2024-03-09T08:33:17.484 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-09T08:33:17.485 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} --verbose --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387 -- /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387/orig.config.yaml 2024-03-09T08:33:17.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-09T08:33:17.639 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387 branch: wip-lusov-qdb description: fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} email: null first_in_suite: false job_id: '7588387' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb ceph: cephfs: fs: - name: a - name: b session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - overall HEALTH_ - \(MON_DOWN\) sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: cephfs_name: a client.1: cephfs_name: b install: ceph: flavor: default sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 kclient: client.0: cephfs_name: a client.1: cephfs_name: b syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 9221 sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 sleep_before_teardown: 0 subset: 111/10000 suite: fs suite_branch: wip-lusov-qdb suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAb/kX4Hsc0tiJSY05cLPFVNhJMK+e4k9JbctP5BGPUVT79kEHsaEBetdjSRmsLxj40HKNERLtYzcfUyQqlqn10= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCLbjgt5vP+vOklGDWn6iDhKnEH0HXyfVYCMuZrpZANGqm64uoLhdXIJPj/Q1yBtpcI7cHfNELey3q2WmQCGrMA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - fs/snaps timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-09_08:15:07 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-09T08:33:17.639 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa; will attempt to use it 2024-03-09T08:33:17.640 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks 2024-03-09T08:33:17.640 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-09T08:33:17.642 INFO:teuthology.task.internal:Checking packages... 2024-03-09T08:33:17.665 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8b896a9e145796119f0451201dc4d53ddac97db0' 2024-03-09T08:33:17.665 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-09T08:33:17.665 INFO:teuthology.packaging:ref: None 2024-03-09T08:33:17.665 INFO:teuthology.packaging:tag: None 2024-03-09T08:33:17.665 INFO:teuthology.packaging:branch: wip-lusov-qdb 2024-03-09T08:33:17.665 INFO:teuthology.packaging:sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-09T08:33:17.666 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb 2024-03-09T08:33:17.808 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1848.g8b896a9e 2024-03-09T08:33:17.810 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-09T08:33:17.816 INFO:teuthology.task.internal:no buildpackages task found 2024-03-09T08:33:17.816 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-09T08:33:17.822 INFO:teuthology.task.internal:Saving configuration 2024-03-09T08:33:17.841 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-09T08:33:17.846 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-09T08:33:17.870 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-09 08:27:53.275226', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAb/kX4Hsc0tiJSY05cLPFVNhJMK+e4k9JbctP5BGPUVT79kEHsaEBetdjSRmsLxj40HKNERLtYzcfUyQqlqn10='} 2024-03-09T08:33:17.894 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-09 08:27:53.273353', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCLbjgt5vP+vOklGDWn6iDhKnEH0HXyfVYCMuZrpZANGqm64uoLhdXIJPj/Q1yBtpcI7cHfNELey3q2WmQCGrMA='} 2024-03-09T08:33:17.894 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-09T08:33:17.900 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-09T08:33:17.900 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-03-09T08:33:17.900 INFO:teuthology.run_tasks:Running task console_log... 2024-03-09T08:33:17.970 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa33926dee0>, signals=[15]) 2024-03-09T08:33:17.971 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-09T08:33:17.977 INFO:teuthology.task.internal:Opening connections... 2024-03-09T08:33:17.977 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T08:33:17.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T08:33:18.082 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T08:33:18.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T08:33:18.172 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-09T08:33:18.178 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2024-03-09T08:33:18.195 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2024-03-09T08:33:18.196 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2024-03-09T08:33:18.253 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2024-03-09T08:33:18.253 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="9" 2024-03-09T08:33:18.253 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2024-03-09T08:33:18.253 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2024-03-09T08:33:18.253 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="9" 2024-03-09T08:33:18.253 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el9" 2024-03-09T08:33:18.253 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-09T08:33:18.253 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2024-03-09T08:33:18.254 INFO:teuthology.orchestra.run.smithi067.stdout:LOGO="fedora-logo-icon" 2024-03-09T08:33:18.254 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-09T08:33:18.254 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2024-03-09T08:33:18.254 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-09T08:33:18.254 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-09T08:33:18.254 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-09T08:33:18.254 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2024-03-09T08:33:18.274 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-03-09T08:33:18.291 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-03-09T08:33:18.292 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-03-09T08:33:18.348 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-03-09T08:33:18.348 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-03-09T08:33:18.348 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-03-09T08:33:18.348 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-03-09T08:33:18.348 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-03-09T08:33:18.348 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-03-09T08:33:18.348 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-09T08:33:18.348 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-03-09T08:33:18.348 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-03-09T08:33:18.348 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-09T08:33:18.348 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-03-09T08:33:18.349 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-09T08:33:18.349 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-09T08:33:18.349 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-09T08:33:18.349 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-03-09T08:33:18.374 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-09T08:33:18.383 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-09T08:33:18.389 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-09T08:33:18.389 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2024-03-09T08:33:18.391 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-03-09T08:33:18.406 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-09T08:33:18.411 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-09T08:33:18.411 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2024-03-09T08:33:18.448 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-03-09T08:33:18.587 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-09T08:33:18.642 INFO:teuthology.run_tasks:Running task kernel... 2024-03-09T08:33:18.657 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-09T08:33:18.658 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}} 2024-03-09T08:33:18.658 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-03-09T08:33:18.658 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-09T08:33:18.658 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-09T08:33:18.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T08:33:18.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-09T08:33:18.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T08:33:18.679 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-09T08:33:18.892 DEBUG:teuthology.task.kernel:sha1 for client.0 is dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-09T08:33:18.892 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412"... 2024-03-09T08:33:18.892 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-03-09T08:33:18.910 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-361.el9.x86_64 2024-03-09T08:33:18.910 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-09T08:33:18.911 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-09T08:33:18.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum upgrade -y linux-firmware 2024-03-09T08:33:19.620 DEBUG:teuthology.task.kernel:sha1 for client.1 is dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-09T08:33:19.620 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412"... 2024-03-09T08:33:19.620 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-03-09T08:33:19.638 INFO:teuthology.orchestra.run.smithi134.stdout:5.14.0-361.el9.x86_64 2024-03-09T08:33:19.638 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-09T08:33:19.638 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-09T08:33:19.638 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum upgrade -y linux-firmware 2024-03-09T08:33:21.761 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - AppStream 5.4 MB/s | 6.7 MB 00:01 2024-03-09T08:33:22.208 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-09T08:33:23.411 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2024-03-09T08:33:24.192 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS-9 - BaseOS 2.2 MB/s | 2.0 MB 00:00 2024-03-09T08:33:39.505 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 549 kB/s | 8.0 MB 00:14 2024-03-09T08:33:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 373 kB/s | 8.0 MB 00:22 2024-03-09T08:34:13.668 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 588 kB/s | 19 MB 00:32 2024-03-09T08:34:16.572 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 659 kB/s | 19 MB 00:29 2024-03-09T08:34:19.018 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.1 MB 00:01 2024-03-09T08:34:20.628 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2024-03-09T08:34:20.823 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-09T08:34:22.246 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-09T08:34:22.323 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 15 kB 00:00 2024-03-09T08:34:23.400 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-03-09T08:34:27.262 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-03-09T08:34:28.508 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-03-09T08:34:29.596 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-03-09T08:34:29.605 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-03-09T08:34:29.605 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-03-09T08:34:29.605 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-03-09T08:34:29.605 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-03-09T08:34:29.605 INFO:teuthology.orchestra.run.smithi134.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-09T08:34:29.605 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:34:29.605 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-03-09T08:34:29.605 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-03-09T08:34:29.605 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 1 Package 2024-03-09T08:34:29.605 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:34:29.606 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 375 M 2024-03-09T08:34:29.606 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-03-09T08:34:30.886 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-03-09T08:34:30.895 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-03-09T08:34:30.895 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-03-09T08:34:30.895 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-03-09T08:34:30.895 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2024-03-09T08:34:30.895 INFO:teuthology.orchestra.run.smithi067.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-09T08:34:30.895 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:34:30.895 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-03-09T08:34:30.895 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-03-09T08:34:30.895 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 1 Package 2024-03-09T08:34:30.896 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:34:30.896 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 375 M 2024-03-09T08:34:30.896 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-03-09T08:35:04.673 INFO:teuthology.orchestra.run.smithi067.stdout:linux-firmware-20240219-143.el9.noarch.rpm 11 MB/s | 375 MB 00:33 2024-03-09T08:35:04.674 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-03-09T08:35:04.674 INFO:teuthology.orchestra.run.smithi067.stdout:Total 11 MB/s | 375 MB 00:33 2024-03-09T08:35:04.674 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-03-09T08:35:04.783 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-03-09T08:35:04.783 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-03-09T08:35:05.667 INFO:teuthology.orchestra.run.smithi134.stdout:linux-firmware-20240219-143.el9.noarch.rpm 10 MB/s | 375 MB 00:36 2024-03-09T08:35:05.668 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-03-09T08:35:05.668 INFO:teuthology.orchestra.run.smithi134.stdout:Total 10 MB/s | 375 MB 00:36 2024-03-09T08:35:05.668 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-03-09T08:35:05.765 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-03-09T08:35:05.765 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-03-09T08:35:07.258 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-03-09T08:35:07.258 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-03-09T08:35:08.216 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-03-09T08:35:08.216 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-03-09T08:35:13.565 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-03-09T08:35:13.776 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-09T08:35:13.866 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-09T08:35:14.478 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-03-09T08:35:14.710 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-09T08:35:14.773 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-09T08:35:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-09T08:35:14.849 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-09T08:35:15.174 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-09T08:35:15.175 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:35:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2024-03-09T08:35:15.175 INFO:teuthology.orchestra.run.smithi067.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-09T08:35:15.175 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:35:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-03-09T08:35:15.455 INFO:teuthology.task.kernel:Downloading kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.0... 2024-03-09T08:35:15.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-09T08:35:15.605 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ 2024-03-09T08:35:15.606 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-09T08:35:15.610 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi067.front.sepia.ceph.com to complete... 2024-03-09T08:35:15.742 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-09T08:35:15.742 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-09T08:35:16.138 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-09T08:35:16.138 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:35:16.138 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-03-09T08:35:16.139 INFO:teuthology.orchestra.run.smithi134.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-09T08:35:16.139 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:35:16.139 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-03-09T08:35:16.449 INFO:teuthology.task.kernel:Downloading kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.1... 2024-03-09T08:35:16.449 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-09T08:35:16.597 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ 2024-03-09T08:35:16.597 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-09T08:35:16.599 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi134.front.sepia.ceph.com to complete... 2024-03-09T08:38:58.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09 08:38:58 URL:https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm [336216379/336216379] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-09T08:38:58.796 INFO:teuthology.orchestra.run.smithi067.stderr:FINISHED --2024-03-09 08:38:58-- 2024-03-09T08:38:58.797 INFO:teuthology.orchestra.run.smithi067.stderr:Total wall clock time: 3m 43s 2024-03-09T08:38:58.797 INFO:teuthology.orchestra.run.smithi067.stderr:Downloaded: 1 files, 321M in 3m 42s (1.44 MB/s) 2024-03-09T08:38:58.801 INFO:teuthology.task.kernel:Installing kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.0... 2024-03-09T08:38:58.801 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-09T08:39:00.837 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying... ######################################## 2024-03-09T08:39:00.902 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing... ######################################## 2024-03-09T08:39:00.986 INFO:teuthology.orchestra.run.smithi067.stdout:Updating / installing... 2024-03-09T08:39:03.124 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09 08:39:03 URL:https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm [336216379/336216379] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-09T08:39:03.124 INFO:teuthology.orchestra.run.smithi134.stderr:FINISHED --2024-03-09 08:39:03-- 2024-03-09T08:39:03.124 INFO:teuthology.orchestra.run.smithi134.stderr:Total wall clock time: 3m 47s 2024-03-09T08:39:03.124 INFO:teuthology.orchestra.run.smithi134.stderr:Downloaded: 1 files, 321M in 3m 46s (1.42 MB/s) 2024-03-09T08:39:03.127 INFO:teuthology.task.kernel:Installing kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.1... 2024-03-09T08:39:03.128 DEBUG:teuthology.orchestra.run.smithi134:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-09T08:39:05.067 INFO:teuthology.orchestra.run.smithi134.stdout:Verifying... ######################################## 2024-03-09T08:39:05.214 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing... ######################################## 2024-03-09T08:39:05.294 INFO:teuthology.orchestra.run.smithi134.stdout:Updating / installing... 2024-03-09T08:39:08.823 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-6.8.0_rc7_gdd5c1cd17167-1 ######################################## 2024-03-09T08:39:12.773 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-6.8.0_rc7_gdd5c1cd17167-1 ######################################## 2024-03-09T08:39:22.722 INFO:teuthology.orchestra.run.smithi067.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-09T08:39:26.759 INFO:teuthology.orchestra.run.smithi134.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-09T08:39:45.968 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-09T08:39:45.969 DEBUG:teuthology.orchestra.run.smithi067:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-09T08:39:46.030 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/System.map 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/config 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-09T08:39:46.031 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-09T08:39:46.032 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm.ko 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ansi_cprng.ko 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/anubis.ko 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/arc4.ko 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_memcpy.ko 2024-03-09T08:39:46.033 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_pq.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_tx.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_xor.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/raid6test.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authenc.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authencesn.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blake2b_generic.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_common.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_generic.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/camellia_generic.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast5_generic.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast6_generic.ko 2024-03-09T08:39:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast_common.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ccm.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cmac.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crc32_generic.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_engine.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_user.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cts.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/des_generic.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/dh_generic.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/echainiv.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/essiv.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/fcrypt.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/khazad.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4hc.ko 2024-03-09T08:39:46.035 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/md4.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/michael_mic.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcbc.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcrypt.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/rmd160.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/seed.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/serpent_generic.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tcrypt.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tea.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_common.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_generic.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/vmac.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/wp512.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xcbc.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xor.ko 2024-03-09T08:39:46.036 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xxhash_generic.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_pad.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/custom_method.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/ec_sys.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/platform_profile.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbs.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbshc.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/video.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/acard-ahci.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ahci_platform.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ata_generic.ko 2024-03-09T08:39:46.037 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/libahci_platform.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_acpi.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ali.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_amd.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_artop.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atiixp.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atp867x.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd640.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd64x.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cypress.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_efar.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt366.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt37x.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-09T08:39:46.038 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it8213.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it821x.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_jmicron.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_marvell.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_mpiix.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_netcell.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ninja32.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87410.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87415.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_oldpiix.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_opti.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_optidma.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pcmcia.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-09T08:39:46.039 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_piccolo.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_rdc.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sch.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_serverworks.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sil680.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sis.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sl82c105.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_triflex.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_via.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pdc_adma.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_inic162x.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_mv.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_nv.ko 2024-03-09T08:39:46.040 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_promise.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_qstor.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil24.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sis.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_svw.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sx4.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_uli.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_via.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_vsc.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay/ks0108.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-09T08:39:46.041 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma/bcma.ko 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe/aoe.ko 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/brd.ko 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd/drbd.ko 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/floppy.ko 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/loop.ko 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/nbd.ko 2024-03-09T08:39:46.042 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk/null_blk.ko 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/pktcdvd.ko 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/rbd.ko 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/virtio_blk.ko 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkfront.ko 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram/zram.ko 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hangcheck-timer.ko 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-09T08:39:46.043 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/via-rng.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/lp.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave/mwave.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ppdev.ko 2024-03-09T08:39:46.044 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tlclk.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/uv_mmtimer.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/virtio_console.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-09T08:39:46.045 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp.ko 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-aes.ko 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-sha.ko 2024-03-09T08:39:46.046 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca/dca.ko 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/amd64_edac.ko 2024-03-09T08:39:46.047 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/e752x_edac.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/edac_mce_amd.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3000_edac.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3200_edac.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5100_edac.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5400_edac.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7300_edac.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7core_edac.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i82975x_edac.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/ie31200_edac.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/sb_edac.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/x38_edac.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/edd.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-09T08:39:46.048 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-ich.ko 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-appleir.ko 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-aureal.ko 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-axff.ko 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-dr.ko 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elecom.ko 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elo.ko 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-emsff.ko 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gaff.ko 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gt683r.ko 2024-03-09T08:39:46.049 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gyration.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtekff.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-hyperv.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-icade.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-keytouch.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-kye.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lcpower.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-led.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lenovo.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-multitouch.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-ortek.ko 2024-03-09T08:39:46.050 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-penmount.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-petalynx.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-picolcd.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-pl.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-plantronics.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-primax.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-rmi.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-common.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-09T08:39:46.051 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-saitek.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-samsung.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sjoy.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sony.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-speedlink.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-steelseries.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sunplus.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-09T08:39:46.052 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tivo.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tmff.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-topseed.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-twinhan.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-uclogic.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-waltop.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-wiimote.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-xinmo.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zpff.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zydacron.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/uhid.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/wacom.ko 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv 2024-03-09T08:39:46.053 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_balloon.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_utils.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_vmbus.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru3.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7414.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7418.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adc128d818.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1025.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1026.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1029.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1031.ko 2024-03-09T08:39:46.054 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm9240.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ads7828.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7410.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7411.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7462.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7470.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7475.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7x10.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/amc6821.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/applesmc.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asb100.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asc7621.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/atxp1.ko 2024-03-09T08:39:46.055 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/coretemp.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dme1737.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds1621.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds620.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc1403.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc6w201.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71805f.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71882fg.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f75375s.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fam15h_power.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fschmd.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g760a.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g762.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl518sm.ko 2024-03-09T08:39:46.056 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl520sm.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5500_temp.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5k_amb.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmaem.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmpex.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina209.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina2xx.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/it87.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k10temp.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k8temp.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lineage-pem.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm63.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm73.ko 2024-03-09T08:39:46.057 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm75.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm77.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm78.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm80.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm83.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm85.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm87.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm90.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm92.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm93.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95234.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95241.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95245.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc2945.ko 2024-03-09T08:39:46.058 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4151.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4215.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4222.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4245.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4260.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4261.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max16065.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1619.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1668.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max197.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6639.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6650.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6697.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/mcp3021.ko 2024-03-09T08:39:46.059 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6683.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775-core.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct7802.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87360.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87427.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pcf8591.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-09T08:39:46.060 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/powr1220.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5627.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5636.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht15.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht21.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/shtc1.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sis5595.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47b397.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m1.ko 2024-03-09T08:39:46.061 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m192.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/thmc50.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp102.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp103.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp401.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp421.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via-cputemp.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via686a.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt1211.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt8231.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627ehf.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627hf.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83781d.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83791d.ko 2024-03-09T08:39:46.062 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83792d.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83793.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83795.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l785ts.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l786ng.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-09T08:39:46.063 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-09T08:39:46.064 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-dev.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-mux.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-smbus.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-stub.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_core.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-09T08:39:46.065 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp 2024-03-09T08:39:46.066 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/gameport.ko 2024-03-09T08:39:46.067 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/joydev.ko 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/appletouch.ko 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/bcm5974.ko 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/cyapatp.ko 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/sermouse.ko 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-09T08:39:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/altera_ps2.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/arc_ps2.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/serio_raw.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/sparse-keymap.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-blinkm.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lm3530.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lp3944.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-ss4200.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-09T08:39:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache/bcache.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-bio-prison.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache-smq.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-crypt.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-delay.ko 2024-03-09T08:39:46.070 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-flakey.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-log-userspace.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-multipath.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-queue-length.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-raid.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-round-robin.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-service-time.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-switch.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-thin-pool.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-verity.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid0.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid1.ko 2024-03-09T08:39:46.071 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid10.ko 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid456.ko 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptbase.ko 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptctl.ko 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptfc.ko 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptlan.ko 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptsas.ko 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptscsih.ko 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptspi.ko 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_ich.ko 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_sch.ko 2024-03-09T08:39:46.072 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/mfd-core.ko 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/sm501.ko 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/viperboard.ko 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/vx855.ko 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/wl1273-core.ko 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds9802als.ko 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds990x.ko 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/bh1770glc.ko 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710/cb710.ko 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/at24.ko 2024-03-09T08:39:46.073 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/max6875.ko 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/enclosure.ko 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/hpilo.ko 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29003.ko 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29020.ko 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-me.ko 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-txe.ko 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei.ko 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru 2024-03-09T08:39:46.074 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_7xx1.ko 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_core.ko 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tsl2550.ko 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_balloon.ko 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding/bonding.ko 2024-03-09T08:39:46.075 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dummy.ko 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/eql.ko 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-09T08:39:46.076 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-09T08:39:46.077 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-09T08:39:46.078 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-09T08:39:46.079 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-09T08:39:46.080 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dnet.ko 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ethoc.ko 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/fealnx.ko 2024-03-09T08:39:46.081 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-09T08:39:46.082 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/jme.ko 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-09T08:39:46.083 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-09T08:39:46.084 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-09T08:39:46.085 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-09T08:39:46.086 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-09T08:39:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/geneve.ko 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ifb.ko 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan 2024-03-09T08:39:46.088 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvlan.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvtap.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mii.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/net_failover.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/netconsole.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/nlmon.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-09T08:39:46.089 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/amd.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm87xx.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/broadcom.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/cicada.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/davicom.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/dp83640.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/et1011c.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/icplus.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/lxt.ko 2024-03-09T08:39:46.090 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/marvell.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/micrel.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/national.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/phylink.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/qsemi.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/realtek.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/smsc.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/ste10Xp.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/vitesse.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_async.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-09T08:39:46.091 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppoe.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppox.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pptp.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slhc.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slip.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/sungem_phy.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tap.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_random.ko 2024-03-09T08:39:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tun.ko 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/veth.ko 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/virtio_net.ko 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan/vxlan.ko 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netfront.ko 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb/ntb.ko 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host 2024-03-09T08:39:46.093 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme-core.ko 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme.ko 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport.ko 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_cs.ko 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_pc.ko 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_serial.ko 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/xen-pcifront.ko 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia 2024-03-09T08:39:46.094 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/i82092.ko 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/pd6729.ko 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acerhdf.ko 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-09T08:39:46.095 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/hdaps.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel_ips.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-09T08:39:46.096 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi.ko 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps_parport.ko 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/pps_core.ko 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp.ko 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp_kvm.ko 2024-03-09T08:39:46.097 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-em3027.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-09T08:39:46.098 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-max6900.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-09T08:39:46.099 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-x1205.ko 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-9xxx.ko 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-sas.ko 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-xxxx.ko 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/BusLogic.ko 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/a100u2w.ko 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/advansys.ko 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-09T08:39:46.100 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/am53c974.ko 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/atp870u.ko 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa/bfa.ko 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ch.ko 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor 2024-03-09T08:39:46.101 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dc395x.ko 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dmx3191d.ko 2024-03-09T08:39:46.102 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esp_scsi.ko 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic/fnic.ko 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hpsa.ko 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hptiop.ko 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hv_storvsc.ko 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/initio.ko 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ipr.ko 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ips.ko 2024-03-09T08:39:46.103 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci/isci.ko 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc/libfc.ko 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi.ko 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas/libsas.ko 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid.ko 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-09T08:39:46.104 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvumi.ko 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pmcraid.ko 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla1280.ko 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx 2024-03-09T08:39:46.105 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/raid_class.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_debug.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ses.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/st.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/stex.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/virtio_scsi.ko 2024-03-09T08:39:46.106 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/wd719x.ko 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb/ssb.ko 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_file.ko 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_iblock.ko 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_mod.ko 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_pscsi.ko 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_user.ko 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc 2024-03-09T08:39:46.107 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-09T08:39:46.108 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_gsm.ko 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_hdlc.ko 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/nozomi.ko 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial 2024-03-09T08:39:46.109 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/arc_uart.ko 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio.ko 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_aec.ko 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_cif.ko 2024-03-09T08:39:46.110 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_pci_generic.ko 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_sercos3.ko 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-acm.ko 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usblp.ko 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usbtmc.ko 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/mdc800.ko 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/microtek.ko 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/adutux.ko 2024-03-09T08:39:46.111 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/appledisplay.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi26.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi62.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ezusb.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/idmouse.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/iowarrior.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ldusb.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/legousbtower.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usb3503.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usblcd.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-09T08:39:46.112 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/uss720.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/yurex.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy/phy-generic.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/aircable.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ark3116.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ch341.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cp210x.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cyberjack.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/empeg.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-09T08:39:46.113 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_ti.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipaq.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipw.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ir-usb.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mct_u232.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7720.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7840.ko 2024-03-09T08:39:46.114 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/navman.ko 2024-03-09T08:39:46.117 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/omninet.ko 2024-03-09T08:39:46.117 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/opticon.ko 2024-03-09T08:39:46.117 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/option.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/oti6858.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/pl2303.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcaux.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcserial.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/quatech2.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/safe_serial.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/sierra.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ssu100.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/symbolserial.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_debug.ko 2024-03-09T08:39:46.118 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/visor.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/whiteheat.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/uas.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-karma.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-09T08:39:46.119 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/usb-storage.ko 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio.ko 2024-03-09T08:39:46.120 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost.ko 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_net.ko 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_scsi.ko 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/apple_bl.ko 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/lcd.ko 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio.ko 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_balloon.ko 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_mmio.ko 2024-03-09T08:39:46.121 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci.ko 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_ring.ko 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2482.ko 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2490.ko 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-09T08:39:46.122 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/wire.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/hpwdt.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/i6300esb.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-09T08:39:46.123 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ib700wdt.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ibmasr.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it87_wdt.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/machzwd.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/nv_tco.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/softdog.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/via_wdt.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-09T08:39:46.124 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/wdt_pci.ko 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/xen_wdt.ko 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-evtchn.ko 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntalloc.ko 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntdev.ko 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-privcmd.ko 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs 2024-03-09T08:39:46.125 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p/9p.ko 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs/affs.ko 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs/befs.ko 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/binfmt_misc.ko 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs/btrfs.ko 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles/cachefiles.ko 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph/ceph.ko 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda 2024-03-09T08:39:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda/coda.ko 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs/cramfs.ko 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm/dlm.ko 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs/f2fs.ko 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/fat.ko 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/msdos.ko 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/vfat.ko 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/cuse.ko 2024-03-09T08:39:46.127 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/fuse.ko 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2/gfs2.ko 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs/hfs.ko 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus/hfsplus.ko 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs/isofs.ko 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs/jfs.ko 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd/lockd.ko 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix 2024-03-09T08:39:46.128 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix/minix.ko 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs/netfs.ko 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfs.ko 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv3.ko 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv4.ko 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/grace.ko 2024-03-09T08:39:46.129 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/nfs_acl.ko 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd/nfsd.ko 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2/nilfs2.ko 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-celtic.ko 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-centeuro.ko 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-croatian.ko 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-cyrillic.ko 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-gaelic.ko 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-greek.ko 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-iceland.ko 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-inuit.ko 2024-03-09T08:39:46.130 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-roman.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-romanian.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-turkish.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1250.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1251.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1255.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp737.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp775.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp850.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp852.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp855.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp857.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp860.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp861.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp862.ko 2024-03-09T08:39:46.131 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp863.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp864.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp865.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp866.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp869.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp874.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp932.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp936.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp949.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp950.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_euc-jp.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-1.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-13.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-14.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-15.ko 2024-03-09T08:39:46.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-2.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-3.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-4.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-5.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-6.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-7.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-9.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-r.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-ru.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-u.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_utf8.ko 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster 2024-03-09T08:39:46.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2.ko 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs/overlay.ko 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore/ramoops.ko 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs/reiserfs.ko 2024-03-09T08:39:46.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs/romfs.ko 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client/cifs.ko 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_arc4.ko 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_md4.ko 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs/squashfs.ko 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv/sysv.ko 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf/udf.ko 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs 2024-03-09T08:39:46.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs/ufs.ko 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs/xfs.ko 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu/rcutorture.ko 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/torture.ko 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/asn1_encoder.ko 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc-itu-t.ko 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc8.ko 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libarc4.ko 2024-03-09T08:39:46.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libdes.ko 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/libcrc32c.ko 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lru_cache.ko 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4_compress.ko 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4hc_compress.ko 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math/cordic.ko 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6/raid6_pq.ko 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_bm.ko 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_fsm.ko 2024-03-09T08:39:46.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_kmp.ko 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd/zstd_compress.ko 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm/hwpoison-inject.ko 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/6lowpan.ko 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_dest.ko 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_fragment.ko 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_hop.ko 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_mobility.ko 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_routing.ko 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_udp.ko 2024-03-09T08:39:46.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/garp.ko 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/mrp.ko 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/p8022.ko 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/psnap.ko 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/stp.ko 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q/8021q.ko 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet.ko 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_fd.ko 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_rdma.ko 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_virtio.ko 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv/batman-adv.ko 2024-03-09T08:39:46.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/br_netfilter.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/bridge.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-09T08:39:46.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtables.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph 2024-03-09T08:39:46.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph/libceph.ko 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/failover.ko 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/pktgen.ko 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp.ko 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_diag.ko 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv4.ko 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv6.ko 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver/dns_resolver.ko 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/dsa_core.ko 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_brcm.ko 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_dsa.ko 2024-03-09T08:39:46.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_none.ko 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_trailer.ko 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154.ko 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ah4.ko 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/esp4.ko 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/fou.ko 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/gre.ko 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/inet_diag.ko 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_gre.ko 2024-03-09T08:39:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_tunnel.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_vti.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipcomp.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipip.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-09T08:39:46.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_bic.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_dctcp.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_diag.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_highspeed.ko 2024-03-09T08:39:46.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_htcp.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_hybla.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_illinois.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_lp.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_scalable.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_vegas.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_veno.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_westwood.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_yeah.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tunnel4.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_diag.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_tunnel.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6 2024-03-09T08:39:46.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ah6.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/esp6.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/fou6.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_tunnel.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_vti.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ipcomp6.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-09T08:39:46.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-09T08:39:46.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/sit.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/tunnel6.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key/af_key.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_core.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_eth.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip6.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_netlink.ko 2024-03-09T08:39:46.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ppp.ko 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc/llc.ko 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154/mac802154.ko 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls/mpls_gso.ko 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set.ko 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-09T08:39:46.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-09T08:39:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conncount.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-09T08:39:46.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_log_syslog.ko 2024-03-09T08:39:46.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_irc.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_sip.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_tables.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_log.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-09T08:39:46.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_chain_nat.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_compat.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_ct.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fib.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_hash.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_limit.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_log.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_masq.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_nat.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_numgen.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_queue.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_quota.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_redir.ko 2024-03-09T08:39:46.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject_inet.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_socket.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_tunnel.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_AUDIT.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CT.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_DSCP.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HL.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HMARK.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LED.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LOG.ko 2024-03-09T08:39:46.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NETMAP.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFLOG.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_RATEEST.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_SECMARK.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TEE.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TPROXY.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TRACE.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_addrtype.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_bpf.ko 2024-03-09T08:39:46.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cgroup.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cluster.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_comment.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connbytes.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlabel.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlimit.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connmark.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_conntrack.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cpu.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dccp.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_devgroup.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dscp.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ecn.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_esp.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hashlimit.ko 2024-03-09T08:39:46.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_helper.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hl.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipcomp.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_iprange.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipvs.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_l2tp.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_length.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_limit.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mac.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mark.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_multiport.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nat.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nfacct.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_osf.ko 2024-03-09T08:39:46.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_owner.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_physdev.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_pkttype.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_policy.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_quota.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_rateest.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_realm.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_recent.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_sctp.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_set.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_socket.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_state.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_statistic.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_string.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_tcpmss.ko 2024-03-09T08:39:46.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_time.ko 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_u32.ko 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink/netlink_diag.ko 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh/nsh.ko 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/openvswitch.ko 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-geneve.ko 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-gre.ko 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-vxlan.ko 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet/af_packet_diag.ko 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds.ko 2024-03-09T08:39:46.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_rdma.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_tcp.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_csum.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_gact.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_mirred.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_nat.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_pedit.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_police.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_simple.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_skbedit.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_vlan.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_basic.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_bpf.ko 2024-03-09T08:39:46.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_flow.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_fw.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_route.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_u32.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_cmp.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_ipset.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_meta.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_nbyte.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_text.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_u32.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_choke.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_codel.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_drr.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_fq.ko 2024-03-09T08:39:46.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_gred.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hfsc.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hhf.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_htb.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_ingress.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio_lib.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_multiq.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_netem.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_pie.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_plug.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_prio.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_qfq.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_red.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfb.ko 2024-03-09T08:39:46.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfq.ko 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_tbf.ko 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_teql.ko 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp.ko 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp_diag.ko 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/sunrpc.ko 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/diag.ko 2024-03-09T08:39:46.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/tipc.ko 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix/unix_diag.ko 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock.ko 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys 2024-03-09T08:39:46.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys/trusted.ko 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib/irqbypass.ko 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias.bin 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.alias.bin 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.bin 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.modinfo 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep.bin 2024-03-09T08:39:46.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.devname 2024-03-09T08:39:46.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.order 2024-03-09T08:39:46.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.softdep 2024-03-09T08:39:46.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols 2024-03-09T08:39:46.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols.bin 2024-03-09T08:39:46.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-09T08:39:46.172 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:46.172 DEBUG:teuthology.orchestra.run.smithi067:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-09T08:39:46.194 INFO:teuthology.orchestra.run.smithi067.stdout:postinstall scriptlet (using /bin/sh): 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gdd5c1cd17167 /lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout:for file in vmlinuz System.map config; do 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167"; then 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout: cp "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167" 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout: fi 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout:done 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout:new-kernel-pkg --remove 6.8.0-rc7-gdd5c1cd17167 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gdd5c1cd17167.img 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-09T08:39:46.195 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-install remove 6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:46.196 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-03-09T08:39:46.196 INFO:teuthology.orchestra.run.smithi067.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-09T08:39:46.196 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-09T08:39:46.196 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:46.196 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-03-09T08:39:46.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2024-03-09T08:39:46.276 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2024-03-09T08:39:46.276 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2024-03-09T08:39:46.276 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.06 2024-03-09T08:39:46.277 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 68.el9 2024-03-09T08:39:46.277 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2024-03-09T08:39:46.277 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-09T08:39:46.277 INFO:teuthology.orchestra.run.smithi067.stdout:Group : Unspecified 2024-03-09T08:39:46.277 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 8264254 2024-03-09T08:39:46.277 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2024-03-09T08:39:46.278 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-09T08:39:46.278 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-09T08:39:46.278 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-09T08:39:46.278 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-09T08:39:46.278 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : builder@centos.org 2024-03-09T08:39:46.278 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : CentOS 2024-03-09T08:39:46.278 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-09T08:39:46.279 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2024-03-09T08:39:46.279 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2024-03-09T08:39:46.279 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:39:46.279 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-09T08:39:46.279 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-09T08:39:46.279 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-09T08:39:46.280 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2024-03-09T08:39:46.280 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:39:46.280 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2024-03-09T08:39:46.281 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-09T08:39:46.282 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:46.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-09T08:39:46.889 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2024-03-09T08:39:48.274 INFO:teuthology.orchestra.run.smithi067.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-09T08:39:48.310 INFO:teuthology.orchestra.run.smithi067.stderr:done 2024-03-09T08:39:48.313 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-03-09T08:39:48.328 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.UmmWuS18EF 2024-03-09T08:39:48.329 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UmmWuS18EF 2024-03-09T08:39:48.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.UmmWuS18EF 2024-03-09T08:39:48.565 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.UmmWuS18EF is 6KB 2024-03-09T08:39:48.590 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.UmmWuS18EF 2024-03-09T08:39:48.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2024-03-09T08:39:48.688 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-09T08:39:48.689 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-09T08:39:48.689 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-09T08:39:48.689 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167.conf 2024-03-09T08:39:48.690 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:48.852 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2024-03-09T08:39:49.097 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi134.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-09T08:39:49.097 DEBUG:teuthology.orchestra.run.smithi134:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-09T08:39:49.168 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:49.168 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/System.map 2024-03-09T08:39:49.168 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/config 2024-03-09T08:39:49.168 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel 2024-03-09T08:39:49.168 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch 2024-03-09T08:39:49.168 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-09T08:39:49.169 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-09T08:39:49.170 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm.ko 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ansi_cprng.ko 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/anubis.ko 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/arc4.ko 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_memcpy.ko 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_pq.ko 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_tx.ko 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_xor.ko 2024-03-09T08:39:49.171 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/raid6test.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authenc.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authencesn.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blake2b_generic.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_common.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_generic.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/camellia_generic.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast5_generic.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast6_generic.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast_common.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ccm.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cmac.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crc32_generic.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_engine.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_user.ko 2024-03-09T08:39:49.172 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cts.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/des_generic.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/dh_generic.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/echainiv.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/essiv.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/fcrypt.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/khazad.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4hc.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/md4.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/michael_mic.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcbc.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcrypt.ko 2024-03-09T08:39:49.173 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/rmd160.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/seed.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/serpent_generic.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tcrypt.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tea.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_common.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_generic.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/vmac.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/wp512.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xcbc.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xor.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xxhash_generic.ko 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi 2024-03-09T08:39:49.174 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_pad.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/custom_method.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/ec_sys.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/platform_profile.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbs.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbshc.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/video.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/acard-ahci.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ahci_platform.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ata_generic.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/libahci_platform.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_acpi.ko 2024-03-09T08:39:49.175 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ali.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_amd.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_artop.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atiixp.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atp867x.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd640.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd64x.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cypress.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_efar.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt366.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt37x.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it8213.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it821x.ko 2024-03-09T08:39:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_jmicron.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_marvell.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_mpiix.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_netcell.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ninja32.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87410.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87415.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_oldpiix.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_opti.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_optidma.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pcmcia.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_piccolo.ko 2024-03-09T08:39:49.177 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_rdc.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sch.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_serverworks.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sil680.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sis.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sl82c105.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_triflex.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_via.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pdc_adma.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_inic162x.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_mv.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_nv.ko 2024-03-09T08:39:49.178 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_promise.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_qstor.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil24.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sis.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_svw.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sx4.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_uli.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_via.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_vsc.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay/ks0108.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-09T08:39:49.179 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma/bcma.ko 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe/aoe.ko 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/brd.ko 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd/drbd.ko 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/floppy.ko 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/loop.ko 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-09T08:39:49.180 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/nbd.ko 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk/null_blk.ko 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/pktcdvd.ko 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/rbd.ko 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/virtio_blk.ko 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkfront.ko 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram/zram.ko 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hangcheck-timer.ko 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-09T08:39:49.181 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/via-rng.ko 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/lp.ko 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave 2024-03-09T08:39:49.182 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave/mwave.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ppdev.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tlclk.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/uv_mmtimer.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/virtio_console.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-09T08:39:49.183 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp.ko 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-aes.ko 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-sha.ko 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-09T08:39:49.184 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca/dca.ko 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/amd64_edac.ko 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/e752x_edac.ko 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/edac_mce_amd.ko 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3000_edac.ko 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3200_edac.ko 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5100_edac.ko 2024-03-09T08:39:49.185 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5400_edac.ko 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7300_edac.ko 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7core_edac.ko 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i82975x_edac.ko 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/ie31200_edac.ko 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/sb_edac.ko 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/x38_edac.ko 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/edd.ko 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-ich.ko 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-09T08:39:49.186 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-appleir.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-aureal.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-axff.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-dr.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elecom.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elo.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-emsff.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gaff.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gt683r.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gyration.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtekff.ko 2024-03-09T08:39:49.187 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-hyperv.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-icade.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-keytouch.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-kye.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lcpower.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-led.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lenovo.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-multitouch.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-ortek.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-penmount.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-petalynx.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-picolcd.ko 2024-03-09T08:39:49.188 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-pl.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-plantronics.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-primax.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-rmi.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-common.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-09T08:39:49.189 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-saitek.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-samsung.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sjoy.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sony.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-speedlink.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-steelseries.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sunplus.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tivo.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tmff.ko 2024-03-09T08:39:49.190 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-topseed.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-twinhan.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-uclogic.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-waltop.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-wiimote.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-xinmo.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zpff.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zydacron.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/uhid.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/wacom.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_balloon.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_utils.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_vmbus.ko 2024-03-09T08:39:49.191 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru3.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7414.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7418.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adc128d818.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1025.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1026.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1029.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1031.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm9240.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ads7828.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7410.ko 2024-03-09T08:39:49.192 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7411.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7462.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7470.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7475.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7x10.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/amc6821.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/applesmc.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asb100.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asc7621.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/atxp1.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/coretemp.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dme1737.ko 2024-03-09T08:39:49.193 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds1621.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds620.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc1403.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc6w201.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71805f.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71882fg.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f75375s.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fam15h_power.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fschmd.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g760a.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g762.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl518sm.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl520sm.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-09T08:39:49.194 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5500_temp.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5k_amb.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmaem.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmpex.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina209.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina2xx.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/it87.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k10temp.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k8temp.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lineage-pem.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm63.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm73.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm75.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm77.ko 2024-03-09T08:39:49.195 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm78.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm80.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm83.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm85.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm87.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm90.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm92.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm93.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95234.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95241.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95245.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc2945.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4151.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4215.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4222.ko 2024-03-09T08:39:49.196 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4245.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4260.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4261.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max16065.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1619.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1668.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max197.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6639.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6650.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6697.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/mcp3021.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6683.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775-core.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775.ko 2024-03-09T08:39:49.197 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct7802.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87360.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87427.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pcf8591.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-09T08:39:49.198 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/powr1220.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5627.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5636.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht15.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht21.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/shtc1.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sis5595.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47b397.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m1.ko 2024-03-09T08:39:49.199 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m192.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/thmc50.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp102.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp103.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp401.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp421.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via-cputemp.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via686a.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt1211.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt8231.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627ehf.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627hf.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83781d.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83791d.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83792d.ko 2024-03-09T08:39:49.200 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83793.ko 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83795.ko 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l785ts.ko 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l786ng.ko 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-09T08:39:49.201 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-09T08:39:49.202 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-dev.ko 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-mux.ko 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-smbus.ko 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-stub.ko 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_core.ko 2024-03-09T08:39:49.203 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp 2024-03-09T08:39:49.204 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-09T08:39:49.205 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/gameport.ko 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/joydev.ko 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/appletouch.ko 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/bcm5974.ko 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/cyapatp.ko 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/sermouse.ko 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4 2024-03-09T08:39:49.206 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/altera_ps2.ko 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/arc_ps2.ko 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/serio_raw.ko 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/sparse-keymap.ko 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-blinkm.ko 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lm3530.ko 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lp3944.ko 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-ss4200.ko 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-09T08:39:49.207 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache/bcache.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-bio-prison.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache-smq.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache.ko 2024-03-09T08:39:49.208 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-crypt.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-delay.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-flakey.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-log-userspace.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-multipath.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-queue-length.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-raid.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-round-robin.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-service-time.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-switch.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-thin-pool.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-verity.ko 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data 2024-03-09T08:39:49.209 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid0.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid1.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid10.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid456.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptbase.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptctl.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptfc.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptlan.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptsas.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptscsih.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptspi.ko 2024-03-09T08:39:49.210 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_ich.ko 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_sch.ko 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/mfd-core.ko 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/sm501.ko 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/viperboard.ko 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/vx855.ko 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/wl1273-core.ko 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds9802als.ko 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds990x.ko 2024-03-09T08:39:49.211 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/bh1770glc.ko 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710/cb710.ko 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/at24.ko 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/max6875.ko 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/enclosure.ko 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/hpilo.ko 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29003.ko 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29020.ko 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei 2024-03-09T08:39:49.212 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-me.ko 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-txe.ko 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei.ko 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_7xx1.ko 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_core.ko 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tsl2550.ko 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_balloon.ko 2024-03-09T08:39:49.213 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding/bonding.ko 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dummy.ko 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/eql.ko 2024-03-09T08:39:49.214 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-09T08:39:49.215 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-09T08:39:49.216 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-09T08:39:49.217 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-09T08:39:49.218 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-09T08:39:49.219 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dnet.ko 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ethoc.ko 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/fealnx.ko 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000 2024-03-09T08:39:49.220 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-09T08:39:49.221 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/jme.ko 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-09T08:39:49.222 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-09T08:39:49.223 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-09T08:39:49.224 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro 2024-03-09T08:39:49.225 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-09T08:39:49.226 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/geneve.ko 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ifb.ko 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-09T08:39:49.227 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvlan.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvtap.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mii.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/net_failover.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/netconsole.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/nlmon.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy 2024-03-09T08:39:49.228 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/amd.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm87xx.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/broadcom.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/cicada.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/davicom.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/dp83640.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/et1011c.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/icplus.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/lxt.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/marvell.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/micrel.ko 2024-03-09T08:39:49.229 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/national.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/phylink.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/qsemi.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/realtek.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/smsc.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/ste10Xp.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/vitesse.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_async.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-09T08:39:49.230 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppoe.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppox.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pptp.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slhc.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slip.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/sungem_phy.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tap.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-09T08:39:49.231 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_random.ko 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tun.ko 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/veth.ko 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/virtio_net.ko 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan/vxlan.ko 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netfront.ko 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb/ntb.ko 2024-03-09T08:39:49.232 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme-core.ko 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme.ko 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport.ko 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_cs.ko 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_pc.ko 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_serial.ko 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-09T08:39:49.233 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/xen-pcifront.ko 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/i82092.ko 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/pd6729.ko 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acerhdf.ko 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-09T08:39:49.234 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/hdaps.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel_ips.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-09T08:39:49.235 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi.ko 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-09T08:39:49.236 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps_parport.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/pps_core.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp_kvm.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-09T08:39:49.237 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-em3027.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-max6900.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-09T08:39:49.238 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-x1205.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-9xxx.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-sas.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-xxxx.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/BusLogic.ko 2024-03-09T08:39:49.239 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/a100u2w.ko 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/advansys.ko 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/am53c974.ko 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/atp870u.ko 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa/bfa.ko 2024-03-09T08:39:49.240 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ch.ko 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dc395x.ko 2024-03-09T08:39:49.241 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dmx3191d.ko 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esp_scsi.ko 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic/fnic.ko 2024-03-09T08:39:49.242 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hpsa.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hptiop.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hv_storvsc.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/initio.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ipr.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ips.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci/isci.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc/libfc.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-09T08:39:49.243 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas/libsas.ko 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid.ko 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvumi.ko 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001 2024-03-09T08:39:49.244 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pmcraid.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla1280.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/raid_class.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_debug.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-09T08:39:49.245 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ses.ko 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/st.ko 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/stex.ko 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/virtio_scsi.ko 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/wd719x.ko 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb/ssb.ko 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-09T08:39:49.246 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_file.ko 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_iblock.ko 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_mod.ko 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_pscsi.ko 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_user.ko 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-09T08:39:49.247 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-09T08:39:49.248 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_gsm.ko 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_hdlc.ko 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/nozomi.ko 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/arc_uart.ko 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-09T08:39:49.249 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio.ko 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_aec.ko 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_cif.ko 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_pci_generic.ko 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_sercos3.ko 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-acm.ko 2024-03-09T08:39:49.250 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usblp.ko 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usbtmc.ko 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/mdc800.ko 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/microtek.ko 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/adutux.ko 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/appledisplay.ko 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi26.ko 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi62.ko 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ezusb.ko 2024-03-09T08:39:49.251 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/idmouse.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/iowarrior.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ldusb.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/legousbtower.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usb3503.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usblcd.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/uss720.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/yurex.ko 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy 2024-03-09T08:39:49.252 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy/phy-generic.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/aircable.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ark3116.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ch341.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cp210x.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cyberjack.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/empeg.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_ti.ko 2024-03-09T08:39:49.253 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipaq.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipw.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ir-usb.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mct_u232.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7720.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7840.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/navman.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/omninet.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/opticon.ko 2024-03-09T08:39:49.254 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/option.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/oti6858.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/pl2303.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcaux.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcserial.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/quatech2.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/safe_serial.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/sierra.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ssu100.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/symbolserial.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_debug.ko 2024-03-09T08:39:49.255 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/visor.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/whiteheat.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/uas.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-karma.ko 2024-03-09T08:39:49.256 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/usb-storage.ko 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-09T08:39:49.257 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio.ko 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost.ko 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_net.ko 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_scsi.ko 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/apple_bl.ko 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/lcd.ko 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio 2024-03-09T08:39:49.258 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio.ko 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_balloon.ko 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_mmio.ko 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci.ko 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_ring.ko 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2482.ko 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2490.ko 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves 2024-03-09T08:39:49.259 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/wire.ko 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog 2024-03-09T08:39:49.260 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/hpwdt.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/i6300esb.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ib700wdt.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ibmasr.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it87_wdt.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/machzwd.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/nv_tco.ko 2024-03-09T08:39:49.261 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/softdog.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/via_wdt.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/wdt_pci.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/xen_wdt.ko 2024-03-09T08:39:49.262 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-evtchn.ko 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntalloc.ko 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntdev.ko 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-privcmd.ko 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p/9p.ko 2024-03-09T08:39:49.263 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs/affs.ko 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs/befs.ko 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/binfmt_misc.ko 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs/btrfs.ko 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles/cachefiles.ko 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph/ceph.ko 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda 2024-03-09T08:39:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda/coda.ko 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs/cramfs.ko 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm/dlm.ko 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs/f2fs.ko 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/fat.ko 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/msdos.ko 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/vfat.ko 2024-03-09T08:39:49.265 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/cuse.ko 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/fuse.ko 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2/gfs2.ko 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs/hfs.ko 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus/hfsplus.ko 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs/isofs.ko 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs/jfs.ko 2024-03-09T08:39:49.266 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd/lockd.ko 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix/minix.ko 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs/netfs.ko 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfs.ko 2024-03-09T08:39:49.267 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv3.ko 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv4.ko 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/grace.ko 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/nfs_acl.ko 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd/nfsd.ko 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2/nilfs2.ko 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-celtic.ko 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-centeuro.ko 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-croatian.ko 2024-03-09T08:39:49.268 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-cyrillic.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-gaelic.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-greek.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-iceland.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-inuit.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-roman.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-romanian.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-turkish.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1250.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1251.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1255.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp737.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp775.ko 2024-03-09T08:39:49.269 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp850.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp852.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp855.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp857.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp860.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp861.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp862.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp863.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp864.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp865.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp866.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp869.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp874.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp932.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp936.ko 2024-03-09T08:39:49.270 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp949.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp950.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_euc-jp.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-1.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-13.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-14.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-15.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-2.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-3.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-4.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-5.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-6.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-7.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-9.ko 2024-03-09T08:39:49.271 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-r.ko 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-ru.ko 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-u.ko 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_utf8.ko 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2.ko 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-09T08:39:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs/overlay.ko 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore/ramoops.ko 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs/reiserfs.ko 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs/romfs.ko 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client/cifs.ko 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_arc4.ko 2024-03-09T08:39:49.273 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_md4.ko 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs/squashfs.ko 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv/sysv.ko 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf/udf.ko 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs/ufs.ko 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs/xfs.ko 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu/rcutorture.ko 2024-03-09T08:39:49.274 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/torture.ko 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/asn1_encoder.ko 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc-itu-t.ko 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc8.ko 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libarc4.ko 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libdes.ko 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/libcrc32c.ko 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lru_cache.ko 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4_compress.ko 2024-03-09T08:39:49.275 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4hc_compress.ko 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math/cordic.ko 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6/raid6_pq.ko 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_bm.ko 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_fsm.ko 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_kmp.ko 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd/zstd_compress.ko 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm/hwpoison-inject.ko 2024-03-09T08:39:49.276 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/6lowpan.ko 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_dest.ko 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_fragment.ko 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_hop.ko 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_mobility.ko 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_routing.ko 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_udp.ko 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/garp.ko 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/mrp.ko 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/p8022.ko 2024-03-09T08:39:49.277 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/psnap.ko 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/stp.ko 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q/8021q.ko 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet.ko 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_fd.ko 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_rdma.ko 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_virtio.ko 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv/batman-adv.ko 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/br_netfilter.ko 2024-03-09T08:39:49.278 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/bridge.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-09T08:39:49.279 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtables.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph 2024-03-09T08:39:49.280 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph/libceph.ko 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/failover.ko 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/pktgen.ko 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp.ko 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_diag.ko 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv4.ko 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv6.ko 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver/dns_resolver.ko 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/dsa_core.ko 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_brcm.ko 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_dsa.ko 2024-03-09T08:39:49.281 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_none.ko 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_trailer.ko 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154.ko 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ah4.ko 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/esp4.ko 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/fou.ko 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/gre.ko 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/inet_diag.ko 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_gre.ko 2024-03-09T08:39:49.282 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_tunnel.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_vti.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipcomp.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipip.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-09T08:39:49.283 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_bic.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_dctcp.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_diag.ko 2024-03-09T08:39:49.284 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_highspeed.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_htcp.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_hybla.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_illinois.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_lp.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_scalable.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_vegas.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_veno.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_westwood.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_yeah.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tunnel4.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_diag.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_tunnel.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-09T08:39:49.285 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ah6.ko 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/esp6.ko 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/fou6.ko 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_tunnel.ko 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_vti.ko 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ipcomp6.ko 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-09T08:39:49.286 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-09T08:39:49.287 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/sit.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/tunnel6.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key/af_key.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_core.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_eth.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip.ko 2024-03-09T08:39:49.288 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip6.ko 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_netlink.ko 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ppp.ko 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc/llc.ko 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154/mac802154.ko 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls/mpls_gso.ko 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set.ko 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-09T08:39:49.289 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-09T08:39:49.290 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conncount.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack.ko 2024-03-09T08:39:49.291 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table.ko 2024-03-09T08:39:49.292 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_log_syslog.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_irc.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_sip.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_tables.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_log.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-09T08:39:49.293 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_chain_nat.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_compat.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_ct.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fib.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_hash.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_limit.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_log.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_masq.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_nat.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_numgen.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_queue.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_quota.ko 2024-03-09T08:39:49.294 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_redir.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject_inet.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_socket.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_tunnel.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_AUDIT.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CT.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_DSCP.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HL.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HMARK.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LED.ko 2024-03-09T08:39:49.295 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LOG.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NETMAP.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFLOG.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_RATEEST.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_SECMARK.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TEE.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TPROXY.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TRACE.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_addrtype.ko 2024-03-09T08:39:49.296 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_bpf.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cgroup.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cluster.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_comment.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connbytes.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlabel.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlimit.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connmark.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_conntrack.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cpu.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dccp.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_devgroup.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dscp.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ecn.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_esp.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hashlimit.ko 2024-03-09T08:39:49.297 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_helper.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hl.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipcomp.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_iprange.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipvs.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_l2tp.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_length.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_limit.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mac.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mark.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_multiport.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nat.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nfacct.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_osf.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_owner.ko 2024-03-09T08:39:49.298 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_physdev.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_pkttype.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_policy.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_quota.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_rateest.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_realm.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_recent.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_sctp.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_set.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_socket.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_state.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_statistic.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_string.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_tcpmss.ko 2024-03-09T08:39:49.299 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_time.ko 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_u32.ko 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink/netlink_diag.ko 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh/nsh.ko 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/openvswitch.ko 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-geneve.ko 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-gre.ko 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-vxlan.ko 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet/af_packet_diag.ko 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds 2024-03-09T08:39:49.300 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_rdma.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_tcp.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_csum.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_gact.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_mirred.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_nat.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_pedit.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_police.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_simple.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_skbedit.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_vlan.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_basic.ko 2024-03-09T08:39:49.301 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_bpf.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_flow.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_fw.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_route.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_u32.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_cmp.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_ipset.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_meta.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_nbyte.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_text.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_u32.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_choke.ko 2024-03-09T08:39:49.302 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_codel.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_drr.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_fq.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_gred.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hfsc.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hhf.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_htb.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_ingress.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio_lib.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_multiq.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_netem.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_pie.ko 2024-03-09T08:39:49.303 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_plug.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_prio.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_qfq.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_red.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfb.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfq.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_tbf.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_teql.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp_diag.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-09T08:39:49.304 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/sunrpc.ko 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/diag.ko 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/tipc.ko 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix/unix_diag.ko 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock.ko 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-09T08:39:49.305 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys/trusted.ko 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib/irqbypass.ko 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias.bin 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin 2024-03-09T08:39:49.306 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.alias.bin 2024-03-09T08:39:49.307 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.bin 2024-03-09T08:39:49.307 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.modinfo 2024-03-09T08:39:49.307 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep 2024-03-09T08:39:49.307 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep.bin 2024-03-09T08:39:49.307 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.devname 2024-03-09T08:39:49.307 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.order 2024-03-09T08:39:49.307 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.softdep 2024-03-09T08:39:49.307 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols 2024-03-09T08:39:49.307 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols.bin 2024-03-09T08:39:49.307 INFO:teuthology.orchestra.run.smithi134.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-09T08:39:49.311 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:49.311 DEBUG:teuthology.orchestra.run.smithi134:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-09T08:39:49.333 INFO:teuthology.orchestra.run.smithi134.stdout:postinstall scriptlet (using /bin/sh): 2024-03-09T08:39:49.333 INFO:teuthology.orchestra.run.smithi134.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-09T08:39:49.333 INFO:teuthology.orchestra.run.smithi134.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gdd5c1cd17167 /lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-09T08:39:49.333 INFO:teuthology.orchestra.run.smithi134.stdout:fi 2024-03-09T08:39:49.333 INFO:teuthology.orchestra.run.smithi134.stdout:for file in vmlinuz System.map config; do 2024-03-09T08:39:49.333 INFO:teuthology.orchestra.run.smithi134.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167"; then 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout: cp "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167" 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout: fi 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout:done 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout:new-kernel-pkg --remove 6.8.0-rc7-gdd5c1cd17167 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gdd5c1cd17167.img 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-install remove 6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout:fi 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-09T08:39:49.334 INFO:teuthology.orchestra.run.smithi134.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:49.335 INFO:teuthology.orchestra.run.smithi134.stdout:fi 2024-03-09T08:39:49.335 DEBUG:teuthology.orchestra.run.smithi134:> sudo rpm -qi grub2-tools 2024-03-09T08:39:49.410 INFO:teuthology.orchestra.run.smithi134.stdout:Name : grub2-tools 2024-03-09T08:39:49.410 INFO:teuthology.orchestra.run.smithi134.stdout:Epoch : 1 2024-03-09T08:39:49.410 INFO:teuthology.orchestra.run.smithi134.stdout:Version : 2.06 2024-03-09T08:39:49.410 INFO:teuthology.orchestra.run.smithi134.stdout:Release : 68.el9 2024-03-09T08:39:49.410 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: x86_64 2024-03-09T08:39:49.410 INFO:teuthology.orchestra.run.smithi134.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-09T08:39:49.410 INFO:teuthology.orchestra.run.smithi134.stdout:Group : Unspecified 2024-03-09T08:39:49.410 INFO:teuthology.orchestra.run.smithi134.stdout:Size : 8264254 2024-03-09T08:39:49.410 INFO:teuthology.orchestra.run.smithi134.stdout:License : GPLv3+ 2024-03-09T08:39:49.410 INFO:teuthology.orchestra.run.smithi134.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-09T08:39:49.410 INFO:teuthology.orchestra.run.smithi134.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:Packager : builder@centos.org 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:Vendor : CentOS 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:Summary : Support tools for GRUB. 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:Description : 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:hardware devices. 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:39:49.411 INFO:teuthology.orchestra.run.smithi134.stdout:This subpackage provides tools for support of all platforms. 2024-03-09T08:39:49.412 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-09T08:39:49.412 INFO:teuthology.task.kernel:Updating grub on smithi134 to boot 6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:49.413 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-09T08:39:50.034 INFO:teuthology.orchestra.run.smithi134.stderr:Generating grub configuration file ... 2024-03-09T08:39:51.434 INFO:teuthology.orchestra.run.smithi134.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-09T08:39:51.462 INFO:teuthology.orchestra.run.smithi134.stderr:done 2024-03-09T08:39:51.465 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-03-09T08:39:51.480 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.q5K54AG14a 2024-03-09T08:39:51.480 DEBUG:teuthology.orchestra.run.smithi134:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.q5K54AG14a 2024-03-09T08:39:51.549 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.q5K54AG14a 2024-03-09T08:39:51.698 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.q5K54AG14a is 6KB 2024-03-09T08:39:51.748 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.q5K54AG14a 2024-03-09T08:39:51.762 DEBUG:teuthology.orchestra.run.smithi134:> sudo /bin/ls /boot/loader/entries || true 2024-03-09T08:39:51.827 INFO:teuthology.orchestra.run.smithi134.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-09T08:39:51.827 INFO:teuthology.orchestra.run.smithi134.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-09T08:39:51.827 INFO:teuthology.orchestra.run.smithi134.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-09T08:39:51.827 INFO:teuthology.orchestra.run.smithi134.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167.conf 2024-03-09T08:39:51.828 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:39:52.018 DEBUG:teuthology.orchestra.run.smithi134:> sudo shutdown -r now 2024-03-09T08:40:18.877 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-09T08:40:18.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-03-09T08:40:18.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T08:40:22.024 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-09T08:40:22.025 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-09T08:40:22.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T08:41:18.936 DEBUG:teuthology.orchestra.remote:timed out 2024-03-09T08:41:22.030 DEBUG:teuthology.orchestra.remote:timed out 2024-03-09T08:41:27.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-03-09T08:41:27.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T08:41:31.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-09T08:41:31.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T08:41:34.536 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-03-09T08:41:35.185 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-09T08:41:35.185 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.8.0-rc7-gdd5c1cd17167"... 2024-03-09T08:41:35.185 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-03-09T08:41:35.230 INFO:teuthology.orchestra.run.smithi134.stdout:6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:41:35.230 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gdd5c1cd17167 vs 6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:41:35.230 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-09T08:41:35.230 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-09T08:41:36.234 DEBUG:teuthology.parallel:result is None 2024-03-09T08:41:43.613 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-03-09T08:41:44.199 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-03-09T08:41:44.199 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc7-gdd5c1cd17167"... 2024-03-09T08:41:44.200 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-03-09T08:41:44.243 INFO:teuthology.orchestra.run.smithi067.stdout:6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:41:44.243 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gdd5c1cd17167 vs 6.8.0-rc7-gdd5c1cd17167 2024-03-09T08:41:44.243 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-09T08:41:44.243 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-09T08:41:45.246 DEBUG:teuthology.parallel:result is None 2024-03-09T08:41:45.247 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-09T08:41:45.254 INFO:teuthology.task.internal:Creating test directory... 2024-03-09T08:41:45.254 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-09T08:41:45.259 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-09T08:41:45.307 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-09T08:41:45.315 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-09T08:41:45.322 INFO:teuthology.task.internal:Creating archive directory... 2024-03-09T08:41:45.322 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-09T08:41:45.346 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-09T08:41:45.435 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-09T08:41:45.442 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-09T08:41:45.442 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-09T08:41:45.447 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-09T08:41:45.585 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-09T08:41:45.591 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-09T08:41:45.634 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-09T08:41:45.681 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-09T08:41:45.686 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-09T08:41:45.694 INFO:teuthology.task.internal:Configuring sudo... 2024-03-09T08:41:45.694 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-09T08:41:45.700 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-09T08:41:45.776 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-09T08:41:45.785 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-09T08:41:45.785 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-09T08:41:45.790 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-09T08:41:45.834 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-09T08:41:45.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-09T08:41:46.064 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-09T08:41:46.134 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-09T08:41:46.239 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:41:46.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-09T08:41:46.335 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-09T08:41:46.405 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-09T08:41:46.501 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-09T08:41:46.568 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-09T08:41:46.652 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:41:46.653 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-09T08:41:46.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2024-03-09T08:41:46.725 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-03-09T08:41:46.845 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-09T08:41:46.861 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-09T08:41:47.366 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-09T08:41:47.373 INFO:teuthology.task.internal:Starting timer... 2024-03-09T08:41:47.374 INFO:teuthology.run_tasks:Running task pcp... 2024-03-09T08:41:47.385 INFO:teuthology.run_tasks:Running task selinux... 2024-03-09T08:41:47.393 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2024-03-09T08:41:47.502 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2024-03-09T08:41:47.507 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-03-09T08:41:47.686 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-03-09T08:41:47.690 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-09T08:41:47.691 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2024-03-09T08:41:47.738 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2024-03-09T08:41:47.738 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-03-09T08:41:47.830 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-03-09T08:41:47.830 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2024-03-09T08:41:47.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-09T08:41:47.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T08:41:47.908 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-09T08:41:47.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T08:41:47.977 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-09T08:41:47.977 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2024-03-09T08:41:48.067 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-03-09T08:41:48.233 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-09T08:41:48.245 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-09T08:41:48.267 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-09T08:41:48.268 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-09T08:47:55.812 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-03-09T08:47:55.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-03-09T08:47:55.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T08:47:55.972 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-03-09T08:47:56.125 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-03-09T08:47:56.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-09T08:47:56.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T08:47:56.263 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-03-09T08:47:56.410 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-09T08:47:56.410 INFO:teuthology.run_tasks:Running task clock... 2024-03-09T08:47:56.422 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-09T08:47:56.422 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-09T08:47:56.422 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-09T08:47:56.430 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-09T08:47:56.430 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-09T08:47:56.524 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-09T08:47:56.536 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-09T08:47:56.571 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-09T08:47:56.592 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-09T08:47:56.659 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-03-09T08:47:56.691 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2024-03-09T08:47:56.702 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-03-09T08:47:56.735 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2024-03-09T08:47:56.752 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-09T08:47:56.789 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-09T08:47:56.805 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-09T08:47:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-09T08:47:56.937 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-03-09T08:47:56.945 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-09T08:47:56.945 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-03-09T08:47:56.945 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T08:47:56.945 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T08:47:56.945 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T08:47:56.945 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T08:47:56.990 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-03-09T08:47:57.000 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-09T08:47:57.000 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-03-09T08:47:57.000 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T08:47:57.000 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T08:47:57.000 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T08:47:57.000 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T08:47:57.002 INFO:teuthology.run_tasks:Running task install... 2024-03-09T08:47:57.670 DEBUG:teuthology.task.install:project ceph 2024-03-09T08:47:57.670 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}} 2024-03-09T08:47:57.670 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'} 2024-03-09T08:47:57.671 INFO:teuthology.task.install:Using flavor: default 2024-03-09T08:47:57.677 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-09T08:47:57.677 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-09T08:47:57.677 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'tag': None, 'wait_for_package': False} 2024-03-09T08:47:57.678 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-09T08:47:57.680 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'tag': None, 'wait_for_package': False} 2024-03-09T08:47:57.681 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-09T08:47:57.921 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/ 2024-03-09T08:47:57.921 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1848.g8b896a9e 2024-03-09T08:47:57.930 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/ 2024-03-09T08:47:57.930 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1848.g8b896a9e 2024-03-09T08:47:58.094 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-09T08:47:58.094 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:47:58.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-09T08:47:58.114 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-09T08:47:58.115 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:47:58.115 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-09T08:47:58.200 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-09T08:47:58.200 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8b896a9e145796119f0451201dc4d53ddac97db0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-09T08:47:58.211 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-09T08:47:58.211 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8b896a9e145796119f0451201dc4d53ddac97db0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-09T08:47:58.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-09T08:47:58.349 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-09T08:47:58.477 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-09T08:47:58.484 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-09T08:47:58.567 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-03-09T08:47:58.571 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-03-09T08:47:58.573 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2024-03-09T08:47:58.578 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2024-03-09T08:47:58.920 INFO:teuthology.orchestra.run.smithi134.stdout:74 files removed 2024-03-09T08:47:58.947 INFO:teuthology.orchestra.run.smithi067.stdout:74 files removed 2024-03-09T08:47:58.961 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-09T08:47:58.987 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-09T08:47:59.965 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 166 kB/s | 83 kB 00:00 2024-03-09T08:48:00.135 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 132 kB/s | 83 kB 00:00 2024-03-09T08:48:00.524 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 22 kB/s | 10 kB 00:00 2024-03-09T08:48:00.636 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-03-09T08:48:01.024 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-03-09T08:48:01.195 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-03-09T08:48:01.595 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-09T08:48:01.874 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-09T08:48:03.337 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-03-09T08:48:03.524 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - BaseOS 5.7 MB/s | 2.0 MB 00:00 2024-03-09T08:48:04.512 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-09T08:48:04.800 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.0 MB 00:01 2024-03-09T08:48:06.471 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-03-09T08:48:22.067 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-03-09T08:48:25.363 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 426 kB/s | 6.1 MB 00:14 2024-03-09T08:48:27.167 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 6.1 MB 00:01 2024-03-09T08:48:27.341 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 15 kB 00:00 2024-03-09T08:48:29.093 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 15 kB 00:00 2024-03-09T08:48:29.316 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-09T08:48:31.659 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 8.6 MB/s | 21 MB 00:02 2024-03-09T08:48:35.350 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-09T08:48:37.395 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-09T08:48:38.231 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-09T08:48:38.233 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-09T08:48:38.257 INFO:teuthology.orchestra.run.smithi067.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-09T08:48:38.259 INFO:teuthology.orchestra.run.smithi067.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-09T08:48:38.260 INFO:teuthology.orchestra.run.smithi067.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-09T08:48:38.262 INFO:teuthology.orchestra.run.smithi067.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-09T08:48:38.265 INFO:teuthology.orchestra.run.smithi067.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-09T08:48:38.269 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-09T08:48:38.270 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-09T08:48:38.367 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-03-09T08:48:38.394 INFO:teuthology.orchestra.run.smithi067.stdout:============================================================================================ 2024-03-09T08:48:38.394 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-03-09T08:48:38.394 INFO:teuthology.orchestra.run.smithi067.stdout:============================================================================================ 2024-03-09T08:48:38.394 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 6.6 k 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.3 M 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 889 k 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 145 k 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 1.5 M 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 138 k 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 1.8 M 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 7.4 M 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 49 k 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 9.8 M 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 47 M 2024-03-09T08:48:38.395 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 264 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 748 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 220 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 27 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 18 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 31 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 746 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 127 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-09T08:48:38.396 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 161 k 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 320 k 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 298 k 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 99 k 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 85 k 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 170 k 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2024-03-09T08:48:38.397 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-09T08:48:38.398 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.4 M 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-03-09T08:48:38.399 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 20 M 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 31 k 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 2.3 M 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 248 k 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 4.7 M 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 17 M 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 16 k 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 25 k 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-09T08:48:38.400 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 163 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 501 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-09T08:48:38.401 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.1 M 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-09T08:48:38.402 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 45 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 131 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-09T08:48:38.403 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-09T08:48:38.404 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-09T08:48:38.405 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-09T08:48:38.406 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-09T08:48:38.407 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout:Installing weak dependencies: 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout:============================================================================================ 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout:Install 150 Packages 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 24 Packages 2024-03-09T08:48:38.408 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:48:38.409 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 224 M 2024-03-09T08:48:38.409 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-03-09T08:48:40.176 INFO:teuthology.orchestra.run.smithi067.stdout:(1/174): ceph-19.0.0-1848.g8b896a9e.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-03-09T08:48:40.226 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-09T08:48:40.228 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-09T08:48:40.250 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-09T08:48:40.251 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-09T08:48:40.252 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-09T08:48:40.254 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-09T08:48:40.256 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-09T08:48:40.260 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-09T08:48:40.261 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-09T08:48:40.351 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-03-09T08:48:40.370 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================ 2024-03-09T08:48:40.370 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-03-09T08:48:40.370 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================ 2024-03-09T08:48:40.370 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-03-09T08:48:40.370 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-09T08:48:40.370 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 6.6 k 2024-03-09T08:48:40.370 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.3 M 2024-03-09T08:48:40.370 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 889 k 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 145 k 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 1.5 M 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 138 k 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 1.8 M 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 7.4 M 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 49 k 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 9.8 M 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 47 M 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 264 k 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 748 k 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 220 k 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 27 k 2024-03-09T08:48:40.371 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 18 k 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 31 k 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 746 k 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 127 k 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-09T08:48:40.372 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 161 k 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 320 k 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 298 k 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 99 k 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 85 k 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 170 k 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-09T08:48:40.373 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.4 M 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-09T08:48:40.374 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 20 M 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 31 k 2024-03-09T08:48:40.375 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 2.3 M 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 248 k 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 4.7 M 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 17 M 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 16 k 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 25 k 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-09T08:48:40.376 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 163 k 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 501 k 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.1 M 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-09T08:48:40.377 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-09T08:48:40.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 45 k 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 131 k 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-09T08:48:40.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-09T08:48:40.380 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-09T08:48:40.381 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-09T08:48:40.382 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-09T08:48:40.383 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-09T08:48:40.384 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-09T08:48:40.384 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-03-09T08:48:40.384 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-09T08:48:40.384 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:48:40.384 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-03-09T08:48:40.384 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================ 2024-03-09T08:48:40.384 INFO:teuthology.orchestra.run.smithi134.stdout:Install 150 Packages 2024-03-09T08:48:40.384 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 24 Packages 2024-03-09T08:48:40.384 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:48:40.385 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 224 M 2024-03-09T08:48:40.385 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-03-09T08:48:41.135 INFO:teuthology.orchestra.run.smithi067.stdout:(2/174): ceph-fuse-19.0.0-1848.g8b896a9e.el9.x8 928 kB/s | 889 kB 00:00 2024-03-09T08:48:41.172 INFO:teuthology.orchestra.run.smithi134.stdout:(1/174): ceph-19.0.0-1848.g8b896a9e.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-03-09T08:48:41.393 INFO:teuthology.orchestra.run.smithi067.stdout:(3/174): ceph-immutable-object-cache-19.0.0-184 565 kB/s | 145 kB 00:00 2024-03-09T08:48:42.172 INFO:teuthology.orchestra.run.smithi134.stdout:(2/174): ceph-fuse-19.0.0-1848.g8b896a9e.el9.x8 890 kB/s | 889 kB 00:00 2024-03-09T08:48:42.347 INFO:teuthology.orchestra.run.smithi134.stdout:(3/174): ceph-immutable-object-cache-19.0.0-184 835 kB/s | 145 kB 00:00 2024-03-09T08:48:43.444 INFO:teuthology.orchestra.run.smithi067.stdout:(4/174): ceph-mds-19.0.0-1848.g8b896a9e.el9.x86 1.1 MB/s | 2.3 MB 00:02 2024-03-09T08:48:43.514 INFO:teuthology.orchestra.run.smithi134.stdout:(4/174): ceph-mds-19.0.0-1848.g8b896a9e.el9.x86 2.0 MB/s | 2.3 MB 00:01 2024-03-09T08:48:44.006 INFO:teuthology.orchestra.run.smithi134.stdout:(5/174): ceph-mgr-19.0.0-1848.g8b896a9e.el9.x86 3.0 MB/s | 1.5 MB 00:00 2024-03-09T08:48:44.777 INFO:teuthology.orchestra.run.smithi067.stdout:(5/174): ceph-mgr-19.0.0-1848.g8b896a9e.el9.x86 1.1 MB/s | 1.5 MB 00:01 2024-03-09T08:48:44.998 INFO:teuthology.orchestra.run.smithi134.stdout:(6/174): ceph-mon-19.0.0-1848.g8b896a9e.el9.x86 4.7 MB/s | 4.7 MB 00:00 2024-03-09T08:48:45.540 INFO:teuthology.orchestra.run.smithi134.stdout:(7/174): ceph-base-19.0.0-1848.g8b896a9e.el9.x8 1.1 MB/s | 5.3 MB 00:04 2024-03-09T08:48:47.915 INFO:teuthology.orchestra.run.smithi134.stdout:(8/174): ceph-osd-19.0.0-1848.g8b896a9e.el9.x86 5.8 MB/s | 17 MB 00:02 2024-03-09T08:48:47.998 INFO:teuthology.orchestra.run.smithi134.stdout:(9/174): ceph-selinux-19.0.0-1848.g8b896a9e.el9 306 kB/s | 25 kB 00:00 2024-03-09T08:48:48.730 INFO:teuthology.orchestra.run.smithi067.stdout:(6/174): ceph-base-19.0.0-1848.g8b896a9e.el9.x8 610 kB/s | 5.3 MB 00:08 2024-03-09T08:48:49.228 INFO:teuthology.orchestra.run.smithi067.stdout:(7/174): ceph-mon-19.0.0-1848.g8b896a9e.el9.x86 1.1 MB/s | 4.7 MB 00:04 2024-03-09T08:48:55.281 INFO:teuthology.orchestra.run.smithi134.stdout:(10/174): ceph-radosgw-19.0.0-1848.g8b896a9e.el 1.0 MB/s | 9.8 MB 00:09 2024-03-09T08:48:55.940 INFO:teuthology.orchestra.run.smithi134.stdout:(11/174): cephfs-mirror-19.0.0-1848.g8b896a9e.e 335 kB/s | 220 kB 00:00 2024-03-09T08:48:56.065 INFO:teuthology.orchestra.run.smithi134.stdout:(12/174): cephfs-shell-19.0.0-1848.g8b896a9e.el 219 kB/s | 27 kB 00:00 2024-03-09T08:48:56.174 INFO:teuthology.orchestra.run.smithi134.stdout:(13/174): libcephfs-devel-19.0.0-1848.g8b896a9e 290 kB/s | 31 kB 00:00 2024-03-09T08:48:57.833 INFO:teuthology.orchestra.run.smithi134.stdout:(14/174): libcephfs2-19.0.0-1848.g8b896a9e.el9. 450 kB/s | 746 kB 00:01 2024-03-09T08:48:58.224 INFO:teuthology.orchestra.run.smithi134.stdout:(15/174): libcephsqlite-19.0.0-1848.g8b896a9e.e 418 kB/s | 163 kB 00:00 2024-03-09T08:48:58.533 INFO:teuthology.orchestra.run.smithi134.stdout:(16/174): librados-devel-19.0.0-1848.g8b896a9e. 412 kB/s | 127 kB 00:00 2024-03-09T08:48:59.467 INFO:teuthology.orchestra.run.smithi134.stdout:(17/174): libradosstriper1-19.0.0-1848.g8b896a9 538 kB/s | 501 kB 00:00 2024-03-09T08:49:00.659 INFO:teuthology.orchestra.run.smithi134.stdout:(18/174): ceph-test-19.0.0-1848.g8b896a9e.el9.x 3.7 MB/s | 47 MB 00:12 2024-03-09T08:49:00.750 INFO:teuthology.orchestra.run.smithi134.stdout:(19/174): python3-ceph-argparse-19.0.0-1848.g8b 500 kB/s | 45 kB 00:00 2024-03-09T08:49:00.901 INFO:teuthology.orchestra.run.smithi134.stdout:(20/174): python3-ceph-common-19.0.0-1848.g8b89 874 kB/s | 131 kB 00:00 2024-03-09T08:49:00.993 INFO:teuthology.orchestra.run.smithi134.stdout:(21/174): python3-cephfs-19.0.0-1848.g8b896a9e. 1.7 MB/s | 161 kB 00:00 2024-03-09T08:49:01.160 INFO:teuthology.orchestra.run.smithi134.stdout:(22/174): python3-rados-19.0.0-1848.g8b896a9e.e 1.9 MB/s | 320 kB 00:00 2024-03-09T08:49:01.327 INFO:teuthology.orchestra.run.smithi134.stdout:(23/174): python3-rbd-19.0.0-1848.g8b896a9e.el9 1.7 MB/s | 298 kB 00:00 2024-03-09T08:49:01.419 INFO:teuthology.orchestra.run.smithi134.stdout:(24/174): python3-rgw-19.0.0-1848.g8b896a9e.el9 1.1 MB/s | 99 kB 00:00 2024-03-09T08:49:01.511 INFO:teuthology.orchestra.run.smithi134.stdout:(25/174): rbd-fuse-19.0.0-1848.g8b896a9e.el9.x8 943 kB/s | 85 kB 00:00 2024-03-09T08:49:02.161 INFO:teuthology.orchestra.run.smithi067.stdout:(8/174): ceph-radosgw-19.0.0-1848.g8b896a9e.el9 774 kB/s | 9.8 MB 00:12 2024-03-09T08:49:02.261 INFO:teuthology.orchestra.run.smithi067.stdout:(9/174): ceph-selinux-19.0.0-1848.g8b896a9e.el9 255 kB/s | 25 kB 00:00 2024-03-09T08:49:02.653 INFO:teuthology.orchestra.run.smithi134.stdout:(26/174): rbd-mirror-19.0.0-1848.g8b896a9e.el9. 2.7 MB/s | 3.1 MB 00:01 2024-03-09T08:49:02.753 INFO:teuthology.orchestra.run.smithi134.stdout:(27/174): rbd-nbd-19.0.0-1848.g8b896a9e.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-09T08:49:02.862 INFO:teuthology.orchestra.run.smithi134.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1848.g 292 kB/s | 31 kB 00:00 2024-03-09T08:49:02.962 INFO:teuthology.orchestra.run.smithi134.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1848.g8b896a9 1.4 MB/s | 138 kB 00:00 2024-03-09T08:49:03.286 INFO:teuthology.orchestra.run.smithi067.stdout:(10/174): ceph-osd-19.0.0-1848.g8b896a9e.el9.x8 1.2 MB/s | 17 MB 00:14 2024-03-09T08:49:03.529 INFO:teuthology.orchestra.run.smithi134.stdout:(30/174): ceph-common-19.0.0-1848.g8b896a9e.el9 918 kB/s | 20 MB 00:22 2024-03-09T08:49:03.570 INFO:teuthology.orchestra.run.smithi067.stdout:(11/174): cephfs-mirror-19.0.0-1848.g8b896a9e.e 781 kB/s | 220 kB 00:00 2024-03-09T08:49:03.654 INFO:teuthology.orchestra.run.smithi134.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1848.g8b896 2.6 MB/s | 1.8 MB 00:00 2024-03-09T08:49:03.670 INFO:teuthology.orchestra.run.smithi067.stdout:(12/174): cephfs-shell-19.0.0-1848.g8b896a9e.el 272 kB/s | 27 kB 00:00 2024-03-09T08:49:03.770 INFO:teuthology.orchestra.run.smithi067.stdout:(13/174): libcephfs-devel-19.0.0-1848.g8b896a9e 315 kB/s | 31 kB 00:00 2024-03-09T08:49:03.829 INFO:teuthology.orchestra.run.smithi134.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1848.g8b 1.4 MB/s | 248 kB 00:00 2024-03-09T08:49:03.921 INFO:teuthology.orchestra.run.smithi134.stdout:(33/174): ceph-mgr-rook-19.0.0-1848.g8b896a9e.e 542 kB/s | 49 kB 00:00 2024-03-09T08:49:04.030 INFO:teuthology.orchestra.run.smithi134.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1848.g8 152 kB/s | 16 kB 00:00 2024-03-09T08:49:04.387 INFO:teuthology.orchestra.run.smithi067.stdout:(14/174): libcephfs2-19.0.0-1848.g8b896a9e.el9. 1.2 MB/s | 746 kB 00:00 2024-03-09T08:49:04.529 INFO:teuthology.orchestra.run.smithi067.stdout:(15/174): libcephsqlite-19.0.0-1848.g8b896a9e.e 1.1 MB/s | 163 kB 00:00 2024-03-09T08:49:04.547 INFO:teuthology.orchestra.run.smithi134.stdout:(35/174): ceph-volume-19.0.0-1848.g8b896a9e.el9 511 kB/s | 264 kB 00:00 2024-03-09T08:49:04.688 INFO:teuthology.orchestra.run.smithi067.stdout:(16/174): librados-devel-19.0.0-1848.g8b896a9e. 802 kB/s | 127 kB 00:00 2024-03-09T08:49:05.146 INFO:teuthology.orchestra.run.smithi067.stdout:(17/174): libradosstriper1-19.0.0-1848.g8b896a9 1.1 MB/s | 501 kB 00:00 2024-03-09T08:49:05.472 INFO:teuthology.orchestra.run.smithi134.stdout:(36/174): cephadm-19.0.0-1848.g8b896a9e.el9.noa 810 kB/s | 748 kB 00:00 2024-03-09T08:49:05.648 INFO:teuthology.orchestra.run.smithi134.stdout:(37/174): cephfs-top-19.0.0-1848.g8b896a9e.el9. 105 kB/s | 18 kB 00:00 2024-03-09T08:49:06.107 INFO:teuthology.orchestra.run.smithi134.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 1.4 MB/s | 662 kB 00:00 2024-03-09T08:49:06.231 INFO:teuthology.orchestra.run.smithi134.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 7.3 MB/s | 920 kB 00:00 2024-03-09T08:49:06.315 INFO:teuthology.orchestra.run.smithi134.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 106 kB 00:00 2024-03-09T08:49:06.407 INFO:teuthology.orchestra.run.smithi134.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-09T08:49:06.466 INFO:teuthology.orchestra.run.smithi134.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 548 kB/s | 32 kB 00:00 2024-03-09T08:49:06.674 INFO:teuthology.orchestra.run.smithi134.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-03-09T08:49:06.733 INFO:teuthology.orchestra.run.smithi134.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 298 kB/s | 17 kB 00:00 2024-03-09T08:49:06.835 INFO:teuthology.orchestra.run.smithi134.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 99 kB/s | 10 kB 00:00 2024-03-09T08:49:07.000 INFO:teuthology.orchestra.run.smithi134.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 968 kB/s | 159 kB 00:00 2024-03-09T08:49:07.059 INFO:teuthology.orchestra.run.smithi134.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 766 kB/s | 44 kB 00:00 2024-03-09T08:49:07.159 INFO:teuthology.orchestra.run.smithi134.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 661 kB 00:00 2024-03-09T08:49:07.251 INFO:teuthology.orchestra.run.smithi134.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 578 kB 00:00 2024-03-09T08:49:07.318 INFO:teuthology.orchestra.run.smithi134.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 703 kB/s | 46 kB 00:00 2024-03-09T08:49:07.393 INFO:teuthology.orchestra.run.smithi134.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-09T08:49:07.485 INFO:teuthology.orchestra.run.smithi134.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-03-09T08:49:07.569 INFO:teuthology.orchestra.run.smithi134.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2024-03-09T08:49:07.636 INFO:teuthology.orchestra.run.smithi134.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 519 kB/s | 34 kB 00:00 2024-03-09T08:49:07.920 INFO:teuthology.orchestra.run.smithi134.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-03-09T08:49:08.111 INFO:teuthology.orchestra.run.smithi134.stdout:(56/174): postgresql-13.11-1.el9.x86_64.rpm 8.0 MB/s | 1.5 MB 00:00 2024-03-09T08:49:08.212 INFO:teuthology.orchestra.run.smithi134.stdout:(57/174): postgresql-contrib-13.11-1.el9.x86_64 8.0 MB/s | 811 kB 00:00 2024-03-09T08:49:08.287 INFO:teuthology.orchestra.run.smithi134.stdout:(58/174): postgresql-private-libs-13.11-1.el9.x 1.8 MB/s | 134 kB 00:00 2024-03-09T08:49:08.422 INFO:teuthology.orchestra.run.smithi067.stdout:(18/174): librgw2-19.0.0-1848.g8b896a9e.el9.x86 1.6 MB/s | 5.1 MB 00:03 2024-03-09T08:49:08.513 INFO:teuthology.orchestra.run.smithi067.stdout:(19/174): python3-ceph-argparse-19.0.0-1848.g8b 499 kB/s | 45 kB 00:00 2024-03-09T08:49:08.605 INFO:teuthology.orchestra.run.smithi067.stdout:(20/174): python3-ceph-common-19.0.0-1848.g8b89 1.4 MB/s | 131 kB 00:00 2024-03-09T08:49:08.764 INFO:teuthology.orchestra.run.smithi067.stdout:(21/174): python3-cephfs-19.0.0-1848.g8b896a9e. 1.0 MB/s | 161 kB 00:00 2024-03-09T08:49:08.838 INFO:teuthology.orchestra.run.smithi134.stdout:(59/174): postgresql-server-13.11-1.el9.x86_64. 10 MB/s | 5.7 MB 00:00 2024-03-09T08:49:08.971 INFO:teuthology.orchestra.run.smithi134.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-03-09T08:49:09.038 INFO:teuthology.orchestra.run.smithi134.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-09T08:49:09.039 INFO:teuthology.orchestra.run.smithi067.stdout:(22/174): ceph-common-19.0.0-1848.g8b896a9e.el9 714 kB/s | 20 MB 00:29 2024-03-09T08:49:09.105 INFO:teuthology.orchestra.run.smithi067.stdout:(23/174): python3-rados-19.0.0-1848.g8b896a9e.e 942 kB/s | 320 kB 00:00 2024-03-09T08:49:09.199 INFO:teuthology.orchestra.run.smithi067.stdout:(24/174): python3-rgw-19.0.0-1848.g8b896a9e.el9 1.1 MB/s | 99 kB 00:00 2024-03-09T08:49:09.292 INFO:teuthology.orchestra.run.smithi067.stdout:(25/174): rbd-fuse-19.0.0-1848.g8b896a9e.el9.x8 936 kB/s | 85 kB 00:00 2024-03-09T08:49:09.490 INFO:teuthology.orchestra.run.smithi134.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 5.8 MB 00:00 2024-03-09T08:49:09.546 INFO:teuthology.orchestra.run.smithi134.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 497 kB/s | 24 kB 00:00 2024-03-09T08:49:09.607 INFO:teuthology.orchestra.run.smithi134.stdout:(64/174): python3-jsonpointer-2.0-4.el9.noarch. 273 kB/s | 18 kB 00:00 2024-03-09T08:49:09.698 INFO:teuthology.orchestra.run.smithi134.stdout:(65/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 144 kB 00:00 2024-03-09T08:49:09.774 INFO:teuthology.orchestra.run.smithi134.stdout:(66/174): python3-markupsafe-1.1.1-12.el9.x86_6 424 kB/s | 32 kB 00:00 2024-03-09T08:49:10.207 INFO:teuthology.orchestra.run.smithi134.stdout:(67/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 4.8 MB 00:00 2024-03-09T08:49:10.391 INFO:teuthology.orchestra.run.smithi134.stdout:(68/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-03-09T08:49:10.458 INFO:teuthology.orchestra.run.smithi134.stdout:(69/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-03-09T08:49:10.524 INFO:teuthology.orchestra.run.smithi134.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-09T08:49:10.849 INFO:teuthology.orchestra.run.smithi067.stdout:(26/174): rbd-mirror-19.0.0-1848.g8b896a9e.el9. 2.0 MB/s | 3.1 MB 00:01 2024-03-09T08:49:10.875 INFO:teuthology.orchestra.run.smithi134.stdout:(71/174): python3-prettytable-0.7.2-27.el9.noar 118 kB/s | 41 kB 00:00 2024-03-09T08:49:10.949 INFO:teuthology.orchestra.run.smithi067.stdout:(27/174): rbd-nbd-19.0.0-1848.g8b896a9e.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-09T08:49:10.967 INFO:teuthology.orchestra.run.smithi134.stdout:(72/174): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 238 kB 00:00 2024-03-09T08:49:11.042 INFO:teuthology.orchestra.run.smithi134.stdout:(73/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-09T08:49:11.049 INFO:teuthology.orchestra.run.smithi067.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1848.g 316 kB/s | 31 kB 00:00 2024-03-09T08:49:11.118 INFO:teuthology.orchestra.run.smithi134.stdout:(74/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-03-09T08:49:11.150 INFO:teuthology.orchestra.run.smithi067.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1848.g8b896a9 1.4 MB/s | 138 kB 00:00 2024-03-09T08:49:11.184 INFO:teuthology.orchestra.run.smithi134.stdout:(75/174): python3-pytz-2021.1-5.el9.noarch.rpm 719 kB/s | 47 kB 00:00 2024-03-09T08:49:11.251 INFO:teuthology.orchestra.run.smithi134.stdout:(76/174): python3-requests-oauthlib-1.3.0-12.el 649 kB/s | 43 kB 00:00 2024-03-09T08:49:11.485 INFO:teuthology.orchestra.run.smithi134.stdout:(77/174): ceph-mgr-diskprediction-local-19.0.0- 955 kB/s | 7.4 MB 00:07 2024-03-09T08:49:11.516 INFO:teuthology.orchestra.run.smithi067.stdout:(30/174): python3-rbd-19.0.0-1848.g8b896a9e.el9 120 kB/s | 298 kB 00:02 2024-03-09T08:49:11.850 INFO:teuthology.orchestra.run.smithi067.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1848.g8b896 2.5 MB/s | 1.8 MB 00:00 2024-03-09T08:49:12.001 INFO:teuthology.orchestra.run.smithi067.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1848.g8b 1.6 MB/s | 248 kB 00:00 2024-03-09T08:49:12.002 INFO:teuthology.orchestra.run.smithi134.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 77 kB/s | 39 kB 00:00 2024-03-09T08:49:12.076 INFO:teuthology.orchestra.run.smithi067.stdout:(33/174): ceph-mgr-rook-19.0.0-1848.g8b896a9e.e 666 kB/s | 49 kB 00:00 2024-03-09T08:49:12.160 INFO:teuthology.orchestra.run.smithi067.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1848.g8 198 kB/s | 16 kB 00:00 2024-03-09T08:49:12.326 INFO:teuthology.orchestra.run.smithi067.stdout:(35/174): ceph-volume-19.0.0-1848.g8b896a9e.el9 1.6 MB/s | 264 kB 00:00 2024-03-09T08:49:12.510 INFO:teuthology.orchestra.run.smithi134.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-03-09T08:49:12.536 INFO:teuthology.orchestra.run.smithi134.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 77 kB/s | 41 kB 00:00 2024-03-09T08:49:12.611 INFO:teuthology.orchestra.run.smithi134.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 300 kB 00:00 2024-03-09T08:49:12.636 INFO:teuthology.orchestra.run.smithi134.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 565 kB/s | 56 kB 00:00 2024-03-09T08:49:12.660 INFO:teuthology.orchestra.run.smithi067.stdout:(36/174): cephadm-19.0.0-1848.g8b896a9e.el9.noa 2.2 MB/s | 748 kB 00:00 2024-03-09T08:49:12.695 INFO:teuthology.orchestra.run.smithi134.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 760 kB/s | 63 kB 00:00 2024-03-09T08:49:12.720 INFO:teuthology.orchestra.run.smithi134.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 529 kB/s | 44 kB 00:00 2024-03-09T08:49:12.754 INFO:teuthology.orchestra.run.smithi134.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-09T08:49:12.760 INFO:teuthology.orchestra.run.smithi067.stdout:(37/174): cephfs-top-19.0.0-1848.g8b896a9e.el9. 183 kB/s | 18 kB 00:00 2024-03-09T08:49:12.796 INFO:teuthology.orchestra.run.smithi134.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-09T08:49:12.955 INFO:teuthology.orchestra.run.smithi134.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.1 MB 00:00 2024-03-09T08:49:12.980 INFO:teuthology.orchestra.run.smithi134.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 563 kB/s | 103 kB 00:00 2024-03-09T08:49:13.005 INFO:teuthology.orchestra.run.smithi134.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-03-09T08:49:13.047 INFO:teuthology.orchestra.run.smithi134.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2024-03-09T08:49:13.252 INFO:teuthology.orchestra.run.smithi067.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 1.3 MB/s | 662 kB 00:00 2024-03-09T08:49:13.428 INFO:teuthology.orchestra.run.smithi067.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 5.2 MB/s | 920 kB 00:00 2024-03-09T08:49:13.511 INFO:teuthology.orchestra.run.smithi067.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 106 kB 00:00 2024-03-09T08:49:13.587 INFO:teuthology.orchestra.run.smithi067.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-03-09T08:49:13.662 INFO:teuthology.orchestra.run.smithi067.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 427 kB/s | 32 kB 00:00 2024-03-09T08:49:14.063 INFO:teuthology.orchestra.run.smithi067.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.5 MB/s | 3.0 MB 00:00 2024-03-09T08:49:14.121 INFO:teuthology.orchestra.run.smithi067.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 299 kB/s | 17 kB 00:00 2024-03-09T08:49:14.188 INFO:teuthology.orchestra.run.smithi067.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 152 kB/s | 10 kB 00:00 2024-03-09T08:49:14.263 INFO:teuthology.orchestra.run.smithi067.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-09T08:49:14.339 INFO:teuthology.orchestra.run.smithi067.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 593 kB/s | 44 kB 00:00 2024-03-09T08:49:14.464 INFO:teuthology.orchestra.run.smithi134.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 128 kB/s | 187 kB 00:01 2024-03-09T08:49:14.681 INFO:teuthology.orchestra.run.smithi067.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 661 kB 00:00 2024-03-09T08:49:14.814 INFO:teuthology.orchestra.run.smithi134.stdout:(92/174): libgfortran-11.4.1-3.el9.x86_64.rpm 455 kB/s | 803 kB 00:01 2024-03-09T08:49:14.840 INFO:teuthology.orchestra.run.smithi134.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 521 kB/s | 195 kB 00:00 2024-03-09T08:49:14.898 INFO:teuthology.orchestra.run.smithi067.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 578 kB 00:00 2024-03-09T08:49:14.948 INFO:teuthology.orchestra.run.smithi134.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 938 kB/s | 125 kB 00:00 2024-03-09T08:49:14.965 INFO:teuthology.orchestra.run.smithi067.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 705 kB/s | 46 kB 00:00 2024-03-09T08:49:15.090 INFO:teuthology.orchestra.run.smithi067.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 282 kB 00:00 2024-03-09T08:49:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2024-03-09T08:49:15.307 INFO:teuthology.orchestra.run.smithi067.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.8 MB/s | 217 kB 00:00 2024-03-09T08:49:15.374 INFO:teuthology.orchestra.run.smithi067.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 517 kB/s | 34 kB 00:00 2024-03-09T08:49:15.749 INFO:teuthology.orchestra.run.smithi134.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 24 kB/s | 21 kB 00:00 2024-03-09T08:49:15.865 INFO:teuthology.orchestra.run.smithi134.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 28 kB/s | 25 kB 00:00 2024-03-09T08:49:16.350 INFO:teuthology.orchestra.run.smithi067.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 4.9 MB/s | 4.7 MB 00:00 2024-03-09T08:49:16.675 INFO:teuthology.orchestra.run.smithi067.stdout:(56/174): postgresql-13.11-1.el9.x86_64.rpm 4.7 MB/s | 1.5 MB 00:00 2024-03-09T08:49:16.975 INFO:teuthology.orchestra.run.smithi067.stdout:(57/174): postgresql-contrib-13.11-1.el9.x86_64 2.6 MB/s | 811 kB 00:00 2024-03-09T08:49:17.067 INFO:teuthology.orchestra.run.smithi067.stdout:(58/174): postgresql-private-libs-13.11-1.el9.x 1.4 MB/s | 134 kB 00:00 2024-03-09T08:49:17.419 INFO:teuthology.orchestra.run.smithi134.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 158 kB/s | 245 kB 00:01 2024-03-09T08:49:17.541 INFO:teuthology.orchestra.run.smithi134.stdout:(98/174): librgw2-19.0.0-1848.g8b896a9e.el9.x86 290 kB/s | 5.1 MB 00:18 2024-03-09T08:49:17.951 INFO:teuthology.orchestra.run.smithi067.stdout:(59/174): postgresql-server-13.11-1.el9.x86_64. 6.4 MB/s | 5.7 MB 00:00 2024-03-09T08:49:18.143 INFO:teuthology.orchestra.run.smithi067.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-03-09T08:49:18.210 INFO:teuthology.orchestra.run.smithi067.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-09T08:49:18.716 INFO:teuthology.orchestra.run.smithi134.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 191 kB/s | 248 kB 00:01 2024-03-09T08:49:18.767 INFO:teuthology.orchestra.run.smithi134.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 177 kB/s | 216 kB 00:01 2024-03-09T08:49:18.893 INFO:teuthology.orchestra.run.smithi067.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 8.5 MB/s | 5.8 MB 00:00 2024-03-09T08:49:18.977 INFO:teuthology.orchestra.run.smithi067.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 296 kB/s | 24 kB 00:00 2024-03-09T08:49:19.044 INFO:teuthology.orchestra.run.smithi067.stdout:(64/174): python3-jsonpointer-2.0-4.el9.noarch. 274 kB/s | 18 kB 00:00 2024-03-09T08:49:19.228 INFO:teuthology.orchestra.run.smithi067.stdout:(65/174): python3-mako-1.1.4-6.el9.noarch.rpm 789 kB/s | 144 kB 00:00 2024-03-09T08:49:19.286 INFO:teuthology.orchestra.run.smithi067.stdout:(66/174): python3-markupsafe-1.1.1-12.el9.x86_6 548 kB/s | 32 kB 00:00 2024-03-09T08:49:19.567 INFO:teuthology.orchestra.run.smithi134.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 41 kB/s | 35 kB 00:00 2024-03-09T08:49:19.642 INFO:teuthology.orchestra.run.smithi134.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 51 kB/s | 44 kB 00:00 2024-03-09T08:49:19.751 INFO:teuthology.orchestra.run.smithi134.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 208 kB/s | 22 kB 00:00 2024-03-09T08:49:19.837 INFO:teuthology.orchestra.run.smithi067.stdout:(67/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.7 MB/s | 4.8 MB 00:00 2024-03-09T08:49:19.926 INFO:teuthology.orchestra.run.smithi134.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-03-09T08:49:19.928 INFO:teuthology.orchestra.run.smithi067.stdout:(68/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 217 kB 00:00 2024-03-09T08:49:20.004 INFO:teuthology.orchestra.run.smithi067.stdout:(69/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.3 MB/s | 174 kB 00:00 2024-03-09T08:49:20.071 INFO:teuthology.orchestra.run.smithi067.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-09T08:49:20.138 INFO:teuthology.orchestra.run.smithi067.stdout:(71/174): python3-prettytable-0.7.2-27.el9.noar 625 kB/s | 41 kB 00:00 2024-03-09T08:49:20.213 INFO:teuthology.orchestra.run.smithi067.stdout:(72/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-09T08:49:20.288 INFO:teuthology.orchestra.run.smithi067.stdout:(73/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-09T08:49:20.380 INFO:teuthology.orchestra.run.smithi067.stdout:(74/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 211 kB 00:00 2024-03-09T08:49:20.456 INFO:teuthology.orchestra.run.smithi067.stdout:(75/174): python3-pytz-2021.1-5.el9.noarch.rpm 636 kB/s | 47 kB 00:00 2024-03-09T08:49:20.523 INFO:teuthology.orchestra.run.smithi067.stdout:(76/174): python3-requests-oauthlib-1.3.0-12.el 650 kB/s | 43 kB 00:00 2024-03-09T08:49:20.543 INFO:teuthology.orchestra.run.smithi134.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-03-09T08:49:20.726 INFO:teuthology.orchestra.run.smithi134.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-03-09T08:49:20.868 INFO:teuthology.orchestra.run.smithi134.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 137 kB/s | 19 kB 00:00 2024-03-09T08:49:20.894 INFO:teuthology.orchestra.run.smithi134.stdout:(108/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 71 kB/s | 94 kB 00:01 2024-03-09T08:49:21.177 INFO:teuthology.orchestra.run.smithi134.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 92 kB/s | 26 kB 00:00 2024-03-09T08:49:21.344 INFO:teuthology.orchestra.run.smithi134.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 292 kB/s | 49 kB 00:00 2024-03-09T08:49:21.462 INFO:teuthology.orchestra.run.smithi134.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 581 kB/s | 67 kB 00:00 2024-03-09T08:49:21.662 INFO:teuthology.orchestra.run.smithi134.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-03-09T08:49:21.678 INFO:teuthology.orchestra.run.smithi134.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 700 kB/s | 151 kB 00:00 2024-03-09T08:49:21.862 INFO:teuthology.orchestra.run.smithi134.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-03-09T08:49:21.904 INFO:teuthology.orchestra.run.smithi134.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-03-09T08:49:21.915 INFO:teuthology.orchestra.run.smithi067.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-03-09T08:49:21.963 INFO:teuthology.orchestra.run.smithi134.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 433 kB/s | 43 kB 00:00 2024-03-09T08:49:21.965 INFO:teuthology.orchestra.run.smithi067.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 791 kB/s | 39 kB 00:00 2024-03-09T08:49:22.015 INFO:teuthology.orchestra.run.smithi067.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 830 kB/s | 41 kB 00:00 2024-03-09T08:49:22.021 INFO:teuthology.orchestra.run.smithi134.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2024-03-09T08:49:22.080 INFO:teuthology.orchestra.run.smithi134.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-03-09T08:49:22.099 INFO:teuthology.orchestra.run.smithi067.stdout:(80/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-03-09T08:49:22.147 INFO:teuthology.orchestra.run.smithi134.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-03-09T08:49:22.231 INFO:teuthology.orchestra.run.smithi134.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-03-09T08:49:22.283 INFO:teuthology.orchestra.run.smithi067.stdout:(81/174): uuid-1.6.2-55.el9.x86_64.rpm 308 kB/s | 56 kB 00:00 2024-03-09T08:49:22.298 INFO:teuthology.orchestra.run.smithi134.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-03-09T08:49:22.341 INFO:teuthology.orchestra.run.smithi067.stdout:(82/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-03-09T08:49:22.348 INFO:teuthology.orchestra.run.smithi134.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-03-09T08:49:22.399 INFO:teuthology.orchestra.run.smithi134.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-03-09T08:49:22.400 INFO:teuthology.orchestra.run.smithi067.stdout:(83/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 765 kB/s | 44 kB 00:00 2024-03-09T08:49:22.492 INFO:teuthology.orchestra.run.smithi067.stdout:(84/174): fuse-2.9.9-15.el9.x86_64.rpm 864 kB/s | 79 kB 00:00 2024-03-09T08:49:22.549 INFO:teuthology.orchestra.run.smithi134.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 968 kB/s | 144 kB 00:00 2024-03-09T08:49:22.576 INFO:teuthology.orchestra.run.smithi067.stdout:(85/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2024-03-09T08:49:22.682 INFO:teuthology.orchestra.run.smithi134.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 81 kB/s | 11 kB 00:00 2024-03-09T08:49:22.710 INFO:teuthology.orchestra.run.smithi067.stdout:(86/174): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.1 MB 00:00 2024-03-09T08:49:22.776 INFO:teuthology.orchestra.run.smithi067.stdout:(87/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-09T08:49:22.801 INFO:teuthology.orchestra.run.smithi134.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-03-09T08:49:22.826 INFO:teuthology.orchestra.run.smithi134.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 123 kB/s | 18 kB 00:00 2024-03-09T08:49:22.843 INFO:teuthology.orchestra.run.smithi067.stdout:(88/174): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 124 kB 00:00 2024-03-09T08:49:22.919 INFO:teuthology.orchestra.run.smithi067.stdout:(89/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 187 kB 00:00 2024-03-09T08:49:22.927 INFO:teuthology.orchestra.run.smithi134.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 186 kB/s | 23 kB 00:00 2024-03-09T08:49:22.961 INFO:teuthology.orchestra.run.smithi134.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 146 kB/s | 19 kB 00:00 2024-03-09T08:49:23.045 INFO:teuthology.orchestra.run.smithi134.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-03-09T08:49:23.170 INFO:teuthology.orchestra.run.smithi134.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 373 kB/s | 46 kB 00:00 2024-03-09T08:49:23.245 INFO:teuthology.orchestra.run.smithi134.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 3.6 MB/s | 1.0 MB 00:00 2024-03-09T08:49:23.295 INFO:teuthology.orchestra.run.smithi134.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 632 kB/s | 79 kB 00:00 2024-03-09T08:49:23.354 INFO:teuthology.orchestra.run.smithi134.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 533 kB/s | 58 kB 00:00 2024-03-09T08:49:23.413 INFO:teuthology.orchestra.run.smithi134.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-03-09T08:49:23.455 INFO:teuthology.orchestra.run.smithi134.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-03-09T08:49:23.602 INFO:teuthology.orchestra.run.smithi067.stdout:(90/174): jq-1.6-16.el9.x86_64.rpm 274 kB/s | 187 kB 00:00 2024-03-09T08:49:23.813 INFO:teuthology.orchestra.run.smithi134.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 225 kB/s | 90 kB 00:00 2024-03-09T08:49:23.838 INFO:teuthology.orchestra.run.smithi134.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 74 kB/s | 28 kB 00:00 2024-03-09T08:49:23.931 INFO:teuthology.orchestra.run.smithi134.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 264 kB/s | 31 kB 00:00 2024-03-09T08:49:23.978 INFO:teuthology.orchestra.run.smithi067.stdout:(91/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-03-09T08:49:23.981 INFO:teuthology.orchestra.run.smithi134.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-03-09T08:49:24.073 INFO:teuthology.orchestra.run.smithi134.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 418 kB/s | 59 kB 00:00 2024-03-09T08:49:24.086 INFO:teuthology.orchestra.run.smithi067.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-03-09T08:49:24.098 INFO:teuthology.orchestra.run.smithi134.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 307 kB/s | 36 kB 00:00 2024-03-09T08:49:24.182 INFO:teuthology.orchestra.run.smithi134.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 473 kB/s | 51 kB 00:00 2024-03-09T08:49:24.195 INFO:teuthology.orchestra.run.smithi067.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-09T08:49:24.249 INFO:teuthology.orchestra.run.smithi134.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-03-09T08:49:24.299 INFO:teuthology.orchestra.run.smithi134.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 770 kB/s | 90 kB 00:00 2024-03-09T08:49:24.391 INFO:teuthology.orchestra.run.smithi134.stdout:(146/174): openssl-devel-3.0.7-27.el9.x86_64.rp 490 kB/s | 4.1 MB 00:08 2024-03-09T08:49:24.417 INFO:teuthology.orchestra.run.smithi134.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 172 kB/s | 20 kB 00:00 2024-03-09T08:49:24.767 INFO:teuthology.orchestra.run.smithi134.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 513 kB/s | 192 kB 00:00 2024-03-09T08:49:24.825 INFO:teuthology.orchestra.run.smithi134.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 742 kB/s | 427 kB 00:00 2024-03-09T08:49:24.884 INFO:teuthology.orchestra.run.smithi134.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-03-09T08:49:24.995 INFO:teuthology.orchestra.run.smithi067.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 27 kB/s | 21 kB 00:00 2024-03-09T08:49:25.026 INFO:teuthology.orchestra.run.smithi134.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.2 MB/s | 321 kB 00:00 2024-03-09T08:49:25.762 INFO:teuthology.orchestra.run.smithi067.stdout:(95/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 33 kB/s | 25 kB 00:00 2024-03-09T08:49:26.118 INFO:teuthology.orchestra.run.smithi134.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 2.1 MB/s | 2.3 MB 00:01 2024-03-09T08:49:26.201 INFO:teuthology.orchestra.run.smithi134.stdout:(153/174): librados2-19.0.0-1848.g8b896a9e.el9. 2.4 MB/s | 3.4 MB 00:01 2024-03-09T08:49:26.393 INFO:teuthology.orchestra.run.smithi134.stdout:(154/174): NetworkManager-libnm-1.46.0-1.el9.x8 6.6 MB/s | 1.8 MB 00:00 2024-03-09T08:49:26.677 INFO:teuthology.orchestra.run.smithi134.stdout:(155/174): NetworkManager-team-1.46.0-1.el9.x86 89 kB/s | 42 kB 00:00 2024-03-09T08:49:27.054 INFO:teuthology.orchestra.run.smithi067.stdout:(96/174): ceph-test-19.0.0-1848.g8b896a9e.el9.x 1.9 MB/s | 47 MB 00:24 2024-03-09T08:49:27.402 INFO:teuthology.orchestra.run.smithi134.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 98 kB/s | 71 kB 00:00 2024-03-09T08:49:27.544 INFO:teuthology.orchestra.run.smithi134.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 258 kB/s | 36 kB 00:00 2024-03-09T08:49:28.045 INFO:teuthology.orchestra.run.smithi134.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 385 kB/s | 192 kB 00:00 2024-03-09T08:49:28.070 INFO:teuthology.orchestra.run.smithi134.stdout:(159/174): NetworkManager-tui-1.46.0-1.el9.x86_ 148 kB/s | 248 kB 00:01 2024-03-09T08:49:28.178 INFO:teuthology.orchestra.run.smithi134.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.9 MB/s | 256 kB 00:00 2024-03-09T08:49:28.204 INFO:teuthology.orchestra.run.smithi134.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 173 kB/s | 23 kB 00:00 2024-03-09T08:49:28.488 INFO:teuthology.orchestra.run.smithi067.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 171 kB/s | 245 kB 00:01 2024-03-09T08:49:28.762 INFO:teuthology.orchestra.run.smithi134.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 278 kB/s | 155 kB 00:00 2024-03-09T08:49:29.271 INFO:teuthology.orchestra.run.smithi134.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 267 kB/s | 136 kB 00:00 2024-03-09T08:49:29.371 INFO:teuthology.orchestra.run.smithi067.stdout:(98/174): ceph-mgr-diskprediction-local-19.0.0- 425 kB/s | 7.4 MB 00:17 2024-03-09T08:49:29.388 INFO:teuthology.orchestra.run.smithi134.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 90 kB/s | 108 kB 00:01 2024-03-09T08:49:29.522 INFO:teuthology.orchestra.run.smithi134.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 216 kB/s | 29 kB 00:00 2024-03-09T08:49:29.547 INFO:teuthology.orchestra.run.smithi134.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 232 kB/s | 64 kB 00:00 2024-03-09T08:49:29.813 INFO:teuthology.orchestra.run.smithi067.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 187 kB/s | 248 kB 00:01 2024-03-09T08:49:30.681 INFO:teuthology.orchestra.run.smithi067.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 166 kB/s | 216 kB 00:01 2024-03-09T08:49:30.763 INFO:teuthology.orchestra.run.smithi067.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 37 kB/s | 35 kB 00:00 2024-03-09T08:49:31.780 INFO:teuthology.orchestra.run.smithi067.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 40 kB/s | 44 kB 00:01 2024-03-09T08:49:31.822 INFO:teuthology.orchestra.run.smithi067.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 89 kB/s | 94 kB 00:01 2024-03-09T08:49:31.890 INFO:teuthology.orchestra.run.smithi067.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 208 kB/s | 22 kB 00:00 2024-03-09T08:49:31.998 INFO:teuthology.orchestra.run.smithi067.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-03-09T08:49:32.164 INFO:teuthology.orchestra.run.smithi134.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 469 kB/s | 1.2 MB 00:02 2024-03-09T08:49:32.405 INFO:teuthology.orchestra.run.smithi134.stdout:(168/174): librbd1-19.0.0-1848.g8b896a9e.el9.x8 424 kB/s | 3.1 MB 00:07 2024-03-09T08:49:32.465 INFO:teuthology.orchestra.run.smithi067.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 954 kB/s | 548 kB 00:00 2024-03-09T08:49:32.499 INFO:teuthology.orchestra.run.smithi067.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 618 kB/s | 309 kB 00:00 2024-03-09T08:49:32.595 INFO:teuthology.orchestra.run.smithi067.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 150 kB/s | 19 kB 00:00 2024-03-09T08:49:33.054 INFO:teuthology.orchestra.run.smithi067.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 57 kB/s | 26 kB 00:00 2024-03-09T08:49:33.162 INFO:teuthology.orchestra.run.smithi067.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 452 kB/s | 49 kB 00:00 2024-03-09T08:49:33.280 INFO:teuthology.orchestra.run.smithi067.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-03-09T08:49:33.380 INFO:teuthology.orchestra.run.smithi067.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-03-09T08:49:33.847 INFO:teuthology.orchestra.run.smithi134.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 323 kB/s | 465 kB 00:01 2024-03-09T08:49:33.947 INFO:teuthology.orchestra.run.smithi067.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.4 MB/s | 838 kB 00:00 2024-03-09T08:49:33.981 INFO:teuthology.orchestra.run.smithi134.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 234 kB/s | 31 kB 00:00 2024-03-09T08:49:34.447 INFO:teuthology.orchestra.run.smithi067.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 1.1 MB/s | 548 kB 00:00 2024-03-09T08:49:34.656 INFO:teuthology.orchestra.run.smithi067.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 208 kB/s | 43 kB 00:00 2024-03-09T08:49:34.773 INFO:teuthology.orchestra.run.smithi067.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 277 kB/s | 32 kB 00:00 2024-03-09T08:49:34.898 INFO:teuthology.orchestra.run.smithi067.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-03-09T08:49:35.141 INFO:teuthology.orchestra.run.smithi067.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 710 kB/s | 172 kB 00:00 2024-03-09T08:49:35.549 INFO:teuthology.orchestra.run.smithi067.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 881 kB/s | 358 kB 00:00 2024-03-09T08:49:36.407 INFO:teuthology.orchestra.run.smithi067.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 353 kB/s | 303 kB 00:00 2024-03-09T08:49:36.641 INFO:teuthology.orchestra.run.smithi067.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 152 kB/s | 35 kB 00:00 2024-03-09T08:49:37.433 INFO:teuthology.orchestra.run.smithi067.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 256 kB/s | 203 kB 00:00 2024-03-09T08:49:39.559 INFO:teuthology.orchestra.run.smithi067.stdout:(123/174): openssl-devel-3.0.7-27.el9.x86_64.rp 307 kB/s | 4.1 MB 00:13 2024-03-09T08:49:39.681 INFO:teuthology.orchestra.run.smithi134.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 193 kB/s | 1.1 MB 00:05 2024-03-09T08:49:39.917 INFO:teuthology.orchestra.run.smithi067.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 405 kB/s | 144 kB 00:00 2024-03-09T08:49:40.017 INFO:teuthology.orchestra.run.smithi067.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-03-09T08:49:40.118 INFO:teuthology.orchestra.run.smithi067.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 178 kB/s | 18 kB 00:00 2024-03-09T08:49:40.185 INFO:teuthology.orchestra.run.smithi067.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 351 kB/s | 23 kB 00:00 2024-03-09T08:49:40.277 INFO:teuthology.orchestra.run.smithi067.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 214 kB/s | 19 kB 00:00 2024-03-09T08:49:40.360 INFO:teuthology.orchestra.run.smithi067.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 238 kB/s | 20 kB 00:00 2024-03-09T08:49:40.756 INFO:teuthology.orchestra.run.smithi134.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 69 kB/s | 74 kB 00:01 2024-03-09T08:49:41.402 INFO:teuthology.orchestra.run.smithi067.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 1.0 MB/s | 1.0 MB 00:01 2024-03-09T08:49:41.494 INFO:teuthology.orchestra.run.smithi067.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 509 kB/s | 46 kB 00:00 2024-03-09T08:49:41.645 INFO:teuthology.orchestra.run.smithi067.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 528 kB/s | 79 kB 00:00 2024-03-09T08:49:41.745 INFO:teuthology.orchestra.run.smithi067.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 581 kB/s | 58 kB 00:00 2024-03-09T08:49:41.956 INFO:teuthology.orchestra.run.smithi134.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 239 kB/s | 2.3 MB 00:09 2024-03-09T08:49:42.028 INFO:teuthology.orchestra.run.smithi067.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 963 kB/s | 272 kB 00:00 2024-03-09T08:49:42.104 INFO:teuthology.orchestra.run.smithi067.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 221 kB/s | 16 kB 00:00 2024-03-09T08:49:42.246 INFO:teuthology.orchestra.run.smithi067.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 638 kB/s | 90 kB 00:00 2024-03-09T08:49:42.338 INFO:teuthology.orchestra.run.smithi067.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 310 kB/s | 28 kB 00:00 2024-03-09T08:49:42.430 INFO:teuthology.orchestra.run.smithi067.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 338 kB/s | 31 kB 00:00 2024-03-09T08:49:42.722 INFO:teuthology.orchestra.run.smithi067.stdout:(139/174): python3-routes-2.5.1-5.el9.noarch.rp 648 kB/s | 188 kB 00:00 2024-03-09T08:49:42.897 INFO:teuthology.orchestra.run.smithi067.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 339 kB/s | 59 kB 00:00 2024-03-09T08:49:42.989 INFO:teuthology.orchestra.run.smithi067.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 394 kB/s | 36 kB 00:00 2024-03-09T08:49:43.172 INFO:teuthology.orchestra.run.smithi067.stdout:(142/174): python3-typing-extensions-4.4.0-2.el 279 kB/s | 51 kB 00:00 2024-03-09T08:49:43.539 INFO:teuthology.orchestra.run.smithi067.stdout:(143/174): python3-webob-1.8.7-6.el9.noarch.rpm 628 kB/s | 230 kB 00:00 2024-03-09T08:49:43.707 INFO:teuthology.orchestra.run.smithi067.stdout:(144/174): python3-grpcio-1.46.7-10.el9.x86_64. 333 kB/s | 2.0 MB 00:06 2024-03-09T08:49:44.564 INFO:teuthology.orchestra.run.smithi134.stdout:(174/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 147 kB/s | 2.2 MB 00:15 2024-03-09T08:49:44.573 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-03-09T08:49:44.573 INFO:teuthology.orchestra.run.smithi134.stdout:Total 3.5 MB/s | 224 MB 01:04 2024-03-09T08:49:44.790 INFO:teuthology.orchestra.run.smithi067.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 72 kB/s | 90 kB 00:01 2024-03-09T08:49:44.874 INFO:teuthology.orchestra.run.smithi067.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 241 kB/s | 20 kB 00:00 2024-03-09T08:49:45.216 INFO:teuthology.orchestra.run.smithi067.stdout:(147/174): re2-20211101-3.el9.x86_64.rpm 562 kB/s | 192 kB 00:00 2024-03-09T08:49:45.366 INFO:teuthology.orchestra.run.smithi067.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 257 kB/s | 427 kB 00:01 2024-03-09T08:49:45.900 INFO:teuthology.orchestra.run.smithi067.stdout:(149/174): librados2-19.0.0-1848.g8b896a9e.el9. 6.4 MB/s | 3.4 MB 00:00 2024-03-09T08:49:46.053 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-03-09T08:49:46.200 INFO:teuthology.orchestra.run.smithi067.stdout:(150/174): libarrow-9.0.0-10.el9.x86_64.rpm 330 kB/s | 4.4 MB 00:13 2024-03-09T08:49:46.300 INFO:teuthology.orchestra.run.smithi067.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.2 MB/s | 321 kB 00:00 2024-03-09T08:49:46.376 INFO:teuthology.orchestra.run.smithi067.stdout:(152/174): librbd1-19.0.0-1848.g8b896a9e.el9.x8 6.6 MB/s | 3.1 MB 00:00 2024-03-09T08:49:46.435 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-03-09T08:49:46.435 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-03-09T08:49:47.301 INFO:teuthology.orchestra.run.smithi067.stdout:(153/174): thrift-0.15.0-2.el9.x86_64.rpm 777 kB/s | 1.6 MB 00:02 2024-03-09T08:49:47.711 INFO:teuthology.orchestra.run.smithi067.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 104 kB/s | 42 kB 00:00 2024-03-09T08:49:48.127 INFO:teuthology.orchestra.run.smithi067.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 597 kB/s | 248 kB 00:00 2024-03-09T08:49:48.261 INFO:teuthology.orchestra.run.smithi067.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 537 kB/s | 71 kB 00:00 2024-03-09T08:49:48.387 INFO:teuthology.orchestra.run.smithi067.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 293 kB/s | 36 kB 00:00 2024-03-09T08:49:48.553 INFO:teuthology.orchestra.run.smithi067.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.1 MB/s | 192 kB 00:00 2024-03-09T08:49:48.696 INFO:teuthology.orchestra.run.smithi067.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.8 MB/s | 256 kB 00:00 2024-03-09T08:49:48.829 INFO:teuthology.orchestra.run.smithi067.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 174 kB/s | 23 kB 00:00 2024-03-09T08:49:48.830 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-03-09T08:49:48.830 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-03-09T08:49:48.963 INFO:teuthology.orchestra.run.smithi067.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 817 kB/s | 108 kB 00:00 2024-03-09T08:49:49.096 INFO:teuthology.orchestra.run.smithi067.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 155 kB 00:00 2024-03-09T08:49:49.247 INFO:teuthology.orchestra.run.smithi067.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 910 kB/s | 136 kB 00:00 2024-03-09T08:49:49.305 INFO:teuthology.orchestra.run.smithi067.stdout:(164/174): NetworkManager-1.46.0-1.el9.x86_64.r 781 kB/s | 2.3 MB 00:03 2024-03-09T08:49:49.364 INFO:teuthology.orchestra.run.smithi067.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 550 kB/s | 64 kB 00:00 2024-03-09T08:49:49.435 INFO:teuthology.orchestra.run.smithi067.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 229 kB/s | 29 kB 00:00 2024-03-09T08:49:49.814 INFO:teuthology.orchestra.run.smithi067.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 2.7 MB/s | 1.2 MB 00:00 2024-03-09T08:49:50.090 INFO:teuthology.orchestra.run.smithi067.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.3 MB/s | 2.3 MB 00:00 2024-03-09T08:49:50.223 INFO:teuthology.orchestra.run.smithi067.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.4 MB/s | 465 kB 00:00 2024-03-09T08:49:50.332 INFO:teuthology.orchestra.run.smithi067.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 289 kB/s | 31 kB 00:00 2024-03-09T08:49:50.482 INFO:teuthology.orchestra.run.smithi067.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.2 MB/s | 1.1 MB 00:00 2024-03-09T08:49:50.858 INFO:teuthology.orchestra.run.smithi067.stdout:(172/174): NetworkManager-libnm-1.46.0-1.el9.x8 415 kB/s | 1.8 MB 00:04 2024-03-09T08:49:51.099 INFO:teuthology.orchestra.run.smithi067.stdout:(173/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 1.3 MB/s | 2.2 MB 00:01 2024-03-09T08:49:51.345 INFO:teuthology.orchestra.run.smithi067.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 86 kB/s | 74 kB 00:00 2024-03-09T08:49:51.351 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-03-09T08:49:51.351 INFO:teuthology.orchestra.run.smithi067.stdout:Total 3.1 MB/s | 224 MB 01:12 2024-03-09T08:49:52.836 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-03-09T08:49:53.234 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-03-09T08:49:53.234 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-03-09T08:49:55.758 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-03-09T08:49:55.759 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-03-09T08:49:55.997 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-03-09T08:49:56.110 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-09T08:49:56.158 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-09T08:49:56.204 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-09T08:49:56.248 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-09T08:49:56.335 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-09T08:49:56.430 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-09T08:49:56.487 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-09T08:49:56.562 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-09T08:49:56.667 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-09T08:49:57.109 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-09T08:49:57.151 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-09T08:49:57.300 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-09T08:49:57.327 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-09T08:49:57.446 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-09T08:49:57.486 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-09T08:49:57.547 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-09T08:49:57.831 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-09T08:49:57.881 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-09T08:49:57.898 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-09T08:49:58.050 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-09T08:49:58.066 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-09T08:49:58.155 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-09T08:49:58.200 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-09T08:49:58.218 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-09T08:49:58.280 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-09T08:49:58.567 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-09T08:49:58.609 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-09T08:49:58.651 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-09T08:49:58.975 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-09T08:49:59.192 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-09T08:49:59.281 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-09T08:49:59.487 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-09T08:49:59.553 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-09T08:49:59.893 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-09T08:50:00.071 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-09T08:50:00.191 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-09T08:50:00.234 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-09T08:50:00.271 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-09T08:50:00.330 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-09T08:50:00.440 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-09T08:50:00.509 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-09T08:50:00.550 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-09T08:50:00.640 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-09T08:50:00.680 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-09T08:50:00.741 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-09T08:50:00.785 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-09T08:50:00.817 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-09T08:50:00.928 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-09T08:50:00.982 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 39/198 2024-03-09T08:50:01.043 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 40/198 2024-03-09T08:50:01.078 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 41/198 2024-03-09T08:50:01.197 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-09T08:50:01.256 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-09T08:50:01.352 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-09T08:50:01.524 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-09T08:50:01.583 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-09T08:50:02.261 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-09T08:50:02.370 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-09T08:50:02.478 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-09T08:50:02.540 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-09T08:50:02.563 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-09T08:50:02.609 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-09T08:50:02.681 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-09T08:50:02.732 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 53/198 2024-03-09T08:50:02.772 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-09T08:50:03.153 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-03-09T08:50:03.174 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-09T08:50:03.266 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-09T08:50:03.332 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-09T08:50:03.377 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-09T08:50:03.379 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-09T08:50:03.422 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-09T08:50:03.489 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-09T08:50:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-09T08:50:03.569 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-09T08:50:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-09T08:50:03.680 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-09T08:50:03.718 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-09T08:50:03.766 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-09T08:50:03.810 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-09T08:50:03.823 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-09T08:50:03.902 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-09T08:50:03.994 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-09T08:50:04.031 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-09T08:50:04.070 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-09T08:50:04.138 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-09T08:50:04.178 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-09T08:50:04.239 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-09T08:50:04.241 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-09T08:50:04.274 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-09T08:50:04.308 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-09T08:50:04.424 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-09T08:50:04.458 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-09T08:50:04.506 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-09T08:50:04.565 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-09T08:50:04.567 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-09T08:50:04.605 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-09T08:50:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-09T08:50:04.671 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-09T08:50:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-09T08:50:04.801 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-09T08:50:04.837 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-09T08:50:04.889 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-09T08:50:04.920 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-09T08:50:04.952 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-09T08:50:05.011 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-09T08:50:05.063 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-09T08:50:05.088 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-09T08:50:05.092 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-09T08:50:05.251 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-09T08:50:05.274 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-09T08:50:05.281 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-09T08:50:05.380 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-09T08:50:05.410 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-09T08:50:05.451 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-09T08:50:05.462 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-09T08:50:05.475 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-09T08:50:05.525 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-09T08:50:05.545 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-09T08:50:05.597 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-09T08:50:05.844 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-09T08:50:05.864 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-09T08:50:05.898 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-09T08:50:05.909 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-09T08:50:05.958 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-09T08:50:06.062 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-09T08:50:06.111 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-09T08:50:06.151 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-09T08:50:06.181 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-09T08:50:06.230 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-09T08:50:06.277 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-09T08:50:06.296 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-09T08:50:06.336 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-09T08:50:06.389 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-09T08:50:06.435 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-09T08:50:06.465 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-09T08:50:06.555 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-09T08:50:06.560 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-09T08:50:06.606 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-09T08:50:06.640 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-09T08:50:06.694 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-09T08:50:06.736 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-09T08:50:06.751 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-09T08:50:06.866 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-09T08:50:06.877 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-09T08:50:07.022 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-09T08:50:07.073 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-09T08:50:07.109 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-09T08:50:07.166 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-09T08:50:07.207 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-09T08:50:07.303 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-09T08:50:07.368 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-09T08:50:07.403 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-09T08:50:07.540 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-09T08:50:07.585 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-09T08:50:07.623 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-09T08:50:07.688 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-09T08:50:07.811 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-09T08:50:07.879 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-09T08:50:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-09T08:50:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-09T08:50:08.074 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-09T08:50:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-09T08:50:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-09T08:50:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-09T08:50:08.301 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-09T08:50:08.358 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-09T08:50:08.391 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-09T08:50:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-09T08:50:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 39/198 2024-03-09T08:50:08.447 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-09T08:50:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 40/198 2024-03-09T08:50:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 41/198 2024-03-09T08:50:08.568 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-09T08:50:08.609 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-09T08:50:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-09T08:50:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-09T08:50:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-09T08:50:08.839 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-09T08:50:08.872 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-09T08:50:08.963 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-09T08:50:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-09T08:50:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-09T08:50:09.747 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-09T08:50:09.771 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-09T08:50:09.864 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-09T08:50:09.981 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-09T08:50:10.041 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-09T08:50:10.067 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-09T08:50:10.122 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-09T08:50:10.196 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-09T08:50:10.255 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 53/198 2024-03-09T08:50:10.260 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-09T08:50:10.299 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-09T08:50:10.772 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-09T08:50:11.257 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-09T08:50:11.333 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-09T08:50:11.405 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-09T08:50:11.625 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-09T08:50:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-09T08:50:11.773 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-09T08:50:11.801 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-09T08:50:11.900 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-09T08:50:11.910 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-09T08:50:11.942 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-09T08:50:11.948 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-09T08:50:11.978 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-09T08:50:11.982 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-09T08:50:12.020 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-09T08:50:12.052 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-09T08:50:12.106 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-09T08:50:12.165 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-09T08:50:12.228 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-09T08:50:12.397 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-09T08:50:12.424 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-09T08:50:12.436 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-09T08:50:12.496 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-09T08:50:12.543 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-09T08:50:12.559 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-09T08:50:12.572 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 125/198 2024-03-09T08:50:12.621 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-09T08:50:12.626 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-09T08:50:12.768 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-09T08:50:12.864 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-09T08:50:12.912 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-09T08:50:12.947 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-09T08:50:12.970 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-09T08:50:13.159 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-09T08:50:13.163 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-09T08:50:13.193 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 128/198 2024-03-09T08:50:13.284 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 128/198 2024-03-09T08:50:13.320 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 129/198 2024-03-09T08:50:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-09T08:50:13.501 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-09T08:50:13.547 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-09T08:50:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-09T08:50:13.675 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-09T08:50:13.941 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-09T08:50:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-09T08:50:14.182 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-09T08:50:14.231 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-09T08:50:14.270 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-09T08:50:14.309 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-09T08:50:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-09T08:50:14.395 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-09T08:50:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-09T08:50:14.513 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-09T08:50:14.563 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-09T08:50:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-09T08:50:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-09T08:50:14.759 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-09T08:50:14.822 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-09T08:50:14.863 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-09T08:50:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-09T08:50:15.134 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-09T08:50:15.185 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-09T08:50:15.220 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-09T08:50:15.279 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-09T08:50:15.410 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-09T08:50:15.478 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-09T08:50:15.844 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 130/198 2024-03-09T08:50:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 130/198 2024-03-09T08:50:16.487 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-09T08:50:16.507 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 130/198 2024-03-09T08:50:16.543 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-09T08:50:16.553 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 131/198 2024-03-09T08:50:16.569 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-09T08:50:16.625 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-09T08:50:16.716 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 131/198 2024-03-09T08:50:16.716 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-09T08:50:16.716 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-09T08:50:16.716 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:16.739 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-09T08:50:16.771 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 132/198 2024-03-09T08:50:16.785 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-09T08:50:17.030 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-09T08:50:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-09T08:50:17.162 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-09T08:50:18.008 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-09T08:50:18.151 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-09T08:50:19.778 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-09T08:50:19.870 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-09T08:50:19.908 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-09T08:50:19.938 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-09T08:50:19.978 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-09T08:50:20.372 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-09T08:50:20.409 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-09T08:50:20.532 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-09T08:50:20.566 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 125/198 2024-03-09T08:50:20.622 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-09T08:50:21.179 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-09T08:50:21.221 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 128/198 2024-03-09T08:50:21.330 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 128/198 2024-03-09T08:50:21.356 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 129/198 2024-03-09T08:50:24.093 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 130/198 2024-03-09T08:50:24.136 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 130/198 2024-03-09T08:50:24.811 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 130/198 2024-03-09T08:50:24.848 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 131/198 2024-03-09T08:50:24.985 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 131/198 2024-03-09T08:50:24.985 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-09T08:50:24.985 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-09T08:50:24.985 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:50:25.023 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 132/198 2024-03-09T08:50:29.542 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 132/198 2024-03-09T08:50:29.543 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-03-09T08:50:29.543 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-03-09T08:50:29.543 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-03-09T08:50:29.543 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-03-09T08:50:29.543 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-03-09T08:50:29.543 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-03-09T08:50:29.543 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-03-09T08:50:29.543 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:29.758 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 133/198 2024-03-09T08:50:29.802 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 133/198 2024-03-09T08:50:29.803 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:29.803 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-09T08:50:29.803 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:31.386 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 134/198 2024-03-09T08:50:31.474 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 134/198 2024-03-09T08:50:31.474 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:31.474 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-09T08:50:31.475 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-09T08:50:31.475 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-09T08:50:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:31.712 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 135/198 2024-03-09T08:50:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 135/198 2024-03-09T08:50:31.780 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:31.780 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-09T08:50:31.780 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-09T08:50:31.780 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-09T08:50:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:32.261 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 136/198 2024-03-09T08:50:32.329 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 136/198 2024-03-09T08:50:32.330 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:32.330 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-09T08:50:32.330 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-09T08:50:32.330 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-09T08:50:32.330 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:32.438 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-09T08:50:33.068 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-09T08:50:33.097 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-09T08:50:33.146 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 139/198 2024-03-09T08:50:33.238 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 140/198 2024-03-09T08:50:33.276 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-09T08:50:33.587 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-09T08:50:33.619 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-09T08:50:34.731 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-09T08:50:34.761 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-09T08:50:34.901 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-09T08:50:35.073 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 144/198 2024-03-09T08:50:35.104 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-09T08:50:35.173 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-09T08:50:35.173 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:35.174 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-09T08:50:35.174 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-09T08:50:35.174 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-09T08:50:35.174 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:35.238 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-09T08:50:35.274 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-09T08:50:35.330 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 147/198 2024-03-09T08:50:36.262 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-09T08:50:36.290 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 149/198 2024-03-09T08:50:36.359 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 149/198 2024-03-09T08:50:36.359 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:36.360 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-09T08:50:36.360 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-09T08:50:36.360 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-09T08:50:36.360 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:36.416 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 150/198 2024-03-09T08:50:36.485 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1848.g8b896 150/198 2024-03-09T08:50:36.485 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:36.485 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-09T08:50:36.485 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:36.567 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-09T08:50:36.642 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-09T08:50:36.643 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:36.643 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-09T08:50:36.643 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-09T08:50:36.643 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-09T08:50:36.643 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:37.017 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 152/198 2024-03-09T08:50:37.098 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 152/198 2024-03-09T08:50:37.098 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:37.098 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-09T08:50:37.098 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-09T08:50:37.098 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-09T08:50:37.098 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:38.146 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 132/198 2024-03-09T08:50:38.146 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2024-03-09T08:50:38.146 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2024-03-09T08:50:38.146 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2024-03-09T08:50:38.146 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2024-03-09T08:50:38.146 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2024-03-09T08:50:38.147 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /root 2024-03-09T08:50:38.147 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2024-03-09T08:50:38.147 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:50:38.371 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 133/198 2024-03-09T08:50:38.422 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 133/198 2024-03-09T08:50:38.422 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:38.422 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-09T08:50:38.422 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:50:40.097 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 134/198 2024-03-09T08:50:40.193 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 134/198 2024-03-09T08:50:40.193 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:40.194 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-09T08:50:40.194 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-09T08:50:40.194 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-09T08:50:40.194 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:50:40.448 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 135/198 2024-03-09T08:50:40.525 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 135/198 2024-03-09T08:50:40.525 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:40.525 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-09T08:50:40.525 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-09T08:50:40.525 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-09T08:50:40.525 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:50:41.032 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 136/198 2024-03-09T08:50:41.114 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 136/198 2024-03-09T08:50:41.114 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:41.114 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-09T08:50:41.115 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-09T08:50:41.115 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-09T08:50:41.115 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:50:41.233 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-09T08:50:41.819 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-09T08:50:41.824 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-09T08:50:41.859 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-09T08:50:41.870 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-09T08:50:41.906 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 155/198 2024-03-09T08:50:41.910 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 139/198 2024-03-09T08:50:41.943 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-09T08:50:42.014 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 140/198 2024-03-09T08:50:42.046 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-09T08:50:42.208 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 157/198 2024-03-09T08:50:42.333 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-09T08:50:42.373 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 159/198 2024-03-09T08:50:42.387 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-09T08:50:42.423 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-09T08:50:42.425 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-09T08:50:42.489 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-09T08:50:42.535 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-09T08:50:42.570 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 163/198 2024-03-09T08:50:42.624 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-09T08:50:42.675 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-09T08:50:42.761 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-09T08:50:42.864 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-09T08:50:42.931 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-09T08:50:43.611 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-09T08:50:43.640 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-09T08:50:43.788 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-09T08:50:43.817 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-09T08:50:43.862 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-09T08:50:43.924 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-09T08:50:43.970 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 144/198 2024-03-09T08:50:44.000 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-09T08:50:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-09T08:50:44.079 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:44.079 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-09T08:50:44.079 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-09T08:50:44.079 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-09T08:50:44.079 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:50:44.134 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-09T08:50:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-09T08:50:44.287 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 147/198 2024-03-09T08:50:45.232 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 170/198 2024-03-09T08:50:45.306 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-09T08:50:45.316 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-09T08:50:45.336 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 149/198 2024-03-09T08:50:45.364 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-09T08:50:45.419 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 149/198 2024-03-09T08:50:45.419 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:45.419 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-09T08:50:45.419 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-09T08:50:45.419 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-09T08:50:45.420 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:50:45.470 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 150/198 2024-03-09T08:50:45.490 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-09T08:50:45.527 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-09T08:50:45.560 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-09T08:50:45.567 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1848.g8b896 150/198 2024-03-09T08:50:45.567 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:45.567 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-09T08:50:45.567 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:50:45.594 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-09T08:50:45.643 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-09T08:50:45.643 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-09T08:50:45.654 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-09T08:50:45.762 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-09T08:50:45.762 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:45.762 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-09T08:50:45.763 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-09T08:50:45.763 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-09T08:50:45.763 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:50:45.789 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-09T08:50:45.832 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-09T08:50:45.832 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-09T08:50:45.970 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-09T08:50:46.014 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-09T08:50:46.060 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-09T08:50:46.093 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-09T08:50:46.126 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-09T08:50:46.154 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 152/198 2024-03-09T08:50:46.200 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-09T08:50:46.238 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 152/198 2024-03-09T08:50:46.238 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-09T08:50:46.238 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-09T08:50:46.239 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-09T08:50:46.239 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-09T08:50:46.239 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:50:46.273 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-09T08:50:46.313 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-09T08:50:46.372 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-09T08:50:46.414 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-09T08:50:46.447 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-09T08:50:46.481 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-09T08:50:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-09T08:50:46.517 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-09T08:50:46.839 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-09T08:50:46.840 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-09T08:50:46.978 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-09T08:50:47.009 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-09T08:50:47.051 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-09T08:50:47.109 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-09T08:50:47.152 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-09T08:50:47.270 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-09T08:50:47.532 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-09T08:50:47.532 INFO:teuthology.orchestra.run.smithi134.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-09T08:50:47.533 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-09T08:50:51.371 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-09T08:50:51.408 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 155/198 2024-03-09T08:50:51.443 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-09T08:50:51.710 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 157/198 2024-03-09T08:50:51.851 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-09T08:50:51.903 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 159/198 2024-03-09T08:50:51.993 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-09T08:50:52.070 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-09T08:50:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-09T08:50:52.168 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 163/198 2024-03-09T08:50:52.219 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-09T08:50:52.270 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-09T08:50:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-09T08:50:52.470 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-09T08:50:52.510 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-09T08:50:52.613 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 1/198 2024-03-09T08:50:52.613 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2/198 2024-03-09T08:50:52.613 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 3/198 2024-03-09T08:50:52.613 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 4/198 2024-03-09T08:50:52.613 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 5/198 2024-03-09T08:50:52.613 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 6/198 2024-03-09T08:50:52.613 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 7/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 8/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 9/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 13/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 14/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 15/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 17/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 18/198 2024-03-09T08:50:52.614 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 19/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 20/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 21/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 22/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 23/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 24/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 25/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 26/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 27/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 28/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 29/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 30/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 31/198 2024-03-09T08:50:52.615 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 32/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 33/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 34/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 35/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 36/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 37/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 38/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 39/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-09T08:50:52.616 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-09T08:50:52.617 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-09T08:50:52.618 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-09T08:50:52.619 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-09T08:50:52.620 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-09T08:50:52.621 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-09T08:50:52.622 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-09T08:50:52.622 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-09T08:50:52.622 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-09T08:50:52.622 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-09T08:50:52.622 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-09T08:50:52.622 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-09T08:50:52.622 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-09T08:50:52.622 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-09T08:50:52.622 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-09T08:50:52.622 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-09T08:50:52.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-09T08:50:52.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-09T08:50:52.625 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-09T08:50:52.626 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-09T08:50:52.627 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-09T08:50:52.627 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-09T08:50:52.627 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-09T08:50:52.627 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-09T08:50:52.627 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-09T08:50:52.627 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-09T08:50:52.627 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-09T08:50:52.628 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-09T08:50:52.629 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-09T08:50:52.629 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-09T08:50:53.436 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-09T08:50:53.484 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-09T08:50:53.538 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-09T08:50:54.990 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 170/198 2024-03-09T08:50:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-09T08:50:55.123 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-09T08:50:55.123 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:55.123 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: acl-2.3.1-4.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-09T08:50:55.124 INFO:teuthology.orchestra.run.smithi134.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-03-09T08:50:55.125 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.126 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:50:55.127 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.128 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.128 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:50:55.128 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-09T08:50:55.128 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-03-09T08:50:55.128 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-09T08:50:55.128 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-09T08:50:55.128 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-09T08:50:55.128 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-09T08:50:55.128 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-09T08:50:55.128 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-09T08:50:55.128 INFO:teuthology.orchestra.run.smithi134.stdout: jq-1.6-16.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.129 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-09T08:50:55.130 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.11-1.el9.x86_64 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-09T08:50:55.131 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-09T08:50:55.132 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-09T08:50:55.133 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-09T08:50:55.134 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-09T08:50:55.135 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-09T08:50:55.135 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-09T08:50:55.135 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-09T08:50:55.135 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-09T08:50:55.135 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.135 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-09T08:50:55.138 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-3.el9.x86_64 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-09T08:50:55.139 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-09T08:50:55.140 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-09T08:50:55.140 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-09T08:50:55.140 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-09T08:50:55.140 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:50:55.140 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-03-09T08:50:55.158 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-09T08:50:55.286 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-09T08:50:55.324 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-09T08:50:55.359 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-09T08:50:55.392 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-09T08:50:55.433 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-09T08:50:55.433 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-09T08:50:55.572 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-09T08:50:55.621 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-09T08:50:55.622 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-09T08:50:55.759 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-09T08:50:55.780 DEBUG:teuthology.parallel:result is None 2024-03-09T08:50:55.806 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-09T08:50:55.849 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-09T08:50:55.882 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-09T08:50:55.915 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-09T08:50:55.986 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-09T08:50:56.064 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-09T08:50:56.111 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-09T08:50:56.153 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-09T08:50:56.211 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-09T08:50:56.245 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-09T08:50:56.278 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-09T08:50:56.316 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-09T08:50:56.316 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-09T08:50:56.645 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-09T08:50:56.645 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-09T08:50:56.768 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-09T08:50:56.798 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-09T08:50:56.832 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-09T08:50:56.914 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-09T08:50:56.957 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-09T08:50:57.082 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-09T08:50:57.360 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-09T08:50:57.361 INFO:teuthology.orchestra.run.smithi067.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-09T08:50:57.361 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:01.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 1/198 2024-03-09T08:51:01.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2/198 2024-03-09T08:51:01.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 3/198 2024-03-09T08:51:01.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 4/198 2024-03-09T08:51:01.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 5/198 2024-03-09T08:51:01.882 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 6/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 7/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 8/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 9/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 13/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 14/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 15/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 17/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 18/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 19/198 2024-03-09T08:51:01.883 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 20/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 21/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 22/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 23/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 24/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 25/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 26/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 27/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 28/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 29/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 30/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 31/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 32/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 33/198 2024-03-09T08:51:01.884 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 34/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 35/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 36/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 37/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 38/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 39/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-09T08:51:01.885 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-09T08:51:01.886 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-09T08:51:01.887 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-09T08:51:01.888 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-09T08:51:01.889 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-09T08:51:01.890 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-09T08:51:01.891 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-09T08:51:01.892 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-09T08:51:01.893 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-09T08:51:01.894 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-09T08:51:01.895 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-09T08:51:01.896 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-09T08:51:04.381 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-09T08:51:04.381 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:04.382 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2024-03-09T08:51:04.382 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-09T08:51:04.382 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-09T08:51:04.382 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-09T08:51:04.382 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-09T08:51:04.382 INFO:teuthology.orchestra.run.smithi067.stdout: acl-2.3.1-4.el9.x86_64 2024-03-09T08:51:04.382 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-09T08:51:04.382 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-09T08:51:04.382 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-09T08:51:04.382 INFO:teuthology.orchestra.run.smithi067.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-09T08:51:04.382 INFO:teuthology.orchestra.run.smithi067.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-09T08:51:04.383 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout: automake-1.16.2-8.el9.noarch 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout: bison-3.7.4-5.el9.x86_64 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.384 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:51:04.385 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: flex-2.6.4-9.el9.x86_64 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: jq-1.6-16.el9.x86_64 2024-03-09T08:51:04.386 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.387 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: lua-5.4.4-4.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-09T08:51:04.388 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-13.11-1.el9.x86_64 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.389 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-09T08:51:04.390 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-09T08:51:04.391 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-09T08:51:04.391 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-09T08:51:04.391 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-09T08:51:04.391 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-09T08:51:04.391 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-09T08:51:04.391 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-09T08:51:04.391 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-09T08:51:04.391 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-09T08:51:04.391 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-09T08:51:04.391 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply-3.11-14.el9.noarch 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-09T08:51:04.392 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-09T08:51:04.393 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-09T08:51:04.394 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-09T08:51:04.395 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.395 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.395 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-09T08:51:04.395 INFO:teuthology.orchestra.run.smithi067.stdout: re2-1:20211101-3.el9.x86_64 2024-03-09T08:51:04.395 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-09T08:51:04.395 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-09T08:51:04.395 INFO:teuthology.orchestra.run.smithi067.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-09T08:51:04.395 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-09T08:51:04.395 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-09T08:51:04.395 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-09T08:51:04.396 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:04.396 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-03-09T08:51:05.075 DEBUG:teuthology.parallel:result is None 2024-03-09T08:51:05.076 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-09T08:51:05.219 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-09T08:51:05.289 INFO:teuthology.orchestra.run.smithi067.stdout:19.0.0-1848.g8b896a9e.el9 2024-03-09T08:51:05.289 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1848.g8b896a9e.el9 2024-03-09T08:51:05.289 INFO:teuthology.task.install:The correct ceph version 19.0.0-1848.g8b896a9e is installed. 2024-03-09T08:51:05.291 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-09T08:51:05.546 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-09T08:51:05.618 INFO:teuthology.orchestra.run.smithi134.stdout:19.0.0-1848.g8b896a9e.el9 2024-03-09T08:51:05.618 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1848.g8b896a9e.el9 2024-03-09T08:51:05.618 INFO:teuthology.task.install:The correct ceph version 19.0.0-1848.g8b896a9e is installed. 2024-03-09T08:51:05.620 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-09T08:51:05.620 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:05.620 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-09T08:51:05.702 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:05.703 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-09T08:51:05.804 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-09T08:51:05.805 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:05.805 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2024-03-09T08:51:05.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-09T08:51:05.997 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:05.997 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-03-09T08:51:06.098 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-09T08:51:06.193 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-09T08:51:06.193 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:06.193 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-09T08:51:06.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-09T08:51:06.380 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:06.381 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-09T08:51:06.479 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-09T08:51:06.618 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-09T08:51:06.618 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:06.618 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2024-03-09T08:51:06.697 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-09T08:51:06.812 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:06.812 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-03-09T08:51:06.912 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-09T08:51:07.007 INFO:teuthology.run_tasks:Running task ceph... 2024-03-09T08:51:07.090 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-09T08:51:07.090 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /var/log/ceph 2024-03-09T08:51:07.095 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /var/log/ceph 2024-03-09T08:51:07.190 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-09T08:51:07.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-09T08:51:07.194 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-09T08:51:07.286 INFO:tasks.ceph:Creating extra log directories... 2024-03-09T08:51:07.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-09T08:51:07.290 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-09T08:51:07.439 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-09T08:51:07.439 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-09T08:51:07.439 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-09T08:51:07.440 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:07.440 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-09T08:51:07.478 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-09T08:51:07.697 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-09T08:51:07.776 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:07.776 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-09T08:51:07.831 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-09T08:51:08.099 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-09T08:51:08.199 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-09T08:51:08.199 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-09T08:51:08.200 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387', 'branch': 'wip-lusov-qdb', 'description': 'fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}}', 'email': None, 'first_in_suite': False, 'job_id': '7588387', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-qdb'}, 'ceph': {'cephfs': {'fs': [{'name': 'a'}, {'name': 'b'}], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'ceph-fuse': {'client.0': {'cephfs_name': 'a'}, 'client.1': {'cephfs_name': 'b'}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}}, 'kclient': {'client.0': {'cephfs_name': 'a'}, 'client.1': {'cephfs_name': 'b'}, 'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-qdb', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 9221, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'sleep_before_teardown': 0, 'subset': '111/10000', 'suite': 'fs', 'suite_branch': 'wip-lusov-qdb', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi067.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAb/kX4Hsc0tiJSY05cLPFVNhJMK+e4k9JbctP5BGPUVT79kEHsaEBetdjSRmsLxj40HKNERLtYzcfUyQqlqn10=', 'smithi134.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCLbjgt5vP+vOklGDWn6iDhKnEH0HXyfVYCMuZrpZANGqm64uoLhdXIJPj/Q1yBtpcI7cHfNELey3q2WmQCGrMA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'fs': [{'name': 'a'}, {'name': 'b'}], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'cluster': 'ceph'}}, {'kclient': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['fs/snaps']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-09_08:15:07', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-09T08:51:08.200 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-09T08:51:08.206 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-09T08:51:08.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/run/ceph 2024-03-09T08:51:08.294 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/run/ceph 2024-03-09T08:51:08.391 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:08.391 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-03-09T08:51:08.439 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-09T08:51:08.439 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2024-03-09T08:51:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-09T08:51:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-09T08:51:08.522 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 638 Links: 1 2024-03-09T08:51:08.522 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-09T08:51:08.522 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-03-09T08:51:08.522 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-03-09 08:50:57.857405994 +0000 2024-03-09T08:51:08.522 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-03-09 08:47:23.039757398 +0000 2024-03-09T08:51:08.522 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-03-09 08:47:23.039757398 +0000 2024-03-09T08:51:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-03-09 08:47:23.039757398 +0000 2024-03-09T08:51:08.523 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-09T08:51:08.631 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-03-09T08:51:08.631 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-03-09T08:51:08.631 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000357766 s, 1.4 MB/s 2024-03-09T08:51:08.634 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-09T08:51:08.676 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2024-03-09T08:51:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-09T08:51:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-09T08:51:08.753 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 657 Links: 1 2024-03-09T08:51:08.753 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-09T08:51:08.753 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-03-09T08:51:08.753 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-03-09 08:50:57.858405973 +0000 2024-03-09T08:51:08.753 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-03-09 08:47:23.625744882 +0000 2024-03-09T08:51:08.753 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-03-09 08:47:23.625744882 +0000 2024-03-09T08:51:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-03-09 08:47:23.625744882 +0000 2024-03-09T08:51:08.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-09T08:51:08.858 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-03-09T08:51:08.858 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-03-09T08:51:08.858 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000277922 s, 1.8 MB/s 2024-03-09T08:51:08.861 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-09T08:51:08.901 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2024-03-09T08:51:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-09T08:51:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-09T08:51:08.982 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 676 Links: 1 2024-03-09T08:51:08.982 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-09T08:51:08.983 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-03-09T08:51:08.983 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-03-09 08:50:57.859405951 +0000 2024-03-09T08:51:08.983 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-03-09 08:47:24.210732387 +0000 2024-03-09T08:51:08.983 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-03-09 08:47:24.210732387 +0000 2024-03-09T08:51:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-03-09 08:47:24.210732387 +0000 2024-03-09T08:51:08.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-09T08:51:09.098 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-03-09T08:51:09.099 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-03-09T08:51:09.099 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000286856 s, 1.8 MB/s 2024-03-09T08:51:09.102 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-09T08:51:09.140 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2024-03-09T08:51:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-09T08:51:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-09T08:51:09.218 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 696 Links: 1 2024-03-09T08:51:09.218 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-09T08:51:09.218 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-03-09T08:51:09.219 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-03-09 08:50:57.860405929 +0000 2024-03-09T08:51:09.219 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-03-09 08:47:24.796719871 +0000 2024-03-09T08:51:09.219 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-03-09 08:47:24.796719871 +0000 2024-03-09T08:51:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-03-09 08:47:24.796719871 +0000 2024-03-09T08:51:09.219 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-09T08:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-03-09T08:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-03-09T08:51:09.326 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000259226 s, 2.0 MB/s 2024-03-09T08:51:09.329 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-09T08:51:09.368 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-09T08:51:09.368 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:09.368 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-03-09T08:51:09.423 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-09T08:51:09.423 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-03-09T08:51:09.473 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-09T08:51:09.474 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-09T08:51:09.474 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-09T08:51:09.474 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-09T08:51:09.474 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-03-09T08:51:09.474 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-03-09 08:50:47.977336594 +0000 2024-03-09T08:51:09.474 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-03-09 08:47:12.286253186 +0000 2024-03-09T08:51:09.474 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-03-09 08:47:12.286253186 +0000 2024-03-09T08:51:09.474 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-03-09 08:47:12.286253186 +0000 2024-03-09T08:51:09.474 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-09T08:51:09.561 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-03-09T08:51:09.561 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-03-09T08:51:09.562 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.00035396 s, 1.4 MB/s 2024-03-09T08:51:09.566 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-09T08:51:09.622 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-03-09T08:51:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-09T08:51:09.671 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-09T08:51:09.671 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-09T08:51:09.671 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-09T08:51:09.671 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-03-09T08:51:09.672 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-03-09 08:50:47.978336570 +0000 2024-03-09T08:51:09.672 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-03-09 08:47:12.868240554 +0000 2024-03-09T08:51:09.672 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-03-09 08:47:12.868240554 +0000 2024-03-09T08:51:09.672 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-03-09 08:47:12.868240554 +0000 2024-03-09T08:51:09.672 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-09T08:51:09.758 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-03-09T08:51:09.758 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-03-09T08:51:09.759 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000496651 s, 1.0 MB/s 2024-03-09T08:51:09.763 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-09T08:51:09.816 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-03-09T08:51:09.866 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-09T08:51:09.866 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-09T08:51:09.866 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-09T08:51:09.866 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-09T08:51:09.866 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-03-09T08:51:09.866 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-03-09 08:50:47.979336546 +0000 2024-03-09T08:51:09.866 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-03-09 08:47:13.410228790 +0000 2024-03-09T08:51:09.866 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-03-09 08:47:13.410228790 +0000 2024-03-09T08:51:09.867 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-03-09 08:47:13.410228790 +0000 2024-03-09T08:51:09.867 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-09T08:51:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-03-09T08:51:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-03-09T08:51:09.952 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000302799 s, 1.7 MB/s 2024-03-09T08:51:09.956 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-09T08:51:10.012 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-03-09T08:51:10.061 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-09T08:51:10.061 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-09T08:51:10.061 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-09T08:51:10.061 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-09T08:51:10.061 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-03-09T08:51:10.061 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-03-09 08:50:47.980336523 +0000 2024-03-09T08:51:10.061 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-03-09 08:47:13.966216721 +0000 2024-03-09T08:51:10.061 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-03-09 08:47:13.966216721 +0000 2024-03-09T08:51:10.061 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-03-09 08:47:13.966216721 +0000 2024-03-09T08:51:10.062 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-09T08:51:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-03-09T08:51:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-03-09T08:51:10.146 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.00034875 s, 1.5 MB/s 2024-03-09T08:51:10.149 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-09T08:51:10.203 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-09T08:51:10.203 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi067.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-09T08:51:10.204 INFO:tasks.ceph:Generating config... 2024-03-09T08:51:10.205 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-09T08:51:10.205 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-03-09T08:51:10.205 INFO:tasks.ceph:[client] debug client = 20 2024-03-09T08:51:10.205 INFO:tasks.ceph:[client] debug ms = 1 2024-03-09T08:51:10.205 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-09T08:51:10.205 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-09T08:51:10.205 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-09T08:51:10.205 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-09T08:51:10.205 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-09T08:51:10.205 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-03-09T08:51:10.206 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-09T08:51:10.207 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-09T08:51:10.207 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-09T08:51:10.207 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-09T08:51:10.207 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-09T08:51:10.207 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-09T08:51:10.207 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-09T08:51:10.207 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-09T08:51:10.207 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-09T08:51:10.207 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-09T08:51:10.207 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-09T08:51:10.207 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-09T08:51:10.207 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-09T08:51:10.207 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-09T08:51:10.207 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-09T08:51:10.207 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-09T08:51:10.208 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-03-09T08:51:10.209 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-09T08:51:10.209 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-09T08:51:10.209 INFO:tasks.ceph:Setting up mon.a... 2024-03-09T08:51:10.209 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-09T08:51:10.364 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.keyring 2024-03-09T08:51:10.373 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-09T08:51:10.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-09T08:51:10.606 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.67'), ('mon.b', '172.21.15.134'), ('mon.c', '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]')] 2024-03-09T08:51:10.607 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.67,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-09T08:51:10.608 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:10.608 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-09T08:51:10.664 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.67 --add b 172.21.15.134 --addv c '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:10.760 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:10.760 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: generated fsid aded9f77-d43e-4509-848a-632eae6a080f 2024-03-09T08:51:10.760 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = quincy 2024-03-09T08:51:10.760 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 0 2024-03-09T08:51:10.760 INFO:teuthology.orchestra.run.smithi067.stdout:fsid aded9f77-d43e-4509-848a-632eae6a080f 2024-03-09T08:51:10.760 INFO:teuthology.orchestra.run.smithi067.stdout:last_changed 2024-03-09T08:51:10.767075+0000 2024-03-09T08:51:10.760 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-03-09T08:51:10.767075+0000 2024-03-09T08:51:10.761 INFO:teuthology.orchestra.run.smithi067.stdout:min_mon_release 17 (quincy) 2024-03-09T08:51:10.761 INFO:teuthology.orchestra.run.smithi067.stdout:election_strategy: 1 2024-03-09T08:51:10.761 INFO:teuthology.orchestra.run.smithi067.stdout:0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-03-09T08:51:10.761 INFO:teuthology.orchestra.run.smithi067.stdout:1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-03-09T08:51:10.761 INFO:teuthology.orchestra.run.smithi067.stdout:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.c 2024-03-09T08:51:10.761 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-09T08:51:10.763 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-09T08:51:10.808 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID aded9f77-d43e-4509-848a-632eae6a080f... 2024-03-09T08:51:10.809 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-09T08:51:10.855 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-09T08:51:11.029 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2024-03-09T08:51:11.029 INFO:teuthology.orchestra.run.smithi067.stdout: chdir = "" 2024-03-09T08:51:11.029 INFO:teuthology.orchestra.run.smithi067.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-09T08:51:11.029 INFO:teuthology.orchestra.run.smithi067.stdout: auth supported = cephx 2024-03-09T08:51:11.029 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.029 INFO:teuthology.orchestra.run.smithi067.stdout: filestore xattr use omap = true 2024-03-09T08:51:11.029 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.029 INFO:teuthology.orchestra.run.smithi067.stdout: mon clock drift allowed = 1.000 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: osd crush chooseleaf type = 0 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: auth debug = true 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on old message = true 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on bug = true 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: mon pg warn max object skew = 0 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: # disable pg_autoscaler by default for new pools 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-09T08:51:11.030 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default size = 2 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow primary affinity = true 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow pg remap = true 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on legacy crush tunables = false 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on crush straw calc version zero = false 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on no sortbitwise = false 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on osd down out interval zero = false 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on too few osds = false 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: mon_allow_pool_size_one = true 2024-03-09T08:51:11.031 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: osd default data pool replay window = 5 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: mon allow pool delete = true 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: mon cluster log file level = debug 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: debug asserts on shutdown = true 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: mon health detail to clog = false 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: mon host = "172.21.15.67,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on skipped message = False 2024-03-09T08:51:11.032 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = aded9f77-d43e-4509-848a-632eae6a080f 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout:[osd] 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: osd journal size = 100 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub load threshold = 5.0 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub max interval = 600 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: osd mclock profile = high_recovery_ops 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: osd recover clone overlap = true 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: osd recovery max chunk = 1048576 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug shutdown = true 2024-03-09T08:51:11.033 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug op order = true 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug verify stray on activate = true 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug trim objects = true 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: osd open classes on start = true 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug pg log writeout = true 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: osd deep scrub update digest min age = 30 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: osd map max advance = 10 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: journal zero on create = true 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: filestore ondisk finisher threads = 3 2024-03-09T08:51:11.034 INFO:teuthology.orchestra.run.smithi067.stdout: filestore apply finisher threads = 3 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: bdev debug aio = true 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug misdirected ops = true 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: bdev async discard = True 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: bdev enable discard = True 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore allocator = bitmap 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore block size = 96636764160 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore fsck on mount = True 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluefs = 1/20 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluestore = 1/20 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: debug osd = 20 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: debug rocksdb = 4/10 2024-03-09T08:51:11.035 INFO:teuthology.orchestra.run.smithi067.stdout: mon client directed command retry = 5 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd backfillfull_ratio = 0.85 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd full ratio = 0.9 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd nearfull ratio = 0.8 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject delay max = 1 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject delay probability = 0.005 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject delay type = client mds 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject socket failures = 2500 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: osd blocked scrub grace period = 3600 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: osd failsafe full ratio = 0.95 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: osd max markdown count = 1000 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: osd objectstore = bluestore 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: osd op complaint time = 180 2024-03-09T08:51:11.036 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr] 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout: debug mgr = 20 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout: mgr/telemetry/nag = false 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout:[mon] 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2024-03-09T08:51:11.037 INFO:teuthology.orchestra.run.smithi067.stdout: debug paxos = 20 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: mon data avail warn = 5 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: mon mgr mkfs grace = 240 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd reporter subtree level = osd 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd prime pg temp = true 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: auth mon ticket ttl = 660 # 11m 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: auth service ticket ttl = 240 # 4m 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: # don't complain about insecure global_id in the test suite 2024-03-09T08:51:11.038 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: # 1m isn't quite enough 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: mon_down_mkfs_grace = 2m 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_filestore_osds = false 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: mon op complaint time = 120 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout:[client] 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: rgw cache enabled = true 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable ops log = true 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable usage log = true 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-09T08:51:11.039 INFO:teuthology.orchestra.run.smithi067.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout: client mount timeout = 600 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout: client_shutdown_timeout = 120 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout: debug client = 20 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout: rados mon op timeout = 900 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout: rados osd op timeout = 900 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.a] 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.b] 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.c] 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a] 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c] 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b] 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d] 2024-03-09T08:51:11.040 INFO:teuthology.orchestra.run.smithi067.stdout:[mds] 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: debug mds = 20 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: debug mds balancer = 20 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: mds bal fragment size max = 10000 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: mds bal merge size = 5 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: mds bal split bits = 3 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: mds bal split size = 100 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: mds debug frag = True 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: mds debug scatterstat = True 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: mds op complaint time = 180 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: mds verify scatter = True 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: mon client directed command retry = 5 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject delay max = 1 2024-03-09T08:51:11.041 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject delay probability = 0.005 2024-03-09T08:51:11.042 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject delay type = client mds osd 2024-03-09T08:51:11.042 INFO:teuthology.orchestra.run.smithi067.stdout: ms inject socket failures = 2500 2024-03-09T08:51:11.042 INFO:teuthology.orchestra.run.smithi067.stdout: osd op complaint time = 180 2024-03-09T08:51:11.042 INFO:teuthology.orchestra.run.smithi067.stdout: rados mon op timeout = 900 2024-03-09T08:51:11.042 INFO:teuthology.orchestra.run.smithi067.stdout: rados osd op timeout = 900 2024-03-09T08:51:11.042 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-03-09T08:51:11.042 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-03-09T08:51:11.042 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-09T08:51:11.042 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-09T08:51:11.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-03-09T08:51:11.044 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.044 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-03-09T08:51:11.044 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-09T08:51:11.044 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.044 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-03-09T08:51:11.044 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.044 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-03-09T08:51:11.044 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-03-09T08:51:11.044 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-03-09T08:51:11.044 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-03-09T08:51:11.044 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-03-09T08:51:11.045 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-03-09T08:51:11.045 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-03-09T08:51:11.045 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-09T08:51:11.045 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-09T08:51:11.045 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-03-09T08:51:11.045 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-09T08:51:11.045 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-03-09T08:51:11.045 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.045 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.67,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on skipped message = False 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = aded9f77-d43e-4509-848a-632eae6a080f 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-03-09T08:51:11.046 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug trim objects = true 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-03-09T08:51:11.047 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-03-09T08:51:11.048 INFO:teuthology.orchestra.run.smithi134.stdout: bdev async discard = True 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: bdev enable discard = True 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore allocator = bitmap 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 1/20 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 1/20 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 4/10 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: mon client directed command retry = 5 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2024-03-09T08:51:11.049 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay max = 1 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay probability = 0.005 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay type = client mds 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject socket failures = 2500 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: osd blocked scrub grace period = 3600 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: osd max markdown count = 1000 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-03-09T08:51:11.050 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-03-09T08:51:11.051 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: mon op complaint time = 120 2024-03-09T08:51:11.052 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: client mount timeout = 600 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: client_shutdown_timeout = 120 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: debug client = 20 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-03-09T08:51:11.053 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout:[mds] 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds = 20 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: debug mds balancer = 20 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal fragment size max = 10000 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal merge size = 5 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal split bits = 3 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: mds bal split size = 100 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug frag = True 2024-03-09T08:51:11.054 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug scatterstat = True 2024-03-09T08:51:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: mds op complaint time = 180 2024-03-09T08:51:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: mds verify scatter = True 2024-03-09T08:51:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: mon client directed command retry = 5 2024-03-09T08:51:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay max = 1 2024-03-09T08:51:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay probability = 0.005 2024-03-09T08:51:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject delay type = client mds osd 2024-03-09T08:51:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: ms inject socket failures = 2500 2024-03-09T08:51:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: osd op complaint time = 180 2024-03-09T08:51:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: rados mon op timeout = 900 2024-03-09T08:51:11.055 INFO:teuthology.orchestra.run.smithi134.stdout: rados osd op timeout = 900 2024-03-09T08:51:11.078 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-09T08:51:11.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-09T08:51:11.230 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-09T08:51:11.231 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:11.231 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-09T08:51:11.280 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:11.280 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-09T08:51:11.379 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T08:51:11.379 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:11.379 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-09T08:51:11.379 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-09T08:51:11.537 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:11.537 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:11.637 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T08:51:11.638 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:11.638 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-09T08:51:11.638 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-09T08:51:11.790 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:11.790 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:11.888 INFO:tasks.ceph:Setting up mon nodes... 2024-03-09T08:51:11.888 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-09T08:51:11.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-09T08:51:12.087 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-09T08:51:12.098 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-09T08:51:12.299 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-09T08:51:12.311 INFO:tasks.ceph:Setting up mds nodes... 2024-03-09T08:51:12.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-09T08:51:12.515 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-09T08:51:12.525 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-09T08:51:12.624 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-09T08:51:12.816 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-09T08:51:12.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-09T08:51:12.921 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-09T08:51:13.096 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-09T08:51:13.106 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-09T08:51:13.194 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-09T08:51:13.366 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-09T08:51:13.375 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-09T08:51:13.471 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-09T08:51:13.471 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-09T08:51:13.608 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-09T08:51:13.675 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-09T08:51:13.800 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-09T08:51:13.856 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-09T08:51:13.856 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi067.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-09T08:51:13.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-09T08:51:13.964 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-09T08:51:13.964 INFO:tasks.ceph:role: osd.0 2024-03-09T08:51:13.964 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T08:51:13.965 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-09T08:51:14.072 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-09T08:51:14.072 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-09T08:51:14.072 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-09T08:51:14.072 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-09T08:51:14.072 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-09T08:51:14.072 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-03-09T08:51:14.072 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-09T08:51:14.073 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-09T08:51:14.073 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-09T08:51:14.073 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-09T08:51:14.589 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-03-09T08:51:14.595 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-03-09T08:51:14.596 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-09T08:51:14.787 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-09T08:51:14.903 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-09T08:51:15.005 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-09T08:51:15.006 INFO:tasks.ceph:role: osd.1 2024-03-09T08:51:15.006 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T08:51:15.006 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-09T08:51:15.110 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-09T08:51:15.110 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-09T08:51:15.111 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-09T08:51:15.111 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-09T08:51:15.111 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-09T08:51:15.111 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-03-09T08:51:15.111 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-09T08:51:15.111 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-09T08:51:15.111 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-09T08:51:15.111 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-09T08:51:15.609 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-03-09T08:51:15.615 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-03-09T08:51:15.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-09T08:51:15.765 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-09T08:51:15.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-09T08:51:15.947 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-09T08:51:15.947 INFO:tasks.ceph:role: osd.2 2024-03-09T08:51:15.947 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T08:51:15.948 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-09T08:51:16.049 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-09T08:51:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-09T08:51:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-09T08:51:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-09T08:51:16.049 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-09T08:51:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-03-09T08:51:16.049 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-09T08:51:16.050 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-09T08:51:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-09T08:51:16.050 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-09T08:51:16.560 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-03-09T08:51:16.566 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-03-09T08:51:16.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-09T08:51:16.770 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-09T08:51:16.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-09T08:51:16.986 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-09T08:51:16.986 INFO:tasks.ceph:role: osd.3 2024-03-09T08:51:16.986 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T08:51:16.986 DEBUG:teuthology.orchestra.run.smithi067:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-09T08:51:17.090 INFO:teuthology.orchestra.run.smithi067.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-09T08:51:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-09T08:51:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-09T08:51:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-09T08:51:17.091 INFO:teuthology.orchestra.run.smithi067.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-09T08:51:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: = sunit=0 swidth=0 blks 2024-03-09T08:51:17.091 INFO:teuthology.orchestra.run.smithi067.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-09T08:51:17.091 INFO:teuthology.orchestra.run.smithi067.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-09T08:51:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-09T08:51:17.091 INFO:teuthology.orchestra.run.smithi067.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-09T08:51:17.572 INFO:teuthology.orchestra.run.smithi067.stdout:Discarding blocks...Done. 2024-03-09T08:51:17.579 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi067.front.sepia.ceph.com -o noatime 2024-03-09T08:51:17.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-09T08:51:17.725 DEBUG:teuthology.orchestra.run.smithi067:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-09T08:51:17.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:17.981+0000 7f03c4a11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-09T08:51:17.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:17.987+0000 7f03c4a11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-09T08:51:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:17.987+0000 7f03c4a11640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-09T08:51:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:17.987+0000 7f03c4a11640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-09T08:51:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:17.987+0000 7f03c4a11640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-09T08:51:17.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:17.997+0000 7f03c4a11640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-09T08:51:20.938 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-09T08:51:21.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:21.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:21.194+0000 7f54c941b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-09T08:51:21.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:21.194+0000 7f54c941b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-09T08:51:21.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:21.194+0000 7f54c941b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-09T08:51:21.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:21.194+0000 7f54c941b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-09T08:51:21.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:21.194+0000 7f54c941b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-09T08:51:21.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:21.197+0000 7f54c941b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-09T08:51:24.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-09T08:51:24.468 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:24.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:24.620+0000 7f5baf59e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-09T08:51:24.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:24.621+0000 7f5baf59e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-09T08:51:24.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:24.621+0000 7f5baf59e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-09T08:51:24.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:24.621+0000 7f5baf59e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-09T08:51:24.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:24.621+0000 7f5baf59e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-09T08:51:24.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:24.623+0000 7f5baf59e640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-09T08:51:27.809 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-09T08:51:27.908 DEBUG:teuthology.orchestra.run.smithi067:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:28.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:28.058+0000 7f32ad611640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-09T08:51:28.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:28.059+0000 7f32ad611640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-09T08:51:28.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:28.059+0000 7f32ad611640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-09T08:51:28.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:28.059+0000 7f32ad611640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-09T08:51:28.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:28.059+0000 7f32ad611640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-09T08:51:28.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:28.061+0000 7f32ad611640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-09T08:51:31.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-09T08:51:31.350 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-09T08:51:31.448 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-09T08:51:31.448 INFO:tasks.ceph:role: osd.4 2024-03-09T08:51:31.448 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T08:51:31.448 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-09T08:51:31.545 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-09T08:51:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-09T08:51:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-09T08:51:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-09T08:51:31.546 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-09T08:51:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-03-09T08:51:31.546 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-09T08:51:31.546 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-09T08:51:31.546 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-09T08:51:31.546 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-09T08:51:32.053 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-03-09T08:51:32.059 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-03-09T08:51:32.059 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-09T08:51:32.234 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-09T08:51:32.346 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-09T08:51:32.424 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-09T08:51:32.425 INFO:tasks.ceph:role: osd.5 2024-03-09T08:51:32.425 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T08:51:32.425 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-09T08:51:32.504 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-09T08:51:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-09T08:51:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-09T08:51:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-09T08:51:32.504 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-09T08:51:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-03-09T08:51:32.504 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-09T08:51:32.504 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-09T08:51:32.504 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-09T08:51:32.505 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-09T08:51:32.961 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-03-09T08:51:32.969 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-03-09T08:51:32.969 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-09T08:51:33.101 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-09T08:51:33.184 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-09T08:51:33.260 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-09T08:51:33.261 INFO:tasks.ceph:role: osd.6 2024-03-09T08:51:33.261 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T08:51:33.261 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-09T08:51:33.341 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-09T08:51:33.341 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-09T08:51:33.341 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-09T08:51:33.341 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-09T08:51:33.341 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-09T08:51:33.341 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-03-09T08:51:33.341 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-09T08:51:33.341 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-09T08:51:33.342 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-09T08:51:33.342 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-09T08:51:33.851 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-03-09T08:51:33.857 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-03-09T08:51:33.857 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-09T08:51:34.013 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-09T08:51:34.112 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-09T08:51:34.203 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-09T08:51:34.203 INFO:tasks.ceph:role: osd.7 2024-03-09T08:51:34.203 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T08:51:34.203 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-09T08:51:34.298 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-09T08:51:34.298 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-09T08:51:34.298 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-09T08:51:34.298 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-09T08:51:34.299 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-09T08:51:34.299 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-03-09T08:51:34.299 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-09T08:51:34.299 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-09T08:51:34.299 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-09T08:51:34.299 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-09T08:51:34.757 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-03-09T08:51:34.764 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-03-09T08:51:34.764 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-09T08:51:34.872 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-09T08:51:34.947 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:35.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:35.096+0000 7f725b80b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-09T08:51:35.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:35.096+0000 7f725b80b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-09T08:51:35.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:35.096+0000 7f725b80b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-09T08:51:35.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:35.096+0000 7f725b80b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-09T08:51:35.091 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:35.097+0000 7f725b80b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-09T08:51:35.093 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:35.100+0000 7f725b80b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-09T08:51:38.027 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-09T08:51:38.118 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:38.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:51:38.209 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:51:38.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:38.259+0000 7f25d740e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-09T08:51:38.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:38.260+0000 7f25d740e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-09T08:51:38.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:38.260+0000 7f25d740e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-09T08:51:38.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:38.260+0000 7f25d740e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-09T08:51:38.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:38.260+0000 7f25d740e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-09T08:51:38.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:38.262+0000 7f25d740e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-09T08:51:41.167 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-09T08:51:41.258 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:41.395 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:41.400+0000 7f776ad24640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-09T08:51:41.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:41.401+0000 7f776ad24640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-09T08:51:41.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:41.401+0000 7f776ad24640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-09T08:51:41.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:41.401+0000 7f776ad24640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-09T08:51:41.396 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:41.401+0000 7f776ad24640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-09T08:51:41.397 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:41.403+0000 7f776ad24640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-09T08:51:44.561 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-09T08:51:44.656 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:44.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:44.793+0000 7f9a298ac640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-09T08:51:44.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:44.794+0000 7f9a298ac640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-09T08:51:44.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:44.794+0000 7f9a298ac640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-09T08:51:44.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:44.794+0000 7f9a298ac640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-09T08:51:44.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:44.794+0000 7f9a298ac640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-09T08:51:44.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:44.796+0000 7f9a298ac640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-09T08:51:47.974 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-09T08:51:48.068 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-09T08:51:48.068 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:48.068 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-09T08:51:48.168 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:48.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-09T08:51:48.270 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:48.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-09T08:51:48.372 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:48.372 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-09T08:51:48.475 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:48.475 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-09T08:51:48.577 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:48.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-09T08:51:48.677 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:48.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-09T08:51:48.776 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:48.777 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-09T08:51:48.874 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:48.875 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-09T08:51:48.958 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:48.958 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-09T08:51:49.093 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:49.093 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-09T08:51:49.187 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:49.187 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-09T08:51:49.280 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:49.280 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-09T08:51:49.373 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:49.374 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-09T08:51:49.467 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:49.467 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-09T08:51:49.528 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:49.529 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-09T08:51:49.584 INFO:tasks.ceph:Adding keys to all mons... 2024-03-09T08:51:49.584 DEBUG:teuthology.orchestra.run.smithi067:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-09T08:51:49.591 DEBUG:teuthology.orchestra.run.smithi134:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-09T08:51:49.676 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.y] 2024-03-09T08:51:49.676 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAAI+xlCV6kBRAA8yNt9yHKeOieO2ZrDRZM9Q== 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a] 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAAI+xlp90hHxAAap8+ve1tRdAVGmxD0VZSLg== 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c] 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAAI+xl8IMYMRAAu4QKhTH08bX6Eq+1oK/hLw== 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.0] 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAFI+xl+2WQOhAAcKf3Ief6vuKAZnVb4K0mpQ== 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.1] 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAJI+xli6SiCxAA8Ki1DYIyWfqGhyXiftA4gA== 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.2] 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAMI+xlOwMOJRAAIRDMcD1R1fEa1jC7c4PXaA== 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.3] 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAQI+xlajeUAxAA042BZ8+C69uAoiqmkNO0oA== 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.x] 2024-03-09T08:51:49.677 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAAI+xlA5NLEhAA1X4tcOAaCOMM4sOFpFRZOQ== 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b] 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQABI+xlVtMvBhAA3uRmB1LMNuiWvNAmzbXudw== 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d] 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQABI+xlAZtCFhAAUGe7R5fg+7BQ+Gq6DZu/sw== 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.4] 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAXI+xlYg3HBRAAsB1FlD8JkDptkwoitXUVJA== 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.5] 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAaI+xl2mGADxAAK0bp1bwAyJgsPKaorIIiEw== 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.6] 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAdI+xlM6PtFxAAgp3Vxvjtl6DNAKOtRC3XBw== 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.7] 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAgI+xl6m9sLxAAtDqClkr72+GbKRdG4VTIAw== 2024-03-09T08:51:49.678 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-03-09T08:51:49.679 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQABI+xl0tizJBAAdhR5EucirfL9jUbGUI0D1w== 2024-03-09T08:51:49.679 INFO:teuthology.orchestra.run.smithi067.stdout:[client.1] 2024-03-09T08:51:49.679 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQABI+xl5zEmMBAAm7LT7/SfM7lAd9AQa0rbxQ== 2024-03-09T08:51:49.680 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.y] 2024-03-09T08:51:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAAI+xlCV6kBRAA8yNt9yHKeOieO2ZrDRZM9Q== 2024-03-09T08:51:49.680 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-03-09T08:51:49.680 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAAI+xlp90hHxAAap8+ve1tRdAVGmxD0VZSLg== 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAAI+xl8IMYMRAAu4QKhTH08bX6Eq+1oK/hLw== 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.0] 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAFI+xl+2WQOhAAcKf3Ief6vuKAZnVb4K0mpQ== 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.1] 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAJI+xli6SiCxAA8Ki1DYIyWfqGhyXiftA4gA== 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.2] 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAMI+xlOwMOJRAAIRDMcD1R1fEa1jC7c4PXaA== 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.3] 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAQI+xlajeUAxAA042BZ8+C69uAoiqmkNO0oA== 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.x] 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAAI+xlA5NLEhAA1X4tcOAaCOMM4sOFpFRZOQ== 2024-03-09T08:51:49.681 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQABI+xlVtMvBhAA3uRmB1LMNuiWvNAmzbXudw== 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQABI+xlAZtCFhAAUGe7R5fg+7BQ+Gq6DZu/sw== 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.4] 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAXI+xlYg3HBRAAsB1FlD8JkDptkwoitXUVJA== 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.5] 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAaI+xl2mGADxAAK0bp1bwAyJgsPKaorIIiEw== 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.6] 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAdI+xlM6PtFxAAgp3Vxvjtl6DNAKOtRC3XBw== 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.7] 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAgI+xl6m9sLxAAtDqClkr72+GbKRdG4VTIAw== 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout:[client.0] 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQABI+xl0tizJBAAdhR5EucirfL9jUbGUI0D1w== 2024-03-09T08:51:49.682 INFO:teuthology.orchestra.run.smithi134.stdout:[client.1] 2024-03-09T08:51:49.683 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQABI+xl5zEmMBAAm7LT7/SfM7lAd9AQa0rbxQ== 2024-03-09T08:51:49.685 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-09T08:51:49.729 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-09T08:51:49.875 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-09T08:51:49.880 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-09T08:51:50.022 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-09T08:51:50.026 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-09T08:51:50.167 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:50.173 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:50.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:50.325 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:50.466 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:50.474 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:50.620 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:50.625 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:50.765 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-09T08:51:50.771 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-09T08:51:50.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-09T08:51:50.920 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-09T08:51:51.068 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-09T08:51:51.073 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-09T08:51:51.220 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:51.227 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:51.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:51.376 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:51.521 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:51.526 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:51.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:51.678 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-09T08:51:51.828 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-09T08:51:51.835 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-09T08:51:51.982 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-09T08:51:51.987 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-09T08:51:52.133 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-09T08:51:52.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-09T08:51:52.236 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-09T08:51:52.405 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-09T08:51:52.500 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-09T08:51:52.593 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-09T08:51:52.747 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-09T08:51:52.824 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-09T08:51:52.914 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-09T08:51:53.063 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-09T08:51:53.149 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:53.157 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-09T08:51:53.209 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-09T08:51:53.209 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-09T08:51:53.209 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-09T08:51:53.214 INFO:tasks.ceph.mon.a:Started 2024-03-09T08:51:53.215 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-09T08:51:53.215 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-09T08:51:53.219 INFO:tasks.ceph.mon.b:Started 2024-03-09T08:51:53.219 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T08:51:53.220 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T08:51:53.224 INFO:tasks.ceph.mon.c:Started 2024-03-09T08:51:53.224 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-09T08:51:53.224 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-09T08:51:53.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-09T08:51:53.231 INFO:tasks.ceph.mgr.y:Started 2024-03-09T08:51:53.231 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-09T08:51:53.231 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-09T08:51:53.236 INFO:tasks.ceph.mgr.x:Started 2024-03-09T08:51:53.236 INFO:tasks.ceph:Setting crush tunables to default 2024-03-09T08:51:53.236 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph osd crush tunables default 2024-03-09T08:51:53.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.428+0000 7fd3bdc00640 1 Processor -- start 2024-03-09T08:51:53.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.428+0000 7fd3bdc00640 1 -- start start 2024-03-09T08:51:53.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.430+0000 7fd3bdc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd3b80c6770 0x7fd3b80abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:53.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.430+0000 7fd3bdc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd3b81563f0 0x7fd3b81568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:53.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.430+0000 7fd3bdc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b8152e70 con 0x7fd3b80ac490 2024-03-09T08:51:53.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.430+0000 7fd3bdc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b81539f0 con 0x7fd3b81563f0 2024-03-09T08:51:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.430+0000 7fd3bdc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b80aa3b0 con 0x7fd3b80c6770 2024-03-09T08:51:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.430+0000 7fd3b6a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd3b80c6770 0x7fd3b80abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.430+0000 7fd3b6000640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd3b80ac490 0x7fd3b814fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:51552/0 (socket says 172.21.15.67:51552) 2024-03-09T08:51:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.430+0000 7fd3b6000640 1 -- 172.21.15.67:0/2295560365 learned_addr learned my addr 172.21.15.67:0/2295560365 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:51:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.430+0000 7fd3b7400640 1 --2- 172.21.15.67:0/2295560365 >> v2:172.21.15.67:3300/0 conn(0x7fd3b81563f0 0x7fd3b81568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.431+0000 7fd3b6a00640 1 -- 172.21.15.67:0/2295560365 >> v2:172.21.15.134:3300/0 conn(0x7fd3b80c6770 msgr2=0x7fd3b80abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:51:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.431+0000 7fd3b6a00640 1 -- 172.21.15.67:0/2295560365 >> v2:172.21.15.134:3300/0 conn(0x7fd3b80c6770 msgr2=0x7fd3b80abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:51:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.431+0000 7fd3b6a00640 1 --2- 172.21.15.67:0/2295560365 >> v2:172.21.15.134:3300/0 conn(0x7fd3b80c6770 0x7fd3b80abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:51:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.431+0000 7fd3b6a00640 1 --2- 172.21.15.67:0/2295560365 >> v2:172.21.15.134:3300/0 conn(0x7fd3b80c6770 0x7fd3b80abf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:51:53.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.431+0000 7fd3b7400640 1 -- 172.21.15.67:0/2295560365 >> v2:172.21.15.67:3300/0 conn(0x7fd3b81563f0 msgr2=0x7fd3b81568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:51:53.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.431+0000 7fd3b7400640 1 -- 172.21.15.67:0/2295560365 >> v2:172.21.15.67:3300/0 conn(0x7fd3b81563f0 msgr2=0x7fd3b81568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:51:53.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.431+0000 7fd3b7400640 1 --2- 172.21.15.67:0/2295560365 >> v2:172.21.15.67:3300/0 conn(0x7fd3b81563f0 0x7fd3b81568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:51:53.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.431+0000 7fd3b7400640 1 --2- 172.21.15.67:0/2295560365 >> v2:172.21.15.67:3300/0 conn(0x7fd3b81563f0 0x7fd3b81568a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:51:53.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.438+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974637150 0 0) 0x7fd3b8152e70 con 0x7fd3b80ac490 2024-03-09T08:51:53.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.438+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd39c003520 con 0x7fd3b80ac490 2024-03-09T08:51:53.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1004142988 0 0) 0x7fd39c003520 con 0x7fd3b80ac490 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 >> v2:172.21.15.134:3300/0 conn(0x7fd3b80c6770 msgr2=0x7fd3b80abf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 --2- 172.21.15.67:0/2295560365 >> v2:172.21.15.134:3300/0 conn(0x7fd3b80c6770 0x7fd3b80abf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 >> v2:172.21.15.67:3300/0 conn(0x7fd3b81563f0 msgr2=0x7fd3b81568a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 --2- 172.21.15.67:0/2295560365 >> v2:172.21.15.67:3300/0 conn(0x7fd3b81563f0 0x7fd3b81568a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8153150 con 0x7fd3b80ac490 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fd3a4001670 con 0x7fd3b80ac490 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 >> v1:172.21.15.134:6789/0 conn(0x7fd3b80ac490 legacy=0x7fd3b814fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c003a00 0x7fd39c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39c004800 0x7fd39c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c0051c0 0x7fd39c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b81539f0 con 0x7fd39c003a00 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c00c090 con 0x7fd39c0051c0 2024-03-09T08:51:53.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c00c290 con 0x7fd39c004800 2024-03-09T08:51:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b7400640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c0051c0 0x7fd39c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.447+0000 7fd3b6000640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39c004800 0x7fd39c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.448+0000 7fd3b6a00640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c003a00 0x7fd39c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.448+0000 7fd3b7400640 1 -- 172.21.15.67:0/2295560365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39c004800 msgr2=0x7fd39c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.448+0000 7fd3b7400640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39c004800 0x7fd39c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.448+0000 7fd3b7400640 1 -- 172.21.15.67:0/2295560365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c003a00 msgr2=0x7fd39c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.448+0000 7fd3b7400640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c003a00 0x7fd39c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.448+0000 7fd3b7400640 1 -- 172.21.15.67:0/2295560365 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c003520 con 0x7fd39c0051c0 2024-03-09T08:51:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.448+0000 7fd3b6000640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd39c004800 0x7fd39c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:51:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.448+0000 7fd3b6a00640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c003a00 0x7fd39c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:51:53.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.449+0000 7fd3b7400640 1 -- 172.21.15.67:0/2295560365 reap_dead start 2024-03-09T08:51:53.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.449+0000 7fd3b7400640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c0051c0 0x7fd39c00b9c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0007330 tx=0x7fd39c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e0428300dc5326f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:53.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.449+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4007c80 con 0x7fd39c0051c0 2024-03-09T08:51:53.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.449+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b002ccd0 con 0x7fd39c0051c0 2024-03-09T08:51:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.449+0000 7fd3b5600640 1 -- 172.21.15.67:0/2295560365 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0009bd0 con 0x7fd39c0051c0 2024-03-09T08:51:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.450+0000 7fd3bdc00640 1 -- 172.21.15.67:0/2295560365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c0051c0 msgr2=0x7fd39c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.450+0000 7fd3bdc00640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c0051c0 0x7fd39c00b9c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0007330 tx=0x7fd39c005830 comp rx=0 tx=0).stop 2024-03-09T08:51:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.450+0000 7fd3bdc00640 1 -- 172.21.15.67:0/2295560365 shutdown_connections 2024-03-09T08:51:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.450+0000 7fd3bdc00640 1 --2- 172.21.15.67:0/2295560365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd39c0051c0 0x7fd39c00b9c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.450+0000 7fd3bdc00640 1 -- 172.21.15.67:0/2295560365 >> 172.21.15.67:0/2295560365 conn(0x7fd3b806b040 msgr2=0x7fd3b806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.450+0000 7fd3bdc00640 1 -- 172.21.15.67:0/2295560365 shutdown_connections 2024-03-09T08:51:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.451+0000 7fd3bdc00640 1 -- 172.21.15.67:0/2295560365 wait complete. 2024-03-09T08:51:53.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.452+0000 7fd3bdc00640 1 Processor -- start 2024-03-09T08:51:53.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.452+0000 7fd3bdc00640 1 -- start start 2024-03-09T08:51:53.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.453+0000 7fd3bdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c0051c0 0x7fd3b80c0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:53.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.453+0000 7fd3bdc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3b80c1200 0x7fd3b815f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:53.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.453+0000 7fd3bdc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3b8154950 0x7fd3b815fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:53.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.453+0000 7fd3bdc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8152a40 con 0x7fd39c0051c0 2024-03-09T08:51:53.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.453+0000 7fd3bdc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8153710 con 0x7fd3b8154950 2024-03-09T08:51:53.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.453+0000 7fd3bdc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8152bc0 con 0x7fd3b80c1200 2024-03-09T08:51:53.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.453+0000 7fd3b6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c0051c0 0x7fd3b80c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:53.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.454+0000 7fd3b6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c0051c0 0x7fd3b80c0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50762/0 (socket says 172.21.15.67:50762) 2024-03-09T08:51:53.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.454+0000 7fd3b6a00640 1 -- 172.21.15.67:0/1985855068 learned_addr learned my addr 172.21.15.67:0/1985855068 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.454+0000 7fd3b6000640 1 --2- 172.21.15.67:0/1985855068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3b80c1200 0x7fd3b815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.454+0000 7fd3b7400640 1 --2- 172.21.15.67:0/1985855068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3b8154950 0x7fd3b815fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.454+0000 7fd3b6a00640 1 -- 172.21.15.67:0/1985855068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3b80c1200 msgr2=0x7fd3b815f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.454+0000 7fd3b6a00640 1 --2- 172.21.15.67:0/1985855068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3b80c1200 0x7fd3b815f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.454+0000 7fd3b6a00640 1 -- 172.21.15.67:0/1985855068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3b8154950 msgr2=0x7fd3b815fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.454+0000 7fd3b6a00640 1 --2- 172.21.15.67:0/1985855068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3b8154950 0x7fd3b815fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.454+0000 7fd3b6a00640 1 -- 172.21.15.67:0/1985855068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b815f130 con 0x7fd39c0051c0 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.454+0000 7fd3b6a00640 1 --2- 172.21.15.67:0/1985855068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c0051c0 0x7fd3b80c0830 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800f4e0 tx=0x7fd3a800f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.455+0000 7fd3aea00640 1 -- 172.21.15.67:0/1985855068 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8019020 con 0x7fd39c0051c0 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.455+0000 7fd3aea00640 1 -- 172.21.15.67:0/1985855068 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a8014870 con 0x7fd39c0051c0 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.455+0000 7fd3bdc00640 1 -- 172.21.15.67:0/1985855068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b81ef110 con 0x7fd39c0051c0 2024-03-09T08:51:53.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.455+0000 7fd3aea00640 1 -- 172.21.15.67:0/1985855068 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8014c20 con 0x7fd39c0051c0 2024-03-09T08:51:53.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.455+0000 7fd3bdc00640 1 -- 172.21.15.67:0/1985855068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b81ef6d0 con 0x7fd39c0051c0 2024-03-09T08:51:53.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.455+0000 7fd3aea00640 1 -- 172.21.15.67:0/1985855068 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd3a801c020 con 0x7fd39c0051c0 2024-03-09T08:51:53.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.455+0000 7fd3aea00640 1 -- 172.21.15.67:0/1985855068 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd3a801e400 con 0x7fd39c0051c0 2024-03-09T08:51:53.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.456+0000 7fd3bdc00640 1 -- 172.21.15.67:0/1985855068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b8075a00 con 0x7fd39c0051c0 2024-03-09T08:51:53.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.459+0000 7fd3aea00640 1 -- 172.21.15.67:0/1985855068 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd3a8023020 con 0x7fd39c0051c0 2024-03-09T08:51:53.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.532+0000 7fd3bdc00640 1 -- 172.21.15.67:0/1985855068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd3b80c4ac0 con 0x7fd39c0051c0 2024-03-09T08:51:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.538+0000 7fd3aea00640 1 -- 172.21.15.67:0/1985855068 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd3a8014dc0 con 0x7fd39c0051c0 2024-03-09T08:51:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2024-03-09T08:51:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.540+0000 7fd3bdc00640 1 -- 172.21.15.67:0/1985855068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c0051c0 msgr2=0x7fd3b80c0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.540+0000 7fd3bdc00640 1 --2- 172.21.15.67:0/1985855068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c0051c0 0x7fd3b80c0830 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800f4e0 tx=0x7fd3a800f9a0 comp rx=0 tx=0).stop 2024-03-09T08:51:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.540+0000 7fd3bdc00640 1 -- 172.21.15.67:0/1985855068 shutdown_connections 2024-03-09T08:51:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.540+0000 7fd3bdc00640 1 --2- 172.21.15.67:0/1985855068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3b8154950 0x7fd3b815fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.540+0000 7fd3bdc00640 1 --2- 172.21.15.67:0/1985855068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3b80c1200 0x7fd3b815f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.540+0000 7fd3bdc00640 1 --2- 172.21.15.67:0/1985855068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd39c0051c0 0x7fd3b80c0830 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.540+0000 7fd3bdc00640 1 -- 172.21.15.67:0/1985855068 >> 172.21.15.67:0/1985855068 conn(0x7fd3b806b040 msgr2=0x7fd3b8145c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.540+0000 7fd3bdc00640 1 -- 172.21.15.67:0/1985855068 shutdown_connections 2024-03-09T08:51:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:53.540+0000 7fd3bdc00640 1 -- 172.21.15.67:0/1985855068 wait complete. 2024-03-09T08:51:53.553 INFO:tasks.ceph:check_enable_crimson: False 2024-03-09T08:51:53.553 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-09T08:51:53.554 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:53.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-09T08:51:53.638 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:53.638 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-09T08:51:53.761 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:53.761 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-09T08:51:53.893 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:51:53.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-09T08:51:53.984 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:53.984 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-09T08:51:54.072 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:54.072 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-09T08:51:54.189 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:54.193+0000 7fef4ca42040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-09T08:51:54.200 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:54.201 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-09T08:51:54.217 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:54.222+0000 7f87aa246040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-09T08:51:54.255 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:54.259+0000 7fef4ca42040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-09T08:51:54.286 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:54.290+0000 7f87aa246040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-09T08:51:54.328 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:51:54.328 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-09T08:51:54.456 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new d3c74a26-9960-44e3-a2cf-eae01cd1f1c9 0 2024-03-09T08:51:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.692+0000 7f9ca4800640 1 Processor -- start 2024-03-09T08:51:54.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.693+0000 7f9ca4800640 1 -- start start 2024-03-09T08:51:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.694+0000 7f9ca4800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9c9c153500 0x7f9c9c0c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.694+0000 7f9ca4800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9c155ed0 con 0x7f9c9c0c9500 2024-03-09T08:51:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.694+0000 7f9ca4800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9c156170 con 0x7f9c9c0caf20 2024-03-09T08:51:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.694+0000 7f9ca4800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9c1567c0 con 0x7f9c9c153500 2024-03-09T08:51:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.694+0000 7f9ca1400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f9c9c0caf20 0x7f9c9c0c8010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:53198/0 (socket says 172.21.15.134:53198) 2024-03-09T08:51:54.690 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.694+0000 7f9ca1400640 1 -- 172.21.15.134:0/1840936742 learned_addr learned my addr 172.21.15.134:0/1840936742 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-03-09T08:51:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca1e00640 1 --2- 172.21.15.134:0/1840936742 >> v2:172.21.15.67:3300/0 conn(0x7f9c9c153500 0x7f9c9c0c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784688404 0 0) 0x7f9c9c156170 con 0x7f9c9c0caf20 2024-03-09T08:51:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c84003520 con 0x7f9c9c0caf20 2024-03-09T08:51:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 399609515 0 0) 0x7f9c84003520 con 0x7f9c9c0caf20 2024-03-09T08:51:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 >> v2:172.21.15.67:3300/0 conn(0x7f9c9c153500 msgr2=0x7f9c9c0c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 --2- 172.21.15.134:0/1840936742 >> v2:172.21.15.67:3300/0 conn(0x7f9c9c153500 0x7f9c9c0c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 >> v1:172.21.15.67:6789/0 conn(0x7f9c9c0c9500 legacy=0x7f9c9c0c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:54.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c156450 con 0x7f9c9c0caf20 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f9c8c001670 con 0x7f9c9c0caf20 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 >> v1:172.21.15.134:6789/0 conn(0x7f9c9c0caf20 legacy=0x7f9c9c0c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 --2- 172.21.15.134:0/1840936742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c84003a00 0x7f9c84003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 --2- 172.21.15.134:0/1840936742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c84004800 0x7f9c84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 --2- 172.21.15.134:0/1840936742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c84005210 0x7f9c8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c1567c0 con 0x7f9c84004800 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8400c160 con 0x7f9c84005210 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.695+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8400c360 con 0x7f9c84003a00 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.696+0000 7f9ca1e00640 1 --2- 172.21.15.134:0/1840936742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c84003a00 0x7f9c84003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.696+0000 7f9ca1400640 1 --2- 172.21.15.134:0/1840936742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c84004800 0x7f9c84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.696+0000 7f9ca1e00640 1 -- 172.21.15.134:0/1840936742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c84005210 msgr2=0x7f9c8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.696+0000 7f9ca1e00640 1 --2- 172.21.15.134:0/1840936742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c84005210 0x7f9c8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.696+0000 7f9ca1e00640 1 -- 172.21.15.134:0/1840936742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c84004800 msgr2=0x7f9c84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:54.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.696+0000 7f9ca1e00640 1 --2- 172.21.15.134:0/1840936742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c84004800 0x7f9c84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.696+0000 7f9ca1e00640 1 -- 172.21.15.134:0/1840936742 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c84003520 con 0x7f9c84003a00 2024-03-09T08:51:54.694 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.696+0000 7f9ca2800640 1 -- 172.21.15.134:0/1840936742 reap_dead start 2024-03-09T08:51:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.697+0000 7f9ca1e00640 1 --2- 172.21.15.134:0/1840936742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c84003a00 0x7f9c84003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009a90 tx=0x7f9c84005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fc03dee8d449ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.697+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90039070 con 0x7f9c84003a00 2024-03-09T08:51:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.698+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9002c6b0 con 0x7f9c84003a00 2024-03-09T08:51:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.698+0000 7f9ca0a00640 1 -- 172.21.15.134:0/1840936742 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90009690 con 0x7f9c84003a00 2024-03-09T08:51:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.698+0000 7f9ca4800640 1 -- 172.21.15.134:0/1840936742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c84003a00 msgr2=0x7f9c84003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.698+0000 7f9ca4800640 1 --2- 172.21.15.134:0/1840936742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c84003a00 0x7f9c84003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009a90 tx=0x7f9c84005830 comp rx=0 tx=0).stop 2024-03-09T08:51:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.698+0000 7f9ca4800640 1 -- 172.21.15.134:0/1840936742 shutdown_connections 2024-03-09T08:51:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.698+0000 7f9ca4800640 1 --2- 172.21.15.134:0/1840936742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c84003a00 0x7f9c84003e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.698+0000 7f9ca4800640 1 -- 172.21.15.134:0/1840936742 >> 172.21.15.134:0/1840936742 conn(0x7f9c9c0ad350 msgr2=0x7f9c9c0ad760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.699+0000 7f9ca4800640 1 -- 172.21.15.134:0/1840936742 shutdown_connections 2024-03-09T08:51:54.695 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.699+0000 7f9ca4800640 1 -- 172.21.15.134:0/1840936742 wait complete. 2024-03-09T08:51:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.700+0000 7f9ca4800640 1 Processor -- start 2024-03-09T08:51:54.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.700+0000 7f9ca4800640 1 -- start start 2024-03-09T08:51:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.701+0000 7f9ca4800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c84003a00 0x7f9c9c1e2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.701+0000 7f9ca4800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c9c1eb8b0 0x7f9c9c0c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.701+0000 7f9ca4800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c9c0c3290 0x7f9c9c1ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:54.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.701+0000 7f9ca4800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c0c6a70 con 0x7f9c9c0c3290 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.701+0000 7f9ca4800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c0c68f0 con 0x7f9c84003a00 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.701+0000 7f9ca4800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c0c9970 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.701+0000 7f9ca1e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c84003a00 0x7f9c9c1e2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.701+0000 7f9ca1400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c9c1eb8b0 0x7f9c9c0c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.701+0000 7f9ca1e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c84003a00 0x7f9c9c1e2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:56692/0 (socket says 172.21.15.134:56692) 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.701+0000 7f9ca1e00640 1 -- 172.21.15.134:0/2698776148 learned_addr learned my addr 172.21.15.134:0/2698776148 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9ca2800640 1 --2- 172.21.15.134:0/2698776148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c9c0c3290 0x7f9c9c1ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9ca1400640 1 -- 172.21.15.134:0/2698776148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c84003a00 msgr2=0x7f9c9c1e2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9ca1400640 1 --2- 172.21.15.134:0/2698776148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c84003a00 0x7f9c9c1e2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9ca1400640 1 -- 172.21.15.134:0/2698776148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c9c0c3290 msgr2=0x7f9c9c1ef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9ca1400640 1 --2- 172.21.15.134:0/2698776148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c9c0c3290 0x7f9c9c1ef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9ca1400640 1 -- 172.21.15.134:0/2698776148 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c0c4f00 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9ca1e00640 1 --2- 172.21.15.134:0/2698776148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c84003a00 0x7f9c9c1e2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:51:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9ca1400640 1 --2- 172.21.15.134:0/2698776148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c9c1eb8b0 0x7f9c9c0c2d50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c050170 tx=0x7f9c8c00eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9c96000640 1 -- 172.21.15.134:0/2698776148 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00c6c0 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9c96000640 1 -- 172.21.15.134:0/2698776148 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8c018070 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9ca4800640 1 -- 172.21.15.134:0/2698776148 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9c0c51e0 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.702+0000 7f9c96000640 1 -- 172.21.15.134:0/2698776148 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00caa0 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.703+0000 7f9ca4800640 1 -- 172.21.15.134:0/2698776148 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9c1e3280 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.703+0000 7f9c96000640 1 -- 172.21.15.134:0/2698776148 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9c8c00cc40 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.703+0000 7f9c96000640 1 -- 172.21.15.134:0/2698776148 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9c8c013460 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.704+0000 7f9ca4800640 1 -- 172.21.15.134:0/2698776148 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9c150d20 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.707+0000 7f9c96000640 1 -- 172.21.15.134:0/2698776148 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9c9c150d20 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.781 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.786+0000 7f9ca4800640 1 -- 172.21.15.134:0/2698776148 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d3c74a26-9960-44e3-a2cf-eae01cd1f1c9", "id": 0} v 0) v1 -- 0x7f9c9c075a00 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.792+0000 7f9c96000640 1 -- 172.21.15.134:0/2698776148 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d3c74a26-9960-44e3-a2cf-eae01cd1f1c9", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9c8c01d460 con 0x7f9c9c1eb8b0 2024-03-09T08:51:54.788 INFO:teuthology.orchestra.run.smithi134.stdout:0 2024-03-09T08:51:54.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.794+0000 7f9ca4800640 1 -- 172.21.15.134:0/2698776148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c9c1eb8b0 msgr2=0x7f9c9c0c2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:54.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.794+0000 7f9ca4800640 1 --2- 172.21.15.134:0/2698776148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c9c1eb8b0 0x7f9c9c0c2d50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c050170 tx=0x7f9c8c00eb90 comp rx=0 tx=0).stop 2024-03-09T08:51:54.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.794+0000 7f9ca4800640 1 -- 172.21.15.134:0/2698776148 shutdown_connections 2024-03-09T08:51:54.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.794+0000 7f9ca4800640 1 --2- 172.21.15.134:0/2698776148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c9c0c3290 0x7f9c9c1ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.794+0000 7f9ca4800640 1 --2- 172.21.15.134:0/2698776148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c9c1eb8b0 0x7f9c9c0c2d50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.794+0000 7f9ca4800640 1 --2- 172.21.15.134:0/2698776148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c84003a00 0x7f9c9c1e2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.794+0000 7f9ca4800640 1 -- 172.21.15.134:0/2698776148 >> 172.21.15.134:0/2698776148 conn(0x7f9c9c0ad350 msgr2=0x7f9c9c154b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:54.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.794+0000 7f9ca4800640 1 -- 172.21.15.134:0/2698776148 shutdown_connections 2024-03-09T08:51:54.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.794+0000 7f9ca4800640 1 -- 172.21.15.134:0/2698776148 wait complete. 2024-03-09T08:51:54.806 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 749c5cf3-91d5-4bb9-bd35-e34f9e83ceac 1 2024-03-09T08:51:54.881 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:54.885+0000 7fef4ca42040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-09T08:51:54.915 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:54.919+0000 7f87aa246040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-09T08:51:54.948 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:54.952+0000 7fef4ca42040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-09T08:51:54.981 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:54.986+0000 7f87aa246040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-09T08:51:54.990 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.995+0000 7fa439200640 1 Processor -- start 2024-03-09T08:51:54.991 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.995+0000 7fa439200640 1 -- start start 2024-03-09T08:51:54.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.996+0000 7fa439200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa434073a60 0x7fa4340c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:54.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.996+0000 7fa439200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4340c4770 0x7fa4340c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:54.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa439200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa434152bf0 con 0x7fa43414cd00 2024-03-09T08:51:54.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa439200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa434152ed0 con 0x7fa434073a60 2024-03-09T08:51:54.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa439200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4341531b0 con 0x7fa4340c4770 2024-03-09T08:51:54.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa433400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fa43414cd00 0x7fa4340c4cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56188/0 (socket says 172.21.15.134:56188) 2024-03-09T08:51:54.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa433400640 1 -- 172.21.15.134:0/3183543074 learned_addr learned my addr 172.21.15.134:0/3183543074 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-03-09T08:51:54.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa433e00640 1 --2- 172.21.15.134:0/3183543074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4340c4770 0x7fa4340c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:54.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa432a00640 1 --2- 172.21.15.134:0/3183543074 >> v2:172.21.15.67:3300/0 conn(0x7fa434073a60 0x7fa4340c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:54.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa432000640 1 -- 172.21.15.134:0/3183543074 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964885989 0 0) 0x7fa434152bf0 con 0x7fa43414cd00 2024-03-09T08:51:54.993 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa433e00640 1 -- 172.21.15.134:0/3183543074 >> v2:172.21.15.67:3300/0 conn(0x7fa434073a60 msgr2=0x7fa4340c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:54.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa433e00640 1 --2- 172.21.15.134:0/3183543074 >> v2:172.21.15.67:3300/0 conn(0x7fa434073a60 0x7fa4340c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa433e00640 1 -- 172.21.15.134:0/3183543074 >> v1:172.21.15.134:6789/0 conn(0x7fa43414cd00 legacy=0x7fa4340c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:54.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.997+0000 7fa433e00640 1 -- 172.21.15.134:0/3183543074 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa434153490 con 0x7fa4340c4770 2024-03-09T08:51:54.994 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.998+0000 7fa433e00640 1 --2- 172.21.15.134:0/3183543074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4340c4770 0x7fa4340c34c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009d90 tx=0x7fa42c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2e9aebf2bfdc6b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:54.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.999+0000 7fa432000640 1 -- 172.21.15.134:0/3183543074 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c039070 con 0x7fa4340c4770 2024-03-09T08:51:54.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.999+0000 7fa432000640 1 -- 172.21.15.134:0/3183543074 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42c02cc50 con 0x7fa4340c4770 2024-03-09T08:51:54.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:54.999+0000 7fa432000640 1 -- 172.21.15.134:0/3183543074 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c034440 con 0x7fa4340c4770 2024-03-09T08:51:54.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.000+0000 7fa439200640 1 -- 172.21.15.134:0/3183543074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4340c4770 msgr2=0x7fa4340c34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:54.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.000+0000 7fa439200640 1 --2- 172.21.15.134:0/3183543074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4340c4770 0x7fa4340c34c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009d90 tx=0x7fa42c02bea0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.000+0000 7fa439200640 1 -- 172.21.15.134:0/3183543074 shutdown_connections 2024-03-09T08:51:54.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.000+0000 7fa439200640 1 --2- 172.21.15.134:0/3183543074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4340c4770 0x7fa4340c34c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.000+0000 7fa439200640 1 --2- 172.21.15.134:0/3183543074 >> v2:172.21.15.67:3300/0 conn(0x7fa434073a60 0x7fa4340c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:54.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.000+0000 7fa439200640 1 -- 172.21.15.134:0/3183543074 >> 172.21.15.134:0/3183543074 conn(0x7fa4340aa890 msgr2=0x7fa43406c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:54.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.000+0000 7fa439200640 1 -- 172.21.15.134:0/3183543074 shutdown_connections 2024-03-09T08:51:54.996 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.000+0000 7fa439200640 1 -- 172.21.15.134:0/3183543074 wait complete. 2024-03-09T08:51:54.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.001+0000 7fa439200640 1 Processor -- start 2024-03-09T08:51:54.997 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.002+0000 7fa439200640 1 -- start start 2024-03-09T08:51:54.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.002+0000 7fa439200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa434073a60 0x7fa434155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:54.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.002+0000 7fa439200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4340c4770 0x7fa434156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:54.999 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.002+0000 7fa439200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa43414cd00 0x7fa4340c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.002+0000 7fa439200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4341528a0 con 0x7fa4340c4770 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.002+0000 7fa439200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa434153770 con 0x7fa43414cd00 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.002+0000 7fa439200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340c5400 con 0x7fa434073a60 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa433400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa434073a60 0x7fa434155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa433e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa43414cd00 0x7fa4340c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa433400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa434073a60 0x7fa434155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:36174/0 (socket says 172.21.15.134:36174) 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa433400640 1 -- 172.21.15.134:0/3522572982 learned_addr learned my addr 172.21.15.134:0/3522572982 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa433e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa43414cd00 0x7fa4340c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38990/0 (socket says 172.21.15.134:38990) 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa432a00640 1 --2- 172.21.15.134:0/3522572982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4340c4770 0x7fa434156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa433e00640 1 -- 172.21.15.134:0/3522572982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa434073a60 msgr2=0x7fa434155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa433e00640 1 --2- 172.21.15.134:0/3522572982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa434073a60 0x7fa434155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa433e00640 1 -- 172.21.15.134:0/3522572982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4340c4770 msgr2=0x7fa434156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa433e00640 1 --2- 172.21.15.134:0/3522572982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4340c4770 0x7fa434156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa433e00640 1 -- 172.21.15.134:0/3522572982 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4341ea060 con 0x7fa43414cd00 2024-03-09T08:51:55.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.003+0000 7fa433e00640 1 --2- 172.21.15.134:0/3522572982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa43414cd00 0x7fa4340c7600 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa42c033000 tx=0x7fa42c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:55.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.004+0000 7fa42b400640 1 -- 172.21.15.134:0/3522572982 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c039050 con 0x7fa43414cd00 2024-03-09T08:51:55.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.004+0000 7fa42b400640 1 -- 172.21.15.134:0/3522572982 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42c045070 con 0x7fa43414cd00 2024-03-09T08:51:55.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.004+0000 7fa42b400640 1 -- 172.21.15.134:0/3522572982 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c03e610 con 0x7fa43414cd00 2024-03-09T08:51:55.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.004+0000 7fa439200640 1 -- 172.21.15.134:0/3522572982 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa434153cf0 con 0x7fa43414cd00 2024-03-09T08:51:55.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.004+0000 7fa439200640 1 -- 172.21.15.134:0/3522572982 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa434156a60 con 0x7fa43414cd00 2024-03-09T08:51:55.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.004+0000 7fa42b400640 1 -- 172.21.15.134:0/3522572982 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa42c03e7b0 con 0x7fa43414cd00 2024-03-09T08:51:55.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.004+0000 7fa42b400640 1 -- 172.21.15.134:0/3522572982 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fa42c040450 con 0x7fa43414cd00 2024-03-09T08:51:55.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.005+0000 7fa439200640 1 -- 172.21.15.134:0/3522572982 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4341e9dd0 con 0x7fa43414cd00 2024-03-09T08:51:55.003 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:55.008+0000 7fef4ca42040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-09T08:51:55.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.008+0000 7fa42b400640 1 -- 172.21.15.134:0/3522572982 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa42c03ec30 con 0x7fa43414cd00 2024-03-09T08:51:55.035 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:55.040+0000 7f87aa246040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-09T08:51:55.069 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:55.073+0000 7fef4ca42040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-09T08:51:55.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.086+0000 7fa439200640 1 -- 172.21.15.134:0/3522572982 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "749c5cf3-91d5-4bb9-bd35-e34f9e83ceac", "id": 1} v 0) v1 -- 0x7fa4340c3e00 con 0x7fa43414cd00 2024-03-09T08:51:55.090 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.094+0000 7fa42b400640 1 -- 172.21.15.134:0/3522572982 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "749c5cf3-91d5-4bb9-bd35-e34f9e83ceac", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa42c04a460 con 0x7fa43414cd00 2024-03-09T08:51:55.090 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-03-09T08:51:55.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.096+0000 7fa439200640 1 -- 172.21.15.134:0/3522572982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa43414cd00 msgr2=0x7fa4340c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.096+0000 7fa439200640 1 --2- 172.21.15.134:0/3522572982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa43414cd00 0x7fa4340c7600 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa42c033000 tx=0x7fa42c004a40 comp rx=0 tx=0).stop 2024-03-09T08:51:55.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.096+0000 7fa439200640 1 -- 172.21.15.134:0/3522572982 shutdown_connections 2024-03-09T08:51:55.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.096+0000 7fa439200640 1 --2- 172.21.15.134:0/3522572982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa43414cd00 0x7fa4340c7600 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.096+0000 7fa439200640 1 --2- 172.21.15.134:0/3522572982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4340c4770 0x7fa434156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.096+0000 7fa439200640 1 --2- 172.21.15.134:0/3522572982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa434073a60 0x7fa434155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.096+0000 7fa439200640 1 -- 172.21.15.134:0/3522572982 >> 172.21.15.134:0/3522572982 conn(0x7fa4340aa890 msgr2=0x7fa434151bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:55.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.096+0000 7fa439200640 1 -- 172.21.15.134:0/3522572982 shutdown_connections 2024-03-09T08:51:55.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.096+0000 7fa439200640 1 -- 172.21.15.134:0/3522572982 wait complete. 2024-03-09T08:51:55.102 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:55.107+0000 7f87aa246040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-09T08:51:55.107 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 2c83ccf2-0bc6-4d05-be62-5d788c8c20ce 2 2024-03-09T08:51:55.122 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:55.127+0000 7fef4ca42040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-09T08:51:55.154 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:55.159+0000 7f87aa246040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-09T08:51:55.261 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:55.265+0000 7fef4ca42040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-09T08:51:55.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.287+0000 7f7f4a200640 1 Processor -- start 2024-03-09T08:51:55.283 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.288+0000 7f7f4a200640 1 -- start start 2024-03-09T08:51:55.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.289+0000 7f7f4a200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7f440c5f10 0x7f7f440c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.289+0000 7f7f4a200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 0x7f7f440cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.289+0000 7f7f4a200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f440c9a30 con 0x7f7f4406b040 2024-03-09T08:51:55.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.289+0000 7f7f4a200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f440c9d10 con 0x7f7f440c5f10 2024-03-09T08:51:55.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.289+0000 7f7f4a200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440ca340 con 0x7f7f440c9490 2024-03-09T08:51:55.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.289+0000 7f7f48e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7f440c5f10 0x7f7f440c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.289+0000 7f7f48e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7f440c5f10 0x7f7f440c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38992/0 (socket says 172.21.15.134:38992) 2024-03-09T08:51:55.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.289+0000 7f7f48e00640 1 -- 172.21.15.134:0/3315280712 learned_addr learned my addr 172.21.15.134:0/3315280712 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:55.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.289+0000 7f7f49800640 1 --2- 172.21.15.134:0/3315280712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 0x7f7f440cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.290+0000 7f7f48e00640 1 -- 172.21.15.134:0/3315280712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 msgr2=0x7f7f440cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:55.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.290+0000 7f7f48e00640 1 --2- 172.21.15.134:0/3315280712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 0x7f7f440cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.290+0000 7f7f48e00640 1 -- 172.21.15.134:0/3315280712 >> v1:172.21.15.134:6789/0 conn(0x7f7f4406b040 legacy=0x7f7f441567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.290+0000 7f7f48e00640 1 -- 172.21.15.134:0/3315280712 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440ca620 con 0x7f7f440c5f10 2024-03-09T08:51:55.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.290+0000 7f7f49800640 1 --2- 172.21.15.134:0/3315280712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 0x7f7f440cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:51:55.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.291+0000 7f7f48e00640 1 --2- 172.21.15.134:0/3315280712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f440c5f10 0x7f7f440c62f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009870 tx=0x7f7f3802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf7ca15a20b73c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:55.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.291+0000 7f7f43400640 1 -- 172.21.15.134:0/3315280712 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f380317c0 con 0x7f7f440c5f10 2024-03-09T08:51:55.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.291+0000 7f7f43400640 1 -- 172.21.15.134:0/3315280712 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f38037070 con 0x7f7f440c5f10 2024-03-09T08:51:55.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.291+0000 7f7f43400640 1 -- 172.21.15.134:0/3315280712 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38031b80 con 0x7f7f440c5f10 2024-03-09T08:51:55.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.292+0000 7f7f4a200640 1 -- 172.21.15.134:0/3315280712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f440c5f10 msgr2=0x7f7f440c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.292+0000 7f7f4a200640 1 --2- 172.21.15.134:0/3315280712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f440c5f10 0x7f7f440c62f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009870 tx=0x7f7f3802b860 comp rx=0 tx=0).stop 2024-03-09T08:51:55.288 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.292+0000 7f7f4a200640 1 -- 172.21.15.134:0/3315280712 shutdown_connections 2024-03-09T08:51:55.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.292+0000 7f7f4a200640 1 --2- 172.21.15.134:0/3315280712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 0x7f7f440cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.292+0000 7f7f4a200640 1 --2- 172.21.15.134:0/3315280712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f440c5f10 0x7f7f440c62f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.292+0000 7f7f4a200640 1 -- 172.21.15.134:0/3315280712 >> 172.21.15.134:0/3315280712 conn(0x7f7f440cb590 msgr2=0x7f7f440cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:55.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.292+0000 7f7f4a200640 1 -- 172.21.15.134:0/3315280712 shutdown_connections 2024-03-09T08:51:55.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.292+0000 7f7f4a200640 1 -- 172.21.15.134:0/3315280712 wait complete. 2024-03-09T08:51:55.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.295+0000 7f7f4a200640 1 Processor -- start 2024-03-09T08:51:55.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.295+0000 7f7f4a200640 1 -- start start 2024-03-09T08:51:55.291 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:55.295+0000 7f87aa246040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-09T08:51:55.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.296+0000 7f7f4a200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f4406b040 0x7f7f440c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f4a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f440c5f10 0x7f7f44154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f4a200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 0x7f7f44159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f4a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441537a0 con 0x7f7f440c5f10 2024-03-09T08:51:55.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f4a200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f44153620 con 0x7f7f4406b040 2024-03-09T08:51:55.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f4a200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440cb080 con 0x7f7f440c9490 2024-03-09T08:51:55.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f48e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f4406b040 0x7f7f440c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f49800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 0x7f7f44159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f48e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f4406b040 0x7f7f440c3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:38996/0 (socket says 172.21.15.134:38996) 2024-03-09T08:51:55.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f49800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 0x7f7f44159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:36196/0 (socket says 172.21.15.134:36196) 2024-03-09T08:51:55.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f48e00640 1 -- 172.21.15.134:0/51769890 learned_addr learned my addr 172.21.15.134:0/51769890 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:55.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f43e00640 1 --2- 172.21.15.134:0/51769890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f440c5f10 0x7f7f44154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f49800640 1 -- 172.21.15.134:0/51769890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f4406b040 msgr2=0x7f7f440c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f49800640 1 --2- 172.21.15.134:0/51769890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f4406b040 0x7f7f440c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f49800640 1 -- 172.21.15.134:0/51769890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f440c5f10 msgr2=0x7f7f44154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f49800640 1 --2- 172.21.15.134:0/51769890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f440c5f10 0x7f7f44154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.297+0000 7f7f49800640 1 -- 172.21.15.134:0/51769890 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f44149900 con 0x7f7f440c9490 2024-03-09T08:51:55.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.298+0000 7f7f49800640 1 --2- 172.21.15.134:0/51769890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 0x7f7f44159250 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009db0 tx=0x7f7f3c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:55.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.298+0000 7f7f40c00640 1 -- 172.21.15.134:0/51769890 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c018030 con 0x7f7f440c9490 2024-03-09T08:51:55.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.298+0000 7f7f40c00640 1 -- 172.21.15.134:0/51769890 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c00cd30 con 0x7f7f440c9490 2024-03-09T08:51:55.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.298+0000 7f7f4a200640 1 -- 172.21.15.134:0/51769890 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f440c3010 con 0x7f7f440c9490 2024-03-09T08:51:55.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.298+0000 7f7f40c00640 1 -- 172.21.15.134:0/51769890 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c003bb0 con 0x7f7f440c9490 2024-03-09T08:51:55.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.298+0000 7f7f4a200640 1 -- 172.21.15.134:0/51769890 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f441552a0 con 0x7f7f440c9490 2024-03-09T08:51:55.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.298+0000 7f7f40c00640 1 -- 172.21.15.134:0/51769890 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7f3c01b070 con 0x7f7f440c9490 2024-03-09T08:51:55.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.298+0000 7f7f40c00640 1 -- 172.21.15.134:0/51769890 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f7f3c0148a0 con 0x7f7f440c9490 2024-03-09T08:51:55.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.299+0000 7f7f4a200640 1 -- 172.21.15.134:0/51769890 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4404f3c0 con 0x7f7f440c9490 2024-03-09T08:51:55.297 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.301+0000 7f7f40c00640 1 -- 172.21.15.134:0/51769890 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7f3c010040 con 0x7f7f440c9490 2024-03-09T08:51:55.348 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:55.352+0000 7fef4ca42040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-09T08:51:55.375 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:55.380+0000 7f87aa246040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-09T08:51:55.375 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.380+0000 7f7f4a200640 1 -- 172.21.15.134:0/51769890 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2c83ccf2-0bc6-4d05-be62-5d788c8c20ce", "id": 2} v 0) v1 -- 0x7f7f440cc260 con 0x7f7f440c9490 2024-03-09T08:51:55.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.388+0000 7f7f40c00640 1 -- 172.21.15.134:0/51769890 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2c83ccf2-0bc6-4d05-be62-5d788c8c20ce", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7f3c014b40 con 0x7f7f440c9490 2024-03-09T08:51:55.384 INFO:teuthology.orchestra.run.smithi134.stdout:2 2024-03-09T08:51:55.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.389+0000 7f7f4a200640 1 -- 172.21.15.134:0/51769890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 msgr2=0x7f7f44159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.389+0000 7f7f4a200640 1 --2- 172.21.15.134:0/51769890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 0x7f7f44159250 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009db0 tx=0x7f7f3c00c490 comp rx=0 tx=0).stop 2024-03-09T08:51:55.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.389+0000 7f7f4a200640 1 -- 172.21.15.134:0/51769890 shutdown_connections 2024-03-09T08:51:55.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.389+0000 7f7f4a200640 1 --2- 172.21.15.134:0/51769890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f440c9490 0x7f7f44159250 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.389+0000 7f7f4a200640 1 --2- 172.21.15.134:0/51769890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f440c5f10 0x7f7f44154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.389+0000 7f7f4a200640 1 --2- 172.21.15.134:0/51769890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f4406b040 0x7f7f440c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.389+0000 7f7f4a200640 1 -- 172.21.15.134:0/51769890 >> 172.21.15.134:0/51769890 conn(0x7f7f440cb590 msgr2=0x7f7f4406c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:55.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.389+0000 7f7f4a200640 1 -- 172.21.15.134:0/51769890 shutdown_connections 2024-03-09T08:51:55.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.390+0000 7f7f4a200640 1 -- 172.21.15.134:0/51769890 wait complete. 2024-03-09T08:51:55.401 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new aab9dcc8-7016-4aa6-aeb2-e030df702b71 3 2024-03-09T08:51:55.405 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:55.409+0000 7fef4ca42040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-09T08:51:55.432 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:55.437+0000 7f87aa246040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-09T08:51:55.470 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:55.474+0000 7fef4ca42040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-09T08:51:55.497 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:55.502+0000 7f87aa246040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-09T08:51:55.528 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:55.532+0000 7fef4ca42040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-09T08:51:55.554 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:55.559+0000 7f87aa246040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-09T08:51:55.577 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.582+0000 7fd087e00640 1 Processor -- start 2024-03-09T08:51:55.578 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.582+0000 7fd087e00640 1 -- start start 2024-03-09T08:51:55.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.583+0000 7fd087e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd088152f20 0x7fd088153380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.583+0000 7fd087e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd0880ca2c0 0x7fd0880ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.583+0000 7fd087e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd088155d70 con 0x7fd0880c8190 2024-03-09T08:51:55.579 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.583+0000 7fd087e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd088156c40 con 0x7fd088152f20 2024-03-09T08:51:55.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.583+0000 7fd087e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd088156030 con 0x7fd0880ca2c0 2024-03-09T08:51:55.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.584+0000 7fd087400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd0880ca2c0 0x7fd0880ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.584+0000 7fd087400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd0880ca2c0 0x7fd0880ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39010/0 (socket says 172.21.15.134:39010) 2024-03-09T08:51:55.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.584+0000 7fd087400640 1 -- 172.21.15.134:0/3953596812 learned_addr learned my addr 172.21.15.134:0/3953596812 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:55.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.584+0000 7fd086000640 1 --2- 172.21.15.134:0/3953596812 >> v2:172.21.15.67:3300/0 conn(0x7fd088152f20 0x7fd088153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.584+0000 7fd087400640 1 -- 172.21.15.134:0/3953596812 >> v2:172.21.15.67:3300/0 conn(0x7fd088152f20 msgr2=0x7fd088153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:55.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.584+0000 7fd087400640 1 --2- 172.21.15.134:0/3953596812 >> v2:172.21.15.67:3300/0 conn(0x7fd088152f20 0x7fd088153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.580 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.584+0000 7fd087400640 1 -- 172.21.15.134:0/3953596812 >> v1:172.21.15.134:6789/0 conn(0x7fd0880c8190 legacy=0x7fd0880c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.581 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.584+0000 7fd087400640 1 -- 172.21.15.134:0/3953596812 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd088156310 con 0x7fd0880ca2c0 2024-03-09T08:51:55.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.585+0000 7fd087400640 1 --2- 172.21.15.134:0/3953596812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880ca2c0 0x7fd0880ca6f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd08806cef0 tx=0x7fd08002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8357e99c73f6ef02 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:55.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.586+0000 7fd085600640 1 -- 172.21.15.134:0/3953596812 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08002ca80 con 0x7fd0880ca2c0 2024-03-09T08:51:55.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.586+0000 7fd085600640 1 -- 172.21.15.134:0/3953596812 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08002cc20 con 0x7fd0880ca2c0 2024-03-09T08:51:55.582 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.586+0000 7fd085600640 1 -- 172.21.15.134:0/3953596812 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0800343f0 con 0x7fd0880ca2c0 2024-03-09T08:51:55.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.586+0000 7fd087e00640 1 -- 172.21.15.134:0/3953596812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880ca2c0 msgr2=0x7fd0880ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.586+0000 7fd087e00640 1 --2- 172.21.15.134:0/3953596812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880ca2c0 0x7fd0880ca6f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd08806cef0 tx=0x7fd08002bee0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.586+0000 7fd087e00640 1 -- 172.21.15.134:0/3953596812 shutdown_connections 2024-03-09T08:51:55.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.586+0000 7fd087e00640 1 --2- 172.21.15.134:0/3953596812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880ca2c0 0x7fd0880ca6f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.586+0000 7fd087e00640 1 --2- 172.21.15.134:0/3953596812 >> v2:172.21.15.67:3300/0 conn(0x7fd088152f20 0x7fd088153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.586+0000 7fd087e00640 1 -- 172.21.15.134:0/3953596812 >> 172.21.15.134:0/3953596812 conn(0x7fd0880ad060 msgr2=0x7fd0880ad440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:55.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.587+0000 7fd087e00640 1 -- 172.21.15.134:0/3953596812 shutdown_connections 2024-03-09T08:51:55.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.587+0000 7fd087e00640 1 -- 172.21.15.134:0/3953596812 wait complete. 2024-03-09T08:51:55.583 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.588+0000 7fd087e00640 1 Processor -- start 2024-03-09T08:51:55.584 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.588+0000 7fd087e00640 1 -- start start 2024-03-09T08:51:55.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.589+0000 7fd087e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0880c8190 0x7fd0880c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.589+0000 7fd087e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880ca2c0 0x7fd0881578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.589+0000 7fd087e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd088152f20 0x7fd0880ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.589+0000 7fd087e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08806c9d0 con 0x7fd0880c8190 2024-03-09T08:51:55.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.589+0000 7fd087e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0881568d0 con 0x7fd088152f20 2024-03-09T08:51:55.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.589+0000 7fd087e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08806cb50 con 0x7fd0880ca2c0 2024-03-09T08:51:55.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.589+0000 7fd086000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880ca2c0 0x7fd0881578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.589+0000 7fd086000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880ca2c0 0x7fd0881578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:36204/0 (socket says 172.21.15.134:36204) 2024-03-09T08:51:55.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.589+0000 7fd086000640 1 -- 172.21.15.134:0/3540894888 learned_addr learned my addr 172.21.15.134:0/3540894888 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:55.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.589+0000 7fd087400640 1 --2- 172.21.15.134:0/3540894888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd088152f20 0x7fd0880ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.590+0000 7fd086a00640 1 --2- 172.21.15.134:0/3540894888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0880c8190 0x7fd0880c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.590+0000 7fd086000640 1 -- 172.21.15.134:0/3540894888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd088152f20 msgr2=0x7fd0880ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.590+0000 7fd086000640 1 --2- 172.21.15.134:0/3540894888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd088152f20 0x7fd0880ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.590+0000 7fd086000640 1 -- 172.21.15.134:0/3540894888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0880c8190 msgr2=0x7fd0880c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.590+0000 7fd086000640 1 --2- 172.21.15.134:0/3540894888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0880c8190 0x7fd0880c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.586 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.590+0000 7fd086000640 1 -- 172.21.15.134:0/3540894888 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd088157e00 con 0x7fd0880ca2c0 2024-03-09T08:51:55.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.590+0000 7fd086000640 1 --2- 172.21.15.134:0/3540894888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880ca2c0 0x7fd0881578c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd07400e870 tx=0x7fd07400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:55.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.590+0000 7fd07ea00640 1 -- 172.21.15.134:0/3540894888 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074017070 con 0x7fd0880ca2c0 2024-03-09T08:51:55.587 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.590+0000 7fd07ea00640 1 -- 172.21.15.134:0/3540894888 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07400c940 con 0x7fd0880ca2c0 2024-03-09T08:51:55.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.590+0000 7fd07ea00640 1 -- 172.21.15.134:0/3540894888 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07400cd30 con 0x7fd0880ca2c0 2024-03-09T08:51:55.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.590+0000 7fd087e00640 1 -- 172.21.15.134:0/3540894888 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0881580a0 con 0x7fd0880ca2c0 2024-03-09T08:51:55.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.591+0000 7fd087e00640 1 -- 172.21.15.134:0/3540894888 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0881eeef0 con 0x7fd0880ca2c0 2024-03-09T08:51:55.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.591+0000 7fd07ea00640 1 -- 172.21.15.134:0/3540894888 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd07401a070 con 0x7fd0880ca2c0 2024-03-09T08:51:55.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.591+0000 7fd07ea00640 1 -- 172.21.15.134:0/3540894888 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd074006020 con 0x7fd0880ca2c0 2024-03-09T08:51:55.588 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.592+0000 7fd087e00640 1 -- 172.21.15.134:0/3540894888 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0880689d0 con 0x7fd0880ca2c0 2024-03-09T08:51:55.589 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.594+0000 7fd07ea00640 1 -- 172.21.15.134:0/3540894888 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd0880689d0 con 0x7fd0880ca2c0 2024-03-09T08:51:55.668 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.673+0000 7fd087e00640 1 -- 172.21.15.134:0/3540894888 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "aab9dcc8-7016-4aa6-aeb2-e030df702b71", "id": 3} v 0) v1 -- 0x7fd08804f3c0 con 0x7fd0880ca2c0 2024-03-09T08:51:55.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.682+0000 7fd07ea00640 1 -- 172.21.15.134:0/3540894888 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aab9dcc8-7016-4aa6-aeb2-e030df702b71", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd0740064d0 con 0x7fd0880ca2c0 2024-03-09T08:51:55.677 INFO:teuthology.orchestra.run.smithi134.stdout:3 2024-03-09T08:51:55.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.683+0000 7fd087e00640 1 -- 172.21.15.134:0/3540894888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880ca2c0 msgr2=0x7fd0881578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.683+0000 7fd087e00640 1 --2- 172.21.15.134:0/3540894888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880ca2c0 0x7fd0881578c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd07400e870 tx=0x7fd07400ed30 comp rx=0 tx=0).stop 2024-03-09T08:51:55.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.683+0000 7fd087e00640 1 -- 172.21.15.134:0/3540894888 shutdown_connections 2024-03-09T08:51:55.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.683+0000 7fd087e00640 1 --2- 172.21.15.134:0/3540894888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd088152f20 0x7fd0880ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.683+0000 7fd087e00640 1 --2- 172.21.15.134:0/3540894888 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880ca2c0 0x7fd0881578c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.683+0000 7fd087e00640 1 --2- 172.21.15.134:0/3540894888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0880c8190 0x7fd0880c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.683+0000 7fd087e00640 1 -- 172.21.15.134:0/3540894888 >> 172.21.15.134:0/3540894888 conn(0x7fd0880ad060 msgr2=0x7fd08814cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:55.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.683+0000 7fd087e00640 1 -- 172.21.15.134:0/3540894888 shutdown_connections 2024-03-09T08:51:55.679 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.683+0000 7fd087e00640 1 -- 172.21.15.134:0/3540894888 wait complete. 2024-03-09T08:51:55.694 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 24203e41-1515-47c8-b596-01c471e0de42 4 2024-03-09T08:51:55.879 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.884+0000 7faf54e00640 1 Processor -- start 2024-03-09T08:51:55.880 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.884+0000 7faf54e00640 1 -- start start 2024-03-09T08:51:55.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.885+0000 7faf54e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf50151370 0x7faf50072f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.885+0000 7faf54e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faf50153b50 0x7faf5006fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.885+0000 7faf54e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf50072960 con 0x7faf500c6020 2024-03-09T08:51:55.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.885+0000 7faf54e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf50154af0 con 0x7faf50151370 2024-03-09T08:51:55.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.885+0000 7faf54e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf50153fa0 con 0x7faf50153b50 2024-03-09T08:51:55.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.886+0000 7faf4f400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7faf500c6020 0x7faf50150c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.134:56210/0 (socket says 172.21.15.134:56210) 2024-03-09T08:51:55.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.886+0000 7faf4f400640 1 -- 172.21.15.134:0/3780115248 learned_addr learned my addr 172.21.15.134:0/3780115248 (peer_addr_for_me v1:172.21.15.134:0/0) 2024-03-09T08:51:55.882 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.886+0000 7faf4fe00640 1 --2- 172.21.15.134:0/3780115248 >> v2:172.21.15.134:3300/0 conn(0x7faf50153b50 0x7faf5006fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.886+0000 7faf4ea00640 1 --2- 172.21.15.134:0/3780115248 >> v2:172.21.15.67:3300/0 conn(0x7faf50151370 0x7faf50072f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.886+0000 7faf4fe00640 1 -- 172.21.15.134:0/3780115248 >> v2:172.21.15.67:3300/0 conn(0x7faf50151370 msgr2=0x7faf50072f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:55.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.886+0000 7faf4fe00640 1 --2- 172.21.15.134:0/3780115248 >> v2:172.21.15.67:3300/0 conn(0x7faf50151370 0x7faf50072f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.886+0000 7faf4fe00640 1 -- 172.21.15.134:0/3780115248 >> v1:172.21.15.134:6789/0 conn(0x7faf500c6020 legacy=0x7faf50150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.886+0000 7faf4fe00640 1 -- 172.21.15.134:0/3780115248 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf50154250 con 0x7faf50153b50 2024-03-09T08:51:55.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.887+0000 7faf4fe00640 1 --2- 172.21.15.134:0/3780115248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf50153b50 0x7faf5006fa70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faf500c45f0 tx=0x7faf4802bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be2cbb715cdc1392 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:55.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.887+0000 7faf4e000640 1 -- 172.21.15.134:0/3780115248 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48039070 con 0x7faf50153b50 2024-03-09T08:51:55.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.887+0000 7faf4e000640 1 -- 172.21.15.134:0/3780115248 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4802cb50 con 0x7faf50153b50 2024-03-09T08:51:55.883 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.887+0000 7faf4e000640 1 -- 172.21.15.134:0/3780115248 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48034440 con 0x7faf50153b50 2024-03-09T08:51:55.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.888+0000 7faf54e00640 1 -- 172.21.15.134:0/3780115248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf50153b50 msgr2=0x7faf5006fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.888+0000 7faf54e00640 1 --2- 172.21.15.134:0/3780115248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf50153b50 0x7faf5006fa70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faf500c45f0 tx=0x7faf4802bda0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.884 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.888+0000 7faf54e00640 1 -- 172.21.15.134:0/3780115248 shutdown_connections 2024-03-09T08:51:55.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.888+0000 7faf54e00640 1 --2- 172.21.15.134:0/3780115248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf50153b50 0x7faf5006fa70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.888+0000 7faf54e00640 1 --2- 172.21.15.134:0/3780115248 >> v2:172.21.15.67:3300/0 conn(0x7faf50151370 0x7faf50072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.888+0000 7faf54e00640 1 -- 172.21.15.134:0/3780115248 >> 172.21.15.134:0/3780115248 conn(0x7faf5006ed30 msgr2=0x7faf50066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:55.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.888+0000 7faf54e00640 1 -- 172.21.15.134:0/3780115248 shutdown_connections 2024-03-09T08:51:55.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.888+0000 7faf54e00640 1 -- 172.21.15.134:0/3780115248 wait complete. 2024-03-09T08:51:55.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.890+0000 7faf54e00640 1 Processor -- start 2024-03-09T08:51:55.885 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.890+0000 7faf54e00640 1 -- start start 2024-03-09T08:51:55.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.891+0000 7faf54e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf500c6020 0x7faf50152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.891+0000 7faf54e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf50151370 0x7faf5014b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.891+0000 7faf54e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf50153b50 0x7faf5014b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:55.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.891+0000 7faf54e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf500aa4a0 con 0x7faf50151370 2024-03-09T08:51:55.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.891+0000 7faf54e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf50154810 con 0x7faf50153b50 2024-03-09T08:51:55.889 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.891+0000 7faf54e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5014bbf0 con 0x7faf500c6020 2024-03-09T08:51:55.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.891+0000 7faf4f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf500c6020 0x7faf50152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.891+0000 7faf4fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf50153b50 0x7faf5014b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.891+0000 7faf4f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf500c6020 0x7faf50152c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:36216/0 (socket says 172.21.15.134:36216) 2024-03-09T08:51:55.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.891+0000 7faf4f400640 1 -- 172.21.15.134:0/2932960160 learned_addr learned my addr 172.21.15.134:0/2932960160 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:55.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.891+0000 7faf4ea00640 1 --2- 172.21.15.134:0/2932960160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf50151370 0x7faf5014b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:55.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.892+0000 7faf4f400640 1 -- 172.21.15.134:0/2932960160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf50153b50 msgr2=0x7faf5014b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.892+0000 7faf4f400640 1 --2- 172.21.15.134:0/2932960160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf50153b50 0x7faf5014b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.892+0000 7faf4f400640 1 -- 172.21.15.134:0/2932960160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf50151370 msgr2=0x7faf5014b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.892+0000 7faf4f400640 1 --2- 172.21.15.134:0/2932960160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf50151370 0x7faf5014b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.892+0000 7faf4f400640 1 -- 172.21.15.134:0/2932960160 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf501eeac0 con 0x7faf500c6020 2024-03-09T08:51:55.891 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.892+0000 7faf4f400640 1 --2- 172.21.15.134:0/2932960160 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf500c6020 0x7faf50152c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faf4000c430 tx=0x7faf4000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:55.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.892+0000 7faf47400640 1 -- 172.21.15.134:0/2932960160 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40017070 con 0x7faf500c6020 2024-03-09T08:51:55.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.892+0000 7faf47400640 1 -- 172.21.15.134:0/2932960160 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf40005ce0 con 0x7faf500c6020 2024-03-09T08:51:55.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.892+0000 7faf47400640 1 -- 172.21.15.134:0/2932960160 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400072f0 con 0x7faf500c6020 2024-03-09T08:51:55.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.892+0000 7faf54e00640 1 -- 172.21.15.134:0/2932960160 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf501eeda0 con 0x7faf500c6020 2024-03-09T08:51:55.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.892+0000 7faf54e00640 1 -- 172.21.15.134:0/2932960160 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf501ef360 con 0x7faf500c6020 2024-03-09T08:51:55.892 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.893+0000 7faf47400640 1 -- 172.21.15.134:0/2932960160 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7faf4001a070 con 0x7faf500c6020 2024-03-09T08:51:55.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.893+0000 7faf47400640 1 -- 172.21.15.134:0/2932960160 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7faf40012de0 con 0x7faf500c6020 2024-03-09T08:51:55.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.894+0000 7faf54e00640 1 -- 172.21.15.134:0/2932960160 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf500c3da0 con 0x7faf500c6020 2024-03-09T08:51:55.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.896+0000 7faf47400640 1 -- 172.21.15.134:0/2932960160 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7faf40007490 con 0x7faf500c6020 2024-03-09T08:51:55.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.975+0000 7faf54e00640 1 -- 172.21.15.134:0/2932960160 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "24203e41-1515-47c8-b596-01c471e0de42", "id": 4} v 0) v1 -- 0x7faf5004f3c0 con 0x7faf500c6020 2024-03-09T08:51:55.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.984+0000 7faf47400640 1 -- 172.21.15.134:0/2932960160 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "24203e41-1515-47c8-b596-01c471e0de42", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7faf40007670 con 0x7faf500c6020 2024-03-09T08:51:55.980 INFO:teuthology.orchestra.run.smithi134.stdout:4 2024-03-09T08:51:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.985+0000 7faf54e00640 1 -- 172.21.15.134:0/2932960160 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf500c6020 msgr2=0x7faf50152c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.985+0000 7faf54e00640 1 --2- 172.21.15.134:0/2932960160 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf500c6020 0x7faf50152c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faf4000c430 tx=0x7faf4000c8f0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.986+0000 7faf54e00640 1 -- 172.21.15.134:0/2932960160 shutdown_connections 2024-03-09T08:51:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.986+0000 7faf54e00640 1 --2- 172.21.15.134:0/2932960160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf50153b50 0x7faf5014b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.986+0000 7faf54e00640 1 --2- 172.21.15.134:0/2932960160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf50151370 0x7faf5014b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.986+0000 7faf54e00640 1 --2- 172.21.15.134:0/2932960160 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf500c6020 0x7faf50152c80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.986+0000 7faf54e00640 1 -- 172.21.15.134:0/2932960160 >> 172.21.15.134:0/2932960160 conn(0x7faf5006ed30 msgr2=0x7faf50066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.986+0000 7faf54e00640 1 -- 172.21.15.134:0/2932960160 shutdown_connections 2024-03-09T08:51:55.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:55.986+0000 7faf54e00640 1 -- 172.21.15.134:0/2932960160 wait complete. 2024-03-09T08:51:55.999 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new db5b5ea7-6a92-4d06-b2f7-dcece773d296 5 2024-03-09T08:51:56.006 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:56.007+0000 7fef4ca42040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-09T08:51:56.029 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:56.033+0000 7f87aa246040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-09T08:51:56.171 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:56.175+0000 7fef4ca42040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-09T08:51:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.187+0000 7fdd2fe00640 1 Processor -- start 2024-03-09T08:51:56.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.187+0000 7fdd2fe00640 1 -- start start 2024-03-09T08:51:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd30152f20 0x7fdd30153380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd30155d70 con 0x7fdd300ca2c0 2024-03-09T08:51:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd30156c40 con 0x7fdd30152f20 2024-03-09T08:51:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd30156030 con 0x7fdd300c8190 2024-03-09T08:51:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2ea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2ea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:36226/0 (socket says 172.21.15.134:36226) 2024-03-09T08:51:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2ea00640 1 -- 172.21.15.134:0/1235862261 learned_addr learned my addr 172.21.15.134:0/1235862261 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2ea00640 1 -- 172.21.15.134:0/1235862261 >> v2:172.21.15.67:3300/0 conn(0x7fdd30152f20 msgr2=0x7fdd30153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2ea00640 1 --2- 172.21.15.134:0/1235862261 >> v2:172.21.15.67:3300/0 conn(0x7fdd30152f20 0x7fdd30153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.185 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2ea00640 1 -- 172.21.15.134:0/1235862261 >> v1:172.21.15.67:6789/0 conn(0x7fdd300ca2c0 legacy=0x7fdd300ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:56.186 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.189+0000 7fdd2ea00640 1 -- 172.21.15.134:0/1235862261 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd30156310 con 0x7fdd300c8190 2024-03-09T08:51:56.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.190+0000 7fdd2ea00640 1 --2- 172.21.15.134:0/1235862261 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300c9640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009870 tx=0x7fdd2002b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a21b6e1b65485846 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:56.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.190+0000 7fdd2d600640 1 -- 172.21.15.134:0/1235862261 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd200317c0 con 0x7fdd300c8190 2024-03-09T08:51:56.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.191+0000 7fdd2d600640 1 -- 172.21.15.134:0/1235862261 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd20037070 con 0x7fdd300c8190 2024-03-09T08:51:56.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.191+0000 7fdd2d600640 1 -- 172.21.15.134:0/1235862261 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20031b80 con 0x7fdd300c8190 2024-03-09T08:51:56.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.191+0000 7fdd2fe00640 1 -- 172.21.15.134:0/1235862261 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 msgr2=0x7fdd300c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.191+0000 7fdd2fe00640 1 --2- 172.21.15.134:0/1235862261 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300c9640 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009870 tx=0x7fdd2002b860 comp rx=0 tx=0).stop 2024-03-09T08:51:56.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.191+0000 7fdd2fe00640 1 -- 172.21.15.134:0/1235862261 shutdown_connections 2024-03-09T08:51:56.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.191+0000 7fdd2fe00640 1 --2- 172.21.15.134:0/1235862261 >> v2:172.21.15.67:3300/0 conn(0x7fdd30152f20 0x7fdd30153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.191+0000 7fdd2fe00640 1 --2- 172.21.15.134:0/1235862261 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300c9640 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.187 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.191+0000 7fdd2fe00640 1 -- 172.21.15.134:0/1235862261 >> 172.21.15.134:0/1235862261 conn(0x7fdd300ad060 msgr2=0x7fdd300ad440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:56.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.191+0000 7fdd2fe00640 1 -- 172.21.15.134:0/1235862261 shutdown_connections 2024-03-09T08:51:56.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.191+0000 7fdd2fe00640 1 -- 172.21.15.134:0/1235862261 wait complete. 2024-03-09T08:51:56.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.193+0000 7fdd2fe00640 1 Processor -- start 2024-03-09T08:51:56.188 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.193+0000 7fdd2fe00640 1 -- start start 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd300ca2c0 0x7fdd30159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd30152f20 0x7fdd30151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd300c9b80 con 0x7fdd300ca2c0 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd301568d0 con 0x7fdd30152f20 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd300cae60 con 0x7fdd300c8190 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2ea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd30152f20 0x7fdd30151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2ea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:36230/0 (socket says 172.21.15.134:36230) 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2ea00640 1 -- 172.21.15.134:0/2959136518 learned_addr learned my addr 172.21.15.134:0/2959136518 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2e000640 1 --2- 172.21.15.134:0/2959136518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd300ca2c0 0x7fdd30159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.191 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2ea00640 1 -- 172.21.15.134:0/2959136518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd30152f20 msgr2=0x7fdd30151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.194+0000 7fdd2ea00640 1 --2- 172.21.15.134:0/2959136518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd30152f20 0x7fdd30151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.195+0000 7fdd2ea00640 1 -- 172.21.15.134:0/2959136518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd300ca2c0 msgr2=0x7fdd30159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.195+0000 7fdd2ea00640 1 --2- 172.21.15.134:0/2959136518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd300ca2c0 0x7fdd30159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.195+0000 7fdd2ea00640 1 -- 172.21.15.134:0/2959136518 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd300c6150 con 0x7fdd300c8190 2024-03-09T08:51:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.195+0000 7fdd2f400640 1 --2- 172.21.15.134:0/2959136518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd30152f20 0x7fdd30151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:51:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.195+0000 7fdd2ea00640 1 --2- 172.21.15.134:0/2959136518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300aaed0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009870 tx=0x7fdd200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.195+0000 7fdd26a00640 1 -- 172.21.15.134:0/2959136518 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20032860 con 0x7fdd300c8190 2024-03-09T08:51:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.195+0000 7fdd26a00640 1 -- 172.21.15.134:0/2959136518 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd20032a00 con 0x7fdd300c8190 2024-03-09T08:51:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.195+0000 7fdd26a00640 1 -- 172.21.15.134:0/2959136518 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20032df0 con 0x7fdd300c8190 2024-03-09T08:51:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.195+0000 7fdd2fe00640 1 -- 172.21.15.134:0/2959136518 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd30157740 con 0x7fdd300c8190 2024-03-09T08:51:56.192 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.195+0000 7fdd2fe00640 1 -- 172.21.15.134:0/2959136518 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd30151210 con 0x7fdd300c8190 2024-03-09T08:51:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.196+0000 7fdd26a00640 1 -- 172.21.15.134:0/2959136518 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdd20045780 con 0x7fdd300c8190 2024-03-09T08:51:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.196+0000 7fdd26a00640 1 -- 172.21.15.134:0/2959136518 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fdd2004d3f0 con 0x7fdd300c8190 2024-03-09T08:51:56.193 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.197+0000 7fdd2fe00640 1 -- 172.21.15.134:0/2959136518 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd30150d50 con 0x7fdd300c8190 2024-03-09T08:51:56.194 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.199+0000 7fdd26a00640 1 -- 172.21.15.134:0/2959136518 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdd30150d50 con 0x7fdd300c8190 2024-03-09T08:51:56.197 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:56.202+0000 7f87aa246040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-09T08:51:56.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.296+0000 7fdd2fe00640 1 -- 172.21.15.134:0/2959136518 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "db5b5ea7-6a92-4d06-b2f7-dcece773d296", "id": 5} v 0) v1 -- 0x7fdd300689d0 con 0x7fdd300c8190 2024-03-09T08:51:56.300 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.305+0000 7fdd26a00640 1 -- 172.21.15.134:0/2959136518 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "db5b5ea7-6a92-4d06-b2f7-dcece773d296", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdd300689d0 con 0x7fdd300c8190 2024-03-09T08:51:56.300 INFO:teuthology.orchestra.run.smithi134.stdout:5 2024-03-09T08:51:56.301 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.306+0000 7fdd2fe00640 1 -- 172.21.15.134:0/2959136518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 msgr2=0x7fdd300aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.306+0000 7fdd2fe00640 1 --2- 172.21.15.134:0/2959136518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300aaed0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009870 tx=0x7fdd200046c0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.306+0000 7fdd2fe00640 1 -- 172.21.15.134:0/2959136518 shutdown_connections 2024-03-09T08:51:56.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.306+0000 7fdd2fe00640 1 --2- 172.21.15.134:0/2959136518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd30152f20 0x7fdd30151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.306+0000 7fdd2fe00640 1 --2- 172.21.15.134:0/2959136518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd300ca2c0 0x7fdd30159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.306+0000 7fdd2fe00640 1 --2- 172.21.15.134:0/2959136518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd300c8190 0x7fdd300aaed0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.306+0000 7fdd2fe00640 1 -- 172.21.15.134:0/2959136518 >> 172.21.15.134:0/2959136518 conn(0x7fdd300ad060 msgr2=0x7fdd3006c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:56.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.306+0000 7fdd2fe00640 1 -- 172.21.15.134:0/2959136518 shutdown_connections 2024-03-09T08:51:56.302 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.306+0000 7fdd2fe00640 1 -- 172.21.15.134:0/2959136518 wait complete. 2024-03-09T08:51:56.318 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 2af12c64-6476-4e86-a977-2420ad5a4679 6 2024-03-09T08:51:56.328 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:56.333+0000 7fef4ca42040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-09T08:51:56.341 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:56.346+0000 7f87aa246040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-09T08:51:56.386 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:56.391+0000 7fef4ca42040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-09T08:51:56.397 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:56.402+0000 7f87aa246040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-09T08:51:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.511+0000 7f6cfd000640 1 Processor -- start 2024-03-09T08:51:56.507 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.512+0000 7f6cfd000640 1 -- start start 2024-03-09T08:51:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.513+0000 7f6cfd000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6cf806b040 0x7f6cf80c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.513+0000 7f6cfd000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80c9a30 con 0x7f6cf80c9490 2024-03-09T08:51:56.508 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.513+0000 7f6cfd000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80c9d10 con 0x7f6cf80c5f10 2024-03-09T08:51:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.513+0000 7f6cfd000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf80ca340 con 0x7f6cf806b040 2024-03-09T08:51:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.513+0000 7f6cf6a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6cf806b040 0x7f6cf80c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.513+0000 7f6cf6a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6cf806b040 0x7f6cf80c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39050/0 (socket says 172.21.15.134:39050) 2024-03-09T08:51:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.513+0000 7f6cf6a00640 1 -- 172.21.15.134:0/2271917629 learned_addr learned my addr 172.21.15.134:0/2271917629 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.513+0000 7f6cf6a00640 1 -- 172.21.15.134:0/2271917629 >> v1:172.21.15.134:6789/0 conn(0x7f6cf80c5f10 legacy=0x7f6cf81567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.513+0000 7f6cf6a00640 1 -- 172.21.15.134:0/2271917629 >> v1:172.21.15.67:6789/0 conn(0x7f6cf80c9490 legacy=0x7f6cf80cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:56.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.513+0000 7f6cf6a00640 1 -- 172.21.15.134:0/2271917629 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80ca620 con 0x7f6cf806b040 2024-03-09T08:51:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.514+0000 7f6cf6a00640 1 --2- 172.21.15.134:0/2271917629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf806b040 0x7f6cf80c62f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009870 tx=0x7f6ce402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5d5cafa36ae47be server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.515+0000 7f6cf6000640 1 -- 172.21.15.134:0/2271917629 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4035020 con 0x7f6cf806b040 2024-03-09T08:51:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.515+0000 7f6cf6000640 1 -- 172.21.15.134:0/2271917629 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce4031870 con 0x7f6cf806b040 2024-03-09T08:51:56.510 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.515+0000 7f6cf6000640 1 -- 172.21.15.134:0/2271917629 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4031c40 con 0x7f6cf806b040 2024-03-09T08:51:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.515+0000 7f6cfd000640 1 -- 172.21.15.134:0/2271917629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf806b040 msgr2=0x7f6cf80c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.515+0000 7f6cfd000640 1 --2- 172.21.15.134:0/2271917629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf806b040 0x7f6cf80c62f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009870 tx=0x7f6ce402b910 comp rx=0 tx=0).stop 2024-03-09T08:51:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.515+0000 7f6cfd000640 1 -- 172.21.15.134:0/2271917629 shutdown_connections 2024-03-09T08:51:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.515+0000 7f6cfd000640 1 --2- 172.21.15.134:0/2271917629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf806b040 0x7f6cf80c62f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.515+0000 7f6cfd000640 1 -- 172.21.15.134:0/2271917629 >> 172.21.15.134:0/2271917629 conn(0x7f6cf80cb590 msgr2=0x7f6cf80cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.516+0000 7f6cfd000640 1 -- 172.21.15.134:0/2271917629 shutdown_connections 2024-03-09T08:51:56.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.516+0000 7f6cfd000640 1 -- 172.21.15.134:0/2271917629 wait complete. 2024-03-09T08:51:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.517+0000 7f6cfd000640 1 Processor -- start 2024-03-09T08:51:56.512 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.517+0000 7f6cfd000640 1 -- start start 2024-03-09T08:51:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cfd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf806b040 0x7f6cf8147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cfd000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80c5f10 0x7f6cf81587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cfd000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c9490 0x7f6cf8157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cfd000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80cb3d0 con 0x7f6cf806b040 2024-03-09T08:51:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cfd000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80cb250 con 0x7f6cf80c9490 2024-03-09T08:51:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cfd000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf8153620 con 0x7f6cf80c5f10 2024-03-09T08:51:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cf7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c9490 0x7f6cf8157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cf6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80c5f10 0x7f6cf81587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cf6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80c5f10 0x7f6cf81587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:36236/0 (socket says 172.21.15.134:36236) 2024-03-09T08:51:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cf6a00640 1 -- 172.21.15.134:0/3948850993 learned_addr learned my addr 172.21.15.134:0/3948850993 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:56.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cf7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c9490 0x7f6cf8157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39062/0 (socket says 172.21.15.134:39062) 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.518+0000 7f6cf7400640 1 --2- 172.21.15.134:0/3948850993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf806b040 0x7f6cf8147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cf7e00640 1 -- 172.21.15.134:0/3948850993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80c5f10 msgr2=0x7f6cf81587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cf7e00640 1 --2- 172.21.15.134:0/3948850993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80c5f10 0x7f6cf81587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cf7e00640 1 -- 172.21.15.134:0/3948850993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf806b040 msgr2=0x7f6cf8147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cf7e00640 1 --2- 172.21.15.134:0/3948850993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf806b040 0x7f6cf8147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cf7e00640 1 -- 172.21.15.134:0/3948850993 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf8147940 con 0x7f6cf80c9490 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cf6a00640 1 --2- 172.21.15.134:0/3948850993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80c5f10 0x7f6cf81587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cf7400640 1 --2- 172.21.15.134:0/3948850993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf806b040 0x7f6cf8147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cf7e00640 1 --2- 172.21.15.134:0/3948850993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c9490 0x7f6cf8157140 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000be00 tx=0x7f6cf000bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cef400640 1 -- 172.21.15.134:0/3948850993 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0003970 con 0x7f6cf80c9490 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cef400640 1 -- 172.21.15.134:0/3948850993 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf0003b10 con 0x7f6cf80c9490 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cef400640 1 -- 172.21.15.134:0/3948850993 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0004020 con 0x7f6cf80c9490 2024-03-09T08:51:56.516 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cfd000640 1 -- 172.21.15.134:0/3948850993 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf8147160 con 0x7f6cf80c9490 2024-03-09T08:51:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.519+0000 7f6cfd000640 1 -- 172.21.15.134:0/3948850993 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf81ebf20 con 0x7f6cf80c9490 2024-03-09T08:51:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.520+0000 7f6cef400640 1 -- 172.21.15.134:0/3948850993 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6cf00041c0 con 0x7f6cf80c9490 2024-03-09T08:51:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.520+0000 7f6cef400640 1 -- 172.21.15.134:0/3948850993 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f6cf0014440 con 0x7f6cf80c9490 2024-03-09T08:51:56.517 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.520+0000 7f6cfd000640 1 -- 172.21.15.134:0/3948850993 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf804f3c0 con 0x7f6cf80c9490 2024-03-09T08:51:56.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.523+0000 7f6cef400640 1 -- 172.21.15.134:0/3948850993 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6cf8147940 con 0x7f6cf80c9490 2024-03-09T08:51:56.522 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:56.527+0000 7fef4ca42040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-09T08:51:56.532 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:56.537+0000 7f87aa246040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-09T08:51:56.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.599+0000 7f6cfd000640 1 -- 172.21.15.134:0/3948850993 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2af12c64-6476-4e86-a977-2420ad5a4679", "id": 6} v 0) v1 -- 0x7f6cf80cc260 con 0x7f6cf80c9490 2024-03-09T08:51:56.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.606+0000 7f6cef400640 1 -- 172.21.15.134:0/3948850993 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2af12c64-6476-4e86-a977-2420ad5a4679", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6cf00044b0 con 0x7f6cf80c9490 2024-03-09T08:51:56.602 INFO:teuthology.orchestra.run.smithi134.stdout:6 2024-03-09T08:51:56.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.607+0000 7f6cfd000640 1 -- 172.21.15.134:0/3948850993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c9490 msgr2=0x7f6cf8157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.607+0000 7f6cfd000640 1 --2- 172.21.15.134:0/3948850993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c9490 0x7f6cf8157140 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000be00 tx=0x7f6cf000bef0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.607+0000 7f6cfd000640 1 -- 172.21.15.134:0/3948850993 shutdown_connections 2024-03-09T08:51:56.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.607+0000 7f6cfd000640 1 --2- 172.21.15.134:0/3948850993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c9490 0x7f6cf8157140 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.607+0000 7f6cfd000640 1 --2- 172.21.15.134:0/3948850993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80c5f10 0x7f6cf81587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.607+0000 7f6cfd000640 1 --2- 172.21.15.134:0/3948850993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf806b040 0x7f6cf8147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.607+0000 7f6cfd000640 1 -- 172.21.15.134:0/3948850993 >> 172.21.15.134:0/3948850993 conn(0x7f6cf80cb590 msgr2=0x7f6cf8151220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:56.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.607+0000 7f6cfd000640 1 -- 172.21.15.134:0/3948850993 shutdown_connections 2024-03-09T08:51:56.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.607+0000 7f6cfd000640 1 -- 172.21.15.134:0/3948850993 wait complete. 2024-03-09T08:51:56.619 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 14fb0a99-8675-41f7-b02d-adfcfd95f841 7 2024-03-09T08:51:56.640 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:56.644+0000 7fef4ca42040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-09T08:51:56.649 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:56.654+0000 7f87aa246040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-09T08:51:56.825 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:56.829+0000 7fef4ca42040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-09T08:51:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.832+0000 7f521de00640 1 Processor -- start 2024-03-09T08:51:56.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.832+0000 7f521de00640 1 -- start start 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.834+0000 7f521de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52180bdb80 0x7f52180c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.834+0000 7f521de00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f52180bef70 0x7f52180c3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.834+0000 7f521de00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52181503b0 con 0x7f521814ffd0 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.834+0000 7f521de00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5218153820 con 0x7f52180bef70 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.834+0000 7f521de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5218152c10 con 0x7f52180bdb80 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.835+0000 7f521d400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f52180bef70 0x7f52180c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.835+0000 7f5217e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52180bdb80 0x7f52180c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.835+0000 7f521d400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f52180bef70 0x7f52180c3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39072/0 (socket says 172.21.15.134:39072) 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.835+0000 7f521d400640 1 -- 172.21.15.134:0/556860480 learned_addr learned my addr 172.21.15.134:0/556860480 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.835+0000 7f5217e00640 1 -- 172.21.15.134:0/556860480 >> v2:172.21.15.134:3300/0 conn(0x7f52180bef70 msgr2=0x7f52180c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.835+0000 7f5217e00640 1 --2- 172.21.15.134:0/556860480 >> v2:172.21.15.134:3300/0 conn(0x7f52180bef70 0x7f52180c3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.835+0000 7f5217e00640 1 -- 172.21.15.134:0/556860480 >> v1:172.21.15.67:6789/0 conn(0x7f521814ffd0 legacy=0x7f52180be0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:56.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.835+0000 7f5217e00640 1 -- 172.21.15.134:0/556860480 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5218152ef0 con 0x7f52180bdb80 2024-03-09T08:51:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.835+0000 7f521d400640 1 --2- 172.21.15.134:0/556860480 >> v2:172.21.15.134:3300/0 conn(0x7f52180bef70 0x7f52180c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:51:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.836+0000 7f5217e00640 1 --2- 172.21.15.134:0/556860480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52180bdb80 0x7f52180c2e50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f52040097a0 tx=0x7f520402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b368a6826abc98 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.836+0000 7f5217400640 1 -- 172.21.15.134:0/556860480 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204035020 con 0x7f52180bdb80 2024-03-09T08:51:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.837+0000 7f5217400640 1 -- 172.21.15.134:0/556860480 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5204031810 con 0x7f52180bdb80 2024-03-09T08:51:56.832 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.837+0000 7f5217400640 1 -- 172.21.15.134:0/556860480 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204031be0 con 0x7f52180bdb80 2024-03-09T08:51:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.837+0000 7f521de00640 1 -- 172.21.15.134:0/556860480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52180bdb80 msgr2=0x7f52180c2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.837+0000 7f521de00640 1 --2- 172.21.15.134:0/556860480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52180bdb80 0x7f52180c2e50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f52040097a0 tx=0x7f520402b8b0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.837+0000 7f521de00640 1 -- 172.21.15.134:0/556860480 shutdown_connections 2024-03-09T08:51:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.837+0000 7f521de00640 1 --2- 172.21.15.134:0/556860480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52180bdb80 0x7f52180c2e50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.837+0000 7f521de00640 1 --2- 172.21.15.134:0/556860480 >> v2:172.21.15.134:3300/0 conn(0x7f52180bef70 0x7f52180c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.833 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.837+0000 7f521de00640 1 -- 172.21.15.134:0/556860480 >> 172.21.15.134:0/556860480 conn(0x7f52180c82a0 msgr2=0x7f52180c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.837+0000 7f521de00640 1 -- 172.21.15.134:0/556860480 shutdown_connections 2024-03-09T08:51:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.837+0000 7f521de00640 1 -- 172.21.15.134:0/556860480 wait complete. 2024-03-09T08:51:56.834 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:56.839+0000 7f87aa246040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-09T08:51:56.834 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.839+0000 7f521de00640 1 Processor -- start 2024-03-09T08:51:56.835 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.839+0000 7f521de00640 1 -- start start 2024-03-09T08:51:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.840+0000 7f521de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52180bdb80 0x7f52180c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.840+0000 7f521de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52180bef70 0x7f52181522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.840+0000 7f521de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f521814ffd0 0x7f52181efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.840+0000 7f521de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521806d440 con 0x7f52180bef70 2024-03-09T08:51:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.840+0000 7f521de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52181534b0 con 0x7f52180bdb80 2024-03-09T08:51:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.840+0000 7f521de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52180be7e0 con 0x7f521814ffd0 2024-03-09T08:51:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.840+0000 7f521d400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f521814ffd0 0x7f52181efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.840+0000 7f521ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52180bdb80 0x7f52180c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.840+0000 7f521d400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f521814ffd0 0x7f52181efaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:36256/0 (socket says 172.21.15.134:36256) 2024-03-09T08:51:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.840+0000 7f521d400640 1 -- 172.21.15.134:0/2450847699 learned_addr learned my addr 172.21.15.134:0/2450847699 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-03-09T08:51:56.836 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.840+0000 7f5217e00640 1 --2- 172.21.15.134:0/2450847699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52180bef70 0x7f52181522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f521ca00640 1 -- 172.21.15.134:0/2450847699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f521814ffd0 msgr2=0x7f52181efaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f521ca00640 1 --2- 172.21.15.134:0/2450847699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f521814ffd0 0x7f52181efaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f521ca00640 1 -- 172.21.15.134:0/2450847699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52180bef70 msgr2=0x7f52181522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f521ca00640 1 --2- 172.21.15.134:0/2450847699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52180bef70 0x7f52181522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f521ca00640 1 -- 172.21.15.134:0/2450847699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5218155db0 con 0x7f52180bdb80 2024-03-09T08:51:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f521d400640 1 --2- 172.21.15.134:0/2450847699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f521814ffd0 0x7f52181efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:51:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f521ca00640 1 --2- 172.21.15.134:0/2450847699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52180bdb80 0x7f52180c5e70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f520c00b3e0 tx=0x7f520c00b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f5214c00640 1 -- 172.21.15.134:0/2450847699 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c015030 con 0x7f52180bdb80 2024-03-09T08:51:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f5214c00640 1 -- 172.21.15.134:0/2450847699 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c004020 con 0x7f52180bdb80 2024-03-09T08:51:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f5214c00640 1 -- 172.21.15.134:0/2450847699 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c004410 con 0x7f52180bdb80 2024-03-09T08:51:56.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f521de00640 1 -- 172.21.15.134:0/2450847699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5218154630 con 0x7f52180bdb80 2024-03-09T08:51:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.841+0000 7f521de00640 1 -- 172.21.15.134:0/2450847699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52181ef2a0 con 0x7f52180bdb80 2024-03-09T08:51:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.842+0000 7f5214c00640 1 -- 172.21.15.134:0/2450847699 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f520c018070 con 0x7f52180bdb80 2024-03-09T08:51:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.842+0000 7f5214c00640 1 -- 172.21.15.134:0/2450847699 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f520c011630 con 0x7f52180bdb80 2024-03-09T08:51:56.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.842+0000 7f521de00640 1 -- 172.21.15.134:0/2450847699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521804f3c0 con 0x7f52180bdb80 2024-03-09T08:51:56.840 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.845+0000 7f5214c00640 1 -- 172.21.15.134:0/2450847699 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f520c0039b0 con 0x7f52180bdb80 2024-03-09T08:51:56.879 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:56.884+0000 7fef4ca42040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-09T08:51:56.886 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:56.891+0000 7f87aa246040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-09T08:51:56.916 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.921+0000 7f521de00640 1 -- 172.21.15.134:0/2450847699 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "14fb0a99-8675-41f7-b02d-adfcfd95f841", "id": 7} v 0) v1 -- 0x7f52180be5c0 con 0x7f52180bdb80 2024-03-09T08:51:56.925 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.930+0000 7f5214c00640 1 -- 172.21.15.134:0/2450847699 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "14fb0a99-8675-41f7-b02d-adfcfd95f841", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f520c003b90 con 0x7f52180bdb80 2024-03-09T08:51:56.925 INFO:teuthology.orchestra.run.smithi134.stdout:7 2024-03-09T08:51:56.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.931+0000 7f521de00640 1 -- 172.21.15.134:0/2450847699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52180bdb80 msgr2=0x7f52180c5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:56.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.931+0000 7f521de00640 1 --2- 172.21.15.134:0/2450847699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52180bdb80 0x7f52180c5e70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f520c00b3e0 tx=0x7f520c00b7e0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.931+0000 7f521de00640 1 -- 172.21.15.134:0/2450847699 shutdown_connections 2024-03-09T08:51:56.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.931+0000 7f521de00640 1 --2- 172.21.15.134:0/2450847699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f521814ffd0 0x7f52181efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.931+0000 7f521de00640 1 --2- 172.21.15.134:0/2450847699 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52180bef70 0x7f52181522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.931+0000 7f521de00640 1 --2- 172.21.15.134:0/2450847699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52180bdb80 0x7f52180c5e70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:56.927 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.931+0000 7f521de00640 1 -- 172.21.15.134:0/2450847699 >> 172.21.15.134:0/2450847699 conn(0x7f52180c82a0 msgr2=0x7f521814d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:56.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.931+0000 7f521de00640 1 -- 172.21.15.134:0/2450847699 shutdown_connections 2024-03-09T08:51:56.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-03-09T08:51:56.931+0000 7f521de00640 1 -- 172.21.15.134:0/2450847699 wait complete. 2024-03-09T08:51:56.943 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-09T08:51:56.943 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-09T08:51:56.951 INFO:tasks.ceph.osd.0:Started 2024-03-09T08:51:56.951 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-09T08:51:56.951 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-09T08:51:56.958 INFO:tasks.ceph.osd.1:Started 2024-03-09T08:51:56.958 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-09T08:51:56.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-09T08:51:56.964 INFO:tasks.ceph.osd.2:Started 2024-03-09T08:51:56.964 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-09T08:51:56.964 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-09T08:51:56.969 INFO:tasks.ceph.osd.3:Started 2024-03-09T08:51:56.969 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-09T08:51:56.969 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-09T08:51:56.974 INFO:tasks.ceph.osd.4:Started 2024-03-09T08:51:56.974 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-09T08:51:56.974 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-09T08:51:56.978 INFO:tasks.ceph.osd.5:Started 2024-03-09T08:51:56.978 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-09T08:51:56.978 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-09T08:51:56.982 INFO:tasks.ceph.osd.6:Started 2024-03-09T08:51:56.983 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-09T08:51:56.983 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-09T08:51:56.987 INFO:tasks.ceph.osd.7:Started 2024-03-09T08:51:56.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:51:57.083 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:57.087+0000 7fef4ca42040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-09T08:51:57.127 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:57.132+0000 7f87aa246040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-09T08:51:57.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.221+0000 7fe110a00640 1 Processor -- start 2024-03-09T08:51:57.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.221+0000 7fe110a00640 1 -- start start 2024-03-09T08:51:57.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.222+0000 7fe110a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe10c06b040 0x7fe10c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:57.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.222+0000 7fe110a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c0c6a40 con 0x7fe10c0c5f50 2024-03-09T08:51:57.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.222+0000 7fe110a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10c06b5b0 con 0x7fe10c06c650 2024-03-09T08:51:57.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.222+0000 7fe110a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10c0c9e00 con 0x7fe10c06b040 2024-03-09T08:51:57.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.222+0000 7fe109e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe10c0c5f50 0x7fe10c0c96f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39516/0 (socket says 172.21.15.67:39516) 2024-03-09T08:51:57.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.222+0000 7fe109e00640 1 -- 172.21.15.67:0/1478603014 learned_addr learned my addr 172.21.15.67:0/1478603014 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:51:57.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.223+0000 7fe109400640 1 --2- 172.21.15.67:0/1478603014 >> v2:172.21.15.134:3300/0 conn(0x7fe10c06b040 0x7fe10c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:57.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.223+0000 7fe108a00640 1 -- 172.21.15.67:0/1478603014 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837752146 0 0) 0x7fe10c0c6a40 con 0x7fe10c0c5f50 2024-03-09T08:51:57.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.223+0000 7fe108a00640 1 -- 172.21.15.67:0/1478603014 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f0003520 con 0x7fe10c0c5f50 2024-03-09T08:51:57.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.223+0000 7fe109400640 1 -- 172.21.15.67:0/1478603014 >> v1:172.21.15.134:6789/0 conn(0x7fe10c06c650 legacy=0x7fe10c0c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:57.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.223+0000 7fe109400640 1 -- 172.21.15.67:0/1478603014 >> v1:172.21.15.67:6789/0 conn(0x7fe10c0c5f50 legacy=0x7fe10c0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:57.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.223+0000 7fe109400640 1 -- 172.21.15.67:0/1478603014 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c06b890 con 0x7fe10c06b040 2024-03-09T08:51:57.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.225+0000 7fe109400640 1 --2- 172.21.15.67:0/1478603014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10c06b040 0x7fe10c155e20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80097a0 tx=0x7fe0f802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44e2d9257eb24299 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:57.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.225+0000 7fe108a00640 1 -- 172.21.15.67:0/1478603014 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8034070 con 0x7fe10c06b040 2024-03-09T08:51:57.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.225+0000 7fe108a00640 1 -- 172.21.15.67:0/1478603014 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8030610 con 0x7fe10c06b040 2024-03-09T08:51:57.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.226+0000 7fe108a00640 1 -- 172.21.15.67:0/1478603014 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80309c0 con 0x7fe10c06b040 2024-03-09T08:51:57.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.226+0000 7fe110a00640 1 -- 172.21.15.67:0/1478603014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10c06b040 msgr2=0x7fe10c155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:57.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.226+0000 7fe110a00640 1 --2- 172.21.15.67:0/1478603014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10c06b040 0x7fe10c155e20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80097a0 tx=0x7fe0f802b960 comp rx=0 tx=0).stop 2024-03-09T08:51:57.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.226+0000 7fe110a00640 1 -- 172.21.15.67:0/1478603014 shutdown_connections 2024-03-09T08:51:57.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.226+0000 7fe110a00640 1 --2- 172.21.15.67:0/1478603014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10c06b040 0x7fe10c155e20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:57.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.226+0000 7fe110a00640 1 -- 172.21.15.67:0/1478603014 >> 172.21.15.67:0/1478603014 conn(0x7fe10c0c8050 msgr2=0x7fe10c0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:57.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.226+0000 7fe110a00640 1 -- 172.21.15.67:0/1478603014 shutdown_connections 2024-03-09T08:51:57.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.226+0000 7fe110a00640 1 -- 172.21.15.67:0/1478603014 wait complete. 2024-03-09T08:51:57.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.228+0000 7fe110a00640 1 Processor -- start 2024-03-09T08:51:57.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.228+0000 7fe110a00640 1 -- start start 2024-03-09T08:51:57.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe110a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10c06b040 0x7fe10c1eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe110a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe10c06c650 0x7fe10c0c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe110a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe10c0c5f50 0x7fe10c0c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe110a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c06be50 con 0x7fe10c06c650 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe110a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c06cac0 con 0x7fe10c06b040 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe110a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c06c130 con 0x7fe10c0c5f50 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe109400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe10c06c650 0x7fe10c0c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe109e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10c06b040 0x7fe10c1eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe109400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe10c06c650 0x7fe10c0c4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50816/0 (socket says 172.21.15.67:50816) 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe109e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10c06b040 0x7fe10c1eb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:34832/0 (socket says 172.21.15.67:34832) 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe109400640 1 -- 172.21.15.67:0/1127861120 learned_addr learned my addr 172.21.15.67:0/1127861120 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe10a800640 1 --2- 172.21.15.67:0/1127861120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe10c0c5f50 0x7fe10c0c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe109e00640 1 -- 172.21.15.67:0/1127861120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe10c0c5f50 msgr2=0x7fe10c0c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:57.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe109e00640 1 --2- 172.21.15.67:0/1127861120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe10c0c5f50 0x7fe10c0c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:57.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe109e00640 1 -- 172.21.15.67:0/1127861120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe10c06c650 msgr2=0x7fe10c0c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:57.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe109e00640 1 --2- 172.21.15.67:0/1127861120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe10c06c650 0x7fe10c0c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:57.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.229+0000 7fe109e00640 1 -- 172.21.15.67:0/1127861120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c0c5260 con 0x7fe10c06b040 2024-03-09T08:51:57.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.230+0000 7fe109e00640 1 --2- 172.21.15.67:0/1127861120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10c06b040 0x7fe10c1eb7c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c3c0 tx=0x7fe0fc001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.230+0000 7fe102000640 1 -- 172.21.15.67:0/1127861120 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc017070 con 0x7fe10c06b040 2024-03-09T08:51:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.230+0000 7fe102000640 1 -- 172.21.15.67:0/1127861120 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc005ce0 con 0x7fe10c06b040 2024-03-09T08:51:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.230+0000 7fe102000640 1 -- 172.21.15.67:0/1127861120 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0072d0 con 0x7fe10c06b040 2024-03-09T08:51:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.230+0000 7fe110a00640 1 -- 172.21.15.67:0/1127861120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10c157ce0 con 0x7fe10c06b040 2024-03-09T08:51:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.230+0000 7fe110a00640 1 -- 172.21.15.67:0/1127861120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10c0c2940 con 0x7fe10c06b040 2024-03-09T08:51:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.230+0000 7fe102000640 1 -- 172.21.15.67:0/1127861120 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe0fc01a070 con 0x7fe10c06b040 2024-03-09T08:51:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.231+0000 7fe102000640 1 -- 172.21.15.67:0/1127861120 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe0fc01cc60 con 0x7fe10c06b040 2024-03-09T08:51:57.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.231+0000 7fe110a00640 1 -- 172.21.15.67:0/1127861120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10c075a00 con 0x7fe10c06b040 2024-03-09T08:51:57.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.236+0000 7fe102000640 1 -- 172.21.15.67:0/1127861120 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe0fc007470 con 0x7fe10c06b040 2024-03-09T08:51:57.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.302+0000 7fe110a00640 1 -- 172.21.15.67:0/1127861120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe10c06cca0 con 0x7fe10c06b040 2024-03-09T08:51:57.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.303+0000 7fe102000640 1 -- 172.21.15.67:0/1127861120 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fe0fc007650 con 0x7fe10c06b040 2024-03-09T08:51:57.298 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:51:57.298 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":10,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:51:56.924885+0000","last_up_change":"0.000000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:51:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.304+0000 7fe110a00640 1 -- 172.21.15.67:0/1127861120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10c06b040 msgr2=0x7fe10c1eb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.304+0000 7fe110a00640 1 --2- 172.21.15.67:0/1127861120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10c06b040 0x7fe10c1eb7c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c3c0 tx=0x7fe0fc001e70 comp rx=0 tx=0).stop 2024-03-09T08:51:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.304+0000 7fe110a00640 1 -- 172.21.15.67:0/1127861120 shutdown_connections 2024-03-09T08:51:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.304+0000 7fe110a00640 1 --2- 172.21.15.67:0/1127861120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe10c0c5f50 0x7fe10c0c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.304+0000 7fe110a00640 1 --2- 172.21.15.67:0/1127861120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe10c06c650 0x7fe10c0c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.304+0000 7fe110a00640 1 --2- 172.21.15.67:0/1127861120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10c06b040 0x7fe10c1eb7c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.304+0000 7fe110a00640 1 -- 172.21.15.67:0/1127861120 >> 172.21.15.67:0/1127861120 conn(0x7fe10c0c8050 msgr2=0x7fe10c153890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.305+0000 7fe110a00640 1 -- 172.21.15.67:0/1127861120 shutdown_connections 2024-03-09T08:51:57.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:51:57.305+0000 7fe110a00640 1 -- 172.21.15.67:0/1127861120 wait complete. 2024-03-09T08:51:57.315 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-09T08:51:57.316 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-09T08:51:57.658 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-03-09T08:51:57.662+0000 7f19a72b3640 -1 Falling back to public interface 2024-03-09T08:51:57.663 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-03-09T08:51:57.668+0000 7f7690019640 -1 Falling back to public interface 2024-03-09T08:51:57.665 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-03-09T08:51:57.670+0000 7f13a8e11640 -1 Falling back to public interface 2024-03-09T08:51:57.665 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-03-09T08:51:57.670+0000 7f1e74806640 -1 Falling back to public interface 2024-03-09T08:51:57.670 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-03-09T08:51:57.675+0000 7f2e0ccce640 -1 Falling back to public interface 2024-03-09T08:51:57.683 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-03-09T08:51:57.688+0000 7f9dcc5ff640 -1 Falling back to public interface 2024-03-09T08:51:57.695 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-03-09T08:51:57.700+0000 7f17d81f8640 -1 Falling back to public interface 2024-03-09T08:51:57.711 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-03-09T08:51:57.716+0000 7fe81400e640 -1 Falling back to public interface 2024-03-09T08:51:58.122 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-09T08:51:58.132 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:58.137+0000 7fef4ca42040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-09T08:51:58.285 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.290+0000 7f2347a00640 1 Processor -- start 2024-03-09T08:51:58.285 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.290+0000 7f2347a00640 1 -- start start 2024-03-09T08:51:58.287 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.291+0000 7f2347a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f23401534e0 0x7f23400c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:58.287 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.291+0000 7f2347a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f234006ca00 0x7f23400c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:58.287 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.291+0000 7f2347a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23401538c0 con 0x7f23400c80f0 2024-03-09T08:51:58.287 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.291+0000 7f2347a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23400c9b20 con 0x7f234006ca00 2024-03-09T08:51:58.287 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.291+0000 7f2347a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23400c8660 con 0x7f23401534e0 2024-03-09T08:51:58.287 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.292+0000 7f2345a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f234006ca00 0x7f23400c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:58.287 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.292+0000 7f233fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f23400c80f0 0x7f23401568e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39532/0 (socket says 172.21.15.67:39532) 2024-03-09T08:51:58.287 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.292+0000 7f233fe00640 1 -- 172.21.15.67:0/2773713220 learned_addr learned my addr 172.21.15.67:0/2773713220 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:51:58.287 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.292+0000 7f2345000640 1 --2- 172.21.15.67:0/2773713220 >> v2:172.21.15.134:3300/0 conn(0x7f23401534e0 0x7f23400c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:58.288 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.292+0000 7f2345a00640 1 -- 172.21.15.67:0/2773713220 >> v2:172.21.15.134:3300/0 conn(0x7f23401534e0 msgr2=0x7f23400c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:51:58.288 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.292+0000 7f2345a00640 1 --2- 172.21.15.67:0/2773713220 >> v2:172.21.15.134:3300/0 conn(0x7f23401534e0 0x7f23400c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:58.288 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.292+0000 7f2345a00640 1 -- 172.21.15.67:0/2773713220 >> v1:172.21.15.67:6789/0 conn(0x7f23400c80f0 legacy=0x7f23401568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:58.288 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.292+0000 7f2345a00640 1 -- 172.21.15.67:0/2773713220 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23400c8940 con 0x7f234006ca00 2024-03-09T08:51:58.288 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.292+0000 7f233f400640 1 -- 172.21.15.67:0/2773713220 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778241145 0 0) 0x7f232c001110 con 0x7f23400c80f0 2024-03-09T08:51:58.289 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.294+0000 7f2345a00640 1 --2- 172.21.15.67:0/2773713220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f234006ca00 0x7f23400c6060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2340074f80 tx=0x7f233802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc24a1c2c160ab9c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:58.289 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.294+0000 7f233f400640 1 -- 172.21.15.67:0/2773713220 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338039070 con 0x7f234006ca00 2024-03-09T08:51:58.289 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.294+0000 7f233f400640 1 -- 172.21.15.67:0/2773713220 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233802cb60 con 0x7f234006ca00 2024-03-09T08:51:58.289 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.294+0000 7f233f400640 1 -- 172.21.15.67:0/2773713220 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338034440 con 0x7f234006ca00 2024-03-09T08:51:58.290 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.295+0000 7f2347a00640 1 -- 172.21.15.67:0/2773713220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f234006ca00 msgr2=0x7f23400c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:58.290 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.295+0000 7f2347a00640 1 --2- 172.21.15.67:0/2773713220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f234006ca00 0x7f23400c6060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2340074f80 tx=0x7f233802bdb0 comp rx=0 tx=0).stop 2024-03-09T08:51:58.290 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.295+0000 7f2347a00640 1 -- 172.21.15.67:0/2773713220 shutdown_connections 2024-03-09T08:51:58.290 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.295+0000 7f2347a00640 1 --2- 172.21.15.67:0/2773713220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f234006ca00 0x7f23400c6060 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:58.290 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.295+0000 7f2347a00640 1 --2- 172.21.15.67:0/2773713220 >> v2:172.21.15.134:3300/0 conn(0x7f23401534e0 0x7f23400c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:58.290 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.295+0000 7f2347a00640 1 -- 172.21.15.67:0/2773713220 >> 172.21.15.67:0/2773713220 conn(0x7f23400ad2d0 msgr2=0x7f23400ad6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:58.290 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.295+0000 7f2347a00640 1 -- 172.21.15.67:0/2773713220 shutdown_connections 2024-03-09T08:51:58.291 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.295+0000 7f2347a00640 1 -- 172.21.15.67:0/2773713220 wait complete. 2024-03-09T08:51:58.295 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.296+0000 7f2347a00640 1 Processor -- start 2024-03-09T08:51:58.295 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.297+0000 7f2347a00640 1 -- start start 2024-03-09T08:51:58.295 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.297+0000 7f2347a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234006ca00 0x7f23401ef880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:58.295 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.297+0000 7f2347a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23400c80f0 0x7f234014ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:58.295 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.297+0000 7f2347a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23401534e0 0x7f23400c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:51:58.295 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.297+0000 7f2347a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234006ceb0 con 0x7f23401534e0 2024-03-09T08:51:58.295 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.297+0000 7f2347a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234006c4e0 con 0x7f23400c80f0 2024-03-09T08:51:58.296 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.297+0000 7f2347a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23401ef3f0 con 0x7f234006ca00 2024-03-09T08:51:58.296 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f2345a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23401534e0 0x7f23400c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:58.296 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f2345a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23401534e0 0x7f23400c28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50840/0 (socket says 172.21.15.67:50840) 2024-03-09T08:51:58.296 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f2345a00640 1 -- 172.21.15.67:0/3525478440 learned_addr learned my addr 172.21.15.67:0/3525478440 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:51:58.296 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f2345000640 1 --2- 172.21.15.67:0/3525478440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234006ca00 0x7f23401ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:58.296 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f233fe00640 1 --2- 172.21.15.67:0/3525478440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23400c80f0 0x7f234014ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:51:58.296 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f2345a00640 1 -- 172.21.15.67:0/3525478440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234006ca00 msgr2=0x7f23401ef880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:58.296 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f2345a00640 1 --2- 172.21.15.67:0/3525478440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234006ca00 0x7f23401ef880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:58.296 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f2345a00640 1 -- 172.21.15.67:0/3525478440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23400c80f0 msgr2=0x7f234014ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:58.296 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f2345a00640 1 --2- 172.21.15.67:0/3525478440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23400c80f0 0x7f234014ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:58.296 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f2345a00640 1 -- 172.21.15.67:0/3525478440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340153d90 con 0x7f23401534e0 2024-03-09T08:51:58.297 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f2345000640 1 --2- 172.21.15.67:0/3525478440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234006ca00 0x7f23401ef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:51:58.297 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f2345a00640 1 --2- 172.21.15.67:0/3525478440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23401534e0 0x7f23400c28d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2338009e80 tx=0x7f2338004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:51:58.297 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.298+0000 7f233cc00640 1 -- 172.21.15.67:0/3525478440 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338044020 con 0x7f23401534e0 2024-03-09T08:51:58.297 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.299+0000 7f233cc00640 1 -- 172.21.15.67:0/3525478440 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233803a040 con 0x7f23401534e0 2024-03-09T08:51:58.297 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.299+0000 7f233cc00640 1 -- 172.21.15.67:0/3525478440 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338039050 con 0x7f23401534e0 2024-03-09T08:51:58.297 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.299+0000 7f2347a00640 1 -- 172.21.15.67:0/3525478440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23400c5420 con 0x7f23401534e0 2024-03-09T08:51:58.297 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.299+0000 7f2347a00640 1 -- 172.21.15.67:0/3525478440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23401eeda0 con 0x7f23401534e0 2024-03-09T08:51:58.297 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.299+0000 7f233cc00640 1 -- 172.21.15.67:0/3525478440 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f233802cd10 con 0x7f23401534e0 2024-03-09T08:51:58.297 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.299+0000 7f233cc00640 1 -- 172.21.15.67:0/3525478440 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f2338004700 con 0x7f23401534e0 2024-03-09T08:51:58.297 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.300+0000 7f2347a00640 1 -- 172.21.15.67:0/3525478440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234004f3c0 con 0x7f23401534e0 2024-03-09T08:51:58.299 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.304+0000 7f233cc00640 1 -- 172.21.15.67:0/3525478440 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f233804e020 con 0x7f23401534e0 2024-03-09T08:51:58.357 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:58.362+0000 7f87aa246040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-09T08:51:58.365 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.370+0000 7f2347a00640 1 -- 172.21.15.67:0/3525478440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2340152b30 con 0x7f23401534e0 2024-03-09T08:51:58.366 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.371+0000 7f233cc00640 1 -- 172.21.15.67:0/3525478440 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f2338004030 con 0x7f23401534e0 2024-03-09T08:51:58.366 INFO:teuthology.misc.health.smithi067.stdout: 2024-03-09T08:51:58.366 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":10,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:51:56.924885+0000","last_up_change":"0.000000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:51:58.368 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.373+0000 7f2347a00640 1 -- 172.21.15.67:0/3525478440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23401534e0 msgr2=0x7f23400c28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:51:58.368 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.373+0000 7f2347a00640 1 --2- 172.21.15.67:0/3525478440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23401534e0 0x7f23400c28d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2338009e80 tx=0x7f2338004a40 comp rx=0 tx=0).stop 2024-03-09T08:51:58.368 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.373+0000 7f2347a00640 1 -- 172.21.15.67:0/3525478440 shutdown_connections 2024-03-09T08:51:58.368 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.373+0000 7f2347a00640 1 --2- 172.21.15.67:0/3525478440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23401534e0 0x7f23400c28d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:58.368 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.373+0000 7f2347a00640 1 --2- 172.21.15.67:0/3525478440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23400c80f0 0x7f234014ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:58.368 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.373+0000 7f2347a00640 1 --2- 172.21.15.67:0/3525478440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f234006ca00 0x7f23401ef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:51:58.368 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.373+0000 7f2347a00640 1 -- 172.21.15.67:0/3525478440 >> 172.21.15.67:0/3525478440 conn(0x7f23400ad2d0 msgr2=0x7f234014f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:51:58.368 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.373+0000 7f2347a00640 1 -- 172.21.15.67:0/3525478440 shutdown_connections 2024-03-09T08:51:58.368 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:51:58.373+0000 7f2347a00640 1 -- 172.21.15.67:0/3525478440 wait complete. 2024-03-09T08:51:58.381 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-09T08:51:58.444 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:58.448+0000 7fef4ca42040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-09T08:51:58.506 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:58.510+0000 7fef4ca42040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-09T08:51:58.635 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:58.640+0000 7fef4ca42040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-09T08:51:58.694 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:58.696+0000 7f87aa246040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-09T08:51:58.756 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:58.761+0000 7f87aa246040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-09T08:51:58.893 INFO:tasks.ceph.mgr.y.smithi067.stderr:2024-03-09T08:51:58.898+0000 7f87aa246040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-09T08:51:59.681 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:59.685+0000 7feeeda00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-09T08:51:59.681 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T08:51:59.686+0000 7feeeda00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-09T08:52:01.193 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-03-09T08:52:01.199+0000 7f7690019640 -1 osd.2 0 log_to_monitors true 2024-03-09T08:52:01.219 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-03-09T08:52:01.224+0000 7f1e74806640 -1 osd.6 0 log_to_monitors true 2024-03-09T08:52:01.227 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-03-09T08:52:01.233+0000 7f2e0ccce640 -1 osd.1 0 log_to_monitors true 2024-03-09T08:52:01.227 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-03-09T08:52:01.231+0000 7f9dcc5ff640 -1 osd.7 0 log_to_monitors true 2024-03-09T08:52:01.289 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-03-09T08:52:01.295+0000 7fe81400e640 -1 osd.3 0 log_to_monitors true 2024-03-09T08:52:01.449 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-03-09T08:52:01.453+0000 7f19a72b3640 -1 osd.5 0 log_to_monitors true 2024-03-09T08:52:01.508 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-03-09T08:52:01.512+0000 7f17d81f8640 -1 osd.4 0 log_to_monitors true 2024-03-09T08:52:01.523 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-03-09T08:52:01.529+0000 7f13a8e11640 -1 osd.0 0 log_to_monitors true 2024-03-09T08:52:04.883 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-03-09T08:52:04.888+0000 7f17d2600640 -1 osd.4 0 waiting for initial osdmap 2024-03-09T08:52:04.893 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-03-09T08:52:04.898+0000 7f17c9000640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-09T08:52:04.897 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-03-09T08:52:04.902+0000 7f9dc6a00640 -1 osd.7 0 waiting for initial osdmap 2024-03-09T08:52:04.912 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-03-09T08:52:04.916+0000 7f9dbd400640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-09T08:52:04.938 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-03-09T08:52:04.942+0000 7f19a3600640 -1 osd.5 0 waiting for initial osdmap 2024-03-09T08:52:04.947 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-03-09T08:52:04.951+0000 7f1998200640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-09T08:52:04.964 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-03-09T08:52:04.968+0000 7f13a2c00640 -1 osd.0 0 waiting for initial osdmap 2024-03-09T08:52:04.974 INFO:tasks.ceph.osd.0.smithi067.stderr:2024-03-09T08:52:04.976+0000 7f1399600640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-09T08:52:04.986 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-03-09T08:52:04.992+0000 7f1e70a00640 -1 osd.6 0 waiting for initial osdmap 2024-03-09T08:52:04.992 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-03-09T08:52:04.997+0000 7f1e65600640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-09T08:52:05.015 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-03-09T08:52:05.020+0000 7fe80de00640 -1 osd.3 0 waiting for initial osdmap 2024-03-09T08:52:05.021 INFO:tasks.ceph.osd.3.smithi067.stderr:2024-03-09T08:52:05.026+0000 7fe804800640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-09T08:52:05.048 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-03-09T08:52:05.053+0000 7f768a000640 -1 osd.2 0 waiting for initial osdmap 2024-03-09T08:52:05.055 INFO:tasks.ceph.osd.2.smithi067.stderr:2024-03-09T08:52:05.060+0000 7f7680a00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-09T08:52:05.080 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-03-09T08:52:05.085+0000 7f2e09000640 -1 osd.1 0 waiting for initial osdmap 2024-03-09T08:52:05.086 INFO:tasks.ceph.osd.1.smithi067.stderr:2024-03-09T08:52:05.091+0000 7f2dfdc00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-09T08:52:05.189 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:05.372 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.377+0000 7f29c3200640 1 Processor -- start 2024-03-09T08:52:05.372 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.377+0000 7f29c3200640 1 -- start start 2024-03-09T08:52:05.373 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.378+0000 7f29c3200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f29c414fc70 0x7f29c4155d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:05.373 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.378+0000 7f29c3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c40cb350 con 0x7f29c406b050 2024-03-09T08:52:05.373 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.378+0000 7f29c3200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c40c97d0 con 0x7f29c4157240 2024-03-09T08:52:05.373 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.378+0000 7f29c3200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c40c9ab0 con 0x7f29c414fc70 2024-03-09T08:52:05.374 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.378+0000 7f29c2800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f29c406b050 0x7f29c41606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42412/0 (socket says 172.21.15.67:42412) 2024-03-09T08:52:05.374 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.378+0000 7f29c2800640 1 -- 172.21.15.67:0/3956937272 learned_addr learned my addr 172.21.15.67:0/3956937272 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:05.374 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c1e00640 1 --2- 172.21.15.67:0/3956937272 >> v2:172.21.15.134:3300/0 conn(0x7f29c414fc70 0x7f29c4155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:05.376 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686728446 0 0) 0x7f29c40cb350 con 0x7f29c406b050 2024-03-09T08:52:05.376 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29a8003520 con 0x7f29c406b050 2024-03-09T08:52:05.376 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2090458697 0 0) 0x7f29a8003520 con 0x7f29c406b050 2024-03-09T08:52:05.376 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 >> v2:172.21.15.134:3300/0 conn(0x7f29c414fc70 msgr2=0x7f29c4155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:05.376 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 --2- 172.21.15.67:0/3956937272 >> v2:172.21.15.134:3300/0 conn(0x7f29c414fc70 0x7f29c4155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:05.376 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 >> v1:172.21.15.134:6789/0 conn(0x7f29c4157240 legacy=0x7f29c40ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:05.376 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40c9d90 con 0x7f29c406b050 2024-03-09T08:52:05.376 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f29bc001a10 con 0x7f29c406b050 2024-03-09T08:52:05.376 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 >> v1:172.21.15.67:6789/0 conn(0x7f29c406b050 legacy=0x7f29c41606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:05.376 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 --2- 172.21.15.67:0/3956937272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a8003a00 0x7f29a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:05.376 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 --2- 172.21.15.67:0/3956937272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29a8004800 0x7f29a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:05.377 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 --2- 172.21.15.67:0/3956937272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8005210 0x7f29a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:05.377 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40c9ab0 con 0x7f29a8003a00 2024-03-09T08:52:05.377 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a800c160 con 0x7f29a8005210 2024-03-09T08:52:05.377 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.379+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a800c360 con 0x7f29a8004800 2024-03-09T08:52:05.377 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.380+0000 7f29c1e00640 1 --2- 172.21.15.67:0/3956937272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a8003a00 0x7f29a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:05.377 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.380+0000 7f29c1e00640 1 -- 172.21.15.67:0/3956937272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29a8004800 msgr2=0x7f29a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:05.377 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.380+0000 7f29c1e00640 1 --2- 172.21.15.67:0/3956937272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29a8004800 0x7f29a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:05.377 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.380+0000 7f29c1e00640 1 -- 172.21.15.67:0/3956937272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8005210 msgr2=0x7f29a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:05.377 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.380+0000 7f29c1e00640 1 --2- 172.21.15.67:0/3956937272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8005210 0x7f29a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:05.377 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.380+0000 7f29c1e00640 1 -- 172.21.15.67:0/3956937272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a8003520 con 0x7f29a8003a00 2024-03-09T08:52:05.377 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.380+0000 7f29c2800640 1 -- 172.21.15.67:0/3956937272 reap_dead start 2024-03-09T08:52:05.378 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.381+0000 7f29c1e00640 1 --2- 172.21.15.67:0/3956937272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a8003a00 0x7f29a8003e20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009a90 tx=0x7f29a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180f67b25684ab6e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:05.378 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.381+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4039070 con 0x7f29a8003a00 2024-03-09T08:52:05.378 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.381+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b402c680 con 0x7f29a8003a00 2024-03-09T08:52:05.379 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.381+0000 7f29c0a00640 1 -- 172.21.15.67:0/3956937272 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4009690 con 0x7f29a8003a00 2024-03-09T08:52:05.379 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.382+0000 7f29c3200640 1 -- 172.21.15.67:0/3956937272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a8003a00 msgr2=0x7f29a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:05.379 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.382+0000 7f29c3200640 1 --2- 172.21.15.67:0/3956937272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a8003a00 0x7f29a8003e20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009a90 tx=0x7f29a8005830 comp rx=0 tx=0).stop 2024-03-09T08:52:05.379 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.382+0000 7f29c3200640 1 -- 172.21.15.67:0/3956937272 shutdown_connections 2024-03-09T08:52:05.379 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.382+0000 7f29c3200640 1 --2- 172.21.15.67:0/3956937272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29a8003a00 0x7f29a8003e20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:05.379 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.382+0000 7f29c3200640 1 -- 172.21.15.67:0/3956937272 >> 172.21.15.67:0/3956937272 conn(0x7f29c40cb580 msgr2=0x7f29c40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:05.379 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.382+0000 7f29c3200640 1 -- 172.21.15.67:0/3956937272 shutdown_connections 2024-03-09T08:52:05.379 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.382+0000 7f29c3200640 1 -- 172.21.15.67:0/3956937272 wait complete. 2024-03-09T08:52:05.380 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.384+0000 7f29c3200640 1 Processor -- start 2024-03-09T08:52:05.380 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.384+0000 7f29c3200640 1 -- start start 2024-03-09T08:52:05.382 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.385+0000 7f29c3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8003a00 0x7f29c40c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:05.382 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.385+0000 7f29c3200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c40beb90 0x7f29c4151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:05.382 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.385+0000 7f29c3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c4152530 0x7f29c40c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:05.382 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.385+0000 7f29c3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41576c0 con 0x7f29c4152530 2024-03-09T08:52:05.382 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.385+0000 7f29c3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40ca350 con 0x7f29a8003a00 2024-03-09T08:52:05.382 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.385+0000 7f29c3200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40c9480 con 0x7f29c40beb90 2024-03-09T08:52:05.382 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.385+0000 7f29c2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c4152530 0x7f29c40c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:05.382 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.385+0000 7f29c2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c4152530 0x7f29c40c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50930/0 (socket says 172.21.15.67:50930) 2024-03-09T08:52:05.382 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.385+0000 7f29c2800640 1 -- 172.21.15.67:0/2775997453 learned_addr learned my addr 172.21.15.67:0/2775997453 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:05.382 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29c2800640 1 -- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c40beb90 msgr2=0x7f29c4151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:05.383 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29c2800640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c40beb90 0x7f29c4151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:05.383 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29c2800640 1 -- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8003a00 msgr2=0x7f29c40c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:05.383 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29c1e00640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8003a00 0x7f29c40c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:05.383 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29c2800640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8003a00 0x7f29c40c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:05.383 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29c2800640 1 -- 172.21.15.67:0/2775997453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40c8110 con 0x7f29c4152530 2024-03-09T08:52:05.383 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29c2800640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c4152530 0x7f29c40c4670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f29c408d840 tx=0x7f29bc00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:05.383 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29ba000640 1 -- 172.21.15.67:0/2775997453 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00a5b0 con 0x7f29c4152530 2024-03-09T08:52:05.383 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29ba000640 1 -- 172.21.15.67:0/2775997453 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc00a770 con 0x7f29c4152530 2024-03-09T08:52:05.383 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29ba000640 1 -- 172.21.15.67:0/2775997453 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00ab60 con 0x7f29c4152530 2024-03-09T08:52:05.383 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29c3200640 1 -- 172.21.15.67:0/2775997453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c41ec5e0 con 0x7f29c4152530 2024-03-09T08:52:05.383 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.386+0000 7f29c3200640 1 -- 172.21.15.67:0/2775997453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c40c6330 con 0x7f29c4152530 2024-03-09T08:52:05.384 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.388+0000 7f29c3200640 1 -- 172.21.15.67:0/2775997453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c406ac30 con 0x7f29c4152530 2024-03-09T08:52:05.384 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.388+0000 7f29ba000640 1 -- 172.21.15.67:0/2775997453 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 89722+0+0 (secure 0 0 0) 0x7f29bc011070 con 0x7f29c4152530 2024-03-09T08:52:05.384 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.389+0000 7f29ba000640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a406b470 0x7f29a406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:05.384 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.389+0000 7f29ba000640 1 -- 172.21.15.67:0/2775997453 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f29bc088ab0 con 0x7f29c4152530 2024-03-09T08:52:05.385 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.390+0000 7f29c1e00640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a406b470 0x7f29a406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:05.386 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.391+0000 7f29c1e00640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a406b470 0x7f29a406d930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f29b4036000 tx=0x7f29b4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:05.390 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.394+0000 7f29ba000640 1 -- 172.21.15.67:0/2775997453 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f29bc01a060 con 0x7f29c4152530 2024-03-09T08:52:05.583 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.587+0000 7f29c3200640 1 -- 172.21.15.67:0/2775997453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29c40cb050 con 0x7f29c4152530 2024-03-09T08:52:05.584 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.588+0000 7f29ba000640 1 -- 172.21.15.67:0/2775997453 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f29bc0102c0 con 0x7f29c4152530 2024-03-09T08:52:05.584 INFO:teuthology.misc.health.smithi067.stdout: 2024-03-09T08:52:05.585 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":12,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:02.675988+0000","last_up_change":"0.000000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 -- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a406b470 msgr2=0x7f29a406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a406b470 0x7f29a406d930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f29b4036000 tx=0x7f29b4030000 comp rx=0 tx=0).stop 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 -- 172.21.15.67:0/2775997453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c4152530 msgr2=0x7f29c40c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c4152530 0x7f29c40c4670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f29c408d840 tx=0x7f29bc00ccd0 comp rx=0 tx=0).stop 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 -- 172.21.15.67:0/2775997453 shutdown_connections 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a406b470 0x7f29a406d930 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c4152530 0x7f29c40c4670 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c40beb90 0x7f29c4151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 --2- 172.21.15.67:0/2775997453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29a8003a00 0x7f29c40c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 -- 172.21.15.67:0/2775997453 >> 172.21.15.67:0/2775997453 conn(0x7f29c40cb580 msgr2=0x7f29c4155bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 -- 172.21.15.67:0/2775997453 shutdown_connections 2024-03-09T08:52:05.587 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:05.591+0000 7f29c3200640 1 -- 172.21.15.67:0/2775997453 wait complete. 2024-03-09T08:52:05.599 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-09T08:52:08.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:52:08.375 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:52:12.405 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:12.604 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.609+0000 7fadce200640 1 Processor -- start 2024-03-09T08:52:12.605 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.609+0000 7fadce200640 1 -- start start 2024-03-09T08:52:12.606 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.611+0000 7fadce200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadc81512a0 0x7fadc8157c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:12.606 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.611+0000 7fadce200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fadc80c31d0 0x7fadc80c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:12.606 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.611+0000 7fadce200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc80c3a20 con 0x7fadc80c4250 2024-03-09T08:52:12.606 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.611+0000 7fadce200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc80c25f0 con 0x7fadc80c31d0 2024-03-09T08:52:12.607 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.611+0000 7fadce200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80c1cc0 con 0x7fadc81512a0 2024-03-09T08:52:12.607 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.611+0000 7fadcce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadc81512a0 0x7fadc8157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:12.607 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.611+0000 7fadc7e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fadc80c31d0 0x7fadc80c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:12.607 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.611+0000 7fadcd800640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fadc80c4250 0x7fadc8153b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:38692/0 (socket says 172.21.15.67:38692) 2024-03-09T08:52:12.607 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.611+0000 7fadcd800640 1 -- 172.21.15.67:0/3532227943 learned_addr learned my addr 172.21.15.67:0/3532227943 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:12.608 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.612+0000 7fadc7400640 1 -- 172.21.15.67:0/3532227943 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215389637 0 0) 0x7fadc80c3a20 con 0x7fadc80c4250 2024-03-09T08:52:12.608 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.612+0000 7fadc7400640 1 -- 172.21.15.67:0/3532227943 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadb0003520 con 0x7fadc80c4250 2024-03-09T08:52:12.608 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.612+0000 7fadcce00640 1 -- 172.21.15.67:0/3532227943 >> v2:172.21.15.134:3300/0 conn(0x7fadc80c31d0 msgr2=0x7fadc80c3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:12.608 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.612+0000 7fadcce00640 1 --2- 172.21.15.67:0/3532227943 >> v2:172.21.15.134:3300/0 conn(0x7fadc80c31d0 0x7fadc80c3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:12.608 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.612+0000 7fadcce00640 1 -- 172.21.15.67:0/3532227943 >> v1:172.21.15.134:6789/0 conn(0x7fadc80c4250 legacy=0x7fadc8153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:12.608 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.612+0000 7fadcce00640 1 -- 172.21.15.67:0/3532227943 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc80c1fa0 con 0x7fadc81512a0 2024-03-09T08:52:12.608 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.612+0000 7fadc7e00640 1 --2- 172.21.15.67:0/3532227943 >> v2:172.21.15.134:3300/0 conn(0x7fadc80c31d0 0x7fadc80c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:52:12.609 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.613+0000 7fadcce00640 1 --2- 172.21.15.67:0/3532227943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadc81512a0 0x7fadc8157c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0097a0 tx=0x7fadbc02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65391cf07c531f18 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:12.609 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.613+0000 7fadc7400640 1 -- 172.21.15.67:0/3532227943 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc031780 con 0x7fadc81512a0 2024-03-09T08:52:12.609 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.613+0000 7fadc7400640 1 -- 172.21.15.67:0/3532227943 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc037070 con 0x7fadc81512a0 2024-03-09T08:52:12.609 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.613+0000 7fadc7400640 1 -- 172.21.15.67:0/3532227943 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc031b40 con 0x7fadc81512a0 2024-03-09T08:52:12.609 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.614+0000 7fadce200640 1 -- 172.21.15.67:0/3532227943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadc81512a0 msgr2=0x7fadc8157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:12.610 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.614+0000 7fadce200640 1 --2- 172.21.15.67:0/3532227943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadc81512a0 0x7fadc8157c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0097a0 tx=0x7fadbc02b820 comp rx=0 tx=0).stop 2024-03-09T08:52:12.610 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.614+0000 7fadce200640 1 -- 172.21.15.67:0/3532227943 shutdown_connections 2024-03-09T08:52:12.610 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.614+0000 7fadce200640 1 --2- 172.21.15.67:0/3532227943 >> v2:172.21.15.134:3300/0 conn(0x7fadc80c31d0 0x7fadc80c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:12.610 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.614+0000 7fadce200640 1 --2- 172.21.15.67:0/3532227943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadc81512a0 0x7fadc8157c90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:12.610 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.614+0000 7fadce200640 1 -- 172.21.15.67:0/3532227943 >> 172.21.15.67:0/3532227943 conn(0x7fadc80c72c0 msgr2=0x7fadc80c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:12.610 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.614+0000 7fadce200640 1 -- 172.21.15.67:0/3532227943 shutdown_connections 2024-03-09T08:52:12.610 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.614+0000 7fadce200640 1 -- 172.21.15.67:0/3532227943 wait complete. 2024-03-09T08:52:12.616 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.616+0000 7fadce200640 1 Processor -- start 2024-03-09T08:52:12.616 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.616+0000 7fadce200640 1 -- start start 2024-03-09T08:52:12.616 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.617+0000 7fadce200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadc80c31d0 0x7fadc814f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:12.616 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.617+0000 7fadce200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadc80c4250 0x7fadc81553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:12.616 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.617+0000 7fadce200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadc81512a0 0x7fadc8155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:12.616 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.617+0000 7fadce200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80c2c00 con 0x7fadc80c4250 2024-03-09T08:52:12.616 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.617+0000 7fadce200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80c2a80 con 0x7fadc80c31d0 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.617+0000 7fadce200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80c3630 con 0x7fadc81512a0 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.617+0000 7fadc7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadc80c4250 0x7fadc81553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.617+0000 7fadc7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadc80c4250 0x7fadc81553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38382/0 (socket says 172.21.15.67:38382) 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.617+0000 7fadc7e00640 1 -- 172.21.15.67:0/3810724254 learned_addr learned my addr 172.21.15.67:0/3810724254 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.617+0000 7fadcd800640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadc81512a0 0x7fadc8155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.617+0000 7fadcce00640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadc80c31d0 0x7fadc814f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.618+0000 7fadc7e00640 1 -- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadc81512a0 msgr2=0x7fadc8155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.618+0000 7fadc7e00640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadc81512a0 0x7fadc8155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.618+0000 7fadc7e00640 1 -- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadc80c31d0 msgr2=0x7fadc814f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.618+0000 7fadc7e00640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadc80c31d0 0x7fadc814f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.618+0000 7fadc7e00640 1 -- 172.21.15.67:0/3810724254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc81ef470 con 0x7fadc80c4250 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.618+0000 7fadc7e00640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadc80c4250 0x7fadc81553c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fadb400ec10 tx=0x7fadb400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.618+0000 7fadc4c00640 1 -- 172.21.15.67:0/3810724254 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4017070 con 0x7fadc80c4250 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.618+0000 7fadc4c00640 1 -- 172.21.15.67:0/3810724254 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb400cc30 con 0x7fadc80c4250 2024-03-09T08:52:12.617 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.618+0000 7fadc4c00640 1 -- 172.21.15.67:0/3810724254 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4013470 con 0x7fadc80c4250 2024-03-09T08:52:12.618 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.618+0000 7fadce200640 1 -- 172.21.15.67:0/3810724254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc81ef750 con 0x7fadc80c4250 2024-03-09T08:52:12.618 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.619+0000 7fadce200640 1 -- 172.21.15.67:0/3810724254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc814fb10 con 0x7fadc80c4250 2024-03-09T08:52:12.618 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.620+0000 7fadce200640 1 -- 172.21.15.67:0/3810724254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc806ac30 con 0x7fadc80c4250 2024-03-09T08:52:12.618 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.621+0000 7fadc4c00640 1 -- 172.21.15.67:0/3810724254 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fadb401a070 con 0x7fadc80c4250 2024-03-09T08:52:12.618 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.621+0000 7fadc4c00640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fada806b4d0 0x7fada806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:12.619 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.622+0000 7fadc4c00640 1 -- 172.21.15.67:0/3810724254 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fadb408a0d0 con 0x7fadc80c4250 2024-03-09T08:52:12.619 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.622+0000 7fadcce00640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fada806b4d0 0x7fada806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:12.619 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.623+0000 7fadcce00640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fada806b4d0 0x7fada806d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fadbc036000 tx=0x7fadbc03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:12.622 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.627+0000 7fadc4c00640 1 -- 172.21.15.67:0/3810724254 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fadb4058c50 con 0x7fadc80c4250 2024-03-09T08:52:12.823 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.828+0000 7fadce200640 1 -- 172.21.15.67:0/3810724254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadc80c6fd0 con 0x7fadc80c4250 2024-03-09T08:52:12.825 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.830+0000 7fadc4c00640 1 -- 172.21.15.67:0/3810724254 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12848 (secure 0 0 0) 0x7fadb405ccf0 con 0x7fadc80c4250 2024-03-09T08:52:12.825 INFO:teuthology.misc.health.smithi067.stdout: 2024-03-09T08:52:12.825 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":16,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:08.940246+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:12.828 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.832+0000 7fadce200640 1 -- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fada806b4d0 msgr2=0x7fada806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:12.828 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.832+0000 7fadce200640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fada806b4d0 0x7fada806d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fadbc036000 tx=0x7fadbc03d040 comp rx=0 tx=0).stop 2024-03-09T08:52:12.828 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.832+0000 7fadce200640 1 -- 172.21.15.67:0/3810724254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadc80c4250 msgr2=0x7fadc81553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:12.828 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.832+0000 7fadce200640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadc80c4250 0x7fadc81553c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fadb400ec10 tx=0x7fadb400c520 comp rx=0 tx=0).stop 2024-03-09T08:52:12.828 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.833+0000 7fadce200640 1 -- 172.21.15.67:0/3810724254 shutdown_connections 2024-03-09T08:52:12.829 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.833+0000 7fadce200640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fada806b4d0 0x7fada806d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:12.829 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.833+0000 7fadce200640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadc81512a0 0x7fadc8155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:12.829 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.833+0000 7fadce200640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadc80c4250 0x7fadc81553c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:12.829 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.833+0000 7fadce200640 1 --2- 172.21.15.67:0/3810724254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadc80c31d0 0x7fadc814f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:12.829 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.833+0000 7fadce200640 1 -- 172.21.15.67:0/3810724254 >> 172.21.15.67:0/3810724254 conn(0x7fadc80c72c0 msgr2=0x7fadc806b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:12.829 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.833+0000 7fadce200640 1 -- 172.21.15.67:0/3810724254 shutdown_connections 2024-03-09T08:52:12.829 INFO:teuthology.misc.health.smithi067.stderr:2024-03-09T08:52:12.833+0000 7fadce200640 1 -- 172.21.15.67:0/3810724254 wait complete. 2024-03-09T08:52:12.842 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-09T08:52:12.842 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-09T08:52:12.843 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-09T08:52:12.843 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-09T08:52:12.848 INFO:tasks.ceph.mds.a:Started 2024-03-09T08:52:12.849 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-09T08:52:12.849 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-09T08:52:12.854 INFO:tasks.ceph.mds.c:Started 2024-03-09T08:52:12.854 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-09T08:52:12.854 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-09T08:52:12.859 INFO:tasks.ceph.mds.b:Started 2024-03-09T08:52:12.859 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-09T08:52:12.860 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-09T08:52:12.865 INFO:tasks.ceph.mds.d:Started 2024-03-09T08:52:12.865 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-09T08:52:12.866 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:13.007 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-03-09T08:52:13.008 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2024-03-09T08:52:13.009 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-03-09T08:52:13.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.056+0000 7f231f800640 1 Processor -- start 2024-03-09T08:52:13.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.056+0000 7f231f800640 1 -- start start 2024-03-09T08:52:13.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.058+0000 7f231f800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f231806e2f0 0x7f23180aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:13.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.058+0000 7f231f800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23180ab3c0 con 0x7f2318152a20 2024-03-09T08:52:13.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.058+0000 7f231f800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23180aab20 con 0x7f23180c30f0 2024-03-09T08:52:13.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.058+0000 7f231f800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23180c82d0 con 0x7f231806e2f0 2024-03-09T08:52:13.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.058+0000 7f2317e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2318152a20 0x7f231806dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40954/0 (socket says 172.21.15.67:40954) 2024-03-09T08:52:13.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.058+0000 7f2317e00640 1 -- 172.21.15.67:0/2495291695 learned_addr learned my addr 172.21.15.67:0/2495291695 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:13.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.058+0000 7f231d800640 1 --2- 172.21.15.67:0/2495291695 >> v2:172.21.15.134:3300/0 conn(0x7f231806e2f0 0x7f23180aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:13.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.059+0000 7f2317400640 1 -- 172.21.15.67:0/2495291695 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866341447 0 0) 0x7f23180aab20 con 0x7f23180c30f0 2024-03-09T08:52:13.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.059+0000 7f2317400640 1 -- 172.21.15.67:0/2495291695 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2300005730 con 0x7f23180c30f0 2024-03-09T08:52:13.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.059+0000 7f231d800640 1 -- 172.21.15.67:0/2495291695 >> v1:172.21.15.134:6789/0 conn(0x7f23180c30f0 legacy=0x7f23181490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.059+0000 7f231d800640 1 -- 172.21.15.67:0/2495291695 >> v1:172.21.15.67:6789/0 conn(0x7f2318152a20 legacy=0x7f231806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.059+0000 7f231d800640 1 -- 172.21.15.67:0/2495291695 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23180aae00 con 0x7f231806e2f0 2024-03-09T08:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.060+0000 7f231d800640 1 --2- 172.21.15.67:0/2495291695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231806e2f0 0x7f23180aa420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2310009d90 tx=0x7f231002be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d72f0fcd48c8696 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.060+0000 7f2317400640 1 -- 172.21.15.67:0/2495291695 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310039070 con 0x7f231806e2f0 2024-03-09T08:52:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.060+0000 7f2317400640 1 -- 172.21.15.67:0/2495291695 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231002cbd0 con 0x7f231806e2f0 2024-03-09T08:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.060+0000 7f2317400640 1 -- 172.21.15.67:0/2495291695 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310034440 con 0x7f231806e2f0 2024-03-09T08:52:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.061+0000 7f231f800640 1 -- 172.21.15.67:0/2495291695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231806e2f0 msgr2=0x7f23180aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.061+0000 7f231f800640 1 --2- 172.21.15.67:0/2495291695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231806e2f0 0x7f23180aa420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2310009d90 tx=0x7f231002be20 comp rx=0 tx=0).stop 2024-03-09T08:52:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.061+0000 7f231f800640 1 -- 172.21.15.67:0/2495291695 shutdown_connections 2024-03-09T08:52:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.061+0000 7f231f800640 1 --2- 172.21.15.67:0/2495291695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231806e2f0 0x7f23180aa420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.061+0000 7f231f800640 1 -- 172.21.15.67:0/2495291695 >> 172.21.15.67:0/2495291695 conn(0x7f23180c7370 msgr2=0x7f23180c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.061+0000 7f231f800640 1 -- 172.21.15.67:0/2495291695 shutdown_connections 2024-03-09T08:52:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.061+0000 7f231f800640 1 -- 172.21.15.67:0/2495291695 wait complete. 2024-03-09T08:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.063+0000 7f231f800640 1 Processor -- start 2024-03-09T08:52:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.063+0000 7f231f800640 1 -- start start 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.064+0000 7f231f800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231806e2f0 0x7f23181ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.064+0000 7f231f800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23180c30f0 0x7f23181ebea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.064+0000 7f231f800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2318152a20 0x7f231814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.064+0000 7f231f800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2318149800 con 0x7f231806e2f0 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.064+0000 7f231f800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231806d6c0 con 0x7f23180c30f0 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.064+0000 7f231f800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2318149980 con 0x7f2318152a20 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.064+0000 7f231ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231806e2f0 0x7f23181ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.064+0000 7f231ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231806e2f0 0x7f23181ef3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38430/0 (socket says 172.21.15.67:38430) 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.064+0000 7f231ce00640 1 -- 172.21.15.67:0/1368878352 learned_addr learned my addr 172.21.15.67:0/1368878352 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.064+0000 7f2317e00640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23180c30f0 0x7f23181ebea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.064+0000 7f231d800640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2318152a20 0x7f231814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.065+0000 7f231ce00640 1 -- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2318152a20 msgr2=0x7f231814e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.065+0000 7f231ce00640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2318152a20 0x7f231814e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.065+0000 7f231ce00640 1 -- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23180c30f0 msgr2=0x7f23181ebea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.065+0000 7f231ce00640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23180c30f0 0x7f23181ebea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.065+0000 7f231ce00640 1 -- 172.21.15.67:0/1368878352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23181ec3e0 con 0x7f231806e2f0 2024-03-09T08:52:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.065+0000 7f231ce00640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231806e2f0 0x7f23181ef3e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f230c00c4a0 tx=0x7f230c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:13.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.105+0000 7f2314c00640 1 -- 172.21.15.67:0/1368878352 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c017070 con 0x7f231806e2f0 2024-03-09T08:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.105+0000 7f2314c00640 1 -- 172.21.15.67:0/1368878352 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230c005480 con 0x7f231806e2f0 2024-03-09T08:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.105+0000 7f231f800640 1 -- 172.21.15.67:0/1368878352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231814eb00 con 0x7f231806e2f0 2024-03-09T08:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.105+0000 7f2314c00640 1 -- 172.21.15.67:0/1368878352 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c003830 con 0x7f231806e2f0 2024-03-09T08:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.105+0000 7f231f800640 1 -- 172.21.15.67:0/1368878352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231814d8a0 con 0x7f231806e2f0 2024-03-09T08:52:13.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.106+0000 7f231f800640 1 -- 172.21.15.67:0/1368878352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2318075a00 con 0x7f231806e2f0 2024-03-09T08:52:13.104 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-03-09T08:52:13.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.110+0000 7f2314c00640 1 -- 172.21.15.67:0/1368878352 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f230c01a070 con 0x7f231806e2f0 2024-03-09T08:52:13.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.111+0000 7f2314c00640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22f806b500 0x7f22f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:13.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.111+0000 7f2314c00640 1 -- 172.21.15.67:0/1368878352 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f230c0896d0 con 0x7f231806e2f0 2024-03-09T08:52:13.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.111+0000 7f2317e00640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22f806b500 0x7f22f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:13.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.112+0000 7f2317e00640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22f806b500 0x7f22f806d9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23180aa070 tx=0x7f2304009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.115+0000 7f2314c00640 1 -- 172.21.15.67:0/1368878352 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f230c021020 con 0x7f231806e2f0 2024-03-09T08:52:13.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.318+0000 7f231f800640 1 -- 172.21.15.67:0/1368878352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2318068440 con 0x7f231806e2f0 2024-03-09T08:52:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.320+0000 7f2314c00640 1 -- 172.21.15.67:0/1368878352 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12848 (secure 0 0 0) 0x7f230c0581b0 con 0x7f231806e2f0 2024-03-09T08:52:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:13.316 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":16,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:08.940246+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:13.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 -- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22f806b500 msgr2=0x7f22f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22f806b500 0x7f22f806d9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23180aa070 tx=0x7f2304009040 comp rx=0 tx=0).stop 2024-03-09T08:52:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 -- 172.21.15.67:0/1368878352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231806e2f0 msgr2=0x7f23181ef3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231806e2f0 0x7f23181ef3e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f230c00c4a0 tx=0x7f230c00c960 comp rx=0 tx=0).stop 2024-03-09T08:52:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 -- 172.21.15.67:0/1368878352 shutdown_connections 2024-03-09T08:52:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22f806b500 0x7f22f806d9c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2318152a20 0x7f231814e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23180c30f0 0x7f23181ebea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 --2- 172.21.15.67:0/1368878352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231806e2f0 0x7f23181ef3e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 -- 172.21.15.67:0/1368878352 >> 172.21.15.67:0/1368878352 conn(0x7f23180c7370 msgr2=0x7f231806dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 -- 172.21.15.67:0/1368878352 shutdown_connections 2024-03-09T08:52:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.323+0000 7f231f800640 1 -- 172.21.15.67:0/1368878352 wait complete. 2024-03-09T08:52:13.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-09T08:52:06.749641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-09T08:52:13.336 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T08:52:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.561+0000 7f2705200640 1 Processor -- start 2024-03-09T08:52:13.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.561+0000 7f2705200640 1 -- start start 2024-03-09T08:52:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.563+0000 7f2705200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27000c3510 0x7f27000c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.563+0000 7f2705200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f270006dd60 0x7f270006e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.563+0000 7f2705200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2700153050 con 0x7f270014c040 2024-03-09T08:52:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.563+0000 7f2705200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27000ab320 con 0x7f270006dd60 2024-03-09T08:52:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.563+0000 7f2705200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2700152ad0 con 0x7f27000c3510 2024-03-09T08:52:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.563+0000 7f26ff400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f270014c040 0x7f27000aac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40960/0 (socket says 172.21.15.67:40960) 2024-03-09T08:52:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.563+0000 7f26ffe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f270006dd60 0x7f270006e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.563+0000 7f26ff400640 1 -- 172.21.15.67:0/1029617713 learned_addr learned my addr 172.21.15.67:0/1029617713 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.564+0000 7f26ffe00640 1 -- 172.21.15.67:0/1029617713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27000c3510 msgr2=0x7f27000c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.564+0000 7f26ffe00640 1 --2- 172.21.15.67:0/1029617713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27000c3510 0x7f27000c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.564+0000 7f26ffe00640 1 -- 172.21.15.67:0/1029617713 >> v1:172.21.15.67:6789/0 conn(0x7f270014c040 legacy=0x7f27000aac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.564+0000 7f26fe000640 1 -- 172.21.15.67:0/1029617713 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328497312 0 0) 0x7f2700153050 con 0x7f270014c040 2024-03-09T08:52:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.564+0000 7f26ffe00640 1 -- 172.21.15.67:0/1029617713 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27000c1dc0 con 0x7f270006dd60 2024-03-09T08:52:13.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.565+0000 7f26ffe00640 1 --2- 172.21.15.67:0/1029617713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f270006dd60 0x7f270006e1d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009d90 tx=0x7f26f802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15596842ec3512d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:13.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.565+0000 7f26fe000640 1 -- 172.21.15.67:0/1029617713 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8039070 con 0x7f270006dd60 2024-03-09T08:52:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.565+0000 7f26fe000640 1 -- 172.21.15.67:0/1029617713 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f802cb30 con 0x7f270006dd60 2024-03-09T08:52:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.565+0000 7f26fe000640 1 -- 172.21.15.67:0/1029617713 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8034440 con 0x7f270006dd60 2024-03-09T08:52:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.566+0000 7f2705200640 1 -- 172.21.15.67:0/1029617713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f270006dd60 msgr2=0x7f270006e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.566+0000 7f2705200640 1 --2- 172.21.15.67:0/1029617713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f270006dd60 0x7f270006e1d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009d90 tx=0x7f26f802bd80 comp rx=0 tx=0).stop 2024-03-09T08:52:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.566+0000 7f2705200640 1 -- 172.21.15.67:0/1029617713 shutdown_connections 2024-03-09T08:52:13.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.566+0000 7f2705200640 1 --2- 172.21.15.67:0/1029617713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f270006dd60 0x7f270006e1d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.566+0000 7f2705200640 1 --2- 172.21.15.67:0/1029617713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27000c3510 0x7f27000c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.566+0000 7f2705200640 1 -- 172.21.15.67:0/1029617713 >> 172.21.15.67:0/1029617713 conn(0x7f270006b8a0 msgr2=0x7f27000c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:13.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.566+0000 7f2705200640 1 -- 172.21.15.67:0/1029617713 shutdown_connections 2024-03-09T08:52:13.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.566+0000 7f2705200640 1 -- 172.21.15.67:0/1029617713 wait complete. 2024-03-09T08:52:13.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.568+0000 7f2705200640 1 Processor -- start 2024-03-09T08:52:13.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.568+0000 7f2705200640 1 -- start start 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f2705200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f270006dd60 0x7f27001436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f2705200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27000c3510 0x7f27001eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f2705200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f270014c040 0x7f27001ebf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f2705200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27000c1910 con 0x7f270014c040 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f2705200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27000c2060 con 0x7f27000c3510 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f2705200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270014c420 con 0x7f270006dd60 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f26ffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f270014c040 0x7f27001ebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f26ffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f270014c040 0x7f27001ebf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38444/0 (socket says 172.21.15.67:38444) 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f26ffe00640 1 -- 172.21.15.67:0/553766483 learned_addr learned my addr 172.21.15.67:0/553766483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f26fea00640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27000c3510 0x7f27001eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f26ff400640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f270006dd60 0x7f27001436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:13.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f26ffe00640 1 -- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f270006dd60 msgr2=0x7f27001436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.569+0000 7f26ffe00640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f270006dd60 0x7f27001436c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.570+0000 7f26ffe00640 1 -- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27000c3510 msgr2=0x7f27001eb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.570+0000 7f26ffe00640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27000c3510 0x7f27001eb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.570+0000 7f26ffe00640 1 -- 172.21.15.67:0/553766483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2700141f80 con 0x7f270014c040 2024-03-09T08:52:13.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.570+0000 7f26fea00640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27000c3510 0x7f27001eb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:13.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.570+0000 7f26ffe00640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f270014c040 0x7f27001ebf10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009eb0 tx=0x7f26f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:13.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.570+0000 7f26f7400640 1 -- 172.21.15.67:0/553766483 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8044020 con 0x7f270014c040 2024-03-09T08:52:13.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.570+0000 7f26f7400640 1 -- 172.21.15.67:0/553766483 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f803f470 con 0x7f270014c040 2024-03-09T08:52:13.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.570+0000 7f2705200640 1 -- 172.21.15.67:0/553766483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27001415d0 con 0x7f270014c040 2024-03-09T08:52:13.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.570+0000 7f26f7400640 1 -- 172.21.15.67:0/553766483 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f8039040 con 0x7f270014c040 2024-03-09T08:52:13.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.570+0000 7f2705200640 1 -- 172.21.15.67:0/553766483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27001ef430 con 0x7f270014c040 2024-03-09T08:52:13.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.571+0000 7f2705200640 1 -- 172.21.15.67:0/553766483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27001eef40 con 0x7f270014c040 2024-03-09T08:52:13.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.573+0000 7f26f7400640 1 -- 172.21.15.67:0/553766483 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f26f8047020 con 0x7f270014c040 2024-03-09T08:52:13.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.574+0000 7f26f7400640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26e006b4d0 0x7f26e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:13.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.574+0000 7f26f7400640 1 -- 172.21.15.67:0/553766483 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f26f80acb10 con 0x7f270014c040 2024-03-09T08:52:13.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.574+0000 7f26ff400640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26e006b4d0 0x7f26e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:13.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.575+0000 7f26ff400640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26e006b4d0 0x7f26e006d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f26f0007fd0 tx=0x7f26f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:13.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.579+0000 7f26f7400640 1 -- 172.21.15.67:0/553766483 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f26f804e030 con 0x7f270014c040 2024-03-09T08:52:13.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.779+0000 7f2705200640 1 -- 172.21.15.67:0/553766483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27000ab020 con 0x7f270014c040 2024-03-09T08:52:13.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.780+0000 7f26f7400640 1 -- 172.21.15.67:0/553766483 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26f807bcd0 con 0x7f270014c040 2024-03-09T08:52:13.775 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-09T08:52:13.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.782+0000 7f2705200640 1 -- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26e006b4d0 msgr2=0x7f26e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.782+0000 7f2705200640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26e006b4d0 0x7f26e006d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f26f0007fd0 tx=0x7f26f0009040 comp rx=0 tx=0).stop 2024-03-09T08:52:13.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.782+0000 7f2705200640 1 -- 172.21.15.67:0/553766483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f270014c040 msgr2=0x7f27001ebf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:13.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.782+0000 7f2705200640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f270014c040 0x7f27001ebf10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009eb0 tx=0x7f26f8004a40 comp rx=0 tx=0).stop 2024-03-09T08:52:13.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.783+0000 7f2705200640 1 -- 172.21.15.67:0/553766483 shutdown_connections 2024-03-09T08:52:13.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.783+0000 7f2705200640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26e006b4d0 0x7f26e006d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.783+0000 7f2705200640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f270014c040 0x7f27001ebf10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.783+0000 7f2705200640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27000c3510 0x7f27001eb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.783+0000 7f2705200640 1 --2- 172.21.15.67:0/553766483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f270006dd60 0x7f27001436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:13.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.783+0000 7f2705200640 1 -- 172.21.15.67:0/553766483 >> 172.21.15.67:0/553766483 conn(0x7f270006b8a0 msgr2=0x7f270006d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:13.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.783+0000 7f2705200640 1 -- 172.21.15.67:0/553766483 shutdown_connections 2024-03-09T08:52:13.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:13.783+0000 7f2705200640 1 -- 172.21.15.67:0/553766483 wait complete. 2024-03-09T08:52:13.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T08:52:14.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.023+0000 7f3787e00640 1 Processor -- start 2024-03-09T08:52:14.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.024+0000 7f3787e00640 1 -- start start 2024-03-09T08:52:14.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.025+0000 7f3787e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f37880c5e40 0x7f3788156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.025+0000 7f3787e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37880cadb0 0x7f3788156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.025+0000 7f3787e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3788156a90 con 0x7f3788153300 2024-03-09T08:52:14.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.025+0000 7f3787e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3788157000 con 0x7f37880c5e40 2024-03-09T08:52:14.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.025+0000 7f3787e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37880ae430 con 0x7f37880cadb0 2024-03-09T08:52:14.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.025+0000 7f3786a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3788153300 0x7f37880b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40972/0 (socket says 172.21.15.67:40972) 2024-03-09T08:52:14.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.025+0000 7f3786a00640 1 -- 172.21.15.67:0/2970055538 learned_addr learned my addr 172.21.15.67:0/2970055538 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:14.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f377e000640 1 --2- 172.21.15.67:0/2970055538 >> v2:172.21.15.134:3300/0 conn(0x7f37880c5e40 0x7f3788156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3787400640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37880cadb0 0x7f3788156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130733044 0 0) 0x7f3788156a90 con 0x7f3788153300 2024-03-09T08:52:14.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f376c003520 con 0x7f3788153300 2024-03-09T08:52:14.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207191952 0 0) 0x7f376c003520 con 0x7f3788153300 2024-03-09T08:52:14.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37880cadb0 msgr2=0x7f3788156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:14.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3786000640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37880cadb0 0x7f3788156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 >> v2:172.21.15.134:3300/0 conn(0x7f37880c5e40 msgr2=0x7f3788156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:14.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3786000640 1 --2- 172.21.15.67:0/2970055538 >> v2:172.21.15.134:3300/0 conn(0x7f37880c5e40 0x7f3788156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880ae6f0 con 0x7f3788153300 2024-03-09T08:52:14.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f3778001670 con 0x7f3788153300 2024-03-09T08:52:14.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 >> v1:172.21.15.67:6789/0 conn(0x7f3788153300 legacy=0x7f37880b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:14.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.026+0000 7f3786000640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f376c003a20 0x7f376c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3786000640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37880cadb0 0x7f376c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3786000640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c004d40 0x7f376c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3788157000 con 0x7f376c003a20 2024-03-09T08:52:14.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37880ae430 con 0x7f376c004d40 2024-03-09T08:52:14.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376c00b9c0 con 0x7f37880cadb0 2024-03-09T08:52:14.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3786a00640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f376c003a20 0x7f376c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3787400640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c004d40 0x7f376c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3786a00640 1 -- 172.21.15.67:0/2970055538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37880cadb0 msgr2=0x7f376c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:14.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3786a00640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37880cadb0 0x7f376c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3786a00640 1 -- 172.21.15.67:0/2970055538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c004d40 msgr2=0x7f376c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:14.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3786a00640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c004d40 0x7f376c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3786a00640 1 -- 172.21.15.67:0/2970055538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c003520 con 0x7f376c003a20 2024-03-09T08:52:14.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.027+0000 7f3787400640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c004d40 0x7f376c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:14.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.028+0000 7f3786a00640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f376c003a20 0x7f376c003e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f377800c160 tx=0x7f376c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a292bd7215bb1f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:14.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.028+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3778007c80 con 0x7f376c003a20 2024-03-09T08:52:14.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.029+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3778030070 con 0x7f376c003a20 2024-03-09T08:52:14.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.029+0000 7f3786000640 1 -- 172.21.15.67:0/2970055538 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3778005d30 con 0x7f376c003a20 2024-03-09T08:52:14.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.029+0000 7f3787e00640 1 -- 172.21.15.67:0/2970055538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f376c003a20 msgr2=0x7f376c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:14.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.029+0000 7f3787e00640 1 --2- 172.21.15.67:0/2970055538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f376c003a20 0x7f376c003e40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f377800c160 tx=0x7f376c005830 comp rx=0 tx=0).stop 2024-03-09T08:52:14.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.029+0000 7f3787400640 1 -- 172.21.15.67:0/2970055538 reap_dead start 2024-03-09T08:52:14.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.029+0000 7f3787e00640 1 -- 172.21.15.67:0/2970055538 shutdown_connections 2024-03-09T08:52:14.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.029+0000 7f3787e00640 1 -- 172.21.15.67:0/2970055538 >> 172.21.15.67:0/2970055538 conn(0x7f37880732d0 msgr2=0x7f37880ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:14.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.029+0000 7f3787e00640 1 -- 172.21.15.67:0/2970055538 shutdown_connections 2024-03-09T08:52:14.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.029+0000 7f3787e00640 1 -- 172.21.15.67:0/2970055538 wait complete. 2024-03-09T08:52:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.031+0000 7f3787e00640 1 Processor -- start 2024-03-09T08:52:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.031+0000 7f3787e00640 1 -- start start 2024-03-09T08:52:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.032+0000 7f3787e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c003a20 0x7f378806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.032+0000 7f3787e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f378814cd70 0x7f37881ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.032+0000 7f3787e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37881ec290 0x7f378814b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.032+0000 7f3787e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3788159e90 con 0x7f37881ec290 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.032+0000 7f3787e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3788159bb0 con 0x7f376c003a20 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.032+0000 7f3787e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37880c59e0 con 0x7f378814cd70 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f3787400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37881ec290 0x7f378814b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f3787400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37881ec290 0x7f378814b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38452/0 (socket says 172.21.15.67:38452) 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f3787400640 1 -- 172.21.15.67:0/2612246587 learned_addr learned my addr 172.21.15.67:0/2612246587 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f3786a00640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c003a20 0x7f378806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f377e000640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f378814cd70 0x7f37881ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f3787400640 1 -- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f378814cd70 msgr2=0x7f37881ebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f3787400640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f378814cd70 0x7f37881ebd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f3787400640 1 -- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c003a20 msgr2=0x7f378806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f3787400640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c003a20 0x7f378806bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f3787400640 1 -- 172.21.15.67:0/2612246587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880c6f10 con 0x7f37881ec290 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f377e000640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f378814cd70 0x7f37881ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.033+0000 7f3787400640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37881ec290 0x7f378814b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f37880c9060 tx=0x7f37800109f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.034+0000 7f377fe00640 1 -- 172.21.15.67:0/2612246587 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378000f040 con 0x7f37881ec290 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.034+0000 7f377fe00640 1 -- 172.21.15.67:0/2612246587 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3780014820 con 0x7f37881ec290 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.034+0000 7f3787e00640 1 -- 172.21.15.67:0/2612246587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37880c71f0 con 0x7f37881ec290 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.034+0000 7f377fe00640 1 -- 172.21.15.67:0/2612246587 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780014c10 con 0x7f37881ec290 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.034+0000 7f3787e00640 1 -- 172.21.15.67:0/2612246587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378806c4b0 con 0x7f37881ec290 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.035+0000 7f3787e00640 1 -- 172.21.15.67:0/2612246587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378804f3c0 con 0x7f37881ec290 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.036+0000 7f377fe00640 1 -- 172.21.15.67:0/2612246587 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f378001c400 con 0x7f37881ec290 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.037+0000 7f377fe00640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f376406b4d0 0x7f376406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.037+0000 7f377fe00640 1 -- 172.21.15.67:0/2612246587 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f378008a1a0 con 0x7f37881ec290 2024-03-09T08:52:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.038+0000 7f3786a00640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f376406b4d0 0x7f376406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.039+0000 7f3786a00640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f376406b4d0 0x7f376406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3778034000 tx=0x7f3778007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.041+0000 7f377fe00640 1 -- 172.21.15.67:0/2612246587 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3780018030 con 0x7f37881ec290 2024-03-09T08:52:14.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.288+0000 7f3787e00640 1 -- 172.21.15.67:0/2612246587 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37880cab90 con 0x7f37881ec290 2024-03-09T08:52:14.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.289+0000 7f377fe00640 1 -- 172.21.15.67:0/2612246587 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f3780059360 con 0x7f37881ec290 2024-03-09T08:52:14.285 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:14.285 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-09T08:52:14.286 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 3 2024-03-09T08:52:14.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.291+0000 7f3787e00640 1 -- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f376406b4d0 msgr2=0x7f376406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:14.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.291+0000 7f3787e00640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f376406b4d0 0x7f376406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3778034000 tx=0x7f3778007000 comp rx=0 tx=0).stop 2024-03-09T08:52:14.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.291+0000 7f3787e00640 1 -- 172.21.15.67:0/2612246587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37881ec290 msgr2=0x7f378814b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:14.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.291+0000 7f3787e00640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37881ec290 0x7f378814b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f37880c9060 tx=0x7f37800109f0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.292+0000 7f3787e00640 1 -- 172.21.15.67:0/2612246587 shutdown_connections 2024-03-09T08:52:14.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.292+0000 7f3787e00640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f376406b4d0 0x7f376406d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.292+0000 7f3787e00640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37881ec290 0x7f378814b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.292+0000 7f3787e00640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f378814cd70 0x7f37881ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.292+0000 7f3787e00640 1 --2- 172.21.15.67:0/2612246587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f376c003a20 0x7f378806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.292+0000 7f3787e00640 1 -- 172.21.15.67:0/2612246587 >> 172.21.15.67:0/2612246587 conn(0x7f37880732d0 msgr2=0x7f3788150720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:14.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.292+0000 7f3787e00640 1 -- 172.21.15.67:0/2612246587 shutdown_connections 2024-03-09T08:52:14.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.292+0000 7f3787e00640 1 -- 172.21.15.67:0/2612246587 wait complete. 2024-03-09T08:52:14.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.536+0000 7f1d28e00640 1 Processor -- start 2024-03-09T08:52:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.536+0000 7f1d28e00640 1 -- start start 2024-03-09T08:52:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d28e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d240ac730 0x7f1d24071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d28e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d240c22a0 0x7f1d240acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d28e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1d24157650 0x7f1d24157b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d28e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d24070ee0 con 0x7f1d240ac730 2024-03-09T08:52:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d28e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d24154d00 con 0x7f1d24157650 2024-03-09T08:52:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d28e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d24153d40 con 0x7f1d240c22a0 2024-03-09T08:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d21e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d240ac730 0x7f1d24071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d21e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1d240ac730 0x7f1d24071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38462/0 (socket says 172.21.15.67:38462) 2024-03-09T08:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d21e00640 1 -- 172.21.15.67:0/1869334481 learned_addr learned my addr 172.21.15.67:0/1869334481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d22800640 1 --2- 172.21.15.67:0/1869334481 >> v2:172.21.15.134:3300/0 conn(0x7f1d24157650 0x7f1d24157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d21400640 1 --2- 172.21.15.67:0/1869334481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d240c22a0 0x7f1d240acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d21e00640 1 -- 172.21.15.67:0/1869334481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d240c22a0 msgr2=0x7f1d240acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.538+0000 7f1d21e00640 1 --2- 172.21.15.67:0/1869334481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d240c22a0 0x7f1d240acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.539+0000 7f1d21e00640 1 -- 172.21.15.67:0/1869334481 >> v2:172.21.15.134:3300/0 conn(0x7f1d24157650 msgr2=0x7f1d24157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.539+0000 7f1d21e00640 1 --2- 172.21.15.67:0/1869334481 >> v2:172.21.15.134:3300/0 conn(0x7f1d24157650 0x7f1d24157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.539+0000 7f1d21e00640 1 -- 172.21.15.67:0/1869334481 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d240c2790 con 0x7f1d240ac730 2024-03-09T08:52:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.540+0000 7f1d21e00640 1 --2- 172.21.15.67:0/1869334481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d240ac730 0x7f1d24071190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d240c3e40 tx=0x7f1d1402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a20bf12cf66d57 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.540+0000 7f1d20a00640 1 -- 172.21.15.67:0/1869334481 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14035020 con 0x7f1d240ac730 2024-03-09T08:52:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.540+0000 7f1d20a00640 1 -- 172.21.15.67:0/1869334481 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d14031870 con 0x7f1d240ac730 2024-03-09T08:52:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.540+0000 7f1d20a00640 1 -- 172.21.15.67:0/1869334481 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14031c40 con 0x7f1d240ac730 2024-03-09T08:52:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.541+0000 7f1d28e00640 1 -- 172.21.15.67:0/1869334481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d240ac730 msgr2=0x7f1d24071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.541+0000 7f1d28e00640 1 --2- 172.21.15.67:0/1869334481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d240ac730 0x7f1d24071190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d240c3e40 tx=0x7f1d1402b910 comp rx=0 tx=0).stop 2024-03-09T08:52:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.541+0000 7f1d28e00640 1 -- 172.21.15.67:0/1869334481 shutdown_connections 2024-03-09T08:52:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.541+0000 7f1d28e00640 1 --2- 172.21.15.67:0/1869334481 >> v2:172.21.15.134:3300/0 conn(0x7f1d24157650 0x7f1d24157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.541+0000 7f1d28e00640 1 --2- 172.21.15.67:0/1869334481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d240c22a0 0x7f1d240acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.541+0000 7f1d28e00640 1 --2- 172.21.15.67:0/1869334481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d240ac730 0x7f1d24071190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.541+0000 7f1d28e00640 1 -- 172.21.15.67:0/1869334481 >> 172.21.15.67:0/1869334481 conn(0x7f1d240c7370 msgr2=0x7f1d240c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.541+0000 7f1d28e00640 1 -- 172.21.15.67:0/1869334481 shutdown_connections 2024-03-09T08:52:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.541+0000 7f1d28e00640 1 -- 172.21.15.67:0/1869334481 wait complete. 2024-03-09T08:52:14.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.543+0000 7f1d28e00640 1 Processor -- start 2024-03-09T08:52:14.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.543+0000 7f1d28e00640 1 -- start start 2024-03-09T08:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.544+0000 7f1d28e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d240ac730 0x7f1d241ec430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.544+0000 7f1d28e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d240c22a0 0x7f1d240bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.544+0000 7f1d28e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d24157650 0x7f1d24159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.544+0000 7f1d28e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d24158040 con 0x7f1d24157650 2024-03-09T08:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.544+0000 7f1d28e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d24070c00 con 0x7f1d240ac730 2024-03-09T08:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.544+0000 7f1d28e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d241581c0 con 0x7f1d240c22a0 2024-03-09T08:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.544+0000 7f1d22800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d24157650 0x7f1d24159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.544+0000 7f1d22800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d24157650 0x7f1d24159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38476/0 (socket says 172.21.15.67:38476) 2024-03-09T08:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.544+0000 7f1d22800640 1 -- 172.21.15.67:0/585529223 learned_addr learned my addr 172.21.15.67:0/585529223 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d21e00640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d240ac730 0x7f1d241ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d21400640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d240c22a0 0x7f1d240bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d22800640 1 -- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d240c22a0 msgr2=0x7f1d240bcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:14.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d22800640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d240c22a0 0x7f1d240bcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d22800640 1 -- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d240ac730 msgr2=0x7f1d241ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:14.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d22800640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d240ac730 0x7f1d241ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d22800640 1 -- 172.21.15.67:0/585529223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d241eb9c0 con 0x7f1d24157650 2024-03-09T08:52:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d22800640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d24157650 0x7f1d24159600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00d5d0 tx=0x7f1d1c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d1a000640 1 -- 172.21.15.67:0/585529223 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0125f0 con 0x7f1d24157650 2024-03-09T08:52:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d1a000640 1 -- 172.21.15.67:0/585529223 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c019070 con 0x7f1d24157650 2024-03-09T08:52:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d28e00640 1 -- 172.21.15.67:0/585529223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2415e200 con 0x7f1d24157650 2024-03-09T08:52:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.545+0000 7f1d1a000640 1 -- 172.21.15.67:0/585529223 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0129f0 con 0x7f1d24157650 2024-03-09T08:52:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.546+0000 7f1d28e00640 1 -- 172.21.15.67:0/585529223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d241ebc30 con 0x7f1d24157650 2024-03-09T08:52:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.547+0000 7f1d28e00640 1 -- 172.21.15.67:0/585529223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d24075a00 con 0x7f1d24157650 2024-03-09T08:52:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.548+0000 7f1d1a000640 1 -- 172.21.15.67:0/585529223 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1d1c012b90 con 0x7f1d24157650 2024-03-09T08:52:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.549+0000 7f1d1a000640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1d0006b500 0x7f1d0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.549+0000 7f1d1a000640 1 -- 172.21.15.67:0/585529223 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1d1c08a270 con 0x7f1d24157650 2024-03-09T08:52:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.549+0000 7f1d21e00640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1d0006b500 0x7f1d0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.550+0000 7f1d21e00640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1d0006b500 0x7f1d0006d9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1d14036000 tx=0x7f1d1403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:14.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.553+0000 7f1d1a000640 1 -- 172.21.15.67:0/585529223 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1d2415e200 con 0x7f1d24157650 2024-03-09T08:52:14.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.760+0000 7f1d28e00640 1 -- 172.21.15.67:0/585529223 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d240c1d70 con 0x7f1d24157650 2024-03-09T08:52:14.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.761+0000 7f1d1a000640 1 -- 172.21.15.67:0/585529223 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12848 (secure 0 0 0) 0x7f1d1c0593b0 con 0x7f1d24157650 2024-03-09T08:52:14.757 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:14.757 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":16,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:08.940246+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.764+0000 7f1d28e00640 1 -- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1d0006b500 msgr2=0x7f1d0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.764+0000 7f1d28e00640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1d0006b500 0x7f1d0006d9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1d14036000 tx=0x7f1d1403c040 comp rx=0 tx=0).stop 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.764+0000 7f1d28e00640 1 -- 172.21.15.67:0/585529223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d24157650 msgr2=0x7f1d24159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.764+0000 7f1d28e00640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d24157650 0x7f1d24159600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00d5d0 tx=0x7f1d1c00da90 comp rx=0 tx=0).stop 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.764+0000 7f1d28e00640 1 -- 172.21.15.67:0/585529223 shutdown_connections 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.764+0000 7f1d28e00640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1d0006b500 0x7f1d0006d9c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.764+0000 7f1d28e00640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d24157650 0x7f1d24159600 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.764+0000 7f1d28e00640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d240c22a0 0x7f1d240bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.764+0000 7f1d28e00640 1 --2- 172.21.15.67:0/585529223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d240ac730 0x7f1d241ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.764+0000 7f1d28e00640 1 -- 172.21.15.67:0/585529223 >> 172.21.15.67:0/585529223 conn(0x7f1d240c7370 msgr2=0x7f1d240c47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.764+0000 7f1d28e00640 1 -- 172.21.15.67:0/585529223 shutdown_connections 2024-03-09T08:52:14.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.765+0000 7f1d28e00640 1 -- 172.21.15.67:0/585529223 wait complete. 2024-03-09T08:52:14.779 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-09T08:52:06.749641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-09T08:52:14.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T08:52:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.999+0000 7febca600640 1 Processor -- start 2024-03-09T08:52:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:14.999+0000 7febca600640 1 -- start start 2024-03-09T08:52:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.001+0000 7febca600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7febc40c5f10 0x7febc40c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.001+0000 7febca600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc406b040 0x7febc41567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.001+0000 7febca600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7febc40c9490 0x7febc40cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.001+0000 7febca600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc4156d10 con 0x7febc40c5f10 2024-03-09T08:52:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.001+0000 7febca600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc40c9a30 con 0x7febc40c9490 2024-03-09T08:52:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.001+0000 7febca600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc40c9d10 con 0x7febc406b040 2024-03-09T08:52:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.001+0000 7febc9200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7febc40c5f10 0x7febc40c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.001+0000 7febc9200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7febc40c5f10 0x7febc40c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38492/0 (socket says 172.21.15.67:38492) 2024-03-09T08:52:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.001+0000 7febc9200640 1 -- 172.21.15.67:0/2737386960 learned_addr learned my addr 172.21.15.67:0/2737386960 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.001+0000 7febc9c00640 1 --2- 172.21.15.67:0/2737386960 >> v2:172.21.15.134:3300/0 conn(0x7febc40c9490 0x7febc40cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.002+0000 7febc3e00640 1 --2- 172.21.15.67:0/2737386960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc406b040 0x7febc41567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.002+0000 7febc9200640 1 -- 172.21.15.67:0/2737386960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc406b040 msgr2=0x7febc41567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.002+0000 7febc9200640 1 --2- 172.21.15.67:0/2737386960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc406b040 0x7febc41567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.002+0000 7febc9200640 1 -- 172.21.15.67:0/2737386960 >> v2:172.21.15.134:3300/0 conn(0x7febc40c9490 msgr2=0x7febc40cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.002+0000 7febc9200640 1 --2- 172.21.15.67:0/2737386960 >> v2:172.21.15.134:3300/0 conn(0x7febc40c9490 0x7febc40cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.002+0000 7febc9200640 1 -- 172.21.15.67:0/2737386960 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc40ca340 con 0x7febc40c5f10 2024-03-09T08:52:14.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.003+0000 7febc9200640 1 --2- 172.21.15.67:0/2737386960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc40c5f10 0x7febc40c62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7febc40c5b50 tx=0x7febb402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978602655ff908cf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:14.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.003+0000 7febc3400640 1 -- 172.21.15.67:0/2737386960 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb402c9e0 con 0x7febc40c5f10 2024-03-09T08:52:14.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.003+0000 7febc3400640 1 -- 172.21.15.67:0/2737386960 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb402cb80 con 0x7febc40c5f10 2024-03-09T08:52:14.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.003+0000 7febc3400640 1 -- 172.21.15.67:0/2737386960 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb40343f0 con 0x7febc40c5f10 2024-03-09T08:52:15.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.004+0000 7febca600640 1 -- 172.21.15.67:0/2737386960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc40c5f10 msgr2=0x7febc40c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.004+0000 7febca600640 1 --2- 172.21.15.67:0/2737386960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc40c5f10 0x7febc40c62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7febc40c5b50 tx=0x7febb402be40 comp rx=0 tx=0).stop 2024-03-09T08:52:15.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.004+0000 7febca600640 1 -- 172.21.15.67:0/2737386960 shutdown_connections 2024-03-09T08:52:15.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.004+0000 7febca600640 1 --2- 172.21.15.67:0/2737386960 >> v2:172.21.15.134:3300/0 conn(0x7febc40c9490 0x7febc40cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.004+0000 7febca600640 1 --2- 172.21.15.67:0/2737386960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc406b040 0x7febc41567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.004+0000 7febca600640 1 --2- 172.21.15.67:0/2737386960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc40c5f10 0x7febc40c62f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.004+0000 7febca600640 1 -- 172.21.15.67:0/2737386960 >> 172.21.15.67:0/2737386960 conn(0x7febc40cb590 msgr2=0x7febc40cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:15.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.004+0000 7febca600640 1 -- 172.21.15.67:0/2737386960 shutdown_connections 2024-03-09T08:52:15.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.004+0000 7febca600640 1 -- 172.21.15.67:0/2737386960 wait complete. 2024-03-09T08:52:15.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.006+0000 7febca600640 1 Processor -- start 2024-03-09T08:52:15.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.006+0000 7febca600640 1 -- start start 2024-03-09T08:52:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febca600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc406b040 0x7febc414c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febca600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc40c5f10 0x7febc4148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febca600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc40c9490 0x7febc4148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febca600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc4153620 con 0x7febc40c5f10 2024-03-09T08:52:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febca600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc40c9ff0 con 0x7febc406b040 2024-03-09T08:52:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febca600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc41537a0 con 0x7febc40c9490 2024-03-09T08:52:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febc3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc40c5f10 0x7febc4148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febc3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc40c5f10 0x7febc4148aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38496/0 (socket says 172.21.15.67:38496) 2024-03-09T08:52:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febc3e00640 1 -- 172.21.15.67:0/3471335521 learned_addr learned my addr 172.21.15.67:0/3471335521 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febc9200640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc406b040 0x7febc414c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febc9c00640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc40c9490 0x7febc4148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febc3e00640 1 -- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc40c9490 msgr2=0x7febc4148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febc3e00640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc40c9490 0x7febc4148fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febc3e00640 1 -- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc406b040 msgr2=0x7febc414c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febc3e00640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc406b040 0x7febc414c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febc3e00640 1 -- 172.21.15.67:0/3471335521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc41ec410 con 0x7febc40c5f10 2024-03-09T08:52:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.007+0000 7febc9c00640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc40c9490 0x7febc4148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.008+0000 7febc9200640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc406b040 0x7febc414c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.008+0000 7febc3e00640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc40c5f10 0x7febc4148aa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7febb000d800 tx=0x7febb000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:15.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.008+0000 7febc0c00640 1 -- 172.21.15.67:0/3471335521 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0018020 con 0x7febc40c5f10 2024-03-09T08:52:15.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.008+0000 7febc0c00640 1 -- 172.21.15.67:0/3471335521 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb00096a0 con 0x7febc40c5f10 2024-03-09T08:52:15.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.008+0000 7febc0c00640 1 -- 172.21.15.67:0/3471335521 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0009a90 con 0x7febc40c5f10 2024-03-09T08:52:15.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.008+0000 7febca600640 1 -- 172.21.15.67:0/3471335521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc41ec6f0 con 0x7febc40c5f10 2024-03-09T08:52:15.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.008+0000 7febca600640 1 -- 172.21.15.67:0/3471335521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc414ce70 con 0x7febc40c5f10 2024-03-09T08:52:15.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.009+0000 7febca600640 1 -- 172.21.15.67:0/3471335521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc404f3c0 con 0x7febc40c5f10 2024-03-09T08:52:15.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.010+0000 7febc0c00640 1 -- 172.21.15.67:0/3471335521 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7febb001b020 con 0x7febc40c5f10 2024-03-09T08:52:15.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.011+0000 7febc0c00640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feba406b4d0 0x7feba406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.011+0000 7febc0c00640 1 -- 172.21.15.67:0/3471335521 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7febb008a210 con 0x7febc40c5f10 2024-03-09T08:52:15.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.012+0000 7febc9200640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feba406b4d0 0x7feba406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.013+0000 7febc9200640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feba406b4d0 0x7feba406d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7febb4035000 tx=0x7febb4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:15.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.015+0000 7febc0c00640 1 -- 172.21.15.67:0/3471335521 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7febb0022030 con 0x7febc40c5f10 2024-03-09T08:52:15.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.221+0000 7febca600640 1 -- 172.21.15.67:0/3471335521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febc40cc260 con 0x7febc40c5f10 2024-03-09T08:52:15.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.222+0000 7febc0c00640 1 -- 172.21.15.67:0/3471335521 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7febb0059340 con 0x7febc40c5f10 2024-03-09T08:52:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-09T08:52:15.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 -- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feba406b4d0 msgr2=0x7feba406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feba406b4d0 0x7feba406d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7febb4035000 tx=0x7febb4030000 comp rx=0 tx=0).stop 2024-03-09T08:52:15.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 -- 172.21.15.67:0/3471335521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc40c5f10 msgr2=0x7febc4148aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc40c5f10 0x7febc4148aa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7febb000d800 tx=0x7febb000dcc0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 -- 172.21.15.67:0/3471335521 shutdown_connections 2024-03-09T08:52:15.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feba406b4d0 0x7feba406d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc40c9490 0x7febc4148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc40c5f10 0x7febc4148aa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 --2- 172.21.15.67:0/3471335521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc406b040 0x7febc414c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 -- 172.21.15.67:0/3471335521 >> 172.21.15.67:0/3471335521 conn(0x7febc40cb590 msgr2=0x7febc40caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:15.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 -- 172.21.15.67:0/3471335521 shutdown_connections 2024-03-09T08:52:15.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.225+0000 7febca600640 1 -- 172.21.15.67:0/3471335521 wait complete. 2024-03-09T08:52:15.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-09T08:52:15.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.441+0000 7f1b72600640 1 Processor -- start 2024-03-09T08:52:15.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.441+0000 7f1b72600640 1 -- start start 2024-03-09T08:52:15.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.443+0000 7f1b72600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1b6c0710a0 0x7f1b6c153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.443+0000 7f1b72600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6c0c81e0 con 0x7f1b6c0c1f70 2024-03-09T08:52:15.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.443+0000 7f1b72600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6c0c32b0 con 0x7f1b6c151360 2024-03-09T08:52:15.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.443+0000 7f1b72600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6c0c2fd0 con 0x7f1b6c0710a0 2024-03-09T08:52:15.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.443+0000 7f1b71200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1b6c0c1f70 0x7f1b6c150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40980/0 (socket says 172.21.15.67:40980) 2024-03-09T08:52:15.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.443+0000 7f1b71200640 1 -- 172.21.15.67:0/3028463658 learned_addr learned my addr 172.21.15.67:0/3028463658 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:15.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.443+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625488313 0 0) 0x7f1b6c0c81e0 con 0x7f1b6c0c1f70 2024-03-09T08:52:15.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.443+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b54003520 con 0x7f1b6c0c1f70 2024-03-09T08:52:15.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.443+0000 7f1b6be00640 1 --2- 172.21.15.67:0/3028463658 >> v2:172.21.15.134:3300/0 conn(0x7f1b6c0710a0 0x7f1b6c153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1082535885 0 0) 0x7f1b54003520 con 0x7f1b6c0c1f70 2024-03-09T08:52:15.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 >> v2:172.21.15.134:3300/0 conn(0x7f1b6c0710a0 msgr2=0x7f1b6c153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:15.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 --2- 172.21.15.67:0/3028463658 >> v2:172.21.15.134:3300/0 conn(0x7f1b6c0710a0 0x7f1b6c153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 >> v1:172.21.15.134:6789/0 conn(0x7f1b6c151360 legacy=0x7f1b6c15e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c0c3700 con 0x7f1b6c0c1f70 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f1b60001670 con 0x7f1b6c0c1f70 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 >> v1:172.21.15.67:6789/0 conn(0x7f1b6c0c1f70 legacy=0x7f1b6c150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b54003a20 0x7f1b54003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b54004800 0x7f1b54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b540052e0 0x7f1b5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c0c2fd0 con 0x7f1b54004800 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5400c230 con 0x7f1b54003a20 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5400c430 con 0x7f1b540052e0 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b6be00640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b54004800 0x7f1b54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.444+0000 7f1b71200640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b54003a20 0x7f1b54003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.445+0000 7f1b71c00640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b540052e0 0x7f1b5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.445+0000 7f1b6be00640 1 -- 172.21.15.67:0/3028463658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b540052e0 msgr2=0x7f1b5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.445+0000 7f1b6be00640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b540052e0 0x7f1b5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.445+0000 7f1b6be00640 1 -- 172.21.15.67:0/3028463658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b54003a20 msgr2=0x7f1b54003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.445+0000 7f1b6be00640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b54003a20 0x7f1b54003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.445+0000 7f1b6be00640 1 -- 172.21.15.67:0/3028463658 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b54003520 con 0x7f1b54004800 2024-03-09T08:52:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.445+0000 7f1b71c00640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b540052e0 0x7f1b5400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.445+0000 7f1b71c00640 1 -- 172.21.15.67:0/3028463658 reap_dead start 2024-03-09T08:52:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.446+0000 7f1b6be00640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b54004800 0x7f1b54004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800ba00 tx=0x7f1b54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b536887f4c5733 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.446+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800b600 con 0x7f1b54004800 2024-03-09T08:52:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.446+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b58038070 con 0x7f1b54004800 2024-03-09T08:52:15.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.446+0000 7f1b6b400640 1 -- 172.21.15.67:0/3028463658 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b580338a0 con 0x7f1b54004800 2024-03-09T08:52:15.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.447+0000 7f1b72600640 1 -- 172.21.15.67:0/3028463658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b54004800 msgr2=0x7f1b54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.447+0000 7f1b72600640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b54004800 0x7f1b54004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800ba00 tx=0x7f1b54005830 comp rx=0 tx=0).stop 2024-03-09T08:52:15.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.447+0000 7f1b72600640 1 -- 172.21.15.67:0/3028463658 shutdown_connections 2024-03-09T08:52:15.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.447+0000 7f1b72600640 1 --2- 172.21.15.67:0/3028463658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b54004800 0x7f1b54004c80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.447+0000 7f1b72600640 1 -- 172.21.15.67:0/3028463658 >> 172.21.15.67:0/3028463658 conn(0x7f1b6c0c7300 msgr2=0x7f1b6c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.447+0000 7f1b72600640 1 -- 172.21.15.67:0/3028463658 shutdown_connections 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.447+0000 7f1b72600640 1 -- 172.21.15.67:0/3028463658 wait complete. 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.449+0000 7f1b72600640 1 Processor -- start 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.449+0000 7f1b72600640 1 -- start start 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.450+0000 7f1b72600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b54004800 0x7f1b6c1517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.450+0000 7f1b72600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b6c0ab640 0x7f1b6c14a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.450+0000 7f1b72600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b6c14a9b0 0x7f1b6c1ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.450+0000 7f1b72600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c070d50 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.450+0000 7f1b72600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c070830 con 0x7f1b6c0ab640 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.450+0000 7f1b72600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c070ed0 con 0x7f1b54004800 2024-03-09T08:52:15.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.450+0000 7f1b71c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b6c14a9b0 0x7f1b6c1ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.450+0000 7f1b71c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b6c14a9b0 0x7f1b6c1ee9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38512/0 (socket says 172.21.15.67:38512) 2024-03-09T08:52:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.450+0000 7f1b71c00640 1 -- 172.21.15.67:0/3752470782 learned_addr learned my addr 172.21.15.67:0/3752470782 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.450+0000 7f1b71200640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b54004800 0x7f1b6c1517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.450+0000 7f1b6be00640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b6c0ab640 0x7f1b6c14a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b71c00640 1 -- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b54004800 msgr2=0x7f1b6c1517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b71c00640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b54004800 0x7f1b6c1517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b71c00640 1 -- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b6c0ab640 msgr2=0x7f1b6c14a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b71c00640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b6c0ab640 0x7f1b6c14a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b71c00640 1 -- 172.21.15.67:0/3752470782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c151f70 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b6be00640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b6c0ab640 0x7f1b6c14a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b71c00640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b6c14a9b0 0x7f1b6c1ee9a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c06c430 tx=0x7f1b6400d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b68c00640 1 -- 172.21.15.67:0/3752470782 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64018070 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b68c00640 1 -- 172.21.15.67:0/3752470782 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b64003da0 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b72600640 1 -- 172.21.15.67:0/3752470782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c154640 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b68c00640 1 -- 172.21.15.67:0/3752470782 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64005330 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.451+0000 7f1b72600640 1 -- 172.21.15.67:0/3752470782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c152c80 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.452+0000 7f1b72600640 1 -- 172.21.15.67:0/3752470782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c04f3c0 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.453+0000 7f1b68c00640 1 -- 172.21.15.67:0/3752470782 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1b6401b070 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.454+0000 7f1b68c00640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b4c06b4d0 0x7f1b4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.454+0000 7f1b68c00640 1 -- 172.21.15.67:0/3752470782 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1b640895a0 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.455+0000 7f1b71200640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b4c06b4d0 0x7f1b4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.456+0000 7f1b71200640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b4c06b4d0 0x7f1b4c06d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0aa070 tx=0x7f1b60009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:15.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.459+0000 7f1b68c00640 1 -- 172.21.15.67:0/3752470782 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1b64058800 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.659+0000 7f1b72600640 1 -- 172.21.15.67:0/3752470782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1b6c150d60 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.660+0000 7f1b68c00640 1 -- 172.21.15.67:0/3752470782 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1b64058100 con 0x7f1b6c14a9b0 2024-03-09T08:52:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:15.655 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-03-09T08:52:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:52:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-03-09T08:52:15.656 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-03-09T08:52:15.656 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:52:15.656 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-03-09T08:52:15.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.662+0000 7f1b72600640 1 -- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b4c06b4d0 msgr2=0x7f1b4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.662+0000 7f1b72600640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b4c06b4d0 0x7f1b4c06d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0aa070 tx=0x7f1b60009040 comp rx=0 tx=0).stop 2024-03-09T08:52:15.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.662+0000 7f1b72600640 1 -- 172.21.15.67:0/3752470782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b6c14a9b0 msgr2=0x7f1b6c1ee9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.662+0000 7f1b72600640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b6c14a9b0 0x7f1b6c1ee9a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c06c430 tx=0x7f1b6400d9f0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.662+0000 7f1b72600640 1 -- 172.21.15.67:0/3752470782 shutdown_connections 2024-03-09T08:52:15.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.663+0000 7f1b72600640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b4c06b4d0 0x7f1b4c06d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.663+0000 7f1b72600640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b6c14a9b0 0x7f1b6c1ee9a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.663+0000 7f1b72600640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b6c0ab640 0x7f1b6c14a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.663+0000 7f1b72600640 1 --2- 172.21.15.67:0/3752470782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b54004800 0x7f1b6c1517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.663+0000 7f1b72600640 1 -- 172.21.15.67:0/3752470782 >> 172.21.15.67:0/3752470782 conn(0x7f1b6c0c7300 msgr2=0x7f1b6c0728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:15.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.663+0000 7f1b72600640 1 -- 172.21.15.67:0/3752470782 shutdown_connections 2024-03-09T08:52:15.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.663+0000 7f1b72600640 1 -- 172.21.15.67:0/3752470782 wait complete. 2024-03-09T08:52:15.677 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'a' 2024-03-09T08:52:15.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create a_metadata --pg_num_min 64 2024-03-09T08:52:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.912+0000 7fd0b1600640 1 Processor -- start 2024-03-09T08:52:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.912+0000 7fd0b1600640 1 -- start start 2024-03-09T08:52:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.914+0000 7fd0b1600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0ac06dac0 0x7fd0ac0c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.914+0000 7fd0b1600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0ac06d3b0 0x7fd0ac0c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.914+0000 7fd0b1600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ac06e420 con 0x7fd0ac150fc0 2024-03-09T08:52:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.914+0000 7fd0b1600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac06e6c0 con 0x7fd0ac06dac0 2024-03-09T08:52:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.914+0000 7fd0b1600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac0706d0 con 0x7fd0ac06d3b0 2024-03-09T08:52:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.914+0000 7fd0ab400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0ac06dac0 0x7fd0ac0c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.914+0000 7fd0ab400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0ac06dac0 0x7fd0ac0c2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38528/0 (socket says 172.21.15.67:38528) 2024-03-09T08:52:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.914+0000 7fd0ab400640 1 -- 172.21.15.67:0/1179986302 learned_addr learned my addr 172.21.15.67:0/1179986302 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.914+0000 7fd0b0c00640 1 --2- 172.21.15.67:0/1179986302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0ac06d3b0 0x7fd0ac0c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.915+0000 7fd0ab400640 1 -- 172.21.15.67:0/1179986302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0ac06d3b0 msgr2=0x7fd0ac0c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.915+0000 7fd0ab400640 1 --2- 172.21.15.67:0/1179986302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0ac06d3b0 0x7fd0ac0c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.915+0000 7fd0ab400640 1 -- 172.21.15.67:0/1179986302 >> v1:172.21.15.134:6789/0 conn(0x7fd0ac150fc0 legacy=0x7fd0ac0ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.915+0000 7fd0ab400640 1 -- 172.21.15.67:0/1179986302 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac0709b0 con 0x7fd0ac06dac0 2024-03-09T08:52:15.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.916+0000 7fd0ab400640 1 --2- 172.21.15.67:0/1179986302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ac06dac0 0x7fd0ac0c2ca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd098009870 tx=0x7fd09802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593197cf29067292 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:15.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.916+0000 7fd0aaa00640 1 -- 172.21.15.67:0/1179986302 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098035020 con 0x7fd0ac06dac0 2024-03-09T08:52:15.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.916+0000 7fd0aaa00640 1 -- 172.21.15.67:0/1179986302 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd098031640 con 0x7fd0ac06dac0 2024-03-09T08:52:15.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.916+0000 7fd0aaa00640 1 -- 172.21.15.67:0/1179986302 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09802d040 con 0x7fd0ac06dac0 2024-03-09T08:52:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.917+0000 7fd0b1600640 1 -- 172.21.15.67:0/1179986302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ac06dac0 msgr2=0x7fd0ac0c2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.917+0000 7fd0b1600640 1 --2- 172.21.15.67:0/1179986302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ac06dac0 0x7fd0ac0c2ca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd098009870 tx=0x7fd09802b8f0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.917+0000 7fd0b1600640 1 -- 172.21.15.67:0/1179986302 shutdown_connections 2024-03-09T08:52:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.917+0000 7fd0b1600640 1 --2- 172.21.15.67:0/1179986302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0ac06d3b0 0x7fd0ac0c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.917+0000 7fd0b1600640 1 --2- 172.21.15.67:0/1179986302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ac06dac0 0x7fd0ac0c2ca0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.917+0000 7fd0b1600640 1 -- 172.21.15.67:0/1179986302 >> 172.21.15.67:0/1179986302 conn(0x7fd0ac0c3d90 msgr2=0x7fd0ac0c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.917+0000 7fd0b1600640 1 -- 172.21.15.67:0/1179986302 shutdown_connections 2024-03-09T08:52:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.917+0000 7fd0b1600640 1 -- 172.21.15.67:0/1179986302 wait complete. 2024-03-09T08:52:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.919+0000 7fd0b1600640 1 Processor -- start 2024-03-09T08:52:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.919+0000 7fd0b1600640 1 -- start start 2024-03-09T08:52:15.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.920+0000 7fd0b1600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ac06d3b0 0x7fd0ac152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.920+0000 7fd0b1600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0ac06dac0 0x7fd0ac0c0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.920+0000 7fd0b1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ac150fc0 0x7fd0ac0aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.920+0000 7fd0b1600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac06e200 con 0x7fd0ac150fc0 2024-03-09T08:52:15.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.920+0000 7fd0b1600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac06e080 con 0x7fd0ac06d3b0 2024-03-09T08:52:15.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.920+0000 7fd0b1600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac0acdd0 con 0x7fd0ac06dac0 2024-03-09T08:52:15.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.920+0000 7fd0b0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ac150fc0 0x7fd0ac0aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.920+0000 7fd0b0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ac150fc0 0x7fd0ac0aada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38536/0 (socket says 172.21.15.67:38536) 2024-03-09T08:52:15.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.920+0000 7fd0b0c00640 1 -- 172.21.15.67:0/708586446 learned_addr learned my addr 172.21.15.67:0/708586446 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0ab400640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0ac06dac0 0x7fd0ac0c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0abe00640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ac06d3b0 0x7fd0ac152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0b0c00640 1 -- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0ac06dac0 msgr2=0x7fd0ac0c0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0b0c00640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0ac06dac0 0x7fd0ac0c0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0b0c00640 1 -- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ac06d3b0 msgr2=0x7fd0ac152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0b0c00640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ac06d3b0 0x7fd0ac152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0b0c00640 1 -- 172.21.15.67:0/708586446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac1ef950 con 0x7fd0ac150fc0 2024-03-09T08:52:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0abe00640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ac06d3b0 0x7fd0ac152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0b0c00640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ac150fc0 0x7fd0ac0aada0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4007e40 tx=0x7fd0a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0a3e00640 1 -- 172.21.15.67:0/708586446 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4014620 con 0x7fd0ac150fc0 2024-03-09T08:52:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0a3e00640 1 -- 172.21.15.67:0/708586446 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a4019070 con 0x7fd0ac150fc0 2024-03-09T08:52:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0a3e00640 1 -- 172.21.15.67:0/708586446 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4014a20 con 0x7fd0ac150fc0 2024-03-09T08:52:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.921+0000 7fd0b1600640 1 -- 172.21.15.67:0/708586446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ac0aa5e0 con 0x7fd0ac150fc0 2024-03-09T08:52:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.922+0000 7fd0b1600640 1 -- 172.21.15.67:0/708586446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ac0c0e70 con 0x7fd0ac150fc0 2024-03-09T08:52:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.923+0000 7fd0b1600640 1 -- 172.21.15.67:0/708586446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ac04f430 con 0x7fd0ac150fc0 2024-03-09T08:52:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.924+0000 7fd0a3e00640 1 -- 172.21.15.67:0/708586446 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd0a4014bc0 con 0x7fd0ac150fc0 2024-03-09T08:52:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.925+0000 7fd0a3e00640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08c06b500 0x7fd08c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.925+0000 7fd0a3e00640 1 -- 172.21.15.67:0/708586446 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd0a408a790 con 0x7fd0ac150fc0 2024-03-09T08:52:15.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.925+0000 7fd0abe00640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08c06b500 0x7fd08c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:15.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.926+0000 7fd0abe00640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08c06b500 0x7fd08c06d9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd09c000f60 tx=0x7fd09c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:15.929+0000 7fd0a3e00640 1 -- 172.21.15.67:0/708586446 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0a4059950 con 0x7fd0ac150fc0 2024-03-09T08:52:16.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.130+0000 7fd0b1600640 1 -- 172.21.15.67:0/708586446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "a_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd0ac06c540 con 0x7fd0ac150fc0 2024-03-09T08:52:16.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.239+0000 7fd0a3e00640 1 -- 172.21.15.67:0/708586446 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "a_metadata", "pg_num_min": 64}]=0 pool 'a_metadata' created v17) v1 ==== 128+0+0 (secure 0 0 0) 0x7fd0a4059250 con 0x7fd0ac150fc0 2024-03-09T08:52:16.235 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'a_metadata' created 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.242+0000 7fd0b1600640 1 -- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08c06b500 msgr2=0x7fd08c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.242+0000 7fd0b1600640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08c06b500 0x7fd08c06d9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd09c000f60 tx=0x7fd09c009040 comp rx=0 tx=0).stop 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.242+0000 7fd0b1600640 1 -- 172.21.15.67:0/708586446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ac150fc0 msgr2=0x7fd0ac0aada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.242+0000 7fd0b1600640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ac150fc0 0x7fd0ac0aada0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4007e40 tx=0x7fd0a400e4b0 comp rx=0 tx=0).stop 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.242+0000 7fd0b1600640 1 -- 172.21.15.67:0/708586446 shutdown_connections 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.243+0000 7fd0b1600640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08c06b500 0x7fd08c06d9c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.243+0000 7fd0b1600640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0ac150fc0 0x7fd0ac0aada0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.243+0000 7fd0b1600640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0ac06dac0 0x7fd0ac0c0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.243+0000 7fd0b1600640 1 --2- 172.21.15.67:0/708586446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0ac06d3b0 0x7fd0ac152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.243+0000 7fd0b1600640 1 -- 172.21.15.67:0/708586446 >> 172.21.15.67:0/708586446 conn(0x7fd0ac0c3d90 msgr2=0x7fd0ac149560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.243+0000 7fd0b1600640 1 -- 172.21.15.67:0/708586446 shutdown_connections 2024-03-09T08:52:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.243+0000 7fd0b1600640 1 -- 172.21.15.67:0/708586446 wait complete. 2024-03-09T08:52:16.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create a_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-09T08:52:16.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.474+0000 7fdb46a00640 1 Processor -- start 2024-03-09T08:52:16.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.474+0000 7fdb46a00640 1 -- start start 2024-03-09T08:52:16.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.476+0000 7fdb46a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdb4006c460 0x7fdb401533f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:16.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.476+0000 7fdb46a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb4006ca50 0x7fdb401594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:16.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.476+0000 7fdb46a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb400c3900 con 0x7fdb400ac310 2024-03-09T08:52:16.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.476+0000 7fdb46a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb400c36b0 con 0x7fdb4006c460 2024-03-09T08:52:16.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.476+0000 7fdb46a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400c3be0 con 0x7fdb4006ca50 2024-03-09T08:52:16.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.476+0000 7fdb45600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdb400ac310 0x7fdb40152ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40992/0 (socket says 172.21.15.67:40992) 2024-03-09T08:52:16.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.476+0000 7fdb45600640 1 -- 172.21.15.67:0/280006390 learned_addr learned my addr 172.21.15.67:0/280006390 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:16.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb44c00640 1 --2- 172.21.15.67:0/280006390 >> v2:172.21.15.134:3300/0 conn(0x7fdb4006c460 0x7fdb401533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:16.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402235311 0 0) 0x7fdb400c3900 con 0x7fdb400ac310 2024-03-09T08:52:16.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb28003520 con 0x7fdb400ac310 2024-03-09T08:52:16.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb46000640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb4006ca50 0x7fdb401594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:16.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1832766479 0 0) 0x7fdb28003520 con 0x7fdb400ac310 2024-03-09T08:52:16.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb4006ca50 msgr2=0x7fdb401594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:16.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb4006ca50 0x7fdb401594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:16.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 >> v2:172.21.15.134:3300/0 conn(0x7fdb4006c460 msgr2=0x7fdb401533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:16.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 --2- 172.21.15.67:0/280006390 >> v2:172.21.15.134:3300/0 conn(0x7fdb4006c460 0x7fdb401533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:16.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb400aa4e0 con 0x7fdb400ac310 2024-03-09T08:52:16.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fdb34001670 con 0x7fdb400ac310 2024-03-09T08:52:16.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 >> v1:172.21.15.67:6789/0 conn(0x7fdb400ac310 legacy=0x7fdb40152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:16.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb4006ca50 0x7fdb280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:16.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb28003f30 0x7fdb28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:16.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb28004d40 0x7fdb280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:16.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400c36b0 con 0x7fdb28003f30 2024-03-09T08:52:16.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400c3be0 con 0x7fdb28004d40 2024-03-09T08:52:16.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.477+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2800bb50 con 0x7fdb4006ca50 2024-03-09T08:52:16.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.478+0000 7fdb44c00640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb28003f30 0x7fdb28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:16.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.478+0000 7fdb46000640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb28004d40 0x7fdb280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:16.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.478+0000 7fdb45600640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb4006ca50 0x7fdb280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:16.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.478+0000 7fdb44c00640 1 -- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb4006ca50 msgr2=0x7fdb280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:16.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.478+0000 7fdb44c00640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb4006ca50 0x7fdb280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:16.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.478+0000 7fdb44c00640 1 -- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb28004d40 msgr2=0x7fdb280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:16.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.478+0000 7fdb44c00640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb28004d40 0x7fdb280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:16.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.478+0000 7fdb44c00640 1 -- 172.21.15.67:0/280006390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28003520 con 0x7fdb28003f30 2024-03-09T08:52:16.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.478+0000 7fdb45600640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb4006ca50 0x7fdb280039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:16.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.480+0000 7fdb44c00640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb28003f30 0x7fdb28004800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009a90 tx=0x7fdb28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293fdc741a45c324 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:16.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.480+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c039070 con 0x7fdb28003f30 2024-03-09T08:52:16.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.480+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c02c6b0 con 0x7fdb28003f30 2024-03-09T08:52:16.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.480+0000 7fdb3fe00640 1 -- 172.21.15.67:0/280006390 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c009690 con 0x7fdb28003f30 2024-03-09T08:52:16.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.481+0000 7fdb46a00640 1 -- 172.21.15.67:0/280006390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb28003f30 msgr2=0x7fdb28004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:16.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.481+0000 7fdb46a00640 1 --2- 172.21.15.67:0/280006390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb28003f30 0x7fdb28004800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009a90 tx=0x7fdb28005830 comp rx=0 tx=0).stop 2024-03-09T08:52:16.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.481+0000 7fdb46000640 1 -- 172.21.15.67:0/280006390 reap_dead start 2024-03-09T08:52:16.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.481+0000 7fdb46a00640 1 -- 172.21.15.67:0/280006390 shutdown_connections 2024-03-09T08:52:16.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.481+0000 7fdb46a00640 1 -- 172.21.15.67:0/280006390 >> 172.21.15.67:0/280006390 conn(0x7fdb400c3d90 msgr2=0x7fdb400c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:16.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.481+0000 7fdb46a00640 1 -- 172.21.15.67:0/280006390 shutdown_connections 2024-03-09T08:52:16.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.481+0000 7fdb46a00640 1 -- 172.21.15.67:0/280006390 wait complete. 2024-03-09T08:52:16.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.482+0000 7fdb46a00640 1 Processor -- start 2024-03-09T08:52:16.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.483+0000 7fdb46a00640 1 -- start start 2024-03-09T08:52:16.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.483+0000 7fdb46a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb400c6650 0x7fdb401422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:16.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.483+0000 7fdb46a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb400c6e80 0x7fdb40142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:16.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb46a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb401e0f50 0x7fdb401eb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:16.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb46a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400ac870 con 0x7fdb400c6650 2024-03-09T08:52:16.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb46a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400ac6f0 con 0x7fdb400c6e80 2024-03-09T08:52:16.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb46a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb401594b0 con 0x7fdb401e0f50 2024-03-09T08:52:16.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb45600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb400c6650 0x7fdb401422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:16.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb45600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb400c6650 0x7fdb401422f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38546/0 (socket says 172.21.15.67:38546) 2024-03-09T08:52:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb45600640 1 -- 172.21.15.67:0/4292790172 learned_addr learned my addr 172.21.15.67:0/4292790172 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb44c00640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb400c6e80 0x7fdb40142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb46000640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb401e0f50 0x7fdb401eb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb45600640 1 -- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb401e0f50 msgr2=0x7fdb401eb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb45600640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb401e0f50 0x7fdb401eb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb45600640 1 -- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb400c6e80 msgr2=0x7fdb40142830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb45600640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb400c6e80 0x7fdb40142830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.484+0000 7fdb45600640 1 -- 172.21.15.67:0/4292790172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb401eeab0 con 0x7fdb400c6650 2024-03-09T08:52:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.485+0000 7fdb45600640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb400c6650 0x7fdb401422f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdb34001120 tx=0x7fdb3400df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.485+0000 7fdb3d600640 1 -- 172.21.15.67:0/4292790172 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34019070 con 0x7fdb400c6650 2024-03-09T08:52:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.485+0000 7fdb3d600640 1 -- 172.21.15.67:0/4292790172 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34004cb0 con 0x7fdb400c6650 2024-03-09T08:52:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.485+0000 7fdb3d600640 1 -- 172.21.15.67:0/4292790172 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34004e60 con 0x7fdb400c6650 2024-03-09T08:52:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.485+0000 7fdb46a00640 1 -- 172.21.15.67:0/4292790172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb401eed50 con 0x7fdb400c6650 2024-03-09T08:52:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.485+0000 7fdb46a00640 1 -- 172.21.15.67:0/4292790172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb401ef310 con 0x7fdb400c6650 2024-03-09T08:52:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.486+0000 7fdb46a00640 1 -- 172.21.15.67:0/4292790172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4004f430 con 0x7fdb400c6650 2024-03-09T08:52:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.488+0000 7fdb3d600640 1 -- 172.21.15.67:0/4292790172 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdb340072d0 con 0x7fdb400c6650 2024-03-09T08:52:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.488+0000 7fdb3d600640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb2006b500 0x7fdb2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.489+0000 7fdb3d600640 1 -- 172.21.15.67:0/4292790172 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5222+0+0 (secure 0 0 0) 0x7fdb34089e40 con 0x7fdb400c6650 2024-03-09T08:52:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.489+0000 7fdb44c00640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb2006b500 0x7fdb2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.490+0000 7fdb44c00640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb2006b500 0x7fdb2006d9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c036000 tx=0x7fdb2c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:16.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.494+0000 7fdb3d600640 1 -- 172.21.15.67:0/4292790172 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdb34058e90 con 0x7fdb400c6650 2024-03-09T08:52:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:16.700+0000 7fdb46a00640 1 -- 172.21.15.67:0/4292790172 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "a_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdb400c0910 con 0x7fdb400c6650 2024-03-09T08:52:17.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.246+0000 7fdb3d600640 1 -- 172.21.15.67:0/4292790172 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "a_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'a_data' created v18) v1 ==== 160+0+0 (secure 0 0 0) 0x7fdb34058790 con 0x7fdb400c6650 2024-03-09T08:52:17.242 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'a_data' created 2024-03-09T08:52:17.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.250+0000 7fdb46a00640 1 -- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb2006b500 msgr2=0x7fdb2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:17.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.250+0000 7fdb46a00640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb2006b500 0x7fdb2006d9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c036000 tx=0x7fdb2c030000 comp rx=0 tx=0).stop 2024-03-09T08:52:17.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.250+0000 7fdb46a00640 1 -- 172.21.15.67:0/4292790172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb400c6650 msgr2=0x7fdb401422f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:17.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.250+0000 7fdb46a00640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb400c6650 0x7fdb401422f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdb34001120 tx=0x7fdb3400df70 comp rx=0 tx=0).stop 2024-03-09T08:52:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.251+0000 7fdb46a00640 1 -- 172.21.15.67:0/4292790172 shutdown_connections 2024-03-09T08:52:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.251+0000 7fdb46a00640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb2006b500 0x7fdb2006d9c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.251+0000 7fdb46a00640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb401e0f50 0x7fdb401eb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.251+0000 7fdb46a00640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb400c6e80 0x7fdb40142830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.251+0000 7fdb46a00640 1 --2- 172.21.15.67:0/4292790172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb400c6650 0x7fdb401422f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.251+0000 7fdb46a00640 1 -- 172.21.15.67:0/4292790172 >> 172.21.15.67:0/4292790172 conn(0x7fdb400c3d90 msgr2=0x7fdb400c1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:17.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.251+0000 7fdb46a00640 1 -- 172.21.15.67:0/4292790172 shutdown_connections 2024-03-09T08:52:17.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.252+0000 7fdb46a00640 1 -- 172.21.15.67:0/4292790172 wait complete. 2024-03-09T08:52:17.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new a a_metadata a_data 2024-03-09T08:52:17.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.475+0000 7f738b200640 1 Processor -- start 2024-03-09T08:52:17.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.476+0000 7f738b200640 1 -- start start 2024-03-09T08:52:17.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.477+0000 7f738b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7384073830 0x7f73840701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:17.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.477+0000 7f738b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7384156d30 con 0x7f7384153510 2024-03-09T08:52:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.477+0000 7f738b200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73840c7640 con 0x7f73841568c0 2024-03-09T08:52:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.477+0000 7f738b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73840c7ee0 con 0x7f7384073830 2024-03-09T08:52:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.477+0000 7f7383e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7384153510 0x7f7384159b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41002/0 (socket says 172.21.15.67:41002) 2024-03-09T08:52:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.477+0000 7f7383e00640 1 -- 172.21.15.67:0/1564781601 learned_addr learned my addr 172.21.15.67:0/1564781601 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.477+0000 7f7383400640 1 --2- 172.21.15.67:0/1564781601 >> v2:172.21.15.67:3300/0 conn(0x7f7384073830 0x7f73840701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:17.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.478+0000 7f7382a00640 1 -- 172.21.15.67:0/1564781601 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347221336 0 0) 0x7f7384156d30 con 0x7f7384153510 2024-03-09T08:52:17.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.478+0000 7f7382a00640 1 -- 172.21.15.67:0/1564781601 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7368003520 con 0x7f7384153510 2024-03-09T08:52:17.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.478+0000 7f7383400640 1 -- 172.21.15.67:0/1564781601 >> v1:172.21.15.134:6789/0 conn(0x7f73841568c0 legacy=0x7f73840c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:17.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.478+0000 7f7383400640 1 -- 172.21.15.67:0/1564781601 >> v1:172.21.15.67:6789/0 conn(0x7f7384153510 legacy=0x7f7384159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:17.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.478+0000 7f7383400640 1 -- 172.21.15.67:0/1564781601 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73840c7920 con 0x7f7384073830 2024-03-09T08:52:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.479+0000 7f7383400640 1 --2- 172.21.15.67:0/1564781601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7384073830 0x7f73840701d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f73700097a0 tx=0x7f737002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9487ee3a7e62527 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.479+0000 7f7382a00640 1 -- 172.21.15.67:0/1564781601 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73700315a0 con 0x7f7384073830 2024-03-09T08:52:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.479+0000 7f7382a00640 1 -- 172.21.15.67:0/1564781601 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7370037070 con 0x7f7384073830 2024-03-09T08:52:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.479+0000 7f7382a00640 1 -- 172.21.15.67:0/1564781601 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370031960 con 0x7f7384073830 2024-03-09T08:52:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.480+0000 7f738b200640 1 -- 172.21.15.67:0/1564781601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7384073830 msgr2=0x7f73840701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.480+0000 7f738b200640 1 --2- 172.21.15.67:0/1564781601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7384073830 0x7f73840701d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f73700097a0 tx=0x7f737002b840 comp rx=0 tx=0).stop 2024-03-09T08:52:17.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.480+0000 7f738b200640 1 -- 172.21.15.67:0/1564781601 shutdown_connections 2024-03-09T08:52:17.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.480+0000 7f738b200640 1 --2- 172.21.15.67:0/1564781601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7384073830 0x7f73840701d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:17.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.480+0000 7f738b200640 1 -- 172.21.15.67:0/1564781601 >> 172.21.15.67:0/1564781601 conn(0x7f7384067180 msgr2=0x7f7384067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:17.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.480+0000 7f738b200640 1 -- 172.21.15.67:0/1564781601 shutdown_connections 2024-03-09T08:52:17.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.480+0000 7f738b200640 1 -- 172.21.15.67:0/1564781601 wait complete. 2024-03-09T08:52:17.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.482+0000 7f738b200640 1 Processor -- start 2024-03-09T08:52:17.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.482+0000 7f738b200640 1 -- start start 2024-03-09T08:52:17.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f738b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7384073830 0x7f738414c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f738b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7384153510 0x7f73841e1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f738b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73841568c0 0x7f73841e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f738b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7384159410 con 0x7f73841568c0 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f738b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73841538f0 con 0x7f7384153510 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f738b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73841596f0 con 0x7f7384073830 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f7389200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73841568c0 0x7f73841e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f7389200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73841568c0 0x7f73841e2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38568/0 (socket says 172.21.15.67:38568) 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f7389200640 1 -- 172.21.15.67:0/975784562 learned_addr learned my addr 172.21.15.67:0/975784562 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f7383400640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7384153510 0x7f73841e1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f7383e00640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7384073830 0x7f738414c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f7389200640 1 -- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7384073830 msgr2=0x7f738414c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f7389200640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7384073830 0x7f738414c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f7389200640 1 -- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7384153510 msgr2=0x7f73841e1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f7389200640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7384153510 0x7f73841e1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.483+0000 7f7389200640 1 -- 172.21.15.67:0/975784562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738406dbe0 con 0x7f73841568c0 2024-03-09T08:52:17.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.484+0000 7f7389200640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73841568c0 0x7f73841e2ac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f737c00c6e0 tx=0x7f737c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:17.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.484+0000 7f737be00640 1 -- 172.21.15.67:0/975784562 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c003b70 con 0x7f73841568c0 2024-03-09T08:52:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.484+0000 7f737be00640 1 -- 172.21.15.67:0/975784562 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c003d30 con 0x7f73841568c0 2024-03-09T08:52:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.484+0000 7f737be00640 1 -- 172.21.15.67:0/975784562 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c005310 con 0x7f73841568c0 2024-03-09T08:52:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.484+0000 7f738b200640 1 -- 172.21.15.67:0/975784562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738406dec0 con 0x7f73841568c0 2024-03-09T08:52:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.484+0000 7f738b200640 1 -- 172.21.15.67:0/975784562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738414bcb0 con 0x7f73841568c0 2024-03-09T08:52:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.485+0000 7f738b200640 1 -- 172.21.15.67:0/975784562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73840c7200 con 0x7f73841568c0 2024-03-09T08:52:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.486+0000 7f737be00640 1 -- 172.21.15.67:0/975784562 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f737c0054b0 con 0x7f73841568c0 2024-03-09T08:52:17.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.487+0000 7f737be00640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f736406b4d0 0x7f736406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:17.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.487+0000 7f737be00640 1 -- 172.21.15.67:0/975784562 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5599+0+0 (secure 0 0 0) 0x7f737c089fe0 con 0x7f73841568c0 2024-03-09T08:52:17.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.488+0000 7f7383e00640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f736406b4d0 0x7f736406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:17.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.489+0000 7f7383e00640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f736406b4d0 0x7f736406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f73740011e0 tx=0x7f7374009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:17.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.493+0000 7f737be00640 1 -- 172.21.15.67:0/975784562 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f73840c7200 con 0x7f73841568c0 2024-03-09T08:52:17.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:17.734+0000 7f738b200640 1 -- 172.21.15.67:0/975784562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "a", "metadata": "a_metadata", "data": "a_data"} v 0) v1 -- 0x7f738414f840 con 0x7f73841568c0 2024-03-09T08:52:18.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.265+0000 7f737be00640 1 -- 172.21.15.67:0/975784562 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "a", "metadata": "a_metadata", "data": "a_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 157+0+0 (secure 0 0 0) 0x7f738414f840 con 0x7f73841568c0 2024-03-09T08:52:18.261 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 2 and data pool 3 2024-03-09T08:52:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 -- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f736406b4d0 msgr2=0x7f736406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f736406b4d0 0x7f736406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f73740011e0 tx=0x7f7374009040 comp rx=0 tx=0).stop 2024-03-09T08:52:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 -- 172.21.15.67:0/975784562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73841568c0 msgr2=0x7f73841e2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73841568c0 0x7f73841e2ac0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f737c00c6e0 tx=0x7f737c00cba0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 -- 172.21.15.67:0/975784562 shutdown_connections 2024-03-09T08:52:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f736406b4d0 0x7f736406d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73841568c0 0x7f73841e2ac0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7384153510 0x7f73841e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 --2- 172.21.15.67:0/975784562 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7384073830 0x7f738414c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 -- 172.21.15.67:0/975784562 >> 172.21.15.67:0/975784562 conn(0x7f7384067180 msgr2=0x7f73840c68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:18.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 -- 172.21.15.67:0/975784562 shutdown_connections 2024-03-09T08:52:18.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.268+0000 7f738b200640 1 -- 172.21.15.67:0/975784562 wait complete. 2024-03-09T08:52:18.287 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:18.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.479+0000 7f1c3fa00640 1 Processor -- start 2024-03-09T08:52:18.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.479+0000 7f1c3fa00640 1 -- start start 2024-03-09T08:52:18.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.480+0000 7f1c3fa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380ac730 0x7f1c38071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.480+0000 7f1c3fa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1c380c22a0 0x7f1c380acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.480+0000 7f1c3fa00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c38153d40 con 0x7f1c38157650 2024-03-09T08:52:18.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.480+0000 7f1c3fa00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c38070ee0 con 0x7f1c380c22a0 2024-03-09T08:52:18.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.480+0000 7f1c3fa00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c38154d00 con 0x7f1c380ac730 2024-03-09T08:52:18.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.481+0000 7f1c3d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380ac730 0x7f1c38071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.481+0000 7f1c3d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380ac730 0x7f1c38071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:35714/0 (socket says 172.21.15.67:35714) 2024-03-09T08:52:18.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.481+0000 7f1c3d000640 1 -- 172.21.15.67:0/3593289637 learned_addr learned my addr 172.21.15.67:0/3593289637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:18.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.481+0000 7f1c37e00640 1 --2- 172.21.15.67:0/3593289637 >> v2:172.21.15.134:3300/0 conn(0x7f1c380c22a0 0x7f1c380acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.482+0000 7f1c3d000640 1 -- 172.21.15.67:0/3593289637 >> v2:172.21.15.134:3300/0 conn(0x7f1c380c22a0 msgr2=0x7f1c380acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.482+0000 7f1c3d000640 1 --2- 172.21.15.67:0/3593289637 >> v2:172.21.15.134:3300/0 conn(0x7f1c380c22a0 0x7f1c380acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.482+0000 7f1c3d000640 1 -- 172.21.15.67:0/3593289637 >> v1:172.21.15.134:6789/0 conn(0x7f1c38157650 legacy=0x7f1c38157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.482+0000 7f1c3d000640 1 -- 172.21.15.67:0/3593289637 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c380c2790 con 0x7f1c380ac730 2024-03-09T08:52:18.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.483+0000 7f1c3d000640 1 --2- 172.21.15.67:0/3593289637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380ac730 0x7f1c38071190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009870 tx=0x7f1c2c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c67ea3e12e7ade22 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:18.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.484+0000 7f1c37400640 1 -- 172.21.15.67:0/3593289637 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c035020 con 0x7f1c380ac730 2024-03-09T08:52:18.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.484+0000 7f1c37400640 1 -- 172.21.15.67:0/3593289637 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2c031640 con 0x7f1c380ac730 2024-03-09T08:52:18.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.484+0000 7f1c37400640 1 -- 172.21.15.67:0/3593289637 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c02d040 con 0x7f1c380ac730 2024-03-09T08:52:18.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.484+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3593289637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380ac730 msgr2=0x7f1c38071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.484+0000 7f1c3fa00640 1 --2- 172.21.15.67:0/3593289637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380ac730 0x7f1c38071190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009870 tx=0x7f1c2c02b8f0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.484+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3593289637 shutdown_connections 2024-03-09T08:52:18.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.484+0000 7f1c3fa00640 1 --2- 172.21.15.67:0/3593289637 >> v2:172.21.15.134:3300/0 conn(0x7f1c380c22a0 0x7f1c380acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.484+0000 7f1c3fa00640 1 --2- 172.21.15.67:0/3593289637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380ac730 0x7f1c38071190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.484+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3593289637 >> 172.21.15.67:0/3593289637 conn(0x7f1c380c7370 msgr2=0x7f1c380c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:18.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.484+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3593289637 shutdown_connections 2024-03-09T08:52:18.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.484+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3593289637 wait complete. 2024-03-09T08:52:18.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.486+0000 7f1c3fa00640 1 Processor -- start 2024-03-09T08:52:18.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.486+0000 7f1c3fa00640 1 -- start start 2024-03-09T08:52:18.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.487+0000 7f1c3fa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c380ac730 0x7f1c381e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.487+0000 7f1c3fa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380c22a0 0x7f1c381eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.487+0000 7f1c3fa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c38157650 0x7f1c380c0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.487+0000 7f1c3fa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c38151610 con 0x7f1c38157650 2024-03-09T08:52:18.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.487+0000 7f1c3fa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c38070c00 con 0x7f1c380ac730 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.487+0000 7f1c3fa00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c38151790 con 0x7f1c380c22a0 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.487+0000 7f1c3da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c38157650 0x7f1c380c0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.487+0000 7f1c3da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c38157650 0x7f1c380c0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38576/0 (socket says 172.21.15.67:38576) 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.487+0000 7f1c3da00640 1 -- 172.21.15.67:0/3947493975 learned_addr learned my addr 172.21.15.67:0/3947493975 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c3d000640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c380ac730 0x7f1c381e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c37e00640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380c22a0 0x7f1c381eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c3da00640 1 -- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380c22a0 msgr2=0x7f1c381eaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c3da00640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380c22a0 0x7f1c381eaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c3da00640 1 -- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c380ac730 msgr2=0x7f1c381e0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c3da00640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c380ac730 0x7f1c381e0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c3da00640 1 -- 172.21.15.67:0/3947493975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c380c17d0 con 0x7f1c38157650 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c3d000640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c380ac730 0x7f1c381e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:18.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c3da00640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c38157650 0x7f1c380c0f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c760 tx=0x7f1c3000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:18.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c34c00640 1 -- 172.21.15.67:0/3947493975 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30016020 con 0x7f1c38157650 2024-03-09T08:52:18.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c34c00640 1 -- 172.21.15.67:0/3947493975 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c30006d50 con 0x7f1c38157650 2024-03-09T08:52:18.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c34c00640 1 -- 172.21.15.67:0/3947493975 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30003bb0 con 0x7f1c38157650 2024-03-09T08:52:18.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.488+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3947493975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c381dff40 con 0x7f1c38157650 2024-03-09T08:52:18.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.489+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3947493975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c381eafb0 con 0x7f1c38157650 2024-03-09T08:52:18.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.490+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3947493975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c38068d50 con 0x7f1c38157650 2024-03-09T08:52:18.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.491+0000 7f1c34c00640 1 -- 172.21.15.67:0/3947493975 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1c300050d0 con 0x7f1c38157650 2024-03-09T08:52:18.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.492+0000 7f1c34c00640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1c1c06b500 0x7f1c1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.492+0000 7f1c34c00640 1 -- 172.21.15.67:0/3947493975 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f1c30089de0 con 0x7f1c38157650 2024-03-09T08:52:18.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.492+0000 7f1c3d000640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1c1c06b500 0x7f1c1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.493+0000 7f1c3d000640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1c1c06b500 0x7f1c1c06d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c036000 tx=0x7f1c2c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:18.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.496+0000 7f1c34c00640 1 -- 172.21.15.67:0/3947493975 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1c38068d50 con 0x7f1c38157650 2024-03-09T08:52:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.695+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3947493975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c380c1440 con 0x7f1c38157650 2024-03-09T08:52:18.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.697+0000 7f1c34c00640 1 -- 172.21.15.67:0/3947493975 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16562 (secure 0 0 0) 0x7f1c30020020 con 0x7f1c38157650 2024-03-09T08:52:18.693 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:18.693 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":19,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:17.736512+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:18.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1c1c06b500 msgr2=0x7f1c1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1c1c06b500 0x7f1c1c06d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c036000 tx=0x7f1c2c03c040 comp rx=0 tx=0).stop 2024-03-09T08:52:18.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3947493975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c38157650 msgr2=0x7f1c380c0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c38157650 0x7f1c380c0f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c760 tx=0x7f1c3000cb60 comp rx=0 tx=0).stop 2024-03-09T08:52:18.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3947493975 shutdown_connections 2024-03-09T08:52:18.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1c1c06b500 0x7f1c1c06d9c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c38157650 0x7f1c380c0f00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c380c22a0 0x7f1c381eaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 --2- 172.21.15.67:0/3947493975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c380ac730 0x7f1c381e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3947493975 >> 172.21.15.67:0/3947493975 conn(0x7f1c380c7370 msgr2=0x7f1c38151bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:18.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3947493975 shutdown_connections 2024-03-09T08:52:18.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.700+0000 7f1c3fa00640 1 -- 172.21.15.67:0/3947493975 wait complete. 2024-03-09T08:52:18.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:18.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.917+0000 7fe217200640 1 Processor -- start 2024-03-09T08:52:18.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.917+0000 7fe217200640 1 -- start start 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.918+0000 7fe217200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe218159770 0x7fe218159be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.918+0000 7fe217200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21815a120 con 0x7fe2180acd90 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.918+0000 7fe217200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21806de60 con 0x7fe2180c1d00 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.918+0000 7fe217200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2181592e0 con 0x7fe218159770 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.919+0000 7fe215e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe2180acd90 0x7fe2181565f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46320/0 (socket says 172.21.15.67:46320) 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.919+0000 7fe215e00640 1 -- 172.21.15.67:0/1170646439 learned_addr learned my addr 172.21.15.67:0/1170646439 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.919+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514399810 0 0) 0x7fe208001110 con 0x7fe2180acd90 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.919+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1f8003520 con 0x7fe2180acd90 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.919+0000 7fe216800640 1 --2- 172.21.15.67:0/1170646439 >> v2:172.21.15.134:3300/0 conn(0x7fe218159770 0x7fe218159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.919+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 754970108 0 0) 0x7fe1f8003520 con 0x7fe2180acd90 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.919+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 >> v2:172.21.15.134:3300/0 conn(0x7fe218159770 msgr2=0x7fe218159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.919+0000 7fe215400640 1 --2- 172.21.15.67:0/1170646439 >> v2:172.21.15.134:3300/0 conn(0x7fe218159770 0x7fe218159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.919+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 >> v1:172.21.15.134:6789/0 conn(0x7fe2180c1d00 legacy=0x7fe2180aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21806d5c0 con 0x7fe2180acd90 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fe208001860 con 0x7fe2180acd90 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 >> v1:172.21.15.67:6789/0 conn(0x7fe2180acd90 legacy=0x7fe2181565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215400640 1 --2- 172.21.15.67:0/1170646439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 0x7fe1f8003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215400640 1 --2- 172.21.15.67:0/1170646439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1f8004800 0x7fe1f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215400640 1 --2- 172.21.15.67:0/1170646439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1f80053c0 0x7fe1f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2181592e0 con 0x7fe1f8003a70 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f800c2a0 con 0x7fe1f8004800 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f800c4a0 con 0x7fe1f80053c0 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215e00640 1 --2- 172.21.15.67:0/1170646439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 0x7fe1f8003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215e00640 1 -- 172.21.15.67:0/1170646439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1f80053c0 msgr2=0x7fe1f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215e00640 1 --2- 172.21.15.67:0/1170646439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1f80053c0 0x7fe1f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215e00640 1 -- 172.21.15.67:0/1170646439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1f8004800 msgr2=0x7fe1f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:18.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215e00640 1 --2- 172.21.15.67:0/1170646439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1f8004800 0x7fe1f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe215e00640 1 -- 172.21.15.67:0/1170646439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f8003520 con 0x7fe1f8003a70 2024-03-09T08:52:18.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.920+0000 7fe216800640 1 -- 172.21.15.67:0/1170646439 reap_dead start 2024-03-09T08:52:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.921+0000 7fe215e00640 1 --2- 172.21.15.67:0/1170646439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 0x7fe1f8003e90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe20800c2f0 tx=0x7fe1f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd12b2167895ee4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.922+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208007e10 con 0x7fe1f8003a70 2024-03-09T08:52:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.922+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208030070 con 0x7fe1f8003a70 2024-03-09T08:52:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.922+0000 7fe215400640 1 -- 172.21.15.67:0/1170646439 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20802ce50 con 0x7fe1f8003a70 2024-03-09T08:52:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.922+0000 7fe217200640 1 -- 172.21.15.67:0/1170646439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 msgr2=0x7fe1f8003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.922+0000 7fe217200640 1 --2- 172.21.15.67:0/1170646439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 0x7fe1f8003e90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe20800c2f0 tx=0x7fe1f8005830 comp rx=0 tx=0).stop 2024-03-09T08:52:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.922+0000 7fe217200640 1 -- 172.21.15.67:0/1170646439 shutdown_connections 2024-03-09T08:52:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.922+0000 7fe217200640 1 --2- 172.21.15.67:0/1170646439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 0x7fe1f8003e90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.922+0000 7fe217200640 1 -- 172.21.15.67:0/1170646439 >> 172.21.15.67:0/1170646439 conn(0x7fe2180c7300 msgr2=0x7fe2180c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.922+0000 7fe217200640 1 -- 172.21.15.67:0/1170646439 shutdown_connections 2024-03-09T08:52:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.922+0000 7fe217200640 1 -- 172.21.15.67:0/1170646439 wait complete. 2024-03-09T08:52:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.924+0000 7fe217200640 1 Processor -- start 2024-03-09T08:52:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.924+0000 7fe217200640 1 -- start start 2024-03-09T08:52:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.925+0000 7fe217200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 0x7fe218149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.925+0000 7fe217200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2180c5160 0x7fe2181e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.925+0000 7fe217200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2181e1280 0x7fe218157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.925+0000 7fe217200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180ad170 con 0x7fe1f8003a70 2024-03-09T08:52:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.925+0000 7fe217200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21806db80 con 0x7fe2180c5160 2024-03-09T08:52:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.925+0000 7fe217200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180ad2f0 con 0x7fe2181e1280 2024-03-09T08:52:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.925+0000 7fe215e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 0x7fe218149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.925+0000 7fe215e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 0x7fe218149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59652/0 (socket says 172.21.15.67:59652) 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.925+0000 7fe215e00640 1 -- 172.21.15.67:0/966898823 learned_addr learned my addr 172.21.15.67:0/966898823 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe216800640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2181e1280 0x7fe218157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe20fe00640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2180c5160 0x7fe2181e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe215e00640 1 -- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2181e1280 msgr2=0x7fe218157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe215e00640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2181e1280 0x7fe218157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe215e00640 1 -- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2180c5160 msgr2=0x7fe2181e0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe215e00640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2180c5160 0x7fe2181e0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe215e00640 1 -- 172.21.15.67:0/966898823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2181df550 con 0x7fe1f8003a70 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe20fe00640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2180c5160 0x7fe2181e0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe215e00640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 0x7fe218149210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe20802c620 tx=0x7fe218159250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe20e000640 1 -- 172.21.15.67:0/966898823 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208046070 con 0x7fe1f8003a70 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe20e000640 1 -- 172.21.15.67:0/966898823 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2080049e0 con 0x7fe1f8003a70 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe20e000640 1 -- 172.21.15.67:0/966898823 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208004dd0 con 0x7fe1f8003a70 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.926+0000 7fe217200640 1 -- 172.21.15.67:0/966898823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2181eb950 con 0x7fe1f8003a70 2024-03-09T08:52:18.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.927+0000 7fe217200640 1 -- 172.21.15.67:0/966898823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2181ebf10 con 0x7fe1f8003a70 2024-03-09T08:52:18.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.928+0000 7fe217200640 1 -- 172.21.15.67:0/966898823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21804f3c0 con 0x7fe1f8003a70 2024-03-09T08:52:18.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.928+0000 7fe20e000640 1 -- 172.21.15.67:0/966898823 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe208007070 con 0x7fe1f8003a70 2024-03-09T08:52:18.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.929+0000 7fe20e000640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1f406b4d0 0x7fe1f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:18.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.929+0000 7fe20e000640 1 -- 172.21.15.67:0/966898823 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5689+0+0 (secure 0 0 0) 0x7fe2080ab3e0 con 0x7fe1f8003a70 2024-03-09T08:52:18.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.930+0000 7fe20fe00640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1f406b4d0 0x7fe1f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:18.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.931+0000 7fe20fe00640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1f406b4d0 0x7fe1f406d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe2181e3160 tx=0x7fe200009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:18.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:18.934+0000 7fe20e000640 1 -- 172.21.15.67:0/966898823 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe21815a120 con 0x7fe1f8003a70 2024-03-09T08:52:19.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.133+0000 7fe217200640 1 -- 172.21.15.67:0/966898823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe21806e320 con 0x7fe1f8003a70 2024-03-09T08:52:19.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.136+0000 7fe20e000640 1 -- 172.21.15.67:0/966898823 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16562 (secure 0 0 0) 0x7fe20807a260 con 0x7fe1f8003a70 2024-03-09T08:52:19.131 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:19.132 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":19,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:17.736512+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.138+0000 7fe217200640 1 -- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1f406b4d0 msgr2=0x7fe1f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.138+0000 7fe217200640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1f406b4d0 0x7fe1f406d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe2181e3160 tx=0x7fe200009040 comp rx=0 tx=0).stop 2024-03-09T08:52:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.138+0000 7fe217200640 1 -- 172.21.15.67:0/966898823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 msgr2=0x7fe218149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.138+0000 7fe217200640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 0x7fe218149210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe20802c620 tx=0x7fe218159250 comp rx=0 tx=0).stop 2024-03-09T08:52:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.138+0000 7fe217200640 1 -- 172.21.15.67:0/966898823 shutdown_connections 2024-03-09T08:52:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.138+0000 7fe217200640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1f406b4d0 0x7fe1f406d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.138+0000 7fe217200640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2181e1280 0x7fe218157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.138+0000 7fe217200640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2180c5160 0x7fe2181e0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:19.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.138+0000 7fe217200640 1 --2- 172.21.15.67:0/966898823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1f8003a70 0x7fe218149210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.138+0000 7fe217200640 1 -- 172.21.15.67:0/966898823 >> 172.21.15.67:0/966898823 conn(0x7fe2180c7300 msgr2=0x7fe2180c2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.139+0000 7fe217200640 1 -- 172.21.15.67:0/966898823 shutdown_connections 2024-03-09T08:52:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.139+0000 7fe217200640 1 -- 172.21.15.67:0/966898823 wait complete. 2024-03-09T08:52:19.141 INFO:tasks.ceph.mds.d.smithi134.stderr:2024-03-09T08:52:19.147+0000 7f9240e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-09T08:52:19.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a standby_count_wanted 0 2024-03-09T08:52:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.370+0000 7ffae3e00640 1 Processor -- start 2024-03-09T08:52:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.370+0000 7ffae3e00640 1 -- start start 2024-03-09T08:52:19.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.371+0000 7ffae3e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae40c5f10 0x7ffae40c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:19.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.371+0000 7ffae3e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffae406b040 0x7ffae41567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.371+0000 7ffae3e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae4156d10 con 0x7ffae40c9490 2024-03-09T08:52:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.371+0000 7ffae3e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae40c9a30 con 0x7ffae406b040 2024-03-09T08:52:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.371+0000 7ffae3e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40c9d10 con 0x7ffae40c5f10 2024-03-09T08:52:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.372+0000 7ffae2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae40c5f10 0x7ffae40c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.372+0000 7ffae2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae40c5f10 0x7ffae40c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:37850/0 (socket says 172.21.15.67:37850) 2024-03-09T08:52:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.372+0000 7ffae2a00640 1 -- 172.21.15.67:0/4288837164 learned_addr learned my addr 172.21.15.67:0/4288837164 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.372+0000 7ffae2000640 1 --2- 172.21.15.67:0/4288837164 >> v2:172.21.15.134:3300/0 conn(0x7ffae406b040 0x7ffae41567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.373+0000 7ffae2a00640 1 -- 172.21.15.67:0/4288837164 >> v2:172.21.15.134:3300/0 conn(0x7ffae406b040 msgr2=0x7ffae41567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.373+0000 7ffae2a00640 1 --2- 172.21.15.67:0/4288837164 >> v2:172.21.15.134:3300/0 conn(0x7ffae406b040 0x7ffae41567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.373+0000 7ffae2a00640 1 -- 172.21.15.67:0/4288837164 >> v1:172.21.15.134:6789/0 conn(0x7ffae40c9490 legacy=0x7ffae40cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.373+0000 7ffae2a00640 1 -- 172.21.15.67:0/4288837164 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae40ca340 con 0x7ffae40c5f10 2024-03-09T08:52:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.373+0000 7ffae1600640 1 -- 172.21.15.67:0/4288837164 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852842675 0 0) 0x7ffadc001110 con 0x7ffae40c9490 2024-03-09T08:52:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.374+0000 7ffae2a00640 1 --2- 172.21.15.67:0/4288837164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae40c5f10 0x7ffae40c62f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009e50 tx=0x7ffad402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a448762a9182ecf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.374+0000 7ffae1600640 1 -- 172.21.15.67:0/4288837164 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad402ca70 con 0x7ffae40c5f10 2024-03-09T08:52:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.374+0000 7ffae1600640 1 -- 172.21.15.67:0/4288837164 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad402cc10 con 0x7ffae40c5f10 2024-03-09T08:52:19.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.375+0000 7ffae1600640 1 -- 172.21.15.67:0/4288837164 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad40343f0 con 0x7ffae40c5f10 2024-03-09T08:52:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.375+0000 7ffae3e00640 1 -- 172.21.15.67:0/4288837164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae40c5f10 msgr2=0x7ffae40c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.375+0000 7ffae3e00640 1 --2- 172.21.15.67:0/4288837164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae40c5f10 0x7ffae40c62f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009e50 tx=0x7ffad402bed0 comp rx=0 tx=0).stop 2024-03-09T08:52:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.375+0000 7ffae3e00640 1 -- 172.21.15.67:0/4288837164 shutdown_connections 2024-03-09T08:52:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.375+0000 7ffae3e00640 1 --2- 172.21.15.67:0/4288837164 >> v2:172.21.15.134:3300/0 conn(0x7ffae406b040 0x7ffae41567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.375+0000 7ffae3e00640 1 --2- 172.21.15.67:0/4288837164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae40c5f10 0x7ffae40c62f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.375+0000 7ffae3e00640 1 -- 172.21.15.67:0/4288837164 >> 172.21.15.67:0/4288837164 conn(0x7ffae40cb590 msgr2=0x7ffae40cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.375+0000 7ffae3e00640 1 -- 172.21.15.67:0/4288837164 shutdown_connections 2024-03-09T08:52:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.376+0000 7ffae3e00640 1 -- 172.21.15.67:0/4288837164 wait complete. 2024-03-09T08:52:19.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.378+0000 7ffae3e00640 1 Processor -- start 2024-03-09T08:52:19.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.378+0000 7ffae3e00640 1 -- start start 2024-03-09T08:52:19.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.379+0000 7ffae3e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae406b040 0x7ffae4148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:19.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.379+0000 7ffae3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae40c5f10 0x7ffae41ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:19.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.379+0000 7ffae3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffae40c9490 0x7ffae41eec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:19.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.379+0000 7ffae3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40cb250 con 0x7ffae40c9490 2024-03-09T08:52:19.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.379+0000 7ffae3e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40c9ff0 con 0x7ffae40c5f10 2024-03-09T08:52:19.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.379+0000 7ffae3e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40cb3d0 con 0x7ffae406b040 2024-03-09T08:52:19.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.379+0000 7ffae3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffae40c9490 0x7ffae41eec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:19.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.380+0000 7ffae3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffae40c9490 0x7ffae41eec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59660/0 (socket says 172.21.15.67:59660) 2024-03-09T08:52:19.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.380+0000 7ffae3400640 1 -- 172.21.15.67:0/1744127662 learned_addr learned my addr 172.21.15.67:0/1744127662 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:19.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.380+0000 7ffae2000640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae40c5f10 0x7ffae41ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:19.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.380+0000 7ffae2a00640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae406b040 0x7ffae4148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:19.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.380+0000 7ffae3400640 1 -- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae406b040 msgr2=0x7ffae4148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:19.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.380+0000 7ffae3400640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae406b040 0x7ffae4148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:19.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.380+0000 7ffae3400640 1 -- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae40c5f10 msgr2=0x7ffae41ee6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:19.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.380+0000 7ffae3400640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae40c5f10 0x7ffae41ee6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:19.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.380+0000 7ffae3400640 1 -- 172.21.15.67:0/1744127662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae41ef6b0 con 0x7ffae40c9490 2024-03-09T08:52:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.380+0000 7ffae3400640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffae40c9490 0x7ffae41eec00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00c3e0 tx=0x7ffadc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.380+0000 7ffadb400640 1 -- 172.21.15.67:0/1744127662 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc017070 con 0x7ffae40c9490 2024-03-09T08:52:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.381+0000 7ffadb400640 1 -- 172.21.15.67:0/1744127662 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc0035f0 con 0x7ffae40c9490 2024-03-09T08:52:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.381+0000 7ffadb400640 1 -- 172.21.15.67:0/1744127662 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0039e0 con 0x7ffae40c9490 2024-03-09T08:52:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.381+0000 7ffae3e00640 1 -- 172.21.15.67:0/1744127662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae41ef990 con 0x7ffae40c9490 2024-03-09T08:52:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.381+0000 7ffae3e00640 1 -- 172.21.15.67:0/1744127662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae41eff50 con 0x7ffae40c9490 2024-03-09T08:52:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.382+0000 7ffae3e00640 1 -- 172.21.15.67:0/1744127662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae404f3c0 con 0x7ffae40c9490 2024-03-09T08:52:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.383+0000 7ffadb400640 1 -- 172.21.15.67:0/1744127662 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ffadc01a070 con 0x7ffae40c9490 2024-03-09T08:52:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.383+0000 7ffadb400640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffac006b4d0 0x7ffac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.383+0000 7ffadb400640 1 -- 172.21.15.67:0/1744127662 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7ffadc0895a0 con 0x7ffae40c9490 2024-03-09T08:52:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.384+0000 7ffae2a00640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffac006b4d0 0x7ffac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.385+0000 7ffae2a00640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffac006b4d0 0x7ffac006d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffad4035000 tx=0x7ffad4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:19.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.390+0000 7ffadb400640 1 -- 172.21.15.67:0/1744127662 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffae4156d10 con 0x7ffae40c9490 2024-03-09T08:52:19.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:19.634+0000 7ffae3e00640 1 -- 172.21.15.67:0/1744127662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffae40cc260 con 0x7ffae40c9490 2024-03-09T08:52:20.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.417+0000 7ffadb400640 1 -- 172.21.15.67:0/1744127662 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 113+0+0 (secure 0 0 0) 0x7ffadc058420 con 0x7ffae40c9490 2024-03-09T08:52:20.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.420+0000 7ffae3e00640 1 -- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffac006b4d0 msgr2=0x7ffac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:20.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.420+0000 7ffae3e00640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffac006b4d0 0x7ffac006d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ffad4035000 tx=0x7ffad4030000 comp rx=0 tx=0).stop 2024-03-09T08:52:20.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.420+0000 7ffae3e00640 1 -- 172.21.15.67:0/1744127662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffae40c9490 msgr2=0x7ffae41eec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:20.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.420+0000 7ffae3e00640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffae40c9490 0x7ffae41eec00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00c3e0 tx=0x7ffadc00c8a0 comp rx=0 tx=0).stop 2024-03-09T08:52:20.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.420+0000 7ffae3e00640 1 -- 172.21.15.67:0/1744127662 shutdown_connections 2024-03-09T08:52:20.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.420+0000 7ffae3e00640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffac006b4d0 0x7ffac006d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:20.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.420+0000 7ffae3e00640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffae40c9490 0x7ffae41eec00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:20.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.420+0000 7ffae3e00640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffae40c5f10 0x7ffae41ee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:20.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.420+0000 7ffae3e00640 1 --2- 172.21.15.67:0/1744127662 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffae406b040 0x7ffae4148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:20.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.420+0000 7ffae3e00640 1 -- 172.21.15.67:0/1744127662 >> 172.21.15.67:0/1744127662 conn(0x7ffae40cb590 msgr2=0x7ffae4155da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:20.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.421+0000 7ffae3e00640 1 -- 172.21.15.67:0/1744127662 shutdown_connections 2024-03-09T08:52:20.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.421+0000 7ffae3e00640 1 -- 172.21.15.67:0/1744127662 wait complete. 2024-03-09T08:52:20.437 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-03-09T08:52:20.438 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a allow_standby_replay false 2024-03-09T08:52:20.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.669+0000 7fb349c00640 1 Processor -- start 2024-03-09T08:52:20.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.669+0000 7fb349c00640 1 -- start start 2024-03-09T08:52:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.671+0000 7fb349c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440ac930 0x7fb3440c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.671+0000 7fb349c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb3441514f0 0x7fb344153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.671+0000 7fb349c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440713a0 con 0x7fb3440c7f60 2024-03-09T08:52:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.671+0000 7fb349c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb344070670 con 0x7fb3441514f0 2024-03-09T08:52:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.671+0000 7fb349c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c39a0 con 0x7fb3440ac930 2024-03-09T08:52:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.671+0000 7fb343400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb3440c7f60 0x7fb344150de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46326/0 (socket says 172.21.15.67:46326) 2024-03-09T08:52:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.671+0000 7fb343400640 1 -- 172.21.15.67:0/3468632632 learned_addr learned my addr 172.21.15.67:0/3468632632 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.671+0000 7fb349200640 1 --2- 172.21.15.67:0/3468632632 >> v2:172.21.15.67:3300/0 conn(0x7fb3441514f0 0x7fb344153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:20.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.671+0000 7fb343e00640 1 --2- 172.21.15.67:0/3468632632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440ac930 0x7fb3440c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:20.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.672+0000 7fb342a00640 1 -- 172.21.15.67:0/3468632632 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205575117 0 0) 0x7fb3440713a0 con 0x7fb3440c7f60 2024-03-09T08:52:20.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.672+0000 7fb342a00640 1 -- 172.21.15.67:0/3468632632 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb328003520 con 0x7fb3440c7f60 2024-03-09T08:52:20.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.672+0000 7fb349200640 1 -- 172.21.15.67:0/3468632632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440ac930 msgr2=0x7fb3440c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:20.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.672+0000 7fb349200640 1 --2- 172.21.15.67:0/3468632632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440ac930 0x7fb3440c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:20.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.672+0000 7fb349200640 1 -- 172.21.15.67:0/3468632632 >> v1:172.21.15.67:6789/0 conn(0x7fb3440c7f60 legacy=0x7fb344150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:20.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.672+0000 7fb349200640 1 -- 172.21.15.67:0/3468632632 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440708c0 con 0x7fb3441514f0 2024-03-09T08:52:20.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.673+0000 7fb349200640 1 --2- 172.21.15.67:0/3468632632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3441514f0 0x7fb344153ac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009d90 tx=0x7fb33c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccb6360c6a7775b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:20.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.674+0000 7fb342a00640 1 -- 172.21.15.67:0/3468632632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c039070 con 0x7fb3441514f0 2024-03-09T08:52:20.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.674+0000 7fb342a00640 1 -- 172.21.15.67:0/3468632632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c02cbc0 con 0x7fb3441514f0 2024-03-09T08:52:20.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.674+0000 7fb342a00640 1 -- 172.21.15.67:0/3468632632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c034440 con 0x7fb3441514f0 2024-03-09T08:52:20.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.674+0000 7fb349c00640 1 -- 172.21.15.67:0/3468632632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3441514f0 msgr2=0x7fb344153ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:20.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.674+0000 7fb349c00640 1 --2- 172.21.15.67:0/3468632632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3441514f0 0x7fb344153ac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009d90 tx=0x7fb33c02be10 comp rx=0 tx=0).stop 2024-03-09T08:52:20.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.675+0000 7fb349c00640 1 -- 172.21.15.67:0/3468632632 shutdown_connections 2024-03-09T08:52:20.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.675+0000 7fb349c00640 1 --2- 172.21.15.67:0/3468632632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3441514f0 0x7fb344153ac0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:20.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.675+0000 7fb349c00640 1 --2- 172.21.15.67:0/3468632632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440ac930 0x7fb3440c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:20.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.675+0000 7fb349c00640 1 -- 172.21.15.67:0/3468632632 >> 172.21.15.67:0/3468632632 conn(0x7fb34406b8a0 msgr2=0x7fb3440c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:20.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.675+0000 7fb349c00640 1 -- 172.21.15.67:0/3468632632 shutdown_connections 2024-03-09T08:52:20.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.675+0000 7fb349c00640 1 -- 172.21.15.67:0/3468632632 wait complete. 2024-03-09T08:52:20.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.676+0000 7fb349c00640 1 Processor -- start 2024-03-09T08:52:20.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.676+0000 7fb349c00640 1 -- start start 2024-03-09T08:52:20.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.677+0000 7fb349c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3440ac930 0x7fb3440c0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:20.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.677+0000 7fb349c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440c7f60 0x7fb3441df710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:20.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.677+0000 7fb349c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3441514f0 0x7fb3441dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:20.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.677+0000 7fb349c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c3570 con 0x7fb3440ac930 2024-03-09T08:52:20.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.677+0000 7fb349c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb344070e80 con 0x7fb3441514f0 2024-03-09T08:52:20.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.677+0000 7fb349c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c36f0 con 0x7fb3440c7f60 2024-03-09T08:52:20.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.678+0000 7fb343e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3440ac930 0x7fb3440c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:20.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.678+0000 7fb343e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3440ac930 0x7fb3440c0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59676/0 (socket says 172.21.15.67:59676) 2024-03-09T08:52:20.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.678+0000 7fb343e00640 1 -- 172.21.15.67:0/188767651 learned_addr learned my addr 172.21.15.67:0/188767651 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:20.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.678+0000 7fb343400640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440c7f60 0x7fb3441df710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:20.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.678+0000 7fb349200640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3441514f0 0x7fb3441dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:20.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.678+0000 7fb343e00640 1 -- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440c7f60 msgr2=0x7fb3441df710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:20.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.678+0000 7fb343e00640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440c7f60 0x7fb3441df710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:20.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.678+0000 7fb343e00640 1 -- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3441514f0 msgr2=0x7fb3441dfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:20.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.678+0000 7fb343e00640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3441514f0 0x7fb3441dfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:20.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.678+0000 7fb343e00640 1 -- 172.21.15.67:0/188767651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3441eb880 con 0x7fb3440ac930 2024-03-09T08:52:20.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.678+0000 7fb343e00640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3440ac930 0x7fb3440c0930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb33400e9e0 tx=0x7fb33400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:20.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.679+0000 7fb33be00640 1 -- 172.21.15.67:0/188767651 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334017070 con 0x7fb3440ac930 2024-03-09T08:52:20.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.679+0000 7fb33be00640 1 -- 172.21.15.67:0/188767651 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33400c9d0 con 0x7fb3440ac930 2024-03-09T08:52:20.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.679+0000 7fb33be00640 1 -- 172.21.15.67:0/188767651 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33400cdc0 con 0x7fb3440ac930 2024-03-09T08:52:20.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.679+0000 7fb349c00640 1 -- 172.21.15.67:0/188767651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3441ebb60 con 0x7fb3440ac930 2024-03-09T08:52:20.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.679+0000 7fb349c00640 1 -- 172.21.15.67:0/188767651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3441ec120 con 0x7fb3440ac930 2024-03-09T08:52:20.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.680+0000 7fb349c00640 1 -- 172.21.15.67:0/188767651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34404f3c0 con 0x7fb3440ac930 2024-03-09T08:52:20.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.681+0000 7fb33be00640 1 -- 172.21.15.67:0/188767651 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb33401a070 con 0x7fb3440ac930 2024-03-09T08:52:20.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.682+0000 7fb33be00640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb32406b4d0 0x7fb32406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:20.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.683+0000 7fb33be00640 1 -- 172.21.15.67:0/188767651 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7fb33408a5a0 con 0x7fb3440ac930 2024-03-09T08:52:20.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.683+0000 7fb343400640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb32406b4d0 0x7fb32406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:20.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.684+0000 7fb343400640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb32406b4d0 0x7fb32406d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb3440c1160 tx=0x7fb330009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.687+0000 7fb33be00640 1 -- 172.21.15.67:0/188767651 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb3340594c0 con 0x7fb3440ac930 2024-03-09T08:52:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:20.930+0000 7fb349c00640 1 -- 172.21.15.67:0/188767651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb3440aa4b0 con 0x7fb3440ac930 2024-03-09T08:52:21.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.427+0000 7fb33be00640 1 -- 172.21.15.67:0/188767651 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 117+0+0 (secure 0 0 0) 0x7fb334058dc0 con 0x7fb3440ac930 2024-03-09T08:52:21.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.431+0000 7fb349c00640 1 -- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb32406b4d0 msgr2=0x7fb32406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:21.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.431+0000 7fb349c00640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb32406b4d0 0x7fb32406d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb3440c1160 tx=0x7fb330009040 comp rx=0 tx=0).stop 2024-03-09T08:52:21.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.431+0000 7fb349c00640 1 -- 172.21.15.67:0/188767651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3440ac930 msgr2=0x7fb3440c0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:21.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.431+0000 7fb349c00640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3440ac930 0x7fb3440c0930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb33400e9e0 tx=0x7fb33400eea0 comp rx=0 tx=0).stop 2024-03-09T08:52:21.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.431+0000 7fb349c00640 1 -- 172.21.15.67:0/188767651 shutdown_connections 2024-03-09T08:52:21.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.431+0000 7fb349c00640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb32406b4d0 0x7fb32406d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:21.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.431+0000 7fb349c00640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3441514f0 0x7fb3441dfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:21.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.431+0000 7fb349c00640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440c7f60 0x7fb3441df710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:21.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.431+0000 7fb349c00640 1 --2- 172.21.15.67:0/188767651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3440ac930 0x7fb3440c0930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:21.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.431+0000 7fb349c00640 1 -- 172.21.15.67:0/188767651 >> 172.21.15.67:0/188767651 conn(0x7fb34406b8a0 msgr2=0x7fb3440c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:21.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.431+0000 7fb349c00640 1 -- 172.21.15.67:0/188767651 shutdown_connections 2024-03-09T08:52:21.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.432+0000 7fb349c00640 1 -- 172.21.15.67:0/188767651 wait complete. 2024-03-09T08:52:21.449 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a session_timeout 300 2024-03-09T08:52:21.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.671+0000 7f7903e00640 1 Processor -- start 2024-03-09T08:52:21.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.671+0000 7f7903e00640 1 -- start start 2024-03-09T08:52:21.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.672+0000 7f7903e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f790406b040 0x7f79040c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:21.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.672+0000 7f7903e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f79040c9490 0x7f79040cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:21.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.672+0000 7f7903e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79040c9a30 con 0x7f79040c5f10 2024-03-09T08:52:21.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.672+0000 7f7903e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79040c9d10 con 0x7f79040c9490 2024-03-09T08:52:21.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.672+0000 7f7903e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79040ca340 con 0x7f790406b040 2024-03-09T08:52:21.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.673+0000 7f7903400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f79040c9490 0x7f79040cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:21.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.673+0000 7f7903400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f79040c9490 0x7f79040cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59684/0 (socket says 172.21.15.67:59684) 2024-03-09T08:52:21.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.673+0000 7f7903400640 1 -- 172.21.15.67:0/3957562891 learned_addr learned my addr 172.21.15.67:0/3957562891 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:21.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.673+0000 7f7902000640 1 --2- 172.21.15.67:0/3957562891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f790406b040 0x7f79040c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:21.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.673+0000 7f7903400640 1 -- 172.21.15.67:0/3957562891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f790406b040 msgr2=0x7f79040c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:21.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.673+0000 7f7903400640 1 --2- 172.21.15.67:0/3957562891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f790406b040 0x7f79040c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:21.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.673+0000 7f7903400640 1 -- 172.21.15.67:0/3957562891 >> v1:172.21.15.134:6789/0 conn(0x7f79040c5f10 legacy=0x7f79041567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:21.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.673+0000 7f7903400640 1 -- 172.21.15.67:0/3957562891 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79040ca620 con 0x7f79040c9490 2024-03-09T08:52:21.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.674+0000 7f7903400640 1 --2- 172.21.15.67:0/3957562891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79040c9490 0x7f79040cab40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009e50 tx=0x7f78fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66e71fe4044f763 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:21.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.675+0000 7f7901600640 1 -- 172.21.15.67:0/3957562891 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc02ca70 con 0x7f79040c9490 2024-03-09T08:52:21.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.675+0000 7f7901600640 1 -- 172.21.15.67:0/3957562891 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc02cc10 con 0x7f79040c9490 2024-03-09T08:52:21.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.675+0000 7f7901600640 1 -- 172.21.15.67:0/3957562891 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0343f0 con 0x7f79040c9490 2024-03-09T08:52:21.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.675+0000 7f7903e00640 1 -- 172.21.15.67:0/3957562891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79040c9490 msgr2=0x7f79040cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:21.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.675+0000 7f7903e00640 1 --2- 172.21.15.67:0/3957562891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79040c9490 0x7f79040cab40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009e50 tx=0x7f78fc02bed0 comp rx=0 tx=0).stop 2024-03-09T08:52:21.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.675+0000 7f7903e00640 1 -- 172.21.15.67:0/3957562891 shutdown_connections 2024-03-09T08:52:21.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.675+0000 7f7903e00640 1 --2- 172.21.15.67:0/3957562891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79040c9490 0x7f79040cab40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:21.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.675+0000 7f7903e00640 1 --2- 172.21.15.67:0/3957562891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f790406b040 0x7f79040c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:21.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.676+0000 7f7903e00640 1 -- 172.21.15.67:0/3957562891 >> 172.21.15.67:0/3957562891 conn(0x7f79040cb590 msgr2=0x7f79040cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:21.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.676+0000 7f7903e00640 1 -- 172.21.15.67:0/3957562891 shutdown_connections 2024-03-09T08:52:21.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.676+0000 7f7903e00640 1 -- 172.21.15.67:0/3957562891 wait complete. 2024-03-09T08:52:21.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.677+0000 7f7903e00640 1 Processor -- start 2024-03-09T08:52:21.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.677+0000 7f7903e00640 1 -- start start 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.678+0000 7f7903e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f790406b040 0x7f7904155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.678+0000 7f7903e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79040c5f10 0x7f79041e1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.678+0000 7f7903e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79040c9490 0x7f79041e1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.678+0000 7f7903e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79041537a0 con 0x7f79040c9490 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.678+0000 7f7903e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7904153620 con 0x7f790406b040 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.678+0000 7f7903e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79040cb080 con 0x7f79040c5f10 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.679+0000 7f7903400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79040c9490 0x7f79041e1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.679+0000 7f7903400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79040c9490 0x7f79041e1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59694/0 (socket says 172.21.15.67:59694) 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.679+0000 7f7903400640 1 -- 172.21.15.67:0/727440501 learned_addr learned my addr 172.21.15.67:0/727440501 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.679+0000 7f7902000640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79040c5f10 0x7f79041e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.679+0000 7f7902a00640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f790406b040 0x7f7904155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:21.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.679+0000 7f7903400640 1 -- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79040c5f10 msgr2=0x7f79041e1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.679+0000 7f7903400640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79040c5f10 0x7f79041e1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.679+0000 7f7903400640 1 -- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f790406b040 msgr2=0x7f7904155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.679+0000 7f7903400640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f790406b040 0x7f7904155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.679+0000 7f7903400640 1 -- 172.21.15.67:0/727440501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79041ef930 con 0x7f79040c9490 2024-03-09T08:52:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.680+0000 7f7903400640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79040c9490 0x7f79041e1c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78fc036040 tx=0x7f78fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:21.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.680+0000 7f78faa00640 1 -- 172.21.15.67:0/727440501 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc043020 con 0x7f79040c9490 2024-03-09T08:52:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.680+0000 7f78faa00640 1 -- 172.21.15.67:0/727440501 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc03e450 con 0x7f79040c9490 2024-03-09T08:52:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.680+0000 7f78faa00640 1 -- 172.21.15.67:0/727440501 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc009510 con 0x7f79040c9490 2024-03-09T08:52:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.680+0000 7f7903e00640 1 -- 172.21.15.67:0/727440501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79041e0f30 con 0x7f79040c9490 2024-03-09T08:52:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.680+0000 7f7903e00640 1 -- 172.21.15.67:0/727440501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790414cda0 con 0x7f79040c9490 2024-03-09T08:52:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.681+0000 7f7903e00640 1 -- 172.21.15.67:0/727440501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7904062c90 con 0x7f79040c9490 2024-03-09T08:52:21.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.682+0000 7f78faa00640 1 -- 172.21.15.67:0/727440501 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f78fc046020 con 0x7f79040c9490 2024-03-09T08:52:21.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.683+0000 7f78faa00640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f78e006b500 0x7f78e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:21.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.683+0000 7f78faa00640 1 -- 172.21.15.67:0/727440501 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f78fc0ac9a0 con 0x7f79040c9490 2024-03-09T08:52:21.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.684+0000 7f7902a00640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f78e006b500 0x7f78e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:21.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.685+0000 7f7902a00640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f78e006b500 0x7f78e006d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78f4000f60 tx=0x7f78f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:21.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.688+0000 7f78faa00640 1 -- 172.21.15.67:0/727440501 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7904062c90 con 0x7f79040c9490 2024-03-09T08:52:21.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:21.929+0000 7f7903e00640 1 -- 172.21.15.67:0/727440501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f790404f3c0 con 0x7f79040c9490 2024-03-09T08:52:22.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.439+0000 7f78faa00640 1 -- 172.21.15.67:0/727440501 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "session_timeout", "val": "300"}]=0 v8) v1 ==== 110+0+0 (secure 0 0 0) 0x7f78fc04d030 con 0x7f79040c9490 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 -- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f78e006b500 msgr2=0x7f78e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f78e006b500 0x7f78e006d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78f4000f60 tx=0x7f78f4009040 comp rx=0 tx=0).stop 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 -- 172.21.15.67:0/727440501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79040c9490 msgr2=0x7f79041e1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79040c9490 0x7f79041e1c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78fc036040 tx=0x7f78fc004a40 comp rx=0 tx=0).stop 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 -- 172.21.15.67:0/727440501 shutdown_connections 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f78e006b500 0x7f78e006d9c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79040c9490 0x7f79041e1c70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79040c5f10 0x7f79041e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 --2- 172.21.15.67:0/727440501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f790406b040 0x7f7904155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 -- 172.21.15.67:0/727440501 >> 172.21.15.67:0/727440501 conn(0x7f79040cb590 msgr2=0x7f79040ac590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 -- 172.21.15.67:0/727440501 shutdown_connections 2024-03-09T08:52:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.443+0000 7f7903e00640 1 -- 172.21.15.67:0/727440501 wait complete. 2024-03-09T08:52:22.460 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'a' 2024-03-09T08:52:22.460 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T08:52:22.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.686+0000 7fd406e00640 1 Processor -- start 2024-03-09T08:52:22.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.686+0000 7fd406e00640 1 -- start start 2024-03-09T08:52:22.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.688+0000 7fd406e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd4000c1f70 0x7fd400150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:22.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.688+0000 7fd406e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4000710a0 0x7fd400151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:22.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.688+0000 7fd406e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4000c81e0 con 0x7fd4000c27a0 2024-03-09T08:52:22.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.688+0000 7fd406e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4000c32b0 con 0x7fd4000c1f70 2024-03-09T08:52:22.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.688+0000 7fd406e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4000c2fd0 con 0x7fd4000710a0 2024-03-09T08:52:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.688+0000 7fd406400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd4000c27a0 0x7fd400153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46330/0 (socket says 172.21.15.67:46330) 2024-03-09T08:52:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.688+0000 7fd406400640 1 -- 172.21.15.67:0/2694947096 learned_addr learned my addr 172.21.15.67:0/2694947096 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.688+0000 7fd405a00640 1 --2- 172.21.15.67:0/2694947096 >> v2:172.21.15.67:3300/0 conn(0x7fd4000c1f70 0x7fd400150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.688+0000 7fd405000640 1 --2- 172.21.15.67:0/2694947096 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4000710a0 0x7fd400151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.689+0000 7fd3ffe00640 1 -- 172.21.15.67:0/2694947096 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906544328 0 0) 0x7fd4000c81e0 con 0x7fd4000c27a0 2024-03-09T08:52:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.689+0000 7fd3ffe00640 1 -- 172.21.15.67:0/2694947096 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3e8003520 con 0x7fd4000c27a0 2024-03-09T08:52:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.689+0000 7fd405a00640 1 -- 172.21.15.67:0/2694947096 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4000710a0 msgr2=0x7fd400151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.689+0000 7fd405a00640 1 --2- 172.21.15.67:0/2694947096 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4000710a0 0x7fd400151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.689+0000 7fd405a00640 1 -- 172.21.15.67:0/2694947096 >> v1:172.21.15.67:6789/0 conn(0x7fd4000c27a0 legacy=0x7fd400153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.689+0000 7fd405a00640 1 -- 172.21.15.67:0/2694947096 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4000c3700 con 0x7fd4000c1f70 2024-03-09T08:52:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.690+0000 7fd405a00640 1 --2- 172.21.15.67:0/2694947096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4000c1f70 0x7fd400150c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40097a0 tx=0x7fd3f402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7f3df1037d9df1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.690+0000 7fd3ffe00640 1 -- 172.21.15.67:0/2694947096 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4031780 con 0x7fd4000c1f70 2024-03-09T08:52:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.690+0000 7fd3ffe00640 1 -- 172.21.15.67:0/2694947096 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f4037070 con 0x7fd4000c1f70 2024-03-09T08:52:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.690+0000 7fd3ffe00640 1 -- 172.21.15.67:0/2694947096 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4031b40 con 0x7fd4000c1f70 2024-03-09T08:52:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.691+0000 7fd406e00640 1 -- 172.21.15.67:0/2694947096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4000c1f70 msgr2=0x7fd400150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.691+0000 7fd406e00640 1 --2- 172.21.15.67:0/2694947096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4000c1f70 0x7fd400150c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40097a0 tx=0x7fd3f402b820 comp rx=0 tx=0).stop 2024-03-09T08:52:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.691+0000 7fd406e00640 1 -- 172.21.15.67:0/2694947096 shutdown_connections 2024-03-09T08:52:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.691+0000 7fd406e00640 1 --2- 172.21.15.67:0/2694947096 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4000710a0 0x7fd400151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.691+0000 7fd406e00640 1 --2- 172.21.15.67:0/2694947096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4000c1f70 0x7fd400150c50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.691+0000 7fd406e00640 1 -- 172.21.15.67:0/2694947096 >> 172.21.15.67:0/2694947096 conn(0x7fd4000c7300 msgr2=0x7fd4000c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.691+0000 7fd406e00640 1 -- 172.21.15.67:0/2694947096 shutdown_connections 2024-03-09T08:52:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.691+0000 7fd406e00640 1 -- 172.21.15.67:0/2694947096 wait complete. 2024-03-09T08:52:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.693+0000 7fd406e00640 1 Processor -- start 2024-03-09T08:52:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.693+0000 7fd406e00640 1 -- start start 2024-03-09T08:52:22.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.694+0000 7fd406e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4000710a0 0x7fd400148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:22.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.694+0000 7fd406e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4000c1f70 0x7fd4000c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:22.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.694+0000 7fd406e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4000c27a0 0x7fd4000c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:22.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.694+0000 7fd406e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd400070d50 con 0x7fd4000710a0 2024-03-09T08:52:22.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.694+0000 7fd406e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd400070830 con 0x7fd4000c1f70 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.694+0000 7fd406e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd400070ed0 con 0x7fd4000c27a0 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.694+0000 7fd405a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4000710a0 0x7fd400148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.694+0000 7fd405a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4000710a0 0x7fd400148070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59710/0 (socket says 172.21.15.67:59710) 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.694+0000 7fd405a00640 1 -- 172.21.15.67:0/2386422510 learned_addr learned my addr 172.21.15.67:0/2386422510 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.694+0000 7fd406400640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4000c27a0 0x7fd4000c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.694+0000 7fd405000640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4000c1f70 0x7fd4000c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.695+0000 7fd405a00640 1 -- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4000c27a0 msgr2=0x7fd4000c0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.695+0000 7fd405a00640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4000c27a0 0x7fd4000c0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.695+0000 7fd405a00640 1 -- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4000c1f70 msgr2=0x7fd4000c07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.695+0000 7fd405a00640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4000c1f70 0x7fd4000c07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.695+0000 7fd405a00640 1 -- 172.21.15.67:0/2386422510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd400157e90 con 0x7fd4000710a0 2024-03-09T08:52:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.695+0000 7fd405a00640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4000710a0 0x7fd400148070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4034040 tx=0x7fd3f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.695+0000 7fd3fd600640 1 -- 172.21.15.67:0/2386422510 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4032610 con 0x7fd4000710a0 2024-03-09T08:52:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.695+0000 7fd3fd600640 1 -- 172.21.15.67:0/2386422510 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f40327d0 con 0x7fd4000710a0 2024-03-09T08:52:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.695+0000 7fd3fd600640 1 -- 172.21.15.67:0/2386422510 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4032bc0 con 0x7fd4000710a0 2024-03-09T08:52:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.695+0000 7fd406e00640 1 -- 172.21.15.67:0/2386422510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd400151880 con 0x7fd4000710a0 2024-03-09T08:52:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.696+0000 7fd406e00640 1 -- 172.21.15.67:0/2386422510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4000beb90 con 0x7fd4000710a0 2024-03-09T08:52:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.697+0000 7fd406e00640 1 -- 172.21.15.67:0/2386422510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd400151da0 con 0x7fd4000710a0 2024-03-09T08:52:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.697+0000 7fd3fd600640 1 -- 172.21.15.67:0/2386422510 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd3f404e430 con 0x7fd4000710a0 2024-03-09T08:52:22.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.698+0000 7fd3fd600640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd3e006b4d0 0x7fd3e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:22.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.698+0000 7fd3fd600640 1 -- 172.21.15.67:0/2386422510 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7fd3f40ac090 con 0x7fd4000710a0 2024-03-09T08:52:22.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.699+0000 7fd405000640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd3e006b4d0 0x7fd3e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.700+0000 7fd405000640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd3e006b4d0 0x7fd3e006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd40014ec10 tx=0x7fd3ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.703+0000 7fd3fd600640 1 -- 172.21.15.67:0/2386422510 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd400151da0 con 0x7fd4000710a0 2024-03-09T08:52:22.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.943+0000 7fd406e00640 1 -- 172.21.15.67:0/2386422510 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4000bed90 con 0x7fd4000710a0 2024-03-09T08:52:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.944+0000 7fd3fd600640 1 -- 172.21.15.67:0/2386422510 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4970 (secure 0 0 0) 0x7fd4000bed90 con 0x7fd4000710a0 2024-03-09T08:52:22.940 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:22.940 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:17.736973+0000","modified":"2024-03-09T08:52:21.931108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1}]} 2024-03-09T08:52:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 -- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd3e006b4d0 msgr2=0x7fd3e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd3e006b4d0 0x7fd3e006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd40014ec10 tx=0x7fd3ec008040 comp rx=0 tx=0).stop 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 -- 172.21.15.67:0/2386422510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4000710a0 msgr2=0x7fd400148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4000710a0 0x7fd400148070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4034040 tx=0x7fd3f40046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 -- 172.21.15.67:0/2386422510 shutdown_connections 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd3e006b4d0 0x7fd3e006d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4000c27a0 0x7fd4000c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4000c1f70 0x7fd4000c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 --2- 172.21.15.67:0/2386422510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4000710a0 0x7fd400148070 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 -- 172.21.15.67:0/2386422510 >> 172.21.15.67:0/2386422510 conn(0x7fd4000c7300 msgr2=0x7fd40014f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 -- 172.21.15.67:0/2386422510 shutdown_connections 2024-03-09T08:52:22.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:22.947+0000 7fd406e00640 1 -- 172.21.15.67:0/2386422510 wait complete. 2024-03-09T08:52:22.963 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:23.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.186+0000 7f1955600640 1 Processor -- start 2024-03-09T08:52:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.186+0000 7f1955600640 1 -- start start 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.187+0000 7f1955600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f19500710a0 0x7f1950153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f1955600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1950151360 0x7f1950153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f1955600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19500c81e0 con 0x7f19500c1f70 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f1955600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19500c32b0 con 0x7f19500710a0 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f1955600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19500c2fd0 con 0x7f1950151360 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f194f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f19500710a0 0x7f1950153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f194f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f19500710a0 0x7f1950153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59720/0 (socket says 172.21.15.67:59720) 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f194f400640 1 -- 172.21.15.67:0/121508807 learned_addr learned my addr 172.21.15.67:0/121508807 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f1954c00640 1 --2- 172.21.15.67:0/121508807 >> v2:172.21.15.134:3300/0 conn(0x7f1950151360 0x7f1950153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f194f400640 1 -- 172.21.15.67:0/121508807 >> v2:172.21.15.134:3300/0 conn(0x7f1950151360 msgr2=0x7f1950153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f194f400640 1 --2- 172.21.15.67:0/121508807 >> v2:172.21.15.134:3300/0 conn(0x7f1950151360 0x7f1950153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f194f400640 1 -- 172.21.15.67:0/121508807 >> v1:172.21.15.134:6789/0 conn(0x7f19500c1f70 legacy=0x7f1950150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f194f400640 1 -- 172.21.15.67:0/121508807 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19500c3700 con 0x7f19500710a0 2024-03-09T08:52:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.188+0000 7f1954c00640 1 --2- 172.21.15.67:0/121508807 >> v2:172.21.15.134:3300/0 conn(0x7f1950151360 0x7f1950153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.189+0000 7f194f400640 1 --2- 172.21.15.67:0/121508807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19500710a0 0x7f1950153970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1950073ef0 tx=0x7f193c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba7f9d3b369878e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.190+0000 7f194ea00640 1 -- 172.21.15.67:0/121508807 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c02e070 con 0x7f19500710a0 2024-03-09T08:52:23.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.190+0000 7f194ea00640 1 -- 172.21.15.67:0/121508807 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c037070 con 0x7f19500710a0 2024-03-09T08:52:23.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.190+0000 7f194ea00640 1 -- 172.21.15.67:0/121508807 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c031a00 con 0x7f19500710a0 2024-03-09T08:52:23.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.190+0000 7f1955600640 1 -- 172.21.15.67:0/121508807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19500710a0 msgr2=0x7f1950153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:23.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.190+0000 7f1955600640 1 --2- 172.21.15.67:0/121508807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19500710a0 0x7f1950153970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1950073ef0 tx=0x7f193c02b880 comp rx=0 tx=0).stop 2024-03-09T08:52:23.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.190+0000 7f1955600640 1 -- 172.21.15.67:0/121508807 shutdown_connections 2024-03-09T08:52:23.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.190+0000 7f1955600640 1 --2- 172.21.15.67:0/121508807 >> v2:172.21.15.134:3300/0 conn(0x7f1950151360 0x7f1950153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.190+0000 7f1955600640 1 --2- 172.21.15.67:0/121508807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f19500710a0 0x7f1950153970 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.190+0000 7f1955600640 1 -- 172.21.15.67:0/121508807 >> 172.21.15.67:0/121508807 conn(0x7f19500c7300 msgr2=0x7f19500c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:23.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.191+0000 7f1955600640 1 -- 172.21.15.67:0/121508807 shutdown_connections 2024-03-09T08:52:23.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.191+0000 7f1955600640 1 -- 172.21.15.67:0/121508807 wait complete. 2024-03-09T08:52:23.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.192+0000 7f1955600640 1 Processor -- start 2024-03-09T08:52:23.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.193+0000 7f1955600640 1 -- start start 2024-03-09T08:52:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.194+0000 7f1955600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19500710a0 0x7f19500c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.194+0000 7f1955600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19500c1f70 0x7f19500c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.194+0000 7f1955600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1950151360 0x7f19501eb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.194+0000 7f1955600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1950070d50 con 0x7f1950151360 2024-03-09T08:52:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.194+0000 7f1955600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1950070830 con 0x7f19500710a0 2024-03-09T08:52:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.194+0000 7f1955600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1950070ed0 con 0x7f19500c1f70 2024-03-09T08:52:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.194+0000 7f1954c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1950151360 0x7f19501eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.194+0000 7f194fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19500710a0 0x7f19500c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.194+0000 7f1954c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1950151360 0x7f19501eb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59722/0 (socket says 172.21.15.67:59722) 2024-03-09T08:52:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.194+0000 7f1954c00640 1 -- 172.21.15.67:0/3713293823 learned_addr learned my addr 172.21.15.67:0/3713293823 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.194+0000 7f194f400640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19500c1f70 0x7f19500c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.195+0000 7f1954c00640 1 -- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19500c1f70 msgr2=0x7f19500c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.195+0000 7f1954c00640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19500c1f70 0x7f19500c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.195+0000 7f1954c00640 1 -- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19500710a0 msgr2=0x7f19500c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.195+0000 7f1954c00640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19500710a0 0x7f19500c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.195+0000 7f1954c00640 1 -- 172.21.15.67:0/3713293823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19501ebc90 con 0x7f1950151360 2024-03-09T08:52:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.195+0000 7f1954c00640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1950151360 0x7f19501eb750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f194800cea0 tx=0x7f194800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:23.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.195+0000 7f1947e00640 1 -- 172.21.15.67:0/3713293823 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1948003b90 con 0x7f1950151360 2024-03-09T08:52:23.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.195+0000 7f1947e00640 1 -- 172.21.15.67:0/3713293823 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1948003d50 con 0x7f1950151360 2024-03-09T08:52:23.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.195+0000 7f1947e00640 1 -- 172.21.15.67:0/3713293823 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1948004260 con 0x7f1950151360 2024-03-09T08:52:23.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.195+0000 7f1955600640 1 -- 172.21.15.67:0/3713293823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19500c5e10 con 0x7f1950151360 2024-03-09T08:52:23.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.195+0000 7f1955600640 1 -- 172.21.15.67:0/3713293823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1950154f00 con 0x7f1950151360 2024-03-09T08:52:23.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.196+0000 7f1955600640 1 -- 172.21.15.67:0/3713293823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195004f3c0 con 0x7f1950151360 2024-03-09T08:52:23.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.197+0000 7f1947e00640 1 -- 172.21.15.67:0/3713293823 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1948004400 con 0x7f1950151360 2024-03-09T08:52:23.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.198+0000 7f1947e00640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f193006b4d0 0x7f193006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.198+0000 7f1947e00640 1 -- 172.21.15.67:0/3713293823 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f194808aac0 con 0x7f1950151360 2024-03-09T08:52:23.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.199+0000 7f194fe00640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f193006b4d0 0x7f193006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.200+0000 7f194fe00640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f193006b4d0 0x7f193006d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1940004f60 tx=0x7f1940007e30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.202+0000 7f1947e00640 1 -- 172.21.15.67:0/3713293823 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1948059390 con 0x7f1950151360 2024-03-09T08:52:23.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.400+0000 7f1955600640 1 -- 172.21.15.67:0/3713293823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19500c2910 con 0x7f1950151360 2024-03-09T08:52:23.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.403+0000 7f1947e00640 1 -- 172.21.15.67:0/3713293823 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16549 (secure 0 0 0) 0x7f194805d430 con 0x7f1950151360 2024-03-09T08:52:23.399 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:23.399 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":20,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:19.196234+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.405+0000 7f1955600640 1 -- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f193006b4d0 msgr2=0x7f193006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.405+0000 7f1955600640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f193006b4d0 0x7f193006d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1940004f60 tx=0x7f1940007e30 comp rx=0 tx=0).stop 2024-03-09T08:52:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.406+0000 7f1955600640 1 -- 172.21.15.67:0/3713293823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1950151360 msgr2=0x7f19501eb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.406+0000 7f1955600640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1950151360 0x7f19501eb750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f194800cea0 tx=0x7f194800cf90 comp rx=0 tx=0).stop 2024-03-09T08:52:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.406+0000 7f1955600640 1 -- 172.21.15.67:0/3713293823 shutdown_connections 2024-03-09T08:52:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.406+0000 7f1955600640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f193006b4d0 0x7f193006d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.406+0000 7f1955600640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1950151360 0x7f19501eb750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.406+0000 7f1955600640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19500c1f70 0x7f19500c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.406+0000 7f1955600640 1 --2- 172.21.15.67:0/3713293823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19500710a0 0x7f19500c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.406+0000 7f1955600640 1 -- 172.21.15.67:0/3713293823 >> 172.21.15.67:0/3713293823 conn(0x7f19500c7300 msgr2=0x7f1950073380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.406+0000 7f1955600640 1 -- 172.21.15.67:0/3713293823 shutdown_connections 2024-03-09T08:52:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.406+0000 7f1955600640 1 -- 172.21.15.67:0/3713293823 wait complete. 2024-03-09T08:52:23.419 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-09T08:52:23.419 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T08:52:23.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.634+0000 7f6cfce00640 1 Processor -- start 2024-03-09T08:52:23.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.634+0000 7f6cfce00640 1 -- start start 2024-03-09T08:52:23.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.636+0000 7f6cfce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6cf80c1f70 0x7f6cf8150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.636+0000 7f6cfce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80710a0 0x7f6cf8151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.636+0000 7f6cfce00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6cf80c27a0 0x7f6cf8153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.636+0000 7f6cfce00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf80c81e0 con 0x7f6cf80c1f70 2024-03-09T08:52:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.636+0000 7f6cfce00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf80c32b0 con 0x7f6cf80c27a0 2024-03-09T08:52:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.636+0000 7f6cfce00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf80c2fd0 con 0x7f6cf80710a0 2024-03-09T08:52:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.636+0000 7f6cf7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6cf80c1f70 0x7f6cf8150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.636+0000 7f6cf7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6cf80c1f70 0x7f6cf8150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59730/0 (socket says 172.21.15.67:59730) 2024-03-09T08:52:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.636+0000 7f6cf7400640 1 -- 172.21.15.67:0/543322920 learned_addr learned my addr 172.21.15.67:0/543322920 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.636+0000 7f6cf7e00640 1 --2- 172.21.15.67:0/543322920 >> v2:172.21.15.134:3300/0 conn(0x7f6cf80c27a0 0x7f6cf8153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.636+0000 7f6cf6a00640 1 --2- 172.21.15.67:0/543322920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80710a0 0x7f6cf8151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.637+0000 7f6cf7400640 1 -- 172.21.15.67:0/543322920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80710a0 msgr2=0x7f6cf8151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:23.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.637+0000 7f6cf7400640 1 --2- 172.21.15.67:0/543322920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80710a0 0x7f6cf8151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.637+0000 7f6cf7400640 1 -- 172.21.15.67:0/543322920 >> v2:172.21.15.134:3300/0 conn(0x7f6cf80c27a0 msgr2=0x7f6cf8153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:23.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.637+0000 7f6cf7400640 1 --2- 172.21.15.67:0/543322920 >> v2:172.21.15.134:3300/0 conn(0x7f6cf80c27a0 0x7f6cf8153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.637+0000 7f6cf7400640 1 -- 172.21.15.67:0/543322920 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80c3700 con 0x7f6cf80c1f70 2024-03-09T08:52:23.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.638+0000 7f6cf7400640 1 --2- 172.21.15.67:0/543322920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf80c1f70 0x7f6cf8150c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6cf806e550 tx=0x7f6ce802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c8a23f650772da server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:23.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.639+0000 7f6cf6000640 1 -- 172.21.15.67:0/543322920 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce802e070 con 0x7f6cf80c1f70 2024-03-09T08:52:23.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.639+0000 7f6cf6000640 1 -- 172.21.15.67:0/543322920 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce8037070 con 0x7f6cf80c1f70 2024-03-09T08:52:23.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.639+0000 7f6cf6000640 1 -- 172.21.15.67:0/543322920 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8031a00 con 0x7f6cf80c1f70 2024-03-09T08:52:23.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.639+0000 7f6cfce00640 1 -- 172.21.15.67:0/543322920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf80c1f70 msgr2=0x7f6cf8150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:23.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.639+0000 7f6cfce00640 1 --2- 172.21.15.67:0/543322920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf80c1f70 0x7f6cf8150c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6cf806e550 tx=0x7f6ce802b880 comp rx=0 tx=0).stop 2024-03-09T08:52:23.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.639+0000 7f6cfce00640 1 -- 172.21.15.67:0/543322920 shutdown_connections 2024-03-09T08:52:23.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.639+0000 7f6cfce00640 1 --2- 172.21.15.67:0/543322920 >> v2:172.21.15.134:3300/0 conn(0x7f6cf80c27a0 0x7f6cf8153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.639+0000 7f6cfce00640 1 --2- 172.21.15.67:0/543322920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80710a0 0x7f6cf8151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.639+0000 7f6cfce00640 1 --2- 172.21.15.67:0/543322920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf80c1f70 0x7f6cf8150c50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.639+0000 7f6cfce00640 1 -- 172.21.15.67:0/543322920 >> 172.21.15.67:0/543322920 conn(0x7f6cf80c7300 msgr2=0x7f6cf80c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:23.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.640+0000 7f6cfce00640 1 -- 172.21.15.67:0/543322920 shutdown_connections 2024-03-09T08:52:23.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.640+0000 7f6cfce00640 1 -- 172.21.15.67:0/543322920 wait complete. 2024-03-09T08:52:23.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.641+0000 7f6cfce00640 1 Processor -- start 2024-03-09T08:52:23.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.642+0000 7f6cfce00640 1 -- start start 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.642+0000 7f6cfce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80710a0 0x7f6cf81eed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.642+0000 7f6cfce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf80c1f70 0x7f6cf80ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.642+0000 7f6cfce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c27a0 0x7f6cf80ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.642+0000 7f6cfce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf8070d50 con 0x7f6cf80c1f70 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.642+0000 7f6cfce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf8070830 con 0x7f6cf80c27a0 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.642+0000 7f6cfce00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf8070ed0 con 0x7f6cf80710a0 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.643+0000 7f6cf6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf80c1f70 0x7f6cf80ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.643+0000 7f6cf6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf80c1f70 0x7f6cf80ab1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59746/0 (socket says 172.21.15.67:59746) 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.643+0000 7f6cf6a00640 1 -- 172.21.15.67:0/3722252758 learned_addr learned my addr 172.21.15.67:0/3722252758 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.643+0000 7f6cf7400640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80710a0 0x7f6cf81eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.643+0000 7f6cf7e00640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c27a0 0x7f6cf80ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.643+0000 7f6cf6a00640 1 -- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80710a0 msgr2=0x7f6cf81eed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:23.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.643+0000 7f6cf6a00640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80710a0 0x7f6cf81eed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.643+0000 7f6cf6a00640 1 -- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c27a0 msgr2=0x7f6cf80ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:23.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.643+0000 7f6cf6a00640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c27a0 0x7f6cf80ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.643+0000 7f6cf6a00640 1 -- 172.21.15.67:0/3722252758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf814c8c0 con 0x7f6cf80c1f70 2024-03-09T08:52:23.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.643+0000 7f6cf6a00640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf80c1f70 0x7f6cf80ab1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400d800 tx=0x7f6ce400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:23.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.644+0000 7f6cef400640 1 -- 172.21.15.67:0/3722252758 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4018030 con 0x7f6cf80c1f70 2024-03-09T08:52:23.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.644+0000 7f6cef400640 1 -- 172.21.15.67:0/3722252758 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce40096a0 con 0x7f6cf80c1f70 2024-03-09T08:52:23.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.644+0000 7f6cef400640 1 -- 172.21.15.67:0/3722252758 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4009870 con 0x7f6cf80c1f70 2024-03-09T08:52:23.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.644+0000 7f6cfce00640 1 -- 172.21.15.67:0/3722252758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf814c130 con 0x7f6cf80c1f70 2024-03-09T08:52:23.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.644+0000 7f6cfce00640 1 -- 172.21.15.67:0/3722252758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf80c70b0 con 0x7f6cf80c1f70 2024-03-09T08:52:23.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.645+0000 7f6cfce00640 1 -- 172.21.15.67:0/3722252758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf804f3c0 con 0x7f6cf80c1f70 2024-03-09T08:52:23.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.646+0000 7f6cef400640 1 -- 172.21.15.67:0/3722252758 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6ce401b070 con 0x7f6cf80c1f70 2024-03-09T08:52:23.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.647+0000 7f6cef400640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cd806b4d0 0x7f6cd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:23.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.647+0000 7f6cef400640 1 -- 172.21.15.67:0/3722252758 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f6ce408a460 con 0x7f6cf80c1f70 2024-03-09T08:52:23.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.647+0000 7f6cf7400640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cd806b4d0 0x7f6cd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.648+0000 7f6cf7400640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cd806b4d0 0x7f6cd806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8036000 tx=0x7f6ce803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.652+0000 7f6cef400640 1 -- 172.21.15.67:0/3722252758 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ce40592e0 con 0x7f6cf80c1f70 2024-03-09T08:52:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.846+0000 7f6cfce00640 1 -- 172.21.15.67:0/3722252758 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6cf80c7880 con 0x7f6cd806b4d0 2024-03-09T08:52:23.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.882+0000 7f6cef400640 1 -- 172.21.15.67:0/3722252758 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251951 (secure 0 0 0) 0x7f6cf80c7880 con 0x7f6cd806b4d0 2024-03-09T08:52:23.882 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:23.883 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T08:52:23.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 -- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cd806b4d0 msgr2=0x7f6cd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:23.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cd806b4d0 0x7f6cd806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8036000 tx=0x7f6ce803d040 comp rx=0 tx=0).stop 2024-03-09T08:52:23.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 -- 172.21.15.67:0/3722252758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf80c1f70 msgr2=0x7f6cf80ab1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:23.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf80c1f70 0x7f6cf80ab1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400d800 tx=0x7f6ce400dcc0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 -- 172.21.15.67:0/3722252758 shutdown_connections 2024-03-09T08:52:23.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cd806b4d0 0x7f6cd806d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cf80c27a0 0x7f6cf80ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cf80c1f70 0x7f6cf80ab1a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 --2- 172.21.15.67:0/3722252758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cf80710a0 0x7f6cf81eed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:23.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 -- 172.21.15.67:0/3722252758 >> 172.21.15.67:0/3722252758 conn(0x7f6cf80c7300 msgr2=0x7f6cf80c0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:23.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 -- 172.21.15.67:0/3722252758 shutdown_connections 2024-03-09T08:52:23.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:23.889+0000 7f6cfce00640 1 -- 172.21.15.67:0/3722252758 wait complete. 2024-03-09T08:52:23.906 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-09T08:52:22.674281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2891776,"data_stored":1752388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":81,"num_write_kb":798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002107"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.947389+0000","last_change":"2024-03-09T08:52:18.248973+0000","last_active":"2024-03-09T08:52:19.947389+0000","last_peered":"2024-03-09T08:52:19.947389+0000","last_clean":"2024-03-09T08:52:19.947389+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T08:52:19.947389+0000","last_undegraded":"2024-03-09T08:52:19.947389+0000","last_fullsized":"2024-03-09T08:52:19.947389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:09:11.372616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203635+0000","last_change":"2024-03-09T08:52:18.225812+0000","last_active":"2024-03-09T08:52:19.203635+0000","last_peered":"2024-03-09T08:52:19.203635+0000","last_clean":"2024-03-09T08:52:19.203635+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T08:52:19.203635+0000","last_undegraded":"2024-03-09T08:52:19.203635+0000","last_fullsized":"2024-03-09T08:52:19.203635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:09:32.044185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344910+0000","last_change":"2024-03-09T08:52:18.222968+0000","last_active":"2024-03-09T08:52:20.344910+0000","last_peered":"2024-03-09T08:52:20.344910+0000","last_clean":"2024-03-09T08:52:20.344910+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T08:52:20.344910+0000","last_undegraded":"2024-03-09T08:52:20.344910+0000","last_fullsized":"2024-03-09T08:52:20.344910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:52:29.523963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171445+0000","last_change":"2024-03-09T08:52:18.248341+0000","last_active":"2024-03-09T08:52:20.171445+0000","last_peered":"2024-03-09T08:52:20.171445+0000","last_clean":"2024-03-09T08:52:20.171445+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T08:52:20.171445+0000","last_undegraded":"2024-03-09T08:52:20.171445+0000","last_fullsized":"2024-03-09T08:52:20.171445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:18:02.716066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946424+0000","last_change":"2024-03-09T08:52:18.242311+0000","last_active":"2024-03-09T08:52:19.946424+0000","last_peered":"2024-03-09T08:52:19.946424+0000","last_clean":"2024-03-09T08:52:19.946424+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T08:52:19.946424+0000","last_undegraded":"2024-03-09T08:52:19.946424+0000","last_fullsized":"2024-03-09T08:52:19.946424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:48:15.559438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170700+0000","last_change":"2024-03-09T08:52:18.233561+0000","last_active":"2024-03-09T08:52:20.170700+0000","last_peered":"2024-03-09T08:52:20.170700+0000","last_clean":"2024-03-09T08:52:20.170700+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T08:52:20.170700+0000","last_undegraded":"2024-03-09T08:52:20.170700+0000","last_fullsized":"2024-03-09T08:52:20.170700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:15:54.027155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946214+0000","last_change":"2024-03-09T08:52:18.249901+0000","last_active":"2024-03-09T08:52:19.946214+0000","last_peered":"2024-03-09T08:52:19.946214+0000","last_clean":"2024-03-09T08:52:19.946214+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T08:52:19.946214+0000","last_undegraded":"2024-03-09T08:52:19.946214+0000","last_fullsized":"2024-03-09T08:52:19.946214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:09.947401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.341993+0000","last_change":"2024-03-09T08:52:18.240975+0000","last_active":"2024-03-09T08:52:20.341993+0000","last_peered":"2024-03-09T08:52:20.341993+0000","last_clean":"2024-03-09T08:52:20.341993+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T08:52:20.341993+0000","last_undegraded":"2024-03-09T08:52:20.341993+0000","last_fullsized":"2024-03-09T08:52:20.341993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:30:09.363339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.947387+0000","last_change":"2024-03-09T08:52:18.253038+0000","last_active":"2024-03-09T08:52:19.947387+0000","last_peered":"2024-03-09T08:52:19.947387+0000","last_clean":"2024-03-09T08:52:19.947387+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T08:52:19.947387+0000","last_undegraded":"2024-03-09T08:52:19.947387+0000","last_fullsized":"2024-03-09T08:52:19.947387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:26:39.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240290+0000","last_change":"2024-03-09T08:52:18.234648+0000","last_active":"2024-03-09T08:52:19.240290+0000","last_peered":"2024-03-09T08:52:19.240290+0000","last_clean":"2024-03-09T08:52:19.240290+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T08:52:19.240290+0000","last_undegraded":"2024-03-09T08:52:19.240290+0000","last_fullsized":"2024-03-09T08:52:19.240290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:03.927280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240422+0000","last_change":"2024-03-09T08:52:18.234158+0000","last_active":"2024-03-09T08:52:19.240422+0000","last_peered":"2024-03-09T08:52:19.240422+0000","last_clean":"2024-03-09T08:52:19.240422+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T08:52:19.240422+0000","last_undegraded":"2024-03-09T08:52:19.240422+0000","last_fullsized":"2024-03-09T08:52:19.240422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:59:33.423558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.207312+0000","last_change":"2024-03-09T08:52:18.233451+0000","last_active":"2024-03-09T08:52:19.207312+0000","last_peered":"2024-03-09T08:52:19.207312+0000","last_clean":"2024-03-09T08:52:19.207312+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T08:52:19.207312+0000","last_undegraded":"2024-03-09T08:52:19.207312+0000","last_fullsized":"2024-03-09T08:52:19.207312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:56:11.270081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342372+0000","last_change":"2024-03-09T08:52:18.243645+0000","last_active":"2024-03-09T08:52:20.342372+0000","last_peered":"2024-03-09T08:52:20.342372+0000","last_clean":"2024-03-09T08:52:20.342372+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T08:52:20.342372+0000","last_undegraded":"2024-03-09T08:52:20.342372+0000","last_fullsized":"2024-03-09T08:52:20.342372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:19.501675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342584+0000","last_change":"2024-03-09T08:52:18.231316+0000","last_active":"2024-03-09T08:52:20.342584+0000","last_peered":"2024-03-09T08:52:20.342584+0000","last_clean":"2024-03-09T08:52:20.342584+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T08:52:20.342584+0000","last_undegraded":"2024-03-09T08:52:20.342584+0000","last_fullsized":"2024-03-09T08:52:20.342584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:25:25.715084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202255+0000","last_change":"2024-03-09T08:52:18.236376+0000","last_active":"2024-03-09T08:52:19.202255+0000","last_peered":"2024-03-09T08:52:19.202255+0000","last_clean":"2024-03-09T08:52:19.202255+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T08:52:19.202255+0000","last_undegraded":"2024-03-09T08:52:19.202255+0000","last_fullsized":"2024-03-09T08:52:19.202255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:34:04.412643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170630+0000","last_change":"2024-03-09T08:52:18.230289+0000","last_active":"2024-03-09T08:52:20.170630+0000","last_peered":"2024-03-09T08:52:20.170630+0000","last_clean":"2024-03-09T08:52:20.170630+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T08:52:20.170630+0000","last_undegraded":"2024-03-09T08:52:20.170630+0000","last_fullsized":"2024-03-09T08:52:20.170630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:34.507924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.172291+0000","last_change":"2024-03-09T08:52:18.242349+0000","last_active":"2024-03-09T08:52:20.172291+0000","last_peered":"2024-03-09T08:52:20.172291+0000","last_clean":"2024-03-09T08:52:20.172291+0000","last_became_active":"2024-03-09T08:52:18.241995+0000","last_became_peered":"2024-03-09T08:52:18.241995+0000","last_unstale":"2024-03-09T08:52:20.172291+0000","last_undegraded":"2024-03-09T08:52:20.172291+0000","last_fullsized":"2024-03-09T08:52:20.172291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:29:09.977984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203534+0000","last_change":"2024-03-09T08:52:18.233582+0000","last_active":"2024-03-09T08:52:19.203534+0000","last_peered":"2024-03-09T08:52:19.203534+0000","last_clean":"2024-03-09T08:52:19.203534+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T08:52:19.203534+0000","last_undegraded":"2024-03-09T08:52:19.203534+0000","last_fullsized":"2024-03-09T08:52:19.203534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:49.313616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.947213+0000","last_change":"2024-03-09T08:52:18.239801+0000","last_active":"2024-03-09T08:52:19.947213+0000","last_peered":"2024-03-09T08:52:19.947213+0000","last_clean":"2024-03-09T08:52:19.947213+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T08:52:19.947213+0000","last_undegraded":"2024-03-09T08:52:19.947213+0000","last_fullsized":"2024-03-09T08:52:19.947213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:38:06.836990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240359+0000","last_change":"2024-03-09T08:52:18.235565+0000","last_active":"2024-03-09T08:52:19.240359+0000","last_peered":"2024-03-09T08:52:19.240359+0000","last_clean":"2024-03-09T08:52:19.240359+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T08:52:19.240359+0000","last_undegraded":"2024-03-09T08:52:19.240359+0000","last_fullsized":"2024-03-09T08:52:19.240359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:52:14.994539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343956+0000","last_change":"2024-03-09T08:52:18.238373+0000","last_active":"2024-03-09T08:52:20.343956+0000","last_peered":"2024-03-09T08:52:20.343956+0000","last_clean":"2024-03-09T08:52:20.343956+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T08:52:20.343956+0000","last_undegraded":"2024-03-09T08:52:20.343956+0000","last_fullsized":"2024-03-09T08:52:20.343956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:05:31.480335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170962+0000","last_change":"2024-03-09T08:52:18.236855+0000","last_active":"2024-03-09T08:52:20.170962+0000","last_peered":"2024-03-09T08:52:20.170962+0000","last_clean":"2024-03-09T08:52:20.170962+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T08:52:20.170962+0000","last_undegraded":"2024-03-09T08:52:20.170962+0000","last_fullsized":"2024-03-09T08:52:20.170962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:21.941242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342904+0000","last_change":"2024-03-09T08:52:18.236642+0000","last_active":"2024-03-09T08:52:20.342904+0000","last_peered":"2024-03-09T08:52:20.342904+0000","last_clean":"2024-03-09T08:52:20.342904+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T08:52:20.342904+0000","last_undegraded":"2024-03-09T08:52:20.342904+0000","last_fullsized":"2024-03-09T08:52:20.342904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:51:46.825153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170679+0000","last_change":"2024-03-09T08:52:18.239943+0000","last_active":"2024-03-09T08:52:20.170679+0000","last_peered":"2024-03-09T08:52:20.170679+0000","last_clean":"2024-03-09T08:52:20.170679+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T08:52:20.170679+0000","last_undegraded":"2024-03-09T08:52:20.170679+0000","last_fullsized":"2024-03-09T08:52:20.170679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:18.727030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343663+0000","last_change":"2024-03-09T08:52:18.247784+0000","last_active":"2024-03-09T08:52:20.343663+0000","last_peered":"2024-03-09T08:52:20.343663+0000","last_clean":"2024-03-09T08:52:20.343663+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T08:52:20.343663+0000","last_undegraded":"2024-03-09T08:52:20.343663+0000","last_fullsized":"2024-03-09T08:52:20.343663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:46:59.008220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240561+0000","last_change":"2024-03-09T08:52:18.247145+0000","last_active":"2024-03-09T08:52:19.240561+0000","last_peered":"2024-03-09T08:52:19.240561+0000","last_clean":"2024-03-09T08:52:19.240561+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T08:52:19.240561+0000","last_undegraded":"2024-03-09T08:52:19.240561+0000","last_fullsized":"2024-03-09T08:52:19.240561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:46:24.111224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203373+0000","last_change":"2024-03-09T08:52:18.241038+0000","last_active":"2024-03-09T08:52:19.203373+0000","last_peered":"2024-03-09T08:52:19.203373+0000","last_clean":"2024-03-09T08:52:19.203373+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T08:52:19.203373+0000","last_undegraded":"2024-03-09T08:52:19.203373+0000","last_fullsized":"2024-03-09T08:52:19.203373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:02.169113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344169+0000","last_change":"2024-03-09T08:52:18.239951+0000","last_active":"2024-03-09T08:52:20.344169+0000","last_peered":"2024-03-09T08:52:20.344169+0000","last_clean":"2024-03-09T08:52:20.344169+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T08:52:20.344169+0000","last_undegraded":"2024-03-09T08:52:20.344169+0000","last_fullsized":"2024-03-09T08:52:20.344169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:51.102540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344029+0000","last_change":"2024-03-09T08:52:18.242257+0000","last_active":"2024-03-09T08:52:20.344029+0000","last_peered":"2024-03-09T08:52:20.344029+0000","last_clean":"2024-03-09T08:52:20.344029+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T08:52:20.344029+0000","last_undegraded":"2024-03-09T08:52:20.344029+0000","last_fullsized":"2024-03-09T08:52:20.344029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:40:00.854066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171019+0000","last_change":"2024-03-09T08:52:18.241628+0000","last_active":"2024-03-09T08:52:20.171019+0000","last_peered":"2024-03-09T08:52:20.171019+0000","last_clean":"2024-03-09T08:52:20.171019+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T08:52:20.171019+0000","last_undegraded":"2024-03-09T08:52:20.171019+0000","last_fullsized":"2024-03-09T08:52:20.171019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:07:58.570641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203397+0000","last_change":"2024-03-09T08:52:18.226980+0000","last_active":"2024-03-09T08:52:19.203397+0000","last_peered":"2024-03-09T08:52:19.203397+0000","last_clean":"2024-03-09T08:52:19.203397+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T08:52:19.203397+0000","last_undegraded":"2024-03-09T08:52:19.203397+0000","last_fullsized":"2024-03-09T08:52:19.203397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:28.312576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240525+0000","last_change":"2024-03-09T08:52:18.237869+0000","last_active":"2024-03-09T08:52:19.240525+0000","last_peered":"2024-03-09T08:52:19.240525+0000","last_clean":"2024-03-09T08:52:19.240525+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T08:52:19.240525+0000","last_undegraded":"2024-03-09T08:52:19.240525+0000","last_fullsized":"2024-03-09T08:52:19.240525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:01.571866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.206020+0000","last_change":"2024-03-09T08:52:18.229533+0000","last_active":"2024-03-09T08:52:19.206020+0000","last_peered":"2024-03-09T08:52:19.206020+0000","last_clean":"2024-03-09T08:52:19.206020+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T08:52:19.206020+0000","last_undegraded":"2024-03-09T08:52:19.206020+0000","last_fullsized":"2024-03-09T08:52:19.206020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:17.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342947+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T08:52:20.342947+0000","last_peered":"2024-03-09T08:52:20.342947+0000","last_clean":"2024-03-09T08:52:20.342947+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T08:52:20.342947+0000","last_undegraded":"2024-03-09T08:52:20.342947+0000","last_fullsized":"2024-03-09T08:52:20.342947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344401+0000","last_change":"2024-03-09T08:52:18.231141+0000","last_active":"2024-03-09T08:52:20.344401+0000","last_peered":"2024-03-09T08:52:20.344401+0000","last_clean":"2024-03-09T08:52:20.344401+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T08:52:20.344401+0000","last_undegraded":"2024-03-09T08:52:20.344401+0000","last_fullsized":"2024-03-09T08:52:20.344401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:25:00.471222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.239683+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T08:52:19.239683+0000","last_peered":"2024-03-09T08:52:19.239683+0000","last_clean":"2024-03-09T08:52:19.239683+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T08:52:19.239683+0000","last_undegraded":"2024-03-09T08:52:19.239683+0000","last_fullsized":"2024-03-09T08:52:19.239683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343053+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T08:52:20.343053+0000","last_peered":"2024-03-09T08:52:20.343053+0000","last_clean":"2024-03-09T08:52:20.343053+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T08:52:20.343053+0000","last_undegraded":"2024-03-09T08:52:20.343053+0000","last_fullsized":"2024-03-09T08:52:20.343053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343110+0000","last_change":"2024-03-09T08:52:18.234400+0000","last_active":"2024-03-09T08:52:20.343110+0000","last_peered":"2024-03-09T08:52:20.343110+0000","last_clean":"2024-03-09T08:52:20.343110+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T08:52:20.343110+0000","last_undegraded":"2024-03-09T08:52:20.343110+0000","last_fullsized":"2024-03-09T08:52:20.343110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:42:14.057367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170801+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T08:52:20.170801+0000","last_peered":"2024-03-09T08:52:20.170801+0000","last_clean":"2024-03-09T08:52:20.170801+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T08:52:20.170801+0000","last_undegraded":"2024-03-09T08:52:20.170801+0000","last_fullsized":"2024-03-09T08:52:20.170801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202195+0000","last_change":"2024-03-09T08:52:18.241156+0000","last_active":"2024-03-09T08:52:19.202195+0000","last_peered":"2024-03-09T08:52:19.202195+0000","last_clean":"2024-03-09T08:52:19.202195+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T08:52:19.202195+0000","last_undegraded":"2024-03-09T08:52:19.202195+0000","last_fullsized":"2024-03-09T08:52:19.202195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:55.323770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240316+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T08:52:19.240316+0000","last_peered":"2024-03-09T08:52:19.240316+0000","last_clean":"2024-03-09T08:52:19.240316+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T08:52:19.240316+0000","last_undegraded":"2024-03-09T08:52:19.240316+0000","last_fullsized":"2024-03-09T08:52:19.240316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:31.731626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171969+0000","last_change":"2024-03-09T08:52:18.251648+0000","last_active":"2024-03-09T08:52:20.171969+0000","last_peered":"2024-03-09T08:52:20.171969+0000","last_clean":"2024-03-09T08:52:20.171969+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T08:52:20.171969+0000","last_undegraded":"2024-03-09T08:52:20.171969+0000","last_fullsized":"2024-03-09T08:52:20.171969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:54:45.721673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171655+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T08:52:20.171655+0000","last_peered":"2024-03-09T08:52:20.171655+0000","last_clean":"2024-03-09T08:52:20.171655+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T08:52:20.171655+0000","last_undegraded":"2024-03-09T08:52:20.171655+0000","last_fullsized":"2024-03-09T08:52:20.171655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.345211+0000","last_change":"2024-03-09T08:52:18.236628+0000","last_active":"2024-03-09T08:52:20.345211+0000","last_peered":"2024-03-09T08:52:20.345211+0000","last_clean":"2024-03-09T08:52:20.345211+0000","last_became_active":"2024-03-09T08:52:18.235321+0000","last_became_peered":"2024-03-09T08:52:18.235321+0000","last_unstale":"2024-03-09T08:52:20.345211+0000","last_undegraded":"2024-03-09T08:52:20.345211+0000","last_fullsized":"2024-03-09T08:52:20.345211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:13:59.221325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203456+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T08:52:19.203456+0000","last_peered":"2024-03-09T08:52:19.203456+0000","last_clean":"2024-03-09T08:52:19.203456+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T08:52:19.203456+0000","last_undegraded":"2024-03-09T08:52:19.203456+0000","last_fullsized":"2024-03-09T08:52:19.203456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:41:10.578669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.207489+0000","last_change":"2024-03-09T08:52:18.244654+0000","last_active":"2024-03-09T08:52:19.207489+0000","last_peered":"2024-03-09T08:52:19.207489+0000","last_clean":"2024-03-09T08:52:19.207489+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T08:52:19.207489+0000","last_undegraded":"2024-03-09T08:52:19.207489+0000","last_fullsized":"2024-03-09T08:52:19.207489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:08:58.611725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240131+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T08:52:19.240131+0000","last_peered":"2024-03-09T08:52:19.240131+0000","last_clean":"2024-03-09T08:52:19.240131+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T08:52:19.240131+0000","last_undegraded":"2024-03-09T08:52:19.240131+0000","last_fullsized":"2024-03-09T08:52:19.240131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:01.724043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946832+0000","last_change":"2024-03-09T08:52:18.242496+0000","last_active":"2024-03-09T08:52:19.946832+0000","last_peered":"2024-03-09T08:52:19.946832+0000","last_clean":"2024-03-09T08:52:19.946832+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T08:52:19.946832+0000","last_undegraded":"2024-03-09T08:52:19.946832+0000","last_fullsized":"2024-03-09T08:52:19.946832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:35:17.142983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344205+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T08:52:20.344205+0000","last_peered":"2024-03-09T08:52:20.344205+0000","last_clean":"2024-03-09T08:52:20.344205+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T08:52:20.344205+0000","last_undegraded":"2024-03-09T08:52:20.344205+0000","last_fullsized":"2024-03-09T08:52:20.344205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:22:05.504557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342999+0000","last_change":"2024-03-09T08:52:18.236146+0000","last_active":"2024-03-09T08:52:20.342999+0000","last_peered":"2024-03-09T08:52:20.342999+0000","last_clean":"2024-03-09T08:52:20.342999+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T08:52:20.342999+0000","last_undegraded":"2024-03-09T08:52:20.342999+0000","last_fullsized":"2024-03-09T08:52:20.342999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:18:51.511926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171050+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T08:52:20.171050+0000","last_peered":"2024-03-09T08:52:20.171050+0000","last_clean":"2024-03-09T08:52:20.171050+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T08:52:20.171050+0000","last_undegraded":"2024-03-09T08:52:20.171050+0000","last_fullsized":"2024-03-09T08:52:20.171050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946410+0000","last_change":"2024-03-09T08:52:18.248093+0000","last_active":"2024-03-09T08:52:19.946410+0000","last_peered":"2024-03-09T08:52:19.946410+0000","last_clean":"2024-03-09T08:52:19.946410+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T08:52:19.946410+0000","last_undegraded":"2024-03-09T08:52:19.946410+0000","last_fullsized":"2024-03-09T08:52:19.946410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:29.414335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.239611+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T08:52:19.239611+0000","last_peered":"2024-03-09T08:52:19.239611+0000","last_clean":"2024-03-09T08:52:19.239611+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T08:52:19.239611+0000","last_undegraded":"2024-03-09T08:52:19.239611+0000","last_fullsized":"2024-03-09T08:52:19.239611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:07:29.577461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.239847+0000","last_change":"2024-03-09T08:52:18.245073+0000","last_active":"2024-03-09T08:52:19.239847+0000","last_peered":"2024-03-09T08:52:19.239847+0000","last_clean":"2024-03-09T08:52:19.239847+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T08:52:19.239847+0000","last_undegraded":"2024-03-09T08:52:19.239847+0000","last_fullsized":"2024-03-09T08:52:19.239847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:40:15.390735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202948+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T08:52:19.202948+0000","last_peered":"2024-03-09T08:52:19.202948+0000","last_clean":"2024-03-09T08:52:19.202948+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T08:52:19.202948+0000","last_undegraded":"2024-03-09T08:52:19.202948+0000","last_fullsized":"2024-03-09T08:52:19.202948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:31.859924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946479+0000","last_change":"2024-03-09T08:52:18.248755+0000","last_active":"2024-03-09T08:52:19.946479+0000","last_peered":"2024-03-09T08:52:19.946479+0000","last_clean":"2024-03-09T08:52:19.946479+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T08:52:19.946479+0000","last_undegraded":"2024-03-09T08:52:19.946479+0000","last_fullsized":"2024-03-09T08:52:19.946479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:37.083738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.207109+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T08:52:19.207109+0000","last_peered":"2024-03-09T08:52:19.207109+0000","last_clean":"2024-03-09T08:52:19.207109+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T08:52:19.207109+0000","last_undegraded":"2024-03-09T08:52:19.207109+0000","last_fullsized":"2024-03-09T08:52:19.207109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:10:34.698235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946939+0000","last_change":"2024-03-09T08:52:18.246145+0000","last_active":"2024-03-09T08:52:19.946939+0000","last_peered":"2024-03-09T08:52:19.946939+0000","last_clean":"2024-03-09T08:52:19.946939+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T08:52:19.946939+0000","last_undegraded":"2024-03-09T08:52:19.946939+0000","last_fullsized":"2024-03-09T08:52:19.946939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:29:36.213774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203616+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T08:52:19.203616+0000","last_peered":"2024-03-09T08:52:19.203616+0000","last_clean":"2024-03-09T08:52:19.203616+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T08:52:19.203616+0000","last_undegraded":"2024-03-09T08:52:19.203616+0000","last_fullsized":"2024-03-09T08:52:19.203616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:16.138216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.947778+0000","last_change":"2024-03-09T08:52:18.250420+0000","last_active":"2024-03-09T08:52:19.947778+0000","last_peered":"2024-03-09T08:52:19.947778+0000","last_clean":"2024-03-09T08:52:19.947778+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T08:52:19.947778+0000","last_undegraded":"2024-03-09T08:52:19.947778+0000","last_fullsized":"2024-03-09T08:52:19.947778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:48.826066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203615+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T08:52:19.203615+0000","last_peered":"2024-03-09T08:52:19.203615+0000","last_clean":"2024-03-09T08:52:19.203615+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T08:52:19.203615+0000","last_undegraded":"2024-03-09T08:52:19.203615+0000","last_fullsized":"2024-03-09T08:52:19.203615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:26:35.574903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171894+0000","last_change":"2024-03-09T08:52:18.250433+0000","last_active":"2024-03-09T08:52:20.171894+0000","last_peered":"2024-03-09T08:52:20.171894+0000","last_clean":"2024-03-09T08:52:20.171894+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T08:52:20.171894+0000","last_undegraded":"2024-03-09T08:52:20.171894+0000","last_fullsized":"2024-03-09T08:52:20.171894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:29.742337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203270+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T08:52:19.203270+0000","last_peered":"2024-03-09T08:52:19.203270+0000","last_clean":"2024-03-09T08:52:19.203270+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T08:52:19.203270+0000","last_undegraded":"2024-03-09T08:52:19.203270+0000","last_fullsized":"2024-03-09T08:52:19.203270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:27.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946885+0000","last_change":"2024-03-09T08:52:18.242671+0000","last_active":"2024-03-09T08:52:19.946885+0000","last_peered":"2024-03-09T08:52:19.946885+0000","last_clean":"2024-03-09T08:52:19.946885+0000","last_became_active":"2024-03-09T08:52:18.242258+0000","last_became_peered":"2024-03-09T08:52:18.242258+0000","last_unstale":"2024-03-09T08:52:19.946885+0000","last_undegraded":"2024-03-09T08:52:19.946885+0000","last_fullsized":"2024-03-09T08:52:19.946885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:15.030669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946012+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T08:52:19.946012+0000","last_peered":"2024-03-09T08:52:19.946012+0000","last_clean":"2024-03-09T08:52:19.946012+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T08:52:19.946012+0000","last_undegraded":"2024-03-09T08:52:19.946012+0000","last_fullsized":"2024-03-09T08:52:19.946012+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170651+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T08:52:20.170651+0000","last_peered":"2024-03-09T08:52:20.170651+0000","last_clean":"2024-03-09T08:52:20.170651+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T08:52:20.170651+0000","last_undegraded":"2024-03-09T08:52:20.170651+0000","last_fullsized":"2024-03-09T08:52:20.170651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342508+0000","last_change":"2024-03-09T08:52:18.235367+0000","last_active":"2024-03-09T08:52:20.342508+0000","last_peered":"2024-03-09T08:52:20.342508+0000","last_clean":"2024-03-09T08:52:20.342508+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T08:52:20.342508+0000","last_undegraded":"2024-03-09T08:52:20.342508+0000","last_fullsized":"2024-03-09T08:52:20.342508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:40.736440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.945746+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T08:52:19.945746+0000","last_peered":"2024-03-09T08:52:19.945746+0000","last_clean":"2024-03-09T08:52:19.945746+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T08:52:19.945746+0000","last_undegraded":"2024-03-09T08:52:19.945746+0000","last_fullsized":"2024-03-09T08:52:19.945746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.341951+0000","last_change":"2024-03-09T08:52:18.214396+0000","last_active":"2024-03-09T08:52:20.341951+0000","last_peered":"2024-03-09T08:52:20.341951+0000","last_clean":"2024-03-09T08:52:20.341951+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T08:52:20.341951+0000","last_undegraded":"2024-03-09T08:52:20.341951+0000","last_fullsized":"2024-03-09T08:52:20.341951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:24:10.719351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203472+0000","last_change":"2024-03-09T08:52:17.271847+0000","last_active":"2024-03-09T08:52:19.203472+0000","last_peered":"2024-03-09T08:52:19.203472+0000","last_clean":"2024-03-09T08:52:19.203472+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T08:52:19.203472+0000","last_undegraded":"2024-03-09T08:52:19.203472+0000","last_fullsized":"2024-03-09T08:52:19.203472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:27.418674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202590+0000","last_change":"2024-03-09T08:52:18.235832+0000","last_active":"2024-03-09T08:52:19.202590+0000","last_peered":"2024-03-09T08:52:19.202590+0000","last_clean":"2024-03-09T08:52:19.202590+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T08:52:19.202590+0000","last_undegraded":"2024-03-09T08:52:19.202590+0000","last_fullsized":"2024-03-09T08:52:19.202590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:38:58.751526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171135+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T08:52:20.171135+0000","last_peered":"2024-03-09T08:52:20.171135+0000","last_clean":"2024-03-09T08:52:20.171135+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T08:52:20.171135+0000","last_undegraded":"2024-03-09T08:52:20.171135+0000","last_fullsized":"2024-03-09T08:52:20.171135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.206422+0000","last_change":"2024-03-09T08:52:18.227365+0000","last_active":"2024-03-09T08:52:19.206422+0000","last_peered":"2024-03-09T08:52:19.206422+0000","last_clean":"2024-03-09T08:52:19.206422+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T08:52:19.206422+0000","last_undegraded":"2024-03-09T08:52:19.206422+0000","last_fullsized":"2024-03-09T08:52:19.206422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:26.357063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203208+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T08:52:19.203208+0000","last_peered":"2024-03-09T08:52:19.203208+0000","last_clean":"2024-03-09T08:52:19.203208+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T08:52:19.203208+0000","last_undegraded":"2024-03-09T08:52:19.203208+0000","last_fullsized":"2024-03-09T08:52:19.203208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.172365+0000","last_change":"2024-03-09T08:52:18.250999+0000","last_active":"2024-03-09T08:52:20.172365+0000","last_peered":"2024-03-09T08:52:20.172365+0000","last_clean":"2024-03-09T08:52:20.172365+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T08:52:20.172365+0000","last_undegraded":"2024-03-09T08:52:20.172365+0000","last_fullsized":"2024-03-09T08:52:20.172365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:49:54.496603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.947173+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T08:52:19.947173+0000","last_peered":"2024-03-09T08:52:19.947173+0000","last_clean":"2024-03-09T08:52:19.947173+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T08:52:19.947173+0000","last_undegraded":"2024-03-09T08:52:19.947173+0000","last_fullsized":"2024-03-09T08:52:19.947173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202531+0000","last_change":"2024-03-09T08:52:18.234887+0000","last_active":"2024-03-09T08:52:19.202531+0000","last_peered":"2024-03-09T08:52:19.202531+0000","last_clean":"2024-03-09T08:52:19.202531+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T08:52:19.202531+0000","last_undegraded":"2024-03-09T08:52:19.202531+0000","last_fullsized":"2024-03-09T08:52:19.202531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:44.661882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203765+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T08:52:19.203765+0000","last_peered":"2024-03-09T08:52:19.203765+0000","last_clean":"2024-03-09T08:52:19.203765+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T08:52:19.203765+0000","last_undegraded":"2024-03-09T08:52:19.203765+0000","last_fullsized":"2024-03-09T08:52:19.203765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:30.201471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344099+0000","last_change":"2024-03-09T08:52:18.245442+0000","last_active":"2024-03-09T08:52:20.344099+0000","last_peered":"2024-03-09T08:52:20.344099+0000","last_clean":"2024-03-09T08:52:20.344099+0000","last_became_active":"2024-03-09T08:52:18.245164+0000","last_became_peered":"2024-03-09T08:52:18.245164+0000","last_unstale":"2024-03-09T08:52:20.344099+0000","last_undegraded":"2024-03-09T08:52:20.344099+0000","last_fullsized":"2024-03-09T08:52:20.344099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:31:08.413041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240686+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T08:52:19.240686+0000","last_peered":"2024-03-09T08:52:19.240686+0000","last_clean":"2024-03-09T08:52:19.240686+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T08:52:19.240686+0000","last_undegraded":"2024-03-09T08:52:19.240686+0000","last_fullsized":"2024-03-09T08:52:19.240686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:35:38.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344838+0000","last_change":"2024-03-09T08:52:18.229739+0000","last_active":"2024-03-09T08:52:20.344838+0000","last_peered":"2024-03-09T08:52:20.344838+0000","last_clean":"2024-03-09T08:52:20.344838+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T08:52:20.344838+0000","last_undegraded":"2024-03-09T08:52:20.344838+0000","last_fullsized":"2024-03-09T08:52:20.344838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:24.394779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.945740+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T08:52:19.945740+0000","last_peered":"2024-03-09T08:52:19.945740+0000","last_clean":"2024-03-09T08:52:19.945740+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T08:52:19.945740+0000","last_undegraded":"2024-03-09T08:52:19.945740+0000","last_fullsized":"2024-03-09T08:52:19.945740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.206203+0000","last_change":"2024-03-09T08:52:18.217046+0000","last_active":"2024-03-09T08:52:19.206203+0000","last_peered":"2024-03-09T08:52:19.206203+0000","last_clean":"2024-03-09T08:52:19.206203+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T08:52:19.206203+0000","last_undegraded":"2024-03-09T08:52:19.206203+0000","last_fullsized":"2024-03-09T08:52:19.206203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:14.884963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202279+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T08:52:19.202279+0000","last_peered":"2024-03-09T08:52:19.202279+0000","last_clean":"2024-03-09T08:52:19.202279+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T08:52:19.202279+0000","last_undegraded":"2024-03-09T08:52:19.202279+0000","last_fullsized":"2024-03-09T08:52:19.202279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:21:59.990396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342910+0000","last_change":"2024-03-09T08:52:18.233208+0000","last_active":"2024-03-09T08:52:20.342910+0000","last_peered":"2024-03-09T08:52:20.342910+0000","last_clean":"2024-03-09T08:52:20.342910+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T08:52:20.342910+0000","last_undegraded":"2024-03-09T08:52:20.342910+0000","last_fullsized":"2024-03-09T08:52:20.342910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:17.589783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202515+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T08:52:19.202515+0000","last_peered":"2024-03-09T08:52:19.202515+0000","last_clean":"2024-03-09T08:52:19.202515+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T08:52:19.202515+0000","last_undegraded":"2024-03-09T08:52:19.202515+0000","last_fullsized":"2024-03-09T08:52:19.202515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:45:45.612953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342792+0000","last_change":"2024-03-09T08:52:18.244306+0000","last_active":"2024-03-09T08:52:20.342792+0000","last_peered":"2024-03-09T08:52:20.342792+0000","last_clean":"2024-03-09T08:52:20.342792+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T08:52:20.342792+0000","last_undegraded":"2024-03-09T08:52:20.342792+0000","last_fullsized":"2024-03-09T08:52:20.342792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:58.365524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946557+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T08:52:19.946557+0000","last_peered":"2024-03-09T08:52:19.946557+0000","last_clean":"2024-03-09T08:52:19.946557+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T08:52:19.946557+0000","last_undegraded":"2024-03-09T08:52:19.946557+0000","last_fullsized":"2024-03-09T08:52:19.946557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.239920+0000","last_change":"2024-03-09T08:52:18.234372+0000","last_active":"2024-03-09T08:52:19.239920+0000","last_peered":"2024-03-09T08:52:19.239920+0000","last_clean":"2024-03-09T08:52:19.239920+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T08:52:19.239920+0000","last_undegraded":"2024-03-09T08:52:19.239920+0000","last_fullsized":"2024-03-09T08:52:19.239920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:09:33.422907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.172082+0000","last_change":"2024-03-09T08:52:18.228419+0000","last_active":"2024-03-09T08:52:20.172082+0000","last_peered":"2024-03-09T08:52:20.172082+0000","last_clean":"2024-03-09T08:52:20.172082+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T08:52:20.172082+0000","last_undegraded":"2024-03-09T08:52:20.172082+0000","last_fullsized":"2024-03-09T08:52:20.172082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:19:12.533402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946320+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T08:52:19.946320+0000","last_peered":"2024-03-09T08:52:19.946320+0000","last_clean":"2024-03-09T08:52:19.946320+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T08:52:19.946320+0000","last_undegraded":"2024-03-09T08:52:19.946320+0000","last_fullsized":"2024-03-09T08:52:19.946320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171447+0000","last_change":"2024-03-09T08:52:18.230629+0000","last_active":"2024-03-09T08:52:20.171447+0000","last_peered":"2024-03-09T08:52:20.171447+0000","last_clean":"2024-03-09T08:52:20.171447+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T08:52:20.171447+0000","last_undegraded":"2024-03-09T08:52:20.171447+0000","last_fullsized":"2024-03-09T08:52:20.171447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:46:38.528839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946964+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T08:52:19.946964+0000","last_peered":"2024-03-09T08:52:19.946964+0000","last_clean":"2024-03-09T08:52:19.946964+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T08:52:19.946964+0000","last_undegraded":"2024-03-09T08:52:19.946964+0000","last_fullsized":"2024-03-09T08:52:19.946964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344989+0000","last_change":"2024-03-09T08:52:18.236946+0000","last_active":"2024-03-09T08:52:20.344989+0000","last_peered":"2024-03-09T08:52:20.344989+0000","last_clean":"2024-03-09T08:52:20.344989+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T08:52:20.344989+0000","last_undegraded":"2024-03-09T08:52:20.344989+0000","last_fullsized":"2024-03-09T08:52:20.344989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:45.372160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343766+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T08:52:20.343766+0000","last_peered":"2024-03-09T08:52:20.343766+0000","last_clean":"2024-03-09T08:52:20.343766+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T08:52:20.343766+0000","last_undegraded":"2024-03-09T08:52:20.343766+0000","last_fullsized":"2024-03-09T08:52:20.343766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343927+0000","last_change":"2024-03-09T08:52:18.235099+0000","last_active":"2024-03-09T08:52:20.343927+0000","last_peered":"2024-03-09T08:52:20.343927+0000","last_clean":"2024-03-09T08:52:20.343927+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T08:52:20.343927+0000","last_undegraded":"2024-03-09T08:52:20.343927+0000","last_fullsized":"2024-03-09T08:52:20.343927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:39:28.882461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.345109+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T08:52:20.345109+0000","last_peered":"2024-03-09T08:52:20.345109+0000","last_clean":"2024-03-09T08:52:20.345109+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T08:52:20.345109+0000","last_undegraded":"2024-03-09T08:52:20.345109+0000","last_fullsized":"2024-03-09T08:52:20.345109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:41:40.332687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T08:52:23.910 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T08:52:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.177+0000 7fe377e00640 1 Processor -- start 2024-03-09T08:52:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.178+0000 7fe377e00640 1 -- start start 2024-03-09T08:52:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.179+0000 7fe377e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3780c1f70 0x7fe378150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.179+0000 7fe377e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe378154080 0x7fe378151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.179+0000 7fe377e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3780c81e0 con 0x7fe3780710a0 2024-03-09T08:52:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.179+0000 7fe377e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3780c32b0 con 0x7fe378154080 2024-03-09T08:52:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.179+0000 7fe377e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3780c2fd0 con 0x7fe3780c1f70 2024-03-09T08:52:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.179+0000 7fe36e000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe3780710a0 0x7fe378153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46338/0 (socket says 172.21.15.67:46338) 2024-03-09T08:52:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.179+0000 7fe36e000640 1 -- 172.21.15.67:0/3847445005 learned_addr learned my addr 172.21.15.67:0/3847445005 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.179+0000 7fe377400640 1 --2- 172.21.15.67:0/3847445005 >> v2:172.21.15.67:3300/0 conn(0x7fe378154080 0x7fe378151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.179+0000 7fe376a00640 1 --2- 172.21.15.67:0/3847445005 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3780c1f70 0x7fe378150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.180+0000 7fe376000640 1 -- 172.21.15.67:0/3847445005 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408079086 0 0) 0x7fe3780c81e0 con 0x7fe3780710a0 2024-03-09T08:52:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.180+0000 7fe376000640 1 -- 172.21.15.67:0/3847445005 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe358003520 con 0x7fe3780710a0 2024-03-09T08:52:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.180+0000 7fe377400640 1 -- 172.21.15.67:0/3847445005 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3780c1f70 msgr2=0x7fe378150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.180+0000 7fe377400640 1 --2- 172.21.15.67:0/3847445005 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3780c1f70 0x7fe378150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.180+0000 7fe377400640 1 -- 172.21.15.67:0/3847445005 >> v1:172.21.15.67:6789/0 conn(0x7fe3780710a0 legacy=0x7fe378153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.180+0000 7fe377400640 1 -- 172.21.15.67:0/3847445005 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3780c3700 con 0x7fe378154080 2024-03-09T08:52:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.181+0000 7fe377400640 1 --2- 172.21.15.67:0/3847445005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378154080 0x7fe378151190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe370009d90 tx=0x7fe37002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e9c4bcb540503c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.181+0000 7fe376000640 1 -- 172.21.15.67:0/3847445005 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370039070 con 0x7fe378154080 2024-03-09T08:52:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.181+0000 7fe376000640 1 -- 172.21.15.67:0/3847445005 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37002cbc0 con 0x7fe378154080 2024-03-09T08:52:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.181+0000 7fe376000640 1 -- 172.21.15.67:0/3847445005 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370034440 con 0x7fe378154080 2024-03-09T08:52:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.182+0000 7fe377e00640 1 -- 172.21.15.67:0/3847445005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378154080 msgr2=0x7fe378151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.182+0000 7fe377e00640 1 --2- 172.21.15.67:0/3847445005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378154080 0x7fe378151190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe370009d90 tx=0x7fe37002be10 comp rx=0 tx=0).stop 2024-03-09T08:52:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.182+0000 7fe377e00640 1 -- 172.21.15.67:0/3847445005 shutdown_connections 2024-03-09T08:52:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.182+0000 7fe377e00640 1 --2- 172.21.15.67:0/3847445005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378154080 0x7fe378151190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.182+0000 7fe377e00640 1 --2- 172.21.15.67:0/3847445005 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3780c1f70 0x7fe378150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.182+0000 7fe377e00640 1 -- 172.21.15.67:0/3847445005 >> 172.21.15.67:0/3847445005 conn(0x7fe3780c7300 msgr2=0x7fe3780c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.182+0000 7fe377e00640 1 -- 172.21.15.67:0/3847445005 shutdown_connections 2024-03-09T08:52:24.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.182+0000 7fe377e00640 1 -- 172.21.15.67:0/3847445005 wait complete. 2024-03-09T08:52:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.184+0000 7fe377e00640 1 Processor -- start 2024-03-09T08:52:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.184+0000 7fe377e00640 1 -- start start 2024-03-09T08:52:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3780710a0 0x7fe378153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3780c1f70 0x7fe3781567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378154080 0x7fe378148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe378070d50 con 0x7fe378154080 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe378070830 con 0x7fe3780710a0 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe378070ed0 con 0x7fe3780c1f70 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378154080 0x7fe378148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378154080 0x7fe378148550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59774/0 (socket says 172.21.15.67:59774) 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377400640 1 -- 172.21.15.67:0/781426913 learned_addr learned my addr 172.21.15.67:0/781426913 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377400640 1 -- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3780c1f70 msgr2=0x7fe3781567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377400640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3780c1f70 0x7fe3781567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377400640 1 -- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3780710a0 msgr2=0x7fe378153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377400640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3780710a0 0x7fe378153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.185+0000 7fe377400640 1 -- 172.21.15.67:0/781426913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe378157910 con 0x7fe378154080 2024-03-09T08:52:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.186+0000 7fe377400640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378154080 0x7fe378148550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe370009eb0 tx=0x7fe370004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.186+0000 7fe36f400640 1 -- 172.21.15.67:0/781426913 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370039050 con 0x7fe378154080 2024-03-09T08:52:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.186+0000 7fe36f400640 1 -- 172.21.15.67:0/781426913 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe370045070 con 0x7fe378154080 2024-03-09T08:52:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.186+0000 7fe36f400640 1 -- 172.21.15.67:0/781426913 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370009450 con 0x7fe378154080 2024-03-09T08:52:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.186+0000 7fe377e00640 1 -- 172.21.15.67:0/781426913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe378157bf0 con 0x7fe378154080 2024-03-09T08:52:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.186+0000 7fe377e00640 1 -- 172.21.15.67:0/781426913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3781eacc0 con 0x7fe378154080 2024-03-09T08:52:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.187+0000 7fe377e00640 1 -- 172.21.15.67:0/781426913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3780c1590 con 0x7fe378154080 2024-03-09T08:52:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.188+0000 7fe36f400640 1 -- 172.21.15.67:0/781426913 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe37003e830 con 0x7fe378154080 2024-03-09T08:52:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.189+0000 7fe36f400640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe35406b500 0x7fe35406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.189+0000 7fe36f400640 1 -- 172.21.15.67:0/781426913 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7fe3700ace40 con 0x7fe378154080 2024-03-09T08:52:24.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.190+0000 7fe376a00640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe35406b500 0x7fe35406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.191+0000 7fe376a00640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe35406b500 0x7fe35406d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe368006a90 tx=0x7fe368006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:24.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.194+0000 7fe36f400640 1 -- 172.21.15.67:0/781426913 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe3780c1590 con 0x7fe378154080 2024-03-09T08:52:24.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.394+0000 7fe377e00640 1 -- 172.21.15.67:0/781426913 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3781eaec0 con 0x7fe35406b500 2024-03-09T08:52:24.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.414+0000 7fe36f400640 1 -- 172.21.15.67:0/781426913 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251951 (secure 0 0 0) 0x7fe3781eaec0 con 0x7fe35406b500 2024-03-09T08:52:24.414 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:24.415 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T08:52:24.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 -- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe35406b500 msgr2=0x7fe35406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe35406b500 0x7fe35406d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe368006a90 tx=0x7fe368006a20 comp rx=0 tx=0).stop 2024-03-09T08:52:24.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 -- 172.21.15.67:0/781426913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378154080 msgr2=0x7fe378148550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378154080 0x7fe378148550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe370009eb0 tx=0x7fe370004a40 comp rx=0 tx=0).stop 2024-03-09T08:52:24.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 -- 172.21.15.67:0/781426913 shutdown_connections 2024-03-09T08:52:24.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe35406b500 0x7fe35406d9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe378154080 0x7fe378148550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3780c1f70 0x7fe3781567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 --2- 172.21.15.67:0/781426913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3780710a0 0x7fe378153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 -- 172.21.15.67:0/781426913 >> 172.21.15.67:0/781426913 conn(0x7fe3780c7300 msgr2=0x7fe37806d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:24.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 -- 172.21.15.67:0/781426913 shutdown_connections 2024-03-09T08:52:24.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.420+0000 7fe377e00640 1 -- 172.21.15.67:0/781426913 wait complete. 2024-03-09T08:52:24.436 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-09T08:52:22.674281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2891776,"data_stored":1752388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":81,"num_write_kb":798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002107"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.947389+0000","last_change":"2024-03-09T08:52:18.248973+0000","last_active":"2024-03-09T08:52:19.947389+0000","last_peered":"2024-03-09T08:52:19.947389+0000","last_clean":"2024-03-09T08:52:19.947389+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T08:52:19.947389+0000","last_undegraded":"2024-03-09T08:52:19.947389+0000","last_fullsized":"2024-03-09T08:52:19.947389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:09:11.372616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203635+0000","last_change":"2024-03-09T08:52:18.225812+0000","last_active":"2024-03-09T08:52:19.203635+0000","last_peered":"2024-03-09T08:52:19.203635+0000","last_clean":"2024-03-09T08:52:19.203635+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T08:52:19.203635+0000","last_undegraded":"2024-03-09T08:52:19.203635+0000","last_fullsized":"2024-03-09T08:52:19.203635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:09:32.044185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344910+0000","last_change":"2024-03-09T08:52:18.222968+0000","last_active":"2024-03-09T08:52:20.344910+0000","last_peered":"2024-03-09T08:52:20.344910+0000","last_clean":"2024-03-09T08:52:20.344910+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T08:52:20.344910+0000","last_undegraded":"2024-03-09T08:52:20.344910+0000","last_fullsized":"2024-03-09T08:52:20.344910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:52:29.523963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171445+0000","last_change":"2024-03-09T08:52:18.248341+0000","last_active":"2024-03-09T08:52:20.171445+0000","last_peered":"2024-03-09T08:52:20.171445+0000","last_clean":"2024-03-09T08:52:20.171445+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T08:52:20.171445+0000","last_undegraded":"2024-03-09T08:52:20.171445+0000","last_fullsized":"2024-03-09T08:52:20.171445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:18:02.716066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946424+0000","last_change":"2024-03-09T08:52:18.242311+0000","last_active":"2024-03-09T08:52:19.946424+0000","last_peered":"2024-03-09T08:52:19.946424+0000","last_clean":"2024-03-09T08:52:19.946424+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T08:52:19.946424+0000","last_undegraded":"2024-03-09T08:52:19.946424+0000","last_fullsized":"2024-03-09T08:52:19.946424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:48:15.559438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170700+0000","last_change":"2024-03-09T08:52:18.233561+0000","last_active":"2024-03-09T08:52:20.170700+0000","last_peered":"2024-03-09T08:52:20.170700+0000","last_clean":"2024-03-09T08:52:20.170700+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T08:52:20.170700+0000","last_undegraded":"2024-03-09T08:52:20.170700+0000","last_fullsized":"2024-03-09T08:52:20.170700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:15:54.027155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946214+0000","last_change":"2024-03-09T08:52:18.249901+0000","last_active":"2024-03-09T08:52:19.946214+0000","last_peered":"2024-03-09T08:52:19.946214+0000","last_clean":"2024-03-09T08:52:19.946214+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T08:52:19.946214+0000","last_undegraded":"2024-03-09T08:52:19.946214+0000","last_fullsized":"2024-03-09T08:52:19.946214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:09.947401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.341993+0000","last_change":"2024-03-09T08:52:18.240975+0000","last_active":"2024-03-09T08:52:20.341993+0000","last_peered":"2024-03-09T08:52:20.341993+0000","last_clean":"2024-03-09T08:52:20.341993+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T08:52:20.341993+0000","last_undegraded":"2024-03-09T08:52:20.341993+0000","last_fullsized":"2024-03-09T08:52:20.341993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:30:09.363339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.947387+0000","last_change":"2024-03-09T08:52:18.253038+0000","last_active":"2024-03-09T08:52:19.947387+0000","last_peered":"2024-03-09T08:52:19.947387+0000","last_clean":"2024-03-09T08:52:19.947387+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T08:52:19.947387+0000","last_undegraded":"2024-03-09T08:52:19.947387+0000","last_fullsized":"2024-03-09T08:52:19.947387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:26:39.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240290+0000","last_change":"2024-03-09T08:52:18.234648+0000","last_active":"2024-03-09T08:52:19.240290+0000","last_peered":"2024-03-09T08:52:19.240290+0000","last_clean":"2024-03-09T08:52:19.240290+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T08:52:19.240290+0000","last_undegraded":"2024-03-09T08:52:19.240290+0000","last_fullsized":"2024-03-09T08:52:19.240290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:03.927280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240422+0000","last_change":"2024-03-09T08:52:18.234158+0000","last_active":"2024-03-09T08:52:19.240422+0000","last_peered":"2024-03-09T08:52:19.240422+0000","last_clean":"2024-03-09T08:52:19.240422+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T08:52:19.240422+0000","last_undegraded":"2024-03-09T08:52:19.240422+0000","last_fullsized":"2024-03-09T08:52:19.240422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:59:33.423558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.207312+0000","last_change":"2024-03-09T08:52:18.233451+0000","last_active":"2024-03-09T08:52:19.207312+0000","last_peered":"2024-03-09T08:52:19.207312+0000","last_clean":"2024-03-09T08:52:19.207312+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T08:52:19.207312+0000","last_undegraded":"2024-03-09T08:52:19.207312+0000","last_fullsized":"2024-03-09T08:52:19.207312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:56:11.270081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342372+0000","last_change":"2024-03-09T08:52:18.243645+0000","last_active":"2024-03-09T08:52:20.342372+0000","last_peered":"2024-03-09T08:52:20.342372+0000","last_clean":"2024-03-09T08:52:20.342372+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T08:52:20.342372+0000","last_undegraded":"2024-03-09T08:52:20.342372+0000","last_fullsized":"2024-03-09T08:52:20.342372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:19.501675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342584+0000","last_change":"2024-03-09T08:52:18.231316+0000","last_active":"2024-03-09T08:52:20.342584+0000","last_peered":"2024-03-09T08:52:20.342584+0000","last_clean":"2024-03-09T08:52:20.342584+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T08:52:20.342584+0000","last_undegraded":"2024-03-09T08:52:20.342584+0000","last_fullsized":"2024-03-09T08:52:20.342584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:25:25.715084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202255+0000","last_change":"2024-03-09T08:52:18.236376+0000","last_active":"2024-03-09T08:52:19.202255+0000","last_peered":"2024-03-09T08:52:19.202255+0000","last_clean":"2024-03-09T08:52:19.202255+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T08:52:19.202255+0000","last_undegraded":"2024-03-09T08:52:19.202255+0000","last_fullsized":"2024-03-09T08:52:19.202255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:34:04.412643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170630+0000","last_change":"2024-03-09T08:52:18.230289+0000","last_active":"2024-03-09T08:52:20.170630+0000","last_peered":"2024-03-09T08:52:20.170630+0000","last_clean":"2024-03-09T08:52:20.170630+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T08:52:20.170630+0000","last_undegraded":"2024-03-09T08:52:20.170630+0000","last_fullsized":"2024-03-09T08:52:20.170630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:34.507924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.172291+0000","last_change":"2024-03-09T08:52:18.242349+0000","last_active":"2024-03-09T08:52:20.172291+0000","last_peered":"2024-03-09T08:52:20.172291+0000","last_clean":"2024-03-09T08:52:20.172291+0000","last_became_active":"2024-03-09T08:52:18.241995+0000","last_became_peered":"2024-03-09T08:52:18.241995+0000","last_unstale":"2024-03-09T08:52:20.172291+0000","last_undegraded":"2024-03-09T08:52:20.172291+0000","last_fullsized":"2024-03-09T08:52:20.172291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:29:09.977984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203534+0000","last_change":"2024-03-09T08:52:18.233582+0000","last_active":"2024-03-09T08:52:19.203534+0000","last_peered":"2024-03-09T08:52:19.203534+0000","last_clean":"2024-03-09T08:52:19.203534+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T08:52:19.203534+0000","last_undegraded":"2024-03-09T08:52:19.203534+0000","last_fullsized":"2024-03-09T08:52:19.203534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:49.313616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.947213+0000","last_change":"2024-03-09T08:52:18.239801+0000","last_active":"2024-03-09T08:52:19.947213+0000","last_peered":"2024-03-09T08:52:19.947213+0000","last_clean":"2024-03-09T08:52:19.947213+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T08:52:19.947213+0000","last_undegraded":"2024-03-09T08:52:19.947213+0000","last_fullsized":"2024-03-09T08:52:19.947213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:38:06.836990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240359+0000","last_change":"2024-03-09T08:52:18.235565+0000","last_active":"2024-03-09T08:52:19.240359+0000","last_peered":"2024-03-09T08:52:19.240359+0000","last_clean":"2024-03-09T08:52:19.240359+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T08:52:19.240359+0000","last_undegraded":"2024-03-09T08:52:19.240359+0000","last_fullsized":"2024-03-09T08:52:19.240359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:52:14.994539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343956+0000","last_change":"2024-03-09T08:52:18.238373+0000","last_active":"2024-03-09T08:52:20.343956+0000","last_peered":"2024-03-09T08:52:20.343956+0000","last_clean":"2024-03-09T08:52:20.343956+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T08:52:20.343956+0000","last_undegraded":"2024-03-09T08:52:20.343956+0000","last_fullsized":"2024-03-09T08:52:20.343956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:05:31.480335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170962+0000","last_change":"2024-03-09T08:52:18.236855+0000","last_active":"2024-03-09T08:52:20.170962+0000","last_peered":"2024-03-09T08:52:20.170962+0000","last_clean":"2024-03-09T08:52:20.170962+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T08:52:20.170962+0000","last_undegraded":"2024-03-09T08:52:20.170962+0000","last_fullsized":"2024-03-09T08:52:20.170962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:21.941242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342904+0000","last_change":"2024-03-09T08:52:18.236642+0000","last_active":"2024-03-09T08:52:20.342904+0000","last_peered":"2024-03-09T08:52:20.342904+0000","last_clean":"2024-03-09T08:52:20.342904+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T08:52:20.342904+0000","last_undegraded":"2024-03-09T08:52:20.342904+0000","last_fullsized":"2024-03-09T08:52:20.342904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:51:46.825153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170679+0000","last_change":"2024-03-09T08:52:18.239943+0000","last_active":"2024-03-09T08:52:20.170679+0000","last_peered":"2024-03-09T08:52:20.170679+0000","last_clean":"2024-03-09T08:52:20.170679+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T08:52:20.170679+0000","last_undegraded":"2024-03-09T08:52:20.170679+0000","last_fullsized":"2024-03-09T08:52:20.170679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:18.727030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343663+0000","last_change":"2024-03-09T08:52:18.247784+0000","last_active":"2024-03-09T08:52:20.343663+0000","last_peered":"2024-03-09T08:52:20.343663+0000","last_clean":"2024-03-09T08:52:20.343663+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T08:52:20.343663+0000","last_undegraded":"2024-03-09T08:52:20.343663+0000","last_fullsized":"2024-03-09T08:52:20.343663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:46:59.008220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240561+0000","last_change":"2024-03-09T08:52:18.247145+0000","last_active":"2024-03-09T08:52:19.240561+0000","last_peered":"2024-03-09T08:52:19.240561+0000","last_clean":"2024-03-09T08:52:19.240561+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T08:52:19.240561+0000","last_undegraded":"2024-03-09T08:52:19.240561+0000","last_fullsized":"2024-03-09T08:52:19.240561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:46:24.111224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203373+0000","last_change":"2024-03-09T08:52:18.241038+0000","last_active":"2024-03-09T08:52:19.203373+0000","last_peered":"2024-03-09T08:52:19.203373+0000","last_clean":"2024-03-09T08:52:19.203373+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T08:52:19.203373+0000","last_undegraded":"2024-03-09T08:52:19.203373+0000","last_fullsized":"2024-03-09T08:52:19.203373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:02.169113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344169+0000","last_change":"2024-03-09T08:52:18.239951+0000","last_active":"2024-03-09T08:52:20.344169+0000","last_peered":"2024-03-09T08:52:20.344169+0000","last_clean":"2024-03-09T08:52:20.344169+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T08:52:20.344169+0000","last_undegraded":"2024-03-09T08:52:20.344169+0000","last_fullsized":"2024-03-09T08:52:20.344169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:51.102540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344029+0000","last_change":"2024-03-09T08:52:18.242257+0000","last_active":"2024-03-09T08:52:20.344029+0000","last_peered":"2024-03-09T08:52:20.344029+0000","last_clean":"2024-03-09T08:52:20.344029+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T08:52:20.344029+0000","last_undegraded":"2024-03-09T08:52:20.344029+0000","last_fullsized":"2024-03-09T08:52:20.344029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:40:00.854066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171019+0000","last_change":"2024-03-09T08:52:18.241628+0000","last_active":"2024-03-09T08:52:20.171019+0000","last_peered":"2024-03-09T08:52:20.171019+0000","last_clean":"2024-03-09T08:52:20.171019+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T08:52:20.171019+0000","last_undegraded":"2024-03-09T08:52:20.171019+0000","last_fullsized":"2024-03-09T08:52:20.171019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:07:58.570641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203397+0000","last_change":"2024-03-09T08:52:18.226980+0000","last_active":"2024-03-09T08:52:19.203397+0000","last_peered":"2024-03-09T08:52:19.203397+0000","last_clean":"2024-03-09T08:52:19.203397+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T08:52:19.203397+0000","last_undegraded":"2024-03-09T08:52:19.203397+0000","last_fullsized":"2024-03-09T08:52:19.203397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:28.312576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240525+0000","last_change":"2024-03-09T08:52:18.237869+0000","last_active":"2024-03-09T08:52:19.240525+0000","last_peered":"2024-03-09T08:52:19.240525+0000","last_clean":"2024-03-09T08:52:19.240525+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T08:52:19.240525+0000","last_undegraded":"2024-03-09T08:52:19.240525+0000","last_fullsized":"2024-03-09T08:52:19.240525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:01.571866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.206020+0000","last_change":"2024-03-09T08:52:18.229533+0000","last_active":"2024-03-09T08:52:19.206020+0000","last_peered":"2024-03-09T08:52:19.206020+0000","last_clean":"2024-03-09T08:52:19.206020+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T08:52:19.206020+0000","last_undegraded":"2024-03-09T08:52:19.206020+0000","last_fullsized":"2024-03-09T08:52:19.206020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:17.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342947+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T08:52:20.342947+0000","last_peered":"2024-03-09T08:52:20.342947+0000","last_clean":"2024-03-09T08:52:20.342947+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T08:52:20.342947+0000","last_undegraded":"2024-03-09T08:52:20.342947+0000","last_fullsized":"2024-03-09T08:52:20.342947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344401+0000","last_change":"2024-03-09T08:52:18.231141+0000","last_active":"2024-03-09T08:52:20.344401+0000","last_peered":"2024-03-09T08:52:20.344401+0000","last_clean":"2024-03-09T08:52:20.344401+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T08:52:20.344401+0000","last_undegraded":"2024-03-09T08:52:20.344401+0000","last_fullsized":"2024-03-09T08:52:20.344401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:25:00.471222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.239683+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T08:52:19.239683+0000","last_peered":"2024-03-09T08:52:19.239683+0000","last_clean":"2024-03-09T08:52:19.239683+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T08:52:19.239683+0000","last_undegraded":"2024-03-09T08:52:19.239683+0000","last_fullsized":"2024-03-09T08:52:19.239683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343053+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T08:52:20.343053+0000","last_peered":"2024-03-09T08:52:20.343053+0000","last_clean":"2024-03-09T08:52:20.343053+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T08:52:20.343053+0000","last_undegraded":"2024-03-09T08:52:20.343053+0000","last_fullsized":"2024-03-09T08:52:20.343053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343110+0000","last_change":"2024-03-09T08:52:18.234400+0000","last_active":"2024-03-09T08:52:20.343110+0000","last_peered":"2024-03-09T08:52:20.343110+0000","last_clean":"2024-03-09T08:52:20.343110+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T08:52:20.343110+0000","last_undegraded":"2024-03-09T08:52:20.343110+0000","last_fullsized":"2024-03-09T08:52:20.343110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:42:14.057367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170801+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T08:52:20.170801+0000","last_peered":"2024-03-09T08:52:20.170801+0000","last_clean":"2024-03-09T08:52:20.170801+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T08:52:20.170801+0000","last_undegraded":"2024-03-09T08:52:20.170801+0000","last_fullsized":"2024-03-09T08:52:20.170801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202195+0000","last_change":"2024-03-09T08:52:18.241156+0000","last_active":"2024-03-09T08:52:19.202195+0000","last_peered":"2024-03-09T08:52:19.202195+0000","last_clean":"2024-03-09T08:52:19.202195+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T08:52:19.202195+0000","last_undegraded":"2024-03-09T08:52:19.202195+0000","last_fullsized":"2024-03-09T08:52:19.202195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:55.323770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240316+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T08:52:19.240316+0000","last_peered":"2024-03-09T08:52:19.240316+0000","last_clean":"2024-03-09T08:52:19.240316+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T08:52:19.240316+0000","last_undegraded":"2024-03-09T08:52:19.240316+0000","last_fullsized":"2024-03-09T08:52:19.240316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:31.731626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171969+0000","last_change":"2024-03-09T08:52:18.251648+0000","last_active":"2024-03-09T08:52:20.171969+0000","last_peered":"2024-03-09T08:52:20.171969+0000","last_clean":"2024-03-09T08:52:20.171969+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T08:52:20.171969+0000","last_undegraded":"2024-03-09T08:52:20.171969+0000","last_fullsized":"2024-03-09T08:52:20.171969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:54:45.721673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171655+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T08:52:20.171655+0000","last_peered":"2024-03-09T08:52:20.171655+0000","last_clean":"2024-03-09T08:52:20.171655+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T08:52:20.171655+0000","last_undegraded":"2024-03-09T08:52:20.171655+0000","last_fullsized":"2024-03-09T08:52:20.171655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.345211+0000","last_change":"2024-03-09T08:52:18.236628+0000","last_active":"2024-03-09T08:52:20.345211+0000","last_peered":"2024-03-09T08:52:20.345211+0000","last_clean":"2024-03-09T08:52:20.345211+0000","last_became_active":"2024-03-09T08:52:18.235321+0000","last_became_peered":"2024-03-09T08:52:18.235321+0000","last_unstale":"2024-03-09T08:52:20.345211+0000","last_undegraded":"2024-03-09T08:52:20.345211+0000","last_fullsized":"2024-03-09T08:52:20.345211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:13:59.221325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203456+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T08:52:19.203456+0000","last_peered":"2024-03-09T08:52:19.203456+0000","last_clean":"2024-03-09T08:52:19.203456+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T08:52:19.203456+0000","last_undegraded":"2024-03-09T08:52:19.203456+0000","last_fullsized":"2024-03-09T08:52:19.203456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:41:10.578669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.207489+0000","last_change":"2024-03-09T08:52:18.244654+0000","last_active":"2024-03-09T08:52:19.207489+0000","last_peered":"2024-03-09T08:52:19.207489+0000","last_clean":"2024-03-09T08:52:19.207489+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T08:52:19.207489+0000","last_undegraded":"2024-03-09T08:52:19.207489+0000","last_fullsized":"2024-03-09T08:52:19.207489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:08:58.611725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240131+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T08:52:19.240131+0000","last_peered":"2024-03-09T08:52:19.240131+0000","last_clean":"2024-03-09T08:52:19.240131+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T08:52:19.240131+0000","last_undegraded":"2024-03-09T08:52:19.240131+0000","last_fullsized":"2024-03-09T08:52:19.240131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:01.724043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946832+0000","last_change":"2024-03-09T08:52:18.242496+0000","last_active":"2024-03-09T08:52:19.946832+0000","last_peered":"2024-03-09T08:52:19.946832+0000","last_clean":"2024-03-09T08:52:19.946832+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T08:52:19.946832+0000","last_undegraded":"2024-03-09T08:52:19.946832+0000","last_fullsized":"2024-03-09T08:52:19.946832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:35:17.142983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344205+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T08:52:20.344205+0000","last_peered":"2024-03-09T08:52:20.344205+0000","last_clean":"2024-03-09T08:52:20.344205+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T08:52:20.344205+0000","last_undegraded":"2024-03-09T08:52:20.344205+0000","last_fullsized":"2024-03-09T08:52:20.344205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:22:05.504557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342999+0000","last_change":"2024-03-09T08:52:18.236146+0000","last_active":"2024-03-09T08:52:20.342999+0000","last_peered":"2024-03-09T08:52:20.342999+0000","last_clean":"2024-03-09T08:52:20.342999+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T08:52:20.342999+0000","last_undegraded":"2024-03-09T08:52:20.342999+0000","last_fullsized":"2024-03-09T08:52:20.342999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:18:51.511926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171050+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T08:52:20.171050+0000","last_peered":"2024-03-09T08:52:20.171050+0000","last_clean":"2024-03-09T08:52:20.171050+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T08:52:20.171050+0000","last_undegraded":"2024-03-09T08:52:20.171050+0000","last_fullsized":"2024-03-09T08:52:20.171050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946410+0000","last_change":"2024-03-09T08:52:18.248093+0000","last_active":"2024-03-09T08:52:19.946410+0000","last_peered":"2024-03-09T08:52:19.946410+0000","last_clean":"2024-03-09T08:52:19.946410+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T08:52:19.946410+0000","last_undegraded":"2024-03-09T08:52:19.946410+0000","last_fullsized":"2024-03-09T08:52:19.946410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:29.414335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.239611+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T08:52:19.239611+0000","last_peered":"2024-03-09T08:52:19.239611+0000","last_clean":"2024-03-09T08:52:19.239611+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T08:52:19.239611+0000","last_undegraded":"2024-03-09T08:52:19.239611+0000","last_fullsized":"2024-03-09T08:52:19.239611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:07:29.577461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.239847+0000","last_change":"2024-03-09T08:52:18.245073+0000","last_active":"2024-03-09T08:52:19.239847+0000","last_peered":"2024-03-09T08:52:19.239847+0000","last_clean":"2024-03-09T08:52:19.239847+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T08:52:19.239847+0000","last_undegraded":"2024-03-09T08:52:19.239847+0000","last_fullsized":"2024-03-09T08:52:19.239847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:40:15.390735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202948+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T08:52:19.202948+0000","last_peered":"2024-03-09T08:52:19.202948+0000","last_clean":"2024-03-09T08:52:19.202948+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T08:52:19.202948+0000","last_undegraded":"2024-03-09T08:52:19.202948+0000","last_fullsized":"2024-03-09T08:52:19.202948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:31.859924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946479+0000","last_change":"2024-03-09T08:52:18.248755+0000","last_active":"2024-03-09T08:52:19.946479+0000","last_peered":"2024-03-09T08:52:19.946479+0000","last_clean":"2024-03-09T08:52:19.946479+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T08:52:19.946479+0000","last_undegraded":"2024-03-09T08:52:19.946479+0000","last_fullsized":"2024-03-09T08:52:19.946479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:37.083738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.207109+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T08:52:19.207109+0000","last_peered":"2024-03-09T08:52:19.207109+0000","last_clean":"2024-03-09T08:52:19.207109+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T08:52:19.207109+0000","last_undegraded":"2024-03-09T08:52:19.207109+0000","last_fullsized":"2024-03-09T08:52:19.207109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:10:34.698235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946939+0000","last_change":"2024-03-09T08:52:18.246145+0000","last_active":"2024-03-09T08:52:19.946939+0000","last_peered":"2024-03-09T08:52:19.946939+0000","last_clean":"2024-03-09T08:52:19.946939+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T08:52:19.946939+0000","last_undegraded":"2024-03-09T08:52:19.946939+0000","last_fullsized":"2024-03-09T08:52:19.946939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:29:36.213774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203616+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T08:52:19.203616+0000","last_peered":"2024-03-09T08:52:19.203616+0000","last_clean":"2024-03-09T08:52:19.203616+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T08:52:19.203616+0000","last_undegraded":"2024-03-09T08:52:19.203616+0000","last_fullsized":"2024-03-09T08:52:19.203616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:16.138216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.947778+0000","last_change":"2024-03-09T08:52:18.250420+0000","last_active":"2024-03-09T08:52:19.947778+0000","last_peered":"2024-03-09T08:52:19.947778+0000","last_clean":"2024-03-09T08:52:19.947778+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T08:52:19.947778+0000","last_undegraded":"2024-03-09T08:52:19.947778+0000","last_fullsized":"2024-03-09T08:52:19.947778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:48.826066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203615+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T08:52:19.203615+0000","last_peered":"2024-03-09T08:52:19.203615+0000","last_clean":"2024-03-09T08:52:19.203615+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T08:52:19.203615+0000","last_undegraded":"2024-03-09T08:52:19.203615+0000","last_fullsized":"2024-03-09T08:52:19.203615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:26:35.574903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171894+0000","last_change":"2024-03-09T08:52:18.250433+0000","last_active":"2024-03-09T08:52:20.171894+0000","last_peered":"2024-03-09T08:52:20.171894+0000","last_clean":"2024-03-09T08:52:20.171894+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T08:52:20.171894+0000","last_undegraded":"2024-03-09T08:52:20.171894+0000","last_fullsized":"2024-03-09T08:52:20.171894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:29.742337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203270+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T08:52:19.203270+0000","last_peered":"2024-03-09T08:52:19.203270+0000","last_clean":"2024-03-09T08:52:19.203270+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T08:52:19.203270+0000","last_undegraded":"2024-03-09T08:52:19.203270+0000","last_fullsized":"2024-03-09T08:52:19.203270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:27.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946885+0000","last_change":"2024-03-09T08:52:18.242671+0000","last_active":"2024-03-09T08:52:19.946885+0000","last_peered":"2024-03-09T08:52:19.946885+0000","last_clean":"2024-03-09T08:52:19.946885+0000","last_became_active":"2024-03-09T08:52:18.242258+0000","last_became_peered":"2024-03-09T08:52:18.242258+0000","last_unstale":"2024-03-09T08:52:19.946885+0000","last_undegraded":"2024-03-09T08:52:19.946885+0000","last_fullsized":"2024-03-09T08:52:19.946885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:15.030669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946012+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T08:52:19.946012+0000","last_peered":"2024-03-09T08:52:19.946012+0000","last_clean":"2024-03-09T08:52:19.946012+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T08:52:19.946012+0000","last_undegraded":"2024-03-09T08:52:19.946012+0000","last_fullsized":"2024-03-09T08:52:19.946012+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.170651+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T08:52:20.170651+0000","last_peered":"2024-03-09T08:52:20.170651+0000","last_clean":"2024-03-09T08:52:20.170651+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T08:52:20.170651+0000","last_undegraded":"2024-03-09T08:52:20.170651+0000","last_fullsized":"2024-03-09T08:52:20.170651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342508+0000","last_change":"2024-03-09T08:52:18.235367+0000","last_active":"2024-03-09T08:52:20.342508+0000","last_peered":"2024-03-09T08:52:20.342508+0000","last_clean":"2024-03-09T08:52:20.342508+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T08:52:20.342508+0000","last_undegraded":"2024-03-09T08:52:20.342508+0000","last_fullsized":"2024-03-09T08:52:20.342508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:40.736440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.945746+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T08:52:19.945746+0000","last_peered":"2024-03-09T08:52:19.945746+0000","last_clean":"2024-03-09T08:52:19.945746+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T08:52:19.945746+0000","last_undegraded":"2024-03-09T08:52:19.945746+0000","last_fullsized":"2024-03-09T08:52:19.945746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.341951+0000","last_change":"2024-03-09T08:52:18.214396+0000","last_active":"2024-03-09T08:52:20.341951+0000","last_peered":"2024-03-09T08:52:20.341951+0000","last_clean":"2024-03-09T08:52:20.341951+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T08:52:20.341951+0000","last_undegraded":"2024-03-09T08:52:20.341951+0000","last_fullsized":"2024-03-09T08:52:20.341951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:24:10.719351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203472+0000","last_change":"2024-03-09T08:52:17.271847+0000","last_active":"2024-03-09T08:52:19.203472+0000","last_peered":"2024-03-09T08:52:19.203472+0000","last_clean":"2024-03-09T08:52:19.203472+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T08:52:19.203472+0000","last_undegraded":"2024-03-09T08:52:19.203472+0000","last_fullsized":"2024-03-09T08:52:19.203472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:27.418674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202590+0000","last_change":"2024-03-09T08:52:18.235832+0000","last_active":"2024-03-09T08:52:19.202590+0000","last_peered":"2024-03-09T08:52:19.202590+0000","last_clean":"2024-03-09T08:52:19.202590+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T08:52:19.202590+0000","last_undegraded":"2024-03-09T08:52:19.202590+0000","last_fullsized":"2024-03-09T08:52:19.202590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:38:58.751526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171135+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T08:52:20.171135+0000","last_peered":"2024-03-09T08:52:20.171135+0000","last_clean":"2024-03-09T08:52:20.171135+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T08:52:20.171135+0000","last_undegraded":"2024-03-09T08:52:20.171135+0000","last_fullsized":"2024-03-09T08:52:20.171135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.206422+0000","last_change":"2024-03-09T08:52:18.227365+0000","last_active":"2024-03-09T08:52:19.206422+0000","last_peered":"2024-03-09T08:52:19.206422+0000","last_clean":"2024-03-09T08:52:19.206422+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T08:52:19.206422+0000","last_undegraded":"2024-03-09T08:52:19.206422+0000","last_fullsized":"2024-03-09T08:52:19.206422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:26.357063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203208+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T08:52:19.203208+0000","last_peered":"2024-03-09T08:52:19.203208+0000","last_clean":"2024-03-09T08:52:19.203208+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T08:52:19.203208+0000","last_undegraded":"2024-03-09T08:52:19.203208+0000","last_fullsized":"2024-03-09T08:52:19.203208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.172365+0000","last_change":"2024-03-09T08:52:18.250999+0000","last_active":"2024-03-09T08:52:20.172365+0000","last_peered":"2024-03-09T08:52:20.172365+0000","last_clean":"2024-03-09T08:52:20.172365+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T08:52:20.172365+0000","last_undegraded":"2024-03-09T08:52:20.172365+0000","last_fullsized":"2024-03-09T08:52:20.172365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:49:54.496603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.947173+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T08:52:19.947173+0000","last_peered":"2024-03-09T08:52:19.947173+0000","last_clean":"2024-03-09T08:52:19.947173+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T08:52:19.947173+0000","last_undegraded":"2024-03-09T08:52:19.947173+0000","last_fullsized":"2024-03-09T08:52:19.947173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202531+0000","last_change":"2024-03-09T08:52:18.234887+0000","last_active":"2024-03-09T08:52:19.202531+0000","last_peered":"2024-03-09T08:52:19.202531+0000","last_clean":"2024-03-09T08:52:19.202531+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T08:52:19.202531+0000","last_undegraded":"2024-03-09T08:52:19.202531+0000","last_fullsized":"2024-03-09T08:52:19.202531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:44.661882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.203765+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T08:52:19.203765+0000","last_peered":"2024-03-09T08:52:19.203765+0000","last_clean":"2024-03-09T08:52:19.203765+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T08:52:19.203765+0000","last_undegraded":"2024-03-09T08:52:19.203765+0000","last_fullsized":"2024-03-09T08:52:19.203765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:30.201471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344099+0000","last_change":"2024-03-09T08:52:18.245442+0000","last_active":"2024-03-09T08:52:20.344099+0000","last_peered":"2024-03-09T08:52:20.344099+0000","last_clean":"2024-03-09T08:52:20.344099+0000","last_became_active":"2024-03-09T08:52:18.245164+0000","last_became_peered":"2024-03-09T08:52:18.245164+0000","last_unstale":"2024-03-09T08:52:20.344099+0000","last_undegraded":"2024-03-09T08:52:20.344099+0000","last_fullsized":"2024-03-09T08:52:20.344099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:31:08.413041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.240686+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T08:52:19.240686+0000","last_peered":"2024-03-09T08:52:19.240686+0000","last_clean":"2024-03-09T08:52:19.240686+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T08:52:19.240686+0000","last_undegraded":"2024-03-09T08:52:19.240686+0000","last_fullsized":"2024-03-09T08:52:19.240686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:35:38.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344838+0000","last_change":"2024-03-09T08:52:18.229739+0000","last_active":"2024-03-09T08:52:20.344838+0000","last_peered":"2024-03-09T08:52:20.344838+0000","last_clean":"2024-03-09T08:52:20.344838+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T08:52:20.344838+0000","last_undegraded":"2024-03-09T08:52:20.344838+0000","last_fullsized":"2024-03-09T08:52:20.344838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:24.394779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.945740+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T08:52:19.945740+0000","last_peered":"2024-03-09T08:52:19.945740+0000","last_clean":"2024-03-09T08:52:19.945740+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T08:52:19.945740+0000","last_undegraded":"2024-03-09T08:52:19.945740+0000","last_fullsized":"2024-03-09T08:52:19.945740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.206203+0000","last_change":"2024-03-09T08:52:18.217046+0000","last_active":"2024-03-09T08:52:19.206203+0000","last_peered":"2024-03-09T08:52:19.206203+0000","last_clean":"2024-03-09T08:52:19.206203+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T08:52:19.206203+0000","last_undegraded":"2024-03-09T08:52:19.206203+0000","last_fullsized":"2024-03-09T08:52:19.206203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:14.884963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202279+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T08:52:19.202279+0000","last_peered":"2024-03-09T08:52:19.202279+0000","last_clean":"2024-03-09T08:52:19.202279+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T08:52:19.202279+0000","last_undegraded":"2024-03-09T08:52:19.202279+0000","last_fullsized":"2024-03-09T08:52:19.202279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:21:59.990396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342910+0000","last_change":"2024-03-09T08:52:18.233208+0000","last_active":"2024-03-09T08:52:20.342910+0000","last_peered":"2024-03-09T08:52:20.342910+0000","last_clean":"2024-03-09T08:52:20.342910+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T08:52:20.342910+0000","last_undegraded":"2024-03-09T08:52:20.342910+0000","last_fullsized":"2024-03-09T08:52:20.342910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:17.589783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.202515+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T08:52:19.202515+0000","last_peered":"2024-03-09T08:52:19.202515+0000","last_clean":"2024-03-09T08:52:19.202515+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T08:52:19.202515+0000","last_undegraded":"2024-03-09T08:52:19.202515+0000","last_fullsized":"2024-03-09T08:52:19.202515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:45:45.612953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.342792+0000","last_change":"2024-03-09T08:52:18.244306+0000","last_active":"2024-03-09T08:52:20.342792+0000","last_peered":"2024-03-09T08:52:20.342792+0000","last_clean":"2024-03-09T08:52:20.342792+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T08:52:20.342792+0000","last_undegraded":"2024-03-09T08:52:20.342792+0000","last_fullsized":"2024-03-09T08:52:20.342792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:58.365524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946557+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T08:52:19.946557+0000","last_peered":"2024-03-09T08:52:19.946557+0000","last_clean":"2024-03-09T08:52:19.946557+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T08:52:19.946557+0000","last_undegraded":"2024-03-09T08:52:19.946557+0000","last_fullsized":"2024-03-09T08:52:19.946557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.239920+0000","last_change":"2024-03-09T08:52:18.234372+0000","last_active":"2024-03-09T08:52:19.239920+0000","last_peered":"2024-03-09T08:52:19.239920+0000","last_clean":"2024-03-09T08:52:19.239920+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T08:52:19.239920+0000","last_undegraded":"2024-03-09T08:52:19.239920+0000","last_fullsized":"2024-03-09T08:52:19.239920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:09:33.422907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.172082+0000","last_change":"2024-03-09T08:52:18.228419+0000","last_active":"2024-03-09T08:52:20.172082+0000","last_peered":"2024-03-09T08:52:20.172082+0000","last_clean":"2024-03-09T08:52:20.172082+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T08:52:20.172082+0000","last_undegraded":"2024-03-09T08:52:20.172082+0000","last_fullsized":"2024-03-09T08:52:20.172082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:19:12.533402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946320+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T08:52:19.946320+0000","last_peered":"2024-03-09T08:52:19.946320+0000","last_clean":"2024-03-09T08:52:19.946320+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T08:52:19.946320+0000","last_undegraded":"2024-03-09T08:52:19.946320+0000","last_fullsized":"2024-03-09T08:52:19.946320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.171447+0000","last_change":"2024-03-09T08:52:18.230629+0000","last_active":"2024-03-09T08:52:20.171447+0000","last_peered":"2024-03-09T08:52:20.171447+0000","last_clean":"2024-03-09T08:52:20.171447+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T08:52:20.171447+0000","last_undegraded":"2024-03-09T08:52:20.171447+0000","last_fullsized":"2024-03-09T08:52:20.171447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:46:38.528839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:19.946964+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T08:52:19.946964+0000","last_peered":"2024-03-09T08:52:19.946964+0000","last_clean":"2024-03-09T08:52:19.946964+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T08:52:19.946964+0000","last_undegraded":"2024-03-09T08:52:19.946964+0000","last_fullsized":"2024-03-09T08:52:19.946964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.344989+0000","last_change":"2024-03-09T08:52:18.236946+0000","last_active":"2024-03-09T08:52:20.344989+0000","last_peered":"2024-03-09T08:52:20.344989+0000","last_clean":"2024-03-09T08:52:20.344989+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T08:52:20.344989+0000","last_undegraded":"2024-03-09T08:52:20.344989+0000","last_fullsized":"2024-03-09T08:52:20.344989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:45.372160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343766+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T08:52:20.343766+0000","last_peered":"2024-03-09T08:52:20.343766+0000","last_clean":"2024-03-09T08:52:20.343766+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T08:52:20.343766+0000","last_undegraded":"2024-03-09T08:52:20.343766+0000","last_fullsized":"2024-03-09T08:52:20.343766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.343927+0000","last_change":"2024-03-09T08:52:18.235099+0000","last_active":"2024-03-09T08:52:20.343927+0000","last_peered":"2024-03-09T08:52:20.343927+0000","last_clean":"2024-03-09T08:52:20.343927+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T08:52:20.343927+0000","last_undegraded":"2024-03-09T08:52:20.343927+0000","last_fullsized":"2024-03-09T08:52:20.343927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:39:28.882461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-09T08:52:20.345109+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T08:52:20.345109+0000","last_peered":"2024-03-09T08:52:20.345109+0000","last_clean":"2024-03-09T08:52:20.345109+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T08:52:20.345109+0000","last_undegraded":"2024-03-09T08:52:20.345109+0000","last_fullsized":"2024-03-09T08:52:20.345109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:41:40.332687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T08:52:24.441 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-09T08:52:24.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.709+0000 7fe478000640 1 Processor -- start 2024-03-09T08:52:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.709+0000 7fe478000640 1 -- start start 2024-03-09T08:52:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.710+0000 7fe478000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe470151290 0x7fe4700c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.710+0000 7fe478000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe470073930 0x7fe4700a8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.710+0000 7fe478000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4701574e0 con 0x7fe4700a9480 2024-03-09T08:52:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.710+0000 7fe478000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47006fa20 con 0x7fe470073930 2024-03-09T08:52:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.710+0000 7fe478000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4701577c0 con 0x7fe470151290 2024-03-09T08:52:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.711+0000 7fe474c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe470073930 0x7fe4700a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.711+0000 7fe474c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe470073930 0x7fe4700a8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59788/0 (socket says 172.21.15.67:59788) 2024-03-09T08:52:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.711+0000 7fe474c00640 1 -- 172.21.15.67:0/1533242778 learned_addr learned my addr 172.21.15.67:0/1533242778 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.711+0000 7fe475600640 1 --2- 172.21.15.67:0/1533242778 >> v2:172.21.15.134:3300/0 conn(0x7fe470151290 0x7fe4700c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.711+0000 7fe474c00640 1 -- 172.21.15.67:0/1533242778 >> v2:172.21.15.134:3300/0 conn(0x7fe470151290 msgr2=0x7fe4700c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.711+0000 7fe474c00640 1 --2- 172.21.15.67:0/1533242778 >> v2:172.21.15.134:3300/0 conn(0x7fe470151290 0x7fe4700c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.711+0000 7fe474c00640 1 -- 172.21.15.67:0/1533242778 >> v1:172.21.15.134:6789/0 conn(0x7fe4700a9480 legacy=0x7fe470153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.711+0000 7fe474c00640 1 -- 172.21.15.67:0/1533242778 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe470157aa0 con 0x7fe470073930 2024-03-09T08:52:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.712+0000 7fe474c00640 1 --2- 172.21.15.67:0/1533242778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe470073930 0x7fe4700a8f40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe470071380 tx=0x7fe45c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa503e8beb378b5d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.713+0000 7fe46fe00640 1 -- 172.21.15.67:0/1533242778 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c02e070 con 0x7fe470073930 2024-03-09T08:52:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.713+0000 7fe46fe00640 1 -- 172.21.15.67:0/1533242778 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c037070 con 0x7fe470073930 2024-03-09T08:52:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.713+0000 7fe46fe00640 1 -- 172.21.15.67:0/1533242778 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c031a00 con 0x7fe470073930 2024-03-09T08:52:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.713+0000 7fe478000640 1 -- 172.21.15.67:0/1533242778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe470073930 msgr2=0x7fe4700a8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.713+0000 7fe478000640 1 --2- 172.21.15.67:0/1533242778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe470073930 0x7fe4700a8f40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe470071380 tx=0x7fe45c02b880 comp rx=0 tx=0).stop 2024-03-09T08:52:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.713+0000 7fe478000640 1 -- 172.21.15.67:0/1533242778 shutdown_connections 2024-03-09T08:52:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.713+0000 7fe478000640 1 --2- 172.21.15.67:0/1533242778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe470073930 0x7fe4700a8f40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.713+0000 7fe478000640 1 --2- 172.21.15.67:0/1533242778 >> v2:172.21.15.134:3300/0 conn(0x7fe470151290 0x7fe4700c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.713+0000 7fe478000640 1 -- 172.21.15.67:0/1533242778 >> 172.21.15.67:0/1533242778 conn(0x7fe4700c9710 msgr2=0x7fe4700c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.713+0000 7fe478000640 1 -- 172.21.15.67:0/1533242778 shutdown_connections 2024-03-09T08:52:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.713+0000 7fe478000640 1 -- 172.21.15.67:0/1533242778 wait complete. 2024-03-09T08:52:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.715+0000 7fe478000640 1 Processor -- start 2024-03-09T08:52:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.715+0000 7fe478000640 1 -- start start 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.716+0000 7fe478000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe470073930 0x7fe47006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.716+0000 7fe478000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4700a9480 0x7fe47014ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.716+0000 7fe478000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe470151290 0x7fe47014a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.716+0000 7fe478000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe470157240 con 0x7fe4700a9480 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.716+0000 7fe478000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4701570c0 con 0x7fe470073930 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.716+0000 7fe478000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4701546b0 con 0x7fe470151290 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.716+0000 7fe474c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4700a9480 0x7fe47014ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.716+0000 7fe474c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4700a9480 0x7fe47014ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59790/0 (socket says 172.21.15.67:59790) 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.716+0000 7fe474c00640 1 -- 172.21.15.67:0/4213876723 learned_addr learned my addr 172.21.15.67:0/4213876723 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe475600640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe470073930 0x7fe47006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe476000640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe470151290 0x7fe47014a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe474c00640 1 -- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe470151290 msgr2=0x7fe47014a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe474c00640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe470151290 0x7fe47014a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe474c00640 1 -- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe470073930 msgr2=0x7fe47006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe474c00640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe470073930 0x7fe47006d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe474c00640 1 -- 172.21.15.67:0/4213876723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47006e070 con 0x7fe4700a9480 2024-03-09T08:52:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe476000640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe470151290 0x7fe47014a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe474c00640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4700a9480 0x7fe47014ca70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe45c000c00 tx=0x7fe45c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe46d600640 1 -- 172.21.15.67:0/4213876723 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c032880 con 0x7fe4700a9480 2024-03-09T08:52:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe46d600640 1 -- 172.21.15.67:0/4213876723 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c032a20 con 0x7fe4700a9480 2024-03-09T08:52:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe478000640 1 -- 172.21.15.67:0/4213876723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe470147c40 con 0x7fe4700a9480 2024-03-09T08:52:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe46d600640 1 -- 172.21.15.67:0/4213876723 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c032e10 con 0x7fe4700a9480 2024-03-09T08:52:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.717+0000 7fe478000640 1 -- 172.21.15.67:0/4213876723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47014cfb0 con 0x7fe4700a9480 2024-03-09T08:52:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.718+0000 7fe478000640 1 -- 172.21.15.67:0/4213876723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4700750f0 con 0x7fe4700a9480 2024-03-09T08:52:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.719+0000 7fe46d600640 1 -- 172.21.15.67:0/4213876723 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe45c04c060 con 0x7fe4700a9480 2024-03-09T08:52:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.721+0000 7fe46d600640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe45006b500 0x7fe45006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.721+0000 7fe46d600640 1 -- 172.21.15.67:0/4213876723 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7fe45c0ab9e0 con 0x7fe4700a9480 2024-03-09T08:52:24.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.721+0000 7fe475600640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe45006b500 0x7fe45006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:24.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.723+0000 7fe475600640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe45006b500 0x7fe45006d9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe460009a30 tx=0x7fe460008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:24.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.725+0000 7fe46d600640 1 -- 172.21.15.67:0/4213876723 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe45c040070 con 0x7fe4700a9480 2024-03-09T08:52:24.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.925+0000 7fe478000640 1 -- 172.21.15.67:0/4213876723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe470075a00 con 0x7fe4700a9480 2024-03-09T08:52:24.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.928+0000 7fe46d600640 1 -- 172.21.15.67:0/4213876723 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16549 (secure 0 0 0) 0x7fe45c07a860 con 0x7fe4700a9480 2024-03-09T08:52:24.924 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:24.924 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":20,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:19.196234+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:24.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 -- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe45006b500 msgr2=0x7fe45006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe45006b500 0x7fe45006d9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe460009a30 tx=0x7fe460008040 comp rx=0 tx=0).stop 2024-03-09T08:52:24.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 -- 172.21.15.67:0/4213876723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4700a9480 msgr2=0x7fe47014ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:24.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4700a9480 0x7fe47014ca70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe45c000c00 tx=0x7fe45c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 -- 172.21.15.67:0/4213876723 shutdown_connections 2024-03-09T08:52:24.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe45006b500 0x7fe45006d9c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe470151290 0x7fe47014a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4700a9480 0x7fe47014ca70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 --2- 172.21.15.67:0/4213876723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe470073930 0x7fe47006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 -- 172.21.15.67:0/4213876723 >> 172.21.15.67:0/4213876723 conn(0x7fe4700c9710 msgr2=0x7fe470074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 -- 172.21.15.67:0/4213876723 shutdown_connections 2024-03-09T08:52:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:24.931+0000 7fe478000640 1 -- 172.21.15.67:0/4213876723 wait complete. 2024-03-09T08:52:24.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-09T08:52:06.749641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-03-09T08:52:16.185356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-03-09T08:52:16.756829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-09T08:52:24.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T08:52:25.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.169+0000 7f6ff0a00640 1 Processor -- start 2024-03-09T08:52:25.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.170+0000 7f6ff0a00640 1 -- start start 2024-03-09T08:52:25.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.171+0000 7f6ff0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fec0bd4d0 0x7f6fec15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.171+0000 7f6ff0a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fec0c4500 0x7f6fec1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.171+0000 7f6ff0a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec06f4c0 con 0x7f6fec06b2b0 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.171+0000 7f6ff0a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fec06f7a0 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.171+0000 7f6ff0a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec15a320 con 0x7f6fec0c4500 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.172+0000 7f6fe9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fec0bd4d0 0x7f6fec15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.172+0000 7f6fe9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6fec0bd4d0 0x7f6fec15ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59800/0 (socket says 172.21.15.67:59800) 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.172+0000 7f6fe9e00640 1 -- 172.21.15.67:0/1186449993 learned_addr learned my addr 172.21.15.67:0/1186449993 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.172+0000 7f6feb200640 1 --2- 172.21.15.67:0/1186449993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fec0c4500 0x7f6fec1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.172+0000 7f6fe9e00640 1 -- 172.21.15.67:0/1186449993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fec0c4500 msgr2=0x7f6fec1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.172+0000 7f6fe9e00640 1 --2- 172.21.15.67:0/1186449993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fec0c4500 0x7f6fec1572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.172+0000 7f6fe9e00640 1 -- 172.21.15.67:0/1186449993 >> v1:172.21.15.134:6789/0 conn(0x7f6fec06b2b0 legacy=0x7f6fec0c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.172+0000 7f6fe9e00640 1 -- 172.21.15.67:0/1186449993 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec15a600 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.173+0000 7f6fe9e00640 1 --2- 172.21.15.67:0/1186449993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fec0bd4d0 0x7f6fec15ae00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009e50 tx=0x7f6fd802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcf3a24fca9890a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:25.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.174+0000 7f6fe9400640 1 -- 172.21.15.67:0/1186449993 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd802ca70 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.174+0000 7f6fe9400640 1 -- 172.21.15.67:0/1186449993 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd802cc10 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.174+0000 7f6fe9400640 1 -- 172.21.15.67:0/1186449993 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80343f0 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.174+0000 7f6ff0a00640 1 -- 172.21.15.67:0/1186449993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fec0bd4d0 msgr2=0x7f6fec15ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:25.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.174+0000 7f6ff0a00640 1 --2- 172.21.15.67:0/1186449993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fec0bd4d0 0x7f6fec15ae00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009e50 tx=0x7f6fd802bed0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.174+0000 7f6ff0a00640 1 -- 172.21.15.67:0/1186449993 shutdown_connections 2024-03-09T08:52:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.174+0000 7f6ff0a00640 1 --2- 172.21.15.67:0/1186449993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fec0c4500 0x7f6fec1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.174+0000 7f6ff0a00640 1 --2- 172.21.15.67:0/1186449993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fec0bd4d0 0x7f6fec15ae00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.174+0000 7f6ff0a00640 1 -- 172.21.15.67:0/1186449993 >> 172.21.15.67:0/1186449993 conn(0x7f6fec0c82a0 msgr2=0x7f6fec0c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.174+0000 7f6ff0a00640 1 -- 172.21.15.67:0/1186449993 shutdown_connections 2024-03-09T08:52:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.174+0000 7f6ff0a00640 1 -- 172.21.15.67:0/1186449993 wait complete. 2024-03-09T08:52:25.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.176+0000 7f6ff0a00640 1 Processor -- start 2024-03-09T08:52:25.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.176+0000 7f6ff0a00640 1 -- start start 2024-03-09T08:52:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.177+0000 7f6ff0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fec06b2b0 0x7f6fec0c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.177+0000 7f6ff0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fec0bd4d0 0x7f6fec0ad150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.177+0000 7f6ff0a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fec0c4500 0x7f6fec0c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.177+0000 7f6ff0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec073b20 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.177+0000 7f6ff0a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0c33b0 con 0x7f6fec06b2b0 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.177+0000 7f6ff0a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec073ca0 con 0x7f6fec0c4500 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.177+0000 7f6fe9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fec0bd4d0 0x7f6fec0ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.177+0000 7f6fe9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fec0bd4d0 0x7f6fec0ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59806/0 (socket says 172.21.15.67:59806) 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.177+0000 7f6fe9e00640 1 -- 172.21.15.67:0/3544605171 learned_addr learned my addr 172.21.15.67:0/3544605171 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.178+0000 7f6fea800640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fec06b2b0 0x7f6fec0c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.178+0000 7f6feb200640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fec0c4500 0x7f6fec0c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.178+0000 7f6fe9e00640 1 -- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fec0c4500 msgr2=0x7f6fec0c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.178+0000 7f6fe9e00640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fec0c4500 0x7f6fec0c08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.178+0000 7f6fe9e00640 1 -- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fec06b2b0 msgr2=0x7f6fec0c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.178+0000 7f6fe9e00640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fec06b2b0 0x7f6fec0c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.178+0000 7f6fe9e00640 1 -- 172.21.15.67:0/3544605171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec1f09e0 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.178+0000 7f6fe9e00640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fec0bd4d0 0x7f6fec0ad150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8000f80 tx=0x7f6fd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:25.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.178+0000 7f6fe2a00640 1 -- 172.21.15.67:0/3544605171 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8043020 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.179+0000 7f6fe2a00640 1 -- 172.21.15.67:0/3544605171 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd803e450 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.179+0000 7f6fe2a00640 1 -- 172.21.15.67:0/3544605171 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8009510 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.179+0000 7f6ff0a00640 1 -- 172.21.15.67:0/3544605171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec1f0cc0 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.179+0000 7f6ff0a00640 1 -- 172.21.15.67:0/3544605171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec0ad690 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.180+0000 7f6ff0a00640 1 -- 172.21.15.67:0/3544605171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec0c2b20 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.181+0000 7f6fe2a00640 1 -- 172.21.15.67:0/3544605171 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6fd8005d10 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.182+0000 7f6fe2a00640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fcc06b500 0x7f6fcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.182+0000 7f6fe2a00640 1 -- 172.21.15.67:0/3544605171 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f6fd80ac860 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.183+0000 7f6fea800640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fcc06b500 0x7f6fcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.184+0000 7f6fea800640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fcc06b500 0x7f6fcc06d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc001640 tx=0x7f6fdc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:25.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.187+0000 7f6fe2a00640 1 -- 172.21.15.67:0/3544605171 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6fd804d070 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.393+0000 7f6ff0a00640 1 -- 172.21.15.67:0/3544605171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6fec04f3c0 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.394+0000 7f6fe2a00640 1 -- 172.21.15.67:0/3544605171 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6fd807b780 con 0x7f6fec0bd4d0 2024-03-09T08:52:25.389 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-09T08:52:25.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 -- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fcc06b500 msgr2=0x7f6fcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:25.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fcc06b500 0x7f6fcc06d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc001640 tx=0x7f6fdc009040 comp rx=0 tx=0).stop 2024-03-09T08:52:25.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 -- 172.21.15.67:0/3544605171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fec0bd4d0 msgr2=0x7f6fec0ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:25.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fec0bd4d0 0x7f6fec0ad150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8000f80 tx=0x7f6fd8004a40 comp rx=0 tx=0).stop 2024-03-09T08:52:25.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 -- 172.21.15.67:0/3544605171 shutdown_connections 2024-03-09T08:52:25.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fcc06b500 0x7f6fcc06d9c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6fec0c4500 0x7f6fec0c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fec0bd4d0 0x7f6fec0ad150 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 --2- 172.21.15.67:0/3544605171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fec06b2b0 0x7f6fec0c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 -- 172.21.15.67:0/3544605171 >> 172.21.15.67:0/3544605171 conn(0x7f6fec0c82a0 msgr2=0x7f6fec157a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:25.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 -- 172.21.15.67:0/3544605171 shutdown_connections 2024-03-09T08:52:25.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.397+0000 7f6ff0a00640 1 -- 172.21.15.67:0/3544605171 wait complete. 2024-03-09T08:52:25.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-03-09T08:52:25.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.646+0000 7f84f6000640 1 Processor -- start 2024-03-09T08:52:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.646+0000 7f84f6000640 1 -- start start 2024-03-09T08:52:25.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.647+0000 7f84f6000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f00c3010 0x7f84f00ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.647+0000 7f84f6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f84f0156980 0x7f84f014fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.647+0000 7f84f6000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f84f0150180 0x7f84f006d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.647+0000 7f84f6000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f0153950 con 0x7f84f0156980 2024-03-09T08:52:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.647+0000 7f84f6000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f01505f0 con 0x7f84f0150180 2024-03-09T08:52:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.647+0000 7f84f6000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f0152b90 con 0x7f84f00c3010 2024-03-09T08:52:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.648+0000 7f84efe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f84f0156980 0x7f84f014fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.648+0000 7f84efe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f84f0156980 0x7f84f014fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59818/0 (socket says 172.21.15.67:59818) 2024-03-09T08:52:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.648+0000 7f84efe00640 1 -- 172.21.15.67:0/2779655951 learned_addr learned my addr 172.21.15.67:0/2779655951 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.648+0000 7f84f4c00640 1 --2- 172.21.15.67:0/2779655951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f00c3010 0x7f84f00ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.648+0000 7f84f5600640 1 --2- 172.21.15.67:0/2779655951 >> v2:172.21.15.134:3300/0 conn(0x7f84f0150180 0x7f84f006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.648+0000 7f84efe00640 1 -- 172.21.15.67:0/2779655951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f00c3010 msgr2=0x7f84f00ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.648+0000 7f84efe00640 1 --2- 172.21.15.67:0/2779655951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f00c3010 0x7f84f00ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.648+0000 7f84efe00640 1 -- 172.21.15.67:0/2779655951 >> v2:172.21.15.134:3300/0 conn(0x7f84f0150180 msgr2=0x7f84f006d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.648+0000 7f84efe00640 1 --2- 172.21.15.67:0/2779655951 >> v2:172.21.15.134:3300/0 conn(0x7f84f0150180 0x7f84f006d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.648+0000 7f84efe00640 1 -- 172.21.15.67:0/2779655951 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f0152e70 con 0x7f84f0156980 2024-03-09T08:52:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.649+0000 7f84efe00640 1 --2- 172.21.15.67:0/2779655951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f0156980 0x7f84f014fc40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84f00c3960 tx=0x7f84dc02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da2ef5e31974f57 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.649+0000 7f84ef400640 1 -- 172.21.15.67:0/2779655951 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc038070 con 0x7f84f0156980 2024-03-09T08:52:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.649+0000 7f84ef400640 1 -- 172.21.15.67:0/2779655951 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc031740 con 0x7f84f0156980 2024-03-09T08:52:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.650+0000 7f84ef400640 1 -- 172.21.15.67:0/2779655951 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc02f050 con 0x7f84f0156980 2024-03-09T08:52:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.650+0000 7f84f6000640 1 -- 172.21.15.67:0/2779655951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f0156980 msgr2=0x7f84f014fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.650+0000 7f84f6000640 1 --2- 172.21.15.67:0/2779655951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f0156980 0x7f84f014fc40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84f00c3960 tx=0x7f84dc02b7e0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.650+0000 7f84f6000640 1 -- 172.21.15.67:0/2779655951 shutdown_connections 2024-03-09T08:52:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.650+0000 7f84f6000640 1 --2- 172.21.15.67:0/2779655951 >> v2:172.21.15.134:3300/0 conn(0x7f84f0150180 0x7f84f006d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.650+0000 7f84f6000640 1 --2- 172.21.15.67:0/2779655951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f0156980 0x7f84f014fc40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.650+0000 7f84f6000640 1 --2- 172.21.15.67:0/2779655951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f00c3010 0x7f84f00ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.650+0000 7f84f6000640 1 -- 172.21.15.67:0/2779655951 >> 172.21.15.67:0/2779655951 conn(0x7f84f006b8a0 msgr2=0x7f84f00c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.651+0000 7f84f6000640 1 -- 172.21.15.67:0/2779655951 shutdown_connections 2024-03-09T08:52:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.651+0000 7f84f6000640 1 -- 172.21.15.67:0/2779655951 wait complete. 2024-03-09T08:52:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.653+0000 7f84f6000640 1 Processor -- start 2024-03-09T08:52:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.653+0000 7f84f6000640 1 -- start start 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.654+0000 7f84f6000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f00c3010 0x7f84f014b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.654+0000 7f84f6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f0150180 0x7f84f00726e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.654+0000 7f84f6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f0156980 0x7f84f0072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.654+0000 7f84f6000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00ac9a0 con 0x7f84f0150180 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.654+0000 7f84f6000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f0153430 con 0x7f84f0156980 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.654+0000 7f84f6000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00acb20 con 0x7f84f00c3010 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.655+0000 7f84efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f0150180 0x7f84f00726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.655+0000 7f84efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f0150180 0x7f84f00726e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59826/0 (socket says 172.21.15.67:59826) 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.655+0000 7f84efe00640 1 -- 172.21.15.67:0/2962309021 learned_addr learned my addr 172.21.15.67:0/2962309021 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.655+0000 7f84f4c00640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f00c3010 0x7f84f014b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.655+0000 7f84f5600640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f0156980 0x7f84f0072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.655+0000 7f84efe00640 1 -- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f00c3010 msgr2=0x7f84f014b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.655+0000 7f84efe00640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f00c3010 0x7f84f014b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.655+0000 7f84efe00640 1 -- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f0156980 msgr2=0x7f84f0072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.655+0000 7f84efe00640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f0156980 0x7f84f0072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.655+0000 7f84efe00640 1 -- 172.21.15.67:0/2962309021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f014bdf0 con 0x7f84f0150180 2024-03-09T08:52:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.656+0000 7f84efe00640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f0150180 0x7f84f00726e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f84dc02b820 tx=0x7f84dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.656+0000 7f84ecc00640 1 -- 172.21.15.67:0/2962309021 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc042070 con 0x7f84f0150180 2024-03-09T08:52:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.656+0000 7f84ecc00640 1 -- 172.21.15.67:0/2962309021 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc031e80 con 0x7f84f0150180 2024-03-09T08:52:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.656+0000 7f84ecc00640 1 -- 172.21.15.67:0/2962309021 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc038040 con 0x7f84f0150180 2024-03-09T08:52:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.656+0000 7f84f6000640 1 -- 172.21.15.67:0/2962309021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f00bc500 con 0x7f84f0150180 2024-03-09T08:52:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.656+0000 7f84f6000640 1 -- 172.21.15.67:0/2962309021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f00c5290 con 0x7f84f0150180 2024-03-09T08:52:25.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.657+0000 7f84f6000640 1 -- 172.21.15.67:0/2962309021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f004f3c0 con 0x7f84f0150180 2024-03-09T08:52:25.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.658+0000 7f84ecc00640 1 -- 172.21.15.67:0/2962309021 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f84dc045070 con 0x7f84f0150180 2024-03-09T08:52:25.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.660+0000 7f84ecc00640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f84d006b500 0x7f84d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:25.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.660+0000 7f84ecc00640 1 -- 172.21.15.67:0/2962309021 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f84dc0abea0 con 0x7f84f0150180 2024-03-09T08:52:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.660+0000 7f84f4c00640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f84d006b500 0x7f84d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.661+0000 7f84f4c00640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f84d006b500 0x7f84d006d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f84e0006fd0 tx=0x7f84e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:25.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.664+0000 7f84ecc00640 1 -- 172.21.15.67:0/2962309021 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f84dc04c030 con 0x7f84f0150180 2024-03-09T08:52:25.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.854+0000 7f84f6000640 1 -- 172.21.15.67:0/2962309021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84f00c2640 con 0x7f84f0150180 2024-03-09T08:52:25.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.855+0000 7f84ecc00640 1 -- 172.21.15.67:0/2962309021 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v20) v1 ==== 99+0+11 (secure 0 0 0) 0x7f84dc045e40 con 0x7f84f0150180 2024-03-09T08:52:25.982 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T08:52:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.857+0000 7f84f6000640 1 -- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f84d006b500 msgr2=0x7f84d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.857+0000 7f84f6000640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f84d006b500 0x7f84d006d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f84e0006fd0 tx=0x7f84e0008040 comp rx=0 tx=0).stop 2024-03-09T08:52:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.857+0000 7f84f6000640 1 -- 172.21.15.67:0/2962309021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f0150180 msgr2=0x7f84f00726e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.857+0000 7f84f6000640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f0150180 0x7f84f00726e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f84dc02b820 tx=0x7f84dc0046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.857+0000 7f84f6000640 1 -- 172.21.15.67:0/2962309021 shutdown_connections 2024-03-09T08:52:25.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.857+0000 7f84f6000640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f84d006b500 0x7f84d006d9c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.857+0000 7f84f6000640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f0156980 0x7f84f0072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.857+0000 7f84f6000640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f0150180 0x7f84f00726e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.857+0000 7f84f6000640 1 --2- 172.21.15.67:0/2962309021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f00c3010 0x7f84f014b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:25.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.857+0000 7f84f6000640 1 -- 172.21.15.67:0/2962309021 >> 172.21.15.67:0/2962309021 conn(0x7f84f006b8a0 msgr2=0x7f84f014c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:25.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.857+0000 7f84f6000640 1 -- 172.21.15.67:0/2962309021 shutdown_connections 2024-03-09T08:52:25.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:25.858+0000 7f84f6000640 1 -- 172.21.15.67:0/2962309021 wait complete. 2024-03-09T08:52:25.989 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-03-09T08:52:26.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.224+0000 7ff348a00640 1 Processor -- start 2024-03-09T08:52:26.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.224+0000 7ff348a00640 1 -- start start 2024-03-09T08:52:26.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.226+0000 7ff348a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff3440c3010 0x7ff3440ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.226+0000 7ff348a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff344150350 0x7ff34406d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.226+0000 7ff348a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff344153950 con 0x7ff344156980 2024-03-09T08:52:26.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.226+0000 7ff348a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff344152b90 con 0x7ff344150350 2024-03-09T08:52:26.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.226+0000 7ff348a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff344152e70 con 0x7ff3440c3010 2024-03-09T08:52:26.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.226+0000 7ff343200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff344150350 0x7ff34406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.226+0000 7ff343200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff344150350 0x7ff34406d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59838/0 (socket says 172.21.15.67:59838) 2024-03-09T08:52:26.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.226+0000 7ff343200640 1 -- 172.21.15.67:0/1400809913 learned_addr learned my addr 172.21.15.67:0/1400809913 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:26.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.227+0000 7ff343200640 1 -- 172.21.15.67:0/1400809913 >> v2:172.21.15.134:3300/0 conn(0x7ff3440c3010 msgr2=0x7ff3440ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:26.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.227+0000 7ff342800640 1 --2- 172.21.15.67:0/1400809913 >> v2:172.21.15.134:3300/0 conn(0x7ff3440c3010 0x7ff3440ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.227+0000 7ff343200640 1 --2- 172.21.15.67:0/1400809913 >> v2:172.21.15.134:3300/0 conn(0x7ff3440c3010 0x7ff3440ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.227+0000 7ff343200640 1 -- 172.21.15.67:0/1400809913 >> v1:172.21.15.134:6789/0 conn(0x7ff344156980 legacy=0x7ff34414fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.227+0000 7ff343200640 1 -- 172.21.15.67:0/1400809913 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff344153150 con 0x7ff344150350 2024-03-09T08:52:26.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.228+0000 7ff343200640 1 --2- 172.21.15.67:0/1400809913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff344150350 0x7ff34406d360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff34406c020 tx=0x7ff33c02e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540489eb9be4bd5a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:26.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.228+0000 7ff341400640 1 -- 172.21.15.67:0/1400809913 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c02ea00 con 0x7ff344150350 2024-03-09T08:52:26.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.228+0000 7ff341400640 1 -- 172.21.15.67:0/1400809913 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c02eba0 con 0x7ff344150350 2024-03-09T08:52:26.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.228+0000 7ff341400640 1 -- 172.21.15.67:0/1400809913 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c009600 con 0x7ff344150350 2024-03-09T08:52:26.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.229+0000 7ff348a00640 1 -- 172.21.15.67:0/1400809913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff344150350 msgr2=0x7ff34406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.229+0000 7ff348a00640 1 --2- 172.21.15.67:0/1400809913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff344150350 0x7ff34406d360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff34406c020 tx=0x7ff33c02e060 comp rx=0 tx=0).stop 2024-03-09T08:52:26.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.229+0000 7ff348a00640 1 -- 172.21.15.67:0/1400809913 shutdown_connections 2024-03-09T08:52:26.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.229+0000 7ff348a00640 1 --2- 172.21.15.67:0/1400809913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff344150350 0x7ff34406d360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.229+0000 7ff348a00640 1 --2- 172.21.15.67:0/1400809913 >> v2:172.21.15.134:3300/0 conn(0x7ff3440c3010 0x7ff3440ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.229+0000 7ff348a00640 1 -- 172.21.15.67:0/1400809913 >> 172.21.15.67:0/1400809913 conn(0x7ff34406b8a0 msgr2=0x7ff3440c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:26.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.229+0000 7ff348a00640 1 -- 172.21.15.67:0/1400809913 shutdown_connections 2024-03-09T08:52:26.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.229+0000 7ff348a00640 1 -- 172.21.15.67:0/1400809913 wait complete. 2024-03-09T08:52:26.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.231+0000 7ff348a00640 1 Processor -- start 2024-03-09T08:52:26.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.231+0000 7ff348a00640 1 -- start start 2024-03-09T08:52:26.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.232+0000 7ff348a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3440c3010 0x7ff344071c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.232+0000 7ff348a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff344150350 0x7ff34414f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.232+0000 7ff348a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff344156980 0x7ff3441eb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.232+0000 7ff348a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440acb20 con 0x7ff344150350 2024-03-09T08:52:26.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.232+0000 7ff348a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440ac9a0 con 0x7ff3440c3010 2024-03-09T08:52:26.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.232+0000 7ff348a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440c33f0 con 0x7ff344156980 2024-03-09T08:52:26.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.232+0000 7ff341e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff344150350 0x7ff34414f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.232+0000 7ff341e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff344150350 0x7ff34414f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59850/0 (socket says 172.21.15.67:59850) 2024-03-09T08:52:26.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.232+0000 7ff341e00640 1 -- 172.21.15.67:0/670619702 learned_addr learned my addr 172.21.15.67:0/670619702 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:26.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.233+0000 7ff342800640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3440c3010 0x7ff344071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.233+0000 7ff341e00640 1 -- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff344156980 msgr2=0x7ff3441eb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.233+0000 7ff341e00640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff344156980 0x7ff3441eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.233+0000 7ff341e00640 1 -- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3440c3010 msgr2=0x7ff344071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.233+0000 7ff341e00640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3440c3010 0x7ff344071c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.233+0000 7ff341e00640 1 -- 172.21.15.67:0/670619702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3440c6b30 con 0x7ff344150350 2024-03-09T08:52:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.233+0000 7ff341e00640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff344150350 0x7ff34414f060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c3c0 tx=0x7ff33000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.233+0000 7ff33aa00640 1 -- 172.21.15.67:0/670619702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3300052d0 con 0x7ff344150350 2024-03-09T08:52:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.234+0000 7ff33aa00640 1 -- 172.21.15.67:0/670619702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330005490 con 0x7ff344150350 2024-03-09T08:52:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.234+0000 7ff33aa00640 1 -- 172.21.15.67:0/670619702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330007260 con 0x7ff344150350 2024-03-09T08:52:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.234+0000 7ff348a00640 1 -- 172.21.15.67:0/670619702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3440c6e10 con 0x7ff344150350 2024-03-09T08:52:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.234+0000 7ff348a00640 1 -- 172.21.15.67:0/670619702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34414e860 con 0x7ff344150350 2024-03-09T08:52:26.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.236+0000 7ff348a00640 1 -- 172.21.15.67:0/670619702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34404f3c0 con 0x7ff344150350 2024-03-09T08:52:26.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.236+0000 7ff33aa00640 1 -- 172.21.15.67:0/670619702 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff330007400 con 0x7ff344150350 2024-03-09T08:52:26.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.237+0000 7ff33aa00640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32406b4d0 0x7ff32406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.238+0000 7ff33aa00640 1 -- 172.21.15.67:0/670619702 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7ff3300896e0 con 0x7ff344150350 2024-03-09T08:52:26.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.238+0000 7ff342800640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32406b4d0 0x7ff32406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.239+0000 7ff342800640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32406b4d0 0x7ff32406d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff334006fd0 tx=0x7ff334008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:26.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.243+0000 7ff33aa00640 1 -- 172.21.15.67:0/670619702 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff330058560 con 0x7ff344150350 2024-03-09T08:52:26.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.445+0000 7ff348a00640 1 -- 172.21.15.67:0/670619702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7ff344155f90 con 0x7ff344150350 2024-03-09T08:52:26.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.446+0000 7ff33aa00640 1 -- 172.21.15.67:0/670619702 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v20) v1 ==== 95+0+11 (secure 0 0 0) 0x7ff330057e60 con 0x7ff344150350 2024-03-09T08:52:26.441 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T08:52:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.448+0000 7ff348a00640 1 -- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32406b4d0 msgr2=0x7ff32406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.448+0000 7ff348a00640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32406b4d0 0x7ff32406d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff334006fd0 tx=0x7ff334008040 comp rx=0 tx=0).stop 2024-03-09T08:52:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.448+0000 7ff348a00640 1 -- 172.21.15.67:0/670619702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff344150350 msgr2=0x7ff34414f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.448+0000 7ff348a00640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff344150350 0x7ff34414f060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c3c0 tx=0x7ff33000c880 comp rx=0 tx=0).stop 2024-03-09T08:52:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.448+0000 7ff348a00640 1 -- 172.21.15.67:0/670619702 shutdown_connections 2024-03-09T08:52:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.448+0000 7ff348a00640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32406b4d0 0x7ff32406d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.448+0000 7ff348a00640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff344156980 0x7ff3441eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.448+0000 7ff348a00640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff344150350 0x7ff34414f060 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.448+0000 7ff348a00640 1 --2- 172.21.15.67:0/670619702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3440c3010 0x7ff344071c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.448+0000 7ff348a00640 1 -- 172.21.15.67:0/670619702 >> 172.21.15.67:0/670619702 conn(0x7ff34406b8a0 msgr2=0x7ff3440c1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.449+0000 7ff348a00640 1 -- 172.21.15.67:0/670619702 shutdown_connections 2024-03-09T08:52:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.449+0000 7ff348a00640 1 -- 172.21.15.67:0/670619702 wait complete. 2024-03-09T08:52:26.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-09T08:52:26.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.737+0000 7fb003c00640 1 Processor -- start 2024-03-09T08:52:26.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.737+0000 7fb003c00640 1 -- start start 2024-03-09T08:52:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.738+0000 7fb003c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc152a20 0x7faffc0aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.739+0000 7fb003c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faffc0ab3c0 con 0x7faffc06dbe0 2024-03-09T08:52:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.739+0000 7fb003c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faffc0aab20 con 0x7faffc0c30f0 2024-03-09T08:52:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.739+0000 7fb003c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0c82d0 con 0x7faffc152a20 2024-03-09T08:52:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.739+0000 7fb001c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faffc06dbe0 0x7faffc072940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46348/0 (socket says 172.21.15.67:46348) 2024-03-09T08:52:26.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.739+0000 7fb001c00640 1 -- 172.21.15.67:0/26592787 learned_addr learned my addr 172.21.15.67:0/26592787 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:26.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.739+0000 7faffbe00640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc152a20 0x7faffc0aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.739+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383124832 0 0) 0x7faff4001110 con 0x7faffc06dbe0 2024-03-09T08:52:26.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe0005730 con 0x7faffc06dbe0 2024-03-09T08:52:26.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845467516 0 0) 0x7faffc0aab20 con 0x7faffc0c30f0 2024-03-09T08:52:26.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faff4001110 con 0x7faffc0c30f0 2024-03-09T08:52:26.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1360233710 0 0) 0x7fafe0005730 con 0x7faffc06dbe0 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc152a20 msgr2=0x7faffc0aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc152a20 0x7faffc0aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 >> v1:172.21.15.134:6789/0 conn(0x7faffc0c30f0 legacy=0x7faffc1490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc0aae00 con 0x7faffc06dbe0 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7faff4001840 con 0x7faffc06dbe0 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 >> v1:172.21.15.67:6789/0 conn(0x7faffc06dbe0 legacy=0x7faffc072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafe0005d80 0x7fafe0006180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0006a10 0x7fafe0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc152a20 0x7fafe0007430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe000dd60 con 0x7fafe0005d80 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe000dee0 con 0x7fafe0006a10 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe000e0b0 con 0x7faffc152a20 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.740+0000 7fb001200640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafe0005d80 0x7fafe0006180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.741+0000 7fb001200640 1 -- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc152a20 msgr2=0x7fafe0007430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.741+0000 7faffbe00640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0006a10 0x7fafe0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.741+0000 7fb001c00640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc152a20 0x7fafe0007430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.741+0000 7fb001200640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc152a20 0x7fafe0007430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.741+0000 7fb001200640 1 -- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0006a10 msgr2=0x7fafe0006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.741+0000 7fb001200640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0006a10 0x7fafe0006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.741+0000 7fb001200640 1 -- 172.21.15.67:0/26592787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe0005730 con 0x7fafe0005d80 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.741+0000 7fb001c00640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc152a20 0x7fafe0007430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.741+0000 7faffbe00640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0006a10 0x7fafe0006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.742+0000 7fb001200640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafe0005d80 0x7fafe0006180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafec002000 tx=0x7fafe0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629b6e540a648dac server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.742+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec001bf0 con 0x7fafe0005d80 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.742+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec003b70 con 0x7fafe0005d80 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.742+0000 7faffb400640 1 -- 172.21.15.67:0/26592787 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec0050d0 con 0x7fafe0005d80 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.743+0000 7fb003c00640 1 -- 172.21.15.67:0/26592787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafe0005d80 msgr2=0x7fafe0006180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.743+0000 7fb003c00640 1 --2- 172.21.15.67:0/26592787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafe0005d80 0x7fafe0006180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafec002000 tx=0x7fafe0007a40 comp rx=0 tx=0).stop 2024-03-09T08:52:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.743+0000 7fb001c00640 1 -- 172.21.15.67:0/26592787 reap_dead start 2024-03-09T08:52:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.743+0000 7fb003c00640 1 -- 172.21.15.67:0/26592787 shutdown_connections 2024-03-09T08:52:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.743+0000 7fb003c00640 1 -- 172.21.15.67:0/26592787 >> 172.21.15.67:0/26592787 conn(0x7faffc0c7370 msgr2=0x7faffc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.743+0000 7fb003c00640 1 -- 172.21.15.67:0/26592787 shutdown_connections 2024-03-09T08:52:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.743+0000 7fb003c00640 1 -- 172.21.15.67:0/26592787 wait complete. 2024-03-09T08:52:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.745+0000 7fb003c00640 1 Processor -- start 2024-03-09T08:52:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.745+0000 7fb003c00640 1 -- start start 2024-03-09T08:52:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb003c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0005d80 0x7faffc072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb003c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc14ad00 0x7faffc1eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb003c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faffc14b180 0x7faffc1ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb003c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc149800 con 0x7faffc14b180 2024-03-09T08:52:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb003c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc06d6c0 con 0x7fafe0005d80 2024-03-09T08:52:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb003c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc149980 con 0x7faffc14ad00 2024-03-09T08:52:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb001c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faffc14b180 0x7faffc1ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb001c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faffc14b180 0x7faffc1ef3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59870/0 (socket says 172.21.15.67:59870) 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb001c00640 1 -- 172.21.15.67:0/1789166706 learned_addr learned my addr 172.21.15.67:0/1789166706 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7faffbe00640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc14ad00 0x7faffc1eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb001200640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0005d80 0x7faffc072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb001c00640 1 -- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc14ad00 msgr2=0x7faffc1eeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb001c00640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc14ad00 0x7faffc1eeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.746+0000 7fb001c00640 1 -- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0005d80 msgr2=0x7faffc072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.747+0000 7fb001c00640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0005d80 0x7faffc072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.747+0000 7fb001c00640 1 -- 172.21.15.67:0/1789166706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc14df00 con 0x7faffc14b180 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.747+0000 7fb001c00640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faffc14b180 0x7faffc1ef3e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faffc0715c0 tx=0x7faff400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.747+0000 7faff8c00640 1 -- 172.21.15.67:0/1789166706 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff40072d0 con 0x7faffc14b180 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.747+0000 7faff8c00640 1 -- 172.21.15.67:0/1789166706 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff4007490 con 0x7faffc14b180 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.747+0000 7faff8c00640 1 -- 172.21.15.67:0/1789166706 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4004a20 con 0x7faffc14b180 2024-03-09T08:52:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.747+0000 7fb003c00640 1 -- 172.21.15.67:0/1789166706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faffc1ef920 con 0x7faffc14b180 2024-03-09T08:52:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.747+0000 7fb003c00640 1 -- 172.21.15.67:0/1789166706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faffc072350 con 0x7faffc14b180 2024-03-09T08:52:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.748+0000 7fb003c00640 1 -- 172.21.15.67:0/1789166706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faffc075a00 con 0x7faffc14b180 2024-03-09T08:52:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.749+0000 7faff8c00640 1 -- 172.21.15.67:0/1789166706 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faff401b020 con 0x7faffc14b180 2024-03-09T08:52:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.750+0000 7faff8c00640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafdc06b520 0x7fafdc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.750+0000 7faff8c00640 1 -- 172.21.15.67:0/1789166706 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7faff4089ce0 con 0x7faffc14b180 2024-03-09T08:52:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.751+0000 7fb001200640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafdc06b520 0x7fafdc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.752+0000 7fb001200640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafdc06b520 0x7fafdc06d9e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fafec034000 tx=0x7fafec009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.755+0000 7faff8c00640 1 -- 172.21.15.67:0/1789166706 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faffc0ab3c0 con 0x7faffc14b180 2024-03-09T08:52:26.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.958+0000 7fb003c00640 1 -- 172.21.15.67:0/1789166706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faffc0a9f70 con 0x7faffc14b180 2024-03-09T08:52:26.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.959+0000 7faff8c00640 1 -- 172.21.15.67:0/1789166706 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+197 (secure 0 0 0) 0x7faff4058be0 con 0x7faffc14b180 2024-03-09T08:52:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:26.955 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-03-09T08:52:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:52:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-03-09T08:52:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-03-09T08:52:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:52:26.955 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:52:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 2, 2024-03-09T08:52:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": "a_metadata" 2024-03-09T08:52:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:52:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:52:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 3, 2024-03-09T08:52:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": "a_data" 2024-03-09T08:52:26.956 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:52:26.956 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-03-09T08:52:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 -- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafdc06b520 msgr2=0x7fafdc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafdc06b520 0x7fafdc06d9e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fafec034000 tx=0x7fafec009030 comp rx=0 tx=0).stop 2024-03-09T08:52:26.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 -- 172.21.15.67:0/1789166706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faffc14b180 msgr2=0x7faffc1ef3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faffc14b180 0x7faffc1ef3e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faffc0715c0 tx=0x7faff400dc00 comp rx=0 tx=0).stop 2024-03-09T08:52:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 -- 172.21.15.67:0/1789166706 shutdown_connections 2024-03-09T08:52:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafdc06b520 0x7fafdc06d9e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faffc14b180 0x7faffc1ef3e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faffc14ad00 0x7faffc1eeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 --2- 172.21.15.67:0/1789166706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe0005d80 0x7faffc072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 -- 172.21.15.67:0/1789166706 >> 172.21.15.67:0/1789166706 conn(0x7faffc0c7370 msgr2=0x7faffc074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 -- 172.21.15.67:0/1789166706 shutdown_connections 2024-03-09T08:52:26.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:26.962+0000 7fb003c00640 1 -- 172.21.15.67:0/1789166706 wait complete. 2024-03-09T08:52:26.976 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'b' 2024-03-09T08:52:26.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create b_metadata --pg_num_min 64 2024-03-09T08:52:27.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.209+0000 7f048ca00640 1 Processor -- start 2024-03-09T08:52:27.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.209+0000 7f048ca00640 1 -- start start 2024-03-09T08:52:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.211+0000 7f048ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04880c73d0 0x7f04881547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.211+0000 7f048ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0488151720 0x7f0488072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.211+0000 7f048ca00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048806d6c0 con 0x7f04880710a0 2024-03-09T08:52:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.211+0000 7f048ca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048806e480 con 0x7f0488151720 2024-03-09T08:52:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.211+0000 7f048ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048806dff0 con 0x7f04880c73d0 2024-03-09T08:52:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.211+0000 7f0486800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0488151720 0x7f0488072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.211+0000 7f0486800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0488151720 0x7f0488072cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59878/0 (socket says 172.21.15.67:59878) 2024-03-09T08:52:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.211+0000 7f0486800640 1 -- 172.21.15.67:0/3728608577 learned_addr learned my addr 172.21.15.67:0/3728608577 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.211+0000 7f0485e00640 1 --2- 172.21.15.67:0/3728608577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04880c73d0 0x7f04881547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.212+0000 7f0486800640 1 -- 172.21.15.67:0/3728608577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04880c73d0 msgr2=0x7f04881547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:27.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.212+0000 7f0486800640 1 --2- 172.21.15.67:0/3728608577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04880c73d0 0x7f04881547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.212+0000 7f0486800640 1 -- 172.21.15.67:0/3728608577 >> v1:172.21.15.134:6789/0 conn(0x7f04880710a0 legacy=0x7f0488151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:27.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.212+0000 7f0486800640 1 -- 172.21.15.67:0/3728608577 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048806d9a0 con 0x7f0488151720 2024-03-09T08:52:27.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.213+0000 7f0486800640 1 --2- 172.21.15.67:0/3728608577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0488151720 0x7f0488072cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0480009870 tx=0x7f048002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe80e8f2887b8670 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:27.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.213+0000 7f0485400640 1 -- 172.21.15.67:0/3728608577 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480034070 con 0x7f0488151720 2024-03-09T08:52:27.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.213+0000 7f0485400640 1 -- 172.21.15.67:0/3728608577 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0480030840 con 0x7f0488151720 2024-03-09T08:52:27.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.213+0000 7f0485400640 1 -- 172.21.15.67:0/3728608577 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480030c10 con 0x7f0488151720 2024-03-09T08:52:27.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.214+0000 7f048ca00640 1 -- 172.21.15.67:0/3728608577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0488151720 msgr2=0x7f0488072cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:27.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.214+0000 7f048ca00640 1 --2- 172.21.15.67:0/3728608577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0488151720 0x7f0488072cd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0480009870 tx=0x7f048002b980 comp rx=0 tx=0).stop 2024-03-09T08:52:27.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.214+0000 7f048ca00640 1 -- 172.21.15.67:0/3728608577 shutdown_connections 2024-03-09T08:52:27.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.214+0000 7f048ca00640 1 --2- 172.21.15.67:0/3728608577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0488151720 0x7f0488072cd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.214+0000 7f048ca00640 1 --2- 172.21.15.67:0/3728608577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04880c73d0 0x7f04881547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.214+0000 7f048ca00640 1 -- 172.21.15.67:0/3728608577 >> 172.21.15.67:0/3728608577 conn(0x7f048806b040 msgr2=0x7f048806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:27.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.214+0000 7f048ca00640 1 -- 172.21.15.67:0/3728608577 shutdown_connections 2024-03-09T08:52:27.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.214+0000 7f048ca00640 1 -- 172.21.15.67:0/3728608577 wait complete. 2024-03-09T08:52:27.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.216+0000 7f048ca00640 1 Processor -- start 2024-03-09T08:52:27.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.216+0000 7f048ca00640 1 -- start start 2024-03-09T08:52:27.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.217+0000 7f048ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04880710a0 0x7f04880c0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.217+0000 7f048ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04880c73d0 0x7f0488150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.217+0000 7f048ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0488151720 0x7f048814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.217+0000 7f048ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880c77b0 con 0x7f04880c73d0 2024-03-09T08:52:27.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.217+0000 7f048ca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0488071500 con 0x7f04880710a0 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.217+0000 7f048ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04880c7930 con 0x7f0488151720 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.217+0000 7f047d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04880c73d0 0x7f0488150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.218+0000 7f047d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04880c73d0 0x7f0488150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59880/0 (socket says 172.21.15.67:59880) 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.218+0000 7f047d400640 1 -- 172.21.15.67:0/4024410359 learned_addr learned my addr 172.21.15.67:0/4024410359 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.218+0000 7f0485e00640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04880710a0 0x7f04880c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.218+0000 7f0486800640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0488151720 0x7f048814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.218+0000 7f047d400640 1 -- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0488151720 msgr2=0x7f048814f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.218+0000 7f047d400640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0488151720 0x7f048814f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.218+0000 7f047d400640 1 -- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04880710a0 msgr2=0x7f04880c0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.218+0000 7f047d400640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04880710a0 0x7f04880c0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.218+0000 7f047d400640 1 -- 172.21.15.67:0/4024410359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04881ec010 con 0x7f04880c73d0 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.218+0000 7f047d400640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04880c73d0 0x7f0488150070 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f047000c450 tx=0x7f047000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.219+0000 7f047ea00640 1 -- 172.21.15.67:0/4024410359 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04700052d0 con 0x7f04880c73d0 2024-03-09T08:52:27.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.219+0000 7f047ea00640 1 -- 172.21.15.67:0/4024410359 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0470005470 con 0x7f04880c73d0 2024-03-09T08:52:27.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.219+0000 7f047ea00640 1 -- 172.21.15.67:0/4024410359 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470012870 con 0x7f04880c73d0 2024-03-09T08:52:27.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.219+0000 7f048ca00640 1 -- 172.21.15.67:0/4024410359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04880ac000 con 0x7f04880c73d0 2024-03-09T08:52:27.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.219+0000 7f048ca00640 1 -- 172.21.15.67:0/4024410359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04881ec2b0 con 0x7f04880c73d0 2024-03-09T08:52:27.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.220+0000 7f048ca00640 1 -- 172.21.15.67:0/4024410359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048806b5e0 con 0x7f04880c73d0 2024-03-09T08:52:27.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.221+0000 7f047ea00640 1 -- 172.21.15.67:0/4024410359 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0470012a10 con 0x7f04880c73d0 2024-03-09T08:52:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.222+0000 7f047ea00640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f046406b520 0x7f046406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.222+0000 7f047ea00640 1 -- 172.21.15.67:0/4024410359 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f0470089c90 con 0x7f04880c73d0 2024-03-09T08:52:27.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.222+0000 7f0485e00640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f046406b520 0x7f046406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.224+0000 7f0485e00640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f046406b520 0x7f046406d9e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f04880abd20 tx=0x7f047800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:27.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.226+0000 7f047ea00640 1 -- 172.21.15.67:0/4024410359 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0470058c90 con 0x7f04880c73d0 2024-03-09T08:52:27.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.428+0000 7f048ca00640 1 -- 172.21.15.67:0/4024410359 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f04880c0d70 con 0x7f04880c73d0 2024-03-09T08:52:27.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.538+0000 7f047ea00640 1 -- 172.21.15.67:0/4024410359 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b_metadata", "pg_num_min": 64}]=0 pool 'b_metadata' created v21) v1 ==== 128+0+0 (secure 0 0 0) 0x7f0470058590 con 0x7f04880c73d0 2024-03-09T08:52:27.534 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'b_metadata' created 2024-03-09T08:52:27.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.541+0000 7f048ca00640 1 -- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f046406b520 msgr2=0x7f046406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:27.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.541+0000 7f048ca00640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f046406b520 0x7f046406d9e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f04880abd20 tx=0x7f047800b040 comp rx=0 tx=0).stop 2024-03-09T08:52:27.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.541+0000 7f048ca00640 1 -- 172.21.15.67:0/4024410359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04880c73d0 msgr2=0x7f0488150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:27.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.541+0000 7f048ca00640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04880c73d0 0x7f0488150070 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f047000c450 tx=0x7f047000c910 comp rx=0 tx=0).stop 2024-03-09T08:52:27.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.541+0000 7f048ca00640 1 -- 172.21.15.67:0/4024410359 shutdown_connections 2024-03-09T08:52:27.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.541+0000 7f048ca00640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f046406b520 0x7f046406d9e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.541+0000 7f048ca00640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0488151720 0x7f048814f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.541+0000 7f048ca00640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04880c73d0 0x7f0488150070 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.541+0000 7f048ca00640 1 --2- 172.21.15.67:0/4024410359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04880710a0 0x7f04880c0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.541+0000 7f048ca00640 1 -- 172.21.15.67:0/4024410359 >> 172.21.15.67:0/4024410359 conn(0x7f048806b040 msgr2=0x7f0488152b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:27.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.542+0000 7f048ca00640 1 -- 172.21.15.67:0/4024410359 shutdown_connections 2024-03-09T08:52:27.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.542+0000 7f048ca00640 1 -- 172.21.15.67:0/4024410359 wait complete. 2024-03-09T08:52:27.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create b_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-09T08:52:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.787+0000 7f83eda00640 1 Processor -- start 2024-03-09T08:52:27.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.787+0000 7f83eda00640 1 -- start start 2024-03-09T08:52:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.789+0000 7f83eda00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83e80ac310 0x7f83e806ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.789+0000 7f83eda00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f83e806c460 0x7f83e8152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.789+0000 7f83eda00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e80c3900 con 0x7f83e8153220 2024-03-09T08:52:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.789+0000 7f83eda00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e80c36b0 con 0x7f83e806c460 2024-03-09T08:52:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.789+0000 7f83eda00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80c3be0 con 0x7f83e80ac310 2024-03-09T08:52:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.789+0000 7f83ed000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f83e8153220 0x7f83e81594b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46364/0 (socket says 172.21.15.67:46364) 2024-03-09T08:52:27.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.789+0000 7f83ed000640 1 -- 172.21.15.67:0/1936206152 learned_addr learned my addr 172.21.15.67:0/1936206152 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.789+0000 7f83e7400640 1 --2- 172.21.15.67:0/1936206152 >> v2:172.21.15.134:3300/0 conn(0x7f83e806c460 0x7f83e8152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.789+0000 7f83e7e00640 1 --2- 172.21.15.67:0/1936206152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83e80ac310 0x7f83e806ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.790+0000 7f83e6a00640 1 -- 172.21.15.67:0/1936206152 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825764114 0 0) 0x7f83e80c3900 con 0x7f83e8153220 2024-03-09T08:52:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.790+0000 7f83e6a00640 1 -- 172.21.15.67:0/1936206152 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83cc003520 con 0x7f83e8153220 2024-03-09T08:52:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.790+0000 7f83e7e00640 1 -- 172.21.15.67:0/1936206152 >> v2:172.21.15.134:3300/0 conn(0x7f83e806c460 msgr2=0x7f83e8152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.790+0000 7f83e7e00640 1 --2- 172.21.15.67:0/1936206152 >> v2:172.21.15.134:3300/0 conn(0x7f83e806c460 0x7f83e8152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.790+0000 7f83e7e00640 1 -- 172.21.15.67:0/1936206152 >> v1:172.21.15.67:6789/0 conn(0x7f83e8153220 legacy=0x7f83e81594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.790+0000 7f83e7e00640 1 -- 172.21.15.67:0/1936206152 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80aa4e0 con 0x7f83e80ac310 2024-03-09T08:52:27.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.790+0000 7f83e7400640 1 --2- 172.21.15.67:0/1936206152 >> v2:172.21.15.134:3300/0 conn(0x7f83e806c460 0x7f83e8152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:52:27.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.791+0000 7f83e7e00640 1 --2- 172.21.15.67:0/1936206152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83e80ac310 0x7f83e806ca50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f83d80097a0 tx=0x7f83d802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=279c844c0af1cde6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:27.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.792+0000 7f83e6a00640 1 -- 172.21.15.67:0/1936206152 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8039070 con 0x7f83e80ac310 2024-03-09T08:52:27.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.792+0000 7f83e6a00640 1 -- 172.21.15.67:0/1936206152 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d80316f0 con 0x7f83e80ac310 2024-03-09T08:52:27.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.792+0000 7f83e6a00640 1 -- 172.21.15.67:0/1936206152 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8031aa0 con 0x7f83e80ac310 2024-03-09T08:52:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.793+0000 7f83eda00640 1 -- 172.21.15.67:0/1936206152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83e80ac310 msgr2=0x7f83e806ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.793+0000 7f83eda00640 1 --2- 172.21.15.67:0/1936206152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83e80ac310 0x7f83e806ca50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f83d80097a0 tx=0x7f83d802b790 comp rx=0 tx=0).stop 2024-03-09T08:52:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.793+0000 7f83eda00640 1 -- 172.21.15.67:0/1936206152 shutdown_connections 2024-03-09T08:52:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.793+0000 7f83eda00640 1 --2- 172.21.15.67:0/1936206152 >> v2:172.21.15.134:3300/0 conn(0x7f83e806c460 0x7f83e8152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.793+0000 7f83eda00640 1 --2- 172.21.15.67:0/1936206152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83e80ac310 0x7f83e806ca50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.793+0000 7f83eda00640 1 -- 172.21.15.67:0/1936206152 >> 172.21.15.67:0/1936206152 conn(0x7f83e80c3d90 msgr2=0x7f83e80c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.793+0000 7f83eda00640 1 -- 172.21.15.67:0/1936206152 shutdown_connections 2024-03-09T08:52:27.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.793+0000 7f83eda00640 1 -- 172.21.15.67:0/1936206152 wait complete. 2024-03-09T08:52:27.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.795+0000 7f83eda00640 1 Processor -- start 2024-03-09T08:52:27.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.796+0000 7f83eda00640 1 -- start start 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83eda00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83e806c460 0x7f83e814d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83eda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e80ac310 0x7f83e8149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83eda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e8153220 0x7f83e8149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83eda00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8153800 con 0x7f83e80ac310 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83eda00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8153680 con 0x7f83e8153220 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83eda00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80ac6f0 con 0x7f83e806c460 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e80ac310 0x7f83e8149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e80ac310 0x7f83e8149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59892/0 (socket says 172.21.15.67:59892) 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83e7400640 1 -- 172.21.15.67:0/109842754 learned_addr learned my addr 172.21.15.67:0/109842754 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83ed000640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e8153220 0x7f83e8149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83e7e00640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83e806c460 0x7f83e814d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83e7400640 1 -- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83e806c460 msgr2=0x7f83e814d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:27.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83e7400640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83e806c460 0x7f83e814d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83e7400640 1 -- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e8153220 msgr2=0x7f83e8149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83e7400640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e8153220 0x7f83e8149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83e7400640 1 -- 172.21.15.67:0/109842754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e8149cc0 con 0x7f83e80ac310 2024-03-09T08:52:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.797+0000 7f83ed000640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e8153220 0x7f83e8149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.798+0000 7f83e7400640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e80ac310 0x7f83e8149240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f83d400ec10 tx=0x7f83d400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.798+0000 7f83dfe00640 1 -- 172.21.15.67:0/109842754 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d4017070 con 0x7f83e80ac310 2024-03-09T08:52:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.798+0000 7f83dfe00640 1 -- 172.21.15.67:0/109842754 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d400cc30 con 0x7f83e80ac310 2024-03-09T08:52:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.798+0000 7f83dfe00640 1 -- 172.21.15.67:0/109842754 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d4013470 con 0x7f83e80ac310 2024-03-09T08:52:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.798+0000 7f83eda00640 1 -- 172.21.15.67:0/109842754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e814f060 con 0x7f83e80ac310 2024-03-09T08:52:27.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.798+0000 7f83eda00640 1 -- 172.21.15.67:0/109842754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e814f620 con 0x7f83e80ac310 2024-03-09T08:52:27.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.799+0000 7f83eda00640 1 -- 172.21.15.67:0/109842754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e8072150 con 0x7f83e80ac310 2024-03-09T08:52:27.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.800+0000 7f83dfe00640 1 -- 172.21.15.67:0/109842754 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f83d401a070 con 0x7f83e80ac310 2024-03-09T08:52:27.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.801+0000 7f83dfe00640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83c806b500 0x7f83c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:27.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.802+0000 7f83dfe00640 1 -- 172.21.15.67:0/109842754 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6054+0+0 (secure 0 0 0) 0x7f83d408a2b0 con 0x7f83e80ac310 2024-03-09T08:52:27.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.802+0000 7f83e7e00640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83c806b500 0x7f83c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:27.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.803+0000 7f83e7e00640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83c806b500 0x7f83c806d9c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f83d8036000 tx=0x7f83d8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:27.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:27.808+0000 7f83dfe00640 1 -- 172.21.15.67:0/109842754 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f83e8072150 con 0x7f83e80ac310 2024-03-09T08:52:28.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.007+0000 7f83eda00640 1 -- 172.21.15.67:0/109842754 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f83e804f430 con 0x7f83e80ac310 2024-03-09T08:52:28.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.544+0000 7f83dfe00640 1 -- 172.21.15.67:0/109842754 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'b_data' created v22) v1 ==== 160+0+0 (secure 0 0 0) 0x7f83d4058fc0 con 0x7f83e80ac310 2024-03-09T08:52:28.540 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'b_data' created 2024-03-09T08:52:28.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.549+0000 7f83eda00640 1 -- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83c806b500 msgr2=0x7f83c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.549+0000 7f83eda00640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83c806b500 0x7f83c806d9c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f83d8036000 tx=0x7f83d8008030 comp rx=0 tx=0).stop 2024-03-09T08:52:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.549+0000 7f83eda00640 1 -- 172.21.15.67:0/109842754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e80ac310 msgr2=0x7f83e8149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.549+0000 7f83eda00640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e80ac310 0x7f83e8149240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f83d400ec10 tx=0x7f83d400c520 comp rx=0 tx=0).stop 2024-03-09T08:52:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.549+0000 7f83eda00640 1 -- 172.21.15.67:0/109842754 shutdown_connections 2024-03-09T08:52:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.549+0000 7f83eda00640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83c806b500 0x7f83c806d9c0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f83d8036000 tx=0x7f83d8008030 comp rx=0 tx=0).stop 2024-03-09T08:52:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.549+0000 7f83eda00640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e8153220 0x7f83e8149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.549+0000 7f83eda00640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e80ac310 0x7f83e8149240 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f83d400ec10 tx=0x7f83d400c520 comp rx=0 tx=0).stop 2024-03-09T08:52:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.549+0000 7f83eda00640 1 --2- 172.21.15.67:0/109842754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83e806c460 0x7f83e814d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.549+0000 7f83eda00640 1 -- 172.21.15.67:0/109842754 >> 172.21.15.67:0/109842754 conn(0x7f83e80c3d90 msgr2=0x7f83e814b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:28.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.551+0000 7f83eda00640 1 -- 172.21.15.67:0/109842754 shutdown_connections 2024-03-09T08:52:28.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.551+0000 7f83eda00640 1 -- 172.21.15.67:0/109842754 wait complete. 2024-03-09T08:52:28.584 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new b b_metadata b_data 2024-03-09T08:52:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.775+0000 7fa7a7600640 1 Processor -- start 2024-03-09T08:52:28.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.775+0000 7fa7a7600640 1 -- start start 2024-03-09T08:52:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.776+0000 7fa7a7600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7a00c73d0 0x7fa7a01547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.776+0000 7fa7a7600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa7a0151720 0x7fa7a0072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.777+0000 7fa7a7600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a006d6c0 con 0x7fa7a00710a0 2024-03-09T08:52:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.777+0000 7fa7a7600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a006e480 con 0x7fa7a0151720 2024-03-09T08:52:28.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.777+0000 7fa7a7600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a006dff0 con 0x7fa7a00c73d0 2024-03-09T08:52:28.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.777+0000 7fa7a4c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7a00c73d0 0x7fa7a01547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:28.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.777+0000 7fa79fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa7a00710a0 0x7fa7a0151010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46376/0 (socket says 172.21.15.67:46376) 2024-03-09T08:52:28.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.777+0000 7fa79fe00640 1 -- 172.21.15.67:0/834692349 learned_addr learned my addr 172.21.15.67:0/834692349 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:28.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.777+0000 7fa7a5600640 1 --2- 172.21.15.67:0/834692349 >> v2:172.21.15.134:3300/0 conn(0x7fa7a0151720 0x7fa7a0072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:28.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.778+0000 7fa79f400640 1 -- 172.21.15.67:0/834692349 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3100450084 0 0) 0x7fa7a006d6c0 con 0x7fa7a00710a0 2024-03-09T08:52:28.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.778+0000 7fa79f400640 1 -- 172.21.15.67:0/834692349 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa788003520 con 0x7fa7a00710a0 2024-03-09T08:52:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.778+0000 7fa7a5600640 1 -- 172.21.15.67:0/834692349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7a00c73d0 msgr2=0x7fa7a01547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.778+0000 7fa7a5600640 1 --2- 172.21.15.67:0/834692349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7a00c73d0 0x7fa7a01547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.778+0000 7fa7a5600640 1 -- 172.21.15.67:0/834692349 >> v1:172.21.15.67:6789/0 conn(0x7fa7a00710a0 legacy=0x7fa7a0151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.778+0000 7fa7a5600640 1 -- 172.21.15.67:0/834692349 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a006d9a0 con 0x7fa7a0151720 2024-03-09T08:52:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.778+0000 7fa7a4c00640 1 --2- 172.21.15.67:0/834692349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7a00c73d0 0x7fa7a01547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:52:28.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.779+0000 7fa7a5600640 1 --2- 172.21.15.67:0/834692349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a0151720 0x7fa7a0072cd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa798009d90 tx=0x7fa79802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741d627119e00703 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.779+0000 7fa79f400640 1 -- 172.21.15.67:0/834692349 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798039070 con 0x7fa7a0151720 2024-03-09T08:52:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.780+0000 7fa79f400640 1 -- 172.21.15.67:0/834692349 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79802cc60 con 0x7fa7a0151720 2024-03-09T08:52:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.780+0000 7fa79f400640 1 -- 172.21.15.67:0/834692349 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798034640 con 0x7fa7a0151720 2024-03-09T08:52:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.780+0000 7fa7a7600640 1 -- 172.21.15.67:0/834692349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a0151720 msgr2=0x7fa7a0072cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.780+0000 7fa7a7600640 1 --2- 172.21.15.67:0/834692349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a0151720 0x7fa7a0072cd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa798009d90 tx=0x7fa79802beb0 comp rx=0 tx=0).stop 2024-03-09T08:52:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.780+0000 7fa7a7600640 1 -- 172.21.15.67:0/834692349 shutdown_connections 2024-03-09T08:52:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.780+0000 7fa7a7600640 1 --2- 172.21.15.67:0/834692349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a0151720 0x7fa7a0072cd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.780+0000 7fa7a7600640 1 --2- 172.21.15.67:0/834692349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7a00c73d0 0x7fa7a01547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.780+0000 7fa7a7600640 1 -- 172.21.15.67:0/834692349 >> 172.21.15.67:0/834692349 conn(0x7fa7a006b040 msgr2=0x7fa7a006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.780+0000 7fa7a7600640 1 -- 172.21.15.67:0/834692349 shutdown_connections 2024-03-09T08:52:28.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.780+0000 7fa7a7600640 1 -- 172.21.15.67:0/834692349 wait complete. 2024-03-09T08:52:28.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.782+0000 7fa7a7600640 1 Processor -- start 2024-03-09T08:52:28.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.782+0000 7fa7a7600640 1 -- start start 2024-03-09T08:52:28.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.783+0000 7fa7a7600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a00710a0 0x7fa7a01ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:28.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.783+0000 7fa7a7600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7a00c73d0 0x7fa7a014c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:28.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.783+0000 7fa7a7600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7a0151720 0x7fa7a014cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:28.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.783+0000 7fa7a7600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00c77b0 con 0x7fa7a00c73d0 2024-03-09T08:52:28.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.783+0000 7fa7a7600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0071500 con 0x7fa7a00710a0 2024-03-09T08:52:28.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.783+0000 7fa7a7600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00c7930 con 0x7fa7a0151720 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.783+0000 7fa79fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7a00c73d0 0x7fa7a014c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.784+0000 7fa79fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7a00c73d0 0x7fa7a014c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59906/0 (socket says 172.21.15.67:59906) 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.784+0000 7fa79fe00640 1 -- 172.21.15.67:0/3268142654 learned_addr learned my addr 172.21.15.67:0/3268142654 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.784+0000 7fa7a4c00640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a00710a0 0x7fa7a01ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.784+0000 7fa7a5600640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7a0151720 0x7fa7a014cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.784+0000 7fa79fe00640 1 -- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7a0151720 msgr2=0x7fa7a014cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.784+0000 7fa79fe00640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7a0151720 0x7fa7a014cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.784+0000 7fa79fe00640 1 -- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a00710a0 msgr2=0x7fa7a01ef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.784+0000 7fa79fe00640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a00710a0 0x7fa7a01ef020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.784+0000 7fa79fe00640 1 -- 172.21.15.67:0/3268142654 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a01efb30 con 0x7fa7a00c73d0 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.784+0000 7fa79fe00640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7a00c73d0 0x7fa7a014c7d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00c430 tx=0x7fa78c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.785+0000 7fa79cc00640 1 -- 172.21.15.67:0/3268142654 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c017070 con 0x7fa7a00c73d0 2024-03-09T08:52:28.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.785+0000 7fa79cc00640 1 -- 172.21.15.67:0/3268142654 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c005ce0 con 0x7fa7a00c73d0 2024-03-09T08:52:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.785+0000 7fa79cc00640 1 -- 172.21.15.67:0/3268142654 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c0072f0 con 0x7fa7a00c73d0 2024-03-09T08:52:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.785+0000 7fa7a7600640 1 -- 172.21.15.67:0/3268142654 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a00aa600 con 0x7fa7a00c73d0 2024-03-09T08:52:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.785+0000 7fa7a7600640 1 -- 172.21.15.67:0/3268142654 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a00aabc0 con 0x7fa7a00c73d0 2024-03-09T08:52:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.786+0000 7fa7a7600640 1 -- 172.21.15.67:0/3268142654 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a0075a00 con 0x7fa7a00c73d0 2024-03-09T08:52:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.787+0000 7fa79cc00640 1 -- 172.21.15.67:0/3268142654 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa78c01a070 con 0x7fa7a00c73d0 2024-03-09T08:52:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.788+0000 7fa79cc00640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa78006b4d0 0x7fa78006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.788+0000 7fa79cc00640 1 -- 172.21.15.67:0/3268142654 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fa78c089890 con 0x7fa7a00c73d0 2024-03-09T08:52:28.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.789+0000 7fa7a4c00640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa78006b4d0 0x7fa78006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:28.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.790+0000 7fa7a4c00640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa78006b4d0 0x7fa78006d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa794009560 tx=0x7fa794009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:28.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:28.794+0000 7fa79cc00640 1 -- 172.21.15.67:0/3268142654 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa78c057e60 con 0x7fa7a00c73d0 2024-03-09T08:52:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.034+0000 7fa7a7600640 1 -- 172.21.15.67:0/3268142654 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "b", "metadata": "b_metadata", "data": "b_data"} v 0) v1 -- 0x7fa7a00c1710 con 0x7fa7a00c73d0 2024-03-09T08:52:29.567 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 4 and data pool 5 2024-03-09T08:52:29.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.569+0000 7fa79cc00640 1 -- 172.21.15.67:0/3268142654 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "b", "metadata": "b_metadata", "data": "b_data"}]=0 new fs with metadata pool 4 and data pool 5 v9) v1 ==== 157+0+0 (secure 0 0 0) 0x7fa78c05ce20 con 0x7fa7a00c73d0 2024-03-09T08:52:29.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.576+0000 7fa7a7600640 1 -- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa78006b4d0 msgr2=0x7fa78006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:29.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.576+0000 7fa7a7600640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa78006b4d0 0x7fa78006d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa794009560 tx=0x7fa794009190 comp rx=0 tx=0).stop 2024-03-09T08:52:29.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.576+0000 7fa7a7600640 1 -- 172.21.15.67:0/3268142654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7a00c73d0 msgr2=0x7fa7a014c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:29.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.576+0000 7fa7a7600640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7a00c73d0 0x7fa7a014c7d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00c430 tx=0x7fa78c00c8f0 comp rx=0 tx=0).stop 2024-03-09T08:52:29.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.578+0000 7fa7a7600640 1 -- 172.21.15.67:0/3268142654 shutdown_connections 2024-03-09T08:52:29.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.578+0000 7fa7a7600640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa78006b4d0 0x7fa78006d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:29.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.578+0000 7fa7a7600640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7a0151720 0x7fa7a014cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:29.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.578+0000 7fa7a7600640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7a00c73d0 0x7fa7a014c7d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.578+0000 7fa7a7600640 1 --2- 172.21.15.67:0/3268142654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7a00710a0 0x7fa7a01ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.578+0000 7fa7a7600640 1 -- 172.21.15.67:0/3268142654 >> 172.21.15.67:0/3268142654 conn(0x7fa7a006b040 msgr2=0x7fa7a00c5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.578+0000 7fa7a7600640 1 -- 172.21.15.67:0/3268142654 shutdown_connections 2024-03-09T08:52:29.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.579+0000 7fa7a7600640 1 -- 172.21.15.67:0/3268142654 wait complete. 2024-03-09T08:52:29.614 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:29.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.794+0000 7f9350c00640 1 Processor -- start 2024-03-09T08:52:29.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.795+0000 7f9350c00640 1 -- start start 2024-03-09T08:52:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.796+0000 7f9350c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c0c1d00 0x7f934c0aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.796+0000 7f9350c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934c06de60 con 0x7f934c0aa8f0 2024-03-09T08:52:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.796+0000 7f9350c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934c06d5c0 con 0x7f934c0acd90 2024-03-09T08:52:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.796+0000 7f9350c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1592e0 con 0x7f934c0c1d00 2024-03-09T08:52:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.796+0000 7f934be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f934c0aa8f0 0x7f934c159770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54352/0 (socket says 172.21.15.67:54352) 2024-03-09T08:52:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.796+0000 7f934be00640 1 -- 172.21.15.67:0/301086143 learned_addr learned my addr 172.21.15.67:0/301086143 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934aa00640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c0c1d00 0x7f934c0aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078446922 0 0) 0x7f934c06de60 con 0x7f934c0aa8f0 2024-03-09T08:52:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9330003520 con 0x7f934c0aa8f0 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 64752425 0 0) 0x7f9330003520 con 0x7f934c0aa8f0 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c0c1d00 msgr2=0x7f934c0aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c0c1d00 0x7f934c0aa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 >> v1:172.21.15.134:6789/0 conn(0x7f934c0acd90 legacy=0x7f934c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c06d8a0 con 0x7f934c0aa8f0 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f93440019f0 con 0x7f934c0aa8f0 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 >> v1:172.21.15.67:6789/0 conn(0x7f934c0aa8f0 legacy=0x7f934c159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9330003a50 0x7f9330003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c0c1d00 0x7f9330004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9330004d90 0x7f9330005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1592e0 con 0x7f9330003a50 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933000bb80 con 0x7f9330004d90 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.797+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933000bd80 con 0x7f934c0c1d00 2024-03-09T08:52:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.798+0000 7f934b400640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9330003a50 0x7f9330003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.798+0000 7f934aa00640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c0c1d00 0x7f9330004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.798+0000 7f934be00640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9330004d90 0x7f9330005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.798+0000 7f934b400640 1 -- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c0c1d00 msgr2=0x7f9330004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.798+0000 7f934b400640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c0c1d00 0x7f9330004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.798+0000 7f934b400640 1 -- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9330004d90 msgr2=0x7f9330005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.798+0000 7f934b400640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9330004d90 0x7f9330005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.798+0000 7f934b400640 1 -- 172.21.15.67:0/301086143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9330003520 con 0x7f9330003a50 2024-03-09T08:52:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.798+0000 7f934aa00640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c0c1d00 0x7f9330004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:29.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.799+0000 7f934b400640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9330003a50 0x7f9330003e70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f933c001750 tx=0x7f9330005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450d34ccb7d7360a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:29.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.799+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c001370 con 0x7f9330003a50 2024-03-09T08:52:29.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.799+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c0052d0 con 0x7f9330003a50 2024-03-09T08:52:29.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.799+0000 7f934a000640 1 -- 172.21.15.67:0/301086143 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c003830 con 0x7f9330003a50 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.800+0000 7f9350c00640 1 -- 172.21.15.67:0/301086143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9330003a50 msgr2=0x7f9330003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.800+0000 7f9350c00640 1 --2- 172.21.15.67:0/301086143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9330003a50 0x7f9330003e70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f933c001750 tx=0x7f9330005830 comp rx=0 tx=0).stop 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.800+0000 7f934be00640 1 -- 172.21.15.67:0/301086143 reap_dead start 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.800+0000 7f9350c00640 1 -- 172.21.15.67:0/301086143 shutdown_connections 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.800+0000 7f9350c00640 1 -- 172.21.15.67:0/301086143 >> 172.21.15.67:0/301086143 conn(0x7f934c0c7300 msgr2=0x7f934c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.800+0000 7f9350c00640 1 -- 172.21.15.67:0/301086143 shutdown_connections 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.800+0000 7f9350c00640 1 -- 172.21.15.67:0/301086143 wait complete. 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.802+0000 7f9350c00640 1 Processor -- start 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.802+0000 7f9350c00640 1 -- start start 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f9350c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9330003a50 0x7f934c0c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f9350c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c1538d0 0x7f934c1eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f9350c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f934c070590 0x7f934c14ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f9350c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0ad2f0 con 0x7f934c070590 2024-03-09T08:52:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f9350c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0ad170 con 0x7f9330003a50 2024-03-09T08:52:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f9350c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0ac200 con 0x7f934c1538d0 2024-03-09T08:52:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f934be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f934c070590 0x7f934c14ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f934be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f934c070590 0x7f934c14ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51190/0 (socket says 172.21.15.67:51190) 2024-03-09T08:52:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f934be00640 1 -- 172.21.15.67:0/1098438615 learned_addr learned my addr 172.21.15.67:0/1098438615 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f934be00640 1 -- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c1538d0 msgr2=0x7f934c1eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f934be00640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c1538d0 0x7f934c1eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f934be00640 1 -- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9330003a50 msgr2=0x7f934c0c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f934be00640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9330003a50 0x7f934c0c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f934be00640 1 -- 172.21.15.67:0/1098438615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c146010 con 0x7f934c070590 2024-03-09T08:52:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.803+0000 7f934be00640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f934c070590 0x7f934c14ab00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f934c06c030 tx=0x7f934400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.804+0000 7f9343400640 1 -- 172.21.15.67:0/1098438615 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344004d60 con 0x7f934c070590 2024-03-09T08:52:29.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.804+0000 7f9343400640 1 -- 172.21.15.67:0/1098438615 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934400bb50 con 0x7f934c070590 2024-03-09T08:52:29.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.804+0000 7f9343400640 1 -- 172.21.15.67:0/1098438615 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344007110 con 0x7f934c070590 2024-03-09T08:52:29.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.804+0000 7f9350c00640 1 -- 172.21.15.67:0/1098438615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c1532c0 con 0x7f934c070590 2024-03-09T08:52:29.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.804+0000 7f9350c00640 1 -- 172.21.15.67:0/1098438615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c145cc0 con 0x7f934c070590 2024-03-09T08:52:29.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.805+0000 7f9350c00640 1 -- 172.21.15.67:0/1098438615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c04f3c0 con 0x7f934c070590 2024-03-09T08:52:29.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.806+0000 7f9343400640 1 -- 172.21.15.67:0/1098438615 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f934401b020 con 0x7f934c070590 2024-03-09T08:52:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.807+0000 7f9343400640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f932806b500 0x7f932806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.807+0000 7f9343400640 1 -- 172.21.15.67:0/1098438615 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f934408a630 con 0x7f934c070590 2024-03-09T08:52:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.807+0000 7f934b400640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f932806b500 0x7f932806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:29.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.808+0000 7f934b400640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f932806b500 0x7f932806d9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f933c033000 tx=0x7f933c02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:29.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:29.811+0000 7f9343400640 1 -- 172.21.15.67:0/1098438615 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9344059170 con 0x7f934c070590 2024-03-09T08:52:30.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.013+0000 7f9350c00640 1 -- 172.21.15.67:0/1098438615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f934c06e320 con 0x7f934c070590 2024-03-09T08:52:30.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.017+0000 7f9343400640 1 -- 172.21.15.67:0/1098438615 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+20320 (secure 0 0 0) 0x7f9344058a70 con 0x7f934c070590 2024-03-09T08:52:30.014 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:30.014 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":23,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:29.036591+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:30.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.021+0000 7f9350c00640 1 -- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f932806b500 msgr2=0x7f932806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.021+0000 7f9350c00640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f932806b500 0x7f932806d9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f933c033000 tx=0x7f933c02f000 comp rx=0 tx=0).stop 2024-03-09T08:52:30.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.021+0000 7f9350c00640 1 -- 172.21.15.67:0/1098438615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f934c070590 msgr2=0x7f934c14ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.021+0000 7f9350c00640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f934c070590 0x7f934c14ab00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f934c06c030 tx=0x7f934400b4a0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.021+0000 7f9350c00640 1 -- 172.21.15.67:0/1098438615 shutdown_connections 2024-03-09T08:52:30.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.021+0000 7f9350c00640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f932806b500 0x7f932806d9c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.021+0000 7f9350c00640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f934c070590 0x7f934c14ab00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.021+0000 7f9350c00640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f934c1538d0 0x7f934c1eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.021+0000 7f9350c00640 1 --2- 172.21.15.67:0/1098438615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9330003a50 0x7f934c0c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.021+0000 7f9350c00640 1 -- 172.21.15.67:0/1098438615 >> 172.21.15.67:0/1098438615 conn(0x7f934c0c7300 msgr2=0x7f934c0c2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.022+0000 7f9350c00640 1 -- 172.21.15.67:0/1098438615 shutdown_connections 2024-03-09T08:52:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.022+0000 7f9350c00640 1 -- 172.21.15.67:0/1098438615 wait complete. 2024-03-09T08:52:30.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:30.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.263+0000 7f85d3e00640 1 Processor -- start 2024-03-09T08:52:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.263+0000 7f85d3e00640 1 -- start start 2024-03-09T08:52:30.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.264+0000 7f85d3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85d40c1f70 0x7f85d4150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:30.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.265+0000 7f85d3e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d40c81e0 con 0x7f85d40710a0 2024-03-09T08:52:30.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.265+0000 7f85d3e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d40c32b0 con 0x7f85d4154080 2024-03-09T08:52:30.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.265+0000 7f85d3e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d40c2fd0 con 0x7f85d40c1f70 2024-03-09T08:52:30.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.265+0000 7f85cbe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f85d40710a0 0x7f85d4153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54364/0 (socket says 172.21.15.67:54364) 2024-03-09T08:52:30.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.265+0000 7f85cbe00640 1 -- 172.21.15.67:0/3864495190 learned_addr learned my addr 172.21.15.67:0/3864495190 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.265+0000 7f85d2a00640 1 --2- 172.21.15.67:0/3864495190 >> v2:172.21.15.67:3300/0 conn(0x7f85d40c1f70 0x7f85d4150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.265+0000 7f85d2000640 1 -- 172.21.15.67:0/3864495190 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840831505 0 0) 0x7f85d40c81e0 con 0x7f85d40710a0 2024-03-09T08:52:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.265+0000 7f85d2000640 1 -- 172.21.15.67:0/3864495190 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b8003520 con 0x7f85d40710a0 2024-03-09T08:52:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.266+0000 7f85d2a00640 1 -- 172.21.15.67:0/3864495190 >> v1:172.21.15.134:6789/0 conn(0x7f85d4154080 legacy=0x7f85d415e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.266+0000 7f85d2a00640 1 -- 172.21.15.67:0/3864495190 >> v1:172.21.15.67:6789/0 conn(0x7f85d40710a0 legacy=0x7f85d4153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.266+0000 7f85d2000640 1 -- 172.21.15.67:0/3864495190 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1964650689 0 0) 0x7f85b8003520 con 0x7f85d40710a0 2024-03-09T08:52:30.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.266+0000 7f85d2a00640 1 -- 172.21.15.67:0/3864495190 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d40c3700 con 0x7f85d40c1f70 2024-03-09T08:52:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.267+0000 7f85d2a00640 1 --2- 172.21.15.67:0/3864495190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d40c1f70 0x7f85d4150c50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f85c40097a0 tx=0x7f85c402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7346054c27220af server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.267+0000 7f85d2000640 1 -- 172.21.15.67:0/3864495190 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4037070 con 0x7f85d40c1f70 2024-03-09T08:52:30.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.267+0000 7f85d2000640 1 -- 172.21.15.67:0/3864495190 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c4030510 con 0x7f85d40c1f70 2024-03-09T08:52:30.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.267+0000 7f85d2000640 1 -- 172.21.15.67:0/3864495190 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc001670 con 0x7f85d40c1f70 2024-03-09T08:52:30.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.268+0000 7f85d3e00640 1 -- 172.21.15.67:0/3864495190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d40c1f70 msgr2=0x7f85d4150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.268+0000 7f85d3e00640 1 --2- 172.21.15.67:0/3864495190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d40c1f70 0x7f85d4150c50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f85c40097a0 tx=0x7f85c402b790 comp rx=0 tx=0).stop 2024-03-09T08:52:30.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.268+0000 7f85d3e00640 1 -- 172.21.15.67:0/3864495190 shutdown_connections 2024-03-09T08:52:30.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.268+0000 7f85d3e00640 1 --2- 172.21.15.67:0/3864495190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d40c1f70 0x7f85d4150c50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.268+0000 7f85d3e00640 1 -- 172.21.15.67:0/3864495190 >> 172.21.15.67:0/3864495190 conn(0x7f85d40c7300 msgr2=0x7f85d40c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.268+0000 7f85d3e00640 1 -- 172.21.15.67:0/3864495190 shutdown_connections 2024-03-09T08:52:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.268+0000 7f85d3e00640 1 -- 172.21.15.67:0/3864495190 wait complete. 2024-03-09T08:52:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.270+0000 7f85d3e00640 1 Processor -- start 2024-03-09T08:52:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.270+0000 7f85d3e00640 1 -- start start 2024-03-09T08:52:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85d3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85d40710a0 0x7f85d414c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85d3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d40c1f70 0x7f85d41579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85d3e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85d4154080 0x7f85d4155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85d3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d4070d50 con 0x7f85d40c1f70 2024-03-09T08:52:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85d3e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d4070830 con 0x7f85d40710a0 2024-03-09T08:52:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85d3e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d4070ed0 con 0x7f85d4154080 2024-03-09T08:52:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d40c1f70 0x7f85d41579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d40c1f70 0x7f85d41579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51210/0 (socket says 172.21.15.67:51210) 2024-03-09T08:52:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85cbe00640 1 -- 172.21.15.67:0/4019807849 learned_addr learned my addr 172.21.15.67:0/4019807849 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:30.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85d2a00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85d40710a0 0x7f85d414c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:30.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85d3400640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85d4154080 0x7f85d4155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:30.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85cbe00640 1 -- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85d4154080 msgr2=0x7f85d4155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.271+0000 7f85cbe00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85d4154080 0x7f85d4155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.272+0000 7f85cbe00640 1 -- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85d40710a0 msgr2=0x7f85d414c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.272+0000 7f85cbe00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85d40710a0 0x7f85d414c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.272+0000 7f85cbe00640 1 -- 172.21.15.67:0/4019807849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d41559a0 con 0x7f85d40c1f70 2024-03-09T08:52:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.272+0000 7f85d3400640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85d4154080 0x7f85d4155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.272+0000 7f85cbe00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d40c1f70 0x7f85d41579e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c700 tx=0x7f85bc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.272+0000 7f85caa00640 1 -- 172.21.15.67:0/4019807849 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc005ad0 con 0x7f85d40c1f70 2024-03-09T08:52:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.272+0000 7f85caa00640 1 -- 172.21.15.67:0/4019807849 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc005c70 con 0x7f85d40c1f70 2024-03-09T08:52:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.272+0000 7f85d3e00640 1 -- 172.21.15.67:0/4019807849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d4155cf0 con 0x7f85d40c1f70 2024-03-09T08:52:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.272+0000 7f85caa00640 1 -- 172.21.15.67:0/4019807849 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc005e60 con 0x7f85d40c1f70 2024-03-09T08:52:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.272+0000 7f85d3e00640 1 -- 172.21.15.67:0/4019807849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d41562b0 con 0x7f85d40c1f70 2024-03-09T08:52:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.273+0000 7f85d3e00640 1 -- 172.21.15.67:0/4019807849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d404f3c0 con 0x7f85d40c1f70 2024-03-09T08:52:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.274+0000 7f85caa00640 1 -- 172.21.15.67:0/4019807849 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f85bc007290 con 0x7f85d40c1f70 2024-03-09T08:52:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.275+0000 7f85caa00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85b006b4d0 0x7f85b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.275+0000 7f85caa00640 1 -- 172.21.15.67:0/4019807849 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f85bc08a2d0 con 0x7f85d40c1f70 2024-03-09T08:52:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.276+0000 7f85d2a00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85b006b4d0 0x7f85b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.277+0000 7f85d2a00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85b006b4d0 0x7f85b006d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f85c4034000 tx=0x7f85c402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.279+0000 7f85caa00640 1 -- 172.21.15.67:0/4019807849 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f85bc058e10 con 0x7f85d40c1f70 2024-03-09T08:52:30.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.474+0000 7f85d3e00640 1 -- 172.21.15.67:0/4019807849 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85d40c7b80 con 0x7f85d40c1f70 2024-03-09T08:52:30.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.478+0000 7f85caa00640 1 -- 172.21.15.67:0/4019807849 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+20320 (secure 0 0 0) 0x7f85bc081e20 con 0x7f85d40c1f70 2024-03-09T08:52:30.474 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:30.474 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":23,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:29.036591+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:30.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.480+0000 7f85d3e00640 1 -- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85b006b4d0 msgr2=0x7f85b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.480+0000 7f85d3e00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85b006b4d0 0x7f85b006d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f85c4034000 tx=0x7f85c402f000 comp rx=0 tx=0).stop 2024-03-09T08:52:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.480+0000 7f85d3e00640 1 -- 172.21.15.67:0/4019807849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d40c1f70 msgr2=0x7f85d41579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.480+0000 7f85d3e00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d40c1f70 0x7f85d41579e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c700 tx=0x7f85bc00cbc0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.480+0000 7f85d3e00640 1 -- 172.21.15.67:0/4019807849 shutdown_connections 2024-03-09T08:52:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.480+0000 7f85d3e00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85b006b4d0 0x7f85b006d990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.480+0000 7f85d3e00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85d4154080 0x7f85d4155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.480+0000 7f85d3e00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85d40c1f70 0x7f85d41579e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.480+0000 7f85d3e00640 1 --2- 172.21.15.67:0/4019807849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85d40710a0 0x7f85d414c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.480+0000 7f85d3e00640 1 -- 172.21.15.67:0/4019807849 >> 172.21.15.67:0/4019807849 conn(0x7f85d40c7300 msgr2=0x7f85d4150a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.481+0000 7f85d3e00640 1 -- 172.21.15.67:0/4019807849 shutdown_connections 2024-03-09T08:52:30.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.481+0000 7f85d3e00640 1 -- 172.21.15.67:0/4019807849 wait complete. 2024-03-09T08:52:30.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b standby_count_wanted 0 2024-03-09T08:52:30.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.776+0000 7f2765600640 1 Processor -- start 2024-03-09T08:52:30.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.776+0000 7f2765600640 1 -- start start 2024-03-09T08:52:30.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.778+0000 7f2765600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2760153390 0x7f2760159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:30.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.778+0000 7f2765600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f27600cb270 0x7f27600cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:30.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.778+0000 7f2765600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276015a0f0 con 0x7f27600c9400 2024-03-09T08:52:30.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.778+0000 7f2765600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2760155e50 con 0x7f27600cb270 2024-03-09T08:52:30.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.778+0000 7f2765600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27600c7480 con 0x7f2760153390 2024-03-09T08:52:30.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.778+0000 7f275f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f27600c9400 0x7f27601562e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54372/0 (socket says 172.21.15.67:54372) 2024-03-09T08:52:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.778+0000 7f275f400640 1 -- 172.21.15.67:0/1244443696 learned_addr learned my addr 172.21.15.67:0/1244443696 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.778+0000 7f275fe00640 1 --2- 172.21.15.67:0/1244443696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2760153390 0x7f2760159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.778+0000 7f2764c00640 1 --2- 172.21.15.67:0/1244443696 >> v2:172.21.15.134:3300/0 conn(0x7f27600cb270 0x7f27600cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.779+0000 7f275ea00640 1 -- 172.21.15.67:0/1244443696 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907395574 0 0) 0x7f276015a0f0 con 0x7f27600c9400 2024-03-09T08:52:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.779+0000 7f275ea00640 1 -- 172.21.15.67:0/1244443696 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2744003520 con 0x7f27600c9400 2024-03-09T08:52:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.779+0000 7f275fe00640 1 -- 172.21.15.67:0/1244443696 >> v2:172.21.15.134:3300/0 conn(0x7f27600cb270 msgr2=0x7f27600cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.779+0000 7f275fe00640 1 --2- 172.21.15.67:0/1244443696 >> v2:172.21.15.134:3300/0 conn(0x7f27600cb270 0x7f27600cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.779+0000 7f275fe00640 1 -- 172.21.15.67:0/1244443696 >> v1:172.21.15.67:6789/0 conn(0x7f27600c9400 legacy=0x7f27601562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.779+0000 7f275ea00640 1 -- 172.21.15.67:0/1244443696 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3097118875 0 0) 0x7f2744003520 con 0x7f27600c9400 2024-03-09T08:52:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.779+0000 7f275fe00640 1 -- 172.21.15.67:0/1244443696 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27600c7760 con 0x7f2760153390 2024-03-09T08:52:30.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.780+0000 7f275fe00640 1 --2- 172.21.15.67:0/1244443696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2760153390 0x7f2760159bb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f27500097a0 tx=0x7f275002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db26b14025e6af6e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:30.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.781+0000 7f275ea00640 1 -- 172.21.15.67:0/1244443696 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750037070 con 0x7f2760153390 2024-03-09T08:52:30.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.781+0000 7f275ea00640 1 -- 172.21.15.67:0/1244443696 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2750030510 con 0x7f2760153390 2024-03-09T08:52:30.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.781+0000 7f275ea00640 1 -- 172.21.15.67:0/1244443696 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c001670 con 0x7f2760153390 2024-03-09T08:52:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.781+0000 7f2765600640 1 -- 172.21.15.67:0/1244443696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2760153390 msgr2=0x7f2760159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.781+0000 7f2765600640 1 --2- 172.21.15.67:0/1244443696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2760153390 0x7f2760159bb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f27500097a0 tx=0x7f275002b790 comp rx=0 tx=0).stop 2024-03-09T08:52:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.782+0000 7f2765600640 1 -- 172.21.15.67:0/1244443696 shutdown_connections 2024-03-09T08:52:30.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.782+0000 7f2765600640 1 --2- 172.21.15.67:0/1244443696 >> v2:172.21.15.134:3300/0 conn(0x7f27600cb270 0x7f27600cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.782+0000 7f2765600640 1 --2- 172.21.15.67:0/1244443696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2760153390 0x7f2760159bb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.782+0000 7f2765600640 1 -- 172.21.15.67:0/1244443696 >> 172.21.15.67:0/1244443696 conn(0x7f27600ad190 msgr2=0x7f27600ad5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.782+0000 7f2765600640 1 -- 172.21.15.67:0/1244443696 shutdown_connections 2024-03-09T08:52:30.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.782+0000 7f2765600640 1 -- 172.21.15.67:0/1244443696 wait complete. 2024-03-09T08:52:30.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.783+0000 7f2765600640 1 Processor -- start 2024-03-09T08:52:30.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.784+0000 7f2765600640 1 -- start start 2024-03-09T08:52:30.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.784+0000 7f2765600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27600c9400 0x7f27601e0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:30.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.784+0000 7f2765600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27600cb270 0x7f27601518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:30.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.784+0000 7f2765600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2760153390 0x7f27601ec210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:30.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.784+0000 7f2765600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601569f0 con 0x7f2760153390 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.784+0000 7f2765600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276006d6c0 con 0x7f27600c9400 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.784+0000 7f2765600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2760156b70 con 0x7f27600cb270 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.785+0000 7f2764c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2760153390 0x7f27601ec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.785+0000 7f2764c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2760153390 0x7f27601ec210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51222/0 (socket says 172.21.15.67:51222) 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.785+0000 7f2764c00640 1 -- 172.21.15.67:0/3353007676 learned_addr learned my addr 172.21.15.67:0/3353007676 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.785+0000 7f275f400640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27600cb270 0x7f27601518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.785+0000 7f275fe00640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27600c9400 0x7f27601e0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.785+0000 7f2764c00640 1 -- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27600cb270 msgr2=0x7f27601518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.785+0000 7f2764c00640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27600cb270 0x7f27601518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.785+0000 7f2764c00640 1 -- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27600c9400 msgr2=0x7f27601e0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.785+0000 7f2764c00640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27600c9400 0x7f27601e0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.785+0000 7f2764c00640 1 -- 172.21.15.67:0/3353007676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27601eba50 con 0x7f2760153390 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.786+0000 7f2764c00640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2760153390 0x7f27601ec210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2758009dd0 tx=0x7f275800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:30.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.786+0000 7f2757e00640 1 -- 172.21.15.67:0/3353007676 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758018020 con 0x7f2760153390 2024-03-09T08:52:30.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.786+0000 7f2757e00640 1 -- 172.21.15.67:0/3353007676 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2758014480 con 0x7f2760153390 2024-03-09T08:52:30.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.786+0000 7f2757e00640 1 -- 172.21.15.67:0/3353007676 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758014870 con 0x7f2760153390 2024-03-09T08:52:30.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.786+0000 7f2765600640 1 -- 172.21.15.67:0/3353007676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27601efa60 con 0x7f2760153390 2024-03-09T08:52:30.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.786+0000 7f2765600640 1 -- 172.21.15.67:0/3353007676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2760151e10 con 0x7f2760153390 2024-03-09T08:52:30.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.787+0000 7f2765600640 1 -- 172.21.15.67:0/3353007676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27600c68c0 con 0x7f2760153390 2024-03-09T08:52:30.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.788+0000 7f2757e00640 1 -- 172.21.15.67:0/3353007676 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2758005d10 con 0x7f2760153390 2024-03-09T08:52:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.789+0000 7f2757e00640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f274006b4d0 0x7f274006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:30.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.790+0000 7f2757e00640 1 -- 172.21.15.67:0/3353007676 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f275808b330 con 0x7f2760153390 2024-03-09T08:52:30.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.790+0000 7f275fe00640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f274006b4d0 0x7f274006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:30.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.791+0000 7f275fe00640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f274006b4d0 0x7f274006d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2750034000 tx=0x7f275002f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:30.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:30.794+0000 7f2757e00640 1 -- 172.21.15.67:0/3353007676 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f27600c68c0 con 0x7f2760153390 2024-03-09T08:52:31.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.040+0000 7f2765600640 1 -- 172.21.15.67:0/3353007676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f276004f3c0 con 0x7f2760153390 2024-03-09T08:52:31.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.590+0000 7f2757e00640 1 -- 172.21.15.67:0/3353007676 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 113+0+0 (secure 0 0 0) 0x7f2758022030 con 0x7f2760153390 2024-03-09T08:52:31.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.593+0000 7f2765600640 1 -- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f274006b4d0 msgr2=0x7f274006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:31.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.593+0000 7f2765600640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f274006b4d0 0x7f274006d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2750034000 tx=0x7f275002f000 comp rx=0 tx=0).stop 2024-03-09T08:52:31.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.593+0000 7f2765600640 1 -- 172.21.15.67:0/3353007676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2760153390 msgr2=0x7f27601ec210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:31.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.593+0000 7f2765600640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2760153390 0x7f27601ec210 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2758009dd0 tx=0x7f275800c490 comp rx=0 tx=0).stop 2024-03-09T08:52:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.594+0000 7f2765600640 1 -- 172.21.15.67:0/3353007676 shutdown_connections 2024-03-09T08:52:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.594+0000 7f2765600640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f274006b4d0 0x7f274006d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.594+0000 7f2765600640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2760153390 0x7f27601ec210 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:31.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.594+0000 7f2765600640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27600cb270 0x7f27601518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.594+0000 7f2765600640 1 --2- 172.21.15.67:0/3353007676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27600c9400 0x7f27601e0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.594+0000 7f2765600640 1 -- 172.21.15.67:0/3353007676 >> 172.21.15.67:0/3353007676 conn(0x7f27600ad190 msgr2=0x7f27600b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.594+0000 7f2765600640 1 -- 172.21.15.67:0/3353007676 shutdown_connections 2024-03-09T08:52:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.594+0000 7f2765600640 1 -- 172.21.15.67:0/3353007676 wait complete. 2024-03-09T08:52:31.611 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-03-09T08:52:31.611 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b allow_standby_replay false 2024-03-09T08:52:31.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.832+0000 7f09c2800640 1 Processor -- start 2024-03-09T08:52:31.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.832+0000 7f09c2800640 1 -- start start 2024-03-09T08:52:31.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.834+0000 7f09c2800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f09bc152f20 0x7f09bc153380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:31.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.834+0000 7f09c2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f09bc0ca2c0 0x7f09bc0ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:31.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.834+0000 7f09c2800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09bc155d70 con 0x7f09bc0c8190 2024-03-09T08:52:31.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.834+0000 7f09c2800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09bc156c40 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.834+0000 7f09c2800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09bc156030 con 0x7f09bc152f20 2024-03-09T08:52:31.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.834+0000 7f09c1400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f09bc0c8190 0x7f09bc0c9640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54386/0 (socket says 172.21.15.67:54386) 2024-03-09T08:52:31.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.834+0000 7f09c1400640 1 -- 172.21.15.67:0/244576971 learned_addr learned my addr 172.21.15.67:0/244576971 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:31.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.834+0000 7f09c1e00640 1 --2- 172.21.15.67:0/244576971 >> v2:172.21.15.67:3300/0 conn(0x7f09bc0ca2c0 0x7f09bc0ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:31.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.834+0000 7f09c0a00640 1 --2- 172.21.15.67:0/244576971 >> v2:172.21.15.134:3300/0 conn(0x7f09bc152f20 0x7f09bc153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:31.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.835+0000 7f09bbe00640 1 -- 172.21.15.67:0/244576971 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667880464 0 0) 0x7f09bc155d70 con 0x7f09bc0c8190 2024-03-09T08:52:31.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.835+0000 7f09bbe00640 1 -- 172.21.15.67:0/244576971 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09a4003520 con 0x7f09bc0c8190 2024-03-09T08:52:31.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.835+0000 7f09c1e00640 1 -- 172.21.15.67:0/244576971 >> v2:172.21.15.134:3300/0 conn(0x7f09bc152f20 msgr2=0x7f09bc153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:31.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.835+0000 7f09c1e00640 1 --2- 172.21.15.67:0/244576971 >> v2:172.21.15.134:3300/0 conn(0x7f09bc152f20 0x7f09bc153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:31.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.835+0000 7f09c1e00640 1 -- 172.21.15.67:0/244576971 >> v1:172.21.15.67:6789/0 conn(0x7f09bc0c8190 legacy=0x7f09bc0c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:31.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.835+0000 7f09c1e00640 1 -- 172.21.15.67:0/244576971 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc156310 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.837+0000 7f09c1e00640 1 --2- 172.21.15.67:0/244576971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09bc0ca2c0 0x7f09bc0ca6f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0ae0a0 tx=0x7f09b402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fe5f50ba507847 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:31.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.837+0000 7f09bbe00640 1 -- 172.21.15.67:0/244576971 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4039070 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.837+0000 7f09bbe00640 1 -- 172.21.15.67:0/244576971 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b402cbe0 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.837+0000 7f09bbe00640 1 -- 172.21.15.67:0/244576971 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4034440 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.838+0000 7f09c2800640 1 -- 172.21.15.67:0/244576971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09bc0ca2c0 msgr2=0x7f09bc0ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:31.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.838+0000 7f09c2800640 1 --2- 172.21.15.67:0/244576971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09bc0ca2c0 0x7f09bc0ca6f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0ae0a0 tx=0x7f09b402be30 comp rx=0 tx=0).stop 2024-03-09T08:52:31.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.838+0000 7f09c2800640 1 -- 172.21.15.67:0/244576971 shutdown_connections 2024-03-09T08:52:31.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.838+0000 7f09c2800640 1 --2- 172.21.15.67:0/244576971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09bc0ca2c0 0x7f09bc0ca6f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:31.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.838+0000 7f09c2800640 1 --2- 172.21.15.67:0/244576971 >> v2:172.21.15.134:3300/0 conn(0x7f09bc152f20 0x7f09bc153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:31.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.838+0000 7f09c2800640 1 -- 172.21.15.67:0/244576971 >> 172.21.15.67:0/244576971 conn(0x7f09bc0ad060 msgr2=0x7f09bc0ad440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:31.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.838+0000 7f09c2800640 1 -- 172.21.15.67:0/244576971 shutdown_connections 2024-03-09T08:52:31.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.838+0000 7f09c2800640 1 -- 172.21.15.67:0/244576971 wait complete. 2024-03-09T08:52:31.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.840+0000 7f09c2800640 1 Processor -- start 2024-03-09T08:52:31.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.840+0000 7f09c2800640 1 -- start start 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.841+0000 7f09c2800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09bc0c8190 0x7f09bc1ec080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.841+0000 7f09c2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09bc0ca2c0 0x7f09bc157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.841+0000 7f09c2800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09bc152f20 0x7f09bc1eea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.841+0000 7f09c2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc06c9d0 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.841+0000 7f09c2800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc1568d0 con 0x7f09bc152f20 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.841+0000 7f09c2800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc06cb50 con 0x7f09bc0c8190 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.841+0000 7f09c0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09bc0ca2c0 0x7f09bc157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.841+0000 7f09c0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09bc0ca2c0 0x7f09bc157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51248/0 (socket says 172.21.15.67:51248) 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.841+0000 7f09c0a00640 1 -- 172.21.15.67:0/3666021591 learned_addr learned my addr 172.21.15.67:0/3666021591 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.841+0000 7f09c1e00640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09bc152f20 0x7f09bc1eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.841+0000 7f09c1400640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09bc0c8190 0x7f09bc1ec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.842+0000 7f09c0a00640 1 -- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09bc0c8190 msgr2=0x7f09bc1ec080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.842+0000 7f09c0a00640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09bc0c8190 0x7f09bc1ec080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.842+0000 7f09c0a00640 1 -- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09bc152f20 msgr2=0x7f09bc1eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.842+0000 7f09c0a00640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09bc152f20 0x7f09bc1eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.842+0000 7f09c0a00640 1 -- 172.21.15.67:0/3666021591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc1eb610 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.842+0000 7f09c0a00640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09bc0ca2c0 0x7f09bc157bc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f09a800e870 tx=0x7f09a800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.842+0000 7f09b9600640 1 -- 172.21.15.67:0/3666021591 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8017070 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.842+0000 7f09b9600640 1 -- 172.21.15.67:0/3666021591 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a800c940 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.842+0000 7f09b9600640 1 -- 172.21.15.67:0/3666021591 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800cd30 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.842+0000 7f09c2800640 1 -- 172.21.15.67:0/3666021591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09bc152640 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.842+0000 7f09c2800640 1 -- 172.21.15.67:0/3666021591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc1ebb60 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.844+0000 7f09c2800640 1 -- 172.21.15.67:0/3666021591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09bc04f3c0 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.844+0000 7f09b9600640 1 -- 172.21.15.67:0/3666021591 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f09a801a070 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.845+0000 7f09b9600640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f099c06b4d0 0x7f099c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.845+0000 7f09b9600640 1 -- 172.21.15.67:0/3666021591 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f09a808a3b0 con 0x7f09bc0ca2c0 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.846+0000 7f09c1400640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f099c06b4d0 0x7f099c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.847+0000 7f09c1400640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f099c06b4d0 0x7f099c06d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f09b0007fd0 tx=0x7f09b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:31.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:31.850+0000 7f09b9600640 1 -- 172.21.15.67:0/3666021591 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f09a8058ef0 con 0x7f09bc0ca2c0 2024-03-09T08:52:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.090+0000 7f09c2800640 1 -- 172.21.15.67:0/3666021591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f09bc0c93b0 con 0x7f09bc0ca2c0 2024-03-09T08:52:32.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.599+0000 7f09b9600640 1 -- 172.21.15.67:0/3666021591 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 117+0+0 (secure 0 0 0) 0x7f09a80587f0 con 0x7f09bc0ca2c0 2024-03-09T08:52:32.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.603+0000 7f09c2800640 1 -- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f099c06b4d0 msgr2=0x7f099c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:32.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.603+0000 7f09c2800640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f099c06b4d0 0x7f099c06d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f09b0007fd0 tx=0x7f09b0009040 comp rx=0 tx=0).stop 2024-03-09T08:52:32.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.604+0000 7f09c2800640 1 -- 172.21.15.67:0/3666021591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09bc0ca2c0 msgr2=0x7f09bc157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:32.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.604+0000 7f09c2800640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09bc0ca2c0 0x7f09bc157bc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f09a800e870 tx=0x7f09a800ed30 comp rx=0 tx=0).stop 2024-03-09T08:52:32.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.604+0000 7f09c2800640 1 -- 172.21.15.67:0/3666021591 shutdown_connections 2024-03-09T08:52:32.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.604+0000 7f09c2800640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f099c06b4d0 0x7f099c06d990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:32.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.604+0000 7f09c2800640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09bc152f20 0x7f09bc1eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:32.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.604+0000 7f09c2800640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09bc0ca2c0 0x7f09bc157bc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:32.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.604+0000 7f09c2800640 1 --2- 172.21.15.67:0/3666021591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09bc0c8190 0x7f09bc1ec080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:32.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.604+0000 7f09c2800640 1 -- 172.21.15.67:0/3666021591 >> 172.21.15.67:0/3666021591 conn(0x7f09bc0ad060 msgr2=0x7f09bc14dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:32.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.604+0000 7f09c2800640 1 -- 172.21.15.67:0/3666021591 shutdown_connections 2024-03-09T08:52:32.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.604+0000 7f09c2800640 1 -- 172.21.15.67:0/3666021591 wait complete. 2024-03-09T08:52:32.635 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b session_timeout 300 2024-03-09T08:52:32.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.857+0000 7f7293e00640 1 Processor -- start 2024-03-09T08:52:32.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.857+0000 7f7293e00640 1 -- start start 2024-03-09T08:52:32.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.858+0000 7f7293e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f72940c5f10 0x7f72940c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:32.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.858+0000 7f7293e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f72940c9490 0x7f72940cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:32.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.858+0000 7f7293e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72940c9a30 con 0x7f729406b040 2024-03-09T08:52:32.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.858+0000 7f7293e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72940c9d10 con 0x7f72940c9490 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.858+0000 7f7293e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72940ca340 con 0x7f72940c5f10 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.858+0000 7f7292000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f729406b040 0x7f72941567d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54402/0 (socket says 172.21.15.67:54402) 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.859+0000 7f7292000640 1 -- 172.21.15.67:0/1465922051 learned_addr learned my addr 172.21.15.67:0/1465922051 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.859+0000 7f7293400640 1 --2- 172.21.15.67:0/1465922051 >> v2:172.21.15.67:3300/0 conn(0x7f72940c9490 0x7f72940cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.859+0000 7f7292a00640 1 --2- 172.21.15.67:0/1465922051 >> v2:172.21.15.134:3300/0 conn(0x7f72940c5f10 0x7f72940c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.859+0000 7f7291600640 1 -- 172.21.15.67:0/1465922051 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264000464 0 0) 0x7f72940c9a30 con 0x7f729406b040 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.859+0000 7f7291600640 1 -- 172.21.15.67:0/1465922051 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7278003520 con 0x7f729406b040 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.859+0000 7f7293400640 1 -- 172.21.15.67:0/1465922051 >> v2:172.21.15.134:3300/0 conn(0x7f72940c5f10 msgr2=0x7f72940c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.859+0000 7f7293400640 1 --2- 172.21.15.67:0/1465922051 >> v2:172.21.15.134:3300/0 conn(0x7f72940c5f10 0x7f72940c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.859+0000 7f7293400640 1 -- 172.21.15.67:0/1465922051 >> v1:172.21.15.67:6789/0 conn(0x7f729406b040 legacy=0x7f72941567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.859+0000 7f7291600640 1 -- 172.21.15.67:0/1465922051 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2225905636 0 0) 0x7f7278003520 con 0x7f729406b040 2024-03-09T08:52:32.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.860+0000 7f7293400640 1 -- 172.21.15.67:0/1465922051 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72940ca620 con 0x7f72940c9490 2024-03-09T08:52:32.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.861+0000 7f7293400640 1 --2- 172.21.15.67:0/1465922051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72940c9490 0x7f72940cab40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f729408d2e0 tx=0x7f728c02bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6dddda2e48f42e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:32.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.862+0000 7f7291600640 1 -- 172.21.15.67:0/1465922051 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c039070 con 0x7f72940c9490 2024-03-09T08:52:32.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.862+0000 7f7291600640 1 -- 172.21.15.67:0/1465922051 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c02c830 con 0x7f72940c9490 2024-03-09T08:52:32.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.862+0000 7f7291600640 1 -- 172.21.15.67:0/1465922051 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280001670 con 0x7f72940c9490 2024-03-09T08:52:32.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.862+0000 7f7293e00640 1 -- 172.21.15.67:0/1465922051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72940c9490 msgr2=0x7f72940cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:32.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.862+0000 7f7293e00640 1 --2- 172.21.15.67:0/1465922051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72940c9490 0x7f72940cab40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f729408d2e0 tx=0x7f728c02bd00 comp rx=0 tx=0).stop 2024-03-09T08:52:32.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.862+0000 7f7293e00640 1 -- 172.21.15.67:0/1465922051 shutdown_connections 2024-03-09T08:52:32.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.862+0000 7f7293e00640 1 --2- 172.21.15.67:0/1465922051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72940c9490 0x7f72940cab40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:32.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.862+0000 7f7293e00640 1 --2- 172.21.15.67:0/1465922051 >> v2:172.21.15.134:3300/0 conn(0x7f72940c5f10 0x7f72940c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:32.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.862+0000 7f7293e00640 1 -- 172.21.15.67:0/1465922051 >> 172.21.15.67:0/1465922051 conn(0x7f72940cb590 msgr2=0x7f72940cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:32.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.862+0000 7f7293e00640 1 -- 172.21.15.67:0/1465922051 shutdown_connections 2024-03-09T08:52:32.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.863+0000 7f7293e00640 1 -- 172.21.15.67:0/1465922051 wait complete. 2024-03-09T08:52:32.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.864+0000 7f7293e00640 1 Processor -- start 2024-03-09T08:52:32.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.864+0000 7f7293e00640 1 -- start start 2024-03-09T08:52:32.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.865+0000 7f7293e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f729406b040 0x7f729414b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:32.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.865+0000 7f7293e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72940c5f10 0x7f72941ef210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:32.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.865+0000 7f7293e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72940c9490 0x7f72940c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:32.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.865+0000 7f7293e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72941537a0 con 0x7f72940c5f10 2024-03-09T08:52:32.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.865+0000 7f7293e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7294153620 con 0x7f729406b040 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.865+0000 7f7293e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72940cb080 con 0x7f72940c9490 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.865+0000 7f7292000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72940c5f10 0x7f72941ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.866+0000 7f7292000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72940c5f10 0x7f72941ef210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51260/0 (socket says 172.21.15.67:51260) 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.866+0000 7f7292000640 1 -- 172.21.15.67:0/4086776518 learned_addr learned my addr 172.21.15.67:0/4086776518 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.866+0000 7f7292a00640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f729406b040 0x7f729414b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.866+0000 7f7293400640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72940c9490 0x7f72940c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.866+0000 7f7292000640 1 -- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72940c9490 msgr2=0x7f72940c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.866+0000 7f7292000640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72940c9490 0x7f72940c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.866+0000 7f7292000640 1 -- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f729406b040 msgr2=0x7f729414b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.866+0000 7f7292000640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f729406b040 0x7f729414b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.866+0000 7f7292000640 1 -- 172.21.15.67:0/4086776518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72941efcc0 con 0x7f72940c5f10 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.866+0000 7f7292000640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72940c5f10 0x7f72941ef210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f728000c760 tx=0x7f728000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.866+0000 7f728aa00640 1 -- 172.21.15.67:0/4086776518 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280017070 con 0x7f72940c5f10 2024-03-09T08:52:32.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.867+0000 7f728aa00640 1 -- 172.21.15.67:0/4086776518 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7280005ad0 con 0x7f72940c5f10 2024-03-09T08:52:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.867+0000 7f728aa00640 1 -- 172.21.15.67:0/4086776518 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280007070 con 0x7f72940c5f10 2024-03-09T08:52:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.867+0000 7f7293e00640 1 -- 172.21.15.67:0/4086776518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72941effa0 con 0x7f72940c5f10 2024-03-09T08:52:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.867+0000 7f7293e00640 1 -- 172.21.15.67:0/4086776518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72940c30c0 con 0x7f72940c5f10 2024-03-09T08:52:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.868+0000 7f7293e00640 1 -- 172.21.15.67:0/4086776518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72940678c0 con 0x7f72940c5f10 2024-03-09T08:52:32.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.869+0000 7f728aa00640 1 -- 172.21.15.67:0/4086776518 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f728001a070 con 0x7f72940c5f10 2024-03-09T08:52:32.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.870+0000 7f728aa00640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f727406b4d0 0x7f727406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:32.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.870+0000 7f728aa00640 1 -- 172.21.15.67:0/4086776518 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f7280089a70 con 0x7f72940c5f10 2024-03-09T08:52:32.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.871+0000 7f7292a00640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f727406b4d0 0x7f727406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:32.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.872+0000 7f7292a00640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f727406b4d0 0x7f727406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7284009ad0 tx=0x7f7284008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:32.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:32.874+0000 7f728aa00640 1 -- 172.21.15.67:0/4086776518 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f72940678c0 con 0x7f72940c5f10 2024-03-09T08:52:33.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.149+0000 7f7293e00640 1 -- 172.21.15.67:0/4086776518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f72940cae70 con 0x7f72940c5f10 2024-03-09T08:52:33.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.609+0000 7f728aa00640 1 -- 172.21.15.67:0/4086776518 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "session_timeout", "val": "300"}]=0 v14) v1 ==== 110+0+0 (secure 0 0 0) 0x7f72940cae70 con 0x7f72940c5f10 2024-03-09T08:52:33.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 -- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f727406b4d0 msgr2=0x7f727406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:33.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f727406b4d0 0x7f727406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7284009ad0 tx=0x7f7284008040 comp rx=0 tx=0).stop 2024-03-09T08:52:33.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 -- 172.21.15.67:0/4086776518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72940c5f10 msgr2=0x7f72941ef210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:33.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72940c5f10 0x7f72941ef210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f728000c760 tx=0x7f728000cc20 comp rx=0 tx=0).stop 2024-03-09T08:52:33.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 -- 172.21.15.67:0/4086776518 shutdown_connections 2024-03-09T08:52:33.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f727406b4d0 0x7f727406d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:33.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72940c9490 0x7f72940c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:33.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72940c5f10 0x7f72941ef210 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:33.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 --2- 172.21.15.67:0/4086776518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f729406b040 0x7f729414b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:33.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 -- 172.21.15.67:0/4086776518 >> 172.21.15.67:0/4086776518 conn(0x7f72940cb590 msgr2=0x7f729406c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:33.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 -- 172.21.15.67:0/4086776518 shutdown_connections 2024-03-09T08:52:33.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.612+0000 7f7293e00640 1 -- 172.21.15.67:0/4086776518 wait complete. 2024-03-09T08:52:33.634 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'b' 2024-03-09T08:52:33.634 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T08:52:33.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.871+0000 7f0021600640 1 Processor -- start 2024-03-09T08:52:33.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.871+0000 7f0021600640 1 -- start start 2024-03-09T08:52:33.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.873+0000 7f0021600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001c0c5e40 0x7f001c156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:33.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.873+0000 7f0021600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f001c0cadb0 0x7f001c156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:33.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.873+0000 7f0021600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001c156a90 con 0x7f001c153300 2024-03-09T08:52:33.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.873+0000 7f0021600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001c157000 con 0x7f001c0cadb0 2024-03-09T08:52:33.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.873+0000 7f0021600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001c0ae430 con 0x7f001c0c5e40 2024-03-09T08:52:33.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.874+0000 7f001b400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001c0c5e40 0x7f001c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:33.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.874+0000 7f001be00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f001c153300 0x7f001c0b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:60878/0 (socket says 172.21.15.67:60878) 2024-03-09T08:52:33.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.874+0000 7f001be00640 1 -- 172.21.15.67:0/490353875 learned_addr learned my addr 172.21.15.67:0/490353875 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:33.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.874+0000 7f0020c00640 1 --2- 172.21.15.67:0/490353875 >> v2:172.21.15.134:3300/0 conn(0x7f001c0cadb0 0x7f001c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.875+0000 7f001aa00640 1 -- 172.21.15.67:0/490353875 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329406409 0 0) 0x7f001c156a90 con 0x7f001c153300 2024-03-09T08:52:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.875+0000 7f001b400640 1 -- 172.21.15.67:0/490353875 >> v2:172.21.15.134:3300/0 conn(0x7f001c0cadb0 msgr2=0x7f001c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.875+0000 7f001b400640 1 --2- 172.21.15.67:0/490353875 >> v2:172.21.15.134:3300/0 conn(0x7f001c0cadb0 0x7f001c156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.875+0000 7f001b400640 1 -- 172.21.15.67:0/490353875 >> v1:172.21.15.134:6789/0 conn(0x7f001c153300 legacy=0x7f001c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.875+0000 7f001b400640 1 -- 172.21.15.67:0/490353875 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c0ae6f0 con 0x7f001c0c5e40 2024-03-09T08:52:33.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.876+0000 7f001b400640 1 --2- 172.21.15.67:0/490353875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001c0c5e40 0x7f001c156010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f00080097a0 tx=0x7f000802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b84ba141fb4021 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:33.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.876+0000 7f001aa00640 1 -- 172.21.15.67:0/490353875 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008035020 con 0x7f001c0c5e40 2024-03-09T08:52:33.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.877+0000 7f001aa00640 1 -- 172.21.15.67:0/490353875 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0008031810 con 0x7f001c0c5e40 2024-03-09T08:52:33.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.877+0000 7f001aa00640 1 -- 172.21.15.67:0/490353875 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008031be0 con 0x7f001c0c5e40 2024-03-09T08:52:33.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.877+0000 7f0021600640 1 -- 172.21.15.67:0/490353875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001c0c5e40 msgr2=0x7f001c156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:33.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.877+0000 7f0021600640 1 --2- 172.21.15.67:0/490353875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001c0c5e40 0x7f001c156010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f00080097a0 tx=0x7f000802b8b0 comp rx=0 tx=0).stop 2024-03-09T08:52:33.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.877+0000 7f0021600640 1 -- 172.21.15.67:0/490353875 shutdown_connections 2024-03-09T08:52:33.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.877+0000 7f0021600640 1 --2- 172.21.15.67:0/490353875 >> v2:172.21.15.134:3300/0 conn(0x7f001c0cadb0 0x7f001c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:33.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.877+0000 7f0021600640 1 --2- 172.21.15.67:0/490353875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001c0c5e40 0x7f001c156010 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:33.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.877+0000 7f0021600640 1 -- 172.21.15.67:0/490353875 >> 172.21.15.67:0/490353875 conn(0x7f001c0732d0 msgr2=0x7f001c0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:33.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.877+0000 7f0021600640 1 -- 172.21.15.67:0/490353875 shutdown_connections 2024-03-09T08:52:33.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.877+0000 7f0021600640 1 -- 172.21.15.67:0/490353875 wait complete. 2024-03-09T08:52:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.879+0000 7f0021600640 1 Processor -- start 2024-03-09T08:52:33.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.879+0000 7f0021600640 1 -- start start 2024-03-09T08:52:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.880+0000 7f0021600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c0c5e40 0x7f001c149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.880+0000 7f0021600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cadb0 0x7f001c0c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:33.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.880+0000 7f0021600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001c153300 0x7f001c0c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.880+0000 7f0021600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c159e90 con 0x7f001c0cadb0 2024-03-09T08:52:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.880+0000 7f0021600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c159bb0 con 0x7f001c0c5e40 2024-03-09T08:52:33.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.880+0000 7f0021600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001c0c59e0 con 0x7f001c153300 2024-03-09T08:52:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.880+0000 7f001b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cadb0 0x7f001c0c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.880+0000 7f001b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cadb0 0x7f001c0c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51270/0 (socket says 172.21.15.67:51270) 2024-03-09T08:52:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.880+0000 7f001b400640 1 -- 172.21.15.67:0/1888413004 learned_addr learned my addr 172.21.15.67:0/1888413004 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:33.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.881+0000 7f0020c00640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001c153300 0x7f001c0c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.881+0000 7f001be00640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c0c5e40 0x7f001c149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.881+0000 7f001b400640 1 -- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001c153300 msgr2=0x7f001c0c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.881+0000 7f001b400640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001c153300 0x7f001c0c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.881+0000 7f001b400640 1 -- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c0c5e40 msgr2=0x7f001c149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.881+0000 7f001b400640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c0c5e40 0x7f001c149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.881+0000 7f001b400640 1 -- 172.21.15.67:0/1888413004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c155500 con 0x7f001c0cadb0 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.881+0000 7f001be00640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c0c5e40 0x7f001c149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.881+0000 7f001b400640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cadb0 0x7f001c0c7fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f00080097a0 tx=0x7f00080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.882+0000 7f0013e00640 1 -- 172.21.15.67:0/1888413004 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008041070 con 0x7f001c0cadb0 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.882+0000 7f0013e00640 1 -- 172.21.15.67:0/1888413004 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000803ac80 con 0x7f001c0cadb0 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.882+0000 7f0021600640 1 -- 172.21.15.67:0/1888413004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c1ef7b0 con 0x7f001c0cadb0 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.882+0000 7f0013e00640 1 -- 172.21.15.67:0/1888413004 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008035040 con 0x7f001c0cadb0 2024-03-09T08:52:33.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.882+0000 7f0021600640 1 -- 172.21.15.67:0/1888413004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c1557c0 con 0x7f001c0cadb0 2024-03-09T08:52:33.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.883+0000 7f0021600640 1 -- 172.21.15.67:0/1888413004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c04f3c0 con 0x7f001c0cadb0 2024-03-09T08:52:33.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.884+0000 7f0013e00640 1 -- 172.21.15.67:0/1888413004 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0008038020 con 0x7f001c0cadb0 2024-03-09T08:52:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.885+0000 7f0013e00640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efffc06b4d0 0x7efffc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:33.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.885+0000 7f0013e00640 1 -- 172.21.15.67:0/1888413004 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f00080ac100 con 0x7f001c0cadb0 2024-03-09T08:52:33.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.885+0000 7f001be00640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efffc06b4d0 0x7efffc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:33.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.886+0000 7f001be00640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efffc06b4d0 0x7efffc06d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f000c000f60 tx=0x7f000c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:33.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:33.889+0000 7f0013e00640 1 -- 172.21.15.67:0/1888413004 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f000807abb0 con 0x7f001c0cadb0 2024-03-09T08:52:34.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.128+0000 7f0021600640 1 -- 172.21.15.67:0/1888413004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f001c0cab90 con 0x7f001c0cadb0 2024-03-09T08:52:34.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.129+0000 7f0013e00640 1 -- 172.21.15.67:0/1888413004 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6247 (secure 0 0 0) 0x7f000807a4b0 con 0x7f001c0cadb0 2024-03-09T08:52:34.125 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:34.125 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:17.736973+0000","modified":"2024-03-09T08:52:21.931108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T08:52:34.125 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 14 2024-03-09T08:52:34.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.131+0000 7f0021600640 1 -- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efffc06b4d0 msgr2=0x7efffc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.131+0000 7f0021600640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efffc06b4d0 0x7efffc06d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f000c000f60 tx=0x7f000c009040 comp rx=0 tx=0).stop 2024-03-09T08:52:34.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.131+0000 7f0021600640 1 -- 172.21.15.67:0/1888413004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cadb0 msgr2=0x7f001c0c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.131+0000 7f0021600640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cadb0 0x7f001c0c7fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f00080097a0 tx=0x7f00080046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.132+0000 7f0021600640 1 -- 172.21.15.67:0/1888413004 shutdown_connections 2024-03-09T08:52:34.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.132+0000 7f0021600640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efffc06b4d0 0x7efffc06d990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.132+0000 7f0021600640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f001c153300 0x7f001c0c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.132+0000 7f0021600640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f001c0cadb0 0x7f001c0c7fb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.132+0000 7f0021600640 1 --2- 172.21.15.67:0/1888413004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f001c0c5e40 0x7f001c149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.132+0000 7f0021600640 1 -- 172.21.15.67:0/1888413004 >> 172.21.15.67:0/1888413004 conn(0x7f001c0732d0 msgr2=0x7f001c156010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:34.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.132+0000 7f0021600640 1 -- 172.21.15.67:0/1888413004 shutdown_connections 2024-03-09T08:52:34.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.132+0000 7f0021600640 1 -- 172.21.15.67:0/1888413004 wait complete. 2024-03-09T08:52:34.141 INFO:tasks.ceph.mds.c.smithi067.stderr:2024-03-09T08:52:34.145+0000 7f88b5800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-09T08:52:34.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:34.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.353+0000 7f7a1d000640 1 Processor -- start 2024-03-09T08:52:34.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.353+0000 7f7a1d000640 1 -- start start 2024-03-09T08:52:34.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.355+0000 7f7a1d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7a18153300 0x7f7a180b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.355+0000 7f7a1d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a180cadb0 0x7f7a180b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.355+0000 7f7a1d000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a18157000 con 0x7f7a180c5e40 2024-03-09T08:52:34.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.355+0000 7f7a1d000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a180ae430 con 0x7f7a18153300 2024-03-09T08:52:34.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.355+0000 7f7a1d000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a180ae6f0 con 0x7f7a180cadb0 2024-03-09T08:52:34.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.355+0000 7f7a17400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7a18153300 0x7f7a180b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.355+0000 7f7a17400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7a18153300 0x7f7a180b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51286/0 (socket says 172.21.15.67:51286) 2024-03-09T08:52:34.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.355+0000 7f7a17400640 1 -- 172.21.15.67:0/2063076900 learned_addr learned my addr 172.21.15.67:0/2063076900 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:34.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.355+0000 7f7a17e00640 1 --2- 172.21.15.67:0/2063076900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a180cadb0 0x7f7a180b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.356+0000 7f7a17400640 1 -- 172.21.15.67:0/2063076900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a180cadb0 msgr2=0x7f7a180b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:34.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.356+0000 7f7a17400640 1 --2- 172.21.15.67:0/2063076900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a180cadb0 0x7f7a180b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.356+0000 7f7a17400640 1 -- 172.21.15.67:0/2063076900 >> v1:172.21.15.134:6789/0 conn(0x7f7a180c5e40 legacy=0x7f7a18156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.356+0000 7f7a17400640 1 -- 172.21.15.67:0/2063076900 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a18155b10 con 0x7f7a18153300 2024-03-09T08:52:34.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.357+0000 7f7a17400640 1 --2- 172.21.15.67:0/2063076900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a18153300 0x7f7a180b0000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009870 tx=0x7f7a0802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269efefc86719567 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:34.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.357+0000 7f7a16000640 1 -- 172.21.15.67:0/2063076900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08034070 con 0x7f7a18153300 2024-03-09T08:52:34.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.357+0000 7f7a16000640 1 -- 172.21.15.67:0/2063076900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a08030840 con 0x7f7a18153300 2024-03-09T08:52:34.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.357+0000 7f7a16000640 1 -- 172.21.15.67:0/2063076900 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08030c10 con 0x7f7a18153300 2024-03-09T08:52:34.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.358+0000 7f7a1d000640 1 -- 172.21.15.67:0/2063076900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a18153300 msgr2=0x7f7a180b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.358+0000 7f7a1d000640 1 --2- 172.21.15.67:0/2063076900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a18153300 0x7f7a180b0000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009870 tx=0x7f7a0802b980 comp rx=0 tx=0).stop 2024-03-09T08:52:34.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.358+0000 7f7a1d000640 1 -- 172.21.15.67:0/2063076900 shutdown_connections 2024-03-09T08:52:34.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.358+0000 7f7a1d000640 1 --2- 172.21.15.67:0/2063076900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a180cadb0 0x7f7a180b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.358+0000 7f7a1d000640 1 --2- 172.21.15.67:0/2063076900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a18153300 0x7f7a180b0000 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.358+0000 7f7a1d000640 1 -- 172.21.15.67:0/2063076900 >> 172.21.15.67:0/2063076900 conn(0x7f7a180732d0 msgr2=0x7f7a180ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:34.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.358+0000 7f7a1d000640 1 -- 172.21.15.67:0/2063076900 shutdown_connections 2024-03-09T08:52:34.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.359+0000 7f7a1d000640 1 -- 172.21.15.67:0/2063076900 wait complete. 2024-03-09T08:52:34.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.360+0000 7f7a1d000640 1 Processor -- start 2024-03-09T08:52:34.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.360+0000 7f7a1d000640 1 -- start start 2024-03-09T08:52:34.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.361+0000 7f7a1d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a180c5e40 0x7f7a1806b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.361+0000 7f7a1d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a180cadb0 0x7f7a180ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.361+0000 7f7a1d000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a18153300 0x7f7a1806bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.361+0000 7f7a1d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a180c59e0 con 0x7f7a180c5e40 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.361+0000 7f7a1d000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a18159e90 con 0x7f7a18153300 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.361+0000 7f7a1d000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a18156720 con 0x7f7a180cadb0 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.361+0000 7f7a17400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a180c5e40 0x7f7a1806b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.361+0000 7f7a17400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a180c5e40 0x7f7a1806b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51302/0 (socket says 172.21.15.67:51302) 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.361+0000 7f7a17400640 1 -- 172.21.15.67:0/2537417561 learned_addr learned my addr 172.21.15.67:0/2537417561 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.362+0000 7f7a17e00640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a18153300 0x7f7a1806bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.362+0000 7f7a16a00640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a180cadb0 0x7f7a180ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.362+0000 7f7a17400640 1 -- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a180cadb0 msgr2=0x7f7a180ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.362+0000 7f7a17400640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a180cadb0 0x7f7a180ab940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.362+0000 7f7a17400640 1 -- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a18153300 msgr2=0x7f7a1806bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.362+0000 7f7a17400640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a18153300 0x7f7a1806bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.362+0000 7f7a17400640 1 -- 172.21.15.67:0/2537417561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a180acf10 con 0x7f7a180c5e40 2024-03-09T08:52:34.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.362+0000 7f7a17e00640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a18153300 0x7f7a1806bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:34.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.362+0000 7f7a17400640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a180c5e40 0x7f7a1806b6a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7a08000c00 tx=0x7f7a080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:34.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.363+0000 7f7a0f400640 1 -- 172.21.15.67:0/2537417561 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08034040 con 0x7f7a180c5e40 2024-03-09T08:52:34.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.363+0000 7f7a0f400640 1 -- 172.21.15.67:0/2537417561 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a08039d90 con 0x7f7a180c5e40 2024-03-09T08:52:34.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.363+0000 7f7a1d000640 1 -- 172.21.15.67:0/2537417561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1806cb70 con 0x7f7a180c5e40 2024-03-09T08:52:34.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.363+0000 7f7a0f400640 1 -- 172.21.15.67:0/2537417561 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08043630 con 0x7f7a180c5e40 2024-03-09T08:52:34.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.363+0000 7f7a1d000640 1 -- 172.21.15.67:0/2537417561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1806ce10 con 0x7f7a180c5e40 2024-03-09T08:52:34.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.364+0000 7f7a1d000640 1 -- 172.21.15.67:0/2537417561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a180682c0 con 0x7f7a180c5e40 2024-03-09T08:52:34.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.365+0000 7f7a0f400640 1 -- 172.21.15.67:0/2537417561 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7a0804a060 con 0x7f7a180c5e40 2024-03-09T08:52:34.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.366+0000 7f7a0f400640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79f806b4d0 0x7f79f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.367+0000 7f7a0f400640 1 -- 172.21.15.67:0/2537417561 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f7a080ac510 con 0x7f7a180c5e40 2024-03-09T08:52:34.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.367+0000 7f7a16a00640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79f806b4d0 0x7f79f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.368+0000 7f7a16a00640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79f806b4d0 0x7f79f806d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a18156340 tx=0x7f7a04009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:34.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.371+0000 7f7a0f400640 1 -- 172.21.15.67:0/2537417561 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7a180682c0 con 0x7f7a180c5e40 2024-03-09T08:52:34.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.563+0000 7f7a1d000640 1 -- 172.21.15.67:0/2537417561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a180c9090 con 0x7f7a180c5e40 2024-03-09T08:52:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.565+0000 7f7a0f400640 1 -- 172.21.15.67:0/2537417561 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20307 (secure 0 0 0) 0x7f7a180c9090 con 0x7f7a180c5e40 2024-03-09T08:52:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:34.561 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:30.565529+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.567+0000 7f7a1d000640 1 -- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79f806b4d0 msgr2=0x7f79f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.567+0000 7f7a1d000640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79f806b4d0 0x7f79f806d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7a18156340 tx=0x7f7a04009040 comp rx=0 tx=0).stop 2024-03-09T08:52:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.567+0000 7f7a1d000640 1 -- 172.21.15.67:0/2537417561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a180c5e40 msgr2=0x7f7a1806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.567+0000 7f7a1d000640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a180c5e40 0x7f7a1806b6a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7a08000c00 tx=0x7f7a080046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.568+0000 7f7a1d000640 1 -- 172.21.15.67:0/2537417561 shutdown_connections 2024-03-09T08:52:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.568+0000 7f7a1d000640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79f806b4d0 0x7f79f806d990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.568+0000 7f7a1d000640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a18153300 0x7f7a1806bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.568+0000 7f7a1d000640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a180cadb0 0x7f7a180ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.568+0000 7f7a1d000640 1 --2- 172.21.15.67:0/2537417561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a180c5e40 0x7f7a1806b6a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.568+0000 7f7a1d000640 1 -- 172.21.15.67:0/2537417561 >> 172.21.15.67:0/2537417561 conn(0x7f7a180732d0 msgr2=0x7f7a180c92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.568+0000 7f7a1d000640 1 -- 172.21.15.67:0/2537417561 shutdown_connections 2024-03-09T08:52:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.568+0000 7f7a1d000640 1 -- 172.21.15.67:0/2537417561 wait complete. 2024-03-09T08:52:34.581 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-09T08:52:34.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T08:52:34.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.793+0000 7fa247200640 1 Processor -- start 2024-03-09T08:52:34.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.793+0000 7fa247200640 1 -- start start 2024-03-09T08:52:34.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.794+0000 7fa247200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa2480c5e40 0x7fa248156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.794+0000 7fa247200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa2480cadb0 0x7fa248156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.794+0000 7fa247200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa248156a90 con 0x7fa248153300 2024-03-09T08:52:34.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.794+0000 7fa247200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa248157000 con 0x7fa2480c5e40 2024-03-09T08:52:34.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.794+0000 7fa247200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2480ae430 con 0x7fa2480cadb0 2024-03-09T08:52:34.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.795+0000 7fa245e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa248153300 0x7fa2480b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54404/0 (socket says 172.21.15.67:54404) 2024-03-09T08:52:34.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.795+0000 7fa245e00640 1 -- 172.21.15.67:0/1540759646 learned_addr learned my addr 172.21.15.67:0/1540759646 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:34.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.795+0000 7fa23d400640 1 --2- 172.21.15.67:0/1540759646 >> v2:172.21.15.67:3300/0 conn(0x7fa2480c5e40 0x7fa248156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.795+0000 7fa246800640 1 --2- 172.21.15.67:0/1540759646 >> v2:172.21.15.134:3300/0 conn(0x7fa2480cadb0 0x7fa248156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.795+0000 7fa23d400640 1 -- 172.21.15.67:0/1540759646 >> v2:172.21.15.134:3300/0 conn(0x7fa2480cadb0 msgr2=0x7fa248156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.795+0000 7fa23d400640 1 --2- 172.21.15.67:0/1540759646 >> v2:172.21.15.134:3300/0 conn(0x7fa2480cadb0 0x7fa248156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.795+0000 7fa23d400640 1 -- 172.21.15.67:0/1540759646 >> v1:172.21.15.67:6789/0 conn(0x7fa248153300 legacy=0x7fa2480b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.795+0000 7fa23d400640 1 -- 172.21.15.67:0/1540759646 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2480ae6f0 con 0x7fa2480c5e40 2024-03-09T08:52:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.795+0000 7fa246800640 1 --2- 172.21.15.67:0/1540759646 >> v2:172.21.15.134:3300/0 conn(0x7fa2480cadb0 0x7fa248156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.796+0000 7fa23d400640 1 --2- 172.21.15.67:0/1540759646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2480c5e40 0x7fa248156010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa2480adc60 tx=0x7fa23002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f470395015e0275 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.796+0000 7fa245400640 1 -- 172.21.15.67:0/1540759646 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230035020 con 0x7fa2480c5e40 2024-03-09T08:52:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.796+0000 7fa245400640 1 -- 172.21.15.67:0/1540759646 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa230031830 con 0x7fa2480c5e40 2024-03-09T08:52:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.796+0000 7fa245400640 1 -- 172.21.15.67:0/1540759646 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230031c00 con 0x7fa2480c5e40 2024-03-09T08:52:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.797+0000 7fa247200640 1 -- 172.21.15.67:0/1540759646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2480c5e40 msgr2=0x7fa248156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.797+0000 7fa247200640 1 --2- 172.21.15.67:0/1540759646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2480c5e40 0x7fa248156010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa2480adc60 tx=0x7fa23002b8d0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.797+0000 7fa247200640 1 -- 172.21.15.67:0/1540759646 shutdown_connections 2024-03-09T08:52:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.797+0000 7fa247200640 1 --2- 172.21.15.67:0/1540759646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2480c5e40 0x7fa248156010 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.797+0000 7fa247200640 1 --2- 172.21.15.67:0/1540759646 >> v2:172.21.15.134:3300/0 conn(0x7fa2480cadb0 0x7fa248156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.797+0000 7fa247200640 1 -- 172.21.15.67:0/1540759646 >> 172.21.15.67:0/1540759646 conn(0x7fa2480732d0 msgr2=0x7fa2480ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.797+0000 7fa247200640 1 -- 172.21.15.67:0/1540759646 shutdown_connections 2024-03-09T08:52:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.797+0000 7fa247200640 1 -- 172.21.15.67:0/1540759646 wait complete. 2024-03-09T08:52:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.799+0000 7fa247200640 1 Processor -- start 2024-03-09T08:52:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.799+0000 7fa247200640 1 -- start start 2024-03-09T08:52:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa247200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2480c5e40 0x7fa24806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa247200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2480cadb0 0x7fa2480c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa247200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa248153300 0x7fa2481ef160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa247200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa248159e90 con 0x7fa248153300 2024-03-09T08:52:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa247200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa248159bb0 con 0x7fa2480cadb0 2024-03-09T08:52:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa247200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2480c59e0 con 0x7fa2480c5e40 2024-03-09T08:52:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa246800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa248153300 0x7fa2481ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa246800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa248153300 0x7fa2481ef160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51322/0 (socket says 172.21.15.67:51322) 2024-03-09T08:52:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa246800640 1 -- 172.21.15.67:0/348881979 learned_addr learned my addr 172.21.15.67:0/348881979 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa245e00640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2480c5e40 0x7fa24806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa246800640 1 -- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2480c5e40 msgr2=0x7fa24806bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa246800640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2480c5e40 0x7fa24806bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa246800640 1 -- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2480cadb0 msgr2=0x7fa2480c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa23d400640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2480cadb0 0x7fa2480c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.800+0000 7fa246800640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2480cadb0 0x7fa2480c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.801+0000 7fa246800640 1 -- 172.21.15.67:0/348881979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24814c260 con 0x7fa248153300 2024-03-09T08:52:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.801+0000 7fa246800640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa248153300 0x7fa2481ef160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa24000c820 tx=0x7fa24000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.801+0000 7fa23ea00640 1 -- 172.21.15.67:0/348881979 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240015070 con 0x7fa248153300 2024-03-09T08:52:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.801+0000 7fa23ea00640 1 -- 172.21.15.67:0/348881979 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa240007a30 con 0x7fa248153300 2024-03-09T08:52:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.801+0000 7fa23ea00640 1 -- 172.21.15.67:0/348881979 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240007e20 con 0x7fa248153300 2024-03-09T08:52:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.801+0000 7fa247200640 1 -- 172.21.15.67:0/348881979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24814bf80 con 0x7fa248153300 2024-03-09T08:52:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.801+0000 7fa247200640 1 -- 172.21.15.67:0/348881979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2480c8230 con 0x7fa248153300 2024-03-09T08:52:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.802+0000 7fa247200640 1 -- 172.21.15.67:0/348881979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2480af140 con 0x7fa248153300 2024-03-09T08:52:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.803+0000 7fa23ea00640 1 -- 172.21.15.67:0/348881979 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa240018070 con 0x7fa248153300 2024-03-09T08:52:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.805+0000 7fa23ea00640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa22406b500 0x7fa22406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.805+0000 7fa23ea00640 1 -- 172.21.15.67:0/348881979 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fa24008a3e0 con 0x7fa248153300 2024-03-09T08:52:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.805+0000 7fa245e00640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa22406b500 0x7fa22406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.806+0000 7fa245e00640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa22406b500 0x7fa22406d9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa2480ca500 tx=0x7fa238009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:34.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:34.809+0000 7fa23ea00640 1 -- 172.21.15.67:0/348881979 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa2480af140 con 0x7fa248153300 2024-03-09T08:52:35.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.004+0000 7fa247200640 1 -- 172.21.15.67:0/348881979 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa24804f3c0 con 0x7fa22406b500 2024-03-09T08:52:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.044+0000 7fa23ea00640 1 -- 172.21.15.67:0/348881979 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490417 (secure 0 0 0) 0x7fa24804f3c0 con 0x7fa22406b500 2024-03-09T08:52:35.043 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T08:52:35.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.053+0000 7fa247200640 1 -- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa22406b500 msgr2=0x7fa22406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.053+0000 7fa247200640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa22406b500 0x7fa22406d9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa2480ca500 tx=0x7fa238009040 comp rx=0 tx=0).stop 2024-03-09T08:52:35.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.053+0000 7fa247200640 1 -- 172.21.15.67:0/348881979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa248153300 msgr2=0x7fa2481ef160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.053+0000 7fa247200640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa248153300 0x7fa2481ef160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa24000c820 tx=0x7fa24000cce0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.053+0000 7fa247200640 1 -- 172.21.15.67:0/348881979 shutdown_connections 2024-03-09T08:52:35.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.053+0000 7fa247200640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa22406b500 0x7fa22406d9c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.053+0000 7fa247200640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa248153300 0x7fa2481ef160 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.053+0000 7fa247200640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2480cadb0 0x7fa2480c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.053+0000 7fa247200640 1 --2- 172.21.15.67:0/348881979 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2480c5e40 0x7fa24806bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.054+0000 7fa247200640 1 -- 172.21.15.67:0/348881979 >> 172.21.15.67:0/348881979 conn(0x7fa2480732d0 msgr2=0x7fa2481514f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:35.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.054+0000 7fa247200640 1 -- 172.21.15.67:0/348881979 shutdown_connections 2024-03-09T08:52:35.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.054+0000 7fa247200640 1 -- 172.21.15.67:0/348881979 wait complete. 2024-03-09T08:52:35.071 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-03-09T08:52:34.679036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":323,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3350528,"data_stored":1984464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.129679"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575471+0000","last_change":"2024-03-09T08:52:29.573571+0000","last_active":"2024-03-09T08:52:30.575471+0000","last_peered":"2024-03-09T08:52:30.575471+0000","last_clean":"2024-03-09T08:52:30.575471+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T08:52:30.575471+0000","last_undegraded":"2024-03-09T08:52:30.575471+0000","last_fullsized":"2024-03-09T08:52:30.575471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:33:33.068463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598024+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T08:52:30.598024+0000","last_peered":"2024-03-09T08:52:30.598024+0000","last_clean":"2024-03-09T08:52:30.598024+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T08:52:30.598024+0000","last_undegraded":"2024-03-09T08:52:30.598024+0000","last_fullsized":"2024-03-09T08:52:30.598024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:41:40.332687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577727+0000","last_change":"2024-03-09T08:52:18.235099+0000","last_active":"2024-03-09T08:52:30.577727+0000","last_peered":"2024-03-09T08:52:30.577727+0000","last_clean":"2024-03-09T08:52:30.577727+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T08:52:30.577727+0000","last_undegraded":"2024-03-09T08:52:30.577727+0000","last_fullsized":"2024-03-09T08:52:30.577727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:39:28.882461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575426+0000","last_change":"2024-03-09T08:52:28.571905+0000","last_active":"2024-03-09T08:52:30.575426+0000","last_peered":"2024-03-09T08:52:30.575426+0000","last_clean":"2024-03-09T08:52:30.575426+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T08:52:30.575426+0000","last_undegraded":"2024-03-09T08:52:30.575426+0000","last_fullsized":"2024-03-09T08:52:30.575426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:01.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.600985+0000","last_change":"2024-03-09T08:52:29.578644+0000","last_active":"2024-03-09T08:52:30.600985+0000","last_peered":"2024-03-09T08:52:30.600985+0000","last_clean":"2024-03-09T08:52:30.600985+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T08:52:30.600985+0000","last_undegraded":"2024-03-09T08:52:30.600985+0000","last_fullsized":"2024-03-09T08:52:30.600985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:00.044620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577762+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T08:52:30.577762+0000","last_peered":"2024-03-09T08:52:30.577762+0000","last_clean":"2024-03-09T08:52:30.577762+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T08:52:30.577762+0000","last_undegraded":"2024-03-09T08:52:30.577762+0000","last_fullsized":"2024-03-09T08:52:30.577762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597762+0000","last_change":"2024-03-09T08:52:18.236946+0000","last_active":"2024-03-09T08:52:30.597762+0000","last_peered":"2024-03-09T08:52:30.597762+0000","last_clean":"2024-03-09T08:52:30.597762+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T08:52:30.597762+0000","last_undegraded":"2024-03-09T08:52:30.597762+0000","last_fullsized":"2024-03-09T08:52:30.597762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:45.372160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575237+0000","last_change":"2024-03-09T08:52:28.573074+0000","last_active":"2024-03-09T08:52:30.575237+0000","last_peered":"2024-03-09T08:52:30.575237+0000","last_clean":"2024-03-09T08:52:30.575237+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T08:52:30.575237+0000","last_undegraded":"2024-03-09T08:52:30.575237+0000","last_fullsized":"2024-03-09T08:52:30.575237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:50.600408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945199+0000","last_change":"2024-03-09T08:52:29.594030+0000","last_active":"2024-03-09T08:52:30.945199+0000","last_peered":"2024-03-09T08:52:30.945199+0000","last_clean":"2024-03-09T08:52:30.945199+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T08:52:30.945199+0000","last_undegraded":"2024-03-09T08:52:30.945199+0000","last_fullsized":"2024-03-09T08:52:30.945199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:57.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948441+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T08:52:30.948441+0000","last_peered":"2024-03-09T08:52:30.948441+0000","last_clean":"2024-03-09T08:52:30.948441+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T08:52:30.948441+0000","last_undegraded":"2024-03-09T08:52:30.948441+0000","last_fullsized":"2024-03-09T08:52:30.948441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947481+0000","last_change":"2024-03-09T08:52:18.230629+0000","last_active":"2024-03-09T08:52:30.947481+0000","last_peered":"2024-03-09T08:52:30.947481+0000","last_clean":"2024-03-09T08:52:30.947481+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T08:52:30.947481+0000","last_undegraded":"2024-03-09T08:52:30.947481+0000","last_fullsized":"2024-03-09T08:52:30.947481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:46:38.528839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576909+0000","last_change":"2024-03-09T08:52:28.577128+0000","last_active":"2024-03-09T08:52:30.576909+0000","last_peered":"2024-03-09T08:52:30.576909+0000","last_clean":"2024-03-09T08:52:30.576909+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T08:52:30.576909+0000","last_undegraded":"2024-03-09T08:52:30.576909+0000","last_fullsized":"2024-03-09T08:52:30.576909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:41.801752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946024+0000","last_change":"2024-03-09T08:52:29.581705+0000","last_active":"2024-03-09T08:52:30.946024+0000","last_peered":"2024-03-09T08:52:30.946024+0000","last_clean":"2024-03-09T08:52:30.946024+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T08:52:30.946024+0000","last_undegraded":"2024-03-09T08:52:30.946024+0000","last_fullsized":"2024-03-09T08:52:30.946024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:27.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947791+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T08:52:30.947791+0000","last_peered":"2024-03-09T08:52:30.947791+0000","last_clean":"2024-03-09T08:52:30.947791+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T08:52:30.947791+0000","last_undegraded":"2024-03-09T08:52:30.947791+0000","last_fullsized":"2024-03-09T08:52:30.947791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947372+0000","last_change":"2024-03-09T08:52:18.228419+0000","last_active":"2024-03-09T08:52:30.947372+0000","last_peered":"2024-03-09T08:52:30.947372+0000","last_clean":"2024-03-09T08:52:30.947372+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T08:52:30.947372+0000","last_undegraded":"2024-03-09T08:52:30.947372+0000","last_fullsized":"2024-03-09T08:52:30.947372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:19:12.533402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577499+0000","last_change":"2024-03-09T08:52:28.577069+0000","last_active":"2024-03-09T08:52:30.577499+0000","last_peered":"2024-03-09T08:52:30.577499+0000","last_clean":"2024-03-09T08:52:30.577499+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T08:52:30.577499+0000","last_undegraded":"2024-03-09T08:52:30.577499+0000","last_fullsized":"2024-03-09T08:52:30.577499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:09:02.410915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601446+0000","last_change":"2024-03-09T08:52:29.588922+0000","last_active":"2024-03-09T08:52:30.601446+0000","last_peered":"2024-03-09T08:52:30.601446+0000","last_clean":"2024-03-09T08:52:30.601446+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T08:52:30.601446+0000","last_undegraded":"2024-03-09T08:52:30.601446+0000","last_fullsized":"2024-03-09T08:52:30.601446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:13.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576353+0000","last_change":"2024-03-09T08:52:18.234372+0000","last_active":"2024-03-09T08:52:30.576353+0000","last_peered":"2024-03-09T08:52:30.576353+0000","last_clean":"2024-03-09T08:52:30.576353+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T08:52:30.576353+0000","last_undegraded":"2024-03-09T08:52:30.576353+0000","last_fullsized":"2024-03-09T08:52:30.576353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:09:33.422907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944406+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T08:52:30.944406+0000","last_peered":"2024-03-09T08:52:30.944406+0000","last_clean":"2024-03-09T08:52:30.944406+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T08:52:30.944406+0000","last_undegraded":"2024-03-09T08:52:30.944406+0000","last_fullsized":"2024-03-09T08:52:30.944406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578381+0000","last_change":"2024-03-09T08:52:28.570631+0000","last_active":"2024-03-09T08:52:30.578381+0000","last_peered":"2024-03-09T08:52:30.578381+0000","last_clean":"2024-03-09T08:52:30.578381+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T08:52:30.578381+0000","last_undegraded":"2024-03-09T08:52:30.578381+0000","last_fullsized":"2024-03-09T08:52:30.578381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:06.630704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943131+0000","last_change":"2024-03-09T08:52:29.596820+0000","last_active":"2024-03-09T08:52:30.943131+0000","last_peered":"2024-03-09T08:52:30.943131+0000","last_clean":"2024-03-09T08:52:30.943131+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T08:52:30.943131+0000","last_undegraded":"2024-03-09T08:52:30.943131+0000","last_fullsized":"2024-03-09T08:52:30.943131+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:01:20.331889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576713+0000","last_change":"2024-03-09T08:52:18.244306+0000","last_active":"2024-03-09T08:52:30.576713+0000","last_peered":"2024-03-09T08:52:30.576713+0000","last_clean":"2024-03-09T08:52:30.576713+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T08:52:30.576713+0000","last_undegraded":"2024-03-09T08:52:30.576713+0000","last_fullsized":"2024-03-09T08:52:30.576713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:58.365524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601528+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T08:52:30.601528+0000","last_peered":"2024-03-09T08:52:30.601528+0000","last_clean":"2024-03-09T08:52:30.601528+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T08:52:30.601528+0000","last_undegraded":"2024-03-09T08:52:30.601528+0000","last_fullsized":"2024-03-09T08:52:30.601528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:45:45.612953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576699+0000","last_change":"2024-03-09T08:52:28.577110+0000","last_active":"2024-03-09T08:52:30.576699+0000","last_peered":"2024-03-09T08:52:30.576699+0000","last_clean":"2024-03-09T08:52:30.576699+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T08:52:30.576699+0000","last_undegraded":"2024-03-09T08:52:30.576699+0000","last_fullsized":"2024-03-09T08:52:30.576699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:57.197373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579257+0000","last_change":"2024-03-09T08:52:29.579929+0000","last_active":"2024-03-09T08:52:30.579257+0000","last_peered":"2024-03-09T08:52:30.579257+0000","last_clean":"2024-03-09T08:52:30.579257+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T08:52:30.579257+0000","last_undegraded":"2024-03-09T08:52:30.579257+0000","last_fullsized":"2024-03-09T08:52:30.579257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:37:23.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576526+0000","last_change":"2024-03-09T08:52:18.233208+0000","last_active":"2024-03-09T08:52:30.576526+0000","last_peered":"2024-03-09T08:52:30.576526+0000","last_clean":"2024-03-09T08:52:30.576526+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T08:52:30.576526+0000","last_undegraded":"2024-03-09T08:52:30.576526+0000","last_fullsized":"2024-03-09T08:52:30.576526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:17.589783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601269+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T08:52:30.601269+0000","last_peered":"2024-03-09T08:52:30.601269+0000","last_clean":"2024-03-09T08:52:30.601269+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T08:52:30.601269+0000","last_undegraded":"2024-03-09T08:52:30.601269+0000","last_fullsized":"2024-03-09T08:52:30.601269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:21:59.990396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602237+0000","last_change":"2024-03-09T08:52:28.576254+0000","last_active":"2024-03-09T08:52:30.602237+0000","last_peered":"2024-03-09T08:52:30.602237+0000","last_clean":"2024-03-09T08:52:30.602237+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T08:52:30.602237+0000","last_undegraded":"2024-03-09T08:52:30.602237+0000","last_fullsized":"2024-03-09T08:52:30.602237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:21:56.330955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946687+0000","last_change":"2024-03-09T08:52:29.578272+0000","last_active":"2024-03-09T08:52:30.946687+0000","last_peered":"2024-03-09T08:52:30.946687+0000","last_clean":"2024-03-09T08:52:30.946687+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T08:52:30.946687+0000","last_undegraded":"2024-03-09T08:52:30.946687+0000","last_fullsized":"2024-03-09T08:52:30.946687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:56.180809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576191+0000","last_change":"2024-03-09T08:52:18.217046+0000","last_active":"2024-03-09T08:52:30.576191+0000","last_peered":"2024-03-09T08:52:30.576191+0000","last_clean":"2024-03-09T08:52:30.576191+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T08:52:30.576191+0000","last_undegraded":"2024-03-09T08:52:30.576191+0000","last_fullsized":"2024-03-09T08:52:30.576191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:14.884963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943757+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T08:52:30.943757+0000","last_peered":"2024-03-09T08:52:30.943757+0000","last_clean":"2024-03-09T08:52:30.943757+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T08:52:30.943757+0000","last_undegraded":"2024-03-09T08:52:30.943757+0000","last_fullsized":"2024-03-09T08:52:30.943757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578428+0000","last_change":"2024-03-09T08:52:28.573638+0000","last_active":"2024-03-09T08:52:30.578428+0000","last_peered":"2024-03-09T08:52:30.578428+0000","last_clean":"2024-03-09T08:52:30.578428+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T08:52:30.578428+0000","last_undegraded":"2024-03-09T08:52:30.578428+0000","last_fullsized":"2024-03-09T08:52:30.578428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:02.478112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575514+0000","last_change":"2024-03-09T08:52:29.597697+0000","last_active":"2024-03-09T08:52:30.575514+0000","last_peered":"2024-03-09T08:52:30.575514+0000","last_clean":"2024-03-09T08:52:30.575514+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T08:52:30.575514+0000","last_undegraded":"2024-03-09T08:52:30.575514+0000","last_fullsized":"2024-03-09T08:52:30.575514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:00.890845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598142+0000","last_change":"2024-03-09T08:52:18.229739+0000","last_active":"2024-03-09T08:52:30.598142+0000","last_peered":"2024-03-09T08:52:30.598142+0000","last_clean":"2024-03-09T08:52:30.598142+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T08:52:30.598142+0000","last_undegraded":"2024-03-09T08:52:30.598142+0000","last_fullsized":"2024-03-09T08:52:30.598142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:24.394779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579139+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T08:52:30.579139+0000","last_peered":"2024-03-09T08:52:30.579139+0000","last_clean":"2024-03-09T08:52:30.579139+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T08:52:30.579139+0000","last_undegraded":"2024-03-09T08:52:30.579139+0000","last_fullsized":"2024-03-09T08:52:30.579139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:35:38.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575645+0000","last_change":"2024-03-09T08:52:28.575630+0000","last_active":"2024-03-09T08:52:30.575645+0000","last_peered":"2024-03-09T08:52:30.575645+0000","last_clean":"2024-03-09T08:52:30.575645+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T08:52:30.575645+0000","last_undegraded":"2024-03-09T08:52:30.575645+0000","last_fullsized":"2024-03-09T08:52:30.575645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:48.210921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575362+0000","last_change":"2024-03-09T08:52:29.576677+0000","last_active":"2024-03-09T08:52:30.575362+0000","last_peered":"2024-03-09T08:52:30.575362+0000","last_clean":"2024-03-09T08:52:30.575362+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T08:52:30.575362+0000","last_undegraded":"2024-03-09T08:52:30.575362+0000","last_fullsized":"2024-03-09T08:52:30.575362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:19.026873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577342+0000","last_change":"2024-03-09T08:52:18.245442+0000","last_active":"2024-03-09T08:52:30.577342+0000","last_peered":"2024-03-09T08:52:30.577342+0000","last_clean":"2024-03-09T08:52:30.577342+0000","last_became_active":"2024-03-09T08:52:18.245164+0000","last_became_peered":"2024-03-09T08:52:18.245164+0000","last_unstale":"2024-03-09T08:52:30.577342+0000","last_undegraded":"2024-03-09T08:52:30.577342+0000","last_fullsized":"2024-03-09T08:52:30.577342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:31:08.413041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602103+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T08:52:30.602103+0000","last_peered":"2024-03-09T08:52:30.602103+0000","last_clean":"2024-03-09T08:52:30.602103+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T08:52:30.602103+0000","last_undegraded":"2024-03-09T08:52:30.602103+0000","last_fullsized":"2024-03-09T08:52:30.602103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:30.201471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575392+0000","last_change":"2024-03-09T08:52:28.586702+0000","last_active":"2024-03-09T08:52:30.575392+0000","last_peered":"2024-03-09T08:52:30.575392+0000","last_clean":"2024-03-09T08:52:30.575392+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T08:52:30.575392+0000","last_undegraded":"2024-03-09T08:52:30.575392+0000","last_fullsized":"2024-03-09T08:52:30.575392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:09.201224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577654+0000","last_change":"2024-03-09T08:52:29.578956+0000","last_active":"2024-03-09T08:52:30.577654+0000","last_peered":"2024-03-09T08:52:30.577654+0000","last_clean":"2024-03-09T08:52:30.577654+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T08:52:30.577654+0000","last_undegraded":"2024-03-09T08:52:30.577654+0000","last_fullsized":"2024-03-09T08:52:30.577654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:00:00.884211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601530+0000","last_change":"2024-03-09T08:52:18.234887+0000","last_active":"2024-03-09T08:52:30.601530+0000","last_peered":"2024-03-09T08:52:30.601530+0000","last_clean":"2024-03-09T08:52:30.601530+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T08:52:30.601530+0000","last_undegraded":"2024-03-09T08:52:30.601530+0000","last_fullsized":"2024-03-09T08:52:30.601530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:44.661882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948828+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T08:52:30.948828+0000","last_peered":"2024-03-09T08:52:30.948828+0000","last_clean":"2024-03-09T08:52:30.948828+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T08:52:30.948828+0000","last_undegraded":"2024-03-09T08:52:30.948828+0000","last_fullsized":"2024-03-09T08:52:30.948828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597558+0000","last_change":"2024-03-09T08:52:28.580879+0000","last_active":"2024-03-09T08:52:30.597558+0000","last_peered":"2024-03-09T08:52:30.597558+0000","last_clean":"2024-03-09T08:52:30.597558+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T08:52:30.597558+0000","last_undegraded":"2024-03-09T08:52:30.597558+0000","last_fullsized":"2024-03-09T08:52:30.597558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:19:44.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598143+0000","last_change":"2024-03-09T08:52:29.580024+0000","last_active":"2024-03-09T08:52:30.598143+0000","last_peered":"2024-03-09T08:52:30.598143+0000","last_clean":"2024-03-09T08:52:30.598143+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T08:52:30.598143+0000","last_undegraded":"2024-03-09T08:52:30.598143+0000","last_fullsized":"2024-03-09T08:52:30.598143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:02:13.195642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947463+0000","last_change":"2024-03-09T08:52:18.250999+0000","last_active":"2024-03-09T08:52:30.947463+0000","last_peered":"2024-03-09T08:52:30.947463+0000","last_clean":"2024-03-09T08:52:30.947463+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T08:52:30.947463+0000","last_undegraded":"2024-03-09T08:52:30.947463+0000","last_fullsized":"2024-03-09T08:52:30.947463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:49:54.496603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601554+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T08:52:30.601554+0000","last_peered":"2024-03-09T08:52:30.601554+0000","last_clean":"2024-03-09T08:52:30.601554+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T08:52:30.601554+0000","last_undegraded":"2024-03-09T08:52:30.601554+0000","last_fullsized":"2024-03-09T08:52:30.601554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601366+0000","last_change":"2024-03-09T08:52:28.584457+0000","last_active":"2024-03-09T08:52:30.601366+0000","last_peered":"2024-03-09T08:52:30.601366+0000","last_clean":"2024-03-09T08:52:30.601366+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T08:52:30.601366+0000","last_undegraded":"2024-03-09T08:52:30.601366+0000","last_fullsized":"2024-03-09T08:52:30.601366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:45.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597465+0000","last_change":"2024-03-09T08:52:29.596486+0000","last_active":"2024-03-09T08:52:30.597465+0000","last_peered":"2024-03-09T08:52:30.597465+0000","last_clean":"2024-03-09T08:52:30.597465+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T08:52:30.597465+0000","last_undegraded":"2024-03-09T08:52:30.597465+0000","last_fullsized":"2024-03-09T08:52:30.597465+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:40:16.587782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577604+0000","last_change":"2024-03-09T08:52:18.227365+0000","last_active":"2024-03-09T08:52:30.577604+0000","last_peered":"2024-03-09T08:52:30.577604+0000","last_clean":"2024-03-09T08:52:30.577604+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T08:52:30.577604+0000","last_undegraded":"2024-03-09T08:52:30.577604+0000","last_fullsized":"2024-03-09T08:52:30.577604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:26.357063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945950+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T08:52:30.945950+0000","last_peered":"2024-03-09T08:52:30.945950+0000","last_clean":"2024-03-09T08:52:30.945950+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T08:52:30.945950+0000","last_undegraded":"2024-03-09T08:52:30.945950+0000","last_fullsized":"2024-03-09T08:52:30.945950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601516+0000","last_change":"2024-03-09T08:52:28.578096+0000","last_active":"2024-03-09T08:52:30.601516+0000","last_peered":"2024-03-09T08:52:30.601516+0000","last_clean":"2024-03-09T08:52:30.601516+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T08:52:30.601516+0000","last_undegraded":"2024-03-09T08:52:30.601516+0000","last_fullsized":"2024-03-09T08:52:30.601516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:42.645517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943375+0000","last_change":"2024-03-09T08:52:29.590834+0000","last_active":"2024-03-09T08:52:30.943375+0000","last_peered":"2024-03-09T08:52:30.943375+0000","last_clean":"2024-03-09T08:52:30.943375+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T08:52:30.943375+0000","last_undegraded":"2024-03-09T08:52:30.943375+0000","last_fullsized":"2024-03-09T08:52:30.943375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:24:06.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.600924+0000","last_change":"2024-03-09T08:52:18.235832+0000","last_active":"2024-03-09T08:52:30.600924+0000","last_peered":"2024-03-09T08:52:30.600924+0000","last_clean":"2024-03-09T08:52:30.600924+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T08:52:30.600924+0000","last_undegraded":"2024-03-09T08:52:30.600924+0000","last_fullsized":"2024-03-09T08:52:30.600924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:38:58.751526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942505+0000","last_change":"2024-03-09T08:52:17.271847+0000","last_active":"2024-03-09T08:52:30.942505+0000","last_peered":"2024-03-09T08:52:30.942505+0000","last_clean":"2024-03-09T08:52:30.942505+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T08:52:30.942505+0000","last_undegraded":"2024-03-09T08:52:30.942505+0000","last_fullsized":"2024-03-09T08:52:30.942505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:27.418674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602301+0000","last_change":"2024-03-09T08:52:28.588636+0000","last_active":"2024-03-09T08:52:30.602301+0000","last_peered":"2024-03-09T08:52:30.602301+0000","last_clean":"2024-03-09T08:52:30.602301+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T08:52:30.602301+0000","last_undegraded":"2024-03-09T08:52:30.602301+0000","last_fullsized":"2024-03-09T08:52:30.602301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:21.601182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947090+0000","last_change":"2024-03-09T08:52:29.606833+0000","last_active":"2024-03-09T08:52:30.947090+0000","last_peered":"2024-03-09T08:52:30.947090+0000","last_clean":"2024-03-09T08:52:30.947090+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T08:52:30.947090+0000","last_undegraded":"2024-03-09T08:52:30.947090+0000","last_fullsized":"2024-03-09T08:52:30.947090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:40.299681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597339+0000","last_change":"2024-03-09T08:52:18.214396+0000","last_active":"2024-03-09T08:52:30.597339+0000","last_peered":"2024-03-09T08:52:30.597339+0000","last_clean":"2024-03-09T08:52:30.597339+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T08:52:30.597339+0000","last_undegraded":"2024-03-09T08:52:30.597339+0000","last_fullsized":"2024-03-09T08:52:30.597339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:24:10.719351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944007+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T08:52:30.944007+0000","last_peered":"2024-03-09T08:52:30.944007+0000","last_clean":"2024-03-09T08:52:30.944007+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T08:52:30.944007+0000","last_undegraded":"2024-03-09T08:52:30.944007+0000","last_fullsized":"2024-03-09T08:52:30.944007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.599908+0000","last_change":"2024-03-09T08:52:28.578599+0000","last_active":"2024-03-09T08:52:30.599908+0000","last_peered":"2024-03-09T08:52:30.599908+0000","last_clean":"2024-03-09T08:52:30.599908+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T08:52:30.599908+0000","last_undegraded":"2024-03-09T08:52:30.599908+0000","last_fullsized":"2024-03-09T08:52:30.599908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:52:31.328248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943920+0000","last_change":"2024-03-09T08:52:29.591872+0000","last_active":"2024-03-09T08:52:30.943920+0000","last_peered":"2024-03-09T08:52:30.943920+0000","last_clean":"2024-03-09T08:52:30.943920+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T08:52:30.943920+0000","last_undegraded":"2024-03-09T08:52:30.943920+0000","last_fullsized":"2024-03-09T08:52:30.943920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:44:17.870183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576290+0000","last_change":"2024-03-09T08:52:18.235367+0000","last_active":"2024-03-09T08:52:30.576290+0000","last_peered":"2024-03-09T08:52:30.576290+0000","last_clean":"2024-03-09T08:52:30.576290+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T08:52:30.576290+0000","last_undegraded":"2024-03-09T08:52:30.576290+0000","last_fullsized":"2024-03-09T08:52:30.576290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:40.736440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946321+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T08:52:30.946321+0000","last_peered":"2024-03-09T08:52:30.946321+0000","last_clean":"2024-03-09T08:52:30.946321+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T08:52:30.946321+0000","last_undegraded":"2024-03-09T08:52:30.946321+0000","last_fullsized":"2024-03-09T08:52:30.946321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944699+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T08:52:30.944699+0000","last_peered":"2024-03-09T08:52:30.944699+0000","last_clean":"2024-03-09T08:52:30.944699+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T08:52:30.944699+0000","last_undegraded":"2024-03-09T08:52:30.944699+0000","last_fullsized":"2024-03-09T08:52:30.944699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577815+0000","last_change":"2024-03-09T08:52:28.577973+0000","last_active":"2024-03-09T08:52:30.577815+0000","last_peered":"2024-03-09T08:52:30.577815+0000","last_clean":"2024-03-09T08:52:30.577815+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T08:52:30.577815+0000","last_undegraded":"2024-03-09T08:52:30.577815+0000","last_fullsized":"2024-03-09T08:52:30.577815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:13.291622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943332+0000","last_change":"2024-03-09T08:52:29.599601+0000","last_active":"2024-03-09T08:52:30.943332+0000","last_peered":"2024-03-09T08:52:30.943332+0000","last_clean":"2024-03-09T08:52:30.943332+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T08:52:30.943332+0000","last_undegraded":"2024-03-09T08:52:30.943332+0000","last_fullsized":"2024-03-09T08:52:30.943332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:11:17.424330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948439+0000","last_change":"2024-03-09T08:52:18.242671+0000","last_active":"2024-03-09T08:52:30.948439+0000","last_peered":"2024-03-09T08:52:30.948439+0000","last_clean":"2024-03-09T08:52:30.948439+0000","last_became_active":"2024-03-09T08:52:18.242258+0000","last_became_peered":"2024-03-09T08:52:18.242258+0000","last_unstale":"2024-03-09T08:52:30.948439+0000","last_undegraded":"2024-03-09T08:52:30.948439+0000","last_fullsized":"2024-03-09T08:52:30.948439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:15.030669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943179+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T08:52:30.943179+0000","last_peered":"2024-03-09T08:52:30.943179+0000","last_clean":"2024-03-09T08:52:30.943179+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T08:52:30.943179+0000","last_undegraded":"2024-03-09T08:52:30.943179+0000","last_fullsized":"2024-03-09T08:52:30.943179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:27.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576159+0000","last_change":"2024-03-09T08:52:28.589278+0000","last_active":"2024-03-09T08:52:30.576159+0000","last_peered":"2024-03-09T08:52:30.576159+0000","last_clean":"2024-03-09T08:52:30.576159+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T08:52:30.576159+0000","last_undegraded":"2024-03-09T08:52:30.576159+0000","last_fullsized":"2024-03-09T08:52:30.576159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:16:26.449275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942717+0000","last_change":"2024-03-09T08:52:29.600332+0000","last_active":"2024-03-09T08:52:30.942717+0000","last_peered":"2024-03-09T08:52:30.942717+0000","last_clean":"2024-03-09T08:52:30.942717+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T08:52:30.942717+0000","last_undegraded":"2024-03-09T08:52:30.942717+0000","last_fullsized":"2024-03-09T08:52:30.942717+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:36:41.422546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947852+0000","last_change":"2024-03-09T08:52:18.250433+0000","last_active":"2024-03-09T08:52:30.947852+0000","last_peered":"2024-03-09T08:52:30.947852+0000","last_clean":"2024-03-09T08:52:30.947852+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T08:52:30.947852+0000","last_undegraded":"2024-03-09T08:52:30.947852+0000","last_fullsized":"2024-03-09T08:52:30.947852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:29.742337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602070+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T08:52:30.602070+0000","last_peered":"2024-03-09T08:52:30.602070+0000","last_clean":"2024-03-09T08:52:30.602070+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T08:52:30.602070+0000","last_undegraded":"2024-03-09T08:52:30.602070+0000","last_fullsized":"2024-03-09T08:52:30.602070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:26:35.574903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597442+0000","last_change":"2024-03-09T08:52:28.583181+0000","last_active":"2024-03-09T08:52:30.597442+0000","last_peered":"2024-03-09T08:52:30.597442+0000","last_clean":"2024-03-09T08:52:30.597442+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T08:52:30.597442+0000","last_undegraded":"2024-03-09T08:52:30.597442+0000","last_fullsized":"2024-03-09T08:52:30.597442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:58:50.061324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943982+0000","last_change":"2024-03-09T08:52:29.590141+0000","last_active":"2024-03-09T08:52:30.943982+0000","last_peered":"2024-03-09T08:52:30.943982+0000","last_clean":"2024-03-09T08:52:30.943982+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T08:52:30.943982+0000","last_undegraded":"2024-03-09T08:52:30.943982+0000","last_fullsized":"2024-03-09T08:52:30.943982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:23.981897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948851+0000","last_change":"2024-03-09T08:52:18.250420+0000","last_active":"2024-03-09T08:52:30.948851+0000","last_peered":"2024-03-09T08:52:30.948851+0000","last_clean":"2024-03-09T08:52:30.948851+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T08:52:30.948851+0000","last_undegraded":"2024-03-09T08:52:30.948851+0000","last_fullsized":"2024-03-09T08:52:30.948851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:48.826066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943461+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T08:52:30.943461+0000","last_peered":"2024-03-09T08:52:30.943461+0000","last_clean":"2024-03-09T08:52:30.943461+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T08:52:30.943461+0000","last_undegraded":"2024-03-09T08:52:30.943461+0000","last_fullsized":"2024-03-09T08:52:30.943461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:16.138216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944811+0000","last_change":"2024-03-09T08:52:28.587637+0000","last_active":"2024-03-09T08:52:30.944811+0000","last_peered":"2024-03-09T08:52:30.944811+0000","last_clean":"2024-03-09T08:52:30.944811+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T08:52:30.944811+0000","last_undegraded":"2024-03-09T08:52:30.944811+0000","last_fullsized":"2024-03-09T08:52:30.944811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:31:32.362061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942477+0000","last_change":"2024-03-09T08:52:29.586180+0000","last_active":"2024-03-09T08:52:30.942477+0000","last_peered":"2024-03-09T08:52:30.942477+0000","last_clean":"2024-03-09T08:52:30.942477+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T08:52:30.942477+0000","last_undegraded":"2024-03-09T08:52:30.942477+0000","last_fullsized":"2024-03-09T08:52:30.942477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:35:21.522181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948836+0000","last_change":"2024-03-09T08:52:18.246145+0000","last_active":"2024-03-09T08:52:30.948836+0000","last_peered":"2024-03-09T08:52:30.948836+0000","last_clean":"2024-03-09T08:52:30.948836+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T08:52:30.948836+0000","last_undegraded":"2024-03-09T08:52:30.948836+0000","last_fullsized":"2024-03-09T08:52:30.948836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:29:36.213774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578625+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T08:52:30.578625+0000","last_peered":"2024-03-09T08:52:30.578625+0000","last_clean":"2024-03-09T08:52:30.578625+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T08:52:30.578625+0000","last_undegraded":"2024-03-09T08:52:30.578625+0000","last_fullsized":"2024-03-09T08:52:30.578625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:10:34.698235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575615+0000","last_change":"2024-03-09T08:52:28.582308+0000","last_active":"2024-03-09T08:52:30.575615+0000","last_peered":"2024-03-09T08:52:30.575615+0000","last_clean":"2024-03-09T08:52:30.575615+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T08:52:30.575615+0000","last_undegraded":"2024-03-09T08:52:30.575615+0000","last_fullsized":"2024-03-09T08:52:30.575615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:04.918893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577162+0000","last_change":"2024-03-09T08:52:29.610596+0000","last_active":"2024-03-09T08:52:30.577162+0000","last_peered":"2024-03-09T08:52:30.577162+0000","last_clean":"2024-03-09T08:52:30.577162+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T08:52:30.577162+0000","last_undegraded":"2024-03-09T08:52:30.577162+0000","last_fullsized":"2024-03-09T08:52:30.577162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:19:47.313981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944192+0000","last_change":"2024-03-09T08:52:18.248755+0000","last_active":"2024-03-09T08:52:30.944192+0000","last_peered":"2024-03-09T08:52:30.944192+0000","last_clean":"2024-03-09T08:52:30.944192+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T08:52:30.944192+0000","last_undegraded":"2024-03-09T08:52:30.944192+0000","last_fullsized":"2024-03-09T08:52:30.944192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:37.083738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942668+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T08:52:30.942668+0000","last_peered":"2024-03-09T08:52:30.942668+0000","last_clean":"2024-03-09T08:52:30.942668+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T08:52:30.942668+0000","last_undegraded":"2024-03-09T08:52:30.942668+0000","last_fullsized":"2024-03-09T08:52:30.942668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:31.859924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598089+0000","last_change":"2024-03-09T08:52:28.575975+0000","last_active":"2024-03-09T08:52:30.598089+0000","last_peered":"2024-03-09T08:52:30.598089+0000","last_clean":"2024-03-09T08:52:30.598089+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T08:52:30.598089+0000","last_undegraded":"2024-03-09T08:52:30.598089+0000","last_fullsized":"2024-03-09T08:52:30.598089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:51:24.405316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947555+0000","last_change":"2024-03-09T08:52:29.575752+0000","last_active":"2024-03-09T08:52:30.947555+0000","last_peered":"2024-03-09T08:52:30.947555+0000","last_clean":"2024-03-09T08:52:30.947555+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T08:52:30.947555+0000","last_undegraded":"2024-03-09T08:52:30.947555+0000","last_fullsized":"2024-03-09T08:52:30.947555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:31:51.847871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575309+0000","last_change":"2024-03-09T08:52:18.245073+0000","last_active":"2024-03-09T08:52:30.575309+0000","last_peered":"2024-03-09T08:52:30.575309+0000","last_clean":"2024-03-09T08:52:30.575309+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T08:52:30.575309+0000","last_undegraded":"2024-03-09T08:52:30.575309+0000","last_fullsized":"2024-03-09T08:52:30.575309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:40:15.390735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577484+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T08:52:30.577484+0000","last_peered":"2024-03-09T08:52:30.577484+0000","last_clean":"2024-03-09T08:52:30.577484+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T08:52:30.577484+0000","last_undegraded":"2024-03-09T08:52:30.577484+0000","last_fullsized":"2024-03-09T08:52:30.577484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:07:29.577461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577273+0000","last_change":"2024-03-09T08:52:28.588312+0000","last_active":"2024-03-09T08:52:30.577273+0000","last_peered":"2024-03-09T08:52:30.577273+0000","last_clean":"2024-03-09T08:52:30.577273+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T08:52:30.577273+0000","last_undegraded":"2024-03-09T08:52:30.577273+0000","last_fullsized":"2024-03-09T08:52:30.577273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:14:52.895618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946368+0000","last_change":"2024-03-09T08:52:29.596647+0000","last_active":"2024-03-09T08:52:30.946368+0000","last_peered":"2024-03-09T08:52:30.946368+0000","last_clean":"2024-03-09T08:52:30.946368+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T08:52:30.946368+0000","last_undegraded":"2024-03-09T08:52:30.946368+0000","last_fullsized":"2024-03-09T08:52:30.946368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:35:28.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944097+0000","last_change":"2024-03-09T08:52:18.248093+0000","last_active":"2024-03-09T08:52:30.944097+0000","last_peered":"2024-03-09T08:52:30.944097+0000","last_clean":"2024-03-09T08:52:30.944097+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T08:52:30.944097+0000","last_undegraded":"2024-03-09T08:52:30.944097+0000","last_fullsized":"2024-03-09T08:52:30.944097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:29.414335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945767+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T08:52:30.945767+0000","last_peered":"2024-03-09T08:52:30.945767+0000","last_clean":"2024-03-09T08:52:30.945767+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T08:52:30.945767+0000","last_undegraded":"2024-03-09T08:52:30.945767+0000","last_fullsized":"2024-03-09T08:52:30.945767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576830+0000","last_change":"2024-03-09T08:52:28.585562+0000","last_active":"2024-03-09T08:52:30.576830+0000","last_peered":"2024-03-09T08:52:30.576830+0000","last_clean":"2024-03-09T08:52:30.576830+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T08:52:30.576830+0000","last_undegraded":"2024-03-09T08:52:30.576830+0000","last_fullsized":"2024-03-09T08:52:30.576830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:28:50.544795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576833+0000","last_change":"2024-03-09T08:52:29.581330+0000","last_active":"2024-03-09T08:52:30.576833+0000","last_peered":"2024-03-09T08:52:30.576833+0000","last_clean":"2024-03-09T08:52:30.576833+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T08:52:30.576833+0000","last_undegraded":"2024-03-09T08:52:30.576833+0000","last_fullsized":"2024-03-09T08:52:30.576833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:55:38.041794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597506+0000","last_change":"2024-03-09T08:52:18.236146+0000","last_active":"2024-03-09T08:52:30.597506+0000","last_peered":"2024-03-09T08:52:30.597506+0000","last_clean":"2024-03-09T08:52:30.597506+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T08:52:30.597506+0000","last_undegraded":"2024-03-09T08:52:30.597506+0000","last_fullsized":"2024-03-09T08:52:30.597506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:18:51.511926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598307+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T08:52:30.598307+0000","last_peered":"2024-03-09T08:52:30.598307+0000","last_clean":"2024-03-09T08:52:30.598307+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T08:52:30.598307+0000","last_undegraded":"2024-03-09T08:52:30.598307+0000","last_fullsized":"2024-03-09T08:52:30.598307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:22:05.504557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947315+0000","last_change":"2024-03-09T08:52:28.585697+0000","last_active":"2024-03-09T08:52:30.947315+0000","last_peered":"2024-03-09T08:52:30.947315+0000","last_clean":"2024-03-09T08:52:30.947315+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T08:52:30.947315+0000","last_undegraded":"2024-03-09T08:52:30.947315+0000","last_fullsized":"2024-03-09T08:52:30.947315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:50:23.095008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576346+0000","last_change":"2024-03-09T08:52:29.578803+0000","last_active":"2024-03-09T08:52:30.576346+0000","last_peered":"2024-03-09T08:52:30.576346+0000","last_clean":"2024-03-09T08:52:30.576346+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T08:52:30.576346+0000","last_undegraded":"2024-03-09T08:52:30.576346+0000","last_fullsized":"2024-03-09T08:52:30.576346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:22:47.690873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948118+0000","last_change":"2024-03-09T08:52:18.242496+0000","last_active":"2024-03-09T08:52:30.948118+0000","last_peered":"2024-03-09T08:52:30.948118+0000","last_clean":"2024-03-09T08:52:30.948118+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T08:52:30.948118+0000","last_undegraded":"2024-03-09T08:52:30.948118+0000","last_fullsized":"2024-03-09T08:52:30.948118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:35:17.142983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578358+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T08:52:30.578358+0000","last_peered":"2024-03-09T08:52:30.578358+0000","last_clean":"2024-03-09T08:52:30.578358+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T08:52:30.578358+0000","last_undegraded":"2024-03-09T08:52:30.578358+0000","last_fullsized":"2024-03-09T08:52:30.578358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:01.724043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601055+0000","last_change":"2024-03-09T08:52:28.583685+0000","last_active":"2024-03-09T08:52:30.601055+0000","last_peered":"2024-03-09T08:52:30.601055+0000","last_clean":"2024-03-09T08:52:30.601055+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T08:52:30.601055+0000","last_undegraded":"2024-03-09T08:52:30.601055+0000","last_fullsized":"2024-03-09T08:52:30.601055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:58:00.899620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601468+0000","last_change":"2024-03-09T08:52:29.585912+0000","last_active":"2024-03-09T08:52:30.601468+0000","last_peered":"2024-03-09T08:52:30.601468+0000","last_clean":"2024-03-09T08:52:30.601468+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T08:52:30.601468+0000","last_undegraded":"2024-03-09T08:52:30.601468+0000","last_fullsized":"2024-03-09T08:52:30.601468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:24:47.261191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577039+0000","last_change":"2024-03-09T08:52:18.244654+0000","last_active":"2024-03-09T08:52:30.577039+0000","last_peered":"2024-03-09T08:52:30.577039+0000","last_clean":"2024-03-09T08:52:30.577039+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T08:52:30.577039+0000","last_undegraded":"2024-03-09T08:52:30.577039+0000","last_fullsized":"2024-03-09T08:52:30.577039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:08:58.611725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601999+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T08:52:30.601999+0000","last_peered":"2024-03-09T08:52:30.601999+0000","last_clean":"2024-03-09T08:52:30.601999+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T08:52:30.601999+0000","last_undegraded":"2024-03-09T08:52:30.601999+0000","last_fullsized":"2024-03-09T08:52:30.601999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:41:10.578669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577095+0000","last_change":"2024-03-09T08:52:28.568794+0000","last_active":"2024-03-09T08:52:30.577095+0000","last_peered":"2024-03-09T08:52:30.577095+0000","last_clean":"2024-03-09T08:52:30.577095+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T08:52:30.577095+0000","last_undegraded":"2024-03-09T08:52:30.577095+0000","last_fullsized":"2024-03-09T08:52:30.577095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:44:21.721511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577942+0000","last_change":"2024-03-09T08:52:29.586705+0000","last_active":"2024-03-09T08:52:30.577942+0000","last_peered":"2024-03-09T08:52:30.577942+0000","last_clean":"2024-03-09T08:52:30.577942+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T08:52:30.577942+0000","last_undegraded":"2024-03-09T08:52:30.577942+0000","last_fullsized":"2024-03-09T08:52:30.577942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:26.353156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597722+0000","last_change":"2024-03-09T08:52:18.236628+0000","last_active":"2024-03-09T08:52:30.597722+0000","last_peered":"2024-03-09T08:52:30.597722+0000","last_clean":"2024-03-09T08:52:30.597722+0000","last_became_active":"2024-03-09T08:52:18.235321+0000","last_became_peered":"2024-03-09T08:52:18.235321+0000","last_unstale":"2024-03-09T08:52:30.597722+0000","last_undegraded":"2024-03-09T08:52:30.597722+0000","last_fullsized":"2024-03-09T08:52:30.597722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:13:59.221325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948276+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T08:52:30.948276+0000","last_peered":"2024-03-09T08:52:30.948276+0000","last_clean":"2024-03-09T08:52:30.948276+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T08:52:30.948276+0000","last_undegraded":"2024-03-09T08:52:30.948276+0000","last_fullsized":"2024-03-09T08:52:30.948276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575808+0000","last_change":"2024-03-09T08:52:28.583697+0000","last_active":"2024-03-09T08:52:30.575808+0000","last_peered":"2024-03-09T08:52:30.575808+0000","last_clean":"2024-03-09T08:52:30.575808+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T08:52:30.575808+0000","last_undegraded":"2024-03-09T08:52:30.575808+0000","last_fullsized":"2024-03-09T08:52:30.575808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:41.552156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945076+0000","last_change":"2024-03-09T08:52:29.590854+0000","last_active":"2024-03-09T08:52:30.945076+0000","last_peered":"2024-03-09T08:52:30.945076+0000","last_clean":"2024-03-09T08:52:30.945076+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T08:52:30.945076+0000","last_undegraded":"2024-03-09T08:52:30.945076+0000","last_fullsized":"2024-03-09T08:52:30.945076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:09.568826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947914+0000","last_change":"2024-03-09T08:52:18.251648+0000","last_active":"2024-03-09T08:52:30.947914+0000","last_peered":"2024-03-09T08:52:30.947914+0000","last_clean":"2024-03-09T08:52:30.947914+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T08:52:30.947914+0000","last_undegraded":"2024-03-09T08:52:30.947914+0000","last_fullsized":"2024-03-09T08:52:30.947914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:54:45.721673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579118+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T08:52:30.579118+0000","last_peered":"2024-03-09T08:52:30.579118+0000","last_clean":"2024-03-09T08:52:30.579118+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T08:52:30.579118+0000","last_undegraded":"2024-03-09T08:52:30.579118+0000","last_fullsized":"2024-03-09T08:52:30.579118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:31.731626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575378+0000","last_change":"2024-03-09T08:52:28.586606+0000","last_active":"2024-03-09T08:52:30.575378+0000","last_peered":"2024-03-09T08:52:30.575378+0000","last_clean":"2024-03-09T08:52:30.575378+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T08:52:30.575378+0000","last_undegraded":"2024-03-09T08:52:30.575378+0000","last_fullsized":"2024-03-09T08:52:30.575378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:17:40.902403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578219+0000","last_change":"2024-03-09T08:52:29.584512+0000","last_active":"2024-03-09T08:52:30.578219+0000","last_peered":"2024-03-09T08:52:30.578219+0000","last_clean":"2024-03-09T08:52:30.578219+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T08:52:30.578219+0000","last_undegraded":"2024-03-09T08:52:30.578219+0000","last_fullsized":"2024-03-09T08:52:30.578219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:39:20.330443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597975+0000","last_change":"2024-03-09T08:52:28.586201+0000","last_active":"2024-03-09T08:52:30.597975+0000","last_peered":"2024-03-09T08:52:30.597975+0000","last_clean":"2024-03-09T08:52:30.597975+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T08:52:30.597975+0000","last_undegraded":"2024-03-09T08:52:30.597975+0000","last_fullsized":"2024-03-09T08:52:30.597975+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:28:03.471646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601039+0000","last_change":"2024-03-09T08:52:18.241156+0000","last_active":"2024-03-09T08:52:30.601039+0000","last_peered":"2024-03-09T08:52:30.601039+0000","last_clean":"2024-03-09T08:52:30.601039+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T08:52:30.601039+0000","last_undegraded":"2024-03-09T08:52:30.601039+0000","last_fullsized":"2024-03-09T08:52:30.601039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:55.323770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946394+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T08:52:30.946394+0000","last_peered":"2024-03-09T08:52:30.946394+0000","last_clean":"2024-03-09T08:52:30.946394+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T08:52:30.946394+0000","last_undegraded":"2024-03-09T08:52:30.946394+0000","last_fullsized":"2024-03-09T08:52:30.946394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597714+0000","last_change":"2024-03-09T08:52:29.591085+0000","last_active":"2024-03-09T08:52:30.597714+0000","last_peered":"2024-03-09T08:52:30.597714+0000","last_clean":"2024-03-09T08:52:30.597714+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T08:52:30.597714+0000","last_undegraded":"2024-03-09T08:52:30.597714+0000","last_fullsized":"2024-03-09T08:52:30.597714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:40:27.878922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578971+0000","last_change":"2024-03-09T08:52:28.585631+0000","last_active":"2024-03-09T08:52:30.578971+0000","last_peered":"2024-03-09T08:52:30.578971+0000","last_clean":"2024-03-09T08:52:30.578971+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T08:52:30.578971+0000","last_undegraded":"2024-03-09T08:52:30.578971+0000","last_fullsized":"2024-03-09T08:52:30.578971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:06:03.569759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576252+0000","last_change":"2024-03-09T08:52:18.234400+0000","last_active":"2024-03-09T08:52:30.576252+0000","last_peered":"2024-03-09T08:52:30.576252+0000","last_clean":"2024-03-09T08:52:30.576252+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T08:52:30.576252+0000","last_undegraded":"2024-03-09T08:52:30.576252+0000","last_fullsized":"2024-03-09T08:52:30.576252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:42:14.057367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576643+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T08:52:30.576643+0000","last_peered":"2024-03-09T08:52:30.576643+0000","last_clean":"2024-03-09T08:52:30.576643+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T08:52:30.576643+0000","last_undegraded":"2024-03-09T08:52:30.576643+0000","last_fullsized":"2024-03-09T08:52:30.576643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601916+0000","last_change":"2024-03-09T08:52:29.588928+0000","last_active":"2024-03-09T08:52:30.601916+0000","last_peered":"2024-03-09T08:52:30.601916+0000","last_clean":"2024-03-09T08:52:30.601916+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T08:52:30.601916+0000","last_undegraded":"2024-03-09T08:52:30.601916+0000","last_fullsized":"2024-03-09T08:52:30.601916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:32:27.061041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576515+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T08:52:30.576515+0000","last_peered":"2024-03-09T08:52:30.576515+0000","last_clean":"2024-03-09T08:52:30.576515+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T08:52:30.576515+0000","last_undegraded":"2024-03-09T08:52:30.576515+0000","last_fullsized":"2024-03-09T08:52:30.576515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598213+0000","last_change":"2024-03-09T08:52:18.231141+0000","last_active":"2024-03-09T08:52:30.598213+0000","last_peered":"2024-03-09T08:52:30.598213+0000","last_clean":"2024-03-09T08:52:30.598213+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T08:52:30.598213+0000","last_undegraded":"2024-03-09T08:52:30.598213+0000","last_fullsized":"2024-03-09T08:52:30.598213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:25:00.471222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943012+0000","last_change":"2024-03-09T08:52:28.579315+0000","last_active":"2024-03-09T08:52:30.943012+0000","last_peered":"2024-03-09T08:52:30.943012+0000","last_clean":"2024-03-09T08:52:30.943012+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T08:52:30.943012+0000","last_undegraded":"2024-03-09T08:52:30.943012+0000","last_fullsized":"2024-03-09T08:52:30.943012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:57:17.301677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576561+0000","last_change":"2024-03-09T08:52:29.585154+0000","last_active":"2024-03-09T08:52:30.576561+0000","last_peered":"2024-03-09T08:52:30.576561+0000","last_clean":"2024-03-09T08:52:30.576561+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T08:52:30.576561+0000","last_undegraded":"2024-03-09T08:52:30.576561+0000","last_fullsized":"2024-03-09T08:52:30.576561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:43:35.679108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576704+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T08:52:30.576704+0000","last_peered":"2024-03-09T08:52:30.576704+0000","last_clean":"2024-03-09T08:52:30.576704+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T08:52:30.576704+0000","last_undegraded":"2024-03-09T08:52:30.576704+0000","last_fullsized":"2024-03-09T08:52:30.576704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577869+0000","last_change":"2024-03-09T08:52:18.229533+0000","last_active":"2024-03-09T08:52:30.577869+0000","last_peered":"2024-03-09T08:52:30.577869+0000","last_clean":"2024-03-09T08:52:30.577869+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T08:52:30.577869+0000","last_undegraded":"2024-03-09T08:52:30.577869+0000","last_fullsized":"2024-03-09T08:52:30.577869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:17.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576387+0000","last_change":"2024-03-09T08:52:28.585877+0000","last_active":"2024-03-09T08:52:30.576387+0000","last_peered":"2024-03-09T08:52:30.576387+0000","last_clean":"2024-03-09T08:52:30.576387+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T08:52:30.576387+0000","last_undegraded":"2024-03-09T08:52:30.576387+0000","last_fullsized":"2024-03-09T08:52:30.576387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:30:20.036385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577546+0000","last_change":"2024-03-09T08:52:29.592253+0000","last_active":"2024-03-09T08:52:30.577546+0000","last_peered":"2024-03-09T08:52:30.577546+0000","last_clean":"2024-03-09T08:52:30.577546+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T08:52:30.577546+0000","last_undegraded":"2024-03-09T08:52:30.577546+0000","last_fullsized":"2024-03-09T08:52:30.577546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:50:33.525541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575154+0000","last_change":"2024-03-09T08:52:18.237869+0000","last_active":"2024-03-09T08:52:30.575154+0000","last_peered":"2024-03-09T08:52:30.575154+0000","last_clean":"2024-03-09T08:52:30.575154+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T08:52:30.575154+0000","last_undegraded":"2024-03-09T08:52:30.575154+0000","last_fullsized":"2024-03-09T08:52:30.575154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:01.571866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602125+0000","last_change":"2024-03-09T08:52:29.601300+0000","last_active":"2024-03-09T08:52:30.602125+0000","last_peered":"2024-03-09T08:52:30.602125+0000","last_clean":"2024-03-09T08:52:30.602125+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T08:52:30.602125+0000","last_undegraded":"2024-03-09T08:52:30.602125+0000","last_fullsized":"2024-03-09T08:52:30.602125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:46:04.870574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942394+0000","last_change":"2024-03-09T08:52:18.226980+0000","last_active":"2024-03-09T08:52:30.942394+0000","last_peered":"2024-03-09T08:52:30.942394+0000","last_clean":"2024-03-09T08:52:30.942394+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T08:52:30.942394+0000","last_undegraded":"2024-03-09T08:52:30.942394+0000","last_fullsized":"2024-03-09T08:52:30.942394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:28.312576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946757+0000","last_change":"2024-03-09T08:52:29.575612+0000","last_active":"2024-03-09T08:52:30.946757+0000","last_peered":"2024-03-09T08:52:30.946757+0000","last_clean":"2024-03-09T08:52:30.946757+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T08:52:30.946757+0000","last_undegraded":"2024-03-09T08:52:30.946757+0000","last_fullsized":"2024-03-09T08:52:30.946757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:58.852983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945644+0000","last_change":"2024-03-09T08:52:18.241628+0000","last_active":"2024-03-09T08:52:30.945644+0000","last_peered":"2024-03-09T08:52:30.945644+0000","last_clean":"2024-03-09T08:52:30.945644+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T08:52:30.945644+0000","last_undegraded":"2024-03-09T08:52:30.945644+0000","last_fullsized":"2024-03-09T08:52:30.945644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:07:58.570641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601917+0000","last_change":"2024-03-09T08:52:29.588660+0000","last_active":"2024-03-09T08:52:30.601917+0000","last_peered":"2024-03-09T08:52:30.601917+0000","last_clean":"2024-03-09T08:52:30.601917+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T08:52:30.601917+0000","last_undegraded":"2024-03-09T08:52:30.601917+0000","last_fullsized":"2024-03-09T08:52:30.601917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:14.305320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598085+0000","last_change":"2024-03-09T08:52:18.242257+0000","last_active":"2024-03-09T08:52:30.598085+0000","last_peered":"2024-03-09T08:52:30.598085+0000","last_clean":"2024-03-09T08:52:30.598085+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T08:52:30.598085+0000","last_undegraded":"2024-03-09T08:52:30.598085+0000","last_fullsized":"2024-03-09T08:52:30.598085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:40:00.854066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944026+0000","last_change":"2024-03-09T08:52:29.595057+0000","last_active":"2024-03-09T08:52:30.944026+0000","last_peered":"2024-03-09T08:52:30.944026+0000","last_clean":"2024-03-09T08:52:30.944026+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T08:52:30.944026+0000","last_undegraded":"2024-03-09T08:52:30.944026+0000","last_fullsized":"2024-03-09T08:52:30.944026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:33:37.474266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577091+0000","last_change":"2024-03-09T08:52:18.239951+0000","last_active":"2024-03-09T08:52:30.577091+0000","last_peered":"2024-03-09T08:52:30.577091+0000","last_clean":"2024-03-09T08:52:30.577091+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T08:52:30.577091+0000","last_undegraded":"2024-03-09T08:52:30.577091+0000","last_fullsized":"2024-03-09T08:52:30.577091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:51.102540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576730+0000","last_change":"2024-03-09T08:52:29.578993+0000","last_active":"2024-03-09T08:52:30.576730+0000","last_peered":"2024-03-09T08:52:30.576730+0000","last_clean":"2024-03-09T08:52:30.576730+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T08:52:30.576730+0000","last_undegraded":"2024-03-09T08:52:30.576730+0000","last_fullsized":"2024-03-09T08:52:30.576730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:50:53.692060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601995+0000","last_change":"2024-03-09T08:52:18.241038+0000","last_active":"2024-03-09T08:52:30.601995+0000","last_peered":"2024-03-09T08:52:30.601995+0000","last_clean":"2024-03-09T08:52:30.601995+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T08:52:30.601995+0000","last_undegraded":"2024-03-09T08:52:30.601995+0000","last_fullsized":"2024-03-09T08:52:30.601995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:02.169113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943444+0000","last_change":"2024-03-09T08:52:29.583869+0000","last_active":"2024-03-09T08:52:30.943444+0000","last_peered":"2024-03-09T08:52:30.943444+0000","last_clean":"2024-03-09T08:52:30.943444+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T08:52:30.943444+0000","last_undegraded":"2024-03-09T08:52:30.943444+0000","last_fullsized":"2024-03-09T08:52:30.943444+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:29:47.636741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578670+0000","last_change":"2024-03-09T08:52:18.247145+0000","last_active":"2024-03-09T08:52:30.578670+0000","last_peered":"2024-03-09T08:52:30.578670+0000","last_clean":"2024-03-09T08:52:30.578670+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T08:52:30.578670+0000","last_undegraded":"2024-03-09T08:52:30.578670+0000","last_fullsized":"2024-03-09T08:52:30.578670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:46:24.111224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577279+0000","last_change":"2024-03-09T08:52:29.587358+0000","last_active":"2024-03-09T08:52:30.577279+0000","last_peered":"2024-03-09T08:52:30.577279+0000","last_clean":"2024-03-09T08:52:30.577279+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T08:52:30.577279+0000","last_undegraded":"2024-03-09T08:52:30.577279+0000","last_fullsized":"2024-03-09T08:52:30.577279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:46.836801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577323+0000","last_change":"2024-03-09T08:52:18.247784+0000","last_active":"2024-03-09T08:52:30.577323+0000","last_peered":"2024-03-09T08:52:30.577323+0000","last_clean":"2024-03-09T08:52:30.577323+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T08:52:30.577323+0000","last_undegraded":"2024-03-09T08:52:30.577323+0000","last_fullsized":"2024-03-09T08:52:30.577323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:46:59.008220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577364+0000","last_change":"2024-03-09T08:52:29.589458+0000","last_active":"2024-03-09T08:52:30.577364+0000","last_peered":"2024-03-09T08:52:30.577364+0000","last_clean":"2024-03-09T08:52:30.577364+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T08:52:30.577364+0000","last_undegraded":"2024-03-09T08:52:30.577364+0000","last_fullsized":"2024-03-09T08:52:30.577364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:19:35.222184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945231+0000","last_change":"2024-03-09T08:52:18.239943+0000","last_active":"2024-03-09T08:52:30.945231+0000","last_peered":"2024-03-09T08:52:30.945231+0000","last_clean":"2024-03-09T08:52:30.945231+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T08:52:30.945231+0000","last_undegraded":"2024-03-09T08:52:30.945231+0000","last_fullsized":"2024-03-09T08:52:30.945231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:18.727030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.949211+0000","last_change":"2024-03-09T08:52:29.597147+0000","last_active":"2024-03-09T08:52:30.949211+0000","last_peered":"2024-03-09T08:52:30.949211+0000","last_clean":"2024-03-09T08:52:30.949211+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T08:52:30.949211+0000","last_undegraded":"2024-03-09T08:52:30.949211+0000","last_fullsized":"2024-03-09T08:52:30.949211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:44:47.144911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575715+0000","last_change":"2024-03-09T08:52:18.236642+0000","last_active":"2024-03-09T08:52:30.575715+0000","last_peered":"2024-03-09T08:52:30.575715+0000","last_clean":"2024-03-09T08:52:30.575715+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T08:52:30.575715+0000","last_undegraded":"2024-03-09T08:52:30.575715+0000","last_fullsized":"2024-03-09T08:52:30.575715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:51:46.825153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943044+0000","last_change":"2024-03-09T08:52:29.590810+0000","last_active":"2024-03-09T08:52:30.943044+0000","last_peered":"2024-03-09T08:52:30.943044+0000","last_clean":"2024-03-09T08:52:30.943044+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T08:52:30.943044+0000","last_undegraded":"2024-03-09T08:52:30.943044+0000","last_fullsized":"2024-03-09T08:52:30.943044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:21.534580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945569+0000","last_change":"2024-03-09T08:52:18.236855+0000","last_active":"2024-03-09T08:52:30.945569+0000","last_peered":"2024-03-09T08:52:30.945569+0000","last_clean":"2024-03-09T08:52:30.945569+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T08:52:30.945569+0000","last_undegraded":"2024-03-09T08:52:30.945569+0000","last_fullsized":"2024-03-09T08:52:30.945569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:21.941242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576311+0000","last_change":"2024-03-09T08:52:29.602209+0000","last_active":"2024-03-09T08:52:30.576311+0000","last_peered":"2024-03-09T08:52:30.576311+0000","last_clean":"2024-03-09T08:52:30.576311+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T08:52:30.576311+0000","last_undegraded":"2024-03-09T08:52:30.576311+0000","last_fullsized":"2024-03-09T08:52:30.576311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:11:33.349464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597936+0000","last_change":"2024-03-09T08:52:18.238373+0000","last_active":"2024-03-09T08:52:30.597936+0000","last_peered":"2024-03-09T08:52:30.597936+0000","last_clean":"2024-03-09T08:52:30.597936+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T08:52:30.597936+0000","last_undegraded":"2024-03-09T08:52:30.597936+0000","last_fullsized":"2024-03-09T08:52:30.597936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:05:31.480335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577230+0000","last_change":"2024-03-09T08:52:29.578120+0000","last_active":"2024-03-09T08:52:30.577230+0000","last_peered":"2024-03-09T08:52:30.577230+0000","last_clean":"2024-03-09T08:52:30.577230+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T08:52:30.577230+0000","last_undegraded":"2024-03-09T08:52:30.577230+0000","last_fullsized":"2024-03-09T08:52:30.577230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:13:27.623995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578214+0000","last_change":"2024-03-09T08:52:18.235565+0000","last_active":"2024-03-09T08:52:30.578214+0000","last_peered":"2024-03-09T08:52:30.578214+0000","last_clean":"2024-03-09T08:52:30.578214+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T08:52:30.578214+0000","last_undegraded":"2024-03-09T08:52:30.578214+0000","last_fullsized":"2024-03-09T08:52:30.578214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:52:14.994539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576439+0000","last_change":"2024-03-09T08:52:29.579668+0000","last_active":"2024-03-09T08:52:30.576439+0000","last_peered":"2024-03-09T08:52:30.576439+0000","last_clean":"2024-03-09T08:52:30.576439+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T08:52:30.576439+0000","last_undegraded":"2024-03-09T08:52:30.576439+0000","last_fullsized":"2024-03-09T08:52:30.576439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:12:15.272875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948818+0000","last_change":"2024-03-09T08:52:18.239801+0000","last_active":"2024-03-09T08:52:30.948818+0000","last_peered":"2024-03-09T08:52:30.948818+0000","last_clean":"2024-03-09T08:52:30.948818+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T08:52:30.948818+0000","last_undegraded":"2024-03-09T08:52:30.948818+0000","last_fullsized":"2024-03-09T08:52:30.948818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:38:06.836990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944799+0000","last_change":"2024-03-09T08:52:29.586986+0000","last_active":"2024-03-09T08:52:30.944799+0000","last_peered":"2024-03-09T08:52:30.944799+0000","last_clean":"2024-03-09T08:52:30.944799+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T08:52:30.944799+0000","last_undegraded":"2024-03-09T08:52:30.944799+0000","last_fullsized":"2024-03-09T08:52:30.944799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:37.382414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943359+0000","last_change":"2024-03-09T08:52:18.233582+0000","last_active":"2024-03-09T08:52:30.943359+0000","last_peered":"2024-03-09T08:52:30.943359+0000","last_clean":"2024-03-09T08:52:30.943359+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T08:52:30.943359+0000","last_undegraded":"2024-03-09T08:52:30.943359+0000","last_fullsized":"2024-03-09T08:52:30.943359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:49.313616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597247+0000","last_change":"2024-03-09T08:52:29.586673+0000","last_active":"2024-03-09T08:52:30.597247+0000","last_peered":"2024-03-09T08:52:30.597247+0000","last_clean":"2024-03-09T08:52:30.597247+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T08:52:30.597247+0000","last_undegraded":"2024-03-09T08:52:30.597247+0000","last_fullsized":"2024-03-09T08:52:30.597247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:23:49.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947293+0000","last_change":"2024-03-09T08:52:18.242349+0000","last_active":"2024-03-09T08:52:30.947293+0000","last_peered":"2024-03-09T08:52:30.947293+0000","last_clean":"2024-03-09T08:52:30.947293+0000","last_became_active":"2024-03-09T08:52:18.241995+0000","last_became_peered":"2024-03-09T08:52:18.241995+0000","last_unstale":"2024-03-09T08:52:30.947293+0000","last_undegraded":"2024-03-09T08:52:30.947293+0000","last_fullsized":"2024-03-09T08:52:30.947293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:29:09.977984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601965+0000","last_change":"2024-03-09T08:52:29.592702+0000","last_active":"2024-03-09T08:52:30.601965+0000","last_peered":"2024-03-09T08:52:30.601965+0000","last_clean":"2024-03-09T08:52:30.601965+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T08:52:30.601965+0000","last_undegraded":"2024-03-09T08:52:30.601965+0000","last_fullsized":"2024-03-09T08:52:30.601965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:39.344726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944970+0000","last_change":"2024-03-09T08:52:18.230289+0000","last_active":"2024-03-09T08:52:30.944970+0000","last_peered":"2024-03-09T08:52:30.944970+0000","last_clean":"2024-03-09T08:52:30.944970+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T08:52:30.944970+0000","last_undegraded":"2024-03-09T08:52:30.944970+0000","last_fullsized":"2024-03-09T08:52:30.944970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:34.507924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578394+0000","last_change":"2024-03-09T08:52:29.608191+0000","last_active":"2024-03-09T08:52:30.578394+0000","last_peered":"2024-03-09T08:52:30.578394+0000","last_clean":"2024-03-09T08:52:30.578394+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T08:52:30.578394+0000","last_undegraded":"2024-03-09T08:52:30.578394+0000","last_fullsized":"2024-03-09T08:52:30.578394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:26:32.282196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601147+0000","last_change":"2024-03-09T08:52:18.236376+0000","last_active":"2024-03-09T08:52:30.601147+0000","last_peered":"2024-03-09T08:52:30.601147+0000","last_clean":"2024-03-09T08:52:30.601147+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T08:52:30.601147+0000","last_undegraded":"2024-03-09T08:52:30.601147+0000","last_fullsized":"2024-03-09T08:52:30.601147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:34:04.412643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946339+0000","last_change":"2024-03-09T08:52:29.573685+0000","last_active":"2024-03-09T08:52:30.946339+0000","last_peered":"2024-03-09T08:52:30.946339+0000","last_clean":"2024-03-09T08:52:30.946339+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T08:52:30.946339+0000","last_undegraded":"2024-03-09T08:52:30.946339+0000","last_fullsized":"2024-03-09T08:52:30.946339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:41:41.839063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597356+0000","last_change":"2024-03-09T08:52:18.231316+0000","last_active":"2024-03-09T08:52:30.597356+0000","last_peered":"2024-03-09T08:52:30.597356+0000","last_clean":"2024-03-09T08:52:30.597356+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T08:52:30.597356+0000","last_undegraded":"2024-03-09T08:52:30.597356+0000","last_fullsized":"2024-03-09T08:52:30.597356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:25:25.715084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948285+0000","last_change":"2024-03-09T08:52:29.603272+0000","last_active":"2024-03-09T08:52:30.948285+0000","last_peered":"2024-03-09T08:52:30.948285+0000","last_clean":"2024-03-09T08:52:30.948285+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T08:52:30.948285+0000","last_undegraded":"2024-03-09T08:52:30.948285+0000","last_fullsized":"2024-03-09T08:52:30.948285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:07:58.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576412+0000","last_change":"2024-03-09T08:52:18.243645+0000","last_active":"2024-03-09T08:52:30.576412+0000","last_peered":"2024-03-09T08:52:30.576412+0000","last_clean":"2024-03-09T08:52:30.576412+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T08:52:30.576412+0000","last_undegraded":"2024-03-09T08:52:30.576412+0000","last_fullsized":"2024-03-09T08:52:30.576412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:19.501675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576249+0000","last_change":"2024-03-09T08:52:29.587462+0000","last_active":"2024-03-09T08:52:30.576249+0000","last_peered":"2024-03-09T08:52:30.576249+0000","last_clean":"2024-03-09T08:52:30.576249+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T08:52:30.576249+0000","last_undegraded":"2024-03-09T08:52:30.576249+0000","last_fullsized":"2024-03-09T08:52:30.576249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:51.652225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577062+0000","last_change":"2024-03-09T08:52:18.233451+0000","last_active":"2024-03-09T08:52:30.577062+0000","last_peered":"2024-03-09T08:52:30.577062+0000","last_clean":"2024-03-09T08:52:30.577062+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T08:52:30.577062+0000","last_undegraded":"2024-03-09T08:52:30.577062+0000","last_fullsized":"2024-03-09T08:52:30.577062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:56:11.270081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942964+0000","last_change":"2024-03-09T08:52:29.599909+0000","last_active":"2024-03-09T08:52:30.942964+0000","last_peered":"2024-03-09T08:52:30.942964+0000","last_clean":"2024-03-09T08:52:30.942964+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T08:52:30.942964+0000","last_undegraded":"2024-03-09T08:52:30.942964+0000","last_fullsized":"2024-03-09T08:52:30.942964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:42.596067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577555+0000","last_change":"2024-03-09T08:52:18.234158+0000","last_active":"2024-03-09T08:52:30.577555+0000","last_peered":"2024-03-09T08:52:30.577555+0000","last_clean":"2024-03-09T08:52:30.577555+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T08:52:30.577555+0000","last_undegraded":"2024-03-09T08:52:30.577555+0000","last_fullsized":"2024-03-09T08:52:30.577555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:59:33.423558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944531+0000","last_change":"2024-03-09T08:52:29.597518+0000","last_active":"2024-03-09T08:52:30.944531+0000","last_peered":"2024-03-09T08:52:30.944531+0000","last_clean":"2024-03-09T08:52:30.944531+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T08:52:30.944531+0000","last_undegraded":"2024-03-09T08:52:30.944531+0000","last_fullsized":"2024-03-09T08:52:30.944531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:32:32.452993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577830+0000","last_change":"2024-03-09T08:52:18.234648+0000","last_active":"2024-03-09T08:52:30.577830+0000","last_peered":"2024-03-09T08:52:30.577830+0000","last_clean":"2024-03-09T08:52:30.577830+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T08:52:30.577830+0000","last_undegraded":"2024-03-09T08:52:30.577830+0000","last_fullsized":"2024-03-09T08:52:30.577830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:03.927280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944900+0000","last_change":"2024-03-09T08:52:29.580608+0000","last_active":"2024-03-09T08:52:30.944900+0000","last_peered":"2024-03-09T08:52:30.944900+0000","last_clean":"2024-03-09T08:52:30.944900+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T08:52:30.944900+0000","last_undegraded":"2024-03-09T08:52:30.944900+0000","last_fullsized":"2024-03-09T08:52:30.944900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:00:37.184351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948628+0000","last_change":"2024-03-09T08:52:18.253038+0000","last_active":"2024-03-09T08:52:30.948628+0000","last_peered":"2024-03-09T08:52:30.948628+0000","last_clean":"2024-03-09T08:52:30.948628+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T08:52:30.948628+0000","last_undegraded":"2024-03-09T08:52:30.948628+0000","last_fullsized":"2024-03-09T08:52:30.948628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:26:39.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577331+0000","last_change":"2024-03-09T08:52:29.578549+0000","last_active":"2024-03-09T08:52:30.577331+0000","last_peered":"2024-03-09T08:52:30.577331+0000","last_clean":"2024-03-09T08:52:30.577331+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T08:52:30.577331+0000","last_undegraded":"2024-03-09T08:52:30.577331+0000","last_fullsized":"2024-03-09T08:52:30.577331+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:42.775199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575758+0000","last_change":"2024-03-09T08:52:18.240975+0000","last_active":"2024-03-09T08:52:30.575758+0000","last_peered":"2024-03-09T08:52:30.575758+0000","last_clean":"2024-03-09T08:52:30.575758+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T08:52:30.575758+0000","last_undegraded":"2024-03-09T08:52:30.575758+0000","last_fullsized":"2024-03-09T08:52:30.575758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:30:09.363339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577959+0000","last_change":"2024-03-09T08:52:29.584159+0000","last_active":"2024-03-09T08:52:30.577959+0000","last_peered":"2024-03-09T08:52:30.577959+0000","last_clean":"2024-03-09T08:52:30.577959+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T08:52:30.577959+0000","last_undegraded":"2024-03-09T08:52:30.577959+0000","last_fullsized":"2024-03-09T08:52:30.577959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:01:41.661806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943943+0000","last_change":"2024-03-09T08:52:18.249901+0000","last_active":"2024-03-09T08:52:30.943943+0000","last_peered":"2024-03-09T08:52:30.943943+0000","last_clean":"2024-03-09T08:52:30.943943+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T08:52:30.943943+0000","last_undegraded":"2024-03-09T08:52:30.943943+0000","last_fullsized":"2024-03-09T08:52:30.943943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:09.947401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601289+0000","last_change":"2024-03-09T08:52:29.595184+0000","last_active":"2024-03-09T08:52:30.601289+0000","last_peered":"2024-03-09T08:52:30.601289+0000","last_clean":"2024-03-09T08:52:30.601289+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T08:52:30.601289+0000","last_undegraded":"2024-03-09T08:52:30.601289+0000","last_fullsized":"2024-03-09T08:52:30.601289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:34:27.662636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945364+0000","last_change":"2024-03-09T08:52:18.233561+0000","last_active":"2024-03-09T08:52:30.945364+0000","last_peered":"2024-03-09T08:52:30.945364+0000","last_clean":"2024-03-09T08:52:30.945364+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T08:52:30.945364+0000","last_undegraded":"2024-03-09T08:52:30.945364+0000","last_fullsized":"2024-03-09T08:52:30.945364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:15:54.027155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947171+0000","last_change":"2024-03-09T08:52:29.595771+0000","last_active":"2024-03-09T08:52:30.947171+0000","last_peered":"2024-03-09T08:52:30.947171+0000","last_clean":"2024-03-09T08:52:30.947171+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T08:52:30.947171+0000","last_undegraded":"2024-03-09T08:52:30.947171+0000","last_fullsized":"2024-03-09T08:52:30.947171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:23:48.898071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943880+0000","last_change":"2024-03-09T08:52:18.242311+0000","last_active":"2024-03-09T08:52:30.943880+0000","last_peered":"2024-03-09T08:52:30.943880+0000","last_clean":"2024-03-09T08:52:30.943880+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T08:52:30.943880+0000","last_undegraded":"2024-03-09T08:52:30.943880+0000","last_fullsized":"2024-03-09T08:52:30.943880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:48:15.559438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575834+0000","last_change":"2024-03-09T08:52:29.576792+0000","last_active":"2024-03-09T08:52:30.575834+0000","last_peered":"2024-03-09T08:52:30.575834+0000","last_clean":"2024-03-09T08:52:30.575834+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T08:52:30.575834+0000","last_undegraded":"2024-03-09T08:52:30.575834+0000","last_fullsized":"2024-03-09T08:52:30.575834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:21.364814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947241+0000","last_change":"2024-03-09T08:52:18.248341+0000","last_active":"2024-03-09T08:52:30.947241+0000","last_peered":"2024-03-09T08:52:30.947241+0000","last_clean":"2024-03-09T08:52:30.947241+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T08:52:30.947241+0000","last_undegraded":"2024-03-09T08:52:30.947241+0000","last_fullsized":"2024-03-09T08:52:30.947241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:18:02.716066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942730+0000","last_change":"2024-03-09T08:52:29.606330+0000","last_active":"2024-03-09T08:52:30.942730+0000","last_peered":"2024-03-09T08:52:30.942730+0000","last_clean":"2024-03-09T08:52:30.942730+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T08:52:30.942730+0000","last_undegraded":"2024-03-09T08:52:30.942730+0000","last_fullsized":"2024-03-09T08:52:30.942730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:44:50.347124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597559+0000","last_change":"2024-03-09T08:52:18.222968+0000","last_active":"2024-03-09T08:52:30.597559+0000","last_peered":"2024-03-09T08:52:30.597559+0000","last_clean":"2024-03-09T08:52:30.597559+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T08:52:30.597559+0000","last_undegraded":"2024-03-09T08:52:30.597559+0000","last_fullsized":"2024-03-09T08:52:30.597559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:52:29.523963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943337+0000","last_change":"2024-03-09T08:52:29.591813+0000","last_active":"2024-03-09T08:52:30.943337+0000","last_peered":"2024-03-09T08:52:30.943337+0000","last_clean":"2024-03-09T08:52:30.943337+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T08:52:30.943337+0000","last_undegraded":"2024-03-09T08:52:30.943337+0000","last_fullsized":"2024-03-09T08:52:30.943337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:40:31.568253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601942+0000","last_change":"2024-03-09T08:52:18.225812+0000","last_active":"2024-03-09T08:52:30.601942+0000","last_peered":"2024-03-09T08:52:30.601942+0000","last_clean":"2024-03-09T08:52:30.601942+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T08:52:30.601942+0000","last_undegraded":"2024-03-09T08:52:30.601942+0000","last_fullsized":"2024-03-09T08:52:30.601942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:09:32.044185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943686+0000","last_change":"2024-03-09T08:52:29.595055+0000","last_active":"2024-03-09T08:52:30.943686+0000","last_peered":"2024-03-09T08:52:30.943686+0000","last_clean":"2024-03-09T08:52:30.943686+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T08:52:30.943686+0000","last_undegraded":"2024-03-09T08:52:30.943686+0000","last_fullsized":"2024-03-09T08:52:30.943686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:53.209169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948709+0000","last_change":"2024-03-09T08:52:18.248973+0000","last_active":"2024-03-09T08:52:30.948709+0000","last_peered":"2024-03-09T08:52:30.948709+0000","last_clean":"2024-03-09T08:52:30.948709+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T08:52:30.948709+0000","last_undegraded":"2024-03-09T08:52:30.948709+0000","last_fullsized":"2024-03-09T08:52:30.948709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:09:11.372616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":101489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":102507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T08:52:35.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T08:52:35.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.322+0000 7fbfd6600640 1 Processor -- start 2024-03-09T08:52:35.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.322+0000 7fbfd6600640 1 -- start start 2024-03-09T08:52:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.323+0000 7fbfd6600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbfd0072aa0 0x7fbfd0072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.323+0000 7fbfd6600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbfd0150d20 0x7fbfd0151180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.323+0000 7fbfd6600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd00c5be0 con 0x7fbfd006fab0 2024-03-09T08:52:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.323+0000 7fbfd6600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd0154780 con 0x7fbfd0150d20 2024-03-09T08:52:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.323+0000 7fbfd6600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd00c3fe0 con 0x7fbfd0072aa0 2024-03-09T08:52:35.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.323+0000 7fbfd5c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbfd006fab0 0x7fbfd006fec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54416/0 (socket says 172.21.15.67:54416) 2024-03-09T08:52:35.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.323+0000 7fbfd5c00640 1 -- 172.21.15.67:0/14041236 learned_addr learned my addr 172.21.15.67:0/14041236 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:35.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.323+0000 7fbfcfe00640 1 --2- 172.21.15.67:0/14041236 >> v2:172.21.15.67:3300/0 conn(0x7fbfd0150d20 0x7fbfd0151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.324+0000 7fbfd5200640 1 --2- 172.21.15.67:0/14041236 >> v2:172.21.15.134:3300/0 conn(0x7fbfd0072aa0 0x7fbfd0072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.324+0000 7fbfcf400640 1 -- 172.21.15.67:0/14041236 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741250020 0 0) 0x7fbfd00c5be0 con 0x7fbfd006fab0 2024-03-09T08:52:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.324+0000 7fbfcf400640 1 -- 172.21.15.67:0/14041236 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb8003520 con 0x7fbfd006fab0 2024-03-09T08:52:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.324+0000 7fbfcfe00640 1 -- 172.21.15.67:0/14041236 >> v2:172.21.15.134:3300/0 conn(0x7fbfd0072aa0 msgr2=0x7fbfd0072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.324+0000 7fbfcfe00640 1 --2- 172.21.15.67:0/14041236 >> v2:172.21.15.134:3300/0 conn(0x7fbfd0072aa0 0x7fbfd0072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.324+0000 7fbfcfe00640 1 -- 172.21.15.67:0/14041236 >> v1:172.21.15.67:6789/0 conn(0x7fbfd006fab0 legacy=0x7fbfd006fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.324+0000 7fbfcfe00640 1 -- 172.21.15.67:0/14041236 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd00c42c0 con 0x7fbfd0150d20 2024-03-09T08:52:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.325+0000 7fbfcfe00640 1 --2- 172.21.15.67:0/14041236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd0150d20 0x7fbfd0151180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0071340 tx=0x7fbfbc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7369dca388833064 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.325+0000 7fbfcf400640 1 -- 172.21.15.67:0/14041236 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0315a0 con 0x7fbfd0150d20 2024-03-09T08:52:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.325+0000 7fbfcf400640 1 -- 172.21.15.67:0/14041236 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc037070 con 0x7fbfd0150d20 2024-03-09T08:52:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.325+0000 7fbfcf400640 1 -- 172.21.15.67:0/14041236 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc031960 con 0x7fbfd0150d20 2024-03-09T08:52:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.326+0000 7fbfd6600640 1 -- 172.21.15.67:0/14041236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd0150d20 msgr2=0x7fbfd0151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.326+0000 7fbfd6600640 1 --2- 172.21.15.67:0/14041236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd0150d20 0x7fbfd0151180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0071340 tx=0x7fbfbc02b840 comp rx=0 tx=0).stop 2024-03-09T08:52:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.326+0000 7fbfd6600640 1 -- 172.21.15.67:0/14041236 shutdown_connections 2024-03-09T08:52:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.326+0000 7fbfd6600640 1 --2- 172.21.15.67:0/14041236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd0150d20 0x7fbfd0151180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.326+0000 7fbfd6600640 1 --2- 172.21.15.67:0/14041236 >> v2:172.21.15.134:3300/0 conn(0x7fbfd0072aa0 0x7fbfd0072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.326+0000 7fbfd6600640 1 -- 172.21.15.67:0/14041236 >> 172.21.15.67:0/14041236 conn(0x7fbfd00c9400 msgr2=0x7fbfd00c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.326+0000 7fbfd6600640 1 -- 172.21.15.67:0/14041236 shutdown_connections 2024-03-09T08:52:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.326+0000 7fbfd6600640 1 -- 172.21.15.67:0/14041236 wait complete. 2024-03-09T08:52:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.328+0000 7fbfd6600640 1 Processor -- start 2024-03-09T08:52:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.328+0000 7fbfd6600640 1 -- start start 2024-03-09T08:52:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfd006fab0 0x7fbfd014bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfd0072aa0 0x7fbfd01424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd6600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd0150d20 0x7fbfd01ec860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd6600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0070750 con 0x7fbfd0150d20 2024-03-09T08:52:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd6600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd00705d0 con 0x7fbfd006fab0 2024-03-09T08:52:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd6600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0073cd0 con 0x7fbfd0072aa0 2024-03-09T08:52:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd5c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd0150d20 0x7fbfd01ec860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd5c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd0150d20 0x7fbfd01ec860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51344/0 (socket says 172.21.15.67:51344) 2024-03-09T08:52:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd5c00640 1 -- 172.21.15.67:0/2401240123 learned_addr learned my addr 172.21.15.67:0/2401240123 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd5200640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfd006fab0 0x7fbfd014bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfcfe00640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfd0072aa0 0x7fbfd01424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd5c00640 1 -- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfd0072aa0 msgr2=0x7fbfd01424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd5c00640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfd0072aa0 0x7fbfd01424f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd5c00640 1 -- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfd006fab0 msgr2=0x7fbfd014bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd5c00640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfd006fab0 0x7fbfd014bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.329+0000 7fbfd5c00640 1 -- 172.21.15.67:0/2401240123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd014d9b0 con 0x7fbfd0150d20 2024-03-09T08:52:35.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.330+0000 7fbfd5c00640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd0150d20 0x7fbfd01ec860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800cae0 tx=0x7fbfc800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:35.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.330+0000 7fbfccc00640 1 -- 172.21.15.67:0/2401240123 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8017070 con 0x7fbfd0150d20 2024-03-09T08:52:35.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.330+0000 7fbfccc00640 1 -- 172.21.15.67:0/2401240123 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc800abe0 con 0x7fbfd0150d20 2024-03-09T08:52:35.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.330+0000 7fbfd6600640 1 -- 172.21.15.67:0/2401240123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd014dc90 con 0x7fbfd0150d20 2024-03-09T08:52:35.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.330+0000 7fbfccc00640 1 -- 172.21.15.67:0/2401240123 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8012600 con 0x7fbfd0150d20 2024-03-09T08:52:35.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.330+0000 7fbfd6600640 1 -- 172.21.15.67:0/2401240123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd01ec340 con 0x7fbfd0150d20 2024-03-09T08:52:35.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.331+0000 7fbfd6600640 1 -- 172.21.15.67:0/2401240123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd004f3c0 con 0x7fbfd0150d20 2024-03-09T08:52:35.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.332+0000 7fbfccc00640 1 -- 172.21.15.67:0/2401240123 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbfc801a070 con 0x7fbfd0150d20 2024-03-09T08:52:35.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.334+0000 7fbfccc00640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbfb006b4d0 0x7fbfb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.334+0000 7fbfccc00640 1 -- 172.21.15.67:0/2401240123 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fbfc808a150 con 0x7fbfd0150d20 2024-03-09T08:52:35.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.334+0000 7fbfd5200640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbfb006b4d0 0x7fbfb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.335+0000 7fbfd5200640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbfb006b4d0 0x7fbfb006d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009aa0 tx=0x7fbfc4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:35.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.338+0000 7fbfccc00640 1 -- 172.21.15.67:0/2401240123 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbfc8058640 con 0x7fbfd0150d20 2024-03-09T08:52:35.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.533+0000 7fbfd6600640 1 -- 172.21.15.67:0/2401240123 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfd0142f60 con 0x7fbfb006b4d0 2024-03-09T08:52:35.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.567+0000 7fbfccc00640 1 -- 172.21.15.67:0/2401240123 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490417 (secure 0 0 0) 0x7fbfc4011a00 con 0x7fbfb006b4d0 2024-03-09T08:52:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T08:52:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.576+0000 7fbfd6600640 1 -- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbfb006b4d0 msgr2=0x7fbfb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.576+0000 7fbfd6600640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbfb006b4d0 0x7fbfb006d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009aa0 tx=0x7fbfc4009190 comp rx=0 tx=0).stop 2024-03-09T08:52:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.576+0000 7fbfd6600640 1 -- 172.21.15.67:0/2401240123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd0150d20 msgr2=0x7fbfd01ec860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.576+0000 7fbfd6600640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd0150d20 0x7fbfd01ec860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800cae0 tx=0x7fbfc800a060 comp rx=0 tx=0).stop 2024-03-09T08:52:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.576+0000 7fbfd6600640 1 -- 172.21.15.67:0/2401240123 shutdown_connections 2024-03-09T08:52:35.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.576+0000 7fbfd6600640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbfb006b4d0 0x7fbfb006d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.576+0000 7fbfd6600640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd0150d20 0x7fbfd01ec860 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.577+0000 7fbfd6600640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfd0072aa0 0x7fbfd01424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.577+0000 7fbfd6600640 1 --2- 172.21.15.67:0/2401240123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfd006fab0 0x7fbfd014bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.577+0000 7fbfd6600640 1 -- 172.21.15.67:0/2401240123 >> 172.21.15.67:0/2401240123 conn(0x7fbfd00c9400 msgr2=0x7fbfd006fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.577+0000 7fbfd6600640 1 -- 172.21.15.67:0/2401240123 shutdown_connections 2024-03-09T08:52:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.577+0000 7fbfd6600640 1 -- 172.21.15.67:0/2401240123 wait complete. 2024-03-09T08:52:35.593 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-03-09T08:52:34.679036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":323,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3350528,"data_stored":1984464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.129679"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575471+0000","last_change":"2024-03-09T08:52:29.573571+0000","last_active":"2024-03-09T08:52:30.575471+0000","last_peered":"2024-03-09T08:52:30.575471+0000","last_clean":"2024-03-09T08:52:30.575471+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T08:52:30.575471+0000","last_undegraded":"2024-03-09T08:52:30.575471+0000","last_fullsized":"2024-03-09T08:52:30.575471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:33:33.068463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598024+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T08:52:30.598024+0000","last_peered":"2024-03-09T08:52:30.598024+0000","last_clean":"2024-03-09T08:52:30.598024+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T08:52:30.598024+0000","last_undegraded":"2024-03-09T08:52:30.598024+0000","last_fullsized":"2024-03-09T08:52:30.598024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:41:40.332687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577727+0000","last_change":"2024-03-09T08:52:18.235099+0000","last_active":"2024-03-09T08:52:30.577727+0000","last_peered":"2024-03-09T08:52:30.577727+0000","last_clean":"2024-03-09T08:52:30.577727+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T08:52:30.577727+0000","last_undegraded":"2024-03-09T08:52:30.577727+0000","last_fullsized":"2024-03-09T08:52:30.577727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:39:28.882461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575426+0000","last_change":"2024-03-09T08:52:28.571905+0000","last_active":"2024-03-09T08:52:30.575426+0000","last_peered":"2024-03-09T08:52:30.575426+0000","last_clean":"2024-03-09T08:52:30.575426+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T08:52:30.575426+0000","last_undegraded":"2024-03-09T08:52:30.575426+0000","last_fullsized":"2024-03-09T08:52:30.575426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:01.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.600985+0000","last_change":"2024-03-09T08:52:29.578644+0000","last_active":"2024-03-09T08:52:30.600985+0000","last_peered":"2024-03-09T08:52:30.600985+0000","last_clean":"2024-03-09T08:52:30.600985+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T08:52:30.600985+0000","last_undegraded":"2024-03-09T08:52:30.600985+0000","last_fullsized":"2024-03-09T08:52:30.600985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:00.044620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577762+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T08:52:30.577762+0000","last_peered":"2024-03-09T08:52:30.577762+0000","last_clean":"2024-03-09T08:52:30.577762+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T08:52:30.577762+0000","last_undegraded":"2024-03-09T08:52:30.577762+0000","last_fullsized":"2024-03-09T08:52:30.577762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597762+0000","last_change":"2024-03-09T08:52:18.236946+0000","last_active":"2024-03-09T08:52:30.597762+0000","last_peered":"2024-03-09T08:52:30.597762+0000","last_clean":"2024-03-09T08:52:30.597762+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T08:52:30.597762+0000","last_undegraded":"2024-03-09T08:52:30.597762+0000","last_fullsized":"2024-03-09T08:52:30.597762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:45.372160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575237+0000","last_change":"2024-03-09T08:52:28.573074+0000","last_active":"2024-03-09T08:52:30.575237+0000","last_peered":"2024-03-09T08:52:30.575237+0000","last_clean":"2024-03-09T08:52:30.575237+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T08:52:30.575237+0000","last_undegraded":"2024-03-09T08:52:30.575237+0000","last_fullsized":"2024-03-09T08:52:30.575237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:50.600408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945199+0000","last_change":"2024-03-09T08:52:29.594030+0000","last_active":"2024-03-09T08:52:30.945199+0000","last_peered":"2024-03-09T08:52:30.945199+0000","last_clean":"2024-03-09T08:52:30.945199+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T08:52:30.945199+0000","last_undegraded":"2024-03-09T08:52:30.945199+0000","last_fullsized":"2024-03-09T08:52:30.945199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:57.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948441+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T08:52:30.948441+0000","last_peered":"2024-03-09T08:52:30.948441+0000","last_clean":"2024-03-09T08:52:30.948441+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T08:52:30.948441+0000","last_undegraded":"2024-03-09T08:52:30.948441+0000","last_fullsized":"2024-03-09T08:52:30.948441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947481+0000","last_change":"2024-03-09T08:52:18.230629+0000","last_active":"2024-03-09T08:52:30.947481+0000","last_peered":"2024-03-09T08:52:30.947481+0000","last_clean":"2024-03-09T08:52:30.947481+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T08:52:30.947481+0000","last_undegraded":"2024-03-09T08:52:30.947481+0000","last_fullsized":"2024-03-09T08:52:30.947481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:46:38.528839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576909+0000","last_change":"2024-03-09T08:52:28.577128+0000","last_active":"2024-03-09T08:52:30.576909+0000","last_peered":"2024-03-09T08:52:30.576909+0000","last_clean":"2024-03-09T08:52:30.576909+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T08:52:30.576909+0000","last_undegraded":"2024-03-09T08:52:30.576909+0000","last_fullsized":"2024-03-09T08:52:30.576909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:41.801752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946024+0000","last_change":"2024-03-09T08:52:29.581705+0000","last_active":"2024-03-09T08:52:30.946024+0000","last_peered":"2024-03-09T08:52:30.946024+0000","last_clean":"2024-03-09T08:52:30.946024+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T08:52:30.946024+0000","last_undegraded":"2024-03-09T08:52:30.946024+0000","last_fullsized":"2024-03-09T08:52:30.946024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:27.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947791+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T08:52:30.947791+0000","last_peered":"2024-03-09T08:52:30.947791+0000","last_clean":"2024-03-09T08:52:30.947791+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T08:52:30.947791+0000","last_undegraded":"2024-03-09T08:52:30.947791+0000","last_fullsized":"2024-03-09T08:52:30.947791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947372+0000","last_change":"2024-03-09T08:52:18.228419+0000","last_active":"2024-03-09T08:52:30.947372+0000","last_peered":"2024-03-09T08:52:30.947372+0000","last_clean":"2024-03-09T08:52:30.947372+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T08:52:30.947372+0000","last_undegraded":"2024-03-09T08:52:30.947372+0000","last_fullsized":"2024-03-09T08:52:30.947372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:19:12.533402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577499+0000","last_change":"2024-03-09T08:52:28.577069+0000","last_active":"2024-03-09T08:52:30.577499+0000","last_peered":"2024-03-09T08:52:30.577499+0000","last_clean":"2024-03-09T08:52:30.577499+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T08:52:30.577499+0000","last_undegraded":"2024-03-09T08:52:30.577499+0000","last_fullsized":"2024-03-09T08:52:30.577499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:09:02.410915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601446+0000","last_change":"2024-03-09T08:52:29.588922+0000","last_active":"2024-03-09T08:52:30.601446+0000","last_peered":"2024-03-09T08:52:30.601446+0000","last_clean":"2024-03-09T08:52:30.601446+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T08:52:30.601446+0000","last_undegraded":"2024-03-09T08:52:30.601446+0000","last_fullsized":"2024-03-09T08:52:30.601446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:13.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576353+0000","last_change":"2024-03-09T08:52:18.234372+0000","last_active":"2024-03-09T08:52:30.576353+0000","last_peered":"2024-03-09T08:52:30.576353+0000","last_clean":"2024-03-09T08:52:30.576353+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T08:52:30.576353+0000","last_undegraded":"2024-03-09T08:52:30.576353+0000","last_fullsized":"2024-03-09T08:52:30.576353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:09:33.422907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944406+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T08:52:30.944406+0000","last_peered":"2024-03-09T08:52:30.944406+0000","last_clean":"2024-03-09T08:52:30.944406+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T08:52:30.944406+0000","last_undegraded":"2024-03-09T08:52:30.944406+0000","last_fullsized":"2024-03-09T08:52:30.944406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578381+0000","last_change":"2024-03-09T08:52:28.570631+0000","last_active":"2024-03-09T08:52:30.578381+0000","last_peered":"2024-03-09T08:52:30.578381+0000","last_clean":"2024-03-09T08:52:30.578381+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T08:52:30.578381+0000","last_undegraded":"2024-03-09T08:52:30.578381+0000","last_fullsized":"2024-03-09T08:52:30.578381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:06.630704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943131+0000","last_change":"2024-03-09T08:52:29.596820+0000","last_active":"2024-03-09T08:52:30.943131+0000","last_peered":"2024-03-09T08:52:30.943131+0000","last_clean":"2024-03-09T08:52:30.943131+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T08:52:30.943131+0000","last_undegraded":"2024-03-09T08:52:30.943131+0000","last_fullsized":"2024-03-09T08:52:30.943131+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:01:20.331889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576713+0000","last_change":"2024-03-09T08:52:18.244306+0000","last_active":"2024-03-09T08:52:30.576713+0000","last_peered":"2024-03-09T08:52:30.576713+0000","last_clean":"2024-03-09T08:52:30.576713+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T08:52:30.576713+0000","last_undegraded":"2024-03-09T08:52:30.576713+0000","last_fullsized":"2024-03-09T08:52:30.576713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:58.365524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601528+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T08:52:30.601528+0000","last_peered":"2024-03-09T08:52:30.601528+0000","last_clean":"2024-03-09T08:52:30.601528+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T08:52:30.601528+0000","last_undegraded":"2024-03-09T08:52:30.601528+0000","last_fullsized":"2024-03-09T08:52:30.601528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:45:45.612953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576699+0000","last_change":"2024-03-09T08:52:28.577110+0000","last_active":"2024-03-09T08:52:30.576699+0000","last_peered":"2024-03-09T08:52:30.576699+0000","last_clean":"2024-03-09T08:52:30.576699+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T08:52:30.576699+0000","last_undegraded":"2024-03-09T08:52:30.576699+0000","last_fullsized":"2024-03-09T08:52:30.576699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:57.197373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579257+0000","last_change":"2024-03-09T08:52:29.579929+0000","last_active":"2024-03-09T08:52:30.579257+0000","last_peered":"2024-03-09T08:52:30.579257+0000","last_clean":"2024-03-09T08:52:30.579257+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T08:52:30.579257+0000","last_undegraded":"2024-03-09T08:52:30.579257+0000","last_fullsized":"2024-03-09T08:52:30.579257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:37:23.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576526+0000","last_change":"2024-03-09T08:52:18.233208+0000","last_active":"2024-03-09T08:52:30.576526+0000","last_peered":"2024-03-09T08:52:30.576526+0000","last_clean":"2024-03-09T08:52:30.576526+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T08:52:30.576526+0000","last_undegraded":"2024-03-09T08:52:30.576526+0000","last_fullsized":"2024-03-09T08:52:30.576526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:17.589783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601269+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T08:52:30.601269+0000","last_peered":"2024-03-09T08:52:30.601269+0000","last_clean":"2024-03-09T08:52:30.601269+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T08:52:30.601269+0000","last_undegraded":"2024-03-09T08:52:30.601269+0000","last_fullsized":"2024-03-09T08:52:30.601269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:21:59.990396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602237+0000","last_change":"2024-03-09T08:52:28.576254+0000","last_active":"2024-03-09T08:52:30.602237+0000","last_peered":"2024-03-09T08:52:30.602237+0000","last_clean":"2024-03-09T08:52:30.602237+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T08:52:30.602237+0000","last_undegraded":"2024-03-09T08:52:30.602237+0000","last_fullsized":"2024-03-09T08:52:30.602237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:21:56.330955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946687+0000","last_change":"2024-03-09T08:52:29.578272+0000","last_active":"2024-03-09T08:52:30.946687+0000","last_peered":"2024-03-09T08:52:30.946687+0000","last_clean":"2024-03-09T08:52:30.946687+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T08:52:30.946687+0000","last_undegraded":"2024-03-09T08:52:30.946687+0000","last_fullsized":"2024-03-09T08:52:30.946687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:56.180809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576191+0000","last_change":"2024-03-09T08:52:18.217046+0000","last_active":"2024-03-09T08:52:30.576191+0000","last_peered":"2024-03-09T08:52:30.576191+0000","last_clean":"2024-03-09T08:52:30.576191+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T08:52:30.576191+0000","last_undegraded":"2024-03-09T08:52:30.576191+0000","last_fullsized":"2024-03-09T08:52:30.576191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:14.884963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943757+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T08:52:30.943757+0000","last_peered":"2024-03-09T08:52:30.943757+0000","last_clean":"2024-03-09T08:52:30.943757+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T08:52:30.943757+0000","last_undegraded":"2024-03-09T08:52:30.943757+0000","last_fullsized":"2024-03-09T08:52:30.943757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578428+0000","last_change":"2024-03-09T08:52:28.573638+0000","last_active":"2024-03-09T08:52:30.578428+0000","last_peered":"2024-03-09T08:52:30.578428+0000","last_clean":"2024-03-09T08:52:30.578428+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T08:52:30.578428+0000","last_undegraded":"2024-03-09T08:52:30.578428+0000","last_fullsized":"2024-03-09T08:52:30.578428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:02.478112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575514+0000","last_change":"2024-03-09T08:52:29.597697+0000","last_active":"2024-03-09T08:52:30.575514+0000","last_peered":"2024-03-09T08:52:30.575514+0000","last_clean":"2024-03-09T08:52:30.575514+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T08:52:30.575514+0000","last_undegraded":"2024-03-09T08:52:30.575514+0000","last_fullsized":"2024-03-09T08:52:30.575514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:00.890845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598142+0000","last_change":"2024-03-09T08:52:18.229739+0000","last_active":"2024-03-09T08:52:30.598142+0000","last_peered":"2024-03-09T08:52:30.598142+0000","last_clean":"2024-03-09T08:52:30.598142+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T08:52:30.598142+0000","last_undegraded":"2024-03-09T08:52:30.598142+0000","last_fullsized":"2024-03-09T08:52:30.598142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:24.394779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579139+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T08:52:30.579139+0000","last_peered":"2024-03-09T08:52:30.579139+0000","last_clean":"2024-03-09T08:52:30.579139+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T08:52:30.579139+0000","last_undegraded":"2024-03-09T08:52:30.579139+0000","last_fullsized":"2024-03-09T08:52:30.579139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:35:38.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575645+0000","last_change":"2024-03-09T08:52:28.575630+0000","last_active":"2024-03-09T08:52:30.575645+0000","last_peered":"2024-03-09T08:52:30.575645+0000","last_clean":"2024-03-09T08:52:30.575645+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T08:52:30.575645+0000","last_undegraded":"2024-03-09T08:52:30.575645+0000","last_fullsized":"2024-03-09T08:52:30.575645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:48.210921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575362+0000","last_change":"2024-03-09T08:52:29.576677+0000","last_active":"2024-03-09T08:52:30.575362+0000","last_peered":"2024-03-09T08:52:30.575362+0000","last_clean":"2024-03-09T08:52:30.575362+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T08:52:30.575362+0000","last_undegraded":"2024-03-09T08:52:30.575362+0000","last_fullsized":"2024-03-09T08:52:30.575362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:19.026873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577342+0000","last_change":"2024-03-09T08:52:18.245442+0000","last_active":"2024-03-09T08:52:30.577342+0000","last_peered":"2024-03-09T08:52:30.577342+0000","last_clean":"2024-03-09T08:52:30.577342+0000","last_became_active":"2024-03-09T08:52:18.245164+0000","last_became_peered":"2024-03-09T08:52:18.245164+0000","last_unstale":"2024-03-09T08:52:30.577342+0000","last_undegraded":"2024-03-09T08:52:30.577342+0000","last_fullsized":"2024-03-09T08:52:30.577342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:31:08.413041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602103+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T08:52:30.602103+0000","last_peered":"2024-03-09T08:52:30.602103+0000","last_clean":"2024-03-09T08:52:30.602103+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T08:52:30.602103+0000","last_undegraded":"2024-03-09T08:52:30.602103+0000","last_fullsized":"2024-03-09T08:52:30.602103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:30.201471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575392+0000","last_change":"2024-03-09T08:52:28.586702+0000","last_active":"2024-03-09T08:52:30.575392+0000","last_peered":"2024-03-09T08:52:30.575392+0000","last_clean":"2024-03-09T08:52:30.575392+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T08:52:30.575392+0000","last_undegraded":"2024-03-09T08:52:30.575392+0000","last_fullsized":"2024-03-09T08:52:30.575392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:09.201224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577654+0000","last_change":"2024-03-09T08:52:29.578956+0000","last_active":"2024-03-09T08:52:30.577654+0000","last_peered":"2024-03-09T08:52:30.577654+0000","last_clean":"2024-03-09T08:52:30.577654+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T08:52:30.577654+0000","last_undegraded":"2024-03-09T08:52:30.577654+0000","last_fullsized":"2024-03-09T08:52:30.577654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:00:00.884211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601530+0000","last_change":"2024-03-09T08:52:18.234887+0000","last_active":"2024-03-09T08:52:30.601530+0000","last_peered":"2024-03-09T08:52:30.601530+0000","last_clean":"2024-03-09T08:52:30.601530+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T08:52:30.601530+0000","last_undegraded":"2024-03-09T08:52:30.601530+0000","last_fullsized":"2024-03-09T08:52:30.601530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:44.661882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948828+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T08:52:30.948828+0000","last_peered":"2024-03-09T08:52:30.948828+0000","last_clean":"2024-03-09T08:52:30.948828+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T08:52:30.948828+0000","last_undegraded":"2024-03-09T08:52:30.948828+0000","last_fullsized":"2024-03-09T08:52:30.948828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597558+0000","last_change":"2024-03-09T08:52:28.580879+0000","last_active":"2024-03-09T08:52:30.597558+0000","last_peered":"2024-03-09T08:52:30.597558+0000","last_clean":"2024-03-09T08:52:30.597558+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T08:52:30.597558+0000","last_undegraded":"2024-03-09T08:52:30.597558+0000","last_fullsized":"2024-03-09T08:52:30.597558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:19:44.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598143+0000","last_change":"2024-03-09T08:52:29.580024+0000","last_active":"2024-03-09T08:52:30.598143+0000","last_peered":"2024-03-09T08:52:30.598143+0000","last_clean":"2024-03-09T08:52:30.598143+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T08:52:30.598143+0000","last_undegraded":"2024-03-09T08:52:30.598143+0000","last_fullsized":"2024-03-09T08:52:30.598143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:02:13.195642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947463+0000","last_change":"2024-03-09T08:52:18.250999+0000","last_active":"2024-03-09T08:52:30.947463+0000","last_peered":"2024-03-09T08:52:30.947463+0000","last_clean":"2024-03-09T08:52:30.947463+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T08:52:30.947463+0000","last_undegraded":"2024-03-09T08:52:30.947463+0000","last_fullsized":"2024-03-09T08:52:30.947463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:49:54.496603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601554+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T08:52:30.601554+0000","last_peered":"2024-03-09T08:52:30.601554+0000","last_clean":"2024-03-09T08:52:30.601554+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T08:52:30.601554+0000","last_undegraded":"2024-03-09T08:52:30.601554+0000","last_fullsized":"2024-03-09T08:52:30.601554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601366+0000","last_change":"2024-03-09T08:52:28.584457+0000","last_active":"2024-03-09T08:52:30.601366+0000","last_peered":"2024-03-09T08:52:30.601366+0000","last_clean":"2024-03-09T08:52:30.601366+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T08:52:30.601366+0000","last_undegraded":"2024-03-09T08:52:30.601366+0000","last_fullsized":"2024-03-09T08:52:30.601366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:45.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597465+0000","last_change":"2024-03-09T08:52:29.596486+0000","last_active":"2024-03-09T08:52:30.597465+0000","last_peered":"2024-03-09T08:52:30.597465+0000","last_clean":"2024-03-09T08:52:30.597465+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T08:52:30.597465+0000","last_undegraded":"2024-03-09T08:52:30.597465+0000","last_fullsized":"2024-03-09T08:52:30.597465+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:40:16.587782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577604+0000","last_change":"2024-03-09T08:52:18.227365+0000","last_active":"2024-03-09T08:52:30.577604+0000","last_peered":"2024-03-09T08:52:30.577604+0000","last_clean":"2024-03-09T08:52:30.577604+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T08:52:30.577604+0000","last_undegraded":"2024-03-09T08:52:30.577604+0000","last_fullsized":"2024-03-09T08:52:30.577604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:26.357063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945950+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T08:52:30.945950+0000","last_peered":"2024-03-09T08:52:30.945950+0000","last_clean":"2024-03-09T08:52:30.945950+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T08:52:30.945950+0000","last_undegraded":"2024-03-09T08:52:30.945950+0000","last_fullsized":"2024-03-09T08:52:30.945950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601516+0000","last_change":"2024-03-09T08:52:28.578096+0000","last_active":"2024-03-09T08:52:30.601516+0000","last_peered":"2024-03-09T08:52:30.601516+0000","last_clean":"2024-03-09T08:52:30.601516+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T08:52:30.601516+0000","last_undegraded":"2024-03-09T08:52:30.601516+0000","last_fullsized":"2024-03-09T08:52:30.601516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:42.645517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943375+0000","last_change":"2024-03-09T08:52:29.590834+0000","last_active":"2024-03-09T08:52:30.943375+0000","last_peered":"2024-03-09T08:52:30.943375+0000","last_clean":"2024-03-09T08:52:30.943375+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T08:52:30.943375+0000","last_undegraded":"2024-03-09T08:52:30.943375+0000","last_fullsized":"2024-03-09T08:52:30.943375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:24:06.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.600924+0000","last_change":"2024-03-09T08:52:18.235832+0000","last_active":"2024-03-09T08:52:30.600924+0000","last_peered":"2024-03-09T08:52:30.600924+0000","last_clean":"2024-03-09T08:52:30.600924+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T08:52:30.600924+0000","last_undegraded":"2024-03-09T08:52:30.600924+0000","last_fullsized":"2024-03-09T08:52:30.600924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:38:58.751526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942505+0000","last_change":"2024-03-09T08:52:17.271847+0000","last_active":"2024-03-09T08:52:30.942505+0000","last_peered":"2024-03-09T08:52:30.942505+0000","last_clean":"2024-03-09T08:52:30.942505+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T08:52:30.942505+0000","last_undegraded":"2024-03-09T08:52:30.942505+0000","last_fullsized":"2024-03-09T08:52:30.942505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:27.418674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602301+0000","last_change":"2024-03-09T08:52:28.588636+0000","last_active":"2024-03-09T08:52:30.602301+0000","last_peered":"2024-03-09T08:52:30.602301+0000","last_clean":"2024-03-09T08:52:30.602301+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T08:52:30.602301+0000","last_undegraded":"2024-03-09T08:52:30.602301+0000","last_fullsized":"2024-03-09T08:52:30.602301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:21.601182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947090+0000","last_change":"2024-03-09T08:52:29.606833+0000","last_active":"2024-03-09T08:52:30.947090+0000","last_peered":"2024-03-09T08:52:30.947090+0000","last_clean":"2024-03-09T08:52:30.947090+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T08:52:30.947090+0000","last_undegraded":"2024-03-09T08:52:30.947090+0000","last_fullsized":"2024-03-09T08:52:30.947090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:40.299681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597339+0000","last_change":"2024-03-09T08:52:18.214396+0000","last_active":"2024-03-09T08:52:30.597339+0000","last_peered":"2024-03-09T08:52:30.597339+0000","last_clean":"2024-03-09T08:52:30.597339+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T08:52:30.597339+0000","last_undegraded":"2024-03-09T08:52:30.597339+0000","last_fullsized":"2024-03-09T08:52:30.597339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:24:10.719351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944007+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T08:52:30.944007+0000","last_peered":"2024-03-09T08:52:30.944007+0000","last_clean":"2024-03-09T08:52:30.944007+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T08:52:30.944007+0000","last_undegraded":"2024-03-09T08:52:30.944007+0000","last_fullsized":"2024-03-09T08:52:30.944007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.599908+0000","last_change":"2024-03-09T08:52:28.578599+0000","last_active":"2024-03-09T08:52:30.599908+0000","last_peered":"2024-03-09T08:52:30.599908+0000","last_clean":"2024-03-09T08:52:30.599908+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T08:52:30.599908+0000","last_undegraded":"2024-03-09T08:52:30.599908+0000","last_fullsized":"2024-03-09T08:52:30.599908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:52:31.328248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943920+0000","last_change":"2024-03-09T08:52:29.591872+0000","last_active":"2024-03-09T08:52:30.943920+0000","last_peered":"2024-03-09T08:52:30.943920+0000","last_clean":"2024-03-09T08:52:30.943920+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T08:52:30.943920+0000","last_undegraded":"2024-03-09T08:52:30.943920+0000","last_fullsized":"2024-03-09T08:52:30.943920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:44:17.870183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576290+0000","last_change":"2024-03-09T08:52:18.235367+0000","last_active":"2024-03-09T08:52:30.576290+0000","last_peered":"2024-03-09T08:52:30.576290+0000","last_clean":"2024-03-09T08:52:30.576290+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T08:52:30.576290+0000","last_undegraded":"2024-03-09T08:52:30.576290+0000","last_fullsized":"2024-03-09T08:52:30.576290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:40.736440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946321+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T08:52:30.946321+0000","last_peered":"2024-03-09T08:52:30.946321+0000","last_clean":"2024-03-09T08:52:30.946321+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T08:52:30.946321+0000","last_undegraded":"2024-03-09T08:52:30.946321+0000","last_fullsized":"2024-03-09T08:52:30.946321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944699+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T08:52:30.944699+0000","last_peered":"2024-03-09T08:52:30.944699+0000","last_clean":"2024-03-09T08:52:30.944699+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T08:52:30.944699+0000","last_undegraded":"2024-03-09T08:52:30.944699+0000","last_fullsized":"2024-03-09T08:52:30.944699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577815+0000","last_change":"2024-03-09T08:52:28.577973+0000","last_active":"2024-03-09T08:52:30.577815+0000","last_peered":"2024-03-09T08:52:30.577815+0000","last_clean":"2024-03-09T08:52:30.577815+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T08:52:30.577815+0000","last_undegraded":"2024-03-09T08:52:30.577815+0000","last_fullsized":"2024-03-09T08:52:30.577815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:13.291622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943332+0000","last_change":"2024-03-09T08:52:29.599601+0000","last_active":"2024-03-09T08:52:30.943332+0000","last_peered":"2024-03-09T08:52:30.943332+0000","last_clean":"2024-03-09T08:52:30.943332+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T08:52:30.943332+0000","last_undegraded":"2024-03-09T08:52:30.943332+0000","last_fullsized":"2024-03-09T08:52:30.943332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:11:17.424330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948439+0000","last_change":"2024-03-09T08:52:18.242671+0000","last_active":"2024-03-09T08:52:30.948439+0000","last_peered":"2024-03-09T08:52:30.948439+0000","last_clean":"2024-03-09T08:52:30.948439+0000","last_became_active":"2024-03-09T08:52:18.242258+0000","last_became_peered":"2024-03-09T08:52:18.242258+0000","last_unstale":"2024-03-09T08:52:30.948439+0000","last_undegraded":"2024-03-09T08:52:30.948439+0000","last_fullsized":"2024-03-09T08:52:30.948439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:15.030669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943179+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T08:52:30.943179+0000","last_peered":"2024-03-09T08:52:30.943179+0000","last_clean":"2024-03-09T08:52:30.943179+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T08:52:30.943179+0000","last_undegraded":"2024-03-09T08:52:30.943179+0000","last_fullsized":"2024-03-09T08:52:30.943179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:27.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576159+0000","last_change":"2024-03-09T08:52:28.589278+0000","last_active":"2024-03-09T08:52:30.576159+0000","last_peered":"2024-03-09T08:52:30.576159+0000","last_clean":"2024-03-09T08:52:30.576159+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T08:52:30.576159+0000","last_undegraded":"2024-03-09T08:52:30.576159+0000","last_fullsized":"2024-03-09T08:52:30.576159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:16:26.449275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942717+0000","last_change":"2024-03-09T08:52:29.600332+0000","last_active":"2024-03-09T08:52:30.942717+0000","last_peered":"2024-03-09T08:52:30.942717+0000","last_clean":"2024-03-09T08:52:30.942717+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T08:52:30.942717+0000","last_undegraded":"2024-03-09T08:52:30.942717+0000","last_fullsized":"2024-03-09T08:52:30.942717+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:36:41.422546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947852+0000","last_change":"2024-03-09T08:52:18.250433+0000","last_active":"2024-03-09T08:52:30.947852+0000","last_peered":"2024-03-09T08:52:30.947852+0000","last_clean":"2024-03-09T08:52:30.947852+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T08:52:30.947852+0000","last_undegraded":"2024-03-09T08:52:30.947852+0000","last_fullsized":"2024-03-09T08:52:30.947852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:29.742337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602070+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T08:52:30.602070+0000","last_peered":"2024-03-09T08:52:30.602070+0000","last_clean":"2024-03-09T08:52:30.602070+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T08:52:30.602070+0000","last_undegraded":"2024-03-09T08:52:30.602070+0000","last_fullsized":"2024-03-09T08:52:30.602070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:26:35.574903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597442+0000","last_change":"2024-03-09T08:52:28.583181+0000","last_active":"2024-03-09T08:52:30.597442+0000","last_peered":"2024-03-09T08:52:30.597442+0000","last_clean":"2024-03-09T08:52:30.597442+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T08:52:30.597442+0000","last_undegraded":"2024-03-09T08:52:30.597442+0000","last_fullsized":"2024-03-09T08:52:30.597442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:58:50.061324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943982+0000","last_change":"2024-03-09T08:52:29.590141+0000","last_active":"2024-03-09T08:52:30.943982+0000","last_peered":"2024-03-09T08:52:30.943982+0000","last_clean":"2024-03-09T08:52:30.943982+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T08:52:30.943982+0000","last_undegraded":"2024-03-09T08:52:30.943982+0000","last_fullsized":"2024-03-09T08:52:30.943982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:23.981897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948851+0000","last_change":"2024-03-09T08:52:18.250420+0000","last_active":"2024-03-09T08:52:30.948851+0000","last_peered":"2024-03-09T08:52:30.948851+0000","last_clean":"2024-03-09T08:52:30.948851+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T08:52:30.948851+0000","last_undegraded":"2024-03-09T08:52:30.948851+0000","last_fullsized":"2024-03-09T08:52:30.948851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:48.826066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943461+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T08:52:30.943461+0000","last_peered":"2024-03-09T08:52:30.943461+0000","last_clean":"2024-03-09T08:52:30.943461+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T08:52:30.943461+0000","last_undegraded":"2024-03-09T08:52:30.943461+0000","last_fullsized":"2024-03-09T08:52:30.943461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:16.138216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944811+0000","last_change":"2024-03-09T08:52:28.587637+0000","last_active":"2024-03-09T08:52:30.944811+0000","last_peered":"2024-03-09T08:52:30.944811+0000","last_clean":"2024-03-09T08:52:30.944811+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T08:52:30.944811+0000","last_undegraded":"2024-03-09T08:52:30.944811+0000","last_fullsized":"2024-03-09T08:52:30.944811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:31:32.362061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942477+0000","last_change":"2024-03-09T08:52:29.586180+0000","last_active":"2024-03-09T08:52:30.942477+0000","last_peered":"2024-03-09T08:52:30.942477+0000","last_clean":"2024-03-09T08:52:30.942477+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T08:52:30.942477+0000","last_undegraded":"2024-03-09T08:52:30.942477+0000","last_fullsized":"2024-03-09T08:52:30.942477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:35:21.522181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948836+0000","last_change":"2024-03-09T08:52:18.246145+0000","last_active":"2024-03-09T08:52:30.948836+0000","last_peered":"2024-03-09T08:52:30.948836+0000","last_clean":"2024-03-09T08:52:30.948836+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T08:52:30.948836+0000","last_undegraded":"2024-03-09T08:52:30.948836+0000","last_fullsized":"2024-03-09T08:52:30.948836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:29:36.213774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578625+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T08:52:30.578625+0000","last_peered":"2024-03-09T08:52:30.578625+0000","last_clean":"2024-03-09T08:52:30.578625+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T08:52:30.578625+0000","last_undegraded":"2024-03-09T08:52:30.578625+0000","last_fullsized":"2024-03-09T08:52:30.578625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:10:34.698235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575615+0000","last_change":"2024-03-09T08:52:28.582308+0000","last_active":"2024-03-09T08:52:30.575615+0000","last_peered":"2024-03-09T08:52:30.575615+0000","last_clean":"2024-03-09T08:52:30.575615+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T08:52:30.575615+0000","last_undegraded":"2024-03-09T08:52:30.575615+0000","last_fullsized":"2024-03-09T08:52:30.575615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:04.918893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577162+0000","last_change":"2024-03-09T08:52:29.610596+0000","last_active":"2024-03-09T08:52:30.577162+0000","last_peered":"2024-03-09T08:52:30.577162+0000","last_clean":"2024-03-09T08:52:30.577162+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T08:52:30.577162+0000","last_undegraded":"2024-03-09T08:52:30.577162+0000","last_fullsized":"2024-03-09T08:52:30.577162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:19:47.313981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944192+0000","last_change":"2024-03-09T08:52:18.248755+0000","last_active":"2024-03-09T08:52:30.944192+0000","last_peered":"2024-03-09T08:52:30.944192+0000","last_clean":"2024-03-09T08:52:30.944192+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T08:52:30.944192+0000","last_undegraded":"2024-03-09T08:52:30.944192+0000","last_fullsized":"2024-03-09T08:52:30.944192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:37.083738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942668+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T08:52:30.942668+0000","last_peered":"2024-03-09T08:52:30.942668+0000","last_clean":"2024-03-09T08:52:30.942668+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T08:52:30.942668+0000","last_undegraded":"2024-03-09T08:52:30.942668+0000","last_fullsized":"2024-03-09T08:52:30.942668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:31.859924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598089+0000","last_change":"2024-03-09T08:52:28.575975+0000","last_active":"2024-03-09T08:52:30.598089+0000","last_peered":"2024-03-09T08:52:30.598089+0000","last_clean":"2024-03-09T08:52:30.598089+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T08:52:30.598089+0000","last_undegraded":"2024-03-09T08:52:30.598089+0000","last_fullsized":"2024-03-09T08:52:30.598089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:51:24.405316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947555+0000","last_change":"2024-03-09T08:52:29.575752+0000","last_active":"2024-03-09T08:52:30.947555+0000","last_peered":"2024-03-09T08:52:30.947555+0000","last_clean":"2024-03-09T08:52:30.947555+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T08:52:30.947555+0000","last_undegraded":"2024-03-09T08:52:30.947555+0000","last_fullsized":"2024-03-09T08:52:30.947555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:31:51.847871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575309+0000","last_change":"2024-03-09T08:52:18.245073+0000","last_active":"2024-03-09T08:52:30.575309+0000","last_peered":"2024-03-09T08:52:30.575309+0000","last_clean":"2024-03-09T08:52:30.575309+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T08:52:30.575309+0000","last_undegraded":"2024-03-09T08:52:30.575309+0000","last_fullsized":"2024-03-09T08:52:30.575309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:40:15.390735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577484+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T08:52:30.577484+0000","last_peered":"2024-03-09T08:52:30.577484+0000","last_clean":"2024-03-09T08:52:30.577484+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T08:52:30.577484+0000","last_undegraded":"2024-03-09T08:52:30.577484+0000","last_fullsized":"2024-03-09T08:52:30.577484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:07:29.577461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577273+0000","last_change":"2024-03-09T08:52:28.588312+0000","last_active":"2024-03-09T08:52:30.577273+0000","last_peered":"2024-03-09T08:52:30.577273+0000","last_clean":"2024-03-09T08:52:30.577273+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T08:52:30.577273+0000","last_undegraded":"2024-03-09T08:52:30.577273+0000","last_fullsized":"2024-03-09T08:52:30.577273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:14:52.895618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946368+0000","last_change":"2024-03-09T08:52:29.596647+0000","last_active":"2024-03-09T08:52:30.946368+0000","last_peered":"2024-03-09T08:52:30.946368+0000","last_clean":"2024-03-09T08:52:30.946368+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T08:52:30.946368+0000","last_undegraded":"2024-03-09T08:52:30.946368+0000","last_fullsized":"2024-03-09T08:52:30.946368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:35:28.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944097+0000","last_change":"2024-03-09T08:52:18.248093+0000","last_active":"2024-03-09T08:52:30.944097+0000","last_peered":"2024-03-09T08:52:30.944097+0000","last_clean":"2024-03-09T08:52:30.944097+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T08:52:30.944097+0000","last_undegraded":"2024-03-09T08:52:30.944097+0000","last_fullsized":"2024-03-09T08:52:30.944097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:29.414335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945767+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T08:52:30.945767+0000","last_peered":"2024-03-09T08:52:30.945767+0000","last_clean":"2024-03-09T08:52:30.945767+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T08:52:30.945767+0000","last_undegraded":"2024-03-09T08:52:30.945767+0000","last_fullsized":"2024-03-09T08:52:30.945767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576830+0000","last_change":"2024-03-09T08:52:28.585562+0000","last_active":"2024-03-09T08:52:30.576830+0000","last_peered":"2024-03-09T08:52:30.576830+0000","last_clean":"2024-03-09T08:52:30.576830+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T08:52:30.576830+0000","last_undegraded":"2024-03-09T08:52:30.576830+0000","last_fullsized":"2024-03-09T08:52:30.576830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:28:50.544795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576833+0000","last_change":"2024-03-09T08:52:29.581330+0000","last_active":"2024-03-09T08:52:30.576833+0000","last_peered":"2024-03-09T08:52:30.576833+0000","last_clean":"2024-03-09T08:52:30.576833+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T08:52:30.576833+0000","last_undegraded":"2024-03-09T08:52:30.576833+0000","last_fullsized":"2024-03-09T08:52:30.576833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:55:38.041794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597506+0000","last_change":"2024-03-09T08:52:18.236146+0000","last_active":"2024-03-09T08:52:30.597506+0000","last_peered":"2024-03-09T08:52:30.597506+0000","last_clean":"2024-03-09T08:52:30.597506+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T08:52:30.597506+0000","last_undegraded":"2024-03-09T08:52:30.597506+0000","last_fullsized":"2024-03-09T08:52:30.597506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:18:51.511926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598307+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T08:52:30.598307+0000","last_peered":"2024-03-09T08:52:30.598307+0000","last_clean":"2024-03-09T08:52:30.598307+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T08:52:30.598307+0000","last_undegraded":"2024-03-09T08:52:30.598307+0000","last_fullsized":"2024-03-09T08:52:30.598307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:22:05.504557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947315+0000","last_change":"2024-03-09T08:52:28.585697+0000","last_active":"2024-03-09T08:52:30.947315+0000","last_peered":"2024-03-09T08:52:30.947315+0000","last_clean":"2024-03-09T08:52:30.947315+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T08:52:30.947315+0000","last_undegraded":"2024-03-09T08:52:30.947315+0000","last_fullsized":"2024-03-09T08:52:30.947315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:50:23.095008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576346+0000","last_change":"2024-03-09T08:52:29.578803+0000","last_active":"2024-03-09T08:52:30.576346+0000","last_peered":"2024-03-09T08:52:30.576346+0000","last_clean":"2024-03-09T08:52:30.576346+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T08:52:30.576346+0000","last_undegraded":"2024-03-09T08:52:30.576346+0000","last_fullsized":"2024-03-09T08:52:30.576346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:22:47.690873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948118+0000","last_change":"2024-03-09T08:52:18.242496+0000","last_active":"2024-03-09T08:52:30.948118+0000","last_peered":"2024-03-09T08:52:30.948118+0000","last_clean":"2024-03-09T08:52:30.948118+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T08:52:30.948118+0000","last_undegraded":"2024-03-09T08:52:30.948118+0000","last_fullsized":"2024-03-09T08:52:30.948118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:35:17.142983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578358+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T08:52:30.578358+0000","last_peered":"2024-03-09T08:52:30.578358+0000","last_clean":"2024-03-09T08:52:30.578358+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T08:52:30.578358+0000","last_undegraded":"2024-03-09T08:52:30.578358+0000","last_fullsized":"2024-03-09T08:52:30.578358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:01.724043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601055+0000","last_change":"2024-03-09T08:52:28.583685+0000","last_active":"2024-03-09T08:52:30.601055+0000","last_peered":"2024-03-09T08:52:30.601055+0000","last_clean":"2024-03-09T08:52:30.601055+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T08:52:30.601055+0000","last_undegraded":"2024-03-09T08:52:30.601055+0000","last_fullsized":"2024-03-09T08:52:30.601055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:58:00.899620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601468+0000","last_change":"2024-03-09T08:52:29.585912+0000","last_active":"2024-03-09T08:52:30.601468+0000","last_peered":"2024-03-09T08:52:30.601468+0000","last_clean":"2024-03-09T08:52:30.601468+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T08:52:30.601468+0000","last_undegraded":"2024-03-09T08:52:30.601468+0000","last_fullsized":"2024-03-09T08:52:30.601468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:24:47.261191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577039+0000","last_change":"2024-03-09T08:52:18.244654+0000","last_active":"2024-03-09T08:52:30.577039+0000","last_peered":"2024-03-09T08:52:30.577039+0000","last_clean":"2024-03-09T08:52:30.577039+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T08:52:30.577039+0000","last_undegraded":"2024-03-09T08:52:30.577039+0000","last_fullsized":"2024-03-09T08:52:30.577039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:08:58.611725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601999+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T08:52:30.601999+0000","last_peered":"2024-03-09T08:52:30.601999+0000","last_clean":"2024-03-09T08:52:30.601999+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T08:52:30.601999+0000","last_undegraded":"2024-03-09T08:52:30.601999+0000","last_fullsized":"2024-03-09T08:52:30.601999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:41:10.578669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577095+0000","last_change":"2024-03-09T08:52:28.568794+0000","last_active":"2024-03-09T08:52:30.577095+0000","last_peered":"2024-03-09T08:52:30.577095+0000","last_clean":"2024-03-09T08:52:30.577095+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T08:52:30.577095+0000","last_undegraded":"2024-03-09T08:52:30.577095+0000","last_fullsized":"2024-03-09T08:52:30.577095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:44:21.721511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577942+0000","last_change":"2024-03-09T08:52:29.586705+0000","last_active":"2024-03-09T08:52:30.577942+0000","last_peered":"2024-03-09T08:52:30.577942+0000","last_clean":"2024-03-09T08:52:30.577942+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T08:52:30.577942+0000","last_undegraded":"2024-03-09T08:52:30.577942+0000","last_fullsized":"2024-03-09T08:52:30.577942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:26.353156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597722+0000","last_change":"2024-03-09T08:52:18.236628+0000","last_active":"2024-03-09T08:52:30.597722+0000","last_peered":"2024-03-09T08:52:30.597722+0000","last_clean":"2024-03-09T08:52:30.597722+0000","last_became_active":"2024-03-09T08:52:18.235321+0000","last_became_peered":"2024-03-09T08:52:18.235321+0000","last_unstale":"2024-03-09T08:52:30.597722+0000","last_undegraded":"2024-03-09T08:52:30.597722+0000","last_fullsized":"2024-03-09T08:52:30.597722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:13:59.221325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948276+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T08:52:30.948276+0000","last_peered":"2024-03-09T08:52:30.948276+0000","last_clean":"2024-03-09T08:52:30.948276+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T08:52:30.948276+0000","last_undegraded":"2024-03-09T08:52:30.948276+0000","last_fullsized":"2024-03-09T08:52:30.948276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575808+0000","last_change":"2024-03-09T08:52:28.583697+0000","last_active":"2024-03-09T08:52:30.575808+0000","last_peered":"2024-03-09T08:52:30.575808+0000","last_clean":"2024-03-09T08:52:30.575808+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T08:52:30.575808+0000","last_undegraded":"2024-03-09T08:52:30.575808+0000","last_fullsized":"2024-03-09T08:52:30.575808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:41.552156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945076+0000","last_change":"2024-03-09T08:52:29.590854+0000","last_active":"2024-03-09T08:52:30.945076+0000","last_peered":"2024-03-09T08:52:30.945076+0000","last_clean":"2024-03-09T08:52:30.945076+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T08:52:30.945076+0000","last_undegraded":"2024-03-09T08:52:30.945076+0000","last_fullsized":"2024-03-09T08:52:30.945076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:09.568826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947914+0000","last_change":"2024-03-09T08:52:18.251648+0000","last_active":"2024-03-09T08:52:30.947914+0000","last_peered":"2024-03-09T08:52:30.947914+0000","last_clean":"2024-03-09T08:52:30.947914+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T08:52:30.947914+0000","last_undegraded":"2024-03-09T08:52:30.947914+0000","last_fullsized":"2024-03-09T08:52:30.947914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:54:45.721673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579118+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T08:52:30.579118+0000","last_peered":"2024-03-09T08:52:30.579118+0000","last_clean":"2024-03-09T08:52:30.579118+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T08:52:30.579118+0000","last_undegraded":"2024-03-09T08:52:30.579118+0000","last_fullsized":"2024-03-09T08:52:30.579118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:31.731626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575378+0000","last_change":"2024-03-09T08:52:28.586606+0000","last_active":"2024-03-09T08:52:30.575378+0000","last_peered":"2024-03-09T08:52:30.575378+0000","last_clean":"2024-03-09T08:52:30.575378+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T08:52:30.575378+0000","last_undegraded":"2024-03-09T08:52:30.575378+0000","last_fullsized":"2024-03-09T08:52:30.575378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:17:40.902403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578219+0000","last_change":"2024-03-09T08:52:29.584512+0000","last_active":"2024-03-09T08:52:30.578219+0000","last_peered":"2024-03-09T08:52:30.578219+0000","last_clean":"2024-03-09T08:52:30.578219+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T08:52:30.578219+0000","last_undegraded":"2024-03-09T08:52:30.578219+0000","last_fullsized":"2024-03-09T08:52:30.578219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:39:20.330443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597975+0000","last_change":"2024-03-09T08:52:28.586201+0000","last_active":"2024-03-09T08:52:30.597975+0000","last_peered":"2024-03-09T08:52:30.597975+0000","last_clean":"2024-03-09T08:52:30.597975+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T08:52:30.597975+0000","last_undegraded":"2024-03-09T08:52:30.597975+0000","last_fullsized":"2024-03-09T08:52:30.597975+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:28:03.471646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601039+0000","last_change":"2024-03-09T08:52:18.241156+0000","last_active":"2024-03-09T08:52:30.601039+0000","last_peered":"2024-03-09T08:52:30.601039+0000","last_clean":"2024-03-09T08:52:30.601039+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T08:52:30.601039+0000","last_undegraded":"2024-03-09T08:52:30.601039+0000","last_fullsized":"2024-03-09T08:52:30.601039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:55.323770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946394+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T08:52:30.946394+0000","last_peered":"2024-03-09T08:52:30.946394+0000","last_clean":"2024-03-09T08:52:30.946394+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T08:52:30.946394+0000","last_undegraded":"2024-03-09T08:52:30.946394+0000","last_fullsized":"2024-03-09T08:52:30.946394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597714+0000","last_change":"2024-03-09T08:52:29.591085+0000","last_active":"2024-03-09T08:52:30.597714+0000","last_peered":"2024-03-09T08:52:30.597714+0000","last_clean":"2024-03-09T08:52:30.597714+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T08:52:30.597714+0000","last_undegraded":"2024-03-09T08:52:30.597714+0000","last_fullsized":"2024-03-09T08:52:30.597714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:40:27.878922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578971+0000","last_change":"2024-03-09T08:52:28.585631+0000","last_active":"2024-03-09T08:52:30.578971+0000","last_peered":"2024-03-09T08:52:30.578971+0000","last_clean":"2024-03-09T08:52:30.578971+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T08:52:30.578971+0000","last_undegraded":"2024-03-09T08:52:30.578971+0000","last_fullsized":"2024-03-09T08:52:30.578971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:06:03.569759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576252+0000","last_change":"2024-03-09T08:52:18.234400+0000","last_active":"2024-03-09T08:52:30.576252+0000","last_peered":"2024-03-09T08:52:30.576252+0000","last_clean":"2024-03-09T08:52:30.576252+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T08:52:30.576252+0000","last_undegraded":"2024-03-09T08:52:30.576252+0000","last_fullsized":"2024-03-09T08:52:30.576252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:42:14.057367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576643+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T08:52:30.576643+0000","last_peered":"2024-03-09T08:52:30.576643+0000","last_clean":"2024-03-09T08:52:30.576643+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T08:52:30.576643+0000","last_undegraded":"2024-03-09T08:52:30.576643+0000","last_fullsized":"2024-03-09T08:52:30.576643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601916+0000","last_change":"2024-03-09T08:52:29.588928+0000","last_active":"2024-03-09T08:52:30.601916+0000","last_peered":"2024-03-09T08:52:30.601916+0000","last_clean":"2024-03-09T08:52:30.601916+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T08:52:30.601916+0000","last_undegraded":"2024-03-09T08:52:30.601916+0000","last_fullsized":"2024-03-09T08:52:30.601916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:32:27.061041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576515+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T08:52:30.576515+0000","last_peered":"2024-03-09T08:52:30.576515+0000","last_clean":"2024-03-09T08:52:30.576515+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T08:52:30.576515+0000","last_undegraded":"2024-03-09T08:52:30.576515+0000","last_fullsized":"2024-03-09T08:52:30.576515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598213+0000","last_change":"2024-03-09T08:52:18.231141+0000","last_active":"2024-03-09T08:52:30.598213+0000","last_peered":"2024-03-09T08:52:30.598213+0000","last_clean":"2024-03-09T08:52:30.598213+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T08:52:30.598213+0000","last_undegraded":"2024-03-09T08:52:30.598213+0000","last_fullsized":"2024-03-09T08:52:30.598213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:25:00.471222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943012+0000","last_change":"2024-03-09T08:52:28.579315+0000","last_active":"2024-03-09T08:52:30.943012+0000","last_peered":"2024-03-09T08:52:30.943012+0000","last_clean":"2024-03-09T08:52:30.943012+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T08:52:30.943012+0000","last_undegraded":"2024-03-09T08:52:30.943012+0000","last_fullsized":"2024-03-09T08:52:30.943012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:57:17.301677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576561+0000","last_change":"2024-03-09T08:52:29.585154+0000","last_active":"2024-03-09T08:52:30.576561+0000","last_peered":"2024-03-09T08:52:30.576561+0000","last_clean":"2024-03-09T08:52:30.576561+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T08:52:30.576561+0000","last_undegraded":"2024-03-09T08:52:30.576561+0000","last_fullsized":"2024-03-09T08:52:30.576561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:43:35.679108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576704+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T08:52:30.576704+0000","last_peered":"2024-03-09T08:52:30.576704+0000","last_clean":"2024-03-09T08:52:30.576704+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T08:52:30.576704+0000","last_undegraded":"2024-03-09T08:52:30.576704+0000","last_fullsized":"2024-03-09T08:52:30.576704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577869+0000","last_change":"2024-03-09T08:52:18.229533+0000","last_active":"2024-03-09T08:52:30.577869+0000","last_peered":"2024-03-09T08:52:30.577869+0000","last_clean":"2024-03-09T08:52:30.577869+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T08:52:30.577869+0000","last_undegraded":"2024-03-09T08:52:30.577869+0000","last_fullsized":"2024-03-09T08:52:30.577869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:17.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576387+0000","last_change":"2024-03-09T08:52:28.585877+0000","last_active":"2024-03-09T08:52:30.576387+0000","last_peered":"2024-03-09T08:52:30.576387+0000","last_clean":"2024-03-09T08:52:30.576387+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T08:52:30.576387+0000","last_undegraded":"2024-03-09T08:52:30.576387+0000","last_fullsized":"2024-03-09T08:52:30.576387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:30:20.036385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577546+0000","last_change":"2024-03-09T08:52:29.592253+0000","last_active":"2024-03-09T08:52:30.577546+0000","last_peered":"2024-03-09T08:52:30.577546+0000","last_clean":"2024-03-09T08:52:30.577546+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T08:52:30.577546+0000","last_undegraded":"2024-03-09T08:52:30.577546+0000","last_fullsized":"2024-03-09T08:52:30.577546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:50:33.525541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575154+0000","last_change":"2024-03-09T08:52:18.237869+0000","last_active":"2024-03-09T08:52:30.575154+0000","last_peered":"2024-03-09T08:52:30.575154+0000","last_clean":"2024-03-09T08:52:30.575154+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T08:52:30.575154+0000","last_undegraded":"2024-03-09T08:52:30.575154+0000","last_fullsized":"2024-03-09T08:52:30.575154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:01.571866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602125+0000","last_change":"2024-03-09T08:52:29.601300+0000","last_active":"2024-03-09T08:52:30.602125+0000","last_peered":"2024-03-09T08:52:30.602125+0000","last_clean":"2024-03-09T08:52:30.602125+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T08:52:30.602125+0000","last_undegraded":"2024-03-09T08:52:30.602125+0000","last_fullsized":"2024-03-09T08:52:30.602125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:46:04.870574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942394+0000","last_change":"2024-03-09T08:52:18.226980+0000","last_active":"2024-03-09T08:52:30.942394+0000","last_peered":"2024-03-09T08:52:30.942394+0000","last_clean":"2024-03-09T08:52:30.942394+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T08:52:30.942394+0000","last_undegraded":"2024-03-09T08:52:30.942394+0000","last_fullsized":"2024-03-09T08:52:30.942394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:28.312576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946757+0000","last_change":"2024-03-09T08:52:29.575612+0000","last_active":"2024-03-09T08:52:30.946757+0000","last_peered":"2024-03-09T08:52:30.946757+0000","last_clean":"2024-03-09T08:52:30.946757+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T08:52:30.946757+0000","last_undegraded":"2024-03-09T08:52:30.946757+0000","last_fullsized":"2024-03-09T08:52:30.946757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:58.852983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945644+0000","last_change":"2024-03-09T08:52:18.241628+0000","last_active":"2024-03-09T08:52:30.945644+0000","last_peered":"2024-03-09T08:52:30.945644+0000","last_clean":"2024-03-09T08:52:30.945644+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T08:52:30.945644+0000","last_undegraded":"2024-03-09T08:52:30.945644+0000","last_fullsized":"2024-03-09T08:52:30.945644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:07:58.570641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601917+0000","last_change":"2024-03-09T08:52:29.588660+0000","last_active":"2024-03-09T08:52:30.601917+0000","last_peered":"2024-03-09T08:52:30.601917+0000","last_clean":"2024-03-09T08:52:30.601917+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T08:52:30.601917+0000","last_undegraded":"2024-03-09T08:52:30.601917+0000","last_fullsized":"2024-03-09T08:52:30.601917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:14.305320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598085+0000","last_change":"2024-03-09T08:52:18.242257+0000","last_active":"2024-03-09T08:52:30.598085+0000","last_peered":"2024-03-09T08:52:30.598085+0000","last_clean":"2024-03-09T08:52:30.598085+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T08:52:30.598085+0000","last_undegraded":"2024-03-09T08:52:30.598085+0000","last_fullsized":"2024-03-09T08:52:30.598085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:40:00.854066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944026+0000","last_change":"2024-03-09T08:52:29.595057+0000","last_active":"2024-03-09T08:52:30.944026+0000","last_peered":"2024-03-09T08:52:30.944026+0000","last_clean":"2024-03-09T08:52:30.944026+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T08:52:30.944026+0000","last_undegraded":"2024-03-09T08:52:30.944026+0000","last_fullsized":"2024-03-09T08:52:30.944026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:33:37.474266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577091+0000","last_change":"2024-03-09T08:52:18.239951+0000","last_active":"2024-03-09T08:52:30.577091+0000","last_peered":"2024-03-09T08:52:30.577091+0000","last_clean":"2024-03-09T08:52:30.577091+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T08:52:30.577091+0000","last_undegraded":"2024-03-09T08:52:30.577091+0000","last_fullsized":"2024-03-09T08:52:30.577091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:51.102540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576730+0000","last_change":"2024-03-09T08:52:29.578993+0000","last_active":"2024-03-09T08:52:30.576730+0000","last_peered":"2024-03-09T08:52:30.576730+0000","last_clean":"2024-03-09T08:52:30.576730+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T08:52:30.576730+0000","last_undegraded":"2024-03-09T08:52:30.576730+0000","last_fullsized":"2024-03-09T08:52:30.576730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:50:53.692060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601995+0000","last_change":"2024-03-09T08:52:18.241038+0000","last_active":"2024-03-09T08:52:30.601995+0000","last_peered":"2024-03-09T08:52:30.601995+0000","last_clean":"2024-03-09T08:52:30.601995+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T08:52:30.601995+0000","last_undegraded":"2024-03-09T08:52:30.601995+0000","last_fullsized":"2024-03-09T08:52:30.601995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:02.169113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943444+0000","last_change":"2024-03-09T08:52:29.583869+0000","last_active":"2024-03-09T08:52:30.943444+0000","last_peered":"2024-03-09T08:52:30.943444+0000","last_clean":"2024-03-09T08:52:30.943444+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T08:52:30.943444+0000","last_undegraded":"2024-03-09T08:52:30.943444+0000","last_fullsized":"2024-03-09T08:52:30.943444+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:29:47.636741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578670+0000","last_change":"2024-03-09T08:52:18.247145+0000","last_active":"2024-03-09T08:52:30.578670+0000","last_peered":"2024-03-09T08:52:30.578670+0000","last_clean":"2024-03-09T08:52:30.578670+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T08:52:30.578670+0000","last_undegraded":"2024-03-09T08:52:30.578670+0000","last_fullsized":"2024-03-09T08:52:30.578670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:46:24.111224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577279+0000","last_change":"2024-03-09T08:52:29.587358+0000","last_active":"2024-03-09T08:52:30.577279+0000","last_peered":"2024-03-09T08:52:30.577279+0000","last_clean":"2024-03-09T08:52:30.577279+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T08:52:30.577279+0000","last_undegraded":"2024-03-09T08:52:30.577279+0000","last_fullsized":"2024-03-09T08:52:30.577279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:46.836801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577323+0000","last_change":"2024-03-09T08:52:18.247784+0000","last_active":"2024-03-09T08:52:30.577323+0000","last_peered":"2024-03-09T08:52:30.577323+0000","last_clean":"2024-03-09T08:52:30.577323+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T08:52:30.577323+0000","last_undegraded":"2024-03-09T08:52:30.577323+0000","last_fullsized":"2024-03-09T08:52:30.577323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:46:59.008220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577364+0000","last_change":"2024-03-09T08:52:29.589458+0000","last_active":"2024-03-09T08:52:30.577364+0000","last_peered":"2024-03-09T08:52:30.577364+0000","last_clean":"2024-03-09T08:52:30.577364+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T08:52:30.577364+0000","last_undegraded":"2024-03-09T08:52:30.577364+0000","last_fullsized":"2024-03-09T08:52:30.577364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:19:35.222184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945231+0000","last_change":"2024-03-09T08:52:18.239943+0000","last_active":"2024-03-09T08:52:30.945231+0000","last_peered":"2024-03-09T08:52:30.945231+0000","last_clean":"2024-03-09T08:52:30.945231+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T08:52:30.945231+0000","last_undegraded":"2024-03-09T08:52:30.945231+0000","last_fullsized":"2024-03-09T08:52:30.945231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:18.727030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.949211+0000","last_change":"2024-03-09T08:52:29.597147+0000","last_active":"2024-03-09T08:52:30.949211+0000","last_peered":"2024-03-09T08:52:30.949211+0000","last_clean":"2024-03-09T08:52:30.949211+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T08:52:30.949211+0000","last_undegraded":"2024-03-09T08:52:30.949211+0000","last_fullsized":"2024-03-09T08:52:30.949211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:44:47.144911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575715+0000","last_change":"2024-03-09T08:52:18.236642+0000","last_active":"2024-03-09T08:52:30.575715+0000","last_peered":"2024-03-09T08:52:30.575715+0000","last_clean":"2024-03-09T08:52:30.575715+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T08:52:30.575715+0000","last_undegraded":"2024-03-09T08:52:30.575715+0000","last_fullsized":"2024-03-09T08:52:30.575715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:51:46.825153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943044+0000","last_change":"2024-03-09T08:52:29.590810+0000","last_active":"2024-03-09T08:52:30.943044+0000","last_peered":"2024-03-09T08:52:30.943044+0000","last_clean":"2024-03-09T08:52:30.943044+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T08:52:30.943044+0000","last_undegraded":"2024-03-09T08:52:30.943044+0000","last_fullsized":"2024-03-09T08:52:30.943044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:21.534580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945569+0000","last_change":"2024-03-09T08:52:18.236855+0000","last_active":"2024-03-09T08:52:30.945569+0000","last_peered":"2024-03-09T08:52:30.945569+0000","last_clean":"2024-03-09T08:52:30.945569+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T08:52:30.945569+0000","last_undegraded":"2024-03-09T08:52:30.945569+0000","last_fullsized":"2024-03-09T08:52:30.945569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:21.941242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576311+0000","last_change":"2024-03-09T08:52:29.602209+0000","last_active":"2024-03-09T08:52:30.576311+0000","last_peered":"2024-03-09T08:52:30.576311+0000","last_clean":"2024-03-09T08:52:30.576311+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T08:52:30.576311+0000","last_undegraded":"2024-03-09T08:52:30.576311+0000","last_fullsized":"2024-03-09T08:52:30.576311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:11:33.349464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597936+0000","last_change":"2024-03-09T08:52:18.238373+0000","last_active":"2024-03-09T08:52:30.597936+0000","last_peered":"2024-03-09T08:52:30.597936+0000","last_clean":"2024-03-09T08:52:30.597936+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T08:52:30.597936+0000","last_undegraded":"2024-03-09T08:52:30.597936+0000","last_fullsized":"2024-03-09T08:52:30.597936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:05:31.480335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577230+0000","last_change":"2024-03-09T08:52:29.578120+0000","last_active":"2024-03-09T08:52:30.577230+0000","last_peered":"2024-03-09T08:52:30.577230+0000","last_clean":"2024-03-09T08:52:30.577230+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T08:52:30.577230+0000","last_undegraded":"2024-03-09T08:52:30.577230+0000","last_fullsized":"2024-03-09T08:52:30.577230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:13:27.623995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578214+0000","last_change":"2024-03-09T08:52:18.235565+0000","last_active":"2024-03-09T08:52:30.578214+0000","last_peered":"2024-03-09T08:52:30.578214+0000","last_clean":"2024-03-09T08:52:30.578214+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T08:52:30.578214+0000","last_undegraded":"2024-03-09T08:52:30.578214+0000","last_fullsized":"2024-03-09T08:52:30.578214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:52:14.994539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576439+0000","last_change":"2024-03-09T08:52:29.579668+0000","last_active":"2024-03-09T08:52:30.576439+0000","last_peered":"2024-03-09T08:52:30.576439+0000","last_clean":"2024-03-09T08:52:30.576439+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T08:52:30.576439+0000","last_undegraded":"2024-03-09T08:52:30.576439+0000","last_fullsized":"2024-03-09T08:52:30.576439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:12:15.272875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948818+0000","last_change":"2024-03-09T08:52:18.239801+0000","last_active":"2024-03-09T08:52:30.948818+0000","last_peered":"2024-03-09T08:52:30.948818+0000","last_clean":"2024-03-09T08:52:30.948818+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T08:52:30.948818+0000","last_undegraded":"2024-03-09T08:52:30.948818+0000","last_fullsized":"2024-03-09T08:52:30.948818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:38:06.836990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944799+0000","last_change":"2024-03-09T08:52:29.586986+0000","last_active":"2024-03-09T08:52:30.944799+0000","last_peered":"2024-03-09T08:52:30.944799+0000","last_clean":"2024-03-09T08:52:30.944799+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T08:52:30.944799+0000","last_undegraded":"2024-03-09T08:52:30.944799+0000","last_fullsized":"2024-03-09T08:52:30.944799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:37.382414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943359+0000","last_change":"2024-03-09T08:52:18.233582+0000","last_active":"2024-03-09T08:52:30.943359+0000","last_peered":"2024-03-09T08:52:30.943359+0000","last_clean":"2024-03-09T08:52:30.943359+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T08:52:30.943359+0000","last_undegraded":"2024-03-09T08:52:30.943359+0000","last_fullsized":"2024-03-09T08:52:30.943359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:49.313616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597247+0000","last_change":"2024-03-09T08:52:29.586673+0000","last_active":"2024-03-09T08:52:30.597247+0000","last_peered":"2024-03-09T08:52:30.597247+0000","last_clean":"2024-03-09T08:52:30.597247+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T08:52:30.597247+0000","last_undegraded":"2024-03-09T08:52:30.597247+0000","last_fullsized":"2024-03-09T08:52:30.597247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:23:49.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947293+0000","last_change":"2024-03-09T08:52:18.242349+0000","last_active":"2024-03-09T08:52:30.947293+0000","last_peered":"2024-03-09T08:52:30.947293+0000","last_clean":"2024-03-09T08:52:30.947293+0000","last_became_active":"2024-03-09T08:52:18.241995+0000","last_became_peered":"2024-03-09T08:52:18.241995+0000","last_unstale":"2024-03-09T08:52:30.947293+0000","last_undegraded":"2024-03-09T08:52:30.947293+0000","last_fullsized":"2024-03-09T08:52:30.947293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:29:09.977984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601965+0000","last_change":"2024-03-09T08:52:29.592702+0000","last_active":"2024-03-09T08:52:30.601965+0000","last_peered":"2024-03-09T08:52:30.601965+0000","last_clean":"2024-03-09T08:52:30.601965+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T08:52:30.601965+0000","last_undegraded":"2024-03-09T08:52:30.601965+0000","last_fullsized":"2024-03-09T08:52:30.601965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:39.344726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944970+0000","last_change":"2024-03-09T08:52:18.230289+0000","last_active":"2024-03-09T08:52:30.944970+0000","last_peered":"2024-03-09T08:52:30.944970+0000","last_clean":"2024-03-09T08:52:30.944970+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T08:52:30.944970+0000","last_undegraded":"2024-03-09T08:52:30.944970+0000","last_fullsized":"2024-03-09T08:52:30.944970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:34.507924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578394+0000","last_change":"2024-03-09T08:52:29.608191+0000","last_active":"2024-03-09T08:52:30.578394+0000","last_peered":"2024-03-09T08:52:30.578394+0000","last_clean":"2024-03-09T08:52:30.578394+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T08:52:30.578394+0000","last_undegraded":"2024-03-09T08:52:30.578394+0000","last_fullsized":"2024-03-09T08:52:30.578394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:26:32.282196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601147+0000","last_change":"2024-03-09T08:52:18.236376+0000","last_active":"2024-03-09T08:52:30.601147+0000","last_peered":"2024-03-09T08:52:30.601147+0000","last_clean":"2024-03-09T08:52:30.601147+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T08:52:30.601147+0000","last_undegraded":"2024-03-09T08:52:30.601147+0000","last_fullsized":"2024-03-09T08:52:30.601147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:34:04.412643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946339+0000","last_change":"2024-03-09T08:52:29.573685+0000","last_active":"2024-03-09T08:52:30.946339+0000","last_peered":"2024-03-09T08:52:30.946339+0000","last_clean":"2024-03-09T08:52:30.946339+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T08:52:30.946339+0000","last_undegraded":"2024-03-09T08:52:30.946339+0000","last_fullsized":"2024-03-09T08:52:30.946339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:41:41.839063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597356+0000","last_change":"2024-03-09T08:52:18.231316+0000","last_active":"2024-03-09T08:52:30.597356+0000","last_peered":"2024-03-09T08:52:30.597356+0000","last_clean":"2024-03-09T08:52:30.597356+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T08:52:30.597356+0000","last_undegraded":"2024-03-09T08:52:30.597356+0000","last_fullsized":"2024-03-09T08:52:30.597356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:25:25.715084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948285+0000","last_change":"2024-03-09T08:52:29.603272+0000","last_active":"2024-03-09T08:52:30.948285+0000","last_peered":"2024-03-09T08:52:30.948285+0000","last_clean":"2024-03-09T08:52:30.948285+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T08:52:30.948285+0000","last_undegraded":"2024-03-09T08:52:30.948285+0000","last_fullsized":"2024-03-09T08:52:30.948285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:07:58.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576412+0000","last_change":"2024-03-09T08:52:18.243645+0000","last_active":"2024-03-09T08:52:30.576412+0000","last_peered":"2024-03-09T08:52:30.576412+0000","last_clean":"2024-03-09T08:52:30.576412+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T08:52:30.576412+0000","last_undegraded":"2024-03-09T08:52:30.576412+0000","last_fullsized":"2024-03-09T08:52:30.576412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:19.501675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576249+0000","last_change":"2024-03-09T08:52:29.587462+0000","last_active":"2024-03-09T08:52:30.576249+0000","last_peered":"2024-03-09T08:52:30.576249+0000","last_clean":"2024-03-09T08:52:30.576249+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T08:52:30.576249+0000","last_undegraded":"2024-03-09T08:52:30.576249+0000","last_fullsized":"2024-03-09T08:52:30.576249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:51.652225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577062+0000","last_change":"2024-03-09T08:52:18.233451+0000","last_active":"2024-03-09T08:52:30.577062+0000","last_peered":"2024-03-09T08:52:30.577062+0000","last_clean":"2024-03-09T08:52:30.577062+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T08:52:30.577062+0000","last_undegraded":"2024-03-09T08:52:30.577062+0000","last_fullsized":"2024-03-09T08:52:30.577062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:56:11.270081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942964+0000","last_change":"2024-03-09T08:52:29.599909+0000","last_active":"2024-03-09T08:52:30.942964+0000","last_peered":"2024-03-09T08:52:30.942964+0000","last_clean":"2024-03-09T08:52:30.942964+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T08:52:30.942964+0000","last_undegraded":"2024-03-09T08:52:30.942964+0000","last_fullsized":"2024-03-09T08:52:30.942964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:42.596067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577555+0000","last_change":"2024-03-09T08:52:18.234158+0000","last_active":"2024-03-09T08:52:30.577555+0000","last_peered":"2024-03-09T08:52:30.577555+0000","last_clean":"2024-03-09T08:52:30.577555+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T08:52:30.577555+0000","last_undegraded":"2024-03-09T08:52:30.577555+0000","last_fullsized":"2024-03-09T08:52:30.577555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:59:33.423558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944531+0000","last_change":"2024-03-09T08:52:29.597518+0000","last_active":"2024-03-09T08:52:30.944531+0000","last_peered":"2024-03-09T08:52:30.944531+0000","last_clean":"2024-03-09T08:52:30.944531+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T08:52:30.944531+0000","last_undegraded":"2024-03-09T08:52:30.944531+0000","last_fullsized":"2024-03-09T08:52:30.944531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:32:32.452993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577830+0000","last_change":"2024-03-09T08:52:18.234648+0000","last_active":"2024-03-09T08:52:30.577830+0000","last_peered":"2024-03-09T08:52:30.577830+0000","last_clean":"2024-03-09T08:52:30.577830+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T08:52:30.577830+0000","last_undegraded":"2024-03-09T08:52:30.577830+0000","last_fullsized":"2024-03-09T08:52:30.577830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:03.927280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944900+0000","last_change":"2024-03-09T08:52:29.580608+0000","last_active":"2024-03-09T08:52:30.944900+0000","last_peered":"2024-03-09T08:52:30.944900+0000","last_clean":"2024-03-09T08:52:30.944900+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T08:52:30.944900+0000","last_undegraded":"2024-03-09T08:52:30.944900+0000","last_fullsized":"2024-03-09T08:52:30.944900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:00:37.184351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948628+0000","last_change":"2024-03-09T08:52:18.253038+0000","last_active":"2024-03-09T08:52:30.948628+0000","last_peered":"2024-03-09T08:52:30.948628+0000","last_clean":"2024-03-09T08:52:30.948628+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T08:52:30.948628+0000","last_undegraded":"2024-03-09T08:52:30.948628+0000","last_fullsized":"2024-03-09T08:52:30.948628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:26:39.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577331+0000","last_change":"2024-03-09T08:52:29.578549+0000","last_active":"2024-03-09T08:52:30.577331+0000","last_peered":"2024-03-09T08:52:30.577331+0000","last_clean":"2024-03-09T08:52:30.577331+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T08:52:30.577331+0000","last_undegraded":"2024-03-09T08:52:30.577331+0000","last_fullsized":"2024-03-09T08:52:30.577331+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:42.775199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575758+0000","last_change":"2024-03-09T08:52:18.240975+0000","last_active":"2024-03-09T08:52:30.575758+0000","last_peered":"2024-03-09T08:52:30.575758+0000","last_clean":"2024-03-09T08:52:30.575758+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T08:52:30.575758+0000","last_undegraded":"2024-03-09T08:52:30.575758+0000","last_fullsized":"2024-03-09T08:52:30.575758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:30:09.363339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577959+0000","last_change":"2024-03-09T08:52:29.584159+0000","last_active":"2024-03-09T08:52:30.577959+0000","last_peered":"2024-03-09T08:52:30.577959+0000","last_clean":"2024-03-09T08:52:30.577959+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T08:52:30.577959+0000","last_undegraded":"2024-03-09T08:52:30.577959+0000","last_fullsized":"2024-03-09T08:52:30.577959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:01:41.661806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943943+0000","last_change":"2024-03-09T08:52:18.249901+0000","last_active":"2024-03-09T08:52:30.943943+0000","last_peered":"2024-03-09T08:52:30.943943+0000","last_clean":"2024-03-09T08:52:30.943943+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T08:52:30.943943+0000","last_undegraded":"2024-03-09T08:52:30.943943+0000","last_fullsized":"2024-03-09T08:52:30.943943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:09.947401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601289+0000","last_change":"2024-03-09T08:52:29.595184+0000","last_active":"2024-03-09T08:52:30.601289+0000","last_peered":"2024-03-09T08:52:30.601289+0000","last_clean":"2024-03-09T08:52:30.601289+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T08:52:30.601289+0000","last_undegraded":"2024-03-09T08:52:30.601289+0000","last_fullsized":"2024-03-09T08:52:30.601289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:34:27.662636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945364+0000","last_change":"2024-03-09T08:52:18.233561+0000","last_active":"2024-03-09T08:52:30.945364+0000","last_peered":"2024-03-09T08:52:30.945364+0000","last_clean":"2024-03-09T08:52:30.945364+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T08:52:30.945364+0000","last_undegraded":"2024-03-09T08:52:30.945364+0000","last_fullsized":"2024-03-09T08:52:30.945364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:15:54.027155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947171+0000","last_change":"2024-03-09T08:52:29.595771+0000","last_active":"2024-03-09T08:52:30.947171+0000","last_peered":"2024-03-09T08:52:30.947171+0000","last_clean":"2024-03-09T08:52:30.947171+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T08:52:30.947171+0000","last_undegraded":"2024-03-09T08:52:30.947171+0000","last_fullsized":"2024-03-09T08:52:30.947171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:23:48.898071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943880+0000","last_change":"2024-03-09T08:52:18.242311+0000","last_active":"2024-03-09T08:52:30.943880+0000","last_peered":"2024-03-09T08:52:30.943880+0000","last_clean":"2024-03-09T08:52:30.943880+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T08:52:30.943880+0000","last_undegraded":"2024-03-09T08:52:30.943880+0000","last_fullsized":"2024-03-09T08:52:30.943880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:48:15.559438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575834+0000","last_change":"2024-03-09T08:52:29.576792+0000","last_active":"2024-03-09T08:52:30.575834+0000","last_peered":"2024-03-09T08:52:30.575834+0000","last_clean":"2024-03-09T08:52:30.575834+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T08:52:30.575834+0000","last_undegraded":"2024-03-09T08:52:30.575834+0000","last_fullsized":"2024-03-09T08:52:30.575834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:21.364814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947241+0000","last_change":"2024-03-09T08:52:18.248341+0000","last_active":"2024-03-09T08:52:30.947241+0000","last_peered":"2024-03-09T08:52:30.947241+0000","last_clean":"2024-03-09T08:52:30.947241+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T08:52:30.947241+0000","last_undegraded":"2024-03-09T08:52:30.947241+0000","last_fullsized":"2024-03-09T08:52:30.947241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:18:02.716066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942730+0000","last_change":"2024-03-09T08:52:29.606330+0000","last_active":"2024-03-09T08:52:30.942730+0000","last_peered":"2024-03-09T08:52:30.942730+0000","last_clean":"2024-03-09T08:52:30.942730+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T08:52:30.942730+0000","last_undegraded":"2024-03-09T08:52:30.942730+0000","last_fullsized":"2024-03-09T08:52:30.942730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:44:50.347124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597559+0000","last_change":"2024-03-09T08:52:18.222968+0000","last_active":"2024-03-09T08:52:30.597559+0000","last_peered":"2024-03-09T08:52:30.597559+0000","last_clean":"2024-03-09T08:52:30.597559+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T08:52:30.597559+0000","last_undegraded":"2024-03-09T08:52:30.597559+0000","last_fullsized":"2024-03-09T08:52:30.597559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:52:29.523963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943337+0000","last_change":"2024-03-09T08:52:29.591813+0000","last_active":"2024-03-09T08:52:30.943337+0000","last_peered":"2024-03-09T08:52:30.943337+0000","last_clean":"2024-03-09T08:52:30.943337+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T08:52:30.943337+0000","last_undegraded":"2024-03-09T08:52:30.943337+0000","last_fullsized":"2024-03-09T08:52:30.943337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:40:31.568253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601942+0000","last_change":"2024-03-09T08:52:18.225812+0000","last_active":"2024-03-09T08:52:30.601942+0000","last_peered":"2024-03-09T08:52:30.601942+0000","last_clean":"2024-03-09T08:52:30.601942+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T08:52:30.601942+0000","last_undegraded":"2024-03-09T08:52:30.601942+0000","last_fullsized":"2024-03-09T08:52:30.601942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:09:32.044185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943686+0000","last_change":"2024-03-09T08:52:29.595055+0000","last_active":"2024-03-09T08:52:30.943686+0000","last_peered":"2024-03-09T08:52:30.943686+0000","last_clean":"2024-03-09T08:52:30.943686+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T08:52:30.943686+0000","last_undegraded":"2024-03-09T08:52:30.943686+0000","last_fullsized":"2024-03-09T08:52:30.943686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:53.209169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948709+0000","last_change":"2024-03-09T08:52:18.248973+0000","last_active":"2024-03-09T08:52:30.948709+0000","last_peered":"2024-03-09T08:52:30.948709+0000","last_clean":"2024-03-09T08:52:30.948709+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T08:52:30.948709+0000","last_undegraded":"2024-03-09T08:52:30.948709+0000","last_fullsized":"2024-03-09T08:52:30.948709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:09:11.372616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":101489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":102507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T08:52:35.601 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-09T08:52:35.601 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-09T08:52:35.601 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-09T08:52:35.601 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-09T08:52:35.602 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-09T08:52:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.857+0000 7f9091600640 1 Processor -- start 2024-03-09T08:52:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.857+0000 7f9091600640 1 -- start start 2024-03-09T08:52:35.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.858+0000 7f9091600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f908c153300 0x7f908c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.858+0000 7f9091600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f908c0cadb0 0x7f908c0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.858+0000 7f9091600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908c157000 con 0x7f908c0c5e40 2024-03-09T08:52:35.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.858+0000 7f9091600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908c0ae430 con 0x7f908c153300 2024-03-09T08:52:35.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.858+0000 7f9091600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908c0ae6f0 con 0x7f908c0cadb0 2024-03-09T08:52:35.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.859+0000 7f908b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f908c0c5e40 0x7f908c156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54424/0 (socket says 172.21.15.67:54424) 2024-03-09T08:52:35.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.859+0000 7f908b400640 1 -- 172.21.15.67:0/1732261218 learned_addr learned my addr 172.21.15.67:0/1732261218 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:35.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.859+0000 7f908be00640 1 --2- 172.21.15.67:0/1732261218 >> v2:172.21.15.134:3300/0 conn(0x7f908c153300 0x7f908c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.859+0000 7f9090c00640 1 --2- 172.21.15.67:0/1732261218 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f908c0cadb0 0x7f908c0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.859+0000 7f908aa00640 1 -- 172.21.15.67:0/1732261218 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257642291 0 0) 0x7f908c157000 con 0x7f908c0c5e40 2024-03-09T08:52:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.859+0000 7f908aa00640 1 -- 172.21.15.67:0/1732261218 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9070003520 con 0x7f908c0c5e40 2024-03-09T08:52:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.860+0000 7f908be00640 1 -- 172.21.15.67:0/1732261218 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f908c0cadb0 msgr2=0x7f908c0b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.860+0000 7f908aa00640 1 -- 172.21.15.67:0/1732261218 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 249363157 0 0) 0x7f9070003520 con 0x7f908c0c5e40 2024-03-09T08:52:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.860+0000 7f908be00640 1 --2- 172.21.15.67:0/1732261218 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f908c0cadb0 0x7f908c0b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.860+0000 7f908be00640 1 -- 172.21.15.67:0/1732261218 >> v1:172.21.15.67:6789/0 conn(0x7f908c0c5e40 legacy=0x7f908c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.860+0000 7f908be00640 1 -- 172.21.15.67:0/1732261218 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c155b10 con 0x7f908c153300 2024-03-09T08:52:35.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.862+0000 7f908be00640 1 --2- 172.21.15.67:0/1732261218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f908c153300 0x7f908c0b0000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f907c0097a0 tx=0x7f907c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c922c373ee143a56 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:35.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.862+0000 7f908aa00640 1 -- 172.21.15.67:0/1732261218 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c037070 con 0x7f908c153300 2024-03-09T08:52:35.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.862+0000 7f908aa00640 1 -- 172.21.15.67:0/1732261218 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c030510 con 0x7f908c153300 2024-03-09T08:52:35.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.862+0000 7f908aa00640 1 -- 172.21.15.67:0/1732261218 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780019f0 con 0x7f908c153300 2024-03-09T08:52:35.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.863+0000 7f9091600640 1 -- 172.21.15.67:0/1732261218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f908c153300 msgr2=0x7f908c0b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.863+0000 7f9091600640 1 --2- 172.21.15.67:0/1732261218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f908c153300 0x7f908c0b0000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f907c0097a0 tx=0x7f907c02b790 comp rx=0 tx=0).stop 2024-03-09T08:52:35.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.863+0000 7f9091600640 1 -- 172.21.15.67:0/1732261218 shutdown_connections 2024-03-09T08:52:35.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.863+0000 7f9091600640 1 --2- 172.21.15.67:0/1732261218 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f908c0cadb0 0x7f908c0b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.863+0000 7f9091600640 1 --2- 172.21.15.67:0/1732261218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f908c153300 0x7f908c0b0000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.863+0000 7f9091600640 1 -- 172.21.15.67:0/1732261218 >> 172.21.15.67:0/1732261218 conn(0x7f908c0732d0 msgr2=0x7f908c0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:35.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.863+0000 7f9091600640 1 -- 172.21.15.67:0/1732261218 shutdown_connections 2024-03-09T08:52:35.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.863+0000 7f9091600640 1 -- 172.21.15.67:0/1732261218 wait complete. 2024-03-09T08:52:35.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.865+0000 7f9091600640 1 Processor -- start 2024-03-09T08:52:35.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.865+0000 7f9091600640 1 -- start start 2024-03-09T08:52:35.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.866+0000 7f9091600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f908c0c5e40 0x7f908c1ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.866+0000 7f9091600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f908c0cadb0 0x7f908c0c0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.866+0000 7f9091600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f908c153300 0x7f908c0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.866+0000 7f9091600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c0c59e0 con 0x7f908c153300 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.866+0000 7f9091600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c159e90 con 0x7f908c0c5e40 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.866+0000 7f9091600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908c156720 con 0x7f908c0cadb0 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.866+0000 7f9090c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f908c153300 0x7f908c0ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.866+0000 7f9090c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f908c153300 0x7f908c0ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51346/0 (socket says 172.21.15.67:51346) 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.866+0000 7f9090c00640 1 -- 172.21.15.67:0/947452933 learned_addr learned my addr 172.21.15.67:0/947452933 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.866+0000 7f908be00640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f908c0c5e40 0x7f908c1ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f9090c00640 1 -- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f908c0cadb0 msgr2=0x7f908c0c0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f908b400640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f908c0cadb0 0x7f908c0c0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f9090c00640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f908c0cadb0 0x7f908c0c0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f9090c00640 1 -- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f908c0c5e40 msgr2=0x7f908c1ebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f9090c00640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f908c0c5e40 0x7f908c1ebc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f9090c00640 1 -- 172.21.15.67:0/947452933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c1ec1d0 con 0x7f908c153300 2024-03-09T08:52:35.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f908be00640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f908c0c5e40 0x7f908c1ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:35.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f9090c00640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f908c153300 0x7f908c0ac970 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f908400e930 tx=0x7f908400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:35.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f9083e00640 1 -- 172.21.15.67:0/947452933 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084017070 con 0x7f908c153300 2024-03-09T08:52:35.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f9083e00640 1 -- 172.21.15.67:0/947452933 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908400c980 con 0x7f908c153300 2024-03-09T08:52:35.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f9083e00640 1 -- 172.21.15.67:0/947452933 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908400cd70 con 0x7f908c153300 2024-03-09T08:52:35.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f9091600640 1 -- 172.21.15.67:0/947452933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908c0ab1f0 con 0x7f908c153300 2024-03-09T08:52:35.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.867+0000 7f9091600640 1 -- 172.21.15.67:0/947452933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c0abe90 con 0x7f908c153300 2024-03-09T08:52:35.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.869+0000 7f9091600640 1 -- 172.21.15.67:0/947452933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908c156a50 con 0x7f908c153300 2024-03-09T08:52:35.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.870+0000 7f9083e00640 1 -- 172.21.15.67:0/947452933 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f908401a070 con 0x7f908c153300 2024-03-09T08:52:35.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.870+0000 7f9083e00640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f906c06b4d0 0x7f906c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:35.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.870+0000 7f9083e00640 1 -- 172.21.15.67:0/947452933 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f908408a910 con 0x7f908c153300 2024-03-09T08:52:35.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.871+0000 7f908be00640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f906c06b4d0 0x7f906c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:35.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.872+0000 7f908be00640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f906c06b4d0 0x7f906c06d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f907c034000 tx=0x7f907c02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:35.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:35.875+0000 7f9083e00640 1 -- 172.21.15.67:0/947452933 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f908c156a50 con 0x7f908c153300 2024-03-09T08:52:36.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.134+0000 7f9091600640 1 -- 172.21.15.67:0/947452933 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f908c04f3c0 con 0x7f908c153300 2024-03-09T08:52:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.139+0000 7f9083e00640 1 -- 172.21.15.67:0/947452933 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171789 (secure 0 0 0) 0x7f908c1ec1d0 con 0x7f908c153300 2024-03-09T08:52:36.139 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:36.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.143+0000 7f9091600640 1 -- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f906c06b4d0 msgr2=0x7f906c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.143+0000 7f9091600640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f906c06b4d0 0x7f906c06d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f907c034000 tx=0x7f907c02f000 comp rx=0 tx=0).stop 2024-03-09T08:52:36.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.143+0000 7f9091600640 1 -- 172.21.15.67:0/947452933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f908c153300 msgr2=0x7f908c0ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.143+0000 7f9091600640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f908c153300 0x7f908c0ac970 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f908400e930 tx=0x7f908400edf0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.143+0000 7f9091600640 1 -- 172.21.15.67:0/947452933 shutdown_connections 2024-03-09T08:52:36.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.143+0000 7f9091600640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f906c06b4d0 0x7f906c06d990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.143+0000 7f9091600640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f908c153300 0x7f908c0ac970 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.143+0000 7f9091600640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f908c0cadb0 0x7f908c0c0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.143+0000 7f9091600640 1 --2- 172.21.15.67:0/947452933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f908c0c5e40 0x7f908c1ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.143+0000 7f9091600640 1 -- 172.21.15.67:0/947452933 >> 172.21.15.67:0/947452933 conn(0x7f908c0732d0 msgr2=0x7f908c0aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:36.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.144+0000 7f9091600640 1 -- 172.21.15.67:0/947452933 shutdown_connections 2024-03-09T08:52:36.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.144+0000 7f9091600640 1 -- 172.21.15.67:0/947452933 wait complete. 2024-03-09T08:52:36.166 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":1750216114},{"type":"v1","addr":"172.21.15.134:6833","nonce":1750216114}]},"active_addr":"172.21.15.134:6833/1750216114","active_change":"2024-03-09T08:51:58.646808+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":3244893796}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":4156163466}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":1025440625}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":944570794}]}]} 2024-03-09T08:52:36.169 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-09T08:52:36.170 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-09T08:52:36.170 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:36.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.384+0000 7fc3fc800640 1 Processor -- start 2024-03-09T08:52:36.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.384+0000 7fc3fc800640 1 -- start start 2024-03-09T08:52:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.386+0000 7fc3fc800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc3f40aefb0 0x7fc3f40c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.386+0000 7fc3fc800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 0x7fc3f40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.386+0000 7fc3fc800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f40c4010 con 0x7fc3f40737a0 2024-03-09T08:52:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.386+0000 7fc3fc800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f406fa10 con 0x7fc3f40aefb0 2024-03-09T08:52:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.386+0000 7fc3fc800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40af390 con 0x7fc3f406fea0 2024-03-09T08:52:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.387+0000 7fc3fa800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 0x7fc3f40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.387+0000 7fc3fa800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 0x7fc3f40702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46150/0 (socket says 172.21.15.67:46150) 2024-03-09T08:52:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.387+0000 7fc3f9400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc3f40737a0 0x7fc3f40c6160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:60902/0 (socket says 172.21.15.67:60902) 2024-03-09T08:52:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.387+0000 7fc3fa800640 1 -- 172.21.15.67:0/3883081118 learned_addr learned my addr 172.21.15.67:0/3883081118 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.387+0000 7fc3f9e00640 1 --2- 172.21.15.67:0/3883081118 >> v2:172.21.15.134:3300/0 conn(0x7fc3f40aefb0 0x7fc3f40c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.388+0000 7fc3fa800640 1 -- 172.21.15.67:0/3883081118 >> v2:172.21.15.134:3300/0 conn(0x7fc3f40aefb0 msgr2=0x7fc3f40c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.388+0000 7fc3fa800640 1 --2- 172.21.15.67:0/3883081118 >> v2:172.21.15.134:3300/0 conn(0x7fc3f40aefb0 0x7fc3f40c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.388+0000 7fc3fa800640 1 -- 172.21.15.67:0/3883081118 >> v1:172.21.15.134:6789/0 conn(0x7fc3f40737a0 legacy=0x7fc3f40c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.388+0000 7fc3fa800640 1 -- 172.21.15.67:0/3883081118 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f4070830 con 0x7fc3f406fea0 2024-03-09T08:52:36.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.389+0000 7fc3fa800640 1 --2- 172.21.15.67:0/3883081118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 0x7fc3f40702f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc3f406fb90 tx=0x7fc3f002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0fb19e4c6d8bffd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.389+0000 7fc3f8a00640 1 -- 172.21.15.67:0/3883081118 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f002ca50 con 0x7fc3f406fea0 2024-03-09T08:52:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.389+0000 7fc3f8a00640 1 -- 172.21.15.67:0/3883081118 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f002cbf0 con 0x7fc3f406fea0 2024-03-09T08:52:36.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.389+0000 7fc3f8a00640 1 -- 172.21.15.67:0/3883081118 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00343f0 con 0x7fc3f406fea0 2024-03-09T08:52:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.390+0000 7fc3fc800640 1 -- 172.21.15.67:0/3883081118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 msgr2=0x7fc3f40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.390+0000 7fc3fc800640 1 --2- 172.21.15.67:0/3883081118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 0x7fc3f40702f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc3f406fb90 tx=0x7fc3f002beb0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.390+0000 7fc3fc800640 1 -- 172.21.15.67:0/3883081118 shutdown_connections 2024-03-09T08:52:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.390+0000 7fc3fc800640 1 --2- 172.21.15.67:0/3883081118 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 0x7fc3f40702f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.390+0000 7fc3fc800640 1 --2- 172.21.15.67:0/3883081118 >> v2:172.21.15.134:3300/0 conn(0x7fc3f40aefb0 0x7fc3f40c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.390+0000 7fc3fc800640 1 -- 172.21.15.67:0/3883081118 >> 172.21.15.67:0/3883081118 conn(0x7fc3f4066d90 msgr2=0x7fc3f40671a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:36.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.390+0000 7fc3fc800640 1 -- 172.21.15.67:0/3883081118 shutdown_connections 2024-03-09T08:52:36.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.390+0000 7fc3fc800640 1 -- 172.21.15.67:0/3883081118 wait complete. 2024-03-09T08:52:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.392+0000 7fc3fc800640 1 Processor -- start 2024-03-09T08:52:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.392+0000 7fc3fc800640 1 -- start start 2024-03-09T08:52:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.393+0000 7fc3fc800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 0x7fc3f40c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.393+0000 7fc3fc800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3f40737a0 0x7fc3f41ef110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.393+0000 7fc3fc800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3f40aefb0 0x7fc3f41ef650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.393+0000 7fc3fc800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41544f0 con 0x7fc3f40737a0 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.393+0000 7fc3fc800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40c42b0 con 0x7fc3f40aefb0 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.393+0000 7fc3fc800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40c4590 con 0x7fc3f406fea0 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.393+0000 7fc3f9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3f40737a0 0x7fc3f41ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.393+0000 7fc3f9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3f40737a0 0x7fc3f41ef110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51354/0 (socket says 172.21.15.67:51354) 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.393+0000 7fc3f9400640 1 -- 172.21.15.67:0/3024810051 learned_addr learned my addr 172.21.15.67:0/3024810051 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.394+0000 7fc3f9e00640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 0x7fc3f40c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.394+0000 7fc3fa800640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3f40aefb0 0x7fc3f41ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.394+0000 7fc3f9400640 1 -- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 msgr2=0x7fc3f40c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.394+0000 7fc3f9400640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 0x7fc3f40c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.394+0000 7fc3f9400640 1 -- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3f40aefb0 msgr2=0x7fc3f41ef650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.394+0000 7fc3f9400640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3f40aefb0 0x7fc3f41ef650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.394+0000 7fc3f9400640 1 -- 172.21.15.67:0/3024810051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f414a800 con 0x7fc3f40737a0 2024-03-09T08:52:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.394+0000 7fc3f9400640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3f40737a0 0x7fc3f41ef110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400c500 tx=0x7fc3e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.395+0000 7fc3ee000640 1 -- 172.21.15.67:0/3024810051 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4018070 con 0x7fc3f40737a0 2024-03-09T08:52:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.395+0000 7fc3ee000640 1 -- 172.21.15.67:0/3024810051 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e4007500 con 0x7fc3f40737a0 2024-03-09T08:52:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.395+0000 7fc3ee000640 1 -- 172.21.15.67:0/3024810051 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40049e0 con 0x7fc3f40737a0 2024-03-09T08:52:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.395+0000 7fc3fc800640 1 -- 172.21.15.67:0/3024810051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f41ec200 con 0x7fc3f40737a0 2024-03-09T08:52:36.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.395+0000 7fc3fc800640 1 -- 172.21.15.67:0/3024810051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f4149790 con 0x7fc3f40737a0 2024-03-09T08:52:36.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.396+0000 7fc3fc800640 1 -- 172.21.15.67:0/3024810051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f4075a00 con 0x7fc3f40737a0 2024-03-09T08:52:36.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.397+0000 7fc3ee000640 1 -- 172.21.15.67:0/3024810051 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc3e401b070 con 0x7fc3f40737a0 2024-03-09T08:52:36.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.398+0000 7fc3ee000640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc3d806b4d0 0x7fc3d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.399+0000 7fc3ee000640 1 -- 172.21.15.67:0/3024810051 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fc3e4089930 con 0x7fc3f40737a0 2024-03-09T08:52:36.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.399+0000 7fc3f9e00640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc3d806b4d0 0x7fc3d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.400+0000 7fc3f9e00640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc3d806b4d0 0x7fc3d806d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40c8310 tx=0x7fc3e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:36.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.403+0000 7fc3ee000640 1 -- 172.21.15.67:0/3024810051 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc3e4058470 con 0x7fc3f40737a0 2024-03-09T08:52:36.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.607+0000 7fc3fc800640 1 -- 172.21.15.67:0/3024810051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3f4073580 con 0x7fc3f40737a0 2024-03-09T08:52:36.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.608+0000 7fc3ee000640 1 -- 172.21.15.67:0/3024810051 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20307 (secure 0 0 0) 0x7fc3e4057d70 con 0x7fc3f40737a0 2024-03-09T08:52:36.605 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:36.605 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:30.565529+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:36.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.611+0000 7fc3fc800640 1 -- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc3d806b4d0 msgr2=0x7fc3d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.611+0000 7fc3fc800640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc3d806b4d0 0x7fc3d806d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40c8310 tx=0x7fc3e8008040 comp rx=0 tx=0).stop 2024-03-09T08:52:36.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.611+0000 7fc3fc800640 1 -- 172.21.15.67:0/3024810051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3f40737a0 msgr2=0x7fc3f41ef110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.611+0000 7fc3fc800640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3f40737a0 0x7fc3f41ef110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400c500 tx=0x7fc3e400c9c0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.611+0000 7fc3fc800640 1 -- 172.21.15.67:0/3024810051 shutdown_connections 2024-03-09T08:52:36.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.611+0000 7fc3fc800640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc3d806b4d0 0x7fc3d806d990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.611+0000 7fc3fc800640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3f40aefb0 0x7fc3f41ef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.611+0000 7fc3fc800640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3f40737a0 0x7fc3f41ef110 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.611+0000 7fc3fc800640 1 --2- 172.21.15.67:0/3024810051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3f406fea0 0x7fc3f40c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.611+0000 7fc3fc800640 1 -- 172.21.15.67:0/3024810051 >> 172.21.15.67:0/3024810051 conn(0x7fc3f4066d90 msgr2=0x7fc3f4067170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:36.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.612+0000 7fc3fc800640 1 -- 172.21.15.67:0/3024810051 shutdown_connections 2024-03-09T08:52:36.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.612+0000 7fc3fc800640 1 -- 172.21.15.67:0/3024810051 wait complete. 2024-03-09T08:52:36.627 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-09T08:52:36.627 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:36.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.841+0000 7fa831800640 1 Processor -- start 2024-03-09T08:52:36.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.841+0000 7fa831800640 1 -- start start 2024-03-09T08:52:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.842+0000 7fa831800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa82c153300 0x7fa82c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.843+0000 7fa831800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c157000 con 0x7fa82c0cadb0 2024-03-09T08:52:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.843+0000 7fa831800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c0ae430 con 0x7fa82c0c5e40 2024-03-09T08:52:36.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.843+0000 7fa831800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0ae6f0 con 0x7fa82c153300 2024-03-09T08:52:36.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.843+0000 7fa830e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa82c0cadb0 0x7fa82c160490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54428/0 (socket says 172.21.15.67:54428) 2024-03-09T08:52:36.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.843+0000 7fa830e00640 1 -- 172.21.15.67:0/509768792 learned_addr learned my addr 172.21.15.67:0/509768792 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:36.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.843+0000 7fa82be00640 1 --2- 172.21.15.67:0/509768792 >> v2:172.21.15.134:3300/0 conn(0x7fa82c153300 0x7fa82c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.843+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659771 0 0) 0x7fa82c157000 con 0x7fa82c0cadb0 2024-03-09T08:52:36.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa810003520 con 0x7fa82c0cadb0 2024-03-09T08:52:36.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3153361597 0 0) 0x7fa810003520 con 0x7fa82c0cadb0 2024-03-09T08:52:36.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 >> v2:172.21.15.134:3300/0 conn(0x7fa82c153300 msgr2=0x7fa82c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:36.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 --2- 172.21.15.67:0/509768792 >> v2:172.21.15.134:3300/0 conn(0x7fa82c153300 0x7fa82c0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 >> v1:172.21.15.134:6789/0 conn(0x7fa82c0c5e40 legacy=0x7fa82c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c155b10 con 0x7fa82c0cadb0 2024-03-09T08:52:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fa8240019f0 con 0x7fa82c0cadb0 2024-03-09T08:52:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 >> v1:172.21.15.67:6789/0 conn(0x7fa82c0cadb0 legacy=0x7fa82c160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 0x7fa810003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa810004800 0x7fa810004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8100051c0 0x7fa81000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0ae6f0 con 0x7fa810003a20 2024-03-09T08:52:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81000c0f0 con 0x7fa810004800 2024-03-09T08:52:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81000c2f0 con 0x7fa8100051c0 2024-03-09T08:52:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.844+0000 7fa82be00640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 0x7fa810003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.845+0000 7fa830e00640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8100051c0 0x7fa81000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.845+0000 7fa82be00640 1 -- 172.21.15.67:0/509768792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8100051c0 msgr2=0x7fa81000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.845+0000 7fa82be00640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8100051c0 0x7fa81000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.845+0000 7fa823e00640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa810004800 0x7fa810004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.845+0000 7fa82be00640 1 -- 172.21.15.67:0/509768792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa810004800 msgr2=0x7fa810004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.845+0000 7fa82be00640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa810004800 0x7fa810004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.845+0000 7fa82be00640 1 -- 172.21.15.67:0/509768792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa810003520 con 0x7fa810003a20 2024-03-09T08:52:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.845+0000 7fa830e00640 1 -- 172.21.15.67:0/509768792 reap_dead start 2024-03-09T08:52:36.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.845+0000 7fa823e00640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa810004800 0x7fa810004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.846+0000 7fa82be00640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 0x7fa810003e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009ac0 tx=0x7fa810005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc2e3d488ed53cf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.846+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c039070 con 0x7fa810003a20 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.846+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81c02c6d0 con 0x7fa810003a20 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.846+0000 7fa82b400640 1 -- 172.21.15.67:0/509768792 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c009690 con 0x7fa810003a20 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.847+0000 7fa831800640 1 -- 172.21.15.67:0/509768792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 msgr2=0x7fa810003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.847+0000 7fa831800640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 0x7fa810003e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009ac0 tx=0x7fa810005830 comp rx=0 tx=0).stop 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.847+0000 7fa831800640 1 -- 172.21.15.67:0/509768792 shutdown_connections 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.847+0000 7fa831800640 1 --2- 172.21.15.67:0/509768792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 0x7fa810003e40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.847+0000 7fa831800640 1 -- 172.21.15.67:0/509768792 >> 172.21.15.67:0/509768792 conn(0x7fa82c0732d0 msgr2=0x7fa82c0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.847+0000 7fa831800640 1 -- 172.21.15.67:0/509768792 shutdown_connections 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.847+0000 7fa831800640 1 -- 172.21.15.67:0/509768792 wait complete. 2024-03-09T08:52:36.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.849+0000 7fa831800640 1 Processor -- start 2024-03-09T08:52:36.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.849+0000 7fa831800640 1 -- start start 2024-03-09T08:52:36.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa831800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 0x7fa82c06b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa831800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa82c0ab010 0x7fa82c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa831800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa82c0c6fd0 0x7fa82c14bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa831800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0c59e0 con 0x7fa810003a20 2024-03-09T08:52:36.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa831800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c159e90 con 0x7fa82c0c6fd0 2024-03-09T08:52:36.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa831800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0b0000 con 0x7fa82c0ab010 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa82be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 0x7fa82c06b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa82be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 0x7fa82c06b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51376/0 (socket says 172.21.15.67:51376) 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa82be00640 1 -- 172.21.15.67:0/3320237925 learned_addr learned my addr 172.21.15.67:0/3320237925 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa823e00640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa82c0ab010 0x7fa82c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa82be00640 1 -- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa82c0ab010 msgr2=0x7fa82c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa82be00640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa82c0ab010 0x7fa82c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa830e00640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa82c0c6fd0 0x7fa82c14bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa82be00640 1 -- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa82c0c6fd0 msgr2=0x7fa82c14bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa82be00640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa82c0c6fd0 0x7fa82c14bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.850+0000 7fa82be00640 1 -- 172.21.15.67:0/3320237925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c06b650 con 0x7fa810003a20 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.851+0000 7fa82be00640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 0x7fa82c06b8a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa81c037040 tx=0x7fa81c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.851+0000 7fa828c00640 1 -- 172.21.15.67:0/3320237925 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c039050 con 0x7fa810003a20 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.851+0000 7fa828c00640 1 -- 172.21.15.67:0/3320237925 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81c045070 con 0x7fa810003a20 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.851+0000 7fa828c00640 1 -- 172.21.15.67:0/3320237925 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c003a70 con 0x7fa810003a20 2024-03-09T08:52:36.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.851+0000 7fa831800640 1 -- 172.21.15.67:0/3320237925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c06b370 con 0x7fa810003a20 2024-03-09T08:52:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.851+0000 7fa831800640 1 -- 172.21.15.67:0/3320237925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c1ec6a0 con 0x7fa810003a20 2024-03-09T08:52:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.853+0000 7fa831800640 1 -- 172.21.15.67:0/3320237925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c04f3c0 con 0x7fa810003a20 2024-03-09T08:52:36.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.853+0000 7fa828c00640 1 -- 172.21.15.67:0/3320237925 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa81c003c10 con 0x7fa810003a20 2024-03-09T08:52:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.855+0000 7fa828c00640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa80806b500 0x7fa80806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.855+0000 7fa828c00640 1 -- 172.21.15.67:0/3320237925 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fa81c0ac420 con 0x7fa810003a20 2024-03-09T08:52:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.855+0000 7fa823e00640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa80806b500 0x7fa80806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.856+0000 7fa823e00640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa80806b500 0x7fa80806d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0c9610 tx=0x7fa814009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:36.858+0000 7fa828c00640 1 -- 172.21.15.67:0/3320237925 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa81c04f020 con 0x7fa810003a20 2024-03-09T08:52:37.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.096+0000 7fa831800640 1 -- 172.21.15.67:0/3320237925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa82c158180 con 0x7fa810003a20 2024-03-09T08:52:37.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.098+0000 7fa828c00640 1 -- 172.21.15.67:0/3320237925 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20307 (secure 0 0 0) 0x7fa81c07b080 con 0x7fa810003a20 2024-03-09T08:52:37.096 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:37.096 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:30.565529+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:37.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.101+0000 7fa831800640 1 -- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa80806b500 msgr2=0x7fa80806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.101+0000 7fa831800640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa80806b500 0x7fa80806d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0c9610 tx=0x7fa814009040 comp rx=0 tx=0).stop 2024-03-09T08:52:37.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.101+0000 7fa831800640 1 -- 172.21.15.67:0/3320237925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 msgr2=0x7fa82c06b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.101+0000 7fa831800640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 0x7fa82c06b8a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa81c037040 tx=0x7fa81c004730 comp rx=0 tx=0).stop 2024-03-09T08:52:37.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.102+0000 7fa831800640 1 -- 172.21.15.67:0/3320237925 shutdown_connections 2024-03-09T08:52:37.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.102+0000 7fa831800640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa80806b500 0x7fa80806d9c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.102+0000 7fa831800640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa82c0c6fd0 0x7fa82c14bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.102+0000 7fa831800640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa82c0ab010 0x7fa82c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.102+0000 7fa831800640 1 --2- 172.21.15.67:0/3320237925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa810003a20 0x7fa82c06b8a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.102+0000 7fa831800640 1 -- 172.21.15.67:0/3320237925 >> 172.21.15.67:0/3320237925 conn(0x7fa82c0732d0 msgr2=0x7fa82c1514d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.102+0000 7fa831800640 1 -- 172.21.15.67:0/3320237925 shutdown_connections 2024-03-09T08:52:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.102+0000 7fa831800640 1 -- 172.21.15.67:0/3320237925 wait complete. 2024-03-09T08:52:37.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-09T08:52:37.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-09T08:52:37.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-09T08:52:37.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-09T08:52:37.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-09T08:52:37.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-09T08:52:37.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-09T08:52:37.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-09T08:52:37.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.459+0000 7f04b3a00640 1 Processor -- start 2024-03-09T08:52:37.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.459+0000 7f04b3a00640 1 -- start start 2024-03-09T08:52:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.462+0000 7f04b3a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f04ac0ac9c0 0x7f04ac0ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.462+0000 7f04b3a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ac0ac4e0 con 0x7f04ac0c1d70 2024-03-09T08:52:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.462+0000 7f04b3a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ac0aa550 con 0x7f04ac14c9a0 2024-03-09T08:52:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.462+0000 7f04b3a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ac06d590 con 0x7f04ac0ac9c0 2024-03-09T08:52:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.462+0000 7f1c28200640 1 Processor -- start 2024-03-09T08:52:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.462+0000 7f1c28200640 1 -- start start 2024-03-09T08:52:37.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.462+0000 7f04b1a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f04ac0ac9c0 0x7f04ac0ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.463+0000 7f04b1000640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f04ac14c9a0 0x7f04ac1567c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:60916/0 (socket says 172.21.15.67:60916) 2024-03-09T08:52:37.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.463+0000 7f04b1000640 1 -- 172.21.15.67:0/1386173922 learned_addr learned my addr 172.21.15.67:0/1386173922 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.464+0000 7f1c28200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d590 0x7f1c20163590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.464+0000 7f1c28200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1c2006d970 0x7f1c200aab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.464+0000 7f1c28200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c20073e50 con 0x7f1c2008c7a0 2024-03-09T08:52:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.464+0000 7f1c28200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2006de00 con 0x7f1c2006d970 2024-03-09T08:52:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.464+0000 7f1c28200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200c39b0 con 0x7f1c2006d590 2024-03-09T08:52:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.465+0000 7f04b1a00640 1 -- 172.21.15.67:0/1386173922 >> v1:172.21.15.134:6789/0 conn(0x7f04ac14c9a0 legacy=0x7f04ac1567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.465+0000 7f1c25800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d590 0x7f1c20163590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.465+0000 7f1c25800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d590 0x7f1c20163590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46176/0 (socket says 172.21.15.67:46176) 2024-03-09T08:52:37.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.465+0000 7f1c25800640 1 -- 172.21.15.67:0/1983501167 learned_addr learned my addr 172.21.15.67:0/1983501167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.465+0000 7f1c26200640 1 --2- 172.21.15.67:0/1983501167 >> v2:172.21.15.134:3300/0 conn(0x7f1c2006d970 0x7f1c200aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.465+0000 7f04b1a00640 1 -- 172.21.15.67:0/1386173922 >> v1:172.21.15.67:6789/0 conn(0x7f04ac0c1d70 legacy=0x7f04ac0c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.465+0000 7f04b1a00640 1 -- 172.21.15.67:0/1386173922 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0aa830 con 0x7f04ac0ac9c0 2024-03-09T08:52:37.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.466+0000 7f1c25800640 1 -- 172.21.15.67:0/1983501167 >> v2:172.21.15.134:3300/0 conn(0x7f1c2006d970 msgr2=0x7f1c200aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.466+0000 7f1c25800640 1 --2- 172.21.15.67:0/1983501167 >> v2:172.21.15.134:3300/0 conn(0x7f1c2006d970 0x7f1c200aab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.466+0000 7f1c25800640 1 -- 172.21.15.67:0/1983501167 >> v1:172.21.15.134:6789/0 conn(0x7f1c2008c7a0 legacy=0x7f1c200aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.466+0000 7f1c25800640 1 -- 172.21.15.67:0/1983501167 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200ac250 con 0x7f1c2006d590 2024-03-09T08:52:37.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.468+0000 7f04b1a00640 1 --2- 172.21.15.67:0/1386173922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04ac0ac9c0 0x7f04ac0ace50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f049c0097a0 tx=0x7f049c02b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9539f8d64d1a161e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.468+0000 7f1c25800640 1 --2- 172.21.15.67:0/1983501167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d590 0x7f1c20163590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009870 tx=0x7f1c0c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=504ed89c0fa3ca11 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.468+0000 7f1c1fe00640 1 -- 172.21.15.67:0/1983501167 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c035020 con 0x7f1c2006d590 2024-03-09T08:52:37.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.468+0000 7f1c1fe00640 1 -- 172.21.15.67:0/1983501167 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c031890 con 0x7f1c2006d590 2024-03-09T08:52:37.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.468+0000 7f1c1fe00640 1 -- 172.21.15.67:0/1983501167 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c031ca0 con 0x7f1c2006d590 2024-03-09T08:52:37.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.469+0000 7f04ab400640 1 -- 172.21.15.67:0/1386173922 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c037070 con 0x7f04ac0ac9c0 2024-03-09T08:52:37.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.469+0000 7f04ab400640 1 -- 172.21.15.67:0/1386173922 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c030710 con 0x7f04ac0ac9c0 2024-03-09T08:52:37.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.469+0000 7f04ab400640 1 -- 172.21.15.67:0/1386173922 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c030ae0 con 0x7f04ac0ac9c0 2024-03-09T08:52:37.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.470+0000 7f04b3a00640 1 -- 172.21.15.67:0/1386173922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04ac0ac9c0 msgr2=0x7f04ac0ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.470+0000 7f04b3a00640 1 --2- 172.21.15.67:0/1386173922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04ac0ac9c0 0x7f04ac0ace50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f049c0097a0 tx=0x7f049c02b570 comp rx=0 tx=0).stop 2024-03-09T08:52:37.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.471+0000 7fb877200640 1 Processor -- start 2024-03-09T08:52:37.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.471+0000 7fb877200640 1 -- start start 2024-03-09T08:52:37.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.470+0000 7fba49c00640 1 Processor -- start 2024-03-09T08:52:37.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.470+0000 7fba49c00640 1 -- start start 2024-03-09T08:52:37.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.472+0000 7f58fce00640 1 Processor -- start 2024-03-09T08:52:37.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.472+0000 7f58fce00640 1 -- start start 2024-03-09T08:52:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.469+0000 7f1c28200640 1 -- 172.21.15.67:0/1983501167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d590 msgr2=0x7f1c20163590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.469+0000 7f1c28200640 1 --2- 172.21.15.67:0/1983501167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d590 0x7f1c20163590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009870 tx=0x7f1c0c02b930 comp rx=0 tx=0).stop 2024-03-09T08:52:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.469+0000 7f1c28200640 1 -- 172.21.15.67:0/1983501167 shutdown_connections 2024-03-09T08:52:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.469+0000 7f1c28200640 1 --2- 172.21.15.67:0/1983501167 >> v2:172.21.15.134:3300/0 conn(0x7f1c2006d970 0x7f1c200aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.469+0000 7f1c28200640 1 --2- 172.21.15.67:0/1983501167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d590 0x7f1c20163590 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.469+0000 7f1c28200640 1 -- 172.21.15.67:0/1983501167 >> 172.21.15.67:0/1983501167 conn(0x7f1c200c7370 msgr2=0x7f1c200c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.470+0000 7f1c28200640 1 -- 172.21.15.67:0/1983501167 shutdown_connections 2024-03-09T08:52:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.470+0000 7f1c28200640 1 -- 172.21.15.67:0/1983501167 wait complete. 2024-03-09T08:52:37.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.472+0000 7f1c28200640 1 Processor -- start 2024-03-09T08:52:37.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.472+0000 7fba49c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fba44157990 0x7fba4406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.472+0000 7fba49c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba440aaf90 con 0x7fba4406b580 2024-03-09T08:52:37.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.472+0000 7fba49c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba440c9710 con 0x7fba440c5ff0 2024-03-09T08:52:37.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.472+0000 7fba49c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba440ca040 con 0x7fba44157990 2024-03-09T08:52:37.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.472+0000 7fba42a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fba44157990 0x7fba4406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.472+0000 7fba42a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fba44157990 0x7fba4406b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51380/0 (socket says 172.21.15.67:51380) 2024-03-09T08:52:37.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.472+0000 7fba42a00640 1 -- 172.21.15.67:0/2946579890 learned_addr learned my addr 172.21.15.67:0/2946579890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.473+0000 7f04b3a00640 1 -- 172.21.15.67:0/1386173922 shutdown_connections 2024-03-09T08:52:37.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.473+0000 7f04b3a00640 1 --2- 172.21.15.67:0/1386173922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04ac0ac9c0 0x7f04ac0ace50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.473+0000 7f04b3a00640 1 -- 172.21.15.67:0/1386173922 >> 172.21.15.67:0/1386173922 conn(0x7f04ac0c7370 msgr2=0x7f04ac0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.474+0000 7fb877200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb87004dff0 0x7fb87004e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.472+0000 7f1c28200640 1 -- start start 2024-03-09T08:52:37.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.474+0000 7f1c28200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c2006d590 0x7f1c2020abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.474+0000 7f1c28200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d970 0x7f1c20166250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.474+0000 7f3c9be00640 1 Processor -- start 2024-03-09T08:52:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.474+0000 7f1c28200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c2008c7a0 0x7f1c2020cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.474+0000 7f3c9be00640 1 -- start start 2024-03-09T08:52:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7f1c24e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d970 0x7f1c20166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7f1c25800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c2006d590 0x7f1c2020abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7f1c25800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c2006d590 0x7f1c2020abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46214/0 (socket says 172.21.15.67:46214) 2024-03-09T08:52:37.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599584058 0 0) 0x7fba3c0013c0 con 0x7fba4406b580 2024-03-09T08:52:37.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba28003520 con 0x7fba4406b580 2024-03-09T08:52:37.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7fb876800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb87004dae0 0x7fb87004ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54462/0 (socket says 172.21.15.67:54462) 2024-03-09T08:52:37.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12218952 0 0) 0x7fba440c9710 con 0x7fba440c5ff0 2024-03-09T08:52:37.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba3c0013c0 con 0x7fba440c5ff0 2024-03-09T08:52:37.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7f1c24e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d970 0x7f1c20166250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46182/0 (socket says 172.21.15.67:46182) 2024-03-09T08:52:37.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7fb875400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb87004dff0 0x7fb87004e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7fb875e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb8700d7ae0 0x7fb8700071b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:60944/0 (socket says 172.21.15.67:60944) 2024-03-09T08:52:37.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7fb875400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb87004dff0 0x7fb87004e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46222/0 (socket says 172.21.15.67:46222) 2024-03-09T08:52:37.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7f1c26200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c2008c7a0 0x7f1c2020cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7f1c26200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c2008c7a0 0x7f1c2020cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51388/0 (socket says 172.21.15.67:51388) 2024-03-09T08:52:37.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7f1c25800640 1 -- 172.21.15.67:0/2480118455 learned_addr learned my addr 172.21.15.67:0/2480118455 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.475+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 110575736 0 0) 0x7fba28003520 con 0x7fba4406b580 2024-03-09T08:52:37.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 >> v2:172.21.15.67:3300/0 conn(0x7fba44157990 msgr2=0x7fba4406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 --2- 172.21.15.67:0/2946579890 >> v2:172.21.15.67:3300/0 conn(0x7fba44157990 0x7fba4406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 >> v1:172.21.15.134:6789/0 conn(0x7fba440c5ff0 legacy=0x7fba440ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba440c99f0 con 0x7fba4406b580 2024-03-09T08:52:37.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fb876800640 1 -- 172.21.15.67:0/774905421 learned_addr learned my addr 172.21.15.67:0/774905421 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:37.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7f1c28200640 1 -- 172.21.15.67:0/2480118455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200c3640 con 0x7f1c2008c7a0 2024-03-09T08:52:37.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7f1c28200640 1 -- 172.21.15.67:0/2480118455 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200abf70 con 0x7f1c2006d590 2024-03-09T08:52:37.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7f1c28200640 1 -- 172.21.15.67:0/2480118455 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200ab070 con 0x7f1c2006d970 2024-03-09T08:52:37.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fba3c001af0 con 0x7fba4406b580 2024-03-09T08:52:37.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 >> v1:172.21.15.67:6789/0 conn(0x7fba4406b580 legacy=0x7fba44164b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 --2- 172.21.15.67:0/2946579890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba28004bf0 0x7fba28004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 --2- 172.21.15.67:0/2946579890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba280037a0 0x7fba28003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 --2- 172.21.15.67:0/2946579890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba280041b0 0x7fba2800ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2800c190 con 0x7fba28004bf0 2024-03-09T08:52:37.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2800c340 con 0x7fba280037a0 2024-03-09T08:52:37.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2800c540 con 0x7fba280041b0 2024-03-09T08:52:37.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c940d8790 0x7f3c940df150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7fb877200640 1 -- 172.21.15.67:0/774905421 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87000f9d0 con 0x7fb87004dae0 2024-03-09T08:52:37.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f1c26200640 1 -- 172.21.15.67:0/2480118455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d970 msgr2=0x7f1c20166250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f1c26200640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c2006d970 0x7f1c20166250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7fb877200640 1 -- 172.21.15.67:0/774905421 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8700da4b0 con 0x7fb8700d7ae0 2024-03-09T08:52:37.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7fba43400640 1 --2- 172.21.15.67:0/2946579890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba280041b0 0x7fba2800ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f3c9be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c940064a0 0x7f3c94006910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f3c9be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9404b8d0 con 0x7f3c94049fa0 2024-03-09T08:52:37.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f3c9be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c9404c4a0 con 0x7f3c940064a0 2024-03-09T08:52:37.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f3c9be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940d8b70 con 0x7f3c940d8790 2024-03-09T08:52:37.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7fb874a00640 1 -- 172.21.15.67:0/774905421 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854298426 0 0) 0x7fb87000f9d0 con 0x7fb87004dae0 2024-03-09T08:52:37.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f1c26200640 1 -- 172.21.15.67:0/2480118455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c2006d590 msgr2=0x7f1c2020abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f1c26200640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c2006d590 0x7f1c2020abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7fba3be00640 1 --2- 172.21.15.67:0/2946579890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba280037a0 0x7fba28003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7fba42a00640 1 --2- 172.21.15.67:0/2946579890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba28004bf0 0x7fba28004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f58fce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f58f80c1d70 0x7f58f80c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f58fce00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f80ac4e0 con 0x7f58f80c0ce0 2024-03-09T08:52:37.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f58fce00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f80aa550 con 0x7f58f814c9a0 2024-03-09T08:52:37.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f58fce00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f806d590 con 0x7f58f80c1d70 2024-03-09T08:52:37.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7fb877200640 1 -- 172.21.15.67:0/774905421 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8700da790 con 0x7fb87004dff0 2024-03-09T08:52:37.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.477+0000 7f1c26200640 1 -- 172.21.15.67:0/2480118455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2020b0f0 con 0x7f1c2008c7a0 2024-03-09T08:52:37.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7fb874a00640 1 -- 172.21.15.67:0/774905421 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb85c003520 con 0x7fb87004dae0 2024-03-09T08:52:37.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7fb874a00640 1 -- 172.21.15.67:0/774905421 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329638616 0 0) 0x7fb8700da4b0 con 0x7fb8700d7ae0 2024-03-09T08:52:37.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.474+0000 7f04b3a00640 1 -- 172.21.15.67:0/1386173922 shutdown_connections 2024-03-09T08:52:37.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.474+0000 7f04b3a00640 1 -- 172.21.15.67:0/1386173922 wait complete. 2024-03-09T08:52:37.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7f1c26200640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c2008c7a0 0x7f1c2020cf20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c14003640 tx=0x7f1c14003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.476+0000 7f04b3a00640 1 Processor -- start 2024-03-09T08:52:37.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7f58f5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f58f80c1d70 0x7f58f80c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7fb874a00640 1 -- 172.21.15.67:0/774905421 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb87000f9d0 con 0x7fb8700d7ae0 2024-03-09T08:52:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7f58f5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f58f80c1d70 0x7f58f80c07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51416/0 (socket says 172.21.15.67:51416) 2024-03-09T08:52:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7f58f5400640 1 -- 172.21.15.67:0/1044512908 learned_addr learned my addr 172.21.15.67:0/1044512908 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7fba42a00640 1 -- 172.21.15.67:0/2946579890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba280041b0 msgr2=0x7fba2800ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7fba42a00640 1 --2- 172.21.15.67:0/2946579890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba280041b0 0x7fba2800ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7fba42a00640 1 -- 172.21.15.67:0/2946579890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba280037a0 msgr2=0x7fba28003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7fba42a00640 1 --2- 172.21.15.67:0/2946579890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba280037a0 0x7fba28003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7f3c9b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c940064a0 0x7f3c94006910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7f3c9b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c940064a0 0x7f3c94006910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51412/0 (socket says 172.21.15.67:51412) 2024-03-09T08:52:37.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.479+0000 7f1c1d600640 1 -- 172.21.15.67:0/2480118455 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1401b070 con 0x7f1c2008c7a0 2024-03-09T08:52:37.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.479+0000 7fb875400640 1 -- 172.21.15.67:0/774905421 >> v1:172.21.15.134:6789/0 conn(0x7fb8700d7ae0 legacy=0x7fb8700071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.479+0000 7f04b3a00640 1 -- start start 2024-03-09T08:52:37.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.479+0000 7fb874a00640 1 -- 172.21.15.67:0/774905421 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3902811759 0 0) 0x7fb87000f9d0 con 0x7fb8700d7ae0 2024-03-09T08:52:37.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7f3c9b400640 1 -- 172.21.15.67:0/701724922 learned_addr learned my addr 172.21.15.67:0/701724922 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.479+0000 7fb875400640 1 -- 172.21.15.67:0/774905421 >> v1:172.21.15.67:6789/0 conn(0x7fb87004dae0 legacy=0x7fb87004ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.478+0000 7f3c9aa00640 1 --2- 172.21.15.67:0/701724922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c940d8790 0x7f3c940df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.480+0000 7f58f5400640 1 -- 172.21.15.67:0/1044512908 >> v1:172.21.15.134:6789/0 conn(0x7f58f814c9a0 legacy=0x7f58f81567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.479+0000 7fba42a00640 1 -- 172.21.15.67:0/2946579890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba28003520 con 0x7fba28004bf0 2024-03-09T08:52:37.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.479+0000 7f1c28200640 1 -- 172.21.15.67:0/2480118455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c20165a90 con 0x7f1c2008c7a0 2024-03-09T08:52:37.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.480+0000 7fba43400640 1 -- 172.21.15.67:0/2946579890 reap_dead start 2024-03-09T08:52:37.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.479+0000 7f1c28200640 1 -- 172.21.15.67:0/2480118455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c20165d30 con 0x7f1c2008c7a0 2024-03-09T08:52:37.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.480+0000 7f1c28200640 1 -- 172.21.15.67:0/2480118455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c20075a00 con 0x7f1c2008c7a0 2024-03-09T08:52:37.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.481+0000 7fb875400640 1 -- 172.21.15.67:0/774905421 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8700daa70 con 0x7fb87004dff0 2024-03-09T08:52:37.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.480+0000 7f58f5400640 1 -- 172.21.15.67:0/1044512908 >> v1:172.21.15.67:6789/0 conn(0x7f58f80c0ce0 legacy=0x7f58f80ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.480+0000 7f58f5400640 1 -- 172.21.15.67:0/1044512908 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f80aa830 con 0x7f58f80c1d70 2024-03-09T08:52:37.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.481+0000 7fba42a00640 1 --2- 172.21.15.67:0/2946579890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba28004bf0 0x7fba28004ff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba34009c10 tx=0x7fba28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108021ae5646dde server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.482+0000 7f58f5400640 1 --2- 172.21.15.67:0/1044512908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f80c1d70 0x7f58f80c07a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009870 tx=0x7f58e402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d290cc48f67f684 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.481+0000 7f1c1d600640 1 -- 172.21.15.67:0/2480118455 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1400db50 con 0x7f1c2008c7a0 2024-03-09T08:52:37.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.481+0000 7f1c1d600640 1 -- 172.21.15.67:0/2480118455 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14016460 con 0x7f1c2008c7a0 2024-03-09T08:52:37.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.482+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34039070 con 0x7fba28004bf0 2024-03-09T08:52:37.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.482+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3402c7a0 con 0x7fba28004bf0 2024-03-09T08:52:37.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.482+0000 7fba42000640 1 -- 172.21.15.67:0/2946579890 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba340097e0 con 0x7fba28004bf0 2024-03-09T08:52:37.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.480+0000 7f3c9aa00640 1 -- 172.21.15.67:0/701724922 >> v2:172.21.15.67:3300/0 conn(0x7f3c940064a0 msgr2=0x7f3c94006910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.480+0000 7f3c9aa00640 1 --2- 172.21.15.67:0/701724922 >> v2:172.21.15.67:3300/0 conn(0x7f3c940064a0 0x7f3c94006910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.480+0000 7f3c9aa00640 1 -- 172.21.15.67:0/701724922 >> v1:172.21.15.67:6789/0 conn(0x7f3c94049fa0 legacy=0x7f3c9400edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.480+0000 7f3c9aa00640 1 -- 172.21.15.67:0/701724922 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9404bb70 con 0x7f3c940d8790 2024-03-09T08:52:37.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.482+0000 7f04b3a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04ac0ac9c0 0x7f04ac1ef750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.482+0000 7fb875400640 1 --2- 172.21.15.67:0/774905421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87004dff0 0x7fb87004e7d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb87802d5c0 tx=0x7fb8780a7b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc06b228671d9afa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.482+0000 7f58f4a00640 1 -- 172.21.15.67:0/1044512908 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4035020 con 0x7f58f80c1d70 2024-03-09T08:52:37.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.483+0000 7f58f4a00640 1 -- 172.21.15.67:0/1044512908 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e4031890 con 0x7f58f80c1d70 2024-03-09T08:52:37.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.483+0000 7f58f4a00640 1 -- 172.21.15.67:0/1044512908 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4031ca0 con 0x7f58f80c1d70 2024-03-09T08:52:37.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.483+0000 7fb874a00640 1 -- 172.21.15.67:0/774905421 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8640015f0 con 0x7fb87004dff0 2024-03-09T08:52:37.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.483+0000 7f04b3a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04ac0c1d70 0x7f04ac1ea350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.483+0000 7fb874a00640 1 -- 172.21.15.67:0/774905421 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8780a96e0 con 0x7fb87004dff0 2024-03-09T08:52:37.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.483+0000 7f04b3a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04ac14c9a0 0x7f04ac0a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.483+0000 7fb874a00640 1 -- 172.21.15.67:0/774905421 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8680015a0 con 0x7fb87004dff0 2024-03-09T08:52:37.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.483+0000 7fba49c00640 1 -- 172.21.15.67:0/2946579890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba28004bf0 msgr2=0x7fba28004ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.483+0000 7fba49c00640 1 --2- 172.21.15.67:0/2946579890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba28004bf0 0x7fba28004ff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba34009c10 tx=0x7fba28005830 comp rx=0 tx=0).stop 2024-03-09T08:52:37.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7fba49c00640 1 -- 172.21.15.67:0/2946579890 shutdown_connections 2024-03-09T08:52:37.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7f04b1000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04ac0ac9c0 0x7f04ac1ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7fba49c00640 1 --2- 172.21.15.67:0/2946579890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba28004bf0 0x7fba28004ff0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7fba49c00640 1 -- 172.21.15.67:0/2946579890 >> 172.21.15.67:0/2946579890 conn(0x7fba440cb670 msgr2=0x7fba440cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7fba49c00640 1 -- 172.21.15.67:0/2946579890 shutdown_connections 2024-03-09T08:52:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7fba49c00640 1 -- 172.21.15.67:0/2946579890 wait complete. 2024-03-09T08:52:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7fb877200640 1 -- 172.21.15.67:0/774905421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87004dff0 msgr2=0x7fb87004e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7fb877200640 1 --2- 172.21.15.67:0/774905421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87004dff0 0x7fb87004e7d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb87802d5c0 tx=0x7fb8780a7b40 comp rx=0 tx=0).stop 2024-03-09T08:52:37.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7f3c9aa00640 1 --2- 172.21.15.67:0/701724922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c940d8790 0x7f3c940df150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0097a0 tx=0x7f3c8c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fd635d3d18dbe server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7f04b1a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04ac14c9a0 0x7f04ac0a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7f04b1000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04ac0ac9c0 0x7f04ac1ef750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46222/0 (socket says 172.21.15.67:46222) 2024-03-09T08:52:37.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.484+0000 7f04abe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04ac0c1d70 0x7f04ac1ea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7f04b1a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04ac14c9a0 0x7f04ac0a8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51420/0 (socket says 172.21.15.67:51420) 2024-03-09T08:52:37.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7fb877200640 1 -- 172.21.15.67:0/774905421 shutdown_connections 2024-03-09T08:52:37.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7fb877200640 1 --2- 172.21.15.67:0/774905421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87004dff0 0x7fb87004e7d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7fb877200640 1 -- 172.21.15.67:0/774905421 >> 172.21.15.67:0/774905421 conn(0x7fb87004c660 msgr2=0x7fb87004ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7f04b1000640 1 -- 172.21.15.67:0/1155325632 learned_addr learned my addr 172.21.15.67:0/1155325632 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0c0eb0 con 0x7f04ac14c9a0 2024-03-09T08:52:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0aadf0 con 0x7f04ac0c1d70 2024-03-09T08:52:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7f58fce00640 1 -- 172.21.15.67:0/1044512908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f80c1d70 msgr2=0x7f58f80c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7fb877200640 1 -- 172.21.15.67:0/774905421 shutdown_connections 2024-03-09T08:52:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0c1030 con 0x7f04ac0ac9c0 2024-03-09T08:52:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7f58fce00640 1 --2- 172.21.15.67:0/1044512908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f80c1d70 0x7f58f80c07a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009870 tx=0x7f58e402b930 comp rx=0 tx=0).stop 2024-03-09T08:52:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7f3c99600640 1 -- 172.21.15.67:0/701724922 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c039070 con 0x7f3c940d8790 2024-03-09T08:52:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.486+0000 7f3c99600640 1 -- 172.21.15.67:0/701724922 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c0316f0 con 0x7f3c940d8790 2024-03-09T08:52:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.486+0000 7f3c99600640 1 -- 172.21.15.67:0/701724922 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c039070 con 0x7f3c940d8790 2024-03-09T08:52:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.487+0000 7fba49c00640 1 Processor -- start 2024-03-09T08:52:37.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.487+0000 7f3c9be00640 1 -- 172.21.15.67:0/701724922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c940d8790 msgr2=0x7f3c940df150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.487+0000 7fba49c00640 1 -- start start 2024-03-09T08:52:37.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.487+0000 7f3c9be00640 1 --2- 172.21.15.67:0/701724922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c940d8790 0x7f3c940df150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0097a0 tx=0x7f3c8c02b790 comp rx=0 tx=0).stop 2024-03-09T08:52:37.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.485+0000 7fb877200640 1 -- 172.21.15.67:0/774905421 wait complete. 2024-03-09T08:52:37.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.487+0000 7fb877200640 1 Processor -- start 2024-03-09T08:52:37.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.486+0000 7f1c1d600640 1 -- 172.21.15.67:0/2480118455 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1c1401e070 con 0x7f1c2008c7a0 2024-03-09T08:52:37.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.487+0000 7f1c1d600640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1c0006b4d0 0x7f1c0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.487+0000 7f1c1d600640 1 -- 172.21.15.67:0/2480118455 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f1c1408a6a0 con 0x7f1c2008c7a0 2024-03-09T08:52:37.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.486+0000 7f04b1a00640 1 -- 172.21.15.67:0/1155325632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04ac0ac9c0 msgr2=0x7f04ac1ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.486+0000 7f04b1a00640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04ac0ac9c0 0x7f04ac1ef750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.486+0000 7f04b1a00640 1 -- 172.21.15.67:0/1155325632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04ac0c1d70 msgr2=0x7f04ac1ea350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.486+0000 7f04b1a00640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04ac0c1d70 0x7f04ac1ea350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.486+0000 7f04b1a00640 1 -- 172.21.15.67:0/1155325632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0715c0 con 0x7f04ac14c9a0 2024-03-09T08:52:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.487+0000 7f04b1a00640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04ac14c9a0 0x7f04ac0a8ef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f04ac070470 tx=0x7f049c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.488+0000 7fb877200640 1 -- start start 2024-03-09T08:52:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.488+0000 7f04b1000640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04ac0ac9c0 0x7f04ac1ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.487+0000 7f58fce00640 1 -- 172.21.15.67:0/1044512908 shutdown_connections 2024-03-09T08:52:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.487+0000 7f58fce00640 1 --2- 172.21.15.67:0/1044512908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f80c1d70 0x7f58f80c07a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.487+0000 7f58fce00640 1 -- 172.21.15.67:0/1044512908 >> 172.21.15.67:0/1044512908 conn(0x7f58f80c7370 msgr2=0x7f58f80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.488+0000 7f04a8c00640 1 -- 172.21.15.67:0/1155325632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c042070 con 0x7f04ac14c9a0 2024-03-09T08:52:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.488+0000 7f1c1d600640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] conn(0x7f1c00075d70 0x7f1c00078190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.488+0000 7f04a8c00640 1 -- 172.21.15.67:0/1155325632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c0329c0 con 0x7f04ac14c9a0 2024-03-09T08:52:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.488+0000 7f04a8c00640 1 -- 172.21.15.67:0/1155325632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c037040 con 0x7f04ac14c9a0 2024-03-09T08:52:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.488+0000 7f1c1d600640 1 -- 172.21.15.67:0/2480118455 --> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c00078840 con 0x7f1c00075d70 2024-03-09T08:52:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.488+0000 7f1c1d600640 1 -- 172.21.15.67:0/2480118455 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c1408aa90 con 0x7f1c2008c7a0 2024-03-09T08:52:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.488+0000 7f04abe00640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04ac0c1d70 0x7f04ac1ea350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:37.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.488+0000 7f1c25800640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1c0006b4d0 0x7f1c0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.488+0000 7f58fce00640 1 -- 172.21.15.67:0/1044512908 shutdown_connections 2024-03-09T08:52:37.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.489+0000 7f1c24e00640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] conn(0x7f1c00075d70 0x7f1c00078190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.489+0000 7fba49c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba28004bf0 0x7fba4415b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.489+0000 7fba49c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba441f4070 0x7fba441524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.489+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ac0718a0 con 0x7f04ac14c9a0 2024-03-09T08:52:37.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.489+0000 7fba49c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba44158030 0x7fba441584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fb877200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87004dae0 0x7fb870171400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fba49c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4415d770 con 0x7fba44158030 2024-03-09T08:52:37.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fba49c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba440c6500 con 0x7fba441f4070 2024-03-09T08:52:37.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fba49c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4415d8f0 con 0x7fba28004bf0 2024-03-09T08:52:37.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7f1c24e00640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] conn(0x7f1c00075d70 0x7f1c00078190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fba3be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba441f4070 0x7fba441524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fb877200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb87004dff0 0x7fb870170a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fba42a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba28004bf0 0x7fba4415b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fba3be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba441f4070 0x7fba441524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46262/0 (socket says 172.21.15.67:46262) 2024-03-09T08:52:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fba42a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba28004bf0 0x7fba4415b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46232/0 (socket says 172.21.15.67:46232) 2024-03-09T08:52:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fba3be00640 1 -- 172.21.15.67:0/2291731014 learned_addr learned my addr 172.21.15.67:0/2291731014 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fb875e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87004dae0 0x7fb870171400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7fb875e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87004dae0 0x7fb870171400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46264/0 (socket says 172.21.15.67:46264) 2024-03-09T08:52:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7fb875e00640 1 -- 172.21.15.67:0/1705471870 learned_addr learned my addr 172.21.15.67:0/1705471870 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.490+0000 7fba43400640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba44158030 0x7fba441584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7fb877200640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8700d7ae0 0x7fb870030d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7fb875400640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb87004dff0 0x7fb870170a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7f3c9be00640 1 -- 172.21.15.67:0/701724922 shutdown_connections 2024-03-09T08:52:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7fba43400640 1 -- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba28004bf0 msgr2=0x7fba4415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7fba43400640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba28004bf0 0x7fba4415b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7fba43400640 1 -- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba441f4070 msgr2=0x7fba441524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7fb876800640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8700d7ae0 0x7fb870030d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.492+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87004e4b0 con 0x7fb87004dff0 2024-03-09T08:52:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7fba43400640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba441f4070 0x7fba441524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7fba43400640 1 -- 172.21.15.67:0/2291731014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba441f1050 con 0x7fba44158030 2024-03-09T08:52:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.492+0000 7fba3be00640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba441f4070 0x7fba441524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:52:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7f3c9be00640 1 --2- 172.21.15.67:0/701724922 >> v2:172.21.15.67:3300/0 conn(0x7f3c940064a0 0x7f3c94006910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.492+0000 7f3c9be00640 1 --2- 172.21.15.67:0/701724922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c940d8790 0x7f3c940df150 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.492+0000 7f3c9be00640 1 -- 172.21.15.67:0/701724922 >> 172.21.15.67:0/701724922 conn(0x7f3c9404fc00 msgr2=0x7f3c94050010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.492+0000 7f1c25800640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1c0006b4d0 0x7f1c0006d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c02b970 tx=0x7f1c0c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.492+0000 7fba43400640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba44158030 0x7fba441584b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba440c9160 tx=0x7fba3c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.492+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700db270 con 0x7fb87004dae0 2024-03-09T08:52:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.492+0000 7f3c9be00640 1 -- 172.21.15.67:0/701724922 shutdown_connections 2024-03-09T08:52:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.492+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700078c0 con 0x7fb8700d7ae0 2024-03-09T08:52:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.492+0000 7f1c1d600640 1 -- 172.21.15.67:0/2480118455 <== osd.1 v2:172.21.15.67:6815/1773797031 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1c00078840 con 0x7f1c00075d70 2024-03-09T08:52:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.492+0000 7f3c9be00640 1 -- 172.21.15.67:0/701724922 wait complete. 2024-03-09T08:52:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.493+0000 7fba3aa00640 1 -- 172.21.15.67:0/2291731014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00db20 con 0x7fba44158030 2024-03-09T08:52:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.493+0000 7fba3aa00640 1 -- 172.21.15.67:0/2291731014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3c018070 con 0x7fba44158030 2024-03-09T08:52:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.493+0000 7fba3aa00640 1 -- 172.21.15.67:0/2291731014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c003970 con 0x7fba44158030 2024-03-09T08:52:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.493+0000 7f04a8c00640 1 -- 172.21.15.67:0/1155325632 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f049c045070 con 0x7f04ac14c9a0 2024-03-09T08:52:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.494+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ac1efc90 con 0x7f04ac14c9a0 2024-03-09T08:52:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.494+0000 7fb875e00640 1 -- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8700d7ae0 msgr2=0x7fb870030d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.494+0000 7fb875e00640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8700d7ae0 0x7fb870030d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.494+0000 7fb875e00640 1 -- 172.21.15.67:0/1705471870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb87004dff0 msgr2=0x7fb870170a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.494+0000 7fb875e00640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb87004dff0 0x7fb870170a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.494+0000 7fb875e00640 1 -- 172.21.15.67:0/1705471870 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8700d9d20 con 0x7fb87004dae0 2024-03-09T08:52:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.494+0000 7fba49c00640 1 -- 172.21.15.67:0/2291731014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba441f0290 con 0x7fba44158030 2024-03-09T08:52:37.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.494+0000 7fba49c00640 1 -- 172.21.15.67:0/2291731014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba441f0850 con 0x7fba44158030 2024-03-09T08:52:37.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.494+0000 7fb875e00640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87004dae0 0x7fb870171400 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb86800c570 tx=0x7fb86800ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.495+0000 7fb86e000640 1 -- 172.21.15.67:0/1705471870 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868017070 con 0x7fb87004dae0 2024-03-09T08:52:37.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.495+0000 7fb86e000640 1 -- 172.21.15.67:0/1705471870 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8680074a0 con 0x7fb87004dae0 2024-03-09T08:52:37.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.495+0000 7fb86e000640 1 -- 172.21.15.67:0/1705471870 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868005a50 con 0x7fb87004dae0 2024-03-09T08:52:37.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.496+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8700da000 con 0x7fb87004dae0 2024-03-09T08:52:37.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.496+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8700d8270 con 0x7fb87004dae0 2024-03-09T08:52:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.496+0000 7f4b66a00640 1 Processor -- start 2024-03-09T08:52:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.495+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f04ac0c11b0 con 0x7f04ac14c9a0 2024-03-09T08:52:37.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.496+0000 7f3c9be00640 1 Processor -- start 2024-03-09T08:52:37.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.495+0000 7fba49c00640 1 -- 172.21.15.67:0/2291731014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fba1c001030 con 0x7fba44158030 2024-03-09T08:52:37.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.497+0000 7fba3aa00640 1 -- 172.21.15.67:0/2291731014 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fba3c003b10 con 0x7fba44158030 2024-03-09T08:52:37.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.498+0000 7f04a8c00640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f048c06b550 0x7f048c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.498+0000 7f3c9be00640 1 -- start start 2024-03-09T08:52:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb8700057e0 con 0x7fb87004dae0 2024-03-09T08:52:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7f04b1000640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f048c06b550 0x7f048c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7f04a8c00640 1 -- 172.21.15.67:0/1155325632 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f049c0b5410 con 0x7f04ac14c9a0 2024-03-09T08:52:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.497+0000 7f4b66a00640 1 -- start start 2024-03-09T08:52:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7f4b66a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b580d7ae0 0x7f4b580071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7f4b66a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b5804dae0 0x7f4b5804eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7f4b66a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5800f9d0 con 0x7f4b5804dff0 2024-03-09T08:52:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7f4b66a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b580da4b0 con 0x7f4b5804dae0 2024-03-09T08:52:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7f4b66a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580da790 con 0x7f4b580d7ae0 2024-03-09T08:52:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7f04a8c00640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] conn(0x7f048c075df0 0x7f048c078210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7fba3aa00640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba2006b550 0x7fba2006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7fba3aa00640 1 -- 172.21.15.67:0/2291731014 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fba3c08b020 con 0x7fba44158030 2024-03-09T08:52:37.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7fba3aa00640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] conn(0x7fba20075de0 0x7fba20078200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7fba3aa00640 1 -- 172.21.15.67:0/2291731014 --> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba3c01ee50 con 0x7fba20075de0 2024-03-09T08:52:37.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7fba3aa00640 1 -- 172.21.15.67:0/2291731014 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba3c08b500 con 0x7fba44158030 2024-03-09T08:52:37.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.499+0000 7f4b66000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b5804dae0 0x7f4b5804eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f4b66000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b5804dae0 0x7f4b5804eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51448/0 (socket says 172.21.15.67:51448) 2024-03-09T08:52:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f4b66000640 1 -- 172.21.15.67:0/3153079640 learned_addr learned my addr 172.21.15.67:0/3153079640 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f4b65600640 1 --2- 172.21.15.67:0/3153079640 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b580d7ae0 0x7f4b580071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940064a0 0x7f3c94174d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7fba42a00640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba2006b550 0x7fba2006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f4b66000640 1 -- 172.21.15.67:0/3153079640 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b580d7ae0 msgr2=0x7f4b580071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f4b66000640 1 --2- 172.21.15.67:0/3153079640 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b580d7ae0 0x7f4b580071b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f4b66000640 1 -- 172.21.15.67:0/3153079640 >> v1:172.21.15.67:6789/0 conn(0x7f4b5804dff0 legacy=0x7f4b5804e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f4b66000640 1 -- 172.21.15.67:0/3153079640 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b580daa70 con 0x7f4b5804dae0 2024-03-09T08:52:37.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7fba3be00640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] conn(0x7fba20075de0 0x7fba20078200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c94049fa0 0x7f3c940d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f3c9be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c940d8790 0x7f3c940338f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f3c9be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940508d0 con 0x7f3c940064a0 2024-03-09T08:52:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f3c9be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9404c130 con 0x7f3c940d8790 2024-03-09T08:52:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f3c9be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9400f4c0 con 0x7f3c94049fa0 2024-03-09T08:52:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.500+0000 7f4b65600640 1 --2- 172.21.15.67:0/3153079640 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b580d7ae0 0x7f4b580071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.501+0000 7f04abe00640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] conn(0x7f048c075df0 0x7f048c078210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.489+0000 7f58fce00640 1 -- 172.21.15.67:0/1044512908 wait complete. 2024-03-09T08:52:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.501+0000 7f3c9aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940064a0 0x7f3c94174d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.491+0000 7f58fce00640 1 Processor -- start 2024-03-09T08:52:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.501+0000 7f3c9aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940064a0 0x7f3c94174d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51464/0 (socket says 172.21.15.67:51464) 2024-03-09T08:52:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.501+0000 7f3c9aa00640 1 -- 172.21.15.67:0/3105292983 learned_addr learned my addr 172.21.15.67:0/3105292983 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.501+0000 7f4b66000640 1 --2- 172.21.15.67:0/3153079640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b5804dae0 0x7f4b5804eee0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b60069a40 tx=0x7f4b600a7da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc92aa2e0f7ed228 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.501+0000 7f3c9b400640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c940d8790 0x7f3c940338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.501+0000 7f3c9a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c94049fa0 0x7f3c940d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7fba42a00640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba2006b550 0x7fba2006da10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fba34036000 tx=0x7fba34030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7fba3be00640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] conn(0x7fba20075de0 0x7fba20078200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f4b5fe00640 1 -- 172.21.15.67:0/3153079640 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600b5070 con 0x7f4b5804dae0 2024-03-09T08:52:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f04a8c00640 1 -- 172.21.15.67:0/1155325632 --> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f049c034ec0 con 0x7f048c075df0 2024-03-09T08:52:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f3c9aa00640 1 -- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c94049fa0 msgr2=0x7f3c940d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f04a8c00640 1 -- 172.21.15.67:0/1155325632 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f049c0b57f0 con 0x7f04ac14c9a0 2024-03-09T08:52:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f3c9aa00640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c94049fa0 0x7f3c940d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f3c9aa00640 1 -- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c940d8790 msgr2=0x7f3c940338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f3c9aa00640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c940d8790 0x7f3c940338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f3c9aa00640 1 -- 172.21.15.67:0/3105292983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c940d1ef0 con 0x7f3c940064a0 2024-03-09T08:52:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f3c9b400640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c940d8790 0x7f3c940338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f4b5fe00640 1 -- 172.21.15.67:0/3153079640 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b600a8950 con 0x7f4b5804dae0 2024-03-09T08:52:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f4b5fe00640 1 -- 172.21.15.67:0/3153079640 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600a8dc0 con 0x7f4b5804dae0 2024-03-09T08:52:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.503+0000 7f4b66a00640 1 -- 172.21.15.67:0/3153079640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b5804dae0 msgr2=0x7f4b5804eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.503+0000 7f4b66a00640 1 --2- 172.21.15.67:0/3153079640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b5804dae0 0x7f4b5804eee0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b60069a40 tx=0x7f4b600a7da0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.503+0000 7f04b1000640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f048c06b550 0x7f048c06da10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f04a4007fd0 tx=0x7f04a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f3c9a000640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c94049fa0 0x7f3c940d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.503+0000 7f4b66a00640 1 -- 172.21.15.67:0/3153079640 shutdown_connections 2024-03-09T08:52:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.503+0000 7f4b66a00640 1 --2- 172.21.15.67:0/3153079640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b5804dae0 0x7f4b5804eee0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.503+0000 7f4b66a00640 1 --2- 172.21.15.67:0/3153079640 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b580d7ae0 0x7f4b580071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.503+0000 7f4b66a00640 1 -- 172.21.15.67:0/3153079640 >> 172.21.15.67:0/3153079640 conn(0x7f4b5804c660 msgr2=0x7f4b5804ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.503+0000 7f04abe00640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] conn(0x7f048c075df0 0x7f048c078210 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.502+0000 7f58fce00640 1 -- start start 2024-03-09T08:52:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.503+0000 7f3c9aa00640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940064a0 0x7f3c94174d20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000c00 tx=0x7f3c8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.503+0000 7f4b66a00640 1 -- 172.21.15.67:0/3153079640 shutdown_connections 2024-03-09T08:52:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.503+0000 7f4b66a00640 1 -- 172.21.15.67:0/3153079640 wait complete. 2024-03-09T08:52:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.505+0000 7f58fce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f80c0ce0 0x7f58f8149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.505+0000 7f58fce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f80c1d70 0x7f58f81f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.505+0000 7f58fce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58f814c9a0 0x7f58f81e0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.505+0000 7fba3aa00640 1 -- 172.21.15.67:0/2291731014 <== osd.5 v2:172.21.15.134:6800/4161897388 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fba3c01ee50 con 0x7fba20075de0 2024-03-09T08:52:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.505+0000 7f04a8c00640 1 -- 172.21.15.67:0/1155325632 <== osd.3 v2:172.21.15.67:6824/1424647284 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f049c034ec0 con 0x7f048c075df0 2024-03-09T08:52:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.505+0000 7f3c92a00640 1 -- 172.21.15.67:0/3105292983 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c043020 con 0x7f3c940064a0 2024-03-09T08:52:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.505+0000 7f3c9be00640 1 -- 172.21.15.67:0/3105292983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c940dda20 con 0x7f3c940064a0 2024-03-09T08:52:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.505+0000 7f3c92a00640 1 -- 172.21.15.67:0/3105292983 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c031aa0 con 0x7f3c940064a0 2024-03-09T08:52:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.506+0000 7f4b66a00640 1 Processor -- start 2024-03-09T08:52:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.505+0000 7f58f5400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f80c1d70 0x7f58f81f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.506+0000 7f58f6800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58f814c9a0 0x7f58f81e0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.506+0000 7f58f5400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f80c1d70 0x7f58f81f3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46280/0 (socket says 172.21.15.67:46280) 2024-03-09T08:52:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.506+0000 7f58f5400640 1 -- 172.21.15.67:0/2403882757 learned_addr learned my addr 172.21.15.67:0/2403882757 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.506+0000 7f58f5e00640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f80c0ce0 0x7f58f8149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.506+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80ad0d0 con 0x7f58f80c0ce0 2024-03-09T08:52:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.506+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80aadf0 con 0x7f58f80c1d70 2024-03-09T08:52:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.506+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80ad250 con 0x7f58f814c9a0 2024-03-09T08:52:37.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.506+0000 7f4b66a00640 1 -- start start 2024-03-09T08:52:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.507+0000 7f3c92a00640 1 -- 172.21.15.67:0/3105292983 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c039040 con 0x7f3c940064a0 2024-03-09T08:52:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.507+0000 7f3c9be00640 1 -- 172.21.15.67:0/3105292983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c940e0da0 con 0x7f3c940064a0 2024-03-09T08:52:37.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f58f5400640 1 -- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58f814c9a0 msgr2=0x7f58f81e0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f58f5400640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58f814c9a0 0x7f58f81e0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f58f5400640 1 -- 172.21.15.67:0/2403882757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f80c0ce0 msgr2=0x7f58f8149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f58f5400640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f58f80c0ce0 0x7f58f8149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f58f5400640 1 -- 172.21.15.67:0/2403882757 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f81f3d20 con 0x7f58f80c1d70 2024-03-09T08:52:37.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f4b66a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b5804dae0 0x7f4b58034ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f3c9be00640 1 -- 172.21.15.67:0/3105292983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3c940e0bf0 con 0x7f3c940064a0 2024-03-09T08:52:37.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f4b66a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b5804dff0 0x7f4b580d8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f4b66a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b580d7ae0 0x7f4b580472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f4b66a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5804e4b0 con 0x7f4b5804dae0 2024-03-09T08:52:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f4b66a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580db270 con 0x7f4b580d7ae0 2024-03-09T08:52:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7f4b66a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5800fdc0 con 0x7f4b5804dff0 2024-03-09T08:52:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.509+0000 7f58f5400640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f80c1d70 0x7f58f81f3270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f58e402b970 tx=0x7f58e40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.508+0000 7fb86e000640 1 -- 172.21.15.67:0/1705471870 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb86801a070 con 0x7fb87004dae0 2024-03-09T08:52:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.509+0000 7f4b65600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b5804dae0 0x7f4b58034ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.509+0000 7f4b65600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b5804dae0 0x7f4b58034ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51490/0 (socket says 172.21.15.67:51490) 2024-03-09T08:52:37.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.509+0000 7f4b65600640 1 -- 172.21.15.67:0/2146543315 learned_addr learned my addr 172.21.15.67:0/2146543315 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.509+0000 7f4b66000640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b580d7ae0 0x7f4b580472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.509+0000 7f4b64c00640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b5804dff0 0x7f4b580d8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.510+0000 7f4b66000640 1 -- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b5804dff0 msgr2=0x7f4b580d8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.510+0000 7f4b66000640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b5804dff0 0x7f4b580d8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.510+0000 7f4b66000640 1 -- 172.21.15.67:0/2146543315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b5804dae0 msgr2=0x7f4b58034ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.510+0000 7f4b66000640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b5804dae0 0x7f4b58034ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.510+0000 7f4b66000640 1 -- 172.21.15.67:0/2146543315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b580da290 con 0x7f4b580d7ae0 2024-03-09T08:52:37.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.509+0000 7f58ee000640 1 -- 172.21.15.67:0/2403882757 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4040070 con 0x7f58f80c1d70 2024-03-09T08:52:37.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.510+0000 7f3c92a00640 1 -- 172.21.15.67:0/3105292983 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3c8c046020 con 0x7f3c940064a0 2024-03-09T08:52:37.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.510+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f81f4000 con 0x7f58f80c1d70 2024-03-09T08:52:37.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.510+0000 7f58ee000640 1 -- 172.21.15.67:0/2403882757 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e403a420 con 0x7f58f80c1d70 2024-03-09T08:52:37.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.510+0000 7f58ee000640 1 -- 172.21.15.67:0/2403882757 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4040070 con 0x7f58f80c1d70 2024-03-09T08:52:37.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.510+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81e1fb0 con 0x7f58f80c1d70 2024-03-09T08:52:37.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.511+0000 7f3c92a00640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c7c06b4d0 0x7f3c7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.511+0000 7f4b66000640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b580d7ae0 0x7f4b580472c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4b6004f8e0 tx=0x7f4b6006a9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7fb86e000640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb85806b5a0 0x7fb85806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7fb86e000640 1 -- 172.21.15.67:0/1705471870 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb868089b10 con 0x7fb87004dae0 2024-03-09T08:52:37.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7f4b5d600640 1 -- 172.21.15.67:0/2146543315 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600bd070 con 0x7f4b580d7ae0 2024-03-09T08:52:37.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7f4b5d600640 1 -- 172.21.15.67:0/2146543315 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b600b0e00 con 0x7f4b580d7ae0 2024-03-09T08:52:37.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7f4b5d600640 1 -- 172.21.15.67:0/2146543315 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600b5040 con 0x7f4b580d7ae0 2024-03-09T08:52:37.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7fb86e000640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] conn(0x7fb858075e40 0x7fb858078260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7fb86e000640 1 -- 172.21.15.67:0/1705471870 --> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb858078910 con 0x7fb858075e40 2024-03-09T08:52:37.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7fb86e000640 1 -- 172.21.15.67:0/1705471870 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb868089f00 con 0x7fb87004dae0 2024-03-09T08:52:37.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.513+0000 7fb876800640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] conn(0x7fb858075e40 0x7fb858078260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.511+0000 7f3c92a00640 1 -- 172.21.15.67:0/3105292983 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f3c8c066060 con 0x7f3c940064a0 2024-03-09T08:52:37.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7f3c92a00640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] conn(0x7f3c7c075c60 0x7f3c7c078080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7f3c92a00640 1 -- 172.21.15.67:0/3105292983 --> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c8c0067f0 con 0x7f3c7c075c60 2024-03-09T08:52:37.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7f3c9b400640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] conn(0x7f3c7c075c60 0x7f3c7c078080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.514+0000 7f3c9b400640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] conn(0x7f3c7c075c60 0x7f3c7c078080 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.514+0000 7f3c9a000640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c7c06b4d0 0x7f3c7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.514+0000 7fb876800640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] conn(0x7fb858075e40 0x7fb858078260 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.515+0000 7fb875400640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb85806b5a0 0x7fb85806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.516+0000 7fb875400640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb85806b5a0 0x7fb85806da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb8700d7f00 tx=0x7fb8780ad000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.516+0000 7fb86e000640 1 -- 172.21.15.67:0/1705471870 <== osd.0 v2:172.21.15.67:6805/3109561126 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb858078910 con 0x7fb858075e40 2024-03-09T08:52:37.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7f4b66a00640 1 -- 172.21.15.67:0/2146543315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b580d99f0 con 0x7f4b580d7ae0 2024-03-09T08:52:37.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.512+0000 7f4b66a00640 1 -- 172.21.15.67:0/2146543315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b580d9fb0 con 0x7f4b580d7ae0 2024-03-09T08:52:37.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.516+0000 7f4b5d600640 1 -- 172.21.15.67:0/2146543315 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4b600c0070 con 0x7f4b580d7ae0 2024-03-09T08:52:37.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.518+0000 7f4b5d600640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b4006b4d0 0x7f4b4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.518+0000 7f4b5d600640 1 -- 172.21.15.67:0/2146543315 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f4b60128220 con 0x7f4b580d7ae0 2024-03-09T08:52:37.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.519+0000 7f4b66a00640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] conn(0x7f4b580d9300 0x7f4b58003190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.520+0000 7f4b66a00640 1 -- 172.21.15.67:0/2146543315 --> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b5800df40 con 0x7f4b580d9300 2024-03-09T08:52:37.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.520+0000 7f4b65600640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b4006b4d0 0x7f4b4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.520+0000 7f4b64c00640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] conn(0x7f4b580d9300 0x7f4b58003190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.515+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f58f81498a0 con 0x7f58f80c1d70 2024-03-09T08:52:37.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.515+0000 7f58ee000640 1 -- 172.21.15.67:0/2403882757 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f58e404b060 con 0x7f58f80c1d70 2024-03-09T08:52:37.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.515+0000 7f3c9a000640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c7c06b4d0 0x7f3c7c06d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c94034500 tx=0x7f3c88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.517+0000 7f3c92a00640 1 -- 172.21.15.67:0/3105292983 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c8c0b2050 con 0x7f3c940064a0 2024-03-09T08:52:37.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.518+0000 7f3c92a00640 1 -- 172.21.15.67:0/3105292983 <== osd.7 v2:172.21.15.134:6816/128562112 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3c8c0067f0 con 0x7f3c7c075c60 2024-03-09T08:52:37.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.521+0000 7f4b64c00640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] conn(0x7f4b580d9300 0x7f4b58003190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.521+0000 7f4b65600640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b4006b4d0 0x7f4b4006d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800eee0 tx=0x7f4b54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.519+0000 7f58ee000640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58d806b6d0 0x7f58d806db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.519+0000 7f58ee000640 1 -- 172.21.15.67:0/2403882757 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f58e40ac000 con 0x7f58f80c1d70 2024-03-09T08:52:37.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.520+0000 7f58f5e00640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58d806b6d0 0x7f58d806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.520+0000 7f58ee000640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] conn(0x7f58d8075ee0 0x7f58d8078300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.520+0000 7f58ee000640 1 -- 172.21.15.67:0/2403882757 --> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58d80789b0 con 0x7f58d8075ee0 2024-03-09T08:52:37.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.521+0000 7f58f6800640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] conn(0x7f58d8075ee0 0x7f58d8078300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.522+0000 7f58f6800640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] conn(0x7f58d8075ee0 0x7f58d8078300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.523+0000 7f4b5d600640 1 -- 172.21.15.67:0/2146543315 <== osd.2 v2:172.21.15.67:6800/74956722 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4b5800df40 con 0x7f4b580d9300 2024-03-09T08:52:37.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.523+0000 7f58ee000640 1 -- 172.21.15.67:0/2403882757 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58e4076650 con 0x7f58f80c1d70 2024-03-09T08:52:37.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.525+0000 7f58ee000640 1 -- 172.21.15.67:0/2403882757 <== osd.4 v2:172.21.15.134:6824/294033811 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f58d80789b0 con 0x7f58d8075ee0 2024-03-09T08:52:37.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.525+0000 7f58f5e00640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58d806b6d0 0x7f58d806db90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f58f80c05f0 tx=0x7f58e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.528+0000 7f1c28200640 1 -- 172.21.15.67:0/2480118455 --> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c2006e7f0 con 0x7f1c00075d70 2024-03-09T08:52:37.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.532+0000 7f1c1d600640 1 -- 172.21.15.67:0/2480118455 <== osd.1 v2:172.21.15.67:6815/1773797031 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1c2006e7f0 con 0x7f1c00075d70 2024-03-09T08:52:37.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.533+0000 7f1c12a00640 1 -- 172.21.15.67:0/2480118455 >> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] conn(0x7f1c00075d70 msgr2=0x7f1c00078190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.533+0000 7f1c12a00640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] conn(0x7f1c00075d70 0x7f1c00078190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.534+0000 7f1c12a00640 1 -- 172.21.15.67:0/2480118455 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1c0006b4d0 msgr2=0x7f1c0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.534+0000 7f1c12a00640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1c0006b4d0 0x7f1c0006d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c02b970 tx=0x7f1c0c03c040 comp rx=0 tx=0).stop 2024-03-09T08:52:37.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.534+0000 7f1c12a00640 1 -- 172.21.15.67:0/2480118455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c2008c7a0 msgr2=0x7f1c2020cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.534+0000 7f1c12a00640 1 --2- 172.21.15.67:0/2480118455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c2008c7a0 0x7f1c2020cf20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c14003640 tx=0x7f1c14003b00 comp rx=0 tx=0).stop 2024-03-09T08:52:37.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.534+0000 7f1c26200640 1 -- 172.21.15.67:0/2480118455 reap_dead start 2024-03-09T08:52:37.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.535+0000 7f1c12a00640 1 -- 172.21.15.67:0/2480118455 shutdown_connections 2024-03-09T08:52:37.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.535+0000 7f1c12a00640 1 -- 172.21.15.67:0/2480118455 >> 172.21.15.67:0/2480118455 conn(0x7f1c200c7370 msgr2=0x7f1c20162100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.535+0000 7fba49c00640 1 -- 172.21.15.67:0/2291731014 --> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fba1c002dd0 con 0x7fba20075de0 2024-03-09T08:52:37.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.536+0000 7f1c12a00640 1 -- 172.21.15.67:0/2480118455 shutdown_connections 2024-03-09T08:52:37.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.536+0000 7f1c12a00640 1 -- 172.21.15.67:0/2480118455 wait complete. 2024-03-09T08:52:37.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.536+0000 7fba3aa00640 1 -- 172.21.15.67:0/2291731014 <== osd.5 v2:172.21.15.134:6800/4161897388 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fba1c002dd0 con 0x7fba20075de0 2024-03-09T08:52:37.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.537+0000 7fba49c00640 1 -- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] conn(0x7fba20075de0 msgr2=0x7fba20078200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.537+0000 7fba49c00640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] conn(0x7fba20075de0 0x7fba20078200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.537+0000 7fba49c00640 1 -- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba2006b550 msgr2=0x7fba2006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.537+0000 7fba49c00640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba2006b550 0x7fba2006da10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fba34036000 tx=0x7fba34030000 comp rx=0 tx=0).stop 2024-03-09T08:52:37.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.538+0000 7fba49c00640 1 -- 172.21.15.67:0/2291731014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba44158030 msgr2=0x7fba441584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.538+0000 7fba49c00640 1 --2- 172.21.15.67:0/2291731014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba44158030 0x7fba441584b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba440c9160 tx=0x7fba3c00d520 comp rx=0 tx=0).stop 2024-03-09T08:52:37.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.538+0000 7fba43400640 1 -- 172.21.15.67:0/2291731014 reap_dead start 2024-03-09T08:52:37.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.538+0000 7fba49c00640 1 -- 172.21.15.67:0/2291731014 shutdown_connections 2024-03-09T08:52:37.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.538+0000 7fba49c00640 1 -- 172.21.15.67:0/2291731014 >> 172.21.15.67:0/2291731014 conn(0x7fba440cb670 msgr2=0x7fba4415db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.539+0000 7fba49c00640 1 -- 172.21.15.67:0/2291731014 shutdown_connections 2024-03-09T08:52:37.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.539+0000 7fba49c00640 1 -- 172.21.15.67:0/2291731014 wait complete. 2024-03-09T08:52:37.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.544+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 --> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f04ac075a00 con 0x7f048c075df0 2024-03-09T08:52:37.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.545+0000 7f04a8c00640 1 -- 172.21.15.67:0/1155325632 <== osd.3 v2:172.21.15.67:6824/1424647284 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f04ac075a00 con 0x7f048c075df0 2024-03-09T08:52:37.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.553+0000 7fb34f400640 1 Processor -- start 2024-03-09T08:52:37.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.553+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 >> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] conn(0x7f048c075df0 msgr2=0x7f048c078210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.553+0000 7f04b3a00640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] conn(0x7f048c075df0 0x7f048c078210 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.553+0000 7fb34f400640 1 -- start start 2024-03-09T08:52:37.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f048c06b550 msgr2=0x7f048c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7f04b3a00640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f048c06b550 0x7f048c06da10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f04a4007fd0 tx=0x7f04a4009040 comp rx=0 tx=0).stop 2024-03-09T08:52:37.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 --> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb87004b940 con 0x7fb858075e40 2024-03-09T08:52:37.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04ac14c9a0 msgr2=0x7f04ac0a8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7f04b3a00640 1 --2- 172.21.15.67:0/1155325632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04ac14c9a0 0x7f04ac0a8ef0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f04ac070470 tx=0x7f049c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7f04b1a00640 1 -- 172.21.15.67:0/1155325632 reap_dead start 2024-03-09T08:52:37.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 shutdown_connections 2024-03-09T08:52:37.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 >> 172.21.15.67:0/1155325632 conn(0x7f04ac0c7370 msgr2=0x7f04ac1539b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7fb34f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb3400d6a90 0x7fb3400d6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7fb34f400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb34000b100 0x7fb3400e1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7fb34f400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb340046900 con 0x7fb3400d3c70 2024-03-09T08:52:37.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7fb34f400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb340047690 con 0x7fb3400d6a90 2024-03-09T08:52:37.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7fb34f400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3400d6600 con 0x7fb34000b100 2024-03-09T08:52:37.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.557+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 shutdown_connections 2024-03-09T08:52:37.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.556+0000 7fb34ca00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb3400d3c70 0x7fb34004aca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:60952/0 (socket says 172.21.15.67:60952) 2024-03-09T08:52:37.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.557+0000 7fb34ca00640 1 -- 172.21.15.67:0/2868265911 learned_addr learned my addr 172.21.15.67:0/2868265911 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:37.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.557+0000 7f04b3a00640 1 -- 172.21.15.67:0/1155325632 wait complete. 2024-03-09T08:52:37.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.557+0000 7fb347e00640 1 --2- 172.21.15.67:0/2868265911 >> v2:172.21.15.67:3300/0 conn(0x7fb3400d6a90 0x7fb3400d6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.557+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 --> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f58f80c3c60 con 0x7f58d8075ee0 2024-03-09T08:52:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.557+0000 7fb347400640 1 -- 172.21.15.67:0/2868265911 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158936502 0 0) 0x7fb340046900 con 0x7fb3400d3c70 2024-03-09T08:52:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.557+0000 7fb347400640 1 -- 172.21.15.67:0/2868265911 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb330003520 con 0x7fb3400d3c70 2024-03-09T08:52:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.558+0000 7fb347e00640 1 -- 172.21.15.67:0/2868265911 >> v2:172.21.15.134:3300/0 conn(0x7fb34000b100 msgr2=0x7fb3400e1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.558+0000 7fb347e00640 1 --2- 172.21.15.67:0/2868265911 >> v2:172.21.15.134:3300/0 conn(0x7fb34000b100 0x7fb3400e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.558+0000 7fb347e00640 1 -- 172.21.15.67:0/2868265911 >> v1:172.21.15.134:6789/0 conn(0x7fb3400d3c70 legacy=0x7fb34004aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.558+0000 7fb347e00640 1 -- 172.21.15.67:0/2868265911 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb340046be0 con 0x7fb3400d6a90 2024-03-09T08:52:37.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.557+0000 7fb86e000640 1 -- 172.21.15.67:0/1705471870 <== osd.0 v2:172.21.15.67:6805/3109561126 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb87004b940 con 0x7fb858075e40 2024-03-09T08:52:37.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.559+0000 7f58ee000640 1 -- 172.21.15.67:0/2403882757 <== osd.4 v2:172.21.15.134:6824/294033811 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f58f80c3c60 con 0x7f58d8075ee0 2024-03-09T08:52:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.560+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] conn(0x7f58d8075ee0 msgr2=0x7f58d8078300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.560+0000 7f58fce00640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] conn(0x7f58d8075ee0 0x7f58d8078300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.560+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58d806b6d0 msgr2=0x7f58d806db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.560+0000 7f58fce00640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58d806b6d0 0x7f58d806db90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f58f80c05f0 tx=0x7f58e8009040 comp rx=0 tx=0).stop 2024-03-09T08:52:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.560+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f80c1d70 msgr2=0x7f58f81f3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.560+0000 7f58fce00640 1 --2- 172.21.15.67:0/2403882757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58f80c1d70 0x7f58f81f3270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f58e402b970 tx=0x7f58e40046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.560+0000 7f58f6800640 1 -- 172.21.15.67:0/2403882757 reap_dead start 2024-03-09T08:52:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.560+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 shutdown_connections 2024-03-09T08:52:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.560+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 >> 172.21.15.67:0/2403882757 conn(0x7f58f80c7370 msgr2=0x7f58f80c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.591 INFO:teuthology.orchestra.run.smithi067.stdout:55834574858 2024-03-09T08:52:37.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-09T08:52:37.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.561+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 shutdown_connections 2024-03-09T08:52:37.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.561+0000 7f58fce00640 1 -- 172.21.15.67:0/2403882757 wait complete. 2024-03-09T08:52:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.561+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 >> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] conn(0x7fb858075e40 msgr2=0x7fb858078260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.561+0000 7fb877200640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] conn(0x7fb858075e40 0x7fb858078260 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.561+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb85806b5a0 msgr2=0x7fb85806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.561+0000 7fb877200640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb85806b5a0 0x7fb85806da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb8700d7f00 tx=0x7fb8780ad000 comp rx=0 tx=0).stop 2024-03-09T08:52:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.561+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87004dae0 msgr2=0x7fb870171400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.561+0000 7fb877200640 1 --2- 172.21.15.67:0/1705471870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb87004dae0 0x7fb870171400 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb86800c570 tx=0x7fb86800ca30 comp rx=0 tx=0).stop 2024-03-09T08:52:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.562+0000 7fb876800640 1 -- 172.21.15.67:0/1705471870 reap_dead start 2024-03-09T08:52:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.562+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 shutdown_connections 2024-03-09T08:52:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.562+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 >> 172.21.15.67:0/1705471870 conn(0x7fb87004c660 msgr2=0x7fb87004ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.561+0000 7fb347e00640 1 --2- 172.21.15.67:0/2868265911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3400d6a90 0x7fb3400d6f10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb34802d5c0 tx=0x7fb3480a7cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68670319e45ee6da server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.563+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 shutdown_connections 2024-03-09T08:52:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.563+0000 7f3c9be00640 1 -- 172.21.15.67:0/3105292983 --> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3c9400fad0 con 0x7f3c7c075c60 2024-03-09T08:52:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.563+0000 7fb347400640 1 -- 172.21.15.67:0/2868265911 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3480b5070 con 0x7fb3400d6a90 2024-03-09T08:52:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.563+0000 7fb347400640 1 -- 172.21.15.67:0/2868265911 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3480a8870 con 0x7fb3400d6a90 2024-03-09T08:52:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.563+0000 7fb347400640 1 -- 172.21.15.67:0/2868265911 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3480a8d00 con 0x7fb3400d6a90 2024-03-09T08:52:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.563+0000 7fb877200640 1 -- 172.21.15.67:0/1705471870 wait complete. 2024-03-09T08:52:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.563+0000 7f4b66a00640 1 -- 172.21.15.67:0/2146543315 --> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b58035810 con 0x7f4b580d9300 2024-03-09T08:52:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.564+0000 7fb34f400640 1 -- 172.21.15.67:0/2868265911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3400d6a90 msgr2=0x7fb3400d6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.564+0000 7fb34f400640 1 --2- 172.21.15.67:0/2868265911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3400d6a90 0x7fb3400d6f10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb34802d5c0 tx=0x7fb3480a7cc0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.564+0000 7f4b5d600640 1 -- 172.21.15.67:0/2146543315 <== osd.2 v2:172.21.15.67:6800/74956722 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4b58035810 con 0x7f4b580d9300 2024-03-09T08:52:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.564+0000 7fb34f400640 1 -- 172.21.15.67:0/2868265911 shutdown_connections 2024-03-09T08:52:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.564+0000 7fb34f400640 1 --2- 172.21.15.67:0/2868265911 >> v2:172.21.15.134:3300/0 conn(0x7fb34000b100 0x7fb3400e1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.564+0000 7fb34f400640 1 --2- 172.21.15.67:0/2868265911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3400d6a90 0x7fb3400d6f10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.564+0000 7fb34f400640 1 -- 172.21.15.67:0/2868265911 >> 172.21.15.67:0/2868265911 conn(0x7fb340005ec0 msgr2=0x7fb3400062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.564+0000 7f3c92a00640 1 -- 172.21.15.67:0/3105292983 <== osd.7 v2:172.21.15.134:6816/128562112 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3c9400fad0 con 0x7f3c7c075c60 2024-03-09T08:52:37.598 INFO:teuthology.orchestra.run.smithi067.stdout:55834574857 2024-03-09T08:52:37.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-09T08:52:37.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f4b66a00640 1 -- 172.21.15.67:0/2146543315 >> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] conn(0x7f4b580d9300 msgr2=0x7f4b58003190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f4b66a00640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] conn(0x7f4b580d9300 0x7f4b58003190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7fb34f400640 1 -- 172.21.15.67:0/2868265911 shutdown_connections 2024-03-09T08:52:37.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f3c9be00640 1 -- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] conn(0x7f3c7c075c60 msgr2=0x7f3c7c078080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f3c9be00640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] conn(0x7f3c7c075c60 0x7f3c7c078080 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f3c9be00640 1 -- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c7c06b4d0 msgr2=0x7f3c7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f3c9be00640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c7c06b4d0 0x7f3c7c06d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c94034500 tx=0x7f3c88009040 comp rx=0 tx=0).stop 2024-03-09T08:52:37.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f3c9be00640 1 -- 172.21.15.67:0/3105292983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940064a0 msgr2=0x7f3c94174d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f3c9be00640 1 --2- 172.21.15.67:0/3105292983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c940064a0 0x7f3c94174d20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000c00 tx=0x7f3c8c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f3c9b400640 1 -- 172.21.15.67:0/3105292983 reap_dead start 2024-03-09T08:52:37.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f4b66a00640 1 -- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b4006b4d0 msgr2=0x7f4b4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f4b66a00640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b4006b4d0 0x7f4b4006d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800eee0 tx=0x7f4b54008040 comp rx=0 tx=0).stop 2024-03-09T08:52:37.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f4b66a00640 1 -- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b580d7ae0 msgr2=0x7f4b580472c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f4b66a00640 1 --2- 172.21.15.67:0/2146543315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b580d7ae0 0x7f4b580472c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4b6004f8e0 tx=0x7f4b6006a9c0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7f4b66000640 1 -- 172.21.15.67:0/2146543315 reap_dead start 2024-03-09T08:52:37.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.566+0000 7f4b66a00640 1 -- 172.21.15.67:0/2146543315 shutdown_connections 2024-03-09T08:52:37.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.566+0000 7f4b66a00640 1 -- 172.21.15.67:0/2146543315 >> 172.21.15.67:0/2146543315 conn(0x7f4b5804c660 msgr2=0x7f4b5804aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.566+0000 7f3c9be00640 1 -- 172.21.15.67:0/3105292983 shutdown_connections 2024-03-09T08:52:37.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.566+0000 7f3c9be00640 1 -- 172.21.15.67:0/3105292983 >> 172.21.15.67:0/3105292983 conn(0x7f3c9404fc00 msgr2=0x7f3c940d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.566+0000 7f4b66a00640 1 -- 172.21.15.67:0/2146543315 shutdown_connections 2024-03-09T08:52:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.566+0000 7f4b66a00640 1 -- 172.21.15.67:0/2146543315 wait complete. 2024-03-09T08:52:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.565+0000 7fb34f400640 1 -- 172.21.15.67:0/2868265911 wait complete. 2024-03-09T08:52:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.567+0000 7f3c9be00640 1 -- 172.21.15.67:0/3105292983 shutdown_connections 2024-03-09T08:52:37.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.567+0000 7f3c9be00640 1 -- 172.21.15.67:0/3105292983 wait complete. 2024-03-09T08:52:37.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.568+0000 7fb34f400640 1 Processor -- start 2024-03-09T08:52:37.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.568+0000 7fb34f400640 1 -- start start 2024-03-09T08:52:37.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.569+0000 7fb34f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb34000b100 0x7fb3400d2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.569+0000 7fb34f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3400d3c70 0x7fb3400d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.569+0000 7fb34f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3400428d0 0x7fb340044420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.569+0000 7fb34f400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb340046390 con 0x7fb34000b100 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.569+0000 7fb34f400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb340046210 con 0x7fb3400d3c70 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.569+0000 7fb34f400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb340046ec0 con 0x7fb3400428d0 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.569+0000 7fb34ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb34000b100 0x7fb3400d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.569+0000 7fb34ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb34000b100 0x7fb3400d2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51516/0 (socket says 172.21.15.67:51516) 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.569+0000 7fb34ca00640 1 -- 172.21.15.67:0/3340130642 learned_addr learned my addr 172.21.15.67:0/3340130642 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.570+0000 7fb34d400640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3400428d0 0x7fb340044420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.570+0000 7fb34d400640 1 -- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3400d3c70 msgr2=0x7fb3400d2ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.570+0000 7fb34d400640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3400d3c70 0x7fb3400d2ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.571+0000 7fb34d400640 1 -- 172.21.15.67:0/3340130642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb34000b100 msgr2=0x7fb3400d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.571+0000 7fb34d400640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb34000b100 0x7fb3400d2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.571+0000 7fb34d400640 1 -- 172.21.15.67:0/3340130642 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb340175330 con 0x7fb3400428d0 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.571+0000 7fb34d400640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3400428d0 0x7fb340044420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb33400b410 tx=0x7fb33400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.572+0000 7fb344c00640 1 -- 172.21.15.67:0/3340130642 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334015020 con 0x7fb3400428d0 2024-03-09T08:52:37.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.572+0000 7fb34f400640 1 -- 172.21.15.67:0/3340130642 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34002ebf0 con 0x7fb3400428d0 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.572+0000 7fb344c00640 1 -- 172.21.15.67:0/3340130642 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb334011830 con 0x7fb3400428d0 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.572+0000 7fb344c00640 1 -- 172.21.15.67:0/3340130642 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334011c00 con 0x7fb3400428d0 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.572+0000 7fb34f400640 1 -- 172.21.15.67:0/3340130642 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34002f1b0 con 0x7fb3400428d0 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.574+0000 7fb33a000640 1 -- 172.21.15.67:0/3340130642 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb3401750e0 con 0x7fb3400428d0 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.576+0000 7fb344c00640 1 -- 172.21.15.67:0/3340130642 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb334018020 con 0x7fb3400428d0 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.579+0000 7fb344c00640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb32806b4d0 0x7fb32806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.579+0000 7fb344c00640 1 -- 172.21.15.67:0/3340130642 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb33408ace0 con 0x7fb3400428d0 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.580+0000 7fb344c00640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] conn(0x7fb328075d70 0x7fb328078190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.580+0000 7fb344c00640 1 -- 172.21.15.67:0/3340130642 --> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb33401fe80 con 0x7fb328075d70 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.580+0000 7fb344c00640 1 -- 172.21.15.67:0/3340130642 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb33408b0d0 con 0x7fb3400428d0 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.580+0000 7fb34ca00640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb32806b4d0 0x7fb32806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.581+0000 7fb347e00640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] conn(0x7fb328075d70 0x7fb328078190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.582+0000 7fb347e00640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] conn(0x7fb328075d70 0x7fb328078190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.582+0000 7fb34ca00640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb32806b4d0 0x7fb32806d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb33c001220 tx=0x7fb33c0011b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.609 INFO:teuthology.orchestra.run.smithi067.stdout:55834574859 2024-03-09T08:52:37.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-09T08:52:37.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.584+0000 7fb344c00640 1 -- 172.21.15.67:0/3340130642 <== osd.6 v2:172.21.15.134:6808/3579212498 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb33401fe80 con 0x7fb328075d70 2024-03-09T08:52:37.611 INFO:teuthology.orchestra.run.smithi067.stdout:55834574858 2024-03-09T08:52:37.611 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-09T08:52:37.612 INFO:teuthology.orchestra.run.smithi067.stdout:55834574858 2024-03-09T08:52:37.612 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-09T08:52:37.613 INFO:teuthology.orchestra.run.smithi067.stdout:55834574857 2024-03-09T08:52:37.613 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-09T08:52:37.614 INFO:teuthology.orchestra.run.smithi067.stdout:55834574859 2024-03-09T08:52:37.614 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-09T08:52:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.605+0000 7fb33a000640 1 -- 172.21.15.67:0/3340130642 --> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb340175610 con 0x7fb328075d70 2024-03-09T08:52:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.606+0000 7fb344c00640 1 -- 172.21.15.67:0/3340130642 <== osd.6 v2:172.21.15.134:6808/3579212498 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb340175610 con 0x7fb328075d70 2024-03-09T08:52:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.606+0000 7fb33a000640 1 -- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] conn(0x7fb328075d70 msgr2=0x7fb328078190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.606+0000 7fb33a000640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] conn(0x7fb328075d70 0x7fb328078190 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.606+0000 7fb33a000640 1 -- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb32806b4d0 msgr2=0x7fb32806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.606+0000 7fb33a000640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb32806b4d0 0x7fb32806d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb33c001220 tx=0x7fb33c0011b0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.607+0000 7fb33a000640 1 -- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3400428d0 msgr2=0x7fb340044420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.607+0000 7fb33a000640 1 --2- 172.21.15.67:0/3340130642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3400428d0 0x7fb340044420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb33400b410 tx=0x7fb33400b8d0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.607+0000 7fb34d400640 1 -- 172.21.15.67:0/3340130642 reap_dead start 2024-03-09T08:52:37.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.607+0000 7fb33a000640 1 -- 172.21.15.67:0/3340130642 shutdown_connections 2024-03-09T08:52:37.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.607+0000 7fb33a000640 1 -- 172.21.15.67:0/3340130642 >> 172.21.15.67:0/3340130642 conn(0x7fb340005ec0 msgr2=0x7fb3400d55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.607+0000 7fb33a000640 1 -- 172.21.15.67:0/3340130642 shutdown_connections 2024-03-09T08:52:37.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.607+0000 7fb33a000640 1 -- 172.21.15.67:0/3340130642 wait complete. 2024-03-09T08:52:37.622 INFO:teuthology.orchestra.run.smithi067.stdout:55834574859 2024-03-09T08:52:37.622 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-09T08:52:37.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.931+0000 7f889c600640 1 Processor -- start 2024-03-09T08:52:37.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.931+0000 7f889c600640 1 -- start start 2024-03-09T08:52:37.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.933+0000 7f889c600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f888c0db800 0x7f888c0dbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.933+0000 7f889c600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f888c0e2c80 0x7f888c0e5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.933+0000 7f889c600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f888c007080 con 0x7f888c0d7ad0 2024-03-09T08:52:37.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.933+0000 7f889c600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f888c007340 con 0x7f888c0e2c80 2024-03-09T08:52:37.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.933+0000 7f889c600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888c0da420 con 0x7f888c0db800 2024-03-09T08:52:37.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.934+0000 7f8899c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f888c0d7ad0 0x7f888c046150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54492/0 (socket says 172.21.15.67:54492) 2024-03-09T08:52:37.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.934+0000 7f8899c00640 1 -- 172.21.15.67:0/2922249822 learned_addr learned my addr 172.21.15.67:0/2922249822 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:37.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.935+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606691291 0 0) 0x7f888c007080 con 0x7f888c0d7ad0 2024-03-09T08:52:37.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.935+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f887c003520 con 0x7f888c0d7ad0 2024-03-09T08:52:37.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.935+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 212685857 0 0) 0x7f887c003520 con 0x7f888c0d7ad0 2024-03-09T08:52:37.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.935+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f888c0db800 msgr2=0x7f888c0dbc60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T08:52:37.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.935+0000 7f8893e00640 1 --2- 172.21.15.67:0/2922249822 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f888c0db800 0x7f888c0dbc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.935+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 >> v2:172.21.15.134:3300/0 conn(0x7f888c0e2c80 msgr2=0x7f888c0e5070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T08:52:37.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.935+0000 7f8893e00640 1 --2- 172.21.15.67:0/2922249822 >> v2:172.21.15.134:3300/0 conn(0x7f888c0e2c80 0x7f888c0e5070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.935+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c0da700 con 0x7f888c0d7ad0 2024-03-09T08:52:37.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.935+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f8880001670 con 0x7f888c0d7ad0 2024-03-09T08:52:37.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.936+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 >> v1:172.21.15.67:6789/0 conn(0x7f888c0d7ad0 legacy=0x7f888c046150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.936+0000 7f8893e00640 1 --2- 172.21.15.67:0/2922249822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f887c0039b0 0x7f887c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.936+0000 7f8893e00640 1 --2- 172.21.15.67:0/2922249822 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f887c004800 0x7f887c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.936+0000 7f8893e00640 1 --2- 172.21.15.67:0/2922249822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 0x7f887c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.936+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c007340 con 0x7f887c0052e0 2024-03-09T08:52:37.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.936+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c0da420 con 0x7f887c0039b0 2024-03-09T08:52:37.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.936+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c00c240 con 0x7f887c004800 2024-03-09T08:52:37.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.937+0000 7f66f7400640 1 Processor -- start 2024-03-09T08:52:37.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.937+0000 7f889a600640 1 --2- 172.21.15.67:0/2922249822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 0x7f887c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.938+0000 7f889a600640 1 -- 172.21.15.67:0/2922249822 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f887c004800 msgr2=0x7f887c004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T08:52:37.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.938+0000 7f889a600640 1 --2- 172.21.15.67:0/2922249822 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f887c004800 0x7f887c004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.938+0000 7f889a600640 1 -- 172.21.15.67:0/2922249822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f887c0039b0 msgr2=0x7f887c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.938+0000 7f889a600640 1 --2- 172.21.15.67:0/2922249822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f887c0039b0 0x7f887c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.938+0000 7f889a600640 1 -- 172.21.15.67:0/2922249822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c0da700 con 0x7f887c0052e0 2024-03-09T08:52:37.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.939+0000 7f901f200640 1 Processor -- start 2024-03-09T08:52:37.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.939+0000 7f889a600640 1 --2- 172.21.15.67:0/2922249822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 0x7f887c00b9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8894064b50 tx=0x7f887c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a6cbd6a686ac82 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.938+0000 7f66f7400640 1 -- start start 2024-03-09T08:52:37.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.939+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894062ab0 con 0x7f887c0052e0 2024-03-09T08:52:37.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.940+0000 7f8587800640 1 Processor -- start 2024-03-09T08:52:37.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.940+0000 7f8587800640 1 -- start start 2024-03-09T08:52:37.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.940+0000 7f901f200640 1 -- start start 2024-03-09T08:52:37.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.942+0000 7f8587800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f85800c1d70 0x7f858015e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.942+0000 7f8587800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85800aad50 0x7f85800ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.940+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88940a95c0 con 0x7f887c0052e0 2024-03-09T08:52:37.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.940+0000 7f889c600640 1 -- 172.21.15.67:0/2922249822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 msgr2=0x7f887c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.940+0000 7f889c600640 1 --2- 172.21.15.67:0/2922249822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 0x7f887c00b9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8894064b50 tx=0x7f887c005830 comp rx=0 tx=0).stop 2024-03-09T08:52:37.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.940+0000 7f8893e00640 1 -- 172.21.15.67:0/2922249822 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88940a99b0 con 0x7f887c0052e0 2024-03-09T08:52:37.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.940+0000 7f889a600640 1 -- 172.21.15.67:0/2922249822 reap_dead start 2024-03-09T08:52:37.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.941+0000 7f889c600640 1 -- 172.21.15.67:0/2922249822 shutdown_connections 2024-03-09T08:52:37.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.941+0000 7f889c600640 1 -- 172.21.15.67:0/2922249822 >> 172.21.15.67:0/2922249822 conn(0x7f888c005e90 msgr2=0x7f888c0062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.942+0000 7f889c600640 1 -- 172.21.15.67:0/2922249822 shutdown_connections 2024-03-09T08:52:37.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.942+0000 7f66f7400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f66f014f4b0 0x7f66f0155430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.942+0000 7fe523000640 1 Processor -- start 2024-03-09T08:52:37.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.942+0000 7fe523000640 1 -- start start 2024-03-09T08:52:37.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.943+0000 7f889c600640 1 -- 172.21.15.67:0/2922249822 wait complete. 2024-03-09T08:52:37.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.943+0000 7f66f7400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66f00c32b0 0x7f66f0152570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.943+0000 7f66f7400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f00c2ec0 con 0x7f66f00aa420 2024-03-09T08:52:37.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.943+0000 7f66f7400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f0155970 con 0x7f66f014f4b0 2024-03-09T08:52:37.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.943+0000 7f66f7400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f014f890 con 0x7f66f00c32b0 2024-03-09T08:52:37.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.943+0000 7f8585800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85800aad50 0x7f85800ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.943+0000 7f8585800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85800aad50 0x7f85800ab1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51534/0 (socket says 172.21.15.67:51534) 2024-03-09T08:52:37.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.943+0000 7f857fe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f85800c1d70 0x7f858015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.943+0000 7f8585800640 1 -- 172.21.15.67:0/2262149482 learned_addr learned my addr 172.21.15.67:0/2262149482 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7f8587800640 1 -- 172.21.15.67:0/2262149482 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858006d970 con 0x7f8580157820 2024-03-09T08:52:37.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7f66efe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f66f00aa420 0x7f66f00abca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54504/0 (socket says 172.21.15.67:54504) 2024-03-09T08:52:37.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7f8587800640 1 -- 172.21.15.67:0/2262149482 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858006df30 con 0x7f85800aad50 2024-03-09T08:52:37.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7f8587800640 1 -- 172.21.15.67:0/2262149482 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858006dc50 con 0x7f85800c1d70 2024-03-09T08:52:37.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7f66efe00640 1 -- 172.21.15.67:0/3061979405 learned_addr learned my addr 172.21.15.67:0/3061979405 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:37.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7f901f200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f90200c1410 0x7f90200c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7f901f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90200c1c30 con 0x7f9020152730 2024-03-09T08:52:37.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7f901f200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90200c1f10 con 0x7f90200c47b0 2024-03-09T08:52:37.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7f901f200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90201538b0 con 0x7f90200c1410 2024-03-09T08:52:37.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7f901e800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f90200c1410 0x7f90200c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7f901e800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f90200c1410 0x7f90200c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46348/0 (socket says 172.21.15.67:46348) 2024-03-09T08:52:37.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7f901e800640 1 -- 172.21.15.67:0/284921922 learned_addr learned my addr 172.21.15.67:0/284921922 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7fe523000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe51c153fc0 0x7fe51c0c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7fe523000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c15aa40 0x7fe51c0c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.944+0000 7fe523000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe51c0c3810 0x7fe51c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7fe523000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51c06fec0 con 0x7fe51c0c3810 2024-03-09T08:52:37.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7fe523000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51c15af20 con 0x7fe51c153fc0 2024-03-09T08:52:37.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7fe523000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0c3f10 con 0x7fe51c15aa40 2024-03-09T08:52:37.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7fe521200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c15aa40 0x7fe51c0c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7fe521200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c15aa40 0x7fe51c0c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46324/0 (socket says 172.21.15.67:46324) 2024-03-09T08:52:37.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7fe521200640 1 -- 172.21.15.67:0/426940376 learned_addr learned my addr 172.21.15.67:0/426940376 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7f889c600640 1 Processor -- start 2024-03-09T08:52:37.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f889c600640 1 -- start start 2024-03-09T08:52:37.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f857fe00640 1 -- 172.21.15.67:0/2262149482 >> v2:172.21.15.67:3300/0 conn(0x7f85800aad50 msgr2=0x7f85800ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f857fe00640 1 --2- 172.21.15.67:0/2262149482 >> v2:172.21.15.67:3300/0 conn(0x7f85800aad50 0x7f85800ab1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f857f400640 1 -- 172.21.15.67:0/2262149482 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319482174 0 0) 0x7f858006d970 con 0x7f8580157820 2024-03-09T08:52:37.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f857fe00640 1 -- 172.21.15.67:0/2262149482 >> v1:172.21.15.134:6789/0 conn(0x7f8580157820 legacy=0x7f85800ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f8585800640 1 --2- 172.21.15.67:0/2262149482 >> v2:172.21.15.67:3300/0 conn(0x7f85800aad50 0x7f85800ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:37.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7f66f5400640 1 --2- 172.21.15.67:0/3061979405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66f00c32b0 0x7f66f0152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591673228 0 0) 0x7f66f00c2ec0 con 0x7f66f00aa420 2024-03-09T08:52:37.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.947+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66d8003db0 con 0x7f66f00aa420 2024-03-09T08:52:37.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.945+0000 7fe521c00640 1 --2- 172.21.15.67:0/426940376 >> v2:172.21.15.134:3300/0 conn(0x7fe51c153fc0 0x7fe51c0c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7fe522600640 1 --2- 172.21.15.67:0/426940376 >> v2:172.21.15.67:3300/0 conn(0x7fe51c0c3810 0x7fe51c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.947+0000 7fe522600640 1 -- 172.21.15.67:0/426940376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c15aa40 msgr2=0x7fe51c0c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.947+0000 7fe522600640 1 --2- 172.21.15.67:0/426940376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c15aa40 0x7fe51c0c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.947+0000 7fe522600640 1 -- 172.21.15.67:0/426940376 >> v2:172.21.15.134:3300/0 conn(0x7fe51c153fc0 msgr2=0x7fe51c0c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.947+0000 7fe522600640 1 --2- 172.21.15.67:0/426940376 >> v2:172.21.15.134:3300/0 conn(0x7fe51c153fc0 0x7fe51c0c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.947+0000 7fe522600640 1 -- 172.21.15.67:0/426940376 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c0c41f0 con 0x7fe51c0c3810 2024-03-09T08:52:37.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.947+0000 7f857fe00640 1 -- 172.21.15.67:0/2262149482 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85800aa550 con 0x7f85800c1d70 2024-03-09T08:52:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.947+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2713815229 0 0) 0x7f66d8003db0 con 0x7f66f00aa420 2024-03-09T08:52:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.947+0000 7f889c600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 0x7f888c0d5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.947+0000 7f889c600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f888c0ca310 0x7f888c0d6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.947+0000 7f889c600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f888c0c9570 0x7f888c049c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66f00c32b0 msgr2=0x7f66f0152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 --2- 172.21.15.67:0/3061979405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66f00c32b0 0x7f66f0152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 >> v2:172.21.15.134:3300/0 conn(0x7f66f014f4b0 msgr2=0x7f66f0155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f889c600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c0d7eb0 con 0x7f887c0052e0 2024-03-09T08:52:37.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 --2- 172.21.15.67:0/3061979405 >> v2:172.21.15.134:3300/0 conn(0x7f66f014f4b0 0x7f66f0155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0155ba0 con 0x7f66f00aa420 2024-03-09T08:52:37.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f66dc001670 con 0x7f66f00aa420 2024-03-09T08:52:37.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 >> v1:172.21.15.67:6789/0 conn(0x7f66f00aa420 legacy=0x7f66f00abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 --2- 172.21.15.67:0/3061979405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d8003420 0x7f66d8004240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 --2- 172.21.15.67:0/3061979405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d8004780 0x7f66d8005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 --2- 172.21.15.67:0/3061979405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 0x7f66d8005a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f014f890 con 0x7f66d80055d0 2024-03-09T08:52:37.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d800c250 con 0x7f66d8003420 2024-03-09T08:52:37.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d800c450 con 0x7f66d8004780 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883453728 0 0) 0x7f90200c1c30 con 0x7f9020152730 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9004003520 con 0x7f9020152730 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017739307 0 0) 0x7f90200c1f10 con 0x7f90200c47b0 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90200c1c30 con 0x7f90200c47b0 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3791601122 0 0) 0x7f9004003520 con 0x7f9020152730 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 >> v2:172.21.15.134:3300/0 conn(0x7f90200c1410 msgr2=0x7f90200c29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 --2- 172.21.15.67:0/284921922 >> v2:172.21.15.134:3300/0 conn(0x7f90200c1410 0x7f90200c29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 >> v1:172.21.15.134:6789/0 conn(0x7f90200c47b0 legacy=0x7f90201531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90200c21f0 con 0x7f9020152730 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f900c001670 con 0x7f9020152730 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 >> v1:172.21.15.67:6789/0 conn(0x7f9020152730 legacy=0x7f9020148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9004003bb0 0x7f9004003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9004004800 0x7f9004004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90040051c0 0x7f900400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f889c600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c0dacc0 con 0x7f888c0ca310 2024-03-09T08:52:37.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f889c600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888c0db1e0 con 0x7f888c0c9570 2024-03-09T08:52:37.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.948+0000 7f8899200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f888c0ca310 0x7f888c0d6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90201538b0 con 0x7f9004003bb0 2024-03-09T08:52:37.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900400c0c0 con 0x7f90040051c0 2024-03-09T08:52:37.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.946+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900400c2c0 con 0x7f9004004800 2024-03-09T08:52:37.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.949+0000 7f901e800640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90040051c0 0x7f900400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.949+0000 7f901de00640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9004003bb0 0x7f9004003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.949+0000 7f66f5400640 1 --2- 172.21.15.67:0/3061979405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 0x7f66d8005a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.949+0000 7f8899c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 0x7f888c0d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.949+0000 7f8899c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 0x7f888c0d5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51578/0 (socket says 172.21.15.67:51578) 2024-03-09T08:52:37.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.949+0000 7f8899c00640 1 -- 172.21.15.67:0/471210809 learned_addr learned my addr 172.21.15.67:0/471210809 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.949+0000 7f889a600640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f888c0c9570 0x7f888c049c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.949+0000 7f857fe00640 1 --2- 172.21.15.67:0/2262149482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85800c1d70 0x7f858015e250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f858006fca0 tx=0x7f856c02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ee4eedeb48fd0e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.949+0000 7fe522600640 1 --2- 172.21.15.67:0/426940376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c0c3810 0x7fe51c0703c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe51c06b980 tx=0x7fe51402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ceed78e555a0316 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f66efe00640 1 --2- 172.21.15.67:0/3061979405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d8004780 0x7f66d8005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f901d400640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9004004800 0x7f9004004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f8899c00640 1 -- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f888c0c9570 msgr2=0x7f888c049c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f8899c00640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f888c0c9570 0x7f888c049c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f8899c00640 1 -- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f888c0ca310 msgr2=0x7f888c0d6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f8899c00640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f888c0ca310 0x7f888c0d6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f8899c00640 1 -- 172.21.15.67:0/471210809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c0ca0c0 con 0x7f887c0052e0 2024-03-09T08:52:37.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f857f400640 1 -- 172.21.15.67:0/2262149482 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c0317c0 con 0x7f85800c1d70 2024-03-09T08:52:37.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f857f400640 1 -- 172.21.15.67:0/2262149482 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c037070 con 0x7f85800c1d70 2024-03-09T08:52:37.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f857f400640 1 -- 172.21.15.67:0/2262149482 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c031b80 con 0x7f85800c1d70 2024-03-09T08:52:37.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7fe51be00640 1 -- 172.21.15.67:0/426940376 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514039070 con 0x7fe51c0c3810 2024-03-09T08:52:37.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7fe51be00640 1 -- 172.21.15.67:0/426940376 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51402cbf0 con 0x7fe51c0c3810 2024-03-09T08:52:37.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7fe51be00640 1 -- 172.21.15.67:0/426940376 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514034440 con 0x7fe51c0c3810 2024-03-09T08:52:37.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f66f5400640 1 -- 172.21.15.67:0/3061979405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d8004780 msgr2=0x7f66d8005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f66f5400640 1 --2- 172.21.15.67:0/3061979405 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66d8004780 0x7f66d8005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f66f5400640 1 -- 172.21.15.67:0/3061979405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d8003420 msgr2=0x7f66d8004240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f66f5400640 1 --2- 172.21.15.67:0/3061979405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66d8003420 0x7f66d8004240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f66f5400640 1 -- 172.21.15.67:0/3061979405 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0155ba0 con 0x7f66d80055d0 2024-03-09T08:52:37.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f901de00640 1 -- 172.21.15.67:0/284921922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9004004800 msgr2=0x7f9004004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f901de00640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9004004800 0x7f9004004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f901de00640 1 -- 172.21.15.67:0/284921922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90040051c0 msgr2=0x7f900400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f901de00640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90040051c0 0x7f900400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f901de00640 1 -- 172.21.15.67:0/284921922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90200c21f0 con 0x7f9004003bb0 2024-03-09T08:52:37.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f901e800640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90040051c0 0x7f900400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:52:37.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.950+0000 7f901e800640 1 -- 172.21.15.67:0/284921922 reap_dead start 2024-03-09T08:52:37.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.951+0000 7f8899c00640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 0x7f888c0d5690 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f888000c7d0 tx=0x7f888000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.952+0000 7fe523000640 1 -- 172.21.15.67:0/426940376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c0c3810 msgr2=0x7fe51c0703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.952+0000 7fe523000640 1 --2- 172.21.15.67:0/426940376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c0c3810 0x7fe51c0703c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe51c06b980 tx=0x7fe51402be40 comp rx=0 tx=0).stop 2024-03-09T08:52:37.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.952+0000 7fe523000640 1 -- 172.21.15.67:0/426940376 shutdown_connections 2024-03-09T08:52:37.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.952+0000 7fe523000640 1 --2- 172.21.15.67:0/426940376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c0c3810 0x7fe51c0703c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.952+0000 7fe523000640 1 --2- 172.21.15.67:0/426940376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c15aa40 0x7fe51c0c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.952+0000 7fe523000640 1 --2- 172.21.15.67:0/426940376 >> v2:172.21.15.134:3300/0 conn(0x7fe51c153fc0 0x7fe51c0c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.952+0000 7fe523000640 1 -- 172.21.15.67:0/426940376 >> 172.21.15.67:0/426940376 conn(0x7fe51c0c9470 msgr2=0x7fe51c0c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.952+0000 7f8891600640 1 -- 172.21.15.67:0/471210809 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880018070 con 0x7f887c0052e0 2024-03-09T08:52:37.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.952+0000 7f889c600640 1 -- 172.21.15.67:0/471210809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888c0d3510 con 0x7f887c0052e0 2024-03-09T08:52:37.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.952+0000 7f889c600640 1 -- 172.21.15.67:0/471210809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888c0c9de0 con 0x7f887c0052e0 2024-03-09T08:52:37.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.952+0000 7f901de00640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9004003bb0 0x7f9004003fb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f90100018b0 tx=0x7f9004005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd1f385ba4f3ac server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.953+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90100014d0 con 0x7f9004003bb0 2024-03-09T08:52:37.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.953+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9010005270 con 0x7f9004003bb0 2024-03-09T08:52:37.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.953+0000 7f8587800640 1 -- 172.21.15.67:0/2262149482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85800c1d70 msgr2=0x7f858015e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.953+0000 7f8587800640 1 --2- 172.21.15.67:0/2262149482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85800c1d70 0x7f858015e250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f858006fca0 tx=0x7f856c02b860 comp rx=0 tx=0).stop 2024-03-09T08:52:37.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.953+0000 7f901ca00640 1 -- 172.21.15.67:0/284921922 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010003830 con 0x7f9004003bb0 2024-03-09T08:52:37.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.953+0000 7fe523000640 1 -- 172.21.15.67:0/426940376 shutdown_connections 2024-03-09T08:52:37.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.954+0000 7f901f200640 1 -- 172.21.15.67:0/284921922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9004003bb0 msgr2=0x7f9004003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.954+0000 7f901f200640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9004003bb0 0x7f9004003fb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f90100018b0 tx=0x7f9004005830 comp rx=0 tx=0).stop 2024-03-09T08:52:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.955+0000 7f901f200640 1 -- 172.21.15.67:0/284921922 shutdown_connections 2024-03-09T08:52:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.955+0000 7f901f200640 1 --2- 172.21.15.67:0/284921922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9004003bb0 0x7f9004003fb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.953+0000 7f8891600640 1 -- 172.21.15.67:0/471210809 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8880007560 con 0x7f887c0052e0 2024-03-09T08:52:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.953+0000 7f8891600640 1 -- 172.21.15.67:0/471210809 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880018070 con 0x7f887c0052e0 2024-03-09T08:52:37.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.956+0000 7f7a03000640 1 Processor -- start 2024-03-09T08:52:37.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.955+0000 7f901f200640 1 -- 172.21.15.67:0/284921922 >> 172.21.15.67:0/284921922 conn(0x7f90200c3c90 msgr2=0x7f90200c40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.955+0000 7f901f200640 1 -- 172.21.15.67:0/284921922 shutdown_connections 2024-03-09T08:52:37.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.956+0000 7f7a03000640 1 -- start start 2024-03-09T08:52:37.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.955+0000 7f901f200640 1 -- 172.21.15.67:0/284921922 wait complete. 2024-03-09T08:52:37.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.955+0000 7f889c600640 1 -- 172.21.15.67:0/471210809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8870005290 con 0x7f887c0052e0 2024-03-09T08:52:37.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.956+0000 7f8587800640 1 -- 172.21.15.67:0/2262149482 shutdown_connections 2024-03-09T08:52:37.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.956+0000 7f8587800640 1 --2- 172.21.15.67:0/2262149482 >> v2:172.21.15.67:3300/0 conn(0x7f85800aad50 0x7f85800ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.956+0000 7f8587800640 1 --2- 172.21.15.67:0/2262149482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85800c1d70 0x7f858015e250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.956+0000 7f8587800640 1 -- 172.21.15.67:0/2262149482 >> 172.21.15.67:0/2262149482 conn(0x7f85800c7370 msgr2=0x7f85800c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.955+0000 7f66f5400640 1 --2- 172.21.15.67:0/3061979405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 0x7f66d8005a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66f00c3690 tx=0x7f66d80060c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fd0bc8a02fbd03 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.956+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8039070 con 0x7f66d80055d0 2024-03-09T08:52:37.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.956+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e802cb90 con 0x7f66d80055d0 2024-03-09T08:52:37.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.956+0000 7f66ef400640 1 -- 172.21.15.67:0/3061979405 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8009ae0 con 0x7f66d80055d0 2024-03-09T08:52:37.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.954+0000 7fe523000640 1 -- 172.21.15.67:0/426940376 wait complete. 2024-03-09T08:52:37.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.956+0000 7fe523000640 1 Processor -- start 2024-03-09T08:52:37.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.957+0000 7fe523000640 1 -- start start 2024-03-09T08:52:37.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.957+0000 7f7a03000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f79fc1503d0 0x7f79fc1560f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.957+0000 7f7a03000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79fc06ee70 0x7f79fc0c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.957+0000 7f7a03000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc1594c0 con 0x7f79fc0c88b0 2024-03-09T08:52:37.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.957+0000 7f7a03000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc157080 con 0x7f79fc1503d0 2024-03-09T08:52:37.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.957+0000 7f7a03000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc154210 con 0x7f79fc06ee70 2024-03-09T08:52:37.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f901f200640 1 Processor -- start 2024-03-09T08:52:37.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.960+0000 7f901f200640 1 -- start start 2024-03-09T08:52:37.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.957+0000 7f8587800640 1 -- 172.21.15.67:0/2262149482 shutdown_connections 2024-03-09T08:52:37.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.958+0000 7f7a01000640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f79fc0c88b0 0x7f79fc1546a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:60988/0 (socket says 172.21.15.67:60988) 2024-03-09T08:52:37.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.958+0000 7f7a01000640 1 -- 172.21.15.67:0/747635583 learned_addr learned my addr 172.21.15.67:0/747635583 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:37.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.958+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475494354 0 0) 0x7f79fc1594c0 con 0x7f79fc0c88b0 2024-03-09T08:52:37.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.958+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79e0003520 con 0x7f79fc0c88b0 2024-03-09T08:52:37.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1284380017 0 0) 0x7f79e0003520 con 0x7f79fc0c88b0 2024-03-09T08:52:37.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79fc06ee70 msgr2=0x7f79fc0c8370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:52:37.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 --2- 172.21.15.67:0/747635583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79fc06ee70 0x7f79fc0c8370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 >> v2:172.21.15.67:3300/0 conn(0x7f79fc1503d0 msgr2=0x7f79fc1560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 --2- 172.21.15.67:0/747635583 >> v2:172.21.15.67:3300/0 conn(0x7f79fc1503d0 0x7f79fc1560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc1567e0 con 0x7f79fc0c88b0 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f79ec001670 con 0x7f79fc0c88b0 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 >> v1:172.21.15.134:6789/0 conn(0x7f79fc0c88b0 legacy=0x7f79fc1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 --2- 172.21.15.67:0/747635583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79e00039b0 0x7f79e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 --2- 172.21.15.67:0/747635583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79e0004800 0x7f79e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 --2- 172.21.15.67:0/747635583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00053c0 0x7f79e000b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc157080 con 0x7f79e00039b0 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc154210 con 0x7f79e0004800 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.959+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e000c2f0 con 0x7f79e00053c0 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.960+0000 7f7a01000640 1 --2- 172.21.15.67:0/747635583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00053c0 0x7f79e000b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.960+0000 7f7a01000640 1 -- 172.21.15.67:0/747635583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79e0004800 msgr2=0x7f79e0004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.960+0000 7f7a01000640 1 --2- 172.21.15.67:0/747635583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79e0004800 0x7f79e0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.960+0000 7f7a01000640 1 -- 172.21.15.67:0/747635583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79e00039b0 msgr2=0x7f79e0003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.960+0000 7f7a01000640 1 --2- 172.21.15.67:0/747635583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79e00039b0 0x7f79e0003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.960+0000 7f7a01000640 1 -- 172.21.15.67:0/747635583 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc1567e0 con 0x7f79e00053c0 2024-03-09T08:52:37.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.957+0000 7f66f7400640 1 -- 172.21.15.67:0/3061979405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 msgr2=0x7f66d8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.957+0000 7f66f7400640 1 --2- 172.21.15.67:0/3061979405 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 0x7f66d8005a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66f00c3690 tx=0x7f66d80060c0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.957+0000 7f66f5400640 1 -- 172.21.15.67:0/3061979405 reap_dead start 2024-03-09T08:52:37.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.961+0000 7f66f7400640 1 -- 172.21.15.67:0/3061979405 shutdown_connections 2024-03-09T08:52:37.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.961+0000 7f66f7400640 1 -- 172.21.15.67:0/3061979405 >> 172.21.15.67:0/3061979405 conn(0x7f66f00c7370 msgr2=0x7f66f00c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.961+0000 7f66f7400640 1 -- 172.21.15.67:0/3061979405 shutdown_connections 2024-03-09T08:52:37.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.958+0000 7f8587800640 1 -- 172.21.15.67:0/2262149482 wait complete. 2024-03-09T08:52:37.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.960+0000 7f8587800640 1 Processor -- start 2024-03-09T08:52:37.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.961+0000 7f66f7400640 1 -- 172.21.15.67:0/3061979405 wait complete. 2024-03-09T08:52:37.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.961+0000 7f901f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9004003bb0 0x7f9020070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.962+0000 7f901f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f902013d610 0x7f9020145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.962+0000 7f8587800640 1 -- start start 2024-03-09T08:52:37.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.962+0000 7f901de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9004003bb0 0x7f9020070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.962+0000 7f901de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9004003bb0 0x7f9020070c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46374/0 (socket says 172.21.15.67:46374) 2024-03-09T08:52:37.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.962+0000 7f901de00640 1 -- 172.21.15.67:0/203503328 learned_addr learned my addr 172.21.15.67:0/203503328 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.962+0000 7f901d400640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f902013d610 0x7f9020145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.962+0000 7fe523000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c0c3810 0x7fe51c1efe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.962+0000 7f901f200640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020072a10 0x7f90200ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.962+0000 7fe523000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe51c153fc0 0x7fe51c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.963+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90200c3370 con 0x7f902013d610 2024-03-09T08:52:37.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.963+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90200c31f0 con 0x7f9020072a10 2024-03-09T08:52:37.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.963+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90200c29f0 con 0x7f9004003bb0 2024-03-09T08:52:37.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.963+0000 7fe521c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c0c3810 0x7fe51c1efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.963+0000 7fe521c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c0c3810 0x7fe51c1efe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46378/0 (socket says 172.21.15.67:46378) 2024-03-09T08:52:37.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.963+0000 7f901e800640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020072a10 0x7f90200ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.963+0000 7f7a01000640 1 --2- 172.21.15.67:0/747635583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00053c0 0x7f79e000b9e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c290 tx=0x7f79e0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b65d139857f17bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7f901d400640 1 -- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9004003bb0 msgr2=0x7f9020070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7f901d400640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9004003bb0 0x7f9020070c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7f901d400640 1 -- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020072a10 msgr2=0x7f90200ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7f901d400640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020072a10 0x7f90200ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7f901d400640 1 -- 172.21.15.67:0/203503328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9020073400 con 0x7f902013d610 2024-03-09T08:52:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.965+0000 7f901d400640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f902013d610 0x7f9020145370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f900c000fb0 tx=0x7f900c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.963+0000 7f8891600640 1 -- 172.21.15.67:0/471210809 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f888001b070 con 0x7f887c0052e0 2024-03-09T08:52:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7f8891600640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f887406b700 0x7f887406dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7f8891600640 1 -- 172.21.15.67:0/471210809 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f88800898e0 con 0x7f887c0052e0 2024-03-09T08:52:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.965+0000 7f8587800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85800aad50 0x7f858015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.965+0000 7f8587800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85800c1d70 0x7f85801539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.965+0000 7f8587800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580157820 0x7f8580153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7f8584e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85800aad50 0x7f858015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7f8584e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85800aad50 0x7f858015c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46424/0 (socket says 172.21.15.67:46424) 2024-03-09T08:52:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7f8585800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580157820 0x7f8580153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7f857fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85800c1d70 0x7f85801539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7f857fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85800c1d70 0x7f85801539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46384/0 (socket says 172.21.15.67:46384) 2024-03-09T08:52:37.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7f857fe00640 1 -- 172.21.15.67:0/1479029705 learned_addr learned my addr 172.21.15.67:0/1479029705 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85800c3b80 con 0x7f8580157820 2024-03-09T08:52:37.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8580157c00 con 0x7f85800aad50 2024-03-09T08:52:37.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85800abf50 con 0x7f85800c1d70 2024-03-09T08:52:37.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7fb99cc00640 1 Processor -- start 2024-03-09T08:52:37.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7fe523000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c15aa40 0x7fe51c1ec670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7fe521200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe51c153fc0 0x7fe51c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0c6670 con 0x7fe51c15aa40 2024-03-09T08:52:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c1543a0 con 0x7fe51c153fc0 2024-03-09T08:52:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0c67f0 con 0x7fe51c0c3810 2024-03-09T08:52:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7fe521c00640 1 -- 172.21.15.67:0/1690228817 learned_addr learned my addr 172.21.15.67:0/1690228817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec007db0 con 0x7f79e00053c0 2024-03-09T08:52:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7f79faa00640 1 -- 172.21.15.67:0/747635583 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec030070 con 0x7f79e00053c0 2024-03-09T08:52:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.965+0000 7f7a03000640 1 -- 172.21.15.67:0/747635583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00053c0 msgr2=0x7f79e000b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.965+0000 7f7a03000640 1 --2- 172.21.15.67:0/747635583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00053c0 0x7f79e000b9e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c290 tx=0x7f79e0005830 comp rx=0 tx=0).stop 2024-03-09T08:52:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.965+0000 7f7a01000640 1 -- 172.21.15.67:0/747635583 reap_dead start 2024-03-09T08:52:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.965+0000 7f7a03000640 1 -- 172.21.15.67:0/747635583 shutdown_connections 2024-03-09T08:52:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.965+0000 7f7a03000640 1 -- 172.21.15.67:0/747635583 >> 172.21.15.67:0/747635583 conn(0x7f79fc0aa890 msgr2=0x7f79fc0ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.965+0000 7f7a03000640 1 -- 172.21.15.67:0/747635583 shutdown_connections 2024-03-09T08:52:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.965+0000 7f7a03000640 1 -- 172.21.15.67:0/747635583 wait complete. 2024-03-09T08:52:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7fb99cc00640 1 -- start start 2024-03-09T08:52:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7fb99cc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb998153ed0 0x7fb99815a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7fb99cc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb9980c4630 0x7fb99806ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7fb99cc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99806d3f0 con 0x7fb99815dc30 2024-03-09T08:52:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7fb99cc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99806d6d0 con 0x7fb9980c4630 2024-03-09T08:52:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7fb99cc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99815d430 con 0x7fb998153ed0 2024-03-09T08:52:37.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7f8585800640 1 -- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85800c1d70 msgr2=0x7f85801539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.966+0000 7fe522600640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c15aa40 0x7fe51c1ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7f8585800640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85800c1d70 0x7f85801539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7f8585800640 1 -- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85800aad50 msgr2=0x7f858015c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7fe521200640 1 -- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c0c3810 msgr2=0x7fe51c1efe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7fe521200640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c0c3810 0x7fe51c1efe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7fe521200640 1 -- 172.21.15.67:0/1690228817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c15aa40 msgr2=0x7fe51c1ec670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7f8585800640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85800aad50 0x7f858015c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7f8585800640 1 -- 172.21.15.67:0/1479029705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8580154840 con 0x7f8580157820 2024-03-09T08:52:37.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7f8584e00640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85800aad50 0x7f858015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:37.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7fe521200640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c15aa40 0x7fe51c1ec670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7fe521200640 1 -- 172.21.15.67:0/1690228817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c1f0390 con 0x7fe51c153fc0 2024-03-09T08:52:37.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7fe522600640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c15aa40 0x7fe51c1ec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:37.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.967+0000 7f8585800640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580157820 0x7f8580153f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8578007d80 tx=0x7f857800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7f7a03000640 1 Processor -- start 2024-03-09T08:52:37.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fe521200640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe51c153fc0 0x7fe51c06d410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe51000e850 tx=0x7fe51000ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fb997e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb99815dc30 0x7fb99806b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:32768/0 (socket says 172.21.15.67:32768) 2024-03-09T08:52:37.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fb997e00640 1 -- 172.21.15.67:0/2560335948 learned_addr learned my addr 172.21.15.67:0/2560335948 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:37.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7f7a03000640 1 -- start start 2024-03-09T08:52:37.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fb996a00640 1 --2- 172.21.15.67:0/2560335948 >> v2:172.21.15.67:3300/0 conn(0x7fb9980c4630 0x7fb99806ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fb997400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb998153ed0 0x7fb99815a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fb996a00640 1 -- 172.21.15.67:0/2560335948 >> v2:172.21.15.134:3300/0 conn(0x7fb998153ed0 msgr2=0x7fb99815a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fb996a00640 1 --2- 172.21.15.67:0/2560335948 >> v2:172.21.15.134:3300/0 conn(0x7fb998153ed0 0x7fb99815a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fe519600640 1 -- 172.21.15.67:0/1690228817 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510017070 con 0x7fe51c153fc0 2024-03-09T08:52:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fe519600640 1 -- 172.21.15.67:0/1690228817 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51000c940 con 0x7fe51c153fc0 2024-03-09T08:52:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fe519600640 1 -- 172.21.15.67:0/1690228817 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000cd30 con 0x7fe51c153fc0 2024-03-09T08:52:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fb996a00640 1 -- 172.21.15.67:0/2560335948 >> v1:172.21.15.134:6789/0 conn(0x7fb99815dc30 legacy=0x7fb99806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.969+0000 7fb996a00640 1 -- 172.21.15.67:0/2560335948 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99815d710 con 0x7fb9980c4630 2024-03-09T08:52:37.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.968+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c1529c0 con 0x7fe51c153fc0 2024-03-09T08:52:37.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.969+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c152400 con 0x7fe51c153fc0 2024-03-09T08:52:37.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.969+0000 7f7a03000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00053c0 0x7f79fc152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.969+0000 7f7a03000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79fc142880 0x7f79fc142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.969+0000 7f7a03000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79fc152b20 0x7f79fc152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.969+0000 7f7a03000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0ae550 con 0x7f79fc152b20 2024-03-09T08:52:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.969+0000 7f7a03000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc156da0 con 0x7f79fc142880 2024-03-09T08:52:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.969+0000 7f7a03000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0c8370 con 0x7f79e00053c0 2024-03-09T08:52:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.969+0000 7fb997400640 1 --2- 172.21.15.67:0/2560335948 >> v2:172.21.15.134:3300/0 conn(0x7fb998153ed0 0x7fb99815a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.969+0000 7f7a01000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79fc152b20 0x7f79fc152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.970+0000 7f7a01000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79fc152b20 0x7f79fc152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51636/0 (socket says 172.21.15.67:51636) 2024-03-09T08:52:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.970+0000 7f7a01000640 1 -- 172.21.15.67:0/753855689 learned_addr learned my addr 172.21.15.67:0/753855689 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.970+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c0c9140 con 0x7fe51c153fc0 2024-03-09T08:52:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.970+0000 7f79fbe00640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00053c0 0x7f79fc152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.970+0000 7f9016000640 1 -- 172.21.15.67:0/203503328 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c0050d0 con 0x7f902013d610 2024-03-09T08:52:37.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.971+0000 7f9016000640 1 -- 172.21.15.67:0/203503328 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c018070 con 0x7f902013d610 2024-03-09T08:52:37.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.971+0000 7f8899200640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f887406b700 0x7f887406dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.971+0000 7f857cc00640 1 -- 172.21.15.67:0/1479029705 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857801a070 con 0x7f8580157820 2024-03-09T08:52:37.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.971+0000 7f857cc00640 1 -- 172.21.15.67:0/1479029705 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857800ddb0 con 0x7f8580157820 2024-03-09T08:52:37.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.971+0000 7f857cc00640 1 -- 172.21.15.67:0/1479029705 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578015600 con 0x7f8580157820 2024-03-09T08:52:37.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.971+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8580158970 con 0x7f8580157820 2024-03-09T08:52:37.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.971+0000 7f9016000640 1 -- 172.21.15.67:0/203503328 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c00c690 con 0x7f902013d610 2024-03-09T08:52:37.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.971+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90201f0850 con 0x7f902013d610 2024-03-09T08:52:37.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.971+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85801f9400 con 0x7f8580157820 2024-03-09T08:52:37.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.971+0000 7f8891600640 1 -- 172.21.15.67:0/471210809 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8880057e50 con 0x7f887c0052e0 2024-03-09T08:52:37.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.971+0000 7fb996a00640 1 --2- 172.21.15.67:0/2560335948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9980c4630 0x7fb99806ed90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb9980c43d0 tx=0x7fb98402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1281e3cb511d4ff6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.972+0000 7f79fb400640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79fc142880 0x7f79fc142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:37.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.972+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90201f0b30 con 0x7f902013d610 2024-03-09T08:52:37.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.972+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85801f9600 con 0x7f8580157820 2024-03-09T08:52:37.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.973+0000 7f8899200640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f887406b700 0x7f887406dbc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f888c04a810 tx=0x7f8888009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.973+0000 7f79fbe00640 1 -- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79fc142880 msgr2=0x7f79fc142d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.973+0000 7f79fbe00640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79fc142880 0x7f79fc142d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.973+0000 7f79fbe00640 1 -- 172.21.15.67:0/753855689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79fc152b20 msgr2=0x7f79fc152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:37.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.974+0000 7f79fbe00640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79fc152b20 0x7f79fc152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:37.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.974+0000 7f79fbe00640 1 -- 172.21.15.67:0/753855689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc151460 con 0x7f79e00053c0 2024-03-09T08:52:37.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.974+0000 7f79fb400640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79fc142880 0x7f79fc142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:37.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.974+0000 7f7a01000640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79fc152b20 0x7f79fc152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:52:37.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.964+0000 7f66f7400640 1 Processor -- start 2024-03-09T08:52:37.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.974+0000 7fe519600640 1 -- 172.21.15.67:0/1690228817 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe51001a070 con 0x7fe51c153fc0 2024-03-09T08:52:37.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.975+0000 7f79fbe00640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00053c0 0x7f79fc152010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f79e800b470 tx=0x7f79e800b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:37.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.975+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902004f3c0 con 0x7f902013d610 2024-03-09T08:52:37.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.975+0000 7f79f3e00640 1 -- 172.21.15.67:0/753855689 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8015020 con 0x7f79e00053c0 2024-03-09T08:52:37.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.976+0000 7f79f3e00640 1 -- 172.21.15.67:0/753855689 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e8011830 con 0x7f79e00053c0 2024-03-09T08:52:37.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.976+0000 7fe519600640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4fc06b4d0 0x7fe4fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:37.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.976+0000 7f79f3e00640 1 -- 172.21.15.67:0/753855689 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8011c20 con 0x7f79e00053c0 2024-03-09T08:52:37.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.976+0000 7f7a03000640 1 -- 172.21.15.67:0/753855689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc143a50 con 0x7f79e00053c0 2024-03-09T08:52:37.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.976+0000 7f9016000640 1 -- 172.21.15.67:0/203503328 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f900c00c830 con 0x7f902013d610 2024-03-09T08:52:37.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.976+0000 7fe519600640 1 -- 172.21.15.67:0/1690228817 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fe51008ac80 con 0x7fe51c153fc0 2024-03-09T08:52:37.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.976+0000 7f7a03000640 1 -- 172.21.15.67:0/753855689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc141440 con 0x7f79e00053c0 2024-03-09T08:52:37.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.977+0000 7f7a03000640 1 -- 172.21.15.67:0/753855689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc14f270 con 0x7f79e00053c0 2024-03-09T08:52:37.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.978+0000 7f9016000640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f900006b550 0x7f900006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.978+0000 7fb996000640 1 -- 172.21.15.67:0/2560335948 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9840315a0 con 0x7fb9980c4630 2024-03-09T08:52:38.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.978+0000 7fe521c00640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4fc06b4d0 0x7fe4fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.978+0000 7fb996000640 1 -- 172.21.15.67:0/2560335948 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb984037070 con 0x7fb9980c4630 2024-03-09T08:52:38.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.978+0000 7fb996000640 1 -- 172.21.15.67:0/2560335948 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984031960 con 0x7fb9980c4630 2024-03-09T08:52:38.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.975+0000 7f66f7400640 1 -- start start 2024-03-09T08:52:38.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.979+0000 7f901de00640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f900006b550 0x7f900006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.979+0000 7f66f7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 0x7f66f014b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7f66f7400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66f0154800 0x7f66f01445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7f66f7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f0144b30 0x7f66f01e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7f66f7400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0152050 con 0x7f66d80055d0 2024-03-09T08:52:38.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7f66f7400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0156160 con 0x7f66f0144b30 2024-03-09T08:52:38.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7f66f7400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00ac3b0 con 0x7f66f0154800 2024-03-09T08:52:38.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7fb99cc00640 1 -- 172.21.15.67:0/2560335948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9980c4630 msgr2=0x7fb99806ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7fb99cc00640 1 --2- 172.21.15.67:0/2560335948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9980c4630 0x7fb99806ed90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb9980c43d0 tx=0x7fb98402b840 comp rx=0 tx=0).stop 2024-03-09T08:52:38.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7fe521c00640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4fc06b4d0 0x7fe4fc06d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe508006fd0 tx=0x7fe508008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:38.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7fb99cc00640 1 -- 172.21.15.67:0/2560335948 shutdown_connections 2024-03-09T08:52:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7fb99cc00640 1 --2- 172.21.15.67:0/2560335948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9980c4630 0x7fb99806ed90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7fb99cc00640 1 --2- 172.21.15.67:0/2560335948 >> v2:172.21.15.134:3300/0 conn(0x7fb998153ed0 0x7fb99815a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7f66f4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 0x7f66f014b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7f9016000640 1 -- 172.21.15.67:0/203503328 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f900c088370 con 0x7f902013d610 2024-03-09T08:52:38.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7fb99cc00640 1 -- 172.21.15.67:0/2560335948 >> 172.21.15.67:0/2560335948 conn(0x7fb9980c8480 msgr2=0x7fb9980c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:38.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7f66f4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 0x7f66f014b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51638/0 (socket says 172.21.15.67:51638) 2024-03-09T08:52:38.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.980+0000 7f66f4a00640 1 -- 172.21.15.67:0/769594782 learned_addr learned my addr 172.21.15.67:0/769594782 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:38.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.981+0000 7f66efe00640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66f0154800 0x7f66f01445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.981+0000 7f66f5400640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f0144b30 0x7f66f01e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.981+0000 7f901de00640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f900006b550 0x7f900006da10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9020145fb0 tx=0x7f901002f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:38.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.981+0000 7f66f4a00640 1 -- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66f0154800 msgr2=0x7f66f01445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.981+0000 7f66f4a00640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66f0154800 0x7f66f01445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.981+0000 7f66f4a00640 1 -- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f0144b30 msgr2=0x7f66f01e6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.981+0000 7f66f4a00640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f0144b30 0x7f66f01e6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.981+0000 7f66f4a00640 1 -- 172.21.15.67:0/769594782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0071a20 con 0x7f66d80055d0 2024-03-09T08:52:38.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.981+0000 7f857cc00640 1 -- 172.21.15.67:0/1479029705 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f857801d070 con 0x7f8580157820 2024-03-09T08:52:38.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.982+0000 7fb99cc00640 1 -- 172.21.15.67:0/2560335948 shutdown_connections 2024-03-09T08:52:38.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.982+0000 7fb99cc00640 1 -- 172.21.15.67:0/2560335948 wait complete. 2024-03-09T08:52:38.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.982+0000 7f857cc00640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f856006b520 0x7f856006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.982+0000 7f857cc00640 1 -- 172.21.15.67:0/1479029705 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f857808a900 con 0x7f8580157820 2024-03-09T08:52:38.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.983+0000 7f66f4a00640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 0x7f66f014b1e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009fd0 tx=0x7f66e400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:38.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.983+0000 7f8584e00640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f856006b520 0x7f856006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.984+0000 7fb99cc00640 1 Processor -- start 2024-03-09T08:52:38.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.984+0000 7fb99cc00640 1 -- start start 2024-03-09T08:52:38.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.985+0000 7f8584e00640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f856006b520 0x7f856006d9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8580068330 tx=0x7f8570009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:38.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.985+0000 7f79f3e00640 1 -- 172.21.15.67:0/753855689 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f79e8018020 con 0x7f79e00053c0 2024-03-09T08:52:38.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.986+0000 7fb99cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9980c4630 0x7fb998147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.986+0000 7fb99cc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb998153ed0 0x7fb998156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.986+0000 7fb99cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb99815dc30 0x7fb9981566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.986+0000 7fb99cc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99806b750 con 0x7fb9980c4630 2024-03-09T08:52:38.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.986+0000 7fb99cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb998071aa0 con 0x7fb99815dc30 2024-03-09T08:52:38.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.986+0000 7f79f3e00640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79dc06b500 0x7f79dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.986+0000 7f79f3e00640 1 -- 172.21.15.67:0/753855689 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f79e808a880 con 0x7f79e00053c0 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.986+0000 7fb99cc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99806b8d0 con 0x7fb998153ed0 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.987+0000 7fb997e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb99815dc30 0x7fb9981566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.987+0000 7fb997e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb99815dc30 0x7fb9981566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46460/0 (socket says 172.21.15.67:46460) 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.987+0000 7fb997e00640 1 -- 172.21.15.67:0/3578618128 learned_addr learned my addr 172.21.15.67:0/3578618128 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.987+0000 7fb996a00640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb998153ed0 0x7fb998156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.987+0000 7fb997400640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9980c4630 0x7fb998147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.988+0000 7fb997e00640 1 -- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb998153ed0 msgr2=0x7fb998156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.988+0000 7fb997e00640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb998153ed0 0x7fb998156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.988+0000 7fb997e00640 1 -- 172.21.15.67:0/3578618128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9980c4630 msgr2=0x7fb998147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.988+0000 7fb997e00640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9980c4630 0x7fb998147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.988+0000 7fb997e00640 1 -- 172.21.15.67:0/3578618128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9980c2780 con 0x7fb99815dc30 2024-03-09T08:52:38.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.988+0000 7fb997400640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9980c4630 0x7fb998147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:38.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.987+0000 7f79fb400640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79dc06b500 0x7f79dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.988+0000 7f857cc00640 1 -- 172.21.15.67:0/1479029705 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f85801f9600 con 0x7f8580157820 2024-03-09T08:52:38.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.988+0000 7fb997e00640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb99815dc30 0x7fb9981566b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb99000cad0 tx=0x7fb99000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:38.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.989+0000 7f79fb400640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79dc06b500 0x7f79dc06d9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f79fc161ac0 tx=0x7f79f40094f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:38.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.990+0000 7fb98f400640 1 -- 172.21.15.67:0/3578618128 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb990017070 con 0x7fb99815dc30 2024-03-09T08:52:38.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.990+0000 7fb98f400640 1 -- 172.21.15.67:0/3578618128 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99000ac30 con 0x7fb99815dc30 2024-03-09T08:52:38.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.990+0000 7fb98f400640 1 -- 172.21.15.67:0/3578618128 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb990012620 con 0x7fb99815dc30 2024-03-09T08:52:38.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.991+0000 7fe519600640 1 -- 172.21.15.67:0/1690228817 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe51c0c9140 con 0x7fe51c153fc0 2024-03-09T08:52:38.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.991+0000 7fb99cc00640 1 -- 172.21.15.67:0/3578618128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb998146260 con 0x7fb99815dc30 2024-03-09T08:52:38.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.991+0000 7fb99cc00640 1 -- 172.21.15.67:0/3578618128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9980ab530 con 0x7fb99815dc30 2024-03-09T08:52:38.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.992+0000 7f66ecc00640 1 -- 172.21.15.67:0/769594782 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4011830 con 0x7f66d80055d0 2024-03-09T08:52:38.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.992+0000 7f66ecc00640 1 -- 172.21.15.67:0/769594782 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e4016070 con 0x7f66d80055d0 2024-03-09T08:52:38.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.992+0000 7f66ecc00640 1 -- 172.21.15.67:0/769594782 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4011bf0 con 0x7f66d80055d0 2024-03-09T08:52:38.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.992+0000 7f66f7400640 1 -- 172.21.15.67:0/769594782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f0070c80 con 0x7f66d80055d0 2024-03-09T08:52:38.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.993+0000 7f66f7400640 1 -- 172.21.15.67:0/769594782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f0070f40 con 0x7f66d80055d0 2024-03-09T08:52:38.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.994+0000 7f66f7400640 1 -- 172.21.15.67:0/769594782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f0075a00 con 0x7f66d80055d0 2024-03-09T08:52:38.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.993+0000 7fb99cc00640 1 -- 172.21.15.67:0/3578618128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9980ab770 con 0x7fb99815dc30 2024-03-09T08:52:38.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.995+0000 7f9016000640 1 -- 172.21.15.67:0/203503328 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f900c0585f0 con 0x7f902013d610 2024-03-09T08:52:38.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.995+0000 7fb98f400640 1 -- 172.21.15.67:0/3578618128 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb99001a070 con 0x7fb99815dc30 2024-03-09T08:52:38.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:37.997+0000 7fb98f400640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb97806b4d0 0x7fb97806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.000+0000 7fb98f400640 1 -- 172.21.15.67:0/3578618128 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb99008a1b0 con 0x7fb99815dc30 2024-03-09T08:52:38.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.000+0000 7fb997400640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb97806b4d0 0x7fb97806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.000+0000 7f79f3e00640 1 -- 172.21.15.67:0/753855689 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f79fc14f270 con 0x7f79e00053c0 2024-03-09T08:52:38.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.002+0000 7fb997400640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb97806b4d0 0x7fb97806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb988006fd0 tx=0x7fb988008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:38.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.006+0000 7f66ecc00640 1 -- 172.21.15.67:0/769594782 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f66e401b460 con 0x7f66d80055d0 2024-03-09T08:52:38.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.008+0000 7fb98f400640 1 -- 172.21.15.67:0/3578618128 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb9980ab770 con 0x7fb99815dc30 2024-03-09T08:52:38.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.008+0000 7f66ecc00640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f66d006b520 0x7f66d006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.009+0000 7f66ecc00640 1 -- 172.21.15.67:0/769594782 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f66e408ab50 con 0x7f66d80055d0 2024-03-09T08:52:38.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.009+0000 7f66ecc00640 1 -- 172.21.15.67:0/769594782 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f66e40b7eb0 con 0x7f66d80055d0 2024-03-09T08:52:38.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.012+0000 7f66efe00640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f66d006b520 0x7f66d006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.016+0000 7f66efe00640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f66d006b520 0x7f66d006d9e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f66dc000fb0 tx=0x7f66dc007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:38.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.057+0000 7f6c63e00640 1 Processor -- start 2024-03-09T08:52:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.057+0000 7f6c63e00640 1 -- start start 2024-03-09T08:52:38.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.059+0000 7f6c63e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6c64072770 0x7f6c64072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.059+0000 7f6c63e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6406bda0 0x7f6c64156650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.059+0000 7f6c63e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6406cd30 con 0x7f6c640bebc0 2024-03-09T08:52:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.059+0000 7f6c63e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6406b200 con 0x7f6c64072770 2024-03-09T08:52:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.059+0000 7f6c63e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640c5eb0 con 0x7f6c6406bda0 2024-03-09T08:52:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.061+0000 7f6c62a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6c64072770 0x7f6c64072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.061+0000 7f6c63400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6406bda0 0x7f6c64156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.061+0000 7f6c62a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6c64072770 0x7f6c64072b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46462/0 (socket says 172.21.15.67:46462) 2024-03-09T08:52:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.061+0000 7f6c62a00640 1 -- 172.21.15.67:0/821041733 learned_addr learned my addr 172.21.15.67:0/821041733 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:38.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.062+0000 7f6c62a00640 1 -- 172.21.15.67:0/821041733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6406bda0 msgr2=0x7f6c64156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:38.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.062+0000 7f6c62a00640 1 --2- 172.21.15.67:0/821041733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6406bda0 0x7f6c64156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.062+0000 7f6c62a00640 1 -- 172.21.15.67:0/821041733 >> v1:172.21.15.134:6789/0 conn(0x7f6c640bebc0 legacy=0x7f6c6406b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.062+0000 7f6c62a00640 1 -- 172.21.15.67:0/821041733 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c640c6150 con 0x7f6c64072770 2024-03-09T08:52:38.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.062+0000 7f6c63400640 1 --2- 172.21.15.67:0/821041733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6406bda0 0x7f6c64156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:52:38.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.063+0000 7f6c62a00640 1 --2- 172.21.15.67:0/821041733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c64072770 0x7f6c64072b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009e50 tx=0x7f6c5c02be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cae2e58ac11b245 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:38.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.064+0000 7f6c61600640 1 -- 172.21.15.67:0/821041733 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c02c9f0 con 0x7f6c64072770 2024-03-09T08:52:38.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.064+0000 7f6c61600640 1 -- 172.21.15.67:0/821041733 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c02cb90 con 0x7f6c64072770 2024-03-09T08:52:38.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.064+0000 7f6c61600640 1 -- 172.21.15.67:0/821041733 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c02c9f0 con 0x7f6c64072770 2024-03-09T08:52:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.066+0000 7f6c63e00640 1 -- 172.21.15.67:0/821041733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c64072770 msgr2=0x7f6c64072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.066+0000 7f6c63e00640 1 --2- 172.21.15.67:0/821041733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c64072770 0x7f6c64072b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009e50 tx=0x7f6c5c02be50 comp rx=0 tx=0).stop 2024-03-09T08:52:38.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.066+0000 7f6c63e00640 1 -- 172.21.15.67:0/821041733 shutdown_connections 2024-03-09T08:52:38.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.066+0000 7f6c63e00640 1 --2- 172.21.15.67:0/821041733 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6406bda0 0x7f6c64156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.066+0000 7f6c63e00640 1 --2- 172.21.15.67:0/821041733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c64072770 0x7f6c64072b50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.066+0000 7f6c63e00640 1 -- 172.21.15.67:0/821041733 >> 172.21.15.67:0/821041733 conn(0x7f6c640c7fb0 msgr2=0x7f6c640c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:38.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.069+0000 7f6c63e00640 1 -- 172.21.15.67:0/821041733 shutdown_connections 2024-03-09T08:52:38.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.069+0000 7f6c63e00640 1 -- 172.21.15.67:0/821041733 wait complete. 2024-03-09T08:52:38.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.072+0000 7f6c63e00640 1 Processor -- start 2024-03-09T08:52:38.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.073+0000 7f6c63e00640 1 -- start start 2024-03-09T08:52:38.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.080+0000 7f6c63e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6406bda0 0x7f6c640c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.080+0000 7f6c63e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64072770 0x7f6c641eede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.080+0000 7f6c63e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c640bebc0 0x7f6c641ef320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.080+0000 7f6c63e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640c9430 con 0x7f6c64072770 2024-03-09T08:52:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.080+0000 7f6c63e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640c6710 con 0x7f6c640bebc0 2024-03-09T08:52:38.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.080+0000 7f6c63e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c641ef9f0 con 0x7f6c6406bda0 2024-03-09T08:52:38.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.084+0000 7f6c63400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c640bebc0 0x7f6c641ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.084+0000 7f6c63400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c640bebc0 0x7f6c641ef320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46474/0 (socket says 172.21.15.67:46474) 2024-03-09T08:52:38.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.084+0000 7f6c63400640 1 -- 172.21.15.67:0/952254649 learned_addr learned my addr 172.21.15.67:0/952254649 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:38.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.084+0000 7f6c62a00640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6406bda0 0x7f6c640c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.084+0000 7f6c62000640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64072770 0x7f6c641eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.084+0000 7f6c63400640 1 -- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6406bda0 msgr2=0x7f6c640c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.084+0000 7f6c63400640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6406bda0 0x7f6c640c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.084+0000 7f6c63400640 1 -- 172.21.15.67:0/952254649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64072770 msgr2=0x7f6c641eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.084+0000 7f6c63400640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64072770 0x7f6c641eede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.084+0000 7f6c63400640 1 -- 172.21.15.67:0/952254649 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c640c30b0 con 0x7f6c640bebc0 2024-03-09T08:52:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.084+0000 7f6c62000640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64072770 0x7f6c641eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.085+0000 7f6c63400640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c640bebc0 0x7f6c641ef320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400e880 tx=0x7f6c5400ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.085+0000 7f6c5aa00640 1 -- 172.21.15.67:0/952254649 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54004020 con 0x7f6c640bebc0 2024-03-09T08:52:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.086+0000 7f6c5aa00640 1 -- 172.21.15.67:0/952254649 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c540041e0 con 0x7f6c640bebc0 2024-03-09T08:52:38.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.086+0000 7f6c5aa00640 1 -- 172.21.15.67:0/952254649 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c540043d0 con 0x7f6c640bebc0 2024-03-09T08:52:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.086+0000 7f6c63e00640 1 -- 172.21.15.67:0/952254649 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c640c3390 con 0x7f6c640bebc0 2024-03-09T08:52:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.086+0000 7f6c63e00640 1 -- 172.21.15.67:0/952254649 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c640c3950 con 0x7f6c640bebc0 2024-03-09T08:52:38.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.089+0000 7f6c63e00640 1 -- 172.21.15.67:0/952254649 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c640689d0 con 0x7f6c640bebc0 2024-03-09T08:52:38.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.095+0000 7f6c5aa00640 1 -- 172.21.15.67:0/952254649 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6c54024020 con 0x7f6c640bebc0 2024-03-09T08:52:38.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.097+0000 7f6c5aa00640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c4006b5d0 0x7f6c4006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:38.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.098+0000 7f6c5aa00640 1 -- 172.21.15.67:0/952254649 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f6c54092c90 con 0x7f6c640bebc0 2024-03-09T08:52:38.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.102+0000 7f6c62a00640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c4006b5d0 0x7f6c4006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:38.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.103+0000 7f6c62a00640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c4006b5d0 0x7f6c4006da90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c035000 tx=0x7f6c5c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:38.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.105+0000 7f6c5aa00640 1 -- 172.21.15.67:0/952254649 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c640689d0 con 0x7f6c640bebc0 2024-03-09T08:52:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.392+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f90200709e0 con 0x7f902013d610 2024-03-09T08:52:38.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.393+0000 7f9016000640 1 -- 172.21.15.67:0/203503328 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f900c05c690 con 0x7f902013d610 2024-03-09T08:52:38.390 INFO:teuthology.orchestra.run.smithi067.stdout:55834574857 2024-03-09T08:52:38.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.397+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f900006b550 msgr2=0x7f900006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.397+0000 7f901f200640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f900006b550 0x7f900006da10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9020145fb0 tx=0x7f901002f000 comp rx=0 tx=0).stop 2024-03-09T08:52:38.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.397+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f902013d610 msgr2=0x7f9020145370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.397+0000 7f901f200640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f902013d610 0x7f9020145370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f900c000fb0 tx=0x7f900c00ec10 comp rx=0 tx=0).stop 2024-03-09T08:52:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.401+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 shutdown_connections 2024-03-09T08:52:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.401+0000 7f901f200640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f900006b550 0x7f900006da10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.401+0000 7f901f200640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9020072a10 0x7f90200ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.401+0000 7f901f200640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f902013d610 0x7f9020145370 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.401+0000 7f901f200640 1 --2- 172.21.15.67:0/203503328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9004003bb0 0x7f9020070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.401+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 >> 172.21.15.67:0/203503328 conn(0x7f90200c3c90 msgr2=0x7f9020150060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.405+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 shutdown_connections 2024-03-09T08:52:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.405+0000 7f889c600640 1 -- 172.21.15.67:0/471210809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8870005850 con 0x7f887c0052e0 2024-03-09T08:52:38.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.405+0000 7f901f200640 1 -- 172.21.15.67:0/203503328 wait complete. 2024-03-09T08:52:38.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.406+0000 7f8891600640 1 -- 172.21.15.67:0/471210809 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f888005bef0 con 0x7f887c0052e0 2024-03-09T08:52:38.404 INFO:teuthology.orchestra.run.smithi067.stdout:55834574855 2024-03-09T08:52:38.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.409+0000 7f889c600640 1 -- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f887406b700 msgr2=0x7f887406dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.409+0000 7f889c600640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f887406b700 0x7f887406dbc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f888c04a810 tx=0x7f8888009060 comp rx=0 tx=0).stop 2024-03-09T08:52:38.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.410+0000 7f889c600640 1 -- 172.21.15.67:0/471210809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 msgr2=0x7f888c0d5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.410+0000 7f889c600640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 0x7f888c0d5690 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f888000c7d0 tx=0x7f888000cc90 comp rx=0 tx=0).stop 2024-03-09T08:52:38.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.417+0000 7f889c600640 1 -- 172.21.15.67:0/471210809 shutdown_connections 2024-03-09T08:52:38.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.417+0000 7f889c600640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f887406b700 0x7f887406dbc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.417+0000 7f889c600640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f888c0c9570 0x7f888c049c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.417+0000 7f889c600640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f888c0ca310 0x7f888c0d6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.417+0000 7f889c600640 1 --2- 172.21.15.67:0/471210809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f887c0052e0 0x7f888c0d5690 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.417+0000 7f889c600640 1 -- 172.21.15.67:0/471210809 >> 172.21.15.67:0/471210809 conn(0x7f888c005e90 msgr2=0x7f888c0055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:38.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.417+0000 7f889c600640 1 -- 172.21.15.67:0/471210809 shutdown_connections 2024-03-09T08:52:38.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.418+0000 7f889c600640 1 -- 172.21.15.67:0/471210809 wait complete. 2024-03-09T08:52:38.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.3 2024-03-09T08:52:38.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.445+0000 7fb99cc00640 1 -- 172.21.15.67:0/3578618128 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb99804f3c0 con 0x7fb99815dc30 2024-03-09T08:52:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.446+0000 7fb98f400640 1 -- 172.21.15.67:0/3578618128 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb99000ae20 con 0x7fb99815dc30 2024-03-09T08:52:38.443 INFO:teuthology.orchestra.run.smithi067.stdout:55834574856 2024-03-09T08:52:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.450+0000 7fb99cc00640 1 -- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb97806b4d0 msgr2=0x7fb97806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.450+0000 7fb99cc00640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb97806b4d0 0x7fb97806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb988006fd0 tx=0x7fb988008040 comp rx=0 tx=0).stop 2024-03-09T08:52:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.450+0000 7fb99cc00640 1 -- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb99815dc30 msgr2=0x7fb9981566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.450+0000 7fb99cc00640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb99815dc30 0x7fb9981566b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb99000cad0 tx=0x7fb99000cf90 comp rx=0 tx=0).stop 2024-03-09T08:52:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.451+0000 7fb99cc00640 1 -- 172.21.15.67:0/3578618128 shutdown_connections 2024-03-09T08:52:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.451+0000 7fb99cc00640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb97806b4d0 0x7fb97806d990 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb988006fd0 tx=0x7fb988008040 comp rx=0 tx=0).stop 2024-03-09T08:52:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.451+0000 7fb99cc00640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb99815dc30 0x7fb9981566b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.451+0000 7fb99cc00640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb998153ed0 0x7fb998156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.451+0000 7fb99cc00640 1 --2- 172.21.15.67:0/3578618128 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9980c4630 0x7fb998147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.451+0000 7fb99cc00640 1 -- 172.21.15.67:0/3578618128 >> 172.21.15.67:0/3578618128 conn(0x7fb9980c8480 msgr2=0x7fb99815b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:38.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.451+0000 7fb99cc00640 1 -- 172.21.15.67:0/3578618128 shutdown_connections 2024-03-09T08:52:38.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.451+0000 7fb99cc00640 1 -- 172.21.15.67:0/3578618128 wait complete. 2024-03-09T08:52:38.449 INFO:teuthology.orchestra.run.smithi067.stdout:55834574856 2024-03-09T08:52:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.453+0000 7f66f7400640 1 -- 172.21.15.67:0/769594782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f66f00c3090 con 0x7f66d80055d0 2024-03-09T08:52:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.453+0000 7f66ecc00640 1 -- 172.21.15.67:0/769594782 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f66e4059810 con 0x7f66d80055d0 2024-03-09T08:52:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 -- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f66d006b520 msgr2=0x7f66d006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f66d006b520 0x7f66d006d9e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f66dc000fb0 tx=0x7f66dc007040 comp rx=0 tx=0).stop 2024-03-09T08:52:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 -- 172.21.15.67:0/769594782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 msgr2=0x7f66f014b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 0x7f66f014b1e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009fd0 tx=0x7f66e400b8e0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 -- 172.21.15.67:0/769594782 shutdown_connections 2024-03-09T08:52:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f66d006b520 0x7f66d006d9e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f0144b30 0x7f66f01e6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66f0154800 0x7f66f01445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 --2- 172.21.15.67:0/769594782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d80055d0 0x7f66f014b1e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 -- 172.21.15.67:0/769594782 >> 172.21.15.67:0/769594782 conn(0x7f66f00c7370 msgr2=0x7f66f006d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:38.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 -- 172.21.15.67:0/769594782 shutdown_connections 2024-03-09T08:52:38.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.457+0000 7f66e2000640 1 -- 172.21.15.67:0/769594782 wait complete. 2024-03-09T08:52:38.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.5 2024-03-09T08:52:38.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.1 2024-03-09T08:52:38.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.4 2024-03-09T08:52:38.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.480+0000 7f7a03000640 1 -- 172.21.15.67:0/753855689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f79fc075a00 con 0x7f79e00053c0 2024-03-09T08:52:38.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.481+0000 7f79f3e00640 1 -- 172.21.15.67:0/753855689 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f79e801f020 con 0x7f79e00053c0 2024-03-09T08:52:38.477 INFO:teuthology.orchestra.run.smithi067.stdout:55834574857 2024-03-09T08:52:38.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 -- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79dc06b500 msgr2=0x7f79dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79dc06b500 0x7f79dc06d9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f79fc161ac0 tx=0x7f79f40094f0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 -- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00053c0 msgr2=0x7f79fc152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00053c0 0x7f79fc152010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f79e800b470 tx=0x7f79e800b930 comp rx=0 tx=0).stop 2024-03-09T08:52:38.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 -- 172.21.15.67:0/753855689 shutdown_connections 2024-03-09T08:52:38.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f79dc06b500 0x7f79dc06d9c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f79fc152b20 0x7f79fc152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79fc142880 0x7f79fc142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 --2- 172.21.15.67:0/753855689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f79e00053c0 0x7f79fc152010 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 -- 172.21.15.67:0/753855689 >> 172.21.15.67:0/753855689 conn(0x7f79fc0aa890 msgr2=0x7f79fc0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:38.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 -- 172.21.15.67:0/753855689 shutdown_connections 2024-03-09T08:52:38.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.484+0000 7f7a03000640 1 -- 172.21.15.67:0/753855689 wait complete. 2024-03-09T08:52:38.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.0 2024-03-09T08:52:38.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.511+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8580075a00 con 0x7f8580157820 2024-03-09T08:52:38.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.512+0000 7f857cc00640 1 -- 172.21.15.67:0/1479029705 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8578010070 con 0x7f8580157820 2024-03-09T08:52:38.507 INFO:teuthology.orchestra.run.smithi067.stdout:55834574856 2024-03-09T08:52:38.508 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:52:38.512 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:52:38.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f856006b520 msgr2=0x7f856006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f856006b520 0x7f856006d9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8580068330 tx=0x7f8570009040 comp rx=0 tx=0).stop 2024-03-09T08:52:38.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580157820 msgr2=0x7f8580153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580157820 0x7f8580153f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8578007d80 tx=0x7f857800d4a0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 shutdown_connections 2024-03-09T08:52:38.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f856006b520 0x7f856006d9e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8580157820 0x7f8580153f00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85800c1d70 0x7f85801539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 --2- 172.21.15.67:0/1479029705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85800aad50 0x7f858015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 >> 172.21.15.67:0/1479029705 conn(0x7f85800c7370 msgr2=0x7f8580156ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 shutdown_connections 2024-03-09T08:52:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7f8587800640 1 -- 172.21.15.67:0/1479029705 wait complete. 2024-03-09T08:52:38.514 INFO:teuthology.orchestra.run.smithi067.stdout:55834574855 2024-03-09T08:52:38.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.515+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe51c04f3c0 con 0x7fe51c153fc0 2024-03-09T08:52:38.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.516+0000 7fe519600640 1 -- 172.21.15.67:0/1690228817 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe51005f030 con 0x7fe51c153fc0 2024-03-09T08:52:38.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.518+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4fc06b4d0 msgr2=0x7fe4fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.518+0000 7fe523000640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4fc06b4d0 0x7fe4fc06d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe508006fd0 tx=0x7fe508008040 comp rx=0 tx=0).stop 2024-03-09T08:52:38.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.519+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe51c153fc0 msgr2=0x7fe51c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.519+0000 7fe523000640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe51c153fc0 0x7fe51c06d410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe51000e850 tx=0x7fe51000ed10 comp rx=0 tx=0).stop 2024-03-09T08:52:38.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.519+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 shutdown_connections 2024-03-09T08:52:38.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.519+0000 7fe523000640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4fc06b4d0 0x7fe4fc06d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.519+0000 7fe523000640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe51c15aa40 0x7fe51c1ec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.519+0000 7fe523000640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe51c153fc0 0x7fe51c06d410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.519+0000 7fe523000640 1 --2- 172.21.15.67:0/1690228817 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe51c0c3810 0x7fe51c1efe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.519+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 >> 172.21.15.67:0/1690228817 conn(0x7fe51c0c9470 msgr2=0x7fe51c073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:38.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.519+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 shutdown_connections 2024-03-09T08:52:38.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.519+0000 7fe523000640 1 -- 172.21.15.67:0/1690228817 wait complete. 2024-03-09T08:52:38.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.2 2024-03-09T08:52:38.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.7 2024-03-09T08:52:38.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.623+0000 7f6c63e00640 1 -- 172.21.15.67:0/952254649 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6c6404f3c0 con 0x7f6c640bebc0 2024-03-09T08:52:38.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.623+0000 7f6c5aa00640 1 -- 172.21.15.67:0/952254649 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c5402b020 con 0x7f6c640bebc0 2024-03-09T08:52:38.619 INFO:teuthology.orchestra.run.smithi067.stdout:55834574857 2024-03-09T08:52:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 -- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c4006b5d0 msgr2=0x7f6c4006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c4006b5d0 0x7f6c4006da90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c035000 tx=0x7f6c5c030000 comp rx=0 tx=0).stop 2024-03-09T08:52:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 -- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c640bebc0 msgr2=0x7f6c641ef320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c640bebc0 0x7f6c641ef320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400e880 tx=0x7f6c5400ed40 comp rx=0 tx=0).stop 2024-03-09T08:52:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 -- 172.21.15.67:0/952254649 shutdown_connections 2024-03-09T08:52:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c4006b5d0 0x7f6c4006da90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c640bebc0 0x7f6c641ef320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64072770 0x7f6c641eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 --2- 172.21.15.67:0/952254649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c6406bda0 0x7f6c640c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 -- 172.21.15.67:0/952254649 >> 172.21.15.67:0/952254649 conn(0x7f6c640c7fb0 msgr2=0x7f6c640ab5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 -- 172.21.15.67:0/952254649 shutdown_connections 2024-03-09T08:52:38.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:38.626+0000 7f6c63e00640 1 -- 172.21.15.67:0/952254649 wait complete. 2024-03-09T08:52:38.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.6 2024-03-09T08:52:39.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-09T08:52:39.455 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-09T08:52:39.471 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-09T08:52:39.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-09T08:52:39.499 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-09T08:52:39.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-09T08:52:39.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-09T08:52:39.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-09T08:52:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.751+0000 7f7d47200640 1 Processor -- start 2024-03-09T08:52:39.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.751+0000 7f7d47200640 1 -- start start 2024-03-09T08:52:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.753+0000 7f7d47200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d480c3d20 0x7f7d481526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.753+0000 7f7d47200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d480abb30 0x7f7d4806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.753+0000 7f7d47200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d48152c10 con 0x7f7d480aae00 2024-03-09T08:52:39.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.753+0000 7f7d47200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d480aa3b0 con 0x7f7d480abb30 2024-03-09T08:52:39.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.753+0000 7f7d46800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7d480aae00 0x7f7d480ab290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53796/0 (socket says 172.21.15.67:53796) 2024-03-09T08:52:39.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.753+0000 7f7d46800640 1 -- 172.21.15.67:0/1249058854 learned_addr learned my addr 172.21.15.67:0/1249058854 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:39.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.754+0000 7f7d47200640 1 -- 172.21.15.67:0/1249058854 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480c1c20 con 0x7f7d480c3d20 2024-03-09T08:52:39.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.754+0000 7f7d45400640 1 --2- 172.21.15.67:0/1249058854 >> v2:172.21.15.67:3300/0 conn(0x7f7d480abb30 0x7f7d4806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.754+0000 7f7d45e00640 1 --2- 172.21.15.67:0/1249058854 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d480c3d20 0x7f7d481526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.755+0000 7f7d45400640 1 -- 172.21.15.67:0/1249058854 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d480c3d20 msgr2=0x7f7d481526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.755+0000 7f7d45400640 1 --2- 172.21.15.67:0/1249058854 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d480c3d20 0x7f7d481526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.755+0000 7f7d45400640 1 -- 172.21.15.67:0/1249058854 >> v1:172.21.15.67:6789/0 conn(0x7f7d480aae00 legacy=0x7f7d480ab290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.755+0000 7f7d45400640 1 -- 172.21.15.67:0/1249058854 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d480ac0f0 con 0x7f7d480abb30 2024-03-09T08:52:39.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.757+0000 7f7d45400640 1 --2- 172.21.15.67:0/1249058854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d480abb30 0x7f7d4806d5b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000b050 tx=0x7f7d4002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a019f8e592f53e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.758+0000 7f7d44a00640 1 -- 172.21.15.67:0/1249058854 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40039070 con 0x7f7d480abb30 2024-03-09T08:52:39.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.758+0000 7f7d44a00640 1 -- 172.21.15.67:0/1249058854 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4002c9d0 con 0x7f7d480abb30 2024-03-09T08:52:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.758+0000 7f7d44a00640 1 -- 172.21.15.67:0/1249058854 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4002cda0 con 0x7f7d480abb30 2024-03-09T08:52:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.758+0000 7f3802800640 1 Processor -- start 2024-03-09T08:52:39.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.758+0000 7f3802800640 1 -- start start 2024-03-09T08:52:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.759+0000 7f7d47200640 1 -- 172.21.15.67:0/1249058854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d480abb30 msgr2=0x7f7d4806d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.759+0000 7f7d47200640 1 --2- 172.21.15.67:0/1249058854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d480abb30 0x7f7d4806d5b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000b050 tx=0x7f7d4002beb0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.759+0000 7f7d47200640 1 -- 172.21.15.67:0/1249058854 shutdown_connections 2024-03-09T08:52:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.759+0000 7f7d47200640 1 --2- 172.21.15.67:0/1249058854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d480abb30 0x7f7d4806d5b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.759+0000 7f7d47200640 1 --2- 172.21.15.67:0/1249058854 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d480c3d20 0x7f7d481526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.759+0000 7f7d47200640 1 -- 172.21.15.67:0/1249058854 >> 172.21.15.67:0/1249058854 conn(0x7f7d4806b490 msgr2=0x7f7d4806b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.759+0000 7f7d47200640 1 -- 172.21.15.67:0/1249058854 shutdown_connections 2024-03-09T08:52:39.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.759+0000 7f7d47200640 1 -- 172.21.15.67:0/1249058854 wait complete. 2024-03-09T08:52:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.760+0000 7f3802800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f37fc0c3240 0x7f37fc155330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.760+0000 7f3802800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc152880 con 0x7f37fc14f2a0 2024-03-09T08:52:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.760+0000 7f3802800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc152b60 con 0x7f37fc0a9d30 2024-03-09T08:52:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.760+0000 7f3802800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc0abaf0 con 0x7f37fc0c3240 2024-03-09T08:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.761+0000 7f37faa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f37fc0c3240 0x7f37fc155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.761+0000 7f37faa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f37fc0c3240 0x7f37fc155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45562/0 (socket says 172.21.15.67:45562) 2024-03-09T08:52:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.761+0000 7f37faa00640 1 -- 172.21.15.67:0/1561067422 learned_addr learned my addr 172.21.15.67:0/1561067422 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.762+0000 7f37fa000640 1 -- 172.21.15.67:0/1561067422 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971184529 0 0) 0x7f37fc152b60 con 0x7f37fc0a9d30 2024-03-09T08:52:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.762+0000 7f37fa000640 1 -- 172.21.15.67:0/1561067422 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e0003520 con 0x7f37fc0a9d30 2024-03-09T08:52:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.762+0000 7f37faa00640 1 -- 172.21.15.67:0/1561067422 >> v1:172.21.15.134:6789/0 conn(0x7f37fc0a9d30 legacy=0x7f37fc15c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.762+0000 7f37faa00640 1 -- 172.21.15.67:0/1561067422 >> v1:172.21.15.67:6789/0 conn(0x7f37fc14f2a0 legacy=0x7f37fc06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.762+0000 7f37faa00640 1 -- 172.21.15.67:0/1561067422 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc0abd40 con 0x7f37fc0c3240 2024-03-09T08:52:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.762+0000 7f7d47200640 1 Processor -- start 2024-03-09T08:52:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.762+0000 7f7d47200640 1 -- start start 2024-03-09T08:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.764+0000 7f7d47200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d480aae00 0x7f7d4813ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.764+0000 7f7d47200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d480abb30 0x7f7d48070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.764+0000 7f7d47200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d480c3d20 0x7f7d481efc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.764+0000 7f37faa00640 1 --2- 172.21.15.67:0/1561067422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37fc0c3240 0x7f37fc155330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f37e8008870 tx=0x7f37e8030130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2782626b240fed0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.764+0000 7f7d47200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480aa8e0 con 0x7f7d480c3d20 2024-03-09T08:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.764+0000 7f7d47200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480aa600 con 0x7f7d480abb30 2024-03-09T08:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.764+0000 7f7d47200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4806daf0 con 0x7f7d480aae00 2024-03-09T08:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.765+0000 7f7d46800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d480c3d20 0x7f7d481efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.765+0000 7f37fa000640 1 -- 172.21.15.67:0/1561067422 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8036070 con 0x7f37fc0c3240 2024-03-09T08:52:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.765+0000 7f7d46800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d480c3d20 0x7f7d481efc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46484/0 (socket says 172.21.15.67:46484) 2024-03-09T08:52:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.765+0000 7f7d46800640 1 -- 172.21.15.67:0/885811890 learned_addr learned my addr 172.21.15.67:0/885811890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.765+0000 7f7d45400640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d480abb30 0x7f7d48070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.765+0000 7f37fa000640 1 -- 172.21.15.67:0/1561067422 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e803a400 con 0x7f37fc0c3240 2024-03-09T08:52:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.765+0000 7f7d45e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d480aae00 0x7f7d4813ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.765+0000 7f37fa000640 1 -- 172.21.15.67:0/1561067422 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e803a7b0 con 0x7f37fc0c3240 2024-03-09T08:52:39.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.765+0000 7f7d46800640 1 -- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d480aae00 msgr2=0x7f7d4813ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.766+0000 7f7d46800640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d480aae00 0x7f7d4813ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.766+0000 7f3802800640 1 -- 172.21.15.67:0/1561067422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37fc0c3240 msgr2=0x7f37fc155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.766+0000 7f3802800640 1 --2- 172.21.15.67:0/1561067422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37fc0c3240 0x7f37fc155330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f37e8008870 tx=0x7f37e8030130 comp rx=0 tx=0).stop 2024-03-09T08:52:39.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.766+0000 7f7d46800640 1 -- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d480abb30 msgr2=0x7f7d48070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.766+0000 7f7d46800640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d480abb30 0x7f7d48070bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.766+0000 7f7d46800640 1 -- 172.21.15.67:0/885811890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4806d740 con 0x7f7d480c3d20 2024-03-09T08:52:39.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.766+0000 7f3802800640 1 -- 172.21.15.67:0/1561067422 shutdown_connections 2024-03-09T08:52:39.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.766+0000 7f3802800640 1 --2- 172.21.15.67:0/1561067422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37fc0c3240 0x7f37fc155330 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.766+0000 7f3802800640 1 -- 172.21.15.67:0/1561067422 >> 172.21.15.67:0/1561067422 conn(0x7f37fc0c7370 msgr2=0x7f37fc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:39.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.766+0000 7f7d46800640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d480c3d20 0x7f7d481efc80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400c520 tx=0x7f7d3400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.767+0000 7f7d3e000640 1 -- 172.21.15.67:0/885811890 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34018070 con 0x7f7d480c3d20 2024-03-09T08:52:39.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.767+0000 7f7d3e000640 1 -- 172.21.15.67:0/885811890 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d340074e0 con 0x7f7d480c3d20 2024-03-09T08:52:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.767+0000 7f7d3e000640 1 -- 172.21.15.67:0/885811890 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34004a00 con 0x7f7d480c3d20 2024-03-09T08:52:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.768+0000 7f7d47200640 1 -- 172.21.15.67:0/885811890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4813bf70 con 0x7f7d480c3d20 2024-03-09T08:52:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.768+0000 7f7d47200640 1 -- 172.21.15.67:0/885811890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4813c530 con 0x7f7d480c3d20 2024-03-09T08:52:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.767+0000 7f3802800640 1 -- 172.21.15.67:0/1561067422 shutdown_connections 2024-03-09T08:52:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.767+0000 7f3802800640 1 -- 172.21.15.67:0/1561067422 wait complete. 2024-03-09T08:52:39.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.769+0000 7f3802800640 1 Processor -- start 2024-03-09T08:52:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.769+0000 7f3802800640 1 -- start start 2024-03-09T08:52:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.771+0000 7f3802800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37fc0a9d30 0x7f37fc1f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.771+0000 7f3802800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37fc0c3240 0x7f37fc14bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.771+0000 7f3802800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37fc14f2a0 0x7f37fc0c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.772+0000 7f37fb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37fc0a9d30 0x7f37fc1f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.772+0000 7f37fb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37fc0a9d30 0x7f37fc1f0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46486/0 (socket says 172.21.15.67:46486) 2024-03-09T08:52:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.773+0000 7f37fb400640 1 -- 172.21.15.67:0/1840971716 learned_addr learned my addr 172.21.15.67:0/1840971716 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.773+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc14f680 con 0x7f37fc0a9d30 2024-03-09T08:52:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.773+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0ac0b0 con 0x7f37fc14f2a0 2024-03-09T08:52:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.773+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc14f800 con 0x7f37fc0c3240 2024-03-09T08:52:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.772+0000 7f7d3e000640 1 -- 172.21.15.67:0/885811890 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7d3401b070 con 0x7f7d480c3d20 2024-03-09T08:52:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.773+0000 7f7d3e000640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d2806b5a0 0x7f7d2806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.774+0000 7f7d3e000640 1 -- 172.21.15.67:0/885811890 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f7d340899f0 con 0x7f7d480c3d20 2024-03-09T08:52:39.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.774+0000 7f37fb400640 1 -- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37fc0c3240 msgr2=0x7f37fc14bbf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T08:52:39.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.774+0000 7f37fb400640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37fc0c3240 0x7f37fc14bbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.775+0000 7f37fbe00640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37fc14f2a0 0x7f37fc0c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.774+0000 7f7d45e00640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d2806b5a0 0x7f7d2806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.775+0000 7f37fb400640 1 -- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37fc14f2a0 msgr2=0x7f37fc0c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.775+0000 7f37fb400640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37fc14f2a0 0x7f37fc0c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.776+0000 7f37fb400640 1 -- 172.21.15.67:0/1840971716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc149670 con 0x7f37fc0a9d30 2024-03-09T08:52:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.776+0000 7f37fbe00640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37fc14f2a0 0x7f37fc0c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.776+0000 7f7d47200640 1 -- 172.21.15.67:0/885811890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d20005290 con 0x7f7d480c3d20 2024-03-09T08:52:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.777+0000 7f37fb400640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37fc0a9d30 0x7f37fc1f0f10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c3f0 tx=0x7f37ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.778+0000 7f37f3400640 1 -- 172.21.15.67:0/1840971716 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec0052d0 con 0x7f37fc0a9d30 2024-03-09T08:52:39.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.781+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc149950 con 0x7f37fc0a9d30 2024-03-09T08:52:39.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.781+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc1455c0 con 0x7f37fc0a9d30 2024-03-09T08:52:39.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.781+0000 7f7d45e00640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d2806b5a0 0x7f7d2806da60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7d38006fd0 tx=0x7f7d38008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.781+0000 7f37f3400640 1 -- 172.21.15.67:0/1840971716 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec005470 con 0x7f37fc0a9d30 2024-03-09T08:52:39.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.781+0000 7f37f3400640 1 -- 172.21.15.67:0/1840971716 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec0052d0 con 0x7f37fc0a9d30 2024-03-09T08:52:39.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.782+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d4005290 con 0x7f37fc0a9d30 2024-03-09T08:52:39.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.783+0000 7fb438c00640 1 Processor -- start 2024-03-09T08:52:39.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.783+0000 7fb438c00640 1 -- start start 2024-03-09T08:52:39.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.785+0000 7fb438c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c0a10 0x7fb4341567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.785+0000 7fb438c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4340c1d70 0x7fb43414c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.785+0000 7fb438c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4340aa420 con 0x7fb43414c9b0 2024-03-09T08:52:39.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.785+0000 7fb438c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb434156cf0 con 0x7fb4340c1d70 2024-03-09T08:52:39.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.785+0000 7fb438c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43406d590 con 0x7fb4340c0a10 2024-03-09T08:52:39.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.786+0000 7fb431e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c0a10 0x7fb4341567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.787+0000 7fb431400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4340c1d70 0x7fb43414c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.787+0000 7fb431400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4340c1d70 0x7fb43414c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46496/0 (socket says 172.21.15.67:46496) 2024-03-09T08:52:39.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.787+0000 7fb431e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c0a10 0x7fb4341567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:39360/0 (socket says 172.21.15.67:39360) 2024-03-09T08:52:39.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.787+0000 7fb431e00640 1 -- 172.21.15.67:0/3385109650 learned_addr learned my addr 172.21.15.67:0/3385109650 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.787+0000 7fb432800640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb43414c9b0 0x7fb4340ac910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:58574/0 (socket says 172.21.15.67:58574) 2024-03-09T08:52:39.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.788+0000 7fb431e00640 1 -- 172.21.15.67:0/3385109650 >> v2:172.21.15.67:3300/0 conn(0x7fb4340c1d70 msgr2=0x7fb43414c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:39.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.788+0000 7fb431e00640 1 --2- 172.21.15.67:0/3385109650 >> v2:172.21.15.67:3300/0 conn(0x7fb4340c1d70 0x7fb43414c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.788+0000 7fb431e00640 1 -- 172.21.15.67:0/3385109650 >> v1:172.21.15.134:6789/0 conn(0x7fb43414c9b0 legacy=0x7fb4340ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.788+0000 7fb431e00640 1 -- 172.21.15.67:0/3385109650 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb434073e00 con 0x7fb4340c0a10 2024-03-09T08:52:39.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.793+0000 7f7d3e000640 1 -- 172.21.15.67:0/885811890 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7d340585b0 con 0x7f7d480c3d20 2024-03-09T08:52:39.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.793+0000 7fb431e00640 1 --2- 172.21.15.67:0/3385109650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c0a10 0x7fb4341567b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb420009870 tx=0x7fb42002b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=111733304fe6f0b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.793+0000 7f37f3400640 1 -- 172.21.15.67:0/1840971716 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f37ec0035f0 con 0x7f37fc0a9d30 2024-03-09T08:52:39.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.795+0000 7fb430a00640 1 -- 172.21.15.67:0/3385109650 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420034070 con 0x7fb4340c0a10 2024-03-09T08:52:39.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.795+0000 7fb430a00640 1 -- 172.21.15.67:0/3385109650 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb420030860 con 0x7fb4340c0a10 2024-03-09T08:52:39.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.795+0000 7fb182c00640 1 Processor -- start 2024-03-09T08:52:39.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.795+0000 7fb430a00640 1 -- 172.21.15.67:0/3385109650 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420030c70 con 0x7fb4340c0a10 2024-03-09T08:52:39.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.796+0000 7fb438c00640 1 -- 172.21.15.67:0/3385109650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c0a10 msgr2=0x7fb4341567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.796+0000 7fb438c00640 1 --2- 172.21.15.67:0/3385109650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c0a10 0x7fb4341567b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb420009870 tx=0x7fb42002b9a0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.797+0000 7fb438c00640 1 -- 172.21.15.67:0/3385109650 shutdown_connections 2024-03-09T08:52:39.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.797+0000 7fb438c00640 1 --2- 172.21.15.67:0/3385109650 >> v2:172.21.15.67:3300/0 conn(0x7fb4340c1d70 0x7fb43414c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.797+0000 7fb438c00640 1 --2- 172.21.15.67:0/3385109650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c0a10 0x7fb4341567b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.797+0000 7fb438c00640 1 -- 172.21.15.67:0/3385109650 >> 172.21.15.67:0/3385109650 conn(0x7fb4340c7370 msgr2=0x7fb4340c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:39.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.797+0000 7fb438c00640 1 -- 172.21.15.67:0/3385109650 shutdown_connections 2024-03-09T08:52:39.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.794+0000 7f37f3400640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f37dc06b8d0 0x7f37dc06dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.794+0000 7f37f3400640 1 -- 172.21.15.67:0/1840971716 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f37ec0050d0 con 0x7f37fc0a9d30 2024-03-09T08:52:39.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.798+0000 7fb438c00640 1 -- 172.21.15.67:0/3385109650 wait complete. 2024-03-09T08:52:39.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.798+0000 7f37faa00640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f37dc06b8d0 0x7f37dc06dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.801+0000 7fb438c00640 1 Processor -- start 2024-03-09T08:52:39.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.801+0000 7fb438c00640 1 -- start start 2024-03-09T08:52:39.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.803+0000 7f37faa00640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f37dc06b8d0 0x7f37dc06dd90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f37e803b000 tx=0x7f37e8036000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.803+0000 7fb438c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4340c0a10 0x7fb434159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.803+0000 7fb438c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c1d70 0x7fb4341599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.804+0000 7fb438c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb43414c9b0 0x7fb4341ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.804+0000 7fb438c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340aa9c0 con 0x7fb4340c0a10 2024-03-09T08:52:39.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.804+0000 7fb438c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340aaca0 con 0x7fb43414c9b0 2024-03-09T08:52:39.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.804+0000 7fb438c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4340c0df0 con 0x7fb4340c1d70 2024-03-09T08:52:39.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.804+0000 7fb431e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4340c0a10 0x7fb434159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.804+0000 7fb431400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c1d70 0x7fb4341599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.804+0000 7fb431e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4340c0a10 0x7fb434159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46502/0 (socket says 172.21.15.67:46502) 2024-03-09T08:52:39.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.804+0000 7fb431e00640 1 -- 172.21.15.67:0/2572068958 learned_addr learned my addr 172.21.15.67:0/2572068958 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.804+0000 7f37f3400640 1 -- 172.21.15.67:0/1840971716 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f37ec0583c0 con 0x7f37fc0a9d30 2024-03-09T08:52:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.804+0000 7fb432800640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb43414c9b0 0x7fb4341ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.805+0000 7fb432800640 1 -- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c1d70 msgr2=0x7fb4341599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.805+0000 7fb432800640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c1d70 0x7fb4341599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.805+0000 7fb432800640 1 -- 172.21.15.67:0/2572068958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4340c0a10 msgr2=0x7fb434159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.805+0000 7fb432800640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4340c0a10 0x7fb434159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.805+0000 7fb432800640 1 -- 172.21.15.67:0/2572068958 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb434151570 con 0x7fb43414c9b0 2024-03-09T08:52:39.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.806+0000 7fb432800640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb43414c9b0 0x7fb4341ef020 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0012e0 tx=0x7fb42c00cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.806+0000 7fb42a000640 1 -- 172.21.15.67:0/2572068958 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c018070 con 0x7fb43414c9b0 2024-03-09T08:52:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.806+0000 7fb438c00640 1 -- 172.21.15.67:0/2572068958 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43414d3c0 con 0x7fb43414c9b0 2024-03-09T08:52:39.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.806+0000 7fb438c00640 1 -- 172.21.15.67:0/2572068958 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43414d980 con 0x7fb43414c9b0 2024-03-09T08:52:39.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.796+0000 7fb182c00640 1 -- start start 2024-03-09T08:52:39.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.797+0000 7fb182c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb17c0bebc0 0x7fb17c072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.797+0000 7fb182c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c06cd30 con 0x7fb17c06bda0 2024-03-09T08:52:39.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.797+0000 7fb182c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17c0c5eb0 con 0x7fb17c072770 2024-03-09T08:52:39.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.797+0000 7fb182c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c06b200 con 0x7fb17c0bebc0 2024-03-09T08:52:39.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.798+0000 7fb180e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb17c06bda0 0x7fb17c15e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53816/0 (socket says 172.21.15.67:53816) 2024-03-09T08:52:39.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.798+0000 7fb180e00640 1 -- 172.21.15.67:0/1427656401 learned_addr learned my addr 172.21.15.67:0/1427656401 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:39.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.800+0000 7fb182200640 1 --2- 172.21.15.67:0/1427656401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb17c0bebc0 0x7fb17c072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.801+0000 7fb182200640 1 -- 172.21.15.67:0/1427656401 >> v1:172.21.15.134:6789/0 conn(0x7fb17c072770 legacy=0x7fb17c06b690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:52:39.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.801+0000 7fb182200640 1 -- 172.21.15.67:0/1427656401 >> v1:172.21.15.67:6789/0 conn(0x7fb17c06bda0 legacy=0x7fb17c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.801+0000 7fb182200640 1 -- 172.21.15.67:0/1427656401 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0c6150 con 0x7fb17c0bebc0 2024-03-09T08:52:39.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.804+0000 7fb182200640 1 --2- 172.21.15.67:0/1427656401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb17c0bebc0 0x7fb17c072b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb174009dc0 tx=0x7fb17402bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16407a116d1a9609 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.806+0000 7fb17be00640 1 -- 172.21.15.67:0/1427656401 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174039070 con 0x7fb17c0bebc0 2024-03-09T08:52:39.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.806+0000 7fb17be00640 1 -- 172.21.15.67:0/1427656401 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17402cb30 con 0x7fb17c0bebc0 2024-03-09T08:52:39.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.806+0000 7fb17be00640 1 -- 172.21.15.67:0/1427656401 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174034440 con 0x7fb17c0bebc0 2024-03-09T08:52:39.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.807+0000 7fb42a000640 1 -- 172.21.15.67:0/2572068958 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c004d60 con 0x7fb43414c9b0 2024-03-09T08:52:39.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.807+0000 7fb42a000640 1 -- 172.21.15.67:0/2572068958 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00a880 con 0x7fb43414c9b0 2024-03-09T08:52:39.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.810+0000 7fb41f400640 1 -- 172.21.15.67:0/2572068958 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb434075a00 con 0x7fb43414c9b0 2024-03-09T08:52:39.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.815+0000 7fb42a000640 1 -- 172.21.15.67:0/2572068958 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb42c01b070 con 0x7fb43414c9b0 2024-03-09T08:52:39.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.815+0000 7fb182c00640 1 -- 172.21.15.67:0/1427656401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb17c0bebc0 msgr2=0x7fb17c072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.815+0000 7fb182c00640 1 --2- 172.21.15.67:0/1427656401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb17c0bebc0 0x7fb17c072b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb174009dc0 tx=0x7fb17402bd80 comp rx=0 tx=0).stop 2024-03-09T08:52:39.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.815+0000 7fb182c00640 1 -- 172.21.15.67:0/1427656401 shutdown_connections 2024-03-09T08:52:39.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.815+0000 7fb182c00640 1 --2- 172.21.15.67:0/1427656401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb17c0bebc0 0x7fb17c072b50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.815+0000 7fb182c00640 1 -- 172.21.15.67:0/1427656401 >> 172.21.15.67:0/1427656401 conn(0x7fb17c0c7fb0 msgr2=0x7fb17c0c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:39.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.817+0000 7fb42a000640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb41406b4d0 0x7fb41406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.817+0000 7fb42a000640 1 -- 172.21.15.67:0/2572068958 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb42c089b50 con 0x7fb43414c9b0 2024-03-09T08:52:39.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.817+0000 7fb182c00640 1 -- 172.21.15.67:0/1427656401 shutdown_connections 2024-03-09T08:52:39.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.819+0000 7fb182c00640 1 -- 172.21.15.67:0/1427656401 wait complete. 2024-03-09T08:52:39.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.820+0000 7fb182c00640 1 Processor -- start 2024-03-09T08:52:39.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.821+0000 7fb182c00640 1 -- start start 2024-03-09T08:52:39.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.822+0000 7fb431e00640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb41406b4d0 0x7fb41406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.822+0000 7fb182c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb17c06bda0 0x7fb17c150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.822+0000 7fb182c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb17c072770 0x7fb17c153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.822+0000 7fb182c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c0bebc0 0x7fb17c155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.822+0000 7fb182c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0c9430 con 0x7fb17c06bda0 2024-03-09T08:52:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.822+0000 7fb182c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c0c6710 con 0x7fb17c0bebc0 2024-03-09T08:52:39.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.822+0000 7fb182c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c1501b0 con 0x7fb17c072770 2024-03-09T08:52:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.823+0000 7fb182200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c0bebc0 0x7fb17c155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.823+0000 7fb182200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c0bebc0 0x7fb17c155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45580/0 (socket says 172.21.15.67:45580) 2024-03-09T08:52:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.823+0000 7fb182200640 1 -- 172.21.15.67:0/433368270 learned_addr learned my addr 172.21.15.67:0/433368270 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.824+0000 7fb181800640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb17c06bda0 0x7fb17c150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.823+0000 7fb431e00640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb41406b4d0 0x7fb41406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb420000c00 tx=0x7fb42003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.824+0000 7fb180e00640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb17c072770 0x7fb17c153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.824+0000 7fb182200640 1 -- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb17c072770 msgr2=0x7fb17c153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.824+0000 7fb182200640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb17c072770 0x7fb17c153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.824+0000 7fb182200640 1 -- 172.21.15.67:0/433368270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb17c06bda0 msgr2=0x7fb17c150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.824+0000 7fb182200640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb17c06bda0 0x7fb17c150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.824+0000 7fb182200640 1 -- 172.21.15.67:0/433368270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c151140 con 0x7fb17c0bebc0 2024-03-09T08:52:39.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.826+0000 7fb182200640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c0bebc0 0x7fb17c155600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb17402c390 tx=0x7fb174004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.827+0000 7fb179600640 1 -- 172.21.15.67:0/433368270 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174044020 con 0x7fb17c0bebc0 2024-03-09T08:52:39.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.827+0000 7fb179600640 1 -- 172.21.15.67:0/433368270 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17403a040 con 0x7fb17c0bebc0 2024-03-09T08:52:39.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.827+0000 7fb179600640 1 -- 172.21.15.67:0/433368270 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174039070 con 0x7fb17c0bebc0 2024-03-09T08:52:39.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.827+0000 7fb42a000640 1 -- 172.21.15.67:0/2572068958 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb42c0580c0 con 0x7fb43414c9b0 2024-03-09T08:52:39.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.829+0000 7fb182c00640 1 -- 172.21.15.67:0/433368270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c14fb60 con 0x7fb17c0bebc0 2024-03-09T08:52:39.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.829+0000 7fb182c00640 1 -- 172.21.15.67:0/433368270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c0afa00 con 0x7fb17c0bebc0 2024-03-09T08:52:39.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.831+0000 7fb182c00640 1 -- 172.21.15.67:0/433368270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17c04f3c0 con 0x7fb17c0bebc0 2024-03-09T08:52:39.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.836+0000 7fb179600640 1 -- 172.21.15.67:0/433368270 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb174047020 con 0x7fb17c0bebc0 2024-03-09T08:52:39.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.837+0000 7fb179600640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb15c06b5a0 0x7fb15c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.837+0000 7fb179600640 1 -- 172.21.15.67:0/433368270 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb1740adeb0 con 0x7fb17c0bebc0 2024-03-09T08:52:39.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.839+0000 7f9ecfe00640 1 Processor -- start 2024-03-09T08:52:39.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.839+0000 7f9ecfe00640 1 -- start start 2024-03-09T08:52:39.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.841+0000 7f9ecfe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9ed00abb30 0x7f9ed014efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.841+0000 7fb181800640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb15c06b5a0 0x7fb15c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.842+0000 7f9ecfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ed014f520 0x7f9ed00aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.842+0000 7f9ecfe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed00c1c20 con 0x7f9ed006d5b0 2024-03-09T08:52:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.842+0000 7f9ecfe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed00aa3b0 con 0x7f9ed014f520 2024-03-09T08:52:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.842+0000 7f9ecfe00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed00ac0f0 con 0x7f9ed00abb30 2024-03-09T08:52:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.842+0000 7f9ece000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9ed00abb30 0x7f9ed014efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.842+0000 7f9ecea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9ed006d5b0 0x7f9ed01526d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53826/0 (socket says 172.21.15.67:53826) 2024-03-09T08:52:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.842+0000 7f9ecea00640 1 -- 172.21.15.67:0/1502168192 learned_addr learned my addr 172.21.15.67:0/1502168192 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.842+0000 7f9ecf400640 1 --2- 172.21.15.67:0/1502168192 >> v2:172.21.15.67:3300/0 conn(0x7f9ed014f520 0x7f9ed00aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.844+0000 7f9ecf400640 1 -- 172.21.15.67:0/1502168192 >> v2:172.21.15.134:3300/0 conn(0x7f9ed00abb30 msgr2=0x7f9ed014efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.844+0000 7f9ecf400640 1 --2- 172.21.15.67:0/1502168192 >> v2:172.21.15.134:3300/0 conn(0x7f9ed00abb30 0x7f9ed014efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.844+0000 7f9ecf400640 1 -- 172.21.15.67:0/1502168192 >> v1:172.21.15.67:6789/0 conn(0x7f9ed006d5b0 legacy=0x7f9ed01526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.844+0000 7f9ecf400640 1 -- 172.21.15.67:0/1502168192 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed00ac3d0 con 0x7f9ed014f520 2024-03-09T08:52:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.846+0000 7f9ecf400640 1 --2- 172.21.15.67:0/1502168192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ed014f520 0x7f9ed00aae00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00c4980 tx=0x7f9ebc02fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb6e8f32615b90f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.847+0000 7f9ecd600640 1 -- 172.21.15.67:0/1502168192 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc03d070 con 0x7f9ed014f520 2024-03-09T08:52:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.847+0000 7f9ecd600640 1 -- 172.21.15.67:0/1502168192 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc030960 con 0x7f9ed014f520 2024-03-09T08:52:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.847+0000 7f9ecd600640 1 -- 172.21.15.67:0/1502168192 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc03d070 con 0x7f9ed014f520 2024-03-09T08:52:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.848+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1502168192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ed014f520 msgr2=0x7f9ed00aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.848+0000 7f9ecfe00640 1 --2- 172.21.15.67:0/1502168192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ed014f520 0x7f9ed00aae00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00c4980 tx=0x7f9ebc02fc30 comp rx=0 tx=0).stop 2024-03-09T08:52:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.849+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1502168192 shutdown_connections 2024-03-09T08:52:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.849+0000 7f9ecfe00640 1 --2- 172.21.15.67:0/1502168192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ed014f520 0x7f9ed00aae00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.849+0000 7f9ecfe00640 1 --2- 172.21.15.67:0/1502168192 >> v2:172.21.15.134:3300/0 conn(0x7f9ed00abb30 0x7f9ed014efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.849+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1502168192 >> 172.21.15.67:0/1502168192 conn(0x7f9ed006b490 msgr2=0x7f9ed006b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:39.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.850+0000 7fb179600640 1 -- 172.21.15.67:0/433368270 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb17404e020 con 0x7fb17c0bebc0 2024-03-09T08:52:39.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.850+0000 7fb181800640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb15c06b5a0 0x7fb15c06da60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb170006fd0 tx=0x7fb170008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.850+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1502168192 shutdown_connections 2024-03-09T08:52:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.850+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1502168192 wait complete. 2024-03-09T08:52:39.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.853+0000 7f9ecfe00640 1 Processor -- start 2024-03-09T08:52:39.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.854+0000 7f9ecfe00640 1 -- start start 2024-03-09T08:52:39.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.856+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed006d5b0 0x7f9ed01da900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.856+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ed00abb30 0x7f9ed00c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.856+0000 7f9ecea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed006d5b0 0x7f9ed01da900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.857+0000 7f9ecea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed006d5b0 0x7f9ed01da900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45596/0 (socket says 172.21.15.67:45596) 2024-03-09T08:52:39.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.857+0000 7f9ecea00640 1 -- 172.21.15.67:0/1995579081 learned_addr learned my addr 172.21.15.67:0/1995579081 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.857+0000 7f9ecfe00640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ed014f520 0x7f9ed01f0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.858+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed006d990 con 0x7f9ed014f520 2024-03-09T08:52:39.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.858+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00aa8e0 con 0x7f9ed006d5b0 2024-03-09T08:52:39.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.858+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed006db10 con 0x7f9ed00abb30 2024-03-09T08:52:39.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.858+0000 7f9ece000640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ed00abb30 0x7f9ed00c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.859+0000 7f9ecf400640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ed014f520 0x7f9ed01f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.860+0000 7f9ecea00640 1 -- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ed00abb30 msgr2=0x7f9ed00c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.860+0000 7f9ecea00640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ed00abb30 0x7f9ed00c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.860+0000 7f9ecea00640 1 -- 172.21.15.67:0/1995579081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ed014f520 msgr2=0x7f9ed01f0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.860+0000 7f9ecea00640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ed014f520 0x7f9ed01f0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.860+0000 7f9ecea00640 1 -- 172.21.15.67:0/1995579081 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed01dd1b0 con 0x7f9ed006d5b0 2024-03-09T08:52:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.861+0000 7f9ecf400640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ed014f520 0x7f9ed01f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:39.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.861+0000 7f9ece000640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ed00abb30 0x7f9ed00c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:52:39.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.861+0000 7f9ecea00640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed006d5b0 0x7f9ed01da900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8001b10 tx=0x7f9ec8001fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.862+0000 7f9ec7400640 1 -- 172.21.15.67:0/1995579081 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8018070 con 0x7f9ed006d5b0 2024-03-09T08:52:39.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.862+0000 7f9ec7400640 1 -- 172.21.15.67:0/1995579081 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec800d660 con 0x7f9ed006d5b0 2024-03-09T08:52:39.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.862+0000 7f9ec7400640 1 -- 172.21.15.67:0/1995579081 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec800da30 con 0x7f9ed006d5b0 2024-03-09T08:52:39.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.864+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed0151be0 con 0x7f9ed006d5b0 2024-03-09T08:52:39.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.864+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed0073670 con 0x7f9ed006d5b0 2024-03-09T08:52:39.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.869+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed004f3c0 con 0x7f9ed006d5b0 2024-03-09T08:52:39.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.870+0000 7f9ec7400640 1 -- 172.21.15.67:0/1995579081 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9ec801b070 con 0x7f9ed006d5b0 2024-03-09T08:52:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.871+0000 7f9ec7400640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9eb006b5d0 0x7f9eb006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.871+0000 7f9ec7400640 1 -- 172.21.15.67:0/1995579081 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f9ec8089d60 con 0x7f9ed006d5b0 2024-03-09T08:52:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.872+0000 7f75a5000640 1 Processor -- start 2024-03-09T08:52:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.872+0000 7f75a5000640 1 -- start start 2024-03-09T08:52:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.874+0000 7f75a5000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f75a00caa20 0x7f75a00cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.874+0000 7f75a5000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a006b040 con 0x7f75a00c8c60 2024-03-09T08:52:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.874+0000 7f75a5000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a0156710 con 0x7f75a0153460 2024-03-09T08:52:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.874+0000 7f75a5000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a00ca590 con 0x7f75a00caa20 2024-03-09T08:52:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.872+0000 7f9ece000640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9eb006b5d0 0x7f9eb006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.875+0000 7f759e800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f75a00caa20 0x7f75a00cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.875+0000 7f759e800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f75a00caa20 0x7f75a00cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45608/0 (socket says 172.21.15.67:45608) 2024-03-09T08:52:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.875+0000 7f759e800640 1 -- 172.21.15.67:0/619551887 learned_addr learned my addr 172.21.15.67:0/619551887 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.875+0000 7f759e800640 1 -- 172.21.15.67:0/619551887 >> v1:172.21.15.134:6789/0 conn(0x7f75a0153460 legacy=0x7f75a0155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.875+0000 7f759e800640 1 -- 172.21.15.67:0/619551887 >> v1:172.21.15.67:6789/0 conn(0x7f75a00c8c60 legacy=0x7f75a00c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.875+0000 7f759e800640 1 -- 172.21.15.67:0/619551887 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a01569f0 con 0x7f75a00caa20 2024-03-09T08:52:39.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.878+0000 7f9ece000640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9eb006b5d0 0x7f9eb006da90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0006fd0 tx=0x7f9ec0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.879+0000 7f759e800640 1 --2- 172.21.15.67:0/619551887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75a00caa20 0x7f75a00cae50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7590009870 tx=0x7f759002b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef59cc5523885738 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.885+0000 7f759ca00640 1 -- 172.21.15.67:0/619551887 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590034070 con 0x7f75a00caa20 2024-03-09T08:52:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.885+0000 7f759ca00640 1 -- 172.21.15.67:0/619551887 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7590030850 con 0x7f75a00caa20 2024-03-09T08:52:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.885+0000 7f759ca00640 1 -- 172.21.15.67:0/619551887 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590030c20 con 0x7f75a00caa20 2024-03-09T08:52:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.885+0000 7f9ec7400640 1 -- 172.21.15.67:0/1995579081 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9ec80582d0 con 0x7f9ed006d5b0 2024-03-09T08:52:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.889+0000 7f75a5000640 1 -- 172.21.15.67:0/619551887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75a00caa20 msgr2=0x7f75a00cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.889+0000 7f75a5000640 1 --2- 172.21.15.67:0/619551887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75a00caa20 0x7f75a00cae50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7590009870 tx=0x7f759002b990 comp rx=0 tx=0).stop 2024-03-09T08:52:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.889+0000 7f75a5000640 1 -- 172.21.15.67:0/619551887 shutdown_connections 2024-03-09T08:52:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.889+0000 7f75a5000640 1 --2- 172.21.15.67:0/619551887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75a00caa20 0x7f75a00cae50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.890+0000 7f75a5000640 1 -- 172.21.15.67:0/619551887 >> 172.21.15.67:0/619551887 conn(0x7f75a00c8170 msgr2=0x7f75a00c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:39.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.890+0000 7f75a5000640 1 -- 172.21.15.67:0/619551887 shutdown_connections 2024-03-09T08:52:39.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.890+0000 7f75a5000640 1 -- 172.21.15.67:0/619551887 wait complete. 2024-03-09T08:52:39.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.894+0000 7f75a5000640 1 Processor -- start 2024-03-09T08:52:39.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.894+0000 7f75a5000640 1 -- start start 2024-03-09T08:52:39.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.896+0000 7f75a5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75a00c8c60 0x7f75a00ac340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.896+0000 7f75a5000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75a00caa20 0x7f75a01ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.896+0000 7f75a5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75a0153460 0x7f75a01ed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.896+0000 7f75a5000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00c9f20 con 0x7f75a0153460 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.896+0000 7f75a5000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a0153840 con 0x7f75a00c8c60 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.896+0000 7f75a5000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00ca0a0 con 0x7f75a00caa20 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.897+0000 7f759de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75a00c8c60 0x7f75a00ac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.897+0000 7f759d400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75a00caa20 0x7f75a01ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.897+0000 7f759d400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75a00caa20 0x7f75a01ecb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:39398/0 (socket says 172.21.15.67:39398) 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.897+0000 7f759d400640 1 -- 172.21.15.67:0/158662575 learned_addr learned my addr 172.21.15.67:0/158662575 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.898+0000 7f759de00640 1 -- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75a00caa20 msgr2=0x7f75a01ecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.898+0000 7f759de00640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75a00caa20 0x7f75a01ecb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.898+0000 7f759de00640 1 -- 172.21.15.67:0/158662575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75a0153460 msgr2=0x7f75a01ed0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.898+0000 7f759de00640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75a0153460 0x7f75a01ed0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.898+0000 7f759de00640 1 -- 172.21.15.67:0/158662575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00c55b0 con 0x7f75a00c8c60 2024-03-09T08:52:39.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.901+0000 7f759de00640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75a00c8c60 0x7f75a00ac340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f759800c730 tx=0x7f759800cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.903+0000 7f7596000640 1 -- 172.21.15.67:0/158662575 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598003b70 con 0x7f75a00c8c60 2024-03-09T08:52:39.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.903+0000 7f75a5000640 1 -- 172.21.15.67:0/158662575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a01548e0 con 0x7f75a00c8c60 2024-03-09T08:52:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.903+0000 7f75a5000640 1 -- 172.21.15.67:0/158662575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a01553b0 con 0x7f75a00c8c60 2024-03-09T08:52:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.905+0000 7f7596000640 1 -- 172.21.15.67:0/158662575 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598003d30 con 0x7f75a00c8c60 2024-03-09T08:52:39.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.905+0000 7f7596000640 1 -- 172.21.15.67:0/158662575 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598005310 con 0x7f75a00c8c60 2024-03-09T08:52:39.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.906+0000 7fa73fa00640 1 Processor -- start 2024-03-09T08:52:39.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.907+0000 7fa73fa00640 1 -- start start 2024-03-09T08:52:39.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.908+0000 7fa73fa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7380c5f50 0x7fa73806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.909+0000 7fa73fa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa7380c96f0 0x7fa7380c6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.909+0000 7fa73d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7380c5f50 0x7fa73806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.909+0000 7fa73d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7380c5f50 0x7fa73806b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46538/0 (socket says 172.21.15.67:46538) 2024-03-09T08:52:39.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.910+0000 7fa73da00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa7381567a0 0x7fa7380cacb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53842/0 (socket says 172.21.15.67:53842) 2024-03-09T08:52:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.910+0000 7fa737e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa7380c96f0 0x7fa7380c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.910+0000 7fa737e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa7380c96f0 0x7fa7380c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45626/0 (socket says 172.21.15.67:45626) 2024-03-09T08:52:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.910+0000 7fa73d000640 1 -- 172.21.15.67:0/608955770 learned_addr learned my addr 172.21.15.67:0/608955770 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.907+0000 7f75a5000640 1 -- 172.21.15.67:0/158662575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7578005290 con 0x7f75a00c8c60 2024-03-09T08:52:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.910+0000 7fa73fa00640 1 -- 172.21.15.67:0/608955770 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7380c9f30 con 0x7fa7381567a0 2024-03-09T08:52:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.910+0000 7fa73fa00640 1 -- 172.21.15.67:0/608955770 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73806cfa0 con 0x7fa7380c5f50 2024-03-09T08:52:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.910+0000 7fa73fa00640 1 -- 172.21.15.67:0/608955770 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7380ca820 con 0x7fa7380c96f0 2024-03-09T08:52:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.911+0000 7fa737400640 1 -- 172.21.15.67:0/608955770 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531901580 0 0) 0x7fa7380c9f30 con 0x7fa7381567a0 2024-03-09T08:52:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.911+0000 7fa737400640 1 -- 172.21.15.67:0/608955770 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa71c003520 con 0x7fa7381567a0 2024-03-09T08:52:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.911+0000 7fa73d000640 1 -- 172.21.15.67:0/608955770 >> v2:172.21.15.134:3300/0 conn(0x7fa7380c96f0 msgr2=0x7fa7380c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.911+0000 7fa73d000640 1 --2- 172.21.15.67:0/608955770 >> v2:172.21.15.134:3300/0 conn(0x7fa7380c96f0 0x7fa7380c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.911+0000 7fa73d000640 1 -- 172.21.15.67:0/608955770 >> v1:172.21.15.67:6789/0 conn(0x7fa7381567a0 legacy=0x7fa7380cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.911+0000 7fa73d000640 1 -- 172.21.15.67:0/608955770 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7380ca1d0 con 0x7fa7380c5f50 2024-03-09T08:52:39.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.915+0000 7fa73d000640 1 --2- 172.21.15.67:0/608955770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7380c5f50 0x7fa73806b040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa738073bd0 tx=0x7fa728034140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a83b0ba0daaf0a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.916+0000 7f7596000640 1 -- 172.21.15.67:0/158662575 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f75980054b0 con 0x7f75a00c8c60 2024-03-09T08:52:39.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.916+0000 7fa737400640 1 -- 172.21.15.67:0/608955770 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728041070 con 0x7fa7380c5f50 2024-03-09T08:52:39.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.916+0000 7fa737400640 1 -- 172.21.15.67:0/608955770 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72803c440 con 0x7fa7380c5f50 2024-03-09T08:52:39.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.917+0000 7fa73fa00640 1 -- 172.21.15.67:0/608955770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7380c5f50 msgr2=0x7fa73806b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.917+0000 7fa737400640 1 -- 172.21.15.67:0/608955770 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728041070 con 0x7fa7380c5f50 2024-03-09T08:52:39.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.917+0000 7fa73fa00640 1 --2- 172.21.15.67:0/608955770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7380c5f50 0x7fa73806b040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa738073bd0 tx=0x7fa728034140 comp rx=0 tx=0).stop 2024-03-09T08:52:39.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.918+0000 7fa73fa00640 1 -- 172.21.15.67:0/608955770 shutdown_connections 2024-03-09T08:52:39.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.918+0000 7fa73fa00640 1 --2- 172.21.15.67:0/608955770 >> v2:172.21.15.134:3300/0 conn(0x7fa7380c96f0 0x7fa7380c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.918+0000 7fa73fa00640 1 --2- 172.21.15.67:0/608955770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7380c5f50 0x7fa73806b040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.918+0000 7fa73fa00640 1 -- 172.21.15.67:0/608955770 >> 172.21.15.67:0/608955770 conn(0x7fa7380c8050 msgr2=0x7fa7380c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.918+0000 7fa73fa00640 1 -- 172.21.15.67:0/608955770 shutdown_connections 2024-03-09T08:52:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.918+0000 7fa73fa00640 1 -- 172.21.15.67:0/608955770 wait complete. 2024-03-09T08:52:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.920+0000 7fa73fa00640 1 Processor -- start 2024-03-09T08:52:39.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.922+0000 7fa73fa00640 1 -- start start 2024-03-09T08:52:39.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.921+0000 7f7596000640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f758006b5d0 0x7f758006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.922+0000 7f7596000640 1 -- 172.21.15.67:0/158662575 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f759808a320 con 0x7f75a00c8c60 2024-03-09T08:52:39.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.924+0000 7fa73fa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380c5f50 0x7fa7381484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.924+0000 7fa73fa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380c96f0 0x7fa738154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.924+0000 7fa73fa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7381567a0 0x7fa738154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.924+0000 7f759d400640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f758006b5d0 0x7f758006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.924+0000 7fa737e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380c96f0 0x7fa738154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.924+0000 7fa73da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7381567a0 0x7fa738154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.924+0000 7fa73fa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738156c10 con 0x7fa7381567a0 2024-03-09T08:52:39.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.924+0000 7fa73fa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7380c9bf0 con 0x7fa7380c96f0 2024-03-09T08:52:39.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.924+0000 7fa73fa00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa738156d90 con 0x7fa7380c5f50 2024-03-09T08:52:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.924+0000 7fa737e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380c96f0 0x7fa738154880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45638/0 (socket says 172.21.15.67:45638) 2024-03-09T08:52:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.924+0000 7fa737e00640 1 -- 172.21.15.67:0/3230110244 learned_addr learned my addr 172.21.15.67:0/3230110244 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.925+0000 7fa73d000640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380c5f50 0x7fa7381484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.925+0000 7fa73da00640 1 -- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380c5f50 msgr2=0x7fa7381484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.925+0000 7fa73da00640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380c5f50 0x7fa7381484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.925+0000 7fa73da00640 1 -- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380c96f0 msgr2=0x7fa738154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.925+0000 7fa73da00640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380c96f0 0x7fa738154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.925+0000 7fa73da00640 1 -- 172.21.15.67:0/3230110244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7380c2060 con 0x7fa7381567a0 2024-03-09T08:52:39.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.926+0000 7fa73da00640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7381567a0 0x7fa738154dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa72400c780 tx=0x7fa72400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.926+0000 7fa734c00640 1 -- 172.21.15.67:0/3230110244 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724018070 con 0x7fa7381567a0 2024-03-09T08:52:39.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.926+0000 7fa73fa00640 1 -- 172.21.15.67:0/3230110244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7381e8790 con 0x7fa7381567a0 2024-03-09T08:52:39.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.926+0000 7fa73fa00640 1 -- 172.21.15.67:0/3230110244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa738154360 con 0x7fa7381567a0 2024-03-09T08:52:39.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.929+0000 7f7596000640 1 -- 172.21.15.67:0/158662575 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7598058dc0 con 0x7f75a00c8c60 2024-03-09T08:52:39.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.927+0000 7fa734c00640 1 -- 172.21.15.67:0/3230110244 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa724007270 con 0x7fa7381567a0 2024-03-09T08:52:39.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.927+0000 7fa734c00640 1 -- 172.21.15.67:0/3230110244 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724018070 con 0x7fa7381567a0 2024-03-09T08:52:39.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.929+0000 7f759d400640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f758006b5d0 0x7f758006da90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f75a014d870 tx=0x7f758c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.928+0000 7fa73fa00640 1 -- 172.21.15.67:0/3230110244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa710005290 con 0x7fa7381567a0 2024-03-09T08:52:39.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.931+0000 7fa734c00640 1 -- 172.21.15.67:0/3230110244 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa72401b070 con 0x7fa7381567a0 2024-03-09T08:52:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.933+0000 7fa734c00640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa71806b700 0x7fa71806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.937+0000 7fa734c00640 1 -- 172.21.15.67:0/3230110244 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fa724055800 con 0x7fa7381567a0 2024-03-09T08:52:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.937+0000 7fa73d000640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa71806b700 0x7fa71806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.943+0000 7fa73d000640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa71806b700 0x7fa71806dbc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa72803e000 tx=0x7fa728038000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.945+0000 7fa734c00640 1 -- 172.21.15.67:0/3230110244 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa724057760 con 0x7fa7381567a0 2024-03-09T08:52:39.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.979+0000 7f2cbb200640 1 Processor -- start 2024-03-09T08:52:39.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.979+0000 7f2cbb200640 1 -- start start 2024-03-09T08:52:39.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.981+0000 7f2cbb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2cbc14c470 0x7f2cbc156540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.981+0000 7f2cbb200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cbc0ac090 0x7f2cbc06d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:39.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.981+0000 7f2cbb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc159150 con 0x7f2cbc0aa3b0 2024-03-09T08:52:39.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.981+0000 7f2cbb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0ac540 con 0x7f2cbc14c470 2024-03-09T08:52:39.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.981+0000 7f2cbb200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc06e110 con 0x7f2cbc0ac090 2024-03-09T08:52:39.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.983+0000 7f2cb9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2cbc14c470 0x7f2cbc156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.983+0000 7f2cb9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2cbc14c470 0x7f2cbc156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46550/0 (socket says 172.21.15.67:46550) 2024-03-09T08:52:39.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.983+0000 7f2cb9e00640 1 -- 172.21.15.67:0/2264044831 learned_addr learned my addr 172.21.15.67:0/2264044831 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:39.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.984+0000 7f2cb8a00640 1 -- 172.21.15.67:0/2264044831 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941941232 0 0) 0x7f2cbc159150 con 0x7f2cbc0aa3b0 2024-03-09T08:52:39.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.984+0000 7f2cb8a00640 1 -- 172.21.15.67:0/2264044831 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ca0003520 con 0x7f2cbc0aa3b0 2024-03-09T08:52:39.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.985+0000 7f2cba800640 1 --2- 172.21.15.67:0/2264044831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cbc0ac090 0x7f2cbc06d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:39.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.988+0000 7f2cb9e00640 1 -- 172.21.15.67:0/2264044831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cbc0ac090 msgr2=0x7f2cbc06d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:39.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.988+0000 7f2cb9e00640 1 --2- 172.21.15.67:0/2264044831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cbc0ac090 0x7f2cbc06d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:39.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.988+0000 7f2cb9e00640 1 -- 172.21.15.67:0/2264044831 >> v1:172.21.15.67:6789/0 conn(0x7f2cbc0aa3b0 legacy=0x7f2cbc159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.988+0000 7f2cb9e00640 1 -- 172.21.15.67:0/2264044831 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc06e3f0 con 0x7f2cbc14c470 2024-03-09T08:52:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:39.998+0000 7f2cb9e00640 1 --2- 172.21.15.67:0/2264044831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc14c470 0x7f2cbc156540 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009870 tx=0x7f2ca802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730f0fcbf771b33c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:39.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.002+0000 7f2cb8a00640 1 -- 172.21.15.67:0/2264044831 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8038070 con 0x7f2cbc14c470 2024-03-09T08:52:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.002+0000 7f2cb8a00640 1 -- 172.21.15.67:0/2264044831 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca8033440 con 0x7f2cbc14c470 2024-03-09T08:52:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.002+0000 7f2cb8a00640 1 -- 172.21.15.67:0/2264044831 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac001670 con 0x7f2cbc14c470 2024-03-09T08:52:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.003+0000 7f2cbb200640 1 -- 172.21.15.67:0/2264044831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc14c470 msgr2=0x7f2cbc156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:39.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.003+0000 7f2cbb200640 1 --2- 172.21.15.67:0/2264044831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc14c470 0x7f2cbc156540 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009870 tx=0x7f2ca802b7a0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.009+0000 7f2cbb200640 1 -- 172.21.15.67:0/2264044831 shutdown_connections 2024-03-09T08:52:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.009+0000 7f2cbb200640 1 --2- 172.21.15.67:0/2264044831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cbc0ac090 0x7f2cbc06d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.009+0000 7f2cbb200640 1 --2- 172.21.15.67:0/2264044831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc14c470 0x7f2cbc156540 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.009+0000 7f2cbb200640 1 -- 172.21.15.67:0/2264044831 >> 172.21.15.67:0/2264044831 conn(0x7f2cbc06b490 msgr2=0x7f2cbc06b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.009+0000 7f2cbb200640 1 -- 172.21.15.67:0/2264044831 shutdown_connections 2024-03-09T08:52:40.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.009+0000 7f2cbb200640 1 -- 172.21.15.67:0/2264044831 wait complete. 2024-03-09T08:52:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.017+0000 7f2cbb200640 1 Processor -- start 2024-03-09T08:52:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.017+0000 7f2cbb200640 1 -- start start 2024-03-09T08:52:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.023+0000 7f2cbb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0aa3b0 0x7f2cbc1f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.024+0000 7f2cbb200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cbc0ac090 0x7f2cbc0be2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.024+0000 7f2cbb200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cbc14c470 0x7f2cbc0be800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:40.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.025+0000 7f2cb9400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cbc0ac090 0x7f2cbc0be2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:40.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.025+0000 7f2cb9400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cbc0ac090 0x7f2cbc0be2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45648/0 (socket says 172.21.15.67:45648) 2024-03-09T08:52:40.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.024+0000 7f2cb9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0aa3b0 0x7f2cbc1f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:40.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.029+0000 7f2cb9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0aa3b0 0x7f2cbc1f2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46552/0 (socket says 172.21.15.67:46552) 2024-03-09T08:52:40.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.029+0000 7f2cbb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc14c9d0 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.029+0000 7f2cbb200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc14c850 con 0x7f2cbc0ac090 2024-03-09T08:52:40.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.030+0000 7f2cb9400640 1 -- 172.21.15.67:0/1460495061 learned_addr learned my addr 172.21.15.67:0/1460495061 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:40.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.030+0000 7f2cba800640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cbc14c470 0x7f2cbc0be800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:40.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.030+0000 7f2cbb200640 1 -- 172.21.15.67:0/1460495061 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc06d880 con 0x7f2cbc14c470 2024-03-09T08:52:40.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.032+0000 7f2cb9e00640 1 -- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cbc14c470 msgr2=0x7f2cbc0be800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.032+0000 7f2cb9e00640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cbc14c470 0x7f2cbc0be800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.032+0000 7f2cb9e00640 1 -- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cbc0ac090 msgr2=0x7f2cbc0be2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.032+0000 7f2cb9e00640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cbc0ac090 0x7f2cbc0be2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.032+0000 7f2cb9e00640 1 -- 172.21.15.67:0/1460495061 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc0bcce0 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.036+0000 7f2cb9e00640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0aa3b0 0x7f2cbc1f2e90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0c4140 tx=0x7f2ca80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:40.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.037+0000 7f2cb2000640 1 -- 172.21.15.67:0/1460495061 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8042070 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.037+0000 7f2cb2000640 1 -- 172.21.15.67:0/1460495061 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca803d870 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.037+0000 7f2cb2000640 1 -- 172.21.15.67:0/1460495061 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8038040 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.037+0000 7f2cbb200640 1 -- 172.21.15.67:0/1460495061 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc14bf10 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.037+0000 7f2cbb200640 1 -- 172.21.15.67:0/1460495061 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc1f39b0 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.042+0000 7f2cbb200640 1 -- 172.21.15.67:0/1460495061 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc04f3c0 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.056+0000 7f2cb2000640 1 -- 172.21.15.67:0/1460495061 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2ca8045070 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.059+0000 7f2cb2000640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c9c06b5a0 0x7f2c9c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:40.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.059+0000 7f2cb2000640 1 -- 172.21.15.67:0/1460495061 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f2ca80ac310 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.060+0000 7f2cb9400640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c9c06b5a0 0x7f2c9c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.062+0000 7f2cb2000640 1 -- 172.21.15.67:0/1460495061 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2ca807a880 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.063+0000 7f2cb9400640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c9c06b5a0 0x7f2c9c06da60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc158ab0 tx=0x7f2cac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:40.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.234+0000 7f7d47200640 1 -- 172.21.15.67:0/885811890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7d20005580 con 0x7f7d480c3d20 2024-03-09T08:52:40.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.235+0000 7f7d3e000640 1 -- 172.21.15.67:0/885811890 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d34057eb0 con 0x7f7d480c3d20 2024-03-09T08:52:40.233 INFO:teuthology.orchestra.run.smithi067.stdout:55834574858 2024-03-09T08:52:40.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.237+0000 7fb41f400640 1 -- 172.21.15.67:0/2572068958 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb4340c1530 con 0x7fb43414c9b0 2024-03-09T08:52:40.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.238+0000 7f7d47200640 1 -- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d2806b5a0 msgr2=0x7f7d2806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.239+0000 7fb42a000640 1 -- 172.21.15.67:0/2572068958 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb42c05c160 con 0x7fb43414c9b0 2024-03-09T08:52:40.237 INFO:teuthology.orchestra.run.smithi067.stdout:55834574857 2024-03-09T08:52:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.238+0000 7f7d47200640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d2806b5a0 0x7f7d2806da60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7d38006fd0 tx=0x7f7d38008040 comp rx=0 tx=0).stop 2024-03-09T08:52:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.239+0000 7f7d47200640 1 -- 172.21.15.67:0/885811890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d480c3d20 msgr2=0x7f7d481efc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.239+0000 7f7d47200640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d480c3d20 0x7f7d481efc80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400c520 tx=0x7f7d3400c9e0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.242+0000 7f7d47200640 1 -- 172.21.15.67:0/885811890 shutdown_connections 2024-03-09T08:52:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.242+0000 7f7d47200640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d2806b5a0 0x7f7d2806da60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.242+0000 7f7d47200640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d480c3d20 0x7f7d481efc80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.242+0000 7f7d47200640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d480abb30 0x7f7d48070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.242+0000 7f7d47200640 1 --2- 172.21.15.67:0/885811890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d480aae00 0x7f7d4813ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.242+0000 7f7d47200640 1 -- 172.21.15.67:0/885811890 >> 172.21.15.67:0/885811890 conn(0x7f7d4806b490 msgr2=0x7f7d4814edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.244+0000 7f7d47200640 1 -- 172.21.15.67:0/885811890 shutdown_connections 2024-03-09T08:52:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.244+0000 7fb41f400640 1 -- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb41406b4d0 msgr2=0x7fb41406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.244+0000 7fb41f400640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb41406b4d0 0x7fb41406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb420000c00 tx=0x7fb42003b040 comp rx=0 tx=0).stop 2024-03-09T08:52:40.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.245+0000 7f7d47200640 1 -- 172.21.15.67:0/885811890 wait complete. 2024-03-09T08:52:40.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.245+0000 7fb41f400640 1 -- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb43414c9b0 msgr2=0x7fb4341ef020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.245+0000 7fb41f400640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb43414c9b0 0x7fb4341ef020 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0012e0 tx=0x7fb42c00cb90 comp rx=0 tx=0).stop 2024-03-09T08:52:40.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.244+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f37d4005850 con 0x7f37fc0a9d30 2024-03-09T08:52:40.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.245+0000 7fb41f400640 1 -- 172.21.15.67:0/2572068958 shutdown_connections 2024-03-09T08:52:40.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.245+0000 7fb41f400640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb41406b4d0 0x7fb41406d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.245+0000 7fb41f400640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb43414c9b0 0x7fb4341ef020 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.245+0000 7fb41f400640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4340c1d70 0x7fb4341599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.245+0000 7fb41f400640 1 --2- 172.21.15.67:0/2572068958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4340c0a10 0x7fb434159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.245+0000 7fb41f400640 1 -- 172.21.15.67:0/2572068958 >> 172.21.15.67:0/2572068958 conn(0x7fb4340c7370 msgr2=0x7fb434154510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:40.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.245+0000 7fb41f400640 1 -- 172.21.15.67:0/2572068958 shutdown_connections 2024-03-09T08:52:40.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.245+0000 7fb41f400640 1 -- 172.21.15.67:0/2572068958 wait complete. 2024-03-09T08:52:40.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.246+0000 7f37f3400640 1 -- 172.21.15.67:0/1840971716 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f37ec057cc0 con 0x7f37fc0a9d30 2024-03-09T08:52:40.243 INFO:teuthology.orchestra.run.smithi067.stdout:55834574859 2024-03-09T08:52:40.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.250+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f37dc06b8d0 msgr2=0x7f37dc06dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.250+0000 7f3802800640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f37dc06b8d0 0x7f37dc06dd90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f37e803b000 tx=0x7f37e8036000 comp rx=0 tx=0).stop 2024-03-09T08:52:40.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.250+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37fc0a9d30 msgr2=0x7f37fc1f0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.250+0000 7f3802800640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37fc0a9d30 0x7f37fc1f0f10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c3f0 tx=0x7f37ec00c8b0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.251+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 shutdown_connections 2024-03-09T08:52:40.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.251+0000 7f3802800640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f37dc06b8d0 0x7f37dc06dd90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.251+0000 7f3802800640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37fc14f2a0 0x7f37fc0c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.251+0000 7f3802800640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f37fc0c3240 0x7f37fc14bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.251+0000 7f3802800640 1 --2- 172.21.15.67:0/1840971716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37fc0a9d30 0x7f37fc1f0f10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.251+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 >> 172.21.15.67:0/1840971716 conn(0x7f37fc0c7370 msgr2=0x7f37fc0aa1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:40.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.252+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 shutdown_connections 2024-03-09T08:52:40.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.252+0000 7f3802800640 1 -- 172.21.15.67:0/1840971716 wait complete. 2024-03-09T08:52:40.271 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2024-03-09T08:52:40.271 DEBUG:teuthology.parallel:result is None 2024-03-09T08:52:40.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-03-09T08:52:40.277 DEBUG:teuthology.parallel:result is None 2024-03-09T08:52:40.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-03-09T08:52:40.280 DEBUG:teuthology.parallel:result is None 2024-03-09T08:52:40.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.300+0000 7fb182c00640 1 -- 172.21.15.67:0/433368270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb17c06c980 con 0x7fb17c0bebc0 2024-03-09T08:52:40.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.301+0000 7fb179600640 1 -- 172.21.15.67:0/433368270 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb17407cb90 con 0x7fb17c0bebc0 2024-03-09T08:52:40.297 INFO:teuthology.orchestra.run.smithi067.stdout:55834574858 2024-03-09T08:52:40.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.303+0000 7fb182c00640 1 -- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb15c06b5a0 msgr2=0x7fb15c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.303+0000 7fb182c00640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb15c06b5a0 0x7fb15c06da60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb170006fd0 tx=0x7fb170008040 comp rx=0 tx=0).stop 2024-03-09T08:52:40.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.303+0000 7fb182c00640 1 -- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c0bebc0 msgr2=0x7fb17c155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.303+0000 7fb182c00640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c0bebc0 0x7fb17c155600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb17402c390 tx=0x7fb174004a40 comp rx=0 tx=0).stop 2024-03-09T08:52:40.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.304+0000 7fb182c00640 1 -- 172.21.15.67:0/433368270 shutdown_connections 2024-03-09T08:52:40.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.304+0000 7fb182c00640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb15c06b5a0 0x7fb15c06da60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.304+0000 7fb182c00640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb17c0bebc0 0x7fb17c155600 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.304+0000 7fb182c00640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb17c072770 0x7fb17c153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.304+0000 7fb182c00640 1 --2- 172.21.15.67:0/433368270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb17c06bda0 0x7fb17c150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.304+0000 7fb182c00640 1 -- 172.21.15.67:0/433368270 >> 172.21.15.67:0/433368270 conn(0x7fb17c0c7fb0 msgr2=0x7fb17c0ca5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:40.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.304+0000 7fb182c00640 1 -- 172.21.15.67:0/433368270 shutdown_connections 2024-03-09T08:52:40.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.304+0000 7fb182c00640 1 -- 172.21.15.67:0/433368270 wait complete. 2024-03-09T08:52:40.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.309+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9ed006e530 con 0x7f9ed006d5b0 2024-03-09T08:52:40.306 INFO:teuthology.orchestra.run.smithi067.stdout:55834574858 2024-03-09T08:52:40.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.310+0000 7f9ec7400640 1 -- 172.21.15.67:0/1995579081 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ec805c370 con 0x7f9ed006d5b0 2024-03-09T08:52:40.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.312+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9eb006b5d0 msgr2=0x7f9eb006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.312+0000 7f9ecfe00640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9eb006b5d0 0x7f9eb006da90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0006fd0 tx=0x7f9ec0008040 comp rx=0 tx=0).stop 2024-03-09T08:52:40.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.312+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed006d5b0 msgr2=0x7f9ed01da900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.312+0000 7f9ecfe00640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed006d5b0 0x7f9ed01da900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8001b10 tx=0x7f9ec8001fd0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.313+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 shutdown_connections 2024-03-09T08:52:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.313+0000 7f9ecfe00640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9eb006b5d0 0x7f9eb006da90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.313+0000 7f9ecfe00640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ed014f520 0x7f9ed01f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.313+0000 7f9ecfe00640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ed00abb30 0x7f9ed00c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.313+0000 7f9ecfe00640 1 --2- 172.21.15.67:0/1995579081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ed006d5b0 0x7f9ed01da900 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.313+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 >> 172.21.15.67:0/1995579081 conn(0x7f9ed006b490 msgr2=0x7f9ed00ab090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.313+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 shutdown_connections 2024-03-09T08:52:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.313+0000 7f9ecfe00640 1 -- 172.21.15.67:0/1995579081 wait complete. 2024-03-09T08:52:40.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-03-09T08:52:40.325 DEBUG:teuthology.parallel:result is None 2024-03-09T08:52:40.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-03-09T08:52:40.327 DEBUG:teuthology.parallel:result is None 2024-03-09T08:52:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.369+0000 7f75a5000640 1 -- 172.21.15.67:0/158662575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7578005580 con 0x7f75a00c8c60 2024-03-09T08:52:40.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.369+0000 7f7596000640 1 -- 172.21.15.67:0/158662575 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f75980586c0 con 0x7f75a00c8c60 2024-03-09T08:52:40.365 INFO:teuthology.orchestra.run.smithi067.stdout:55834574859 2024-03-09T08:52:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 -- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f758006b5d0 msgr2=0x7f758006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f758006b5d0 0x7f758006da90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f75a014d870 tx=0x7f758c009040 comp rx=0 tx=0).stop 2024-03-09T08:52:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 -- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75a00c8c60 msgr2=0x7f75a00ac340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75a00c8c60 0x7f75a00ac340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f759800c730 tx=0x7f759800cbf0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 -- 172.21.15.67:0/158662575 shutdown_connections 2024-03-09T08:52:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f758006b5d0 0x7f758006da90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75a0153460 0x7f75a01ed0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75a00caa20 0x7f75a01ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 --2- 172.21.15.67:0/158662575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75a00c8c60 0x7f75a00ac340 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 -- 172.21.15.67:0/158662575 >> 172.21.15.67:0/158662575 conn(0x7f75a00c8170 msgr2=0x7f75a01507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:40.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 -- 172.21.15.67:0/158662575 shutdown_connections 2024-03-09T08:52:40.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.372+0000 7f75a5000640 1 -- 172.21.15.67:0/158662575 wait complete. 2024-03-09T08:52:40.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.389+0000 7fa73fa00640 1 -- 172.21.15.67:0/3230110244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa710005850 con 0x7fa7381567a0 2024-03-09T08:52:40.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.390+0000 7fa734c00640 1 -- 172.21.15.67:0/3230110244 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa72405b800 con 0x7fa7381567a0 2024-03-09T08:52:40.386 INFO:teuthology.orchestra.run.smithi067.stdout:55834574857 2024-03-09T08:52:40.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2024-03-09T08:52:40.387 DEBUG:teuthology.parallel:result is None 2024-03-09T08:52:40.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 -- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa71806b700 msgr2=0x7fa71806dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa71806b700 0x7fa71806dbc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa72803e000 tx=0x7fa728038000 comp rx=0 tx=0).stop 2024-03-09T08:52:40.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 -- 172.21.15.67:0/3230110244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7381567a0 msgr2=0x7fa738154dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7381567a0 0x7fa738154dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa72400c780 tx=0x7fa72400cc40 comp rx=0 tx=0).stop 2024-03-09T08:52:40.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 -- 172.21.15.67:0/3230110244 shutdown_connections 2024-03-09T08:52:40.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa71806b700 0x7fa71806dbc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7381567a0 0x7fa738154dc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7380c96f0 0x7fa738154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 --2- 172.21.15.67:0/3230110244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7380c5f50 0x7fa7381484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 -- 172.21.15.67:0/3230110244 >> 172.21.15.67:0/3230110244 conn(0x7fa7380c8050 msgr2=0x7fa73806b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:40.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 -- 172.21.15.67:0/3230110244 shutdown_connections 2024-03-09T08:52:40.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.393+0000 7fa73fa00640 1 -- 172.21.15.67:0/3230110244 wait complete. 2024-03-09T08:52:40.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2024-03-09T08:52:40.405 DEBUG:teuthology.parallel:result is None 2024-03-09T08:52:40.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.444+0000 7f2cbb200640 1 -- 172.21.15.67:0/1460495061 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2cbc0c80f0 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.444+0000 7f2cb2000640 1 -- 172.21.15.67:0/1460495061 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ca807e920 con 0x7f2cbc0aa3b0 2024-03-09T08:52:40.440 INFO:teuthology.orchestra.run.smithi067.stdout:55834574859 2024-03-09T08:52:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 -- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c9c06b5a0 msgr2=0x7f2c9c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c9c06b5a0 0x7f2c9c06da60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc158ab0 tx=0x7f2cac009040 comp rx=0 tx=0).stop 2024-03-09T08:52:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 -- 172.21.15.67:0/1460495061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0aa3b0 msgr2=0x7f2cbc1f2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0aa3b0 0x7f2cbc1f2e90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0c4140 tx=0x7f2ca80046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 -- 172.21.15.67:0/1460495061 shutdown_connections 2024-03-09T08:52:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c9c06b5a0 0x7f2c9c06da60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cbc14c470 0x7f2cbc0be800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cbc0ac090 0x7f2cbc0be2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 --2- 172.21.15.67:0/1460495061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cbc0aa3b0 0x7f2cbc1f2e90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 -- 172.21.15.67:0/1460495061 >> 172.21.15.67:0/1460495061 conn(0x7f2cbc06b490 msgr2=0x7f2cbc156240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 -- 172.21.15.67:0/1460495061 shutdown_connections 2024-03-09T08:52:40.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.447+0000 7f2cbb200640 1 -- 172.21.15.67:0/1460495061 wait complete. 2024-03-09T08:52:40.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-03-09T08:52:40.458 DEBUG:teuthology.parallel:result is None 2024-03-09T08:52:40.458 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-09T08:52:40.458 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T08:52:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.706+0000 7f38df200640 1 Processor -- start 2024-03-09T08:52:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.706+0000 7f38df200640 1 -- start start 2024-03-09T08:52:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.707+0000 7f38df200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e00710a0 0x7f38e0153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.708+0000 7f38df200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f38e0151360 0x7f38e0153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.708+0000 7f38df200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e00c81e0 con 0x7f38e00c1f70 2024-03-09T08:52:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.708+0000 7f38df200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e00c32b0 con 0x7f38e0151360 2024-03-09T08:52:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.708+0000 7f38df200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e00c2fd0 con 0x7f38e00710a0 2024-03-09T08:52:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.708+0000 7f38dde00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f38e00c1f70 0x7f38e0150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:58586/0 (socket says 172.21.15.67:58586) 2024-03-09T08:52:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.708+0000 7f38dde00640 1 -- 172.21.15.67:0/2419036791 learned_addr learned my addr 172.21.15.67:0/2419036791 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.708+0000 7f38de800640 1 --2- 172.21.15.67:0/2419036791 >> v2:172.21.15.67:3300/0 conn(0x7f38e0151360 0x7f38e0153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.708+0000 7f38dd400640 1 --2- 172.21.15.67:0/2419036791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e00710a0 0x7f38e0153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.709+0000 7f38de800640 1 -- 172.21.15.67:0/2419036791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e00710a0 msgr2=0x7f38e0153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.709+0000 7f38de800640 1 --2- 172.21.15.67:0/2419036791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e00710a0 0x7f38e0153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.709+0000 7f38de800640 1 -- 172.21.15.67:0/2419036791 >> v1:172.21.15.134:6789/0 conn(0x7f38e00c1f70 legacy=0x7f38e0150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.709+0000 7f38de800640 1 -- 172.21.15.67:0/2419036791 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e00c3700 con 0x7f38e0151360 2024-03-09T08:52:40.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.710+0000 7f38de800640 1 --2- 172.21.15.67:0/2419036791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38e0151360 0x7f38e0153eb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009d90 tx=0x7f38d802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51aa8e0be1fe8dd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:40.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.710+0000 7f38dca00640 1 -- 172.21.15.67:0/2419036791 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8039070 con 0x7f38e0151360 2024-03-09T08:52:40.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.710+0000 7f38dca00640 1 -- 172.21.15.67:0/2419036791 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d802cbc0 con 0x7f38e0151360 2024-03-09T08:52:40.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.710+0000 7f38dca00640 1 -- 172.21.15.67:0/2419036791 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8034440 con 0x7f38e0151360 2024-03-09T08:52:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.711+0000 7f38df200640 1 -- 172.21.15.67:0/2419036791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38e0151360 msgr2=0x7f38e0153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.711+0000 7f38df200640 1 --2- 172.21.15.67:0/2419036791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38e0151360 0x7f38e0153eb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009d90 tx=0x7f38d802be10 comp rx=0 tx=0).stop 2024-03-09T08:52:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.711+0000 7f38df200640 1 -- 172.21.15.67:0/2419036791 shutdown_connections 2024-03-09T08:52:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.711+0000 7f38df200640 1 --2- 172.21.15.67:0/2419036791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38e0151360 0x7f38e0153eb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.711+0000 7f38df200640 1 --2- 172.21.15.67:0/2419036791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e00710a0 0x7f38e0153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.711+0000 7f38df200640 1 -- 172.21.15.67:0/2419036791 >> 172.21.15.67:0/2419036791 conn(0x7f38e00c7300 msgr2=0x7f38e00c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.711+0000 7f38df200640 1 -- 172.21.15.67:0/2419036791 shutdown_connections 2024-03-09T08:52:40.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.711+0000 7f38df200640 1 -- 172.21.15.67:0/2419036791 wait complete. 2024-03-09T08:52:40.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.712+0000 7f38df200640 1 Processor -- start 2024-03-09T08:52:40.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.712+0000 7f38df200640 1 -- start start 2024-03-09T08:52:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.713+0000 7f38df200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38e00710a0 0x7f38e01ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.713+0000 7f38df200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e00c1f70 0x7f38e0144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.713+0000 7f38df200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e0151360 0x7f38e00c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.713+0000 7f38df200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0070d50 con 0x7f38e00710a0 2024-03-09T08:52:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.713+0000 7f38df200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0070830 con 0x7f38e0151360 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.713+0000 7f38df200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0070ed0 con 0x7f38e00c1f70 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.713+0000 7f38dde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38e00710a0 0x7f38e01ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38dde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38e00710a0 0x7f38e01ec2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46568/0 (socket says 172.21.15.67:46568) 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38dde00640 1 -- 172.21.15.67:0/2907657666 learned_addr learned my addr 172.21.15.67:0/2907657666 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38de800640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e0151360 0x7f38e00c09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38dde00640 1 -- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e00c1f70 msgr2=0x7f38e0144ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38dd400640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e00c1f70 0x7f38e0144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38dde00640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e00c1f70 0x7f38e0144ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38dde00640 1 -- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e0151360 msgr2=0x7f38e00c09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38dde00640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e0151360 0x7f38e00c09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38dde00640 1 -- 172.21.15.67:0/2907657666 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e00c4c90 con 0x7f38e00710a0 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38de800640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e0151360 0x7f38e00c09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38dde00640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38e00710a0 0x7f38e01ec2d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f38d000c480 tx=0x7f38d000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38d6000640 1 -- 172.21.15.67:0/2907657666 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0017070 con 0x7f38e00710a0 2024-03-09T08:52:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38d6000640 1 -- 172.21.15.67:0/2907657666 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d0005ce0 con 0x7f38e00710a0 2024-03-09T08:52:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.714+0000 7f38d6000640 1 -- 172.21.15.67:0/2907657666 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00072d0 con 0x7f38e00710a0 2024-03-09T08:52:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.715+0000 7f38df200640 1 -- 172.21.15.67:0/2907657666 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e00c46d0 con 0x7f38e00710a0 2024-03-09T08:52:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.715+0000 7f38df200640 1 -- 172.21.15.67:0/2907657666 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e01eee80 con 0x7f38e00710a0 2024-03-09T08:52:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.716+0000 7f38df200640 1 -- 172.21.15.67:0/2907657666 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e014bf00 con 0x7f38e00710a0 2024-03-09T08:52:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.717+0000 7f38d6000640 1 -- 172.21.15.67:0/2907657666 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f38d001a070 con 0x7f38e00710a0 2024-03-09T08:52:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.718+0000 7f38d6000640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38c006b4d0 0x7f38c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.718+0000 7f38d6000640 1 -- 172.21.15.67:0/2907657666 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f38d0089810 con 0x7f38e00710a0 2024-03-09T08:52:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.718+0000 7f38dd400640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38c006b4d0 0x7f38c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.719+0000 7f38dd400640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38c006b4d0 0x7f38c006d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f38cc006fd0 tx=0x7f38cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:40.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.722+0000 7f38d6000640 1 -- 172.21.15.67:0/2907657666 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f38e014bf00 con 0x7f38e00710a0 2024-03-09T08:52:40.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.908+0000 7f38df200640 1 -- 172.21.15.67:0/2907657666 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38e01450e0 con 0x7f38c006b4d0 2024-03-09T08:52:40.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.942+0000 7f38d6000640 1 -- 172.21.15.67:0/2907657666 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490306 (secure 0 0 0) 0x7f38e01450e0 con 0x7f38c006b4d0 2024-03-09T08:52:40.940 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:40.948 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T08:52:40.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 -- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38c006b4d0 msgr2=0x7f38c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38c006b4d0 0x7f38c006d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f38cc006fd0 tx=0x7f38cc008040 comp rx=0 tx=0).stop 2024-03-09T08:52:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 -- 172.21.15.67:0/2907657666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38e00710a0 msgr2=0x7f38e01ec2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38e00710a0 0x7f38e01ec2d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f38d000c480 tx=0x7f38d000c940 comp rx=0 tx=0).stop 2024-03-09T08:52:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 -- 172.21.15.67:0/2907657666 shutdown_connections 2024-03-09T08:52:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38c006b4d0 0x7f38c006d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38e0151360 0x7f38e00c09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38e00c1f70 0x7f38e0144ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 --2- 172.21.15.67:0/2907657666 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38e00710a0 0x7f38e01ec2d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 -- 172.21.15.67:0/2907657666 >> 172.21.15.67:0/2907657666 conn(0x7f38e00c7300 msgr2=0x7f38e014b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 -- 172.21.15.67:0/2907657666 shutdown_connections 2024-03-09T08:52:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:40.952+0000 7f38df200640 1 -- 172.21.15.67:0/2907657666 wait complete. 2024-03-09T08:52:40.965 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-03-09T08:52:40.681725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3350528,"data_stored":1984464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.110573"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575471+0000","last_change":"2024-03-09T08:52:29.573571+0000","last_active":"2024-03-09T08:52:30.575471+0000","last_peered":"2024-03-09T08:52:30.575471+0000","last_clean":"2024-03-09T08:52:30.575471+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T08:52:30.575471+0000","last_undegraded":"2024-03-09T08:52:30.575471+0000","last_fullsized":"2024-03-09T08:52:30.575471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:33:33.068463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598024+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T08:52:30.598024+0000","last_peered":"2024-03-09T08:52:30.598024+0000","last_clean":"2024-03-09T08:52:30.598024+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T08:52:30.598024+0000","last_undegraded":"2024-03-09T08:52:30.598024+0000","last_fullsized":"2024-03-09T08:52:30.598024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:41:40.332687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577727+0000","last_change":"2024-03-09T08:52:18.235099+0000","last_active":"2024-03-09T08:52:30.577727+0000","last_peered":"2024-03-09T08:52:30.577727+0000","last_clean":"2024-03-09T08:52:30.577727+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T08:52:30.577727+0000","last_undegraded":"2024-03-09T08:52:30.577727+0000","last_fullsized":"2024-03-09T08:52:30.577727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:39:28.882461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575426+0000","last_change":"2024-03-09T08:52:28.571905+0000","last_active":"2024-03-09T08:52:30.575426+0000","last_peered":"2024-03-09T08:52:30.575426+0000","last_clean":"2024-03-09T08:52:30.575426+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T08:52:30.575426+0000","last_undegraded":"2024-03-09T08:52:30.575426+0000","last_fullsized":"2024-03-09T08:52:30.575426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:01.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.600985+0000","last_change":"2024-03-09T08:52:29.578644+0000","last_active":"2024-03-09T08:52:30.600985+0000","last_peered":"2024-03-09T08:52:30.600985+0000","last_clean":"2024-03-09T08:52:30.600985+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T08:52:30.600985+0000","last_undegraded":"2024-03-09T08:52:30.600985+0000","last_fullsized":"2024-03-09T08:52:30.600985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:00.044620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577762+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T08:52:30.577762+0000","last_peered":"2024-03-09T08:52:30.577762+0000","last_clean":"2024-03-09T08:52:30.577762+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T08:52:30.577762+0000","last_undegraded":"2024-03-09T08:52:30.577762+0000","last_fullsized":"2024-03-09T08:52:30.577762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597762+0000","last_change":"2024-03-09T08:52:18.236946+0000","last_active":"2024-03-09T08:52:30.597762+0000","last_peered":"2024-03-09T08:52:30.597762+0000","last_clean":"2024-03-09T08:52:30.597762+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T08:52:30.597762+0000","last_undegraded":"2024-03-09T08:52:30.597762+0000","last_fullsized":"2024-03-09T08:52:30.597762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:45.372160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575237+0000","last_change":"2024-03-09T08:52:28.573074+0000","last_active":"2024-03-09T08:52:30.575237+0000","last_peered":"2024-03-09T08:52:30.575237+0000","last_clean":"2024-03-09T08:52:30.575237+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T08:52:30.575237+0000","last_undegraded":"2024-03-09T08:52:30.575237+0000","last_fullsized":"2024-03-09T08:52:30.575237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:50.600408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945199+0000","last_change":"2024-03-09T08:52:29.594030+0000","last_active":"2024-03-09T08:52:30.945199+0000","last_peered":"2024-03-09T08:52:30.945199+0000","last_clean":"2024-03-09T08:52:30.945199+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T08:52:30.945199+0000","last_undegraded":"2024-03-09T08:52:30.945199+0000","last_fullsized":"2024-03-09T08:52:30.945199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:57.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948441+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T08:52:30.948441+0000","last_peered":"2024-03-09T08:52:30.948441+0000","last_clean":"2024-03-09T08:52:30.948441+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T08:52:30.948441+0000","last_undegraded":"2024-03-09T08:52:30.948441+0000","last_fullsized":"2024-03-09T08:52:30.948441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947481+0000","last_change":"2024-03-09T08:52:18.230629+0000","last_active":"2024-03-09T08:52:30.947481+0000","last_peered":"2024-03-09T08:52:30.947481+0000","last_clean":"2024-03-09T08:52:30.947481+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T08:52:30.947481+0000","last_undegraded":"2024-03-09T08:52:30.947481+0000","last_fullsized":"2024-03-09T08:52:30.947481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:46:38.528839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576909+0000","last_change":"2024-03-09T08:52:28.577128+0000","last_active":"2024-03-09T08:52:30.576909+0000","last_peered":"2024-03-09T08:52:30.576909+0000","last_clean":"2024-03-09T08:52:30.576909+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T08:52:30.576909+0000","last_undegraded":"2024-03-09T08:52:30.576909+0000","last_fullsized":"2024-03-09T08:52:30.576909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:41.801752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946024+0000","last_change":"2024-03-09T08:52:29.581705+0000","last_active":"2024-03-09T08:52:30.946024+0000","last_peered":"2024-03-09T08:52:30.946024+0000","last_clean":"2024-03-09T08:52:30.946024+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T08:52:30.946024+0000","last_undegraded":"2024-03-09T08:52:30.946024+0000","last_fullsized":"2024-03-09T08:52:30.946024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:27.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947791+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T08:52:30.947791+0000","last_peered":"2024-03-09T08:52:30.947791+0000","last_clean":"2024-03-09T08:52:30.947791+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T08:52:30.947791+0000","last_undegraded":"2024-03-09T08:52:30.947791+0000","last_fullsized":"2024-03-09T08:52:30.947791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947372+0000","last_change":"2024-03-09T08:52:18.228419+0000","last_active":"2024-03-09T08:52:30.947372+0000","last_peered":"2024-03-09T08:52:30.947372+0000","last_clean":"2024-03-09T08:52:30.947372+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T08:52:30.947372+0000","last_undegraded":"2024-03-09T08:52:30.947372+0000","last_fullsized":"2024-03-09T08:52:30.947372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:19:12.533402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577499+0000","last_change":"2024-03-09T08:52:28.577069+0000","last_active":"2024-03-09T08:52:30.577499+0000","last_peered":"2024-03-09T08:52:30.577499+0000","last_clean":"2024-03-09T08:52:30.577499+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T08:52:30.577499+0000","last_undegraded":"2024-03-09T08:52:30.577499+0000","last_fullsized":"2024-03-09T08:52:30.577499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:09:02.410915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601446+0000","last_change":"2024-03-09T08:52:29.588922+0000","last_active":"2024-03-09T08:52:30.601446+0000","last_peered":"2024-03-09T08:52:30.601446+0000","last_clean":"2024-03-09T08:52:30.601446+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T08:52:30.601446+0000","last_undegraded":"2024-03-09T08:52:30.601446+0000","last_fullsized":"2024-03-09T08:52:30.601446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:13.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576353+0000","last_change":"2024-03-09T08:52:18.234372+0000","last_active":"2024-03-09T08:52:30.576353+0000","last_peered":"2024-03-09T08:52:30.576353+0000","last_clean":"2024-03-09T08:52:30.576353+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T08:52:30.576353+0000","last_undegraded":"2024-03-09T08:52:30.576353+0000","last_fullsized":"2024-03-09T08:52:30.576353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:09:33.422907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944406+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T08:52:30.944406+0000","last_peered":"2024-03-09T08:52:30.944406+0000","last_clean":"2024-03-09T08:52:30.944406+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T08:52:30.944406+0000","last_undegraded":"2024-03-09T08:52:30.944406+0000","last_fullsized":"2024-03-09T08:52:30.944406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578381+0000","last_change":"2024-03-09T08:52:28.570631+0000","last_active":"2024-03-09T08:52:30.578381+0000","last_peered":"2024-03-09T08:52:30.578381+0000","last_clean":"2024-03-09T08:52:30.578381+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T08:52:30.578381+0000","last_undegraded":"2024-03-09T08:52:30.578381+0000","last_fullsized":"2024-03-09T08:52:30.578381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:06.630704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943131+0000","last_change":"2024-03-09T08:52:29.596820+0000","last_active":"2024-03-09T08:52:30.943131+0000","last_peered":"2024-03-09T08:52:30.943131+0000","last_clean":"2024-03-09T08:52:30.943131+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T08:52:30.943131+0000","last_undegraded":"2024-03-09T08:52:30.943131+0000","last_fullsized":"2024-03-09T08:52:30.943131+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:01:20.331889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576713+0000","last_change":"2024-03-09T08:52:18.244306+0000","last_active":"2024-03-09T08:52:30.576713+0000","last_peered":"2024-03-09T08:52:30.576713+0000","last_clean":"2024-03-09T08:52:30.576713+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T08:52:30.576713+0000","last_undegraded":"2024-03-09T08:52:30.576713+0000","last_fullsized":"2024-03-09T08:52:30.576713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:58.365524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601528+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T08:52:30.601528+0000","last_peered":"2024-03-09T08:52:30.601528+0000","last_clean":"2024-03-09T08:52:30.601528+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T08:52:30.601528+0000","last_undegraded":"2024-03-09T08:52:30.601528+0000","last_fullsized":"2024-03-09T08:52:30.601528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:45:45.612953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576699+0000","last_change":"2024-03-09T08:52:28.577110+0000","last_active":"2024-03-09T08:52:30.576699+0000","last_peered":"2024-03-09T08:52:30.576699+0000","last_clean":"2024-03-09T08:52:30.576699+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T08:52:30.576699+0000","last_undegraded":"2024-03-09T08:52:30.576699+0000","last_fullsized":"2024-03-09T08:52:30.576699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:57.197373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579257+0000","last_change":"2024-03-09T08:52:29.579929+0000","last_active":"2024-03-09T08:52:30.579257+0000","last_peered":"2024-03-09T08:52:30.579257+0000","last_clean":"2024-03-09T08:52:30.579257+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T08:52:30.579257+0000","last_undegraded":"2024-03-09T08:52:30.579257+0000","last_fullsized":"2024-03-09T08:52:30.579257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:37:23.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576526+0000","last_change":"2024-03-09T08:52:18.233208+0000","last_active":"2024-03-09T08:52:30.576526+0000","last_peered":"2024-03-09T08:52:30.576526+0000","last_clean":"2024-03-09T08:52:30.576526+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T08:52:30.576526+0000","last_undegraded":"2024-03-09T08:52:30.576526+0000","last_fullsized":"2024-03-09T08:52:30.576526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:17.589783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601269+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T08:52:30.601269+0000","last_peered":"2024-03-09T08:52:30.601269+0000","last_clean":"2024-03-09T08:52:30.601269+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T08:52:30.601269+0000","last_undegraded":"2024-03-09T08:52:30.601269+0000","last_fullsized":"2024-03-09T08:52:30.601269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:21:59.990396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602237+0000","last_change":"2024-03-09T08:52:28.576254+0000","last_active":"2024-03-09T08:52:30.602237+0000","last_peered":"2024-03-09T08:52:30.602237+0000","last_clean":"2024-03-09T08:52:30.602237+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T08:52:30.602237+0000","last_undegraded":"2024-03-09T08:52:30.602237+0000","last_fullsized":"2024-03-09T08:52:30.602237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:21:56.330955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946687+0000","last_change":"2024-03-09T08:52:29.578272+0000","last_active":"2024-03-09T08:52:30.946687+0000","last_peered":"2024-03-09T08:52:30.946687+0000","last_clean":"2024-03-09T08:52:30.946687+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T08:52:30.946687+0000","last_undegraded":"2024-03-09T08:52:30.946687+0000","last_fullsized":"2024-03-09T08:52:30.946687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:56.180809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576191+0000","last_change":"2024-03-09T08:52:18.217046+0000","last_active":"2024-03-09T08:52:30.576191+0000","last_peered":"2024-03-09T08:52:30.576191+0000","last_clean":"2024-03-09T08:52:30.576191+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T08:52:30.576191+0000","last_undegraded":"2024-03-09T08:52:30.576191+0000","last_fullsized":"2024-03-09T08:52:30.576191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:14.884963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943757+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T08:52:30.943757+0000","last_peered":"2024-03-09T08:52:30.943757+0000","last_clean":"2024-03-09T08:52:30.943757+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T08:52:30.943757+0000","last_undegraded":"2024-03-09T08:52:30.943757+0000","last_fullsized":"2024-03-09T08:52:30.943757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578428+0000","last_change":"2024-03-09T08:52:28.573638+0000","last_active":"2024-03-09T08:52:30.578428+0000","last_peered":"2024-03-09T08:52:30.578428+0000","last_clean":"2024-03-09T08:52:30.578428+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T08:52:30.578428+0000","last_undegraded":"2024-03-09T08:52:30.578428+0000","last_fullsized":"2024-03-09T08:52:30.578428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:02.478112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575514+0000","last_change":"2024-03-09T08:52:29.597697+0000","last_active":"2024-03-09T08:52:30.575514+0000","last_peered":"2024-03-09T08:52:30.575514+0000","last_clean":"2024-03-09T08:52:30.575514+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T08:52:30.575514+0000","last_undegraded":"2024-03-09T08:52:30.575514+0000","last_fullsized":"2024-03-09T08:52:30.575514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:00.890845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598142+0000","last_change":"2024-03-09T08:52:18.229739+0000","last_active":"2024-03-09T08:52:30.598142+0000","last_peered":"2024-03-09T08:52:30.598142+0000","last_clean":"2024-03-09T08:52:30.598142+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T08:52:30.598142+0000","last_undegraded":"2024-03-09T08:52:30.598142+0000","last_fullsized":"2024-03-09T08:52:30.598142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:24.394779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579139+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T08:52:30.579139+0000","last_peered":"2024-03-09T08:52:30.579139+0000","last_clean":"2024-03-09T08:52:30.579139+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T08:52:30.579139+0000","last_undegraded":"2024-03-09T08:52:30.579139+0000","last_fullsized":"2024-03-09T08:52:30.579139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:35:38.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575645+0000","last_change":"2024-03-09T08:52:28.575630+0000","last_active":"2024-03-09T08:52:30.575645+0000","last_peered":"2024-03-09T08:52:30.575645+0000","last_clean":"2024-03-09T08:52:30.575645+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T08:52:30.575645+0000","last_undegraded":"2024-03-09T08:52:30.575645+0000","last_fullsized":"2024-03-09T08:52:30.575645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:48.210921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575362+0000","last_change":"2024-03-09T08:52:29.576677+0000","last_active":"2024-03-09T08:52:30.575362+0000","last_peered":"2024-03-09T08:52:30.575362+0000","last_clean":"2024-03-09T08:52:30.575362+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T08:52:30.575362+0000","last_undegraded":"2024-03-09T08:52:30.575362+0000","last_fullsized":"2024-03-09T08:52:30.575362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:19.026873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577342+0000","last_change":"2024-03-09T08:52:18.245442+0000","last_active":"2024-03-09T08:52:30.577342+0000","last_peered":"2024-03-09T08:52:30.577342+0000","last_clean":"2024-03-09T08:52:30.577342+0000","last_became_active":"2024-03-09T08:52:18.245164+0000","last_became_peered":"2024-03-09T08:52:18.245164+0000","last_unstale":"2024-03-09T08:52:30.577342+0000","last_undegraded":"2024-03-09T08:52:30.577342+0000","last_fullsized":"2024-03-09T08:52:30.577342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:31:08.413041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602103+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T08:52:30.602103+0000","last_peered":"2024-03-09T08:52:30.602103+0000","last_clean":"2024-03-09T08:52:30.602103+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T08:52:30.602103+0000","last_undegraded":"2024-03-09T08:52:30.602103+0000","last_fullsized":"2024-03-09T08:52:30.602103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:30.201471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575392+0000","last_change":"2024-03-09T08:52:28.586702+0000","last_active":"2024-03-09T08:52:30.575392+0000","last_peered":"2024-03-09T08:52:30.575392+0000","last_clean":"2024-03-09T08:52:30.575392+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T08:52:30.575392+0000","last_undegraded":"2024-03-09T08:52:30.575392+0000","last_fullsized":"2024-03-09T08:52:30.575392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:09.201224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577654+0000","last_change":"2024-03-09T08:52:29.578956+0000","last_active":"2024-03-09T08:52:30.577654+0000","last_peered":"2024-03-09T08:52:30.577654+0000","last_clean":"2024-03-09T08:52:30.577654+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T08:52:30.577654+0000","last_undegraded":"2024-03-09T08:52:30.577654+0000","last_fullsized":"2024-03-09T08:52:30.577654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:00:00.884211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601530+0000","last_change":"2024-03-09T08:52:18.234887+0000","last_active":"2024-03-09T08:52:30.601530+0000","last_peered":"2024-03-09T08:52:30.601530+0000","last_clean":"2024-03-09T08:52:30.601530+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T08:52:30.601530+0000","last_undegraded":"2024-03-09T08:52:30.601530+0000","last_fullsized":"2024-03-09T08:52:30.601530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:44.661882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948828+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T08:52:30.948828+0000","last_peered":"2024-03-09T08:52:30.948828+0000","last_clean":"2024-03-09T08:52:30.948828+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T08:52:30.948828+0000","last_undegraded":"2024-03-09T08:52:30.948828+0000","last_fullsized":"2024-03-09T08:52:30.948828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597558+0000","last_change":"2024-03-09T08:52:28.580879+0000","last_active":"2024-03-09T08:52:30.597558+0000","last_peered":"2024-03-09T08:52:30.597558+0000","last_clean":"2024-03-09T08:52:30.597558+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T08:52:30.597558+0000","last_undegraded":"2024-03-09T08:52:30.597558+0000","last_fullsized":"2024-03-09T08:52:30.597558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:19:44.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598143+0000","last_change":"2024-03-09T08:52:29.580024+0000","last_active":"2024-03-09T08:52:30.598143+0000","last_peered":"2024-03-09T08:52:30.598143+0000","last_clean":"2024-03-09T08:52:30.598143+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T08:52:30.598143+0000","last_undegraded":"2024-03-09T08:52:30.598143+0000","last_fullsized":"2024-03-09T08:52:30.598143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:02:13.195642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947463+0000","last_change":"2024-03-09T08:52:18.250999+0000","last_active":"2024-03-09T08:52:30.947463+0000","last_peered":"2024-03-09T08:52:30.947463+0000","last_clean":"2024-03-09T08:52:30.947463+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T08:52:30.947463+0000","last_undegraded":"2024-03-09T08:52:30.947463+0000","last_fullsized":"2024-03-09T08:52:30.947463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:49:54.496603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601554+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T08:52:30.601554+0000","last_peered":"2024-03-09T08:52:30.601554+0000","last_clean":"2024-03-09T08:52:30.601554+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T08:52:30.601554+0000","last_undegraded":"2024-03-09T08:52:30.601554+0000","last_fullsized":"2024-03-09T08:52:30.601554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601366+0000","last_change":"2024-03-09T08:52:28.584457+0000","last_active":"2024-03-09T08:52:30.601366+0000","last_peered":"2024-03-09T08:52:30.601366+0000","last_clean":"2024-03-09T08:52:30.601366+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T08:52:30.601366+0000","last_undegraded":"2024-03-09T08:52:30.601366+0000","last_fullsized":"2024-03-09T08:52:30.601366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:45.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597465+0000","last_change":"2024-03-09T08:52:29.596486+0000","last_active":"2024-03-09T08:52:30.597465+0000","last_peered":"2024-03-09T08:52:30.597465+0000","last_clean":"2024-03-09T08:52:30.597465+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T08:52:30.597465+0000","last_undegraded":"2024-03-09T08:52:30.597465+0000","last_fullsized":"2024-03-09T08:52:30.597465+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:40:16.587782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577604+0000","last_change":"2024-03-09T08:52:18.227365+0000","last_active":"2024-03-09T08:52:30.577604+0000","last_peered":"2024-03-09T08:52:30.577604+0000","last_clean":"2024-03-09T08:52:30.577604+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T08:52:30.577604+0000","last_undegraded":"2024-03-09T08:52:30.577604+0000","last_fullsized":"2024-03-09T08:52:30.577604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:26.357063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945950+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T08:52:30.945950+0000","last_peered":"2024-03-09T08:52:30.945950+0000","last_clean":"2024-03-09T08:52:30.945950+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T08:52:30.945950+0000","last_undegraded":"2024-03-09T08:52:30.945950+0000","last_fullsized":"2024-03-09T08:52:30.945950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601516+0000","last_change":"2024-03-09T08:52:28.578096+0000","last_active":"2024-03-09T08:52:30.601516+0000","last_peered":"2024-03-09T08:52:30.601516+0000","last_clean":"2024-03-09T08:52:30.601516+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T08:52:30.601516+0000","last_undegraded":"2024-03-09T08:52:30.601516+0000","last_fullsized":"2024-03-09T08:52:30.601516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:42.645517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943375+0000","last_change":"2024-03-09T08:52:29.590834+0000","last_active":"2024-03-09T08:52:30.943375+0000","last_peered":"2024-03-09T08:52:30.943375+0000","last_clean":"2024-03-09T08:52:30.943375+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T08:52:30.943375+0000","last_undegraded":"2024-03-09T08:52:30.943375+0000","last_fullsized":"2024-03-09T08:52:30.943375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:24:06.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.600924+0000","last_change":"2024-03-09T08:52:18.235832+0000","last_active":"2024-03-09T08:52:30.600924+0000","last_peered":"2024-03-09T08:52:30.600924+0000","last_clean":"2024-03-09T08:52:30.600924+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T08:52:30.600924+0000","last_undegraded":"2024-03-09T08:52:30.600924+0000","last_fullsized":"2024-03-09T08:52:30.600924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:38:58.751526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942505+0000","last_change":"2024-03-09T08:52:17.271847+0000","last_active":"2024-03-09T08:52:30.942505+0000","last_peered":"2024-03-09T08:52:30.942505+0000","last_clean":"2024-03-09T08:52:30.942505+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T08:52:30.942505+0000","last_undegraded":"2024-03-09T08:52:30.942505+0000","last_fullsized":"2024-03-09T08:52:30.942505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:27.418674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602301+0000","last_change":"2024-03-09T08:52:28.588636+0000","last_active":"2024-03-09T08:52:30.602301+0000","last_peered":"2024-03-09T08:52:30.602301+0000","last_clean":"2024-03-09T08:52:30.602301+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T08:52:30.602301+0000","last_undegraded":"2024-03-09T08:52:30.602301+0000","last_fullsized":"2024-03-09T08:52:30.602301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:21.601182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947090+0000","last_change":"2024-03-09T08:52:29.606833+0000","last_active":"2024-03-09T08:52:30.947090+0000","last_peered":"2024-03-09T08:52:30.947090+0000","last_clean":"2024-03-09T08:52:30.947090+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T08:52:30.947090+0000","last_undegraded":"2024-03-09T08:52:30.947090+0000","last_fullsized":"2024-03-09T08:52:30.947090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:40.299681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597339+0000","last_change":"2024-03-09T08:52:18.214396+0000","last_active":"2024-03-09T08:52:30.597339+0000","last_peered":"2024-03-09T08:52:30.597339+0000","last_clean":"2024-03-09T08:52:30.597339+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T08:52:30.597339+0000","last_undegraded":"2024-03-09T08:52:30.597339+0000","last_fullsized":"2024-03-09T08:52:30.597339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:24:10.719351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944007+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T08:52:30.944007+0000","last_peered":"2024-03-09T08:52:30.944007+0000","last_clean":"2024-03-09T08:52:30.944007+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T08:52:30.944007+0000","last_undegraded":"2024-03-09T08:52:30.944007+0000","last_fullsized":"2024-03-09T08:52:30.944007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.599908+0000","last_change":"2024-03-09T08:52:28.578599+0000","last_active":"2024-03-09T08:52:30.599908+0000","last_peered":"2024-03-09T08:52:30.599908+0000","last_clean":"2024-03-09T08:52:30.599908+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T08:52:30.599908+0000","last_undegraded":"2024-03-09T08:52:30.599908+0000","last_fullsized":"2024-03-09T08:52:30.599908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:52:31.328248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943920+0000","last_change":"2024-03-09T08:52:29.591872+0000","last_active":"2024-03-09T08:52:30.943920+0000","last_peered":"2024-03-09T08:52:30.943920+0000","last_clean":"2024-03-09T08:52:30.943920+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T08:52:30.943920+0000","last_undegraded":"2024-03-09T08:52:30.943920+0000","last_fullsized":"2024-03-09T08:52:30.943920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:44:17.870183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576290+0000","last_change":"2024-03-09T08:52:18.235367+0000","last_active":"2024-03-09T08:52:30.576290+0000","last_peered":"2024-03-09T08:52:30.576290+0000","last_clean":"2024-03-09T08:52:30.576290+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T08:52:30.576290+0000","last_undegraded":"2024-03-09T08:52:30.576290+0000","last_fullsized":"2024-03-09T08:52:30.576290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:40.736440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946321+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T08:52:30.946321+0000","last_peered":"2024-03-09T08:52:30.946321+0000","last_clean":"2024-03-09T08:52:30.946321+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T08:52:30.946321+0000","last_undegraded":"2024-03-09T08:52:30.946321+0000","last_fullsized":"2024-03-09T08:52:30.946321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944699+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T08:52:30.944699+0000","last_peered":"2024-03-09T08:52:30.944699+0000","last_clean":"2024-03-09T08:52:30.944699+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T08:52:30.944699+0000","last_undegraded":"2024-03-09T08:52:30.944699+0000","last_fullsized":"2024-03-09T08:52:30.944699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577815+0000","last_change":"2024-03-09T08:52:28.577973+0000","last_active":"2024-03-09T08:52:30.577815+0000","last_peered":"2024-03-09T08:52:30.577815+0000","last_clean":"2024-03-09T08:52:30.577815+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T08:52:30.577815+0000","last_undegraded":"2024-03-09T08:52:30.577815+0000","last_fullsized":"2024-03-09T08:52:30.577815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:13.291622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943332+0000","last_change":"2024-03-09T08:52:29.599601+0000","last_active":"2024-03-09T08:52:30.943332+0000","last_peered":"2024-03-09T08:52:30.943332+0000","last_clean":"2024-03-09T08:52:30.943332+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T08:52:30.943332+0000","last_undegraded":"2024-03-09T08:52:30.943332+0000","last_fullsized":"2024-03-09T08:52:30.943332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:11:17.424330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948439+0000","last_change":"2024-03-09T08:52:18.242671+0000","last_active":"2024-03-09T08:52:30.948439+0000","last_peered":"2024-03-09T08:52:30.948439+0000","last_clean":"2024-03-09T08:52:30.948439+0000","last_became_active":"2024-03-09T08:52:18.242258+0000","last_became_peered":"2024-03-09T08:52:18.242258+0000","last_unstale":"2024-03-09T08:52:30.948439+0000","last_undegraded":"2024-03-09T08:52:30.948439+0000","last_fullsized":"2024-03-09T08:52:30.948439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:15.030669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943179+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T08:52:30.943179+0000","last_peered":"2024-03-09T08:52:30.943179+0000","last_clean":"2024-03-09T08:52:30.943179+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T08:52:30.943179+0000","last_undegraded":"2024-03-09T08:52:30.943179+0000","last_fullsized":"2024-03-09T08:52:30.943179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:27.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576159+0000","last_change":"2024-03-09T08:52:28.589278+0000","last_active":"2024-03-09T08:52:30.576159+0000","last_peered":"2024-03-09T08:52:30.576159+0000","last_clean":"2024-03-09T08:52:30.576159+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T08:52:30.576159+0000","last_undegraded":"2024-03-09T08:52:30.576159+0000","last_fullsized":"2024-03-09T08:52:30.576159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:16:26.449275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942717+0000","last_change":"2024-03-09T08:52:29.600332+0000","last_active":"2024-03-09T08:52:30.942717+0000","last_peered":"2024-03-09T08:52:30.942717+0000","last_clean":"2024-03-09T08:52:30.942717+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T08:52:30.942717+0000","last_undegraded":"2024-03-09T08:52:30.942717+0000","last_fullsized":"2024-03-09T08:52:30.942717+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:36:41.422546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947852+0000","last_change":"2024-03-09T08:52:18.250433+0000","last_active":"2024-03-09T08:52:30.947852+0000","last_peered":"2024-03-09T08:52:30.947852+0000","last_clean":"2024-03-09T08:52:30.947852+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T08:52:30.947852+0000","last_undegraded":"2024-03-09T08:52:30.947852+0000","last_fullsized":"2024-03-09T08:52:30.947852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:29.742337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602070+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T08:52:30.602070+0000","last_peered":"2024-03-09T08:52:30.602070+0000","last_clean":"2024-03-09T08:52:30.602070+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T08:52:30.602070+0000","last_undegraded":"2024-03-09T08:52:30.602070+0000","last_fullsized":"2024-03-09T08:52:30.602070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:26:35.574903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597442+0000","last_change":"2024-03-09T08:52:28.583181+0000","last_active":"2024-03-09T08:52:30.597442+0000","last_peered":"2024-03-09T08:52:30.597442+0000","last_clean":"2024-03-09T08:52:30.597442+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T08:52:30.597442+0000","last_undegraded":"2024-03-09T08:52:30.597442+0000","last_fullsized":"2024-03-09T08:52:30.597442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:58:50.061324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943982+0000","last_change":"2024-03-09T08:52:29.590141+0000","last_active":"2024-03-09T08:52:30.943982+0000","last_peered":"2024-03-09T08:52:30.943982+0000","last_clean":"2024-03-09T08:52:30.943982+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T08:52:30.943982+0000","last_undegraded":"2024-03-09T08:52:30.943982+0000","last_fullsized":"2024-03-09T08:52:30.943982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:23.981897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948851+0000","last_change":"2024-03-09T08:52:18.250420+0000","last_active":"2024-03-09T08:52:30.948851+0000","last_peered":"2024-03-09T08:52:30.948851+0000","last_clean":"2024-03-09T08:52:30.948851+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T08:52:30.948851+0000","last_undegraded":"2024-03-09T08:52:30.948851+0000","last_fullsized":"2024-03-09T08:52:30.948851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:48.826066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943461+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T08:52:30.943461+0000","last_peered":"2024-03-09T08:52:30.943461+0000","last_clean":"2024-03-09T08:52:30.943461+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T08:52:30.943461+0000","last_undegraded":"2024-03-09T08:52:30.943461+0000","last_fullsized":"2024-03-09T08:52:30.943461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:16.138216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944811+0000","last_change":"2024-03-09T08:52:28.587637+0000","last_active":"2024-03-09T08:52:30.944811+0000","last_peered":"2024-03-09T08:52:30.944811+0000","last_clean":"2024-03-09T08:52:30.944811+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T08:52:30.944811+0000","last_undegraded":"2024-03-09T08:52:30.944811+0000","last_fullsized":"2024-03-09T08:52:30.944811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:31:32.362061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942477+0000","last_change":"2024-03-09T08:52:29.586180+0000","last_active":"2024-03-09T08:52:30.942477+0000","last_peered":"2024-03-09T08:52:30.942477+0000","last_clean":"2024-03-09T08:52:30.942477+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T08:52:30.942477+0000","last_undegraded":"2024-03-09T08:52:30.942477+0000","last_fullsized":"2024-03-09T08:52:30.942477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:35:21.522181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948836+0000","last_change":"2024-03-09T08:52:18.246145+0000","last_active":"2024-03-09T08:52:30.948836+0000","last_peered":"2024-03-09T08:52:30.948836+0000","last_clean":"2024-03-09T08:52:30.948836+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T08:52:30.948836+0000","last_undegraded":"2024-03-09T08:52:30.948836+0000","last_fullsized":"2024-03-09T08:52:30.948836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:29:36.213774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578625+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T08:52:30.578625+0000","last_peered":"2024-03-09T08:52:30.578625+0000","last_clean":"2024-03-09T08:52:30.578625+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T08:52:30.578625+0000","last_undegraded":"2024-03-09T08:52:30.578625+0000","last_fullsized":"2024-03-09T08:52:30.578625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:10:34.698235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575615+0000","last_change":"2024-03-09T08:52:28.582308+0000","last_active":"2024-03-09T08:52:30.575615+0000","last_peered":"2024-03-09T08:52:30.575615+0000","last_clean":"2024-03-09T08:52:30.575615+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T08:52:30.575615+0000","last_undegraded":"2024-03-09T08:52:30.575615+0000","last_fullsized":"2024-03-09T08:52:30.575615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:04.918893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577162+0000","last_change":"2024-03-09T08:52:29.610596+0000","last_active":"2024-03-09T08:52:30.577162+0000","last_peered":"2024-03-09T08:52:30.577162+0000","last_clean":"2024-03-09T08:52:30.577162+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T08:52:30.577162+0000","last_undegraded":"2024-03-09T08:52:30.577162+0000","last_fullsized":"2024-03-09T08:52:30.577162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:19:47.313981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944192+0000","last_change":"2024-03-09T08:52:18.248755+0000","last_active":"2024-03-09T08:52:30.944192+0000","last_peered":"2024-03-09T08:52:30.944192+0000","last_clean":"2024-03-09T08:52:30.944192+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T08:52:30.944192+0000","last_undegraded":"2024-03-09T08:52:30.944192+0000","last_fullsized":"2024-03-09T08:52:30.944192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:37.083738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942668+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T08:52:30.942668+0000","last_peered":"2024-03-09T08:52:30.942668+0000","last_clean":"2024-03-09T08:52:30.942668+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T08:52:30.942668+0000","last_undegraded":"2024-03-09T08:52:30.942668+0000","last_fullsized":"2024-03-09T08:52:30.942668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:31.859924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598089+0000","last_change":"2024-03-09T08:52:28.575975+0000","last_active":"2024-03-09T08:52:30.598089+0000","last_peered":"2024-03-09T08:52:30.598089+0000","last_clean":"2024-03-09T08:52:30.598089+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T08:52:30.598089+0000","last_undegraded":"2024-03-09T08:52:30.598089+0000","last_fullsized":"2024-03-09T08:52:30.598089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:51:24.405316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947555+0000","last_change":"2024-03-09T08:52:29.575752+0000","last_active":"2024-03-09T08:52:30.947555+0000","last_peered":"2024-03-09T08:52:30.947555+0000","last_clean":"2024-03-09T08:52:30.947555+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T08:52:30.947555+0000","last_undegraded":"2024-03-09T08:52:30.947555+0000","last_fullsized":"2024-03-09T08:52:30.947555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:31:51.847871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575309+0000","last_change":"2024-03-09T08:52:18.245073+0000","last_active":"2024-03-09T08:52:30.575309+0000","last_peered":"2024-03-09T08:52:30.575309+0000","last_clean":"2024-03-09T08:52:30.575309+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T08:52:30.575309+0000","last_undegraded":"2024-03-09T08:52:30.575309+0000","last_fullsized":"2024-03-09T08:52:30.575309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:40:15.390735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577484+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T08:52:30.577484+0000","last_peered":"2024-03-09T08:52:30.577484+0000","last_clean":"2024-03-09T08:52:30.577484+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T08:52:30.577484+0000","last_undegraded":"2024-03-09T08:52:30.577484+0000","last_fullsized":"2024-03-09T08:52:30.577484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:07:29.577461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577273+0000","last_change":"2024-03-09T08:52:28.588312+0000","last_active":"2024-03-09T08:52:30.577273+0000","last_peered":"2024-03-09T08:52:30.577273+0000","last_clean":"2024-03-09T08:52:30.577273+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T08:52:30.577273+0000","last_undegraded":"2024-03-09T08:52:30.577273+0000","last_fullsized":"2024-03-09T08:52:30.577273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:14:52.895618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946368+0000","last_change":"2024-03-09T08:52:29.596647+0000","last_active":"2024-03-09T08:52:30.946368+0000","last_peered":"2024-03-09T08:52:30.946368+0000","last_clean":"2024-03-09T08:52:30.946368+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T08:52:30.946368+0000","last_undegraded":"2024-03-09T08:52:30.946368+0000","last_fullsized":"2024-03-09T08:52:30.946368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:35:28.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944097+0000","last_change":"2024-03-09T08:52:18.248093+0000","last_active":"2024-03-09T08:52:30.944097+0000","last_peered":"2024-03-09T08:52:30.944097+0000","last_clean":"2024-03-09T08:52:30.944097+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T08:52:30.944097+0000","last_undegraded":"2024-03-09T08:52:30.944097+0000","last_fullsized":"2024-03-09T08:52:30.944097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:29.414335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945767+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T08:52:30.945767+0000","last_peered":"2024-03-09T08:52:30.945767+0000","last_clean":"2024-03-09T08:52:30.945767+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T08:52:30.945767+0000","last_undegraded":"2024-03-09T08:52:30.945767+0000","last_fullsized":"2024-03-09T08:52:30.945767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576830+0000","last_change":"2024-03-09T08:52:28.585562+0000","last_active":"2024-03-09T08:52:30.576830+0000","last_peered":"2024-03-09T08:52:30.576830+0000","last_clean":"2024-03-09T08:52:30.576830+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T08:52:30.576830+0000","last_undegraded":"2024-03-09T08:52:30.576830+0000","last_fullsized":"2024-03-09T08:52:30.576830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:28:50.544795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576833+0000","last_change":"2024-03-09T08:52:29.581330+0000","last_active":"2024-03-09T08:52:30.576833+0000","last_peered":"2024-03-09T08:52:30.576833+0000","last_clean":"2024-03-09T08:52:30.576833+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T08:52:30.576833+0000","last_undegraded":"2024-03-09T08:52:30.576833+0000","last_fullsized":"2024-03-09T08:52:30.576833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:55:38.041794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597506+0000","last_change":"2024-03-09T08:52:18.236146+0000","last_active":"2024-03-09T08:52:30.597506+0000","last_peered":"2024-03-09T08:52:30.597506+0000","last_clean":"2024-03-09T08:52:30.597506+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T08:52:30.597506+0000","last_undegraded":"2024-03-09T08:52:30.597506+0000","last_fullsized":"2024-03-09T08:52:30.597506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:18:51.511926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598307+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T08:52:30.598307+0000","last_peered":"2024-03-09T08:52:30.598307+0000","last_clean":"2024-03-09T08:52:30.598307+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T08:52:30.598307+0000","last_undegraded":"2024-03-09T08:52:30.598307+0000","last_fullsized":"2024-03-09T08:52:30.598307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:22:05.504557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947315+0000","last_change":"2024-03-09T08:52:28.585697+0000","last_active":"2024-03-09T08:52:30.947315+0000","last_peered":"2024-03-09T08:52:30.947315+0000","last_clean":"2024-03-09T08:52:30.947315+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T08:52:30.947315+0000","last_undegraded":"2024-03-09T08:52:30.947315+0000","last_fullsized":"2024-03-09T08:52:30.947315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:50:23.095008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576346+0000","last_change":"2024-03-09T08:52:29.578803+0000","last_active":"2024-03-09T08:52:30.576346+0000","last_peered":"2024-03-09T08:52:30.576346+0000","last_clean":"2024-03-09T08:52:30.576346+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T08:52:30.576346+0000","last_undegraded":"2024-03-09T08:52:30.576346+0000","last_fullsized":"2024-03-09T08:52:30.576346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:22:47.690873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948118+0000","last_change":"2024-03-09T08:52:18.242496+0000","last_active":"2024-03-09T08:52:30.948118+0000","last_peered":"2024-03-09T08:52:30.948118+0000","last_clean":"2024-03-09T08:52:30.948118+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T08:52:30.948118+0000","last_undegraded":"2024-03-09T08:52:30.948118+0000","last_fullsized":"2024-03-09T08:52:30.948118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:35:17.142983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578358+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T08:52:30.578358+0000","last_peered":"2024-03-09T08:52:30.578358+0000","last_clean":"2024-03-09T08:52:30.578358+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T08:52:30.578358+0000","last_undegraded":"2024-03-09T08:52:30.578358+0000","last_fullsized":"2024-03-09T08:52:30.578358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:01.724043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601055+0000","last_change":"2024-03-09T08:52:28.583685+0000","last_active":"2024-03-09T08:52:30.601055+0000","last_peered":"2024-03-09T08:52:30.601055+0000","last_clean":"2024-03-09T08:52:30.601055+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T08:52:30.601055+0000","last_undegraded":"2024-03-09T08:52:30.601055+0000","last_fullsized":"2024-03-09T08:52:30.601055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:58:00.899620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601468+0000","last_change":"2024-03-09T08:52:29.585912+0000","last_active":"2024-03-09T08:52:30.601468+0000","last_peered":"2024-03-09T08:52:30.601468+0000","last_clean":"2024-03-09T08:52:30.601468+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T08:52:30.601468+0000","last_undegraded":"2024-03-09T08:52:30.601468+0000","last_fullsized":"2024-03-09T08:52:30.601468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:24:47.261191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577039+0000","last_change":"2024-03-09T08:52:18.244654+0000","last_active":"2024-03-09T08:52:30.577039+0000","last_peered":"2024-03-09T08:52:30.577039+0000","last_clean":"2024-03-09T08:52:30.577039+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T08:52:30.577039+0000","last_undegraded":"2024-03-09T08:52:30.577039+0000","last_fullsized":"2024-03-09T08:52:30.577039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:08:58.611725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601999+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T08:52:30.601999+0000","last_peered":"2024-03-09T08:52:30.601999+0000","last_clean":"2024-03-09T08:52:30.601999+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T08:52:30.601999+0000","last_undegraded":"2024-03-09T08:52:30.601999+0000","last_fullsized":"2024-03-09T08:52:30.601999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:41:10.578669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577095+0000","last_change":"2024-03-09T08:52:28.568794+0000","last_active":"2024-03-09T08:52:30.577095+0000","last_peered":"2024-03-09T08:52:30.577095+0000","last_clean":"2024-03-09T08:52:30.577095+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T08:52:30.577095+0000","last_undegraded":"2024-03-09T08:52:30.577095+0000","last_fullsized":"2024-03-09T08:52:30.577095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:44:21.721511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577942+0000","last_change":"2024-03-09T08:52:29.586705+0000","last_active":"2024-03-09T08:52:30.577942+0000","last_peered":"2024-03-09T08:52:30.577942+0000","last_clean":"2024-03-09T08:52:30.577942+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T08:52:30.577942+0000","last_undegraded":"2024-03-09T08:52:30.577942+0000","last_fullsized":"2024-03-09T08:52:30.577942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:26.353156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597722+0000","last_change":"2024-03-09T08:52:18.236628+0000","last_active":"2024-03-09T08:52:30.597722+0000","last_peered":"2024-03-09T08:52:30.597722+0000","last_clean":"2024-03-09T08:52:30.597722+0000","last_became_active":"2024-03-09T08:52:18.235321+0000","last_became_peered":"2024-03-09T08:52:18.235321+0000","last_unstale":"2024-03-09T08:52:30.597722+0000","last_undegraded":"2024-03-09T08:52:30.597722+0000","last_fullsized":"2024-03-09T08:52:30.597722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:13:59.221325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948276+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T08:52:30.948276+0000","last_peered":"2024-03-09T08:52:30.948276+0000","last_clean":"2024-03-09T08:52:30.948276+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T08:52:30.948276+0000","last_undegraded":"2024-03-09T08:52:30.948276+0000","last_fullsized":"2024-03-09T08:52:30.948276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575808+0000","last_change":"2024-03-09T08:52:28.583697+0000","last_active":"2024-03-09T08:52:30.575808+0000","last_peered":"2024-03-09T08:52:30.575808+0000","last_clean":"2024-03-09T08:52:30.575808+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T08:52:30.575808+0000","last_undegraded":"2024-03-09T08:52:30.575808+0000","last_fullsized":"2024-03-09T08:52:30.575808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:41.552156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945076+0000","last_change":"2024-03-09T08:52:29.590854+0000","last_active":"2024-03-09T08:52:30.945076+0000","last_peered":"2024-03-09T08:52:30.945076+0000","last_clean":"2024-03-09T08:52:30.945076+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T08:52:30.945076+0000","last_undegraded":"2024-03-09T08:52:30.945076+0000","last_fullsized":"2024-03-09T08:52:30.945076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:09.568826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947914+0000","last_change":"2024-03-09T08:52:18.251648+0000","last_active":"2024-03-09T08:52:30.947914+0000","last_peered":"2024-03-09T08:52:30.947914+0000","last_clean":"2024-03-09T08:52:30.947914+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T08:52:30.947914+0000","last_undegraded":"2024-03-09T08:52:30.947914+0000","last_fullsized":"2024-03-09T08:52:30.947914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:54:45.721673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579118+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T08:52:30.579118+0000","last_peered":"2024-03-09T08:52:30.579118+0000","last_clean":"2024-03-09T08:52:30.579118+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T08:52:30.579118+0000","last_undegraded":"2024-03-09T08:52:30.579118+0000","last_fullsized":"2024-03-09T08:52:30.579118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:31.731626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575378+0000","last_change":"2024-03-09T08:52:28.586606+0000","last_active":"2024-03-09T08:52:30.575378+0000","last_peered":"2024-03-09T08:52:30.575378+0000","last_clean":"2024-03-09T08:52:30.575378+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T08:52:30.575378+0000","last_undegraded":"2024-03-09T08:52:30.575378+0000","last_fullsized":"2024-03-09T08:52:30.575378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:17:40.902403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578219+0000","last_change":"2024-03-09T08:52:29.584512+0000","last_active":"2024-03-09T08:52:30.578219+0000","last_peered":"2024-03-09T08:52:30.578219+0000","last_clean":"2024-03-09T08:52:30.578219+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T08:52:30.578219+0000","last_undegraded":"2024-03-09T08:52:30.578219+0000","last_fullsized":"2024-03-09T08:52:30.578219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:39:20.330443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597975+0000","last_change":"2024-03-09T08:52:28.586201+0000","last_active":"2024-03-09T08:52:30.597975+0000","last_peered":"2024-03-09T08:52:30.597975+0000","last_clean":"2024-03-09T08:52:30.597975+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T08:52:30.597975+0000","last_undegraded":"2024-03-09T08:52:30.597975+0000","last_fullsized":"2024-03-09T08:52:30.597975+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:28:03.471646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601039+0000","last_change":"2024-03-09T08:52:18.241156+0000","last_active":"2024-03-09T08:52:30.601039+0000","last_peered":"2024-03-09T08:52:30.601039+0000","last_clean":"2024-03-09T08:52:30.601039+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T08:52:30.601039+0000","last_undegraded":"2024-03-09T08:52:30.601039+0000","last_fullsized":"2024-03-09T08:52:30.601039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:55.323770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946394+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T08:52:30.946394+0000","last_peered":"2024-03-09T08:52:30.946394+0000","last_clean":"2024-03-09T08:52:30.946394+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T08:52:30.946394+0000","last_undegraded":"2024-03-09T08:52:30.946394+0000","last_fullsized":"2024-03-09T08:52:30.946394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597714+0000","last_change":"2024-03-09T08:52:29.591085+0000","last_active":"2024-03-09T08:52:30.597714+0000","last_peered":"2024-03-09T08:52:30.597714+0000","last_clean":"2024-03-09T08:52:30.597714+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T08:52:30.597714+0000","last_undegraded":"2024-03-09T08:52:30.597714+0000","last_fullsized":"2024-03-09T08:52:30.597714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:40:27.878922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578971+0000","last_change":"2024-03-09T08:52:28.585631+0000","last_active":"2024-03-09T08:52:30.578971+0000","last_peered":"2024-03-09T08:52:30.578971+0000","last_clean":"2024-03-09T08:52:30.578971+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T08:52:30.578971+0000","last_undegraded":"2024-03-09T08:52:30.578971+0000","last_fullsized":"2024-03-09T08:52:30.578971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:06:03.569759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576252+0000","last_change":"2024-03-09T08:52:18.234400+0000","last_active":"2024-03-09T08:52:30.576252+0000","last_peered":"2024-03-09T08:52:30.576252+0000","last_clean":"2024-03-09T08:52:30.576252+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T08:52:30.576252+0000","last_undegraded":"2024-03-09T08:52:30.576252+0000","last_fullsized":"2024-03-09T08:52:30.576252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:42:14.057367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576643+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T08:52:30.576643+0000","last_peered":"2024-03-09T08:52:30.576643+0000","last_clean":"2024-03-09T08:52:30.576643+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T08:52:30.576643+0000","last_undegraded":"2024-03-09T08:52:30.576643+0000","last_fullsized":"2024-03-09T08:52:30.576643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601916+0000","last_change":"2024-03-09T08:52:29.588928+0000","last_active":"2024-03-09T08:52:30.601916+0000","last_peered":"2024-03-09T08:52:30.601916+0000","last_clean":"2024-03-09T08:52:30.601916+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T08:52:30.601916+0000","last_undegraded":"2024-03-09T08:52:30.601916+0000","last_fullsized":"2024-03-09T08:52:30.601916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:32:27.061041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576515+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T08:52:30.576515+0000","last_peered":"2024-03-09T08:52:30.576515+0000","last_clean":"2024-03-09T08:52:30.576515+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T08:52:30.576515+0000","last_undegraded":"2024-03-09T08:52:30.576515+0000","last_fullsized":"2024-03-09T08:52:30.576515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598213+0000","last_change":"2024-03-09T08:52:18.231141+0000","last_active":"2024-03-09T08:52:30.598213+0000","last_peered":"2024-03-09T08:52:30.598213+0000","last_clean":"2024-03-09T08:52:30.598213+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T08:52:30.598213+0000","last_undegraded":"2024-03-09T08:52:30.598213+0000","last_fullsized":"2024-03-09T08:52:30.598213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:25:00.471222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943012+0000","last_change":"2024-03-09T08:52:28.579315+0000","last_active":"2024-03-09T08:52:30.943012+0000","last_peered":"2024-03-09T08:52:30.943012+0000","last_clean":"2024-03-09T08:52:30.943012+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T08:52:30.943012+0000","last_undegraded":"2024-03-09T08:52:30.943012+0000","last_fullsized":"2024-03-09T08:52:30.943012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:57:17.301677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576561+0000","last_change":"2024-03-09T08:52:29.585154+0000","last_active":"2024-03-09T08:52:30.576561+0000","last_peered":"2024-03-09T08:52:30.576561+0000","last_clean":"2024-03-09T08:52:30.576561+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T08:52:30.576561+0000","last_undegraded":"2024-03-09T08:52:30.576561+0000","last_fullsized":"2024-03-09T08:52:30.576561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:43:35.679108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576704+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T08:52:30.576704+0000","last_peered":"2024-03-09T08:52:30.576704+0000","last_clean":"2024-03-09T08:52:30.576704+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T08:52:30.576704+0000","last_undegraded":"2024-03-09T08:52:30.576704+0000","last_fullsized":"2024-03-09T08:52:30.576704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577869+0000","last_change":"2024-03-09T08:52:18.229533+0000","last_active":"2024-03-09T08:52:30.577869+0000","last_peered":"2024-03-09T08:52:30.577869+0000","last_clean":"2024-03-09T08:52:30.577869+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T08:52:30.577869+0000","last_undegraded":"2024-03-09T08:52:30.577869+0000","last_fullsized":"2024-03-09T08:52:30.577869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:17.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576387+0000","last_change":"2024-03-09T08:52:28.585877+0000","last_active":"2024-03-09T08:52:30.576387+0000","last_peered":"2024-03-09T08:52:30.576387+0000","last_clean":"2024-03-09T08:52:30.576387+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T08:52:30.576387+0000","last_undegraded":"2024-03-09T08:52:30.576387+0000","last_fullsized":"2024-03-09T08:52:30.576387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:30:20.036385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577546+0000","last_change":"2024-03-09T08:52:29.592253+0000","last_active":"2024-03-09T08:52:30.577546+0000","last_peered":"2024-03-09T08:52:30.577546+0000","last_clean":"2024-03-09T08:52:30.577546+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T08:52:30.577546+0000","last_undegraded":"2024-03-09T08:52:30.577546+0000","last_fullsized":"2024-03-09T08:52:30.577546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:50:33.525541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575154+0000","last_change":"2024-03-09T08:52:18.237869+0000","last_active":"2024-03-09T08:52:30.575154+0000","last_peered":"2024-03-09T08:52:30.575154+0000","last_clean":"2024-03-09T08:52:30.575154+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T08:52:30.575154+0000","last_undegraded":"2024-03-09T08:52:30.575154+0000","last_fullsized":"2024-03-09T08:52:30.575154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:01.571866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602125+0000","last_change":"2024-03-09T08:52:29.601300+0000","last_active":"2024-03-09T08:52:30.602125+0000","last_peered":"2024-03-09T08:52:30.602125+0000","last_clean":"2024-03-09T08:52:30.602125+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T08:52:30.602125+0000","last_undegraded":"2024-03-09T08:52:30.602125+0000","last_fullsized":"2024-03-09T08:52:30.602125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:46:04.870574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942394+0000","last_change":"2024-03-09T08:52:18.226980+0000","last_active":"2024-03-09T08:52:30.942394+0000","last_peered":"2024-03-09T08:52:30.942394+0000","last_clean":"2024-03-09T08:52:30.942394+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T08:52:30.942394+0000","last_undegraded":"2024-03-09T08:52:30.942394+0000","last_fullsized":"2024-03-09T08:52:30.942394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:28.312576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946757+0000","last_change":"2024-03-09T08:52:29.575612+0000","last_active":"2024-03-09T08:52:30.946757+0000","last_peered":"2024-03-09T08:52:30.946757+0000","last_clean":"2024-03-09T08:52:30.946757+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T08:52:30.946757+0000","last_undegraded":"2024-03-09T08:52:30.946757+0000","last_fullsized":"2024-03-09T08:52:30.946757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:58.852983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945644+0000","last_change":"2024-03-09T08:52:18.241628+0000","last_active":"2024-03-09T08:52:30.945644+0000","last_peered":"2024-03-09T08:52:30.945644+0000","last_clean":"2024-03-09T08:52:30.945644+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T08:52:30.945644+0000","last_undegraded":"2024-03-09T08:52:30.945644+0000","last_fullsized":"2024-03-09T08:52:30.945644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:07:58.570641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601917+0000","last_change":"2024-03-09T08:52:29.588660+0000","last_active":"2024-03-09T08:52:30.601917+0000","last_peered":"2024-03-09T08:52:30.601917+0000","last_clean":"2024-03-09T08:52:30.601917+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T08:52:30.601917+0000","last_undegraded":"2024-03-09T08:52:30.601917+0000","last_fullsized":"2024-03-09T08:52:30.601917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:14.305320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598085+0000","last_change":"2024-03-09T08:52:18.242257+0000","last_active":"2024-03-09T08:52:30.598085+0000","last_peered":"2024-03-09T08:52:30.598085+0000","last_clean":"2024-03-09T08:52:30.598085+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T08:52:30.598085+0000","last_undegraded":"2024-03-09T08:52:30.598085+0000","last_fullsized":"2024-03-09T08:52:30.598085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:40:00.854066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944026+0000","last_change":"2024-03-09T08:52:29.595057+0000","last_active":"2024-03-09T08:52:30.944026+0000","last_peered":"2024-03-09T08:52:30.944026+0000","last_clean":"2024-03-09T08:52:30.944026+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T08:52:30.944026+0000","last_undegraded":"2024-03-09T08:52:30.944026+0000","last_fullsized":"2024-03-09T08:52:30.944026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:33:37.474266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577091+0000","last_change":"2024-03-09T08:52:18.239951+0000","last_active":"2024-03-09T08:52:30.577091+0000","last_peered":"2024-03-09T08:52:30.577091+0000","last_clean":"2024-03-09T08:52:30.577091+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T08:52:30.577091+0000","last_undegraded":"2024-03-09T08:52:30.577091+0000","last_fullsized":"2024-03-09T08:52:30.577091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:51.102540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576730+0000","last_change":"2024-03-09T08:52:29.578993+0000","last_active":"2024-03-09T08:52:30.576730+0000","last_peered":"2024-03-09T08:52:30.576730+0000","last_clean":"2024-03-09T08:52:30.576730+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T08:52:30.576730+0000","last_undegraded":"2024-03-09T08:52:30.576730+0000","last_fullsized":"2024-03-09T08:52:30.576730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:50:53.692060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601995+0000","last_change":"2024-03-09T08:52:18.241038+0000","last_active":"2024-03-09T08:52:30.601995+0000","last_peered":"2024-03-09T08:52:30.601995+0000","last_clean":"2024-03-09T08:52:30.601995+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T08:52:30.601995+0000","last_undegraded":"2024-03-09T08:52:30.601995+0000","last_fullsized":"2024-03-09T08:52:30.601995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:02.169113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943444+0000","last_change":"2024-03-09T08:52:29.583869+0000","last_active":"2024-03-09T08:52:30.943444+0000","last_peered":"2024-03-09T08:52:30.943444+0000","last_clean":"2024-03-09T08:52:30.943444+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T08:52:30.943444+0000","last_undegraded":"2024-03-09T08:52:30.943444+0000","last_fullsized":"2024-03-09T08:52:30.943444+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:29:47.636741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578670+0000","last_change":"2024-03-09T08:52:18.247145+0000","last_active":"2024-03-09T08:52:30.578670+0000","last_peered":"2024-03-09T08:52:30.578670+0000","last_clean":"2024-03-09T08:52:30.578670+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T08:52:30.578670+0000","last_undegraded":"2024-03-09T08:52:30.578670+0000","last_fullsized":"2024-03-09T08:52:30.578670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:46:24.111224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577279+0000","last_change":"2024-03-09T08:52:29.587358+0000","last_active":"2024-03-09T08:52:30.577279+0000","last_peered":"2024-03-09T08:52:30.577279+0000","last_clean":"2024-03-09T08:52:30.577279+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T08:52:30.577279+0000","last_undegraded":"2024-03-09T08:52:30.577279+0000","last_fullsized":"2024-03-09T08:52:30.577279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:46.836801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577323+0000","last_change":"2024-03-09T08:52:18.247784+0000","last_active":"2024-03-09T08:52:30.577323+0000","last_peered":"2024-03-09T08:52:30.577323+0000","last_clean":"2024-03-09T08:52:30.577323+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T08:52:30.577323+0000","last_undegraded":"2024-03-09T08:52:30.577323+0000","last_fullsized":"2024-03-09T08:52:30.577323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:46:59.008220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577364+0000","last_change":"2024-03-09T08:52:29.589458+0000","last_active":"2024-03-09T08:52:30.577364+0000","last_peered":"2024-03-09T08:52:30.577364+0000","last_clean":"2024-03-09T08:52:30.577364+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T08:52:30.577364+0000","last_undegraded":"2024-03-09T08:52:30.577364+0000","last_fullsized":"2024-03-09T08:52:30.577364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:19:35.222184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945231+0000","last_change":"2024-03-09T08:52:18.239943+0000","last_active":"2024-03-09T08:52:30.945231+0000","last_peered":"2024-03-09T08:52:30.945231+0000","last_clean":"2024-03-09T08:52:30.945231+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T08:52:30.945231+0000","last_undegraded":"2024-03-09T08:52:30.945231+0000","last_fullsized":"2024-03-09T08:52:30.945231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:18.727030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.949211+0000","last_change":"2024-03-09T08:52:29.597147+0000","last_active":"2024-03-09T08:52:30.949211+0000","last_peered":"2024-03-09T08:52:30.949211+0000","last_clean":"2024-03-09T08:52:30.949211+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T08:52:30.949211+0000","last_undegraded":"2024-03-09T08:52:30.949211+0000","last_fullsized":"2024-03-09T08:52:30.949211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:44:47.144911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575715+0000","last_change":"2024-03-09T08:52:18.236642+0000","last_active":"2024-03-09T08:52:30.575715+0000","last_peered":"2024-03-09T08:52:30.575715+0000","last_clean":"2024-03-09T08:52:30.575715+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T08:52:30.575715+0000","last_undegraded":"2024-03-09T08:52:30.575715+0000","last_fullsized":"2024-03-09T08:52:30.575715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:51:46.825153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943044+0000","last_change":"2024-03-09T08:52:29.590810+0000","last_active":"2024-03-09T08:52:30.943044+0000","last_peered":"2024-03-09T08:52:30.943044+0000","last_clean":"2024-03-09T08:52:30.943044+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T08:52:30.943044+0000","last_undegraded":"2024-03-09T08:52:30.943044+0000","last_fullsized":"2024-03-09T08:52:30.943044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:21.534580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945569+0000","last_change":"2024-03-09T08:52:18.236855+0000","last_active":"2024-03-09T08:52:30.945569+0000","last_peered":"2024-03-09T08:52:30.945569+0000","last_clean":"2024-03-09T08:52:30.945569+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T08:52:30.945569+0000","last_undegraded":"2024-03-09T08:52:30.945569+0000","last_fullsized":"2024-03-09T08:52:30.945569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:21.941242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576311+0000","last_change":"2024-03-09T08:52:29.602209+0000","last_active":"2024-03-09T08:52:30.576311+0000","last_peered":"2024-03-09T08:52:30.576311+0000","last_clean":"2024-03-09T08:52:30.576311+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T08:52:30.576311+0000","last_undegraded":"2024-03-09T08:52:30.576311+0000","last_fullsized":"2024-03-09T08:52:30.576311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:11:33.349464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597936+0000","last_change":"2024-03-09T08:52:18.238373+0000","last_active":"2024-03-09T08:52:30.597936+0000","last_peered":"2024-03-09T08:52:30.597936+0000","last_clean":"2024-03-09T08:52:30.597936+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T08:52:30.597936+0000","last_undegraded":"2024-03-09T08:52:30.597936+0000","last_fullsized":"2024-03-09T08:52:30.597936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:05:31.480335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577230+0000","last_change":"2024-03-09T08:52:29.578120+0000","last_active":"2024-03-09T08:52:30.577230+0000","last_peered":"2024-03-09T08:52:30.577230+0000","last_clean":"2024-03-09T08:52:30.577230+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T08:52:30.577230+0000","last_undegraded":"2024-03-09T08:52:30.577230+0000","last_fullsized":"2024-03-09T08:52:30.577230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:13:27.623995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578214+0000","last_change":"2024-03-09T08:52:18.235565+0000","last_active":"2024-03-09T08:52:30.578214+0000","last_peered":"2024-03-09T08:52:30.578214+0000","last_clean":"2024-03-09T08:52:30.578214+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T08:52:30.578214+0000","last_undegraded":"2024-03-09T08:52:30.578214+0000","last_fullsized":"2024-03-09T08:52:30.578214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:52:14.994539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576439+0000","last_change":"2024-03-09T08:52:29.579668+0000","last_active":"2024-03-09T08:52:30.576439+0000","last_peered":"2024-03-09T08:52:30.576439+0000","last_clean":"2024-03-09T08:52:30.576439+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T08:52:30.576439+0000","last_undegraded":"2024-03-09T08:52:30.576439+0000","last_fullsized":"2024-03-09T08:52:30.576439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:12:15.272875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948818+0000","last_change":"2024-03-09T08:52:18.239801+0000","last_active":"2024-03-09T08:52:30.948818+0000","last_peered":"2024-03-09T08:52:30.948818+0000","last_clean":"2024-03-09T08:52:30.948818+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T08:52:30.948818+0000","last_undegraded":"2024-03-09T08:52:30.948818+0000","last_fullsized":"2024-03-09T08:52:30.948818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:38:06.836990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944799+0000","last_change":"2024-03-09T08:52:29.586986+0000","last_active":"2024-03-09T08:52:30.944799+0000","last_peered":"2024-03-09T08:52:30.944799+0000","last_clean":"2024-03-09T08:52:30.944799+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T08:52:30.944799+0000","last_undegraded":"2024-03-09T08:52:30.944799+0000","last_fullsized":"2024-03-09T08:52:30.944799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:37.382414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943359+0000","last_change":"2024-03-09T08:52:18.233582+0000","last_active":"2024-03-09T08:52:30.943359+0000","last_peered":"2024-03-09T08:52:30.943359+0000","last_clean":"2024-03-09T08:52:30.943359+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T08:52:30.943359+0000","last_undegraded":"2024-03-09T08:52:30.943359+0000","last_fullsized":"2024-03-09T08:52:30.943359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:49.313616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597247+0000","last_change":"2024-03-09T08:52:29.586673+0000","last_active":"2024-03-09T08:52:30.597247+0000","last_peered":"2024-03-09T08:52:30.597247+0000","last_clean":"2024-03-09T08:52:30.597247+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T08:52:30.597247+0000","last_undegraded":"2024-03-09T08:52:30.597247+0000","last_fullsized":"2024-03-09T08:52:30.597247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:23:49.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947293+0000","last_change":"2024-03-09T08:52:18.242349+0000","last_active":"2024-03-09T08:52:30.947293+0000","last_peered":"2024-03-09T08:52:30.947293+0000","last_clean":"2024-03-09T08:52:30.947293+0000","last_became_active":"2024-03-09T08:52:18.241995+0000","last_became_peered":"2024-03-09T08:52:18.241995+0000","last_unstale":"2024-03-09T08:52:30.947293+0000","last_undegraded":"2024-03-09T08:52:30.947293+0000","last_fullsized":"2024-03-09T08:52:30.947293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:29:09.977984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601965+0000","last_change":"2024-03-09T08:52:29.592702+0000","last_active":"2024-03-09T08:52:30.601965+0000","last_peered":"2024-03-09T08:52:30.601965+0000","last_clean":"2024-03-09T08:52:30.601965+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T08:52:30.601965+0000","last_undegraded":"2024-03-09T08:52:30.601965+0000","last_fullsized":"2024-03-09T08:52:30.601965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:39.344726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944970+0000","last_change":"2024-03-09T08:52:18.230289+0000","last_active":"2024-03-09T08:52:30.944970+0000","last_peered":"2024-03-09T08:52:30.944970+0000","last_clean":"2024-03-09T08:52:30.944970+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T08:52:30.944970+0000","last_undegraded":"2024-03-09T08:52:30.944970+0000","last_fullsized":"2024-03-09T08:52:30.944970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:34.507924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578394+0000","last_change":"2024-03-09T08:52:29.608191+0000","last_active":"2024-03-09T08:52:30.578394+0000","last_peered":"2024-03-09T08:52:30.578394+0000","last_clean":"2024-03-09T08:52:30.578394+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T08:52:30.578394+0000","last_undegraded":"2024-03-09T08:52:30.578394+0000","last_fullsized":"2024-03-09T08:52:30.578394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:26:32.282196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601147+0000","last_change":"2024-03-09T08:52:18.236376+0000","last_active":"2024-03-09T08:52:30.601147+0000","last_peered":"2024-03-09T08:52:30.601147+0000","last_clean":"2024-03-09T08:52:30.601147+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T08:52:30.601147+0000","last_undegraded":"2024-03-09T08:52:30.601147+0000","last_fullsized":"2024-03-09T08:52:30.601147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:34:04.412643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946339+0000","last_change":"2024-03-09T08:52:29.573685+0000","last_active":"2024-03-09T08:52:30.946339+0000","last_peered":"2024-03-09T08:52:30.946339+0000","last_clean":"2024-03-09T08:52:30.946339+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T08:52:30.946339+0000","last_undegraded":"2024-03-09T08:52:30.946339+0000","last_fullsized":"2024-03-09T08:52:30.946339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:41:41.839063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597356+0000","last_change":"2024-03-09T08:52:18.231316+0000","last_active":"2024-03-09T08:52:30.597356+0000","last_peered":"2024-03-09T08:52:30.597356+0000","last_clean":"2024-03-09T08:52:30.597356+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T08:52:30.597356+0000","last_undegraded":"2024-03-09T08:52:30.597356+0000","last_fullsized":"2024-03-09T08:52:30.597356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:25:25.715084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948285+0000","last_change":"2024-03-09T08:52:29.603272+0000","last_active":"2024-03-09T08:52:30.948285+0000","last_peered":"2024-03-09T08:52:30.948285+0000","last_clean":"2024-03-09T08:52:30.948285+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T08:52:30.948285+0000","last_undegraded":"2024-03-09T08:52:30.948285+0000","last_fullsized":"2024-03-09T08:52:30.948285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:07:58.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576412+0000","last_change":"2024-03-09T08:52:18.243645+0000","last_active":"2024-03-09T08:52:30.576412+0000","last_peered":"2024-03-09T08:52:30.576412+0000","last_clean":"2024-03-09T08:52:30.576412+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T08:52:30.576412+0000","last_undegraded":"2024-03-09T08:52:30.576412+0000","last_fullsized":"2024-03-09T08:52:30.576412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:19.501675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576249+0000","last_change":"2024-03-09T08:52:29.587462+0000","last_active":"2024-03-09T08:52:30.576249+0000","last_peered":"2024-03-09T08:52:30.576249+0000","last_clean":"2024-03-09T08:52:30.576249+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T08:52:30.576249+0000","last_undegraded":"2024-03-09T08:52:30.576249+0000","last_fullsized":"2024-03-09T08:52:30.576249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:51.652225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577062+0000","last_change":"2024-03-09T08:52:18.233451+0000","last_active":"2024-03-09T08:52:30.577062+0000","last_peered":"2024-03-09T08:52:30.577062+0000","last_clean":"2024-03-09T08:52:30.577062+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T08:52:30.577062+0000","last_undegraded":"2024-03-09T08:52:30.577062+0000","last_fullsized":"2024-03-09T08:52:30.577062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:56:11.270081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942964+0000","last_change":"2024-03-09T08:52:29.599909+0000","last_active":"2024-03-09T08:52:30.942964+0000","last_peered":"2024-03-09T08:52:30.942964+0000","last_clean":"2024-03-09T08:52:30.942964+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T08:52:30.942964+0000","last_undegraded":"2024-03-09T08:52:30.942964+0000","last_fullsized":"2024-03-09T08:52:30.942964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:42.596067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577555+0000","last_change":"2024-03-09T08:52:18.234158+0000","last_active":"2024-03-09T08:52:30.577555+0000","last_peered":"2024-03-09T08:52:30.577555+0000","last_clean":"2024-03-09T08:52:30.577555+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T08:52:30.577555+0000","last_undegraded":"2024-03-09T08:52:30.577555+0000","last_fullsized":"2024-03-09T08:52:30.577555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:59:33.423558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944531+0000","last_change":"2024-03-09T08:52:29.597518+0000","last_active":"2024-03-09T08:52:30.944531+0000","last_peered":"2024-03-09T08:52:30.944531+0000","last_clean":"2024-03-09T08:52:30.944531+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T08:52:30.944531+0000","last_undegraded":"2024-03-09T08:52:30.944531+0000","last_fullsized":"2024-03-09T08:52:30.944531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:32:32.452993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577830+0000","last_change":"2024-03-09T08:52:18.234648+0000","last_active":"2024-03-09T08:52:30.577830+0000","last_peered":"2024-03-09T08:52:30.577830+0000","last_clean":"2024-03-09T08:52:30.577830+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T08:52:30.577830+0000","last_undegraded":"2024-03-09T08:52:30.577830+0000","last_fullsized":"2024-03-09T08:52:30.577830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:03.927280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944900+0000","last_change":"2024-03-09T08:52:29.580608+0000","last_active":"2024-03-09T08:52:30.944900+0000","last_peered":"2024-03-09T08:52:30.944900+0000","last_clean":"2024-03-09T08:52:30.944900+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T08:52:30.944900+0000","last_undegraded":"2024-03-09T08:52:30.944900+0000","last_fullsized":"2024-03-09T08:52:30.944900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:00:37.184351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948628+0000","last_change":"2024-03-09T08:52:18.253038+0000","last_active":"2024-03-09T08:52:30.948628+0000","last_peered":"2024-03-09T08:52:30.948628+0000","last_clean":"2024-03-09T08:52:30.948628+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T08:52:30.948628+0000","last_undegraded":"2024-03-09T08:52:30.948628+0000","last_fullsized":"2024-03-09T08:52:30.948628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:26:39.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577331+0000","last_change":"2024-03-09T08:52:29.578549+0000","last_active":"2024-03-09T08:52:30.577331+0000","last_peered":"2024-03-09T08:52:30.577331+0000","last_clean":"2024-03-09T08:52:30.577331+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T08:52:30.577331+0000","last_undegraded":"2024-03-09T08:52:30.577331+0000","last_fullsized":"2024-03-09T08:52:30.577331+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:42.775199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575758+0000","last_change":"2024-03-09T08:52:18.240975+0000","last_active":"2024-03-09T08:52:30.575758+0000","last_peered":"2024-03-09T08:52:30.575758+0000","last_clean":"2024-03-09T08:52:30.575758+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T08:52:30.575758+0000","last_undegraded":"2024-03-09T08:52:30.575758+0000","last_fullsized":"2024-03-09T08:52:30.575758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:30:09.363339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577959+0000","last_change":"2024-03-09T08:52:29.584159+0000","last_active":"2024-03-09T08:52:30.577959+0000","last_peered":"2024-03-09T08:52:30.577959+0000","last_clean":"2024-03-09T08:52:30.577959+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T08:52:30.577959+0000","last_undegraded":"2024-03-09T08:52:30.577959+0000","last_fullsized":"2024-03-09T08:52:30.577959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:01:41.661806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943943+0000","last_change":"2024-03-09T08:52:18.249901+0000","last_active":"2024-03-09T08:52:30.943943+0000","last_peered":"2024-03-09T08:52:30.943943+0000","last_clean":"2024-03-09T08:52:30.943943+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T08:52:30.943943+0000","last_undegraded":"2024-03-09T08:52:30.943943+0000","last_fullsized":"2024-03-09T08:52:30.943943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:09.947401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601289+0000","last_change":"2024-03-09T08:52:29.595184+0000","last_active":"2024-03-09T08:52:30.601289+0000","last_peered":"2024-03-09T08:52:30.601289+0000","last_clean":"2024-03-09T08:52:30.601289+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T08:52:30.601289+0000","last_undegraded":"2024-03-09T08:52:30.601289+0000","last_fullsized":"2024-03-09T08:52:30.601289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:34:27.662636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945364+0000","last_change":"2024-03-09T08:52:18.233561+0000","last_active":"2024-03-09T08:52:30.945364+0000","last_peered":"2024-03-09T08:52:30.945364+0000","last_clean":"2024-03-09T08:52:30.945364+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T08:52:30.945364+0000","last_undegraded":"2024-03-09T08:52:30.945364+0000","last_fullsized":"2024-03-09T08:52:30.945364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:15:54.027155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947171+0000","last_change":"2024-03-09T08:52:29.595771+0000","last_active":"2024-03-09T08:52:30.947171+0000","last_peered":"2024-03-09T08:52:30.947171+0000","last_clean":"2024-03-09T08:52:30.947171+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T08:52:30.947171+0000","last_undegraded":"2024-03-09T08:52:30.947171+0000","last_fullsized":"2024-03-09T08:52:30.947171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:23:48.898071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943880+0000","last_change":"2024-03-09T08:52:18.242311+0000","last_active":"2024-03-09T08:52:30.943880+0000","last_peered":"2024-03-09T08:52:30.943880+0000","last_clean":"2024-03-09T08:52:30.943880+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T08:52:30.943880+0000","last_undegraded":"2024-03-09T08:52:30.943880+0000","last_fullsized":"2024-03-09T08:52:30.943880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:48:15.559438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575834+0000","last_change":"2024-03-09T08:52:29.576792+0000","last_active":"2024-03-09T08:52:30.575834+0000","last_peered":"2024-03-09T08:52:30.575834+0000","last_clean":"2024-03-09T08:52:30.575834+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T08:52:30.575834+0000","last_undegraded":"2024-03-09T08:52:30.575834+0000","last_fullsized":"2024-03-09T08:52:30.575834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:21.364814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947241+0000","last_change":"2024-03-09T08:52:18.248341+0000","last_active":"2024-03-09T08:52:30.947241+0000","last_peered":"2024-03-09T08:52:30.947241+0000","last_clean":"2024-03-09T08:52:30.947241+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T08:52:30.947241+0000","last_undegraded":"2024-03-09T08:52:30.947241+0000","last_fullsized":"2024-03-09T08:52:30.947241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:18:02.716066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942730+0000","last_change":"2024-03-09T08:52:29.606330+0000","last_active":"2024-03-09T08:52:30.942730+0000","last_peered":"2024-03-09T08:52:30.942730+0000","last_clean":"2024-03-09T08:52:30.942730+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T08:52:30.942730+0000","last_undegraded":"2024-03-09T08:52:30.942730+0000","last_fullsized":"2024-03-09T08:52:30.942730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:44:50.347124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597559+0000","last_change":"2024-03-09T08:52:18.222968+0000","last_active":"2024-03-09T08:52:30.597559+0000","last_peered":"2024-03-09T08:52:30.597559+0000","last_clean":"2024-03-09T08:52:30.597559+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T08:52:30.597559+0000","last_undegraded":"2024-03-09T08:52:30.597559+0000","last_fullsized":"2024-03-09T08:52:30.597559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:52:29.523963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943337+0000","last_change":"2024-03-09T08:52:29.591813+0000","last_active":"2024-03-09T08:52:30.943337+0000","last_peered":"2024-03-09T08:52:30.943337+0000","last_clean":"2024-03-09T08:52:30.943337+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T08:52:30.943337+0000","last_undegraded":"2024-03-09T08:52:30.943337+0000","last_fullsized":"2024-03-09T08:52:30.943337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:40:31.568253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601942+0000","last_change":"2024-03-09T08:52:18.225812+0000","last_active":"2024-03-09T08:52:30.601942+0000","last_peered":"2024-03-09T08:52:30.601942+0000","last_clean":"2024-03-09T08:52:30.601942+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T08:52:30.601942+0000","last_undegraded":"2024-03-09T08:52:30.601942+0000","last_fullsized":"2024-03-09T08:52:30.601942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:09:32.044185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943686+0000","last_change":"2024-03-09T08:52:29.595055+0000","last_active":"2024-03-09T08:52:30.943686+0000","last_peered":"2024-03-09T08:52:30.943686+0000","last_clean":"2024-03-09T08:52:30.943686+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T08:52:30.943686+0000","last_undegraded":"2024-03-09T08:52:30.943686+0000","last_fullsized":"2024-03-09T08:52:30.943686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:53.209169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948709+0000","last_change":"2024-03-09T08:52:18.248973+0000","last_active":"2024-03-09T08:52:30.948709+0000","last_peered":"2024-03-09T08:52:30.948709+0000","last_clean":"2024-03-09T08:52:30.948709+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T08:52:30.948709+0000","last_undegraded":"2024-03-09T08:52:30.948709+0000","last_fullsized":"2024-03-09T08:52:30.948709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:09:11.372616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":101489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":102507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T08:52:40.972 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T08:52:41.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.207+0000 7f061ee00640 1 Processor -- start 2024-03-09T08:52:41.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.207+0000 7f061ee00640 1 -- start start 2024-03-09T08:52:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.208+0000 7f061ee00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f06180c5f50 0x7f0618155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.208+0000 7f061ee00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06180c6a40 con 0x7f061806b040 2024-03-09T08:52:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.208+0000 7f061ee00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061806b5b0 con 0x7f061806c650 2024-03-09T08:52:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.208+0000 7f061ee00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06180c9e00 con 0x7f06180c5f50 2024-03-09T08:52:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.209+0000 7f060f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f061806b040 0x7f06180c96f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53848/0 (socket says 172.21.15.67:53848) 2024-03-09T08:52:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.209+0000 7f060f400640 1 -- 172.21.15.67:0/2811596226 learned_addr learned my addr 172.21.15.67:0/2811596226 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.209+0000 7f0617e00640 1 --2- 172.21.15.67:0/2811596226 >> v2:172.21.15.67:3300/0 conn(0x7f06180c5f50 0x7f0618155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.209+0000 7f0617e00640 1 -- 172.21.15.67:0/2811596226 >> v1:172.21.15.134:6789/0 conn(0x7f061806c650 legacy=0x7f06180c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.209+0000 7f0617400640 1 -- 172.21.15.67:0/2811596226 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454177241 0 0) 0x7f06180c6a40 con 0x7f061806b040 2024-03-09T08:52:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.209+0000 7f0617e00640 1 -- 172.21.15.67:0/2811596226 >> v1:172.21.15.67:6789/0 conn(0x7f061806b040 legacy=0x7f06180c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.209+0000 7f0617e00640 1 -- 172.21.15.67:0/2811596226 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061806b890 con 0x7f06180c5f50 2024-03-09T08:52:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.210+0000 7f0617e00640 1 --2- 172.21.15.67:0/2811596226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06180c5f50 0x7f0618155e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0608009d90 tx=0x7f060802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f7d908ebb8bce server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.211+0000 7f0617400640 1 -- 172.21.15.67:0/2811596226 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608039070 con 0x7f06180c5f50 2024-03-09T08:52:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.211+0000 7f0617400640 1 -- 172.21.15.67:0/2811596226 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060802cd00 con 0x7f06180c5f50 2024-03-09T08:52:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.211+0000 7f0617400640 1 -- 172.21.15.67:0/2811596226 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608034660 con 0x7f06180c5f50 2024-03-09T08:52:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.211+0000 7f061ee00640 1 -- 172.21.15.67:0/2811596226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06180c5f50 msgr2=0x7f0618155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.211+0000 7f061ee00640 1 --2- 172.21.15.67:0/2811596226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06180c5f50 0x7f0618155e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0608009d90 tx=0x7f060802bf50 comp rx=0 tx=0).stop 2024-03-09T08:52:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.211+0000 7f061ee00640 1 -- 172.21.15.67:0/2811596226 shutdown_connections 2024-03-09T08:52:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.211+0000 7f061ee00640 1 --2- 172.21.15.67:0/2811596226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06180c5f50 0x7f0618155e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.211+0000 7f061ee00640 1 -- 172.21.15.67:0/2811596226 >> 172.21.15.67:0/2811596226 conn(0x7f06180c8050 msgr2=0x7f06180c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:41.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.211+0000 7f061ee00640 1 -- 172.21.15.67:0/2811596226 shutdown_connections 2024-03-09T08:52:41.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.211+0000 7f061ee00640 1 -- 172.21.15.67:0/2811596226 wait complete. 2024-03-09T08:52:41.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.213+0000 7f061ee00640 1 Processor -- start 2024-03-09T08:52:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.213+0000 7f061ee00640 1 -- start start 2024-03-09T08:52:41.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.214+0000 7f061ee00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f061806b040 0x7f061814e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:41.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.214+0000 7f061ee00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f061806c650 0x7f06180c1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:41.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.214+0000 7f061ee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06180c5f50 0x7f061814d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:41.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.214+0000 7f061ee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061806be50 con 0x7f06180c5f50 2024-03-09T08:52:41.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.214+0000 7f061ee00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061806cac0 con 0x7f061806b040 2024-03-09T08:52:41.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.214+0000 7f061ee00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061806c130 con 0x7f061806c650 2024-03-09T08:52:41.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.214+0000 7f061ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06180c5f50 0x7f061814d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:41.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.214+0000 7f061ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06180c5f50 0x7f061814d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46582/0 (socket says 172.21.15.67:46582) 2024-03-09T08:52:41.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.214+0000 7f061ce00640 1 -- 172.21.15.67:0/4069459242 learned_addr learned my addr 172.21.15.67:0/4069459242 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:41.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.214+0000 7f060f400640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f061806c650 0x7f06180c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:41.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.214+0000 7f0617e00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f061806b040 0x7f061814e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:41.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.215+0000 7f061ce00640 1 -- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f061806c650 msgr2=0x7f06180c1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:41.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.215+0000 7f061ce00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f061806c650 0x7f06180c1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.215+0000 7f061ce00640 1 -- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f061806b040 msgr2=0x7f061814e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:41.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.215+0000 7f061ce00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f061806b040 0x7f061814e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.215+0000 7f061ce00640 1 -- 172.21.15.67:0/4069459242 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06181eefb0 con 0x7f06180c5f50 2024-03-09T08:52:41.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.215+0000 7f061ce00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06180c5f50 0x7f061814d030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f061000c3c0 tx=0x7f061000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:41.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.215+0000 7f0614c00640 1 -- 172.21.15.67:0/4069459242 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610017070 con 0x7f06180c5f50 2024-03-09T08:52:41.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.215+0000 7f0614c00640 1 -- 172.21.15.67:0/4069459242 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06100052d0 con 0x7f06180c5f50 2024-03-09T08:52:41.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.215+0000 7f0614c00640 1 -- 172.21.15.67:0/4069459242 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610003830 con 0x7f06180c5f50 2024-03-09T08:52:41.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.215+0000 7f061ee00640 1 -- 172.21.15.67:0/4069459242 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061814cc90 con 0x7f06180c5f50 2024-03-09T08:52:41.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.215+0000 7f061ee00640 1 -- 172.21.15.67:0/4069459242 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06181ef500 con 0x7f06180c5f50 2024-03-09T08:52:41.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.217+0000 7f061ee00640 1 -- 172.21.15.67:0/4069459242 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061806d500 con 0x7f06180c5f50 2024-03-09T08:52:41.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.217+0000 7f0614c00640 1 -- 172.21.15.67:0/4069459242 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f061001a070 con 0x7f06180c5f50 2024-03-09T08:52:41.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.218+0000 7f0614c00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05f406b500 0x7f05f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:41.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.219+0000 7f0614c00640 1 -- 172.21.15.67:0/4069459242 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f0610089720 con 0x7f06180c5f50 2024-03-09T08:52:41.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.219+0000 7f0617e00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05f406b500 0x7f05f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:41.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.220+0000 7f0617e00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05f406b500 0x7f05f406d9c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0608036000 tx=0x7f0608030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:41.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.223+0000 7f0614c00640 1 -- 172.21.15.67:0/4069459242 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f061806d500 con 0x7f06180c5f50 2024-03-09T08:52:41.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.414+0000 7f061ee00640 1 -- 172.21.15.67:0/4069459242 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f061806cca0 con 0x7f05f406b500 2024-03-09T08:52:41.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.449+0000 7f0614c00640 1 -- 172.21.15.67:0/4069459242 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490306 (secure 0 0 0) 0x7f061806cca0 con 0x7f05f406b500 2024-03-09T08:52:41.453 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:41.458 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T08:52:41.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.458+0000 7f061ee00640 1 -- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05f406b500 msgr2=0x7f05f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:41.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.458+0000 7f061ee00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05f406b500 0x7f05f406d9c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0608036000 tx=0x7f0608030000 comp rx=0 tx=0).stop 2024-03-09T08:52:41.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.458+0000 7f061ee00640 1 -- 172.21.15.67:0/4069459242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06180c5f50 msgr2=0x7f061814d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:41.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.458+0000 7f061ee00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06180c5f50 0x7f061814d030 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f061000c3c0 tx=0x7f061000c880 comp rx=0 tx=0).stop 2024-03-09T08:52:41.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.459+0000 7f061ee00640 1 -- 172.21.15.67:0/4069459242 shutdown_connections 2024-03-09T08:52:41.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.459+0000 7f061ee00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05f406b500 0x7f05f406d9c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.459+0000 7f061ee00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06180c5f50 0x7f061814d030 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.459+0000 7f061ee00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f061806c650 0x7f06180c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.459+0000 7f061ee00640 1 --2- 172.21.15.67:0/4069459242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f061806b040 0x7f061814e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.459+0000 7f061ee00640 1 -- 172.21.15.67:0/4069459242 >> 172.21.15.67:0/4069459242 conn(0x7f06180c8050 msgr2=0x7f06180ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:41.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.459+0000 7f061ee00640 1 -- 172.21.15.67:0/4069459242 shutdown_connections 2024-03-09T08:52:41.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.459+0000 7f061ee00640 1 -- 172.21.15.67:0/4069459242 wait complete. 2024-03-09T08:52:41.475 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-03-09T08:52:40.681725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3350528,"data_stored":1984464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.110573"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575471+0000","last_change":"2024-03-09T08:52:29.573571+0000","last_active":"2024-03-09T08:52:30.575471+0000","last_peered":"2024-03-09T08:52:30.575471+0000","last_clean":"2024-03-09T08:52:30.575471+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T08:52:30.575471+0000","last_undegraded":"2024-03-09T08:52:30.575471+0000","last_fullsized":"2024-03-09T08:52:30.575471+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:33:33.068463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598024+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T08:52:30.598024+0000","last_peered":"2024-03-09T08:52:30.598024+0000","last_clean":"2024-03-09T08:52:30.598024+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T08:52:30.598024+0000","last_undegraded":"2024-03-09T08:52:30.598024+0000","last_fullsized":"2024-03-09T08:52:30.598024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:41:40.332687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577727+0000","last_change":"2024-03-09T08:52:18.235099+0000","last_active":"2024-03-09T08:52:30.577727+0000","last_peered":"2024-03-09T08:52:30.577727+0000","last_clean":"2024-03-09T08:52:30.577727+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T08:52:30.577727+0000","last_undegraded":"2024-03-09T08:52:30.577727+0000","last_fullsized":"2024-03-09T08:52:30.577727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:39:28.882461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575426+0000","last_change":"2024-03-09T08:52:28.571905+0000","last_active":"2024-03-09T08:52:30.575426+0000","last_peered":"2024-03-09T08:52:30.575426+0000","last_clean":"2024-03-09T08:52:30.575426+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T08:52:30.575426+0000","last_undegraded":"2024-03-09T08:52:30.575426+0000","last_fullsized":"2024-03-09T08:52:30.575426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:01.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.600985+0000","last_change":"2024-03-09T08:52:29.578644+0000","last_active":"2024-03-09T08:52:30.600985+0000","last_peered":"2024-03-09T08:52:30.600985+0000","last_clean":"2024-03-09T08:52:30.600985+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T08:52:30.600985+0000","last_undegraded":"2024-03-09T08:52:30.600985+0000","last_fullsized":"2024-03-09T08:52:30.600985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:00.044620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577762+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T08:52:30.577762+0000","last_peered":"2024-03-09T08:52:30.577762+0000","last_clean":"2024-03-09T08:52:30.577762+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T08:52:30.577762+0000","last_undegraded":"2024-03-09T08:52:30.577762+0000","last_fullsized":"2024-03-09T08:52:30.577762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597762+0000","last_change":"2024-03-09T08:52:18.236946+0000","last_active":"2024-03-09T08:52:30.597762+0000","last_peered":"2024-03-09T08:52:30.597762+0000","last_clean":"2024-03-09T08:52:30.597762+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T08:52:30.597762+0000","last_undegraded":"2024-03-09T08:52:30.597762+0000","last_fullsized":"2024-03-09T08:52:30.597762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:45.372160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575237+0000","last_change":"2024-03-09T08:52:28.573074+0000","last_active":"2024-03-09T08:52:30.575237+0000","last_peered":"2024-03-09T08:52:30.575237+0000","last_clean":"2024-03-09T08:52:30.575237+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T08:52:30.575237+0000","last_undegraded":"2024-03-09T08:52:30.575237+0000","last_fullsized":"2024-03-09T08:52:30.575237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:50.600408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945199+0000","last_change":"2024-03-09T08:52:29.594030+0000","last_active":"2024-03-09T08:52:30.945199+0000","last_peered":"2024-03-09T08:52:30.945199+0000","last_clean":"2024-03-09T08:52:30.945199+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T08:52:30.945199+0000","last_undegraded":"2024-03-09T08:52:30.945199+0000","last_fullsized":"2024-03-09T08:52:30.945199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:57.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948441+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T08:52:30.948441+0000","last_peered":"2024-03-09T08:52:30.948441+0000","last_clean":"2024-03-09T08:52:30.948441+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T08:52:30.948441+0000","last_undegraded":"2024-03-09T08:52:30.948441+0000","last_fullsized":"2024-03-09T08:52:30.948441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947481+0000","last_change":"2024-03-09T08:52:18.230629+0000","last_active":"2024-03-09T08:52:30.947481+0000","last_peered":"2024-03-09T08:52:30.947481+0000","last_clean":"2024-03-09T08:52:30.947481+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T08:52:30.947481+0000","last_undegraded":"2024-03-09T08:52:30.947481+0000","last_fullsized":"2024-03-09T08:52:30.947481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:46:38.528839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576909+0000","last_change":"2024-03-09T08:52:28.577128+0000","last_active":"2024-03-09T08:52:30.576909+0000","last_peered":"2024-03-09T08:52:30.576909+0000","last_clean":"2024-03-09T08:52:30.576909+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T08:52:30.576909+0000","last_undegraded":"2024-03-09T08:52:30.576909+0000","last_fullsized":"2024-03-09T08:52:30.576909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:41.801752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946024+0000","last_change":"2024-03-09T08:52:29.581705+0000","last_active":"2024-03-09T08:52:30.946024+0000","last_peered":"2024-03-09T08:52:30.946024+0000","last_clean":"2024-03-09T08:52:30.946024+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T08:52:30.946024+0000","last_undegraded":"2024-03-09T08:52:30.946024+0000","last_fullsized":"2024-03-09T08:52:30.946024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:27.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947791+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T08:52:30.947791+0000","last_peered":"2024-03-09T08:52:30.947791+0000","last_clean":"2024-03-09T08:52:30.947791+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T08:52:30.947791+0000","last_undegraded":"2024-03-09T08:52:30.947791+0000","last_fullsized":"2024-03-09T08:52:30.947791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947372+0000","last_change":"2024-03-09T08:52:18.228419+0000","last_active":"2024-03-09T08:52:30.947372+0000","last_peered":"2024-03-09T08:52:30.947372+0000","last_clean":"2024-03-09T08:52:30.947372+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T08:52:30.947372+0000","last_undegraded":"2024-03-09T08:52:30.947372+0000","last_fullsized":"2024-03-09T08:52:30.947372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:19:12.533402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577499+0000","last_change":"2024-03-09T08:52:28.577069+0000","last_active":"2024-03-09T08:52:30.577499+0000","last_peered":"2024-03-09T08:52:30.577499+0000","last_clean":"2024-03-09T08:52:30.577499+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T08:52:30.577499+0000","last_undegraded":"2024-03-09T08:52:30.577499+0000","last_fullsized":"2024-03-09T08:52:30.577499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:09:02.410915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601446+0000","last_change":"2024-03-09T08:52:29.588922+0000","last_active":"2024-03-09T08:52:30.601446+0000","last_peered":"2024-03-09T08:52:30.601446+0000","last_clean":"2024-03-09T08:52:30.601446+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T08:52:30.601446+0000","last_undegraded":"2024-03-09T08:52:30.601446+0000","last_fullsized":"2024-03-09T08:52:30.601446+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:13.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576353+0000","last_change":"2024-03-09T08:52:18.234372+0000","last_active":"2024-03-09T08:52:30.576353+0000","last_peered":"2024-03-09T08:52:30.576353+0000","last_clean":"2024-03-09T08:52:30.576353+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T08:52:30.576353+0000","last_undegraded":"2024-03-09T08:52:30.576353+0000","last_fullsized":"2024-03-09T08:52:30.576353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:09:33.422907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944406+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T08:52:30.944406+0000","last_peered":"2024-03-09T08:52:30.944406+0000","last_clean":"2024-03-09T08:52:30.944406+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T08:52:30.944406+0000","last_undegraded":"2024-03-09T08:52:30.944406+0000","last_fullsized":"2024-03-09T08:52:30.944406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578381+0000","last_change":"2024-03-09T08:52:28.570631+0000","last_active":"2024-03-09T08:52:30.578381+0000","last_peered":"2024-03-09T08:52:30.578381+0000","last_clean":"2024-03-09T08:52:30.578381+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T08:52:30.578381+0000","last_undegraded":"2024-03-09T08:52:30.578381+0000","last_fullsized":"2024-03-09T08:52:30.578381+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:06.630704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943131+0000","last_change":"2024-03-09T08:52:29.596820+0000","last_active":"2024-03-09T08:52:30.943131+0000","last_peered":"2024-03-09T08:52:30.943131+0000","last_clean":"2024-03-09T08:52:30.943131+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T08:52:30.943131+0000","last_undegraded":"2024-03-09T08:52:30.943131+0000","last_fullsized":"2024-03-09T08:52:30.943131+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:01:20.331889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576713+0000","last_change":"2024-03-09T08:52:18.244306+0000","last_active":"2024-03-09T08:52:30.576713+0000","last_peered":"2024-03-09T08:52:30.576713+0000","last_clean":"2024-03-09T08:52:30.576713+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T08:52:30.576713+0000","last_undegraded":"2024-03-09T08:52:30.576713+0000","last_fullsized":"2024-03-09T08:52:30.576713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:58.365524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601528+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T08:52:30.601528+0000","last_peered":"2024-03-09T08:52:30.601528+0000","last_clean":"2024-03-09T08:52:30.601528+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T08:52:30.601528+0000","last_undegraded":"2024-03-09T08:52:30.601528+0000","last_fullsized":"2024-03-09T08:52:30.601528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:45:45.612953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576699+0000","last_change":"2024-03-09T08:52:28.577110+0000","last_active":"2024-03-09T08:52:30.576699+0000","last_peered":"2024-03-09T08:52:30.576699+0000","last_clean":"2024-03-09T08:52:30.576699+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T08:52:30.576699+0000","last_undegraded":"2024-03-09T08:52:30.576699+0000","last_fullsized":"2024-03-09T08:52:30.576699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:57.197373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579257+0000","last_change":"2024-03-09T08:52:29.579929+0000","last_active":"2024-03-09T08:52:30.579257+0000","last_peered":"2024-03-09T08:52:30.579257+0000","last_clean":"2024-03-09T08:52:30.579257+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T08:52:30.579257+0000","last_undegraded":"2024-03-09T08:52:30.579257+0000","last_fullsized":"2024-03-09T08:52:30.579257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:37:23.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576526+0000","last_change":"2024-03-09T08:52:18.233208+0000","last_active":"2024-03-09T08:52:30.576526+0000","last_peered":"2024-03-09T08:52:30.576526+0000","last_clean":"2024-03-09T08:52:30.576526+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T08:52:30.576526+0000","last_undegraded":"2024-03-09T08:52:30.576526+0000","last_fullsized":"2024-03-09T08:52:30.576526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:17.589783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601269+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T08:52:30.601269+0000","last_peered":"2024-03-09T08:52:30.601269+0000","last_clean":"2024-03-09T08:52:30.601269+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T08:52:30.601269+0000","last_undegraded":"2024-03-09T08:52:30.601269+0000","last_fullsized":"2024-03-09T08:52:30.601269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:21:59.990396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602237+0000","last_change":"2024-03-09T08:52:28.576254+0000","last_active":"2024-03-09T08:52:30.602237+0000","last_peered":"2024-03-09T08:52:30.602237+0000","last_clean":"2024-03-09T08:52:30.602237+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T08:52:30.602237+0000","last_undegraded":"2024-03-09T08:52:30.602237+0000","last_fullsized":"2024-03-09T08:52:30.602237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:21:56.330955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946687+0000","last_change":"2024-03-09T08:52:29.578272+0000","last_active":"2024-03-09T08:52:30.946687+0000","last_peered":"2024-03-09T08:52:30.946687+0000","last_clean":"2024-03-09T08:52:30.946687+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T08:52:30.946687+0000","last_undegraded":"2024-03-09T08:52:30.946687+0000","last_fullsized":"2024-03-09T08:52:30.946687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:56.180809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576191+0000","last_change":"2024-03-09T08:52:18.217046+0000","last_active":"2024-03-09T08:52:30.576191+0000","last_peered":"2024-03-09T08:52:30.576191+0000","last_clean":"2024-03-09T08:52:30.576191+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T08:52:30.576191+0000","last_undegraded":"2024-03-09T08:52:30.576191+0000","last_fullsized":"2024-03-09T08:52:30.576191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:14.884963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943757+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T08:52:30.943757+0000","last_peered":"2024-03-09T08:52:30.943757+0000","last_clean":"2024-03-09T08:52:30.943757+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T08:52:30.943757+0000","last_undegraded":"2024-03-09T08:52:30.943757+0000","last_fullsized":"2024-03-09T08:52:30.943757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578428+0000","last_change":"2024-03-09T08:52:28.573638+0000","last_active":"2024-03-09T08:52:30.578428+0000","last_peered":"2024-03-09T08:52:30.578428+0000","last_clean":"2024-03-09T08:52:30.578428+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T08:52:30.578428+0000","last_undegraded":"2024-03-09T08:52:30.578428+0000","last_fullsized":"2024-03-09T08:52:30.578428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:02.478112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575514+0000","last_change":"2024-03-09T08:52:29.597697+0000","last_active":"2024-03-09T08:52:30.575514+0000","last_peered":"2024-03-09T08:52:30.575514+0000","last_clean":"2024-03-09T08:52:30.575514+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T08:52:30.575514+0000","last_undegraded":"2024-03-09T08:52:30.575514+0000","last_fullsized":"2024-03-09T08:52:30.575514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:00.890845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598142+0000","last_change":"2024-03-09T08:52:18.229739+0000","last_active":"2024-03-09T08:52:30.598142+0000","last_peered":"2024-03-09T08:52:30.598142+0000","last_clean":"2024-03-09T08:52:30.598142+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T08:52:30.598142+0000","last_undegraded":"2024-03-09T08:52:30.598142+0000","last_fullsized":"2024-03-09T08:52:30.598142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:24.394779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579139+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T08:52:30.579139+0000","last_peered":"2024-03-09T08:52:30.579139+0000","last_clean":"2024-03-09T08:52:30.579139+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T08:52:30.579139+0000","last_undegraded":"2024-03-09T08:52:30.579139+0000","last_fullsized":"2024-03-09T08:52:30.579139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:35:38.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575645+0000","last_change":"2024-03-09T08:52:28.575630+0000","last_active":"2024-03-09T08:52:30.575645+0000","last_peered":"2024-03-09T08:52:30.575645+0000","last_clean":"2024-03-09T08:52:30.575645+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T08:52:30.575645+0000","last_undegraded":"2024-03-09T08:52:30.575645+0000","last_fullsized":"2024-03-09T08:52:30.575645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:48.210921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575362+0000","last_change":"2024-03-09T08:52:29.576677+0000","last_active":"2024-03-09T08:52:30.575362+0000","last_peered":"2024-03-09T08:52:30.575362+0000","last_clean":"2024-03-09T08:52:30.575362+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T08:52:30.575362+0000","last_undegraded":"2024-03-09T08:52:30.575362+0000","last_fullsized":"2024-03-09T08:52:30.575362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:19.026873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577342+0000","last_change":"2024-03-09T08:52:18.245442+0000","last_active":"2024-03-09T08:52:30.577342+0000","last_peered":"2024-03-09T08:52:30.577342+0000","last_clean":"2024-03-09T08:52:30.577342+0000","last_became_active":"2024-03-09T08:52:18.245164+0000","last_became_peered":"2024-03-09T08:52:18.245164+0000","last_unstale":"2024-03-09T08:52:30.577342+0000","last_undegraded":"2024-03-09T08:52:30.577342+0000","last_fullsized":"2024-03-09T08:52:30.577342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:31:08.413041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602103+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T08:52:30.602103+0000","last_peered":"2024-03-09T08:52:30.602103+0000","last_clean":"2024-03-09T08:52:30.602103+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T08:52:30.602103+0000","last_undegraded":"2024-03-09T08:52:30.602103+0000","last_fullsized":"2024-03-09T08:52:30.602103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:30.201471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575392+0000","last_change":"2024-03-09T08:52:28.586702+0000","last_active":"2024-03-09T08:52:30.575392+0000","last_peered":"2024-03-09T08:52:30.575392+0000","last_clean":"2024-03-09T08:52:30.575392+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T08:52:30.575392+0000","last_undegraded":"2024-03-09T08:52:30.575392+0000","last_fullsized":"2024-03-09T08:52:30.575392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:09.201224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577654+0000","last_change":"2024-03-09T08:52:29.578956+0000","last_active":"2024-03-09T08:52:30.577654+0000","last_peered":"2024-03-09T08:52:30.577654+0000","last_clean":"2024-03-09T08:52:30.577654+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T08:52:30.577654+0000","last_undegraded":"2024-03-09T08:52:30.577654+0000","last_fullsized":"2024-03-09T08:52:30.577654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:00:00.884211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601530+0000","last_change":"2024-03-09T08:52:18.234887+0000","last_active":"2024-03-09T08:52:30.601530+0000","last_peered":"2024-03-09T08:52:30.601530+0000","last_clean":"2024-03-09T08:52:30.601530+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T08:52:30.601530+0000","last_undegraded":"2024-03-09T08:52:30.601530+0000","last_fullsized":"2024-03-09T08:52:30.601530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:44.661882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948828+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T08:52:30.948828+0000","last_peered":"2024-03-09T08:52:30.948828+0000","last_clean":"2024-03-09T08:52:30.948828+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T08:52:30.948828+0000","last_undegraded":"2024-03-09T08:52:30.948828+0000","last_fullsized":"2024-03-09T08:52:30.948828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597558+0000","last_change":"2024-03-09T08:52:28.580879+0000","last_active":"2024-03-09T08:52:30.597558+0000","last_peered":"2024-03-09T08:52:30.597558+0000","last_clean":"2024-03-09T08:52:30.597558+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T08:52:30.597558+0000","last_undegraded":"2024-03-09T08:52:30.597558+0000","last_fullsized":"2024-03-09T08:52:30.597558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:19:44.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598143+0000","last_change":"2024-03-09T08:52:29.580024+0000","last_active":"2024-03-09T08:52:30.598143+0000","last_peered":"2024-03-09T08:52:30.598143+0000","last_clean":"2024-03-09T08:52:30.598143+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T08:52:30.598143+0000","last_undegraded":"2024-03-09T08:52:30.598143+0000","last_fullsized":"2024-03-09T08:52:30.598143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:02:13.195642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947463+0000","last_change":"2024-03-09T08:52:18.250999+0000","last_active":"2024-03-09T08:52:30.947463+0000","last_peered":"2024-03-09T08:52:30.947463+0000","last_clean":"2024-03-09T08:52:30.947463+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T08:52:30.947463+0000","last_undegraded":"2024-03-09T08:52:30.947463+0000","last_fullsized":"2024-03-09T08:52:30.947463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:49:54.496603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601554+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T08:52:30.601554+0000","last_peered":"2024-03-09T08:52:30.601554+0000","last_clean":"2024-03-09T08:52:30.601554+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T08:52:30.601554+0000","last_undegraded":"2024-03-09T08:52:30.601554+0000","last_fullsized":"2024-03-09T08:52:30.601554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601366+0000","last_change":"2024-03-09T08:52:28.584457+0000","last_active":"2024-03-09T08:52:30.601366+0000","last_peered":"2024-03-09T08:52:30.601366+0000","last_clean":"2024-03-09T08:52:30.601366+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T08:52:30.601366+0000","last_undegraded":"2024-03-09T08:52:30.601366+0000","last_fullsized":"2024-03-09T08:52:30.601366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:45.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597465+0000","last_change":"2024-03-09T08:52:29.596486+0000","last_active":"2024-03-09T08:52:30.597465+0000","last_peered":"2024-03-09T08:52:30.597465+0000","last_clean":"2024-03-09T08:52:30.597465+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T08:52:30.597465+0000","last_undegraded":"2024-03-09T08:52:30.597465+0000","last_fullsized":"2024-03-09T08:52:30.597465+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:40:16.587782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577604+0000","last_change":"2024-03-09T08:52:18.227365+0000","last_active":"2024-03-09T08:52:30.577604+0000","last_peered":"2024-03-09T08:52:30.577604+0000","last_clean":"2024-03-09T08:52:30.577604+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T08:52:30.577604+0000","last_undegraded":"2024-03-09T08:52:30.577604+0000","last_fullsized":"2024-03-09T08:52:30.577604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:26.357063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945950+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T08:52:30.945950+0000","last_peered":"2024-03-09T08:52:30.945950+0000","last_clean":"2024-03-09T08:52:30.945950+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T08:52:30.945950+0000","last_undegraded":"2024-03-09T08:52:30.945950+0000","last_fullsized":"2024-03-09T08:52:30.945950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601516+0000","last_change":"2024-03-09T08:52:28.578096+0000","last_active":"2024-03-09T08:52:30.601516+0000","last_peered":"2024-03-09T08:52:30.601516+0000","last_clean":"2024-03-09T08:52:30.601516+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T08:52:30.601516+0000","last_undegraded":"2024-03-09T08:52:30.601516+0000","last_fullsized":"2024-03-09T08:52:30.601516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:42.645517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943375+0000","last_change":"2024-03-09T08:52:29.590834+0000","last_active":"2024-03-09T08:52:30.943375+0000","last_peered":"2024-03-09T08:52:30.943375+0000","last_clean":"2024-03-09T08:52:30.943375+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T08:52:30.943375+0000","last_undegraded":"2024-03-09T08:52:30.943375+0000","last_fullsized":"2024-03-09T08:52:30.943375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:24:06.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.600924+0000","last_change":"2024-03-09T08:52:18.235832+0000","last_active":"2024-03-09T08:52:30.600924+0000","last_peered":"2024-03-09T08:52:30.600924+0000","last_clean":"2024-03-09T08:52:30.600924+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T08:52:30.600924+0000","last_undegraded":"2024-03-09T08:52:30.600924+0000","last_fullsized":"2024-03-09T08:52:30.600924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:38:58.751526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942505+0000","last_change":"2024-03-09T08:52:17.271847+0000","last_active":"2024-03-09T08:52:30.942505+0000","last_peered":"2024-03-09T08:52:30.942505+0000","last_clean":"2024-03-09T08:52:30.942505+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T08:52:30.942505+0000","last_undegraded":"2024-03-09T08:52:30.942505+0000","last_fullsized":"2024-03-09T08:52:30.942505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:27.418674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602301+0000","last_change":"2024-03-09T08:52:28.588636+0000","last_active":"2024-03-09T08:52:30.602301+0000","last_peered":"2024-03-09T08:52:30.602301+0000","last_clean":"2024-03-09T08:52:30.602301+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T08:52:30.602301+0000","last_undegraded":"2024-03-09T08:52:30.602301+0000","last_fullsized":"2024-03-09T08:52:30.602301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:21.601182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947090+0000","last_change":"2024-03-09T08:52:29.606833+0000","last_active":"2024-03-09T08:52:30.947090+0000","last_peered":"2024-03-09T08:52:30.947090+0000","last_clean":"2024-03-09T08:52:30.947090+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T08:52:30.947090+0000","last_undegraded":"2024-03-09T08:52:30.947090+0000","last_fullsized":"2024-03-09T08:52:30.947090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:40.299681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597339+0000","last_change":"2024-03-09T08:52:18.214396+0000","last_active":"2024-03-09T08:52:30.597339+0000","last_peered":"2024-03-09T08:52:30.597339+0000","last_clean":"2024-03-09T08:52:30.597339+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T08:52:30.597339+0000","last_undegraded":"2024-03-09T08:52:30.597339+0000","last_fullsized":"2024-03-09T08:52:30.597339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:24:10.719351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944007+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T08:52:30.944007+0000","last_peered":"2024-03-09T08:52:30.944007+0000","last_clean":"2024-03-09T08:52:30.944007+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T08:52:30.944007+0000","last_undegraded":"2024-03-09T08:52:30.944007+0000","last_fullsized":"2024-03-09T08:52:30.944007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.599908+0000","last_change":"2024-03-09T08:52:28.578599+0000","last_active":"2024-03-09T08:52:30.599908+0000","last_peered":"2024-03-09T08:52:30.599908+0000","last_clean":"2024-03-09T08:52:30.599908+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T08:52:30.599908+0000","last_undegraded":"2024-03-09T08:52:30.599908+0000","last_fullsized":"2024-03-09T08:52:30.599908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:52:31.328248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943920+0000","last_change":"2024-03-09T08:52:29.591872+0000","last_active":"2024-03-09T08:52:30.943920+0000","last_peered":"2024-03-09T08:52:30.943920+0000","last_clean":"2024-03-09T08:52:30.943920+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T08:52:30.943920+0000","last_undegraded":"2024-03-09T08:52:30.943920+0000","last_fullsized":"2024-03-09T08:52:30.943920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:44:17.870183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576290+0000","last_change":"2024-03-09T08:52:18.235367+0000","last_active":"2024-03-09T08:52:30.576290+0000","last_peered":"2024-03-09T08:52:30.576290+0000","last_clean":"2024-03-09T08:52:30.576290+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T08:52:30.576290+0000","last_undegraded":"2024-03-09T08:52:30.576290+0000","last_fullsized":"2024-03-09T08:52:30.576290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:40.736440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946321+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T08:52:30.946321+0000","last_peered":"2024-03-09T08:52:30.946321+0000","last_clean":"2024-03-09T08:52:30.946321+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T08:52:30.946321+0000","last_undegraded":"2024-03-09T08:52:30.946321+0000","last_fullsized":"2024-03-09T08:52:30.946321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944699+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T08:52:30.944699+0000","last_peered":"2024-03-09T08:52:30.944699+0000","last_clean":"2024-03-09T08:52:30.944699+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T08:52:30.944699+0000","last_undegraded":"2024-03-09T08:52:30.944699+0000","last_fullsized":"2024-03-09T08:52:30.944699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577815+0000","last_change":"2024-03-09T08:52:28.577973+0000","last_active":"2024-03-09T08:52:30.577815+0000","last_peered":"2024-03-09T08:52:30.577815+0000","last_clean":"2024-03-09T08:52:30.577815+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T08:52:30.577815+0000","last_undegraded":"2024-03-09T08:52:30.577815+0000","last_fullsized":"2024-03-09T08:52:30.577815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:13.291622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943332+0000","last_change":"2024-03-09T08:52:29.599601+0000","last_active":"2024-03-09T08:52:30.943332+0000","last_peered":"2024-03-09T08:52:30.943332+0000","last_clean":"2024-03-09T08:52:30.943332+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T08:52:30.943332+0000","last_undegraded":"2024-03-09T08:52:30.943332+0000","last_fullsized":"2024-03-09T08:52:30.943332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:11:17.424330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948439+0000","last_change":"2024-03-09T08:52:18.242671+0000","last_active":"2024-03-09T08:52:30.948439+0000","last_peered":"2024-03-09T08:52:30.948439+0000","last_clean":"2024-03-09T08:52:30.948439+0000","last_became_active":"2024-03-09T08:52:18.242258+0000","last_became_peered":"2024-03-09T08:52:18.242258+0000","last_unstale":"2024-03-09T08:52:30.948439+0000","last_undegraded":"2024-03-09T08:52:30.948439+0000","last_fullsized":"2024-03-09T08:52:30.948439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:15.030669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943179+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T08:52:30.943179+0000","last_peered":"2024-03-09T08:52:30.943179+0000","last_clean":"2024-03-09T08:52:30.943179+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T08:52:30.943179+0000","last_undegraded":"2024-03-09T08:52:30.943179+0000","last_fullsized":"2024-03-09T08:52:30.943179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:27.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576159+0000","last_change":"2024-03-09T08:52:28.589278+0000","last_active":"2024-03-09T08:52:30.576159+0000","last_peered":"2024-03-09T08:52:30.576159+0000","last_clean":"2024-03-09T08:52:30.576159+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T08:52:30.576159+0000","last_undegraded":"2024-03-09T08:52:30.576159+0000","last_fullsized":"2024-03-09T08:52:30.576159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:16:26.449275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942717+0000","last_change":"2024-03-09T08:52:29.600332+0000","last_active":"2024-03-09T08:52:30.942717+0000","last_peered":"2024-03-09T08:52:30.942717+0000","last_clean":"2024-03-09T08:52:30.942717+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T08:52:30.942717+0000","last_undegraded":"2024-03-09T08:52:30.942717+0000","last_fullsized":"2024-03-09T08:52:30.942717+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:36:41.422546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947852+0000","last_change":"2024-03-09T08:52:18.250433+0000","last_active":"2024-03-09T08:52:30.947852+0000","last_peered":"2024-03-09T08:52:30.947852+0000","last_clean":"2024-03-09T08:52:30.947852+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T08:52:30.947852+0000","last_undegraded":"2024-03-09T08:52:30.947852+0000","last_fullsized":"2024-03-09T08:52:30.947852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:29.742337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602070+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T08:52:30.602070+0000","last_peered":"2024-03-09T08:52:30.602070+0000","last_clean":"2024-03-09T08:52:30.602070+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T08:52:30.602070+0000","last_undegraded":"2024-03-09T08:52:30.602070+0000","last_fullsized":"2024-03-09T08:52:30.602070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:26:35.574903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597442+0000","last_change":"2024-03-09T08:52:28.583181+0000","last_active":"2024-03-09T08:52:30.597442+0000","last_peered":"2024-03-09T08:52:30.597442+0000","last_clean":"2024-03-09T08:52:30.597442+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T08:52:30.597442+0000","last_undegraded":"2024-03-09T08:52:30.597442+0000","last_fullsized":"2024-03-09T08:52:30.597442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:58:50.061324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943982+0000","last_change":"2024-03-09T08:52:29.590141+0000","last_active":"2024-03-09T08:52:30.943982+0000","last_peered":"2024-03-09T08:52:30.943982+0000","last_clean":"2024-03-09T08:52:30.943982+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T08:52:30.943982+0000","last_undegraded":"2024-03-09T08:52:30.943982+0000","last_fullsized":"2024-03-09T08:52:30.943982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:23.981897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948851+0000","last_change":"2024-03-09T08:52:18.250420+0000","last_active":"2024-03-09T08:52:30.948851+0000","last_peered":"2024-03-09T08:52:30.948851+0000","last_clean":"2024-03-09T08:52:30.948851+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T08:52:30.948851+0000","last_undegraded":"2024-03-09T08:52:30.948851+0000","last_fullsized":"2024-03-09T08:52:30.948851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:48.826066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943461+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T08:52:30.943461+0000","last_peered":"2024-03-09T08:52:30.943461+0000","last_clean":"2024-03-09T08:52:30.943461+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T08:52:30.943461+0000","last_undegraded":"2024-03-09T08:52:30.943461+0000","last_fullsized":"2024-03-09T08:52:30.943461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:16.138216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944811+0000","last_change":"2024-03-09T08:52:28.587637+0000","last_active":"2024-03-09T08:52:30.944811+0000","last_peered":"2024-03-09T08:52:30.944811+0000","last_clean":"2024-03-09T08:52:30.944811+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T08:52:30.944811+0000","last_undegraded":"2024-03-09T08:52:30.944811+0000","last_fullsized":"2024-03-09T08:52:30.944811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:31:32.362061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942477+0000","last_change":"2024-03-09T08:52:29.586180+0000","last_active":"2024-03-09T08:52:30.942477+0000","last_peered":"2024-03-09T08:52:30.942477+0000","last_clean":"2024-03-09T08:52:30.942477+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T08:52:30.942477+0000","last_undegraded":"2024-03-09T08:52:30.942477+0000","last_fullsized":"2024-03-09T08:52:30.942477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:35:21.522181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948836+0000","last_change":"2024-03-09T08:52:18.246145+0000","last_active":"2024-03-09T08:52:30.948836+0000","last_peered":"2024-03-09T08:52:30.948836+0000","last_clean":"2024-03-09T08:52:30.948836+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T08:52:30.948836+0000","last_undegraded":"2024-03-09T08:52:30.948836+0000","last_fullsized":"2024-03-09T08:52:30.948836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:29:36.213774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578625+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T08:52:30.578625+0000","last_peered":"2024-03-09T08:52:30.578625+0000","last_clean":"2024-03-09T08:52:30.578625+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T08:52:30.578625+0000","last_undegraded":"2024-03-09T08:52:30.578625+0000","last_fullsized":"2024-03-09T08:52:30.578625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:10:34.698235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575615+0000","last_change":"2024-03-09T08:52:28.582308+0000","last_active":"2024-03-09T08:52:30.575615+0000","last_peered":"2024-03-09T08:52:30.575615+0000","last_clean":"2024-03-09T08:52:30.575615+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T08:52:30.575615+0000","last_undegraded":"2024-03-09T08:52:30.575615+0000","last_fullsized":"2024-03-09T08:52:30.575615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:04.918893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577162+0000","last_change":"2024-03-09T08:52:29.610596+0000","last_active":"2024-03-09T08:52:30.577162+0000","last_peered":"2024-03-09T08:52:30.577162+0000","last_clean":"2024-03-09T08:52:30.577162+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T08:52:30.577162+0000","last_undegraded":"2024-03-09T08:52:30.577162+0000","last_fullsized":"2024-03-09T08:52:30.577162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:19:47.313981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944192+0000","last_change":"2024-03-09T08:52:18.248755+0000","last_active":"2024-03-09T08:52:30.944192+0000","last_peered":"2024-03-09T08:52:30.944192+0000","last_clean":"2024-03-09T08:52:30.944192+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T08:52:30.944192+0000","last_undegraded":"2024-03-09T08:52:30.944192+0000","last_fullsized":"2024-03-09T08:52:30.944192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:37.083738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942668+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T08:52:30.942668+0000","last_peered":"2024-03-09T08:52:30.942668+0000","last_clean":"2024-03-09T08:52:30.942668+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T08:52:30.942668+0000","last_undegraded":"2024-03-09T08:52:30.942668+0000","last_fullsized":"2024-03-09T08:52:30.942668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:31.859924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598089+0000","last_change":"2024-03-09T08:52:28.575975+0000","last_active":"2024-03-09T08:52:30.598089+0000","last_peered":"2024-03-09T08:52:30.598089+0000","last_clean":"2024-03-09T08:52:30.598089+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T08:52:30.598089+0000","last_undegraded":"2024-03-09T08:52:30.598089+0000","last_fullsized":"2024-03-09T08:52:30.598089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:51:24.405316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947555+0000","last_change":"2024-03-09T08:52:29.575752+0000","last_active":"2024-03-09T08:52:30.947555+0000","last_peered":"2024-03-09T08:52:30.947555+0000","last_clean":"2024-03-09T08:52:30.947555+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T08:52:30.947555+0000","last_undegraded":"2024-03-09T08:52:30.947555+0000","last_fullsized":"2024-03-09T08:52:30.947555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:31:51.847871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575309+0000","last_change":"2024-03-09T08:52:18.245073+0000","last_active":"2024-03-09T08:52:30.575309+0000","last_peered":"2024-03-09T08:52:30.575309+0000","last_clean":"2024-03-09T08:52:30.575309+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T08:52:30.575309+0000","last_undegraded":"2024-03-09T08:52:30.575309+0000","last_fullsized":"2024-03-09T08:52:30.575309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:40:15.390735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577484+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T08:52:30.577484+0000","last_peered":"2024-03-09T08:52:30.577484+0000","last_clean":"2024-03-09T08:52:30.577484+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T08:52:30.577484+0000","last_undegraded":"2024-03-09T08:52:30.577484+0000","last_fullsized":"2024-03-09T08:52:30.577484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:07:29.577461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577273+0000","last_change":"2024-03-09T08:52:28.588312+0000","last_active":"2024-03-09T08:52:30.577273+0000","last_peered":"2024-03-09T08:52:30.577273+0000","last_clean":"2024-03-09T08:52:30.577273+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T08:52:30.577273+0000","last_undegraded":"2024-03-09T08:52:30.577273+0000","last_fullsized":"2024-03-09T08:52:30.577273+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:14:52.895618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946368+0000","last_change":"2024-03-09T08:52:29.596647+0000","last_active":"2024-03-09T08:52:30.946368+0000","last_peered":"2024-03-09T08:52:30.946368+0000","last_clean":"2024-03-09T08:52:30.946368+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T08:52:30.946368+0000","last_undegraded":"2024-03-09T08:52:30.946368+0000","last_fullsized":"2024-03-09T08:52:30.946368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:35:28.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944097+0000","last_change":"2024-03-09T08:52:18.248093+0000","last_active":"2024-03-09T08:52:30.944097+0000","last_peered":"2024-03-09T08:52:30.944097+0000","last_clean":"2024-03-09T08:52:30.944097+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T08:52:30.944097+0000","last_undegraded":"2024-03-09T08:52:30.944097+0000","last_fullsized":"2024-03-09T08:52:30.944097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:29.414335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945767+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T08:52:30.945767+0000","last_peered":"2024-03-09T08:52:30.945767+0000","last_clean":"2024-03-09T08:52:30.945767+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T08:52:30.945767+0000","last_undegraded":"2024-03-09T08:52:30.945767+0000","last_fullsized":"2024-03-09T08:52:30.945767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576830+0000","last_change":"2024-03-09T08:52:28.585562+0000","last_active":"2024-03-09T08:52:30.576830+0000","last_peered":"2024-03-09T08:52:30.576830+0000","last_clean":"2024-03-09T08:52:30.576830+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T08:52:30.576830+0000","last_undegraded":"2024-03-09T08:52:30.576830+0000","last_fullsized":"2024-03-09T08:52:30.576830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:28:50.544795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576833+0000","last_change":"2024-03-09T08:52:29.581330+0000","last_active":"2024-03-09T08:52:30.576833+0000","last_peered":"2024-03-09T08:52:30.576833+0000","last_clean":"2024-03-09T08:52:30.576833+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T08:52:30.576833+0000","last_undegraded":"2024-03-09T08:52:30.576833+0000","last_fullsized":"2024-03-09T08:52:30.576833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:55:38.041794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597506+0000","last_change":"2024-03-09T08:52:18.236146+0000","last_active":"2024-03-09T08:52:30.597506+0000","last_peered":"2024-03-09T08:52:30.597506+0000","last_clean":"2024-03-09T08:52:30.597506+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T08:52:30.597506+0000","last_undegraded":"2024-03-09T08:52:30.597506+0000","last_fullsized":"2024-03-09T08:52:30.597506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:18:51.511926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598307+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T08:52:30.598307+0000","last_peered":"2024-03-09T08:52:30.598307+0000","last_clean":"2024-03-09T08:52:30.598307+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T08:52:30.598307+0000","last_undegraded":"2024-03-09T08:52:30.598307+0000","last_fullsized":"2024-03-09T08:52:30.598307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:22:05.504557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947315+0000","last_change":"2024-03-09T08:52:28.585697+0000","last_active":"2024-03-09T08:52:30.947315+0000","last_peered":"2024-03-09T08:52:30.947315+0000","last_clean":"2024-03-09T08:52:30.947315+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T08:52:30.947315+0000","last_undegraded":"2024-03-09T08:52:30.947315+0000","last_fullsized":"2024-03-09T08:52:30.947315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:50:23.095008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576346+0000","last_change":"2024-03-09T08:52:29.578803+0000","last_active":"2024-03-09T08:52:30.576346+0000","last_peered":"2024-03-09T08:52:30.576346+0000","last_clean":"2024-03-09T08:52:30.576346+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T08:52:30.576346+0000","last_undegraded":"2024-03-09T08:52:30.576346+0000","last_fullsized":"2024-03-09T08:52:30.576346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:22:47.690873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948118+0000","last_change":"2024-03-09T08:52:18.242496+0000","last_active":"2024-03-09T08:52:30.948118+0000","last_peered":"2024-03-09T08:52:30.948118+0000","last_clean":"2024-03-09T08:52:30.948118+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T08:52:30.948118+0000","last_undegraded":"2024-03-09T08:52:30.948118+0000","last_fullsized":"2024-03-09T08:52:30.948118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:35:17.142983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578358+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T08:52:30.578358+0000","last_peered":"2024-03-09T08:52:30.578358+0000","last_clean":"2024-03-09T08:52:30.578358+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T08:52:30.578358+0000","last_undegraded":"2024-03-09T08:52:30.578358+0000","last_fullsized":"2024-03-09T08:52:30.578358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:01.724043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601055+0000","last_change":"2024-03-09T08:52:28.583685+0000","last_active":"2024-03-09T08:52:30.601055+0000","last_peered":"2024-03-09T08:52:30.601055+0000","last_clean":"2024-03-09T08:52:30.601055+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T08:52:30.601055+0000","last_undegraded":"2024-03-09T08:52:30.601055+0000","last_fullsized":"2024-03-09T08:52:30.601055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:58:00.899620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601468+0000","last_change":"2024-03-09T08:52:29.585912+0000","last_active":"2024-03-09T08:52:30.601468+0000","last_peered":"2024-03-09T08:52:30.601468+0000","last_clean":"2024-03-09T08:52:30.601468+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T08:52:30.601468+0000","last_undegraded":"2024-03-09T08:52:30.601468+0000","last_fullsized":"2024-03-09T08:52:30.601468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:24:47.261191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577039+0000","last_change":"2024-03-09T08:52:18.244654+0000","last_active":"2024-03-09T08:52:30.577039+0000","last_peered":"2024-03-09T08:52:30.577039+0000","last_clean":"2024-03-09T08:52:30.577039+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T08:52:30.577039+0000","last_undegraded":"2024-03-09T08:52:30.577039+0000","last_fullsized":"2024-03-09T08:52:30.577039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:08:58.611725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601999+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T08:52:30.601999+0000","last_peered":"2024-03-09T08:52:30.601999+0000","last_clean":"2024-03-09T08:52:30.601999+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T08:52:30.601999+0000","last_undegraded":"2024-03-09T08:52:30.601999+0000","last_fullsized":"2024-03-09T08:52:30.601999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:41:10.578669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577095+0000","last_change":"2024-03-09T08:52:28.568794+0000","last_active":"2024-03-09T08:52:30.577095+0000","last_peered":"2024-03-09T08:52:30.577095+0000","last_clean":"2024-03-09T08:52:30.577095+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T08:52:30.577095+0000","last_undegraded":"2024-03-09T08:52:30.577095+0000","last_fullsized":"2024-03-09T08:52:30.577095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:44:21.721511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577942+0000","last_change":"2024-03-09T08:52:29.586705+0000","last_active":"2024-03-09T08:52:30.577942+0000","last_peered":"2024-03-09T08:52:30.577942+0000","last_clean":"2024-03-09T08:52:30.577942+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T08:52:30.577942+0000","last_undegraded":"2024-03-09T08:52:30.577942+0000","last_fullsized":"2024-03-09T08:52:30.577942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:26.353156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597722+0000","last_change":"2024-03-09T08:52:18.236628+0000","last_active":"2024-03-09T08:52:30.597722+0000","last_peered":"2024-03-09T08:52:30.597722+0000","last_clean":"2024-03-09T08:52:30.597722+0000","last_became_active":"2024-03-09T08:52:18.235321+0000","last_became_peered":"2024-03-09T08:52:18.235321+0000","last_unstale":"2024-03-09T08:52:30.597722+0000","last_undegraded":"2024-03-09T08:52:30.597722+0000","last_fullsized":"2024-03-09T08:52:30.597722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:13:59.221325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948276+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T08:52:30.948276+0000","last_peered":"2024-03-09T08:52:30.948276+0000","last_clean":"2024-03-09T08:52:30.948276+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T08:52:30.948276+0000","last_undegraded":"2024-03-09T08:52:30.948276+0000","last_fullsized":"2024-03-09T08:52:30.948276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575808+0000","last_change":"2024-03-09T08:52:28.583697+0000","last_active":"2024-03-09T08:52:30.575808+0000","last_peered":"2024-03-09T08:52:30.575808+0000","last_clean":"2024-03-09T08:52:30.575808+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T08:52:30.575808+0000","last_undegraded":"2024-03-09T08:52:30.575808+0000","last_fullsized":"2024-03-09T08:52:30.575808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:41.552156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945076+0000","last_change":"2024-03-09T08:52:29.590854+0000","last_active":"2024-03-09T08:52:30.945076+0000","last_peered":"2024-03-09T08:52:30.945076+0000","last_clean":"2024-03-09T08:52:30.945076+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T08:52:30.945076+0000","last_undegraded":"2024-03-09T08:52:30.945076+0000","last_fullsized":"2024-03-09T08:52:30.945076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:09.568826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947914+0000","last_change":"2024-03-09T08:52:18.251648+0000","last_active":"2024-03-09T08:52:30.947914+0000","last_peered":"2024-03-09T08:52:30.947914+0000","last_clean":"2024-03-09T08:52:30.947914+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T08:52:30.947914+0000","last_undegraded":"2024-03-09T08:52:30.947914+0000","last_fullsized":"2024-03-09T08:52:30.947914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:54:45.721673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.579118+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T08:52:30.579118+0000","last_peered":"2024-03-09T08:52:30.579118+0000","last_clean":"2024-03-09T08:52:30.579118+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T08:52:30.579118+0000","last_undegraded":"2024-03-09T08:52:30.579118+0000","last_fullsized":"2024-03-09T08:52:30.579118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:31.731626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575378+0000","last_change":"2024-03-09T08:52:28.586606+0000","last_active":"2024-03-09T08:52:30.575378+0000","last_peered":"2024-03-09T08:52:30.575378+0000","last_clean":"2024-03-09T08:52:30.575378+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T08:52:30.575378+0000","last_undegraded":"2024-03-09T08:52:30.575378+0000","last_fullsized":"2024-03-09T08:52:30.575378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:17:40.902403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578219+0000","last_change":"2024-03-09T08:52:29.584512+0000","last_active":"2024-03-09T08:52:30.578219+0000","last_peered":"2024-03-09T08:52:30.578219+0000","last_clean":"2024-03-09T08:52:30.578219+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T08:52:30.578219+0000","last_undegraded":"2024-03-09T08:52:30.578219+0000","last_fullsized":"2024-03-09T08:52:30.578219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:39:20.330443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597975+0000","last_change":"2024-03-09T08:52:28.586201+0000","last_active":"2024-03-09T08:52:30.597975+0000","last_peered":"2024-03-09T08:52:30.597975+0000","last_clean":"2024-03-09T08:52:30.597975+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T08:52:30.597975+0000","last_undegraded":"2024-03-09T08:52:30.597975+0000","last_fullsized":"2024-03-09T08:52:30.597975+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:28:03.471646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601039+0000","last_change":"2024-03-09T08:52:18.241156+0000","last_active":"2024-03-09T08:52:30.601039+0000","last_peered":"2024-03-09T08:52:30.601039+0000","last_clean":"2024-03-09T08:52:30.601039+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T08:52:30.601039+0000","last_undegraded":"2024-03-09T08:52:30.601039+0000","last_fullsized":"2024-03-09T08:52:30.601039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:55.323770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946394+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T08:52:30.946394+0000","last_peered":"2024-03-09T08:52:30.946394+0000","last_clean":"2024-03-09T08:52:30.946394+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T08:52:30.946394+0000","last_undegraded":"2024-03-09T08:52:30.946394+0000","last_fullsized":"2024-03-09T08:52:30.946394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597714+0000","last_change":"2024-03-09T08:52:29.591085+0000","last_active":"2024-03-09T08:52:30.597714+0000","last_peered":"2024-03-09T08:52:30.597714+0000","last_clean":"2024-03-09T08:52:30.597714+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T08:52:30.597714+0000","last_undegraded":"2024-03-09T08:52:30.597714+0000","last_fullsized":"2024-03-09T08:52:30.597714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:40:27.878922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578971+0000","last_change":"2024-03-09T08:52:28.585631+0000","last_active":"2024-03-09T08:52:30.578971+0000","last_peered":"2024-03-09T08:52:30.578971+0000","last_clean":"2024-03-09T08:52:30.578971+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T08:52:30.578971+0000","last_undegraded":"2024-03-09T08:52:30.578971+0000","last_fullsized":"2024-03-09T08:52:30.578971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:06:03.569759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576252+0000","last_change":"2024-03-09T08:52:18.234400+0000","last_active":"2024-03-09T08:52:30.576252+0000","last_peered":"2024-03-09T08:52:30.576252+0000","last_clean":"2024-03-09T08:52:30.576252+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T08:52:30.576252+0000","last_undegraded":"2024-03-09T08:52:30.576252+0000","last_fullsized":"2024-03-09T08:52:30.576252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:42:14.057367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576643+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T08:52:30.576643+0000","last_peered":"2024-03-09T08:52:30.576643+0000","last_clean":"2024-03-09T08:52:30.576643+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T08:52:30.576643+0000","last_undegraded":"2024-03-09T08:52:30.576643+0000","last_fullsized":"2024-03-09T08:52:30.576643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601916+0000","last_change":"2024-03-09T08:52:29.588928+0000","last_active":"2024-03-09T08:52:30.601916+0000","last_peered":"2024-03-09T08:52:30.601916+0000","last_clean":"2024-03-09T08:52:30.601916+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T08:52:30.601916+0000","last_undegraded":"2024-03-09T08:52:30.601916+0000","last_fullsized":"2024-03-09T08:52:30.601916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:32:27.061041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576515+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T08:52:30.576515+0000","last_peered":"2024-03-09T08:52:30.576515+0000","last_clean":"2024-03-09T08:52:30.576515+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T08:52:30.576515+0000","last_undegraded":"2024-03-09T08:52:30.576515+0000","last_fullsized":"2024-03-09T08:52:30.576515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598213+0000","last_change":"2024-03-09T08:52:18.231141+0000","last_active":"2024-03-09T08:52:30.598213+0000","last_peered":"2024-03-09T08:52:30.598213+0000","last_clean":"2024-03-09T08:52:30.598213+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T08:52:30.598213+0000","last_undegraded":"2024-03-09T08:52:30.598213+0000","last_fullsized":"2024-03-09T08:52:30.598213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:25:00.471222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943012+0000","last_change":"2024-03-09T08:52:28.579315+0000","last_active":"2024-03-09T08:52:30.943012+0000","last_peered":"2024-03-09T08:52:30.943012+0000","last_clean":"2024-03-09T08:52:30.943012+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T08:52:30.943012+0000","last_undegraded":"2024-03-09T08:52:30.943012+0000","last_fullsized":"2024-03-09T08:52:30.943012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:57:17.301677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576561+0000","last_change":"2024-03-09T08:52:29.585154+0000","last_active":"2024-03-09T08:52:30.576561+0000","last_peered":"2024-03-09T08:52:30.576561+0000","last_clean":"2024-03-09T08:52:30.576561+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T08:52:30.576561+0000","last_undegraded":"2024-03-09T08:52:30.576561+0000","last_fullsized":"2024-03-09T08:52:30.576561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:43:35.679108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576704+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T08:52:30.576704+0000","last_peered":"2024-03-09T08:52:30.576704+0000","last_clean":"2024-03-09T08:52:30.576704+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T08:52:30.576704+0000","last_undegraded":"2024-03-09T08:52:30.576704+0000","last_fullsized":"2024-03-09T08:52:30.576704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577869+0000","last_change":"2024-03-09T08:52:18.229533+0000","last_active":"2024-03-09T08:52:30.577869+0000","last_peered":"2024-03-09T08:52:30.577869+0000","last_clean":"2024-03-09T08:52:30.577869+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T08:52:30.577869+0000","last_undegraded":"2024-03-09T08:52:30.577869+0000","last_fullsized":"2024-03-09T08:52:30.577869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:17.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576387+0000","last_change":"2024-03-09T08:52:28.585877+0000","last_active":"2024-03-09T08:52:30.576387+0000","last_peered":"2024-03-09T08:52:30.576387+0000","last_clean":"2024-03-09T08:52:30.576387+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T08:52:30.576387+0000","last_undegraded":"2024-03-09T08:52:30.576387+0000","last_fullsized":"2024-03-09T08:52:30.576387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:30:20.036385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577546+0000","last_change":"2024-03-09T08:52:29.592253+0000","last_active":"2024-03-09T08:52:30.577546+0000","last_peered":"2024-03-09T08:52:30.577546+0000","last_clean":"2024-03-09T08:52:30.577546+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T08:52:30.577546+0000","last_undegraded":"2024-03-09T08:52:30.577546+0000","last_fullsized":"2024-03-09T08:52:30.577546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:50:33.525541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575154+0000","last_change":"2024-03-09T08:52:18.237869+0000","last_active":"2024-03-09T08:52:30.575154+0000","last_peered":"2024-03-09T08:52:30.575154+0000","last_clean":"2024-03-09T08:52:30.575154+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T08:52:30.575154+0000","last_undegraded":"2024-03-09T08:52:30.575154+0000","last_fullsized":"2024-03-09T08:52:30.575154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:01.571866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.602125+0000","last_change":"2024-03-09T08:52:29.601300+0000","last_active":"2024-03-09T08:52:30.602125+0000","last_peered":"2024-03-09T08:52:30.602125+0000","last_clean":"2024-03-09T08:52:30.602125+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T08:52:30.602125+0000","last_undegraded":"2024-03-09T08:52:30.602125+0000","last_fullsized":"2024-03-09T08:52:30.602125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:46:04.870574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942394+0000","last_change":"2024-03-09T08:52:18.226980+0000","last_active":"2024-03-09T08:52:30.942394+0000","last_peered":"2024-03-09T08:52:30.942394+0000","last_clean":"2024-03-09T08:52:30.942394+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T08:52:30.942394+0000","last_undegraded":"2024-03-09T08:52:30.942394+0000","last_fullsized":"2024-03-09T08:52:30.942394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:28.312576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946757+0000","last_change":"2024-03-09T08:52:29.575612+0000","last_active":"2024-03-09T08:52:30.946757+0000","last_peered":"2024-03-09T08:52:30.946757+0000","last_clean":"2024-03-09T08:52:30.946757+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T08:52:30.946757+0000","last_undegraded":"2024-03-09T08:52:30.946757+0000","last_fullsized":"2024-03-09T08:52:30.946757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:58.852983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945644+0000","last_change":"2024-03-09T08:52:18.241628+0000","last_active":"2024-03-09T08:52:30.945644+0000","last_peered":"2024-03-09T08:52:30.945644+0000","last_clean":"2024-03-09T08:52:30.945644+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T08:52:30.945644+0000","last_undegraded":"2024-03-09T08:52:30.945644+0000","last_fullsized":"2024-03-09T08:52:30.945644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:07:58.570641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601917+0000","last_change":"2024-03-09T08:52:29.588660+0000","last_active":"2024-03-09T08:52:30.601917+0000","last_peered":"2024-03-09T08:52:30.601917+0000","last_clean":"2024-03-09T08:52:30.601917+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T08:52:30.601917+0000","last_undegraded":"2024-03-09T08:52:30.601917+0000","last_fullsized":"2024-03-09T08:52:30.601917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:14.305320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.598085+0000","last_change":"2024-03-09T08:52:18.242257+0000","last_active":"2024-03-09T08:52:30.598085+0000","last_peered":"2024-03-09T08:52:30.598085+0000","last_clean":"2024-03-09T08:52:30.598085+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T08:52:30.598085+0000","last_undegraded":"2024-03-09T08:52:30.598085+0000","last_fullsized":"2024-03-09T08:52:30.598085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:40:00.854066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944026+0000","last_change":"2024-03-09T08:52:29.595057+0000","last_active":"2024-03-09T08:52:30.944026+0000","last_peered":"2024-03-09T08:52:30.944026+0000","last_clean":"2024-03-09T08:52:30.944026+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T08:52:30.944026+0000","last_undegraded":"2024-03-09T08:52:30.944026+0000","last_fullsized":"2024-03-09T08:52:30.944026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:33:37.474266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577091+0000","last_change":"2024-03-09T08:52:18.239951+0000","last_active":"2024-03-09T08:52:30.577091+0000","last_peered":"2024-03-09T08:52:30.577091+0000","last_clean":"2024-03-09T08:52:30.577091+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T08:52:30.577091+0000","last_undegraded":"2024-03-09T08:52:30.577091+0000","last_fullsized":"2024-03-09T08:52:30.577091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:51.102540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576730+0000","last_change":"2024-03-09T08:52:29.578993+0000","last_active":"2024-03-09T08:52:30.576730+0000","last_peered":"2024-03-09T08:52:30.576730+0000","last_clean":"2024-03-09T08:52:30.576730+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T08:52:30.576730+0000","last_undegraded":"2024-03-09T08:52:30.576730+0000","last_fullsized":"2024-03-09T08:52:30.576730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:50:53.692060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601995+0000","last_change":"2024-03-09T08:52:18.241038+0000","last_active":"2024-03-09T08:52:30.601995+0000","last_peered":"2024-03-09T08:52:30.601995+0000","last_clean":"2024-03-09T08:52:30.601995+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T08:52:30.601995+0000","last_undegraded":"2024-03-09T08:52:30.601995+0000","last_fullsized":"2024-03-09T08:52:30.601995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:02.169113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943444+0000","last_change":"2024-03-09T08:52:29.583869+0000","last_active":"2024-03-09T08:52:30.943444+0000","last_peered":"2024-03-09T08:52:30.943444+0000","last_clean":"2024-03-09T08:52:30.943444+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T08:52:30.943444+0000","last_undegraded":"2024-03-09T08:52:30.943444+0000","last_fullsized":"2024-03-09T08:52:30.943444+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:29:47.636741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578670+0000","last_change":"2024-03-09T08:52:18.247145+0000","last_active":"2024-03-09T08:52:30.578670+0000","last_peered":"2024-03-09T08:52:30.578670+0000","last_clean":"2024-03-09T08:52:30.578670+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T08:52:30.578670+0000","last_undegraded":"2024-03-09T08:52:30.578670+0000","last_fullsized":"2024-03-09T08:52:30.578670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:46:24.111224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577279+0000","last_change":"2024-03-09T08:52:29.587358+0000","last_active":"2024-03-09T08:52:30.577279+0000","last_peered":"2024-03-09T08:52:30.577279+0000","last_clean":"2024-03-09T08:52:30.577279+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T08:52:30.577279+0000","last_undegraded":"2024-03-09T08:52:30.577279+0000","last_fullsized":"2024-03-09T08:52:30.577279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:46.836801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577323+0000","last_change":"2024-03-09T08:52:18.247784+0000","last_active":"2024-03-09T08:52:30.577323+0000","last_peered":"2024-03-09T08:52:30.577323+0000","last_clean":"2024-03-09T08:52:30.577323+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T08:52:30.577323+0000","last_undegraded":"2024-03-09T08:52:30.577323+0000","last_fullsized":"2024-03-09T08:52:30.577323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:46:59.008220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577364+0000","last_change":"2024-03-09T08:52:29.589458+0000","last_active":"2024-03-09T08:52:30.577364+0000","last_peered":"2024-03-09T08:52:30.577364+0000","last_clean":"2024-03-09T08:52:30.577364+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T08:52:30.577364+0000","last_undegraded":"2024-03-09T08:52:30.577364+0000","last_fullsized":"2024-03-09T08:52:30.577364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:19:35.222184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945231+0000","last_change":"2024-03-09T08:52:18.239943+0000","last_active":"2024-03-09T08:52:30.945231+0000","last_peered":"2024-03-09T08:52:30.945231+0000","last_clean":"2024-03-09T08:52:30.945231+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T08:52:30.945231+0000","last_undegraded":"2024-03-09T08:52:30.945231+0000","last_fullsized":"2024-03-09T08:52:30.945231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:18.727030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.949211+0000","last_change":"2024-03-09T08:52:29.597147+0000","last_active":"2024-03-09T08:52:30.949211+0000","last_peered":"2024-03-09T08:52:30.949211+0000","last_clean":"2024-03-09T08:52:30.949211+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T08:52:30.949211+0000","last_undegraded":"2024-03-09T08:52:30.949211+0000","last_fullsized":"2024-03-09T08:52:30.949211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:44:47.144911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575715+0000","last_change":"2024-03-09T08:52:18.236642+0000","last_active":"2024-03-09T08:52:30.575715+0000","last_peered":"2024-03-09T08:52:30.575715+0000","last_clean":"2024-03-09T08:52:30.575715+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T08:52:30.575715+0000","last_undegraded":"2024-03-09T08:52:30.575715+0000","last_fullsized":"2024-03-09T08:52:30.575715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:51:46.825153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943044+0000","last_change":"2024-03-09T08:52:29.590810+0000","last_active":"2024-03-09T08:52:30.943044+0000","last_peered":"2024-03-09T08:52:30.943044+0000","last_clean":"2024-03-09T08:52:30.943044+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T08:52:30.943044+0000","last_undegraded":"2024-03-09T08:52:30.943044+0000","last_fullsized":"2024-03-09T08:52:30.943044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:21.534580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945569+0000","last_change":"2024-03-09T08:52:18.236855+0000","last_active":"2024-03-09T08:52:30.945569+0000","last_peered":"2024-03-09T08:52:30.945569+0000","last_clean":"2024-03-09T08:52:30.945569+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T08:52:30.945569+0000","last_undegraded":"2024-03-09T08:52:30.945569+0000","last_fullsized":"2024-03-09T08:52:30.945569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:21.941242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576311+0000","last_change":"2024-03-09T08:52:29.602209+0000","last_active":"2024-03-09T08:52:30.576311+0000","last_peered":"2024-03-09T08:52:30.576311+0000","last_clean":"2024-03-09T08:52:30.576311+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T08:52:30.576311+0000","last_undegraded":"2024-03-09T08:52:30.576311+0000","last_fullsized":"2024-03-09T08:52:30.576311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:11:33.349464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597936+0000","last_change":"2024-03-09T08:52:18.238373+0000","last_active":"2024-03-09T08:52:30.597936+0000","last_peered":"2024-03-09T08:52:30.597936+0000","last_clean":"2024-03-09T08:52:30.597936+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T08:52:30.597936+0000","last_undegraded":"2024-03-09T08:52:30.597936+0000","last_fullsized":"2024-03-09T08:52:30.597936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:05:31.480335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577230+0000","last_change":"2024-03-09T08:52:29.578120+0000","last_active":"2024-03-09T08:52:30.577230+0000","last_peered":"2024-03-09T08:52:30.577230+0000","last_clean":"2024-03-09T08:52:30.577230+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T08:52:30.577230+0000","last_undegraded":"2024-03-09T08:52:30.577230+0000","last_fullsized":"2024-03-09T08:52:30.577230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:13:27.623995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578214+0000","last_change":"2024-03-09T08:52:18.235565+0000","last_active":"2024-03-09T08:52:30.578214+0000","last_peered":"2024-03-09T08:52:30.578214+0000","last_clean":"2024-03-09T08:52:30.578214+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T08:52:30.578214+0000","last_undegraded":"2024-03-09T08:52:30.578214+0000","last_fullsized":"2024-03-09T08:52:30.578214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:52:14.994539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576439+0000","last_change":"2024-03-09T08:52:29.579668+0000","last_active":"2024-03-09T08:52:30.576439+0000","last_peered":"2024-03-09T08:52:30.576439+0000","last_clean":"2024-03-09T08:52:30.576439+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T08:52:30.576439+0000","last_undegraded":"2024-03-09T08:52:30.576439+0000","last_fullsized":"2024-03-09T08:52:30.576439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:12:15.272875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948818+0000","last_change":"2024-03-09T08:52:18.239801+0000","last_active":"2024-03-09T08:52:30.948818+0000","last_peered":"2024-03-09T08:52:30.948818+0000","last_clean":"2024-03-09T08:52:30.948818+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T08:52:30.948818+0000","last_undegraded":"2024-03-09T08:52:30.948818+0000","last_fullsized":"2024-03-09T08:52:30.948818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:38:06.836990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944799+0000","last_change":"2024-03-09T08:52:29.586986+0000","last_active":"2024-03-09T08:52:30.944799+0000","last_peered":"2024-03-09T08:52:30.944799+0000","last_clean":"2024-03-09T08:52:30.944799+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T08:52:30.944799+0000","last_undegraded":"2024-03-09T08:52:30.944799+0000","last_fullsized":"2024-03-09T08:52:30.944799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:37.382414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943359+0000","last_change":"2024-03-09T08:52:18.233582+0000","last_active":"2024-03-09T08:52:30.943359+0000","last_peered":"2024-03-09T08:52:30.943359+0000","last_clean":"2024-03-09T08:52:30.943359+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T08:52:30.943359+0000","last_undegraded":"2024-03-09T08:52:30.943359+0000","last_fullsized":"2024-03-09T08:52:30.943359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:49.313616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597247+0000","last_change":"2024-03-09T08:52:29.586673+0000","last_active":"2024-03-09T08:52:30.597247+0000","last_peered":"2024-03-09T08:52:30.597247+0000","last_clean":"2024-03-09T08:52:30.597247+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T08:52:30.597247+0000","last_undegraded":"2024-03-09T08:52:30.597247+0000","last_fullsized":"2024-03-09T08:52:30.597247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:23:49.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947293+0000","last_change":"2024-03-09T08:52:18.242349+0000","last_active":"2024-03-09T08:52:30.947293+0000","last_peered":"2024-03-09T08:52:30.947293+0000","last_clean":"2024-03-09T08:52:30.947293+0000","last_became_active":"2024-03-09T08:52:18.241995+0000","last_became_peered":"2024-03-09T08:52:18.241995+0000","last_unstale":"2024-03-09T08:52:30.947293+0000","last_undegraded":"2024-03-09T08:52:30.947293+0000","last_fullsized":"2024-03-09T08:52:30.947293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:29:09.977984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601965+0000","last_change":"2024-03-09T08:52:29.592702+0000","last_active":"2024-03-09T08:52:30.601965+0000","last_peered":"2024-03-09T08:52:30.601965+0000","last_clean":"2024-03-09T08:52:30.601965+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T08:52:30.601965+0000","last_undegraded":"2024-03-09T08:52:30.601965+0000","last_fullsized":"2024-03-09T08:52:30.601965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:39.344726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944970+0000","last_change":"2024-03-09T08:52:18.230289+0000","last_active":"2024-03-09T08:52:30.944970+0000","last_peered":"2024-03-09T08:52:30.944970+0000","last_clean":"2024-03-09T08:52:30.944970+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T08:52:30.944970+0000","last_undegraded":"2024-03-09T08:52:30.944970+0000","last_fullsized":"2024-03-09T08:52:30.944970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:34.507924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.578394+0000","last_change":"2024-03-09T08:52:29.608191+0000","last_active":"2024-03-09T08:52:30.578394+0000","last_peered":"2024-03-09T08:52:30.578394+0000","last_clean":"2024-03-09T08:52:30.578394+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T08:52:30.578394+0000","last_undegraded":"2024-03-09T08:52:30.578394+0000","last_fullsized":"2024-03-09T08:52:30.578394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:26:32.282196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601147+0000","last_change":"2024-03-09T08:52:18.236376+0000","last_active":"2024-03-09T08:52:30.601147+0000","last_peered":"2024-03-09T08:52:30.601147+0000","last_clean":"2024-03-09T08:52:30.601147+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T08:52:30.601147+0000","last_undegraded":"2024-03-09T08:52:30.601147+0000","last_fullsized":"2024-03-09T08:52:30.601147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:34:04.412643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.946339+0000","last_change":"2024-03-09T08:52:29.573685+0000","last_active":"2024-03-09T08:52:30.946339+0000","last_peered":"2024-03-09T08:52:30.946339+0000","last_clean":"2024-03-09T08:52:30.946339+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T08:52:30.946339+0000","last_undegraded":"2024-03-09T08:52:30.946339+0000","last_fullsized":"2024-03-09T08:52:30.946339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:41:41.839063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597356+0000","last_change":"2024-03-09T08:52:18.231316+0000","last_active":"2024-03-09T08:52:30.597356+0000","last_peered":"2024-03-09T08:52:30.597356+0000","last_clean":"2024-03-09T08:52:30.597356+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T08:52:30.597356+0000","last_undegraded":"2024-03-09T08:52:30.597356+0000","last_fullsized":"2024-03-09T08:52:30.597356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:25:25.715084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948285+0000","last_change":"2024-03-09T08:52:29.603272+0000","last_active":"2024-03-09T08:52:30.948285+0000","last_peered":"2024-03-09T08:52:30.948285+0000","last_clean":"2024-03-09T08:52:30.948285+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T08:52:30.948285+0000","last_undegraded":"2024-03-09T08:52:30.948285+0000","last_fullsized":"2024-03-09T08:52:30.948285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:07:58.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576412+0000","last_change":"2024-03-09T08:52:18.243645+0000","last_active":"2024-03-09T08:52:30.576412+0000","last_peered":"2024-03-09T08:52:30.576412+0000","last_clean":"2024-03-09T08:52:30.576412+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T08:52:30.576412+0000","last_undegraded":"2024-03-09T08:52:30.576412+0000","last_fullsized":"2024-03-09T08:52:30.576412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:19.501675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.576249+0000","last_change":"2024-03-09T08:52:29.587462+0000","last_active":"2024-03-09T08:52:30.576249+0000","last_peered":"2024-03-09T08:52:30.576249+0000","last_clean":"2024-03-09T08:52:30.576249+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T08:52:30.576249+0000","last_undegraded":"2024-03-09T08:52:30.576249+0000","last_fullsized":"2024-03-09T08:52:30.576249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:51.652225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577062+0000","last_change":"2024-03-09T08:52:18.233451+0000","last_active":"2024-03-09T08:52:30.577062+0000","last_peered":"2024-03-09T08:52:30.577062+0000","last_clean":"2024-03-09T08:52:30.577062+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T08:52:30.577062+0000","last_undegraded":"2024-03-09T08:52:30.577062+0000","last_fullsized":"2024-03-09T08:52:30.577062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:56:11.270081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942964+0000","last_change":"2024-03-09T08:52:29.599909+0000","last_active":"2024-03-09T08:52:30.942964+0000","last_peered":"2024-03-09T08:52:30.942964+0000","last_clean":"2024-03-09T08:52:30.942964+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T08:52:30.942964+0000","last_undegraded":"2024-03-09T08:52:30.942964+0000","last_fullsized":"2024-03-09T08:52:30.942964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:42.596067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577555+0000","last_change":"2024-03-09T08:52:18.234158+0000","last_active":"2024-03-09T08:52:30.577555+0000","last_peered":"2024-03-09T08:52:30.577555+0000","last_clean":"2024-03-09T08:52:30.577555+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T08:52:30.577555+0000","last_undegraded":"2024-03-09T08:52:30.577555+0000","last_fullsized":"2024-03-09T08:52:30.577555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:59:33.423558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944531+0000","last_change":"2024-03-09T08:52:29.597518+0000","last_active":"2024-03-09T08:52:30.944531+0000","last_peered":"2024-03-09T08:52:30.944531+0000","last_clean":"2024-03-09T08:52:30.944531+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T08:52:30.944531+0000","last_undegraded":"2024-03-09T08:52:30.944531+0000","last_fullsized":"2024-03-09T08:52:30.944531+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:32:32.452993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577830+0000","last_change":"2024-03-09T08:52:18.234648+0000","last_active":"2024-03-09T08:52:30.577830+0000","last_peered":"2024-03-09T08:52:30.577830+0000","last_clean":"2024-03-09T08:52:30.577830+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T08:52:30.577830+0000","last_undegraded":"2024-03-09T08:52:30.577830+0000","last_fullsized":"2024-03-09T08:52:30.577830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:03.927280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.944900+0000","last_change":"2024-03-09T08:52:29.580608+0000","last_active":"2024-03-09T08:52:30.944900+0000","last_peered":"2024-03-09T08:52:30.944900+0000","last_clean":"2024-03-09T08:52:30.944900+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T08:52:30.944900+0000","last_undegraded":"2024-03-09T08:52:30.944900+0000","last_fullsized":"2024-03-09T08:52:30.944900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:00:37.184351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948628+0000","last_change":"2024-03-09T08:52:18.253038+0000","last_active":"2024-03-09T08:52:30.948628+0000","last_peered":"2024-03-09T08:52:30.948628+0000","last_clean":"2024-03-09T08:52:30.948628+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T08:52:30.948628+0000","last_undegraded":"2024-03-09T08:52:30.948628+0000","last_fullsized":"2024-03-09T08:52:30.948628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:26:39.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577331+0000","last_change":"2024-03-09T08:52:29.578549+0000","last_active":"2024-03-09T08:52:30.577331+0000","last_peered":"2024-03-09T08:52:30.577331+0000","last_clean":"2024-03-09T08:52:30.577331+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T08:52:30.577331+0000","last_undegraded":"2024-03-09T08:52:30.577331+0000","last_fullsized":"2024-03-09T08:52:30.577331+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:42.775199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575758+0000","last_change":"2024-03-09T08:52:18.240975+0000","last_active":"2024-03-09T08:52:30.575758+0000","last_peered":"2024-03-09T08:52:30.575758+0000","last_clean":"2024-03-09T08:52:30.575758+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T08:52:30.575758+0000","last_undegraded":"2024-03-09T08:52:30.575758+0000","last_fullsized":"2024-03-09T08:52:30.575758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:30:09.363339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.577959+0000","last_change":"2024-03-09T08:52:29.584159+0000","last_active":"2024-03-09T08:52:30.577959+0000","last_peered":"2024-03-09T08:52:30.577959+0000","last_clean":"2024-03-09T08:52:30.577959+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T08:52:30.577959+0000","last_undegraded":"2024-03-09T08:52:30.577959+0000","last_fullsized":"2024-03-09T08:52:30.577959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:01:41.661806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943943+0000","last_change":"2024-03-09T08:52:18.249901+0000","last_active":"2024-03-09T08:52:30.943943+0000","last_peered":"2024-03-09T08:52:30.943943+0000","last_clean":"2024-03-09T08:52:30.943943+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T08:52:30.943943+0000","last_undegraded":"2024-03-09T08:52:30.943943+0000","last_fullsized":"2024-03-09T08:52:30.943943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:09.947401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601289+0000","last_change":"2024-03-09T08:52:29.595184+0000","last_active":"2024-03-09T08:52:30.601289+0000","last_peered":"2024-03-09T08:52:30.601289+0000","last_clean":"2024-03-09T08:52:30.601289+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T08:52:30.601289+0000","last_undegraded":"2024-03-09T08:52:30.601289+0000","last_fullsized":"2024-03-09T08:52:30.601289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:34:27.662636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.945364+0000","last_change":"2024-03-09T08:52:18.233561+0000","last_active":"2024-03-09T08:52:30.945364+0000","last_peered":"2024-03-09T08:52:30.945364+0000","last_clean":"2024-03-09T08:52:30.945364+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T08:52:30.945364+0000","last_undegraded":"2024-03-09T08:52:30.945364+0000","last_fullsized":"2024-03-09T08:52:30.945364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:15:54.027155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947171+0000","last_change":"2024-03-09T08:52:29.595771+0000","last_active":"2024-03-09T08:52:30.947171+0000","last_peered":"2024-03-09T08:52:30.947171+0000","last_clean":"2024-03-09T08:52:30.947171+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T08:52:30.947171+0000","last_undegraded":"2024-03-09T08:52:30.947171+0000","last_fullsized":"2024-03-09T08:52:30.947171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:23:48.898071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943880+0000","last_change":"2024-03-09T08:52:18.242311+0000","last_active":"2024-03-09T08:52:30.943880+0000","last_peered":"2024-03-09T08:52:30.943880+0000","last_clean":"2024-03-09T08:52:30.943880+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T08:52:30.943880+0000","last_undegraded":"2024-03-09T08:52:30.943880+0000","last_fullsized":"2024-03-09T08:52:30.943880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:48:15.559438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.575834+0000","last_change":"2024-03-09T08:52:29.576792+0000","last_active":"2024-03-09T08:52:30.575834+0000","last_peered":"2024-03-09T08:52:30.575834+0000","last_clean":"2024-03-09T08:52:30.575834+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T08:52:30.575834+0000","last_undegraded":"2024-03-09T08:52:30.575834+0000","last_fullsized":"2024-03-09T08:52:30.575834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:21.364814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.947241+0000","last_change":"2024-03-09T08:52:18.248341+0000","last_active":"2024-03-09T08:52:30.947241+0000","last_peered":"2024-03-09T08:52:30.947241+0000","last_clean":"2024-03-09T08:52:30.947241+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T08:52:30.947241+0000","last_undegraded":"2024-03-09T08:52:30.947241+0000","last_fullsized":"2024-03-09T08:52:30.947241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:18:02.716066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.942730+0000","last_change":"2024-03-09T08:52:29.606330+0000","last_active":"2024-03-09T08:52:30.942730+0000","last_peered":"2024-03-09T08:52:30.942730+0000","last_clean":"2024-03-09T08:52:30.942730+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T08:52:30.942730+0000","last_undegraded":"2024-03-09T08:52:30.942730+0000","last_fullsized":"2024-03-09T08:52:30.942730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:44:50.347124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.597559+0000","last_change":"2024-03-09T08:52:18.222968+0000","last_active":"2024-03-09T08:52:30.597559+0000","last_peered":"2024-03-09T08:52:30.597559+0000","last_clean":"2024-03-09T08:52:30.597559+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T08:52:30.597559+0000","last_undegraded":"2024-03-09T08:52:30.597559+0000","last_fullsized":"2024-03-09T08:52:30.597559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:52:29.523963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943337+0000","last_change":"2024-03-09T08:52:29.591813+0000","last_active":"2024-03-09T08:52:30.943337+0000","last_peered":"2024-03-09T08:52:30.943337+0000","last_clean":"2024-03-09T08:52:30.943337+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T08:52:30.943337+0000","last_undegraded":"2024-03-09T08:52:30.943337+0000","last_fullsized":"2024-03-09T08:52:30.943337+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:40:31.568253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.601942+0000","last_change":"2024-03-09T08:52:18.225812+0000","last_active":"2024-03-09T08:52:30.601942+0000","last_peered":"2024-03-09T08:52:30.601942+0000","last_clean":"2024-03-09T08:52:30.601942+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T08:52:30.601942+0000","last_undegraded":"2024-03-09T08:52:30.601942+0000","last_fullsized":"2024-03-09T08:52:30.601942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:09:32.044185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.943686+0000","last_change":"2024-03-09T08:52:29.595055+0000","last_active":"2024-03-09T08:52:30.943686+0000","last_peered":"2024-03-09T08:52:30.943686+0000","last_clean":"2024-03-09T08:52:30.943686+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T08:52:30.943686+0000","last_undegraded":"2024-03-09T08:52:30.943686+0000","last_fullsized":"2024-03-09T08:52:30.943686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:53.209169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-09T08:52:30.948709+0000","last_change":"2024-03-09T08:52:18.248973+0000","last_active":"2024-03-09T08:52:30.948709+0000","last_peered":"2024-03-09T08:52:30.948709+0000","last_clean":"2024-03-09T08:52:30.948709+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T08:52:30.948709+0000","last_undegraded":"2024-03-09T08:52:30.948709+0000","last_fullsized":"2024-03-09T08:52:30.948709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:09:11.372616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":856064,"data_stored":690301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":101489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":100023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":99491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":102507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":99331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":101055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":690267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T08:52:41.485 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-09T08:52:41.485 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-09T08:52:41.485 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-09T08:52:41.486 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-09T08:52:41.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.728+0000 7fb8b0e00640 1 Processor -- start 2024-03-09T08:52:41.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.728+0000 7fb8b0e00640 1 -- start start 2024-03-09T08:52:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.730+0000 7fb8b0e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 0x7fb8ac0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.730+0000 7fb8b0e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb8ac0cadb0 0x7fb8ac0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.730+0000 7fb8b0e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ac157000 con 0x7fb8ac0c5e40 2024-03-09T08:52:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.730+0000 7fb8b0e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ac0ae430 con 0x7fb8ac0cadb0 2024-03-09T08:52:41.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.730+0000 7fb8b0e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0ae6f0 con 0x7fb8ac153300 2024-03-09T08:52:41.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.731+0000 7fb8ab400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 0x7fb8ac0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:41.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.731+0000 7fb8ab400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 0x7fb8ac0b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:39462/0 (socket says 172.21.15.67:39462) 2024-03-09T08:52:41.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.731+0000 7fb8ab400640 1 -- 172.21.15.67:0/2870973473 learned_addr learned my addr 172.21.15.67:0/2870973473 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:41.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.731+0000 7fb8abe00640 1 --2- 172.21.15.67:0/2870973473 >> v2:172.21.15.134:3300/0 conn(0x7fb8ac0cadb0 0x7fb8ac0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:41.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.732+0000 7fb8ab400640 1 -- 172.21.15.67:0/2870973473 >> v2:172.21.15.134:3300/0 conn(0x7fb8ac0cadb0 msgr2=0x7fb8ac0b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:41.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.732+0000 7fb8aaa00640 1 -- 172.21.15.67:0/2870973473 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053144581 0 0) 0x7fb8ac157000 con 0x7fb8ac0c5e40 2024-03-09T08:52:41.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.732+0000 7fb8ab400640 1 --2- 172.21.15.67:0/2870973473 >> v2:172.21.15.134:3300/0 conn(0x7fb8ac0cadb0 0x7fb8ac0b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.732+0000 7fb8ab400640 1 -- 172.21.15.67:0/2870973473 >> v1:172.21.15.134:6789/0 conn(0x7fb8ac0c5e40 legacy=0x7fb8ac156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:41.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.732+0000 7fb8ab400640 1 -- 172.21.15.67:0/2870973473 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac155b10 con 0x7fb8ac153300 2024-03-09T08:52:41.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.733+0000 7fb8ab400640 1 --2- 172.21.15.67:0/2870973473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 0x7fb8ac0b0000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009870 tx=0x7fb89c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2944f395b6949758 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:41.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.734+0000 7fb8aaa00640 1 -- 172.21.15.67:0/2870973473 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c034070 con 0x7fb8ac153300 2024-03-09T08:52:41.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.734+0000 7fb8aaa00640 1 -- 172.21.15.67:0/2870973473 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c030840 con 0x7fb8ac153300 2024-03-09T08:52:41.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.734+0000 7fb8aaa00640 1 -- 172.21.15.67:0/2870973473 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c030c10 con 0x7fb8ac153300 2024-03-09T08:52:41.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.735+0000 7fb8b0e00640 1 -- 172.21.15.67:0/2870973473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 msgr2=0x7fb8ac0b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:41.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.735+0000 7fb8b0e00640 1 --2- 172.21.15.67:0/2870973473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 0x7fb8ac0b0000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009870 tx=0x7fb89c02b980 comp rx=0 tx=0).stop 2024-03-09T08:52:41.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.735+0000 7fb8b0e00640 1 -- 172.21.15.67:0/2870973473 shutdown_connections 2024-03-09T08:52:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.735+0000 7fb8b0e00640 1 --2- 172.21.15.67:0/2870973473 >> v2:172.21.15.134:3300/0 conn(0x7fb8ac0cadb0 0x7fb8ac0b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.735+0000 7fb8b0e00640 1 --2- 172.21.15.67:0/2870973473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 0x7fb8ac0b0000 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.735+0000 7fb8b0e00640 1 -- 172.21.15.67:0/2870973473 >> 172.21.15.67:0/2870973473 conn(0x7fb8ac0732d0 msgr2=0x7fb8ac0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.735+0000 7fb8b0e00640 1 -- 172.21.15.67:0/2870973473 shutdown_connections 2024-03-09T08:52:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.735+0000 7fb8b0e00640 1 -- 172.21.15.67:0/2870973473 wait complete. 2024-03-09T08:52:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.737+0000 7fb8b0e00640 1 Processor -- start 2024-03-09T08:52:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.737+0000 7fb8b0e00640 1 -- start start 2024-03-09T08:52:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8b0e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8ac0c5e40 0x7fb8ac1e2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8b0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8ac0cadb0 0x7fb8ac06cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8b0e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 0x7fb8ac0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8b0e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0c59e0 con 0x7fb8ac0cadb0 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8b0e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac159e90 con 0x7fb8ac0c5e40 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8b0e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac156720 con 0x7fb8ac153300 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8ac0cadb0 0x7fb8ac06cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8ac0cadb0 0x7fb8ac06cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46586/0 (socket says 172.21.15.67:46586) 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8a3e00640 1 -- 172.21.15.67:0/1076069735 learned_addr learned my addr 172.21.15.67:0/1076069735 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8ab400640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8ac0c5e40 0x7fb8ac1e2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8abe00640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 0x7fb8ac0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8a3e00640 1 -- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 msgr2=0x7fb8ac0ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8a3e00640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 0x7fb8ac0ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8a3e00640 1 -- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8ac0c5e40 msgr2=0x7fb8ac1e2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8a3e00640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8ac0c5e40 0x7fb8ac1e2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8a3e00640 1 -- 172.21.15.67:0/1076069735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac0ab480 con 0x7fb8ac0cadb0 2024-03-09T08:52:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.739+0000 7fb8abe00640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 0x7fb8ac0ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.740+0000 7fb8a3e00640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8ac0cadb0 0x7fb8ac06cfb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb89400c460 tx=0x7fb89400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.740+0000 7fb8a3400640 1 -- 172.21.15.67:0/1076069735 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894005480 con 0x7fb8ac0cadb0 2024-03-09T08:52:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.740+0000 7fb8a3400640 1 -- 172.21.15.67:0/1076069735 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8940035f0 con 0x7fb8ac0cadb0 2024-03-09T08:52:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.740+0000 7fb8a3400640 1 -- 172.21.15.67:0/1076069735 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894003a00 con 0x7fb8ac0cadb0 2024-03-09T08:52:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.740+0000 7fb8b0e00640 1 -- 172.21.15.67:0/1076069735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ac0ab760 con 0x7fb8ac0cadb0 2024-03-09T08:52:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.740+0000 7fb8b0e00640 1 -- 172.21.15.67:0/1076069735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ac0ac910 con 0x7fb8ac0cadb0 2024-03-09T08:52:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.742+0000 7fb8b0e00640 1 -- 172.21.15.67:0/1076069735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ac04f3c0 con 0x7fb8ac0cadb0 2024-03-09T08:52:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.743+0000 7fb8a3400640 1 -- 172.21.15.67:0/1076069735 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb894003ba0 con 0x7fb8ac0cadb0 2024-03-09T08:52:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.744+0000 7fb8a3400640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb88806b670 0x7fb88806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.744+0000 7fb8a3400640 1 -- 172.21.15.67:0/1076069735 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb89408a1d0 con 0x7fb8ac0cadb0 2024-03-09T08:52:41.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.744+0000 7fb8ab400640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb88806b670 0x7fb88806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.745+0000 7fb8ab400640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb88806b670 0x7fb88806db30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb89c036000 tx=0x7fb89c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:41.752+0000 7fb8a3400640 1 -- 172.21.15.67:0/1076069735 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb8940586d0 con 0x7fb8ac0cadb0 2024-03-09T08:52:42.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.032+0000 7fb8b0e00640 1 -- 172.21.15.67:0/1076069735 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb8ac0cab90 con 0x7fb8ac0cadb0 2024-03-09T08:52:42.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.033+0000 7fb8a3400640 1 -- 172.21.15.67:0/1076069735 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb89405c770 con 0x7fb8ac0cadb0 2024-03-09T08:52:42.028 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:42.028 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-09T08:52:42.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 -- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb88806b670 msgr2=0x7fb88806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb88806b670 0x7fb88806db30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb89c036000 tx=0x7fb89c03b040 comp rx=0 tx=0).stop 2024-03-09T08:52:42.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 -- 172.21.15.67:0/1076069735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8ac0cadb0 msgr2=0x7fb8ac06cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8ac0cadb0 0x7fb8ac06cfb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb89400c460 tx=0x7fb89400c920 comp rx=0 tx=0).stop 2024-03-09T08:52:42.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 -- 172.21.15.67:0/1076069735 shutdown_connections 2024-03-09T08:52:42.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb88806b670 0x7fb88806db30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8ac153300 0x7fb8ac0ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb8ac0cadb0 0x7fb8ac06cfb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 --2- 172.21.15.67:0/1076069735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8ac0c5e40 0x7fb8ac1e2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 -- 172.21.15.67:0/1076069735 >> 172.21.15.67:0/1076069735 conn(0x7fb8ac0732d0 msgr2=0x7fb8ac157900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:42.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 -- 172.21.15.67:0/1076069735 shutdown_connections 2024-03-09T08:52:42.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.035+0000 7fb8b0e00640 1 -- 172.21.15.67:0/1076069735 wait complete. 2024-03-09T08:52:42.048 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-09T08:52:42.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:42.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.272+0000 7f8082800640 1 Processor -- start 2024-03-09T08:52:42.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.272+0000 7f8082800640 1 -- start start 2024-03-09T08:52:42.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.274+0000 7f8082800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c0710a0 0x7f807c153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.274+0000 7f8082800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f807c151360 0x7f807c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.274+0000 7f8082800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807c0c81e0 con 0x7f807c0c1f70 2024-03-09T08:52:42.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.274+0000 7f8082800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c0c32b0 con 0x7f807c151360 2024-03-09T08:52:42.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.274+0000 7f8082800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0c2fd0 con 0x7f807c0710a0 2024-03-09T08:52:42.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.274+0000 7f8081400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f807c0c1f70 0x7f807c150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53854/0 (socket says 172.21.15.67:53854) 2024-03-09T08:52:42.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.274+0000 7f8081400640 1 -- 172.21.15.67:0/709419543 learned_addr learned my addr 172.21.15.67:0/709419543 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.274+0000 7f8080a00640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c0710a0 0x7f807c153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.274+0000 7f8081e00640 1 --2- 172.21.15.67:0/709419543 >> v2:172.21.15.134:3300/0 conn(0x7f807c151360 0x7f807c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590927210 0 0) 0x7f807c0c81e0 con 0x7f807c0c1f70 2024-03-09T08:52:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8064003520 con 0x7f807c0c1f70 2024-03-09T08:52:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3429722853 0 0) 0x7f8064003520 con 0x7f807c0c1f70 2024-03-09T08:52:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c0710a0 msgr2=0x7f807c153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:42.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c0710a0 0x7f807c153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 >> v2:172.21.15.134:3300/0 conn(0x7f807c151360 msgr2=0x7f807c153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 --2- 172.21.15.67:0/709419543 >> v2:172.21.15.134:3300/0 conn(0x7f807c151360 0x7f807c153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c0c3700 con 0x7f807c0c1f70 2024-03-09T08:52:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f8070001670 con 0x7f807c0c1f70 2024-03-09T08:52:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 >> v1:172.21.15.67:6789/0 conn(0x7f807c0c1f70 legacy=0x7f807c150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8064003a20 0x7f8064003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c0710a0 0x7f8064004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 0x7f80640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0c32b0 con 0x7f8064004d40 2024-03-09T08:52:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0c2fd0 con 0x7f8064003a20 2024-03-09T08:52:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.275+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806400ba70 con 0x7f807c0710a0 2024-03-09T08:52:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.276+0000 7f8081e00640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 0x7f80640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.276+0000 7f8080a00640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c0710a0 0x7f8064004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.276+0000 7f8081400640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8064003a20 0x7f8064003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.276+0000 7f8081e00640 1 -- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c0710a0 msgr2=0x7f8064004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.276+0000 7f8081e00640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c0710a0 0x7f8064004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.276+0000 7f8081e00640 1 -- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8064003a20 msgr2=0x7f8064003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.276+0000 7f8081e00640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8064003a20 0x7f8064003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.276+0000 7f8081e00640 1 -- 172.21.15.67:0/709419543 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8064003520 con 0x7f8064004d40 2024-03-09T08:52:42.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.276+0000 7f8081400640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8064003a20 0x7f8064003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.277+0000 7f8081e00640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 0x7f80640051c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f807c1517b0 tx=0x7f8064005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f411dd13b15e94e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.277+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074039070 con 0x7f8064004d40 2024-03-09T08:52:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.277+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807402cbd0 con 0x7f8064004d40 2024-03-09T08:52:42.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.277+0000 7f807be00640 1 -- 172.21.15.67:0/709419543 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074009c10 con 0x7f8064004d40 2024-03-09T08:52:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.278+0000 7f8082800640 1 -- 172.21.15.67:0/709419543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 msgr2=0x7f80640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.278+0000 7f8082800640 1 --2- 172.21.15.67:0/709419543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 0x7f80640051c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f807c1517b0 tx=0x7f8064005830 comp rx=0 tx=0).stop 2024-03-09T08:52:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.278+0000 7f8081e00640 1 -- 172.21.15.67:0/709419543 reap_dead start 2024-03-09T08:52:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.278+0000 7f8082800640 1 -- 172.21.15.67:0/709419543 shutdown_connections 2024-03-09T08:52:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.278+0000 7f8082800640 1 -- 172.21.15.67:0/709419543 >> 172.21.15.67:0/709419543 conn(0x7f807c0c7300 msgr2=0x7f807c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.278+0000 7f8082800640 1 -- 172.21.15.67:0/709419543 shutdown_connections 2024-03-09T08:52:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.278+0000 7f8082800640 1 -- 172.21.15.67:0/709419543 wait complete. 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.281+0000 7f8082800640 1 Processor -- start 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.281+0000 7f8082800640 1 -- start start 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.282+0000 7f8082800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 0x7f807c154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.282+0000 7f8082800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f807c0ac570 0x7f807c156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.282+0000 7f8082800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c1570a0 0x7f807c157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.282+0000 7f8082800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c070d50 con 0x7f8064004d40 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.282+0000 7f8082800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c070830 con 0x7f807c0ac570 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.282+0000 7f8082800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c070ed0 con 0x7f807c1570a0 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8081400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 0x7f807c154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8081400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 0x7f807c154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46592/0 (socket says 172.21.15.67:46592) 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8081400640 1 -- 172.21.15.67:0/1822693035 learned_addr learned my addr 172.21.15.67:0/1822693035 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8081e00640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c1570a0 0x7f807c157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8080a00640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f807c0ac570 0x7f807c156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8081400640 1 -- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c1570a0 msgr2=0x7f807c157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8081400640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c1570a0 0x7f807c157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8081400640 1 -- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f807c0ac570 msgr2=0x7f807c156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8081400640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f807c0ac570 0x7f807c156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8081400640 1 -- 172.21.15.67:0/1822693035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c1eb6b0 con 0x7f8064004d40 2024-03-09T08:52:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8081e00640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c1570a0 0x7f807c157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.283+0000 7f8081400640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 0x7f807c154f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f807000d830 tx=0x7f807000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.284+0000 7f8079600640 1 -- 172.21.15.67:0/1822693035 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070019070 con 0x7f8064004d40 2024-03-09T08:52:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.284+0000 7f8079600640 1 -- 172.21.15.67:0/1822693035 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80700074e0 con 0x7f8064004d40 2024-03-09T08:52:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.284+0000 7f8082800640 1 -- 172.21.15.67:0/1822693035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c1eb990 con 0x7f8064004d40 2024-03-09T08:52:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.284+0000 7f8079600640 1 -- 172.21.15.67:0/1822693035 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070004a00 con 0x7f8064004d40 2024-03-09T08:52:42.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.284+0000 7f8082800640 1 -- 172.21.15.67:0/1822693035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c1ebf50 con 0x7f8064004d40 2024-03-09T08:52:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.285+0000 7f8082800640 1 -- 172.21.15.67:0/1822693035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c04f3c0 con 0x7f8064004d40 2024-03-09T08:52:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.287+0000 7f8079600640 1 -- 172.21.15.67:0/1822693035 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f807001c070 con 0x7f8064004d40 2024-03-09T08:52:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.288+0000 7f8079600640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f805c06b500 0x7f805c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.288+0000 7f8079600640 1 -- 172.21.15.67:0/1822693035 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f807008a110 con 0x7f8064004d40 2024-03-09T08:52:42.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.288+0000 7f8080a00640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f805c06b500 0x7f805c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.289+0000 7f8080a00640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f805c06b500 0x7f805c06d9c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f807c072cf0 tx=0x7f8068008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:42.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.292+0000 7f8079600640 1 -- 172.21.15.67:0/1822693035 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8070058c80 con 0x7f8064004d40 2024-03-09T08:52:42.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.491+0000 7f8082800640 1 -- 172.21.15.67:0/1822693035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f807c0c7ed0 con 0x7f8064004d40 2024-03-09T08:52:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.495+0000 7f8079600640 1 -- 172.21.15.67:0/1822693035 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20307 (secure 0 0 0) 0x7f8070058580 con 0x7f8064004d40 2024-03-09T08:52:42.491 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:42.491 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:30.565529+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:42.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.497+0000 7f8082800640 1 -- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f805c06b500 msgr2=0x7f805c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.497+0000 7f8082800640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f805c06b500 0x7f805c06d9c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f807c072cf0 tx=0x7f8068008040 comp rx=0 tx=0).stop 2024-03-09T08:52:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.497+0000 7f8082800640 1 -- 172.21.15.67:0/1822693035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 msgr2=0x7f807c154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.497+0000 7f8082800640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 0x7f807c154f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f807000d830 tx=0x7f807000dcf0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.498+0000 7f8082800640 1 -- 172.21.15.67:0/1822693035 shutdown_connections 2024-03-09T08:52:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.498+0000 7f8082800640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f805c06b500 0x7f805c06d9c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.498+0000 7f8082800640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f807c1570a0 0x7f807c157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.498+0000 7f8082800640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f807c0ac570 0x7f807c156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.498+0000 7f8082800640 1 --2- 172.21.15.67:0/1822693035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8064004d40 0x7f807c154f60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.498+0000 7f8082800640 1 -- 172.21.15.67:0/1822693035 >> 172.21.15.67:0/1822693035 conn(0x7f807c0c7300 msgr2=0x7f807c0c2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.498+0000 7f8082800640 1 -- 172.21.15.67:0/1822693035 shutdown_connections 2024-03-09T08:52:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.498+0000 7f8082800640 1 -- 172.21.15.67:0/1822693035 wait complete. 2024-03-09T08:52:42.513 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-09T08:52:06.749641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-03-09T08:52:16.185356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-03-09T08:52:16.756829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-03-09T08:52:27.484297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-03-09T08:52:28.060754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-09T08:52:42.514 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T08:52:42.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.739+0000 7f4f3fa00640 1 Processor -- start 2024-03-09T08:52:42.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.740+0000 7f4f3fa00640 1 -- start start 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.741+0000 7f4f3fa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f380c6770 0x7f4f380abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.741+0000 7f4f3fa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4f381563f0 0x7f4f381568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.741+0000 7f4f3fa00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f38152e70 con 0x7f4f380ac490 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.741+0000 7f4f3fa00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f381539f0 con 0x7f4f380c6770 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.741+0000 7f4f3fa00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f380aa3b0 con 0x7f4f381563f0 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.741+0000 7f4f3d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f380c6770 0x7f4f380abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.742+0000 7f4f3d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f380c6770 0x7f4f380abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46594/0 (socket says 172.21.15.67:46594) 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.742+0000 7f4f3d000640 1 -- 172.21.15.67:0/128659662 learned_addr learned my addr 172.21.15.67:0/128659662 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.741+0000 7f4f37e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4f380ac490 0x7f4f3814fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53870/0 (socket says 172.21.15.67:53870) 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.742+0000 7f4f3da00640 1 --2- 172.21.15.67:0/128659662 >> v2:172.21.15.134:3300/0 conn(0x7f4f381563f0 0x7f4f381568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.742+0000 7f4f3d000640 1 -- 172.21.15.67:0/128659662 >> v2:172.21.15.134:3300/0 conn(0x7f4f381563f0 msgr2=0x7f4f381568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.742+0000 7f4f3d000640 1 --2- 172.21.15.67:0/128659662 >> v2:172.21.15.134:3300/0 conn(0x7f4f381563f0 0x7f4f381568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.742+0000 7f4f3d000640 1 -- 172.21.15.67:0/128659662 >> v1:172.21.15.67:6789/0 conn(0x7f4f380ac490 legacy=0x7f4f3814fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.742+0000 7f4f37400640 1 -- 172.21.15.67:0/128659662 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082788472 0 0) 0x7f4f38152e70 con 0x7f4f380ac490 2024-03-09T08:52:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.742+0000 7f4f3d000640 1 -- 172.21.15.67:0/128659662 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38153150 con 0x7f4f380c6770 2024-03-09T08:52:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.743+0000 7f4f3d000640 1 --2- 172.21.15.67:0/128659662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f380c6770 0x7f4f380abf50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4f380abf00 tx=0x7f4f2802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab17a7ce0fb7a83c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.743+0000 7f4f37400640 1 -- 172.21.15.67:0/128659662 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28034070 con 0x7f4f380c6770 2024-03-09T08:52:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.743+0000 7f4f37400640 1 -- 172.21.15.67:0/128659662 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f28030860 con 0x7f4f380c6770 2024-03-09T08:52:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.743+0000 7f4f37400640 1 -- 172.21.15.67:0/128659662 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28030c30 con 0x7f4f380c6770 2024-03-09T08:52:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.744+0000 7f4f3fa00640 1 -- 172.21.15.67:0/128659662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f380c6770 msgr2=0x7f4f380abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.744+0000 7f4f3fa00640 1 --2- 172.21.15.67:0/128659662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f380c6770 0x7f4f380abf50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4f380abf00 tx=0x7f4f2802b9a0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.744+0000 7f4f3fa00640 1 -- 172.21.15.67:0/128659662 shutdown_connections 2024-03-09T08:52:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.744+0000 7f4f3fa00640 1 --2- 172.21.15.67:0/128659662 >> v2:172.21.15.134:3300/0 conn(0x7f4f381563f0 0x7f4f381568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.744+0000 7f4f3fa00640 1 --2- 172.21.15.67:0/128659662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f380c6770 0x7f4f380abf50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.744+0000 7f4f3fa00640 1 -- 172.21.15.67:0/128659662 >> 172.21.15.67:0/128659662 conn(0x7f4f3806b040 msgr2=0x7f4f3806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.744+0000 7f4f3fa00640 1 -- 172.21.15.67:0/128659662 shutdown_connections 2024-03-09T08:52:42.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.744+0000 7f4f3fa00640 1 -- 172.21.15.67:0/128659662 wait complete. 2024-03-09T08:52:42.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.746+0000 7f4f3fa00640 1 Processor -- start 2024-03-09T08:52:42.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.746+0000 7f4f3fa00640 1 -- start start 2024-03-09T08:52:42.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.747+0000 7f4f3fa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f380ac490 0x7f4f3814c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.747+0000 7f4f3fa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f380c6770 0x7f4f3814c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.747+0000 7f4f3fa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f381563f0 0x7f4f381ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.747+0000 7f4f3fa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f38156de0 con 0x7f4f381563f0 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.747+0000 7f4f3fa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f38153710 con 0x7f4f380ac490 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.747+0000 7f4f3fa00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f38150eb0 con 0x7f4f380c6770 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.747+0000 7f4f3da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f381563f0 0x7f4f381ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.747+0000 7f4f3da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f381563f0 0x7f4f381ec9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46604/0 (socket says 172.21.15.67:46604) 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.747+0000 7f4f3da00640 1 -- 172.21.15.67:0/3823679605 learned_addr learned my addr 172.21.15.67:0/3823679605 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f3d000640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f380ac490 0x7f4f3814c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f37e00640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f380c6770 0x7f4f3814c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f3da00640 1 -- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f380c6770 msgr2=0x7f4f3814c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f3da00640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f380c6770 0x7f4f3814c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f3da00640 1 -- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f380ac490 msgr2=0x7f4f3814c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f3da00640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f380ac490 0x7f4f3814c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f3da00640 1 -- 172.21.15.67:0/3823679605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38152a30 con 0x7f4f381563f0 2024-03-09T08:52:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f3d000640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f380ac490 0x7f4f3814c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f3da00640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f381563f0 0x7f4f381ec9c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d600 tx=0x7f4f3000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f34c00640 1 -- 172.21.15.67:0/3823679605 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f300125f0 con 0x7f4f381563f0 2024-03-09T08:52:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f34c00640 1 -- 172.21.15.67:0/3823679605 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f30019070 con 0x7f4f381563f0 2024-03-09T08:52:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f34c00640 1 -- 172.21.15.67:0/3823679605 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f300129f0 con 0x7f4f381563f0 2024-03-09T08:52:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.748+0000 7f4f3fa00640 1 -- 172.21.15.67:0/3823679605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f380bc990 con 0x7f4f381563f0 2024-03-09T08:52:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.749+0000 7f4f3fa00640 1 -- 172.21.15.67:0/3823679605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f38150b40 con 0x7f4f381563f0 2024-03-09T08:52:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.750+0000 7f4f3fa00640 1 -- 172.21.15.67:0/3823679605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f38075a00 con 0x7f4f381563f0 2024-03-09T08:52:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.751+0000 7f4f34c00640 1 -- 172.21.15.67:0/3823679605 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4f30012b90 con 0x7f4f381563f0 2024-03-09T08:52:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.751+0000 7f4f34c00640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f1406b500 0x7f4f1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.752+0000 7f4f34c00640 1 -- 172.21.15.67:0/3823679605 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f4f3008a980 con 0x7f4f381563f0 2024-03-09T08:52:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.752+0000 7f4f3d000640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f1406b500 0x7f4f1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.753+0000 7f4f3d000640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f1406b500 0x7f4f1406d9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f28036000 tx=0x7f4f2803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:42.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.756+0000 7f4f34c00640 1 -- 172.21.15.67:0/3823679605 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4f30059440 con 0x7f4f381563f0 2024-03-09T08:52:42.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.952+0000 7f4f3fa00640 1 -- 172.21.15.67:0/3823679605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f380c1870 con 0x7f4f381563f0 2024-03-09T08:52:42.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.953+0000 7f4f34c00640 1 -- 172.21.15.67:0/3823679605 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f30058d40 con 0x7f4f381563f0 2024-03-09T08:52:42.949 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-09T08:52:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.956+0000 7f4f3fa00640 1 -- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f1406b500 msgr2=0x7f4f1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.956+0000 7f4f3fa00640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f1406b500 0x7f4f1406d9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4f28036000 tx=0x7f4f2803b040 comp rx=0 tx=0).stop 2024-03-09T08:52:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.956+0000 7f4f3fa00640 1 -- 172.21.15.67:0/3823679605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f381563f0 msgr2=0x7f4f381ec9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.956+0000 7f4f3fa00640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f381563f0 0x7f4f381ec9c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d600 tx=0x7f4f3000dac0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.956+0000 7f4f3fa00640 1 -- 172.21.15.67:0/3823679605 shutdown_connections 2024-03-09T08:52:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.956+0000 7f4f3fa00640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f1406b500 0x7f4f1406d9c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.957+0000 7f4f3fa00640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f381563f0 0x7f4f381ec9c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.957+0000 7f4f3fa00640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f380c6770 0x7f4f3814c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.957+0000 7f4f3fa00640 1 --2- 172.21.15.67:0/3823679605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f380ac490 0x7f4f3814c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.957+0000 7f4f3fa00640 1 -- 172.21.15.67:0/3823679605 >> 172.21.15.67:0/3823679605 conn(0x7f4f3806b040 msgr2=0x7f4f3806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.957+0000 7f4f3fa00640 1 -- 172.21.15.67:0/3823679605 shutdown_connections 2024-03-09T08:52:42.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:42.957+0000 7f4f3fa00640 1 -- 172.21.15.67:0/3823679605 wait complete. 2024-03-09T08:52:42.972 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-03-09T08:52:43.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.193+0000 7f90ec600640 1 Processor -- start 2024-03-09T08:52:43.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.193+0000 7f90ec600640 1 -- start start 2024-03-09T08:52:43.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.194+0000 7f90ec600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f90e40c84f0 0x7f90e4073a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.194+0000 7f90ec600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e40c2870 con 0x7f90e40c7500 2024-03-09T08:52:43.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.194+0000 7f90ec600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e40c2590 con 0x7f90e40a6af0 2024-03-09T08:52:43.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.194+0000 7f90ec600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e41568c0 con 0x7f90e40c84f0 2024-03-09T08:52:43.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.195+0000 7f90e9200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f90e40c7500 0x7f90e40c3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53880/0 (socket says 172.21.15.67:53880) 2024-03-09T08:52:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.195+0000 7f90e9200640 1 -- 172.21.15.67:0/1467234328 learned_addr learned my addr 172.21.15.67:0/1467234328 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.195+0000 7f90e9c00640 1 --2- 172.21.15.67:0/1467234328 >> v2:172.21.15.67:3300/0 conn(0x7f90e40c84f0 0x7f90e4073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.195+0000 7f90e3e00640 1 -- 172.21.15.67:0/1467234328 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801515264 0 0) 0x7f90e40c2870 con 0x7f90e40c7500 2024-03-09T08:52:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.195+0000 7f90e3e00640 1 -- 172.21.15.67:0/1467234328 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90cc003520 con 0x7f90e40c7500 2024-03-09T08:52:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.195+0000 7f90e9c00640 1 -- 172.21.15.67:0/1467234328 >> v1:172.21.15.134:6789/0 conn(0x7f90e40a6af0 legacy=0x7f90e40a6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.195+0000 7f90e9c00640 1 -- 172.21.15.67:0/1467234328 >> v1:172.21.15.67:6789/0 conn(0x7f90e40c7500 legacy=0x7f90e40c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.195+0000 7f90e9c00640 1 -- 172.21.15.67:0/1467234328 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e406ed70 con 0x7f90e40c84f0 2024-03-09T08:52:43.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.196+0000 7f90e9c00640 1 --2- 172.21.15.67:0/1467234328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90e40c84f0 0x7f90e4073a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f90d80097a0 tx=0x7f90d802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ae63c7df7b9e90 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:43.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.197+0000 7f90e3e00640 1 -- 172.21.15.67:0/1467234328 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8034070 con 0x7f90e40c84f0 2024-03-09T08:52:43.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.197+0000 7f90e3e00640 1 -- 172.21.15.67:0/1467234328 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d8030610 con 0x7f90e40c84f0 2024-03-09T08:52:43.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.197+0000 7f90e3e00640 1 -- 172.21.15.67:0/1467234328 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80309e0 con 0x7f90e40c84f0 2024-03-09T08:52:43.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.197+0000 7f90ec600640 1 -- 172.21.15.67:0/1467234328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90e40c84f0 msgr2=0x7f90e4073a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.197+0000 7f90ec600640 1 --2- 172.21.15.67:0/1467234328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90e40c84f0 0x7f90e4073a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f90d80097a0 tx=0x7f90d802b960 comp rx=0 tx=0).stop 2024-03-09T08:52:43.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.197+0000 7f90ec600640 1 -- 172.21.15.67:0/1467234328 shutdown_connections 2024-03-09T08:52:43.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.197+0000 7f90ec600640 1 --2- 172.21.15.67:0/1467234328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90e40c84f0 0x7f90e4073a30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.197+0000 7f90ec600640 1 -- 172.21.15.67:0/1467234328 >> 172.21.15.67:0/1467234328 conn(0x7f90e40a9fd0 msgr2=0x7f90e40c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:43.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.198+0000 7f90ec600640 1 -- 172.21.15.67:0/1467234328 shutdown_connections 2024-03-09T08:52:43.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.198+0000 7f90ec600640 1 -- 172.21.15.67:0/1467234328 wait complete. 2024-03-09T08:52:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.199+0000 7f90ec600640 1 Processor -- start 2024-03-09T08:52:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.199+0000 7f90ec600640 1 -- start start 2024-03-09T08:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.200+0000 7f90ec600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e40a6af0 0x7f90e41e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.200+0000 7f90ec600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e40c7500 0x7f90e41df9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.200+0000 7f90ec600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90e40c84f0 0x7f90e4152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.200+0000 7f90ec600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40c17d0 con 0x7f90e40c84f0 2024-03-09T08:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.200+0000 7f90ec600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e4156d50 con 0x7f90e40a6af0 2024-03-09T08:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.200+0000 7f90ec600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40c88d0 con 0x7f90e40c7500 2024-03-09T08:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.200+0000 7f90ea600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90e40c84f0 0x7f90e4152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.201+0000 7f90ea600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90e40c84f0 0x7f90e4152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46628/0 (socket says 172.21.15.67:46628) 2024-03-09T08:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.201+0000 7f90ea600640 1 -- 172.21.15.67:0/3410203695 learned_addr learned my addr 172.21.15.67:0/3410203695 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:43.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.201+0000 7f90e9c00640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e40a6af0 0x7f90e41e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.201+0000 7f90e9200640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e40c7500 0x7f90e41df9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.201+0000 7f90ea600640 1 -- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e40c7500 msgr2=0x7f90e41df9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.201+0000 7f90ea600640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e40c7500 0x7f90e41df9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.201+0000 7f90ea600640 1 -- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e40a6af0 msgr2=0x7f90e41e2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.201+0000 7f90ea600640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e40a6af0 0x7f90e41e2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.201+0000 7f90ea600640 1 -- 172.21.15.67:0/3410203695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e41530d0 con 0x7f90e40c84f0 2024-03-09T08:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.201+0000 7f90e9c00640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e40a6af0 0x7f90e41e2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.201+0000 7f90ea600640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90e40c84f0 0x7f90e4152370 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c6e0 tx=0x7f90dc00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.202+0000 7f90e1600640 1 -- 172.21.15.67:0/3410203695 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc003b70 con 0x7f90e40c84f0 2024-03-09T08:52:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.202+0000 7f90e1600640 1 -- 172.21.15.67:0/3410203695 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc003d30 con 0x7f90e40c84f0 2024-03-09T08:52:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.202+0000 7f90e1600640 1 -- 172.21.15.67:0/3410203695 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc005310 con 0x7f90e40c84f0 2024-03-09T08:52:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.202+0000 7f90ec600640 1 -- 172.21.15.67:0/3410203695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e41533b0 con 0x7f90e40c84f0 2024-03-09T08:52:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.202+0000 7f90ec600640 1 -- 172.21.15.67:0/3410203695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e40ac800 con 0x7f90e40c84f0 2024-03-09T08:52:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.203+0000 7f90ec600640 1 -- 172.21.15.67:0/3410203695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e4075a00 con 0x7f90e40c84f0 2024-03-09T08:52:43.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.204+0000 7f90e1600640 1 -- 172.21.15.67:0/3410203695 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f90dc0054b0 con 0x7f90e40c84f0 2024-03-09T08:52:43.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.205+0000 7f90e1600640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90c406b4d0 0x7f90c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.205+0000 7f90e1600640 1 -- 172.21.15.67:0/3410203695 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f90dc08a3d0 con 0x7f90e40c84f0 2024-03-09T08:52:43.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.205+0000 7f90e9c00640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90c406b4d0 0x7f90c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:43.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.206+0000 7f90e9c00640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90c406b4d0 0x7f90c406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90d8036000 tx=0x7f90d803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:43.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.214+0000 7f90e1600640 1 -- 172.21.15.67:0/3410203695 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f90dc058f10 con 0x7f90e40c84f0 2024-03-09T08:52:43.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.415+0000 7f90ec600640 1 -- 172.21.15.67:0/3410203695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90e40c2870 con 0x7f90e40c84f0 2024-03-09T08:52:43.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.416+0000 7f90e1600640 1 -- 172.21.15.67:0/3410203695 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7f90dc058810 con 0x7f90e40c84f0 2024-03-09T08:52:43.412 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T08:52:43.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.418+0000 7f90ec600640 1 -- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90c406b4d0 msgr2=0x7f90c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.418+0000 7f90ec600640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90c406b4d0 0x7f90c406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f90d8036000 tx=0x7f90d803b040 comp rx=0 tx=0).stop 2024-03-09T08:52:43.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.418+0000 7f90ec600640 1 -- 172.21.15.67:0/3410203695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90e40c84f0 msgr2=0x7f90e4152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.418+0000 7f90ec600640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90e40c84f0 0x7f90e4152370 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c6e0 tx=0x7f90dc00cba0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.418+0000 7f90ec600640 1 -- 172.21.15.67:0/3410203695 shutdown_connections 2024-03-09T08:52:43.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.418+0000 7f90ec600640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90c406b4d0 0x7f90c406d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.418+0000 7f90ec600640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90e40c84f0 0x7f90e4152370 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.418+0000 7f90ec600640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90e40c7500 0x7f90e41df9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.418+0000 7f90ec600640 1 --2- 172.21.15.67:0/3410203695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90e40a6af0 0x7f90e41e2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.418+0000 7f90ec600640 1 -- 172.21.15.67:0/3410203695 >> 172.21.15.67:0/3410203695 conn(0x7f90e40a9fd0 msgr2=0x7f90e4155420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:43.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.419+0000 7f90ec600640 1 -- 172.21.15.67:0/3410203695 shutdown_connections 2024-03-09T08:52:43.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.419+0000 7f90ec600640 1 -- 172.21.15.67:0/3410203695 wait complete. 2024-03-09T08:52:43.433 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-03-09T08:52:43.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.647+0000 7ff001600640 1 Processor -- start 2024-03-09T08:52:43.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.648+0000 7ff001600640 1 -- start start 2024-03-09T08:52:43.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.649+0000 7ff001600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7feffc0c5f10 0x7feffc0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.649+0000 7ff001600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feffc06b040 0x7feffc1567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.649+0000 7ff001600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feffc0c9490 0x7feffc0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.649+0000 7ff001600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feffc156d10 con 0x7feffc0c9490 2024-03-09T08:52:43.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.649+0000 7ff001600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feffc0c9a30 con 0x7feffc0c5f10 2024-03-09T08:52:43.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.649+0000 7ff001600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc0c9d10 con 0x7feffc06b040 2024-03-09T08:52:43.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.650+0000 7ff000c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feffc0c9490 0x7feffc0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:43.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.650+0000 7ff000c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feffc0c9490 0x7feffc0cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46640/0 (socket says 172.21.15.67:46640) 2024-03-09T08:52:43.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.650+0000 7ff000c00640 1 -- 172.21.15.67:0/3280118955 learned_addr learned my addr 172.21.15.67:0/3280118955 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:43.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.650+0000 7feffb400640 1 --2- 172.21.15.67:0/3280118955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feffc06b040 0x7feffc1567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:43.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.650+0000 7feffbe00640 1 --2- 172.21.15.67:0/3280118955 >> v2:172.21.15.134:3300/0 conn(0x7feffc0c5f10 0x7feffc0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:43.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.650+0000 7ff000c00640 1 -- 172.21.15.67:0/3280118955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feffc06b040 msgr2=0x7feffc1567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:43.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.650+0000 7ff000c00640 1 --2- 172.21.15.67:0/3280118955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feffc06b040 0x7feffc1567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.650+0000 7ff000c00640 1 -- 172.21.15.67:0/3280118955 >> v2:172.21.15.134:3300/0 conn(0x7feffc0c5f10 msgr2=0x7feffc0c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:43.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.650+0000 7ff000c00640 1 --2- 172.21.15.67:0/3280118955 >> v2:172.21.15.134:3300/0 conn(0x7feffc0c5f10 0x7feffc0c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.650+0000 7ff000c00640 1 -- 172.21.15.67:0/3280118955 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc0ca340 con 0x7feffc0c9490 2024-03-09T08:52:43.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.652+0000 7ff000c00640 1 --2- 172.21.15.67:0/3280118955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0c9490 0x7feffc0cab40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feffc08d2e0 tx=0x7feff402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39c1f114f0abb94 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:43.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.652+0000 7feffaa00640 1 -- 172.21.15.67:0/3280118955 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff402c9e0 con 0x7feffc0c9490 2024-03-09T08:52:43.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.652+0000 7feffaa00640 1 -- 172.21.15.67:0/3280118955 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff402cb80 con 0x7feffc0c9490 2024-03-09T08:52:43.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.652+0000 7feffaa00640 1 -- 172.21.15.67:0/3280118955 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40343f0 con 0x7feffc0c9490 2024-03-09T08:52:43.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.653+0000 7ff001600640 1 -- 172.21.15.67:0/3280118955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0c9490 msgr2=0x7feffc0cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.653+0000 7ff001600640 1 --2- 172.21.15.67:0/3280118955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0c9490 0x7feffc0cab40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feffc08d2e0 tx=0x7feff402be40 comp rx=0 tx=0).stop 2024-03-09T08:52:43.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.653+0000 7ff001600640 1 -- 172.21.15.67:0/3280118955 shutdown_connections 2024-03-09T08:52:43.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.653+0000 7ff001600640 1 --2- 172.21.15.67:0/3280118955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0c9490 0x7feffc0cab40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.653+0000 7ff001600640 1 --2- 172.21.15.67:0/3280118955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feffc06b040 0x7feffc1567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.653+0000 7ff001600640 1 --2- 172.21.15.67:0/3280118955 >> v2:172.21.15.134:3300/0 conn(0x7feffc0c5f10 0x7feffc0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.653+0000 7ff001600640 1 -- 172.21.15.67:0/3280118955 >> 172.21.15.67:0/3280118955 conn(0x7feffc0cb590 msgr2=0x7feffc0cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:43.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.653+0000 7ff001600640 1 -- 172.21.15.67:0/3280118955 shutdown_connections 2024-03-09T08:52:43.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.653+0000 7ff001600640 1 -- 172.21.15.67:0/3280118955 wait complete. 2024-03-09T08:52:43.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.655+0000 7ff001600640 1 Processor -- start 2024-03-09T08:52:43.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.655+0000 7ff001600640 1 -- start start 2024-03-09T08:52:43.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff001600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feffc06b040 0x7feffc1ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff001600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feffc0c5f10 0x7feffc1ef920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff001600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0c9490 0x7feffc1edd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff001600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc153620 con 0x7feffc0c9490 2024-03-09T08:52:43.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff001600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc0c9ff0 con 0x7feffc0c5f10 2024-03-09T08:52:43.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff001600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc1537a0 con 0x7feffc06b040 2024-03-09T08:52:43.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff000c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0c9490 0x7feffc1edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:43.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff000c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0c9490 0x7feffc1edd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46652/0 (socket says 172.21.15.67:46652) 2024-03-09T08:52:43.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff000c00640 1 -- 172.21.15.67:0/2785670187 learned_addr learned my addr 172.21.15.67:0/2785670187 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:43.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7feffbe00640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feffc06b040 0x7feffc1ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:43.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff000c00640 1 -- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feffc06b040 msgr2=0x7feffc1ef3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff000c00640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feffc06b040 0x7feffc1ef3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff000c00640 1 -- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feffc0c5f10 msgr2=0x7feffc1ef920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff000c00640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feffc0c5f10 0x7feffc1ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.656+0000 7ff000c00640 1 -- 172.21.15.67:0/2785670187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc1eff00 con 0x7feffc0c9490 2024-03-09T08:52:43.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.657+0000 7ff000c00640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0c9490 0x7feffc1edd90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feff4036040 tx=0x7feff4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:43.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.657+0000 7feff3e00640 1 -- 172.21.15.67:0/2785670187 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4043020 con 0x7feffc0c9490 2024-03-09T08:52:43.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.657+0000 7feff3e00640 1 -- 172.21.15.67:0/2785670187 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff403e450 con 0x7feffc0c9490 2024-03-09T08:52:43.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.657+0000 7feff3e00640 1 -- 172.21.15.67:0/2785670187 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4009510 con 0x7feffc0c9490 2024-03-09T08:52:43.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.657+0000 7ff001600640 1 -- 172.21.15.67:0/2785670187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feffc1ed2b0 con 0x7feffc0c9490 2024-03-09T08:52:43.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.657+0000 7ff001600640 1 -- 172.21.15.67:0/2785670187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc14ce30 con 0x7feffc0c9490 2024-03-09T08:52:43.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.659+0000 7ff001600640 1 -- 172.21.15.67:0/2785670187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feffc1ed0a0 con 0x7feffc0c9490 2024-03-09T08:52:43.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.660+0000 7feff3e00640 1 -- 172.21.15.67:0/2785670187 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7feff4046020 con 0x7feffc0c9490 2024-03-09T08:52:43.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.660+0000 7feff3e00640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd806b500 0x7fefd806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:43.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.661+0000 7feff3e00640 1 -- 172.21.15.67:0/2785670187 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7feff40acd80 con 0x7feffc0c9490 2024-03-09T08:52:43.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.661+0000 7feffbe00640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd806b500 0x7fefd806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:43.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.662+0000 7feffbe00640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd806b500 0x7fefd806d9c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fefe4006fd0 tx=0x7fefe4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:43.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.665+0000 7feff3e00640 1 -- 172.21.15.67:0/2785670187 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feff404d030 con 0x7feffc0c9490 2024-03-09T08:52:43.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.895+0000 7ff001600640 1 -- 172.21.15.67:0/2785670187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7feffc04f3c0 con 0x7feffc0c9490 2024-03-09T08:52:43.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.897+0000 7feff3e00640 1 -- 172.21.15.67:0/2785670187 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7feff407b8d0 con 0x7feffc0c9490 2024-03-09T08:52:43.893 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T08:52:43.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 -- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd806b500 msgr2=0x7fefd806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd806b500 0x7fefd806d9c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fefe4006fd0 tx=0x7fefe4008040 comp rx=0 tx=0).stop 2024-03-09T08:52:43.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 -- 172.21.15.67:0/2785670187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0c9490 msgr2=0x7feffc1edd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:43.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0c9490 0x7feffc1edd90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feff4036040 tx=0x7feff4004a40 comp rx=0 tx=0).stop 2024-03-09T08:52:43.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 -- 172.21.15.67:0/2785670187 shutdown_connections 2024-03-09T08:52:43.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd806b500 0x7fefd806d9c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feffc0c9490 0x7feffc1edd90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feffc0c5f10 0x7feffc1ef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 --2- 172.21.15.67:0/2785670187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feffc06b040 0x7feffc1ef3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:43.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 -- 172.21.15.67:0/2785670187 >> 172.21.15.67:0/2785670187 conn(0x7feffc0cb590 msgr2=0x7feffc0ac360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:43.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 -- 172.21.15.67:0/2785670187 shutdown_connections 2024-03-09T08:52:43.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:43.900+0000 7ff001600640 1 -- 172.21.15.67:0/2785670187 wait complete. 2024-03-09T08:52:43.923 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-03-09T08:52:44.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.147+0000 7feb3fe00640 1 Processor -- start 2024-03-09T08:52:44.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.147+0000 7feb3fe00640 1 -- start start 2024-03-09T08:52:44.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.149+0000 7feb3fe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7feb400c8190 0x7feb400c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:44.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.149+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb40152f20 0x7feb40153380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:44.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.149+0000 7feb3fe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb40155d70 con 0x7feb400ca2c0 2024-03-09T08:52:44.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.149+0000 7feb3fe00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb40156c40 con 0x7feb400c8190 2024-03-09T08:52:44.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.149+0000 7feb3fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb40156030 con 0x7feb40152f20 2024-03-09T08:52:44.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.149+0000 7feb3e000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb40152f20 0x7feb40153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:44.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.149+0000 7feb3f400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7feb400ca2c0 0x7feb400ca750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:58632/0 (socket says 172.21.15.67:58632) 2024-03-09T08:52:44.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.149+0000 7feb3e000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb40152f20 0x7feb40153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:39548/0 (socket says 172.21.15.67:39548) 2024-03-09T08:52:44.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.150+0000 7feb3f400640 1 -- 172.21.15.67:0/3049154821 learned_addr learned my addr 172.21.15.67:0/3049154821 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:44.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.150+0000 7feb3ea00640 1 --2- 172.21.15.67:0/3049154821 >> v2:172.21.15.134:3300/0 conn(0x7feb400c8190 0x7feb400c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:44.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.150+0000 7feb3ea00640 1 -- 172.21.15.67:0/3049154821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb40152f20 msgr2=0x7feb40153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:44.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.150+0000 7feb3ea00640 1 --2- 172.21.15.67:0/3049154821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb40152f20 0x7feb40153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.150+0000 7feb3ea00640 1 -- 172.21.15.67:0/3049154821 >> v1:172.21.15.134:6789/0 conn(0x7feb400ca2c0 legacy=0x7feb400ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.150+0000 7feb3ea00640 1 -- 172.21.15.67:0/3049154821 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb40156310 con 0x7feb400c8190 2024-03-09T08:52:44.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.152+0000 7feb3ea00640 1 --2- 172.21.15.67:0/3049154821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb400c8190 0x7feb400c9640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feb300097a0 tx=0x7feb3002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27559348a91b581a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.152+0000 7feb3d600640 1 -- 172.21.15.67:0/3049154821 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb300315a0 con 0x7feb400c8190 2024-03-09T08:52:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.152+0000 7feb3d600640 1 -- 172.21.15.67:0/3049154821 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb30037070 con 0x7feb400c8190 2024-03-09T08:52:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.152+0000 7feb3d600640 1 -- 172.21.15.67:0/3049154821 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30031960 con 0x7feb400c8190 2024-03-09T08:52:44.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.153+0000 7feb3fe00640 1 -- 172.21.15.67:0/3049154821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb400c8190 msgr2=0x7feb400c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.153+0000 7feb3fe00640 1 --2- 172.21.15.67:0/3049154821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb400c8190 0x7feb400c9640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feb300097a0 tx=0x7feb3002b840 comp rx=0 tx=0).stop 2024-03-09T08:52:44.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.153+0000 7feb3fe00640 1 -- 172.21.15.67:0/3049154821 shutdown_connections 2024-03-09T08:52:44.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.153+0000 7feb3fe00640 1 --2- 172.21.15.67:0/3049154821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb40152f20 0x7feb40153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.153+0000 7feb3fe00640 1 --2- 172.21.15.67:0/3049154821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb400c8190 0x7feb400c9640 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.153+0000 7feb3fe00640 1 -- 172.21.15.67:0/3049154821 >> 172.21.15.67:0/3049154821 conn(0x7feb400ad060 msgr2=0x7feb400ad440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:44.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.153+0000 7feb3fe00640 1 -- 172.21.15.67:0/3049154821 shutdown_connections 2024-03-09T08:52:44.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.153+0000 7feb3fe00640 1 -- 172.21.15.67:0/3049154821 wait complete. 2024-03-09T08:52:44.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.155+0000 7feb3fe00640 1 Processor -- start 2024-03-09T08:52:44.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.155+0000 7feb3fe00640 1 -- start start 2024-03-09T08:52:44.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.156+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb400c8190 0x7feb401522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:44.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.156+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb400ca2c0 0x7feb401eb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:44.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.156+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb40152f20 0x7feb401ebc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:44.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.156+0000 7feb3fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400c9b80 con 0x7feb400c8190 2024-03-09T08:52:44.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.156+0000 7feb3fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb401568d0 con 0x7feb40152f20 2024-03-09T08:52:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.156+0000 7feb3fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb400cae60 con 0x7feb400ca2c0 2024-03-09T08:52:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.157+0000 7feb3ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb400c8190 0x7feb401522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.157+0000 7feb3ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb400c8190 0x7feb401522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46666/0 (socket says 172.21.15.67:46666) 2024-03-09T08:52:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.157+0000 7feb3ea00640 1 -- 172.21.15.67:0/3077415427 learned_addr learned my addr 172.21.15.67:0/3077415427 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.157+0000 7feb3e000640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb400ca2c0 0x7feb401eb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.157+0000 7feb3f400640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb40152f20 0x7feb401ebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.157+0000 7feb3ea00640 1 -- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb400ca2c0 msgr2=0x7feb401eb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.157+0000 7feb3ea00640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb400ca2c0 0x7feb401eb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.157+0000 7feb3ea00640 1 -- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb40152f20 msgr2=0x7feb401ebc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.157+0000 7feb3ea00640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb40152f20 0x7feb401ebc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.157+0000 7feb3ea00640 1 -- 172.21.15.67:0/3077415427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4014cc20 con 0x7feb400c8190 2024-03-09T08:52:44.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.158+0000 7feb3ea00640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb400c8190 0x7feb401522b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feb3002b880 tx=0x7feb300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.158+0000 7feb36a00640 1 -- 172.21.15.67:0/3077415427 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30032630 con 0x7feb400c8190 2024-03-09T08:52:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.158+0000 7feb36a00640 1 -- 172.21.15.67:0/3077415427 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb300327d0 con 0x7feb400c8190 2024-03-09T08:52:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.158+0000 7feb3fe00640 1 -- 172.21.15.67:0/3077415427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb401eec80 con 0x7feb400c8190 2024-03-09T08:52:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.158+0000 7feb36a00640 1 -- 172.21.15.67:0/3077415427 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30032bc0 con 0x7feb400c8190 2024-03-09T08:52:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.158+0000 7feb3fe00640 1 -- 172.21.15.67:0/3077415427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb401ef240 con 0x7feb400c8190 2024-03-09T08:52:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.160+0000 7feb3fe00640 1 -- 172.21.15.67:0/3077415427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4004f3c0 con 0x7feb400c8190 2024-03-09T08:52:44.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.160+0000 7feb36a00640 1 -- 172.21.15.67:0/3077415427 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7feb3004d3f0 con 0x7feb400c8190 2024-03-09T08:52:44.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.162+0000 7feb36a00640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb2006b4d0 0x7feb2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:44.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.162+0000 7feb36a00640 1 -- 172.21.15.67:0/3077415427 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7feb3002f030 con 0x7feb400c8190 2024-03-09T08:52:44.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.162+0000 7feb3e000640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb2006b4d0 0x7feb2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.163+0000 7feb3e000640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb2006b4d0 0x7feb2006d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb400c55b0 tx=0x7feb2c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:44.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.165+0000 7feb36a00640 1 -- 172.21.15.67:0/3077415427 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feb30040070 con 0x7feb400c8190 2024-03-09T08:52:44.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.364+0000 7feb3fe00640 1 -- 172.21.15.67:0/3077415427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb4014df60 con 0x7feb400c8190 2024-03-09T08:52:44.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.365+0000 7feb36a00640 1 -- 172.21.15.67:0/3077415427 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7feb3007ac30 con 0x7feb400c8190 2024-03-09T08:52:44.361 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T08:52:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 -- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb2006b4d0 msgr2=0x7feb2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb2006b4d0 0x7feb2006d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb400c55b0 tx=0x7feb2c009190 comp rx=0 tx=0).stop 2024-03-09T08:52:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 -- 172.21.15.67:0/3077415427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb400c8190 msgr2=0x7feb401522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb400c8190 0x7feb401522b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feb3002b880 tx=0x7feb300046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 -- 172.21.15.67:0/3077415427 shutdown_connections 2024-03-09T08:52:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb2006b4d0 0x7feb2006d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb40152f20 0x7feb401ebc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb400ca2c0 0x7feb401eb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 --2- 172.21.15.67:0/3077415427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb400c8190 0x7feb401522b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 -- 172.21.15.67:0/3077415427 >> 172.21.15.67:0/3077415427 conn(0x7feb400ad060 msgr2=0x7feb4006b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 -- 172.21.15.67:0/3077415427 shutdown_connections 2024-03-09T08:52:44.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.368+0000 7feb3fe00640 1 -- 172.21.15.67:0/3077415427 wait complete. 2024-03-09T08:52:44.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-03-09T08:52:44.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.599+0000 7fa183e00640 1 Processor -- start 2024-03-09T08:52:44.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.599+0000 7fa183e00640 1 -- start start 2024-03-09T08:52:44.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.601+0000 7fa183e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa184151030 0x7fa18406e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:44.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.601+0000 7fa183e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18406bf80 con 0x7fa184153ed0 2024-03-09T08:52:44.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.601+0000 7fa183e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1840c3b30 con 0x7fa1840c3160 2024-03-09T08:52:44.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.601+0000 7fa183e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1840c4ac0 con 0x7fa184151030 2024-03-09T08:52:44.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.601+0000 7fa182a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa184151030 0x7fa18406e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:44.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.601+0000 7fa183400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa184153ed0 0x7fa1841542e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53884/0 (socket says 172.21.15.67:53884) 2024-03-09T08:52:44.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.601+0000 7fa183400640 1 -- 172.21.15.67:0/966925270 learned_addr learned my addr 172.21.15.67:0/966925270 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.602+0000 7fa182a00640 1 -- 172.21.15.67:0/966925270 >> v1:172.21.15.134:6789/0 conn(0x7fa1840c3160 legacy=0x7fa1840c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.602+0000 7fa182a00640 1 -- 172.21.15.67:0/966925270 >> v1:172.21.15.67:6789/0 conn(0x7fa184153ed0 legacy=0x7fa1841542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.602+0000 7fa182a00640 1 -- 172.21.15.67:0/966925270 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18406b9c0 con 0x7fa184151030 2024-03-09T08:52:44.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.603+0000 7fa182a00640 1 --2- 172.21.15.67:0/966925270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa184151030 0x7fa18406e6a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa1740097a0 tx=0x7fa17402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c12a89689ac1a3a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:44.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.603+0000 7fa181600640 1 -- 172.21.15.67:0/966925270 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740315a0 con 0x7fa184151030 2024-03-09T08:52:44.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.603+0000 7fa181600640 1 -- 172.21.15.67:0/966925270 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174037070 con 0x7fa184151030 2024-03-09T08:52:44.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.603+0000 7fa181600640 1 -- 172.21.15.67:0/966925270 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174031960 con 0x7fa184151030 2024-03-09T08:52:44.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.604+0000 7fa183e00640 1 -- 172.21.15.67:0/966925270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa184151030 msgr2=0x7fa18406e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.604+0000 7fa183e00640 1 --2- 172.21.15.67:0/966925270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa184151030 0x7fa18406e6a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa1740097a0 tx=0x7fa17402b840 comp rx=0 tx=0).stop 2024-03-09T08:52:44.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.604+0000 7fa183e00640 1 -- 172.21.15.67:0/966925270 shutdown_connections 2024-03-09T08:52:44.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.604+0000 7fa183e00640 1 --2- 172.21.15.67:0/966925270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa184151030 0x7fa18406e6a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.604+0000 7fa183e00640 1 -- 172.21.15.67:0/966925270 >> 172.21.15.67:0/966925270 conn(0x7fa1840aade0 msgr2=0x7fa18406d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:44.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.604+0000 7fa183e00640 1 -- 172.21.15.67:0/966925270 shutdown_connections 2024-03-09T08:52:44.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.604+0000 7fa183e00640 1 -- 172.21.15.67:0/966925270 wait complete. 2024-03-09T08:52:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.606+0000 7fa183e00640 1 Processor -- start 2024-03-09T08:52:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.607+0000 7fa183e00640 1 -- start start 2024-03-09T08:52:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.608+0000 7fa183e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1840c3160 0x7fa1841efd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.608+0000 7fa183e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa184151030 0x7fa1840c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.608+0000 7fa183e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa184153ed0 0x7fa1841ec430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.608+0000 7fa183e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840c89e0 con 0x7fa1840c3160 2024-03-09T08:52:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.608+0000 7fa183e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1841549f0 con 0x7fa184153ed0 2024-03-09T08:52:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.608+0000 7fa183e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840c8b60 con 0x7fa184151030 2024-03-09T08:52:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.608+0000 7fa182a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1840c3160 0x7fa1841efd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.609+0000 7fa182a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1840c3160 0x7fa1841efd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46674/0 (socket says 172.21.15.67:46674) 2024-03-09T08:52:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.609+0000 7fa182a00640 1 -- 172.21.15.67:0/1858026384 learned_addr learned my addr 172.21.15.67:0/1858026384 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.609+0000 7fa182000640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa184151030 0x7fa1840c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.609+0000 7fa183400640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa184153ed0 0x7fa1841ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:44.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.609+0000 7fa182a00640 1 -- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa184151030 msgr2=0x7fa1840c1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.609+0000 7fa182a00640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa184151030 0x7fa1840c1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.609+0000 7fa182a00640 1 -- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa184153ed0 msgr2=0x7fa1841ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.609+0000 7fa182a00640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa184153ed0 0x7fa1841ec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.609+0000 7fa182a00640 1 -- 172.21.15.67:0/1858026384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa184073170 con 0x7fa1840c3160 2024-03-09T08:52:44.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.609+0000 7fa182a00640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1840c3160 0x7fa1841efd80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa174000c00 tx=0x7fa1740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.610+0000 7fa17aa00640 1 -- 172.21.15.67:0/1858026384 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174032840 con 0x7fa1840c3160 2024-03-09T08:52:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.610+0000 7fa17aa00640 1 -- 172.21.15.67:0/1858026384 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1740329e0 con 0x7fa1840c3160 2024-03-09T08:52:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.610+0000 7fa17aa00640 1 -- 172.21.15.67:0/1858026384 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174032dd0 con 0x7fa1840c3160 2024-03-09T08:52:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.610+0000 7fa183e00640 1 -- 172.21.15.67:0/1858026384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1840c63b0 con 0x7fa1840c3160 2024-03-09T08:52:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.610+0000 7fa183e00640 1 -- 172.21.15.67:0/1858026384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1840c6970 con 0x7fa1840c3160 2024-03-09T08:52:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.611+0000 7fa183e00640 1 -- 172.21.15.67:0/1858026384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18404f3c0 con 0x7fa1840c3160 2024-03-09T08:52:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.612+0000 7fa17aa00640 1 -- 172.21.15.67:0/1858026384 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa17403b3f0 con 0x7fa1840c3160 2024-03-09T08:52:44.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.613+0000 7fa17aa00640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa16406b4d0 0x7fa16406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:44.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.613+0000 7fa17aa00640 1 -- 172.21.15.67:0/1858026384 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fa17402f030 con 0x7fa1840c3160 2024-03-09T08:52:44.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.613+0000 7fa182000640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa16406b4d0 0x7fa16406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:44.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.614+0000 7fa182000640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa16406b4d0 0x7fa16406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa184073820 tx=0x7fa170009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:44.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.617+0000 7fa17aa00640 1 -- 172.21.15.67:0/1858026384 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa174040070 con 0x7fa1840c3160 2024-03-09T08:52:44.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.827+0000 7fa183e00640 1 -- 172.21.15.67:0/1858026384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7fa18406b600 con 0x7fa1840c3160 2024-03-09T08:52:44.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.828+0000 7fa17aa00640 1 -- 172.21.15.67:0/1858026384 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7fa17407b030 con 0x7fa1840c3160 2024-03-09T08:52:44.824 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T08:52:44.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 -- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa16406b4d0 msgr2=0x7fa16406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa16406b4d0 0x7fa16406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa184073820 tx=0x7fa170009040 comp rx=0 tx=0).stop 2024-03-09T08:52:44.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 -- 172.21.15.67:0/1858026384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1840c3160 msgr2=0x7fa1841efd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:44.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1840c3160 0x7fa1841efd80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa174000c00 tx=0x7fa1740046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 -- 172.21.15.67:0/1858026384 shutdown_connections 2024-03-09T08:52:44.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa16406b4d0 0x7fa16406d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa184153ed0 0x7fa1841ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa184151030 0x7fa1840c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 --2- 172.21.15.67:0/1858026384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1840c3160 0x7fa1841efd80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:44.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 -- 172.21.15.67:0/1858026384 >> 172.21.15.67:0/1858026384 conn(0x7fa1840aade0 msgr2=0x7fa184153340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:44.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 -- 172.21.15.67:0/1858026384 shutdown_connections 2024-03-09T08:52:44.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:44.830+0000 7fa183e00640 1 -- 172.21.15.67:0/1858026384 wait complete. 2024-03-09T08:52:44.843 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T08:52:45.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.029+0000 7f12cfe00640 1 Processor -- start 2024-03-09T08:52:45.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.029+0000 7f12cfe00640 1 -- start start 2024-03-09T08:52:45.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.031+0000 7f12cfe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0150fb0 0x7f12d00c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.031+0000 7f12cfe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d0157f30 con 0x7f12d0157a30 2024-03-09T08:52:45.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.031+0000 7f12cfe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d0153b40 con 0x7f12d00c5350 2024-03-09T08:52:45.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.031+0000 7f12cfe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0153e20 con 0x7f12d0150fb0 2024-03-09T08:52:45.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.031+0000 7f12ce000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f12d0157a30 0x7f12d00c2d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53898/0 (socket says 172.21.15.67:53898) 2024-03-09T08:52:45.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.031+0000 7f12ce000640 1 -- 172.21.15.67:0/2537384604 learned_addr learned my addr 172.21.15.67:0/2537384604 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:45.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cea00640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0150fb0 0x7f12d00c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 645479478 0 0) 0x7f12d0157f30 con 0x7f12d0157a30 2024-03-09T08:52:45.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b4003520 con 0x7f12d0157a30 2024-03-09T08:52:45.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2735469236 0 0) 0x7f12b4003520 con 0x7f12d0157a30 2024-03-09T08:52:45.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0150fb0 msgr2=0x7f12d00c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:45.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0150fb0 0x7f12d00c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 >> v1:172.21.15.134:6789/0 conn(0x7f12d00c5350 legacy=0x7f12d015e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d0154100 con 0x7f12d0157a30 2024-03-09T08:52:45.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f12bc001670 con 0x7f12d0157a30 2024-03-09T08:52:45.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 >> v1:172.21.15.67:6789/0 conn(0x7f12d0157a30 legacy=0x7f12d00c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0150fb0 0x7f12b4003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b4003fd0 0x7f12b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4004d40 0x7f12b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0153e20 con 0x7f12b4004d40 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b400bbb0 con 0x7f12b4003fd0 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.032+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b400bdb0 con 0x7f12d0150fb0 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.033+0000 7f12cf400640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4004d40 0x7f12b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.033+0000 7f12cea00640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0150fb0 0x7f12b4003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.033+0000 7f12ce000640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b4003fd0 0x7f12b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.033+0000 7f12cf400640 1 -- 172.21.15.67:0/2537384604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0150fb0 msgr2=0x7f12b4003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.033+0000 7f12cf400640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0150fb0 0x7f12b4003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.033+0000 7f12cf400640 1 -- 172.21.15.67:0/2537384604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b4003fd0 msgr2=0x7f12b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:45.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.033+0000 7f12cf400640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12b4003fd0 0x7f12b4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.033+0000 7f12cf400640 1 -- 172.21.15.67:0/2537384604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b4003520 con 0x7f12b4004d40 2024-03-09T08:52:45.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.034+0000 7f12cf400640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4004d40 0x7f12b40051c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f12c8001e50 tx=0x7f12b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afbbe149f0a0e67 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:45.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.034+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8001a70 con 0x7f12b4004d40 2024-03-09T08:52:45.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.034+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c8003b70 con 0x7f12b4004d40 2024-03-09T08:52:45.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.034+0000 7f12cd600640 1 -- 172.21.15.67:0/2537384604 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c80050d0 con 0x7f12b4004d40 2024-03-09T08:52:45.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.035+0000 7f12cfe00640 1 -- 172.21.15.67:0/2537384604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4004d40 msgr2=0x7f12b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.035+0000 7f12cfe00640 1 --2- 172.21.15.67:0/2537384604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12b4004d40 0x7f12b40051c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f12c8001e50 tx=0x7f12b4005830 comp rx=0 tx=0).stop 2024-03-09T08:52:45.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.035+0000 7f12cf400640 1 -- 172.21.15.67:0/2537384604 reap_dead start 2024-03-09T08:52:45.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.035+0000 7f12cfe00640 1 -- 172.21.15.67:0/2537384604 shutdown_connections 2024-03-09T08:52:45.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.035+0000 7f12cfe00640 1 -- 172.21.15.67:0/2537384604 >> 172.21.15.67:0/2537384604 conn(0x7f12d00c8380 msgr2=0x7f12d00c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:45.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.035+0000 7f12cfe00640 1 -- 172.21.15.67:0/2537384604 shutdown_connections 2024-03-09T08:52:45.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.035+0000 7f12cfe00640 1 -- 172.21.15.67:0/2537384604 wait complete. 2024-03-09T08:52:45.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.037+0000 7f12cfe00640 1 Processor -- start 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.037+0000 7f12cfe00640 1 -- start start 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.037+0000 7f12cfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d0155ec0 0x7f12d014dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cfe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0073530 0x7f12d01eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01ef040 0x7f12d014cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cfe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0151390 con 0x7f12d01ef040 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cfe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0154900 con 0x7f12d0155ec0 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cfe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d006e8a0 con 0x7f12d0073530 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01ef040 0x7f12d014cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01ef040 0x7f12d014cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46680/0 (socket says 172.21.15.67:46680) 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cf400640 1 -- 172.21.15.67:0/1941447547 learned_addr learned my addr 172.21.15.67:0/1941447547 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cea00640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d0155ec0 0x7f12d014dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12ce000640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0073530 0x7f12d01eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cf400640 1 -- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0073530 msgr2=0x7f12d01eeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cf400640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0073530 0x7f12d01eeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cf400640 1 -- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d0155ec0 msgr2=0x7f12d014dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cf400640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d0155ec0 0x7f12d014dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.038+0000 7f12cf400640 1 -- 172.21.15.67:0/1941447547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d00739c0 con 0x7f12d01ef040 2024-03-09T08:52:45.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.039+0000 7f12cf400640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01ef040 0x7f12d014cb50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f12c8001f70 tx=0x7f12c80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:45.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.039+0000 7f12c6a00640 1 -- 172.21.15.67:0/1941447547 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8001a70 con 0x7f12d01ef040 2024-03-09T08:52:45.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.039+0000 7f12c6a00640 1 -- 172.21.15.67:0/1941447547 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c8003970 con 0x7f12d01ef040 2024-03-09T08:52:45.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.039+0000 7f12c6a00640 1 -- 172.21.15.67:0/1941447547 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8007db0 con 0x7f12d01ef040 2024-03-09T08:52:45.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.039+0000 7f12cfe00640 1 -- 172.21.15.67:0/1941447547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d00c6cd0 con 0x7f12d01ef040 2024-03-09T08:52:45.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.039+0000 7f12cfe00640 1 -- 172.21.15.67:0/1941447547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d00c7290 con 0x7f12d01ef040 2024-03-09T08:52:45.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.040+0000 7f12cfe00640 1 -- 172.21.15.67:0/1941447547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d004f3c0 con 0x7f12d01ef040 2024-03-09T08:52:45.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.041+0000 7f12c6a00640 1 -- 172.21.15.67:0/1941447547 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f12c8041030 con 0x7f12d01ef040 2024-03-09T08:52:45.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.042+0000 7f12c6a00640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12b006b4d0 0x7f12b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.042+0000 7f12c6a00640 1 -- 172.21.15.67:0/1941447547 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f12c80ab4d0 con 0x7f12d01ef040 2024-03-09T08:52:45.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.042+0000 7f12cea00640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12b006b4d0 0x7f12b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.043+0000 7f12cea00640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12b006b4d0 0x7f12b006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12d00c4400 tx=0x7f12c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:45.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.046+0000 7f12c6a00640 1 -- 172.21.15.67:0/1941447547 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f12c8033020 con 0x7f12d01ef040 2024-03-09T08:52:45.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.277+0000 7f12cfe00640 1 -- 172.21.15.67:0/1941447547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12d00c5d30 con 0x7f12d01ef040 2024-03-09T08:52:45.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.278+0000 7f12c6a00640 1 -- 172.21.15.67:0/1941447547 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6247 (secure 0 0 0) 0x7f12c8079ef0 con 0x7f12d01ef040 2024-03-09T08:52:45.274 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:45.274 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:17.736973+0000","modified":"2024-03-09T08:52:21.931108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T08:52:45.274 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 14 2024-03-09T08:52:45.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.280+0000 7f12cfe00640 1 -- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12b006b4d0 msgr2=0x7f12b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.280+0000 7f12cfe00640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12b006b4d0 0x7f12b006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12d00c4400 tx=0x7f12c0008040 comp rx=0 tx=0).stop 2024-03-09T08:52:45.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.280+0000 7f12cfe00640 1 -- 172.21.15.67:0/1941447547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01ef040 msgr2=0x7f12d014cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.280+0000 7f12cfe00640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01ef040 0x7f12d014cb50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f12c8001f70 tx=0x7f12c80014a0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.280+0000 7f12cfe00640 1 -- 172.21.15.67:0/1941447547 shutdown_connections 2024-03-09T08:52:45.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.280+0000 7f12cfe00640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12b006b4d0 0x7f12b006d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.280+0000 7f12cfe00640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d01ef040 0x7f12d014cb50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.280+0000 7f12cfe00640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d0073530 0x7f12d01eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.280+0000 7f12cfe00640 1 --2- 172.21.15.67:0/1941447547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d0155ec0 0x7f12d014dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.280+0000 7f12cfe00640 1 -- 172.21.15.67:0/1941447547 >> 172.21.15.67:0/1941447547 conn(0x7f12d00c8380 msgr2=0x7f12d01518a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:45.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.281+0000 7f12cfe00640 1 -- 172.21.15.67:0/1941447547 shutdown_connections 2024-03-09T08:52:45.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.281+0000 7f12cfe00640 1 -- 172.21.15.67:0/1941447547 wait complete. 2024-03-09T08:52:45.293 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:45.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.492+0000 7f2691600640 1 Processor -- start 2024-03-09T08:52:45.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.493+0000 7f2691600640 1 -- start start 2024-03-09T08:52:45.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.495+0000 7f2691600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c153370 0x7f268c0c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.495+0000 7f2691600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268c156d50 con 0x7f268c0c2400 2024-03-09T08:52:45.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.495+0000 7f2691600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268c0aa4c0 con 0x7f268c0c1d70 2024-03-09T08:52:45.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.495+0000 7f2691600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0aadf0 con 0x7f268c153370 2024-03-09T08:52:45.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.495+0000 7f268b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f268c0c2400 0x7f268c0acd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53900/0 (socket says 172.21.15.67:53900) 2024-03-09T08:52:45.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.495+0000 7f268b400640 1 -- 172.21.15.67:0/1230051229 learned_addr learned my addr 172.21.15.67:0/1230051229 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:45.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.495+0000 7f268aa00640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c153370 0x7f268c0c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079391843 0 0) 0x7f268c156d50 con 0x7f268c0c2400 2024-03-09T08:52:45.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2670003520 con 0x7f268c0c2400 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1424764697 0 0) 0x7f2670003520 con 0x7f268c0c2400 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c153370 msgr2=0x7f268c0c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c153370 0x7f268c0c07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 >> v1:172.21.15.134:6789/0 conn(0x7f268c0c1d70 legacy=0x7f268c0ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0aa7a0 con 0x7f268c0c2400 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f2684001670 con 0x7f268c0c2400 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 >> v1:172.21.15.67:6789/0 conn(0x7f268c0c2400 legacy=0x7f268c0acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 0x7f2670003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c153370 0x7f2670004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2670004d40 0x7f26700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0aadf0 con 0x7f26700039e0 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267000bb80 con 0x7f2670004d40 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.496+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267000bd80 con 0x7f268c153370 2024-03-09T08:52:45.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.497+0000 7f268aa00640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 0x7f2670003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.497+0000 7f2682000640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c153370 0x7f2670004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.497+0000 7f268b400640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2670004d40 0x7f26700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.497+0000 7f268aa00640 1 -- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c153370 msgr2=0x7f2670004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:45.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.497+0000 7f268aa00640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c153370 0x7f2670004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.497+0000 7f268aa00640 1 -- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2670004d40 msgr2=0x7f26700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:45.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.497+0000 7f268aa00640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2670004d40 0x7f26700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.497+0000 7f268aa00640 1 -- 172.21.15.67:0/1230051229 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2670003520 con 0x7f26700039e0 2024-03-09T08:52:45.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.497+0000 7f268b400640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2670004d40 0x7f26700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:45.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.498+0000 7f268aa00640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 0x7f2670003e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f267c009fc0 tx=0x7f2670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf2049e02559055 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:45.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.498+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c039070 con 0x7f26700039e0 2024-03-09T08:52:45.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.498+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c02cc20 con 0x7f26700039e0 2024-03-09T08:52:45.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.498+0000 7f268a000640 1 -- 172.21.15.67:0/1230051229 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c009c10 con 0x7f26700039e0 2024-03-09T08:52:45.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.499+0000 7f2691600640 1 -- 172.21.15.67:0/1230051229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 msgr2=0x7f2670003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.499+0000 7f2691600640 1 --2- 172.21.15.67:0/1230051229 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 0x7f2670003e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f267c009fc0 tx=0x7f2670005830 comp rx=0 tx=0).stop 2024-03-09T08:52:45.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.499+0000 7f268b400640 1 -- 172.21.15.67:0/1230051229 reap_dead start 2024-03-09T08:52:45.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.499+0000 7f2691600640 1 -- 172.21.15.67:0/1230051229 shutdown_connections 2024-03-09T08:52:45.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.499+0000 7f2691600640 1 -- 172.21.15.67:0/1230051229 >> 172.21.15.67:0/1230051229 conn(0x7f268c0c7370 msgr2=0x7f268c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:45.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.499+0000 7f2691600640 1 -- 172.21.15.67:0/1230051229 shutdown_connections 2024-03-09T08:52:45.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.499+0000 7f2691600640 1 -- 172.21.15.67:0/1230051229 wait complete. 2024-03-09T08:52:45.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.501+0000 7f2691600640 1 Processor -- start 2024-03-09T08:52:45.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.501+0000 7f2691600640 1 -- start start 2024-03-09T08:52:45.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.502+0000 7f2691600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 0x7f268c1500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.502+0000 7f2691600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c159140 0x7f268c14a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.502+0000 7f2691600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f268c1ef420 0x7f268c1486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.502+0000 7f2691600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1538d0 con 0x7f26700039e0 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.502+0000 7f2691600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c153750 con 0x7f268c1ef420 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.502+0000 7f2691600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c1568e0 con 0x7f268c159140 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.504+0000 7f268aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 0x7f268c1500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.504+0000 7f268aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 0x7f268c1500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46706/0 (socket says 172.21.15.67:46706) 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.504+0000 7f268aa00640 1 -- 172.21.15.67:0/621339871 learned_addr learned my addr 172.21.15.67:0/621339871 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.504+0000 7f2682000640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c159140 0x7f268c14a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.504+0000 7f268aa00640 1 -- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c159140 msgr2=0x7f268c14a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.504+0000 7f268aa00640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c159140 0x7f268c14a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.504+0000 7f268aa00640 1 -- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f268c1ef420 msgr2=0x7f268c1486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.504+0000 7f268aa00640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f268c1ef420 0x7f268c1486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.504+0000 7f268aa00640 1 -- 172.21.15.67:0/621339871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c1595c0 con 0x7f26700039e0 2024-03-09T08:52:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.504+0000 7f268aa00640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 0x7f268c1500d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f267c00e200 tx=0x7f267c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.505+0000 7f2683400640 1 -- 172.21.15.67:0/621339871 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c039050 con 0x7f26700039e0 2024-03-09T08:52:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.505+0000 7f2683400640 1 -- 172.21.15.67:0/621339871 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c03a040 con 0x7f26700039e0 2024-03-09T08:52:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.505+0000 7f2683400640 1 -- 172.21.15.67:0/621339871 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c009c10 con 0x7f26700039e0 2024-03-09T08:52:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.505+0000 7f2691600640 1 -- 172.21.15.67:0/621339871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c073270 con 0x7f26700039e0 2024-03-09T08:52:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.505+0000 7f2691600640 1 -- 172.21.15.67:0/621339871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c1eec20 con 0x7f26700039e0 2024-03-09T08:52:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.505+0000 7f2691600640 1 -- 172.21.15.67:0/621339871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c075a00 con 0x7f26700039e0 2024-03-09T08:52:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.507+0000 7f2683400640 1 -- 172.21.15.67:0/621339871 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f267c003bb0 con 0x7f26700039e0 2024-03-09T08:52:45.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.507+0000 7f2683400640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f266806b500 0x7f266806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:45.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.508+0000 7f2683400640 1 -- 172.21.15.67:0/621339871 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f267c0acce0 con 0x7f26700039e0 2024-03-09T08:52:45.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.508+0000 7f2682000640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f266806b500 0x7f266806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.510+0000 7f2682000640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f266806b500 0x7f266806d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2674007fd0 tx=0x7f2674009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.511+0000 7f2683400640 1 -- 172.21.15.67:0/621339871 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f267c07b820 con 0x7f26700039e0 2024-03-09T08:52:45.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.804+0000 7f2691600640 1 -- 172.21.15.67:0/621339871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f268c0c1530 con 0x7f26700039e0 2024-03-09T08:52:45.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.806+0000 7f2683400640 1 -- 172.21.15.67:0/621339871 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20307 (secure 0 0 0) 0x7f267c07b120 con 0x7f26700039e0 2024-03-09T08:52:45.802 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:45.802 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:30.565529+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.809+0000 7f2691600640 1 -- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f266806b500 msgr2=0x7f266806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.809+0000 7f2691600640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f266806b500 0x7f266806d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2674007fd0 tx=0x7f2674009040 comp rx=0 tx=0).stop 2024-03-09T08:52:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.809+0000 7f2691600640 1 -- 172.21.15.67:0/621339871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 msgr2=0x7f268c1500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.809+0000 7f2691600640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 0x7f268c1500d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f267c00e200 tx=0x7f267c009750 comp rx=0 tx=0).stop 2024-03-09T08:52:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.809+0000 7f2691600640 1 -- 172.21.15.67:0/621339871 shutdown_connections 2024-03-09T08:52:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.809+0000 7f2691600640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f266806b500 0x7f266806d9c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.809+0000 7f2691600640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f268c1ef420 0x7f268c1486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.809+0000 7f2691600640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f268c159140 0x7f268c14a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.809+0000 7f2691600640 1 --2- 172.21.15.67:0/621339871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26700039e0 0x7f268c1500d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.809+0000 7f2691600640 1 -- 172.21.15.67:0/621339871 >> 172.21.15.67:0/621339871 conn(0x7f268c0c7370 msgr2=0x7f268c154140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.810+0000 7f2691600640 1 -- 172.21.15.67:0/621339871 shutdown_connections 2024-03-09T08:52:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:45.810+0000 7f2691600640 1 -- 172.21.15.67:0/621339871 wait complete. 2024-03-09T08:52:45.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-09T08:52:06.749641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-03-09T08:52:16.185356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-03-09T08:52:16.756829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-03-09T08:52:27.484297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-03-09T08:52:28.060754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-09T08:52:45.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T08:52:46.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.044+0000 7f05fb200640 1 Processor -- start 2024-03-09T08:52:46.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.045+0000 7f05fb200640 1 -- start start 2024-03-09T08:52:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.046+0000 7f05fb200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05f40c6240 0x7f05f40c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.046+0000 7f05fb200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f05f40c6620 0x7f05f4156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.046+0000 7f05fb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f4156bc0 con 0x7f05f40734d0 2024-03-09T08:52:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.046+0000 7f05fb200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f40c6a90 con 0x7f05f40c6620 2024-03-09T08:52:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.046+0000 7f05fb200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f4159d00 con 0x7f05f40c6240 2024-03-09T08:52:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.046+0000 7f05f9400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f05f40734d0 0x7f05f4155f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53908/0 (socket says 172.21.15.67:53908) 2024-03-09T08:52:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.046+0000 7f05f9400640 1 -- 172.21.15.67:0/3608911509 learned_addr learned my addr 172.21.15.67:0/3608911509 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.047+0000 7f05f9e00640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05f40c6240 0x7f05f40c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.047+0000 7f05fa800640 1 --2- 172.21.15.67:0/3608911509 >> v2:172.21.15.134:3300/0 conn(0x7f05f40c6620 0x7f05f4156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.047+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437073274 0 0) 0x7f05f4156bc0 con 0x7f05f40734d0 2024-03-09T08:52:46.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.047+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05dc003520 con 0x7f05f40734d0 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.047+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2500842357 0 0) 0x7f05dc003520 con 0x7f05f40734d0 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.047+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05f40c6240 msgr2=0x7f05f40c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.047+0000 7f05f8a00640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05f40c6240 0x7f05f40c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.047+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 >> v2:172.21.15.134:3300/0 conn(0x7f05f40c6620 msgr2=0x7f05f4156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.047+0000 7f05f8a00640 1 --2- 172.21.15.67:0/3608911509 >> v2:172.21.15.134:3300/0 conn(0x7f05f40c6620 0x7f05f4156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.047+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f4159fe0 con 0x7f05f40734d0 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.047+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f05e4001670 con 0x7f05f40734d0 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 >> v1:172.21.15.67:6789/0 conn(0x7f05f40734d0 legacy=0x7f05f4155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f8a00640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05dc0039b0 0x7f05dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f8a00640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05f40c6240 0x7f05dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f8a00640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05dc004d40 0x7f05dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40c6a90 con 0x7f05dc0039b0 2024-03-09T08:52:46.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f4159d00 con 0x7f05dc004d40 2024-03-09T08:52:46.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc00b9c0 con 0x7f05f40c6240 2024-03-09T08:52:46.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f9e00640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05dc0039b0 0x7f05dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05fa800640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05dc004d40 0x7f05dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f9e00640 1 -- 172.21.15.67:0/3608911509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05f40c6240 msgr2=0x7f05dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:46.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f9e00640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05f40c6240 0x7f05dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f9e00640 1 -- 172.21.15.67:0/3608911509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05dc004d40 msgr2=0x7f05dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:46.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f9e00640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05dc004d40 0x7f05dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05f9e00640 1 -- 172.21.15.67:0/3608911509 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc003520 con 0x7f05dc0039b0 2024-03-09T08:52:46.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.048+0000 7f05fa800640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05dc004d40 0x7f05dc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.049+0000 7f05f9e00640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05dc0039b0 0x7f05dc003dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009ac0 tx=0x7f05dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d837d48411e6686e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.050+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8039070 con 0x7f05dc0039b0 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.050+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e802c710 con 0x7f05dc0039b0 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.050+0000 7f05f8a00640 1 -- 172.21.15.67:0/3608911509 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8009690 con 0x7f05dc0039b0 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.050+0000 7f05fb200640 1 -- 172.21.15.67:0/3608911509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05dc0039b0 msgr2=0x7f05dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.050+0000 7f05fb200640 1 --2- 172.21.15.67:0/3608911509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05dc0039b0 0x7f05dc003dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009ac0 tx=0x7f05dc005830 comp rx=0 tx=0).stop 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.050+0000 7f05fa800640 1 -- 172.21.15.67:0/3608911509 reap_dead start 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.050+0000 7f05fb200640 1 -- 172.21.15.67:0/3608911509 shutdown_connections 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.050+0000 7f05fb200640 1 -- 172.21.15.67:0/3608911509 >> 172.21.15.67:0/3608911509 conn(0x7f05f4066fa0 msgr2=0x7f05f40673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.050+0000 7f05fb200640 1 -- 172.21.15.67:0/3608911509 shutdown_connections 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.050+0000 7f05fb200640 1 -- 172.21.15.67:0/3608911509 wait complete. 2024-03-09T08:52:46.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.053+0000 7f05fb200640 1 Processor -- start 2024-03-09T08:52:46.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.053+0000 7f05fb200640 1 -- start start 2024-03-09T08:52:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05fb200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05dc0039b0 0x7f05f414fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05fb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f40c8320 0x7f05f41df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05fb200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f4157400 0x7f05f41578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05fb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40c44c0 con 0x7f05f40c8320 2024-03-09T08:52:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05fb200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40c41e0 con 0x7f05f4157400 2024-03-09T08:52:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05fb200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f406dfc0 con 0x7f05dc0039b0 2024-03-09T08:52:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05f9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f40c8320 0x7f05f41df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05f9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f40c8320 0x7f05f41df5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46724/0 (socket says 172.21.15.67:46724) 2024-03-09T08:52:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05f9400640 1 -- 172.21.15.67:0/2619178458 learned_addr learned my addr 172.21.15.67:0/2619178458 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05f9e00640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05dc0039b0 0x7f05f414fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05fa800640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f4157400 0x7f05f41578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05f9400640 1 -- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05dc0039b0 msgr2=0x7f05f414fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05f9400640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05dc0039b0 0x7f05f414fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.055+0000 7f05f9400640 1 -- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f4157400 msgr2=0x7f05f41578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.056+0000 7f05f9400640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f4157400 0x7f05f41578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.056+0000 7f05f9400640 1 -- 172.21.15.67:0/2619178458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f41e0a80 con 0x7f05f40c8320 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.056+0000 7f05fa800640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f4157400 0x7f05f41578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.056+0000 7f05f9400640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f40c8320 0x7f05f41df5f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f05e400c6c0 tx=0x7f05e400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.056+0000 7f05ee000640 1 -- 172.21.15.67:0/2619178458 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4018070 con 0x7f05f40c8320 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.056+0000 7f05ee000640 1 -- 172.21.15.67:0/2619178458 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e40072f0 con 0x7f05f40c8320 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.056+0000 7f05fb200640 1 -- 172.21.15.67:0/2619178458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f41e0cf0 con 0x7f05f40c8320 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.056+0000 7f05ee000640 1 -- 172.21.15.67:0/2619178458 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40047e0 con 0x7f05f40c8320 2024-03-09T08:52:46.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.056+0000 7f05fb200640 1 -- 172.21.15.67:0/2619178458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f40c9e60 con 0x7f05f40c8320 2024-03-09T08:52:46.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.058+0000 7f05fb200640 1 -- 172.21.15.67:0/2619178458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f40c9c50 con 0x7f05f40c8320 2024-03-09T08:52:46.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.059+0000 7f05ee000640 1 -- 172.21.15.67:0/2619178458 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f05e401b070 con 0x7f05f40c8320 2024-03-09T08:52:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.060+0000 7f05ee000640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05d806b4d0 0x7f05d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.060+0000 7f05ee000640 1 -- 172.21.15.67:0/2619178458 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f05e4089990 con 0x7f05f40c8320 2024-03-09T08:52:46.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.060+0000 7f05f9e00640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05d806b4d0 0x7f05d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.061+0000 7f05f9e00640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05d806b4d0 0x7f05d806d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f05e8036000 tx=0x7f05e8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:46.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.064+0000 7f05ee000640 1 -- 172.21.15.67:0/2619178458 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f05e4058570 con 0x7f05f40c8320 2024-03-09T08:52:46.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.319+0000 7f05fb200640 1 -- 172.21.15.67:0/2619178458 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05f404f3c0 con 0x7f05f40c8320 2024-03-09T08:52:46.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.320+0000 7f05ee000640 1 -- 172.21.15.67:0/2619178458 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05e4057e70 con 0x7f05f40c8320 2024-03-09T08:52:46.315 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-09T08:52:46.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.322+0000 7f05fb200640 1 -- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05d806b4d0 msgr2=0x7f05d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.322+0000 7f05fb200640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05d806b4d0 0x7f05d806d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f05e8036000 tx=0x7f05e8030000 comp rx=0 tx=0).stop 2024-03-09T08:52:46.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.322+0000 7f05fb200640 1 -- 172.21.15.67:0/2619178458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f40c8320 msgr2=0x7f05f41df5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.322+0000 7f05fb200640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f40c8320 0x7f05f41df5f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f05e400c6c0 tx=0x7f05e400cb80 comp rx=0 tx=0).stop 2024-03-09T08:52:46.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.323+0000 7f05fb200640 1 -- 172.21.15.67:0/2619178458 shutdown_connections 2024-03-09T08:52:46.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.323+0000 7f05fb200640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05d806b4d0 0x7f05d806d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.323+0000 7f05fb200640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f4157400 0x7f05f41578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.323+0000 7f05fb200640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05f40c8320 0x7f05f41df5f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.323+0000 7f05fb200640 1 --2- 172.21.15.67:0/2619178458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05dc0039b0 0x7f05f414fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.323+0000 7f05fb200640 1 -- 172.21.15.67:0/2619178458 >> 172.21.15.67:0/2619178458 conn(0x7f05f4066fa0 msgr2=0x7f05f4155690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:46.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.323+0000 7f05fb200640 1 -- 172.21.15.67:0/2619178458 shutdown_connections 2024-03-09T08:52:46.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.323+0000 7f05fb200640 1 -- 172.21.15.67:0/2619178458 wait complete. 2024-03-09T08:52:46.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-03-09T08:52:46.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.558+0000 7feff9400640 1 Processor -- start 2024-03-09T08:52:46.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.558+0000 7feff9400640 1 -- start start 2024-03-09T08:52:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.560+0000 7feff9400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7feff40c5f10 0x7feff40c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.560+0000 7feff9400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff406b040 0x7feff41567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.560+0000 7feff9400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff4156d10 con 0x7feff40c9490 2024-03-09T08:52:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.560+0000 7feff9400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff40c9a30 con 0x7feff40c5f10 2024-03-09T08:52:46.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.560+0000 7feff9400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff40c9d10 con 0x7feff406b040 2024-03-09T08:52:46.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.560+0000 7feff8a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7feff40c9490 0x7feff40cab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53922/0 (socket says 172.21.15.67:53922) 2024-03-09T08:52:46.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.560+0000 7feff8a00640 1 -- 172.21.15.67:0/3110385591 learned_addr learned my addr 172.21.15.67:0/3110385591 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:46.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.560+0000 7feff3e00640 1 --2- 172.21.15.67:0/3110385591 >> v2:172.21.15.134:3300/0 conn(0x7feff40c5f10 0x7feff40c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.560+0000 7fefebe00640 1 --2- 172.21.15.67:0/3110385591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff406b040 0x7feff41567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.561+0000 7feff3400640 1 -- 172.21.15.67:0/3110385591 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215028184 0 0) 0x7fefec001490 con 0x7feff40c9490 2024-03-09T08:52:46.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.561+0000 7feff3400640 1 -- 172.21.15.67:0/3110385591 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd8003520 con 0x7feff40c9490 2024-03-09T08:52:46.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.561+0000 7feff3e00640 1 -- 172.21.15.67:0/3110385591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff406b040 msgr2=0x7feff41567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:46.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.561+0000 7feff3e00640 1 --2- 172.21.15.67:0/3110385591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff406b040 0x7feff41567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.561+0000 7feff3e00640 1 -- 172.21.15.67:0/3110385591 >> v1:172.21.15.67:6789/0 conn(0x7feff40c9490 legacy=0x7feff40cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.561+0000 7feff3400640 1 -- 172.21.15.67:0/3110385591 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1620526354 0 0) 0x7fefd8003520 con 0x7feff40c9490 2024-03-09T08:52:46.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.561+0000 7feff3e00640 1 -- 172.21.15.67:0/3110385591 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff40ca340 con 0x7feff40c5f10 2024-03-09T08:52:46.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.561+0000 7fefebe00640 1 --2- 172.21.15.67:0/3110385591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff406b040 0x7feff41567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:52:46.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.563+0000 7feff3e00640 1 --2- 172.21.15.67:0/3110385591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feff40c5f10 0x7feff40c62f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefe40097a0 tx=0x7fefe402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a54e5ab6e1145301 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:46.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.563+0000 7feff3400640 1 -- 172.21.15.67:0/3110385591 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4037070 con 0x7feff40c5f10 2024-03-09T08:52:46.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.563+0000 7feff3400640 1 -- 172.21.15.67:0/3110385591 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe4030510 con 0x7feff40c5f10 2024-03-09T08:52:46.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.563+0000 7feff3400640 1 -- 172.21.15.67:0/3110385591 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec001bc0 con 0x7feff40c5f10 2024-03-09T08:52:46.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.564+0000 7feff9400640 1 -- 172.21.15.67:0/3110385591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feff40c5f10 msgr2=0x7feff40c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.564+0000 7feff9400640 1 --2- 172.21.15.67:0/3110385591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feff40c5f10 0x7feff40c62f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefe40097a0 tx=0x7fefe402b710 comp rx=0 tx=0).stop 2024-03-09T08:52:46.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.564+0000 7feff9400640 1 -- 172.21.15.67:0/3110385591 shutdown_connections 2024-03-09T08:52:46.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.564+0000 7feff9400640 1 --2- 172.21.15.67:0/3110385591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff406b040 0x7feff41567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.564+0000 7feff9400640 1 --2- 172.21.15.67:0/3110385591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feff40c5f10 0x7feff40c62f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.564+0000 7feff9400640 1 -- 172.21.15.67:0/3110385591 >> 172.21.15.67:0/3110385591 conn(0x7feff40cb590 msgr2=0x7feff40cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:46.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.564+0000 7feff9400640 1 -- 172.21.15.67:0/3110385591 shutdown_connections 2024-03-09T08:52:46.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.564+0000 7feff9400640 1 -- 172.21.15.67:0/3110385591 wait complete. 2024-03-09T08:52:46.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.566+0000 7feff9400640 1 Processor -- start 2024-03-09T08:52:46.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.566+0000 7feff9400640 1 -- start start 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7feff9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff406b040 0x7feff41ef070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7feff9400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feff40c5f10 0x7feff414ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7feff9400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff40c9490 0x7feff414cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7feff9400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff40cb250 con 0x7feff406b040 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7feff9400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff40c9ff0 con 0x7feff40c5f10 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7feff9400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff40cb3d0 con 0x7feff40c9490 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7feff3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff406b040 0x7feff41ef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7feff3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff406b040 0x7feff41ef070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46730/0 (socket says 172.21.15.67:46730) 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7feff3e00640 1 -- 172.21.15.67:0/797850038 learned_addr learned my addr 172.21.15.67:0/797850038 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7fefebe00640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feff40c5f10 0x7feff414ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7feff8a00640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff40c9490 0x7feff414cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.567+0000 7feff3e00640 1 -- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff40c9490 msgr2=0x7feff414cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7feff3e00640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff40c9490 0x7feff414cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7feff3e00640 1 -- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feff40c5f10 msgr2=0x7feff414ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7feff3e00640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feff40c5f10 0x7feff414ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7feff3e00640 1 -- 172.21.15.67:0/797850038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4145b20 con 0x7feff406b040 2024-03-09T08:52:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7feff8a00640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff40c9490 0x7feff414cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7fefebe00640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feff40c5f10 0x7feff414ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7feff3e00640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff406b040 0x7feff41ef070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fefe402b750 tx=0x7fefe40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7feff0c00640 1 -- 172.21.15.67:0/797850038 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4042020 con 0x7feff406b040 2024-03-09T08:52:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7feff0c00640 1 -- 172.21.15.67:0/797850038 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe40044c0 con 0x7feff406b040 2024-03-09T08:52:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7feff0c00640 1 -- 172.21.15.67:0/797850038 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4037040 con 0x7feff406b040 2024-03-09T08:52:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7feff9400640 1 -- 172.21.15.67:0/797850038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff4145840 con 0x7feff406b040 2024-03-09T08:52:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.568+0000 7feff9400640 1 -- 172.21.15.67:0/797850038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff41efb90 con 0x7feff406b040 2024-03-09T08:52:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.569+0000 7feff9400640 1 -- 172.21.15.67:0/797850038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff404f3c0 con 0x7feff406b040 2024-03-09T08:52:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.571+0000 7feff0c00640 1 -- 172.21.15.67:0/797850038 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fefe4045020 con 0x7feff406b040 2024-03-09T08:52:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.572+0000 7feff0c00640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd006b4d0 0x7fefd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.572+0000 7feff0c00640 1 -- 172.21.15.67:0/797850038 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fefe40ac150 con 0x7feff406b040 2024-03-09T08:52:46.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.572+0000 7fefebe00640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd006b4d0 0x7fefd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:46.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.573+0000 7fefebe00640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd006b4d0 0x7fefd006d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feff40ab380 tx=0x7fefdc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:46.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.576+0000 7feff0c00640 1 -- 172.21.15.67:0/797850038 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fefe404c030 con 0x7feff406b040 2024-03-09T08:52:46.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.780+0000 7feff9400640 1 -- 172.21.15.67:0/797850038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7feff40cc260 con 0x7feff406b040 2024-03-09T08:52:46.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.781+0000 7feff0c00640 1 -- 172.21.15.67:0/797850038 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7fefe407ad30 con 0x7feff406b040 2024-03-09T08:52:46.776 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T08:52:46.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.783+0000 7feff9400640 1 -- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd006b4d0 msgr2=0x7fefd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.783+0000 7feff9400640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd006b4d0 0x7fefd006d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feff40ab380 tx=0x7fefdc008040 comp rx=0 tx=0).stop 2024-03-09T08:52:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.783+0000 7feff9400640 1 -- 172.21.15.67:0/797850038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff406b040 msgr2=0x7feff41ef070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.783+0000 7feff9400640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff406b040 0x7feff41ef070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fefe402b750 tx=0x7fefe40046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.784+0000 7feff9400640 1 -- 172.21.15.67:0/797850038 shutdown_connections 2024-03-09T08:52:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.784+0000 7feff9400640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fefd006b4d0 0x7fefd006d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.784+0000 7feff9400640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feff40c9490 0x7feff414cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.784+0000 7feff9400640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feff40c5f10 0x7feff414ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.784+0000 7feff9400640 1 --2- 172.21.15.67:0/797850038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feff406b040 0x7feff41ef070 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.784+0000 7feff9400640 1 -- 172.21.15.67:0/797850038 >> 172.21.15.67:0/797850038 conn(0x7feff40cb590 msgr2=0x7feff406b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.784+0000 7feff9400640 1 -- 172.21.15.67:0/797850038 shutdown_connections 2024-03-09T08:52:46.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:46.784+0000 7feff9400640 1 -- 172.21.15.67:0/797850038 wait complete. 2024-03-09T08:52:46.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-03-09T08:52:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.011+0000 7f9925400640 1 Processor -- start 2024-03-09T08:52:47.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.012+0000 7f9925400640 1 -- start start 2024-03-09T08:52:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.013+0000 7f9925400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99200becf0 0x7f9920072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.013+0000 7f9925400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99200c9bb0 0x7f99200ca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.013+0000 7f9925400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99200c6080 con 0x7f9920072770 2024-03-09T08:52:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.013+0000 7f9925400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99200c9720 con 0x7f99200c9bb0 2024-03-09T08:52:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.013+0000 7f9925400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99200c62f0 con 0x7f99200becf0 2024-03-09T08:52:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.013+0000 7f991e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9920072770 0x7f9920156750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53938/0 (socket says 172.21.15.67:53938) 2024-03-09T08:52:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.013+0000 7f991e800640 1 -- 172.21.15.67:0/1037696122 learned_addr learned my addr 172.21.15.67:0/1037696122 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.013+0000 7f991f200640 1 --2- 172.21.15.67:0/1037696122 >> v2:172.21.15.67:3300/0 conn(0x7f99200c9bb0 0x7f99200ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.014+0000 7f991de00640 1 --2- 172.21.15.67:0/1037696122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99200becf0 0x7f9920072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.014+0000 7f991d400640 1 -- 172.21.15.67:0/1037696122 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591533084 0 0) 0x7f99200c6080 con 0x7f9920072770 2024-03-09T08:52:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.014+0000 7f991d400640 1 -- 172.21.15.67:0/1037696122 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9904003520 con 0x7f9920072770 2024-03-09T08:52:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.014+0000 7f991f200640 1 -- 172.21.15.67:0/1037696122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99200becf0 msgr2=0x7f9920072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.014+0000 7f991f200640 1 --2- 172.21.15.67:0/1037696122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99200becf0 0x7f9920072b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.014+0000 7f991f200640 1 -- 172.21.15.67:0/1037696122 >> v1:172.21.15.67:6789/0 conn(0x7f9920072770 legacy=0x7f9920156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.014+0000 7f991f200640 1 -- 172.21.15.67:0/1037696122 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99200c65d0 con 0x7f99200c9bb0 2024-03-09T08:52:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.015+0000 7f991f200640 1 --2- 172.21.15.67:0/1037696122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99200c9bb0 0x7f99200ca030 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9918009d90 tx=0x7f991802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e01ca017707a17 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.015+0000 7f991d400640 1 -- 172.21.15.67:0/1037696122 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918039070 con 0x7f99200c9bb0 2024-03-09T08:52:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.016+0000 7f991d400640 1 -- 172.21.15.67:0/1037696122 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991802cb30 con 0x7f99200c9bb0 2024-03-09T08:52:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.016+0000 7f991d400640 1 -- 172.21.15.67:0/1037696122 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918034440 con 0x7f99200c9bb0 2024-03-09T08:52:47.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.016+0000 7f9925400640 1 -- 172.21.15.67:0/1037696122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99200c9bb0 msgr2=0x7f99200ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.016+0000 7f9925400640 1 --2- 172.21.15.67:0/1037696122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99200c9bb0 0x7f99200ca030 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9918009d90 tx=0x7f991802bd80 comp rx=0 tx=0).stop 2024-03-09T08:52:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.016+0000 7f9925400640 1 -- 172.21.15.67:0/1037696122 shutdown_connections 2024-03-09T08:52:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.016+0000 7f9925400640 1 --2- 172.21.15.67:0/1037696122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99200c9bb0 0x7f99200ca030 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.016+0000 7f9925400640 1 --2- 172.21.15.67:0/1037696122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99200becf0 0x7f9920072b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.016+0000 7f9925400640 1 -- 172.21.15.67:0/1037696122 >> 172.21.15.67:0/1037696122 conn(0x7f99200cb840 msgr2=0x7f99200cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.016+0000 7f9925400640 1 -- 172.21.15.67:0/1037696122 shutdown_connections 2024-03-09T08:52:47.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.016+0000 7f9925400640 1 -- 172.21.15.67:0/1037696122 wait complete. 2024-03-09T08:52:47.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.018+0000 7f9925400640 1 Processor -- start 2024-03-09T08:52:47.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.018+0000 7f9925400640 1 -- start start 2024-03-09T08:52:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.019+0000 7f9925400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9920072770 0x7f992014f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.019+0000 7f9925400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99200becf0 0x7f9920153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.019+0000 7f9925400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99200c9bb0 0x7f99200bf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.019+0000 7f9925400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9920153700 con 0x7f99200becf0 2024-03-09T08:52:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.019+0000 7f9925400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9920153580 con 0x7f99200c9bb0 2024-03-09T08:52:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.019+0000 7f9925400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99201571c0 con 0x7f9920072770 2024-03-09T08:52:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.019+0000 7f991de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99200becf0 0x7f9920153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.019+0000 7f991de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99200becf0 0x7f9920153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46748/0 (socket says 172.21.15.67:46748) 2024-03-09T08:52:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.019+0000 7f991de00640 1 -- 172.21.15.67:0/1726359129 learned_addr learned my addr 172.21.15.67:0/1726359129 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.020+0000 7f991e800640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9920072770 0x7f992014f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.020+0000 7f991f200640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99200c9bb0 0x7f99200bf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.020+0000 7f991de00640 1 -- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9920072770 msgr2=0x7f992014f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.020+0000 7f991de00640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9920072770 0x7f992014f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.020+0000 7f991de00640 1 -- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99200c9bb0 msgr2=0x7f99200bf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.020+0000 7f991de00640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99200c9bb0 0x7f99200bf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.020+0000 7f991de00640 1 -- 172.21.15.67:0/1726359129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99201ef290 con 0x7f99200becf0 2024-03-09T08:52:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.020+0000 7f991de00640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99200becf0 0x7f9920153b40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f990800e9b0 tx=0x7f990800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.020+0000 7f9916a00640 1 -- 172.21.15.67:0/1726359129 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908017070 con 0x7f99200becf0 2024-03-09T08:52:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.021+0000 7f9916a00640 1 -- 172.21.15.67:0/1726359129 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990800c9a0 con 0x7f99200becf0 2024-03-09T08:52:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.021+0000 7f9916a00640 1 -- 172.21.15.67:0/1726359129 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800cd70 con 0x7f99200becf0 2024-03-09T08:52:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.021+0000 7f9925400640 1 -- 172.21.15.67:0/1726359129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99201ef510 con 0x7f99200becf0 2024-03-09T08:52:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.021+0000 7f9925400640 1 -- 172.21.15.67:0/1726359129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992014f990 con 0x7f99200becf0 2024-03-09T08:52:47.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.022+0000 7f9925400640 1 -- 172.21.15.67:0/1726359129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9920152340 con 0x7f99200becf0 2024-03-09T08:52:47.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.024+0000 7f9916a00640 1 -- 172.21.15.67:0/1726359129 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f990801a070 con 0x7f99200becf0 2024-03-09T08:52:47.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.025+0000 7f9916a00640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.025+0000 7f9916a00640 1 -- 172.21.15.67:0/1726359129 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f990808a810 con 0x7f99200becf0 2024-03-09T08:52:47.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.025+0000 7f991e800640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.027+0000 7f991e800640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98fc06b500 0x7f98fc06d9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9910005880 tx=0x7f9910009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:47.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.029+0000 7f9916a00640 1 -- 172.21.15.67:0/1726359129 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9920152340 con 0x7f99200becf0 2024-03-09T08:52:47.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.228+0000 7f9925400640 1 -- 172.21.15.67:0/1726359129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f9920075a00 con 0x7f99200becf0 2024-03-09T08:52:47.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.228+0000 7f9916a00640 1 -- 172.21.15.67:0/1726359129 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7f99201ef290 con 0x7f99200becf0 2024-03-09T08:52:47.223 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T08:52:47.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 -- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98fc06b500 msgr2=0x7f98fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98fc06b500 0x7f98fc06d9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9910005880 tx=0x7f9910009040 comp rx=0 tx=0).stop 2024-03-09T08:52:47.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 -- 172.21.15.67:0/1726359129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99200becf0 msgr2=0x7f9920153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99200becf0 0x7f9920153b40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f990800e9b0 tx=0x7f990800ee70 comp rx=0 tx=0).stop 2024-03-09T08:52:47.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 -- 172.21.15.67:0/1726359129 shutdown_connections 2024-03-09T08:52:47.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99200c9bb0 0x7f99200bf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99200becf0 0x7f9920153b40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 --2- 172.21.15.67:0/1726359129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9920072770 0x7f992014f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 -- 172.21.15.67:0/1726359129 >> 172.21.15.67:0/1726359129 conn(0x7f99200cb840 msgr2=0x7f9920151460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:47.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 -- 172.21.15.67:0/1726359129 shutdown_connections 2024-03-09T08:52:47.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.231+0000 7f9925400640 1 -- 172.21.15.67:0/1726359129 wait complete. 2024-03-09T08:52:47.244 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-03-09T08:52:47.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.472+0000 7f20c9600640 1 Processor -- start 2024-03-09T08:52:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.473+0000 7f20c9600640 1 -- start start 2024-03-09T08:52:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.474+0000 7f20c9600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20c40c73d0 0x7f20c41547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.474+0000 7f20c9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f20c4151720 0x7f20c4072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.474+0000 7f20c9600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c406d6c0 con 0x7f20c40710a0 2024-03-09T08:52:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.474+0000 7f20c9600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c406e480 con 0x7f20c4151720 2024-03-09T08:52:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.474+0000 7f20c9600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c406dff0 con 0x7f20c40c73d0 2024-03-09T08:52:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.475+0000 7f20c3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f20c4151720 0x7f20c4072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.475+0000 7f20c3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f20c4151720 0x7f20c4072cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46760/0 (socket says 172.21.15.67:46760) 2024-03-09T08:52:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.475+0000 7f20c3400640 1 -- 172.21.15.67:0/1351771382 learned_addr learned my addr 172.21.15.67:0/1351771382 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:47.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.475+0000 7f20c2a00640 1 --2- 172.21.15.67:0/1351771382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20c40c73d0 0x7f20c41547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.475+0000 7f20c3400640 1 -- 172.21.15.67:0/1351771382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20c40c73d0 msgr2=0x7f20c41547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.475+0000 7f20c3400640 1 --2- 172.21.15.67:0/1351771382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20c40c73d0 0x7f20c41547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.475+0000 7f20c3400640 1 -- 172.21.15.67:0/1351771382 >> v1:172.21.15.134:6789/0 conn(0x7f20c40710a0 legacy=0x7f20c4151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.475+0000 7f20c3400640 1 -- 172.21.15.67:0/1351771382 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c406d9a0 con 0x7f20c4151720 2024-03-09T08:52:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.475+0000 7f20c2a00640 1 --2- 172.21.15.67:0/1351771382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20c40c73d0 0x7f20c41547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:47.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.477+0000 7f20c3400640 1 --2- 172.21.15.67:0/1351771382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20c4151720 0x7f20c4072cd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009e50 tx=0x7f20bc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb02d3afa1275ff5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:47.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.477+0000 7f20c1600640 1 -- 172.21.15.67:0/1351771382 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc02cb00 con 0x7f20c4151720 2024-03-09T08:52:47.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.477+0000 7f20c1600640 1 -- 172.21.15.67:0/1351771382 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc02cca0 con 0x7f20c4151720 2024-03-09T08:52:47.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.477+0000 7f20c1600640 1 -- 172.21.15.67:0/1351771382 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc034610 con 0x7f20c4151720 2024-03-09T08:52:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.478+0000 7f20c9600640 1 -- 172.21.15.67:0/1351771382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20c4151720 msgr2=0x7f20c4072cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.478+0000 7f20c9600640 1 --2- 172.21.15.67:0/1351771382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20c4151720 0x7f20c4072cd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009e50 tx=0x7f20bc02bf60 comp rx=0 tx=0).stop 2024-03-09T08:52:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.478+0000 7f20c9600640 1 -- 172.21.15.67:0/1351771382 shutdown_connections 2024-03-09T08:52:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.478+0000 7f20c9600640 1 --2- 172.21.15.67:0/1351771382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20c4151720 0x7f20c4072cd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.478+0000 7f20c9600640 1 --2- 172.21.15.67:0/1351771382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20c40c73d0 0x7f20c41547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.478+0000 7f20c9600640 1 -- 172.21.15.67:0/1351771382 >> 172.21.15.67:0/1351771382 conn(0x7f20c406b040 msgr2=0x7f20c406b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.478+0000 7f20c9600640 1 -- 172.21.15.67:0/1351771382 shutdown_connections 2024-03-09T08:52:47.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.478+0000 7f20c9600640 1 -- 172.21.15.67:0/1351771382 wait complete. 2024-03-09T08:52:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.480+0000 7f20c9600640 1 Processor -- start 2024-03-09T08:52:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.480+0000 7f20c9600640 1 -- start start 2024-03-09T08:52:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20c40710a0 0x7f20c40bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c9600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20c40c73d0 0x7f20c41e0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20c4151720 0x7f20c414d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c9600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40c77b0 con 0x7f20c40710a0 2024-03-09T08:52:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c9600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c4071500 con 0x7f20c4151720 2024-03-09T08:52:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c9600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40c7930 con 0x7f20c40c73d0 2024-03-09T08:52:47.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20c40710a0 0x7f20c40bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20c40710a0 0x7f20c40bfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46770/0 (socket says 172.21.15.67:46770) 2024-03-09T08:52:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c2a00640 1 -- 172.21.15.67:0/732073073 learned_addr learned my addr 172.21.15.67:0/732073073 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c2000640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20c40c73d0 0x7f20c41e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c2a00640 1 -- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20c40c73d0 msgr2=0x7f20c41e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c2a00640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20c40c73d0 0x7f20c41e0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c2a00640 1 -- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20c4151720 msgr2=0x7f20c414d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c2a00640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20c4151720 0x7f20c414d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.481+0000 7f20c2a00640 1 -- 172.21.15.67:0/732073073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c41de840 con 0x7f20c40710a0 2024-03-09T08:52:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.482+0000 7f20c2a00640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20c40710a0 0x7f20c40bfee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f20b400e730 tx=0x7f20b400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.482+0000 7f20baa00640 1 -- 172.21.15.67:0/732073073 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b400f040 con 0x7f20c40710a0 2024-03-09T08:52:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.482+0000 7f20baa00640 1 -- 172.21.15.67:0/732073073 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b4004230 con 0x7f20c40710a0 2024-03-09T08:52:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.482+0000 7f20c9600640 1 -- 172.21.15.67:0/732073073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c41deae0 con 0x7f20c40710a0 2024-03-09T08:52:47.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.482+0000 7f20baa00640 1 -- 172.21.15.67:0/732073073 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b400ee40 con 0x7f20c40710a0 2024-03-09T08:52:47.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.482+0000 7f20c9600640 1 -- 172.21.15.67:0/732073073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c40be8f0 con 0x7f20c40710a0 2024-03-09T08:52:47.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.483+0000 7f20c9600640 1 -- 172.21.15.67:0/732073073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c4075a00 con 0x7f20c40710a0 2024-03-09T08:52:47.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.484+0000 7f20baa00640 1 -- 172.21.15.67:0/732073073 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f20b400c6d0 con 0x7f20c40710a0 2024-03-09T08:52:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.485+0000 7f20baa00640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f20a006b4d0 0x7f20a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.485+0000 7f20baa00640 1 -- 172.21.15.67:0/732073073 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f20b408a2b0 con 0x7f20c40710a0 2024-03-09T08:52:47.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.486+0000 7f20c2000640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f20a006b4d0 0x7f20a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.487+0000 7f20c2000640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f20a006b4d0 0x7f20a006d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f20c40c5df0 tx=0x7f20ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:47.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.489+0000 7f20baa00640 1 -- 172.21.15.67:0/732073073 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f20b4017020 con 0x7f20c40710a0 2024-03-09T08:52:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.692+0000 7f20c9600640 1 -- 172.21.15.67:0/732073073 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20c40c33e0 con 0x7f20c40710a0 2024-03-09T08:52:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.693+0000 7f20baa00640 1 -- 172.21.15.67:0/732073073 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7f20b4058d60 con 0x7f20c40710a0 2024-03-09T08:52:47.687 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T08:52:47.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.695+0000 7f20c9600640 1 -- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f20a006b4d0 msgr2=0x7f20a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.695+0000 7f20c9600640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f20a006b4d0 0x7f20a006d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f20c40c5df0 tx=0x7f20ac009040 comp rx=0 tx=0).stop 2024-03-09T08:52:47.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.695+0000 7f20c9600640 1 -- 172.21.15.67:0/732073073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20c40710a0 msgr2=0x7f20c40bfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.695+0000 7f20c9600640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20c40710a0 0x7f20c40bfee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f20b400e730 tx=0x7f20b400ebf0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.695+0000 7f20c9600640 1 -- 172.21.15.67:0/732073073 shutdown_connections 2024-03-09T08:52:47.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.695+0000 7f20c9600640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f20a006b4d0 0x7f20a006d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.695+0000 7f20c9600640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20c4151720 0x7f20c414d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.695+0000 7f20c9600640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20c40c73d0 0x7f20c41e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.695+0000 7f20c9600640 1 --2- 172.21.15.67:0/732073073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f20c40710a0 0x7f20c40bfee0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.695+0000 7f20c9600640 1 -- 172.21.15.67:0/732073073 >> 172.21.15.67:0/732073073 conn(0x7f20c406b040 msgr2=0x7f20c40c2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:47.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.696+0000 7f20c9600640 1 -- 172.21.15.67:0/732073073 shutdown_connections 2024-03-09T08:52:47.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.696+0000 7f20c9600640 1 -- 172.21.15.67:0/732073073 wait complete. 2024-03-09T08:52:47.709 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-03-09T08:52:47.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.945+0000 7fe724c00640 1 Processor -- start 2024-03-09T08:52:47.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.945+0000 7fe724c00640 1 -- start start 2024-03-09T08:52:47.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.947+0000 7fe724c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe7200c73d0 0x7fe7201547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.947+0000 7fe724c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe7200710a0 0x7fe720151010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.947+0000 7fe724c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72006d6c0 con 0x7fe720151550 2024-03-09T08:52:47.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.947+0000 7fe724c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72006e480 con 0x7fe7200710a0 2024-03-09T08:52:47.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.947+0000 7fe724c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7201519c0 con 0x7fe7200c73d0 2024-03-09T08:52:47.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.947+0000 7fe715400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe7200710a0 0x7fe720151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.947+0000 7fe71e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe720151550 0x7fe720072cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53944/0 (socket says 172.21.15.67:53944) 2024-03-09T08:52:47.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.947+0000 7fe71e800640 1 -- 172.21.15.67:0/1438699160 learned_addr learned my addr 172.21.15.67:0/1438699160 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:47.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.947+0000 7fe71de00640 1 --2- 172.21.15.67:0/1438699160 >> v2:172.21.15.134:3300/0 conn(0x7fe7200c73d0 0x7fe7201547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.948+0000 7fe715400640 1 -- 172.21.15.67:0/1438699160 >> v2:172.21.15.134:3300/0 conn(0x7fe7200c73d0 msgr2=0x7fe7201547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:47.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.948+0000 7fe715400640 1 --2- 172.21.15.67:0/1438699160 >> v2:172.21.15.134:3300/0 conn(0x7fe7200c73d0 0x7fe7201547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.948+0000 7fe715400640 1 -- 172.21.15.67:0/1438699160 >> v1:172.21.15.67:6789/0 conn(0x7fe720151550 legacy=0x7fe720072cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.948+0000 7fe715400640 1 -- 172.21.15.67:0/1438699160 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72006d9a0 con 0x7fe7200710a0 2024-03-09T08:52:47.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.948+0000 7fe71d400640 1 -- 172.21.15.67:0/1438699160 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796795088 0 0) 0x7fe72006d6c0 con 0x7fe720151550 2024-03-09T08:52:47.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.949+0000 7fe715400640 1 --2- 172.21.15.67:0/1438699160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7200710a0 0x7fe720151010 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe7200ac6b0 tx=0x7fe70802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940af725e7a64662 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:47.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.949+0000 7fe71d400640 1 -- 172.21.15.67:0/1438699160 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708035020 con 0x7fe7200710a0 2024-03-09T08:52:47.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.949+0000 7fe71d400640 1 -- 172.21.15.67:0/1438699160 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708031830 con 0x7fe7200710a0 2024-03-09T08:52:47.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.949+0000 7fe71d400640 1 -- 172.21.15.67:0/1438699160 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708031c00 con 0x7fe7200710a0 2024-03-09T08:52:47.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.950+0000 7fe724c00640 1 -- 172.21.15.67:0/1438699160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7200710a0 msgr2=0x7fe720151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.950+0000 7fe724c00640 1 --2- 172.21.15.67:0/1438699160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7200710a0 0x7fe720151010 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe7200ac6b0 tx=0x7fe70802b8d0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.950+0000 7fe724c00640 1 -- 172.21.15.67:0/1438699160 shutdown_connections 2024-03-09T08:52:47.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.950+0000 7fe724c00640 1 --2- 172.21.15.67:0/1438699160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7200710a0 0x7fe720151010 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.950+0000 7fe724c00640 1 --2- 172.21.15.67:0/1438699160 >> v2:172.21.15.134:3300/0 conn(0x7fe7200c73d0 0x7fe7201547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.950+0000 7fe724c00640 1 -- 172.21.15.67:0/1438699160 >> 172.21.15.67:0/1438699160 conn(0x7fe72006b040 msgr2=0x7fe72006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:47.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.950+0000 7fe724c00640 1 -- 172.21.15.67:0/1438699160 shutdown_connections 2024-03-09T08:52:47.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.950+0000 7fe724c00640 1 -- 172.21.15.67:0/1438699160 wait complete. 2024-03-09T08:52:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.952+0000 7fe724c00640 1 Processor -- start 2024-03-09T08:52:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.952+0000 7fe724c00640 1 -- start start 2024-03-09T08:52:47.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe724c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7200710a0 0x7fe7201e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe724c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7200c73d0 0x7fe7200bc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe724c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720151550 0x7fe7200be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe724c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720071500 con 0x7fe720151550 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe724c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72006dff0 con 0x7fe7200c73d0 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe724c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200c77b0 con 0x7fe7200710a0 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe71e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720151550 0x7fe7200be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe71e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720151550 0x7fe7200be6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46790/0 (socket says 172.21.15.67:46790) 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe71e800640 1 -- 172.21.15.67:0/2319215391 learned_addr learned my addr 172.21.15.67:0/2319215391 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe71de00640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7200710a0 0x7fe7201e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe715400640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7200c73d0 0x7fe7200bc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe71e800640 1 -- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7200710a0 msgr2=0x7fe7201e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe71e800640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7200710a0 0x7fe7201e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.953+0000 7fe71e800640 1 -- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7200c73d0 msgr2=0x7fe7200bc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:47.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.954+0000 7fe71e800640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7200c73d0 0x7fe7200bc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:47.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.954+0000 7fe71e800640 1 -- 172.21.15.67:0/2319215391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7200bc8b0 con 0x7fe720151550 2024-03-09T08:52:47.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.954+0000 7fe715400640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7200c73d0 0x7fe7200bc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:47.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.954+0000 7fe71e800640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720151550 0x7fe7200be6f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe71800c730 tx=0x7fe71800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:47.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.954+0000 7fe716a00640 1 -- 172.21.15.67:0/2319215391 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718018070 con 0x7fe720151550 2024-03-09T08:52:47.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.954+0000 7fe716a00640 1 -- 172.21.15.67:0/2319215391 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71800ce40 con 0x7fe720151550 2024-03-09T08:52:47.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.954+0000 7fe716a00640 1 -- 172.21.15.67:0/2319215391 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71800a860 con 0x7fe720151550 2024-03-09T08:52:47.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.954+0000 7fe724c00640 1 -- 172.21.15.67:0/2319215391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7200bcb90 con 0x7fe720151550 2024-03-09T08:52:47.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.954+0000 7fe724c00640 1 -- 172.21.15.67:0/2319215391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7200bd150 con 0x7fe720151550 2024-03-09T08:52:47.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.956+0000 7fe724c00640 1 -- 172.21.15.67:0/2319215391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe720075a00 con 0x7fe720151550 2024-03-09T08:52:47.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.957+0000 7fe716a00640 1 -- 172.21.15.67:0/2319215391 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe71801b070 con 0x7fe720151550 2024-03-09T08:52:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.957+0000 7fe716a00640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6fc06b4d0 0x7fe6fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:47.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.958+0000 7fe716a00640 1 -- 172.21.15.67:0/2319215391 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fe718089aa0 con 0x7fe720151550 2024-03-09T08:52:47.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.958+0000 7fe71de00640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6fc06b4d0 0x7fe6fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:47.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.959+0000 7fe71de00640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6fc06b4d0 0x7fe6fc06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe710006fd0 tx=0x7fe710008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:47.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:47.962+0000 7fe716a00640 1 -- 172.21.15.67:0/2319215391 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe718058680 con 0x7fe720151550 2024-03-09T08:52:48.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.163+0000 7fe724c00640 1 -- 172.21.15.67:0/2319215391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7fe7200c4b20 con 0x7fe720151550 2024-03-09T08:52:48.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.164+0000 7fe716a00640 1 -- 172.21.15.67:0/2319215391 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7fe718057f80 con 0x7fe720151550 2024-03-09T08:52:48.159 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T08:52:48.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.167+0000 7fe724c00640 1 -- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6fc06b4d0 msgr2=0x7fe6fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.167+0000 7fe724c00640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6fc06b4d0 0x7fe6fc06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe710006fd0 tx=0x7fe710008040 comp rx=0 tx=0).stop 2024-03-09T08:52:48.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.167+0000 7fe724c00640 1 -- 172.21.15.67:0/2319215391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720151550 msgr2=0x7fe7200be6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.167+0000 7fe724c00640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720151550 0x7fe7200be6f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe71800c730 tx=0x7fe71800cbf0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.167+0000 7fe724c00640 1 -- 172.21.15.67:0/2319215391 shutdown_connections 2024-03-09T08:52:48.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.167+0000 7fe724c00640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6fc06b4d0 0x7fe6fc06d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.167+0000 7fe724c00640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe720151550 0x7fe7200be6f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.167+0000 7fe724c00640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7200c73d0 0x7fe7200bc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.167+0000 7fe724c00640 1 --2- 172.21.15.67:0/2319215391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe7200710a0 0x7fe7201e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.167+0000 7fe724c00640 1 -- 172.21.15.67:0/2319215391 >> 172.21.15.67:0/2319215391 conn(0x7fe72006b040 msgr2=0x7fe72014f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:48.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.168+0000 7fe724c00640 1 -- 172.21.15.67:0/2319215391 shutdown_connections 2024-03-09T08:52:48.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.168+0000 7fe724c00640 1 -- 172.21.15.67:0/2319215391 wait complete. 2024-03-09T08:52:48.183 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T08:52:48.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.404+0000 7fc6f9c00640 1 Processor -- start 2024-03-09T08:52:48.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.404+0000 7fc6f9c00640 1 -- start start 2024-03-09T08:52:48.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.406+0000 7fc6f9c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6f4153300 0x7fc6f40b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:48.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.406+0000 7fc6f9c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f4157000 con 0x7fc6f40cadb0 2024-03-09T08:52:48.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.406+0000 7fc6f9c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f40ae430 con 0x7fc6f40c5e40 2024-03-09T08:52:48.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.406+0000 7fc6f9c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f40ae6f0 con 0x7fc6f4153300 2024-03-09T08:52:48.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.406+0000 7fc6f3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6f4153300 0x7fc6f40b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:48.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.406+0000 7fc6f9200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc6f40cadb0 0x7fc6f4160490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53960/0 (socket says 172.21.15.67:53960) 2024-03-09T08:52:48.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.406+0000 7fc6f9200640 1 -- 172.21.15.67:0/3254516578 learned_addr learned my addr 172.21.15.67:0/3254516578 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:48.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.407+0000 7fc6f3e00640 1 -- 172.21.15.67:0/3254516578 >> v1:172.21.15.134:6789/0 conn(0x7fc6f40c5e40 legacy=0x7fc6f4156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.407+0000 7fc6f3e00640 1 -- 172.21.15.67:0/3254516578 >> v1:172.21.15.67:6789/0 conn(0x7fc6f40cadb0 legacy=0x7fc6f4160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.407+0000 7fc6f3e00640 1 -- 172.21.15.67:0/3254516578 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f4155b10 con 0x7fc6f4153300 2024-03-09T08:52:48.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.408+0000 7fc6f3e00640 1 --2- 172.21.15.67:0/3254516578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f4153300 0x7fc6f40b0000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40097a0 tx=0x7fc6e402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1fed1a21923a69 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:48.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.408+0000 7fc6f3400640 1 -- 172.21.15.67:0/3254516578 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4034070 con 0x7fc6f4153300 2024-03-09T08:52:48.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.409+0000 7fc6f3400640 1 -- 172.21.15.67:0/3254516578 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e4030610 con 0x7fc6f4153300 2024-03-09T08:52:48.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.409+0000 7fc6f3400640 1 -- 172.21.15.67:0/3254516578 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40309e0 con 0x7fc6f4153300 2024-03-09T08:52:48.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.409+0000 7fc6f9c00640 1 -- 172.21.15.67:0/3254516578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f4153300 msgr2=0x7fc6f40b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.409+0000 7fc6f9c00640 1 --2- 172.21.15.67:0/3254516578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f4153300 0x7fc6f40b0000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40097a0 tx=0x7fc6e402b960 comp rx=0 tx=0).stop 2024-03-09T08:52:48.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.409+0000 7fc6f9c00640 1 -- 172.21.15.67:0/3254516578 shutdown_connections 2024-03-09T08:52:48.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.409+0000 7fc6f9c00640 1 --2- 172.21.15.67:0/3254516578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f4153300 0x7fc6f40b0000 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.409+0000 7fc6f9c00640 1 -- 172.21.15.67:0/3254516578 >> 172.21.15.67:0/3254516578 conn(0x7fc6f40732d0 msgr2=0x7fc6f40ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:48.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.409+0000 7fc6f9c00640 1 -- 172.21.15.67:0/3254516578 shutdown_connections 2024-03-09T08:52:48.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.409+0000 7fc6f9c00640 1 -- 172.21.15.67:0/3254516578 wait complete. 2024-03-09T08:52:48.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.411+0000 7fc6f9c00640 1 Processor -- start 2024-03-09T08:52:48.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.411+0000 7fc6f9c00640 1 -- start start 2024-03-09T08:52:48.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.412+0000 7fc6f9c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f40c5e40 0x7fc6f40c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.412+0000 7fc6f9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f40cadb0 0x7fc6f414b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.412+0000 7fc6f9c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6f4153300 0x7fc6f414bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.412+0000 7fc6f9c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40c59e0 con 0x7fc6f40cadb0 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.412+0000 7fc6f9c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f4159e90 con 0x7fc6f40c5e40 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.412+0000 7fc6f9c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40b0540 con 0x7fc6f4153300 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.413+0000 7fc6ebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f40cadb0 0x7fc6f414b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.413+0000 7fc6ebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f40cadb0 0x7fc6f414b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46800/0 (socket says 172.21.15.67:46800) 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.413+0000 7fc6ebe00640 1 -- 172.21.15.67:0/899971387 learned_addr learned my addr 172.21.15.67:0/899971387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.413+0000 7fc6f3e00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f40c5e40 0x7fc6f40c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.413+0000 7fc6f9200640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6f4153300 0x7fc6f414bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.413+0000 7fc6ebe00640 1 -- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6f4153300 msgr2=0x7fc6f414bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.413+0000 7fc6ebe00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6f4153300 0x7fc6f414bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.413+0000 7fc6ebe00640 1 -- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f40c5e40 msgr2=0x7fc6f40c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.413+0000 7fc6ebe00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f40c5e40 0x7fc6f40c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.413+0000 7fc6ebe00640 1 -- 172.21.15.67:0/899971387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f406d7b0 con 0x7fc6f40cadb0 2024-03-09T08:52:48.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.413+0000 7fc6ebe00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f40cadb0 0x7fc6f414b660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00c430 tx=0x7fc6dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:48.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.414+0000 7fc6f0c00640 1 -- 172.21.15.67:0/899971387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0052d0 con 0x7fc6f40cadb0 2024-03-09T08:52:48.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.414+0000 7fc6f0c00640 1 -- 172.21.15.67:0/899971387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc005490 con 0x7fc6f40cadb0 2024-03-09T08:52:48.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.414+0000 7fc6f0c00640 1 -- 172.21.15.67:0/899971387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc007260 con 0x7fc6f40cadb0 2024-03-09T08:52:48.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.414+0000 7fc6f9c00640 1 -- 172.21.15.67:0/899971387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f40c6450 con 0x7fc6f40cadb0 2024-03-09T08:52:48.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.414+0000 7fc6f9c00640 1 -- 172.21.15.67:0/899971387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f40ca190 con 0x7fc6f40cadb0 2024-03-09T08:52:48.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.415+0000 7fc6f9c00640 1 -- 172.21.15.67:0/899971387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f4155080 con 0x7fc6f40cadb0 2024-03-09T08:52:48.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.417+0000 7fc6f0c00640 1 -- 172.21.15.67:0/899971387 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc6dc007400 con 0x7fc6f40cadb0 2024-03-09T08:52:48.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.418+0000 7fc6f0c00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc6d006b4d0 0x7fc6d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:48.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.418+0000 7fc6f0c00640 1 -- 172.21.15.67:0/899971387 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fc6dc089a30 con 0x7fc6f40cadb0 2024-03-09T08:52:48.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.419+0000 7fc6f3e00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc6d006b4d0 0x7fc6d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:48.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.420+0000 7fc6f3e00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc6d006b4d0 0x7fc6d006d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4036000 tx=0x7fc6e403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:48.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.423+0000 7fc6f0c00640 1 -- 172.21.15.67:0/899971387 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc6f4155080 con 0x7fc6f40cadb0 2024-03-09T08:52:48.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.664+0000 7fc6f9c00640 1 -- 172.21.15.67:0/899971387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6f404f3c0 con 0x7fc6f40cadb0 2024-03-09T08:52:48.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.665+0000 7fc6f0c00640 1 -- 172.21.15.67:0/899971387 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6247 (secure 0 0 0) 0x7fc6dc058570 con 0x7fc6f40cadb0 2024-03-09T08:52:48.661 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:48.661 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:17.736973+0000","modified":"2024-03-09T08:52:21.931108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T08:52:48.662 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 14 2024-03-09T08:52:48.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 -- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc6d006b4d0 msgr2=0x7fc6d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc6d006b4d0 0x7fc6d006d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4036000 tx=0x7fc6e403b040 comp rx=0 tx=0).stop 2024-03-09T08:52:48.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 -- 172.21.15.67:0/899971387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f40cadb0 msgr2=0x7fc6f414b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f40cadb0 0x7fc6f414b660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00c430 tx=0x7fc6dc00c8f0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 -- 172.21.15.67:0/899971387 shutdown_connections 2024-03-09T08:52:48.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc6d006b4d0 0x7fc6d006d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6f4153300 0x7fc6f414bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6f40cadb0 0x7fc6f414b660 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 --2- 172.21.15.67:0/899971387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6f40c5e40 0x7fc6f40c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 -- 172.21.15.67:0/899971387 >> 172.21.15.67:0/899971387 conn(0x7fc6f40732d0 msgr2=0x7fc6f40ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:48.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 -- 172.21.15.67:0/899971387 shutdown_connections 2024-03-09T08:52:48.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.668+0000 7fc6f9c00640 1 -- 172.21.15.67:0/899971387 wait complete. 2024-03-09T08:52:48.683 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:48.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.895+0000 7fe3e7e00640 1 Processor -- start 2024-03-09T08:52:48.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.895+0000 7fe3e7e00640 1 -- start start 2024-03-09T08:52:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.897+0000 7fe3e7e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe3e8153300 0x7fe3e80b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.897+0000 7fe3e7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e8157000 con 0x7fe3e80cadb0 2024-03-09T08:52:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.897+0000 7fe3e7e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e80ae430 con 0x7fe3e80c5e40 2024-03-09T08:52:48.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.897+0000 7fe3e7e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e80ae6f0 con 0x7fe3e8153300 2024-03-09T08:52:48.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.897+0000 7fe3e7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe3e80cadb0 0x7fe3e8160490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49020/0 (socket says 172.21.15.67:49020) 2024-03-09T08:52:48.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.898+0000 7fe3e7400640 1 -- 172.21.15.67:0/3048771267 learned_addr learned my addr 172.21.15.67:0/3048771267 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:48.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.898+0000 7fe3e6a00640 1 --2- 172.21.15.67:0/3048771267 >> v2:172.21.15.134:3300/0 conn(0x7fe3e8153300 0x7fe3e80b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:48.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.898+0000 7fe3e6000640 1 -- 172.21.15.67:0/3048771267 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631164573 0 0) 0x7fe3e8157000 con 0x7fe3e80cadb0 2024-03-09T08:52:48.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.898+0000 7fe3e6000640 1 -- 172.21.15.67:0/3048771267 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3cc003520 con 0x7fe3e80cadb0 2024-03-09T08:52:48.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.899+0000 7fe3e6000640 1 -- 172.21.15.67:0/3048771267 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445353161 0 0) 0x7fe3e80ae430 con 0x7fe3e80c5e40 2024-03-09T08:52:48.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.899+0000 7fe3e6000640 1 -- 172.21.15.67:0/3048771267 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3e8157000 con 0x7fe3e80c5e40 2024-03-09T08:52:48.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.899+0000 7fe3e6000640 1 -- 172.21.15.67:0/3048771267 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1820835206 0 0) 0x7fe3cc003520 con 0x7fe3e80cadb0 2024-03-09T08:52:48.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.899+0000 7fe3e6a00640 1 -- 172.21.15.67:0/3048771267 >> v1:172.21.15.134:6789/0 conn(0x7fe3e80c5e40 legacy=0x7fe3e8156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.899+0000 7fe3e6a00640 1 -- 172.21.15.67:0/3048771267 >> v1:172.21.15.67:6789/0 conn(0x7fe3e80cadb0 legacy=0x7fe3e8160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.899+0000 7fe3e6a00640 1 -- 172.21.15.67:0/3048771267 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e8155b10 con 0x7fe3e8153300 2024-03-09T08:52:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.900+0000 7fe3e6a00640 1 --2- 172.21.15.67:0/3048771267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e8153300 0x7fe3e80b0000 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80097a0 tx=0x7fe3d802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73209f4df8f4b4e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.901+0000 7fe3e6000640 1 -- 172.21.15.67:0/3048771267 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8038070 con 0x7fe3e8153300 2024-03-09T08:52:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.901+0000 7fe3e6000640 1 -- 172.21.15.67:0/3048771267 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8031500 con 0x7fe3e8153300 2024-03-09T08:52:48.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.901+0000 7fe3e6000640 1 -- 172.21.15.67:0/3048771267 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0001970 con 0x7fe3e8153300 2024-03-09T08:52:48.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.901+0000 7fe3e7e00640 1 -- 172.21.15.67:0/3048771267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e8153300 msgr2=0x7fe3e80b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.901+0000 7fe3e7e00640 1 --2- 172.21.15.67:0/3048771267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e8153300 0x7fe3e80b0000 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80097a0 tx=0x7fe3d802b820 comp rx=0 tx=0).stop 2024-03-09T08:52:48.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.901+0000 7fe3e7e00640 1 -- 172.21.15.67:0/3048771267 shutdown_connections 2024-03-09T08:52:48.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.901+0000 7fe3e7e00640 1 --2- 172.21.15.67:0/3048771267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e8153300 0x7fe3e80b0000 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.901+0000 7fe3e7e00640 1 -- 172.21.15.67:0/3048771267 >> 172.21.15.67:0/3048771267 conn(0x7fe3e80732d0 msgr2=0x7fe3e80ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.901+0000 7fe3e7e00640 1 -- 172.21.15.67:0/3048771267 shutdown_connections 2024-03-09T08:52:48.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.901+0000 7fe3e7e00640 1 -- 172.21.15.67:0/3048771267 wait complete. 2024-03-09T08:52:48.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.903+0000 7fe3e7e00640 1 Processor -- start 2024-03-09T08:52:48.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.903+0000 7fe3e7e00640 1 -- start start 2024-03-09T08:52:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.904+0000 7fe3e7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e80c5e40 0x7fe3e80c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.904+0000 7fe3e7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3e80cadb0 0x7fe3e806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.904+0000 7fe3e7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3e8153300 0x7fe3e814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.904+0000 7fe3e7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80c59e0 con 0x7fe3e80cadb0 2024-03-09T08:52:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.904+0000 7fe3e7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8159e90 con 0x7fe3e80c5e40 2024-03-09T08:52:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.904+0000 7fe3e7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80b0540 con 0x7fe3e8153300 2024-03-09T08:52:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.904+0000 7fe3de000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3e80cadb0 0x7fe3e806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.905+0000 7fe3de000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3e80cadb0 0x7fe3e806c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48482/0 (socket says 172.21.15.67:48482) 2024-03-09T08:52:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.905+0000 7fe3de000640 1 -- 172.21.15.67:0/2649304859 learned_addr learned my addr 172.21.15.67:0/2649304859 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.905+0000 7fe3e6a00640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e80c5e40 0x7fe3e80c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:48.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.905+0000 7fe3e7400640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3e8153300 0x7fe3e814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:48.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.905+0000 7fe3de000640 1 -- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3e8153300 msgr2=0x7fe3e814ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.905+0000 7fe3de000640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3e8153300 0x7fe3e814ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.905+0000 7fe3de000640 1 -- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e80c5e40 msgr2=0x7fe3e80c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:48.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.905+0000 7fe3de000640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e80c5e40 0x7fe3e80c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:48.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.905+0000 7fe3de000640 1 -- 172.21.15.67:0/2649304859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e806b7e0 con 0x7fe3e80cadb0 2024-03-09T08:52:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.905+0000 7fe3de000640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3e80cadb0 0x7fe3e806c5a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c550 tx=0x7fe3d000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.906+0000 7fe3dfe00640 1 -- 172.21.15.67:0/2649304859 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0017070 con 0x7fe3e80cadb0 2024-03-09T08:52:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.906+0000 7fe3dfe00640 1 -- 172.21.15.67:0/2649304859 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d00052d0 con 0x7fe3e80cadb0 2024-03-09T08:52:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.906+0000 7fe3e7e00640 1 -- 172.21.15.67:0/2649304859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e806bac0 con 0x7fe3e80cadb0 2024-03-09T08:52:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.906+0000 7fe3dfe00640 1 -- 172.21.15.67:0/2649304859 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0012870 con 0x7fe3e80cadb0 2024-03-09T08:52:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.906+0000 7fe3e7e00640 1 -- 172.21.15.67:0/2649304859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e806c080 con 0x7fe3e80cadb0 2024-03-09T08:52:48.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.907+0000 7fe3e7e00640 1 -- 172.21.15.67:0/2649304859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e806c280 con 0x7fe3e80cadb0 2024-03-09T08:52:48.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.908+0000 7fe3dfe00640 1 -- 172.21.15.67:0/2649304859 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe3d001a070 con 0x7fe3e80cadb0 2024-03-09T08:52:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.909+0000 7fe3dfe00640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe3c406b4d0 0x7fe3c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.909+0000 7fe3dfe00640 1 -- 172.21.15.67:0/2649304859 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fe3d0089910 con 0x7fe3e80cadb0 2024-03-09T08:52:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.910+0000 7fe3e6a00640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe3c406b4d0 0x7fe3c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:48.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.911+0000 7fe3e6a00640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe3c406b4d0 0x7fe3c406d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8035000 tx=0x7fe3d8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:48.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:48.914+0000 7fe3dfe00640 1 -- 172.21.15.67:0/2649304859 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe3e806c280 con 0x7fe3e80cadb0 2024-03-09T08:52:49.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.117+0000 7fe3e7e00640 1 -- 172.21.15.67:0/2649304859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3e8158930 con 0x7fe3e80cadb0 2024-03-09T08:52:49.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.120+0000 7fe3dfe00640 1 -- 172.21.15.67:0/2649304859 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20307 (secure 0 0 0) 0x7fe3d00584f0 con 0x7fe3e80cadb0 2024-03-09T08:52:49.116 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:49.116 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:30.565529+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 -- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe3c406b4d0 msgr2=0x7fe3c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe3c406b4d0 0x7fe3c406d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8035000 tx=0x7fe3d8008030 comp rx=0 tx=0).stop 2024-03-09T08:52:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 -- 172.21.15.67:0/2649304859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3e80cadb0 msgr2=0x7fe3e806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3e80cadb0 0x7fe3e806c5a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c550 tx=0x7fe3d000ca10 comp rx=0 tx=0).stop 2024-03-09T08:52:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 -- 172.21.15.67:0/2649304859 shutdown_connections 2024-03-09T08:52:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe3c406b4d0 0x7fe3c406d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3e8153300 0x7fe3e814ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3e80cadb0 0x7fe3e806c5a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 --2- 172.21.15.67:0/2649304859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e80c5e40 0x7fe3e80c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:49.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 -- 172.21.15.67:0/2649304859 >> 172.21.15.67:0/2649304859 conn(0x7fe3e80732d0 msgr2=0x7fe3e8151f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 -- 172.21.15.67:0/2649304859 shutdown_connections 2024-03-09T08:52:49.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.123+0000 7fe3e7e00640 1 -- 172.21.15.67:0/2649304859 wait complete. 2024-03-09T08:52:49.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T08:52:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.361+0000 7f5d7da00640 1 Processor -- start 2024-03-09T08:52:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.361+0000 7f5d7da00640 1 -- start start 2024-03-09T08:52:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.363+0000 7f5d7da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5d78153300 0x7f5d780b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.363+0000 7f5d7da00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d78157000 con 0x7f5d780cadb0 2024-03-09T08:52:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.363+0000 7f5d7da00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780ae430 con 0x7f5d780c5e40 2024-03-09T08:52:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.363+0000 7f5d7da00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d780ae6f0 con 0x7f5d78153300 2024-03-09T08:52:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.363+0000 7f5d7d000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5d780cadb0 0x7f5d78160490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49036/0 (socket says 172.21.15.67:49036) 2024-03-09T08:52:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.363+0000 7f5d7d000640 1 -- 172.21.15.67:0/2450358598 learned_addr learned my addr 172.21.15.67:0/2450358598 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:49.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.364+0000 7f5d77e00640 1 --2- 172.21.15.67:0/2450358598 >> v2:172.21.15.67:3300/0 conn(0x7f5d78153300 0x7f5d780b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:49.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.364+0000 7f5d77400640 1 -- 172.21.15.67:0/2450358598 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877808050 0 0) 0x7f5d78157000 con 0x7f5d780cadb0 2024-03-09T08:52:49.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.364+0000 7f5d77400640 1 -- 172.21.15.67:0/2450358598 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d5c003520 con 0x7f5d780cadb0 2024-03-09T08:52:49.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.364+0000 7f5d77e00640 1 -- 172.21.15.67:0/2450358598 >> v1:172.21.15.134:6789/0 conn(0x7f5d780c5e40 legacy=0x7f5d78156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:49.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.364+0000 7f5d77e00640 1 -- 172.21.15.67:0/2450358598 >> v1:172.21.15.67:6789/0 conn(0x7f5d780cadb0 legacy=0x7f5d78160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:49.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.364+0000 7f5d77e00640 1 -- 172.21.15.67:0/2450358598 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d78155b10 con 0x7f5d78153300 2024-03-09T08:52:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.365+0000 7f5d77e00640 1 --2- 172.21.15.67:0/2450358598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d78153300 0x7f5d780b0000 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009d90 tx=0x7f5d6802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5f701815f35db3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.365+0000 7f5d77400640 1 -- 172.21.15.67:0/2450358598 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68039070 con 0x7f5d78153300 2024-03-09T08:52:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.366+0000 7f5d77400640 1 -- 172.21.15.67:0/2450358598 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6802cd00 con 0x7f5d78153300 2024-03-09T08:52:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.366+0000 7f5d77400640 1 -- 172.21.15.67:0/2450358598 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68034640 con 0x7f5d78153300 2024-03-09T08:52:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.366+0000 7f5d7da00640 1 -- 172.21.15.67:0/2450358598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d78153300 msgr2=0x7f5d780b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.366+0000 7f5d7da00640 1 --2- 172.21.15.67:0/2450358598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d78153300 0x7f5d780b0000 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009d90 tx=0x7f5d6802bf50 comp rx=0 tx=0).stop 2024-03-09T08:52:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.366+0000 7f5d7da00640 1 -- 172.21.15.67:0/2450358598 shutdown_connections 2024-03-09T08:52:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.366+0000 7f5d7da00640 1 --2- 172.21.15.67:0/2450358598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d78153300 0x7f5d780b0000 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.366+0000 7f5d7da00640 1 -- 172.21.15.67:0/2450358598 >> 172.21.15.67:0/2450358598 conn(0x7f5d780732d0 msgr2=0x7f5d780ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.366+0000 7f5d7da00640 1 -- 172.21.15.67:0/2450358598 shutdown_connections 2024-03-09T08:52:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.366+0000 7f5d7da00640 1 -- 172.21.15.67:0/2450358598 wait complete. 2024-03-09T08:52:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.369+0000 7f5d7da00640 1 Processor -- start 2024-03-09T08:52:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.369+0000 7f5d7da00640 1 -- start start 2024-03-09T08:52:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d7da00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d780c5e40 0x7f5d78157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d7da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d780cadb0 0x7f5d781eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d7da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d78153300 0x7f5d781ef230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d7da00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780c59e0 con 0x7f5d780cadb0 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d7da00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78159e90 con 0x7f5d78153300 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d7da00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780b0540 con 0x7f5d780c5e40 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d6f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d780cadb0 0x7f5d781eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d6f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d780cadb0 0x7f5d781eecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48506/0 (socket says 172.21.15.67:48506) 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d6f400640 1 -- 172.21.15.67:0/3534591977 learned_addr learned my addr 172.21.15.67:0/3534591977 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d77e00640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d780c5e40 0x7f5d78157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d7d000640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d78153300 0x7f5d781ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d6f400640 1 -- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d780c5e40 msgr2=0x7f5d78157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d6f400640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d780c5e40 0x7f5d78157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d6f400640 1 -- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d78153300 msgr2=0x7f5d781ef230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d6f400640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d78153300 0x7f5d781ef230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d6f400640 1 -- 172.21.15.67:0/3534591977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d781ef770 con 0x7f5d780cadb0 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.371+0000 7f5d7d000640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d78153300 0x7f5d781ef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.372+0000 7f5d6f400640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d780cadb0 0x7f5d781eecf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c450 tx=0x7f5d6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.372+0000 7f5d74c00640 1 -- 172.21.15.67:0/3534591977 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d600052d0 con 0x7f5d780cadb0 2024-03-09T08:52:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.372+0000 7f5d74c00640 1 -- 172.21.15.67:0/3534591977 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d60005470 con 0x7f5d780cadb0 2024-03-09T08:52:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.372+0000 7f5d7da00640 1 -- 172.21.15.67:0/3534591977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7814bee0 con 0x7f5d780cadb0 2024-03-09T08:52:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.372+0000 7f5d74c00640 1 -- 172.21.15.67:0/3534591977 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60012870 con 0x7f5d780cadb0 2024-03-09T08:52:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.372+0000 7f5d7da00640 1 -- 172.21.15.67:0/3534591977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d780c67a0 con 0x7f5d780cadb0 2024-03-09T08:52:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.374+0000 7f5d7da00640 1 -- 172.21.15.67:0/3534591977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7804f3c0 con 0x7f5d780cadb0 2024-03-09T08:52:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.374+0000 7f5d74c00640 1 -- 172.21.15.67:0/3534591977 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5d60012a10 con 0x7f5d780cadb0 2024-03-09T08:52:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.375+0000 7f5d74c00640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d5406b500 0x7f5d5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.375+0000 7f5d74c00640 1 -- 172.21.15.67:0/3534591977 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f5d6008a0a0 con 0x7f5d780cadb0 2024-03-09T08:52:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.376+0000 7f5d77e00640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d5406b500 0x7f5d5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.377+0000 7f5d77e00640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d5406b500 0x7f5d5406d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d68036000 tx=0x7f5d68030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.380+0000 7f5d74c00640 1 -- 172.21.15.67:0/3534591977 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5d781ef770 con 0x7f5d780cadb0 2024-03-09T08:52:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.618+0000 7f5d7da00640 1 -- 172.21.15.67:0/3534591977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d780cab90 con 0x7f5d780cadb0 2024-03-09T08:52:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.619+0000 7f5d74c00640 1 -- 172.21.15.67:0/3534591977 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6247 (secure 0 0 0) 0x7f5d60058c80 con 0x7f5d780cadb0 2024-03-09T08:52:49.615 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:49.615 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:17.736973+0000","modified":"2024-03-09T08:52:21.931108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T08:52:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 14 2024-03-09T08:52:49.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.621+0000 7f5d7da00640 1 -- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d5406b500 msgr2=0x7f5d5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.621+0000 7f5d7da00640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d5406b500 0x7f5d5406d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d68036000 tx=0x7f5d68030000 comp rx=0 tx=0).stop 2024-03-09T08:52:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.621+0000 7f5d7da00640 1 -- 172.21.15.67:0/3534591977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d780cadb0 msgr2=0x7f5d781eecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.621+0000 7f5d7da00640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d780cadb0 0x7f5d781eecf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c450 tx=0x7f5d6000c910 comp rx=0 tx=0).stop 2024-03-09T08:52:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.621+0000 7f5d7da00640 1 -- 172.21.15.67:0/3534591977 shutdown_connections 2024-03-09T08:52:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.621+0000 7f5d7da00640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d5406b500 0x7f5d5406d9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.621+0000 7f5d7da00640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d78153300 0x7f5d781ef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.621+0000 7f5d7da00640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d780cadb0 0x7f5d781eecf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.621+0000 7f5d7da00640 1 --2- 172.21.15.67:0/3534591977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d780c5e40 0x7f5d78157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.621+0000 7f5d7da00640 1 -- 172.21.15.67:0/3534591977 >> 172.21.15.67:0/3534591977 conn(0x7f5d780732d0 msgr2=0x7f5d781503d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.622+0000 7f5d7da00640 1 -- 172.21.15.67:0/3534591977 shutdown_connections 2024-03-09T08:52:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:49.622+0000 7f5d7da00640 1 -- 172.21.15.67:0/3534591977 wait complete. 2024-03-09T08:52:50.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-09T08:52:17.736973+0000', 'modified': '2024-03-09T08:52:21.931108+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.134:6835/340976284', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 340976284}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 340976284}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'a', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4282, 'qdb_cluster': [4282]} 2024-03-09T08:52:50.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-09T08:52:50.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-09T08:52:50.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.297+0000 7f7d12400640 1 Processor -- start 2024-03-09T08:52:50.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.297+0000 7f7d12400640 1 -- start start 2024-03-09T08:52:50.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.299+0000 7f7d12400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d0c0ac930 0x7f7d0c0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.299+0000 7f7d12400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0c0c2780 con 0x7f7d0c0c30a0 2024-03-09T08:52:50.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.299+0000 7f7d12400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0c0708c0 con 0x7f7d0c151360 2024-03-09T08:52:50.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.299+0000 7f7d12400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d0c070670 con 0x7f7d0c0ac930 2024-03-09T08:52:50.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.299+0000 7f7d0be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7d0c0c30a0 0x7f7d0c150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49050/0 (socket says 172.21.15.67:49050) 2024-03-09T08:52:50.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.299+0000 7f7d0be00640 1 -- 172.21.15.67:0/328430763 learned_addr learned my addr 172.21.15.67:0/328430763 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:50.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.299+0000 7f7d11000640 1 --2- 172.21.15.67:0/328430763 >> v2:172.21.15.67:3300/0 conn(0x7f7d0c0ac930 0x7f7d0c0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.300+0000 7f7d0b400640 1 -- 172.21.15.67:0/328430763 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258521490 0 0) 0x7f7d0c0c2780 con 0x7f7d0c0c30a0 2024-03-09T08:52:50.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.300+0000 7f7d0b400640 1 -- 172.21.15.67:0/328430763 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cf4003520 con 0x7f7d0c0c30a0 2024-03-09T08:52:50.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.300+0000 7f7d0b400640 1 -- 172.21.15.67:0/328430763 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609685346 0 0) 0x7f7d0c0708c0 con 0x7f7d0c151360 2024-03-09T08:52:50.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.300+0000 7f7d0b400640 1 -- 172.21.15.67:0/328430763 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d0c0c2780 con 0x7f7d0c151360 2024-03-09T08:52:50.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.300+0000 7f7d0b400640 1 -- 172.21.15.67:0/328430763 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 638499121 0 0) 0x7f7cf4003520 con 0x7f7d0c0c30a0 2024-03-09T08:52:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.300+0000 7f7d11000640 1 -- 172.21.15.67:0/328430763 >> v1:172.21.15.134:6789/0 conn(0x7f7d0c151360 legacy=0x7f7d0c1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.300+0000 7f7d11000640 1 -- 172.21.15.67:0/328430763 >> v1:172.21.15.67:6789/0 conn(0x7f7d0c0c30a0 legacy=0x7f7d0c150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.300+0000 7f7d11000640 1 -- 172.21.15.67:0/328430763 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c070ba0 con 0x7f7d0c0ac930 2024-03-09T08:52:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.301+0000 7f7d11000640 1 --2- 172.21.15.67:0/328430763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d0c0ac930 0x7f7d0c0c3660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d000097a0 tx=0x7f7d0002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797a2f4423837c27 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.301+0000 7f7d0b400640 1 -- 172.21.15.67:0/328430763 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00038070 con 0x7f7d0c0ac930 2024-03-09T08:52:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.302+0000 7f7d0b400640 1 -- 172.21.15.67:0/328430763 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d00031500 con 0x7f7d0c0ac930 2024-03-09T08:52:50.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.302+0000 7f7d0b400640 1 -- 172.21.15.67:0/328430763 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8001670 con 0x7f7d0c0ac930 2024-03-09T08:52:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.302+0000 7f7d12400640 1 -- 172.21.15.67:0/328430763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d0c0ac930 msgr2=0x7f7d0c0c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.302+0000 7f7d12400640 1 --2- 172.21.15.67:0/328430763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d0c0ac930 0x7f7d0c0c3660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d000097a0 tx=0x7f7d0002b820 comp rx=0 tx=0).stop 2024-03-09T08:52:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.302+0000 7f7d12400640 1 -- 172.21.15.67:0/328430763 shutdown_connections 2024-03-09T08:52:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.302+0000 7f7d12400640 1 --2- 172.21.15.67:0/328430763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d0c0ac930 0x7f7d0c0c3660 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.302+0000 7f7d12400640 1 -- 172.21.15.67:0/328430763 >> 172.21.15.67:0/328430763 conn(0x7f7d0c0c7220 msgr2=0x7f7d0c0c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.302+0000 7f7d12400640 1 -- 172.21.15.67:0/328430763 shutdown_connections 2024-03-09T08:52:50.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.302+0000 7f7d12400640 1 -- 172.21.15.67:0/328430763 wait complete. 2024-03-09T08:52:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.304+0000 7f7d12400640 1 Processor -- start 2024-03-09T08:52:50.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.304+0000 7f7d12400640 1 -- start start 2024-03-09T08:52:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.305+0000 7f7d12400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d0c0ac930 0x7f7d0c152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.305+0000 7f7d12400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d0c0c30a0 0x7f7d0c1e8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.305+0000 7f7d12400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c151360 0x7f7d0c1e8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.305+0000 7f7d12400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c0ace90 con 0x7f7d0c0ac930 2024-03-09T08:52:50.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.305+0000 7f7d12400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c0acd10 con 0x7f7d0c151360 2024-03-09T08:52:50.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.305+0000 7f7d12400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c1540f0 con 0x7f7d0c0c30a0 2024-03-09T08:52:50.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.305+0000 7f7d11000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d0c0ac930 0x7f7d0c152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.305+0000 7f7d11000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d0c0ac930 0x7f7d0c152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48514/0 (socket says 172.21.15.67:48514) 2024-03-09T08:52:50.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.305+0000 7f7d11000640 1 -- 172.21.15.67:0/3687978868 learned_addr learned my addr 172.21.15.67:0/3687978868 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:50.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.306+0000 7f7d0be00640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d0c0c30a0 0x7f7d0c1e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.306+0000 7f7d11a00640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c151360 0x7f7d0c1e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.306+0000 7f7d11000640 1 -- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d0c0c30a0 msgr2=0x7f7d0c1e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.306+0000 7f7d11000640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d0c0c30a0 0x7f7d0c1e8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.306+0000 7f7d11000640 1 -- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c151360 msgr2=0x7f7d0c1e8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.306+0000 7f7d11000640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c151360 0x7f7d0c1e8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.306+0000 7f7d11000640 1 -- 172.21.15.67:0/3687978868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c156740 con 0x7f7d0c0ac930 2024-03-09T08:52:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.306+0000 7f7d11a00640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c151360 0x7f7d0c1e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.306+0000 7f7d11000640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d0c0ac930 0x7f7d0c152f70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d0002b860 tx=0x7f7d000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.307+0000 7f7d08c00640 1 -- 172.21.15.67:0/3687978868 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00038050 con 0x7f7d0c0ac930 2024-03-09T08:52:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.307+0000 7f7d08c00640 1 -- 172.21.15.67:0/3687978868 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d00044070 con 0x7f7d0c0ac930 2024-03-09T08:52:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.307+0000 7f7d08c00640 1 -- 172.21.15.67:0/3687978868 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00003a30 con 0x7f7d0c0ac930 2024-03-09T08:52:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.307+0000 7f7d12400640 1 -- 172.21.15.67:0/3687978868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0c1ebd40 con 0x7f7d0c0ac930 2024-03-09T08:52:50.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.307+0000 7f7d12400640 1 -- 172.21.15.67:0/3687978868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0c14b4d0 con 0x7f7d0c0ac930 2024-03-09T08:52:50.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.310+0000 7f7d08c00640 1 -- 172.21.15.67:0/3687978868 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7d00003bd0 con 0x7f7d0c0ac930 2024-03-09T08:52:50.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.311+0000 7f7d08c00640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7cec06b4d0 0x7f7cec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.311+0000 7f7d08c00640 1 -- 172.21.15.67:0/3687978868 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f7d000ac310 con 0x7f7d0c0ac930 2024-03-09T08:52:50.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.311+0000 7f7d16779740 1 Processor -- start 2024-03-09T08:52:50.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.311+0000 7f7d16779740 1 -- start start 2024-03-09T08:52:50.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.311+0000 7f7d0be00640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7cec06b4d0 0x7f7cec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.312+0000 7f7d16779740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f1ef20 0x561568f20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.312+0000 7f7d16779740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568ee52c0 0x561568f1f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.312+0000 7f7d16779740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568ee4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.312+0000 7f7d16779740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561568e486f0 con 0x561568ee52c0 2024-03-09T08:52:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.312+0000 7f7d16779740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561568df19e0 con 0x561568f22090 2024-03-09T08:52:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.312+0000 7f7d16779740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x561568e44790 con 0x561568f1ef20 2024-03-09T08:52:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.312+0000 7f7d0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568ee52c0 0x561568f1f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568ee52c0 0x561568f1f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48518/0 (socket says 172.21.15.67:48518) 2024-03-09T08:52:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d0be00640 1 -- 172.21.15.67:0/893343674 learned_addr learned my addr 172.21.15.67:0/893343674 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d11000640 1 --2- 172.21.15.67:0/893343674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f1ef20 0x561568f20090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d11a00640 1 --2- 172.21.15.67:0/893343674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568ee4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d0be00640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7cec06b4d0 0x7f7cec06d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8001380 tx=0x7f7cf8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d0be00640 1 -- 172.21.15.67:0/893343674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f1ef20 msgr2=0x561568f20090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d0be00640 1 --2- 172.21.15.67:0/893343674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f1ef20 0x561568f20090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d0be00640 1 -- 172.21.15.67:0/893343674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 msgr2=0x561568ee4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d0be00640 1 --2- 172.21.15.67:0/893343674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568ee4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d0be00640 1 -- 172.21.15.67:0/893343674 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561568de5720 con 0x561568ee52c0 2024-03-09T08:52:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d11a00640 1 --2- 172.21.15.67:0/893343674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568ee4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.313+0000 7f7d0be00640 1 --2- 172.21.15.67:0/893343674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568ee52c0 0x561568f1f300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800dc20 tx=0x7f7cf8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebd1f8189cdc47c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.314+0000 7f7cfd600640 1 -- 172.21.15.67:0/893343674 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8024070 con 0x561568ee52c0 2024-03-09T08:52:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.314+0000 7f7cfd600640 1 -- 172.21.15.67:0/893343674 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8017030 con 0x561568ee52c0 2024-03-09T08:52:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.314+0000 7f7cfd600640 1 -- 172.21.15.67:0/893343674 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8019050 con 0x561568ee52c0 2024-03-09T08:52:50.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.314+0000 7f7d16779740 1 -- 172.21.15.67:0/893343674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568ee52c0 msgr2=0x561568f1f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.314+0000 7f7d16779740 1 --2- 172.21.15.67:0/893343674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568ee52c0 0x561568f1f300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800dc20 tx=0x7f7cf8010000 comp rx=0 tx=0).stop 2024-03-09T08:52:50.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.314+0000 7f7d16779740 1 -- 172.21.15.67:0/893343674 shutdown_connections 2024-03-09T08:52:50.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.314+0000 7f7d16779740 1 --2- 172.21.15.67:0/893343674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568ee4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.314+0000 7f7d16779740 1 --2- 172.21.15.67:0/893343674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568ee52c0 0x561568f1f300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.314+0000 7f7d16779740 1 --2- 172.21.15.67:0/893343674 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f1ef20 0x561568f20090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.314+0000 7f7d16779740 1 -- 172.21.15.67:0/893343674 >> 172.21.15.67:0/893343674 conn(0x561568ee0900 msgr2=0x561568f05310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:50.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.314+0000 7f7d16779740 1 -- 172.21.15.67:0/893343674 shutdown_connections 2024-03-09T08:52:50.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.315+0000 7f7d16779740 1 -- 172.21.15.67:0/893343674 wait complete. 2024-03-09T08:52:50.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.315+0000 7f7d16779740 1 -- start start 2024-03-09T08:52:50.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.315+0000 7f7d16779740 1 Processor -- start 2024-03-09T08:52:50.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.316+0000 7f7d16779740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f1ef20 0x561568f15fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.316+0000 7f7d16779740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f22090 0x561568f172d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.316+0000 7f7d16779740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f02320 0x561568f01800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.316+0000 7f7d16779740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561568f21c00 con 0x561568f22090 2024-03-09T08:52:50.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.316+0000 7f7d16779740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561568e04e20 con 0x561568f1ef20 2024-03-09T08:52:50.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.316+0000 7f7d16779740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x561568f216c0 con 0x561568f02320 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.316+0000 7f7d16779740 10 client.0 _ll_register_callbacks cb 0x561568e3f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7d0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f22090 0x561568f172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7d0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f22090 0x561568f172d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48522/0 (socket says 172.21.15.67:48522) 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7d0be00640 1 -- 172.21.15.67:0/766326889 learned_addr learned my addr 172.21.15.67:0/766326889 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7d11000640 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f1ef20 0x561568f15fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7d11a00640 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f02320 0x561568f01800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7d0be00640 1 -- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f02320 msgr2=0x561568f01800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7d0be00640 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f02320 0x561568f01800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7d0be00640 1 -- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f1ef20 msgr2=0x561568f15fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7d0be00640 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f1ef20 0x561568f15fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7d0be00640 1 -- 172.21.15.67:0/766326889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561568b5d2d0 con 0x561568f22090 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7d0be00640 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f22090 0x561568f172d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8001320 tx=0x7f7cf8001290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1049b2f06c0ab5a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:50.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7cf2000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.317+0000 7f7cf2000640 1 -- 172.21.15.67:0/766326889 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf802d020 con 0x561568f22090 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7cf2000640 1 -- 172.21.15.67:0/766326889 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f7cf801eba0 con 0x561568f22090 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7d16779740 1 -- 172.21.15.67:0/766326889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cf8017030 con 0x561568f22090 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7cf2000640 1 -- 172.21.15.67:0/766326889 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cf8037050 con 0x561568f22090 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7d16779740 10 client.4788 fetch_fsmap learned FSMap version 14 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7d16779740 1 -- 172.21.15.67:0/766326889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x561568c276b0 con 0x561568f22090 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7cf2000640 1 -- 172.21.15.67:0/766326889 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8017030 con 0x561568f22090 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7cf2000640 1 -- 172.21.15.67:0/766326889 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 14) v1 ==== 3918+0+0 (secure 0 0 0) 0x7f7cf803a020 con 0x561568f22090 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7d16779740 10 client.4788 fetch_fsmap finished waiting for FSMap version 14 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7d16779740 10 client.4788 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7d16779740 20 client.4788 populate_metadata read hostname 'smithi067' 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7d16779740 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568f30d40 0x561568f3f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7d16779740 4 client.4788 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.318+0000 7f7d16779740 1 -- 172.21.15.67:0/766326889 --> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561568e55c30 con 0x561568f30d40 2024-03-09T08:52:50.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.319+0000 7f7d11000640 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568f30d40 0x561568f3f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.319+0000 7f7d11000640 1 -- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568f30d40 msgr2=0x561568f3f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-03-09T08:52:50.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.319+0000 7f7d11000640 1 -- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568f30d40 msgr2=0x561568f3f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:52:50.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.319+0000 7f7d11000640 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568f30d40 0x561568f3f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:52:50.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.319+0000 7f7d11000640 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568f30d40 0x561568f3f150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:52:50.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.520+0000 7f7d11000640 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568f30d40 0x561568f3f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.521+0000 7f7cf2000640 10 client.4788 ms_handle_connect on v2:172.21.15.134:6834/340976284 2024-03-09T08:52:50.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.521+0000 7f7d11000640 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568f30d40 0x561568f3f150 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6924b2942bffd3e server_cookie=8fc4c484bf72fad0 in_seq=0 out_seq=0 2024-03-09T08:52:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.524+0000 7f7cf2000640 1 -- 172.21.15.67:0/766326889 <== mds.0 v2:172.21.15.134:6834/340976284 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561568e55c30 con 0x561568f30d40 2024-03-09T08:52:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.524+0000 7f7cf2000640 10 client.4788 handle_command_reply: tid=0 2024-03-09T08:52:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.524+0000 7f7d16779740 1 client.4788 shutdown 2024-03-09T08:52:50.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.525+0000 7f7d16779740 1 -- 172.21.15.67:0/766326889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f22090 msgr2=0x561568f172d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.525+0000 7f7d16779740 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f22090 0x561568f172d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8001320 tx=0x7f7cf8001290 comp rx=0 tx=0).stop 2024-03-09T08:52:50.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.525+0000 7f7d16779740 1 -- 172.21.15.67:0/766326889 shutdown_connections 2024-03-09T08:52:50.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.525+0000 7f7d16779740 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568f30d40 0x561568f3f150 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.525+0000 7f7d16779740 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f02320 0x561568f01800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.525+0000 7f7d16779740 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f22090 0x561568f172d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.525+0000 7f7d16779740 1 --2- 172.21.15.67:0/766326889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f1ef20 0x561568f15fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.525+0000 7f7cf2000640 0 client.4788 ms_handle_reset on v2:172.21.15.134:6834/340976284 2024-03-09T08:52:50.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.525+0000 7f7d16779740 1 -- 172.21.15.67:0/766326889 >> 172.21.15.67:0/766326889 conn(0x561568ee52c0 msgr2=0x561568edd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:50.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.525+0000 7f7d16779740 1 -- 172.21.15.67:0/766326889 shutdown_connections 2024-03-09T08:52:50.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.525+0000 7f7d16779740 1 -- 172.21.15.67:0/766326889 wait complete. 2024-03-09T08:52:50.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.526+0000 7f7d16779740 20 client.4788 trim_cache size 0 max 16384 2024-03-09T08:52:50.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.561+0000 7f7d16779740 1 Processor -- start 2024-03-09T08:52:50.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.561+0000 7f7d16779740 1 -- start start 2024-03-09T08:52:50.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.562+0000 7f7d16779740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f02320 0x561568f20620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.562+0000 7f7d16779740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f1ef20 0x561568edf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.563+0000 7f7d16779740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568edf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.563+0000 7f7d16779740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c1fae60 con 0x561568f02320 2024-03-09T08:52:50.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.563+0000 7f7d16779740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c0c6af0 con 0x561568f22090 2024-03-09T08:52:50.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.563+0000 7f7d16779740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c1f90b0 con 0x561568f1ef20 2024-03-09T08:52:50.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.563+0000 7f7d11000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f02320 0x561568f20620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.563+0000 7f7d11000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f02320 0x561568f20620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48536/0 (socket says 172.21.15.67:48536) 2024-03-09T08:52:50.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.563+0000 7f7d11000640 1 -- 172.21.15.67:0/2820739621 learned_addr learned my addr 172.21.15.67:0/2820739621 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:50.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.563+0000 7f7d11a00640 1 --2- 172.21.15.67:0/2820739621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568edf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.563+0000 7f7d0be00640 1 --2- 172.21.15.67:0/2820739621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f1ef20 0x561568edf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.564+0000 7f7d11000640 1 -- 172.21.15.67:0/2820739621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f1ef20 msgr2=0x561568edf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:50.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.564+0000 7f7d11000640 1 --2- 172.21.15.67:0/2820739621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f1ef20 0x561568edf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.564+0000 7f7d11000640 1 -- 172.21.15.67:0/2820739621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 msgr2=0x561568edf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:50.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.564+0000 7f7d11000640 1 --2- 172.21.15.67:0/2820739621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568edf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.564+0000 7f7d11000640 1 -- 172.21.15.67:0/2820739621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561568f02700 con 0x561568f02320 2024-03-09T08:52:50.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.564+0000 7f7d11000640 1 --2- 172.21.15.67:0/2820739621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f02320 0x561568f20620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d0007a6e0 tx=0x7f7d00009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b569f1b1a88162 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:50.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.564+0000 7f7cfcc00640 1 -- 172.21.15.67:0/2820739621 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000a3e60 con 0x561568f02320 2024-03-09T08:52:50.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.564+0000 7f7cfcc00640 1 -- 172.21.15.67:0/2820739621 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d00039040 con 0x561568f02320 2024-03-09T08:52:50.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.564+0000 7f7cfcc00640 1 -- 172.21.15.67:0/2820739621 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0007ae50 con 0x561568f02320 2024-03-09T08:52:50.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.565+0000 7f7d16779740 1 -- 172.21.15.67:0/2820739621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f02320 msgr2=0x561568f20620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.565+0000 7f7d16779740 1 --2- 172.21.15.67:0/2820739621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f02320 0x561568f20620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d0007a6e0 tx=0x7f7d00009190 comp rx=0 tx=0).stop 2024-03-09T08:52:50.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.565+0000 7f7d16779740 1 -- 172.21.15.67:0/2820739621 shutdown_connections 2024-03-09T08:52:50.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.565+0000 7f7d16779740 1 --2- 172.21.15.67:0/2820739621 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568edf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.565+0000 7f7d16779740 1 --2- 172.21.15.67:0/2820739621 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f1ef20 0x561568edf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.565+0000 7f7d16779740 1 --2- 172.21.15.67:0/2820739621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f02320 0x561568f20620 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.565+0000 7f7d16779740 1 -- 172.21.15.67:0/2820739621 >> 172.21.15.67:0/2820739621 conn(0x561568ee52c0 msgr2=0x561568ed9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:50.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.566+0000 7f7d16779740 1 -- 172.21.15.67:0/2820739621 shutdown_connections 2024-03-09T08:52:50.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.566+0000 7f7d16779740 1 -- 172.21.15.67:0/2820739621 wait complete. 2024-03-09T08:52:50.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.566+0000 7f7d16779740 1 -- start start 2024-03-09T08:52:50.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.567+0000 7f7d16779740 1 Processor -- start 2024-03-09T08:52:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d16779740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f02320 0x561568f2d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d16779740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f1ef20 0x561568f939f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d16779740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568f8d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d16779740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561568f02990 con 0x561568f1ef20 2024-03-09T08:52:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d16779740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561568f19930 con 0x561568f22090 2024-03-09T08:52:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d16779740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x561568f307d0 con 0x561568f02320 2024-03-09T08:52:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d16779740 10 client.0 _ll_register_callbacks cb 0x7f7cf8022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T08:52:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f1ef20 0x561568f939f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f1ef20 0x561568f939f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48548/0 (socket says 172.21.15.67:48548) 2024-03-09T08:52:50.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d0be00640 1 -- 172.21.15.67:0/4052698827 learned_addr learned my addr 172.21.15.67:0/4052698827 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:50.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d11a00640 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568f8d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.568+0000 7f7d11000640 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f02320 0x561568f2d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.569+0000 7f7d0be00640 1 -- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f02320 msgr2=0x561568f2d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:50.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.569+0000 7f7d0be00640 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f02320 0x561568f2d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.569+0000 7f7d0be00640 1 -- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 msgr2=0x561568f8d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:50.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.569+0000 7f7d0be00640 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568f8d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.569+0000 7f7d0be00640 1 -- 172.21.15.67:0/4052698827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561568f16c60 con 0x561568f1ef20 2024-03-09T08:52:50.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.569+0000 7f7d11000640 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f02320 0x561568f2d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.569+0000 7f7d0be00640 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f1ef20 0x561568f939f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8001320 tx=0x7f7cf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad19fe8c57047c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.569+0000 7f7cf2000640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-09T08:52:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.570+0000 7f7cf2000640 1 -- 172.21.15.67:0/4052698827 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf802d070 con 0x561568f1ef20 2024-03-09T08:52:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.570+0000 7f7cf2000640 1 -- 172.21.15.67:0/4052698827 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f7cf802be00 con 0x561568f1ef20 2024-03-09T08:52:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.570+0000 7f7d16779740 1 -- 172.21.15.67:0/4052698827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d00039040 con 0x561568f1ef20 2024-03-09T08:52:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.570+0000 7f7cf2000640 1 -- 172.21.15.67:0/4052698827 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cf8019040 con 0x561568f1ef20 2024-03-09T08:52:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.570+0000 7f7d16779740 10 client.4794 fetch_fsmap learned FSMap version 14 2024-03-09T08:52:50.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.570+0000 7f7d16779740 1 -- 172.21.15.67:0/4052698827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x561568f93f30 con 0x561568f1ef20 2024-03-09T08:52:50.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.570+0000 7f7cf2000640 1 -- 172.21.15.67:0/4052698827 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d00039040 con 0x561568f1ef20 2024-03-09T08:52:50.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.571+0000 7f7cf2000640 1 -- 172.21.15.67:0/4052698827 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 14) v1 ==== 3918+0+0 (secure 0 0 0) 0x7f7cf803a040 con 0x561568f1ef20 2024-03-09T08:52:50.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.571+0000 7f7d16779740 10 client.4794 fetch_fsmap finished waiting for FSMap version 14 2024-03-09T08:52:50.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.571+0000 7f7d16779740 10 client.4794 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-09T08:52:50.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.571+0000 7f7d16779740 20 client.4794 populate_metadata read hostname 'smithi067' 2024-03-09T08:52:50.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.571+0000 7f7d16779740 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568efde90 0x561568efe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.571+0000 7f7d16779740 4 client.4794 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-09T08:52:50.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.571+0000 7f7d16779740 1 -- 172.21.15.67:0/4052698827 --> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561568ee1060 con 0x561568efde90 2024-03-09T08:52:50.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.571+0000 7f7d11000640 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568efde90 0x561568efe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.572+0000 7f7d11000640 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568efde90 0x561568efe270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f190ff6d091a03 server_cookie=3bafd8223c6ee1ff in_seq=0 out_seq=0 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.572+0000 7f7cf2000640 10 client.4794 ms_handle_connect on v2:172.21.15.134:6834/340976284 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7cf2000640 1 -- 172.21.15.67:0/4052698827 <== mds.0 v2:172.21.15.134:6834/340976284 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561568ee1060 con 0x561568efde90 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7cf2000640 10 client.4794 handle_command_reply: tid=0 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7d16779740 1 client.4794 shutdown 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7d16779740 1 -- 172.21.15.67:0/4052698827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f1ef20 msgr2=0x561568f939f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7d16779740 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f1ef20 0x561568f939f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8001320 tx=0x7f7cf80057f0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7d16779740 1 -- 172.21.15.67:0/4052698827 shutdown_connections 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7d16779740 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:6834/340976284,v1:172.21.15.134:6835/340976284] conn(0x561568efde90 0x561568efe270 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7d16779740 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561568f22090 0x561568f8d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7d16779740 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561568f1ef20 0x561568f939f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7d16779740 1 --2- 172.21.15.67:0/4052698827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x561568f02320 0x561568f2d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7cf2000640 0 client.4794 ms_handle_reset on v2:172.21.15.134:6834/340976284 2024-03-09T08:52:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7d16779740 1 -- 172.21.15.67:0/4052698827 >> 172.21.15.67:0/4052698827 conn(0x561568ee52c0 msgr2=0x561568f18500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7d16779740 1 -- 172.21.15.67:0/4052698827 shutdown_connections 2024-03-09T08:52:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.573+0000 7f7d16779740 1 -- 172.21.15.67:0/4052698827 wait complete. 2024-03-09T08:52:50.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.574+0000 7f7d16779740 20 client.4794 trim_cache size 0 max 16384 2024-03-09T08:52:50.571 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4282, 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "a", 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 32.305888611999997, 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 8, 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 24, 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 20, 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 37.555256090999997 2024-03-09T08:52:50.572 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:52:50.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.574+0000 7f7d12400640 1 -- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7cec06b4d0 msgr2=0x7f7cec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.574+0000 7f7d12400640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7cec06b4d0 0x7f7cec06d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8001380 tx=0x7f7cf8009040 comp rx=0 tx=0).stop 2024-03-09T08:52:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.574+0000 7f7d12400640 1 -- 172.21.15.67:0/3687978868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d0c0ac930 msgr2=0x7f7d0c152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.574+0000 7f7d12400640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d0c0ac930 0x7f7d0c152f70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d0002b860 tx=0x7f7d000046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.575+0000 7f7d12400640 1 -- 172.21.15.67:0/3687978868 shutdown_connections 2024-03-09T08:52:50.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.575+0000 7f7d12400640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7cec06b4d0 0x7f7cec06d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.575+0000 7f7d12400640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d0c151360 0x7f7d0c1e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.575+0000 7f7d12400640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d0c0c30a0 0x7f7d0c1e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.575+0000 7f7d12400640 1 --2- 172.21.15.67:0/3687978868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d0c0ac930 0x7f7d0c152f70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.575+0000 7f7d12400640 1 -- 172.21.15.67:0/3687978868 >> 172.21.15.67:0/3687978868 conn(0x7f7d0c0c7220 msgr2=0x7f7d0c14eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:50.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.575+0000 7f7d12400640 1 -- 172.21.15.67:0/3687978868 shutdown_connections 2024-03-09T08:52:50.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.575+0000 7f7d12400640 1 -- 172.21.15.67:0/3687978868 wait complete. 2024-03-09T08:52:50.593 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:50.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.809+0000 7f4500a00640 1 Processor -- start 2024-03-09T08:52:50.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.809+0000 7f4500a00640 1 -- start start 2024-03-09T08:52:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.811+0000 7f4500a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44fc14dbf0 0x7f44fc0c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.811+0000 7f4500a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f44fc0c5aa0 0x7f44fc153c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.811+0000 7f4500a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44fc06f9e0 0x7f44fc154140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.811+0000 7f4500a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc157b10 con 0x7f44fc06f9e0 2024-03-09T08:52:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.811+0000 7f4500a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc0c36a0 con 0x7f44fc0c5aa0 2024-03-09T08:52:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.811+0000 7f4500a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0c3980 con 0x7f44fc14dbf0 2024-03-09T08:52:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.811+0000 7f44fb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44fc06f9e0 0x7f44fc154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.811+0000 7f44fb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44fc06f9e0 0x7f44fc154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48554/0 (socket says 172.21.15.67:48554) 2024-03-09T08:52:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.811+0000 7f44fb200640 1 -- 172.21.15.67:0/3894185036 learned_addr learned my addr 172.21.15.67:0/3894185036 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:50.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.812+0000 7f44f1e00640 1 --2- 172.21.15.67:0/3894185036 >> v2:172.21.15.134:3300/0 conn(0x7f44fc0c5aa0 0x7f44fc153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.812+0000 7f44fa800640 1 --2- 172.21.15.67:0/3894185036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44fc14dbf0 0x7f44fc0c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.812+0000 7f44fb200640 1 -- 172.21.15.67:0/3894185036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44fc14dbf0 msgr2=0x7f44fc0c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:50.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.812+0000 7f44fb200640 1 --2- 172.21.15.67:0/3894185036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44fc14dbf0 0x7f44fc0c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.812+0000 7f44fb200640 1 -- 172.21.15.67:0/3894185036 >> v2:172.21.15.134:3300/0 conn(0x7f44fc0c5aa0 msgr2=0x7f44fc153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:50.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.812+0000 7f44fb200640 1 --2- 172.21.15.67:0/3894185036 >> v2:172.21.15.134:3300/0 conn(0x7f44fc0c5aa0 0x7f44fc153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.812+0000 7f44fb200640 1 -- 172.21.15.67:0/3894185036 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc157df0 con 0x7f44fc06f9e0 2024-03-09T08:52:50.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.812+0000 7f44fa800640 1 --2- 172.21.15.67:0/3894185036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44fc14dbf0 0x7f44fc0c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:50.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.813+0000 7f44fb200640 1 --2- 172.21.15.67:0/3894185036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44fc06f9e0 0x7f44fc154140 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f44fc068630 tx=0x7f44f402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a349d9ee01b831 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:50.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.813+0000 7f44f9e00640 1 -- 172.21.15.67:0/3894185036 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f402c950 con 0x7f44fc06f9e0 2024-03-09T08:52:50.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.813+0000 7f44f9e00640 1 -- 172.21.15.67:0/3894185036 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f402caf0 con 0x7f44fc06f9e0 2024-03-09T08:52:50.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.813+0000 7f44f9e00640 1 -- 172.21.15.67:0/3894185036 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f40343f0 con 0x7f44fc06f9e0 2024-03-09T08:52:50.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.814+0000 7f4500a00640 1 -- 172.21.15.67:0/3894185036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44fc06f9e0 msgr2=0x7f44fc154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.814+0000 7f4500a00640 1 --2- 172.21.15.67:0/3894185036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44fc06f9e0 0x7f44fc154140 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f44fc068630 tx=0x7f44f402bdb0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.814+0000 7f4500a00640 1 -- 172.21.15.67:0/3894185036 shutdown_connections 2024-03-09T08:52:50.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.814+0000 7f4500a00640 1 --2- 172.21.15.67:0/3894185036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44fc06f9e0 0x7f44fc154140 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.814+0000 7f4500a00640 1 --2- 172.21.15.67:0/3894185036 >> v2:172.21.15.134:3300/0 conn(0x7f44fc0c5aa0 0x7f44fc153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.814+0000 7f4500a00640 1 --2- 172.21.15.67:0/3894185036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44fc14dbf0 0x7f44fc0c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.814+0000 7f4500a00640 1 -- 172.21.15.67:0/3894185036 >> 172.21.15.67:0/3894185036 conn(0x7f44fc0c8310 msgr2=0x7f44fc0c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:50.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.814+0000 7f4500a00640 1 -- 172.21.15.67:0/3894185036 shutdown_connections 2024-03-09T08:52:50.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.814+0000 7f4500a00640 1 -- 172.21.15.67:0/3894185036 wait complete. 2024-03-09T08:52:50.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.816+0000 7f4500a00640 1 Processor -- start 2024-03-09T08:52:50.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.817+0000 7f4500a00640 1 -- start start 2024-03-09T08:52:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.818+0000 7f4500a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44fc06f9e0 0x7f44fc153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.818+0000 7f4500a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44fc14dbf0 0x7f44fc0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.818+0000 7f4500a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44fc14e060 0x7f44fc149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.818+0000 7f4500a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc154800 con 0x7f44fc14dbf0 2024-03-09T08:52:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.818+0000 7f4500a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc154680 con 0x7f44fc14e060 2024-03-09T08:52:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.818+0000 7f4500a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0c52e0 con 0x7f44fc06f9e0 2024-03-09T08:52:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.818+0000 7f44f1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44fc14dbf0 0x7f44fc0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.818+0000 7f44f1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44fc14dbf0 0x7f44fc0c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48558/0 (socket says 172.21.15.67:48558) 2024-03-09T08:52:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.818+0000 7f44f1e00640 1 -- 172.21.15.67:0/505480192 learned_addr learned my addr 172.21.15.67:0/505480192 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.818+0000 7f44fa800640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44fc06f9e0 0x7f44fc153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.818+0000 7f44fb200640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44fc14e060 0x7f44fc149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.819+0000 7f44f1e00640 1 -- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44fc06f9e0 msgr2=0x7f44fc153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.819+0000 7f44f1e00640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44fc06f9e0 0x7f44fc153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.819+0000 7f44f1e00640 1 -- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44fc14e060 msgr2=0x7f44fc149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:50.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.819+0000 7f44f1e00640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44fc14e060 0x7f44fc149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:50.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.819+0000 7f44f1e00640 1 -- 172.21.15.67:0/505480192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc1e9a90 con 0x7f44fc14dbf0 2024-03-09T08:52:50.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.819+0000 7f44f1e00640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44fc14dbf0 0x7f44fc0c7ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f44e400d830 tx=0x7f44e400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:50.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.819+0000 7f44f3400640 1 -- 172.21.15.67:0/505480192 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4018020 con 0x7f44fc14dbf0 2024-03-09T08:52:50.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.819+0000 7f44f3400640 1 -- 172.21.15.67:0/505480192 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4009640 con 0x7f44fc14dbf0 2024-03-09T08:52:50.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.819+0000 7f44f3400640 1 -- 172.21.15.67:0/505480192 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4009a10 con 0x7f44fc14dbf0 2024-03-09T08:52:50.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.819+0000 7f4500a00640 1 -- 172.21.15.67:0/505480192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44fc1ec220 con 0x7f44fc14dbf0 2024-03-09T08:52:50.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.820+0000 7f4500a00640 1 -- 172.21.15.67:0/505480192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44fc1ec7e0 con 0x7f44fc14dbf0 2024-03-09T08:52:50.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.821+0000 7f4500a00640 1 -- 172.21.15.67:0/505480192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44fc04f3c0 con 0x7f44fc14dbf0 2024-03-09T08:52:50.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.823+0000 7f44f3400640 1 -- 172.21.15.67:0/505480192 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f44e401b020 con 0x7f44fc14dbf0 2024-03-09T08:52:50.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.823+0000 7f44f3400640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f44d806b4d0 0x7f44d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:50.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.823+0000 7f44f3400640 1 -- 172.21.15.67:0/505480192 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f44e408a8c0 con 0x7f44fc14dbf0 2024-03-09T08:52:50.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.824+0000 7f44fa800640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f44d806b4d0 0x7f44d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:50.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.825+0000 7f44fa800640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f44d806b4d0 0x7f44d806d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00a950 tx=0x7f44ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:50.829+0000 7f44f3400640 1 -- 172.21.15.67:0/505480192 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f44e4022030 con 0x7f44fc14dbf0 2024-03-09T08:52:51.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.031+0000 7f4500a00640 1 -- 172.21.15.67:0/505480192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44fc0c7ff0 con 0x7f44fc14dbf0 2024-03-09T08:52:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.034+0000 7f44f3400640 1 -- 172.21.15.67:0/505480192 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20307 (secure 0 0 0) 0x7f44e40594a0 con 0x7f44fc14dbf0 2024-03-09T08:52:51.030 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:51.030 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:30.565529+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:51.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.036+0000 7f4500a00640 1 -- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f44d806b4d0 msgr2=0x7f44d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.036+0000 7f4500a00640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f44d806b4d0 0x7f44d806d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00a950 tx=0x7f44ec008040 comp rx=0 tx=0).stop 2024-03-09T08:52:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.036+0000 7f4500a00640 1 -- 172.21.15.67:0/505480192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44fc14dbf0 msgr2=0x7f44fc0c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.036+0000 7f4500a00640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44fc14dbf0 0x7f44fc0c7ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f44e400d830 tx=0x7f44e400dcf0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.036+0000 7f4500a00640 1 -- 172.21.15.67:0/505480192 shutdown_connections 2024-03-09T08:52:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.036+0000 7f4500a00640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f44d806b4d0 0x7f44d806d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.036+0000 7f4500a00640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44fc14e060 0x7f44fc149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.036+0000 7f4500a00640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44fc14dbf0 0x7f44fc0c7ab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.036+0000 7f4500a00640 1 --2- 172.21.15.67:0/505480192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44fc06f9e0 0x7f44fc153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.036+0000 7f4500a00640 1 -- 172.21.15.67:0/505480192 >> 172.21.15.67:0/505480192 conn(0x7f44fc0c8310 msgr2=0x7f44fc06e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.037+0000 7f4500a00640 1 -- 172.21.15.67:0/505480192 shutdown_connections 2024-03-09T08:52:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.037+0000 7f4500a00640 1 -- 172.21.15.67:0/505480192 wait complete. 2024-03-09T08:52:51.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-09T08:52:06.749641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-03-09T08:52:16.185356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-03-09T08:52:16.756829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-03-09T08:52:27.484297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-03-09T08:52:28.060754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-09T08:52:51.050 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T08:52:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.264+0000 7f6503200640 1 Processor -- start 2024-03-09T08:52:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.264+0000 7f6503200640 1 -- start start 2024-03-09T08:52:51.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.266+0000 7f6503200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f65040ac930 0x7f65040c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.266+0000 7f6503200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f65040c7f60 0x7f6504150de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.266+0000 7f6503200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6504151320 0x7f6504153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.266+0000 7f6503200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6504070670 con 0x7f65040c7f60 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.266+0000 7f6503200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6504151760 con 0x7f65040ac930 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.266+0000 7f6503200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65040713a0 con 0x7f6504151320 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.266+0000 7f6501400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f65040c7f60 0x7f6504150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.266+0000 7f6501400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f65040c7f60 0x7f6504150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48562/0 (socket says 172.21.15.67:48562) 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.266+0000 7f6501400640 1 -- 172.21.15.67:0/1339300627 learned_addr learned my addr 172.21.15.67:0/1339300627 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.266+0000 7f6501e00640 1 --2- 172.21.15.67:0/1339300627 >> v2:172.21.15.134:3300/0 conn(0x7f65040ac930 0x7f65040c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.266+0000 7f6502800640 1 --2- 172.21.15.67:0/1339300627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6504151320 0x7f6504153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.267+0000 7f6501400640 1 -- 172.21.15.67:0/1339300627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6504151320 msgr2=0x7f6504153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.267+0000 7f6501400640 1 --2- 172.21.15.67:0/1339300627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6504151320 0x7f6504153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.267+0000 7f6501400640 1 -- 172.21.15.67:0/1339300627 >> v2:172.21.15.134:3300/0 conn(0x7f65040ac930 msgr2=0x7f65040c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.267+0000 7f6501400640 1 --2- 172.21.15.67:0/1339300627 >> v2:172.21.15.134:3300/0 conn(0x7f65040ac930 0x7f65040c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.267+0000 7f6501400640 1 -- 172.21.15.67:0/1339300627 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040c39a0 con 0x7f65040c7f60 2024-03-09T08:52:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.268+0000 7f6501400640 1 --2- 172.21.15.67:0/1339300627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65040c7f60 0x7f6504150de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f65040c3cf0 tx=0x7f64f002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6011924dc2f39db0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.268+0000 7f6500a00640 1 -- 172.21.15.67:0/1339300627 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f002e070 con 0x7f65040c7f60 2024-03-09T08:52:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.268+0000 7f6500a00640 1 -- 172.21.15.67:0/1339300627 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f0037070 con 0x7f65040c7f60 2024-03-09T08:52:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.268+0000 7f6500a00640 1 -- 172.21.15.67:0/1339300627 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f0031a00 con 0x7f65040c7f60 2024-03-09T08:52:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.269+0000 7f6503200640 1 -- 172.21.15.67:0/1339300627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65040c7f60 msgr2=0x7f6504150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.269+0000 7f6503200640 1 --2- 172.21.15.67:0/1339300627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65040c7f60 0x7f6504150de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f65040c3cf0 tx=0x7f64f002b880 comp rx=0 tx=0).stop 2024-03-09T08:52:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.269+0000 7f6503200640 1 -- 172.21.15.67:0/1339300627 shutdown_connections 2024-03-09T08:52:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.269+0000 7f6503200640 1 --2- 172.21.15.67:0/1339300627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6504151320 0x7f6504153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.269+0000 7f6503200640 1 --2- 172.21.15.67:0/1339300627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65040c7f60 0x7f6504150de0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.269+0000 7f6503200640 1 --2- 172.21.15.67:0/1339300627 >> v2:172.21.15.134:3300/0 conn(0x7f65040ac930 0x7f65040c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.269+0000 7f6503200640 1 -- 172.21.15.67:0/1339300627 >> 172.21.15.67:0/1339300627 conn(0x7f650406b8a0 msgr2=0x7f65040c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.269+0000 7f6503200640 1 -- 172.21.15.67:0/1339300627 shutdown_connections 2024-03-09T08:52:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.269+0000 7f6503200640 1 -- 172.21.15.67:0/1339300627 wait complete. 2024-03-09T08:52:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.271+0000 7f6503200640 1 Processor -- start 2024-03-09T08:52:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.271+0000 7f6503200640 1 -- start start 2024-03-09T08:52:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6503200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65040ac930 0x7f65040c0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6503200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65040c7f60 0x7f6504149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6503200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6504151320 0x7f65041498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6503200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504070e80 con 0x7f65040ac930 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6503200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504070ba0 con 0x7f65040c7f60 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6503200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65040c3570 con 0x7f6504151320 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6501e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65040ac930 0x7f65040c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6501e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65040ac930 0x7f65040c0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48578/0 (socket says 172.21.15.67:48578) 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6501e00640 1 -- 172.21.15.67:0/2245498294 learned_addr learned my addr 172.21.15.67:0/2245498294 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6501400640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65040c7f60 0x7f6504149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6502800640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6504151320 0x7f65041498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6501e00640 1 -- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6504151320 msgr2=0x7f65041498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6501e00640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6504151320 0x7f65041498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6501e00640 1 -- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65040c7f60 msgr2=0x7f6504149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.272+0000 7f6501e00640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65040c7f60 0x7f6504149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.273+0000 7f6501e00640 1 -- 172.21.15.67:0/2245498294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65041d9de0 con 0x7f65040ac930 2024-03-09T08:52:51.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.273+0000 7f6501e00640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65040ac930 0x7f65040c0930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f64f400d800 tx=0x7f64f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:51.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.273+0000 7f64fa000640 1 -- 172.21.15.67:0/2245498294 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4018030 con 0x7f65040ac930 2024-03-09T08:52:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.273+0000 7f64fa000640 1 -- 172.21.15.67:0/2245498294 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f40096a0 con 0x7f65040ac930 2024-03-09T08:52:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.273+0000 7f64fa000640 1 -- 172.21.15.67:0/2245498294 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4009870 con 0x7f65040ac930 2024-03-09T08:52:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.273+0000 7f6503200640 1 -- 172.21.15.67:0/2245498294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65041ec4b0 con 0x7f65040ac930 2024-03-09T08:52:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.273+0000 7f6503200640 1 -- 172.21.15.67:0/2245498294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65040c0e70 con 0x7f65040ac930 2024-03-09T08:52:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.275+0000 7f6503200640 1 -- 172.21.15.67:0/2245498294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650404f3c0 con 0x7f65040ac930 2024-03-09T08:52:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.275+0000 7f64fa000640 1 -- 172.21.15.67:0/2245498294 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f64f401b070 con 0x7f65040ac930 2024-03-09T08:52:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.276+0000 7f64fa000640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f64e406b4d0 0x7f64e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.276+0000 7f64fa000640 1 -- 172.21.15.67:0/2245498294 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f64f408a750 con 0x7f65040ac930 2024-03-09T08:52:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.277+0000 7f6501400640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f64e406b4d0 0x7f64e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.278+0000 7f6501400640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f64e406b4d0 0x7f64e406d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f64f0036000 tx=0x7f64f003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:51.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.285+0000 7f64fa000640 1 -- 172.21.15.67:0/2245498294 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f64f4059330 con 0x7f65040ac930 2024-03-09T08:52:51.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.485+0000 7f6503200640 1 -- 172.21.15.67:0/2245498294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f650414a350 con 0x7f65040ac930 2024-03-09T08:52:51.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.486+0000 7f64fa000640 1 -- 172.21.15.67:0/2245498294 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64f4058c30 con 0x7f65040ac930 2024-03-09T08:52:51.482 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-09T08:52:51.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 -- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f64e406b4d0 msgr2=0x7f64e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f64e406b4d0 0x7f64e406d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f64f0036000 tx=0x7f64f003d040 comp rx=0 tx=0).stop 2024-03-09T08:52:51.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 -- 172.21.15.67:0/2245498294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65040ac930 msgr2=0x7f65040c0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65040ac930 0x7f65040c0930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f64f400d800 tx=0x7f64f400dcc0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 -- 172.21.15.67:0/2245498294 shutdown_connections 2024-03-09T08:52:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f64e406b4d0 0x7f64e406d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6504151320 0x7f65041498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65040c7f60 0x7f6504149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 --2- 172.21.15.67:0/2245498294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65040ac930 0x7f65040c0930 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 -- 172.21.15.67:0/2245498294 >> 172.21.15.67:0/2245498294 conn(0x7f650406b8a0 msgr2=0x7f65040c3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 -- 172.21.15.67:0/2245498294 shutdown_connections 2024-03-09T08:52:51.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.488+0000 7f6503200640 1 -- 172.21.15.67:0/2245498294 wait complete. 2024-03-09T08:52:51.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-03-09T08:52:51.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.730+0000 7f00dca00640 1 Processor -- start 2024-03-09T08:52:51.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.731+0000 7f00dca00640 1 -- start start 2024-03-09T08:52:51.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.733+0000 7f00dca00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f00d8150010 0x7f00d80c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.733+0000 7f00dca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d80c3620 con 0x7f00d80c3000 2024-03-09T08:52:51.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.733+0000 7f00dca00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d80c3900 con 0x7f00d8153360 2024-03-09T08:52:51.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.734+0000 7f00dca00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d8153840 con 0x7f00d8150010 2024-03-09T08:52:51.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.734+0000 7f00d7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f00d80c3000 0x7f00d815d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49058/0 (socket says 172.21.15.67:49058) 2024-03-09T08:52:51.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.734+0000 7f00d7e00640 1 -- 172.21.15.67:0/2403752819 learned_addr learned my addr 172.21.15.67:0/2403752819 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.734+0000 7f00d7400640 1 --2- 172.21.15.67:0/2403752819 >> v2:172.21.15.134:3300/0 conn(0x7f00d8150010 0x7f00d80c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.734+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068016443 0 0) 0x7f00d80c3620 con 0x7f00d80c3000 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.734+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00bc003520 con 0x7f00d80c3000 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3972954539 0 0) 0x7f00bc003520 con 0x7f00d80c3000 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 >> v2:172.21.15.134:3300/0 conn(0x7f00d8150010 msgr2=0x7f00d80c2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 --2- 172.21.15.67:0/2403752819 >> v2:172.21.15.134:3300/0 conn(0x7f00d8150010 0x7f00d80c2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 >> v1:172.21.15.134:6789/0 conn(0x7f00d8153360 legacy=0x7f00d80c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d80c70b0 con 0x7f00d80c3000 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f00d00019f0 con 0x7f00d80c3000 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 >> v1:172.21.15.67:6789/0 conn(0x7f00d80c3000 legacy=0x7f00d815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 --2- 172.21.15.67:0/2403752819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00bc0039b0 0x7f00bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 --2- 172.21.15.67:0/2403752819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc0043b0 0x7f00bc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 --2- 172.21.15.67:0/2403752819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 0x7f00bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d8153840 con 0x7f00bc005120 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc00c060 con 0x7f00bc0043b0 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc00c260 con 0x7f00bc0039b0 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d7e00640 1 --2- 172.21.15.67:0/2403752819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 0x7f00bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.735+0000 7f00d6a00640 1 --2- 172.21.15.67:0/2403752819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc0043b0 0x7f00bc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.736+0000 7f00d7400640 1 --2- 172.21.15.67:0/2403752819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00bc0039b0 0x7f00bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.736+0000 7f00d7e00640 1 -- 172.21.15.67:0/2403752819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00bc0039b0 msgr2=0x7f00bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.736+0000 7f00d7e00640 1 --2- 172.21.15.67:0/2403752819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00bc0039b0 0x7f00bc003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.736+0000 7f00d7e00640 1 -- 172.21.15.67:0/2403752819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc0043b0 msgr2=0x7f00bc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.736+0000 7f00d7e00640 1 --2- 172.21.15.67:0/2403752819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc0043b0 0x7f00bc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.736+0000 7f00d7e00640 1 -- 172.21.15.67:0/2403752819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc003520 con 0x7f00bc005120 2024-03-09T08:52:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.737+0000 7f00d7e00640 1 -- 172.21.15.67:0/2403752819 reap_dead start 2024-03-09T08:52:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.737+0000 7f00d7e00640 1 --2- 172.21.15.67:0/2403752819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 0x7f00bc00b9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f00d000a920 tx=0x7f00bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb609f5c7bb9a84 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.737+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d000a4c0 con 0x7f00bc005120 2024-03-09T08:52:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.737+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d0034650 con 0x7f00bc005120 2024-03-09T08:52:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.737+0000 7f00d6000640 1 -- 172.21.15.67:0/2403752819 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0034a40 con 0x7f00bc005120 2024-03-09T08:52:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.738+0000 7f00dca00640 1 -- 172.21.15.67:0/2403752819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 msgr2=0x7f00bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.738+0000 7f00dca00640 1 --2- 172.21.15.67:0/2403752819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 0x7f00bc00b9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f00d000a920 tx=0x7f00bc005830 comp rx=0 tx=0).stop 2024-03-09T08:52:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.738+0000 7f00dca00640 1 -- 172.21.15.67:0/2403752819 shutdown_connections 2024-03-09T08:52:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.738+0000 7f00dca00640 1 --2- 172.21.15.67:0/2403752819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 0x7f00bc00b9c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.738+0000 7f00dca00640 1 -- 172.21.15.67:0/2403752819 >> 172.21.15.67:0/2403752819 conn(0x7f00d80a9740 msgr2=0x7f00d80a9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.738+0000 7f00dca00640 1 -- 172.21.15.67:0/2403752819 shutdown_connections 2024-03-09T08:52:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.738+0000 7f00dca00640 1 -- 172.21.15.67:0/2403752819 wait complete. 2024-03-09T08:52:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.740+0000 7f00dca00640 1 Processor -- start 2024-03-09T08:52:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.740+0000 7f00dca00640 1 -- start start 2024-03-09T08:52:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.741+0000 7f00dca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 0x7f00d80acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.741+0000 7f00dca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00d81ebaf0 0x7f00d81eedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.741+0000 7f00dca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00d80a7d10 0x7f00d8071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.741+0000 7f00dca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80c4000 con 0x7f00bc005120 2024-03-09T08:52:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.741+0000 7f00dca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d8150600 con 0x7f00d81ebaf0 2024-03-09T08:52:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.741+0000 7f00dca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80c4180 con 0x7f00d80a7d10 2024-03-09T08:52:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.741+0000 7f00d7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 0x7f00d80acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.741+0000 7f00d7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 0x7f00d80acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48598/0 (socket says 172.21.15.67:48598) 2024-03-09T08:52:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.741+0000 7f00d7400640 1 -- 172.21.15.67:0/23404183 learned_addr learned my addr 172.21.15.67:0/23404183 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.742+0000 7f00d6a00640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00d81ebaf0 0x7f00d81eedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.742+0000 7f00d7400640 1 -- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00d80a7d10 msgr2=0x7f00d8071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.742+0000 7f00d7e00640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00d80a7d10 0x7f00d8071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.742+0000 7f00d7400640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00d80a7d10 0x7f00d8071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.742+0000 7f00d7400640 1 -- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00d81ebaf0 msgr2=0x7f00d81eedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.742+0000 7f00d7400640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00d81ebaf0 0x7f00d81eedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.742+0000 7f00d7400640 1 -- 172.21.15.67:0/23404183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d81ef2f0 con 0x7f00bc005120 2024-03-09T08:52:51.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.742+0000 7f00d7400640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 0x7f00d80acd00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f00c80108f0 tx=0x7f00c8010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:51.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.742+0000 7f00cf400640 1 -- 172.21.15.67:0/23404183 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c800f040 con 0x7f00bc005120 2024-03-09T08:52:51.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.743+0000 7f00cf400640 1 -- 172.21.15.67:0/23404183 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c8005b90 con 0x7f00bc005120 2024-03-09T08:52:51.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.743+0000 7f00cf400640 1 -- 172.21.15.67:0/23404183 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c800e900 con 0x7f00bc005120 2024-03-09T08:52:51.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.743+0000 7f00dca00640 1 -- 172.21.15.67:0/23404183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d80ad2e0 con 0x7f00bc005120 2024-03-09T08:52:51.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.743+0000 7f00dca00640 1 -- 172.21.15.67:0/23404183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d81eb2f0 con 0x7f00bc005120 2024-03-09T08:52:51.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.744+0000 7f00dca00640 1 -- 172.21.15.67:0/23404183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d81eae00 con 0x7f00bc005120 2024-03-09T08:52:51.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.745+0000 7f00cf400640 1 -- 172.21.15.67:0/23404183 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f00c800eaa0 con 0x7f00bc005120 2024-03-09T08:52:51.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.746+0000 7f00cf400640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f00b806b4a0 0x7f00b806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:51.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.746+0000 7f00cf400640 1 -- 172.21.15.67:0/23404183 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f00c808a7b0 con 0x7f00bc005120 2024-03-09T08:52:51.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.746+0000 7f00d6a00640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f00b806b4a0 0x7f00b806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:51.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.747+0000 7f00d6a00640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f00b806b4a0 0x7f00b806d960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f00d80adc80 tx=0x7f00c4007bb0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:51.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.751+0000 7f00cf400640 1 -- 172.21.15.67:0/23404183 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f00c8018030 con 0x7f00bc005120 2024-03-09T08:52:51.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.955+0000 7f00dca00640 1 -- 172.21.15.67:0/23404183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00d804f3c0 con 0x7f00bc005120 2024-03-09T08:52:51.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.955+0000 7f00cf400640 1 -- 172.21.15.67:0/23404183 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7f00c8059390 con 0x7f00bc005120 2024-03-09T08:52:51.951 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T08:52:51.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 -- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f00b806b4a0 msgr2=0x7f00b806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f00b806b4a0 0x7f00b806d960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f00d80adc80 tx=0x7f00c4007bb0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 -- 172.21.15.67:0/23404183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 msgr2=0x7f00d80acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:51.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 0x7f00d80acd00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f00c80108f0 tx=0x7f00c8010db0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 -- 172.21.15.67:0/23404183 shutdown_connections 2024-03-09T08:52:51.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f00b806b4a0 0x7f00b806d960 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00d80a7d10 0x7f00d8071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00d81ebaf0 0x7f00d81eedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 --2- 172.21.15.67:0/23404183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f00bc005120 0x7f00d80acd00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:51.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 -- 172.21.15.67:0/23404183 >> 172.21.15.67:0/23404183 conn(0x7f00d80a9740 msgr2=0x7f00d8151d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:51.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 -- 172.21.15.67:0/23404183 shutdown_connections 2024-03-09T08:52:51.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:51.958+0000 7f00dca00640 1 -- 172.21.15.67:0/23404183 wait complete. 2024-03-09T08:52:51.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-03-09T08:52:52.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.184+0000 7f2af2c00640 1 Processor -- start 2024-03-09T08:52:52.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.185+0000 7f2af2c00640 1 -- start start 2024-03-09T08:52:52.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.186+0000 7f2af2c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2aec0702d0 0x7f2aec0c8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.186+0000 7f2af2c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2aec0c8c70 0x7f2aec0c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.186+0000 7f2af2c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec073140 con 0x7f2aec157290 2024-03-09T08:52:52.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.186+0000 7f2af2c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aec157d80 con 0x7f2aec0702d0 2024-03-09T08:52:52.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.186+0000 7f2af2c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aec070840 con 0x7f2aec0c8c70 2024-03-09T08:52:52.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.186+0000 7f2ae3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2aec0702d0 0x7f2aec0c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.186+0000 7f2ae3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2aec0702d0 0x7f2aec0c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48610/0 (socket says 172.21.15.67:48610) 2024-03-09T08:52:52.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.187+0000 7f2ae3400640 1 -- 172.21.15.67:0/3960525656 learned_addr learned my addr 172.21.15.67:0/3960525656 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:52.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.187+0000 7f2af0c00640 1 --2- 172.21.15.67:0/3960525656 >> v2:172.21.15.134:3300/0 conn(0x7f2aec0c8c70 0x7f2aec0c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.187+0000 7f2ae3400640 1 -- 172.21.15.67:0/3960525656 >> v2:172.21.15.134:3300/0 conn(0x7f2aec0c8c70 msgr2=0x7f2aec0c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:52.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.187+0000 7f2ae3400640 1 --2- 172.21.15.67:0/3960525656 >> v2:172.21.15.134:3300/0 conn(0x7f2aec0c8c70 0x7f2aec0c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.187+0000 7f2ae3400640 1 -- 172.21.15.67:0/3960525656 >> v1:172.21.15.134:6789/0 conn(0x7f2aec157290 legacy=0x7f2aec157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:52.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.187+0000 7f2ae3400640 1 -- 172.21.15.67:0/3960525656 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec0c90a0 con 0x7f2aec0702d0 2024-03-09T08:52:52.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.188+0000 7f2ae3400640 1 --2- 172.21.15.67:0/3960525656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec0702d0 0x7f2aec0c8730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0c4380 tx=0x7f2ad402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde619a8a3a7b5d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:52.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.188+0000 7f2aeb400640 1 -- 172.21.15.67:0/3960525656 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4035020 con 0x7f2aec0702d0 2024-03-09T08:52:52.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.189+0000 7f2aeb400640 1 -- 172.21.15.67:0/3960525656 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad4031870 con 0x7f2aec0702d0 2024-03-09T08:52:52.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.189+0000 7f2aeb400640 1 -- 172.21.15.67:0/3960525656 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4031c40 con 0x7f2aec0702d0 2024-03-09T08:52:52.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.189+0000 7f2af2c00640 1 -- 172.21.15.67:0/3960525656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec0702d0 msgr2=0x7f2aec0c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:52.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.189+0000 7f2af2c00640 1 --2- 172.21.15.67:0/3960525656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec0702d0 0x7f2aec0c8730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0c4380 tx=0x7f2ad402b910 comp rx=0 tx=0).stop 2024-03-09T08:52:52.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.189+0000 7f2af2c00640 1 -- 172.21.15.67:0/3960525656 shutdown_connections 2024-03-09T08:52:52.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.189+0000 7f2af2c00640 1 --2- 172.21.15.67:0/3960525656 >> v2:172.21.15.134:3300/0 conn(0x7f2aec0c8c70 0x7f2aec0c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.189+0000 7f2af2c00640 1 --2- 172.21.15.67:0/3960525656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec0702d0 0x7f2aec0c8730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.189+0000 7f2af2c00640 1 -- 172.21.15.67:0/3960525656 >> 172.21.15.67:0/3960525656 conn(0x7f2aec06b460 msgr2=0x7f2aec066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:52.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.189+0000 7f2af2c00640 1 -- 172.21.15.67:0/3960525656 shutdown_connections 2024-03-09T08:52:52.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.189+0000 7f2af2c00640 1 -- 172.21.15.67:0/3960525656 wait complete. 2024-03-09T08:52:52.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.191+0000 7f2af2c00640 1 Processor -- start 2024-03-09T08:52:52.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.191+0000 7f2af2c00640 1 -- start start 2024-03-09T08:52:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.192+0000 7f2af2c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2aec0702d0 0x7f2aec1ed590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.192+0000 7f2af2c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec0c8c70 0x7f2aec1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.192+0000 7f2af2c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2aec157290 0x7f2aec0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.192+0000 7f2af2c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec072e60 con 0x7f2aec0c8c70 2024-03-09T08:52:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.192+0000 7f2af2c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0c5130 con 0x7f2aec157290 2024-03-09T08:52:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.192+0000 7f2af2c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec073ab0 con 0x7f2aec0702d0 2024-03-09T08:52:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.192+0000 7f2ae3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec0c8c70 0x7f2aec1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.193+0000 7f2ae3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec0c8c70 0x7f2aec1518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48614/0 (socket says 172.21.15.67:48614) 2024-03-09T08:52:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.193+0000 7f2ae3400640 1 -- 172.21.15.67:0/115818952 learned_addr learned my addr 172.21.15.67:0/115818952 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:52.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.193+0000 7f2aebe00640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2aec0702d0 0x7f2aec1ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.193+0000 7f2af0c00640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2aec157290 0x7f2aec0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.193+0000 7f2ae3400640 1 -- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2aec0702d0 msgr2=0x7f2aec1ed590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:52.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.193+0000 7f2ae3400640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2aec0702d0 0x7f2aec1ed590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.193+0000 7f2ae3400640 1 -- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2aec157290 msgr2=0x7f2aec0ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:52.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.193+0000 7f2ae3400640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2aec157290 0x7f2aec0ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.193+0000 7f2ae3400640 1 -- 172.21.15.67:0/115818952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec0ae240 con 0x7f2aec0c8c70 2024-03-09T08:52:52.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.193+0000 7f2ae3400640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec0c8c70 0x7f2aec1518a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4000c00 tx=0x7f2ad40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:52.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.194+0000 7f2ae8c00640 1 -- 172.21.15.67:0/115818952 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4035040 con 0x7f2aec0c8c70 2024-03-09T08:52:52.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.194+0000 7f2ae8c00640 1 -- 172.21.15.67:0/115818952 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad4030d00 con 0x7f2aec0c8c70 2024-03-09T08:52:52.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.194+0000 7f2af2c00640 1 -- 172.21.15.67:0/115818952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec154370 con 0x7f2aec0c8c70 2024-03-09T08:52:52.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.194+0000 7f2ae8c00640 1 -- 172.21.15.67:0/115818952 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40456a0 con 0x7f2aec0c8c70 2024-03-09T08:52:52.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.194+0000 7f2af2c00640 1 -- 172.21.15.67:0/115818952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec1e97b0 con 0x7f2aec0c8c70 2024-03-09T08:52:52.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.195+0000 7f2af2c00640 1 -- 172.21.15.67:0/115818952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec06ff90 con 0x7f2aec0c8c70 2024-03-09T08:52:52.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.196+0000 7f2ae8c00640 1 -- 172.21.15.67:0/115818952 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2ad4038020 con 0x7f2aec0c8c70 2024-03-09T08:52:52.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.198+0000 7f2ae8c00640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ac806b4d0 0x7f2ac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.198+0000 7f2ae8c00640 1 -- 172.21.15.67:0/115818952 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f2ad40ac890 con 0x7f2aec0c8c70 2024-03-09T08:52:52.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.198+0000 7f2aebe00640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ac806b4d0 0x7f2ac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.199+0000 7f2aebe00640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ac806b4d0 0x7f2ac806d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0012e0 tx=0x7f2adc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:52.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.201+0000 7f2ae8c00640 1 -- 172.21.15.67:0/115818952 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2aec0ae240 con 0x7f2aec0c8c70 2024-03-09T08:52:52.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.399+0000 7f2af2c00640 1 -- 172.21.15.67:0/115818952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f2aec0747f0 con 0x7f2aec0c8c70 2024-03-09T08:52:52.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.399+0000 7f2ae8c00640 1 -- 172.21.15.67:0/115818952 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7f2ad407b3d0 con 0x7f2aec0c8c70 2024-03-09T08:52:52.395 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T08:52:52.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 -- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ac806b4d0 msgr2=0x7f2ac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ac806b4d0 0x7f2ac806d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0012e0 tx=0x7f2adc009040 comp rx=0 tx=0).stop 2024-03-09T08:52:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 -- 172.21.15.67:0/115818952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec0c8c70 msgr2=0x7f2aec1518a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec0c8c70 0x7f2aec1518a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4000c00 tx=0x7f2ad40046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 -- 172.21.15.67:0/115818952 shutdown_connections 2024-03-09T08:52:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ac806b4d0 0x7f2ac806d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2aec157290 0x7f2aec0ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec0c8c70 0x7f2aec1518a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 --2- 172.21.15.67:0/115818952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2aec0702d0 0x7f2aec1ed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 -- 172.21.15.67:0/115818952 >> 172.21.15.67:0/115818952 conn(0x7f2aec06b460 msgr2=0x7f2aec157670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 -- 172.21.15.67:0/115818952 shutdown_connections 2024-03-09T08:52:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.402+0000 7f2af2c00640 1 -- 172.21.15.67:0/115818952 wait complete. 2024-03-09T08:52:52.415 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-03-09T08:52:52.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.641+0000 7fd305400640 1 Processor -- start 2024-03-09T08:52:52.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.641+0000 7fd305400640 1 -- start start 2024-03-09T08:52:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.642+0000 7fd305400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3000c5f10 0x7fd3000c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.642+0000 7fd305400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd30006b040 0x7fd3001567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.642+0000 7fd305400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd300156d10 con 0x7fd3000c9490 2024-03-09T08:52:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.642+0000 7fd305400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3000c9a30 con 0x7fd30006b040 2024-03-09T08:52:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.642+0000 7fd305400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000c9d10 con 0x7fd3000c5f10 2024-03-09T08:52:52.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.643+0000 7fd2f7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd30006b040 0x7fd3001567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.643+0000 7fd2f7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd30006b040 0x7fd3001567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48618/0 (socket says 172.21.15.67:48618) 2024-03-09T08:52:52.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.643+0000 7fd2f7400640 1 -- 172.21.15.67:0/2617981548 learned_addr learned my addr 172.21.15.67:0/2617981548 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:52.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.643+0000 7fd2ffe00640 1 --2- 172.21.15.67:0/2617981548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3000c5f10 0x7fd3000c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.643+0000 7fd2f7400640 1 -- 172.21.15.67:0/2617981548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3000c5f10 msgr2=0x7fd3000c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:52.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.643+0000 7fd2f7400640 1 --2- 172.21.15.67:0/2617981548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3000c5f10 0x7fd3000c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.643+0000 7fd2f7400640 1 -- 172.21.15.67:0/2617981548 >> v1:172.21.15.134:6789/0 conn(0x7fd3000c9490 legacy=0x7fd3000cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:52.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.643+0000 7fd2f7400640 1 -- 172.21.15.67:0/2617981548 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3000ca340 con 0x7fd30006b040 2024-03-09T08:52:52.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.644+0000 7fd2f7400640 1 --2- 172.21.15.67:0/2617981548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30006b040 0x7fd3001567d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009870 tx=0x7fd2e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19c81816d867de7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:52.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.645+0000 7fd2ff400640 1 -- 172.21.15.67:0/2617981548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8035020 con 0x7fd30006b040 2024-03-09T08:52:52.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.645+0000 7fd2ff400640 1 -- 172.21.15.67:0/2617981548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8031640 con 0x7fd30006b040 2024-03-09T08:52:52.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.645+0000 7fd2ff400640 1 -- 172.21.15.67:0/2617981548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e802d040 con 0x7fd30006b040 2024-03-09T08:52:52.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.645+0000 7fd305400640 1 -- 172.21.15.67:0/2617981548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30006b040 msgr2=0x7fd3001567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:52.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.645+0000 7fd305400640 1 --2- 172.21.15.67:0/2617981548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30006b040 0x7fd3001567d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009870 tx=0x7fd2e802b8f0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.646+0000 7fd305400640 1 -- 172.21.15.67:0/2617981548 shutdown_connections 2024-03-09T08:52:52.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.646+0000 7fd305400640 1 --2- 172.21.15.67:0/2617981548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd30006b040 0x7fd3001567d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.646+0000 7fd305400640 1 --2- 172.21.15.67:0/2617981548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3000c5f10 0x7fd3000c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.646+0000 7fd305400640 1 -- 172.21.15.67:0/2617981548 >> 172.21.15.67:0/2617981548 conn(0x7fd3000cb590 msgr2=0x7fd3000cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:52.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.646+0000 7fd305400640 1 -- 172.21.15.67:0/2617981548 shutdown_connections 2024-03-09T08:52:52.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.646+0000 7fd305400640 1 -- 172.21.15.67:0/2617981548 wait complete. 2024-03-09T08:52:52.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.647+0000 7fd305400640 1 Processor -- start 2024-03-09T08:52:52.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.647+0000 7fd305400640 1 -- start start 2024-03-09T08:52:52.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.648+0000 7fd305400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30006b040 0x7fd3000c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.648+0000 7fd305400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3000c5f10 0x7fd3001e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.648+0000 7fd305400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3000c9490 0x7fd3000c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.648+0000 7fd305400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000cb250 con 0x7fd3000c9490 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.648+0000 7fd305400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000c9ff0 con 0x7fd30006b040 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.648+0000 7fd305400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000cb3d0 con 0x7fd3000c5f10 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.649+0000 7fd304a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3000c9490 0x7fd3000c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.649+0000 7fd304a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3000c9490 0x7fd3000c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48630/0 (socket says 172.21.15.67:48630) 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.649+0000 7fd304a00640 1 -- 172.21.15.67:0/3218470977 learned_addr learned my addr 172.21.15.67:0/3218470977 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.649+0000 7fd2ffe00640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30006b040 0x7fd3000c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.649+0000 7fd2f7400640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3000c5f10 0x7fd3001e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.649+0000 7fd304a00640 1 -- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3000c5f10 msgr2=0x7fd3001e17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:52.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.649+0000 7fd304a00640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3000c5f10 0x7fd3001e17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.649+0000 7fd304a00640 1 -- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30006b040 msgr2=0x7fd3000c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:52.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.649+0000 7fd304a00640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30006b040 0x7fd3000c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.649+0000 7fd304a00640 1 -- 172.21.15.67:0/3218470977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3000bdb40 con 0x7fd3000c9490 2024-03-09T08:52:52.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.650+0000 7fd304a00640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3000c9490 0x7fd3000c4470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c390 tx=0x7fd2f800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:52.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.650+0000 7fd2fcc00640 1 -- 172.21.15.67:0/3218470977 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8017070 con 0x7fd3000c9490 2024-03-09T08:52:52.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.650+0000 7fd2fcc00640 1 -- 172.21.15.67:0/3218470977 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f80052d0 con 0x7fd3000c9490 2024-03-09T08:52:52.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.650+0000 7fd2fcc00640 1 -- 172.21.15.67:0/3218470977 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8003810 con 0x7fd3000c9490 2024-03-09T08:52:52.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.650+0000 7fd305400640 1 -- 172.21.15.67:0/3218470977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3000bde20 con 0x7fd3000c9490 2024-03-09T08:52:52.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.650+0000 7fd305400640 1 -- 172.21.15.67:0/3218470977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd300150700 con 0x7fd3000c9490 2024-03-09T08:52:52.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.652+0000 7fd305400640 1 -- 172.21.15.67:0/3218470977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3000c4ec0 con 0x7fd3000c9490 2024-03-09T08:52:52.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.652+0000 7fd2fcc00640 1 -- 172.21.15.67:0/3218470977 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd2f801a070 con 0x7fd3000c9490 2024-03-09T08:52:52.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.653+0000 7fd2fcc00640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd2dc06b500 0x7fd2dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:52.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.653+0000 7fd2fcc00640 1 -- 172.21.15.67:0/3218470977 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fd2f8089800 con 0x7fd3000c9490 2024-03-09T08:52:52.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.654+0000 7fd2ffe00640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd2dc06b500 0x7fd2dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:52.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.655+0000 7fd2ffe00640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd2dc06b500 0x7fd2dc06d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0003aa0 tx=0x7fd2f0009830 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:52.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.657+0000 7fd2fcc00640 1 -- 172.21.15.67:0/3218470977 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd300156d10 con 0x7fd3000c9490 2024-03-09T08:52:52.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.898+0000 7fd305400640 1 -- 172.21.15.67:0/3218470977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd30004f3c0 con 0x7fd3000c9490 2024-03-09T08:52:52.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.899+0000 7fd2fcc00640 1 -- 172.21.15.67:0/3218470977 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7fd2f8057c60 con 0x7fd3000c9490 2024-03-09T08:52:52.898 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T08:52:52.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 -- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd2dc06b500 msgr2=0x7fd2dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:52.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd2dc06b500 0x7fd2dc06d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0003aa0 tx=0x7fd2f0009830 comp rx=0 tx=0).stop 2024-03-09T08:52:52.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 -- 172.21.15.67:0/3218470977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3000c9490 msgr2=0x7fd3000c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:52.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3000c9490 0x7fd3000c4470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c390 tx=0x7fd2f800c850 comp rx=0 tx=0).stop 2024-03-09T08:52:52.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 -- 172.21.15.67:0/3218470977 shutdown_connections 2024-03-09T08:52:52.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd2dc06b500 0x7fd2dc06d9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3000c9490 0x7fd3000c4470 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3000c5f10 0x7fd3001e17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 --2- 172.21.15.67:0/3218470977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30006b040 0x7fd3000c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:52.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 -- 172.21.15.67:0/3218470977 >> 172.21.15.67:0/3218470977 conn(0x7fd3000cb590 msgr2=0x7fd30006b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:52.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 -- 172.21.15.67:0/3218470977 shutdown_connections 2024-03-09T08:52:52.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:52.902+0000 7fd305400640 1 -- 172.21.15.67:0/3218470977 wait complete. 2024-03-09T08:52:52.921 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-03-09T08:52:53.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.159+0000 7fdd33200640 1 Processor -- start 2024-03-09T08:52:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.160+0000 7fdd33200640 1 -- start start 2024-03-09T08:52:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.161+0000 7fdd33200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340c8190 0x7fdd340c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.161+0000 7fdd33200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd34152f20 0x7fdd34153380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.161+0000 7fdd33200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd34155d70 con 0x7fdd340ca2c0 2024-03-09T08:52:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.161+0000 7fdd33200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd34156c40 con 0x7fdd34152f20 2024-03-09T08:52:53.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.161+0000 7fdd33200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd34156030 con 0x7fdd340c8190 2024-03-09T08:52:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.161+0000 7fdd32800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdd340ca2c0 0x7fdd340ca750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49066/0 (socket says 172.21.15.67:49066) 2024-03-09T08:52:53.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.161+0000 7fdd2be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd34152f20 0x7fdd34153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.162+0000 7fdd32800640 1 -- 172.21.15.67:0/3386908550 learned_addr learned my addr 172.21.15.67:0/3386908550 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.162+0000 7fdd31e00640 1 --2- 172.21.15.67:0/3386908550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340c8190 0x7fdd340c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.162+0000 7fdd2be00640 1 -- 172.21.15.67:0/3386908550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340c8190 msgr2=0x7fdd340c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.162+0000 7fdd2be00640 1 --2- 172.21.15.67:0/3386908550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340c8190 0x7fdd340c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.162+0000 7fdd2be00640 1 -- 172.21.15.67:0/3386908550 >> v1:172.21.15.67:6789/0 conn(0x7fdd340ca2c0 legacy=0x7fdd340ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.162+0000 7fdd2be00640 1 -- 172.21.15.67:0/3386908550 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd34156310 con 0x7fdd34152f20 2024-03-09T08:52:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.162+0000 7fdd31400640 1 -- 172.21.15.67:0/3386908550 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888436518 0 0) 0x7fdd34155d70 con 0x7fdd340ca2c0 2024-03-09T08:52:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.162+0000 7fdd31e00640 1 --2- 172.21.15.67:0/3386908550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340c8190 0x7fdd340c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:53.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.163+0000 7fdd2be00640 1 --2- 172.21.15.67:0/3386908550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd34152f20 0x7fdd34153380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0097a0 tx=0x7fdd1c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2558dc75c4808412 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:53.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.164+0000 7fdd31400640 1 -- 172.21.15.67:0/3386908550 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c031780 con 0x7fdd34152f20 2024-03-09T08:52:53.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.164+0000 7fdd31400640 1 -- 172.21.15.67:0/3386908550 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c037070 con 0x7fdd34152f20 2024-03-09T08:52:53.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.164+0000 7fdd31400640 1 -- 172.21.15.67:0/3386908550 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c031b40 con 0x7fdd34152f20 2024-03-09T08:52:53.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.164+0000 7fdd33200640 1 -- 172.21.15.67:0/3386908550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd34152f20 msgr2=0x7fdd34153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.164+0000 7fdd33200640 1 --2- 172.21.15.67:0/3386908550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd34152f20 0x7fdd34153380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0097a0 tx=0x7fdd1c02b820 comp rx=0 tx=0).stop 2024-03-09T08:52:53.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.164+0000 7fdd33200640 1 -- 172.21.15.67:0/3386908550 shutdown_connections 2024-03-09T08:52:53.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.164+0000 7fdd33200640 1 --2- 172.21.15.67:0/3386908550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd34152f20 0x7fdd34153380 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.164+0000 7fdd33200640 1 --2- 172.21.15.67:0/3386908550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340c8190 0x7fdd340c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.164+0000 7fdd33200640 1 -- 172.21.15.67:0/3386908550 >> 172.21.15.67:0/3386908550 conn(0x7fdd340ad060 msgr2=0x7fdd340ad440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:53.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.164+0000 7fdd33200640 1 -- 172.21.15.67:0/3386908550 shutdown_connections 2024-03-09T08:52:53.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.165+0000 7fdd33200640 1 -- 172.21.15.67:0/3386908550 wait complete. 2024-03-09T08:52:53.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.166+0000 7fdd33200640 1 Processor -- start 2024-03-09T08:52:53.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.166+0000 7fdd33200640 1 -- start start 2024-03-09T08:52:53.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.167+0000 7fdd33200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd340c8190 0x7fdd341ebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.167+0000 7fdd33200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340ca2c0 0x7fdd34152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.167+0000 7fdd33200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34152f20 0x7fdd3414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.167+0000 7fdd33200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd340c9b80 con 0x7fdd340c8190 2024-03-09T08:52:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.167+0000 7fdd33200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341568d0 con 0x7fdd34152f20 2024-03-09T08:52:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.167+0000 7fdd33200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd340cae60 con 0x7fdd340ca2c0 2024-03-09T08:52:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.168+0000 7fdd31e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd340c8190 0x7fdd341ebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.168+0000 7fdd31e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd340c8190 0x7fdd341ebfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48654/0 (socket says 172.21.15.67:48654) 2024-03-09T08:52:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.168+0000 7fdd31e00640 1 -- 172.21.15.67:0/3349316964 learned_addr learned my addr 172.21.15.67:0/3349316964 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.168+0000 7fdd2be00640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340ca2c0 0x7fdd34152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.168+0000 7fdd31e00640 1 -- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340ca2c0 msgr2=0x7fdd34152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.168+0000 7fdd31e00640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340ca2c0 0x7fdd34152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.168+0000 7fdd31e00640 1 -- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34152f20 msgr2=0x7fdd3414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.168+0000 7fdd31e00640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34152f20 0x7fdd3414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.168+0000 7fdd31e00640 1 -- 172.21.15.67:0/3349316964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3414e160 con 0x7fdd340c8190 2024-03-09T08:52:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.168+0000 7fdd2be00640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340ca2c0 0x7fdd34152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.169+0000 7fdd31e00640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd340c8190 0x7fdd341ebfe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400e7e0 tx=0x7fdd2400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.169+0000 7fdd2a000640 1 -- 172.21.15.67:0/3349316964 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400f040 con 0x7fdd340c8190 2024-03-09T08:52:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.169+0000 7fdd2a000640 1 -- 172.21.15.67:0/3349316964 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd24004030 con 0x7fdd340c8190 2024-03-09T08:52:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.169+0000 7fdd2a000640 1 -- 172.21.15.67:0/3349316964 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24004400 con 0x7fdd340c8190 2024-03-09T08:52:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.169+0000 7fdd33200640 1 -- 172.21.15.67:0/3349316964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd341529a0 con 0x7fdd340c8190 2024-03-09T08:52:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.169+0000 7fdd33200640 1 -- 172.21.15.67:0/3349316964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd341eb500 con 0x7fdd340c8190 2024-03-09T08:52:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.170+0000 7fdd33200640 1 -- 172.21.15.67:0/3349316964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3404f3c0 con 0x7fdd340c8190 2024-03-09T08:52:53.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.171+0000 7fdd2a000640 1 -- 172.21.15.67:0/3349316964 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdd2400c6d0 con 0x7fdd340c8190 2024-03-09T08:52:53.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.172+0000 7fdd2a000640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd1006b500 0x7fdd1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.172+0000 7fdd2a000640 1 -- 172.21.15.67:0/3349316964 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fdd2408a3a0 con 0x7fdd340c8190 2024-03-09T08:52:53.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.173+0000 7fdd2be00640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd1006b500 0x7fdd1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.174+0000 7fdd2be00640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd1006b500 0x7fdd1006d9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c036000 tx=0x7fdd1c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.176+0000 7fdd2a000640 1 -- 172.21.15.67:0/3349316964 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdd24017020 con 0x7fdd340c8190 2024-03-09T08:52:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.374+0000 7fdd33200640 1 -- 172.21.15.67:0/3349316964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7fdd3406c9d0 con 0x7fdd340c8190 2024-03-09T08:52:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.375+0000 7fdd2a000640 1 -- 172.21.15.67:0/3349316964 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7fdd24058e50 con 0x7fdd340c8190 2024-03-09T08:52:53.375 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T08:52:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 -- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd1006b500 msgr2=0x7fdd1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd1006b500 0x7fdd1006d9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c036000 tx=0x7fdd1c008030 comp rx=0 tx=0).stop 2024-03-09T08:52:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 -- 172.21.15.67:0/3349316964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd340c8190 msgr2=0x7fdd341ebfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd340c8190 0x7fdd341ebfe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400e7e0 tx=0x7fdd2400eca0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 -- 172.21.15.67:0/3349316964 shutdown_connections 2024-03-09T08:52:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd1006b500 0x7fdd1006d9c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd34152f20 0x7fdd3414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd340ca2c0 0x7fdd34152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 --2- 172.21.15.67:0/3349316964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd340c8190 0x7fdd341ebfe0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 -- 172.21.15.67:0/3349316964 >> 172.21.15.67:0/3349316964 conn(0x7fdd340ad060 msgr2=0x7fdd340ca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:53.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 -- 172.21.15.67:0/3349316964 shutdown_connections 2024-03-09T08:52:53.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.378+0000 7fdd33200640 1 -- 172.21.15.67:0/3349316964 wait complete. 2024-03-09T08:52:53.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T08:52:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.619+0000 7f338a000640 1 Processor -- start 2024-03-09T08:52:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.620+0000 7f338a000640 1 -- start start 2024-03-09T08:52:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.621+0000 7f338a000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3384153370 0x7f33840c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.621+0000 7f338a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3384153750 0x7f33840c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.621+0000 7f338a000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33840c4010 con 0x7f338406dc00 2024-03-09T08:52:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.621+0000 7f338a000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3384156dc0 con 0x7f3384153370 2024-03-09T08:52:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.621+0000 7f338a000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33840c42f0 con 0x7f3384153750 2024-03-09T08:52:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.622+0000 7f337aa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3384153750 0x7f33840c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.622+0000 7f337aa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3384153750 0x7f33840c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:42018/0 (socket says 172.21.15.67:42018) 2024-03-09T08:52:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.622+0000 7f337aa00640 1 -- 172.21.15.67:0/2014115487 learned_addr learned my addr 172.21.15.67:0/2014115487 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.622+0000 7f3383400640 1 --2- 172.21.15.67:0/2014115487 >> v2:172.21.15.134:3300/0 conn(0x7f3384153370 0x7f33840c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.623+0000 7f337aa00640 1 -- 172.21.15.67:0/2014115487 >> v2:172.21.15.134:3300/0 conn(0x7f3384153370 msgr2=0x7f33840c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.623+0000 7f337aa00640 1 --2- 172.21.15.67:0/2014115487 >> v2:172.21.15.134:3300/0 conn(0x7f3384153370 0x7f33840c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.623+0000 7f337aa00640 1 -- 172.21.15.67:0/2014115487 >> v1:172.21.15.134:6789/0 conn(0x7f338406dc00 legacy=0x7f33840729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.623+0000 7f337aa00640 1 -- 172.21.15.67:0/2014115487 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33840c5a50 con 0x7f3384153750 2024-03-09T08:52:53.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.624+0000 7f337aa00640 1 --2- 172.21.15.67:0/2014115487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3384153750 0x7f33840c2ee0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f336c009870 tx=0x7f336c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71191f2e8c398b32 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:53.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.625+0000 7f3382a00640 1 -- 172.21.15.67:0/2014115487 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c035020 con 0x7f3384153750 2024-03-09T08:52:53.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.625+0000 7f3382a00640 1 -- 172.21.15.67:0/2014115487 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336c031640 con 0x7f3384153750 2024-03-09T08:52:53.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.625+0000 7f3382a00640 1 -- 172.21.15.67:0/2014115487 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c02d040 con 0x7f3384153750 2024-03-09T08:52:53.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.625+0000 7f338a000640 1 -- 172.21.15.67:0/2014115487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3384153750 msgr2=0x7f33840c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.625+0000 7f338a000640 1 --2- 172.21.15.67:0/2014115487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3384153750 0x7f33840c2ee0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f336c009870 tx=0x7f336c02b8f0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.625+0000 7f338a000640 1 -- 172.21.15.67:0/2014115487 shutdown_connections 2024-03-09T08:52:53.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.625+0000 7f338a000640 1 --2- 172.21.15.67:0/2014115487 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3384153750 0x7f33840c2ee0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.625+0000 7f338a000640 1 --2- 172.21.15.67:0/2014115487 >> v2:172.21.15.134:3300/0 conn(0x7f3384153370 0x7f33840c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.625+0000 7f338a000640 1 -- 172.21.15.67:0/2014115487 >> 172.21.15.67:0/2014115487 conn(0x7f33840ab320 msgr2=0x7f33840ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:53.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.625+0000 7f338a000640 1 -- 172.21.15.67:0/2014115487 shutdown_connections 2024-03-09T08:52:53.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.625+0000 7f338a000640 1 -- 172.21.15.67:0/2014115487 wait complete. 2024-03-09T08:52:53.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.627+0000 7f338a000640 1 Processor -- start 2024-03-09T08:52:53.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.627+0000 7f338a000640 1 -- start start 2024-03-09T08:52:53.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.628+0000 7f338a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f338406dc00 0x7f33841ec330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.628+0000 7f338a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3384153370 0x7f33841ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.628+0000 7f338a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3384153750 0x7f3384154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.628+0000 7f338a000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33840c5d30 con 0x7f3384153370 2024-03-09T08:52:53.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.628+0000 7f338a000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3384073dc0 con 0x7f3384153750 2024-03-09T08:52:53.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.628+0000 7f338a000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33840c3420 con 0x7f338406dc00 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f337aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3384153370 0x7f33841ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f337aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3384153370 0x7f33841ef7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48656/0 (socket says 172.21.15.67:48656) 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f337aa00640 1 -- 172.21.15.67:0/2380883281 learned_addr learned my addr 172.21.15.67:0/2380883281 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f3383400640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f338406dc00 0x7f33841ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f3383e00640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3384153750 0x7f3384154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f337aa00640 1 -- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f338406dc00 msgr2=0x7f33841ec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f337aa00640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f338406dc00 0x7f33841ec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f337aa00640 1 -- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3384153750 msgr2=0x7f3384154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f337aa00640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3384153750 0x7f3384154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f337aa00640 1 -- 172.21.15.67:0/2380883281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33841eb870 con 0x7f3384153370 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f3383e00640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3384153750 0x7f3384154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.629+0000 7f337aa00640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3384153370 0x7f33841ef7f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f336c02b930 tx=0x7f336c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.630+0000 7f337be00640 1 -- 172.21.15.67:0/2380883281 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c040070 con 0x7f3384153370 2024-03-09T08:52:53.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.630+0000 7f337be00640 1 -- 172.21.15.67:0/2380883281 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336c03e070 con 0x7f3384153370 2024-03-09T08:52:53.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.630+0000 7f337be00640 1 -- 172.21.15.67:0/2380883281 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c035040 con 0x7f3384153370 2024-03-09T08:52:53.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.630+0000 7f338a000640 1 -- 172.21.15.67:0/2380883281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3384154370 con 0x7f3384153370 2024-03-09T08:52:53.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.630+0000 7f338a000640 1 -- 172.21.15.67:0/2380883281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33841ebb30 con 0x7f3384153370 2024-03-09T08:52:53.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.631+0000 7f338a000640 1 -- 172.21.15.67:0/2380883281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3384075a00 con 0x7f3384153370 2024-03-09T08:52:53.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.632+0000 7f337be00640 1 -- 172.21.15.67:0/2380883281 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f336c038020 con 0x7f3384153370 2024-03-09T08:52:53.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.633+0000 7f337be00640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f336006b4d0 0x7f336006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:53.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.633+0000 7f337be00640 1 -- 172.21.15.67:0/2380883281 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f336c0abcb0 con 0x7f3384153370 2024-03-09T08:52:53.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.634+0000 7f3383400640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f336006b4d0 0x7f336006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:53.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.635+0000 7f3383400640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f336006b4d0 0x7f336006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3374011340 tx=0x7f337400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:53.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.638+0000 7f337be00640 1 -- 172.21.15.67:0/2380883281 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f336c07a890 con 0x7f3384153370 2024-03-09T08:52:53.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.911+0000 7f338a000640 1 -- 172.21.15.67:0/2380883281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33840c7d60 con 0x7f3384153370 2024-03-09T08:52:53.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.912+0000 7f337be00640 1 -- 172.21.15.67:0/2380883281 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6247 (secure 0 0 0) 0x7f336c07a190 con 0x7f3384153370 2024-03-09T08:52:53.908 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:53.908 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:17.736973+0000","modified":"2024-03-09T08:52:21.931108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T08:52:53.909 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 14 2024-03-09T08:52:53.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.914+0000 7f338a000640 1 -- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f336006b4d0 msgr2=0x7f336006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.914+0000 7f338a000640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f336006b4d0 0x7f336006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3374011340 tx=0x7f337400b040 comp rx=0 tx=0).stop 2024-03-09T08:52:53.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.914+0000 7f338a000640 1 -- 172.21.15.67:0/2380883281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3384153370 msgr2=0x7f33841ef7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:53.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.914+0000 7f338a000640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3384153370 0x7f33841ef7f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f336c02b930 tx=0x7f336c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.915+0000 7f338a000640 1 -- 172.21.15.67:0/2380883281 shutdown_connections 2024-03-09T08:52:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.915+0000 7f338a000640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f336006b4d0 0x7f336006d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.915+0000 7f338a000640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3384153750 0x7f3384154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.915+0000 7f338a000640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3384153370 0x7f33841ef7f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.915+0000 7f338a000640 1 --2- 172.21.15.67:0/2380883281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f338406dc00 0x7f33841ec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.915+0000 7f338a000640 1 -- 172.21.15.67:0/2380883281 >> 172.21.15.67:0/2380883281 conn(0x7f33840ab320 msgr2=0x7f338406c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.915+0000 7f338a000640 1 -- 172.21.15.67:0/2380883281 shutdown_connections 2024-03-09T08:52:53.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:53.915+0000 7f338a000640 1 -- 172.21.15.67:0/2380883281 wait complete. 2024-03-09T08:52:53.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:52:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.150+0000 7fcdcbe00640 1 Processor -- start 2024-03-09T08:52:54.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.150+0000 7fcdcbe00640 1 -- start start 2024-03-09T08:52:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.151+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc0c5e40 0x7fcdcc156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.151+0000 7fcdcbe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdcc157000 con 0x7fcdcc0cadb0 2024-03-09T08:52:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.151+0000 7fcdcbe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdcc0ae430 con 0x7fcdcc153300 2024-03-09T08:52:54.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.151+0000 7fcdcbe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0ae6f0 con 0x7fcdcc0c5e40 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.152+0000 7fcdcb400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcdcc0cadb0 0x7fcdcc156550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49076/0 (socket says 172.21.15.67:49076) 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.152+0000 7fcdcb400640 1 -- 172.21.15.67:0/3549364637 learned_addr learned my addr 172.21.15.67:0/3549364637 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.152+0000 7fcdc3e00640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc0c5e40 0x7fcdcc156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.152+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188485018 0 0) 0x7fcdcc157000 con 0x7fcdcc0cadb0 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.152+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdb0003520 con 0x7fcdcc0cadb0 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3438758493 0 0) 0x7fcdb0003520 con 0x7fcdcc0cadb0 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc0c5e40 msgr2=0x7fcdcc156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc0c5e40 0x7fcdcc156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 >> v1:172.21.15.134:6789/0 conn(0x7fcdcc153300 legacy=0x7fcdcc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc155b10 con 0x7fcdcc0cadb0 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fcdc40019f0 con 0x7fcdcc0cadb0 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 >> v1:172.21.15.67:6789/0 conn(0x7fcdcc0cadb0 legacy=0x7fcdcc156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdb0003a50 0x7fcdb0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0004800 0x7fcdb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc0c5e40 0x7fcdb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0ae6f0 con 0x7fcdb0003a50 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb000ba70 con 0x7fcdb0004800 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb000bc40 con 0x7fcdcc0c5e40 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdcaa00640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdb0003a50 0x7fcdb0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdc3e00640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0004800 0x7fcdb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdcb400640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc0c5e40 0x7fcdb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdcaa00640 1 -- 172.21.15.67:0/3549364637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc0c5e40 msgr2=0x7fcdb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.153+0000 7fcdcaa00640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc0c5e40 0x7fcdb00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.154+0000 7fcdcaa00640 1 -- 172.21.15.67:0/3549364637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0004800 msgr2=0x7fcdb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.154+0000 7fcdcaa00640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0004800 0x7fcdb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.154+0000 7fcdcaa00640 1 -- 172.21.15.67:0/3549364637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0003520 con 0x7fcdb0003a50 2024-03-09T08:52:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.155+0000 7fcdcaa00640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdb0003a50 0x7fcdb0003e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc001860 tx=0x7fcdb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c962f600da80fe5a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.155+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc001430 con 0x7fcdb0003a50 2024-03-09T08:52:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.155+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc033610 con 0x7fcdb0003a50 2024-03-09T08:52:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.155+0000 7fcdca000640 1 -- 172.21.15.67:0/3549364637 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc033a00 con 0x7fcdb0003a50 2024-03-09T08:52:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.156+0000 7fcdcbe00640 1 -- 172.21.15.67:0/3549364637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdb0003a50 msgr2=0x7fcdb0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.156+0000 7fcdcbe00640 1 --2- 172.21.15.67:0/3549364637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdb0003a50 0x7fcdb0003e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc001860 tx=0x7fcdb0005830 comp rx=0 tx=0).stop 2024-03-09T08:52:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.156+0000 7fcdcb400640 1 -- 172.21.15.67:0/3549364637 reap_dead start 2024-03-09T08:52:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.156+0000 7fcdcbe00640 1 -- 172.21.15.67:0/3549364637 shutdown_connections 2024-03-09T08:52:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.156+0000 7fcdcbe00640 1 -- 172.21.15.67:0/3549364637 >> 172.21.15.67:0/3549364637 conn(0x7fcdcc0732d0 msgr2=0x7fcdcc0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.156+0000 7fcdcbe00640 1 -- 172.21.15.67:0/3549364637 shutdown_connections 2024-03-09T08:52:54.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.156+0000 7fcdcbe00640 1 -- 172.21.15.67:0/3549364637 wait complete. 2024-03-09T08:52:54.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.158+0000 7fcdcbe00640 1 Processor -- start 2024-03-09T08:52:54.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.158+0000 7fcdcbe00640 1 -- start start 2024-03-09T08:52:54.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0003a50 0x7fcdcc1eec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdcc06ccb0 0x7fcdcc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc06bc90 0x7fcdcc14b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdcbe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0c59e0 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdcbe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc159e90 con 0x7fcdb0003a50 2024-03-09T08:52:54.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdcbe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0b0710 con 0x7fcdcc06bc90 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdc3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdcc06ccb0 0x7fcdcc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdc3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdcc06ccb0 0x7fcdcc06b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48668/0 (socket says 172.21.15.67:48668) 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdc3e00640 1 -- 172.21.15.67:0/1705021106 learned_addr learned my addr 172.21.15.67:0/1705021106 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdcaa00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0003a50 0x7fcdcc1eec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdcb400640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc06bc90 0x7fcdcc14b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdc3e00640 1 -- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc06bc90 msgr2=0x7fcdcc14b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdc3e00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc06bc90 0x7fcdcc14b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdc3e00640 1 -- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0003a50 msgr2=0x7fcdcc1eec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdc3e00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0003a50 0x7fcdcc1eec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.159+0000 7fcdc3e00640 1 -- 172.21.15.67:0/1705021106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc0c6df0 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.160+0000 7fcdc3e00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdcc06ccb0 0x7fcdcc06b750 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4011630 tx=0x7fcdb4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.160+0000 7fcdc2a00640 1 -- 172.21.15.67:0/1705021106 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400f8c0 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.160+0000 7fcdc2a00640 1 -- 172.21.15.67:0/1705021106 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb400fa60 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.160+0000 7fcdcbe00640 1 -- 172.21.15.67:0/1705021106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdcc14c360 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.160+0000 7fcdc2a00640 1 -- 172.21.15.67:0/1705021106 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400fe10 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.160+0000 7fcdcbe00640 1 -- 172.21.15.67:0/1705021106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdcc1ef4d0 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.162+0000 7fcdcbe00640 1 -- 172.21.15.67:0/1705021106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdcc0ca8f0 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.162+0000 7fcdc2a00640 1 -- 172.21.15.67:0/1705021106 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcdb4015460 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.163+0000 7fcdc2a00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcda806b500 0x7fcda806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.163+0000 7fcdc2a00640 1 -- 172.21.15.67:0/1705021106 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fcdb408ada0 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.164+0000 7fcdcaa00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcda806b500 0x7fcda806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.165+0000 7fcdcaa00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcda806b500 0x7fcda806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc036000 tx=0x7fcdbc031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:54.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.167+0000 7fcdc2a00640 1 -- 172.21.15.67:0/1705021106 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcdcc0ca8f0 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.399+0000 7fcdcbe00640 1 -- 172.21.15.67:0/1705021106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdcc04f3c0 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.402+0000 7fcdc2a00640 1 -- 172.21.15.67:0/1705021106 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20307 (secure 0 0 0) 0x7fcdb4019030 con 0x7fcdcc06ccb0 2024-03-09T08:52:54.399 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:54.399 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":24,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:30.565529+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:52:54.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 -- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcda806b500 msgr2=0x7fcda806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcda806b500 0x7fcda806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc036000 tx=0x7fcdbc031000 comp rx=0 tx=0).stop 2024-03-09T08:52:54.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 -- 172.21.15.67:0/1705021106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdcc06ccb0 msgr2=0x7fcdcc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdcc06ccb0 0x7fcdcc06b750 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4011630 tx=0x7fcdb4011af0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 -- 172.21.15.67:0/1705021106 shutdown_connections 2024-03-09T08:52:54.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcda806b500 0x7fcda806d9c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdcc06bc90 0x7fcdcc14b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdcc06ccb0 0x7fcdcc06b750 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 --2- 172.21.15.67:0/1705021106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdb0003a50 0x7fcdcc1eec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 -- 172.21.15.67:0/1705021106 >> 172.21.15.67:0/1705021106 conn(0x7fcdcc0732d0 msgr2=0x7fcdcc0ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:54.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 -- 172.21.15.67:0/1705021106 shutdown_connections 2024-03-09T08:52:54.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.406+0000 7fcdcbe00640 1 -- 172.21.15.67:0/1705021106 wait complete. 2024-03-09T08:52:54.422 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T08:52:54.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.645+0000 7f9dc2600640 1 Processor -- start 2024-03-09T08:52:54.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.645+0000 7f9dc2600640 1 -- start start 2024-03-09T08:52:54.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.646+0000 7f9dc2600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9dbc073180 0x7f9dbc0c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.646+0000 7f9dc2600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0c5320 0x7f9dbc1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9dc2600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dbc155b30 con 0x7f9dbc06ed50 2024-03-09T08:52:54.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9dc2600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dbc155df0 con 0x7f9dbc073180 2024-03-09T08:52:54.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9dc2600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc1560d0 con 0x7f9dbc0c5320 2024-03-09T08:52:54.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9db3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9dbc06ed50 0x7f9dbc1592d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49088/0 (socket says 172.21.15.67:49088) 2024-03-09T08:52:54.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9dc1200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9dbc073180 0x7f9dbc0c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9db3e00640 1 -- 172.21.15.67:0/823369633 learned_addr learned my addr 172.21.15.67:0/823369633 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:54.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9dc1c00640 1 --2- 172.21.15.67:0/823369633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0c5320 0x7f9dbc1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9dc1200640 1 -- 172.21.15.67:0/823369633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0c5320 msgr2=0x7f9dbc1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9dc1200640 1 --2- 172.21.15.67:0/823369633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0c5320 0x7f9dbc1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9dc1200640 1 -- 172.21.15.67:0/823369633 >> v1:172.21.15.67:6789/0 conn(0x7f9dbc06ed50 legacy=0x7f9dbc1592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9dc1200640 1 -- 172.21.15.67:0/823369633 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc1563b0 con 0x7f9dbc073180 2024-03-09T08:52:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.647+0000 7f9dbbe00640 1 -- 172.21.15.67:0/823369633 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778195870 0 0) 0x7f9dbc155b30 con 0x7f9dbc06ed50 2024-03-09T08:52:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.649+0000 7f9dc1200640 1 --2- 172.21.15.67:0/823369633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc073180 0x7f9dbc0c4de0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0097a0 tx=0x7f9dac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9320da505738af3b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.649+0000 7f9dbbe00640 1 -- 172.21.15.67:0/823369633 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0315a0 con 0x7f9dbc073180 2024-03-09T08:52:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.649+0000 7f9dbbe00640 1 -- 172.21.15.67:0/823369633 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac037070 con 0x7f9dbc073180 2024-03-09T08:52:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.649+0000 7f9dbbe00640 1 -- 172.21.15.67:0/823369633 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac031960 con 0x7f9dbc073180 2024-03-09T08:52:54.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.649+0000 7f9dc2600640 1 -- 172.21.15.67:0/823369633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc073180 msgr2=0x7f9dbc0c4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.649+0000 7f9dc2600640 1 --2- 172.21.15.67:0/823369633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc073180 0x7f9dbc0c4de0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0097a0 tx=0x7f9dac02b840 comp rx=0 tx=0).stop 2024-03-09T08:52:54.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.650+0000 7f9dc2600640 1 -- 172.21.15.67:0/823369633 shutdown_connections 2024-03-09T08:52:54.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.650+0000 7f9dc2600640 1 --2- 172.21.15.67:0/823369633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0c5320 0x7f9dbc1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.650+0000 7f9dc2600640 1 --2- 172.21.15.67:0/823369633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc073180 0x7f9dbc0c4de0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.650+0000 7f9dc2600640 1 -- 172.21.15.67:0/823369633 >> 172.21.15.67:0/823369633 conn(0x7f9dbc06c150 msgr2=0x7f9dbc06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:54.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.650+0000 7f9dc2600640 1 -- 172.21.15.67:0/823369633 shutdown_connections 2024-03-09T08:52:54.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.650+0000 7f9dc2600640 1 -- 172.21.15.67:0/823369633 wait complete. 2024-03-09T08:52:54.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.651+0000 7f9dc2600640 1 Processor -- start 2024-03-09T08:52:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.651+0000 7f9dc2600640 1 -- start start 2024-03-09T08:52:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.652+0000 7f9dc2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc06ed50 0x7f9dbc14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.652+0000 7f9dc2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc073180 0x7f9dbc071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.652+0000 7f9dc2600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0c5320 0x7f9dbc154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.652+0000 7f9dc2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc159f20 con 0x7f9dbc06ed50 2024-03-09T08:52:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.652+0000 7f9dc2600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc0c3920 con 0x7f9dbc073180 2024-03-09T08:52:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.652+0000 7f9dc2600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc073560 con 0x7f9dbc0c5320 2024-03-09T08:52:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9dc1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc06ed50 0x7f9dbc14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9dc1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc06ed50 0x7f9dbc14c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48690/0 (socket says 172.21.15.67:48690) 2024-03-09T08:52:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9dc1200640 1 -- 172.21.15.67:0/3305694614 learned_addr learned my addr 172.21.15.67:0/3305694614 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9db3e00640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc073180 0x7f9dbc071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9dc1c00640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0c5320 0x7f9dbc154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9dc1200640 1 -- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0c5320 msgr2=0x7f9dbc154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9dc1200640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0c5320 0x7f9dbc154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9dc1200640 1 -- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc073180 msgr2=0x7f9dbc071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9dc1200640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc073180 0x7f9dbc071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9dc1200640 1 -- 172.21.15.67:0/3305694614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc0723b0 con 0x7f9dbc06ed50 2024-03-09T08:52:54.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9db3e00640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc073180 0x7f9dbc071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:54.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9dc1c00640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0c5320 0x7f9dbc154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:54.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.653+0000 7f9dc1200640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc06ed50 0x7f9dbc14c840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9dac034040 tx=0x7f9dac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.654+0000 7f9db9600640 1 -- 172.21.15.67:0/3305694614 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac004230 con 0x7f9dbc06ed50 2024-03-09T08:52:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.654+0000 7f9db9600640 1 -- 172.21.15.67:0/3305694614 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac0043f0 con 0x7f9dbc06ed50 2024-03-09T08:52:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.654+0000 7f9db9600640 1 -- 172.21.15.67:0/3305694614 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac003850 con 0x7f9dbc06ed50 2024-03-09T08:52:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.654+0000 7f9dc2600640 1 -- 172.21.15.67:0/3305694614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dbc150d40 con 0x7f9dbc06ed50 2024-03-09T08:52:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.654+0000 7f9dc2600640 1 -- 172.21.15.67:0/3305694614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dbc14e320 con 0x7f9dbc06ed50 2024-03-09T08:52:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.655+0000 7f9dc2600640 1 -- 172.21.15.67:0/3305694614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dbc04f3c0 con 0x7f9dbc06ed50 2024-03-09T08:52:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.656+0000 7f9db9600640 1 -- 172.21.15.67:0/3305694614 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9dac0039f0 con 0x7f9dbc06ed50 2024-03-09T08:52:54.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.657+0000 7f9db9600640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d9806b4d0 0x7f9d9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:54.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.657+0000 7f9db9600640 1 -- 172.21.15.67:0/3305694614 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f9dac02f030 con 0x7f9dbc06ed50 2024-03-09T08:52:54.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.657+0000 7f9db3e00640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d9806b4d0 0x7f9d9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:54.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.658+0000 7f9db3e00640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d9806b4d0 0x7f9d9806d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc073e20 tx=0x7f9da4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:54.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.661+0000 7f9db9600640 1 -- 172.21.15.67:0/3305694614 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9dac041070 con 0x7f9dbc06ed50 2024-03-09T08:52:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.910+0000 7f9dc2600640 1 -- 172.21.15.67:0/3305694614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dbc159720 con 0x7f9dbc06ed50 2024-03-09T08:52:54.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.912+0000 7f9db9600640 1 -- 172.21.15.67:0/3305694614 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6247 (secure 0 0 0) 0x7f9dac044340 con 0x7f9dbc06ed50 2024-03-09T08:52:54.908 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:52:54.908 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:17.736973+0000","modified":"2024-03-09T08:52:21.931108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T08:52:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 14 2024-03-09T08:52:54.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.915+0000 7f9dc2600640 1 -- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d9806b4d0 msgr2=0x7f9d9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.915+0000 7f9dc2600640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d9806b4d0 0x7f9d9806d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc073e20 tx=0x7f9da4009040 comp rx=0 tx=0).stop 2024-03-09T08:52:54.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.915+0000 7f9dc2600640 1 -- 172.21.15.67:0/3305694614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc06ed50 msgr2=0x7f9dbc14c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:54.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.916+0000 7f9dc2600640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc06ed50 0x7f9dbc14c840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9dac034040 tx=0x7f9dac0046c0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.916+0000 7f9dc2600640 1 -- 172.21.15.67:0/3305694614 shutdown_connections 2024-03-09T08:52:54.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.916+0000 7f9dc2600640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d9806b4d0 0x7f9d9806d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.916+0000 7f9dc2600640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dbc0c5320 0x7f9dbc154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.916+0000 7f9dc2600640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dbc073180 0x7f9dbc071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.916+0000 7f9dc2600640 1 --2- 172.21.15.67:0/3305694614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dbc06ed50 0x7f9dbc14c840 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:54.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.916+0000 7f9dc2600640 1 -- 172.21.15.67:0/3305694614 >> 172.21.15.67:0/3305694614 conn(0x7f9dbc06c150 msgr2=0x7f9dbc0c2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:54.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.916+0000 7f9dc2600640 1 -- 172.21.15.67:0/3305694614 shutdown_connections 2024-03-09T08:52:54.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:54.916+0000 7f9dc2600640 1 -- 172.21.15.67:0/3305694614 wait complete. 2024-03-09T08:52:55.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-09T08:52:29.037519+0000', 'modified': '2024-03-09T08:52:33.151162+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'c', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.67:6834/1699514524', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6832', 'nonce': 1699514524}, {'type': 'v1', 'addr': '172.21.15.67:6834', 'nonce': 1699514524}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'b', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4279, 'qdb_cluster': [4279]} 2024-03-09T08:52:55.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-09T08:52:55.342 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-09T08:52:55.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.573+0000 7f7147200640 1 Processor -- start 2024-03-09T08:52:55.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.574+0000 7f7147200640 1 -- start start 2024-03-09T08:52:55.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.575+0000 7f7147200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714806c1d0 0x7f714806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.575+0000 7f7147200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71480c9890 con 0x7f714814fd90 2024-03-09T08:52:55.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.575+0000 7f7147200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71480c5f80 con 0x7f71480c9380 2024-03-09T08:52:55.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.575+0000 7f7147200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714806ba60 con 0x7f714806c1d0 2024-03-09T08:52:55.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.575+0000 7f7145e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f714814fd90 0x7f7148156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49092/0 (socket says 172.21.15.67:49092) 2024-03-09T08:52:55.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.575+0000 7f7145e00640 1 -- 172.21.15.67:0/4065532766 learned_addr learned my addr 172.21.15.67:0/4065532766 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:55.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7146800640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714806c1d0 0x7f714806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701995806 0 0) 0x7f71480c9890 con 0x7f714814fd90 2024-03-09T08:52:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f712c003520 con 0x7f714814fd90 2024-03-09T08:52:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 511856282 0 0) 0x7f712c003520 con 0x7f714814fd90 2024-03-09T08:52:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714806c1d0 msgr2=0x7f714806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714806c1d0 0x7f714806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 >> v1:172.21.15.134:6789/0 conn(0x7f71480c9380 legacy=0x7f71480acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71480c6240 con 0x7f714814fd90 2024-03-09T08:52:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f7138001670 con 0x7f714814fd90 2024-03-09T08:52:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 >> v1:172.21.15.67:6789/0 conn(0x7f714814fd90 legacy=0x7f7148156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714806c1d0 0x7f712c003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f712c003fd0 0x7f712c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 0x7f712c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714806ba60 con 0x7f712c004d40 2024-03-09T08:52:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c00bc50 con 0x7f712c003fd0 2024-03-09T08:52:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.576+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c00be50 con 0x7f714806c1d0 2024-03-09T08:52:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.577+0000 7f7146800640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 0x7f712c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.577+0000 7f7145400640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f712c003fd0 0x7f712c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.577+0000 7f7145e00640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714806c1d0 0x7f712c003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.577+0000 7f7146800640 1 -- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714806c1d0 msgr2=0x7f712c003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.577+0000 7f7146800640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714806c1d0 0x7f712c003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.577+0000 7f7146800640 1 -- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f712c003fd0 msgr2=0x7f712c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.577+0000 7f7146800640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f712c003fd0 0x7f712c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.577+0000 7f7146800640 1 -- 172.21.15.67:0/4065532766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c003520 con 0x7f712c004d40 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.577+0000 7f7145e00640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714806c1d0 0x7f712c003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.578+0000 7f7146800640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 0x7f712c0051c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7140009fc0 tx=0x7f712c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717cabf59a0a121e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.579+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140039070 con 0x7f712c004d40 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.579+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714002cc20 con 0x7f712c004d40 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.579+0000 7f7144a00640 1 -- 172.21.15.67:0/4065532766 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140009c10 con 0x7f712c004d40 2024-03-09T08:52:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.579+0000 7f7147200640 1 -- 172.21.15.67:0/4065532766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 msgr2=0x7f712c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:55.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.579+0000 7f7147200640 1 --2- 172.21.15.67:0/4065532766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 0x7f712c0051c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7140009fc0 tx=0x7f712c005830 comp rx=0 tx=0).stop 2024-03-09T08:52:55.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.579+0000 7f7146800640 1 -- 172.21.15.67:0/4065532766 reap_dead start 2024-03-09T08:52:55.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.579+0000 7f7147200640 1 -- 172.21.15.67:0/4065532766 shutdown_connections 2024-03-09T08:52:55.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.579+0000 7f7147200640 1 -- 172.21.15.67:0/4065532766 >> 172.21.15.67:0/4065532766 conn(0x7f71480cb480 msgr2=0x7f71480cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:55.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.579+0000 7f7147200640 1 -- 172.21.15.67:0/4065532766 shutdown_connections 2024-03-09T08:52:55.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.579+0000 7f7147200640 1 -- 172.21.15.67:0/4065532766 wait complete. 2024-03-09T08:52:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.581+0000 7f7147200640 1 Processor -- start 2024-03-09T08:52:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.581+0000 7f7147200640 1 -- start start 2024-03-09T08:52:55.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.582+0000 7f7147200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 0x7f71481eed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.582+0000 7f7147200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7148156db0 0x7f7148150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.582+0000 7f7147200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7148151270 0x7f7148149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.582+0000 7f7147200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480ad610 con 0x7f712c004d40 2024-03-09T08:52:55.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.582+0000 7f7147200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714806bcb0 con 0x7f7148156db0 2024-03-09T08:52:55.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.582+0000 7f7147200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71480ad790 con 0x7f7148151270 2024-03-09T08:52:55.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.582+0000 7f7145e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 0x7f71481eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.582+0000 7f7145e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 0x7f71481eed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48700/0 (socket says 172.21.15.67:48700) 2024-03-09T08:52:55.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.582+0000 7f7145e00640 1 -- 172.21.15.67:0/2654796780 learned_addr learned my addr 172.21.15.67:0/2654796780 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f7145400640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7148156db0 0x7f7148150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f7146800640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7148151270 0x7f7148149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f7145e00640 1 -- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7148151270 msgr2=0x7f7148149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f7145e00640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7148151270 0x7f7148149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f7145e00640 1 -- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7148156db0 msgr2=0x7f7148150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f7145e00640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7148156db0 0x7f7148150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f7145e00640 1 -- 172.21.15.67:0/2654796780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7148151d30 con 0x7f712c004d40 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f7146800640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7148151270 0x7f7148149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f7145e00640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 0x7f71481eed80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f713800d8b0 tx=0x7f713800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f713e000640 1 -- 172.21.15.67:0/2654796780 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138019070 con 0x7f712c004d40 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f713e000640 1 -- 172.21.15.67:0/2654796780 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71380074e0 con 0x7f712c004d40 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f713e000640 1 -- 172.21.15.67:0/2654796780 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71380049e0 con 0x7f712c004d40 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.583+0000 7f7147200640 1 -- 172.21.15.67:0/2654796780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7148152d60 con 0x7f712c004d40 2024-03-09T08:52:55.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.584+0000 7f7147200640 1 -- 172.21.15.67:0/2654796780 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71481ef5c0 con 0x7f712c004d40 2024-03-09T08:52:55.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.586+0000 7f713e000640 1 -- 172.21.15.67:0/2654796780 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f713801c070 con 0x7f712c004d40 2024-03-09T08:52:55.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.587+0000 7f713e000640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712406b500 0x7f712406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.587+0000 7f713e000640 1 -- 172.21.15.67:0/2654796780 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f7138089ec0 con 0x7f712c004d40 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.587+0000 7f7145400640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712406b500 0x7f712406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.588+0000 7f7145400640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712406b500 0x7f712406d9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f71480cbe90 tx=0x7f7130009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.589+0000 7f714f837740 1 Processor -- start 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.589+0000 7f714f837740 1 -- start start 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.591+0000 7f714f837740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f326090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.591+0000 7f714f837740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f2eb2c0 0x55770f325300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.591+0000 7f714f837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f2ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.591+0000 7f714f837740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55770f24e6f0 con 0x55770f2eb2c0 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.591+0000 7f714f837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55770f1f79e0 con 0x55770f328090 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.591+0000 7f714f837740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x55770f24a790 con 0x55770f324f20 2024-03-09T08:52:55.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.591+0000 7f7145400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f2eb2c0 0x55770f325300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.591+0000 7f7145400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f2eb2c0 0x55770f325300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48714/0 (socket says 172.21.15.67:48714) 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.591+0000 7f7145400640 1 -- 172.21.15.67:0/3830318213 learned_addr learned my addr 172.21.15.67:0/3830318213 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.591+0000 7f7145e00640 1 --2- 172.21.15.67:0/3830318213 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f326090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.592+0000 7f7146800640 1 --2- 172.21.15.67:0/3830318213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f2ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.592+0000 7f7145400640 1 -- 172.21.15.67:0/3830318213 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 msgr2=0x55770f326090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.592+0000 7f7145400640 1 --2- 172.21.15.67:0/3830318213 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f326090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.592+0000 7f7145400640 1 -- 172.21.15.67:0/3830318213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 msgr2=0x55770f2ea040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.592+0000 7f7145400640 1 --2- 172.21.15.67:0/3830318213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f2ea040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.592+0000 7f7145400640 1 -- 172.21.15.67:0/3830318213 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55770f1eb720 con 0x55770f2eb2c0 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.592+0000 7f7146800640 1 --2- 172.21.15.67:0/3830318213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f2ea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.592+0000 7f7145400640 1 --2- 172.21.15.67:0/3830318213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f2eb2c0 0x55770f325300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7130001540 tx=0x7f7130013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc33f6339e3b033 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.592+0000 7f7136600640 1 -- 172.21.15.67:0/3830318213 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130027070 con 0x55770f2eb2c0 2024-03-09T08:52:55.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.592+0000 7f7136600640 1 -- 172.21.15.67:0/3830318213 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71300146c0 con 0x55770f2eb2c0 2024-03-09T08:52:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.592+0000 7f7136600640 1 -- 172.21.15.67:0/3830318213 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130014a70 con 0x55770f2eb2c0 2024-03-09T08:52:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.593+0000 7f714f837740 1 -- 172.21.15.67:0/3830318213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f2eb2c0 msgr2=0x55770f325300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.593+0000 7f714f837740 1 --2- 172.21.15.67:0/3830318213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f2eb2c0 0x55770f325300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7130001540 tx=0x7f7130013000 comp rx=0 tx=0).stop 2024-03-09T08:52:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.593+0000 7f714f837740 1 -- 172.21.15.67:0/3830318213 shutdown_connections 2024-03-09T08:52:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.593+0000 7f714f837740 1 --2- 172.21.15.67:0/3830318213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f2ea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.593+0000 7f714f837740 1 --2- 172.21.15.67:0/3830318213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f2eb2c0 0x55770f325300 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.593+0000 7f714f837740 1 --2- 172.21.15.67:0/3830318213 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f326090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.593+0000 7f714f837740 1 -- 172.21.15.67:0/3830318213 >> 172.21.15.67:0/3830318213 conn(0x55770f2e6900 msgr2=0x55770f30b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.593+0000 7f714f837740 1 -- 172.21.15.67:0/3830318213 shutdown_connections 2024-03-09T08:52:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.593+0000 7f714f837740 1 -- 172.21.15.67:0/3830318213 wait complete. 2024-03-09T08:52:55.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.594+0000 7f714f837740 1 -- start start 2024-03-09T08:52:55.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.594+0000 7f714f837740 1 Processor -- start 2024-03-09T08:52:55.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.595+0000 7f714f837740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f31ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.595+0000 7f714f837740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f328090 0x55770f2e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.595+0000 7f714f837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f3396f0 0x55770f2fd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.595+0000 7f714f837740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55770f327c00 con 0x55770f328090 2024-03-09T08:52:55.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.595+0000 7f714f837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55770f20ae20 con 0x55770f3396f0 2024-03-09T08:52:55.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.595+0000 7f714f837740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x55770f3276c0 con 0x55770f324f20 2024-03-09T08:52:55.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.595+0000 7f714f837740 10 client.0 _ll_register_callbacks cb 0x55770f2458d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T08:52:55.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.595+0000 7f7145400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f328090 0x55770f2e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.595+0000 7f7145400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f328090 0x55770f2e4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48730/0 (socket says 172.21.15.67:48730) 2024-03-09T08:52:55.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.595+0000 7f7145400640 1 -- 172.21.15.67:0/807679428 learned_addr learned my addr 172.21.15.67:0/807679428 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:55.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f7145e00640 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f31ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f7146800640 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f3396f0 0x55770f2fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f7145400640 1 -- 172.21.15.67:0/807679428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 msgr2=0x55770f31ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:55.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f7145400640 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f31ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f7145400640 1 -- 172.21.15.67:0/807679428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f3396f0 msgr2=0x55770f2fd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:55.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f7145400640 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f3396f0 0x55770f2fd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f7145400640 1 -- 172.21.15.67:0/807679428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55770ef632d0 con 0x55770f328090 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f7146800640 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f3396f0 0x55770f2fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f7145400640 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f328090 0x55770f2e4a00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55770f325bd0 tx=0x7f7130001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104c4b54c09a97f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f712aa00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f712aa00640 1 -- 172.21.15.67:0/807679428 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130027070 con 0x55770f328090 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.596+0000 7f712aa00640 1 -- 172.21.15.67:0/807679428 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f7130007020 con 0x55770f328090 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f714f837740 1 -- 172.21.15.67:0/807679428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71300146c0 con 0x55770f328090 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f712aa00640 1 -- 172.21.15.67:0/807679428 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7130039050 con 0x55770f328090 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f714f837740 10 client.4863 fetch_fsmap learned FSMap version 14 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f714f837740 1 -- 172.21.15.67:0/807679428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x55770f02d6b0 con 0x55770f328090 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f712aa00640 1 -- 172.21.15.67:0/807679428 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71300146c0 con 0x55770f328090 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f712aa00640 1 -- 172.21.15.67:0/807679428 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 14) v1 ==== 3918+0+0 (secure 0 0 0) 0x7f713003c020 con 0x55770f328090 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f714f837740 10 client.4863 fetch_fsmap finished waiting for FSMap version 14 2024-03-09T08:52:55.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f714f837740 10 client.4863 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-09T08:52:55.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f714f837740 20 client.4863 populate_metadata read hostname 'smithi067' 2024-03-09T08:52:55.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f714f837740 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.67:6832/1699514524,v1:172.21.15.67:6834/1699514524] conn(0x55770f31d970 0x55770f344d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f714f837740 4 client.4863 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-09T08:52:55.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.597+0000 7f714f837740 1 -- 172.21.15.67:0/807679428 --> [v2:172.21.15.67:6832/1699514524,v1:172.21.15.67:6834/1699514524] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55770f25bc30 con 0x55770f31d970 2024-03-09T08:52:55.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.598+0000 7f7145e00640 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.67:6832/1699514524,v1:172.21.15.67:6834/1699514524] conn(0x55770f31d970 0x55770f344d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.598+0000 7f7145e00640 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.67:6832/1699514524,v1:172.21.15.67:6834/1699514524] conn(0x55770f31d970 0x55770f344d90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7603cddf0f7460c2 server_cookie=373546a7388eded9 in_seq=0 out_seq=0 2024-03-09T08:52:55.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.598+0000 7f712aa00640 10 client.4863 ms_handle_connect on v2:172.21.15.67:6832/1699514524 2024-03-09T08:52:55.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.599+0000 7f712aa00640 1 -- 172.21.15.67:0/807679428 <== mds.0 v2:172.21.15.67:6832/1699514524 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55770f25bc30 con 0x55770f31d970 2024-03-09T08:52:55.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.599+0000 7f712aa00640 10 client.4863 handle_command_reply: tid=0 2024-03-09T08:52:55.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.599+0000 7f714f837740 1 client.4863 shutdown 2024-03-09T08:52:55.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f714f837740 1 -- 172.21.15.67:0/807679428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f328090 msgr2=0x55770f2e4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f714f837740 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f328090 0x55770f2e4a00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55770f325bd0 tx=0x7f7130001190 comp rx=0 tx=0).stop 2024-03-09T08:52:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f714f837740 1 -- 172.21.15.67:0/807679428 shutdown_connections 2024-03-09T08:52:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f714f837740 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.67:6832/1699514524,v1:172.21.15.67:6834/1699514524] conn(0x55770f31d970 0x55770f344d90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f714f837740 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f3396f0 0x55770f2fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f714f837740 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f328090 0x55770f2e4a00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f714f837740 1 --2- 172.21.15.67:0/807679428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f31ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f714f837740 1 -- 172.21.15.67:0/807679428 >> 172.21.15.67:0/807679428 conn(0x55770f2eb2c0 msgr2=0x55770f30b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f712aa00640 0 client.4863 ms_handle_reset on v2:172.21.15.67:6832/1699514524 2024-03-09T08:52:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f714f837740 1 -- 172.21.15.67:0/807679428 shutdown_connections 2024-03-09T08:52:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f714f837740 1 -- 172.21.15.67:0/807679428 wait complete. 2024-03-09T08:52:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.600+0000 7f714f837740 20 client.4863 trim_cache size 0 max 16384 2024-03-09T08:52:55.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.618+0000 7f714f837740 1 Processor -- start 2024-03-09T08:52:55.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.618+0000 7f714f837740 1 -- start start 2024-03-09T08:52:55.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.619+0000 7f714f837740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f326810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.619+0000 7f714f837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f2e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.619+0000 7f714f837740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f3396f0 0x55770f2e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.619+0000 7f714f837740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71481fb690 con 0x55770f3396f0 2024-03-09T08:52:55.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.619+0000 7f714f837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71481ec410 con 0x55770f328090 2024-03-09T08:52:55.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.619+0000 7f714f837740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71481f9880 con 0x55770f324f20 2024-03-09T08:52:55.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.619+0000 7f7146800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f3396f0 0x55770f2e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.619+0000 7f7146800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f3396f0 0x55770f2e5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48742/0 (socket says 172.21.15.67:48742) 2024-03-09T08:52:55.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.619+0000 7f7146800640 1 -- 172.21.15.67:0/3583365998 learned_addr learned my addr 172.21.15.67:0/3583365998 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:55.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.620+0000 7f7145400640 1 --2- 172.21.15.67:0/3583365998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f2e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.620+0000 7f7145e00640 1 --2- 172.21.15.67:0/3583365998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f326810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.620+0000 7f7146800640 1 -- 172.21.15.67:0/3583365998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 msgr2=0x55770f326810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.620+0000 7f7146800640 1 --2- 172.21.15.67:0/3583365998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f326810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.620+0000 7f7146800640 1 -- 172.21.15.67:0/3583365998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 msgr2=0x55770f2e55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.620+0000 7f7146800640 1 --2- 172.21.15.67:0/3583365998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f2e55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.620+0000 7f7146800640 1 -- 172.21.15.67:0/3583365998 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55770f34a5a0 con 0x55770f3396f0 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.620+0000 7f7146800640 1 --2- 172.21.15.67:0/3583365998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f3396f0 0x55770f2e5b10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f714000e230 tx=0x7f714002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1480e32f618350ea server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.620+0000 7f7135c00640 1 -- 172.21.15.67:0/3583365998 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140044070 con 0x55770f3396f0 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.621+0000 7f7135c00640 1 -- 172.21.15.67:0/3583365998 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714003a040 con 0x55770f3396f0 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.621+0000 7f7135c00640 1 -- 172.21.15.67:0/3583365998 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140039070 con 0x55770f3396f0 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.621+0000 7f714f837740 1 -- 172.21.15.67:0/3583365998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f3396f0 msgr2=0x55770f2e5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.621+0000 7f714f837740 1 --2- 172.21.15.67:0/3583365998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f3396f0 0x55770f2e5b10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f714000e230 tx=0x7f714002e070 comp rx=0 tx=0).stop 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.621+0000 7f714f837740 1 -- 172.21.15.67:0/3583365998 shutdown_connections 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.621+0000 7f714f837740 1 --2- 172.21.15.67:0/3583365998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f3396f0 0x55770f2e5b10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.621+0000 7f714f837740 1 --2- 172.21.15.67:0/3583365998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f2e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.621+0000 7f714f837740 1 --2- 172.21.15.67:0/3583365998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f324f20 0x55770f326810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.621+0000 7f714f837740 1 -- 172.21.15.67:0/3583365998 >> 172.21.15.67:0/3583365998 conn(0x55770f2eb2c0 msgr2=0x55770f3081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.621+0000 7f714f837740 1 -- 172.21.15.67:0/3583365998 shutdown_connections 2024-03-09T08:52:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.621+0000 7f714f837740 1 -- 172.21.15.67:0/3583365998 wait complete. 2024-03-09T08:52:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.622+0000 7f714f837740 1 -- start start 2024-03-09T08:52:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.623+0000 7f714f837740 1 Processor -- start 2024-03-09T08:52:55.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.624+0000 7f714f837740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f324f20 0x55770f307eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.624+0000 7f714f837740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f3333b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.624+0000 7f714f837740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f3396f0 0x55770f3987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.624+0000 7f714f837740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55770f2e5220 con 0x55770f324f20 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.624+0000 7f714f837740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55770f336860 con 0x55770f328090 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.624+0000 7f714f837740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x55770f31d3c0 con 0x55770f3396f0 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.624+0000 7f714f837740 10 client.0 _ll_register_callbacks cb 0x7f7130025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.624+0000 7f7145e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f324f20 0x55770f307eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.624+0000 7f7145e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f324f20 0x55770f307eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48746/0 (socket says 172.21.15.67:48746) 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.624+0000 7f7145e00640 1 -- 172.21.15.67:0/321818135 learned_addr learned my addr 172.21.15.67:0/321818135 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f7146800640 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f3396f0 0x55770f3987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f7145400640 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f3333b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f7145e00640 1 -- 172.21.15.67:0/321818135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f3396f0 msgr2=0x55770f3987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f7145e00640 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f3396f0 0x55770f3987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f7145e00640 1 -- 172.21.15.67:0/321818135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 msgr2=0x55770f3333b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f7145e00640 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f3333b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f7145e00640 1 -- 172.21.15.67:0/321818135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55770f332860 con 0x55770f324f20 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f7146800640 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f3396f0 0x55770f3987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f7145e00640 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f324f20 0x55770f307eb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f713800b4a0 tx=0x7f7138000f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d86d9450a364f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f712aa00640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f712aa00640 1 -- 172.21.15.67:0/321818135 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138073070 con 0x55770f324f20 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f712aa00640 1 -- 172.21.15.67:0/321818135 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f713805c0c0 con 0x55770f324f20 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f714f837740 1 -- 172.21.15.67:0/321818135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f714003a040 con 0x55770f324f20 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.625+0000 7f712aa00640 1 -- 172.21.15.67:0/321818135 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7138060a10 con 0x55770f324f20 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.626+0000 7f714f837740 10 client.4869 fetch_fsmap learned FSMap version 14 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.626+0000 7f714f837740 1 -- 172.21.15.67:0/321818135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x55770f329560 con 0x55770f324f20 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.626+0000 7f712aa00640 1 -- 172.21.15.67:0/321818135 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714003a040 con 0x55770f324f20 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.626+0000 7f712aa00640 1 -- 172.21.15.67:0/321818135 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 14) v1 ==== 3918+0+0 (secure 0 0 0) 0x7f7138054a20 con 0x55770f324f20 2024-03-09T08:52:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.626+0000 7f714f837740 10 client.4869 fetch_fsmap finished waiting for FSMap version 14 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.626+0000 7f714f837740 10 client.4869 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.626+0000 7f714f837740 20 client.4869 populate_metadata read hostname 'smithi067' 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.626+0000 7f714f837740 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.67:6832/1699514524,v1:172.21.15.67:6834/1699514524] conn(0x55770f391f40 0x55770f392320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.626+0000 7f714f837740 4 client.4869 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.626+0000 7f714f837740 1 -- 172.21.15.67:0/321818135 --> [v2:172.21.15.67:6832/1699514524,v1:172.21.15.67:6834/1699514524] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55770f2e7060 con 0x55770f391f40 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.626+0000 7f7145400640 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.67:6832/1699514524,v1:172.21.15.67:6834/1699514524] conn(0x55770f391f40 0x55770f392320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.627+0000 7f7145400640 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.67:6832/1699514524,v1:172.21.15.67:6834/1699514524] conn(0x55770f391f40 0x55770f392320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa4bc7b4015816f server_cookie=f3ccc883c3d14e5f in_seq=0 out_seq=0 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.627+0000 7f712aa00640 10 client.4869 ms_handle_connect on v2:172.21.15.67:6832/1699514524 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.627+0000 7f712aa00640 1 -- 172.21.15.67:0/321818135 <== mds.0 v2:172.21.15.67:6832/1699514524 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55770f2e7060 con 0x55770f391f40 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.627+0000 7f712aa00640 10 client.4869 handle_command_reply: tid=0 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.627+0000 7f714f837740 1 client.4869 shutdown 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.627+0000 7f714f837740 1 -- 172.21.15.67:0/321818135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f324f20 msgr2=0x55770f307eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:55.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.627+0000 7f714f837740 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f324f20 0x55770f307eb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f713800b4a0 tx=0x7f7138000f60 comp rx=0 tx=0).stop 2024-03-09T08:52:55.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f714f837740 1 -- 172.21.15.67:0/321818135 shutdown_connections 2024-03-09T08:52:55.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f714f837740 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.67:6832/1699514524,v1:172.21.15.67:6834/1699514524] conn(0x55770f391f40 0x55770f392320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f714f837740 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55770f3396f0 0x55770f3987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f714f837740 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55770f328090 0x55770f3333b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f714f837740 1 --2- 172.21.15.67:0/321818135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55770f324f20 0x55770f307eb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f712aa00640 0 client.4869 ms_handle_reset on v2:172.21.15.67:6832/1699514524 2024-03-09T08:52:55.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f714f837740 1 -- 172.21.15.67:0/321818135 >> 172.21.15.67:0/321818135 conn(0x55770f2eb2c0 msgr2=0x55770f2e37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:55.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f714f837740 1 -- 172.21.15.67:0/321818135 shutdown_connections 2024-03-09T08:52:55.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f714f837740 1 -- 172.21.15.67:0/321818135 wait complete. 2024-03-09T08:52:55.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f714f837740 20 client.4869 trim_cache size 0 max 16384 2024-03-09T08:52:55.628 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4279, 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "b", 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 26.052069024000001, 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 14, 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 24, 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 24, 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 42.611212745000003 2024-03-09T08:52:55.629 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:52:55.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f7147200640 1 -- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712406b500 msgr2=0x7f712406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:55.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f7147200640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712406b500 0x7f712406d9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f71480cbe90 tx=0x7f7130009040 comp rx=0 tx=0).stop 2024-03-09T08:52:55.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f7147200640 1 -- 172.21.15.67:0/2654796780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 msgr2=0x7f71481eed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:55.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.628+0000 7f7147200640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 0x7f71481eed80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f713800d8b0 tx=0x7f713800dd70 comp rx=0 tx=0).stop 2024-03-09T08:52:55.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.629+0000 7f7147200640 1 -- 172.21.15.67:0/2654796780 shutdown_connections 2024-03-09T08:52:55.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.629+0000 7f7147200640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712406b500 0x7f712406d9c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.629+0000 7f7147200640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7148151270 0x7f7148149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.629+0000 7f7147200640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7148156db0 0x7f7148150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.629+0000 7f7147200640 1 --2- 172.21.15.67:0/2654796780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f712c004d40 0x7f71481eed80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:55.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.629+0000 7f7147200640 1 -- 172.21.15.67:0/2654796780 >> 172.21.15.67:0/2654796780 conn(0x7f71480cb480 msgr2=0x7f71481552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:55.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.629+0000 7f7147200640 1 -- 172.21.15.67:0/2654796780 shutdown_connections 2024-03-09T08:52:55.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:55.629+0000 7f7147200640 1 -- 172.21.15.67:0/2654796780 wait complete. 2024-03-09T08:52:55.644 INFO:teuthology.run_tasks:Running task kclient... 2024-03-09T08:52:55.668 INFO:tasks.kclient:Mounting kernel clients... 2024-03-09T08:52:55.668 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-09T08:52:55.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:55.668 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-03-09T08:52:55.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:55.722 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-03-09T08:52:55.821 INFO:teuthology.orchestra.run.smithi067.stderr:Cannot find device "ceph-brx" 2024-03-09T08:52:55.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T08:52:55.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:55.825 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-03-09T08:52:55.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:55.880 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-03-09T08:52:55.981 INFO:teuthology.orchestra.run.smithi134.stderr:Cannot find device "ceph-brx" 2024-03-09T08:52:55.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T08:52:55.985 INFO:tasks.kclient:client.0 config is {'syntax': 'v1', 'cephfs_name': 'a'} 2024-03-09T08:52:55.985 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-09T08:52:55.985 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-09T08:52:55.985 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-09T08:52:56.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.243+0000 7fea5a200640 1 Processor -- start 2024-03-09T08:52:56.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.243+0000 7fea5a200640 1 -- start start 2024-03-09T08:52:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.245+0000 7fea5a200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea540c53a0 0x7fea54153470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.245+0000 7fea5a200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea540c5be0 con 0x7fea54156880 2024-03-09T08:52:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.246+0000 7fea5a200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea54071ff0 con 0x7fea540c5780 2024-03-09T08:52:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.246+0000 7fea5a200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea54152fe0 con 0x7fea540c53a0 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.246+0000 7fea59800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fea54156880 0x7fea5415d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49102/0 (socket says 172.21.15.67:49102) 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.246+0000 7fea59800640 1 -- 172.21.15.67:0/3441022949 learned_addr learned my addr 172.21.15.67:0/3441022949 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.246+0000 7fea58e00640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea540c53a0 0x7fea54153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.246+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910912316 0 0) 0x7fea540c5be0 con 0x7fea54156880 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.246+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea38005730 con 0x7fea54156880 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208180196 0 0) 0x7fea54071ff0 con 0x7fea540c5780 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea540c5be0 con 0x7fea540c5780 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 827233791 0 0) 0x7fea38005730 con 0x7fea54156880 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea540c53a0 msgr2=0x7fea54153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea540c53a0 0x7fea54153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 >> v1:172.21.15.134:6789/0 conn(0x7fea540c5780 legacy=0x7fea540718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea54150290 con 0x7fea54156880 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fea4c001670 con 0x7fea54156880 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 >> v1:172.21.15.67:6789/0 conn(0x7fea54156880 legacy=0x7fea5415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea540c53a0 0x7fea38005df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea380063a0 0x7fea38006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 0x7fea38007400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea54152fe0 con 0x7fea38006fa0 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3800de90 con 0x7fea380063a0 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.247+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3800e090 con 0x7fea540c53a0 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.248+0000 7fea59800640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 0x7fea38007400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.248+0000 7fea4be00640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea380063a0 0x7fea38006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.248+0000 7fea58e00640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea540c53a0 0x7fea38005df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.248+0000 7fea59800640 1 -- 172.21.15.67:0/3441022949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea540c53a0 msgr2=0x7fea38005df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.248+0000 7fea59800640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea540c53a0 0x7fea38005df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.248+0000 7fea59800640 1 -- 172.21.15.67:0/3441022949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea380063a0 msgr2=0x7fea38006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:52:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.248+0000 7fea59800640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea380063a0 0x7fea38006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.248+0000 7fea59800640 1 -- 172.21.15.67:0/3441022949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea38005730 con 0x7fea38006fa0 2024-03-09T08:52:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.249+0000 7fea59800640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 0x7fea38007400 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00c0e0 tx=0x7fea38007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12190ea18dbb3366 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.249+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c007c40 con 0x7fea38006fa0 2024-03-09T08:52:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.249+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c030070 con 0x7fea38006fa0 2024-03-09T08:52:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.249+0000 7fea53e00640 1 -- 172.21.15.67:0/3441022949 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c005d90 con 0x7fea38006fa0 2024-03-09T08:52:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.250+0000 7fea5a200640 1 -- 172.21.15.67:0/3441022949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 msgr2=0x7fea38007400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.250+0000 7fea5a200640 1 --2- 172.21.15.67:0/3441022949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 0x7fea38007400 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00c0e0 tx=0x7fea38007a40 comp rx=0 tx=0).stop 2024-03-09T08:52:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.250+0000 7fea59800640 1 -- 172.21.15.67:0/3441022949 reap_dead start 2024-03-09T08:52:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.250+0000 7fea5a200640 1 -- 172.21.15.67:0/3441022949 shutdown_connections 2024-03-09T08:52:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.250+0000 7fea5a200640 1 -- 172.21.15.67:0/3441022949 >> 172.21.15.67:0/3441022949 conn(0x7fea540c88e0 msgr2=0x7fea540c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.250+0000 7fea5a200640 1 -- 172.21.15.67:0/3441022949 shutdown_connections 2024-03-09T08:52:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.250+0000 7fea5a200640 1 -- 172.21.15.67:0/3441022949 wait complete. 2024-03-09T08:52:56.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.252+0000 7fea5a200640 1 Processor -- start 2024-03-09T08:52:56.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.252+0000 7fea5a200640 1 -- start start 2024-03-09T08:52:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea5a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 0x7fea540c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea5a200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea54149b20 0x7fea5406b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea5a200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea541495d0 0x7fea5406b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea5a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea54152c70 con 0x7fea38006fa0 2024-03-09T08:52:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea5a200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea54152990 con 0x7fea541495d0 2024-03-09T08:52:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea5a200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea54156d10 con 0x7fea54149b20 2024-03-09T08:52:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea58e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 0x7fea540c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea58e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 0x7fea540c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48762/0 (socket says 172.21.15.67:48762) 2024-03-09T08:52:56.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea58e00640 1 -- 172.21.15.67:0/318832097 learned_addr learned my addr 172.21.15.67:0/318832097 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:52:56.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea59800640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea541495d0 0x7fea5406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:56.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea58e00640 1 -- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea54149b20 msgr2=0x7fea5406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea58e00640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea54149b20 0x7fea5406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea58e00640 1 -- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea541495d0 msgr2=0x7fea5406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea58e00640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea541495d0 0x7fea5406b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.253+0000 7fea58e00640 1 -- 172.21.15.67:0/318832097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea540c7e80 con 0x7fea38006fa0 2024-03-09T08:52:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.254+0000 7fea58e00640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 0x7fea540c74d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fea44007f30 tx=0x7fea44011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.254+0000 7fea51600640 1 -- 172.21.15.67:0/318832097 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4401a020 con 0x7fea38006fa0 2024-03-09T08:52:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.254+0000 7fea51600640 1 -- 172.21.15.67:0/318832097 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44016470 con 0x7fea38006fa0 2024-03-09T08:52:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.254+0000 7fea51600640 1 -- 172.21.15.67:0/318832097 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44016860 con 0x7fea38006fa0 2024-03-09T08:52:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.254+0000 7fea5a200640 1 -- 172.21.15.67:0/318832097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea54153d00 con 0x7fea38006fa0 2024-03-09T08:52:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.254+0000 7fea5a200640 1 -- 172.21.15.67:0/318832097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea54150b00 con 0x7fea38006fa0 2024-03-09T08:52:56.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.255+0000 7fea5a200640 1 -- 172.21.15.67:0/318832097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5404f3c0 con 0x7fea38006fa0 2024-03-09T08:52:56.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.256+0000 7fea51600640 1 -- 172.21.15.67:0/318832097 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fea44005d10 con 0x7fea38006fa0 2024-03-09T08:52:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.257+0000 7fea51600640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea3006b500 0x7fea3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:52:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.257+0000 7fea51600640 1 -- 172.21.15.67:0/318832097 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fea4408b480 con 0x7fea38006fa0 2024-03-09T08:52:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.258+0000 7fea4be00640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea3006b500 0x7fea3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:52:56.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.259+0000 7fea4be00640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea3006b500 0x7fea3006d9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fea3c007fd0 tx=0x7fea3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:52:56.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.266+0000 7fea51600640 1 -- 172.21.15.67:0/318832097 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fea44024030 con 0x7fea38006fa0 2024-03-09T08:52:56.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.509+0000 7fea5a200640 1 -- 172.21.15.67:0/318832097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fea540c3b70 con 0x7fea38006fa0 2024-03-09T08:52:56.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.509+0000 7fea51600640 1 -- 172.21.15.67:0/318832097 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+116 (secure 0 0 0) 0x7fea44059fc0 con 0x7fea38006fa0 2024-03-09T08:52:56.505 INFO:teuthology.orchestra.run.smithi067.stdout:name: a, metadata pool: a_metadata, data pools: [a_data ] 2024-03-09T08:52:56.505 INFO:teuthology.orchestra.run.smithi067.stdout:name: b, metadata pool: b_metadata, data pools: [b_data ] 2024-03-09T08:52:56.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 -- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea3006b500 msgr2=0x7fea3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:56.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea3006b500 0x7fea3006d9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fea3c007fd0 tx=0x7fea3c009040 comp rx=0 tx=0).stop 2024-03-09T08:52:56.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 -- 172.21.15.67:0/318832097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 msgr2=0x7fea540c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:52:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 0x7fea540c74d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fea44007f30 tx=0x7fea44011660 comp rx=0 tx=0).stop 2024-03-09T08:52:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 -- 172.21.15.67:0/318832097 shutdown_connections 2024-03-09T08:52:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea3006b500 0x7fea3006d9c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea541495d0 0x7fea5406b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea54149b20 0x7fea5406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 --2- 172.21.15.67:0/318832097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea38006fa0 0x7fea540c74d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:52:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 -- 172.21.15.67:0/318832097 >> 172.21.15.67:0/318832097 conn(0x7fea540c88e0 msgr2=0x7fea5414e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:52:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 -- 172.21.15.67:0/318832097 shutdown_connections 2024-03-09T08:52:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:52:56.512+0000 7fea5a200640 1 -- 172.21.15.67:0/318832097 wait complete. 2024-03-09T08:52:56.531 INFO:tasks.cephfs.mount:Mounting Ceph FS a; just confirmed its presence on cluster 2024-03-09T08:52:56.531 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-09T08:52:56.531 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-03-09T08:52:56.531 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-09T08:52:56.531 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-09T08:52:56.531 INFO:tasks.cephfs.mount:self.cephfs_name = a 2024-03-09T08:52:56.532 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-09T08:52:56.532 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-09T08:52:56.532 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-09T08:52:56.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:56.532 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-03-09T08:52:56.586 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-09T08:52:56.586 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-09T08:52:56.586 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-03-09T08:52:56.586 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-03-09T08:52:56.586 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-03-09T08:52:56.586 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-03-09T08:52:56.586 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42523sec preferred_lft 42523sec 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-03-09T08:52:56.587 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-03-09T08:52:56.588 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-09T08:52:56.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:56.588 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-09T08:52:56.588 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-09T08:52:56.588 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-03-09T08:52:56.588 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-03-09T08:52:56.588 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-03-09T08:52:56.589 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-09T08:52:56.589 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-09T08:52:57.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:57.040 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-09T08:52:57.118 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-03-09T08:52:57.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:57.122 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-03-09T08:52:57.178 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-03-09T08:52:57.178 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-09T08:52:57.178 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-09T08:52:57.178 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-09T08:52:57.178 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-09T08:52:57.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:57.179 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-09T08:52:57.179 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-09T08:52:57.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-09T08:52:57.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-09T08:52:57.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-09T08:52:57.179 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-09T08:52:57.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:57.853 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-03-09T08:52:57.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:57.897 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-03-09T08:52:57.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:57.981 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-09T08:52:57.981 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-09T08:52:57.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-09T08:52:57.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-09T08:52:57.982 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-09T08:52:58.156 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-09T08:52:58.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:58.157 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-09T08:52:58.157 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-09T08:52:58.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-09T08:52:58.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-09T08:52:58.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-09T08:52:58.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-09T08:52:58.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-09T08:52:58.157 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-09T08:52:58.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:58.439 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-09T08:52:58.439 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-09T08:52:58.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-03-09T08:52:58.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-03-09T08:52:58.439 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-09T08:52:58.605 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-09T08:52:58.605 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-09T08:52:58.648 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-09T08:52:58.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-09T08:52:58.648 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-09T08:52:58.742 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-09T08:52:58.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:58.743 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=a 2024-03-09T08:52:58.862 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=a 2024-03-09T08:52:58.863 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=a". 2024-03-09T08:52:58.863 INFO:teuthology.orchestra.run.smithi067.stdout:invalid new device string format 2024-03-09T08:52:59.478 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=a 2024-03-09T08:52:59.478 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=a". 2024-03-09T08:52:59.478 INFO:teuthology.orchestra.run.smithi067.stdout:invalid new device string format 2024-03-09T08:52:59.478 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-03-09T08:52:59.478 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-09T08:52:59.513 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=a 2024-03-09T08:52:59.513 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=a". 2024-03-09T08:52:59.513 INFO:teuthology.orchestra.run.smithi067.stdout:invalid new device string format 2024-03-09T08:52:59.513 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-03-09T08:52:59.513 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-09T08:52:59.513 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-03-09T08:52:59.513 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.67:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-03-09T08:52:59.514 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=a,key=0,fsid=aded9f77-d43e-4509-848a-632eae6a080f" will pass to kernel 2024-03-09T08:52:59.516 INFO:teuthology.orchestra.run.smithi067.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-09T08:52:59.517 INFO:teuthology.orchestra.run.smithi067.stdout: You just mounted a file system that supports labels which does not 2024-03-09T08:52:59.517 INFO:teuthology.orchestra.run.smithi067.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-09T08:52:59.517 INFO:teuthology.orchestra.run.smithi067.stdout: applications will generate AVC messages and not be allowed access to 2024-03-09T08:52:59.517 INFO:teuthology.orchestra.run.smithi067.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-09T08:52:59.520 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-09T08:52:59.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:59.521 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-09T08:52:59.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:52:59.608 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-09T08:53:00.534 INFO:teuthology.orchestra.run.smithi067.stdout:client4878 2024-03-09T08:53:00.534 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:53:00.534 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-09T08:53:00.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:00.592 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-09T08:53:00.660 INFO:teuthology.orchestra.run.smithi067.stdout:client4878 2024-03-09T08:53:00.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:00.661 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-09T08:53:00.729 INFO:teuthology.orchestra.run.smithi067.stdout:client4878 2024-03-09T08:53:00.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:00.730 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/aded9f77-d43e-4509-848a-632eae6a080f.client4878/status') 2024-03-09T08:53:00.898 INFO:teuthology.orchestra.run.smithi067.stdout:instance: client.4878 (3)192.168.0.1:0/4015138569 2024-03-09T08:53:00.899 INFO:teuthology.orchestra.run.smithi067.stdout:blocklisted: false 2024-03-09T08:53:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records in 2024-03-09T08:53:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records out 2024-03-09T08:53:00.899 INFO:teuthology.orchestra.run.smithi067.stderr:69 bytes copied, 7.997e-05 s, 863 kB/s 2024-03-09T08:53:00.904 INFO:tasks.kclient:client.1 config is {'syntax': 'v1', 'cephfs_name': 'b'} 2024-03-09T08:53:00.904 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-09T08:53:00.905 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-09T08:53:00.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-09T08:53:01.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.128+0000 7f7b83e00640 1 Processor -- start 2024-03-09T08:53:01.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.128+0000 7f7b83e00640 1 -- start start 2024-03-09T08:53:01.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.130+0000 7f7b83e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7b840c5e40 0x7f7b84156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:01.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.130+0000 7f7b83e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7b840cadb0 0x7f7b84156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:01.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.130+0000 7f7b83e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b84156a90 con 0x7f7b84153300 2024-03-09T08:53:01.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.130+0000 7f7b83e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b84157000 con 0x7f7b840c5e40 2024-03-09T08:53:01.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.130+0000 7f7b83e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b840ae430 con 0x7f7b840cadb0 2024-03-09T08:53:01.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.130+0000 7f7b82000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7b840c5e40 0x7f7b84156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:01.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.130+0000 7f7b82000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7b840c5e40 0x7f7b84156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35406/0 (socket says 172.21.15.67:35406) 2024-03-09T08:53:01.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.130+0000 7f7b82000640 1 -- 172.21.15.67:0/2468972958 learned_addr learned my addr 172.21.15.67:0/2468972958 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:01.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.130+0000 7f7b83400640 1 --2- 172.21.15.67:0/2468972958 >> v2:172.21.15.134:3300/0 conn(0x7f7b840cadb0 0x7f7b84156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:01.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.131+0000 7f7b82000640 1 -- 172.21.15.67:0/2468972958 >> v2:172.21.15.134:3300/0 conn(0x7f7b840cadb0 msgr2=0x7f7b84156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:01.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.131+0000 7f7b82000640 1 --2- 172.21.15.67:0/2468972958 >> v2:172.21.15.134:3300/0 conn(0x7f7b840cadb0 0x7f7b84156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:01.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.131+0000 7f7b82000640 1 -- 172.21.15.67:0/2468972958 >> v1:172.21.15.134:6789/0 conn(0x7f7b84153300 legacy=0x7f7b840b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:01.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.131+0000 7f7b82000640 1 -- 172.21.15.67:0/2468972958 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b840ae6f0 con 0x7f7b840c5e40 2024-03-09T08:53:01.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.132+0000 7f7b82000640 1 --2- 172.21.15.67:0/2468972958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b840c5e40 0x7f7b84156010 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7b840adc60 tx=0x7f7b6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16633bd0404b5463 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.132+0000 7f7b81600640 1 -- 172.21.15.67:0/2468972958 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c035020 con 0x7f7b840c5e40 2024-03-09T08:53:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.132+0000 7f7b81600640 1 -- 172.21.15.67:0/2468972958 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c031870 con 0x7f7b840c5e40 2024-03-09T08:53:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.132+0000 7f7b81600640 1 -- 172.21.15.67:0/2468972958 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c031c40 con 0x7f7b840c5e40 2024-03-09T08:53:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.133+0000 7f7b83e00640 1 -- 172.21.15.67:0/2468972958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b840c5e40 msgr2=0x7f7b84156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.133+0000 7f7b83e00640 1 --2- 172.21.15.67:0/2468972958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b840c5e40 0x7f7b84156010 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7b840adc60 tx=0x7f7b6c02b910 comp rx=0 tx=0).stop 2024-03-09T08:53:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.133+0000 7f7b83e00640 1 -- 172.21.15.67:0/2468972958 shutdown_connections 2024-03-09T08:53:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.133+0000 7f7b83e00640 1 --2- 172.21.15.67:0/2468972958 >> v2:172.21.15.134:3300/0 conn(0x7f7b840cadb0 0x7f7b84156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.133+0000 7f7b83e00640 1 --2- 172.21.15.67:0/2468972958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b840c5e40 0x7f7b84156010 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.133+0000 7f7b83e00640 1 -- 172.21.15.67:0/2468972958 >> 172.21.15.67:0/2468972958 conn(0x7f7b840732d0 msgr2=0x7f7b840ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.133+0000 7f7b83e00640 1 -- 172.21.15.67:0/2468972958 shutdown_connections 2024-03-09T08:53:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.133+0000 7f7b83e00640 1 -- 172.21.15.67:0/2468972958 wait complete. 2024-03-09T08:53:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.135+0000 7f7b83e00640 1 Processor -- start 2024-03-09T08:53:01.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.135+0000 7f7b83e00640 1 -- start start 2024-03-09T08:53:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b840c5e40 0x7f7b8414e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b840cadb0 0x7f7b84153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b84153300 0x7f7b841eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b84159e90 con 0x7f7b84153300 2024-03-09T08:53:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b84159bb0 con 0x7f7b840c5e40 2024-03-09T08:53:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b840c59e0 con 0x7f7b840cadb0 2024-03-09T08:53:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b84153300 0x7f7b841eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b84153300 0x7f7b841eecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35414/0 (socket says 172.21.15.67:35414) 2024-03-09T08:53:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83400640 1 -- 172.21.15.67:0/281700438 learned_addr learned my addr 172.21.15.67:0/281700438 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b82a00640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b840c5e40 0x7f7b8414e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83400640 1 -- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b840cadb0 msgr2=0x7f7b84153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b82000640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b840cadb0 0x7f7b84153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83400640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b840cadb0 0x7f7b84153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.136+0000 7f7b83400640 1 -- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b840c5e40 msgr2=0x7f7b8414e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.137+0000 7f7b83400640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b840c5e40 0x7f7b8414e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.137+0000 7f7b83400640 1 -- 172.21.15.67:0/281700438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8414a320 con 0x7f7b84153300 2024-03-09T08:53:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.137+0000 7f7b82a00640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b840c5e40 0x7f7b8414e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.137+0000 7f7b83400640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b84153300 0x7f7b841eecc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c007e40 tx=0x7f7b7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.137+0000 7f7b7aa00640 1 -- 172.21.15.67:0/281700438 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c014620 con 0x7f7b84153300 2024-03-09T08:53:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.137+0000 7f7b7aa00640 1 -- 172.21.15.67:0/281700438 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c019070 con 0x7f7b84153300 2024-03-09T08:53:01.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.137+0000 7f7b7aa00640 1 -- 172.21.15.67:0/281700438 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c014a20 con 0x7f7b84153300 2024-03-09T08:53:01.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.137+0000 7f7b83e00640 1 -- 172.21.15.67:0/281700438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b84146150 con 0x7f7b84153300 2024-03-09T08:53:01.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.137+0000 7f7b83e00640 1 -- 172.21.15.67:0/281700438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b841ec4b0 con 0x7f7b84153300 2024-03-09T08:53:01.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.138+0000 7f7b83e00640 1 -- 172.21.15.67:0/281700438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b8404f3c0 con 0x7f7b84153300 2024-03-09T08:53:01.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.141+0000 7f7b7aa00640 1 -- 172.21.15.67:0/281700438 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7b7c014bc0 con 0x7f7b84153300 2024-03-09T08:53:01.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.141+0000 7f7b7aa00640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b6006b520 0x7f7b6006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:01.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.141+0000 7f7b7aa00640 1 -- 172.21.15.67:0/281700438 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f7b7c08ae80 con 0x7f7b84153300 2024-03-09T08:53:01.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.142+0000 7f7b82a00640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b6006b520 0x7f7b6006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:01.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.143+0000 7f7b82a00640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b6006b520 0x7f7b6006d9e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400a830 tx=0x7f7b74008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:01.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.151+0000 7f7b7aa00640 1 -- 172.21.15.67:0/281700438 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7b8414a320 con 0x7f7b84153300 2024-03-09T08:53:01.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.396+0000 7f7b83e00640 1 -- 172.21.15.67:0/281700438 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7b840cab90 con 0x7f7b84153300 2024-03-09T08:53:01.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.397+0000 7f7b7aa00640 1 -- 172.21.15.67:0/281700438 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+116 (secure 0 0 0) 0x7f7b7c059940 con 0x7f7b84153300 2024-03-09T08:53:01.393 INFO:teuthology.orchestra.run.smithi067.stdout:name: a, metadata pool: a_metadata, data pools: [a_data ] 2024-03-09T08:53:01.393 INFO:teuthology.orchestra.run.smithi067.stdout:name: b, metadata pool: b_metadata, data pools: [b_data ] 2024-03-09T08:53:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.399+0000 7f7b83e00640 1 -- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b6006b520 msgr2=0x7f7b6006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.399+0000 7f7b83e00640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b6006b520 0x7f7b6006d9e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400a830 tx=0x7f7b74008040 comp rx=0 tx=0).stop 2024-03-09T08:53:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.400+0000 7f7b83e00640 1 -- 172.21.15.67:0/281700438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b84153300 msgr2=0x7f7b841eecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.400+0000 7f7b83e00640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b84153300 0x7f7b841eecc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c007e40 tx=0x7f7b7c00e4b0 comp rx=0 tx=0).stop 2024-03-09T08:53:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.400+0000 7f7b83e00640 1 -- 172.21.15.67:0/281700438 shutdown_connections 2024-03-09T08:53:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.400+0000 7f7b83e00640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b6006b520 0x7f7b6006d9e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.400+0000 7f7b83e00640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b84153300 0x7f7b841eecc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.400+0000 7f7b83e00640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b840cadb0 0x7f7b84153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.400+0000 7f7b83e00640 1 --2- 172.21.15.67:0/281700438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b840c5e40 0x7f7b8414e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:01.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.400+0000 7f7b83e00640 1 -- 172.21.15.67:0/281700438 >> 172.21.15.67:0/281700438 conn(0x7f7b840732d0 msgr2=0x7f7b84151460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:01.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.400+0000 7f7b83e00640 1 -- 172.21.15.67:0/281700438 shutdown_connections 2024-03-09T08:53:01.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:01.400+0000 7f7b83e00640 1 -- 172.21.15.67:0/281700438 wait complete. 2024-03-09T08:53:01.414 INFO:tasks.cephfs.mount:Mounting Ceph FS b; just confirmed its presence on cluster 2024-03-09T08:53:01.414 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-09T08:53:01.414 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi134.front.sepia.ceph.com 2024-03-09T08:53:01.414 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-09T08:53:01.414 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-09T08:53:01.414 INFO:tasks.cephfs.mount:self.cephfs_name = b 2024-03-09T08:53:01.414 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-09T08:53:01.414 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-09T08:53:01.414 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-09T08:53:01.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:01.415 DEBUG:teuthology.orchestra.run.smithi134:> ip addr 2024-03-09T08:53:01.469 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-09T08:53:01.469 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-09T08:53:01.469 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-03-09T08:53:01.469 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-03-09T08:53:01.469 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-03-09T08:53:01.469 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-03-09T08:53:01.469 INFO:teuthology.orchestra.run.smithi134.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-09T08:53:01.469 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e6 brd ff:ff:ff:ff:ff:ff 2024-03-09T08:53:01.469 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f0 2024-03-09T08:53:01.469 INFO:teuthology.orchestra.run.smithi134.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-09T08:53:01.470 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e7 brd ff:ff:ff:ff:ff:ff 2024-03-09T08:53:01.470 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f1 2024-03-09T08:53:01.470 INFO:teuthology.orchestra.run.smithi134.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-09T08:53:01.470 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e8 brd ff:ff:ff:ff:ff:ff 2024-03-09T08:53:01.470 INFO:teuthology.orchestra.run.smithi134.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-09T08:53:01.470 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e9 brd ff:ff:ff:ff:ff:ff 2024-03-09T08:53:01.470 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.21.15.134/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-09T08:53:01.470 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft 42511sec preferred_lft 42511sec 2024-03-09T08:53:01.470 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::ec4:7aff:fe88:6de9/64 scope link 2024-03-09T08:53:01.470 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-03-09T08:53:01.471 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-09T08:53:01.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:01.471 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec bash -c ' 2024-03-09T08:53:01.472 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-03-09T08:53:01.472 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add name ceph-brx type bridge 2024-03-09T08:53:01.472 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip addr flush dev ceph-brx 2024-03-09T08:53:01.472 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set ceph-brx up 2024-03-09T08:53:01.472 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-09T08:53:01.472 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-03-09T08:53:01.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:01.911 DEBUG:teuthology.orchestra.run.smithi134:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-09T08:53:01.991 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-03-09T08:53:01.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:01.995 DEBUG:teuthology.orchestra.run.smithi134:> route 2024-03-09T08:53:02.051 INFO:teuthology.orchestra.run.smithi134.stdout:Kernel IP routing table 2024-03-09T08:53:02.051 INFO:teuthology.orchestra.run.smithi134.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-09T08:53:02.051 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-09T08:53:02.051 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-09T08:53:02.051 INFO:teuthology.orchestra.run.smithi134.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-09T08:53:02.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:02.053 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec bash -c ' 2024-03-09T08:53:02.053 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-03-09T08:53:02.053 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-09T08:53:02.053 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-09T08:53:02.053 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-09T08:53:02.053 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-03-09T08:53:02.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:02.698 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-03-09T08:53:02.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:02.746 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list-id 2024-03-09T08:53:02.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:02.792 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec bash -c ' 2024-03-09T08:53:02.792 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-03-09T08:53:02.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-09T08:53:02.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-09T08:53:02.792 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-03-09T08:53:02.968 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-09T08:53:02.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:02.968 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec bash -c ' 2024-03-09T08:53:02.968 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-03-09T08:53:02.968 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-09T08:53:02.968 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-09T08:53:02.968 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-09T08:53:02.969 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-09T08:53:02.969 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-09T08:53:02.969 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-03-09T08:53:03.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:03.248 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec bash -c ' 2024-03-09T08:53:03.248 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-03-09T08:53:03.248 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set brx.0 up 2024-03-09T08:53:03.248 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set dev brx.0 master ceph-brx 2024-03-09T08:53:03.248 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-03-09T08:53:03.394 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-09T08:53:03.395 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-09T08:53:03.434 INFO:teuthology.orchestra.run.smithi134.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-09T08:53:03.435 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-09T08:53:03.435 DEBUG:teuthology.orchestra.run.smithi134:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-09T08:53:03.524 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-09T08:53:03.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:03.525 DEBUG:teuthology.orchestra.run.smithi134:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=b 2024-03-09T08:53:03.641 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=b 2024-03-09T08:53:03.642 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=b". 2024-03-09T08:53:03.642 INFO:teuthology.orchestra.run.smithi134.stdout:invalid new device string format 2024-03-09T08:53:04.234 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=b 2024-03-09T08:53:04.234 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=b". 2024-03-09T08:53:04.234 INFO:teuthology.orchestra.run.smithi134.stdout:invalid new device string format 2024-03-09T08:53:04.234 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-03-09T08:53:04.234 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-09T08:53:04.535 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=b 2024-03-09T08:53:04.535 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=b". 2024-03-09T08:53:04.535 INFO:teuthology.orchestra.run.smithi134.stdout:invalid new device string format 2024-03-09T08:53:04.535 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-03-09T08:53:04.535 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-09T08:53:04.535 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.67:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-03-09T08:53:04.535 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.67:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-03-09T08:53:04.536 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=b,key=1,fsid=aded9f77-d43e-4509-848a-632eae6a080f" will pass to kernel 2024-03-09T08:53:04.540 INFO:teuthology.orchestra.run.smithi134.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-09T08:53:04.540 INFO:teuthology.orchestra.run.smithi134.stdout: You just mounted a file system that supports labels which does not 2024-03-09T08:53:04.540 INFO:teuthology.orchestra.run.smithi134.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-09T08:53:04.540 INFO:teuthology.orchestra.run.smithi134.stdout: applications will generate AVC messages and not be allowed access to 2024-03-09T08:53:04.540 INFO:teuthology.orchestra.run.smithi134.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-09T08:53:04.548 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-09T08:53:04.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:04.548 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-09T08:53:04.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:04.637 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-09T08:53:04.711 INFO:teuthology.orchestra.run.smithi134.stdout:client4780 2024-03-09T08:53:04.711 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:53:04.711 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-09T08:53:04.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:04.764 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-09T08:53:04.834 INFO:teuthology.orchestra.run.smithi134.stdout:client4780 2024-03-09T08:53:04.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:04.835 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-09T08:53:04.909 INFO:teuthology.orchestra.run.smithi134.stdout:client4780 2024-03-09T08:53:04.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T08:53:04.909 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/aded9f77-d43e-4509-848a-632eae6a080f.client4780/status') 2024-03-09T08:53:05.014 INFO:teuthology.orchestra.run.smithi134.stdout:instance: client.4780 (3)192.168.0.1:0/198315456 2024-03-09T08:53:05.014 INFO:teuthology.orchestra.run.smithi134.stdout:blocklisted: false 2024-03-09T08:53:05.014 INFO:teuthology.orchestra.run.smithi134.stderr:0+1 records in 2024-03-09T08:53:05.014 INFO:teuthology.orchestra.run.smithi134.stderr:0+1 records out 2024-03-09T08:53:05.014 INFO:teuthology.orchestra.run.smithi134.stderr:68 bytes copied, 6.4166e-05 s, 1.1 MB/s 2024-03-09T08:53:05.019 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-03-09T08:53:05.031 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-03-09T08:53:05.031 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:53:05.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.266+0000 7f6cdd800640 1 Processor -- start 2024-03-09T08:53:05.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.266+0000 7f6cdd800640 1 -- start start 2024-03-09T08:53:05.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.268+0000 7f6cdd800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6cd80c1f70 0x7f6cd8150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.268+0000 7f6cdd800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd80c81e0 con 0x7f6cd80710a0 2024-03-09T08:53:05.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.268+0000 7f6cdd800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd80c32b0 con 0x7f6cd8154080 2024-03-09T08:53:05.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.268+0000 7f6cdd800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd80c2fd0 con 0x7f6cd80c1f70 2024-03-09T08:53:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.268+0000 7f6cd7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6cd80710a0 0x7f6cd8153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55606/0 (socket says 172.21.15.67:55606) 2024-03-09T08:53:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.268+0000 7f6cd7400640 1 -- 172.21.15.67:0/1918512575 learned_addr learned my addr 172.21.15.67:0/1918512575 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.268+0000 7f6cd7e00640 1 --2- 172.21.15.67:0/1918512575 >> v2:172.21.15.134:3300/0 conn(0x7f6cd80c1f70 0x7f6cd8150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733930930 0 0) 0x7f6cd80c81e0 con 0x7f6cd80710a0 2024-03-09T08:53:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cbc003520 con 0x7f6cd80710a0 2024-03-09T08:53:05.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2480700879 0 0) 0x7f6cbc003520 con 0x7f6cd80710a0 2024-03-09T08:53:05.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 >> v2:172.21.15.134:3300/0 conn(0x7f6cd80c1f70 msgr2=0x7f6cd8150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:05.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 --2- 172.21.15.67:0/1918512575 >> v2:172.21.15.134:3300/0 conn(0x7f6cd80c1f70 0x7f6cd8150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 >> v1:172.21.15.134:6789/0 conn(0x7f6cd8154080 legacy=0x7f6cd815e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd80c3700 con 0x7f6cd80710a0 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f6cc4001670 con 0x7f6cd80710a0 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 >> v1:172.21.15.67:6789/0 conn(0x7f6cd80710a0 legacy=0x7f6cd8153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cbc003a20 0x7f6cbc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc004800 0x7f6cbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc005230 0x7f6cbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd80c2fd0 con 0x7f6cbc005230 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc00c190 con 0x7f6cbc004800 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.269+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc00c390 con 0x7f6cbc003a20 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.270+0000 7f6cdce00640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc005230 0x7f6cbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.270+0000 7f6cd7e00640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cbc003a20 0x7f6cbc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.270+0000 7f6cd7400640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc004800 0x7f6cbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.270+0000 7f6cdce00640 1 -- 172.21.15.67:0/1918512575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cbc003a20 msgr2=0x7f6cbc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:05.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.270+0000 7f6cdce00640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cbc003a20 0x7f6cbc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.270+0000 7f6cdce00640 1 -- 172.21.15.67:0/1918512575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc004800 msgr2=0x7f6cbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:05.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.270+0000 7f6cdce00640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc004800 0x7f6cbc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.270+0000 7f6cdce00640 1 -- 172.21.15.67:0/1918512575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd80c3700 con 0x7f6cbc005230 2024-03-09T08:53:05.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.270+0000 7f6cd7400640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc004800 0x7f6cbc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:05.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.271+0000 7f6cdce00640 1 -- 172.21.15.67:0/1918512575 reap_dead start 2024-03-09T08:53:05.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.271+0000 7f6cdce00640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc005230 0x7f6cbc00b9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0001e50 tx=0x7f6cbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3728f308663455d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:05.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.272+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0034070 con 0x7f6cbc005230 2024-03-09T08:53:05.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.272+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd0003b90 con 0x7f6cbc005230 2024-03-09T08:53:05.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.272+0000 7f6cd6a00640 1 -- 172.21.15.67:0/1918512575 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0001a70 con 0x7f6cbc005230 2024-03-09T08:53:05.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.272+0000 7f6cdd800640 1 -- 172.21.15.67:0/1918512575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc005230 msgr2=0x7f6cbc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.272+0000 7f6cdd800640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc005230 0x7f6cbc00b9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0001e50 tx=0x7f6cbc005830 comp rx=0 tx=0).stop 2024-03-09T08:53:05.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.273+0000 7f6cdd800640 1 -- 172.21.15.67:0/1918512575 shutdown_connections 2024-03-09T08:53:05.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.273+0000 7f6cdd800640 1 --2- 172.21.15.67:0/1918512575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cbc005230 0x7f6cbc00b9c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.273+0000 7f6cdd800640 1 -- 172.21.15.67:0/1918512575 >> 172.21.15.67:0/1918512575 conn(0x7f6cd80c7300 msgr2=0x7f6cd80c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:05.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.273+0000 7f6cdd800640 1 -- 172.21.15.67:0/1918512575 shutdown_connections 2024-03-09T08:53:05.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.273+0000 7f6cdd800640 1 -- 172.21.15.67:0/1918512575 wait complete. 2024-03-09T08:53:05.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.275+0000 7f6cdd800640 1 Processor -- start 2024-03-09T08:53:05.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.275+0000 7f6cdd800640 1 -- start start 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.276+0000 7f6cdd800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc005230 0x7f6cd80ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.276+0000 7f6cdd800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd814cce0 0x7f6cd81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.276+0000 7f6cdd800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cd814c790 0x7f6cd8156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.276+0000 7f6cdd800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd8070d50 con 0x7f6cd814c790 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.276+0000 7f6cdd800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd8070830 con 0x7f6cbc005230 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.276+0000 7f6cdd800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd8070ed0 con 0x7f6cd814cce0 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.276+0000 7f6cdce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cd814c790 0x7f6cd8156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.276+0000 7f6cdce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cd814c790 0x7f6cd8156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35422/0 (socket says 172.21.15.67:35422) 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.276+0000 7f6cdce00640 1 -- 172.21.15.67:0/1845592380 learned_addr learned my addr 172.21.15.67:0/1845592380 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.276+0000 7f6cd7400640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd814cce0 0x7f6cd81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.276+0000 7f6cd7e00640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc005230 0x7f6cd80ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.277+0000 7f6cdce00640 1 -- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd814cce0 msgr2=0x7f6cd81562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.277+0000 7f6cdce00640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd814cce0 0x7f6cd81562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.277+0000 7f6cdce00640 1 -- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc005230 msgr2=0x7f6cd80ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.277+0000 7f6cdce00640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc005230 0x7f6cd80ab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.277+0000 7f6cdce00640 1 -- 172.21.15.67:0/1845592380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd8152f00 con 0x7f6cd814c790 2024-03-09T08:53:05.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.277+0000 7f6cd7e00640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc005230 0x7f6cd80ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:05.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.277+0000 7f6cdce00640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cd814c790 0x7f6cd8156830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0001e80 tx=0x7f6cd00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:05.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.277+0000 7f6ccfe00640 1 -- 172.21.15.67:0/1845592380 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0034030 con 0x7f6cd814c790 2024-03-09T08:53:05.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.277+0000 7f6ccfe00640 1 -- 172.21.15.67:0/1845592380 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd0006d60 con 0x7f6cd814c790 2024-03-09T08:53:05.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.277+0000 7f6ccfe00640 1 -- 172.21.15.67:0/1845592380 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0007db0 con 0x7f6cd814c790 2024-03-09T08:53:05.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.278+0000 7f6cdd800640 1 -- 172.21.15.67:0/1845592380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd81ef960 con 0x7f6cd814c790 2024-03-09T08:53:05.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.278+0000 7f6cdd800640 1 -- 172.21.15.67:0/1845592380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd8153480 con 0x7f6cd814c790 2024-03-09T08:53:05.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.279+0000 7f6cdd800640 1 -- 172.21.15.67:0/1845592380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd8153680 con 0x7f6cd814c790 2024-03-09T08:53:05.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.280+0000 7f6ccfe00640 1 -- 172.21.15.67:0/1845592380 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6cd0043020 con 0x7f6cd814c790 2024-03-09T08:53:05.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.281+0000 7f6ccfe00640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb806b500 0x7f6cb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.281+0000 7f6ccfe00640 1 -- 172.21.15.67:0/1845592380 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f6cd00abbf0 con 0x7f6cd814c790 2024-03-09T08:53:05.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.281+0000 7f6cd7e00640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb806b500 0x7f6cb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.283+0000 7f6cd7e00640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb806b500 0x7f6cb806d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009980 tx=0x7f6cc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:05.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.289+0000 7f6ccfe00640 1 -- 172.21.15.67:0/1845592380 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6cd8153680 con 0x7f6cd814c790 2024-03-09T08:53:05.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.488+0000 7f6cdd800640 1 -- 172.21.15.67:0/1845592380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cd804f3c0 con 0x7f6cd814c790 2024-03-09T08:53:05.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.491+0000 7f6ccfe00640 1 -- 172.21.15.67:0/1845592380 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20555 (secure 0 0 0) 0x7f6cd007a6c0 con 0x7f6cd814c790 2024-03-09T08:53:05.488 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:53:05.488 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:59.780300+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.494+0000 7f6cdd800640 1 -- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb806b500 msgr2=0x7f6cb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.494+0000 7f6cdd800640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb806b500 0x7f6cb806d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009980 tx=0x7f6cc8008040 comp rx=0 tx=0).stop 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.494+0000 7f6cdd800640 1 -- 172.21.15.67:0/1845592380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cd814c790 msgr2=0x7f6cd8156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.494+0000 7f6cdd800640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cd814c790 0x7f6cd8156830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0001e80 tx=0x7f6cd00014a0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.494+0000 7f6cdd800640 1 -- 172.21.15.67:0/1845592380 shutdown_connections 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.494+0000 7f6cdd800640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb806b500 0x7f6cb806d9c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.494+0000 7f6cdd800640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cd814c790 0x7f6cd8156830 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.494+0000 7f6cdd800640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd814cce0 0x7f6cd81562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.494+0000 7f6cdd800640 1 --2- 172.21.15.67:0/1845592380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cbc005230 0x7f6cd80ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.494+0000 7f6cdd800640 1 -- 172.21.15.67:0/1845592380 >> 172.21.15.67:0/1845592380 conn(0x7f6cd80c7300 msgr2=0x7f6cd8072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.495+0000 7f6cdd800640 1 -- 172.21.15.67:0/1845592380 shutdown_connections 2024-03-09T08:53:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.495+0000 7f6cdd800640 1 -- 172.21.15.67:0/1845592380 wait complete. 2024-03-09T08:53:05.508 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-09T08:52:06.749641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-03-09T08:52:16.185356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-03-09T08:52:16.756829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-03-09T08:52:27.484297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-03-09T08:52:28.060754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-09T08:53:05.509 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T08:53:05.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.730+0000 7f9d12200640 1 Processor -- start 2024-03-09T08:53:05.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.731+0000 7f9d12200640 1 -- start start 2024-03-09T08:53:05.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.732+0000 7f9d12200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9d0c0becf0 0x7f9d0c072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.733+0000 7f9d12200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0c6080 con 0x7f9d0c0c9bb0 2024-03-09T08:53:05.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.733+0000 7f9d12200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0c62f0 con 0x7f9d0c072770 2024-03-09T08:53:05.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.733+0000 7f9d12200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c9720 con 0x7f9d0c0becf0 2024-03-09T08:53:05.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.733+0000 7f9d0be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9d0c0c9bb0 0x7f9d0c160650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55622/0 (socket says 172.21.15.67:55622) 2024-03-09T08:53:05.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.733+0000 7f9d0be00640 1 -- 172.21.15.67:0/2641355668 learned_addr learned my addr 172.21.15.67:0/2641355668 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.733+0000 7f9d0aa00640 1 --2- 172.21.15.67:0/2641355668 >> v2:172.21.15.134:3300/0 conn(0x7f9d0c0becf0 0x7f9d0c072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.733+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385196571 0 0) 0x7f9d0c0c6080 con 0x7f9d0c0c9bb0 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf0003520 con 0x7f9d0c0c9bb0 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1359927591 0 0) 0x7f9cf0003520 con 0x7f9d0c0c9bb0 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 >> v2:172.21.15.134:3300/0 conn(0x7f9d0c0becf0 msgr2=0x7f9d0c072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 --2- 172.21.15.67:0/2641355668 >> v2:172.21.15.134:3300/0 conn(0x7f9d0c0becf0 0x7f9d0c072b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 >> v1:172.21.15.134:6789/0 conn(0x7f9d0c072770 legacy=0x7f9d0c156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0c65d0 con 0x7f9d0c0c9bb0 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f9d040019f0 con 0x7f9d0c0c9bb0 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 >> v1:172.21.15.67:6789/0 conn(0x7f9d0c0c9bb0 legacy=0x7f9d0c160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 --2- 172.21.15.67:0/2641355668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf0003a00 0x7f9cf0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 --2- 172.21.15.67:0/2641355668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf0004800 0x7f9cf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 --2- 172.21.15.67:0/2641355668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf00052e0 0x7f9cf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0c9720 con 0x7f9cf0004800 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf000c1e0 con 0x7f9cf00052e0 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf000c3e0 con 0x7f9cf0003a00 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.734+0000 7f9d0aa00640 1 --2- 172.21.15.67:0/2641355668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf0004800 0x7f9cf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.735+0000 7f9d0b400640 1 --2- 172.21.15.67:0/2641355668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf0003a00 0x7f9cf0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.735+0000 7f9d0aa00640 1 -- 172.21.15.67:0/2641355668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf0003a00 msgr2=0x7f9cf0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.735+0000 7f9d0aa00640 1 --2- 172.21.15.67:0/2641355668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf0003a00 0x7f9cf0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.735+0000 7f9d0aa00640 1 -- 172.21.15.67:0/2641355668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf00052e0 msgr2=0x7f9cf000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.735+0000 7f9d0aa00640 1 --2- 172.21.15.67:0/2641355668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf00052e0 0x7f9cf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.735+0000 7f9d0aa00640 1 -- 172.21.15.67:0/2641355668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0003520 con 0x7f9cf0004800 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.735+0000 7f9d0be00640 1 -- 172.21.15.67:0/2641355668 reap_dead start 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.735+0000 7f9d0b400640 1 --2- 172.21.15.67:0/2641355668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf0003a00 0x7f9cf0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:05.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.736+0000 7f9d0aa00640 1 --2- 172.21.15.67:0/2641355668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf0004800 0x7f9cf0004c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009a90 tx=0x7f9cf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8688ea08f3589725 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.736+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8039070 con 0x7f9cf0004800 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.736+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf802c6b0 con 0x7f9cf0004800 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.736+0000 7f9d0a000640 1 -- 172.21.15.67:0/2641355668 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8009690 con 0x7f9cf0004800 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.737+0000 7f9d12200640 1 -- 172.21.15.67:0/2641355668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf0004800 msgr2=0x7f9cf0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.737+0000 7f9d12200640 1 --2- 172.21.15.67:0/2641355668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf0004800 0x7f9cf0004c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009a90 tx=0x7f9cf0005830 comp rx=0 tx=0).stop 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.737+0000 7f9d12200640 1 -- 172.21.15.67:0/2641355668 shutdown_connections 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.737+0000 7f9d12200640 1 --2- 172.21.15.67:0/2641355668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf0004800 0x7f9cf0004c80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.737+0000 7f9d12200640 1 -- 172.21.15.67:0/2641355668 >> 172.21.15.67:0/2641355668 conn(0x7f9d0c0cb840 msgr2=0x7f9d0c0cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.737+0000 7f9d12200640 1 -- 172.21.15.67:0/2641355668 shutdown_connections 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.737+0000 7f9d12200640 1 -- 172.21.15.67:0/2641355668 wait complete. 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.739+0000 7f9d12200640 1 Processor -- start 2024-03-09T08:53:05.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.739+0000 7f9d12200640 1 -- start start 2024-03-09T08:53:05.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d12200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf0004800 0x7f9d0c0c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d12200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d0c0bfb10 0x7f9d0c1ebd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d12200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ec260 0x7f9d0c0ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d12200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c153700 con 0x7f9d0c1ec260 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d12200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c153580 con 0x7f9d0c0bfb10 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d12200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c1571c0 con 0x7f9cf0004800 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ec260 0x7f9d0c0ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d0be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ec260 0x7f9d0c0ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35436/0 (socket says 172.21.15.67:35436) 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d0be00640 1 -- 172.21.15.67:0/2136239421 learned_addr learned my addr 172.21.15.67:0/2136239421 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d0b400640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf0004800 0x7f9d0c0c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d0aa00640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d0c0bfb10 0x7f9d0c1ebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d0be00640 1 -- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf0004800 msgr2=0x7f9d0c0c1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d0be00640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf0004800 0x7f9d0c0c1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.740+0000 7f9d0be00640 1 -- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d0c0bfb10 msgr2=0x7f9d0c1ebd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.741+0000 7f9d0be00640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d0c0bfb10 0x7f9d0c1ebd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.741+0000 7f9d0be00640 1 -- 172.21.15.67:0/2136239421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c1ef2d0 con 0x7f9d0c1ec260 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.741+0000 7f9d0b400640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf0004800 0x7f9d0c0c1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:05.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.741+0000 7f9d0be00640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ec260 0x7f9d0c0ac160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0c9f90 tx=0x7f9d0400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:05.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.741+0000 7f9d03400640 1 -- 172.21.15.67:0/2136239421 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0400a5b0 con 0x7f9d0c1ec260 2024-03-09T08:53:05.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.741+0000 7f9d03400640 1 -- 172.21.15.67:0/2136239421 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0400a750 con 0x7f9d0c1ec260 2024-03-09T08:53:05.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.741+0000 7f9d12200640 1 -- 172.21.15.67:0/2136239421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c1ef5b0 con 0x7f9d0c1ec260 2024-03-09T08:53:05.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.741+0000 7f9d03400640 1 -- 172.21.15.67:0/2136239421 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0400ab40 con 0x7f9d0c1ec260 2024-03-09T08:53:05.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.741+0000 7f9d12200640 1 -- 172.21.15.67:0/2136239421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c1efb70 con 0x7f9d0c1ec260 2024-03-09T08:53:05.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.742+0000 7f9d12200640 1 -- 172.21.15.67:0/2136239421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c075a00 con 0x7f9d0c1ec260 2024-03-09T08:53:05.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.743+0000 7f9d03400640 1 -- 172.21.15.67:0/2136239421 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9d04011070 con 0x7f9d0c1ec260 2024-03-09T08:53:05.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.744+0000 7f9d03400640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cec06b4d0 0x7f9cec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:05.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.744+0000 7f9d03400640 1 -- 172.21.15.67:0/2136239421 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f9d04089fe0 con 0x7f9d0c1ec260 2024-03-09T08:53:05.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.745+0000 7f9d0b400640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cec06b4d0 0x7f9cec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:05.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.746+0000 7f9d0b400640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cec06b4d0 0x7f9cec06d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c050630 tx=0x7f9cfc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:05.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.749+0000 7f9d03400640 1 -- 172.21.15.67:0/2136239421 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9d0401a060 con 0x7f9d0c1ec260 2024-03-09T08:53:05.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.955+0000 7f9d12200640 1 -- 172.21.15.67:0/2136239421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d0c0cb030 con 0x7f9d0c1ec260 2024-03-09T08:53:05.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.956+0000 7f9d03400640 1 -- 172.21.15.67:0/2136239421 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d040102c0 con 0x7f9d0c1ec260 2024-03-09T08:53:05.952 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-09T08:53:05.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.958+0000 7f9d12200640 1 -- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cec06b4d0 msgr2=0x7f9cec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.958+0000 7f9d12200640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cec06b4d0 0x7f9cec06d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c050630 tx=0x7f9cfc009040 comp rx=0 tx=0).stop 2024-03-09T08:53:05.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.958+0000 7f9d12200640 1 -- 172.21.15.67:0/2136239421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ec260 msgr2=0x7f9d0c0ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:05.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.958+0000 7f9d12200640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ec260 0x7f9d0c0ac160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0c9f90 tx=0x7f9d0400cbe0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.958+0000 7f9d12200640 1 -- 172.21.15.67:0/2136239421 shutdown_connections 2024-03-09T08:53:05.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.958+0000 7f9d12200640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cec06b4d0 0x7f9cec06d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.958+0000 7f9d12200640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d0c1ec260 0x7f9d0c0ac160 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.959+0000 7f9d12200640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d0c0bfb10 0x7f9d0c1ebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.959+0000 7f9d12200640 1 --2- 172.21.15.67:0/2136239421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf0004800 0x7f9d0c0c1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:05.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.959+0000 7f9d12200640 1 -- 172.21.15.67:0/2136239421 >> 172.21.15.67:0/2136239421 conn(0x7f9d0c0cb840 msgr2=0x7f9d0c150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.959+0000 7f9d12200640 1 -- 172.21.15.67:0/2136239421 shutdown_connections 2024-03-09T08:53:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:05.959+0000 7f9d12200640 1 -- 172.21.15.67:0/2136239421 wait complete. 2024-03-09T08:53:05.972 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-03-09T08:53:06.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.204+0000 7fe4cda00640 1 Processor -- start 2024-03-09T08:53:06.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.204+0000 7fe4cda00640 1 -- start start 2024-03-09T08:53:06.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.206+0000 7fe4cda00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe4c80c9490 0x7fe4c80c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.206+0000 7fe4cda00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c80c9a30 con 0x7fe4c806b040 2024-03-09T08:53:06.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.206+0000 7fe4cda00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c80c9d10 con 0x7fe4c80c5f10 2024-03-09T08:53:06.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.206+0000 7fe4cda00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c80ca340 con 0x7fe4c80c9490 2024-03-09T08:53:06.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.206+0000 7fe4c7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe4c806b040 0x7fe4c80cab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55632/0 (socket says 172.21.15.67:55632) 2024-03-09T08:53:06.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.206+0000 7fe4c7400640 1 -- 172.21.15.67:0/2329478667 learned_addr learned my addr 172.21.15.67:0/2329478667 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:06.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.206+0000 7fe4cd000640 1 --2- 172.21.15.67:0/2329478667 >> v2:172.21.15.134:3300/0 conn(0x7fe4c80c9490 0x7fe4c80c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808449365 0 0) 0x7fe4c80c9a30 con 0x7fe4c806b040 2024-03-09T08:53:06.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4ac003520 con 0x7fe4c806b040 2024-03-09T08:53:06.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1360075761 0 0) 0x7fe4ac003520 con 0x7fe4c806b040 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 >> v2:172.21.15.134:3300/0 conn(0x7fe4c80c9490 msgr2=0x7fe4c80c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 --2- 172.21.15.67:0/2329478667 >> v2:172.21.15.134:3300/0 conn(0x7fe4c80c9490 0x7fe4c80c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 >> v1:172.21.15.134:6789/0 conn(0x7fe4c80c5f10 legacy=0x7fe4c81567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c80ca620 con 0x7fe4c806b040 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fe4b0001670 con 0x7fe4c806b040 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 >> v1:172.21.15.67:6789/0 conn(0x7fe4c806b040 legacy=0x7fe4c80cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 --2- 172.21.15.67:0/2329478667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac003a00 0x7fe4ac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 --2- 172.21.15.67:0/2329478667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4ac004800 0x7fe4ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 --2- 172.21.15.67:0/2329478667 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4ac005210 0x7fe4ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80ca340 con 0x7fe4ac004800 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac00c160 con 0x7fe4ac003a00 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac00c360 con 0x7fe4ac005210 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.207+0000 7fe4c7400640 1 --2- 172.21.15.67:0/2329478667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4ac004800 0x7fe4ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.208+0000 7fe4c7e00640 1 --2- 172.21.15.67:0/2329478667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac003a00 0x7fe4ac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.208+0000 7fe4cd000640 1 --2- 172.21.15.67:0/2329478667 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4ac005210 0x7fe4ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.208+0000 7fe4c7400640 1 -- 172.21.15.67:0/2329478667 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4ac005210 msgr2=0x7fe4ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:06.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.208+0000 7fe4c7400640 1 --2- 172.21.15.67:0/2329478667 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4ac005210 0x7fe4ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.208+0000 7fe4c7400640 1 -- 172.21.15.67:0/2329478667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac003a00 msgr2=0x7fe4ac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:06.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.208+0000 7fe4c7400640 1 --2- 172.21.15.67:0/2329478667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac003a00 0x7fe4ac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.208+0000 7fe4c7400640 1 -- 172.21.15.67:0/2329478667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac003520 con 0x7fe4ac004800 2024-03-09T08:53:06.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.208+0000 7fe4cd000640 1 -- 172.21.15.67:0/2329478667 reap_dead start 2024-03-09T08:53:06.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.210+0000 7fe4c7400640 1 --2- 172.21.15.67:0/2329478667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4ac004800 0x7fe4ac004c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000c160 tx=0x7fe4ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a20e218416b54 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:06.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.210+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0007c60 con 0x7fe4ac004800 2024-03-09T08:53:06.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.210+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b0030070 con 0x7fe4ac004800 2024-03-09T08:53:06.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.210+0000 7fe4c6a00640 1 -- 172.21.15.67:0/2329478667 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0005d90 con 0x7fe4ac004800 2024-03-09T08:53:06.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.211+0000 7fe4cda00640 1 -- 172.21.15.67:0/2329478667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4ac004800 msgr2=0x7fe4ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:06.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.211+0000 7fe4cda00640 1 --2- 172.21.15.67:0/2329478667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4ac004800 0x7fe4ac004c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000c160 tx=0x7fe4ac005830 comp rx=0 tx=0).stop 2024-03-09T08:53:06.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.211+0000 7fe4cda00640 1 -- 172.21.15.67:0/2329478667 shutdown_connections 2024-03-09T08:53:06.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.211+0000 7fe4cda00640 1 --2- 172.21.15.67:0/2329478667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4ac004800 0x7fe4ac004c80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.211+0000 7fe4cda00640 1 -- 172.21.15.67:0/2329478667 >> 172.21.15.67:0/2329478667 conn(0x7fe4c80cb590 msgr2=0x7fe4c80cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:06.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.211+0000 7fe4cda00640 1 -- 172.21.15.67:0/2329478667 shutdown_connections 2024-03-09T08:53:06.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.211+0000 7fe4cda00640 1 -- 172.21.15.67:0/2329478667 wait complete. 2024-03-09T08:53:06.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.214+0000 7fe4cda00640 1 Processor -- start 2024-03-09T08:53:06.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.214+0000 7fe4cda00640 1 -- start start 2024-03-09T08:53:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.215+0000 7fe4cda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac004800 0x7fe4c8149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.215+0000 7fe4cda00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4c814d280 0x7fe4c814c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.215+0000 7fe4cda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4c8157140 0x7fe4c814af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.215+0000 7fe4cda00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c81537a0 con 0x7fe4c8157140 2024-03-09T08:53:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.215+0000 7fe4cda00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c8153620 con 0x7fe4ac004800 2024-03-09T08:53:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.215+0000 7fe4cda00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80c62f0 con 0x7fe4c814d280 2024-03-09T08:53:06.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.216+0000 7fe4cd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4c8157140 0x7fe4c814af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.216+0000 7fe4cd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4c8157140 0x7fe4c814af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35456/0 (socket says 172.21.15.67:35456) 2024-03-09T08:53:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.216+0000 7fe4cd000640 1 -- 172.21.15.67:0/839476461 learned_addr learned my addr 172.21.15.67:0/839476461 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.216+0000 7fe4c7e00640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac004800 0x7fe4c8149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.216+0000 7fe4c7400640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4c814d280 0x7fe4c814c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.216+0000 7fe4cd000640 1 -- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4c814d280 msgr2=0x7fe4c814c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.216+0000 7fe4cd000640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4c814d280 0x7fe4c814c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.216+0000 7fe4cd000640 1 -- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac004800 msgr2=0x7fe4c8149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.216+0000 7fe4cd000640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac004800 0x7fe4c8149740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.216+0000 7fe4cd000640 1 -- 172.21.15.67:0/839476461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c8149460 con 0x7fe4c8157140 2024-03-09T08:53:06.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.216+0000 7fe4cd000640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4c8157140 0x7fe4c814af50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe4c808d7a0 tx=0x7fe4c00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:06.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.217+0000 7fe4bfe00640 1 -- 172.21.15.67:0/839476461 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0011db0 con 0x7fe4c8157140 2024-03-09T08:53:06.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.217+0000 7fe4bfe00640 1 -- 172.21.15.67:0/839476461 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c001b070 con 0x7fe4c8157140 2024-03-09T08:53:06.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.217+0000 7fe4bfe00640 1 -- 172.21.15.67:0/839476461 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0003990 con 0x7fe4c8157140 2024-03-09T08:53:06.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.217+0000 7fe4cda00640 1 -- 172.21.15.67:0/839476461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c814a450 con 0x7fe4c8157140 2024-03-09T08:53:06.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.217+0000 7fe4cda00640 1 -- 172.21.15.67:0/839476461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c814aa10 con 0x7fe4c8157140 2024-03-09T08:53:06.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.218+0000 7fe4cda00640 1 -- 172.21.15.67:0/839476461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c8075a00 con 0x7fe4c8157140 2024-03-09T08:53:06.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.219+0000 7fe4bfe00640 1 -- 172.21.15.67:0/839476461 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe4c0003b30 con 0x7fe4c8157140 2024-03-09T08:53:06.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.221+0000 7fe4bfe00640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4a406b500 0x7fe4a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.221+0000 7fe4bfe00640 1 -- 172.21.15.67:0/839476461 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fe4c008ae90 con 0x7fe4c8157140 2024-03-09T08:53:06.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.221+0000 7fe4c7e00640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4a406b500 0x7fe4a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.223+0000 7fe4c7e00640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4a406b500 0x7fe4a406d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8000c00 tx=0x7fe4b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:06.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.225+0000 7fe4bfe00640 1 -- 172.21.15.67:0/839476461 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe4c8075a00 con 0x7fe4c8157140 2024-03-09T08:53:06.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.427+0000 7fe4cda00640 1 -- 172.21.15.67:0/839476461 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe4c804f3c0 con 0x7fe4c8157140 2024-03-09T08:53:06.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.428+0000 7fe4bfe00640 1 -- 172.21.15.67:0/839476461 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v26) v1 ==== 99+0+11 (secure 0 0 0) 0x7fe4c0003e20 con 0x7fe4c8157140 2024-03-09T08:53:06.424 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T08:53:06.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 -- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4a406b500 msgr2=0x7fe4a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:06.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4a406b500 0x7fe4a406d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8000c00 tx=0x7fe4b8009040 comp rx=0 tx=0).stop 2024-03-09T08:53:06.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 -- 172.21.15.67:0/839476461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4c8157140 msgr2=0x7fe4c814af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:06.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4c8157140 0x7fe4c814af50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe4c808d7a0 tx=0x7fe4c00115a0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 -- 172.21.15.67:0/839476461 shutdown_connections 2024-03-09T08:53:06.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4a406b500 0x7fe4a406d9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4c8157140 0x7fe4c814af50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4c814d280 0x7fe4c814c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 --2- 172.21.15.67:0/839476461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac004800 0x7fe4c8149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 -- 172.21.15.67:0/839476461 >> 172.21.15.67:0/839476461 conn(0x7fe4c80cb590 msgr2=0x7fe4c81526c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:06.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 -- 172.21.15.67:0/839476461 shutdown_connections 2024-03-09T08:53:06.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.431+0000 7fe4cda00640 1 -- 172.21.15.67:0/839476461 wait complete. 2024-03-09T08:53:06.455 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-03-09T08:53:06.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.677+0000 7ff31be00640 1 Processor -- start 2024-03-09T08:53:06.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.677+0000 7ff31be00640 1 -- start start 2024-03-09T08:53:06.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.679+0000 7ff31be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff31c0ac930 0x7ff31c0c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.679+0000 7ff31be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff31c1514f0 0x7ff31c153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.679+0000 7ff31be00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31c0713a0 con 0x7ff31c0c7f60 2024-03-09T08:53:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.679+0000 7ff31be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31c070670 con 0x7ff31c0ac930 2024-03-09T08:53:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.679+0000 7ff31be00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31c0c39a0 con 0x7ff31c1514f0 2024-03-09T08:53:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.679+0000 7ff31aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff31c0ac930 0x7ff31c0c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.679+0000 7ff31aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff31c0ac930 0x7ff31c0c3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35470/0 (socket says 172.21.15.67:35470) 2024-03-09T08:53:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.679+0000 7ff31aa00640 1 -- 172.21.15.67:0/1516385614 learned_addr learned my addr 172.21.15.67:0/1516385614 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.680+0000 7ff31b400640 1 --2- 172.21.15.67:0/1516385614 >> v2:172.21.15.134:3300/0 conn(0x7ff31c1514f0 0x7ff31c153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.680+0000 7ff31aa00640 1 -- 172.21.15.67:0/1516385614 >> v2:172.21.15.134:3300/0 conn(0x7ff31c1514f0 msgr2=0x7ff31c153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:06.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.680+0000 7ff31aa00640 1 --2- 172.21.15.67:0/1516385614 >> v2:172.21.15.134:3300/0 conn(0x7ff31c1514f0 0x7ff31c153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.680+0000 7ff31aa00640 1 -- 172.21.15.67:0/1516385614 >> v1:172.21.15.134:6789/0 conn(0x7ff31c0c7f60 legacy=0x7ff31c150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:06.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.680+0000 7ff31aa00640 1 -- 172.21.15.67:0/1516385614 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c0708c0 con 0x7ff31c0ac930 2024-03-09T08:53:06.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.682+0000 7ff31aa00640 1 --2- 172.21.15.67:0/1516385614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff31c0ac930 0x7ff31c0c3030 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0c2930 tx=0x7ff30c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8418d727e549baf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:06.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.683+0000 7ff31a000640 1 -- 172.21.15.67:0/1516385614 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c035020 con 0x7ff31c0ac930 2024-03-09T08:53:06.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.683+0000 7ff31a000640 1 -- 172.21.15.67:0/1516385614 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c031870 con 0x7ff31c0ac930 2024-03-09T08:53:06.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.683+0000 7ff31a000640 1 -- 172.21.15.67:0/1516385614 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c031c40 con 0x7ff31c0ac930 2024-03-09T08:53:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.683+0000 7ff31be00640 1 -- 172.21.15.67:0/1516385614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff31c0ac930 msgr2=0x7ff31c0c3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.683+0000 7ff31be00640 1 --2- 172.21.15.67:0/1516385614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff31c0ac930 0x7ff31c0c3030 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0c2930 tx=0x7ff30c02b910 comp rx=0 tx=0).stop 2024-03-09T08:53:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.683+0000 7ff31be00640 1 -- 172.21.15.67:0/1516385614 shutdown_connections 2024-03-09T08:53:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.683+0000 7ff31be00640 1 --2- 172.21.15.67:0/1516385614 >> v2:172.21.15.134:3300/0 conn(0x7ff31c1514f0 0x7ff31c153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.683+0000 7ff31be00640 1 --2- 172.21.15.67:0/1516385614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff31c0ac930 0x7ff31c0c3030 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.683+0000 7ff31be00640 1 -- 172.21.15.67:0/1516385614 >> 172.21.15.67:0/1516385614 conn(0x7ff31c06b8a0 msgr2=0x7ff31c0c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.684+0000 7ff31be00640 1 -- 172.21.15.67:0/1516385614 shutdown_connections 2024-03-09T08:53:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.684+0000 7ff31be00640 1 -- 172.21.15.67:0/1516385614 wait complete. 2024-03-09T08:53:06.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.685+0000 7ff31be00640 1 Processor -- start 2024-03-09T08:53:06.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.685+0000 7ff31be00640 1 -- start start 2024-03-09T08:53:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.686+0000 7ff31be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff31c0ac930 0x7ff31c1ea470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.686+0000 7ff31be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff31c0c7f60 0x7ff31c152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.686+0000 7ff31be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff31c1514f0 0x7ff31c1e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.686+0000 7ff31be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0c3570 con 0x7ff31c0c7f60 2024-03-09T08:53:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.686+0000 7ff31be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c070e80 con 0x7ff31c0ac930 2024-03-09T08:53:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.686+0000 7ff31be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c0c36f0 con 0x7ff31c1514f0 2024-03-09T08:53:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.686+0000 7ff313e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff31c0c7f60 0x7ff31c152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.686+0000 7ff313e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff31c0c7f60 0x7ff31c152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35480/0 (socket says 172.21.15.67:35480) 2024-03-09T08:53:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.686+0000 7ff313e00640 1 -- 172.21.15.67:0/1660760216 learned_addr learned my addr 172.21.15.67:0/1660760216 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff31aa00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff31c0ac930 0x7ff31c1ea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff313e00640 1 -- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff31c1514f0 msgr2=0x7ff31c1e40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff31b400640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff31c1514f0 0x7ff31c1e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff313e00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff31c1514f0 0x7ff31c1e40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff313e00640 1 -- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff31c0ac930 msgr2=0x7ff31c1ea470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff313e00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff31c0ac930 0x7ff31c1ea470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff313e00640 1 -- 172.21.15.67:0/1660760216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c0ab4d0 con 0x7ff31c0c7f60 2024-03-09T08:53:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff31aa00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff31c0ac930 0x7ff31c1ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff313e00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff31c0c7f60 0x7ff31c152170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c470 tx=0x7ff304001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff312a00640 1 -- 172.21.15.67:0/1660760216 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3040052d0 con 0x7ff31c0c7f60 2024-03-09T08:53:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff312a00640 1 -- 172.21.15.67:0/1660760216 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff304005470 con 0x7ff31c0c7f60 2024-03-09T08:53:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.687+0000 7ff312a00640 1 -- 172.21.15.67:0/1660760216 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304007260 con 0x7ff31c0c7f60 2024-03-09T08:53:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.688+0000 7ff31be00640 1 -- 172.21.15.67:0/1660760216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31c14cbb0 con 0x7ff31c0c7f60 2024-03-09T08:53:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.688+0000 7ff31be00640 1 -- 172.21.15.67:0/1660760216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31c14d170 con 0x7ff31c0c7f60 2024-03-09T08:53:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.689+0000 7ff31be00640 1 -- 172.21.15.67:0/1660760216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31c04f3c0 con 0x7ff31c0c7f60 2024-03-09T08:53:06.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.690+0000 7ff312a00640 1 -- 172.21.15.67:0/1660760216 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff304007400 con 0x7ff31c0c7f60 2024-03-09T08:53:06.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.691+0000 7ff312a00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2f806b550 0x7ff2f806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:06.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.691+0000 7ff312a00640 1 -- 172.21.15.67:0/1660760216 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7ff304089930 con 0x7ff31c0c7f60 2024-03-09T08:53:06.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.692+0000 7ff31aa00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2f806b550 0x7ff2f806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:06.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.693+0000 7ff31aa00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2f806b550 0x7ff2f806da10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff30c036000 tx=0x7ff30c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:06.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:06.696+0000 7ff312a00640 1 -- 172.21.15.67:0/1660760216 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff31c0ab4d0 con 0x7ff31c0c7f60 2024-03-09T08:53:07.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.039+0000 7ff31be00640 1 -- 172.21.15.67:0/1660760216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7ff31c14ade0 con 0x7ff31c0c7f60 2024-03-09T08:53:07.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.040+0000 7ff312a00640 1 -- 172.21.15.67:0/1660760216 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v26) v1 ==== 95+0+11 (secure 0 0 0) 0x7ff304058580 con 0x7ff31c0c7f60 2024-03-09T08:53:07.036 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T08:53:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.043+0000 7ff31be00640 1 -- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2f806b550 msgr2=0x7ff2f806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.043+0000 7ff31be00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2f806b550 0x7ff2f806da10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff30c036000 tx=0x7ff30c008030 comp rx=0 tx=0).stop 2024-03-09T08:53:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.043+0000 7ff31be00640 1 -- 172.21.15.67:0/1660760216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff31c0c7f60 msgr2=0x7ff31c152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.043+0000 7ff31be00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff31c0c7f60 0x7ff31c152170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c470 tx=0x7ff304001e70 comp rx=0 tx=0).stop 2024-03-09T08:53:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.044+0000 7ff31be00640 1 -- 172.21.15.67:0/1660760216 shutdown_connections 2024-03-09T08:53:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.044+0000 7ff31be00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2f806b550 0x7ff2f806da10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.044+0000 7ff31be00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff31c1514f0 0x7ff31c1e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.044+0000 7ff31be00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff31c0c7f60 0x7ff31c152170 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.044+0000 7ff31be00640 1 --2- 172.21.15.67:0/1660760216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff31c0ac930 0x7ff31c1ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.044+0000 7ff31be00640 1 -- 172.21.15.67:0/1660760216 >> 172.21.15.67:0/1660760216 conn(0x7ff31c06b8a0 msgr2=0x7ff31c073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:07.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.044+0000 7ff31be00640 1 -- 172.21.15.67:0/1660760216 shutdown_connections 2024-03-09T08:53:07.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.044+0000 7ff31be00640 1 -- 172.21.15.67:0/1660760216 wait complete. 2024-03-09T08:53:07.068 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-03-09T08:53:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.293+0000 7fe1cd400640 1 Processor -- start 2024-03-09T08:53:07.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.293+0000 7fe1cd400640 1 -- start start 2024-03-09T08:53:07.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.295+0000 7fe1cd400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe1c806b040 0x7fe1c80c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.295+0000 7fe1cd400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c80c9a30 con 0x7fe1c80c5f10 2024-03-09T08:53:07.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.295+0000 7fe1cd400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c80c9d10 con 0x7fe1c80c9490 2024-03-09T08:53:07.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.295+0000 7fe1cd400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c80ca340 con 0x7fe1c806b040 2024-03-09T08:53:07.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.295+0000 7fe1c7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe1c80c5f10 0x7fe1c81567d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55636/0 (socket says 172.21.15.67:55636) 2024-03-09T08:53:07.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.295+0000 7fe1c7e00640 1 -- 172.21.15.67:0/4044229627 learned_addr learned my addr 172.21.15.67:0/4044229627 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.296+0000 7fe1bfe00640 1 --2- 172.21.15.67:0/4044229627 >> v2:172.21.15.134:3300/0 conn(0x7fe1c806b040 0x7fe1c80c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.296+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112211025 0 0) 0x7fe1c80c9a30 con 0x7fe1c80c5f10 2024-03-09T08:53:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.296+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1ac003520 con 0x7fe1c80c5f10 2024-03-09T08:53:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.296+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 73687804 0 0) 0x7fe1ac003520 con 0x7fe1c80c5f10 2024-03-09T08:53:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.296+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 >> v2:172.21.15.134:3300/0 conn(0x7fe1c806b040 msgr2=0x7fe1c80c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.296+0000 7fe1c7400640 1 --2- 172.21.15.67:0/4044229627 >> v2:172.21.15.134:3300/0 conn(0x7fe1c806b040 0x7fe1c80c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.296+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 >> v1:172.21.15.134:6789/0 conn(0x7fe1c80c9490 legacy=0x7fe1c80cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.296+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80ca620 con 0x7fe1c80c5f10 2024-03-09T08:53:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.296+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fe1b8001670 con 0x7fe1c80c5f10 2024-03-09T08:53:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.296+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 >> v1:172.21.15.67:6789/0 conn(0x7fe1c80c5f10 legacy=0x7fe1c81567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.297+0000 7fe1c7400640 1 --2- 172.21.15.67:0/4044229627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ac0039b0 0x7fe1ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.297+0000 7fe1c7400640 1 --2- 172.21.15.67:0/4044229627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac004800 0x7fe1ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.297+0000 7fe1c7400640 1 --2- 172.21.15.67:0/4044229627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0051c0 0x7fe1ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.297+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80ca340 con 0x7fe1ac0039b0 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.297+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac00c160 con 0x7fe1ac0051c0 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.297+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac00c360 con 0x7fe1ac004800 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.297+0000 7fe1c7e00640 1 --2- 172.21.15.67:0/4044229627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ac0039b0 0x7fe1ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.297+0000 7fe1cca00640 1 --2- 172.21.15.67:0/4044229627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0051c0 0x7fe1ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.297+0000 7fe1bfe00640 1 --2- 172.21.15.67:0/4044229627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac004800 0x7fe1ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.298+0000 7fe1c7e00640 1 -- 172.21.15.67:0/4044229627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac004800 msgr2=0x7fe1ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.298+0000 7fe1c7e00640 1 --2- 172.21.15.67:0/4044229627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1ac004800 0x7fe1ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.298+0000 7fe1c7e00640 1 -- 172.21.15.67:0/4044229627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0051c0 msgr2=0x7fe1ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.298+0000 7fe1c7e00640 1 --2- 172.21.15.67:0/4044229627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0051c0 0x7fe1ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.298+0000 7fe1c7e00640 1 -- 172.21.15.67:0/4044229627 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac003520 con 0x7fe1ac0039b0 2024-03-09T08:53:07.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.298+0000 7fe1cca00640 1 -- 172.21.15.67:0/4044229627 reap_dead start 2024-03-09T08:53:07.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.300+0000 7fe1c7e00640 1 --2- 172.21.15.67:0/4044229627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ac0039b0 0x7fe1ac003dd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c210 tx=0x7fe1ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c33ff3de2d7341 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:07.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.300+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8007d30 con 0x7fe1ac0039b0 2024-03-09T08:53:07.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.300+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b8030070 con 0x7fe1ac0039b0 2024-03-09T08:53:07.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.300+0000 7fe1c7400640 1 -- 172.21.15.67:0/4044229627 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b802cda0 con 0x7fe1ac0039b0 2024-03-09T08:53:07.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.301+0000 7fe1cd400640 1 -- 172.21.15.67:0/4044229627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ac0039b0 msgr2=0x7fe1ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.301+0000 7fe1cd400640 1 --2- 172.21.15.67:0/4044229627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ac0039b0 0x7fe1ac003dd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c210 tx=0x7fe1ac005830 comp rx=0 tx=0).stop 2024-03-09T08:53:07.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.301+0000 7fe1cd400640 1 -- 172.21.15.67:0/4044229627 shutdown_connections 2024-03-09T08:53:07.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.301+0000 7fe1cd400640 1 --2- 172.21.15.67:0/4044229627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1ac0039b0 0x7fe1ac003dd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.301+0000 7fe1cd400640 1 -- 172.21.15.67:0/4044229627 >> 172.21.15.67:0/4044229627 conn(0x7fe1c80cb590 msgr2=0x7fe1c80cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:07.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.301+0000 7fe1cd400640 1 -- 172.21.15.67:0/4044229627 shutdown_connections 2024-03-09T08:53:07.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.301+0000 7fe1cd400640 1 -- 172.21.15.67:0/4044229627 wait complete. 2024-03-09T08:53:07.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.303+0000 7fe1cd400640 1 Processor -- start 2024-03-09T08:53:07.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.304+0000 7fe1cd400640 1 -- start start 2024-03-09T08:53:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cd400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0039b0 0x7fe1c81e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cd400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1c8159f90 0x7fe1c8155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c81ee7e0 0x7fe1c81eec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cd400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c81537a0 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cd400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c8153620 con 0x7fe1ac0039b0 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cd400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80c62f0 con 0x7fe1c8159f90 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c81ee7e0 0x7fe1c81eec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c81ee7e0 0x7fe1c81eec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35506/0 (socket says 172.21.15.67:35506) 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cca00640 1 -- 172.21.15.67:0/537620158 learned_addr learned my addr 172.21.15.67:0/537620158 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1c7e00640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0039b0 0x7fe1c81e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1bfe00640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1c8159f90 0x7fe1c8155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cca00640 1 -- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1c8159f90 msgr2=0x7fe1c8155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cca00640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1c8159f90 0x7fe1c8155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cca00640 1 -- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0039b0 msgr2=0x7fe1c81e2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cca00640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0039b0 0x7fe1c81e2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1cca00640 1 -- 172.21.15.67:0/537620158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c8155b70 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1bfe00640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1c8159f90 0x7fe1c8155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:07.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.305+0000 7fe1c7e00640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0039b0 0x7fe1c81e2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:07.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.306+0000 7fe1cca00640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c81ee7e0 0x7fe1c81eec90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80c5b50 tx=0x7fe1c000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:07.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.306+0000 7fe1c4c00640 1 -- 172.21.15.67:0/537620158 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c001a070 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.306+0000 7fe1c4c00640 1 -- 172.21.15.67:0/537620158 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c0006d60 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.306+0000 7fe1cd400640 1 -- 172.21.15.67:0/537620158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c81594b0 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.306+0000 7fe1c4c00640 1 -- 172.21.15.67:0/537620158 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0003bb0 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.306+0000 7fe1cd400640 1 -- 172.21.15.67:0/537620158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c8159a70 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.307+0000 7fe1cd400640 1 -- 172.21.15.67:0/537620158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c8159c70 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.308+0000 7fe1c4c00640 1 -- 172.21.15.67:0/537620158 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe1c001d070 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.309+0000 7fe1c4c00640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1a406b4d0 0x7fe1a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.309+0000 7fe1c4c00640 1 -- 172.21.15.67:0/537620158 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fe1c0089b20 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.310+0000 7fe1c7e00640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1a406b4d0 0x7fe1a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.311+0000 7fe1c7e00640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1a406b4d0 0x7fe1a406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8034000 tx=0x7fe1b8039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:07.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.314+0000 7fe1c4c00640 1 -- 172.21.15.67:0/537620158 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe1c0058560 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.514+0000 7fe1cd400640 1 -- 172.21.15.67:0/537620158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1c804f3c0 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.515+0000 7fe1c4c00640 1 -- 172.21.15.67:0/537620158 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v26) v1 ==== 99+0+11 (secure 0 0 0) 0x7fe1c0057e60 con 0x7fe1c81ee7e0 2024-03-09T08:53:07.511 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 -- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1a406b4d0 msgr2=0x7fe1a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1a406b4d0 0x7fe1a406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8034000 tx=0x7fe1b8039040 comp rx=0 tx=0).stop 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 -- 172.21.15.67:0/537620158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c81ee7e0 msgr2=0x7fe1c81eec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c81ee7e0 0x7fe1c81eec90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80c5b50 tx=0x7fe1c000fbe0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 -- 172.21.15.67:0/537620158 shutdown_connections 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe1a406b4d0 0x7fe1a406d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c81ee7e0 0x7fe1c81eec90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1c8159f90 0x7fe1c8155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 --2- 172.21.15.67:0/537620158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ac0039b0 0x7fe1c81e2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 -- 172.21.15.67:0/537620158 >> 172.21.15.67:0/537620158 conn(0x7fe1c80cb590 msgr2=0x7fe1c806b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 -- 172.21.15.67:0/537620158 shutdown_connections 2024-03-09T08:53:07.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.518+0000 7fe1cd400640 1 -- 172.21.15.67:0/537620158 wait complete. 2024-03-09T08:53:07.541 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-03-09T08:53:07.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.768+0000 7f818d000640 1 Processor -- start 2024-03-09T08:53:07.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.768+0000 7f818d000640 1 -- start start 2024-03-09T08:53:07.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.770+0000 7f818d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81880c7f60 0x7f8188150de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.770+0000 7f818d000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81880713a0 con 0x7f8188151320 2024-03-09T08:53:07.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.770+0000 7f818d000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81880c39a0 con 0x7f81880ac930 2024-03-09T08:53:07.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.770+0000 7f818d000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8188070670 con 0x7f81880c7f60 2024-03-09T08:53:07.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.770+0000 7f8187e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8188151320 0x7f8188153ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55642/0 (socket says 172.21.15.67:55642) 2024-03-09T08:53:07.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.770+0000 7f8187e00640 1 -- 172.21.15.67:0/7673719 learned_addr learned my addr 172.21.15.67:0/7673719 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:07.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.770+0000 7f817fe00640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81880c7f60 0x7f8188150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302322145 0 0) 0x7f81880713a0 con 0x7f8188151320 2024-03-09T08:53:07.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f816c003520 con 0x7f8188151320 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438518981 0 0) 0x7f81880c39a0 con 0x7f81880ac930 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81880713a0 con 0x7f81880ac930 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2614862034 0 0) 0x7f816c003520 con 0x7f8188151320 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81880c7f60 msgr2=0x7f8188150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81880c7f60 0x7f8188150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 >> v1:172.21.15.134:6789/0 conn(0x7f81880ac930 legacy=0x7f81880c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880708c0 con 0x7f8188151320 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f8180001970 con 0x7f8188151320 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 >> v1:172.21.15.67:6789/0 conn(0x7f8188151320 legacy=0x7f8188153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f816c003b70 0x7f816c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81880c7f60 0x7f816c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f816c004d40 0x7f816c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8188070670 con 0x7f816c003b70 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c00bb50 con 0x7f816c004d40 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.771+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c00bd20 con 0x7f81880c7f60 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.772+0000 7f8187400640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f816c003b70 0x7f816c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.772+0000 7f8187e00640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f816c004d40 0x7f816c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.772+0000 7f817fe00640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81880c7f60 0x7f816c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.772+0000 7f8187400640 1 -- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81880c7f60 msgr2=0x7f816c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.772+0000 7f8187400640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81880c7f60 0x7f816c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.772+0000 7f8187400640 1 -- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f816c004d40 msgr2=0x7f816c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.772+0000 7f8187400640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f816c004d40 0x7f816c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.772+0000 7f8187400640 1 -- 172.21.15.67:0/7673719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c003520 con 0x7f816c003b70 2024-03-09T08:53:07.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.772+0000 7f817fe00640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81880c7f60 0x7f816c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:07.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.773+0000 7f8187400640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f816c003b70 0x7f816c003f70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8178001900 tx=0x7f816c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e313a48eb0e637c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.774+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178037070 con 0x7f816c003b70 2024-03-09T08:53:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.774+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81780052d0 con 0x7f816c003b70 2024-03-09T08:53:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.774+0000 7f8186a00640 1 -- 172.21.15.67:0/7673719 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780014f0 con 0x7f816c003b70 2024-03-09T08:53:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.774+0000 7f818d000640 1 -- 172.21.15.67:0/7673719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f816c003b70 msgr2=0x7f816c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.774+0000 7f818d000640 1 --2- 172.21.15.67:0/7673719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f816c003b70 0x7f816c003f70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8178001900 tx=0x7f816c005830 comp rx=0 tx=0).stop 2024-03-09T08:53:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.774+0000 7f8187e00640 1 -- 172.21.15.67:0/7673719 reap_dead start 2024-03-09T08:53:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.774+0000 7f818d000640 1 -- 172.21.15.67:0/7673719 shutdown_connections 2024-03-09T08:53:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.774+0000 7f818d000640 1 -- 172.21.15.67:0/7673719 >> 172.21.15.67:0/7673719 conn(0x7f818806b8a0 msgr2=0x7f81880c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.774+0000 7f818d000640 1 -- 172.21.15.67:0/7673719 shutdown_connections 2024-03-09T08:53:07.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.774+0000 7f818d000640 1 -- 172.21.15.67:0/7673719 wait complete. 2024-03-09T08:53:07.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.776+0000 7f818d000640 1 Processor -- start 2024-03-09T08:53:07.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.776+0000 7f818d000640 1 -- start start 2024-03-09T08:53:07.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.777+0000 7f818d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f816c003b70 0x7f818814c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.777+0000 7f818d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81881547d0 0x7f818814c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.777+0000 7f818d000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81881ee640 0x7f81881eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.777+0000 7f818d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8188151770 con 0x7f81881547d0 2024-03-09T08:53:07.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.777+0000 7f818d000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81880c3740 con 0x7f81881ee640 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.777+0000 7f818d000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8188070e80 con 0x7f816c003b70 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.777+0000 7f817fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81881547d0 0x7f818814c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.778+0000 7f817fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81881547d0 0x7f818814c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35524/0 (socket says 172.21.15.67:35524) 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.778+0000 7f817fe00640 1 -- 172.21.15.67:0/251215323 learned_addr learned my addr 172.21.15.67:0/251215323 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.778+0000 7f8187400640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f816c003b70 0x7f818814c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.778+0000 7f8187e00640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81881ee640 0x7f81881eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.778+0000 7f817fe00640 1 -- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f816c003b70 msgr2=0x7f818814c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.778+0000 7f817fe00640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f816c003b70 0x7f818814c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.778+0000 7f817fe00640 1 -- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81881ee640 msgr2=0x7f81881eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.778+0000 7f817fe00640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81881ee640 0x7f81881eeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.778+0000 7f817fe00640 1 -- 172.21.15.67:0/251215323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81881528e0 con 0x7f81881547d0 2024-03-09T08:53:07.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.778+0000 7f817fe00640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81881547d0 0x7f818814c8e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f81700108e0 tx=0x7f8170010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:07.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.778+0000 7f817f400640 1 -- 172.21.15.67:0/251215323 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817000f040 con 0x7f81881547d0 2024-03-09T08:53:07.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.779+0000 7f817f400640 1 -- 172.21.15.67:0/251215323 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817000ec20 con 0x7f81881547d0 2024-03-09T08:53:07.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.779+0000 7f818d000640 1 -- 172.21.15.67:0/251215323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8188151d10 con 0x7f81881547d0 2024-03-09T08:53:07.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.779+0000 7f817f400640 1 -- 172.21.15.67:0/251215323 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817001c400 con 0x7f81881547d0 2024-03-09T08:53:07.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.779+0000 7f818d000640 1 -- 172.21.15.67:0/251215323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818814be80 con 0x7f81881547d0 2024-03-09T08:53:07.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.780+0000 7f818d000640 1 -- 172.21.15.67:0/251215323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818804f3c0 con 0x7f81881547d0 2024-03-09T08:53:07.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.781+0000 7f817f400640 1 -- 172.21.15.67:0/251215323 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f817001c5a0 con 0x7f81881547d0 2024-03-09T08:53:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.782+0000 7f817f400640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f816406b4d0 0x7f816406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.782+0000 7f817f400640 1 -- 172.21.15.67:0/251215323 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f817008acf0 con 0x7f81881547d0 2024-03-09T08:53:07.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.782+0000 7f8187400640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f816406b4d0 0x7f816406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:07.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.784+0000 7f8187400640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f816406b4d0 0x7f816406d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8178034000 tx=0x7f8178031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:07.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.786+0000 7f817f400640 1 -- 172.21.15.67:0/251215323 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f817008f070 con 0x7f81881547d0 2024-03-09T08:53:07.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.986+0000 7f818d000640 1 -- 172.21.15.67:0/251215323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7f818814ce20 con 0x7f81881547d0 2024-03-09T08:53:07.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.987+0000 7f817f400640 1 -- 172.21.15.67:0/251215323 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v26) v1 ==== 95+0+11 (secure 0 0 0) 0x7f8170018030 con 0x7f81881547d0 2024-03-09T08:53:07.983 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T08:53:07.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.989+0000 7f818d000640 1 -- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f816406b4d0 msgr2=0x7f816406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.989+0000 7f818d000640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f816406b4d0 0x7f816406d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8178034000 tx=0x7f8178031000 comp rx=0 tx=0).stop 2024-03-09T08:53:07.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.989+0000 7f818d000640 1 -- 172.21.15.67:0/251215323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81881547d0 msgr2=0x7f818814c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:07.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.989+0000 7f818d000640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81881547d0 0x7f818814c8e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f81700108e0 tx=0x7f8170010da0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.989+0000 7f818d000640 1 -- 172.21.15.67:0/251215323 shutdown_connections 2024-03-09T08:53:07.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.989+0000 7f818d000640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f816406b4d0 0x7f816406d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.990+0000 7f818d000640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81881ee640 0x7f81881eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.990+0000 7f818d000640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81881547d0 0x7f818814c8e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.990+0000 7f818d000640 1 --2- 172.21.15.67:0/251215323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f816c003b70 0x7f818814c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:07.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.990+0000 7f818d000640 1 -- 172.21.15.67:0/251215323 >> 172.21.15.67:0/251215323 conn(0x7f818806b8a0 msgr2=0x7f818814ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:07.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.990+0000 7f818d000640 1 -- 172.21.15.67:0/251215323 shutdown_connections 2024-03-09T08:53:07.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:07.990+0000 7f818d000640 1 -- 172.21.15.67:0/251215323 wait complete. 2024-03-09T08:53:08.005 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-03-09T08:53:08.005 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T08:53:08.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.215+0000 7fc954a00640 1 Processor -- start 2024-03-09T08:53:08.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.215+0000 7fc954a00640 1 -- start start 2024-03-09T08:53:08.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.216+0000 7fc954a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc95014c5c0 0x7fc950152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.216+0000 7fc954a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9500c3780 0x7fc95006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.216+0000 7fc954a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc95006e4d0 0x7fc9500aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.216+0000 7fc954a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9500aaae0 con 0x7fc95014c5c0 2024-03-09T08:53:08.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.216+0000 7fc954a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95014c9a0 con 0x7fc95006e4d0 2024-03-09T08:53:08.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.216+0000 7fc954a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500c1d70 con 0x7fc9500c3780 2024-03-09T08:53:08.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.216+0000 7fc94de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc95014c5c0 0x7fc950152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.216+0000 7fc94de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc95014c5c0 0x7fc950152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35526/0 (socket says 172.21.15.67:35526) 2024-03-09T08:53:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.217+0000 7fc94de00640 1 -- 172.21.15.67:0/4048322135 learned_addr learned my addr 172.21.15.67:0/4048322135 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.217+0000 7fc94d400640 1 --2- 172.21.15.67:0/4048322135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9500c3780 0x7fc95006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.217+0000 7fc94e800640 1 --2- 172.21.15.67:0/4048322135 >> v2:172.21.15.134:3300/0 conn(0x7fc95006e4d0 0x7fc9500aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.217+0000 7fc94de00640 1 -- 172.21.15.67:0/4048322135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9500c3780 msgr2=0x7fc95006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.217+0000 7fc94de00640 1 --2- 172.21.15.67:0/4048322135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9500c3780 0x7fc95006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.217+0000 7fc94de00640 1 -- 172.21.15.67:0/4048322135 >> v2:172.21.15.134:3300/0 conn(0x7fc95006e4d0 msgr2=0x7fc9500aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.217+0000 7fc94de00640 1 --2- 172.21.15.67:0/4048322135 >> v2:172.21.15.134:3300/0 conn(0x7fc95006e4d0 0x7fc9500aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.217+0000 7fc94de00640 1 -- 172.21.15.67:0/4048322135 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9500c2010 con 0x7fc95014c5c0 2024-03-09T08:53:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.217+0000 7fc94e800640 1 --2- 172.21.15.67:0/4048322135 >> v2:172.21.15.134:3300/0 conn(0x7fc95006e4d0 0x7fc9500aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:08.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.218+0000 7fc94de00640 1 --2- 172.21.15.67:0/4048322135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc95014c5c0 0x7fc950152ce0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc950070710 tx=0x7fc94002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1185580c4c9a2cbd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:08.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc94ca00640 1 -- 172.21.15.67:0/4048322135 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9400317d0 con 0x7fc95014c5c0 2024-03-09T08:53:08.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc94ca00640 1 -- 172.21.15.67:0/4048322135 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc940037070 con 0x7fc95014c5c0 2024-03-09T08:53:08.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc94ca00640 1 -- 172.21.15.67:0/4048322135 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940031b90 con 0x7fc95014c5c0 2024-03-09T08:53:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc954a00640 1 -- 172.21.15.67:0/4048322135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc95014c5c0 msgr2=0x7fc950152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc954a00640 1 --2- 172.21.15.67:0/4048322135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc95014c5c0 0x7fc950152ce0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc950070710 tx=0x7fc94002b870 comp rx=0 tx=0).stop 2024-03-09T08:53:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc954a00640 1 -- 172.21.15.67:0/4048322135 shutdown_connections 2024-03-09T08:53:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc954a00640 1 --2- 172.21.15.67:0/4048322135 >> v2:172.21.15.134:3300/0 conn(0x7fc95006e4d0 0x7fc9500aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc954a00640 1 --2- 172.21.15.67:0/4048322135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9500c3780 0x7fc95006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc954a00640 1 --2- 172.21.15.67:0/4048322135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc95014c5c0 0x7fc950152ce0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc954a00640 1 -- 172.21.15.67:0/4048322135 >> 172.21.15.67:0/4048322135 conn(0x7fc9500c7370 msgr2=0x7fc9500c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc954a00640 1 -- 172.21.15.67:0/4048322135 shutdown_connections 2024-03-09T08:53:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.219+0000 7fc954a00640 1 -- 172.21.15.67:0/4048322135 wait complete. 2024-03-09T08:53:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.221+0000 7fc954a00640 1 Processor -- start 2024-03-09T08:53:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.221+0000 7fc954a00640 1 -- start start 2024-03-09T08:53:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.222+0000 7fc954a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc95006e4d0 0x7fc950143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.222+0000 7fc954a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9500c3780 0x7fc95014ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.222+0000 7fc954a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc95014c5c0 0x7fc9501403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.222+0000 7fc954a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500c28b0 con 0x7fc95014c5c0 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.222+0000 7fc954a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500c25d0 con 0x7fc9500c3780 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.222+0000 7fc954a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500ab240 con 0x7fc95006e4d0 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.222+0000 7fc94e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc95014c5c0 0x7fc9501403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.222+0000 7fc94e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc95014c5c0 0x7fc9501403d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35540/0 (socket says 172.21.15.67:35540) 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.222+0000 7fc94e800640 1 -- 172.21.15.67:0/2359431566 learned_addr learned my addr 172.21.15.67:0/2359431566 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.222+0000 7fc94d400640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9500c3780 0x7fc95014ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.223+0000 7fc94de00640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc95006e4d0 0x7fc950143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.223+0000 7fc94e800640 1 -- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc95006e4d0 msgr2=0x7fc950143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.223+0000 7fc94e800640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc95006e4d0 0x7fc950143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.223+0000 7fc94e800640 1 -- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9500c3780 msgr2=0x7fc95014ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.223+0000 7fc94e800640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9500c3780 0x7fc95014ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.223+0000 7fc94e800640 1 -- 172.21.15.67:0/2359431566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950143c90 con 0x7fc95014c5c0 2024-03-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.223+0000 7fc94e800640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc95014c5c0 0x7fc9501403d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc94800cef0 tx=0x7fc94800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.223+0000 7fc946000640 1 -- 172.21.15.67:0/2359431566 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94801a070 con 0x7fc95014c5c0 2024-03-09T08:53:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.223+0000 7fc946000640 1 -- 172.21.15.67:0/2359431566 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94800d840 con 0x7fc95014c5c0 2024-03-09T08:53:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.223+0000 7fc954a00640 1 -- 172.21.15.67:0/2359431566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc950143f70 con 0x7fc95014c5c0 2024-03-09T08:53:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.223+0000 7fc946000640 1 -- 172.21.15.67:0/2359431566 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800dc30 con 0x7fc95014c5c0 2024-03-09T08:53:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.224+0000 7fc954a00640 1 -- 172.21.15.67:0/2359431566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc950144530 con 0x7fc95014c5c0 2024-03-09T08:53:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.225+0000 7fc954a00640 1 -- 172.21.15.67:0/2359431566 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc950075a00 con 0x7fc95014c5c0 2024-03-09T08:53:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.226+0000 7fc946000640 1 -- 172.21.15.67:0/2359431566 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc94801d070 con 0x7fc95014c5c0 2024-03-09T08:53:08.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.227+0000 7fc946000640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc93006b4d0 0x7fc93006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.227+0000 7fc946000640 1 -- 172.21.15.67:0/2359431566 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fc94808ae40 con 0x7fc95014c5c0 2024-03-09T08:53:08.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.227+0000 7fc94de00640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc93006b4d0 0x7fc93006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.228+0000 7fc94de00640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc93006b4d0 0x7fc93006d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc940036000 tx=0x7fc94003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:08.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.232+0000 7fc946000640 1 -- 172.21.15.67:0/2359431566 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc948059910 con 0x7fc95014c5c0 2024-03-09T08:53:08.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.432+0000 7fc954a00640 1 -- 172.21.15.67:0/2359431566 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc95014fc80 con 0x7fc93006b4d0 2024-03-09T08:53:08.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.463+0000 7fc946000640 1 -- 172.21.15.67:0/2359431566 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490412 (secure 0 0 0) 0x7fc95014fc80 con 0x7fc93006b4d0 2024-03-09T08:53:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:53:08.469 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T08:53:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 -- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc93006b4d0 msgr2=0x7fc93006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc93006b4d0 0x7fc93006d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc940036000 tx=0x7fc94003d040 comp rx=0 tx=0).stop 2024-03-09T08:53:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 -- 172.21.15.67:0/2359431566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc95014c5c0 msgr2=0x7fc9501403d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc95014c5c0 0x7fc9501403d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc94800cef0 tx=0x7fc94800a710 comp rx=0 tx=0).stop 2024-03-09T08:53:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 -- 172.21.15.67:0/2359431566 shutdown_connections 2024-03-09T08:53:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc93006b4d0 0x7fc93006d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc95014c5c0 0x7fc9501403d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9500c3780 0x7fc95014ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 --2- 172.21.15.67:0/2359431566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc95006e4d0 0x7fc950143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 -- 172.21.15.67:0/2359431566 >> 172.21.15.67:0/2359431566 conn(0x7fc9500c7370 msgr2=0x7fc950073030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 -- 172.21.15.67:0/2359431566 shutdown_connections 2024-03-09T08:53:08.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.470+0000 7fc954a00640 1 -- 172.21.15.67:0/2359431566 wait complete. 2024-03-09T08:53:08.489 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-03-09T08:53:06.694034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595446,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":251,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219644,"kb_used_data":3464,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755076,"statfs":{"total":773094113280,"available":772869197824,"internally_reserved":0,"allocated":3547136,"data_stored":2095428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004342"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790031+0000","last_change":"2024-03-09T08:52:29.573571+0000","last_active":"2024-03-09T08:52:59.790031+0000","last_peered":"2024-03-09T08:52:59.790031+0000","last_clean":"2024-03-09T08:52:59.790031+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T08:52:59.790031+0000","last_undegraded":"2024-03-09T08:52:59.790031+0000","last_fullsized":"2024-03-09T08:52:59.790031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:33:33.068463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794135+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T08:52:59.794135+0000","last_peered":"2024-03-09T08:52:59.794135+0000","last_clean":"2024-03-09T08:52:59.794135+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T08:52:59.794135+0000","last_undegraded":"2024-03-09T08:52:59.794135+0000","last_fullsized":"2024-03-09T08:52:59.794135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:41:40.332687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793042+0000","last_change":"2024-03-09T08:52:18.235099+0000","last_active":"2024-03-09T08:52:59.793042+0000","last_peered":"2024-03-09T08:52:59.793042+0000","last_clean":"2024-03-09T08:52:59.793042+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T08:52:59.793042+0000","last_undegraded":"2024-03-09T08:52:59.793042+0000","last_fullsized":"2024-03-09T08:52:59.793042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:39:28.882461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790160+0000","last_change":"2024-03-09T08:52:28.571905+0000","last_active":"2024-03-09T08:52:59.790160+0000","last_peered":"2024-03-09T08:52:59.790160+0000","last_clean":"2024-03-09T08:52:59.790160+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T08:52:59.790160+0000","last_undegraded":"2024-03-09T08:52:59.790160+0000","last_fullsized":"2024-03-09T08:52:59.790160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:01.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.249405+0000","last_change":"2024-03-09T08:52:29.578644+0000","last_active":"2024-03-09T08:53:00.249405+0000","last_peered":"2024-03-09T08:53:00.249405+0000","last_clean":"2024-03-09T08:53:00.249405+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T08:53:00.249405+0000","last_undegraded":"2024-03-09T08:53:00.249405+0000","last_fullsized":"2024-03-09T08:53:00.249405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:00.044620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793123+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T08:52:59.793123+0000","last_peered":"2024-03-09T08:52:59.793123+0000","last_clean":"2024-03-09T08:52:59.793123+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T08:52:59.793123+0000","last_undegraded":"2024-03-09T08:52:59.793123+0000","last_fullsized":"2024-03-09T08:52:59.793123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793800+0000","last_change":"2024-03-09T08:52:18.236946+0000","last_active":"2024-03-09T08:52:59.793800+0000","last_peered":"2024-03-09T08:52:59.793800+0000","last_clean":"2024-03-09T08:52:59.793800+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T08:52:59.793800+0000","last_undegraded":"2024-03-09T08:52:59.793800+0000","last_fullsized":"2024-03-09T08:52:59.793800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:45.372160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790242+0000","last_change":"2024-03-09T08:52:28.573074+0000","last_active":"2024-03-09T08:52:59.790242+0000","last_peered":"2024-03-09T08:52:59.790242+0000","last_clean":"2024-03-09T08:52:59.790242+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T08:52:59.790242+0000","last_undegraded":"2024-03-09T08:52:59.790242+0000","last_fullsized":"2024-03-09T08:52:59.790242+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:50.600408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236942+0000","last_change":"2024-03-09T08:52:29.594030+0000","last_active":"2024-03-09T08:53:00.236942+0000","last_peered":"2024-03-09T08:53:00.236942+0000","last_clean":"2024-03-09T08:53:00.236942+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T08:53:00.236942+0000","last_undegraded":"2024-03-09T08:53:00.236942+0000","last_fullsized":"2024-03-09T08:53:00.236942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:57.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238298+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T08:53:00.238298+0000","last_peered":"2024-03-09T08:53:00.238298+0000","last_clean":"2024-03-09T08:53:00.238298+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T08:53:00.238298+0000","last_undegraded":"2024-03-09T08:53:00.238298+0000","last_fullsized":"2024-03-09T08:53:00.238298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435811+0000","last_change":"2024-03-09T08:52:18.230629+0000","last_active":"2024-03-09T08:53:00.435811+0000","last_peered":"2024-03-09T08:53:00.435811+0000","last_clean":"2024-03-09T08:53:00.435811+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T08:53:00.435811+0000","last_undegraded":"2024-03-09T08:53:00.435811+0000","last_fullsized":"2024-03-09T08:53:00.435811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:46:38.528839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792389+0000","last_change":"2024-03-09T08:52:28.577128+0000","last_active":"2024-03-09T08:52:59.792389+0000","last_peered":"2024-03-09T08:52:59.792389+0000","last_clean":"2024-03-09T08:52:59.792389+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T08:52:59.792389+0000","last_undegraded":"2024-03-09T08:52:59.792389+0000","last_fullsized":"2024-03-09T08:52:59.792389+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:41.801752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434188+0000","last_change":"2024-03-09T08:52:29.581705+0000","last_active":"2024-03-09T08:53:00.434188+0000","last_peered":"2024-03-09T08:53:00.434188+0000","last_clean":"2024-03-09T08:53:00.434188+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T08:53:00.434188+0000","last_undegraded":"2024-03-09T08:53:00.434188+0000","last_fullsized":"2024-03-09T08:53:00.434188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:27.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238318+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T08:53:00.238318+0000","last_peered":"2024-03-09T08:53:00.238318+0000","last_clean":"2024-03-09T08:53:00.238318+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T08:53:00.238318+0000","last_undegraded":"2024-03-09T08:53:00.238318+0000","last_fullsized":"2024-03-09T08:53:00.238318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435922+0000","last_change":"2024-03-09T08:52:18.228419+0000","last_active":"2024-03-09T08:53:00.435922+0000","last_peered":"2024-03-09T08:53:00.435922+0000","last_clean":"2024-03-09T08:53:00.435922+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T08:53:00.435922+0000","last_undegraded":"2024-03-09T08:53:00.435922+0000","last_fullsized":"2024-03-09T08:53:00.435922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:19:12.533402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791700+0000","last_change":"2024-03-09T08:52:28.577069+0000","last_active":"2024-03-09T08:52:59.791700+0000","last_peered":"2024-03-09T08:52:59.791700+0000","last_clean":"2024-03-09T08:52:59.791700+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T08:52:59.791700+0000","last_undegraded":"2024-03-09T08:52:59.791700+0000","last_fullsized":"2024-03-09T08:52:59.791700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:09:02.410915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250908+0000","last_change":"2024-03-09T08:52:29.588922+0000","last_active":"2024-03-09T08:53:00.250908+0000","last_peered":"2024-03-09T08:53:00.250908+0000","last_clean":"2024-03-09T08:53:00.250908+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T08:53:00.250908+0000","last_undegraded":"2024-03-09T08:53:00.250908+0000","last_fullsized":"2024-03-09T08:53:00.250908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:13.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793277+0000","last_change":"2024-03-09T08:52:18.234372+0000","last_active":"2024-03-09T08:52:59.793277+0000","last_peered":"2024-03-09T08:52:59.793277+0000","last_clean":"2024-03-09T08:52:59.793277+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T08:52:59.793277+0000","last_undegraded":"2024-03-09T08:52:59.793277+0000","last_fullsized":"2024-03-09T08:52:59.793277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:09:33.422907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238094+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T08:53:00.238094+0000","last_peered":"2024-03-09T08:53:00.238094+0000","last_clean":"2024-03-09T08:53:00.238094+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T08:53:00.238094+0000","last_undegraded":"2024-03-09T08:53:00.238094+0000","last_fullsized":"2024-03-09T08:53:00.238094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794427+0000","last_change":"2024-03-09T08:52:28.570631+0000","last_active":"2024-03-09T08:52:59.794427+0000","last_peered":"2024-03-09T08:52:59.794427+0000","last_clean":"2024-03-09T08:52:59.794427+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T08:52:59.794427+0000","last_undegraded":"2024-03-09T08:52:59.794427+0000","last_fullsized":"2024-03-09T08:52:59.794427+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:06.630704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.799005+0000","last_change":"2024-03-09T08:52:29.596820+0000","last_active":"2024-03-09T08:52:59.799005+0000","last_peered":"2024-03-09T08:52:59.799005+0000","last_clean":"2024-03-09T08:52:59.799005+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T08:52:59.799005+0000","last_undegraded":"2024-03-09T08:52:59.799005+0000","last_fullsized":"2024-03-09T08:52:59.799005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:01:20.331889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790671+0000","last_change":"2024-03-09T08:52:18.244306+0000","last_active":"2024-03-09T08:52:59.790671+0000","last_peered":"2024-03-09T08:52:59.790671+0000","last_clean":"2024-03-09T08:52:59.790671+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T08:52:59.790671+0000","last_undegraded":"2024-03-09T08:52:59.790671+0000","last_fullsized":"2024-03-09T08:52:59.790671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:58.365524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250537+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T08:53:00.250537+0000","last_peered":"2024-03-09T08:53:00.250537+0000","last_clean":"2024-03-09T08:53:00.250537+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T08:53:00.250537+0000","last_undegraded":"2024-03-09T08:53:00.250537+0000","last_fullsized":"2024-03-09T08:53:00.250537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:45:45.612953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792259+0000","last_change":"2024-03-09T08:52:28.577110+0000","last_active":"2024-03-09T08:52:59.792259+0000","last_peered":"2024-03-09T08:52:59.792259+0000","last_clean":"2024-03-09T08:52:59.792259+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T08:52:59.792259+0000","last_undegraded":"2024-03-09T08:52:59.792259+0000","last_fullsized":"2024-03-09T08:52:59.792259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:57.197373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794138+0000","last_change":"2024-03-09T08:52:29.579929+0000","last_active":"2024-03-09T08:52:59.794138+0000","last_peered":"2024-03-09T08:52:59.794138+0000","last_clean":"2024-03-09T08:52:59.794138+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T08:52:59.794138+0000","last_undegraded":"2024-03-09T08:52:59.794138+0000","last_fullsized":"2024-03-09T08:52:59.794138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:37:23.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789838+0000","last_change":"2024-03-09T08:52:18.233208+0000","last_active":"2024-03-09T08:52:59.789838+0000","last_peered":"2024-03-09T08:52:59.789838+0000","last_clean":"2024-03-09T08:52:59.789838+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T08:52:59.789838+0000","last_undegraded":"2024-03-09T08:52:59.789838+0000","last_fullsized":"2024-03-09T08:52:59.789838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:17.589783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.249867+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T08:53:00.249867+0000","last_peered":"2024-03-09T08:53:00.249867+0000","last_clean":"2024-03-09T08:53:00.249867+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T08:53:00.249867+0000","last_undegraded":"2024-03-09T08:53:00.249867+0000","last_fullsized":"2024-03-09T08:53:00.249867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:21:59.990396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252140+0000","last_change":"2024-03-09T08:52:28.576254+0000","last_active":"2024-03-09T08:53:00.252140+0000","last_peered":"2024-03-09T08:53:00.252140+0000","last_clean":"2024-03-09T08:53:00.252140+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T08:53:00.252140+0000","last_undegraded":"2024-03-09T08:53:00.252140+0000","last_fullsized":"2024-03-09T08:53:00.252140+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:21:56.330955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.436326+0000","last_change":"2024-03-09T08:52:29.578272+0000","last_active":"2024-03-09T08:53:00.436326+0000","last_peered":"2024-03-09T08:53:00.436326+0000","last_clean":"2024-03-09T08:53:00.436326+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T08:53:00.436326+0000","last_undegraded":"2024-03-09T08:53:00.436326+0000","last_fullsized":"2024-03-09T08:53:00.436326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:56.180809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791263+0000","last_change":"2024-03-09T08:52:18.217046+0000","last_active":"2024-03-09T08:52:59.791263+0000","last_peered":"2024-03-09T08:52:59.791263+0000","last_clean":"2024-03-09T08:52:59.791263+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T08:52:59.791263+0000","last_undegraded":"2024-03-09T08:52:59.791263+0000","last_fullsized":"2024-03-09T08:52:59.791263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:14.884963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237306+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T08:53:00.237306+0000","last_peered":"2024-03-09T08:53:00.237306+0000","last_clean":"2024-03-09T08:53:00.237306+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T08:53:00.237306+0000","last_undegraded":"2024-03-09T08:53:00.237306+0000","last_fullsized":"2024-03-09T08:53:00.237306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794145+0000","last_change":"2024-03-09T08:52:28.573638+0000","last_active":"2024-03-09T08:52:59.794145+0000","last_peered":"2024-03-09T08:52:59.794145+0000","last_clean":"2024-03-09T08:52:59.794145+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T08:52:59.794145+0000","last_undegraded":"2024-03-09T08:52:59.794145+0000","last_fullsized":"2024-03-09T08:52:59.794145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:02.478112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.788764+0000","last_change":"2024-03-09T08:52:29.597697+0000","last_active":"2024-03-09T08:52:59.788764+0000","last_peered":"2024-03-09T08:52:59.788764+0000","last_clean":"2024-03-09T08:52:59.788764+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T08:52:59.788764+0000","last_undegraded":"2024-03-09T08:52:59.788764+0000","last_fullsized":"2024-03-09T08:52:59.788764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:00.890845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796799+0000","last_change":"2024-03-09T08:52:18.229739+0000","last_active":"2024-03-09T08:52:59.796799+0000","last_peered":"2024-03-09T08:52:59.796799+0000","last_clean":"2024-03-09T08:52:59.796799+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T08:52:59.796799+0000","last_undegraded":"2024-03-09T08:52:59.796799+0000","last_fullsized":"2024-03-09T08:52:59.796799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:24.394779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795614+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T08:52:59.795614+0000","last_peered":"2024-03-09T08:52:59.795614+0000","last_clean":"2024-03-09T08:52:59.795614+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T08:52:59.795614+0000","last_undegraded":"2024-03-09T08:52:59.795614+0000","last_fullsized":"2024-03-09T08:52:59.795614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:35:38.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790482+0000","last_change":"2024-03-09T08:52:28.575630+0000","last_active":"2024-03-09T08:52:59.790482+0000","last_peered":"2024-03-09T08:52:59.790482+0000","last_clean":"2024-03-09T08:52:59.790482+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T08:52:59.790482+0000","last_undegraded":"2024-03-09T08:52:59.790482+0000","last_fullsized":"2024-03-09T08:52:59.790482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:48.210921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789770+0000","last_change":"2024-03-09T08:52:29.576677+0000","last_active":"2024-03-09T08:52:59.789770+0000","last_peered":"2024-03-09T08:52:59.789770+0000","last_clean":"2024-03-09T08:52:59.789770+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T08:52:59.789770+0000","last_undegraded":"2024-03-09T08:52:59.789770+0000","last_fullsized":"2024-03-09T08:52:59.789770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:19.026873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.244221+0000","last_change":"2024-03-09T08:53:00.241853+0000","last_active":"2024-03-09T08:53:00.244221+0000","last_peered":"2024-03-09T08:53:00.244221+0000","last_clean":"2024-03-09T08:53:00.244221+0000","last_became_active":"2024-03-09T08:53:00.240844+0000","last_became_peered":"2024-03-09T08:53:00.240844+0000","last_unstale":"2024-03-09T08:53:00.244221+0000","last_undegraded":"2024-03-09T08:53:00.244221+0000","last_fullsized":"2024-03-09T08:53:00.244221+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:18:38.901416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252466+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T08:53:00.252466+0000","last_peered":"2024-03-09T08:53:00.252466+0000","last_clean":"2024-03-09T08:53:00.252466+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T08:53:00.252466+0000","last_undegraded":"2024-03-09T08:53:00.252466+0000","last_fullsized":"2024-03-09T08:53:00.252466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:30.201471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.788952+0000","last_change":"2024-03-09T08:52:28.586702+0000","last_active":"2024-03-09T08:52:59.788952+0000","last_peered":"2024-03-09T08:52:59.788952+0000","last_clean":"2024-03-09T08:52:59.788952+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T08:52:59.788952+0000","last_undegraded":"2024-03-09T08:52:59.788952+0000","last_fullsized":"2024-03-09T08:52:59.788952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:09.201224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792707+0000","last_change":"2024-03-09T08:52:29.578956+0000","last_active":"2024-03-09T08:52:59.792707+0000","last_peered":"2024-03-09T08:52:59.792707+0000","last_clean":"2024-03-09T08:52:59.792707+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T08:52:59.792707+0000","last_undegraded":"2024-03-09T08:52:59.792707+0000","last_fullsized":"2024-03-09T08:52:59.792707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:00:00.884211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251496+0000","last_change":"2024-03-09T08:52:18.234887+0000","last_active":"2024-03-09T08:53:00.251496+0000","last_peered":"2024-03-09T08:53:00.251496+0000","last_clean":"2024-03-09T08:53:00.251496+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T08:53:00.251496+0000","last_undegraded":"2024-03-09T08:53:00.251496+0000","last_fullsized":"2024-03-09T08:53:00.251496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:44.661882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238623+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T08:53:00.238623+0000","last_peered":"2024-03-09T08:53:00.238623+0000","last_clean":"2024-03-09T08:53:00.238623+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T08:53:00.238623+0000","last_undegraded":"2024-03-09T08:53:00.238623+0000","last_fullsized":"2024-03-09T08:53:00.238623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794898+0000","last_change":"2024-03-09T08:52:28.580879+0000","last_active":"2024-03-09T08:52:59.794898+0000","last_peered":"2024-03-09T08:52:59.794898+0000","last_clean":"2024-03-09T08:52:59.794898+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T08:52:59.794898+0000","last_undegraded":"2024-03-09T08:52:59.794898+0000","last_fullsized":"2024-03-09T08:52:59.794898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:19:44.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795587+0000","last_change":"2024-03-09T08:52:29.580024+0000","last_active":"2024-03-09T08:52:59.795587+0000","last_peered":"2024-03-09T08:52:59.795587+0000","last_clean":"2024-03-09T08:52:59.795587+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T08:52:59.795587+0000","last_undegraded":"2024-03-09T08:52:59.795587+0000","last_fullsized":"2024-03-09T08:52:59.795587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:02:13.195642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435983+0000","last_change":"2024-03-09T08:52:18.250999+0000","last_active":"2024-03-09T08:53:00.435983+0000","last_peered":"2024-03-09T08:53:00.435983+0000","last_clean":"2024-03-09T08:53:00.435983+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T08:53:00.435983+0000","last_undegraded":"2024-03-09T08:53:00.435983+0000","last_fullsized":"2024-03-09T08:53:00.435983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:49:54.496603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251763+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T08:53:00.251763+0000","last_peered":"2024-03-09T08:53:00.251763+0000","last_clean":"2024-03-09T08:53:00.251763+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T08:53:00.251763+0000","last_undegraded":"2024-03-09T08:53:00.251763+0000","last_fullsized":"2024-03-09T08:53:00.251763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250224+0000","last_change":"2024-03-09T08:52:28.584457+0000","last_active":"2024-03-09T08:53:00.250224+0000","last_peered":"2024-03-09T08:53:00.250224+0000","last_clean":"2024-03-09T08:53:00.250224+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T08:53:00.250224+0000","last_undegraded":"2024-03-09T08:53:00.250224+0000","last_fullsized":"2024-03-09T08:53:00.250224+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:45.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795381+0000","last_change":"2024-03-09T08:52:29.596486+0000","last_active":"2024-03-09T08:52:59.795381+0000","last_peered":"2024-03-09T08:52:59.795381+0000","last_clean":"2024-03-09T08:52:59.795381+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T08:52:59.795381+0000","last_undegraded":"2024-03-09T08:52:59.795381+0000","last_fullsized":"2024-03-09T08:52:59.795381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:40:16.587782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793658+0000","last_change":"2024-03-09T08:52:18.227365+0000","last_active":"2024-03-09T08:52:59.793658+0000","last_peered":"2024-03-09T08:52:59.793658+0000","last_clean":"2024-03-09T08:52:59.793658+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T08:52:59.793658+0000","last_undegraded":"2024-03-09T08:52:59.793658+0000","last_fullsized":"2024-03-09T08:52:59.793658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:26.357063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435231+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T08:53:00.435231+0000","last_peered":"2024-03-09T08:53:00.435231+0000","last_clean":"2024-03-09T08:53:00.435231+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T08:53:00.435231+0000","last_undegraded":"2024-03-09T08:53:00.435231+0000","last_fullsized":"2024-03-09T08:53:00.435231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251559+0000","last_change":"2024-03-09T08:52:28.578096+0000","last_active":"2024-03-09T08:53:00.251559+0000","last_peered":"2024-03-09T08:53:00.251559+0000","last_clean":"2024-03-09T08:53:00.251559+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T08:53:00.251559+0000","last_undegraded":"2024-03-09T08:53:00.251559+0000","last_fullsized":"2024-03-09T08:53:00.251559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:42.645517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.800380+0000","last_change":"2024-03-09T08:52:29.590834+0000","last_active":"2024-03-09T08:52:59.800380+0000","last_peered":"2024-03-09T08:52:59.800380+0000","last_clean":"2024-03-09T08:52:59.800380+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T08:52:59.800380+0000","last_undegraded":"2024-03-09T08:52:59.800380+0000","last_fullsized":"2024-03-09T08:52:59.800380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:24:06.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.249571+0000","last_change":"2024-03-09T08:52:18.235832+0000","last_active":"2024-03-09T08:53:00.249571+0000","last_peered":"2024-03-09T08:53:00.249571+0000","last_clean":"2024-03-09T08:53:00.249571+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T08:53:00.249571+0000","last_undegraded":"2024-03-09T08:53:00.249571+0000","last_fullsized":"2024-03-09T08:53:00.249571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:38:58.751526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798104+0000","last_change":"2024-03-09T08:52:17.271847+0000","last_active":"2024-03-09T08:52:59.798104+0000","last_peered":"2024-03-09T08:52:59.798104+0000","last_clean":"2024-03-09T08:52:59.798104+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T08:52:59.798104+0000","last_undegraded":"2024-03-09T08:52:59.798104+0000","last_fullsized":"2024-03-09T08:52:59.798104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:27.418674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252245+0000","last_change":"2024-03-09T08:52:28.588636+0000","last_active":"2024-03-09T08:53:00.252245+0000","last_peered":"2024-03-09T08:53:00.252245+0000","last_clean":"2024-03-09T08:53:00.252245+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T08:53:00.252245+0000","last_undegraded":"2024-03-09T08:53:00.252245+0000","last_fullsized":"2024-03-09T08:53:00.252245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:21.601182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.436536+0000","last_change":"2024-03-09T08:52:29.606833+0000","last_active":"2024-03-09T08:53:00.436536+0000","last_peered":"2024-03-09T08:53:00.436536+0000","last_clean":"2024-03-09T08:53:00.436536+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T08:53:00.436536+0000","last_undegraded":"2024-03-09T08:53:00.436536+0000","last_fullsized":"2024-03-09T08:53:00.436536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:40.299681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793719+0000","last_change":"2024-03-09T08:52:18.214396+0000","last_active":"2024-03-09T08:52:59.793719+0000","last_peered":"2024-03-09T08:52:59.793719+0000","last_clean":"2024-03-09T08:52:59.793719+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T08:52:59.793719+0000","last_undegraded":"2024-03-09T08:52:59.793719+0000","last_fullsized":"2024-03-09T08:52:59.793719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:24:10.719351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236741+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T08:53:00.236741+0000","last_peered":"2024-03-09T08:53:00.236741+0000","last_clean":"2024-03-09T08:53:00.236741+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T08:53:00.236741+0000","last_undegraded":"2024-03-09T08:53:00.236741+0000","last_fullsized":"2024-03-09T08:53:00.236741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.797004+0000","last_change":"2024-03-09T08:52:28.578599+0000","last_active":"2024-03-09T08:52:59.797004+0000","last_peered":"2024-03-09T08:52:59.797004+0000","last_clean":"2024-03-09T08:52:59.797004+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T08:52:59.797004+0000","last_undegraded":"2024-03-09T08:52:59.797004+0000","last_fullsized":"2024-03-09T08:52:59.797004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:52:31.328248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238697+0000","last_change":"2024-03-09T08:52:29.591872+0000","last_active":"2024-03-09T08:53:00.238697+0000","last_peered":"2024-03-09T08:53:00.238697+0000","last_clean":"2024-03-09T08:53:00.238697+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T08:53:00.238697+0000","last_undegraded":"2024-03-09T08:53:00.238697+0000","last_fullsized":"2024-03-09T08:53:00.238697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:44:17.870183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791322+0000","last_change":"2024-03-09T08:52:18.235367+0000","last_active":"2024-03-09T08:52:59.791322+0000","last_peered":"2024-03-09T08:52:59.791322+0000","last_clean":"2024-03-09T08:52:59.791322+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T08:52:59.791322+0000","last_undegraded":"2024-03-09T08:52:59.791322+0000","last_fullsized":"2024-03-09T08:52:59.791322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:40.736440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434369+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T08:53:00.434369+0000","last_peered":"2024-03-09T08:53:00.434369+0000","last_clean":"2024-03-09T08:53:00.434369+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T08:53:00.434369+0000","last_undegraded":"2024-03-09T08:53:00.434369+0000","last_fullsized":"2024-03-09T08:53:00.434369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237218+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T08:53:00.237218+0000","last_peered":"2024-03-09T08:53:00.237218+0000","last_clean":"2024-03-09T08:53:00.237218+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T08:53:00.237218+0000","last_undegraded":"2024-03-09T08:53:00.237218+0000","last_fullsized":"2024-03-09T08:53:00.237218+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793707+0000","last_change":"2024-03-09T08:52:28.577973+0000","last_active":"2024-03-09T08:52:59.793707+0000","last_peered":"2024-03-09T08:52:59.793707+0000","last_clean":"2024-03-09T08:52:59.793707+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T08:52:59.793707+0000","last_undegraded":"2024-03-09T08:52:59.793707+0000","last_fullsized":"2024-03-09T08:52:59.793707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:13.291622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798312+0000","last_change":"2024-03-09T08:52:29.599601+0000","last_active":"2024-03-09T08:52:59.798312+0000","last_peered":"2024-03-09T08:52:59.798312+0000","last_clean":"2024-03-09T08:52:59.798312+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T08:52:59.798312+0000","last_undegraded":"2024-03-09T08:52:59.798312+0000","last_fullsized":"2024-03-09T08:52:59.798312+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:11:17.424330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.440926+0000","last_change":"2024-03-09T08:53:00.438594+0000","last_active":"2024-03-09T08:53:00.440926+0000","last_peered":"2024-03-09T08:53:00.440926+0000","last_clean":"2024-03-09T08:53:00.440926+0000","last_became_active":"2024-03-09T08:53:00.437859+0000","last_became_peered":"2024-03-09T08:53:00.437859+0000","last_unstale":"2024-03-09T08:53:00.440926+0000","last_undegraded":"2024-03-09T08:53:00.440926+0000","last_fullsized":"2024-03-09T08:53:00.440926+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:49:53.715132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.799007+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T08:52:59.799007+0000","last_peered":"2024-03-09T08:52:59.799007+0000","last_clean":"2024-03-09T08:52:59.799007+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T08:52:59.799007+0000","last_undegraded":"2024-03-09T08:52:59.799007+0000","last_fullsized":"2024-03-09T08:52:59.799007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:27.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793000+0000","last_change":"2024-03-09T08:52:28.589278+0000","last_active":"2024-03-09T08:52:59.793000+0000","last_peered":"2024-03-09T08:52:59.793000+0000","last_clean":"2024-03-09T08:52:59.793000+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T08:52:59.793000+0000","last_undegraded":"2024-03-09T08:52:59.793000+0000","last_fullsized":"2024-03-09T08:52:59.793000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:16:26.449275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798056+0000","last_change":"2024-03-09T08:52:29.600332+0000","last_active":"2024-03-09T08:52:59.798056+0000","last_peered":"2024-03-09T08:52:59.798056+0000","last_clean":"2024-03-09T08:52:59.798056+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T08:52:59.798056+0000","last_undegraded":"2024-03-09T08:52:59.798056+0000","last_fullsized":"2024-03-09T08:52:59.798056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:36:41.422546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.436028+0000","last_change":"2024-03-09T08:52:18.250433+0000","last_active":"2024-03-09T08:53:00.436028+0000","last_peered":"2024-03-09T08:53:00.436028+0000","last_clean":"2024-03-09T08:53:00.436028+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T08:53:00.436028+0000","last_undegraded":"2024-03-09T08:53:00.436028+0000","last_fullsized":"2024-03-09T08:53:00.436028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:29.742337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252598+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T08:53:00.252598+0000","last_peered":"2024-03-09T08:53:00.252598+0000","last_clean":"2024-03-09T08:53:00.252598+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T08:53:00.252598+0000","last_undegraded":"2024-03-09T08:53:00.252598+0000","last_fullsized":"2024-03-09T08:53:00.252598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:26:35.574903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794661+0000","last_change":"2024-03-09T08:52:28.583181+0000","last_active":"2024-03-09T08:52:59.794661+0000","last_peered":"2024-03-09T08:52:59.794661+0000","last_clean":"2024-03-09T08:52:59.794661+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T08:52:59.794661+0000","last_undegraded":"2024-03-09T08:52:59.794661+0000","last_fullsized":"2024-03-09T08:52:59.794661+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:58:50.061324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236170+0000","last_change":"2024-03-09T08:52:29.590141+0000","last_active":"2024-03-09T08:53:00.236170+0000","last_peered":"2024-03-09T08:53:00.236170+0000","last_clean":"2024-03-09T08:53:00.236170+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T08:53:00.236170+0000","last_undegraded":"2024-03-09T08:53:00.236170+0000","last_fullsized":"2024-03-09T08:53:00.236170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:23.981897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238927+0000","last_change":"2024-03-09T08:52:18.250420+0000","last_active":"2024-03-09T08:53:00.238927+0000","last_peered":"2024-03-09T08:53:00.238927+0000","last_clean":"2024-03-09T08:53:00.238927+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T08:53:00.238927+0000","last_undegraded":"2024-03-09T08:53:00.238927+0000","last_fullsized":"2024-03-09T08:53:00.238927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:48.826066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.800760+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T08:52:59.800760+0000","last_peered":"2024-03-09T08:52:59.800760+0000","last_clean":"2024-03-09T08:52:59.800760+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T08:52:59.800760+0000","last_undegraded":"2024-03-09T08:52:59.800760+0000","last_fullsized":"2024-03-09T08:52:59.800760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:16.138216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.433973+0000","last_change":"2024-03-09T08:52:28.587637+0000","last_active":"2024-03-09T08:53:00.433973+0000","last_peered":"2024-03-09T08:53:00.433973+0000","last_clean":"2024-03-09T08:53:00.433973+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T08:53:00.433973+0000","last_undegraded":"2024-03-09T08:53:00.433973+0000","last_fullsized":"2024-03-09T08:53:00.433973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:31:32.362061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.797119+0000","last_change":"2024-03-09T08:52:29.586180+0000","last_active":"2024-03-09T08:52:59.797119+0000","last_peered":"2024-03-09T08:52:59.797119+0000","last_clean":"2024-03-09T08:52:59.797119+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T08:52:59.797119+0000","last_undegraded":"2024-03-09T08:52:59.797119+0000","last_fullsized":"2024-03-09T08:52:59.797119+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:35:21.522181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238280+0000","last_change":"2024-03-09T08:52:18.246145+0000","last_active":"2024-03-09T08:53:00.238280+0000","last_peered":"2024-03-09T08:53:00.238280+0000","last_clean":"2024-03-09T08:53:00.238280+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T08:53:00.238280+0000","last_undegraded":"2024-03-09T08:53:00.238280+0000","last_fullsized":"2024-03-09T08:53:00.238280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:29:36.213774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795394+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T08:52:59.795394+0000","last_peered":"2024-03-09T08:52:59.795394+0000","last_clean":"2024-03-09T08:52:59.795394+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T08:52:59.795394+0000","last_undegraded":"2024-03-09T08:52:59.795394+0000","last_fullsized":"2024-03-09T08:52:59.795394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:10:34.698235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790421+0000","last_change":"2024-03-09T08:52:28.582308+0000","last_active":"2024-03-09T08:52:59.790421+0000","last_peered":"2024-03-09T08:52:59.790421+0000","last_clean":"2024-03-09T08:52:59.790421+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T08:52:59.790421+0000","last_undegraded":"2024-03-09T08:52:59.790421+0000","last_fullsized":"2024-03-09T08:52:59.790421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:04.918893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792433+0000","last_change":"2024-03-09T08:52:29.610596+0000","last_active":"2024-03-09T08:52:59.792433+0000","last_peered":"2024-03-09T08:52:59.792433+0000","last_clean":"2024-03-09T08:52:59.792433+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T08:52:59.792433+0000","last_undegraded":"2024-03-09T08:52:59.792433+0000","last_fullsized":"2024-03-09T08:52:59.792433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:19:47.313981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237024+0000","last_change":"2024-03-09T08:52:18.248755+0000","last_active":"2024-03-09T08:53:00.237024+0000","last_peered":"2024-03-09T08:53:00.237024+0000","last_clean":"2024-03-09T08:53:00.237024+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T08:53:00.237024+0000","last_undegraded":"2024-03-09T08:53:00.237024+0000","last_fullsized":"2024-03-09T08:53:00.237024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:37.083738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.797789+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T08:52:59.797789+0000","last_peered":"2024-03-09T08:52:59.797789+0000","last_clean":"2024-03-09T08:52:59.797789+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T08:52:59.797789+0000","last_undegraded":"2024-03-09T08:52:59.797789+0000","last_fullsized":"2024-03-09T08:52:59.797789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:31.859924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796777+0000","last_change":"2024-03-09T08:52:28.575975+0000","last_active":"2024-03-09T08:52:59.796777+0000","last_peered":"2024-03-09T08:52:59.796777+0000","last_clean":"2024-03-09T08:52:59.796777+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T08:52:59.796777+0000","last_undegraded":"2024-03-09T08:52:59.796777+0000","last_fullsized":"2024-03-09T08:52:59.796777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:51:24.405316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435825+0000","last_change":"2024-03-09T08:52:29.575752+0000","last_active":"2024-03-09T08:53:00.435825+0000","last_peered":"2024-03-09T08:53:00.435825+0000","last_clean":"2024-03-09T08:53:00.435825+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T08:53:00.435825+0000","last_undegraded":"2024-03-09T08:53:00.435825+0000","last_fullsized":"2024-03-09T08:53:00.435825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:31:51.847871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789742+0000","last_change":"2024-03-09T08:52:18.245073+0000","last_active":"2024-03-09T08:52:59.789742+0000","last_peered":"2024-03-09T08:52:59.789742+0000","last_clean":"2024-03-09T08:52:59.789742+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T08:52:59.789742+0000","last_undegraded":"2024-03-09T08:52:59.789742+0000","last_fullsized":"2024-03-09T08:52:59.789742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:40:15.390735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791951+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T08:52:59.791951+0000","last_peered":"2024-03-09T08:52:59.791951+0000","last_clean":"2024-03-09T08:52:59.791951+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T08:52:59.791951+0000","last_undegraded":"2024-03-09T08:52:59.791951+0000","last_fullsized":"2024-03-09T08:52:59.791951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:07:29.577461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792780+0000","last_change":"2024-03-09T08:52:28.588312+0000","last_active":"2024-03-09T08:52:59.792780+0000","last_peered":"2024-03-09T08:52:59.792780+0000","last_clean":"2024-03-09T08:52:59.792780+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T08:52:59.792780+0000","last_undegraded":"2024-03-09T08:52:59.792780+0000","last_fullsized":"2024-03-09T08:52:59.792780+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:14:52.895618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434697+0000","last_change":"2024-03-09T08:52:29.596647+0000","last_active":"2024-03-09T08:53:00.434697+0000","last_peered":"2024-03-09T08:53:00.434697+0000","last_clean":"2024-03-09T08:53:00.434697+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T08:53:00.434697+0000","last_undegraded":"2024-03-09T08:53:00.434697+0000","last_fullsized":"2024-03-09T08:53:00.434697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:35:28.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237560+0000","last_change":"2024-03-09T08:52:18.248093+0000","last_active":"2024-03-09T08:53:00.237560+0000","last_peered":"2024-03-09T08:53:00.237560+0000","last_clean":"2024-03-09T08:53:00.237560+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T08:53:00.237560+0000","last_undegraded":"2024-03-09T08:53:00.237560+0000","last_fullsized":"2024-03-09T08:53:00.237560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:29.414335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435050+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T08:53:00.435050+0000","last_peered":"2024-03-09T08:53:00.435050+0000","last_clean":"2024-03-09T08:53:00.435050+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T08:53:00.435050+0000","last_undegraded":"2024-03-09T08:53:00.435050+0000","last_fullsized":"2024-03-09T08:53:00.435050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791924+0000","last_change":"2024-03-09T08:52:28.585562+0000","last_active":"2024-03-09T08:52:59.791924+0000","last_peered":"2024-03-09T08:52:59.791924+0000","last_clean":"2024-03-09T08:52:59.791924+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T08:52:59.791924+0000","last_undegraded":"2024-03-09T08:52:59.791924+0000","last_fullsized":"2024-03-09T08:52:59.791924+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:28:50.544795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791730+0000","last_change":"2024-03-09T08:52:29.581330+0000","last_active":"2024-03-09T08:52:59.791730+0000","last_peered":"2024-03-09T08:52:59.791730+0000","last_clean":"2024-03-09T08:52:59.791730+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T08:52:59.791730+0000","last_undegraded":"2024-03-09T08:52:59.791730+0000","last_fullsized":"2024-03-09T08:52:59.791730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:55:38.041794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794362+0000","last_change":"2024-03-09T08:52:18.236146+0000","last_active":"2024-03-09T08:52:59.794362+0000","last_peered":"2024-03-09T08:52:59.794362+0000","last_clean":"2024-03-09T08:52:59.794362+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T08:52:59.794362+0000","last_undegraded":"2024-03-09T08:52:59.794362+0000","last_fullsized":"2024-03-09T08:52:59.794362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:18:51.511926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796997+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T08:52:59.796997+0000","last_peered":"2024-03-09T08:52:59.796997+0000","last_clean":"2024-03-09T08:52:59.796997+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T08:52:59.796997+0000","last_undegraded":"2024-03-09T08:52:59.796997+0000","last_fullsized":"2024-03-09T08:52:59.796997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:22:05.504557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435636+0000","last_change":"2024-03-09T08:52:28.585697+0000","last_active":"2024-03-09T08:53:00.435636+0000","last_peered":"2024-03-09T08:53:00.435636+0000","last_clean":"2024-03-09T08:53:00.435636+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T08:53:00.435636+0000","last_undegraded":"2024-03-09T08:53:00.435636+0000","last_fullsized":"2024-03-09T08:53:00.435636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:50:23.095008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790381+0000","last_change":"2024-03-09T08:52:29.578803+0000","last_active":"2024-03-09T08:52:59.790381+0000","last_peered":"2024-03-09T08:52:59.790381+0000","last_clean":"2024-03-09T08:52:59.790381+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T08:52:59.790381+0000","last_undegraded":"2024-03-09T08:52:59.790381+0000","last_fullsized":"2024-03-09T08:52:59.790381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:22:47.690873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237874+0000","last_change":"2024-03-09T08:52:18.242496+0000","last_active":"2024-03-09T08:53:00.237874+0000","last_peered":"2024-03-09T08:53:00.237874+0000","last_clean":"2024-03-09T08:53:00.237874+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T08:53:00.237874+0000","last_undegraded":"2024-03-09T08:53:00.237874+0000","last_fullsized":"2024-03-09T08:53:00.237874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:35:17.142983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793249+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T08:52:59.793249+0000","last_peered":"2024-03-09T08:52:59.793249+0000","last_clean":"2024-03-09T08:52:59.793249+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T08:52:59.793249+0000","last_undegraded":"2024-03-09T08:52:59.793249+0000","last_fullsized":"2024-03-09T08:52:59.793249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:01.724043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1870,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250057+0000","last_change":"2024-03-09T08:52:28.583685+0000","last_active":"2024-03-09T08:53:00.250057+0000","last_peered":"2024-03-09T08:53:00.250057+0000","last_clean":"2024-03-09T08:53:00.250057+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T08:53:00.250057+0000","last_undegraded":"2024-03-09T08:53:00.250057+0000","last_fullsized":"2024-03-09T08:53:00.250057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:58:00.899620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250427+0000","last_change":"2024-03-09T08:52:29.585912+0000","last_active":"2024-03-09T08:53:00.250427+0000","last_peered":"2024-03-09T08:53:00.250427+0000","last_clean":"2024-03-09T08:53:00.250427+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T08:53:00.250427+0000","last_undegraded":"2024-03-09T08:53:00.250427+0000","last_fullsized":"2024-03-09T08:53:00.250427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:24:47.261191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793994+0000","last_change":"2024-03-09T08:52:18.244654+0000","last_active":"2024-03-09T08:52:59.793994+0000","last_peered":"2024-03-09T08:52:59.793994+0000","last_clean":"2024-03-09T08:52:59.793994+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T08:52:59.793994+0000","last_undegraded":"2024-03-09T08:52:59.793994+0000","last_fullsized":"2024-03-09T08:52:59.793994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:08:58.611725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252327+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T08:53:00.252327+0000","last_peered":"2024-03-09T08:53:00.252327+0000","last_clean":"2024-03-09T08:53:00.252327+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T08:53:00.252327+0000","last_undegraded":"2024-03-09T08:53:00.252327+0000","last_fullsized":"2024-03-09T08:53:00.252327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:41:10.578669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792707+0000","last_change":"2024-03-09T08:52:28.568794+0000","last_active":"2024-03-09T08:52:59.792707+0000","last_peered":"2024-03-09T08:52:59.792707+0000","last_clean":"2024-03-09T08:52:59.792707+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T08:52:59.792707+0000","last_undegraded":"2024-03-09T08:52:59.792707+0000","last_fullsized":"2024-03-09T08:52:59.792707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:44:21.721511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791693+0000","last_change":"2024-03-09T08:52:29.586705+0000","last_active":"2024-03-09T08:52:59.791693+0000","last_peered":"2024-03-09T08:52:59.791693+0000","last_clean":"2024-03-09T08:52:59.791693+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T08:52:59.791693+0000","last_undegraded":"2024-03-09T08:52:59.791693+0000","last_fullsized":"2024-03-09T08:52:59.791693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:26.353156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.255334+0000","last_change":"2024-03-09T08:53:00.255334+0000","last_active":"2024-03-09T08:53:00.255334+0000","last_peered":"2024-03-09T08:53:00.255334+0000","last_clean":"2024-03-09T08:53:00.255334+0000","last_became_active":"2024-03-09T08:53:00.254589+0000","last_became_peered":"2024-03-09T08:53:00.254589+0000","last_unstale":"2024-03-09T08:53:00.255334+0000","last_undegraded":"2024-03-09T08:53:00.255334+0000","last_fullsized":"2024-03-09T08:53:00.255334+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:30:00.005142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435997+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T08:53:00.435997+0000","last_peered":"2024-03-09T08:53:00.435997+0000","last_clean":"2024-03-09T08:53:00.435997+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T08:53:00.435997+0000","last_undegraded":"2024-03-09T08:53:00.435997+0000","last_fullsized":"2024-03-09T08:53:00.435997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789400+0000","last_change":"2024-03-09T08:52:28.583697+0000","last_active":"2024-03-09T08:52:59.789400+0000","last_peered":"2024-03-09T08:52:59.789400+0000","last_clean":"2024-03-09T08:52:59.789400+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T08:52:59.789400+0000","last_undegraded":"2024-03-09T08:52:59.789400+0000","last_fullsized":"2024-03-09T08:52:59.789400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:41.552156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236217+0000","last_change":"2024-03-09T08:52:29.590854+0000","last_active":"2024-03-09T08:53:00.236217+0000","last_peered":"2024-03-09T08:53:00.236217+0000","last_clean":"2024-03-09T08:53:00.236217+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T08:53:00.236217+0000","last_undegraded":"2024-03-09T08:53:00.236217+0000","last_fullsized":"2024-03-09T08:53:00.236217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:09.568826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435804+0000","last_change":"2024-03-09T08:52:18.251648+0000","last_active":"2024-03-09T08:53:00.435804+0000","last_peered":"2024-03-09T08:53:00.435804+0000","last_clean":"2024-03-09T08:53:00.435804+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T08:53:00.435804+0000","last_undegraded":"2024-03-09T08:53:00.435804+0000","last_fullsized":"2024-03-09T08:53:00.435804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:54:45.721673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795610+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T08:52:59.795610+0000","last_peered":"2024-03-09T08:52:59.795610+0000","last_clean":"2024-03-09T08:52:59.795610+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T08:52:59.795610+0000","last_undegraded":"2024-03-09T08:52:59.795610+0000","last_fullsized":"2024-03-09T08:52:59.795610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:31.731626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789940+0000","last_change":"2024-03-09T08:52:28.586606+0000","last_active":"2024-03-09T08:52:59.789940+0000","last_peered":"2024-03-09T08:52:59.789940+0000","last_clean":"2024-03-09T08:52:59.789940+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T08:52:59.789940+0000","last_undegraded":"2024-03-09T08:52:59.789940+0000","last_fullsized":"2024-03-09T08:52:59.789940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:17:40.902403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795143+0000","last_change":"2024-03-09T08:52:29.584512+0000","last_active":"2024-03-09T08:52:59.795143+0000","last_peered":"2024-03-09T08:52:59.795143+0000","last_clean":"2024-03-09T08:52:59.795143+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T08:52:59.795143+0000","last_undegraded":"2024-03-09T08:52:59.795143+0000","last_fullsized":"2024-03-09T08:52:59.795143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:39:20.330443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796012+0000","last_change":"2024-03-09T08:52:28.586201+0000","last_active":"2024-03-09T08:52:59.796012+0000","last_peered":"2024-03-09T08:52:59.796012+0000","last_clean":"2024-03-09T08:52:59.796012+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T08:52:59.796012+0000","last_undegraded":"2024-03-09T08:52:59.796012+0000","last_fullsized":"2024-03-09T08:52:59.796012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:28:03.471646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.249633+0000","last_change":"2024-03-09T08:52:18.241156+0000","last_active":"2024-03-09T08:53:00.249633+0000","last_peered":"2024-03-09T08:53:00.249633+0000","last_clean":"2024-03-09T08:53:00.249633+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T08:53:00.249633+0000","last_undegraded":"2024-03-09T08:53:00.249633+0000","last_fullsized":"2024-03-09T08:53:00.249633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:55.323770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434627+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T08:53:00.434627+0000","last_peered":"2024-03-09T08:53:00.434627+0000","last_clean":"2024-03-09T08:53:00.434627+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T08:53:00.434627+0000","last_undegraded":"2024-03-09T08:53:00.434627+0000","last_fullsized":"2024-03-09T08:53:00.434627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795848+0000","last_change":"2024-03-09T08:52:29.591085+0000","last_active":"2024-03-09T08:52:59.795848+0000","last_peered":"2024-03-09T08:52:59.795848+0000","last_clean":"2024-03-09T08:52:59.795848+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T08:52:59.795848+0000","last_undegraded":"2024-03-09T08:52:59.795848+0000","last_fullsized":"2024-03-09T08:52:59.795848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:40:27.878922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795316+0000","last_change":"2024-03-09T08:52:28.585631+0000","last_active":"2024-03-09T08:52:59.795316+0000","last_peered":"2024-03-09T08:52:59.795316+0000","last_clean":"2024-03-09T08:52:59.795316+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T08:52:59.795316+0000","last_undegraded":"2024-03-09T08:52:59.795316+0000","last_fullsized":"2024-03-09T08:52:59.795316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:06:03.569759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789727+0000","last_change":"2024-03-09T08:52:18.234400+0000","last_active":"2024-03-09T08:52:59.789727+0000","last_peered":"2024-03-09T08:52:59.789727+0000","last_clean":"2024-03-09T08:52:59.789727+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T08:52:59.789727+0000","last_undegraded":"2024-03-09T08:52:59.789727+0000","last_fullsized":"2024-03-09T08:52:59.789727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:42:14.057367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790714+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T08:52:59.790714+0000","last_peered":"2024-03-09T08:52:59.790714+0000","last_clean":"2024-03-09T08:52:59.790714+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T08:52:59.790714+0000","last_undegraded":"2024-03-09T08:52:59.790714+0000","last_fullsized":"2024-03-09T08:52:59.790714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251596+0000","last_change":"2024-03-09T08:52:29.588928+0000","last_active":"2024-03-09T08:53:00.251596+0000","last_peered":"2024-03-09T08:53:00.251596+0000","last_clean":"2024-03-09T08:53:00.251596+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T08:53:00.251596+0000","last_undegraded":"2024-03-09T08:53:00.251596+0000","last_fullsized":"2024-03-09T08:53:00.251596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:32:27.061041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793333+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T08:52:59.793333+0000","last_peered":"2024-03-09T08:52:59.793333+0000","last_clean":"2024-03-09T08:52:59.793333+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T08:52:59.793333+0000","last_undegraded":"2024-03-09T08:52:59.793333+0000","last_fullsized":"2024-03-09T08:52:59.793333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796845+0000","last_change":"2024-03-09T08:52:18.231141+0000","last_active":"2024-03-09T08:52:59.796845+0000","last_peered":"2024-03-09T08:52:59.796845+0000","last_clean":"2024-03-09T08:52:59.796845+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T08:52:59.796845+0000","last_undegraded":"2024-03-09T08:52:59.796845+0000","last_fullsized":"2024-03-09T08:52:59.796845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:25:00.471222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798738+0000","last_change":"2024-03-09T08:52:28.579315+0000","last_active":"2024-03-09T08:52:59.798738+0000","last_peered":"2024-03-09T08:52:59.798738+0000","last_clean":"2024-03-09T08:52:59.798738+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T08:52:59.798738+0000","last_undegraded":"2024-03-09T08:52:59.798738+0000","last_fullsized":"2024-03-09T08:52:59.798738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:57:17.301677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791987+0000","last_change":"2024-03-09T08:52:29.585154+0000","last_active":"2024-03-09T08:52:59.791987+0000","last_peered":"2024-03-09T08:52:59.791987+0000","last_clean":"2024-03-09T08:52:59.791987+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T08:52:59.791987+0000","last_undegraded":"2024-03-09T08:52:59.791987+0000","last_fullsized":"2024-03-09T08:52:59.791987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:43:35.679108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791033+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T08:52:59.791033+0000","last_peered":"2024-03-09T08:52:59.791033+0000","last_clean":"2024-03-09T08:52:59.791033+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T08:52:59.791033+0000","last_undegraded":"2024-03-09T08:52:59.791033+0000","last_fullsized":"2024-03-09T08:52:59.791033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792089+0000","last_change":"2024-03-09T08:52:18.229533+0000","last_active":"2024-03-09T08:52:59.792089+0000","last_peered":"2024-03-09T08:52:59.792089+0000","last_clean":"2024-03-09T08:52:59.792089+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T08:52:59.792089+0000","last_undegraded":"2024-03-09T08:52:59.792089+0000","last_fullsized":"2024-03-09T08:52:59.792089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:17.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791797+0000","last_change":"2024-03-09T08:52:28.585877+0000","last_active":"2024-03-09T08:52:59.791797+0000","last_peered":"2024-03-09T08:52:59.791797+0000","last_clean":"2024-03-09T08:52:59.791797+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T08:52:59.791797+0000","last_undegraded":"2024-03-09T08:52:59.791797+0000","last_fullsized":"2024-03-09T08:52:59.791797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:30:20.036385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792761+0000","last_change":"2024-03-09T08:52:29.592253+0000","last_active":"2024-03-09T08:52:59.792761+0000","last_peered":"2024-03-09T08:52:59.792761+0000","last_clean":"2024-03-09T08:52:59.792761+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T08:52:59.792761+0000","last_undegraded":"2024-03-09T08:52:59.792761+0000","last_fullsized":"2024-03-09T08:52:59.792761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:50:33.525541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789659+0000","last_change":"2024-03-09T08:52:18.237869+0000","last_active":"2024-03-09T08:52:59.789659+0000","last_peered":"2024-03-09T08:52:59.789659+0000","last_clean":"2024-03-09T08:52:59.789659+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T08:52:59.789659+0000","last_undegraded":"2024-03-09T08:52:59.789659+0000","last_fullsized":"2024-03-09T08:52:59.789659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:01.571866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251959+0000","last_change":"2024-03-09T08:52:29.601300+0000","last_active":"2024-03-09T08:53:00.251959+0000","last_peered":"2024-03-09T08:53:00.251959+0000","last_clean":"2024-03-09T08:53:00.251959+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T08:53:00.251959+0000","last_undegraded":"2024-03-09T08:53:00.251959+0000","last_fullsized":"2024-03-09T08:53:00.251959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:46:04.870574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.797859+0000","last_change":"2024-03-09T08:52:18.226980+0000","last_active":"2024-03-09T08:52:59.797859+0000","last_peered":"2024-03-09T08:52:59.797859+0000","last_clean":"2024-03-09T08:52:59.797859+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T08:52:59.797859+0000","last_undegraded":"2024-03-09T08:52:59.797859+0000","last_fullsized":"2024-03-09T08:52:59.797859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:28.312576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434867+0000","last_change":"2024-03-09T08:52:29.575612+0000","last_active":"2024-03-09T08:53:00.434867+0000","last_peered":"2024-03-09T08:53:00.434867+0000","last_clean":"2024-03-09T08:53:00.434867+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T08:53:00.434867+0000","last_undegraded":"2024-03-09T08:53:00.434867+0000","last_fullsized":"2024-03-09T08:53:00.434867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:58.852983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434982+0000","last_change":"2024-03-09T08:52:18.241628+0000","last_active":"2024-03-09T08:53:00.434982+0000","last_peered":"2024-03-09T08:53:00.434982+0000","last_clean":"2024-03-09T08:53:00.434982+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T08:53:00.434982+0000","last_undegraded":"2024-03-09T08:53:00.434982+0000","last_fullsized":"2024-03-09T08:53:00.434982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:07:58.570641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252835+0000","last_change":"2024-03-09T08:52:29.588660+0000","last_active":"2024-03-09T08:53:00.252835+0000","last_peered":"2024-03-09T08:53:00.252835+0000","last_clean":"2024-03-09T08:53:00.252835+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T08:53:00.252835+0000","last_undegraded":"2024-03-09T08:53:00.252835+0000","last_fullsized":"2024-03-09T08:53:00.252835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:14.305320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796333+0000","last_change":"2024-03-09T08:52:18.242257+0000","last_active":"2024-03-09T08:52:59.796333+0000","last_peered":"2024-03-09T08:52:59.796333+0000","last_clean":"2024-03-09T08:52:59.796333+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T08:52:59.796333+0000","last_undegraded":"2024-03-09T08:52:59.796333+0000","last_fullsized":"2024-03-09T08:52:59.796333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:40:00.854066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236632+0000","last_change":"2024-03-09T08:52:29.595057+0000","last_active":"2024-03-09T08:53:00.236632+0000","last_peered":"2024-03-09T08:53:00.236632+0000","last_clean":"2024-03-09T08:53:00.236632+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T08:53:00.236632+0000","last_undegraded":"2024-03-09T08:53:00.236632+0000","last_fullsized":"2024-03-09T08:53:00.236632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:33:37.474266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792473+0000","last_change":"2024-03-09T08:52:18.239951+0000","last_active":"2024-03-09T08:52:59.792473+0000","last_peered":"2024-03-09T08:52:59.792473+0000","last_clean":"2024-03-09T08:52:59.792473+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T08:52:59.792473+0000","last_undegraded":"2024-03-09T08:52:59.792473+0000","last_fullsized":"2024-03-09T08:52:59.792473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:51.102540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792106+0000","last_change":"2024-03-09T08:52:29.578993+0000","last_active":"2024-03-09T08:52:59.792106+0000","last_peered":"2024-03-09T08:52:59.792106+0000","last_clean":"2024-03-09T08:52:59.792106+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T08:52:59.792106+0000","last_undegraded":"2024-03-09T08:52:59.792106+0000","last_fullsized":"2024-03-09T08:52:59.792106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:50:53.692060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252030+0000","last_change":"2024-03-09T08:52:18.241038+0000","last_active":"2024-03-09T08:53:00.252030+0000","last_peered":"2024-03-09T08:53:00.252030+0000","last_clean":"2024-03-09T08:53:00.252030+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T08:53:00.252030+0000","last_undegraded":"2024-03-09T08:53:00.252030+0000","last_fullsized":"2024-03-09T08:53:00.252030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:02.169113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236170+0000","last_change":"2024-03-09T08:52:29.583869+0000","last_active":"2024-03-09T08:53:00.236170+0000","last_peered":"2024-03-09T08:53:00.236170+0000","last_clean":"2024-03-09T08:53:00.236170+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T08:53:00.236170+0000","last_undegraded":"2024-03-09T08:53:00.236170+0000","last_fullsized":"2024-03-09T08:53:00.236170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:29:47.636741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794839+0000","last_change":"2024-03-09T08:52:18.247145+0000","last_active":"2024-03-09T08:52:59.794839+0000","last_peered":"2024-03-09T08:52:59.794839+0000","last_clean":"2024-03-09T08:52:59.794839+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T08:52:59.794839+0000","last_undegraded":"2024-03-09T08:52:59.794839+0000","last_fullsized":"2024-03-09T08:52:59.794839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:46:24.111224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791477+0000","last_change":"2024-03-09T08:52:29.587358+0000","last_active":"2024-03-09T08:52:59.791477+0000","last_peered":"2024-03-09T08:52:59.791477+0000","last_clean":"2024-03-09T08:52:59.791477+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T08:52:59.791477+0000","last_undegraded":"2024-03-09T08:52:59.791477+0000","last_fullsized":"2024-03-09T08:52:59.791477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:46.836801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792891+0000","last_change":"2024-03-09T08:52:18.247784+0000","last_active":"2024-03-09T08:52:59.792891+0000","last_peered":"2024-03-09T08:52:59.792891+0000","last_clean":"2024-03-09T08:52:59.792891+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T08:52:59.792891+0000","last_undegraded":"2024-03-09T08:52:59.792891+0000","last_fullsized":"2024-03-09T08:52:59.792891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:46:59.008220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792459+0000","last_change":"2024-03-09T08:52:29.589458+0000","last_active":"2024-03-09T08:52:59.792459+0000","last_peered":"2024-03-09T08:52:59.792459+0000","last_clean":"2024-03-09T08:52:59.792459+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T08:52:59.792459+0000","last_undegraded":"2024-03-09T08:52:59.792459+0000","last_fullsized":"2024-03-09T08:52:59.792459+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:19:35.222184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434277+0000","last_change":"2024-03-09T08:52:18.239943+0000","last_active":"2024-03-09T08:53:00.434277+0000","last_peered":"2024-03-09T08:53:00.434277+0000","last_clean":"2024-03-09T08:53:00.434277+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T08:53:00.434277+0000","last_undegraded":"2024-03-09T08:53:00.434277+0000","last_fullsized":"2024-03-09T08:53:00.434277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:18.727030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.239021+0000","last_change":"2024-03-09T08:52:29.597147+0000","last_active":"2024-03-09T08:53:00.239021+0000","last_peered":"2024-03-09T08:53:00.239021+0000","last_clean":"2024-03-09T08:53:00.239021+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T08:53:00.239021+0000","last_undegraded":"2024-03-09T08:53:00.239021+0000","last_fullsized":"2024-03-09T08:53:00.239021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:44:47.144911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790530+0000","last_change":"2024-03-09T08:52:18.236642+0000","last_active":"2024-03-09T08:52:59.790530+0000","last_peered":"2024-03-09T08:52:59.790530+0000","last_clean":"2024-03-09T08:52:59.790530+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T08:52:59.790530+0000","last_undegraded":"2024-03-09T08:52:59.790530+0000","last_fullsized":"2024-03-09T08:52:59.790530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:51:46.825153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798693+0000","last_change":"2024-03-09T08:52:29.590810+0000","last_active":"2024-03-09T08:52:59.798693+0000","last_peered":"2024-03-09T08:52:59.798693+0000","last_clean":"2024-03-09T08:52:59.798693+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T08:52:59.798693+0000","last_undegraded":"2024-03-09T08:52:59.798693+0000","last_fullsized":"2024-03-09T08:52:59.798693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:21.534580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434797+0000","last_change":"2024-03-09T08:52:18.236855+0000","last_active":"2024-03-09T08:53:00.434797+0000","last_peered":"2024-03-09T08:53:00.434797+0000","last_clean":"2024-03-09T08:53:00.434797+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T08:53:00.434797+0000","last_undegraded":"2024-03-09T08:53:00.434797+0000","last_fullsized":"2024-03-09T08:53:00.434797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:21.941242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791664+0000","last_change":"2024-03-09T08:52:29.602209+0000","last_active":"2024-03-09T08:52:59.791664+0000","last_peered":"2024-03-09T08:52:59.791664+0000","last_clean":"2024-03-09T08:52:59.791664+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T08:52:59.791664+0000","last_undegraded":"2024-03-09T08:52:59.791664+0000","last_fullsized":"2024-03-09T08:52:59.791664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:11:33.349464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795388+0000","last_change":"2024-03-09T08:52:18.238373+0000","last_active":"2024-03-09T08:52:59.795388+0000","last_peered":"2024-03-09T08:52:59.795388+0000","last_clean":"2024-03-09T08:52:59.795388+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T08:52:59.795388+0000","last_undegraded":"2024-03-09T08:52:59.795388+0000","last_fullsized":"2024-03-09T08:52:59.795388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:05:31.480335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791332+0000","last_change":"2024-03-09T08:52:29.578120+0000","last_active":"2024-03-09T08:52:59.791332+0000","last_peered":"2024-03-09T08:52:59.791332+0000","last_clean":"2024-03-09T08:52:59.791332+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T08:52:59.791332+0000","last_undegraded":"2024-03-09T08:52:59.791332+0000","last_fullsized":"2024-03-09T08:52:59.791332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:13:27.623995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793083+0000","last_change":"2024-03-09T08:52:18.235565+0000","last_active":"2024-03-09T08:52:59.793083+0000","last_peered":"2024-03-09T08:52:59.793083+0000","last_clean":"2024-03-09T08:52:59.793083+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T08:52:59.793083+0000","last_undegraded":"2024-03-09T08:52:59.793083+0000","last_fullsized":"2024-03-09T08:52:59.793083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:52:14.994539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791813+0000","last_change":"2024-03-09T08:52:29.579668+0000","last_active":"2024-03-09T08:52:59.791813+0000","last_peered":"2024-03-09T08:52:59.791813+0000","last_clean":"2024-03-09T08:52:59.791813+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T08:52:59.791813+0000","last_undegraded":"2024-03-09T08:52:59.791813+0000","last_fullsized":"2024-03-09T08:52:59.791813+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:12:15.272875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238341+0000","last_change":"2024-03-09T08:52:18.239801+0000","last_active":"2024-03-09T08:53:00.238341+0000","last_peered":"2024-03-09T08:53:00.238341+0000","last_clean":"2024-03-09T08:53:00.238341+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T08:53:00.238341+0000","last_undegraded":"2024-03-09T08:53:00.238341+0000","last_fullsized":"2024-03-09T08:53:00.238341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:38:06.836990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.433665+0000","last_change":"2024-03-09T08:52:29.586986+0000","last_active":"2024-03-09T08:53:00.433665+0000","last_peered":"2024-03-09T08:53:00.433665+0000","last_clean":"2024-03-09T08:53:00.433665+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T08:53:00.433665+0000","last_undegraded":"2024-03-09T08:53:00.433665+0000","last_fullsized":"2024-03-09T08:53:00.433665+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:37.382414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.801226+0000","last_change":"2024-03-09T08:52:18.233582+0000","last_active":"2024-03-09T08:52:59.801226+0000","last_peered":"2024-03-09T08:52:59.801226+0000","last_clean":"2024-03-09T08:52:59.801226+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T08:52:59.801226+0000","last_undegraded":"2024-03-09T08:52:59.801226+0000","last_fullsized":"2024-03-09T08:52:59.801226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:49.313616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793385+0000","last_change":"2024-03-09T08:52:29.586673+0000","last_active":"2024-03-09T08:52:59.793385+0000","last_peered":"2024-03-09T08:52:59.793385+0000","last_clean":"2024-03-09T08:52:59.793385+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T08:52:59.793385+0000","last_undegraded":"2024-03-09T08:52:59.793385+0000","last_fullsized":"2024-03-09T08:52:59.793385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:23:49.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.243414+0000","last_change":"2024-03-09T08:53:00.241216+0000","last_active":"2024-03-09T08:53:00.243414+0000","last_peered":"2024-03-09T08:53:00.243414+0000","last_clean":"2024-03-09T08:53:00.243414+0000","last_became_active":"2024-03-09T08:53:00.240319+0000","last_became_peered":"2024-03-09T08:53:00.240319+0000","last_unstale":"2024-03-09T08:53:00.243414+0000","last_undegraded":"2024-03-09T08:53:00.243414+0000","last_fullsized":"2024-03-09T08:53:00.243414+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:32:34.111501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251554+0000","last_change":"2024-03-09T08:52:29.592702+0000","last_active":"2024-03-09T08:53:00.251554+0000","last_peered":"2024-03-09T08:53:00.251554+0000","last_clean":"2024-03-09T08:53:00.251554+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T08:53:00.251554+0000","last_undegraded":"2024-03-09T08:53:00.251554+0000","last_fullsized":"2024-03-09T08:53:00.251554+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:39.344726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434012+0000","last_change":"2024-03-09T08:52:18.230289+0000","last_active":"2024-03-09T08:53:00.434012+0000","last_peered":"2024-03-09T08:53:00.434012+0000","last_clean":"2024-03-09T08:53:00.434012+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T08:53:00.434012+0000","last_undegraded":"2024-03-09T08:53:00.434012+0000","last_fullsized":"2024-03-09T08:53:00.434012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:34.507924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794831+0000","last_change":"2024-03-09T08:52:29.608191+0000","last_active":"2024-03-09T08:52:59.794831+0000","last_peered":"2024-03-09T08:52:59.794831+0000","last_clean":"2024-03-09T08:52:59.794831+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T08:52:59.794831+0000","last_undegraded":"2024-03-09T08:52:59.794831+0000","last_fullsized":"2024-03-09T08:52:59.794831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:26:32.282196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.249757+0000","last_change":"2024-03-09T08:52:18.236376+0000","last_active":"2024-03-09T08:53:00.249757+0000","last_peered":"2024-03-09T08:53:00.249757+0000","last_clean":"2024-03-09T08:53:00.249757+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T08:53:00.249757+0000","last_undegraded":"2024-03-09T08:53:00.249757+0000","last_fullsized":"2024-03-09T08:53:00.249757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:34:04.412643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434645+0000","last_change":"2024-03-09T08:52:29.573685+0000","last_active":"2024-03-09T08:53:00.434645+0000","last_peered":"2024-03-09T08:53:00.434645+0000","last_clean":"2024-03-09T08:53:00.434645+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T08:53:00.434645+0000","last_undegraded":"2024-03-09T08:53:00.434645+0000","last_fullsized":"2024-03-09T08:53:00.434645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:41:41.839063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794192+0000","last_change":"2024-03-09T08:52:18.231316+0000","last_active":"2024-03-09T08:52:59.794192+0000","last_peered":"2024-03-09T08:52:59.794192+0000","last_clean":"2024-03-09T08:52:59.794192+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T08:52:59.794192+0000","last_undegraded":"2024-03-09T08:52:59.794192+0000","last_fullsized":"2024-03-09T08:52:59.794192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:25:25.715084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237839+0000","last_change":"2024-03-09T08:52:29.603272+0000","last_active":"2024-03-09T08:53:00.237839+0000","last_peered":"2024-03-09T08:53:00.237839+0000","last_clean":"2024-03-09T08:53:00.237839+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T08:53:00.237839+0000","last_undegraded":"2024-03-09T08:53:00.237839+0000","last_fullsized":"2024-03-09T08:53:00.237839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:07:58.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790491+0000","last_change":"2024-03-09T08:52:18.243645+0000","last_active":"2024-03-09T08:52:59.790491+0000","last_peered":"2024-03-09T08:52:59.790491+0000","last_clean":"2024-03-09T08:52:59.790491+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T08:52:59.790491+0000","last_undegraded":"2024-03-09T08:52:59.790491+0000","last_fullsized":"2024-03-09T08:52:59.790491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:19.501675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791144+0000","last_change":"2024-03-09T08:52:29.587462+0000","last_active":"2024-03-09T08:52:59.791144+0000","last_peered":"2024-03-09T08:52:59.791144+0000","last_clean":"2024-03-09T08:52:59.791144+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T08:52:59.791144+0000","last_undegraded":"2024-03-09T08:52:59.791144+0000","last_fullsized":"2024-03-09T08:52:59.791144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:51.652225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793895+0000","last_change":"2024-03-09T08:52:18.233451+0000","last_active":"2024-03-09T08:52:59.793895+0000","last_peered":"2024-03-09T08:52:59.793895+0000","last_clean":"2024-03-09T08:52:59.793895+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T08:52:59.793895+0000","last_undegraded":"2024-03-09T08:52:59.793895+0000","last_fullsized":"2024-03-09T08:52:59.793895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:56:11.270081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798239+0000","last_change":"2024-03-09T08:52:29.599909+0000","last_active":"2024-03-09T08:52:59.798239+0000","last_peered":"2024-03-09T08:52:59.798239+0000","last_clean":"2024-03-09T08:52:59.798239+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T08:52:59.798239+0000","last_undegraded":"2024-03-09T08:52:59.798239+0000","last_fullsized":"2024-03-09T08:52:59.798239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:42.596067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794296+0000","last_change":"2024-03-09T08:52:18.234158+0000","last_active":"2024-03-09T08:52:59.794296+0000","last_peered":"2024-03-09T08:52:59.794296+0000","last_clean":"2024-03-09T08:52:59.794296+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T08:52:59.794296+0000","last_undegraded":"2024-03-09T08:52:59.794296+0000","last_fullsized":"2024-03-09T08:52:59.794296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:59:33.423558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237041+0000","last_change":"2024-03-09T08:52:29.597518+0000","last_active":"2024-03-09T08:53:00.237041+0000","last_peered":"2024-03-09T08:53:00.237041+0000","last_clean":"2024-03-09T08:53:00.237041+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T08:53:00.237041+0000","last_undegraded":"2024-03-09T08:53:00.237041+0000","last_fullsized":"2024-03-09T08:53:00.237041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:32:32.452993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794838+0000","last_change":"2024-03-09T08:52:18.234648+0000","last_active":"2024-03-09T08:52:59.794838+0000","last_peered":"2024-03-09T08:52:59.794838+0000","last_clean":"2024-03-09T08:52:59.794838+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T08:52:59.794838+0000","last_undegraded":"2024-03-09T08:52:59.794838+0000","last_fullsized":"2024-03-09T08:52:59.794838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:03.927280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434105+0000","last_change":"2024-03-09T08:52:29.580608+0000","last_active":"2024-03-09T08:53:00.434105+0000","last_peered":"2024-03-09T08:53:00.434105+0000","last_clean":"2024-03-09T08:53:00.434105+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T08:53:00.434105+0000","last_undegraded":"2024-03-09T08:53:00.434105+0000","last_fullsized":"2024-03-09T08:53:00.434105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:00:37.184351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238778+0000","last_change":"2024-03-09T08:52:18.253038+0000","last_active":"2024-03-09T08:53:00.238778+0000","last_peered":"2024-03-09T08:53:00.238778+0000","last_clean":"2024-03-09T08:53:00.238778+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T08:53:00.238778+0000","last_undegraded":"2024-03-09T08:53:00.238778+0000","last_fullsized":"2024-03-09T08:53:00.238778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:26:39.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792413+0000","last_change":"2024-03-09T08:52:29.578549+0000","last_active":"2024-03-09T08:52:59.792413+0000","last_peered":"2024-03-09T08:52:59.792413+0000","last_clean":"2024-03-09T08:52:59.792413+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T08:52:59.792413+0000","last_undegraded":"2024-03-09T08:52:59.792413+0000","last_fullsized":"2024-03-09T08:52:59.792413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:42.775199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789574+0000","last_change":"2024-03-09T08:52:18.240975+0000","last_active":"2024-03-09T08:52:59.789574+0000","last_peered":"2024-03-09T08:52:59.789574+0000","last_clean":"2024-03-09T08:52:59.789574+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T08:52:59.789574+0000","last_undegraded":"2024-03-09T08:52:59.789574+0000","last_fullsized":"2024-03-09T08:52:59.789574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:30:09.363339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794998+0000","last_change":"2024-03-09T08:52:29.584159+0000","last_active":"2024-03-09T08:52:59.794998+0000","last_peered":"2024-03-09T08:52:59.794998+0000","last_clean":"2024-03-09T08:52:59.794998+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T08:52:59.794998+0000","last_undegraded":"2024-03-09T08:52:59.794998+0000","last_fullsized":"2024-03-09T08:52:59.794998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:01:41.661806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236628+0000","last_change":"2024-03-09T08:52:18.249901+0000","last_active":"2024-03-09T08:53:00.236628+0000","last_peered":"2024-03-09T08:53:00.236628+0000","last_clean":"2024-03-09T08:53:00.236628+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T08:53:00.236628+0000","last_undegraded":"2024-03-09T08:53:00.236628+0000","last_fullsized":"2024-03-09T08:53:00.236628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:09.947401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250862+0000","last_change":"2024-03-09T08:52:29.595184+0000","last_active":"2024-03-09T08:53:00.250862+0000","last_peered":"2024-03-09T08:53:00.250862+0000","last_clean":"2024-03-09T08:53:00.250862+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T08:53:00.250862+0000","last_undegraded":"2024-03-09T08:53:00.250862+0000","last_fullsized":"2024-03-09T08:53:00.250862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:34:27.662636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434708+0000","last_change":"2024-03-09T08:52:18.233561+0000","last_active":"2024-03-09T08:53:00.434708+0000","last_peered":"2024-03-09T08:53:00.434708+0000","last_clean":"2024-03-09T08:53:00.434708+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T08:53:00.434708+0000","last_undegraded":"2024-03-09T08:53:00.434708+0000","last_fullsized":"2024-03-09T08:53:00.434708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:15:54.027155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435347+0000","last_change":"2024-03-09T08:52:29.595771+0000","last_active":"2024-03-09T08:53:00.435347+0000","last_peered":"2024-03-09T08:53:00.435347+0000","last_clean":"2024-03-09T08:53:00.435347+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T08:53:00.435347+0000","last_undegraded":"2024-03-09T08:53:00.435347+0000","last_fullsized":"2024-03-09T08:53:00.435347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:23:48.898071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237460+0000","last_change":"2024-03-09T08:52:18.242311+0000","last_active":"2024-03-09T08:53:00.237460+0000","last_peered":"2024-03-09T08:53:00.237460+0000","last_clean":"2024-03-09T08:53:00.237460+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T08:53:00.237460+0000","last_undegraded":"2024-03-09T08:53:00.237460+0000","last_fullsized":"2024-03-09T08:53:00.237460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:48:15.559438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790367+0000","last_change":"2024-03-09T08:52:29.576792+0000","last_active":"2024-03-09T08:52:59.790367+0000","last_peered":"2024-03-09T08:52:59.790367+0000","last_clean":"2024-03-09T08:52:59.790367+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T08:52:59.790367+0000","last_undegraded":"2024-03-09T08:52:59.790367+0000","last_fullsized":"2024-03-09T08:52:59.790367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:21.364814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435606+0000","last_change":"2024-03-09T08:52:18.248341+0000","last_active":"2024-03-09T08:53:00.435606+0000","last_peered":"2024-03-09T08:53:00.435606+0000","last_clean":"2024-03-09T08:53:00.435606+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T08:53:00.435606+0000","last_undegraded":"2024-03-09T08:53:00.435606+0000","last_fullsized":"2024-03-09T08:53:00.435606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:18:02.716066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798091+0000","last_change":"2024-03-09T08:52:29.606330+0000","last_active":"2024-03-09T08:52:59.798091+0000","last_peered":"2024-03-09T08:52:59.798091+0000","last_clean":"2024-03-09T08:52:59.798091+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T08:52:59.798091+0000","last_undegraded":"2024-03-09T08:52:59.798091+0000","last_fullsized":"2024-03-09T08:52:59.798091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:44:50.347124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795661+0000","last_change":"2024-03-09T08:52:18.222968+0000","last_active":"2024-03-09T08:52:59.795661+0000","last_peered":"2024-03-09T08:52:59.795661+0000","last_clean":"2024-03-09T08:52:59.795661+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T08:52:59.795661+0000","last_undegraded":"2024-03-09T08:52:59.795661+0000","last_fullsized":"2024-03-09T08:52:59.795661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:52:29.523963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236013+0000","last_change":"2024-03-09T08:52:29.591813+0000","last_active":"2024-03-09T08:53:00.236013+0000","last_peered":"2024-03-09T08:53:00.236013+0000","last_clean":"2024-03-09T08:53:00.236013+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T08:53:00.236013+0000","last_undegraded":"2024-03-09T08:53:00.236013+0000","last_fullsized":"2024-03-09T08:53:00.236013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:40:31.568253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251999+0000","last_change":"2024-03-09T08:52:18.225812+0000","last_active":"2024-03-09T08:53:00.251999+0000","last_peered":"2024-03-09T08:53:00.251999+0000","last_clean":"2024-03-09T08:53:00.251999+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T08:53:00.251999+0000","last_undegraded":"2024-03-09T08:53:00.251999+0000","last_fullsized":"2024-03-09T08:53:00.251999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:09:32.044185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236176+0000","last_change":"2024-03-09T08:52:29.595055+0000","last_active":"2024-03-09T08:53:00.236176+0000","last_peered":"2024-03-09T08:53:00.236176+0000","last_clean":"2024-03-09T08:53:00.236176+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T08:53:00.236176+0000","last_undegraded":"2024-03-09T08:53:00.236176+0000","last_fullsized":"2024-03-09T08:53:00.236176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:53.209169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238706+0000","last_change":"2024-03-09T08:52:18.248973+0000","last_active":"2024-03-09T08:53:00.238706+0000","last_peered":"2024-03-09T08:53:00.238706+0000","last_clean":"2024-03-09T08:53:00.238706+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T08:53:00.238706+0000","last_undegraded":"2024-03-09T08:53:00.238706+0000","last_fullsized":"2024-03-09T08:53:00.238706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:09:11.372616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2668,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":704107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":115553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":113829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":113297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":307200,"data_stored":116571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":113137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":114861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":884736,"data_stored":704073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T08:53:08.497 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T08:53:08.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.733+0000 7fc50e200640 1 Processor -- start 2024-03-09T08:53:08.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.733+0000 7fc50e200640 1 -- start start 2024-03-09T08:53:08.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.735+0000 7fc50e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5080c22a0 0x7fc508071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.735+0000 7fc50e200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc508153d40 con 0x7fc5080ac730 2024-03-09T08:53:08.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.735+0000 7fc50e200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5080c2790 con 0x7fc508157d60 2024-03-09T08:53:08.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.735+0000 7fc50e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc508070ee0 con 0x7fc5080c22a0 2024-03-09T08:53:08.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.735+0000 7fc507400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc5080ac730 0x7fc508157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55654/0 (socket says 172.21.15.67:55654) 2024-03-09T08:53:08.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.735+0000 7fc507400640 1 -- 172.21.15.67:0/1546724823 learned_addr learned my addr 172.21.15.67:0/1546724823 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:08.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.736+0000 7fc506a00640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5080c22a0 0x7fc508071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.736+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120874310 0 0) 0x7fc508153d40 con 0x7fc5080ac730 2024-03-09T08:53:08.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.736+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4ec003520 con 0x7fc5080ac730 2024-03-09T08:53:08.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.736+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3682322593 0 0) 0x7fc4ec003520 con 0x7fc5080ac730 2024-03-09T08:53:08.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.736+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5080c22a0 msgr2=0x7fc508071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:08.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.736+0000 7fc506000640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5080c22a0 0x7fc508071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 >> v1:172.21.15.134:6789/0 conn(0x7fc508157d60 legacy=0x7fc50815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc508070920 con 0x7fc5080ac730 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fc4f8001670 con 0x7fc5080ac730 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 >> v1:172.21.15.67:6789/0 conn(0x7fc5080ac730 legacy=0x7fc508157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc506000640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 0x7fc4ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc506000640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4ec004800 0x7fc4ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc506000640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4ec0051c0 0x7fc4ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc508070ee0 con 0x7fc4ec0039e0 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec00c090 con 0x7fc4ec0051c0 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec00c290 con 0x7fc4ec004800 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc506a00640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5080c22a0 0x7fc508071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc507400640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 0x7fc4ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.737+0000 7fc507e00640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4ec0051c0 0x7fc4ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.738+0000 7fc506a00640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4ec004800 0x7fc4ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.738+0000 7fc507400640 1 -- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4ec004800 msgr2=0x7fc4ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:08.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.738+0000 7fc507400640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4ec004800 0x7fc4ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.738+0000 7fc507400640 1 -- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4ec0051c0 msgr2=0x7fc4ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:08.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.738+0000 7fc507400640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4ec0051c0 0x7fc4ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.738+0000 7fc507400640 1 -- 172.21.15.67:0/1546724823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec003520 con 0x7fc4ec0039e0 2024-03-09T08:53:08.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.738+0000 7fc507e00640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4ec0051c0 0x7fc4ec00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:53:08.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.739+0000 7fc507400640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 0x7fc4ec003e00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800c240 tx=0x7fc4ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bef34591af063af server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:08.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.739+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8007d10 con 0x7fc4ec0039e0 2024-03-09T08:53:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.740+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f8030070 con 0x7fc4ec0039e0 2024-03-09T08:53:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.740+0000 7fc506000640 1 -- 172.21.15.67:0/1546724823 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f802cd60 con 0x7fc4ec0039e0 2024-03-09T08:53:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.740+0000 7fc50e200640 1 -- 172.21.15.67:0/1546724823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 msgr2=0x7fc4ec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.740+0000 7fc50e200640 1 --2- 172.21.15.67:0/1546724823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 0x7fc4ec003e00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800c240 tx=0x7fc4ec005830 comp rx=0 tx=0).stop 2024-03-09T08:53:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.740+0000 7fc507e00640 1 -- 172.21.15.67:0/1546724823 reap_dead start 2024-03-09T08:53:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.741+0000 7fc50e200640 1 -- 172.21.15.67:0/1546724823 shutdown_connections 2024-03-09T08:53:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.741+0000 7fc50e200640 1 -- 172.21.15.67:0/1546724823 >> 172.21.15.67:0/1546724823 conn(0x7fc5080c7370 msgr2=0x7fc5080c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.741+0000 7fc50e200640 1 -- 172.21.15.67:0/1546724823 shutdown_connections 2024-03-09T08:53:08.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.741+0000 7fc50e200640 1 -- 172.21.15.67:0/1546724823 wait complete. 2024-03-09T08:53:08.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.743+0000 7fc50e200640 1 Processor -- start 2024-03-09T08:53:08.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.743+0000 7fc50e200640 1 -- start start 2024-03-09T08:53:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc50e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 0x7fc5081ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc50e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc50809bcc0 0x7fc508153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc50e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50809d9e0 0x7fc50809de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc50e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc508151610 con 0x7fc4ec0039e0 2024-03-09T08:53:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc50e200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc508154d00 con 0x7fc50809d9e0 2024-03-09T08:53:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc50e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc508151790 con 0x7fc50809bcc0 2024-03-09T08:53:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc507400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 0x7fc5081ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc507400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 0x7fc5081ef1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35546/0 (socket says 172.21.15.67:35546) 2024-03-09T08:53:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc507400640 1 -- 172.21.15.67:0/1447226855 learned_addr learned my addr 172.21.15.67:0/1447226855 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:08.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc507e00640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50809d9e0 0x7fc50809de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc506a00640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc50809bcc0 0x7fc508153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc507400640 1 -- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc50809bcc0 msgr2=0x7fc508153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:08.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc507400640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc50809bcc0 0x7fc508153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc507400640 1 -- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50809d9e0 msgr2=0x7fc50809de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:08.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc507400640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50809d9e0 0x7fc50809de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:08.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.744+0000 7fc507400640 1 -- 172.21.15.67:0/1447226855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5081ef700 con 0x7fc4ec0039e0 2024-03-09T08:53:08.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.745+0000 7fc507e00640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50809d9e0 0x7fc50809de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:08.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.745+0000 7fc507400640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 0x7fc5081ef1c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800c240 tx=0x7fc508070af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:08.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.745+0000 7fc4ff400640 1 -- 172.21.15.67:0/1447226855 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8004bf0 con 0x7fc4ec0039e0 2024-03-09T08:53:08.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.745+0000 7fc4ff400640 1 -- 172.21.15.67:0/1447226855 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f8004db0 con 0x7fc4ec0039e0 2024-03-09T08:53:08.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.745+0000 7fc50e200640 1 -- 172.21.15.67:0/1447226855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50809c1c0 con 0x7fc4ec0039e0 2024-03-09T08:53:08.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.745+0000 7fc4ff400640 1 -- 172.21.15.67:0/1447226855 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80072b0 con 0x7fc4ec0039e0 2024-03-09T08:53:08.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.745+0000 7fc50e200640 1 -- 172.21.15.67:0/1447226855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50809d1e0 con 0x7fc4ec0039e0 2024-03-09T08:53:08.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.747+0000 7fc50e200640 1 -- 172.21.15.67:0/1447226855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc508075a00 con 0x7fc4ec0039e0 2024-03-09T08:53:08.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.748+0000 7fc4ff400640 1 -- 172.21.15.67:0/1447226855 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc4f8007450 con 0x7fc4ec0039e0 2024-03-09T08:53:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.750+0000 7fc4ff400640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc4e806b4d0 0x7fc4e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.750+0000 7fc4ff400640 1 -- 172.21.15.67:0/1447226855 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fc4f80ac000 con 0x7fc4ec0039e0 2024-03-09T08:53:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.750+0000 7fc506a00640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc4e806b4d0 0x7fc4e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.752+0000 7fc506a00640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc4e806b4d0 0x7fc4e806d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5081e9040 tx=0x7fc4f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.755+0000 7fc4ff400640 1 -- 172.21.15.67:0/1447226855 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc4f807ab70 con 0x7fc4ec0039e0 2024-03-09T08:53:08.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:53:08.953 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:53:08.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.958+0000 7fc50e200640 1 -- 172.21.15.67:0/1447226855 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5080c1d70 con 0x7fc4e806b4d0 2024-03-09T08:53:08.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.991+0000 7fc4ff400640 1 -- 172.21.15.67:0/1447226855 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490337 (secure 0 0 0) 0x7fc5080c1d70 con 0x7fc4e806b4d0 2024-03-09T08:53:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:53:08.999 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T08:53:09.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.997+0000 7fc50e200640 1 -- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc4e806b4d0 msgr2=0x7fc4e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.998+0000 7fc50e200640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc4e806b4d0 0x7fc4e806d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5081e9040 tx=0x7fc4f4008040 comp rx=0 tx=0).stop 2024-03-09T08:53:09.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.998+0000 7fc50e200640 1 -- 172.21.15.67:0/1447226855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 msgr2=0x7fc5081ef1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.998+0000 7fc50e200640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 0x7fc5081ef1c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800c240 tx=0x7fc508070af0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.998+0000 7fc50e200640 1 -- 172.21.15.67:0/1447226855 shutdown_connections 2024-03-09T08:53:09.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.998+0000 7fc50e200640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc4e806b4d0 0x7fc4e806d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.998+0000 7fc50e200640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc50809d9e0 0x7fc50809de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.998+0000 7fc50e200640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc50809bcc0 0x7fc508153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.998+0000 7fc50e200640 1 --2- 172.21.15.67:0/1447226855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4ec0039e0 0x7fc5081ef1c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.998+0000 7fc50e200640 1 -- 172.21.15.67:0/1447226855 >> 172.21.15.67:0/1447226855 conn(0x7fc5080c7370 msgr2=0x7fc5081558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:09.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.998+0000 7fc50e200640 1 -- 172.21.15.67:0/1447226855 shutdown_connections 2024-03-09T08:53:09.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:08.998+0000 7fc50e200640 1 -- 172.21.15.67:0/1447226855 wait complete. 2024-03-09T08:53:09.012 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-03-09T08:53:08.694703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596691,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":254,"num_write_kb":2449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219628,"kb_used_data":3464,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755092,"statfs":{"total":773094113280,"available":772869214208,"internally_reserved":0,"allocated":3547136,"data_stored":2097918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1503,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.911378"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790031+0000","last_change":"2024-03-09T08:52:29.573571+0000","last_active":"2024-03-09T08:52:59.790031+0000","last_peered":"2024-03-09T08:52:59.790031+0000","last_clean":"2024-03-09T08:52:59.790031+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T08:52:59.790031+0000","last_undegraded":"2024-03-09T08:52:59.790031+0000","last_fullsized":"2024-03-09T08:52:59.790031+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:33:33.068463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794135+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T08:52:59.794135+0000","last_peered":"2024-03-09T08:52:59.794135+0000","last_clean":"2024-03-09T08:52:59.794135+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T08:52:59.794135+0000","last_undegraded":"2024-03-09T08:52:59.794135+0000","last_fullsized":"2024-03-09T08:52:59.794135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:41:40.332687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793042+0000","last_change":"2024-03-09T08:52:18.235099+0000","last_active":"2024-03-09T08:52:59.793042+0000","last_peered":"2024-03-09T08:52:59.793042+0000","last_clean":"2024-03-09T08:52:59.793042+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T08:52:59.793042+0000","last_undegraded":"2024-03-09T08:52:59.793042+0000","last_fullsized":"2024-03-09T08:52:59.793042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:39:28.882461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790160+0000","last_change":"2024-03-09T08:52:28.571905+0000","last_active":"2024-03-09T08:52:59.790160+0000","last_peered":"2024-03-09T08:52:59.790160+0000","last_clean":"2024-03-09T08:52:59.790160+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T08:52:59.790160+0000","last_undegraded":"2024-03-09T08:52:59.790160+0000","last_fullsized":"2024-03-09T08:52:59.790160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:01.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.249405+0000","last_change":"2024-03-09T08:52:29.578644+0000","last_active":"2024-03-09T08:53:00.249405+0000","last_peered":"2024-03-09T08:53:00.249405+0000","last_clean":"2024-03-09T08:53:00.249405+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T08:53:00.249405+0000","last_undegraded":"2024-03-09T08:53:00.249405+0000","last_fullsized":"2024-03-09T08:53:00.249405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:00.044620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793123+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T08:52:59.793123+0000","last_peered":"2024-03-09T08:52:59.793123+0000","last_clean":"2024-03-09T08:52:59.793123+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T08:52:59.793123+0000","last_undegraded":"2024-03-09T08:52:59.793123+0000","last_fullsized":"2024-03-09T08:52:59.793123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793800+0000","last_change":"2024-03-09T08:52:18.236946+0000","last_active":"2024-03-09T08:52:59.793800+0000","last_peered":"2024-03-09T08:52:59.793800+0000","last_clean":"2024-03-09T08:52:59.793800+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T08:52:59.793800+0000","last_undegraded":"2024-03-09T08:52:59.793800+0000","last_fullsized":"2024-03-09T08:52:59.793800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:45.372160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790242+0000","last_change":"2024-03-09T08:52:28.573074+0000","last_active":"2024-03-09T08:52:59.790242+0000","last_peered":"2024-03-09T08:52:59.790242+0000","last_clean":"2024-03-09T08:52:59.790242+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T08:52:59.790242+0000","last_undegraded":"2024-03-09T08:52:59.790242+0000","last_fullsized":"2024-03-09T08:52:59.790242+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:50.600408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236942+0000","last_change":"2024-03-09T08:52:29.594030+0000","last_active":"2024-03-09T08:53:00.236942+0000","last_peered":"2024-03-09T08:53:00.236942+0000","last_clean":"2024-03-09T08:53:00.236942+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T08:53:00.236942+0000","last_undegraded":"2024-03-09T08:53:00.236942+0000","last_fullsized":"2024-03-09T08:53:00.236942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:57.279882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238298+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T08:53:00.238298+0000","last_peered":"2024-03-09T08:53:00.238298+0000","last_clean":"2024-03-09T08:53:00.238298+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T08:53:00.238298+0000","last_undegraded":"2024-03-09T08:53:00.238298+0000","last_fullsized":"2024-03-09T08:53:00.238298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435811+0000","last_change":"2024-03-09T08:52:18.230629+0000","last_active":"2024-03-09T08:53:00.435811+0000","last_peered":"2024-03-09T08:53:00.435811+0000","last_clean":"2024-03-09T08:53:00.435811+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T08:53:00.435811+0000","last_undegraded":"2024-03-09T08:53:00.435811+0000","last_fullsized":"2024-03-09T08:53:00.435811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:46:38.528839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792389+0000","last_change":"2024-03-09T08:52:28.577128+0000","last_active":"2024-03-09T08:52:59.792389+0000","last_peered":"2024-03-09T08:52:59.792389+0000","last_clean":"2024-03-09T08:52:59.792389+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T08:52:59.792389+0000","last_undegraded":"2024-03-09T08:52:59.792389+0000","last_fullsized":"2024-03-09T08:52:59.792389+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:41.801752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434188+0000","last_change":"2024-03-09T08:52:29.581705+0000","last_active":"2024-03-09T08:53:00.434188+0000","last_peered":"2024-03-09T08:53:00.434188+0000","last_clean":"2024-03-09T08:53:00.434188+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T08:53:00.434188+0000","last_undegraded":"2024-03-09T08:53:00.434188+0000","last_fullsized":"2024-03-09T08:53:00.434188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:27.066135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238318+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T08:53:00.238318+0000","last_peered":"2024-03-09T08:53:00.238318+0000","last_clean":"2024-03-09T08:53:00.238318+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T08:53:00.238318+0000","last_undegraded":"2024-03-09T08:53:00.238318+0000","last_fullsized":"2024-03-09T08:53:00.238318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435922+0000","last_change":"2024-03-09T08:52:18.228419+0000","last_active":"2024-03-09T08:53:00.435922+0000","last_peered":"2024-03-09T08:53:00.435922+0000","last_clean":"2024-03-09T08:53:00.435922+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T08:53:00.435922+0000","last_undegraded":"2024-03-09T08:53:00.435922+0000","last_fullsized":"2024-03-09T08:53:00.435922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:19:12.533402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791700+0000","last_change":"2024-03-09T08:52:28.577069+0000","last_active":"2024-03-09T08:52:59.791700+0000","last_peered":"2024-03-09T08:52:59.791700+0000","last_clean":"2024-03-09T08:52:59.791700+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T08:52:59.791700+0000","last_undegraded":"2024-03-09T08:52:59.791700+0000","last_fullsized":"2024-03-09T08:52:59.791700+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:09:02.410915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250908+0000","last_change":"2024-03-09T08:52:29.588922+0000","last_active":"2024-03-09T08:53:00.250908+0000","last_peered":"2024-03-09T08:53:00.250908+0000","last_clean":"2024-03-09T08:53:00.250908+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T08:53:00.250908+0000","last_undegraded":"2024-03-09T08:53:00.250908+0000","last_fullsized":"2024-03-09T08:53:00.250908+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:13.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793277+0000","last_change":"2024-03-09T08:52:18.234372+0000","last_active":"2024-03-09T08:52:59.793277+0000","last_peered":"2024-03-09T08:52:59.793277+0000","last_clean":"2024-03-09T08:52:59.793277+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T08:52:59.793277+0000","last_undegraded":"2024-03-09T08:52:59.793277+0000","last_fullsized":"2024-03-09T08:52:59.793277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:09:33.422907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238094+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T08:53:00.238094+0000","last_peered":"2024-03-09T08:53:00.238094+0000","last_clean":"2024-03-09T08:53:00.238094+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T08:53:00.238094+0000","last_undegraded":"2024-03-09T08:53:00.238094+0000","last_fullsized":"2024-03-09T08:53:00.238094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794427+0000","last_change":"2024-03-09T08:52:28.570631+0000","last_active":"2024-03-09T08:52:59.794427+0000","last_peered":"2024-03-09T08:52:59.794427+0000","last_clean":"2024-03-09T08:52:59.794427+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T08:52:59.794427+0000","last_undegraded":"2024-03-09T08:52:59.794427+0000","last_fullsized":"2024-03-09T08:52:59.794427+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:06.630704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.799005+0000","last_change":"2024-03-09T08:52:29.596820+0000","last_active":"2024-03-09T08:52:59.799005+0000","last_peered":"2024-03-09T08:52:59.799005+0000","last_clean":"2024-03-09T08:52:59.799005+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T08:52:59.799005+0000","last_undegraded":"2024-03-09T08:52:59.799005+0000","last_fullsized":"2024-03-09T08:52:59.799005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:01:20.331889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790671+0000","last_change":"2024-03-09T08:52:18.244306+0000","last_active":"2024-03-09T08:52:59.790671+0000","last_peered":"2024-03-09T08:52:59.790671+0000","last_clean":"2024-03-09T08:52:59.790671+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T08:52:59.790671+0000","last_undegraded":"2024-03-09T08:52:59.790671+0000","last_fullsized":"2024-03-09T08:52:59.790671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:58.365524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250537+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T08:53:00.250537+0000","last_peered":"2024-03-09T08:53:00.250537+0000","last_clean":"2024-03-09T08:53:00.250537+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T08:53:00.250537+0000","last_undegraded":"2024-03-09T08:53:00.250537+0000","last_fullsized":"2024-03-09T08:53:00.250537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:45:45.612953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792259+0000","last_change":"2024-03-09T08:52:28.577110+0000","last_active":"2024-03-09T08:52:59.792259+0000","last_peered":"2024-03-09T08:52:59.792259+0000","last_clean":"2024-03-09T08:52:59.792259+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T08:52:59.792259+0000","last_undegraded":"2024-03-09T08:52:59.792259+0000","last_fullsized":"2024-03-09T08:52:59.792259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:57.197373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794138+0000","last_change":"2024-03-09T08:52:29.579929+0000","last_active":"2024-03-09T08:52:59.794138+0000","last_peered":"2024-03-09T08:52:59.794138+0000","last_clean":"2024-03-09T08:52:59.794138+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T08:52:59.794138+0000","last_undegraded":"2024-03-09T08:52:59.794138+0000","last_fullsized":"2024-03-09T08:52:59.794138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:37:23.205337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789838+0000","last_change":"2024-03-09T08:52:18.233208+0000","last_active":"2024-03-09T08:52:59.789838+0000","last_peered":"2024-03-09T08:52:59.789838+0000","last_clean":"2024-03-09T08:52:59.789838+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T08:52:59.789838+0000","last_undegraded":"2024-03-09T08:52:59.789838+0000","last_fullsized":"2024-03-09T08:52:59.789838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:17.589783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.249867+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T08:53:00.249867+0000","last_peered":"2024-03-09T08:53:00.249867+0000","last_clean":"2024-03-09T08:53:00.249867+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T08:53:00.249867+0000","last_undegraded":"2024-03-09T08:53:00.249867+0000","last_fullsized":"2024-03-09T08:53:00.249867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:21:59.990396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252140+0000","last_change":"2024-03-09T08:52:28.576254+0000","last_active":"2024-03-09T08:53:00.252140+0000","last_peered":"2024-03-09T08:53:00.252140+0000","last_clean":"2024-03-09T08:53:00.252140+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T08:53:00.252140+0000","last_undegraded":"2024-03-09T08:53:00.252140+0000","last_fullsized":"2024-03-09T08:53:00.252140+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:21:56.330955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.436326+0000","last_change":"2024-03-09T08:52:29.578272+0000","last_active":"2024-03-09T08:53:00.436326+0000","last_peered":"2024-03-09T08:53:00.436326+0000","last_clean":"2024-03-09T08:53:00.436326+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T08:53:00.436326+0000","last_undegraded":"2024-03-09T08:53:00.436326+0000","last_fullsized":"2024-03-09T08:53:00.436326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:56.180809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791263+0000","last_change":"2024-03-09T08:52:18.217046+0000","last_active":"2024-03-09T08:52:59.791263+0000","last_peered":"2024-03-09T08:52:59.791263+0000","last_clean":"2024-03-09T08:52:59.791263+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T08:52:59.791263+0000","last_undegraded":"2024-03-09T08:52:59.791263+0000","last_fullsized":"2024-03-09T08:52:59.791263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:14.884963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237306+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T08:53:00.237306+0000","last_peered":"2024-03-09T08:53:00.237306+0000","last_clean":"2024-03-09T08:53:00.237306+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T08:53:00.237306+0000","last_undegraded":"2024-03-09T08:53:00.237306+0000","last_fullsized":"2024-03-09T08:53:00.237306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794145+0000","last_change":"2024-03-09T08:52:28.573638+0000","last_active":"2024-03-09T08:52:59.794145+0000","last_peered":"2024-03-09T08:52:59.794145+0000","last_clean":"2024-03-09T08:52:59.794145+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T08:52:59.794145+0000","last_undegraded":"2024-03-09T08:52:59.794145+0000","last_fullsized":"2024-03-09T08:52:59.794145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:02.478112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.788764+0000","last_change":"2024-03-09T08:52:29.597697+0000","last_active":"2024-03-09T08:52:59.788764+0000","last_peered":"2024-03-09T08:52:59.788764+0000","last_clean":"2024-03-09T08:52:59.788764+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T08:52:59.788764+0000","last_undegraded":"2024-03-09T08:52:59.788764+0000","last_fullsized":"2024-03-09T08:52:59.788764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:00.890845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796799+0000","last_change":"2024-03-09T08:52:18.229739+0000","last_active":"2024-03-09T08:52:59.796799+0000","last_peered":"2024-03-09T08:52:59.796799+0000","last_clean":"2024-03-09T08:52:59.796799+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T08:52:59.796799+0000","last_undegraded":"2024-03-09T08:52:59.796799+0000","last_fullsized":"2024-03-09T08:52:59.796799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:24.394779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795614+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T08:52:59.795614+0000","last_peered":"2024-03-09T08:52:59.795614+0000","last_clean":"2024-03-09T08:52:59.795614+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T08:52:59.795614+0000","last_undegraded":"2024-03-09T08:52:59.795614+0000","last_fullsized":"2024-03-09T08:52:59.795614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:35:38.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790482+0000","last_change":"2024-03-09T08:52:28.575630+0000","last_active":"2024-03-09T08:52:59.790482+0000","last_peered":"2024-03-09T08:52:59.790482+0000","last_clean":"2024-03-09T08:52:59.790482+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T08:52:59.790482+0000","last_undegraded":"2024-03-09T08:52:59.790482+0000","last_fullsized":"2024-03-09T08:52:59.790482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:48.210921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789770+0000","last_change":"2024-03-09T08:52:29.576677+0000","last_active":"2024-03-09T08:52:59.789770+0000","last_peered":"2024-03-09T08:52:59.789770+0000","last_clean":"2024-03-09T08:52:59.789770+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T08:52:59.789770+0000","last_undegraded":"2024-03-09T08:52:59.789770+0000","last_fullsized":"2024-03-09T08:52:59.789770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:19.026873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.244221+0000","last_change":"2024-03-09T08:53:00.241853+0000","last_active":"2024-03-09T08:53:00.244221+0000","last_peered":"2024-03-09T08:53:00.244221+0000","last_clean":"2024-03-09T08:53:00.244221+0000","last_became_active":"2024-03-09T08:53:00.240844+0000","last_became_peered":"2024-03-09T08:53:00.240844+0000","last_unstale":"2024-03-09T08:53:00.244221+0000","last_undegraded":"2024-03-09T08:53:00.244221+0000","last_fullsized":"2024-03-09T08:53:00.244221+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:18:38.901416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252466+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T08:53:00.252466+0000","last_peered":"2024-03-09T08:53:00.252466+0000","last_clean":"2024-03-09T08:53:00.252466+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T08:53:00.252466+0000","last_undegraded":"2024-03-09T08:53:00.252466+0000","last_fullsized":"2024-03-09T08:53:00.252466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:30.201471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.788952+0000","last_change":"2024-03-09T08:52:28.586702+0000","last_active":"2024-03-09T08:52:59.788952+0000","last_peered":"2024-03-09T08:52:59.788952+0000","last_clean":"2024-03-09T08:52:59.788952+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T08:52:59.788952+0000","last_undegraded":"2024-03-09T08:52:59.788952+0000","last_fullsized":"2024-03-09T08:52:59.788952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:09.201224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792707+0000","last_change":"2024-03-09T08:52:29.578956+0000","last_active":"2024-03-09T08:52:59.792707+0000","last_peered":"2024-03-09T08:52:59.792707+0000","last_clean":"2024-03-09T08:52:59.792707+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T08:52:59.792707+0000","last_undegraded":"2024-03-09T08:52:59.792707+0000","last_fullsized":"2024-03-09T08:52:59.792707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:00:00.884211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251496+0000","last_change":"2024-03-09T08:52:18.234887+0000","last_active":"2024-03-09T08:53:00.251496+0000","last_peered":"2024-03-09T08:53:00.251496+0000","last_clean":"2024-03-09T08:53:00.251496+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T08:53:00.251496+0000","last_undegraded":"2024-03-09T08:53:00.251496+0000","last_fullsized":"2024-03-09T08:53:00.251496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:44.661882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238623+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T08:53:00.238623+0000","last_peered":"2024-03-09T08:53:00.238623+0000","last_clean":"2024-03-09T08:53:00.238623+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T08:53:00.238623+0000","last_undegraded":"2024-03-09T08:53:00.238623+0000","last_fullsized":"2024-03-09T08:53:00.238623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794898+0000","last_change":"2024-03-09T08:52:28.580879+0000","last_active":"2024-03-09T08:52:59.794898+0000","last_peered":"2024-03-09T08:52:59.794898+0000","last_clean":"2024-03-09T08:52:59.794898+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T08:52:59.794898+0000","last_undegraded":"2024-03-09T08:52:59.794898+0000","last_fullsized":"2024-03-09T08:52:59.794898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:19:44.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795587+0000","last_change":"2024-03-09T08:52:29.580024+0000","last_active":"2024-03-09T08:52:59.795587+0000","last_peered":"2024-03-09T08:52:59.795587+0000","last_clean":"2024-03-09T08:52:59.795587+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T08:52:59.795587+0000","last_undegraded":"2024-03-09T08:52:59.795587+0000","last_fullsized":"2024-03-09T08:52:59.795587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:02:13.195642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435983+0000","last_change":"2024-03-09T08:52:18.250999+0000","last_active":"2024-03-09T08:53:00.435983+0000","last_peered":"2024-03-09T08:53:00.435983+0000","last_clean":"2024-03-09T08:53:00.435983+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T08:53:00.435983+0000","last_undegraded":"2024-03-09T08:53:00.435983+0000","last_fullsized":"2024-03-09T08:53:00.435983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:49:54.496603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251763+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T08:53:00.251763+0000","last_peered":"2024-03-09T08:53:00.251763+0000","last_clean":"2024-03-09T08:53:00.251763+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T08:53:00.251763+0000","last_undegraded":"2024-03-09T08:53:00.251763+0000","last_fullsized":"2024-03-09T08:53:00.251763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250224+0000","last_change":"2024-03-09T08:52:28.584457+0000","last_active":"2024-03-09T08:53:00.250224+0000","last_peered":"2024-03-09T08:53:00.250224+0000","last_clean":"2024-03-09T08:53:00.250224+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T08:53:00.250224+0000","last_undegraded":"2024-03-09T08:53:00.250224+0000","last_fullsized":"2024-03-09T08:53:00.250224+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:45.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795381+0000","last_change":"2024-03-09T08:52:29.596486+0000","last_active":"2024-03-09T08:52:59.795381+0000","last_peered":"2024-03-09T08:52:59.795381+0000","last_clean":"2024-03-09T08:52:59.795381+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T08:52:59.795381+0000","last_undegraded":"2024-03-09T08:52:59.795381+0000","last_fullsized":"2024-03-09T08:52:59.795381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:40:16.587782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793658+0000","last_change":"2024-03-09T08:52:18.227365+0000","last_active":"2024-03-09T08:52:59.793658+0000","last_peered":"2024-03-09T08:52:59.793658+0000","last_clean":"2024-03-09T08:52:59.793658+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T08:52:59.793658+0000","last_undegraded":"2024-03-09T08:52:59.793658+0000","last_fullsized":"2024-03-09T08:52:59.793658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:26.357063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435231+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T08:53:00.435231+0000","last_peered":"2024-03-09T08:53:00.435231+0000","last_clean":"2024-03-09T08:53:00.435231+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T08:53:00.435231+0000","last_undegraded":"2024-03-09T08:53:00.435231+0000","last_fullsized":"2024-03-09T08:53:00.435231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251559+0000","last_change":"2024-03-09T08:52:28.578096+0000","last_active":"2024-03-09T08:53:00.251559+0000","last_peered":"2024-03-09T08:53:00.251559+0000","last_clean":"2024-03-09T08:53:00.251559+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T08:53:00.251559+0000","last_undegraded":"2024-03-09T08:53:00.251559+0000","last_fullsized":"2024-03-09T08:53:00.251559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:42.645517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.800380+0000","last_change":"2024-03-09T08:52:29.590834+0000","last_active":"2024-03-09T08:52:59.800380+0000","last_peered":"2024-03-09T08:52:59.800380+0000","last_clean":"2024-03-09T08:52:59.800380+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T08:52:59.800380+0000","last_undegraded":"2024-03-09T08:52:59.800380+0000","last_fullsized":"2024-03-09T08:52:59.800380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:24:06.310348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.249571+0000","last_change":"2024-03-09T08:52:18.235832+0000","last_active":"2024-03-09T08:53:00.249571+0000","last_peered":"2024-03-09T08:53:00.249571+0000","last_clean":"2024-03-09T08:53:00.249571+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T08:53:00.249571+0000","last_undegraded":"2024-03-09T08:53:00.249571+0000","last_fullsized":"2024-03-09T08:53:00.249571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:38:58.751526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798104+0000","last_change":"2024-03-09T08:52:17.271847+0000","last_active":"2024-03-09T08:52:59.798104+0000","last_peered":"2024-03-09T08:52:59.798104+0000","last_clean":"2024-03-09T08:52:59.798104+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T08:52:59.798104+0000","last_undegraded":"2024-03-09T08:52:59.798104+0000","last_fullsized":"2024-03-09T08:52:59.798104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:27.418674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252245+0000","last_change":"2024-03-09T08:52:28.588636+0000","last_active":"2024-03-09T08:53:00.252245+0000","last_peered":"2024-03-09T08:53:00.252245+0000","last_clean":"2024-03-09T08:53:00.252245+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T08:53:00.252245+0000","last_undegraded":"2024-03-09T08:53:00.252245+0000","last_fullsized":"2024-03-09T08:53:00.252245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:21.601182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.436536+0000","last_change":"2024-03-09T08:52:29.606833+0000","last_active":"2024-03-09T08:53:00.436536+0000","last_peered":"2024-03-09T08:53:00.436536+0000","last_clean":"2024-03-09T08:53:00.436536+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T08:53:00.436536+0000","last_undegraded":"2024-03-09T08:53:00.436536+0000","last_fullsized":"2024-03-09T08:53:00.436536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:40.299681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793719+0000","last_change":"2024-03-09T08:52:18.214396+0000","last_active":"2024-03-09T08:52:59.793719+0000","last_peered":"2024-03-09T08:52:59.793719+0000","last_clean":"2024-03-09T08:52:59.793719+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T08:52:59.793719+0000","last_undegraded":"2024-03-09T08:52:59.793719+0000","last_fullsized":"2024-03-09T08:52:59.793719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:24:10.719351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236741+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T08:53:00.236741+0000","last_peered":"2024-03-09T08:53:00.236741+0000","last_clean":"2024-03-09T08:53:00.236741+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T08:53:00.236741+0000","last_undegraded":"2024-03-09T08:53:00.236741+0000","last_fullsized":"2024-03-09T08:53:00.236741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.797004+0000","last_change":"2024-03-09T08:52:28.578599+0000","last_active":"2024-03-09T08:52:59.797004+0000","last_peered":"2024-03-09T08:52:59.797004+0000","last_clean":"2024-03-09T08:52:59.797004+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T08:52:59.797004+0000","last_undegraded":"2024-03-09T08:52:59.797004+0000","last_fullsized":"2024-03-09T08:52:59.797004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:52:31.328248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238697+0000","last_change":"2024-03-09T08:52:29.591872+0000","last_active":"2024-03-09T08:53:00.238697+0000","last_peered":"2024-03-09T08:53:00.238697+0000","last_clean":"2024-03-09T08:53:00.238697+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T08:53:00.238697+0000","last_undegraded":"2024-03-09T08:53:00.238697+0000","last_fullsized":"2024-03-09T08:53:00.238697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:44:17.870183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791322+0000","last_change":"2024-03-09T08:52:18.235367+0000","last_active":"2024-03-09T08:52:59.791322+0000","last_peered":"2024-03-09T08:52:59.791322+0000","last_clean":"2024-03-09T08:52:59.791322+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T08:52:59.791322+0000","last_undegraded":"2024-03-09T08:52:59.791322+0000","last_fullsized":"2024-03-09T08:52:59.791322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:40.736440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434369+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T08:53:00.434369+0000","last_peered":"2024-03-09T08:53:00.434369+0000","last_clean":"2024-03-09T08:53:00.434369+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T08:53:00.434369+0000","last_undegraded":"2024-03-09T08:53:00.434369+0000","last_fullsized":"2024-03-09T08:53:00.434369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237218+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T08:53:00.237218+0000","last_peered":"2024-03-09T08:53:00.237218+0000","last_clean":"2024-03-09T08:53:00.237218+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T08:53:00.237218+0000","last_undegraded":"2024-03-09T08:53:00.237218+0000","last_fullsized":"2024-03-09T08:53:00.237218+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793707+0000","last_change":"2024-03-09T08:52:28.577973+0000","last_active":"2024-03-09T08:52:59.793707+0000","last_peered":"2024-03-09T08:52:59.793707+0000","last_clean":"2024-03-09T08:52:59.793707+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T08:52:59.793707+0000","last_undegraded":"2024-03-09T08:52:59.793707+0000","last_fullsized":"2024-03-09T08:52:59.793707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:13.291622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798312+0000","last_change":"2024-03-09T08:52:29.599601+0000","last_active":"2024-03-09T08:52:59.798312+0000","last_peered":"2024-03-09T08:52:59.798312+0000","last_clean":"2024-03-09T08:52:59.798312+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T08:52:59.798312+0000","last_undegraded":"2024-03-09T08:52:59.798312+0000","last_fullsized":"2024-03-09T08:52:59.798312+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:11:17.424330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.440926+0000","last_change":"2024-03-09T08:53:00.438594+0000","last_active":"2024-03-09T08:53:00.440926+0000","last_peered":"2024-03-09T08:53:00.440926+0000","last_clean":"2024-03-09T08:53:00.440926+0000","last_became_active":"2024-03-09T08:53:00.437859+0000","last_became_peered":"2024-03-09T08:53:00.437859+0000","last_unstale":"2024-03-09T08:53:00.440926+0000","last_undegraded":"2024-03-09T08:53:00.440926+0000","last_fullsized":"2024-03-09T08:53:00.440926+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:49:53.715132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.799007+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T08:52:59.799007+0000","last_peered":"2024-03-09T08:52:59.799007+0000","last_clean":"2024-03-09T08:52:59.799007+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T08:52:59.799007+0000","last_undegraded":"2024-03-09T08:52:59.799007+0000","last_fullsized":"2024-03-09T08:52:59.799007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:27.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793000+0000","last_change":"2024-03-09T08:52:28.589278+0000","last_active":"2024-03-09T08:52:59.793000+0000","last_peered":"2024-03-09T08:52:59.793000+0000","last_clean":"2024-03-09T08:52:59.793000+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T08:52:59.793000+0000","last_undegraded":"2024-03-09T08:52:59.793000+0000","last_fullsized":"2024-03-09T08:52:59.793000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:16:26.449275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798056+0000","last_change":"2024-03-09T08:52:29.600332+0000","last_active":"2024-03-09T08:52:59.798056+0000","last_peered":"2024-03-09T08:52:59.798056+0000","last_clean":"2024-03-09T08:52:59.798056+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T08:52:59.798056+0000","last_undegraded":"2024-03-09T08:52:59.798056+0000","last_fullsized":"2024-03-09T08:52:59.798056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:36:41.422546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.436028+0000","last_change":"2024-03-09T08:52:18.250433+0000","last_active":"2024-03-09T08:53:00.436028+0000","last_peered":"2024-03-09T08:53:00.436028+0000","last_clean":"2024-03-09T08:53:00.436028+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T08:53:00.436028+0000","last_undegraded":"2024-03-09T08:53:00.436028+0000","last_fullsized":"2024-03-09T08:53:00.436028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:29.742337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252598+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T08:53:00.252598+0000","last_peered":"2024-03-09T08:53:00.252598+0000","last_clean":"2024-03-09T08:53:00.252598+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T08:53:00.252598+0000","last_undegraded":"2024-03-09T08:53:00.252598+0000","last_fullsized":"2024-03-09T08:53:00.252598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:26:35.574903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794661+0000","last_change":"2024-03-09T08:52:28.583181+0000","last_active":"2024-03-09T08:52:59.794661+0000","last_peered":"2024-03-09T08:52:59.794661+0000","last_clean":"2024-03-09T08:52:59.794661+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T08:52:59.794661+0000","last_undegraded":"2024-03-09T08:52:59.794661+0000","last_fullsized":"2024-03-09T08:52:59.794661+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:58:50.061324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236170+0000","last_change":"2024-03-09T08:52:29.590141+0000","last_active":"2024-03-09T08:53:00.236170+0000","last_peered":"2024-03-09T08:53:00.236170+0000","last_clean":"2024-03-09T08:53:00.236170+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T08:53:00.236170+0000","last_undegraded":"2024-03-09T08:53:00.236170+0000","last_fullsized":"2024-03-09T08:53:00.236170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:23.981897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238927+0000","last_change":"2024-03-09T08:52:18.250420+0000","last_active":"2024-03-09T08:53:00.238927+0000","last_peered":"2024-03-09T08:53:00.238927+0000","last_clean":"2024-03-09T08:53:00.238927+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T08:53:00.238927+0000","last_undegraded":"2024-03-09T08:53:00.238927+0000","last_fullsized":"2024-03-09T08:53:00.238927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:48.826066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.800760+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T08:52:59.800760+0000","last_peered":"2024-03-09T08:52:59.800760+0000","last_clean":"2024-03-09T08:52:59.800760+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T08:52:59.800760+0000","last_undegraded":"2024-03-09T08:52:59.800760+0000","last_fullsized":"2024-03-09T08:52:59.800760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:16.138216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.433973+0000","last_change":"2024-03-09T08:52:28.587637+0000","last_active":"2024-03-09T08:53:00.433973+0000","last_peered":"2024-03-09T08:53:00.433973+0000","last_clean":"2024-03-09T08:53:00.433973+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T08:53:00.433973+0000","last_undegraded":"2024-03-09T08:53:00.433973+0000","last_fullsized":"2024-03-09T08:53:00.433973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:31:32.362061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.797119+0000","last_change":"2024-03-09T08:52:29.586180+0000","last_active":"2024-03-09T08:52:59.797119+0000","last_peered":"2024-03-09T08:52:59.797119+0000","last_clean":"2024-03-09T08:52:59.797119+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T08:52:59.797119+0000","last_undegraded":"2024-03-09T08:52:59.797119+0000","last_fullsized":"2024-03-09T08:52:59.797119+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:35:21.522181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238280+0000","last_change":"2024-03-09T08:52:18.246145+0000","last_active":"2024-03-09T08:53:00.238280+0000","last_peered":"2024-03-09T08:53:00.238280+0000","last_clean":"2024-03-09T08:53:00.238280+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T08:53:00.238280+0000","last_undegraded":"2024-03-09T08:53:00.238280+0000","last_fullsized":"2024-03-09T08:53:00.238280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:29:36.213774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795394+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T08:52:59.795394+0000","last_peered":"2024-03-09T08:52:59.795394+0000","last_clean":"2024-03-09T08:52:59.795394+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T08:52:59.795394+0000","last_undegraded":"2024-03-09T08:52:59.795394+0000","last_fullsized":"2024-03-09T08:52:59.795394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:10:34.698235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790421+0000","last_change":"2024-03-09T08:52:28.582308+0000","last_active":"2024-03-09T08:52:59.790421+0000","last_peered":"2024-03-09T08:52:59.790421+0000","last_clean":"2024-03-09T08:52:59.790421+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T08:52:59.790421+0000","last_undegraded":"2024-03-09T08:52:59.790421+0000","last_fullsized":"2024-03-09T08:52:59.790421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:04.918893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792433+0000","last_change":"2024-03-09T08:52:29.610596+0000","last_active":"2024-03-09T08:52:59.792433+0000","last_peered":"2024-03-09T08:52:59.792433+0000","last_clean":"2024-03-09T08:52:59.792433+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T08:52:59.792433+0000","last_undegraded":"2024-03-09T08:52:59.792433+0000","last_fullsized":"2024-03-09T08:52:59.792433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:19:47.313981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237024+0000","last_change":"2024-03-09T08:52:18.248755+0000","last_active":"2024-03-09T08:53:00.237024+0000","last_peered":"2024-03-09T08:53:00.237024+0000","last_clean":"2024-03-09T08:53:00.237024+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T08:53:00.237024+0000","last_undegraded":"2024-03-09T08:53:00.237024+0000","last_fullsized":"2024-03-09T08:53:00.237024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:37.083738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.797789+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T08:52:59.797789+0000","last_peered":"2024-03-09T08:52:59.797789+0000","last_clean":"2024-03-09T08:52:59.797789+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T08:52:59.797789+0000","last_undegraded":"2024-03-09T08:52:59.797789+0000","last_fullsized":"2024-03-09T08:52:59.797789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:31.859924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796777+0000","last_change":"2024-03-09T08:52:28.575975+0000","last_active":"2024-03-09T08:52:59.796777+0000","last_peered":"2024-03-09T08:52:59.796777+0000","last_clean":"2024-03-09T08:52:59.796777+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T08:52:59.796777+0000","last_undegraded":"2024-03-09T08:52:59.796777+0000","last_fullsized":"2024-03-09T08:52:59.796777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:51:24.405316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435825+0000","last_change":"2024-03-09T08:52:29.575752+0000","last_active":"2024-03-09T08:53:00.435825+0000","last_peered":"2024-03-09T08:53:00.435825+0000","last_clean":"2024-03-09T08:53:00.435825+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T08:53:00.435825+0000","last_undegraded":"2024-03-09T08:53:00.435825+0000","last_fullsized":"2024-03-09T08:53:00.435825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:31:51.847871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789742+0000","last_change":"2024-03-09T08:52:18.245073+0000","last_active":"2024-03-09T08:52:59.789742+0000","last_peered":"2024-03-09T08:52:59.789742+0000","last_clean":"2024-03-09T08:52:59.789742+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T08:52:59.789742+0000","last_undegraded":"2024-03-09T08:52:59.789742+0000","last_fullsized":"2024-03-09T08:52:59.789742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:40:15.390735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791951+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T08:52:59.791951+0000","last_peered":"2024-03-09T08:52:59.791951+0000","last_clean":"2024-03-09T08:52:59.791951+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T08:52:59.791951+0000","last_undegraded":"2024-03-09T08:52:59.791951+0000","last_fullsized":"2024-03-09T08:52:59.791951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:07:29.577461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792780+0000","last_change":"2024-03-09T08:52:28.588312+0000","last_active":"2024-03-09T08:52:59.792780+0000","last_peered":"2024-03-09T08:52:59.792780+0000","last_clean":"2024-03-09T08:52:59.792780+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T08:52:59.792780+0000","last_undegraded":"2024-03-09T08:52:59.792780+0000","last_fullsized":"2024-03-09T08:52:59.792780+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:14:52.895618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434697+0000","last_change":"2024-03-09T08:52:29.596647+0000","last_active":"2024-03-09T08:53:00.434697+0000","last_peered":"2024-03-09T08:53:00.434697+0000","last_clean":"2024-03-09T08:53:00.434697+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T08:53:00.434697+0000","last_undegraded":"2024-03-09T08:53:00.434697+0000","last_fullsized":"2024-03-09T08:53:00.434697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:35:28.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237560+0000","last_change":"2024-03-09T08:52:18.248093+0000","last_active":"2024-03-09T08:53:00.237560+0000","last_peered":"2024-03-09T08:53:00.237560+0000","last_clean":"2024-03-09T08:53:00.237560+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T08:53:00.237560+0000","last_undegraded":"2024-03-09T08:53:00.237560+0000","last_fullsized":"2024-03-09T08:53:00.237560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:29.414335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435050+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T08:53:00.435050+0000","last_peered":"2024-03-09T08:53:00.435050+0000","last_clean":"2024-03-09T08:53:00.435050+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T08:53:00.435050+0000","last_undegraded":"2024-03-09T08:53:00.435050+0000","last_fullsized":"2024-03-09T08:53:00.435050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"26'8","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:04.533728+0000","last_change":"2024-03-09T08:52:28.585562+0000","last_active":"2024-03-09T08:53:04.533728+0000","last_peered":"2024-03-09T08:53:04.533728+0000","last_clean":"2024-03-09T08:53:04.533728+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T08:53:04.533728+0000","last_undegraded":"2024-03-09T08:53:04.533728+0000","last_fullsized":"2024-03-09T08:53:04.533728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:28:50.544795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1870,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791730+0000","last_change":"2024-03-09T08:52:29.581330+0000","last_active":"2024-03-09T08:52:59.791730+0000","last_peered":"2024-03-09T08:52:59.791730+0000","last_clean":"2024-03-09T08:52:59.791730+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T08:52:59.791730+0000","last_undegraded":"2024-03-09T08:52:59.791730+0000","last_fullsized":"2024-03-09T08:52:59.791730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:55:38.041794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794362+0000","last_change":"2024-03-09T08:52:18.236146+0000","last_active":"2024-03-09T08:52:59.794362+0000","last_peered":"2024-03-09T08:52:59.794362+0000","last_clean":"2024-03-09T08:52:59.794362+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T08:52:59.794362+0000","last_undegraded":"2024-03-09T08:52:59.794362+0000","last_fullsized":"2024-03-09T08:52:59.794362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:18:51.511926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796997+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T08:52:59.796997+0000","last_peered":"2024-03-09T08:52:59.796997+0000","last_clean":"2024-03-09T08:52:59.796997+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T08:52:59.796997+0000","last_undegraded":"2024-03-09T08:52:59.796997+0000","last_fullsized":"2024-03-09T08:52:59.796997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:22:05.504557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435636+0000","last_change":"2024-03-09T08:52:28.585697+0000","last_active":"2024-03-09T08:53:00.435636+0000","last_peered":"2024-03-09T08:53:00.435636+0000","last_clean":"2024-03-09T08:53:00.435636+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T08:53:00.435636+0000","last_undegraded":"2024-03-09T08:53:00.435636+0000","last_fullsized":"2024-03-09T08:53:00.435636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:50:23.095008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790381+0000","last_change":"2024-03-09T08:52:29.578803+0000","last_active":"2024-03-09T08:52:59.790381+0000","last_peered":"2024-03-09T08:52:59.790381+0000","last_clean":"2024-03-09T08:52:59.790381+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T08:52:59.790381+0000","last_undegraded":"2024-03-09T08:52:59.790381+0000","last_fullsized":"2024-03-09T08:52:59.790381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:22:47.690873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237874+0000","last_change":"2024-03-09T08:52:18.242496+0000","last_active":"2024-03-09T08:53:00.237874+0000","last_peered":"2024-03-09T08:53:00.237874+0000","last_clean":"2024-03-09T08:53:00.237874+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T08:53:00.237874+0000","last_undegraded":"2024-03-09T08:53:00.237874+0000","last_fullsized":"2024-03-09T08:53:00.237874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:35:17.142983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":35,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793306+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T08:52:59.793306+0000","last_peered":"2024-03-09T08:52:59.793306+0000","last_clean":"2024-03-09T08:52:59.793306+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T08:52:59.793306+0000","last_undegraded":"2024-03-09T08:52:59.793306+0000","last_fullsized":"2024-03-09T08:52:59.793306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:01.724043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250057+0000","last_change":"2024-03-09T08:52:28.583685+0000","last_active":"2024-03-09T08:53:00.250057+0000","last_peered":"2024-03-09T08:53:00.250057+0000","last_clean":"2024-03-09T08:53:00.250057+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T08:53:00.250057+0000","last_undegraded":"2024-03-09T08:53:00.250057+0000","last_fullsized":"2024-03-09T08:53:00.250057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:58:00.899620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250427+0000","last_change":"2024-03-09T08:52:29.585912+0000","last_active":"2024-03-09T08:53:00.250427+0000","last_peered":"2024-03-09T08:53:00.250427+0000","last_clean":"2024-03-09T08:53:00.250427+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T08:53:00.250427+0000","last_undegraded":"2024-03-09T08:53:00.250427+0000","last_fullsized":"2024-03-09T08:53:00.250427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:24:47.261191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793994+0000","last_change":"2024-03-09T08:52:18.244654+0000","last_active":"2024-03-09T08:52:59.793994+0000","last_peered":"2024-03-09T08:52:59.793994+0000","last_clean":"2024-03-09T08:52:59.793994+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T08:52:59.793994+0000","last_undegraded":"2024-03-09T08:52:59.793994+0000","last_fullsized":"2024-03-09T08:52:59.793994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:08:58.611725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252327+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T08:53:00.252327+0000","last_peered":"2024-03-09T08:53:00.252327+0000","last_clean":"2024-03-09T08:53:00.252327+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T08:53:00.252327+0000","last_undegraded":"2024-03-09T08:53:00.252327+0000","last_fullsized":"2024-03-09T08:53:00.252327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:41:10.578669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792707+0000","last_change":"2024-03-09T08:52:28.568794+0000","last_active":"2024-03-09T08:52:59.792707+0000","last_peered":"2024-03-09T08:52:59.792707+0000","last_clean":"2024-03-09T08:52:59.792707+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T08:52:59.792707+0000","last_undegraded":"2024-03-09T08:52:59.792707+0000","last_fullsized":"2024-03-09T08:52:59.792707+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:44:21.721511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791693+0000","last_change":"2024-03-09T08:52:29.586705+0000","last_active":"2024-03-09T08:52:59.791693+0000","last_peered":"2024-03-09T08:52:59.791693+0000","last_clean":"2024-03-09T08:52:59.791693+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T08:52:59.791693+0000","last_undegraded":"2024-03-09T08:52:59.791693+0000","last_fullsized":"2024-03-09T08:52:59.791693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:26.353156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.255334+0000","last_change":"2024-03-09T08:53:00.255334+0000","last_active":"2024-03-09T08:53:00.255334+0000","last_peered":"2024-03-09T08:53:00.255334+0000","last_clean":"2024-03-09T08:53:00.255334+0000","last_became_active":"2024-03-09T08:53:00.254589+0000","last_became_peered":"2024-03-09T08:53:00.254589+0000","last_unstale":"2024-03-09T08:53:00.255334+0000","last_undegraded":"2024-03-09T08:53:00.255334+0000","last_fullsized":"2024-03-09T08:53:00.255334+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:30:00.005142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435997+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T08:53:00.435997+0000","last_peered":"2024-03-09T08:53:00.435997+0000","last_clean":"2024-03-09T08:53:00.435997+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T08:53:00.435997+0000","last_undegraded":"2024-03-09T08:53:00.435997+0000","last_fullsized":"2024-03-09T08:53:00.435997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789400+0000","last_change":"2024-03-09T08:52:28.583697+0000","last_active":"2024-03-09T08:52:59.789400+0000","last_peered":"2024-03-09T08:52:59.789400+0000","last_clean":"2024-03-09T08:52:59.789400+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T08:52:59.789400+0000","last_undegraded":"2024-03-09T08:52:59.789400+0000","last_fullsized":"2024-03-09T08:52:59.789400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:41.552156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236217+0000","last_change":"2024-03-09T08:52:29.590854+0000","last_active":"2024-03-09T08:53:00.236217+0000","last_peered":"2024-03-09T08:53:00.236217+0000","last_clean":"2024-03-09T08:53:00.236217+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T08:53:00.236217+0000","last_undegraded":"2024-03-09T08:53:00.236217+0000","last_fullsized":"2024-03-09T08:53:00.236217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:11:09.568826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435804+0000","last_change":"2024-03-09T08:52:18.251648+0000","last_active":"2024-03-09T08:53:00.435804+0000","last_peered":"2024-03-09T08:53:00.435804+0000","last_clean":"2024-03-09T08:53:00.435804+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T08:53:00.435804+0000","last_undegraded":"2024-03-09T08:53:00.435804+0000","last_fullsized":"2024-03-09T08:53:00.435804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:54:45.721673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795610+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T08:52:59.795610+0000","last_peered":"2024-03-09T08:52:59.795610+0000","last_clean":"2024-03-09T08:52:59.795610+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T08:52:59.795610+0000","last_undegraded":"2024-03-09T08:52:59.795610+0000","last_fullsized":"2024-03-09T08:52:59.795610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:31.731626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789940+0000","last_change":"2024-03-09T08:52:28.586606+0000","last_active":"2024-03-09T08:52:59.789940+0000","last_peered":"2024-03-09T08:52:59.789940+0000","last_clean":"2024-03-09T08:52:59.789940+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T08:52:59.789940+0000","last_undegraded":"2024-03-09T08:52:59.789940+0000","last_fullsized":"2024-03-09T08:52:59.789940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:17:40.902403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795143+0000","last_change":"2024-03-09T08:52:29.584512+0000","last_active":"2024-03-09T08:52:59.795143+0000","last_peered":"2024-03-09T08:52:59.795143+0000","last_clean":"2024-03-09T08:52:59.795143+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T08:52:59.795143+0000","last_undegraded":"2024-03-09T08:52:59.795143+0000","last_fullsized":"2024-03-09T08:52:59.795143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:39:20.330443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796012+0000","last_change":"2024-03-09T08:52:28.586201+0000","last_active":"2024-03-09T08:52:59.796012+0000","last_peered":"2024-03-09T08:52:59.796012+0000","last_clean":"2024-03-09T08:52:59.796012+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T08:52:59.796012+0000","last_undegraded":"2024-03-09T08:52:59.796012+0000","last_fullsized":"2024-03-09T08:52:59.796012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:28:03.471646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.249633+0000","last_change":"2024-03-09T08:52:18.241156+0000","last_active":"2024-03-09T08:53:00.249633+0000","last_peered":"2024-03-09T08:53:00.249633+0000","last_clean":"2024-03-09T08:53:00.249633+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T08:53:00.249633+0000","last_undegraded":"2024-03-09T08:53:00.249633+0000","last_fullsized":"2024-03-09T08:53:00.249633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:58:55.323770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434627+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T08:53:00.434627+0000","last_peered":"2024-03-09T08:53:00.434627+0000","last_clean":"2024-03-09T08:53:00.434627+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T08:53:00.434627+0000","last_undegraded":"2024-03-09T08:53:00.434627+0000","last_fullsized":"2024-03-09T08:53:00.434627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795848+0000","last_change":"2024-03-09T08:52:29.591085+0000","last_active":"2024-03-09T08:52:59.795848+0000","last_peered":"2024-03-09T08:52:59.795848+0000","last_clean":"2024-03-09T08:52:59.795848+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T08:52:59.795848+0000","last_undegraded":"2024-03-09T08:52:59.795848+0000","last_fullsized":"2024-03-09T08:52:59.795848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:40:27.878922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795316+0000","last_change":"2024-03-09T08:52:28.585631+0000","last_active":"2024-03-09T08:52:59.795316+0000","last_peered":"2024-03-09T08:52:59.795316+0000","last_clean":"2024-03-09T08:52:59.795316+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T08:52:59.795316+0000","last_undegraded":"2024-03-09T08:52:59.795316+0000","last_fullsized":"2024-03-09T08:52:59.795316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:06:03.569759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789727+0000","last_change":"2024-03-09T08:52:18.234400+0000","last_active":"2024-03-09T08:52:59.789727+0000","last_peered":"2024-03-09T08:52:59.789727+0000","last_clean":"2024-03-09T08:52:59.789727+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T08:52:59.789727+0000","last_undegraded":"2024-03-09T08:52:59.789727+0000","last_fullsized":"2024-03-09T08:52:59.789727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:42:14.057367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790714+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T08:52:59.790714+0000","last_peered":"2024-03-09T08:52:59.790714+0000","last_clean":"2024-03-09T08:52:59.790714+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T08:52:59.790714+0000","last_undegraded":"2024-03-09T08:52:59.790714+0000","last_fullsized":"2024-03-09T08:52:59.790714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251596+0000","last_change":"2024-03-09T08:52:29.588928+0000","last_active":"2024-03-09T08:53:00.251596+0000","last_peered":"2024-03-09T08:53:00.251596+0000","last_clean":"2024-03-09T08:53:00.251596+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T08:53:00.251596+0000","last_undegraded":"2024-03-09T08:53:00.251596+0000","last_fullsized":"2024-03-09T08:53:00.251596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:32:27.061041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793333+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T08:52:59.793333+0000","last_peered":"2024-03-09T08:52:59.793333+0000","last_clean":"2024-03-09T08:52:59.793333+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T08:52:59.793333+0000","last_undegraded":"2024-03-09T08:52:59.793333+0000","last_fullsized":"2024-03-09T08:52:59.793333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796845+0000","last_change":"2024-03-09T08:52:18.231141+0000","last_active":"2024-03-09T08:52:59.796845+0000","last_peered":"2024-03-09T08:52:59.796845+0000","last_clean":"2024-03-09T08:52:59.796845+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T08:52:59.796845+0000","last_undegraded":"2024-03-09T08:52:59.796845+0000","last_fullsized":"2024-03-09T08:52:59.796845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:25:00.471222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798738+0000","last_change":"2024-03-09T08:52:28.579315+0000","last_active":"2024-03-09T08:52:59.798738+0000","last_peered":"2024-03-09T08:52:59.798738+0000","last_clean":"2024-03-09T08:52:59.798738+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T08:52:59.798738+0000","last_undegraded":"2024-03-09T08:52:59.798738+0000","last_fullsized":"2024-03-09T08:52:59.798738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:57:17.301677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791987+0000","last_change":"2024-03-09T08:52:29.585154+0000","last_active":"2024-03-09T08:52:59.791987+0000","last_peered":"2024-03-09T08:52:59.791987+0000","last_clean":"2024-03-09T08:52:59.791987+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T08:52:59.791987+0000","last_undegraded":"2024-03-09T08:52:59.791987+0000","last_fullsized":"2024-03-09T08:52:59.791987+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:43:35.679108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791033+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T08:52:59.791033+0000","last_peered":"2024-03-09T08:52:59.791033+0000","last_clean":"2024-03-09T08:52:59.791033+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T08:52:59.791033+0000","last_undegraded":"2024-03-09T08:52:59.791033+0000","last_fullsized":"2024-03-09T08:52:59.791033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792089+0000","last_change":"2024-03-09T08:52:18.229533+0000","last_active":"2024-03-09T08:52:59.792089+0000","last_peered":"2024-03-09T08:52:59.792089+0000","last_clean":"2024-03-09T08:52:59.792089+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T08:52:59.792089+0000","last_undegraded":"2024-03-09T08:52:59.792089+0000","last_fullsized":"2024-03-09T08:52:59.792089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:17.241048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791797+0000","last_change":"2024-03-09T08:52:28.585877+0000","last_active":"2024-03-09T08:52:59.791797+0000","last_peered":"2024-03-09T08:52:59.791797+0000","last_clean":"2024-03-09T08:52:59.791797+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T08:52:59.791797+0000","last_undegraded":"2024-03-09T08:52:59.791797+0000","last_fullsized":"2024-03-09T08:52:59.791797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:30:20.036385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792761+0000","last_change":"2024-03-09T08:52:29.592253+0000","last_active":"2024-03-09T08:52:59.792761+0000","last_peered":"2024-03-09T08:52:59.792761+0000","last_clean":"2024-03-09T08:52:59.792761+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T08:52:59.792761+0000","last_undegraded":"2024-03-09T08:52:59.792761+0000","last_fullsized":"2024-03-09T08:52:59.792761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:50:33.525541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789659+0000","last_change":"2024-03-09T08:52:18.237869+0000","last_active":"2024-03-09T08:52:59.789659+0000","last_peered":"2024-03-09T08:52:59.789659+0000","last_clean":"2024-03-09T08:52:59.789659+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T08:52:59.789659+0000","last_undegraded":"2024-03-09T08:52:59.789659+0000","last_fullsized":"2024-03-09T08:52:59.789659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:01.571866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251959+0000","last_change":"2024-03-09T08:52:29.601300+0000","last_active":"2024-03-09T08:53:00.251959+0000","last_peered":"2024-03-09T08:53:00.251959+0000","last_clean":"2024-03-09T08:53:00.251959+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T08:53:00.251959+0000","last_undegraded":"2024-03-09T08:53:00.251959+0000","last_fullsized":"2024-03-09T08:53:00.251959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:46:04.870574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.797859+0000","last_change":"2024-03-09T08:52:18.226980+0000","last_active":"2024-03-09T08:52:59.797859+0000","last_peered":"2024-03-09T08:52:59.797859+0000","last_clean":"2024-03-09T08:52:59.797859+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T08:52:59.797859+0000","last_undegraded":"2024-03-09T08:52:59.797859+0000","last_fullsized":"2024-03-09T08:52:59.797859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:45:28.312576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434867+0000","last_change":"2024-03-09T08:52:29.575612+0000","last_active":"2024-03-09T08:53:00.434867+0000","last_peered":"2024-03-09T08:53:00.434867+0000","last_clean":"2024-03-09T08:53:00.434867+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T08:53:00.434867+0000","last_undegraded":"2024-03-09T08:53:00.434867+0000","last_fullsized":"2024-03-09T08:53:00.434867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:58.852983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434982+0000","last_change":"2024-03-09T08:52:18.241628+0000","last_active":"2024-03-09T08:53:00.434982+0000","last_peered":"2024-03-09T08:53:00.434982+0000","last_clean":"2024-03-09T08:53:00.434982+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T08:53:00.434982+0000","last_undegraded":"2024-03-09T08:53:00.434982+0000","last_fullsized":"2024-03-09T08:53:00.434982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:07:58.570641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252835+0000","last_change":"2024-03-09T08:52:29.588660+0000","last_active":"2024-03-09T08:53:00.252835+0000","last_peered":"2024-03-09T08:53:00.252835+0000","last_clean":"2024-03-09T08:53:00.252835+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T08:53:00.252835+0000","last_undegraded":"2024-03-09T08:53:00.252835+0000","last_fullsized":"2024-03-09T08:53:00.252835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:14.305320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.796333+0000","last_change":"2024-03-09T08:52:18.242257+0000","last_active":"2024-03-09T08:52:59.796333+0000","last_peered":"2024-03-09T08:52:59.796333+0000","last_clean":"2024-03-09T08:52:59.796333+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T08:52:59.796333+0000","last_undegraded":"2024-03-09T08:52:59.796333+0000","last_fullsized":"2024-03-09T08:52:59.796333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:40:00.854066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236632+0000","last_change":"2024-03-09T08:52:29.595057+0000","last_active":"2024-03-09T08:53:00.236632+0000","last_peered":"2024-03-09T08:53:00.236632+0000","last_clean":"2024-03-09T08:53:00.236632+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T08:53:00.236632+0000","last_undegraded":"2024-03-09T08:53:00.236632+0000","last_fullsized":"2024-03-09T08:53:00.236632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:33:37.474266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792473+0000","last_change":"2024-03-09T08:52:18.239951+0000","last_active":"2024-03-09T08:52:59.792473+0000","last_peered":"2024-03-09T08:52:59.792473+0000","last_clean":"2024-03-09T08:52:59.792473+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T08:52:59.792473+0000","last_undegraded":"2024-03-09T08:52:59.792473+0000","last_fullsized":"2024-03-09T08:52:59.792473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:51.102540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792106+0000","last_change":"2024-03-09T08:52:29.578993+0000","last_active":"2024-03-09T08:52:59.792106+0000","last_peered":"2024-03-09T08:52:59.792106+0000","last_clean":"2024-03-09T08:52:59.792106+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T08:52:59.792106+0000","last_undegraded":"2024-03-09T08:52:59.792106+0000","last_fullsized":"2024-03-09T08:52:59.792106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:50:53.692060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.252030+0000","last_change":"2024-03-09T08:52:18.241038+0000","last_active":"2024-03-09T08:53:00.252030+0000","last_peered":"2024-03-09T08:53:00.252030+0000","last_clean":"2024-03-09T08:53:00.252030+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T08:53:00.252030+0000","last_undegraded":"2024-03-09T08:53:00.252030+0000","last_fullsized":"2024-03-09T08:53:00.252030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:02.169113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236170+0000","last_change":"2024-03-09T08:52:29.583869+0000","last_active":"2024-03-09T08:53:00.236170+0000","last_peered":"2024-03-09T08:53:00.236170+0000","last_clean":"2024-03-09T08:53:00.236170+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T08:53:00.236170+0000","last_undegraded":"2024-03-09T08:53:00.236170+0000","last_fullsized":"2024-03-09T08:53:00.236170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:29:47.636741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794839+0000","last_change":"2024-03-09T08:52:18.247145+0000","last_active":"2024-03-09T08:52:59.794839+0000","last_peered":"2024-03-09T08:52:59.794839+0000","last_clean":"2024-03-09T08:52:59.794839+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T08:52:59.794839+0000","last_undegraded":"2024-03-09T08:52:59.794839+0000","last_fullsized":"2024-03-09T08:52:59.794839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:46:24.111224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791477+0000","last_change":"2024-03-09T08:52:29.587358+0000","last_active":"2024-03-09T08:52:59.791477+0000","last_peered":"2024-03-09T08:52:59.791477+0000","last_clean":"2024-03-09T08:52:59.791477+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T08:52:59.791477+0000","last_undegraded":"2024-03-09T08:52:59.791477+0000","last_fullsized":"2024-03-09T08:52:59.791477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:46.836801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792891+0000","last_change":"2024-03-09T08:52:18.247784+0000","last_active":"2024-03-09T08:52:59.792891+0000","last_peered":"2024-03-09T08:52:59.792891+0000","last_clean":"2024-03-09T08:52:59.792891+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T08:52:59.792891+0000","last_undegraded":"2024-03-09T08:52:59.792891+0000","last_fullsized":"2024-03-09T08:52:59.792891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:46:59.008220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792459+0000","last_change":"2024-03-09T08:52:29.589458+0000","last_active":"2024-03-09T08:52:59.792459+0000","last_peered":"2024-03-09T08:52:59.792459+0000","last_clean":"2024-03-09T08:52:59.792459+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T08:52:59.792459+0000","last_undegraded":"2024-03-09T08:52:59.792459+0000","last_fullsized":"2024-03-09T08:52:59.792459+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:19:35.222184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434277+0000","last_change":"2024-03-09T08:52:18.239943+0000","last_active":"2024-03-09T08:53:00.434277+0000","last_peered":"2024-03-09T08:53:00.434277+0000","last_clean":"2024-03-09T08:53:00.434277+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T08:53:00.434277+0000","last_undegraded":"2024-03-09T08:53:00.434277+0000","last_fullsized":"2024-03-09T08:53:00.434277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:18.727030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.239021+0000","last_change":"2024-03-09T08:52:29.597147+0000","last_active":"2024-03-09T08:53:00.239021+0000","last_peered":"2024-03-09T08:53:00.239021+0000","last_clean":"2024-03-09T08:53:00.239021+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T08:53:00.239021+0000","last_undegraded":"2024-03-09T08:53:00.239021+0000","last_fullsized":"2024-03-09T08:53:00.239021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:44:47.144911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790530+0000","last_change":"2024-03-09T08:52:18.236642+0000","last_active":"2024-03-09T08:52:59.790530+0000","last_peered":"2024-03-09T08:52:59.790530+0000","last_clean":"2024-03-09T08:52:59.790530+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T08:52:59.790530+0000","last_undegraded":"2024-03-09T08:52:59.790530+0000","last_fullsized":"2024-03-09T08:52:59.790530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:51:46.825153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798693+0000","last_change":"2024-03-09T08:52:29.590810+0000","last_active":"2024-03-09T08:52:59.798693+0000","last_peered":"2024-03-09T08:52:59.798693+0000","last_clean":"2024-03-09T08:52:59.798693+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T08:52:59.798693+0000","last_undegraded":"2024-03-09T08:52:59.798693+0000","last_fullsized":"2024-03-09T08:52:59.798693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:34:21.534580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434797+0000","last_change":"2024-03-09T08:52:18.236855+0000","last_active":"2024-03-09T08:53:00.434797+0000","last_peered":"2024-03-09T08:53:00.434797+0000","last_clean":"2024-03-09T08:53:00.434797+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T08:53:00.434797+0000","last_undegraded":"2024-03-09T08:53:00.434797+0000","last_fullsized":"2024-03-09T08:53:00.434797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:20:21.941242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791664+0000","last_change":"2024-03-09T08:52:29.602209+0000","last_active":"2024-03-09T08:52:59.791664+0000","last_peered":"2024-03-09T08:52:59.791664+0000","last_clean":"2024-03-09T08:52:59.791664+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T08:52:59.791664+0000","last_undegraded":"2024-03-09T08:52:59.791664+0000","last_fullsized":"2024-03-09T08:52:59.791664+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:11:33.349464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795388+0000","last_change":"2024-03-09T08:52:18.238373+0000","last_active":"2024-03-09T08:52:59.795388+0000","last_peered":"2024-03-09T08:52:59.795388+0000","last_clean":"2024-03-09T08:52:59.795388+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T08:52:59.795388+0000","last_undegraded":"2024-03-09T08:52:59.795388+0000","last_fullsized":"2024-03-09T08:52:59.795388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:05:31.480335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791332+0000","last_change":"2024-03-09T08:52:29.578120+0000","last_active":"2024-03-09T08:52:59.791332+0000","last_peered":"2024-03-09T08:52:59.791332+0000","last_clean":"2024-03-09T08:52:59.791332+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T08:52:59.791332+0000","last_undegraded":"2024-03-09T08:52:59.791332+0000","last_fullsized":"2024-03-09T08:52:59.791332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:13:27.623995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793083+0000","last_change":"2024-03-09T08:52:18.235565+0000","last_active":"2024-03-09T08:52:59.793083+0000","last_peered":"2024-03-09T08:52:59.793083+0000","last_clean":"2024-03-09T08:52:59.793083+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T08:52:59.793083+0000","last_undegraded":"2024-03-09T08:52:59.793083+0000","last_fullsized":"2024-03-09T08:52:59.793083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:52:14.994539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791813+0000","last_change":"2024-03-09T08:52:29.579668+0000","last_active":"2024-03-09T08:52:59.791813+0000","last_peered":"2024-03-09T08:52:59.791813+0000","last_clean":"2024-03-09T08:52:59.791813+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T08:52:59.791813+0000","last_undegraded":"2024-03-09T08:52:59.791813+0000","last_fullsized":"2024-03-09T08:52:59.791813+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:12:15.272875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238341+0000","last_change":"2024-03-09T08:52:18.239801+0000","last_active":"2024-03-09T08:53:00.238341+0000","last_peered":"2024-03-09T08:53:00.238341+0000","last_clean":"2024-03-09T08:53:00.238341+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T08:53:00.238341+0000","last_undegraded":"2024-03-09T08:53:00.238341+0000","last_fullsized":"2024-03-09T08:53:00.238341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:38:06.836990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.433665+0000","last_change":"2024-03-09T08:52:29.586986+0000","last_active":"2024-03-09T08:53:00.433665+0000","last_peered":"2024-03-09T08:53:00.433665+0000","last_clean":"2024-03-09T08:53:00.433665+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T08:53:00.433665+0000","last_undegraded":"2024-03-09T08:53:00.433665+0000","last_fullsized":"2024-03-09T08:53:00.433665+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:37.382414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.801226+0000","last_change":"2024-03-09T08:52:18.233582+0000","last_active":"2024-03-09T08:52:59.801226+0000","last_peered":"2024-03-09T08:52:59.801226+0000","last_clean":"2024-03-09T08:52:59.801226+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T08:52:59.801226+0000","last_undegraded":"2024-03-09T08:52:59.801226+0000","last_fullsized":"2024-03-09T08:52:59.801226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:49.313616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793385+0000","last_change":"2024-03-09T08:52:29.586673+0000","last_active":"2024-03-09T08:52:59.793385+0000","last_peered":"2024-03-09T08:52:59.793385+0000","last_clean":"2024-03-09T08:52:59.793385+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T08:52:59.793385+0000","last_undegraded":"2024-03-09T08:52:59.793385+0000","last_fullsized":"2024-03-09T08:52:59.793385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:23:49.495003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.243414+0000","last_change":"2024-03-09T08:53:00.241216+0000","last_active":"2024-03-09T08:53:00.243414+0000","last_peered":"2024-03-09T08:53:00.243414+0000","last_clean":"2024-03-09T08:53:00.243414+0000","last_became_active":"2024-03-09T08:53:00.240319+0000","last_became_peered":"2024-03-09T08:53:00.240319+0000","last_unstale":"2024-03-09T08:53:00.243414+0000","last_undegraded":"2024-03-09T08:53:00.243414+0000","last_fullsized":"2024-03-09T08:53:00.243414+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:32:34.111501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251554+0000","last_change":"2024-03-09T08:52:29.592702+0000","last_active":"2024-03-09T08:53:00.251554+0000","last_peered":"2024-03-09T08:53:00.251554+0000","last_clean":"2024-03-09T08:53:00.251554+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T08:53:00.251554+0000","last_undegraded":"2024-03-09T08:53:00.251554+0000","last_fullsized":"2024-03-09T08:53:00.251554+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:39.344726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434012+0000","last_change":"2024-03-09T08:52:18.230289+0000","last_active":"2024-03-09T08:53:00.434012+0000","last_peered":"2024-03-09T08:53:00.434012+0000","last_clean":"2024-03-09T08:53:00.434012+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T08:53:00.434012+0000","last_undegraded":"2024-03-09T08:53:00.434012+0000","last_fullsized":"2024-03-09T08:53:00.434012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:34.507924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794831+0000","last_change":"2024-03-09T08:52:29.608191+0000","last_active":"2024-03-09T08:52:59.794831+0000","last_peered":"2024-03-09T08:52:59.794831+0000","last_clean":"2024-03-09T08:52:59.794831+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T08:52:59.794831+0000","last_undegraded":"2024-03-09T08:52:59.794831+0000","last_fullsized":"2024-03-09T08:52:59.794831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:26:32.282196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.249757+0000","last_change":"2024-03-09T08:52:18.236376+0000","last_active":"2024-03-09T08:53:00.249757+0000","last_peered":"2024-03-09T08:53:00.249757+0000","last_clean":"2024-03-09T08:53:00.249757+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T08:53:00.249757+0000","last_undegraded":"2024-03-09T08:53:00.249757+0000","last_fullsized":"2024-03-09T08:53:00.249757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:34:04.412643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434645+0000","last_change":"2024-03-09T08:52:29.573685+0000","last_active":"2024-03-09T08:53:00.434645+0000","last_peered":"2024-03-09T08:53:00.434645+0000","last_clean":"2024-03-09T08:53:00.434645+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T08:53:00.434645+0000","last_undegraded":"2024-03-09T08:53:00.434645+0000","last_fullsized":"2024-03-09T08:53:00.434645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:41:41.839063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794192+0000","last_change":"2024-03-09T08:52:18.231316+0000","last_active":"2024-03-09T08:52:59.794192+0000","last_peered":"2024-03-09T08:52:59.794192+0000","last_clean":"2024-03-09T08:52:59.794192+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T08:52:59.794192+0000","last_undegraded":"2024-03-09T08:52:59.794192+0000","last_fullsized":"2024-03-09T08:52:59.794192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:25:25.715084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237839+0000","last_change":"2024-03-09T08:52:29.603272+0000","last_active":"2024-03-09T08:53:00.237839+0000","last_peered":"2024-03-09T08:53:00.237839+0000","last_clean":"2024-03-09T08:53:00.237839+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T08:53:00.237839+0000","last_undegraded":"2024-03-09T08:53:00.237839+0000","last_fullsized":"2024-03-09T08:53:00.237839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:07:58.017672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790491+0000","last_change":"2024-03-09T08:52:18.243645+0000","last_active":"2024-03-09T08:52:59.790491+0000","last_peered":"2024-03-09T08:52:59.790491+0000","last_clean":"2024-03-09T08:52:59.790491+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T08:52:59.790491+0000","last_undegraded":"2024-03-09T08:52:59.790491+0000","last_fullsized":"2024-03-09T08:52:59.790491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:19.501675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.791144+0000","last_change":"2024-03-09T08:52:29.587462+0000","last_active":"2024-03-09T08:52:59.791144+0000","last_peered":"2024-03-09T08:52:59.791144+0000","last_clean":"2024-03-09T08:52:59.791144+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T08:52:59.791144+0000","last_undegraded":"2024-03-09T08:52:59.791144+0000","last_fullsized":"2024-03-09T08:52:59.791144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:43:51.652225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.793895+0000","last_change":"2024-03-09T08:52:18.233451+0000","last_active":"2024-03-09T08:52:59.793895+0000","last_peered":"2024-03-09T08:52:59.793895+0000","last_clean":"2024-03-09T08:52:59.793895+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T08:52:59.793895+0000","last_undegraded":"2024-03-09T08:52:59.793895+0000","last_fullsized":"2024-03-09T08:52:59.793895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:56:11.270081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798239+0000","last_change":"2024-03-09T08:52:29.599909+0000","last_active":"2024-03-09T08:52:59.798239+0000","last_peered":"2024-03-09T08:52:59.798239+0000","last_clean":"2024-03-09T08:52:59.798239+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T08:52:59.798239+0000","last_undegraded":"2024-03-09T08:52:59.798239+0000","last_fullsized":"2024-03-09T08:52:59.798239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:42.596067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794296+0000","last_change":"2024-03-09T08:52:18.234158+0000","last_active":"2024-03-09T08:52:59.794296+0000","last_peered":"2024-03-09T08:52:59.794296+0000","last_clean":"2024-03-09T08:52:59.794296+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T08:52:59.794296+0000","last_undegraded":"2024-03-09T08:52:59.794296+0000","last_fullsized":"2024-03-09T08:52:59.794296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:59:33.423558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237041+0000","last_change":"2024-03-09T08:52:29.597518+0000","last_active":"2024-03-09T08:53:00.237041+0000","last_peered":"2024-03-09T08:53:00.237041+0000","last_clean":"2024-03-09T08:53:00.237041+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T08:53:00.237041+0000","last_undegraded":"2024-03-09T08:53:00.237041+0000","last_fullsized":"2024-03-09T08:53:00.237041+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:32:32.452993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794838+0000","last_change":"2024-03-09T08:52:18.234648+0000","last_active":"2024-03-09T08:52:59.794838+0000","last_peered":"2024-03-09T08:52:59.794838+0000","last_clean":"2024-03-09T08:52:59.794838+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T08:52:59.794838+0000","last_undegraded":"2024-03-09T08:52:59.794838+0000","last_fullsized":"2024-03-09T08:52:59.794838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:45:03.927280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434105+0000","last_change":"2024-03-09T08:52:29.580608+0000","last_active":"2024-03-09T08:53:00.434105+0000","last_peered":"2024-03-09T08:53:00.434105+0000","last_clean":"2024-03-09T08:53:00.434105+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T08:53:00.434105+0000","last_undegraded":"2024-03-09T08:53:00.434105+0000","last_fullsized":"2024-03-09T08:53:00.434105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:00:37.184351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238778+0000","last_change":"2024-03-09T08:52:18.253038+0000","last_active":"2024-03-09T08:53:00.238778+0000","last_peered":"2024-03-09T08:53:00.238778+0000","last_clean":"2024-03-09T08:53:00.238778+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T08:53:00.238778+0000","last_undegraded":"2024-03-09T08:53:00.238778+0000","last_fullsized":"2024-03-09T08:53:00.238778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:26:39.137498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.792413+0000","last_change":"2024-03-09T08:52:29.578549+0000","last_active":"2024-03-09T08:52:59.792413+0000","last_peered":"2024-03-09T08:52:59.792413+0000","last_clean":"2024-03-09T08:52:59.792413+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T08:52:59.792413+0000","last_undegraded":"2024-03-09T08:52:59.792413+0000","last_fullsized":"2024-03-09T08:52:59.792413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:42.775199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.789574+0000","last_change":"2024-03-09T08:52:18.240975+0000","last_active":"2024-03-09T08:52:59.789574+0000","last_peered":"2024-03-09T08:52:59.789574+0000","last_clean":"2024-03-09T08:52:59.789574+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T08:52:59.789574+0000","last_undegraded":"2024-03-09T08:52:59.789574+0000","last_fullsized":"2024-03-09T08:52:59.789574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:30:09.363339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.794998+0000","last_change":"2024-03-09T08:52:29.584159+0000","last_active":"2024-03-09T08:52:59.794998+0000","last_peered":"2024-03-09T08:52:59.794998+0000","last_clean":"2024-03-09T08:52:59.794998+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T08:52:59.794998+0000","last_undegraded":"2024-03-09T08:52:59.794998+0000","last_fullsized":"2024-03-09T08:52:59.794998+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:01:41.661806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236628+0000","last_change":"2024-03-09T08:52:18.249901+0000","last_active":"2024-03-09T08:53:00.236628+0000","last_peered":"2024-03-09T08:53:00.236628+0000","last_clean":"2024-03-09T08:53:00.236628+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T08:53:00.236628+0000","last_undegraded":"2024-03-09T08:53:00.236628+0000","last_fullsized":"2024-03-09T08:53:00.236628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:59:09.947401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.250862+0000","last_change":"2024-03-09T08:52:29.595184+0000","last_active":"2024-03-09T08:53:00.250862+0000","last_peered":"2024-03-09T08:53:00.250862+0000","last_clean":"2024-03-09T08:53:00.250862+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T08:53:00.250862+0000","last_undegraded":"2024-03-09T08:53:00.250862+0000","last_fullsized":"2024-03-09T08:53:00.250862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:34:27.662636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.434708+0000","last_change":"2024-03-09T08:52:18.233561+0000","last_active":"2024-03-09T08:53:00.434708+0000","last_peered":"2024-03-09T08:53:00.434708+0000","last_clean":"2024-03-09T08:53:00.434708+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T08:53:00.434708+0000","last_undegraded":"2024-03-09T08:53:00.434708+0000","last_fullsized":"2024-03-09T08:53:00.434708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:15:54.027155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435347+0000","last_change":"2024-03-09T08:52:29.595771+0000","last_active":"2024-03-09T08:53:00.435347+0000","last_peered":"2024-03-09T08:53:00.435347+0000","last_clean":"2024-03-09T08:53:00.435347+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T08:53:00.435347+0000","last_undegraded":"2024-03-09T08:53:00.435347+0000","last_fullsized":"2024-03-09T08:53:00.435347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:23:48.898071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.237460+0000","last_change":"2024-03-09T08:52:18.242311+0000","last_active":"2024-03-09T08:53:00.237460+0000","last_peered":"2024-03-09T08:53:00.237460+0000","last_clean":"2024-03-09T08:53:00.237460+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T08:53:00.237460+0000","last_undegraded":"2024-03-09T08:53:00.237460+0000","last_fullsized":"2024-03-09T08:53:00.237460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:48:15.559438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.790367+0000","last_change":"2024-03-09T08:52:29.576792+0000","last_active":"2024-03-09T08:52:59.790367+0000","last_peered":"2024-03-09T08:52:59.790367+0000","last_clean":"2024-03-09T08:52:59.790367+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T08:52:59.790367+0000","last_undegraded":"2024-03-09T08:52:59.790367+0000","last_fullsized":"2024-03-09T08:52:59.790367+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:36:21.364814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.435606+0000","last_change":"2024-03-09T08:52:18.248341+0000","last_active":"2024-03-09T08:53:00.435606+0000","last_peered":"2024-03-09T08:53:00.435606+0000","last_clean":"2024-03-09T08:53:00.435606+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T08:53:00.435606+0000","last_undegraded":"2024-03-09T08:53:00.435606+0000","last_fullsized":"2024-03-09T08:53:00.435606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:18:02.716066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.798091+0000","last_change":"2024-03-09T08:52:29.606330+0000","last_active":"2024-03-09T08:52:59.798091+0000","last_peered":"2024-03-09T08:52:59.798091+0000","last_clean":"2024-03-09T08:52:59.798091+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T08:52:59.798091+0000","last_undegraded":"2024-03-09T08:52:59.798091+0000","last_fullsized":"2024-03-09T08:52:59.798091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:44:50.347124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:52:59.795661+0000","last_change":"2024-03-09T08:52:18.222968+0000","last_active":"2024-03-09T08:52:59.795661+0000","last_peered":"2024-03-09T08:52:59.795661+0000","last_clean":"2024-03-09T08:52:59.795661+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T08:52:59.795661+0000","last_undegraded":"2024-03-09T08:52:59.795661+0000","last_fullsized":"2024-03-09T08:52:59.795661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:52:29.523963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236013+0000","last_change":"2024-03-09T08:52:29.591813+0000","last_active":"2024-03-09T08:53:00.236013+0000","last_peered":"2024-03-09T08:53:00.236013+0000","last_clean":"2024-03-09T08:53:00.236013+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T08:53:00.236013+0000","last_undegraded":"2024-03-09T08:53:00.236013+0000","last_fullsized":"2024-03-09T08:53:00.236013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:40:31.568253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.251999+0000","last_change":"2024-03-09T08:52:18.225812+0000","last_active":"2024-03-09T08:53:00.251999+0000","last_peered":"2024-03-09T08:53:00.251999+0000","last_clean":"2024-03-09T08:53:00.251999+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T08:53:00.251999+0000","last_undegraded":"2024-03-09T08:53:00.251999+0000","last_fullsized":"2024-03-09T08:53:00.251999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:09:32.044185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.236176+0000","last_change":"2024-03-09T08:52:29.595055+0000","last_active":"2024-03-09T08:53:00.236176+0000","last_peered":"2024-03-09T08:53:00.236176+0000","last_clean":"2024-03-09T08:53:00.236176+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T08:53:00.236176+0000","last_undegraded":"2024-03-09T08:53:00.236176+0000","last_fullsized":"2024-03-09T08:53:00.236176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:53.209169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-09T08:53:00.238706+0000","last_change":"2024-03-09T08:52:18.248973+0000","last_active":"2024-03-09T08:53:00.238706+0000","last_peered":"2024-03-09T08:53:00.238706+0000","last_clean":"2024-03-09T08:53:00.238706+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T08:53:00.238706+0000","last_undegraded":"2024-03-09T08:53:00.238706+0000","last_fullsized":"2024-03-09T08:53:00.238706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:09:11.372616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2668,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":704107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":116540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":113829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":113297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":307200,"data_stored":117816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":113137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":115119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":884736,"data_stored":704073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T08:53:09.026 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-03-09T08:53:09.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T08:53:09.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.264+0000 7f6dacc00640 1 Processor -- start 2024-03-09T08:53:09.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.264+0000 7f6dacc00640 1 -- start start 2024-03-09T08:53:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.265+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6da80c1f70 0x7f6da8150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.265+0000 7f6dacc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6da8154080 0x7f6da8151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.265+0000 7f6dacc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da80c81e0 con 0x7f6da80710a0 2024-03-09T08:53:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.265+0000 7f6dacc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da80c32b0 con 0x7f6da8154080 2024-03-09T08:53:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.265+0000 7f6dacc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da80c2fd0 con 0x7f6da80c1f70 2024-03-09T08:53:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.266+0000 7f6da7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6da8154080 0x7f6da8151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.266+0000 7f6da7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6da8154080 0x7f6da8151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60136/0 (socket says 172.21.15.67:60136) 2024-03-09T08:53:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.266+0000 7f6da7e00640 1 -- 172.21.15.67:0/1153779568 learned_addr learned my addr 172.21.15.67:0/1153779568 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.266+0000 7f6da7400640 1 --2- 172.21.15.67:0/1153779568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6da80c1f70 0x7f6da8150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.266+0000 7f6da7e00640 1 -- 172.21.15.67:0/1153779568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6da80c1f70 msgr2=0x7f6da8150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.266+0000 7f6da7e00640 1 --2- 172.21.15.67:0/1153779568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6da80c1f70 0x7f6da8150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.266+0000 7f6da7e00640 1 -- 172.21.15.67:0/1153779568 >> v1:172.21.15.134:6789/0 conn(0x7f6da80710a0 legacy=0x7f6da8153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.266+0000 7f6da7e00640 1 -- 172.21.15.67:0/1153779568 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da80c3700 con 0x7f6da8154080 2024-03-09T08:53:09.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.268+0000 7f6da7e00640 1 --2- 172.21.15.67:0/1153779568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da8154080 0x7f6da8151190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009e50 tx=0x7f6da002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47de3f5d00ac3833 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:09.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.268+0000 7f6da6000640 1 -- 172.21.15.67:0/1153779568 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da002c9e0 con 0x7f6da8154080 2024-03-09T08:53:09.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.268+0000 7f6da6000640 1 -- 172.21.15.67:0/1153779568 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da002cb80 con 0x7f6da8154080 2024-03-09T08:53:09.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.268+0000 7f6da6000640 1 -- 172.21.15.67:0/1153779568 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da00343f0 con 0x7f6da8154080 2024-03-09T08:53:09.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.269+0000 7f6dacc00640 1 -- 172.21.15.67:0/1153779568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da8154080 msgr2=0x7f6da8151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.269+0000 7f6dacc00640 1 --2- 172.21.15.67:0/1153779568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da8154080 0x7f6da8151190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009e50 tx=0x7f6da002be40 comp rx=0 tx=0).stop 2024-03-09T08:53:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.269+0000 7f6dacc00640 1 -- 172.21.15.67:0/1153779568 shutdown_connections 2024-03-09T08:53:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.269+0000 7f6dacc00640 1 --2- 172.21.15.67:0/1153779568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da8154080 0x7f6da8151190 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.269+0000 7f6dacc00640 1 --2- 172.21.15.67:0/1153779568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6da80c1f70 0x7f6da8150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.269+0000 7f6dacc00640 1 -- 172.21.15.67:0/1153779568 >> 172.21.15.67:0/1153779568 conn(0x7f6da80c7300 msgr2=0x7f6da80c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.269+0000 7f6dacc00640 1 -- 172.21.15.67:0/1153779568 shutdown_connections 2024-03-09T08:53:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.269+0000 7f6dacc00640 1 -- 172.21.15.67:0/1153779568 wait complete. 2024-03-09T08:53:09.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.271+0000 7f6dacc00640 1 Processor -- start 2024-03-09T08:53:09.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.271+0000 7f6dacc00640 1 -- start start 2024-03-09T08:53:09.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.272+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da80710a0 0x7f6da8151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.272+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6da80c1f70 0x7f6da814bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.272+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6da8154080 0x7f6da81eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.272+0000 7f6dacc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8070d50 con 0x7f6da80710a0 2024-03-09T08:53:09.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.272+0000 7f6dacc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8070830 con 0x7f6da80c1f70 2024-03-09T08:53:09.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.272+0000 7f6dacc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8070ed0 con 0x7f6da8154080 2024-03-09T08:53:09.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.272+0000 7f6da7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da80710a0 0x7f6da8151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.272+0000 7f6da7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da80710a0 0x7f6da8151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60138/0 (socket says 172.21.15.67:60138) 2024-03-09T08:53:09.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.272+0000 7f6da7400640 1 -- 172.21.15.67:0/1427288563 learned_addr learned my addr 172.21.15.67:0/1427288563 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.273+0000 7f6da6a00640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6da80c1f70 0x7f6da814bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.273+0000 7f6da7e00640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6da8154080 0x7f6da81eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.273+0000 7f6da7400640 1 -- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6da8154080 msgr2=0x7f6da81eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.273+0000 7f6da7400640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6da8154080 0x7f6da81eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.273+0000 7f6da7400640 1 -- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6da80c1f70 msgr2=0x7f6da814bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.273+0000 7f6da7400640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6da80c1f70 0x7f6da814bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.273+0000 7f6da7400640 1 -- 172.21.15.67:0/1427288563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da8148e80 con 0x7f6da80710a0 2024-03-09T08:53:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.273+0000 7f6da7e00640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6da8154080 0x7f6da81eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.273+0000 7f6da7400640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da80710a0 0x7f6da8151b30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800d620 tx=0x7f6d9800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.273+0000 7f6d9f400640 1 -- 172.21.15.67:0/1427288563 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9800b620 con 0x7f6da80710a0 2024-03-09T08:53:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.274+0000 7f6d9f400640 1 -- 172.21.15.67:0/1427288563 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d98019070 con 0x7f6da80710a0 2024-03-09T08:53:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.274+0000 7f6d9f400640 1 -- 172.21.15.67:0/1427288563 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9800ba00 con 0x7f6da80710a0 2024-03-09T08:53:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.274+0000 7f6dacc00640 1 -- 172.21.15.67:0/1427288563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da8148780 con 0x7f6da80710a0 2024-03-09T08:53:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.274+0000 7f6dacc00640 1 -- 172.21.15.67:0/1427288563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da8148b80 con 0x7f6da80710a0 2024-03-09T08:53:09.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.275+0000 7f6dacc00640 1 -- 172.21.15.67:0/1427288563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da804f3c0 con 0x7f6da80710a0 2024-03-09T08:53:09.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.276+0000 7f6d9f400640 1 -- 172.21.15.67:0/1427288563 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6d9800bba0 con 0x7f6da80710a0 2024-03-09T08:53:09.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.277+0000 7f6d9f400640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6d8806b4d0 0x7f6d8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.277+0000 7f6d9f400640 1 -- 172.21.15.67:0/1427288563 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f6d9808aa20 con 0x7f6da80710a0 2024-03-09T08:53:09.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.277+0000 7f6da6a00640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6d8806b4d0 0x7f6d8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.279+0000 7f6da6a00640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6d8806b4d0 0x7f6d8806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d94006fd0 tx=0x7f6d94008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:09.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.281+0000 7f6d9f400640 1 -- 172.21.15.67:0/1427288563 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6d980594f0 con 0x7f6da80710a0 2024-03-09T08:53:09.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.488+0000 7f6dacc00640 1 -- 172.21.15.67:0/1427288563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6da8148950 con 0x7f6da80710a0 2024-03-09T08:53:09.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.491+0000 7f6d9f400640 1 -- 172.21.15.67:0/1427288563 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20555 (secure 0 0 0) 0x7f6d98058df0 con 0x7f6da80710a0 2024-03-09T08:53:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:53:09.488 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":26,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T08:52:59.780300+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T08:53:09.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.493+0000 7f6dacc00640 1 -- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6d8806b4d0 msgr2=0x7f6d8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.493+0000 7f6dacc00640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6d8806b4d0 0x7f6d8806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d94006fd0 tx=0x7f6d94008040 comp rx=0 tx=0).stop 2024-03-09T08:53:09.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.493+0000 7f6dacc00640 1 -- 172.21.15.67:0/1427288563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da80710a0 msgr2=0x7f6da8151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.493+0000 7f6dacc00640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da80710a0 0x7f6da8151b30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800d620 tx=0x7f6d9800dae0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.493+0000 7f6dacc00640 1 -- 172.21.15.67:0/1427288563 shutdown_connections 2024-03-09T08:53:09.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.493+0000 7f6dacc00640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6d8806b4d0 0x7f6d8806d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.494+0000 7f6dacc00640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6da8154080 0x7f6da81eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.494+0000 7f6dacc00640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6da80c1f70 0x7f6da814bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.494+0000 7f6dacc00640 1 --2- 172.21.15.67:0/1427288563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6da80710a0 0x7f6da8151b30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.494+0000 7f6dacc00640 1 -- 172.21.15.67:0/1427288563 >> 172.21.15.67:0/1427288563 conn(0x7f6da80c7300 msgr2=0x7f6da80ac6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:09.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.494+0000 7f6dacc00640 1 -- 172.21.15.67:0/1427288563 shutdown_connections 2024-03-09T08:53:09.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.494+0000 7f6dacc00640 1 -- 172.21.15.67:0/1427288563 wait complete. 2024-03-09T08:53:09.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-09T08:52:06.749641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-03-09T08:52:16.185356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-03-09T08:52:16.756829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-03-09T08:52:27.484297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-03-09T08:52:28.060754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-09T08:53:09.513 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T08:53:09.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.732+0000 7f11ffe00640 1 Processor -- start 2024-03-09T08:53:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.732+0000 7f11ffe00640 1 -- start start 2024-03-09T08:53:09.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.734+0000 7f11ffe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12000c7f60 0x7f1200150de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.734+0000 7f11ffe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1200151320 0x7f1200153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.734+0000 7f11ffe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12000713a0 con 0x7f12000ac930 2024-03-09T08:53:09.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.734+0000 7f11ffe00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1200070670 con 0x7f1200151320 2024-03-09T08:53:09.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.734+0000 7f11ffe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12000c3740 con 0x7f12000c7f60 2024-03-09T08:53:09.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.735+0000 7f11fe000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12000c7f60 0x7f1200150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.735+0000 7f11ff400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1200151320 0x7f1200153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.735+0000 7f11fe000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12000c7f60 0x7f1200150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:34270/0 (socket says 172.21.15.67:34270) 2024-03-09T08:53:09.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.735+0000 7f11fe000640 1 -- 172.21.15.67:0/1546746507 learned_addr learned my addr 172.21.15.67:0/1546746507 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:09.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.735+0000 7f11fd600640 1 -- 172.21.15.67:0/1546746507 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393736465 0 0) 0x7f12000713a0 con 0x7f12000ac930 2024-03-09T08:53:09.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.735+0000 7f11fe000640 1 -- 172.21.15.67:0/1546746507 >> v2:172.21.15.134:3300/0 conn(0x7f1200151320 msgr2=0x7f1200153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:09.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.735+0000 7f11fe000640 1 --2- 172.21.15.67:0/1546746507 >> v2:172.21.15.134:3300/0 conn(0x7f1200151320 0x7f1200153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.735+0000 7f11fe000640 1 -- 172.21.15.67:0/1546746507 >> v1:172.21.15.134:6789/0 conn(0x7f12000ac930 legacy=0x7f12000c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.735+0000 7f11fe000640 1 -- 172.21.15.67:0/1546746507 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12000c39a0 con 0x7f12000c7f60 2024-03-09T08:53:09.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.737+0000 7f11fe000640 1 --2- 172.21.15.67:0/1546746507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12000c7f60 0x7f1200150de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f12000707f0 tx=0x7f11ec02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4da07ac20f36184d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:09.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.737+0000 7f11fd600640 1 -- 172.21.15.67:0/1546746507 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec035020 con 0x7f12000c7f60 2024-03-09T08:53:09.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.737+0000 7f11fd600640 1 -- 172.21.15.67:0/1546746507 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec031640 con 0x7f12000c7f60 2024-03-09T08:53:09.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.737+0000 7f11fd600640 1 -- 172.21.15.67:0/1546746507 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec02d040 con 0x7f12000c7f60 2024-03-09T08:53:09.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.738+0000 7f11ffe00640 1 -- 172.21.15.67:0/1546746507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12000c7f60 msgr2=0x7f1200150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.738+0000 7f11ffe00640 1 --2- 172.21.15.67:0/1546746507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12000c7f60 0x7f1200150de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f12000707f0 tx=0x7f11ec02b8f0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.738+0000 7f11ffe00640 1 -- 172.21.15.67:0/1546746507 shutdown_connections 2024-03-09T08:53:09.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.738+0000 7f11ffe00640 1 --2- 172.21.15.67:0/1546746507 >> v2:172.21.15.134:3300/0 conn(0x7f1200151320 0x7f1200153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.738+0000 7f11ffe00640 1 --2- 172.21.15.67:0/1546746507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12000c7f60 0x7f1200150de0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.738+0000 7f11ffe00640 1 -- 172.21.15.67:0/1546746507 >> 172.21.15.67:0/1546746507 conn(0x7f120006b8a0 msgr2=0x7f12000c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:09.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.738+0000 7f11ffe00640 1 -- 172.21.15.67:0/1546746507 shutdown_connections 2024-03-09T08:53:09.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.738+0000 7f11ffe00640 1 -- 172.21.15.67:0/1546746507 wait complete. 2024-03-09T08:53:09.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.740+0000 7f11ffe00640 1 Processor -- start 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.740+0000 7f11ffe00640 1 -- start start 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.741+0000 7f11ffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12000ac930 0x7f120014dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.741+0000 7f11ffe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12000c7f60 0x7f12001e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.741+0000 7f11ffe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1200151320 0x7f12001e8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.741+0000 7f11ffe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200070ba0 con 0x7f12000ac930 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.741+0000 7f11ffe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200151770 con 0x7f12000c7f60 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.741+0000 7f11ffe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1200070e80 con 0x7f1200151320 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.741+0000 7f11fea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12000ac930 0x7f120014dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.741+0000 7f11fea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12000ac930 0x7f120014dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60144/0 (socket says 172.21.15.67:60144) 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.741+0000 7f11fea00640 1 -- 172.21.15.67:0/3826690882 learned_addr learned my addr 172.21.15.67:0/3826690882 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.741+0000 7f11ff400640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1200151320 0x7f12001e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.741+0000 7f11fe000640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12000c7f60 0x7f12001e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.742+0000 7f11fea00640 1 -- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1200151320 msgr2=0x7f12001e8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.742+0000 7f11fea00640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1200151320 0x7f12001e8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.742+0000 7f11fea00640 1 -- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12000c7f60 msgr2=0x7f12001e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.742+0000 7f11fea00640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12000c7f60 0x7f12001e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.742+0000 7f11fea00640 1 -- 172.21.15.67:0/3826690882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12001eb7a0 con 0x7f12000ac930 2024-03-09T08:53:09.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.742+0000 7f11fea00640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12000ac930 0x7f120014dd40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c490 tx=0x7f11f000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:09.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.742+0000 7f11f6a00640 1 -- 172.21.15.67:0/3826690882 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0005480 con 0x7f12000ac930 2024-03-09T08:53:09.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.742+0000 7f11f6a00640 1 -- 172.21.15.67:0/3826690882 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f00035f0 con 0x7f12000ac930 2024-03-09T08:53:09.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.742+0000 7f11ffe00640 1 -- 172.21.15.67:0/3826690882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12001eba80 con 0x7f12000ac930 2024-03-09T08:53:09.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.743+0000 7f11f6a00640 1 -- 172.21.15.67:0/3826690882 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f00039c0 con 0x7f12000ac930 2024-03-09T08:53:09.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.743+0000 7f11ffe00640 1 -- 172.21.15.67:0/3826690882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1200152f20 con 0x7f12000ac930 2024-03-09T08:53:09.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.744+0000 7f11ffe00640 1 -- 172.21.15.67:0/3826690882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120004f3c0 con 0x7f12000ac930 2024-03-09T08:53:09.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.746+0000 7f11f6a00640 1 -- 172.21.15.67:0/3826690882 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f11f0003b60 con 0x7f12000ac930 2024-03-09T08:53:09.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.747+0000 7f11f6a00640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f11e006b670 0x7f11e006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:09.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.747+0000 7f11f6a00640 1 -- 172.21.15.67:0/3826690882 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f11f008a160 con 0x7f12000ac930 2024-03-09T08:53:09.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.747+0000 7f11fe000640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f11e006b670 0x7f11e006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:09.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.749+0000 7f11fe000640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f11e006b670 0x7f11e006db30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f11ec036000 tx=0x7f11ec008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:09.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.752+0000 7f11f6a00640 1 -- 172.21.15.67:0/3826690882 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f11f0058c30 con 0x7f12000ac930 2024-03-09T08:53:09.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.952+0000 7f11ffe00640 1 -- 172.21.15.67:0/3826690882 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f12001ebc50 con 0x7f12000ac930 2024-03-09T08:53:09.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.952+0000 7f11f6a00640 1 -- 172.21.15.67:0/3826690882 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11f0058530 con 0x7f12000ac930 2024-03-09T08:53:09.993 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-09T08:53:09.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.955+0000 7f11ffe00640 1 -- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f11e006b670 msgr2=0x7f11e006db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.955+0000 7f11ffe00640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f11e006b670 0x7f11e006db30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f11ec036000 tx=0x7f11ec008030 comp rx=0 tx=0).stop 2024-03-09T08:53:09.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.955+0000 7f11ffe00640 1 -- 172.21.15.67:0/3826690882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12000ac930 msgr2=0x7f120014dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.955+0000 7f11ffe00640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12000ac930 0x7f120014dd40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c490 tx=0x7f11f000c950 comp rx=0 tx=0).stop 2024-03-09T08:53:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.956+0000 7f11ffe00640 1 -- 172.21.15.67:0/3826690882 shutdown_connections 2024-03-09T08:53:09.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.956+0000 7f11ffe00640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f11e006b670 0x7f11e006db30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.956+0000 7f11ffe00640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1200151320 0x7f12001e8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.956+0000 7f11ffe00640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12000c7f60 0x7f12001e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.956+0000 7f11ffe00640 1 --2- 172.21.15.67:0/3826690882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12000ac930 0x7f120014dd40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:09.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.956+0000 7f11ffe00640 1 -- 172.21.15.67:0/3826690882 >> 172.21.15.67:0/3826690882 conn(0x7f120006b8a0 msgr2=0x7f12000c2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:09.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.956+0000 7f11ffe00640 1 -- 172.21.15.67:0/3826690882 shutdown_connections 2024-03-09T08:53:09.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:09.956+0000 7f11ffe00640 1 -- 172.21.15.67:0/3826690882 wait complete. 2024-03-09T08:53:10.000 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-03-09T08:53:10.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.230+0000 7f66aa000640 1 Processor -- start 2024-03-09T08:53:10.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.230+0000 7f66aa000640 1 -- start start 2024-03-09T08:53:10.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.232+0000 7f66aa000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f66a40c9490 0x7f66a40c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:10.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.232+0000 7f66aa000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a40c9a30 con 0x7f66a40c5f10 2024-03-09T08:53:10.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.232+0000 7f66aa000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a40c9d10 con 0x7f66a406b040 2024-03-09T08:53:10.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.232+0000 7f66aa000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a40ca340 con 0x7f66a40c9490 2024-03-09T08:53:10.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.232+0000 7f66a8c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f66a40c5f10 0x7f66a41567d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38004/0 (socket says 172.21.15.67:38004) 2024-03-09T08:53:10.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.232+0000 7f66a8c00640 1 -- 172.21.15.67:0/2235170942 learned_addr learned my addr 172.21.15.67:0/2235170942 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:10.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.232+0000 7f66a9600640 1 --2- 172.21.15.67:0/2235170942 >> v2:172.21.15.67:3300/0 conn(0x7f66a40c9490 0x7f66a40c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:10.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.233+0000 7f66a9600640 1 -- 172.21.15.67:0/2235170942 >> v1:172.21.15.134:6789/0 conn(0x7f66a406b040 legacy=0x7f66a40cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:10.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.233+0000 7f66a9600640 1 -- 172.21.15.67:0/2235170942 >> v1:172.21.15.67:6789/0 conn(0x7f66a40c5f10 legacy=0x7f66a41567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:10.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.233+0000 7f66a9600640 1 -- 172.21.15.67:0/2235170942 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a40ca620 con 0x7f66a40c9490 2024-03-09T08:53:10.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.234+0000 7f66a9600640 1 --2- 172.21.15.67:0/2235170942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a40c9490 0x7f66a40c62f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f669c009d90 tx=0x7f669c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8615490426867c3c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:10.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.234+0000 7f66a3400640 1 -- 172.21.15.67:0/2235170942 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c039070 con 0x7f66a40c9490 2024-03-09T08:53:10.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.234+0000 7f66a3400640 1 -- 172.21.15.67:0/2235170942 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669c02cbd0 con 0x7f66a40c9490 2024-03-09T08:53:10.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.234+0000 7f66a3400640 1 -- 172.21.15.67:0/2235170942 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c034440 con 0x7f66a40c9490 2024-03-09T08:53:10.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.235+0000 7f66aa000640 1 -- 172.21.15.67:0/2235170942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a40c9490 msgr2=0x7f66a40c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:10.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.235+0000 7f66aa000640 1 --2- 172.21.15.67:0/2235170942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a40c9490 0x7f66a40c62f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f669c009d90 tx=0x7f669c02be20 comp rx=0 tx=0).stop 2024-03-09T08:53:10.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.235+0000 7f66aa000640 1 -- 172.21.15.67:0/2235170942 shutdown_connections 2024-03-09T08:53:10.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.235+0000 7f66aa000640 1 --2- 172.21.15.67:0/2235170942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a40c9490 0x7f66a40c62f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:10.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.235+0000 7f66aa000640 1 -- 172.21.15.67:0/2235170942 >> 172.21.15.67:0/2235170942 conn(0x7f66a40cb590 msgr2=0x7f66a40cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:10.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.235+0000 7f66aa000640 1 -- 172.21.15.67:0/2235170942 shutdown_connections 2024-03-09T08:53:10.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.235+0000 7f66aa000640 1 -- 172.21.15.67:0/2235170942 wait complete. 2024-03-09T08:53:10.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.237+0000 7f66aa000640 1 Processor -- start 2024-03-09T08:53:10.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.237+0000 7f66aa000640 1 -- start start 2024-03-09T08:53:10.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.239+0000 7f66aa000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a406b040 0x7f66a41eff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.239+0000 7f66aa000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66a40c5f10 0x7f66a41e3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.239+0000 7f66aa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a40c9490 0x7f66a414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.239+0000 7f66aa000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a40cb3d0 con 0x7f66a40c9490 2024-03-09T08:53:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.239+0000 7f66aa000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a40cb250 con 0x7f66a406b040 2024-03-09T08:53:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.239+0000 7f66aa000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a4153620 con 0x7f66a40c5f10 2024-03-09T08:53:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.239+0000 7f66a9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a40c9490 0x7f66a414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.239+0000 7f66a9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a40c9490 0x7f66a414f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60174/0 (socket says 172.21.15.67:60174) 2024-03-09T08:53:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.239+0000 7f66a9600640 1 -- 172.21.15.67:0/1077282323 learned_addr learned my addr 172.21.15.67:0/1077282323 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.239+0000 7f66a8c00640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a406b040 0x7f66a41eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.239+0000 7f66a3e00640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66a40c5f10 0x7f66a41e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:10.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.240+0000 7f66a9600640 1 -- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66a40c5f10 msgr2=0x7f66a41e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:10.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.240+0000 7f66a9600640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66a40c5f10 0x7f66a41e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:10.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.240+0000 7f66a9600640 1 -- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a406b040 msgr2=0x7f66a41eff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:10.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.240+0000 7f66a9600640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a406b040 0x7f66a41eff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:10.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.240+0000 7f66a9600640 1 -- 172.21.15.67:0/1077282323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a41ef490 con 0x7f66a40c9490 2024-03-09T08:53:10.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.240+0000 7f66a9600640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a40c9490 0x7f66a414f120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f669c037040 tx=0x7f669c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:10.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.241+0000 7f66a0c00640 1 -- 172.21.15.67:0/1077282323 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c039050 con 0x7f66a40c9490 2024-03-09T08:53:10.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.241+0000 7f66a0c00640 1 -- 172.21.15.67:0/1077282323 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669c03a040 con 0x7f66a40c9490 2024-03-09T08:53:10.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.241+0000 7f66a0c00640 1 -- 172.21.15.67:0/1077282323 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c03e610 con 0x7f66a40c9490 2024-03-09T08:53:10.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.241+0000 7f66aa000640 1 -- 172.21.15.67:0/1077282323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a40c5110 con 0x7f66a40c9490 2024-03-09T08:53:10.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.241+0000 7f66aa000640 1 -- 172.21.15.67:0/1077282323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a41ef750 con 0x7f66a40c9490 2024-03-09T08:53:10.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.242+0000 7f66aa000640 1 -- 172.21.15.67:0/1077282323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a404f3c0 con 0x7f66a40c9490 2024-03-09T08:53:10.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.243+0000 7f66a0c00640 1 -- 172.21.15.67:0/1077282323 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f669c047020 con 0x7f66a40c9490 2024-03-09T08:53:10.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.244+0000 7f66a0c00640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f668406b4d0 0x7f668406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:10.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.244+0000 7f66a0c00640 1 -- 172.21.15.67:0/1077282323 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f669c0ac7e0 con 0x7f66a40c9490 2024-03-09T08:53:10.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.244+0000 7f66a8c00640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f668406b4d0 0x7f668406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:10.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.245+0000 7f66a8c00640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f668406b4d0 0x7f668406d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f66a4152460 tx=0x7f6698007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.248+0000 7f66a0c00640 1 -- 172.21.15.67:0/1077282323 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f669c04e020 con 0x7f66a40c9490 2024-03-09T08:53:10.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.606+0000 7f66aa000640 1 -- 172.21.15.67:0/1077282323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66a40cc260 con 0x7f66a40c9490 2024-03-09T08:53:10.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.607+0000 7f66a0c00640 1 -- 172.21.15.67:0/1077282323 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v26) v1 ==== 99+0+11 (secure 0 0 0) 0x7f669c07ad30 con 0x7f66a40c9490 2024-03-09T08:53:10.603 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T08:53:10.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.610+0000 7f66aa000640 1 -- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f668406b4d0 msgr2=0x7f668406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:10.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.610+0000 7f66aa000640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f668406b4d0 0x7f668406d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f66a4152460 tx=0x7f6698007e80 comp rx=0 tx=0).stop 2024-03-09T08:53:10.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.611+0000 7f66aa000640 1 -- 172.21.15.67:0/1077282323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a40c9490 msgr2=0x7f66a414f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:10.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.611+0000 7f66aa000640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a40c9490 0x7f66a414f120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f669c037040 tx=0x7f669c004a40 comp rx=0 tx=0).stop 2024-03-09T08:53:10.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.611+0000 7f66aa000640 1 -- 172.21.15.67:0/1077282323 shutdown_connections 2024-03-09T08:53:10.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.611+0000 7f66aa000640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f668406b4d0 0x7f668406d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:10.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.611+0000 7f66aa000640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a40c9490 0x7f66a414f120 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.611+0000 7f66aa000640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66a40c5f10 0x7f66a41e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.611+0000 7f66aa000640 1 --2- 172.21.15.67:0/1077282323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66a406b040 0x7f66a41eff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.611+0000 7f66aa000640 1 -- 172.21.15.67:0/1077282323 >> 172.21.15.67:0/1077282323 conn(0x7f66a40cb590 msgr2=0x7f66a4150e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.611+0000 7f66aa000640 1 -- 172.21.15.67:0/1077282323 shutdown_connections 2024-03-09T08:53:10.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.611+0000 7f66aa000640 1 -- 172.21.15.67:0/1077282323 wait complete. 2024-03-09T08:53:10.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-03-09T08:53:10.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.843+0000 7f96ce600640 1 Processor -- start 2024-03-09T08:53:10.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.843+0000 7f96ce600640 1 -- start start 2024-03-09T08:53:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.845+0000 7f96ce600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f96c80ac930 0x7f96c80c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.845+0000 7f96ce600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c80713a0 con 0x7f96c81514f0 2024-03-09T08:53:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.845+0000 7f96ce600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c80c39a0 con 0x7f96c80c7f60 2024-03-09T08:53:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.845+0000 7f96ce600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c8070670 con 0x7f96c80ac930 2024-03-09T08:53:10.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.845+0000 7f96cd200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f96c80ac930 0x7f96c80c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:10.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.845+0000 7f96cd200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f96c80ac930 0x7f96c80c3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60188/0 (socket says 172.21.15.67:60188) 2024-03-09T08:53:10.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.845+0000 7f96cd200640 1 -- 172.21.15.67:0/2159428787 learned_addr learned my addr 172.21.15.67:0/2159428787 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:10.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.846+0000 7f96cd200640 1 -- 172.21.15.67:0/2159428787 >> v1:172.21.15.134:6789/0 conn(0x7f96c80c7f60 legacy=0x7f96c8150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:10.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.846+0000 7f96cd200640 1 -- 172.21.15.67:0/2159428787 >> v1:172.21.15.67:6789/0 conn(0x7f96c81514f0 legacy=0x7f96c8153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:10.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.846+0000 7f96cd200640 1 -- 172.21.15.67:0/2159428787 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c80708c0 con 0x7f96c80ac930 2024-03-09T08:53:10.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.847+0000 7f96cd200640 1 --2- 172.21.15.67:0/2159428787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80ac930 0x7f96c80c3030 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f96c80707f0 tx=0x7f96bc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5408c53cca13d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:10.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.847+0000 7f96c7400640 1 -- 172.21.15.67:0/2159428787 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc034070 con 0x7f96c80ac930 2024-03-09T08:53:10.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.847+0000 7f96c7400640 1 -- 172.21.15.67:0/2159428787 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96bc030840 con 0x7f96c80ac930 2024-03-09T08:53:10.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.847+0000 7f96c7400640 1 -- 172.21.15.67:0/2159428787 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc030c10 con 0x7f96c80ac930 2024-03-09T08:53:10.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.848+0000 7f96ce600640 1 -- 172.21.15.67:0/2159428787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80ac930 msgr2=0x7f96c80c3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:10.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.848+0000 7f96ce600640 1 --2- 172.21.15.67:0/2159428787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80ac930 0x7f96c80c3030 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f96c80707f0 tx=0x7f96bc02b980 comp rx=0 tx=0).stop 2024-03-09T08:53:10.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.848+0000 7f96ce600640 1 -- 172.21.15.67:0/2159428787 shutdown_connections 2024-03-09T08:53:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.848+0000 7f96ce600640 1 --2- 172.21.15.67:0/2159428787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80ac930 0x7f96c80c3030 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.848+0000 7f96ce600640 1 -- 172.21.15.67:0/2159428787 >> 172.21.15.67:0/2159428787 conn(0x7f96c806b8a0 msgr2=0x7f96c80c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.848+0000 7f96ce600640 1 -- 172.21.15.67:0/2159428787 shutdown_connections 2024-03-09T08:53:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.848+0000 7f96ce600640 1 -- 172.21.15.67:0/2159428787 wait complete. 2024-03-09T08:53:10.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.850+0000 7f96ce600640 1 Processor -- start 2024-03-09T08:53:10.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.850+0000 7f96ce600640 1 -- start start 2024-03-09T08:53:10.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.851+0000 7f96ce600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac930 0x7f96c80c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:10.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.851+0000 7f96ce600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80c7f60 0x7f96c81dfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:10.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.851+0000 7f96ce600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c81514f0 0x7f96c8148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:10.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.851+0000 7f96ce600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80c3570 con 0x7f96c80c7f60 2024-03-09T08:53:10.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.851+0000 7f96ce600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c8070e80 con 0x7f96c81514f0 2024-03-09T08:53:10.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.851+0000 7f96ce600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80c36f0 con 0x7f96c80ac930 2024-03-09T08:53:10.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.851+0000 7f96c7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80c7f60 0x7f96c81dfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:10.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.851+0000 7f96c7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80c7f60 0x7f96c81dfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60196/0 (socket says 172.21.15.67:60196) 2024-03-09T08:53:10.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.851+0000 7f96c7e00640 1 -- 172.21.15.67:0/1819249652 learned_addr learned my addr 172.21.15.67:0/1819249652 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:10.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.851+0000 7f96cd200640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac930 0x7f96c80c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:10.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.851+0000 7f96cdc00640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c81514f0 0x7f96c8148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:10.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.852+0000 7f96c7e00640 1 -- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac930 msgr2=0x7f96c80c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:10.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.852+0000 7f96c7e00640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac930 0x7f96c80c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:10.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.852+0000 7f96c7e00640 1 -- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c81514f0 msgr2=0x7f96c8148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:10.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.852+0000 7f96c7e00640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c81514f0 0x7f96c8148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:10.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.852+0000 7f96c7e00640 1 -- 172.21.15.67:0/1819249652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c814dbe0 con 0x7f96c80c7f60 2024-03-09T08:53:10.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.852+0000 7f96c7e00640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80c7f60 0x7f96c81dfa70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f96b400c400 tx=0x7f96b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:10.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.852+0000 7f96c4c00640 1 -- 172.21.15.67:0/1819249652 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4017070 con 0x7f96c80c7f60 2024-03-09T08:53:10.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.852+0000 7f96c4c00640 1 -- 172.21.15.67:0/1819249652 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b40052d0 con 0x7f96c80c7f60 2024-03-09T08:53:10.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.852+0000 7f96c4c00640 1 -- 172.21.15.67:0/1819249652 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4003830 con 0x7f96c80c7f60 2024-03-09T08:53:10.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.852+0000 7f96ce600640 1 -- 172.21.15.67:0/1819249652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c81de550 con 0x7f96c80c7f60 2024-03-09T08:53:10.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.853+0000 7f96ce600640 1 -- 172.21.15.67:0/1819249652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c80c6f60 con 0x7f96c80c7f60 2024-03-09T08:53:10.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.854+0000 7f96ce600640 1 -- 172.21.15.67:0/1819249652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c804f3c0 con 0x7f96c80c7f60 2024-03-09T08:53:10.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.855+0000 7f96c4c00640 1 -- 172.21.15.67:0/1819249652 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f96b401a070 con 0x7f96c80c7f60 2024-03-09T08:53:10.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.856+0000 7f96c4c00640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a806b4d0 0x7f96a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:10.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.856+0000 7f96c4c00640 1 -- 172.21.15.67:0/1819249652 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f96b4089820 con 0x7f96c80c7f60 2024-03-09T08:53:10.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.856+0000 7f96cd200640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a806b4d0 0x7f96a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:10.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.858+0000 7f96cd200640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a806b4d0 0x7f96a806d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f96bc036000 tx=0x7f96bc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:10.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:10.861+0000 7f96c4c00640 1 -- 172.21.15.67:0/1819249652 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f96b4057d40 con 0x7f96c80c7f60 2024-03-09T08:53:11.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.052+0000 7f96ce600640 1 -- 172.21.15.67:0/1819249652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f96c81e0be0 con 0x7f96c80c7f60 2024-03-09T08:53:11.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.053+0000 7f96c4c00640 1 -- 172.21.15.67:0/1819249652 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v26) v1 ==== 95+0+11 (secure 0 0 0) 0x7f96b405bde0 con 0x7f96c80c7f60 2024-03-09T08:53:11.049 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.055+0000 7f96ce600640 1 -- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a806b4d0 msgr2=0x7f96a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.055+0000 7f96ce600640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a806b4d0 0x7f96a806d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f96bc036000 tx=0x7f96bc008030 comp rx=0 tx=0).stop 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.055+0000 7f96ce600640 1 -- 172.21.15.67:0/1819249652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80c7f60 msgr2=0x7f96c81dfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.055+0000 7f96ce600640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80c7f60 0x7f96c81dfa70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f96b400c400 tx=0x7f96b400c8c0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.055+0000 7f96ce600640 1 -- 172.21.15.67:0/1819249652 shutdown_connections 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.055+0000 7f96ce600640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a806b4d0 0x7f96a806d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.055+0000 7f96ce600640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c81514f0 0x7f96c8148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.055+0000 7f96ce600640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80c7f60 0x7f96c81dfa70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.055+0000 7f96ce600640 1 --2- 172.21.15.67:0/1819249652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac930 0x7f96c80c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.055+0000 7f96ce600640 1 -- 172.21.15.67:0/1819249652 >> 172.21.15.67:0/1819249652 conn(0x7f96c806b8a0 msgr2=0x7f96c8072940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.055+0000 7f96ce600640 1 -- 172.21.15.67:0/1819249652 shutdown_connections 2024-03-09T08:53:11.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.056+0000 7f96ce600640 1 -- 172.21.15.67:0/1819249652 wait complete. 2024-03-09T08:53:11.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-03-09T08:53:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.292+0000 7fe431800640 1 Processor -- start 2024-03-09T08:53:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.292+0000 7fe431800640 1 -- start start 2024-03-09T08:53:11.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.293+0000 7fe431800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe42c0c6770 0x7fe42c0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.293+0000 7fe431800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe42c0ac490 0x7fe42c14fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.294+0000 7fe431800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42c152e70 con 0x7fe42c150090 2024-03-09T08:53:11.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.294+0000 7fe431800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c1539f0 con 0x7fe42c0ac490 2024-03-09T08:53:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.294+0000 7fe431800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c0aa3b0 con 0x7fe42c0c6770 2024-03-09T08:53:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.294+0000 7fe422000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe42c0ac490 0x7fe42c14fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.294+0000 7fe422000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe42c0ac490 0x7fe42c14fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60198/0 (socket says 172.21.15.67:60198) 2024-03-09T08:53:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.294+0000 7fe422000640 1 -- 172.21.15.67:0/1946395276 learned_addr learned my addr 172.21.15.67:0/1946395276 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:11.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.294+0000 7fe42aa00640 1 --2- 172.21.15.67:0/1946395276 >> v2:172.21.15.134:3300/0 conn(0x7fe42c0c6770 0x7fe42c0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.295+0000 7fe422000640 1 -- 172.21.15.67:0/1946395276 >> v2:172.21.15.134:3300/0 conn(0x7fe42c0c6770 msgr2=0x7fe42c0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:11.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.295+0000 7fe422000640 1 --2- 172.21.15.67:0/1946395276 >> v2:172.21.15.134:3300/0 conn(0x7fe42c0c6770 0x7fe42c0abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.295+0000 7fe422000640 1 -- 172.21.15.67:0/1946395276 >> v1:172.21.15.134:6789/0 conn(0x7fe42c150090 legacy=0x7fe42c1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:11.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.295+0000 7fe422000640 1 -- 172.21.15.67:0/1946395276 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c153150 con 0x7fe42c0ac490 2024-03-09T08:53:11.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.296+0000 7fe422000640 1 --2- 172.21.15.67:0/1946395276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe42c0ac490 0x7fe42c14fb50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe42c06ea50 tx=0x7fe41c02e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98538696c60966a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.296+0000 7fe42a000640 1 -- 172.21.15.67:0/1946395276 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c02ea90 con 0x7fe42c0ac490 2024-03-09T08:53:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.296+0000 7fe42a000640 1 -- 172.21.15.67:0/1946395276 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41c02ec30 con 0x7fe42c0ac490 2024-03-09T08:53:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.296+0000 7fe42a000640 1 -- 172.21.15.67:0/1946395276 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c009600 con 0x7fe42c0ac490 2024-03-09T08:53:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.297+0000 7fe431800640 1 -- 172.21.15.67:0/1946395276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe42c0ac490 msgr2=0x7fe42c14fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.297+0000 7fe431800640 1 --2- 172.21.15.67:0/1946395276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe42c0ac490 0x7fe42c14fb50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe42c06ea50 tx=0x7fe41c02e0f0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.297+0000 7fe431800640 1 -- 172.21.15.67:0/1946395276 shutdown_connections 2024-03-09T08:53:11.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.297+0000 7fe431800640 1 --2- 172.21.15.67:0/1946395276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe42c0ac490 0x7fe42c14fb50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.297+0000 7fe431800640 1 --2- 172.21.15.67:0/1946395276 >> v2:172.21.15.134:3300/0 conn(0x7fe42c0c6770 0x7fe42c0abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.297+0000 7fe431800640 1 -- 172.21.15.67:0/1946395276 >> 172.21.15.67:0/1946395276 conn(0x7fe42c06b040 msgr2=0x7fe42c06b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:11.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.297+0000 7fe431800640 1 -- 172.21.15.67:0/1946395276 shutdown_connections 2024-03-09T08:53:11.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.297+0000 7fe431800640 1 -- 172.21.15.67:0/1946395276 wait complete. 2024-03-09T08:53:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.299+0000 7fe431800640 1 Processor -- start 2024-03-09T08:53:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.299+0000 7fe431800640 1 -- start start 2024-03-09T08:53:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.301+0000 7fe431800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe42c0ac490 0x7fe42c073460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.301+0000 7fe431800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe42c0c6770 0x7fe42c1521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.301+0000 7fe431800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe42c150090 0x7fe42c13f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.301+0000 7fe431800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c152a40 con 0x7fe42c150090 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.301+0000 7fe431800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c153710 con 0x7fe42c0c6770 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.301+0000 7fe431800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c152bc0 con 0x7fe42c0ac490 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.301+0000 7fe42b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe42c150090 0x7fe42c13f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.301+0000 7fe42b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe42c150090 0x7fe42c13f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60208/0 (socket says 172.21.15.67:60208) 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.301+0000 7fe42b400640 1 -- 172.21.15.67:0/2950759919 learned_addr learned my addr 172.21.15.67:0/2950759919 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.301+0000 7fe422000640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe42c0c6770 0x7fe42c1521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.301+0000 7fe42aa00640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe42c0ac490 0x7fe42c073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.302+0000 7fe42b400640 1 -- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe42c0ac490 msgr2=0x7fe42c073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.302+0000 7fe42b400640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe42c0ac490 0x7fe42c073460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.302+0000 7fe42b400640 1 -- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe42c0c6770 msgr2=0x7fe42c1521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.302+0000 7fe42b400640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe42c0c6770 0x7fe42c1521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.302+0000 7fe42b400640 1 -- 172.21.15.67:0/2950759919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c1eabe0 con 0x7fe42c150090 2024-03-09T08:53:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.302+0000 7fe42b400640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe42c150090 0x7fe42c13f2d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c390 tx=0x7fe42400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:11.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.302+0000 7fe423400640 1 -- 172.21.15.67:0/2950759919 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424017070 con 0x7fe42c150090 2024-03-09T08:53:11.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.302+0000 7fe423400640 1 -- 172.21.15.67:0/2950759919 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4240052d0 con 0x7fe42c150090 2024-03-09T08:53:11.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.302+0000 7fe423400640 1 -- 172.21.15.67:0/2950759919 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424003810 con 0x7fe42c150090 2024-03-09T08:53:11.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.302+0000 7fe431800640 1 -- 172.21.15.67:0/2950759919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c1eaec0 con 0x7fe42c150090 2024-03-09T08:53:11.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.302+0000 7fe431800640 1 -- 172.21.15.67:0/2950759919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c1eb480 con 0x7fe42c150090 2024-03-09T08:53:11.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.304+0000 7fe431800640 1 -- 172.21.15.67:0/2950759919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42c1eb860 con 0x7fe42c150090 2024-03-09T08:53:11.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.305+0000 7fe423400640 1 -- 172.21.15.67:0/2950759919 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe42401a070 con 0x7fe42c150090 2024-03-09T08:53:11.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.306+0000 7fe423400640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe40806b4d0 0x7fe40806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.306+0000 7fe423400640 1 -- 172.21.15.67:0/2950759919 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fe4240898f0 con 0x7fe42c150090 2024-03-09T08:53:11.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.307+0000 7fe42aa00640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe40806b4d0 0x7fe40806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.308+0000 7fe42aa00640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe40806b4d0 0x7fe40806d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe414006fd0 tx=0x7fe414008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:11.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.312+0000 7fe423400640 1 -- 172.21.15.67:0/2950759919 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe42c1eb860 con 0x7fe42c150090 2024-03-09T08:53:11.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.503+0000 7fe431800640 1 -- 172.21.15.67:0/2950759919 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe42c075a00 con 0x7fe42c150090 2024-03-09T08:53:11.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.504+0000 7fe423400640 1 -- 172.21.15.67:0/2950759919 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v26) v1 ==== 99+0+11 (secure 0 0 0) 0x7fe424058330 con 0x7fe42c150090 2024-03-09T08:53:11.500 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T08:53:11.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 -- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe40806b4d0 msgr2=0x7fe40806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:11.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe40806b4d0 0x7fe40806d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe414006fd0 tx=0x7fe414008040 comp rx=0 tx=0).stop 2024-03-09T08:53:11.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 -- 172.21.15.67:0/2950759919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe42c150090 msgr2=0x7fe42c13f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:11.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe42c150090 0x7fe42c13f2d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe42400c390 tx=0x7fe42400c850 comp rx=0 tx=0).stop 2024-03-09T08:53:11.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 -- 172.21.15.67:0/2950759919 shutdown_connections 2024-03-09T08:53:11.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe40806b4d0 0x7fe40806d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe42c150090 0x7fe42c13f2d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe42c0c6770 0x7fe42c1521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 --2- 172.21.15.67:0/2950759919 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe42c0ac490 0x7fe42c073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 -- 172.21.15.67:0/2950759919 >> 172.21.15.67:0/2950759919 conn(0x7fe42c06b040 msgr2=0x7fe42c1493c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:11.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 -- 172.21.15.67:0/2950759919 shutdown_connections 2024-03-09T08:53:11.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.506+0000 7fe431800640 1 -- 172.21.15.67:0/2950759919 wait complete. 2024-03-09T08:53:11.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-03-09T08:53:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.739+0000 7f049ce00640 1 Processor -- start 2024-03-09T08:53:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.739+0000 7f049ce00640 1 -- start start 2024-03-09T08:53:11.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.741+0000 7f049ce00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f049814a5a0 0x7f04980aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.741+0000 7f049ce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981542c0 0x7f04980c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.741+0000 7f049ce00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04981577f0 con 0x7f04980c9ea0 2024-03-09T08:53:11.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.741+0000 7f049ce00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0498157ad0 con 0x7f049814a5a0 2024-03-09T08:53:11.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.741+0000 7f049ce00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0498157db0 con 0x7f04981542c0 2024-03-09T08:53:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.741+0000 7f0496a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f04980c9ea0 0x7f0498156fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38016/0 (socket says 172.21.15.67:38016) 2024-03-09T08:53:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.741+0000 7f0496a00640 1 -- 172.21.15.67:0/747761462 learned_addr learned my addr 172.21.15.67:0/747761462 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.741+0000 7f0497400640 1 --2- 172.21.15.67:0/747761462 >> v2:172.21.15.134:3300/0 conn(0x7f049814a5a0 0x7f04980aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0497e00640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981542c0 0x7f04980c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636160087 0 0) 0x7f04981577f0 con 0x7f04980c9ea0 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f047c003520 con 0x7f04980c9ea0 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1767593687 0 0) 0x7f047c003520 con 0x7f04980c9ea0 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981542c0 msgr2=0x7f04980c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981542c0 0x7f04980c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 >> v2:172.21.15.134:3300/0 conn(0x7f049814a5a0 msgr2=0x7f04980aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 --2- 172.21.15.67:0/747761462 >> v2:172.21.15.134:3300/0 conn(0x7f049814a5a0 0x7f04980aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0498153ac0 con 0x7f04980c9ea0 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f0484001670 con 0x7f04980c9ea0 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 >> v1:172.21.15.67:6789/0 conn(0x7f04980c9ea0 legacy=0x7f0498156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 0x7f047c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f047c004800 0x7f047c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981542c0 0x7f047c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0498157ad0 con 0x7f047c0039b0 2024-03-09T08:53:11.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0498157db0 con 0x7f047c004800 2024-03-09T08:53:11.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c00b9c0 con 0x7f04981542c0 2024-03-09T08:53:11.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.742+0000 7f0497400640 1 --2- 172.21.15.67:0/747761462 >> v2:172.21.15.134:3300/0 conn(0x7f049814a5a0 0x7f04980aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:53:11.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.743+0000 7f0497400640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 0x7f047c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.743+0000 7f0496a00640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f047c004800 0x7f047c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.743+0000 7f0497e00640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981542c0 0x7f047c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.743+0000 7f0497400640 1 -- 172.21.15.67:0/747761462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981542c0 msgr2=0x7f047c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:11.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.743+0000 7f0497400640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981542c0 0x7f047c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.743+0000 7f0497400640 1 -- 172.21.15.67:0/747761462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f047c004800 msgr2=0x7f047c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:11.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.743+0000 7f0497400640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f047c004800 0x7f047c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.743+0000 7f0497400640 1 -- 172.21.15.67:0/747761462 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c003520 con 0x7f047c0039b0 2024-03-09T08:53:11.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.744+0000 7f0497400640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 0x7f047c003dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0488009c70 tx=0x7f047c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f40fb74e9764297 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:11.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.745+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488009840 con 0x7f047c0039b0 2024-03-09T08:53:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.745+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0488004020 con 0x7f047c0039b0 2024-03-09T08:53:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.745+0000 7f0496000640 1 -- 172.21.15.67:0/747761462 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488004410 con 0x7f047c0039b0 2024-03-09T08:53:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.745+0000 7f049ce00640 1 -- 172.21.15.67:0/747761462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 msgr2=0x7f047c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.745+0000 7f049ce00640 1 --2- 172.21.15.67:0/747761462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 0x7f047c003dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0488009c70 tx=0x7f047c005830 comp rx=0 tx=0).stop 2024-03-09T08:53:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.745+0000 7f0497e00640 1 -- 172.21.15.67:0/747761462 reap_dead start 2024-03-09T08:53:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.745+0000 7f049ce00640 1 -- 172.21.15.67:0/747761462 shutdown_connections 2024-03-09T08:53:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.745+0000 7f049ce00640 1 -- 172.21.15.67:0/747761462 >> 172.21.15.67:0/747761462 conn(0x7f04980673c0 msgr2=0x7f04980c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.745+0000 7f049ce00640 1 -- 172.21.15.67:0/747761462 shutdown_connections 2024-03-09T08:53:11.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.745+0000 7f049ce00640 1 -- 172.21.15.67:0/747761462 wait complete. 2024-03-09T08:53:11.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.747+0000 7f049ce00640 1 Processor -- start 2024-03-09T08:53:11.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.747+0000 7f049ce00640 1 -- start start 2024-03-09T08:53:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.748+0000 7f049ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 0x7f04980c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.748+0000 7f049ce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f049814e450 0x7f04981e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.748+0000 7f049ce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981e6930 0x7f0498150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.748+0000 7f049ce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04981548d0 con 0x7f047c0039b0 2024-03-09T08:53:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.748+0000 7f049ce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0498154750 con 0x7f049814e450 2024-03-09T08:53:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.748+0000 7f049ce00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04980c3e00 con 0x7f04981e6930 2024-03-09T08:53:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.748+0000 7f0497400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 0x7f04980c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.748+0000 7f0497400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 0x7f04980c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60216/0 (socket says 172.21.15.67:60216) 2024-03-09T08:53:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.748+0000 7f0497400640 1 -- 172.21.15.67:0/1902471146 learned_addr learned my addr 172.21.15.67:0/1902471146 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.749+0000 7f0496a00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f049814e450 0x7f04981e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.749+0000 7f0497e00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981e6930 0x7f0498150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.749+0000 7f0497400640 1 -- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981e6930 msgr2=0x7f0498150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.749+0000 7f0497400640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981e6930 0x7f0498150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.749+0000 7f0497400640 1 -- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f049814e450 msgr2=0x7f04981e63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.749+0000 7f0497400640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f049814e450 0x7f04981e63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.749+0000 7f0497400640 1 -- 172.21.15.67:0/1902471146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04981e6db0 con 0x7f047c0039b0 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.749+0000 7f0497e00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981e6930 0x7f0498150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.749+0000 7f0496a00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f049814e450 0x7f04981e63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.749+0000 7f0497400640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 0x7f04980c8290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0488009d90 tx=0x7f0488004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.750+0000 7f048f400640 1 -- 172.21.15.67:0/1902471146 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488043020 con 0x7f047c0039b0 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.750+0000 7f048f400640 1 -- 172.21.15.67:0/1902471146 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048802c960 con 0x7f047c0039b0 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.750+0000 7f049ce00640 1 -- 172.21.15.67:0/1902471146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04980ae790 con 0x7f047c0039b0 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.750+0000 7f048f400640 1 -- 172.21.15.67:0/1902471146 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488006e10 con 0x7f047c0039b0 2024-03-09T08:53:11.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.750+0000 7f049ce00640 1 -- 172.21.15.67:0/1902471146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04980adb10 con 0x7f047c0039b0 2024-03-09T08:53:11.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.751+0000 7f049ce00640 1 -- 172.21.15.67:0/1902471146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0498075a00 con 0x7f047c0039b0 2024-03-09T08:53:11.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.752+0000 7f048f400640 1 -- 172.21.15.67:0/1902471146 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0488046020 con 0x7f047c0039b0 2024-03-09T08:53:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.753+0000 7f048f400640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f047806b4d0 0x7f047806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.753+0000 7f048f400640 1 -- 172.21.15.67:0/1902471146 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f04880ac410 con 0x7f047c0039b0 2024-03-09T08:53:11.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.754+0000 7f0496a00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f047806b4d0 0x7f047806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:11.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.755+0000 7f0496a00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f047806b4d0 0x7f047806d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0484007fd0 tx=0x7f0484009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:11.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:11.758+0000 7f048f400640 1 -- 172.21.15.67:0/1902471146 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0498075a00 con 0x7f047c0039b0 2024-03-09T08:53:12.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.024+0000 7f049ce00640 1 -- 172.21.15.67:0/1902471146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7f049804f3c0 con 0x7f047c0039b0 2024-03-09T08:53:12.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.025+0000 7f048f400640 1 -- 172.21.15.67:0/1902471146 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v26) v1 ==== 95+0+11 (secure 0 0 0) 0x7f048804d030 con 0x7f047c0039b0 2024-03-09T08:53:12.021 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T08:53:12.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 -- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f047806b4d0 msgr2=0x7f047806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f047806b4d0 0x7f047806d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0484007fd0 tx=0x7f0484009040 comp rx=0 tx=0).stop 2024-03-09T08:53:12.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 -- 172.21.15.67:0/1902471146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 msgr2=0x7f04980c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 0x7f04980c8290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0488009d90 tx=0x7f0488004730 comp rx=0 tx=0).stop 2024-03-09T08:53:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 -- 172.21.15.67:0/1902471146 shutdown_connections 2024-03-09T08:53:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f047806b4d0 0x7f047806d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04981e6930 0x7f0498150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f049814e450 0x7f04981e63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 --2- 172.21.15.67:0/1902471146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f047c0039b0 0x7f04980c8290 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 -- 172.21.15.67:0/1902471146 >> 172.21.15.67:0/1902471146 conn(0x7f04980673c0 msgr2=0x7f049806f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 -- 172.21.15.67:0/1902471146 shutdown_connections 2024-03-09T08:53:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.028+0000 7f049ce00640 1 -- 172.21.15.67:0/1902471146 wait complete. 2024-03-09T08:53:12.043 DEBUG:tasks.mon_thrash:Yielding 2024-03-09T08:53:12.043 INFO:teuthology.run_tasks:Running task workunit... 2024-03-09T08:53:12.057 INFO:tasks.workunit:Pulling workunits from ref 8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-09T08:53:12.058 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-09T08:53:12.058 INFO:tasks.workunit:timeout=6h 2024-03-09T08:53:12.058 INFO:tasks.workunit:cleanup=True 2024-03-09T08:53:12.059 DEBUG:teuthology.orchestra.run.smithi067:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-09T08:53:12.059 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T08:53:12.110 INFO:teuthology.orchestra.run.smithi067.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-09T08:53:12.110 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-09T08:53:12.110 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 28h/40d Inode: 1 Links: 2 2024-03-09T08:53:12.110 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-09T08:53:12.110 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-09T08:53:12.110 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-09T08:53:12.110 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-03-09 08:52:18.269210358 +0000 2024-03-09T08:53:12.110 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-03-09 08:52:59.605755535 +0000 2024-03-09T08:53:12.110 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-03-09 08:52:18.269210358 +0000 2024-03-09T08:53:12.111 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-09T08:53:12.111 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-09T08:53:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.267+0000 7f7d3da00640 1 Processor -- start 2024-03-09T08:53:12.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.267+0000 7f7d3da00640 1 -- start start 2024-03-09T08:53:12.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.268+0000 7f7d3da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d38153300 0x7f7d380b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.268+0000 7f7d3da00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d38157000 con 0x7f7d380c5e40 2024-03-09T08:53:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.268+0000 7f7d3da00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d380ae430 con 0x7f7d380cadb0 2024-03-09T08:53:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.268+0000 7f7d3da00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d380ae6f0 con 0x7f7d38153300 2024-03-09T08:53:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.269+0000 7f7d37e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7d38153300 0x7f7d380b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.269+0000 7f7d2f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7d380c5e40 0x7f7d38156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38028/0 (socket says 172.21.15.67:38028) 2024-03-09T08:53:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.269+0000 7f7d2f400640 1 -- 172.21.15.67:0/2261851948 learned_addr learned my addr 172.21.15.67:0/2261851948 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.269+0000 7f7d37e00640 1 -- 172.21.15.67:0/2261851948 >> v1:172.21.15.134:6789/0 conn(0x7f7d380cadb0 legacy=0x7f7d38160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.269+0000 7f7d37e00640 1 -- 172.21.15.67:0/2261851948 >> v1:172.21.15.67:6789/0 conn(0x7f7d380c5e40 legacy=0x7f7d38156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.269+0000 7f7d37e00640 1 -- 172.21.15.67:0/2261851948 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d38155b10 con 0x7f7d38153300 2024-03-09T08:53:12.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.269+0000 7f7d37400640 1 -- 172.21.15.67:0/2261851948 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019181534 0 0) 0x7f7d38157000 con 0x7f7d380c5e40 2024-03-09T08:53:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.270+0000 7f7d37e00640 1 --2- 172.21.15.67:0/2261851948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d38153300 0x7f7d380b0000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7d280097a0 tx=0x7f7d2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b822ccb888d460e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.271+0000 7f7d37400640 1 -- 172.21.15.67:0/2261851948 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28035020 con 0x7f7d38153300 2024-03-09T08:53:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.271+0000 7f7d37400640 1 -- 172.21.15.67:0/2261851948 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d28031830 con 0x7f7d38153300 2024-03-09T08:53:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.271+0000 7f7d37400640 1 -- 172.21.15.67:0/2261851948 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28031c00 con 0x7f7d38153300 2024-03-09T08:53:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.271+0000 7f7d3da00640 1 -- 172.21.15.67:0/2261851948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d38153300 msgr2=0x7f7d380b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.271+0000 7f7d3da00640 1 --2- 172.21.15.67:0/2261851948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d38153300 0x7f7d380b0000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7d280097a0 tx=0x7f7d2802b8d0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.271+0000 7f7d3da00640 1 -- 172.21.15.67:0/2261851948 shutdown_connections 2024-03-09T08:53:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.271+0000 7f7d3da00640 1 --2- 172.21.15.67:0/2261851948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d38153300 0x7f7d380b0000 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.271+0000 7f7d3da00640 1 -- 172.21.15.67:0/2261851948 >> 172.21.15.67:0/2261851948 conn(0x7f7d380732d0 msgr2=0x7f7d380ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.271+0000 7f7d3da00640 1 -- 172.21.15.67:0/2261851948 shutdown_connections 2024-03-09T08:53:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.271+0000 7f7d3da00640 1 -- 172.21.15.67:0/2261851948 wait complete. 2024-03-09T08:53:12.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.273+0000 7f7d3da00640 1 Processor -- start 2024-03-09T08:53:12.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.273+0000 7f7d3da00640 1 -- start start 2024-03-09T08:53:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.274+0000 7f7d3da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d380c5e40 0x7f7d381eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.274+0000 7f7d3da00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d380cadb0 0x7f7d3814b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.274+0000 7f7d3da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d38153300 0x7f7d3814be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.274+0000 7f7d3da00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d380c59e0 con 0x7f7d38153300 2024-03-09T08:53:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.274+0000 7f7d3da00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d38159e90 con 0x7f7d380c5e40 2024-03-09T08:53:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.274+0000 7f7d3da00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d380b0540 con 0x7f7d380cadb0 2024-03-09T08:53:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.274+0000 7f7d3d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d38153300 0x7f7d3814be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.274+0000 7f7d3d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d38153300 0x7f7d3814be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60246/0 (socket says 172.21.15.67:60246) 2024-03-09T08:53:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.275+0000 7f7d3d000640 1 -- 172.21.15.67:0/890760971 learned_addr learned my addr 172.21.15.67:0/890760971 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:12.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.275+0000 7f7d37e00640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d380c5e40 0x7f7d381eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:12.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.275+0000 7f7d2f400640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d380cadb0 0x7f7d3814b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:12.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.275+0000 7f7d3d000640 1 -- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d380cadb0 msgr2=0x7f7d3814b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.275+0000 7f7d3d000640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d380cadb0 0x7f7d3814b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.275+0000 7f7d3d000640 1 -- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d380c5e40 msgr2=0x7f7d381eeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.275+0000 7f7d3d000640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d380c5e40 0x7f7d381eeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.275+0000 7f7d3d000640 1 -- 172.21.15.67:0/890760971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d381ef410 con 0x7f7d38153300 2024-03-09T08:53:12.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.275+0000 7f7d3d000640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d38153300 0x7f7d3814be30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c780 tx=0x7f7d3000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:12.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.276+0000 7f7d34c00640 1 -- 172.21.15.67:0/890760971 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30004b60 con 0x7f7d38153300 2024-03-09T08:53:12.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.276+0000 7f7d34c00640 1 -- 172.21.15.67:0/890760971 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d30004d00 con 0x7f7d38153300 2024-03-09T08:53:12.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.276+0000 7f7d34c00640 1 -- 172.21.15.67:0/890760971 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000aa90 con 0x7f7d38153300 2024-03-09T08:53:12.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.276+0000 7f7d3da00640 1 -- 172.21.15.67:0/890760971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3806bb10 con 0x7f7d38153300 2024-03-09T08:53:12.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.276+0000 7f7d3da00640 1 -- 172.21.15.67:0/890760971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3814c370 con 0x7f7d38153300 2024-03-09T08:53:12.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.277+0000 7f7d3da00640 1 -- 172.21.15.67:0/890760971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3804f3c0 con 0x7f7d38153300 2024-03-09T08:53:12.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.278+0000 7f7d34c00640 1 -- 172.21.15.67:0/890760971 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7d3000ac30 con 0x7f7d38153300 2024-03-09T08:53:12.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.279+0000 7f7d34c00640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d1406b4d0 0x7f7d1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.279+0000 7f7d34c00640 1 -- 172.21.15.67:0/890760971 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f7d3008a7f0 con 0x7f7d38153300 2024-03-09T08:53:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.280+0000 7f7d37e00640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d1406b4d0 0x7f7d1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:12.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.281+0000 7f7d37e00640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d1406b4d0 0x7f7d1406d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7d28036000 tx=0x7f7d2803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:12.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.284+0000 7f7d34c00640 1 -- 172.21.15.67:0/890760971 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7d30020070 con 0x7f7d38153300 2024-03-09T08:53:12.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.521+0000 7f7d3da00640 1 -- 172.21.15.67:0/890760971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d380cab90 con 0x7f7d38153300 2024-03-09T08:53:12.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.523+0000 7f7d34c00640 1 -- 172.21.15.67:0/890760971 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6247 (secure 0 0 0) 0x7f7d30059360 con 0x7f7d38153300 2024-03-09T08:53:12.519 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T08:53:12.519 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:17.736973+0000","modified":"2024-03-09T08:52:21.931108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T08:53:12.519 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 14 2024-03-09T08:53:12.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 -- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d1406b4d0 msgr2=0x7f7d1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d1406b4d0 0x7f7d1406d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7d28036000 tx=0x7f7d2803c040 comp rx=0 tx=0).stop 2024-03-09T08:53:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 -- 172.21.15.67:0/890760971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d38153300 msgr2=0x7f7d3814be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d38153300 0x7f7d3814be30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c780 tx=0x7f7d3000cc40 comp rx=0 tx=0).stop 2024-03-09T08:53:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 -- 172.21.15.67:0/890760971 shutdown_connections 2024-03-09T08:53:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d1406b4d0 0x7f7d1406d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d38153300 0x7f7d3814be30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d380cadb0 0x7f7d3814b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 --2- 172.21.15.67:0/890760971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d380c5e40 0x7f7d381eeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 -- 172.21.15.67:0/890760971 >> 172.21.15.67:0/890760971 conn(0x7f7d380732d0 msgr2=0x7f7d3814f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 -- 172.21.15.67:0/890760971 shutdown_connections 2024-03-09T08:53:12.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.525+0000 7f7d3da00640 1 -- 172.21.15.67:0/890760971 wait complete. 2024-03-09T08:53:12.539 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-03-09T08:53:12.539 INFO:tasks.mon_thrash.mon_thrasher:seed: 1709974389, revive delay: 20.0, thrash delay: 10.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-03-09T08:53:12.540 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:53:12.540 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:53:12.727 DEBUG:teuthology.orchestra.run.smithi134:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-09T08:53:12.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.745+0000 7f5993400640 1 Processor -- start 2024-03-09T08:53:12.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.745+0000 7f5993400640 1 -- start start 2024-03-09T08:53:12.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.747+0000 7f5993400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f598c06b040 0x7f598c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:12.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.747+0000 7f5993400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f598c06c650 0x7f598c0c6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:12.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.747+0000 7f5993400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598c06b5b0 con 0x7f598c0c5f50 2024-03-09T08:53:12.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.747+0000 7f5993400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598c0c9e00 con 0x7f598c06c650 2024-03-09T08:53:12.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.747+0000 7f5993400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c06cae0 con 0x7f598c06b040 2024-03-09T08:53:12.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.747+0000 7f5991400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f598c06c650 0x7f598c0c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:12.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.748+0000 7f5991400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f598c06c650 0x7f598c0c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52354/0 (socket says 172.21.15.67:52354) 2024-03-09T08:53:12.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.748+0000 7f598be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f598c06b040 0x7f598c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:12.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.748+0000 7f5991400640 1 -- 172.21.15.67:0/1898793713 learned_addr learned my addr 172.21.15.67:0/1898793713 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:12.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.748+0000 7f5991400640 1 -- 172.21.15.67:0/1898793713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f598c06b040 msgr2=0x7f598c155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:12.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.748+0000 7f5991400640 1 --2- 172.21.15.67:0/1898793713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f598c06b040 0x7f598c155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.748+0000 7f5991400640 1 -- 172.21.15.67:0/1898793713 >> v1:172.21.15.134:6789/0 conn(0x7f598c0c5f50 legacy=0x7f598c0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.748+0000 7f5991400640 1 -- 172.21.15.67:0/1898793713 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c06b890 con 0x7f598c06c650 2024-03-09T08:53:12.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.749+0000 7f5991400640 1 --2- 172.21.15.67:0/1898793713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c06c650 0x7f598c0c6330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5984009e50 tx=0x7f598402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d03cdb855614d6f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:12.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.750+0000 7f598b400640 1 -- 172.21.15.67:0/1898793713 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598402cb00 con 0x7f598c06c650 2024-03-09T08:53:12.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.750+0000 7f598b400640 1 -- 172.21.15.67:0/1898793713 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598402cca0 con 0x7f598c06c650 2024-03-09T08:53:12.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.750+0000 7f598b400640 1 -- 172.21.15.67:0/1898793713 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984034610 con 0x7f598c06c650 2024-03-09T08:53:12.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.750+0000 7f5993400640 1 -- 172.21.15.67:0/1898793713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c06c650 msgr2=0x7f598c0c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.750+0000 7f5993400640 1 --2- 172.21.15.67:0/1898793713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c06c650 0x7f598c0c6330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5984009e50 tx=0x7f598402bf60 comp rx=0 tx=0).stop 2024-03-09T08:53:12.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.750+0000 7f5993400640 1 -- 172.21.15.67:0/1898793713 shutdown_connections 2024-03-09T08:53:12.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.750+0000 7f5993400640 1 --2- 172.21.15.67:0/1898793713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c06c650 0x7f598c0c6330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.750+0000 7f5993400640 1 --2- 172.21.15.67:0/1898793713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f598c06b040 0x7f598c155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.750+0000 7f5993400640 1 -- 172.21.15.67:0/1898793713 >> 172.21.15.67:0/1898793713 conn(0x7f598c0c8050 msgr2=0x7f598c0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:12.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.751+0000 7f5993400640 1 -- 172.21.15.67:0/1898793713 shutdown_connections 2024-03-09T08:53:12.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.751+0000 7f5993400640 1 -- 172.21.15.67:0/1898793713 wait complete. 2024-03-09T08:53:12.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.752+0000 7f5993400640 1 Processor -- start 2024-03-09T08:53:12.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.752+0000 7f5993400640 1 -- start start 2024-03-09T08:53:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.753+0000 7f5993400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f598c06b040 0x7f598c0c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.753+0000 7f5993400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c06c650 0x7f598c0bf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.753+0000 7f5993400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f598c0c5f50 0x7f598c0bf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.753+0000 7f5993400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c06c130 con 0x7f598c06b040 2024-03-09T08:53:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.753+0000 7f5993400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c06be50 con 0x7f598c06c650 2024-03-09T08:53:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.753+0000 7f5993400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c14cc00 con 0x7f598c0c5f50 2024-03-09T08:53:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.754+0000 7f5990a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f598c06b040 0x7f598c0c1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.754+0000 7f5990a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f598c06b040 0x7f598c0c1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60254/0 (socket says 172.21.15.67:60254) 2024-03-09T08:53:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.754+0000 7f5990a00640 1 -- 172.21.15.67:0/3682491961 learned_addr learned my addr 172.21.15.67:0/3682491961 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.754+0000 7f5991400640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f598c0c5f50 0x7f598c0bf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.754+0000 7f598be00640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c06c650 0x7f598c0bf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:12.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.754+0000 7f5990a00640 1 -- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f598c0c5f50 msgr2=0x7f598c0bf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.754+0000 7f5990a00640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f598c0c5f50 0x7f598c0bf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.754+0000 7f5990a00640 1 -- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c06c650 msgr2=0x7f598c0bf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.754+0000 7f5990a00640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c06c650 0x7f598c0bf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.754+0000 7f5990a00640 1 -- 172.21.15.67:0/3682491961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c1eb9c0 con 0x7f598c06b040 2024-03-09T08:53:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.754+0000 7f5990a00640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f598c06b040 0x7f598c0c1b60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f598000c3c0 tx=0x7f598000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.755+0000 7f5988c00640 1 -- 172.21.15.67:0/3682491961 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59800052d0 con 0x7f598c06b040 2024-03-09T08:53:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.755+0000 7f5988c00640 1 -- 172.21.15.67:0/3682491961 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5980005490 con 0x7f598c06b040 2024-03-09T08:53:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.755+0000 7f5988c00640 1 -- 172.21.15.67:0/3682491961 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980007260 con 0x7f598c06b040 2024-03-09T08:53:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.755+0000 7f5993400640 1 -- 172.21.15.67:0/3682491961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598c0c5090 con 0x7f598c06b040 2024-03-09T08:53:12.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.755+0000 7f5993400640 1 -- 172.21.15.67:0/3682491961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598c1ebf30 con 0x7f598c06b040 2024-03-09T08:53:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.756+0000 7f5993400640 1 -- 172.21.15.67:0/3682491961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598c068d50 con 0x7f598c06b040 2024-03-09T08:53:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.757+0000 7f5988c00640 1 -- 172.21.15.67:0/3682491961 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5980007400 con 0x7f598c06b040 2024-03-09T08:53:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.759+0000 7f5988c00640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f596c06b4d0 0x7f596c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.759+0000 7f5988c00640 1 -- 172.21.15.67:0/3682491961 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f5980089a60 con 0x7f598c06b040 2024-03-09T08:53:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.759+0000 7f598be00640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f596c06b4d0 0x7f596c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.760+0000 7f598be00640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f596c06b4d0 0x7f596c06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f598c1f0070 tx=0x7f5978009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:12.763+0000 7f5988c00640 1 -- 172.21.15.67:0/3682491961 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f598c068d50 con 0x7f598c06b040 2024-03-09T08:53:12.783 INFO:teuthology.orchestra.run.smithi134.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-09T08:53:12.783 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-09T08:53:12.783 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 28h/40d Inode: 1 Links: 2 2024-03-09T08:53:12.783 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-09T08:53:12.784 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-09T08:53:12.784 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-09T08:53:12.784 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-03-09 08:52:29.578548063 +0000 2024-03-09T08:53:12.784 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-03-09 08:53:04.635092417 +0000 2024-03-09T08:53:12.784 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-03-09 08:52:29.578548063 +0000 2024-03-09T08:53:12.784 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-09T08:53:12.784 DEBUG:teuthology.orchestra.run.smithi134:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-09T08:53:12.879 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-09T08:53:12.879 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-09T08:53:12.927 INFO:tasks.workunit.client.0.smithi067.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-09T08:53:12.933 INFO:tasks.workunit.client.1.smithi134.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-09T08:53:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.039+0000 7f5993400640 1 -- 172.21.15.67:0/3682491961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f598c075a00 con 0x7f598c06b040 2024-03-09T08:53:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.040+0000 7f5988c00640 1 -- 172.21.15.67:0/3682491961 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7f598005ce20 con 0x7f598c06b040 2024-03-09T08:53:13.037 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":79,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:53:13.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.042+0000 7f5993400640 1 -- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f596c06b4d0 msgr2=0x7f596c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.042+0000 7f5993400640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f596c06b4d0 0x7f596c06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f598c1f0070 tx=0x7f5978009190 comp rx=0 tx=0).stop 2024-03-09T08:53:13.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.042+0000 7f5993400640 1 -- 172.21.15.67:0/3682491961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f598c06b040 msgr2=0x7f598c0c1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.042+0000 7f5993400640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f598c06b040 0x7f598c0c1b60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f598000c3c0 tx=0x7f598000c880 comp rx=0 tx=0).stop 2024-03-09T08:53:13.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.043+0000 7f5993400640 1 -- 172.21.15.67:0/3682491961 shutdown_connections 2024-03-09T08:53:13.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.043+0000 7f5993400640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f596c06b4d0 0x7f596c06d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.043+0000 7f5993400640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f598c0c5f50 0x7f598c0bf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.043+0000 7f5993400640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f598c06c650 0x7f598c0bf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.043+0000 7f5993400640 1 --2- 172.21.15.67:0/3682491961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f598c06b040 0x7f598c0c1b60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.043+0000 7f5993400640 1 -- 172.21.15.67:0/3682491961 >> 172.21.15.67:0/3682491961 conn(0x7f598c0c8050 msgr2=0x7f598c0c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:13.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.043+0000 7f5993400640 1 -- 172.21.15.67:0/3682491961 shutdown_connections 2024-03-09T08:53:13.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.043+0000 7f5993400640 1 -- 172.21.15.67:0/3682491961 wait complete. 2024-03-09T08:53:13.056 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:53:13.057 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T08:53:13.057 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:53:13.057 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:53:13.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.287+0000 7f2c51800640 1 Processor -- start 2024-03-09T08:53:13.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.288+0000 7f2c51800640 1 -- start start 2024-03-09T08:53:13.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.289+0000 7f2c51800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c4c14d930 0x7f2c4c1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.289+0000 7f2c51800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c14dd10 con 0x7f2c4c154ed0 2024-03-09T08:53:13.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.289+0000 7f2c51800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c1547f0 con 0x7f2c4c0c4790 2024-03-09T08:53:13.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.289+0000 7f2c51800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c157c80 con 0x7f2c4c14d930 2024-03-09T08:53:13.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.290+0000 7f2c50e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2c4c154ed0 0x7f2c4c155350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38040/0 (socket says 172.21.15.67:38040) 2024-03-09T08:53:13.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.290+0000 7f2c50e00640 1 -- 172.21.15.67:0/384080398 learned_addr learned my addr 172.21.15.67:0/384080398 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:13.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.290+0000 7f2c4be00640 1 --2- 172.21.15.67:0/384080398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c4c14d930 0x7f2c4c1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.290+0000 7f2c4aa00640 1 -- 172.21.15.67:0/384080398 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497713790 0 0) 0x7f2c44001490 con 0x7f2c4c154ed0 2024-03-09T08:53:13.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.290+0000 7f2c4aa00640 1 -- 172.21.15.67:0/384080398 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c30003520 con 0x7f2c4c154ed0 2024-03-09T08:53:13.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.291+0000 7f2c4be00640 1 -- 172.21.15.67:0/384080398 >> v1:172.21.15.134:6789/0 conn(0x7f2c4c0c4790 legacy=0x7f2c4c06b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.291+0000 7f2c4aa00640 1 -- 172.21.15.67:0/384080398 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3432947036 0 0) 0x7f2c30003520 con 0x7f2c4c154ed0 2024-03-09T08:53:13.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.291+0000 7f2c4be00640 1 -- 172.21.15.67:0/384080398 >> v1:172.21.15.67:6789/0 conn(0x7f2c4c154ed0 legacy=0x7f2c4c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.291+0000 7f2c4be00640 1 -- 172.21.15.67:0/384080398 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c157070 con 0x7f2c4c14d930 2024-03-09T08:53:13.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.292+0000 7f2c4be00640 1 --2- 172.21.15.67:0/384080398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c4c14d930 0x7f2c4c1542b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0097d0 tx=0x7f2c3c02b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c5e7052e72f9f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.292+0000 7f2c4aa00640 1 -- 172.21.15.67:0/384080398 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c037070 con 0x7f2c4c14d930 2024-03-09T08:53:13.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.293+0000 7f2c4aa00640 1 -- 172.21.15.67:0/384080398 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c030510 con 0x7f2c4c14d930 2024-03-09T08:53:13.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.293+0000 7f2c4aa00640 1 -- 172.21.15.67:0/384080398 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44001bc0 con 0x7f2c4c14d930 2024-03-09T08:53:13.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.293+0000 7f2c51800640 1 -- 172.21.15.67:0/384080398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c4c14d930 msgr2=0x7f2c4c1542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.293+0000 7f2c51800640 1 --2- 172.21.15.67:0/384080398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c4c14d930 0x7f2c4c1542b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0097d0 tx=0x7f2c3c02b770 comp rx=0 tx=0).stop 2024-03-09T08:53:13.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.293+0000 7f2c51800640 1 -- 172.21.15.67:0/384080398 shutdown_connections 2024-03-09T08:53:13.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.293+0000 7f2c51800640 1 --2- 172.21.15.67:0/384080398 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c4c14d930 0x7f2c4c1542b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.293+0000 7f2c51800640 1 -- 172.21.15.67:0/384080398 >> 172.21.15.67:0/384080398 conn(0x7f2c4c0c8480 msgr2=0x7f2c4c0c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:13.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.293+0000 7f2c51800640 1 -- 172.21.15.67:0/384080398 shutdown_connections 2024-03-09T08:53:13.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.293+0000 7f2c51800640 1 -- 172.21.15.67:0/384080398 wait complete. 2024-03-09T08:53:13.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.295+0000 7f2c51800640 1 Processor -- start 2024-03-09T08:53:13.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.295+0000 7f2c51800640 1 -- start start 2024-03-09T08:53:13.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.296+0000 7f2c51800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c0c4790 0x7f2c4c14ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.296+0000 7f2c51800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c4c14d930 0x7f2c4c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.296+0000 7f2c51800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c4c154ed0 0x7f2c4c1ece80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.296+0000 7f2c51800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c157630 con 0x7f2c4c0c4790 2024-03-09T08:53:13.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.296+0000 7f2c51800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c156e20 con 0x7f2c4c154ed0 2024-03-09T08:53:13.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.296+0000 7f2c51800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c157910 con 0x7f2c4c14d930 2024-03-09T08:53:13.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.296+0000 7f2c4be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c0c4790 0x7f2c4c14ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.296+0000 7f2c4be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c0c4790 0x7f2c4c14ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60266/0 (socket says 172.21.15.67:60266) 2024-03-09T08:53:13.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.296+0000 7f2c4be00640 1 -- 172.21.15.67:0/218098421 learned_addr learned my addr 172.21.15.67:0/218098421 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:13.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.297+0000 7f2c50e00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c4c154ed0 0x7f2c4c1ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.297+0000 7f2c4be00640 1 -- 172.21.15.67:0/218098421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c4c14d930 msgr2=0x7f2c4c06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.297+0000 7f2c4be00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c4c14d930 0x7f2c4c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.297+0000 7f2c4be00640 1 -- 172.21.15.67:0/218098421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c4c154ed0 msgr2=0x7f2c4c1ece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.297+0000 7f2c4be00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c4c154ed0 0x7f2c4c1ece80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.297+0000 7f2c4be00640 1 -- 172.21.15.67:0/218098421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c14c690 con 0x7f2c4c0c4790 2024-03-09T08:53:13.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.297+0000 7f2c50e00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c4c154ed0 0x7f2c4c1ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:13.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.297+0000 7f2c4be00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c0c4790 0x7f2c4c14ba90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c035040 tx=0x7f2c3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.297+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c042020 con 0x7f2c4c0c4790 2024-03-09T08:53:13.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.298+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c030cf0 con 0x7f2c4c0c4790 2024-03-09T08:53:13.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.298+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c037040 con 0x7f2c4c0c4790 2024-03-09T08:53:13.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.298+0000 7f2c51800640 1 -- 172.21.15.67:0/218098421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c1ed7c0 con 0x7f2c4c0c4790 2024-03-09T08:53:13.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.298+0000 7f2c51800640 1 -- 172.21.15.67:0/218098421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c14cc10 con 0x7f2c4c0c4790 2024-03-09T08:53:13.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.299+0000 7f2c51800640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c146bc0 0x7f2c4c1501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.299+0000 7f2c51800640 1 -- 172.21.15.67:0/218098421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c14b7b0 con 0x7f2c4c146bc0 2024-03-09T08:53:13.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.299+0000 7f2c4b400640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c146bc0 0x7f2c4c1501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.299+0000 7f2c4b400640 1 -- 172.21.15.67:0/218098421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c4c0c7fc0 con 0x7f2c4c146bc0 2024-03-09T08:53:13.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.300+0000 7f2c4b400640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c146bc0 0x7f2c4c1501a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0c7970 tx=0x7f2c3800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.300+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38005280 con 0x7f2c4c146bc0 2024-03-09T08:53:13.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.300+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f2c38005420 con 0x7f2c4c146bc0 2024-03-09T08:53:13.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.300+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c146bc0 msgr2=0x7f2c4c1501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.300+0000 7f2c43e00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c146bc0 0x7f2c4c1501a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0c7970 tx=0x7f2c3800c7a0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.300+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c146bc0 msgr2=0x7f2c4c1501a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:13.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.301+0000 7f2c43e00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c146bc0 0x7f2c4c1501a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.301+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2c3c045020 con 0x7f2c4c0c4790 2024-03-09T08:53:13.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.302+0000 7f2c43e00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c2c06f3c0 0x7f2c2c071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.302+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f2c3c0ac270 con 0x7f2c4c0c4790 2024-03-09T08:53:13.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.303+0000 7f2c4b400640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c2c06f3c0 0x7f2c2c071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.304+0000 7f2c4b400640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c2c06f3c0 0x7f2c2c071880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2c38000f60 tx=0x7f2c38009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.308+0000 7f2c51800640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c147110 0x7f2c4c1506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.308+0000 7f2c51800640 1 -- 172.21.15.67:0/218098421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c04f3c0 con 0x7f2c4c147110 2024-03-09T08:53:13.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.308+0000 7f2c50e00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c147110 0x7f2c4c1506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.309+0000 7f2c50e00640 1 -- 172.21.15.67:0/218098421 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2c4c156c00 con 0x7f2c4c147110 2024-03-09T08:53:13.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.309+0000 7f2c50e00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c147110 0x7f2c4c1506e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2c44001d30 tx=0x7f2c4400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.309+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4400bc20 con 0x7f2c4c147110 2024-03-09T08:53:13.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5018 (secure 0 0 0) 0x7f2c4400bc20 con 0x7f2c4c147110 2024-03-09T08:53:13.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c147110 msgr2=0x7f2c4c1506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c43e00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c147110 0x7f2c4c1506e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2c44001d30 tx=0x7f2c4400b4a0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c43e00640 1 -- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c147110 msgr2=0x7f2c4c1506e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:13.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c43e00640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c147110 0x7f2c4c1506e0 secure :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2c44001d30 tx=0x7f2c4400b4a0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.307 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:53:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:53:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:53:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:53:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 4, 2024-03-09T08:53:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:53:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:53:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:53:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:53:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 79, 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:53:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:53:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:53:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:53:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:53:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:53:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:53:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:53:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:53:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:53:13.323 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:53:13.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c51800640 1 -- 172.21.15.67:0/218098421 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c2c06f3c0 msgr2=0x7f2c2c071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c51800640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c2c06f3c0 0x7f2c2c071880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2c38000f60 tx=0x7f2c38009030 comp rx=0 tx=0).stop 2024-03-09T08:53:13.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c51800640 1 -- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c0c4790 msgr2=0x7f2c4c14ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c51800640 1 --2- 172.21.15.67:0/218098421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c4c0c4790 0x7f2c4c14ba90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c035040 tx=0x7f2c3c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c50e00640 1 -- 172.21.15.67:0/218098421 reap_dead start 2024-03-09T08:53:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c50e00640 1 -- 172.21.15.67:0/218098421 reap_dead start 2024-03-09T08:53:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c51800640 1 -- 172.21.15.67:0/218098421 shutdown_connections 2024-03-09T08:53:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.310+0000 7f2c51800640 1 -- 172.21.15.67:0/218098421 >> 172.21.15.67:0/218098421 conn(0x7f2c4c0c8480 msgr2=0x7f2c4c155740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.311+0000 7f2c51800640 1 -- 172.21.15.67:0/218098421 shutdown_connections 2024-03-09T08:53:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.311+0000 7f2c51800640 1 -- 172.21.15.67:0/218098421 wait complete. 2024-03-09T08:53:13.326 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:53:13.326 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:53:13.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:53:13.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.518+0000 7fbb04400640 1 Processor -- start 2024-03-09T08:53:13.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.518+0000 7fbb04400640 1 -- start start 2024-03-09T08:53:13.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.519+0000 7fbb04400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbafc0c5f50 0x7fbafc06b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.519+0000 7fbb04400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbafc0c96f0 0x7fbafc0c6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.519+0000 7fbb04400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbafc0c9f30 con 0x7fbafc1567a0 2024-03-09T08:53:13.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.519+0000 7fbb04400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc06cfa0 con 0x7fbafc0c96f0 2024-03-09T08:53:13.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.519+0000 7fbb04400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc0ca820 con 0x7fbafc0c5f50 2024-03-09T08:53:13.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.520+0000 7fbb01000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbafc0c96f0 0x7fbafc0c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.520+0000 7fbb01000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbafc0c96f0 0x7fbafc0c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60294/0 (socket says 172.21.15.67:60294) 2024-03-09T08:53:13.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.520+0000 7fbb01000640 1 -- 172.21.15.67:0/2182791931 learned_addr learned my addr 172.21.15.67:0/2182791931 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:13.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.520+0000 7fbb01a00640 1 --2- 172.21.15.67:0/2182791931 >> v2:172.21.15.134:3300/0 conn(0x7fbafc0c5f50 0x7fbafc06b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.520+0000 7fbb01000640 1 -- 172.21.15.67:0/2182791931 >> v2:172.21.15.134:3300/0 conn(0x7fbafc0c5f50 msgr2=0x7fbafc06b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:13.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.520+0000 7fbb01000640 1 --2- 172.21.15.67:0/2182791931 >> v2:172.21.15.134:3300/0 conn(0x7fbafc0c5f50 0x7fbafc06b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.520+0000 7fbb01000640 1 -- 172.21.15.67:0/2182791931 >> v1:172.21.15.134:6789/0 conn(0x7fbafc1567a0 legacy=0x7fbafc0cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:13.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.520+0000 7fbb01000640 1 -- 172.21.15.67:0/2182791931 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc0ca1d0 con 0x7fbafc0c96f0 2024-03-09T08:53:13.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.520+0000 7fbb01a00640 1 --2- 172.21.15.67:0/2182791931 >> v2:172.21.15.134:3300/0 conn(0x7fbafc0c5f50 0x7fbafc06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:53:13.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.521+0000 7fbb01000640 1 --2- 172.21.15.67:0/2182791931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbafc0c96f0 0x7fbafc0c6330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0753c0 tx=0x7fbae802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe4537da6bd70f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.521+0000 7fbafbe00640 1 -- 172.21.15.67:0/2182791931 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8035020 con 0x7fbafc0c96f0 2024-03-09T08:53:13.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.522+0000 7fbafbe00640 1 -- 172.21.15.67:0/2182791931 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae8031870 con 0x7fbafc0c96f0 2024-03-09T08:53:13.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.522+0000 7fbafbe00640 1 -- 172.21.15.67:0/2182791931 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8031c40 con 0x7fbafc0c96f0 2024-03-09T08:53:13.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.522+0000 7fbb04400640 1 -- 172.21.15.67:0/2182791931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbafc0c96f0 msgr2=0x7fbafc0c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.522+0000 7fbb04400640 1 --2- 172.21.15.67:0/2182791931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbafc0c96f0 0x7fbafc0c6330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0753c0 tx=0x7fbae802b910 comp rx=0 tx=0).stop 2024-03-09T08:53:13.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.522+0000 7fbb04400640 1 -- 172.21.15.67:0/2182791931 shutdown_connections 2024-03-09T08:53:13.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.522+0000 7fbb04400640 1 --2- 172.21.15.67:0/2182791931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbafc0c96f0 0x7fbafc0c6330 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.522+0000 7fbb04400640 1 --2- 172.21.15.67:0/2182791931 >> v2:172.21.15.134:3300/0 conn(0x7fbafc0c5f50 0x7fbafc06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.522+0000 7fbb04400640 1 -- 172.21.15.67:0/2182791931 >> 172.21.15.67:0/2182791931 conn(0x7fbafc0c8050 msgr2=0x7fbafc0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:13.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.522+0000 7fbb04400640 1 -- 172.21.15.67:0/2182791931 shutdown_connections 2024-03-09T08:53:13.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.522+0000 7fbb04400640 1 -- 172.21.15.67:0/2182791931 wait complete. 2024-03-09T08:53:13.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.524+0000 7fbb04400640 1 Processor -- start 2024-03-09T08:53:13.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.524+0000 7fbb04400640 1 -- start start 2024-03-09T08:53:13.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.525+0000 7fbb04400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbafc0c5f50 0x7fbafc14a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.525+0000 7fbb04400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbafc0c96f0 0x7fbafc154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.525+0000 7fbb04400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc1567a0 0x7fbafc154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.525+0000 7fbb04400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc156c10 con 0x7fbafc0c5f50 2024-03-09T08:53:13.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.525+0000 7fbb04400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc0c9bf0 con 0x7fbafc1567a0 2024-03-09T08:53:13.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.525+0000 7fbb04400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc156d90 con 0x7fbafc0c96f0 2024-03-09T08:53:13.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.525+0000 7fbb01a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbafc0c5f50 0x7fbafc14a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.525+0000 7fbb01a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbafc0c5f50 0x7fbafc14a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60302/0 (socket says 172.21.15.67:60302) 2024-03-09T08:53:13.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.525+0000 7fbb01a00640 1 -- 172.21.15.67:0/4238904890 learned_addr learned my addr 172.21.15.67:0/4238904890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:13.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.525+0000 7fbb01000640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbafc0c96f0 0x7fbafc154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.525+0000 7fbb02400640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc1567a0 0x7fbafc154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.526+0000 7fbb01a00640 1 -- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbafc0c96f0 msgr2=0x7fbafc154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.526+0000 7fbb01a00640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbafc0c96f0 0x7fbafc154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.526+0000 7fbb01a00640 1 -- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc1567a0 msgr2=0x7fbafc154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.526+0000 7fbb01a00640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc1567a0 0x7fbafc154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.526+0000 7fbb01a00640 1 -- 172.21.15.67:0/4238904890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc0ad9c0 con 0x7fbafc0c5f50 2024-03-09T08:53:13.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.526+0000 7fbb01a00640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbafc0c5f50 0x7fbafc14a3d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000d800 tx=0x7fbaf000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.526+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0004030 con 0x7fbafc0c5f50 2024-03-09T08:53:13.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.526+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf00041d0 con 0x7fbafc0c5f50 2024-03-09T08:53:13.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.526+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf000b6c0 con 0x7fbafc0c5f50 2024-03-09T08:53:13.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.526+0000 7fbb04400640 1 -- 172.21.15.67:0/4238904890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc0adca0 con 0x7fbafc0c5f50 2024-03-09T08:53:13.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.526+0000 7fbb04400640 1 -- 172.21.15.67:0/4238904890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc14aa70 con 0x7fbafc0c5f50 2024-03-09T08:53:13.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.527+0000 7fbb04400640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc1588f0 0x7fbafc14c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.527+0000 7fbb04400640 1 -- 172.21.15.67:0/4238904890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc075a00 con 0x7fbafc1588f0 2024-03-09T08:53:13.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.528+0000 7fbb01000640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc1588f0 0x7fbafc14c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.528+0000 7fbb01000640 1 -- 172.21.15.67:0/4238904890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbafc06cd10 con 0x7fbafc1588f0 2024-03-09T08:53:13.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.528+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbaf000b860 con 0x7fbafc0c5f50 2024-03-09T08:53:13.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.529+0000 7fbb01000640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc1588f0 0x7fbafc14c3e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0c90a0 tx=0x7fbae8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.529+0000 7fbaf9600640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbadc06b500 0x7fbadc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.529+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fbaf008a770 con 0x7fbafc0c5f50 2024-03-09T08:53:13.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.530+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8035040 con 0x7fbafc1588f0 2024-03-09T08:53:13.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.530+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fbae8044400 con 0x7fbafc1588f0 2024-03-09T08:53:13.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.530+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc1588f0 msgr2=0x7fbafc14c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.530+0000 7fbaf9600640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc1588f0 0x7fbafc14c3e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0c90a0 tx=0x7fbae8004730 comp rx=0 tx=0).stop 2024-03-09T08:53:13.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.530+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc1588f0 msgr2=0x7fbafc14c3e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:13.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.530+0000 7fbaf9600640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc1588f0 0x7fbafc14c3e0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0c90a0 tx=0x7fbae8004730 comp rx=0 tx=0).stop 2024-03-09T08:53:13.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.530+0000 7fbb02400640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbadc06b500 0x7fbadc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.531+0000 7fbb02400640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbadc06b500 0x7fbadc06d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005520 tx=0x7fbaf4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.537+0000 7fbb04400640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc14c920 0x7fbafc159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.537+0000 7fbb04400640 1 -- 172.21.15.67:0/4238904890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc152b40 con 0x7fbafc14c920 2024-03-09T08:53:13.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.537+0000 7fbb01000640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc14c920 0x7fbafc159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.538+0000 7fbb01000640 1 -- 172.21.15.67:0/4238904890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbafc14dba0 con 0x7fbafc14c920 2024-03-09T08:53:13.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.538+0000 7fbb01000640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc14c920 0x7fbafc159300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbae8000c00 tx=0x7fbae80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.538+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8035040 con 0x7fbafc14c920 2024-03-09T08:53:13.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.539+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5186 (secure 0 0 0) 0x7fbae80303c0 con 0x7fbafc14c920 2024-03-09T08:53:13.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.539+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc14c920 msgr2=0x7fbafc159300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.539+0000 7fbaf9600640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc14c920 0x7fbafc159300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbae8000c00 tx=0x7fbae80046c0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.539+0000 7fbaf9600640 1 -- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc14c920 msgr2=0x7fbafc159300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:13.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.539+0000 7fbaf9600640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbafc14c920 0x7fbafc159300 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbae8000c00 tx=0x7fbae80046c0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.536 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:53:13.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:53:13.536 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:53:13.537 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:53:13.537 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 4, 2024-03-09T08:53:13.537 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:53:13.537 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:53:13.537 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:53:13.537 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:53:13.537 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.537 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 80, 2024-03-09T08:53:13.537 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:13.537 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:53:13.537 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:53:13.538 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:13.538 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:13.538 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:13.538 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:13.538 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:13.538 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:13.538 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:13.538 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:13.538 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:13.538 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:13.539 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:13.539 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.539 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:53:13.539 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:53:13.539 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:13.539 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:13.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:13.539 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:13.539 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:13.539 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:13.539 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:13.540 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:13.540 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:13.540 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:13.540 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:13.540 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.540 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:53:13.540 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:53:13.540 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:53:13.540 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:53:13.541 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:53:13.541 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:53:13.541 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:13.541 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:13.541 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:53:13.541 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:53:13.541 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:53:13.541 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:53:13.541 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:53:13.542 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:53:13.542 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:53:13.542 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:13.542 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:53:13.542 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:13.542 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:13.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:13.542 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:13.542 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:53:13.543 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.544 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:13.545 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:13.546 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:13.547 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:13.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.547 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:53:13.547 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:53:13.547 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:13.547 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:13.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.547 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:53:13.548 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.550 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.551 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.552 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.552 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:13.552 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.552 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.552 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:53:13.552 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:53:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.540+0000 7fbb04400640 1 -- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbadc06b500 msgr2=0x7fbadc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.540+0000 7fbb04400640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbadc06b500 0x7fbadc06d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005520 tx=0x7fbaf4009040 comp rx=0 tx=0).stop 2024-03-09T08:53:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.540+0000 7fbb04400640 1 -- 172.21.15.67:0/4238904890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbafc0c5f50 msgr2=0x7fbafc14a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.540+0000 7fbb04400640 1 --2- 172.21.15.67:0/4238904890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbafc0c5f50 0x7fbafc14a3d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000d800 tx=0x7fbaf000dcc0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.540+0000 7fbb02400640 1 -- 172.21.15.67:0/4238904890 reap_dead start 2024-03-09T08:53:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.540+0000 7fbb02400640 1 -- 172.21.15.67:0/4238904890 reap_dead start 2024-03-09T08:53:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.540+0000 7fbb04400640 1 -- 172.21.15.67:0/4238904890 shutdown_connections 2024-03-09T08:53:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.540+0000 7fbb04400640 1 -- 172.21.15.67:0/4238904890 >> 172.21.15.67:0/4238904890 conn(0x7fbafc0c8050 msgr2=0x7fbafc14f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.540+0000 7fbb04400640 1 -- 172.21.15.67:0/4238904890 shutdown_connections 2024-03-09T08:53:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.540+0000 7fbb04400640 1 -- 172.21.15.67:0/4238904890 wait complete. 2024-03-09T08:53:13.554 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:53:13.554 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:53:13.555 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:53:13.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.760+0000 7f662ee00640 1 Processor -- start 2024-03-09T08:53:13.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.760+0000 7f662ee00640 1 -- start start 2024-03-09T08:53:13.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662ee00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f662806b040 0x7f66280c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662ee00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f66280717f0 0x7f662806b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662ee00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6628159240 0x7f66281596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662ee00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6628071d40 con 0x7f662806b040 2024-03-09T08:53:13.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662ee00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6628072ba0 con 0x7f66280717f0 2024-03-09T08:53:13.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662ee00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662814ff60 con 0x7f6628159240 2024-03-09T08:53:13.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f662806b040 0x7f66280c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f662806b040 0x7f66280c3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60310/0 (socket says 172.21.15.67:60310) 2024-03-09T08:53:13.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662da00640 1 -- 172.21.15.67:0/2073338354 learned_addr learned my addr 172.21.15.67:0/2073338354 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:13.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662da00640 1 -- 172.21.15.67:0/2073338354 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6628159240 msgr2=0x7f66281596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:13.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662da00640 1 --2- 172.21.15.67:0/2073338354 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6628159240 0x7f66281596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662da00640 1 -- 172.21.15.67:0/2073338354 >> v2:172.21.15.134:3300/0 conn(0x7f66280717f0 msgr2=0x7f662806b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:13.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662da00640 1 --2- 172.21.15.67:0/2073338354 >> v2:172.21.15.134:3300/0 conn(0x7f66280717f0 0x7f662806b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.762+0000 7f662da00640 1 -- 172.21.15.67:0/2073338354 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628071f90 con 0x7f662806b040 2024-03-09T08:53:13.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.764+0000 7f662da00640 1 --2- 172.21.15.67:0/2073338354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f662806b040 0x7f66280c3630 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f66280ae610 tx=0x7f661c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9949fa992ac9ea2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.764+0000 7f6627e00640 1 -- 172.21.15.67:0/2073338354 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c02e070 con 0x7f662806b040 2024-03-09T08:53:13.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.764+0000 7f6627e00640 1 -- 172.21.15.67:0/2073338354 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c037070 con 0x7f662806b040 2024-03-09T08:53:13.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.764+0000 7f6627e00640 1 -- 172.21.15.67:0/2073338354 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c031a00 con 0x7f662806b040 2024-03-09T08:53:13.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.765+0000 7f662ee00640 1 -- 172.21.15.67:0/2073338354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f662806b040 msgr2=0x7f66280c3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.765+0000 7f662ee00640 1 --2- 172.21.15.67:0/2073338354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f662806b040 0x7f66280c3630 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f66280ae610 tx=0x7f661c02b880 comp rx=0 tx=0).stop 2024-03-09T08:53:13.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.765+0000 7f662ee00640 1 -- 172.21.15.67:0/2073338354 shutdown_connections 2024-03-09T08:53:13.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.765+0000 7f662ee00640 1 --2- 172.21.15.67:0/2073338354 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6628159240 0x7f66281596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.765+0000 7f662ee00640 1 --2- 172.21.15.67:0/2073338354 >> v2:172.21.15.134:3300/0 conn(0x7f66280717f0 0x7f662806b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.765+0000 7f662ee00640 1 --2- 172.21.15.67:0/2073338354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f662806b040 0x7f66280c3630 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.765+0000 7f662ee00640 1 -- 172.21.15.67:0/2073338354 >> 172.21.15.67:0/2073338354 conn(0x7f66280c8480 msgr2=0x7f66280c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:13.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.765+0000 7f662ee00640 1 -- 172.21.15.67:0/2073338354 shutdown_connections 2024-03-09T08:53:13.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.765+0000 7f662ee00640 1 -- 172.21.15.67:0/2073338354 wait complete. 2024-03-09T08:53:13.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.767+0000 7f662ee00640 1 Processor -- start 2024-03-09T08:53:13.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.767+0000 7f662ee00640 1 -- start start 2024-03-09T08:53:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.768+0000 7f662ee00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f662806b040 0x7f6628153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.768+0000 7f662ee00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66280717f0 0x7f662806ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.768+0000 7f662ee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628159240 0x7f6628145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.768+0000 7f662ee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628072830 con 0x7f6628159240 2024-03-09T08:53:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.768+0000 7f662ee00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628072550 con 0x7f662806b040 2024-03-09T08:53:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.768+0000 7f662ee00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6628159c10 con 0x7f66280717f0 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.768+0000 7f662e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628159240 0x7f6628145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.768+0000 7f662e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628159240 0x7f6628145b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60326/0 (socket says 172.21.15.67:60326) 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.768+0000 7f662e400640 1 -- 172.21.15.67:0/3765831143 learned_addr learned my addr 172.21.15.67:0/3765831143 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.768+0000 7f662da00640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f662806b040 0x7f6628153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.769+0000 7f662d000640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66280717f0 0x7f662806ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.769+0000 7f662e400640 1 -- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66280717f0 msgr2=0x7f662806ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.769+0000 7f662e400640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66280717f0 0x7f662806ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.769+0000 7f662e400640 1 -- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f662806b040 msgr2=0x7f6628153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.769+0000 7f662e400640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f662806b040 0x7f6628153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.769+0000 7f662e400640 1 -- 172.21.15.67:0/3765831143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662806f650 con 0x7f6628159240 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.769+0000 7f662da00640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f662806b040 0x7f6628153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:53:13.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.769+0000 7f662e400640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628159240 0x7f6628145b40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f662000b9a0 tx=0x7f662000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.769+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620019070 con 0x7f6628159240 2024-03-09T08:53:13.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.769+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662000cbc0 con 0x7f6628159240 2024-03-09T08:53:13.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.769+0000 7f662ee00640 1 -- 172.21.15.67:0/3765831143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66281546d0 con 0x7f6628159240 2024-03-09T08:53:13.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.770+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620014440 con 0x7f6628159240 2024-03-09T08:53:13.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.770+0000 7f662ee00640 1 -- 172.21.15.67:0/3765831143 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66281ef700 con 0x7f6628159240 2024-03-09T08:53:13.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.770+0000 7f662ee00640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281567c0 0x7f6628146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.771+0000 7f662ee00640 1 -- 172.21.15.67:0/3765831143 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66281544a0 con 0x7f66281567c0 2024-03-09T08:53:13.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.771+0000 7f662da00640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281567c0 0x7f6628146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.772+0000 7f662da00640 1 -- 172.21.15.67:0/3765831143 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f662804f3c0 con 0x7f66281567c0 2024-03-09T08:53:13.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.772+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f662001c070 con 0x7f6628159240 2024-03-09T08:53:13.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.772+0000 7f662da00640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281567c0 0x7f6628146080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f66280c5e30 tx=0x7f661c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.773+0000 7f6625600640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f660806b4d0 0x7f660806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.773+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f662008a3c0 con 0x7f6628159240 2024-03-09T08:53:13.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.773+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c04a070 con 0x7f66281567c0 2024-03-09T08:53:13.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.773+0000 7f662d000640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f660806b4d0 0x7f660806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.774+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f661c037040 con 0x7f66281567c0 2024-03-09T08:53:13.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.774+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281567c0 msgr2=0x7f6628146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.774+0000 7f6625600640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281567c0 0x7f6628146080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f66280c5e30 tx=0x7f661c004730 comp rx=0 tx=0).stop 2024-03-09T08:53:13.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.774+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281567c0 msgr2=0x7f6628146080 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:13.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.774+0000 7f6625600640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281567c0 0x7f6628146080 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f66280c5e30 tx=0x7f661c004730 comp rx=0 tx=0).stop 2024-03-09T08:53:13.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.775+0000 7f662d000640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f660806b4d0 0x7f660806d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f66281e9a10 tx=0x7f6614008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.781+0000 7f662ee00640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281ef8d0 0x7f6628151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:13.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.781+0000 7f662ee00640 1 -- 172.21.15.67:0/3765831143 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66281534c0 con 0x7f66281ef8d0 2024-03-09T08:53:13.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.781+0000 7f662da00640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281ef8d0 0x7f6628151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:13.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.782+0000 7f662da00640 1 -- 172.21.15.67:0/3765831143 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f66280c3410 con 0x7f66281ef8d0 2024-03-09T08:53:13.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.782+0000 7f662da00640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281ef8d0 0x7f6628151380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f661c000c00 tx=0x7f661c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:13.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.783+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c04a040 con 0x7f66281ef8d0 2024-03-09T08:53:13.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.783+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4873 (secure 0 0 0) 0x7f661c04c820 con 0x7f66281ef8d0 2024-03-09T08:53:13.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.783+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281ef8d0 msgr2=0x7f6628151380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.783+0000 7f6625600640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281ef8d0 0x7f6628151380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f661c000c00 tx=0x7f661c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.783+0000 7f6625600640 1 -- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281ef8d0 msgr2=0x7f6628151380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:13.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.783+0000 7f6625600640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66281ef8d0 0x7f6628151380 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f661c000c00 tx=0x7f661c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:53:13.781 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:53:13.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:53:13.781 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:53:13.781 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:53:13.781 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 4, 2024-03-09T08:53:13.781 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:53:13.781 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:53:13.781 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 80, 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:13.782 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:13.783 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:53:13.784 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:13.785 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:53:13.786 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.787 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.788 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.788 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:53:13.788 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:53:13.788 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:13.788 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:13.788 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:13.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.788 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:53:13.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:53:13.789 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:13.789 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:13.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:13.789 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:53:13.789 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:13.789 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:53:13.790 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.790 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.790 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.790 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:53:13.790 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:53:13.790 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:13.790 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:13.790 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.792 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:13.792 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:53:13.792 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:13.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.792 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.792 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:53:13.792 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:53:13.793 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:13.793 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:13.793 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:13.793 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.793 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.793 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:53:13.793 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:53:13.794 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.794 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.794 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.794 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:13.794 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.794 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.794 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:53:13.794 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.794 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:13.795 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:53:13.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:13.796 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:13.796 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:13.796 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:53:13.796 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:13.796 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:13.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:13.796 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:53:13.796 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:53:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.784+0000 7f662ee00640 1 -- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f660806b4d0 msgr2=0x7f660806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.784+0000 7f662ee00640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f660806b4d0 0x7f660806d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f66281e9a10 tx=0x7f6614008040 comp rx=0 tx=0).stop 2024-03-09T08:53:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.784+0000 7f662ee00640 1 -- 172.21.15.67:0/3765831143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628159240 msgr2=0x7f6628145b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.784+0000 7f662ee00640 1 --2- 172.21.15.67:0/3765831143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6628159240 0x7f6628145b40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f662000b9a0 tx=0x7f662000be60 comp rx=0 tx=0).stop 2024-03-09T08:53:13.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.784+0000 7f662e400640 1 -- 172.21.15.67:0/3765831143 reap_dead start 2024-03-09T08:53:13.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.784+0000 7f662e400640 1 -- 172.21.15.67:0/3765831143 reap_dead start 2024-03-09T08:53:13.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.784+0000 7f662ee00640 1 -- 172.21.15.67:0/3765831143 shutdown_connections 2024-03-09T08:53:13.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.784+0000 7f662ee00640 1 -- 172.21.15.67:0/3765831143 >> 172.21.15.67:0/3765831143 conn(0x7f66280c8480 msgr2=0x7f662814f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:13.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.784+0000 7f662ee00640 1 -- 172.21.15.67:0/3765831143 shutdown_connections 2024-03-09T08:53:13.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:13.784+0000 7f662ee00640 1 -- 172.21.15.67:0/3765831143 wait complete. 2024-03-09T08:53:13.799 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:53:13.799 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-03-09T08:53:13.800 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T08:53:13.800 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-03-09T08:53:13.800 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T08:53:13.800 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-09T08:53:13.800 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T08:53:13.835 INFO:tasks.ceph.mon.b:Stopped 2024-03-09T08:53:13.835 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T08:53:13.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:53:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.042+0000 7fc7b7200640 1 Processor -- start 2024-03-09T08:53:14.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.042+0000 7fc7b7200640 1 -- start start 2024-03-09T08:53:14.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.044+0000 7fc7b7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc7b80c3510 0x7fc7b8156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:14.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.044+0000 7fc7b7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b80c3960 con 0x7fc7b81532c0 2024-03-09T08:53:14.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.044+0000 7fc7b7200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b80729e0 con 0x7fc7b80c2e80 2024-03-09T08:53:14.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.044+0000 7fc7b7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b8073310 con 0x7fc7b80c3510 2024-03-09T08:53:14.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.044+0000 7fc7b5400640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7fc7b80c2e80 legacy=0x7fc7b8155e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:53:14.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.044+0000 7fc7b5e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc7b81532c0 0x7fc7b806fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38042/0 (socket says 172.21.15.67:38042) 2024-03-09T08:53:14.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.044+0000 7fc7b5e00640 1 -- 172.21.15.67:0/3833847488 learned_addr learned my addr 172.21.15.67:0/3833847488 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:14.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.044+0000 7fc7b6800640 1 --2- 172.21.15.67:0/3833847488 >> v2:172.21.15.67:3300/0 conn(0x7fc7b80c3510 0x7fc7b8156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:14.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.045+0000 7fc7b6800640 1 -- 172.21.15.67:0/3833847488 >> v1:172.21.15.134:6789/0 conn(0x7fc7b80c2e80 legacy=0x7fc7b8155e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:53:14.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.045+0000 7fc7b6800640 1 -- 172.21.15.67:0/3833847488 >> v1:172.21.15.67:6789/0 conn(0x7fc7b81532c0 legacy=0x7fc7b806fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:14.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.044+0000 7fc7b4a00640 1 -- 172.21.15.67:0/3833847488 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735189091 0 0) 0x7fc7b80c3960 con 0x7fc7b81532c0 2024-03-09T08:53:14.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.045+0000 7fc7b6800640 1 -- 172.21.15.67:0/3833847488 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b8072cc0 con 0x7fc7b80c3510 2024-03-09T08:53:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.046+0000 7fc7b6800640 1 --2- 172.21.15.67:0/3833847488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7b80c3510 0x7fc7b8156550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009d90 tx=0x7fc7b002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a640e79345840 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.046+0000 7fc7b4a00640 1 -- 172.21.15.67:0/3833847488 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0039070 con 0x7fc7b80c3510 2024-03-09T08:53:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.046+0000 7fc7b4a00640 1 -- 172.21.15.67:0/3833847488 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b002cbd0 con 0x7fc7b80c3510 2024-03-09T08:53:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.046+0000 7fc7b4a00640 1 -- 172.21.15.67:0/3833847488 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0034440 con 0x7fc7b80c3510 2024-03-09T08:53:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.047+0000 7fc7b7200640 1 -- 172.21.15.67:0/3833847488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7b80c3510 msgr2=0x7fc7b8156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.047+0000 7fc7b7200640 1 --2- 172.21.15.67:0/3833847488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7b80c3510 0x7fc7b8156550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009d90 tx=0x7fc7b002be20 comp rx=0 tx=0).stop 2024-03-09T08:53:14.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.047+0000 7fc7b7200640 1 -- 172.21.15.67:0/3833847488 shutdown_connections 2024-03-09T08:53:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.047+0000 7fc7b7200640 1 --2- 172.21.15.67:0/3833847488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7b80c3510 0x7fc7b8156550 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.047+0000 7fc7b7200640 1 -- 172.21.15.67:0/3833847488 >> 172.21.15.67:0/3833847488 conn(0x7fc7b8066d90 msgr2=0x7fc7b8067170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.047+0000 7fc7b7200640 1 -- 172.21.15.67:0/3833847488 shutdown_connections 2024-03-09T08:53:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.047+0000 7fc7b7200640 1 -- 172.21.15.67:0/3833847488 wait complete. 2024-03-09T08:53:14.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.048+0000 7fc7b7200640 1 Processor -- start 2024-03-09T08:53:14.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.049+0000 7fc7b7200640 1 -- start start 2024-03-09T08:53:14.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.049+0000 7fc7b7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b80c2e80 0x7fc7b814bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:14.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.049+0000 7fc7b7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7b80c3510 0x7fc7b81eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:14.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.049+0000 7fc7b7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7b81532c0 0x7fc7b814b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.049+0000 7fc7b7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b8156a90 con 0x7fc7b80c3510 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.049+0000 7fc7b7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b80c2ab0 con 0x7fc7b80c2e80 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.049+0000 7fc7b7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b8070600 con 0x7fc7b81532c0 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b5e00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b80c2e80 msgr2=0x7fc7b814bf60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b80c2e80 0x7fc7b814bf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7b80c3510 0x7fc7b81eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7b80c3510 0x7fc7b81eeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60348/0 (socket says 172.21.15.67:60348) 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b5400640 1 -- 172.21.15.67:0/2373399457 learned_addr learned my addr 172.21.15.67:0/2373399457 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b6800640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7b81532c0 0x7fc7b814b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b5400640 1 -- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7b81532c0 msgr2=0x7fc7b814b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b5400640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7b81532c0 0x7fc7b814b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:14.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b5400640 1 -- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b80c2e80 msgr2=0x7fc7b814bf60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b5400640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b80c2e80 0x7fc7b814bf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b5400640 1 -- 172.21.15.67:0/2373399457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b80c9650 con 0x7fc7b80c3510 2024-03-09T08:53:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.050+0000 7fc7b5400640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7b80c3510 0x7fc7b81eeed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b550 tx=0x7fc7a400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.051+0000 7fc7ae000640 1 -- 172.21.15.67:0/2373399457 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4003d70 con 0x7fc7b80c3510 2024-03-09T08:53:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.051+0000 7fc7ae000640 1 -- 172.21.15.67:0/2373399457 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a4015070 con 0x7fc7b80c3510 2024-03-09T08:53:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.051+0000 7fc7ae000640 1 -- 172.21.15.67:0/2373399457 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40052b0 con 0x7fc7b80c3510 2024-03-09T08:53:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.051+0000 7fc7b7200640 1 -- 172.21.15.67:0/2373399457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b814bc80 con 0x7fc7b80c3510 2024-03-09T08:53:14.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.051+0000 7fc7b7200640 1 -- 172.21.15.67:0/2373399457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b81ef9f0 con 0x7fc7b80c3510 2024-03-09T08:53:14.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.052+0000 7fc7b7200640 1 -- 172.21.15.67:0/2373399457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b8075a00 con 0x7fc7b80c3510 2024-03-09T08:53:14.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.054+0000 7fc7ae000640 1 -- 172.21.15.67:0/2373399457 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc7a4005450 con 0x7fc7b80c3510 2024-03-09T08:53:14.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.055+0000 7fc7ae000640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc79806b480 0x7fc79806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:14.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.055+0000 7fc7ae000640 1 -- 172.21.15.67:0/2373399457 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fc7a408a3e0 con 0x7fc7b80c3510 2024-03-09T08:53:14.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.056+0000 7fc7b5e00640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc79806b480 0x7fc79806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:14.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.057+0000 7fc7b5e00640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc79806b480 0x7fc79806d940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8159e70 tx=0x7fc7a8007a00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:14.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.060+0000 7fc7ae000640 1 -- 172.21.15.67:0/2373399457 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc7b8075a00 con 0x7fc7b80c3510 2024-03-09T08:53:14.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.339+0000 7fc7b7200640 1 -- 172.21.15.67:0/2373399457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc7b8073900 con 0x7fc7b80c3510 2024-03-09T08:53:14.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.340+0000 7fc7ae000640 1 -- 172.21.15.67:0/2373399457 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7fc7b8073900 con 0x7fc7b80c3510 2024-03-09T08:53:14.336 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":80,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:53:14.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.342+0000 7fc7b7200640 1 -- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc79806b480 msgr2=0x7fc79806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:14.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.342+0000 7fc7b7200640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc79806b480 0x7fc79806d940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8159e70 tx=0x7fc7a8007a00 comp rx=0 tx=0).stop 2024-03-09T08:53:14.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.342+0000 7fc7b7200640 1 -- 172.21.15.67:0/2373399457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7b80c3510 msgr2=0x7fc7b81eeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:14.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.342+0000 7fc7b7200640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7b80c3510 0x7fc7b81eeed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b550 tx=0x7fc7a400ba10 comp rx=0 tx=0).stop 2024-03-09T08:53:14.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.343+0000 7fc7b7200640 1 -- 172.21.15.67:0/2373399457 shutdown_connections 2024-03-09T08:53:14.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.343+0000 7fc7b7200640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc79806b480 0x7fc79806d940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:14.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.343+0000 7fc7b7200640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7b81532c0 0x7fc7b814b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:14.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.343+0000 7fc7b7200640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7b80c3510 0x7fc7b81eeed0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:14.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.343+0000 7fc7b7200640 1 --2- 172.21.15.67:0/2373399457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b80c2e80 0x7fc7b814bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:14.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.343+0000 7fc7b7200640 1 -- 172.21.15.67:0/2373399457 >> 172.21.15.67:0/2373399457 conn(0x7fc7b8066d90 msgr2=0x7fc7b8150a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:14.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.343+0000 7fc7b7200640 1 -- 172.21.15.67:0/2373399457 shutdown_connections 2024-03-09T08:53:14.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:14.343+0000 7fc7b7200640 1 -- 172.21.15.67:0/2373399457 wait complete. 2024-03-09T08:53:17.358 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:53:17.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.589+0000 7f714e600640 1 Processor -- start 2024-03-09T08:53:17.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.589+0000 7f714e600640 1 -- start start 2024-03-09T08:53:17.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.590+0000 7f714e600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f714806d420 0x7f71481527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:17.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.590+0000 7f714e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f714814c9f0 0x7f71481540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:17.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f714e600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f714806d920 con 0x7f71480c3080 2024-03-09T08:53:17.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f714e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7148155ba0 con 0x7f714814c9f0 2024-03-09T08:53:17.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f714e600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714806dc00 con 0x7f714806d420 2024-03-09T08:53:17.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f714d200640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f71480c3080 legacy=0x7f714814c2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:53:17.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f7147e00640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f714806d420 msgr2=0x7f71481527d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:17.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f714dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f714814c9f0 0x7f71481540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:17.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f7147e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f714806d420 0x7f71481527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:17.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f714dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f714814c9f0 0x7f71481540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60352/0 (socket says 172.21.15.67:60352) 2024-03-09T08:53:17.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f714dc00640 1 -- 172.21.15.67:0/3544059989 learned_addr learned my addr 172.21.15.67:0/3544059989 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:17.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f714dc00640 1 -- 172.21.15.67:0/3544059989 >> v2:172.21.15.134:3300/0 conn(0x7f714806d420 msgr2=0x7f71481527d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:53:17.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f714dc00640 1 --2- 172.21.15.67:0/3544059989 >> v2:172.21.15.134:3300/0 conn(0x7f714806d420 0x7f71481527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:17.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f714dc00640 1 -- 172.21.15.67:0/3544059989 >> v1:172.21.15.134:6789/0 conn(0x7f71480c3080 legacy=0x7f714814c2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:53:17.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.591+0000 7f714dc00640 1 -- 172.21.15.67:0/3544059989 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714806dee0 con 0x7f714814c9f0 2024-03-09T08:53:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.593+0000 7f714dc00640 1 --2- 172.21.15.67:0/3544059989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f714814c9f0 0x7f71481540a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f714806e660 tx=0x7f714002be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3464c7f13773195a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.594+0000 7f7147400640 1 -- 172.21.15.67:0/3544059989 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714002ca10 con 0x7f714814c9f0 2024-03-09T08:53:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.594+0000 7f7147400640 1 -- 172.21.15.67:0/3544059989 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714002cbb0 con 0x7f714814c9f0 2024-03-09T08:53:17.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.594+0000 7f7147400640 1 -- 172.21.15.67:0/3544059989 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71400343f0 con 0x7f714814c9f0 2024-03-09T08:53:17.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.594+0000 7f714e600640 1 -- 172.21.15.67:0/3544059989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f714814c9f0 msgr2=0x7f71481540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:17.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.594+0000 7f714e600640 1 --2- 172.21.15.67:0/3544059989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f714814c9f0 0x7f71481540a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f714806e660 tx=0x7f714002be70 comp rx=0 tx=0).stop 2024-03-09T08:53:17.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.594+0000 7f714e600640 1 -- 172.21.15.67:0/3544059989 shutdown_connections 2024-03-09T08:53:17.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.594+0000 7f714e600640 1 --2- 172.21.15.67:0/3544059989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f714814c9f0 0x7f71481540a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:17.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.594+0000 7f714e600640 1 --2- 172.21.15.67:0/3544059989 >> v2:172.21.15.134:3300/0 conn(0x7f714806d420 0x7f71481527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:17.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.594+0000 7f714e600640 1 -- 172.21.15.67:0/3544059989 >> 172.21.15.67:0/3544059989 conn(0x7f71480c7300 msgr2=0x7f71480c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:17.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.594+0000 7f714e600640 1 -- 172.21.15.67:0/3544059989 shutdown_connections 2024-03-09T08:53:17.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.594+0000 7f714e600640 1 -- 172.21.15.67:0/3544059989 wait complete. 2024-03-09T08:53:17.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.596+0000 7f714e600640 1 Processor -- start 2024-03-09T08:53:17.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.596+0000 7f714e600640 1 -- start start 2024-03-09T08:53:17.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.597+0000 7f714e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f714806d420 0x7f714814f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:17.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.597+0000 7f714e600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71480c3080 0x7f714814b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:17.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.597+0000 7f714e600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714814c9f0 0x7f71481dd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:17.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.597+0000 7f714e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480c8030 con 0x7f714806d420 2024-03-09T08:53:17.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.598+0000 7f714e600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480c3460 con 0x7f71480c3080 2024-03-09T08:53:17.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.598+0000 7f714e600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714814f330 con 0x7f714814c9f0 2024-03-09T08:53:17.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.598+0000 7f7147e00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71480c3080 msgr2=0x7f714814b0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:17.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.598+0000 7f7147e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71480c3080 0x7f714814b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:17.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.598+0000 7f714d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f714806d420 0x7f714814f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:17.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.598+0000 7f714d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f714806d420 0x7f714814f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60364/0 (socket says 172.21.15.67:60364) 2024-03-09T08:53:17.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.598+0000 7f714d200640 1 -- 172.21.15.67:0/1765292418 learned_addr learned my addr 172.21.15.67:0/1765292418 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.598+0000 7f714dc00640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714814c9f0 0x7f71481dd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.599+0000 7f714d200640 1 -- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714814c9f0 msgr2=0x7f71481dd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.599+0000 7f714d200640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714814c9f0 0x7f71481dd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.599+0000 7f714d200640 1 -- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71480c3080 msgr2=0x7f714814b0a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.599+0000 7f714d200640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71480c3080 0x7f714814b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.599+0000 7f714d200640 1 -- 172.21.15.67:0/1765292418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71481eb860 con 0x7f714806d420 2024-03-09T08:53:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.599+0000 7f714d200640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f714806d420 0x7f714814f960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f713c00b550 tx=0x7f713c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:17.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.599+0000 7f7144c00640 1 -- 172.21.15.67:0/1765292418 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c013020 con 0x7f714806d420 2024-03-09T08:53:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.599+0000 7f7144c00640 1 -- 172.21.15.67:0/1765292418 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c003de0 con 0x7f714806d420 2024-03-09T08:53:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.599+0000 7f7144c00640 1 -- 172.21.15.67:0/1765292418 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c0052b0 con 0x7f714806d420 2024-03-09T08:53:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.599+0000 7f714e600640 1 -- 172.21.15.67:0/1765292418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71481e9210 con 0x7f714806d420 2024-03-09T08:53:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.600+0000 7f714e600640 1 -- 172.21.15.67:0/1765292418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71480ab9e0 con 0x7f714806d420 2024-03-09T08:53:17.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.601+0000 7f714e600640 1 -- 172.21.15.67:0/1765292418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714804f3c0 con 0x7f714806d420 2024-03-09T08:53:17.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.603+0000 7f7144c00640 1 -- 172.21.15.67:0/1765292418 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f713c016020 con 0x7f714806d420 2024-03-09T08:53:17.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.603+0000 7f7144c00640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712806d690 0x7f712806fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:17.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.604+0000 7f7144c00640 1 -- 172.21.15.67:0/1765292418 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f713c08a490 con 0x7f714806d420 2024-03-09T08:53:17.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.604+0000 7f7147e00640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712806d690 0x7f712806fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:17.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.605+0000 7f7147e00640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712806d690 0x7f712806fb50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7134006fd0 tx=0x7f7134006d80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:17.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.613+0000 7f7144c00640 1 -- 172.21.15.67:0/1765292418 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f713c01d020 con 0x7f714806d420 2024-03-09T08:53:17.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.906+0000 7f714e600640 1 -- 172.21.15.67:0/1765292418 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f71481501f0 con 0x7f714806d420 2024-03-09T08:53:17.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.907+0000 7f7144c00640 1 -- 172.21.15.67:0/1765292418 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7f713c058e40 con 0x7f714806d420 2024-03-09T08:53:17.904 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":84,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:53:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 -- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712806d690 msgr2=0x7f712806fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712806d690 0x7f712806fb50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7134006fd0 tx=0x7f7134006d80 comp rx=0 tx=0).stop 2024-03-09T08:53:17.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 -- 172.21.15.67:0/1765292418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f714806d420 msgr2=0x7f714814f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:17.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f714806d420 0x7f714814f960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f713c00b550 tx=0x7f713c00ba10 comp rx=0 tx=0).stop 2024-03-09T08:53:17.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 -- 172.21.15.67:0/1765292418 shutdown_connections 2024-03-09T08:53:17.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f712806d690 0x7f712806fb50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:17.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f714814c9f0 0x7f71481dd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:17.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71480c3080 0x7f714814b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:17.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 --2- 172.21.15.67:0/1765292418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f714806d420 0x7f714814f960 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:17.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 -- 172.21.15.67:0/1765292418 >> 172.21.15.67:0/1765292418 conn(0x7f71480c7300 msgr2=0x7f71480c6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:17.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 -- 172.21.15.67:0/1765292418 shutdown_connections 2024-03-09T08:53:17.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:17.909+0000 7f714e600640 1 -- 172.21.15.67:0/1765292418 wait complete. 2024-03-09T08:53:20.924 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:53:21.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.156+0000 7ff0ea400640 1 Processor -- start 2024-03-09T08:53:21.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.157+0000 7ff0ea400640 1 -- start start 2024-03-09T08:53:21.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.158+0000 7ff0ea400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4153300 0x7ff0e40b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:21.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.158+0000 7ff0ea400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff0e40cadb0 0x7ff0e40b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:21.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.158+0000 7ff0ea400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e4157000 con 0x7ff0e40c5e40 2024-03-09T08:53:21.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.158+0000 7ff0ea400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e40ae430 con 0x7ff0e40cadb0 2024-03-09T08:53:21.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.158+0000 7ff0ea400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40ae6f0 con 0x7ff0e4153300 2024-03-09T08:53:21.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.159+0000 7ff0e3e00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7ff0e40c5e40 legacy=0x7ff0e4156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:53:21.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.159+0000 7ff0e9a00640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7ff0e40cadb0 msgr2=0x7ff0e40b0540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:21.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.159+0000 7ff0e9a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff0e40cadb0 0x7ff0e40b0540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:21.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.159+0000 7ff0e9000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4153300 0x7ff0e40b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:21.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.159+0000 7ff0e9000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4153300 0x7ff0e40b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:36022/0 (socket says 172.21.15.67:36022) 2024-03-09T08:53:21.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.159+0000 7ff0e9000640 1 -- 172.21.15.67:0/2035916144 learned_addr learned my addr 172.21.15.67:0/2035916144 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:21.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.160+0000 7ff0e9000640 1 -- 172.21.15.67:0/2035916144 >> v2:172.21.15.134:3300/0 conn(0x7ff0e40cadb0 msgr2=0x7ff0e40b0540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:53:21.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.160+0000 7ff0e9000640 1 --2- 172.21.15.67:0/2035916144 >> v2:172.21.15.134:3300/0 conn(0x7ff0e40cadb0 0x7ff0e40b0540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:21.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.160+0000 7ff0e9000640 1 -- 172.21.15.67:0/2035916144 >> v1:172.21.15.134:6789/0 conn(0x7ff0e40c5e40 legacy=0x7ff0e4156010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:53:21.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.160+0000 7ff0e9000640 1 -- 172.21.15.67:0/2035916144 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e4155b10 con 0x7ff0e4153300 2024-03-09T08:53:21.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.162+0000 7ff0e9000640 1 --2- 172.21.15.67:0/2035916144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4153300 0x7ff0e40b0000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009870 tx=0x7ff0d802b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92afca425b53e76 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:21.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.163+0000 7ff0e3400640 1 -- 172.21.15.67:0/2035916144 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8035020 con 0x7ff0e4153300 2024-03-09T08:53:21.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.163+0000 7ff0e3400640 1 -- 172.21.15.67:0/2035916144 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d8031870 con 0x7ff0e4153300 2024-03-09T08:53:21.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.163+0000 7ff0e3400640 1 -- 172.21.15.67:0/2035916144 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8031c40 con 0x7ff0e4153300 2024-03-09T08:53:21.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.163+0000 7ff0ea400640 1 -- 172.21.15.67:0/2035916144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4153300 msgr2=0x7ff0e40b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:21.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.163+0000 7ff0ea400640 1 --2- 172.21.15.67:0/2035916144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4153300 0x7ff0e40b0000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009870 tx=0x7ff0d802b910 comp rx=0 tx=0).stop 2024-03-09T08:53:21.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.164+0000 7ff0ea400640 1 -- 172.21.15.67:0/2035916144 shutdown_connections 2024-03-09T08:53:21.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.164+0000 7ff0ea400640 1 --2- 172.21.15.67:0/2035916144 >> v2:172.21.15.134:3300/0 conn(0x7ff0e40cadb0 0x7ff0e40b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:21.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.164+0000 7ff0ea400640 1 --2- 172.21.15.67:0/2035916144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4153300 0x7ff0e40b0000 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:21.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.164+0000 7ff0ea400640 1 -- 172.21.15.67:0/2035916144 >> 172.21.15.67:0/2035916144 conn(0x7ff0e40732d0 msgr2=0x7ff0e40ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:21.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.164+0000 7ff0ea400640 1 -- 172.21.15.67:0/2035916144 shutdown_connections 2024-03-09T08:53:21.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.164+0000 7ff0ea400640 1 -- 172.21.15.67:0/2035916144 wait complete. 2024-03-09T08:53:21.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.165+0000 7ff0ea400640 1 Processor -- start 2024-03-09T08:53:21.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.165+0000 7ff0ea400640 1 -- start start 2024-03-09T08:53:21.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.166+0000 7ff0ea400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e40c5e40 0x7ff0e41eb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:21.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.166+0000 7ff0ea400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e40cadb0 0x7ff0e4158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.166+0000 7ff0ea400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e4153300 0x7ff0e41ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.166+0000 7ff0ea400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40c59e0 con 0x7ff0e40cadb0 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.166+0000 7ff0ea400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4159e90 con 0x7ff0e4153300 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.166+0000 7ff0ea400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4156720 con 0x7ff0e40c5e40 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e40cadb0 0x7ff0e4158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e40cadb0 0x7ff0e4158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55684/0 (socket says 172.21.15.67:55684) 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e3e00640 1 -- 172.21.15.67:0/3088036444 learned_addr learned my addr 172.21.15.67:0/3088036444 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e9a00640 1 -- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e4153300 msgr2=0x7ff0e41ef6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e9a00640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e4153300 0x7ff0e41ef6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e9000640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e40c5e40 0x7ff0e41eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e3e00640 1 -- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e40c5e40 msgr2=0x7ff0e41eb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e3e00640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e40c5e40 0x7ff0e41eb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e3e00640 1 -- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e4153300 msgr2=0x7ff0e41ef6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e3e00640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e4153300 0x7ff0e41ef6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:21.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e3e00640 1 -- 172.21.15.67:0/3088036444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40ac210 con 0x7ff0e40cadb0 2024-03-09T08:53:21.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.167+0000 7ff0e9000640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e40c5e40 0x7ff0e41eb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:53:21.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.168+0000 7ff0e3e00640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e40cadb0 0x7ff0e4158190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000b550 tx=0x7ff0d000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:21.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.168+0000 7ff0e0c00640 1 -- 172.21.15.67:0/3088036444 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0003d70 con 0x7ff0e40cadb0 2024-03-09T08:53:21.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.168+0000 7ff0e0c00640 1 -- 172.21.15.67:0/3088036444 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d0015070 con 0x7ff0e40cadb0 2024-03-09T08:53:21.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.168+0000 7ff0e0c00640 1 -- 172.21.15.67:0/3088036444 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00052b0 con 0x7ff0e40cadb0 2024-03-09T08:53:21.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.168+0000 7ff0ea400640 1 -- 172.21.15.67:0/3088036444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e40ac4f0 con 0x7ff0e40cadb0 2024-03-09T08:53:21.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.168+0000 7ff0ea400640 1 -- 172.21.15.67:0/3088036444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e41ef1c0 con 0x7ff0e40cadb0 2024-03-09T08:53:21.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.170+0000 7ff0ea400640 1 -- 172.21.15.67:0/3088036444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e40c8d40 con 0x7ff0e40cadb0 2024-03-09T08:53:21.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.170+0000 7ff0e0c00640 1 -- 172.21.15.67:0/3088036444 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff0d0005450 con 0x7ff0e40cadb0 2024-03-09T08:53:21.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.171+0000 7ff0e0c00640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0c406b4b0 0x7ff0c406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:21.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.172+0000 7ff0e0c00640 1 -- 172.21.15.67:0/3088036444 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7ff0d008a3d0 con 0x7ff0e40cadb0 2024-03-09T08:53:21.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.172+0000 7ff0e9000640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0c406b4b0 0x7ff0c406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:21.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.173+0000 7ff0e9000640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0c406b4b0 0x7ff0c406d970 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8036000 tx=0x7ff0d803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:21.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.176+0000 7ff0e0c00640 1 -- 172.21.15.67:0/3088036444 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff0e40c8d40 con 0x7ff0e40cadb0 2024-03-09T08:53:21.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.494+0000 7ff0ea400640 1 -- 172.21.15.67:0/3088036444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff0e404f3c0 con 0x7ff0e40cadb0 2024-03-09T08:53:21.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.494+0000 7ff0e0c00640 1 -- 172.21.15.67:0/3088036444 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7ff0d000e340 con 0x7ff0e40cadb0 2024-03-09T08:53:21.491 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":88,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:53:21.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.497+0000 7ff0ea400640 1 -- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0c406b4b0 msgr2=0x7ff0c406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:21.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.497+0000 7ff0ea400640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0c406b4b0 0x7ff0c406d970 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8036000 tx=0x7ff0d803c040 comp rx=0 tx=0).stop 2024-03-09T08:53:21.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.497+0000 7ff0ea400640 1 -- 172.21.15.67:0/3088036444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e40cadb0 msgr2=0x7ff0e4158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:21.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.497+0000 7ff0ea400640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e40cadb0 0x7ff0e4158190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000b550 tx=0x7ff0d000ba10 comp rx=0 tx=0).stop 2024-03-09T08:53:21.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.497+0000 7ff0ea400640 1 -- 172.21.15.67:0/3088036444 shutdown_connections 2024-03-09T08:53:21.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.497+0000 7ff0ea400640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0c406b4b0 0x7ff0c406d970 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:21.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.497+0000 7ff0ea400640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e4153300 0x7ff0e41ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:21.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.497+0000 7ff0ea400640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e40cadb0 0x7ff0e4158190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:21.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.497+0000 7ff0ea400640 1 --2- 172.21.15.67:0/3088036444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e40c5e40 0x7ff0e41eb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:21.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.497+0000 7ff0ea400640 1 -- 172.21.15.67:0/3088036444 >> 172.21.15.67:0/3088036444 conn(0x7ff0e40732d0 msgr2=0x7ff0e4158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:21.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.498+0000 7ff0ea400640 1 -- 172.21.15.67:0/3088036444 shutdown_connections 2024-03-09T08:53:21.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:21.498+0000 7ff0ea400640 1 -- 172.21.15.67:0/3088036444 wait complete. 2024-03-09T08:53:24.513 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:53:24.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.751+0000 7f5766600640 1 Processor -- start 2024-03-09T08:53:24.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.752+0000 7f5766600640 1 -- start start 2024-03-09T08:53:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.753+0000 7f5766600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f57600c5e40 0x7f5760156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.753+0000 7f5766600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.753+0000 7f5766600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5760156a90 con 0x7f5760153300 2024-03-09T08:53:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.753+0000 7f5766600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5760157000 con 0x7f57600c5e40 2024-03-09T08:53:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.753+0000 7f5766600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57600ae430 con 0x7f57600cadb0 2024-03-09T08:53:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.753+0000 7f575fe00640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f57600c5e40 msgr2=0x7f5760156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.754+0000 7f5765200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5760153300 0x7f57600b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37018/0 (socket says 172.21.15.67:37018) 2024-03-09T08:53:24.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.754+0000 7f5765200640 1 -- 172.21.15.67:0/2525648841 learned_addr learned my addr 172.21.15.67:0/2525648841 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:24.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.754+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.134:3300/0 conn(0x7f57600c5e40 0x7f5760156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:24.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.754+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:24.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.754+0000 7f5765c00640 1 -- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 msgr2=0x7f5760156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:53:24.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.754+0000 7f5765c00640 1 -- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 msgr2=0x7f5760156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:53:24.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.754+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:24.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.754+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:24.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.955+0000 7f575fe00640 1 -- 172.21.15.67:0/2525648841 >> v2:172.21.15.134:3300/0 conn(0x7f57600c5e40 msgr2=0x7f5760156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:24.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.955+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.134:3300/0 conn(0x7f57600c5e40 0x7f5760156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:53:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.955+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.956+0000 7f5765c00640 1 -- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 msgr2=0x7f5760156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:53:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.956+0000 7f5765c00640 1 -- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 msgr2=0x7f5760156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:53:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.956+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:24.956+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:53:25.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:25.356+0000 7f575fe00640 1 -- 172.21.15.67:0/2525648841 >> v2:172.21.15.134:3300/0 conn(0x7f57600c5e40 msgr2=0x7f5760156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:25.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:25.356+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.134:3300/0 conn(0x7f57600c5e40 0x7f5760156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:53:25.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:25.357+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:25.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:25.358+0000 7f5765c00640 1 -- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 msgr2=0x7f5760156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:53:25.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:25.358+0000 7f5765c00640 1 -- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 msgr2=0x7f5760156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:53:25.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:25.358+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:25.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:25.358+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:53:26.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:26.157+0000 7f575fe00640 1 -- 172.21.15.67:0/2525648841 >> v2:172.21.15.134:3300/0 conn(0x7f57600c5e40 msgr2=0x7f5760156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:26.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:26.157+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.134:3300/0 conn(0x7f57600c5e40 0x7f5760156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:53:26.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:26.160+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:26.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:26.160+0000 7f5765c00640 1 -- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 msgr2=0x7f5760156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:53:26.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:26.160+0000 7f5765c00640 1 -- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 msgr2=0x7f5760156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:53:26.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:26.160+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:26.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:26.160+0000 7f5765c00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:53:27.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.753+0000 7f575e000640 1 -- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 msgr2=0x7f5760156550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:53:27.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.753+0000 7f575e000640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600cadb0 0x7f5760156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:27.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.753+0000 7f575e000640 1 -- 172.21.15.67:0/2525648841 >> v2:172.21.15.134:3300/0 conn(0x7f57600c5e40 msgr2=0x7f5760156010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:53:27.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.753+0000 7f575e000640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.134:3300/0 conn(0x7f57600c5e40 0x7f5760156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:27.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.753+0000 7f575e000640 1 -- 172.21.15.67:0/2525648841 >> v1:172.21.15.67:6789/0 conn(0x7f5760153300 legacy=0x7f57600b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:27.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.754+0000 7f575e000640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 0x7f5740009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:27.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.754+0000 7f575e000640 1 -- 172.21.15.67:0/2525648841 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57600ae6f0 con 0x7f57400036c0 2024-03-09T08:53:27.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.754+0000 7f575e000640 1 -- 172.21.15.67:0/2525648841 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5760157000 con 0x7f5740007370 2024-03-09T08:53:27.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.754+0000 7f5765200640 1 -- 172.21.15.67:0/2525648841 >> v1:172.21.15.134:6789/0 conn(0x7f57400036c0 legacy=0x7f5740005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:53:27.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.754+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 0x7f5740009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:27.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.755+0000 7f575fe00640 1 -- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 msgr2=0x7f5740009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:53:27.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.755+0000 7f575fe00640 1 -- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 msgr2=0x7f5740009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:53:27.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.755+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 0x7f5740009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:27.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.755+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 0x7f5740009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:27.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.955+0000 7f5765200640 1 -- 172.21.15.67:0/2525648841 >> v1:172.21.15.134:6789/0 conn(0x7f57400036c0 legacy=0x7f5740005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:53:27.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.956+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 0x7f5740009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:27.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.957+0000 7f575fe00640 1 -- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 msgr2=0x7f5740009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:53:27.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.957+0000 7f575fe00640 1 -- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 msgr2=0x7f5740009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:53:27.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.957+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 0x7f5740009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:27.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:27.957+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 0x7f5740009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:53:28.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.356+0000 7f5765200640 1 -- 172.21.15.67:0/2525648841 >> v1:172.21.15.134:6789/0 conn(0x7f57400036c0 legacy=0x7f5740005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:53:28.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.358+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> v2:172.21.15.67:3300/0 conn(0x7f5740007370 0x7f5740009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.359+0000 7f575fe00640 1 -- 172.21.15.67:0/2525648841 >> v1:172.21.15.134:6789/0 conn(0x7f57400036c0 legacy=0x7f5740005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:53:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.359+0000 7f575fe00640 1 -- 172.21.15.67:0/2525648841 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574000af90 con 0x7f5740007370 2024-03-09T08:53:28.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.362+0000 7f575fe00640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5740007370 0x7f5740009830 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f574c003930 tx=0x7f574c02c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc22691a1dbe1165 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:28.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.363+0000 7f575f400640 1 -- 172.21.15.67:0/2525648841 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c038070 con 0x7f5740007370 2024-03-09T08:53:28.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.363+0000 7f575f400640 1 -- 172.21.15.67:0/2525648841 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c031710 con 0x7f5740007370 2024-03-09T08:53:28.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.363+0000 7f575f400640 1 -- 172.21.15.67:0/2525648841 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c031ac0 con 0x7f5740007370 2024-03-09T08:53:28.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.364+0000 7f5766600640 1 -- 172.21.15.67:0/2525648841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5740007370 msgr2=0x7f5740009830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:28.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.364+0000 7f5766600640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5740007370 0x7f5740009830 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f574c003930 tx=0x7f574c02c760 comp rx=0 tx=0).stop 2024-03-09T08:53:28.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.364+0000 7f5765c00640 1 -- 172.21.15.67:0/2525648841 reap_dead start 2024-03-09T08:53:28.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.364+0000 7f5766600640 1 -- 172.21.15.67:0/2525648841 shutdown_connections 2024-03-09T08:53:28.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.364+0000 7f5766600640 1 --2- 172.21.15.67:0/2525648841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5740007370 0x7f5740009830 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.364+0000 7f5766600640 1 -- 172.21.15.67:0/2525648841 >> 172.21.15.67:0/2525648841 conn(0x7f57600732d0 msgr2=0x7f57600ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:28.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.364+0000 7f5766600640 1 -- 172.21.15.67:0/2525648841 shutdown_connections 2024-03-09T08:53:28.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.365+0000 7f5766600640 1 -- 172.21.15.67:0/2525648841 wait complete. 2024-03-09T08:53:28.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.367+0000 7f5766600640 1 Processor -- start 2024-03-09T08:53:28.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.368+0000 7f5766600640 1 -- start start 2024-03-09T08:53:28.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.369+0000 7f5766600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57600ac260 0x7f576006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.369+0000 7f5766600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57601ebca0 0x7f57600c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.369+0000 7f5766600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600c7120 0x7f576014d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.369+0000 7f5766600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5760159e90 con 0x7f57600ac260 2024-03-09T08:53:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.369+0000 7f5766600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5760159bb0 con 0x7f57601ebca0 2024-03-09T08:53:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.369+0000 7f5766600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57600c59e0 con 0x7f57600c7120 2024-03-09T08:53:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.369+0000 7f575fe00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57601ebca0 msgr2=0x7f57600c6be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.369+0000 7f575fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57601ebca0 0x7f57600c6be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.369+0000 7f5765200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57600ac260 0x7f576006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:28.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.369+0000 7f5765200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57600ac260 0x7f576006b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55712/0 (socket says 172.21.15.67:55712) 2024-03-09T08:53:28.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.369+0000 7f5765200640 1 -- 172.21.15.67:0/4222124632 learned_addr learned my addr 172.21.15.67:0/4222124632 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.370+0000 7f5765c00640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600c7120 0x7f576014d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.370+0000 7f5765200640 1 -- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600c7120 msgr2=0x7f576014d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.370+0000 7f5765200640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600c7120 0x7f576014d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.370+0000 7f5765200640 1 -- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57601ebca0 msgr2=0x7f57600c6be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.370+0000 7f5765200640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57601ebca0 0x7f57600c6be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.370+0000 7f5765200640 1 -- 172.21.15.67:0/4222124632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600c1170 con 0x7f57600ac260 2024-03-09T08:53:28.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.370+0000 7f5765200640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57600ac260 0x7f576006b9e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f575400c550 tx=0x7f575400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:28.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.371+0000 7f575cc00640 1 -- 172.21.15.67:0/4222124632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754017070 con 0x7f57600ac260 2024-03-09T08:53:28.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.371+0000 7f575cc00640 1 -- 172.21.15.67:0/4222124632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5754005ce0 con 0x7f57600ac260 2024-03-09T08:53:28.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.371+0000 7f575cc00640 1 -- 172.21.15.67:0/4222124632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754007270 con 0x7f57600ac260 2024-03-09T08:53:28.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.371+0000 7f5766600640 1 -- 172.21.15.67:0/4222124632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57600c7e60 con 0x7f57600ac260 2024-03-09T08:53:28.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.371+0000 7f5766600640 1 -- 172.21.15.67:0/4222124632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57600c8400 con 0x7f57600ac260 2024-03-09T08:53:28.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.373+0000 7f5766600640 1 -- 172.21.15.67:0/4222124632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576004f3c0 con 0x7f57600ac260 2024-03-09T08:53:28.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.375+0000 7f575cc00640 1 -- 172.21.15.67:0/4222124632 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f575401a070 con 0x7f57600ac260 2024-03-09T08:53:28.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.377+0000 7f575cc00640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f574006c2c0 0x7f574006e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:28.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.377+0000 7f575cc00640 1 -- 172.21.15.67:0/4222124632 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f5754089870 con 0x7f57600ac260 2024-03-09T08:53:28.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.378+0000 7f575fe00640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f574006c2c0 0x7f574006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:28.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.379+0000 7f575fe00640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f574006c2c0 0x7f574006e780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f574c035000 tx=0x7f574c00d030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:28.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.383+0000 7f575cc00640 1 -- 172.21.15.67:0/4222124632 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5754057c60 con 0x7f57600ac260 2024-03-09T08:53:28.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.677+0000 7f5766600640 1 -- 172.21.15.67:0/4222124632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f57600cab90 con 0x7f57600ac260 2024-03-09T08:53:28.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.678+0000 7f575cc00640 1 -- 172.21.15.67:0/4222124632 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1557 (secure 0 0 0) 0x7f575405bd00 con 0x7f57600ac260 2024-03-09T08:53:28.675 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":8,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:53:28.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.680+0000 7f5766600640 1 -- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f574006c2c0 msgr2=0x7f574006e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:28.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.680+0000 7f5766600640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f574006c2c0 0x7f574006e780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f574c035000 tx=0x7f574c00d030 comp rx=0 tx=0).stop 2024-03-09T08:53:28.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.680+0000 7f5766600640 1 -- 172.21.15.67:0/4222124632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57600ac260 msgr2=0x7f576006b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:28.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.680+0000 7f5766600640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57600ac260 0x7f576006b9e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f575400c550 tx=0x7f575400ca10 comp rx=0 tx=0).stop 2024-03-09T08:53:28.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.680+0000 7f5766600640 1 -- 172.21.15.67:0/4222124632 shutdown_connections 2024-03-09T08:53:28.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.680+0000 7f5766600640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f574006c2c0 0x7f574006e780 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.680+0000 7f5766600640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57600c7120 0x7f576014d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.680+0000 7f5766600640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57601ebca0 0x7f57600c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.680+0000 7f5766600640 1 --2- 172.21.15.67:0/4222124632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57600ac260 0x7f576006b9e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.680+0000 7f5766600640 1 -- 172.21.15.67:0/4222124632 >> 172.21.15.67:0/4222124632 conn(0x7f57600732d0 msgr2=0x7f5760151260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:28.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.681+0000 7f5766600640 1 -- 172.21.15.67:0/4222124632 shutdown_connections 2024-03-09T08:53:28.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.681+0000 7f5766600640 1 -- 172.21.15.67:0/4222124632 wait complete. 2024-03-09T08:53:28.705 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T08:53:28.705 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T08:53:28.705 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:53:28.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.910+0000 7f0e34c00640 1 Processor -- start 2024-03-09T08:53:28.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.910+0000 7f0e34c00640 1 -- start start 2024-03-09T08:53:28.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.912+0000 7f0e34c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0e30151360 0x7f0e300c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:28.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.912+0000 7f0e34c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e300c2780 con 0x7f0e300c30a0 2024-03-09T08:53:28.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.912+0000 7f0e34c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e300708c0 con 0x7f0e300ac930 2024-03-09T08:53:28.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.912+0000 7f0e34c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e30070670 con 0x7f0e30151360 2024-03-09T08:53:28.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.912+0000 7f0e2f400640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f0e300ac930 legacy=0x7f0e30150c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:53:28.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.912+0000 7f0e27e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0e30151360 0x7f0e300c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:28.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.912+0000 7f0e2fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0e300c30a0 0x7f0e301539e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59294/0 (socket says 172.21.15.67:59294) 2024-03-09T08:53:28.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.912+0000 7f0e2fe00640 1 -- 172.21.15.67:0/4142926848 learned_addr learned my addr 172.21.15.67:0/4142926848 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:28.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.913+0000 7f0e27e00640 1 -- 172.21.15.67:0/4142926848 >> v1:172.21.15.134:6789/0 conn(0x7f0e300ac930 legacy=0x7f0e30150c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:53:28.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.913+0000 7f0e27e00640 1 -- 172.21.15.67:0/4142926848 >> v1:172.21.15.67:6789/0 conn(0x7f0e300c30a0 legacy=0x7f0e301539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.913+0000 7f0e27e00640 1 -- 172.21.15.67:0/4142926848 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e30070ba0 con 0x7f0e30151360 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.913+0000 7f0e2ea00640 1 -- 172.21.15.67:0/4142926848 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648802263 0 0) 0x7f0e300c2780 con 0x7f0e300c30a0 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.914+0000 7f0e27e00640 1 --2- 172.21.15.67:0/4142926848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e30151360 0x7f0e300c3660 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0e180097a0 tx=0x7f0e1802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2101dd797edfd6c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.915+0000 7f0e2ea00640 1 -- 172.21.15.67:0/4142926848 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18035020 con 0x7f0e30151360 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.915+0000 7f0e2ea00640 1 -- 172.21.15.67:0/4142926848 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e18031830 con 0x7f0e30151360 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.915+0000 7f0e2ea00640 1 -- 172.21.15.67:0/4142926848 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18031c00 con 0x7f0e30151360 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.915+0000 7f0e34c00640 1 -- 172.21.15.67:0/4142926848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e30151360 msgr2=0x7f0e300c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.915+0000 7f0e34c00640 1 --2- 172.21.15.67:0/4142926848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e30151360 0x7f0e300c3660 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0e180097a0 tx=0x7f0e1802b8d0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.916+0000 7f0e34c00640 1 -- 172.21.15.67:0/4142926848 shutdown_connections 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.916+0000 7f0e34c00640 1 --2- 172.21.15.67:0/4142926848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e30151360 0x7f0e300c3660 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.916+0000 7f0e34c00640 1 -- 172.21.15.67:0/4142926848 >> 172.21.15.67:0/4142926848 conn(0x7f0e300c7220 msgr2=0x7f0e300c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.916+0000 7f0e34c00640 1 -- 172.21.15.67:0/4142926848 shutdown_connections 2024-03-09T08:53:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.916+0000 7f0e34c00640 1 -- 172.21.15.67:0/4142926848 wait complete. 2024-03-09T08:53:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.918+0000 7f0e34c00640 1 Processor -- start 2024-03-09T08:53:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.918+0000 7f0e34c00640 1 -- start start 2024-03-09T08:53:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.919+0000 7f0e34c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e300ac930 0x7f0e3014b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.919+0000 7f0e34c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c30a0 0x7f0e30152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.919+0000 7f0e34c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e30151360 0x7f0e301483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.919+0000 7f0e34c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300ace90 con 0x7f0e300c30a0 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.919+0000 7f0e34c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e300acd10 con 0x7f0e30151360 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.919+0000 7f0e34c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301540f0 con 0x7f0e300ac930 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.919+0000 7f0e2fe00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e30151360 msgr2=0x7f0e301483e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.920+0000 7f0e27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c30a0 0x7f0e30152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.920+0000 7f0e2fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e30151360 0x7f0e301483e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.920+0000 7f0e27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c30a0 0x7f0e30152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58838/0 (socket says 172.21.15.67:58838) 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.920+0000 7f0e27e00640 1 -- 172.21.15.67:0/1650665191 learned_addr learned my addr 172.21.15.67:0/1650665191 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.920+0000 7f0e2f400640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e300ac930 0x7f0e3014b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.920+0000 7f0e27e00640 1 -- 172.21.15.67:0/1650665191 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e300ac930 msgr2=0x7f0e3014b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.920+0000 7f0e27e00640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e300ac930 0x7f0e3014b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.920+0000 7f0e27e00640 1 -- 172.21.15.67:0/1650665191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e30151360 msgr2=0x7f0e301483e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:28.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.920+0000 7f0e27e00640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e30151360 0x7f0e301483e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.920+0000 7f0e27e00640 1 -- 172.21.15.67:0/1650665191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e301ef250 con 0x7f0e300c30a0 2024-03-09T08:53:28.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.920+0000 7f0e27e00640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c30a0 0x7f0e30152b10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0e180098c0 tx=0x7f0e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:28.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.921+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1803aa70 con 0x7f0e300c30a0 2024-03-09T08:53:28.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.921+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1803ac10 con 0x7f0e300c30a0 2024-03-09T08:53:28.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.921+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e180443f0 con 0x7f0e300c30a0 2024-03-09T08:53:28.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.921+0000 7f0e34c00640 1 -- 172.21.15.67:0/1650665191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e300c62b0 con 0x7f0e300c30a0 2024-03-09T08:53:28.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.921+0000 7f0e34c00640 1 -- 172.21.15.67:0/1650665191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e30153050 con 0x7f0e300c30a0 2024-03-09T08:53:28.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.922+0000 7f0e34c00640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e30148920 0x7f0e300c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:28.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.922+0000 7f0e34c00640 1 -- 172.21.15.67:0/1650665191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3004f3c0 con 0x7f0e30148920 2024-03-09T08:53:28.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.922+0000 7f0e2f400640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e30148920 0x7f0e300c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:28.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.923+0000 7f0e2f400640 1 -- 172.21.15.67:0/1650665191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e300c67d0 con 0x7f0e30148920 2024-03-09T08:53:28.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.923+0000 7f0e2f400640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e30148920 0x7f0e300c4e10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000d700 tx=0x7f0e2000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:28.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.923+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0e18038020 con 0x7f0e300c30a0 2024-03-09T08:53:28.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.924+0000 7f0e27400640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e0c06b4b0 0x7f0e0c06d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:28.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.924+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e200039b0 con 0x7f0e30148920 2024-03-09T08:53:28.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.924+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f0e180ac2e0 con 0x7f0e300c30a0 2024-03-09T08:53:28.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.924+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f0e20003b50 con 0x7f0e30148920 2024-03-09T08:53:28.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.924+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e30148920 msgr2=0x7f0e300c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:28.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.924+0000 7f0e27400640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e30148920 0x7f0e300c4e10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000d700 tx=0x7f0e2000dbc0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.924+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e30148920 msgr2=0x7f0e300c4e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:28.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.924+0000 7f0e27400640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e30148920 0x7f0e300c4e10 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000d700 tx=0x7f0e2000dbc0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.925+0000 7f0e2fe00640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e0c06b4b0 0x7f0e0c06d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:28.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.926+0000 7f0e2fe00640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e0c06b4b0 0x7f0e0c06d970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0e28001460 tx=0x7f0e28001760 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:28.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.932+0000 7f0e34c00640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c5350 0x7f0e301eb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:28.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.932+0000 7f0e34c00640 1 -- 172.21.15.67:0/1650665191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30067b30 con 0x7f0e300c5350 2024-03-09T08:53:28.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.932+0000 7f0e2f400640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c5350 0x7f0e301eb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:28.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.933+0000 7f0e2f400640 1 -- 172.21.15.67:0/1650665191 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0e300c38b0 con 0x7f0e300c5350 2024-03-09T08:53:28.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.933+0000 7f0e2f400640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c5350 0x7f0e301eb1a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000b550 tx=0x7f0e200059d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:28.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.933+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e200039b0 con 0x7f0e300c5350 2024-03-09T08:53:28.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.933+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5176 (secure 0 0 0) 0x7f0e2000be40 con 0x7f0e300c5350 2024-03-09T08:53:28.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.933+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c5350 msgr2=0x7f0e301eb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:28.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.933+0000 7f0e27400640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c5350 0x7f0e301eb1a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000b550 tx=0x7f0e200059d0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.933+0000 7f0e27400640 1 -- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c5350 msgr2=0x7f0e301eb1a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:28.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.933+0000 7f0e27400640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c5350 0x7f0e301eb1a0 secure :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000b550 tx=0x7f0e200059d0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.931 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:53:28.931 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:53:28.931 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:53:28.931 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:53:28.931 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-03-09T08:53:28.931 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:53:28.931 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:53:28.931 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:53:28.931 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:28.931 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:28.932 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:28.933 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:53:28.934 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:28.936 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:28.937 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.938 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:53:28.939 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:53:28.940 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:28.941 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:28.942 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:28.942 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:28.942 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:53:28.942 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.942 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:28.942 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:28.942 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:53:28.942 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:28.942 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:28.942 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:28.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:28.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:28.945 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:53:28.945 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:53:28.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.934+0000 7f0e34c00640 1 -- 172.21.15.67:0/1650665191 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e0c06b4b0 msgr2=0x7f0e0c06d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.934+0000 7f0e34c00640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e0c06b4b0 0x7f0e0c06d970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0e28001460 tx=0x7f0e28001760 comp rx=0 tx=0).stop 2024-03-09T08:53:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.934+0000 7f0e34c00640 1 -- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c30a0 msgr2=0x7f0e30152b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.934+0000 7f0e34c00640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c30a0 0x7f0e30152b10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0e180098c0 tx=0x7f0e180046c0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.934+0000 7f0e2fe00640 1 -- 172.21.15.67:0/1650665191 reap_dead start 2024-03-09T08:53:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.934+0000 7f0e34c00640 1 -- 172.21.15.67:0/1650665191 shutdown_connections 2024-03-09T08:53:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.934+0000 7f0e34c00640 1 --2- 172.21.15.67:0/1650665191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e300c30a0 0x7f0e30152b10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.934+0000 7f0e34c00640 1 -- 172.21.15.67:0/1650665191 >> 172.21.15.67:0/1650665191 conn(0x7f0e300c7220 msgr2=0x7f0e30072940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.934+0000 7f0e34c00640 1 -- 172.21.15.67:0/1650665191 shutdown_connections 2024-03-09T08:53:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:28.934+0000 7f0e34c00640 1 -- 172.21.15.67:0/1650665191 wait complete. 2024-03-09T08:53:28.952 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:53:28.952 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T08:53:28.952 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:53:29.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.171+0000 7f7dd2600640 1 Processor -- start 2024-03-09T08:53:29.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.172+0000 7f7dd2600640 1 -- start start 2024-03-09T08:53:29.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.173+0000 7f7dd2600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0acf00 0x7f7dcc0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:29.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.173+0000 7f7dd2600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7dcc0c9380 0x7f7dcc156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:29.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.173+0000 7f7dd2600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dcc0c9810 con 0x7f7dcc0c5e00 2024-03-09T08:53:29.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.173+0000 7f7dd2600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dcc06d740 con 0x7f7dcc0c9380 2024-03-09T08:53:29.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.173+0000 7f7dd2600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc153680 con 0x7f7dcc0acf00 2024-03-09T08:53:29.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.174+0000 7f7dd1200640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f7dcc0c5e00 legacy=0x7f7dcc06c5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:53:29.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.174+0000 7f7dd1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7dcc0c9380 0x7f7dcc156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:29.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.174+0000 7f7dd1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7dcc0c9380 0x7f7dcc156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58874/0 (socket says 172.21.15.67:58874) 2024-03-09T08:53:29.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.174+0000 7f7dd1c00640 1 -- 172.21.15.67:0/1255003258 learned_addr learned my addr 172.21.15.67:0/1255003258 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:29.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.174+0000 7f7dcbe00640 1 --2- 172.21.15.67:0/1255003258 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0acf00 0x7f7dcc0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:29.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.174+0000 7f7dd1c00640 1 -- 172.21.15.67:0/1255003258 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0acf00 msgr2=0x7f7dcc0c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:29.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.174+0000 7f7dd1c00640 1 --2- 172.21.15.67:0/1255003258 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0acf00 0x7f7dcc0c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:29.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.174+0000 7f7dd1c00640 1 -- 172.21.15.67:0/1255003258 >> v1:172.21.15.134:6789/0 conn(0x7f7dcc0c5e00 legacy=0x7f7dcc06c5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:53:29.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.174+0000 7f7dd1c00640 1 -- 172.21.15.67:0/1255003258 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc0ad7c0 con 0x7f7dcc0c9380 2024-03-09T08:53:29.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.175+0000 7f7dd1c00640 1 --2- 172.21.15.67:0/1255003258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dcc0c9380 0x7f7dcc156660 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009e80 tx=0x7f7dc402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952954ae444a2a7d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:29.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.176+0000 7f7dcb400640 1 -- 172.21.15.67:0/1255003258 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc402ca50 con 0x7f7dcc0c9380 2024-03-09T08:53:29.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.176+0000 7f7dcb400640 1 -- 172.21.15.67:0/1255003258 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc402cbf0 con 0x7f7dcc0c9380 2024-03-09T08:53:29.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.176+0000 7f7dcb400640 1 -- 172.21.15.67:0/1255003258 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40343f0 con 0x7f7dcc0c9380 2024-03-09T08:53:29.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.176+0000 7f7dd2600640 1 -- 172.21.15.67:0/1255003258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dcc0c9380 msgr2=0x7f7dcc156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:29.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.176+0000 7f7dd2600640 1 --2- 172.21.15.67:0/1255003258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dcc0c9380 0x7f7dcc156660 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009e80 tx=0x7f7dc402beb0 comp rx=0 tx=0).stop 2024-03-09T08:53:29.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.176+0000 7f7dd2600640 1 -- 172.21.15.67:0/1255003258 shutdown_connections 2024-03-09T08:53:29.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.177+0000 7f7dd2600640 1 --2- 172.21.15.67:0/1255003258 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dcc0c9380 0x7f7dcc156660 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:29.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.177+0000 7f7dd2600640 1 --2- 172.21.15.67:0/1255003258 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0acf00 0x7f7dcc0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:29.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.177+0000 7f7dd2600640 1 -- 172.21.15.67:0/1255003258 >> 172.21.15.67:0/1255003258 conn(0x7f7dcc0cb480 msgr2=0x7f7dcc0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:29.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.177+0000 7f7dd2600640 1 -- 172.21.15.67:0/1255003258 shutdown_connections 2024-03-09T08:53:29.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.177+0000 7f7dd2600640 1 -- 172.21.15.67:0/1255003258 wait complete. 2024-03-09T08:53:29.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.178+0000 7f7dd2600640 1 Processor -- start 2024-03-09T08:53:29.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.178+0000 7f7dd2600640 1 -- start start 2024-03-09T08:53:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.179+0000 7f7dd2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dcc0acf00 0x7f7dcc1eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.179+0000 7f7dd2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dcc0c5e00 0x7f7dcc0bf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.179+0000 7f7dd2600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0c9380 0x7f7dcc0bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.179+0000 7f7dd2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc06bde0 con 0x7f7dcc0c5e00 2024-03-09T08:53:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.179+0000 7f7dd2600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc06bb00 con 0x7f7dcc0acf00 2024-03-09T08:53:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.179+0000 7f7dd2600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc06c0c0 con 0x7f7dcc0c9380 2024-03-09T08:53:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.179+0000 7f7dd1200640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dcc0acf00 msgr2=0x7f7dcc1eecf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:53:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.180+0000 7f7dd1200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dcc0acf00 0x7f7dcc1eecf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:29.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.180+0000 7f7dcbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dcc0c5e00 0x7f7dcc0bf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:29.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.180+0000 7f7dcbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dcc0c5e00 0x7f7dcc0bf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58878/0 (socket says 172.21.15.67:58878) 2024-03-09T08:53:29.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.180+0000 7f7dcbe00640 1 -- 172.21.15.67:0/1681107279 learned_addr learned my addr 172.21.15.67:0/1681107279 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.180+0000 7f7dcbe00640 1 -- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0c9380 msgr2=0x7f7dcc0bfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.180+0000 7f7dd1c00640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0c9380 0x7f7dcc0bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.180+0000 7f7dcbe00640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0c9380 0x7f7dcc0bfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.180+0000 7f7dcbe00640 1 -- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dcc0acf00 msgr2=0x7f7dcc1eecf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.180+0000 7f7dcbe00640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7dcc0acf00 0x7f7dcc1eecf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.180+0000 7f7dcbe00640 1 -- 172.21.15.67:0/1681107279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc0bebe0 con 0x7f7dcc0c5e00 2024-03-09T08:53:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.180+0000 7f7dcbe00640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dcc0c5e00 0x7f7dcc0bf8f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7db800d620 tx=0x7f7db800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.181+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db800b620 con 0x7f7dcc0c5e00 2024-03-09T08:53:29.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.181+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db8019070 con 0x7f7dcc0c5e00 2024-03-09T08:53:29.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.181+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db800ba00 con 0x7f7dcc0c5e00 2024-03-09T08:53:29.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.181+0000 7f7dd2600640 1 -- 172.21.15.67:0/1681107279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc1ef770 con 0x7f7dcc0c5e00 2024-03-09T08:53:29.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.181+0000 7f7dd2600640 1 -- 172.21.15.67:0/1681107279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc0c24c0 con 0x7f7dcc0c5e00 2024-03-09T08:53:29.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.182+0000 7f7dd2600640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0c5190 0x7f7dcc1ef230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:29.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.182+0000 7f7dd2600640 1 -- 172.21.15.67:0/1681107279 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc0c1480 con 0x7f7dcc0c5190 2024-03-09T08:53:29.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.183+0000 7f7dd1200640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0c5190 0x7f7dcc1ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:29.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.183+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7db800bba0 con 0x7f7dcc0c5e00 2024-03-09T08:53:29.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.184+0000 7f7dd1200640 1 -- 172.21.15.67:0/1681107279 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7dcc0be990 con 0x7f7dcc0c5190 2024-03-09T08:53:29.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.184+0000 7f7dd1200640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0c5190 0x7f7dcc1ef230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc159550 tx=0x7f7dc000b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:29.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.185+0000 7f7dc8c00640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7dac06b480 0x7f7dac06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:29.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.185+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f7db808aa20 con 0x7f7dcc0c5e00 2024-03-09T08:53:29.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.185+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0003e10 con 0x7f7dcc0c5190 2024-03-09T08:53:29.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.186+0000 7f7dd1c00640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7dac06b480 0x7f7dac06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:29.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.186+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f7dc0015070 con 0x7f7dcc0c5190 2024-03-09T08:53:29.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.186+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0c5190 msgr2=0x7f7dcc1ef230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:29.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.186+0000 7f7dc8c00640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0c5190 0x7f7dcc1ef230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc159550 tx=0x7f7dc000b8c0 comp rx=0 tx=0).stop 2024-03-09T08:53:29.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.186+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0c5190 msgr2=0x7f7dcc1ef230 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:29.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.186+0000 7f7dc8c00640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc0c5190 0x7f7dcc1ef230 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc159550 tx=0x7f7dc000b8c0 comp rx=0 tx=0).stop 2024-03-09T08:53:29.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.187+0000 7f7dd1c00640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7dac06b480 0x7f7dac06d940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4035000 tx=0x7f7dc4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:29.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.200+0000 7f7dd2600640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc1eb510 0x7f7dcc1eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:29.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.200+0000 7f7dd2600640 1 -- 172.21.15.67:0/1681107279 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc0c0370 con 0x7f7dcc1eb510 2024-03-09T08:53:29.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.201+0000 7f7dd1200640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc1eb510 0x7f7dcc1eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:29.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.202+0000 7f7dd1200640 1 -- 172.21.15.67:0/1681107279 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7dcc04f3c0 con 0x7f7dcc1eb510 2024-03-09T08:53:29.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.202+0000 7f7dd1200640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc1eb510 0x7f7dcc1eb8f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc159550 tx=0x7f7dc0005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:29.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.203+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0005550 con 0x7f7dcc1eb510 2024-03-09T08:53:29.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.204+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5174 (secure 0 0 0) 0x7f7dc0005550 con 0x7f7dcc1eb510 2024-03-09T08:53:29.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.204+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc1eb510 msgr2=0x7f7dcc1eb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:29.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.204+0000 7f7dc8c00640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc1eb510 0x7f7dcc1eb8f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc159550 tx=0x7f7dc0005a20 comp rx=0 tx=0).stop 2024-03-09T08:53:29.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.204+0000 7f7dc8c00640 1 -- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc1eb510 msgr2=0x7f7dcc1eb8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:29.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.204+0000 7f7dc8c00640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7dcc1eb510 0x7f7dcc1eb8f0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc159550 tx=0x7f7dc0005a20 comp rx=0 tx=0).stop 2024-03-09T08:53:29.201 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:53:29.201 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:53:29.201 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:53:29.201 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:53:29.201 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 8, 2024-03-09T08:53:29.201 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:53:29.201 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:53:29.201 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:53:29.201 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:29.201 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:29.202 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:29.203 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:53:29.204 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:29.205 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:53:29.206 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:29.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:29.208 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:29.209 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:53:29.210 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:29.211 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:29.212 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:29.214 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:29.214 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:29.214 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:29.214 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:29.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:29.214 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:53:29.214 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:53:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.204+0000 7f7dd2600640 1 -- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7dac06b480 msgr2=0x7f7dac06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.204+0000 7f7dd2600640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7dac06b480 0x7f7dac06d940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4035000 tx=0x7f7dc4030000 comp rx=0 tx=0).stop 2024-03-09T08:53:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.204+0000 7f7dd2600640 1 -- 172.21.15.67:0/1681107279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dcc0c5e00 msgr2=0x7f7dcc0bf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.204+0000 7f7dd2600640 1 --2- 172.21.15.67:0/1681107279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7dcc0c5e00 0x7f7dcc0bf8f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7db800d620 tx=0x7f7db800dae0 comp rx=0 tx=0).stop 2024-03-09T08:53:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.204+0000 7f7dd1c00640 1 -- 172.21.15.67:0/1681107279 reap_dead start 2024-03-09T08:53:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.205+0000 7f7dd1c00640 1 -- 172.21.15.67:0/1681107279 reap_dead start 2024-03-09T08:53:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.205+0000 7f7dd2600640 1 -- 172.21.15.67:0/1681107279 shutdown_connections 2024-03-09T08:53:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.205+0000 7f7dd2600640 1 -- 172.21.15.67:0/1681107279 >> 172.21.15.67:0/1681107279 conn(0x7f7dcc0cb480 msgr2=0x7f7dcc0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.205+0000 7f7dd2600640 1 -- 172.21.15.67:0/1681107279 shutdown_connections 2024-03-09T08:53:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:29.205+0000 7f7dd2600640 1 -- 172.21.15.67:0/1681107279 wait complete. 2024-03-09T08:53:29.220 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:53:29.221 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T08:53:39.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:53:39.099 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:53:49.222 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T08:53:49.222 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-03-09T08:53:49.222 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-09T08:53:49.222 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-09T08:53:49.233 INFO:tasks.ceph.mon.b:Started 2024-03-09T08:53:49.233 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:53:49.233 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:53:49.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.460+0000 7f9a00a00640 1 Processor -- start 2024-03-09T08:53:49.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.460+0000 7f9a00a00640 1 -- start start 2024-03-09T08:53:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.461+0000 7f9a00a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f99fc0c1f70 0x7f99fc150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.461+0000 7f9a00a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0710a0 0x7f99fc151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.461+0000 7f9a00a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99fc0c81e0 con 0x7f99fc0c27a0 2024-03-09T08:53:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.461+0000 7f9a00a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99fc0c32b0 con 0x7f99fc0c1f70 2024-03-09T08:53:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.461+0000 7f9a00a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0c2fd0 con 0x7f99fc0710a0 2024-03-09T08:53:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.462+0000 7f99fa800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f99fc0c1f70 0x7f99fc150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.462+0000 7f99f1e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0710a0 0x7f99fc151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:49.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.462+0000 7f99f1e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0710a0 0x7f99fc151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:52288/0 (socket says 172.21.15.67:52288) 2024-03-09T08:53:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.462+0000 7f99f1e00640 1 -- 172.21.15.67:0/36794414 learned_addr learned my addr 172.21.15.67:0/36794414 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.462+0000 7f99fb200640 1 --1- 172.21.15.67:0/36794414 >> v1:172.21.15.134:6789/0 conn(0x7f99fc0c27a0 0x7f99fc153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:46498/0 (socket says 172.21.15.67:46498) 2024-03-09T08:53:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.462+0000 7f99fa800640 1 -- 172.21.15.67:0/36794414 >> v2:172.21.15.134:3300/0 conn(0x7f99fc0c1f70 msgr2=0x7f99fc150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:53:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.462+0000 7f99fa800640 1 -- 172.21.15.67:0/36794414 >> v2:172.21.15.134:3300/0 conn(0x7f99fc0c1f70 msgr2=0x7f99fc150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:53:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.462+0000 7f99fa800640 1 --2- 172.21.15.67:0/36794414 >> v2:172.21.15.134:3300/0 conn(0x7f99fc0c1f70 0x7f99fc150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.462+0000 7f99fa800640 1 --2- 172.21.15.67:0/36794414 >> v2:172.21.15.134:3300/0 conn(0x7f99fc0c1f70 0x7f99fc150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:49.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.463+0000 7f99f1e00640 1 -- 172.21.15.67:0/36794414 >> v2:172.21.15.134:3300/0 conn(0x7f99fc0c1f70 msgr2=0x7f99fc150c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:53:49.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.463+0000 7f99f1e00640 1 --2- 172.21.15.67:0/36794414 >> v2:172.21.15.134:3300/0 conn(0x7f99fc0c1f70 0x7f99fc150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:49.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.463+0000 7f99f1e00640 1 -- 172.21.15.67:0/36794414 >> v1:172.21.15.134:6789/0 conn(0x7f99fc0c27a0 legacy=0x7f99fc153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:49.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.463+0000 7f99f1e00640 1 -- 172.21.15.67:0/36794414 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc0c3700 con 0x7f99fc0710a0 2024-03-09T08:53:49.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.465+0000 7f99f1e00640 1 --2- 172.21.15.67:0/36794414 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0710a0 0x7f99fc151190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009870 tx=0x7f99e402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7979dfb47ab64fe server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:49.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.467+0000 7f99f9e00640 1 -- 172.21.15.67:0/36794414 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4035020 con 0x7f99fc0710a0 2024-03-09T08:53:49.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.467+0000 7f99f9e00640 1 -- 172.21.15.67:0/36794414 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e4031640 con 0x7f99fc0710a0 2024-03-09T08:53:49.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.467+0000 7f99f9e00640 1 -- 172.21.15.67:0/36794414 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e402d040 con 0x7f99fc0710a0 2024-03-09T08:53:49.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.467+0000 7f9a00a00640 1 -- 172.21.15.67:0/36794414 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0710a0 msgr2=0x7f99fc151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.467+0000 7f9a00a00640 1 --2- 172.21.15.67:0/36794414 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0710a0 0x7f99fc151190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009870 tx=0x7f99e402b8f0 comp rx=0 tx=0).stop 2024-03-09T08:53:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.468+0000 7f9a00a00640 1 -- 172.21.15.67:0/36794414 shutdown_connections 2024-03-09T08:53:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.468+0000 7f9a00a00640 1 --2- 172.21.15.67:0/36794414 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0710a0 0x7f99fc151190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.468+0000 7f9a00a00640 1 --2- 172.21.15.67:0/36794414 >> v2:172.21.15.134:3300/0 conn(0x7f99fc0c1f70 0x7f99fc150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.468+0000 7f9a00a00640 1 -- 172.21.15.67:0/36794414 >> 172.21.15.67:0/36794414 conn(0x7f99fc0c7300 msgr2=0x7f99fc0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.468+0000 7f9a00a00640 1 -- 172.21.15.67:0/36794414 shutdown_connections 2024-03-09T08:53:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.468+0000 7f9a00a00640 1 -- 172.21.15.67:0/36794414 wait complete. 2024-03-09T08:53:49.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.469+0000 7f9a00a00640 1 Processor -- start 2024-03-09T08:53:49.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.470+0000 7f9a00a00640 1 -- start start 2024-03-09T08:53:49.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.470+0000 7f9a00a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:49.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.470+0000 7f9a00a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:49.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.470+0000 7f9a00a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:49.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.470+0000 7f9a00a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc070d50 con 0x7f99fc0c27a0 2024-03-09T08:53:49.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.470+0000 7f9a00a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc070830 con 0x7f99fc0710a0 2024-03-09T08:53:49.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.470+0000 7f9a00a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc070ed0 con 0x7f99fc0c1f70 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99fb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99f1e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99f1e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:52300/0 (socket says 172.21.15.67:52300) 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 learned_addr learned my addr 172.21.15.67:0/622431216 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99fa800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99fc1e2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99fc1e2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99fc149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99fc149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99fc1dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:53:49.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99fc1dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:49.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:49.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:49.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.471+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.672+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.672+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.672+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99fc149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:53:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99fc149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:53:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99fc1e2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T08:53:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99fc1e2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99fc1dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T08:53:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99fc1dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:53:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:49.673+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:53:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.074+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.075+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99fc149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T08:53:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.075+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99fc149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.075+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.075+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.075+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:53:50.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.075+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:50.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.076+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99fc1e2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:53:50.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.076+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99fc1e2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:50.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.076+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:50.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.076+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:53:50.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.076+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99fc1dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T08:53:50.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.076+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99fc1dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:50.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.076+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:50.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.077+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:53:50.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.877+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:50.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.877+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99fc149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:53:50.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.877+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99fc149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:50.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.877+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:50.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.878+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:53:50.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.878+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:50.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.879+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:50.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.879+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99fc1e2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T08:53:50.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.879+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99fc1e2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:50.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.879+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:50.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.879+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:53:50.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.879+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99fc1dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:53:50.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.879+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99fc1dd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:50.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.879+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:50.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:50.879+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:53:52.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99fc1e2750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:52.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99fc1e2750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99fc1dd790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99fc1dd790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99fc149de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99fc149de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc070d50 con 0x7f99fc0c27a0 2024-03-09T08:53:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc070830 con 0x7f99fc0710a0 2024-03-09T08:53:52.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.471+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc070ed0 con 0x7f99fc0c1f70 2024-03-09T08:53:52.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.472+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:52.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.472+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:52.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.472+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:52.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99d800a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T08:53:52.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99d800a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99d8007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T08:53:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99d8007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:53:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:52.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.473+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:52.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.674+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:52.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.675+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.675+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.675+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99d800a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.675+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99d800a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.675+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.676+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99d8007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.676+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99d8007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.676+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.676+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.676+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.676+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.676+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.676+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:52.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:52.676+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:53:53.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.077+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:53.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.077+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:53.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:53.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99d800a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:53:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T08:53:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99d800a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99d8007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T08:53:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:53.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99d8007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:53:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:53:53.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.078+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:53:53.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.879+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:53.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:53.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:53.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99d800a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:53:53.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99d800a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:53.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T08:53:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:53:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:53:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99d8007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T08:53:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.880+0000 7f99f1e00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99d8007c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.881+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:53.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:53.881+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:53:55.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.471+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99d8007c80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:55.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.471+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d8007c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:55.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.471+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80056b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.471+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80056b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.471+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 msgr2=0x7f99d800a250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.471+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99fc0c27a0 0x7f99d800a250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.471+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d800a990 0x7f99d800cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.472+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d80081e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.472+0000 7f99f1400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.472+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc070d50 con 0x7f99d800a990 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.472+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc070830 con 0x7f99fc0710a0 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.472+0000 7f99f1400640 1 -- 172.21.15.67:0/622431216 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc070ed0 con 0x7f99fc0c1f70 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.472+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d800a990 0x7f99d800cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.472+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d80081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:55.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.473+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:55.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.473+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T08:53:55.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.473+0000 7f99fb200640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:55.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.473+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:55.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.473+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 msgr2=0x7f99d80081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:55.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.473+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d80081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:55.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.473+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 msgr2=0x7f99d80036a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:55.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.473+0000 7f99fb200640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80036a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:55.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.474+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80036a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:55.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.474+0000 7f99fa800640 1 -- 172.21.15.67:0/622431216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc0c6a10 con 0x7f99d800a990 2024-03-09T08:53:55.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.474+0000 7f99fa800640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d800a990 0x7f99d800cd80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f99ec004660 tx=0x7f99ec01a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:55.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.475+0000 7f99f3400640 1 -- 172.21.15.67:0/622431216 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec024420 con 0x7f99d800a990 2024-03-09T08:53:55.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.475+0000 7f99f3400640 1 -- 172.21.15.67:0/622431216 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec0245c0 con 0x7f99d800a990 2024-03-09T08:53:55.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.475+0000 7f99f3400640 1 -- 172.21.15.67:0/622431216 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0247b0 con 0x7f99d800a990 2024-03-09T08:53:55.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.475+0000 7f9a00a00640 1 -- 172.21.15.67:0/622431216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc14aa80 con 0x7f99d800a990 2024-03-09T08:53:55.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.476+0000 7f9a00a00640 1 -- 172.21.15.67:0/622431216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc1de930 con 0x7f99d800a990 2024-03-09T08:53:55.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.477+0000 7f9a00a00640 1 -- 172.21.15.67:0/622431216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc04f3c0 con 0x7f99d800a990 2024-03-09T08:53:55.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.479+0000 7f99f3400640 1 -- 172.21.15.67:0/622431216 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f99ec015070 con 0x7f99d800a990 2024-03-09T08:53:55.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.481+0000 7f99f3400640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99d406b4d0 0x7f99d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:55.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.481+0000 7f99f3400640 1 -- 172.21.15.67:0/622431216 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f99ec08c430 con 0x7f99d800a990 2024-03-09T08:53:55.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.482+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99d406b4d0 0x7f99d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:55.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.484+0000 7f99f1e00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99d406b4d0 0x7f99d406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0734d0 tx=0x7f99e4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:55.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.485+0000 7f99f3400640 1 -- 172.21.15.67:0/622431216 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f99ec05afa0 con 0x7f99d800a990 2024-03-09T08:53:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.805+0000 7f9a00a00640 1 -- 172.21.15.67:0/622431216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f99fc0c4cd0 con 0x7f99d800a990 2024-03-09T08:53:55.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.805+0000 7f99f3400640 1 -- 172.21.15.67:0/622431216 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7f99ec05a8a0 con 0x7f99d800a990 2024-03-09T08:53:55.803 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:53:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.808+0000 7f9a00a00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99d406b4d0 msgr2=0x7f99d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.808+0000 7f9a00a00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99d406b4d0 0x7f99d406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0734d0 tx=0x7f99e4008030 comp rx=0 tx=0).stop 2024-03-09T08:53:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.808+0000 7f9a00a00640 1 -- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d800a990 msgr2=0x7f99d800cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.808+0000 7f9a00a00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d800a990 0x7f99d800cd80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f99ec004660 tx=0x7f99ec01a810 comp rx=0 tx=0).stop 2024-03-09T08:53:55.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.808+0000 7f9a00a00640 1 -- 172.21.15.67:0/622431216 shutdown_connections 2024-03-09T08:53:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.808+0000 7f9a00a00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99d406b4d0 0x7f99d406d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.808+0000 7f9a00a00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99fc0710a0 0x7f99d80036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.808+0000 7f9a00a00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99fc0c1f70 0x7f99d80081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.808+0000 7f9a00a00640 1 --2- 172.21.15.67:0/622431216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d800a990 0x7f99d800cd80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.808+0000 7f9a00a00640 1 -- 172.21.15.67:0/622431216 >> 172.21.15.67:0/622431216 conn(0x7f99fc0c7300 msgr2=0x7f99fc0c0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.809+0000 7f9a00a00640 1 -- 172.21.15.67:0/622431216 shutdown_connections 2024-03-09T08:53:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:55.809+0000 7f9a00a00640 1 -- 172.21.15.67:0/622431216 wait complete. 2024-03-09T08:53:55.824 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:53:55.824 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:53:55.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:53:56.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.022+0000 7f4cd3600640 1 Processor -- start 2024-03-09T08:53:56.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.022+0000 7f4cd3600640 1 -- start start 2024-03-09T08:53:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.025+0000 7f4cd3600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4ccc0caa20 0x7f4ccc0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.025+0000 7f4cd3600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ccc06b040 con 0x7f4ccc0c8c60 2024-03-09T08:53:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.025+0000 7f4cd3600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ccc156710 con 0x7f4ccc153460 2024-03-09T08:53:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.025+0000 7f4cd3600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ccc0ca590 con 0x7f4ccc0caa20 2024-03-09T08:53:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.025+0000 7f4cd1600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4ccc0caa20 0x7f4ccc0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.025+0000 7f4cc3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4ccc0c8c60 0x7f4ccc0c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41952/0 (socket says 172.21.15.67:41952) 2024-03-09T08:53:56.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.025+0000 7f4cc3e00640 1 -- 172.21.15.67:0/4061205047 learned_addr learned my addr 172.21.15.67:0/4061205047 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:56.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.026+0000 7f4cd1600640 1 -- 172.21.15.67:0/4061205047 >> v1:172.21.15.134:6789/0 conn(0x7f4ccc153460 legacy=0x7f4ccc155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:56.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.026+0000 7f4cd1600640 1 -- 172.21.15.67:0/4061205047 >> v1:172.21.15.67:6789/0 conn(0x7f4ccc0c8c60 legacy=0x7f4ccc0c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.026+0000 7f4cd1600640 1 -- 172.21.15.67:0/4061205047 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc1569f0 con 0x7f4ccc0caa20 2024-03-09T08:53:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.027+0000 7f4cd1600640 1 --2- 172.21.15.67:0/4061205047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc0caa20 0x7f4ccc0cae50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40097a0 tx=0x7f4cc402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1763a75a4172094e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.028+0000 7f4ccbe00640 1 -- 172.21.15.67:0/4061205047 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40315a0 con 0x7f4ccc0caa20 2024-03-09T08:53:56.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.028+0000 7f4ccbe00640 1 -- 172.21.15.67:0/4061205047 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc4037070 con 0x7f4ccc0caa20 2024-03-09T08:53:56.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.028+0000 7f4ccbe00640 1 -- 172.21.15.67:0/4061205047 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4031960 con 0x7f4ccc0caa20 2024-03-09T08:53:56.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.028+0000 7f4cd3600640 1 -- 172.21.15.67:0/4061205047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc0caa20 msgr2=0x7f4ccc0cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.028+0000 7f4cd3600640 1 --2- 172.21.15.67:0/4061205047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc0caa20 0x7f4ccc0cae50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40097a0 tx=0x7f4cc402b840 comp rx=0 tx=0).stop 2024-03-09T08:53:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.028+0000 7f4cd3600640 1 -- 172.21.15.67:0/4061205047 shutdown_connections 2024-03-09T08:53:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.028+0000 7f4cd3600640 1 --2- 172.21.15.67:0/4061205047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc0caa20 0x7f4ccc0cae50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.028+0000 7f4cd3600640 1 -- 172.21.15.67:0/4061205047 >> 172.21.15.67:0/4061205047 conn(0x7f4ccc0c8170 msgr2=0x7f4ccc0c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.028+0000 7f4cd3600640 1 -- 172.21.15.67:0/4061205047 shutdown_connections 2024-03-09T08:53:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.029+0000 7f4cd3600640 1 -- 172.21.15.67:0/4061205047 wait complete. 2024-03-09T08:53:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.030+0000 7f4cd3600640 1 Processor -- start 2024-03-09T08:53:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.030+0000 7f4cd3600640 1 -- start start 2024-03-09T08:53:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.031+0000 7f4cd3600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc0c8c60 0x7f4ccc0ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.031+0000 7f4cd3600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ccc0caa20 0x7f4ccc0c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.031+0000 7f4cd3600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc153460 0x7f4ccc1eed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.031+0000 7f4cd3600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc0c9f20 con 0x7f4ccc153460 2024-03-09T08:53:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.031+0000 7f4cd3600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc153840 con 0x7f4ccc0c8c60 2024-03-09T08:53:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.031+0000 7f4cd3600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc0ca0a0 con 0x7f4ccc0caa20 2024-03-09T08:53:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.031+0000 7f4cd1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc153460 0x7f4ccc1eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.031+0000 7f4cd1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc153460 0x7f4ccc1eed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38324/0 (socket says 172.21.15.67:38324) 2024-03-09T08:53:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.031+0000 7f4cd1600640 1 -- 172.21.15.67:0/2842151474 learned_addr learned my addr 172.21.15.67:0/2842151474 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.032+0000 7f4cd0c00640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc0c8c60 0x7f4ccc0ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.032+0000 7f4cc3e00640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ccc0caa20 0x7f4ccc0c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.032+0000 7f4cd0c00640 1 -- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc0c8c60 msgr2=0x7f4ccc0ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:53:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.032+0000 7f4cd0c00640 1 -- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc0c8c60 msgr2=0x7f4ccc0ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:53:56.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.032+0000 7f4cd0c00640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc0c8c60 0x7f4ccc0ac470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.032+0000 7f4cd0c00640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc0c8c60 0x7f4ccc0ac470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.032+0000 7f4cd1600640 1 -- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ccc0caa20 msgr2=0x7f4ccc0c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.032+0000 7f4cd1600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ccc0caa20 0x7f4ccc0c4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.032+0000 7f4cd1600640 1 -- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc0c8c60 msgr2=0x7f4ccc0ac470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.032+0000 7f4cd1600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc0c8c60 0x7f4ccc0ac470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.032+0000 7f4cd1600640 1 -- 172.21.15.67:0/2842151474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc155790 con 0x7f4ccc153460 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.033+0000 7f4cd1600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc153460 0x7f4ccc1eed20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40098c0 tx=0x7f4cc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.033+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4032840 con 0x7f4ccc153460 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.033+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc40329e0 con 0x7f4ccc153460 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.033+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4032dd0 con 0x7f4ccc153460 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.033+0000 7f4cd3600640 1 -- 172.21.15.67:0/2842151474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ccc155a70 con 0x7f4ccc153460 2024-03-09T08:53:56.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.033+0000 7f4cd3600640 1 -- 172.21.15.67:0/2842151474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ccc156030 con 0x7f4ccc153460 2024-03-09T08:53:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.034+0000 7f4cd3600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc0c4c40 0x7f4ccc1ef260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.034+0000 7f4cd3600640 1 -- 172.21.15.67:0/2842151474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc0752b0 con 0x7f4ccc0c4c40 2024-03-09T08:53:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.034+0000 7f4cd0c00640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc0c4c40 0x7f4ccc1ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.035+0000 7f4cd0c00640 1 -- 172.21.15.67:0/2842151474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ccc075a00 con 0x7f4ccc0c4c40 2024-03-09T08:53:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.035+0000 7f4cd0c00640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc0c4c40 0x7f4ccc1ef260 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00ce10 tx=0x7f4cbc007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.036+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc01a070 con 0x7f4ccc0c4c40 2024-03-09T08:53:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.036+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4cc404c020 con 0x7f4ccc153460 2024-03-09T08:53:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.037+0000 7f4cc9600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4ca806b3d0 0x7f4ca806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.037+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f4cc402f020 con 0x7f4ccc153460 2024-03-09T08:53:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.037+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4cbc007e20 con 0x7f4ccc0c4c40 2024-03-09T08:53:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.037+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc0c4c40 msgr2=0x7f4ccc1ef260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.037+0000 7f4cc9600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc0c4c40 0x7f4ccc1ef260 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00ce10 tx=0x7f4cbc007660 comp rx=0 tx=0).stop 2024-03-09T08:53:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.037+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc0c4c40 msgr2=0x7f4ccc1ef260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.037+0000 7f4cc9600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc0c4c40 0x7f4ccc1ef260 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.037+0000 7f4cc3e00640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4ca806b3d0 0x7f4ca806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.039+0000 7f4cc3e00640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4ca806b3d0 0x7f4ca806d890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc150c20 tx=0x7f4cb4007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.045+0000 7f4cd3600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1ef7a0 0x7f4ccc0ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.045+0000 7f4cd3600640 1 -- 172.21.15.67:0/2842151474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc1efb80 con 0x7f4ccc1ef7a0 2024-03-09T08:53:56.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.045+0000 7f4cd0c00640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1ef7a0 0x7f4ccc0ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.045+0000 7f4cd0c00640 1 -- 172.21.15.67:0/2842151474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4ccc0c95f0 con 0x7f4ccc1ef7a0 2024-03-09T08:53:56.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.045+0000 7f4cd0c00640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1ef7a0 0x7f4ccc0ab080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00ce10 tx=0x7f4cbc004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.046+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc015610 con 0x7f4ccc1ef7a0 2024-03-09T08:53:56.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.046+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f4cbc0253f0 con 0x7f4ccc1ef7a0 2024-03-09T08:53:56.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.046+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1ef7a0 msgr2=0x7f4ccc0ab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.046+0000 7f4cc9600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1ef7a0 0x7f4ccc0ab080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00ce10 tx=0x7f4cbc004a70 comp rx=0 tx=0).stop 2024-03-09T08:53:56.043 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:53:56.043 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 16, 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:53:56.044 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:53:56.045 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:53:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:53:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:53:56.057 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:53:56.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.046+0000 7f4cc9600640 1 -- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1ef7a0 msgr2=0x7f4ccc0ab080 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:56.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.046+0000 7f4cc9600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc1ef7a0 0x7f4ccc0ab080 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00ce10 tx=0x7f4cbc004a70 comp rx=0 tx=0).stop 2024-03-09T08:53:56.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.046+0000 7f4cd3600640 1 -- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4ca806b3d0 msgr2=0x7f4ca806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.046+0000 7f4cd3600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4ca806b3d0 0x7f4ca806d890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc150c20 tx=0x7f4cb4007e80 comp rx=0 tx=0).stop 2024-03-09T08:53:56.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.047+0000 7f4cd3600640 1 -- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc153460 msgr2=0x7f4ccc1eed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.047+0000 7f4cd3600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc153460 0x7f4ccc1eed20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40098c0 tx=0x7f4cc40046c0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.047+0000 7f4cd1600640 1 -- 172.21.15.67:0/2842151474 reap_dead start 2024-03-09T08:53:56.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.047+0000 7f4cd3600640 1 -- 172.21.15.67:0/2842151474 shutdown_connections 2024-03-09T08:53:56.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.047+0000 7f4cd3600640 1 --2- 172.21.15.67:0/2842151474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ccc153460 0x7f4ccc1eed20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.047+0000 7f4cd3600640 1 -- 172.21.15.67:0/2842151474 >> 172.21.15.67:0/2842151474 conn(0x7f4ccc0c8170 msgr2=0x7f4ccc152150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.047+0000 7f4cd3600640 1 -- 172.21.15.67:0/2842151474 shutdown_connections 2024-03-09T08:53:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.047+0000 7f4cd3600640 1 -- 172.21.15.67:0/2842151474 wait complete. 2024-03-09T08:53:56.060 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:53:56.060 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:53:56.060 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:53:56.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.253+0000 7f5129a00640 1 Processor -- start 2024-03-09T08:53:56.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.254+0000 7f5129a00640 1 -- start start 2024-03-09T08:53:56.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.255+0000 7f5129a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f512414fd90 0x7f5124156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.255+0000 7f5129a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f512406c1d0 0x7f512406c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.255+0000 7f5129a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5124156bc0 con 0x7f51240c9380 2024-03-09T08:53:56.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.255+0000 7f5129a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512406ba60 con 0x7f512406c1d0 2024-03-09T08:53:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.255+0000 7f5129a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51240c9890 con 0x7f512414fd90 2024-03-09T08:53:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.256+0000 7f5123e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f512414fd90 0x7f5124156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.256+0000 7f5129000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f512406c1d0 0x7f512406c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.256+0000 7f5123e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f512414fd90 0x7f5124156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:52386/0 (socket says 172.21.15.67:52386) 2024-03-09T08:53:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.256+0000 7f5123e00640 1 -- 172.21.15.67:0/2090403645 learned_addr learned my addr 172.21.15.67:0/2090403645 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.256+0000 7f5129000640 1 -- 172.21.15.67:0/2090403645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f512414fd90 msgr2=0x7f5124156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.256+0000 7f5129000640 1 --2- 172.21.15.67:0/2090403645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f512414fd90 0x7f5124156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.256+0000 7f5129000640 1 -- 172.21.15.67:0/2090403645 >> v1:172.21.15.134:6789/0 conn(0x7f51240c9380 legacy=0x7f51240acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.256+0000 7f5129000640 1 -- 172.21.15.67:0/2090403645 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240c5f80 con 0x7f512406c1d0 2024-03-09T08:53:56.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.257+0000 7f5129000640 1 --2- 172.21.15.67:0/2090403645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f512406c1d0 0x7f512406c600 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f511c009d90 tx=0x7f511c02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17347e9011f09f14 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.258+0000 7f5122a00640 1 -- 172.21.15.67:0/2090403645 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c039070 con 0x7f512406c1d0 2024-03-09T08:53:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.258+0000 7f5122a00640 1 -- 172.21.15.67:0/2090403645 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c02cb30 con 0x7f512406c1d0 2024-03-09T08:53:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.258+0000 7f5122a00640 1 -- 172.21.15.67:0/2090403645 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c034440 con 0x7f512406c1d0 2024-03-09T08:53:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.259+0000 7f5129a00640 1 -- 172.21.15.67:0/2090403645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f512406c1d0 msgr2=0x7f512406c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.259+0000 7f5129a00640 1 --2- 172.21.15.67:0/2090403645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f512406c1d0 0x7f512406c600 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f511c009d90 tx=0x7f511c02bd80 comp rx=0 tx=0).stop 2024-03-09T08:53:56.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.259+0000 7f5129a00640 1 -- 172.21.15.67:0/2090403645 shutdown_connections 2024-03-09T08:53:56.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.259+0000 7f5129a00640 1 --2- 172.21.15.67:0/2090403645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f512406c1d0 0x7f512406c600 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.259+0000 7f5129a00640 1 --2- 172.21.15.67:0/2090403645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f512414fd90 0x7f5124156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.259+0000 7f5129a00640 1 -- 172.21.15.67:0/2090403645 >> 172.21.15.67:0/2090403645 conn(0x7f51240cb480 msgr2=0x7f51240cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:56.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.259+0000 7f5129a00640 1 -- 172.21.15.67:0/2090403645 shutdown_connections 2024-03-09T08:53:56.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.259+0000 7f5129a00640 1 -- 172.21.15.67:0/2090403645 wait complete. 2024-03-09T08:53:56.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.261+0000 7f5129a00640 1 Processor -- start 2024-03-09T08:53:56.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.262+0000 7f5129a00640 1 -- start start 2024-03-09T08:53:56.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.262+0000 7f5129a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f512406c1d0 0x7f51241ef140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5129a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51240c9380 0x7f51241ee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5129a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f512414fd90 0x7f51241ee900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5129a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512406bcb0 con 0x7f512406c1d0 2024-03-09T08:53:56.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5129a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51240c6520 con 0x7f512414fd90 2024-03-09T08:53:56.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5129a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51240ad610 con 0x7f51240c9380 2024-03-09T08:53:56.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5123e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f512406c1d0 0x7f51241ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5123e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f512406c1d0 0x7f51241ef140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38348/0 (socket says 172.21.15.67:38348) 2024-03-09T08:53:56.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5123e00640 1 -- 172.21.15.67:0/2452090541 learned_addr learned my addr 172.21.15.67:0/2452090541 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:56.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5129000640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f512414fd90 0x7f51241ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5123e00640 1 -- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51240c9380 msgr2=0x7f51241ee3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5123e00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51240c9380 0x7f51241ee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5123e00640 1 -- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f512414fd90 msgr2=0x7f51241ee900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5123e00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f512414fd90 0x7f51241ee900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.263+0000 7f5123e00640 1 -- 172.21.15.67:0/2452090541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51241ecdd0 con 0x7f512406c1d0 2024-03-09T08:53:56.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.264+0000 7f5123e00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f512406c1d0 0x7f51241ef140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f511400e870 tx=0x7f511400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.264+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114017070 con 0x7f512406c1d0 2024-03-09T08:53:56.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.264+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511400c8e0 con 0x7f512406c1d0 2024-03-09T08:53:56.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.264+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511400ccd0 con 0x7f512406c1d0 2024-03-09T08:53:56.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.264+0000 7f5129a00640 1 -- 172.21.15.67:0/2452090541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5124159730 con 0x7f512406c1d0 2024-03-09T08:53:56.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.264+0000 7f5129a00640 1 -- 172.21.15.67:0/2452090541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51241efb30 con 0x7f512406c1d0 2024-03-09T08:53:56.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.265+0000 7f5129a00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51240c5250 0x7f5124157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.265+0000 7f5129a00640 1 -- 172.21.15.67:0/2452090541 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51240c26a0 con 0x7f51240c5250 2024-03-09T08:53:56.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.266+0000 7f5123400640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51240c5250 0x7f5124157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.267+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f511401a070 con 0x7f512406c1d0 2024-03-09T08:53:56.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.268+0000 7f5123400640 1 -- 172.21.15.67:0/2452090541 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5124157800 con 0x7f51240c5250 2024-03-09T08:53:56.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.268+0000 7f5123400640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51240c5250 0x7f5124157bc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f512414b900 tx=0x7f511000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.268+0000 7f511be00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f510406b4d0 0x7f510406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.268+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f511408a3b0 con 0x7f512406c1d0 2024-03-09T08:53:56.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.269+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110017070 con 0x7f51240c5250 2024-03-09T08:53:56.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.270+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f51100052d0 con 0x7f51240c5250 2024-03-09T08:53:56.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.270+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51240c5250 msgr2=0x7f5124157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.270+0000 7f511be00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51240c5250 0x7f5124157bc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f512414b900 tx=0x7f511000c7a0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.270+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51240c5250 msgr2=0x7f5124157bc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:56.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.270+0000 7f511be00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51240c5250 0x7f5124157bc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.271+0000 7f5129000640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f510406b4d0 0x7f510406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.275+0000 7f5129000640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f510406b4d0 0x7f510406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f511c036000 tx=0x7f511c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.277+0000 7f5129a00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5124159db0 0x7f5124158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.277+0000 7f5129a00640 1 -- 172.21.15.67:0/2452090541 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512404f3c0 con 0x7f5124159db0 2024-03-09T08:53:56.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.278+0000 7f5123400640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5124159db0 0x7f5124158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.279+0000 7f5123400640 1 -- 172.21.15.67:0/2452090541 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f512414d140 con 0x7f5124159db0 2024-03-09T08:53:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.280+0000 7f5123400640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5124159db0 0x7f5124158100 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f512414b900 tx=0x7f5110000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.280+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110017070 con 0x7f5124159db0 2024-03-09T08:53:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.280+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4533 (secure 0 0 0) 0x7f5110018040 con 0x7f5124159db0 2024-03-09T08:53:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.280+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5124159db0 msgr2=0x7f5124158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.280+0000 7f511be00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5124159db0 0x7f5124158100 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f512414b900 tx=0x7f5110000ed0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f511be00640 1 -- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5124159db0 msgr2=0x7f5124158100 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:56.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f511be00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5124159db0 0x7f5124158100 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.278 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:53:56.278 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:53:56.278 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 16, 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:56.279 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:56.280 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:53:56.281 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:53:56.282 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:53:56.282 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:56.282 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:56.282 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:53:56.282 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:53:56.282 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:53:56.283 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:53:56.283 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:53:56.283 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:53:56.283 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:53:56.283 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:56.283 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:53:56.284 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:56.284 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:56.284 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:56.284 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:56.284 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:56.285 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:56.285 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:56.285 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:56.285 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:56.285 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:56.285 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:56.286 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.286 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:53:56.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.286 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:53:56.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.287 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:53:56.287 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:53:56.287 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:56.287 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:56.287 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.287 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:56.288 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:53:56.288 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.288 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:56.289 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:53:56.289 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.289 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.289 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.290 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:53:56.290 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:53:56.290 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:56.290 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:56.290 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:56.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.291 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.291 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:53:56.291 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:53:56.291 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:56.291 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:56.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.292 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:56.292 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:53:56.292 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.293 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:56.293 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:53:56.293 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.293 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.293 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:53:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:53:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:56.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.295 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:53:56.295 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:53:56.295 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:56.296 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:56.296 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.296 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:56.296 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:53:56.296 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.297 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:56.297 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:53:56.297 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.297 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.298 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.298 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:53:56.298 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:53:56.298 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:56.298 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:56.299 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:56.299 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.299 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.299 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:53:56.300 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:53:56.300 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.300 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.300 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.300 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:56.300 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.301 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.301 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:53:56.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.301 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.301 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.302 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:56.302 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.302 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.302 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:53:56.302 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:53:56.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f5129a00640 1 -- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f510406b4d0 msgr2=0x7f510406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f5129a00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f510406b4d0 0x7f510406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f511c036000 tx=0x7f511c030000 comp rx=0 tx=0).stop 2024-03-09T08:53:56.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f5129a00640 1 -- 172.21.15.67:0/2452090541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f512406c1d0 msgr2=0x7f51241ef140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f5129a00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f512406c1d0 0x7f51241ef140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f511400e870 tx=0x7f511400ed30 comp rx=0 tx=0).stop 2024-03-09T08:53:56.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f5129000640 1 -- 172.21.15.67:0/2452090541 reap_dead start 2024-03-09T08:53:56.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f5129a00640 1 -- 172.21.15.67:0/2452090541 shutdown_connections 2024-03-09T08:53:56.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f5129a00640 1 --2- 172.21.15.67:0/2452090541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f512406c1d0 0x7f51241ef140 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f5129a00640 1 -- 172.21.15.67:0/2452090541 >> 172.21.15.67:0/2452090541 conn(0x7f51240cb480 msgr2=0x7f512414bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:56.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f5129a00640 1 -- 172.21.15.67:0/2452090541 shutdown_connections 2024-03-09T08:53:56.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.281+0000 7f5129a00640 1 -- 172.21.15.67:0/2452090541 wait complete. 2024-03-09T08:53:56.307 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:53:56.307 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:53:56.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:53:56.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.518+0000 7fb76d200640 1 Processor -- start 2024-03-09T08:53:56.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.518+0000 7fb76d200640 1 -- start start 2024-03-09T08:53:56.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.520+0000 7fb76d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb768072540 0x7fb7680c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.520+0000 7fb76d200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb76806dd80 con 0x7fb76806d250 2024-03-09T08:53:56.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.520+0000 7fb76d200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb76806d7c0 con 0x7fb76814ff40 2024-03-09T08:53:56.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.520+0000 7fb76d200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb768153760 con 0x7fb768072540 2024-03-09T08:53:56.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.520+0000 7fb767e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb76806d250 0x7fb76806dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41958/0 (socket says 172.21.15.67:41958) 2024-03-09T08:53:56.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.520+0000 7fb767e00640 1 -- 172.21.15.67:0/1796781887 learned_addr learned my addr 172.21.15.67:0/1796781887 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:53:56.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.520+0000 7fb766a00640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb768072540 0x7fb7680c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329326151 0 0) 0x7fb76806dd80 con 0x7fb76806d250 2024-03-09T08:53:56.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb74c003520 con 0x7fb76806d250 2024-03-09T08:53:56.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2413156400 0 0) 0x7fb74c003520 con 0x7fb76806d250 2024-03-09T08:53:56.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb768072540 msgr2=0x7fb7680c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:56.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb768072540 0x7fb7680c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 >> v1:172.21.15.134:6789/0 conn(0x7fb76814ff40 legacy=0x7fb768153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:56.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76806daa0 con 0x7fb76806d250 2024-03-09T08:53:56.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb7600019f0 con 0x7fb76806d250 2024-03-09T08:53:56.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 >> v1:172.21.15.67:6789/0 conn(0x7fb76806d250 legacy=0x7fb76806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb768072540 0x7fb74c003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb74c003fb0 0x7fb74c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb74c004d40 0x7fb74c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768153760 con 0x7fb74c003fb0 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c00bbb0 con 0x7fb74c004d40 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.521+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c00bdb0 con 0x7fb768072540 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.522+0000 7fb766a00640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb74c003fb0 0x7fb74c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.522+0000 7fb767e00640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb74c004d40 0x7fb74c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.522+0000 7fb767400640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb768072540 0x7fb74c003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.522+0000 7fb766a00640 1 -- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb768072540 msgr2=0x7fb74c003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.522+0000 7fb766a00640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb768072540 0x7fb74c003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.522+0000 7fb766a00640 1 -- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb74c004d40 msgr2=0x7fb74c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.522+0000 7fb766a00640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb74c004d40 0x7fb74c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.522+0000 7fb766a00640 1 -- 172.21.15.67:0/1796781887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c003520 con 0x7fb74c003fb0 2024-03-09T08:53:56.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.522+0000 7fb767400640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb768072540 0x7fb74c003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:53:56.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.523+0000 7fb766a00640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb74c003fb0 0x7fb74c004800 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb754009ac0 tx=0x7fb74c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a7074b7906160a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.524+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754039070 con 0x7fb74c003fb0 2024-03-09T08:53:56.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.524+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75402c6b0 con 0x7fb74c003fb0 2024-03-09T08:53:56.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.524+0000 7fb766000640 1 -- 172.21.15.67:0/1796781887 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754009690 con 0x7fb74c003fb0 2024-03-09T08:53:56.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.524+0000 7fb76d200640 1 -- 172.21.15.67:0/1796781887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb74c003fb0 msgr2=0x7fb74c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.524+0000 7fb76d200640 1 --2- 172.21.15.67:0/1796781887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb74c003fb0 0x7fb74c004800 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb754009ac0 tx=0x7fb74c005830 comp rx=0 tx=0).stop 2024-03-09T08:53:56.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.524+0000 7fb767e00640 1 -- 172.21.15.67:0/1796781887 reap_dead start 2024-03-09T08:53:56.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.524+0000 7fb76d200640 1 -- 172.21.15.67:0/1796781887 shutdown_connections 2024-03-09T08:53:56.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.524+0000 7fb76d200640 1 -- 172.21.15.67:0/1796781887 >> 172.21.15.67:0/1796781887 conn(0x7fb7680c8480 msgr2=0x7fb7680c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:56.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.524+0000 7fb76d200640 1 -- 172.21.15.67:0/1796781887 shutdown_connections 2024-03-09T08:53:56.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.524+0000 7fb76d200640 1 -- 172.21.15.67:0/1796781887 wait complete. 2024-03-09T08:53:56.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.526+0000 7fb76d200640 1 Processor -- start 2024-03-09T08:53:56.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.526+0000 7fb76d200640 1 -- start start 2024-03-09T08:53:56.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.527+0000 7fb76d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb74c003fb0 0x7fb768148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.527+0000 7fb76d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7680c6ab0 0x7fb7680c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.527+0000 7fb76d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7681e8ee0 0x7fb7681e9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.527+0000 7fb76d200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76806e6e0 con 0x7fb7681e8ee0 2024-03-09T08:53:56.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.527+0000 7fb76d200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768152b30 con 0x7fb7680c6ab0 2024-03-09T08:53:56.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.527+0000 7fb76d200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76806e860 con 0x7fb74c003fb0 2024-03-09T08:53:56.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.527+0000 7fb767e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7681e8ee0 0x7fb7681e9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.527+0000 7fb767e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7681e8ee0 0x7fb7681e9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38360/0 (socket says 172.21.15.67:38360) 2024-03-09T08:53:56.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.527+0000 7fb767e00640 1 -- 172.21.15.67:0/3673212982 learned_addr learned my addr 172.21.15.67:0/3673212982 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:56.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.528+0000 7fb766a00640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7680c6ab0 0x7fb7680c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.528+0000 7fb767e00640 1 -- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb74c003fb0 msgr2=0x7fb768148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.528+0000 7fb767e00640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb74c003fb0 0x7fb768148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.528+0000 7fb767e00640 1 -- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7680c6ab0 msgr2=0x7fb7680c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.528+0000 7fb767e00640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7680c6ab0 0x7fb7680c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.528+0000 7fb767e00640 1 -- 172.21.15.67:0/3673212982 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76814e950 con 0x7fb7681e8ee0 2024-03-09T08:53:56.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.528+0000 7fb766a00640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7680c6ab0 0x7fb7680c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:53:56.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.528+0000 7fb767e00640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7681e8ee0 0x7fb7681e9360 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb7680715f0 tx=0x7fb760007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.529+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76000ea50 con 0x7fb7681e8ee0 2024-03-09T08:53:56.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.529+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76000ec10 con 0x7fb7681e8ee0 2024-03-09T08:53:56.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.529+0000 7fb76d200640 1 -- 172.21.15.67:0/3673212982 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7680c7020 con 0x7fb7681e8ee0 2024-03-09T08:53:56.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.529+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76000ee00 con 0x7fb7681e8ee0 2024-03-09T08:53:56.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.529+0000 7fb76d200640 1 -- 172.21.15.67:0/3673212982 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76814eed0 con 0x7fb7681e8ee0 2024-03-09T08:53:56.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.530+0000 7fb76d200640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb76814f3f0 0x7fb7681e6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.530+0000 7fb76d200640 1 -- 172.21.15.67:0/3673212982 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7680c7300 con 0x7fb76814f3f0 2024-03-09T08:53:56.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.530+0000 7fb767400640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb76814f3f0 0x7fb7681e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.531+0000 7fb767400640 1 -- 172.21.15.67:0/3673212982 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb76804f3c0 con 0x7fb76814f3f0 2024-03-09T08:53:56.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.531+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb76001a020 con 0x7fb7681e8ee0 2024-03-09T08:53:56.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.532+0000 7fb767400640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb76814f3f0 0x7fb7681e6070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb75800d420 tx=0x7fb75800d8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.532+0000 7fb75f400640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb74806b4d0 0x7fb74806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.533+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fb760089f90 con 0x7fb7681e8ee0 2024-03-09T08:53:56.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.533+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758005480 con 0x7fb76814f3f0 2024-03-09T08:53:56.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.533+0000 7fb766a00640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb74806b4d0 0x7fb74806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.534+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb758017070 con 0x7fb76814f3f0 2024-03-09T08:53:56.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.534+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb76814f3f0 msgr2=0x7fb7681e6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.534+0000 7fb75f400640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb76814f3f0 0x7fb7681e6070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb75800d420 tx=0x7fb75800d8e0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.534+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb76814f3f0 msgr2=0x7fb7681e6070 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:56.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.534+0000 7fb75f400640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb76814f3f0 0x7fb7681e6070 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb75800d420 tx=0x7fb75800d8e0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.534+0000 7fb766a00640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb74806b4d0 0x7fb74806d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb754036000 tx=0x7fb754030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.542+0000 7fb76d200640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7681e65b0 0x7fb768150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.542+0000 7fb76d200640 1 -- 172.21.15.67:0/3673212982 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7681e98a0 con 0x7fb7681e65b0 2024-03-09T08:53:56.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.543+0000 7fb767400640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7681e65b0 0x7fb768150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.543+0000 7fb767400640 1 -- 172.21.15.67:0/3673212982 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb768072320 con 0x7fb7681e65b0 2024-03-09T08:53:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.544+0000 7fb767400640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7681e65b0 0x7fb768150320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb758016000 tx=0x7fb758000fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.544+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758003dd0 con 0x7fb7681e65b0 2024-03-09T08:53:56.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.545+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5186 (secure 0 0 0) 0x7fb758012410 con 0x7fb7681e65b0 2024-03-09T08:53:56.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.545+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7681e65b0 msgr2=0x7fb768150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.545+0000 7fb75f400640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7681e65b0 0x7fb768150320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb758016000 tx=0x7fb758000fb0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.545+0000 7fb75f400640 1 -- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7681e65b0 msgr2=0x7fb768150320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:53:56.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.545+0000 7fb75f400640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7681e65b0 0x7fb768150320 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb758016000 tx=0x7fb758000fb0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.542 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 16, 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:53:56.543 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:53:56.544 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.545 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:53:56.546 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:53:56.547 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:53:56.550 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:53:56.551 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:53:56.552 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.553 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.554 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:53:56.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:53:56.556 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:53:56.556 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:53:56.556 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:53:56.556 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:53:56.556 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:53:56.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:53:56.556 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:53:56.556 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:53:56.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.545+0000 7fb76d200640 1 -- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb74806b4d0 msgr2=0x7fb74806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.545+0000 7fb76d200640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb74806b4d0 0x7fb74806d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb754036000 tx=0x7fb754030000 comp rx=0 tx=0).stop 2024-03-09T08:53:56.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.545+0000 7fb76d200640 1 -- 172.21.15.67:0/3673212982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7681e8ee0 msgr2=0x7fb7681e9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.545+0000 7fb76d200640 1 --2- 172.21.15.67:0/3673212982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7681e8ee0 0x7fb7681e9360 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb7680715f0 tx=0x7fb760007bc0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.545+0000 7fb767e00640 1 -- 172.21.15.67:0/3673212982 reap_dead start 2024-03-09T08:53:56.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.545+0000 7fb767e00640 1 -- 172.21.15.67:0/3673212982 reap_dead start 2024-03-09T08:53:56.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.546+0000 7fb76d200640 1 -- 172.21.15.67:0/3673212982 shutdown_connections 2024-03-09T08:53:56.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.546+0000 7fb76d200640 1 -- 172.21.15.67:0/3673212982 >> 172.21.15.67:0/3673212982 conn(0x7fb7680c8480 msgr2=0x7fb7680c35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:56.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.546+0000 7fb76d200640 1 -- 172.21.15.67:0/3673212982 shutdown_connections 2024-03-09T08:53:56.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.546+0000 7fb76d200640 1 -- 172.21.15.67:0/3673212982 wait complete. 2024-03-09T08:53:56.562 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:53:56.562 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T08:53:56.562 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T08:53:56.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.763+0000 7f81c0a00640 1 Processor -- start 2024-03-09T08:53:56.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.763+0000 7f81c0a00640 1 -- start start 2024-03-09T08:53:56.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.764+0000 7f81c0a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f81bc153300 0x7f81bc0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.764+0000 7f81c0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f81bc0cadb0 0x7f81bc0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.764+0000 7f81c0a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc157000 con 0x7f81bc0c5e40 2024-03-09T08:53:56.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.764+0000 7f81c0a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc0ae430 con 0x7f81bc0cadb0 2024-03-09T08:53:56.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.764+0000 7f81c0a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc0ae6f0 con 0x7f81bc153300 2024-03-09T08:53:56.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.765+0000 7f81bb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f81bc0cadb0 0x7f81bc0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.765+0000 7f81bb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f81bc0cadb0 0x7f81bc0b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38362/0 (socket says 172.21.15.67:38362) 2024-03-09T08:53:56.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.765+0000 7f81bb200640 1 -- 172.21.15.67:0/932774336 learned_addr learned my addr 172.21.15.67:0/932774336 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:56.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.765+0000 7f81bb200640 1 -- 172.21.15.67:0/932774336 >> v2:172.21.15.134:3300/0 conn(0x7f81bc153300 msgr2=0x7f81bc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:56.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.765+0000 7f81bb200640 1 --2- 172.21.15.67:0/932774336 >> v2:172.21.15.134:3300/0 conn(0x7f81bc153300 0x7f81bc0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.765+0000 7f81bb200640 1 -- 172.21.15.67:0/932774336 >> v1:172.21.15.134:6789/0 conn(0x7f81bc0c5e40 legacy=0x7f81bc156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:53:56.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.765+0000 7f81bb200640 1 -- 172.21.15.67:0/932774336 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc155b10 con 0x7f81bc0cadb0 2024-03-09T08:53:56.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.766+0000 7f81bb200640 1 --2- 172.21.15.67:0/932774336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc0cadb0 0x7f81bc0b0540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0ade40 tx=0x7f81b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bdd1b9b533f8f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.767+0000 7f81b9400640 1 -- 172.21.15.67:0/932774336 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b402ca80 con 0x7f81bc0cadb0 2024-03-09T08:53:56.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.767+0000 7f81b9400640 1 -- 172.21.15.67:0/932774336 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b402cc20 con 0x7f81bc0cadb0 2024-03-09T08:53:56.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.767+0000 7f81b9400640 1 -- 172.21.15.67:0/932774336 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b40343f0 con 0x7f81bc0cadb0 2024-03-09T08:53:56.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.767+0000 7f81c0a00640 1 -- 172.21.15.67:0/932774336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc0cadb0 msgr2=0x7f81bc0b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.767+0000 7f81c0a00640 1 --2- 172.21.15.67:0/932774336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc0cadb0 0x7f81bc0b0540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0ade40 tx=0x7f81b402bee0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.767+0000 7f81c0a00640 1 -- 172.21.15.67:0/932774336 shutdown_connections 2024-03-09T08:53:56.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.767+0000 7f81c0a00640 1 --2- 172.21.15.67:0/932774336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc0cadb0 0x7f81bc0b0540 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.767+0000 7f81c0a00640 1 --2- 172.21.15.67:0/932774336 >> v2:172.21.15.134:3300/0 conn(0x7f81bc153300 0x7f81bc0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.767+0000 7f81c0a00640 1 -- 172.21.15.67:0/932774336 >> 172.21.15.67:0/932774336 conn(0x7f81bc0732d0 msgr2=0x7f81bc0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:56.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.767+0000 7f81c0a00640 1 -- 172.21.15.67:0/932774336 shutdown_connections 2024-03-09T08:53:56.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.767+0000 7f81c0a00640 1 -- 172.21.15.67:0/932774336 wait complete. 2024-03-09T08:53:56.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.769+0000 7f81c0a00640 1 Processor -- start 2024-03-09T08:53:56.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.769+0000 7f81c0a00640 1 -- start start 2024-03-09T08:53:56.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.770+0000 7f81c0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81bc0c5e40 0x7f81bc1eed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.770+0000 7f81c0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc0cadb0 0x7f81bc0c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.770+0000 7f81c0a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81bc153300 0x7f81bc0c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.770+0000 7f81c0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c59e0 con 0x7f81bc0cadb0 2024-03-09T08:53:56.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.770+0000 7f81c0a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc159e90 con 0x7f81bc0c5e40 2024-03-09T08:53:56.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.770+0000 7f81c0a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc156720 con 0x7f81bc153300 2024-03-09T08:53:56.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.771+0000 7f81b9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc0cadb0 0x7f81bc0c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.771+0000 7f81b9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc0cadb0 0x7f81bc0c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38370/0 (socket says 172.21.15.67:38370) 2024-03-09T08:53:56.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.771+0000 7f81b9e00640 1 -- 172.21.15.67:0/2528291646 learned_addr learned my addr 172.21.15.67:0/2528291646 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:53:56.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.771+0000 7f81ba800640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81bc0c5e40 0x7f81bc1eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.771+0000 7f81bb200640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81bc153300 0x7f81bc0c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.771+0000 7f81b9e00640 1 -- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81bc153300 msgr2=0x7f81bc0c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.771+0000 7f81b9e00640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81bc153300 0x7f81bc0c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.771+0000 7f81b9e00640 1 -- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81bc0c5e40 msgr2=0x7f81bc1eed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:56.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.771+0000 7f81b9e00640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81bc0c5e40 0x7f81bc1eed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:56.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.771+0000 7f81b9e00640 1 -- 172.21.15.67:0/2528291646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc14e980 con 0x7f81bc0cadb0 2024-03-09T08:53:56.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.771+0000 7f81b9e00640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc0cadb0 0x7f81bc0c6c60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c440 tx=0x7f81a800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.772+0000 7f81b2a00640 1 -- 172.21.15.67:0/2528291646 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8017070 con 0x7f81bc0cadb0 2024-03-09T08:53:56.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.772+0000 7f81b2a00640 1 -- 172.21.15.67:0/2528291646 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a80035f0 con 0x7f81bc0cadb0 2024-03-09T08:53:56.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.772+0000 7f81b2a00640 1 -- 172.21.15.67:0/2528291646 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a80039e0 con 0x7f81bc0cadb0 2024-03-09T08:53:56.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.772+0000 7f81c0a00640 1 -- 172.21.15.67:0/2528291646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc1ef5f0 con 0x7f81bc0cadb0 2024-03-09T08:53:56.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.772+0000 7f81c0a00640 1 -- 172.21.15.67:0/2528291646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc06bfe0 con 0x7f81bc0cadb0 2024-03-09T08:53:56.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.774+0000 7f81c0a00640 1 -- 172.21.15.67:0/2528291646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc04f3c0 con 0x7f81bc0cadb0 2024-03-09T08:53:56.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.774+0000 7f81b2a00640 1 -- 172.21.15.67:0/2528291646 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f81a8003b80 con 0x7f81bc0cadb0 2024-03-09T08:53:56.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.776+0000 7f81b2a00640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f819c06b4d0 0x7f819c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:53:56.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.776+0000 7f81b2a00640 1 -- 172.21.15.67:0/2528291646 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f81a80897e0 con 0x7f81bc0cadb0 2024-03-09T08:53:56.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.776+0000 7f81ba800640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f819c06b4d0 0x7f819c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:53:56.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.777+0000 7f81ba800640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f819c06b4d0 0x7f819c06d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f81bc1551c0 tx=0x7f81ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:53:56.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:56.786+0000 7f81b2a00640 1 -- 172.21.15.67:0/2528291646 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f81bc14e980 con 0x7f81bc0cadb0 2024-03-09T08:53:57.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.113+0000 7f81c0a00640 1 -- 172.21.15.67:0/2528291646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f81bc0cab90 con 0x7f81bc0cadb0 2024-03-09T08:53:57.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.114+0000 7f81b2a00640 1 -- 172.21.15.67:0/2528291646 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f81a8058350 con 0x7f81bc0cadb0 2024-03-09T08:53:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 -- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f819c06b4d0 msgr2=0x7f819c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f819c06b4d0 0x7f819c06d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f81bc1551c0 tx=0x7f81ac008040 comp rx=0 tx=0).stop 2024-03-09T08:53:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 -- 172.21.15.67:0/2528291646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc0cadb0 msgr2=0x7f81bc0c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:53:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc0cadb0 0x7f81bc0c6c60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c440 tx=0x7f81a800c900 comp rx=0 tx=0).stop 2024-03-09T08:53:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 -- 172.21.15.67:0/2528291646 shutdown_connections 2024-03-09T08:53:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f819c06b4d0 0x7f819c06d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:57.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81bc153300 0x7f81bc0c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:57.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81bc0cadb0 0x7f81bc0c6c60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:57.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 --2- 172.21.15.67:0/2528291646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81bc0c5e40 0x7f81bc1eed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:53:57.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 -- 172.21.15.67:0/2528291646 >> 172.21.15.67:0/2528291646 conn(0x7f81bc0732d0 msgr2=0x7f81bc1520b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:53:57.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 -- 172.21.15.67:0/2528291646 shutdown_connections 2024-03-09T08:53:57.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:53:57.117+0000 7f81c0a00640 1 -- 172.21.15.67:0/2528291646 wait complete. 2024-03-09T08:53:57.132 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T08:54:07.132 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:54:07.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:54:07.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.337+0000 7f95bb600640 1 Processor -- start 2024-03-09T08:54:07.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.337+0000 7f95bb600640 1 -- start start 2024-03-09T08:54:07.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.339+0000 7f95bb600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f95b40c22a0 0x7f95b4071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.339+0000 7f95bb600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f95b4157d60 0x7f95b40acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.339+0000 7f95bb600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b41581a0 con 0x7f95b40ac730 2024-03-09T08:54:07.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.339+0000 7f95bb600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b4070ee0 con 0x7f95b4157d60 2024-03-09T08:54:07.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.339+0000 7f95bb600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b4154d00 con 0x7f95b40c22a0 2024-03-09T08:54:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.339+0000 7f95b9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f95b4157d60 0x7f95b40acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.339+0000 7f95b9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f95b4157d60 0x7f95b40acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57498/0 (socket says 172.21.15.67:57498) 2024-03-09T08:54:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.339+0000 7f95b9600640 1 -- 172.21.15.67:0/3496306247 learned_addr learned my addr 172.21.15.67:0/3496306247 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.340+0000 7f95b9600640 1 -- 172.21.15.67:0/3496306247 >> v2:172.21.15.134:3300/0 conn(0x7f95b40c22a0 msgr2=0x7f95b4071190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T08:54:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.340+0000 7f95b9600640 1 --2- 172.21.15.67:0/3496306247 >> v2:172.21.15.134:3300/0 conn(0x7f95b40c22a0 0x7f95b4071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.340+0000 7f95b9600640 1 -- 172.21.15.67:0/3496306247 >> v1:172.21.15.134:6789/0 conn(0x7f95b40ac730 legacy=0x7f95b4157650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T08:54:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.340+0000 7f95b9600640 1 -- 172.21.15.67:0/3496306247 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b4153d40 con 0x7f95b4157d60 2024-03-09T08:54:07.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.341+0000 7f95b9600640 1 --2- 172.21.15.67:0/3496306247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4157d60 0x7f95b40acb10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f95b40c1a30 tx=0x7f95ac02be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586922fedb4dfbff server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:07.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.341+0000 7f95b3400640 1 -- 172.21.15.67:0/3496306247 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac02ca10 con 0x7f95b4157d60 2024-03-09T08:54:07.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.341+0000 7f95b3400640 1 -- 172.21.15.67:0/3496306247 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac02cbb0 con 0x7f95b4157d60 2024-03-09T08:54:07.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.341+0000 7f95b3400640 1 -- 172.21.15.67:0/3496306247 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0343f0 con 0x7f95b4157d60 2024-03-09T08:54:07.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.342+0000 7f95bb600640 1 -- 172.21.15.67:0/3496306247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4157d60 msgr2=0x7f95b40acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:07.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.342+0000 7f95bb600640 1 --2- 172.21.15.67:0/3496306247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4157d60 0x7f95b40acb10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f95b40c1a30 tx=0x7f95ac02be70 comp rx=0 tx=0).stop 2024-03-09T08:54:07.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.342+0000 7f95bb600640 1 -- 172.21.15.67:0/3496306247 shutdown_connections 2024-03-09T08:54:07.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.342+0000 7f95bb600640 1 --2- 172.21.15.67:0/3496306247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4157d60 0x7f95b40acb10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.342+0000 7f95bb600640 1 --2- 172.21.15.67:0/3496306247 >> v2:172.21.15.134:3300/0 conn(0x7f95b40c22a0 0x7f95b4071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.342+0000 7f95bb600640 1 -- 172.21.15.67:0/3496306247 >> 172.21.15.67:0/3496306247 conn(0x7f95b40c7370 msgr2=0x7f95b40c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.342+0000 7f95bb600640 1 -- 172.21.15.67:0/3496306247 shutdown_connections 2024-03-09T08:54:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.342+0000 7f95bb600640 1 -- 172.21.15.67:0/3496306247 wait complete. 2024-03-09T08:54:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.344+0000 7f95bb600640 1 Processor -- start 2024-03-09T08:54:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.344+0000 7f95bb600640 1 -- start start 2024-03-09T08:54:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.346+0000 7f95bb600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95b40ac730 0x7f95b41eb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.346+0000 7f95bb600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b40c22a0 0x7f95b40c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.346+0000 7f95bb600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4157d60 0x7f95b41ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.346+0000 7f95bb600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b4070c00 con 0x7f95b4157d60 2024-03-09T08:54:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.346+0000 7f95bb600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b4070920 con 0x7f95b40ac730 2024-03-09T08:54:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.346+0000 7f95bb600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b4151610 con 0x7f95b40c22a0 2024-03-09T08:54:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.346+0000 7f95b9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4157d60 0x7f95b41ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.346+0000 7f95b9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4157d60 0x7f95b41ef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57504/0 (socket says 172.21.15.67:57504) 2024-03-09T08:54:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.346+0000 7f95b9600640 1 -- 172.21.15.67:0/3700781113 learned_addr learned my addr 172.21.15.67:0/3700781113 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.346+0000 7f95b8c00640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95b40ac730 0x7f95b41eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.346+0000 7f95b3e00640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b40c22a0 0x7f95b40c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.347+0000 7f95b9600640 1 -- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b40c22a0 msgr2=0x7f95b40c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:07.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.347+0000 7f95b9600640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b40c22a0 0x7f95b40c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.347+0000 7f95b9600640 1 -- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95b40ac730 msgr2=0x7f95b41eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:07.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.347+0000 7f95b9600640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95b40ac730 0x7f95b41eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.347+0000 7f95b9600640 1 -- 172.21.15.67:0/3700781113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b41e30d0 con 0x7f95b4157d60 2024-03-09T08:54:07.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.347+0000 7f95b3e00640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b40c22a0 0x7f95b40c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:54:07.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.347+0000 7f95b9600640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4157d60 0x7f95b41ef380 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f95ac036040 tx=0x7f95ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.348+0000 7f95b0c00640 1 -- 172.21.15.67:0/3700781113 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac043020 con 0x7f95b4157d60 2024-03-09T08:54:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.348+0000 7f95b0c00640 1 -- 172.21.15.67:0/3700781113 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac03e450 con 0x7f95b4157d60 2024-03-09T08:54:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.348+0000 7f95b0c00640 1 -- 172.21.15.67:0/3700781113 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac03e820 con 0x7f95b4157d60 2024-03-09T08:54:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.348+0000 7f95bb600640 1 -- 172.21.15.67:0/3700781113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b41ebec0 con 0x7f95b4157d60 2024-03-09T08:54:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.348+0000 7f95bb600640 1 -- 172.21.15.67:0/3700781113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b41ec480 con 0x7f95b4157d60 2024-03-09T08:54:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.349+0000 7f95bb600640 1 -- 172.21.15.67:0/3700781113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b4075a00 con 0x7f95b4157d60 2024-03-09T08:54:07.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.352+0000 7f95b0c00640 1 -- 172.21.15.67:0/3700781113 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f95ac046020 con 0x7f95b4157d60 2024-03-09T08:54:07.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.352+0000 7f95b0c00640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f959406b520 0x7f959406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.353+0000 7f95b0c00640 1 -- 172.21.15.67:0/3700781113 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f95ac0ad390 con 0x7f95b4157d60 2024-03-09T08:54:07.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.353+0000 7f95b8c00640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f959406b520 0x7f959406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.354+0000 7f95b8c00640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f959406b520 0x7f959406d9e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f95b40c67b0 tx=0x7f95a80090e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:07.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.359+0000 7f95b0c00640 1 -- 172.21.15.67:0/3700781113 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f95ac04d030 con 0x7f95b4157d60 2024-03-09T08:54:07.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.714+0000 7f95bb600640 1 -- 172.21.15.67:0/3700781113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f95b40c4d40 con 0x7f95b4157d60 2024-03-09T08:54:07.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.715+0000 7f95b0c00640 1 -- 172.21.15.67:0/3700781113 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f95ac07c100 con 0x7f95b4157d60 2024-03-09T08:54:07.712 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:54:07.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.718+0000 7f95bb600640 1 -- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f959406b520 msgr2=0x7f959406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:07.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.718+0000 7f95bb600640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f959406b520 0x7f959406d9e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f95b40c67b0 tx=0x7f95a80090e0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.718+0000 7f95bb600640 1 -- 172.21.15.67:0/3700781113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4157d60 msgr2=0x7f95b41ef380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:07.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.718+0000 7f95bb600640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4157d60 0x7f95b41ef380 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f95ac036040 tx=0x7f95ac004a40 comp rx=0 tx=0).stop 2024-03-09T08:54:07.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.719+0000 7f95bb600640 1 -- 172.21.15.67:0/3700781113 shutdown_connections 2024-03-09T08:54:07.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.719+0000 7f95bb600640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f959406b520 0x7f959406d9e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.719+0000 7f95bb600640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4157d60 0x7f95b41ef380 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.719+0000 7f95bb600640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f95b40c22a0 0x7f95b40c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.719+0000 7f95bb600640 1 --2- 172.21.15.67:0/3700781113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95b40ac730 0x7f95b41eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.719+0000 7f95bb600640 1 -- 172.21.15.67:0/3700781113 >> 172.21.15.67:0/3700781113 conn(0x7f95b40c7370 msgr2=0x7f95b40c78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:07.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.719+0000 7f95bb600640 1 -- 172.21.15.67:0/3700781113 shutdown_connections 2024-03-09T08:54:07.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.719+0000 7f95bb600640 1 -- 172.21.15.67:0/3700781113 wait complete. 2024-03-09T08:54:07.738 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:54:07.738 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T08:54:07.738 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:54:07.738 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:54:07.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.966+0000 7f4e6ac00640 1 Processor -- start 2024-03-09T08:54:07.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.966+0000 7f4e6ac00640 1 -- start start 2024-03-09T08:54:07.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.968+0000 7f4e6ac00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4e6406d340 0x7f4e64152790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.968+0000 7f4e6ac00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6406d940 con 0x7f4e64150140 2024-03-09T08:54:07.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.968+0000 7f4e6ac00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6406dc20 con 0x7f4e640c2fa0 2024-03-09T08:54:07.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.968+0000 7f4e6ac00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e64071b20 con 0x7f4e6406d340 2024-03-09T08:54:07.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.968+0000 7f4e68e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4e64150140 0x7f4e64071fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44658/0 (socket says 172.21.15.67:44658) 2024-03-09T08:54:07.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.968+0000 7f4e68e00640 1 -- 172.21.15.67:0/3890906842 learned_addr learned my addr 172.21.15.67:0/3890906842 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:54:07.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.969+0000 7f4e6a200640 1 --2- 172.21.15.67:0/3890906842 >> v2:172.21.15.134:3300/0 conn(0x7f4e6406d340 0x7f4e64152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.969+0000 7f4e63e00640 1 -- 172.21.15.67:0/3890906842 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044523710 0 0) 0x7f4e6406d940 con 0x7f4e64150140 2024-03-09T08:54:07.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.969+0000 7f4e63e00640 1 -- 172.21.15.67:0/3890906842 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e4c003520 con 0x7f4e64150140 2024-03-09T08:54:07.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.969+0000 7f4e6a200640 1 -- 172.21.15.67:0/3890906842 >> v1:172.21.15.134:6789/0 conn(0x7f4e640c2fa0 legacy=0x7f4e6414fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:07.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.969+0000 7f4e6a200640 1 -- 172.21.15.67:0/3890906842 >> v1:172.21.15.67:6789/0 conn(0x7f4e64150140 legacy=0x7f4e64071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:07.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.969+0000 7f4e6a200640 1 -- 172.21.15.67:0/3890906842 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6406df00 con 0x7f4e6406d340 2024-03-09T08:54:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.971+0000 7f4e6a200640 1 --2- 172.21.15.67:0/3890906842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e6406d340 0x7f4e64152790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009d90 tx=0x7f4e5c02be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed465c7017a7cde5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.972+0000 7f4e63e00640 1 -- 172.21.15.67:0/3890906842 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c039070 con 0x7f4e6406d340 2024-03-09T08:54:07.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.972+0000 7f4e63e00640 1 -- 172.21.15.67:0/3890906842 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c02cbd0 con 0x7f4e6406d340 2024-03-09T08:54:07.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.972+0000 7f4e63e00640 1 -- 172.21.15.67:0/3890906842 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c034440 con 0x7f4e6406d340 2024-03-09T08:54:07.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.973+0000 7f4e6ac00640 1 -- 172.21.15.67:0/3890906842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e6406d340 msgr2=0x7f4e64152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:07.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.973+0000 7f4e6ac00640 1 --2- 172.21.15.67:0/3890906842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e6406d340 0x7f4e64152790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009d90 tx=0x7f4e5c02be20 comp rx=0 tx=0).stop 2024-03-09T08:54:07.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.973+0000 7f4e6ac00640 1 -- 172.21.15.67:0/3890906842 shutdown_connections 2024-03-09T08:54:07.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.973+0000 7f4e6ac00640 1 --2- 172.21.15.67:0/3890906842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e6406d340 0x7f4e64152790 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.973+0000 7f4e6ac00640 1 -- 172.21.15.67:0/3890906842 >> 172.21.15.67:0/3890906842 conn(0x7f4e640c7220 msgr2=0x7f4e640c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:07.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.973+0000 7f4e6ac00640 1 -- 172.21.15.67:0/3890906842 shutdown_connections 2024-03-09T08:54:07.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.973+0000 7f4e6ac00640 1 -- 172.21.15.67:0/3890906842 wait complete. 2024-03-09T08:54:07.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.975+0000 7f4e6ac00640 1 Processor -- start 2024-03-09T08:54:07.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.975+0000 7f4e6ac00640 1 -- start start 2024-03-09T08:54:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.976+0000 7f4e6ac00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e6406d340 0x7f4e6414d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.976+0000 7f4e6ac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640c2fa0 0x7f4e640a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.976+0000 7f4e6ac00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e64150140 0x7f4e640c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.976+0000 7f4e6ac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e64071340 con 0x7f4e640c2fa0 2024-03-09T08:54:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.976+0000 7f4e6ac00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640711c0 con 0x7f4e6406d340 2024-03-09T08:54:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.976+0000 7f4e6ac00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640c3380 con 0x7f4e64150140 2024-03-09T08:54:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.976+0000 7f4e69800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e6406d340 0x7f4e6414d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.976+0000 7f4e69800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e6406d340 0x7f4e6414d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:47138/0 (socket says 172.21.15.67:47138) 2024-03-09T08:54:07.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.976+0000 7f4e68e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640c2fa0 0x7f4e640a93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.976+0000 7f4e69800640 1 -- 172.21.15.67:0/4204446479 learned_addr learned my addr 172.21.15.67:0/4204446479 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:07.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.976+0000 7f4e6a200640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e64150140 0x7f4e640c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.977+0000 7f4e68e00640 1 -- 172.21.15.67:0/4204446479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e64150140 msgr2=0x7f4e640c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:07.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.977+0000 7f4e68e00640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e64150140 0x7f4e640c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.977+0000 7f4e68e00640 1 -- 172.21.15.67:0/4204446479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e6406d340 msgr2=0x7f4e6414d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:07.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.977+0000 7f4e68e00640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e6406d340 0x7f4e6414d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:07.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.977+0000 7f4e68e00640 1 -- 172.21.15.67:0/4204446479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e64160aa0 con 0x7f4e640c2fa0 2024-03-09T08:54:07.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.978+0000 7f4e68e00640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640c2fa0 0x7f4e640a93d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000c390 tx=0x7f4e50001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:07.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.978+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50017070 con 0x7f4e640c2fa0 2024-03-09T08:54:07.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.978+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e50005ce0 con 0x7f4e640c2fa0 2024-03-09T08:54:07.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.978+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500072f0 con 0x7f4e640c2fa0 2024-03-09T08:54:07.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.978+0000 7f4e6ac00640 1 -- 172.21.15.67:0/4204446479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6414da00 con 0x7f4e640c2fa0 2024-03-09T08:54:07.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.978+0000 7f4e6ac00640 1 -- 172.21.15.67:0/4204446479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e640a8610 con 0x7f4e640c2fa0 2024-03-09T08:54:07.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.979+0000 7f4e6ac00640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6413df00 0x7f4e640a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.979+0000 7f4e6ac00640 1 -- 172.21.15.67:0/4204446479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6404f3c0 con 0x7f4e6413df00 2024-03-09T08:54:07.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.980+0000 7f4e69800640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6413df00 0x7f4e640a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.981+0000 7f4e69800640 1 -- 172.21.15.67:0/4204446479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e640c7b10 con 0x7f4e6413df00 2024-03-09T08:54:07.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.981+0000 7f4e69800640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6413df00 0x7f4e640a8a10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800f7a0 tx=0x7f4e5800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:07.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.982+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e580050d0 con 0x7f4e6413df00 2024-03-09T08:54:07.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.982+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4e58018070 con 0x7f4e6413df00 2024-03-09T08:54:07.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.982+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6413df00 msgr2=0x7f4e640a8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:07.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.982+0000 7f4e61600640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6413df00 0x7f4e640a8a10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800f7a0 tx=0x7f4e5800fc60 comp rx=0 tx=0).stop 2024-03-09T08:54:07.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.982+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6413df00 msgr2=0x7f4e640a8a10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:07.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.982+0000 7f4e61600640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e6413df00 0x7f4e640a8a10 secure :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800f7a0 tx=0x7f4e5800fc60 comp rx=0 tx=0).stop 2024-03-09T08:54:07.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.983+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4e5001a070 con 0x7f4e640c2fa0 2024-03-09T08:54:07.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.983+0000 7f4e61600640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e4406afb0 0x7f4e4406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.983+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f4e50089840 con 0x7f4e640c2fa0 2024-03-09T08:54:07.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.984+0000 7f4e69800640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e4406afb0 0x7f4e4406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.985+0000 7f4e69800640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e4406afb0 0x7f4e4406d470 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800fca0 tx=0x7f4e58005000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:07.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.996+0000 7f4e6ac00640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640a8f50 0x7f4e6413c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:07.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.996+0000 7f4e6ac00640 1 -- 172.21.15.67:0/4204446479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640c0e00 con 0x7f4e640a8f50 2024-03-09T08:54:07.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.996+0000 7f4e6a200640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640a8f50 0x7f4e6413c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.997+0000 7f4e6a200640 1 -- 172.21.15.67:0/4204446479 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4e64067b30 con 0x7f4e640a8f50 2024-03-09T08:54:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.997+0000 7f4e6a200640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640a8f50 0x7f4e6413c800 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4e640aa170 tx=0x7f4e5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.998+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c046070 con 0x7f4e640a8f50 2024-03-09T08:54:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.998+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f4e5c034dd0 con 0x7f4e640a8f50 2024-03-09T08:54:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.998+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640a8f50 msgr2=0x7f4e6413c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.998+0000 7f4e61600640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640a8f50 0x7f4e6413c800 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4e640aa170 tx=0x7f4e5c004a40 comp rx=0 tx=0).stop 2024-03-09T08:54:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.998+0000 7f4e61600640 1 -- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640a8f50 msgr2=0x7f4e6413c800 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.998+0000 7f4e61600640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640a8f50 0x7f4e6413c800 secure :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4e640aa170 tx=0x7f4e5c004a40 comp rx=0 tx=0).stop 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 16, 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:54:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:54:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:54:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:54:08.009 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:54:08.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.999+0000 7f4e6ac00640 1 -- 172.21.15.67:0/4204446479 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e4406afb0 msgr2=0x7f4e4406d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.999+0000 7f4e6ac00640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e4406afb0 0x7f4e4406d470 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800fca0 tx=0x7f4e58005000 comp rx=0 tx=0).stop 2024-03-09T08:54:08.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.999+0000 7f4e6ac00640 1 -- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640c2fa0 msgr2=0x7f4e640a93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.999+0000 7f4e6ac00640 1 --2- 172.21.15.67:0/4204446479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e640c2fa0 0x7f4e640a93d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000c390 tx=0x7f4e50001e70 comp rx=0 tx=0).stop 2024-03-09T08:54:08.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.999+0000 7f4e6a200640 1 -- 172.21.15.67:0/4204446479 reap_dead start 2024-03-09T08:54:08.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.999+0000 7f4e6ac00640 1 -- 172.21.15.67:0/4204446479 shutdown_connections 2024-03-09T08:54:08.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.999+0000 7f4e6ac00640 1 -- 172.21.15.67:0/4204446479 >> 172.21.15.67:0/4204446479 conn(0x7f4e640c7220 msgr2=0x7f4e6414b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:08.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.999+0000 7f4e6ac00640 1 -- 172.21.15.67:0/4204446479 shutdown_connections 2024-03-09T08:54:08.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:07.999+0000 7f4e6ac00640 1 -- 172.21.15.67:0/4204446479 wait complete. 2024-03-09T08:54:08.011 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:54:08.011 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:54:08.012 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:54:08.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.204+0000 7fe173200640 1 Processor -- start 2024-03-09T08:54:08.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.204+0000 7fe173200640 1 -- start start 2024-03-09T08:54:08.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.206+0000 7fe173200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe174154410 0x7fe17406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.206+0000 7fe173200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe17406fcd0 0x7fe17415e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.206+0000 7fe173200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1740c51b0 con 0x7fe174150fb0 2024-03-09T08:54:08.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.206+0000 7fe173200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1740c5aa0 con 0x7fe174154410 2024-03-09T08:54:08.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.206+0000 7fe173200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe174151390 con 0x7fe17406fcd0 2024-03-09T08:54:08.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.207+0000 7fe171e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe174150fb0 0x7fe1740c5e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44670/0 (socket says 172.21.15.67:44670) 2024-03-09T08:54:08.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.207+0000 7fe171e00640 1 -- 172.21.15.67:0/2148100146 learned_addr learned my addr 172.21.15.67:0/2148100146 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:54:08.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.207+0000 7fe171400640 1 --2- 172.21.15.67:0/2148100146 >> v2:172.21.15.67:3300/0 conn(0x7fe174154410 0x7fe17406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.207+0000 7fe172800640 1 --2- 172.21.15.67:0/2148100146 >> v2:172.21.15.134:3300/0 conn(0x7fe17406fcd0 0x7fe17415e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.208+0000 7fe170a00640 1 -- 172.21.15.67:0/2148100146 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683546736 0 0) 0x7fe1740c51b0 con 0x7fe174150fb0 2024-03-09T08:54:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.208+0000 7fe170a00640 1 -- 172.21.15.67:0/2148100146 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe158003520 con 0x7fe174150fb0 2024-03-09T08:54:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.208+0000 7fe172800640 1 -- 172.21.15.67:0/2148100146 >> v2:172.21.15.67:3300/0 conn(0x7fe174154410 msgr2=0x7fe17406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.208+0000 7fe172800640 1 --2- 172.21.15.67:0/2148100146 >> v2:172.21.15.67:3300/0 conn(0x7fe174154410 0x7fe17406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.208+0000 7fe172800640 1 -- 172.21.15.67:0/2148100146 >> v1:172.21.15.67:6789/0 conn(0x7fe174150fb0 legacy=0x7fe1740c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.208+0000 7fe172800640 1 -- 172.21.15.67:0/2148100146 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1740c5450 con 0x7fe17406fcd0 2024-03-09T08:54:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.211+0000 7fe172800640 1 --2- 172.21.15.67:0/2148100146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe17406fcd0 0x7fe17415e310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe174067a60 tx=0x7fe16c02bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b5ec33c274e2bc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.212+0000 7fe170a00640 1 -- 172.21.15.67:0/2148100146 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c039070 con 0x7fe17406fcd0 2024-03-09T08:54:08.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.212+0000 7fe170a00640 1 -- 172.21.15.67:0/2148100146 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c02cb50 con 0x7fe17406fcd0 2024-03-09T08:54:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.212+0000 7fe170a00640 1 -- 172.21.15.67:0/2148100146 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c034440 con 0x7fe17406fcd0 2024-03-09T08:54:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.212+0000 7fe173200640 1 -- 172.21.15.67:0/2148100146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe17406fcd0 msgr2=0x7fe17415e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.212+0000 7fe173200640 1 --2- 172.21.15.67:0/2148100146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe17406fcd0 0x7fe17415e310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe174067a60 tx=0x7fe16c02bda0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.212+0000 7fe173200640 1 -- 172.21.15.67:0/2148100146 shutdown_connections 2024-03-09T08:54:08.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.213+0000 7fe173200640 1 --2- 172.21.15.67:0/2148100146 >> v2:172.21.15.67:3300/0 conn(0x7fe174154410 0x7fe17406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.213+0000 7fe173200640 1 --2- 172.21.15.67:0/2148100146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe17406fcd0 0x7fe17415e310 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.213+0000 7fe173200640 1 -- 172.21.15.67:0/2148100146 >> 172.21.15.67:0/2148100146 conn(0x7fe17406f060 msgr2=0x7fe174066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:08.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.213+0000 7fe173200640 1 -- 172.21.15.67:0/2148100146 shutdown_connections 2024-03-09T08:54:08.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.213+0000 7fe173200640 1 -- 172.21.15.67:0/2148100146 wait complete. 2024-03-09T08:54:08.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.215+0000 7fe173200640 1 Processor -- start 2024-03-09T08:54:08.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.215+0000 7fe173200640 1 -- start start 2024-03-09T08:54:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.217+0000 7fe173200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe17406fcd0 0x7fe17406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.217+0000 7fe173200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe174150fb0 0x7fe1740c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.217+0000 7fe173200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe174154410 0x7fe174147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.217+0000 7fe173200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1741548f0 con 0x7fe174150fb0 2024-03-09T08:54:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.217+0000 7fe173200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe174072a50 con 0x7fe17406fcd0 2024-03-09T08:54:08.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.217+0000 7fe173200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe174073bc0 con 0x7fe174154410 2024-03-09T08:54:08.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.217+0000 7fe171400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe174150fb0 0x7fe1740c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.217+0000 7fe171e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe17406fcd0 0x7fe17406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.217+0000 7fe171400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe174150fb0 0x7fe1740c2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57548/0 (socket says 172.21.15.67:57548) 2024-03-09T08:54:08.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.217+0000 7fe171400640 1 -- 172.21.15.67:0/841590221 learned_addr learned my addr 172.21.15.67:0/841590221 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:08.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.218+0000 7fe172800640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe174154410 0x7fe174147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.218+0000 7fe171400640 1 -- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe174154410 msgr2=0x7fe174147d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.218+0000 7fe171400640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe174154410 0x7fe174147d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.218+0000 7fe171400640 1 -- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe17406fcd0 msgr2=0x7fe17406cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.218+0000 7fe171400640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe17406fcd0 0x7fe17406cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.218+0000 7fe171400640 1 -- 172.21.15.67:0/841590221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17406c1f0 con 0x7fe174150fb0 2024-03-09T08:54:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.218+0000 7fe171400640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe174150fb0 0x7fe1740c2a30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe16000e980 tx=0x7fe16000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.219+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe160017070 con 0x7fe174150fb0 2024-03-09T08:54:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.219+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16000ca00 con 0x7fe174150fb0 2024-03-09T08:54:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.219+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16000cdd0 con 0x7fe174150fb0 2024-03-09T08:54:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.219+0000 7fe173200640 1 -- 172.21.15.67:0/841590221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1740c73e0 con 0x7fe174150fb0 2024-03-09T08:54:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.219+0000 7fe173200640 1 -- 172.21.15.67:0/841590221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1740c2f70 con 0x7fe174150fb0 2024-03-09T08:54:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.220+0000 7fe173200640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe174148290 0x7fe1741eba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.220+0000 7fe173200640 1 -- 172.21.15.67:0/841590221 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17406d1a0 con 0x7fe174148290 2024-03-09T08:54:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.222+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe16001a070 con 0x7fe174150fb0 2024-03-09T08:54:08.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.222+0000 7fe171e00640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe174148290 0x7fe1741eba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.223+0000 7fe171e00640 1 -- 172.21.15.67:0/841590221 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1740c7190 con 0x7fe174148290 2024-03-09T08:54:08.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.223+0000 7fe171e00640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe174148290 0x7fe1741eba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe17406d550 tx=0x7fe16400fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.223+0000 7fe16a000640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe15406b4d0 0x7fe15406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.224+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fe16008a460 con 0x7fe174150fb0 2024-03-09T08:54:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.224+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1640058b0 con 0x7fe174148290 2024-03-09T08:54:08.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.225+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fe164018070 con 0x7fe174148290 2024-03-09T08:54:08.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.225+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe174148290 msgr2=0x7fe1741eba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.225+0000 7fe16a000640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe174148290 0x7fe1741eba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe17406d550 tx=0x7fe16400fc00 comp rx=0 tx=0).stop 2024-03-09T08:54:08.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.225+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe174148290 msgr2=0x7fe1741eba70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:08.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.225+0000 7fe16a000640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe174148290 0x7fe1741eba70 secure :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe17406d550 tx=0x7fe16400fc00 comp rx=0 tx=0).stop 2024-03-09T08:54:08.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.226+0000 7fe172800640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe15406b4d0 0x7fe15406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.230+0000 7fe172800640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe15406b4d0 0x7fe15406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe16c036000 tx=0x7fe16c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.233+0000 7fe173200640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1741ebfb0 0x7fe1741eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.233+0000 7fe173200640 1 -- 172.21.15.67:0/841590221 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1740c3850 con 0x7fe1741ebfb0 2024-03-09T08:54:08.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.233+0000 7fe171e00640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1741ebfb0 0x7fe1741eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.234+0000 7fe171e00640 1 -- 172.21.15.67:0/841590221 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe174073790 con 0x7fe1741ebfb0 2024-03-09T08:54:08.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.234+0000 7fe171e00640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1741ebfb0 0x7fe1741eefe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe17406d550 tx=0x7fe1640010a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.235+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164007720 con 0x7fe1741ebfb0 2024-03-09T08:54:08.232 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:54:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:54:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:54:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:54:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 16, 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:54:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:54:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:54:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:54:08.244 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:54:08.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.235+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fe164007720 con 0x7fe1741ebfb0 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.235+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1741ebfb0 msgr2=0x7fe1741eefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.235+0000 7fe16a000640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1741ebfb0 0x7fe1741eefe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe17406d550 tx=0x7fe1640010a0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.235+0000 7fe16a000640 1 -- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1741ebfb0 msgr2=0x7fe1741eefe0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.235+0000 7fe16a000640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1741ebfb0 0x7fe1741eefe0 secure :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe17406d550 tx=0x7fe1640010a0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.236+0000 7fe173200640 1 -- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe15406b4d0 msgr2=0x7fe15406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.236+0000 7fe173200640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe15406b4d0 0x7fe15406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe16c036000 tx=0x7fe16c030000 comp rx=0 tx=0).stop 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.236+0000 7fe173200640 1 -- 172.21.15.67:0/841590221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe174150fb0 msgr2=0x7fe1740c2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.236+0000 7fe173200640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe174150fb0 0x7fe1740c2a30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe16000e980 tx=0x7fe16000ee40 comp rx=0 tx=0).stop 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.236+0000 7fe172800640 1 -- 172.21.15.67:0/841590221 reap_dead start 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.236+0000 7fe173200640 1 -- 172.21.15.67:0/841590221 shutdown_connections 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.236+0000 7fe173200640 1 --2- 172.21.15.67:0/841590221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe174150fb0 0x7fe1740c2a30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.236+0000 7fe173200640 1 -- 172.21.15.67:0/841590221 >> 172.21.15.67:0/841590221 conn(0x7fe17406f060 msgr2=0x7fe174066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:08.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.236+0000 7fe173200640 1 -- 172.21.15.67:0/841590221 shutdown_connections 2024-03-09T08:54:08.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.236+0000 7fe173200640 1 -- 172.21.15.67:0/841590221 wait complete. 2024-03-09T08:54:08.254 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:54:08.255 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:54:08.255 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:54:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.442+0000 7f855a600640 1 Processor -- start 2024-03-09T08:54:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.442+0000 7f855a600640 1 -- start start 2024-03-09T08:54:08.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.444+0000 7f855a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f855414fd90 0x7f8554156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.444+0000 7f855a600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f85540c9380 0x7f85540acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.444+0000 7f855a600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8554156bc0 con 0x7f85540ad440 2024-03-09T08:54:08.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.444+0000 7f855a600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855406ba60 con 0x7f855414fd90 2024-03-09T08:54:08.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.444+0000 7f855a600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85540c9890 con 0x7f85540c9380 2024-03-09T08:54:08.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.445+0000 7f8559200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f855414fd90 0x7f8554156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.445+0000 7f8559200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f855414fd90 0x7f8554156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57552/0 (socket says 172.21.15.67:57552) 2024-03-09T08:54:08.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.445+0000 7f8559200640 1 -- 172.21.15.67:0/2446182654 learned_addr learned my addr 172.21.15.67:0/2446182654 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:08.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.445+0000 7f8553e00640 1 --2- 172.21.15.67:0/2446182654 >> v2:172.21.15.134:3300/0 conn(0x7f85540c9380 0x7f85540acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.445+0000 7f8559200640 1 -- 172.21.15.67:0/2446182654 >> v2:172.21.15.134:3300/0 conn(0x7f85540c9380 msgr2=0x7f85540acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:08.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.445+0000 7f8559200640 1 --2- 172.21.15.67:0/2446182654 >> v2:172.21.15.134:3300/0 conn(0x7f85540c9380 0x7f85540acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.445+0000 7f8559200640 1 -- 172.21.15.67:0/2446182654 >> v1:172.21.15.67:6789/0 conn(0x7f85540ad440 legacy=0x7f855406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:08.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.445+0000 7f8559200640 1 -- 172.21.15.67:0/2446182654 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85540c5f80 con 0x7f855414fd90 2024-03-09T08:54:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.447+0000 7f8559200640 1 --2- 172.21.15.67:0/2446182654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f855414fd90 0x7f8554156680 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f85540ae040 tx=0x7f854802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41134e9d30d059bb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.447+0000 7f8553400640 1 -- 172.21.15.67:0/2446182654 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548038070 con 0x7f855414fd90 2024-03-09T08:54:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.447+0000 7f8553400640 1 -- 172.21.15.67:0/2446182654 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854802f050 con 0x7f855414fd90 2024-03-09T08:54:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.447+0000 7f8553400640 1 -- 172.21.15.67:0/2446182654 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548031970 con 0x7f855414fd90 2024-03-09T08:54:08.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.448+0000 7f855a600640 1 -- 172.21.15.67:0/2446182654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f855414fd90 msgr2=0x7f8554156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.448+0000 7f855a600640 1 --2- 172.21.15.67:0/2446182654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f855414fd90 0x7f8554156680 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f85540ae040 tx=0x7f854802b7f0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.448+0000 7f855a600640 1 -- 172.21.15.67:0/2446182654 shutdown_connections 2024-03-09T08:54:08.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.448+0000 7f855a600640 1 --2- 172.21.15.67:0/2446182654 >> v2:172.21.15.134:3300/0 conn(0x7f85540c9380 0x7f85540acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.448+0000 7f855a600640 1 --2- 172.21.15.67:0/2446182654 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f855414fd90 0x7f8554156680 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.448+0000 7f855a600640 1 -- 172.21.15.67:0/2446182654 >> 172.21.15.67:0/2446182654 conn(0x7f85540cb480 msgr2=0x7f85540cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:08.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.448+0000 7f855a600640 1 -- 172.21.15.67:0/2446182654 shutdown_connections 2024-03-09T08:54:08.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.449+0000 7f855a600640 1 -- 172.21.15.67:0/2446182654 wait complete. 2024-03-09T08:54:08.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.451+0000 7f855a600640 1 Processor -- start 2024-03-09T08:54:08.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.451+0000 7f855a600640 1 -- start start 2024-03-09T08:54:08.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.453+0000 7f855a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85540ad440 0x7f85540b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.453+0000 7f855a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85540c9380 0x7f85541e1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.453+0000 7f855a600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f855414fd90 0x7f85541ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.453+0000 7f855a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855406bcb0 con 0x7f85540ad440 2024-03-09T08:54:08.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.453+0000 7f855a600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85540c6520 con 0x7f85540c9380 2024-03-09T08:54:08.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.453+0000 7f855a600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855406c8e0 con 0x7f855414fd90 2024-03-09T08:54:08.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.453+0000 7f8559200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85540ad440 0x7f85540b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.453+0000 7f8553e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85540c9380 0x7f85541e1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.453+0000 7f8559200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85540ad440 0x7f85540b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57568/0 (socket says 172.21.15.67:57568) 2024-03-09T08:54:08.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.454+0000 7f8553e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85540c9380 0x7f85541e1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:47198/0 (socket says 172.21.15.67:47198) 2024-03-09T08:54:08.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.453+0000 7f8559c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f855414fd90 0x7f85541ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.454+0000 7f8559200640 1 -- 172.21.15.67:0/1959279989 learned_addr learned my addr 172.21.15.67:0/1959279989 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:08.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.454+0000 7f8553e00640 1 -- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f855414fd90 msgr2=0x7f85541ef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.454+0000 7f8553e00640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f855414fd90 0x7f85541ef380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.454+0000 7f8553e00640 1 -- 172.21.15.67:0/1959279989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85540ad440 msgr2=0x7f85540b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.454+0000 7f8553e00640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85540ad440 0x7f85540b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.454+0000 7f8553e00640 1 -- 172.21.15.67:0/1959279989 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855414f540 con 0x7f85540c9380 2024-03-09T08:54:08.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.454+0000 7f8559c00640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f855414fd90 0x7f85541ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:54:08.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.454+0000 7f8559200640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85540ad440 0x7f85540b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:54:08.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.455+0000 7f8553e00640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85540c9380 0x7f85541e1b40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f854000c610 tx=0x7f854000cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.455+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540015070 con 0x7f85540c9380 2024-03-09T08:54:08.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.455+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8540004220 con 0x7f85540c9380 2024-03-09T08:54:08.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.455+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540003610 con 0x7f85540c9380 2024-03-09T08:54:08.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.455+0000 7f855a600640 1 -- 172.21.15.67:0/1959279989 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85540ab110 con 0x7f85540c9380 2024-03-09T08:54:08.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.455+0000 7f855a600640 1 -- 172.21.15.67:0/1959279989 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85540ab6d0 con 0x7f85540c9380 2024-03-09T08:54:08.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.457+0000 7f855a600640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541eeef0 0x7f8554145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.457+0000 7f855a600640 1 -- 172.21.15.67:0/1959279989 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855404f3c0 con 0x7f85541eeef0 2024-03-09T08:54:08.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.457+0000 7f8559200640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541eeef0 0x7f8554145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.458+0000 7f8559200640 1 -- 172.21.15.67:0/1959279989 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f855406d2d0 con 0x7f85541eeef0 2024-03-09T08:54:08.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.458+0000 7f8559200640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541eeef0 0x7f8554145530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f85480090b0 tx=0x7f85480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.458+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f85400037b0 con 0x7f85540c9380 2024-03-09T08:54:08.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.460+0000 7f8550c00640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f853406b4d0 0x7f853406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.460+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f854008a6e0 con 0x7f85540c9380 2024-03-09T08:54:08.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.460+0000 7f8559c00640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f853406b4d0 0x7f853406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.461+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548038040 con 0x7f85541eeef0 2024-03-09T08:54:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.461+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f8548033b50 con 0x7f85541eeef0 2024-03-09T08:54:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.461+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541eeef0 msgr2=0x7f8554145530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.461+0000 7f8550c00640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541eeef0 0x7f8554145530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f85480090b0 tx=0x7f85480046c0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.461+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541eeef0 msgr2=0x7f8554145530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:08.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.461+0000 7f8550c00640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541eeef0 0x7f8554145530 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f85480090b0 tx=0x7f85480046c0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.461+0000 7f8559c00640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f853406b4d0 0x7f853406d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f854c00f1d0 tx=0x7f854c003d10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.474+0000 7f855a600640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541ef8c0 0x7f8554145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.474+0000 7f855a600640 1 -- 172.21.15.67:0/1959279989 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855414e920 con 0x7f85541ef8c0 2024-03-09T08:54:08.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.475+0000 7f8559200640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541ef8c0 0x7f8554145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.475+0000 7f8559200640 1 -- 172.21.15.67:0/1959279989 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f85540008d0 con 0x7f85541ef8c0 2024-03-09T08:54:08.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.476+0000 7f8559200640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541ef8c0 0x7f8554145a70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f85540ab8a0 tx=0x7f8548031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.476+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548052070 con 0x7f85541ef8c0 2024-03-09T08:54:08.473 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 16, 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:54:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:54:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:54:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:54:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:54:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:54:08.487 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:54:08.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.476+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7f854802f040 con 0x7f85541ef8c0 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.476+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541ef8c0 msgr2=0x7f8554145a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.476+0000 7f8550c00640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541ef8c0 0x7f8554145a70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f85540ab8a0 tx=0x7f8548031000 comp rx=0 tx=0).stop 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.476+0000 7f8550c00640 1 -- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541ef8c0 msgr2=0x7f8554145a70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.476+0000 7f8550c00640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85541ef8c0 0x7f8554145a70 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f85540ab8a0 tx=0x7f8548031000 comp rx=0 tx=0).stop 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.477+0000 7f855a600640 1 -- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f853406b4d0 msgr2=0x7f853406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.477+0000 7f855a600640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f853406b4d0 0x7f853406d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f854c00f1d0 tx=0x7f854c003d10 comp rx=0 tx=0).stop 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.477+0000 7f855a600640 1 -- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85540c9380 msgr2=0x7f85541e1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.477+0000 7f855a600640 1 --2- 172.21.15.67:0/1959279989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85540c9380 0x7f85541e1b40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f854000c610 tx=0x7f854000cad0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.477+0000 7f8559c00640 1 -- 172.21.15.67:0/1959279989 reap_dead start 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.477+0000 7f8559c00640 1 -- 172.21.15.67:0/1959279989 reap_dead start 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.477+0000 7f855a600640 1 -- 172.21.15.67:0/1959279989 shutdown_connections 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.477+0000 7f855a600640 1 -- 172.21.15.67:0/1959279989 >> 172.21.15.67:0/1959279989 conn(0x7f85540cb480 msgr2=0x7f855414bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:08.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.478+0000 7f855a600640 1 -- 172.21.15.67:0/1959279989 shutdown_connections 2024-03-09T08:54:08.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.478+0000 7f855a600640 1 -- 172.21.15.67:0/1959279989 wait complete. 2024-03-09T08:54:08.495 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:54:08.496 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T08:54:08.496 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T08:54:08.496 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T08:54:08.496 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T08:54:08.496 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T08:54:08.496 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T08:54:08.525 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T08:54:08.525 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T08:54:08.525 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:54:08.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.757+0000 7fd1f7200640 1 Processor -- start 2024-03-09T08:54:08.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.758+0000 7fd1f7200640 1 -- start start 2024-03-09T08:54:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.759+0000 7fd1f7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1f00710a0 0x7fd1f0153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.759+0000 7fd1f7200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd1f0151360 0x7fd1f0153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.759+0000 7fd1f7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f00c81e0 con 0x7fd1f00c1f70 2024-03-09T08:54:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.759+0000 7fd1f7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f00c32b0 con 0x7fd1f00710a0 2024-03-09T08:54:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.759+0000 7fd1f7200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f00c2fd0 con 0x7fd1f0151360 2024-03-09T08:54:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.759+0000 7fd1f5e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd1f00c1f70 0x7fd1f0150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44694/0 (socket says 172.21.15.67:44694) 2024-03-09T08:54:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.759+0000 7fd1f5e00640 1 -- 172.21.15.67:0/64540745 learned_addr learned my addr 172.21.15.67:0/64540745 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:54:08.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.759+0000 7fd1f5400640 1 --2- 172.21.15.67:0/64540745 >> v2:172.21.15.67:3300/0 conn(0x7fd1f00710a0 0x7fd1f0153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.759+0000 7fd1f6800640 1 --2- 172.21.15.67:0/64540745 >> v2:172.21.15.134:3300/0 conn(0x7fd1f0151360 0x7fd1f0153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.760+0000 7fd1f4a00640 1 -- 172.21.15.67:0/64540745 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507830465 0 0) 0x7fd1f00c81e0 con 0x7fd1f00c1f70 2024-03-09T08:54:08.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.760+0000 7fd1f4a00640 1 -- 172.21.15.67:0/64540745 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1d8003520 con 0x7fd1f00c1f70 2024-03-09T08:54:08.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.760+0000 7fd1f5400640 1 -- 172.21.15.67:0/64540745 >> v2:172.21.15.134:3300/0 conn(0x7fd1f0151360 msgr2=0x7fd1f0153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:08.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.760+0000 7fd1f5400640 1 --2- 172.21.15.67:0/64540745 >> v2:172.21.15.134:3300/0 conn(0x7fd1f0151360 0x7fd1f0153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.760+0000 7fd1f5400640 1 -- 172.21.15.67:0/64540745 >> v1:172.21.15.67:6789/0 conn(0x7fd1f00c1f70 legacy=0x7fd1f0150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.760+0000 7fd1f5400640 1 -- 172.21.15.67:0/64540745 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f00c3700 con 0x7fd1f00710a0 2024-03-09T08:54:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.760+0000 7fd1f6800640 1 --2- 172.21.15.67:0/64540745 >> v2:172.21.15.134:3300/0 conn(0x7fd1f0151360 0x7fd1f0153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:54:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.762+0000 7fd1f5400640 1 --2- 172.21.15.67:0/64540745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f00710a0 0x7fd1f0153970 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0073ef0 tx=0x7fd1e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877dac597987f4e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.762+0000 7fd1f4a00640 1 -- 172.21.15.67:0/64540745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00315a0 con 0x7fd1f00710a0 2024-03-09T08:54:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.762+0000 7fd1f4a00640 1 -- 172.21.15.67:0/64540745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e0037070 con 0x7fd1f00710a0 2024-03-09T08:54:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.762+0000 7fd1f4a00640 1 -- 172.21.15.67:0/64540745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0031960 con 0x7fd1f00710a0 2024-03-09T08:54:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.763+0000 7fd1f7200640 1 -- 172.21.15.67:0/64540745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f00710a0 msgr2=0x7fd1f0153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.763+0000 7fd1f7200640 1 --2- 172.21.15.67:0/64540745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f00710a0 0x7fd1f0153970 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0073ef0 tx=0x7fd1e002b840 comp rx=0 tx=0).stop 2024-03-09T08:54:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.763+0000 7fd1f7200640 1 -- 172.21.15.67:0/64540745 shutdown_connections 2024-03-09T08:54:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.763+0000 7fd1f7200640 1 --2- 172.21.15.67:0/64540745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f00710a0 0x7fd1f0153970 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.763+0000 7fd1f7200640 1 --2- 172.21.15.67:0/64540745 >> v2:172.21.15.134:3300/0 conn(0x7fd1f0151360 0x7fd1f0153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.763+0000 7fd1f7200640 1 -- 172.21.15.67:0/64540745 >> 172.21.15.67:0/64540745 conn(0x7fd1f00c7300 msgr2=0x7fd1f00c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.763+0000 7fd1f7200640 1 -- 172.21.15.67:0/64540745 shutdown_connections 2024-03-09T08:54:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.763+0000 7fd1f7200640 1 -- 172.21.15.67:0/64540745 wait complete. 2024-03-09T08:54:08.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.766+0000 7fd1f7200640 1 Processor -- start 2024-03-09T08:54:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.766+0000 7fd1f7200640 1 -- start start 2024-03-09T08:54:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1f00710a0 0x7fd1f01420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f00c1f70 0x7fd1f01eb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1f0151360 0x7fd1f0144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0070d50 con 0x7fd1f00c1f70 2024-03-09T08:54:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0070830 con 0x7fd1f0151360 2024-03-09T08:54:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0070ed0 con 0x7fd1f00710a0 2024-03-09T08:54:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f00c1f70 0x7fd1f01eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f5e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1f00710a0 msgr2=0x7fd1f01420a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f00c1f70 0x7fd1f01eb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57604/0 (socket says 172.21.15.67:57604) 2024-03-09T08:54:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f5400640 1 -- 172.21.15.67:0/185440898 learned_addr learned my addr 172.21.15.67:0/185440898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f5e00640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1f00710a0 0x7fd1f01420a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.767+0000 7fd1f6800640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1f0151360 0x7fd1f0144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.768+0000 7fd1f5400640 1 -- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1f00710a0 msgr2=0x7fd1f01420a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:54:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.768+0000 7fd1f5400640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1f00710a0 0x7fd1f01420a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.768+0000 7fd1f5400640 1 -- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1f0151360 msgr2=0x7fd1f0144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.768+0000 7fd1f5400640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1f0151360 0x7fd1f0144fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.768+0000 7fd1f5400640 1 -- 172.21.15.67:0/185440898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f014b310 con 0x7fd1f00c1f70 2024-03-09T08:54:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.768+0000 7fd1f5400640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f00c1f70 0x7fd1f01eb1e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0000c00 tx=0x7fd1e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.768+0000 7fd1ea000640 1 -- 172.21.15.67:0/185440898 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0032630 con 0x7fd1f00c1f70 2024-03-09T08:54:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.769+0000 7fd1ea000640 1 -- 172.21.15.67:0/185440898 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e00327f0 con 0x7fd1f00c1f70 2024-03-09T08:54:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.769+0000 7fd1ea000640 1 -- 172.21.15.67:0/185440898 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0032be0 con 0x7fd1f00c1f70 2024-03-09T08:54:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.769+0000 7fd1f7200640 1 -- 172.21.15.67:0/185440898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f01ea740 con 0x7fd1f00c1f70 2024-03-09T08:54:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.769+0000 7fd1f7200640 1 -- 172.21.15.67:0/185440898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f01eacc0 con 0x7fd1f00c1f70 2024-03-09T08:54:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.771+0000 7fd1f7200640 1 -- 172.21.15.67:0/185440898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f004f3c0 con 0x7fd1f00c1f70 2024-03-09T08:54:08.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.772+0000 7fd1ea000640 1 -- 172.21.15.67:0/185440898 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd1e003b3f0 con 0x7fd1f00c1f70 2024-03-09T08:54:08.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.774+0000 7fd1ea000640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd1d406b4d0 0x7fd1d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:08.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.774+0000 7fd1ea000640 1 -- 172.21.15.67:0/185440898 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fd1e002f030 con 0x7fd1f00c1f70 2024-03-09T08:54:08.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.775+0000 7fd1f5e00640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd1d406b4d0 0x7fd1d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:08.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.775+0000 7fd1f5e00640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd1d406b4d0 0x7fd1d406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4000c00 tx=0x7fd1e4007a10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:08.779+0000 7fd1ea000640 1 -- 172.21.15.67:0/185440898 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd1e0040070 con 0x7fd1f00c1f70 2024-03-09T08:54:09.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.117+0000 7fd1f7200640 1 -- 172.21.15.67:0/185440898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd1f0149d90 con 0x7fd1f00c1f70 2024-03-09T08:54:09.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.118+0000 7fd1ea000640 1 -- 172.21.15.67:0/185440898 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fd1e007af50 con 0x7fd1f00c1f70 2024-03-09T08:54:09.117 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:54:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.121+0000 7fd1f7200640 1 -- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd1d406b4d0 msgr2=0x7fd1d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.121+0000 7fd1f7200640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd1d406b4d0 0x7fd1d406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4000c00 tx=0x7fd1e4007a10 comp rx=0 tx=0).stop 2024-03-09T08:54:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.121+0000 7fd1f7200640 1 -- 172.21.15.67:0/185440898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f00c1f70 msgr2=0x7fd1f01eb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.121+0000 7fd1f7200640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f00c1f70 0x7fd1f01eb1e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0000c00 tx=0x7fd1e00046c0 comp rx=0 tx=0).stop 2024-03-09T08:54:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.121+0000 7fd1f7200640 1 -- 172.21.15.67:0/185440898 shutdown_connections 2024-03-09T08:54:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.121+0000 7fd1f7200640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd1d406b4d0 0x7fd1d406d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.122+0000 7fd1f7200640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1f0151360 0x7fd1f0144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.122+0000 7fd1f7200640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f00c1f70 0x7fd1f01eb1e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.122+0000 7fd1f7200640 1 --2- 172.21.15.67:0/185440898 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1f00710a0 0x7fd1f01420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.122+0000 7fd1f7200640 1 -- 172.21.15.67:0/185440898 >> 172.21.15.67:0/185440898 conn(0x7fd1f00c7300 msgr2=0x7fd1f00c2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.122+0000 7fd1f7200640 1 -- 172.21.15.67:0/185440898 shutdown_connections 2024-03-09T08:54:09.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:09.122+0000 7fd1f7200640 1 -- 172.21.15.67:0/185440898 wait complete. 2024-03-09T08:54:09.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:54:09.241 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:54:12.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:54:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.393+0000 7fe735a00640 1 Processor -- start 2024-03-09T08:54:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.393+0000 7fe735a00640 1 -- start start 2024-03-09T08:54:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.394+0000 7fe735a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 0x7fe7300b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.394+0000 7fe735a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe7300c5e40 0x7fe7300b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.394+0000 7fe735a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe730157000 con 0x7fe7300cadb0 2024-03-09T08:54:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.394+0000 7fe735a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7300ae430 con 0x7fe7300c5e40 2024-03-09T08:54:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.394+0000 7fe735a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7300ae6f0 con 0x7fe730153300 2024-03-09T08:54:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.395+0000 7fe72fe00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 msgr2=0x7fe7300b0000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.395+0000 7fe72fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 0x7fe7300b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.395+0000 7fe735000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe7300cadb0 0x7fe730156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59872/0 (socket says 172.21.15.67:59872) 2024-03-09T08:54:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.395+0000 7fe735000640 1 -- 172.21.15.67:0/2753358199 learned_addr learned my addr 172.21.15.67:0/2753358199 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:54:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.395+0000 7fe72f400640 1 --2- 172.21.15.67:0/2753358199 >> v2:172.21.15.134:3300/0 conn(0x7fe7300c5e40 0x7fe7300b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.396+0000 7fe72ea00640 1 -- 172.21.15.67:0/2753358199 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919184346 0 0) 0x7fe730157000 con 0x7fe7300cadb0 2024-03-09T08:54:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.396+0000 7fe72ea00640 1 -- 172.21.15.67:0/2753358199 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe714003520 con 0x7fe7300cadb0 2024-03-09T08:54:12.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.396+0000 7fe72f400640 1 -- 172.21.15.67:0/2753358199 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 msgr2=0x7fe7300b0000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:54:12.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.396+0000 7fe72f400640 1 --2- 172.21.15.67:0/2753358199 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 0x7fe7300b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:12.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.396+0000 7fe72f400640 1 -- 172.21.15.67:0/2753358199 >> v1:172.21.15.67:6789/0 conn(0x7fe7300cadb0 legacy=0x7fe730156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:12.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.396+0000 7fe72f400640 1 -- 172.21.15.67:0/2753358199 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe730155b10 con 0x7fe7300c5e40 2024-03-09T08:54:12.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.398+0000 7fe72f400640 1 --2- 172.21.15.67:0/2753358199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7300c5e40 0x7fe7300b0540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d8752d408812fb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.398+0000 7fe72ea00640 1 -- 172.21.15.67:0/2753358199 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c035020 con 0x7fe7300c5e40 2024-03-09T08:54:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.398+0000 7fe72ea00640 1 -- 172.21.15.67:0/2753358199 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c031830 con 0x7fe7300c5e40 2024-03-09T08:54:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.398+0000 7fe72ea00640 1 -- 172.21.15.67:0/2753358199 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c031be0 con 0x7fe7300c5e40 2024-03-09T08:54:12.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.399+0000 7fe735a00640 1 -- 172.21.15.67:0/2753358199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7300c5e40 msgr2=0x7fe7300b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:12.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.399+0000 7fe735a00640 1 --2- 172.21.15.67:0/2753358199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7300c5e40 0x7fe7300b0540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02b8d0 comp rx=0 tx=0).stop 2024-03-09T08:54:12.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.399+0000 7fe735a00640 1 -- 172.21.15.67:0/2753358199 shutdown_connections 2024-03-09T08:54:12.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.399+0000 7fe735a00640 1 --2- 172.21.15.67:0/2753358199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7300c5e40 0x7fe7300b0540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:12.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.399+0000 7fe735a00640 1 --2- 172.21.15.67:0/2753358199 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 0x7fe7300b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:12.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.399+0000 7fe735a00640 1 -- 172.21.15.67:0/2753358199 >> 172.21.15.67:0/2753358199 conn(0x7fe7300732d0 msgr2=0x7fe7300ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:12.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.399+0000 7fe735a00640 1 -- 172.21.15.67:0/2753358199 shutdown_connections 2024-03-09T08:54:12.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.399+0000 7fe735a00640 1 -- 172.21.15.67:0/2753358199 wait complete. 2024-03-09T08:54:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.402+0000 7fe735a00640 1 Processor -- start 2024-03-09T08:54:12.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.402+0000 7fe735a00640 1 -- start start 2024-03-09T08:54:12.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.403+0000 7fe735a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7300c5e40 0x7fe7301eb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:12.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.403+0000 7fe735a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7300cadb0 0x7fe730154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:12.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.403+0000 7fe735a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 0x7fe73014bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:12.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.403+0000 7fe735a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7300c59e0 con 0x7fe7300cadb0 2024-03-09T08:54:12.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.403+0000 7fe735a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe730159e90 con 0x7fe7300c5e40 2024-03-09T08:54:12.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.403+0000 7fe735a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe730156720 con 0x7fe730153300 2024-03-09T08:54:12.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.404+0000 7fe72fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7300c5e40 0x7fe7301eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:12.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.404+0000 7fe72fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7300c5e40 0x7fe7301eb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:54854/0 (socket says 172.21.15.67:54854) 2024-03-09T08:54:12.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.404+0000 7fe72fe00640 1 -- 172.21.15.67:0/1159854814 learned_addr learned my addr 172.21.15.67:0/1159854814 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:12.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.404+0000 7fe735000640 1 -- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 msgr2=0x7fe73014bdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:12.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.404+0000 7fe735000640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 0x7fe73014bdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:12.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.404+0000 7fe72f400640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7300cadb0 0x7fe730154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:12.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.404+0000 7fe72fe00640 1 -- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 msgr2=0x7fe73014bdd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:54:12.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.404+0000 7fe72fe00640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 0x7fe73014bdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:12.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.404+0000 7fe72fe00640 1 -- 172.21.15.67:0/1159854814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7300cadb0 msgr2=0x7fe730154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:12.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.404+0000 7fe72fe00640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7300cadb0 0x7fe730154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:12.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.404+0000 7fe72fe00640 1 -- 172.21.15.67:0/1159854814 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73006b1d0 con 0x7fe7300c5e40 2024-03-09T08:54:12.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.405+0000 7fe72fe00640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7300c5e40 0x7fe7301eb310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b550 tx=0x7fe72000ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:12.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.405+0000 7fe727e00640 1 -- 172.21.15.67:0/1159854814 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720013020 con 0x7fe7300c5e40 2024-03-09T08:54:12.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.405+0000 7fe727e00640 1 -- 172.21.15.67:0/1159854814 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe720003de0 con 0x7fe7300c5e40 2024-03-09T08:54:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.405+0000 7fe735a00640 1 -- 172.21.15.67:0/1159854814 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7301eba10 con 0x7fe7300c5e40 2024-03-09T08:54:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.406+0000 7fe727e00640 1 -- 172.21.15.67:0/1159854814 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7200052b0 con 0x7fe7300c5e40 2024-03-09T08:54:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.406+0000 7fe735a00640 1 -- 172.21.15.67:0/1159854814 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7301ebfd0 con 0x7fe7300c5e40 2024-03-09T08:54:12.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.407+0000 7fe735a00640 1 -- 172.21.15.67:0/1159854814 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73004f3c0 con 0x7fe7300c5e40 2024-03-09T08:54:12.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.410+0000 7fe727e00640 1 -- 172.21.15.67:0/1159854814 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe720016020 con 0x7fe7300c5e40 2024-03-09T08:54:12.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.411+0000 7fe727e00640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe71006b480 0x7fe71006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:12.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.412+0000 7fe727e00640 1 -- 172.21.15.67:0/1159854814 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fe72008a910 con 0x7fe7300c5e40 2024-03-09T08:54:12.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.412+0000 7fe72f400640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe71006b480 0x7fe71006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:12.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.413+0000 7fe72f400640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe71006b480 0x7fe71006d940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe71c036000 tx=0x7fe71c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:12.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.419+0000 7fe727e00640 1 -- 172.21.15.67:0/1159854814 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe72001d020 con 0x7fe7300c5e40 2024-03-09T08:54:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.833+0000 7fe735a00640 1 -- 172.21.15.67:0/1159854814 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe7300af120 con 0x7fe7300c5e40 2024-03-09T08:54:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.833+0000 7fe727e00640 1 -- 172.21.15.67:0/1159854814 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fe720059460 con 0x7fe7300c5e40 2024-03-09T08:54:12.830 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:54:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 -- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe71006b480 msgr2=0x7fe71006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe71006b480 0x7fe71006d940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe71c036000 tx=0x7fe71c008030 comp rx=0 tx=0).stop 2024-03-09T08:54:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 -- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7300c5e40 msgr2=0x7fe7301eb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7300c5e40 0x7fe7301eb310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b550 tx=0x7fe72000ba10 comp rx=0 tx=0).stop 2024-03-09T08:54:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 -- 172.21.15.67:0/1159854814 shutdown_connections 2024-03-09T08:54:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe71006b480 0x7fe71006d940 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe730153300 0x7fe73014bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe7300cadb0 0x7fe730154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 --2- 172.21.15.67:0/1159854814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7300c5e40 0x7fe7301eb310 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 -- 172.21.15.67:0/1159854814 >> 172.21.15.67:0/1159854814 conn(0x7fe7300732d0 msgr2=0x7fe73014ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 -- 172.21.15.67:0/1159854814 shutdown_connections 2024-03-09T08:54:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:12.836+0000 7fe735a00640 1 -- 172.21.15.67:0/1159854814 wait complete. 2024-03-09T08:54:15.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:54:16.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.094+0000 7f0693200640 1 Processor -- start 2024-03-09T08:54:16.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.094+0000 7f0693200640 1 -- start start 2024-03-09T08:54:16.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.095+0000 7f0693200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06940c1f70 0x7f0694150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:16.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.095+0000 7f0693200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0694154080 0x7f0694151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:16.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.095+0000 7f0693200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06940c81e0 con 0x7f06940710a0 2024-03-09T08:54:16.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.095+0000 7f0693200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06940c32b0 con 0x7f0694154080 2024-03-09T08:54:16.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.095+0000 7f0693200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06940c2fd0 con 0x7f06940c1f70 2024-03-09T08:54:16.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.095+0000 7f0691e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06940c1f70 msgr2=0x7f0694150c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:16.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.096+0000 7f0691e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06940c1f70 0x7f0694150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:16.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.096+0000 7f0691400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f06940710a0 0x7f0694153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59878/0 (socket says 172.21.15.67:59878) 2024-03-09T08:54:16.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.096+0000 7f0691400640 1 -- 172.21.15.67:0/2848671338 learned_addr learned my addr 172.21.15.67:0/2848671338 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:54:16.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.096+0000 7f0692800640 1 --2- 172.21.15.67:0/2848671338 >> v2:172.21.15.134:3300/0 conn(0x7f0694154080 0x7f0694151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:16.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.096+0000 7f0692800640 1 -- 172.21.15.67:0/2848671338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06940c1f70 msgr2=0x7f0694150c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:54:16.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.096+0000 7f0692800640 1 --2- 172.21.15.67:0/2848671338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06940c1f70 0x7f0694150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:16.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.096+0000 7f0692800640 1 -- 172.21.15.67:0/2848671338 >> v1:172.21.15.67:6789/0 conn(0x7f06940710a0 legacy=0x7f0694153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:16.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.096+0000 7f0692800640 1 -- 172.21.15.67:0/2848671338 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06940c3700 con 0x7f0694154080 2024-03-09T08:54:16.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.098+0000 7f0692800640 1 --2- 172.21.15.67:0/2848671338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0694154080 0x7f0694151190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f068c009d90 tx=0x7f068c02be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f94fe87bf53487d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:16.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.098+0000 7f0690a00640 1 -- 172.21.15.67:0/2848671338 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c039070 con 0x7f0694154080 2024-03-09T08:54:16.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.098+0000 7f0690a00640 1 -- 172.21.15.67:0/2848671338 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c02cbf0 con 0x7f0694154080 2024-03-09T08:54:16.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.098+0000 7f0690a00640 1 -- 172.21.15.67:0/2848671338 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c034440 con 0x7f0694154080 2024-03-09T08:54:16.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.099+0000 7f0693200640 1 -- 172.21.15.67:0/2848671338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0694154080 msgr2=0x7f0694151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:16.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.099+0000 7f0693200640 1 --2- 172.21.15.67:0/2848671338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0694154080 0x7f0694151190 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f068c009d90 tx=0x7f068c02be40 comp rx=0 tx=0).stop 2024-03-09T08:54:16.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.099+0000 7f0693200640 1 -- 172.21.15.67:0/2848671338 shutdown_connections 2024-03-09T08:54:16.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.099+0000 7f0693200640 1 --2- 172.21.15.67:0/2848671338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0694154080 0x7f0694151190 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:16.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.099+0000 7f0693200640 1 --2- 172.21.15.67:0/2848671338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06940c1f70 0x7f0694150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:16.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.099+0000 7f0693200640 1 -- 172.21.15.67:0/2848671338 >> 172.21.15.67:0/2848671338 conn(0x7f06940c7300 msgr2=0x7f06940c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:16.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.099+0000 7f0693200640 1 -- 172.21.15.67:0/2848671338 shutdown_connections 2024-03-09T08:54:16.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.099+0000 7f0693200640 1 -- 172.21.15.67:0/2848671338 wait complete. 2024-03-09T08:54:16.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.102+0000 7f0693200640 1 Processor -- start 2024-03-09T08:54:16.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.102+0000 7f0693200640 1 -- start start 2024-03-09T08:54:16.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0693200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06940710a0 0x7f06940abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:16.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0693200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06940c1f70 0x7f0694143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:16.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0693200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0694154080 0x7f06941406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:16.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0693200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0694070d50 con 0x7f06940710a0 2024-03-09T08:54:16.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0693200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0694070830 con 0x7f06940c1f70 2024-03-09T08:54:16.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0693200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0694070ed0 con 0x7f0694154080 2024-03-09T08:54:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0691e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06940710a0 0x7f06940abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0691e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06940710a0 0x7f06940abc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56556/0 (socket says 172.21.15.67:56556) 2024-03-09T08:54:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0691e00640 1 -- 172.21.15.67:0/3099218434 learned_addr learned my addr 172.21.15.67:0/3099218434 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0691400640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06940c1f70 0x7f0694143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0692800640 1 -- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0694154080 msgr2=0x7f06941406e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.104+0000 7f0692800640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0694154080 0x7f06941406e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:16.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.105+0000 7f0691e00640 1 -- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0694154080 msgr2=0x7f06941406e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:54:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.105+0000 7f0691e00640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0694154080 0x7f06941406e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.105+0000 7f0691e00640 1 -- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06940c1f70 msgr2=0x7f0694143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.105+0000 7f0691e00640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06940c1f70 0x7f0694143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.105+0000 7f0691e00640 1 -- 172.21.15.67:0/3099218434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06940ac7f0 con 0x7f06940710a0 2024-03-09T08:54:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.105+0000 7f0691400640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06940c1f70 0x7f0694143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:54:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.105+0000 7f0691e00640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06940710a0 0x7f06940abc40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f068400b7b0 tx=0x7f068400bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.106+0000 7f068a000640 1 -- 172.21.15.67:0/3099218434 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0684003bb0 con 0x7f06940710a0 2024-03-09T08:54:16.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.106+0000 7f068a000640 1 -- 172.21.15.67:0/3099218434 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0684010070 con 0x7f06940710a0 2024-03-09T08:54:16.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.106+0000 7f068a000640 1 -- 172.21.15.67:0/3099218434 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0684005070 con 0x7f06940710a0 2024-03-09T08:54:16.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.106+0000 7f0693200640 1 -- 172.21.15.67:0/3099218434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06940c6490 con 0x7f06940710a0 2024-03-09T08:54:16.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.106+0000 7f0693200640 1 -- 172.21.15.67:0/3099218434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06940c6a50 con 0x7f06940710a0 2024-03-09T08:54:16.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.108+0000 7f0693200640 1 -- 172.21.15.67:0/3099218434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06940c7e30 con 0x7f06940710a0 2024-03-09T08:54:16.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.109+0000 7f068a000640 1 -- 172.21.15.67:0/3099218434 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0684005210 con 0x7f06940710a0 2024-03-09T08:54:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.109+0000 7f068a000640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f067406b500 0x7f067406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:16.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.110+0000 7f068a000640 1 -- 172.21.15.67:0/3099218434 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f0684089ca0 con 0x7f06940710a0 2024-03-09T08:54:16.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.111+0000 7f0691400640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f067406b500 0x7f067406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:16.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.112+0000 7f0691400640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f067406b500 0x7f067406d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f06940ad0f0 tx=0x7f0680007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:16.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.115+0000 7f068a000640 1 -- 172.21.15.67:0/3099218434 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f06940c7e30 con 0x7f06940710a0 2024-03-09T08:54:16.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.486+0000 7f0693200640 1 -- 172.21.15.67:0/3099218434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0694073310 con 0x7f06940710a0 2024-03-09T08:54:16.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.487+0000 7f068a000640 1 -- 172.21.15.67:0/3099218434 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f0694073310 con 0x7f06940710a0 2024-03-09T08:54:16.484 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:54:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 -- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f067406b500 msgr2=0x7f067406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:16.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f067406b500 0x7f067406d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f06940ad0f0 tx=0x7f0680007e80 comp rx=0 tx=0).stop 2024-03-09T08:54:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 -- 172.21.15.67:0/3099218434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06940710a0 msgr2=0x7f06940abc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06940710a0 0x7f06940abc40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f068400b7b0 tx=0x7f068400bc70 comp rx=0 tx=0).stop 2024-03-09T08:54:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 -- 172.21.15.67:0/3099218434 shutdown_connections 2024-03-09T08:54:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f067406b500 0x7f067406d9c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0694154080 0x7f06941406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06940c1f70 0x7f0694143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 --2- 172.21.15.67:0/3099218434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06940710a0 0x7f06940abc40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 -- 172.21.15.67:0/3099218434 >> 172.21.15.67:0/3099218434 conn(0x7f06940c7300 msgr2=0x7f069414fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 -- 172.21.15.67:0/3099218434 shutdown_connections 2024-03-09T08:54:16.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:16.489+0000 7f0693200640 1 -- 172.21.15.67:0/3099218434 wait complete. 2024-03-09T08:54:19.504 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:54:19.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.687+0000 7fb5a9800640 1 Processor -- start 2024-03-09T08:54:19.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.688+0000 7fb5a9800640 1 -- start start 2024-03-09T08:54:19.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.689+0000 7fb5a9800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:19.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.689+0000 7fb5a9800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:19.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.689+0000 7fb5a9800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 0x7fb5a4153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:19.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.689+0000 7fb5a9800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a40c81e0 con 0x7fb5a40c1f70 2024-03-09T08:54:19.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.689+0000 7fb5a9800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a40c32b0 con 0x7fb5a40710a0 2024-03-09T08:54:19.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.689+0000 7fb5a9800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a40c2fd0 con 0x7fb5a40c27a0 2024-03-09T08:54:19.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb5a3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:19.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb5a8e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 msgr2=0x7fb5a4153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:19.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb5a3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56248/0 (socket says 172.21.15.67:56248) 2024-03-09T08:54:19.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb5a3e00640 1 -- 172.21.15.67:0/1734457170 learned_addr learned my addr 172.21.15.67:0/1734457170 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:19.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb5a8e00640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 0x7fb5a4153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:19.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb5a3e00640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 msgr2=0x7fb5a4150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:54:19.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb5a3e00640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 msgr2=0x7fb5a4150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:54:19.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:19.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:19.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:19.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb59b400640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 msgr2=0x7fb5a4151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:54:19.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb59b400640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 msgr2=0x7fb5a4151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:54:19.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:19.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.690+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:19.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.891+0000 7fb5a8e00640 1 -- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 msgr2=0x7fb5a4153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:19.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.891+0000 7fb5a8e00640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 0x7fb5a4153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:54:19.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.891+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:19.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.891+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:19.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.891+0000 7fb5a3e00640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 msgr2=0x7fb5a4150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:54:19.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.891+0000 7fb5a3e00640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 msgr2=0x7fb5a4150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:54:19.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.891+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:19.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.891+0000 7fb59b400640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 msgr2=0x7fb5a4151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:54:19.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.891+0000 7fb59b400640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 msgr2=0x7fb5a4151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:54:19.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.891+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:19.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.891+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:54:19.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:19.892+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:54:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.291+0000 7fb5a8e00640 1 -- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 msgr2=0x7fb5a4153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.291+0000 7fb5a8e00640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 0x7fb5a4153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:54:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.291+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.292+0000 7fb5a3e00640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 msgr2=0x7fb5a4150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:54:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.292+0000 7fb5a3e00640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 msgr2=0x7fb5a4150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:54:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.292+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:20.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.292+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:54:20.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.292+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:20.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.292+0000 7fb59b400640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 msgr2=0x7fb5a4151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:54:20.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.292+0000 7fb59b400640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 msgr2=0x7fb5a4151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:54:20.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.292+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:20.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:20.292+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:54:21.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.092+0000 7fb5a8e00640 1 -- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 msgr2=0x7fb5a4153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:21.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.092+0000 7fb5a8e00640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 0x7fb5a4153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:54:21.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.093+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:21.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.093+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:21.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.093+0000 7fb5a3e00640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 msgr2=0x7fb5a4150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:54:21.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.093+0000 7fb5a3e00640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 msgr2=0x7fb5a4150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:54:21.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.093+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:21.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.093+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:54:21.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.093+0000 7fb59b400640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 msgr2=0x7fb5a4151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:54:21.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.093+0000 7fb59b400640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 msgr2=0x7fb5a4151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:54:21.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.093+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:21.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:21.093+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:54:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.688+0000 7fb5a2000640 1 -- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 msgr2=0x7fb5a4153970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:54:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.688+0000 7fb5a2000640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 0x7fb5a4153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.688+0000 7fb5a2000640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 msgr2=0x7fb5a4151190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:54:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.688+0000 7fb5a2000640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.134:3300/0 conn(0x7fb5a40710a0 0x7fb5a4151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.688+0000 7fb5a2000640 1 -- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 msgr2=0x7fb5a4150c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:54:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.688+0000 7fb5a2000640 1 --2- 172.21.15.67:0/1734457170 >> v2:172.21.15.67:3300/0 conn(0x7fb5a40c1f70 0x7fb5a4150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.689+0000 7fb5a2000640 1 -- 172.21.15.67:0/1734457170 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a40c3700 con 0x7fb5800073e0 2024-03-09T08:54:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.689+0000 7fb5a2000640 1 -- 172.21.15.67:0/1734457170 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58000c140 con 0x7fb5800036b0 2024-03-09T08:54:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566185740 0 0) 0x7fb5a40c3700 con 0x7fb5800073e0 2024-03-09T08:54:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb588003520 con 0x7fb5800073e0 2024-03-09T08:54:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174811976 0 0) 0x7fb58000c140 con 0x7fb5800036b0 2024-03-09T08:54:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5a40c3700 con 0x7fb5800036b0 2024-03-09T08:54:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1466182281 0 0) 0x7fb588003520 con 0x7fb5800073e0 2024-03-09T08:54:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 >> v1:172.21.15.134:6789/0 conn(0x7fb5800036b0 legacy=0x7fb580005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58000d3e0 con 0x7fb5800073e0 2024-03-09T08:54:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb58c00d580 con 0x7fb5800073e0 2024-03-09T08:54:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 >> v1:172.21.15.67:6789/0 conn(0x7fb5800073e0 legacy=0x7fb580009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:22.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 0x7fb588004c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:22.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5880051d0 0x7fb5880037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:22.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb588003d90 0x7fb588004210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:22.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58800bce0 con 0x7fb588003d90 2024-03-09T08:54:22.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58800be90 con 0x7fb5880051d0 2024-03-09T08:54:22.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58800c090 con 0x7fb5a40c27a0 2024-03-09T08:54:22.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a8e00640 1 -- 172.21.15.67:0/1734457170 reap_dead start 2024-03-09T08:54:22.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.690+0000 7fb5a3e00640 1 -- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 msgr2=0x7fb588004c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:22.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.691+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 0x7fb588004c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:22.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.691+0000 7fb5a8e00640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb588003d90 0x7fb588004210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:22.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.691+0000 7fb59b400640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5880051d0 0x7fb5880037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:22.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.691+0000 7fb5a8e00640 1 -- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 msgr2=0x7fb588004c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:54:22.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.691+0000 7fb5a8e00640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 0x7fb588004c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:22.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.691+0000 7fb5a8e00640 1 -- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5880051d0 msgr2=0x7fb5880037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:22.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.691+0000 7fb5a8e00640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5880051d0 0x7fb5880037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:22.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.691+0000 7fb5a8e00640 1 -- 172.21.15.67:0/1734457170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58000d3e0 con 0x7fb588003d90 2024-03-09T08:54:22.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.693+0000 7fb5a8e00640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb588003d90 0x7fb588004210 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb5a406e660 tx=0x7fb588005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee08941932476086 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.693+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c008050 con 0x7fb588003d90 2024-03-09T08:54:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.693+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59c034070 con 0x7fb588003d90 2024-03-09T08:54:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.693+0000 7fb5a3400640 1 -- 172.21.15.67:0/1734457170 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c009910 con 0x7fb588003d90 2024-03-09T08:54:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.694+0000 7fb5a9800640 1 -- 172.21.15.67:0/1734457170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb588003d90 msgr2=0x7fb588004210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.694+0000 7fb5a9800640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb588003d90 0x7fb588004210 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb5a406e660 tx=0x7fb588005830 comp rx=0 tx=0).stop 2024-03-09T08:54:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.695+0000 7fb5a9800640 1 -- 172.21.15.67:0/1734457170 shutdown_connections 2024-03-09T08:54:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.695+0000 7fb5a9800640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb588003d90 0x7fb588004210 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.695+0000 7fb5a9800640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5880051d0 0x7fb5880037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.695+0000 7fb5a9800640 1 --2- 172.21.15.67:0/1734457170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5a40c27a0 0x7fb588004c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.695+0000 7fb5a9800640 1 -- 172.21.15.67:0/1734457170 >> 172.21.15.67:0/1734457170 conn(0x7fb5a40c7300 msgr2=0x7fb5a40c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.695+0000 7fb5a9800640 1 -- 172.21.15.67:0/1734457170 shutdown_connections 2024-03-09T08:54:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.695+0000 7fb5a9800640 1 -- 172.21.15.67:0/1734457170 wait complete. 2024-03-09T08:54:22.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.698+0000 7fb5a9800640 1 Processor -- start 2024-03-09T08:54:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.698+0000 7fb5a9800640 1 -- start start 2024-03-09T08:54:22.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.699+0000 7fb5a9800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb588003d90 0x7fb5a4149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:22.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.699+0000 7fb5a9800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5880051d0 0x7fb5a40a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:22.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.699+0000 7fb5a9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a40c27a0 0x7fb5a40a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:22.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.699+0000 7fb5a9800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a4070d50 con 0x7fb5a40c27a0 2024-03-09T08:54:22.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.699+0000 7fb5a9800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a4070830 con 0x7fb5880051d0 2024-03-09T08:54:22.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.699+0000 7fb5a9800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a4070ed0 con 0x7fb588003d90 2024-03-09T08:54:22.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.699+0000 7fb5a3e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb588003d90 msgr2=0x7fb5a4149720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:22.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.700+0000 7fb5a3e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb588003d90 0x7fb5a4149720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:22.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.700+0000 7fb59b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5880051d0 0x7fb5a40a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.700+0000 7fb5a8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a40c27a0 0x7fb5a40a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.700+0000 7fb5a8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a40c27a0 0x7fb5a40a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56292/0 (socket says 172.21.15.67:56292) 2024-03-09T08:54:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.700+0000 7fb5a8e00640 1 -- 172.21.15.67:0/303350298 learned_addr learned my addr 172.21.15.67:0/303350298 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.701+0000 7fb5a8e00640 1 -- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb588003d90 msgr2=0x7fb5a4149720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:54:22.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.701+0000 7fb5a8e00640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb588003d90 0x7fb5a4149720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.701+0000 7fb5a8e00640 1 -- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5880051d0 msgr2=0x7fb5a40a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.701+0000 7fb5a8e00640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5880051d0 0x7fb5a40a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.701+0000 7fb5a8e00640 1 -- 172.21.15.67:0/303350298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a41ef110 con 0x7fb5a40c27a0 2024-03-09T08:54:22.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.702+0000 7fb5a8e00640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a40c27a0 0x7fb5a40a9a10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb59c030880 tx=0x7fb59c005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:22.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.702+0000 7fb5a0c00640 1 -- 172.21.15.67:0/303350298 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c039880 con 0x7fb5a40c27a0 2024-03-09T08:54:22.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.702+0000 7fb5a0c00640 1 -- 172.21.15.67:0/303350298 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59c006e50 con 0x7fb5a40c27a0 2024-03-09T08:54:22.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.702+0000 7fb5a0c00640 1 -- 172.21.15.67:0/303350298 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c039c80 con 0x7fb5a40c27a0 2024-03-09T08:54:22.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.702+0000 7fb5a9800640 1 -- 172.21.15.67:0/303350298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a41ef3f0 con 0x7fb5a40c27a0 2024-03-09T08:54:22.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.703+0000 7fb5a9800640 1 -- 172.21.15.67:0/303350298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a41ef9b0 con 0x7fb5a40c27a0 2024-03-09T08:54:22.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.705+0000 7fb5a9800640 1 -- 172.21.15.67:0/303350298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a404f3c0 con 0x7fb5a40c27a0 2024-03-09T08:54:22.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.706+0000 7fb5a0c00640 1 -- 172.21.15.67:0/303350298 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb59c042440 con 0x7fb5a40c27a0 2024-03-09T08:54:22.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.707+0000 7fb5a0c00640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb58006c0d0 0x7fb58006e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:22.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.708+0000 7fb5a0c00640 1 -- 172.21.15.67:0/303350298 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fb59c0ac400 con 0x7fb5a40c27a0 2024-03-09T08:54:22.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.709+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb58006c0d0 0x7fb58006e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:22.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.710+0000 7fb5a3e00640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb58006c0d0 0x7fb58006e590 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb594007130 tx=0x7fb594009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:22.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:22.715+0000 7fb5a0c00640 1 -- 172.21.15.67:0/303350298 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb59c07af50 con 0x7fb5a40c27a0 2024-03-09T08:54:23.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.237+0000 7fb5a9800640 1 -- 172.21.15.67:0/303350298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb5a414fff0 con 0x7fb5a40c27a0 2024-03-09T08:54:23.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.237+0000 7fb5a0c00640 1 -- 172.21.15.67:0/303350298 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7fb59c039e20 con 0x7fb5a40c27a0 2024-03-09T08:54:23.235 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":20,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:54:23.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 -- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb58006c0d0 msgr2=0x7fb58006e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb58006c0d0 0x7fb58006e590 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb594007130 tx=0x7fb594009040 comp rx=0 tx=0).stop 2024-03-09T08:54:23.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 -- 172.21.15.67:0/303350298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a40c27a0 msgr2=0x7fb5a40a9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a40c27a0 0x7fb5a40a9a10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb59c030880 tx=0x7fb59c005d00 comp rx=0 tx=0).stop 2024-03-09T08:54:23.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 -- 172.21.15.67:0/303350298 shutdown_connections 2024-03-09T08:54:23.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb58006c0d0 0x7fb58006e590 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5a40c27a0 0x7fb5a40a9a10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5880051d0 0x7fb5a40a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 --2- 172.21.15.67:0/303350298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb588003d90 0x7fb5a4149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 -- 172.21.15.67:0/303350298 >> 172.21.15.67:0/303350298 conn(0x7fb5a40c7300 msgr2=0x7fb5a414cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:23.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 -- 172.21.15.67:0/303350298 shutdown_connections 2024-03-09T08:54:23.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.240+0000 7fb5a9800640 1 -- 172.21.15.67:0/303350298 wait complete. 2024-03-09T08:54:23.255 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T08:54:23.255 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T08:54:23.256 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:54:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.510+0000 7f75b3e00640 1 Processor -- start 2024-03-09T08:54:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.510+0000 7f75b3e00640 1 -- start start 2024-03-09T08:54:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.512+0000 7f75b3e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b40c9380 0x7f75b40acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.512+0000 7f75b3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f75b40ad440 0x7f75b406c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.512+0000 7f75b3e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b40c9890 con 0x7f75b414fd90 2024-03-09T08:54:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.512+0000 7f75b3e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b406ba60 con 0x7f75b40ad440 2024-03-09T08:54:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.512+0000 7f75b3e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40c5f80 con 0x7f75b40c9380 2024-03-09T08:54:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.512+0000 7f75b2a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f75b414fd90 0x7f75b4156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43378/0 (socket says 172.21.15.67:43378) 2024-03-09T08:54:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.512+0000 7f75b2a00640 1 -- 172.21.15.67:0/2320323787 learned_addr learned my addr 172.21.15.67:0/2320323787 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:54:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.512+0000 7f75b2000640 1 -- 172.21.15.67:0/2320323787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b40c9380 msgr2=0x7f75b40acf00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.512+0000 7f75b2000640 1 --2- 172.21.15.67:0/2320323787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b40c9380 0x7f75b40acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:23.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.513+0000 7f75b3400640 1 --2- 172.21.15.67:0/2320323787 >> v2:172.21.15.67:3300/0 conn(0x7f75b40ad440 0x7f75b406c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.513+0000 7f75b1600640 1 -- 172.21.15.67:0/2320323787 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 940030596 0 0) 0x7f75b40c9890 con 0x7f75b414fd90 2024-03-09T08:54:23.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.513+0000 7f75b1600640 1 -- 172.21.15.67:0/2320323787 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7598003520 con 0x7f75b414fd90 2024-03-09T08:54:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.514+0000 7f75b3400640 1 -- 172.21.15.67:0/2320323787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b40c9380 msgr2=0x7f75b40acf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:54:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.514+0000 7f75b3400640 1 --2- 172.21.15.67:0/2320323787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b40c9380 0x7f75b40acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.514+0000 7f75b3400640 1 -- 172.21.15.67:0/2320323787 >> v1:172.21.15.67:6789/0 conn(0x7f75b414fd90 legacy=0x7f75b4156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.514+0000 7f75b3400640 1 -- 172.21.15.67:0/2320323787 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b40c6240 con 0x7f75b40ad440 2024-03-09T08:54:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.515+0000 7f75b3400640 1 --2- 172.21.15.67:0/2320323787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40ad440 0x7f75b406c1d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f75b406bbe0 tx=0x7f75ac02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bc9375fffb5953 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.516+0000 7f75b1600640 1 -- 172.21.15.67:0/2320323787 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac039070 con 0x7f75b40ad440 2024-03-09T08:54:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.516+0000 7f75b1600640 1 -- 172.21.15.67:0/2320323787 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ac02cb40 con 0x7f75b40ad440 2024-03-09T08:54:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.516+0000 7f75b1600640 1 -- 172.21.15.67:0/2320323787 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac034440 con 0x7f75b40ad440 2024-03-09T08:54:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.516+0000 7f75b3e00640 1 -- 172.21.15.67:0/2320323787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40ad440 msgr2=0x7f75b406c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.516+0000 7f75b3e00640 1 --2- 172.21.15.67:0/2320323787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40ad440 0x7f75b406c1d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f75b406bbe0 tx=0x7f75ac02bd90 comp rx=0 tx=0).stop 2024-03-09T08:54:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.517+0000 7f75b3e00640 1 -- 172.21.15.67:0/2320323787 shutdown_connections 2024-03-09T08:54:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.517+0000 7f75b3e00640 1 --2- 172.21.15.67:0/2320323787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40ad440 0x7f75b406c1d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.517+0000 7f75b3e00640 1 --2- 172.21.15.67:0/2320323787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b40c9380 0x7f75b40acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.517+0000 7f75b3e00640 1 -- 172.21.15.67:0/2320323787 >> 172.21.15.67:0/2320323787 conn(0x7f75b40cb480 msgr2=0x7f75b40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.517+0000 7f75b3e00640 1 -- 172.21.15.67:0/2320323787 shutdown_connections 2024-03-09T08:54:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.517+0000 7f75b3e00640 1 -- 172.21.15.67:0/2320323787 wait complete. 2024-03-09T08:54:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.519+0000 7f75b3e00640 1 Processor -- start 2024-03-09T08:54:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.520+0000 7f75b3e00640 1 -- start start 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.521+0000 7f75b3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40ad440 0x7f75b41ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.521+0000 7f75b3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b40c9380 0x7f75b414da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.521+0000 7f75b3e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b414fd90 0x7f75b414c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.521+0000 7f75b3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b406c710 con 0x7f75b40ad440 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.521+0000 7f75b3e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b406bcb0 con 0x7f75b40c9380 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.521+0000 7f75b3e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b406c890 con 0x7f75b414fd90 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.521+0000 7f75b2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40ad440 0x7f75b41ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.521+0000 7f75b3400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b414fd90 msgr2=0x7f75b414c250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.521+0000 7f75b3400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b414fd90 0x7f75b414c250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.522+0000 7f75b2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40ad440 0x7f75b41ef8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56312/0 (socket says 172.21.15.67:56312) 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.522+0000 7f75b2a00640 1 -- 172.21.15.67:0/727685190 learned_addr learned my addr 172.21.15.67:0/727685190 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.522+0000 7f75b2000640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b40c9380 0x7f75b414da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.522+0000 7f75b2a00640 1 -- 172.21.15.67:0/727685190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b414fd90 msgr2=0x7f75b414c250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:54:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.522+0000 7f75b2a00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75b414fd90 0x7f75b414c250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.522+0000 7f75b2a00640 1 -- 172.21.15.67:0/727685190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b40c9380 msgr2=0x7f75b414da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.522+0000 7f75b2a00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75b40c9380 0x7f75b414da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.522+0000 7f75b2a00640 1 -- 172.21.15.67:0/727685190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b4158e40 con 0x7f75b40ad440 2024-03-09T08:54:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.522+0000 7f75b2a00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40ad440 0x7f75b41ef8f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c390 tx=0x7f75a400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.523+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4017070 con 0x7f75b40ad440 2024-03-09T08:54:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.523+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a4005ce0 con 0x7f75b40ad440 2024-03-09T08:54:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.523+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a40072f0 con 0x7f75b40ad440 2024-03-09T08:54:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.523+0000 7f75b3e00640 1 -- 172.21.15.67:0/727685190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b41ebb10 con 0x7f75b40ad440 2024-03-09T08:54:23.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.523+0000 7f75b3e00640 1 -- 172.21.15.67:0/727685190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b41ec0d0 con 0x7f75b40ad440 2024-03-09T08:54:23.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.525+0000 7f75b3e00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414df70 0x7f75b41ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.525+0000 7f75b3e00640 1 -- 172.21.15.67:0/727685190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b404f3c0 con 0x7f75b414df70 2024-03-09T08:54:23.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.526+0000 7f75b2000640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414df70 0x7f75b41ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.526+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f75a401a070 con 0x7f75b40ad440 2024-03-09T08:54:23.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.526+0000 7f75b2000640 1 -- 172.21.15.67:0/727685190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75b40ad220 con 0x7f75b414df70 2024-03-09T08:54:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.527+0000 7f75b2000640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414df70 0x7f75b41ec2d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f75b40c7060 tx=0x7f75a000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:23.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.527+0000 7f75aaa00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f759406b500 0x7f759406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.528+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f75a4089920 con 0x7f75b40ad440 2024-03-09T08:54:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.528+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a00039b0 con 0x7f75b414df70 2024-03-09T08:54:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.528+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f75a0017070 con 0x7f75b414df70 2024-03-09T08:54:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.528+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414df70 msgr2=0x7f75b41ec2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.528+0000 7f75aaa00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414df70 0x7f75b41ec2d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f75b40c7060 tx=0x7f75a000db90 comp rx=0 tx=0).stop 2024-03-09T08:54:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.528+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414df70 msgr2=0x7f75b41ec2d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.528+0000 7f75aaa00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414df70 0x7f75b41ec2d0 secure :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f75b40c7060 tx=0x7f75a000db90 comp rx=0 tx=0).stop 2024-03-09T08:54:23.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.528+0000 7f75b3400640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f759406b500 0x7f759406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.530+0000 7f75b3400640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f759406b500 0x7f759406d9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f75ac036000 tx=0x7f75ac030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:23.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.535+0000 7f75b3e00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414c790 0x7f75b414cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.535+0000 7f75b3e00640 1 -- 172.21.15.67:0/727685190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b406d2d0 con 0x7f75b414c790 2024-03-09T08:54:23.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.536+0000 7f75b2000640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414c790 0x7f75b414cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.536+0000 7f75b2000640 1 -- 172.21.15.67:0/727685190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f75b40cba00 con 0x7f75b414c790 2024-03-09T08:54:23.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.537+0000 7f75b2000640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414c790 0x7f75b414cb70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f75a0013000 tx=0x7f75a00059d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:23.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.537+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a001ca90 con 0x7f75b414c790 2024-03-09T08:54:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.537+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5177 (secure 0 0 0) 0x7f75a001ca90 con 0x7f75b414c790 2024-03-09T08:54:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.537+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414c790 msgr2=0x7f75b414cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.537+0000 7f75aaa00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414c790 0x7f75b414cb70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f75a0013000 tx=0x7f75a00059d0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.537+0000 7f75aaa00640 1 -- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414c790 msgr2=0x7f75b414cb70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.537+0000 7f75aaa00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b414c790 0x7f75b414cb70 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f75a0013000 tx=0x7f75a00059d0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.535 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:54:23.535 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:54:23.535 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:54:23.535 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 20, 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:23.536 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:23.537 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:54:23.538 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:54:23.539 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:54:23.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:23.541 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:54:23.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.543 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:23.544 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:23.545 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.546 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.547 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:54:23.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.549 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:54:23.549 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:23.549 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:54:23.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.550 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:23.550 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:23.550 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T08:54:23.550 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.550 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:54:23.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.551 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:23.551 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:23.551 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:23.551 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.551 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.551 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:54:23.551 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:54:23.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.538+0000 7f75b3e00640 1 -- 172.21.15.67:0/727685190 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f759406b500 msgr2=0x7f759406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.538+0000 7f75b3e00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f759406b500 0x7f759406d9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f75ac036000 tx=0x7f75ac030000 comp rx=0 tx=0).stop 2024-03-09T08:54:23.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.538+0000 7f75b3e00640 1 -- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40ad440 msgr2=0x7f75b41ef8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.538+0000 7f75b3e00640 1 --2- 172.21.15.67:0/727685190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f75b40ad440 0x7f75b41ef8f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c390 tx=0x7f75a400c850 comp rx=0 tx=0).stop 2024-03-09T08:54:23.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.538+0000 7f75b3400640 1 -- 172.21.15.67:0/727685190 reap_dead start 2024-03-09T08:54:23.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.538+0000 7f75b3400640 1 -- 172.21.15.67:0/727685190 reap_dead start 2024-03-09T08:54:23.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.538+0000 7f75b3e00640 1 -- 172.21.15.67:0/727685190 shutdown_connections 2024-03-09T08:54:23.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.538+0000 7f75b3e00640 1 -- 172.21.15.67:0/727685190 >> 172.21.15.67:0/727685190 conn(0x7f75b40cb480 msgr2=0x7f75b40c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:23.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.539+0000 7f75b3e00640 1 -- 172.21.15.67:0/727685190 shutdown_connections 2024-03-09T08:54:23.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.539+0000 7f75b3e00640 1 -- 172.21.15.67:0/727685190 wait complete. 2024-03-09T08:54:23.558 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:54:23.558 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T08:54:23.558 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:54:23.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.747+0000 7f5517000640 1 Processor -- start 2024-03-09T08:54:23.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.747+0000 7f5517000640 1 -- start start 2024-03-09T08:54:23.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.748+0000 7f5517000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5510157890 0x7f551006d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.748+0000 7f5517000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f551006dad0 0x7f551015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.748+0000 7f5517000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551015a370 con 0x7f551006e630 2024-03-09T08:54:23.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.748+0000 7f5517000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f551015b130 con 0x7f5510157890 2024-03-09T08:54:23.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.748+0000 7f5517000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551015a650 con 0x7f551006dad0 2024-03-09T08:54:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.748+0000 7f550fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5510157890 0x7f551006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.749+0000 7f550f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f551006e630 0x7f551015b5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43392/0 (socket says 172.21.15.67:43392) 2024-03-09T08:54:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.749+0000 7f550f400640 1 -- 172.21.15.67:0/3108971301 learned_addr learned my addr 172.21.15.67:0/3108971301 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:54:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.749+0000 7f5515000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f551006dad0 msgr2=0x7f551015bce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.749+0000 7f5515000640 1 --2- 172.21.15.67:0/3108971301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f551006dad0 0x7f551015bce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.749+0000 7f550ea00640 1 -- 172.21.15.67:0/3108971301 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781045147 0 0) 0x7f551015a370 con 0x7f551006e630 2024-03-09T08:54:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.749+0000 7f550ea00640 1 -- 172.21.15.67:0/3108971301 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f4003520 con 0x7f551006e630 2024-03-09T08:54:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.750+0000 7f550fe00640 1 -- 172.21.15.67:0/3108971301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f551006dad0 msgr2=0x7f551015bce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:54:23.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.750+0000 7f550fe00640 1 --2- 172.21.15.67:0/3108971301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f551006dad0 0x7f551015bce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.750+0000 7f550fe00640 1 -- 172.21.15.67:0/3108971301 >> v1:172.21.15.67:6789/0 conn(0x7f551006e630 legacy=0x7f551015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.750+0000 7f550ea00640 1 -- 172.21.15.67:0/3108971301 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1664540361 0 0) 0x7f54f4003520 con 0x7f551006e630 2024-03-09T08:54:23.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.750+0000 7f550fe00640 1 -- 172.21.15.67:0/3108971301 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551015a930 con 0x7f5510157890 2024-03-09T08:54:23.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.751+0000 7f550fe00640 1 --2- 172.21.15.67:0/3108971301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5510157890 0x7f551006d590 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55000097a0 tx=0x7f550002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e829894a60ce41 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:23.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.752+0000 7f550ea00640 1 -- 172.21.15.67:0/3108971301 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500038070 con 0x7f5510157890 2024-03-09T08:54:23.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.752+0000 7f550ea00640 1 -- 172.21.15.67:0/3108971301 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5500031520 con 0x7f5510157890 2024-03-09T08:54:23.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.752+0000 7f550ea00640 1 -- 172.21.15.67:0/3108971301 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500038070 con 0x7f5510157890 2024-03-09T08:54:23.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.753+0000 7f5517000640 1 -- 172.21.15.67:0/3108971301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5510157890 msgr2=0x7f551006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.753+0000 7f5517000640 1 --2- 172.21.15.67:0/3108971301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5510157890 0x7f551006d590 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55000097a0 tx=0x7f550002b840 comp rx=0 tx=0).stop 2024-03-09T08:54:23.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.753+0000 7f5517000640 1 -- 172.21.15.67:0/3108971301 shutdown_connections 2024-03-09T08:54:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.753+0000 7f5517000640 1 --2- 172.21.15.67:0/3108971301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f551006dad0 0x7f551015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.753+0000 7f5517000640 1 --2- 172.21.15.67:0/3108971301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5510157890 0x7f551006d590 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.753+0000 7f5517000640 1 -- 172.21.15.67:0/3108971301 >> 172.21.15.67:0/3108971301 conn(0x7f55100c7370 msgr2=0x7f55100c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.753+0000 7f5517000640 1 -- 172.21.15.67:0/3108971301 shutdown_connections 2024-03-09T08:54:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.753+0000 7f5517000640 1 -- 172.21.15.67:0/3108971301 wait complete. 2024-03-09T08:54:23.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.755+0000 7f5517000640 1 Processor -- start 2024-03-09T08:54:23.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.755+0000 7f5517000640 1 -- start start 2024-03-09T08:54:23.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.756+0000 7f5517000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f551006dad0 0x7f55100712f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.756+0000 7f5517000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f551006e630 0x7f551015b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.756+0000 7f5517000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5510157890 0x7f5510156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.756+0000 7f5517000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100c1ef0 con 0x7f551006e630 2024-03-09T08:54:23.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.756+0000 7f5517000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100c1d70 con 0x7f551006dad0 2024-03-09T08:54:23.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.756+0000 7f5517000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5510157c70 con 0x7f5510157890 2024-03-09T08:54:23.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.757+0000 7f5515000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5510157890 msgr2=0x7f5510156260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:54:23.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.757+0000 7f5515000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5510157890 0x7f5510156260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:23.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.757+0000 7f550f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f551006e630 0x7f551015b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.757+0000 7f550f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f551006e630 0x7f551015b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56336/0 (socket says 172.21.15.67:56336) 2024-03-09T08:54:23.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.757+0000 7f550f400640 1 -- 172.21.15.67:0/163221608 learned_addr learned my addr 172.21.15.67:0/163221608 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:23.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.757+0000 7f550fe00640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f551006dad0 0x7f55100712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.757+0000 7f550f400640 1 -- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5510157890 msgr2=0x7f5510156260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:54:23.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.757+0000 7f550f400640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5510157890 0x7f5510156260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.757+0000 7f550f400640 1 -- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f551006dad0 msgr2=0x7f55100712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.757+0000 7f550f400640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f551006dad0 0x7f55100712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.757+0000 7f550f400640 1 -- 172.21.15.67:0/163221608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55101f0990 con 0x7f551006e630 2024-03-09T08:54:23.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.758+0000 7f550f400640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f551006e630 0x7f551015b740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00caf0 tx=0x7f54fc007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:23.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.758+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc017070 con 0x7f551006e630 2024-03-09T08:54:23.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.758+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc005ce0 con 0x7f551006e630 2024-03-09T08:54:23.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.758+0000 7f5517000640 1 -- 172.21.15.67:0/163221608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5510155a60 con 0x7f551006e630 2024-03-09T08:54:23.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.758+0000 7f5517000640 1 -- 172.21.15.67:0/163221608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551015bc80 con 0x7f551006e630 2024-03-09T08:54:23.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.758+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00a9a0 con 0x7f551006e630 2024-03-09T08:54:23.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.760+0000 7f5517000640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8002a70 0x7f54e8004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.760+0000 7f5517000640 1 -- 172.21.15.67:0/163221608 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e8007ec0 con 0x7f54e8002a70 2024-03-09T08:54:23.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.760+0000 7f550fe00640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8002a70 0x7f54e8004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.761+0000 7f550fe00640 1 -- 172.21.15.67:0/163221608 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54e80080c0 con 0x7f54e8002a70 2024-03-09T08:54:23.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.761+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f54fc01a070 con 0x7f551006e630 2024-03-09T08:54:23.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.761+0000 7f550fe00640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8002a70 0x7f54e8004f30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5510070320 tx=0x7f55000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:23.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.763+0000 7f5507e00640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f54f006b480 0x7f54f006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.763+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f54fc089d90 con 0x7f551006e630 2024-03-09T08:54:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.763+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500045070 con 0x7f54e8002a70 2024-03-09T08:54:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.763+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f5500003b50 con 0x7f54e8002a70 2024-03-09T08:54:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.763+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8002a70 msgr2=0x7f54e8004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.763+0000 7f5507e00640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8002a70 0x7f54e8004f30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5510070320 tx=0x7f55000046c0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.763+0000 7f5515000640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f54f006b480 0x7f54f006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.763+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8002a70 msgr2=0x7f54e8004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.763+0000 7f5507e00640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8002a70 0x7f54e8004f30 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5510070320 tx=0x7f55000046c0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.765+0000 7f5515000640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f54f006b480 0x7f54f006d940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5508003cb0 tx=0x7f55080091f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.771+0000 7f5517000640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8006f00 0x7f54e8007390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.771+0000 7f5517000640 1 -- 172.21.15.67:0/163221608 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e800a4a0 con 0x7f54e8006f00 2024-03-09T08:54:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.772+0000 7f550fe00640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8006f00 0x7f54e8007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:23.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.772+0000 7f550fe00640 1 -- 172.21.15.67:0/163221608 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f54e80078d0 con 0x7f54e8006f00 2024-03-09T08:54:23.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.773+0000 7f550fe00640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8006f00 0x7f54e8007390 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5500000c00 tx=0x7f5500008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:23.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.773+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500057020 con 0x7f54e8006f00 2024-03-09T08:54:23.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.774+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5032 (secure 0 0 0) 0x7f550003f660 con 0x7f54e8006f00 2024-03-09T08:54:23.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.774+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8006f00 msgr2=0x7f54e8007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.774+0000 7f5507e00640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8006f00 0x7f54e8007390 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5500000c00 tx=0x7f5500008030 comp rx=0 tx=0).stop 2024-03-09T08:54:23.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.774+0000 7f5507e00640 1 -- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8006f00 msgr2=0x7f54e8007390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:23.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.774+0000 7f5507e00640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e8006f00 0x7f54e8007390 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5500000c00 tx=0x7f5500008030 comp rx=0 tx=0).stop 2024-03-09T08:54:23.771 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:54:23.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 20, 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:23.772 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:23.773 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:23.773 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:23.773 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:23.773 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:23.773 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:23.773 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:23.773 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:23.773 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:23.773 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:23.773 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:23.774 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:54:23.775 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:23.776 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:23.777 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:23.777 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.777 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:54:23.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.777 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:54:23.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.777 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:54:23.777 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:54:23.777 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:23.777 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:54:23.778 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.780 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.781 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.782 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:23.783 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:23.784 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:23.785 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:23.785 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:23.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:23.785 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:54:23.785 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:54:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.774+0000 7f5517000640 1 -- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f54f006b480 msgr2=0x7f54f006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.774+0000 7f5517000640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f54f006b480 0x7f54f006d940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5508003cb0 tx=0x7f55080091f0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.774+0000 7f5517000640 1 -- 172.21.15.67:0/163221608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f551006e630 msgr2=0x7f551015b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.774+0000 7f5517000640 1 --2- 172.21.15.67:0/163221608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f551006e630 0x7f551015b740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00caf0 tx=0x7f54fc007ed0 comp rx=0 tx=0).stop 2024-03-09T08:54:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.774+0000 7f5515000640 1 -- 172.21.15.67:0/163221608 reap_dead start 2024-03-09T08:54:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.775+0000 7f5515000640 1 -- 172.21.15.67:0/163221608 reap_dead start 2024-03-09T08:54:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.775+0000 7f5517000640 1 -- 172.21.15.67:0/163221608 shutdown_connections 2024-03-09T08:54:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.775+0000 7f5517000640 1 -- 172.21.15.67:0/163221608 >> 172.21.15.67:0/163221608 conn(0x7f55100c7370 msgr2=0x7f55100c6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:23.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.775+0000 7f5517000640 1 -- 172.21.15.67:0/163221608 shutdown_connections 2024-03-09T08:54:23.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:23.775+0000 7f5517000640 1 -- 172.21.15.67:0/163221608 wait complete. 2024-03-09T08:54:23.788 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:54:23.788 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T08:54:40.324 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:54:40.332 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:54:41.968 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 59% (7250/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 99% (12257/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-09T08:54:42.692 INFO:tasks.workunit.client.1.smithi134.stderr:Note: switching to '8b896a9e145796119f0451201dc4d53ddac97db0'. 2024-03-09T08:54:42.692 INFO:tasks.workunit.client.1.smithi134.stderr: 2024-03-09T08:54:42.692 INFO:tasks.workunit.client.1.smithi134.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr:state without impacting any branches by switching back to a branch. 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr: 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr: 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr: git switch -c 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr: 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr:Or undo this operation with: 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr: 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr: git switch - 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr: 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-09T08:54:42.693 INFO:tasks.workunit.client.1.smithi134.stderr: 2024-03-09T08:54:42.694 INFO:tasks.workunit.client.1.smithi134.stderr:HEAD is now at 8b896a9e145 mds/quiesce-db: keep the db thread alive until shutdown 2024-03-09T08:54:42.699 DEBUG:teuthology.orchestra.run.smithi134:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-09T08:54:42.755 INFO:tasks.workunit.client.1.smithi134.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-09T08:54:42.761 INFO:tasks.workunit.client.1.smithi134.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-09T08:54:42.761 INFO:tasks.workunit.client.1.smithi134.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-09T08:54:43.554 INFO:tasks.workunit.client.1.smithi134.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-09T08:54:43.674 INFO:tasks.workunit.client.1.smithi134.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-09T08:54:43.759 INFO:tasks.workunit.client.1.smithi134.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-09T08:54:43.765 INFO:tasks.workunit.client.1.smithi134.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-09T08:54:43.765 INFO:tasks.workunit.client.1.smithi134.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-09T08:54:43.788 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T08:54:43.788 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T08:54:43.788 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T08:54:43.789 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T08:54:43.793 INFO:tasks.ceph.mon.c:Started 2024-03-09T08:54:43.794 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:54:43.794 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:54:43.838 INFO:tasks.workunit.client.1.smithi134.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-09T08:54:43.846 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T08:54:43.846 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-09T08:54:43.930 INFO:tasks.workunit:Running workunits matching fs/snaps on client.1... 2024-03-09T08:54:43.931 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2024-03-09T08:54:43.931 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snap-rm-diff.sh 2024-03-09T08:54:43.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:43.999+0000 7fc452400640 1 Processor -- start 2024-03-09T08:54:43.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:43.999+0000 7fc452400640 1 -- start start 2024-03-09T08:54:43.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.001+0000 7fc452400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc44c153300 0x7fc44c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:43.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.001+0000 7fc452400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c157000 con 0x7fc44c0cadb0 2024-03-09T08:54:43.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.001+0000 7fc452400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c0ae430 con 0x7fc44c0c5e40 2024-03-09T08:54:43.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.001+0000 7fc452400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c0ae6f0 con 0x7fc44c153300 2024-03-09T08:54:43.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.001+0000 7fc451a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc44c0cadb0 0x7fc44c160490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46398/0 (socket says 172.21.15.67:46398) 2024-03-09T08:54:43.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.001+0000 7fc451a00640 1 -- 172.21.15.67:0/3970617990 learned_addr learned my addr 172.21.15.67:0/3970617990 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:54:43.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.001+0000 7fc451000640 1 --2- 172.21.15.67:0/3970617990 >> v2:172.21.15.134:3300/0 conn(0x7fc44c153300 0x7fc44c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:43.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.002+0000 7fc44be00640 1 -- 172.21.15.67:0/3970617990 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662385501 0 0) 0x7fc44c157000 con 0x7fc44c0cadb0 2024-03-09T08:54:43.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.002+0000 7fc451000640 1 -- 172.21.15.67:0/3970617990 >> v1:172.21.15.134:6789/0 conn(0x7fc44c0c5e40 legacy=0x7fc44c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:43.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.002+0000 7fc451000640 1 -- 172.21.15.67:0/3970617990 >> v1:172.21.15.67:6789/0 conn(0x7fc44c0cadb0 legacy=0x7fc44c160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:43.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.002+0000 7fc451000640 1 -- 172.21.15.67:0/3970617990 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c155b10 con 0x7fc44c153300 2024-03-09T08:54:44.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.003+0000 7fc451000640 1 --2- 172.21.15.67:0/3970617990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c153300 0x7fc44c0b0000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0097a0 tx=0x7fc43c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd4049453a1afa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.003+0000 7fc44be00640 1 -- 172.21.15.67:0/3970617990 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c034070 con 0x7fc44c153300 2024-03-09T08:54:44.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.003+0000 7fc44be00640 1 -- 172.21.15.67:0/3970617990 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c030610 con 0x7fc44c153300 2024-03-09T08:54:44.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.003+0000 7fc44be00640 1 -- 172.21.15.67:0/3970617990 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0309e0 con 0x7fc44c153300 2024-03-09T08:54:44.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.004+0000 7fc452400640 1 -- 172.21.15.67:0/3970617990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c153300 msgr2=0x7fc44c0b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.004+0000 7fc452400640 1 --2- 172.21.15.67:0/3970617990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c153300 0x7fc44c0b0000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0097a0 tx=0x7fc43c02b960 comp rx=0 tx=0).stop 2024-03-09T08:54:44.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.004+0000 7fc452400640 1 -- 172.21.15.67:0/3970617990 shutdown_connections 2024-03-09T08:54:44.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.004+0000 7fc452400640 1 --2- 172.21.15.67:0/3970617990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c153300 0x7fc44c0b0000 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.004+0000 7fc452400640 1 -- 172.21.15.67:0/3970617990 >> 172.21.15.67:0/3970617990 conn(0x7fc44c0732d0 msgr2=0x7fc44c0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:44.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.004+0000 7fc452400640 1 -- 172.21.15.67:0/3970617990 shutdown_connections 2024-03-09T08:54:44.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.004+0000 7fc452400640 1 -- 172.21.15.67:0/3970617990 wait complete. 2024-03-09T08:54:44.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.006+0000 7fc452400640 1 Processor -- start 2024-03-09T08:54:44.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.006+0000 7fc452400640 1 -- start start 2024-03-09T08:54:44.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc452400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc44c0c5e40 0x7fc44c1ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc452400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c0cadb0 0x7fc44c0c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc452400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc44c153300 0x7fc44c14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc452400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0c59e0 con 0x7fc44c0c5e40 2024-03-09T08:54:44.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc452400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c159e90 con 0x7fc44c0cadb0 2024-03-09T08:54:44.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc452400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0b0540 con 0x7fc44c153300 2024-03-09T08:54:44.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc451000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc44c0c5e40 0x7fc44c1ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc451000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc44c0c5e40 0x7fc44c1ebe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50922/0 (socket says 172.21.15.67:50922) 2024-03-09T08:54:44.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc451000640 1 -- 172.21.15.67:0/2391681082 learned_addr learned my addr 172.21.15.67:0/2391681082 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:44.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc443e00640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c0cadb0 0x7fc44c0c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc451a00640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc44c153300 0x7fc44c14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc451a00640 1 -- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc44c153300 msgr2=0x7fc44c14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc451a00640 1 -- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc44c153300 msgr2=0x7fc44c14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc451a00640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc44c153300 0x7fc44c14bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc451000640 1 -- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc44c153300 msgr2=0x7fc44c14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc451a00640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc44c153300 0x7fc44c14bd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.007+0000 7fc451000640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc44c153300 0x7fc44c14bd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.008+0000 7fc451000640 1 -- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c0cadb0 msgr2=0x7fc44c0c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.008+0000 7fc451000640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c0cadb0 0x7fc44c0c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.008+0000 7fc451000640 1 -- 172.21.15.67:0/2391681082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c155060 con 0x7fc44c0c5e40 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.008+0000 7fc451000640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc44c0c5e40 0x7fc44c1ebe30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0098c0 tx=0x7fc43c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.008+0000 7fc449600640 1 -- 172.21.15.67:0/2391681082 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c034040 con 0x7fc44c0c5e40 2024-03-09T08:54:44.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.008+0000 7fc449600640 1 -- 172.21.15.67:0/2391681082 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c03acc0 con 0x7fc44c0c5e40 2024-03-09T08:54:44.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.008+0000 7fc449600640 1 -- 172.21.15.67:0/2391681082 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c044630 con 0x7fc44c0c5e40 2024-03-09T08:54:44.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.008+0000 7fc452400640 1 -- 172.21.15.67:0/2391681082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c155340 con 0x7fc44c0c5e40 2024-03-09T08:54:44.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.008+0000 7fc452400640 1 -- 172.21.15.67:0/2391681082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c0c7c50 con 0x7fc44c0c5e40 2024-03-09T08:54:44.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.010+0000 7fc452400640 1 -- 172.21.15.67:0/2391681082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c04f3c0 con 0x7fc44c0c5e40 2024-03-09T08:54:44.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.011+0000 7fc449600640 1 -- 172.21.15.67:0/2391681082 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc43c037070 con 0x7fc44c0c5e40 2024-03-09T08:54:44.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.012+0000 7fc449600640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc42806b4d0 0x7fc42806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.012+0000 7fc449600640 1 -- 172.21.15.67:0/2391681082 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fc43c0ac410 con 0x7fc44c0c5e40 2024-03-09T08:54:44.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.012+0000 7fc443e00640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc42806b4d0 0x7fc42806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.013+0000 7fc443e00640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc42806b4d0 0x7fc42806d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0b01e0 tx=0x7fc434009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.022+0000 7fc449600640 1 -- 172.21.15.67:0/2391681082 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc43c03f070 con 0x7fc44c0c5e40 2024-03-09T08:54:44.022 INFO:tasks.workunit.client.1.smithi134.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2024-03-09T08:54:44.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.308+0000 7fc452400640 1 -- 172.21.15.67:0/2391681082 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc44c0cab90 con 0x7fc44c0c5e40 2024-03-09T08:54:44.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.309+0000 7fc449600640 1 -- 172.21.15.67:0/2391681082 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7fc43c07aa50 con 0x7fc44c0c5e40 2024-03-09T08:54:44.307 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:54:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.311+0000 7fc452400640 1 -- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc42806b4d0 msgr2=0x7fc42806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.311+0000 7fc452400640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc42806b4d0 0x7fc42806d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0b01e0 tx=0x7fc434009040 comp rx=0 tx=0).stop 2024-03-09T08:54:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.311+0000 7fc452400640 1 -- 172.21.15.67:0/2391681082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc44c0c5e40 msgr2=0x7fc44c1ebe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.311+0000 7fc452400640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc44c0c5e40 0x7fc44c1ebe30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0098c0 tx=0x7fc43c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.312+0000 7fc452400640 1 -- 172.21.15.67:0/2391681082 shutdown_connections 2024-03-09T08:54:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.312+0000 7fc452400640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc42806b4d0 0x7fc42806d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.312+0000 7fc452400640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc44c153300 0x7fc44c14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.312+0000 7fc452400640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc44c0cadb0 0x7fc44c0c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.312+0000 7fc452400640 1 --2- 172.21.15.67:0/2391681082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc44c0c5e40 0x7fc44c1ebe30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.312+0000 7fc452400640 1 -- 172.21.15.67:0/2391681082 >> 172.21.15.67:0/2391681082 conn(0x7fc44c0732d0 msgr2=0x7fc44c0ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.312+0000 7fc452400640 1 -- 172.21.15.67:0/2391681082 shutdown_connections 2024-03-09T08:54:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.312+0000 7fc452400640 1 -- 172.21.15.67:0/2391681082 wait complete. 2024-03-09T08:54:44.335 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:54:44.335 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:54:44.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:54:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.523+0000 7fd596400640 1 Processor -- start 2024-03-09T08:54:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.524+0000 7fd596400640 1 -- start start 2024-03-09T08:54:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.525+0000 7fd596400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd590153c00 0x7fd590153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.525+0000 7fd596400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd5900c7130 0x7fd590154520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.525+0000 7fd596400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5900aaa40 con 0x7fd590066e90 2024-03-09T08:54:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.525+0000 7fd596400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5900ab000 con 0x7fd5900c7130 2024-03-09T08:54:44.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.525+0000 7fd596400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900aad20 con 0x7fd590153c00 2024-03-09T08:54:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.526+0000 7fd595000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd590153c00 0x7fd590153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.526+0000 7fd595a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd5900c7130 0x7fd590154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.526+0000 7fd595000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd590153c00 0x7fd590153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:36268/0 (socket says 172.21.15.67:36268) 2024-03-09T08:54:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.526+0000 7fd595000640 1 -- 172.21.15.67:0/1253822721 learned_addr learned my addr 172.21.15.67:0/1253822721 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.526+0000 7fd595000640 1 -- 172.21.15.67:0/1253822721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd590153c00 msgr2=0x7fd590153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:54:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.526+0000 7fd595000640 1 -- 172.21.15.67:0/1253822721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd590153c00 msgr2=0x7fd590153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:54:44.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.526+0000 7fd595000640 1 --2- 172.21.15.67:0/1253822721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd590153c00 0x7fd590153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.526+0000 7fd595000640 1 --2- 172.21.15.67:0/1253822721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd590153c00 0x7fd590153fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.527+0000 7fd595a00640 1 -- 172.21.15.67:0/1253822721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd590153c00 msgr2=0x7fd590153fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:54:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.527+0000 7fd595a00640 1 --2- 172.21.15.67:0/1253822721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd590153c00 0x7fd590153fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.527+0000 7fd595a00640 1 -- 172.21.15.67:0/1253822721 >> v1:172.21.15.134:6789/0 conn(0x7fd590066e90 legacy=0x7fd5900c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.527+0000 7fd595a00640 1 -- 172.21.15.67:0/1253822721 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5900ad100 con 0x7fd5900c7130 2024-03-09T08:54:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.529+0000 7fd595a00640 1 --2- 172.21.15.67:0/1253822721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5900c7130 0x7fd590154520 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd588009d90 tx=0x7fd58802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0bc33d5e4f395d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.529+0000 7fd58f400640 1 -- 172.21.15.67:0/1253822721 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588039070 con 0x7fd5900c7130 2024-03-09T08:54:44.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.529+0000 7fd58f400640 1 -- 172.21.15.67:0/1253822721 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58802cbc0 con 0x7fd5900c7130 2024-03-09T08:54:44.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.529+0000 7fd58f400640 1 -- 172.21.15.67:0/1253822721 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588034440 con 0x7fd5900c7130 2024-03-09T08:54:44.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.530+0000 7fd596400640 1 -- 172.21.15.67:0/1253822721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5900c7130 msgr2=0x7fd590154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.530+0000 7fd596400640 1 --2- 172.21.15.67:0/1253822721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5900c7130 0x7fd590154520 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd588009d90 tx=0x7fd58802be10 comp rx=0 tx=0).stop 2024-03-09T08:54:44.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.530+0000 7fd596400640 1 -- 172.21.15.67:0/1253822721 shutdown_connections 2024-03-09T08:54:44.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.530+0000 7fd596400640 1 --2- 172.21.15.67:0/1253822721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5900c7130 0x7fd590154520 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.530+0000 7fd596400640 1 --2- 172.21.15.67:0/1253822721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd590153c00 0x7fd590153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.530+0000 7fd596400640 1 -- 172.21.15.67:0/1253822721 >> 172.21.15.67:0/1253822721 conn(0x7fd5900a9740 msgr2=0x7fd5900a9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:44.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.530+0000 7fd596400640 1 -- 172.21.15.67:0/1253822721 shutdown_connections 2024-03-09T08:54:44.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.530+0000 7fd596400640 1 -- 172.21.15.67:0/1253822721 wait complete. 2024-03-09T08:54:44.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.532+0000 7fd596400640 1 Processor -- start 2024-03-09T08:54:44.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.532+0000 7fd596400640 1 -- start start 2024-03-09T08:54:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.533+0000 7fd596400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd590066e90 0x7fd5901de310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.533+0000 7fd596400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5900c7130 0x7fd5900c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd596400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd590153c00 0x7fd5901ee560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd596400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900c5890 con 0x7fd590153c00 2024-03-09T08:54:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd596400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900ad6c0 con 0x7fd590066e90 2024-03-09T08:54:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd596400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900c5a10 con 0x7fd5900c7130 2024-03-09T08:54:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd595a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd590153c00 0x7fd5901ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd595a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd590153c00 0x7fd5901ee560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50934/0 (socket says 172.21.15.67:50934) 2024-03-09T08:54:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd595a00640 1 -- 172.21.15.67:0/1214289496 learned_addr learned my addr 172.21.15.67:0/1214289496 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd595000640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd590066e90 0x7fd5901de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd58fe00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5900c7130 0x7fd5900c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd595a00640 1 -- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5900c7130 msgr2=0x7fd5900c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd595a00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5900c7130 0x7fd5900c3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd595a00640 1 -- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd590066e90 msgr2=0x7fd5901de310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd595a00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd590066e90 0x7fd5901de310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd595a00640 1 -- 172.21.15.67:0/1214289496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5901eb100 con 0x7fd590153c00 2024-03-09T08:54:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.534+0000 7fd58fe00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5900c7130 0x7fd5900c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:54:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.535+0000 7fd595a00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd590153c00 0x7fd5901ee560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd588009d90 tx=0x7fd588004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.535+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588039050 con 0x7fd590153c00 2024-03-09T08:54:44.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.535+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd588045070 con 0x7fd590153c00 2024-03-09T08:54:44.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.535+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58803e830 con 0x7fd590153c00 2024-03-09T08:54:44.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.535+0000 7fd596400640 1 -- 172.21.15.67:0/1214289496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5901eb3a0 con 0x7fd590153c00 2024-03-09T08:54:44.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.535+0000 7fd596400640 1 -- 172.21.15.67:0/1214289496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5901eb960 con 0x7fd590153c00 2024-03-09T08:54:44.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.536+0000 7fd596400640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5901e09a0 0x7fd5901eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.536+0000 7fd596400640 1 -- 172.21.15.67:0/1214289496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5901eefe0 con 0x7fd5901e09a0 2024-03-09T08:54:44.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.537+0000 7fd595000640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5901e09a0 0x7fd5901eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.537+0000 7fd595000640 1 -- 172.21.15.67:0/1214289496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5900c37b0 con 0x7fd5901e09a0 2024-03-09T08:54:44.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.537+0000 7fd595000640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5901e09a0 0x7fd5901eeaa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd584010790 tx=0x7fd584010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.538+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd58803e9d0 con 0x7fd590153c00 2024-03-09T08:54:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.539+0000 7fd58cc00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd57006b4d0 0x7fd57006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.539+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58401a020 con 0x7fd5901e09a0 2024-03-09T08:54:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.539+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fd5880ad620 con 0x7fd590153c00 2024-03-09T08:54:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.539+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd58400e9c0 con 0x7fd5901e09a0 2024-03-09T08:54:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.539+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5901e09a0 msgr2=0x7fd5901eeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.539+0000 7fd58cc00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5901e09a0 0x7fd5901eeaa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd584010790 tx=0x7fd584010b90 comp rx=0 tx=0).stop 2024-03-09T08:54:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.539+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5901e09a0 msgr2=0x7fd5901eeaa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.539+0000 7fd58cc00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5901e09a0 0x7fd5901eeaa0 secure :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd584010790 tx=0x7fd584010b90 comp rx=0 tx=0).stop 2024-03-09T08:54:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.539+0000 7fd58fe00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd57006b4d0 0x7fd57006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.540+0000 7fd58fe00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd57006b4d0 0x7fd57006d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd590150c80 tx=0x7fd57c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.546+0000 7fd596400640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd59014ec40 0x7fd59014f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.546+0000 7fd596400640 1 -- 172.21.15.67:0/1214289496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59004f3c0 con 0x7fd59014ec40 2024-03-09T08:54:44.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.547+0000 7fd595000640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd59014ec40 0x7fd59014f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.547+0000 7fd595000640 1 -- 172.21.15.67:0/1214289496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd5900c0ef0 con 0x7fd59014ec40 2024-03-09T08:54:44.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.547+0000 7fd595000640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd59014ec40 0x7fd59014f050 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd584010790 tx=0x7fd584004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.548+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58401a030 con 0x7fd59014ec40 2024-03-09T08:54:44.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.548+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7fd584021c70 con 0x7fd59014ec40 2024-03-09T08:54:44.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.548+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd59014ec40 msgr2=0x7fd59014f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.548+0000 7fd58cc00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd59014ec40 0x7fd59014f050 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd584010790 tx=0x7fd584004730 comp rx=0 tx=0).stop 2024-03-09T08:54:44.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.548+0000 7fd58cc00640 1 -- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd59014ec40 msgr2=0x7fd59014f050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:44.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.548+0000 7fd58cc00640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd59014ec40 0x7fd59014f050 secure :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd584010790 tx=0x7fd584004730 comp rx=0 tx=0).stop 2024-03-09T08:54:44.547 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:54:44.547 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:54:44.547 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:54:44.547 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:54:44.547 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 22, 2024-03-09T08:54:44.547 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:54:44.547 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:54:44.547 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:54:44.547 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:54:44.548 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.548 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:54:44.548 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:44.548 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:54:44.548 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:54:44.548 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:44.548 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:44.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:44.548 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:44.548 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:44.549 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:44.549 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:44.549 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:44.549 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:44.549 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:44.549 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:44.549 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.549 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:54:44.549 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:54:44.549 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:44.550 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:44.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:44.550 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:44.550 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:44.550 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:44.550 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:44.550 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:44.550 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:44.550 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:44.550 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:44.551 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.551 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:54:44.551 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:54:44.551 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:54:44.551 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:54:44.551 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:54:44.551 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:54:44.551 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:44.551 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:54:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:54:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:54:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:54:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:54:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:54:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:54:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:54:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:54:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:54:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:54:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:54:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:54:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:54:44.562 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:54:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.548+0000 7fd596400640 1 -- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd57006b4d0 msgr2=0x7fd57006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.548+0000 7fd596400640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd57006b4d0 0x7fd57006d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd590150c80 tx=0x7fd57c009040 comp rx=0 tx=0).stop 2024-03-09T08:54:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.548+0000 7fd596400640 1 -- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd590153c00 msgr2=0x7fd5901ee560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.548+0000 7fd596400640 1 --2- 172.21.15.67:0/1214289496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd590153c00 0x7fd5901ee560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd588009d90 tx=0x7fd588004a40 comp rx=0 tx=0).stop 2024-03-09T08:54:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.548+0000 7fd595a00640 1 -- 172.21.15.67:0/1214289496 reap_dead start 2024-03-09T08:54:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.549+0000 7fd595a00640 1 -- 172.21.15.67:0/1214289496 reap_dead start 2024-03-09T08:54:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.549+0000 7fd596400640 1 -- 172.21.15.67:0/1214289496 shutdown_connections 2024-03-09T08:54:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.549+0000 7fd596400640 1 -- 172.21.15.67:0/1214289496 >> 172.21.15.67:0/1214289496 conn(0x7fd5900a9740 msgr2=0x7fd5900a7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.549+0000 7fd596400640 1 -- 172.21.15.67:0/1214289496 shutdown_connections 2024-03-09T08:54:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.549+0000 7fd596400640 1 -- 172.21.15.67:0/1214289496 wait complete. 2024-03-09T08:54:44.565 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:54:44.565 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:54:44.565 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:54:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.757+0000 7f6667e00640 1 Processor -- start 2024-03-09T08:54:44.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.757+0000 7f6667e00640 1 -- start start 2024-03-09T08:54:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.759+0000 7f6667e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66680c5e00 0x7f66680acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.759+0000 7f6667e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f66680c61e0 0x7f66680c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.759+0000 7f6667e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6668156660 0x7f666806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.759+0000 7f6667e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666806d740 con 0x7f6668156660 2024-03-09T08:54:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.759+0000 7f6667e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66680c6660 con 0x7f66680c61e0 2024-03-09T08:54:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.759+0000 7f6667e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6668153680 con 0x7f66680c5e00 2024-03-09T08:54:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.759+0000 7f6667400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6668156660 0x7f666806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.759+0000 7f6667400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6668156660 0x7f666806c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50970/0 (socket says 172.21.15.67:50970) 2024-03-09T08:54:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.759+0000 7f6667400640 1 -- 172.21.15.67:0/2793657581 learned_addr learned my addr 172.21.15.67:0/2793657581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.759+0000 7f6666000640 1 --2- 172.21.15.67:0/2793657581 >> v2:172.21.15.134:3300/0 conn(0x7f66680c61e0 0x7f66680c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.759+0000 7f6666a00640 1 --2- 172.21.15.67:0/2793657581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66680c5e00 0x7f66680acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.760+0000 7f6667400640 1 -- 172.21.15.67:0/2793657581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66680c5e00 msgr2=0x7f66680acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.760+0000 7f6667400640 1 --2- 172.21.15.67:0/2793657581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66680c5e00 0x7f66680acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.760+0000 7f6667400640 1 -- 172.21.15.67:0/2793657581 >> v2:172.21.15.134:3300/0 conn(0x7f66680c61e0 msgr2=0x7f66680c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.760+0000 7f6667400640 1 --2- 172.21.15.67:0/2793657581 >> v2:172.21.15.134:3300/0 conn(0x7f66680c61e0 0x7f66680c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.760+0000 7f6667400640 1 -- 172.21.15.67:0/2793657581 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66680ad7c0 con 0x7f6668156660 2024-03-09T08:54:44.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.760+0000 7f6666a00640 1 --2- 172.21.15.67:0/2793657581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66680c5e00 0x7f66680acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:54:44.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.761+0000 7f6667400640 1 --2- 172.21.15.67:0/2793657581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6668156660 0x7f666806c5e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f666808d1a0 tx=0x7f666002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c008bcfe7b3ff0e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6665600640 1 -- 172.21.15.67:0/2793657581 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666002ca70 con 0x7f6668156660 2024-03-09T08:54:44.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6665600640 1 -- 172.21.15.67:0/2793657581 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666002cc10 con 0x7f6668156660 2024-03-09T08:54:44.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6665600640 1 -- 172.21.15.67:0/2793657581 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66600343f0 con 0x7f6668156660 2024-03-09T08:54:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6667e00640 1 -- 172.21.15.67:0/2793657581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6668156660 msgr2=0x7f666806c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6667e00640 1 --2- 172.21.15.67:0/2793657581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6668156660 0x7f666806c5e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f666808d1a0 tx=0x7f666002bed0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6667e00640 1 -- 172.21.15.67:0/2793657581 shutdown_connections 2024-03-09T08:54:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6667e00640 1 --2- 172.21.15.67:0/2793657581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6668156660 0x7f666806c5e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6667e00640 1 --2- 172.21.15.67:0/2793657581 >> v2:172.21.15.134:3300/0 conn(0x7f66680c61e0 0x7f66680c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6667e00640 1 --2- 172.21.15.67:0/2793657581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66680c5e00 0x7f66680acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6667e00640 1 -- 172.21.15.67:0/2793657581 >> 172.21.15.67:0/2793657581 conn(0x7f66680cb480 msgr2=0x7f66680cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:44.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6667e00640 1 -- 172.21.15.67:0/2793657581 shutdown_connections 2024-03-09T08:54:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.762+0000 7f6667e00640 1 -- 172.21.15.67:0/2793657581 wait complete. 2024-03-09T08:54:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.764+0000 7f6667e00640 1 Processor -- start 2024-03-09T08:54:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.764+0000 7f6667e00640 1 -- start start 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.765+0000 7f6667e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66680c5e00 0x7f66680c0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.765+0000 7f6667e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c61e0 0x7f666814e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.765+0000 7f6667e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6668156660 0x7f666814ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.765+0000 7f6667e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666806bb00 con 0x7f66680c5e00 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.765+0000 7f6667e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66680ad4e0 con 0x7f66680c61e0 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.765+0000 7f6667e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666806bde0 con 0x7f6668156660 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.765+0000 7f6666a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66680c5e00 0x7f66680c0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.765+0000 7f6666a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66680c5e00 0x7f66680c0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50972/0 (socket says 172.21.15.67:50972) 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.765+0000 7f6666a00640 1 -- 172.21.15.67:0/3679551749 learned_addr learned my addr 172.21.15.67:0/3679551749 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f6666000640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c61e0 0x7f666814e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f6667400640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6668156660 0x7f666814ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f6666a00640 1 -- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6668156660 msgr2=0x7f666814ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f6666a00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6668156660 0x7f666814ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f6666a00640 1 -- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c61e0 msgr2=0x7f666814e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f6666a00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c61e0 0x7f666814e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f6666a00640 1 -- 172.21.15.67:0/3679551749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66681578f0 con 0x7f66680c5e00 2024-03-09T08:54:44.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f6666a00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66680c5e00 0x7f66680c0420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f665800c7f0 tx=0x7f665800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658015070 con 0x7f66680c5e00 2024-03-09T08:54:44.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6658007820 con 0x7f66680c5e00 2024-03-09T08:54:44.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658007c10 con 0x7f66680c5e00 2024-03-09T08:54:44.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f6667e00640 1 -- 172.21.15.67:0/3679551749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6668157b90 con 0x7f66680c5e00 2024-03-09T08:54:44.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.766+0000 7f6667e00640 1 -- 172.21.15.67:0/3679551749 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66680c1860 con 0x7f66680c5e00 2024-03-09T08:54:44.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.768+0000 7f6667e00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c1d80 0x7f6668159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.768+0000 7f6667e00640 1 -- 172.21.15.67:0/3679551749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666804f3c0 con 0x7f66680c1d80 2024-03-09T08:54:44.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.768+0000 7f6666000640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c1d80 0x7f6668159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.769+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6658018070 con 0x7f66680c5e00 2024-03-09T08:54:44.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.769+0000 7f6666000640 1 -- 172.21.15.67:0/3679551749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f666806d2f0 con 0x7f66680c1d80 2024-03-09T08:54:44.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.769+0000 7f665ea00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f664806b4d0 0x7f664806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.770+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f665808a6f0 con 0x7f66680c5e00 2024-03-09T08:54:44.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.770+0000 7f6666000640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c1d80 0x7f6668159550 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6668156ef0 tx=0x7f665400e970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.770+0000 7f6667400640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f664806b4d0 0x7f664806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.770+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654017070 con 0x7f66680c1d80 2024-03-09T08:54:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.771+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f6654013610 con 0x7f66680c1d80 2024-03-09T08:54:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.771+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c1d80 msgr2=0x7f6668159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.771+0000 7f665ea00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c1d80 0x7f6668159550 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6668156ef0 tx=0x7f665400e970 comp rx=0 tx=0).stop 2024-03-09T08:54:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.771+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c1d80 msgr2=0x7f6668159550 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.771+0000 7f665ea00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c1d80 0x7f6668159550 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6668156ef0 tx=0x7f665400e970 comp rx=0 tx=0).stop 2024-03-09T08:54:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.772+0000 7f6667400640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f664806b4d0 0x7f664806d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6660035000 tx=0x7f6660030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.779+0000 7f6667e00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c0e20 0x7f6668159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.779+0000 7f6667e00640 1 -- 172.21.15.67:0/3679551749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6668152f80 con 0x7f66680c0e20 2024-03-09T08:54:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.780+0000 7f6666000640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c0e20 0x7f6668159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.780+0000 7f6666000640 1 -- 172.21.15.67:0/3679551749 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f66680c0960 con 0x7f66680c0e20 2024-03-09T08:54:44.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.781+0000 7f6666000640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c0e20 0x7f6668159a90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f66680c5a10 tx=0x7f6654004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.781+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665400f040 con 0x7f66680c0e20 2024-03-09T08:54:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5043 (secure 0 0 0) 0x7f665400efe0 con 0x7f66680c0e20 2024-03-09T08:54:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c0e20 msgr2=0x7f6668159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f665ea00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c0e20 0x7f6668159a90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f66680c5a10 tx=0x7f6654004730 comp rx=0 tx=0).stop 2024-03-09T08:54:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f665ea00640 1 -- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c0e20 msgr2=0x7f6668159a90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f665ea00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66680c0e20 0x7f6668159a90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.780 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:54:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:54:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:54:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:54:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 22, 2024-03-09T08:54:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:44.783 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:44.784 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:54:44.785 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:54:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:54:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:54:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:54:44.793 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:54:44.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f6667e00640 1 -- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f664806b4d0 msgr2=0x7f664806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f6667e00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f664806b4d0 0x7f664806d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6660035000 tx=0x7f6660030000 comp rx=0 tx=0).stop 2024-03-09T08:54:44.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f6667e00640 1 -- 172.21.15.67:0/3679551749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66680c5e00 msgr2=0x7f66680c0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:44.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f6667e00640 1 --2- 172.21.15.67:0/3679551749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66680c5e00 0x7f66680c0420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f665800c7f0 tx=0x7f665800ccb0 comp rx=0 tx=0).stop 2024-03-09T08:54:44.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f6667400640 1 -- 172.21.15.67:0/3679551749 reap_dead start 2024-03-09T08:54:44.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f6667400640 1 -- 172.21.15.67:0/3679551749 reap_dead start 2024-03-09T08:54:44.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f6667e00640 1 -- 172.21.15.67:0/3679551749 shutdown_connections 2024-03-09T08:54:44.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.782+0000 7f6667e00640 1 -- 172.21.15.67:0/3679551749 >> 172.21.15.67:0/3679551749 conn(0x7f66680cb480 msgr2=0x7f66680aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:44.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.783+0000 7f6667e00640 1 -- 172.21.15.67:0/3679551749 shutdown_connections 2024-03-09T08:54:44.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:44.783+0000 7f6667e00640 1 -- 172.21.15.67:0/3679551749 wait complete. 2024-03-09T08:54:44.800 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:54:44.800 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:54:44.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:54:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.003+0000 7f29c5200640 1 Processor -- start 2024-03-09T08:54:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.004+0000 7f29c5200640 1 -- start start 2024-03-09T08:54:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.006+0000 7f29c5200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f29c00acf30 0x7f29c006d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.006+0000 7f29c5200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f29c006cbf0 0x7f29c00c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.006+0000 7f29c5200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c006d270 con 0x7f29c00c6ee0 2024-03-09T08:54:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.006+0000 7f29c5200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00c3e70 con 0x7f29c00acf30 2024-03-09T08:54:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.006+0000 7f29c5200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00c35c0 con 0x7f29c006cbf0 2024-03-09T08:54:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.006+0000 7f29bf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f29c00acf30 0x7f29c006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.007+0000 7f29bf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f29c00acf30 0x7f29c006d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50986/0 (socket says 172.21.15.67:50986) 2024-03-09T08:54:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.007+0000 7f29bf400640 1 -- 172.21.15.67:0/1637475688 learned_addr learned my addr 172.21.15.67:0/1637475688 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.007+0000 7f29bea00640 1 --2- 172.21.15.67:0/1637475688 >> v2:172.21.15.134:3300/0 conn(0x7f29c006cbf0 0x7f29c00c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.007+0000 7f29bf400640 1 -- 172.21.15.67:0/1637475688 >> v2:172.21.15.134:3300/0 conn(0x7f29c006cbf0 msgr2=0x7f29c00c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.007+0000 7f29bf400640 1 --2- 172.21.15.67:0/1637475688 >> v2:172.21.15.134:3300/0 conn(0x7f29c006cbf0 0x7f29c00c69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.007+0000 7f29bf400640 1 -- 172.21.15.67:0/1637475688 >> v1:172.21.15.134:6789/0 conn(0x7f29c00c6ee0 legacy=0x7f29c01560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.007+0000 7f29bf400640 1 -- 172.21.15.67:0/1637475688 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c0067360 con 0x7f29c00acf30 2024-03-09T08:54:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.008+0000 7f29bf400640 1 --2- 172.21.15.67:0/1637475688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c00acf30 0x7f29c006d510 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f29c006f8b0 tx=0x7f29b002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1333a32af96c80c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:45.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.009+0000 7f29be000640 1 -- 172.21.15.67:0/1637475688 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00317c0 con 0x7f29c00acf30 2024-03-09T08:54:45.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.009+0000 7f29be000640 1 -- 172.21.15.67:0/1637475688 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0037070 con 0x7f29c00acf30 2024-03-09T08:54:45.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.009+0000 7f29be000640 1 -- 172.21.15.67:0/1637475688 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0031b80 con 0x7f29c00acf30 2024-03-09T08:54:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.009+0000 7f29c5200640 1 -- 172.21.15.67:0/1637475688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c00acf30 msgr2=0x7f29c006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.009+0000 7f29c5200640 1 --2- 172.21.15.67:0/1637475688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c00acf30 0x7f29c006d510 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f29c006f8b0 tx=0x7f29b002b860 comp rx=0 tx=0).stop 2024-03-09T08:54:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.009+0000 7f29c5200640 1 -- 172.21.15.67:0/1637475688 shutdown_connections 2024-03-09T08:54:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.009+0000 7f29c5200640 1 --2- 172.21.15.67:0/1637475688 >> v2:172.21.15.134:3300/0 conn(0x7f29c006cbf0 0x7f29c00c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.009+0000 7f29c5200640 1 --2- 172.21.15.67:0/1637475688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c00acf30 0x7f29c006d510 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.009+0000 7f29c5200640 1 -- 172.21.15.67:0/1637475688 >> 172.21.15.67:0/1637475688 conn(0x7f29c00a9740 msgr2=0x7f29c00a9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.010+0000 7f29c5200640 1 -- 172.21.15.67:0/1637475688 shutdown_connections 2024-03-09T08:54:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.010+0000 7f29c5200640 1 -- 172.21.15.67:0/1637475688 wait complete. 2024-03-09T08:54:45.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.012+0000 7f29c5200640 1 Processor -- start 2024-03-09T08:54:45.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.012+0000 7f29c5200640 1 -- start start 2024-03-09T08:54:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.014+0000 7f29c5200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c006cbf0 0x7f29c00a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.014+0000 7f29c5200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29c00acf30 0x7f29c014fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.014+0000 7f29c5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c00c6ee0 0x7f29c0150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.014+0000 7f29c5200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00ad310 con 0x7f29c00c6ee0 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.014+0000 7f29c5200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00c38a0 con 0x7f29c00acf30 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.014+0000 7f29c5200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c01567c0 con 0x7f29c006cbf0 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.014+0000 7f29bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c00c6ee0 0x7f29c0150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.014+0000 7f29bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c00c6ee0 0x7f29c0150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50994/0 (socket says 172.21.15.67:50994) 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.014+0000 7f29bfe00640 1 -- 172.21.15.67:0/2757673456 learned_addr learned my addr 172.21.15.67:0/2757673456 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29bea00640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29c00acf30 0x7f29c014fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c006cbf0 0x7f29c00a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29bfe00640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c006cbf0 msgr2=0x7f29c00a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29bfe00640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c006cbf0 0x7f29c00a7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29bfe00640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29c00acf30 msgr2=0x7f29c014fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29bfe00640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29c00acf30 0x7f29c014fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29bfe00640 1 -- 172.21.15.67:0/2757673456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c01515a0 con 0x7f29c00c6ee0 2024-03-09T08:54:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29bfe00640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c00c6ee0 0x7f29c0150190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f29b800b9a0 tx=0x7f29b800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:45.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8019070 con 0x7f29c00c6ee0 2024-03-09T08:54:45.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b800cba0 con 0x7f29c00c6ee0 2024-03-09T08:54:45.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8014440 con 0x7f29c00c6ee0 2024-03-09T08:54:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.015+0000 7f29c5200640 1 -- 172.21.15.67:0/2757673456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c0071ca0 con 0x7f29c00c6ee0 2024-03-09T08:54:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.016+0000 7f29c5200640 1 -- 172.21.15.67:0/2757673456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c00a8260 con 0x7f29c00c6ee0 2024-03-09T08:54:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.017+0000 7f29c5200640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.017+0000 7f29c5200640 1 -- 172.21.15.67:0/2757673456 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0155bf0 con 0x7f29c0150990 2024-03-09T08:54:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.018+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f29b801c070 con 0x7f29c00c6ee0 2024-03-09T08:54:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.018+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:45.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.019+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-03-09T08:54:45.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.019+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:54:45.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.019+0000 7f29b7400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a006b4d0 0x7f29a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:45.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.019+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.019+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:54:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.019+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f29b808a170 con 0x7f29c00c6ee0 2024-03-09T08:54:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.019+0000 7f29bea00640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a006b4d0 0x7f29a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.021+0000 7f29bea00640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a006b4d0 0x7f29a006d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29c0155eb0 tx=0x7f29ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:45.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.220+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:45.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.221+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T08:54:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.221+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:54:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.221+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:45.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.221+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:54:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.624+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.625+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T08:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.625+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.625+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:45.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:45.625+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:54:46.025 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 55% (6777/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 92% (11293/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-09T08:54:46.280 INFO:tasks.workunit.client.0.smithi067.stderr:Note: switching to '8b896a9e145796119f0451201dc4d53ddac97db0'. 2024-03-09T08:54:46.280 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-09T08:54:46.280 INFO:tasks.workunit.client.0.smithi067.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-09T08:54:46.280 INFO:tasks.workunit.client.0.smithi067.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-09T08:54:46.280 INFO:tasks.workunit.client.0.smithi067.stderr:state without impacting any branches by switching back to a branch. 2024-03-09T08:54:46.280 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-09T08:54:46.280 INFO:tasks.workunit.client.0.smithi067.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-09T08:54:46.280 INFO:tasks.workunit.client.0.smithi067.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-09T08:54:46.280 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-09T08:54:46.280 INFO:tasks.workunit.client.0.smithi067.stderr: git switch -c 2024-03-09T08:54:46.280 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-09T08:54:46.281 INFO:tasks.workunit.client.0.smithi067.stderr:Or undo this operation with: 2024-03-09T08:54:46.281 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-09T08:54:46.281 INFO:tasks.workunit.client.0.smithi067.stderr: git switch - 2024-03-09T08:54:46.281 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-09T08:54:46.281 INFO:tasks.workunit.client.0.smithi067.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-09T08:54:46.281 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-09T08:54:46.281 INFO:tasks.workunit.client.0.smithi067.stderr:HEAD is now at 8b896a9e145 mds/quiesce-db: keep the db thread alive until shutdown 2024-03-09T08:54:46.289 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-09T08:54:46.347 INFO:tasks.workunit.client.0.smithi067.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-09T08:54:46.353 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-09T08:54:46.353 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-09T08:54:46.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:46.427+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:46.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:46.427+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T08:54:46.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:46.427+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:54:46.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:46.427+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:46.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:46.428+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:54:47.153 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir foo 2024-03-09T08:54:47.166 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-09T08:54:47.169 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 foo 2024-03-09T08:54:47.280 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir foo/.snap/barsnap 2024-03-09T08:54:47.297 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-09T08:54:47.320 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm foo/linux-2.6.33.tar.bz2 2024-03-09T08:54:47.338 INFO:tasks.workunit.client.1.smithi134.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2024-03-09T08:54:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-09T08:54:47.390 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-09T08:54:47.390 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-09T08:54:47.512 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-09T08:54:47.526 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T08:54:47.526 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-09T08:54:47.571 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2024-03-09T08:54:47.572 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2024-03-09T08:54:47.573 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2024-03-09T08:54:47.683 INFO:tasks.workunit.client.0.smithi067.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2024-03-09T08:54:47.736 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'passed: files are identical' 2024-03-09T08:54:47.736 INFO:tasks.workunit.client.1.smithi134.stdout:passed: files are identical 2024-03-09T08:54:47.737 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir foo/.snap/barsnap 2024-03-09T08:54:47.808 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T08:54:47.809 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T08:54:47.809 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:54:47.809 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T08:54:47.914 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2024-03-09T08:54:47.915 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-1.sh 2024-03-09T08:54:47.981 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 1 2024-03-09T08:54:47.983 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 2 2024-03-09T08:54:47.985 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 3 2024-03-09T08:54:47.985 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' -e file4 ']' 2024-03-09T08:54:47.985 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/snap1 2024-03-09T08:54:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:48.029+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:48.029+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T08:54:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:48.029+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:54:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:48.029+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:54:48.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:48.030+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-09T08:54:48.032 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 4 2024-03-09T08:54:48.035 INFO:tasks.workunit.client.1.smithi134.stderr:++ ls 2024-03-09T08:54:48.048 INFO:tasks.workunit.client.1.smithi134.stderr:+ now='file1 2024-03-09T08:54:48.048 INFO:tasks.workunit.client.1.smithi134.stderr:file2 2024-03-09T08:54:48.048 INFO:tasks.workunit.client.1.smithi134.stderr:file3 2024-03-09T08:54:48.048 INFO:tasks.workunit.client.1.smithi134.stderr:file4' 2024-03-09T08:54:48.049 INFO:tasks.workunit.client.1.smithi134.stderr:++ ls .snap/snap1 2024-03-09T08:54:48.056 INFO:tasks.workunit.client.1.smithi134.stderr:+ then='file1 2024-03-09T08:54:48.057 INFO:tasks.workunit.client.1.smithi134.stderr:file2 2024-03-09T08:54:48.057 INFO:tasks.workunit.client.1.smithi134.stderr:file3' 2024-03-09T08:54:48.057 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/snap1 2024-03-09T08:54:48.074 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 'file1 2024-03-09T08:54:48.074 INFO:tasks.workunit.client.1.smithi134.stderr:file2 2024-03-09T08:54:48.074 INFO:tasks.workunit.client.1.smithi134.stderr:file3 2024-03-09T08:54:48.075 INFO:tasks.workunit.client.1.smithi134.stderr:file4' = 'file1 2024-03-09T08:54:48.075 INFO:tasks.workunit.client.1.smithi134.stderr:file2 2024-03-09T08:54:48.075 INFO:tasks.workunit.client.1.smithi134.stderr:file3' ']' 2024-03-09T08:54:48.075 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 1 2024-03-09T08:54:48.090 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 2 2024-03-09T08:54:48.107 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 3 2024-03-09T08:54:48.125 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/snap1 2024-03-09T08:54:48.162 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 4 2024-03-09T08:54:48.179 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/snap1 2024-03-09T08:54:48.201 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm file1 file2 file3 file4 2024-03-09T08:54:48.218 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T08:54:48.218 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T08:54:48.220 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:54:48.220 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T08:54:48.328 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2024-03-09T08:54:48.329 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-2.sh 2024-03-09T08:54:48.387 INFO:tasks.workunit.client.1.smithi134.stdout:Create dir 100 to 199 ... 2024-03-09T08:54:48.407 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 100 2024-03-09T08:54:48.418 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 101 2024-03-09T08:54:48.441 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 102 2024-03-09T08:54:48.473 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 103 2024-03-09T08:54:48.822 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 104 2024-03-09T08:54:48.983 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 105 2024-03-09T08:54:49.010 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 106 2024-03-09T08:54:49.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.014+0000 7f29b6000640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:54:49.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.014+0000 7f29b6000640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.014+0000 7f29b6000640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2994003540 0x7f29940059d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:49.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.014+0000 7f29b6000640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 msgr2=0x7f29c01ef3d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:49.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.014+0000 7f29b6000640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c0150990 0x7f29c01ef3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.014+0000 7f29b6000640 1 -- 172.21.15.67:0/2757673456 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0155bf0 con 0x7f2994003540 2024-03-09T08:54:49.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.016+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2994003540 0x7f29940059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.017+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29940060b0 con 0x7f2994003540 2024-03-09T08:54:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.018+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2994003540 0x7f29940059d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f29c00a96d0 tx=0x7f29b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:49.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.019+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0032650 con 0x7f2994003540 2024-03-09T08:54:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.020+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f29b0037040 con 0x7f2994003540 2024-03-09T08:54:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.020+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2994003540 msgr2=0x7f29940059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.020+0000 7f29b7400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2994003540 0x7f29940059d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f29c00a96d0 tx=0x7f29b00046c0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.020+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2994003540 msgr2=0x7f29940059d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.020+0000 7f29b7400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2994003540 0x7f29940059d0 secure :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f29c00a96d0 tx=0x7f29b00046c0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.037+0000 7f29c5200640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c01ef910 0x7f29c01efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.037+0000 7f29c5200640 1 -- 172.21.15.67:0/2757673456 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c004f3c0 con 0x7f29c01ef910 2024-03-09T08:54:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.038+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c01ef910 0x7f29c01efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.039+0000 7f29bf400640 1 -- 172.21.15.67:0/2757673456 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f29c0066fb0 con 0x7f29c01ef910 2024-03-09T08:54:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.039+0000 7f29bf400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c01ef910 0x7f29c01efcf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2994006080 tx=0x7f29b003d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.040+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0003610 con 0x7f29c01ef910 2024-03-09T08:54:49.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.040+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4533 (secure 0 0 0) 0x7f29b0003610 con 0x7f29c01ef910 2024-03-09T08:54:49.039 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 107 2024-03-09T08:54:49.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.040+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c01ef910 msgr2=0x7f29c01efcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:49.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.040+0000 7f29b7400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c01ef910 0x7f29c01efcf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2994006080 tx=0x7f29b003d040 comp rx=0 tx=0).stop 2024-03-09T08:54:49.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.041+0000 7f29b7400640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c01ef910 msgr2=0x7f29c01efcf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:54:49.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.041+0000 7f29b7400640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c01ef910 0x7f29c01efcf0 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2994006080 tx=0x7f29b003d040 comp rx=0 tx=0).stop 2024-03-09T08:54:49.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.041+0000 7f29bfe00640 1 -- 172.21.15.67:0/2757673456 reap_dead start 2024-03-09T08:54:49.040 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:54:49.040 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:54:49.040 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:54:49.040 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:54:49.040 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 22, 2024-03-09T08:54:49.040 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:54:49.040 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:54:49.040 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:54:49.040 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:54:49.040 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 4, 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:49.041 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:54:49.043 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:54:49.044 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:49.045 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.046 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:49.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.048 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:54:49.049 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:54:49.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:54:49.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:54:49.052 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:54:49.052 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:54:49.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.042+0000 7f29c5200640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a006b4d0 msgr2=0x7f29a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:49.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.042+0000 7f29c5200640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a006b4d0 0x7f29a006d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29c0155eb0 tx=0x7f29ac008040 comp rx=0 tx=0).stop 2024-03-09T08:54:49.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.042+0000 7f29c5200640 1 -- 172.21.15.67:0/2757673456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c00c6ee0 msgr2=0x7f29c0150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:49.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.042+0000 7f29c5200640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c00c6ee0 0x7f29c0150190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f29b800b9a0 tx=0x7f29b800be60 comp rx=0 tx=0).stop 2024-03-09T08:54:49.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.042+0000 7f29c5200640 1 -- 172.21.15.67:0/2757673456 shutdown_connections 2024-03-09T08:54:49.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.042+0000 7f29c5200640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a006b4d0 0x7f29a006d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.042+0000 7f29c5200640 1 --2- 172.21.15.67:0/2757673456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c00c6ee0 0x7f29c0150190 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.042+0000 7f29c5200640 1 -- 172.21.15.67:0/2757673456 >> 172.21.15.67:0/2757673456 conn(0x7f29c00a9740 msgr2=0x7f29c00c20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:49.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.043+0000 7f29c5200640 1 -- 172.21.15.67:0/2757673456 shutdown_connections 2024-03-09T08:54:49.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.043+0000 7f29c5200640 1 -- 172.21.15.67:0/2757673456 wait complete. 2024-03-09T08:54:49.059 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 108 2024-03-09T08:54:49.067 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:54:49.067 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T08:54:49.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T08:54:49.206 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 109 2024-03-09T08:54:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.283+0000 7f347cc00640 1 Processor -- start 2024-03-09T08:54:49.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.283+0000 7f347cc00640 1 -- start start 2024-03-09T08:54:49.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.285+0000 7f347cc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3478153290 0x7f3478159b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:49.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.285+0000 7f347cc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34780cb960 0x7f3478156190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:49.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.285+0000 7f347cc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347815a0a0 con 0x7f347806c940 2024-03-09T08:54:49.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.285+0000 7f347cc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3478155d00 con 0x7f34780cb960 2024-03-09T08:54:49.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.285+0000 7f347cc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780c7340 con 0x7f3478153290 2024-03-09T08:54:49.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.285+0000 7f3477e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34780cb960 0x7f3478156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:49.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.285+0000 7f3477e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34780cb960 0x7f3478156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58312/0 (socket says 172.21.15.67:58312) 2024-03-09T08:54:49.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.285+0000 7f3477e00640 1 -- 172.21.15.67:0/2108137629 learned_addr learned my addr 172.21.15.67:0/2108137629 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:49.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.285+0000 7f3477400640 1 --2- 172.21.15.67:0/2108137629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3478153290 0x7f3478159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.286+0000 7f3477e00640 1 -- 172.21.15.67:0/2108137629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3478153290 msgr2=0x7f3478159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.286+0000 7f3477e00640 1 --2- 172.21.15.67:0/2108137629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3478153290 0x7f3478159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.286+0000 7f3477e00640 1 -- 172.21.15.67:0/2108137629 >> v1:172.21.15.67:6789/0 conn(0x7f347806c940 legacy=0x7f34780cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.286+0000 7f3477e00640 1 -- 172.21.15.67:0/2108137629 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780c7620 con 0x7f34780cb960 2024-03-09T08:54:49.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.287+0000 7f3477e00640 1 --2- 172.21.15.67:0/2108137629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34780cb960 0x7f3478156190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3478155e80 tx=0x7f347002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f2d8f31d397498 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:49.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.287+0000 7f3476a00640 1 -- 172.21.15.67:0/2108137629 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347002cae0 con 0x7f34780cb960 2024-03-09T08:54:49.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.287+0000 7f3476a00640 1 -- 172.21.15.67:0/2108137629 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347002cc80 con 0x7f34780cb960 2024-03-09T08:54:49.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.287+0000 7f3476a00640 1 -- 172.21.15.67:0/2108137629 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470034610 con 0x7f34780cb960 2024-03-09T08:54:49.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.288+0000 7f347cc00640 1 -- 172.21.15.67:0/2108137629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34780cb960 msgr2=0x7f3478156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:49.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.288+0000 7f347cc00640 1 --2- 172.21.15.67:0/2108137629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34780cb960 0x7f3478156190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3478155e80 tx=0x7f347002bf40 comp rx=0 tx=0).stop 2024-03-09T08:54:49.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.288+0000 7f347cc00640 1 -- 172.21.15.67:0/2108137629 shutdown_connections 2024-03-09T08:54:49.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.288+0000 7f347cc00640 1 --2- 172.21.15.67:0/2108137629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34780cb960 0x7f3478156190 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.288+0000 7f347cc00640 1 --2- 172.21.15.67:0/2108137629 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3478153290 0x7f3478159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.288+0000 7f347cc00640 1 -- 172.21.15.67:0/2108137629 >> 172.21.15.67:0/2108137629 conn(0x7f34780ad170 msgr2=0x7f34780ad580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:49.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.288+0000 7f347cc00640 1 -- 172.21.15.67:0/2108137629 shutdown_connections 2024-03-09T08:54:49.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.288+0000 7f347cc00640 1 -- 172.21.15.67:0/2108137629 wait complete. 2024-03-09T08:54:49.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.290+0000 7f347cc00640 1 Processor -- start 2024-03-09T08:54:49.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.290+0000 7f347cc00640 1 -- start start 2024-03-09T08:54:49.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.291+0000 7f347cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f347806c940 0x7f34781e08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:49.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.291+0000 7f347cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34780cb960 0x7f34781eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:49.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.291+0000 7f347cc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3478153290 0x7f34781ebbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:49.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.291+0000 7f347cc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34780c7010 con 0x7f34780cb960 2024-03-09T08:54:49.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.291+0000 7f347cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34780c7be0 con 0x7f347806c940 2024-03-09T08:54:49.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.291+0000 7f347cc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780c93e0 con 0x7f3478153290 2024-03-09T08:54:49.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.292+0000 7f346ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34780cb960 0x7f34781eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:49.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.292+0000 7f346ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34780cb960 0x7f34781eb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58324/0 (socket says 172.21.15.67:58324) 2024-03-09T08:54:49.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.292+0000 7f346ea00640 1 -- 172.21.15.67:0/1493493190 learned_addr learned my addr 172.21.15.67:0/1493493190 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:49.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.292+0000 7f3477400640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f347806c940 0x7f34781e08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:49.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.292+0000 7f3477e00640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3478153290 0x7f34781ebbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:49.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.292+0000 7f346ea00640 1 -- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3478153290 msgr2=0x7f34781ebbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:49.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.292+0000 7f346ea00640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3478153290 0x7f34781ebbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.292+0000 7f346ea00640 1 -- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f347806c940 msgr2=0x7f34781e08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:49.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.292+0000 7f346ea00640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f347806c940 0x7f34781e08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.292+0000 7f346ea00640 1 -- 172.21.15.67:0/1493493190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780c0630 con 0x7f34780cb960 2024-03-09T08:54:49.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.293+0000 7f346ea00640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34780cb960 0x7f34781eb6a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f346000c3c0 tx=0x7f346000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:49.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.293+0000 7f346fe00640 1 -- 172.21.15.67:0/1493493190 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34600052d0 con 0x7f34780cb960 2024-03-09T08:54:49.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.293+0000 7f346fe00640 1 -- 172.21.15.67:0/1493493190 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3460005490 con 0x7f34780cb960 2024-03-09T08:54:49.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.293+0000 7f347cc00640 1 -- 172.21.15.67:0/1493493190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34781eee60 con 0x7f34780cb960 2024-03-09T08:54:49.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.293+0000 7f346fe00640 1 -- 172.21.15.67:0/1493493190 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460007260 con 0x7f34780cb960 2024-03-09T08:54:49.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.293+0000 7f347cc00640 1 -- 172.21.15.67:0/1493493190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34781e0de0 con 0x7f34780cb960 2024-03-09T08:54:49.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.295+0000 7f347cc00640 1 -- 172.21.15.67:0/1493493190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347804f3c0 con 0x7f34780cb960 2024-03-09T08:54:49.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.295+0000 7f346fe00640 1 -- 172.21.15.67:0/1493493190 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3460007400 con 0x7f34780cb960 2024-03-09T08:54:49.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.297+0000 7f346fe00640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f345406b4d0 0x7f345406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:49.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.297+0000 7f346fe00640 1 -- 172.21.15.67:0/1493493190 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f3460089940 con 0x7f34780cb960 2024-03-09T08:54:49.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.297+0000 7f3477400640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f345406b4d0 0x7f345406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:49.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.298+0000 7f3477400640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f345406b4d0 0x7f345406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3468009ad0 tx=0x7f3468008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:49.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.302+0000 7f346fe00640 1 -- 172.21.15.67:0/1493493190 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3460057e20 con 0x7f34780cb960 2024-03-09T08:54:49.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.598+0000 7f347cc00640 1 -- 172.21.15.67:0/1493493190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f34780acf50 con 0x7f34780cb960 2024-03-09T08:54:49.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.600+0000 7f346fe00640 1 -- 172.21.15.67:0/1493493190 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f346005bec0 con 0x7f34780cb960 2024-03-09T08:54:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.605+0000 7f347cc00640 1 -- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f345406b4d0 msgr2=0x7f345406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.605+0000 7f347cc00640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f345406b4d0 0x7f345406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3468009ad0 tx=0x7f3468008040 comp rx=0 tx=0).stop 2024-03-09T08:54:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.605+0000 7f347cc00640 1 -- 172.21.15.67:0/1493493190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34780cb960 msgr2=0x7f34781eb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.605+0000 7f347cc00640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34780cb960 0x7f34781eb6a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f346000c3c0 tx=0x7f346000c880 comp rx=0 tx=0).stop 2024-03-09T08:54:49.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.605+0000 7f347cc00640 1 -- 172.21.15.67:0/1493493190 shutdown_connections 2024-03-09T08:54:49.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.605+0000 7f347cc00640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f345406b4d0 0x7f345406d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.605+0000 7f347cc00640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3478153290 0x7f34781ebbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.605+0000 7f347cc00640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34780cb960 0x7f34781eb6a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.605+0000 7f347cc00640 1 --2- 172.21.15.67:0/1493493190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f347806c940 0x7f34781e08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:49.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.605+0000 7f347cc00640 1 -- 172.21.15.67:0/1493493190 >> 172.21.15.67:0/1493493190 conn(0x7f34780ad170 msgr2=0x7f34780b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:49.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.605+0000 7f347cc00640 1 -- 172.21.15.67:0/1493493190 shutdown_connections 2024-03-09T08:54:49.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:49.606+0000 7f347cc00640 1 -- 172.21.15.67:0/1493493190 wait complete. 2024-03-09T08:54:49.629 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T08:54:49.781 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 110 2024-03-09T08:54:50.056 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 111 2024-03-09T08:54:50.804 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 112 2024-03-09T08:54:50.828 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 113 2024-03-09T08:54:50.854 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 114 2024-03-09T08:54:50.885 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 115 2024-03-09T08:54:51.419 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 116 2024-03-09T08:54:51.444 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 117 2024-03-09T08:54:51.470 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 118 2024-03-09T08:54:51.495 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 119 2024-03-09T08:54:51.515 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 120 2024-03-09T08:54:51.830 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 121 2024-03-09T08:54:51.915 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 122 2024-03-09T08:54:51.941 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 123 2024-03-09T08:54:51.979 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 124 2024-03-09T08:54:52.005 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 125 2024-03-09T08:54:52.026 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 126 2024-03-09T08:54:52.051 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 127 2024-03-09T08:54:52.069 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 128 2024-03-09T08:54:52.093 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 129 2024-03-09T08:54:52.114 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 130 2024-03-09T08:54:52.133 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 131 2024-03-09T08:54:52.153 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 132 2024-03-09T08:54:52.440 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 133 2024-03-09T08:54:52.464 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 134 2024-03-09T08:54:52.490 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 135 2024-03-09T08:54:52.517 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 136 2024-03-09T08:54:52.791 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 137 2024-03-09T08:54:52.942 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 138 2024-03-09T08:54:52.970 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 139 2024-03-09T08:54:52.994 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 140 2024-03-09T08:54:53.017 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 141 2024-03-09T08:54:53.054 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 142 2024-03-09T08:54:53.102 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 143 2024-03-09T08:54:53.145 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 144 2024-03-09T08:54:53.189 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 145 2024-03-09T08:54:53.814 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 146 2024-03-09T08:54:54.051 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 147 2024-03-09T08:54:54.094 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 148 2024-03-09T08:54:54.124 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 149 2024-03-09T08:54:54.147 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 150 2024-03-09T08:54:54.171 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 151 2024-03-09T08:54:54.194 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 152 2024-03-09T08:54:54.215 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 153 2024-03-09T08:54:54.237 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 154 2024-03-09T08:54:54.839 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 155 2024-03-09T08:54:55.024 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 156 2024-03-09T08:54:55.312 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 157 2024-03-09T08:54:55.344 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 158 2024-03-09T08:54:55.374 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 159 2024-03-09T08:54:55.405 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 160 2024-03-09T08:54:55.624 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir foo 2024-03-09T08:54:55.640 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 foo 2024-03-09T08:54:55.694 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 161 2024-03-09T08:54:55.995 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir foo/.snap/barsnap 2024-03-09T08:54:56.039 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm foo/linux-2.6.33.tar.bz2 2024-03-09T08:54:56.062 INFO:tasks.workunit.client.0.smithi067.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2024-03-09T08:54:56.249 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 162 2024-03-09T08:54:56.273 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 163 2024-03-09T08:54:56.290 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 164 2024-03-09T08:54:56.304 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 165 2024-03-09T08:54:56.318 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 166 2024-03-09T08:54:56.338 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 167 2024-03-09T08:54:56.357 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 168 2024-03-09T08:54:56.368 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 169 2024-03-09T08:54:56.382 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 170 2024-03-09T08:54:56.397 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 171 2024-03-09T08:54:56.412 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 172 2024-03-09T08:54:56.622 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'passed: files are identical' 2024-03-09T08:54:56.623 INFO:tasks.workunit.client.0.smithi067.stdout:passed: files are identical 2024-03-09T08:54:56.623 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir foo/.snap/barsnap 2024-03-09T08:54:56.662 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T08:54:56.662 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T08:54:56.663 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:54:56.663 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T08:54:56.759 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 173 2024-03-09T08:54:56.775 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 174 2024-03-09T08:54:56.780 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2024-03-09T08:54:56.780 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-1.sh 2024-03-09T08:54:56.793 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 175 2024-03-09T08:54:56.821 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 176 2024-03-09T08:54:56.837 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 177 2024-03-09T08:54:56.846 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 1 2024-03-09T08:54:56.848 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 2 2024-03-09T08:54:56.848 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 3 2024-03-09T08:54:56.849 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -e file4 ']' 2024-03-09T08:54:56.849 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/snap1 2024-03-09T08:54:56.859 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 178 2024-03-09T08:54:56.876 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 179 2024-03-09T08:54:56.895 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 4 2024-03-09T08:54:56.896 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 180 2024-03-09T08:54:56.898 INFO:tasks.workunit.client.0.smithi067.stderr:++ ls 2024-03-09T08:54:56.908 INFO:tasks.workunit.client.0.smithi067.stderr:+ now='file1 2024-03-09T08:54:56.908 INFO:tasks.workunit.client.0.smithi067.stderr:file2 2024-03-09T08:54:56.908 INFO:tasks.workunit.client.0.smithi067.stderr:file3 2024-03-09T08:54:56.908 INFO:tasks.workunit.client.0.smithi067.stderr:file4' 2024-03-09T08:54:56.908 INFO:tasks.workunit.client.0.smithi067.stderr:++ ls .snap/snap1 2024-03-09T08:54:56.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ then='file1 2024-03-09T08:54:56.915 INFO:tasks.workunit.client.0.smithi067.stderr:file2 2024-03-09T08:54:56.915 INFO:tasks.workunit.client.0.smithi067.stderr:file3' 2024-03-09T08:54:56.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/snap1 2024-03-09T08:54:56.916 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 181 2024-03-09T08:54:56.924 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 'file1 2024-03-09T08:54:56.924 INFO:tasks.workunit.client.0.smithi067.stderr:file2 2024-03-09T08:54:56.925 INFO:tasks.workunit.client.0.smithi067.stderr:file3 2024-03-09T08:54:56.925 INFO:tasks.workunit.client.0.smithi067.stderr:file4' = 'file1 2024-03-09T08:54:56.925 INFO:tasks.workunit.client.0.smithi067.stderr:file2 2024-03-09T08:54:56.925 INFO:tasks.workunit.client.0.smithi067.stderr:file3' ']' 2024-03-09T08:54:56.925 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 1 2024-03-09T08:54:56.933 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 182 2024-03-09T08:54:56.934 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 2 2024-03-09T08:54:56.948 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 3 2024-03-09T08:54:56.951 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 183 2024-03-09T08:54:56.961 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/snap1 2024-03-09T08:54:56.972 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 184 2024-03-09T08:54:56.991 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 185 2024-03-09T08:54:56.993 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 4 2024-03-09T08:54:57.001 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/snap1 2024-03-09T08:54:57.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm file1 file2 file3 file4 2024-03-09T08:54:57.023 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T08:54:57.023 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T08:54:57.024 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:54:57.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T08:54:57.103 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2024-03-09T08:54:57.103 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-2.sh 2024-03-09T08:54:57.160 INFO:tasks.workunit.client.0.smithi067.stdout:Create dir 100 to 199 ... 2024-03-09T08:54:57.172 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 100 2024-03-09T08:54:57.184 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 101 2024-03-09T08:54:57.207 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 102 2024-03-09T08:54:57.232 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 103 2024-03-09T08:54:57.258 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 104 2024-03-09T08:54:57.280 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 105 2024-03-09T08:54:57.298 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 106 2024-03-09T08:54:57.316 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 107 2024-03-09T08:54:57.334 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 108 2024-03-09T08:54:57.351 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 109 2024-03-09T08:54:57.367 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 110 2024-03-09T08:54:57.385 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 111 2024-03-09T08:54:57.401 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 112 2024-03-09T08:54:57.416 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 113 2024-03-09T08:54:57.431 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 114 2024-03-09T08:54:57.446 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 115 2024-03-09T08:54:57.788 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 186 2024-03-09T08:54:57.989 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 116 2024-03-09T08:54:58.019 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 117 2024-03-09T08:54:58.048 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 118 2024-03-09T08:54:58.088 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 119 2024-03-09T08:54:58.095 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 187 2024-03-09T08:54:58.124 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 120 2024-03-09T08:54:58.155 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 121 2024-03-09T08:54:58.195 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 122 2024-03-09T08:54:58.233 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 123 2024-03-09T08:54:58.259 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 124 2024-03-09T08:54:58.289 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 125 2024-03-09T08:54:58.320 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 126 2024-03-09T08:54:58.350 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 127 2024-03-09T08:54:58.374 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 188 2024-03-09T08:54:58.390 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 189 2024-03-09T08:54:58.407 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 190 2024-03-09T08:54:58.425 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 191 2024-03-09T08:54:58.446 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 192 2024-03-09T08:54:58.465 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 193 2024-03-09T08:54:58.492 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 194 2024-03-09T08:54:58.508 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 195 2024-03-09T08:54:58.525 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 196 2024-03-09T08:54:58.541 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 197 2024-03-09T08:54:58.557 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 198 2024-03-09T08:54:58.572 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 199 2024-03-09T08:54:58.587 INFO:tasks.workunit.client.1.smithi134.stdout:Take first snapshot .snap/test1 2024-03-09T08:54:58.637 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 128 2024-03-09T08:54:58.659 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 129 2024-03-09T08:54:58.681 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 130 2024-03-09T08:54:58.847 INFO:tasks.workunit.client.1.smithi134.stdout:Create dir 200 to 299 ... 2024-03-09T08:54:58.850 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 200 2024-03-09T08:54:58.896 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 131 2024-03-09T08:54:59.189 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 132 2024-03-09T08:54:59.237 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 201 2024-03-09T08:54:59.449 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 202 2024-03-09T08:54:59.472 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 203 2024-03-09T08:54:59.504 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 204 2024-03-09T08:54:59.535 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 205 2024-03-09T08:54:59.560 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 206 2024-03-09T08:54:59.583 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 207 2024-03-09T08:54:59.629 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:54:59.630 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:54:59.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.846+0000 7fb36f200640 1 Processor -- start 2024-03-09T08:54:59.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.847+0000 7fb36f200640 1 -- start start 2024-03-09T08:54:59.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.848+0000 7fb36f200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb3700c5e40 0x7fb370156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:59.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.848+0000 7fb36f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700cadb0 0x7fb370156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:59.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.848+0000 7fb36f200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb370156a90 con 0x7fb370153300 2024-03-09T08:54:59.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.848+0000 7fb36f200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb370157000 con 0x7fb3700c5e40 2024-03-09T08:54:59.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.848+0000 7fb36f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700ae430 con 0x7fb3700cadb0 2024-03-09T08:54:59.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.849+0000 7fb36e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700cadb0 0x7fb370156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:59.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.849+0000 7fb365400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb3700c5e40 0x7fb370156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:59.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.849+0000 7fb36e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700cadb0 0x7fb370156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:43192/0 (socket says 172.21.15.67:43192) 2024-03-09T08:54:59.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.849+0000 7fb36e800640 1 -- 172.21.15.67:0/791233895 learned_addr learned my addr 172.21.15.67:0/791233895 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:59.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.849+0000 7fb365400640 1 -- 172.21.15.67:0/791233895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700cadb0 msgr2=0x7fb370156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:54:59.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.849+0000 7fb365400640 1 --2- 172.21.15.67:0/791233895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700cadb0 0x7fb370156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:59.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.849+0000 7fb365400640 1 -- 172.21.15.67:0/791233895 >> v1:172.21.15.134:6789/0 conn(0x7fb370153300 legacy=0x7fb3700b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:59.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.849+0000 7fb365400640 1 -- 172.21.15.67:0/791233895 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700ae6f0 con 0x7fb3700c5e40 2024-03-09T08:54:59.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.849+0000 7fb36e800640 1 --2- 172.21.15.67:0/791233895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700cadb0 0x7fb370156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:54:59.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.851+0000 7fb365400640 1 --2- 172.21.15.67:0/791233895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3700c5e40 0x7fb370156010 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3580097a0 tx=0x7fb35802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad09430841fd36fe server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:59.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.851+0000 7fb36d400640 1 -- 172.21.15.67:0/791233895 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358035020 con 0x7fb3700c5e40 2024-03-09T08:54:59.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.851+0000 7fb36d400640 1 -- 172.21.15.67:0/791233895 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb358031810 con 0x7fb3700c5e40 2024-03-09T08:54:59.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.851+0000 7fb36d400640 1 -- 172.21.15.67:0/791233895 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358031be0 con 0x7fb3700c5e40 2024-03-09T08:54:59.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.852+0000 7fb36f200640 1 -- 172.21.15.67:0/791233895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3700c5e40 msgr2=0x7fb370156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:59.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.852+0000 7fb36f200640 1 --2- 172.21.15.67:0/791233895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3700c5e40 0x7fb370156010 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3580097a0 tx=0x7fb35802b8b0 comp rx=0 tx=0).stop 2024-03-09T08:54:59.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.852+0000 7fb36f200640 1 -- 172.21.15.67:0/791233895 shutdown_connections 2024-03-09T08:54:59.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.852+0000 7fb36f200640 1 --2- 172.21.15.67:0/791233895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700cadb0 0x7fb370156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:59.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.852+0000 7fb36f200640 1 --2- 172.21.15.67:0/791233895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3700c5e40 0x7fb370156010 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:59.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.852+0000 7fb36f200640 1 -- 172.21.15.67:0/791233895 >> 172.21.15.67:0/791233895 conn(0x7fb3700732d0 msgr2=0x7fb3700ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:54:59.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.852+0000 7fb36f200640 1 -- 172.21.15.67:0/791233895 shutdown_connections 2024-03-09T08:54:59.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.852+0000 7fb36f200640 1 -- 172.21.15.67:0/791233895 wait complete. 2024-03-09T08:54:59.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.854+0000 7fb36f200640 1 Processor -- start 2024-03-09T08:54:59.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.854+0000 7fb36f200640 1 -- start start 2024-03-09T08:54:59.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.855+0000 7fb36f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700c5e40 0x7fb3700ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:59.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.855+0000 7fb36f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3700cadb0 0x7fb3700abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:59.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.855+0000 7fb36f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb370153300 0x7fb37014d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:59.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.855+0000 7fb36f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb370159e90 con 0x7fb3700cadb0 2024-03-09T08:54:59.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.855+0000 7fb36f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb370159bb0 con 0x7fb370153300 2024-03-09T08:54:59.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.855+0000 7fb36f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700c59e0 con 0x7fb3700c5e40 2024-03-09T08:54:59.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.855+0000 7fb365400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3700cadb0 0x7fb3700abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:59.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.855+0000 7fb365400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3700cadb0 0x7fb3700abb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54220/0 (socket says 172.21.15.67:54220) 2024-03-09T08:54:59.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.855+0000 7fb36e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb370153300 0x7fb37014d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:59.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.855+0000 7fb365400640 1 -- 172.21.15.67:0/3499269119 learned_addr learned my addr 172.21.15.67:0/3499269119 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:54:59.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.856+0000 7fb36de00640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700c5e40 0x7fb3700ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:59.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.856+0000 7fb365400640 1 -- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700c5e40 msgr2=0x7fb3700ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:59.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.856+0000 7fb365400640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700c5e40 0x7fb3700ab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:59.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.856+0000 7fb365400640 1 -- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb370153300 msgr2=0x7fb37014d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:54:59.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.856+0000 7fb365400640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb370153300 0x7fb37014d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:54:59.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.856+0000 7fb365400640 1 -- 172.21.15.67:0/3499269119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700ac0c0 con 0x7fb3700cadb0 2024-03-09T08:54:59.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.857+0000 7fb365400640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3700cadb0 0x7fb3700abb80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb358034040 tx=0x7fb3580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:59.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.857+0000 7fb366a00640 1 -- 172.21.15.67:0/3499269119 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358035040 con 0x7fb3700cadb0 2024-03-09T08:54:59.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.857+0000 7fb366a00640 1 -- 172.21.15.67:0/3499269119 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb358039cf0 con 0x7fb3700cadb0 2024-03-09T08:54:59.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.857+0000 7fb36f200640 1 -- 172.21.15.67:0/3499269119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3700c76b0 con 0x7fb3700cadb0 2024-03-09T08:54:59.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.857+0000 7fb366a00640 1 -- 172.21.15.67:0/3499269119 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358039ea0 con 0x7fb3700cadb0 2024-03-09T08:54:59.856 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 133 2024-03-09T08:54:59.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.857+0000 7fb36f200640 1 -- 172.21.15.67:0/3499269119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3700c7c70 con 0x7fb3700cadb0 2024-03-09T08:54:59.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.859+0000 7fb36f200640 1 -- 172.21.15.67:0/3499269119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37004f3c0 con 0x7fb3700cadb0 2024-03-09T08:54:59.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.859+0000 7fb366a00640 1 -- 172.21.15.67:0/3499269119 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb358038020 con 0x7fb3700cadb0 2024-03-09T08:54:59.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.860+0000 7fb366a00640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb34c06b500 0x7fb34c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:54:59.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.860+0000 7fb366a00640 1 -- 172.21.15.67:0/3499269119 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6693+0+0 (secure 0 0 0) 0x7fb3580ac6b0 con 0x7fb3700cadb0 2024-03-09T08:54:59.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.861+0000 7fb36de00640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb34c06b500 0x7fb34c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:54:59.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.862+0000 7fb36de00640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb34c06b500 0x7fb34c06d9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb360001640 tx=0x7fb360009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:54:59.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:54:59.869+0000 7fb366a00640 1 -- 172.21.15.67:0/3499269119 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb35807b020 con 0x7fb3700cadb0 2024-03-09T08:55:00.116 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 134 2024-03-09T08:55:00.141 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 135 2024-03-09T08:55:00.166 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 136 2024-03-09T08:55:00.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.179+0000 7fb36f200640 1 -- 172.21.15.67:0/3499269119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb3700cab90 con 0x7fb3700cadb0 2024-03-09T08:55:00.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.180+0000 7fb366a00640 1 -- 172.21.15.67:0/3499269119 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fb35807a920 con 0x7fb3700cadb0 2024-03-09T08:55:00.178 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:55:00.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 -- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb34c06b500 msgr2=0x7fb34c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb34c06b500 0x7fb34c06d9c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb360001640 tx=0x7fb360009040 comp rx=0 tx=0).stop 2024-03-09T08:55:00.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 -- 172.21.15.67:0/3499269119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3700cadb0 msgr2=0x7fb3700abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3700cadb0 0x7fb3700abb80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb358034040 tx=0x7fb3580046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 -- 172.21.15.67:0/3499269119 shutdown_connections 2024-03-09T08:55:00.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb34c06b500 0x7fb34c06d9c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb370153300 0x7fb37014d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3700cadb0 0x7fb3700abb80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 --2- 172.21.15.67:0/3499269119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3700c5e40 0x7fb3700ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 -- 172.21.15.67:0/3499269119 >> 172.21.15.67:0/3499269119 conn(0x7fb3700732d0 msgr2=0x7fb370158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:00.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 -- 172.21.15.67:0/3499269119 shutdown_connections 2024-03-09T08:55:00.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.183+0000 7fb36f200640 1 -- 172.21.15.67:0/3499269119 wait complete. 2024-03-09T08:55:00.188 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 137 2024-03-09T08:55:00.201 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:55:00.201 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T08:55:00.201 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:55:00.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:55:00.207 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 208 2024-03-09T08:55:00.213 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 138 2024-03-09T08:55:00.223 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 209 2024-03-09T08:55:00.235 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 139 2024-03-09T08:55:00.256 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 140 2024-03-09T08:55:00.284 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 141 2024-03-09T08:55:00.308 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 142 2024-03-09T08:55:00.326 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 143 2024-03-09T08:55:00.342 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 144 2024-03-09T08:55:00.358 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 145 2024-03-09T08:55:00.375 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 146 2024-03-09T08:55:00.391 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 147 2024-03-09T08:55:00.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.417+0000 7f0608400640 1 Processor -- start 2024-03-09T08:55:00.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.418+0000 7f0608400640 1 -- start start 2024-03-09T08:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.420+0000 7f0608400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f060014fef0 0x7f060006b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.420+0000 7f0608400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f06000bed00 0x7f06000ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.420+0000 7f0608400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0600155e70 con 0x7f06000c5ff0 2024-03-09T08:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.420+0000 7f0608400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06000c9d60 con 0x7f06000bed00 2024-03-09T08:55:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.420+0000 7f0608400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06000c9710 con 0x7f060014fef0 2024-03-09T08:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.420+0000 7f0606400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f06000bed00 0x7f06000ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.420+0000 7f0606400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f06000bed00 0x7f06000ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54230/0 (socket says 172.21.15.67:54230) 2024-03-09T08:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.420+0000 7f0606400640 1 -- 172.21.15.67:0/1899862675 learned_addr learned my addr 172.21.15.67:0/1899862675 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.420+0000 7f0605a00640 1 --2- 172.21.15.67:0/1899862675 >> v2:172.21.15.134:3300/0 conn(0x7f060014fef0 0x7f060006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.421+0000 7f0606400640 1 -- 172.21.15.67:0/1899862675 >> v2:172.21.15.134:3300/0 conn(0x7f060014fef0 msgr2=0x7f060006b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.421+0000 7f0606400640 1 --2- 172.21.15.67:0/1899862675 >> v2:172.21.15.134:3300/0 conn(0x7f060014fef0 0x7f060006b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.421+0000 7f0606400640 1 -- 172.21.15.67:0/1899862675 >> v1:172.21.15.67:6789/0 conn(0x7f06000c5ff0 legacy=0x7f06000ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.421+0000 7f0606400640 1 -- 172.21.15.67:0/1899862675 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06000c99f0 con 0x7f06000bed00 2024-03-09T08:55:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.422+0000 7f0606400640 1 --2- 172.21.15.67:0/1899862675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06000bed00 0x7f06000ca900 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f06000c9160 tx=0x7f05f802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932229be033ead63 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.422+0000 7f05ffe00640 1 -- 172.21.15.67:0/1899862675 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f802ca80 con 0x7f06000bed00 2024-03-09T08:55:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.422+0000 7f05ffe00640 1 -- 172.21.15.67:0/1899862675 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f802cc20 con 0x7f06000bed00 2024-03-09T08:55:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.422+0000 7f05ffe00640 1 -- 172.21.15.67:0/1899862675 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f80343f0 con 0x7f06000bed00 2024-03-09T08:55:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.423+0000 7f0608400640 1 -- 172.21.15.67:0/1899862675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06000bed00 msgr2=0x7f06000ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.423+0000 7f0608400640 1 --2- 172.21.15.67:0/1899862675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06000bed00 0x7f06000ca900 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f06000c9160 tx=0x7f05f802bee0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.423+0000 7f0608400640 1 -- 172.21.15.67:0/1899862675 shutdown_connections 2024-03-09T08:55:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.423+0000 7f0608400640 1 --2- 172.21.15.67:0/1899862675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06000bed00 0x7f06000ca900 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.423+0000 7f0608400640 1 --2- 172.21.15.67:0/1899862675 >> v2:172.21.15.134:3300/0 conn(0x7f060014fef0 0x7f060006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.423+0000 7f0608400640 1 -- 172.21.15.67:0/1899862675 >> 172.21.15.67:0/1899862675 conn(0x7f06000cb670 msgr2=0x7f06000cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:00.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.423+0000 7f0608400640 1 -- 172.21.15.67:0/1899862675 shutdown_connections 2024-03-09T08:55:00.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.423+0000 7f0608400640 1 -- 172.21.15.67:0/1899862675 wait complete. 2024-03-09T08:55:00.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.424+0000 7f0608400640 1 Processor -- start 2024-03-09T08:55:00.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.425+0000 7f0608400640 1 -- start start 2024-03-09T08:55:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.425+0000 7f0608400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06000bed00 0x7f06001530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.425+0000 7f0608400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06000c5ff0 0x7f0600150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0608400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f060014fef0 0x7f0600154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0608400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000cb470 con 0x7f06000c5ff0 2024-03-09T08:55:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0608400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000cb2f0 con 0x7f060014fef0 2024-03-09T08:55:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0608400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060006b580 con 0x7f06000bed00 2024-03-09T08:55:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0605000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06000c5ff0 0x7f0600150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0605000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06000c5ff0 0x7f0600150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54242/0 (socket says 172.21.15.67:54242) 2024-03-09T08:55:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0605000640 1 -- 172.21.15.67:0/2487821281 learned_addr learned my addr 172.21.15.67:0/2487821281 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:00.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0605a00640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06000bed00 0x7f06001530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0606400640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f060014fef0 0x7f0600154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0605000640 1 -- 172.21.15.67:0/2487821281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06000bed00 msgr2=0x7f06001530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0605000640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06000bed00 0x7f06001530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0605000640 1 -- 172.21.15.67:0/2487821281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f060014fef0 msgr2=0x7f0600154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0605000640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f060014fef0 0x7f0600154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.426+0000 7f0605000640 1 -- 172.21.15.67:0/2487821281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06001502d0 con 0x7f06000c5ff0 2024-03-09T08:55:00.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.427+0000 7f0605000640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06000c5ff0 0x7f0600150870 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00c440 tx=0x7f05ec001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:00.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.427+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec0052d0 con 0x7f06000c5ff0 2024-03-09T08:55:00.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.427+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec005470 con 0x7f06000c5ff0 2024-03-09T08:55:00.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.427+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec007260 con 0x7f06000c5ff0 2024-03-09T08:55:00.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.427+0000 7f0608400640 1 -- 172.21.15.67:0/2487821281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0600150db0 con 0x7f06000c5ff0 2024-03-09T08:55:00.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.427+0000 7f0608400640 1 -- 172.21.15.67:0/2487821281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06000c5380 con 0x7f06000c5ff0 2024-03-09T08:55:00.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.428+0000 7f0608400640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001488e0 0x7f0600148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.428+0000 7f0608400640 1 -- 172.21.15.67:0/2487821281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000c58a0 con 0x7f06001488e0 2024-03-09T08:55:00.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.429+0000 7f0605a00640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001488e0 0x7f0600148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.429+0000 7f0605a00640 1 -- 172.21.15.67:0/2487821281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0600075a00 con 0x7f06001488e0 2024-03-09T08:55:00.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.429+0000 7f0605a00640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001488e0 0x7f0600148cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f05f400f440 tx=0x7f05f400f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:00.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.430+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4019030 con 0x7f06001488e0 2024-03-09T08:55:00.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.430+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f05ec007400 con 0x7f06000c5ff0 2024-03-09T08:55:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.431+0000 7f05fd600640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05e006b3d0 0x7f05e006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.431+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f05ec089970 con 0x7f06000c5ff0 2024-03-09T08:55:00.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.432+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f05f4004020 con 0x7f06001488e0 2024-03-09T08:55:00.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.432+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001488e0 msgr2=0x7f0600148cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.432+0000 7f05fd600640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001488e0 0x7f0600148cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f05f400f440 tx=0x7f05f400f900 comp rx=0 tx=0).stop 2024-03-09T08:55:00.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.432+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001488e0 msgr2=0x7f0600148cc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:00.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.432+0000 7f05fd600640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001488e0 0x7f0600148cc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.432+0000 7f0606400640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05e006b3d0 0x7f05e006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.433+0000 7f0606400640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05e006b3d0 0x7f05e006d890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f05f8035000 tx=0x7f05f8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:00.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.439+0000 7f0608400640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001eb800 0x7f06001ebbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.439+0000 7f0608400640 1 -- 172.21.15.67:0/2487821281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06000ab460 con 0x7f06001eb800 2024-03-09T08:55:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.439+0000 7f0605a00640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001eb800 0x7f06001ebbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.440+0000 7f0605a00640 1 -- 172.21.15.67:0/2487821281 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f06000cb0d0 con 0x7f06001eb800 2024-03-09T08:55:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.440+0000 7f0605a00640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001eb800 0x7f06001ebbe0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f05f400f440 tx=0x7f05f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.440+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4003610 con 0x7f06001eb800 2024-03-09T08:55:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f05f4003610 con 0x7f06001eb800 2024-03-09T08:55:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001eb800 msgr2=0x7f06001ebbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f05fd600640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001eb800 0x7f06001ebbe0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f05f400f440 tx=0x7f05f40046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f05fd600640 1 -- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001eb800 msgr2=0x7f06001ebbe0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:00.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f05fd600640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06001eb800 0x7f06001ebbe0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.439 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:55:00.439 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:00.439 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:00.439 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:55:00.439 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 22, 2024-03-09T08:55:00.440 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:55:00.440 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:55:00.440 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:55:00.440 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:55:00.440 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.440 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T08:55:00.440 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:00.441 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:55:00.441 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:55:00.441 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:00.441 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:00.441 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:00.441 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:00.442 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:00.442 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:00.442 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:00.442 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:00.442 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:00.442 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:00.442 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:00.442 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.442 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:55:00.443 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:55:00.443 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:00.443 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:00.443 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:00.443 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:00.443 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:00.443 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:00.443 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:00.443 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:00.444 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:00.444 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:00.444 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:00.444 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.444 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:55:00.445 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:55:00.445 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:55:00.445 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:55:00.445 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:55:00.445 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:55:00.445 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:00.445 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:00.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:55:00.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:55:00.446 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:55:00.446 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:55:00.446 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:55:00.446 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:55:00.446 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:55:00.446 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.447 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.450 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:00.450 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:00.450 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:00.450 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:00.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.450 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:55:00.451 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.453 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:00.454 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:55:00.455 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:00.456 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:00.457 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.457 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.457 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:55:00.457 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:55:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f0608400640 1 -- 172.21.15.67:0/2487821281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05e006b3d0 msgr2=0x7f05e006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f0608400640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f05e006b3d0 0x7f05e006d890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f05f8035000 tx=0x7f05f8030000 comp rx=0 tx=0).stop 2024-03-09T08:55:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f0608400640 1 -- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06000c5ff0 msgr2=0x7f0600150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f0608400640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06000c5ff0 0x7f0600150870 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00c440 tx=0x7f05ec001e70 comp rx=0 tx=0).stop 2024-03-09T08:55:00.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f0606400640 1 -- 172.21.15.67:0/2487821281 reap_dead start 2024-03-09T08:55:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f0608400640 1 -- 172.21.15.67:0/2487821281 shutdown_connections 2024-03-09T08:55:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f0608400640 1 --2- 172.21.15.67:0/2487821281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06000c5ff0 0x7f0600150870 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.441+0000 7f0608400640 1 -- 172.21.15.67:0/2487821281 >> 172.21.15.67:0/2487821281 conn(0x7f06000cb670 msgr2=0x7f060014b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.442+0000 7f0608400640 1 -- 172.21.15.67:0/2487821281 shutdown_connections 2024-03-09T08:55:00.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.442+0000 7f0608400640 1 -- 172.21.15.67:0/2487821281 wait complete. 2024-03-09T08:55:00.463 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:55:00.463 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:55:00.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:55:00.518 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 210 2024-03-09T08:55:00.534 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 211 2024-03-09T08:55:00.550 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 212 2024-03-09T08:55:00.570 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 213 2024-03-09T08:55:00.593 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 214 2024-03-09T08:55:00.612 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 215 2024-03-09T08:55:00.627 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 216 2024-03-09T08:55:00.629 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 148 2024-03-09T08:55:00.648 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 217 2024-03-09T08:55:00.662 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 218 2024-03-09T08:55:00.678 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 219 2024-03-09T08:55:00.697 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 220 2024-03-09T08:55:00.715 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 221 2024-03-09T08:55:00.730 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 222 2024-03-09T08:55:00.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.743+0000 7fbfcec00640 1 Processor -- start 2024-03-09T08:55:00.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.743+0000 7fbfcec00640 1 -- start start 2024-03-09T08:55:00.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.745+0000 7fbfcec00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbfc80c95d0 0x7fbfc8159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.745+0000 7fbfcec00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc8159d60 con 0x7fbfc8153220 2024-03-09T08:55:00.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.745+0000 7fbfcec00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc815a040 con 0x7fbfc8159850 2024-03-09T08:55:00.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.745+0000 7fbfcec00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc80c8060 con 0x7fbfc80c95d0 2024-03-09T08:55:00.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.745+0000 7fbfcd800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbfc8153220 0x7fbfc80c8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44252/0 (socket says 172.21.15.67:44252) 2024-03-09T08:55:00.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.745+0000 7fbfcd800640 1 -- 172.21.15.67:0/786182990 learned_addr learned my addr 172.21.15.67:0/786182990 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:00.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.745+0000 7fbfcce00640 1 --2- 172.21.15.67:0/786182990 >> v2:172.21.15.134:3300/0 conn(0x7fbfc80c95d0 0x7fbfc8159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.744 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 223 2024-03-09T08:55:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29054234 0 0) 0x7fbfc8159d60 con 0x7fbfc8153220 2024-03-09T08:55:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb0003520 con 0x7fbfc8153220 2024-03-09T08:55:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14756815 0 0) 0x7fbfb0003520 con 0x7fbfc8153220 2024-03-09T08:55:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 >> v2:172.21.15.134:3300/0 conn(0x7fbfc80c95d0 msgr2=0x7fbfc8159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 --2- 172.21.15.67:0/786182990 >> v2:172.21.15.134:3300/0 conn(0x7fbfc80c95d0 0x7fbfc8159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 >> v1:172.21.15.134:6789/0 conn(0x7fbfc8159850 legacy=0x7fbfc80c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80c8340 con 0x7fbfc8153220 2024-03-09T08:55:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fbfbc001670 con 0x7fbfc8153220 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 >> v1:172.21.15.67:6789/0 conn(0x7fbfc8153220 legacy=0x7fbfc80c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 --2- 172.21.15.67:0/786182990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfb0003a20 0x7fbfb0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 --2- 172.21.15.67:0/786182990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfb0004800 0x7fbfb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 --2- 172.21.15.67:0/786182990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 0x7fbfb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80c8060 con 0x7fbfb0005230 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb000c190 con 0x7fbfb0003a20 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.746+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb000c390 con 0x7fbfb0004800 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.747+0000 7fbfce200640 1 --2- 172.21.15.67:0/786182990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 0x7fbfb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.747+0000 7fbfcd800640 1 --2- 172.21.15.67:0/786182990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfb0003a20 0x7fbfb0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.747+0000 7fbfcce00640 1 --2- 172.21.15.67:0/786182990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfb0004800 0x7fbfb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.747+0000 7fbfce200640 1 -- 172.21.15.67:0/786182990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfb0004800 msgr2=0x7fbfb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.747+0000 7fbfce200640 1 --2- 172.21.15.67:0/786182990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfb0004800 0x7fbfb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.747+0000 7fbfce200640 1 -- 172.21.15.67:0/786182990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfb0003a20 msgr2=0x7fbfb0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.747+0000 7fbfce200640 1 --2- 172.21.15.67:0/786182990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfb0003a20 0x7fbfb0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.747+0000 7fbfce200640 1 -- 172.21.15.67:0/786182990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb0003520 con 0x7fbfb0005230 2024-03-09T08:55:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.748+0000 7fbfce200640 1 -- 172.21.15.67:0/786182990 reap_dead start 2024-03-09T08:55:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.748+0000 7fbfce200640 1 --2- 172.21.15.67:0/786182990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 0x7fbfb000b9c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0001e50 tx=0x7fbfb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e816f9e2daf89a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.749+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0034070 con 0x7fbfb0005230 2024-03-09T08:55:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.749+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc0003b90 con 0x7fbfb0005230 2024-03-09T08:55:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.749+0000 7fbfc7e00640 1 -- 172.21.15.67:0/786182990 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0001a70 con 0x7fbfb0005230 2024-03-09T08:55:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.750+0000 7fbfcec00640 1 -- 172.21.15.67:0/786182990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 msgr2=0x7fbfb000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.750+0000 7fbfcec00640 1 --2- 172.21.15.67:0/786182990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 0x7fbfb000b9c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0001e50 tx=0x7fbfb0005830 comp rx=0 tx=0).stop 2024-03-09T08:55:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.750+0000 7fbfcec00640 1 -- 172.21.15.67:0/786182990 shutdown_connections 2024-03-09T08:55:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.750+0000 7fbfcec00640 1 --2- 172.21.15.67:0/786182990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 0x7fbfb000b9c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.750+0000 7fbfcec00640 1 -- 172.21.15.67:0/786182990 >> 172.21.15.67:0/786182990 conn(0x7fbfc80ad060 msgr2=0x7fbfc80ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.750+0000 7fbfcec00640 1 -- 172.21.15.67:0/786182990 shutdown_connections 2024-03-09T08:55:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.750+0000 7fbfcec00640 1 -- 172.21.15.67:0/786182990 wait complete. 2024-03-09T08:55:00.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.753+0000 7fbfcec00640 1 Processor -- start 2024-03-09T08:55:00.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.753+0000 7fbfcec00640 1 -- start start 2024-03-09T08:55:00.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.754+0000 7fbfcec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 0x7fbfc8151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.754+0000 7fbfcec00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc80af570 0x7fbfc814a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.754+0000 7fbfcec00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfc8152100 0x7fbfc814a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.754+0000 7fbfcec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc81560a0 con 0x7fbfb0005230 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.754+0000 7fbfcec00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8155f20 con 0x7fbfc80af570 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.754+0000 7fbfcec00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8153600 con 0x7fbfc8152100 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.754+0000 7fbfcd800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 0x7fbfc8151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.754+0000 7fbfcd800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 0x7fbfc8151bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54292/0 (socket says 172.21.15.67:54292) 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.754+0000 7fbfcd800640 1 -- 172.21.15.67:0/2469545690 learned_addr learned my addr 172.21.15.67:0/2469545690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.754+0000 7fbfcce00640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc80af570 0x7fbfc814a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.755+0000 7fbfce200640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfc8152100 0x7fbfc814a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.755+0000 7fbfcd800640 1 -- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfc8152100 msgr2=0x7fbfc814a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.755+0000 7fbfcd800640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfc8152100 0x7fbfc814a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.755+0000 7fbfcd800640 1 -- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc80af570 msgr2=0x7fbfc814a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.755+0000 7fbfcd800640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc80af570 0x7fbfc814a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.755+0000 7fbfcd800640 1 -- 172.21.15.67:0/2469545690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc814d910 con 0x7fbfb0005230 2024-03-09T08:55:00.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.755+0000 7fbfce200640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbfc8152100 0x7fbfc814a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:55:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.755+0000 7fbfcd800640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 0x7fbfc8151bc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007340 tx=0x7fbfbc007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.755+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00f940 con 0x7fbfb0005230 2024-03-09T08:55:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.756+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc018070 con 0x7fbfb0005230 2024-03-09T08:55:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.756+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00fd40 con 0x7fbfb0005230 2024-03-09T08:55:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.756+0000 7fbfcec00640 1 -- 172.21.15.67:0/2469545690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc814dbf0 con 0x7fbfb0005230 2024-03-09T08:55:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.756+0000 7fbfcec00640 1 -- 172.21.15.67:0/2469545690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc81ebae0 con 0x7fbfb0005230 2024-03-09T08:55:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.757+0000 7fbfcec00640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc81ef950 0x7fbfc81ec000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.757+0000 7fbfcec00640 1 -- 172.21.15.67:0/2469545690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc81ebd20 con 0x7fbfc81ef950 2024-03-09T08:55:00.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.758+0000 7fbfcce00640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc81ef950 0x7fbfc81ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.758+0000 7fbfcce00640 1 -- 172.21.15.67:0/2469545690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfc806d3c0 con 0x7fbfc81ef950 2024-03-09T08:55:00.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.759+0000 7fbfcce00640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc81ef950 0x7fbfc81ec000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80b1120 tx=0x7fbfb40139a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:00.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.759+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbfbc01d460 con 0x7fbfb0005230 2024-03-09T08:55:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.760+0000 7fbfc5600640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbfa806b500 0x7fbfa806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.760+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6693+0+0 (secure 0 0 0) 0x7fbfbc08a780 con 0x7fbfb0005230 2024-03-09T08:55:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.760+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb401c070 con 0x7fbfc81ef950 2024-03-09T08:55:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.760+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fbfb4015950 con 0x7fbfc81ef950 2024-03-09T08:55:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.760+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc81ef950 msgr2=0x7fbfc81ec000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.760+0000 7fbfc5600640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc81ef950 0x7fbfc81ec000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80b1120 tx=0x7fbfb40139a0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.760+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc81ef950 msgr2=0x7fbfc81ec000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.760+0000 7fbfc5600640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc81ef950 0x7fbfc81ec000 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80b1120 tx=0x7fbfb40139a0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.760+0000 7fbfce200640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbfa806b500 0x7fbfa806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.761+0000 7fbfce200640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbfa806b500 0x7fbfa806d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000c160 tx=0x7fbfc0009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:00.761 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 224 2024-03-09T08:55:00.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.772+0000 7fbfcec00640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc814ad00 0x7fbfc81ec540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.772+0000 7fbfcec00640 1 -- 172.21.15.67:0/2469545690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8152dd0 con 0x7fbfc814ad00 2024-03-09T08:55:00.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.772+0000 7fbfcce00640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc814ad00 0x7fbfc81ec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.773+0000 7fbfcce00640 1 -- 172.21.15.67:0/2469545690 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbfc804f3c0 con 0x7fbfc814ad00 2024-03-09T08:55:00.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.774+0000 7fbfcce00640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc814ad00 0x7fbfc81ec540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbfc806b140 tx=0x7fbfb40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:00.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.774+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb401c040 con 0x7fbfc814ad00 2024-03-09T08:55:00.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.775+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5044 (secure 0 0 0) 0x7fbfb4021450 con 0x7fbfc814ad00 2024-03-09T08:55:00.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.775+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc814ad00 msgr2=0x7fbfc81ec540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.775+0000 7fbfc5600640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc814ad00 0x7fbfc81ec540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbfc806b140 tx=0x7fbfb40046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.775+0000 7fbfc5600640 1 -- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc814ad00 msgr2=0x7fbfc81ec540 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:00.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.775+0000 7fbfc5600640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbfc814ad00 0x7fbfc81ec540 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbfc806b140 tx=0x7fbfb40046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.773 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:55:00.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:00.773 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:00.773 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:55:00.773 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 22, 2024-03-09T08:55:00.773 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:55:00.773 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:55:00.773 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:55:00.773 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:00.774 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:00.775 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:00.776 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:00.777 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:00.778 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:00.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:00.779 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:55:00.779 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.782 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.782 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:00.782 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:55:00.782 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.782 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.782 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.783 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:00.784 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.785 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.786 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.787 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.788 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:00.789 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:55:00.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:00.789 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:00.789 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:00.789 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:00.789 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:00.789 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:00.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:00.789 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:55:00.789 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:55:00.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.775+0000 7fbfcec00640 1 -- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbfa806b500 msgr2=0x7fbfa806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.775+0000 7fbfcec00640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbfa806b500 0x7fbfa806d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000c160 tx=0x7fbfc0009030 comp rx=0 tx=0).stop 2024-03-09T08:55:00.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.775+0000 7fbfcec00640 1 -- 172.21.15.67:0/2469545690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 msgr2=0x7fbfc8151bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.775+0000 7fbfcec00640 1 --2- 172.21.15.67:0/2469545690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfb0005230 0x7fbfc8151bc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007340 tx=0x7fbfbc007800 comp rx=0 tx=0).stop 2024-03-09T08:55:00.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.775+0000 7fbfce200640 1 -- 172.21.15.67:0/2469545690 reap_dead start 2024-03-09T08:55:00.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.775+0000 7fbfce200640 1 -- 172.21.15.67:0/2469545690 reap_dead start 2024-03-09T08:55:00.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.776+0000 7fbfcec00640 1 -- 172.21.15.67:0/2469545690 shutdown_connections 2024-03-09T08:55:00.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.776+0000 7fbfcec00640 1 -- 172.21.15.67:0/2469545690 >> 172.21.15.67:0/2469545690 conn(0x7fbfc80ad060 msgr2=0x7fbfc80ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:00.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.776+0000 7fbfcec00640 1 -- 172.21.15.67:0/2469545690 shutdown_connections 2024-03-09T08:55:00.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.776+0000 7fbfcec00640 1 -- 172.21.15.67:0/2469545690 wait complete. 2024-03-09T08:55:00.791 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 225 2024-03-09T08:55:00.791 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 226 2024-03-09T08:55:00.792 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:55:00.792 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:55:00.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:55:00.802 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 227 2024-03-09T08:55:00.815 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 228 2024-03-09T08:55:00.829 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 229 2024-03-09T08:55:00.843 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 230 2024-03-09T08:55:00.860 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 231 2024-03-09T08:55:00.876 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 232 2024-03-09T08:55:00.895 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 233 2024-03-09T08:55:00.913 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 234 2024-03-09T08:55:00.932 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 235 2024-03-09T08:55:00.942 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 149 2024-03-09T08:55:00.949 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 236 2024-03-09T08:55:00.969 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 237 2024-03-09T08:55:00.987 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 238 2024-03-09T08:55:00.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.994+0000 7f127f200640 1 Processor -- start 2024-03-09T08:55:00.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.994+0000 7f127f200640 1 -- start start 2024-03-09T08:55:00.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.996+0000 7f127f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1278153340 0x7f12780c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.997+0000 7f127f200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f12780c26c0 0x7f1278160440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:00.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.997+0000 7f127f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12780c4ac0 con 0x7f12780c1cf0 2024-03-09T08:55:00.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.997+0000 7f127f200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1278155d10 con 0x7f12780c26c0 2024-03-09T08:55:00.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.997+0000 7f127f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12780ab460 con 0x7f1278153340 2024-03-09T08:55:00.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.997+0000 7f127d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f12780c1cf0 0x7f12781561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44256/0 (socket says 172.21.15.67:44256) 2024-03-09T08:55:00.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.997+0000 7f127d400640 1 -- 172.21.15.67:0/2304775052 learned_addr learned my addr 172.21.15.67:0/2304775052 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:00.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.997+0000 7f127e800640 1 --2- 172.21.15.67:0/2304775052 >> v2:172.21.15.134:3300/0 conn(0x7f12780c26c0 0x7f1278160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.997+0000 7f127de00640 1 --2- 172.21.15.67:0/2304775052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1278153340 0x7f12780c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:00.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.998+0000 7f127ca00640 1 -- 172.21.15.67:0/2304775052 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058231450 0 0) 0x7f12780c4ac0 con 0x7f12780c1cf0 2024-03-09T08:55:00.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.998+0000 7f127ca00640 1 -- 172.21.15.67:0/2304775052 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1260003520 con 0x7f12780c1cf0 2024-03-09T08:55:00.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.998+0000 7f127e800640 1 -- 172.21.15.67:0/2304775052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1278153340 msgr2=0x7f12780c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:00.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.998+0000 7f127e800640 1 --2- 172.21.15.67:0/2304775052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1278153340 0x7f12780c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.998+0000 7f127e800640 1 -- 172.21.15.67:0/2304775052 >> v1:172.21.15.67:6789/0 conn(0x7f12780c1cf0 legacy=0x7f12781561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.998+0000 7f127e800640 1 -- 172.21.15.67:0/2304775052 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12780c47e0 con 0x7f12780c26c0 2024-03-09T08:55:00.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.999+0000 7f127e800640 1 --2- 172.21.15.67:0/2304775052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12780c26c0 0x7f1278160440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1274009d90 tx=0x7f127402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6cb39c61552720c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:00.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.999+0000 7f127ca00640 1 -- 172.21.15.67:0/2304775052 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274039070 con 0x7f12780c26c0 2024-03-09T08:55:00.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:00.999+0000 7f127ca00640 1 -- 172.21.15.67:0/2304775052 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127402cc60 con 0x7f12780c26c0 2024-03-09T08:55:00.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.000+0000 7f127ca00640 1 -- 172.21.15.67:0/2304775052 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274034660 con 0x7f12780c26c0 2024-03-09T08:55:00.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.000+0000 7f127f200640 1 -- 172.21.15.67:0/2304775052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12780c26c0 msgr2=0x7f1278160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:00.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.000+0000 7f127f200640 1 --2- 172.21.15.67:0/2304775052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12780c26c0 0x7f1278160440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1274009d90 tx=0x7f127402beb0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.000+0000 7f127f200640 1 -- 172.21.15.67:0/2304775052 shutdown_connections 2024-03-09T08:55:00.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.000+0000 7f127f200640 1 --2- 172.21.15.67:0/2304775052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12780c26c0 0x7f1278160440 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.000+0000 7f127f200640 1 --2- 172.21.15.67:0/2304775052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1278153340 0x7f12780c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:00.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.000+0000 7f127f200640 1 -- 172.21.15.67:0/2304775052 >> 172.21.15.67:0/2304775052 conn(0x7f127806c150 msgr2=0x7f127806c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:00.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.000+0000 7f127f200640 1 -- 172.21.15.67:0/2304775052 shutdown_connections 2024-03-09T08:55:00.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.000+0000 7f127f200640 1 -- 172.21.15.67:0/2304775052 wait complete. 2024-03-09T08:55:01.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.003+0000 7f127f200640 1 Processor -- start 2024-03-09T08:55:01.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.003+0000 7f127f200640 1 -- start start 2024-03-09T08:55:01.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.005+0000 7f127f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12780c1cf0 0x7f12780c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.005+0000 7f127f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12780c26c0 0x7f127814ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.005+0000 7f127f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1278153340 0x7f12780c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.005+0000 7f127f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1278156a30 con 0x7f12780c1cf0 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.005+0000 7f127f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12781568b0 con 0x7f12780c26c0 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.005+0000 7f127f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12780c5230 con 0x7f1278153340 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.005+0000 7f127de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12780c1cf0 0x7f12780c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.005+0000 7f127de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12780c1cf0 0x7f12780c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54298/0 (socket says 172.21.15.67:54298) 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.005+0000 7f127d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12780c26c0 0x7f127814ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.005+0000 7f127de00640 1 -- 172.21.15.67:0/3144554313 learned_addr learned my addr 172.21.15.67:0/3144554313 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.005+0000 7f127e800640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1278153340 0x7f12780c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.006+0000 7f127d400640 1 -- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1278153340 msgr2=0x7f12780c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.006+0000 7f127d400640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1278153340 0x7f12780c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.006+0000 7f127d400640 1 -- 172.21.15.67:0/3144554313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12780c1cf0 msgr2=0x7f12780c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:01.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.006+0000 7f127d400640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12780c1cf0 0x7f12780c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.006+0000 7f127d400640 1 -- 172.21.15.67:0/3144554313 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127806f730 con 0x7f12780c26c0 2024-03-09T08:55:01.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.006+0000 7f127de00640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12780c1cf0 0x7f12780c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:55:01.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.006+0000 7f127d400640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12780c26c0 0x7f127814ab00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f126800c460 tx=0x7f126800c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:01.005 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 239 2024-03-09T08:55:01.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.006+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268017070 con 0x7f12780c26c0 2024-03-09T08:55:01.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.006+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1268005ce0 con 0x7f12780c26c0 2024-03-09T08:55:01.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.007+0000 7f127f200640 1 -- 172.21.15.67:0/3144554313 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127806f450 con 0x7f12780c26c0 2024-03-09T08:55:01.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.007+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1268007290 con 0x7f12780c26c0 2024-03-09T08:55:01.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.007+0000 7f127f200640 1 -- 172.21.15.67:0/3144554313 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12781ec470 con 0x7f12780c26c0 2024-03-09T08:55:01.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.008+0000 7f127f200640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f127814b040 0x7f12780c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.008+0000 7f127f200640 1 -- 172.21.15.67:0/3144554313 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12780c5af0 con 0x7f127814b040 2024-03-09T08:55:01.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.008+0000 7f127de00640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f127814b040 0x7f12780c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:01.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.009+0000 7f127de00640 1 -- 172.21.15.67:0/3144554313 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f127804f3c0 con 0x7f127814b040 2024-03-09T08:55:01.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.009+0000 7f127de00640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f127814b040 0x7f12780c63d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f126c009fd0 tx=0x7f126c00f610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:01.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.009+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f126801a070 con 0x7f12780c26c0 2024-03-09T08:55:01.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.011+0000 7f1272000640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f125c06b4d0 0x7f125c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.011+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f1268089770 con 0x7f12780c26c0 2024-03-09T08:55:01.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.011+0000 7f127e800640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f125c06b4d0 0x7f125c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:01.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.011+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c019070 con 0x7f127814b040 2024-03-09T08:55:01.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.011+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f126c004030 con 0x7f127814b040 2024-03-09T08:55:01.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.011+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f127814b040 msgr2=0x7f12780c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:01.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.011+0000 7f1272000640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f127814b040 0x7f12780c63d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f126c009fd0 tx=0x7f126c00f610 comp rx=0 tx=0).stop 2024-03-09T08:55:01.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.011+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f127814b040 msgr2=0x7f12780c63d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:01.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.011+0000 7f1272000640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f127814b040 0x7f12780c63d0 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f126c009fd0 tx=0x7f126c00f610 comp rx=0 tx=0).stop 2024-03-09T08:55:01.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.012+0000 7f127e800640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f125c06b4d0 0x7f125c06d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1274036000 tx=0x7f1274030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:01.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.019+0000 7f127f200640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12781eebe0 0x7f12781eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.019+0000 7f127f200640 1 -- 172.21.15.67:0/3144554313 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127806d550 con 0x7f12781eebe0 2024-03-09T08:55:01.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.019+0000 7f127de00640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12781eebe0 0x7f12781eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:01.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.020+0000 7f127de00640 1 -- 172.21.15.67:0/3144554313 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1278156480 con 0x7f12781eebe0 2024-03-09T08:55:01.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.020+0000 7f127de00640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12781eebe0 0x7f12781eeff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f126c009fd0 tx=0x7f126c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:01.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.021+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c019040 con 0x7f12781eebe0 2024-03-09T08:55:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.021+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f126c01e660 con 0x7f12781eebe0 2024-03-09T08:55:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.021+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12781eebe0 msgr2=0x7f12781eeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.021+0000 7f1272000640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12781eebe0 0x7f12781eeff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f126c009fd0 tx=0x7f126c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f1272000640 1 -- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12781eebe0 msgr2=0x7f12781eeff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:01.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f1272000640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12781eebe0 0x7f12781eeff0 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f126c009fd0 tx=0x7f126c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.020 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:55:01.020 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:01.020 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:01.020 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:55:01.020 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 22, 2024-03-09T08:55:01.020 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:55:01.020 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:55:01.020 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:55:01.020 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:01.021 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:01.022 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:55:01.023 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:01.024 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:01.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:55:01.026 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:01.028 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.029 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:01.030 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:01.031 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:01.031 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:55:01.031 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:01.031 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:01.031 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:01.031 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:01.031 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:01.031 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:01.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:01.031 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:55:01.031 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:55:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f127f200640 1 -- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f125c06b4d0 msgr2=0x7f125c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f127f200640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f125c06b4d0 0x7f125c06d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1274036000 tx=0x7f1274030000 comp rx=0 tx=0).stop 2024-03-09T08:55:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f127f200640 1 -- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12780c26c0 msgr2=0x7f127814ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f127f200640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12780c26c0 0x7f127814ab00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f126800c460 tx=0x7f126800c920 comp rx=0 tx=0).stop 2024-03-09T08:55:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f127e800640 1 -- 172.21.15.67:0/3144554313 reap_dead start 2024-03-09T08:55:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f127f200640 1 -- 172.21.15.67:0/3144554313 shutdown_connections 2024-03-09T08:55:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f127f200640 1 --2- 172.21.15.67:0/3144554313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12780c26c0 0x7f127814ab00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f127f200640 1 -- 172.21.15.67:0/3144554313 >> 172.21.15.67:0/3144554313 conn(0x7f127806c150 msgr2=0x7f12780c8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:01.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f127f200640 1 -- 172.21.15.67:0/3144554313 shutdown_connections 2024-03-09T08:55:01.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.022+0000 7f127f200640 1 -- 172.21.15.67:0/3144554313 wait complete. 2024-03-09T08:55:01.033 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 240 2024-03-09T08:55:01.038 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:55:01.038 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-03-09T08:55:01.038 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T08:55:01.038 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-03-09T08:55:01.038 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T08:55:01.038 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-09T08:55:01.039 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T08:55:01.039 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 241 2024-03-09T08:55:01.063 INFO:tasks.ceph.mon.b:Stopped 2024-03-09T08:55:01.063 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T08:55:01.063 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:55:01.185 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 242 2024-03-09T08:55:01.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.266+0000 7ff34fc00640 1 Processor -- start 2024-03-09T08:55:01.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.266+0000 7ff34fc00640 1 -- start start 2024-03-09T08:55:01.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.267+0000 7ff34fc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3480c22a0 0x7ff348071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.267+0000 7ff34fc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff348157d60 0x7ff3480acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.267+0000 7ff34fc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff348153d40 con 0x7ff3480ac730 2024-03-09T08:55:01.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.267+0000 7ff34fc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff348070ee0 con 0x7ff348157d60 2024-03-09T08:55:01.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.267+0000 7ff34fc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff348154d00 con 0x7ff3480c22a0 2024-03-09T08:55:01.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.267+0000 7ff34d200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff3480ac730 0x7ff348157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44270/0 (socket says 172.21.15.67:44270) 2024-03-09T08:55:01.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.267+0000 7ff34d200640 1 -- 172.21.15.67:0/124132084 learned_addr learned my addr 172.21.15.67:0/124132084 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:01.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.268+0000 7ff34dc00640 1 --2- 172.21.15.67:0/124132084 >> v2:172.21.15.67:3300/0 conn(0x7ff348157d60 0x7ff3480acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:01.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.268+0000 7ff347e00640 1 --2- 172.21.15.67:0/124132084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3480c22a0 0x7ff348071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:01.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.268+0000 7ff347400640 1 -- 172.21.15.67:0/124132084 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848682722 0 0) 0x7ff348153d40 con 0x7ff3480ac730 2024-03-09T08:55:01.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.268+0000 7ff347400640 1 -- 172.21.15.67:0/124132084 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff330003520 con 0x7ff3480ac730 2024-03-09T08:55:01.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.268+0000 7ff34dc00640 1 -- 172.21.15.67:0/124132084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3480c22a0 msgr2=0x7ff348071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:01.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.268+0000 7ff34dc00640 1 --2- 172.21.15.67:0/124132084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3480c22a0 0x7ff348071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.268+0000 7ff34dc00640 1 -- 172.21.15.67:0/124132084 >> v1:172.21.15.67:6789/0 conn(0x7ff3480ac730 legacy=0x7ff348157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:01.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.268+0000 7ff34dc00640 1 -- 172.21.15.67:0/124132084 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3480c2790 con 0x7ff348157d60 2024-03-09T08:55:01.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.269+0000 7ff34dc00640 1 --2- 172.21.15.67:0/124132084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff348157d60 0x7ff3480acb10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff340009d90 tx=0x7ff34002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38d4c7914fcbaee server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:01.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.270+0000 7ff347400640 1 -- 172.21.15.67:0/124132084 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340039070 con 0x7ff348157d60 2024-03-09T08:55:01.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.270+0000 7ff347400640 1 -- 172.21.15.67:0/124132084 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34002cbc0 con 0x7ff348157d60 2024-03-09T08:55:01.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.270+0000 7ff347400640 1 -- 172.21.15.67:0/124132084 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340034440 con 0x7ff348157d60 2024-03-09T08:55:01.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.270+0000 7ff34fc00640 1 -- 172.21.15.67:0/124132084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff348157d60 msgr2=0x7ff3480acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:01.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.270+0000 7ff34fc00640 1 --2- 172.21.15.67:0/124132084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff348157d60 0x7ff3480acb10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff340009d90 tx=0x7ff34002be10 comp rx=0 tx=0).stop 2024-03-09T08:55:01.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.270+0000 7ff34fc00640 1 -- 172.21.15.67:0/124132084 shutdown_connections 2024-03-09T08:55:01.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.270+0000 7ff34fc00640 1 --2- 172.21.15.67:0/124132084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff348157d60 0x7ff3480acb10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.270+0000 7ff34fc00640 1 --2- 172.21.15.67:0/124132084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3480c22a0 0x7ff348071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.270+0000 7ff34fc00640 1 -- 172.21.15.67:0/124132084 >> 172.21.15.67:0/124132084 conn(0x7ff3480c7370 msgr2=0x7ff3480c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:01.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.271+0000 7ff34fc00640 1 -- 172.21.15.67:0/124132084 shutdown_connections 2024-03-09T08:55:01.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.271+0000 7ff34fc00640 1 -- 172.21.15.67:0/124132084 wait complete. 2024-03-09T08:55:01.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.272+0000 7ff34fc00640 1 Processor -- start 2024-03-09T08:55:01.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.272+0000 7ff34fc00640 1 -- start start 2024-03-09T08:55:01.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.273+0000 7ff34fc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3480ac730 0x7ff3481509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.273+0000 7ff34fc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3480c22a0 0x7ff3481ec130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.273+0000 7ff34fc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff348157d60 0x7ff3480c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.273+0000 7ff34fc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff348151610 con 0x7ff3480c22a0 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.273+0000 7ff34fc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff348070c00 con 0x7ff3480ac730 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.273+0000 7ff34fc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff348151790 con 0x7ff348157d60 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.273+0000 7ff34d200640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3480ac730 msgr2=0x7ff3481509b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.273+0000 7ff34d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3480ac730 0x7ff3481509b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.274+0000 7ff347e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3480c22a0 0x7ff3481ec130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.274+0000 7ff347e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3480c22a0 0x7ff3481ec130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54318/0 (socket says 172.21.15.67:54318) 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.274+0000 7ff347e00640 1 -- 172.21.15.67:0/1175821000 learned_addr learned my addr 172.21.15.67:0/1175821000 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.274+0000 7ff34dc00640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff348157d60 0x7ff3480c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.274+0000 7ff347e00640 1 -- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff348157d60 msgr2=0x7ff3480c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:01.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.274+0000 7ff347e00640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff348157d60 0x7ff3480c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.274+0000 7ff347e00640 1 -- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3480ac730 msgr2=0x7ff3481509b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:55:01.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.274+0000 7ff347e00640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3480ac730 0x7ff3481509b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.274+0000 7ff347e00640 1 -- 172.21.15.67:0/1175821000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3481ddc80 con 0x7ff3480c22a0 2024-03-09T08:55:01.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.274+0000 7ff347e00640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3480c22a0 0x7ff3481ec130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff33400e980 tx=0x7ff33400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:01.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.275+0000 7ff344c00640 1 -- 172.21.15.67:0/1175821000 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334017070 con 0x7ff3480c22a0 2024-03-09T08:55:01.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.275+0000 7ff344c00640 1 -- 172.21.15.67:0/1175821000 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33400ca00 con 0x7ff3480c22a0 2024-03-09T08:55:01.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.275+0000 7ff344c00640 1 -- 172.21.15.67:0/1175821000 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400cdd0 con 0x7ff3480c22a0 2024-03-09T08:55:01.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.275+0000 7ff34fc00640 1 -- 172.21.15.67:0/1175821000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3481ddf60 con 0x7ff3480c22a0 2024-03-09T08:55:01.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.275+0000 7ff34fc00640 1 -- 172.21.15.67:0/1175821000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3480c0f50 con 0x7ff3480c22a0 2024-03-09T08:55:01.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.276+0000 7ff34fc00640 1 -- 172.21.15.67:0/1175821000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff348075a00 con 0x7ff3480c22a0 2024-03-09T08:55:01.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.277+0000 7ff344c00640 1 -- 172.21.15.67:0/1175821000 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff33401a070 con 0x7ff3480c22a0 2024-03-09T08:55:01.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.278+0000 7ff344c00640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32806b480 0x7ff32806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:01.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.278+0000 7ff344c00640 1 -- 172.21.15.67:0/1175821000 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6693+0+0 (secure 0 0 0) 0x7ff33408a8c0 con 0x7ff3480c22a0 2024-03-09T08:55:01.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.279+0000 7ff34d200640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32806b480 0x7ff32806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:01.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.280+0000 7ff34d200640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32806b480 0x7ff32806d940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000c00 tx=0x7ff33c007a10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:01.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.287+0000 7ff344c00640 1 -- 172.21.15.67:0/1175821000 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff3340593f0 con 0x7ff3480c22a0 2024-03-09T08:55:01.396 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 150 2024-03-09T08:55:01.421 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 151 2024-03-09T08:55:01.446 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 152 2024-03-09T08:55:01.471 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 153 2024-03-09T08:55:01.495 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 154 2024-03-09T08:55:01.522 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 155 2024-03-09T08:55:01.544 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 156 2024-03-09T08:55:01.566 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 157 2024-03-09T08:55:01.589 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 158 2024-03-09T08:55:01.612 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 159 2024-03-09T08:55:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.625+0000 7ff34fc00640 1 -- 172.21.15.67:0/1175821000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff3480c1d70 con 0x7ff3480c22a0 2024-03-09T08:55:01.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.626+0000 7ff344c00640 1 -- 172.21.15.67:0/1175821000 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7ff334058cf0 con 0x7ff3480c22a0 2024-03-09T08:55:01.624 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:55:01.624 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 243 2024-03-09T08:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 -- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32806b480 msgr2=0x7ff32806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32806b480 0x7ff32806d940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000c00 tx=0x7ff33c007a10 comp rx=0 tx=0).stop 2024-03-09T08:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 -- 172.21.15.67:0/1175821000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3480c22a0 msgr2=0x7ff3481ec130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3480c22a0 0x7ff3481ec130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff33400e980 tx=0x7ff33400ee40 comp rx=0 tx=0).stop 2024-03-09T08:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 -- 172.21.15.67:0/1175821000 shutdown_connections 2024-03-09T08:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff32806b480 0x7ff32806d940 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff348157d60 0x7ff3480c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3480c22a0 0x7ff3481ec130 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 --2- 172.21.15.67:0/1175821000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3480ac730 0x7ff3481509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 -- 172.21.15.67:0/1175821000 >> 172.21.15.67:0/1175821000 conn(0x7ff3480c7370 msgr2=0x7ff3480c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 -- 172.21.15.67:0/1175821000 shutdown_connections 2024-03-09T08:55:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:01.629+0000 7ff34fc00640 1 -- 172.21.15.67:0/1175821000 wait complete. 2024-03-09T08:55:01.632 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 160 2024-03-09T08:55:01.642 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 244 2024-03-09T08:55:01.648 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 161 2024-03-09T08:55:01.661 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 162 2024-03-09T08:55:01.662 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 245 2024-03-09T08:55:01.674 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 163 2024-03-09T08:55:01.685 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 246 2024-03-09T08:55:01.686 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 164 2024-03-09T08:55:01.703 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 247 2024-03-09T08:55:01.722 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 248 2024-03-09T08:55:01.742 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 249 2024-03-09T08:55:01.763 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 250 2024-03-09T08:55:01.810 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 165 2024-03-09T08:55:01.829 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 166 2024-03-09T08:55:01.849 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 167 2024-03-09T08:55:01.871 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 168 2024-03-09T08:55:01.893 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 169 2024-03-09T08:55:01.913 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 170 2024-03-09T08:55:01.933 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 171 2024-03-09T08:55:01.952 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 172 2024-03-09T08:55:01.970 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 173 2024-03-09T08:55:01.989 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 174 2024-03-09T08:55:02.268 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 175 2024-03-09T08:55:02.865 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 176 2024-03-09T08:55:03.071 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 251 2024-03-09T08:55:03.086 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 252 2024-03-09T08:55:03.113 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 253 2024-03-09T08:55:03.134 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 254 2024-03-09T08:55:03.157 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 255 2024-03-09T08:55:03.177 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 256 2024-03-09T08:55:03.179 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 177 2024-03-09T08:55:03.198 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 257 2024-03-09T08:55:03.216 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 258 2024-03-09T08:55:03.232 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 259 2024-03-09T08:55:03.248 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 260 2024-03-09T08:55:03.263 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 261 2024-03-09T08:55:03.543 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 262 2024-03-09T08:55:03.558 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 263 2024-03-09T08:55:03.934 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 178 2024-03-09T08:55:03.968 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 179 2024-03-09T08:55:04.156 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 264 2024-03-09T08:55:04.303 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 265 2024-03-09T08:55:04.328 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 266 2024-03-09T08:55:04.353 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 267 2024-03-09T08:55:04.379 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 268 2024-03-09T08:55:04.405 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 269 2024-03-09T08:55:04.431 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 270 2024-03-09T08:55:04.452 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 271 2024-03-09T08:55:04.473 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 272 2024-03-09T08:55:04.494 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 273 2024-03-09T08:55:04.515 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 274 2024-03-09T08:55:04.536 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 275 2024-03-09T08:55:04.556 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 276 2024-03-09T08:55:04.577 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 277 2024-03-09T08:55:04.596 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 278 2024-03-09T08:55:04.616 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 279 2024-03-09T08:55:04.636 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 280 2024-03-09T08:55:04.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:55:04.744 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 180 2024-03-09T08:55:04.779 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 181 2024-03-09T08:55:04.810 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 182 2024-03-09T08:55:04.825 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 281 2024-03-09T08:55:04.828 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 183 2024-03-09T08:55:04.849 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 184 2024-03-09T08:55:04.873 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 185 2024-03-09T08:55:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.877+0000 7f85f6e00640 1 Processor -- start 2024-03-09T08:55:04.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.878+0000 7f85f6e00640 1 -- start start 2024-03-09T08:55:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.879+0000 7f85f6e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f00ac730 0x7f85f0071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.879+0000 7f85f6e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85f0157d60 0x7f85f00acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.879+0000 7f85f6e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f0153d40 con 0x7f85f00c22a0 2024-03-09T08:55:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.879+0000 7f85f6e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f0070ee0 con 0x7f85f0157d60 2024-03-09T08:55:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.879+0000 7f85f6e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0154d00 con 0x7f85f00ac730 2024-03-09T08:55:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.879+0000 7f85e7e00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f85f00c22a0 legacy=0x7f85f0157650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:55:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.880+0000 7f85f4e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85f0157d60 0x7f85f00acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.880+0000 7f85f4e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f85f0157d60 0x7f85f00acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54324/0 (socket says 172.21.15.67:54324) 2024-03-09T08:55:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.880+0000 7f85f4e00640 1 -- 172.21.15.67:0/4167556314 learned_addr learned my addr 172.21.15.67:0/4167556314 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.880+0000 7f85efe00640 1 --2- 172.21.15.67:0/4167556314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f00ac730 0x7f85f0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.880+0000 7f85f4e00640 1 -- 172.21.15.67:0/4167556314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f00ac730 msgr2=0x7f85f0071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.880+0000 7f85f4e00640 1 --2- 172.21.15.67:0/4167556314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f00ac730 0x7f85f0071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.880+0000 7f85f4e00640 1 -- 172.21.15.67:0/4167556314 >> v1:172.21.15.134:6789/0 conn(0x7f85f00c22a0 legacy=0x7f85f0157650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:55:04.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.880+0000 7f85f4e00640 1 -- 172.21.15.67:0/4167556314 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f00c2790 con 0x7f85f0157d60 2024-03-09T08:55:04.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.882+0000 7f85f4e00640 1 --2- 172.21.15.67:0/4167556314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85f0157d60 0x7f85f00acb10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009e50 tx=0x7f85e802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d187b19d0143fca server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:04.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.883+0000 7f85ef400640 1 -- 172.21.15.67:0/4167556314 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e802c9e0 con 0x7f85f0157d60 2024-03-09T08:55:04.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.883+0000 7f85ef400640 1 -- 172.21.15.67:0/4167556314 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e802cb80 con 0x7f85f0157d60 2024-03-09T08:55:04.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.883+0000 7f85ef400640 1 -- 172.21.15.67:0/4167556314 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80343f0 con 0x7f85f0157d60 2024-03-09T08:55:04.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.884+0000 7f85f6e00640 1 -- 172.21.15.67:0/4167556314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85f0157d60 msgr2=0x7f85f00acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:04.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.884+0000 7f85f6e00640 1 --2- 172.21.15.67:0/4167556314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85f0157d60 0x7f85f00acb10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009e50 tx=0x7f85e802be40 comp rx=0 tx=0).stop 2024-03-09T08:55:04.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.884+0000 7f85f6e00640 1 -- 172.21.15.67:0/4167556314 shutdown_connections 2024-03-09T08:55:04.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.884+0000 7f85f6e00640 1 --2- 172.21.15.67:0/4167556314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85f0157d60 0x7f85f00acb10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:04.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.884+0000 7f85f6e00640 1 --2- 172.21.15.67:0/4167556314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f00ac730 0x7f85f0071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:04.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.884+0000 7f85f6e00640 1 -- 172.21.15.67:0/4167556314 >> 172.21.15.67:0/4167556314 conn(0x7f85f00c7370 msgr2=0x7f85f00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:04.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.884+0000 7f85f6e00640 1 -- 172.21.15.67:0/4167556314 shutdown_connections 2024-03-09T08:55:04.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.884+0000 7f85f6e00640 1 -- 172.21.15.67:0/4167556314 wait complete. 2024-03-09T08:55:04.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.886+0000 7f85f6e00640 1 Processor -- start 2024-03-09T08:55:04.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.887+0000 7f85f6e00640 1 -- start start 2024-03-09T08:55:04.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85f6e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85f00ac730 0x7f85f00bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:04.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85f6e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f00c22a0 0x7f85f01eed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:04.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85f6e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f0157d60 0x7f85f01ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:04.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85f6e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0151610 con 0x7f85f00ac730 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85f6e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0070c00 con 0x7f85f00c22a0 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85f6e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0151790 con 0x7f85f0157d60 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85e7e00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f00c22a0 msgr2=0x7f85f01eed60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85e7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f00c22a0 0x7f85f01eed60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85f00ac730 0x7f85f00bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85f00ac730 0x7f85f00bdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54332/0 (socket says 172.21.15.67:54332) 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85efe00640 1 -- 172.21.15.67:0/775397272 learned_addr learned my addr 172.21.15.67:0/775397272 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.888+0000 7f85f4e00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f0157d60 0x7f85f01ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.889+0000 7f85efe00640 1 -- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f0157d60 msgr2=0x7f85f01ef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.889+0000 7f85efe00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f0157d60 0x7f85f01ef2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.889+0000 7f85efe00640 1 -- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f00c22a0 msgr2=0x7f85f01eed60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.889+0000 7f85efe00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f00c22a0 0x7f85f01eed60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:04.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.889+0000 7f85efe00640 1 -- 172.21.15.67:0/775397272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f01ef7e0 con 0x7f85f00ac730 2024-03-09T08:55:04.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.889+0000 7f85efe00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85f00ac730 0x7f85f00bdeb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f85e000e9b0 tx=0x7f85e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:04.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.889+0000 7f85ecc00640 1 -- 172.21.15.67:0/775397272 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0017070 con 0x7f85f00ac730 2024-03-09T08:55:04.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.890+0000 7f85ecc00640 1 -- 172.21.15.67:0/775397272 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e000c9a0 con 0x7f85f00ac730 2024-03-09T08:55:04.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.890+0000 7f85ecc00640 1 -- 172.21.15.67:0/775397272 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e000cd90 con 0x7f85f00ac730 2024-03-09T08:55:04.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.890+0000 7f85f6e00640 1 -- 172.21.15.67:0/775397272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f00bcc20 con 0x7f85f00ac730 2024-03-09T08:55:04.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.890+0000 7f85f6e00640 1 -- 172.21.15.67:0/775397272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f01ecb90 con 0x7f85f00ac730 2024-03-09T08:55:04.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.891+0000 7f85f6e00640 1 -- 172.21.15.67:0/775397272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f0075a00 con 0x7f85f00ac730 2024-03-09T08:55:04.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.893+0000 7f85ecc00640 1 -- 172.21.15.67:0/775397272 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f85e001a070 con 0x7f85f00ac730 2024-03-09T08:55:04.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.894+0000 7f85ecc00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85cc06b480 0x7f85cc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:04.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.894+0000 7f85ecc00640 1 -- 172.21.15.67:0/775397272 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f85e008a5e0 con 0x7f85f00ac730 2024-03-09T08:55:04.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.895+0000 7f85e7e00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85cc06b480 0x7f85cc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:04.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.896+0000 7f85e7e00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85cc06b480 0x7f85cc06d940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f85f014bd20 tx=0x7f85d8006d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:04.895 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 186 2024-03-09T08:55:04.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:04.900+0000 7f85ecc00640 1 -- 172.21.15.67:0/775397272 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f85e0059070 con 0x7f85f00ac730 2024-03-09T08:55:04.921 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 187 2024-03-09T08:55:04.957 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 188 2024-03-09T08:55:05.079 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 282 2024-03-09T08:55:05.107 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 283 2024-03-09T08:55:05.127 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 284 2024-03-09T08:55:05.145 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 285 2024-03-09T08:55:05.164 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 286 2024-03-09T08:55:05.183 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 287 2024-03-09T08:55:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.200+0000 7f85f6e00640 1 -- 172.21.15.67:0/775397272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f85f00c1d70 con 0x7f85f00ac730 2024-03-09T08:55:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.201+0000 7f85ecc00640 1 -- 172.21.15.67:0/775397272 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f85e0058970 con 0x7f85f00ac730 2024-03-09T08:55:05.198 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:55:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.203+0000 7f85f6e00640 1 -- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85cc06b480 msgr2=0x7f85cc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.203+0000 7f85f6e00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85cc06b480 0x7f85cc06d940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f85f014bd20 tx=0x7f85d8006d30 comp rx=0 tx=0).stop 2024-03-09T08:55:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.203+0000 7f85f6e00640 1 -- 172.21.15.67:0/775397272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85f00ac730 msgr2=0x7f85f00bdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.203+0000 7f85f6e00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85f00ac730 0x7f85f00bdeb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f85e000e9b0 tx=0x7f85e000ee70 comp rx=0 tx=0).stop 2024-03-09T08:55:05.202 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 288 2024-03-09T08:55:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.203+0000 7f85f6e00640 1 -- 172.21.15.67:0/775397272 shutdown_connections 2024-03-09T08:55:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.203+0000 7f85f6e00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85cc06b480 0x7f85cc06d940 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.203+0000 7f85f6e00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f0157d60 0x7f85f01ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.203+0000 7f85f6e00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f00c22a0 0x7f85f01eed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.203+0000 7f85f6e00640 1 --2- 172.21.15.67:0/775397272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85f00ac730 0x7f85f00bdeb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.203+0000 7f85f6e00640 1 -- 172.21.15.67:0/775397272 >> 172.21.15.67:0/775397272 conn(0x7f85f00c7370 msgr2=0x7f85f0072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.204+0000 7f85f6e00640 1 -- 172.21.15.67:0/775397272 shutdown_connections 2024-03-09T08:55:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:05.204+0000 7f85f6e00640 1 -- 172.21.15.67:0/775397272 wait complete. 2024-03-09T08:55:05.215 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 289 2024-03-09T08:55:05.253 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 189 2024-03-09T08:55:05.280 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 190 2024-03-09T08:55:05.307 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 191 2024-03-09T08:55:05.332 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 192 2024-03-09T08:55:05.608 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 193 2024-03-09T08:55:05.782 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 290 2024-03-09T08:55:05.898 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 194 2024-03-09T08:55:05.924 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 195 2024-03-09T08:55:05.949 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 196 2024-03-09T08:55:05.972 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 197 2024-03-09T08:55:05.998 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 198 2024-03-09T08:55:06.020 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 199 2024-03-09T08:55:06.041 INFO:tasks.workunit.client.0.smithi067.stdout:Take first snapshot .snap/test1 2024-03-09T08:55:06.185 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 291 2024-03-09T08:55:06.214 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 292 2024-03-09T08:55:06.241 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 293 2024-03-09T08:55:06.259 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 294 2024-03-09T08:55:06.281 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 295 2024-03-09T08:55:06.297 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 296 2024-03-09T08:55:06.317 INFO:tasks.workunit.client.0.smithi067.stdout:Create dir 200 to 299 ... 2024-03-09T08:55:06.321 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 200 2024-03-09T08:55:06.336 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 201 2024-03-09T08:55:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 202 2024-03-09T08:55:06.389 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 203 2024-03-09T08:55:06.412 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 204 2024-03-09T08:55:06.434 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 205 2024-03-09T08:55:06.455 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 206 2024-03-09T08:55:06.476 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 207 2024-03-09T08:55:06.496 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 208 2024-03-09T08:55:06.516 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 209 2024-03-09T08:55:06.536 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 210 2024-03-09T08:55:06.555 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 211 2024-03-09T08:55:06.573 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 212 2024-03-09T08:55:06.591 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 213 2024-03-09T08:55:06.609 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 214 2024-03-09T08:55:06.627 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 215 2024-03-09T08:55:06.643 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 216 2024-03-09T08:55:06.660 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 217 2024-03-09T08:55:06.806 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 297 2024-03-09T08:55:06.945 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 218 2024-03-09T08:55:06.968 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 219 2024-03-09T08:55:07.016 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 220 2024-03-09T08:55:07.039 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 221 2024-03-09T08:55:07.063 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 222 2024-03-09T08:55:07.083 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 223 2024-03-09T08:55:07.105 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 224 2024-03-09T08:55:07.127 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 225 2024-03-09T08:55:07.164 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 298 2024-03-09T08:55:07.188 INFO:tasks.workunit.client.1.smithi134.stdout: create dir 299 2024-03-09T08:55:07.211 INFO:tasks.workunit.client.1.smithi134.stdout:Create a snapshot in every first level dir ... 2024-03-09T08:55:07.404 INFO:tasks.workunit.client.1.smithi134.stdout: create 100/.snap/snap-subdir-test 2024-03-09T08:55:07.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 100/30 file after the snapshot 2024-03-09T08:55:07.471 INFO:tasks.workunit.client.1.smithi134.stdout: create 100/31 file after the snapshot 2024-03-09T08:55:07.471 INFO:tasks.workunit.client.1.smithi134.stdout: create 100/32 file after the snapshot 2024-03-09T08:55:07.472 INFO:tasks.workunit.client.1.smithi134.stdout: create 100/33 file after the snapshot 2024-03-09T08:55:07.472 INFO:tasks.workunit.client.1.smithi134.stdout: create 100/34 file after the snapshot 2024-03-09T08:55:07.472 INFO:tasks.workunit.client.1.smithi134.stdout: create 100/35 file after the snapshot 2024-03-09T08:55:07.473 INFO:tasks.workunit.client.1.smithi134.stdout: create 100/36 file after the snapshot 2024-03-09T08:55:07.473 INFO:tasks.workunit.client.1.smithi134.stdout: create 100/37 file after the snapshot 2024-03-09T08:55:07.473 INFO:tasks.workunit.client.1.smithi134.stdout: create 100/38 file after the snapshot 2024-03-09T08:55:07.473 INFO:tasks.workunit.client.1.smithi134.stdout: create 100/39 file after the snapshot 2024-03-09T08:55:07.473 INFO:tasks.workunit.client.1.smithi134.stdout: create 101/.snap/snap-subdir-test 2024-03-09T08:55:07.563 INFO:tasks.workunit.client.1.smithi134.stdout: create 101/30 file after the snapshot 2024-03-09T08:55:07.566 INFO:tasks.workunit.client.1.smithi134.stdout: create 101/31 file after the snapshot 2024-03-09T08:55:07.566 INFO:tasks.workunit.client.1.smithi134.stdout: create 101/32 file after the snapshot 2024-03-09T08:55:07.567 INFO:tasks.workunit.client.1.smithi134.stdout: create 101/33 file after the snapshot 2024-03-09T08:55:07.567 INFO:tasks.workunit.client.1.smithi134.stdout: create 101/34 file after the snapshot 2024-03-09T08:55:07.567 INFO:tasks.workunit.client.1.smithi134.stdout: create 101/35 file after the snapshot 2024-03-09T08:55:07.568 INFO:tasks.workunit.client.1.smithi134.stdout: create 101/36 file after the snapshot 2024-03-09T08:55:07.568 INFO:tasks.workunit.client.1.smithi134.stdout: create 101/37 file after the snapshot 2024-03-09T08:55:07.568 INFO:tasks.workunit.client.1.smithi134.stdout: create 101/38 file after the snapshot 2024-03-09T08:55:07.569 INFO:tasks.workunit.client.1.smithi134.stdout: create 101/39 file after the snapshot 2024-03-09T08:55:07.569 INFO:tasks.workunit.client.1.smithi134.stdout: create 102/.snap/snap-subdir-test 2024-03-09T08:55:07.665 INFO:tasks.workunit.client.1.smithi134.stdout: create 102/30 file after the snapshot 2024-03-09T08:55:07.669 INFO:tasks.workunit.client.1.smithi134.stdout: create 102/31 file after the snapshot 2024-03-09T08:55:07.669 INFO:tasks.workunit.client.1.smithi134.stdout: create 102/32 file after the snapshot 2024-03-09T08:55:07.670 INFO:tasks.workunit.client.1.smithi134.stdout: create 102/33 file after the snapshot 2024-03-09T08:55:07.670 INFO:tasks.workunit.client.1.smithi134.stdout: create 102/34 file after the snapshot 2024-03-09T08:55:07.670 INFO:tasks.workunit.client.1.smithi134.stdout: create 102/35 file after the snapshot 2024-03-09T08:55:07.671 INFO:tasks.workunit.client.1.smithi134.stdout: create 102/36 file after the snapshot 2024-03-09T08:55:07.671 INFO:tasks.workunit.client.1.smithi134.stdout: create 102/37 file after the snapshot 2024-03-09T08:55:07.671 INFO:tasks.workunit.client.1.smithi134.stdout: create 102/38 file after the snapshot 2024-03-09T08:55:07.672 INFO:tasks.workunit.client.1.smithi134.stdout: create 102/39 file after the snapshot 2024-03-09T08:55:07.672 INFO:tasks.workunit.client.1.smithi134.stdout: create 103/.snap/snap-subdir-test 2024-03-09T08:55:07.857 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 226 2024-03-09T08:55:07.916 INFO:tasks.workunit.client.1.smithi134.stdout: create 103/30 file after the snapshot 2024-03-09T08:55:07.919 INFO:tasks.workunit.client.1.smithi134.stdout: create 103/31 file after the snapshot 2024-03-09T08:55:07.920 INFO:tasks.workunit.client.1.smithi134.stdout: create 103/32 file after the snapshot 2024-03-09T08:55:07.920 INFO:tasks.workunit.client.1.smithi134.stdout: create 103/33 file after the snapshot 2024-03-09T08:55:07.920 INFO:tasks.workunit.client.1.smithi134.stdout: create 103/34 file after the snapshot 2024-03-09T08:55:07.921 INFO:tasks.workunit.client.1.smithi134.stdout: create 103/35 file after the snapshot 2024-03-09T08:55:07.921 INFO:tasks.workunit.client.1.smithi134.stdout: create 103/36 file after the snapshot 2024-03-09T08:55:07.922 INFO:tasks.workunit.client.1.smithi134.stdout: create 103/37 file after the snapshot 2024-03-09T08:55:07.922 INFO:tasks.workunit.client.1.smithi134.stdout: create 103/38 file after the snapshot 2024-03-09T08:55:07.923 INFO:tasks.workunit.client.1.smithi134.stdout: create 103/39 file after the snapshot 2024-03-09T08:55:07.924 INFO:tasks.workunit.client.1.smithi134.stdout: create 104/.snap/snap-subdir-test 2024-03-09T08:55:07.927 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 227 2024-03-09T08:55:07.951 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 228 2024-03-09T08:55:07.976 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 229 2024-03-09T08:55:07.997 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 230 2024-03-09T08:55:08.020 INFO:tasks.workunit.client.1.smithi134.stdout: create 104/30 file after the snapshot 2024-03-09T08:55:08.022 INFO:tasks.workunit.client.1.smithi134.stdout: create 104/31 file after the snapshot 2024-03-09T08:55:08.023 INFO:tasks.workunit.client.1.smithi134.stdout: create 104/32 file after the snapshot 2024-03-09T08:55:08.023 INFO:tasks.workunit.client.1.smithi134.stdout: create 104/33 file after the snapshot 2024-03-09T08:55:08.024 INFO:tasks.workunit.client.1.smithi134.stdout: create 104/34 file after the snapshot 2024-03-09T08:55:08.024 INFO:tasks.workunit.client.1.smithi134.stdout: create 104/35 file after the snapshot 2024-03-09T08:55:08.024 INFO:tasks.workunit.client.1.smithi134.stdout: create 104/36 file after the snapshot 2024-03-09T08:55:08.025 INFO:tasks.workunit.client.1.smithi134.stdout: create 104/37 file after the snapshot 2024-03-09T08:55:08.025 INFO:tasks.workunit.client.1.smithi134.stdout: create 104/38 file after the snapshot 2024-03-09T08:55:08.026 INFO:tasks.workunit.client.1.smithi134.stdout: create 104/39 file after the snapshot 2024-03-09T08:55:08.026 INFO:tasks.workunit.client.1.smithi134.stdout: create 105/.snap/snap-subdir-test 2024-03-09T08:55:08.114 INFO:tasks.workunit.client.1.smithi134.stdout: create 105/30 file after the snapshot 2024-03-09T08:55:08.117 INFO:tasks.workunit.client.1.smithi134.stdout: create 105/31 file after the snapshot 2024-03-09T08:55:08.118 INFO:tasks.workunit.client.1.smithi134.stdout: create 105/32 file after the snapshot 2024-03-09T08:55:08.118 INFO:tasks.workunit.client.1.smithi134.stdout: create 105/33 file after the snapshot 2024-03-09T08:55:08.118 INFO:tasks.workunit.client.1.smithi134.stdout: create 105/34 file after the snapshot 2024-03-09T08:55:08.118 INFO:tasks.workunit.client.1.smithi134.stdout: create 105/35 file after the snapshot 2024-03-09T08:55:08.118 INFO:tasks.workunit.client.1.smithi134.stdout: create 105/36 file after the snapshot 2024-03-09T08:55:08.118 INFO:tasks.workunit.client.1.smithi134.stdout: create 105/37 file after the snapshot 2024-03-09T08:55:08.118 INFO:tasks.workunit.client.1.smithi134.stdout: create 105/38 file after the snapshot 2024-03-09T08:55:08.118 INFO:tasks.workunit.client.1.smithi134.stdout: create 105/39 file after the snapshot 2024-03-09T08:55:08.118 INFO:tasks.workunit.client.1.smithi134.stdout: create 106/.snap/snap-subdir-test 2024-03-09T08:55:08.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:55:08.223 INFO:tasks.workunit.client.1.smithi134.stdout: create 106/30 file after the snapshot 2024-03-09T08:55:08.227 INFO:tasks.workunit.client.1.smithi134.stdout: create 106/31 file after the snapshot 2024-03-09T08:55:08.227 INFO:tasks.workunit.client.1.smithi134.stdout: create 106/32 file after the snapshot 2024-03-09T08:55:08.227 INFO:tasks.workunit.client.1.smithi134.stdout: create 106/33 file after the snapshot 2024-03-09T08:55:08.228 INFO:tasks.workunit.client.1.smithi134.stdout: create 106/34 file after the snapshot 2024-03-09T08:55:08.228 INFO:tasks.workunit.client.1.smithi134.stdout: create 106/35 file after the snapshot 2024-03-09T08:55:08.228 INFO:tasks.workunit.client.1.smithi134.stdout: create 106/36 file after the snapshot 2024-03-09T08:55:08.228 INFO:tasks.workunit.client.1.smithi134.stdout: create 106/37 file after the snapshot 2024-03-09T08:55:08.228 INFO:tasks.workunit.client.1.smithi134.stdout: create 106/38 file after the snapshot 2024-03-09T08:55:08.228 INFO:tasks.workunit.client.1.smithi134.stdout: create 106/39 file after the snapshot 2024-03-09T08:55:08.228 INFO:tasks.workunit.client.1.smithi134.stdout: create 107/.snap/snap-subdir-test 2024-03-09T08:55:08.303 INFO:tasks.workunit.client.1.smithi134.stdout: create 107/30 file after the snapshot 2024-03-09T08:55:08.305 INFO:tasks.workunit.client.1.smithi134.stdout: create 107/31 file after the snapshot 2024-03-09T08:55:08.305 INFO:tasks.workunit.client.1.smithi134.stdout: create 107/32 file after the snapshot 2024-03-09T08:55:08.305 INFO:tasks.workunit.client.1.smithi134.stdout: create 107/33 file after the snapshot 2024-03-09T08:55:08.306 INFO:tasks.workunit.client.1.smithi134.stdout: create 107/34 file after the snapshot 2024-03-09T08:55:08.306 INFO:tasks.workunit.client.1.smithi134.stdout: create 107/35 file after the snapshot 2024-03-09T08:55:08.306 INFO:tasks.workunit.client.1.smithi134.stdout: create 107/36 file after the snapshot 2024-03-09T08:55:08.306 INFO:tasks.workunit.client.1.smithi134.stdout: create 107/37 file after the snapshot 2024-03-09T08:55:08.306 INFO:tasks.workunit.client.1.smithi134.stdout: create 107/38 file after the snapshot 2024-03-09T08:55:08.306 INFO:tasks.workunit.client.1.smithi134.stdout: create 107/39 file after the snapshot 2024-03-09T08:55:08.306 INFO:tasks.workunit.client.1.smithi134.stdout: create 108/.snap/snap-subdir-test 2024-03-09T08:55:08.418 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 231 2024-03-09T08:55:08.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.429+0000 7ff301600640 1 Processor -- start 2024-03-09T08:55:08.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.429+0000 7ff301600640 1 -- start start 2024-03-09T08:55:08.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.431+0000 7ff301600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0c1f70 0x7ff2fc150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:08.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.431+0000 7ff301600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff2fc0710a0 0x7ff2fc151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:08.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.431+0000 7ff301600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2fc0c81e0 con 0x7ff2fc0c27a0 2024-03-09T08:55:08.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.431+0000 7ff301600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2fc0c32b0 con 0x7ff2fc0710a0 2024-03-09T08:55:08.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.431+0000 7ff301600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0c2fd0 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.431+0000 7ff2fb400640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7ff2fc0710a0 msgr2=0x7ff2fc151190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:08.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.431+0000 7ff2fb400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff2fc0710a0 0x7ff2fc151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:08.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.431+0000 7ff300c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff2fc0c27a0 0x7ff2fc153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44272/0 (socket says 172.21.15.67:44272) 2024-03-09T08:55:08.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.431+0000 7ff300c00640 1 -- 172.21.15.67:0/3671421528 learned_addr learned my addr 172.21.15.67:0/3671421528 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:08.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.431+0000 7ff2fbe00640 1 --2- 172.21.15.67:0/3671421528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0c1f70 0x7ff2fc150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:08.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.432+0000 7ff2fbe00640 1 -- 172.21.15.67:0/3671421528 >> v2:172.21.15.134:3300/0 conn(0x7ff2fc0710a0 msgr2=0x7ff2fc151190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:55:08.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.432+0000 7ff2fbe00640 1 --2- 172.21.15.67:0/3671421528 >> v2:172.21.15.134:3300/0 conn(0x7ff2fc0710a0 0x7ff2fc151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:08.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.432+0000 7ff2fbe00640 1 -- 172.21.15.67:0/3671421528 >> v1:172.21.15.67:6789/0 conn(0x7ff2fc0c27a0 legacy=0x7ff2fc153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:08.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.432+0000 7ff2fbe00640 1 -- 172.21.15.67:0/3671421528 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc0c3700 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.434+0000 7ff2fbe00640 1 --2- 172.21.15.67:0/3671421528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0c1f70 0x7ff2fc150c50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0097a0 tx=0x7ff2ec02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=557adca4946d5354 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.434+0000 7ff2faa00640 1 -- 172.21.15.67:0/3671421528 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0315a0 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.434+0000 7ff2faa00640 1 -- 172.21.15.67:0/3671421528 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec037070 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.434+0000 7ff2faa00640 1 -- 172.21.15.67:0/3671421528 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec031960 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.435+0000 7ff301600640 1 -- 172.21.15.67:0/3671421528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0c1f70 msgr2=0x7ff2fc150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.435+0000 7ff301600640 1 --2- 172.21.15.67:0/3671421528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0c1f70 0x7ff2fc150c50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0097a0 tx=0x7ff2ec02b840 comp rx=0 tx=0).stop 2024-03-09T08:55:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.435+0000 7ff301600640 1 -- 172.21.15.67:0/3671421528 shutdown_connections 2024-03-09T08:55:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.435+0000 7ff301600640 1 --2- 172.21.15.67:0/3671421528 >> v2:172.21.15.134:3300/0 conn(0x7ff2fc0710a0 0x7ff2fc151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.435+0000 7ff301600640 1 --2- 172.21.15.67:0/3671421528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0c1f70 0x7ff2fc150c50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.435+0000 7ff301600640 1 -- 172.21.15.67:0/3671421528 >> 172.21.15.67:0/3671421528 conn(0x7ff2fc0c7300 msgr2=0x7ff2fc0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.435+0000 7ff301600640 1 -- 172.21.15.67:0/3671421528 shutdown_connections 2024-03-09T08:55:08.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.435+0000 7ff301600640 1 -- 172.21.15.67:0/3671421528 wait complete. 2024-03-09T08:55:08.433 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 232 2024-03-09T08:55:08.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.437+0000 7ff301600640 1 Processor -- start 2024-03-09T08:55:08.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.437+0000 7ff301600640 1 -- start start 2024-03-09T08:55:08.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff301600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0710a0 0x7ff2fc0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff301600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2fc0c1f70 0x7ff2fc145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff301600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2fc0c27a0 0x7ff2fc1457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff301600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc070d50 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff301600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc070830 con 0x7ff2fc0c27a0 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff301600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc070ed0 con 0x7ff2fc0710a0 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff300c00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2fc0c27a0 msgr2=0x7ff2fc1457b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff300c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2fc0c27a0 0x7ff2fc1457b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff2fb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2fc0c1f70 0x7ff2fc145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff2fb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2fc0c1f70 0x7ff2fc145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54340/0 (socket says 172.21.15.67:54340) 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff2fb400640 1 -- 172.21.15.67:0/3244756179 learned_addr learned my addr 172.21.15.67:0/3244756179 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.438+0000 7ff2fbe00640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0710a0 0x7ff2fc0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.439+0000 7ff2fb400640 1 -- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0710a0 msgr2=0x7ff2fc0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.439+0000 7ff2fb400640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0710a0 0x7ff2fc0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:08.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.439+0000 7ff2fb400640 1 -- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2fc0c27a0 msgr2=0x7ff2fc1457b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:55:08.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.439+0000 7ff2fb400640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2fc0c27a0 0x7ff2fc1457b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:08.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.439+0000 7ff2fb400640 1 -- 172.21.15.67:0/3244756179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc145cf0 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.439+0000 7ff2fbe00640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0710a0 0x7ff2fc0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:55:08.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.439+0000 7ff2fb400640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2fc0c1f70 0x7ff2fc145270 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800b550 tx=0x7ff2e800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.439+0000 7ff2f3e00640 1 -- 172.21.15.67:0/3244756179 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8013020 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.440+0000 7ff2f3e00640 1 -- 172.21.15.67:0/3244756179 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e8003de0 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.440+0000 7ff2f3e00640 1 -- 172.21.15.67:0/3244756179 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80052b0 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.440+0000 7ff301600640 1 -- 172.21.15.67:0/3244756179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc1eb6b0 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.440+0000 7ff301600640 1 -- 172.21.15.67:0/3244756179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc1ebc70 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.441+0000 7ff301600640 1 -- 172.21.15.67:0/3244756179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc04f3c0 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.442+0000 7ff2f3e00640 1 -- 172.21.15.67:0/3244756179 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff2e8016020 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.443+0000 7ff2f3e00640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2dc06b480 0x7ff2dc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:08.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.443+0000 7ff2f3e00640 1 -- 172.21.15.67:0/3244756179 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6693+0+0 (secure 0 0 0) 0x7ff2e808a390 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.444+0000 7ff2fbe00640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2dc06b480 0x7ff2dc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:08.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.444+0000 7ff2fbe00640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2dc06b480 0x7ff2dc06d940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec036000 tx=0x7ff2ec03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:08.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.447+0000 7ff2f3e00640 1 -- 172.21.15.67:0/3244756179 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff2e801d020 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.445 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 233 2024-03-09T08:55:08.460 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 234 2024-03-09T08:55:08.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.730+0000 7ff301600640 1 -- 172.21.15.67:0/3244756179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff2fc14c680 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.731+0000 7ff2f3e00640 1 -- 172.21.15.67:0/3244756179 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7ff2e8058d90 con 0x7ff2fc0c1f70 2024-03-09T08:55:08.729 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:55:08.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 -- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2dc06b480 msgr2=0x7ff2dc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:08.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2dc06b480 0x7ff2dc06d940 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec036000 tx=0x7ff2ec03d040 comp rx=0 tx=0).stop 2024-03-09T08:55:08.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 -- 172.21.15.67:0/3244756179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2fc0c1f70 msgr2=0x7ff2fc145270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:08.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2fc0c1f70 0x7ff2fc145270 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800b550 tx=0x7ff2e800ba10 comp rx=0 tx=0).stop 2024-03-09T08:55:08.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 -- 172.21.15.67:0/3244756179 shutdown_connections 2024-03-09T08:55:08.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2dc06b480 0x7ff2dc06d940 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:08.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2fc0c27a0 0x7ff2fc1457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:08.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2fc0c1f70 0x7ff2fc145270 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:08.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 --2- 172.21.15.67:0/3244756179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2fc0710a0 0x7ff2fc0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:08.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 -- 172.21.15.67:0/3244756179 >> 172.21.15.67:0/3244756179 conn(0x7ff2fc0c7300 msgr2=0x7ff2fc14d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:08.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 -- 172.21.15.67:0/3244756179 shutdown_connections 2024-03-09T08:55:08.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:08.735+0000 7ff301600640 1 -- 172.21.15.67:0/3244756179 wait complete. 2024-03-09T08:55:08.880 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 235 2024-03-09T08:55:08.912 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 236 2024-03-09T08:55:08.935 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 237 2024-03-09T08:55:08.953 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 238 2024-03-09T08:55:08.971 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 239 2024-03-09T08:55:08.988 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 240 2024-03-09T08:55:09.009 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 241 2024-03-09T08:55:09.035 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 242 2024-03-09T08:55:09.059 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 243 2024-03-09T08:55:09.082 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 244 2024-03-09T08:55:09.104 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 245 2024-03-09T08:55:09.126 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 246 2024-03-09T08:55:09.145 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 247 2024-03-09T08:55:09.163 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 248 2024-03-09T08:55:09.181 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 249 2024-03-09T08:55:09.302 INFO:tasks.workunit.client.1.smithi134.stdout: create 108/30 file after the snapshot 2024-03-09T08:55:09.305 INFO:tasks.workunit.client.1.smithi134.stdout: create 108/31 file after the snapshot 2024-03-09T08:55:09.305 INFO:tasks.workunit.client.1.smithi134.stdout: create 108/32 file after the snapshot 2024-03-09T08:55:09.306 INFO:tasks.workunit.client.1.smithi134.stdout: create 108/33 file after the snapshot 2024-03-09T08:55:09.307 INFO:tasks.workunit.client.1.smithi134.stdout: create 108/34 file after the snapshot 2024-03-09T08:55:09.307 INFO:tasks.workunit.client.1.smithi134.stdout: create 108/35 file after the snapshot 2024-03-09T08:55:09.308 INFO:tasks.workunit.client.1.smithi134.stdout: create 108/36 file after the snapshot 2024-03-09T08:55:09.308 INFO:tasks.workunit.client.1.smithi134.stdout: create 108/37 file after the snapshot 2024-03-09T08:55:09.308 INFO:tasks.workunit.client.1.smithi134.stdout: create 108/38 file after the snapshot 2024-03-09T08:55:09.309 INFO:tasks.workunit.client.1.smithi134.stdout: create 108/39 file after the snapshot 2024-03-09T08:55:09.310 INFO:tasks.workunit.client.1.smithi134.stdout: create 109/.snap/snap-subdir-test 2024-03-09T08:55:09.409 INFO:tasks.workunit.client.1.smithi134.stdout: create 109/30 file after the snapshot 2024-03-09T08:55:09.412 INFO:tasks.workunit.client.1.smithi134.stdout: create 109/31 file after the snapshot 2024-03-09T08:55:09.413 INFO:tasks.workunit.client.1.smithi134.stdout: create 109/32 file after the snapshot 2024-03-09T08:55:09.413 INFO:tasks.workunit.client.1.smithi134.stdout: create 109/33 file after the snapshot 2024-03-09T08:55:09.413 INFO:tasks.workunit.client.1.smithi134.stdout: create 109/34 file after the snapshot 2024-03-09T08:55:09.413 INFO:tasks.workunit.client.1.smithi134.stdout: create 109/35 file after the snapshot 2024-03-09T08:55:09.414 INFO:tasks.workunit.client.1.smithi134.stdout: create 109/36 file after the snapshot 2024-03-09T08:55:09.414 INFO:tasks.workunit.client.1.smithi134.stdout: create 109/37 file after the snapshot 2024-03-09T08:55:09.414 INFO:tasks.workunit.client.1.smithi134.stdout: create 109/38 file after the snapshot 2024-03-09T08:55:09.414 INFO:tasks.workunit.client.1.smithi134.stdout: create 109/39 file after the snapshot 2024-03-09T08:55:09.415 INFO:tasks.workunit.client.1.smithi134.stdout: create 110/.snap/snap-subdir-test 2024-03-09T08:55:09.499 INFO:tasks.workunit.client.1.smithi134.stdout: create 110/30 file after the snapshot 2024-03-09T08:55:09.502 INFO:tasks.workunit.client.1.smithi134.stdout: create 110/31 file after the snapshot 2024-03-09T08:55:09.503 INFO:tasks.workunit.client.1.smithi134.stdout: create 110/32 file after the snapshot 2024-03-09T08:55:09.503 INFO:tasks.workunit.client.1.smithi134.stdout: create 110/33 file after the snapshot 2024-03-09T08:55:09.503 INFO:tasks.workunit.client.1.smithi134.stdout: create 110/34 file after the snapshot 2024-03-09T08:55:09.504 INFO:tasks.workunit.client.1.smithi134.stdout: create 110/35 file after the snapshot 2024-03-09T08:55:09.504 INFO:tasks.workunit.client.1.smithi134.stdout: create 110/36 file after the snapshot 2024-03-09T08:55:09.504 INFO:tasks.workunit.client.1.smithi134.stdout: create 110/37 file after the snapshot 2024-03-09T08:55:09.504 INFO:tasks.workunit.client.1.smithi134.stdout: create 110/38 file after the snapshot 2024-03-09T08:55:09.504 INFO:tasks.workunit.client.1.smithi134.stdout: create 110/39 file after the snapshot 2024-03-09T08:55:09.505 INFO:tasks.workunit.client.1.smithi134.stdout: create 111/.snap/snap-subdir-test 2024-03-09T08:55:09.910 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 250 2024-03-09T08:55:10.015 INFO:tasks.workunit.client.1.smithi134.stdout: create 111/30 file after the snapshot 2024-03-09T08:55:10.032 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 251 2024-03-09T08:55:10.207 INFO:tasks.workunit.client.1.smithi134.stdout: create 111/31 file after the snapshot 2024-03-09T08:55:10.207 INFO:tasks.workunit.client.1.smithi134.stdout: create 111/32 file after the snapshot 2024-03-09T08:55:10.207 INFO:tasks.workunit.client.1.smithi134.stdout: create 111/33 file after the snapshot 2024-03-09T08:55:10.207 INFO:tasks.workunit.client.1.smithi134.stdout: create 111/34 file after the snapshot 2024-03-09T08:55:10.207 INFO:tasks.workunit.client.1.smithi134.stdout: create 111/35 file after the snapshot 2024-03-09T08:55:10.207 INFO:tasks.workunit.client.1.smithi134.stdout: create 111/36 file after the snapshot 2024-03-09T08:55:10.208 INFO:tasks.workunit.client.1.smithi134.stdout: create 111/37 file after the snapshot 2024-03-09T08:55:10.208 INFO:tasks.workunit.client.1.smithi134.stdout: create 111/38 file after the snapshot 2024-03-09T08:55:10.208 INFO:tasks.workunit.client.1.smithi134.stdout: create 111/39 file after the snapshot 2024-03-09T08:55:10.208 INFO:tasks.workunit.client.1.smithi134.stdout: create 112/.snap/snap-subdir-test 2024-03-09T08:55:10.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:55:10.487 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:55:10.701 INFO:tasks.workunit.client.1.smithi134.stdout: create 112/30 file after the snapshot 2024-03-09T08:55:10.704 INFO:tasks.workunit.client.1.smithi134.stdout: create 112/31 file after the snapshot 2024-03-09T08:55:10.704 INFO:tasks.workunit.client.1.smithi134.stdout: create 112/32 file after the snapshot 2024-03-09T08:55:10.705 INFO:tasks.workunit.client.1.smithi134.stdout: create 112/33 file after the snapshot 2024-03-09T08:55:10.705 INFO:tasks.workunit.client.1.smithi134.stdout: create 112/34 file after the snapshot 2024-03-09T08:55:10.705 INFO:tasks.workunit.client.1.smithi134.stdout: create 112/35 file after the snapshot 2024-03-09T08:55:10.706 INFO:tasks.workunit.client.1.smithi134.stdout: create 112/36 file after the snapshot 2024-03-09T08:55:10.706 INFO:tasks.workunit.client.1.smithi134.stdout: create 112/37 file after the snapshot 2024-03-09T08:55:10.706 INFO:tasks.workunit.client.1.smithi134.stdout: create 112/38 file after the snapshot 2024-03-09T08:55:10.707 INFO:tasks.workunit.client.1.smithi134.stdout: create 112/39 file after the snapshot 2024-03-09T08:55:10.707 INFO:tasks.workunit.client.1.smithi134.stdout: create 113/.snap/snap-subdir-test 2024-03-09T08:55:10.778 INFO:tasks.workunit.client.1.smithi134.stdout: create 113/30 file after the snapshot 2024-03-09T08:55:10.781 INFO:tasks.workunit.client.1.smithi134.stdout: create 113/31 file after the snapshot 2024-03-09T08:55:10.782 INFO:tasks.workunit.client.1.smithi134.stdout: create 113/32 file after the snapshot 2024-03-09T08:55:10.782 INFO:tasks.workunit.client.1.smithi134.stdout: create 113/33 file after the snapshot 2024-03-09T08:55:10.782 INFO:tasks.workunit.client.1.smithi134.stdout: create 113/34 file after the snapshot 2024-03-09T08:55:10.782 INFO:tasks.workunit.client.1.smithi134.stdout: create 113/35 file after the snapshot 2024-03-09T08:55:10.783 INFO:tasks.workunit.client.1.smithi134.stdout: create 113/36 file after the snapshot 2024-03-09T08:55:10.783 INFO:tasks.workunit.client.1.smithi134.stdout: create 113/37 file after the snapshot 2024-03-09T08:55:10.783 INFO:tasks.workunit.client.1.smithi134.stdout: create 113/38 file after the snapshot 2024-03-09T08:55:10.783 INFO:tasks.workunit.client.1.smithi134.stdout: create 113/39 file after the snapshot 2024-03-09T08:55:10.784 INFO:tasks.workunit.client.1.smithi134.stdout: create 114/.snap/snap-subdir-test 2024-03-09T08:55:11.465 INFO:tasks.workunit.client.1.smithi134.stdout: create 114/30 file after the snapshot 2024-03-09T08:55:11.468 INFO:tasks.workunit.client.1.smithi134.stdout: create 114/31 file after the snapshot 2024-03-09T08:55:11.468 INFO:tasks.workunit.client.1.smithi134.stdout: create 114/32 file after the snapshot 2024-03-09T08:55:11.468 INFO:tasks.workunit.client.1.smithi134.stdout: create 114/33 file after the snapshot 2024-03-09T08:55:11.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 114/34 file after the snapshot 2024-03-09T08:55:11.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 114/35 file after the snapshot 2024-03-09T08:55:11.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 114/36 file after the snapshot 2024-03-09T08:55:11.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 114/37 file after the snapshot 2024-03-09T08:55:11.470 INFO:tasks.workunit.client.1.smithi134.stdout: create 114/38 file after the snapshot 2024-03-09T08:55:11.470 INFO:tasks.workunit.client.1.smithi134.stdout: create 114/39 file after the snapshot 2024-03-09T08:55:11.470 INFO:tasks.workunit.client.1.smithi134.stdout: create 115/.snap/snap-subdir-test 2024-03-09T08:55:11.548 INFO:tasks.workunit.client.1.smithi134.stdout: create 115/30 file after the snapshot 2024-03-09T08:55:11.551 INFO:tasks.workunit.client.1.smithi134.stdout: create 115/31 file after the snapshot 2024-03-09T08:55:11.551 INFO:tasks.workunit.client.1.smithi134.stdout: create 115/32 file after the snapshot 2024-03-09T08:55:11.551 INFO:tasks.workunit.client.1.smithi134.stdout: create 115/33 file after the snapshot 2024-03-09T08:55:11.552 INFO:tasks.workunit.client.1.smithi134.stdout: create 115/34 file after the snapshot 2024-03-09T08:55:11.552 INFO:tasks.workunit.client.1.smithi134.stdout: create 115/35 file after the snapshot 2024-03-09T08:55:11.552 INFO:tasks.workunit.client.1.smithi134.stdout: create 115/36 file after the snapshot 2024-03-09T08:55:11.552 INFO:tasks.workunit.client.1.smithi134.stdout: create 115/37 file after the snapshot 2024-03-09T08:55:11.553 INFO:tasks.workunit.client.1.smithi134.stdout: create 115/38 file after the snapshot 2024-03-09T08:55:11.553 INFO:tasks.workunit.client.1.smithi134.stdout: create 115/39 file after the snapshot 2024-03-09T08:55:11.553 INFO:tasks.workunit.client.1.smithi134.stdout: create 116/.snap/snap-subdir-test 2024-03-09T08:55:11.634 INFO:tasks.workunit.client.1.smithi134.stdout: create 116/30 file after the snapshot 2024-03-09T08:55:11.636 INFO:tasks.workunit.client.1.smithi134.stdout: create 116/31 file after the snapshot 2024-03-09T08:55:11.637 INFO:tasks.workunit.client.1.smithi134.stdout: create 116/32 file after the snapshot 2024-03-09T08:55:11.637 INFO:tasks.workunit.client.1.smithi134.stdout: create 116/33 file after the snapshot 2024-03-09T08:55:11.637 INFO:tasks.workunit.client.1.smithi134.stdout: create 116/34 file after the snapshot 2024-03-09T08:55:11.638 INFO:tasks.workunit.client.1.smithi134.stdout: create 116/35 file after the snapshot 2024-03-09T08:55:11.638 INFO:tasks.workunit.client.1.smithi134.stdout: create 116/36 file after the snapshot 2024-03-09T08:55:11.638 INFO:tasks.workunit.client.1.smithi134.stdout: create 116/37 file after the snapshot 2024-03-09T08:55:11.639 INFO:tasks.workunit.client.1.smithi134.stdout: create 116/38 file after the snapshot 2024-03-09T08:55:11.639 INFO:tasks.workunit.client.1.smithi134.stdout: create 116/39 file after the snapshot 2024-03-09T08:55:11.639 INFO:tasks.workunit.client.1.smithi134.stdout: create 117/.snap/snap-subdir-test 2024-03-09T08:55:11.743 INFO:tasks.workunit.client.1.smithi134.stdout: create 117/30 file after the snapshot 2024-03-09T08:55:11.745 INFO:tasks.workunit.client.1.smithi134.stdout: create 117/31 file after the snapshot 2024-03-09T08:55:11.746 INFO:tasks.workunit.client.1.smithi134.stdout: create 117/32 file after the snapshot 2024-03-09T08:55:11.746 INFO:tasks.workunit.client.1.smithi134.stdout: create 117/33 file after the snapshot 2024-03-09T08:55:11.747 INFO:tasks.workunit.client.1.smithi134.stdout: create 117/34 file after the snapshot 2024-03-09T08:55:11.747 INFO:tasks.workunit.client.1.smithi134.stdout: create 117/35 file after the snapshot 2024-03-09T08:55:11.747 INFO:tasks.workunit.client.1.smithi134.stdout: create 117/36 file after the snapshot 2024-03-09T08:55:11.748 INFO:tasks.workunit.client.1.smithi134.stdout: create 117/37 file after the snapshot 2024-03-09T08:55:11.748 INFO:tasks.workunit.client.1.smithi134.stdout: create 117/38 file after the snapshot 2024-03-09T08:55:11.748 INFO:tasks.workunit.client.1.smithi134.stdout: create 117/39 file after the snapshot 2024-03-09T08:55:11.750 INFO:tasks.workunit.client.1.smithi134.stdout: create 118/.snap/snap-subdir-test 2024-03-09T08:55:11.752 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:55:11.817 INFO:tasks.workunit.client.1.smithi134.stdout: create 118/30 file after the snapshot 2024-03-09T08:55:11.818 INFO:tasks.workunit.client.1.smithi134.stdout: create 118/31 file after the snapshot 2024-03-09T08:55:11.819 INFO:tasks.workunit.client.1.smithi134.stdout: create 118/32 file after the snapshot 2024-03-09T08:55:11.819 INFO:tasks.workunit.client.1.smithi134.stdout: create 118/33 file after the snapshot 2024-03-09T08:55:11.820 INFO:tasks.workunit.client.1.smithi134.stdout: create 118/34 file after the snapshot 2024-03-09T08:55:11.820 INFO:tasks.workunit.client.1.smithi134.stdout: create 118/35 file after the snapshot 2024-03-09T08:55:11.820 INFO:tasks.workunit.client.1.smithi134.stdout: create 118/36 file after the snapshot 2024-03-09T08:55:11.820 INFO:tasks.workunit.client.1.smithi134.stdout: create 118/37 file after the snapshot 2024-03-09T08:55:11.820 INFO:tasks.workunit.client.1.smithi134.stdout: create 118/38 file after the snapshot 2024-03-09T08:55:11.821 INFO:tasks.workunit.client.1.smithi134.stdout: create 118/39 file after the snapshot 2024-03-09T08:55:11.821 INFO:tasks.workunit.client.1.smithi134.stdout: create 119/.snap/snap-subdir-test 2024-03-09T08:55:11.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 119/30 file after the snapshot 2024-03-09T08:55:11.915 INFO:tasks.workunit.client.1.smithi134.stdout: create 119/31 file after the snapshot 2024-03-09T08:55:11.915 INFO:tasks.workunit.client.1.smithi134.stdout: create 119/32 file after the snapshot 2024-03-09T08:55:11.915 INFO:tasks.workunit.client.1.smithi134.stdout: create 119/33 file after the snapshot 2024-03-09T08:55:11.916 INFO:tasks.workunit.client.1.smithi134.stdout: create 119/34 file after the snapshot 2024-03-09T08:55:11.916 INFO:tasks.workunit.client.1.smithi134.stdout: create 119/35 file after the snapshot 2024-03-09T08:55:11.916 INFO:tasks.workunit.client.1.smithi134.stdout: create 119/36 file after the snapshot 2024-03-09T08:55:11.916 INFO:tasks.workunit.client.1.smithi134.stdout: create 119/37 file after the snapshot 2024-03-09T08:55:11.917 INFO:tasks.workunit.client.1.smithi134.stdout: create 119/38 file after the snapshot 2024-03-09T08:55:11.917 INFO:tasks.workunit.client.1.smithi134.stdout: create 119/39 file after the snapshot 2024-03-09T08:55:11.917 INFO:tasks.workunit.client.1.smithi134.stdout: create 120/.snap/snap-subdir-test 2024-03-09T08:55:11.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.969+0000 7f642e400640 1 Processor -- start 2024-03-09T08:55:11.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.969+0000 7f642e400640 1 -- start start 2024-03-09T08:55:11.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.970+0000 7f642e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:11.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f642e400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f642e400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64280c81e0 con 0x7f64280710a0 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f642e400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64280c32b0 con 0x7f6428154080 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f642e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64280c2fd0 con 0x7f64280c1f70 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f6427e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f64280710a0 0x7f6428153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55802/0 (socket says 172.21.15.67:55802) 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f6427e00640 1 -- 172.21.15.67:0/227509221 learned_addr learned my addr 172.21.15.67:0/227509221 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 msgr2=0x7f6428151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 msgr2=0x7f6428151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.971+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.972+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 msgr2=0x7f6428150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:55:11.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.972+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 msgr2=0x7f6428150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:55:11.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.972+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:55:11.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:11.972+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:11.991 INFO:tasks.workunit.client.1.smithi134.stdout: create 120/30 file after the snapshot 2024-03-09T08:55:11.993 INFO:tasks.workunit.client.1.smithi134.stdout: create 120/31 file after the snapshot 2024-03-09T08:55:11.994 INFO:tasks.workunit.client.1.smithi134.stdout: create 120/32 file after the snapshot 2024-03-09T08:55:11.994 INFO:tasks.workunit.client.1.smithi134.stdout: create 120/33 file after the snapshot 2024-03-09T08:55:11.994 INFO:tasks.workunit.client.1.smithi134.stdout: create 120/34 file after the snapshot 2024-03-09T08:55:11.995 INFO:tasks.workunit.client.1.smithi134.stdout: create 120/35 file after the snapshot 2024-03-09T08:55:11.995 INFO:tasks.workunit.client.1.smithi134.stdout: create 120/36 file after the snapshot 2024-03-09T08:55:11.995 INFO:tasks.workunit.client.1.smithi134.stdout: create 120/37 file after the snapshot 2024-03-09T08:55:11.995 INFO:tasks.workunit.client.1.smithi134.stdout: create 120/38 file after the snapshot 2024-03-09T08:55:11.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 120/39 file after the snapshot 2024-03-09T08:55:11.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 121/.snap/snap-subdir-test 2024-03-09T08:55:12.065 INFO:tasks.workunit.client.1.smithi134.stdout: create 121/30 file after the snapshot 2024-03-09T08:55:12.067 INFO:tasks.workunit.client.1.smithi134.stdout: create 121/31 file after the snapshot 2024-03-09T08:55:12.067 INFO:tasks.workunit.client.1.smithi134.stdout: create 121/32 file after the snapshot 2024-03-09T08:55:12.068 INFO:tasks.workunit.client.1.smithi134.stdout: create 121/33 file after the snapshot 2024-03-09T08:55:12.069 INFO:tasks.workunit.client.1.smithi134.stdout: create 121/34 file after the snapshot 2024-03-09T08:55:12.069 INFO:tasks.workunit.client.1.smithi134.stdout: create 121/35 file after the snapshot 2024-03-09T08:55:12.069 INFO:tasks.workunit.client.1.smithi134.stdout: create 121/36 file after the snapshot 2024-03-09T08:55:12.070 INFO:tasks.workunit.client.1.smithi134.stdout: create 121/37 file after the snapshot 2024-03-09T08:55:12.070 INFO:tasks.workunit.client.1.smithi134.stdout: create 121/38 file after the snapshot 2024-03-09T08:55:12.071 INFO:tasks.workunit.client.1.smithi134.stdout: create 121/39 file after the snapshot 2024-03-09T08:55:12.071 INFO:tasks.workunit.client.1.smithi134.stdout: create 122/.snap/snap-subdir-test 2024-03-09T08:55:12.145 INFO:tasks.workunit.client.1.smithi134.stdout: create 122/30 file after the snapshot 2024-03-09T08:55:12.147 INFO:tasks.workunit.client.1.smithi134.stdout: create 122/31 file after the snapshot 2024-03-09T08:55:12.148 INFO:tasks.workunit.client.1.smithi134.stdout: create 122/32 file after the snapshot 2024-03-09T08:55:12.148 INFO:tasks.workunit.client.1.smithi134.stdout: create 122/33 file after the snapshot 2024-03-09T08:55:12.148 INFO:tasks.workunit.client.1.smithi134.stdout: create 122/34 file after the snapshot 2024-03-09T08:55:12.149 INFO:tasks.workunit.client.1.smithi134.stdout: create 122/35 file after the snapshot 2024-03-09T08:55:12.149 INFO:tasks.workunit.client.1.smithi134.stdout: create 122/36 file after the snapshot 2024-03-09T08:55:12.149 INFO:tasks.workunit.client.1.smithi134.stdout: create 122/37 file after the snapshot 2024-03-09T08:55:12.150 INFO:tasks.workunit.client.1.smithi134.stdout: create 122/38 file after the snapshot 2024-03-09T08:55:12.150 INFO:tasks.workunit.client.1.smithi134.stdout: create 122/39 file after the snapshot 2024-03-09T08:55:12.150 INFO:tasks.workunit.client.1.smithi134.stdout: create 123/.snap/snap-subdir-test 2024-03-09T08:55:12.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.172+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:12.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.173+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 msgr2=0x7f6428151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:55:12.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.173+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 msgr2=0x7f6428151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:55:12.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.173+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:55:12.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.173+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:55:12.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.173+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:12.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.174+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 msgr2=0x7f6428150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:55:12.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.174+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 msgr2=0x7f6428150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:55:12.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.174+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:55:12.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.174+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:55:12.228 INFO:tasks.workunit.client.1.smithi134.stdout: create 123/30 file after the snapshot 2024-03-09T08:55:12.230 INFO:tasks.workunit.client.1.smithi134.stdout: create 123/31 file after the snapshot 2024-03-09T08:55:12.231 INFO:tasks.workunit.client.1.smithi134.stdout: create 123/32 file after the snapshot 2024-03-09T08:55:12.231 INFO:tasks.workunit.client.1.smithi134.stdout: create 123/33 file after the snapshot 2024-03-09T08:55:12.231 INFO:tasks.workunit.client.1.smithi134.stdout: create 123/34 file after the snapshot 2024-03-09T08:55:12.232 INFO:tasks.workunit.client.1.smithi134.stdout: create 123/35 file after the snapshot 2024-03-09T08:55:12.232 INFO:tasks.workunit.client.1.smithi134.stdout: create 123/36 file after the snapshot 2024-03-09T08:55:12.232 INFO:tasks.workunit.client.1.smithi134.stdout: create 123/37 file after the snapshot 2024-03-09T08:55:12.232 INFO:tasks.workunit.client.1.smithi134.stdout: create 123/38 file after the snapshot 2024-03-09T08:55:12.233 INFO:tasks.workunit.client.1.smithi134.stdout: create 123/39 file after the snapshot 2024-03-09T08:55:12.233 INFO:tasks.workunit.client.1.smithi134.stdout: create 124/.snap/snap-subdir-test 2024-03-09T08:55:12.312 INFO:tasks.workunit.client.1.smithi134.stdout: create 124/30 file after the snapshot 2024-03-09T08:55:12.315 INFO:tasks.workunit.client.1.smithi134.stdout: create 124/31 file after the snapshot 2024-03-09T08:55:12.315 INFO:tasks.workunit.client.1.smithi134.stdout: create 124/32 file after the snapshot 2024-03-09T08:55:12.316 INFO:tasks.workunit.client.1.smithi134.stdout: create 124/33 file after the snapshot 2024-03-09T08:55:12.316 INFO:tasks.workunit.client.1.smithi134.stdout: create 124/34 file after the snapshot 2024-03-09T08:55:12.316 INFO:tasks.workunit.client.1.smithi134.stdout: create 124/35 file after the snapshot 2024-03-09T08:55:12.316 INFO:tasks.workunit.client.1.smithi134.stdout: create 124/36 file after the snapshot 2024-03-09T08:55:12.317 INFO:tasks.workunit.client.1.smithi134.stdout: create 124/37 file after the snapshot 2024-03-09T08:55:12.317 INFO:tasks.workunit.client.1.smithi134.stdout: create 124/38 file after the snapshot 2024-03-09T08:55:12.317 INFO:tasks.workunit.client.1.smithi134.stdout: create 124/39 file after the snapshot 2024-03-09T08:55:12.317 INFO:tasks.workunit.client.1.smithi134.stdout: create 125/.snap/snap-subdir-test 2024-03-09T08:55:12.396 INFO:tasks.workunit.client.1.smithi134.stdout: create 125/30 file after the snapshot 2024-03-09T08:55:12.399 INFO:tasks.workunit.client.1.smithi134.stdout: create 125/31 file after the snapshot 2024-03-09T08:55:12.399 INFO:tasks.workunit.client.1.smithi134.stdout: create 125/32 file after the snapshot 2024-03-09T08:55:12.399 INFO:tasks.workunit.client.1.smithi134.stdout: create 125/33 file after the snapshot 2024-03-09T08:55:12.399 INFO:tasks.workunit.client.1.smithi134.stdout: create 125/34 file after the snapshot 2024-03-09T08:55:12.400 INFO:tasks.workunit.client.1.smithi134.stdout: create 125/35 file after the snapshot 2024-03-09T08:55:12.400 INFO:tasks.workunit.client.1.smithi134.stdout: create 125/36 file after the snapshot 2024-03-09T08:55:12.400 INFO:tasks.workunit.client.1.smithi134.stdout: create 125/37 file after the snapshot 2024-03-09T08:55:12.400 INFO:tasks.workunit.client.1.smithi134.stdout: create 125/38 file after the snapshot 2024-03-09T08:55:12.401 INFO:tasks.workunit.client.1.smithi134.stdout: create 125/39 file after the snapshot 2024-03-09T08:55:12.401 INFO:tasks.workunit.client.1.smithi134.stdout: create 126/.snap/snap-subdir-test 2024-03-09T08:55:12.495 INFO:tasks.workunit.client.1.smithi134.stdout: create 126/30 file after the snapshot 2024-03-09T08:55:12.497 INFO:tasks.workunit.client.1.smithi134.stdout: create 126/31 file after the snapshot 2024-03-09T08:55:12.498 INFO:tasks.workunit.client.1.smithi134.stdout: create 126/32 file after the snapshot 2024-03-09T08:55:12.498 INFO:tasks.workunit.client.1.smithi134.stdout: create 126/33 file after the snapshot 2024-03-09T08:55:12.498 INFO:tasks.workunit.client.1.smithi134.stdout: create 126/34 file after the snapshot 2024-03-09T08:55:12.499 INFO:tasks.workunit.client.1.smithi134.stdout: create 126/35 file after the snapshot 2024-03-09T08:55:12.499 INFO:tasks.workunit.client.1.smithi134.stdout: create 126/36 file after the snapshot 2024-03-09T08:55:12.499 INFO:tasks.workunit.client.1.smithi134.stdout: create 126/37 file after the snapshot 2024-03-09T08:55:12.500 INFO:tasks.workunit.client.1.smithi134.stdout: create 126/38 file after the snapshot 2024-03-09T08:55:12.500 INFO:tasks.workunit.client.1.smithi134.stdout: create 126/39 file after the snapshot 2024-03-09T08:55:12.500 INFO:tasks.workunit.client.1.smithi134.stdout: create 127/.snap/snap-subdir-test 2024-03-09T08:55:12.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.574+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:12.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.575+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 msgr2=0x7f6428151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:55:12.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.575+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 msgr2=0x7f6428151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:55:12.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.575+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:55:12.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.575+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:55:12.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.575+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:12.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.576+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 msgr2=0x7f6428150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:55:12.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.576+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 msgr2=0x7f6428150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:55:12.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.576+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:55:12.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:12.576+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:55:12.587 INFO:tasks.workunit.client.1.smithi134.stdout: create 127/30 file after the snapshot 2024-03-09T08:55:12.590 INFO:tasks.workunit.client.1.smithi134.stdout: create 127/31 file after the snapshot 2024-03-09T08:55:12.590 INFO:tasks.workunit.client.1.smithi134.stdout: create 127/32 file after the snapshot 2024-03-09T08:55:12.590 INFO:tasks.workunit.client.1.smithi134.stdout: create 127/33 file after the snapshot 2024-03-09T08:55:12.591 INFO:tasks.workunit.client.1.smithi134.stdout: create 127/34 file after the snapshot 2024-03-09T08:55:12.591 INFO:tasks.workunit.client.1.smithi134.stdout: create 127/35 file after the snapshot 2024-03-09T08:55:12.591 INFO:tasks.workunit.client.1.smithi134.stdout: create 127/36 file after the snapshot 2024-03-09T08:55:12.591 INFO:tasks.workunit.client.1.smithi134.stdout: create 127/37 file after the snapshot 2024-03-09T08:55:12.592 INFO:tasks.workunit.client.1.smithi134.stdout: create 127/38 file after the snapshot 2024-03-09T08:55:12.592 INFO:tasks.workunit.client.1.smithi134.stdout: create 127/39 file after the snapshot 2024-03-09T08:55:12.592 INFO:tasks.workunit.client.1.smithi134.stdout: create 128/.snap/snap-subdir-test 2024-03-09T08:55:12.845 INFO:tasks.workunit.client.1.smithi134.stdout: create 128/30 file after the snapshot 2024-03-09T08:55:12.847 INFO:tasks.workunit.client.1.smithi134.stdout: create 128/31 file after the snapshot 2024-03-09T08:55:12.848 INFO:tasks.workunit.client.1.smithi134.stdout: create 128/32 file after the snapshot 2024-03-09T08:55:12.848 INFO:tasks.workunit.client.1.smithi134.stdout: create 128/33 file after the snapshot 2024-03-09T08:55:12.848 INFO:tasks.workunit.client.1.smithi134.stdout: create 128/34 file after the snapshot 2024-03-09T08:55:12.848 INFO:tasks.workunit.client.1.smithi134.stdout: create 128/35 file after the snapshot 2024-03-09T08:55:12.849 INFO:tasks.workunit.client.1.smithi134.stdout: create 128/36 file after the snapshot 2024-03-09T08:55:12.849 INFO:tasks.workunit.client.1.smithi134.stdout: create 128/37 file after the snapshot 2024-03-09T08:55:12.849 INFO:tasks.workunit.client.1.smithi134.stdout: create 128/38 file after the snapshot 2024-03-09T08:55:12.849 INFO:tasks.workunit.client.1.smithi134.stdout: create 128/39 file after the snapshot 2024-03-09T08:55:12.849 INFO:tasks.workunit.client.1.smithi134.stdout: create 129/.snap/snap-subdir-test 2024-03-09T08:55:12.936 INFO:tasks.workunit.client.1.smithi134.stdout: create 129/30 file after the snapshot 2024-03-09T08:55:12.938 INFO:tasks.workunit.client.1.smithi134.stdout: create 129/31 file after the snapshot 2024-03-09T08:55:12.939 INFO:tasks.workunit.client.1.smithi134.stdout: create 129/32 file after the snapshot 2024-03-09T08:55:12.939 INFO:tasks.workunit.client.1.smithi134.stdout: create 129/33 file after the snapshot 2024-03-09T08:55:12.939 INFO:tasks.workunit.client.1.smithi134.stdout: create 129/34 file after the snapshot 2024-03-09T08:55:12.940 INFO:tasks.workunit.client.1.smithi134.stdout: create 129/35 file after the snapshot 2024-03-09T08:55:12.940 INFO:tasks.workunit.client.1.smithi134.stdout: create 129/36 file after the snapshot 2024-03-09T08:55:12.941 INFO:tasks.workunit.client.1.smithi134.stdout: create 129/37 file after the snapshot 2024-03-09T08:55:12.941 INFO:tasks.workunit.client.1.smithi134.stdout: create 129/38 file after the snapshot 2024-03-09T08:55:12.942 INFO:tasks.workunit.client.1.smithi134.stdout: create 129/39 file after the snapshot 2024-03-09T08:55:12.942 INFO:tasks.workunit.client.1.smithi134.stdout: create 130/.snap/snap-subdir-test 2024-03-09T08:55:13.028 INFO:tasks.workunit.client.1.smithi134.stdout: create 130/30 file after the snapshot 2024-03-09T08:55:13.031 INFO:tasks.workunit.client.1.smithi134.stdout: create 130/31 file after the snapshot 2024-03-09T08:55:13.032 INFO:tasks.workunit.client.1.smithi134.stdout: create 130/32 file after the snapshot 2024-03-09T08:55:13.032 INFO:tasks.workunit.client.1.smithi134.stdout: create 130/33 file after the snapshot 2024-03-09T08:55:13.033 INFO:tasks.workunit.client.1.smithi134.stdout: create 130/34 file after the snapshot 2024-03-09T08:55:13.033 INFO:tasks.workunit.client.1.smithi134.stdout: create 130/35 file after the snapshot 2024-03-09T08:55:13.033 INFO:tasks.workunit.client.1.smithi134.stdout: create 130/36 file after the snapshot 2024-03-09T08:55:13.033 INFO:tasks.workunit.client.1.smithi134.stdout: create 130/37 file after the snapshot 2024-03-09T08:55:13.034 INFO:tasks.workunit.client.1.smithi134.stdout: create 130/38 file after the snapshot 2024-03-09T08:55:13.034 INFO:tasks.workunit.client.1.smithi134.stdout: create 130/39 file after the snapshot 2024-03-09T08:55:13.034 INFO:tasks.workunit.client.1.smithi134.stdout: create 131/.snap/snap-subdir-test 2024-03-09T08:55:13.077 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 252 2024-03-09T08:55:13.096 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 253 2024-03-09T08:55:13.117 INFO:tasks.workunit.client.1.smithi134.stdout: create 131/30 file after the snapshot 2024-03-09T08:55:13.120 INFO:tasks.workunit.client.1.smithi134.stdout: create 131/31 file after the snapshot 2024-03-09T08:55:13.120 INFO:tasks.workunit.client.1.smithi134.stdout: create 131/32 file after the snapshot 2024-03-09T08:55:13.121 INFO:tasks.workunit.client.1.smithi134.stdout: create 131/33 file after the snapshot 2024-03-09T08:55:13.121 INFO:tasks.workunit.client.1.smithi134.stdout: create 131/34 file after the snapshot 2024-03-09T08:55:13.121 INFO:tasks.workunit.client.1.smithi134.stdout: create 131/35 file after the snapshot 2024-03-09T08:55:13.121 INFO:tasks.workunit.client.1.smithi134.stdout: create 131/36 file after the snapshot 2024-03-09T08:55:13.121 INFO:tasks.workunit.client.1.smithi134.stdout: create 131/37 file after the snapshot 2024-03-09T08:55:13.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 131/38 file after the snapshot 2024-03-09T08:55:13.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 131/39 file after the snapshot 2024-03-09T08:55:13.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 132/.snap/snap-subdir-test 2024-03-09T08:55:13.157 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 254 2024-03-09T08:55:13.176 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 255 2024-03-09T08:55:13.193 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 256 2024-03-09T08:55:13.208 INFO:tasks.workunit.client.1.smithi134.stdout: create 132/30 file after the snapshot 2024-03-09T08:55:13.210 INFO:tasks.workunit.client.1.smithi134.stdout: create 132/31 file after the snapshot 2024-03-09T08:55:13.210 INFO:tasks.workunit.client.1.smithi134.stdout: create 132/32 file after the snapshot 2024-03-09T08:55:13.211 INFO:tasks.workunit.client.1.smithi134.stdout: create 132/33 file after the snapshot 2024-03-09T08:55:13.211 INFO:tasks.workunit.client.1.smithi134.stdout: create 132/34 file after the snapshot 2024-03-09T08:55:13.211 INFO:tasks.workunit.client.1.smithi134.stdout: create 132/35 file after the snapshot 2024-03-09T08:55:13.212 INFO:tasks.workunit.client.1.smithi134.stdout: create 132/36 file after the snapshot 2024-03-09T08:55:13.212 INFO:tasks.workunit.client.1.smithi134.stdout: create 132/37 file after the snapshot 2024-03-09T08:55:13.212 INFO:tasks.workunit.client.1.smithi134.stdout: create 132/38 file after the snapshot 2024-03-09T08:55:13.212 INFO:tasks.workunit.client.1.smithi134.stdout: create 132/39 file after the snapshot 2024-03-09T08:55:13.213 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 257 2024-03-09T08:55:13.213 INFO:tasks.workunit.client.1.smithi134.stdout: create 133/.snap/snap-subdir-test 2024-03-09T08:55:13.233 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 258 2024-03-09T08:55:13.248 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 259 2024-03-09T08:55:13.263 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 260 2024-03-09T08:55:13.277 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 261 2024-03-09T08:55:13.289 INFO:tasks.workunit.client.1.smithi134.stdout: create 133/30 file after the snapshot 2024-03-09T08:55:13.291 INFO:tasks.workunit.client.1.smithi134.stdout: create 133/31 file after the snapshot 2024-03-09T08:55:13.291 INFO:tasks.workunit.client.1.smithi134.stdout: create 133/32 file after the snapshot 2024-03-09T08:55:13.291 INFO:tasks.workunit.client.1.smithi134.stdout: create 133/33 file after the snapshot 2024-03-09T08:55:13.292 INFO:tasks.workunit.client.1.smithi134.stdout: create 133/34 file after the snapshot 2024-03-09T08:55:13.292 INFO:tasks.workunit.client.1.smithi134.stdout: create 133/35 file after the snapshot 2024-03-09T08:55:13.292 INFO:tasks.workunit.client.1.smithi134.stdout: create 133/36 file after the snapshot 2024-03-09T08:55:13.292 INFO:tasks.workunit.client.1.smithi134.stdout: create 133/37 file after the snapshot 2024-03-09T08:55:13.292 INFO:tasks.workunit.client.1.smithi134.stdout: create 133/38 file after the snapshot 2024-03-09T08:55:13.292 INFO:tasks.workunit.client.1.smithi134.stdout: create 133/39 file after the snapshot 2024-03-09T08:55:13.293 INFO:tasks.workunit.client.1.smithi134.stdout: create 134/.snap/snap-subdir-test 2024-03-09T08:55:13.294 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 262 2024-03-09T08:55:13.360 INFO:tasks.workunit.client.1.smithi134.stdout: create 134/30 file after the snapshot 2024-03-09T08:55:13.362 INFO:tasks.workunit.client.1.smithi134.stdout: create 134/31 file after the snapshot 2024-03-09T08:55:13.362 INFO:tasks.workunit.client.1.smithi134.stdout: create 134/32 file after the snapshot 2024-03-09T08:55:13.363 INFO:tasks.workunit.client.1.smithi134.stdout: create 134/33 file after the snapshot 2024-03-09T08:55:13.363 INFO:tasks.workunit.client.1.smithi134.stdout: create 134/34 file after the snapshot 2024-03-09T08:55:13.363 INFO:tasks.workunit.client.1.smithi134.stdout: create 134/35 file after the snapshot 2024-03-09T08:55:13.363 INFO:tasks.workunit.client.1.smithi134.stdout: create 134/36 file after the snapshot 2024-03-09T08:55:13.364 INFO:tasks.workunit.client.1.smithi134.stdout: create 134/37 file after the snapshot 2024-03-09T08:55:13.364 INFO:tasks.workunit.client.1.smithi134.stdout: create 134/38 file after the snapshot 2024-03-09T08:55:13.364 INFO:tasks.workunit.client.1.smithi134.stdout: create 134/39 file after the snapshot 2024-03-09T08:55:13.364 INFO:tasks.workunit.client.1.smithi134.stdout: create 135/.snap/snap-subdir-test 2024-03-09T08:55:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:13.376+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:13.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:13.377+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 msgr2=0x7f6428151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:55:13.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:13.377+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 msgr2=0x7f6428151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:55:13.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:13.377+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:55:13.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:13.377+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:13.377+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:55:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:13.378+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 msgr2=0x7f6428150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:55:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:13.378+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 msgr2=0x7f6428150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:55:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:13.378+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:55:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:13.378+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:55:13.382 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 263 2024-03-09T08:55:13.400 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 264 2024-03-09T08:55:13.420 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 265 2024-03-09T08:55:13.703 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 266 2024-03-09T08:55:13.741 INFO:tasks.workunit.client.1.smithi134.stdout: create 135/30 file after the snapshot 2024-03-09T08:55:13.744 INFO:tasks.workunit.client.1.smithi134.stdout: create 135/31 file after the snapshot 2024-03-09T08:55:13.744 INFO:tasks.workunit.client.1.smithi134.stdout: create 135/32 file after the snapshot 2024-03-09T08:55:13.745 INFO:tasks.workunit.client.1.smithi134.stdout: create 135/33 file after the snapshot 2024-03-09T08:55:13.745 INFO:tasks.workunit.client.1.smithi134.stdout: create 135/34 file after the snapshot 2024-03-09T08:55:13.745 INFO:tasks.workunit.client.1.smithi134.stdout: create 135/35 file after the snapshot 2024-03-09T08:55:13.746 INFO:tasks.workunit.client.1.smithi134.stdout: create 135/36 file after the snapshot 2024-03-09T08:55:13.746 INFO:tasks.workunit.client.1.smithi134.stdout: create 135/37 file after the snapshot 2024-03-09T08:55:13.746 INFO:tasks.workunit.client.1.smithi134.stdout: create 135/38 file after the snapshot 2024-03-09T08:55:13.747 INFO:tasks.workunit.client.1.smithi134.stdout: create 135/39 file after the snapshot 2024-03-09T08:55:13.748 INFO:tasks.workunit.client.1.smithi134.stdout: create 136/.snap/snap-subdir-test 2024-03-09T08:55:13.824 INFO:tasks.workunit.client.1.smithi134.stdout: create 136/30 file after the snapshot 2024-03-09T08:55:13.826 INFO:tasks.workunit.client.1.smithi134.stdout: create 136/31 file after the snapshot 2024-03-09T08:55:13.826 INFO:tasks.workunit.client.1.smithi134.stdout: create 136/32 file after the snapshot 2024-03-09T08:55:13.827 INFO:tasks.workunit.client.1.smithi134.stdout: create 136/33 file after the snapshot 2024-03-09T08:55:13.827 INFO:tasks.workunit.client.1.smithi134.stdout: create 136/34 file after the snapshot 2024-03-09T08:55:13.827 INFO:tasks.workunit.client.1.smithi134.stdout: create 136/35 file after the snapshot 2024-03-09T08:55:13.828 INFO:tasks.workunit.client.1.smithi134.stdout: create 136/36 file after the snapshot 2024-03-09T08:55:13.828 INFO:tasks.workunit.client.1.smithi134.stdout: create 136/37 file after the snapshot 2024-03-09T08:55:13.828 INFO:tasks.workunit.client.1.smithi134.stdout: create 136/38 file after the snapshot 2024-03-09T08:55:13.828 INFO:tasks.workunit.client.1.smithi134.stdout: create 136/39 file after the snapshot 2024-03-09T08:55:13.829 INFO:tasks.workunit.client.1.smithi134.stdout: create 137/.snap/snap-subdir-test 2024-03-09T08:55:13.984 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 267 2024-03-09T08:55:14.009 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 268 2024-03-09T08:55:14.032 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 269 2024-03-09T08:55:14.053 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 270 2024-03-09T08:55:14.076 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 271 2024-03-09T08:55:14.099 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 272 2024-03-09T08:55:14.119 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 273 2024-03-09T08:55:14.140 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 274 2024-03-09T08:55:14.165 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 275 2024-03-09T08:55:14.185 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 276 2024-03-09T08:55:14.206 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 277 2024-03-09T08:55:14.393 INFO:tasks.workunit.client.1.smithi134.stdout: create 137/30 file after the snapshot 2024-03-09T08:55:14.396 INFO:tasks.workunit.client.1.smithi134.stdout: create 137/31 file after the snapshot 2024-03-09T08:55:14.396 INFO:tasks.workunit.client.1.smithi134.stdout: create 137/32 file after the snapshot 2024-03-09T08:55:14.396 INFO:tasks.workunit.client.1.smithi134.stdout: create 137/33 file after the snapshot 2024-03-09T08:55:14.397 INFO:tasks.workunit.client.1.smithi134.stdout: create 137/34 file after the snapshot 2024-03-09T08:55:14.397 INFO:tasks.workunit.client.1.smithi134.stdout: create 137/35 file after the snapshot 2024-03-09T08:55:14.397 INFO:tasks.workunit.client.1.smithi134.stdout: create 137/36 file after the snapshot 2024-03-09T08:55:14.398 INFO:tasks.workunit.client.1.smithi134.stdout: create 137/37 file after the snapshot 2024-03-09T08:55:14.398 INFO:tasks.workunit.client.1.smithi134.stdout: create 137/38 file after the snapshot 2024-03-09T08:55:14.398 INFO:tasks.workunit.client.1.smithi134.stdout: create 137/39 file after the snapshot 2024-03-09T08:55:14.398 INFO:tasks.workunit.client.1.smithi134.stdout: create 138/.snap/snap-subdir-test 2024-03-09T08:55:14.530 INFO:tasks.workunit.client.1.smithi134.stdout: create 138/30 file after the snapshot 2024-03-09T08:55:14.533 INFO:tasks.workunit.client.1.smithi134.stdout: create 138/31 file after the snapshot 2024-03-09T08:55:14.534 INFO:tasks.workunit.client.1.smithi134.stdout: create 138/32 file after the snapshot 2024-03-09T08:55:14.534 INFO:tasks.workunit.client.1.smithi134.stdout: create 138/33 file after the snapshot 2024-03-09T08:55:14.535 INFO:tasks.workunit.client.1.smithi134.stdout: create 138/34 file after the snapshot 2024-03-09T08:55:14.535 INFO:tasks.workunit.client.1.smithi134.stdout: create 138/35 file after the snapshot 2024-03-09T08:55:14.535 INFO:tasks.workunit.client.1.smithi134.stdout: create 138/36 file after the snapshot 2024-03-09T08:55:14.536 INFO:tasks.workunit.client.1.smithi134.stdout: create 138/37 file after the snapshot 2024-03-09T08:55:14.536 INFO:tasks.workunit.client.1.smithi134.stdout: create 138/38 file after the snapshot 2024-03-09T08:55:14.536 INFO:tasks.workunit.client.1.smithi134.stdout: create 138/39 file after the snapshot 2024-03-09T08:55:14.537 INFO:tasks.workunit.client.1.smithi134.stdout: create 139/.snap/snap-subdir-test 2024-03-09T08:55:14.614 INFO:tasks.workunit.client.1.smithi134.stdout: create 139/30 file after the snapshot 2024-03-09T08:55:14.616 INFO:tasks.workunit.client.1.smithi134.stdout: create 139/31 file after the snapshot 2024-03-09T08:55:14.617 INFO:tasks.workunit.client.1.smithi134.stdout: create 139/32 file after the snapshot 2024-03-09T08:55:14.617 INFO:tasks.workunit.client.1.smithi134.stdout: create 139/33 file after the snapshot 2024-03-09T08:55:14.617 INFO:tasks.workunit.client.1.smithi134.stdout: create 139/34 file after the snapshot 2024-03-09T08:55:14.617 INFO:tasks.workunit.client.1.smithi134.stdout: create 139/35 file after the snapshot 2024-03-09T08:55:14.618 INFO:tasks.workunit.client.1.smithi134.stdout: create 139/36 file after the snapshot 2024-03-09T08:55:14.618 INFO:tasks.workunit.client.1.smithi134.stdout: create 139/37 file after the snapshot 2024-03-09T08:55:14.618 INFO:tasks.workunit.client.1.smithi134.stdout: create 139/38 file after the snapshot 2024-03-09T08:55:14.618 INFO:tasks.workunit.client.1.smithi134.stdout: create 139/39 file after the snapshot 2024-03-09T08:55:14.618 INFO:tasks.workunit.client.1.smithi134.stdout: create 140/.snap/snap-subdir-test 2024-03-09T08:55:14.697 INFO:tasks.workunit.client.1.smithi134.stdout: create 140/30 file after the snapshot 2024-03-09T08:55:14.700 INFO:tasks.workunit.client.1.smithi134.stdout: create 140/31 file after the snapshot 2024-03-09T08:55:14.700 INFO:tasks.workunit.client.1.smithi134.stdout: create 140/32 file after the snapshot 2024-03-09T08:55:14.701 INFO:tasks.workunit.client.1.smithi134.stdout: create 140/33 file after the snapshot 2024-03-09T08:55:14.701 INFO:tasks.workunit.client.1.smithi134.stdout: create 140/34 file after the snapshot 2024-03-09T08:55:14.702 INFO:tasks.workunit.client.1.smithi134.stdout: create 140/35 file after the snapshot 2024-03-09T08:55:14.702 INFO:tasks.workunit.client.1.smithi134.stdout: create 140/36 file after the snapshot 2024-03-09T08:55:14.703 INFO:tasks.workunit.client.1.smithi134.stdout: create 140/37 file after the snapshot 2024-03-09T08:55:14.703 INFO:tasks.workunit.client.1.smithi134.stdout: create 140/38 file after the snapshot 2024-03-09T08:55:14.703 INFO:tasks.workunit.client.1.smithi134.stdout: create 140/39 file after the snapshot 2024-03-09T08:55:14.704 INFO:tasks.workunit.client.1.smithi134.stdout: create 141/.snap/snap-subdir-test 2024-03-09T08:55:14.795 INFO:tasks.workunit.client.1.smithi134.stdout: create 141/30 file after the snapshot 2024-03-09T08:55:14.798 INFO:tasks.workunit.client.1.smithi134.stdout: create 141/31 file after the snapshot 2024-03-09T08:55:14.798 INFO:tasks.workunit.client.1.smithi134.stdout: create 141/32 file after the snapshot 2024-03-09T08:55:14.799 INFO:tasks.workunit.client.1.smithi134.stdout: create 141/33 file after the snapshot 2024-03-09T08:55:14.799 INFO:tasks.workunit.client.1.smithi134.stdout: create 141/34 file after the snapshot 2024-03-09T08:55:14.799 INFO:tasks.workunit.client.1.smithi134.stdout: create 141/35 file after the snapshot 2024-03-09T08:55:14.800 INFO:tasks.workunit.client.1.smithi134.stdout: create 141/36 file after the snapshot 2024-03-09T08:55:14.800 INFO:tasks.workunit.client.1.smithi134.stdout: create 141/37 file after the snapshot 2024-03-09T08:55:14.800 INFO:tasks.workunit.client.1.smithi134.stdout: create 141/38 file after the snapshot 2024-03-09T08:55:14.801 INFO:tasks.workunit.client.1.smithi134.stdout: create 141/39 file after the snapshot 2024-03-09T08:55:14.801 INFO:tasks.workunit.client.1.smithi134.stdout: create 142/.snap/snap-subdir-test 2024-03-09T08:55:14.863 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 278 2024-03-09T08:55:14.891 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 279 2024-03-09T08:55:14.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 142/30 file after the snapshot 2024-03-09T08:55:14.898 INFO:tasks.workunit.client.1.smithi134.stdout: create 142/31 file after the snapshot 2024-03-09T08:55:14.898 INFO:tasks.workunit.client.1.smithi134.stdout: create 142/32 file after the snapshot 2024-03-09T08:55:14.898 INFO:tasks.workunit.client.1.smithi134.stdout: create 142/33 file after the snapshot 2024-03-09T08:55:14.898 INFO:tasks.workunit.client.1.smithi134.stdout: create 142/34 file after the snapshot 2024-03-09T08:55:14.898 INFO:tasks.workunit.client.1.smithi134.stdout: create 142/35 file after the snapshot 2024-03-09T08:55:14.898 INFO:tasks.workunit.client.1.smithi134.stdout: create 142/36 file after the snapshot 2024-03-09T08:55:14.899 INFO:tasks.workunit.client.1.smithi134.stdout: create 142/37 file after the snapshot 2024-03-09T08:55:14.899 INFO:tasks.workunit.client.1.smithi134.stdout: create 142/38 file after the snapshot 2024-03-09T08:55:14.899 INFO:tasks.workunit.client.1.smithi134.stdout: create 142/39 file after the snapshot 2024-03-09T08:55:14.899 INFO:tasks.workunit.client.1.smithi134.stdout: create 143/.snap/snap-subdir-test 2024-03-09T08:55:14.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:14.971+0000 7f6426000640 1 -- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 msgr2=0x7f6428150c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:55:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:14.971+0000 7f6426000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64280c1f70 0x7f6428150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:14.971+0000 7f6426000640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 msgr2=0x7f6428151190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:55:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:14.971+0000 7f6426000640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6428154080 0x7f6428151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:14.971+0000 7f6426000640 1 -- 172.21.15.67:0/227509221 >> v1:172.21.15.67:6789/0 conn(0x7f64280710a0 legacy=0x7f6428153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:14.971+0000 7f6426000640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f6408007350 0x7f6408009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:14.971+0000 7f6426000640 1 -- 172.21.15.67:0/227509221 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64280c3700 con 0x7f64080036e0 2024-03-09T08:55:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:14.971+0000 7f6426000640 1 -- 172.21.15.67:0/227509221 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64280c32b0 con 0x7f6408007350 2024-03-09T08:55:14.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:14.971+0000 7f6427e00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f6408007350 msgr2=0x7f6408009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:14.971+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> v1:172.21.15.134:6789/0 conn(0x7f64080036e0 legacy=0x7f6408005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:55:14.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:14.972+0000 7f6427e00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f6408007350 0x7f6408009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:15.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:15.173+0000 7f6427e00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f6408007350 msgr2=0x7f6408009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:15.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:15.173+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> v1:172.21.15.134:6789/0 conn(0x7f64080036e0 legacy=0x7f6408005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:55:15.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:15.173+0000 7f6427e00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f6408007350 0x7f6408009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:55:15.237 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 280 2024-03-09T08:55:15.261 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 281 2024-03-09T08:55:15.288 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 282 2024-03-09T08:55:15.334 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 283 2024-03-09T08:55:15.384 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 284 2024-03-09T08:55:15.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:15.574+0000 7f6427e00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f6408007350 msgr2=0x7f6408009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:15.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:15.574+0000 7f6427e00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f6408007350 0x7f6408009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:55:15.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:15.574+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> v1:172.21.15.134:6789/0 conn(0x7f64080036e0 legacy=0x7f6408005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:55:15.862 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 285 2024-03-09T08:55:15.892 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 286 2024-03-09T08:55:15.915 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 287 2024-03-09T08:55:15.937 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 288 2024-03-09T08:55:15.959 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 289 2024-03-09T08:55:15.979 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 290 2024-03-09T08:55:15.998 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 291 2024-03-09T08:55:16.019 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 292 2024-03-09T08:55:16.039 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 293 2024-03-09T08:55:16.060 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 294 2024-03-09T08:55:16.078 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 295 2024-03-09T08:55:16.096 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 296 2024-03-09T08:55:16.115 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 297 2024-03-09T08:55:16.135 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 298 2024-03-09T08:55:16.154 INFO:tasks.workunit.client.0.smithi067.stdout: create dir 299 2024-03-09T08:55:16.170 INFO:tasks.workunit.client.0.smithi067.stdout:Create a snapshot in every first level dir ... 2024-03-09T08:55:16.275 INFO:tasks.workunit.client.0.smithi067.stdout: create 100/.snap/snap-subdir-test 2024-03-09T08:55:16.330 INFO:tasks.workunit.client.0.smithi067.stdout: create 100/30 file after the snapshot 2024-03-09T08:55:16.333 INFO:tasks.workunit.client.0.smithi067.stdout: create 100/31 file after the snapshot 2024-03-09T08:55:16.334 INFO:tasks.workunit.client.0.smithi067.stdout: create 100/32 file after the snapshot 2024-03-09T08:55:16.334 INFO:tasks.workunit.client.0.smithi067.stdout: create 100/33 file after the snapshot 2024-03-09T08:55:16.334 INFO:tasks.workunit.client.0.smithi067.stdout: create 100/34 file after the snapshot 2024-03-09T08:55:16.335 INFO:tasks.workunit.client.0.smithi067.stdout: create 100/35 file after the snapshot 2024-03-09T08:55:16.335 INFO:tasks.workunit.client.0.smithi067.stdout: create 100/36 file after the snapshot 2024-03-09T08:55:16.335 INFO:tasks.workunit.client.0.smithi067.stdout: create 100/37 file after the snapshot 2024-03-09T08:55:16.335 INFO:tasks.workunit.client.0.smithi067.stdout: create 100/38 file after the snapshot 2024-03-09T08:55:16.336 INFO:tasks.workunit.client.0.smithi067.stdout: create 100/39 file after the snapshot 2024-03-09T08:55:16.336 INFO:tasks.workunit.client.0.smithi067.stdout: create 101/.snap/snap-subdir-test 2024-03-09T08:55:16.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:16.375+0000 7f6427e00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f6408007350 msgr2=0x7f6408009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:16.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:16.375+0000 7f6427e00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f6408007350 0x7f6408009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:55:16.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:16.375+0000 7f642d000640 1 -- 172.21.15.67:0/227509221 >> v1:172.21.15.134:6789/0 conn(0x7f64080036e0 legacy=0x7f6408005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:55:16.415 INFO:tasks.workunit.client.0.smithi067.stdout: create 101/30 file after the snapshot 2024-03-09T08:55:16.418 INFO:tasks.workunit.client.0.smithi067.stdout: create 101/31 file after the snapshot 2024-03-09T08:55:16.418 INFO:tasks.workunit.client.0.smithi067.stdout: create 101/32 file after the snapshot 2024-03-09T08:55:16.418 INFO:tasks.workunit.client.0.smithi067.stdout: create 101/33 file after the snapshot 2024-03-09T08:55:16.419 INFO:tasks.workunit.client.0.smithi067.stdout: create 101/34 file after the snapshot 2024-03-09T08:55:16.419 INFO:tasks.workunit.client.0.smithi067.stdout: create 101/35 file after the snapshot 2024-03-09T08:55:16.419 INFO:tasks.workunit.client.0.smithi067.stdout: create 101/36 file after the snapshot 2024-03-09T08:55:16.420 INFO:tasks.workunit.client.0.smithi067.stdout: create 101/37 file after the snapshot 2024-03-09T08:55:16.420 INFO:tasks.workunit.client.0.smithi067.stdout: create 101/38 file after the snapshot 2024-03-09T08:55:16.420 INFO:tasks.workunit.client.0.smithi067.stdout: create 101/39 file after the snapshot 2024-03-09T08:55:16.421 INFO:tasks.workunit.client.0.smithi067.stdout: create 102/.snap/snap-subdir-test 2024-03-09T08:55:16.497 INFO:tasks.workunit.client.0.smithi067.stdout: create 102/30 file after the snapshot 2024-03-09T08:55:16.501 INFO:tasks.workunit.client.0.smithi067.stdout: create 102/31 file after the snapshot 2024-03-09T08:55:16.501 INFO:tasks.workunit.client.0.smithi067.stdout: create 102/32 file after the snapshot 2024-03-09T08:55:16.502 INFO:tasks.workunit.client.0.smithi067.stdout: create 102/33 file after the snapshot 2024-03-09T08:55:16.502 INFO:tasks.workunit.client.0.smithi067.stdout: create 102/34 file after the snapshot 2024-03-09T08:55:16.502 INFO:tasks.workunit.client.0.smithi067.stdout: create 102/35 file after the snapshot 2024-03-09T08:55:16.503 INFO:tasks.workunit.client.0.smithi067.stdout: create 102/36 file after the snapshot 2024-03-09T08:55:16.503 INFO:tasks.workunit.client.0.smithi067.stdout: create 102/37 file after the snapshot 2024-03-09T08:55:16.503 INFO:tasks.workunit.client.0.smithi067.stdout: create 102/38 file after the snapshot 2024-03-09T08:55:16.504 INFO:tasks.workunit.client.0.smithi067.stdout: create 102/39 file after the snapshot 2024-03-09T08:55:16.504 INFO:tasks.workunit.client.0.smithi067.stdout: create 103/.snap/snap-subdir-test 2024-03-09T08:55:16.604 INFO:tasks.workunit.client.0.smithi067.stdout: create 103/30 file after the snapshot 2024-03-09T08:55:16.607 INFO:tasks.workunit.client.0.smithi067.stdout: create 103/31 file after the snapshot 2024-03-09T08:55:16.607 INFO:tasks.workunit.client.0.smithi067.stdout: create 103/32 file after the snapshot 2024-03-09T08:55:16.608 INFO:tasks.workunit.client.0.smithi067.stdout: create 103/33 file after the snapshot 2024-03-09T08:55:16.609 INFO:tasks.workunit.client.0.smithi067.stdout: create 103/34 file after the snapshot 2024-03-09T08:55:16.609 INFO:tasks.workunit.client.0.smithi067.stdout: create 103/35 file after the snapshot 2024-03-09T08:55:16.609 INFO:tasks.workunit.client.0.smithi067.stdout: create 103/36 file after the snapshot 2024-03-09T08:55:16.610 INFO:tasks.workunit.client.0.smithi067.stdout: create 103/37 file after the snapshot 2024-03-09T08:55:16.610 INFO:tasks.workunit.client.0.smithi067.stdout: create 103/38 file after the snapshot 2024-03-09T08:55:16.611 INFO:tasks.workunit.client.0.smithi067.stdout: create 103/39 file after the snapshot 2024-03-09T08:55:16.612 INFO:tasks.workunit.client.0.smithi067.stdout: create 104/.snap/snap-subdir-test 2024-03-09T08:55:16.640 INFO:tasks.workunit.client.1.smithi134.stdout: create 143/30 file after the snapshot 2024-03-09T08:55:16.703 INFO:tasks.workunit.client.0.smithi067.stdout: create 104/30 file after the snapshot 2024-03-09T08:55:16.708 INFO:tasks.workunit.client.0.smithi067.stdout: create 104/31 file after the snapshot 2024-03-09T08:55:16.708 INFO:tasks.workunit.client.0.smithi067.stdout: create 104/32 file after the snapshot 2024-03-09T08:55:16.709 INFO:tasks.workunit.client.0.smithi067.stdout: create 104/33 file after the snapshot 2024-03-09T08:55:16.709 INFO:tasks.workunit.client.0.smithi067.stdout: create 104/34 file after the snapshot 2024-03-09T08:55:16.709 INFO:tasks.workunit.client.0.smithi067.stdout: create 104/35 file after the snapshot 2024-03-09T08:55:16.709 INFO:tasks.workunit.client.0.smithi067.stdout: create 104/36 file after the snapshot 2024-03-09T08:55:16.709 INFO:tasks.workunit.client.0.smithi067.stdout: create 104/37 file after the snapshot 2024-03-09T08:55:16.709 INFO:tasks.workunit.client.0.smithi067.stdout: create 104/38 file after the snapshot 2024-03-09T08:55:16.709 INFO:tasks.workunit.client.0.smithi067.stdout: create 104/39 file after the snapshot 2024-03-09T08:55:16.710 INFO:tasks.workunit.client.0.smithi067.stdout: create 105/.snap/snap-subdir-test 2024-03-09T08:55:16.802 INFO:tasks.workunit.client.0.smithi067.stdout: create 105/30 file after the snapshot 2024-03-09T08:55:16.805 INFO:tasks.workunit.client.0.smithi067.stdout: create 105/31 file after the snapshot 2024-03-09T08:55:16.805 INFO:tasks.workunit.client.0.smithi067.stdout: create 105/32 file after the snapshot 2024-03-09T08:55:16.805 INFO:tasks.workunit.client.0.smithi067.stdout: create 105/33 file after the snapshot 2024-03-09T08:55:16.805 INFO:tasks.workunit.client.0.smithi067.stdout: create 105/34 file after the snapshot 2024-03-09T08:55:16.805 INFO:tasks.workunit.client.0.smithi067.stdout: create 105/35 file after the snapshot 2024-03-09T08:55:16.806 INFO:tasks.workunit.client.0.smithi067.stdout: create 105/36 file after the snapshot 2024-03-09T08:55:16.806 INFO:tasks.workunit.client.0.smithi067.stdout: create 105/37 file after the snapshot 2024-03-09T08:55:16.806 INFO:tasks.workunit.client.0.smithi067.stdout: create 105/38 file after the snapshot 2024-03-09T08:55:16.806 INFO:tasks.workunit.client.0.smithi067.stdout: create 105/39 file after the snapshot 2024-03-09T08:55:16.806 INFO:tasks.workunit.client.0.smithi067.stdout: create 106/.snap/snap-subdir-test 2024-03-09T08:55:16.858 INFO:tasks.workunit.client.1.smithi134.stdout: create 143/31 file after the snapshot 2024-03-09T08:55:16.859 INFO:tasks.workunit.client.1.smithi134.stdout: create 143/32 file after the snapshot 2024-03-09T08:55:16.859 INFO:tasks.workunit.client.1.smithi134.stdout: create 143/33 file after the snapshot 2024-03-09T08:55:16.860 INFO:tasks.workunit.client.1.smithi134.stdout: create 143/34 file after the snapshot 2024-03-09T08:55:16.861 INFO:tasks.workunit.client.1.smithi134.stdout: create 143/35 file after the snapshot 2024-03-09T08:55:16.862 INFO:tasks.workunit.client.1.smithi134.stdout: create 143/36 file after the snapshot 2024-03-09T08:55:16.862 INFO:tasks.workunit.client.1.smithi134.stdout: create 143/37 file after the snapshot 2024-03-09T08:55:16.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 143/38 file after the snapshot 2024-03-09T08:55:16.866 INFO:tasks.workunit.client.1.smithi134.stdout: create 143/39 file after the snapshot 2024-03-09T08:55:16.866 INFO:tasks.workunit.client.1.smithi134.stdout: create 144/.snap/snap-subdir-test 2024-03-09T08:55:16.897 INFO:tasks.workunit.client.0.smithi067.stdout: create 106/30 file after the snapshot 2024-03-09T08:55:16.898 INFO:tasks.workunit.client.0.smithi067.stdout: create 106/31 file after the snapshot 2024-03-09T08:55:16.898 INFO:tasks.workunit.client.0.smithi067.stdout: create 106/32 file after the snapshot 2024-03-09T08:55:16.899 INFO:tasks.workunit.client.0.smithi067.stdout: create 106/33 file after the snapshot 2024-03-09T08:55:16.899 INFO:tasks.workunit.client.0.smithi067.stdout: create 106/34 file after the snapshot 2024-03-09T08:55:16.899 INFO:tasks.workunit.client.0.smithi067.stdout: create 106/35 file after the snapshot 2024-03-09T08:55:16.899 INFO:tasks.workunit.client.0.smithi067.stdout: create 106/36 file after the snapshot 2024-03-09T08:55:16.900 INFO:tasks.workunit.client.0.smithi067.stdout: create 106/37 file after the snapshot 2024-03-09T08:55:16.900 INFO:tasks.workunit.client.0.smithi067.stdout: create 106/38 file after the snapshot 2024-03-09T08:55:16.900 INFO:tasks.workunit.client.0.smithi067.stdout: create 106/39 file after the snapshot 2024-03-09T08:55:16.900 INFO:tasks.workunit.client.0.smithi067.stdout: create 107/.snap/snap-subdir-test 2024-03-09T08:55:16.994 INFO:tasks.workunit.client.0.smithi067.stdout: create 107/30 file after the snapshot 2024-03-09T08:55:16.997 INFO:tasks.workunit.client.0.smithi067.stdout: create 107/31 file after the snapshot 2024-03-09T08:55:16.997 INFO:tasks.workunit.client.0.smithi067.stdout: create 107/32 file after the snapshot 2024-03-09T08:55:16.998 INFO:tasks.workunit.client.0.smithi067.stdout: create 107/33 file after the snapshot 2024-03-09T08:55:16.998 INFO:tasks.workunit.client.0.smithi067.stdout: create 107/34 file after the snapshot 2024-03-09T08:55:16.999 INFO:tasks.workunit.client.0.smithi067.stdout: create 107/35 file after the snapshot 2024-03-09T08:55:16.999 INFO:tasks.workunit.client.0.smithi067.stdout: create 107/36 file after the snapshot 2024-03-09T08:55:16.999 INFO:tasks.workunit.client.0.smithi067.stdout: create 107/37 file after the snapshot 2024-03-09T08:55:16.999 INFO:tasks.workunit.client.0.smithi067.stdout: create 107/38 file after the snapshot 2024-03-09T08:55:16.999 INFO:tasks.workunit.client.0.smithi067.stdout: create 107/39 file after the snapshot 2024-03-09T08:55:17.000 INFO:tasks.workunit.client.0.smithi067.stdout: create 108/.snap/snap-subdir-test 2024-03-09T08:55:17.806 INFO:tasks.workunit.client.0.smithi067.stdout: create 108/30 file after the snapshot 2024-03-09T08:55:17.843 INFO:tasks.workunit.client.1.smithi134.stdout: create 144/30 file after the snapshot 2024-03-09T08:55:17.846 INFO:tasks.workunit.client.1.smithi134.stdout: create 144/31 file after the snapshot 2024-03-09T08:55:17.846 INFO:tasks.workunit.client.1.smithi134.stdout: create 144/32 file after the snapshot 2024-03-09T08:55:17.846 INFO:tasks.workunit.client.1.smithi134.stdout: create 144/33 file after the snapshot 2024-03-09T08:55:17.847 INFO:tasks.workunit.client.1.smithi134.stdout: create 144/34 file after the snapshot 2024-03-09T08:55:17.847 INFO:tasks.workunit.client.1.smithi134.stdout: create 144/35 file after the snapshot 2024-03-09T08:55:17.847 INFO:tasks.workunit.client.1.smithi134.stdout: create 144/36 file after the snapshot 2024-03-09T08:55:17.847 INFO:tasks.workunit.client.1.smithi134.stdout: create 144/37 file after the snapshot 2024-03-09T08:55:17.848 INFO:tasks.workunit.client.1.smithi134.stdout: create 144/38 file after the snapshot 2024-03-09T08:55:17.848 INFO:tasks.workunit.client.1.smithi134.stdout: create 144/39 file after the snapshot 2024-03-09T08:55:17.848 INFO:tasks.workunit.client.1.smithi134.stdout: create 145/.snap/snap-subdir-test 2024-03-09T08:55:17.902 INFO:tasks.workunit.client.0.smithi067.stdout: create 108/31 file after the snapshot 2024-03-09T08:55:17.902 INFO:tasks.workunit.client.0.smithi067.stdout: create 108/32 file after the snapshot 2024-03-09T08:55:17.903 INFO:tasks.workunit.client.0.smithi067.stdout: create 108/33 file after the snapshot 2024-03-09T08:55:17.903 INFO:tasks.workunit.client.0.smithi067.stdout: create 108/34 file after the snapshot 2024-03-09T08:55:17.903 INFO:tasks.workunit.client.0.smithi067.stdout: create 108/35 file after the snapshot 2024-03-09T08:55:17.904 INFO:tasks.workunit.client.0.smithi067.stdout: create 108/36 file after the snapshot 2024-03-09T08:55:17.904 INFO:tasks.workunit.client.0.smithi067.stdout: create 108/37 file after the snapshot 2024-03-09T08:55:17.904 INFO:tasks.workunit.client.0.smithi067.stdout: create 108/38 file after the snapshot 2024-03-09T08:55:17.904 INFO:tasks.workunit.client.0.smithi067.stdout: create 108/39 file after the snapshot 2024-03-09T08:55:17.905 INFO:tasks.workunit.client.0.smithi067.stdout: create 109/.snap/snap-subdir-test 2024-03-09T08:55:17.931 INFO:tasks.workunit.client.1.smithi134.stdout: create 145/30 file after the snapshot 2024-03-09T08:55:17.933 INFO:tasks.workunit.client.1.smithi134.stdout: create 145/31 file after the snapshot 2024-03-09T08:55:17.934 INFO:tasks.workunit.client.1.smithi134.stdout: create 145/32 file after the snapshot 2024-03-09T08:55:17.934 INFO:tasks.workunit.client.1.smithi134.stdout: create 145/33 file after the snapshot 2024-03-09T08:55:17.934 INFO:tasks.workunit.client.1.smithi134.stdout: create 145/34 file after the snapshot 2024-03-09T08:55:17.934 INFO:tasks.workunit.client.1.smithi134.stdout: create 145/35 file after the snapshot 2024-03-09T08:55:17.935 INFO:tasks.workunit.client.1.smithi134.stdout: create 145/36 file after the snapshot 2024-03-09T08:55:17.935 INFO:tasks.workunit.client.1.smithi134.stdout: create 145/37 file after the snapshot 2024-03-09T08:55:17.935 INFO:tasks.workunit.client.1.smithi134.stdout: create 145/38 file after the snapshot 2024-03-09T08:55:17.935 INFO:tasks.workunit.client.1.smithi134.stdout: create 145/39 file after the snapshot 2024-03-09T08:55:17.935 INFO:tasks.workunit.client.1.smithi134.stdout: create 146/.snap/snap-subdir-test 2024-03-09T08:55:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.971+0000 7f6426000640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f6408007350 msgr2=0x7f6408009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:55:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.971+0000 7f6426000640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f6408007350 0x7f6408009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.971+0000 7f6426000640 1 -- 172.21.15.67:0/227509221 >> v1:172.21.15.134:6789/0 conn(0x7f64080036e0 legacy=0x7f6408005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:55:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.971+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 reap_dead start 2024-03-09T08:55:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.971+0000 7f6426000640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6408006250 0x7f6408009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.971+0000 7f6426000640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f64080066d0 0x7f6408005ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.971+0000 7f6426000640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6408006b40 0x7f6408009ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.971+0000 7f6426000640 1 -- 172.21.15.67:0/227509221 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64280c32b0 con 0x7f6408006250 2024-03-09T08:55:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.971+0000 7f6426000640 1 -- 172.21.15.67:0/227509221 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64280c2fd0 con 0x7f64080066d0 2024-03-09T08:55:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.971+0000 7f6426000640 1 -- 172.21.15.67:0/227509221 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640800a3e0 con 0x7f6408006b40 2024-03-09T08:55:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.972+0000 7f6427e00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f64080066d0 msgr2=0x7f6408005ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.972+0000 7f6427e00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f64080066d0 0x7f6408005ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.972+0000 7f642d000640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6408006250 0x7f6408009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.972+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6408006b40 0x7f6408009ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.973+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f64080066d0 msgr2=0x7f6408005ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:55:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.973+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f64080066d0 0x7f6408005ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.973+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6408006250 msgr2=0x7f6408009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.973+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6408006250 0x7f6408009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:17.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.973+0000 7f642da00640 1 -- 172.21.15.67:0/227509221 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640800afe0 con 0x7f6408006b40 2024-03-09T08:55:17.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.975+0000 7f642da00640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6408006b40 0x7f6408009ea0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6428073ec0 tx=0x7f642002c5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8c5b5d1553431e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:17.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.976+0000 7f6427400640 1 -- 172.21.15.67:0/227509221 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642003a070 con 0x7f6408006b40 2024-03-09T08:55:17.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.976+0000 7f6427400640 1 -- 172.21.15.67:0/227509221 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6420032480 con 0x7f6408006b40 2024-03-09T08:55:17.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.976+0000 7f6427400640 1 -- 172.21.15.67:0/227509221 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420032650 con 0x7f6408006b40 2024-03-09T08:55:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.978+0000 7f642e400640 1 -- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6408006b40 msgr2=0x7f6408009ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.978+0000 7f642e400640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6408006b40 0x7f6408009ea0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6428073ec0 tx=0x7f642002c5b0 comp rx=0 tx=0).stop 2024-03-09T08:55:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.978+0000 7f642e400640 1 -- 172.21.15.67:0/227509221 shutdown_connections 2024-03-09T08:55:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.978+0000 7f642e400640 1 --2- 172.21.15.67:0/227509221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6408006b40 0x7f6408009ea0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.978+0000 7f642e400640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.134:3300/0 conn(0x7f64080066d0 0x7f6408005ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.978+0000 7f642e400640 1 --2- 172.21.15.67:0/227509221 >> v2:172.21.15.67:3300/0 conn(0x7f6408006250 0x7f6408009810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.978+0000 7f642e400640 1 -- 172.21.15.67:0/227509221 >> 172.21.15.67:0/227509221 conn(0x7f64280c7300 msgr2=0x7f64280c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.978+0000 7f642e400640 1 -- 172.21.15.67:0/227509221 shutdown_connections 2024-03-09T08:55:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.978+0000 7f642e400640 1 -- 172.21.15.67:0/227509221 wait complete. 2024-03-09T08:55:17.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.982+0000 7f642e400640 1 Processor -- start 2024-03-09T08:55:17.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.982+0000 7f642e400640 1 -- start start 2024-03-09T08:55:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.983+0000 7f642e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6408006250 0x7f64280ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.983+0000 7f642e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64080066d0 0x7f64281f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.983+0000 7f642e400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6408006b40 0x7f64281f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.983+0000 7f642e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6428070d50 con 0x7f64080066d0 2024-03-09T08:55:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.983+0000 7f642e400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6428070830 con 0x7f6408006b40 2024-03-09T08:55:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.983+0000 7f642e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6428070ed0 con 0x7f6408006250 2024-03-09T08:55:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.983+0000 7f642da00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6408006b40 msgr2=0x7f64281f3610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.983+0000 7f642da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6408006b40 0x7f64281f3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.983+0000 7f6427e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64080066d0 0x7f64281f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.983+0000 7f6427e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64080066d0 0x7f64281f30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41014/0 (socket says 172.21.15.67:41014) 2024-03-09T08:55:17.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.983+0000 7f6427e00640 1 -- 172.21.15.67:0/3034774137 learned_addr learned my addr 172.21.15.67:0/3034774137 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:17.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.984+0000 7f6427e00640 1 -- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6408006250 msgr2=0x7f64280ab770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T08:55:17.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.984+0000 7f6427e00640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6408006250 0x7f64280ab770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:17.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.984+0000 7f6427e00640 1 -- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6408006b40 msgr2=0x7f64281f3610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:55:17.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.984+0000 7f6427e00640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6408006b40 0x7f64281f3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:17.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.984+0000 7f6427e00640 1 -- 172.21.15.67:0/3034774137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6428155ba0 con 0x7f64080066d0 2024-03-09T08:55:17.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.984+0000 7f6427e00640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64080066d0 0x7f64281f30d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6414001260 tx=0x7f641400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.984+0000 7f6424c00640 1 -- 172.21.15.67:0/3034774137 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414017070 con 0x7f64080066d0 2024-03-09T08:55:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.985+0000 7f6424c00640 1 -- 172.21.15.67:0/3034774137 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6414005ab0 con 0x7f64080066d0 2024-03-09T08:55:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.985+0000 7f642e400640 1 -- 172.21.15.67:0/3034774137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6428155e80 con 0x7f64080066d0 2024-03-09T08:55:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.985+0000 7f6424c00640 1 -- 172.21.15.67:0/3034774137 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414007070 con 0x7f64080066d0 2024-03-09T08:55:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.985+0000 7f642e400640 1 -- 172.21.15.67:0/3034774137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6428156440 con 0x7f64080066d0 2024-03-09T08:55:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.986+0000 7f642e400640 1 -- 172.21.15.67:0/3034774137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64280ad1e0 con 0x7f64080066d0 2024-03-09T08:55:17.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.987+0000 7f6424c00640 1 -- 172.21.15.67:0/3034774137 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f641401a070 con 0x7f64080066d0 2024-03-09T08:55:17.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.987+0000 7f6424c00640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f640806d070 0x7f640806f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:17.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.988+0000 7f6424c00640 1 -- 172.21.15.67:0/3034774137 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f6414089a30 con 0x7f64080066d0 2024-03-09T08:55:17.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.988+0000 7f642d000640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f640806d070 0x7f640806f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:17.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.989+0000 7f642d000640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f640806d070 0x7f640806f530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f641c00f280 tx=0x7f641c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:17.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:17.992+0000 7f6424c00640 1 -- 172.21.15.67:0/3034774137 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f64280ad1e0 con 0x7f64080066d0 2024-03-09T08:55:18.001 INFO:tasks.workunit.client.1.smithi134.stdout: create 146/30 file after the snapshot 2024-03-09T08:55:18.004 INFO:tasks.workunit.client.1.smithi134.stdout: create 146/31 file after the snapshot 2024-03-09T08:55:18.004 INFO:tasks.workunit.client.1.smithi134.stdout: create 146/32 file after the snapshot 2024-03-09T08:55:18.004 INFO:tasks.workunit.client.1.smithi134.stdout: create 146/33 file after the snapshot 2024-03-09T08:55:18.005 INFO:tasks.workunit.client.1.smithi134.stdout: create 146/34 file after the snapshot 2024-03-09T08:55:18.005 INFO:tasks.workunit.client.1.smithi134.stdout: create 146/35 file after the snapshot 2024-03-09T08:55:18.005 INFO:tasks.workunit.client.1.smithi134.stdout: create 146/36 file after the snapshot 2024-03-09T08:55:18.005 INFO:tasks.workunit.client.1.smithi134.stdout: create 146/37 file after the snapshot 2024-03-09T08:55:18.005 INFO:tasks.workunit.client.1.smithi134.stdout: create 146/38 file after the snapshot 2024-03-09T08:55:18.006 INFO:tasks.workunit.client.1.smithi134.stdout: create 146/39 file after the snapshot 2024-03-09T08:55:18.006 INFO:tasks.workunit.client.1.smithi134.stdout: create 147/.snap/snap-subdir-test 2024-03-09T08:55:18.093 INFO:tasks.workunit.client.1.smithi134.stdout: create 147/30 file after the snapshot 2024-03-09T08:55:18.096 INFO:tasks.workunit.client.1.smithi134.stdout: create 147/31 file after the snapshot 2024-03-09T08:55:18.096 INFO:tasks.workunit.client.1.smithi134.stdout: create 147/32 file after the snapshot 2024-03-09T08:55:18.096 INFO:tasks.workunit.client.1.smithi134.stdout: create 147/33 file after the snapshot 2024-03-09T08:55:18.097 INFO:tasks.workunit.client.1.smithi134.stdout: create 147/34 file after the snapshot 2024-03-09T08:55:18.097 INFO:tasks.workunit.client.1.smithi134.stdout: create 147/35 file after the snapshot 2024-03-09T08:55:18.097 INFO:tasks.workunit.client.1.smithi134.stdout: create 147/36 file after the snapshot 2024-03-09T08:55:18.097 INFO:tasks.workunit.client.1.smithi134.stdout: create 147/37 file after the snapshot 2024-03-09T08:55:18.097 INFO:tasks.workunit.client.1.smithi134.stdout: create 147/38 file after the snapshot 2024-03-09T08:55:18.097 INFO:tasks.workunit.client.1.smithi134.stdout: create 147/39 file after the snapshot 2024-03-09T08:55:18.098 INFO:tasks.workunit.client.1.smithi134.stdout: create 148/.snap/snap-subdir-test 2024-03-09T08:55:18.138 INFO:tasks.workunit.client.0.smithi067.stdout: create 109/30 file after the snapshot 2024-03-09T08:55:18.140 INFO:tasks.workunit.client.0.smithi067.stdout: create 109/31 file after the snapshot 2024-03-09T08:55:18.140 INFO:tasks.workunit.client.0.smithi067.stdout: create 109/32 file after the snapshot 2024-03-09T08:55:18.140 INFO:tasks.workunit.client.0.smithi067.stdout: create 109/33 file after the snapshot 2024-03-09T08:55:18.141 INFO:tasks.workunit.client.0.smithi067.stdout: create 109/34 file after the snapshot 2024-03-09T08:55:18.141 INFO:tasks.workunit.client.0.smithi067.stdout: create 109/35 file after the snapshot 2024-03-09T08:55:18.142 INFO:tasks.workunit.client.0.smithi067.stdout: create 109/36 file after the snapshot 2024-03-09T08:55:18.142 INFO:tasks.workunit.client.0.smithi067.stdout: create 109/37 file after the snapshot 2024-03-09T08:55:18.142 INFO:tasks.workunit.client.0.smithi067.stdout: create 109/38 file after the snapshot 2024-03-09T08:55:18.142 INFO:tasks.workunit.client.0.smithi067.stdout: create 109/39 file after the snapshot 2024-03-09T08:55:18.143 INFO:tasks.workunit.client.0.smithi067.stdout: create 110/.snap/snap-subdir-test 2024-03-09T08:55:18.180 INFO:tasks.workunit.client.1.smithi134.stdout: create 148/30 file after the snapshot 2024-03-09T08:55:18.183 INFO:tasks.workunit.client.1.smithi134.stdout: create 148/31 file after the snapshot 2024-03-09T08:55:18.183 INFO:tasks.workunit.client.1.smithi134.stdout: create 148/32 file after the snapshot 2024-03-09T08:55:18.183 INFO:tasks.workunit.client.1.smithi134.stdout: create 148/33 file after the snapshot 2024-03-09T08:55:18.184 INFO:tasks.workunit.client.1.smithi134.stdout: create 148/34 file after the snapshot 2024-03-09T08:55:18.184 INFO:tasks.workunit.client.1.smithi134.stdout: create 148/35 file after the snapshot 2024-03-09T08:55:18.185 INFO:tasks.workunit.client.1.smithi134.stdout: create 148/36 file after the snapshot 2024-03-09T08:55:18.185 INFO:tasks.workunit.client.1.smithi134.stdout: create 148/37 file after the snapshot 2024-03-09T08:55:18.186 INFO:tasks.workunit.client.1.smithi134.stdout: create 148/38 file after the snapshot 2024-03-09T08:55:18.186 INFO:tasks.workunit.client.1.smithi134.stdout: create 148/39 file after the snapshot 2024-03-09T08:55:18.187 INFO:tasks.workunit.client.1.smithi134.stdout: create 149/.snap/snap-subdir-test 2024-03-09T08:55:18.211 INFO:tasks.workunit.client.0.smithi067.stdout: create 110/30 file after the snapshot 2024-03-09T08:55:18.212 INFO:tasks.workunit.client.0.smithi067.stdout: create 110/31 file after the snapshot 2024-03-09T08:55:18.213 INFO:tasks.workunit.client.0.smithi067.stdout: create 110/32 file after the snapshot 2024-03-09T08:55:18.213 INFO:tasks.workunit.client.0.smithi067.stdout: create 110/33 file after the snapshot 2024-03-09T08:55:18.214 INFO:tasks.workunit.client.0.smithi067.stdout: create 110/34 file after the snapshot 2024-03-09T08:55:18.214 INFO:tasks.workunit.client.0.smithi067.stdout: create 110/35 file after the snapshot 2024-03-09T08:55:18.214 INFO:tasks.workunit.client.0.smithi067.stdout: create 110/36 file after the snapshot 2024-03-09T08:55:18.214 INFO:tasks.workunit.client.0.smithi067.stdout: create 110/37 file after the snapshot 2024-03-09T08:55:18.214 INFO:tasks.workunit.client.0.smithi067.stdout: create 110/38 file after the snapshot 2024-03-09T08:55:18.214 INFO:tasks.workunit.client.0.smithi067.stdout: create 110/39 file after the snapshot 2024-03-09T08:55:18.214 INFO:tasks.workunit.client.0.smithi067.stdout: create 111/.snap/snap-subdir-test 2024-03-09T08:55:18.256 INFO:tasks.workunit.client.1.smithi134.stdout: create 149/30 file after the snapshot 2024-03-09T08:55:18.258 INFO:tasks.workunit.client.1.smithi134.stdout: create 149/31 file after the snapshot 2024-03-09T08:55:18.258 INFO:tasks.workunit.client.1.smithi134.stdout: create 149/32 file after the snapshot 2024-03-09T08:55:18.258 INFO:tasks.workunit.client.1.smithi134.stdout: create 149/33 file after the snapshot 2024-03-09T08:55:18.259 INFO:tasks.workunit.client.1.smithi134.stdout: create 149/34 file after the snapshot 2024-03-09T08:55:18.259 INFO:tasks.workunit.client.1.smithi134.stdout: create 149/35 file after the snapshot 2024-03-09T08:55:18.259 INFO:tasks.workunit.client.1.smithi134.stdout: create 149/36 file after the snapshot 2024-03-09T08:55:18.260 INFO:tasks.workunit.client.1.smithi134.stdout: create 149/37 file after the snapshot 2024-03-09T08:55:18.260 INFO:tasks.workunit.client.1.smithi134.stdout: create 149/38 file after the snapshot 2024-03-09T08:55:18.260 INFO:tasks.workunit.client.1.smithi134.stdout: create 149/39 file after the snapshot 2024-03-09T08:55:18.260 INFO:tasks.workunit.client.1.smithi134.stdout: create 150/.snap/snap-subdir-test 2024-03-09T08:55:18.285 INFO:tasks.workunit.client.0.smithi067.stdout: create 111/30 file after the snapshot 2024-03-09T08:55:18.288 INFO:tasks.workunit.client.0.smithi067.stdout: create 111/31 file after the snapshot 2024-03-09T08:55:18.288 INFO:tasks.workunit.client.0.smithi067.stdout: create 111/32 file after the snapshot 2024-03-09T08:55:18.288 INFO:tasks.workunit.client.0.smithi067.stdout: create 111/33 file after the snapshot 2024-03-09T08:55:18.289 INFO:tasks.workunit.client.0.smithi067.stdout: create 111/34 file after the snapshot 2024-03-09T08:55:18.289 INFO:tasks.workunit.client.0.smithi067.stdout: create 111/35 file after the snapshot 2024-03-09T08:55:18.289 INFO:tasks.workunit.client.0.smithi067.stdout: create 111/36 file after the snapshot 2024-03-09T08:55:18.289 INFO:tasks.workunit.client.0.smithi067.stdout: create 111/37 file after the snapshot 2024-03-09T08:55:18.289 INFO:tasks.workunit.client.0.smithi067.stdout: create 111/38 file after the snapshot 2024-03-09T08:55:18.289 INFO:tasks.workunit.client.0.smithi067.stdout: create 111/39 file after the snapshot 2024-03-09T08:55:18.290 INFO:tasks.workunit.client.0.smithi067.stdout: create 112/.snap/snap-subdir-test 2024-03-09T08:55:18.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.299+0000 7f642e400640 1 -- 172.21.15.67:0/3034774137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f642804f3c0 con 0x7f64080066d0 2024-03-09T08:55:18.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.300+0000 7f6424c00640 1 -- 172.21.15.67:0/3034774137 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7f6414058500 con 0x7f64080066d0 2024-03-09T08:55:18.298 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":26,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:55:18.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.302+0000 7f642e400640 1 -- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f640806d070 msgr2=0x7f640806f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.302+0000 7f642e400640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f640806d070 0x7f640806f530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f641c00f280 tx=0x7f641c008040 comp rx=0 tx=0).stop 2024-03-09T08:55:18.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.302+0000 7f642e400640 1 -- 172.21.15.67:0/3034774137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64080066d0 msgr2=0x7f64281f30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.302+0000 7f642e400640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64080066d0 0x7f64281f30d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6414001260 tx=0x7f641400cb90 comp rx=0 tx=0).stop 2024-03-09T08:55:18.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.302+0000 7f642e400640 1 -- 172.21.15.67:0/3034774137 shutdown_connections 2024-03-09T08:55:18.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.302+0000 7f642e400640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f640806d070 0x7f640806f530 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.302+0000 7f642e400640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6408006b40 0x7f64281f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.302+0000 7f642e400640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f64080066d0 0x7f64281f30d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.302+0000 7f642e400640 1 --2- 172.21.15.67:0/3034774137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6408006250 0x7f64280ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.303+0000 7f642e400640 1 -- 172.21.15.67:0/3034774137 >> 172.21.15.67:0/3034774137 conn(0x7f64280c7300 msgr2=0x7f64280c0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:18.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.303+0000 7f642e400640 1 -- 172.21.15.67:0/3034774137 shutdown_connections 2024-03-09T08:55:18.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.303+0000 7f642e400640 1 -- 172.21.15.67:0/3034774137 wait complete. 2024-03-09T08:55:18.324 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T08:55:18.325 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T08:55:18.325 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:55:18.331 INFO:tasks.workunit.client.1.smithi134.stdout: create 150/30 file after the snapshot 2024-03-09T08:55:18.334 INFO:tasks.workunit.client.1.smithi134.stdout: create 150/31 file after the snapshot 2024-03-09T08:55:18.334 INFO:tasks.workunit.client.1.smithi134.stdout: create 150/32 file after the snapshot 2024-03-09T08:55:18.334 INFO:tasks.workunit.client.1.smithi134.stdout: create 150/33 file after the snapshot 2024-03-09T08:55:18.335 INFO:tasks.workunit.client.1.smithi134.stdout: create 150/34 file after the snapshot 2024-03-09T08:55:18.335 INFO:tasks.workunit.client.1.smithi134.stdout: create 150/35 file after the snapshot 2024-03-09T08:55:18.335 INFO:tasks.workunit.client.1.smithi134.stdout: create 150/36 file after the snapshot 2024-03-09T08:55:18.336 INFO:tasks.workunit.client.1.smithi134.stdout: create 150/37 file after the snapshot 2024-03-09T08:55:18.336 INFO:tasks.workunit.client.1.smithi134.stdout: create 150/38 file after the snapshot 2024-03-09T08:55:18.337 INFO:tasks.workunit.client.1.smithi134.stdout: create 150/39 file after the snapshot 2024-03-09T08:55:18.337 INFO:tasks.workunit.client.1.smithi134.stdout: create 151/.snap/snap-subdir-test 2024-03-09T08:55:18.366 INFO:tasks.workunit.client.0.smithi067.stdout: create 112/30 file after the snapshot 2024-03-09T08:55:18.403 INFO:tasks.workunit.client.1.smithi134.stdout: create 151/30 file after the snapshot 2024-03-09T08:55:18.404 INFO:tasks.workunit.client.1.smithi134.stdout: create 151/31 file after the snapshot 2024-03-09T08:55:18.405 INFO:tasks.workunit.client.1.smithi134.stdout: create 151/32 file after the snapshot 2024-03-09T08:55:18.405 INFO:tasks.workunit.client.1.smithi134.stdout: create 151/33 file after the snapshot 2024-03-09T08:55:18.405 INFO:tasks.workunit.client.1.smithi134.stdout: create 151/34 file after the snapshot 2024-03-09T08:55:18.406 INFO:tasks.workunit.client.1.smithi134.stdout: create 151/35 file after the snapshot 2024-03-09T08:55:18.406 INFO:tasks.workunit.client.1.smithi134.stdout: create 151/36 file after the snapshot 2024-03-09T08:55:18.406 INFO:tasks.workunit.client.1.smithi134.stdout: create 151/37 file after the snapshot 2024-03-09T08:55:18.407 INFO:tasks.workunit.client.1.smithi134.stdout: create 151/38 file after the snapshot 2024-03-09T08:55:18.407 INFO:tasks.workunit.client.1.smithi134.stdout: create 151/39 file after the snapshot 2024-03-09T08:55:18.407 INFO:tasks.workunit.client.1.smithi134.stdout: create 152/.snap/snap-subdir-test 2024-03-09T08:55:18.495 INFO:tasks.workunit.client.0.smithi067.stdout: create 112/31 file after the snapshot 2024-03-09T08:55:18.495 INFO:tasks.workunit.client.0.smithi067.stdout: create 112/32 file after the snapshot 2024-03-09T08:55:18.496 INFO:tasks.workunit.client.0.smithi067.stdout: create 112/33 file after the snapshot 2024-03-09T08:55:18.496 INFO:tasks.workunit.client.0.smithi067.stdout: create 112/34 file after the snapshot 2024-03-09T08:55:18.496 INFO:tasks.workunit.client.0.smithi067.stdout: create 112/35 file after the snapshot 2024-03-09T08:55:18.496 INFO:tasks.workunit.client.0.smithi067.stdout: create 112/36 file after the snapshot 2024-03-09T08:55:18.496 INFO:tasks.workunit.client.0.smithi067.stdout: create 112/37 file after the snapshot 2024-03-09T08:55:18.496 INFO:tasks.workunit.client.0.smithi067.stdout: create 112/38 file after the snapshot 2024-03-09T08:55:18.496 INFO:tasks.workunit.client.0.smithi067.stdout: create 112/39 file after the snapshot 2024-03-09T08:55:18.496 INFO:tasks.workunit.client.0.smithi067.stdout: create 113/.snap/snap-subdir-test 2024-03-09T08:55:18.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.573+0000 7f2b7be00640 1 Processor -- start 2024-03-09T08:55:18.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.573+0000 7f2b7be00640 1 -- start start 2024-03-09T08:55:18.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.575+0000 7f2b7be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2b7c0c9380 0x7f2b7c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.575+0000 7f2b7be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2b7c0ad440 0x7f2b7c06c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.575+0000 7f2b7be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7c0c9890 con 0x7f2b7c14fd90 2024-03-09T08:55:18.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.575+0000 7f2b7be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7c06ba60 con 0x7f2b7c0c9380 2024-03-09T08:55:18.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.575+0000 7f2b7be00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7c0c5f80 con 0x7f2b7c0ad440 2024-03-09T08:55:18.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.576+0000 7f2b7a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2b7c0c9380 0x7f2b7c0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.576+0000 7f2b7a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2b7c0c9380 0x7f2b7c0acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41022/0 (socket says 172.21.15.67:41022) 2024-03-09T08:55:18.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.576+0000 7f2b7a000640 1 -- 172.21.15.67:0/2490878553 learned_addr learned my addr 172.21.15.67:0/2490878553 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:18.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.577+0000 7f2b7a000640 1 -- 172.21.15.67:0/2490878553 >> v2:172.21.15.134:3300/0 conn(0x7f2b7c0ad440 msgr2=0x7f2b7c06c1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T08:55:18.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.577+0000 7f2b7a000640 1 --2- 172.21.15.67:0/2490878553 >> v2:172.21.15.134:3300/0 conn(0x7f2b7c0ad440 0x7f2b7c06c1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.577+0000 7f2b7a000640 1 -- 172.21.15.67:0/2490878553 >> v1:172.21.15.67:6789/0 conn(0x7f2b7c14fd90 legacy=0x7f2b7c156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.577+0000 7f2b7a000640 1 -- 172.21.15.67:0/2490878553 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c0c6240 con 0x7f2b7c0c9380 2024-03-09T08:55:18.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.578+0000 7f2b7a000640 1 --2- 172.21.15.67:0/2490878553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0c9380 0x7f2b7c0acf00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c06d270 tx=0x7f2b6802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ed4a9163c2784d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:18.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.579+0000 7f2b79600640 1 -- 172.21.15.67:0/2490878553 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68035020 con 0x7f2b7c0c9380 2024-03-09T08:55:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.579+0000 7f2b79600640 1 -- 172.21.15.67:0/2490878553 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b68031810 con 0x7f2b7c0c9380 2024-03-09T08:55:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.579+0000 7f2b79600640 1 -- 172.21.15.67:0/2490878553 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68031be0 con 0x7f2b7c0c9380 2024-03-09T08:55:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.580+0000 7f2b7be00640 1 -- 172.21.15.67:0/2490878553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0c9380 msgr2=0x7f2b7c0acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.580+0000 7f2b7be00640 1 --2- 172.21.15.67:0/2490878553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0c9380 0x7f2b7c0acf00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c06d270 tx=0x7f2b6802b8b0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.580+0000 7f2b7be00640 1 -- 172.21.15.67:0/2490878553 shutdown_connections 2024-03-09T08:55:18.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.580+0000 7f2b7be00640 1 --2- 172.21.15.67:0/2490878553 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0c9380 0x7f2b7c0acf00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.580+0000 7f2b7be00640 1 --2- 172.21.15.67:0/2490878553 >> v2:172.21.15.134:3300/0 conn(0x7f2b7c0ad440 0x7f2b7c06c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.580+0000 7f2b7be00640 1 -- 172.21.15.67:0/2490878553 >> 172.21.15.67:0/2490878553 conn(0x7f2b7c0cb480 msgr2=0x7f2b7c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:18.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.581+0000 7f2b7be00640 1 -- 172.21.15.67:0/2490878553 shutdown_connections 2024-03-09T08:55:18.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.581+0000 7f2b7be00640 1 -- 172.21.15.67:0/2490878553 wait complete. 2024-03-09T08:55:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.582+0000 7f2b7be00640 1 Processor -- start 2024-03-09T08:55:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.583+0000 7f2b7be00640 1 -- start start 2024-03-09T08:55:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0ad440 0x7f2b7c1ec920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b7c0c9380 0x7f2b7c159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b7c14fd90 0x7f2b7c0c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c06c710 con 0x7f2b7c0ad440 2024-03-09T08:55:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c06bcb0 con 0x7f2b7c0c9380 2024-03-09T08:55:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c06c890 con 0x7f2b7c14fd90 2024-03-09T08:55:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7a000640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b7c0c9380 msgr2=0x7f2b7c159670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b7c0c9380 0x7f2b7c159670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0ad440 0x7f2b7c1ec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0ad440 0x7f2b7c1ec920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41034/0 (socket says 172.21.15.67:41034) 2024-03-09T08:55:18.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7aa00640 1 -- 172.21.15.67:0/2741487527 learned_addr learned my addr 172.21.15.67:0/2741487527 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:18.583 INFO:tasks.workunit.client.0.smithi067.stdout: create 113/30 file after the snapshot 2024-03-09T08:55:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.584+0000 7f2b7b400640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b7c14fd90 0x7f2b7c0c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.585+0000 7f2b7aa00640 1 -- 172.21.15.67:0/2741487527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b7c14fd90 msgr2=0x7f2b7c0c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.585+0000 7f2b7aa00640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b7c14fd90 0x7f2b7c0c40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.585+0000 7f2b7aa00640 1 -- 172.21.15.67:0/2741487527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b7c0c9380 msgr2=0x7f2b7c159670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:55:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.585+0000 7f2b7aa00640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b7c0c9380 0x7f2b7c159670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.585+0000 7f2b7aa00640 1 -- 172.21.15.67:0/2741487527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c1ebe60 con 0x7f2b7c0ad440 2024-03-09T08:55:18.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.585+0000 7f2b7aa00640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0ad440 0x7f2b7c1ec920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00c400 tx=0x7f2b6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:18.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.585+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c017070 con 0x7f2b7c0ad440 2024-03-09T08:55:18.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.585+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c0052d0 con 0x7f2b7c0ad440 2024-03-09T08:55:18.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.585+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c003830 con 0x7f2b7c0ad440 2024-03-09T08:55:18.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.585+0000 7f2b7be00640 1 -- 172.21.15.67:0/2741487527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c14c910 con 0x7f2b7c0ad440 2024-03-09T08:55:18.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.586+0000 7f2b7be00640 1 -- 172.21.15.67:0/2741487527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c1f0160 con 0x7f2b7c0ad440 2024-03-09T08:55:18.585 INFO:tasks.workunit.client.0.smithi067.stdout: create 113/31 file after the snapshot 2024-03-09T08:55:18.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.587+0000 7f2b7be00640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0b1210 0x7f2b7c1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.587+0000 7f2b7be00640 1 -- 172.21.15.67:0/2741487527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c159bb0 con 0x7f2b7c0b1210 2024-03-09T08:55:18.586 INFO:tasks.workunit.client.0.smithi067.stdout: create 113/32 file after the snapshot 2024-03-09T08:55:18.586 INFO:tasks.workunit.client.0.smithi067.stdout: create 113/33 file after the snapshot 2024-03-09T08:55:18.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.587+0000 7f2b7a000640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0b1210 0x7f2b7c1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.587 INFO:tasks.workunit.client.0.smithi067.stdout: create 113/34 file after the snapshot 2024-03-09T08:55:18.587 INFO:tasks.workunit.client.0.smithi067.stdout: create 113/35 file after the snapshot 2024-03-09T08:55:18.587 INFO:tasks.workunit.client.0.smithi067.stdout: create 113/36 file after the snapshot 2024-03-09T08:55:18.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.588+0000 7f2b7a000640 1 -- 172.21.15.67:0/2741487527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b7c04f3c0 con 0x7f2b7c0b1210 2024-03-09T08:55:18.588 INFO:tasks.workunit.client.0.smithi067.stdout: create 113/37 file after the snapshot 2024-03-09T08:55:18.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.588+0000 7f2b7a000640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0b1210 0x7f2b7c1582f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0ada40 tx=0x7f2b680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:18.588 INFO:tasks.workunit.client.0.smithi067.stdout: create 113/38 file after the snapshot 2024-03-09T08:55:18.588 INFO:tasks.workunit.client.0.smithi067.stdout: create 113/39 file after the snapshot 2024-03-09T08:55:18.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.589+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2b6c01a070 con 0x7f2b7c0ad440 2024-03-09T08:55:18.589 INFO:tasks.workunit.client.0.smithi067.stdout: create 114/.snap/snap-subdir-test 2024-03-09T08:55:18.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.590+0000 7f2b73400640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2b5c06b480 0x7f2b5c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.590+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68031b80 con 0x7f2b7c0b1210 2024-03-09T08:55:18.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.590+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f2b6c089920 con 0x7f2b7c0ad440 2024-03-09T08:55:18.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.590+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f2b6803bce0 con 0x7f2b7c0b1210 2024-03-09T08:55:18.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.590+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0b1210 msgr2=0x7f2b7c1582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.590+0000 7f2b73400640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0b1210 0x7f2b7c1582f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0ada40 tx=0x7f2b680046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.590+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0b1210 msgr2=0x7f2b7c1582f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:18.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.590+0000 7f2b73400640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0b1210 0x7f2b7c1582f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.590+0000 7f2b7b400640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2b5c06b480 0x7f2b5c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.592+0000 7f2b7b400640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2b5c06b480 0x7f2b5c06d940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009ee0 tx=0x7f2b74009450 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:18.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.598+0000 7f2b7be00640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c1ec080 0x7f2b7c0abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.598+0000 7f2b7be00640 1 -- 172.21.15.67:0/2741487527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c06cf30 con 0x7f2b7c1ec080 2024-03-09T08:55:18.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.598+0000 7f2b7a000640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c1ec080 0x7f2b7c0abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.598+0000 7f2b7a000640 1 -- 172.21.15.67:0/2741487527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2b7c06cc50 con 0x7f2b7c1ec080 2024-03-09T08:55:18.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.599+0000 7f2b7a000640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c1ec080 0x7f2b7c0abe30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2b68000c00 tx=0x7f2b68009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:18.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.599+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68003d40 con 0x7f2b7c1ec080 2024-03-09T08:55:18.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.599+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5177 (secure 0 0 0) 0x7f2b68009490 con 0x7f2b7c1ec080 2024-03-09T08:55:18.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.599+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c1ec080 msgr2=0x7f2b7c0abe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.599+0000 7f2b73400640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c1ec080 0x7f2b7c0abe30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2b68000c00 tx=0x7f2b68009240 comp rx=0 tx=0).stop 2024-03-09T08:55:18.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.599+0000 7f2b73400640 1 -- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c1ec080 msgr2=0x7f2b7c0abe30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:18.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.599+0000 7f2b73400640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c1ec080 0x7f2b7c0abe30 secure :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2b68000c00 tx=0x7f2b68009240 comp rx=0 tx=0).stop 2024-03-09T08:55:18.598 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:55:18.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:18.598 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:18.598 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:55:18.598 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 26, 2024-03-09T08:55:18.598 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:18.599 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:18.600 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:55:18.601 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:18.602 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.603 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:18.604 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:18.604 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:18.604 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:18.604 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.604 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:18.604 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:18.605 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:18.607 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.609 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:55:18.609 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:55:18.609 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:18.609 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:18.609 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:18.609 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.609 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.609 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:55:18.609 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:55:18.609 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.610 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:55:18.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.613 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:55:18.613 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:55:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.600+0000 7f2b7be00640 1 -- 172.21.15.67:0/2741487527 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2b5c06b480 msgr2=0x7f2b5c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.600+0000 7f2b7be00640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2b5c06b480 0x7f2b5c06d940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009ee0 tx=0x7f2b74009450 comp rx=0 tx=0).stop 2024-03-09T08:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.600+0000 7f2b7be00640 1 -- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0ad440 msgr2=0x7f2b7c1ec920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.600+0000 7f2b7be00640 1 --2- 172.21.15.67:0/2741487527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b7c0ad440 0x7f2b7c1ec920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00c400 tx=0x7f2b6c00c8c0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.600+0000 7f2b7b400640 1 -- 172.21.15.67:0/2741487527 reap_dead start 2024-03-09T08:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.600+0000 7f2b7b400640 1 -- 172.21.15.67:0/2741487527 reap_dead start 2024-03-09T08:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.600+0000 7f2b7be00640 1 -- 172.21.15.67:0/2741487527 shutdown_connections 2024-03-09T08:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.600+0000 7f2b7be00640 1 -- 172.21.15.67:0/2741487527 >> 172.21.15.67:0/2741487527 conn(0x7f2b7c0cb480 msgr2=0x7f2b7c0c9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.601+0000 7f2b7be00640 1 -- 172.21.15.67:0/2741487527 shutdown_connections 2024-03-09T08:55:18.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.601+0000 7f2b7be00640 1 -- 172.21.15.67:0/2741487527 wait complete. 2024-03-09T08:55:18.615 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:55:18.615 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T08:55:18.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:55:18.666 INFO:tasks.workunit.client.0.smithi067.stdout: create 114/30 file after the snapshot 2024-03-09T08:55:18.668 INFO:tasks.workunit.client.0.smithi067.stdout: create 114/31 file after the snapshot 2024-03-09T08:55:18.668 INFO:tasks.workunit.client.0.smithi067.stdout: create 114/32 file after the snapshot 2024-03-09T08:55:18.668 INFO:tasks.workunit.client.0.smithi067.stdout: create 114/33 file after the snapshot 2024-03-09T08:55:18.668 INFO:tasks.workunit.client.0.smithi067.stdout: create 114/34 file after the snapshot 2024-03-09T08:55:18.668 INFO:tasks.workunit.client.0.smithi067.stdout: create 114/35 file after the snapshot 2024-03-09T08:55:18.668 INFO:tasks.workunit.client.0.smithi067.stdout: create 114/36 file after the snapshot 2024-03-09T08:55:18.668 INFO:tasks.workunit.client.0.smithi067.stdout: create 114/37 file after the snapshot 2024-03-09T08:55:18.669 INFO:tasks.workunit.client.0.smithi067.stdout: create 114/38 file after the snapshot 2024-03-09T08:55:18.669 INFO:tasks.workunit.client.0.smithi067.stdout: create 114/39 file after the snapshot 2024-03-09T08:55:18.669 INFO:tasks.workunit.client.0.smithi067.stdout: create 115/.snap/snap-subdir-test 2024-03-09T08:55:18.738 INFO:tasks.workunit.client.0.smithi067.stdout: create 115/30 file after the snapshot 2024-03-09T08:55:18.740 INFO:tasks.workunit.client.0.smithi067.stdout: create 115/31 file after the snapshot 2024-03-09T08:55:18.740 INFO:tasks.workunit.client.0.smithi067.stdout: create 115/32 file after the snapshot 2024-03-09T08:55:18.741 INFO:tasks.workunit.client.0.smithi067.stdout: create 115/33 file after the snapshot 2024-03-09T08:55:18.741 INFO:tasks.workunit.client.0.smithi067.stdout: create 115/34 file after the snapshot 2024-03-09T08:55:18.741 INFO:tasks.workunit.client.0.smithi067.stdout: create 115/35 file after the snapshot 2024-03-09T08:55:18.741 INFO:tasks.workunit.client.0.smithi067.stdout: create 115/36 file after the snapshot 2024-03-09T08:55:18.742 INFO:tasks.workunit.client.0.smithi067.stdout: create 115/37 file after the snapshot 2024-03-09T08:55:18.742 INFO:tasks.workunit.client.0.smithi067.stdout: create 115/38 file after the snapshot 2024-03-09T08:55:18.742 INFO:tasks.workunit.client.0.smithi067.stdout: create 115/39 file after the snapshot 2024-03-09T08:55:18.742 INFO:tasks.workunit.client.0.smithi067.stdout: create 116/.snap/snap-subdir-test 2024-03-09T08:55:18.811 INFO:tasks.workunit.client.0.smithi067.stdout: create 116/30 file after the snapshot 2024-03-09T08:55:18.814 INFO:tasks.workunit.client.0.smithi067.stdout: create 116/31 file after the snapshot 2024-03-09T08:55:18.814 INFO:tasks.workunit.client.0.smithi067.stdout: create 116/32 file after the snapshot 2024-03-09T08:55:18.814 INFO:tasks.workunit.client.0.smithi067.stdout: create 116/33 file after the snapshot 2024-03-09T08:55:18.814 INFO:tasks.workunit.client.0.smithi067.stdout: create 116/34 file after the snapshot 2024-03-09T08:55:18.814 INFO:tasks.workunit.client.0.smithi067.stdout: create 116/35 file after the snapshot 2024-03-09T08:55:18.814 INFO:tasks.workunit.client.0.smithi067.stdout: create 116/36 file after the snapshot 2024-03-09T08:55:18.814 INFO:tasks.workunit.client.0.smithi067.stdout: create 116/37 file after the snapshot 2024-03-09T08:55:18.815 INFO:tasks.workunit.client.0.smithi067.stdout: create 116/38 file after the snapshot 2024-03-09T08:55:18.815 INFO:tasks.workunit.client.0.smithi067.stdout: create 116/39 file after the snapshot 2024-03-09T08:55:18.815 INFO:tasks.workunit.client.0.smithi067.stdout: create 117/.snap/snap-subdir-test 2024-03-09T08:55:18.862 INFO:tasks.workunit.client.1.smithi134.stdout: create 152/30 file after the snapshot 2024-03-09T08:55:18.863 INFO:tasks.workunit.client.1.smithi134.stdout: create 152/31 file after the snapshot 2024-03-09T08:55:18.863 INFO:tasks.workunit.client.1.smithi134.stdout: create 152/32 file after the snapshot 2024-03-09T08:55:18.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 152/33 file after the snapshot 2024-03-09T08:55:18.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 152/34 file after the snapshot 2024-03-09T08:55:18.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 152/35 file after the snapshot 2024-03-09T08:55:18.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 152/36 file after the snapshot 2024-03-09T08:55:18.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 152/37 file after the snapshot 2024-03-09T08:55:18.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 152/38 file after the snapshot 2024-03-09T08:55:18.865 INFO:tasks.workunit.client.1.smithi134.stdout: create 152/39 file after the snapshot 2024-03-09T08:55:18.865 INFO:tasks.workunit.client.1.smithi134.stdout: create 153/.snap/snap-subdir-test 2024-03-09T08:55:18.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.870+0000 7ff2f4c00640 1 Processor -- start 2024-03-09T08:55:18.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.870+0000 7ff2f4c00640 1 -- start start 2024-03-09T08:55:18.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.872+0000 7ff2f4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff2f006c1d0 0x7ff2f006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.873+0000 7ff2f4c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f00c9890 con 0x7ff2f00c9380 2024-03-09T08:55:18.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.873+0000 7ff2f4c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f00c5f80 con 0x7ff2f014fd90 2024-03-09T08:55:18.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.873+0000 7ff2f4c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f006ba60 con 0x7ff2f006c1d0 2024-03-09T08:55:18.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.873+0000 7ff2ef400640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7ff2f014fd90 legacy=0x7ff2f0156680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T08:55:18.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.873+0000 7ff2eea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff2f00c9380 0x7ff2f00acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38410/0 (socket says 172.21.15.67:38410) 2024-03-09T08:55:18.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.873+0000 7ff2eea00640 1 -- 172.21.15.67:0/1031890462 learned_addr learned my addr 172.21.15.67:0/1031890462 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:18.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.873+0000 7ff2efe00640 1 --2- 172.21.15.67:0/1031890462 >> v2:172.21.15.67:3300/0 conn(0x7ff2f006c1d0 0x7ff2f006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.874+0000 7ff2efe00640 1 -- 172.21.15.67:0/1031890462 >> v1:172.21.15.134:6789/0 conn(0x7ff2f014fd90 legacy=0x7ff2f0156680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:55:18.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.874+0000 7ff2efe00640 1 -- 172.21.15.67:0/1031890462 >> v1:172.21.15.67:6789/0 conn(0x7ff2f00c9380 legacy=0x7ff2f00acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.874+0000 7ff2efe00640 1 -- 172.21.15.67:0/1031890462 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f00c6240 con 0x7ff2f006c1d0 2024-03-09T08:55:18.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.875+0000 7ff2efe00640 1 --2- 172.21.15.67:0/1031890462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f006c1d0 0x7ff2f006c5f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff2f006bbe0 tx=0x7ff2e802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb723993a2b48bc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:18.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.875+0000 7ff2ee000640 1 -- 172.21.15.67:0/1031890462 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8039070 con 0x7ff2f006c1d0 2024-03-09T08:55:18.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.875+0000 7ff2ee000640 1 -- 172.21.15.67:0/1031890462 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e802cb40 con 0x7ff2f006c1d0 2024-03-09T08:55:18.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.875+0000 7ff2ee000640 1 -- 172.21.15.67:0/1031890462 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8034440 con 0x7ff2f006c1d0 2024-03-09T08:55:18.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.876+0000 7ff2f4c00640 1 -- 172.21.15.67:0/1031890462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f006c1d0 msgr2=0x7ff2f006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.876+0000 7ff2f4c00640 1 --2- 172.21.15.67:0/1031890462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f006c1d0 0x7ff2f006c5f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff2f006bbe0 tx=0x7ff2e802bd90 comp rx=0 tx=0).stop 2024-03-09T08:55:18.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.876+0000 7ff2f4c00640 1 -- 172.21.15.67:0/1031890462 shutdown_connections 2024-03-09T08:55:18.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.876+0000 7ff2f4c00640 1 --2- 172.21.15.67:0/1031890462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f006c1d0 0x7ff2f006c5f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.876+0000 7ff2f4c00640 1 -- 172.21.15.67:0/1031890462 >> 172.21.15.67:0/1031890462 conn(0x7ff2f00cb480 msgr2=0x7ff2f00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:18.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.876+0000 7ff2f4c00640 1 -- 172.21.15.67:0/1031890462 shutdown_connections 2024-03-09T08:55:18.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.876+0000 7ff2f4c00640 1 -- 172.21.15.67:0/1031890462 wait complete. 2024-03-09T08:55:18.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.879+0000 7ff2f4c00640 1 Processor -- start 2024-03-09T08:55:18.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.879+0000 7ff2f4c00640 1 -- start start 2024-03-09T08:55:18.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.880+0000 7ff2f4c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f006c1d0 0x7ff2f00ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.880+0000 7ff2f4c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f00c9380 0x7ff2f0159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.880+0000 7ff2f4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f014fd90 0x7ff2f01596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.880+0000 7ff2f4c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f00ad610 con 0x7ff2f014fd90 2024-03-09T08:55:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.880+0000 7ff2f4c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f006bcb0 con 0x7ff2f00c9380 2024-03-09T08:55:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.880+0000 7ff2f4c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f00ad790 con 0x7ff2f006c1d0 2024-03-09T08:55:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2eea00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f00c9380 msgr2=0x7ff2f0159180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T08:55:18.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2eea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f00c9380 0x7ff2f0159180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:18.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f014fd90 0x7ff2f01596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2ef400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f006c1d0 0x7ff2f00ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f014fd90 0x7ff2f01596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37540/0 (socket says 172.21.15.67:37540) 2024-03-09T08:55:18.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2efe00640 1 -- 172.21.15.67:0/497067481 learned_addr learned my addr 172.21.15.67:0/497067481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:18.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2efe00640 1 -- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f006c1d0 msgr2=0x7ff2f00ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2efe00640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f006c1d0 0x7ff2f00ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2efe00640 1 -- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f00c9380 msgr2=0x7ff2f0159180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:55:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2efe00640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2f00c9380 0x7ff2f0159180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2efe00640 1 -- 172.21.15.67:0/497067481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f00ae1f0 con 0x7ff2f014fd90 2024-03-09T08:55:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2efe00640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f014fd90 0x7ff2f01596c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8037040 tx=0x7ff2e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8044020 con 0x7ff2f014fd90 2024-03-09T08:55:18.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e803f470 con 0x7ff2f014fd90 2024-03-09T08:55:18.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8039040 con 0x7ff2f014fd90 2024-03-09T08:55:18.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2f4c00640 1 -- 172.21.15.67:0/497067481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f00c5530 con 0x7ff2f014fd90 2024-03-09T08:55:18.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2f4c00640 1 -- 172.21.15.67:0/497067481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f0158c60 con 0x7ff2f014fd90 2024-03-09T08:55:18.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2f4c00640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0159c00 0x7ff2f0148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2f4c00640 1 -- 172.21.15.67:0/497067481 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f004f3c0 con 0x7ff2f0159c00 2024-03-09T08:55:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff2e8047020 con 0x7ff2f014fd90 2024-03-09T08:55:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2ef400640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0159c00 0x7ff2f0148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.881+0000 7ff2ef400640 1 -- 172.21.15.67:0/497067481 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2f006d2d0 con 0x7ff2f0159c00 2024-03-09T08:55:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.887+0000 7ff2ef400640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0159c00 0x7ff2f0148210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0003930 tx=0x7ff2e0009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.887+0000 7ff2e7400640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2d006b500 0x7ff2d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.887+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7ff2e80ace10 con 0x7ff2f014fd90 2024-03-09T08:55:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.888+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0016070 con 0x7ff2f0159c00 2024-03-09T08:55:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.888+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff2e0012600 con 0x7ff2f0159c00 2024-03-09T08:55:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.888+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0159c00 msgr2=0x7ff2f0148210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.888+0000 7ff2e7400640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0159c00 0x7ff2f0148210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0003930 tx=0x7ff2e0009b00 comp rx=0 tx=0).stop 2024-03-09T08:55:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.888+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0159c00 msgr2=0x7ff2f0148210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.888+0000 7ff2e7400640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f0159c00 0x7ff2f0148210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.888+0000 7ff2eea00640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2d006b500 0x7ff2d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.890+0000 7ff2eea00640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2d006b500 0x7ff2d006d9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00ae650 tx=0x7ff2dc007b00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:18.890 INFO:tasks.workunit.client.0.smithi067.stdout: create 117/30 file after the snapshot 2024-03-09T08:55:18.892 INFO:tasks.workunit.client.0.smithi067.stdout: create 117/31 file after the snapshot 2024-03-09T08:55:18.892 INFO:tasks.workunit.client.0.smithi067.stdout: create 117/32 file after the snapshot 2024-03-09T08:55:18.892 INFO:tasks.workunit.client.0.smithi067.stdout: create 117/33 file after the snapshot 2024-03-09T08:55:18.892 INFO:tasks.workunit.client.0.smithi067.stdout: create 117/34 file after the snapshot 2024-03-09T08:55:18.892 INFO:tasks.workunit.client.0.smithi067.stdout: create 117/35 file after the snapshot 2024-03-09T08:55:18.892 INFO:tasks.workunit.client.0.smithi067.stdout: create 117/36 file after the snapshot 2024-03-09T08:55:18.892 INFO:tasks.workunit.client.0.smithi067.stdout: create 117/37 file after the snapshot 2024-03-09T08:55:18.892 INFO:tasks.workunit.client.0.smithi067.stdout: create 117/38 file after the snapshot 2024-03-09T08:55:18.892 INFO:tasks.workunit.client.0.smithi067.stdout: create 117/39 file after the snapshot 2024-03-09T08:55:18.893 INFO:tasks.workunit.client.0.smithi067.stdout: create 118/.snap/snap-subdir-test 2024-03-09T08:55:18.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.903+0000 7ff2f4c00640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f00b04d0 0x7ff2f0148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:18.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.903+0000 7ff2f4c00640 1 -- 172.21.15.67:0/497067481 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f00ca710 con 0x7ff2f00b04d0 2024-03-09T08:55:18.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.904+0000 7ff2ef400640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f00b04d0 0x7ff2f0148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:18.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.904+0000 7ff2ef400640 1 -- 172.21.15.67:0/497067481 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff2f0158ea0 con 0x7ff2f00b04d0 2024-03-09T08:55:18.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.905+0000 7ff2ef400640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f00b04d0 0x7ff2f0148750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0003930 tx=0x7ff2e00059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:18.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.905+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0016040 con 0x7ff2f00b04d0 2024-03-09T08:55:18.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4862 (secure 0 0 0) 0x7ff2e001d0d0 con 0x7ff2f00b04d0 2024-03-09T08:55:18.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f00b04d0 msgr2=0x7ff2f0148750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2e7400640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f00b04d0 0x7ff2f0148750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0003930 tx=0x7ff2e00059d0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2e7400640 1 -- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f00b04d0 msgr2=0x7ff2f0148750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:18.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2e7400640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2f00b04d0 0x7ff2f0148750 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0003930 tx=0x7ff2e00059d0 comp rx=0 tx=0).stop 2024-03-09T08:55:18.904 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:55:18.904 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:18.905 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:18.905 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:55:18.905 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 26, 2024-03-09T08:55:18.905 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:55:18.905 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:55:18.905 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:55:18.905 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.905 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T08:55:18.905 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:18.905 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:55:18.905 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:55:18.906 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:18.906 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:18.906 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:18.906 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:18.906 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:18.906 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:18.906 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:18.906 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:18.906 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:18.907 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:18.907 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:18.907 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.907 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:55:18.907 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:55:18.907 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:18.907 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:18.907 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:18.907 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:18.907 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:55:18.908 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:55:18.909 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.910 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.911 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:55:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:55:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.912 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:55:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.914 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:18.915 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:18.916 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:18.917 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:55:18.918 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:55:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2f4c00640 1 -- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2d006b500 msgr2=0x7ff2d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2f4c00640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2d006b500 0x7ff2d006d9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00ae650 tx=0x7ff2dc007b00 comp rx=0 tx=0).stop 2024-03-09T08:55:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2f4c00640 1 -- 172.21.15.67:0/497067481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f014fd90 msgr2=0x7ff2f01596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2f4c00640 1 --2- 172.21.15.67:0/497067481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f014fd90 0x7ff2f01596c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8037040 tx=0x7ff2e8004a40 comp rx=0 tx=0).stop 2024-03-09T08:55:18.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2efe00640 1 -- 172.21.15.67:0/497067481 reap_dead start 2024-03-09T08:55:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2efe00640 1 -- 172.21.15.67:0/497067481 reap_dead start 2024-03-09T08:55:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2f4c00640 1 -- 172.21.15.67:0/497067481 shutdown_connections 2024-03-09T08:55:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2f4c00640 1 -- 172.21.15.67:0/497067481 >> 172.21.15.67:0/497067481 conn(0x7ff2f00cb480 msgr2=0x7ff2f00c9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2f4c00640 1 -- 172.21.15.67:0/497067481 shutdown_connections 2024-03-09T08:55:18.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:18.906+0000 7ff2f4c00640 1 -- 172.21.15.67:0/497067481 wait complete. 2024-03-09T08:55:18.921 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:55:18.921 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T08:55:18.927 INFO:tasks.workunit.client.1.smithi134.stdout: create 153/30 file after the snapshot 2024-03-09T08:55:18.929 INFO:tasks.workunit.client.1.smithi134.stdout: create 153/31 file after the snapshot 2024-03-09T08:55:18.930 INFO:tasks.workunit.client.1.smithi134.stdout: create 153/32 file after the snapshot 2024-03-09T08:55:18.930 INFO:tasks.workunit.client.1.smithi134.stdout: create 153/33 file after the snapshot 2024-03-09T08:55:18.930 INFO:tasks.workunit.client.1.smithi134.stdout: create 153/34 file after the snapshot 2024-03-09T08:55:18.931 INFO:tasks.workunit.client.1.smithi134.stdout: create 153/35 file after the snapshot 2024-03-09T08:55:18.931 INFO:tasks.workunit.client.1.smithi134.stdout: create 153/36 file after the snapshot 2024-03-09T08:55:18.931 INFO:tasks.workunit.client.1.smithi134.stdout: create 153/37 file after the snapshot 2024-03-09T08:55:18.931 INFO:tasks.workunit.client.1.smithi134.stdout: create 153/38 file after the snapshot 2024-03-09T08:55:18.932 INFO:tasks.workunit.client.1.smithi134.stdout: create 153/39 file after the snapshot 2024-03-09T08:55:18.932 INFO:tasks.workunit.client.1.smithi134.stdout: create 154/.snap/snap-subdir-test 2024-03-09T08:55:18.959 INFO:tasks.workunit.client.0.smithi067.stdout: create 118/30 file after the snapshot 2024-03-09T08:55:19.001 INFO:tasks.workunit.client.1.smithi134.stdout: create 154/30 file after the snapshot 2024-03-09T08:55:19.004 INFO:tasks.workunit.client.1.smithi134.stdout: create 154/31 file after the snapshot 2024-03-09T08:55:19.005 INFO:tasks.workunit.client.1.smithi134.stdout: create 154/32 file after the snapshot 2024-03-09T08:55:19.005 INFO:tasks.workunit.client.1.smithi134.stdout: create 154/33 file after the snapshot 2024-03-09T08:55:19.005 INFO:tasks.workunit.client.1.smithi134.stdout: create 154/34 file after the snapshot 2024-03-09T08:55:19.005 INFO:tasks.workunit.client.1.smithi134.stdout: create 154/35 file after the snapshot 2024-03-09T08:55:19.006 INFO:tasks.workunit.client.1.smithi134.stdout: create 154/36 file after the snapshot 2024-03-09T08:55:19.006 INFO:tasks.workunit.client.1.smithi134.stdout: create 154/37 file after the snapshot 2024-03-09T08:55:19.006 INFO:tasks.workunit.client.1.smithi134.stdout: create 154/38 file after the snapshot 2024-03-09T08:55:19.007 INFO:tasks.workunit.client.1.smithi134.stdout: create 154/39 file after the snapshot 2024-03-09T08:55:19.007 INFO:tasks.workunit.client.1.smithi134.stdout: create 155/.snap/snap-subdir-test 2024-03-09T08:55:19.082 INFO:tasks.workunit.client.1.smithi134.stdout: create 155/30 file after the snapshot 2024-03-09T08:55:19.084 INFO:tasks.workunit.client.1.smithi134.stdout: create 155/31 file after the snapshot 2024-03-09T08:55:19.084 INFO:tasks.workunit.client.1.smithi134.stdout: create 155/32 file after the snapshot 2024-03-09T08:55:19.085 INFO:tasks.workunit.client.1.smithi134.stdout: create 155/33 file after the snapshot 2024-03-09T08:55:19.085 INFO:tasks.workunit.client.1.smithi134.stdout: create 155/34 file after the snapshot 2024-03-09T08:55:19.086 INFO:tasks.workunit.client.1.smithi134.stdout: create 155/35 file after the snapshot 2024-03-09T08:55:19.086 INFO:tasks.workunit.client.1.smithi134.stdout: create 155/36 file after the snapshot 2024-03-09T08:55:19.086 INFO:tasks.workunit.client.1.smithi134.stdout: create 155/37 file after the snapshot 2024-03-09T08:55:19.086 INFO:tasks.workunit.client.1.smithi134.stdout: create 155/38 file after the snapshot 2024-03-09T08:55:19.087 INFO:tasks.workunit.client.1.smithi134.stdout: create 155/39 file after the snapshot 2024-03-09T08:55:19.087 INFO:tasks.workunit.client.1.smithi134.stdout: create 156/.snap/snap-subdir-test 2024-03-09T08:55:19.191 INFO:tasks.workunit.client.1.smithi134.stdout: create 156/30 file after the snapshot 2024-03-09T08:55:19.194 INFO:tasks.workunit.client.1.smithi134.stdout: create 156/31 file after the snapshot 2024-03-09T08:55:19.194 INFO:tasks.workunit.client.1.smithi134.stdout: create 156/32 file after the snapshot 2024-03-09T08:55:19.195 INFO:tasks.workunit.client.1.smithi134.stdout: create 156/33 file after the snapshot 2024-03-09T08:55:19.195 INFO:tasks.workunit.client.1.smithi134.stdout: create 156/34 file after the snapshot 2024-03-09T08:55:19.195 INFO:tasks.workunit.client.1.smithi134.stdout: create 156/35 file after the snapshot 2024-03-09T08:55:19.195 INFO:tasks.workunit.client.1.smithi134.stdout: create 156/36 file after the snapshot 2024-03-09T08:55:19.195 INFO:tasks.workunit.client.1.smithi134.stdout: create 156/37 file after the snapshot 2024-03-09T08:55:19.196 INFO:tasks.workunit.client.1.smithi134.stdout: create 156/38 file after the snapshot 2024-03-09T08:55:19.196 INFO:tasks.workunit.client.1.smithi134.stdout: create 156/39 file after the snapshot 2024-03-09T08:55:19.196 INFO:tasks.workunit.client.1.smithi134.stdout: create 157/.snap/snap-subdir-test 2024-03-09T08:55:19.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 157/30 file after the snapshot 2024-03-09T08:55:19.277 INFO:tasks.workunit.client.1.smithi134.stdout: create 157/31 file after the snapshot 2024-03-09T08:55:19.277 INFO:tasks.workunit.client.1.smithi134.stdout: create 157/32 file after the snapshot 2024-03-09T08:55:19.277 INFO:tasks.workunit.client.1.smithi134.stdout: create 157/33 file after the snapshot 2024-03-09T08:55:19.277 INFO:tasks.workunit.client.1.smithi134.stdout: create 157/34 file after the snapshot 2024-03-09T08:55:19.278 INFO:tasks.workunit.client.1.smithi134.stdout: create 157/35 file after the snapshot 2024-03-09T08:55:19.278 INFO:tasks.workunit.client.1.smithi134.stdout: create 157/36 file after the snapshot 2024-03-09T08:55:19.278 INFO:tasks.workunit.client.1.smithi134.stdout: create 157/37 file after the snapshot 2024-03-09T08:55:19.278 INFO:tasks.workunit.client.1.smithi134.stdout: create 157/38 file after the snapshot 2024-03-09T08:55:19.278 INFO:tasks.workunit.client.1.smithi134.stdout: create 157/39 file after the snapshot 2024-03-09T08:55:19.279 INFO:tasks.workunit.client.1.smithi134.stdout: create 158/.snap/snap-subdir-test 2024-03-09T08:55:19.367 INFO:tasks.workunit.client.1.smithi134.stdout: create 158/30 file after the snapshot 2024-03-09T08:55:19.370 INFO:tasks.workunit.client.1.smithi134.stdout: create 158/31 file after the snapshot 2024-03-09T08:55:19.370 INFO:tasks.workunit.client.1.smithi134.stdout: create 158/32 file after the snapshot 2024-03-09T08:55:19.370 INFO:tasks.workunit.client.1.smithi134.stdout: create 158/33 file after the snapshot 2024-03-09T08:55:19.370 INFO:tasks.workunit.client.1.smithi134.stdout: create 158/34 file after the snapshot 2024-03-09T08:55:19.371 INFO:tasks.workunit.client.1.smithi134.stdout: create 158/35 file after the snapshot 2024-03-09T08:55:19.371 INFO:tasks.workunit.client.1.smithi134.stdout: create 158/36 file after the snapshot 2024-03-09T08:55:19.371 INFO:tasks.workunit.client.1.smithi134.stdout: create 158/37 file after the snapshot 2024-03-09T08:55:19.372 INFO:tasks.workunit.client.1.smithi134.stdout: create 158/38 file after the snapshot 2024-03-09T08:55:19.372 INFO:tasks.workunit.client.1.smithi134.stdout: create 158/39 file after the snapshot 2024-03-09T08:55:19.372 INFO:tasks.workunit.client.1.smithi134.stdout: create 159/.snap/snap-subdir-test 2024-03-09T08:55:19.447 INFO:tasks.workunit.client.1.smithi134.stdout: create 159/30 file after the snapshot 2024-03-09T08:55:19.451 INFO:tasks.workunit.client.1.smithi134.stdout: create 159/31 file after the snapshot 2024-03-09T08:55:19.451 INFO:tasks.workunit.client.1.smithi134.stdout: create 159/32 file after the snapshot 2024-03-09T08:55:19.451 INFO:tasks.workunit.client.1.smithi134.stdout: create 159/33 file after the snapshot 2024-03-09T08:55:19.453 INFO:tasks.workunit.client.1.smithi134.stdout: create 159/34 file after the snapshot 2024-03-09T08:55:19.453 INFO:tasks.workunit.client.1.smithi134.stdout: create 159/35 file after the snapshot 2024-03-09T08:55:19.453 INFO:tasks.workunit.client.1.smithi134.stdout: create 159/36 file after the snapshot 2024-03-09T08:55:19.453 INFO:tasks.workunit.client.1.smithi134.stdout: create 159/37 file after the snapshot 2024-03-09T08:55:19.453 INFO:tasks.workunit.client.1.smithi134.stdout: create 159/38 file after the snapshot 2024-03-09T08:55:19.453 INFO:tasks.workunit.client.1.smithi134.stdout: create 159/39 file after the snapshot 2024-03-09T08:55:19.453 INFO:tasks.workunit.client.1.smithi134.stdout: create 160/.snap/snap-subdir-test 2024-03-09T08:55:19.473 INFO:tasks.workunit.client.0.smithi067.stdout: create 118/31 file after the snapshot 2024-03-09T08:55:19.474 INFO:tasks.workunit.client.0.smithi067.stdout: create 118/32 file after the snapshot 2024-03-09T08:55:19.474 INFO:tasks.workunit.client.0.smithi067.stdout: create 118/33 file after the snapshot 2024-03-09T08:55:19.475 INFO:tasks.workunit.client.0.smithi067.stdout: create 118/34 file after the snapshot 2024-03-09T08:55:19.476 INFO:tasks.workunit.client.0.smithi067.stdout: create 118/35 file after the snapshot 2024-03-09T08:55:19.476 INFO:tasks.workunit.client.0.smithi067.stdout: create 118/36 file after the snapshot 2024-03-09T08:55:19.477 INFO:tasks.workunit.client.0.smithi067.stdout: create 118/37 file after the snapshot 2024-03-09T08:55:19.477 INFO:tasks.workunit.client.0.smithi067.stdout: create 118/38 file after the snapshot 2024-03-09T08:55:19.477 INFO:tasks.workunit.client.0.smithi067.stdout: create 118/39 file after the snapshot 2024-03-09T08:55:19.477 INFO:tasks.workunit.client.0.smithi067.stdout: create 119/.snap/snap-subdir-test 2024-03-09T08:55:19.590 INFO:tasks.workunit.client.0.smithi067.stdout: create 119/30 file after the snapshot 2024-03-09T08:55:19.594 INFO:tasks.workunit.client.0.smithi067.stdout: create 119/31 file after the snapshot 2024-03-09T08:55:19.594 INFO:tasks.workunit.client.0.smithi067.stdout: create 119/32 file after the snapshot 2024-03-09T08:55:19.594 INFO:tasks.workunit.client.0.smithi067.stdout: create 119/33 file after the snapshot 2024-03-09T08:55:19.595 INFO:tasks.workunit.client.0.smithi067.stdout: create 119/34 file after the snapshot 2024-03-09T08:55:19.595 INFO:tasks.workunit.client.0.smithi067.stdout: create 119/35 file after the snapshot 2024-03-09T08:55:19.595 INFO:tasks.workunit.client.0.smithi067.stdout: create 119/36 file after the snapshot 2024-03-09T08:55:19.595 INFO:tasks.workunit.client.0.smithi067.stdout: create 119/37 file after the snapshot 2024-03-09T08:55:19.596 INFO:tasks.workunit.client.0.smithi067.stdout: create 119/38 file after the snapshot 2024-03-09T08:55:19.596 INFO:tasks.workunit.client.0.smithi067.stdout: create 119/39 file after the snapshot 2024-03-09T08:55:19.596 INFO:tasks.workunit.client.0.smithi067.stdout: create 120/.snap/snap-subdir-test 2024-03-09T08:55:19.652 INFO:tasks.workunit.client.0.smithi067.stdout: create 120/30 file after the snapshot 2024-03-09T08:55:19.654 INFO:tasks.workunit.client.0.smithi067.stdout: create 120/31 file after the snapshot 2024-03-09T08:55:19.654 INFO:tasks.workunit.client.0.smithi067.stdout: create 120/32 file after the snapshot 2024-03-09T08:55:19.655 INFO:tasks.workunit.client.0.smithi067.stdout: create 120/33 file after the snapshot 2024-03-09T08:55:19.655 INFO:tasks.workunit.client.0.smithi067.stdout: create 120/34 file after the snapshot 2024-03-09T08:55:19.655 INFO:tasks.workunit.client.0.smithi067.stdout: create 120/35 file after the snapshot 2024-03-09T08:55:19.656 INFO:tasks.workunit.client.0.smithi067.stdout: create 120/36 file after the snapshot 2024-03-09T08:55:19.656 INFO:tasks.workunit.client.0.smithi067.stdout: create 120/37 file after the snapshot 2024-03-09T08:55:19.656 INFO:tasks.workunit.client.0.smithi067.stdout: create 120/38 file after the snapshot 2024-03-09T08:55:19.656 INFO:tasks.workunit.client.0.smithi067.stdout: create 120/39 file after the snapshot 2024-03-09T08:55:19.656 INFO:tasks.workunit.client.0.smithi067.stdout: create 121/.snap/snap-subdir-test 2024-03-09T08:55:19.731 INFO:tasks.workunit.client.0.smithi067.stdout: create 121/30 file after the snapshot 2024-03-09T08:55:19.761 INFO:tasks.workunit.client.1.smithi134.stdout: create 160/30 file after the snapshot 2024-03-09T08:55:19.764 INFO:tasks.workunit.client.1.smithi134.stdout: create 160/31 file after the snapshot 2024-03-09T08:55:19.764 INFO:tasks.workunit.client.1.smithi134.stdout: create 160/32 file after the snapshot 2024-03-09T08:55:19.764 INFO:tasks.workunit.client.1.smithi134.stdout: create 160/33 file after the snapshot 2024-03-09T08:55:19.765 INFO:tasks.workunit.client.1.smithi134.stdout: create 160/34 file after the snapshot 2024-03-09T08:55:19.765 INFO:tasks.workunit.client.1.smithi134.stdout: create 160/35 file after the snapshot 2024-03-09T08:55:19.765 INFO:tasks.workunit.client.1.smithi134.stdout: create 160/36 file after the snapshot 2024-03-09T08:55:19.765 INFO:tasks.workunit.client.1.smithi134.stdout: create 160/37 file after the snapshot 2024-03-09T08:55:19.765 INFO:tasks.workunit.client.1.smithi134.stdout: create 160/38 file after the snapshot 2024-03-09T08:55:19.766 INFO:tasks.workunit.client.1.smithi134.stdout: create 160/39 file after the snapshot 2024-03-09T08:55:19.766 INFO:tasks.workunit.client.1.smithi134.stdout: create 161/.snap/snap-subdir-test 2024-03-09T08:55:19.841 INFO:tasks.workunit.client.1.smithi134.stdout: create 161/30 file after the snapshot 2024-03-09T08:55:19.844 INFO:tasks.workunit.client.1.smithi134.stdout: create 161/31 file after the snapshot 2024-03-09T08:55:19.844 INFO:tasks.workunit.client.1.smithi134.stdout: create 161/32 file after the snapshot 2024-03-09T08:55:19.844 INFO:tasks.workunit.client.1.smithi134.stdout: create 161/33 file after the snapshot 2024-03-09T08:55:19.845 INFO:tasks.workunit.client.1.smithi134.stdout: create 161/34 file after the snapshot 2024-03-09T08:55:19.845 INFO:tasks.workunit.client.1.smithi134.stdout: create 161/35 file after the snapshot 2024-03-09T08:55:19.845 INFO:tasks.workunit.client.1.smithi134.stdout: create 161/36 file after the snapshot 2024-03-09T08:55:19.846 INFO:tasks.workunit.client.1.smithi134.stdout: create 161/37 file after the snapshot 2024-03-09T08:55:19.846 INFO:tasks.workunit.client.1.smithi134.stdout: create 161/38 file after the snapshot 2024-03-09T08:55:19.846 INFO:tasks.workunit.client.1.smithi134.stdout: create 161/39 file after the snapshot 2024-03-09T08:55:19.846 INFO:tasks.workunit.client.1.smithi134.stdout: create 162/.snap/snap-subdir-test 2024-03-09T08:55:19.887 INFO:tasks.workunit.client.0.smithi067.stdout: create 121/31 file after the snapshot 2024-03-09T08:55:19.887 INFO:tasks.workunit.client.0.smithi067.stdout: create 121/32 file after the snapshot 2024-03-09T08:55:19.887 INFO:tasks.workunit.client.0.smithi067.stdout: create 121/33 file after the snapshot 2024-03-09T08:55:19.887 INFO:tasks.workunit.client.0.smithi067.stdout: create 121/34 file after the snapshot 2024-03-09T08:55:19.888 INFO:tasks.workunit.client.0.smithi067.stdout: create 121/35 file after the snapshot 2024-03-09T08:55:19.888 INFO:tasks.workunit.client.0.smithi067.stdout: create 121/36 file after the snapshot 2024-03-09T08:55:19.888 INFO:tasks.workunit.client.0.smithi067.stdout: create 121/37 file after the snapshot 2024-03-09T08:55:19.888 INFO:tasks.workunit.client.0.smithi067.stdout: create 121/38 file after the snapshot 2024-03-09T08:55:19.888 INFO:tasks.workunit.client.0.smithi067.stdout: create 121/39 file after the snapshot 2024-03-09T08:55:19.888 INFO:tasks.workunit.client.0.smithi067.stdout: create 122/.snap/snap-subdir-test 2024-03-09T08:55:19.924 INFO:tasks.workunit.client.1.smithi134.stdout: create 162/30 file after the snapshot 2024-03-09T08:55:19.927 INFO:tasks.workunit.client.1.smithi134.stdout: create 162/31 file after the snapshot 2024-03-09T08:55:19.927 INFO:tasks.workunit.client.1.smithi134.stdout: create 162/32 file after the snapshot 2024-03-09T08:55:19.927 INFO:tasks.workunit.client.1.smithi134.stdout: create 162/33 file after the snapshot 2024-03-09T08:55:19.928 INFO:tasks.workunit.client.1.smithi134.stdout: create 162/34 file after the snapshot 2024-03-09T08:55:19.928 INFO:tasks.workunit.client.1.smithi134.stdout: create 162/35 file after the snapshot 2024-03-09T08:55:19.928 INFO:tasks.workunit.client.1.smithi134.stdout: create 162/36 file after the snapshot 2024-03-09T08:55:19.928 INFO:tasks.workunit.client.1.smithi134.stdout: create 162/37 file after the snapshot 2024-03-09T08:55:19.928 INFO:tasks.workunit.client.1.smithi134.stdout: create 162/38 file after the snapshot 2024-03-09T08:55:19.929 INFO:tasks.workunit.client.1.smithi134.stdout: create 162/39 file after the snapshot 2024-03-09T08:55:19.929 INFO:tasks.workunit.client.1.smithi134.stdout: create 163/.snap/snap-subdir-test 2024-03-09T08:55:20.005 INFO:tasks.workunit.client.1.smithi134.stdout: create 163/30 file after the snapshot 2024-03-09T08:55:20.010 INFO:tasks.workunit.client.1.smithi134.stdout: create 163/31 file after the snapshot 2024-03-09T08:55:20.010 INFO:tasks.workunit.client.1.smithi134.stdout: create 163/32 file after the snapshot 2024-03-09T08:55:20.010 INFO:tasks.workunit.client.1.smithi134.stdout: create 163/33 file after the snapshot 2024-03-09T08:55:20.010 INFO:tasks.workunit.client.1.smithi134.stdout: create 163/34 file after the snapshot 2024-03-09T08:55:20.010 INFO:tasks.workunit.client.1.smithi134.stdout: create 163/35 file after the snapshot 2024-03-09T08:55:20.010 INFO:tasks.workunit.client.1.smithi134.stdout: create 163/36 file after the snapshot 2024-03-09T08:55:20.011 INFO:tasks.workunit.client.1.smithi134.stdout: create 163/37 file after the snapshot 2024-03-09T08:55:20.011 INFO:tasks.workunit.client.1.smithi134.stdout: create 163/38 file after the snapshot 2024-03-09T08:55:20.011 INFO:tasks.workunit.client.1.smithi134.stdout: create 163/39 file after the snapshot 2024-03-09T08:55:20.012 INFO:tasks.workunit.client.1.smithi134.stdout: create 164/.snap/snap-subdir-test 2024-03-09T08:55:20.085 INFO:tasks.workunit.client.1.smithi134.stdout: create 164/30 file after the snapshot 2024-03-09T08:55:20.088 INFO:tasks.workunit.client.1.smithi134.stdout: create 164/31 file after the snapshot 2024-03-09T08:55:20.089 INFO:tasks.workunit.client.1.smithi134.stdout: create 164/32 file after the snapshot 2024-03-09T08:55:20.089 INFO:tasks.workunit.client.1.smithi134.stdout: create 164/33 file after the snapshot 2024-03-09T08:55:20.089 INFO:tasks.workunit.client.1.smithi134.stdout: create 164/34 file after the snapshot 2024-03-09T08:55:20.089 INFO:tasks.workunit.client.1.smithi134.stdout: create 164/35 file after the snapshot 2024-03-09T08:55:20.090 INFO:tasks.workunit.client.1.smithi134.stdout: create 164/36 file after the snapshot 2024-03-09T08:55:20.090 INFO:tasks.workunit.client.1.smithi134.stdout: create 164/37 file after the snapshot 2024-03-09T08:55:20.091 INFO:tasks.workunit.client.1.smithi134.stdout: create 164/38 file after the snapshot 2024-03-09T08:55:20.091 INFO:tasks.workunit.client.1.smithi134.stdout: create 164/39 file after the snapshot 2024-03-09T08:55:20.091 INFO:tasks.workunit.client.1.smithi134.stdout: create 165/.snap/snap-subdir-test 2024-03-09T08:55:20.175 INFO:tasks.workunit.client.1.smithi134.stdout: create 165/30 file after the snapshot 2024-03-09T08:55:20.178 INFO:tasks.workunit.client.1.smithi134.stdout: create 165/31 file after the snapshot 2024-03-09T08:55:20.179 INFO:tasks.workunit.client.1.smithi134.stdout: create 165/32 file after the snapshot 2024-03-09T08:55:20.179 INFO:tasks.workunit.client.1.smithi134.stdout: create 165/33 file after the snapshot 2024-03-09T08:55:20.179 INFO:tasks.workunit.client.1.smithi134.stdout: create 165/34 file after the snapshot 2024-03-09T08:55:20.180 INFO:tasks.workunit.client.1.smithi134.stdout: create 165/35 file after the snapshot 2024-03-09T08:55:20.180 INFO:tasks.workunit.client.1.smithi134.stdout: create 165/36 file after the snapshot 2024-03-09T08:55:20.180 INFO:tasks.workunit.client.1.smithi134.stdout: create 165/37 file after the snapshot 2024-03-09T08:55:20.180 INFO:tasks.workunit.client.1.smithi134.stdout: create 165/38 file after the snapshot 2024-03-09T08:55:20.181 INFO:tasks.workunit.client.1.smithi134.stdout: create 165/39 file after the snapshot 2024-03-09T08:55:20.181 INFO:tasks.workunit.client.1.smithi134.stdout: create 166/.snap/snap-subdir-test 2024-03-09T08:55:20.230 INFO:tasks.workunit.client.0.smithi067.stdout: create 122/30 file after the snapshot 2024-03-09T08:55:20.235 INFO:tasks.workunit.client.0.smithi067.stdout: create 122/31 file after the snapshot 2024-03-09T08:55:20.235 INFO:tasks.workunit.client.0.smithi067.stdout: create 122/32 file after the snapshot 2024-03-09T08:55:20.236 INFO:tasks.workunit.client.0.smithi067.stdout: create 122/33 file after the snapshot 2024-03-09T08:55:20.236 INFO:tasks.workunit.client.0.smithi067.stdout: create 122/34 file after the snapshot 2024-03-09T08:55:20.236 INFO:tasks.workunit.client.0.smithi067.stdout: create 122/35 file after the snapshot 2024-03-09T08:55:20.236 INFO:tasks.workunit.client.0.smithi067.stdout: create 122/36 file after the snapshot 2024-03-09T08:55:20.236 INFO:tasks.workunit.client.0.smithi067.stdout: create 122/37 file after the snapshot 2024-03-09T08:55:20.237 INFO:tasks.workunit.client.0.smithi067.stdout: create 122/38 file after the snapshot 2024-03-09T08:55:20.237 INFO:tasks.workunit.client.0.smithi067.stdout: create 122/39 file after the snapshot 2024-03-09T08:55:20.237 INFO:tasks.workunit.client.0.smithi067.stdout: create 123/.snap/snap-subdir-test 2024-03-09T08:55:20.270 INFO:tasks.workunit.client.1.smithi134.stdout: create 166/30 file after the snapshot 2024-03-09T08:55:20.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 166/31 file after the snapshot 2024-03-09T08:55:20.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 166/32 file after the snapshot 2024-03-09T08:55:20.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 166/33 file after the snapshot 2024-03-09T08:55:20.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 166/34 file after the snapshot 2024-03-09T08:55:20.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 166/35 file after the snapshot 2024-03-09T08:55:20.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 166/36 file after the snapshot 2024-03-09T08:55:20.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 166/37 file after the snapshot 2024-03-09T08:55:20.275 INFO:tasks.workunit.client.1.smithi134.stdout: create 166/38 file after the snapshot 2024-03-09T08:55:20.275 INFO:tasks.workunit.client.1.smithi134.stdout: create 166/39 file after the snapshot 2024-03-09T08:55:20.275 INFO:tasks.workunit.client.1.smithi134.stdout: create 167/.snap/snap-subdir-test 2024-03-09T08:55:20.306 INFO:tasks.workunit.client.0.smithi067.stdout: create 123/30 file after the snapshot 2024-03-09T08:55:20.309 INFO:tasks.workunit.client.0.smithi067.stdout: create 123/31 file after the snapshot 2024-03-09T08:55:20.309 INFO:tasks.workunit.client.0.smithi067.stdout: create 123/32 file after the snapshot 2024-03-09T08:55:20.309 INFO:tasks.workunit.client.0.smithi067.stdout: create 123/33 file after the snapshot 2024-03-09T08:55:20.310 INFO:tasks.workunit.client.0.smithi067.stdout: create 123/34 file after the snapshot 2024-03-09T08:55:20.310 INFO:tasks.workunit.client.0.smithi067.stdout: create 123/35 file after the snapshot 2024-03-09T08:55:20.311 INFO:tasks.workunit.client.0.smithi067.stdout: create 123/36 file after the snapshot 2024-03-09T08:55:20.311 INFO:tasks.workunit.client.0.smithi067.stdout: create 123/37 file after the snapshot 2024-03-09T08:55:20.312 INFO:tasks.workunit.client.0.smithi067.stdout: create 123/38 file after the snapshot 2024-03-09T08:55:20.312 INFO:tasks.workunit.client.0.smithi067.stdout: create 123/39 file after the snapshot 2024-03-09T08:55:20.314 INFO:tasks.workunit.client.0.smithi067.stdout: create 124/.snap/snap-subdir-test 2024-03-09T08:55:20.356 INFO:tasks.workunit.client.1.smithi134.stdout: create 167/30 file after the snapshot 2024-03-09T08:55:20.358 INFO:tasks.workunit.client.1.smithi134.stdout: create 167/31 file after the snapshot 2024-03-09T08:55:20.359 INFO:tasks.workunit.client.1.smithi134.stdout: create 167/32 file after the snapshot 2024-03-09T08:55:20.359 INFO:tasks.workunit.client.1.smithi134.stdout: create 167/33 file after the snapshot 2024-03-09T08:55:20.360 INFO:tasks.workunit.client.1.smithi134.stdout: create 167/34 file after the snapshot 2024-03-09T08:55:20.360 INFO:tasks.workunit.client.1.smithi134.stdout: create 167/35 file after the snapshot 2024-03-09T08:55:20.361 INFO:tasks.workunit.client.1.smithi134.stdout: create 167/36 file after the snapshot 2024-03-09T08:55:20.361 INFO:tasks.workunit.client.1.smithi134.stdout: create 167/37 file after the snapshot 2024-03-09T08:55:20.361 INFO:tasks.workunit.client.1.smithi134.stdout: create 167/38 file after the snapshot 2024-03-09T08:55:20.361 INFO:tasks.workunit.client.1.smithi134.stdout: create 167/39 file after the snapshot 2024-03-09T08:55:20.361 INFO:tasks.workunit.client.1.smithi134.stdout: create 168/.snap/snap-subdir-test 2024-03-09T08:55:20.395 INFO:tasks.workunit.client.0.smithi067.stdout: create 124/30 file after the snapshot 2024-03-09T08:55:20.399 INFO:tasks.workunit.client.0.smithi067.stdout: create 124/31 file after the snapshot 2024-03-09T08:55:20.399 INFO:tasks.workunit.client.0.smithi067.stdout: create 124/32 file after the snapshot 2024-03-09T08:55:20.399 INFO:tasks.workunit.client.0.smithi067.stdout: create 124/33 file after the snapshot 2024-03-09T08:55:20.400 INFO:tasks.workunit.client.0.smithi067.stdout: create 124/34 file after the snapshot 2024-03-09T08:55:20.400 INFO:tasks.workunit.client.0.smithi067.stdout: create 124/35 file after the snapshot 2024-03-09T08:55:20.401 INFO:tasks.workunit.client.0.smithi067.stdout: create 124/36 file after the snapshot 2024-03-09T08:55:20.401 INFO:tasks.workunit.client.0.smithi067.stdout: create 124/37 file after the snapshot 2024-03-09T08:55:20.401 INFO:tasks.workunit.client.0.smithi067.stdout: create 124/38 file after the snapshot 2024-03-09T08:55:20.401 INFO:tasks.workunit.client.0.smithi067.stdout: create 124/39 file after the snapshot 2024-03-09T08:55:20.402 INFO:tasks.workunit.client.0.smithi067.stdout: create 125/.snap/snap-subdir-test 2024-03-09T08:55:20.431 INFO:tasks.workunit.client.1.smithi134.stdout: create 168/30 file after the snapshot 2024-03-09T08:55:20.434 INFO:tasks.workunit.client.1.smithi134.stdout: create 168/31 file after the snapshot 2024-03-09T08:55:20.434 INFO:tasks.workunit.client.1.smithi134.stdout: create 168/32 file after the snapshot 2024-03-09T08:55:20.435 INFO:tasks.workunit.client.1.smithi134.stdout: create 168/33 file after the snapshot 2024-03-09T08:55:20.435 INFO:tasks.workunit.client.1.smithi134.stdout: create 168/34 file after the snapshot 2024-03-09T08:55:20.435 INFO:tasks.workunit.client.1.smithi134.stdout: create 168/35 file after the snapshot 2024-03-09T08:55:20.436 INFO:tasks.workunit.client.1.smithi134.stdout: create 168/36 file after the snapshot 2024-03-09T08:55:20.436 INFO:tasks.workunit.client.1.smithi134.stdout: create 168/37 file after the snapshot 2024-03-09T08:55:20.436 INFO:tasks.workunit.client.1.smithi134.stdout: create 168/38 file after the snapshot 2024-03-09T08:55:20.437 INFO:tasks.workunit.client.1.smithi134.stdout: create 168/39 file after the snapshot 2024-03-09T08:55:20.437 INFO:tasks.workunit.client.1.smithi134.stdout: create 169/.snap/snap-subdir-test 2024-03-09T08:55:20.466 INFO:tasks.workunit.client.0.smithi067.stdout: create 125/30 file after the snapshot 2024-03-09T08:55:20.470 INFO:tasks.workunit.client.0.smithi067.stdout: create 125/31 file after the snapshot 2024-03-09T08:55:20.470 INFO:tasks.workunit.client.0.smithi067.stdout: create 125/32 file after the snapshot 2024-03-09T08:55:20.471 INFO:tasks.workunit.client.0.smithi067.stdout: create 125/33 file after the snapshot 2024-03-09T08:55:20.471 INFO:tasks.workunit.client.0.smithi067.stdout: create 125/34 file after the snapshot 2024-03-09T08:55:20.471 INFO:tasks.workunit.client.0.smithi067.stdout: create 125/35 file after the snapshot 2024-03-09T08:55:20.471 INFO:tasks.workunit.client.0.smithi067.stdout: create 125/36 file after the snapshot 2024-03-09T08:55:20.471 INFO:tasks.workunit.client.0.smithi067.stdout: create 125/37 file after the snapshot 2024-03-09T08:55:20.472 INFO:tasks.workunit.client.0.smithi067.stdout: create 125/38 file after the snapshot 2024-03-09T08:55:20.472 INFO:tasks.workunit.client.0.smithi067.stdout: create 125/39 file after the snapshot 2024-03-09T08:55:20.472 INFO:tasks.workunit.client.0.smithi067.stdout: create 126/.snap/snap-subdir-test 2024-03-09T08:55:20.512 INFO:tasks.workunit.client.1.smithi134.stdout: create 169/30 file after the snapshot 2024-03-09T08:55:20.514 INFO:tasks.workunit.client.1.smithi134.stdout: create 169/31 file after the snapshot 2024-03-09T08:55:20.515 INFO:tasks.workunit.client.1.smithi134.stdout: create 169/32 file after the snapshot 2024-03-09T08:55:20.516 INFO:tasks.workunit.client.1.smithi134.stdout: create 169/33 file after the snapshot 2024-03-09T08:55:20.516 INFO:tasks.workunit.client.1.smithi134.stdout: create 169/34 file after the snapshot 2024-03-09T08:55:20.516 INFO:tasks.workunit.client.1.smithi134.stdout: create 169/35 file after the snapshot 2024-03-09T08:55:20.516 INFO:tasks.workunit.client.1.smithi134.stdout: create 169/36 file after the snapshot 2024-03-09T08:55:20.517 INFO:tasks.workunit.client.1.smithi134.stdout: create 169/37 file after the snapshot 2024-03-09T08:55:20.517 INFO:tasks.workunit.client.1.smithi134.stdout: create 169/38 file after the snapshot 2024-03-09T08:55:20.518 INFO:tasks.workunit.client.1.smithi134.stdout: create 169/39 file after the snapshot 2024-03-09T08:55:20.519 INFO:tasks.workunit.client.1.smithi134.stdout: create 170/.snap/snap-subdir-test 2024-03-09T08:55:20.553 INFO:tasks.workunit.client.0.smithi067.stdout: create 126/30 file after the snapshot 2024-03-09T08:55:20.555 INFO:tasks.workunit.client.0.smithi067.stdout: create 126/31 file after the snapshot 2024-03-09T08:55:20.556 INFO:tasks.workunit.client.0.smithi067.stdout: create 126/32 file after the snapshot 2024-03-09T08:55:20.556 INFO:tasks.workunit.client.0.smithi067.stdout: create 126/33 file after the snapshot 2024-03-09T08:55:20.556 INFO:tasks.workunit.client.0.smithi067.stdout: create 126/34 file after the snapshot 2024-03-09T08:55:20.557 INFO:tasks.workunit.client.0.smithi067.stdout: create 126/35 file after the snapshot 2024-03-09T08:55:20.557 INFO:tasks.workunit.client.0.smithi067.stdout: create 126/36 file after the snapshot 2024-03-09T08:55:20.557 INFO:tasks.workunit.client.0.smithi067.stdout: create 126/37 file after the snapshot 2024-03-09T08:55:20.558 INFO:tasks.workunit.client.0.smithi067.stdout: create 126/38 file after the snapshot 2024-03-09T08:55:20.558 INFO:tasks.workunit.client.0.smithi067.stdout: create 126/39 file after the snapshot 2024-03-09T08:55:20.558 INFO:tasks.workunit.client.0.smithi067.stdout: create 127/.snap/snap-subdir-test 2024-03-09T08:55:20.587 INFO:tasks.workunit.client.1.smithi134.stdout: create 170/30 file after the snapshot 2024-03-09T08:55:20.594 INFO:tasks.workunit.client.1.smithi134.stdout: create 170/31 file after the snapshot 2024-03-09T08:55:20.596 INFO:tasks.workunit.client.1.smithi134.stdout: create 170/32 file after the snapshot 2024-03-09T08:55:20.596 INFO:tasks.workunit.client.1.smithi134.stdout: create 170/33 file after the snapshot 2024-03-09T08:55:20.597 INFO:tasks.workunit.client.1.smithi134.stdout: create 170/34 file after the snapshot 2024-03-09T08:55:20.597 INFO:tasks.workunit.client.1.smithi134.stdout: create 170/35 file after the snapshot 2024-03-09T08:55:20.598 INFO:tasks.workunit.client.1.smithi134.stdout: create 170/36 file after the snapshot 2024-03-09T08:55:20.598 INFO:tasks.workunit.client.1.smithi134.stdout: create 170/37 file after the snapshot 2024-03-09T08:55:20.598 INFO:tasks.workunit.client.1.smithi134.stdout: create 170/38 file after the snapshot 2024-03-09T08:55:20.598 INFO:tasks.workunit.client.1.smithi134.stdout: create 170/39 file after the snapshot 2024-03-09T08:55:20.598 INFO:tasks.workunit.client.1.smithi134.stdout: create 171/.snap/snap-subdir-test 2024-03-09T08:55:20.630 INFO:tasks.workunit.client.0.smithi067.stdout: create 127/30 file after the snapshot 2024-03-09T08:55:20.633 INFO:tasks.workunit.client.0.smithi067.stdout: create 127/31 file after the snapshot 2024-03-09T08:55:20.634 INFO:tasks.workunit.client.0.smithi067.stdout: create 127/32 file after the snapshot 2024-03-09T08:55:20.634 INFO:tasks.workunit.client.0.smithi067.stdout: create 127/33 file after the snapshot 2024-03-09T08:55:20.634 INFO:tasks.workunit.client.0.smithi067.stdout: create 127/34 file after the snapshot 2024-03-09T08:55:20.634 INFO:tasks.workunit.client.0.smithi067.stdout: create 127/35 file after the snapshot 2024-03-09T08:55:20.634 INFO:tasks.workunit.client.0.smithi067.stdout: create 127/36 file after the snapshot 2024-03-09T08:55:20.635 INFO:tasks.workunit.client.0.smithi067.stdout: create 127/37 file after the snapshot 2024-03-09T08:55:20.635 INFO:tasks.workunit.client.0.smithi067.stdout: create 127/38 file after the snapshot 2024-03-09T08:55:20.635 INFO:tasks.workunit.client.0.smithi067.stdout: create 127/39 file after the snapshot 2024-03-09T08:55:20.635 INFO:tasks.workunit.client.0.smithi067.stdout: create 128/.snap/snap-subdir-test 2024-03-09T08:55:20.673 INFO:tasks.workunit.client.1.smithi134.stdout: create 171/30 file after the snapshot 2024-03-09T08:55:20.678 INFO:tasks.workunit.client.1.smithi134.stdout: create 171/31 file after the snapshot 2024-03-09T08:55:20.678 INFO:tasks.workunit.client.1.smithi134.stdout: create 171/32 file after the snapshot 2024-03-09T08:55:20.678 INFO:tasks.workunit.client.1.smithi134.stdout: create 171/33 file after the snapshot 2024-03-09T08:55:20.678 INFO:tasks.workunit.client.1.smithi134.stdout: create 171/34 file after the snapshot 2024-03-09T08:55:20.679 INFO:tasks.workunit.client.1.smithi134.stdout: create 171/35 file after the snapshot 2024-03-09T08:55:20.679 INFO:tasks.workunit.client.1.smithi134.stdout: create 171/36 file after the snapshot 2024-03-09T08:55:20.679 INFO:tasks.workunit.client.1.smithi134.stdout: create 171/37 file after the snapshot 2024-03-09T08:55:20.679 INFO:tasks.workunit.client.1.smithi134.stdout: create 171/38 file after the snapshot 2024-03-09T08:55:20.679 INFO:tasks.workunit.client.1.smithi134.stdout: create 171/39 file after the snapshot 2024-03-09T08:55:20.680 INFO:tasks.workunit.client.1.smithi134.stdout: create 172/.snap/snap-subdir-test 2024-03-09T08:55:20.688 INFO:tasks.workunit.client.0.smithi067.stdout: create 128/30 file after the snapshot 2024-03-09T08:55:20.691 INFO:tasks.workunit.client.0.smithi067.stdout: create 128/31 file after the snapshot 2024-03-09T08:55:20.691 INFO:tasks.workunit.client.0.smithi067.stdout: create 128/32 file after the snapshot 2024-03-09T08:55:20.691 INFO:tasks.workunit.client.0.smithi067.stdout: create 128/33 file after the snapshot 2024-03-09T08:55:20.692 INFO:tasks.workunit.client.0.smithi067.stdout: create 128/34 file after the snapshot 2024-03-09T08:55:20.692 INFO:tasks.workunit.client.0.smithi067.stdout: create 128/35 file after the snapshot 2024-03-09T08:55:20.692 INFO:tasks.workunit.client.0.smithi067.stdout: create 128/36 file after the snapshot 2024-03-09T08:55:20.692 INFO:tasks.workunit.client.0.smithi067.stdout: create 128/37 file after the snapshot 2024-03-09T08:55:20.692 INFO:tasks.workunit.client.0.smithi067.stdout: create 128/38 file after the snapshot 2024-03-09T08:55:20.692 INFO:tasks.workunit.client.0.smithi067.stdout: create 128/39 file after the snapshot 2024-03-09T08:55:20.693 INFO:tasks.workunit.client.0.smithi067.stdout: create 129/.snap/snap-subdir-test 2024-03-09T08:55:20.763 INFO:tasks.workunit.client.0.smithi067.stdout: create 129/30 file after the snapshot 2024-03-09T08:55:20.766 INFO:tasks.workunit.client.0.smithi067.stdout: create 129/31 file after the snapshot 2024-03-09T08:55:20.766 INFO:tasks.workunit.client.0.smithi067.stdout: create 129/32 file after the snapshot 2024-03-09T08:55:20.767 INFO:tasks.workunit.client.0.smithi067.stdout: create 129/33 file after the snapshot 2024-03-09T08:55:20.767 INFO:tasks.workunit.client.0.smithi067.stdout: create 129/34 file after the snapshot 2024-03-09T08:55:20.767 INFO:tasks.workunit.client.0.smithi067.stdout: create 129/35 file after the snapshot 2024-03-09T08:55:20.767 INFO:tasks.workunit.client.0.smithi067.stdout: create 129/36 file after the snapshot 2024-03-09T08:55:20.767 INFO:tasks.workunit.client.0.smithi067.stdout: create 129/37 file after the snapshot 2024-03-09T08:55:20.768 INFO:tasks.workunit.client.0.smithi067.stdout: create 129/38 file after the snapshot 2024-03-09T08:55:20.768 INFO:tasks.workunit.client.0.smithi067.stdout: create 129/39 file after the snapshot 2024-03-09T08:55:20.768 INFO:tasks.workunit.client.0.smithi067.stdout: create 130/.snap/snap-subdir-test 2024-03-09T08:55:20.839 INFO:tasks.workunit.client.0.smithi067.stdout: create 130/30 file after the snapshot 2024-03-09T08:55:20.842 INFO:tasks.workunit.client.0.smithi067.stdout: create 130/31 file after the snapshot 2024-03-09T08:55:20.843 INFO:tasks.workunit.client.0.smithi067.stdout: create 130/32 file after the snapshot 2024-03-09T08:55:20.843 INFO:tasks.workunit.client.0.smithi067.stdout: create 130/33 file after the snapshot 2024-03-09T08:55:20.844 INFO:tasks.workunit.client.0.smithi067.stdout: create 130/34 file after the snapshot 2024-03-09T08:55:20.844 INFO:tasks.workunit.client.0.smithi067.stdout: create 130/35 file after the snapshot 2024-03-09T08:55:20.845 INFO:tasks.workunit.client.0.smithi067.stdout: create 130/36 file after the snapshot 2024-03-09T08:55:20.845 INFO:tasks.workunit.client.0.smithi067.stdout: create 130/37 file after the snapshot 2024-03-09T08:55:20.845 INFO:tasks.workunit.client.0.smithi067.stdout: create 130/38 file after the snapshot 2024-03-09T08:55:20.845 INFO:tasks.workunit.client.0.smithi067.stdout: create 130/39 file after the snapshot 2024-03-09T08:55:20.845 INFO:tasks.workunit.client.0.smithi067.stdout: create 131/.snap/snap-subdir-test 2024-03-09T08:55:20.923 INFO:tasks.workunit.client.0.smithi067.stdout: create 131/30 file after the snapshot 2024-03-09T08:55:20.925 INFO:tasks.workunit.client.0.smithi067.stdout: create 131/31 file after the snapshot 2024-03-09T08:55:20.926 INFO:tasks.workunit.client.0.smithi067.stdout: create 131/32 file after the snapshot 2024-03-09T08:55:20.926 INFO:tasks.workunit.client.0.smithi067.stdout: create 131/33 file after the snapshot 2024-03-09T08:55:20.927 INFO:tasks.workunit.client.0.smithi067.stdout: create 131/34 file after the snapshot 2024-03-09T08:55:20.927 INFO:tasks.workunit.client.0.smithi067.stdout: create 131/35 file after the snapshot 2024-03-09T08:55:20.928 INFO:tasks.workunit.client.0.smithi067.stdout: create 131/36 file after the snapshot 2024-03-09T08:55:20.928 INFO:tasks.workunit.client.0.smithi067.stdout: create 131/37 file after the snapshot 2024-03-09T08:55:20.928 INFO:tasks.workunit.client.0.smithi067.stdout: create 131/38 file after the snapshot 2024-03-09T08:55:20.928 INFO:tasks.workunit.client.0.smithi067.stdout: create 131/39 file after the snapshot 2024-03-09T08:55:20.928 INFO:tasks.workunit.client.0.smithi067.stdout: create 132/.snap/snap-subdir-test 2024-03-09T08:55:21.008 INFO:tasks.workunit.client.0.smithi067.stdout: create 132/30 file after the snapshot 2024-03-09T08:55:21.010 INFO:tasks.workunit.client.0.smithi067.stdout: create 132/31 file after the snapshot 2024-03-09T08:55:21.011 INFO:tasks.workunit.client.0.smithi067.stdout: create 132/32 file after the snapshot 2024-03-09T08:55:21.011 INFO:tasks.workunit.client.0.smithi067.stdout: create 132/33 file after the snapshot 2024-03-09T08:55:21.011 INFO:tasks.workunit.client.0.smithi067.stdout: create 132/34 file after the snapshot 2024-03-09T08:55:21.012 INFO:tasks.workunit.client.0.smithi067.stdout: create 132/35 file after the snapshot 2024-03-09T08:55:21.012 INFO:tasks.workunit.client.0.smithi067.stdout: create 132/36 file after the snapshot 2024-03-09T08:55:21.012 INFO:tasks.workunit.client.0.smithi067.stdout: create 132/37 file after the snapshot 2024-03-09T08:55:21.013 INFO:tasks.workunit.client.0.smithi067.stdout: create 132/38 file after the snapshot 2024-03-09T08:55:21.013 INFO:tasks.workunit.client.0.smithi067.stdout: create 132/39 file after the snapshot 2024-03-09T08:55:21.013 INFO:tasks.workunit.client.0.smithi067.stdout: create 133/.snap/snap-subdir-test 2024-03-09T08:55:21.014 INFO:tasks.workunit.client.1.smithi134.stdout: create 172/30 file after the snapshot 2024-03-09T08:55:21.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 172/31 file after the snapshot 2024-03-09T08:55:21.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 172/32 file after the snapshot 2024-03-09T08:55:21.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 172/33 file after the snapshot 2024-03-09T08:55:21.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 172/34 file after the snapshot 2024-03-09T08:55:21.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 172/35 file after the snapshot 2024-03-09T08:55:21.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 172/36 file after the snapshot 2024-03-09T08:55:21.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 172/37 file after the snapshot 2024-03-09T08:55:21.019 INFO:tasks.workunit.client.1.smithi134.stdout: create 172/38 file after the snapshot 2024-03-09T08:55:21.019 INFO:tasks.workunit.client.1.smithi134.stdout: create 172/39 file after the snapshot 2024-03-09T08:55:21.019 INFO:tasks.workunit.client.1.smithi134.stdout: create 173/.snap/snap-subdir-test 2024-03-09T08:55:21.085 INFO:tasks.workunit.client.1.smithi134.stdout: create 173/30 file after the snapshot 2024-03-09T08:55:21.089 INFO:tasks.workunit.client.1.smithi134.stdout: create 173/31 file after the snapshot 2024-03-09T08:55:21.089 INFO:tasks.workunit.client.1.smithi134.stdout: create 173/32 file after the snapshot 2024-03-09T08:55:21.089 INFO:tasks.workunit.client.1.smithi134.stdout: create 173/33 file after the snapshot 2024-03-09T08:55:21.090 INFO:tasks.workunit.client.1.smithi134.stdout: create 173/34 file after the snapshot 2024-03-09T08:55:21.090 INFO:tasks.workunit.client.1.smithi134.stdout: create 173/35 file after the snapshot 2024-03-09T08:55:21.090 INFO:tasks.workunit.client.1.smithi134.stdout: create 173/36 file after the snapshot 2024-03-09T08:55:21.090 INFO:tasks.workunit.client.1.smithi134.stdout: create 173/37 file after the snapshot 2024-03-09T08:55:21.090 INFO:tasks.workunit.client.1.smithi134.stdout: create 173/38 file after the snapshot 2024-03-09T08:55:21.090 INFO:tasks.workunit.client.1.smithi134.stdout: create 173/39 file after the snapshot 2024-03-09T08:55:21.091 INFO:tasks.workunit.client.0.smithi067.stdout: create 133/30 file after the snapshot 2024-03-09T08:55:21.091 INFO:tasks.workunit.client.1.smithi134.stdout: create 174/.snap/snap-subdir-test 2024-03-09T08:55:21.092 INFO:tasks.workunit.client.0.smithi067.stdout: create 133/31 file after the snapshot 2024-03-09T08:55:21.092 INFO:tasks.workunit.client.0.smithi067.stdout: create 133/32 file after the snapshot 2024-03-09T08:55:21.092 INFO:tasks.workunit.client.0.smithi067.stdout: create 133/33 file after the snapshot 2024-03-09T08:55:21.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 133/34 file after the snapshot 2024-03-09T08:55:21.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 133/35 file after the snapshot 2024-03-09T08:55:21.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 133/36 file after the snapshot 2024-03-09T08:55:21.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 133/37 file after the snapshot 2024-03-09T08:55:21.094 INFO:tasks.workunit.client.0.smithi067.stdout: create 133/38 file after the snapshot 2024-03-09T08:55:21.094 INFO:tasks.workunit.client.0.smithi067.stdout: create 133/39 file after the snapshot 2024-03-09T08:55:21.094 INFO:tasks.workunit.client.0.smithi067.stdout: create 134/.snap/snap-subdir-test 2024-03-09T08:55:21.153 INFO:tasks.workunit.client.0.smithi067.stdout: create 134/30 file after the snapshot 2024-03-09T08:55:21.155 INFO:tasks.workunit.client.0.smithi067.stdout: create 134/31 file after the snapshot 2024-03-09T08:55:21.156 INFO:tasks.workunit.client.0.smithi067.stdout: create 134/32 file after the snapshot 2024-03-09T08:55:21.156 INFO:tasks.workunit.client.0.smithi067.stdout: create 134/33 file after the snapshot 2024-03-09T08:55:21.157 INFO:tasks.workunit.client.0.smithi067.stdout: create 134/34 file after the snapshot 2024-03-09T08:55:21.157 INFO:tasks.workunit.client.0.smithi067.stdout: create 134/35 file after the snapshot 2024-03-09T08:55:21.157 INFO:tasks.workunit.client.0.smithi067.stdout: create 134/36 file after the snapshot 2024-03-09T08:55:21.158 INFO:tasks.workunit.client.0.smithi067.stdout: create 134/37 file after the snapshot 2024-03-09T08:55:21.158 INFO:tasks.workunit.client.0.smithi067.stdout: create 134/38 file after the snapshot 2024-03-09T08:55:21.158 INFO:tasks.workunit.client.0.smithi067.stdout: create 134/39 file after the snapshot 2024-03-09T08:55:21.158 INFO:tasks.workunit.client.0.smithi067.stdout: create 135/.snap/snap-subdir-test 2024-03-09T08:55:21.159 INFO:tasks.workunit.client.1.smithi134.stdout: create 174/30 file after the snapshot 2024-03-09T08:55:21.163 INFO:tasks.workunit.client.1.smithi134.stdout: create 174/31 file after the snapshot 2024-03-09T08:55:21.163 INFO:tasks.workunit.client.1.smithi134.stdout: create 174/32 file after the snapshot 2024-03-09T08:55:21.163 INFO:tasks.workunit.client.1.smithi134.stdout: create 174/33 file after the snapshot 2024-03-09T08:55:21.164 INFO:tasks.workunit.client.1.smithi134.stdout: create 174/34 file after the snapshot 2024-03-09T08:55:21.164 INFO:tasks.workunit.client.1.smithi134.stdout: create 174/35 file after the snapshot 2024-03-09T08:55:21.164 INFO:tasks.workunit.client.1.smithi134.stdout: create 174/36 file after the snapshot 2024-03-09T08:55:21.164 INFO:tasks.workunit.client.1.smithi134.stdout: create 174/37 file after the snapshot 2024-03-09T08:55:21.164 INFO:tasks.workunit.client.1.smithi134.stdout: create 174/38 file after the snapshot 2024-03-09T08:55:21.164 INFO:tasks.workunit.client.1.smithi134.stdout: create 174/39 file after the snapshot 2024-03-09T08:55:21.164 INFO:tasks.workunit.client.1.smithi134.stdout: create 175/.snap/snap-subdir-test 2024-03-09T08:55:21.229 INFO:tasks.workunit.client.0.smithi067.stdout: create 135/30 file after the snapshot 2024-03-09T08:55:21.234 INFO:tasks.workunit.client.0.smithi067.stdout: create 135/31 file after the snapshot 2024-03-09T08:55:21.234 INFO:tasks.workunit.client.0.smithi067.stdout: create 135/32 file after the snapshot 2024-03-09T08:55:21.234 INFO:tasks.workunit.client.0.smithi067.stdout: create 135/33 file after the snapshot 2024-03-09T08:55:21.234 INFO:tasks.workunit.client.0.smithi067.stdout: create 135/34 file after the snapshot 2024-03-09T08:55:21.234 INFO:tasks.workunit.client.0.smithi067.stdout: create 135/35 file after the snapshot 2024-03-09T08:55:21.235 INFO:tasks.workunit.client.0.smithi067.stdout: create 135/36 file after the snapshot 2024-03-09T08:55:21.235 INFO:tasks.workunit.client.0.smithi067.stdout: create 135/37 file after the snapshot 2024-03-09T08:55:21.235 INFO:tasks.workunit.client.0.smithi067.stdout: create 135/38 file after the snapshot 2024-03-09T08:55:21.235 INFO:tasks.workunit.client.0.smithi067.stdout: create 135/39 file after the snapshot 2024-03-09T08:55:21.235 INFO:tasks.workunit.client.0.smithi067.stdout: create 136/.snap/snap-subdir-test 2024-03-09T08:55:21.253 INFO:tasks.workunit.client.1.smithi134.stdout: create 175/30 file after the snapshot 2024-03-09T08:55:21.254 INFO:tasks.workunit.client.1.smithi134.stdout: create 175/31 file after the snapshot 2024-03-09T08:55:21.255 INFO:tasks.workunit.client.1.smithi134.stdout: create 175/32 file after the snapshot 2024-03-09T08:55:21.255 INFO:tasks.workunit.client.1.smithi134.stdout: create 175/33 file after the snapshot 2024-03-09T08:55:21.255 INFO:tasks.workunit.client.1.smithi134.stdout: create 175/34 file after the snapshot 2024-03-09T08:55:21.256 INFO:tasks.workunit.client.1.smithi134.stdout: create 175/35 file after the snapshot 2024-03-09T08:55:21.256 INFO:tasks.workunit.client.1.smithi134.stdout: create 175/36 file after the snapshot 2024-03-09T08:55:21.256 INFO:tasks.workunit.client.1.smithi134.stdout: create 175/37 file after the snapshot 2024-03-09T08:55:21.256 INFO:tasks.workunit.client.1.smithi134.stdout: create 175/38 file after the snapshot 2024-03-09T08:55:21.256 INFO:tasks.workunit.client.1.smithi134.stdout: create 175/39 file after the snapshot 2024-03-09T08:55:21.257 INFO:tasks.workunit.client.1.smithi134.stdout: create 176/.snap/snap-subdir-test 2024-03-09T08:55:21.297 INFO:tasks.workunit.client.0.smithi067.stdout: create 136/30 file after the snapshot 2024-03-09T08:55:21.298 INFO:tasks.workunit.client.0.smithi067.stdout: create 136/31 file after the snapshot 2024-03-09T08:55:21.299 INFO:tasks.workunit.client.0.smithi067.stdout: create 136/32 file after the snapshot 2024-03-09T08:55:21.299 INFO:tasks.workunit.client.0.smithi067.stdout: create 136/33 file after the snapshot 2024-03-09T08:55:21.299 INFO:tasks.workunit.client.0.smithi067.stdout: create 136/34 file after the snapshot 2024-03-09T08:55:21.299 INFO:tasks.workunit.client.0.smithi067.stdout: create 136/35 file after the snapshot 2024-03-09T08:55:21.300 INFO:tasks.workunit.client.0.smithi067.stdout: create 136/36 file after the snapshot 2024-03-09T08:55:21.300 INFO:tasks.workunit.client.0.smithi067.stdout: create 136/37 file after the snapshot 2024-03-09T08:55:21.300 INFO:tasks.workunit.client.0.smithi067.stdout: create 136/38 file after the snapshot 2024-03-09T08:55:21.301 INFO:tasks.workunit.client.0.smithi067.stdout: create 136/39 file after the snapshot 2024-03-09T08:55:21.301 INFO:tasks.workunit.client.0.smithi067.stdout: create 137/.snap/snap-subdir-test 2024-03-09T08:55:21.333 INFO:tasks.workunit.client.1.smithi134.stdout: create 176/30 file after the snapshot 2024-03-09T08:55:21.335 INFO:tasks.workunit.client.1.smithi134.stdout: create 176/31 file after the snapshot 2024-03-09T08:55:21.335 INFO:tasks.workunit.client.1.smithi134.stdout: create 176/32 file after the snapshot 2024-03-09T08:55:21.335 INFO:tasks.workunit.client.1.smithi134.stdout: create 176/33 file after the snapshot 2024-03-09T08:55:21.336 INFO:tasks.workunit.client.1.smithi134.stdout: create 176/34 file after the snapshot 2024-03-09T08:55:21.336 INFO:tasks.workunit.client.1.smithi134.stdout: create 176/35 file after the snapshot 2024-03-09T08:55:21.336 INFO:tasks.workunit.client.1.smithi134.stdout: create 176/36 file after the snapshot 2024-03-09T08:55:21.336 INFO:tasks.workunit.client.1.smithi134.stdout: create 176/37 file after the snapshot 2024-03-09T08:55:21.337 INFO:tasks.workunit.client.1.smithi134.stdout: create 176/38 file after the snapshot 2024-03-09T08:55:21.337 INFO:tasks.workunit.client.1.smithi134.stdout: create 176/39 file after the snapshot 2024-03-09T08:55:21.337 INFO:tasks.workunit.client.1.smithi134.stdout: create 177/.snap/snap-subdir-test 2024-03-09T08:55:21.382 INFO:tasks.workunit.client.0.smithi067.stdout: create 137/30 file after the snapshot 2024-03-09T08:55:21.387 INFO:tasks.workunit.client.0.smithi067.stdout: create 137/31 file after the snapshot 2024-03-09T08:55:21.388 INFO:tasks.workunit.client.0.smithi067.stdout: create 137/32 file after the snapshot 2024-03-09T08:55:21.388 INFO:tasks.workunit.client.0.smithi067.stdout: create 137/33 file after the snapshot 2024-03-09T08:55:21.388 INFO:tasks.workunit.client.0.smithi067.stdout: create 137/34 file after the snapshot 2024-03-09T08:55:21.388 INFO:tasks.workunit.client.0.smithi067.stdout: create 137/35 file after the snapshot 2024-03-09T08:55:21.388 INFO:tasks.workunit.client.0.smithi067.stdout: create 137/36 file after the snapshot 2024-03-09T08:55:21.389 INFO:tasks.workunit.client.0.smithi067.stdout: create 137/37 file after the snapshot 2024-03-09T08:55:21.389 INFO:tasks.workunit.client.0.smithi067.stdout: create 137/38 file after the snapshot 2024-03-09T08:55:21.389 INFO:tasks.workunit.client.0.smithi067.stdout: create 137/39 file after the snapshot 2024-03-09T08:55:21.389 INFO:tasks.workunit.client.0.smithi067.stdout: create 138/.snap/snap-subdir-test 2024-03-09T08:55:21.420 INFO:tasks.workunit.client.1.smithi134.stdout: create 177/30 file after the snapshot 2024-03-09T08:55:21.423 INFO:tasks.workunit.client.1.smithi134.stdout: create 177/31 file after the snapshot 2024-03-09T08:55:21.424 INFO:tasks.workunit.client.1.smithi134.stdout: create 177/32 file after the snapshot 2024-03-09T08:55:21.424 INFO:tasks.workunit.client.1.smithi134.stdout: create 177/33 file after the snapshot 2024-03-09T08:55:21.424 INFO:tasks.workunit.client.1.smithi134.stdout: create 177/34 file after the snapshot 2024-03-09T08:55:21.424 INFO:tasks.workunit.client.1.smithi134.stdout: create 177/35 file after the snapshot 2024-03-09T08:55:21.424 INFO:tasks.workunit.client.1.smithi134.stdout: create 177/36 file after the snapshot 2024-03-09T08:55:21.425 INFO:tasks.workunit.client.1.smithi134.stdout: create 177/37 file after the snapshot 2024-03-09T08:55:21.425 INFO:tasks.workunit.client.1.smithi134.stdout: create 177/38 file after the snapshot 2024-03-09T08:55:21.426 INFO:tasks.workunit.client.1.smithi134.stdout: create 177/39 file after the snapshot 2024-03-09T08:55:21.427 INFO:tasks.workunit.client.1.smithi134.stdout: create 178/.snap/snap-subdir-test 2024-03-09T08:55:21.486 INFO:tasks.workunit.client.0.smithi067.stdout: create 138/30 file after the snapshot 2024-03-09T08:55:21.490 INFO:tasks.workunit.client.0.smithi067.stdout: create 138/31 file after the snapshot 2024-03-09T08:55:21.490 INFO:tasks.workunit.client.0.smithi067.stdout: create 138/32 file after the snapshot 2024-03-09T08:55:21.490 INFO:tasks.workunit.client.0.smithi067.stdout: create 138/33 file after the snapshot 2024-03-09T08:55:21.491 INFO:tasks.workunit.client.0.smithi067.stdout: create 138/34 file after the snapshot 2024-03-09T08:55:21.491 INFO:tasks.workunit.client.0.smithi067.stdout: create 138/35 file after the snapshot 2024-03-09T08:55:21.492 INFO:tasks.workunit.client.0.smithi067.stdout: create 138/36 file after the snapshot 2024-03-09T08:55:21.492 INFO:tasks.workunit.client.0.smithi067.stdout: create 138/37 file after the snapshot 2024-03-09T08:55:21.492 INFO:tasks.workunit.client.0.smithi067.stdout: create 138/38 file after the snapshot 2024-03-09T08:55:21.492 INFO:tasks.workunit.client.0.smithi067.stdout: create 138/39 file after the snapshot 2024-03-09T08:55:21.492 INFO:tasks.workunit.client.0.smithi067.stdout: create 139/.snap/snap-subdir-test 2024-03-09T08:55:21.508 INFO:tasks.workunit.client.1.smithi134.stdout: create 178/30 file after the snapshot 2024-03-09T08:55:21.512 INFO:tasks.workunit.client.1.smithi134.stdout: create 178/31 file after the snapshot 2024-03-09T08:55:21.512 INFO:tasks.workunit.client.1.smithi134.stdout: create 178/32 file after the snapshot 2024-03-09T08:55:21.513 INFO:tasks.workunit.client.1.smithi134.stdout: create 178/33 file after the snapshot 2024-03-09T08:55:21.513 INFO:tasks.workunit.client.1.smithi134.stdout: create 178/34 file after the snapshot 2024-03-09T08:55:21.513 INFO:tasks.workunit.client.1.smithi134.stdout: create 178/35 file after the snapshot 2024-03-09T08:55:21.514 INFO:tasks.workunit.client.1.smithi134.stdout: create 178/36 file after the snapshot 2024-03-09T08:55:21.514 INFO:tasks.workunit.client.1.smithi134.stdout: create 178/37 file after the snapshot 2024-03-09T08:55:21.514 INFO:tasks.workunit.client.1.smithi134.stdout: create 178/38 file after the snapshot 2024-03-09T08:55:21.514 INFO:tasks.workunit.client.1.smithi134.stdout: create 178/39 file after the snapshot 2024-03-09T08:55:21.514 INFO:tasks.workunit.client.1.smithi134.stdout: create 179/.snap/snap-subdir-test 2024-03-09T08:55:21.855 INFO:tasks.workunit.client.0.smithi067.stdout: create 139/30 file after the snapshot 2024-03-09T08:55:21.859 INFO:tasks.workunit.client.0.smithi067.stdout: create 139/31 file after the snapshot 2024-03-09T08:55:21.860 INFO:tasks.workunit.client.1.smithi134.stdout: create 179/30 file after the snapshot 2024-03-09T08:55:21.860 INFO:tasks.workunit.client.0.smithi067.stdout: create 139/32 file after the snapshot 2024-03-09T08:55:21.860 INFO:tasks.workunit.client.0.smithi067.stdout: create 139/33 file after the snapshot 2024-03-09T08:55:21.860 INFO:tasks.workunit.client.0.smithi067.stdout: create 139/34 file after the snapshot 2024-03-09T08:55:21.861 INFO:tasks.workunit.client.0.smithi067.stdout: create 139/35 file after the snapshot 2024-03-09T08:55:21.861 INFO:tasks.workunit.client.0.smithi067.stdout: create 139/36 file after the snapshot 2024-03-09T08:55:21.861 INFO:tasks.workunit.client.0.smithi067.stdout: create 139/37 file after the snapshot 2024-03-09T08:55:21.861 INFO:tasks.workunit.client.0.smithi067.stdout: create 139/38 file after the snapshot 2024-03-09T08:55:21.862 INFO:tasks.workunit.client.0.smithi067.stdout: create 139/39 file after the snapshot 2024-03-09T08:55:21.863 INFO:tasks.workunit.client.0.smithi067.stdout: create 140/.snap/snap-subdir-test 2024-03-09T08:55:21.863 INFO:tasks.workunit.client.1.smithi134.stdout: create 179/31 file after the snapshot 2024-03-09T08:55:21.863 INFO:tasks.workunit.client.1.smithi134.stdout: create 179/32 file after the snapshot 2024-03-09T08:55:21.863 INFO:tasks.workunit.client.1.smithi134.stdout: create 179/33 file after the snapshot 2024-03-09T08:55:21.863 INFO:tasks.workunit.client.1.smithi134.stdout: create 179/34 file after the snapshot 2024-03-09T08:55:21.863 INFO:tasks.workunit.client.1.smithi134.stdout: create 179/35 file after the snapshot 2024-03-09T08:55:21.863 INFO:tasks.workunit.client.1.smithi134.stdout: create 179/36 file after the snapshot 2024-03-09T08:55:21.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 179/37 file after the snapshot 2024-03-09T08:55:21.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 179/38 file after the snapshot 2024-03-09T08:55:21.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 179/39 file after the snapshot 2024-03-09T08:55:21.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 180/.snap/snap-subdir-test 2024-03-09T08:55:21.939 INFO:tasks.workunit.client.1.smithi134.stdout: create 180/30 file after the snapshot 2024-03-09T08:55:22.008 INFO:tasks.workunit.client.1.smithi134.stdout: create 180/31 file after the snapshot 2024-03-09T08:55:22.008 INFO:tasks.workunit.client.1.smithi134.stdout: create 180/32 file after the snapshot 2024-03-09T08:55:22.008 INFO:tasks.workunit.client.1.smithi134.stdout: create 180/33 file after the snapshot 2024-03-09T08:55:22.008 INFO:tasks.workunit.client.1.smithi134.stdout: create 180/34 file after the snapshot 2024-03-09T08:55:22.008 INFO:tasks.workunit.client.1.smithi134.stdout: create 180/35 file after the snapshot 2024-03-09T08:55:22.008 INFO:tasks.workunit.client.1.smithi134.stdout: create 180/36 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.1.smithi134.stdout: create 180/37 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.1.smithi134.stdout: create 180/38 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.1.smithi134.stdout: create 180/39 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.1.smithi134.stdout: create 181/.snap/snap-subdir-test 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.0.smithi067.stdout: create 140/30 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.0.smithi067.stdout: create 140/31 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.0.smithi067.stdout: create 140/32 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.0.smithi067.stdout: create 140/33 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.0.smithi067.stdout: create 140/34 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.0.smithi067.stdout: create 140/35 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.0.smithi067.stdout: create 140/36 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.0.smithi067.stdout: create 140/37 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.0.smithi067.stdout: create 140/38 file after the snapshot 2024-03-09T08:55:22.009 INFO:tasks.workunit.client.0.smithi067.stdout: create 140/39 file after the snapshot 2024-03-09T08:55:22.010 INFO:tasks.workunit.client.0.smithi067.stdout: create 141/.snap/snap-subdir-test 2024-03-09T08:55:22.014 INFO:tasks.workunit.client.1.smithi134.stdout: create 181/30 file after the snapshot 2024-03-09T08:55:22.016 INFO:tasks.workunit.client.1.smithi134.stdout: create 181/31 file after the snapshot 2024-03-09T08:55:22.016 INFO:tasks.workunit.client.1.smithi134.stdout: create 181/32 file after the snapshot 2024-03-09T08:55:22.016 INFO:tasks.workunit.client.1.smithi134.stdout: create 181/33 file after the snapshot 2024-03-09T08:55:22.016 INFO:tasks.workunit.client.1.smithi134.stdout: create 181/34 file after the snapshot 2024-03-09T08:55:22.016 INFO:tasks.workunit.client.1.smithi134.stdout: create 181/35 file after the snapshot 2024-03-09T08:55:22.016 INFO:tasks.workunit.client.1.smithi134.stdout: create 181/36 file after the snapshot 2024-03-09T08:55:22.016 INFO:tasks.workunit.client.1.smithi134.stdout: create 181/37 file after the snapshot 2024-03-09T08:55:22.016 INFO:tasks.workunit.client.1.smithi134.stdout: create 181/38 file after the snapshot 2024-03-09T08:55:22.016 INFO:tasks.workunit.client.1.smithi134.stdout: create 181/39 file after the snapshot 2024-03-09T08:55:22.016 INFO:tasks.workunit.client.1.smithi134.stdout: create 182/.snap/snap-subdir-test 2024-03-09T08:55:22.049 INFO:tasks.workunit.client.0.smithi067.stdout: create 141/30 file after the snapshot 2024-03-09T08:55:22.053 INFO:tasks.workunit.client.0.smithi067.stdout: create 141/31 file after the snapshot 2024-03-09T08:55:22.053 INFO:tasks.workunit.client.0.smithi067.stdout: create 141/32 file after the snapshot 2024-03-09T08:55:22.053 INFO:tasks.workunit.client.0.smithi067.stdout: create 141/33 file after the snapshot 2024-03-09T08:55:22.054 INFO:tasks.workunit.client.0.smithi067.stdout: create 141/34 file after the snapshot 2024-03-09T08:55:22.054 INFO:tasks.workunit.client.0.smithi067.stdout: create 141/35 file after the snapshot 2024-03-09T08:55:22.054 INFO:tasks.workunit.client.0.smithi067.stdout: create 141/36 file after the snapshot 2024-03-09T08:55:22.054 INFO:tasks.workunit.client.0.smithi067.stdout: create 141/37 file after the snapshot 2024-03-09T08:55:22.054 INFO:tasks.workunit.client.0.smithi067.stdout: create 141/38 file after the snapshot 2024-03-09T08:55:22.054 INFO:tasks.workunit.client.0.smithi067.stdout: create 141/39 file after the snapshot 2024-03-09T08:55:22.057 INFO:tasks.workunit.client.0.smithi067.stdout: create 142/.snap/snap-subdir-test 2024-03-09T08:55:22.096 INFO:tasks.workunit.client.1.smithi134.stdout: create 182/30 file after the snapshot 2024-03-09T08:55:22.104 INFO:tasks.workunit.client.1.smithi134.stdout: create 182/31 file after the snapshot 2024-03-09T08:55:22.105 INFO:tasks.workunit.client.1.smithi134.stdout: create 182/32 file after the snapshot 2024-03-09T08:55:22.105 INFO:tasks.workunit.client.1.smithi134.stdout: create 182/33 file after the snapshot 2024-03-09T08:55:22.105 INFO:tasks.workunit.client.1.smithi134.stdout: create 182/34 file after the snapshot 2024-03-09T08:55:22.105 INFO:tasks.workunit.client.1.smithi134.stdout: create 182/35 file after the snapshot 2024-03-09T08:55:22.105 INFO:tasks.workunit.client.1.smithi134.stdout: create 182/36 file after the snapshot 2024-03-09T08:55:22.105 INFO:tasks.workunit.client.1.smithi134.stdout: create 182/37 file after the snapshot 2024-03-09T08:55:22.105 INFO:tasks.workunit.client.1.smithi134.stdout: create 182/38 file after the snapshot 2024-03-09T08:55:22.105 INFO:tasks.workunit.client.1.smithi134.stdout: create 182/39 file after the snapshot 2024-03-09T08:55:22.105 INFO:tasks.workunit.client.1.smithi134.stdout: create 183/.snap/snap-subdir-test 2024-03-09T08:55:22.147 INFO:tasks.workunit.client.0.smithi067.stdout: create 142/30 file after the snapshot 2024-03-09T08:55:22.151 INFO:tasks.workunit.client.0.smithi067.stdout: create 142/31 file after the snapshot 2024-03-09T08:55:22.152 INFO:tasks.workunit.client.0.smithi067.stdout: create 142/32 file after the snapshot 2024-03-09T08:55:22.152 INFO:tasks.workunit.client.0.smithi067.stdout: create 142/33 file after the snapshot 2024-03-09T08:55:22.153 INFO:tasks.workunit.client.0.smithi067.stdout: create 142/34 file after the snapshot 2024-03-09T08:55:22.154 INFO:tasks.workunit.client.0.smithi067.stdout: create 142/35 file after the snapshot 2024-03-09T08:55:22.154 INFO:tasks.workunit.client.0.smithi067.stdout: create 142/36 file after the snapshot 2024-03-09T08:55:22.154 INFO:tasks.workunit.client.0.smithi067.stdout: create 142/37 file after the snapshot 2024-03-09T08:55:22.154 INFO:tasks.workunit.client.0.smithi067.stdout: create 142/38 file after the snapshot 2024-03-09T08:55:22.154 INFO:tasks.workunit.client.0.smithi067.stdout: create 142/39 file after the snapshot 2024-03-09T08:55:22.154 INFO:tasks.workunit.client.0.smithi067.stdout: create 143/.snap/snap-subdir-test 2024-03-09T08:55:22.168 INFO:tasks.workunit.client.1.smithi134.stdout: create 183/30 file after the snapshot 2024-03-09T08:55:22.173 INFO:tasks.workunit.client.1.smithi134.stdout: create 183/31 file after the snapshot 2024-03-09T08:55:22.173 INFO:tasks.workunit.client.1.smithi134.stdout: create 183/32 file after the snapshot 2024-03-09T08:55:22.173 INFO:tasks.workunit.client.1.smithi134.stdout: create 183/33 file after the snapshot 2024-03-09T08:55:22.173 INFO:tasks.workunit.client.1.smithi134.stdout: create 183/34 file after the snapshot 2024-03-09T08:55:22.173 INFO:tasks.workunit.client.1.smithi134.stdout: create 183/35 file after the snapshot 2024-03-09T08:55:22.174 INFO:tasks.workunit.client.1.smithi134.stdout: create 183/36 file after the snapshot 2024-03-09T08:55:22.174 INFO:tasks.workunit.client.1.smithi134.stdout: create 183/37 file after the snapshot 2024-03-09T08:55:22.174 INFO:tasks.workunit.client.1.smithi134.stdout: create 183/38 file after the snapshot 2024-03-09T08:55:22.174 INFO:tasks.workunit.client.1.smithi134.stdout: create 183/39 file after the snapshot 2024-03-09T08:55:22.174 INFO:tasks.workunit.client.1.smithi134.stdout: create 184/.snap/snap-subdir-test 2024-03-09T08:55:22.249 INFO:tasks.workunit.client.1.smithi134.stdout: create 184/30 file after the snapshot 2024-03-09T08:55:22.249 INFO:tasks.workunit.client.1.smithi134.stdout: create 184/31 file after the snapshot 2024-03-09T08:55:22.250 INFO:tasks.workunit.client.1.smithi134.stdout: create 184/32 file after the snapshot 2024-03-09T08:55:22.250 INFO:tasks.workunit.client.1.smithi134.stdout: create 184/33 file after the snapshot 2024-03-09T08:55:22.250 INFO:tasks.workunit.client.1.smithi134.stdout: create 184/34 file after the snapshot 2024-03-09T08:55:22.250 INFO:tasks.workunit.client.1.smithi134.stdout: create 184/35 file after the snapshot 2024-03-09T08:55:22.252 INFO:tasks.workunit.client.1.smithi134.stdout: create 184/36 file after the snapshot 2024-03-09T08:55:22.252 INFO:tasks.workunit.client.1.smithi134.stdout: create 184/37 file after the snapshot 2024-03-09T08:55:22.252 INFO:tasks.workunit.client.1.smithi134.stdout: create 184/38 file after the snapshot 2024-03-09T08:55:22.253 INFO:tasks.workunit.client.1.smithi134.stdout: create 184/39 file after the snapshot 2024-03-09T08:55:22.253 INFO:tasks.workunit.client.1.smithi134.stdout: create 185/.snap/snap-subdir-test 2024-03-09T08:55:22.254 INFO:tasks.workunit.client.0.smithi067.stdout: create 143/30 file after the snapshot 2024-03-09T08:55:22.258 INFO:tasks.workunit.client.0.smithi067.stdout: create 143/31 file after the snapshot 2024-03-09T08:55:22.258 INFO:tasks.workunit.client.0.smithi067.stdout: create 143/32 file after the snapshot 2024-03-09T08:55:22.258 INFO:tasks.workunit.client.0.smithi067.stdout: create 143/33 file after the snapshot 2024-03-09T08:55:22.258 INFO:tasks.workunit.client.0.smithi067.stdout: create 143/34 file after the snapshot 2024-03-09T08:55:22.259 INFO:tasks.workunit.client.0.smithi067.stdout: create 143/35 file after the snapshot 2024-03-09T08:55:22.259 INFO:tasks.workunit.client.0.smithi067.stdout: create 143/36 file after the snapshot 2024-03-09T08:55:22.259 INFO:tasks.workunit.client.0.smithi067.stdout: create 143/37 file after the snapshot 2024-03-09T08:55:22.259 INFO:tasks.workunit.client.0.smithi067.stdout: create 143/38 file after the snapshot 2024-03-09T08:55:22.259 INFO:tasks.workunit.client.0.smithi067.stdout: create 143/39 file after the snapshot 2024-03-09T08:55:22.259 INFO:tasks.workunit.client.0.smithi067.stdout: create 144/.snap/snap-subdir-test 2024-03-09T08:55:22.320 INFO:tasks.workunit.client.1.smithi134.stdout: create 185/30 file after the snapshot 2024-03-09T08:55:22.323 INFO:tasks.workunit.client.1.smithi134.stdout: create 185/31 file after the snapshot 2024-03-09T08:55:22.323 INFO:tasks.workunit.client.1.smithi134.stdout: create 185/32 file after the snapshot 2024-03-09T08:55:22.323 INFO:tasks.workunit.client.1.smithi134.stdout: create 185/33 file after the snapshot 2024-03-09T08:55:22.323 INFO:tasks.workunit.client.1.smithi134.stdout: create 185/34 file after the snapshot 2024-03-09T08:55:22.324 INFO:tasks.workunit.client.1.smithi134.stdout: create 185/35 file after the snapshot 2024-03-09T08:55:22.324 INFO:tasks.workunit.client.1.smithi134.stdout: create 185/36 file after the snapshot 2024-03-09T08:55:22.324 INFO:tasks.workunit.client.1.smithi134.stdout: create 185/37 file after the snapshot 2024-03-09T08:55:22.324 INFO:tasks.workunit.client.1.smithi134.stdout: create 185/38 file after the snapshot 2024-03-09T08:55:22.324 INFO:tasks.workunit.client.1.smithi134.stdout: create 185/39 file after the snapshot 2024-03-09T08:55:22.324 INFO:tasks.workunit.client.1.smithi134.stdout: create 186/.snap/snap-subdir-test 2024-03-09T08:55:22.352 INFO:tasks.workunit.client.0.smithi067.stdout: create 144/30 file after the snapshot 2024-03-09T08:55:22.357 INFO:tasks.workunit.client.0.smithi067.stdout: create 144/31 file after the snapshot 2024-03-09T08:55:22.357 INFO:tasks.workunit.client.0.smithi067.stdout: create 144/32 file after the snapshot 2024-03-09T08:55:22.357 INFO:tasks.workunit.client.0.smithi067.stdout: create 144/33 file after the snapshot 2024-03-09T08:55:22.359 INFO:tasks.workunit.client.0.smithi067.stdout: create 144/34 file after the snapshot 2024-03-09T08:55:22.360 INFO:tasks.workunit.client.0.smithi067.stdout: create 144/35 file after the snapshot 2024-03-09T08:55:22.360 INFO:tasks.workunit.client.0.smithi067.stdout: create 144/36 file after the snapshot 2024-03-09T08:55:22.360 INFO:tasks.workunit.client.0.smithi067.stdout: create 144/37 file after the snapshot 2024-03-09T08:55:22.360 INFO:tasks.workunit.client.0.smithi067.stdout: create 144/38 file after the snapshot 2024-03-09T08:55:22.360 INFO:tasks.workunit.client.0.smithi067.stdout: create 144/39 file after the snapshot 2024-03-09T08:55:22.360 INFO:tasks.workunit.client.0.smithi067.stdout: create 145/.snap/snap-subdir-test 2024-03-09T08:55:22.381 INFO:tasks.workunit.client.1.smithi134.stdout: create 186/30 file after the snapshot 2024-03-09T08:55:22.383 INFO:tasks.workunit.client.1.smithi134.stdout: create 186/31 file after the snapshot 2024-03-09T08:55:22.383 INFO:tasks.workunit.client.1.smithi134.stdout: create 186/32 file after the snapshot 2024-03-09T08:55:22.384 INFO:tasks.workunit.client.1.smithi134.stdout: create 186/33 file after the snapshot 2024-03-09T08:55:22.384 INFO:tasks.workunit.client.1.smithi134.stdout: create 186/34 file after the snapshot 2024-03-09T08:55:22.384 INFO:tasks.workunit.client.1.smithi134.stdout: create 186/35 file after the snapshot 2024-03-09T08:55:22.385 INFO:tasks.workunit.client.1.smithi134.stdout: create 186/36 file after the snapshot 2024-03-09T08:55:22.385 INFO:tasks.workunit.client.1.smithi134.stdout: create 186/37 file after the snapshot 2024-03-09T08:55:22.385 INFO:tasks.workunit.client.1.smithi134.stdout: create 186/38 file after the snapshot 2024-03-09T08:55:22.386 INFO:tasks.workunit.client.1.smithi134.stdout: create 186/39 file after the snapshot 2024-03-09T08:55:22.386 INFO:tasks.workunit.client.1.smithi134.stdout: create 187/.snap/snap-subdir-test 2024-03-09T08:55:22.434 INFO:tasks.workunit.client.0.smithi067.stdout: create 145/30 file after the snapshot 2024-03-09T08:55:22.438 INFO:tasks.workunit.client.0.smithi067.stdout: create 145/31 file after the snapshot 2024-03-09T08:55:22.438 INFO:tasks.workunit.client.0.smithi067.stdout: create 145/32 file after the snapshot 2024-03-09T08:55:22.438 INFO:tasks.workunit.client.0.smithi067.stdout: create 145/33 file after the snapshot 2024-03-09T08:55:22.438 INFO:tasks.workunit.client.0.smithi067.stdout: create 145/34 file after the snapshot 2024-03-09T08:55:22.440 INFO:tasks.workunit.client.0.smithi067.stdout: create 145/35 file after the snapshot 2024-03-09T08:55:22.440 INFO:tasks.workunit.client.0.smithi067.stdout: create 145/36 file after the snapshot 2024-03-09T08:55:22.440 INFO:tasks.workunit.client.0.smithi067.stdout: create 145/37 file after the snapshot 2024-03-09T08:55:22.441 INFO:tasks.workunit.client.0.smithi067.stdout: create 145/38 file after the snapshot 2024-03-09T08:55:22.441 INFO:tasks.workunit.client.0.smithi067.stdout: create 145/39 file after the snapshot 2024-03-09T08:55:22.441 INFO:tasks.workunit.client.0.smithi067.stdout: create 146/.snap/snap-subdir-test 2024-03-09T08:55:22.528 INFO:tasks.workunit.client.0.smithi067.stdout: create 146/30 file after the snapshot 2024-03-09T08:55:22.530 INFO:tasks.workunit.client.0.smithi067.stdout: create 146/31 file after the snapshot 2024-03-09T08:55:22.533 INFO:tasks.workunit.client.0.smithi067.stdout: create 146/32 file after the snapshot 2024-03-09T08:55:22.533 INFO:tasks.workunit.client.0.smithi067.stdout: create 146/33 file after the snapshot 2024-03-09T08:55:22.534 INFO:tasks.workunit.client.0.smithi067.stdout: create 146/34 file after the snapshot 2024-03-09T08:55:22.534 INFO:tasks.workunit.client.0.smithi067.stdout: create 146/35 file after the snapshot 2024-03-09T08:55:22.534 INFO:tasks.workunit.client.0.smithi067.stdout: create 146/36 file after the snapshot 2024-03-09T08:55:22.534 INFO:tasks.workunit.client.0.smithi067.stdout: create 146/37 file after the snapshot 2024-03-09T08:55:22.534 INFO:tasks.workunit.client.0.smithi067.stdout: create 146/38 file after the snapshot 2024-03-09T08:55:22.534 INFO:tasks.workunit.client.0.smithi067.stdout: create 146/39 file after the snapshot 2024-03-09T08:55:22.541 INFO:tasks.workunit.client.0.smithi067.stdout: create 147/.snap/snap-subdir-test 2024-03-09T08:55:22.868 INFO:tasks.workunit.client.0.smithi067.stdout: create 147/30 file after the snapshot 2024-03-09T08:55:22.874 INFO:tasks.workunit.client.0.smithi067.stdout: create 147/31 file after the snapshot 2024-03-09T08:55:22.875 INFO:tasks.workunit.client.0.smithi067.stdout: create 147/32 file after the snapshot 2024-03-09T08:55:22.875 INFO:tasks.workunit.client.0.smithi067.stdout: create 147/33 file after the snapshot 2024-03-09T08:55:22.875 INFO:tasks.workunit.client.0.smithi067.stdout: create 147/34 file after the snapshot 2024-03-09T08:55:22.875 INFO:tasks.workunit.client.0.smithi067.stdout: create 147/35 file after the snapshot 2024-03-09T08:55:22.875 INFO:tasks.workunit.client.0.smithi067.stdout: create 147/36 file after the snapshot 2024-03-09T08:55:22.876 INFO:tasks.workunit.client.0.smithi067.stdout: create 147/37 file after the snapshot 2024-03-09T08:55:22.876 INFO:tasks.workunit.client.0.smithi067.stdout: create 147/38 file after the snapshot 2024-03-09T08:55:22.876 INFO:tasks.workunit.client.0.smithi067.stdout: create 147/39 file after the snapshot 2024-03-09T08:55:22.876 INFO:tasks.workunit.client.0.smithi067.stdout: create 148/.snap/snap-subdir-test 2024-03-09T08:55:22.880 INFO:tasks.workunit.client.1.smithi134.stdout: create 187/30 file after the snapshot 2024-03-09T08:55:22.883 INFO:tasks.workunit.client.1.smithi134.stdout: create 187/31 file after the snapshot 2024-03-09T08:55:22.883 INFO:tasks.workunit.client.1.smithi134.stdout: create 187/32 file after the snapshot 2024-03-09T08:55:22.884 INFO:tasks.workunit.client.1.smithi134.stdout: create 187/33 file after the snapshot 2024-03-09T08:55:22.884 INFO:tasks.workunit.client.1.smithi134.stdout: create 187/34 file after the snapshot 2024-03-09T08:55:22.885 INFO:tasks.workunit.client.1.smithi134.stdout: create 187/35 file after the snapshot 2024-03-09T08:55:22.885 INFO:tasks.workunit.client.1.smithi134.stdout: create 187/36 file after the snapshot 2024-03-09T08:55:22.885 INFO:tasks.workunit.client.1.smithi134.stdout: create 187/37 file after the snapshot 2024-03-09T08:55:22.885 INFO:tasks.workunit.client.1.smithi134.stdout: create 187/38 file after the snapshot 2024-03-09T08:55:22.885 INFO:tasks.workunit.client.1.smithi134.stdout: create 187/39 file after the snapshot 2024-03-09T08:55:22.885 INFO:tasks.workunit.client.1.smithi134.stdout: create 188/.snap/snap-subdir-test 2024-03-09T08:55:22.949 INFO:tasks.workunit.client.0.smithi067.stdout: create 148/30 file after the snapshot 2024-03-09T08:55:22.953 INFO:tasks.workunit.client.0.smithi067.stdout: create 148/31 file after the snapshot 2024-03-09T08:55:22.955 INFO:tasks.workunit.client.0.smithi067.stdout: create 148/32 file after the snapshot 2024-03-09T08:55:22.955 INFO:tasks.workunit.client.0.smithi067.stdout: create 148/33 file after the snapshot 2024-03-09T08:55:22.955 INFO:tasks.workunit.client.0.smithi067.stdout: create 148/34 file after the snapshot 2024-03-09T08:55:22.955 INFO:tasks.workunit.client.0.smithi067.stdout: create 148/35 file after the snapshot 2024-03-09T08:55:22.956 INFO:tasks.workunit.client.0.smithi067.stdout: create 148/36 file after the snapshot 2024-03-09T08:55:22.957 INFO:tasks.workunit.client.0.smithi067.stdout: create 148/37 file after the snapshot 2024-03-09T08:55:22.957 INFO:tasks.workunit.client.0.smithi067.stdout: create 148/38 file after the snapshot 2024-03-09T08:55:22.957 INFO:tasks.workunit.client.0.smithi067.stdout: create 148/39 file after the snapshot 2024-03-09T08:55:22.957 INFO:tasks.workunit.client.0.smithi067.stdout: create 149/.snap/snap-subdir-test 2024-03-09T08:55:22.972 INFO:tasks.workunit.client.1.smithi134.stdout: create 188/30 file after the snapshot 2024-03-09T08:55:22.974 INFO:tasks.workunit.client.1.smithi134.stdout: create 188/31 file after the snapshot 2024-03-09T08:55:22.974 INFO:tasks.workunit.client.1.smithi134.stdout: create 188/32 file after the snapshot 2024-03-09T08:55:22.974 INFO:tasks.workunit.client.1.smithi134.stdout: create 188/33 file after the snapshot 2024-03-09T08:55:22.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 188/34 file after the snapshot 2024-03-09T08:55:22.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 188/35 file after the snapshot 2024-03-09T08:55:22.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 188/36 file after the snapshot 2024-03-09T08:55:22.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 188/37 file after the snapshot 2024-03-09T08:55:22.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 188/38 file after the snapshot 2024-03-09T08:55:22.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 188/39 file after the snapshot 2024-03-09T08:55:22.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 189/.snap/snap-subdir-test 2024-03-09T08:55:23.028 INFO:tasks.workunit.client.0.smithi067.stdout: create 149/30 file after the snapshot 2024-03-09T08:55:23.031 INFO:tasks.workunit.client.0.smithi067.stdout: create 149/31 file after the snapshot 2024-03-09T08:55:23.031 INFO:tasks.workunit.client.0.smithi067.stdout: create 149/32 file after the snapshot 2024-03-09T08:55:23.032 INFO:tasks.workunit.client.0.smithi067.stdout: create 149/33 file after the snapshot 2024-03-09T08:55:23.032 INFO:tasks.workunit.client.0.smithi067.stdout: create 149/34 file after the snapshot 2024-03-09T08:55:23.033 INFO:tasks.workunit.client.0.smithi067.stdout: create 149/35 file after the snapshot 2024-03-09T08:55:23.033 INFO:tasks.workunit.client.0.smithi067.stdout: create 149/36 file after the snapshot 2024-03-09T08:55:23.033 INFO:tasks.workunit.client.0.smithi067.stdout: create 149/37 file after the snapshot 2024-03-09T08:55:23.033 INFO:tasks.workunit.client.0.smithi067.stdout: create 149/38 file after the snapshot 2024-03-09T08:55:23.033 INFO:tasks.workunit.client.0.smithi067.stdout: create 149/39 file after the snapshot 2024-03-09T08:55:23.034 INFO:tasks.workunit.client.0.smithi067.stdout: create 150/.snap/snap-subdir-test 2024-03-09T08:55:23.104 INFO:tasks.workunit.client.0.smithi067.stdout: create 150/30 file after the snapshot 2024-03-09T08:55:23.106 INFO:tasks.workunit.client.0.smithi067.stdout: create 150/31 file after the snapshot 2024-03-09T08:55:23.106 INFO:tasks.workunit.client.0.smithi067.stdout: create 150/32 file after the snapshot 2024-03-09T08:55:23.107 INFO:tasks.workunit.client.0.smithi067.stdout: create 150/33 file after the snapshot 2024-03-09T08:55:23.107 INFO:tasks.workunit.client.0.smithi067.stdout: create 150/34 file after the snapshot 2024-03-09T08:55:23.108 INFO:tasks.workunit.client.0.smithi067.stdout: create 150/35 file after the snapshot 2024-03-09T08:55:23.108 INFO:tasks.workunit.client.0.smithi067.stdout: create 150/36 file after the snapshot 2024-03-09T08:55:23.108 INFO:tasks.workunit.client.0.smithi067.stdout: create 150/37 file after the snapshot 2024-03-09T08:55:23.108 INFO:tasks.workunit.client.0.smithi067.stdout: create 150/38 file after the snapshot 2024-03-09T08:55:23.108 INFO:tasks.workunit.client.0.smithi067.stdout: create 150/39 file after the snapshot 2024-03-09T08:55:23.108 INFO:tasks.workunit.client.0.smithi067.stdout: create 151/.snap/snap-subdir-test 2024-03-09T08:55:23.173 INFO:tasks.workunit.client.0.smithi067.stdout: create 151/30 file after the snapshot 2024-03-09T08:55:23.176 INFO:tasks.workunit.client.0.smithi067.stdout: create 151/31 file after the snapshot 2024-03-09T08:55:23.176 INFO:tasks.workunit.client.0.smithi067.stdout: create 151/32 file after the snapshot 2024-03-09T08:55:23.176 INFO:tasks.workunit.client.0.smithi067.stdout: create 151/33 file after the snapshot 2024-03-09T08:55:23.177 INFO:tasks.workunit.client.0.smithi067.stdout: create 151/34 file after the snapshot 2024-03-09T08:55:23.177 INFO:tasks.workunit.client.0.smithi067.stdout: create 151/35 file after the snapshot 2024-03-09T08:55:23.177 INFO:tasks.workunit.client.0.smithi067.stdout: create 151/36 file after the snapshot 2024-03-09T08:55:23.178 INFO:tasks.workunit.client.0.smithi067.stdout: create 151/37 file after the snapshot 2024-03-09T08:55:23.178 INFO:tasks.workunit.client.0.smithi067.stdout: create 151/38 file after the snapshot 2024-03-09T08:55:23.178 INFO:tasks.workunit.client.0.smithi067.stdout: create 151/39 file after the snapshot 2024-03-09T08:55:23.178 INFO:tasks.workunit.client.0.smithi067.stdout: create 152/.snap/snap-subdir-test 2024-03-09T08:55:23.872 INFO:tasks.workunit.client.0.smithi067.stdout: create 152/30 file after the snapshot 2024-03-09T08:55:23.875 INFO:tasks.workunit.client.0.smithi067.stdout: create 152/31 file after the snapshot 2024-03-09T08:55:23.875 INFO:tasks.workunit.client.0.smithi067.stdout: create 152/32 file after the snapshot 2024-03-09T08:55:23.875 INFO:tasks.workunit.client.0.smithi067.stdout: create 152/33 file after the snapshot 2024-03-09T08:55:23.876 INFO:tasks.workunit.client.0.smithi067.stdout: create 152/34 file after the snapshot 2024-03-09T08:55:23.876 INFO:tasks.workunit.client.0.smithi067.stdout: create 152/35 file after the snapshot 2024-03-09T08:55:23.876 INFO:tasks.workunit.client.0.smithi067.stdout: create 152/36 file after the snapshot 2024-03-09T08:55:23.877 INFO:tasks.workunit.client.0.smithi067.stdout: create 152/37 file after the snapshot 2024-03-09T08:55:23.877 INFO:tasks.workunit.client.0.smithi067.stdout: create 152/38 file after the snapshot 2024-03-09T08:55:23.877 INFO:tasks.workunit.client.0.smithi067.stdout: create 152/39 file after the snapshot 2024-03-09T08:55:23.877 INFO:tasks.workunit.client.0.smithi067.stdout: create 153/.snap/snap-subdir-test 2024-03-09T08:55:23.891 INFO:tasks.workunit.client.1.smithi134.stdout: create 189/30 file after the snapshot 2024-03-09T08:55:23.894 INFO:tasks.workunit.client.1.smithi134.stdout: create 189/31 file after the snapshot 2024-03-09T08:55:23.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 189/32 file after the snapshot 2024-03-09T08:55:23.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 189/33 file after the snapshot 2024-03-09T08:55:23.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 189/34 file after the snapshot 2024-03-09T08:55:23.896 INFO:tasks.workunit.client.1.smithi134.stdout: create 189/35 file after the snapshot 2024-03-09T08:55:23.896 INFO:tasks.workunit.client.1.smithi134.stdout: create 189/36 file after the snapshot 2024-03-09T08:55:23.896 INFO:tasks.workunit.client.1.smithi134.stdout: create 189/37 file after the snapshot 2024-03-09T08:55:23.896 INFO:tasks.workunit.client.1.smithi134.stdout: create 189/38 file after the snapshot 2024-03-09T08:55:23.897 INFO:tasks.workunit.client.1.smithi134.stdout: create 189/39 file after the snapshot 2024-03-09T08:55:23.897 INFO:tasks.workunit.client.1.smithi134.stdout: create 190/.snap/snap-subdir-test 2024-03-09T08:55:23.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 153/30 file after the snapshot 2024-03-09T08:55:23.971 INFO:tasks.workunit.client.0.smithi067.stdout: create 153/31 file after the snapshot 2024-03-09T08:55:23.972 INFO:tasks.workunit.client.0.smithi067.stdout: create 153/32 file after the snapshot 2024-03-09T08:55:23.972 INFO:tasks.workunit.client.0.smithi067.stdout: create 153/33 file after the snapshot 2024-03-09T08:55:23.973 INFO:tasks.workunit.client.0.smithi067.stdout: create 153/34 file after the snapshot 2024-03-09T08:55:23.973 INFO:tasks.workunit.client.0.smithi067.stdout: create 153/35 file after the snapshot 2024-03-09T08:55:23.973 INFO:tasks.workunit.client.0.smithi067.stdout: create 153/36 file after the snapshot 2024-03-09T08:55:23.974 INFO:tasks.workunit.client.0.smithi067.stdout: create 153/37 file after the snapshot 2024-03-09T08:55:23.975 INFO:tasks.workunit.client.0.smithi067.stdout: create 153/38 file after the snapshot 2024-03-09T08:55:23.975 INFO:tasks.workunit.client.0.smithi067.stdout: create 153/39 file after the snapshot 2024-03-09T08:55:23.975 INFO:tasks.workunit.client.0.smithi067.stdout: create 154/.snap/snap-subdir-test 2024-03-09T08:55:23.990 INFO:tasks.workunit.client.1.smithi134.stdout: create 190/30 file after the snapshot 2024-03-09T08:55:23.993 INFO:tasks.workunit.client.1.smithi134.stdout: create 190/31 file after the snapshot 2024-03-09T08:55:23.994 INFO:tasks.workunit.client.1.smithi134.stdout: create 190/32 file after the snapshot 2024-03-09T08:55:23.994 INFO:tasks.workunit.client.1.smithi134.stdout: create 190/33 file after the snapshot 2024-03-09T08:55:23.995 INFO:tasks.workunit.client.1.smithi134.stdout: create 190/34 file after the snapshot 2024-03-09T08:55:23.995 INFO:tasks.workunit.client.1.smithi134.stdout: create 190/35 file after the snapshot 2024-03-09T08:55:23.995 INFO:tasks.workunit.client.1.smithi134.stdout: create 190/36 file after the snapshot 2024-03-09T08:55:23.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 190/37 file after the snapshot 2024-03-09T08:55:23.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 190/38 file after the snapshot 2024-03-09T08:55:23.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 190/39 file after the snapshot 2024-03-09T08:55:23.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 191/.snap/snap-subdir-test 2024-03-09T08:55:24.047 INFO:tasks.workunit.client.0.smithi067.stdout: create 154/30 file after the snapshot 2024-03-09T08:55:24.050 INFO:tasks.workunit.client.0.smithi067.stdout: create 154/31 file after the snapshot 2024-03-09T08:55:24.050 INFO:tasks.workunit.client.0.smithi067.stdout: create 154/32 file after the snapshot 2024-03-09T08:55:24.050 INFO:tasks.workunit.client.0.smithi067.stdout: create 154/33 file after the snapshot 2024-03-09T08:55:24.051 INFO:tasks.workunit.client.0.smithi067.stdout: create 154/34 file after the snapshot 2024-03-09T08:55:24.051 INFO:tasks.workunit.client.0.smithi067.stdout: create 154/35 file after the snapshot 2024-03-09T08:55:24.051 INFO:tasks.workunit.client.0.smithi067.stdout: create 154/36 file after the snapshot 2024-03-09T08:55:24.051 INFO:tasks.workunit.client.0.smithi067.stdout: create 154/37 file after the snapshot 2024-03-09T08:55:24.051 INFO:tasks.workunit.client.0.smithi067.stdout: create 154/38 file after the snapshot 2024-03-09T08:55:24.051 INFO:tasks.workunit.client.0.smithi067.stdout: create 154/39 file after the snapshot 2024-03-09T08:55:24.052 INFO:tasks.workunit.client.0.smithi067.stdout: create 155/.snap/snap-subdir-test 2024-03-09T08:55:24.095 INFO:tasks.workunit.client.1.smithi134.stdout: create 191/30 file after the snapshot 2024-03-09T08:55:24.098 INFO:tasks.workunit.client.1.smithi134.stdout: create 191/31 file after the snapshot 2024-03-09T08:55:24.098 INFO:tasks.workunit.client.1.smithi134.stdout: create 191/32 file after the snapshot 2024-03-09T08:55:24.099 INFO:tasks.workunit.client.1.smithi134.stdout: create 191/33 file after the snapshot 2024-03-09T08:55:24.099 INFO:tasks.workunit.client.1.smithi134.stdout: create 191/34 file after the snapshot 2024-03-09T08:55:24.100 INFO:tasks.workunit.client.1.smithi134.stdout: create 191/35 file after the snapshot 2024-03-09T08:55:24.100 INFO:tasks.workunit.client.1.smithi134.stdout: create 191/36 file after the snapshot 2024-03-09T08:55:24.101 INFO:tasks.workunit.client.1.smithi134.stdout: create 191/37 file after the snapshot 2024-03-09T08:55:24.101 INFO:tasks.workunit.client.1.smithi134.stdout: create 191/38 file after the snapshot 2024-03-09T08:55:24.102 INFO:tasks.workunit.client.1.smithi134.stdout: create 191/39 file after the snapshot 2024-03-09T08:55:24.102 INFO:tasks.workunit.client.1.smithi134.stdout: create 192/.snap/snap-subdir-test 2024-03-09T08:55:24.111 INFO:tasks.workunit.client.0.smithi067.stdout: create 155/30 file after the snapshot 2024-03-09T08:55:24.114 INFO:tasks.workunit.client.0.smithi067.stdout: create 155/31 file after the snapshot 2024-03-09T08:55:24.114 INFO:tasks.workunit.client.0.smithi067.stdout: create 155/32 file after the snapshot 2024-03-09T08:55:24.114 INFO:tasks.workunit.client.0.smithi067.stdout: create 155/33 file after the snapshot 2024-03-09T08:55:24.114 INFO:tasks.workunit.client.0.smithi067.stdout: create 155/34 file after the snapshot 2024-03-09T08:55:24.115 INFO:tasks.workunit.client.0.smithi067.stdout: create 155/35 file after the snapshot 2024-03-09T08:55:24.115 INFO:tasks.workunit.client.0.smithi067.stdout: create 155/36 file after the snapshot 2024-03-09T08:55:24.115 INFO:tasks.workunit.client.0.smithi067.stdout: create 155/37 file after the snapshot 2024-03-09T08:55:24.115 INFO:tasks.workunit.client.0.smithi067.stdout: create 155/38 file after the snapshot 2024-03-09T08:55:24.116 INFO:tasks.workunit.client.0.smithi067.stdout: create 155/39 file after the snapshot 2024-03-09T08:55:24.116 INFO:tasks.workunit.client.0.smithi067.stdout: create 156/.snap/snap-subdir-test 2024-03-09T08:55:24.182 INFO:tasks.workunit.client.1.smithi134.stdout: create 192/30 file after the snapshot 2024-03-09T08:55:24.184 INFO:tasks.workunit.client.1.smithi134.stdout: create 192/31 file after the snapshot 2024-03-09T08:55:24.184 INFO:tasks.workunit.client.1.smithi134.stdout: create 192/32 file after the snapshot 2024-03-09T08:55:24.185 INFO:tasks.workunit.client.1.smithi134.stdout: create 192/33 file after the snapshot 2024-03-09T08:55:24.185 INFO:tasks.workunit.client.1.smithi134.stdout: create 192/34 file after the snapshot 2024-03-09T08:55:24.185 INFO:tasks.workunit.client.1.smithi134.stdout: create 192/35 file after the snapshot 2024-03-09T08:55:24.185 INFO:tasks.workunit.client.1.smithi134.stdout: create 192/36 file after the snapshot 2024-03-09T08:55:24.186 INFO:tasks.workunit.client.1.smithi134.stdout: create 192/37 file after the snapshot 2024-03-09T08:55:24.186 INFO:tasks.workunit.client.1.smithi134.stdout: create 192/38 file after the snapshot 2024-03-09T08:55:24.186 INFO:tasks.workunit.client.1.smithi134.stdout: create 192/39 file after the snapshot 2024-03-09T08:55:24.187 INFO:tasks.workunit.client.1.smithi134.stdout: create 193/.snap/snap-subdir-test 2024-03-09T08:55:24.253 INFO:tasks.workunit.client.0.smithi067.stdout: create 156/30 file after the snapshot 2024-03-09T08:55:24.256 INFO:tasks.workunit.client.0.smithi067.stdout: create 156/31 file after the snapshot 2024-03-09T08:55:24.256 INFO:tasks.workunit.client.0.smithi067.stdout: create 156/32 file after the snapshot 2024-03-09T08:55:24.257 INFO:tasks.workunit.client.0.smithi067.stdout: create 156/33 file after the snapshot 2024-03-09T08:55:24.257 INFO:tasks.workunit.client.0.smithi067.stdout: create 156/34 file after the snapshot 2024-03-09T08:55:24.257 INFO:tasks.workunit.client.0.smithi067.stdout: create 156/35 file after the snapshot 2024-03-09T08:55:24.258 INFO:tasks.workunit.client.0.smithi067.stdout: create 156/36 file after the snapshot 2024-03-09T08:55:24.258 INFO:tasks.workunit.client.0.smithi067.stdout: create 156/37 file after the snapshot 2024-03-09T08:55:24.258 INFO:tasks.workunit.client.0.smithi067.stdout: create 156/38 file after the snapshot 2024-03-09T08:55:24.258 INFO:tasks.workunit.client.0.smithi067.stdout: create 156/39 file after the snapshot 2024-03-09T08:55:24.258 INFO:tasks.workunit.client.0.smithi067.stdout: create 157/.snap/snap-subdir-test 2024-03-09T08:55:24.269 INFO:tasks.workunit.client.1.smithi134.stdout: create 193/30 file after the snapshot 2024-03-09T08:55:24.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 193/31 file after the snapshot 2024-03-09T08:55:24.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 193/32 file after the snapshot 2024-03-09T08:55:24.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 193/33 file after the snapshot 2024-03-09T08:55:24.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 193/34 file after the snapshot 2024-03-09T08:55:24.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 193/35 file after the snapshot 2024-03-09T08:55:24.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 193/36 file after the snapshot 2024-03-09T08:55:24.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 193/37 file after the snapshot 2024-03-09T08:55:24.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 193/38 file after the snapshot 2024-03-09T08:55:24.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 193/39 file after the snapshot 2024-03-09T08:55:24.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 194/.snap/snap-subdir-test 2024-03-09T08:55:24.334 INFO:tasks.workunit.client.0.smithi067.stdout: create 157/30 file after the snapshot 2024-03-09T08:55:24.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 157/31 file after the snapshot 2024-03-09T08:55:24.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 157/32 file after the snapshot 2024-03-09T08:55:24.338 INFO:tasks.workunit.client.0.smithi067.stdout: create 157/33 file after the snapshot 2024-03-09T08:55:24.338 INFO:tasks.workunit.client.0.smithi067.stdout: create 157/34 file after the snapshot 2024-03-09T08:55:24.338 INFO:tasks.workunit.client.0.smithi067.stdout: create 157/35 file after the snapshot 2024-03-09T08:55:24.339 INFO:tasks.workunit.client.0.smithi067.stdout: create 157/36 file after the snapshot 2024-03-09T08:55:24.339 INFO:tasks.workunit.client.0.smithi067.stdout: create 157/37 file after the snapshot 2024-03-09T08:55:24.339 INFO:tasks.workunit.client.0.smithi067.stdout: create 157/38 file after the snapshot 2024-03-09T08:55:24.339 INFO:tasks.workunit.client.0.smithi067.stdout: create 157/39 file after the snapshot 2024-03-09T08:55:24.340 INFO:tasks.workunit.client.0.smithi067.stdout: create 158/.snap/snap-subdir-test 2024-03-09T08:55:24.359 INFO:tasks.workunit.client.1.smithi134.stdout: create 194/30 file after the snapshot 2024-03-09T08:55:24.361 INFO:tasks.workunit.client.1.smithi134.stdout: create 194/31 file after the snapshot 2024-03-09T08:55:24.361 INFO:tasks.workunit.client.1.smithi134.stdout: create 194/32 file after the snapshot 2024-03-09T08:55:24.361 INFO:tasks.workunit.client.1.smithi134.stdout: create 194/33 file after the snapshot 2024-03-09T08:55:24.362 INFO:tasks.workunit.client.1.smithi134.stdout: create 194/34 file after the snapshot 2024-03-09T08:55:24.362 INFO:tasks.workunit.client.1.smithi134.stdout: create 194/35 file after the snapshot 2024-03-09T08:55:24.362 INFO:tasks.workunit.client.1.smithi134.stdout: create 194/36 file after the snapshot 2024-03-09T08:55:24.362 INFO:tasks.workunit.client.1.smithi134.stdout: create 194/37 file after the snapshot 2024-03-09T08:55:24.363 INFO:tasks.workunit.client.1.smithi134.stdout: create 194/38 file after the snapshot 2024-03-09T08:55:24.364 INFO:tasks.workunit.client.1.smithi134.stdout: create 194/39 file after the snapshot 2024-03-09T08:55:24.364 INFO:tasks.workunit.client.1.smithi134.stdout: create 195/.snap/snap-subdir-test 2024-03-09T08:55:24.414 INFO:tasks.workunit.client.0.smithi067.stdout: create 158/30 file after the snapshot 2024-03-09T08:55:24.417 INFO:tasks.workunit.client.0.smithi067.stdout: create 158/31 file after the snapshot 2024-03-09T08:55:24.417 INFO:tasks.workunit.client.0.smithi067.stdout: create 158/32 file after the snapshot 2024-03-09T08:55:24.418 INFO:tasks.workunit.client.0.smithi067.stdout: create 158/33 file after the snapshot 2024-03-09T08:55:24.418 INFO:tasks.workunit.client.0.smithi067.stdout: create 158/34 file after the snapshot 2024-03-09T08:55:24.418 INFO:tasks.workunit.client.0.smithi067.stdout: create 158/35 file after the snapshot 2024-03-09T08:55:24.418 INFO:tasks.workunit.client.0.smithi067.stdout: create 158/36 file after the snapshot 2024-03-09T08:55:24.419 INFO:tasks.workunit.client.0.smithi067.stdout: create 158/37 file after the snapshot 2024-03-09T08:55:24.419 INFO:tasks.workunit.client.0.smithi067.stdout: create 158/38 file after the snapshot 2024-03-09T08:55:24.419 INFO:tasks.workunit.client.0.smithi067.stdout: create 158/39 file after the snapshot 2024-03-09T08:55:24.419 INFO:tasks.workunit.client.0.smithi067.stdout: create 159/.snap/snap-subdir-test 2024-03-09T08:55:24.440 INFO:tasks.workunit.client.1.smithi134.stdout: create 195/30 file after the snapshot 2024-03-09T08:55:24.442 INFO:tasks.workunit.client.1.smithi134.stdout: create 195/31 file after the snapshot 2024-03-09T08:55:24.443 INFO:tasks.workunit.client.1.smithi134.stdout: create 195/32 file after the snapshot 2024-03-09T08:55:24.443 INFO:tasks.workunit.client.1.smithi134.stdout: create 195/33 file after the snapshot 2024-03-09T08:55:24.443 INFO:tasks.workunit.client.1.smithi134.stdout: create 195/34 file after the snapshot 2024-03-09T08:55:24.444 INFO:tasks.workunit.client.1.smithi134.stdout: create 195/35 file after the snapshot 2024-03-09T08:55:24.444 INFO:tasks.workunit.client.1.smithi134.stdout: create 195/36 file after the snapshot 2024-03-09T08:55:24.444 INFO:tasks.workunit.client.1.smithi134.stdout: create 195/37 file after the snapshot 2024-03-09T08:55:24.445 INFO:tasks.workunit.client.1.smithi134.stdout: create 195/38 file after the snapshot 2024-03-09T08:55:24.445 INFO:tasks.workunit.client.1.smithi134.stdout: create 195/39 file after the snapshot 2024-03-09T08:55:24.446 INFO:tasks.workunit.client.1.smithi134.stdout: create 196/.snap/snap-subdir-test 2024-03-09T08:55:24.486 INFO:tasks.workunit.client.0.smithi067.stdout: create 159/30 file after the snapshot 2024-03-09T08:55:24.489 INFO:tasks.workunit.client.0.smithi067.stdout: create 159/31 file after the snapshot 2024-03-09T08:55:24.490 INFO:tasks.workunit.client.0.smithi067.stdout: create 159/32 file after the snapshot 2024-03-09T08:55:24.490 INFO:tasks.workunit.client.0.smithi067.stdout: create 159/33 file after the snapshot 2024-03-09T08:55:24.490 INFO:tasks.workunit.client.0.smithi067.stdout: create 159/34 file after the snapshot 2024-03-09T08:55:24.490 INFO:tasks.workunit.client.0.smithi067.stdout: create 159/35 file after the snapshot 2024-03-09T08:55:24.490 INFO:tasks.workunit.client.0.smithi067.stdout: create 159/36 file after the snapshot 2024-03-09T08:55:24.491 INFO:tasks.workunit.client.0.smithi067.stdout: create 159/37 file after the snapshot 2024-03-09T08:55:24.491 INFO:tasks.workunit.client.0.smithi067.stdout: create 159/38 file after the snapshot 2024-03-09T08:55:24.491 INFO:tasks.workunit.client.0.smithi067.stdout: create 159/39 file after the snapshot 2024-03-09T08:55:24.491 INFO:tasks.workunit.client.0.smithi067.stdout: create 160/.snap/snap-subdir-test 2024-03-09T08:55:24.532 INFO:tasks.workunit.client.1.smithi134.stdout: create 196/30 file after the snapshot 2024-03-09T08:55:24.539 INFO:tasks.workunit.client.1.smithi134.stdout: create 196/31 file after the snapshot 2024-03-09T08:55:24.539 INFO:tasks.workunit.client.1.smithi134.stdout: create 196/32 file after the snapshot 2024-03-09T08:55:24.539 INFO:tasks.workunit.client.1.smithi134.stdout: create 196/33 file after the snapshot 2024-03-09T08:55:24.539 INFO:tasks.workunit.client.1.smithi134.stdout: create 196/34 file after the snapshot 2024-03-09T08:55:24.539 INFO:tasks.workunit.client.1.smithi134.stdout: create 196/35 file after the snapshot 2024-03-09T08:55:24.539 INFO:tasks.workunit.client.1.smithi134.stdout: create 196/36 file after the snapshot 2024-03-09T08:55:24.539 INFO:tasks.workunit.client.1.smithi134.stdout: create 196/37 file after the snapshot 2024-03-09T08:55:24.539 INFO:tasks.workunit.client.1.smithi134.stdout: create 196/38 file after the snapshot 2024-03-09T08:55:24.540 INFO:tasks.workunit.client.1.smithi134.stdout: create 196/39 file after the snapshot 2024-03-09T08:55:24.540 INFO:tasks.workunit.client.1.smithi134.stdout: create 197/.snap/snap-subdir-test 2024-03-09T08:55:24.711 INFO:tasks.workunit.client.1.smithi134.stdout: create 197/30 file after the snapshot 2024-03-09T08:55:24.724 INFO:tasks.workunit.client.1.smithi134.stdout: create 197/31 file after the snapshot 2024-03-09T08:55:24.725 INFO:tasks.workunit.client.1.smithi134.stdout: create 197/32 file after the snapshot 2024-03-09T08:55:24.726 INFO:tasks.workunit.client.1.smithi134.stdout: create 197/33 file after the snapshot 2024-03-09T08:55:24.726 INFO:tasks.workunit.client.1.smithi134.stdout: create 197/34 file after the snapshot 2024-03-09T08:55:24.726 INFO:tasks.workunit.client.1.smithi134.stdout: create 197/35 file after the snapshot 2024-03-09T08:55:24.727 INFO:tasks.workunit.client.1.smithi134.stdout: create 197/36 file after the snapshot 2024-03-09T08:55:24.727 INFO:tasks.workunit.client.1.smithi134.stdout: create 197/37 file after the snapshot 2024-03-09T08:55:24.728 INFO:tasks.workunit.client.1.smithi134.stdout: create 197/38 file after the snapshot 2024-03-09T08:55:24.728 INFO:tasks.workunit.client.1.smithi134.stdout: create 197/39 file after the snapshot 2024-03-09T08:55:24.729 INFO:tasks.workunit.client.1.smithi134.stdout: create 198/.snap/snap-subdir-test 2024-03-09T08:55:24.916 INFO:tasks.workunit.client.1.smithi134.stdout: create 198/30 file after the snapshot 2024-03-09T08:55:24.918 INFO:tasks.workunit.client.1.smithi134.stdout: create 198/31 file after the snapshot 2024-03-09T08:55:24.918 INFO:tasks.workunit.client.1.smithi134.stdout: create 198/32 file after the snapshot 2024-03-09T08:55:24.918 INFO:tasks.workunit.client.1.smithi134.stdout: create 198/33 file after the snapshot 2024-03-09T08:55:24.919 INFO:tasks.workunit.client.1.smithi134.stdout: create 198/34 file after the snapshot 2024-03-09T08:55:24.919 INFO:tasks.workunit.client.1.smithi134.stdout: create 198/35 file after the snapshot 2024-03-09T08:55:24.920 INFO:tasks.workunit.client.1.smithi134.stdout: create 198/36 file after the snapshot 2024-03-09T08:55:24.920 INFO:tasks.workunit.client.1.smithi134.stdout: create 198/37 file after the snapshot 2024-03-09T08:55:24.920 INFO:tasks.workunit.client.1.smithi134.stdout: create 198/38 file after the snapshot 2024-03-09T08:55:24.921 INFO:tasks.workunit.client.1.smithi134.stdout: create 198/39 file after the snapshot 2024-03-09T08:55:24.921 INFO:tasks.workunit.client.1.smithi134.stdout: create 199/.snap/snap-subdir-test 2024-03-09T08:55:24.970 INFO:tasks.workunit.client.1.smithi134.stdout: create 199/30 file after the snapshot 2024-03-09T08:55:24.972 INFO:tasks.workunit.client.1.smithi134.stdout: create 199/31 file after the snapshot 2024-03-09T08:55:24.974 INFO:tasks.workunit.client.1.smithi134.stdout: create 199/32 file after the snapshot 2024-03-09T08:55:24.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 199/33 file after the snapshot 2024-03-09T08:55:24.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 199/34 file after the snapshot 2024-03-09T08:55:24.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 199/35 file after the snapshot 2024-03-09T08:55:24.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 199/36 file after the snapshot 2024-03-09T08:55:24.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 199/37 file after the snapshot 2024-03-09T08:55:24.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 199/38 file after the snapshot 2024-03-09T08:55:24.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 199/39 file after the snapshot 2024-03-09T08:55:24.975 INFO:tasks.workunit.client.1.smithi134.stdout: create 200/.snap/snap-subdir-test 2024-03-09T08:55:25.016 INFO:tasks.workunit.client.1.smithi134.stdout: create 200/30 file after the snapshot 2024-03-09T08:55:25.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 200/31 file after the snapshot 2024-03-09T08:55:25.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 200/32 file after the snapshot 2024-03-09T08:55:25.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 200/33 file after the snapshot 2024-03-09T08:55:25.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 200/34 file after the snapshot 2024-03-09T08:55:25.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 200/35 file after the snapshot 2024-03-09T08:55:25.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 200/36 file after the snapshot 2024-03-09T08:55:25.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 200/37 file after the snapshot 2024-03-09T08:55:25.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 200/38 file after the snapshot 2024-03-09T08:55:25.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 200/39 file after the snapshot 2024-03-09T08:55:25.019 INFO:tasks.workunit.client.1.smithi134.stdout: create 201/.snap/snap-subdir-test 2024-03-09T08:55:25.060 INFO:tasks.workunit.client.1.smithi134.stdout: create 201/30 file after the snapshot 2024-03-09T08:55:25.063 INFO:tasks.workunit.client.1.smithi134.stdout: create 201/31 file after the snapshot 2024-03-09T08:55:25.063 INFO:tasks.workunit.client.1.smithi134.stdout: create 201/32 file after the snapshot 2024-03-09T08:55:25.063 INFO:tasks.workunit.client.1.smithi134.stdout: create 201/33 file after the snapshot 2024-03-09T08:55:25.064 INFO:tasks.workunit.client.1.smithi134.stdout: create 201/34 file after the snapshot 2024-03-09T08:55:25.064 INFO:tasks.workunit.client.1.smithi134.stdout: create 201/35 file after the snapshot 2024-03-09T08:55:25.064 INFO:tasks.workunit.client.1.smithi134.stdout: create 201/36 file after the snapshot 2024-03-09T08:55:25.064 INFO:tasks.workunit.client.1.smithi134.stdout: create 201/37 file after the snapshot 2024-03-09T08:55:25.064 INFO:tasks.workunit.client.1.smithi134.stdout: create 201/38 file after the snapshot 2024-03-09T08:55:25.064 INFO:tasks.workunit.client.1.smithi134.stdout: create 201/39 file after the snapshot 2024-03-09T08:55:25.064 INFO:tasks.workunit.client.1.smithi134.stdout: create 202/.snap/snap-subdir-test 2024-03-09T08:55:25.109 INFO:tasks.workunit.client.1.smithi134.stdout: create 202/30 file after the snapshot 2024-03-09T08:55:25.112 INFO:tasks.workunit.client.1.smithi134.stdout: create 202/31 file after the snapshot 2024-03-09T08:55:25.112 INFO:tasks.workunit.client.1.smithi134.stdout: create 202/32 file after the snapshot 2024-03-09T08:55:25.112 INFO:tasks.workunit.client.1.smithi134.stdout: create 202/33 file after the snapshot 2024-03-09T08:55:25.112 INFO:tasks.workunit.client.1.smithi134.stdout: create 202/34 file after the snapshot 2024-03-09T08:55:25.112 INFO:tasks.workunit.client.1.smithi134.stdout: create 202/35 file after the snapshot 2024-03-09T08:55:25.112 INFO:tasks.workunit.client.1.smithi134.stdout: create 202/36 file after the snapshot 2024-03-09T08:55:25.112 INFO:tasks.workunit.client.1.smithi134.stdout: create 202/37 file after the snapshot 2024-03-09T08:55:25.113 INFO:tasks.workunit.client.1.smithi134.stdout: create 202/38 file after the snapshot 2024-03-09T08:55:25.113 INFO:tasks.workunit.client.1.smithi134.stdout: create 202/39 file after the snapshot 2024-03-09T08:55:25.113 INFO:tasks.workunit.client.1.smithi134.stdout: create 203/.snap/snap-subdir-test 2024-03-09T08:55:25.175 INFO:tasks.workunit.client.1.smithi134.stdout: create 203/30 file after the snapshot 2024-03-09T08:55:25.178 INFO:tasks.workunit.client.1.smithi134.stdout: create 203/31 file after the snapshot 2024-03-09T08:55:25.178 INFO:tasks.workunit.client.1.smithi134.stdout: create 203/32 file after the snapshot 2024-03-09T08:55:25.178 INFO:tasks.workunit.client.1.smithi134.stdout: create 203/33 file after the snapshot 2024-03-09T08:55:25.178 INFO:tasks.workunit.client.1.smithi134.stdout: create 203/34 file after the snapshot 2024-03-09T08:55:25.178 INFO:tasks.workunit.client.1.smithi134.stdout: create 203/35 file after the snapshot 2024-03-09T08:55:25.179 INFO:tasks.workunit.client.1.smithi134.stdout: create 203/36 file after the snapshot 2024-03-09T08:55:25.179 INFO:tasks.workunit.client.1.smithi134.stdout: create 203/37 file after the snapshot 2024-03-09T08:55:25.179 INFO:tasks.workunit.client.1.smithi134.stdout: create 203/38 file after the snapshot 2024-03-09T08:55:25.179 INFO:tasks.workunit.client.1.smithi134.stdout: create 203/39 file after the snapshot 2024-03-09T08:55:25.179 INFO:tasks.workunit.client.1.smithi134.stdout: create 204/.snap/snap-subdir-test 2024-03-09T08:55:25.255 INFO:tasks.workunit.client.1.smithi134.stdout: create 204/30 file after the snapshot 2024-03-09T08:55:25.519 INFO:tasks.workunit.client.1.smithi134.stdout: create 204/31 file after the snapshot 2024-03-09T08:55:25.519 INFO:tasks.workunit.client.1.smithi134.stdout: create 204/32 file after the snapshot 2024-03-09T08:55:25.519 INFO:tasks.workunit.client.1.smithi134.stdout: create 204/33 file after the snapshot 2024-03-09T08:55:25.520 INFO:tasks.workunit.client.1.smithi134.stdout: create 204/34 file after the snapshot 2024-03-09T08:55:25.520 INFO:tasks.workunit.client.1.smithi134.stdout: create 204/35 file after the snapshot 2024-03-09T08:55:25.520 INFO:tasks.workunit.client.1.smithi134.stdout: create 204/36 file after the snapshot 2024-03-09T08:55:25.520 INFO:tasks.workunit.client.1.smithi134.stdout: create 204/37 file after the snapshot 2024-03-09T08:55:25.520 INFO:tasks.workunit.client.1.smithi134.stdout: create 204/38 file after the snapshot 2024-03-09T08:55:25.521 INFO:tasks.workunit.client.1.smithi134.stdout: create 204/39 file after the snapshot 2024-03-09T08:55:25.521 INFO:tasks.workunit.client.1.smithi134.stdout: create 205/.snap/snap-subdir-test 2024-03-09T08:55:25.798 INFO:tasks.workunit.client.0.smithi067.stdout: create 160/30 file after the snapshot 2024-03-09T08:55:26.116 INFO:tasks.workunit.client.1.smithi134.stdout: create 205/30 file after the snapshot 2024-03-09T08:55:26.121 INFO:tasks.workunit.client.1.smithi134.stdout: create 205/31 file after the snapshot 2024-03-09T08:55:26.121 INFO:tasks.workunit.client.1.smithi134.stdout: create 205/32 file after the snapshot 2024-03-09T08:55:26.121 INFO:tasks.workunit.client.1.smithi134.stdout: create 205/33 file after the snapshot 2024-03-09T08:55:26.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 205/34 file after the snapshot 2024-03-09T08:55:26.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 205/35 file after the snapshot 2024-03-09T08:55:26.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 205/36 file after the snapshot 2024-03-09T08:55:26.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 205/37 file after the snapshot 2024-03-09T08:55:26.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 205/38 file after the snapshot 2024-03-09T08:55:26.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 205/39 file after the snapshot 2024-03-09T08:55:26.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 206/.snap/snap-subdir-test 2024-03-09T08:55:26.183 INFO:tasks.workunit.client.1.smithi134.stdout: create 206/30 file after the snapshot 2024-03-09T08:55:26.186 INFO:tasks.workunit.client.1.smithi134.stdout: create 206/31 file after the snapshot 2024-03-09T08:55:26.186 INFO:tasks.workunit.client.1.smithi134.stdout: create 206/32 file after the snapshot 2024-03-09T08:55:26.186 INFO:tasks.workunit.client.1.smithi134.stdout: create 206/33 file after the snapshot 2024-03-09T08:55:26.187 INFO:tasks.workunit.client.1.smithi134.stdout: create 206/34 file after the snapshot 2024-03-09T08:55:26.187 INFO:tasks.workunit.client.1.smithi134.stdout: create 206/35 file after the snapshot 2024-03-09T08:55:26.187 INFO:tasks.workunit.client.1.smithi134.stdout: create 206/36 file after the snapshot 2024-03-09T08:55:26.187 INFO:tasks.workunit.client.1.smithi134.stdout: create 206/37 file after the snapshot 2024-03-09T08:55:26.187 INFO:tasks.workunit.client.1.smithi134.stdout: create 206/38 file after the snapshot 2024-03-09T08:55:26.187 INFO:tasks.workunit.client.1.smithi134.stdout: create 206/39 file after the snapshot 2024-03-09T08:55:26.187 INFO:tasks.workunit.client.1.smithi134.stdout: create 207/.snap/snap-subdir-test 2024-03-09T08:55:26.256 INFO:tasks.workunit.client.1.smithi134.stdout: create 207/30 file after the snapshot 2024-03-09T08:55:26.259 INFO:tasks.workunit.client.1.smithi134.stdout: create 207/31 file after the snapshot 2024-03-09T08:55:26.259 INFO:tasks.workunit.client.1.smithi134.stdout: create 207/32 file after the snapshot 2024-03-09T08:55:26.259 INFO:tasks.workunit.client.1.smithi134.stdout: create 207/33 file after the snapshot 2024-03-09T08:55:26.260 INFO:tasks.workunit.client.1.smithi134.stdout: create 207/34 file after the snapshot 2024-03-09T08:55:26.260 INFO:tasks.workunit.client.1.smithi134.stdout: create 207/35 file after the snapshot 2024-03-09T08:55:26.260 INFO:tasks.workunit.client.1.smithi134.stdout: create 207/36 file after the snapshot 2024-03-09T08:55:26.261 INFO:tasks.workunit.client.1.smithi134.stdout: create 207/37 file after the snapshot 2024-03-09T08:55:26.261 INFO:tasks.workunit.client.1.smithi134.stdout: create 207/38 file after the snapshot 2024-03-09T08:55:26.261 INFO:tasks.workunit.client.1.smithi134.stdout: create 207/39 file after the snapshot 2024-03-09T08:55:26.261 INFO:tasks.workunit.client.1.smithi134.stdout: create 208/.snap/snap-subdir-test 2024-03-09T08:55:26.335 INFO:tasks.workunit.client.1.smithi134.stdout: create 208/30 file after the snapshot 2024-03-09T08:55:26.338 INFO:tasks.workunit.client.1.smithi134.stdout: create 208/31 file after the snapshot 2024-03-09T08:55:26.339 INFO:tasks.workunit.client.1.smithi134.stdout: create 208/32 file after the snapshot 2024-03-09T08:55:26.339 INFO:tasks.workunit.client.1.smithi134.stdout: create 208/33 file after the snapshot 2024-03-09T08:55:26.339 INFO:tasks.workunit.client.1.smithi134.stdout: create 208/34 file after the snapshot 2024-03-09T08:55:26.339 INFO:tasks.workunit.client.1.smithi134.stdout: create 208/35 file after the snapshot 2024-03-09T08:55:26.339 INFO:tasks.workunit.client.1.smithi134.stdout: create 208/36 file after the snapshot 2024-03-09T08:55:26.339 INFO:tasks.workunit.client.1.smithi134.stdout: create 208/37 file after the snapshot 2024-03-09T08:55:26.339 INFO:tasks.workunit.client.1.smithi134.stdout: create 208/38 file after the snapshot 2024-03-09T08:55:26.339 INFO:tasks.workunit.client.1.smithi134.stdout: create 208/39 file after the snapshot 2024-03-09T08:55:26.339 INFO:tasks.workunit.client.1.smithi134.stdout: create 209/.snap/snap-subdir-test 2024-03-09T08:55:26.409 INFO:tasks.workunit.client.1.smithi134.stdout: create 209/30 file after the snapshot 2024-03-09T08:55:26.411 INFO:tasks.workunit.client.1.smithi134.stdout: create 209/31 file after the snapshot 2024-03-09T08:55:26.412 INFO:tasks.workunit.client.1.smithi134.stdout: create 209/32 file after the snapshot 2024-03-09T08:55:26.412 INFO:tasks.workunit.client.1.smithi134.stdout: create 209/33 file after the snapshot 2024-03-09T08:55:26.412 INFO:tasks.workunit.client.1.smithi134.stdout: create 209/34 file after the snapshot 2024-03-09T08:55:26.412 INFO:tasks.workunit.client.1.smithi134.stdout: create 209/35 file after the snapshot 2024-03-09T08:55:26.413 INFO:tasks.workunit.client.1.smithi134.stdout: create 209/36 file after the snapshot 2024-03-09T08:55:26.413 INFO:tasks.workunit.client.1.smithi134.stdout: create 209/37 file after the snapshot 2024-03-09T08:55:26.413 INFO:tasks.workunit.client.1.smithi134.stdout: create 209/38 file after the snapshot 2024-03-09T08:55:26.414 INFO:tasks.workunit.client.1.smithi134.stdout: create 209/39 file after the snapshot 2024-03-09T08:55:26.414 INFO:tasks.workunit.client.1.smithi134.stdout: create 210/.snap/snap-subdir-test 2024-03-09T08:55:26.433 INFO:tasks.workunit.client.0.smithi067.stdout: create 160/31 file after the snapshot 2024-03-09T08:55:26.434 INFO:tasks.workunit.client.0.smithi067.stdout: create 160/32 file after the snapshot 2024-03-09T08:55:26.434 INFO:tasks.workunit.client.0.smithi067.stdout: create 160/33 file after the snapshot 2024-03-09T08:55:26.434 INFO:tasks.workunit.client.0.smithi067.stdout: create 160/34 file after the snapshot 2024-03-09T08:55:26.435 INFO:tasks.workunit.client.0.smithi067.stdout: create 160/35 file after the snapshot 2024-03-09T08:55:26.435 INFO:tasks.workunit.client.0.smithi067.stdout: create 160/36 file after the snapshot 2024-03-09T08:55:26.435 INFO:tasks.workunit.client.0.smithi067.stdout: create 160/37 file after the snapshot 2024-03-09T08:55:26.436 INFO:tasks.workunit.client.0.smithi067.stdout: create 160/38 file after the snapshot 2024-03-09T08:55:26.437 INFO:tasks.workunit.client.0.smithi067.stdout: create 160/39 file after the snapshot 2024-03-09T08:55:26.438 INFO:tasks.workunit.client.0.smithi067.stdout: create 161/.snap/snap-subdir-test 2024-03-09T08:55:26.875 INFO:tasks.workunit.client.1.smithi134.stdout: create 210/30 file after the snapshot 2024-03-09T08:55:26.878 INFO:tasks.workunit.client.1.smithi134.stdout: create 210/31 file after the snapshot 2024-03-09T08:55:26.879 INFO:tasks.workunit.client.1.smithi134.stdout: create 210/32 file after the snapshot 2024-03-09T08:55:26.879 INFO:tasks.workunit.client.1.smithi134.stdout: create 210/33 file after the snapshot 2024-03-09T08:55:26.879 INFO:tasks.workunit.client.1.smithi134.stdout: create 210/34 file after the snapshot 2024-03-09T08:55:26.879 INFO:tasks.workunit.client.1.smithi134.stdout: create 210/35 file after the snapshot 2024-03-09T08:55:26.880 INFO:tasks.workunit.client.1.smithi134.stdout: create 210/36 file after the snapshot 2024-03-09T08:55:26.880 INFO:tasks.workunit.client.1.smithi134.stdout: create 210/37 file after the snapshot 2024-03-09T08:55:26.880 INFO:tasks.workunit.client.1.smithi134.stdout: create 210/38 file after the snapshot 2024-03-09T08:55:26.880 INFO:tasks.workunit.client.1.smithi134.stdout: create 210/39 file after the snapshot 2024-03-09T08:55:26.881 INFO:tasks.workunit.client.1.smithi134.stdout: create 211/.snap/snap-subdir-test 2024-03-09T08:55:27.208 INFO:tasks.workunit.client.0.smithi067.stdout: create 161/30 file after the snapshot 2024-03-09T08:55:27.211 INFO:tasks.workunit.client.0.smithi067.stdout: create 161/31 file after the snapshot 2024-03-09T08:55:27.212 INFO:tasks.workunit.client.0.smithi067.stdout: create 161/32 file after the snapshot 2024-03-09T08:55:27.212 INFO:tasks.workunit.client.0.smithi067.stdout: create 161/33 file after the snapshot 2024-03-09T08:55:27.212 INFO:tasks.workunit.client.0.smithi067.stdout: create 161/34 file after the snapshot 2024-03-09T08:55:27.213 INFO:tasks.workunit.client.0.smithi067.stdout: create 161/35 file after the snapshot 2024-03-09T08:55:27.213 INFO:tasks.workunit.client.0.smithi067.stdout: create 161/36 file after the snapshot 2024-03-09T08:55:27.213 INFO:tasks.workunit.client.0.smithi067.stdout: create 161/37 file after the snapshot 2024-03-09T08:55:27.214 INFO:tasks.workunit.client.0.smithi067.stdout: create 161/38 file after the snapshot 2024-03-09T08:55:27.214 INFO:tasks.workunit.client.0.smithi067.stdout: create 161/39 file after the snapshot 2024-03-09T08:55:27.214 INFO:tasks.workunit.client.0.smithi067.stdout: create 162/.snap/snap-subdir-test 2024-03-09T08:55:27.301 INFO:tasks.workunit.client.0.smithi067.stdout: create 162/30 file after the snapshot 2024-03-09T08:55:27.304 INFO:tasks.workunit.client.0.smithi067.stdout: create 162/31 file after the snapshot 2024-03-09T08:55:27.305 INFO:tasks.workunit.client.0.smithi067.stdout: create 162/32 file after the snapshot 2024-03-09T08:55:27.306 INFO:tasks.workunit.client.0.smithi067.stdout: create 162/33 file after the snapshot 2024-03-09T08:55:27.306 INFO:tasks.workunit.client.0.smithi067.stdout: create 162/34 file after the snapshot 2024-03-09T08:55:27.306 INFO:tasks.workunit.client.0.smithi067.stdout: create 162/35 file after the snapshot 2024-03-09T08:55:27.306 INFO:tasks.workunit.client.0.smithi067.stdout: create 162/36 file after the snapshot 2024-03-09T08:55:27.307 INFO:tasks.workunit.client.0.smithi067.stdout: create 162/37 file after the snapshot 2024-03-09T08:55:27.307 INFO:tasks.workunit.client.0.smithi067.stdout: create 162/38 file after the snapshot 2024-03-09T08:55:27.307 INFO:tasks.workunit.client.0.smithi067.stdout: create 162/39 file after the snapshot 2024-03-09T08:55:27.309 INFO:tasks.workunit.client.0.smithi067.stdout: create 163/.snap/snap-subdir-test 2024-03-09T08:55:27.404 INFO:tasks.workunit.client.0.smithi067.stdout: create 163/30 file after the snapshot 2024-03-09T08:55:27.407 INFO:tasks.workunit.client.0.smithi067.stdout: create 163/31 file after the snapshot 2024-03-09T08:55:27.408 INFO:tasks.workunit.client.0.smithi067.stdout: create 163/32 file after the snapshot 2024-03-09T08:55:27.408 INFO:tasks.workunit.client.0.smithi067.stdout: create 163/33 file after the snapshot 2024-03-09T08:55:27.408 INFO:tasks.workunit.client.0.smithi067.stdout: create 163/34 file after the snapshot 2024-03-09T08:55:27.409 INFO:tasks.workunit.client.0.smithi067.stdout: create 163/35 file after the snapshot 2024-03-09T08:55:27.409 INFO:tasks.workunit.client.0.smithi067.stdout: create 163/36 file after the snapshot 2024-03-09T08:55:27.409 INFO:tasks.workunit.client.0.smithi067.stdout: create 163/37 file after the snapshot 2024-03-09T08:55:27.409 INFO:tasks.workunit.client.0.smithi067.stdout: create 163/38 file after the snapshot 2024-03-09T08:55:27.410 INFO:tasks.workunit.client.0.smithi067.stdout: create 163/39 file after the snapshot 2024-03-09T08:55:27.410 INFO:tasks.workunit.client.0.smithi067.stdout: create 164/.snap/snap-subdir-test 2024-03-09T08:55:27.499 INFO:tasks.workunit.client.0.smithi067.stdout: create 164/30 file after the snapshot 2024-03-09T08:55:27.502 INFO:tasks.workunit.client.0.smithi067.stdout: create 164/31 file after the snapshot 2024-03-09T08:55:27.502 INFO:tasks.workunit.client.0.smithi067.stdout: create 164/32 file after the snapshot 2024-03-09T08:55:27.503 INFO:tasks.workunit.client.0.smithi067.stdout: create 164/33 file after the snapshot 2024-03-09T08:55:27.503 INFO:tasks.workunit.client.0.smithi067.stdout: create 164/34 file after the snapshot 2024-03-09T08:55:27.503 INFO:tasks.workunit.client.0.smithi067.stdout: create 164/35 file after the snapshot 2024-03-09T08:55:27.504 INFO:tasks.workunit.client.0.smithi067.stdout: create 164/36 file after the snapshot 2024-03-09T08:55:27.504 INFO:tasks.workunit.client.0.smithi067.stdout: create 164/37 file after the snapshot 2024-03-09T08:55:27.504 INFO:tasks.workunit.client.0.smithi067.stdout: create 164/38 file after the snapshot 2024-03-09T08:55:27.505 INFO:tasks.workunit.client.0.smithi067.stdout: create 164/39 file after the snapshot 2024-03-09T08:55:27.505 INFO:tasks.workunit.client.0.smithi067.stdout: create 165/.snap/snap-subdir-test 2024-03-09T08:55:27.595 INFO:tasks.workunit.client.0.smithi067.stdout: create 165/30 file after the snapshot 2024-03-09T08:55:27.599 INFO:tasks.workunit.client.0.smithi067.stdout: create 165/31 file after the snapshot 2024-03-09T08:55:27.599 INFO:tasks.workunit.client.0.smithi067.stdout: create 165/32 file after the snapshot 2024-03-09T08:55:27.600 INFO:tasks.workunit.client.0.smithi067.stdout: create 165/33 file after the snapshot 2024-03-09T08:55:27.600 INFO:tasks.workunit.client.0.smithi067.stdout: create 165/34 file after the snapshot 2024-03-09T08:55:27.600 INFO:tasks.workunit.client.0.smithi067.stdout: create 165/35 file after the snapshot 2024-03-09T08:55:27.601 INFO:tasks.workunit.client.0.smithi067.stdout: create 165/36 file after the snapshot 2024-03-09T08:55:27.601 INFO:tasks.workunit.client.0.smithi067.stdout: create 165/37 file after the snapshot 2024-03-09T08:55:27.601 INFO:tasks.workunit.client.0.smithi067.stdout: create 165/38 file after the snapshot 2024-03-09T08:55:27.602 INFO:tasks.workunit.client.0.smithi067.stdout: create 165/39 file after the snapshot 2024-03-09T08:55:27.602 INFO:tasks.workunit.client.0.smithi067.stdout: create 166/.snap/snap-subdir-test 2024-03-09T08:55:27.695 INFO:tasks.workunit.client.0.smithi067.stdout: create 166/30 file after the snapshot 2024-03-09T08:55:27.698 INFO:tasks.workunit.client.0.smithi067.stdout: create 166/31 file after the snapshot 2024-03-09T08:55:27.699 INFO:tasks.workunit.client.0.smithi067.stdout: create 166/32 file after the snapshot 2024-03-09T08:55:27.700 INFO:tasks.workunit.client.0.smithi067.stdout: create 166/33 file after the snapshot 2024-03-09T08:55:27.700 INFO:tasks.workunit.client.0.smithi067.stdout: create 166/34 file after the snapshot 2024-03-09T08:55:27.700 INFO:tasks.workunit.client.0.smithi067.stdout: create 166/35 file after the snapshot 2024-03-09T08:55:27.701 INFO:tasks.workunit.client.0.smithi067.stdout: create 166/36 file after the snapshot 2024-03-09T08:55:27.701 INFO:tasks.workunit.client.0.smithi067.stdout: create 166/37 file after the snapshot 2024-03-09T08:55:27.702 INFO:tasks.workunit.client.0.smithi067.stdout: create 166/38 file after the snapshot 2024-03-09T08:55:27.702 INFO:tasks.workunit.client.0.smithi067.stdout: create 166/39 file after the snapshot 2024-03-09T08:55:27.703 INFO:tasks.workunit.client.0.smithi067.stdout: create 167/.snap/snap-subdir-test 2024-03-09T08:55:27.795 INFO:tasks.workunit.client.0.smithi067.stdout: create 167/30 file after the snapshot 2024-03-09T08:55:27.798 INFO:tasks.workunit.client.0.smithi067.stdout: create 167/31 file after the snapshot 2024-03-09T08:55:27.798 INFO:tasks.workunit.client.0.smithi067.stdout: create 167/32 file after the snapshot 2024-03-09T08:55:27.799 INFO:tasks.workunit.client.0.smithi067.stdout: create 167/33 file after the snapshot 2024-03-09T08:55:27.799 INFO:tasks.workunit.client.0.smithi067.stdout: create 167/34 file after the snapshot 2024-03-09T08:55:27.799 INFO:tasks.workunit.client.0.smithi067.stdout: create 167/35 file after the snapshot 2024-03-09T08:55:27.800 INFO:tasks.workunit.client.0.smithi067.stdout: create 167/36 file after the snapshot 2024-03-09T08:55:27.800 INFO:tasks.workunit.client.0.smithi067.stdout: create 167/37 file after the snapshot 2024-03-09T08:55:27.800 INFO:tasks.workunit.client.0.smithi067.stdout: create 167/38 file after the snapshot 2024-03-09T08:55:27.801 INFO:tasks.workunit.client.0.smithi067.stdout: create 167/39 file after the snapshot 2024-03-09T08:55:27.802 INFO:tasks.workunit.client.0.smithi067.stdout: create 168/.snap/snap-subdir-test 2024-03-09T08:55:27.810 INFO:tasks.workunit.client.1.smithi134.stdout: create 211/30 file after the snapshot 2024-03-09T08:55:27.813 INFO:tasks.workunit.client.1.smithi134.stdout: create 211/31 file after the snapshot 2024-03-09T08:55:27.813 INFO:tasks.workunit.client.1.smithi134.stdout: create 211/32 file after the snapshot 2024-03-09T08:55:27.814 INFO:tasks.workunit.client.1.smithi134.stdout: create 211/33 file after the snapshot 2024-03-09T08:55:27.814 INFO:tasks.workunit.client.1.smithi134.stdout: create 211/34 file after the snapshot 2024-03-09T08:55:27.814 INFO:tasks.workunit.client.1.smithi134.stdout: create 211/35 file after the snapshot 2024-03-09T08:55:27.815 INFO:tasks.workunit.client.1.smithi134.stdout: create 211/36 file after the snapshot 2024-03-09T08:55:27.815 INFO:tasks.workunit.client.1.smithi134.stdout: create 211/37 file after the snapshot 2024-03-09T08:55:27.815 INFO:tasks.workunit.client.1.smithi134.stdout: create 211/38 file after the snapshot 2024-03-09T08:55:27.815 INFO:tasks.workunit.client.1.smithi134.stdout: create 211/39 file after the snapshot 2024-03-09T08:55:27.816 INFO:tasks.workunit.client.1.smithi134.stdout: create 212/.snap/snap-subdir-test 2024-03-09T08:55:27.890 INFO:tasks.workunit.client.0.smithi067.stdout: create 168/30 file after the snapshot 2024-03-09T08:55:27.893 INFO:tasks.workunit.client.0.smithi067.stdout: create 168/31 file after the snapshot 2024-03-09T08:55:27.893 INFO:tasks.workunit.client.0.smithi067.stdout: create 168/32 file after the snapshot 2024-03-09T08:55:27.894 INFO:tasks.workunit.client.0.smithi067.stdout: create 168/33 file after the snapshot 2024-03-09T08:55:27.894 INFO:tasks.workunit.client.0.smithi067.stdout: create 168/34 file after the snapshot 2024-03-09T08:55:27.894 INFO:tasks.workunit.client.0.smithi067.stdout: create 168/35 file after the snapshot 2024-03-09T08:55:27.895 INFO:tasks.workunit.client.0.smithi067.stdout: create 168/36 file after the snapshot 2024-03-09T08:55:27.895 INFO:tasks.workunit.client.0.smithi067.stdout: create 168/37 file after the snapshot 2024-03-09T08:55:27.895 INFO:tasks.workunit.client.0.smithi067.stdout: create 168/38 file after the snapshot 2024-03-09T08:55:27.895 INFO:tasks.workunit.client.0.smithi067.stdout: create 168/39 file after the snapshot 2024-03-09T08:55:27.896 INFO:tasks.workunit.client.0.smithi067.stdout: create 169/.snap/snap-subdir-test 2024-03-09T08:55:27.915 INFO:tasks.workunit.client.1.smithi134.stdout: create 212/30 file after the snapshot 2024-03-09T08:55:27.918 INFO:tasks.workunit.client.1.smithi134.stdout: create 212/31 file after the snapshot 2024-03-09T08:55:27.918 INFO:tasks.workunit.client.1.smithi134.stdout: create 212/32 file after the snapshot 2024-03-09T08:55:27.918 INFO:tasks.workunit.client.1.smithi134.stdout: create 212/33 file after the snapshot 2024-03-09T08:55:27.918 INFO:tasks.workunit.client.1.smithi134.stdout: create 212/34 file after the snapshot 2024-03-09T08:55:27.919 INFO:tasks.workunit.client.1.smithi134.stdout: create 212/35 file after the snapshot 2024-03-09T08:55:27.919 INFO:tasks.workunit.client.1.smithi134.stdout: create 212/36 file after the snapshot 2024-03-09T08:55:27.919 INFO:tasks.workunit.client.1.smithi134.stdout: create 212/37 file after the snapshot 2024-03-09T08:55:27.919 INFO:tasks.workunit.client.1.smithi134.stdout: create 212/38 file after the snapshot 2024-03-09T08:55:27.919 INFO:tasks.workunit.client.1.smithi134.stdout: create 212/39 file after the snapshot 2024-03-09T08:55:27.920 INFO:tasks.workunit.client.1.smithi134.stdout: create 213/.snap/snap-subdir-test 2024-03-09T08:55:27.984 INFO:tasks.workunit.client.0.smithi067.stdout: create 169/30 file after the snapshot 2024-03-09T08:55:27.986 INFO:tasks.workunit.client.0.smithi067.stdout: create 169/31 file after the snapshot 2024-03-09T08:55:27.987 INFO:tasks.workunit.client.0.smithi067.stdout: create 169/32 file after the snapshot 2024-03-09T08:55:27.987 INFO:tasks.workunit.client.0.smithi067.stdout: create 169/33 file after the snapshot 2024-03-09T08:55:27.987 INFO:tasks.workunit.client.0.smithi067.stdout: create 169/34 file after the snapshot 2024-03-09T08:55:27.988 INFO:tasks.workunit.client.0.smithi067.stdout: create 169/35 file after the snapshot 2024-03-09T08:55:27.988 INFO:tasks.workunit.client.0.smithi067.stdout: create 169/36 file after the snapshot 2024-03-09T08:55:27.988 INFO:tasks.workunit.client.0.smithi067.stdout: create 169/37 file after the snapshot 2024-03-09T08:55:27.988 INFO:tasks.workunit.client.0.smithi067.stdout: create 169/38 file after the snapshot 2024-03-09T08:55:27.989 INFO:tasks.workunit.client.0.smithi067.stdout: create 169/39 file after the snapshot 2024-03-09T08:55:27.989 INFO:tasks.workunit.client.0.smithi067.stdout: create 170/.snap/snap-subdir-test 2024-03-09T08:55:28.079 INFO:tasks.workunit.client.0.smithi067.stdout: create 170/30 file after the snapshot 2024-03-09T08:55:28.081 INFO:tasks.workunit.client.0.smithi067.stdout: create 170/31 file after the snapshot 2024-03-09T08:55:28.082 INFO:tasks.workunit.client.0.smithi067.stdout: create 170/32 file after the snapshot 2024-03-09T08:55:28.082 INFO:tasks.workunit.client.0.smithi067.stdout: create 170/33 file after the snapshot 2024-03-09T08:55:28.083 INFO:tasks.workunit.client.0.smithi067.stdout: create 170/34 file after the snapshot 2024-03-09T08:55:28.083 INFO:tasks.workunit.client.0.smithi067.stdout: create 170/35 file after the snapshot 2024-03-09T08:55:28.083 INFO:tasks.workunit.client.0.smithi067.stdout: create 170/36 file after the snapshot 2024-03-09T08:55:28.083 INFO:tasks.workunit.client.0.smithi067.stdout: create 170/37 file after the snapshot 2024-03-09T08:55:28.084 INFO:tasks.workunit.client.0.smithi067.stdout: create 170/38 file after the snapshot 2024-03-09T08:55:28.084 INFO:tasks.workunit.client.0.smithi067.stdout: create 170/39 file after the snapshot 2024-03-09T08:55:28.084 INFO:tasks.workunit.client.0.smithi067.stdout: create 171/.snap/snap-subdir-test 2024-03-09T08:55:28.163 INFO:tasks.workunit.client.0.smithi067.stdout: create 171/30 file after the snapshot 2024-03-09T08:55:28.166 INFO:tasks.workunit.client.0.smithi067.stdout: create 171/31 file after the snapshot 2024-03-09T08:55:28.166 INFO:tasks.workunit.client.0.smithi067.stdout: create 171/32 file after the snapshot 2024-03-09T08:55:28.167 INFO:tasks.workunit.client.0.smithi067.stdout: create 171/33 file after the snapshot 2024-03-09T08:55:28.167 INFO:tasks.workunit.client.0.smithi067.stdout: create 171/34 file after the snapshot 2024-03-09T08:55:28.167 INFO:tasks.workunit.client.0.smithi067.stdout: create 171/35 file after the snapshot 2024-03-09T08:55:28.167 INFO:tasks.workunit.client.0.smithi067.stdout: create 171/36 file after the snapshot 2024-03-09T08:55:28.167 INFO:tasks.workunit.client.0.smithi067.stdout: create 171/37 file after the snapshot 2024-03-09T08:55:28.167 INFO:tasks.workunit.client.0.smithi067.stdout: create 171/38 file after the snapshot 2024-03-09T08:55:28.168 INFO:tasks.workunit.client.0.smithi067.stdout: create 171/39 file after the snapshot 2024-03-09T08:55:28.168 INFO:tasks.workunit.client.0.smithi067.stdout: create 172/.snap/snap-subdir-test 2024-03-09T08:55:28.199 INFO:tasks.workunit.client.1.smithi134.stdout: create 213/30 file after the snapshot 2024-03-09T08:55:28.201 INFO:tasks.workunit.client.1.smithi134.stdout: create 213/31 file after the snapshot 2024-03-09T08:55:28.202 INFO:tasks.workunit.client.1.smithi134.stdout: create 213/32 file after the snapshot 2024-03-09T08:55:28.202 INFO:tasks.workunit.client.1.smithi134.stdout: create 213/33 file after the snapshot 2024-03-09T08:55:28.202 INFO:tasks.workunit.client.1.smithi134.stdout: create 213/34 file after the snapshot 2024-03-09T08:55:28.203 INFO:tasks.workunit.client.1.smithi134.stdout: create 213/35 file after the snapshot 2024-03-09T08:55:28.203 INFO:tasks.workunit.client.1.smithi134.stdout: create 213/36 file after the snapshot 2024-03-09T08:55:28.203 INFO:tasks.workunit.client.1.smithi134.stdout: create 213/37 file after the snapshot 2024-03-09T08:55:28.204 INFO:tasks.workunit.client.1.smithi134.stdout: create 213/38 file after the snapshot 2024-03-09T08:55:28.204 INFO:tasks.workunit.client.1.smithi134.stdout: create 213/39 file after the snapshot 2024-03-09T08:55:28.204 INFO:tasks.workunit.client.1.smithi134.stdout: create 214/.snap/snap-subdir-test 2024-03-09T08:55:28.236 INFO:tasks.workunit.client.0.smithi067.stdout: create 172/30 file after the snapshot 2024-03-09T08:55:28.241 INFO:tasks.workunit.client.0.smithi067.stdout: create 172/31 file after the snapshot 2024-03-09T08:55:28.241 INFO:tasks.workunit.client.0.smithi067.stdout: create 172/32 file after the snapshot 2024-03-09T08:55:28.241 INFO:tasks.workunit.client.0.smithi067.stdout: create 172/33 file after the snapshot 2024-03-09T08:55:28.241 INFO:tasks.workunit.client.0.smithi067.stdout: create 172/34 file after the snapshot 2024-03-09T08:55:28.241 INFO:tasks.workunit.client.0.smithi067.stdout: create 172/35 file after the snapshot 2024-03-09T08:55:28.241 INFO:tasks.workunit.client.0.smithi067.stdout: create 172/36 file after the snapshot 2024-03-09T08:55:28.241 INFO:tasks.workunit.client.0.smithi067.stdout: create 172/37 file after the snapshot 2024-03-09T08:55:28.242 INFO:tasks.workunit.client.0.smithi067.stdout: create 172/38 file after the snapshot 2024-03-09T08:55:28.242 INFO:tasks.workunit.client.0.smithi067.stdout: create 172/39 file after the snapshot 2024-03-09T08:55:28.242 INFO:tasks.workunit.client.0.smithi067.stdout: create 173/.snap/snap-subdir-test 2024-03-09T08:55:28.281 INFO:tasks.workunit.client.1.smithi134.stdout: create 214/30 file after the snapshot 2024-03-09T08:55:28.284 INFO:tasks.workunit.client.1.smithi134.stdout: create 214/31 file after the snapshot 2024-03-09T08:55:28.284 INFO:tasks.workunit.client.1.smithi134.stdout: create 214/32 file after the snapshot 2024-03-09T08:55:28.285 INFO:tasks.workunit.client.1.smithi134.stdout: create 214/33 file after the snapshot 2024-03-09T08:55:28.285 INFO:tasks.workunit.client.1.smithi134.stdout: create 214/34 file after the snapshot 2024-03-09T08:55:28.286 INFO:tasks.workunit.client.1.smithi134.stdout: create 214/35 file after the snapshot 2024-03-09T08:55:28.286 INFO:tasks.workunit.client.1.smithi134.stdout: create 214/36 file after the snapshot 2024-03-09T08:55:28.286 INFO:tasks.workunit.client.1.smithi134.stdout: create 214/37 file after the snapshot 2024-03-09T08:55:28.287 INFO:tasks.workunit.client.1.smithi134.stdout: create 214/38 file after the snapshot 2024-03-09T08:55:28.287 INFO:tasks.workunit.client.1.smithi134.stdout: create 214/39 file after the snapshot 2024-03-09T08:55:28.287 INFO:tasks.workunit.client.1.smithi134.stdout: create 215/.snap/snap-subdir-test 2024-03-09T08:55:28.571 INFO:tasks.workunit.client.0.smithi067.stdout: create 173/30 file after the snapshot 2024-03-09T08:55:28.574 INFO:tasks.workunit.client.0.smithi067.stdout: create 173/31 file after the snapshot 2024-03-09T08:55:28.575 INFO:tasks.workunit.client.0.smithi067.stdout: create 173/32 file after the snapshot 2024-03-09T08:55:28.575 INFO:tasks.workunit.client.0.smithi067.stdout: create 173/33 file after the snapshot 2024-03-09T08:55:28.575 INFO:tasks.workunit.client.0.smithi067.stdout: create 173/34 file after the snapshot 2024-03-09T08:55:28.576 INFO:tasks.workunit.client.0.smithi067.stdout: create 173/35 file after the snapshot 2024-03-09T08:55:28.576 INFO:tasks.workunit.client.0.smithi067.stdout: create 173/36 file after the snapshot 2024-03-09T08:55:28.576 INFO:tasks.workunit.client.0.smithi067.stdout: create 173/37 file after the snapshot 2024-03-09T08:55:28.576 INFO:tasks.workunit.client.0.smithi067.stdout: create 173/38 file after the snapshot 2024-03-09T08:55:28.577 INFO:tasks.workunit.client.0.smithi067.stdout: create 173/39 file after the snapshot 2024-03-09T08:55:28.577 INFO:tasks.workunit.client.0.smithi067.stdout: create 174/.snap/snap-subdir-test 2024-03-09T08:55:28.658 INFO:tasks.workunit.client.0.smithi067.stdout: create 174/30 file after the snapshot 2024-03-09T08:55:28.660 INFO:tasks.workunit.client.0.smithi067.stdout: create 174/31 file after the snapshot 2024-03-09T08:55:28.661 INFO:tasks.workunit.client.0.smithi067.stdout: create 174/32 file after the snapshot 2024-03-09T08:55:28.661 INFO:tasks.workunit.client.0.smithi067.stdout: create 174/33 file after the snapshot 2024-03-09T08:55:28.661 INFO:tasks.workunit.client.0.smithi067.stdout: create 174/34 file after the snapshot 2024-03-09T08:55:28.662 INFO:tasks.workunit.client.0.smithi067.stdout: create 174/35 file after the snapshot 2024-03-09T08:55:28.662 INFO:tasks.workunit.client.0.smithi067.stdout: create 174/36 file after the snapshot 2024-03-09T08:55:28.662 INFO:tasks.workunit.client.0.smithi067.stdout: create 174/37 file after the snapshot 2024-03-09T08:55:28.663 INFO:tasks.workunit.client.0.smithi067.stdout: create 174/38 file after the snapshot 2024-03-09T08:55:28.663 INFO:tasks.workunit.client.0.smithi067.stdout: create 174/39 file after the snapshot 2024-03-09T08:55:28.663 INFO:tasks.workunit.client.0.smithi067.stdout: create 175/.snap/snap-subdir-test 2024-03-09T08:55:28.741 INFO:tasks.workunit.client.0.smithi067.stdout: create 175/30 file after the snapshot 2024-03-09T08:55:28.744 INFO:tasks.workunit.client.0.smithi067.stdout: create 175/31 file after the snapshot 2024-03-09T08:55:28.744 INFO:tasks.workunit.client.0.smithi067.stdout: create 175/32 file after the snapshot 2024-03-09T08:55:28.745 INFO:tasks.workunit.client.0.smithi067.stdout: create 175/33 file after the snapshot 2024-03-09T08:55:28.745 INFO:tasks.workunit.client.0.smithi067.stdout: create 175/34 file after the snapshot 2024-03-09T08:55:28.745 INFO:tasks.workunit.client.0.smithi067.stdout: create 175/35 file after the snapshot 2024-03-09T08:55:28.746 INFO:tasks.workunit.client.0.smithi067.stdout: create 175/36 file after the snapshot 2024-03-09T08:55:28.746 INFO:tasks.workunit.client.0.smithi067.stdout: create 175/37 file after the snapshot 2024-03-09T08:55:28.746 INFO:tasks.workunit.client.0.smithi067.stdout: create 175/38 file after the snapshot 2024-03-09T08:55:28.746 INFO:tasks.workunit.client.0.smithi067.stdout: create 175/39 file after the snapshot 2024-03-09T08:55:28.747 INFO:tasks.workunit.client.0.smithi067.stdout: create 176/.snap/snap-subdir-test 2024-03-09T08:55:28.850 INFO:tasks.workunit.client.0.smithi067.stdout: create 176/30 file after the snapshot 2024-03-09T08:55:28.853 INFO:tasks.workunit.client.0.smithi067.stdout: create 176/31 file after the snapshot 2024-03-09T08:55:28.853 INFO:tasks.workunit.client.0.smithi067.stdout: create 176/32 file after the snapshot 2024-03-09T08:55:28.853 INFO:tasks.workunit.client.0.smithi067.stdout: create 176/33 file after the snapshot 2024-03-09T08:55:28.854 INFO:tasks.workunit.client.0.smithi067.stdout: create 176/34 file after the snapshot 2024-03-09T08:55:28.854 INFO:tasks.workunit.client.0.smithi067.stdout: create 176/35 file after the snapshot 2024-03-09T08:55:28.854 INFO:tasks.workunit.client.0.smithi067.stdout: create 176/36 file after the snapshot 2024-03-09T08:55:28.855 INFO:tasks.workunit.client.0.smithi067.stdout: create 176/37 file after the snapshot 2024-03-09T08:55:28.855 INFO:tasks.workunit.client.0.smithi067.stdout: create 176/38 file after the snapshot 2024-03-09T08:55:28.856 INFO:tasks.workunit.client.0.smithi067.stdout: create 176/39 file after the snapshot 2024-03-09T08:55:28.856 INFO:tasks.workunit.client.0.smithi067.stdout: create 177/.snap/snap-subdir-test 2024-03-09T08:55:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: create 177/30 file after the snapshot 2024-03-09T08:55:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: create 177/31 file after the snapshot 2024-03-09T08:55:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: create 177/32 file after the snapshot 2024-03-09T08:55:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: create 177/33 file after the snapshot 2024-03-09T08:55:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: create 177/34 file after the snapshot 2024-03-09T08:55:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: create 177/35 file after the snapshot 2024-03-09T08:55:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: create 177/36 file after the snapshot 2024-03-09T08:55:28.965 INFO:tasks.workunit.client.0.smithi067.stdout: create 177/37 file after the snapshot 2024-03-09T08:55:28.965 INFO:tasks.workunit.client.0.smithi067.stdout: create 177/38 file after the snapshot 2024-03-09T08:55:28.965 INFO:tasks.workunit.client.0.smithi067.stdout: create 177/39 file after the snapshot 2024-03-09T08:55:28.965 INFO:tasks.workunit.client.0.smithi067.stdout: create 178/.snap/snap-subdir-test 2024-03-09T08:55:29.050 INFO:tasks.workunit.client.0.smithi067.stdout: create 178/30 file after the snapshot 2024-03-09T08:55:29.052 INFO:tasks.workunit.client.0.smithi067.stdout: create 178/31 file after the snapshot 2024-03-09T08:55:29.053 INFO:tasks.workunit.client.0.smithi067.stdout: create 178/32 file after the snapshot 2024-03-09T08:55:29.053 INFO:tasks.workunit.client.0.smithi067.stdout: create 178/33 file after the snapshot 2024-03-09T08:55:29.053 INFO:tasks.workunit.client.0.smithi067.stdout: create 178/34 file after the snapshot 2024-03-09T08:55:29.054 INFO:tasks.workunit.client.0.smithi067.stdout: create 178/35 file after the snapshot 2024-03-09T08:55:29.054 INFO:tasks.workunit.client.0.smithi067.stdout: create 178/36 file after the snapshot 2024-03-09T08:55:29.054 INFO:tasks.workunit.client.1.smithi134.stdout: create 215/30 file after the snapshot 2024-03-09T08:55:29.055 INFO:tasks.workunit.client.0.smithi067.stdout: create 178/37 file after the snapshot 2024-03-09T08:55:29.055 INFO:tasks.workunit.client.0.smithi067.stdout: create 178/38 file after the snapshot 2024-03-09T08:55:29.055 INFO:tasks.workunit.client.0.smithi067.stdout: create 178/39 file after the snapshot 2024-03-09T08:55:29.055 INFO:tasks.workunit.client.0.smithi067.stdout: create 179/.snap/snap-subdir-test 2024-03-09T08:55:29.056 INFO:tasks.workunit.client.1.smithi134.stdout: create 215/31 file after the snapshot 2024-03-09T08:55:29.057 INFO:tasks.workunit.client.1.smithi134.stdout: create 215/32 file after the snapshot 2024-03-09T08:55:29.057 INFO:tasks.workunit.client.1.smithi134.stdout: create 215/33 file after the snapshot 2024-03-09T08:55:29.058 INFO:tasks.workunit.client.1.smithi134.stdout: create 215/34 file after the snapshot 2024-03-09T08:55:29.058 INFO:tasks.workunit.client.1.smithi134.stdout: create 215/35 file after the snapshot 2024-03-09T08:55:29.058 INFO:tasks.workunit.client.1.smithi134.stdout: create 215/36 file after the snapshot 2024-03-09T08:55:29.059 INFO:tasks.workunit.client.1.smithi134.stdout: create 215/37 file after the snapshot 2024-03-09T08:55:29.059 INFO:tasks.workunit.client.1.smithi134.stdout: create 215/38 file after the snapshot 2024-03-09T08:55:29.060 INFO:tasks.workunit.client.1.smithi134.stdout: create 215/39 file after the snapshot 2024-03-09T08:55:29.060 INFO:tasks.workunit.client.1.smithi134.stdout: create 216/.snap/snap-subdir-test 2024-03-09T08:55:29.130 INFO:tasks.workunit.client.0.smithi067.stdout: create 179/30 file after the snapshot 2024-03-09T08:55:29.132 INFO:tasks.workunit.client.0.smithi067.stdout: create 179/31 file after the snapshot 2024-03-09T08:55:29.132 INFO:tasks.workunit.client.0.smithi067.stdout: create 179/32 file after the snapshot 2024-03-09T08:55:29.133 INFO:tasks.workunit.client.0.smithi067.stdout: create 179/33 file after the snapshot 2024-03-09T08:55:29.133 INFO:tasks.workunit.client.0.smithi067.stdout: create 179/34 file after the snapshot 2024-03-09T08:55:29.133 INFO:tasks.workunit.client.0.smithi067.stdout: create 179/35 file after the snapshot 2024-03-09T08:55:29.134 INFO:tasks.workunit.client.0.smithi067.stdout: create 179/36 file after the snapshot 2024-03-09T08:55:29.134 INFO:tasks.workunit.client.0.smithi067.stdout: create 179/37 file after the snapshot 2024-03-09T08:55:29.134 INFO:tasks.workunit.client.0.smithi067.stdout: create 179/38 file after the snapshot 2024-03-09T08:55:29.135 INFO:tasks.workunit.client.0.smithi067.stdout: create 179/39 file after the snapshot 2024-03-09T08:55:29.135 INFO:tasks.workunit.client.0.smithi067.stdout: create 180/.snap/snap-subdir-test 2024-03-09T08:55:29.462 INFO:tasks.workunit.client.0.smithi067.stdout: create 180/30 file after the snapshot 2024-03-09T08:55:29.465 INFO:tasks.workunit.client.0.smithi067.stdout: create 180/31 file after the snapshot 2024-03-09T08:55:29.465 INFO:tasks.workunit.client.0.smithi067.stdout: create 180/32 file after the snapshot 2024-03-09T08:55:29.465 INFO:tasks.workunit.client.0.smithi067.stdout: create 180/33 file after the snapshot 2024-03-09T08:55:29.466 INFO:tasks.workunit.client.0.smithi067.stdout: create 180/34 file after the snapshot 2024-03-09T08:55:29.466 INFO:tasks.workunit.client.0.smithi067.stdout: create 180/35 file after the snapshot 2024-03-09T08:55:29.467 INFO:tasks.workunit.client.0.smithi067.stdout: create 180/36 file after the snapshot 2024-03-09T08:55:29.467 INFO:tasks.workunit.client.0.smithi067.stdout: create 180/37 file after the snapshot 2024-03-09T08:55:29.467 INFO:tasks.workunit.client.0.smithi067.stdout: create 180/38 file after the snapshot 2024-03-09T08:55:29.468 INFO:tasks.workunit.client.0.smithi067.stdout: create 180/39 file after the snapshot 2024-03-09T08:55:29.469 INFO:tasks.workunit.client.0.smithi067.stdout: create 181/.snap/snap-subdir-test 2024-03-09T08:55:29.537 INFO:tasks.workunit.client.0.smithi067.stdout: create 181/30 file after the snapshot 2024-03-09T08:55:29.540 INFO:tasks.workunit.client.0.smithi067.stdout: create 181/31 file after the snapshot 2024-03-09T08:55:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: create 181/32 file after the snapshot 2024-03-09T08:55:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: create 181/33 file after the snapshot 2024-03-09T08:55:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: create 181/34 file after the snapshot 2024-03-09T08:55:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: create 181/35 file after the snapshot 2024-03-09T08:55:29.542 INFO:tasks.workunit.client.0.smithi067.stdout: create 181/36 file after the snapshot 2024-03-09T08:55:29.542 INFO:tasks.workunit.client.0.smithi067.stdout: create 181/37 file after the snapshot 2024-03-09T08:55:29.543 INFO:tasks.workunit.client.0.smithi067.stdout: create 181/38 file after the snapshot 2024-03-09T08:55:29.543 INFO:tasks.workunit.client.0.smithi067.stdout: create 181/39 file after the snapshot 2024-03-09T08:55:29.543 INFO:tasks.workunit.client.0.smithi067.stdout: create 182/.snap/snap-subdir-test 2024-03-09T08:55:29.612 INFO:tasks.workunit.client.0.smithi067.stdout: create 182/30 file after the snapshot 2024-03-09T08:55:29.620 INFO:tasks.workunit.client.0.smithi067.stdout: create 182/31 file after the snapshot 2024-03-09T08:55:29.620 INFO:tasks.workunit.client.0.smithi067.stdout: create 182/32 file after the snapshot 2024-03-09T08:55:29.620 INFO:tasks.workunit.client.0.smithi067.stdout: create 182/33 file after the snapshot 2024-03-09T08:55:29.620 INFO:tasks.workunit.client.0.smithi067.stdout: create 182/34 file after the snapshot 2024-03-09T08:55:29.620 INFO:tasks.workunit.client.0.smithi067.stdout: create 182/35 file after the snapshot 2024-03-09T08:55:29.620 INFO:tasks.workunit.client.0.smithi067.stdout: create 182/36 file after the snapshot 2024-03-09T08:55:29.620 INFO:tasks.workunit.client.0.smithi067.stdout: create 182/37 file after the snapshot 2024-03-09T08:55:29.621 INFO:tasks.workunit.client.0.smithi067.stdout: create 182/38 file after the snapshot 2024-03-09T08:55:29.621 INFO:tasks.workunit.client.0.smithi067.stdout: create 182/39 file after the snapshot 2024-03-09T08:55:29.621 INFO:tasks.workunit.client.0.smithi067.stdout: create 183/.snap/snap-subdir-test 2024-03-09T08:55:29.691 INFO:tasks.workunit.client.0.smithi067.stdout: create 183/30 file after the snapshot 2024-03-09T08:55:29.694 INFO:tasks.workunit.client.0.smithi067.stdout: create 183/31 file after the snapshot 2024-03-09T08:55:29.695 INFO:tasks.workunit.client.0.smithi067.stdout: create 183/32 file after the snapshot 2024-03-09T08:55:29.695 INFO:tasks.workunit.client.0.smithi067.stdout: create 183/33 file after the snapshot 2024-03-09T08:55:29.696 INFO:tasks.workunit.client.0.smithi067.stdout: create 183/34 file after the snapshot 2024-03-09T08:55:29.696 INFO:tasks.workunit.client.0.smithi067.stdout: create 183/35 file after the snapshot 2024-03-09T08:55:29.697 INFO:tasks.workunit.client.0.smithi067.stdout: create 183/36 file after the snapshot 2024-03-09T08:55:29.697 INFO:tasks.workunit.client.0.smithi067.stdout: create 183/37 file after the snapshot 2024-03-09T08:55:29.698 INFO:tasks.workunit.client.0.smithi067.stdout: create 183/38 file after the snapshot 2024-03-09T08:55:29.698 INFO:tasks.workunit.client.0.smithi067.stdout: create 183/39 file after the snapshot 2024-03-09T08:55:29.699 INFO:tasks.workunit.client.0.smithi067.stdout: create 184/.snap/snap-subdir-test 2024-03-09T08:55:29.780 INFO:tasks.workunit.client.0.smithi067.stdout: create 184/30 file after the snapshot 2024-03-09T08:55:29.783 INFO:tasks.workunit.client.0.smithi067.stdout: create 184/31 file after the snapshot 2024-03-09T08:55:29.783 INFO:tasks.workunit.client.0.smithi067.stdout: create 184/32 file after the snapshot 2024-03-09T08:55:29.784 INFO:tasks.workunit.client.0.smithi067.stdout: create 184/33 file after the snapshot 2024-03-09T08:55:29.784 INFO:tasks.workunit.client.0.smithi067.stdout: create 184/34 file after the snapshot 2024-03-09T08:55:29.784 INFO:tasks.workunit.client.0.smithi067.stdout: create 184/35 file after the snapshot 2024-03-09T08:55:29.785 INFO:tasks.workunit.client.0.smithi067.stdout: create 184/36 file after the snapshot 2024-03-09T08:55:29.785 INFO:tasks.workunit.client.0.smithi067.stdout: create 184/37 file after the snapshot 2024-03-09T08:55:29.786 INFO:tasks.workunit.client.0.smithi067.stdout: create 184/38 file after the snapshot 2024-03-09T08:55:29.786 INFO:tasks.workunit.client.0.smithi067.stdout: create 184/39 file after the snapshot 2024-03-09T08:55:29.787 INFO:tasks.workunit.client.0.smithi067.stdout: create 185/.snap/snap-subdir-test 2024-03-09T08:55:29.993 INFO:tasks.workunit.client.1.smithi134.stdout: create 216/30 file after the snapshot 2024-03-09T08:55:29.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 216/31 file after the snapshot 2024-03-09T08:55:29.997 INFO:tasks.workunit.client.1.smithi134.stdout: create 216/32 file after the snapshot 2024-03-09T08:55:29.997 INFO:tasks.workunit.client.1.smithi134.stdout: create 216/33 file after the snapshot 2024-03-09T08:55:29.997 INFO:tasks.workunit.client.1.smithi134.stdout: create 216/34 file after the snapshot 2024-03-09T08:55:29.998 INFO:tasks.workunit.client.1.smithi134.stdout: create 216/35 file after the snapshot 2024-03-09T08:55:29.998 INFO:tasks.workunit.client.1.smithi134.stdout: create 216/36 file after the snapshot 2024-03-09T08:55:29.998 INFO:tasks.workunit.client.1.smithi134.stdout: create 216/37 file after the snapshot 2024-03-09T08:55:29.998 INFO:tasks.workunit.client.1.smithi134.stdout: create 216/38 file after the snapshot 2024-03-09T08:55:29.999 INFO:tasks.workunit.client.1.smithi134.stdout: create 216/39 file after the snapshot 2024-03-09T08:55:29.999 INFO:tasks.workunit.client.1.smithi134.stdout: create 217/.snap/snap-subdir-test 2024-03-09T08:55:30.088 INFO:tasks.workunit.client.1.smithi134.stdout: create 217/30 file after the snapshot 2024-03-09T08:55:30.092 INFO:tasks.workunit.client.1.smithi134.stdout: create 217/31 file after the snapshot 2024-03-09T08:55:30.092 INFO:tasks.workunit.client.1.smithi134.stdout: create 217/32 file after the snapshot 2024-03-09T08:55:30.092 INFO:tasks.workunit.client.1.smithi134.stdout: create 217/33 file after the snapshot 2024-03-09T08:55:30.093 INFO:tasks.workunit.client.1.smithi134.stdout: create 217/34 file after the snapshot 2024-03-09T08:55:30.093 INFO:tasks.workunit.client.1.smithi134.stdout: create 217/35 file after the snapshot 2024-03-09T08:55:30.093 INFO:tasks.workunit.client.1.smithi134.stdout: create 217/36 file after the snapshot 2024-03-09T08:55:30.094 INFO:tasks.workunit.client.1.smithi134.stdout: create 217/37 file after the snapshot 2024-03-09T08:55:30.094 INFO:tasks.workunit.client.1.smithi134.stdout: create 217/38 file after the snapshot 2024-03-09T08:55:30.094 INFO:tasks.workunit.client.1.smithi134.stdout: create 217/39 file after the snapshot 2024-03-09T08:55:30.095 INFO:tasks.workunit.client.1.smithi134.stdout: create 218/.snap/snap-subdir-test 2024-03-09T08:55:30.178 INFO:tasks.workunit.client.1.smithi134.stdout: create 218/30 file after the snapshot 2024-03-09T08:55:30.180 INFO:tasks.workunit.client.1.smithi134.stdout: create 218/31 file after the snapshot 2024-03-09T08:55:30.181 INFO:tasks.workunit.client.1.smithi134.stdout: create 218/32 file after the snapshot 2024-03-09T08:55:30.181 INFO:tasks.workunit.client.1.smithi134.stdout: create 218/33 file after the snapshot 2024-03-09T08:55:30.181 INFO:tasks.workunit.client.1.smithi134.stdout: create 218/34 file after the snapshot 2024-03-09T08:55:30.182 INFO:tasks.workunit.client.1.smithi134.stdout: create 218/35 file after the snapshot 2024-03-09T08:55:30.182 INFO:tasks.workunit.client.1.smithi134.stdout: create 218/36 file after the snapshot 2024-03-09T08:55:30.183 INFO:tasks.workunit.client.1.smithi134.stdout: create 218/37 file after the snapshot 2024-03-09T08:55:30.183 INFO:tasks.workunit.client.1.smithi134.stdout: create 218/38 file after the snapshot 2024-03-09T08:55:30.183 INFO:tasks.workunit.client.1.smithi134.stdout: create 218/39 file after the snapshot 2024-03-09T08:55:30.183 INFO:tasks.workunit.client.1.smithi134.stdout: create 219/.snap/snap-subdir-test 2024-03-09T08:55:30.269 INFO:tasks.workunit.client.1.smithi134.stdout: create 219/30 file after the snapshot 2024-03-09T08:55:30.272 INFO:tasks.workunit.client.1.smithi134.stdout: create 219/31 file after the snapshot 2024-03-09T08:55:30.272 INFO:tasks.workunit.client.1.smithi134.stdout: create 219/32 file after the snapshot 2024-03-09T08:55:30.272 INFO:tasks.workunit.client.1.smithi134.stdout: create 219/33 file after the snapshot 2024-03-09T08:55:30.272 INFO:tasks.workunit.client.1.smithi134.stdout: create 219/34 file after the snapshot 2024-03-09T08:55:30.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 219/35 file after the snapshot 2024-03-09T08:55:30.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 219/36 file after the snapshot 2024-03-09T08:55:30.273 INFO:tasks.workunit.client.1.smithi134.stdout: create 219/37 file after the snapshot 2024-03-09T08:55:30.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 219/38 file after the snapshot 2024-03-09T08:55:30.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 219/39 file after the snapshot 2024-03-09T08:55:30.274 INFO:tasks.workunit.client.1.smithi134.stdout: create 220/.snap/snap-subdir-test 2024-03-09T08:55:30.369 INFO:tasks.workunit.client.1.smithi134.stdout: create 220/30 file after the snapshot 2024-03-09T08:55:30.372 INFO:tasks.workunit.client.1.smithi134.stdout: create 220/31 file after the snapshot 2024-03-09T08:55:30.372 INFO:tasks.workunit.client.1.smithi134.stdout: create 220/32 file after the snapshot 2024-03-09T08:55:30.373 INFO:tasks.workunit.client.1.smithi134.stdout: create 220/33 file after the snapshot 2024-03-09T08:55:30.373 INFO:tasks.workunit.client.1.smithi134.stdout: create 220/34 file after the snapshot 2024-03-09T08:55:30.374 INFO:tasks.workunit.client.1.smithi134.stdout: create 220/35 file after the snapshot 2024-03-09T08:55:30.374 INFO:tasks.workunit.client.1.smithi134.stdout: create 220/36 file after the snapshot 2024-03-09T08:55:30.374 INFO:tasks.workunit.client.1.smithi134.stdout: create 220/37 file after the snapshot 2024-03-09T08:55:30.375 INFO:tasks.workunit.client.1.smithi134.stdout: create 220/38 file after the snapshot 2024-03-09T08:55:30.376 INFO:tasks.workunit.client.1.smithi134.stdout: create 220/39 file after the snapshot 2024-03-09T08:55:30.376 INFO:tasks.workunit.client.1.smithi134.stdout: create 221/.snap/snap-subdir-test 2024-03-09T08:55:30.468 INFO:tasks.workunit.client.1.smithi134.stdout: create 221/30 file after the snapshot 2024-03-09T08:55:30.470 INFO:tasks.workunit.client.1.smithi134.stdout: create 221/31 file after the snapshot 2024-03-09T08:55:30.471 INFO:tasks.workunit.client.1.smithi134.stdout: create 221/32 file after the snapshot 2024-03-09T08:55:30.471 INFO:tasks.workunit.client.1.smithi134.stdout: create 221/33 file after the snapshot 2024-03-09T08:55:30.471 INFO:tasks.workunit.client.1.smithi134.stdout: create 221/34 file after the snapshot 2024-03-09T08:55:30.472 INFO:tasks.workunit.client.1.smithi134.stdout: create 221/35 file after the snapshot 2024-03-09T08:55:30.472 INFO:tasks.workunit.client.1.smithi134.stdout: create 221/36 file after the snapshot 2024-03-09T08:55:30.473 INFO:tasks.workunit.client.1.smithi134.stdout: create 221/37 file after the snapshot 2024-03-09T08:55:30.473 INFO:tasks.workunit.client.1.smithi134.stdout: create 221/38 file after the snapshot 2024-03-09T08:55:30.473 INFO:tasks.workunit.client.1.smithi134.stdout: create 221/39 file after the snapshot 2024-03-09T08:55:30.474 INFO:tasks.workunit.client.1.smithi134.stdout: create 222/.snap/snap-subdir-test 2024-03-09T08:55:30.558 INFO:tasks.workunit.client.1.smithi134.stdout: create 222/30 file after the snapshot 2024-03-09T08:55:30.561 INFO:tasks.workunit.client.1.smithi134.stdout: create 222/31 file after the snapshot 2024-03-09T08:55:30.562 INFO:tasks.workunit.client.1.smithi134.stdout: create 222/32 file after the snapshot 2024-03-09T08:55:30.562 INFO:tasks.workunit.client.1.smithi134.stdout: create 222/33 file after the snapshot 2024-03-09T08:55:30.563 INFO:tasks.workunit.client.1.smithi134.stdout: create 222/34 file after the snapshot 2024-03-09T08:55:30.563 INFO:tasks.workunit.client.1.smithi134.stdout: create 222/35 file after the snapshot 2024-03-09T08:55:30.564 INFO:tasks.workunit.client.1.smithi134.stdout: create 222/36 file after the snapshot 2024-03-09T08:55:30.564 INFO:tasks.workunit.client.1.smithi134.stdout: create 222/37 file after the snapshot 2024-03-09T08:55:30.565 INFO:tasks.workunit.client.1.smithi134.stdout: create 222/38 file after the snapshot 2024-03-09T08:55:30.566 INFO:tasks.workunit.client.1.smithi134.stdout: create 222/39 file after the snapshot 2024-03-09T08:55:30.566 INFO:tasks.workunit.client.1.smithi134.stdout: create 223/.snap/snap-subdir-test 2024-03-09T08:55:30.586 INFO:tasks.workunit.client.0.smithi067.stdout: create 185/30 file after the snapshot 2024-03-09T08:55:30.588 INFO:tasks.workunit.client.0.smithi067.stdout: create 185/31 file after the snapshot 2024-03-09T08:55:30.588 INFO:tasks.workunit.client.0.smithi067.stdout: create 185/32 file after the snapshot 2024-03-09T08:55:30.589 INFO:tasks.workunit.client.0.smithi067.stdout: create 185/33 file after the snapshot 2024-03-09T08:55:30.589 INFO:tasks.workunit.client.0.smithi067.stdout: create 185/34 file after the snapshot 2024-03-09T08:55:30.589 INFO:tasks.workunit.client.0.smithi067.stdout: create 185/35 file after the snapshot 2024-03-09T08:55:30.589 INFO:tasks.workunit.client.0.smithi067.stdout: create 185/36 file after the snapshot 2024-03-09T08:55:30.590 INFO:tasks.workunit.client.0.smithi067.stdout: create 185/37 file after the snapshot 2024-03-09T08:55:30.590 INFO:tasks.workunit.client.0.smithi067.stdout: create 185/38 file after the snapshot 2024-03-09T08:55:30.591 INFO:tasks.workunit.client.0.smithi067.stdout: create 185/39 file after the snapshot 2024-03-09T08:55:30.591 INFO:tasks.workunit.client.0.smithi067.stdout: create 186/.snap/snap-subdir-test 2024-03-09T08:55:30.664 INFO:tasks.workunit.client.0.smithi067.stdout: create 186/30 file after the snapshot 2024-03-09T08:55:30.667 INFO:tasks.workunit.client.0.smithi067.stdout: create 186/31 file after the snapshot 2024-03-09T08:55:30.667 INFO:tasks.workunit.client.0.smithi067.stdout: create 186/32 file after the snapshot 2024-03-09T08:55:30.667 INFO:tasks.workunit.client.0.smithi067.stdout: create 186/33 file after the snapshot 2024-03-09T08:55:30.667 INFO:tasks.workunit.client.0.smithi067.stdout: create 186/34 file after the snapshot 2024-03-09T08:55:30.668 INFO:tasks.workunit.client.0.smithi067.stdout: create 186/35 file after the snapshot 2024-03-09T08:55:30.668 INFO:tasks.workunit.client.0.smithi067.stdout: create 186/36 file after the snapshot 2024-03-09T08:55:30.668 INFO:tasks.workunit.client.0.smithi067.stdout: create 186/37 file after the snapshot 2024-03-09T08:55:30.669 INFO:tasks.workunit.client.0.smithi067.stdout: create 186/38 file after the snapshot 2024-03-09T08:55:30.669 INFO:tasks.workunit.client.0.smithi067.stdout: create 186/39 file after the snapshot 2024-03-09T08:55:30.669 INFO:tasks.workunit.client.0.smithi067.stdout: create 187/.snap/snap-subdir-test 2024-03-09T08:55:30.745 INFO:tasks.workunit.client.0.smithi067.stdout: create 187/30 file after the snapshot 2024-03-09T08:55:30.748 INFO:tasks.workunit.client.0.smithi067.stdout: create 187/31 file after the snapshot 2024-03-09T08:55:30.748 INFO:tasks.workunit.client.0.smithi067.stdout: create 187/32 file after the snapshot 2024-03-09T08:55:30.748 INFO:tasks.workunit.client.0.smithi067.stdout: create 187/33 file after the snapshot 2024-03-09T08:55:30.749 INFO:tasks.workunit.client.0.smithi067.stdout: create 187/34 file after the snapshot 2024-03-09T08:55:30.749 INFO:tasks.workunit.client.0.smithi067.stdout: create 187/35 file after the snapshot 2024-03-09T08:55:30.749 INFO:tasks.workunit.client.0.smithi067.stdout: create 187/36 file after the snapshot 2024-03-09T08:55:30.749 INFO:tasks.workunit.client.0.smithi067.stdout: create 187/37 file after the snapshot 2024-03-09T08:55:30.749 INFO:tasks.workunit.client.0.smithi067.stdout: create 187/38 file after the snapshot 2024-03-09T08:55:30.749 INFO:tasks.workunit.client.0.smithi067.stdout: create 187/39 file after the snapshot 2024-03-09T08:55:30.749 INFO:tasks.workunit.client.0.smithi067.stdout: create 188/.snap/snap-subdir-test 2024-03-09T08:55:30.754 INFO:tasks.workunit.client.1.smithi134.stdout: create 223/30 file after the snapshot 2024-03-09T08:55:30.756 INFO:tasks.workunit.client.1.smithi134.stdout: create 223/31 file after the snapshot 2024-03-09T08:55:30.756 INFO:tasks.workunit.client.1.smithi134.stdout: create 223/32 file after the snapshot 2024-03-09T08:55:30.757 INFO:tasks.workunit.client.1.smithi134.stdout: create 223/33 file after the snapshot 2024-03-09T08:55:30.757 INFO:tasks.workunit.client.1.smithi134.stdout: create 223/34 file after the snapshot 2024-03-09T08:55:30.758 INFO:tasks.workunit.client.1.smithi134.stdout: create 223/35 file after the snapshot 2024-03-09T08:55:30.758 INFO:tasks.workunit.client.1.smithi134.stdout: create 223/36 file after the snapshot 2024-03-09T08:55:30.758 INFO:tasks.workunit.client.1.smithi134.stdout: create 223/37 file after the snapshot 2024-03-09T08:55:30.758 INFO:tasks.workunit.client.1.smithi134.stdout: create 223/38 file after the snapshot 2024-03-09T08:55:30.759 INFO:tasks.workunit.client.1.smithi134.stdout: create 223/39 file after the snapshot 2024-03-09T08:55:30.759 INFO:tasks.workunit.client.1.smithi134.stdout: create 224/.snap/snap-subdir-test 2024-03-09T08:55:30.802 INFO:tasks.workunit.client.0.smithi067.stdout: create 188/30 file after the snapshot 2024-03-09T08:55:30.805 INFO:tasks.workunit.client.0.smithi067.stdout: create 188/31 file after the snapshot 2024-03-09T08:55:30.805 INFO:tasks.workunit.client.0.smithi067.stdout: create 188/32 file after the snapshot 2024-03-09T08:55:30.806 INFO:tasks.workunit.client.0.smithi067.stdout: create 188/33 file after the snapshot 2024-03-09T08:55:30.806 INFO:tasks.workunit.client.0.smithi067.stdout: create 188/34 file after the snapshot 2024-03-09T08:55:30.806 INFO:tasks.workunit.client.0.smithi067.stdout: create 188/35 file after the snapshot 2024-03-09T08:55:30.807 INFO:tasks.workunit.client.0.smithi067.stdout: create 188/36 file after the snapshot 2024-03-09T08:55:30.807 INFO:tasks.workunit.client.0.smithi067.stdout: create 188/37 file after the snapshot 2024-03-09T08:55:30.807 INFO:tasks.workunit.client.0.smithi067.stdout: create 188/38 file after the snapshot 2024-03-09T08:55:30.808 INFO:tasks.workunit.client.0.smithi067.stdout: create 188/39 file after the snapshot 2024-03-09T08:55:30.808 INFO:tasks.workunit.client.0.smithi067.stdout: create 189/.snap/snap-subdir-test 2024-03-09T08:55:30.818 INFO:tasks.workunit.client.1.smithi134.stdout: create 224/30 file after the snapshot 2024-03-09T08:55:30.822 INFO:tasks.workunit.client.1.smithi134.stdout: create 224/31 file after the snapshot 2024-03-09T08:55:30.822 INFO:tasks.workunit.client.1.smithi134.stdout: create 224/32 file after the snapshot 2024-03-09T08:55:30.823 INFO:tasks.workunit.client.1.smithi134.stdout: create 224/33 file after the snapshot 2024-03-09T08:55:30.823 INFO:tasks.workunit.client.1.smithi134.stdout: create 224/34 file after the snapshot 2024-03-09T08:55:30.823 INFO:tasks.workunit.client.1.smithi134.stdout: create 224/35 file after the snapshot 2024-03-09T08:55:30.823 INFO:tasks.workunit.client.1.smithi134.stdout: create 224/36 file after the snapshot 2024-03-09T08:55:30.823 INFO:tasks.workunit.client.1.smithi134.stdout: create 224/37 file after the snapshot 2024-03-09T08:55:30.824 INFO:tasks.workunit.client.1.smithi134.stdout: create 224/38 file after the snapshot 2024-03-09T08:55:30.824 INFO:tasks.workunit.client.1.smithi134.stdout: create 224/39 file after the snapshot 2024-03-09T08:55:30.824 INFO:tasks.workunit.client.1.smithi134.stdout: create 225/.snap/snap-subdir-test 2024-03-09T08:55:30.869 INFO:tasks.workunit.client.0.smithi067.stdout: create 189/30 file after the snapshot 2024-03-09T08:55:30.872 INFO:tasks.workunit.client.0.smithi067.stdout: create 189/31 file after the snapshot 2024-03-09T08:55:30.872 INFO:tasks.workunit.client.0.smithi067.stdout: create 189/32 file after the snapshot 2024-03-09T08:55:30.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 189/33 file after the snapshot 2024-03-09T08:55:30.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 189/34 file after the snapshot 2024-03-09T08:55:30.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 189/35 file after the snapshot 2024-03-09T08:55:30.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 189/36 file after the snapshot 2024-03-09T08:55:30.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 189/37 file after the snapshot 2024-03-09T08:55:30.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 189/38 file after the snapshot 2024-03-09T08:55:30.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 189/39 file after the snapshot 2024-03-09T08:55:30.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 190/.snap/snap-subdir-test 2024-03-09T08:55:30.908 INFO:tasks.workunit.client.1.smithi134.stdout: create 225/30 file after the snapshot 2024-03-09T08:55:30.911 INFO:tasks.workunit.client.1.smithi134.stdout: create 225/31 file after the snapshot 2024-03-09T08:55:30.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 225/32 file after the snapshot 2024-03-09T08:55:30.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 225/33 file after the snapshot 2024-03-09T08:55:30.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 225/34 file after the snapshot 2024-03-09T08:55:30.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 225/35 file after the snapshot 2024-03-09T08:55:30.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 225/36 file after the snapshot 2024-03-09T08:55:30.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 225/37 file after the snapshot 2024-03-09T08:55:30.913 INFO:tasks.workunit.client.1.smithi134.stdout: create 225/38 file after the snapshot 2024-03-09T08:55:30.913 INFO:tasks.workunit.client.1.smithi134.stdout: create 225/39 file after the snapshot 2024-03-09T08:55:30.913 INFO:tasks.workunit.client.1.smithi134.stdout: create 226/.snap/snap-subdir-test 2024-03-09T08:55:30.982 INFO:tasks.workunit.client.1.smithi134.stdout: create 226/30 file after the snapshot 2024-03-09T08:55:30.985 INFO:tasks.workunit.client.1.smithi134.stdout: create 226/31 file after the snapshot 2024-03-09T08:55:30.985 INFO:tasks.workunit.client.1.smithi134.stdout: create 226/32 file after the snapshot 2024-03-09T08:55:30.985 INFO:tasks.workunit.client.1.smithi134.stdout: create 226/33 file after the snapshot 2024-03-09T08:55:30.985 INFO:tasks.workunit.client.1.smithi134.stdout: create 226/34 file after the snapshot 2024-03-09T08:55:30.986 INFO:tasks.workunit.client.1.smithi134.stdout: create 226/35 file after the snapshot 2024-03-09T08:55:30.986 INFO:tasks.workunit.client.1.smithi134.stdout: create 226/36 file after the snapshot 2024-03-09T08:55:30.987 INFO:tasks.workunit.client.1.smithi134.stdout: create 226/37 file after the snapshot 2024-03-09T08:55:30.987 INFO:tasks.workunit.client.1.smithi134.stdout: create 226/38 file after the snapshot 2024-03-09T08:55:30.987 INFO:tasks.workunit.client.1.smithi134.stdout: create 226/39 file after the snapshot 2024-03-09T08:55:30.987 INFO:tasks.workunit.client.1.smithi134.stdout: create 227/.snap/snap-subdir-test 2024-03-09T08:55:31.055 INFO:tasks.workunit.client.1.smithi134.stdout: create 227/30 file after the snapshot 2024-03-09T08:55:31.059 INFO:tasks.workunit.client.1.smithi134.stdout: create 227/31 file after the snapshot 2024-03-09T08:55:31.059 INFO:tasks.workunit.client.1.smithi134.stdout: create 227/32 file after the snapshot 2024-03-09T08:55:31.059 INFO:tasks.workunit.client.1.smithi134.stdout: create 227/33 file after the snapshot 2024-03-09T08:55:31.059 INFO:tasks.workunit.client.1.smithi134.stdout: create 227/34 file after the snapshot 2024-03-09T08:55:31.059 INFO:tasks.workunit.client.1.smithi134.stdout: create 227/35 file after the snapshot 2024-03-09T08:55:31.060 INFO:tasks.workunit.client.1.smithi134.stdout: create 227/36 file after the snapshot 2024-03-09T08:55:31.060 INFO:tasks.workunit.client.1.smithi134.stdout: create 227/37 file after the snapshot 2024-03-09T08:55:31.060 INFO:tasks.workunit.client.1.smithi134.stdout: create 227/38 file after the snapshot 2024-03-09T08:55:31.060 INFO:tasks.workunit.client.1.smithi134.stdout: create 227/39 file after the snapshot 2024-03-09T08:55:31.060 INFO:tasks.workunit.client.1.smithi134.stdout: create 228/.snap/snap-subdir-test 2024-03-09T08:55:31.143 INFO:tasks.workunit.client.1.smithi134.stdout: create 228/30 file after the snapshot 2024-03-09T08:55:31.146 INFO:tasks.workunit.client.1.smithi134.stdout: create 228/31 file after the snapshot 2024-03-09T08:55:31.146 INFO:tasks.workunit.client.1.smithi134.stdout: create 228/32 file after the snapshot 2024-03-09T08:55:31.146 INFO:tasks.workunit.client.1.smithi134.stdout: create 228/33 file after the snapshot 2024-03-09T08:55:31.147 INFO:tasks.workunit.client.1.smithi134.stdout: create 228/34 file after the snapshot 2024-03-09T08:55:31.147 INFO:tasks.workunit.client.1.smithi134.stdout: create 228/35 file after the snapshot 2024-03-09T08:55:31.147 INFO:tasks.workunit.client.1.smithi134.stdout: create 228/36 file after the snapshot 2024-03-09T08:55:31.147 INFO:tasks.workunit.client.1.smithi134.stdout: create 228/37 file after the snapshot 2024-03-09T08:55:31.148 INFO:tasks.workunit.client.1.smithi134.stdout: create 228/38 file after the snapshot 2024-03-09T08:55:31.148 INFO:tasks.workunit.client.1.smithi134.stdout: create 228/39 file after the snapshot 2024-03-09T08:55:31.148 INFO:tasks.workunit.client.1.smithi134.stdout: create 229/.snap/snap-subdir-test 2024-03-09T08:55:31.214 INFO:tasks.workunit.client.0.smithi067.stdout: create 190/30 file after the snapshot 2024-03-09T08:55:31.216 INFO:tasks.workunit.client.0.smithi067.stdout: create 190/31 file after the snapshot 2024-03-09T08:55:31.216 INFO:tasks.workunit.client.0.smithi067.stdout: create 190/32 file after the snapshot 2024-03-09T08:55:31.216 INFO:tasks.workunit.client.0.smithi067.stdout: create 190/33 file after the snapshot 2024-03-09T08:55:31.217 INFO:tasks.workunit.client.0.smithi067.stdout: create 190/34 file after the snapshot 2024-03-09T08:55:31.217 INFO:tasks.workunit.client.0.smithi067.stdout: create 190/35 file after the snapshot 2024-03-09T08:55:31.218 INFO:tasks.workunit.client.0.smithi067.stdout: create 190/36 file after the snapshot 2024-03-09T08:55:31.218 INFO:tasks.workunit.client.0.smithi067.stdout: create 190/37 file after the snapshot 2024-03-09T08:55:31.218 INFO:tasks.workunit.client.0.smithi067.stdout: create 190/38 file after the snapshot 2024-03-09T08:55:31.218 INFO:tasks.workunit.client.0.smithi067.stdout: create 190/39 file after the snapshot 2024-03-09T08:55:31.218 INFO:tasks.workunit.client.0.smithi067.stdout: create 191/.snap/snap-subdir-test 2024-03-09T08:55:31.233 INFO:tasks.workunit.client.1.smithi134.stdout: create 229/30 file after the snapshot 2024-03-09T08:55:31.236 INFO:tasks.workunit.client.1.smithi134.stdout: create 229/31 file after the snapshot 2024-03-09T08:55:31.236 INFO:tasks.workunit.client.1.smithi134.stdout: create 229/32 file after the snapshot 2024-03-09T08:55:31.236 INFO:tasks.workunit.client.1.smithi134.stdout: create 229/33 file after the snapshot 2024-03-09T08:55:31.237 INFO:tasks.workunit.client.1.smithi134.stdout: create 229/34 file after the snapshot 2024-03-09T08:55:31.237 INFO:tasks.workunit.client.1.smithi134.stdout: create 229/35 file after the snapshot 2024-03-09T08:55:31.237 INFO:tasks.workunit.client.1.smithi134.stdout: create 229/36 file after the snapshot 2024-03-09T08:55:31.237 INFO:tasks.workunit.client.1.smithi134.stdout: create 229/37 file after the snapshot 2024-03-09T08:55:31.238 INFO:tasks.workunit.client.1.smithi134.stdout: create 229/38 file after the snapshot 2024-03-09T08:55:31.238 INFO:tasks.workunit.client.1.smithi134.stdout: create 229/39 file after the snapshot 2024-03-09T08:55:31.239 INFO:tasks.workunit.client.1.smithi134.stdout: create 230/.snap/snap-subdir-test 2024-03-09T08:55:31.293 INFO:tasks.workunit.client.0.smithi067.stdout: create 191/30 file after the snapshot 2024-03-09T08:55:31.295 INFO:tasks.workunit.client.0.smithi067.stdout: create 191/31 file after the snapshot 2024-03-09T08:55:31.295 INFO:tasks.workunit.client.0.smithi067.stdout: create 191/32 file after the snapshot 2024-03-09T08:55:31.295 INFO:tasks.workunit.client.0.smithi067.stdout: create 191/33 file after the snapshot 2024-03-09T08:55:31.295 INFO:tasks.workunit.client.0.smithi067.stdout: create 191/34 file after the snapshot 2024-03-09T08:55:31.296 INFO:tasks.workunit.client.0.smithi067.stdout: create 191/35 file after the snapshot 2024-03-09T08:55:31.296 INFO:tasks.workunit.client.0.smithi067.stdout: create 191/36 file after the snapshot 2024-03-09T08:55:31.296 INFO:tasks.workunit.client.0.smithi067.stdout: create 191/37 file after the snapshot 2024-03-09T08:55:31.297 INFO:tasks.workunit.client.0.smithi067.stdout: create 191/38 file after the snapshot 2024-03-09T08:55:31.297 INFO:tasks.workunit.client.0.smithi067.stdout: create 191/39 file after the snapshot 2024-03-09T08:55:31.297 INFO:tasks.workunit.client.0.smithi067.stdout: create 192/.snap/snap-subdir-test 2024-03-09T08:55:31.313 INFO:tasks.workunit.client.1.smithi134.stdout: create 230/30 file after the snapshot 2024-03-09T08:55:31.314 INFO:tasks.workunit.client.1.smithi134.stdout: create 230/31 file after the snapshot 2024-03-09T08:55:31.315 INFO:tasks.workunit.client.1.smithi134.stdout: create 230/32 file after the snapshot 2024-03-09T08:55:31.315 INFO:tasks.workunit.client.1.smithi134.stdout: create 230/33 file after the snapshot 2024-03-09T08:55:31.316 INFO:tasks.workunit.client.1.smithi134.stdout: create 230/34 file after the snapshot 2024-03-09T08:55:31.316 INFO:tasks.workunit.client.1.smithi134.stdout: create 230/35 file after the snapshot 2024-03-09T08:55:31.317 INFO:tasks.workunit.client.1.smithi134.stdout: create 230/36 file after the snapshot 2024-03-09T08:55:31.317 INFO:tasks.workunit.client.1.smithi134.stdout: create 230/37 file after the snapshot 2024-03-09T08:55:31.319 INFO:tasks.workunit.client.1.smithi134.stdout: create 230/38 file after the snapshot 2024-03-09T08:55:31.320 INFO:tasks.workunit.client.1.smithi134.stdout: create 230/39 file after the snapshot 2024-03-09T08:55:31.320 INFO:tasks.workunit.client.1.smithi134.stdout: create 231/.snap/snap-subdir-test 2024-03-09T08:55:31.356 INFO:tasks.workunit.client.0.smithi067.stdout: create 192/30 file after the snapshot 2024-03-09T08:55:31.358 INFO:tasks.workunit.client.0.smithi067.stdout: create 192/31 file after the snapshot 2024-03-09T08:55:31.359 INFO:tasks.workunit.client.0.smithi067.stdout: create 192/32 file after the snapshot 2024-03-09T08:55:31.359 INFO:tasks.workunit.client.0.smithi067.stdout: create 192/33 file after the snapshot 2024-03-09T08:55:31.359 INFO:tasks.workunit.client.0.smithi067.stdout: create 192/34 file after the snapshot 2024-03-09T08:55:31.360 INFO:tasks.workunit.client.0.smithi067.stdout: create 192/35 file after the snapshot 2024-03-09T08:55:31.360 INFO:tasks.workunit.client.0.smithi067.stdout: create 192/36 file after the snapshot 2024-03-09T08:55:31.360 INFO:tasks.workunit.client.0.smithi067.stdout: create 192/37 file after the snapshot 2024-03-09T08:55:31.360 INFO:tasks.workunit.client.0.smithi067.stdout: create 192/38 file after the snapshot 2024-03-09T08:55:31.360 INFO:tasks.workunit.client.0.smithi067.stdout: create 192/39 file after the snapshot 2024-03-09T08:55:31.361 INFO:tasks.workunit.client.0.smithi067.stdout: create 193/.snap/snap-subdir-test 2024-03-09T08:55:31.418 INFO:tasks.workunit.client.0.smithi067.stdout: create 193/30 file after the snapshot 2024-03-09T08:55:31.420 INFO:tasks.workunit.client.0.smithi067.stdout: create 193/31 file after the snapshot 2024-03-09T08:55:31.421 INFO:tasks.workunit.client.0.smithi067.stdout: create 193/32 file after the snapshot 2024-03-09T08:55:31.421 INFO:tasks.workunit.client.0.smithi067.stdout: create 193/33 file after the snapshot 2024-03-09T08:55:31.422 INFO:tasks.workunit.client.0.smithi067.stdout: create 193/34 file after the snapshot 2024-03-09T08:55:31.422 INFO:tasks.workunit.client.0.smithi067.stdout: create 193/35 file after the snapshot 2024-03-09T08:55:31.422 INFO:tasks.workunit.client.0.smithi067.stdout: create 193/36 file after the snapshot 2024-03-09T08:55:31.423 INFO:tasks.workunit.client.0.smithi067.stdout: create 193/37 file after the snapshot 2024-03-09T08:55:31.423 INFO:tasks.workunit.client.0.smithi067.stdout: create 193/38 file after the snapshot 2024-03-09T08:55:31.423 INFO:tasks.workunit.client.0.smithi067.stdout: create 193/39 file after the snapshot 2024-03-09T08:55:31.424 INFO:tasks.workunit.client.0.smithi067.stdout: create 194/.snap/snap-subdir-test 2024-03-09T08:55:31.490 INFO:tasks.workunit.client.0.smithi067.stdout: create 194/30 file after the snapshot 2024-03-09T08:55:31.493 INFO:tasks.workunit.client.0.smithi067.stdout: create 194/31 file after the snapshot 2024-03-09T08:55:31.493 INFO:tasks.workunit.client.0.smithi067.stdout: create 194/32 file after the snapshot 2024-03-09T08:55:31.494 INFO:tasks.workunit.client.0.smithi067.stdout: create 194/33 file after the snapshot 2024-03-09T08:55:31.494 INFO:tasks.workunit.client.0.smithi067.stdout: create 194/34 file after the snapshot 2024-03-09T08:55:31.494 INFO:tasks.workunit.client.0.smithi067.stdout: create 194/35 file after the snapshot 2024-03-09T08:55:31.494 INFO:tasks.workunit.client.0.smithi067.stdout: create 194/36 file after the snapshot 2024-03-09T08:55:31.494 INFO:tasks.workunit.client.0.smithi067.stdout: create 194/37 file after the snapshot 2024-03-09T08:55:31.495 INFO:tasks.workunit.client.0.smithi067.stdout: create 194/38 file after the snapshot 2024-03-09T08:55:31.495 INFO:tasks.workunit.client.0.smithi067.stdout: create 194/39 file after the snapshot 2024-03-09T08:55:31.495 INFO:tasks.workunit.client.0.smithi067.stdout: create 195/.snap/snap-subdir-test 2024-03-09T08:55:31.555 INFO:tasks.workunit.client.0.smithi067.stdout: create 195/30 file after the snapshot 2024-03-09T08:55:31.558 INFO:tasks.workunit.client.0.smithi067.stdout: create 195/31 file after the snapshot 2024-03-09T08:55:31.558 INFO:tasks.workunit.client.0.smithi067.stdout: create 195/32 file after the snapshot 2024-03-09T08:55:31.558 INFO:tasks.workunit.client.0.smithi067.stdout: create 195/33 file after the snapshot 2024-03-09T08:55:31.559 INFO:tasks.workunit.client.0.smithi067.stdout: create 195/34 file after the snapshot 2024-03-09T08:55:31.559 INFO:tasks.workunit.client.0.smithi067.stdout: create 195/35 file after the snapshot 2024-03-09T08:55:31.559 INFO:tasks.workunit.client.0.smithi067.stdout: create 195/36 file after the snapshot 2024-03-09T08:55:31.559 INFO:tasks.workunit.client.0.smithi067.stdout: create 195/37 file after the snapshot 2024-03-09T08:55:31.560 INFO:tasks.workunit.client.0.smithi067.stdout: create 195/38 file after the snapshot 2024-03-09T08:55:31.560 INFO:tasks.workunit.client.0.smithi067.stdout: create 195/39 file after the snapshot 2024-03-09T08:55:31.560 INFO:tasks.workunit.client.0.smithi067.stdout: create 196/.snap/snap-subdir-test 2024-03-09T08:55:31.626 INFO:tasks.workunit.client.0.smithi067.stdout: create 196/30 file after the snapshot 2024-03-09T08:55:31.629 INFO:tasks.workunit.client.0.smithi067.stdout: create 196/31 file after the snapshot 2024-03-09T08:55:31.630 INFO:tasks.workunit.client.0.smithi067.stdout: create 196/32 file after the snapshot 2024-03-09T08:55:31.630 INFO:tasks.workunit.client.0.smithi067.stdout: create 196/33 file after the snapshot 2024-03-09T08:55:31.630 INFO:tasks.workunit.client.0.smithi067.stdout: create 196/34 file after the snapshot 2024-03-09T08:55:31.630 INFO:tasks.workunit.client.0.smithi067.stdout: create 196/35 file after the snapshot 2024-03-09T08:55:31.630 INFO:tasks.workunit.client.0.smithi067.stdout: create 196/36 file after the snapshot 2024-03-09T08:55:31.631 INFO:tasks.workunit.client.0.smithi067.stdout: create 196/37 file after the snapshot 2024-03-09T08:55:31.631 INFO:tasks.workunit.client.0.smithi067.stdout: create 196/38 file after the snapshot 2024-03-09T08:55:31.631 INFO:tasks.workunit.client.0.smithi067.stdout: create 196/39 file after the snapshot 2024-03-09T08:55:31.631 INFO:tasks.workunit.client.0.smithi067.stdout: create 197/.snap/snap-subdir-test 2024-03-09T08:55:31.721 INFO:tasks.workunit.client.0.smithi067.stdout: create 197/30 file after the snapshot 2024-03-09T08:55:31.724 INFO:tasks.workunit.client.0.smithi067.stdout: create 197/31 file after the snapshot 2024-03-09T08:55:31.724 INFO:tasks.workunit.client.0.smithi067.stdout: create 197/32 file after the snapshot 2024-03-09T08:55:31.725 INFO:tasks.workunit.client.0.smithi067.stdout: create 197/33 file after the snapshot 2024-03-09T08:55:31.725 INFO:tasks.workunit.client.0.smithi067.stdout: create 197/34 file after the snapshot 2024-03-09T08:55:31.725 INFO:tasks.workunit.client.0.smithi067.stdout: create 197/35 file after the snapshot 2024-03-09T08:55:31.726 INFO:tasks.workunit.client.0.smithi067.stdout: create 197/36 file after the snapshot 2024-03-09T08:55:31.726 INFO:tasks.workunit.client.0.smithi067.stdout: create 197/37 file after the snapshot 2024-03-09T08:55:31.726 INFO:tasks.workunit.client.0.smithi067.stdout: create 197/38 file after the snapshot 2024-03-09T08:55:31.726 INFO:tasks.workunit.client.0.smithi067.stdout: create 197/39 file after the snapshot 2024-03-09T08:55:31.727 INFO:tasks.workunit.client.0.smithi067.stdout: create 198/.snap/snap-subdir-test 2024-03-09T08:55:31.796 INFO:tasks.workunit.client.0.smithi067.stdout: create 198/30 file after the snapshot 2024-03-09T08:55:31.799 INFO:tasks.workunit.client.0.smithi067.stdout: create 198/31 file after the snapshot 2024-03-09T08:55:31.799 INFO:tasks.workunit.client.0.smithi067.stdout: create 198/32 file after the snapshot 2024-03-09T08:55:31.800 INFO:tasks.workunit.client.0.smithi067.stdout: create 198/33 file after the snapshot 2024-03-09T08:55:31.800 INFO:tasks.workunit.client.0.smithi067.stdout: create 198/34 file after the snapshot 2024-03-09T08:55:31.801 INFO:tasks.workunit.client.0.smithi067.stdout: create 198/35 file after the snapshot 2024-03-09T08:55:31.801 INFO:tasks.workunit.client.0.smithi067.stdout: create 198/36 file after the snapshot 2024-03-09T08:55:31.802 INFO:tasks.workunit.client.0.smithi067.stdout: create 198/37 file after the snapshot 2024-03-09T08:55:31.803 INFO:tasks.workunit.client.0.smithi067.stdout: create 198/38 file after the snapshot 2024-03-09T08:55:31.804 INFO:tasks.workunit.client.0.smithi067.stdout: create 198/39 file after the snapshot 2024-03-09T08:55:31.804 INFO:tasks.workunit.client.0.smithi067.stdout: create 199/.snap/snap-subdir-test 2024-03-09T08:55:31.878 INFO:tasks.workunit.client.0.smithi067.stdout: create 199/30 file after the snapshot 2024-03-09T08:55:31.882 INFO:tasks.workunit.client.0.smithi067.stdout: create 199/31 file after the snapshot 2024-03-09T08:55:31.882 INFO:tasks.workunit.client.0.smithi067.stdout: create 199/32 file after the snapshot 2024-03-09T08:55:31.882 INFO:tasks.workunit.client.0.smithi067.stdout: create 199/33 file after the snapshot 2024-03-09T08:55:31.883 INFO:tasks.workunit.client.0.smithi067.stdout: create 199/34 file after the snapshot 2024-03-09T08:55:31.883 INFO:tasks.workunit.client.0.smithi067.stdout: create 199/35 file after the snapshot 2024-03-09T08:55:31.883 INFO:tasks.workunit.client.0.smithi067.stdout: create 199/36 file after the snapshot 2024-03-09T08:55:31.883 INFO:tasks.workunit.client.0.smithi067.stdout: create 199/37 file after the snapshot 2024-03-09T08:55:31.884 INFO:tasks.workunit.client.0.smithi067.stdout: create 199/38 file after the snapshot 2024-03-09T08:55:31.884 INFO:tasks.workunit.client.0.smithi067.stdout: create 199/39 file after the snapshot 2024-03-09T08:55:31.884 INFO:tasks.workunit.client.0.smithi067.stdout: create 200/.snap/snap-subdir-test 2024-03-09T08:55:31.980 INFO:tasks.workunit.client.0.smithi067.stdout: create 200/30 file after the snapshot 2024-03-09T08:55:32.015 INFO:tasks.workunit.client.1.smithi134.stdout: create 231/30 file after the snapshot 2024-03-09T08:55:32.017 INFO:tasks.workunit.client.1.smithi134.stdout: create 231/31 file after the snapshot 2024-03-09T08:55:32.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 231/32 file after the snapshot 2024-03-09T08:55:32.018 INFO:tasks.workunit.client.1.smithi134.stdout: create 231/33 file after the snapshot 2024-03-09T08:55:32.019 INFO:tasks.workunit.client.1.smithi134.stdout: create 231/34 file after the snapshot 2024-03-09T08:55:32.019 INFO:tasks.workunit.client.1.smithi134.stdout: create 231/35 file after the snapshot 2024-03-09T08:55:32.020 INFO:tasks.workunit.client.1.smithi134.stdout: create 231/36 file after the snapshot 2024-03-09T08:55:32.020 INFO:tasks.workunit.client.1.smithi134.stdout: create 231/37 file after the snapshot 2024-03-09T08:55:32.021 INFO:tasks.workunit.client.1.smithi134.stdout: create 231/38 file after the snapshot 2024-03-09T08:55:32.022 INFO:tasks.workunit.client.1.smithi134.stdout: create 231/39 file after the snapshot 2024-03-09T08:55:32.022 INFO:tasks.workunit.client.1.smithi134.stdout: create 232/.snap/snap-subdir-test 2024-03-09T08:55:32.108 INFO:tasks.workunit.client.1.smithi134.stdout: create 232/30 file after the snapshot 2024-03-09T08:55:32.111 INFO:tasks.workunit.client.1.smithi134.stdout: create 232/31 file after the snapshot 2024-03-09T08:55:32.111 INFO:tasks.workunit.client.1.smithi134.stdout: create 232/32 file after the snapshot 2024-03-09T08:55:32.111 INFO:tasks.workunit.client.1.smithi134.stdout: create 232/33 file after the snapshot 2024-03-09T08:55:32.112 INFO:tasks.workunit.client.1.smithi134.stdout: create 232/34 file after the snapshot 2024-03-09T08:55:32.112 INFO:tasks.workunit.client.1.smithi134.stdout: create 232/35 file after the snapshot 2024-03-09T08:55:32.113 INFO:tasks.workunit.client.1.smithi134.stdout: create 232/36 file after the snapshot 2024-03-09T08:55:32.113 INFO:tasks.workunit.client.1.smithi134.stdout: create 232/37 file after the snapshot 2024-03-09T08:55:32.113 INFO:tasks.workunit.client.1.smithi134.stdout: create 232/38 file after the snapshot 2024-03-09T08:55:32.114 INFO:tasks.workunit.client.1.smithi134.stdout: create 232/39 file after the snapshot 2024-03-09T08:55:32.115 INFO:tasks.workunit.client.1.smithi134.stdout: create 233/.snap/snap-subdir-test 2024-03-09T08:55:32.188 INFO:tasks.workunit.client.1.smithi134.stdout: create 233/30 file after the snapshot 2024-03-09T08:55:32.190 INFO:tasks.workunit.client.1.smithi134.stdout: create 233/31 file after the snapshot 2024-03-09T08:55:32.191 INFO:tasks.workunit.client.1.smithi134.stdout: create 233/32 file after the snapshot 2024-03-09T08:55:32.191 INFO:tasks.workunit.client.1.smithi134.stdout: create 233/33 file after the snapshot 2024-03-09T08:55:32.191 INFO:tasks.workunit.client.1.smithi134.stdout: create 233/34 file after the snapshot 2024-03-09T08:55:32.191 INFO:tasks.workunit.client.1.smithi134.stdout: create 233/35 file after the snapshot 2024-03-09T08:55:32.192 INFO:tasks.workunit.client.1.smithi134.stdout: create 233/36 file after the snapshot 2024-03-09T08:55:32.192 INFO:tasks.workunit.client.1.smithi134.stdout: create 233/37 file after the snapshot 2024-03-09T08:55:32.192 INFO:tasks.workunit.client.1.smithi134.stdout: create 233/38 file after the snapshot 2024-03-09T08:55:32.193 INFO:tasks.workunit.client.1.smithi134.stdout: create 233/39 file after the snapshot 2024-03-09T08:55:32.193 INFO:tasks.workunit.client.1.smithi134.stdout: create 234/.snap/snap-subdir-test 2024-03-09T08:55:32.263 INFO:tasks.workunit.client.1.smithi134.stdout: create 234/30 file after the snapshot 2024-03-09T08:55:32.265 INFO:tasks.workunit.client.1.smithi134.stdout: create 234/31 file after the snapshot 2024-03-09T08:55:32.266 INFO:tasks.workunit.client.1.smithi134.stdout: create 234/32 file after the snapshot 2024-03-09T08:55:32.266 INFO:tasks.workunit.client.1.smithi134.stdout: create 234/33 file after the snapshot 2024-03-09T08:55:32.266 INFO:tasks.workunit.client.1.smithi134.stdout: create 234/34 file after the snapshot 2024-03-09T08:55:32.266 INFO:tasks.workunit.client.1.smithi134.stdout: create 234/35 file after the snapshot 2024-03-09T08:55:32.267 INFO:tasks.workunit.client.1.smithi134.stdout: create 234/36 file after the snapshot 2024-03-09T08:55:32.267 INFO:tasks.workunit.client.1.smithi134.stdout: create 234/37 file after the snapshot 2024-03-09T08:55:32.267 INFO:tasks.workunit.client.1.smithi134.stdout: create 234/38 file after the snapshot 2024-03-09T08:55:32.267 INFO:tasks.workunit.client.1.smithi134.stdout: create 234/39 file after the snapshot 2024-03-09T08:55:32.268 INFO:tasks.workunit.client.1.smithi134.stdout: create 235/.snap/snap-subdir-test 2024-03-09T08:55:32.342 INFO:tasks.workunit.client.1.smithi134.stdout: create 235/30 file after the snapshot 2024-03-09T08:55:32.345 INFO:tasks.workunit.client.1.smithi134.stdout: create 235/31 file after the snapshot 2024-03-09T08:55:32.345 INFO:tasks.workunit.client.1.smithi134.stdout: create 235/32 file after the snapshot 2024-03-09T08:55:32.345 INFO:tasks.workunit.client.1.smithi134.stdout: create 235/33 file after the snapshot 2024-03-09T08:55:32.346 INFO:tasks.workunit.client.1.smithi134.stdout: create 235/34 file after the snapshot 2024-03-09T08:55:32.346 INFO:tasks.workunit.client.1.smithi134.stdout: create 235/35 file after the snapshot 2024-03-09T08:55:32.346 INFO:tasks.workunit.client.1.smithi134.stdout: create 235/36 file after the snapshot 2024-03-09T08:55:32.347 INFO:tasks.workunit.client.1.smithi134.stdout: create 235/37 file after the snapshot 2024-03-09T08:55:32.347 INFO:tasks.workunit.client.1.smithi134.stdout: create 235/38 file after the snapshot 2024-03-09T08:55:32.348 INFO:tasks.workunit.client.1.smithi134.stdout: create 235/39 file after the snapshot 2024-03-09T08:55:32.348 INFO:tasks.workunit.client.1.smithi134.stdout: create 236/.snap/snap-subdir-test 2024-03-09T08:55:32.744 INFO:tasks.workunit.client.0.smithi067.stdout: create 200/31 file after the snapshot 2024-03-09T08:55:32.745 INFO:tasks.workunit.client.0.smithi067.stdout: create 200/32 file after the snapshot 2024-03-09T08:55:32.745 INFO:tasks.workunit.client.0.smithi067.stdout: create 200/33 file after the snapshot 2024-03-09T08:55:32.745 INFO:tasks.workunit.client.0.smithi067.stdout: create 200/34 file after the snapshot 2024-03-09T08:55:32.746 INFO:tasks.workunit.client.0.smithi067.stdout: create 200/35 file after the snapshot 2024-03-09T08:55:32.746 INFO:tasks.workunit.client.0.smithi067.stdout: create 200/36 file after the snapshot 2024-03-09T08:55:32.747 INFO:tasks.workunit.client.0.smithi067.stdout: create 200/37 file after the snapshot 2024-03-09T08:55:32.747 INFO:tasks.workunit.client.0.smithi067.stdout: create 200/38 file after the snapshot 2024-03-09T08:55:32.748 INFO:tasks.workunit.client.0.smithi067.stdout: create 200/39 file after the snapshot 2024-03-09T08:55:32.748 INFO:tasks.workunit.client.0.smithi067.stdout: create 201/.snap/snap-subdir-test 2024-03-09T08:55:32.805 INFO:tasks.workunit.client.1.smithi134.stdout: create 236/30 file after the snapshot 2024-03-09T08:55:32.807 INFO:tasks.workunit.client.1.smithi134.stdout: create 236/31 file after the snapshot 2024-03-09T08:55:32.808 INFO:tasks.workunit.client.1.smithi134.stdout: create 236/32 file after the snapshot 2024-03-09T08:55:32.808 INFO:tasks.workunit.client.1.smithi134.stdout: create 236/33 file after the snapshot 2024-03-09T08:55:32.808 INFO:tasks.workunit.client.1.smithi134.stdout: create 236/34 file after the snapshot 2024-03-09T08:55:32.809 INFO:tasks.workunit.client.1.smithi134.stdout: create 236/35 file after the snapshot 2024-03-09T08:55:32.809 INFO:tasks.workunit.client.1.smithi134.stdout: create 236/36 file after the snapshot 2024-03-09T08:55:32.809 INFO:tasks.workunit.client.1.smithi134.stdout: create 236/37 file after the snapshot 2024-03-09T08:55:32.810 INFO:tasks.workunit.client.1.smithi134.stdout: create 236/38 file after the snapshot 2024-03-09T08:55:32.810 INFO:tasks.workunit.client.1.smithi134.stdout: create 236/39 file after the snapshot 2024-03-09T08:55:32.810 INFO:tasks.workunit.client.1.smithi134.stdout: create 237/.snap/snap-subdir-test 2024-03-09T08:55:32.845 INFO:tasks.workunit.client.0.smithi067.stdout: create 201/30 file after the snapshot 2024-03-09T08:55:32.849 INFO:tasks.workunit.client.0.smithi067.stdout: create 201/31 file after the snapshot 2024-03-09T08:55:32.849 INFO:tasks.workunit.client.0.smithi067.stdout: create 201/32 file after the snapshot 2024-03-09T08:55:32.849 INFO:tasks.workunit.client.0.smithi067.stdout: create 201/33 file after the snapshot 2024-03-09T08:55:32.850 INFO:tasks.workunit.client.0.smithi067.stdout: create 201/34 file after the snapshot 2024-03-09T08:55:32.850 INFO:tasks.workunit.client.0.smithi067.stdout: create 201/35 file after the snapshot 2024-03-09T08:55:32.850 INFO:tasks.workunit.client.0.smithi067.stdout: create 201/36 file after the snapshot 2024-03-09T08:55:32.850 INFO:tasks.workunit.client.0.smithi067.stdout: create 201/37 file after the snapshot 2024-03-09T08:55:32.850 INFO:tasks.workunit.client.0.smithi067.stdout: create 201/38 file after the snapshot 2024-03-09T08:55:32.851 INFO:tasks.workunit.client.0.smithi067.stdout: create 201/39 file after the snapshot 2024-03-09T08:55:32.851 INFO:tasks.workunit.client.0.smithi067.stdout: create 202/.snap/snap-subdir-test 2024-03-09T08:55:32.890 INFO:tasks.workunit.client.1.smithi134.stdout: create 237/30 file after the snapshot 2024-03-09T08:55:32.894 INFO:tasks.workunit.client.1.smithi134.stdout: create 237/31 file after the snapshot 2024-03-09T08:55:32.894 INFO:tasks.workunit.client.1.smithi134.stdout: create 237/32 file after the snapshot 2024-03-09T08:55:32.894 INFO:tasks.workunit.client.1.smithi134.stdout: create 237/33 file after the snapshot 2024-03-09T08:55:32.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 237/34 file after the snapshot 2024-03-09T08:55:32.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 237/35 file after the snapshot 2024-03-09T08:55:32.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 237/36 file after the snapshot 2024-03-09T08:55:32.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 237/37 file after the snapshot 2024-03-09T08:55:32.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 237/38 file after the snapshot 2024-03-09T08:55:32.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 237/39 file after the snapshot 2024-03-09T08:55:32.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 238/.snap/snap-subdir-test 2024-03-09T08:55:32.940 INFO:tasks.workunit.client.0.smithi067.stdout: create 202/30 file after the snapshot 2024-03-09T08:55:32.942 INFO:tasks.workunit.client.0.smithi067.stdout: create 202/31 file after the snapshot 2024-03-09T08:55:32.943 INFO:tasks.workunit.client.0.smithi067.stdout: create 202/32 file after the snapshot 2024-03-09T08:55:32.943 INFO:tasks.workunit.client.0.smithi067.stdout: create 202/33 file after the snapshot 2024-03-09T08:55:32.943 INFO:tasks.workunit.client.0.smithi067.stdout: create 202/34 file after the snapshot 2024-03-09T08:55:32.944 INFO:tasks.workunit.client.0.smithi067.stdout: create 202/35 file after the snapshot 2024-03-09T08:55:32.944 INFO:tasks.workunit.client.0.smithi067.stdout: create 202/36 file after the snapshot 2024-03-09T08:55:32.944 INFO:tasks.workunit.client.0.smithi067.stdout: create 202/37 file after the snapshot 2024-03-09T08:55:32.945 INFO:tasks.workunit.client.0.smithi067.stdout: create 202/38 file after the snapshot 2024-03-09T08:55:32.945 INFO:tasks.workunit.client.0.smithi067.stdout: create 202/39 file after the snapshot 2024-03-09T08:55:32.946 INFO:tasks.workunit.client.0.smithi067.stdout: create 203/.snap/snap-subdir-test 2024-03-09T08:55:33.026 INFO:tasks.workunit.client.0.smithi067.stdout: create 203/30 file after the snapshot 2024-03-09T08:55:33.039 INFO:tasks.workunit.client.0.smithi067.stdout: create 203/31 file after the snapshot 2024-03-09T08:55:33.039 INFO:tasks.workunit.client.0.smithi067.stdout: create 203/32 file after the snapshot 2024-03-09T08:55:33.040 INFO:tasks.workunit.client.0.smithi067.stdout: create 203/33 file after the snapshot 2024-03-09T08:55:33.040 INFO:tasks.workunit.client.0.smithi067.stdout: create 203/34 file after the snapshot 2024-03-09T08:55:33.041 INFO:tasks.workunit.client.0.smithi067.stdout: create 203/35 file after the snapshot 2024-03-09T08:55:33.041 INFO:tasks.workunit.client.0.smithi067.stdout: create 203/36 file after the snapshot 2024-03-09T08:55:33.041 INFO:tasks.workunit.client.0.smithi067.stdout: create 203/37 file after the snapshot 2024-03-09T08:55:33.041 INFO:tasks.workunit.client.0.smithi067.stdout: create 203/38 file after the snapshot 2024-03-09T08:55:33.042 INFO:tasks.workunit.client.0.smithi067.stdout: create 203/39 file after the snapshot 2024-03-09T08:55:33.042 INFO:tasks.workunit.client.0.smithi067.stdout: create 204/.snap/snap-subdir-test 2024-03-09T08:55:33.119 INFO:tasks.workunit.client.0.smithi067.stdout: create 204/30 file after the snapshot 2024-03-09T08:55:33.122 INFO:tasks.workunit.client.0.smithi067.stdout: create 204/31 file after the snapshot 2024-03-09T08:55:33.122 INFO:tasks.workunit.client.0.smithi067.stdout: create 204/32 file after the snapshot 2024-03-09T08:55:33.123 INFO:tasks.workunit.client.0.smithi067.stdout: create 204/33 file after the snapshot 2024-03-09T08:55:33.123 INFO:tasks.workunit.client.0.smithi067.stdout: create 204/34 file after the snapshot 2024-03-09T08:55:33.124 INFO:tasks.workunit.client.0.smithi067.stdout: create 204/35 file after the snapshot 2024-03-09T08:55:33.124 INFO:tasks.workunit.client.0.smithi067.stdout: create 204/36 file after the snapshot 2024-03-09T08:55:33.124 INFO:tasks.workunit.client.0.smithi067.stdout: create 204/37 file after the snapshot 2024-03-09T08:55:33.125 INFO:tasks.workunit.client.0.smithi067.stdout: create 204/38 file after the snapshot 2024-03-09T08:55:33.125 INFO:tasks.workunit.client.0.smithi067.stdout: create 204/39 file after the snapshot 2024-03-09T08:55:33.126 INFO:tasks.workunit.client.0.smithi067.stdout: create 205/.snap/snap-subdir-test 2024-03-09T08:55:33.228 INFO:tasks.workunit.client.1.smithi134.stdout: create 238/30 file after the snapshot 2024-03-09T08:55:33.231 INFO:tasks.workunit.client.1.smithi134.stdout: create 238/31 file after the snapshot 2024-03-09T08:55:33.231 INFO:tasks.workunit.client.1.smithi134.stdout: create 238/32 file after the snapshot 2024-03-09T08:55:33.231 INFO:tasks.workunit.client.1.smithi134.stdout: create 238/33 file after the snapshot 2024-03-09T08:55:33.231 INFO:tasks.workunit.client.1.smithi134.stdout: create 238/34 file after the snapshot 2024-03-09T08:55:33.232 INFO:tasks.workunit.client.1.smithi134.stdout: create 238/35 file after the snapshot 2024-03-09T08:55:33.232 INFO:tasks.workunit.client.1.smithi134.stdout: create 238/36 file after the snapshot 2024-03-09T08:55:33.232 INFO:tasks.workunit.client.1.smithi134.stdout: create 238/37 file after the snapshot 2024-03-09T08:55:33.232 INFO:tasks.workunit.client.1.smithi134.stdout: create 238/38 file after the snapshot 2024-03-09T08:55:33.232 INFO:tasks.workunit.client.1.smithi134.stdout: create 238/39 file after the snapshot 2024-03-09T08:55:33.232 INFO:tasks.workunit.client.1.smithi134.stdout: create 239/.snap/snap-subdir-test 2024-03-09T08:55:33.312 INFO:tasks.workunit.client.1.smithi134.stdout: create 239/30 file after the snapshot 2024-03-09T08:55:33.315 INFO:tasks.workunit.client.1.smithi134.stdout: create 239/31 file after the snapshot 2024-03-09T08:55:33.316 INFO:tasks.workunit.client.1.smithi134.stdout: create 239/32 file after the snapshot 2024-03-09T08:55:33.316 INFO:tasks.workunit.client.1.smithi134.stdout: create 239/33 file after the snapshot 2024-03-09T08:55:33.316 INFO:tasks.workunit.client.1.smithi134.stdout: create 239/34 file after the snapshot 2024-03-09T08:55:33.316 INFO:tasks.workunit.client.1.smithi134.stdout: create 239/35 file after the snapshot 2024-03-09T08:55:33.317 INFO:tasks.workunit.client.1.smithi134.stdout: create 239/36 file after the snapshot 2024-03-09T08:55:33.317 INFO:tasks.workunit.client.1.smithi134.stdout: create 239/37 file after the snapshot 2024-03-09T08:55:33.317 INFO:tasks.workunit.client.1.smithi134.stdout: create 239/38 file after the snapshot 2024-03-09T08:55:33.318 INFO:tasks.workunit.client.1.smithi134.stdout: create 239/39 file after the snapshot 2024-03-09T08:55:33.318 INFO:tasks.workunit.client.1.smithi134.stdout: create 240/.snap/snap-subdir-test 2024-03-09T08:55:33.456 INFO:tasks.workunit.client.0.smithi067.stdout: create 205/30 file after the snapshot 2024-03-09T08:55:33.459 INFO:tasks.workunit.client.0.smithi067.stdout: create 205/31 file after the snapshot 2024-03-09T08:55:33.459 INFO:tasks.workunit.client.0.smithi067.stdout: create 205/32 file after the snapshot 2024-03-09T08:55:33.460 INFO:tasks.workunit.client.0.smithi067.stdout: create 205/33 file after the snapshot 2024-03-09T08:55:33.460 INFO:tasks.workunit.client.0.smithi067.stdout: create 205/34 file after the snapshot 2024-03-09T08:55:33.460 INFO:tasks.workunit.client.0.smithi067.stdout: create 205/35 file after the snapshot 2024-03-09T08:55:33.460 INFO:tasks.workunit.client.0.smithi067.stdout: create 205/36 file after the snapshot 2024-03-09T08:55:33.460 INFO:tasks.workunit.client.0.smithi067.stdout: create 205/37 file after the snapshot 2024-03-09T08:55:33.461 INFO:tasks.workunit.client.0.smithi067.stdout: create 205/38 file after the snapshot 2024-03-09T08:55:33.461 INFO:tasks.workunit.client.0.smithi067.stdout: create 205/39 file after the snapshot 2024-03-09T08:55:33.461 INFO:tasks.workunit.client.0.smithi067.stdout: create 206/.snap/snap-subdir-test 2024-03-09T08:55:33.539 INFO:tasks.workunit.client.0.smithi067.stdout: create 206/30 file after the snapshot 2024-03-09T08:55:33.542 INFO:tasks.workunit.client.0.smithi067.stdout: create 206/31 file after the snapshot 2024-03-09T08:55:33.542 INFO:tasks.workunit.client.0.smithi067.stdout: create 206/32 file after the snapshot 2024-03-09T08:55:33.543 INFO:tasks.workunit.client.0.smithi067.stdout: create 206/33 file after the snapshot 2024-03-09T08:55:33.543 INFO:tasks.workunit.client.0.smithi067.stdout: create 206/34 file after the snapshot 2024-03-09T08:55:33.543 INFO:tasks.workunit.client.0.smithi067.stdout: create 206/35 file after the snapshot 2024-03-09T08:55:33.544 INFO:tasks.workunit.client.0.smithi067.stdout: create 206/36 file after the snapshot 2024-03-09T08:55:33.544 INFO:tasks.workunit.client.0.smithi067.stdout: create 206/37 file after the snapshot 2024-03-09T08:55:33.544 INFO:tasks.workunit.client.0.smithi067.stdout: create 206/38 file after the snapshot 2024-03-09T08:55:33.545 INFO:tasks.workunit.client.0.smithi067.stdout: create 206/39 file after the snapshot 2024-03-09T08:55:33.545 INFO:tasks.workunit.client.0.smithi067.stdout: create 207/.snap/snap-subdir-test 2024-03-09T08:55:33.629 INFO:tasks.workunit.client.0.smithi067.stdout: create 207/30 file after the snapshot 2024-03-09T08:55:33.632 INFO:tasks.workunit.client.0.smithi067.stdout: create 207/31 file after the snapshot 2024-03-09T08:55:33.633 INFO:tasks.workunit.client.0.smithi067.stdout: create 207/32 file after the snapshot 2024-03-09T08:55:33.633 INFO:tasks.workunit.client.0.smithi067.stdout: create 207/33 file after the snapshot 2024-03-09T08:55:33.633 INFO:tasks.workunit.client.0.smithi067.stdout: create 207/34 file after the snapshot 2024-03-09T08:55:33.633 INFO:tasks.workunit.client.0.smithi067.stdout: create 207/35 file after the snapshot 2024-03-09T08:55:33.634 INFO:tasks.workunit.client.0.smithi067.stdout: create 207/36 file after the snapshot 2024-03-09T08:55:33.634 INFO:tasks.workunit.client.0.smithi067.stdout: create 207/37 file after the snapshot 2024-03-09T08:55:33.634 INFO:tasks.workunit.client.0.smithi067.stdout: create 207/38 file after the snapshot 2024-03-09T08:55:33.634 INFO:tasks.workunit.client.0.smithi067.stdout: create 207/39 file after the snapshot 2024-03-09T08:55:33.634 INFO:tasks.workunit.client.0.smithi067.stdout: create 208/.snap/snap-subdir-test 2024-03-09T08:55:33.650 INFO:tasks.workunit.client.1.smithi134.stdout: create 240/30 file after the snapshot 2024-03-09T08:55:33.651 INFO:tasks.workunit.client.1.smithi134.stdout: create 240/31 file after the snapshot 2024-03-09T08:55:33.652 INFO:tasks.workunit.client.1.smithi134.stdout: create 240/32 file after the snapshot 2024-03-09T08:55:33.652 INFO:tasks.workunit.client.1.smithi134.stdout: create 240/33 file after the snapshot 2024-03-09T08:55:33.653 INFO:tasks.workunit.client.1.smithi134.stdout: create 240/34 file after the snapshot 2024-03-09T08:55:33.653 INFO:tasks.workunit.client.1.smithi134.stdout: create 240/35 file after the snapshot 2024-03-09T08:55:33.653 INFO:tasks.workunit.client.1.smithi134.stdout: create 240/36 file after the snapshot 2024-03-09T08:55:33.653 INFO:tasks.workunit.client.1.smithi134.stdout: create 240/37 file after the snapshot 2024-03-09T08:55:33.653 INFO:tasks.workunit.client.1.smithi134.stdout: create 240/38 file after the snapshot 2024-03-09T08:55:33.653 INFO:tasks.workunit.client.1.smithi134.stdout: create 240/39 file after the snapshot 2024-03-09T08:55:33.653 INFO:tasks.workunit.client.1.smithi134.stdout: create 241/.snap/snap-subdir-test 2024-03-09T08:55:33.838 INFO:tasks.workunit.client.1.smithi134.stdout: create 241/30 file after the snapshot 2024-03-09T08:55:33.841 INFO:tasks.workunit.client.1.smithi134.stdout: create 241/31 file after the snapshot 2024-03-09T08:55:33.841 INFO:tasks.workunit.client.1.smithi134.stdout: create 241/32 file after the snapshot 2024-03-09T08:55:33.842 INFO:tasks.workunit.client.1.smithi134.stdout: create 241/33 file after the snapshot 2024-03-09T08:55:33.842 INFO:tasks.workunit.client.1.smithi134.stdout: create 241/34 file after the snapshot 2024-03-09T08:55:33.842 INFO:tasks.workunit.client.1.smithi134.stdout: create 241/35 file after the snapshot 2024-03-09T08:55:33.843 INFO:tasks.workunit.client.1.smithi134.stdout: create 241/36 file after the snapshot 2024-03-09T08:55:33.843 INFO:tasks.workunit.client.1.smithi134.stdout: create 241/37 file after the snapshot 2024-03-09T08:55:33.843 INFO:tasks.workunit.client.1.smithi134.stdout: create 241/38 file after the snapshot 2024-03-09T08:55:33.844 INFO:tasks.workunit.client.1.smithi134.stdout: create 241/39 file after the snapshot 2024-03-09T08:55:33.844 INFO:tasks.workunit.client.1.smithi134.stdout: create 242/.snap/snap-subdir-test 2024-03-09T08:55:33.924 INFO:tasks.workunit.client.1.smithi134.stdout: create 242/30 file after the snapshot 2024-03-09T08:55:33.927 INFO:tasks.workunit.client.1.smithi134.stdout: create 242/31 file after the snapshot 2024-03-09T08:55:33.928 INFO:tasks.workunit.client.1.smithi134.stdout: create 242/32 file after the snapshot 2024-03-09T08:55:33.928 INFO:tasks.workunit.client.1.smithi134.stdout: create 242/33 file after the snapshot 2024-03-09T08:55:33.928 INFO:tasks.workunit.client.1.smithi134.stdout: create 242/34 file after the snapshot 2024-03-09T08:55:33.928 INFO:tasks.workunit.client.1.smithi134.stdout: create 242/35 file after the snapshot 2024-03-09T08:55:33.929 INFO:tasks.workunit.client.1.smithi134.stdout: create 242/36 file after the snapshot 2024-03-09T08:55:33.929 INFO:tasks.workunit.client.1.smithi134.stdout: create 242/37 file after the snapshot 2024-03-09T08:55:33.929 INFO:tasks.workunit.client.1.smithi134.stdout: create 242/38 file after the snapshot 2024-03-09T08:55:33.929 INFO:tasks.workunit.client.1.smithi134.stdout: create 242/39 file after the snapshot 2024-03-09T08:55:33.929 INFO:tasks.workunit.client.1.smithi134.stdout: create 243/.snap/snap-subdir-test 2024-03-09T08:55:33.933 INFO:tasks.workunit.client.0.smithi067.stdout: create 208/30 file after the snapshot 2024-03-09T08:55:33.935 INFO:tasks.workunit.client.0.smithi067.stdout: create 208/31 file after the snapshot 2024-03-09T08:55:33.936 INFO:tasks.workunit.client.0.smithi067.stdout: create 208/32 file after the snapshot 2024-03-09T08:55:33.936 INFO:tasks.workunit.client.0.smithi067.stdout: create 208/33 file after the snapshot 2024-03-09T08:55:33.937 INFO:tasks.workunit.client.0.smithi067.stdout: create 208/34 file after the snapshot 2024-03-09T08:55:33.937 INFO:tasks.workunit.client.0.smithi067.stdout: create 208/35 file after the snapshot 2024-03-09T08:55:33.938 INFO:tasks.workunit.client.0.smithi067.stdout: create 208/36 file after the snapshot 2024-03-09T08:55:33.939 INFO:tasks.workunit.client.0.smithi067.stdout: create 208/37 file after the snapshot 2024-03-09T08:55:33.940 INFO:tasks.workunit.client.0.smithi067.stdout: create 208/38 file after the snapshot 2024-03-09T08:55:33.940 INFO:tasks.workunit.client.0.smithi067.stdout: create 208/39 file after the snapshot 2024-03-09T08:55:33.941 INFO:tasks.workunit.client.0.smithi067.stdout: create 209/.snap/snap-subdir-test 2024-03-09T08:55:33.992 INFO:tasks.workunit.client.1.smithi134.stdout: create 243/30 file after the snapshot 2024-03-09T08:55:33.995 INFO:tasks.workunit.client.1.smithi134.stdout: create 243/31 file after the snapshot 2024-03-09T08:55:33.995 INFO:tasks.workunit.client.1.smithi134.stdout: create 243/32 file after the snapshot 2024-03-09T08:55:33.995 INFO:tasks.workunit.client.1.smithi134.stdout: create 243/33 file after the snapshot 2024-03-09T08:55:33.995 INFO:tasks.workunit.client.1.smithi134.stdout: create 243/34 file after the snapshot 2024-03-09T08:55:33.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 243/35 file after the snapshot 2024-03-09T08:55:33.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 243/36 file after the snapshot 2024-03-09T08:55:33.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 243/37 file after the snapshot 2024-03-09T08:55:33.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 243/38 file after the snapshot 2024-03-09T08:55:33.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 243/39 file after the snapshot 2024-03-09T08:55:33.996 INFO:tasks.workunit.client.1.smithi134.stdout: create 244/.snap/snap-subdir-test 2024-03-09T08:55:34.022 INFO:tasks.workunit.client.0.smithi067.stdout: create 209/30 file after the snapshot 2024-03-09T08:55:34.024 INFO:tasks.workunit.client.0.smithi067.stdout: create 209/31 file after the snapshot 2024-03-09T08:55:34.025 INFO:tasks.workunit.client.0.smithi067.stdout: create 209/32 file after the snapshot 2024-03-09T08:55:34.025 INFO:tasks.workunit.client.0.smithi067.stdout: create 209/33 file after the snapshot 2024-03-09T08:55:34.026 INFO:tasks.workunit.client.0.smithi067.stdout: create 209/34 file after the snapshot 2024-03-09T08:55:34.026 INFO:tasks.workunit.client.0.smithi067.stdout: create 209/35 file after the snapshot 2024-03-09T08:55:34.027 INFO:tasks.workunit.client.0.smithi067.stdout: create 209/36 file after the snapshot 2024-03-09T08:55:34.027 INFO:tasks.workunit.client.0.smithi067.stdout: create 209/37 file after the snapshot 2024-03-09T08:55:34.029 INFO:tasks.workunit.client.0.smithi067.stdout: create 209/38 file after the snapshot 2024-03-09T08:55:34.029 INFO:tasks.workunit.client.0.smithi067.stdout: create 209/39 file after the snapshot 2024-03-09T08:55:34.030 INFO:tasks.workunit.client.0.smithi067.stdout: create 210/.snap/snap-subdir-test 2024-03-09T08:55:34.083 INFO:tasks.workunit.client.1.smithi134.stdout: create 244/30 file after the snapshot 2024-03-09T08:55:34.086 INFO:tasks.workunit.client.1.smithi134.stdout: create 244/31 file after the snapshot 2024-03-09T08:55:34.086 INFO:tasks.workunit.client.1.smithi134.stdout: create 244/32 file after the snapshot 2024-03-09T08:55:34.086 INFO:tasks.workunit.client.1.smithi134.stdout: create 244/33 file after the snapshot 2024-03-09T08:55:34.086 INFO:tasks.workunit.client.1.smithi134.stdout: create 244/34 file after the snapshot 2024-03-09T08:55:34.087 INFO:tasks.workunit.client.1.smithi134.stdout: create 244/35 file after the snapshot 2024-03-09T08:55:34.087 INFO:tasks.workunit.client.1.smithi134.stdout: create 244/36 file after the snapshot 2024-03-09T08:55:34.087 INFO:tasks.workunit.client.1.smithi134.stdout: create 244/37 file after the snapshot 2024-03-09T08:55:34.088 INFO:tasks.workunit.client.1.smithi134.stdout: create 244/38 file after the snapshot 2024-03-09T08:55:34.088 INFO:tasks.workunit.client.1.smithi134.stdout: create 244/39 file after the snapshot 2024-03-09T08:55:34.088 INFO:tasks.workunit.client.1.smithi134.stdout: create 245/.snap/snap-subdir-test 2024-03-09T08:55:34.102 INFO:tasks.workunit.client.0.smithi067.stdout: create 210/30 file after the snapshot 2024-03-09T08:55:34.103 INFO:tasks.workunit.client.0.smithi067.stdout: create 210/31 file after the snapshot 2024-03-09T08:55:34.104 INFO:tasks.workunit.client.0.smithi067.stdout: create 210/32 file after the snapshot 2024-03-09T08:55:34.104 INFO:tasks.workunit.client.0.smithi067.stdout: create 210/33 file after the snapshot 2024-03-09T08:55:34.104 INFO:tasks.workunit.client.0.smithi067.stdout: create 210/34 file after the snapshot 2024-03-09T08:55:34.105 INFO:tasks.workunit.client.0.smithi067.stdout: create 210/35 file after the snapshot 2024-03-09T08:55:34.105 INFO:tasks.workunit.client.0.smithi067.stdout: create 210/36 file after the snapshot 2024-03-09T08:55:34.105 INFO:tasks.workunit.client.0.smithi067.stdout: create 210/37 file after the snapshot 2024-03-09T08:55:34.105 INFO:tasks.workunit.client.0.smithi067.stdout: create 210/38 file after the snapshot 2024-03-09T08:55:34.106 INFO:tasks.workunit.client.0.smithi067.stdout: create 210/39 file after the snapshot 2024-03-09T08:55:34.106 INFO:tasks.workunit.client.0.smithi067.stdout: create 211/.snap/snap-subdir-test 2024-03-09T08:55:34.166 INFO:tasks.workunit.client.1.smithi134.stdout: create 245/30 file after the snapshot 2024-03-09T08:55:34.169 INFO:tasks.workunit.client.1.smithi134.stdout: create 245/31 file after the snapshot 2024-03-09T08:55:34.170 INFO:tasks.workunit.client.1.smithi134.stdout: create 245/32 file after the snapshot 2024-03-09T08:55:34.170 INFO:tasks.workunit.client.1.smithi134.stdout: create 245/33 file after the snapshot 2024-03-09T08:55:34.170 INFO:tasks.workunit.client.1.smithi134.stdout: create 245/34 file after the snapshot 2024-03-09T08:55:34.170 INFO:tasks.workunit.client.1.smithi134.stdout: create 245/35 file after the snapshot 2024-03-09T08:55:34.172 INFO:tasks.workunit.client.1.smithi134.stdout: create 245/36 file after the snapshot 2024-03-09T08:55:34.172 INFO:tasks.workunit.client.1.smithi134.stdout: create 245/37 file after the snapshot 2024-03-09T08:55:34.172 INFO:tasks.workunit.client.1.smithi134.stdout: create 245/38 file after the snapshot 2024-03-09T08:55:34.172 INFO:tasks.workunit.client.1.smithi134.stdout: create 245/39 file after the snapshot 2024-03-09T08:55:34.172 INFO:tasks.workunit.client.1.smithi134.stdout: create 246/.snap/snap-subdir-test 2024-03-09T08:55:34.248 INFO:tasks.workunit.client.1.smithi134.stdout: create 246/30 file after the snapshot 2024-03-09T08:55:34.251 INFO:tasks.workunit.client.1.smithi134.stdout: create 246/31 file after the snapshot 2024-03-09T08:55:34.252 INFO:tasks.workunit.client.1.smithi134.stdout: create 246/32 file after the snapshot 2024-03-09T08:55:34.252 INFO:tasks.workunit.client.1.smithi134.stdout: create 246/33 file after the snapshot 2024-03-09T08:55:34.252 INFO:tasks.workunit.client.1.smithi134.stdout: create 246/34 file after the snapshot 2024-03-09T08:55:34.252 INFO:tasks.workunit.client.1.smithi134.stdout: create 246/35 file after the snapshot 2024-03-09T08:55:34.252 INFO:tasks.workunit.client.1.smithi134.stdout: create 246/36 file after the snapshot 2024-03-09T08:55:34.253 INFO:tasks.workunit.client.1.smithi134.stdout: create 246/37 file after the snapshot 2024-03-09T08:55:34.253 INFO:tasks.workunit.client.1.smithi134.stdout: create 246/38 file after the snapshot 2024-03-09T08:55:34.253 INFO:tasks.workunit.client.1.smithi134.stdout: create 246/39 file after the snapshot 2024-03-09T08:55:34.254 INFO:tasks.workunit.client.1.smithi134.stdout: create 247/.snap/snap-subdir-test 2024-03-09T08:55:34.330 INFO:tasks.workunit.client.1.smithi134.stdout: create 247/30 file after the snapshot 2024-03-09T08:55:34.333 INFO:tasks.workunit.client.1.smithi134.stdout: create 247/31 file after the snapshot 2024-03-09T08:55:34.333 INFO:tasks.workunit.client.1.smithi134.stdout: create 247/32 file after the snapshot 2024-03-09T08:55:34.333 INFO:tasks.workunit.client.1.smithi134.stdout: create 247/33 file after the snapshot 2024-03-09T08:55:34.334 INFO:tasks.workunit.client.1.smithi134.stdout: create 247/34 file after the snapshot 2024-03-09T08:55:34.334 INFO:tasks.workunit.client.1.smithi134.stdout: create 247/35 file after the snapshot 2024-03-09T08:55:34.334 INFO:tasks.workunit.client.1.smithi134.stdout: create 247/36 file after the snapshot 2024-03-09T08:55:34.334 INFO:tasks.workunit.client.1.smithi134.stdout: create 247/37 file after the snapshot 2024-03-09T08:55:34.335 INFO:tasks.workunit.client.1.smithi134.stdout: create 247/38 file after the snapshot 2024-03-09T08:55:34.335 INFO:tasks.workunit.client.1.smithi134.stdout: create 247/39 file after the snapshot 2024-03-09T08:55:34.335 INFO:tasks.workunit.client.1.smithi134.stdout: create 248/.snap/snap-subdir-test 2024-03-09T08:55:34.408 INFO:tasks.workunit.client.1.smithi134.stdout: create 248/30 file after the snapshot 2024-03-09T08:55:34.411 INFO:tasks.workunit.client.1.smithi134.stdout: create 248/31 file after the snapshot 2024-03-09T08:55:34.412 INFO:tasks.workunit.client.1.smithi134.stdout: create 248/32 file after the snapshot 2024-03-09T08:55:34.412 INFO:tasks.workunit.client.1.smithi134.stdout: create 248/33 file after the snapshot 2024-03-09T08:55:34.412 INFO:tasks.workunit.client.1.smithi134.stdout: create 248/34 file after the snapshot 2024-03-09T08:55:34.413 INFO:tasks.workunit.client.1.smithi134.stdout: create 248/35 file after the snapshot 2024-03-09T08:55:34.413 INFO:tasks.workunit.client.1.smithi134.stdout: create 248/36 file after the snapshot 2024-03-09T08:55:34.413 INFO:tasks.workunit.client.1.smithi134.stdout: create 248/37 file after the snapshot 2024-03-09T08:55:34.414 INFO:tasks.workunit.client.1.smithi134.stdout: create 248/38 file after the snapshot 2024-03-09T08:55:34.414 INFO:tasks.workunit.client.1.smithi134.stdout: create 248/39 file after the snapshot 2024-03-09T08:55:34.414 INFO:tasks.workunit.client.1.smithi134.stdout: create 249/.snap/snap-subdir-test 2024-03-09T08:55:34.495 INFO:tasks.workunit.client.1.smithi134.stdout: create 249/30 file after the snapshot 2024-03-09T08:55:34.498 INFO:tasks.workunit.client.1.smithi134.stdout: create 249/31 file after the snapshot 2024-03-09T08:55:34.498 INFO:tasks.workunit.client.1.smithi134.stdout: create 249/32 file after the snapshot 2024-03-09T08:55:34.498 INFO:tasks.workunit.client.1.smithi134.stdout: create 249/33 file after the snapshot 2024-03-09T08:55:34.499 INFO:tasks.workunit.client.1.smithi134.stdout: create 249/34 file after the snapshot 2024-03-09T08:55:34.499 INFO:tasks.workunit.client.1.smithi134.stdout: create 249/35 file after the snapshot 2024-03-09T08:55:34.500 INFO:tasks.workunit.client.1.smithi134.stdout: create 249/36 file after the snapshot 2024-03-09T08:55:34.500 INFO:tasks.workunit.client.1.smithi134.stdout: create 249/37 file after the snapshot 2024-03-09T08:55:34.501 INFO:tasks.workunit.client.1.smithi134.stdout: create 249/38 file after the snapshot 2024-03-09T08:55:34.501 INFO:tasks.workunit.client.1.smithi134.stdout: create 249/39 file after the snapshot 2024-03-09T08:55:34.502 INFO:tasks.workunit.client.1.smithi134.stdout: create 250/.snap/snap-subdir-test 2024-03-09T08:55:34.574 INFO:tasks.workunit.client.1.smithi134.stdout: create 250/30 file after the snapshot 2024-03-09T08:55:34.577 INFO:tasks.workunit.client.1.smithi134.stdout: create 250/31 file after the snapshot 2024-03-09T08:55:34.577 INFO:tasks.workunit.client.1.smithi134.stdout: create 250/32 file after the snapshot 2024-03-09T08:55:34.577 INFO:tasks.workunit.client.1.smithi134.stdout: create 250/33 file after the snapshot 2024-03-09T08:55:34.578 INFO:tasks.workunit.client.1.smithi134.stdout: create 250/34 file after the snapshot 2024-03-09T08:55:34.578 INFO:tasks.workunit.client.1.smithi134.stdout: create 250/35 file after the snapshot 2024-03-09T08:55:34.579 INFO:tasks.workunit.client.1.smithi134.stdout: create 250/36 file after the snapshot 2024-03-09T08:55:34.579 INFO:tasks.workunit.client.1.smithi134.stdout: create 250/37 file after the snapshot 2024-03-09T08:55:34.580 INFO:tasks.workunit.client.1.smithi134.stdout: create 250/38 file after the snapshot 2024-03-09T08:55:34.581 INFO:tasks.workunit.client.1.smithi134.stdout: create 250/39 file after the snapshot 2024-03-09T08:55:34.581 INFO:tasks.workunit.client.1.smithi134.stdout: create 251/.snap/snap-subdir-test 2024-03-09T08:55:34.662 INFO:tasks.workunit.client.1.smithi134.stdout: create 251/30 file after the snapshot 2024-03-09T08:55:34.665 INFO:tasks.workunit.client.1.smithi134.stdout: create 251/31 file after the snapshot 2024-03-09T08:55:34.665 INFO:tasks.workunit.client.1.smithi134.stdout: create 251/32 file after the snapshot 2024-03-09T08:55:34.666 INFO:tasks.workunit.client.1.smithi134.stdout: create 251/33 file after the snapshot 2024-03-09T08:55:34.666 INFO:tasks.workunit.client.1.smithi134.stdout: create 251/34 file after the snapshot 2024-03-09T08:55:34.666 INFO:tasks.workunit.client.1.smithi134.stdout: create 251/35 file after the snapshot 2024-03-09T08:55:34.666 INFO:tasks.workunit.client.1.smithi134.stdout: create 251/36 file after the snapshot 2024-03-09T08:55:34.667 INFO:tasks.workunit.client.1.smithi134.stdout: create 251/37 file after the snapshot 2024-03-09T08:55:34.667 INFO:tasks.workunit.client.1.smithi134.stdout: create 251/38 file after the snapshot 2024-03-09T08:55:34.668 INFO:tasks.workunit.client.1.smithi134.stdout: create 251/39 file after the snapshot 2024-03-09T08:55:34.668 INFO:tasks.workunit.client.1.smithi134.stdout: create 252/.snap/snap-subdir-test 2024-03-09T08:55:34.751 INFO:tasks.workunit.client.0.smithi067.stdout: create 211/30 file after the snapshot 2024-03-09T08:55:34.754 INFO:tasks.workunit.client.0.smithi067.stdout: create 211/31 file after the snapshot 2024-03-09T08:55:34.754 INFO:tasks.workunit.client.0.smithi067.stdout: create 211/32 file after the snapshot 2024-03-09T08:55:34.755 INFO:tasks.workunit.client.0.smithi067.stdout: create 211/33 file after the snapshot 2024-03-09T08:55:34.756 INFO:tasks.workunit.client.0.smithi067.stdout: create 211/34 file after the snapshot 2024-03-09T08:55:34.756 INFO:tasks.workunit.client.0.smithi067.stdout: create 211/35 file after the snapshot 2024-03-09T08:55:34.757 INFO:tasks.workunit.client.0.smithi067.stdout: create 211/36 file after the snapshot 2024-03-09T08:55:34.757 INFO:tasks.workunit.client.0.smithi067.stdout: create 211/37 file after the snapshot 2024-03-09T08:55:34.758 INFO:tasks.workunit.client.0.smithi067.stdout: create 211/38 file after the snapshot 2024-03-09T08:55:34.758 INFO:tasks.workunit.client.0.smithi067.stdout: create 211/39 file after the snapshot 2024-03-09T08:55:34.759 INFO:tasks.workunit.client.0.smithi067.stdout: create 212/.snap/snap-subdir-test 2024-03-09T08:55:34.829 INFO:tasks.workunit.client.0.smithi067.stdout: create 212/30 file after the snapshot 2024-03-09T08:55:34.833 INFO:tasks.workunit.client.0.smithi067.stdout: create 212/31 file after the snapshot 2024-03-09T08:55:34.834 INFO:tasks.workunit.client.0.smithi067.stdout: create 212/32 file after the snapshot 2024-03-09T08:55:34.834 INFO:tasks.workunit.client.0.smithi067.stdout: create 212/33 file after the snapshot 2024-03-09T08:55:34.835 INFO:tasks.workunit.client.0.smithi067.stdout: create 212/34 file after the snapshot 2024-03-09T08:55:34.835 INFO:tasks.workunit.client.0.smithi067.stdout: create 212/35 file after the snapshot 2024-03-09T08:55:34.836 INFO:tasks.workunit.client.0.smithi067.stdout: create 212/36 file after the snapshot 2024-03-09T08:55:34.836 INFO:tasks.workunit.client.0.smithi067.stdout: create 212/37 file after the snapshot 2024-03-09T08:55:34.837 INFO:tasks.workunit.client.0.smithi067.stdout: create 212/38 file after the snapshot 2024-03-09T08:55:34.837 INFO:tasks.workunit.client.0.smithi067.stdout: create 212/39 file after the snapshot 2024-03-09T08:55:34.837 INFO:tasks.workunit.client.0.smithi067.stdout: create 213/.snap/snap-subdir-test 2024-03-09T08:55:34.931 INFO:tasks.workunit.client.0.smithi067.stdout: create 213/30 file after the snapshot 2024-03-09T08:55:34.934 INFO:tasks.workunit.client.0.smithi067.stdout: create 213/31 file after the snapshot 2024-03-09T08:55:34.935 INFO:tasks.workunit.client.0.smithi067.stdout: create 213/32 file after the snapshot 2024-03-09T08:55:34.935 INFO:tasks.workunit.client.0.smithi067.stdout: create 213/33 file after the snapshot 2024-03-09T08:55:34.935 INFO:tasks.workunit.client.0.smithi067.stdout: create 213/34 file after the snapshot 2024-03-09T08:55:34.935 INFO:tasks.workunit.client.0.smithi067.stdout: create 213/35 file after the snapshot 2024-03-09T08:55:34.936 INFO:tasks.workunit.client.0.smithi067.stdout: create 213/36 file after the snapshot 2024-03-09T08:55:34.936 INFO:tasks.workunit.client.0.smithi067.stdout: create 213/37 file after the snapshot 2024-03-09T08:55:34.936 INFO:tasks.workunit.client.0.smithi067.stdout: create 213/38 file after the snapshot 2024-03-09T08:55:34.936 INFO:tasks.workunit.client.0.smithi067.stdout: create 213/39 file after the snapshot 2024-03-09T08:55:34.936 INFO:tasks.workunit.client.0.smithi067.stdout: create 214/.snap/snap-subdir-test 2024-03-09T08:55:35.014 INFO:tasks.workunit.client.0.smithi067.stdout: create 214/30 file after the snapshot 2024-03-09T08:55:35.016 INFO:tasks.workunit.client.0.smithi067.stdout: create 214/31 file after the snapshot 2024-03-09T08:55:35.017 INFO:tasks.workunit.client.0.smithi067.stdout: create 214/32 file after the snapshot 2024-03-09T08:55:35.017 INFO:tasks.workunit.client.0.smithi067.stdout: create 214/33 file after the snapshot 2024-03-09T08:55:35.018 INFO:tasks.workunit.client.0.smithi067.stdout: create 214/34 file after the snapshot 2024-03-09T08:55:35.018 INFO:tasks.workunit.client.0.smithi067.stdout: create 214/35 file after the snapshot 2024-03-09T08:55:35.018 INFO:tasks.workunit.client.0.smithi067.stdout: create 214/36 file after the snapshot 2024-03-09T08:55:35.019 INFO:tasks.workunit.client.0.smithi067.stdout: create 214/37 file after the snapshot 2024-03-09T08:55:35.019 INFO:tasks.workunit.client.0.smithi067.stdout: create 214/38 file after the snapshot 2024-03-09T08:55:35.020 INFO:tasks.workunit.client.0.smithi067.stdout: create 214/39 file after the snapshot 2024-03-09T08:55:35.021 INFO:tasks.workunit.client.0.smithi067.stdout: create 215/.snap/snap-subdir-test 2024-03-09T08:55:35.090 INFO:tasks.workunit.client.0.smithi067.stdout: create 215/30 file after the snapshot 2024-03-09T08:55:35.092 INFO:tasks.workunit.client.0.smithi067.stdout: create 215/31 file after the snapshot 2024-03-09T08:55:35.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 215/32 file after the snapshot 2024-03-09T08:55:35.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 215/33 file after the snapshot 2024-03-09T08:55:35.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 215/34 file after the snapshot 2024-03-09T08:55:35.094 INFO:tasks.workunit.client.0.smithi067.stdout: create 215/35 file after the snapshot 2024-03-09T08:55:35.094 INFO:tasks.workunit.client.0.smithi067.stdout: create 215/36 file after the snapshot 2024-03-09T08:55:35.094 INFO:tasks.workunit.client.0.smithi067.stdout: create 215/37 file after the snapshot 2024-03-09T08:55:35.095 INFO:tasks.workunit.client.0.smithi067.stdout: create 215/38 file after the snapshot 2024-03-09T08:55:35.095 INFO:tasks.workunit.client.0.smithi067.stdout: create 215/39 file after the snapshot 2024-03-09T08:55:35.095 INFO:tasks.workunit.client.0.smithi067.stdout: create 216/.snap/snap-subdir-test 2024-03-09T08:55:35.186 INFO:tasks.workunit.client.0.smithi067.stdout: create 216/30 file after the snapshot 2024-03-09T08:55:35.189 INFO:tasks.workunit.client.0.smithi067.stdout: create 216/31 file after the snapshot 2024-03-09T08:55:35.190 INFO:tasks.workunit.client.0.smithi067.stdout: create 216/32 file after the snapshot 2024-03-09T08:55:35.190 INFO:tasks.workunit.client.0.smithi067.stdout: create 216/33 file after the snapshot 2024-03-09T08:55:35.191 INFO:tasks.workunit.client.0.smithi067.stdout: create 216/34 file after the snapshot 2024-03-09T08:55:35.192 INFO:tasks.workunit.client.0.smithi067.stdout: create 216/35 file after the snapshot 2024-03-09T08:55:35.192 INFO:tasks.workunit.client.0.smithi067.stdout: create 216/36 file after the snapshot 2024-03-09T08:55:35.193 INFO:tasks.workunit.client.0.smithi067.stdout: create 216/37 file after the snapshot 2024-03-09T08:55:35.193 INFO:tasks.workunit.client.0.smithi067.stdout: create 216/38 file after the snapshot 2024-03-09T08:55:35.193 INFO:tasks.workunit.client.0.smithi067.stdout: create 216/39 file after the snapshot 2024-03-09T08:55:35.194 INFO:tasks.workunit.client.0.smithi067.stdout: create 217/.snap/snap-subdir-test 2024-03-09T08:55:35.281 INFO:tasks.workunit.client.0.smithi067.stdout: create 217/30 file after the snapshot 2024-03-09T08:55:35.284 INFO:tasks.workunit.client.0.smithi067.stdout: create 217/31 file after the snapshot 2024-03-09T08:55:35.284 INFO:tasks.workunit.client.0.smithi067.stdout: create 217/32 file after the snapshot 2024-03-09T08:55:35.285 INFO:tasks.workunit.client.0.smithi067.stdout: create 217/33 file after the snapshot 2024-03-09T08:55:35.285 INFO:tasks.workunit.client.0.smithi067.stdout: create 217/34 file after the snapshot 2024-03-09T08:55:35.285 INFO:tasks.workunit.client.0.smithi067.stdout: create 217/35 file after the snapshot 2024-03-09T08:55:35.286 INFO:tasks.workunit.client.0.smithi067.stdout: create 217/36 file after the snapshot 2024-03-09T08:55:35.286 INFO:tasks.workunit.client.0.smithi067.stdout: create 217/37 file after the snapshot 2024-03-09T08:55:35.287 INFO:tasks.workunit.client.0.smithi067.stdout: create 217/38 file after the snapshot 2024-03-09T08:55:35.288 INFO:tasks.workunit.client.0.smithi067.stdout: create 217/39 file after the snapshot 2024-03-09T08:55:35.288 INFO:tasks.workunit.client.0.smithi067.stdout: create 218/.snap/snap-subdir-test 2024-03-09T08:55:35.359 INFO:tasks.workunit.client.0.smithi067.stdout: create 218/30 file after the snapshot 2024-03-09T08:55:35.363 INFO:tasks.workunit.client.0.smithi067.stdout: create 218/31 file after the snapshot 2024-03-09T08:55:35.363 INFO:tasks.workunit.client.0.smithi067.stdout: create 218/32 file after the snapshot 2024-03-09T08:55:35.363 INFO:tasks.workunit.client.0.smithi067.stdout: create 218/33 file after the snapshot 2024-03-09T08:55:35.364 INFO:tasks.workunit.client.0.smithi067.stdout: create 218/34 file after the snapshot 2024-03-09T08:55:35.364 INFO:tasks.workunit.client.0.smithi067.stdout: create 218/35 file after the snapshot 2024-03-09T08:55:35.365 INFO:tasks.workunit.client.0.smithi067.stdout: create 218/36 file after the snapshot 2024-03-09T08:55:35.365 INFO:tasks.workunit.client.0.smithi067.stdout: create 218/37 file after the snapshot 2024-03-09T08:55:35.365 INFO:tasks.workunit.client.0.smithi067.stdout: create 218/38 file after the snapshot 2024-03-09T08:55:35.365 INFO:tasks.workunit.client.0.smithi067.stdout: create 218/39 file after the snapshot 2024-03-09T08:55:35.365 INFO:tasks.workunit.client.0.smithi067.stdout: create 219/.snap/snap-subdir-test 2024-03-09T08:55:35.443 INFO:tasks.workunit.client.1.smithi134.stdout: create 252/30 file after the snapshot 2024-03-09T08:55:35.446 INFO:tasks.workunit.client.1.smithi134.stdout: create 252/31 file after the snapshot 2024-03-09T08:55:35.447 INFO:tasks.workunit.client.1.smithi134.stdout: create 252/32 file after the snapshot 2024-03-09T08:55:35.447 INFO:tasks.workunit.client.1.smithi134.stdout: create 252/33 file after the snapshot 2024-03-09T08:55:35.447 INFO:tasks.workunit.client.1.smithi134.stdout: create 252/34 file after the snapshot 2024-03-09T08:55:35.447 INFO:tasks.workunit.client.0.smithi067.stdout: create 219/30 file after the snapshot 2024-03-09T08:55:35.448 INFO:tasks.workunit.client.1.smithi134.stdout: create 252/35 file after the snapshot 2024-03-09T08:55:35.448 INFO:tasks.workunit.client.1.smithi134.stdout: create 252/36 file after the snapshot 2024-03-09T08:55:35.448 INFO:tasks.workunit.client.1.smithi134.stdout: create 252/37 file after the snapshot 2024-03-09T08:55:35.448 INFO:tasks.workunit.client.1.smithi134.stdout: create 252/38 file after the snapshot 2024-03-09T08:55:35.448 INFO:tasks.workunit.client.1.smithi134.stdout: create 252/39 file after the snapshot 2024-03-09T08:55:35.449 INFO:tasks.workunit.client.1.smithi134.stdout: create 253/.snap/snap-subdir-test 2024-03-09T08:55:35.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 219/31 file after the snapshot 2024-03-09T08:55:35.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 219/32 file after the snapshot 2024-03-09T08:55:35.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 219/33 file after the snapshot 2024-03-09T08:55:35.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 219/34 file after the snapshot 2024-03-09T08:55:35.451 INFO:tasks.workunit.client.0.smithi067.stdout: create 219/35 file after the snapshot 2024-03-09T08:55:35.451 INFO:tasks.workunit.client.0.smithi067.stdout: create 219/36 file after the snapshot 2024-03-09T08:55:35.451 INFO:tasks.workunit.client.0.smithi067.stdout: create 219/37 file after the snapshot 2024-03-09T08:55:35.451 INFO:tasks.workunit.client.0.smithi067.stdout: create 219/38 file after the snapshot 2024-03-09T08:55:35.451 INFO:tasks.workunit.client.0.smithi067.stdout: create 219/39 file after the snapshot 2024-03-09T08:55:35.451 INFO:tasks.workunit.client.0.smithi067.stdout: create 220/.snap/snap-subdir-test 2024-03-09T08:55:35.636 INFO:tasks.workunit.client.0.smithi067.stdout: create 220/30 file after the snapshot 2024-03-09T08:55:35.638 INFO:tasks.workunit.client.0.smithi067.stdout: create 220/31 file after the snapshot 2024-03-09T08:55:35.639 INFO:tasks.workunit.client.0.smithi067.stdout: create 220/32 file after the snapshot 2024-03-09T08:55:35.639 INFO:tasks.workunit.client.0.smithi067.stdout: create 220/33 file after the snapshot 2024-03-09T08:55:35.640 INFO:tasks.workunit.client.0.smithi067.stdout: create 220/34 file after the snapshot 2024-03-09T08:55:35.640 INFO:tasks.workunit.client.0.smithi067.stdout: create 220/35 file after the snapshot 2024-03-09T08:55:35.640 INFO:tasks.workunit.client.0.smithi067.stdout: create 220/36 file after the snapshot 2024-03-09T08:55:35.641 INFO:tasks.workunit.client.0.smithi067.stdout: create 220/37 file after the snapshot 2024-03-09T08:55:35.641 INFO:tasks.workunit.client.0.smithi067.stdout: create 220/38 file after the snapshot 2024-03-09T08:55:35.642 INFO:tasks.workunit.client.0.smithi067.stdout: create 220/39 file after the snapshot 2024-03-09T08:55:35.643 INFO:tasks.workunit.client.0.smithi067.stdout: create 221/.snap/snap-subdir-test 2024-03-09T08:55:35.708 INFO:tasks.workunit.client.0.smithi067.stdout: create 221/30 file after the snapshot 2024-03-09T08:55:35.711 INFO:tasks.workunit.client.0.smithi067.stdout: create 221/31 file after the snapshot 2024-03-09T08:55:35.711 INFO:tasks.workunit.client.0.smithi067.stdout: create 221/32 file after the snapshot 2024-03-09T08:55:35.711 INFO:tasks.workunit.client.0.smithi067.stdout: create 221/33 file after the snapshot 2024-03-09T08:55:35.712 INFO:tasks.workunit.client.0.smithi067.stdout: create 221/34 file after the snapshot 2024-03-09T08:55:35.712 INFO:tasks.workunit.client.0.smithi067.stdout: create 221/35 file after the snapshot 2024-03-09T08:55:35.712 INFO:tasks.workunit.client.0.smithi067.stdout: create 221/36 file after the snapshot 2024-03-09T08:55:35.713 INFO:tasks.workunit.client.0.smithi067.stdout: create 221/37 file after the snapshot 2024-03-09T08:55:35.713 INFO:tasks.workunit.client.0.smithi067.stdout: create 221/38 file after the snapshot 2024-03-09T08:55:35.713 INFO:tasks.workunit.client.0.smithi067.stdout: create 221/39 file after the snapshot 2024-03-09T08:55:35.714 INFO:tasks.workunit.client.0.smithi067.stdout: create 222/.snap/snap-subdir-test 2024-03-09T08:55:35.782 INFO:tasks.workunit.client.0.smithi067.stdout: create 222/30 file after the snapshot 2024-03-09T08:55:35.785 INFO:tasks.workunit.client.0.smithi067.stdout: create 222/31 file after the snapshot 2024-03-09T08:55:35.785 INFO:tasks.workunit.client.0.smithi067.stdout: create 222/32 file after the snapshot 2024-03-09T08:55:35.785 INFO:tasks.workunit.client.0.smithi067.stdout: create 222/33 file after the snapshot 2024-03-09T08:55:35.786 INFO:tasks.workunit.client.0.smithi067.stdout: create 222/34 file after the snapshot 2024-03-09T08:55:35.786 INFO:tasks.workunit.client.0.smithi067.stdout: create 222/35 file after the snapshot 2024-03-09T08:55:35.786 INFO:tasks.workunit.client.0.smithi067.stdout: create 222/36 file after the snapshot 2024-03-09T08:55:35.787 INFO:tasks.workunit.client.0.smithi067.stdout: create 222/37 file after the snapshot 2024-03-09T08:55:35.787 INFO:tasks.workunit.client.0.smithi067.stdout: create 222/38 file after the snapshot 2024-03-09T08:55:35.787 INFO:tasks.workunit.client.0.smithi067.stdout: create 222/39 file after the snapshot 2024-03-09T08:55:35.788 INFO:tasks.workunit.client.0.smithi067.stdout: create 223/.snap/snap-subdir-test 2024-03-09T08:55:35.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 223/30 file after the snapshot 2024-03-09T08:55:35.875 INFO:tasks.workunit.client.0.smithi067.stdout: create 223/31 file after the snapshot 2024-03-09T08:55:35.875 INFO:tasks.workunit.client.0.smithi067.stdout: create 223/32 file after the snapshot 2024-03-09T08:55:35.876 INFO:tasks.workunit.client.0.smithi067.stdout: create 223/33 file after the snapshot 2024-03-09T08:55:35.876 INFO:tasks.workunit.client.0.smithi067.stdout: create 223/34 file after the snapshot 2024-03-09T08:55:35.876 INFO:tasks.workunit.client.0.smithi067.stdout: create 223/35 file after the snapshot 2024-03-09T08:55:35.877 INFO:tasks.workunit.client.0.smithi067.stdout: create 223/36 file after the snapshot 2024-03-09T08:55:35.877 INFO:tasks.workunit.client.0.smithi067.stdout: create 223/37 file after the snapshot 2024-03-09T08:55:35.878 INFO:tasks.workunit.client.0.smithi067.stdout: create 223/38 file after the snapshot 2024-03-09T08:55:35.878 INFO:tasks.workunit.client.0.smithi067.stdout: create 223/39 file after the snapshot 2024-03-09T08:55:35.878 INFO:tasks.workunit.client.0.smithi067.stdout: create 224/.snap/snap-subdir-test 2024-03-09T08:55:35.943 INFO:tasks.workunit.client.0.smithi067.stdout: create 224/30 file after the snapshot 2024-03-09T08:55:35.946 INFO:tasks.workunit.client.0.smithi067.stdout: create 224/31 file after the snapshot 2024-03-09T08:55:35.946 INFO:tasks.workunit.client.0.smithi067.stdout: create 224/32 file after the snapshot 2024-03-09T08:55:35.947 INFO:tasks.workunit.client.0.smithi067.stdout: create 224/33 file after the snapshot 2024-03-09T08:55:35.947 INFO:tasks.workunit.client.0.smithi067.stdout: create 224/34 file after the snapshot 2024-03-09T08:55:35.947 INFO:tasks.workunit.client.0.smithi067.stdout: create 224/35 file after the snapshot 2024-03-09T08:55:35.948 INFO:tasks.workunit.client.0.smithi067.stdout: create 224/36 file after the snapshot 2024-03-09T08:55:35.948 INFO:tasks.workunit.client.0.smithi067.stdout: create 224/37 file after the snapshot 2024-03-09T08:55:35.948 INFO:tasks.workunit.client.0.smithi067.stdout: create 224/38 file after the snapshot 2024-03-09T08:55:35.949 INFO:tasks.workunit.client.0.smithi067.stdout: create 224/39 file after the snapshot 2024-03-09T08:55:35.949 INFO:tasks.workunit.client.0.smithi067.stdout: create 225/.snap/snap-subdir-test 2024-03-09T08:55:36.018 INFO:tasks.workunit.client.0.smithi067.stdout: create 225/30 file after the snapshot 2024-03-09T08:55:36.021 INFO:tasks.workunit.client.0.smithi067.stdout: create 225/31 file after the snapshot 2024-03-09T08:55:36.021 INFO:tasks.workunit.client.0.smithi067.stdout: create 225/32 file after the snapshot 2024-03-09T08:55:36.022 INFO:tasks.workunit.client.0.smithi067.stdout: create 225/33 file after the snapshot 2024-03-09T08:55:36.022 INFO:tasks.workunit.client.0.smithi067.stdout: create 225/34 file after the snapshot 2024-03-09T08:55:36.023 INFO:tasks.workunit.client.0.smithi067.stdout: create 225/35 file after the snapshot 2024-03-09T08:55:36.023 INFO:tasks.workunit.client.0.smithi067.stdout: create 225/36 file after the snapshot 2024-03-09T08:55:36.023 INFO:tasks.workunit.client.0.smithi067.stdout: create 225/37 file after the snapshot 2024-03-09T08:55:36.023 INFO:tasks.workunit.client.0.smithi067.stdout: create 225/38 file after the snapshot 2024-03-09T08:55:36.023 INFO:tasks.workunit.client.0.smithi067.stdout: create 225/39 file after the snapshot 2024-03-09T08:55:36.024 INFO:tasks.workunit.client.0.smithi067.stdout: create 226/.snap/snap-subdir-test 2024-03-09T08:55:36.087 INFO:tasks.workunit.client.0.smithi067.stdout: create 226/30 file after the snapshot 2024-03-09T08:55:36.090 INFO:tasks.workunit.client.0.smithi067.stdout: create 226/31 file after the snapshot 2024-03-09T08:55:36.091 INFO:tasks.workunit.client.0.smithi067.stdout: create 226/32 file after the snapshot 2024-03-09T08:55:36.091 INFO:tasks.workunit.client.0.smithi067.stdout: create 226/33 file after the snapshot 2024-03-09T08:55:36.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 226/34 file after the snapshot 2024-03-09T08:55:36.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 226/35 file after the snapshot 2024-03-09T08:55:36.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 226/36 file after the snapshot 2024-03-09T08:55:36.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 226/37 file after the snapshot 2024-03-09T08:55:36.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 226/38 file after the snapshot 2024-03-09T08:55:36.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 226/39 file after the snapshot 2024-03-09T08:55:36.093 INFO:tasks.workunit.client.0.smithi067.stdout: create 227/.snap/snap-subdir-test 2024-03-09T08:55:36.162 INFO:tasks.workunit.client.0.smithi067.stdout: create 227/30 file after the snapshot 2024-03-09T08:55:36.165 INFO:tasks.workunit.client.0.smithi067.stdout: create 227/31 file after the snapshot 2024-03-09T08:55:36.165 INFO:tasks.workunit.client.0.smithi067.stdout: create 227/32 file after the snapshot 2024-03-09T08:55:36.165 INFO:tasks.workunit.client.0.smithi067.stdout: create 227/33 file after the snapshot 2024-03-09T08:55:36.166 INFO:tasks.workunit.client.0.smithi067.stdout: create 227/34 file after the snapshot 2024-03-09T08:55:36.166 INFO:tasks.workunit.client.0.smithi067.stdout: create 227/35 file after the snapshot 2024-03-09T08:55:36.166 INFO:tasks.workunit.client.0.smithi067.stdout: create 227/36 file after the snapshot 2024-03-09T08:55:36.166 INFO:tasks.workunit.client.0.smithi067.stdout: create 227/37 file after the snapshot 2024-03-09T08:55:36.167 INFO:tasks.workunit.client.0.smithi067.stdout: create 227/38 file after the snapshot 2024-03-09T08:55:36.167 INFO:tasks.workunit.client.0.smithi067.stdout: create 227/39 file after the snapshot 2024-03-09T08:55:36.167 INFO:tasks.workunit.client.0.smithi067.stdout: create 228/.snap/snap-subdir-test 2024-03-09T08:55:36.227 INFO:tasks.workunit.client.0.smithi067.stdout: create 228/30 file after the snapshot 2024-03-09T08:55:36.229 INFO:tasks.workunit.client.0.smithi067.stdout: create 228/31 file after the snapshot 2024-03-09T08:55:36.230 INFO:tasks.workunit.client.0.smithi067.stdout: create 228/32 file after the snapshot 2024-03-09T08:55:36.230 INFO:tasks.workunit.client.0.smithi067.stdout: create 228/33 file after the snapshot 2024-03-09T08:55:36.231 INFO:tasks.workunit.client.0.smithi067.stdout: create 228/34 file after the snapshot 2024-03-09T08:55:36.232 INFO:tasks.workunit.client.0.smithi067.stdout: create 228/35 file after the snapshot 2024-03-09T08:55:36.232 INFO:tasks.workunit.client.0.smithi067.stdout: create 228/36 file after the snapshot 2024-03-09T08:55:36.233 INFO:tasks.workunit.client.0.smithi067.stdout: create 228/37 file after the snapshot 2024-03-09T08:55:36.233 INFO:tasks.workunit.client.0.smithi067.stdout: create 228/38 file after the snapshot 2024-03-09T08:55:36.233 INFO:tasks.workunit.client.0.smithi067.stdout: create 228/39 file after the snapshot 2024-03-09T08:55:36.234 INFO:tasks.workunit.client.0.smithi067.stdout: create 229/.snap/snap-subdir-test 2024-03-09T08:55:36.500 INFO:tasks.workunit.client.1.smithi134.stdout: create 253/30 file after the snapshot 2024-03-09T08:55:36.503 INFO:tasks.workunit.client.1.smithi134.stdout: create 253/31 file after the snapshot 2024-03-09T08:55:36.504 INFO:tasks.workunit.client.1.smithi134.stdout: create 253/32 file after the snapshot 2024-03-09T08:55:36.504 INFO:tasks.workunit.client.1.smithi134.stdout: create 253/33 file after the snapshot 2024-03-09T08:55:36.504 INFO:tasks.workunit.client.1.smithi134.stdout: create 253/34 file after the snapshot 2024-03-09T08:55:36.505 INFO:tasks.workunit.client.1.smithi134.stdout: create 253/35 file after the snapshot 2024-03-09T08:55:36.505 INFO:tasks.workunit.client.1.smithi134.stdout: create 253/36 file after the snapshot 2024-03-09T08:55:36.506 INFO:tasks.workunit.client.1.smithi134.stdout: create 253/37 file after the snapshot 2024-03-09T08:55:36.506 INFO:tasks.workunit.client.1.smithi134.stdout: create 253/38 file after the snapshot 2024-03-09T08:55:36.506 INFO:tasks.workunit.client.1.smithi134.stdout: create 253/39 file after the snapshot 2024-03-09T08:55:36.506 INFO:tasks.workunit.client.1.smithi134.stdout: create 254/.snap/snap-subdir-test 2024-03-09T08:55:36.838 INFO:tasks.workunit.client.1.smithi134.stdout: create 254/30 file after the snapshot 2024-03-09T08:55:36.841 INFO:tasks.workunit.client.1.smithi134.stdout: create 254/31 file after the snapshot 2024-03-09T08:55:36.841 INFO:tasks.workunit.client.1.smithi134.stdout: create 254/32 file after the snapshot 2024-03-09T08:55:36.841 INFO:tasks.workunit.client.1.smithi134.stdout: create 254/33 file after the snapshot 2024-03-09T08:55:36.842 INFO:tasks.workunit.client.1.smithi134.stdout: create 254/34 file after the snapshot 2024-03-09T08:55:36.842 INFO:tasks.workunit.client.1.smithi134.stdout: create 254/35 file after the snapshot 2024-03-09T08:55:36.842 INFO:tasks.workunit.client.1.smithi134.stdout: create 254/36 file after the snapshot 2024-03-09T08:55:36.843 INFO:tasks.workunit.client.1.smithi134.stdout: create 254/37 file after the snapshot 2024-03-09T08:55:36.843 INFO:tasks.workunit.client.1.smithi134.stdout: create 254/38 file after the snapshot 2024-03-09T08:55:36.844 INFO:tasks.workunit.client.1.smithi134.stdout: create 254/39 file after the snapshot 2024-03-09T08:55:36.845 INFO:tasks.workunit.client.1.smithi134.stdout: create 255/.snap/snap-subdir-test 2024-03-09T08:55:36.855 INFO:tasks.workunit.client.0.smithi067.stdout: create 229/30 file after the snapshot 2024-03-09T08:55:36.857 INFO:tasks.workunit.client.0.smithi067.stdout: create 229/31 file after the snapshot 2024-03-09T08:55:36.857 INFO:tasks.workunit.client.0.smithi067.stdout: create 229/32 file after the snapshot 2024-03-09T08:55:36.858 INFO:tasks.workunit.client.0.smithi067.stdout: create 229/33 file after the snapshot 2024-03-09T08:55:36.858 INFO:tasks.workunit.client.0.smithi067.stdout: create 229/34 file after the snapshot 2024-03-09T08:55:36.858 INFO:tasks.workunit.client.0.smithi067.stdout: create 229/35 file after the snapshot 2024-03-09T08:55:36.859 INFO:tasks.workunit.client.0.smithi067.stdout: create 229/36 file after the snapshot 2024-03-09T08:55:36.859 INFO:tasks.workunit.client.0.smithi067.stdout: create 229/37 file after the snapshot 2024-03-09T08:55:36.859 INFO:tasks.workunit.client.0.smithi067.stdout: create 229/38 file after the snapshot 2024-03-09T08:55:36.859 INFO:tasks.workunit.client.0.smithi067.stdout: create 229/39 file after the snapshot 2024-03-09T08:55:36.860 INFO:tasks.workunit.client.0.smithi067.stdout: create 230/.snap/snap-subdir-test 2024-03-09T08:55:36.908 INFO:tasks.workunit.client.1.smithi134.stdout: create 255/30 file after the snapshot 2024-03-09T08:55:36.911 INFO:tasks.workunit.client.1.smithi134.stdout: create 255/31 file after the snapshot 2024-03-09T08:55:36.911 INFO:tasks.workunit.client.1.smithi134.stdout: create 255/32 file after the snapshot 2024-03-09T08:55:36.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 255/33 file after the snapshot 2024-03-09T08:55:36.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 255/34 file after the snapshot 2024-03-09T08:55:36.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 255/35 file after the snapshot 2024-03-09T08:55:36.913 INFO:tasks.workunit.client.1.smithi134.stdout: create 255/36 file after the snapshot 2024-03-09T08:55:36.913 INFO:tasks.workunit.client.1.smithi134.stdout: create 255/37 file after the snapshot 2024-03-09T08:55:36.913 INFO:tasks.workunit.client.1.smithi134.stdout: create 255/38 file after the snapshot 2024-03-09T08:55:36.914 INFO:tasks.workunit.client.1.smithi134.stdout: create 255/39 file after the snapshot 2024-03-09T08:55:36.914 INFO:tasks.workunit.client.1.smithi134.stdout: create 256/.snap/snap-subdir-test 2024-03-09T08:55:36.918 INFO:tasks.workunit.client.0.smithi067.stdout: create 230/30 file after the snapshot 2024-03-09T08:55:36.920 INFO:tasks.workunit.client.0.smithi067.stdout: create 230/31 file after the snapshot 2024-03-09T08:55:36.921 INFO:tasks.workunit.client.0.smithi067.stdout: create 230/32 file after the snapshot 2024-03-09T08:55:36.921 INFO:tasks.workunit.client.0.smithi067.stdout: create 230/33 file after the snapshot 2024-03-09T08:55:36.921 INFO:tasks.workunit.client.0.smithi067.stdout: create 230/34 file after the snapshot 2024-03-09T08:55:36.921 INFO:tasks.workunit.client.0.smithi067.stdout: create 230/35 file after the snapshot 2024-03-09T08:55:36.922 INFO:tasks.workunit.client.0.smithi067.stdout: create 230/36 file after the snapshot 2024-03-09T08:55:36.922 INFO:tasks.workunit.client.0.smithi067.stdout: create 230/37 file after the snapshot 2024-03-09T08:55:36.922 INFO:tasks.workunit.client.0.smithi067.stdout: create 230/38 file after the snapshot 2024-03-09T08:55:36.922 INFO:tasks.workunit.client.0.smithi067.stdout: create 230/39 file after the snapshot 2024-03-09T08:55:36.923 INFO:tasks.workunit.client.0.smithi067.stdout: create 231/.snap/snap-subdir-test 2024-03-09T08:55:36.981 INFO:tasks.workunit.client.1.smithi134.stdout: create 256/30 file after the snapshot 2024-03-09T08:55:36.983 INFO:tasks.workunit.client.1.smithi134.stdout: create 256/31 file after the snapshot 2024-03-09T08:55:36.984 INFO:tasks.workunit.client.1.smithi134.stdout: create 256/32 file after the snapshot 2024-03-09T08:55:36.984 INFO:tasks.workunit.client.1.smithi134.stdout: create 256/33 file after the snapshot 2024-03-09T08:55:36.984 INFO:tasks.workunit.client.1.smithi134.stdout: create 256/34 file after the snapshot 2024-03-09T08:55:36.985 INFO:tasks.workunit.client.1.smithi134.stdout: create 256/35 file after the snapshot 2024-03-09T08:55:36.985 INFO:tasks.workunit.client.1.smithi134.stdout: create 256/36 file after the snapshot 2024-03-09T08:55:36.985 INFO:tasks.workunit.client.1.smithi134.stdout: create 256/37 file after the snapshot 2024-03-09T08:55:36.985 INFO:tasks.workunit.client.1.smithi134.stdout: create 256/38 file after the snapshot 2024-03-09T08:55:36.985 INFO:tasks.workunit.client.1.smithi134.stdout: create 256/39 file after the snapshot 2024-03-09T08:55:36.986 INFO:tasks.workunit.client.1.smithi134.stdout: create 257/.snap/snap-subdir-test 2024-03-09T08:55:36.991 INFO:tasks.workunit.client.0.smithi067.stdout: create 231/30 file after the snapshot 2024-03-09T08:55:36.994 INFO:tasks.workunit.client.0.smithi067.stdout: create 231/31 file after the snapshot 2024-03-09T08:55:36.994 INFO:tasks.workunit.client.0.smithi067.stdout: create 231/32 file after the snapshot 2024-03-09T08:55:36.995 INFO:tasks.workunit.client.0.smithi067.stdout: create 231/33 file after the snapshot 2024-03-09T08:55:36.995 INFO:tasks.workunit.client.0.smithi067.stdout: create 231/34 file after the snapshot 2024-03-09T08:55:36.996 INFO:tasks.workunit.client.0.smithi067.stdout: create 231/35 file after the snapshot 2024-03-09T08:55:36.996 INFO:tasks.workunit.client.0.smithi067.stdout: create 231/36 file after the snapshot 2024-03-09T08:55:36.998 INFO:tasks.workunit.client.0.smithi067.stdout: create 231/37 file after the snapshot 2024-03-09T08:55:36.999 INFO:tasks.workunit.client.0.smithi067.stdout: create 231/38 file after the snapshot 2024-03-09T08:55:36.999 INFO:tasks.workunit.client.0.smithi067.stdout: create 231/39 file after the snapshot 2024-03-09T08:55:37.000 INFO:tasks.workunit.client.0.smithi067.stdout: create 232/.snap/snap-subdir-test 2024-03-09T08:55:37.055 INFO:tasks.workunit.client.1.smithi134.stdout: create 257/30 file after the snapshot 2024-03-09T08:55:37.060 INFO:tasks.workunit.client.1.smithi134.stdout: create 257/31 file after the snapshot 2024-03-09T08:55:37.060 INFO:tasks.workunit.client.1.smithi134.stdout: create 257/32 file after the snapshot 2024-03-09T08:55:37.061 INFO:tasks.workunit.client.1.smithi134.stdout: create 257/33 file after the snapshot 2024-03-09T08:55:37.061 INFO:tasks.workunit.client.1.smithi134.stdout: create 257/34 file after the snapshot 2024-03-09T08:55:37.061 INFO:tasks.workunit.client.1.smithi134.stdout: create 257/35 file after the snapshot 2024-03-09T08:55:37.061 INFO:tasks.workunit.client.1.smithi134.stdout: create 257/36 file after the snapshot 2024-03-09T08:55:37.062 INFO:tasks.workunit.client.1.smithi134.stdout: create 257/37 file after the snapshot 2024-03-09T08:55:37.062 INFO:tasks.workunit.client.1.smithi134.stdout: create 257/38 file after the snapshot 2024-03-09T08:55:37.062 INFO:tasks.workunit.client.1.smithi134.stdout: create 257/39 file after the snapshot 2024-03-09T08:55:37.062 INFO:tasks.workunit.client.1.smithi134.stdout: create 258/.snap/snap-subdir-test 2024-03-09T08:55:37.081 INFO:tasks.workunit.client.0.smithi067.stdout: create 232/30 file after the snapshot 2024-03-09T08:55:37.083 INFO:tasks.workunit.client.0.smithi067.stdout: create 232/31 file after the snapshot 2024-03-09T08:55:37.084 INFO:tasks.workunit.client.0.smithi067.stdout: create 232/32 file after the snapshot 2024-03-09T08:55:37.084 INFO:tasks.workunit.client.0.smithi067.stdout: create 232/33 file after the snapshot 2024-03-09T08:55:37.084 INFO:tasks.workunit.client.0.smithi067.stdout: create 232/34 file after the snapshot 2024-03-09T08:55:37.084 INFO:tasks.workunit.client.0.smithi067.stdout: create 232/35 file after the snapshot 2024-03-09T08:55:37.084 INFO:tasks.workunit.client.0.smithi067.stdout: create 232/36 file after the snapshot 2024-03-09T08:55:37.085 INFO:tasks.workunit.client.0.smithi067.stdout: create 232/37 file after the snapshot 2024-03-09T08:55:37.085 INFO:tasks.workunit.client.0.smithi067.stdout: create 232/38 file after the snapshot 2024-03-09T08:55:37.086 INFO:tasks.workunit.client.0.smithi067.stdout: create 232/39 file after the snapshot 2024-03-09T08:55:37.086 INFO:tasks.workunit.client.0.smithi067.stdout: create 233/.snap/snap-subdir-test 2024-03-09T08:55:37.129 INFO:tasks.workunit.client.1.smithi134.stdout: create 258/30 file after the snapshot 2024-03-09T08:55:37.133 INFO:tasks.workunit.client.1.smithi134.stdout: create 258/31 file after the snapshot 2024-03-09T08:55:37.133 INFO:tasks.workunit.client.1.smithi134.stdout: create 258/32 file after the snapshot 2024-03-09T08:55:37.134 INFO:tasks.workunit.client.1.smithi134.stdout: create 258/33 file after the snapshot 2024-03-09T08:55:37.134 INFO:tasks.workunit.client.1.smithi134.stdout: create 258/34 file after the snapshot 2024-03-09T08:55:37.134 INFO:tasks.workunit.client.1.smithi134.stdout: create 258/35 file after the snapshot 2024-03-09T08:55:37.134 INFO:tasks.workunit.client.1.smithi134.stdout: create 258/36 file after the snapshot 2024-03-09T08:55:37.134 INFO:tasks.workunit.client.1.smithi134.stdout: create 258/37 file after the snapshot 2024-03-09T08:55:37.135 INFO:tasks.workunit.client.1.smithi134.stdout: create 258/38 file after the snapshot 2024-03-09T08:55:37.135 INFO:tasks.workunit.client.1.smithi134.stdout: create 258/39 file after the snapshot 2024-03-09T08:55:37.135 INFO:tasks.workunit.client.1.smithi134.stdout: create 259/.snap/snap-subdir-test 2024-03-09T08:55:37.153 INFO:tasks.workunit.client.0.smithi067.stdout: create 233/30 file after the snapshot 2024-03-09T08:55:37.155 INFO:tasks.workunit.client.0.smithi067.stdout: create 233/31 file after the snapshot 2024-03-09T08:55:37.155 INFO:tasks.workunit.client.0.smithi067.stdout: create 233/32 file after the snapshot 2024-03-09T08:55:37.156 INFO:tasks.workunit.client.0.smithi067.stdout: create 233/33 file after the snapshot 2024-03-09T08:55:37.156 INFO:tasks.workunit.client.0.smithi067.stdout: create 233/34 file after the snapshot 2024-03-09T08:55:37.156 INFO:tasks.workunit.client.0.smithi067.stdout: create 233/35 file after the snapshot 2024-03-09T08:55:37.157 INFO:tasks.workunit.client.0.smithi067.stdout: create 233/36 file after the snapshot 2024-03-09T08:55:37.157 INFO:tasks.workunit.client.0.smithi067.stdout: create 233/37 file after the snapshot 2024-03-09T08:55:37.157 INFO:tasks.workunit.client.0.smithi067.stdout: create 233/38 file after the snapshot 2024-03-09T08:55:37.157 INFO:tasks.workunit.client.0.smithi067.stdout: create 233/39 file after the snapshot 2024-03-09T08:55:37.157 INFO:tasks.workunit.client.0.smithi067.stdout: create 234/.snap/snap-subdir-test 2024-03-09T08:55:37.782 INFO:tasks.workunit.client.1.smithi134.stdout: create 259/30 file after the snapshot 2024-03-09T08:55:37.785 INFO:tasks.workunit.client.1.smithi134.stdout: create 259/31 file after the snapshot 2024-03-09T08:55:37.785 INFO:tasks.workunit.client.1.smithi134.stdout: create 259/32 file after the snapshot 2024-03-09T08:55:37.787 INFO:tasks.workunit.client.1.smithi134.stdout: create 259/33 file after the snapshot 2024-03-09T08:55:37.787 INFO:tasks.workunit.client.1.smithi134.stdout: create 259/34 file after the snapshot 2024-03-09T08:55:37.788 INFO:tasks.workunit.client.1.smithi134.stdout: create 259/35 file after the snapshot 2024-03-09T08:55:37.788 INFO:tasks.workunit.client.1.smithi134.stdout: create 259/36 file after the snapshot 2024-03-09T08:55:37.788 INFO:tasks.workunit.client.1.smithi134.stdout: create 259/37 file after the snapshot 2024-03-09T08:55:37.789 INFO:tasks.workunit.client.1.smithi134.stdout: create 259/38 file after the snapshot 2024-03-09T08:55:37.789 INFO:tasks.workunit.client.1.smithi134.stdout: create 259/39 file after the snapshot 2024-03-09T08:55:37.789 INFO:tasks.workunit.client.1.smithi134.stdout: create 260/.snap/snap-subdir-test 2024-03-09T08:55:37.864 INFO:tasks.workunit.client.1.smithi134.stdout: create 260/30 file after the snapshot 2024-03-09T08:55:37.867 INFO:tasks.workunit.client.1.smithi134.stdout: create 260/31 file after the snapshot 2024-03-09T08:55:37.868 INFO:tasks.workunit.client.1.smithi134.stdout: create 260/32 file after the snapshot 2024-03-09T08:55:37.868 INFO:tasks.workunit.client.1.smithi134.stdout: create 260/33 file after the snapshot 2024-03-09T08:55:37.868 INFO:tasks.workunit.client.1.smithi134.stdout: create 260/34 file after the snapshot 2024-03-09T08:55:37.869 INFO:tasks.workunit.client.1.smithi134.stdout: create 260/35 file after the snapshot 2024-03-09T08:55:37.869 INFO:tasks.workunit.client.1.smithi134.stdout: create 260/36 file after the snapshot 2024-03-09T08:55:37.869 INFO:tasks.workunit.client.1.smithi134.stdout: create 260/37 file after the snapshot 2024-03-09T08:55:37.870 INFO:tasks.workunit.client.1.smithi134.stdout: create 260/38 file after the snapshot 2024-03-09T08:55:37.870 INFO:tasks.workunit.client.1.smithi134.stdout: create 260/39 file after the snapshot 2024-03-09T08:55:37.870 INFO:tasks.workunit.client.1.smithi134.stdout: create 261/.snap/snap-subdir-test 2024-03-09T08:55:38.165 INFO:tasks.workunit.client.0.smithi067.stdout: create 234/30 file after the snapshot 2024-03-09T08:55:38.168 INFO:tasks.workunit.client.0.smithi067.stdout: create 234/31 file after the snapshot 2024-03-09T08:55:38.168 INFO:tasks.workunit.client.0.smithi067.stdout: create 234/32 file after the snapshot 2024-03-09T08:55:38.169 INFO:tasks.workunit.client.0.smithi067.stdout: create 234/33 file after the snapshot 2024-03-09T08:55:38.170 INFO:tasks.workunit.client.0.smithi067.stdout: create 234/34 file after the snapshot 2024-03-09T08:55:38.170 INFO:tasks.workunit.client.0.smithi067.stdout: create 234/35 file after the snapshot 2024-03-09T08:55:38.171 INFO:tasks.workunit.client.0.smithi067.stdout: create 234/36 file after the snapshot 2024-03-09T08:55:38.171 INFO:tasks.workunit.client.0.smithi067.stdout: create 234/37 file after the snapshot 2024-03-09T08:55:38.171 INFO:tasks.workunit.client.0.smithi067.stdout: create 234/38 file after the snapshot 2024-03-09T08:55:38.172 INFO:tasks.workunit.client.0.smithi067.stdout: create 234/39 file after the snapshot 2024-03-09T08:55:38.172 INFO:tasks.workunit.client.0.smithi067.stdout: create 235/.snap/snap-subdir-test 2024-03-09T08:55:38.250 INFO:tasks.workunit.client.0.smithi067.stdout: create 235/30 file after the snapshot 2024-03-09T08:55:38.252 INFO:tasks.workunit.client.0.smithi067.stdout: create 235/31 file after the snapshot 2024-03-09T08:55:38.252 INFO:tasks.workunit.client.0.smithi067.stdout: create 235/32 file after the snapshot 2024-03-09T08:55:38.252 INFO:tasks.workunit.client.0.smithi067.stdout: create 235/33 file after the snapshot 2024-03-09T08:55:38.253 INFO:tasks.workunit.client.0.smithi067.stdout: create 235/34 file after the snapshot 2024-03-09T08:55:38.253 INFO:tasks.workunit.client.0.smithi067.stdout: create 235/35 file after the snapshot 2024-03-09T08:55:38.253 INFO:tasks.workunit.client.0.smithi067.stdout: create 235/36 file after the snapshot 2024-03-09T08:55:38.253 INFO:tasks.workunit.client.0.smithi067.stdout: create 235/37 file after the snapshot 2024-03-09T08:55:38.254 INFO:tasks.workunit.client.0.smithi067.stdout: create 235/38 file after the snapshot 2024-03-09T08:55:38.254 INFO:tasks.workunit.client.0.smithi067.stdout: create 235/39 file after the snapshot 2024-03-09T08:55:38.254 INFO:tasks.workunit.client.0.smithi067.stdout: create 236/.snap/snap-subdir-test 2024-03-09T08:55:38.335 INFO:tasks.workunit.client.0.smithi067.stdout: create 236/30 file after the snapshot 2024-03-09T08:55:38.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 236/31 file after the snapshot 2024-03-09T08:55:38.338 INFO:tasks.workunit.client.0.smithi067.stdout: create 236/32 file after the snapshot 2024-03-09T08:55:38.339 INFO:tasks.workunit.client.0.smithi067.stdout: create 236/33 file after the snapshot 2024-03-09T08:55:38.339 INFO:tasks.workunit.client.0.smithi067.stdout: create 236/34 file after the snapshot 2024-03-09T08:55:38.339 INFO:tasks.workunit.client.0.smithi067.stdout: create 236/35 file after the snapshot 2024-03-09T08:55:38.339 INFO:tasks.workunit.client.0.smithi067.stdout: create 236/36 file after the snapshot 2024-03-09T08:55:38.340 INFO:tasks.workunit.client.0.smithi067.stdout: create 236/37 file after the snapshot 2024-03-09T08:55:38.340 INFO:tasks.workunit.client.0.smithi067.stdout: create 236/38 file after the snapshot 2024-03-09T08:55:38.340 INFO:tasks.workunit.client.0.smithi067.stdout: create 236/39 file after the snapshot 2024-03-09T08:55:38.340 INFO:tasks.workunit.client.0.smithi067.stdout: create 237/.snap/snap-subdir-test 2024-03-09T08:55:38.517 INFO:tasks.workunit.client.1.smithi134.stdout: create 261/30 file after the snapshot 2024-03-09T08:55:38.520 INFO:tasks.workunit.client.1.smithi134.stdout: create 261/31 file after the snapshot 2024-03-09T08:55:38.520 INFO:tasks.workunit.client.1.smithi134.stdout: create 261/32 file after the snapshot 2024-03-09T08:55:38.521 INFO:tasks.workunit.client.1.smithi134.stdout: create 261/33 file after the snapshot 2024-03-09T08:55:38.521 INFO:tasks.workunit.client.1.smithi134.stdout: create 261/34 file after the snapshot 2024-03-09T08:55:38.522 INFO:tasks.workunit.client.1.smithi134.stdout: create 261/35 file after the snapshot 2024-03-09T08:55:38.522 INFO:tasks.workunit.client.1.smithi134.stdout: create 261/36 file after the snapshot 2024-03-09T08:55:38.522 INFO:tasks.workunit.client.1.smithi134.stdout: create 261/37 file after the snapshot 2024-03-09T08:55:38.523 INFO:tasks.workunit.client.1.smithi134.stdout: create 261/38 file after the snapshot 2024-03-09T08:55:38.523 INFO:tasks.workunit.client.1.smithi134.stdout: create 261/39 file after the snapshot 2024-03-09T08:55:38.524 INFO:tasks.workunit.client.1.smithi134.stdout: create 262/.snap/snap-subdir-test 2024-03-09T08:55:38.610 INFO:tasks.workunit.client.1.smithi134.stdout: create 262/30 file after the snapshot 2024-03-09T08:55:38.614 INFO:tasks.workunit.client.1.smithi134.stdout: create 262/31 file after the snapshot 2024-03-09T08:55:38.614 INFO:tasks.workunit.client.1.smithi134.stdout: create 262/32 file after the snapshot 2024-03-09T08:55:38.615 INFO:tasks.workunit.client.1.smithi134.stdout: create 262/33 file after the snapshot 2024-03-09T08:55:38.615 INFO:tasks.workunit.client.1.smithi134.stdout: create 262/34 file after the snapshot 2024-03-09T08:55:38.615 INFO:tasks.workunit.client.1.smithi134.stdout: create 262/35 file after the snapshot 2024-03-09T08:55:38.615 INFO:tasks.workunit.client.1.smithi134.stdout: create 262/36 file after the snapshot 2024-03-09T08:55:38.615 INFO:tasks.workunit.client.1.smithi134.stdout: create 262/37 file after the snapshot 2024-03-09T08:55:38.615 INFO:tasks.workunit.client.1.smithi134.stdout: create 262/38 file after the snapshot 2024-03-09T08:55:38.615 INFO:tasks.workunit.client.1.smithi134.stdout: create 262/39 file after the snapshot 2024-03-09T08:55:38.615 INFO:tasks.workunit.client.1.smithi134.stdout: create 263/.snap/snap-subdir-test 2024-03-09T08:55:38.682 INFO:tasks.workunit.client.0.smithi067.stdout: create 237/30 file after the snapshot 2024-03-09T08:55:38.684 INFO:tasks.workunit.client.0.smithi067.stdout: create 237/31 file after the snapshot 2024-03-09T08:55:38.685 INFO:tasks.workunit.client.0.smithi067.stdout: create 237/32 file after the snapshot 2024-03-09T08:55:38.685 INFO:tasks.workunit.client.0.smithi067.stdout: create 237/33 file after the snapshot 2024-03-09T08:55:38.686 INFO:tasks.workunit.client.0.smithi067.stdout: create 237/34 file after the snapshot 2024-03-09T08:55:38.686 INFO:tasks.workunit.client.0.smithi067.stdout: create 237/35 file after the snapshot 2024-03-09T08:55:38.686 INFO:tasks.workunit.client.0.smithi067.stdout: create 237/36 file after the snapshot 2024-03-09T08:55:38.686 INFO:tasks.workunit.client.0.smithi067.stdout: create 237/37 file after the snapshot 2024-03-09T08:55:38.686 INFO:tasks.workunit.client.0.smithi067.stdout: create 237/38 file after the snapshot 2024-03-09T08:55:38.686 INFO:tasks.workunit.client.0.smithi067.stdout: create 237/39 file after the snapshot 2024-03-09T08:55:38.686 INFO:tasks.workunit.client.0.smithi067.stdout: create 238/.snap/snap-subdir-test 2024-03-09T08:55:38.706 INFO:tasks.workunit.client.1.smithi134.stdout: create 263/30 file after the snapshot 2024-03-09T08:55:38.709 INFO:tasks.workunit.client.1.smithi134.stdout: create 263/31 file after the snapshot 2024-03-09T08:55:38.709 INFO:tasks.workunit.client.1.smithi134.stdout: create 263/32 file after the snapshot 2024-03-09T08:55:38.710 INFO:tasks.workunit.client.1.smithi134.stdout: create 263/33 file after the snapshot 2024-03-09T08:55:38.710 INFO:tasks.workunit.client.1.smithi134.stdout: create 263/34 file after the snapshot 2024-03-09T08:55:38.710 INFO:tasks.workunit.client.1.smithi134.stdout: create 263/35 file after the snapshot 2024-03-09T08:55:38.710 INFO:tasks.workunit.client.1.smithi134.stdout: create 263/36 file after the snapshot 2024-03-09T08:55:38.710 INFO:tasks.workunit.client.1.smithi134.stdout: create 263/37 file after the snapshot 2024-03-09T08:55:38.710 INFO:tasks.workunit.client.1.smithi134.stdout: create 263/38 file after the snapshot 2024-03-09T08:55:38.710 INFO:tasks.workunit.client.1.smithi134.stdout: create 263/39 file after the snapshot 2024-03-09T08:55:38.710 INFO:tasks.workunit.client.1.smithi134.stdout: create 264/.snap/snap-subdir-test 2024-03-09T08:55:38.760 INFO:tasks.workunit.client.0.smithi067.stdout: create 238/30 file after the snapshot 2024-03-09T08:55:38.762 INFO:tasks.workunit.client.0.smithi067.stdout: create 238/31 file after the snapshot 2024-03-09T08:55:38.763 INFO:tasks.workunit.client.0.smithi067.stdout: create 238/32 file after the snapshot 2024-03-09T08:55:38.763 INFO:tasks.workunit.client.0.smithi067.stdout: create 238/33 file after the snapshot 2024-03-09T08:55:38.763 INFO:tasks.workunit.client.0.smithi067.stdout: create 238/34 file after the snapshot 2024-03-09T08:55:38.764 INFO:tasks.workunit.client.0.smithi067.stdout: create 238/35 file after the snapshot 2024-03-09T08:55:38.764 INFO:tasks.workunit.client.0.smithi067.stdout: create 238/36 file after the snapshot 2024-03-09T08:55:38.765 INFO:tasks.workunit.client.0.smithi067.stdout: create 238/37 file after the snapshot 2024-03-09T08:55:38.765 INFO:tasks.workunit.client.0.smithi067.stdout: create 238/38 file after the snapshot 2024-03-09T08:55:38.765 INFO:tasks.workunit.client.0.smithi067.stdout: create 238/39 file after the snapshot 2024-03-09T08:55:38.766 INFO:tasks.workunit.client.0.smithi067.stdout: create 239/.snap/snap-subdir-test 2024-03-09T08:55:38.791 INFO:tasks.workunit.client.1.smithi134.stdout: create 264/30 file after the snapshot 2024-03-09T08:55:38.795 INFO:tasks.workunit.client.1.smithi134.stdout: create 264/31 file after the snapshot 2024-03-09T08:55:38.796 INFO:tasks.workunit.client.1.smithi134.stdout: create 264/32 file after the snapshot 2024-03-09T08:55:38.796 INFO:tasks.workunit.client.1.smithi134.stdout: create 264/33 file after the snapshot 2024-03-09T08:55:38.796 INFO:tasks.workunit.client.1.smithi134.stdout: create 264/34 file after the snapshot 2024-03-09T08:55:38.796 INFO:tasks.workunit.client.1.smithi134.stdout: create 264/35 file after the snapshot 2024-03-09T08:55:38.797 INFO:tasks.workunit.client.1.smithi134.stdout: create 264/36 file after the snapshot 2024-03-09T08:55:38.797 INFO:tasks.workunit.client.1.smithi134.stdout: create 264/37 file after the snapshot 2024-03-09T08:55:38.797 INFO:tasks.workunit.client.1.smithi134.stdout: create 264/38 file after the snapshot 2024-03-09T08:55:38.797 INFO:tasks.workunit.client.1.smithi134.stdout: create 264/39 file after the snapshot 2024-03-09T08:55:38.798 INFO:tasks.workunit.client.1.smithi134.stdout: create 265/.snap/snap-subdir-test 2024-03-09T08:55:38.909 INFO:tasks.workunit.client.1.smithi134.stdout: create 265/30 file after the snapshot 2024-03-09T08:55:38.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 265/31 file after the snapshot 2024-03-09T08:55:38.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 265/32 file after the snapshot 2024-03-09T08:55:38.912 INFO:tasks.workunit.client.1.smithi134.stdout: create 265/33 file after the snapshot 2024-03-09T08:55:38.913 INFO:tasks.workunit.client.1.smithi134.stdout: create 265/34 file after the snapshot 2024-03-09T08:55:38.913 INFO:tasks.workunit.client.1.smithi134.stdout: create 265/35 file after the snapshot 2024-03-09T08:55:38.914 INFO:tasks.workunit.client.1.smithi134.stdout: create 265/36 file after the snapshot 2024-03-09T08:55:38.914 INFO:tasks.workunit.client.1.smithi134.stdout: create 265/37 file after the snapshot 2024-03-09T08:55:38.914 INFO:tasks.workunit.client.1.smithi134.stdout: create 265/38 file after the snapshot 2024-03-09T08:55:38.915 INFO:tasks.workunit.client.1.smithi134.stdout: create 265/39 file after the snapshot 2024-03-09T08:55:38.916 INFO:tasks.workunit.client.1.smithi134.stdout: create 266/.snap/snap-subdir-test 2024-03-09T08:55:38.922 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T08:55:38.922 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-03-09T08:55:38.922 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-09T08:55:38.922 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-09T08:55:38.930 INFO:tasks.ceph.mon.b:Started 2024-03-09T08:55:38.930 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:55:38.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:55:39.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.151+0000 7f23d8c00640 1 Processor -- start 2024-03-09T08:55:39.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.151+0000 7f23d8c00640 1 -- start start 2024-03-09T08:55:39.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.153+0000 7f23d8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f23d406d420 0x7f23d41527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.153+0000 7f23d8c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d406d920 con 0x7f23d414c9f0 2024-03-09T08:55:39.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.153+0000 7f23d8c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d406dc00 con 0x7f23d40c3080 2024-03-09T08:55:39.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.153+0000 7f23d8c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d4155ba0 con 0x7f23d406d420 2024-03-09T08:55:39.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.153+0000 7f23d3400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f23d40c3080 0x7f23d414c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:40824/0 (socket says 172.21.15.67:40824) 2024-03-09T08:55:39.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.153+0000 7f23d3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f23d414c9f0 0x7f23d41540a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60332/0 (socket says 172.21.15.67:60332) 2024-03-09T08:55:39.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.153+0000 7f23d3400640 1 -- 172.21.15.67:0/3864131057 learned_addr learned my addr 172.21.15.67:0/3864131057 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:39.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.153+0000 7f23cbe00640 1 --2- 172.21.15.67:0/3864131057 >> v2:172.21.15.67:3300/0 conn(0x7f23d406d420 0x7f23d41527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.154+0000 7f23cbe00640 1 -- 172.21.15.67:0/3864131057 >> v1:172.21.15.134:6789/0 conn(0x7f23d40c3080 legacy=0x7f23d414c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.154+0000 7f23cbe00640 1 -- 172.21.15.67:0/3864131057 >> v1:172.21.15.67:6789/0 conn(0x7f23d414c9f0 legacy=0x7f23d41540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.154+0000 7f23cbe00640 1 -- 172.21.15.67:0/3864131057 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d406dee0 con 0x7f23d406d420 2024-03-09T08:55:39.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.155+0000 7f23cbe00640 1 --2- 172.21.15.67:0/3864131057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d406d420 0x7f23d41527d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0097a0 tx=0x7f23bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6413a890fcf29bd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:39.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.155+0000 7f23d2a00640 1 -- 172.21.15.67:0/3864131057 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc035020 con 0x7f23d406d420 2024-03-09T08:55:39.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.155+0000 7f23d2a00640 1 -- 172.21.15.67:0/3864131057 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc031830 con 0x7f23d406d420 2024-03-09T08:55:39.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.155+0000 7f23d2a00640 1 -- 172.21.15.67:0/3864131057 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc031c00 con 0x7f23d406d420 2024-03-09T08:55:39.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.156+0000 7f23d8c00640 1 -- 172.21.15.67:0/3864131057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d406d420 msgr2=0x7f23d41527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.156+0000 7f23d8c00640 1 --2- 172.21.15.67:0/3864131057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d406d420 0x7f23d41527d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0097a0 tx=0x7f23bc02b8d0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.156+0000 7f23d8c00640 1 -- 172.21.15.67:0/3864131057 shutdown_connections 2024-03-09T08:55:39.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.156+0000 7f23d8c00640 1 --2- 172.21.15.67:0/3864131057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d406d420 0x7f23d41527d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.156+0000 7f23d8c00640 1 -- 172.21.15.67:0/3864131057 >> 172.21.15.67:0/3864131057 conn(0x7f23d40c7300 msgr2=0x7f23d40c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:39.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.156+0000 7f23d8c00640 1 -- 172.21.15.67:0/3864131057 shutdown_connections 2024-03-09T08:55:39.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.156+0000 7f23d8c00640 1 -- 172.21.15.67:0/3864131057 wait complete. 2024-03-09T08:55:39.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.158+0000 7f23d8c00640 1 Processor -- start 2024-03-09T08:55:39.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.158+0000 7f23d8c00640 1 -- start start 2024-03-09T08:55:39.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.159+0000 7f23d8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d406d420 0x7f23d41e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.159+0000 7f23d8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d40c3080 0x7f23d40ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.159+0000 7f23d8c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d414c9f0 0x7f23d414ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.159+0000 7f23d8c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40c8030 con 0x7f23d40c3080 2024-03-09T08:55:39.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.159+0000 7f23d8c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40c3460 con 0x7f23d406d420 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.159+0000 7f23d8c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40ad2a0 con 0x7f23d414c9f0 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.159+0000 7f23cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d40c3080 0x7f23d40ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d40c3080 0x7f23d40ab990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37902/0 (socket says 172.21.15.67:37902) 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23cbe00640 1 -- 172.21.15.67:0/2135113021 learned_addr learned my addr 172.21.15.67:0/2135113021 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23d3400640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d406d420 0x7f23d41e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23d3e00640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d414c9f0 0x7f23d414ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23d3400640 1 -- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d406d420 msgr2=0x7f23d41e0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23d3400640 1 -- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d406d420 msgr2=0x7f23d41e0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23d3400640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d406d420 0x7f23d41e0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23d3400640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d406d420 0x7f23d41e0fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23cbe00640 1 -- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d414c9f0 msgr2=0x7f23d414ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23cbe00640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d414c9f0 0x7f23d414ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23cbe00640 1 -- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d406d420 msgr2=0x7f23d41e0fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:55:39.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23cbe00640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d406d420 0x7f23d41e0fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23cbe00640 1 -- 172.21.15.67:0/2135113021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40c0ff0 con 0x7f23d40c3080 2024-03-09T08:55:39.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23d3e00640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d414c9f0 0x7f23d414ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:55:39.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.160+0000 7f23cbe00640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d40c3080 0x7f23d40ab990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009890 tx=0x7f23bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:39.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.161+0000 7f23cb400640 1 -- 172.21.15.67:0/2135113021 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc041070 con 0x7f23d40c3080 2024-03-09T08:55:39.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.161+0000 7f23cb400640 1 -- 172.21.15.67:0/2135113021 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc03aa30 con 0x7f23d40c3080 2024-03-09T08:55:39.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.161+0000 7f23cb400640 1 -- 172.21.15.67:0/2135113021 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc035040 con 0x7f23d40c3080 2024-03-09T08:55:39.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.161+0000 7f23d8c00640 1 -- 172.21.15.67:0/2135113021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d414f980 con 0x7f23d40c3080 2024-03-09T08:55:39.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.161+0000 7f23d8c00640 1 -- 172.21.15.67:0/2135113021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d414ff40 con 0x7f23d40c3080 2024-03-09T08:55:39.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.162+0000 7f23d8c00640 1 -- 172.21.15.67:0/2135113021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d404f3c0 con 0x7f23d40c3080 2024-03-09T08:55:39.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.164+0000 7f23cb400640 1 -- 172.21.15.67:0/2135113021 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f23bc04c060 con 0x7f23d40c3080 2024-03-09T08:55:39.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.165+0000 7f23cb400640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b500 0x7f23b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.166+0000 7f23cb400640 1 -- 172.21.15.67:0/2135113021 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f23bc0ac0c0 con 0x7f23d40c3080 2024-03-09T08:55:39.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.166+0000 7f23d3400640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b500 0x7f23b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.167+0000 7f23d3400640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b500 0x7f23b006d9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23c4005880 tx=0x7f23c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:39.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.171+0000 7f23cb400640 1 -- 172.21.15.67:0/2135113021 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f23bc07aae0 con 0x7f23d40c3080 2024-03-09T08:55:39.246 INFO:tasks.workunit.client.1.smithi134.stdout: create 266/30 file after the snapshot 2024-03-09T08:55:39.248 INFO:tasks.workunit.client.1.smithi134.stdout: create 266/31 file after the snapshot 2024-03-09T08:55:39.248 INFO:tasks.workunit.client.1.smithi134.stdout: create 266/32 file after the snapshot 2024-03-09T08:55:39.249 INFO:tasks.workunit.client.1.smithi134.stdout: create 266/33 file after the snapshot 2024-03-09T08:55:39.249 INFO:tasks.workunit.client.1.smithi134.stdout: create 266/34 file after the snapshot 2024-03-09T08:55:39.249 INFO:tasks.workunit.client.1.smithi134.stdout: create 266/35 file after the snapshot 2024-03-09T08:55:39.250 INFO:tasks.workunit.client.1.smithi134.stdout: create 266/36 file after the snapshot 2024-03-09T08:55:39.250 INFO:tasks.workunit.client.1.smithi134.stdout: create 266/37 file after the snapshot 2024-03-09T08:55:39.251 INFO:tasks.workunit.client.1.smithi134.stdout: create 266/38 file after the snapshot 2024-03-09T08:55:39.251 INFO:tasks.workunit.client.1.smithi134.stdout: create 266/39 file after the snapshot 2024-03-09T08:55:39.252 INFO:tasks.workunit.client.1.smithi134.stdout: create 267/.snap/snap-subdir-test 2024-03-09T08:55:39.315 INFO:tasks.workunit.client.1.smithi134.stdout: create 267/30 file after the snapshot 2024-03-09T08:55:39.318 INFO:tasks.workunit.client.1.smithi134.stdout: create 267/31 file after the snapshot 2024-03-09T08:55:39.318 INFO:tasks.workunit.client.1.smithi134.stdout: create 267/32 file after the snapshot 2024-03-09T08:55:39.319 INFO:tasks.workunit.client.1.smithi134.stdout: create 267/33 file after the snapshot 2024-03-09T08:55:39.319 INFO:tasks.workunit.client.1.smithi134.stdout: create 267/34 file after the snapshot 2024-03-09T08:55:39.320 INFO:tasks.workunit.client.1.smithi134.stdout: create 267/35 file after the snapshot 2024-03-09T08:55:39.320 INFO:tasks.workunit.client.1.smithi134.stdout: create 267/36 file after the snapshot 2024-03-09T08:55:39.320 INFO:tasks.workunit.client.1.smithi134.stdout: create 267/37 file after the snapshot 2024-03-09T08:55:39.321 INFO:tasks.workunit.client.1.smithi134.stdout: create 267/38 file after the snapshot 2024-03-09T08:55:39.322 INFO:tasks.workunit.client.1.smithi134.stdout: create 267/39 file after the snapshot 2024-03-09T08:55:39.322 INFO:tasks.workunit.client.1.smithi134.stdout: create 268/.snap/snap-subdir-test 2024-03-09T08:55:39.404 INFO:tasks.workunit.client.1.smithi134.stdout: create 268/30 file after the snapshot 2024-03-09T08:55:39.406 INFO:tasks.workunit.client.1.smithi134.stdout: create 268/31 file after the snapshot 2024-03-09T08:55:39.407 INFO:tasks.workunit.client.1.smithi134.stdout: create 268/32 file after the snapshot 2024-03-09T08:55:39.407 INFO:tasks.workunit.client.1.smithi134.stdout: create 268/33 file after the snapshot 2024-03-09T08:55:39.407 INFO:tasks.workunit.client.1.smithi134.stdout: create 268/34 file after the snapshot 2024-03-09T08:55:39.408 INFO:tasks.workunit.client.1.smithi134.stdout: create 268/35 file after the snapshot 2024-03-09T08:55:39.408 INFO:tasks.workunit.client.1.smithi134.stdout: create 268/36 file after the snapshot 2024-03-09T08:55:39.409 INFO:tasks.workunit.client.1.smithi134.stdout: create 268/37 file after the snapshot 2024-03-09T08:55:39.409 INFO:tasks.workunit.client.1.smithi134.stdout: create 268/38 file after the snapshot 2024-03-09T08:55:39.409 INFO:tasks.workunit.client.1.smithi134.stdout: create 268/39 file after the snapshot 2024-03-09T08:55:39.410 INFO:tasks.workunit.client.1.smithi134.stdout: create 269/.snap/snap-subdir-test 2024-03-09T08:55:39.470 INFO:tasks.workunit.client.0.smithi067.stdout: create 239/30 file after the snapshot 2024-03-09T08:55:39.473 INFO:tasks.workunit.client.0.smithi067.stdout: create 239/31 file after the snapshot 2024-03-09T08:55:39.473 INFO:tasks.workunit.client.0.smithi067.stdout: create 239/32 file after the snapshot 2024-03-09T08:55:39.474 INFO:tasks.workunit.client.0.smithi067.stdout: create 239/33 file after the snapshot 2024-03-09T08:55:39.474 INFO:tasks.workunit.client.0.smithi067.stdout: create 239/34 file after the snapshot 2024-03-09T08:55:39.475 INFO:tasks.workunit.client.0.smithi067.stdout: create 239/35 file after the snapshot 2024-03-09T08:55:39.475 INFO:tasks.workunit.client.0.smithi067.stdout: create 239/36 file after the snapshot 2024-03-09T08:55:39.475 INFO:tasks.workunit.client.0.smithi067.stdout: create 239/37 file after the snapshot 2024-03-09T08:55:39.476 INFO:tasks.workunit.client.0.smithi067.stdout: create 239/38 file after the snapshot 2024-03-09T08:55:39.476 INFO:tasks.workunit.client.0.smithi067.stdout: create 239/39 file after the snapshot 2024-03-09T08:55:39.477 INFO:tasks.workunit.client.0.smithi067.stdout: create 240/.snap/snap-subdir-test 2024-03-09T08:55:39.481 INFO:tasks.workunit.client.1.smithi134.stdout: create 269/30 file after the snapshot 2024-03-09T08:55:39.484 INFO:tasks.workunit.client.1.smithi134.stdout: create 269/31 file after the snapshot 2024-03-09T08:55:39.484 INFO:tasks.workunit.client.1.smithi134.stdout: create 269/32 file after the snapshot 2024-03-09T08:55:39.484 INFO:tasks.workunit.client.1.smithi134.stdout: create 269/33 file after the snapshot 2024-03-09T08:55:39.485 INFO:tasks.workunit.client.1.smithi134.stdout: create 269/34 file after the snapshot 2024-03-09T08:55:39.485 INFO:tasks.workunit.client.1.smithi134.stdout: create 269/35 file after the snapshot 2024-03-09T08:55:39.485 INFO:tasks.workunit.client.1.smithi134.stdout: create 269/36 file after the snapshot 2024-03-09T08:55:39.485 INFO:tasks.workunit.client.1.smithi134.stdout: create 269/37 file after the snapshot 2024-03-09T08:55:39.485 INFO:tasks.workunit.client.1.smithi134.stdout: create 269/38 file after the snapshot 2024-03-09T08:55:39.486 INFO:tasks.workunit.client.1.smithi134.stdout: create 269/39 file after the snapshot 2024-03-09T08:55:39.486 INFO:tasks.workunit.client.1.smithi134.stdout: create 270/.snap/snap-subdir-test 2024-03-09T08:55:39.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.518+0000 7f23d8c00640 1 -- 172.21.15.67:0/2135113021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f23d41519f0 con 0x7f23d40c3080 2024-03-09T08:55:39.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.519+0000 7f23cb400640 1 -- 172.21.15.67:0/2135113021 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7f23bc07a3e0 con 0x7f23d40c3080 2024-03-09T08:55:39.517 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:55:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.522+0000 7f23d8c00640 1 -- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b500 msgr2=0x7f23b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.522+0000 7f23d8c00640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b500 0x7f23b006d9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f23c4005880 tx=0x7f23c4009040 comp rx=0 tx=0).stop 2024-03-09T08:55:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.522+0000 7f23d8c00640 1 -- 172.21.15.67:0/2135113021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d40c3080 msgr2=0x7f23d40ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.522+0000 7f23d8c00640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d40c3080 0x7f23d40ab990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009890 tx=0x7f23bc0046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.522+0000 7f23d8c00640 1 -- 172.21.15.67:0/2135113021 shutdown_connections 2024-03-09T08:55:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.522+0000 7f23d8c00640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b500 0x7f23b006d9c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.522+0000 7f23d8c00640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d414c9f0 0x7f23d414ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.522+0000 7f23d8c00640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d40c3080 0x7f23d40ab990 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.522+0000 7f23d8c00640 1 --2- 172.21.15.67:0/2135113021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d406d420 0x7f23d41e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.522+0000 7f23d8c00640 1 -- 172.21.15.67:0/2135113021 >> 172.21.15.67:0/2135113021 conn(0x7f23d40c7300 msgr2=0x7f23d40c1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:39.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.522+0000 7f23d8c00640 1 -- 172.21.15.67:0/2135113021 shutdown_connections 2024-03-09T08:55:39.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.523+0000 7f23d8c00640 1 -- 172.21.15.67:0/2135113021 wait complete. 2024-03-09T08:55:39.541 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:55:39.541 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:55:39.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:55:39.555 INFO:tasks.workunit.client.1.smithi134.stdout: create 270/30 file after the snapshot 2024-03-09T08:55:39.557 INFO:tasks.workunit.client.1.smithi134.stdout: create 270/31 file after the snapshot 2024-03-09T08:55:39.557 INFO:tasks.workunit.client.1.smithi134.stdout: create 270/32 file after the snapshot 2024-03-09T08:55:39.557 INFO:tasks.workunit.client.1.smithi134.stdout: create 270/33 file after the snapshot 2024-03-09T08:55:39.558 INFO:tasks.workunit.client.1.smithi134.stdout: create 270/34 file after the snapshot 2024-03-09T08:55:39.558 INFO:tasks.workunit.client.1.smithi134.stdout: create 270/35 file after the snapshot 2024-03-09T08:55:39.558 INFO:tasks.workunit.client.1.smithi134.stdout: create 270/36 file after the snapshot 2024-03-09T08:55:39.559 INFO:tasks.workunit.client.1.smithi134.stdout: create 270/37 file after the snapshot 2024-03-09T08:55:39.559 INFO:tasks.workunit.client.1.smithi134.stdout: create 270/38 file after the snapshot 2024-03-09T08:55:39.559 INFO:tasks.workunit.client.1.smithi134.stdout: create 270/39 file after the snapshot 2024-03-09T08:55:39.560 INFO:tasks.workunit.client.1.smithi134.stdout: create 271/.snap/snap-subdir-test 2024-03-09T08:55:39.604 INFO:tasks.workunit.client.1.smithi134.stdout: create 271/30 file after the snapshot 2024-03-09T08:55:39.606 INFO:tasks.workunit.client.1.smithi134.stdout: create 271/31 file after the snapshot 2024-03-09T08:55:39.606 INFO:tasks.workunit.client.1.smithi134.stdout: create 271/32 file after the snapshot 2024-03-09T08:55:39.607 INFO:tasks.workunit.client.1.smithi134.stdout: create 271/33 file after the snapshot 2024-03-09T08:55:39.607 INFO:tasks.workunit.client.1.smithi134.stdout: create 271/34 file after the snapshot 2024-03-09T08:55:39.607 INFO:tasks.workunit.client.1.smithi134.stdout: create 271/35 file after the snapshot 2024-03-09T08:55:39.608 INFO:tasks.workunit.client.1.smithi134.stdout: create 271/36 file after the snapshot 2024-03-09T08:55:39.608 INFO:tasks.workunit.client.1.smithi134.stdout: create 271/37 file after the snapshot 2024-03-09T08:55:39.608 INFO:tasks.workunit.client.1.smithi134.stdout: create 271/38 file after the snapshot 2024-03-09T08:55:39.608 INFO:tasks.workunit.client.1.smithi134.stdout: create 271/39 file after the snapshot 2024-03-09T08:55:39.609 INFO:tasks.workunit.client.1.smithi134.stdout: create 272/.snap/snap-subdir-test 2024-03-09T08:55:39.672 INFO:tasks.workunit.client.1.smithi134.stdout: create 272/30 file after the snapshot 2024-03-09T08:55:39.675 INFO:tasks.workunit.client.1.smithi134.stdout: create 272/31 file after the snapshot 2024-03-09T08:55:39.675 INFO:tasks.workunit.client.1.smithi134.stdout: create 272/32 file after the snapshot 2024-03-09T08:55:39.675 INFO:tasks.workunit.client.1.smithi134.stdout: create 272/33 file after the snapshot 2024-03-09T08:55:39.676 INFO:tasks.workunit.client.1.smithi134.stdout: create 272/34 file after the snapshot 2024-03-09T08:55:39.676 INFO:tasks.workunit.client.1.smithi134.stdout: create 272/35 file after the snapshot 2024-03-09T08:55:39.676 INFO:tasks.workunit.client.1.smithi134.stdout: create 272/36 file after the snapshot 2024-03-09T08:55:39.677 INFO:tasks.workunit.client.1.smithi134.stdout: create 272/37 file after the snapshot 2024-03-09T08:55:39.677 INFO:tasks.workunit.client.1.smithi134.stdout: create 272/38 file after the snapshot 2024-03-09T08:55:39.678 INFO:tasks.workunit.client.1.smithi134.stdout: create 272/39 file after the snapshot 2024-03-09T08:55:39.678 INFO:tasks.workunit.client.1.smithi134.stdout: create 273/.snap/snap-subdir-test 2024-03-09T08:55:39.747 INFO:tasks.workunit.client.1.smithi134.stdout: create 273/30 file after the snapshot 2024-03-09T08:55:39.749 INFO:tasks.workunit.client.1.smithi134.stdout: create 273/31 file after the snapshot 2024-03-09T08:55:39.750 INFO:tasks.workunit.client.1.smithi134.stdout: create 273/32 file after the snapshot 2024-03-09T08:55:39.750 INFO:tasks.workunit.client.1.smithi134.stdout: create 273/33 file after the snapshot 2024-03-09T08:55:39.751 INFO:tasks.workunit.client.1.smithi134.stdout: create 273/34 file after the snapshot 2024-03-09T08:55:39.751 INFO:tasks.workunit.client.1.smithi134.stdout: create 273/35 file after the snapshot 2024-03-09T08:55:39.751 INFO:tasks.workunit.client.1.smithi134.stdout: create 273/36 file after the snapshot 2024-03-09T08:55:39.751 INFO:tasks.workunit.client.1.smithi134.stdout: create 273/37 file after the snapshot 2024-03-09T08:55:39.751 INFO:tasks.workunit.client.1.smithi134.stdout: create 273/38 file after the snapshot 2024-03-09T08:55:39.752 INFO:tasks.workunit.client.1.smithi134.stdout: create 273/39 file after the snapshot 2024-03-09T08:55:39.752 INFO:tasks.workunit.client.1.smithi134.stdout: create 274/.snap/snap-subdir-test 2024-03-09T08:55:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.758+0000 7f470a800640 1 Processor -- start 2024-03-09T08:55:39.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.758+0000 7f470a800640 1 -- start start 2024-03-09T08:55:39.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.759+0000 7f470a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f470414fd90 0x7f4704156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.759+0000 7f470a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4704156bc0 con 0x7f47040c9380 2024-03-09T08:55:39.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.759+0000 7f470a800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47040c9890 con 0x7f470406c1d0 2024-03-09T08:55:39.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.759+0000 7f470a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470406cd00 con 0x7f470414fd90 2024-03-09T08:55:39.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.760+0000 7f4709400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f470414fd90 0x7f4704156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.760+0000 7f4709400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f470414fd90 0x7f4704156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37910/0 (socket says 172.21.15.67:37910) 2024-03-09T08:55:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.760+0000 7f4709400640 1 -- 172.21.15.67:0/2230675648 learned_addr learned my addr 172.21.15.67:0/2230675648 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.760+0000 7f4708a00640 1 --1- 172.21.15.67:0/2230675648 >> v1:172.21.15.67:6789/0 conn(0x7f47040c9380 0x7f47040acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60336/0 (socket says 172.21.15.67:60336) 2024-03-09T08:55:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.760+0000 7f4709400640 1 -- 172.21.15.67:0/2230675648 >> v1:172.21.15.134:6789/0 conn(0x7f470406c1d0 legacy=0x7f470406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.760+0000 7f4709400640 1 -- 172.21.15.67:0/2230675648 >> v1:172.21.15.67:6789/0 conn(0x7f47040c9380 legacy=0x7f47040acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.760+0000 7f4709400640 1 -- 172.21.15.67:0/2230675648 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47040c5f80 con 0x7f470414fd90 2024-03-09T08:55:39.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.762+0000 7f4709400640 1 --2- 172.21.15.67:0/2230675648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470414fd90 0x7f4704156680 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009870 tx=0x7f46f802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e7d64e4bb3789 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.763+0000 7f4703e00640 1 -- 172.21.15.67:0/2230675648 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8035020 con 0x7f470414fd90 2024-03-09T08:55:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.763+0000 7f4703e00640 1 -- 172.21.15.67:0/2230675648 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f8031880 con 0x7f470414fd90 2024-03-09T08:55:39.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.763+0000 7f4703e00640 1 -- 172.21.15.67:0/2230675648 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8031c50 con 0x7f470414fd90 2024-03-09T08:55:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.764+0000 7f470a800640 1 -- 172.21.15.67:0/2230675648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470414fd90 msgr2=0x7f4704156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.764+0000 7f470a800640 1 --2- 172.21.15.67:0/2230675648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470414fd90 0x7f4704156680 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009870 tx=0x7f46f802b920 comp rx=0 tx=0).stop 2024-03-09T08:55:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.764+0000 7f470a800640 1 -- 172.21.15.67:0/2230675648 shutdown_connections 2024-03-09T08:55:39.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.764+0000 7f470a800640 1 --2- 172.21.15.67:0/2230675648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470414fd90 0x7f4704156680 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.764+0000 7f470a800640 1 -- 172.21.15.67:0/2230675648 >> 172.21.15.67:0/2230675648 conn(0x7f47040cb480 msgr2=0x7f47040cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:39.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.764+0000 7f470a800640 1 -- 172.21.15.67:0/2230675648 shutdown_connections 2024-03-09T08:55:39.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.764+0000 7f470a800640 1 -- 172.21.15.67:0/2230675648 wait complete. 2024-03-09T08:55:39.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.766+0000 7f470a800640 1 Processor -- start 2024-03-09T08:55:39.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.766+0000 7f470a800640 1 -- start start 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.768+0000 7f470a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406c1d0 0x7f470414ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.768+0000 7f470a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47040c9380 0x7f4704148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.768+0000 7f470a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470414fd90 0x7f4704158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.768+0000 7f470a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47040c6520 con 0x7f470414fd90 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.768+0000 7f470a800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470406ba60 con 0x7f47040c9380 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.768+0000 7f470a800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470406bcb0 con 0x7f470406c1d0 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.768+0000 7f4709e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470414fd90 0x7f4704158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.768+0000 7f4709e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470414fd90 0x7f4704158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37916/0 (socket says 172.21.15.67:37916) 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.768+0000 7f4709e00640 1 -- 172.21.15.67:0/972463416 learned_addr learned my addr 172.21.15.67:0/972463416 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.768+0000 7f4709400640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406c1d0 0x7f470414ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.768+0000 7f4708a00640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47040c9380 0x7f4704148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.769+0000 7f4709e00640 1 -- 172.21.15.67:0/972463416 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406c1d0 msgr2=0x7f470414ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.769+0000 7f4709e00640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470406c1d0 0x7f470414ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.769+0000 7f4709e00640 1 -- 172.21.15.67:0/972463416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47040c9380 msgr2=0x7f4704148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.769+0000 7f4709e00640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47040c9380 0x7f4704148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.769+0000 7f4709e00640 1 -- 172.21.15.67:0/972463416 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4704159a30 con 0x7f470414fd90 2024-03-09T08:55:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.769+0000 7f4709e00640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470414fd90 0x7f4704158000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00c6e0 tx=0x7f46fc00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.769+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc003b70 con 0x7f470414fd90 2024-03-09T08:55:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.769+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46fc003d30 con 0x7f470414fd90 2024-03-09T08:55:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.769+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc005310 con 0x7f470414fd90 2024-03-09T08:55:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.769+0000 7f470a800640 1 -- 172.21.15.67:0/972463416 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47041efb80 con 0x7f470414fd90 2024-03-09T08:55:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.770+0000 7f470a800640 1 -- 172.21.15.67:0/972463416 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47041ef8a0 con 0x7f470414fd90 2024-03-09T08:55:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.771+0000 7f470a800640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4704148aa0 0x7f47041eba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.771+0000 7f470a800640 1 -- 172.21.15.67:0/972463416 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470404f3c0 con 0x7f4704148aa0 2024-03-09T08:55:39.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.771+0000 7f4709400640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4704148aa0 0x7f47041eba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.771+0000 7f4709400640 1 -- 172.21.15.67:0/972463416 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47041ebfd0 con 0x7f4704148aa0 2024-03-09T08:55:39.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.772+0000 7f4709400640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4704148aa0 0x7f47041eba90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f47040ae040 tx=0x7f46f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:39.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.772+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f46fc0054b0 con 0x7f470414fd90 2024-03-09T08:55:39.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.773+0000 7f4701600640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46e406b4d0 0x7f46e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.773+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f46fc08a320 con 0x7f470414fd90 2024-03-09T08:55:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.773+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8035040 con 0x7f4704148aa0 2024-03-09T08:55:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.773+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f46f803abb0 con 0x7f4704148aa0 2024-03-09T08:55:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.773+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4704148aa0 msgr2=0x7f47041eba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.773+0000 7f4701600640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4704148aa0 0x7f47041eba90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f47040ae040 tx=0x7f46f8004730 comp rx=0 tx=0).stop 2024-03-09T08:55:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.773+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4704148aa0 msgr2=0x7f47041eba90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.773+0000 7f4701600640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4704148aa0 0x7f47041eba90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.774+0000 7f4708a00640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46e406b4d0 0x7f46e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.775+0000 7f4708a00640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46e406b4d0 0x7f46e406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f470415a310 tx=0x7f46f0007ce0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:39.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.781+0000 7f470a800640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47040b0f70 0x7f4704158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:39.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.781+0000 7f470a800640 1 -- 172.21.15.67:0/972463416 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4704153560 con 0x7f47040b0f70 2024-03-09T08:55:39.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.781+0000 7f4709400640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47040b0f70 0x7f4704158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:39.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.781+0000 7f4709400640 1 -- 172.21.15.67:0/972463416 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4704073a30 con 0x7f47040b0f70 2024-03-09T08:55:39.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.782+0000 7f4709400640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47040b0f70 0x7f4704158540 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f46f8032000 tx=0x7f46f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:39.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.782+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8035030 con 0x7f47040b0f70 2024-03-09T08:55:39.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.782+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f46f804ba80 con 0x7f47040b0f70 2024-03-09T08:55:39.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.782+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47040b0f70 msgr2=0x7f4704158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.782+0000 7f4701600640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47040b0f70 0x7f4704158540 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f46f8032000 tx=0x7f46f80046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.782+0000 7f4701600640 1 -- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47040b0f70 msgr2=0x7f4704158540 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:39.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.782+0000 7f4701600640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47040b0f70 0x7f4704158540 secure :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f46f8032000 tx=0x7f46f80046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.781 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:55:39.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:39.781 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:39.781 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:55:39.781 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 28, 2024-03-09T08:55:39.781 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:39.782 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:39.783 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:55:39.784 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:39.785 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.786 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:55:39.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.787 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:39.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:39.787 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:39.787 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:39.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.787 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:39.787 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:55:39.787 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:39.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.788 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:39.788 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:55:39.788 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:39.788 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:39.788 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:39.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.788 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:55:39.788 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:55:39.788 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:39.788 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:39.788 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:39.790 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:39.791 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:39.792 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:39.793 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:39.794 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:55:39.795 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:55:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.783+0000 7f470a800640 1 -- 172.21.15.67:0/972463416 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46e406b4d0 msgr2=0x7f46e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.783+0000 7f470a800640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46e406b4d0 0x7f46e406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f470415a310 tx=0x7f46f0007ce0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.783+0000 7f470a800640 1 -- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470414fd90 msgr2=0x7f4704158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.783+0000 7f470a800640 1 --2- 172.21.15.67:0/972463416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470414fd90 0x7f4704158000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00c6e0 tx=0x7f46fc00cba0 comp rx=0 tx=0).stop 2024-03-09T08:55:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.783+0000 7f4709e00640 1 -- 172.21.15.67:0/972463416 reap_dead start 2024-03-09T08:55:39.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.783+0000 7f4709e00640 1 -- 172.21.15.67:0/972463416 reap_dead start 2024-03-09T08:55:39.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.783+0000 7f470a800640 1 -- 172.21.15.67:0/972463416 shutdown_connections 2024-03-09T08:55:39.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.783+0000 7f470a800640 1 -- 172.21.15.67:0/972463416 >> 172.21.15.67:0/972463416 conn(0x7f47040cb480 msgr2=0x7f4704154870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:39.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.783+0000 7f470a800640 1 -- 172.21.15.67:0/972463416 shutdown_connections 2024-03-09T08:55:39.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:39.783+0000 7f470a800640 1 -- 172.21.15.67:0/972463416 wait complete. 2024-03-09T08:55:39.804 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:55:39.804 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:55:39.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:55:39.878 INFO:tasks.workunit.client.0.smithi067.stdout: create 240/30 file after the snapshot 2024-03-09T08:55:39.880 INFO:tasks.workunit.client.0.smithi067.stdout: create 240/31 file after the snapshot 2024-03-09T08:55:39.880 INFO:tasks.workunit.client.0.smithi067.stdout: create 240/32 file after the snapshot 2024-03-09T08:55:39.881 INFO:tasks.workunit.client.0.smithi067.stdout: create 240/33 file after the snapshot 2024-03-09T08:55:39.881 INFO:tasks.workunit.client.0.smithi067.stdout: create 240/34 file after the snapshot 2024-03-09T08:55:39.881 INFO:tasks.workunit.client.0.smithi067.stdout: create 240/35 file after the snapshot 2024-03-09T08:55:39.881 INFO:tasks.workunit.client.0.smithi067.stdout: create 240/36 file after the snapshot 2024-03-09T08:55:39.882 INFO:tasks.workunit.client.0.smithi067.stdout: create 240/37 file after the snapshot 2024-03-09T08:55:39.882 INFO:tasks.workunit.client.0.smithi067.stdout: create 240/38 file after the snapshot 2024-03-09T08:55:39.882 INFO:tasks.workunit.client.0.smithi067.stdout: create 240/39 file after the snapshot 2024-03-09T08:55:39.882 INFO:tasks.workunit.client.0.smithi067.stdout: create 241/.snap/snap-subdir-test 2024-03-09T08:55:39.965 INFO:tasks.workunit.client.0.smithi067.stdout: create 241/30 file after the snapshot 2024-03-09T08:55:39.967 INFO:tasks.workunit.client.0.smithi067.stdout: create 241/31 file after the snapshot 2024-03-09T08:55:39.968 INFO:tasks.workunit.client.0.smithi067.stdout: create 241/32 file after the snapshot 2024-03-09T08:55:39.968 INFO:tasks.workunit.client.0.smithi067.stdout: create 241/33 file after the snapshot 2024-03-09T08:55:39.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 241/34 file after the snapshot 2024-03-09T08:55:39.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 241/35 file after the snapshot 2024-03-09T08:55:39.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 241/36 file after the snapshot 2024-03-09T08:55:39.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 241/37 file after the snapshot 2024-03-09T08:55:39.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 241/38 file after the snapshot 2024-03-09T08:55:39.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 241/39 file after the snapshot 2024-03-09T08:55:39.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 242/.snap/snap-subdir-test 2024-03-09T08:55:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.013+0000 7fa0fb200640 1 Processor -- start 2024-03-09T08:55:40.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.013+0000 7fa0fb200640 1 -- start start 2024-03-09T08:55:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.016+0000 7fa0fb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa0f4154ed0 0x7fa0f4155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.016+0000 7fa0fb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f414dd10 con 0x7fa0f414d930 2024-03-09T08:55:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.016+0000 7fa0fb200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f4157070 con 0x7fa0f40c4790 2024-03-09T08:55:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.016+0000 7fa0fb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f4157c80 con 0x7fa0f4154ed0 2024-03-09T08:55:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.016+0000 7fa0f9e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa0f414d930 0x7fa0f41542b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60344/0 (socket says 172.21.15.67:60344) 2024-03-09T08:55:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.016+0000 7fa0f9e00640 1 -- 172.21.15.67:0/966412813 learned_addr learned my addr 172.21.15.67:0/966412813 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.017+0000 7fa0fa800640 1 --2- 172.21.15.67:0/966412813 >> v2:172.21.15.67:3300/0 conn(0x7fa0f4154ed0 0x7fa0f4155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.017+0000 7fa0f8a00640 1 -- 172.21.15.67:0/966412813 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591482450 0 0) 0x7fa0e8001110 con 0x7fa0f414d930 2024-03-09T08:55:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.017+0000 7fa0f8a00640 1 -- 172.21.15.67:0/966412813 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0dc003520 con 0x7fa0f414d930 2024-03-09T08:55:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.017+0000 7fa0fa800640 1 -- 172.21.15.67:0/966412813 >> v1:172.21.15.134:6789/0 conn(0x7fa0f40c4790 legacy=0x7fa0f406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.017+0000 7fa0fa800640 1 -- 172.21.15.67:0/966412813 >> v1:172.21.15.67:6789/0 conn(0x7fa0f414d930 legacy=0x7fa0f41542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.017+0000 7fa0fa800640 1 -- 172.21.15.67:0/966412813 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f4157350 con 0x7fa0f4154ed0 2024-03-09T08:55:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.018+0000 7fa0fa800640 1 --2- 172.21.15.67:0/966412813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f4154ed0 0x7fa0f4155380 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009d90 tx=0x7fa0f002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ea5945996fc666 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.019+0000 7fa0f8a00640 1 -- 172.21.15.67:0/966412813 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0039070 con 0x7fa0f4154ed0 2024-03-09T08:55:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.019+0000 7fa0f8a00640 1 -- 172.21.15.67:0/966412813 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f002cc60 con 0x7fa0f4154ed0 2024-03-09T08:55:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.019+0000 7fa0f8a00640 1 -- 172.21.15.67:0/966412813 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0034660 con 0x7fa0f4154ed0 2024-03-09T08:55:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.020+0000 7fa0fb200640 1 -- 172.21.15.67:0/966412813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f4154ed0 msgr2=0x7fa0f4155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.020+0000 7fa0fb200640 1 --2- 172.21.15.67:0/966412813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f4154ed0 0x7fa0f4155380 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009d90 tx=0x7fa0f002beb0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.020+0000 7fa0fb200640 1 -- 172.21.15.67:0/966412813 shutdown_connections 2024-03-09T08:55:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.020+0000 7fa0fb200640 1 --2- 172.21.15.67:0/966412813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f4154ed0 0x7fa0f4155380 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.020+0000 7fa0fb200640 1 -- 172.21.15.67:0/966412813 >> 172.21.15.67:0/966412813 conn(0x7fa0f40c8480 msgr2=0x7fa0f40c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.020+0000 7fa0fb200640 1 -- 172.21.15.67:0/966412813 shutdown_connections 2024-03-09T08:55:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.020+0000 7fa0fb200640 1 -- 172.21.15.67:0/966412813 wait complete. 2024-03-09T08:55:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.022+0000 7fa0fb200640 1 Processor -- start 2024-03-09T08:55:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.022+0000 7fa0fb200640 1 -- start start 2024-03-09T08:55:40.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.023+0000 7fa0fb200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0f40c4790 0x7fa0f4150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.023+0000 7fa0fb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f414d930 0x7fa0f414f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.023+0000 7fa0fb200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f4154ed0 0x7fa0f4142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.023+0000 7fa0fb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f4157910 con 0x7fa0f414d930 2024-03-09T08:55:40.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.023+0000 7fa0fb200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f4156e20 con 0x7fa0f4154ed0 2024-03-09T08:55:40.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.023+0000 7fa0fb200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f406b750 con 0x7fa0f40c4790 2024-03-09T08:55:40.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.023+0000 7fa0f9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f414d930 0x7fa0f414f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.023+0000 7fa0f9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f414d930 0x7fa0f414f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37962/0 (socket says 172.21.15.67:37962) 2024-03-09T08:55:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.023+0000 7fa0f9400640 1 -- 172.21.15.67:0/1349940116 learned_addr learned my addr 172.21.15.67:0/1349940116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.023+0000 7fa0f9e00640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0f40c4790 0x7fa0f4150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.023+0000 7fa0fa800640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f4154ed0 0x7fa0f4142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.024+0000 7fa0f9400640 1 -- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0f40c4790 msgr2=0x7fa0f4150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.024+0000 7fa0f9400640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0f40c4790 0x7fa0f4150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.024+0000 7fa0f9400640 1 -- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f4154ed0 msgr2=0x7fa0f4142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.024+0000 7fa0f9400640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f4154ed0 0x7fa0f4142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.024+0000 7fa0f9400640 1 -- 172.21.15.67:0/1349940116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f41ec390 con 0x7fa0f414d930 2024-03-09T08:55:40.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.024+0000 7fa0f9400640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f414d930 0x7fa0f414f0c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c400 tx=0x7fa0e400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.024+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4017070 con 0x7fa0f414d930 2024-03-09T08:55:40.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.024+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e40052d0 con 0x7fa0f414d930 2024-03-09T08:55:40.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.024+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4003830 con 0x7fa0f414d930 2024-03-09T08:55:40.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.024+0000 7fa0fb200640 1 -- 172.21.15.67:0/1349940116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f41ec610 con 0x7fa0f414d930 2024-03-09T08:55:40.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.025+0000 7fa0fb200640 1 -- 172.21.15.67:0/1349940116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f40c7f80 con 0x7fa0f414d930 2024-03-09T08:55:40.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.026+0000 7fa0fb200640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414f600 0x7fa0f4142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.026+0000 7fa0fb200640 1 -- 172.21.15.67:0/1349940116 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f404f3c0 con 0x7fa0f414f600 2024-03-09T08:55:40.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.027+0000 7fa0f9e00640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414f600 0x7fa0f4142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.027+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa0e401a070 con 0x7fa0f414d930 2024-03-09T08:55:40.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.028+0000 7fa0f9e00640 1 -- 172.21.15.67:0/1349940116 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0f40c4490 con 0x7fa0f414f600 2024-03-09T08:55:40.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.028+0000 7fa0ee000640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0d806b500 0x7fa0d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.028+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fa0e40898c0 con 0x7fa0f414d930 2024-03-09T08:55:40.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.028+0000 7fa0f9e00640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414f600 0x7fa0f4142b70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80012b0 tx=0x7fa0e8001b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.029+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8017020 con 0x7fa0f414f600 2024-03-09T08:55:40.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.029+0000 7fa0fa800640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0d806b500 0x7fa0d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.030+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa0e800f7a0 con 0x7fa0f414f600 2024-03-09T08:55:40.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.030+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414f600 msgr2=0x7fa0f4142b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.030+0000 7fa0ee000640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414f600 0x7fa0f4142b70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80012b0 tx=0x7fa0e8001b60 comp rx=0 tx=0).stop 2024-03-09T08:55:40.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.030+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414f600 msgr2=0x7fa0f4142b70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:40.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.030+0000 7fa0ee000640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414f600 0x7fa0f4142b70 secure :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80012b0 tx=0x7fa0e8001b60 comp rx=0 tx=0).stop 2024-03-09T08:55:40.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.030+0000 7fa0fa800640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0d806b500 0x7fa0d806d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0036000 tx=0x7fa0f0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.042+0000 7fa0fb200640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414def0 0x7fa0f414e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.042+0000 7fa0fb200640 1 -- 172.21.15.67:0/1349940116 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f40c33f0 con 0x7fa0f414def0 2024-03-09T08:55:40.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.043+0000 7fa0f9e00640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414def0 0x7fa0f414e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.044+0000 7fa0f9e00640 1 -- 172.21.15.67:0/1349940116 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa0f406b150 con 0x7fa0f414def0 2024-03-09T08:55:40.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.044+0000 7fa0f9e00640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414def0 0x7fa0f414e300 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8018000 tx=0x7fa0e8001320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.045+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8017070 con 0x7fa0f414def0 2024-03-09T08:55:40.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.045+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4533 (secure 0 0 0) 0x7fa0e8004bf0 con 0x7fa0f414def0 2024-03-09T08:55:40.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.045+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414def0 msgr2=0x7fa0f414e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.045+0000 7fa0ee000640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414def0 0x7fa0f414e300 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8018000 tx=0x7fa0e8001320 comp rx=0 tx=0).stop 2024-03-09T08:55:40.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.045+0000 7fa0ee000640 1 -- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414def0 msgr2=0x7fa0f414e300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:40.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.045+0000 7fa0ee000640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f414def0 0x7fa0f414e300 secure :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8018000 tx=0x7fa0e8001320 comp rx=0 tx=0).stop 2024-03-09T08:55:40.044 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:55:40.044 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 28, 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:55:40.045 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:55:40.046 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:55:40.048 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:40.049 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:55:40.050 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:40.051 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:40.052 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:40.053 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:40.055 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:55:40.056 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:55:40.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.046+0000 7fa0fb200640 1 -- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0d806b500 msgr2=0x7fa0d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.046+0000 7fa0fb200640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0d806b500 0x7fa0d806d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0036000 tx=0x7fa0f0030000 comp rx=0 tx=0).stop 2024-03-09T08:55:40.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.046+0000 7fa0fb200640 1 -- 172.21.15.67:0/1349940116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f414d930 msgr2=0x7fa0f414f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.046+0000 7fa0fb200640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f414d930 0x7fa0f414f0c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c400 tx=0x7fa0e400c8c0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.046+0000 7fa0fa800640 1 -- 172.21.15.67:0/1349940116 reap_dead start 2024-03-09T08:55:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.046+0000 7fa0fb200640 1 -- 172.21.15.67:0/1349940116 shutdown_connections 2024-03-09T08:55:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.046+0000 7fa0fb200640 1 --2- 172.21.15.67:0/1349940116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0f414d930 0x7fa0f414f0c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.046+0000 7fa0fb200640 1 -- 172.21.15.67:0/1349940116 >> 172.21.15.67:0/1349940116 conn(0x7fa0f40c8480 msgr2=0x7fa0f41516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.046+0000 7fa0fb200640 1 -- 172.21.15.67:0/1349940116 shutdown_connections 2024-03-09T08:55:40.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.046+0000 7fa0fb200640 1 -- 172.21.15.67:0/1349940116 wait complete. 2024-03-09T08:55:40.061 INFO:tasks.workunit.client.1.smithi134.stdout: create 274/30 file after the snapshot 2024-03-09T08:55:40.063 INFO:tasks.workunit.client.1.smithi134.stdout: create 274/31 file after the snapshot 2024-03-09T08:55:40.063 INFO:tasks.workunit.client.1.smithi134.stdout: create 274/32 file after the snapshot 2024-03-09T08:55:40.064 INFO:tasks.workunit.client.1.smithi134.stdout: create 274/33 file after the snapshot 2024-03-09T08:55:40.064 INFO:tasks.workunit.client.1.smithi134.stdout: create 274/34 file after the snapshot 2024-03-09T08:55:40.064 INFO:tasks.workunit.client.1.smithi134.stdout: create 274/35 file after the snapshot 2024-03-09T08:55:40.065 INFO:tasks.workunit.client.1.smithi134.stdout: create 274/36 file after the snapshot 2024-03-09T08:55:40.065 INFO:tasks.workunit.client.1.smithi134.stdout: create 274/37 file after the snapshot 2024-03-09T08:55:40.065 INFO:tasks.workunit.client.1.smithi134.stdout: create 274/38 file after the snapshot 2024-03-09T08:55:40.065 INFO:tasks.workunit.client.1.smithi134.stdout: create 274/39 file after the snapshot 2024-03-09T08:55:40.066 INFO:tasks.workunit.client.1.smithi134.stdout: create 275/.snap/snap-subdir-test 2024-03-09T08:55:40.067 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:55:40.067 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:55:40.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:55:40.120 INFO:tasks.workunit.client.1.smithi134.stdout: create 275/30 file after the snapshot 2024-03-09T08:55:40.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 275/31 file after the snapshot 2024-03-09T08:55:40.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 275/32 file after the snapshot 2024-03-09T08:55:40.122 INFO:tasks.workunit.client.1.smithi134.stdout: create 275/33 file after the snapshot 2024-03-09T08:55:40.123 INFO:tasks.workunit.client.1.smithi134.stdout: create 275/34 file after the snapshot 2024-03-09T08:55:40.123 INFO:tasks.workunit.client.1.smithi134.stdout: create 275/35 file after the snapshot 2024-03-09T08:55:40.123 INFO:tasks.workunit.client.1.smithi134.stdout: create 275/36 file after the snapshot 2024-03-09T08:55:40.124 INFO:tasks.workunit.client.1.smithi134.stdout: create 275/37 file after the snapshot 2024-03-09T08:55:40.124 INFO:tasks.workunit.client.1.smithi134.stdout: create 275/38 file after the snapshot 2024-03-09T08:55:40.124 INFO:tasks.workunit.client.1.smithi134.stdout: create 275/39 file after the snapshot 2024-03-09T08:55:40.125 INFO:tasks.workunit.client.1.smithi134.stdout: create 276/.snap/snap-subdir-test 2024-03-09T08:55:40.173 INFO:tasks.workunit.client.1.smithi134.stdout: create 276/30 file after the snapshot 2024-03-09T08:55:40.175 INFO:tasks.workunit.client.1.smithi134.stdout: create 276/31 file after the snapshot 2024-03-09T08:55:40.175 INFO:tasks.workunit.client.1.smithi134.stdout: create 276/32 file after the snapshot 2024-03-09T08:55:40.175 INFO:tasks.workunit.client.1.smithi134.stdout: create 276/33 file after the snapshot 2024-03-09T08:55:40.176 INFO:tasks.workunit.client.1.smithi134.stdout: create 276/34 file after the snapshot 2024-03-09T08:55:40.176 INFO:tasks.workunit.client.1.smithi134.stdout: create 276/35 file after the snapshot 2024-03-09T08:55:40.177 INFO:tasks.workunit.client.1.smithi134.stdout: create 276/36 file after the snapshot 2024-03-09T08:55:40.177 INFO:tasks.workunit.client.1.smithi134.stdout: create 276/37 file after the snapshot 2024-03-09T08:55:40.177 INFO:tasks.workunit.client.1.smithi134.stdout: create 276/38 file after the snapshot 2024-03-09T08:55:40.177 INFO:tasks.workunit.client.1.smithi134.stdout: create 276/39 file after the snapshot 2024-03-09T08:55:40.178 INFO:tasks.workunit.client.1.smithi134.stdout: create 277/.snap/snap-subdir-test 2024-03-09T08:55:40.235 INFO:tasks.workunit.client.1.smithi134.stdout: create 277/30 file after the snapshot 2024-03-09T08:55:40.237 INFO:tasks.workunit.client.1.smithi134.stdout: create 277/31 file after the snapshot 2024-03-09T08:55:40.238 INFO:tasks.workunit.client.1.smithi134.stdout: create 277/32 file after the snapshot 2024-03-09T08:55:40.238 INFO:tasks.workunit.client.1.smithi134.stdout: create 277/33 file after the snapshot 2024-03-09T08:55:40.239 INFO:tasks.workunit.client.1.smithi134.stdout: create 277/34 file after the snapshot 2024-03-09T08:55:40.239 INFO:tasks.workunit.client.1.smithi134.stdout: create 277/35 file after the snapshot 2024-03-09T08:55:40.239 INFO:tasks.workunit.client.1.smithi134.stdout: create 277/36 file after the snapshot 2024-03-09T08:55:40.240 INFO:tasks.workunit.client.1.smithi134.stdout: create 277/37 file after the snapshot 2024-03-09T08:55:40.240 INFO:tasks.workunit.client.1.smithi134.stdout: create 277/38 file after the snapshot 2024-03-09T08:55:40.240 INFO:tasks.workunit.client.1.smithi134.stdout: create 277/39 file after the snapshot 2024-03-09T08:55:40.240 INFO:tasks.workunit.client.1.smithi134.stdout: create 278/.snap/snap-subdir-test 2024-03-09T08:55:40.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.309+0000 7f977be00640 1 Processor -- start 2024-03-09T08:55:40.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.310+0000 7f977be00640 1 -- start start 2024-03-09T08:55:40.308 INFO:tasks.workunit.client.1.smithi134.stdout: create 278/30 file after the snapshot 2024-03-09T08:55:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.311+0000 7f977be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9774153600 0x7f977406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.311+0000 7f977be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977406b580 0x7f97740c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.311+0000 7f977be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97741539e0 con 0x7f97740c5ff0 2024-03-09T08:55:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.311+0000 7f977be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977415a490 con 0x7f9774153600 2024-03-09T08:55:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.311+0000 7f977be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97740c6990 con 0x7f977406b580 2024-03-09T08:55:40.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.311+0000 7f9779400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9774153600 0x7f977406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.312+0000 7f9779400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9774153600 0x7f977406b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37964/0 (socket says 172.21.15.67:37964) 2024-03-09T08:55:40.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.312+0000 7f9779400640 1 -- 172.21.15.67:0/1075053410 learned_addr learned my addr 172.21.15.67:0/1075053410 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:40.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.312+0000 7f9778a00640 1 --2- 172.21.15.67:0/1075053410 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977406b580 0x7f97740c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.312+0000 7f9779400640 1 -- 172.21.15.67:0/1075053410 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977406b580 msgr2=0x7f97740c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:40.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.312+0000 7f9779400640 1 --2- 172.21.15.67:0/1075053410 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977406b580 0x7f97740c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.312+0000 7f9779400640 1 -- 172.21.15.67:0/1075053410 >> v1:172.21.15.67:6789/0 conn(0x7f97740c5ff0 legacy=0x7f9774155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.312+0000 7f9779400640 1 -- 172.21.15.67:0/1075053410 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97740c7a00 con 0x7f9774153600 2024-03-09T08:55:40.311 INFO:tasks.workunit.client.1.smithi134.stdout: create 278/31 file after the snapshot 2024-03-09T08:55:40.312 INFO:tasks.workunit.client.1.smithi134.stdout: create 278/32 file after the snapshot 2024-03-09T08:55:40.312 INFO:tasks.workunit.client.1.smithi134.stdout: create 278/33 file after the snapshot 2024-03-09T08:55:40.312 INFO:tasks.workunit.client.1.smithi134.stdout: create 278/34 file after the snapshot 2024-03-09T08:55:40.312 INFO:tasks.workunit.client.1.smithi134.stdout: create 278/35 file after the snapshot 2024-03-09T08:55:40.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.313+0000 7f9779400640 1 --2- 172.21.15.67:0/1075053410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9774153600 0x7f977406b040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9768009870 tx=0x7f976802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3668c46cd6cd499e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.314+0000 7f9773e00640 1 -- 172.21.15.67:0/1075053410 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768035020 con 0x7f9774153600 2024-03-09T08:55:40.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.314+0000 7f9773e00640 1 -- 172.21.15.67:0/1075053410 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768031640 con 0x7f9774153600 2024-03-09T08:55:40.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.314+0000 7f9773e00640 1 -- 172.21.15.67:0/1075053410 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976802d040 con 0x7f9774153600 2024-03-09T08:55:40.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.315+0000 7f977be00640 1 -- 172.21.15.67:0/1075053410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9774153600 msgr2=0x7f977406b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.315+0000 7f977be00640 1 --2- 172.21.15.67:0/1075053410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9774153600 0x7f977406b040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9768009870 tx=0x7f976802b8f0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.315+0000 7f977be00640 1 -- 172.21.15.67:0/1075053410 shutdown_connections 2024-03-09T08:55:40.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.315+0000 7f977be00640 1 --2- 172.21.15.67:0/1075053410 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977406b580 0x7f97740c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.315+0000 7f977be00640 1 --2- 172.21.15.67:0/1075053410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9774153600 0x7f977406b040 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.315+0000 7f977be00640 1 -- 172.21.15.67:0/1075053410 >> 172.21.15.67:0/1075053410 conn(0x7f97740ad160 msgr2=0x7f97740ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:40.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.315+0000 7f977be00640 1 -- 172.21.15.67:0/1075053410 shutdown_connections 2024-03-09T08:55:40.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.315+0000 7f977be00640 1 -- 172.21.15.67:0/1075053410 wait complete. 2024-03-09T08:55:40.315 INFO:tasks.workunit.client.1.smithi134.stdout: create 278/36 file after the snapshot 2024-03-09T08:55:40.315 INFO:tasks.workunit.client.1.smithi134.stdout: create 278/37 file after the snapshot 2024-03-09T08:55:40.315 INFO:tasks.workunit.client.1.smithi134.stdout: create 278/38 file after the snapshot 2024-03-09T08:55:40.315 INFO:tasks.workunit.client.1.smithi134.stdout: create 278/39 file after the snapshot 2024-03-09T08:55:40.315 INFO:tasks.workunit.client.1.smithi134.stdout: create 279/.snap/snap-subdir-test 2024-03-09T08:55:40.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.317+0000 7f977be00640 1 Processor -- start 2024-03-09T08:55:40.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.317+0000 7f977be00640 1 -- start start 2024-03-09T08:55:40.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.319+0000 7f977be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977406b580 0x7f97740affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.319+0000 7f977be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97740c5ff0 0x7f97740aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.319+0000 7f977be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9774153600 0x7f97740af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.319+0000 7f977be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97740c6e20 con 0x7f97740c5ff0 2024-03-09T08:55:40.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.319+0000 7f977be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97740c7580 con 0x7f9774153600 2024-03-09T08:55:40.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.319+0000 7f977be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97740c6fa0 con 0x7f977406b580 2024-03-09T08:55:40.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.319+0000 7f9778a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97740c5ff0 0x7f97740aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.319+0000 7f9778a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97740c5ff0 0x7f97740aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37974/0 (socket says 172.21.15.67:37974) 2024-03-09T08:55:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.319+0000 7f9778a00640 1 -- 172.21.15.67:0/3139390349 learned_addr learned my addr 172.21.15.67:0/3139390349 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.319+0000 7f9779400640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977406b580 0x7f97740affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.320+0000 7f9778a00640 1 -- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977406b580 msgr2=0x7f97740affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.320+0000 7f9778a00640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977406b580 0x7f97740affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.320+0000 7f9779e00640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9774153600 0x7f97740af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.320+0000 7f9778a00640 1 -- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9774153600 msgr2=0x7f97740af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.320+0000 7f9778a00640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9774153600 0x7f97740af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.320+0000 7f9778a00640 1 -- 172.21.15.67:0/3139390349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97741ec020 con 0x7f97740c5ff0 2024-03-09T08:55:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.320+0000 7f9779400640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977406b580 0x7f97740affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:55:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.320+0000 7f9779e00640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9774153600 0x7f97740af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:55:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.320+0000 7f9778a00640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97740c5ff0 0x7f97740aeb00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f976000e870 tx=0x7f976000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.320+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9760017070 con 0x7f97740c5ff0 2024-03-09T08:55:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.321+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976000ee60 con 0x7f97740c5ff0 2024-03-09T08:55:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.321+0000 7f977be00640 1 -- 172.21.15.67:0/3139390349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97741ec300 con 0x7f97740c5ff0 2024-03-09T08:55:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.321+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976000cb00 con 0x7f97740c5ff0 2024-03-09T08:55:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.321+0000 7f977be00640 1 -- 172.21.15.67:0/3139390349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97741ec8c0 con 0x7f97740c5ff0 2024-03-09T08:55:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.322+0000 7f977be00640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740af580 0x7f977414c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.322+0000 7f977be00640 1 -- 172.21.15.67:0/3139390349 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97740c5ab0 con 0x7f97740af580 2024-03-09T08:55:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.322+0000 7f9779400640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740af580 0x7f977414c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.323+0000 7f9779400640 1 -- 172.21.15.67:0/3139390349 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9774150c10 con 0x7f97740af580 2024-03-09T08:55:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.324+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f976001a070 con 0x7f97740c5ff0 2024-03-09T08:55:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.324+0000 7f9779400640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740af580 0x7f977414c440 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f97740c6480 tx=0x7f9768004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.325+0000 7f9771600640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f975406b4d0 0x7f975406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.325+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f976008a7c0 con 0x7f97740c5ff0 2024-03-09T08:55:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.325+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768049070 con 0x7f97740af580 2024-03-09T08:55:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.326+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f976803a630 con 0x7f97740af580 2024-03-09T08:55:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.326+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740af580 msgr2=0x7f977414c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.326+0000 7f9771600640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740af580 0x7f977414c440 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f97740c6480 tx=0x7f9768004730 comp rx=0 tx=0).stop 2024-03-09T08:55:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.326+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740af580 msgr2=0x7f977414c440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.326+0000 7f9771600640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97740af580 0x7f977414c440 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f97740c6480 tx=0x7f9768004730 comp rx=0 tx=0).stop 2024-03-09T08:55:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.326+0000 7f9779e00640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f975406b4d0 0x7f975406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.327+0000 7f9779e00640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f975406b4d0 0x7f975406d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f976c0012e0 tx=0x7f976c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.334+0000 7f977be00640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977414c980 0x7f977414cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.334+0000 7f977be00640 1 -- 172.21.15.67:0/3139390349 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97740ae820 con 0x7f977414c980 2024-03-09T08:55:40.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.335+0000 7f9779400640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977414c980 0x7f977414cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.336+0000 7f9779400640 1 -- 172.21.15.67:0/3139390349 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9774075a00 con 0x7f977414c980 2024-03-09T08:55:40.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.337+0000 7f9779400640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977414c980 0x7f977414cd80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9768000c00 tx=0x7f97680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.337+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768049040 con 0x7f977414c980 2024-03-09T08:55:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4873 (secure 0 0 0) 0x7f9768009490 con 0x7f977414c980 2024-03-09T08:55:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977414c980 msgr2=0x7f977414cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f9771600640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977414c980 0x7f977414cd80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9768000c00 tx=0x7f97680046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f9771600640 1 -- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977414c980 msgr2=0x7f977414cd80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f9771600640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f977414c980 0x7f977414cd80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.336 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:55:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 28, 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:55:40.338 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.339 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:55:40.340 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.344 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:40.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:55:40.346 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:40.347 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:40.348 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:55:40.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:40.349 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:40.349 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:40.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:40.349 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:40.349 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:40.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:40.349 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:55:40.349 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:55:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f977be00640 1 -- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f975406b4d0 msgr2=0x7f975406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f977be00640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f975406b4d0 0x7f975406d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f976c0012e0 tx=0x7f976c009040 comp rx=0 tx=0).stop 2024-03-09T08:55:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f977be00640 1 -- 172.21.15.67:0/3139390349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97740c5ff0 msgr2=0x7f97740aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f977be00640 1 --2- 172.21.15.67:0/3139390349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f97740c5ff0 0x7f97740aeb00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f976000e870 tx=0x7f976000ed30 comp rx=0 tx=0).stop 2024-03-09T08:55:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f9779e00640 1 -- 172.21.15.67:0/3139390349 reap_dead start 2024-03-09T08:55:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f9779e00640 1 -- 172.21.15.67:0/3139390349 reap_dead start 2024-03-09T08:55:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f977be00640 1 -- 172.21.15.67:0/3139390349 shutdown_connections 2024-03-09T08:55:40.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.338+0000 7f977be00640 1 -- 172.21.15.67:0/3139390349 >> 172.21.15.67:0/3139390349 conn(0x7f97740ad160 msgr2=0x7f977414e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:40.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.339+0000 7f977be00640 1 -- 172.21.15.67:0/3139390349 shutdown_connections 2024-03-09T08:55:40.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.339+0000 7f977be00640 1 -- 172.21.15.67:0/3139390349 wait complete. 2024-03-09T08:55:40.353 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:55:40.353 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T08:55:40.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T08:55:40.354 INFO:tasks.workunit.client.1.smithi134.stdout: create 279/30 file after the snapshot 2024-03-09T08:55:40.356 INFO:tasks.workunit.client.1.smithi134.stdout: create 279/31 file after the snapshot 2024-03-09T08:55:40.356 INFO:tasks.workunit.client.1.smithi134.stdout: create 279/32 file after the snapshot 2024-03-09T08:55:40.356 INFO:tasks.workunit.client.1.smithi134.stdout: create 279/33 file after the snapshot 2024-03-09T08:55:40.356 INFO:tasks.workunit.client.1.smithi134.stdout: create 279/34 file after the snapshot 2024-03-09T08:55:40.356 INFO:tasks.workunit.client.1.smithi134.stdout: create 279/35 file after the snapshot 2024-03-09T08:55:40.356 INFO:tasks.workunit.client.1.smithi134.stdout: create 279/36 file after the snapshot 2024-03-09T08:55:40.356 INFO:tasks.workunit.client.1.smithi134.stdout: create 279/37 file after the snapshot 2024-03-09T08:55:40.356 INFO:tasks.workunit.client.1.smithi134.stdout: create 279/38 file after the snapshot 2024-03-09T08:55:40.356 INFO:tasks.workunit.client.1.smithi134.stdout: create 279/39 file after the snapshot 2024-03-09T08:55:40.356 INFO:tasks.workunit.client.1.smithi134.stdout: create 280/.snap/snap-subdir-test 2024-03-09T08:55:40.394 INFO:tasks.workunit.client.1.smithi134.stdout: create 280/30 file after the snapshot 2024-03-09T08:55:40.396 INFO:tasks.workunit.client.1.smithi134.stdout: create 280/31 file after the snapshot 2024-03-09T08:55:40.396 INFO:tasks.workunit.client.1.smithi134.stdout: create 280/32 file after the snapshot 2024-03-09T08:55:40.396 INFO:tasks.workunit.client.1.smithi134.stdout: create 280/33 file after the snapshot 2024-03-09T08:55:40.397 INFO:tasks.workunit.client.1.smithi134.stdout: create 280/34 file after the snapshot 2024-03-09T08:55:40.397 INFO:tasks.workunit.client.1.smithi134.stdout: create 280/35 file after the snapshot 2024-03-09T08:55:40.397 INFO:tasks.workunit.client.1.smithi134.stdout: create 280/36 file after the snapshot 2024-03-09T08:55:40.397 INFO:tasks.workunit.client.1.smithi134.stdout: create 280/37 file after the snapshot 2024-03-09T08:55:40.398 INFO:tasks.workunit.client.1.smithi134.stdout: create 280/38 file after the snapshot 2024-03-09T08:55:40.398 INFO:tasks.workunit.client.1.smithi134.stdout: create 280/39 file after the snapshot 2024-03-09T08:55:40.398 INFO:tasks.workunit.client.1.smithi134.stdout: create 281/.snap/snap-subdir-test 2024-03-09T08:55:40.465 INFO:tasks.workunit.client.1.smithi134.stdout: create 281/30 file after the snapshot 2024-03-09T08:55:40.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 281/31 file after the snapshot 2024-03-09T08:55:40.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 281/32 file after the snapshot 2024-03-09T08:55:40.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 281/33 file after the snapshot 2024-03-09T08:55:40.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 281/34 file after the snapshot 2024-03-09T08:55:40.470 INFO:tasks.workunit.client.1.smithi134.stdout: create 281/35 file after the snapshot 2024-03-09T08:55:40.470 INFO:tasks.workunit.client.1.smithi134.stdout: create 281/36 file after the snapshot 2024-03-09T08:55:40.471 INFO:tasks.workunit.client.1.smithi134.stdout: create 281/37 file after the snapshot 2024-03-09T08:55:40.471 INFO:tasks.workunit.client.1.smithi134.stdout: create 281/38 file after the snapshot 2024-03-09T08:55:40.471 INFO:tasks.workunit.client.1.smithi134.stdout: create 281/39 file after the snapshot 2024-03-09T08:55:40.471 INFO:tasks.workunit.client.1.smithi134.stdout: create 282/.snap/snap-subdir-test 2024-03-09T08:55:40.552 INFO:tasks.workunit.client.1.smithi134.stdout: create 282/30 file after the snapshot 2024-03-09T08:55:40.554 INFO:tasks.workunit.client.1.smithi134.stdout: create 282/31 file after the snapshot 2024-03-09T08:55:40.555 INFO:tasks.workunit.client.1.smithi134.stdout: create 282/32 file after the snapshot 2024-03-09T08:55:40.555 INFO:tasks.workunit.client.1.smithi134.stdout: create 282/33 file after the snapshot 2024-03-09T08:55:40.556 INFO:tasks.workunit.client.1.smithi134.stdout: create 282/34 file after the snapshot 2024-03-09T08:55:40.557 INFO:tasks.workunit.client.1.smithi134.stdout: create 282/35 file after the snapshot 2024-03-09T08:55:40.557 INFO:tasks.workunit.client.1.smithi134.stdout: create 282/36 file after the snapshot 2024-03-09T08:55:40.557 INFO:tasks.workunit.client.1.smithi134.stdout: create 282/37 file after the snapshot 2024-03-09T08:55:40.557 INFO:tasks.workunit.client.1.smithi134.stdout: create 282/38 file after the snapshot 2024-03-09T08:55:40.558 INFO:tasks.workunit.client.1.smithi134.stdout: create 282/39 file after the snapshot 2024-03-09T08:55:40.558 INFO:tasks.workunit.client.1.smithi134.stdout: create 283/.snap/snap-subdir-test 2024-03-09T08:55:40.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.566+0000 7f40de800640 1 Processor -- start 2024-03-09T08:55:40.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.566+0000 7f40de800640 1 -- start start 2024-03-09T08:55:40.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.569+0000 7f40de800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f40d80c84e0 0x7f40d806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.569+0000 7f40de800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d806bd30 0x7f40d80c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.569+0000 7f40de800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d8155c60 con 0x7f40d806f4f0 2024-03-09T08:55:40.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.569+0000 7f40de800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d8155f40 con 0x7f40d80c84e0 2024-03-09T08:55:40.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.569+0000 7f40de800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8156220 con 0x7f40d806bd30 2024-03-09T08:55:40.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.569+0000 7f40dca00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f40d806f4f0 0x7f40d80c53b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60358/0 (socket says 172.21.15.67:60358) 2024-03-09T08:55:40.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.569+0000 7f40dca00640 1 -- 172.21.15.67:0/1186045371 learned_addr learned my addr 172.21.15.67:0/1186045371 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:40.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.570+0000 7f40dd400640 1 --2- 172.21.15.67:0/1186045371 >> v2:172.21.15.134:3300/0 conn(0x7f40d80c84e0 0x7f40d806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.570+0000 7f40dde00640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d806bd30 0x7f40d80c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.570+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384662430 0 0) 0x7f40d8155c60 con 0x7f40d806f4f0 2024-03-09T08:55:40.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.570+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40c0003520 con 0x7f40d806f4f0 2024-03-09T08:55:40.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.570+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2426646010 0 0) 0x7f40c0003520 con 0x7f40d806f4f0 2024-03-09T08:55:40.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.570+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d806bd30 msgr2=0x7f40d80c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:40.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.570+0000 7f40d7e00640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d806bd30 0x7f40d80c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.570+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 >> v2:172.21.15.134:3300/0 conn(0x7f40d80c84e0 msgr2=0x7f40d806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:40.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.570+0000 7f40d7e00640 1 --2- 172.21.15.67:0/1186045371 >> v2:172.21.15.134:3300/0 conn(0x7f40d80c84e0 0x7f40d806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.570+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d8156500 con 0x7f40d806f4f0 2024-03-09T08:55:40.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.571+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f40c4001670 con 0x7f40d806f4f0 2024-03-09T08:55:40.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.571+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 >> v1:172.21.15.67:6789/0 conn(0x7f40d806f4f0 legacy=0x7f40d80c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.571+0000 7f40d7e00640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 0x7f40c0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.571+0000 7f40d7e00640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c0004800 0x7f40c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.571+0000 7f40d7e00640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d806bd30 0x7f40c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.571+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8155f40 con 0x7f40c0003a20 2024-03-09T08:55:40.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.571+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8156220 con 0x7f40c0004800 2024-03-09T08:55:40.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.571+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c000baa0 con 0x7f40d806bd30 2024-03-09T08:55:40.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.571+0000 7f40dd400640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 0x7f40c0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.571+0000 7f40dca00640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c0004800 0x7f40c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.571+0000 7f40dde00640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d806bd30 0x7f40c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.572+0000 7f40dd400640 1 -- 172.21.15.67:0/1186045371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d806bd30 msgr2=0x7f40c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:40.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.572+0000 7f40dd400640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d806bd30 0x7f40c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.572+0000 7f40dd400640 1 -- 172.21.15.67:0/1186045371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c0004800 msgr2=0x7f40c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:40.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.572+0000 7f40dd400640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c0004800 0x7f40c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.572+0000 7f40dd400640 1 -- 172.21.15.67:0/1186045371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0003520 con 0x7f40c0003a20 2024-03-09T08:55:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.573+0000 7f40dd400640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 0x7f40c0003e40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009a90 tx=0x7f40c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2d6ff98f7166fa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.573+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc039070 con 0x7f40c0003a20 2024-03-09T08:55:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.573+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc02c690 con 0x7f40c0003a20 2024-03-09T08:55:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.573+0000 7f40d7e00640 1 -- 172.21.15.67:0/1186045371 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc009690 con 0x7f40c0003a20 2024-03-09T08:55:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.574+0000 7f40de800640 1 -- 172.21.15.67:0/1186045371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 msgr2=0x7f40c0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.574+0000 7f40de800640 1 --2- 172.21.15.67:0/1186045371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 0x7f40c0003e40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009a90 tx=0x7f40c0005830 comp rx=0 tx=0).stop 2024-03-09T08:55:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.574+0000 7f40dde00640 1 -- 172.21.15.67:0/1186045371 reap_dead start 2024-03-09T08:55:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.574+0000 7f40de800640 1 -- 172.21.15.67:0/1186045371 shutdown_connections 2024-03-09T08:55:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.574+0000 7f40de800640 1 -- 172.21.15.67:0/1186045371 >> 172.21.15.67:0/1186045371 conn(0x7f40d806d650 msgr2=0x7f40d806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.574+0000 7f40de800640 1 -- 172.21.15.67:0/1186045371 shutdown_connections 2024-03-09T08:55:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.574+0000 7f40de800640 1 -- 172.21.15.67:0/1186045371 wait complete. 2024-03-09T08:55:40.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.576+0000 7f40de800640 1 Processor -- start 2024-03-09T08:55:40.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.576+0000 7f40de800640 1 -- start start 2024-03-09T08:55:40.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.577+0000 7f40de800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 0x7f40d8153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.577+0000 7f40de800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d814f3a0 0x7f40d8153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.577+0000 7f40de800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40d8159480 0x7f40d8159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.577+0000 7f40de800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80c8a40 con 0x7f40c0003a20 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.577+0000 7f40de800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80c88c0 con 0x7f40d8159480 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.577+0000 7f40de800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80c1840 con 0x7f40d814f3a0 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.578+0000 7f40dd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 0x7f40d8153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.578+0000 7f40dd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 0x7f40d8153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37986/0 (socket says 172.21.15.67:37986) 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.578+0000 7f40dd400640 1 -- 172.21.15.67:0/2488398373 learned_addr learned my addr 172.21.15.67:0/2488398373 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.578+0000 7f40dca00640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d814f3a0 0x7f40d8153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.578+0000 7f40dde00640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40d8159480 0x7f40d8159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.578+0000 7f40dd400640 1 -- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d814f3a0 msgr2=0x7f40d8153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.578+0000 7f40dd400640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d814f3a0 0x7f40d8153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.578+0000 7f40dd400640 1 -- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40d8159480 msgr2=0x7f40d8159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:40.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.578+0000 7f40dd400640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40d8159480 0x7f40d8159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:40.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.578+0000 7f40dd400640 1 -- 172.21.15.67:0/2488398373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d8157740 con 0x7f40c0003a20 2024-03-09T08:55:40.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.579+0000 7f40dd400640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 0x7f40d8153680 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f40cc000c00 tx=0x7f40cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.579+0000 7f40d5600640 1 -- 172.21.15.67:0/2488398373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc039050 con 0x7f40c0003a20 2024-03-09T08:55:40.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.579+0000 7f40d5600640 1 -- 172.21.15.67:0/2488398373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc045070 con 0x7f40c0003a20 2024-03-09T08:55:40.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.579+0000 7f40d5600640 1 -- 172.21.15.67:0/2488398373 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0044e0 con 0x7f40c0003a20 2024-03-09T08:55:40.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.579+0000 7f40de800640 1 -- 172.21.15.67:0/2488398373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d81ec0f0 con 0x7f40c0003a20 2024-03-09T08:55:40.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.579+0000 7f40de800640 1 -- 172.21.15.67:0/2488398373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d814d7a0 con 0x7f40c0003a20 2024-03-09T08:55:40.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.580+0000 7f40de800640 1 -- 172.21.15.67:0/2488398373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d804f3c0 con 0x7f40c0003a20 2024-03-09T08:55:40.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.583+0000 7f40d5600640 1 -- 172.21.15.67:0/2488398373 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f40cc003a50 con 0x7f40c0003a20 2024-03-09T08:55:40.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.584+0000 7f40d5600640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40b806b4d0 0x7f40b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:40.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.584+0000 7f40d5600640 1 -- 172.21.15.67:0/2488398373 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f40cc0ac700 con 0x7f40c0003a20 2024-03-09T08:55:40.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.585+0000 7f40dca00640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40b806b4d0 0x7f40b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:40.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.586+0000 7f40dca00640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40b806b4d0 0x7f40b806d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f40c4001050 tx=0x7f40c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:40.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:40.591+0000 7f40d5600640 1 -- 172.21.15.67:0/2488398373 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f40cc07b250 con 0x7f40c0003a20 2024-03-09T08:55:40.615 INFO:tasks.workunit.client.1.smithi134.stdout: create 283/30 file after the snapshot 2024-03-09T08:55:40.617 INFO:tasks.workunit.client.1.smithi134.stdout: create 283/31 file after the snapshot 2024-03-09T08:55:40.617 INFO:tasks.workunit.client.1.smithi134.stdout: create 283/32 file after the snapshot 2024-03-09T08:55:40.617 INFO:tasks.workunit.client.1.smithi134.stdout: create 283/33 file after the snapshot 2024-03-09T08:55:40.618 INFO:tasks.workunit.client.1.smithi134.stdout: create 283/34 file after the snapshot 2024-03-09T08:55:40.618 INFO:tasks.workunit.client.1.smithi134.stdout: create 283/35 file after the snapshot 2024-03-09T08:55:40.618 INFO:tasks.workunit.client.1.smithi134.stdout: create 283/36 file after the snapshot 2024-03-09T08:55:40.618 INFO:tasks.workunit.client.1.smithi134.stdout: create 283/37 file after the snapshot 2024-03-09T08:55:40.618 INFO:tasks.workunit.client.1.smithi134.stdout: create 283/38 file after the snapshot 2024-03-09T08:55:40.619 INFO:tasks.workunit.client.1.smithi134.stdout: create 283/39 file after the snapshot 2024-03-09T08:55:40.619 INFO:tasks.workunit.client.1.smithi134.stdout: create 284/.snap/snap-subdir-test 2024-03-09T08:55:40.681 INFO:tasks.workunit.client.1.smithi134.stdout: create 284/30 file after the snapshot 2024-03-09T08:55:40.684 INFO:tasks.workunit.client.1.smithi134.stdout: create 284/31 file after the snapshot 2024-03-09T08:55:40.684 INFO:tasks.workunit.client.1.smithi134.stdout: create 284/32 file after the snapshot 2024-03-09T08:55:40.685 INFO:tasks.workunit.client.1.smithi134.stdout: create 284/33 file after the snapshot 2024-03-09T08:55:40.685 INFO:tasks.workunit.client.1.smithi134.stdout: create 284/34 file after the snapshot 2024-03-09T08:55:40.685 INFO:tasks.workunit.client.1.smithi134.stdout: create 284/35 file after the snapshot 2024-03-09T08:55:40.685 INFO:tasks.workunit.client.1.smithi134.stdout: create 284/36 file after the snapshot 2024-03-09T08:55:40.686 INFO:tasks.workunit.client.1.smithi134.stdout: create 284/37 file after the snapshot 2024-03-09T08:55:40.686 INFO:tasks.workunit.client.1.smithi134.stdout: create 284/38 file after the snapshot 2024-03-09T08:55:40.686 INFO:tasks.workunit.client.1.smithi134.stdout: create 284/39 file after the snapshot 2024-03-09T08:55:40.686 INFO:tasks.workunit.client.1.smithi134.stdout: create 285/.snap/snap-subdir-test 2024-03-09T08:55:40.748 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:55:40.753 INFO:tasks.workunit.client.1.smithi134.stdout: create 285/30 file after the snapshot 2024-03-09T08:55:40.757 INFO:tasks.workunit.client.1.smithi134.stdout: create 285/31 file after the snapshot 2024-03-09T08:55:40.757 INFO:tasks.workunit.client.1.smithi134.stdout: create 285/32 file after the snapshot 2024-03-09T08:55:40.757 INFO:tasks.workunit.client.1.smithi134.stdout: create 285/33 file after the snapshot 2024-03-09T08:55:40.758 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:55:40.759 INFO:tasks.workunit.client.1.smithi134.stdout: create 285/34 file after the snapshot 2024-03-09T08:55:40.759 INFO:tasks.workunit.client.1.smithi134.stdout: create 285/35 file after the snapshot 2024-03-09T08:55:40.760 INFO:tasks.workunit.client.1.smithi134.stdout: create 285/36 file after the snapshot 2024-03-09T08:55:40.760 INFO:tasks.workunit.client.1.smithi134.stdout: create 285/37 file after the snapshot 2024-03-09T08:55:40.760 INFO:tasks.workunit.client.1.smithi134.stdout: create 285/38 file after the snapshot 2024-03-09T08:55:40.760 INFO:tasks.workunit.client.1.smithi134.stdout: create 285/39 file after the snapshot 2024-03-09T08:55:40.760 INFO:tasks.workunit.client.1.smithi134.stdout: create 286/.snap/snap-subdir-test 2024-03-09T08:55:40.942 INFO:tasks.workunit.client.0.smithi067.stdout: create 242/30 file after the snapshot 2024-03-09T08:55:40.949 INFO:tasks.workunit.client.0.smithi067.stdout: create 242/31 file after the snapshot 2024-03-09T08:55:40.950 INFO:tasks.workunit.client.0.smithi067.stdout: create 242/32 file after the snapshot 2024-03-09T08:55:40.951 INFO:tasks.workunit.client.0.smithi067.stdout: create 242/33 file after the snapshot 2024-03-09T08:55:40.951 INFO:tasks.workunit.client.0.smithi067.stdout: create 242/34 file after the snapshot 2024-03-09T08:55:40.951 INFO:tasks.workunit.client.0.smithi067.stdout: create 242/35 file after the snapshot 2024-03-09T08:55:40.952 INFO:tasks.workunit.client.0.smithi067.stdout: create 242/36 file after the snapshot 2024-03-09T08:55:40.952 INFO:tasks.workunit.client.0.smithi067.stdout: create 242/37 file after the snapshot 2024-03-09T08:55:40.953 INFO:tasks.workunit.client.0.smithi067.stdout: create 242/38 file after the snapshot 2024-03-09T08:55:40.953 INFO:tasks.workunit.client.0.smithi067.stdout: create 242/39 file after the snapshot 2024-03-09T08:55:40.955 INFO:tasks.workunit.client.0.smithi067.stdout: create 243/.snap/snap-subdir-test 2024-03-09T08:55:41.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.009+0000 7f40de800640 1 -- 172.21.15.67:0/2488398373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f40d8072b10 con 0x7f40c0003a20 2024-03-09T08:55:41.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.012+0000 7f40d5600640 1 -- 172.21.15.67:0/2488398373 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f40cc07ab50 con 0x7f40c0003a20 2024-03-09T08:55:41.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.021+0000 7f40de800640 1 -- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40b806b4d0 msgr2=0x7f40b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:41.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.021+0000 7f40de800640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40b806b4d0 0x7f40b806d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f40c4001050 tx=0x7f40c4009040 comp rx=0 tx=0).stop 2024-03-09T08:55:41.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.021+0000 7f40de800640 1 -- 172.21.15.67:0/2488398373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 msgr2=0x7f40d8153680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:41.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.021+0000 7f40de800640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 0x7f40d8153680 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f40cc000c00 tx=0x7f40cc004730 comp rx=0 tx=0).stop 2024-03-09T08:55:41.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.022+0000 7f40de800640 1 -- 172.21.15.67:0/2488398373 shutdown_connections 2024-03-09T08:55:41.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.022+0000 7f40de800640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40b806b4d0 0x7f40b806d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:41.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.022+0000 7f40de800640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40d8159480 0x7f40d8159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:41.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.022+0000 7f40de800640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40d814f3a0 0x7f40d8153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:41.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.022+0000 7f40de800640 1 --2- 172.21.15.67:0/2488398373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c0003a20 0x7f40d8153680 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:41.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.022+0000 7f40de800640 1 -- 172.21.15.67:0/2488398373 >> 172.21.15.67:0/2488398373 conn(0x7f40d806d650 msgr2=0x7f40d806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:41.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.023+0000 7f40de800640 1 -- 172.21.15.67:0/2488398373 shutdown_connections 2024-03-09T08:55:41.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:41.023+0000 7f40de800640 1 -- 172.21.15.67:0/2488398373 wait complete. 2024-03-09T08:55:41.057 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T08:55:41.257 INFO:tasks.workunit.client.0.smithi067.stdout: create 243/30 file after the snapshot 2024-03-09T08:55:41.260 INFO:tasks.workunit.client.0.smithi067.stdout: create 243/31 file after the snapshot 2024-03-09T08:55:41.260 INFO:tasks.workunit.client.0.smithi067.stdout: create 243/32 file after the snapshot 2024-03-09T08:55:41.260 INFO:tasks.workunit.client.0.smithi067.stdout: create 243/33 file after the snapshot 2024-03-09T08:55:41.260 INFO:tasks.workunit.client.0.smithi067.stdout: create 243/34 file after the snapshot 2024-03-09T08:55:41.260 INFO:tasks.workunit.client.0.smithi067.stdout: create 243/35 file after the snapshot 2024-03-09T08:55:41.260 INFO:tasks.workunit.client.0.smithi067.stdout: create 243/36 file after the snapshot 2024-03-09T08:55:41.261 INFO:tasks.workunit.client.0.smithi067.stdout: create 243/37 file after the snapshot 2024-03-09T08:55:41.261 INFO:tasks.workunit.client.0.smithi067.stdout: create 243/38 file after the snapshot 2024-03-09T08:55:41.261 INFO:tasks.workunit.client.0.smithi067.stdout: create 243/39 file after the snapshot 2024-03-09T08:55:41.261 INFO:tasks.workunit.client.0.smithi067.stdout: create 244/.snap/snap-subdir-test 2024-03-09T08:55:41.296 INFO:tasks.workunit.client.0.smithi067.stdout: create 244/30 file after the snapshot 2024-03-09T08:55:41.298 INFO:tasks.workunit.client.0.smithi067.stdout: create 244/31 file after the snapshot 2024-03-09T08:55:41.298 INFO:tasks.workunit.client.0.smithi067.stdout: create 244/32 file after the snapshot 2024-03-09T08:55:41.298 INFO:tasks.workunit.client.0.smithi067.stdout: create 244/33 file after the snapshot 2024-03-09T08:55:41.298 INFO:tasks.workunit.client.0.smithi067.stdout: create 244/34 file after the snapshot 2024-03-09T08:55:41.298 INFO:tasks.workunit.client.0.smithi067.stdout: create 244/35 file after the snapshot 2024-03-09T08:55:41.299 INFO:tasks.workunit.client.0.smithi067.stdout: create 244/36 file after the snapshot 2024-03-09T08:55:41.299 INFO:tasks.workunit.client.0.smithi067.stdout: create 244/37 file after the snapshot 2024-03-09T08:55:41.299 INFO:tasks.workunit.client.0.smithi067.stdout: create 244/38 file after the snapshot 2024-03-09T08:55:41.299 INFO:tasks.workunit.client.0.smithi067.stdout: create 244/39 file after the snapshot 2024-03-09T08:55:41.299 INFO:tasks.workunit.client.0.smithi067.stdout: create 245/.snap/snap-subdir-test 2024-03-09T08:55:41.342 INFO:tasks.workunit.client.0.smithi067.stdout: create 245/30 file after the snapshot 2024-03-09T08:55:41.344 INFO:tasks.workunit.client.0.smithi067.stdout: create 245/31 file after the snapshot 2024-03-09T08:55:41.344 INFO:tasks.workunit.client.0.smithi067.stdout: create 245/32 file after the snapshot 2024-03-09T08:55:41.344 INFO:tasks.workunit.client.0.smithi067.stdout: create 245/33 file after the snapshot 2024-03-09T08:55:41.344 INFO:tasks.workunit.client.0.smithi067.stdout: create 245/34 file after the snapshot 2024-03-09T08:55:41.344 INFO:tasks.workunit.client.0.smithi067.stdout: create 245/35 file after the snapshot 2024-03-09T08:55:41.345 INFO:tasks.workunit.client.0.smithi067.stdout: create 245/36 file after the snapshot 2024-03-09T08:55:41.345 INFO:tasks.workunit.client.0.smithi067.stdout: create 245/37 file after the snapshot 2024-03-09T08:55:41.345 INFO:tasks.workunit.client.0.smithi067.stdout: create 245/38 file after the snapshot 2024-03-09T08:55:41.345 INFO:tasks.workunit.client.0.smithi067.stdout: create 245/39 file after the snapshot 2024-03-09T08:55:41.345 INFO:tasks.workunit.client.0.smithi067.stdout: create 246/.snap/snap-subdir-test 2024-03-09T08:55:41.390 INFO:tasks.workunit.client.0.smithi067.stdout: create 246/30 file after the snapshot 2024-03-09T08:55:41.391 INFO:tasks.workunit.client.0.smithi067.stdout: create 246/31 file after the snapshot 2024-03-09T08:55:41.392 INFO:tasks.workunit.client.0.smithi067.stdout: create 246/32 file after the snapshot 2024-03-09T08:55:41.392 INFO:tasks.workunit.client.0.smithi067.stdout: create 246/33 file after the snapshot 2024-03-09T08:55:41.392 INFO:tasks.workunit.client.0.smithi067.stdout: create 246/34 file after the snapshot 2024-03-09T08:55:41.392 INFO:tasks.workunit.client.0.smithi067.stdout: create 246/35 file after the snapshot 2024-03-09T08:55:41.393 INFO:tasks.workunit.client.0.smithi067.stdout: create 246/36 file after the snapshot 2024-03-09T08:55:41.393 INFO:tasks.workunit.client.0.smithi067.stdout: create 246/37 file after the snapshot 2024-03-09T08:55:41.393 INFO:tasks.workunit.client.0.smithi067.stdout: create 246/38 file after the snapshot 2024-03-09T08:55:41.393 INFO:tasks.workunit.client.0.smithi067.stdout: create 246/39 file after the snapshot 2024-03-09T08:55:41.393 INFO:tasks.workunit.client.0.smithi067.stdout: create 247/.snap/snap-subdir-test 2024-03-09T08:55:41.448 INFO:tasks.workunit.client.0.smithi067.stdout: create 247/30 file after the snapshot 2024-03-09T08:55:41.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 247/31 file after the snapshot 2024-03-09T08:55:41.452 INFO:tasks.workunit.client.0.smithi067.stdout: create 247/32 file after the snapshot 2024-03-09T08:55:41.452 INFO:tasks.workunit.client.0.smithi067.stdout: create 247/33 file after the snapshot 2024-03-09T08:55:41.452 INFO:tasks.workunit.client.0.smithi067.stdout: create 247/34 file after the snapshot 2024-03-09T08:55:41.452 INFO:tasks.workunit.client.0.smithi067.stdout: create 247/35 file after the snapshot 2024-03-09T08:55:41.452 INFO:tasks.workunit.client.0.smithi067.stdout: create 247/36 file after the snapshot 2024-03-09T08:55:41.452 INFO:tasks.workunit.client.0.smithi067.stdout: create 247/37 file after the snapshot 2024-03-09T08:55:41.452 INFO:tasks.workunit.client.0.smithi067.stdout: create 247/38 file after the snapshot 2024-03-09T08:55:41.453 INFO:tasks.workunit.client.0.smithi067.stdout: create 247/39 file after the snapshot 2024-03-09T08:55:41.453 INFO:tasks.workunit.client.0.smithi067.stdout: create 248/.snap/snap-subdir-test 2024-03-09T08:55:41.515 INFO:tasks.workunit.client.0.smithi067.stdout: create 248/30 file after the snapshot 2024-03-09T08:55:41.517 INFO:tasks.workunit.client.0.smithi067.stdout: create 248/31 file after the snapshot 2024-03-09T08:55:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: create 248/32 file after the snapshot 2024-03-09T08:55:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: create 248/33 file after the snapshot 2024-03-09T08:55:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: create 248/34 file after the snapshot 2024-03-09T08:55:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: create 248/35 file after the snapshot 2024-03-09T08:55:41.519 INFO:tasks.workunit.client.0.smithi067.stdout: create 248/36 file after the snapshot 2024-03-09T08:55:41.519 INFO:tasks.workunit.client.0.smithi067.stdout: create 248/37 file after the snapshot 2024-03-09T08:55:41.519 INFO:tasks.workunit.client.0.smithi067.stdout: create 248/38 file after the snapshot 2024-03-09T08:55:41.519 INFO:tasks.workunit.client.0.smithi067.stdout: create 248/39 file after the snapshot 2024-03-09T08:55:41.520 INFO:tasks.workunit.client.0.smithi067.stdout: create 249/.snap/snap-subdir-test 2024-03-09T08:55:41.585 INFO:tasks.workunit.client.0.smithi067.stdout: create 249/30 file after the snapshot 2024-03-09T08:55:41.586 INFO:tasks.workunit.client.0.smithi067.stdout: create 249/31 file after the snapshot 2024-03-09T08:55:41.586 INFO:tasks.workunit.client.0.smithi067.stdout: create 249/32 file after the snapshot 2024-03-09T08:55:41.587 INFO:tasks.workunit.client.0.smithi067.stdout: create 249/33 file after the snapshot 2024-03-09T08:55:41.588 INFO:tasks.workunit.client.0.smithi067.stdout: create 249/34 file after the snapshot 2024-03-09T08:55:41.588 INFO:tasks.workunit.client.0.smithi067.stdout: create 249/35 file after the snapshot 2024-03-09T08:55:41.588 INFO:tasks.workunit.client.0.smithi067.stdout: create 249/36 file after the snapshot 2024-03-09T08:55:41.588 INFO:tasks.workunit.client.0.smithi067.stdout: create 249/37 file after the snapshot 2024-03-09T08:55:41.588 INFO:tasks.workunit.client.0.smithi067.stdout: create 249/38 file after the snapshot 2024-03-09T08:55:41.589 INFO:tasks.workunit.client.0.smithi067.stdout: create 249/39 file after the snapshot 2024-03-09T08:55:41.589 INFO:tasks.workunit.client.0.smithi067.stdout: create 250/.snap/snap-subdir-test 2024-03-09T08:55:41.639 INFO:tasks.workunit.client.0.smithi067.stdout: create 250/30 file after the snapshot 2024-03-09T08:55:41.642 INFO:tasks.workunit.client.0.smithi067.stdout: create 250/31 file after the snapshot 2024-03-09T08:55:41.643 INFO:tasks.workunit.client.0.smithi067.stdout: create 250/32 file after the snapshot 2024-03-09T08:55:41.643 INFO:tasks.workunit.client.0.smithi067.stdout: create 250/33 file after the snapshot 2024-03-09T08:55:41.643 INFO:tasks.workunit.client.0.smithi067.stdout: create 250/34 file after the snapshot 2024-03-09T08:55:41.644 INFO:tasks.workunit.client.0.smithi067.stdout: create 250/35 file after the snapshot 2024-03-09T08:55:41.644 INFO:tasks.workunit.client.0.smithi067.stdout: create 250/36 file after the snapshot 2024-03-09T08:55:41.644 INFO:tasks.workunit.client.0.smithi067.stdout: create 250/37 file after the snapshot 2024-03-09T08:55:41.644 INFO:tasks.workunit.client.0.smithi067.stdout: create 250/38 file after the snapshot 2024-03-09T08:55:41.645 INFO:tasks.workunit.client.0.smithi067.stdout: create 250/39 file after the snapshot 2024-03-09T08:55:41.645 INFO:tasks.workunit.client.0.smithi067.stdout: create 251/.snap/snap-subdir-test 2024-03-09T08:55:41.709 INFO:tasks.workunit.client.0.smithi067.stdout: create 251/30 file after the snapshot 2024-03-09T08:55:41.712 INFO:tasks.workunit.client.0.smithi067.stdout: create 251/31 file after the snapshot 2024-03-09T08:55:41.713 INFO:tasks.workunit.client.0.smithi067.stdout: create 251/32 file after the snapshot 2024-03-09T08:55:41.713 INFO:tasks.workunit.client.0.smithi067.stdout: create 251/33 file after the snapshot 2024-03-09T08:55:41.713 INFO:tasks.workunit.client.0.smithi067.stdout: create 251/34 file after the snapshot 2024-03-09T08:55:41.714 INFO:tasks.workunit.client.0.smithi067.stdout: create 251/35 file after the snapshot 2024-03-09T08:55:41.714 INFO:tasks.workunit.client.0.smithi067.stdout: create 251/36 file after the snapshot 2024-03-09T08:55:41.714 INFO:tasks.workunit.client.0.smithi067.stdout: create 251/37 file after the snapshot 2024-03-09T08:55:41.715 INFO:tasks.workunit.client.0.smithi067.stdout: create 251/38 file after the snapshot 2024-03-09T08:55:41.715 INFO:tasks.workunit.client.0.smithi067.stdout: create 251/39 file after the snapshot 2024-03-09T08:55:41.715 INFO:tasks.workunit.client.0.smithi067.stdout: create 252/.snap/snap-subdir-test 2024-03-09T08:55:41.787 INFO:tasks.workunit.client.0.smithi067.stdout: create 252/30 file after the snapshot 2024-03-09T08:55:41.789 INFO:tasks.workunit.client.0.smithi067.stdout: create 252/31 file after the snapshot 2024-03-09T08:55:41.790 INFO:tasks.workunit.client.0.smithi067.stdout: create 252/32 file after the snapshot 2024-03-09T08:55:41.790 INFO:tasks.workunit.client.0.smithi067.stdout: create 252/33 file after the snapshot 2024-03-09T08:55:41.791 INFO:tasks.workunit.client.0.smithi067.stdout: create 252/34 file after the snapshot 2024-03-09T08:55:41.791 INFO:tasks.workunit.client.0.smithi067.stdout: create 252/35 file after the snapshot 2024-03-09T08:55:41.791 INFO:tasks.workunit.client.0.smithi067.stdout: create 252/36 file after the snapshot 2024-03-09T08:55:41.792 INFO:tasks.workunit.client.0.smithi067.stdout: create 252/37 file after the snapshot 2024-03-09T08:55:41.792 INFO:tasks.workunit.client.0.smithi067.stdout: create 252/38 file after the snapshot 2024-03-09T08:55:41.793 INFO:tasks.workunit.client.0.smithi067.stdout: create 252/39 file after the snapshot 2024-03-09T08:55:41.793 INFO:tasks.workunit.client.0.smithi067.stdout: create 253/.snap/snap-subdir-test 2024-03-09T08:55:41.860 INFO:tasks.workunit.client.0.smithi067.stdout: create 253/30 file after the snapshot 2024-03-09T08:55:41.862 INFO:tasks.workunit.client.0.smithi067.stdout: create 253/31 file after the snapshot 2024-03-09T08:55:41.862 INFO:tasks.workunit.client.0.smithi067.stdout: create 253/32 file after the snapshot 2024-03-09T08:55:41.863 INFO:tasks.workunit.client.0.smithi067.stdout: create 253/33 file after the snapshot 2024-03-09T08:55:41.863 INFO:tasks.workunit.client.0.smithi067.stdout: create 253/34 file after the snapshot 2024-03-09T08:55:41.863 INFO:tasks.workunit.client.0.smithi067.stdout: create 253/35 file after the snapshot 2024-03-09T08:55:41.864 INFO:tasks.workunit.client.0.smithi067.stdout: create 253/36 file after the snapshot 2024-03-09T08:55:41.864 INFO:tasks.workunit.client.0.smithi067.stdout: create 253/37 file after the snapshot 2024-03-09T08:55:41.865 INFO:tasks.workunit.client.0.smithi067.stdout: create 253/38 file after the snapshot 2024-03-09T08:55:41.865 INFO:tasks.workunit.client.0.smithi067.stdout: create 253/39 file after the snapshot 2024-03-09T08:55:41.865 INFO:tasks.workunit.client.0.smithi067.stdout: create 254/.snap/snap-subdir-test 2024-03-09T08:55:41.938 INFO:tasks.workunit.client.0.smithi067.stdout: create 254/30 file after the snapshot 2024-03-09T08:55:41.940 INFO:tasks.workunit.client.0.smithi067.stdout: create 254/31 file after the snapshot 2024-03-09T08:55:41.940 INFO:tasks.workunit.client.0.smithi067.stdout: create 254/32 file after the snapshot 2024-03-09T08:55:41.940 INFO:tasks.workunit.client.0.smithi067.stdout: create 254/33 file after the snapshot 2024-03-09T08:55:41.940 INFO:tasks.workunit.client.0.smithi067.stdout: create 254/34 file after the snapshot 2024-03-09T08:55:41.941 INFO:tasks.workunit.client.0.smithi067.stdout: create 254/35 file after the snapshot 2024-03-09T08:55:41.941 INFO:tasks.workunit.client.0.smithi067.stdout: create 254/36 file after the snapshot 2024-03-09T08:55:41.941 INFO:tasks.workunit.client.0.smithi067.stdout: create 254/37 file after the snapshot 2024-03-09T08:55:41.941 INFO:tasks.workunit.client.0.smithi067.stdout: create 254/38 file after the snapshot 2024-03-09T08:55:41.941 INFO:tasks.workunit.client.0.smithi067.stdout: create 254/39 file after the snapshot 2024-03-09T08:55:41.941 INFO:tasks.workunit.client.0.smithi067.stdout: create 255/.snap/snap-subdir-test 2024-03-09T08:55:42.267 INFO:tasks.workunit.client.0.smithi067.stdout: create 255/30 file after the snapshot 2024-03-09T08:55:42.269 INFO:tasks.workunit.client.0.smithi067.stdout: create 255/31 file after the snapshot 2024-03-09T08:55:42.270 INFO:tasks.workunit.client.0.smithi067.stdout: create 255/32 file after the snapshot 2024-03-09T08:55:42.270 INFO:tasks.workunit.client.0.smithi067.stdout: create 255/33 file after the snapshot 2024-03-09T08:55:42.270 INFO:tasks.workunit.client.0.smithi067.stdout: create 255/34 file after the snapshot 2024-03-09T08:55:42.271 INFO:tasks.workunit.client.0.smithi067.stdout: create 255/35 file after the snapshot 2024-03-09T08:55:42.271 INFO:tasks.workunit.client.0.smithi067.stdout: create 255/36 file after the snapshot 2024-03-09T08:55:42.271 INFO:tasks.workunit.client.0.smithi067.stdout: create 255/37 file after the snapshot 2024-03-09T08:55:42.272 INFO:tasks.workunit.client.0.smithi067.stdout: create 255/38 file after the snapshot 2024-03-09T08:55:42.272 INFO:tasks.workunit.client.0.smithi067.stdout: create 255/39 file after the snapshot 2024-03-09T08:55:42.272 INFO:tasks.workunit.client.0.smithi067.stdout: create 256/.snap/snap-subdir-test 2024-03-09T08:55:42.341 INFO:tasks.workunit.client.0.smithi067.stdout: create 256/30 file after the snapshot 2024-03-09T08:55:42.344 INFO:tasks.workunit.client.0.smithi067.stdout: create 256/31 file after the snapshot 2024-03-09T08:55:42.344 INFO:tasks.workunit.client.0.smithi067.stdout: create 256/32 file after the snapshot 2024-03-09T08:55:42.345 INFO:tasks.workunit.client.0.smithi067.stdout: create 256/33 file after the snapshot 2024-03-09T08:55:42.345 INFO:tasks.workunit.client.0.smithi067.stdout: create 256/34 file after the snapshot 2024-03-09T08:55:42.346 INFO:tasks.workunit.client.0.smithi067.stdout: create 256/35 file after the snapshot 2024-03-09T08:55:42.346 INFO:tasks.workunit.client.0.smithi067.stdout: create 256/36 file after the snapshot 2024-03-09T08:55:42.346 INFO:tasks.workunit.client.0.smithi067.stdout: create 256/37 file after the snapshot 2024-03-09T08:55:42.347 INFO:tasks.workunit.client.0.smithi067.stdout: create 256/38 file after the snapshot 2024-03-09T08:55:42.347 INFO:tasks.workunit.client.0.smithi067.stdout: create 256/39 file after the snapshot 2024-03-09T08:55:42.348 INFO:tasks.workunit.client.0.smithi067.stdout: create 257/.snap/snap-subdir-test 2024-03-09T08:55:42.415 INFO:tasks.workunit.client.0.smithi067.stdout: create 257/30 file after the snapshot 2024-03-09T08:55:42.418 INFO:tasks.workunit.client.0.smithi067.stdout: create 257/31 file after the snapshot 2024-03-09T08:55:42.418 INFO:tasks.workunit.client.0.smithi067.stdout: create 257/32 file after the snapshot 2024-03-09T08:55:42.419 INFO:tasks.workunit.client.0.smithi067.stdout: create 257/33 file after the snapshot 2024-03-09T08:55:42.419 INFO:tasks.workunit.client.0.smithi067.stdout: create 257/34 file after the snapshot 2024-03-09T08:55:42.419 INFO:tasks.workunit.client.0.smithi067.stdout: create 257/35 file after the snapshot 2024-03-09T08:55:42.420 INFO:tasks.workunit.client.0.smithi067.stdout: create 257/36 file after the snapshot 2024-03-09T08:55:42.421 INFO:tasks.workunit.client.0.smithi067.stdout: create 257/37 file after the snapshot 2024-03-09T08:55:42.422 INFO:tasks.workunit.client.0.smithi067.stdout: create 257/38 file after the snapshot 2024-03-09T08:55:42.423 INFO:tasks.workunit.client.0.smithi067.stdout: create 257/39 file after the snapshot 2024-03-09T08:55:42.423 INFO:tasks.workunit.client.0.smithi067.stdout: create 258/.snap/snap-subdir-test 2024-03-09T08:55:42.491 INFO:tasks.workunit.client.0.smithi067.stdout: create 258/30 file after the snapshot 2024-03-09T08:55:42.494 INFO:tasks.workunit.client.0.smithi067.stdout: create 258/31 file after the snapshot 2024-03-09T08:55:42.495 INFO:tasks.workunit.client.0.smithi067.stdout: create 258/32 file after the snapshot 2024-03-09T08:55:42.495 INFO:tasks.workunit.client.0.smithi067.stdout: create 258/33 file after the snapshot 2024-03-09T08:55:42.495 INFO:tasks.workunit.client.0.smithi067.stdout: create 258/34 file after the snapshot 2024-03-09T08:55:42.495 INFO:tasks.workunit.client.0.smithi067.stdout: create 258/35 file after the snapshot 2024-03-09T08:55:42.496 INFO:tasks.workunit.client.0.smithi067.stdout: create 258/36 file after the snapshot 2024-03-09T08:55:42.496 INFO:tasks.workunit.client.0.smithi067.stdout: create 258/37 file after the snapshot 2024-03-09T08:55:42.496 INFO:tasks.workunit.client.0.smithi067.stdout: create 258/38 file after the snapshot 2024-03-09T08:55:42.497 INFO:tasks.workunit.client.0.smithi067.stdout: create 258/39 file after the snapshot 2024-03-09T08:55:42.497 INFO:tasks.workunit.client.0.smithi067.stdout: create 259/.snap/snap-subdir-test 2024-03-09T08:55:42.572 INFO:tasks.workunit.client.0.smithi067.stdout: create 259/30 file after the snapshot 2024-03-09T08:55:42.575 INFO:tasks.workunit.client.0.smithi067.stdout: create 259/31 file after the snapshot 2024-03-09T08:55:42.576 INFO:tasks.workunit.client.0.smithi067.stdout: create 259/32 file after the snapshot 2024-03-09T08:55:42.576 INFO:tasks.workunit.client.0.smithi067.stdout: create 259/33 file after the snapshot 2024-03-09T08:55:42.576 INFO:tasks.workunit.client.0.smithi067.stdout: create 259/34 file after the snapshot 2024-03-09T08:55:42.576 INFO:tasks.workunit.client.0.smithi067.stdout: create 259/35 file after the snapshot 2024-03-09T08:55:42.577 INFO:tasks.workunit.client.0.smithi067.stdout: create 259/36 file after the snapshot 2024-03-09T08:55:42.577 INFO:tasks.workunit.client.0.smithi067.stdout: create 259/37 file after the snapshot 2024-03-09T08:55:42.577 INFO:tasks.workunit.client.0.smithi067.stdout: create 259/38 file after the snapshot 2024-03-09T08:55:42.577 INFO:tasks.workunit.client.0.smithi067.stdout: create 259/39 file after the snapshot 2024-03-09T08:55:42.577 INFO:tasks.workunit.client.0.smithi067.stdout: create 260/.snap/snap-subdir-test 2024-03-09T08:55:42.644 INFO:tasks.workunit.client.0.smithi067.stdout: create 260/30 file after the snapshot 2024-03-09T08:55:42.648 INFO:tasks.workunit.client.0.smithi067.stdout: create 260/31 file after the snapshot 2024-03-09T08:55:42.648 INFO:tasks.workunit.client.0.smithi067.stdout: create 260/32 file after the snapshot 2024-03-09T08:55:42.648 INFO:tasks.workunit.client.0.smithi067.stdout: create 260/33 file after the snapshot 2024-03-09T08:55:42.649 INFO:tasks.workunit.client.0.smithi067.stdout: create 260/34 file after the snapshot 2024-03-09T08:55:42.649 INFO:tasks.workunit.client.0.smithi067.stdout: create 260/35 file after the snapshot 2024-03-09T08:55:42.649 INFO:tasks.workunit.client.0.smithi067.stdout: create 260/36 file after the snapshot 2024-03-09T08:55:42.649 INFO:tasks.workunit.client.0.smithi067.stdout: create 260/37 file after the snapshot 2024-03-09T08:55:42.650 INFO:tasks.workunit.client.0.smithi067.stdout: create 260/38 file after the snapshot 2024-03-09T08:55:42.650 INFO:tasks.workunit.client.0.smithi067.stdout: create 260/39 file after the snapshot 2024-03-09T08:55:42.650 INFO:tasks.workunit.client.0.smithi067.stdout: create 261/.snap/snap-subdir-test 2024-03-09T08:55:42.731 INFO:tasks.workunit.client.0.smithi067.stdout: create 261/30 file after the snapshot 2024-03-09T08:55:42.733 INFO:tasks.workunit.client.0.smithi067.stdout: create 261/31 file after the snapshot 2024-03-09T08:55:42.733 INFO:tasks.workunit.client.0.smithi067.stdout: create 261/32 file after the snapshot 2024-03-09T08:55:42.734 INFO:tasks.workunit.client.0.smithi067.stdout: create 261/33 file after the snapshot 2024-03-09T08:55:42.734 INFO:tasks.workunit.client.0.smithi067.stdout: create 261/34 file after the snapshot 2024-03-09T08:55:42.734 INFO:tasks.workunit.client.0.smithi067.stdout: create 261/35 file after the snapshot 2024-03-09T08:55:42.734 INFO:tasks.workunit.client.0.smithi067.stdout: create 261/36 file after the snapshot 2024-03-09T08:55:42.734 INFO:tasks.workunit.client.0.smithi067.stdout: create 261/37 file after the snapshot 2024-03-09T08:55:42.735 INFO:tasks.workunit.client.0.smithi067.stdout: create 261/38 file after the snapshot 2024-03-09T08:55:42.735 INFO:tasks.workunit.client.0.smithi067.stdout: create 261/39 file after the snapshot 2024-03-09T08:55:42.735 INFO:tasks.workunit.client.0.smithi067.stdout: create 262/.snap/snap-subdir-test 2024-03-09T08:55:42.806 INFO:tasks.workunit.client.0.smithi067.stdout: create 262/30 file after the snapshot 2024-03-09T08:55:42.809 INFO:tasks.workunit.client.0.smithi067.stdout: create 262/31 file after the snapshot 2024-03-09T08:55:42.809 INFO:tasks.workunit.client.0.smithi067.stdout: create 262/32 file after the snapshot 2024-03-09T08:55:42.809 INFO:tasks.workunit.client.0.smithi067.stdout: create 262/33 file after the snapshot 2024-03-09T08:55:42.809 INFO:tasks.workunit.client.0.smithi067.stdout: create 262/34 file after the snapshot 2024-03-09T08:55:42.809 INFO:tasks.workunit.client.0.smithi067.stdout: create 262/35 file after the snapshot 2024-03-09T08:55:42.810 INFO:tasks.workunit.client.0.smithi067.stdout: create 262/36 file after the snapshot 2024-03-09T08:55:42.810 INFO:tasks.workunit.client.0.smithi067.stdout: create 262/37 file after the snapshot 2024-03-09T08:55:42.810 INFO:tasks.workunit.client.0.smithi067.stdout: create 262/38 file after the snapshot 2024-03-09T08:55:42.810 INFO:tasks.workunit.client.0.smithi067.stdout: create 262/39 file after the snapshot 2024-03-09T08:55:42.810 INFO:tasks.workunit.client.0.smithi067.stdout: create 263/.snap/snap-subdir-test 2024-03-09T08:55:42.881 INFO:tasks.workunit.client.0.smithi067.stdout: create 263/30 file after the snapshot 2024-03-09T08:55:42.886 INFO:tasks.workunit.client.0.smithi067.stdout: create 263/31 file after the snapshot 2024-03-09T08:55:42.886 INFO:tasks.workunit.client.0.smithi067.stdout: create 263/32 file after the snapshot 2024-03-09T08:55:42.886 INFO:tasks.workunit.client.0.smithi067.stdout: create 263/33 file after the snapshot 2024-03-09T08:55:42.886 INFO:tasks.workunit.client.0.smithi067.stdout: create 263/34 file after the snapshot 2024-03-09T08:55:42.887 INFO:tasks.workunit.client.0.smithi067.stdout: create 263/35 file after the snapshot 2024-03-09T08:55:42.887 INFO:tasks.workunit.client.0.smithi067.stdout: create 263/36 file after the snapshot 2024-03-09T08:55:42.887 INFO:tasks.workunit.client.0.smithi067.stdout: create 263/37 file after the snapshot 2024-03-09T08:55:42.887 INFO:tasks.workunit.client.0.smithi067.stdout: create 263/38 file after the snapshot 2024-03-09T08:55:42.887 INFO:tasks.workunit.client.0.smithi067.stdout: create 263/39 file after the snapshot 2024-03-09T08:55:42.887 INFO:tasks.workunit.client.0.smithi067.stdout: create 264/.snap/snap-subdir-test 2024-03-09T08:55:42.964 INFO:tasks.workunit.client.0.smithi067.stdout: create 264/30 file after the snapshot 2024-03-09T08:55:42.967 INFO:tasks.workunit.client.0.smithi067.stdout: create 264/31 file after the snapshot 2024-03-09T08:55:42.968 INFO:tasks.workunit.client.0.smithi067.stdout: create 264/32 file after the snapshot 2024-03-09T08:55:42.968 INFO:tasks.workunit.client.0.smithi067.stdout: create 264/33 file after the snapshot 2024-03-09T08:55:42.968 INFO:tasks.workunit.client.0.smithi067.stdout: create 264/34 file after the snapshot 2024-03-09T08:55:42.968 INFO:tasks.workunit.client.0.smithi067.stdout: create 264/35 file after the snapshot 2024-03-09T08:55:42.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 264/36 file after the snapshot 2024-03-09T08:55:42.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 264/37 file after the snapshot 2024-03-09T08:55:42.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 264/38 file after the snapshot 2024-03-09T08:55:42.969 INFO:tasks.workunit.client.0.smithi067.stdout: create 264/39 file after the snapshot 2024-03-09T08:55:42.970 INFO:tasks.workunit.client.0.smithi067.stdout: create 265/.snap/snap-subdir-test 2024-03-09T08:55:43.035 INFO:tasks.workunit.client.0.smithi067.stdout: create 265/30 file after the snapshot 2024-03-09T08:55:43.044 INFO:tasks.workunit.client.0.smithi067.stdout: create 265/31 file after the snapshot 2024-03-09T08:55:43.044 INFO:tasks.workunit.client.0.smithi067.stdout: create 265/32 file after the snapshot 2024-03-09T08:55:43.044 INFO:tasks.workunit.client.0.smithi067.stdout: create 265/33 file after the snapshot 2024-03-09T08:55:43.045 INFO:tasks.workunit.client.0.smithi067.stdout: create 265/34 file after the snapshot 2024-03-09T08:55:43.045 INFO:tasks.workunit.client.0.smithi067.stdout: create 265/35 file after the snapshot 2024-03-09T08:55:43.045 INFO:tasks.workunit.client.0.smithi067.stdout: create 265/36 file after the snapshot 2024-03-09T08:55:43.045 INFO:tasks.workunit.client.0.smithi067.stdout: create 265/37 file after the snapshot 2024-03-09T08:55:43.045 INFO:tasks.workunit.client.0.smithi067.stdout: create 265/38 file after the snapshot 2024-03-09T08:55:43.045 INFO:tasks.workunit.client.0.smithi067.stdout: create 265/39 file after the snapshot 2024-03-09T08:55:43.046 INFO:tasks.workunit.client.0.smithi067.stdout: create 266/.snap/snap-subdir-test 2024-03-09T08:55:43.400 INFO:tasks.workunit.client.0.smithi067.stdout: create 266/30 file after the snapshot 2024-03-09T08:55:43.404 INFO:tasks.workunit.client.0.smithi067.stdout: create 266/31 file after the snapshot 2024-03-09T08:55:43.405 INFO:tasks.workunit.client.0.smithi067.stdout: create 266/32 file after the snapshot 2024-03-09T08:55:43.405 INFO:tasks.workunit.client.0.smithi067.stdout: create 266/33 file after the snapshot 2024-03-09T08:55:43.405 INFO:tasks.workunit.client.0.smithi067.stdout: create 266/34 file after the snapshot 2024-03-09T08:55:43.405 INFO:tasks.workunit.client.0.smithi067.stdout: create 266/35 file after the snapshot 2024-03-09T08:55:43.406 INFO:tasks.workunit.client.0.smithi067.stdout: create 266/36 file after the snapshot 2024-03-09T08:55:43.406 INFO:tasks.workunit.client.0.smithi067.stdout: create 266/37 file after the snapshot 2024-03-09T08:55:43.406 INFO:tasks.workunit.client.0.smithi067.stdout: create 266/38 file after the snapshot 2024-03-09T08:55:43.407 INFO:tasks.workunit.client.0.smithi067.stdout: create 266/39 file after the snapshot 2024-03-09T08:55:43.407 INFO:tasks.workunit.client.0.smithi067.stdout: create 267/.snap/snap-subdir-test 2024-03-09T08:55:43.493 INFO:tasks.workunit.client.0.smithi067.stdout: create 267/30 file after the snapshot 2024-03-09T08:55:43.497 INFO:tasks.workunit.client.0.smithi067.stdout: create 267/31 file after the snapshot 2024-03-09T08:55:43.497 INFO:tasks.workunit.client.0.smithi067.stdout: create 267/32 file after the snapshot 2024-03-09T08:55:43.498 INFO:tasks.workunit.client.0.smithi067.stdout: create 267/33 file after the snapshot 2024-03-09T08:55:43.498 INFO:tasks.workunit.client.0.smithi067.stdout: create 267/34 file after the snapshot 2024-03-09T08:55:43.499 INFO:tasks.workunit.client.0.smithi067.stdout: create 267/35 file after the snapshot 2024-03-09T08:55:43.499 INFO:tasks.workunit.client.0.smithi067.stdout: create 267/36 file after the snapshot 2024-03-09T08:55:43.499 INFO:tasks.workunit.client.0.smithi067.stdout: create 267/37 file after the snapshot 2024-03-09T08:55:43.500 INFO:tasks.workunit.client.0.smithi067.stdout: create 267/38 file after the snapshot 2024-03-09T08:55:43.500 INFO:tasks.workunit.client.0.smithi067.stdout: create 267/39 file after the snapshot 2024-03-09T08:55:43.501 INFO:tasks.workunit.client.0.smithi067.stdout: create 268/.snap/snap-subdir-test 2024-03-09T08:55:43.588 INFO:tasks.workunit.client.1.smithi134.stdout: create 286/30 file after the snapshot 2024-03-09T08:55:43.590 INFO:tasks.workunit.client.1.smithi134.stdout: create 286/31 file after the snapshot 2024-03-09T08:55:43.591 INFO:tasks.workunit.client.1.smithi134.stdout: create 286/32 file after the snapshot 2024-03-09T08:55:43.591 INFO:tasks.workunit.client.1.smithi134.stdout: create 286/33 file after the snapshot 2024-03-09T08:55:43.591 INFO:tasks.workunit.client.1.smithi134.stdout: create 286/34 file after the snapshot 2024-03-09T08:55:43.591 INFO:tasks.workunit.client.1.smithi134.stdout: create 286/35 file after the snapshot 2024-03-09T08:55:43.592 INFO:tasks.workunit.client.1.smithi134.stdout: create 286/36 file after the snapshot 2024-03-09T08:55:43.592 INFO:tasks.workunit.client.1.smithi134.stdout: create 286/37 file after the snapshot 2024-03-09T08:55:43.592 INFO:tasks.workunit.client.1.smithi134.stdout: create 286/38 file after the snapshot 2024-03-09T08:55:43.592 INFO:tasks.workunit.client.1.smithi134.stdout: create 286/39 file after the snapshot 2024-03-09T08:55:43.593 INFO:tasks.workunit.client.1.smithi134.stdout: create 287/.snap/snap-subdir-test 2024-03-09T08:55:43.841 INFO:tasks.workunit.client.0.smithi067.stdout: create 268/30 file after the snapshot 2024-03-09T08:55:43.845 INFO:tasks.workunit.client.0.smithi067.stdout: create 268/31 file after the snapshot 2024-03-09T08:55:43.845 INFO:tasks.workunit.client.0.smithi067.stdout: create 268/32 file after the snapshot 2024-03-09T08:55:43.845 INFO:tasks.workunit.client.0.smithi067.stdout: create 268/33 file after the snapshot 2024-03-09T08:55:43.846 INFO:tasks.workunit.client.0.smithi067.stdout: create 268/34 file after the snapshot 2024-03-09T08:55:43.846 INFO:tasks.workunit.client.0.smithi067.stdout: create 268/35 file after the snapshot 2024-03-09T08:55:43.846 INFO:tasks.workunit.client.0.smithi067.stdout: create 268/36 file after the snapshot 2024-03-09T08:55:43.846 INFO:tasks.workunit.client.0.smithi067.stdout: create 268/37 file after the snapshot 2024-03-09T08:55:43.847 INFO:tasks.workunit.client.0.smithi067.stdout: create 268/38 file after the snapshot 2024-03-09T08:55:43.847 INFO:tasks.workunit.client.0.smithi067.stdout: create 268/39 file after the snapshot 2024-03-09T08:55:43.847 INFO:tasks.workunit.client.0.smithi067.stdout: create 269/.snap/snap-subdir-test 2024-03-09T08:55:43.943 INFO:tasks.workunit.client.0.smithi067.stdout: create 269/30 file after the snapshot 2024-03-09T08:55:43.946 INFO:tasks.workunit.client.0.smithi067.stdout: create 269/31 file after the snapshot 2024-03-09T08:55:43.946 INFO:tasks.workunit.client.0.smithi067.stdout: create 269/32 file after the snapshot 2024-03-09T08:55:43.947 INFO:tasks.workunit.client.0.smithi067.stdout: create 269/33 file after the snapshot 2024-03-09T08:55:43.947 INFO:tasks.workunit.client.0.smithi067.stdout: create 269/34 file after the snapshot 2024-03-09T08:55:43.947 INFO:tasks.workunit.client.0.smithi067.stdout: create 269/35 file after the snapshot 2024-03-09T08:55:43.947 INFO:tasks.workunit.client.0.smithi067.stdout: create 269/36 file after the snapshot 2024-03-09T08:55:43.948 INFO:tasks.workunit.client.0.smithi067.stdout: create 269/37 file after the snapshot 2024-03-09T08:55:43.948 INFO:tasks.workunit.client.0.smithi067.stdout: create 269/38 file after the snapshot 2024-03-09T08:55:43.948 INFO:tasks.workunit.client.0.smithi067.stdout: create 269/39 file after the snapshot 2024-03-09T08:55:43.949 INFO:tasks.workunit.client.0.smithi067.stdout: create 270/.snap/snap-subdir-test 2024-03-09T08:55:44.634 INFO:tasks.workunit.client.1.smithi134.stdout: create 287/30 file after the snapshot 2024-03-09T08:55:44.637 INFO:tasks.workunit.client.1.smithi134.stdout: create 287/31 file after the snapshot 2024-03-09T08:55:44.638 INFO:tasks.workunit.client.1.smithi134.stdout: create 287/32 file after the snapshot 2024-03-09T08:55:44.638 INFO:tasks.workunit.client.1.smithi134.stdout: create 287/33 file after the snapshot 2024-03-09T08:55:44.638 INFO:tasks.workunit.client.1.smithi134.stdout: create 287/34 file after the snapshot 2024-03-09T08:55:44.639 INFO:tasks.workunit.client.1.smithi134.stdout: create 287/35 file after the snapshot 2024-03-09T08:55:44.639 INFO:tasks.workunit.client.1.smithi134.stdout: create 287/36 file after the snapshot 2024-03-09T08:55:44.639 INFO:tasks.workunit.client.1.smithi134.stdout: create 287/37 file after the snapshot 2024-03-09T08:55:44.640 INFO:tasks.workunit.client.1.smithi134.stdout: create 287/38 file after the snapshot 2024-03-09T08:55:44.640 INFO:tasks.workunit.client.1.smithi134.stdout: create 287/39 file after the snapshot 2024-03-09T08:55:44.640 INFO:tasks.workunit.client.1.smithi134.stdout: create 288/.snap/snap-subdir-test 2024-03-09T08:55:45.007 INFO:tasks.workunit.client.0.smithi067.stdout: create 270/30 file after the snapshot 2024-03-09T08:55:45.010 INFO:tasks.workunit.client.0.smithi067.stdout: create 270/31 file after the snapshot 2024-03-09T08:55:45.010 INFO:tasks.workunit.client.0.smithi067.stdout: create 270/32 file after the snapshot 2024-03-09T08:55:45.010 INFO:tasks.workunit.client.0.smithi067.stdout: create 270/33 file after the snapshot 2024-03-09T08:55:45.011 INFO:tasks.workunit.client.0.smithi067.stdout: create 270/34 file after the snapshot 2024-03-09T08:55:45.011 INFO:tasks.workunit.client.0.smithi067.stdout: create 270/35 file after the snapshot 2024-03-09T08:55:45.012 INFO:tasks.workunit.client.0.smithi067.stdout: create 270/36 file after the snapshot 2024-03-09T08:55:45.012 INFO:tasks.workunit.client.0.smithi067.stdout: create 270/37 file after the snapshot 2024-03-09T08:55:45.012 INFO:tasks.workunit.client.0.smithi067.stdout: create 270/38 file after the snapshot 2024-03-09T08:55:45.012 INFO:tasks.workunit.client.0.smithi067.stdout: create 270/39 file after the snapshot 2024-03-09T08:55:45.012 INFO:tasks.workunit.client.0.smithi067.stdout: create 271/.snap/snap-subdir-test 2024-03-09T08:55:45.094 INFO:tasks.workunit.client.0.smithi067.stdout: create 271/30 file after the snapshot 2024-03-09T08:55:45.097 INFO:tasks.workunit.client.0.smithi067.stdout: create 271/31 file after the snapshot 2024-03-09T08:55:45.097 INFO:tasks.workunit.client.0.smithi067.stdout: create 271/32 file after the snapshot 2024-03-09T08:55:45.097 INFO:tasks.workunit.client.0.smithi067.stdout: create 271/33 file after the snapshot 2024-03-09T08:55:45.098 INFO:tasks.workunit.client.0.smithi067.stdout: create 271/34 file after the snapshot 2024-03-09T08:55:45.098 INFO:tasks.workunit.client.0.smithi067.stdout: create 271/35 file after the snapshot 2024-03-09T08:55:45.099 INFO:tasks.workunit.client.0.smithi067.stdout: create 271/36 file after the snapshot 2024-03-09T08:55:45.099 INFO:tasks.workunit.client.0.smithi067.stdout: create 271/37 file after the snapshot 2024-03-09T08:55:45.099 INFO:tasks.workunit.client.0.smithi067.stdout: create 271/38 file after the snapshot 2024-03-09T08:55:45.099 INFO:tasks.workunit.client.0.smithi067.stdout: create 271/39 file after the snapshot 2024-03-09T08:55:45.100 INFO:tasks.workunit.client.0.smithi067.stdout: create 272/.snap/snap-subdir-test 2024-03-09T08:55:45.181 INFO:tasks.workunit.client.0.smithi067.stdout: create 272/30 file after the snapshot 2024-03-09T08:55:45.184 INFO:tasks.workunit.client.0.smithi067.stdout: create 272/31 file after the snapshot 2024-03-09T08:55:45.184 INFO:tasks.workunit.client.0.smithi067.stdout: create 272/32 file after the snapshot 2024-03-09T08:55:45.185 INFO:tasks.workunit.client.0.smithi067.stdout: create 272/33 file after the snapshot 2024-03-09T08:55:45.185 INFO:tasks.workunit.client.0.smithi067.stdout: create 272/34 file after the snapshot 2024-03-09T08:55:45.185 INFO:tasks.workunit.client.0.smithi067.stdout: create 272/35 file after the snapshot 2024-03-09T08:55:45.186 INFO:tasks.workunit.client.0.smithi067.stdout: create 272/36 file after the snapshot 2024-03-09T08:55:45.186 INFO:tasks.workunit.client.0.smithi067.stdout: create 272/37 file after the snapshot 2024-03-09T08:55:45.186 INFO:tasks.workunit.client.0.smithi067.stdout: create 272/38 file after the snapshot 2024-03-09T08:55:45.186 INFO:tasks.workunit.client.0.smithi067.stdout: create 272/39 file after the snapshot 2024-03-09T08:55:45.187 INFO:tasks.workunit.client.0.smithi067.stdout: create 273/.snap/snap-subdir-test 2024-03-09T08:55:45.266 INFO:tasks.workunit.client.0.smithi067.stdout: create 273/30 file after the snapshot 2024-03-09T08:55:45.269 INFO:tasks.workunit.client.0.smithi067.stdout: create 273/31 file after the snapshot 2024-03-09T08:55:45.269 INFO:tasks.workunit.client.0.smithi067.stdout: create 273/32 file after the snapshot 2024-03-09T08:55:45.270 INFO:tasks.workunit.client.0.smithi067.stdout: create 273/33 file after the snapshot 2024-03-09T08:55:45.270 INFO:tasks.workunit.client.0.smithi067.stdout: create 273/34 file after the snapshot 2024-03-09T08:55:45.270 INFO:tasks.workunit.client.0.smithi067.stdout: create 273/35 file after the snapshot 2024-03-09T08:55:45.271 INFO:tasks.workunit.client.0.smithi067.stdout: create 273/36 file after the snapshot 2024-03-09T08:55:45.271 INFO:tasks.workunit.client.0.smithi067.stdout: create 273/37 file after the snapshot 2024-03-09T08:55:45.271 INFO:tasks.workunit.client.0.smithi067.stdout: create 273/38 file after the snapshot 2024-03-09T08:55:45.272 INFO:tasks.workunit.client.0.smithi067.stdout: create 273/39 file after the snapshot 2024-03-09T08:55:45.272 INFO:tasks.workunit.client.0.smithi067.stdout: create 274/.snap/snap-subdir-test 2024-03-09T08:55:45.299 INFO:tasks.workunit.client.1.smithi134.stdout: create 288/30 file after the snapshot 2024-03-09T08:55:45.302 INFO:tasks.workunit.client.1.smithi134.stdout: create 288/31 file after the snapshot 2024-03-09T08:55:45.302 INFO:tasks.workunit.client.1.smithi134.stdout: create 288/32 file after the snapshot 2024-03-09T08:55:45.302 INFO:tasks.workunit.client.1.smithi134.stdout: create 288/33 file after the snapshot 2024-03-09T08:55:45.302 INFO:tasks.workunit.client.1.smithi134.stdout: create 288/34 file after the snapshot 2024-03-09T08:55:45.302 INFO:tasks.workunit.client.1.smithi134.stdout: create 288/35 file after the snapshot 2024-03-09T08:55:45.303 INFO:tasks.workunit.client.1.smithi134.stdout: create 288/36 file after the snapshot 2024-03-09T08:55:45.303 INFO:tasks.workunit.client.1.smithi134.stdout: create 288/37 file after the snapshot 2024-03-09T08:55:45.303 INFO:tasks.workunit.client.1.smithi134.stdout: create 288/38 file after the snapshot 2024-03-09T08:55:45.303 INFO:tasks.workunit.client.1.smithi134.stdout: create 288/39 file after the snapshot 2024-03-09T08:55:45.303 INFO:tasks.workunit.client.1.smithi134.stdout: create 289/.snap/snap-subdir-test 2024-03-09T08:55:45.370 INFO:tasks.workunit.client.0.smithi067.stdout: create 274/30 file after the snapshot 2024-03-09T08:55:45.373 INFO:tasks.workunit.client.0.smithi067.stdout: create 274/31 file after the snapshot 2024-03-09T08:55:45.373 INFO:tasks.workunit.client.0.smithi067.stdout: create 274/32 file after the snapshot 2024-03-09T08:55:45.374 INFO:tasks.workunit.client.0.smithi067.stdout: create 274/33 file after the snapshot 2024-03-09T08:55:45.374 INFO:tasks.workunit.client.0.smithi067.stdout: create 274/34 file after the snapshot 2024-03-09T08:55:45.375 INFO:tasks.workunit.client.0.smithi067.stdout: create 274/35 file after the snapshot 2024-03-09T08:55:45.375 INFO:tasks.workunit.client.0.smithi067.stdout: create 274/36 file after the snapshot 2024-03-09T08:55:45.375 INFO:tasks.workunit.client.0.smithi067.stdout: create 274/37 file after the snapshot 2024-03-09T08:55:45.375 INFO:tasks.workunit.client.0.smithi067.stdout: create 274/38 file after the snapshot 2024-03-09T08:55:45.375 INFO:tasks.workunit.client.0.smithi067.stdout: create 274/39 file after the snapshot 2024-03-09T08:55:45.376 INFO:tasks.workunit.client.0.smithi067.stdout: create 275/.snap/snap-subdir-test 2024-03-09T08:55:45.388 INFO:tasks.workunit.client.1.smithi134.stdout: create 289/30 file after the snapshot 2024-03-09T08:55:45.390 INFO:tasks.workunit.client.1.smithi134.stdout: create 289/31 file after the snapshot 2024-03-09T08:55:45.391 INFO:tasks.workunit.client.1.smithi134.stdout: create 289/32 file after the snapshot 2024-03-09T08:55:45.391 INFO:tasks.workunit.client.1.smithi134.stdout: create 289/33 file after the snapshot 2024-03-09T08:55:45.392 INFO:tasks.workunit.client.1.smithi134.stdout: create 289/34 file after the snapshot 2024-03-09T08:55:45.392 INFO:tasks.workunit.client.1.smithi134.stdout: create 289/35 file after the snapshot 2024-03-09T08:55:45.392 INFO:tasks.workunit.client.1.smithi134.stdout: create 289/36 file after the snapshot 2024-03-09T08:55:45.392 INFO:tasks.workunit.client.1.smithi134.stdout: create 289/37 file after the snapshot 2024-03-09T08:55:45.392 INFO:tasks.workunit.client.1.smithi134.stdout: create 289/38 file after the snapshot 2024-03-09T08:55:45.392 INFO:tasks.workunit.client.1.smithi134.stdout: create 289/39 file after the snapshot 2024-03-09T08:55:45.393 INFO:tasks.workunit.client.1.smithi134.stdout: create 290/.snap/snap-subdir-test 2024-03-09T08:55:45.446 INFO:tasks.workunit.client.0.smithi067.stdout: create 275/30 file after the snapshot 2024-03-09T08:55:45.449 INFO:tasks.workunit.client.0.smithi067.stdout: create 275/31 file after the snapshot 2024-03-09T08:55:45.449 INFO:tasks.workunit.client.0.smithi067.stdout: create 275/32 file after the snapshot 2024-03-09T08:55:45.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 275/33 file after the snapshot 2024-03-09T08:55:45.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 275/34 file after the snapshot 2024-03-09T08:55:45.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 275/35 file after the snapshot 2024-03-09T08:55:45.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 275/36 file after the snapshot 2024-03-09T08:55:45.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 275/37 file after the snapshot 2024-03-09T08:55:45.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 275/38 file after the snapshot 2024-03-09T08:55:45.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 275/39 file after the snapshot 2024-03-09T08:55:45.450 INFO:tasks.workunit.client.0.smithi067.stdout: create 276/.snap/snap-subdir-test 2024-03-09T08:55:45.464 INFO:tasks.workunit.client.1.smithi134.stdout: create 290/30 file after the snapshot 2024-03-09T08:55:45.468 INFO:tasks.workunit.client.1.smithi134.stdout: create 290/31 file after the snapshot 2024-03-09T08:55:45.468 INFO:tasks.workunit.client.1.smithi134.stdout: create 290/32 file after the snapshot 2024-03-09T08:55:45.468 INFO:tasks.workunit.client.1.smithi134.stdout: create 290/33 file after the snapshot 2024-03-09T08:55:45.468 INFO:tasks.workunit.client.1.smithi134.stdout: create 290/34 file after the snapshot 2024-03-09T08:55:45.468 INFO:tasks.workunit.client.1.smithi134.stdout: create 290/35 file after the snapshot 2024-03-09T08:55:45.468 INFO:tasks.workunit.client.1.smithi134.stdout: create 290/36 file after the snapshot 2024-03-09T08:55:45.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 290/37 file after the snapshot 2024-03-09T08:55:45.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 290/38 file after the snapshot 2024-03-09T08:55:45.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 290/39 file after the snapshot 2024-03-09T08:55:45.469 INFO:tasks.workunit.client.1.smithi134.stdout: create 291/.snap/snap-subdir-test 2024-03-09T08:55:45.503 INFO:tasks.workunit.client.0.smithi067.stdout: create 276/30 file after the snapshot 2024-03-09T08:55:45.506 INFO:tasks.workunit.client.0.smithi067.stdout: create 276/31 file after the snapshot 2024-03-09T08:55:45.507 INFO:tasks.workunit.client.0.smithi067.stdout: create 276/32 file after the snapshot 2024-03-09T08:55:45.507 INFO:tasks.workunit.client.0.smithi067.stdout: create 276/33 file after the snapshot 2024-03-09T08:55:45.507 INFO:tasks.workunit.client.0.smithi067.stdout: create 276/34 file after the snapshot 2024-03-09T08:55:45.508 INFO:tasks.workunit.client.0.smithi067.stdout: create 276/35 file after the snapshot 2024-03-09T08:55:45.508 INFO:tasks.workunit.client.0.smithi067.stdout: create 276/36 file after the snapshot 2024-03-09T08:55:45.508 INFO:tasks.workunit.client.0.smithi067.stdout: create 276/37 file after the snapshot 2024-03-09T08:55:45.508 INFO:tasks.workunit.client.0.smithi067.stdout: create 276/38 file after the snapshot 2024-03-09T08:55:45.508 INFO:tasks.workunit.client.0.smithi067.stdout: create 276/39 file after the snapshot 2024-03-09T08:55:45.508 INFO:tasks.workunit.client.0.smithi067.stdout: create 277/.snap/snap-subdir-test 2024-03-09T08:55:45.540 INFO:tasks.workunit.client.1.smithi134.stdout: create 291/30 file after the snapshot 2024-03-09T08:55:45.543 INFO:tasks.workunit.client.1.smithi134.stdout: create 291/31 file after the snapshot 2024-03-09T08:55:45.543 INFO:tasks.workunit.client.1.smithi134.stdout: create 291/32 file after the snapshot 2024-03-09T08:55:45.543 INFO:tasks.workunit.client.1.smithi134.stdout: create 291/33 file after the snapshot 2024-03-09T08:55:45.544 INFO:tasks.workunit.client.1.smithi134.stdout: create 291/34 file after the snapshot 2024-03-09T08:55:45.544 INFO:tasks.workunit.client.1.smithi134.stdout: create 291/35 file after the snapshot 2024-03-09T08:55:45.545 INFO:tasks.workunit.client.1.smithi134.stdout: create 291/36 file after the snapshot 2024-03-09T08:55:45.545 INFO:tasks.workunit.client.1.smithi134.stdout: create 291/37 file after the snapshot 2024-03-09T08:55:45.546 INFO:tasks.workunit.client.1.smithi134.stdout: create 291/38 file after the snapshot 2024-03-09T08:55:45.547 INFO:tasks.workunit.client.1.smithi134.stdout: create 291/39 file after the snapshot 2024-03-09T08:55:45.548 INFO:tasks.workunit.client.1.smithi134.stdout: create 292/.snap/snap-subdir-test 2024-03-09T08:55:45.581 INFO:tasks.workunit.client.0.smithi067.stdout: create 277/30 file after the snapshot 2024-03-09T08:55:45.584 INFO:tasks.workunit.client.0.smithi067.stdout: create 277/31 file after the snapshot 2024-03-09T08:55:45.584 INFO:tasks.workunit.client.0.smithi067.stdout: create 277/32 file after the snapshot 2024-03-09T08:55:45.584 INFO:tasks.workunit.client.0.smithi067.stdout: create 277/33 file after the snapshot 2024-03-09T08:55:45.584 INFO:tasks.workunit.client.0.smithi067.stdout: create 277/34 file after the snapshot 2024-03-09T08:55:45.585 INFO:tasks.workunit.client.0.smithi067.stdout: create 277/35 file after the snapshot 2024-03-09T08:55:45.585 INFO:tasks.workunit.client.0.smithi067.stdout: create 277/36 file after the snapshot 2024-03-09T08:55:45.585 INFO:tasks.workunit.client.0.smithi067.stdout: create 277/37 file after the snapshot 2024-03-09T08:55:45.585 INFO:tasks.workunit.client.0.smithi067.stdout: create 277/38 file after the snapshot 2024-03-09T08:55:45.585 INFO:tasks.workunit.client.0.smithi067.stdout: create 277/39 file after the snapshot 2024-03-09T08:55:45.585 INFO:tasks.workunit.client.0.smithi067.stdout: create 278/.snap/snap-subdir-test 2024-03-09T08:55:45.617 INFO:tasks.workunit.client.1.smithi134.stdout: create 292/30 file after the snapshot 2024-03-09T08:55:45.621 INFO:tasks.workunit.client.1.smithi134.stdout: create 292/31 file after the snapshot 2024-03-09T08:55:45.621 INFO:tasks.workunit.client.1.smithi134.stdout: create 292/32 file after the snapshot 2024-03-09T08:55:45.622 INFO:tasks.workunit.client.1.smithi134.stdout: create 292/33 file after the snapshot 2024-03-09T08:55:45.622 INFO:tasks.workunit.client.1.smithi134.stdout: create 292/34 file after the snapshot 2024-03-09T08:55:45.622 INFO:tasks.workunit.client.1.smithi134.stdout: create 292/35 file after the snapshot 2024-03-09T08:55:45.623 INFO:tasks.workunit.client.1.smithi134.stdout: create 292/36 file after the snapshot 2024-03-09T08:55:45.623 INFO:tasks.workunit.client.1.smithi134.stdout: create 292/37 file after the snapshot 2024-03-09T08:55:45.623 INFO:tasks.workunit.client.1.smithi134.stdout: create 292/38 file after the snapshot 2024-03-09T08:55:45.623 INFO:tasks.workunit.client.1.smithi134.stdout: create 292/39 file after the snapshot 2024-03-09T08:55:45.624 INFO:tasks.workunit.client.1.smithi134.stdout: create 293/.snap/snap-subdir-test 2024-03-09T08:55:45.649 INFO:tasks.workunit.client.0.smithi067.stdout: create 278/30 file after the snapshot 2024-03-09T08:55:45.652 INFO:tasks.workunit.client.0.smithi067.stdout: create 278/31 file after the snapshot 2024-03-09T08:55:45.652 INFO:tasks.workunit.client.0.smithi067.stdout: create 278/32 file after the snapshot 2024-03-09T08:55:45.653 INFO:tasks.workunit.client.0.smithi067.stdout: create 278/33 file after the snapshot 2024-03-09T08:55:45.653 INFO:tasks.workunit.client.0.smithi067.stdout: create 278/34 file after the snapshot 2024-03-09T08:55:45.653 INFO:tasks.workunit.client.0.smithi067.stdout: create 278/35 file after the snapshot 2024-03-09T08:55:45.653 INFO:tasks.workunit.client.0.smithi067.stdout: create 278/36 file after the snapshot 2024-03-09T08:55:45.653 INFO:tasks.workunit.client.0.smithi067.stdout: create 278/37 file after the snapshot 2024-03-09T08:55:45.654 INFO:tasks.workunit.client.0.smithi067.stdout: create 278/38 file after the snapshot 2024-03-09T08:55:45.654 INFO:tasks.workunit.client.0.smithi067.stdout: create 278/39 file after the snapshot 2024-03-09T08:55:45.654 INFO:tasks.workunit.client.0.smithi067.stdout: create 279/.snap/snap-subdir-test 2024-03-09T08:55:45.710 INFO:tasks.workunit.client.0.smithi067.stdout: create 279/30 file after the snapshot 2024-03-09T08:55:45.712 INFO:tasks.workunit.client.1.smithi134.stdout: create 293/30 file after the snapshot 2024-03-09T08:55:45.713 INFO:tasks.workunit.client.0.smithi067.stdout: create 279/31 file after the snapshot 2024-03-09T08:55:45.713 INFO:tasks.workunit.client.0.smithi067.stdout: create 279/32 file after the snapshot 2024-03-09T08:55:45.713 INFO:tasks.workunit.client.0.smithi067.stdout: create 279/33 file after the snapshot 2024-03-09T08:55:45.714 INFO:tasks.workunit.client.0.smithi067.stdout: create 279/34 file after the snapshot 2024-03-09T08:55:45.714 INFO:tasks.workunit.client.0.smithi067.stdout: create 279/35 file after the snapshot 2024-03-09T08:55:45.714 INFO:tasks.workunit.client.0.smithi067.stdout: create 279/36 file after the snapshot 2024-03-09T08:55:45.714 INFO:tasks.workunit.client.0.smithi067.stdout: create 279/37 file after the snapshot 2024-03-09T08:55:45.714 INFO:tasks.workunit.client.0.smithi067.stdout: create 279/38 file after the snapshot 2024-03-09T08:55:45.715 INFO:tasks.workunit.client.0.smithi067.stdout: create 279/39 file after the snapshot 2024-03-09T08:55:45.715 INFO:tasks.workunit.client.0.smithi067.stdout: create 280/.snap/snap-subdir-test 2024-03-09T08:55:45.717 INFO:tasks.workunit.client.1.smithi134.stdout: create 293/31 file after the snapshot 2024-03-09T08:55:45.717 INFO:tasks.workunit.client.1.smithi134.stdout: create 293/32 file after the snapshot 2024-03-09T08:55:45.717 INFO:tasks.workunit.client.1.smithi134.stdout: create 293/33 file after the snapshot 2024-03-09T08:55:45.717 INFO:tasks.workunit.client.1.smithi134.stdout: create 293/34 file after the snapshot 2024-03-09T08:55:45.718 INFO:tasks.workunit.client.1.smithi134.stdout: create 293/35 file after the snapshot 2024-03-09T08:55:45.718 INFO:tasks.workunit.client.1.smithi134.stdout: create 293/36 file after the snapshot 2024-03-09T08:55:45.719 INFO:tasks.workunit.client.1.smithi134.stdout: create 293/37 file after the snapshot 2024-03-09T08:55:45.719 INFO:tasks.workunit.client.1.smithi134.stdout: create 293/38 file after the snapshot 2024-03-09T08:55:45.719 INFO:tasks.workunit.client.1.smithi134.stdout: create 293/39 file after the snapshot 2024-03-09T08:55:45.719 INFO:tasks.workunit.client.1.smithi134.stdout: create 294/.snap/snap-subdir-test 2024-03-09T08:55:45.810 INFO:tasks.workunit.client.1.smithi134.stdout: create 294/30 file after the snapshot 2024-03-09T08:55:45.812 INFO:tasks.workunit.client.1.smithi134.stdout: create 294/31 file after the snapshot 2024-03-09T08:55:45.813 INFO:tasks.workunit.client.1.smithi134.stdout: create 294/32 file after the snapshot 2024-03-09T08:55:45.813 INFO:tasks.workunit.client.1.smithi134.stdout: create 294/33 file after the snapshot 2024-03-09T08:55:45.813 INFO:tasks.workunit.client.1.smithi134.stdout: create 294/34 file after the snapshot 2024-03-09T08:55:45.814 INFO:tasks.workunit.client.1.smithi134.stdout: create 294/35 file after the snapshot 2024-03-09T08:55:45.814 INFO:tasks.workunit.client.1.smithi134.stdout: create 294/36 file after the snapshot 2024-03-09T08:55:45.814 INFO:tasks.workunit.client.1.smithi134.stdout: create 294/37 file after the snapshot 2024-03-09T08:55:45.814 INFO:tasks.workunit.client.1.smithi134.stdout: create 294/38 file after the snapshot 2024-03-09T08:55:45.815 INFO:tasks.workunit.client.1.smithi134.stdout: create 294/39 file after the snapshot 2024-03-09T08:55:45.815 INFO:tasks.workunit.client.1.smithi134.stdout: create 295/.snap/snap-subdir-test 2024-03-09T08:55:45.891 INFO:tasks.workunit.client.1.smithi134.stdout: create 295/30 file after the snapshot 2024-03-09T08:55:45.893 INFO:tasks.workunit.client.1.smithi134.stdout: create 295/31 file after the snapshot 2024-03-09T08:55:45.894 INFO:tasks.workunit.client.1.smithi134.stdout: create 295/32 file after the snapshot 2024-03-09T08:55:45.894 INFO:tasks.workunit.client.1.smithi134.stdout: create 295/33 file after the snapshot 2024-03-09T08:55:45.894 INFO:tasks.workunit.client.1.smithi134.stdout: create 295/34 file after the snapshot 2024-03-09T08:55:45.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 295/35 file after the snapshot 2024-03-09T08:55:45.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 295/36 file after the snapshot 2024-03-09T08:55:45.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 295/37 file after the snapshot 2024-03-09T08:55:45.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 295/38 file after the snapshot 2024-03-09T08:55:45.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 295/39 file after the snapshot 2024-03-09T08:55:45.895 INFO:tasks.workunit.client.1.smithi134.stdout: create 296/.snap/snap-subdir-test 2024-03-09T08:55:46.153 INFO:tasks.workunit.client.1.smithi134.stdout: create 296/30 file after the snapshot 2024-03-09T08:55:46.422 INFO:tasks.workunit.client.1.smithi134.stdout: create 296/31 file after the snapshot 2024-03-09T08:55:46.422 INFO:tasks.workunit.client.1.smithi134.stdout: create 296/32 file after the snapshot 2024-03-09T08:55:46.422 INFO:tasks.workunit.client.1.smithi134.stdout: create 296/33 file after the snapshot 2024-03-09T08:55:46.423 INFO:tasks.workunit.client.1.smithi134.stdout: create 296/34 file after the snapshot 2024-03-09T08:55:46.423 INFO:tasks.workunit.client.1.smithi134.stdout: create 296/35 file after the snapshot 2024-03-09T08:55:46.423 INFO:tasks.workunit.client.1.smithi134.stdout: create 296/36 file after the snapshot 2024-03-09T08:55:46.424 INFO:tasks.workunit.client.1.smithi134.stdout: create 296/37 file after the snapshot 2024-03-09T08:55:46.424 INFO:tasks.workunit.client.1.smithi134.stdout: create 296/38 file after the snapshot 2024-03-09T08:55:46.424 INFO:tasks.workunit.client.1.smithi134.stdout: create 296/39 file after the snapshot 2024-03-09T08:55:46.424 INFO:tasks.workunit.client.1.smithi134.stdout: create 297/.snap/snap-subdir-test 2024-03-09T08:55:46.507 INFO:tasks.workunit.client.1.smithi134.stdout: create 297/30 file after the snapshot 2024-03-09T08:55:46.510 INFO:tasks.workunit.client.1.smithi134.stdout: create 297/31 file after the snapshot 2024-03-09T08:55:46.511 INFO:tasks.workunit.client.1.smithi134.stdout: create 297/32 file after the snapshot 2024-03-09T08:55:46.511 INFO:tasks.workunit.client.1.smithi134.stdout: create 297/33 file after the snapshot 2024-03-09T08:55:46.511 INFO:tasks.workunit.client.1.smithi134.stdout: create 297/34 file after the snapshot 2024-03-09T08:55:46.511 INFO:tasks.workunit.client.1.smithi134.stdout: create 297/35 file after the snapshot 2024-03-09T08:55:46.512 INFO:tasks.workunit.client.1.smithi134.stdout: create 297/36 file after the snapshot 2024-03-09T08:55:46.512 INFO:tasks.workunit.client.1.smithi134.stdout: create 297/37 file after the snapshot 2024-03-09T08:55:46.512 INFO:tasks.workunit.client.1.smithi134.stdout: create 297/38 file after the snapshot 2024-03-09T08:55:46.513 INFO:tasks.workunit.client.1.smithi134.stdout: create 297/39 file after the snapshot 2024-03-09T08:55:46.513 INFO:tasks.workunit.client.1.smithi134.stdout: create 298/.snap/snap-subdir-test 2024-03-09T08:55:46.580 INFO:tasks.workunit.client.1.smithi134.stdout: create 298/30 file after the snapshot 2024-03-09T08:55:46.583 INFO:tasks.workunit.client.1.smithi134.stdout: create 298/31 file after the snapshot 2024-03-09T08:55:46.583 INFO:tasks.workunit.client.1.smithi134.stdout: create 298/32 file after the snapshot 2024-03-09T08:55:46.583 INFO:tasks.workunit.client.1.smithi134.stdout: create 298/33 file after the snapshot 2024-03-09T08:55:46.583 INFO:tasks.workunit.client.1.smithi134.stdout: create 298/34 file after the snapshot 2024-03-09T08:55:46.583 INFO:tasks.workunit.client.1.smithi134.stdout: create 298/35 file after the snapshot 2024-03-09T08:55:46.583 INFO:tasks.workunit.client.1.smithi134.stdout: create 298/36 file after the snapshot 2024-03-09T08:55:46.584 INFO:tasks.workunit.client.1.smithi134.stdout: create 298/37 file after the snapshot 2024-03-09T08:55:46.584 INFO:tasks.workunit.client.1.smithi134.stdout: create 298/38 file after the snapshot 2024-03-09T08:55:46.584 INFO:tasks.workunit.client.1.smithi134.stdout: create 298/39 file after the snapshot 2024-03-09T08:55:46.584 INFO:tasks.workunit.client.1.smithi134.stdout: create 299/.snap/snap-subdir-test 2024-03-09T08:55:46.620 INFO:tasks.workunit.client.0.smithi067.stdout: create 280/30 file after the snapshot 2024-03-09T08:55:46.624 INFO:tasks.workunit.client.0.smithi067.stdout: create 280/31 file after the snapshot 2024-03-09T08:55:46.624 INFO:tasks.workunit.client.0.smithi067.stdout: create 280/32 file after the snapshot 2024-03-09T08:55:46.625 INFO:tasks.workunit.client.0.smithi067.stdout: create 280/33 file after the snapshot 2024-03-09T08:55:46.625 INFO:tasks.workunit.client.0.smithi067.stdout: create 280/34 file after the snapshot 2024-03-09T08:55:46.625 INFO:tasks.workunit.client.0.smithi067.stdout: create 280/35 file after the snapshot 2024-03-09T08:55:46.626 INFO:tasks.workunit.client.0.smithi067.stdout: create 280/36 file after the snapshot 2024-03-09T08:55:46.626 INFO:tasks.workunit.client.0.smithi067.stdout: create 280/37 file after the snapshot 2024-03-09T08:55:46.626 INFO:tasks.workunit.client.0.smithi067.stdout: create 280/38 file after the snapshot 2024-03-09T08:55:46.626 INFO:tasks.workunit.client.0.smithi067.stdout: create 280/39 file after the snapshot 2024-03-09T08:55:46.627 INFO:tasks.workunit.client.0.smithi067.stdout: create 281/.snap/snap-subdir-test 2024-03-09T08:55:46.656 INFO:tasks.workunit.client.1.smithi134.stdout: create 299/30 file after the snapshot 2024-03-09T08:55:46.658 INFO:tasks.workunit.client.1.smithi134.stdout: create 299/31 file after the snapshot 2024-03-09T08:55:46.659 INFO:tasks.workunit.client.1.smithi134.stdout: create 299/32 file after the snapshot 2024-03-09T08:55:46.659 INFO:tasks.workunit.client.1.smithi134.stdout: create 299/33 file after the snapshot 2024-03-09T08:55:46.659 INFO:tasks.workunit.client.1.smithi134.stdout: create 299/34 file after the snapshot 2024-03-09T08:55:46.660 INFO:tasks.workunit.client.1.smithi134.stdout: create 299/35 file after the snapshot 2024-03-09T08:55:46.660 INFO:tasks.workunit.client.1.smithi134.stdout: create 299/36 file after the snapshot 2024-03-09T08:55:46.661 INFO:tasks.workunit.client.1.smithi134.stdout: create 299/37 file after the snapshot 2024-03-09T08:55:46.661 INFO:tasks.workunit.client.1.smithi134.stdout: create 299/38 file after the snapshot 2024-03-09T08:55:46.661 INFO:tasks.workunit.client.1.smithi134.stdout: create 299/39 file after the snapshot 2024-03-09T08:55:46.661 INFO:tasks.workunit.client.1.smithi134.stdout:Take second snapshot .snap/test2 2024-03-09T08:55:46.689 INFO:tasks.workunit.client.0.smithi067.stdout: create 281/30 file after the snapshot 2024-03-09T08:55:46.694 INFO:tasks.workunit.client.0.smithi067.stdout: create 281/31 file after the snapshot 2024-03-09T08:55:46.696 INFO:tasks.workunit.client.0.smithi067.stdout: create 281/32 file after the snapshot 2024-03-09T08:55:46.697 INFO:tasks.workunit.client.0.smithi067.stdout: create 281/33 file after the snapshot 2024-03-09T08:55:46.698 INFO:tasks.workunit.client.0.smithi067.stdout: create 281/34 file after the snapshot 2024-03-09T08:55:46.698 INFO:tasks.workunit.client.0.smithi067.stdout: create 281/35 file after the snapshot 2024-03-09T08:55:46.699 INFO:tasks.workunit.client.0.smithi067.stdout: create 281/36 file after the snapshot 2024-03-09T08:55:46.699 INFO:tasks.workunit.client.0.smithi067.stdout: create 281/37 file after the snapshot 2024-03-09T08:55:46.700 INFO:tasks.workunit.client.0.smithi067.stdout: create 281/38 file after the snapshot 2024-03-09T08:55:46.700 INFO:tasks.workunit.client.0.smithi067.stdout: create 281/39 file after the snapshot 2024-03-09T08:55:46.701 INFO:tasks.workunit.client.0.smithi067.stdout: create 282/.snap/snap-subdir-test 2024-03-09T08:55:46.774 INFO:tasks.workunit.client.0.smithi067.stdout: create 282/30 file after the snapshot 2024-03-09T08:55:46.777 INFO:tasks.workunit.client.0.smithi067.stdout: create 282/31 file after the snapshot 2024-03-09T08:55:46.777 INFO:tasks.workunit.client.0.smithi067.stdout: create 282/32 file after the snapshot 2024-03-09T08:55:46.777 INFO:tasks.workunit.client.0.smithi067.stdout: create 282/33 file after the snapshot 2024-03-09T08:55:46.782 INFO:tasks.workunit.client.0.smithi067.stdout: create 282/34 file after the snapshot 2024-03-09T08:55:46.782 INFO:tasks.workunit.client.0.smithi067.stdout: create 282/35 file after the snapshot 2024-03-09T08:55:46.782 INFO:tasks.workunit.client.0.smithi067.stdout: create 282/36 file after the snapshot 2024-03-09T08:55:46.783 INFO:tasks.workunit.client.0.smithi067.stdout: create 282/37 file after the snapshot 2024-03-09T08:55:46.783 INFO:tasks.workunit.client.0.smithi067.stdout: create 282/38 file after the snapshot 2024-03-09T08:55:46.783 INFO:tasks.workunit.client.0.smithi067.stdout: create 282/39 file after the snapshot 2024-03-09T08:55:46.783 INFO:tasks.workunit.client.0.smithi067.stdout: create 283/.snap/snap-subdir-test 2024-03-09T08:55:46.822 INFO:tasks.workunit.client.1.smithi134.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2024-03-09T08:55:46.844 INFO:tasks.workunit.client.0.smithi067.stdout: create 283/30 file after the snapshot 2024-03-09T08:55:46.848 INFO:tasks.workunit.client.0.smithi067.stdout: create 283/31 file after the snapshot 2024-03-09T08:55:46.848 INFO:tasks.workunit.client.0.smithi067.stdout: create 283/32 file after the snapshot 2024-03-09T08:55:46.849 INFO:tasks.workunit.client.0.smithi067.stdout: create 283/33 file after the snapshot 2024-03-09T08:55:46.849 INFO:tasks.workunit.client.0.smithi067.stdout: create 283/34 file after the snapshot 2024-03-09T08:55:46.849 INFO:tasks.workunit.client.0.smithi067.stdout: create 283/35 file after the snapshot 2024-03-09T08:55:46.851 INFO:tasks.workunit.client.0.smithi067.stdout: create 283/36 file after the snapshot 2024-03-09T08:55:46.852 INFO:tasks.workunit.client.0.smithi067.stdout: create 283/37 file after the snapshot 2024-03-09T08:55:46.852 INFO:tasks.workunit.client.0.smithi067.stdout: create 283/38 file after the snapshot 2024-03-09T08:55:46.852 INFO:tasks.workunit.client.0.smithi067.stdout: create 283/39 file after the snapshot 2024-03-09T08:55:46.852 INFO:tasks.workunit.client.0.smithi067.stdout: create 284/.snap/snap-subdir-test 2024-03-09T08:55:46.910 INFO:tasks.workunit.client.0.smithi067.stdout: create 284/30 file after the snapshot 2024-03-09T08:55:46.912 INFO:tasks.workunit.client.0.smithi067.stdout: create 284/31 file after the snapshot 2024-03-09T08:55:46.912 INFO:tasks.workunit.client.0.smithi067.stdout: create 284/32 file after the snapshot 2024-03-09T08:55:46.912 INFO:tasks.workunit.client.0.smithi067.stdout: create 284/33 file after the snapshot 2024-03-09T08:55:46.913 INFO:tasks.workunit.client.0.smithi067.stdout: create 284/34 file after the snapshot 2024-03-09T08:55:46.914 INFO:tasks.workunit.client.0.smithi067.stdout: create 284/35 file after the snapshot 2024-03-09T08:55:46.914 INFO:tasks.workunit.client.0.smithi067.stdout: create 284/36 file after the snapshot 2024-03-09T08:55:46.915 INFO:tasks.workunit.client.0.smithi067.stdout: create 284/37 file after the snapshot 2024-03-09T08:55:46.916 INFO:tasks.workunit.client.0.smithi067.stdout: create 284/38 file after the snapshot 2024-03-09T08:55:46.917 INFO:tasks.workunit.client.0.smithi067.stdout: create 284/39 file after the snapshot 2024-03-09T08:55:46.918 INFO:tasks.workunit.client.0.smithi067.stdout: create 285/.snap/snap-subdir-test 2024-03-09T08:55:47.869 INFO:tasks.workunit.client.0.smithi067.stdout: create 285/30 file after the snapshot 2024-03-09T08:55:47.872 INFO:tasks.workunit.client.0.smithi067.stdout: create 285/31 file after the snapshot 2024-03-09T08:55:47.872 INFO:tasks.workunit.client.0.smithi067.stdout: create 285/32 file after the snapshot 2024-03-09T08:55:47.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 285/33 file after the snapshot 2024-03-09T08:55:47.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 285/34 file after the snapshot 2024-03-09T08:55:47.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 285/35 file after the snapshot 2024-03-09T08:55:47.873 INFO:tasks.workunit.client.0.smithi067.stdout: create 285/36 file after the snapshot 2024-03-09T08:55:47.874 INFO:tasks.workunit.client.0.smithi067.stdout: create 285/37 file after the snapshot 2024-03-09T08:55:47.874 INFO:tasks.workunit.client.0.smithi067.stdout: create 285/38 file after the snapshot 2024-03-09T08:55:47.874 INFO:tasks.workunit.client.0.smithi067.stdout: create 285/39 file after the snapshot 2024-03-09T08:55:47.875 INFO:tasks.workunit.client.0.smithi067.stdout: create 286/.snap/snap-subdir-test 2024-03-09T08:55:47.945 INFO:tasks.workunit.client.0.smithi067.stdout: create 286/30 file after the snapshot 2024-03-09T08:55:47.947 INFO:tasks.workunit.client.0.smithi067.stdout: create 286/31 file after the snapshot 2024-03-09T08:55:47.948 INFO:tasks.workunit.client.0.smithi067.stdout: create 286/32 file after the snapshot 2024-03-09T08:55:47.948 INFO:tasks.workunit.client.0.smithi067.stdout: create 286/33 file after the snapshot 2024-03-09T08:55:47.948 INFO:tasks.workunit.client.0.smithi067.stdout: create 286/34 file after the snapshot 2024-03-09T08:55:47.949 INFO:tasks.workunit.client.0.smithi067.stdout: create 286/35 file after the snapshot 2024-03-09T08:55:47.949 INFO:tasks.workunit.client.0.smithi067.stdout: create 286/36 file after the snapshot 2024-03-09T08:55:47.950 INFO:tasks.workunit.client.0.smithi067.stdout: create 286/37 file after the snapshot 2024-03-09T08:55:47.950 INFO:tasks.workunit.client.0.smithi067.stdout: create 286/38 file after the snapshot 2024-03-09T08:55:47.950 INFO:tasks.workunit.client.0.smithi067.stdout: create 286/39 file after the snapshot 2024-03-09T08:55:47.951 INFO:tasks.workunit.client.0.smithi067.stdout: create 287/.snap/snap-subdir-test 2024-03-09T08:55:48.018 INFO:tasks.workunit.client.0.smithi067.stdout: create 287/30 file after the snapshot 2024-03-09T08:55:48.019 INFO:tasks.workunit.client.0.smithi067.stdout: create 287/31 file after the snapshot 2024-03-09T08:55:48.019 INFO:tasks.workunit.client.0.smithi067.stdout: create 287/32 file after the snapshot 2024-03-09T08:55:48.019 INFO:tasks.workunit.client.0.smithi067.stdout: create 287/33 file after the snapshot 2024-03-09T08:55:48.020 INFO:tasks.workunit.client.0.smithi067.stdout: create 287/34 file after the snapshot 2024-03-09T08:55:48.020 INFO:tasks.workunit.client.0.smithi067.stdout: create 287/35 file after the snapshot 2024-03-09T08:55:48.020 INFO:tasks.workunit.client.0.smithi067.stdout: create 287/36 file after the snapshot 2024-03-09T08:55:48.021 INFO:tasks.workunit.client.0.smithi067.stdout: create 287/37 file after the snapshot 2024-03-09T08:55:48.021 INFO:tasks.workunit.client.0.smithi067.stdout: create 287/38 file after the snapshot 2024-03-09T08:55:48.021 INFO:tasks.workunit.client.0.smithi067.stdout: create 287/39 file after the snapshot 2024-03-09T08:55:48.023 INFO:tasks.workunit.client.0.smithi067.stdout: create 288/.snap/snap-subdir-test 2024-03-09T08:55:48.098 INFO:tasks.workunit.client.0.smithi067.stdout: create 288/30 file after the snapshot 2024-03-09T08:55:48.101 INFO:tasks.workunit.client.0.smithi067.stdout: create 288/31 file after the snapshot 2024-03-09T08:55:48.101 INFO:tasks.workunit.client.0.smithi067.stdout: create 288/32 file after the snapshot 2024-03-09T08:55:48.102 INFO:tasks.workunit.client.0.smithi067.stdout: create 288/33 file after the snapshot 2024-03-09T08:55:48.102 INFO:tasks.workunit.client.0.smithi067.stdout: create 288/34 file after the snapshot 2024-03-09T08:55:48.103 INFO:tasks.workunit.client.0.smithi067.stdout: create 288/35 file after the snapshot 2024-03-09T08:55:48.103 INFO:tasks.workunit.client.0.smithi067.stdout: create 288/36 file after the snapshot 2024-03-09T08:55:48.103 INFO:tasks.workunit.client.0.smithi067.stdout: create 288/37 file after the snapshot 2024-03-09T08:55:48.103 INFO:tasks.workunit.client.0.smithi067.stdout: create 288/38 file after the snapshot 2024-03-09T08:55:48.103 INFO:tasks.workunit.client.0.smithi067.stdout: create 288/39 file after the snapshot 2024-03-09T08:55:48.104 INFO:tasks.workunit.client.0.smithi067.stdout: create 289/.snap/snap-subdir-test 2024-03-09T08:55:48.458 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2024-03-09T08:55:48.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2024-03-09T08:55:48.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2024-03-09T08:55:48.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2024-03-09T08:55:48.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2024-03-09T08:55:48.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2024-03-09T08:55:48.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2024-03-09T08:55:49.118 INFO:tasks.workunit.client.0.smithi067.stdout: create 289/30 file after the snapshot 2024-03-09T08:55:49.121 INFO:tasks.workunit.client.0.smithi067.stdout: create 289/31 file after the snapshot 2024-03-09T08:55:49.121 INFO:tasks.workunit.client.0.smithi067.stdout: create 289/32 file after the snapshot 2024-03-09T08:55:49.121 INFO:tasks.workunit.client.0.smithi067.stdout: create 289/33 file after the snapshot 2024-03-09T08:55:49.121 INFO:tasks.workunit.client.0.smithi067.stdout: create 289/34 file after the snapshot 2024-03-09T08:55:49.121 INFO:tasks.workunit.client.0.smithi067.stdout: create 289/35 file after the snapshot 2024-03-09T08:55:49.122 INFO:tasks.workunit.client.0.smithi067.stdout: create 289/36 file after the snapshot 2024-03-09T08:55:49.122 INFO:tasks.workunit.client.0.smithi067.stdout: create 289/37 file after the snapshot 2024-03-09T08:55:49.122 INFO:tasks.workunit.client.0.smithi067.stdout: create 289/38 file after the snapshot 2024-03-09T08:55:49.123 INFO:tasks.workunit.client.0.smithi067.stdout: create 289/39 file after the snapshot 2024-03-09T08:55:49.123 INFO:tasks.workunit.client.0.smithi067.stdout: create 290/.snap/snap-subdir-test 2024-03-09T08:55:49.530 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2024-03-09T08:55:49.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2024-03-09T08:55:49.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2024-03-09T08:55:49.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2024-03-09T08:55:49.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2024-03-09T08:55:49.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2024-03-09T08:55:49.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2024-03-09T08:55:49.537 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2024-03-09T08:55:49.537 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2024-03-09T08:55:49.537 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2024-03-09T08:55:49.537 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2024-03-09T08:55:49.537 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2024-03-09T08:55:49.537 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2024-03-09T08:55:49.537 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2024-03-09T08:55:49.537 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2024-03-09T08:55:49.931 INFO:tasks.workunit.client.0.smithi067.stdout: create 290/30 file after the snapshot 2024-03-09T08:55:49.934 INFO:tasks.workunit.client.0.smithi067.stdout: create 290/31 file after the snapshot 2024-03-09T08:55:49.934 INFO:tasks.workunit.client.0.smithi067.stdout: create 290/32 file after the snapshot 2024-03-09T08:55:49.935 INFO:tasks.workunit.client.0.smithi067.stdout: create 290/33 file after the snapshot 2024-03-09T08:55:49.935 INFO:tasks.workunit.client.0.smithi067.stdout: create 290/34 file after the snapshot 2024-03-09T08:55:49.935 INFO:tasks.workunit.client.0.smithi067.stdout: create 290/35 file after the snapshot 2024-03-09T08:55:49.935 INFO:tasks.workunit.client.0.smithi067.stdout: create 290/36 file after the snapshot 2024-03-09T08:55:49.936 INFO:tasks.workunit.client.0.smithi067.stdout: create 290/37 file after the snapshot 2024-03-09T08:55:49.936 INFO:tasks.workunit.client.0.smithi067.stdout: create 290/38 file after the snapshot 2024-03-09T08:55:49.936 INFO:tasks.workunit.client.0.smithi067.stdout: create 290/39 file after the snapshot 2024-03-09T08:55:49.937 INFO:tasks.workunit.client.0.smithi067.stdout: create 291/.snap/snap-subdir-test 2024-03-09T08:55:50.023 INFO:tasks.workunit.client.0.smithi067.stdout: create 291/30 file after the snapshot 2024-03-09T08:55:50.026 INFO:tasks.workunit.client.0.smithi067.stdout: create 291/31 file after the snapshot 2024-03-09T08:55:50.026 INFO:tasks.workunit.client.0.smithi067.stdout: create 291/32 file after the snapshot 2024-03-09T08:55:50.027 INFO:tasks.workunit.client.0.smithi067.stdout: create 291/33 file after the snapshot 2024-03-09T08:55:50.027 INFO:tasks.workunit.client.0.smithi067.stdout: create 291/34 file after the snapshot 2024-03-09T08:55:50.027 INFO:tasks.workunit.client.0.smithi067.stdout: create 291/35 file after the snapshot 2024-03-09T08:55:50.027 INFO:tasks.workunit.client.0.smithi067.stdout: create 291/36 file after the snapshot 2024-03-09T08:55:50.027 INFO:tasks.workunit.client.0.smithi067.stdout: create 291/37 file after the snapshot 2024-03-09T08:55:50.028 INFO:tasks.workunit.client.0.smithi067.stdout: create 291/38 file after the snapshot 2024-03-09T08:55:50.028 INFO:tasks.workunit.client.0.smithi067.stdout: create 291/39 file after the snapshot 2024-03-09T08:55:50.028 INFO:tasks.workunit.client.0.smithi067.stdout: create 292/.snap/snap-subdir-test 2024-03-09T08:55:50.112 INFO:tasks.workunit.client.0.smithi067.stdout: create 292/30 file after the snapshot 2024-03-09T08:55:50.143 INFO:tasks.workunit.client.0.smithi067.stdout: create 292/31 file after the snapshot 2024-03-09T08:55:50.143 INFO:tasks.workunit.client.0.smithi067.stdout: create 292/32 file after the snapshot 2024-03-09T08:55:50.143 INFO:tasks.workunit.client.0.smithi067.stdout: create 292/33 file after the snapshot 2024-03-09T08:55:50.143 INFO:tasks.workunit.client.0.smithi067.stdout: create 292/34 file after the snapshot 2024-03-09T08:55:50.143 INFO:tasks.workunit.client.0.smithi067.stdout: create 292/35 file after the snapshot 2024-03-09T08:55:50.144 INFO:tasks.workunit.client.0.smithi067.stdout: create 292/36 file after the snapshot 2024-03-09T08:55:50.144 INFO:tasks.workunit.client.0.smithi067.stdout: create 292/37 file after the snapshot 2024-03-09T08:55:50.144 INFO:tasks.workunit.client.0.smithi067.stdout: create 292/38 file after the snapshot 2024-03-09T08:55:50.144 INFO:tasks.workunit.client.0.smithi067.stdout: create 292/39 file after the snapshot 2024-03-09T08:55:50.144 INFO:tasks.workunit.client.0.smithi067.stdout: create 293/.snap/snap-subdir-test 2024-03-09T08:55:50.317 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2024-03-09T08:55:50.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2024-03-09T08:55:50.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2024-03-09T08:55:50.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2024-03-09T08:55:50.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2024-03-09T08:55:50.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2024-03-09T08:55:50.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2024-03-09T08:55:50.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2024-03-09T08:55:50.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2024-03-09T08:55:50.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2024-03-09T08:55:50.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2024-03-09T08:55:50.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2024-03-09T08:55:50.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2024-03-09T08:55:50.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2024-03-09T08:55:50.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2024-03-09T08:55:50.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2024-03-09T08:55:50.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2024-03-09T08:55:50.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2024-03-09T08:55:50.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2024-03-09T08:55:50.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2024-03-09T08:55:50.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2024-03-09T08:55:50.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2024-03-09T08:55:50.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2024-03-09T08:55:50.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2024-03-09T08:55:50.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2024-03-09T08:55:50.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2024-03-09T08:55:50.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2024-03-09T08:55:50.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2024-03-09T08:55:50.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2024-03-09T08:55:50.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2024-03-09T08:55:50.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2024-03-09T08:55:50.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2024-03-09T08:55:50.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2024-03-09T08:55:50.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2024-03-09T08:55:50.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2024-03-09T08:55:50.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2024-03-09T08:55:50.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2024-03-09T08:55:50.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2024-03-09T08:55:50.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2024-03-09T08:55:50.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2024-03-09T08:55:50.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2024-03-09T08:55:50.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2024-03-09T08:55:50.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2024-03-09T08:55:50.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2024-03-09T08:55:50.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2024-03-09T08:55:50.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2024-03-09T08:55:50.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2024-03-09T08:55:50.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2024-03-09T08:55:50.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2024-03-09T08:55:50.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2024-03-09T08:55:50.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2024-03-09T08:55:50.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2024-03-09T08:55:50.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2024-03-09T08:55:50.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2024-03-09T08:55:50.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2024-03-09T08:55:50.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2024-03-09T08:55:50.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2024-03-09T08:55:51.059 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:55:51.059 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:55:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.273+0000 7fed55400640 1 Processor -- start 2024-03-09T08:55:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.274+0000 7fed55400640 1 -- start start 2024-03-09T08:55:51.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.275+0000 7fed55400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed50150080 0x7fed5006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.275+0000 7fed55400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fed5006e8d0 0x7fed500c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.275+0000 7fed55400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed50155dc0 con 0x7fed500c8930 2024-03-09T08:55:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.275+0000 7fed55400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed50156c90 con 0x7fed5006e8d0 2024-03-09T08:55:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.275+0000 7fed55400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed50156080 con 0x7fed50150080 2024-03-09T08:55:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.275+0000 7fed4f200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fed500c8930 0x7fed50159570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46446/0 (socket says 172.21.15.67:46446) 2024-03-09T08:55:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.275+0000 7fed4f200640 1 -- 172.21.15.67:0/3501671172 learned_addr learned my addr 172.21.15.67:0/3501671172 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.275+0000 7fed45e00640 1 --2- 172.21.15.67:0/3501671172 >> v2:172.21.15.134:3300/0 conn(0x7fed5006e8d0 0x7fed500c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.276+0000 7fed4e800640 1 --2- 172.21.15.67:0/3501671172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed50150080 0x7fed5006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.276+0000 7fed4de00640 1 -- 172.21.15.67:0/3501671172 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226459448 0 0) 0x7fed50155dc0 con 0x7fed500c8930 2024-03-09T08:55:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.276+0000 7fed4de00640 1 -- 172.21.15.67:0/3501671172 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed34003520 con 0x7fed500c8930 2024-03-09T08:55:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.276+0000 7fed45e00640 1 -- 172.21.15.67:0/3501671172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed50150080 msgr2=0x7fed5006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.276+0000 7fed45e00640 1 --2- 172.21.15.67:0/3501671172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed50150080 0x7fed5006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.276+0000 7fed45e00640 1 -- 172.21.15.67:0/3501671172 >> v1:172.21.15.67:6789/0 conn(0x7fed500c8930 legacy=0x7fed50159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.276+0000 7fed4de00640 1 -- 172.21.15.67:0/3501671172 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2425636564 0 0) 0x7fed34003520 con 0x7fed500c8930 2024-03-09T08:55:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.276+0000 7fed45e00640 1 -- 172.21.15.67:0/3501671172 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed50156360 con 0x7fed5006e8d0 2024-03-09T08:55:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.278+0000 7fed45e00640 1 --2- 172.21.15.67:0/3501671172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5006e8d0 0x7fed500c83f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fed380097a0 tx=0x7fed3802b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=789399b262528906 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.278+0000 7fed4de00640 1 -- 172.21.15.67:0/3501671172 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38037070 con 0x7fed5006e8d0 2024-03-09T08:55:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.278+0000 7fed4de00640 1 -- 172.21.15.67:0/3501671172 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed38030510 con 0x7fed5006e8d0 2024-03-09T08:55:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.278+0000 7fed4de00640 1 -- 172.21.15.67:0/3501671172 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48001670 con 0x7fed5006e8d0 2024-03-09T08:55:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.278+0000 7fed55400640 1 -- 172.21.15.67:0/3501671172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5006e8d0 msgr2=0x7fed500c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.278+0000 7fed55400640 1 --2- 172.21.15.67:0/3501671172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5006e8d0 0x7fed500c83f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fed380097a0 tx=0x7fed3802b710 comp rx=0 tx=0).stop 2024-03-09T08:55:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.279+0000 7fed55400640 1 -- 172.21.15.67:0/3501671172 shutdown_connections 2024-03-09T08:55:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.279+0000 7fed55400640 1 --2- 172.21.15.67:0/3501671172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5006e8d0 0x7fed500c83f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.279+0000 7fed55400640 1 --2- 172.21.15.67:0/3501671172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed50150080 0x7fed5006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.279+0000 7fed55400640 1 -- 172.21.15.67:0/3501671172 >> 172.21.15.67:0/3501671172 conn(0x7fed500aa890 msgr2=0x7fed500ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:51.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.279+0000 7fed55400640 1 -- 172.21.15.67:0/3501671172 shutdown_connections 2024-03-09T08:55:51.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.279+0000 7fed55400640 1 -- 172.21.15.67:0/3501671172 wait complete. 2024-03-09T08:55:51.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.280+0000 7fed55400640 1 Processor -- start 2024-03-09T08:55:51.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.281+0000 7fed55400640 1 -- start start 2024-03-09T08:55:51.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.281+0000 7fed55400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5006e8d0 0x7fed50158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.281+0000 7fed55400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed500c8930 0x7fed50157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.281+0000 7fed55400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed50150080 0x7fed500c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.281+0000 7fed55400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed50150460 con 0x7fed500c8930 2024-03-09T08:55:51.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.281+0000 7fed55400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed50156920 con 0x7fed5006e8d0 2024-03-09T08:55:51.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.281+0000 7fed55400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed50159c80 con 0x7fed50150080 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.282+0000 7fed45e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed500c8930 0x7fed50157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.282+0000 7fed45e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed500c8930 0x7fed50157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56328/0 (socket says 172.21.15.67:56328) 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.282+0000 7fed45e00640 1 -- 172.21.15.67:0/1875405310 learned_addr learned my addr 172.21.15.67:0/1875405310 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.282+0000 7fed4e800640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5006e8d0 0x7fed50158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.282+0000 7fed4f200640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed50150080 0x7fed500c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.282+0000 7fed45e00640 1 -- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed50150080 msgr2=0x7fed500c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.282+0000 7fed45e00640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed50150080 0x7fed500c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.282+0000 7fed45e00640 1 -- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5006e8d0 msgr2=0x7fed50158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.282+0000 7fed45e00640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5006e8d0 0x7fed50158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.282+0000 7fed45e00640 1 -- 172.21.15.67:0/1875405310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed501423e0 con 0x7fed500c8930 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.283+0000 7fed45e00640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed500c8930 0x7fed50157c40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed380098c0 tx=0x7fed380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.283+0000 7fed47400640 1 -- 172.21.15.67:0/1875405310 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38041020 con 0x7fed500c8930 2024-03-09T08:55:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.283+0000 7fed47400640 1 -- 172.21.15.67:0/1875405310 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3803c870 con 0x7fed500c8930 2024-03-09T08:55:51.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.283+0000 7fed47400640 1 -- 172.21.15.67:0/1875405310 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38037040 con 0x7fed500c8930 2024-03-09T08:55:51.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.283+0000 7fed55400640 1 -- 172.21.15.67:0/1875405310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed50157160 con 0x7fed500c8930 2024-03-09T08:55:51.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.283+0000 7fed55400640 1 -- 172.21.15.67:0/1875405310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed500c7a60 con 0x7fed500c8930 2024-03-09T08:55:51.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.284+0000 7fed55400640 1 -- 172.21.15.67:0/1875405310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed50075a00 con 0x7fed500c8930 2024-03-09T08:55:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.285+0000 7fed47400640 1 -- 172.21.15.67:0/1875405310 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fed38044020 con 0x7fed500c8930 2024-03-09T08:55:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.286+0000 7fed47400640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed2c06b500 0x7fed2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.286+0000 7fed47400640 1 -- 172.21.15.67:0/1875405310 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fed380abde0 con 0x7fed500c8930 2024-03-09T08:55:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.287+0000 7fed4e800640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed2c06b500 0x7fed2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.287+0000 7fed4e800640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed2c06b500 0x7fed2c06d9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fed40011340 tx=0x7fed4000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:51.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.296+0000 7fed47400640 1 -- 172.21.15.67:0/1875405310 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fed3804b030 con 0x7fed500c8930 2024-03-09T08:55:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.569+0000 7fed55400640 1 -- 172.21.15.67:0/1875405310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fed500ae1f0 con 0x7fed500c8930 2024-03-09T08:55:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.570+0000 7fed47400640 1 -- 172.21.15.67:0/1875405310 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fed3807a240 con 0x7fed500c8930 2024-03-09T08:55:51.568 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:55:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.572+0000 7fed55400640 1 -- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed2c06b500 msgr2=0x7fed2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.572+0000 7fed55400640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed2c06b500 0x7fed2c06d9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fed40011340 tx=0x7fed4000b040 comp rx=0 tx=0).stop 2024-03-09T08:55:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.572+0000 7fed55400640 1 -- 172.21.15.67:0/1875405310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed500c8930 msgr2=0x7fed50157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.572+0000 7fed55400640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed500c8930 0x7fed50157c40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed380098c0 tx=0x7fed380046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.572+0000 7fed55400640 1 -- 172.21.15.67:0/1875405310 shutdown_connections 2024-03-09T08:55:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.572+0000 7fed55400640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed2c06b500 0x7fed2c06d9c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.572+0000 7fed55400640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed50150080 0x7fed500c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.572+0000 7fed55400640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed500c8930 0x7fed50157c40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.572+0000 7fed55400640 1 --2- 172.21.15.67:0/1875405310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed5006e8d0 0x7fed50158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.572+0000 7fed55400640 1 -- 172.21.15.67:0/1875405310 >> 172.21.15.67:0/1875405310 conn(0x7fed500aa890 msgr2=0x7fed5006ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:51.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.573+0000 7fed55400640 1 -- 172.21.15.67:0/1875405310 shutdown_connections 2024-03-09T08:55:51.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.573+0000 7fed55400640 1 -- 172.21.15.67:0/1875405310 wait complete. 2024-03-09T08:55:51.591 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:55:51.591 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T08:55:51.591 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:55:51.591 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:55:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.843+0000 7fab4d800640 1 Processor -- start 2024-03-09T08:55:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.843+0000 7fab4d800640 1 -- start start 2024-03-09T08:55:51.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.844+0000 7fab4d800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fab480c95d0 0x7fab48159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.844+0000 7fab4d800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab48159850 0x7fab480c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.845+0000 7fab4d800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab48159d60 con 0x7fab48153220 2024-03-09T08:55:51.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.845+0000 7fab4d800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4815a040 con 0x7fab48159850 2024-03-09T08:55:51.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.845+0000 7fab4d800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab480c8060 con 0x7fab480c95d0 2024-03-09T08:55:51.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.845+0000 7fab4ce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fab48159850 0x7fab480c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.845+0000 7fab47e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fab48153220 0x7fab480c8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46462/0 (socket says 172.21.15.67:46462) 2024-03-09T08:55:51.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.845+0000 7fab47e00640 1 -- 172.21.15.67:0/3760732171 learned_addr learned my addr 172.21.15.67:0/3760732171 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:51.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.845+0000 7fab47400640 1 --2- 172.21.15.67:0/3760732171 >> v2:172.21.15.134:3300/0 conn(0x7fab480c95d0 0x7fab48159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.845+0000 7fab46a00640 1 -- 172.21.15.67:0/3760732171 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469192809 0 0) 0x7fab48159d60 con 0x7fab48153220 2024-03-09T08:55:51.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.845+0000 7fab4ce00640 1 -- 172.21.15.67:0/3760732171 >> v2:172.21.15.134:3300/0 conn(0x7fab480c95d0 msgr2=0x7fab48159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:51.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.845+0000 7fab4ce00640 1 --2- 172.21.15.67:0/3760732171 >> v2:172.21.15.134:3300/0 conn(0x7fab480c95d0 0x7fab48159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.845+0000 7fab4ce00640 1 -- 172.21.15.67:0/3760732171 >> v1:172.21.15.67:6789/0 conn(0x7fab48153220 legacy=0x7fab480c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.846+0000 7fab4ce00640 1 -- 172.21.15.67:0/3760732171 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480c8340 con 0x7fab48159850 2024-03-09T08:55:51.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.847+0000 7fab4ce00640 1 --2- 172.21.15.67:0/3760732171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab48159850 0x7fab480c5e30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fab48156e90 tx=0x7fab4002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acdadf45f7ac019 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:51.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.847+0000 7fab46a00640 1 -- 172.21.15.67:0/3760732171 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40039070 con 0x7fab48159850 2024-03-09T08:55:51.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.847+0000 7fab46a00640 1 -- 172.21.15.67:0/3760732171 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4002cbf0 con 0x7fab48159850 2024-03-09T08:55:51.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.847+0000 7fab46a00640 1 -- 172.21.15.67:0/3760732171 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40034440 con 0x7fab48159850 2024-03-09T08:55:51.846 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2024-03-09T08:55:51.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2024-03-09T08:55:51.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2024-03-09T08:55:51.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2024-03-09T08:55:51.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2024-03-09T08:55:51.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2024-03-09T08:55:51.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2024-03-09T08:55:51.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2024-03-09T08:55:51.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2024-03-09T08:55:51.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2024-03-09T08:55:51.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2024-03-09T08:55:51.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2024-03-09T08:55:51.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2024-03-09T08:55:51.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2024-03-09T08:55:51.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2024-03-09T08:55:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.848+0000 7fab4d800640 1 -- 172.21.15.67:0/3760732171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab48159850 msgr2=0x7fab480c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.848+0000 7fab4d800640 1 --2- 172.21.15.67:0/3760732171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab48159850 0x7fab480c5e30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fab48156e90 tx=0x7fab4002be40 comp rx=0 tx=0).stop 2024-03-09T08:55:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.848+0000 7fab4d800640 1 -- 172.21.15.67:0/3760732171 shutdown_connections 2024-03-09T08:55:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.848+0000 7fab4d800640 1 --2- 172.21.15.67:0/3760732171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab48159850 0x7fab480c5e30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.848+0000 7fab4d800640 1 --2- 172.21.15.67:0/3760732171 >> v2:172.21.15.134:3300/0 conn(0x7fab480c95d0 0x7fab48159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.848+0000 7fab4d800640 1 -- 172.21.15.67:0/3760732171 >> 172.21.15.67:0/3760732171 conn(0x7fab480ad060 msgr2=0x7fab480ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.848+0000 7fab4d800640 1 -- 172.21.15.67:0/3760732171 shutdown_connections 2024-03-09T08:55:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.848+0000 7fab4d800640 1 -- 172.21.15.67:0/3760732171 wait complete. 2024-03-09T08:55:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.851+0000 7fab4d800640 1 Processor -- start 2024-03-09T08:55:51.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.851+0000 7fab4d800640 1 -- start start 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.852+0000 7fab4d800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab480c95d0 0x7fab48157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.852+0000 7fab4d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab48153220 0x7fab480c1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.852+0000 7fab4d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab48159850 0x7fab48158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.852+0000 7fab4d800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab481560a0 con 0x7fab48153220 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.852+0000 7fab4d800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48155f20 con 0x7fab48159850 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.852+0000 7fab4d800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab48153600 con 0x7fab480c95d0 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.852+0000 7fab47400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab48153220 0x7fab480c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.852+0000 7fab47400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab48153220 0x7fab480c1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56354/0 (socket says 172.21.15.67:56354) 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.852+0000 7fab47400640 1 -- 172.21.15.67:0/3153460637 learned_addr learned my addr 172.21.15.67:0/3153460637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.853+0000 7fab47e00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab480c95d0 0x7fab48157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.853+0000 7fab4ce00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab48159850 0x7fab48158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.853+0000 7fab47400640 1 -- 172.21.15.67:0/3153460637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab480c95d0 msgr2=0x7fab48157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.853+0000 7fab47400640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab480c95d0 0x7fab48157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.853+0000 7fab47400640 1 -- 172.21.15.67:0/3153460637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab48159850 msgr2=0x7fab48158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.853+0000 7fab47400640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab48159850 0x7fab48158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.853+0000 7fab47400640 1 -- 172.21.15.67:0/3153460637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480bef20 con 0x7fab48153220 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.854+0000 7fab47400640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab48153220 0x7fab480c1950 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fab3000e9b0 tx=0x7fab3000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.854+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30017070 con 0x7fab48153220 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.854+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3000c9a0 con 0x7fab48153220 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.854+0000 7fab4d800640 1 -- 172.21.15.67:0/3153460637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab480bf200 con 0x7fab48153220 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.854+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3000cd90 con 0x7fab48153220 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.854+0000 7fab4d800640 1 -- 172.21.15.67:0/3153460637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab481592e0 con 0x7fab48153220 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.855+0000 7fab4d800640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab4814f710 0x7fab4814faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.855+0000 7fab4d800640 1 -- 172.21.15.67:0/3153460637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48153d50 con 0x7fab4814f710 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.856+0000 7fab47e00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab4814f710 0x7fab4814faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.856+0000 7fab47e00640 1 -- 172.21.15.67:0/3153460637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab4804f3c0 con 0x7fab4814f710 2024-03-09T08:55:51.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.857+0000 7fab47e00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab4814f710 0x7fab4814faf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fab4814d3d0 tx=0x7fab38001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.857+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fab3001a070 con 0x7fab48153220 2024-03-09T08:55:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.858+0000 7fab3fe00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fab2406b4d0 0x7fab2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.858+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fab3008a490 con 0x7fab48153220 2024-03-09T08:55:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.858+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38012590 con 0x7fab4814f710 2024-03-09T08:55:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.858+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fab38018070 con 0x7fab4814f710 2024-03-09T08:55:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.858+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab4814f710 msgr2=0x7fab4814faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.858+0000 7fab3fe00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab4814f710 0x7fab4814faf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fab4814d3d0 tx=0x7fab38001970 comp rx=0 tx=0).stop 2024-03-09T08:55:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.858+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab4814f710 msgr2=0x7fab4814faf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.858+0000 7fab3fe00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab4814f710 0x7fab4814faf0 secure :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fab4814d3d0 tx=0x7fab38001970 comp rx=0 tx=0).stop 2024-03-09T08:55:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.858+0000 7fab4ce00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fab2406b4d0 0x7fab2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.859+0000 7fab4ce00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fab2406b4d0 0x7fab2406d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab40036000 tx=0x7fab40030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.866+0000 7fab4d800640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab481eb3c0 0x7fab481eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.866+0000 7fab4d800640 1 -- 172.21.15.67:0/3153460637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480c93b0 con 0x7fab481eb3c0 2024-03-09T08:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.866+0000 7fab47e00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab481eb3c0 0x7fab481eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:51.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.866+0000 7fab47e00640 1 -- 172.21.15.67:0/3153460637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fab48073a30 con 0x7fab481eb3c0 2024-03-09T08:55:51.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.867+0000 7fab47e00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab481eb3c0 0x7fab481eb7d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fab38005fd0 tx=0x7fab38001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:51.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.867+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38012a80 con 0x7fab481eb3c0 2024-03-09T08:55:51.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7fab38012a80 con 0x7fab481eb3c0 2024-03-09T08:55:51.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab481eb3c0 msgr2=0x7fab481eb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab3fe00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab481eb3c0 0x7fab481eb7d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fab38005fd0 tx=0x7fab38001080 comp rx=0 tx=0).stop 2024-03-09T08:55:51.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab3fe00640 1 -- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab481eb3c0 msgr2=0x7fab481eb7d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:51.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab3fe00640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab481eb3c0 0x7fab481eb7d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 28, 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:55:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:51.875 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:55:51.876 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:51.877 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:55:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:51.880 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:51.880 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:51.880 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:51.880 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:51.880 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:51.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:51.880 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:55:51.880 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:55:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab4d800640 1 -- 172.21.15.67:0/3153460637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fab2406b4d0 msgr2=0x7fab2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab4d800640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fab2406b4d0 0x7fab2406d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fab40036000 tx=0x7fab40030000 comp rx=0 tx=0).stop 2024-03-09T08:55:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab4d800640 1 -- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab48153220 msgr2=0x7fab480c1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab4d800640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab48153220 0x7fab480c1950 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fab3000e9b0 tx=0x7fab3000ee70 comp rx=0 tx=0).stop 2024-03-09T08:55:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab4ce00640 1 -- 172.21.15.67:0/3153460637 reap_dead start 2024-03-09T08:55:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab4d800640 1 -- 172.21.15.67:0/3153460637 shutdown_connections 2024-03-09T08:55:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab4d800640 1 --2- 172.21.15.67:0/3153460637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab48153220 0x7fab480c1950 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab4d800640 1 -- 172.21.15.67:0/3153460637 >> 172.21.15.67:0/3153460637 conn(0x7fab480ad060 msgr2=0x7fab4806c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab4d800640 1 -- 172.21.15.67:0/3153460637 shutdown_connections 2024-03-09T08:55:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:51.868+0000 7fab4d800640 1 -- 172.21.15.67:0/3153460637 wait complete. 2024-03-09T08:55:51.889 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:55:51.889 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:55:51.889 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:55:52.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.092+0000 7fb365400640 1 Processor -- start 2024-03-09T08:55:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.092+0000 7fb365400640 1 -- start start 2024-03-09T08:55:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.094+0000 7fb365400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb36014fd90 0x7fb360156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.094+0000 7fb365400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb36006c1d0 0x7fb36006c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.094+0000 7fb365400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb360156bc0 con 0x7fb3600c9380 2024-03-09T08:55:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.094+0000 7fb365400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36006ba60 con 0x7fb36006c1d0 2024-03-09T08:55:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.094+0000 7fb365400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3600c9890 con 0x7fb36014fd90 2024-03-09T08:55:52.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.094+0000 7fb35f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb3600c9380 0x7fb3600acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46470/0 (socket says 172.21.15.67:46470) 2024-03-09T08:55:52.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.094+0000 7fb35f400640 1 -- 172.21.15.67:0/827749668 learned_addr learned my addr 172.21.15.67:0/827749668 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.094+0000 7fb35fe00640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb36014fd90 0x7fb360156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.094+0000 7fb364a00640 1 --2- 172.21.15.67:0/827749668 >> v2:172.21.15.134:3300/0 conn(0x7fb36006c1d0 0x7fb36006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155116278 0 0) 0x7fb348001110 con 0x7fb3600c9380 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb344003520 con 0x7fb3600c9380 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2431777379 0 0) 0x7fb344003520 con 0x7fb3600c9380 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb36014fd90 msgr2=0x7fb360156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb36014fd90 0x7fb360156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 >> v2:172.21.15.134:3300/0 conn(0x7fb36006c1d0 msgr2=0x7fb36006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 --2- 172.21.15.67:0/827749668 >> v2:172.21.15.134:3300/0 conn(0x7fb36006c1d0 0x7fb36006c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3600c5f80 con 0x7fb3600c9380 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb348001840 con 0x7fb3600c9380 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 >> v1:172.21.15.67:6789/0 conn(0x7fb3600c9380 legacy=0x7fb3600acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb36014fd90 0x7fb344003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb344003ff0 0x7fb344004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb344004d40 0x7fb3440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36006ba60 con 0x7fb344004d40 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3600c9890 con 0x7fb344003ff0 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.095+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34400bb80 con 0x7fb36014fd90 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.096+0000 7fb364a00640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb344004d40 0x7fb3440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.096+0000 7fb35fe00640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb36014fd90 0x7fb344003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.096+0000 7fb35f400640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb344003ff0 0x7fb344004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.096+0000 7fb364a00640 1 -- 172.21.15.67:0/827749668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb36014fd90 msgr2=0x7fb344003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.096+0000 7fb364a00640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb36014fd90 0x7fb344003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.096+0000 7fb364a00640 1 -- 172.21.15.67:0/827749668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb344003ff0 msgr2=0x7fb344004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.096+0000 7fb364a00640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb344003ff0 0x7fb344004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.096+0000 7fb364a00640 1 -- 172.21.15.67:0/827749668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb344003520 con 0x7fb344004d40 2024-03-09T08:55:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.097+0000 7fb364a00640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb344004d40 0x7fb3440051c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb358009ac0 tx=0x7fb344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea4a811bd9d507b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.098+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358039070 con 0x7fb344004d40 2024-03-09T08:55:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.098+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35802c710 con 0x7fb344004d40 2024-03-09T08:55:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.098+0000 7fb35ea00640 1 -- 172.21.15.67:0/827749668 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358009690 con 0x7fb344004d40 2024-03-09T08:55:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.098+0000 7fb365400640 1 -- 172.21.15.67:0/827749668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb344004d40 msgr2=0x7fb3440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.098+0000 7fb365400640 1 --2- 172.21.15.67:0/827749668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb344004d40 0x7fb3440051c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb358009ac0 tx=0x7fb344005830 comp rx=0 tx=0).stop 2024-03-09T08:55:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.098+0000 7fb364a00640 1 -- 172.21.15.67:0/827749668 reap_dead start 2024-03-09T08:55:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.098+0000 7fb365400640 1 -- 172.21.15.67:0/827749668 shutdown_connections 2024-03-09T08:55:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.098+0000 7fb365400640 1 -- 172.21.15.67:0/827749668 >> 172.21.15.67:0/827749668 conn(0x7fb3600cb480 msgr2=0x7fb3600cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.098+0000 7fb365400640 1 -- 172.21.15.67:0/827749668 shutdown_connections 2024-03-09T08:55:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.098+0000 7fb365400640 1 -- 172.21.15.67:0/827749668 wait complete. 2024-03-09T08:55:52.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.100+0000 7fb365400640 1 Processor -- start 2024-03-09T08:55:52.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.100+0000 7fb365400640 1 -- start start 2024-03-09T08:55:52.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.101+0000 7fb365400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb344004d40 0x7fb360153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.101+0000 7fb365400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3601ef540 0x7fb3600af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.101+0000 7fb365400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af5b0 0x7fb360158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.101+0000 7fb365400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36006bcb0 con 0x7fb3601ef540 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.101+0000 7fb365400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3600c6520 con 0x7fb3600af5b0 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.101+0000 7fb365400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3600ad610 con 0x7fb344004d40 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.101+0000 7fb35f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3601ef540 0x7fb3600af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.102+0000 7fb35f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3601ef540 0x7fb3600af070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56384/0 (socket says 172.21.15.67:56384) 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.102+0000 7fb35f400640 1 -- 172.21.15.67:0/2308031989 learned_addr learned my addr 172.21.15.67:0/2308031989 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.102+0000 7fb35fe00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb344004d40 0x7fb360153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.102+0000 7fb364a00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af5b0 0x7fb360158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.102+0000 7fb35f400640 1 -- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb344004d40 msgr2=0x7fb360153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.102+0000 7fb35f400640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb344004d40 0x7fb360153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.102+0000 7fb35f400640 1 -- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af5b0 msgr2=0x7fb360158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.102+0000 7fb35f400640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af5b0 0x7fb360158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.102+0000 7fb35f400640 1 -- 172.21.15.67:0/2308031989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3601581f0 con 0x7fb3601ef540 2024-03-09T08:55:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.102+0000 7fb364a00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af5b0 0x7fb360158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:55:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.102+0000 7fb35f400640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3601ef540 0x7fb3600af070 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb34800d7a0 tx=0x7fb34800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.103+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348019070 con 0x7fb3601ef540 2024-03-09T08:55:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.103+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3480072d0 con 0x7fb3601ef540 2024-03-09T08:55:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.103+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3480047e0 con 0x7fb3601ef540 2024-03-09T08:55:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.103+0000 7fb365400640 1 -- 172.21.15.67:0/2308031989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3601ebfd0 con 0x7fb3601ef540 2024-03-09T08:55:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.103+0000 7fb365400640 1 -- 172.21.15.67:0/2308031989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3601ec570 con 0x7fb3601ef540 2024-03-09T08:55:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.104+0000 7fb365400640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af990 0x7fb3601ec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.104+0000 7fb365400640 1 -- 172.21.15.67:0/2308031989 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb360156da0 con 0x7fb3600af990 2024-03-09T08:55:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.105+0000 7fb35fe00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af990 0x7fb3601ec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.105+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb34801c070 con 0x7fb3601ef540 2024-03-09T08:55:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.105+0000 7fb35fe00640 1 -- 172.21.15.67:0/2308031989 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb36004f3c0 con 0x7fb3600af990 2024-03-09T08:55:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.106+0000 7fb35fe00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af990 0x7fb3601ec7b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb350007f00 tx=0x7fb350012610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.106+0000 7fb357e00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb33c06b500 0x7fb33c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.106+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fb348089f30 con 0x7fb3601ef540 2024-03-09T08:55:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.106+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35001c020 con 0x7fb3600af990 2024-03-09T08:55:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.106+0000 7fb364a00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb33c06b500 0x7fb33c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.107+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb350018670 con 0x7fb3600af990 2024-03-09T08:55:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.107+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af990 msgr2=0x7fb3601ec7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.107+0000 7fb357e00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af990 0x7fb3601ec7b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb350007f00 tx=0x7fb350012610 comp rx=0 tx=0).stop 2024-03-09T08:55:52.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.107+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af990 msgr2=0x7fb3601ec7b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:52.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.107+0000 7fb357e00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3600af990 0x7fb3601ec7b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.108+0000 7fb364a00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb33c06b500 0x7fb33c06d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb358036000 tx=0x7fb358030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:52.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.114+0000 7fb365400640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3601eccf0 0x7fb3601594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.114+0000 7fb365400640 1 -- 172.21.15.67:0/2308031989 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36006d2d0 con 0x7fb3601eccf0 2024-03-09T08:55:52.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.115+0000 7fb35fe00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3601eccf0 0x7fb3601594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.115+0000 7fb35fe00640 1 -- 172.21.15.67:0/2308031989 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb3600ca070 con 0x7fb3601eccf0 2024-03-09T08:55:52.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.116+0000 7fb35fe00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3601eccf0 0x7fb3601594e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb350007f00 tx=0x7fb350004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:52.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.116+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35001c030 con 0x7fb3601eccf0 2024-03-09T08:55:52.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.117+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fb350023860 con 0x7fb3601eccf0 2024-03-09T08:55:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.117+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3601eccf0 msgr2=0x7fb3601594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.117+0000 7fb357e00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3601eccf0 0x7fb3601594e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb350007f00 tx=0x7fb350004ab0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.117+0000 7fb357e00640 1 -- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3601eccf0 msgr2=0x7fb3601594e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.117+0000 7fb357e00640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3601eccf0 0x7fb3601594e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.116 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:55:52.116 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:52.116 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:52.116 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:55:52.116 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 28, 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:52.117 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:52.118 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:55:52.119 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:52.120 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.121 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.122 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:52.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:55:52.124 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:52.127 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.128 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.128 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:55:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:55:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.118+0000 7fb365400640 1 -- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb33c06b500 msgr2=0x7fb33c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.118+0000 7fb365400640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb33c06b500 0x7fb33c06d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb358036000 tx=0x7fb358030000 comp rx=0 tx=0).stop 2024-03-09T08:55:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.118+0000 7fb365400640 1 -- 172.21.15.67:0/2308031989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3601ef540 msgr2=0x7fb3600af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.118+0000 7fb365400640 1 --2- 172.21.15.67:0/2308031989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3601ef540 0x7fb3600af070 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb34800d7a0 tx=0x7fb34800dc60 comp rx=0 tx=0).stop 2024-03-09T08:55:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.118+0000 7fb364a00640 1 -- 172.21.15.67:0/2308031989 reap_dead start 2024-03-09T08:55:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.118+0000 7fb364a00640 1 -- 172.21.15.67:0/2308031989 reap_dead start 2024-03-09T08:55:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.118+0000 7fb365400640 1 -- 172.21.15.67:0/2308031989 shutdown_connections 2024-03-09T08:55:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.118+0000 7fb365400640 1 -- 172.21.15.67:0/2308031989 >> 172.21.15.67:0/2308031989 conn(0x7fb3600cb480 msgr2=0x7fb36006c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.118+0000 7fb365400640 1 -- 172.21.15.67:0/2308031989 shutdown_connections 2024-03-09T08:55:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.118+0000 7fb365400640 1 -- 172.21.15.67:0/2308031989 wait complete. 2024-03-09T08:55:52.134 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:55:52.134 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:55:52.135 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:55:52.224 INFO:tasks.workunit.client.0.smithi067.stdout: create 293/30 file after the snapshot 2024-03-09T08:55:52.227 INFO:tasks.workunit.client.0.smithi067.stdout: create 293/31 file after the snapshot 2024-03-09T08:55:52.228 INFO:tasks.workunit.client.0.smithi067.stdout: create 293/32 file after the snapshot 2024-03-09T08:55:52.228 INFO:tasks.workunit.client.0.smithi067.stdout: create 293/33 file after the snapshot 2024-03-09T08:55:52.228 INFO:tasks.workunit.client.0.smithi067.stdout: create 293/34 file after the snapshot 2024-03-09T08:55:52.228 INFO:tasks.workunit.client.0.smithi067.stdout: create 293/35 file after the snapshot 2024-03-09T08:55:52.228 INFO:tasks.workunit.client.0.smithi067.stdout: create 293/36 file after the snapshot 2024-03-09T08:55:52.228 INFO:tasks.workunit.client.0.smithi067.stdout: create 293/37 file after the snapshot 2024-03-09T08:55:52.229 INFO:tasks.workunit.client.0.smithi067.stdout: create 293/38 file after the snapshot 2024-03-09T08:55:52.229 INFO:tasks.workunit.client.0.smithi067.stdout: create 293/39 file after the snapshot 2024-03-09T08:55:52.229 INFO:tasks.workunit.client.0.smithi067.stdout: create 294/.snap/snap-subdir-test 2024-03-09T08:55:52.294 INFO:tasks.workunit.client.0.smithi067.stdout: create 294/30 file after the snapshot 2024-03-09T08:55:52.296 INFO:tasks.workunit.client.0.smithi067.stdout: create 294/31 file after the snapshot 2024-03-09T08:55:52.296 INFO:tasks.workunit.client.0.smithi067.stdout: create 294/32 file after the snapshot 2024-03-09T08:55:52.297 INFO:tasks.workunit.client.0.smithi067.stdout: create 294/33 file after the snapshot 2024-03-09T08:55:52.297 INFO:tasks.workunit.client.0.smithi067.stdout: create 294/34 file after the snapshot 2024-03-09T08:55:52.297 INFO:tasks.workunit.client.0.smithi067.stdout: create 294/35 file after the snapshot 2024-03-09T08:55:52.298 INFO:tasks.workunit.client.0.smithi067.stdout: create 294/36 file after the snapshot 2024-03-09T08:55:52.298 INFO:tasks.workunit.client.0.smithi067.stdout: create 294/37 file after the snapshot 2024-03-09T08:55:52.298 INFO:tasks.workunit.client.0.smithi067.stdout: create 294/38 file after the snapshot 2024-03-09T08:55:52.299 INFO:tasks.workunit.client.0.smithi067.stdout: create 294/39 file after the snapshot 2024-03-09T08:55:52.299 INFO:tasks.workunit.client.0.smithi067.stdout: create 295/.snap/snap-subdir-test 2024-03-09T08:55:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.347+0000 7fc4ea600640 1 Processor -- start 2024-03-09T08:55:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.347+0000 7fc4ea600640 1 -- start start 2024-03-09T08:55:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.349+0000 7fc4ea600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4e4153560 0x7fc4e406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.349+0000 7fc4ea600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc4e40c5f50 0x7fc4e40c6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.349+0000 7fc4ea600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e406cf30 con 0x7fc4e40c96f0 2024-03-09T08:55:52.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.349+0000 7fc4ea600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e4159d80 con 0x7fc4e4153560 2024-03-09T08:55:52.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.349+0000 7fc4ea600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e415a060 con 0x7fc4e40c5f50 2024-03-09T08:55:52.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.349+0000 7fc4e3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4e4153560 0x7fc4e406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.349+0000 7fc4e3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4e4153560 0x7fc4e406b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56400/0 (socket says 172.21.15.67:56400) 2024-03-09T08:55:52.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.349+0000 7fc4e3400640 1 -- 172.21.15.67:0/3659485610 learned_addr learned my addr 172.21.15.67:0/3659485610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:52.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.349+0000 7fc4e3e00640 1 --2- 172.21.15.67:0/3659485610 >> v2:172.21.15.134:3300/0 conn(0x7fc4e40c5f50 0x7fc4e40c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.350+0000 7fc4e3400640 1 -- 172.21.15.67:0/3659485610 >> v2:172.21.15.134:3300/0 conn(0x7fc4e40c5f50 msgr2=0x7fc4e40c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.350+0000 7fc4e3400640 1 --2- 172.21.15.67:0/3659485610 >> v2:172.21.15.134:3300/0 conn(0x7fc4e40c5f50 0x7fc4e40c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.350+0000 7fc4e3400640 1 -- 172.21.15.67:0/3659485610 >> v1:172.21.15.134:6789/0 conn(0x7fc4e40c96f0 legacy=0x7fc4e40cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:55:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.350+0000 7fc4e3400640 1 -- 172.21.15.67:0/3659485610 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e40ca440 con 0x7fc4e4153560 2024-03-09T08:55:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.351+0000 7fc4e3400640 1 --2- 172.21.15.67:0/3659485610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4e4153560 0x7fc4e406b040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4073c90 tx=0x7fc4d402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88acb05e602dae8e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.351+0000 7fc4e2000640 1 -- 172.21.15.67:0/3659485610 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d402e070 con 0x7fc4e4153560 2024-03-09T08:55:52.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.351+0000 7fc4e2000640 1 -- 172.21.15.67:0/3659485610 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d4037070 con 0x7fc4e4153560 2024-03-09T08:55:52.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.351+0000 7fc4e2000640 1 -- 172.21.15.67:0/3659485610 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4031a00 con 0x7fc4e4153560 2024-03-09T08:55:52.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.352+0000 7fc4ea600640 1 -- 172.21.15.67:0/3659485610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4e4153560 msgr2=0x7fc4e406b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.352+0000 7fc4ea600640 1 --2- 172.21.15.67:0/3659485610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4e4153560 0x7fc4e406b040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4073c90 tx=0x7fc4d402b880 comp rx=0 tx=0).stop 2024-03-09T08:55:52.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.352+0000 7fc4ea600640 1 -- 172.21.15.67:0/3659485610 shutdown_connections 2024-03-09T08:55:52.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.352+0000 7fc4ea600640 1 --2- 172.21.15.67:0/3659485610 >> v2:172.21.15.134:3300/0 conn(0x7fc4e40c5f50 0x7fc4e40c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.352+0000 7fc4ea600640 1 --2- 172.21.15.67:0/3659485610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4e4153560 0x7fc4e406b040 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.352+0000 7fc4ea600640 1 -- 172.21.15.67:0/3659485610 >> 172.21.15.67:0/3659485610 conn(0x7fc4e40c8050 msgr2=0x7fc4e40c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:52.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.352+0000 7fc4ea600640 1 -- 172.21.15.67:0/3659485610 shutdown_connections 2024-03-09T08:55:52.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.352+0000 7fc4ea600640 1 -- 172.21.15.67:0/3659485610 wait complete. 2024-03-09T08:55:52.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.354+0000 7fc4ea600640 1 Processor -- start 2024-03-09T08:55:52.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.354+0000 7fc4ea600640 1 -- start start 2024-03-09T08:55:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.355+0000 7fc4ea600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e40c5f50 0x7fc4e40ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.355+0000 7fc4ea600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e40c96f0 0x7fc4e40b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.355+0000 7fc4ea600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4e4153560 0x7fc4e40b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.355+0000 7fc4ea600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40c68a0 con 0x7fc4e4153560 2024-03-09T08:55:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.355+0000 7fc4ea600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40c9ff0 con 0x7fc4e40c5f50 2024-03-09T08:55:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.355+0000 7fc4ea600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40c6a20 con 0x7fc4e40c96f0 2024-03-09T08:55:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.355+0000 7fc4e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4e4153560 0x7fc4e40b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.355+0000 7fc4e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4e4153560 0x7fc4e40b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56402/0 (socket says 172.21.15.67:56402) 2024-03-09T08:55:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.355+0000 7fc4e3e00640 1 -- 172.21.15.67:0/2411410374 learned_addr learned my addr 172.21.15.67:0/2411410374 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:55:52.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.355+0000 7fc4e3400640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e40c5f50 0x7fc4e40ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.355+0000 7fc4e2a00640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e40c96f0 0x7fc4e40b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.356+0000 7fc4e3e00640 1 -- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e40c96f0 msgr2=0x7fc4e40b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.356+0000 7fc4e3e00640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e40c96f0 0x7fc4e40b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.356+0000 7fc4e3e00640 1 -- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e40c5f50 msgr2=0x7fc4e40ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.356+0000 7fc4e3e00640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4e40c5f50 0x7fc4e40ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.356+0000 7fc4e3e00640 1 -- 172.21.15.67:0/2411410374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e414bc60 con 0x7fc4e4153560 2024-03-09T08:55:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.356+0000 7fc4e3e00640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4e4153560 0x7fc4e40b0d20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc007e40 tx=0x7fc4dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.356+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc014620 con 0x7fc4e4153560 2024-03-09T08:55:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.356+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc019070 con 0x7fc4e4153560 2024-03-09T08:55:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.356+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc014a20 con 0x7fc4e4153560 2024-03-09T08:55:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.356+0000 7fc4ea600640 1 -- 172.21.15.67:0/2411410374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e414c220 con 0x7fc4e4153560 2024-03-09T08:55:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.357+0000 7fc4ea600640 1 -- 172.21.15.67:0/2411410374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e4149f20 con 0x7fc4e4153560 2024-03-09T08:55:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.358+0000 7fc4ea600640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c3f0 0x7fc4e41efb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.358+0000 7fc4ea600640 1 -- 172.21.15.67:0/2411410374 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e414c080 con 0x7fc4e414c3f0 2024-03-09T08:55:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.358+0000 7fc4e3400640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c3f0 0x7fc4e41efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.359+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc4dc014bc0 con 0x7fc4e4153560 2024-03-09T08:55:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.359+0000 7fc4e3400640 1 -- 172.21.15.67:0/2411410374 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4e41597d0 con 0x7fc4e414c3f0 2024-03-09T08:55:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.360+0000 7fc4e3400640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c3f0 0x7fc4e41efb50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc4e408d920 tx=0x7fc4d4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.360+0000 7fc4db400640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc4c406b4d0 0x7fc4c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.360+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fc4dc08afa0 con 0x7fc4e4153560 2024-03-09T08:55:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.360+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d404a070 con 0x7fc4e414c3f0 2024-03-09T08:55:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.361+0000 7fc4e2a00640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc4c406b4d0 0x7fc4c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.361+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc4d4037040 con 0x7fc4e414c3f0 2024-03-09T08:55:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.361+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c3f0 msgr2=0x7fc4e41efb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.361+0000 7fc4db400640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c3f0 0x7fc4e41efb50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc4e408d920 tx=0x7fc4d4004730 comp rx=0 tx=0).stop 2024-03-09T08:55:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.361+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c3f0 msgr2=0x7fc4e41efb50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.361+0000 7fc4db400640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c3f0 0x7fc4e41efb50 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc4e408d920 tx=0x7fc4d4004730 comp rx=0 tx=0).stop 2024-03-09T08:55:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.362+0000 7fc4e2a00640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc4c406b4d0 0x7fc4c406d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0007fd0 tx=0x7fc4d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:52.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.369+0000 7fc4ea600640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c8b0 0x7fc4e41f0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:55:52.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.369+0000 7fc4ea600640 1 -- 172.21.15.67:0/2411410374 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e4075a00 con 0x7fc4e414c8b0 2024-03-09T08:55:52.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.369+0000 7fc4e3400640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c8b0 0x7fc4e41f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:55:52.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.370+0000 7fc4e3400640 1 -- 172.21.15.67:0/2411410374 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc4e414be30 con 0x7fc4e414c8b0 2024-03-09T08:55:52.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.370+0000 7fc4e3400640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c8b0 0x7fc4e41f0090 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4000c00 tx=0x7fc4d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:55:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.371+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d404a040 con 0x7fc4e414c8b0 2024-03-09T08:55:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.371+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4874 (secure 0 0 0) 0x7fc4d4045410 con 0x7fc4e414c8b0 2024-03-09T08:55:52.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.371+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c8b0 msgr2=0x7fc4e41f0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.371+0000 7fc4db400640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c8b0 0x7fc4e41f0090 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4000c00 tx=0x7fc4d40046c0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.371+0000 7fc4db400640 1 -- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c8b0 msgr2=0x7fc4e41f0090 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:55:52.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.371+0000 7fc4db400640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4e414c8b0 0x7fc4e41f0090 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.370 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:55:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:55:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 28, 2024-03-09T08:55:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:55:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:55:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:55:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:55:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.376 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:55:52.377 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:52.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:55:52.379 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:55:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:55:52.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:55:52.382 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:55:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.372+0000 7fc4ea600640 1 -- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc4c406b4d0 msgr2=0x7fc4c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.372+0000 7fc4ea600640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc4c406b4d0 0x7fc4c406d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0007fd0 tx=0x7fc4d0009040 comp rx=0 tx=0).stop 2024-03-09T08:55:52.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.372+0000 7fc4ea600640 1 -- 172.21.15.67:0/2411410374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4e4153560 msgr2=0x7fc4e40b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:55:52.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.372+0000 7fc4ea600640 1 --2- 172.21.15.67:0/2411410374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4e4153560 0x7fc4e40b0d20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc007e40 tx=0x7fc4dc00e4b0 comp rx=0 tx=0).stop 2024-03-09T08:55:52.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.372+0000 7fc4e3e00640 1 -- 172.21.15.67:0/2411410374 reap_dead start 2024-03-09T08:55:52.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.372+0000 7fc4e3e00640 1 -- 172.21.15.67:0/2411410374 reap_dead start 2024-03-09T08:55:52.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.372+0000 7fc4ea600640 1 -- 172.21.15.67:0/2411410374 shutdown_connections 2024-03-09T08:55:52.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.372+0000 7fc4ea600640 1 -- 172.21.15.67:0/2411410374 >> 172.21.15.67:0/2411410374 conn(0x7fc4e40c8050 msgr2=0x7fc4e4152490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:55:52.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.372+0000 7fc4ea600640 1 -- 172.21.15.67:0/2411410374 shutdown_connections 2024-03-09T08:55:52.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:55:52.372+0000 7fc4ea600640 1 -- 172.21.15.67:0/2411410374 wait complete. 2024-03-09T08:55:52.385 INFO:tasks.workunit.client.0.smithi067.stdout: create 295/30 file after the snapshot 2024-03-09T08:55:52.385 INFO:tasks.workunit.client.0.smithi067.stdout: create 295/31 file after the snapshot 2024-03-09T08:55:52.385 INFO:tasks.workunit.client.0.smithi067.stdout: create 295/32 file after the snapshot 2024-03-09T08:55:52.385 INFO:tasks.workunit.client.0.smithi067.stdout: create 295/33 file after the snapshot 2024-03-09T08:55:52.385 INFO:tasks.workunit.client.0.smithi067.stdout: create 295/34 file after the snapshot 2024-03-09T08:55:52.385 INFO:tasks.workunit.client.0.smithi067.stdout: create 295/35 file after the snapshot 2024-03-09T08:55:52.386 INFO:tasks.workunit.client.0.smithi067.stdout: create 295/36 file after the snapshot 2024-03-09T08:55:52.386 INFO:tasks.workunit.client.0.smithi067.stdout: create 295/37 file after the snapshot 2024-03-09T08:55:52.386 INFO:tasks.workunit.client.0.smithi067.stdout: create 295/38 file after the snapshot 2024-03-09T08:55:52.386 INFO:tasks.workunit.client.0.smithi067.stdout: create 295/39 file after the snapshot 2024-03-09T08:55:52.386 INFO:tasks.workunit.client.0.smithi067.stdout: create 296/.snap/snap-subdir-test 2024-03-09T08:55:52.388 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:55:52.388 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T08:55:52.388 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a'] 2024-03-09T08:55:52.389 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T08:55:52.389 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T08:55:52.389 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T08:55:52.389 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T08:55:52.415 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T08:55:52.415 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-03-09T08:55:52.415 INFO:tasks.ceph.mon.a:Sent signal 19 2024-03-09T08:55:52.416 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-03-09T08:55:52.461 INFO:tasks.workunit.client.0.smithi067.stdout: create 296/30 file after the snapshot 2024-03-09T08:55:52.464 INFO:tasks.workunit.client.0.smithi067.stdout: create 296/31 file after the snapshot 2024-03-09T08:55:52.464 INFO:tasks.workunit.client.0.smithi067.stdout: create 296/32 file after the snapshot 2024-03-09T08:55:52.464 INFO:tasks.workunit.client.0.smithi067.stdout: create 296/33 file after the snapshot 2024-03-09T08:55:52.465 INFO:tasks.workunit.client.0.smithi067.stdout: create 296/34 file after the snapshot 2024-03-09T08:55:52.465 INFO:tasks.workunit.client.0.smithi067.stdout: create 296/35 file after the snapshot 2024-03-09T08:55:52.465 INFO:tasks.workunit.client.0.smithi067.stdout: create 296/36 file after the snapshot 2024-03-09T08:55:52.465 INFO:tasks.workunit.client.0.smithi067.stdout: create 296/37 file after the snapshot 2024-03-09T08:55:52.466 INFO:tasks.workunit.client.0.smithi067.stdout: create 296/38 file after the snapshot 2024-03-09T08:55:52.466 INFO:tasks.workunit.client.0.smithi067.stdout: create 296/39 file after the snapshot 2024-03-09T08:55:52.466 INFO:tasks.workunit.client.0.smithi067.stdout: create 297/.snap/snap-subdir-test 2024-03-09T08:55:53.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2024-03-09T08:55:53.329 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2024-03-09T08:55:53.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2024-03-09T08:55:53.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2024-03-09T08:55:53.332 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2024-03-09T08:55:53.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2024-03-09T08:55:53.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2024-03-09T08:55:53.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2024-03-09T08:55:53.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 297/30 file after the snapshot 2024-03-09T08:55:53.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 297/31 file after the snapshot 2024-03-09T08:55:53.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 297/32 file after the snapshot 2024-03-09T08:55:53.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 297/33 file after the snapshot 2024-03-09T08:55:53.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 297/34 file after the snapshot 2024-03-09T08:55:53.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 297/35 file after the snapshot 2024-03-09T08:55:53.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 297/36 file after the snapshot 2024-03-09T08:55:53.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 297/37 file after the snapshot 2024-03-09T08:55:53.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 297/38 file after the snapshot 2024-03-09T08:55:53.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 297/39 file after the snapshot 2024-03-09T08:55:53.337 INFO:tasks.workunit.client.0.smithi067.stdout: create 298/.snap/snap-subdir-test 2024-03-09T08:55:53.348 INFO:tasks.workunit.client.0.smithi067.stdout: create 298/30 file after the snapshot 2024-03-09T08:55:53.352 INFO:tasks.workunit.client.0.smithi067.stdout: create 298/31 file after the snapshot 2024-03-09T08:55:53.352 INFO:tasks.workunit.client.0.smithi067.stdout: create 298/32 file after the snapshot 2024-03-09T08:55:53.352 INFO:tasks.workunit.client.0.smithi067.stdout: create 298/33 file after the snapshot 2024-03-09T08:55:53.352 INFO:tasks.workunit.client.0.smithi067.stdout: create 298/34 file after the snapshot 2024-03-09T08:55:53.352 INFO:tasks.workunit.client.0.smithi067.stdout: create 298/35 file after the snapshot 2024-03-09T08:55:53.352 INFO:tasks.workunit.client.0.smithi067.stdout: create 298/36 file after the snapshot 2024-03-09T08:55:53.352 INFO:tasks.workunit.client.0.smithi067.stdout: create 298/37 file after the snapshot 2024-03-09T08:55:53.352 INFO:tasks.workunit.client.0.smithi067.stdout: create 298/38 file after the snapshot 2024-03-09T08:55:53.352 INFO:tasks.workunit.client.0.smithi067.stdout: create 298/39 file after the snapshot 2024-03-09T08:55:53.352 INFO:tasks.workunit.client.0.smithi067.stdout: create 299/.snap/snap-subdir-test 2024-03-09T08:55:53.717 INFO:tasks.workunit.client.0.smithi067.stdout: create 299/30 file after the snapshot 2024-03-09T08:55:53.718 INFO:tasks.workunit.client.0.smithi067.stdout: create 299/31 file after the snapshot 2024-03-09T08:55:53.718 INFO:tasks.workunit.client.0.smithi067.stdout: create 299/32 file after the snapshot 2024-03-09T08:55:53.719 INFO:tasks.workunit.client.0.smithi067.stdout: create 299/33 file after the snapshot 2024-03-09T08:55:53.719 INFO:tasks.workunit.client.0.smithi067.stdout: create 299/34 file after the snapshot 2024-03-09T08:55:53.720 INFO:tasks.workunit.client.0.smithi067.stdout: create 299/35 file after the snapshot 2024-03-09T08:55:53.720 INFO:tasks.workunit.client.0.smithi067.stdout: create 299/36 file after the snapshot 2024-03-09T08:55:53.721 INFO:tasks.workunit.client.0.smithi067.stdout: create 299/37 file after the snapshot 2024-03-09T08:55:53.721 INFO:tasks.workunit.client.0.smithi067.stdout: create 299/38 file after the snapshot 2024-03-09T08:55:53.721 INFO:tasks.workunit.client.0.smithi067.stdout: create 299/39 file after the snapshot 2024-03-09T08:55:53.722 INFO:tasks.workunit.client.0.smithi067.stdout:Take second snapshot .snap/test2 2024-03-09T08:55:54.515 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2024-03-09T08:55:54.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2024-03-09T08:55:54.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2024-03-09T08:55:54.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2024-03-09T08:55:54.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2024-03-09T08:55:54.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2024-03-09T08:55:54.521 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2024-03-09T08:55:54.521 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2024-03-09T08:55:54.521 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2024-03-09T08:55:54.521 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2024-03-09T08:55:54.521 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2024-03-09T08:55:54.521 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2024-03-09T08:55:54.521 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2024-03-09T08:55:54.521 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2024-03-09T08:55:54.638 INFO:tasks.workunit.client.0.smithi067.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2024-03-09T08:55:56.238 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2024-03-09T08:55:56.238 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2024-03-09T08:55:56.238 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2024-03-09T08:55:56.238 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2024-03-09T08:55:56.238 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2024-03-09T08:55:56.238 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2024-03-09T08:55:56.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2024-03-09T08:55:56.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2024-03-09T08:55:56.241 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2024-03-09T08:55:56.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2024-03-09T08:55:56.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2024-03-09T08:55:56.244 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2024-03-09T08:55:56.244 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2024-03-09T08:55:56.244 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2024-03-09T08:55:56.244 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2024-03-09T08:55:56.244 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2024-03-09T08:55:56.244 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2024-03-09T08:55:56.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2024-03-09T08:55:56.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2024-03-09T08:55:56.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2024-03-09T08:55:56.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2024-03-09T08:55:56.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2024-03-09T08:55:56.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2024-03-09T08:55:56.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2024-03-09T08:55:56.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2024-03-09T08:55:56.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2024-03-09T08:55:56.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2024-03-09T08:55:56.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2024-03-09T08:55:56.465 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2024-03-09T08:55:56.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2024-03-09T08:55:56.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2024-03-09T08:55:57.076 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2024-03-09T08:55:57.076 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2024-03-09T08:55:57.076 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2024-03-09T08:55:57.076 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2024-03-09T08:55:57.076 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2024-03-09T08:55:57.076 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2024-03-09T08:55:57.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2024-03-09T08:55:57.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2024-03-09T08:55:57.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2024-03-09T08:55:57.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2024-03-09T08:55:57.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2024-03-09T08:55:57.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2024-03-09T08:55:57.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2024-03-09T08:55:57.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2024-03-09T08:55:57.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2024-03-09T08:55:57.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2024-03-09T08:55:57.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2024-03-09T08:55:57.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2024-03-09T08:55:57.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2024-03-09T08:55:57.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2024-03-09T08:55:57.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2024-03-09T08:55:57.082 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2024-03-09T08:55:57.082 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2024-03-09T08:55:57.082 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2024-03-09T08:55:57.082 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2024-03-09T08:55:57.082 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2024-03-09T08:55:57.082 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2024-03-09T08:55:57.082 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2024-03-09T08:55:57.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2024-03-09T08:55:57.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2024-03-09T08:55:57.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2024-03-09T08:55:57.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2024-03-09T08:55:57.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2024-03-09T08:55:57.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2024-03-09T08:55:57.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2024-03-09T08:55:57.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2024-03-09T08:55:57.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2024-03-09T08:55:57.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2024-03-09T08:55:57.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2024-03-09T08:55:57.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2024-03-09T08:55:57.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2024-03-09T08:55:57.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2024-03-09T08:55:57.508 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2024-03-09T08:55:57.508 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2024-03-09T08:55:57.508 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2024-03-09T08:55:57.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2024-03-09T08:55:57.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2024-03-09T08:55:57.811 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2024-03-09T08:55:57.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2024-03-09T08:55:57.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2024-03-09T08:55:57.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2024-03-09T08:55:59.393 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2024-03-09T08:55:59.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2024-03-09T08:55:59.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2024-03-09T08:55:59.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2024-03-09T08:55:59.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2024-03-09T08:55:59.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2024-03-09T08:55:59.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2024-03-09T08:55:59.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2024-03-09T08:55:59.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2024-03-09T08:55:59.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2024-03-09T08:55:59.395 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2024-03-09T08:55:59.395 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2024-03-09T08:55:59.395 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2024-03-09T08:55:59.395 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2024-03-09T08:55:59.395 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2024-03-09T08:55:59.395 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2024-03-09T08:55:59.395 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2024-03-09T08:55:59.395 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2024-03-09T08:55:59.396 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2024-03-09T08:55:59.396 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2024-03-09T08:55:59.396 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2024-03-09T08:55:59.396 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2024-03-09T08:55:59.396 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2024-03-09T08:55:59.396 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2024-03-09T08:55:59.397 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2024-03-09T08:55:59.397 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2024-03-09T08:55:59.397 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2024-03-09T08:55:59.397 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2024-03-09T08:55:59.397 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2024-03-09T08:55:59.397 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2024-03-09T08:55:59.397 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2024-03-09T08:55:59.397 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2024-03-09T08:55:59.397 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2024-03-09T08:55:59.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2024-03-09T08:55:59.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2024-03-09T08:55:59.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2024-03-09T08:55:59.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2024-03-09T08:55:59.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2024-03-09T08:55:59.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2024-03-09T08:55:59.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2024-03-09T08:55:59.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2024-03-09T08:55:59.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2024-03-09T08:55:59.399 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2024-03-09T08:55:59.399 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2024-03-09T08:55:59.399 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2024-03-09T08:55:59.399 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2024-03-09T08:55:59.399 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2024-03-09T08:55:59.399 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2024-03-09T08:55:59.399 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2024-03-09T08:55:59.399 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2024-03-09T08:55:59.400 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2024-03-09T08:55:59.400 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2024-03-09T08:55:59.400 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2024-03-09T08:55:59.400 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2024-03-09T08:55:59.400 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2024-03-09T08:55:59.400 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2024-03-09T08:55:59.400 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2024-03-09T08:55:59.400 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2024-03-09T08:55:59.401 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2024-03-09T08:55:59.401 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2024-03-09T08:55:59.401 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2024-03-09T08:55:59.401 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2024-03-09T08:55:59.401 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2024-03-09T08:55:59.401 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2024-03-09T08:55:59.401 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2024-03-09T08:55:59.401 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2024-03-09T08:55:59.401 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2024-03-09T08:55:59.401 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2024-03-09T08:55:59.401 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2024-03-09T08:55:59.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2024-03-09T08:56:00.984 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2024-03-09T08:56:00.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2024-03-09T08:56:00.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2024-03-09T08:56:00.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2024-03-09T08:56:00.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2024-03-09T08:56:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2024-03-09T08:56:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2024-03-09T08:56:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2024-03-09T08:56:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2024-03-09T08:56:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2024-03-09T08:56:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2024-03-09T08:56:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2024-03-09T08:56:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2024-03-09T08:56:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2024-03-09T08:56:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2024-03-09T08:56:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2024-03-09T08:56:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2024-03-09T08:56:01.256 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2024-03-09T08:56:01.256 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2024-03-09T08:56:01.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2024-03-09T08:56:01.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2024-03-09T08:56:01.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2024-03-09T08:56:01.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2024-03-09T08:56:01.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2024-03-09T08:56:01.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2024-03-09T08:56:01.570 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2024-03-09T08:56:01.570 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2024-03-09T08:56:01.570 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2024-03-09T08:56:01.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2024-03-09T08:56:01.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2024-03-09T08:56:01.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2024-03-09T08:56:01.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2024-03-09T08:56:01.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2024-03-09T08:56:01.576 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2024-03-09T08:56:01.576 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2024-03-09T08:56:01.576 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2024-03-09T08:56:01.576 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2024-03-09T08:56:01.576 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2024-03-09T08:56:01.576 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2024-03-09T08:56:02.152 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2024-03-09T08:56:02.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2024-03-09T08:56:02.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2024-03-09T08:56:02.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2024-03-09T08:56:02.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2024-03-09T08:56:02.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2024-03-09T08:56:02.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2024-03-09T08:56:02.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2024-03-09T08:56:02.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2024-03-09T08:56:02.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2024-03-09T08:56:02.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2024-03-09T08:56:02.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2024-03-09T08:56:02.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2024-03-09T08:56:02.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2024-03-09T08:56:02.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2024-03-09T08:56:02.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2024-03-09T08:56:02.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2024-03-09T08:56:02.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2024-03-09T08:56:02.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2024-03-09T08:56:02.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2024-03-09T08:56:02.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2024-03-09T08:56:02.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2024-03-09T08:56:02.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2024-03-09T08:56:02.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2024-03-09T08:56:02.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2024-03-09T08:56:02.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2024-03-09T08:56:02.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2024-03-09T08:56:02.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2024-03-09T08:56:02.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2024-03-09T08:56:02.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2024-03-09T08:56:02.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2024-03-09T08:56:02.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2024-03-09T08:56:02.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2024-03-09T08:56:02.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2024-03-09T08:56:02.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2024-03-09T08:56:02.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2024-03-09T08:56:02.158 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2024-03-09T08:56:02.158 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2024-03-09T08:56:02.158 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2024-03-09T08:56:02.158 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2024-03-09T08:56:02.158 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2024-03-09T08:56:02.158 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2024-03-09T08:56:02.158 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2024-03-09T08:56:02.158 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2024-03-09T08:56:02.158 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2024-03-09T08:56:02.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2024-03-09T08:56:02.160 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2024-03-09T08:56:02.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2024-03-09T08:56:02.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2024-03-09T08:56:02.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2024-03-09T08:56:02.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2024-03-09T08:56:02.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2024-03-09T08:56:02.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2024-03-09T08:56:02.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2024-03-09T08:56:02.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2024-03-09T08:56:02.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2024-03-09T08:56:02.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2024-03-09T08:56:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2024-03-09T08:56:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2024-03-09T08:56:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2024-03-09T08:56:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2024-03-09T08:56:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2024-03-09T08:56:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2024-03-09T08:56:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2024-03-09T08:56:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2024-03-09T08:56:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2024-03-09T08:56:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2024-03-09T08:56:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2024-03-09T08:56:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2024-03-09T08:56:02.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2024-03-09T08:56:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2024-03-09T08:56:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2024-03-09T08:56:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2024-03-09T08:56:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2024-03-09T08:56:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2024-03-09T08:56:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2024-03-09T08:56:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2024-03-09T08:56:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2024-03-09T08:56:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2024-03-09T08:56:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2024-03-09T08:56:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2024-03-09T08:56:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2024-03-09T08:56:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2024-03-09T08:56:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2024-03-09T08:56:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2024-03-09T08:56:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2024-03-09T08:56:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2024-03-09T08:56:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2024-03-09T08:56:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2024-03-09T08:56:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2024-03-09T08:56:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2024-03-09T08:56:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2024-03-09T08:56:03.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2024-03-09T08:56:03.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2024-03-09T08:56:03.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2024-03-09T08:56:03.321 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2024-03-09T08:56:03.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2024-03-09T08:56:03.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2024-03-09T08:56:03.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2024-03-09T08:56:03.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2024-03-09T08:56:03.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2024-03-09T08:56:03.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2024-03-09T08:56:03.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2024-03-09T08:56:03.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2024-03-09T08:56:03.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2024-03-09T08:56:03.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2024-03-09T08:56:03.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2024-03-09T08:56:03.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2024-03-09T08:56:03.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2024-03-09T08:56:03.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2024-03-09T08:56:03.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2024-03-09T08:56:03.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2024-03-09T08:56:04.869 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2024-03-09T08:56:04.869 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2024-03-09T08:56:04.869 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2024-03-09T08:56:04.869 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2024-03-09T08:56:04.869 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2024-03-09T08:56:04.869 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2024-03-09T08:56:04.869 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2024-03-09T08:56:04.870 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2024-03-09T08:56:04.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2024-03-09T08:56:04.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2024-03-09T08:56:04.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2024-03-09T08:56:04.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2024-03-09T08:56:04.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2024-03-09T08:56:04.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2024-03-09T08:56:04.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2024-03-09T08:56:04.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2024-03-09T08:56:04.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2024-03-09T08:56:04.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2024-03-09T08:56:04.872 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2024-03-09T08:56:04.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2024-03-09T08:56:04.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2024-03-09T08:56:04.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2024-03-09T08:56:04.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2024-03-09T08:56:04.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2024-03-09T08:56:04.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2024-03-09T08:56:04.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2024-03-09T08:56:04.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2024-03-09T08:56:04.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2024-03-09T08:56:04.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2024-03-09T08:56:04.874 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2024-03-09T08:56:04.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2024-03-09T08:56:04.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2024-03-09T08:56:04.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2024-03-09T08:56:04.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2024-03-09T08:56:04.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2024-03-09T08:56:04.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2024-03-09T08:56:04.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2024-03-09T08:56:04.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2024-03-09T08:56:04.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2024-03-09T08:56:04.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2024-03-09T08:56:04.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2024-03-09T08:56:07.416 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-03-09T08:56:07.417 INFO:tasks.ceph.mon.a:Sent signal 18 2024-03-09T08:56:07.417 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T08:56:07.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:56:07.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.679+0000 7f948fe00640 1 Processor -- start 2024-03-09T08:56:07.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.679+0000 7f948fe00640 1 -- start start 2024-03-09T08:56:07.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.681+0000 7f948fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94900c5e40 0x7f9490156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:07.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.681+0000 7f948fe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:07.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.681+0000 7f948fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9490156a90 con 0x7f9490153300 2024-03-09T08:56:07.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.681+0000 7f948fe00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9490157000 con 0x7f94900cadb0 2024-03-09T08:56:07.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.681+0000 7f948fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94900ae430 con 0x7f94900c5e40 2024-03-09T08:56:07.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.681+0000 7f948e000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94900c5e40 msgr2=0x7f9490156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:56:07.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.681+0000 7f948e000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94900c5e40 0x7f9490156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:56:07.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.681+0000 7f948ea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9490153300 0x7f94900b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55644/0 (socket says 172.21.15.67:55644) 2024-03-09T08:56:07.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.681+0000 7f948ea00640 1 -- 172.21.15.67:0/41447633 learned_addr learned my addr 172.21.15.67:0/41447633 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:56:07.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.681+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:07.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.682+0000 7f948f400640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 msgr2=0x7f9490156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:56:07.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.682+0000 7f948f400640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 msgr2=0x7f9490156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:56:07.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.682+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:56:07.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.682+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:56:07.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.882+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94900c5e40 msgr2=0x7f9490156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:56:07.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.882+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94900c5e40 0x7f9490156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:56:07.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.883+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:07.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.884+0000 7f948f400640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 msgr2=0x7f9490156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:56:07.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.884+0000 7f948f400640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 msgr2=0x7f9490156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:56:07.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.884+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:56:07.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:07.884+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:56:08.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:08.283+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94900c5e40 msgr2=0x7f9490156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:56:08.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:08.283+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94900c5e40 0x7f9490156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:56:08.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:08.285+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:08.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:08.286+0000 7f948f400640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 msgr2=0x7f9490156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:56:08.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:08.286+0000 7f948f400640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 msgr2=0x7f9490156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:56:08.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:08.286+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:56:08.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:08.286+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:56:09.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:09.084+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94900c5e40 msgr2=0x7f9490156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:56:09.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:09.084+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94900c5e40 0x7f9490156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:56:09.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:09.088+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:09.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:09.088+0000 7f948f400640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 msgr2=0x7f9490156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:56:09.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:09.088+0000 7f948f400640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 msgr2=0x7f9490156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:56:09.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:09.088+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:56:09.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:09.088+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:56:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.681+0000 7f9487e00640 1 -- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94900c5e40 msgr2=0x7f9490156010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:56:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.681+0000 7f9487e00640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94900c5e40 0x7f9490156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.681+0000 7f9487e00640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 msgr2=0x7f9490156550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:56:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.681+0000 7f9487e00640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.134:3300/0 conn(0x7f94900cadb0 0x7f9490156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.681+0000 7f9487e00640 1 -- 172.21.15.67:0/41447633 >> v1:172.21.15.67:6789/0 conn(0x7f9490153300 legacy=0x7f94900b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.681+0000 7f9487e00640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.681+0000 7f9487e00640 1 -- 172.21.15.67:0/41447633 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94900ae6f0 con 0x7f94700036e0 2024-03-09T08:56:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.681+0000 7f9487e00640 1 -- 172.21.15.67:0/41447633 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9490157000 con 0x7f9470007390 2024-03-09T08:56:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.684+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.684+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 msgr2=0x7f9470009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:56:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.684+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 msgr2=0x7f9470009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:56:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.684+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:56:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.685+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:56:10.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:56:10.875 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:56:10.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.886+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:10.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.887+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 msgr2=0x7f9470009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:56:10.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.887+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 msgr2=0x7f9470009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:56:10.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.887+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:56:10.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:10.887+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:56:11.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:11.288+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:11.289+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 msgr2=0x7f9470009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:56:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:11.289+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 msgr2=0x7f9470009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:56:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:11.289+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:56:11.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:11.289+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:56:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.090+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.091+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 msgr2=0x7f9470009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:56:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.091+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 msgr2=0x7f9470009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:56:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.091+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:56:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.091+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:56:12.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.460+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361154711 0 0) 0x7f94900ae6f0 con 0x7f94700036e0 2024-03-09T08:56:12.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.460+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9474003520 con 0x7f94700036e0 2024-03-09T08:56:12.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.479+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1075354904 0 0) 0x7f9474003520 con 0x7f94700036e0 2024-03-09T08:56:12.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.479+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 msgr2=0x7f9470009850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:56:12.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.479+0000 7f948d600640 1 --2- 172.21.15.67:0/41447633 >> v2:172.21.15.67:3300/0 conn(0x7f9470007390 0x7f9470009850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.479+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9470009ed0 con 0x7f94700036e0 2024-03-09T08:56:12.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.479+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f94800016b0 con 0x7f94700036e0 2024-03-09T08:56:12.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.480+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 >> v1:172.21.15.134:6789/0 conn(0x7f94700036e0 legacy=0x7f9470005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:12.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.480+0000 7f948f400640 1 -- 172.21.15.67:0/41447633 reap_dead start 2024-03-09T08:56:12.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.480+0000 7f948d600640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 0x7f9474003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:12.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.480+0000 7f948d600640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 0x7f9474004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:12.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.480+0000 7f948d600640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 0x7f947400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:12.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.480+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490157000 con 0x7f94740052e0 2024-03-09T08:56:12.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.480+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947400c1b0 con 0x7f9474003a20 2024-03-09T08:56:12.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.480+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947400c3b0 con 0x7f9474004800 2024-03-09T08:56:12.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.480+0000 7f948e000640 1 -- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 msgr2=0x7f9474004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:56:12.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.480+0000 7f948e000640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 0x7f9474004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:56:12.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.481+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 0x7f947400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:12.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.481+0000 7f948ea00640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 0x7f9474003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:12.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.482+0000 7f948ea00640 1 -- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 msgr2=0x7f9474004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:56:12.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.482+0000 7f948ea00640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 0x7f9474004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.482+0000 7f948ea00640 1 -- 172.21.15.67:0/41447633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 msgr2=0x7f947400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:56:12.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.482+0000 7f948ea00640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 0x7f947400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.482+0000 7f948ea00640 1 -- 172.21.15.67:0/41447633 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474003520 con 0x7f9474003a20 2024-03-09T08:56:12.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.482+0000 7f948f400640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 0x7f947400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:56:12.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.484+0000 7f948ea00640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 0x7f9474003e20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9480007f70 tx=0x7f9474005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd9ad596ca117673 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:12.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.485+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480007bc0 con 0x7f9474003a20 2024-03-09T08:56:12.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.485+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9480030070 con 0x7f9474003a20 2024-03-09T08:56:12.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.485+0000 7f948d600640 1 -- 172.21.15.67:0/41447633 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480005cf0 con 0x7f9474003a20 2024-03-09T08:56:12.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.486+0000 7f948fe00640 1 -- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 msgr2=0x7f9474003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:12.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.486+0000 7f948fe00640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 0x7f9474003e20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9480007f70 tx=0x7f9474005830 comp rx=0 tx=0).stop 2024-03-09T08:56:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.487+0000 7f948fe00640 1 -- 172.21.15.67:0/41447633 shutdown_connections 2024-03-09T08:56:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.487+0000 7f948fe00640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 0x7f947400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.487+0000 7f948fe00640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 0x7f9474004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.487+0000 7f948fe00640 1 --2- 172.21.15.67:0/41447633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 0x7f9474003e20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.487+0000 7f948fe00640 1 -- 172.21.15.67:0/41447633 >> 172.21.15.67:0/41447633 conn(0x7f94900732d0 msgr2=0x7f94900ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.487+0000 7f948fe00640 1 -- 172.21.15.67:0/41447633 shutdown_connections 2024-03-09T08:56:12.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.487+0000 7f948fe00640 1 -- 172.21.15.67:0/41447633 wait complete. 2024-03-09T08:56:12.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.491+0000 7f948fe00640 1 Processor -- start 2024-03-09T08:56:12.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.491+0000 7f948fe00640 1 -- start start 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.493+0000 7f948fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 0x7f949014db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.493+0000 7f948fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 0x7f949014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.494+0000 7f948fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 0x7f949006c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.494+0000 7f948fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490159e90 con 0x7f94740052e0 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.494+0000 7f948fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490159bb0 con 0x7f9474003a20 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.494+0000 7f948fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94900c59e0 con 0x7f9474004800 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.494+0000 7f948e000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 msgr2=0x7f949014e080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.494+0000 7f948e000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 0x7f949014e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.494+0000 7f948ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 0x7f949014db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.494+0000 7f948f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 0x7f949006c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.494+0000 7f948ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 0x7f949014db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:47690/0 (socket says 172.21.15.67:47690) 2024-03-09T08:56:12.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.494+0000 7f948ea00640 1 -- 172.21.15.67:0/3336233024 learned_addr learned my addr 172.21.15.67:0/3336233024 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:56:12.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.495+0000 7f948f400640 1 -- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 msgr2=0x7f949014e080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:56:12.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.495+0000 7f948f400640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 0x7f949014e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.495+0000 7f948f400640 1 -- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 msgr2=0x7f949014db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:12.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.495+0000 7f948f400640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 0x7f949014db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.495+0000 7f948f400640 1 -- 172.21.15.67:0/3336233024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949014d180 con 0x7f94740052e0 2024-03-09T08:56:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.496+0000 7f948f400640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 0x7f949006c810 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f94900c8920 tx=0x7f9488015700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.496+0000 7f9486a00640 1 -- 172.21.15.67:0/3336233024 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948801c030 con 0x7f94740052e0 2024-03-09T08:56:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.496+0000 7f9486a00640 1 -- 172.21.15.67:0/3336233024 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9488021410 con 0x7f94740052e0 2024-03-09T08:56:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.496+0000 7f9486a00640 1 -- 172.21.15.67:0/3336233024 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94880217e0 con 0x7f94740052e0 2024-03-09T08:56:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.496+0000 7f948fe00640 1 -- 172.21.15.67:0/3336233024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949006b710 con 0x7f94740052e0 2024-03-09T08:56:12.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.497+0000 7f948fe00640 1 -- 172.21.15.67:0/3336233024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949006b430 con 0x7f94740052e0 2024-03-09T08:56:12.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.499+0000 7f948fe00640 1 -- 172.21.15.67:0/3336233024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94900aefb0 con 0x7f94740052e0 2024-03-09T08:56:12.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.500+0000 7f9486a00640 1 -- 172.21.15.67:0/3336233024 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9488005d10 con 0x7f94740052e0 2024-03-09T08:56:12.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.500+0000 7f9486a00640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9470074560 0x7f9470076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:12.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.501+0000 7f9486a00640 1 -- 172.21.15.67:0/3336233024 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f948808b040 con 0x7f94740052e0 2024-03-09T08:56:12.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.501+0000 7f948ea00640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9470074560 0x7f9470076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:12.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.503+0000 7f948ea00640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9470074560 0x7f9470076a20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9480000c00 tx=0x7f9480009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:12.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.506+0000 7f9486a00640 1 -- 172.21.15.67:0/3336233024 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f94900aefb0 con 0x7f94740052e0 2024-03-09T08:56:12.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.823+0000 7f948fe00640 1 -- 172.21.15.67:0/3336233024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f94900ca3d0 con 0x7f94740052e0 2024-03-09T08:56:12.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.824+0000 7f9486a00640 1 -- 172.21.15.67:0/3336233024 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7f9488026070 con 0x7f94740052e0 2024-03-09T08:56:12.823 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":32,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:56:12.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.827+0000 7f948fe00640 1 -- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9470074560 msgr2=0x7f9470076a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:12.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.827+0000 7f948fe00640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9470074560 0x7f9470076a20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9480000c00 tx=0x7f9480009030 comp rx=0 tx=0).stop 2024-03-09T08:56:12.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.827+0000 7f948fe00640 1 -- 172.21.15.67:0/3336233024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 msgr2=0x7f949006c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:12.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.827+0000 7f948fe00640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 0x7f949006c810 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f94900c8920 tx=0x7f9488015700 comp rx=0 tx=0).stop 2024-03-09T08:56:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.828+0000 7f948fe00640 1 -- 172.21.15.67:0/3336233024 shutdown_connections 2024-03-09T08:56:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.828+0000 7f948fe00640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9470074560 0x7f9470076a20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.828+0000 7f948fe00640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94740052e0 0x7f949006c810 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.828+0000 7f948fe00640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9474004800 0x7f949014e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.828+0000 7f948fe00640 1 --2- 172.21.15.67:0/3336233024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9474003a20 0x7f949014db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.828+0000 7f948fe00640 1 -- 172.21.15.67:0/3336233024 >> 172.21.15.67:0/3336233024 conn(0x7f94900732d0 msgr2=0x7f9490150570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.828+0000 7f948fe00640 1 -- 172.21.15.67:0/3336233024 shutdown_connections 2024-03-09T08:56:12.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:12.828+0000 7f948fe00640 1 -- 172.21.15.67:0/3336233024 wait complete. 2024-03-09T08:56:12.848 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T08:56:12.848 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T08:56:12.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:56:13.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.053+0000 7fee07e00640 1 Processor -- start 2024-03-09T08:56:13.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.053+0000 7fee07e00640 1 -- start start 2024-03-09T08:56:13.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.055+0000 7fee07e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fee080c9380 0x7fee080acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.055+0000 7fee07e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee080ad440 0x7fee0806c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.055+0000 7fee07e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee080c9890 con 0x7fee0814fd90 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.055+0000 7fee07e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0806ba60 con 0x7fee080c9380 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.055+0000 7fee07e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee080c5f80 con 0x7fee080ad440 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.055+0000 7fedfe000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fee080c9380 0x7fee080acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.055+0000 7fee07400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee080ad440 msgr2=0x7fee0806c1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.056+0000 7fee07400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee080ad440 0x7fee0806c1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.056+0000 7fedfe000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fee080c9380 0x7fee080acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51228/0 (socket says 172.21.15.67:51228) 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.056+0000 7fedfe000640 1 -- 172.21.15.67:0/741901429 learned_addr learned my addr 172.21.15.67:0/741901429 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.056+0000 7fedfe000640 1 -- 172.21.15.67:0/741901429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee080ad440 msgr2=0x7fee0806c1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.056+0000 7fedfe000640 1 --2- 172.21.15.67:0/741901429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee080ad440 0x7fee0806c1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.056+0000 7fedfe000640 1 -- 172.21.15.67:0/741901429 >> v1:172.21.15.134:6789/0 conn(0x7fee0814fd90 legacy=0x7fee08156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.056+0000 7fedfe000640 1 -- 172.21.15.67:0/741901429 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee080c6240 con 0x7fee080c9380 2024-03-09T08:56:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.058+0000 7fedfe000640 1 --2- 172.21.15.67:0/741901429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080c9380 0x7fee080acf00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009870 tx=0x7fedf002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7967786e8a8b2037 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.058+0000 7fee06000640 1 -- 172.21.15.67:0/741901429 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf0035020 con 0x7fee080c9380 2024-03-09T08:56:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.058+0000 7fee06000640 1 -- 172.21.15.67:0/741901429 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf0031800 con 0x7fee080c9380 2024-03-09T08:56:13.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.058+0000 7fee06000640 1 -- 172.21.15.67:0/741901429 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf0031c10 con 0x7fee080c9380 2024-03-09T08:56:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.059+0000 7fee07e00640 1 -- 172.21.15.67:0/741901429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080c9380 msgr2=0x7fee080acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.059+0000 7fee07e00640 1 --2- 172.21.15.67:0/741901429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080c9380 0x7fee080acf00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009870 tx=0x7fedf002b8a0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.059+0000 7fee07e00640 1 -- 172.21.15.67:0/741901429 shutdown_connections 2024-03-09T08:56:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.059+0000 7fee07e00640 1 --2- 172.21.15.67:0/741901429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee080ad440 0x7fee0806c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.059+0000 7fee07e00640 1 --2- 172.21.15.67:0/741901429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080c9380 0x7fee080acf00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.059+0000 7fee07e00640 1 -- 172.21.15.67:0/741901429 >> 172.21.15.67:0/741901429 conn(0x7fee080cb480 msgr2=0x7fee080cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.059+0000 7fee07e00640 1 -- 172.21.15.67:0/741901429 shutdown_connections 2024-03-09T08:56:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.059+0000 7fee07e00640 1 -- 172.21.15.67:0/741901429 wait complete. 2024-03-09T08:56:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.062+0000 7fee07e00640 1 Processor -- start 2024-03-09T08:56:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.062+0000 7fee07e00640 1 -- start start 2024-03-09T08:56:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fee07e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ad440 0x7fee08156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fee07e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee080c9380 0x7fee0814c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fee07e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee0814fd90 0x7fee080ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fee07e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0806c710 con 0x7fee080ad440 2024-03-09T08:56:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fee07e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0806bcb0 con 0x7fee0814fd90 2024-03-09T08:56:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fee07e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0806c890 con 0x7fee080c9380 2024-03-09T08:56:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fedfe000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee080c9380 msgr2=0x7fee0814c280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:56:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fedfe000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee080c9380 0x7fee0814c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:56:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fee06a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ad440 0x7fee08156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fee06a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ad440 0x7fee08156fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51236/0 (socket says 172.21.15.67:51236) 2024-03-09T08:56:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fee06a00640 1 -- 172.21.15.67:0/4214266453 learned_addr learned my addr 172.21.15.67:0/4214266453 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:56:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.064+0000 7fee07400640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee0814fd90 0x7fee080ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.065+0000 7fee06a00640 1 -- 172.21.15.67:0/4214266453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee080c9380 msgr2=0x7fee0814c280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:56:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.065+0000 7fee06a00640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee080c9380 0x7fee0814c280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.065+0000 7fee06a00640 1 -- 172.21.15.67:0/4214266453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee0814fd90 msgr2=0x7fee080ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.065+0000 7fee06a00640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee0814fd90 0x7fee080ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.065+0000 7fee06a00640 1 -- 172.21.15.67:0/4214266453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee080aba50 con 0x7fee080ad440 2024-03-09T08:56:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.065+0000 7fee06a00640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ad440 0x7fee08156fb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c760 tx=0x7fedf800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.065+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8003b70 con 0x7fee080ad440 2024-03-09T08:56:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.065+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf8003d30 con 0x7fee080ad440 2024-03-09T08:56:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.065+0000 7fee07e00640 1 -- 172.21.15.67:0/4214266453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee08159ad0 con 0x7fee080ad440 2024-03-09T08:56:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.065+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800aa90 con 0x7fee080ad440 2024-03-09T08:56:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.065+0000 7fee07e00640 1 -- 172.21.15.67:0/4214266453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee081574f0 con 0x7fee080ad440 2024-03-09T08:56:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.067+0000 7fee07e00640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee081eedd0 0x7fee08146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.067+0000 7fee07e00640 1 -- 172.21.15.67:0/4214266453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee080ab820 con 0x7fee081eedd0 2024-03-09T08:56:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.067+0000 7fedfe000640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee081eedd0 0x7fee08146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.067+0000 7fedfe000640 1 -- 172.21.15.67:0/4214266453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee080abd30 con 0x7fee081eedd0 2024-03-09T08:56:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.068+0000 7fedfe000640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee081eedd0 0x7fee08146540 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fee080ada40 tx=0x7fedf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.068+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf003bc60 con 0x7fee081eedd0 2024-03-09T08:56:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.069+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fedf003be00 con 0x7fee081eedd0 2024-03-09T08:56:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.069+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee081eedd0 msgr2=0x7fee08146540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.069+0000 7fedff400640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee081eedd0 0x7fee08146540 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fee080ada40 tx=0x7fedf00046c0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.069+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee081eedd0 msgr2=0x7fee08146540 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:56:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.069+0000 7fedff400640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee081eedd0 0x7fee08146540 secure :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fee080ada40 tx=0x7fedf00046c0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.069+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fedf800ac30 con 0x7fee080ad440 2024-03-09T08:56:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.070+0000 7fedff400640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fede406afe0 0x7fede406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.070+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fedf808a530 con 0x7fee080ad440 2024-03-09T08:56:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.071+0000 7fedfe000640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fede406afe0 0x7fede406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.072+0000 7fedfe000640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fede406afe0 0x7fede406d4a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fee081576c0 tx=0x7fedf0041000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:13.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.077+0000 7fee07e00640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ac820 0x7fee08146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.077+0000 7fee07e00640 1 -- 172.21.15.67:0/4214266453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee08159910 con 0x7fee080ac820 2024-03-09T08:56:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.077+0000 7fee07400640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ac820 0x7fee08146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.078+0000 7fee07400640 1 -- 172.21.15.67:0/4214266453 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fee0804f3c0 con 0x7fee080ac820 2024-03-09T08:56:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.078+0000 7fee07400640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ac820 0x7fee08146a80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fee00003840 tx=0x7fee0000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.079+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00016030 con 0x7fee080ac820 2024-03-09T08:56:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.079+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5177 (secure 0 0 0) 0x7fee00003960 con 0x7fee080ac820 2024-03-09T08:56:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.079+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ac820 msgr2=0x7fee08146a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.079+0000 7fedff400640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ac820 0x7fee08146a80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fee00003840 tx=0x7fee0000db90 comp rx=0 tx=0).stop 2024-03-09T08:56:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.079+0000 7fedff400640 1 -- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ac820 msgr2=0x7fee08146a80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:56:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.079+0000 7fedff400640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ac820 0x7fee08146a80 secure :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fee00003840 tx=0x7fee0000db90 comp rx=0 tx=0).stop 2024-03-09T08:56:13.078 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:56:13.078 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 32, 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:56:13.079 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:13.080 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:13.080 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:13.080 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:13.080 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:13.080 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:13.080 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:13.080 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:13.080 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:13.080 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:13.080 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:13.081 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:56:13.082 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:13.083 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.086 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.087 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:56:13.088 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:13.089 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:13.090 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T08:56:13.091 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.091 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.091 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:56:13.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.091 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:56:13.091 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:13.091 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:56:13.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.093 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:56:13.093 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:56:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.080+0000 7fee07e00640 1 -- 172.21.15.67:0/4214266453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fede406afe0 msgr2=0x7fede406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.080+0000 7fee07e00640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fede406afe0 0x7fede406d4a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fee081576c0 tx=0x7fedf0041000 comp rx=0 tx=0).stop 2024-03-09T08:56:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.080+0000 7fee07e00640 1 -- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ad440 msgr2=0x7fee08156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.080+0000 7fee07e00640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ad440 0x7fee08156fb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c760 tx=0x7fedf800cc20 comp rx=0 tx=0).stop 2024-03-09T08:56:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.080+0000 7fee07400640 1 -- 172.21.15.67:0/4214266453 reap_dead start 2024-03-09T08:56:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.080+0000 7fee07e00640 1 -- 172.21.15.67:0/4214266453 shutdown_connections 2024-03-09T08:56:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.080+0000 7fee07e00640 1 --2- 172.21.15.67:0/4214266453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee080ad440 0x7fee08156fb0 secure :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c760 tx=0x7fedf800cc20 comp rx=0 tx=0).stop 2024-03-09T08:56:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.080+0000 7fee07e00640 1 -- 172.21.15.67:0/4214266453 >> 172.21.15.67:0/4214266453 conn(0x7fee080cb480 msgr2=0x7fee080c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.080+0000 7fee07e00640 1 -- 172.21.15.67:0/4214266453 shutdown_connections 2024-03-09T08:56:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.081+0000 7fee07e00640 1 -- 172.21.15.67:0/4214266453 wait complete. 2024-03-09T08:56:13.103 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:56:13.103 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T08:56:13.103 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:56:13.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.313+0000 7facebe00640 1 Processor -- start 2024-03-09T08:56:13.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.313+0000 7facebe00640 1 -- start start 2024-03-09T08:56:13.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.314+0000 7facebe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facec0ac930 0x7facec0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.314+0000 7facebe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7facec151360 0x7facec1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.314+0000 7facebe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec0c2780 con 0x7facec0c30a0 2024-03-09T08:56:13.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.314+0000 7facebe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec070670 con 0x7facec151360 2024-03-09T08:56:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.314+0000 7facebe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec0708c0 con 0x7facec0ac930 2024-03-09T08:56:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.315+0000 7faceaa00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facec0ac930 msgr2=0x7facec0c3660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:56:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.315+0000 7faceaa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facec0ac930 0x7facec0c3660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:56:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.315+0000 7faceb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7facec151360 0x7facec1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.315+0000 7faceb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7facec151360 0x7facec1539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51254/0 (socket says 172.21.15.67:51254) 2024-03-09T08:56:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.315+0000 7faceb400640 1 -- 172.21.15.67:0/2544704449 learned_addr learned my addr 172.21.15.67:0/2544704449 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:56:13.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.315+0000 7faceb400640 1 -- 172.21.15.67:0/2544704449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facec0ac930 msgr2=0x7facec0c3660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:56:13.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.315+0000 7faceb400640 1 --2- 172.21.15.67:0/2544704449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facec0ac930 0x7facec0c3660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.315+0000 7faceb400640 1 -- 172.21.15.67:0/2544704449 >> v1:172.21.15.134:6789/0 conn(0x7facec0c30a0 legacy=0x7facec150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.315+0000 7faceb400640 1 -- 172.21.15.67:0/2544704449 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec070ba0 con 0x7facec151360 2024-03-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.316+0000 7faceb400640 1 --2- 172.21.15.67:0/2544704449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facec151360 0x7facec1539e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7face4009e50 tx=0x7face402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d9ee140bd46b6a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.317+0000 7face9600640 1 -- 172.21.15.67:0/2544704449 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face402cb20 con 0x7facec151360 2024-03-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.317+0000 7face9600640 1 -- 172.21.15.67:0/2544704449 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face402ccc0 con 0x7facec151360 2024-03-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.317+0000 7face9600640 1 -- 172.21.15.67:0/2544704449 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4034610 con 0x7facec151360 2024-03-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.317+0000 7facebe00640 1 -- 172.21.15.67:0/2544704449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facec151360 msgr2=0x7facec1539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.317+0000 7facebe00640 1 --2- 172.21.15.67:0/2544704449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facec151360 0x7facec1539e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7face4009e50 tx=0x7face402bf80 comp rx=0 tx=0).stop 2024-03-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.317+0000 7facebe00640 1 -- 172.21.15.67:0/2544704449 shutdown_connections 2024-03-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.317+0000 7facebe00640 1 --2- 172.21.15.67:0/2544704449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facec151360 0x7facec1539e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.317+0000 7facebe00640 1 --2- 172.21.15.67:0/2544704449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facec0ac930 0x7facec0c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.317+0000 7facebe00640 1 -- 172.21.15.67:0/2544704449 >> 172.21.15.67:0/2544704449 conn(0x7facec0c7220 msgr2=0x7facec0c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.317+0000 7facebe00640 1 -- 172.21.15.67:0/2544704449 shutdown_connections 2024-03-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.318+0000 7facebe00640 1 -- 172.21.15.67:0/2544704449 wait complete. 2024-03-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.320+0000 7facebe00640 1 Processor -- start 2024-03-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.320+0000 7facebe00640 1 -- start start 2024-03-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7facebe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facec0ac930 0x7facec14f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7facebe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec0c30a0 0x7facec148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7facebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facec151360 0x7facec0c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7facebe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec0ace90 con 0x7facec151360 2024-03-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7facebe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec0acd10 con 0x7facec0c30a0 2024-03-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7facebe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec153f20 con 0x7facec0ac930 2024-03-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7faceaa00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facec0ac930 msgr2=0x7facec14f760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7faceaa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facec0ac930 0x7facec14f760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7faceb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facec151360 0x7facec0c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7faceb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facec151360 0x7facec0c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51264/0 (socket says 172.21.15.67:51264) 2024-03-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7faceb400640 1 -- 172.21.15.67:0/3364181013 learned_addr learned my addr 172.21.15.67:0/3364181013 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.322+0000 7facea000640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec0c30a0 0x7facec148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.323+0000 7faceb400640 1 -- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facec0ac930 msgr2=0x7facec14f760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.323+0000 7faceb400640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7facec0ac930 0x7facec14f760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.323+0000 7faceb400640 1 -- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec0c30a0 msgr2=0x7facec148510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.323+0000 7faceb400640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec0c30a0 0x7facec148510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.323+0000 7faceb400640 1 -- 172.21.15.67:0/3364181013 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec150570 con 0x7facec151360 2024-03-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.323+0000 7faceb400640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facec151360 0x7facec0c6670 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7face4036040 tx=0x7face4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.323+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4043020 con 0x7facec151360 2024-03-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.323+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face403e450 con 0x7facec151360 2024-03-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.323+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face403e860 con 0x7facec151360 2024-03-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.323+0000 7facebe00640 1 -- 172.21.15.67:0/3364181013 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec148230 con 0x7facec151360 2024-03-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.324+0000 7facebe00640 1 -- 172.21.15.67:0/3364181013 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec06dcc0 con 0x7facec151360 2024-03-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.325+0000 7facebe00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec154d70 0x7facec1ee580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.325+0000 7facebe00640 1 -- 172.21.15.67:0/3364181013 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec14fca0 con 0x7facec154d70 2024-03-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.325+0000 7faceaa00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec154d70 0x7facec1ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.326+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7face4046020 con 0x7facec151360 2024-03-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.326+0000 7faceaa00640 1 -- 172.21.15.67:0/3364181013 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facec148a50 con 0x7facec154d70 2024-03-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.326+0000 7faceaa00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec154d70 0x7facec1ee580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7facdc003930 tx=0x7facdc00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.326+0000 7face2a00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faccc06b480 0x7faccc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.327+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7face40aca90 con 0x7facec151360 2024-03-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.327+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc003e10 con 0x7facec154d70 2024-03-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.327+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7facdc015070 con 0x7facec154d70 2024-03-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.327+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec154d70 msgr2=0x7facec1ee580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.327+0000 7face2a00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec154d70 0x7facec1ee580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7facdc003930 tx=0x7facdc00b8c0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.327+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec154d70 msgr2=0x7facec1ee580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.327+0000 7face2a00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec154d70 0x7facec1ee580 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7facdc003930 tx=0x7facdc00b8c0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.327+0000 7facea000640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faccc06b480 0x7faccc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.328+0000 7facea000640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faccc06b480 0x7faccc06d940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7facec072c90 tx=0x7facd8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.334+0000 7facebe00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec155150 0x7facec1eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.334+0000 7facebe00640 1 -- 172.21.15.67:0/3364181013 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec04f3c0 con 0x7facec155150 2024-03-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.335+0000 7faceaa00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec155150 0x7facec1eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.335+0000 7faceaa00640 1 -- 172.21.15.67:0/3364181013 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7facec0729e0 con 0x7facec155150 2024-03-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.336+0000 7faceaa00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec155150 0x7facec1eeac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7facdc003930 tx=0x7facdc005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:13.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.336+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc005850 con 0x7facec155150 2024-03-09T08:56:13.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.337+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4862 (secure 0 0 0) 0x7facdc005850 con 0x7facec155150 2024-03-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.337+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec155150 msgr2=0x7facec1eeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.337+0000 7face2a00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec155150 0x7facec1eeac0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7facdc003930 tx=0x7facdc005a20 comp rx=0 tx=0).stop 2024-03-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.337+0000 7face2a00640 1 -- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec155150 msgr2=0x7facec1eeac0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.337+0000 7face2a00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7facec155150 0x7facec1eeac0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 32, 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:56:13.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.337+0000 7facebe00640 1 -- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faccc06b480 msgr2=0x7faccc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.337+0000 7facebe00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faccc06b480 0x7faccc06d940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7facec072c90 tx=0x7facd8009040 comp rx=0 tx=0).stop 2024-03-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.338+0000 7facebe00640 1 -- 172.21.15.67:0/3364181013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facec151360 msgr2=0x7facec0c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.338+0000 7facebe00640 1 --2- 172.21.15.67:0/3364181013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7facec151360 0x7facec0c6670 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7face4036040 tx=0x7face4004a40 comp rx=0 tx=0).stop 2024-03-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.338+0000 7faceb400640 1 -- 172.21.15.67:0/3364181013 reap_dead start 2024-03-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.338+0000 7faceb400640 1 -- 172.21.15.67:0/3364181013 reap_dead start 2024-03-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.338+0000 7facebe00640 1 -- 172.21.15.67:0/3364181013 shutdown_connections 2024-03-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.338+0000 7facebe00640 1 -- 172.21.15.67:0/3364181013 >> 172.21.15.67:0/3364181013 conn(0x7facec0c7220 msgr2=0x7facec06d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.338+0000 7facebe00640 1 -- 172.21.15.67:0/3364181013 shutdown_connections 2024-03-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:13.338+0000 7facebe00640 1 -- 172.21.15.67:0/3364181013 wait complete. 2024-03-09T08:56:13.356 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:56:13.356 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T08:56:14.828 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2024-03-09T08:56:14.828 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2024-03-09T08:56:14.828 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2024-03-09T08:56:14.828 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2024-03-09T08:56:14.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2024-03-09T08:56:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2024-03-09T08:56:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2024-03-09T08:56:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2024-03-09T08:56:14.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2024-03-09T08:56:14.834 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2024-03-09T08:56:14.834 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2024-03-09T08:56:14.834 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2024-03-09T08:56:14.834 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2024-03-09T08:56:14.834 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2024-03-09T08:56:14.834 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2024-03-09T08:56:14.834 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2024-03-09T08:56:14.834 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2024-03-09T08:56:15.197 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2024-03-09T08:56:15.197 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2024-03-09T08:56:15.197 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2024-03-09T08:56:15.197 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2024-03-09T08:56:15.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2024-03-09T08:56:15.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2024-03-09T08:56:15.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2024-03-09T08:56:15.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2024-03-09T08:56:15.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2024-03-09T08:56:15.203 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2024-03-09T08:56:15.203 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2024-03-09T08:56:15.203 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2024-03-09T08:56:15.203 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2024-03-09T08:56:15.203 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2024-03-09T08:56:15.203 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2024-03-09T08:56:15.203 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2024-03-09T08:56:15.946 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2024-03-09T08:56:15.947 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2024-03-09T08:56:15.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2024-03-09T08:56:15.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2024-03-09T08:56:15.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2024-03-09T08:56:15.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2024-03-09T08:56:15.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2024-03-09T08:56:15.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2024-03-09T08:56:15.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2024-03-09T08:56:15.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2024-03-09T08:56:15.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2024-03-09T08:56:15.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2024-03-09T08:56:15.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2024-03-09T08:56:15.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2024-03-09T08:56:15.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2024-03-09T08:56:15.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2024-03-09T08:56:15.952 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2024-03-09T08:56:15.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2024-03-09T08:56:15.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2024-03-09T08:56:15.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2024-03-09T08:56:15.956 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2024-03-09T08:56:15.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2024-03-09T08:56:15.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2024-03-09T08:56:15.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2024-03-09T08:56:15.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2024-03-09T08:56:15.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2024-03-09T08:56:15.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2024-03-09T08:56:15.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2024-03-09T08:56:15.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2024-03-09T08:56:15.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2024-03-09T08:56:15.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2024-03-09T08:56:15.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2024-03-09T08:56:17.285 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2024-03-09T08:56:17.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2024-03-09T08:56:17.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2024-03-09T08:56:17.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2024-03-09T08:56:17.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2024-03-09T08:56:17.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2024-03-09T08:56:17.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2024-03-09T08:56:17.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2024-03-09T08:56:17.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2024-03-09T08:56:17.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2024-03-09T08:56:17.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2024-03-09T08:56:17.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2024-03-09T08:56:17.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2024-03-09T08:56:17.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2024-03-09T08:56:17.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2024-03-09T08:56:17.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2024-03-09T08:56:17.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2024-03-09T08:56:17.670 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2024-03-09T08:56:17.670 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2024-03-09T08:56:17.670 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2024-03-09T08:56:17.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2024-03-09T08:56:17.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2024-03-09T08:56:17.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2024-03-09T08:56:17.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2024-03-09T08:56:17.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2024-03-09T08:56:17.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2024-03-09T08:56:17.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2024-03-09T08:56:17.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2024-03-09T08:56:17.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2024-03-09T08:56:17.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2024-03-09T08:56:17.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2024-03-09T08:56:17.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2024-03-09T08:56:17.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2024-03-09T08:56:17.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2024-03-09T08:56:17.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2024-03-09T08:56:17.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2024-03-09T08:56:19.019 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2024-03-09T08:56:19.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2024-03-09T08:56:19.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2024-03-09T08:56:19.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2024-03-09T08:56:19.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2024-03-09T08:56:19.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2024-03-09T08:56:19.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2024-03-09T08:56:19.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2024-03-09T08:56:19.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2024-03-09T08:56:19.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2024-03-09T08:56:19.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2024-03-09T08:56:19.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2024-03-09T08:56:19.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2024-03-09T08:56:19.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2024-03-09T08:56:19.166 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2024-03-09T08:56:19.166 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2024-03-09T08:56:19.166 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2024-03-09T08:56:19.166 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2024-03-09T08:56:19.166 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2024-03-09T08:56:19.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2024-03-09T08:56:19.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2024-03-09T08:56:19.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2024-03-09T08:56:19.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2024-03-09T08:56:19.171 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2024-03-09T08:56:19.171 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2024-03-09T08:56:19.171 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2024-03-09T08:56:19.171 INFO:tasks.workunit.client.1.smithi134.stdout:Take third snapshot .snap/test3 2024-03-09T08:56:19.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2024-03-09T08:56:19.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2024-03-09T08:56:19.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2024-03-09T08:56:19.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2024-03-09T08:56:19.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2024-03-09T08:56:19.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2024-03-09T08:56:19.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2024-03-09T08:56:19.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2024-03-09T08:56:19.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2024-03-09T08:56:19.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2024-03-09T08:56:19.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2024-03-09T08:56:19.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2024-03-09T08:56:19.349 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2024-03-09T08:56:19.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2024-03-09T08:56:19.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2024-03-09T08:56:19.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2024-03-09T08:56:19.430 INFO:tasks.workunit.client.1.smithi134.stdout:Delete the snapshots... 2024-03-09T08:56:20.833 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2024-03-09T08:56:20.833 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2024-03-09T08:56:20.833 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2024-03-09T08:56:20.833 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2024-03-09T08:56:20.833 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2024-03-09T08:56:20.833 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2024-03-09T08:56:20.834 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2024-03-09T08:56:20.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2024-03-09T08:56:20.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2024-03-09T08:56:20.837 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2024-03-09T08:56:20.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2024-03-09T08:56:20.839 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2024-03-09T08:56:20.839 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2024-03-09T08:56:20.839 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2024-03-09T08:56:20.839 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2024-03-09T08:56:21.814 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2024-03-09T08:56:21.814 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2024-03-09T08:56:21.814 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2024-03-09T08:56:21.814 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2024-03-09T08:56:21.814 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2024-03-09T08:56:21.814 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2024-03-09T08:56:21.814 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2024-03-09T08:56:21.815 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2024-03-09T08:56:21.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2024-03-09T08:56:21.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2024-03-09T08:56:21.818 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2024-03-09T08:56:21.819 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2024-03-09T08:56:21.819 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2024-03-09T08:56:21.819 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2024-03-09T08:56:21.819 INFO:tasks.workunit.client.0.smithi067.stdout:Take third snapshot .snap/test3 2024-03-09T08:56:22.616 INFO:tasks.workunit.client.1.smithi134.stdout:.//.snap/test1 2024-03-09T08:56:22.616 INFO:tasks.workunit.client.1.smithi134.stdout:.//.snap/test2 2024-03-09T08:56:22.616 INFO:tasks.workunit.client.1.smithi134.stdout:.//.snap/test3 2024-03-09T08:56:22.630 INFO:tasks.workunit.client.1.smithi134.stdout:./274/.snap/snap-subdir-test 2024-03-09T08:56:22.650 INFO:tasks.workunit.client.1.smithi134.stdout:./166/.snap/snap-subdir-test 2024-03-09T08:56:22.670 INFO:tasks.workunit.client.1.smithi134.stdout:./256/.snap/snap-subdir-test 2024-03-09T08:56:22.813 INFO:tasks.workunit.client.0.smithi067.stdout:Delete the snapshots... 2024-03-09T08:56:23.217 INFO:tasks.workunit.client.1.smithi134.stdout:./247/.snap/snap-subdir-test 2024-03-09T08:56:23.232 INFO:tasks.workunit.client.1.smithi134.stdout:./248/.snap/snap-subdir-test 2024-03-09T08:56:23.246 INFO:tasks.workunit.client.1.smithi134.stdout:./147/.snap/snap-subdir-test 2024-03-09T08:56:23.261 INFO:tasks.workunit.client.1.smithi134.stdout:./151/.snap/snap-subdir-test 2024-03-09T08:56:23.276 INFO:tasks.workunit.client.1.smithi134.stdout:./170/.snap/snap-subdir-test 2024-03-09T08:56:23.290 INFO:tasks.workunit.client.1.smithi134.stdout:./167/.snap/snap-subdir-test 2024-03-09T08:56:23.305 INFO:tasks.workunit.client.1.smithi134.stdout:./140/.snap/snap-subdir-test 2024-03-09T08:56:23.318 INFO:tasks.workunit.client.1.smithi134.stdout:./142/.snap/snap-subdir-test 2024-03-09T08:56:23.336 INFO:tasks.workunit.client.1.smithi134.stdout:./150/.snap/snap-subdir-test 2024-03-09T08:56:23.351 INFO:tasks.workunit.client.1.smithi134.stdout:./273/.snap/snap-subdir-test 2024-03-09T08:56:23.367 INFO:tasks.workunit.client.1.smithi134.stdout:./269/.snap/snap-subdir-test 2024-03-09T08:56:23.401 INFO:tasks.workunit.client.1.smithi134.stdout:./105/.snap/snap-subdir-test 2024-03-09T08:56:23.432 INFO:tasks.workunit.client.1.smithi134.stdout:./210/.snap/snap-subdir-test 2024-03-09T08:56:23.923 INFO:tasks.workunit.client.1.smithi134.stdout:./265/.snap/snap-subdir-test 2024-03-09T08:56:23.939 INFO:tasks.workunit.client.1.smithi134.stdout:./218/.snap/snap-subdir-test 2024-03-09T08:56:23.955 INFO:tasks.workunit.client.1.smithi134.stdout:./290/.snap/snap-subdir-test 2024-03-09T08:56:23.971 INFO:tasks.workunit.client.1.smithi134.stdout:./149/.snap/snap-subdir-test 2024-03-09T08:56:23.987 INFO:tasks.workunit.client.1.smithi134.stdout:./109/.snap/snap-subdir-test 2024-03-09T08:56:24.003 INFO:tasks.workunit.client.1.smithi134.stdout:./292/.snap/snap-subdir-test 2024-03-09T08:56:24.016 INFO:tasks.workunit.client.1.smithi134.stdout:./242/.snap/snap-subdir-test 2024-03-09T08:56:24.030 INFO:tasks.workunit.client.1.smithi134.stdout:./158/.snap/snap-subdir-test 2024-03-09T08:56:24.044 INFO:tasks.workunit.client.1.smithi134.stdout:./243/.snap/snap-subdir-test 2024-03-09T08:56:24.060 INFO:tasks.workunit.client.1.smithi134.stdout:./106/.snap/snap-subdir-test 2024-03-09T08:56:24.076 INFO:tasks.workunit.client.1.smithi134.stdout:./250/.snap/snap-subdir-test 2024-03-09T08:56:24.092 INFO:tasks.workunit.client.1.smithi134.stdout:./221/.snap/snap-subdir-test 2024-03-09T08:56:24.108 INFO:tasks.workunit.client.1.smithi134.stdout:./244/.snap/snap-subdir-test 2024-03-09T08:56:24.123 INFO:tasks.workunit.client.1.smithi134.stdout:./258/.snap/snap-subdir-test 2024-03-09T08:56:24.139 INFO:tasks.workunit.client.1.smithi134.stdout:./268/.snap/snap-subdir-test 2024-03-09T08:56:24.154 INFO:tasks.workunit.client.1.smithi134.stdout:./189/.snap/snap-subdir-test 2024-03-09T08:56:24.170 INFO:tasks.workunit.client.1.smithi134.stdout:./285/.snap/snap-subdir-test 2024-03-09T08:56:24.186 INFO:tasks.workunit.client.1.smithi134.stdout:./208/.snap/snap-subdir-test 2024-03-09T08:56:24.201 INFO:tasks.workunit.client.1.smithi134.stdout:./288/.snap/snap-subdir-test 2024-03-09T08:56:24.216 INFO:tasks.workunit.client.1.smithi134.stdout:./209/.snap/snap-subdir-test 2024-03-09T08:56:24.232 INFO:tasks.workunit.client.1.smithi134.stdout:./253/.snap/snap-subdir-test 2024-03-09T08:56:24.247 INFO:tasks.workunit.client.1.smithi134.stdout:./249/.snap/snap-subdir-test 2024-03-09T08:56:24.263 INFO:tasks.workunit.client.1.smithi134.stdout:./133/.snap/snap-subdir-test 2024-03-09T08:56:24.277 INFO:tasks.workunit.client.1.smithi134.stdout:./291/.snap/snap-subdir-test 2024-03-09T08:56:24.297 INFO:tasks.workunit.client.1.smithi134.stdout:./226/.snap/snap-subdir-test 2024-03-09T08:56:24.316 INFO:tasks.workunit.client.1.smithi134.stdout:./173/.snap/snap-subdir-test 2024-03-09T08:56:24.332 INFO:tasks.workunit.client.1.smithi134.stdout:./174/.snap/snap-subdir-test 2024-03-09T08:56:24.349 INFO:tasks.workunit.client.1.smithi134.stdout:./299/.snap/snap-subdir-test 2024-03-09T08:56:24.365 INFO:tasks.workunit.client.1.smithi134.stdout:./262/.snap/snap-subdir-test 2024-03-09T08:56:24.433 INFO:tasks.workunit.client.1.smithi134.stdout:./231/.snap/snap-subdir-test 2024-03-09T08:56:24.433 INFO:tasks.workunit.client.1.smithi134.stdout:./214/.snap/snap-subdir-test 2024-03-09T08:56:24.433 INFO:tasks.workunit.client.1.smithi134.stdout:./281/.snap/snap-subdir-test 2024-03-09T08:56:24.434 INFO:tasks.workunit.client.1.smithi134.stdout:./289/.snap/snap-subdir-test 2024-03-09T08:56:24.442 INFO:tasks.workunit.client.1.smithi134.stdout:./283/.snap/snap-subdir-test 2024-03-09T08:56:24.458 INFO:tasks.workunit.client.1.smithi134.stdout:./232/.snap/snap-subdir-test 2024-03-09T08:56:24.477 INFO:tasks.workunit.client.1.smithi134.stdout:./298/.snap/snap-subdir-test 2024-03-09T08:56:24.492 INFO:tasks.workunit.client.1.smithi134.stdout:./233/.snap/snap-subdir-test 2024-03-09T08:56:24.510 INFO:tasks.workunit.client.1.smithi134.stdout:./155/.snap/snap-subdir-test 2024-03-09T08:56:24.671 INFO:tasks.workunit.client.0.smithi067.stdout:.//.snap/test1 2024-03-09T08:56:24.671 INFO:tasks.workunit.client.0.smithi067.stdout:.//.snap/test2 2024-03-09T08:56:24.671 INFO:tasks.workunit.client.0.smithi067.stdout:.//.snap/test3 2024-03-09T08:56:24.784 INFO:tasks.workunit.client.1.smithi134.stdout:./114/.snap/snap-subdir-test 2024-03-09T08:56:24.800 INFO:tasks.workunit.client.1.smithi134.stdout:./159/.snap/snap-subdir-test 2024-03-09T08:56:24.818 INFO:tasks.workunit.client.1.smithi134.stdout:./187/.snap/snap-subdir-test 2024-03-09T08:56:25.020 INFO:tasks.workunit.client.0.smithi067.stdout:./274/.snap/snap-subdir-test 2024-03-09T08:56:25.040 INFO:tasks.workunit.client.0.smithi067.stdout:./166/.snap/snap-subdir-test 2024-03-09T08:56:25.059 INFO:tasks.workunit.client.0.smithi067.stdout:./256/.snap/snap-subdir-test 2024-03-09T08:56:25.076 INFO:tasks.workunit.client.0.smithi067.stdout:./247/.snap/snap-subdir-test 2024-03-09T08:56:25.093 INFO:tasks.workunit.client.0.smithi067.stdout:./248/.snap/snap-subdir-test 2024-03-09T08:56:25.107 INFO:tasks.workunit.client.0.smithi067.stdout:./147/.snap/snap-subdir-test 2024-03-09T08:56:25.122 INFO:tasks.workunit.client.0.smithi067.stdout:./151/.snap/snap-subdir-test 2024-03-09T08:56:25.138 INFO:tasks.workunit.client.0.smithi067.stdout:./170/.snap/snap-subdir-test 2024-03-09T08:56:25.153 INFO:tasks.workunit.client.0.smithi067.stdout:./167/.snap/snap-subdir-test 2024-03-09T08:56:25.168 INFO:tasks.workunit.client.0.smithi067.stdout:./140/.snap/snap-subdir-test 2024-03-09T08:56:25.184 INFO:tasks.workunit.client.0.smithi067.stdout:./142/.snap/snap-subdir-test 2024-03-09T08:56:25.461 INFO:tasks.workunit.client.0.smithi067.stdout:./150/.snap/snap-subdir-test 2024-03-09T08:56:25.478 INFO:tasks.workunit.client.0.smithi067.stdout:./273/.snap/snap-subdir-test 2024-03-09T08:56:25.496 INFO:tasks.workunit.client.0.smithi067.stdout:./269/.snap/snap-subdir-test 2024-03-09T08:56:25.511 INFO:tasks.workunit.client.0.smithi067.stdout:./105/.snap/snap-subdir-test 2024-03-09T08:56:25.528 INFO:tasks.workunit.client.0.smithi067.stdout:./210/.snap/snap-subdir-test 2024-03-09T08:56:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:./265/.snap/snap-subdir-test 2024-03-09T08:56:25.562 INFO:tasks.workunit.client.0.smithi067.stdout:./218/.snap/snap-subdir-test 2024-03-09T08:56:25.580 INFO:tasks.workunit.client.0.smithi067.stdout:./290/.snap/snap-subdir-test 2024-03-09T08:56:25.588 INFO:tasks.workunit.client.1.smithi134.stdout:./185/.snap/snap-subdir-test 2024-03-09T08:56:25.600 INFO:tasks.workunit.client.0.smithi067.stdout:./149/.snap/snap-subdir-test 2024-03-09T08:56:25.605 INFO:tasks.workunit.client.1.smithi134.stdout:./293/.snap/snap-subdir-test 2024-03-09T08:56:25.618 INFO:tasks.workunit.client.0.smithi067.stdout:./109/.snap/snap-subdir-test 2024-03-09T08:56:25.619 INFO:tasks.workunit.client.1.smithi134.stdout:./228/.snap/snap-subdir-test 2024-03-09T08:56:25.631 INFO:tasks.workunit.client.1.smithi134.stdout:./120/.snap/snap-subdir-test 2024-03-09T08:56:25.635 INFO:tasks.workunit.client.0.smithi067.stdout:./292/.snap/snap-subdir-test 2024-03-09T08:56:25.643 INFO:tasks.workunit.client.1.smithi134.stdout:./110/.snap/snap-subdir-test 2024-03-09T08:56:25.648 INFO:tasks.workunit.client.0.smithi067.stdout:./242/.snap/snap-subdir-test 2024-03-09T08:56:25.659 INFO:tasks.workunit.client.1.smithi134.stdout:./267/.snap/snap-subdir-test 2024-03-09T08:56:25.660 INFO:tasks.workunit.client.0.smithi067.stdout:./158/.snap/snap-subdir-test 2024-03-09T08:56:25.673 INFO:tasks.workunit.client.0.smithi067.stdout:./243/.snap/snap-subdir-test 2024-03-09T08:56:25.673 INFO:tasks.workunit.client.1.smithi134.stdout:./202/.snap/snap-subdir-test 2024-03-09T08:56:25.683 INFO:tasks.workunit.client.1.smithi134.stdout:./137/.snap/snap-subdir-test 2024-03-09T08:56:25.684 INFO:tasks.workunit.client.0.smithi067.stdout:./106/.snap/snap-subdir-test 2024-03-09T08:56:25.694 INFO:tasks.workunit.client.0.smithi067.stdout:./250/.snap/snap-subdir-test 2024-03-09T08:56:25.695 INFO:tasks.workunit.client.1.smithi134.stdout:./246/.snap/snap-subdir-test 2024-03-09T08:56:25.707 INFO:tasks.workunit.client.0.smithi067.stdout:./221/.snap/snap-subdir-test 2024-03-09T08:56:25.708 INFO:tasks.workunit.client.1.smithi134.stdout:./219/.snap/snap-subdir-test 2024-03-09T08:56:25.721 INFO:tasks.workunit.client.1.smithi134.stdout:./198/.snap/snap-subdir-test 2024-03-09T08:56:25.721 INFO:tasks.workunit.client.0.smithi067.stdout:./244/.snap/snap-subdir-test 2024-03-09T08:56:25.731 INFO:tasks.workunit.client.0.smithi067.stdout:./258/.snap/snap-subdir-test 2024-03-09T08:56:25.731 INFO:tasks.workunit.client.1.smithi134.stdout:./136/.snap/snap-subdir-test 2024-03-09T08:56:25.741 INFO:tasks.workunit.client.0.smithi067.stdout:./268/.snap/snap-subdir-test 2024-03-09T08:56:25.742 INFO:tasks.workunit.client.1.smithi134.stdout:./270/.snap/snap-subdir-test 2024-03-09T08:56:25.749 INFO:tasks.workunit.client.0.smithi067.stdout:./189/.snap/snap-subdir-test 2024-03-09T08:56:25.751 INFO:tasks.workunit.client.1.smithi134.stdout:./125/.snap/snap-subdir-test 2024-03-09T08:56:25.762 INFO:tasks.workunit.client.1.smithi134.stdout:./194/.snap/snap-subdir-test 2024-03-09T08:56:25.768 INFO:tasks.workunit.client.0.smithi067.stdout:./285/.snap/snap-subdir-test 2024-03-09T08:56:25.769 INFO:tasks.workunit.client.1.smithi134.stdout:./119/.snap/snap-subdir-test 2024-03-09T08:56:25.778 INFO:tasks.workunit.client.1.smithi134.stdout:./297/.snap/snap-subdir-test 2024-03-09T08:56:25.784 INFO:tasks.workunit.client.0.smithi067.stdout:./208/.snap/snap-subdir-test 2024-03-09T08:56:25.789 INFO:tasks.workunit.client.1.smithi134.stdout:./230/.snap/snap-subdir-test 2024-03-09T08:56:25.794 INFO:tasks.workunit.client.0.smithi067.stdout:./288/.snap/snap-subdir-test 2024-03-09T08:56:25.799 INFO:tasks.workunit.client.1.smithi134.stdout:./260/.snap/snap-subdir-test 2024-03-09T08:56:25.802 INFO:tasks.workunit.client.0.smithi067.stdout:./209/.snap/snap-subdir-test 2024-03-09T08:56:25.812 INFO:tasks.workunit.client.1.smithi134.stdout:./157/.snap/snap-subdir-test 2024-03-09T08:56:25.813 INFO:tasks.workunit.client.0.smithi067.stdout:./253/.snap/snap-subdir-test 2024-03-09T08:56:25.824 INFO:tasks.workunit.client.0.smithi067.stdout:./249/.snap/snap-subdir-test 2024-03-09T08:56:25.834 INFO:tasks.workunit.client.0.smithi067.stdout:./133/.snap/snap-subdir-test 2024-03-09T08:56:25.845 INFO:tasks.workunit.client.0.smithi067.stdout:./291/.snap/snap-subdir-test 2024-03-09T08:56:25.856 INFO:tasks.workunit.client.0.smithi067.stdout:./226/.snap/snap-subdir-test 2024-03-09T08:56:25.867 INFO:tasks.workunit.client.0.smithi067.stdout:./173/.snap/snap-subdir-test 2024-03-09T08:56:25.879 INFO:tasks.workunit.client.0.smithi067.stdout:./174/.snap/snap-subdir-test 2024-03-09T08:56:26.319 INFO:tasks.workunit.client.1.smithi134.stdout:./172/.snap/snap-subdir-test 2024-03-09T08:56:26.335 INFO:tasks.workunit.client.1.smithi134.stdout:./277/.snap/snap-subdir-test 2024-03-09T08:56:26.352 INFO:tasks.workunit.client.1.smithi134.stdout:./186/.snap/snap-subdir-test 2024-03-09T08:56:26.367 INFO:tasks.workunit.client.1.smithi134.stdout:./237/.snap/snap-subdir-test 2024-03-09T08:56:26.384 INFO:tasks.workunit.client.1.smithi134.stdout:./190/.snap/snap-subdir-test 2024-03-09T08:56:26.400 INFO:tasks.workunit.client.1.smithi134.stdout:./213/.snap/snap-subdir-test 2024-03-09T08:56:26.416 INFO:tasks.workunit.client.1.smithi134.stdout:./104/.snap/snap-subdir-test 2024-03-09T08:56:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:./164/.snap/snap-subdir-test 2024-03-09T08:56:26.453 INFO:tasks.workunit.client.1.smithi134.stdout:./112/.snap/snap-subdir-test 2024-03-09T08:56:26.471 INFO:tasks.workunit.client.1.smithi134.stdout:./295/.snap/snap-subdir-test 2024-03-09T08:56:26.489 INFO:tasks.workunit.client.1.smithi134.stdout:./272/.snap/snap-subdir-test 2024-03-09T08:56:26.505 INFO:tasks.workunit.client.1.smithi134.stdout:./100/.snap/snap-subdir-test 2024-03-09T08:56:26.520 INFO:tasks.workunit.client.1.smithi134.stdout:./134/.snap/snap-subdir-test 2024-03-09T08:56:26.537 INFO:tasks.workunit.client.1.smithi134.stdout:./200/.snap/snap-subdir-test 2024-03-09T08:56:26.553 INFO:tasks.workunit.client.1.smithi134.stdout:./115/.snap/snap-subdir-test 2024-03-09T08:56:26.567 INFO:tasks.workunit.client.1.smithi134.stdout:./284/.snap/snap-subdir-test 2024-03-09T08:56:26.582 INFO:tasks.workunit.client.1.smithi134.stdout:./280/.snap/snap-subdir-test 2024-03-09T08:56:26.597 INFO:tasks.workunit.client.1.smithi134.stdout:./116/.snap/snap-subdir-test 2024-03-09T08:56:26.613 INFO:tasks.workunit.client.1.smithi134.stdout:./245/.snap/snap-subdir-test 2024-03-09T08:56:26.630 INFO:tasks.workunit.client.1.smithi134.stdout:./195/.snap/snap-subdir-test 2024-03-09T08:56:26.645 INFO:tasks.workunit.client.1.smithi134.stdout:./138/.snap/snap-subdir-test 2024-03-09T08:56:26.735 INFO:tasks.workunit.client.0.smithi067.stdout:./299/.snap/snap-subdir-test 2024-03-09T08:56:26.751 INFO:tasks.workunit.client.0.smithi067.stdout:./262/.snap/snap-subdir-test 2024-03-09T08:56:26.767 INFO:tasks.workunit.client.0.smithi067.stdout:./231/.snap/snap-subdir-test 2024-03-09T08:56:26.783 INFO:tasks.workunit.client.0.smithi067.stdout:./214/.snap/snap-subdir-test 2024-03-09T08:56:26.798 INFO:tasks.workunit.client.0.smithi067.stdout:./281/.snap/snap-subdir-test 2024-03-09T08:56:26.815 INFO:tasks.workunit.client.0.smithi067.stdout:./289/.snap/snap-subdir-test 2024-03-09T08:56:26.833 INFO:tasks.workunit.client.0.smithi067.stdout:./283/.snap/snap-subdir-test 2024-03-09T08:56:26.848 INFO:tasks.workunit.client.0.smithi067.stdout:./232/.snap/snap-subdir-test 2024-03-09T08:56:26.863 INFO:tasks.workunit.client.0.smithi067.stdout:./298/.snap/snap-subdir-test 2024-03-09T08:56:26.879 INFO:tasks.workunit.client.0.smithi067.stdout:./233/.snap/snap-subdir-test 2024-03-09T08:56:26.893 INFO:tasks.workunit.client.0.smithi067.stdout:./155/.snap/snap-subdir-test 2024-03-09T08:56:26.906 INFO:tasks.workunit.client.0.smithi067.stdout:./114/.snap/snap-subdir-test 2024-03-09T08:56:26.921 INFO:tasks.workunit.client.1.smithi134.stdout:./135/.snap/snap-subdir-test 2024-03-09T08:56:27.185 INFO:tasks.workunit.client.0.smithi067.stdout:./159/.snap/snap-subdir-test 2024-03-09T08:56:27.203 INFO:tasks.workunit.client.0.smithi067.stdout:./187/.snap/snap-subdir-test 2024-03-09T08:56:27.220 INFO:tasks.workunit.client.0.smithi067.stdout:./185/.snap/snap-subdir-test 2024-03-09T08:56:27.234 INFO:tasks.workunit.client.0.smithi067.stdout:./293/.snap/snap-subdir-test 2024-03-09T08:56:27.250 INFO:tasks.workunit.client.0.smithi067.stdout:./228/.snap/snap-subdir-test 2024-03-09T08:56:27.265 INFO:tasks.workunit.client.0.smithi067.stdout:./120/.snap/snap-subdir-test 2024-03-09T08:56:27.275 INFO:tasks.workunit.client.1.smithi134.stdout:./117/.snap/snap-subdir-test 2024-03-09T08:56:27.279 INFO:tasks.workunit.client.0.smithi067.stdout:./110/.snap/snap-subdir-test 2024-03-09T08:56:27.294 INFO:tasks.workunit.client.0.smithi067.stdout:./267/.snap/snap-subdir-test 2024-03-09T08:56:27.294 INFO:tasks.workunit.client.1.smithi134.stdout:./241/.snap/snap-subdir-test 2024-03-09T08:56:27.309 INFO:tasks.workunit.client.0.smithi067.stdout:./202/.snap/snap-subdir-test 2024-03-09T08:56:27.310 INFO:tasks.workunit.client.1.smithi134.stdout:./143/.snap/snap-subdir-test 2024-03-09T08:56:27.320 INFO:tasks.workunit.client.0.smithi067.stdout:./137/.snap/snap-subdir-test 2024-03-09T08:56:27.328 INFO:tasks.workunit.client.1.smithi134.stdout:./180/.snap/snap-subdir-test 2024-03-09T08:56:27.330 INFO:tasks.workunit.client.0.smithi067.stdout:./246/.snap/snap-subdir-test 2024-03-09T08:56:27.343 INFO:tasks.workunit.client.1.smithi134.stdout:./126/.snap/snap-subdir-test 2024-03-09T08:56:27.358 INFO:tasks.workunit.client.1.smithi134.stdout:./161/.snap/snap-subdir-test 2024-03-09T08:56:27.372 INFO:tasks.workunit.client.1.smithi134.stdout:./204/.snap/snap-subdir-test 2024-03-09T08:56:27.386 INFO:tasks.workunit.client.1.smithi134.stdout:./176/.snap/snap-subdir-test 2024-03-09T08:56:27.401 INFO:tasks.workunit.client.1.smithi134.stdout:./255/.snap/snap-subdir-test 2024-03-09T08:56:27.416 INFO:tasks.workunit.client.1.smithi134.stdout:./215/.snap/snap-subdir-test 2024-03-09T08:56:27.430 INFO:tasks.workunit.client.1.smithi134.stdout:./111/.snap/snap-subdir-test 2024-03-09T08:56:27.445 INFO:tasks.workunit.client.1.smithi134.stdout:./238/.snap/snap-subdir-test 2024-03-09T08:56:27.459 INFO:tasks.workunit.client.1.smithi134.stdout:./191/.snap/snap-subdir-test 2024-03-09T08:56:27.474 INFO:tasks.workunit.client.1.smithi134.stdout:./184/.snap/snap-subdir-test 2024-03-09T08:56:27.489 INFO:tasks.workunit.client.1.smithi134.stdout:./192/.snap/snap-subdir-test 2024-03-09T08:56:27.505 INFO:tasks.workunit.client.1.smithi134.stdout:./103/.snap/snap-subdir-test 2024-03-09T08:56:27.522 INFO:tasks.workunit.client.1.smithi134.stdout:./128/.snap/snap-subdir-test 2024-03-09T08:56:27.533 INFO:tasks.workunit.client.1.smithi134.stdout:./229/.snap/snap-subdir-test 2024-03-09T08:56:27.548 INFO:tasks.workunit.client.1.smithi134.stdout:./144/.snap/snap-subdir-test 2024-03-09T08:56:27.561 INFO:tasks.workunit.client.1.smithi134.stdout:./146/.snap/snap-subdir-test 2024-03-09T08:56:27.577 INFO:tasks.workunit.client.1.smithi134.stdout:./227/.snap/snap-subdir-test 2024-03-09T08:56:27.592 INFO:tasks.workunit.client.1.smithi134.stdout:./278/.snap/snap-subdir-test 2024-03-09T08:56:27.607 INFO:tasks.workunit.client.1.smithi134.stdout:./141/.snap/snap-subdir-test 2024-03-09T08:56:27.621 INFO:tasks.workunit.client.1.smithi134.stdout:./234/.snap/snap-subdir-test 2024-03-09T08:56:27.636 INFO:tasks.workunit.client.1.smithi134.stdout:./225/.snap/snap-subdir-test 2024-03-09T08:56:28.180 INFO:tasks.workunit.client.0.smithi067.stdout:./219/.snap/snap-subdir-test 2024-03-09T08:56:28.201 INFO:tasks.workunit.client.0.smithi067.stdout:./198/.snap/snap-subdir-test 2024-03-09T08:56:28.220 INFO:tasks.workunit.client.0.smithi067.stdout:./136/.snap/snap-subdir-test 2024-03-09T08:56:28.242 INFO:tasks.workunit.client.0.smithi067.stdout:./270/.snap/snap-subdir-test 2024-03-09T08:56:28.262 INFO:tasks.workunit.client.0.smithi067.stdout:./125/.snap/snap-subdir-test 2024-03-09T08:56:28.281 INFO:tasks.workunit.client.0.smithi067.stdout:./194/.snap/snap-subdir-test 2024-03-09T08:56:28.301 INFO:tasks.workunit.client.0.smithi067.stdout:./119/.snap/snap-subdir-test 2024-03-09T08:56:28.321 INFO:tasks.workunit.client.0.smithi067.stdout:./297/.snap/snap-subdir-test 2024-03-09T08:56:28.339 INFO:tasks.workunit.client.0.smithi067.stdout:./230/.snap/snap-subdir-test 2024-03-09T08:56:28.355 INFO:tasks.workunit.client.0.smithi067.stdout:./260/.snap/snap-subdir-test 2024-03-09T08:56:28.372 INFO:tasks.workunit.client.0.smithi067.stdout:./157/.snap/snap-subdir-test 2024-03-09T08:56:28.387 INFO:tasks.workunit.client.0.smithi067.stdout:./172/.snap/snap-subdir-test 2024-03-09T08:56:28.402 INFO:tasks.workunit.client.0.smithi067.stdout:./277/.snap/snap-subdir-test 2024-03-09T08:56:28.416 INFO:tasks.workunit.client.0.smithi067.stdout:./186/.snap/snap-subdir-test 2024-03-09T08:56:28.433 INFO:tasks.workunit.client.0.smithi067.stdout:./237/.snap/snap-subdir-test 2024-03-09T08:56:28.447 INFO:tasks.workunit.client.0.smithi067.stdout:./190/.snap/snap-subdir-test 2024-03-09T08:56:28.463 INFO:tasks.workunit.client.0.smithi067.stdout:./213/.snap/snap-subdir-test 2024-03-09T08:56:28.479 INFO:tasks.workunit.client.0.smithi067.stdout:./104/.snap/snap-subdir-test 2024-03-09T08:56:28.497 INFO:tasks.workunit.client.0.smithi067.stdout:./164/.snap/snap-subdir-test 2024-03-09T08:56:28.513 INFO:tasks.workunit.client.0.smithi067.stdout:./112/.snap/snap-subdir-test 2024-03-09T08:56:28.528 INFO:tasks.workunit.client.0.smithi067.stdout:./295/.snap/snap-subdir-test 2024-03-09T08:56:28.542 INFO:tasks.workunit.client.0.smithi067.stdout:./272/.snap/snap-subdir-test 2024-03-09T08:56:28.557 INFO:tasks.workunit.client.0.smithi067.stdout:./100/.snap/snap-subdir-test 2024-03-09T08:56:28.572 INFO:tasks.workunit.client.0.smithi067.stdout:./134/.snap/snap-subdir-test 2024-03-09T08:56:28.589 INFO:tasks.workunit.client.0.smithi067.stdout:./200/.snap/snap-subdir-test 2024-03-09T08:56:28.606 INFO:tasks.workunit.client.0.smithi067.stdout:./115/.snap/snap-subdir-test 2024-03-09T08:56:28.623 INFO:tasks.workunit.client.0.smithi067.stdout:./284/.snap/snap-subdir-test 2024-03-09T08:56:28.632 INFO:tasks.workunit.client.1.smithi134.stdout:./182/.snap/snap-subdir-test 2024-03-09T08:56:28.640 INFO:tasks.workunit.client.0.smithi067.stdout:./280/.snap/snap-subdir-test 2024-03-09T08:56:28.649 INFO:tasks.workunit.client.1.smithi134.stdout:./107/.snap/snap-subdir-test 2024-03-09T08:56:28.659 INFO:tasks.workunit.client.0.smithi067.stdout:./116/.snap/snap-subdir-test 2024-03-09T08:56:28.858 INFO:tasks.workunit.client.0.smithi067.stdout:./245/.snap/snap-subdir-test 2024-03-09T08:56:28.876 INFO:tasks.workunit.client.0.smithi067.stdout:./195/.snap/snap-subdir-test 2024-03-09T08:56:28.893 INFO:tasks.workunit.client.0.smithi067.stdout:./138/.snap/snap-subdir-test 2024-03-09T08:56:28.908 INFO:tasks.workunit.client.0.smithi067.stdout:./135/.snap/snap-subdir-test 2024-03-09T08:56:28.936 INFO:tasks.workunit.client.0.smithi067.stdout:./117/.snap/snap-subdir-test 2024-03-09T08:56:28.952 INFO:tasks.workunit.client.0.smithi067.stdout:./241/.snap/snap-subdir-test 2024-03-09T08:56:28.968 INFO:tasks.workunit.client.0.smithi067.stdout:./143/.snap/snap-subdir-test 2024-03-09T08:56:28.982 INFO:tasks.workunit.client.0.smithi067.stdout:./180/.snap/snap-subdir-test 2024-03-09T08:56:28.998 INFO:tasks.workunit.client.0.smithi067.stdout:./126/.snap/snap-subdir-test 2024-03-09T08:56:29.014 INFO:tasks.workunit.client.0.smithi067.stdout:./161/.snap/snap-subdir-test 2024-03-09T08:56:29.028 INFO:tasks.workunit.client.0.smithi067.stdout:./204/.snap/snap-subdir-test 2024-03-09T08:56:29.043 INFO:tasks.workunit.client.0.smithi067.stdout:./176/.snap/snap-subdir-test 2024-03-09T08:56:29.063 INFO:tasks.workunit.client.0.smithi067.stdout:./255/.snap/snap-subdir-test 2024-03-09T08:56:29.075 INFO:tasks.workunit.client.0.smithi067.stdout:./215/.snap/snap-subdir-test 2024-03-09T08:56:29.085 INFO:tasks.workunit.client.0.smithi067.stdout:./111/.snap/snap-subdir-test 2024-03-09T08:56:29.095 INFO:tasks.workunit.client.0.smithi067.stdout:./238/.snap/snap-subdir-test 2024-03-09T08:56:29.106 INFO:tasks.workunit.client.0.smithi067.stdout:./191/.snap/snap-subdir-test 2024-03-09T08:56:29.111 INFO:tasks.workunit.client.1.smithi134.stdout:./178/.snap/snap-subdir-test 2024-03-09T08:56:29.118 INFO:tasks.workunit.client.0.smithi067.stdout:./184/.snap/snap-subdir-test 2024-03-09T08:56:29.125 INFO:tasks.workunit.client.1.smithi134.stdout:./266/.snap/snap-subdir-test 2024-03-09T08:56:29.130 INFO:tasks.workunit.client.0.smithi067.stdout:./192/.snap/snap-subdir-test 2024-03-09T08:56:29.139 INFO:tasks.workunit.client.1.smithi134.stdout:./240/.snap/snap-subdir-test 2024-03-09T08:56:29.141 INFO:tasks.workunit.client.0.smithi067.stdout:./103/.snap/snap-subdir-test 2024-03-09T08:56:29.148 INFO:tasks.workunit.client.1.smithi134.stdout:./156/.snap/snap-subdir-test 2024-03-09T08:56:29.152 INFO:tasks.workunit.client.0.smithi067.stdout:./128/.snap/snap-subdir-test 2024-03-09T08:56:29.159 INFO:tasks.workunit.client.1.smithi134.stdout:./131/.snap/snap-subdir-test 2024-03-09T08:56:29.163 INFO:tasks.workunit.client.0.smithi067.stdout:./229/.snap/snap-subdir-test 2024-03-09T08:56:29.170 INFO:tasks.workunit.client.1.smithi134.stdout:./123/.snap/snap-subdir-test 2024-03-09T08:56:29.183 INFO:tasks.workunit.client.1.smithi134.stdout:./160/.snap/snap-subdir-test 2024-03-09T08:56:29.195 INFO:tasks.workunit.client.1.smithi134.stdout:./165/.snap/snap-subdir-test 2024-03-09T08:56:29.210 INFO:tasks.workunit.client.1.smithi134.stdout:./101/.snap/snap-subdir-test 2024-03-09T08:56:29.225 INFO:tasks.workunit.client.1.smithi134.stdout:./148/.snap/snap-subdir-test 2024-03-09T08:56:29.240 INFO:tasks.workunit.client.1.smithi134.stdout:./168/.snap/snap-subdir-test 2024-03-09T08:56:29.255 INFO:tasks.workunit.client.1.smithi134.stdout:./179/.snap/snap-subdir-test 2024-03-09T08:56:29.269 INFO:tasks.workunit.client.1.smithi134.stdout:./257/.snap/snap-subdir-test 2024-03-09T08:56:29.284 INFO:tasks.workunit.client.1.smithi134.stdout:./252/.snap/snap-subdir-test 2024-03-09T08:56:29.299 INFO:tasks.workunit.client.1.smithi134.stdout:./132/.snap/snap-subdir-test 2024-03-09T08:56:29.312 INFO:tasks.workunit.client.1.smithi134.stdout:./236/.snap/snap-subdir-test 2024-03-09T08:56:29.327 INFO:tasks.workunit.client.1.smithi134.stdout:./211/.snap/snap-subdir-test 2024-03-09T08:56:29.341 INFO:tasks.workunit.client.1.smithi134.stdout:./199/.snap/snap-subdir-test 2024-03-09T08:56:29.354 INFO:tasks.workunit.client.1.smithi134.stdout:./205/.snap/snap-subdir-test 2024-03-09T08:56:29.368 INFO:tasks.workunit.client.1.smithi134.stdout:./163/.snap/snap-subdir-test 2024-03-09T08:56:29.382 INFO:tasks.workunit.client.1.smithi134.stdout:./261/.snap/snap-subdir-test 2024-03-09T08:56:29.398 INFO:tasks.workunit.client.1.smithi134.stdout:./201/.snap/snap-subdir-test 2024-03-09T08:56:29.413 INFO:tasks.workunit.client.1.smithi134.stdout:./282/.snap/snap-subdir-test 2024-03-09T08:56:29.430 INFO:tasks.workunit.client.1.smithi134.stdout:./296/.snap/snap-subdir-test 2024-03-09T08:56:29.439 INFO:tasks.workunit.client.0.smithi067.stdout:./144/.snap/snap-subdir-test 2024-03-09T08:56:29.446 INFO:tasks.workunit.client.1.smithi134.stdout:./217/.snap/snap-subdir-test 2024-03-09T08:56:29.454 INFO:tasks.workunit.client.0.smithi067.stdout:./146/.snap/snap-subdir-test 2024-03-09T08:56:29.462 INFO:tasks.workunit.client.1.smithi134.stdout:./153/.snap/snap-subdir-test 2024-03-09T08:56:29.472 INFO:tasks.workunit.client.0.smithi067.stdout:./227/.snap/snap-subdir-test 2024-03-09T08:56:29.477 INFO:tasks.workunit.client.1.smithi134.stdout:./206/.snap/snap-subdir-test 2024-03-09T08:56:29.487 INFO:tasks.workunit.client.0.smithi067.stdout:./278/.snap/snap-subdir-test 2024-03-09T08:56:29.489 INFO:tasks.workunit.client.1.smithi134.stdout:./122/.snap/snap-subdir-test 2024-03-09T08:56:29.502 INFO:tasks.workunit.client.0.smithi067.stdout:./141/.snap/snap-subdir-test 2024-03-09T08:56:29.505 INFO:tasks.workunit.client.1.smithi134.stdout:./162/.snap/snap-subdir-test 2024-03-09T08:56:29.516 INFO:tasks.workunit.client.0.smithi067.stdout:./234/.snap/snap-subdir-test 2024-03-09T08:56:29.519 INFO:tasks.workunit.client.1.smithi134.stdout:./197/.snap/snap-subdir-test 2024-03-09T08:56:29.528 INFO:tasks.workunit.client.0.smithi067.stdout:./225/.snap/snap-subdir-test 2024-03-09T08:56:29.535 INFO:tasks.workunit.client.1.smithi134.stdout:./216/.snap/snap-subdir-test 2024-03-09T08:56:29.544 INFO:tasks.workunit.client.0.smithi067.stdout:./182/.snap/snap-subdir-test 2024-03-09T08:56:29.560 INFO:tasks.workunit.client.0.smithi067.stdout:./107/.snap/snap-subdir-test 2024-03-09T08:56:29.577 INFO:tasks.workunit.client.0.smithi067.stdout:./178/.snap/snap-subdir-test 2024-03-09T08:56:29.590 INFO:tasks.workunit.client.0.smithi067.stdout:./266/.snap/snap-subdir-test 2024-03-09T08:56:29.605 INFO:tasks.workunit.client.0.smithi067.stdout:./240/.snap/snap-subdir-test 2024-03-09T08:56:29.619 INFO:tasks.workunit.client.0.smithi067.stdout:./156/.snap/snap-subdir-test 2024-03-09T08:56:29.632 INFO:tasks.workunit.client.0.smithi067.stdout:./131/.snap/snap-subdir-test 2024-03-09T08:56:29.646 INFO:tasks.workunit.client.0.smithi067.stdout:./123/.snap/snap-subdir-test 2024-03-09T08:56:29.660 INFO:tasks.workunit.client.0.smithi067.stdout:./160/.snap/snap-subdir-test 2024-03-09T08:56:29.675 INFO:tasks.workunit.client.0.smithi067.stdout:./165/.snap/snap-subdir-test 2024-03-09T08:56:29.690 INFO:tasks.workunit.client.0.smithi067.stdout:./101/.snap/snap-subdir-test 2024-03-09T08:56:29.706 INFO:tasks.workunit.client.0.smithi067.stdout:./148/.snap/snap-subdir-test 2024-03-09T08:56:29.720 INFO:tasks.workunit.client.0.smithi067.stdout:./168/.snap/snap-subdir-test 2024-03-09T08:56:29.735 INFO:tasks.workunit.client.0.smithi067.stdout:./179/.snap/snap-subdir-test 2024-03-09T08:56:29.751 INFO:tasks.workunit.client.0.smithi067.stdout:./257/.snap/snap-subdir-test 2024-03-09T08:56:29.769 INFO:tasks.workunit.client.0.smithi067.stdout:./252/.snap/snap-subdir-test 2024-03-09T08:56:29.784 INFO:tasks.workunit.client.0.smithi067.stdout:./132/.snap/snap-subdir-test 2024-03-09T08:56:29.800 INFO:tasks.workunit.client.0.smithi067.stdout:./236/.snap/snap-subdir-test 2024-03-09T08:56:29.816 INFO:tasks.workunit.client.0.smithi067.stdout:./211/.snap/snap-subdir-test 2024-03-09T08:56:29.830 INFO:tasks.workunit.client.0.smithi067.stdout:./199/.snap/snap-subdir-test 2024-03-09T08:56:29.843 INFO:tasks.workunit.client.0.smithi067.stdout:./205/.snap/snap-subdir-test 2024-03-09T08:56:29.859 INFO:tasks.workunit.client.0.smithi067.stdout:./163/.snap/snap-subdir-test 2024-03-09T08:56:29.873 INFO:tasks.workunit.client.0.smithi067.stdout:./261/.snap/snap-subdir-test 2024-03-09T08:56:29.886 INFO:tasks.workunit.client.0.smithi067.stdout:./201/.snap/snap-subdir-test 2024-03-09T08:56:29.899 INFO:tasks.workunit.client.0.smithi067.stdout:./282/.snap/snap-subdir-test 2024-03-09T08:56:29.915 INFO:tasks.workunit.client.0.smithi067.stdout:./296/.snap/snap-subdir-test 2024-03-09T08:56:29.924 INFO:tasks.workunit.client.0.smithi067.stdout:./217/.snap/snap-subdir-test 2024-03-09T08:56:29.938 INFO:tasks.workunit.client.0.smithi067.stdout:./153/.snap/snap-subdir-test 2024-03-09T08:56:30.214 INFO:tasks.workunit.client.0.smithi067.stdout:./206/.snap/snap-subdir-test 2024-03-09T08:56:30.229 INFO:tasks.workunit.client.0.smithi067.stdout:./122/.snap/snap-subdir-test 2024-03-09T08:56:30.246 INFO:tasks.workunit.client.0.smithi067.stdout:./162/.snap/snap-subdir-test 2024-03-09T08:56:30.260 INFO:tasks.workunit.client.0.smithi067.stdout:./197/.snap/snap-subdir-test 2024-03-09T08:56:30.274 INFO:tasks.workunit.client.0.smithi067.stdout:./216/.snap/snap-subdir-test 2024-03-09T08:56:32.122 INFO:tasks.workunit.client.1.smithi134.stdout:./254/.snap/snap-subdir-test 2024-03-09T08:56:32.137 INFO:tasks.workunit.client.1.smithi134.stdout:./113/.snap/snap-subdir-test 2024-03-09T08:56:32.418 INFO:tasks.workunit.client.1.smithi134.stdout:./152/.snap/snap-subdir-test 2024-03-09T08:56:32.435 INFO:tasks.workunit.client.1.smithi134.stdout:./121/.snap/snap-subdir-test 2024-03-09T08:56:32.980 INFO:tasks.workunit.client.1.smithi134.stdout:./118/.snap/snap-subdir-test 2024-03-09T08:56:32.996 INFO:tasks.workunit.client.1.smithi134.stdout:./196/.snap/snap-subdir-test 2024-03-09T08:56:33.012 INFO:tasks.workunit.client.1.smithi134.stdout:./124/.snap/snap-subdir-test 2024-03-09T08:56:33.030 INFO:tasks.workunit.client.1.smithi134.stdout:./239/.snap/snap-subdir-test 2024-03-09T08:56:33.059 INFO:tasks.workunit.client.1.smithi134.stdout:./188/.snap/snap-subdir-test 2024-03-09T08:56:33.076 INFO:tasks.workunit.client.1.smithi134.stdout:./287/.snap/snap-subdir-test 2024-03-09T08:56:33.091 INFO:tasks.workunit.client.1.smithi134.stdout:./275/.snap/snap-subdir-test 2024-03-09T08:56:33.107 INFO:tasks.workunit.client.1.smithi134.stdout:./259/.snap/snap-subdir-test 2024-03-09T08:56:33.122 INFO:tasks.workunit.client.1.smithi134.stdout:./224/.snap/snap-subdir-test 2024-03-09T08:56:33.137 INFO:tasks.workunit.client.1.smithi134.stdout:./264/.snap/snap-subdir-test 2024-03-09T08:56:33.153 INFO:tasks.workunit.client.1.smithi134.stdout:./145/.snap/snap-subdir-test 2024-03-09T08:56:33.168 INFO:tasks.workunit.client.1.smithi134.stdout:./279/.snap/snap-subdir-test 2024-03-09T08:56:33.183 INFO:tasks.workunit.client.1.smithi134.stdout:./263/.snap/snap-subdir-test 2024-03-09T08:56:33.198 INFO:tasks.workunit.client.1.smithi134.stdout:./130/.snap/snap-subdir-test 2024-03-09T08:56:33.357 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T08:56:33.357 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T08:56:33.357 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T08:56:33.357 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T08:56:33.365 INFO:tasks.ceph.mon.c:Started 2024-03-09T08:56:33.365 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-03-09T08:56:33.366 INFO:tasks.ceph.mon.a:Sent signal 19 2024-03-09T08:56:33.366 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-03-09T08:56:33.920 INFO:tasks.workunit.client.1.smithi134.stdout:./193/.snap/snap-subdir-test 2024-03-09T08:56:40.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:56:40.990 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:56:48.367 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-03-09T08:56:48.367 INFO:tasks.ceph.mon.a:Sent signal 18 2024-03-09T08:56:48.368 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:56:48.368 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:56:48.412 INFO:tasks.workunit.client.1.smithi134.stdout:./223/.snap/snap-subdir-test 2024-03-09T08:56:48.427 INFO:tasks.workunit.client.1.smithi134.stdout:./207/.snap/snap-subdir-test 2024-03-09T08:56:48.438 INFO:tasks.workunit.client.1.smithi134.stdout:./276/.snap/snap-subdir-test 2024-03-09T08:56:48.448 INFO:tasks.workunit.client.1.smithi134.stdout:./271/.snap/snap-subdir-test 2024-03-09T08:56:48.458 INFO:tasks.workunit.client.1.smithi134.stdout:./251/.snap/snap-subdir-test 2024-03-09T08:56:48.470 INFO:tasks.workunit.client.1.smithi134.stdout:./175/.snap/snap-subdir-test 2024-03-09T08:56:48.480 INFO:tasks.workunit.client.1.smithi134.stdout:./222/.snap/snap-subdir-test 2024-03-09T08:56:48.492 INFO:tasks.workunit.client.1.smithi134.stdout:./220/.snap/snap-subdir-test 2024-03-09T08:56:48.505 INFO:tasks.workunit.client.1.smithi134.stdout:./203/.snap/snap-subdir-test 2024-03-09T08:56:48.519 INFO:tasks.workunit.client.1.smithi134.stdout:./181/.snap/snap-subdir-test 2024-03-09T08:56:48.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.593+0000 7f8a1f800640 1 Processor -- start 2024-03-09T08:56:48.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.593+0000 7f8a1f800640 1 -- start start 2024-03-09T08:56:48.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.594+0000 7f8a1f800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a180ac730 0x7f8a18071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:48.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.594+0000 7f8a1f800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a18157d60 0x7f8a180acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:48.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.594+0000 7f8a1f800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a18153d40 con 0x7f8a180c22a0 2024-03-09T08:56:48.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.594+0000 7f8a1f800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a18070ee0 con 0x7f8a18157d60 2024-03-09T08:56:48.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.594+0000 7f8a1f800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18154d00 con 0x7f8a180ac730 2024-03-09T08:56:48.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.594+0000 7f8a1d800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8a18157d60 0x7f8a180acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:48.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.594+0000 7f8a17e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8a180c22a0 0x7f8a18157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50306/0 (socket says 172.21.15.67:50306) 2024-03-09T08:56:48.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.594+0000 7f8a17e00640 1 -- 172.21.15.67:0/1816389701 learned_addr learned my addr 172.21.15.67:0/1816389701 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:56:48.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.595+0000 7f8a1ce00640 1 --2- 172.21.15.67:0/1816389701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a180ac730 0x7f8a18071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:48.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.595+0000 7f8a1d800640 1 -- 172.21.15.67:0/1816389701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a180ac730 msgr2=0x7f8a18071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:56:48.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.595+0000 7f8a1d800640 1 --2- 172.21.15.67:0/1816389701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a180ac730 0x7f8a18071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:48.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.595+0000 7f8a1d800640 1 -- 172.21.15.67:0/1816389701 >> v1:172.21.15.67:6789/0 conn(0x7f8a180c22a0 legacy=0x7f8a18157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:48.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.595+0000 7f8a1d800640 1 -- 172.21.15.67:0/1816389701 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a180c2790 con 0x7f8a18157d60 2024-03-09T08:56:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.597+0000 7f8a1d800640 1 --2- 172.21.15.67:0/1816389701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a18157d60 0x7f8a180acb10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009d90 tx=0x7f8a1002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6729d05ab7b63bb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.597+0000 7f8a17400640 1 -- 172.21.15.67:0/1816389701 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10039070 con 0x7f8a18157d60 2024-03-09T08:56:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.597+0000 7f8a17400640 1 -- 172.21.15.67:0/1816389701 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1002cbc0 con 0x7f8a18157d60 2024-03-09T08:56:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.597+0000 7f8a17400640 1 -- 172.21.15.67:0/1816389701 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10034440 con 0x7f8a18157d60 2024-03-09T08:56:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.598+0000 7f8a1f800640 1 -- 172.21.15.67:0/1816389701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a18157d60 msgr2=0x7f8a180acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.598+0000 7f8a1f800640 1 --2- 172.21.15.67:0/1816389701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a18157d60 0x7f8a180acb10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009d90 tx=0x7f8a1002be10 comp rx=0 tx=0).stop 2024-03-09T08:56:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.598+0000 7f8a1f800640 1 -- 172.21.15.67:0/1816389701 shutdown_connections 2024-03-09T08:56:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.598+0000 7f8a1f800640 1 --2- 172.21.15.67:0/1816389701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a18157d60 0x7f8a180acb10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.598+0000 7f8a1f800640 1 --2- 172.21.15.67:0/1816389701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a180ac730 0x7f8a18071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:48.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.598+0000 7f8a1f800640 1 -- 172.21.15.67:0/1816389701 >> 172.21.15.67:0/1816389701 conn(0x7f8a180c7370 msgr2=0x7f8a180c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:48.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.598+0000 7f8a1f800640 1 -- 172.21.15.67:0/1816389701 shutdown_connections 2024-03-09T08:56:48.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.598+0000 7f8a1f800640 1 -- 172.21.15.67:0/1816389701 wait complete. 2024-03-09T08:56:48.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.599+0000 7f8a1f800640 1 Processor -- start 2024-03-09T08:56:48.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.600+0000 7f8a1f800640 1 -- start start 2024-03-09T08:56:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.600+0000 7f8a1f800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a180ac730 0x7f8a18152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.600+0000 7f8a1f800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a180c22a0 0x7f8a18155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:48.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.600+0000 7f8a1f800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a18157d60 0x7f8a181de160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:48.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.600+0000 7f8a1f800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18151610 con 0x7f8a18157d60 2024-03-09T08:56:48.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.600+0000 7f8a1f800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18070c00 con 0x7f8a180ac730 2024-03-09T08:56:48.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.600+0000 7f8a1f800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18151790 con 0x7f8a180c22a0 2024-03-09T08:56:48.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.601+0000 7f8a1d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a18157d60 0x7f8a181de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:48.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.601+0000 7f8a1d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a18157d60 0x7f8a181de160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37804/0 (socket says 172.21.15.67:37804) 2024-03-09T08:56:48.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.601+0000 7f8a1d800640 1 -- 172.21.15.67:0/3125441628 learned_addr learned my addr 172.21.15.67:0/3125441628 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:56:48.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.601+0000 7f8a17e00640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a180c22a0 0x7f8a18155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:48.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.601+0000 7f8a1ce00640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a180ac730 0x7f8a18152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.601+0000 7f8a1d800640 1 -- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a180c22a0 msgr2=0x7f8a18155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.601+0000 7f8a1d800640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a180c22a0 0x7f8a18155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.601+0000 7f8a1d800640 1 -- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a180ac730 msgr2=0x7f8a18152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.601+0000 7f8a1d800640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a180ac730 0x7f8a18152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.601+0000 7f8a1d800640 1 -- 172.21.15.67:0/3125441628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a181564b0 con 0x7f8a18157d60 2024-03-09T08:56:48.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.602+0000 7f8a1d800640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a18157d60 0x7f8a181de160 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009eb0 tx=0x7f8a10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.602+0000 7f8a14c00640 1 -- 172.21.15.67:0/3125441628 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10039050 con 0x7f8a18157d60 2024-03-09T08:56:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.602+0000 7f8a14c00640 1 -- 172.21.15.67:0/3125441628 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a10045070 con 0x7f8a18157d60 2024-03-09T08:56:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.602+0000 7f8a14c00640 1 -- 172.21.15.67:0/3125441628 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1003e830 con 0x7f8a18157d60 2024-03-09T08:56:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.602+0000 7f8a1f800640 1 -- 172.21.15.67:0/3125441628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a18156790 con 0x7f8a18157d60 2024-03-09T08:56:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.602+0000 7f8a1f800640 1 -- 172.21.15.67:0/3125441628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a181dd960 con 0x7f8a18157d60 2024-03-09T08:56:48.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.604+0000 7f8a1f800640 1 -- 172.21.15.67:0/3125441628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a18075a00 con 0x7f8a18157d60 2024-03-09T08:56:48.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.605+0000 7f8a14c00640 1 -- 172.21.15.67:0/3125441628 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8a1003e9d0 con 0x7f8a18157d60 2024-03-09T08:56:48.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.605+0000 7f8a14c00640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f89f806b4d0 0x7f89f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:48.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.606+0000 7f8a14c00640 1 -- 172.21.15.67:0/3125441628 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f8a100ad690 con 0x7f8a18157d60 2024-03-09T08:56:48.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.606+0000 7f8a1ce00640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f89f806b4d0 0x7f89f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:48.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.607+0000 7f8a1ce00640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f89f806b4d0 0x7f89f806d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c006fd0 tx=0x7f8a0c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:48.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.611+0000 7f8a14c00640 1 -- 172.21.15.67:0/3125441628 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8a1007c1d0 con 0x7f8a18157d60 2024-03-09T08:56:48.797 INFO:tasks.workunit.client.1.smithi134.stdout:./286/.snap/snap-subdir-test 2024-03-09T08:56:48.811 INFO:tasks.workunit.client.1.smithi134.stdout:./127/.snap/snap-subdir-test 2024-03-09T08:56:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:./108/.snap/snap-subdir-test 2024-03-09T08:56:48.837 INFO:tasks.workunit.client.1.smithi134.stdout:./183/.snap/snap-subdir-test 2024-03-09T08:56:48.853 INFO:tasks.workunit.client.1.smithi134.stdout:./294/.snap/snap-subdir-test 2024-03-09T08:56:48.872 INFO:tasks.workunit.client.1.smithi134.stdout:./235/.snap/snap-subdir-test 2024-03-09T08:56:48.889 INFO:tasks.workunit.client.1.smithi134.stdout:./177/.snap/snap-subdir-test 2024-03-09T08:56:48.904 INFO:tasks.workunit.client.1.smithi134.stdout:./169/.snap/snap-subdir-test 2024-03-09T08:56:48.917 INFO:tasks.workunit.client.1.smithi134.stdout:./212/.snap/snap-subdir-test 2024-03-09T08:56:48.931 INFO:tasks.workunit.client.1.smithi134.stdout:./154/.snap/snap-subdir-test 2024-03-09T08:56:48.946 INFO:tasks.workunit.client.1.smithi134.stdout:./171/.snap/snap-subdir-test 2024-03-09T08:56:48.962 INFO:tasks.workunit.client.1.smithi134.stdout:./139/.snap/snap-subdir-test 2024-03-09T08:56:48.975 INFO:tasks.workunit.client.1.smithi134.stdout:./102/.snap/snap-subdir-test 2024-03-09T08:56:48.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.979+0000 7f8a1f800640 1 -- 172.21.15.67:0/3125441628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8a180c1d70 con 0x7f8a18157d60 2024-03-09T08:56:48.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.979+0000 7f8a14c00640 1 -- 172.21.15.67:0/3125441628 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7f8a1007bad0 con 0x7f8a18157d60 2024-03-09T08:56:48.978 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:56:48.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 -- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f89f806b4d0 msgr2=0x7f89f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:48.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f89f806b4d0 0x7f89f806d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c006fd0 tx=0x7f8a0c008040 comp rx=0 tx=0).stop 2024-03-09T08:56:48.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 -- 172.21.15.67:0/3125441628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a18157d60 msgr2=0x7f8a181de160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:48.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a18157d60 0x7f8a181de160 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009eb0 tx=0x7f8a10004a40 comp rx=0 tx=0).stop 2024-03-09T08:56:48.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 -- 172.21.15.67:0/3125441628 shutdown_connections 2024-03-09T08:56:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f89f806b4d0 0x7f89f806d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a18157d60 0x7f8a181de160 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a180c22a0 0x7f8a18155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 --2- 172.21.15.67:0/3125441628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a180ac730 0x7f8a18152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 -- 172.21.15.67:0/3125441628 >> 172.21.15.67:0/3125441628 conn(0x7f8a180c7370 msgr2=0x7f8a180c45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 -- 172.21.15.67:0/3125441628 shutdown_connections 2024-03-09T08:56:48.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:48.982+0000 7f8a1f800640 1 -- 172.21.15.67:0/3125441628 wait complete. 2024-03-09T08:56:48.985 INFO:tasks.workunit.client.1.smithi134.stdout:./129/.snap/snap-subdir-test 2024-03-09T08:56:49.001 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:56:49.001 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:56:49.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.244+0000 7f0aee200640 1 Processor -- start 2024-03-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.244+0000 7f0aee200640 1 -- start start 2024-03-09T08:56:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.245+0000 7f0aee200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0ae80ac930 0x7f0ae80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.245+0000 7f0aee200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ae80c30a0 0x7f0ae8150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.245+0000 7f0aee200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae80c2780 con 0x7f0ae8151190 2024-03-09T08:56:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.245+0000 7f0aee200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae8070670 con 0x7f0ae80ac930 2024-03-09T08:56:49.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.245+0000 7f0aee200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80708c0 con 0x7f0ae80c30a0 2024-03-09T08:56:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.245+0000 7f0aece00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0ae80ac930 0x7f0ae80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.245+0000 7f0aed800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0ae8151190 0x7f0ae81539e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53988/0 (socket says 172.21.15.67:53988) 2024-03-09T08:56:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.245+0000 7f0aed800640 1 -- 172.21.15.67:0/2790212993 learned_addr learned my addr 172.21.15.67:0/2790212993 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:56:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.245+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/2790212993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ae80c30a0 0x7f0ae8150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.246+0000 7f0ae7400640 1 -- 172.21.15.67:0/2790212993 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454758484 0 0) 0x7f0ae80c2780 con 0x7f0ae8151190 2024-03-09T08:56:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.246+0000 7f0ae7400640 1 -- 172.21.15.67:0/2790212993 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ad0003520 con 0x7f0ae8151190 2024-03-09T08:56:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.246+0000 7f0ae7e00640 1 -- 172.21.15.67:0/2790212993 >> v2:172.21.15.67:3300/0 conn(0x7f0ae80ac930 msgr2=0x7f0ae80c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:56:49.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.246+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/2790212993 >> v2:172.21.15.67:3300/0 conn(0x7f0ae80ac930 0x7f0ae80c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.246+0000 7f0ae7e00640 1 -- 172.21.15.67:0/2790212993 >> v1:172.21.15.67:6789/0 conn(0x7f0ae8151190 legacy=0x7f0ae81539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.246+0000 7f0ae7e00640 1 -- 172.21.15.67:0/2790212993 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae8070ba0 con 0x7f0ae80c30a0 2024-03-09T08:56:49.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.248+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/2790212993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ae80c30a0 0x7f0ae8150c50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40097a0 tx=0x7f0ad402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81934cbcd4daf471 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.248+0000 7f0ae7400640 1 -- 172.21.15.67:0/2790212993 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4035020 con 0x7f0ae80c30a0 2024-03-09T08:56:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.248+0000 7f0ae7400640 1 -- 172.21.15.67:0/2790212993 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad4031810 con 0x7f0ae80c30a0 2024-03-09T08:56:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.248+0000 7f0ae7400640 1 -- 172.21.15.67:0/2790212993 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4031bc0 con 0x7f0ae80c30a0 2024-03-09T08:56:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.249+0000 7f0aee200640 1 -- 172.21.15.67:0/2790212993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ae80c30a0 msgr2=0x7f0ae8150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.249+0000 7f0aee200640 1 --2- 172.21.15.67:0/2790212993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ae80c30a0 0x7f0ae8150c50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40097a0 tx=0x7f0ad402b8b0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.249+0000 7f0aee200640 1 -- 172.21.15.67:0/2790212993 shutdown_connections 2024-03-09T08:56:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.249+0000 7f0aee200640 1 --2- 172.21.15.67:0/2790212993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ae80c30a0 0x7f0ae8150c50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.249+0000 7f0aee200640 1 --2- 172.21.15.67:0/2790212993 >> v2:172.21.15.67:3300/0 conn(0x7f0ae80ac930 0x7f0ae80c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.249+0000 7f0aee200640 1 -- 172.21.15.67:0/2790212993 >> 172.21.15.67:0/2790212993 conn(0x7f0ae80c7220 msgr2=0x7f0ae80c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.249+0000 7f0aee200640 1 -- 172.21.15.67:0/2790212993 shutdown_connections 2024-03-09T08:56:49.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.249+0000 7f0aee200640 1 -- 172.21.15.67:0/2790212993 wait complete. 2024-03-09T08:56:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.250+0000 7f0aee200640 1 Processor -- start 2024-03-09T08:56:49.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.251+0000 7f0aee200640 1 -- start start 2024-03-09T08:56:49.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.251+0000 7f0aee200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ae80ac930 0x7f0ae81eca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.251+0000 7f0aee200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae80c30a0 0x7f0ae81567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0aee200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8151190 0x7f0ae8156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0aee200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80acd10 con 0x7f0ae80c30a0 2024-03-09T08:56:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0aee200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae81515e0 con 0x7f0ae8151190 2024-03-09T08:56:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0aee200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80ace90 con 0x7f0ae80ac930 2024-03-09T08:56:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0ae7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae80c30a0 0x7f0ae81567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0ae7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae80c30a0 0x7f0ae81567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35952/0 (socket says 172.21.15.67:35952) 2024-03-09T08:56:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0ae7e00640 1 -- 172.21.15.67:0/503484332 learned_addr learned my addr 172.21.15.67:0/503484332 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:56:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0aece00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ae80ac930 0x7f0ae81eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0aed800640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8151190 0x7f0ae8156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0ae7e00640 1 -- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ae80ac930 msgr2=0x7f0ae81eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ae80ac930 0x7f0ae81eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0ae7e00640 1 -- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8151190 msgr2=0x7f0ae8156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8151190 0x7f0ae8156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0ae7e00640 1 -- 172.21.15.67:0/503484332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae8151dc0 con 0x7f0ae80c30a0 2024-03-09T08:56:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.252+0000 7f0aed800640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ae8151190 0x7f0ae8156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:56:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.253+0000 7f0ae7e00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae80c30a0 0x7f0ae81567d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0ad402b8f0 tx=0x7f0ad40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.253+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4040070 con 0x7f0ae80c30a0 2024-03-09T08:56:49.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.253+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad4039cf0 con 0x7f0ae80c30a0 2024-03-09T08:56:49.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.253+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4035040 con 0x7f0ae80c30a0 2024-03-09T08:56:49.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.253+0000 7f0aee200640 1 -- 172.21.15.67:0/503484332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae81464a0 con 0x7f0ae80c30a0 2024-03-09T08:56:49.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.254+0000 7f0aee200640 1 -- 172.21.15.67:0/503484332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae8146cf0 con 0x7f0ae80c30a0 2024-03-09T08:56:49.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.255+0000 7f0aee200640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae814bdf0 0x7f0ae81ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.255+0000 7f0aee200640 1 -- 172.21.15.67:0/503484332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae8151c40 con 0x7f0ae814bdf0 2024-03-09T08:56:49.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.255+0000 7f0aece00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae814bdf0 0x7f0ae81ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.255+0000 7f0aece00640 1 -- 172.21.15.67:0/503484332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ae8146250 con 0x7f0ae814bdf0 2024-03-09T08:56:49.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.256+0000 7f0aece00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae814bdf0 0x7f0ae81ecf60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0adc011860 tx=0x7f0adc011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.256+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc019070 con 0x7f0ae814bdf0 2024-03-09T08:56:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.256+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f0adc004240 con 0x7f0ae814bdf0 2024-03-09T08:56:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.256+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae814bdf0 msgr2=0x7f0ae81ecf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.256+0000 7f0ae4c00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae814bdf0 0x7f0ae81ecf60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0adc011860 tx=0x7f0adc011d20 comp rx=0 tx=0).stop 2024-03-09T08:56:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.256+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae814bdf0 msgr2=0x7f0ae81ecf60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:56:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.256+0000 7f0ae4c00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae814bdf0 0x7f0ae81ecf60 secure :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0adc011860 tx=0x7f0adc011d20 comp rx=0 tx=0).stop 2024-03-09T08:56:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.256+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0ad404b060 con 0x7f0ae80c30a0 2024-03-09T08:56:49.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.257+0000 7f0ae4c00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ac806b140 0x7f0ac806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.257+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f0ad40abee0 con 0x7f0ae80c30a0 2024-03-09T08:56:49.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.258+0000 7f0aece00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ac806b140 0x7f0ac806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.259+0000 7f0aece00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ac806b140 0x7f0ac806d600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00f6b0 tx=0x7f0adc008070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.264+0000 7f0aee200640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae8146780 0x7f0ae8154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.264+0000 7f0aee200640 1 -- 172.21.15.67:0/503484332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae81ed6e0 con 0x7f0ae8146780 2024-03-09T08:56:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.264+0000 7f0aed800640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae8146780 0x7f0ae8154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.265+0000 7f0aed800640 1 -- 172.21.15.67:0/503484332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0ae806e620 con 0x7f0ae8146780 2024-03-09T08:56:49.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.265+0000 7f0aed800640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae8146780 0x7f0ae8154170 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00012e0 tx=0x7f0ae000c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.265+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000cdf0 con 0x7f0ae8146780 2024-03-09T08:56:49.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.265+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f0ae000cdf0 con 0x7f0ae8146780 2024-03-09T08:56:49.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.265+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae8146780 msgr2=0x7f0ae8154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.265+0000 7f0ae4c00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae8146780 0x7f0ae8154170 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00012e0 tx=0x7f0ae000c4b0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.265+0000 7f0ae4c00640 1 -- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae8146780 msgr2=0x7f0ae8154170 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:56:49.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.265+0000 7f0ae4c00640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae8146780 0x7f0ae8154170 secure :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00012e0 tx=0x7f0ae000c4b0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.265 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:56:49.265 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:56:49.266 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:56:49.266 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:56:49.266 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 36, 2024-03-09T08:56:49.266 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:56:49.266 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:56:49.266 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:56:49.266 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:56:49.266 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.266 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:56:49.266 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:56:49.266 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:49.267 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:49.268 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.268 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:56:49.268 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:56:49.268 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:49.268 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:49.268 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:49.268 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:49.268 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:49.268 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:49.269 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:49.269 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:49.269 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:49.269 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:49.269 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:49.269 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.269 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.269 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:56:49.270 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:56:49.270 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:56:49.270 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:56:49.270 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:56:49.270 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:56:49.270 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:49.271 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.272 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.273 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:56:49.274 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.276 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:56:49.276 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:56:49.276 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:49.276 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:49.276 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:49.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:49.277 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.278 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:56:49.279 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:56:49.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.279 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:49.279 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.279 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:49.279 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.279 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.279 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:56:49.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.279 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:49.279 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:56:49.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.281 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:56:49.281 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.281 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:56:49.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.281 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.281 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:49.281 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.281 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:56:49.281 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.281 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.281 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:56:49.282 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.282 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:49.282 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.282 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:49.282 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.282 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.282 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:56:49.282 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:56:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.266+0000 7f0aee200640 1 -- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ac806b140 msgr2=0x7f0ac806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.266+0000 7f0aee200640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ac806b140 0x7f0ac806d600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00f6b0 tx=0x7f0adc008070 comp rx=0 tx=0).stop 2024-03-09T08:56:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.266+0000 7f0aee200640 1 -- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae80c30a0 msgr2=0x7f0ae81567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.266+0000 7f0aee200640 1 --2- 172.21.15.67:0/503484332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae80c30a0 0x7f0ae81567d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0ad402b8f0 tx=0x7f0ad40046c0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.266+0000 7f0aed800640 1 -- 172.21.15.67:0/503484332 reap_dead start 2024-03-09T08:56:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.266+0000 7f0aee200640 1 -- 172.21.15.67:0/503484332 shutdown_connections 2024-03-09T08:56:49.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.266+0000 7f0aee200640 1 -- 172.21.15.67:0/503484332 >> 172.21.15.67:0/503484332 conn(0x7f0ae80c7220 msgr2=0x7f0ae8149d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:49.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.266+0000 7f0aee200640 1 -- 172.21.15.67:0/503484332 shutdown_connections 2024-03-09T08:56:49.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.266+0000 7f0aee200640 1 -- 172.21.15.67:0/503484332 wait complete. 2024-03-09T08:56:49.285 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:56:49.285 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:56:49.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:56:49.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.507+0000 7f2c49e00640 1 Processor -- start 2024-03-09T08:56:49.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.507+0000 7f2c49e00640 1 -- start start 2024-03-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.508+0000 7f2c49e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2c44153460 0x7f2c44155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.508+0000 7f2c49e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c440caa20 0x7f2c440cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.508+0000 7f2c49e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4406b040 con 0x7f2c440c8c60 2024-03-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.508+0000 7f2c49e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c440ca590 con 0x7f2c44153460 2024-03-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.508+0000 7f2c49e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c44156710 con 0x7f2c440caa20 2024-03-09T08:56:49.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.509+0000 7f2c3be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2c440c8c60 0x7f2c440c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54004/0 (socket says 172.21.15.67:54004) 2024-03-09T08:56:49.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.509+0000 7f2c3be00640 1 -- 172.21.15.67:0/4211197150 learned_addr learned my addr 172.21.15.67:0/4211197150 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:56:49.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.509+0000 7f2c42a00640 1 --2- 172.21.15.67:0/4211197150 >> v2:172.21.15.134:3300/0 conn(0x7f2c44153460 0x7f2c44155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.509+0000 7f2c43400640 1 --2- 172.21.15.67:0/4211197150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c440caa20 0x7f2c440cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.509+0000 7f2c42000640 1 -- 172.21.15.67:0/4211197150 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182436578 0 0) 0x7f2c2c001110 con 0x7f2c440c8c60 2024-03-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.509+0000 7f2c42000640 1 -- 172.21.15.67:0/4211197150 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c28003520 con 0x7f2c440c8c60 2024-03-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.509+0000 7f2c43400640 1 -- 172.21.15.67:0/4211197150 >> v2:172.21.15.134:3300/0 conn(0x7f2c44153460 msgr2=0x7f2c44155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.509+0000 7f2c43400640 1 --2- 172.21.15.67:0/4211197150 >> v2:172.21.15.134:3300/0 conn(0x7f2c44153460 0x7f2c44155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.509+0000 7f2c43400640 1 -- 172.21.15.67:0/4211197150 >> v1:172.21.15.67:6789/0 conn(0x7f2c440c8c60 legacy=0x7f2c440c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.509+0000 7f2c43400640 1 -- 172.21.15.67:0/4211197150 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c441569f0 con 0x7f2c440caa20 2024-03-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.511+0000 7f2c43400640 1 --2- 172.21.15.67:0/4211197150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c440caa20 0x7f2c440cae90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009d90 tx=0x7f2c3c02bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a34acd067aaff14 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.511+0000 7f2c42000640 1 -- 172.21.15.67:0/4211197150 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c039070 con 0x7f2c440caa20 2024-03-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.511+0000 7f2c42000640 1 -- 172.21.15.67:0/4211197150 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c02c740 con 0x7f2c440caa20 2024-03-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.511+0000 7f2c42000640 1 -- 172.21.15.67:0/4211197150 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c001840 con 0x7f2c440caa20 2024-03-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.512+0000 7f2c49e00640 1 -- 172.21.15.67:0/4211197150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c440caa20 msgr2=0x7f2c440cae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.512+0000 7f2c49e00640 1 --2- 172.21.15.67:0/4211197150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c440caa20 0x7f2c440cae90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009d90 tx=0x7f2c3c02bc10 comp rx=0 tx=0).stop 2024-03-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.512+0000 7f2c49e00640 1 -- 172.21.15.67:0/4211197150 shutdown_connections 2024-03-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.512+0000 7f2c49e00640 1 --2- 172.21.15.67:0/4211197150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c440caa20 0x7f2c440cae90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.512+0000 7f2c49e00640 1 --2- 172.21.15.67:0/4211197150 >> v2:172.21.15.134:3300/0 conn(0x7f2c44153460 0x7f2c44155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.512+0000 7f2c49e00640 1 -- 172.21.15.67:0/4211197150 >> 172.21.15.67:0/4211197150 conn(0x7f2c440c8170 msgr2=0x7f2c440c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.512+0000 7f2c49e00640 1 -- 172.21.15.67:0/4211197150 shutdown_connections 2024-03-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.512+0000 7f2c49e00640 1 -- 172.21.15.67:0/4211197150 wait complete. 2024-03-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.514+0000 7f2c49e00640 1 Processor -- start 2024-03-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.514+0000 7f2c49e00640 1 -- start start 2024-03-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.515+0000 7f2c49e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c440c8c60 0x7f2c440c4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.515+0000 7f2c49e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c440caa20 0x7f2c441e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.515+0000 7f2c49e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c44153460 0x7f2c440c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.515+0000 7f2c49e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c44153840 con 0x7f2c44153460 2024-03-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.515+0000 7f2c49e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c44156400 con 0x7f2c440caa20 2024-03-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.515+0000 7f2c49e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c440c9f20 con 0x7f2c440c8c60 2024-03-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.515+0000 7f2c43400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c44153460 0x7f2c440c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.515+0000 7f2c43400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c44153460 0x7f2c440c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35982/0 (socket says 172.21.15.67:35982) 2024-03-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.515+0000 7f2c43400640 1 -- 172.21.15.67:0/708958195 learned_addr learned my addr 172.21.15.67:0/708958195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.516+0000 7f2c42a00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c440c8c60 0x7f2c440c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.516+0000 7f2c3be00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c440caa20 0x7f2c441e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.516+0000 7f2c43400640 1 -- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c440c8c60 msgr2=0x7f2c440c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.516+0000 7f2c43400640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c440c8c60 0x7f2c440c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.516+0000 7f2c43400640 1 -- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c440caa20 msgr2=0x7f2c441e2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.516+0000 7f2c43400640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c440caa20 0x7f2c441e2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.516+0000 7f2c43400640 1 -- 172.21.15.67:0/708958195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c441573a0 con 0x7f2c44153460 2024-03-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.516+0000 7f2c43400640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c44153460 0x7f2c440c2b90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c02bc50 tx=0x7f2c3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.517+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c039050 con 0x7f2c44153460 2024-03-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.517+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c045070 con 0x7f2c44153460 2024-03-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.517+0000 7f2c49e00640 1 -- 172.21.15.67:0/708958195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c44151970 con 0x7f2c44153460 2024-03-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.517+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c009450 con 0x7f2c44153460 2024-03-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.517+0000 7f2c49e00640 1 -- 172.21.15.67:0/708958195 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c44151f30 con 0x7f2c44153460 2024-03-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.518+0000 7f2c49e00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c440c2730 0x7f2c4414c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.518+0000 7f2c49e00640 1 -- 172.21.15.67:0/708958195 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c440c4840 con 0x7f2c440c2730 2024-03-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.518+0000 7f2c42a00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c440c2730 0x7f2c4414c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.519+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2c3c0046b0 con 0x7f2c44153460 2024-03-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.519+0000 7f2c42a00640 1 -- 172.21.15.67:0/708958195 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c44075a00 con 0x7f2c440c2730 2024-03-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.520+0000 7f2c42a00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c440c2730 0x7f2c4414c880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c34011860 tx=0x7f2c34011d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.520+0000 7f2c3b400640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c2006b4d0 0x7f2c2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.520+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f2c3c0aca70 con 0x7f2c44153460 2024-03-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.520+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34019070 con 0x7f2c440c2730 2024-03-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.521+0000 7f2c3be00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c2006b4d0 0x7f2c2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.521+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f2c34004240 con 0x7f2c440c2730 2024-03-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.521+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c440c2730 msgr2=0x7f2c4414c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.521+0000 7f2c3b400640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c440c2730 0x7f2c4414c880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c34011860 tx=0x7f2c34011d20 comp rx=0 tx=0).stop 2024-03-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.521+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c440c2730 msgr2=0x7f2c4414c880 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.521+0000 7f2c3b400640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c440c2730 0x7f2c4414c880 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.522+0000 7f2c3be00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c2006b4d0 0x7f2c2006d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c001eb0 tx=0x7f2c2c001d40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.529+0000 7f2c49e00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c441e3180 0x7f2c440c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.529+0000 7f2c49e00640 1 -- 172.21.15.67:0/708958195 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c440752b0 con 0x7f2c441e3180 2024-03-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.529+0000 7f2c42a00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c441e3180 0x7f2c440c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.530+0000 7f2c42a00640 1 -- 172.21.15.67:0/708958195 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2c440c95f0 con 0x7f2c441e3180 2024-03-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.530+0000 7f2c42a00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c441e3180 0x7f2c440c30d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c34011860 tx=0x7f2c340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.531+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34019040 con 0x7f2c441e3180 2024-03-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4873 (secure 0 0 0) 0x7f2c3401e450 con 0x7f2c441e3180 2024-03-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c441e3180 msgr2=0x7f2c440c30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c3b400640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c441e3180 0x7f2c440c30d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c34011860 tx=0x7f2c340046c0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c3b400640 1 -- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c441e3180 msgr2=0x7f2c440c30d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c3b400640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c441e3180 0x7f2c440c30d0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c34011860 tx=0x7f2c340046c0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 36, 2024-03-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c49e00640 1 -- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c2006b4d0 msgr2=0x7f2c2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c49e00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c2006b4d0 0x7f2c2006d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c001eb0 tx=0x7f2c2c001d40 comp rx=0 tx=0).stop 2024-03-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c49e00640 1 -- 172.21.15.67:0/708958195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c44153460 msgr2=0x7f2c440c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c49e00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c44153460 0x7f2c440c2b90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c02bc50 tx=0x7f2c3c004a40 comp rx=0 tx=0).stop 2024-03-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c43400640 1 -- 172.21.15.67:0/708958195 reap_dead start 2024-03-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c49e00640 1 -- 172.21.15.67:0/708958195 shutdown_connections 2024-03-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c49e00640 1 --2- 172.21.15.67:0/708958195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c44153460 0x7f2c440c2b90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.532+0000 7f2c49e00640 1 -- 172.21.15.67:0/708958195 >> 172.21.15.67:0/708958195 conn(0x7f2c440c8170 msgr2=0x7f2c440c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.533+0000 7f2c49e00640 1 -- 172.21.15.67:0/708958195 shutdown_connections 2024-03-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.533+0000 7f2c49e00640 1 -- 172.21.15.67:0/708958195 wait complete. 2024-03-09T08:56:49.549 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:56:49.549 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:56:49.550 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:56:49.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.747+0000 7fd393200640 1 Processor -- start 2024-03-09T08:56:49.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.748+0000 7fd393200640 1 -- start start 2024-03-09T08:56:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.749+0000 7fd393200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd39414a470 0x7fd394154330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.749+0000 7fd393200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3940c3060 con 0x7fd39406b4b0 2024-03-09T08:56:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.749+0000 7fd393200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3940c3340 con 0x7fd39406b040 2024-03-09T08:56:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.749+0000 7fd393200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3940c3620 con 0x7fd39414a470 2024-03-09T08:56:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.749+0000 7fd392800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd39406b4b0 0x7fd3941555e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54008/0 (socket says 172.21.15.67:54008) 2024-03-09T08:56:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.749+0000 7fd392800640 1 -- 172.21.15.67:0/3064219193 learned_addr learned my addr 172.21.15.67:0/3064219193 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:56:49.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391e00640 1 --2- 172.21.15.67:0/3064219193 >> v2:172.21.15.134:3300/0 conn(0x7fd39414a470 0x7fd394154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404985831 0 0) 0x7fd3940c3060 con 0x7fd39406b4b0 2024-03-09T08:56:49.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd378003520 con 0x7fd39406b4b0 2024-03-09T08:56:49.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3006687727 0 0) 0x7fd378003520 con 0x7fd39406b4b0 2024-03-09T08:56:49.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 >> v2:172.21.15.134:3300/0 conn(0x7fd39414a470 msgr2=0x7fd394154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 --2- 172.21.15.67:0/3064219193 >> v2:172.21.15.134:3300/0 conn(0x7fd39414a470 0x7fd394154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 >> v1:172.21.15.134:6789/0 conn(0x7fd39406b040 legacy=0x7fd394154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3940c3900 con 0x7fd39406b4b0 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fd38c0019f0 con 0x7fd39406b4b0 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 >> v1:172.21.15.67:6789/0 conn(0x7fd39406b4b0 legacy=0x7fd3941555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 --2- 172.21.15.67:0/3064219193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd378003a70 0x7fd378003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 --2- 172.21.15.67:0/3064219193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd378004800 0x7fd378004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 --2- 172.21.15.67:0/3064219193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd378005210 0x7fd37800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940c3620 con 0x7fd378004800 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37800c140 con 0x7fd378003a70 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.750+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37800c340 con 0x7fd378005210 2024-03-09T08:56:49.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.751+0000 7fd38be00640 1 --2- 172.21.15.67:0/3064219193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd378004800 0x7fd378004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.751+0000 7fd392800640 1 --2- 172.21.15.67:0/3064219193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd378005210 0x7fd37800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.751+0000 7fd391e00640 1 --2- 172.21.15.67:0/3064219193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd378003a70 0x7fd378003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.751+0000 7fd38be00640 1 -- 172.21.15.67:0/3064219193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd378005210 msgr2=0x7fd37800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:56:49.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.751+0000 7fd38be00640 1 --2- 172.21.15.67:0/3064219193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd378005210 0x7fd37800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.751+0000 7fd38be00640 1 -- 172.21.15.67:0/3064219193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd378003a70 msgr2=0x7fd378003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:56:49.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.751+0000 7fd38be00640 1 --2- 172.21.15.67:0/3064219193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd378003a70 0x7fd378003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.751+0000 7fd38be00640 1 -- 172.21.15.67:0/3064219193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd378003520 con 0x7fd378004800 2024-03-09T08:56:49.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.751+0000 7fd392800640 1 -- 172.21.15.67:0/3064219193 reap_dead start 2024-03-09T08:56:49.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.752+0000 7fd38be00640 1 --2- 172.21.15.67:0/3064219193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd378004800 0x7fd378004c80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd37c001750 tx=0x7fd378005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4b7a1f6d5122ef server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.753+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c001370 con 0x7fd378004800 2024-03-09T08:56:49.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.753+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c0052d0 con 0x7fd378004800 2024-03-09T08:56:49.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.753+0000 7fd391400640 1 -- 172.21.15.67:0/3064219193 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c003830 con 0x7fd378004800 2024-03-09T08:56:49.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.753+0000 7fd393200640 1 -- 172.21.15.67:0/3064219193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd378004800 msgr2=0x7fd378004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.753+0000 7fd393200640 1 --2- 172.21.15.67:0/3064219193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd378004800 0x7fd378004c80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd37c001750 tx=0x7fd378005830 comp rx=0 tx=0).stop 2024-03-09T08:56:49.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.753+0000 7fd393200640 1 -- 172.21.15.67:0/3064219193 shutdown_connections 2024-03-09T08:56:49.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.753+0000 7fd393200640 1 --2- 172.21.15.67:0/3064219193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd378004800 0x7fd378004c80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.753+0000 7fd393200640 1 -- 172.21.15.67:0/3064219193 >> 172.21.15.67:0/3064219193 conn(0x7fd3940c82a0 msgr2=0x7fd3940c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:49.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.753+0000 7fd393200640 1 -- 172.21.15.67:0/3064219193 shutdown_connections 2024-03-09T08:56:49.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.753+0000 7fd393200640 1 -- 172.21.15.67:0/3064219193 wait complete. 2024-03-09T08:56:49.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.755+0000 7fd393200640 1 Processor -- start 2024-03-09T08:56:49.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.755+0000 7fd393200640 1 -- start start 2024-03-09T08:56:49.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.756+0000 7fd393200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd378004800 0x7fd3941ef430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.756+0000 7fd393200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3940c7130 0x7fd3941eb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.756+0000 7fd393200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd394145a20 0x7fd394145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.756+0000 7fd393200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940c2e20 con 0x7fd3940c7130 2024-03-09T08:56:49.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.756+0000 7fd393200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940c2ca0 con 0x7fd394145a20 2024-03-09T08:56:49.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.756+0000 7fd393200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940c4840 con 0x7fd378004800 2024-03-09T08:56:49.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.756+0000 7fd38be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3940c7130 0x7fd3941eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.756+0000 7fd38be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3940c7130 0x7fd3941eb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35990/0 (socket says 172.21.15.67:35990) 2024-03-09T08:56:49.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.756+0000 7fd38be00640 1 -- 172.21.15.67:0/100752380 learned_addr learned my addr 172.21.15.67:0/100752380 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:56:49.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.757+0000 7fd391e00640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd378004800 0x7fd3941ef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.757+0000 7fd392800640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd394145a20 0x7fd394145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.757+0000 7fd38be00640 1 -- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd378004800 msgr2=0x7fd3941ef430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.757+0000 7fd38be00640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd378004800 0x7fd3941ef430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.757+0000 7fd38be00640 1 -- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd394145a20 msgr2=0x7fd394145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.757+0000 7fd38be00640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd394145a20 0x7fd394145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.757+0000 7fd38be00640 1 -- 172.21.15.67:0/100752380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3941ef970 con 0x7fd3940c7130 2024-03-09T08:56:49.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.757+0000 7fd38be00640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3940c7130 0x7fd3941eb520 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd37c005040 tx=0x7fd37c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.757+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c041070 con 0x7fd3940c7130 2024-03-09T08:56:49.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.758+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c005480 con 0x7fd3940c7130 2024-03-09T08:56:49.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.758+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c007a30 con 0x7fd3940c7130 2024-03-09T08:56:49.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.758+0000 7fd393200640 1 -- 172.21.15.67:0/100752380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39406fa60 con 0x7fd3940c7130 2024-03-09T08:56:49.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.758+0000 7fd393200640 1 -- 172.21.15.67:0/100752380 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd394072cf0 con 0x7fd3940c7130 2024-03-09T08:56:49.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.759+0000 7fd393200640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eea90 0x7fd3941e0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.759+0000 7fd393200640 1 -- 172.21.15.67:0/100752380 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd394072f30 con 0x7fd3941eea90 2024-03-09T08:56:49.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.759+0000 7fd391e00640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eea90 0x7fd3941e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.760+0000 7fd391e00640 1 -- 172.21.15.67:0/100752380 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd39404f3c0 con 0x7fd3941eea90 2024-03-09T08:56:49.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.760+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd37c044070 con 0x7fd3940c7130 2024-03-09T08:56:49.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.760+0000 7fd391e00640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eea90 0x7fd3941e0410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd3840135d0 tx=0x7fd384013a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.761+0000 7fd38a000640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd37006b4d0 0x7fd37006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.761+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fd37c0ac050 con 0x7fd3940c7130 2024-03-09T08:56:49.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.761+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38401c070 con 0x7fd3941eea90 2024-03-09T08:56:49.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.762+0000 7fd392800640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd37006b4d0 0x7fd37006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.762+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd384011aa0 con 0x7fd3941eea90 2024-03-09T08:56:49.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.762+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eea90 msgr2=0x7fd3941e0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.762+0000 7fd38a000640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eea90 0x7fd3941e0410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd3840135d0 tx=0x7fd384013a90 comp rx=0 tx=0).stop 2024-03-09T08:56:49.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.762+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eea90 msgr2=0x7fd3941e0410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:56:49.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.762+0000 7fd38a000640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eea90 0x7fd3941e0410 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd3840135d0 tx=0x7fd384013a90 comp rx=0 tx=0).stop 2024-03-09T08:56:49.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.763+0000 7fd392800640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd37006b4d0 0x7fd37006d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd394073c40 tx=0x7fd38c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.770+0000 7fd393200640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eee70 0x7fd3941e0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.770+0000 7fd393200640 1 -- 172.21.15.67:0/100752380 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3941e0e90 con 0x7fd3941eee70 2024-03-09T08:56:49.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.770+0000 7fd391e00640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eee70 0x7fd3941e0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.771+0000 7fd391e00640 1 -- 172.21.15.67:0/100752380 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd3940c44e0 con 0x7fd3941eee70 2024-03-09T08:56:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.771+0000 7fd391e00640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eee70 0x7fd3941e0950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd3840135d0 tx=0x7fd384004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.772+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384006450 con 0x7fd3941eee70 2024-03-09T08:56:49.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.773+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4533 (secure 0 0 0) 0x7fd384006450 con 0x7fd3941eee70 2024-03-09T08:56:49.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.773+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eee70 msgr2=0x7fd3941e0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.773+0000 7fd38a000640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eee70 0x7fd3941e0950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd3840135d0 tx=0x7fd384004730 comp rx=0 tx=0).stop 2024-03-09T08:56:49.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.773+0000 7fd38a000640 1 -- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eee70 msgr2=0x7fd3941e0950 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:56:49.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.773+0000 7fd38a000640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3941eee70 0x7fd3941e0950 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd3840135d0 tx=0x7fd384004730 comp rx=0 tx=0).stop 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 36, 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:56:49.773 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.774 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:56:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:56:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:56:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:56:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:56:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:56:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:56:49.776 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:56:49.777 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:56:49.778 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:56:49.779 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:49.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:56:49.782 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:56:49.783 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.783 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.783 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:56:49.783 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:56:49.783 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:56:49.783 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:56:49.783 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:56:49.784 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.784 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.784 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:56:49.784 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:56:49.784 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.784 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:49.784 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.784 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:49.784 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.784 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:56:49.785 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:56:49.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:56:49.785 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:56:49.785 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:56:49.785 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:56:49.785 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:56:49.785 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:56:49.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:56:49.785 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:56:49.785 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:56:49.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.774+0000 7fd393200640 1 -- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd37006b4d0 msgr2=0x7fd37006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.774+0000 7fd393200640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd37006b4d0 0x7fd37006d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd394073c40 tx=0x7fd38c009040 comp rx=0 tx=0).stop 2024-03-09T08:56:49.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.774+0000 7fd393200640 1 -- 172.21.15.67:0/100752380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3940c7130 msgr2=0x7fd3941eb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.774+0000 7fd393200640 1 --2- 172.21.15.67:0/100752380 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3940c7130 0x7fd3941eb520 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd37c005040 tx=0x7fd37c001120 comp rx=0 tx=0).stop 2024-03-09T08:56:49.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.774+0000 7fd392800640 1 -- 172.21.15.67:0/100752380 reap_dead start 2024-03-09T08:56:49.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.774+0000 7fd392800640 1 -- 172.21.15.67:0/100752380 reap_dead start 2024-03-09T08:56:49.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.774+0000 7fd393200640 1 -- 172.21.15.67:0/100752380 shutdown_connections 2024-03-09T08:56:49.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.774+0000 7fd393200640 1 -- 172.21.15.67:0/100752380 >> 172.21.15.67:0/100752380 conn(0x7fd3940c82a0 msgr2=0x7fd394152020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:49.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.774+0000 7fd393200640 1 -- 172.21.15.67:0/100752380 shutdown_connections 2024-03-09T08:56:49.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.774+0000 7fd393200640 1 -- 172.21.15.67:0/100752380 wait complete. 2024-03-09T08:56:49.791 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:56:49.791 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T08:56:49.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T08:56:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.986+0000 7fd76f200640 1 Processor -- start 2024-03-09T08:56:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.986+0000 7fd76f200640 1 -- start start 2024-03-09T08:56:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.987+0000 7fd76f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7680c1d00 0x7fd7680aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.987+0000 7fd76f200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd7680aa8f0 0x7fd768159770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.987+0000 7fd76f200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76806de60 con 0x7fd7680acd90 2024-03-09T08:56:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.987+0000 7fd76f200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7681592e0 con 0x7fd7680aa8f0 2024-03-09T08:56:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.987+0000 7fd76f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76806d5c0 con 0x7fd7680c1d00 2024-03-09T08:56:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.988+0000 7fd76de00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd7680acd90 0x7fd7681565f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:38862/0 (socket says 172.21.15.67:38862) 2024-03-09T08:56:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.988+0000 7fd76de00640 1 -- 172.21.15.67:0/2808548055 learned_addr learned my addr 172.21.15.67:0/2808548055 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:56:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.988+0000 7fd76e800640 1 --2- 172.21.15.67:0/2808548055 >> v2:172.21.15.134:3300/0 conn(0x7fd7680aa8f0 0x7fd768159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.988+0000 7fd76d400640 1 --2- 172.21.15.67:0/2808548055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7680c1d00 0x7fd7680aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.988+0000 7fd76e800640 1 -- 172.21.15.67:0/2808548055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7680c1d00 msgr2=0x7fd7680aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:56:49.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.988+0000 7fd76e800640 1 --2- 172.21.15.67:0/2808548055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7680c1d00 0x7fd7680aa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.988+0000 7fd76e800640 1 -- 172.21.15.67:0/2808548055 >> v1:172.21.15.134:6789/0 conn(0x7fd7680acd90 legacy=0x7fd7681565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.988+0000 7fd76e800640 1 -- 172.21.15.67:0/2808548055 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76806d8a0 con 0x7fd7680aa8f0 2024-03-09T08:56:49.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.989+0000 7fd76e800640 1 --2- 172.21.15.67:0/2808548055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7680aa8f0 0x7fd768159770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd764009d90 tx=0x7fd76402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48742af663bdb99 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.990+0000 7fd76ca00640 1 -- 172.21.15.67:0/2808548055 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764039070 con 0x7fd7680aa8f0 2024-03-09T08:56:49.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.990+0000 7fd76ca00640 1 -- 172.21.15.67:0/2808548055 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76402cc50 con 0x7fd7680aa8f0 2024-03-09T08:56:49.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.990+0000 7fd76ca00640 1 -- 172.21.15.67:0/2808548055 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764034440 con 0x7fd7680aa8f0 2024-03-09T08:56:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.990+0000 7fd76f200640 1 -- 172.21.15.67:0/2808548055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7680aa8f0 msgr2=0x7fd768159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.990+0000 7fd76f200640 1 --2- 172.21.15.67:0/2808548055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7680aa8f0 0x7fd768159770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd764009d90 tx=0x7fd76402bea0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.991+0000 7fd76f200640 1 -- 172.21.15.67:0/2808548055 shutdown_connections 2024-03-09T08:56:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.991+0000 7fd76f200640 1 --2- 172.21.15.67:0/2808548055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7680aa8f0 0x7fd768159770 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.991+0000 7fd76f200640 1 --2- 172.21.15.67:0/2808548055 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7680c1d00 0x7fd7680aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.991+0000 7fd76f200640 1 -- 172.21.15.67:0/2808548055 >> 172.21.15.67:0/2808548055 conn(0x7fd7680c7300 msgr2=0x7fd7680c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:49.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.991+0000 7fd76f200640 1 -- 172.21.15.67:0/2808548055 shutdown_connections 2024-03-09T08:56:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.991+0000 7fd76f200640 1 -- 172.21.15.67:0/2808548055 wait complete. 2024-03-09T08:56:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.992+0000 7fd76f200640 1 Processor -- start 2024-03-09T08:56:49.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.993+0000 7fd76f200640 1 -- start start 2024-03-09T08:56:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.993+0000 7fd76f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7680aa8f0 0x7fd76814b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.993+0000 7fd76f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7680acd90 0x7fd7681eee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.993+0000 7fd76f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7680c1d00 0x7fd76814a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.993+0000 7fd76f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680ad2f0 con 0x7fd7680acd90 2024-03-09T08:56:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.993+0000 7fd76f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680ad170 con 0x7fd7680c1d00 2024-03-09T08:56:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.993+0000 7fd76f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680ac200 con 0x7fd7680aa8f0 2024-03-09T08:56:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.994+0000 7fd76d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7680acd90 0x7fd7681eee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.994+0000 7fd76e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7680c1d00 0x7fd76814a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.994+0000 7fd76d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7680acd90 0x7fd7681eee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35994/0 (socket says 172.21.15.67:35994) 2024-03-09T08:56:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.994+0000 7fd76d400640 1 -- 172.21.15.67:0/880817341 learned_addr learned my addr 172.21.15.67:0/880817341 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:56:49.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.994+0000 7fd76de00640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7680aa8f0 0x7fd76814b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.994+0000 7fd76d400640 1 -- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7680aa8f0 msgr2=0x7fd76814b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.994+0000 7fd76d400640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7680aa8f0 0x7fd76814b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.994+0000 7fd76d400640 1 -- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7680c1d00 msgr2=0x7fd76814a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:49.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.994+0000 7fd76d400640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7680c1d00 0x7fd76814a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:49.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.995+0000 7fd76d400640 1 -- 172.21.15.67:0/880817341 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd768155820 con 0x7fd7680acd90 2024-03-09T08:56:49.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.995+0000 7fd76d400640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7680acd90 0x7fd7681eee40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd75800e870 tx=0x7fd75800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:49.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.995+0000 7fd762000640 1 -- 172.21.15.67:0/880817341 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd758017070 con 0x7fd7680acd90 2024-03-09T08:56:49.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.995+0000 7fd762000640 1 -- 172.21.15.67:0/880817341 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75800c940 con 0x7fd7680acd90 2024-03-09T08:56:49.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.995+0000 7fd762000640 1 -- 172.21.15.67:0/880817341 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75800cd30 con 0x7fd7680acd90 2024-03-09T08:56:49.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.995+0000 7fd76f200640 1 -- 172.21.15.67:0/880817341 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7681ef710 con 0x7fd7680acd90 2024-03-09T08:56:49.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.995+0000 7fd76f200640 1 -- 172.21.15.67:0/880817341 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76814b3a0 con 0x7fd7680acd90 2024-03-09T08:56:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.997+0000 7fd76f200640 1 -- 172.21.15.67:0/880817341 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76804f3c0 con 0x7fd7680acd90 2024-03-09T08:56:49.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.998+0000 7fd762000640 1 -- 172.21.15.67:0/880817341 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd75801a070 con 0x7fd7680acd90 2024-03-09T08:56:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.999+0000 7fd762000640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd74c06b500 0x7fd74c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:56:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.999+0000 7fd762000640 1 -- 172.21.15.67:0/880817341 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fd75808a390 con 0x7fd7680acd90 2024-03-09T08:56:49.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:49.999+0000 7fd76de00640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd74c06b500 0x7fd74c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:56:49.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.000+0000 7fd76de00640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd74c06b500 0x7fd74c06d9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd75c005880 tx=0x7fd75c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:56:50.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.003+0000 7fd762000640 1 -- 172.21.15.67:0/880817341 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd7580587f0 con 0x7fd7680acd90 2024-03-09T08:56:50.064 INFO:tasks.workunit.client.0.smithi067.stdout:./254/.snap/snap-subdir-test 2024-03-09T08:56:50.074 INFO:tasks.workunit.client.0.smithi067.stdout:./113/.snap/snap-subdir-test 2024-03-09T08:56:50.083 INFO:tasks.workunit.client.0.smithi067.stdout:./152/.snap/snap-subdir-test 2024-03-09T08:56:50.091 INFO:tasks.workunit.client.0.smithi067.stdout:./121/.snap/snap-subdir-test 2024-03-09T08:56:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:./118/.snap/snap-subdir-test 2024-03-09T08:56:50.109 INFO:tasks.workunit.client.0.smithi067.stdout:./196/.snap/snap-subdir-test 2024-03-09T08:56:50.117 INFO:tasks.workunit.client.0.smithi067.stdout:./124/.snap/snap-subdir-test 2024-03-09T08:56:50.127 INFO:tasks.workunit.client.0.smithi067.stdout:./239/.snap/snap-subdir-test 2024-03-09T08:56:50.136 INFO:tasks.workunit.client.0.smithi067.stdout:./188/.snap/snap-subdir-test 2024-03-09T08:56:50.145 INFO:tasks.workunit.client.0.smithi067.stdout:./287/.snap/snap-subdir-test 2024-03-09T08:56:50.153 INFO:tasks.workunit.client.0.smithi067.stdout:./275/.snap/snap-subdir-test 2024-03-09T08:56:50.162 INFO:tasks.workunit.client.0.smithi067.stdout:./259/.snap/snap-subdir-test 2024-03-09T08:56:50.170 INFO:tasks.workunit.client.0.smithi067.stdout:./224/.snap/snap-subdir-test 2024-03-09T08:56:50.177 INFO:tasks.workunit.client.0.smithi067.stdout:./264/.snap/snap-subdir-test 2024-03-09T08:56:50.186 INFO:tasks.workunit.client.0.smithi067.stdout:./145/.snap/snap-subdir-test 2024-03-09T08:56:50.194 INFO:tasks.workunit.client.0.smithi067.stdout:./279/.snap/snap-subdir-test 2024-03-09T08:56:50.201 INFO:tasks.workunit.client.0.smithi067.stdout:./263/.snap/snap-subdir-test 2024-03-09T08:56:50.210 INFO:tasks.workunit.client.0.smithi067.stdout:./130/.snap/snap-subdir-test 2024-03-09T08:56:50.219 INFO:tasks.workunit.client.0.smithi067.stdout:./193/.snap/snap-subdir-test 2024-03-09T08:56:50.229 INFO:tasks.workunit.client.0.smithi067.stdout:./223/.snap/snap-subdir-test 2024-03-09T08:56:50.239 INFO:tasks.workunit.client.0.smithi067.stdout:./207/.snap/snap-subdir-test 2024-03-09T08:56:50.250 INFO:tasks.workunit.client.0.smithi067.stdout:./276/.snap/snap-subdir-test 2024-03-09T08:56:50.261 INFO:tasks.workunit.client.0.smithi067.stdout:./271/.snap/snap-subdir-test 2024-03-09T08:56:50.284 INFO:tasks.workunit.client.0.smithi067.stdout:./251/.snap/snap-subdir-test 2024-03-09T08:56:50.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.334+0000 7fd76f200640 1 -- 172.21.15.67:0/880817341 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fd7680c36d0 con 0x7fd7680acd90 2024-03-09T08:56:50.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.335+0000 7fd762000640 1 -- 172.21.15.67:0/880817341 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fd75805c890 con 0x7fd7680acd90 2024-03-09T08:56:50.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 -- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd74c06b500 msgr2=0x7fd74c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:50.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd74c06b500 0x7fd74c06d9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd75c005880 tx=0x7fd75c009040 comp rx=0 tx=0).stop 2024-03-09T08:56:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 -- 172.21.15.67:0/880817341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7680acd90 msgr2=0x7fd7681eee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:56:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7680acd90 0x7fd7681eee40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd75800e870 tx=0x7fd75800ed30 comp rx=0 tx=0).stop 2024-03-09T08:56:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 -- 172.21.15.67:0/880817341 shutdown_connections 2024-03-09T08:56:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd74c06b500 0x7fd74c06d9c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7680c1d00 0x7fd76814a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7680acd90 0x7fd7681eee40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 --2- 172.21.15.67:0/880817341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7680aa8f0 0x7fd76814b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:56:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 -- 172.21.15.67:0/880817341 >> 172.21.15.67:0/880817341 conn(0x7fd7680c7300 msgr2=0x7fd7680c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:56:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 -- 172.21.15.67:0/880817341 shutdown_connections 2024-03-09T08:56:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:56:50.338+0000 7fd76f200640 1 -- 172.21.15.67:0/880817341 wait complete. 2024-03-09T08:56:50.356 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T08:56:50.563 INFO:tasks.workunit.client.0.smithi067.stdout:./175/.snap/snap-subdir-test 2024-03-09T08:56:50.574 INFO:tasks.workunit.client.0.smithi067.stdout:./222/.snap/snap-subdir-test 2024-03-09T08:56:50.586 INFO:tasks.workunit.client.0.smithi067.stdout:./220/.snap/snap-subdir-test 2024-03-09T08:56:50.600 INFO:tasks.workunit.client.0.smithi067.stdout:./203/.snap/snap-subdir-test 2024-03-09T08:56:50.614 INFO:tasks.workunit.client.0.smithi067.stdout:./181/.snap/snap-subdir-test 2024-03-09T08:56:50.893 INFO:tasks.workunit.client.0.smithi067.stdout:./286/.snap/snap-subdir-test 2024-03-09T08:56:50.906 INFO:tasks.workunit.client.0.smithi067.stdout:./127/.snap/snap-subdir-test 2024-03-09T08:56:50.918 INFO:tasks.workunit.client.0.smithi067.stdout:./108/.snap/snap-subdir-test 2024-03-09T08:56:50.931 INFO:tasks.workunit.client.0.smithi067.stdout:./183/.snap/snap-subdir-test 2024-03-09T08:56:51.027 INFO:tasks.workunit.client.0.smithi067.stdout:./294/.snap/snap-subdir-test 2024-03-09T08:56:51.043 INFO:tasks.workunit.client.0.smithi067.stdout:./235/.snap/snap-subdir-test 2024-03-09T08:56:51.058 INFO:tasks.workunit.client.0.smithi067.stdout:./177/.snap/snap-subdir-test 2024-03-09T08:56:51.076 INFO:tasks.workunit.client.0.smithi067.stdout:./169/.snap/snap-subdir-test 2024-03-09T08:56:51.090 INFO:tasks.workunit.client.0.smithi067.stdout:./212/.snap/snap-subdir-test 2024-03-09T08:56:51.107 INFO:tasks.workunit.client.0.smithi067.stdout:./154/.snap/snap-subdir-test 2024-03-09T08:56:51.123 INFO:tasks.workunit.client.0.smithi067.stdout:./171/.snap/snap-subdir-test 2024-03-09T08:56:51.142 INFO:tasks.workunit.client.0.smithi067.stdout:./139/.snap/snap-subdir-test 2024-03-09T08:56:51.160 INFO:tasks.workunit.client.0.smithi067.stdout:./102/.snap/snap-subdir-test 2024-03-09T08:56:51.179 INFO:tasks.workunit.client.0.smithi067.stdout:./129/.snap/snap-subdir-test 2024-03-09T08:56:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:Delete all the files and directories ... 2024-03-09T08:56:57.957 INFO:tasks.workunit.client.1.smithi134.stdout:Delete all the files and directories ... 2024-03-09T08:56:59.068 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/19' 2024-03-09T08:56:59.068 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/15' 2024-03-09T08:56:59.068 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/31' 2024-03-09T08:56:59.068 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/36' 2024-03-09T08:56:59.068 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/20' 2024-03-09T08:56:59.068 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/12' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/39' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/10' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/34' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/38' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/11' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/16' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/32' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/30' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/14' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/18' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/13' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/33' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/17' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/35' 2024-03-09T08:56:59.069 INFO:tasks.workunit.client.0.smithi067.stdout:removed './100/37' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './100' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/19' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/15' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/31' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/36' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/20' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/12' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/39' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/10' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/34' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/38' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/11' 2024-03-09T08:56:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/16' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/32' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/30' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/14' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/18' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/13' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/33' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/17' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/35' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed './101/37' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './101' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/19' 2024-03-09T08:56:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/15' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/31' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/36' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/20' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/12' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/39' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/10' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/34' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/38' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/11' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/16' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/32' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/30' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/14' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/18' 2024-03-09T08:56:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/13' 2024-03-09T08:56:59.073 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/33' 2024-03-09T08:56:59.073 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/17' 2024-03-09T08:56:59.073 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/35' 2024-03-09T08:56:59.073 INFO:tasks.workunit.client.0.smithi067.stdout:removed './102/37' 2024-03-09T08:56:59.073 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './102' 2024-03-09T08:56:59.073 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/19' 2024-03-09T08:56:59.073 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/15' 2024-03-09T08:56:59.073 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/31' 2024-03-09T08:56:59.074 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/36' 2024-03-09T08:56:59.074 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/20' 2024-03-09T08:56:59.074 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/12' 2024-03-09T08:56:59.074 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/39' 2024-03-09T08:56:59.074 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/10' 2024-03-09T08:56:59.074 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/34' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/38' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/11' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/16' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/32' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/30' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/14' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/18' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/13' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/33' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/17' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/35' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './103/37' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './103' 2024-03-09T08:56:59.075 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/19' 2024-03-09T08:56:59.076 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/15' 2024-03-09T08:56:59.076 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/31' 2024-03-09T08:56:59.076 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/36' 2024-03-09T08:56:59.076 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/20' 2024-03-09T08:56:59.076 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/12' 2024-03-09T08:56:59.076 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/39' 2024-03-09T08:56:59.076 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/10' 2024-03-09T08:56:59.076 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/34' 2024-03-09T08:56:59.076 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/38' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/11' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/16' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/32' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/30' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/14' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/18' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/13' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/33' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/17' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/35' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './104/37' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './104' 2024-03-09T08:56:59.077 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/19' 2024-03-09T08:56:59.078 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/15' 2024-03-09T08:56:59.078 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/31' 2024-03-09T08:56:59.078 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/36' 2024-03-09T08:56:59.078 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/20' 2024-03-09T08:56:59.078 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/12' 2024-03-09T08:56:59.078 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/39' 2024-03-09T08:56:59.078 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/10' 2024-03-09T08:56:59.078 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/34' 2024-03-09T08:56:59.078 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/38' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/11' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/16' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/32' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/30' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/14' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/18' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/13' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/33' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/17' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/35' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './105/37' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './105' 2024-03-09T08:56:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/19' 2024-03-09T08:56:59.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/15' 2024-03-09T08:56:59.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/31' 2024-03-09T08:56:59.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/36' 2024-03-09T08:56:59.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/20' 2024-03-09T08:56:59.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/12' 2024-03-09T08:56:59.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/39' 2024-03-09T08:56:59.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/10' 2024-03-09T08:56:59.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/34' 2024-03-09T08:56:59.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/38' 2024-03-09T08:56:59.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/11' 2024-03-09T08:56:59.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/16' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/32' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/30' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/14' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/18' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/13' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/33' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/17' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/35' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './106/37' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './106' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/19' 2024-03-09T08:56:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/15' 2024-03-09T08:56:59.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/31' 2024-03-09T08:56:59.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/36' 2024-03-09T08:56:59.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/20' 2024-03-09T08:56:59.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/12' 2024-03-09T08:56:59.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/39' 2024-03-09T08:56:59.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/10' 2024-03-09T08:56:59.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/34' 2024-03-09T08:56:59.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/38' 2024-03-09T08:56:59.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/11' 2024-03-09T08:56:59.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/16' 2024-03-09T08:56:59.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/32' 2024-03-09T08:56:59.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/30' 2024-03-09T08:56:59.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/14' 2024-03-09T08:56:59.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/18' 2024-03-09T08:56:59.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/13' 2024-03-09T08:56:59.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/33' 2024-03-09T08:56:59.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/17' 2024-03-09T08:56:59.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/35' 2024-03-09T08:56:59.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './107/37' 2024-03-09T08:56:59.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './107' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/19' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/15' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/31' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/36' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/20' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/12' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/39' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/10' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/34' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/38' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/11' 2024-03-09T08:56:59.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/16' 2024-03-09T08:56:59.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/32' 2024-03-09T08:56:59.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/30' 2024-03-09T08:56:59.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/14' 2024-03-09T08:56:59.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/18' 2024-03-09T08:56:59.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/13' 2024-03-09T08:56:59.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/33' 2024-03-09T08:56:59.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/17' 2024-03-09T08:56:59.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/35' 2024-03-09T08:56:59.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './108/37' 2024-03-09T08:56:59.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './108' 2024-03-09T08:56:59.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/19' 2024-03-09T08:56:59.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/15' 2024-03-09T08:56:59.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/31' 2024-03-09T08:56:59.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/36' 2024-03-09T08:56:59.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/20' 2024-03-09T08:56:59.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/12' 2024-03-09T08:56:59.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/39' 2024-03-09T08:56:59.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/10' 2024-03-09T08:56:59.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/34' 2024-03-09T08:56:59.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/38' 2024-03-09T08:56:59.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/11' 2024-03-09T08:56:59.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/16' 2024-03-09T08:56:59.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/32' 2024-03-09T08:56:59.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/30' 2024-03-09T08:56:59.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/19' 2024-03-09T08:56:59.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/15' 2024-03-09T08:56:59.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/31' 2024-03-09T08:56:59.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/36' 2024-03-09T08:56:59.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/20' 2024-03-09T08:56:59.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/12' 2024-03-09T08:56:59.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/39' 2024-03-09T08:56:59.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/10' 2024-03-09T08:56:59.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/34' 2024-03-09T08:56:59.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/38' 2024-03-09T08:56:59.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/11' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/16' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/32' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/30' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/14' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/18' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/13' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/33' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/17' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/35' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './100/37' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './100' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/19' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/15' 2024-03-09T08:56:59.737 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/31' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/36' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/20' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/12' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/39' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/10' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/34' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/38' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/11' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/16' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/32' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/30' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/14' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/18' 2024-03-09T08:56:59.738 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/13' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/33' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/17' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/35' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './101/37' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './101' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/19' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/15' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/31' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/36' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/20' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/12' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/39' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/10' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/34' 2024-03-09T08:56:59.739 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/38' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/11' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/16' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/32' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/30' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/14' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/18' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/13' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/33' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/17' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/35' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './102/37' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './102' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/19' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/15' 2024-03-09T08:56:59.740 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/31' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/36' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/20' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/12' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/39' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/10' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/34' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/38' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/11' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/16' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/32' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/30' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/14' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/18' 2024-03-09T08:56:59.741 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/13' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/33' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/17' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/35' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './103/37' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './103' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/19' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/15' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/31' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/36' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/20' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/12' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/39' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/10' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/34' 2024-03-09T08:56:59.742 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/38' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/11' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/16' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/32' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/30' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/14' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/18' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/13' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/33' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/17' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/35' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './104/37' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './104' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/19' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/15' 2024-03-09T08:56:59.743 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/31' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/36' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/20' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/12' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/39' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/10' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/34' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/38' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/11' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/16' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/32' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/30' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/14' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/18' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/13' 2024-03-09T08:56:59.744 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/33' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/17' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/35' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './105/37' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './105' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/19' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/15' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/31' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/36' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/20' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/12' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/39' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/10' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/34' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/38' 2024-03-09T08:56:59.745 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/11' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/16' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/32' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/30' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/14' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/18' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/13' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/33' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/17' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/35' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './106/37' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './106' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/19' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/15' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/31' 2024-03-09T08:56:59.746 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/36' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/20' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/12' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/39' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/10' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/34' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/38' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/11' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/16' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/32' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/30' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/14' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/18' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/13' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/33' 2024-03-09T08:56:59.747 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/17' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/35' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './107/37' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './107' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/19' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/15' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/31' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/36' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/20' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/12' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/39' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/10' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/34' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/38' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/11' 2024-03-09T08:56:59.748 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/16' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/32' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/30' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/14' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/18' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/13' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/33' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/17' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/35' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './108/37' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './108' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/19' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/15' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/31' 2024-03-09T08:56:59.749 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/36' 2024-03-09T08:56:59.750 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/20' 2024-03-09T08:56:59.750 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/12' 2024-03-09T08:56:59.750 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/39' 2024-03-09T08:56:59.750 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/10' 2024-03-09T08:56:59.750 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/34' 2024-03-09T08:56:59.750 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/38' 2024-03-09T08:56:59.750 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/11' 2024-03-09T08:56:59.750 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/16' 2024-03-09T08:56:59.750 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/32' 2024-03-09T08:56:59.750 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/30' 2024-03-09T08:57:00.357 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:57:00.358 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:57:00.517 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/14' 2024-03-09T08:57:00.518 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/18' 2024-03-09T08:57:00.518 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/13' 2024-03-09T08:57:00.518 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/33' 2024-03-09T08:57:00.518 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/17' 2024-03-09T08:57:00.518 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/35' 2024-03-09T08:57:00.518 INFO:tasks.workunit.client.0.smithi067.stdout:removed './109/37' 2024-03-09T08:57:00.518 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './109' 2024-03-09T08:57:00.518 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/19' 2024-03-09T08:57:00.518 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/15' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/31' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/36' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/20' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/12' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/39' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/10' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/34' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/38' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/11' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/16' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/32' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/30' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/14' 2024-03-09T08:57:00.519 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/18' 2024-03-09T08:57:00.520 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/13' 2024-03-09T08:57:00.520 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/33' 2024-03-09T08:57:00.520 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/17' 2024-03-09T08:57:00.520 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/35' 2024-03-09T08:57:00.520 INFO:tasks.workunit.client.0.smithi067.stdout:removed './110/37' 2024-03-09T08:57:00.520 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './110' 2024-03-09T08:57:00.520 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/19' 2024-03-09T08:57:00.520 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/15' 2024-03-09T08:57:00.520 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/31' 2024-03-09T08:57:00.520 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/36' 2024-03-09T08:57:00.521 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/20' 2024-03-09T08:57:00.521 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/12' 2024-03-09T08:57:00.521 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/39' 2024-03-09T08:57:00.521 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/10' 2024-03-09T08:57:00.521 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/34' 2024-03-09T08:57:00.521 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/38' 2024-03-09T08:57:00.521 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/11' 2024-03-09T08:57:00.521 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/16' 2024-03-09T08:57:00.521 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/32' 2024-03-09T08:57:00.521 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/30' 2024-03-09T08:57:00.522 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/14' 2024-03-09T08:57:00.522 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/18' 2024-03-09T08:57:00.522 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/13' 2024-03-09T08:57:00.522 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/33' 2024-03-09T08:57:00.522 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/17' 2024-03-09T08:57:00.522 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/35' 2024-03-09T08:57:00.522 INFO:tasks.workunit.client.0.smithi067.stdout:removed './111/37' 2024-03-09T08:57:00.522 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './111' 2024-03-09T08:57:00.522 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/19' 2024-03-09T08:57:00.522 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/15' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/31' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/36' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/20' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/12' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/39' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/10' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/34' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/38' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/11' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/16' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/32' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/30' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/14' 2024-03-09T08:57:00.523 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/18' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/13' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/33' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/17' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/35' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed './112/37' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './112' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/19' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/15' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/31' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/36' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/20' 2024-03-09T08:57:00.524 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/12' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/39' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/10' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/34' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/38' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/11' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/16' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/32' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/30' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/14' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/18' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/13' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/33' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/17' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/35' 2024-03-09T08:57:00.525 INFO:tasks.workunit.client.0.smithi067.stdout:removed './113/37' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './113' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/19' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/15' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/31' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/36' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/20' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/12' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/39' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/10' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/34' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/38' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/11' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/16' 2024-03-09T08:57:00.526 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/32' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/30' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/14' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/18' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/13' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/33' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/17' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/35' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './114/37' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './114' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/19' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/15' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/31' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/36' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/20' 2024-03-09T08:57:00.527 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/12' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/39' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/10' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/34' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/38' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/11' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/16' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/32' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/30' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/14' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/18' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/13' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/33' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/17' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/35' 2024-03-09T08:57:00.528 INFO:tasks.workunit.client.0.smithi067.stdout:removed './115/37' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './115' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/19' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/15' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/31' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/36' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/20' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/12' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/39' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/10' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/34' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/38' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/11' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/16' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/32' 2024-03-09T08:57:00.529 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/30' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/14' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/18' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/13' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/33' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/17' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/35' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './116/37' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './116' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/19' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/15' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/31' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/36' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/20' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/12' 2024-03-09T08:57:00.530 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/39' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/10' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/34' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/38' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/11' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/16' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/32' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/30' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/14' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/18' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/13' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/33' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/17' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/35' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed './117/37' 2024-03-09T08:57:00.531 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './117' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/19' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/15' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/31' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/36' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/20' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/12' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/39' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/10' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/34' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/38' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/11' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/16' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/32' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/30' 2024-03-09T08:57:00.532 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/14' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/18' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/13' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/33' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/17' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/35' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './118/37' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './118' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/19' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/15' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/31' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/36' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/20' 2024-03-09T08:57:00.533 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/12' 2024-03-09T08:57:00.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.572+0000 7fcae6400640 1 Processor -- start 2024-03-09T08:57:00.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.572+0000 7fcae6400640 1 -- start start 2024-03-09T08:57:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.574+0000 7fcae6400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c5e40 0x7fcae0156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.574+0000 7fcae6400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcae00cadb0 0x7fcae0156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.574+0000 7fcae6400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae0156a90 con 0x7fcae0153300 2024-03-09T08:57:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.574+0000 7fcae6400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae0157000 con 0x7fcae00cadb0 2024-03-09T08:57:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.574+0000 7fcae6400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00ae430 con 0x7fcae00c5e40 2024-03-09T08:57:00.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.574+0000 7fcae5000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcae0153300 0x7fcae00b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35208/0 (socket says 172.21.15.67:35208) 2024-03-09T08:57:00.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.574+0000 7fcae5000640 1 -- 172.21.15.67:0/3264467975 learned_addr learned my addr 172.21.15.67:0/3264467975 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:00.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.574+0000 7fcadfe00640 1 --2- 172.21.15.67:0/3264467975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c5e40 0x7fcae0156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.574+0000 7fcae5a00640 1 --2- 172.21.15.67:0/3264467975 >> v2:172.21.15.134:3300/0 conn(0x7fcae00cadb0 0x7fcae0156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.575+0000 7fcadf400640 1 -- 172.21.15.67:0/3264467975 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412350558 0 0) 0x7fcae0156a90 con 0x7fcae0153300 2024-03-09T08:57:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.575+0000 7fcadf400640 1 -- 172.21.15.67:0/3264467975 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcac8003520 con 0x7fcae0153300 2024-03-09T08:57:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.575+0000 7fcadfe00640 1 -- 172.21.15.67:0/3264467975 >> v2:172.21.15.134:3300/0 conn(0x7fcae00cadb0 msgr2=0x7fcae0156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.575+0000 7fcadfe00640 1 --2- 172.21.15.67:0/3264467975 >> v2:172.21.15.134:3300/0 conn(0x7fcae00cadb0 0x7fcae0156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.575+0000 7fcadfe00640 1 -- 172.21.15.67:0/3264467975 >> v1:172.21.15.67:6789/0 conn(0x7fcae0153300 legacy=0x7fcae00b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.575+0000 7fcadfe00640 1 -- 172.21.15.67:0/3264467975 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae00ae6f0 con 0x7fcae00c5e40 2024-03-09T08:57:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.577+0000 7fcadfe00640 1 --2- 172.21.15.67:0/3264467975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c5e40 0x7fcae0156010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0097a0 tx=0x7fcacc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28ba009c7414efd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:00.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.577+0000 7fcadf400640 1 -- 172.21.15.67:0/3264467975 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc035020 con 0x7fcae00c5e40 2024-03-09T08:57:00.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.577+0000 7fcadf400640 1 -- 172.21.15.67:0/3264467975 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc031830 con 0x7fcae00c5e40 2024-03-09T08:57:00.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.577+0000 7fcadf400640 1 -- 172.21.15.67:0/3264467975 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc031be0 con 0x7fcae00c5e40 2024-03-09T08:57:00.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.578+0000 7fcae6400640 1 -- 172.21.15.67:0/3264467975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c5e40 msgr2=0x7fcae0156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:00.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.578+0000 7fcae6400640 1 --2- 172.21.15.67:0/3264467975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c5e40 0x7fcae0156010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0097a0 tx=0x7fcacc02b8d0 comp rx=0 tx=0).stop 2024-03-09T08:57:00.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.578+0000 7fcae6400640 1 -- 172.21.15.67:0/3264467975 shutdown_connections 2024-03-09T08:57:00.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.578+0000 7fcae6400640 1 --2- 172.21.15.67:0/3264467975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae00c5e40 0x7fcae0156010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:00.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.578+0000 7fcae6400640 1 --2- 172.21.15.67:0/3264467975 >> v2:172.21.15.134:3300/0 conn(0x7fcae00cadb0 0x7fcae0156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:00.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.578+0000 7fcae6400640 1 -- 172.21.15.67:0/3264467975 >> 172.21.15.67:0/3264467975 conn(0x7fcae00732d0 msgr2=0x7fcae00ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:00.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.578+0000 7fcae6400640 1 -- 172.21.15.67:0/3264467975 shutdown_connections 2024-03-09T08:57:00.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.578+0000 7fcae6400640 1 -- 172.21.15.67:0/3264467975 wait complete. 2024-03-09T08:57:00.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.580+0000 7fcae6400640 1 Processor -- start 2024-03-09T08:57:00.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.580+0000 7fcae6400640 1 -- start start 2024-03-09T08:57:00.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae6400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae00c5e40 0x7fcae00ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae6400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae00cadb0 0x7fcae006c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae6400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae0153300 0x7fcae01eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae6400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae0159e90 con 0x7fcae00c5e40 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae6400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae0159bb0 con 0x7fcae00cadb0 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae6400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00c59e0 con 0x7fcae0153300 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae00c5e40 0x7fcae00ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae00c5e40 0x7fcae00ab240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33726/0 (socket says 172.21.15.67:33726) 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae5000640 1 -- 172.21.15.67:0/857083063 learned_addr learned my addr 172.21.15.67:0/857083063 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae5a00640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae0153300 0x7fcae01eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcadfe00640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae00cadb0 0x7fcae006c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae5000640 1 -- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae0153300 msgr2=0x7fcae01eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:00.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae5000640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae0153300 0x7fcae01eb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:00.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae5000640 1 -- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae00cadb0 msgr2=0x7fcae006c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:00.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae5000640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae00cadb0 0x7fcae006c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:00.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.581+0000 7fcae5000640 1 -- 172.21.15.67:0/857083063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae00ad2f0 con 0x7fcae00c5e40 2024-03-09T08:57:00.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.582+0000 7fcadfe00640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae00cadb0 0x7fcae006c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:57:00.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.582+0000 7fcae5000640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae00c5e40 0x7fcae00ab240 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcad400c430 tx=0x7fcad400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:00.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.582+0000 7fcadcc00640 1 -- 172.21.15.67:0/857083063 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4017070 con 0x7fcae00c5e40 2024-03-09T08:57:00.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.582+0000 7fcadcc00640 1 -- 172.21.15.67:0/857083063 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad4005ce0 con 0x7fcae00c5e40 2024-03-09T08:57:00.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.582+0000 7fcadcc00640 1 -- 172.21.15.67:0/857083063 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad40072f0 con 0x7fcae00c5e40 2024-03-09T08:57:00.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.582+0000 7fcae6400640 1 -- 172.21.15.67:0/857083063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae00ad5d0 con 0x7fcae00c5e40 2024-03-09T08:57:00.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.582+0000 7fcae6400640 1 -- 172.21.15.67:0/857083063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae01ec320 con 0x7fcae00c5e40 2024-03-09T08:57:00.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.583+0000 7fcae6400640 1 -- 172.21.15.67:0/857083063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae00abc40 con 0x7fcae00c5e40 2024-03-09T08:57:00.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.585+0000 7fcadcc00640 1 -- 172.21.15.67:0/857083063 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcad401a070 con 0x7fcae00c5e40 2024-03-09T08:57:00.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.587+0000 7fcadcc00640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcac006b4d0 0x7fcac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:00.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.587+0000 7fcadcc00640 1 -- 172.21.15.67:0/857083063 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 8361+0+0 (secure 0 0 0) 0x7fcad408a160 con 0x7fcae00c5e40 2024-03-09T08:57:00.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.587+0000 7fcadfe00640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcac006b4d0 0x7fcac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:00.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.588+0000 7fcadfe00640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcac006b4d0 0x7fcac006d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcacc036000 tx=0x7fcacc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:00.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.593+0000 7fcadcc00640 1 -- 172.21.15.67:0/857083063 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcae00abc40 con 0x7fcae00c5e40 2024-03-09T08:57:00.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.916+0000 7fcae6400640 1 -- 172.21.15.67:0/857083063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcae004f3c0 con 0x7fcae00c5e40 2024-03-09T08:57:00.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.917+0000 7fcadcc00640 1 -- 172.21.15.67:0/857083063 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fcad4058530 con 0x7fcae00c5e40 2024-03-09T08:57:00.916 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:57:00.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 -- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcac006b4d0 msgr2=0x7fcac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:00.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcac006b4d0 0x7fcac006d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcacc036000 tx=0x7fcacc008030 comp rx=0 tx=0).stop 2024-03-09T08:57:00.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 -- 172.21.15.67:0/857083063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae00c5e40 msgr2=0x7fcae00ab240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:00.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae00c5e40 0x7fcae00ab240 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcad400c430 tx=0x7fcad400c8f0 comp rx=0 tx=0).stop 2024-03-09T08:57:00.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 -- 172.21.15.67:0/857083063 shutdown_connections 2024-03-09T08:57:00.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcac006b4d0 0x7fcac006d990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:00.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcae0153300 0x7fcae01eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:00.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcae00cadb0 0x7fcae006c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:00.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 --2- 172.21.15.67:0/857083063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcae00c5e40 0x7fcae00ab240 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:00.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 -- 172.21.15.67:0/857083063 >> 172.21.15.67:0/857083063 conn(0x7fcae00732d0 msgr2=0x7fcae0156010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:00.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 -- 172.21.15.67:0/857083063 shutdown_connections 2024-03-09T08:57:00.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:00.920+0000 7fcae6400640 1 -- 172.21.15.67:0/857083063 wait complete. 2024-03-09T08:57:00.946 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:57:00.946 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T08:57:00.946 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:57:00.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:57:01.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.178+0000 7f353a200640 1 Processor -- start 2024-03-09T08:57:01.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.178+0000 7f353a200640 1 -- start start 2024-03-09T08:57:01.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.180+0000 7f353a200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3534153600 0x7f353406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.180+0000 7f353a200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35340c9c80 0x7f3534160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.180+0000 7f353a200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35341539e0 con 0x7f353406b580 2024-03-09T08:57:01.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.180+0000 7f353a200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f353415a490 con 0x7f3534153600 2024-03-09T08:57:01.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.180+0000 7f353a200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35340c6990 con 0x7f35340c9c80 2024-03-09T08:57:01.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.180+0000 7f3532a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f353406b580 0x7f35340c9570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35220/0 (socket says 172.21.15.67:35220) 2024-03-09T08:57:01.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.180+0000 7f3532a00640 1 -- 172.21.15.67:0/2115104852 learned_addr learned my addr 172.21.15.67:0/2115104852 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:01.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.180+0000 7f3533400640 1 --2- 172.21.15.67:0/2115104852 >> v2:172.21.15.134:3300/0 conn(0x7f3534153600 0x7f353406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.181+0000 7f3533e00640 1 --2- 172.21.15.67:0/2115104852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35340c9c80 0x7f3534160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.181+0000 7f3532000640 1 -- 172.21.15.67:0/2115104852 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250064529 0 0) 0x7f35341539e0 con 0x7f353406b580 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.181+0000 7f3532000640 1 -- 172.21.15.67:0/2115104852 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3518003520 con 0x7f353406b580 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.181+0000 7f3533400640 1 -- 172.21.15.67:0/2115104852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35340c9c80 msgr2=0x7f3534160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.181+0000 7f3533400640 1 --2- 172.21.15.67:0/2115104852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35340c9c80 0x7f3534160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.181+0000 7f3533400640 1 -- 172.21.15.67:0/2115104852 >> v1:172.21.15.67:6789/0 conn(0x7f353406b580 legacy=0x7f35340c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.181+0000 7f3532000640 1 -- 172.21.15.67:0/2115104852 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2903404574 0 0) 0x7f3518003520 con 0x7f353406b580 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.181+0000 7f3533400640 1 -- 172.21.15.67:0/2115104852 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35340c7a00 con 0x7f3534153600 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.183+0000 7f3533400640 1 --2- 172.21.15.67:0/2115104852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3534153600 0x7f353406b040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35240097a0 tx=0x7f352402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86539a7c1c3a712c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.183+0000 7f3532000640 1 -- 172.21.15.67:0/2115104852 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524037070 con 0x7f3534153600 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.183+0000 7f3532000640 1 -- 172.21.15.67:0/2115104852 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3524030510 con 0x7f3534153600 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.183+0000 7f3532000640 1 -- 172.21.15.67:0/2115104852 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520001670 con 0x7f3534153600 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.184+0000 7f353a200640 1 -- 172.21.15.67:0/2115104852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3534153600 msgr2=0x7f353406b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.184+0000 7f353a200640 1 --2- 172.21.15.67:0/2115104852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3534153600 0x7f353406b040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35240097a0 tx=0x7f352402b710 comp rx=0 tx=0).stop 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.184+0000 7f353a200640 1 -- 172.21.15.67:0/2115104852 shutdown_connections 2024-03-09T08:57:01.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.184+0000 7f353a200640 1 --2- 172.21.15.67:0/2115104852 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35340c9c80 0x7f3534160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.184+0000 7f353a200640 1 --2- 172.21.15.67:0/2115104852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3534153600 0x7f353406b040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.184+0000 7f353a200640 1 -- 172.21.15.67:0/2115104852 >> 172.21.15.67:0/2115104852 conn(0x7f35340ad160 msgr2=0x7f35340ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:01.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.184+0000 7f353a200640 1 -- 172.21.15.67:0/2115104852 shutdown_connections 2024-03-09T08:57:01.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.184+0000 7f353a200640 1 -- 172.21.15.67:0/2115104852 wait complete. 2024-03-09T08:57:01.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.187+0000 7f353a200640 1 Processor -- start 2024-03-09T08:57:01.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.187+0000 7f353a200640 1 -- start start 2024-03-09T08:57:01.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.188+0000 7f353a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353406b580 0x7f3534154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.188+0000 7f353a200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35340c9c80 0x7f3534154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.188+0000 7f353a200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3534153600 0x7f35340c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.188+0000 7f353a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35340c6e20 con 0x7f353406b580 2024-03-09T08:57:01.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.188+0000 7f353a200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35340c7580 con 0x7f3534153600 2024-03-09T08:57:01.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.188+0000 7f353a200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35340c6fa0 con 0x7f35340c9c80 2024-03-09T08:57:01.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.188+0000 7f3533400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353406b580 0x7f3534154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.189+0000 7f3533400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353406b580 0x7f3534154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33730/0 (socket says 172.21.15.67:33730) 2024-03-09T08:57:01.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.189+0000 7f3533400640 1 -- 172.21.15.67:0/1375488226 learned_addr learned my addr 172.21.15.67:0/1375488226 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:01.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.189+0000 7f3533e00640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3534153600 0x7f35340c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.189+0000 7f3532a00640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35340c9c80 0x7f3534154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.189+0000 7f3533400640 1 -- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35340c9c80 msgr2=0x7f3534154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.189+0000 7f3533400640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35340c9c80 0x7f3534154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.189+0000 7f3533400640 1 -- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3534153600 msgr2=0x7f35340c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.189+0000 7f3533400640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3534153600 0x7f35340c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.189+0000 7f3533400640 1 -- 172.21.15.67:0/1375488226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35341effc0 con 0x7f353406b580 2024-03-09T08:57:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.189+0000 7f3533e00640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3534153600 0x7f35340c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:57:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.189+0000 7f3533400640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353406b580 0x7f3534154050 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3524000c00 tx=0x7f35240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.190+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524040020 con 0x7f353406b580 2024-03-09T08:57:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.190+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3524038040 con 0x7f353406b580 2024-03-09T08:57:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.190+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524037070 con 0x7f353406b580 2024-03-09T08:57:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.190+0000 7f353a200640 1 -- 172.21.15.67:0/1375488226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3534157410 con 0x7f353406b580 2024-03-09T08:57:01.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.190+0000 7f353a200640 1 -- 172.21.15.67:0/1375488226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35341593f0 con 0x7f353406b580 2024-03-09T08:57:01.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.191+0000 7f353a200640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353415a010 0x7f35340c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.191+0000 7f353a200640 1 -- 172.21.15.67:0/1375488226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35341efa90 con 0x7f353415a010 2024-03-09T08:57:01.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.191+0000 7f3532a00640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353415a010 0x7f35340c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.192+0000 7f3532a00640 1 -- 172.21.15.67:0/1375488226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3534075a00 con 0x7f353415a010 2024-03-09T08:57:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.192+0000 7f3532a00640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353415a010 0x7f35340c5060 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f353414d970 tx=0x7f352000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.192+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35200058d0 con 0x7f353415a010 2024-03-09T08:57:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.193+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f3520018070 con 0x7f353415a010 2024-03-09T08:57:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.193+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353415a010 msgr2=0x7f35340c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.193+0000 7f352b400640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353415a010 0x7f35340c5060 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f353414d970 tx=0x7f352000eb90 comp rx=0 tx=0).stop 2024-03-09T08:57:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.193+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353415a010 msgr2=0x7f35340c5060 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.193+0000 7f352b400640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353415a010 0x7f35340c5060 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.193+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3524043020 con 0x7f353406b580 2024-03-09T08:57:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.194+0000 7f352b400640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f351406af40 0x7f351406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.194+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 8361+0+0 (secure 0 0 0) 0x7f35240ac170 con 0x7f353406b580 2024-03-09T08:57:01.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.195+0000 7f3532a00640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f351406af40 0x7f351406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.196+0000 7f3532a00640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f351406af40 0x7f351406d400 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3520014000 tx=0x7f3520012000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.200+0000 7f353a200640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35340c2e90 0x7f3534157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.200+0000 7f353a200640 1 -- 172.21.15.67:0/1375488226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3534152180 con 0x7f35340c2e90 2024-03-09T08:57:01.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.200+0000 7f3533e00640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35340c2e90 0x7f3534157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.201+0000 7f3533e00640 1 -- 172.21.15.67:0/1375488226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f35340cb360 con 0x7f35340c2e90 2024-03-09T08:57:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.201+0000 7f3533e00640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35340c2e90 0x7f3534157bd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f352c009f50 tx=0x7f352c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.201+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c01a030 con 0x7f35340c2e90 2024-03-09T08:57:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.201+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f352c016470 con 0x7f35340c2e90 2024-03-09T08:57:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.201+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35340c2e90 msgr2=0x7f3534157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.201+0000 7f352b400640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35340c2e90 0x7f3534157bd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f352c009f50 tx=0x7f352c0115a0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.201+0000 7f352b400640 1 -- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35340c2e90 msgr2=0x7f3534157bd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:01.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.201+0000 7f352b400640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35340c2e90 0x7f3534157bd0 secure :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f352c009f50 tx=0x7f352c0115a0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.209 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:57:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:01.209 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 36, 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:01.210 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:01.211 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:57:01.212 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:01.213 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:57:01.214 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.215 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:01.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.218 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:57:01.219 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.220 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:01.221 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:57:01.222 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:57:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.202+0000 7f353a200640 1 -- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f351406af40 msgr2=0x7f351406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.202+0000 7f353a200640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f351406af40 0x7f351406d400 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3520014000 tx=0x7f3520012000 comp rx=0 tx=0).stop 2024-03-09T08:57:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.202+0000 7f353a200640 1 -- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353406b580 msgr2=0x7f3534154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.202+0000 7f353a200640 1 --2- 172.21.15.67:0/1375488226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f353406b580 0x7f3534154050 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3524000c00 tx=0x7f35240046c0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.202+0000 7f3533e00640 1 -- 172.21.15.67:0/1375488226 reap_dead start 2024-03-09T08:57:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.202+0000 7f353a200640 1 -- 172.21.15.67:0/1375488226 shutdown_connections 2024-03-09T08:57:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.202+0000 7f353a200640 1 -- 172.21.15.67:0/1375488226 >> 172.21.15.67:0/1375488226 conn(0x7f35340ad160 msgr2=0x7f3534152650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.202+0000 7f353a200640 1 -- 172.21.15.67:0/1375488226 shutdown_connections 2024-03-09T08:57:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.202+0000 7f353a200640 1 -- 172.21.15.67:0/1375488226 wait complete. 2024-03-09T08:57:01.225 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:57:01.225 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:57:01.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:57:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.429+0000 7ff477200640 1 Processor -- start 2024-03-09T08:57:01.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.429+0000 7ff477200640 1 -- start start 2024-03-09T08:57:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.431+0000 7ff477200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff47814fd90 0x7ff478156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.431+0000 7ff477200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff47806c1d0 0x7ff47806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.431+0000 7ff477200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff478156bc0 con 0x7ff4780c9380 2024-03-09T08:57:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.431+0000 7ff477200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47806ba60 con 0x7ff47814fd90 2024-03-09T08:57:01.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.431+0000 7ff477200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4780c9890 con 0x7ff47806c1d0 2024-03-09T08:57:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.431+0000 7ff476800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff47806c1d0 0x7ff47806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.432+0000 7ff475e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff47814fd90 0x7ff478156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.432+0000 7ff476800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff47806c1d0 0x7ff47806c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:52480/0 (socket says 172.21.15.67:52480) 2024-03-09T08:57:01.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.432+0000 7ff476800640 1 -- 172.21.15.67:0/2088999791 learned_addr learned my addr 172.21.15.67:0/2088999791 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:01.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.432+0000 7ff476800640 1 -- 172.21.15.67:0/2088999791 >> v2:172.21.15.134:3300/0 conn(0x7ff47814fd90 msgr2=0x7ff478156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:01.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.432+0000 7ff476800640 1 --2- 172.21.15.67:0/2088999791 >> v2:172.21.15.134:3300/0 conn(0x7ff47814fd90 0x7ff478156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.432+0000 7ff476800640 1 -- 172.21.15.67:0/2088999791 >> v1:172.21.15.134:6789/0 conn(0x7ff4780c9380 legacy=0x7ff4780acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.432+0000 7ff476800640 1 -- 172.21.15.67:0/2088999791 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4780c5f80 con 0x7ff47806c1d0 2024-03-09T08:57:01.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.433+0000 7ff476800640 1 --2- 172.21.15.67:0/2088999791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff47806c1d0 0x7ff47806c5f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff470009e50 tx=0x7ff47002be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a35125d3be4c05 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.434+0000 7ff475400640 1 -- 172.21.15.67:0/2088999791 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47002c9e0 con 0x7ff47806c1d0 2024-03-09T08:57:01.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.434+0000 7ff475400640 1 -- 172.21.15.67:0/2088999791 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47002cb80 con 0x7ff47806c1d0 2024-03-09T08:57:01.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.434+0000 7ff475400640 1 -- 172.21.15.67:0/2088999791 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700343f0 con 0x7ff47806c1d0 2024-03-09T08:57:01.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.435+0000 7ff477200640 1 -- 172.21.15.67:0/2088999791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff47806c1d0 msgr2=0x7ff47806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.435+0000 7ff477200640 1 --2- 172.21.15.67:0/2088999791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff47806c1d0 0x7ff47806c5f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff470009e50 tx=0x7ff47002be40 comp rx=0 tx=0).stop 2024-03-09T08:57:01.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.435+0000 7ff477200640 1 -- 172.21.15.67:0/2088999791 shutdown_connections 2024-03-09T08:57:01.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.435+0000 7ff477200640 1 --2- 172.21.15.67:0/2088999791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff47806c1d0 0x7ff47806c5f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.435+0000 7ff477200640 1 --2- 172.21.15.67:0/2088999791 >> v2:172.21.15.134:3300/0 conn(0x7ff47814fd90 0x7ff478156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.435+0000 7ff477200640 1 -- 172.21.15.67:0/2088999791 >> 172.21.15.67:0/2088999791 conn(0x7ff4780cb480 msgr2=0x7ff4780cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:01.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.435+0000 7ff477200640 1 -- 172.21.15.67:0/2088999791 shutdown_connections 2024-03-09T08:57:01.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.435+0000 7ff477200640 1 -- 172.21.15.67:0/2088999791 wait complete. 2024-03-09T08:57:01.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.437+0000 7ff477200640 1 Processor -- start 2024-03-09T08:57:01.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.437+0000 7ff477200640 1 -- start start 2024-03-09T08:57:01.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.439+0000 7ff477200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff47806c1d0 0x7ff47814dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.439+0000 7ff477200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4780c9380 0x7ff478156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.439+0000 7ff477200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff47814fd90 0x7ff478157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.439+0000 7ff477200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47806bcb0 con 0x7ff47806c1d0 2024-03-09T08:57:01.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.439+0000 7ff477200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4780c6520 con 0x7ff4780c9380 2024-03-09T08:57:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.439+0000 7ff477200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4780ad610 con 0x7ff47814fd90 2024-03-09T08:57:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.439+0000 7ff475e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff47806c1d0 0x7ff47814dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.439+0000 7ff475e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff47806c1d0 0x7ff47814dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33754/0 (socket says 172.21.15.67:33754) 2024-03-09T08:57:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.439+0000 7ff475e00640 1 -- 172.21.15.67:0/2463008183 learned_addr learned my addr 172.21.15.67:0/2463008183 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.439+0000 7ff476800640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff47814fd90 0x7ff478157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.439+0000 7ff46d400640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4780c9380 0x7ff478156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.440+0000 7ff475e00640 1 -- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff47814fd90 msgr2=0x7ff478157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.440+0000 7ff475e00640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff47814fd90 0x7ff478157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.440+0000 7ff475e00640 1 -- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4780c9380 msgr2=0x7ff478156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.440+0000 7ff475e00640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4780c9380 0x7ff478156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.440+0000 7ff475e00640 1 -- 172.21.15.67:0/2463008183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4780af1c0 con 0x7ff47806c1d0 2024-03-09T08:57:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.440+0000 7ff46d400640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4780c9380 0x7ff478156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:57:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.440+0000 7ff475e00640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff47806c1d0 0x7ff47814dd30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e9e0 tx=0x7ff46800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.440+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468017070 con 0x7ff47806c1d0 2024-03-09T08:57:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.440+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46800c9d0 con 0x7ff47806c1d0 2024-03-09T08:57:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.440+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46800cdc0 con 0x7ff47806c1d0 2024-03-09T08:57:01.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.440+0000 7ff477200640 1 -- 172.21.15.67:0/2463008183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4780b0eb0 con 0x7ff47806c1d0 2024-03-09T08:57:01.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.441+0000 7ff477200640 1 -- 172.21.15.67:0/2463008183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4780afea0 con 0x7ff47806c1d0 2024-03-09T08:57:01.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.442+0000 7ff477200640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff47814ce10 0x7ff478148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.442+0000 7ff477200640 1 -- 172.21.15.67:0/2463008183 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47804f3c0 con 0x7ff47814ce10 2024-03-09T08:57:01.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.442+0000 7ff46d400640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff47814ce10 0x7ff478148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.443+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff46801a070 con 0x7ff47806c1d0 2024-03-09T08:57:01.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.443+0000 7ff46d400640 1 -- 172.21.15.67:0/2463008183 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff47806d2d0 con 0x7ff47814ce10 2024-03-09T08:57:01.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.444+0000 7ff46d400640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff47814ce10 0x7ff478148210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4780afd00 tx=0x7ff46000e960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.444+0000 7ff46ea00640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff45406b4d0 0x7ff45406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.444+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 8361+0+0 (secure 0 0 0) 0x7ff46808afb0 con 0x7ff47806c1d0 2024-03-09T08:57:01.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.444+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4600052e0 con 0x7ff47814ce10 2024-03-09T08:57:01.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.444+0000 7ff476800640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff45406b4d0 0x7ff45406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.445+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff4600052e0 con 0x7ff47814ce10 2024-03-09T08:57:01.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.445+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff47814ce10 msgr2=0x7ff478148210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.445+0000 7ff46ea00640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff47814ce10 0x7ff478148210 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4780afd00 tx=0x7ff46000e960 comp rx=0 tx=0).stop 2024-03-09T08:57:01.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.445+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff47814ce10 msgr2=0x7ff478148210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:01.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.445+0000 7ff46ea00640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff47814ce10 0x7ff478148210 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4780afd00 tx=0x7ff46000e960 comp rx=0 tx=0).stop 2024-03-09T08:57:01.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.446+0000 7ff476800640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff45406b4d0 0x7ff45406d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff470035000 tx=0x7ff470030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.452+0000 7ff477200640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff478159e40 0x7ff478148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.452+0000 7ff477200640 1 -- 172.21.15.67:0/2463008183 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff478154c20 con 0x7ff478159e40 2024-03-09T08:57:01.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.453+0000 7ff46d400640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff478159e40 0x7ff478148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.454+0000 7ff46d400640 1 -- 172.21.15.67:0/2463008183 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff4780cbac0 con 0x7ff478159e40 2024-03-09T08:57:01.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.454+0000 7ff46d400640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff478159e40 0x7ff478148750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4780afd00 tx=0x7ff460000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.455+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4600069e0 con 0x7ff478159e40 2024-03-09T08:57:01.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.455+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4874 (secure 0 0 0) 0x7ff4600069e0 con 0x7ff478159e40 2024-03-09T08:57:01.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.455+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff478159e40 msgr2=0x7ff478148750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.455+0000 7ff46ea00640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff478159e40 0x7ff478148750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4780afd00 tx=0x7ff460000ed0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.455+0000 7ff46ea00640 1 -- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff478159e40 msgr2=0x7ff478148750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:01.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.455+0000 7ff46ea00640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff478159e40 0x7ff478148750 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4780afd00 tx=0x7ff460000ed0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.455 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:57:01.456 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:01.456 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:01.456 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:57:01.456 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 36, 2024-03-09T08:57:01.456 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:57:01.456 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:57:01.456 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:57:01.456 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:57:01.456 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.456 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T08:57:01.456 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:01.457 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.458 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:57:01.458 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:57:01.458 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:01.458 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:01.458 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:01.458 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:01.458 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:01.458 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:01.458 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:01.458 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:01.458 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:01.459 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:57:01.460 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:57:01.460 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:57:01.460 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:57:01.460 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:57:01.460 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:57:01.460 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:57:01.460 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:01.460 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:57:01.460 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:01.461 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:01.461 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:01.461 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:01.461 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:01.461 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:01.461 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:01.461 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:01.461 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:01.461 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:01.461 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:01.461 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:57:01.462 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:01.463 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:01.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:01.468 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.469 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:01.470 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.470 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:01.470 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.470 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.470 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:57:01.470 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:57:01.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.456+0000 7ff477200640 1 -- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff45406b4d0 msgr2=0x7ff45406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.456+0000 7ff477200640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff45406b4d0 0x7ff45406d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff470035000 tx=0x7ff470030000 comp rx=0 tx=0).stop 2024-03-09T08:57:01.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.456+0000 7ff477200640 1 -- 172.21.15.67:0/2463008183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff47806c1d0 msgr2=0x7ff47814dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.456+0000 7ff477200640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff47806c1d0 0x7ff47814dd30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e9e0 tx=0x7ff46800eea0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.456+0000 7ff476800640 1 -- 172.21.15.67:0/2463008183 reap_dead start 2024-03-09T08:57:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.456+0000 7ff477200640 1 -- 172.21.15.67:0/2463008183 shutdown_connections 2024-03-09T08:57:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.456+0000 7ff477200640 1 --2- 172.21.15.67:0/2463008183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff47806c1d0 0x7ff47814dd30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.456+0000 7ff477200640 1 -- 172.21.15.67:0/2463008183 >> 172.21.15.67:0/2463008183 conn(0x7ff4780cb480 msgr2=0x7ff4781e24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.457+0000 7ff477200640 1 -- 172.21.15.67:0/2463008183 shutdown_connections 2024-03-09T08:57:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.457+0000 7ff477200640 1 -- 172.21.15.67:0/2463008183 wait complete. 2024-03-09T08:57:01.475 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:57:01.476 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:57:01.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:57:01.539 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/14' 2024-03-09T08:57:01.539 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/18' 2024-03-09T08:57:01.539 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/13' 2024-03-09T08:57:01.539 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/33' 2024-03-09T08:57:01.539 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/17' 2024-03-09T08:57:01.539 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/35' 2024-03-09T08:57:01.539 INFO:tasks.workunit.client.1.smithi134.stdout:removed './109/37' 2024-03-09T08:57:01.539 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './109' 2024-03-09T08:57:01.540 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/19' 2024-03-09T08:57:01.540 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/15' 2024-03-09T08:57:01.540 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/31' 2024-03-09T08:57:01.540 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/36' 2024-03-09T08:57:01.540 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/20' 2024-03-09T08:57:01.540 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/12' 2024-03-09T08:57:01.540 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/39' 2024-03-09T08:57:01.540 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/10' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/34' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/38' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/11' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/16' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/32' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/30' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/14' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/18' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/13' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/33' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/17' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/35' 2024-03-09T08:57:01.541 INFO:tasks.workunit.client.1.smithi134.stdout:removed './110/37' 2024-03-09T08:57:01.542 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './110' 2024-03-09T08:57:01.542 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/19' 2024-03-09T08:57:01.542 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/15' 2024-03-09T08:57:01.542 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/31' 2024-03-09T08:57:01.542 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/36' 2024-03-09T08:57:01.542 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/20' 2024-03-09T08:57:01.542 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/12' 2024-03-09T08:57:01.543 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/39' 2024-03-09T08:57:01.543 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/10' 2024-03-09T08:57:01.543 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/34' 2024-03-09T08:57:01.543 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/38' 2024-03-09T08:57:01.543 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/11' 2024-03-09T08:57:01.543 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/16' 2024-03-09T08:57:01.543 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/32' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/30' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/14' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/18' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/13' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/33' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/17' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/35' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './111/37' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './111' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/19' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/15' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/31' 2024-03-09T08:57:01.544 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/36' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/20' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/12' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/39' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/10' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/34' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/38' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/11' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/16' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/32' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/30' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/14' 2024-03-09T08:57:01.545 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/18' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/13' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/33' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/17' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/35' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed './112/37' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './112' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/19' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/15' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/31' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/36' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/20' 2024-03-09T08:57:01.546 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/12' 2024-03-09T08:57:01.547 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/39' 2024-03-09T08:57:01.547 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/10' 2024-03-09T08:57:01.547 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/34' 2024-03-09T08:57:01.547 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/38' 2024-03-09T08:57:01.547 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/11' 2024-03-09T08:57:01.547 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/16' 2024-03-09T08:57:01.547 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/32' 2024-03-09T08:57:01.547 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/30' 2024-03-09T08:57:01.547 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/14' 2024-03-09T08:57:01.548 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/18' 2024-03-09T08:57:01.548 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/13' 2024-03-09T08:57:01.548 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/33' 2024-03-09T08:57:01.548 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/17' 2024-03-09T08:57:01.548 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/35' 2024-03-09T08:57:01.548 INFO:tasks.workunit.client.1.smithi134.stdout:removed './113/37' 2024-03-09T08:57:01.548 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './113' 2024-03-09T08:57:01.548 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/19' 2024-03-09T08:57:01.548 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/15' 2024-03-09T08:57:01.548 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/31' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/36' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/20' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/12' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/39' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/10' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/34' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/38' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/11' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/16' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/32' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/30' 2024-03-09T08:57:01.549 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/14' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/18' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/13' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/33' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/17' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/35' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './114/37' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './114' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/19' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/15' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/31' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/36' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/20' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/12' 2024-03-09T08:57:01.550 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/39' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/10' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/34' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/38' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/11' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/16' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/32' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/30' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/14' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/18' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/13' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/33' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/17' 2024-03-09T08:57:01.551 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/35' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './115/37' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './115' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/19' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/15' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/31' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/36' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/20' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/12' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/39' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/10' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/34' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/38' 2024-03-09T08:57:01.552 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/11' 2024-03-09T08:57:01.553 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/16' 2024-03-09T08:57:01.553 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/32' 2024-03-09T08:57:01.553 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/30' 2024-03-09T08:57:01.553 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/14' 2024-03-09T08:57:01.553 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/18' 2024-03-09T08:57:01.553 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/13' 2024-03-09T08:57:01.553 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/33' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/17' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/35' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './116/37' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './116' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/19' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/15' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/31' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/36' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/20' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/12' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/39' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/10' 2024-03-09T08:57:01.554 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/34' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/38' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/11' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/16' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/32' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/30' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/14' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/18' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/13' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/33' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/17' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/35' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed './117/37' 2024-03-09T08:57:01.555 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './117' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/19' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/15' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/31' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/36' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/20' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/12' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/39' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/10' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/34' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/38' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/11' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/16' 2024-03-09T08:57:01.556 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/32' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/30' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/14' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/18' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/13' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/33' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/17' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/35' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed './118/37' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './118' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/19' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/15' 2024-03-09T08:57:01.557 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/31' 2024-03-09T08:57:01.558 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/36' 2024-03-09T08:57:01.558 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/20' 2024-03-09T08:57:01.558 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/12' 2024-03-09T08:57:01.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.710+0000 7f51cba00640 1 Processor -- start 2024-03-09T08:57:01.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.711+0000 7f51cba00640 1 -- start start 2024-03-09T08:57:01.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.712+0000 7f51cba00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c41535e0 0x7f51c406dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.712+0000 7f51cba00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f51c40730d0 0x7f51c40c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.712+0000 7f51cba00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c4156a40 con 0x7f51c40c5900 2024-03-09T08:57:01.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.712+0000 7f51cba00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c4073850 con 0x7f51c40730d0 2024-03-09T08:57:01.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.712+0000 7f51cba00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4073570 con 0x7f51c41535e0 2024-03-09T08:57:01.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.713+0000 7f51c9000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c41535e0 0x7f51c406dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.713+0000 7f51c9a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f51c40730d0 0x7f51c40c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.713+0000 7f51c9a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f51c40730d0 0x7f51c40c2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:50176/0 (socket says 172.21.15.67:50176) 2024-03-09T08:57:01.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.713+0000 7f51c9a00640 1 -- 172.21.15.67:0/4167751133 learned_addr learned my addr 172.21.15.67:0/4167751133 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:01.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.714+0000 7f51c9000640 1 -- 172.21.15.67:0/4167751133 >> v2:172.21.15.134:3300/0 conn(0x7f51c40730d0 msgr2=0x7f51c40c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:01.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.714+0000 7f51c9000640 1 --2- 172.21.15.67:0/4167751133 >> v2:172.21.15.134:3300/0 conn(0x7f51c40730d0 0x7f51c40c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.714+0000 7f51c9000640 1 -- 172.21.15.67:0/4167751133 >> v1:172.21.15.134:6789/0 conn(0x7f51c40c5900 legacy=0x7f51c40729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.714+0000 7f51c9000640 1 -- 172.21.15.67:0/4167751133 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c4156ce0 con 0x7f51c41535e0 2024-03-09T08:57:01.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.714+0000 7f51c9a00640 1 --2- 172.21.15.67:0/4167751133 >> v2:172.21.15.134:3300/0 conn(0x7f51c40730d0 0x7f51c40c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:57:01.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.715+0000 7f51c9000640 1 --2- 172.21.15.67:0/4167751133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c41535e0 0x7f51c406dce0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f51b80097a0 tx=0x7f51b802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ded34342860c21 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.716+0000 7f51c3400640 1 -- 172.21.15.67:0/4167751133 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8031780 con 0x7f51c41535e0 2024-03-09T08:57:01.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.716+0000 7f51c3400640 1 -- 172.21.15.67:0/4167751133 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b8037070 con 0x7f51c41535e0 2024-03-09T08:57:01.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.716+0000 7f51c3400640 1 -- 172.21.15.67:0/4167751133 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8031b40 con 0x7f51c41535e0 2024-03-09T08:57:01.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.717+0000 7f51cba00640 1 -- 172.21.15.67:0/4167751133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c41535e0 msgr2=0x7f51c406dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.717+0000 7f51cba00640 1 --2- 172.21.15.67:0/4167751133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c41535e0 0x7f51c406dce0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f51b80097a0 tx=0x7f51b802b820 comp rx=0 tx=0).stop 2024-03-09T08:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.717+0000 7f51cba00640 1 -- 172.21.15.67:0/4167751133 shutdown_connections 2024-03-09T08:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.717+0000 7f51cba00640 1 --2- 172.21.15.67:0/4167751133 >> v2:172.21.15.134:3300/0 conn(0x7f51c40730d0 0x7f51c40c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.717+0000 7f51cba00640 1 --2- 172.21.15.67:0/4167751133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c41535e0 0x7f51c406dce0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.717+0000 7f51cba00640 1 -- 172.21.15.67:0/4167751133 >> 172.21.15.67:0/4167751133 conn(0x7f51c40ab320 msgr2=0x7f51c40ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.717+0000 7f51cba00640 1 -- 172.21.15.67:0/4167751133 shutdown_connections 2024-03-09T08:57:01.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.717+0000 7f51cba00640 1 -- 172.21.15.67:0/4167751133 wait complete. 2024-03-09T08:57:01.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.720+0000 7f51cba00640 1 Processor -- start 2024-03-09T08:57:01.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.720+0000 7f51cba00640 1 -- start start 2024-03-09T08:57:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.721+0000 7f51cba00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c40730d0 0x7f51c414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.721+0000 7f51cba00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c40c5900 0x7f51c41ebd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.721+0000 7f51cba00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c41535e0 0x7f51c41ec280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51cba00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4153b40 con 0x7f51c40c5900 2024-03-09T08:57:01.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51cba00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c41539c0 con 0x7f51c40730d0 2024-03-09T08:57:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51cba00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40c3090 con 0x7f51c41535e0 2024-03-09T08:57:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c40c5900 0x7f51c41ebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c40c5900 0x7f51c41ebd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33756/0 (socket says 172.21.15.67:33756) 2024-03-09T08:57:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51c3e00640 1 -- 172.21.15.67:0/2411895923 learned_addr learned my addr 172.21.15.67:0/2411895923 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51c9000640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c40730d0 0x7f51c414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51c9a00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c41535e0 0x7f51c41ec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51c3e00640 1 -- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c41535e0 msgr2=0x7f51c41ec280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51c3e00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c41535e0 0x7f51c41ec280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51c3e00640 1 -- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c40730d0 msgr2=0x7f51c414df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.722+0000 7f51c3e00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51c40730d0 0x7f51c414df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.723+0000 7f51c3e00640 1 -- 172.21.15.67:0/2411895923 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c41eef60 con 0x7f51c40c5900 2024-03-09T08:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.723+0000 7f51c9a00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c41535e0 0x7f51c41ec280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.723+0000 7f51c3e00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c40c5900 0x7f51c41ebd40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f51b000c400 tx=0x7f51b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.723+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b00052d0 con 0x7f51c40c5900 2024-03-09T08:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.723+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b0005490 con 0x7f51c40c5900 2024-03-09T08:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.723+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0007260 con 0x7f51c40c5900 2024-03-09T08:57:01.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.723+0000 7f51cba00640 1 -- 172.21.15.67:0/2411895923 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c41efe10 con 0x7f51c40c5900 2024-03-09T08:57:01.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.723+0000 7f51cba00640 1 -- 172.21.15.67:0/2411895923 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c41ef4e0 con 0x7f51c40c5900 2024-03-09T08:57:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.725+0000 7f51cba00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c4159fd0 0x7f51c41ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.725+0000 7f51cba00640 1 -- 172.21.15.67:0/2411895923 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4075a00 con 0x7f51c4159fd0 2024-03-09T08:57:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.725+0000 7f51c9000640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c4159fd0 0x7f51c41ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.726+0000 7f51c9000640 1 -- 172.21.15.67:0/2411895923 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51c4153110 con 0x7f51c4159fd0 2024-03-09T08:57:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.726+0000 7f51c9000640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c4159fd0 0x7f51c41ef6e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51c40723d0 tx=0x7f51b8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.727+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8045410 con 0x7f51c4159fd0 2024-03-09T08:57:01.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.727+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f51b0007400 con 0x7f51c40c5900 2024-03-09T08:57:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.728+0000 7f51c0c00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f51a406b3d0 0x7f51a406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.728+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 8361+0+0 (secure 0 0 0) 0x7f51b008a180 con 0x7f51c40c5900 2024-03-09T08:57:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.728+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f51b8037040 con 0x7f51c4159fd0 2024-03-09T08:57:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.728+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c4159fd0 msgr2=0x7f51c41ef6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.728+0000 7f51c0c00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c4159fd0 0x7f51c41ef6e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51c40723d0 tx=0x7f51b8004730 comp rx=0 tx=0).stop 2024-03-09T08:57:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.728+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c4159fd0 msgr2=0x7f51c41ef6e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.728+0000 7f51c0c00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c4159fd0 0x7f51c41ef6e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.728+0000 7f51c9a00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f51a406b3d0 0x7f51a406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.729+0000 7f51c9a00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f51a406b3d0 0x7f51a406d890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f51c40af000 tx=0x7f51bc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.736+0000 7f51cba00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c406e9b0 0x7f51c406ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:01.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.736+0000 7f51cba00640 1 -- 172.21.15.67:0/2411895923 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40682c0 con 0x7f51c406e9b0 2024-03-09T08:57:01.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.737+0000 7f51c9000640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c406e9b0 0x7f51c406ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:01.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.738+0000 7f51c9000640 1 -- 172.21.15.67:0/2411895923 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f51c40ab020 con 0x7f51c406e9b0 2024-03-09T08:57:01.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.738+0000 7f51c9000640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c406e9b0 0x7f51c406ed90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f51b8044000 tx=0x7f51b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:01.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.739+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8050070 con 0x7f51c406e9b0 2024-03-09T08:57:01.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.739+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f51b804c2b0 con 0x7f51c406e9b0 2024-03-09T08:57:01.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.739+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c406e9b0 msgr2=0x7f51c406ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.739+0000 7f51c0c00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c406e9b0 0x7f51c406ed90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f51b8044000 tx=0x7f51b80046c0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.739+0000 7f51c0c00640 1 -- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c406e9b0 msgr2=0x7f51c406ed90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:01.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.739+0000 7f51c0c00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51c406e9b0 0x7f51c406ed90 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f51b8044000 tx=0x7f51b80046c0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.739 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:57:01.740 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:01.740 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:01.740 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:57:01.740 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 36, 2024-03-09T08:57:01.740 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:57:01.740 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:57:01.740 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:57:01.740 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:57:01.740 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.740 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T08:57:01.741 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:01.741 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:57:01.741 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:57:01.741 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:01.741 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:01.741 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:01.741 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:01.741 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:01.741 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:01.742 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:01.742 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:01.742 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:01.742 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:01.742 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:01.742 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.743 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:57:01.743 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:57:01.743 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:01.743 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:01.743 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:01.743 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:01.743 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:01.743 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:01.744 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:01.744 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:01.744 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:01.744 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:01.744 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:01.744 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.744 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:57:01.744 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:57:01.744 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:57:01.745 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:57:01.745 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:57:01.745 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:57:01.745 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:01.745 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:01.745 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:57:01.745 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:57:01.745 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:57:01.745 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:57:01.746 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:57:01.746 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:57:01.746 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:57:01.746 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:01.746 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:57:01.746 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:01.746 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:01.746 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:01.746 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:01.746 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:57:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:57:01.747 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:57:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.749 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.749 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.749 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:57:01.749 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:57:01.749 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:01.749 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:01.749 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:01.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.750 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:01.750 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:01.750 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:01.750 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:01.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.750 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:01.750 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:57:01.750 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:01.751 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:57:01.751 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.751 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.751 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.751 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:01.753 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:57:01.753 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.754 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:01.754 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:57:01.754 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:01.754 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.754 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.754 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:57:01.754 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:57:01.754 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:01.754 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:01.754 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:01.755 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:57:01.756 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:01.756 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:01.756 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:01.756 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:01.756 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:01.756 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:01.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:01.756 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:57:01.756 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:57:01.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.740+0000 7f51cba00640 1 -- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f51a406b3d0 msgr2=0x7f51a406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.740+0000 7f51cba00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f51a406b3d0 0x7f51a406d890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f51c40af000 tx=0x7f51bc00b040 comp rx=0 tx=0).stop 2024-03-09T08:57:01.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.740+0000 7f51cba00640 1 -- 172.21.15.67:0/2411895923 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c40c5900 msgr2=0x7f51c41ebd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:01.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.740+0000 7f51cba00640 1 --2- 172.21.15.67:0/2411895923 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c40c5900 0x7f51c41ebd40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f51b000c400 tx=0x7f51b000c8c0 comp rx=0 tx=0).stop 2024-03-09T08:57:01.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.740+0000 7f51c9a00640 1 -- 172.21.15.67:0/2411895923 reap_dead start 2024-03-09T08:57:01.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.740+0000 7f51c9a00640 1 -- 172.21.15.67:0/2411895923 reap_dead start 2024-03-09T08:57:01.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.740+0000 7f51cba00640 1 -- 172.21.15.67:0/2411895923 shutdown_connections 2024-03-09T08:57:01.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.740+0000 7f51cba00640 1 -- 172.21.15.67:0/2411895923 >> 172.21.15.67:0/2411895923 conn(0x7f51c40ab320 msgr2=0x7f51c40c7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:01.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.740+0000 7f51cba00640 1 -- 172.21.15.67:0/2411895923 shutdown_connections 2024-03-09T08:57:01.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.740+0000 7f51cba00640 1 -- 172.21.15.67:0/2411895923 wait complete. 2024-03-09T08:57:01.759 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:57:01.760 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T08:57:01.760 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T08:57:01.760 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T08:57:01.760 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T08:57:01.760 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T08:57:01.760 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T08:57:01.786 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T08:57:01.787 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T08:57:01.787 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:57:01.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/39' 2024-03-09T08:57:01.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/10' 2024-03-09T08:57:01.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/34' 2024-03-09T08:57:01.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/38' 2024-03-09T08:57:01.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/11' 2024-03-09T08:57:01.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/16' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/32' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/30' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/14' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/18' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/13' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/33' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/17' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/35' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './119/37' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './119' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/19' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/15' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/31' 2024-03-09T08:57:01.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/36' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/20' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/12' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/39' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/10' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/34' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/38' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/11' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/16' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/32' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/30' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/14' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/18' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/13' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/33' 2024-03-09T08:57:01.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/17' 2024-03-09T08:57:01.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/35' 2024-03-09T08:57:01.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed './120/37' 2024-03-09T08:57:01.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './120' 2024-03-09T08:57:01.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/19' 2024-03-09T08:57:01.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/15' 2024-03-09T08:57:01.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/31' 2024-03-09T08:57:01.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/36' 2024-03-09T08:57:01.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/20' 2024-03-09T08:57:01.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/12' 2024-03-09T08:57:01.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/39' 2024-03-09T08:57:01.950 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/10' 2024-03-09T08:57:01.950 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/34' 2024-03-09T08:57:01.950 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/38' 2024-03-09T08:57:01.950 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/11' 2024-03-09T08:57:01.950 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/16' 2024-03-09T08:57:01.950 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/32' 2024-03-09T08:57:01.950 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/30' 2024-03-09T08:57:01.950 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/14' 2024-03-09T08:57:01.950 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/18' 2024-03-09T08:57:01.951 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/13' 2024-03-09T08:57:01.951 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/33' 2024-03-09T08:57:01.951 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/17' 2024-03-09T08:57:01.951 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/35' 2024-03-09T08:57:01.951 INFO:tasks.workunit.client.0.smithi067.stdout:removed './121/37' 2024-03-09T08:57:01.951 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './121' 2024-03-09T08:57:01.951 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/19' 2024-03-09T08:57:01.951 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/15' 2024-03-09T08:57:01.951 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/31' 2024-03-09T08:57:01.952 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/36' 2024-03-09T08:57:01.952 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/20' 2024-03-09T08:57:01.952 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/12' 2024-03-09T08:57:01.952 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/39' 2024-03-09T08:57:01.952 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/10' 2024-03-09T08:57:01.952 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/34' 2024-03-09T08:57:01.952 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/38' 2024-03-09T08:57:01.952 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/11' 2024-03-09T08:57:01.952 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/16' 2024-03-09T08:57:01.952 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/32' 2024-03-09T08:57:01.953 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/30' 2024-03-09T08:57:01.953 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/14' 2024-03-09T08:57:01.953 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/18' 2024-03-09T08:57:01.953 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/13' 2024-03-09T08:57:01.953 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/33' 2024-03-09T08:57:01.953 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/17' 2024-03-09T08:57:01.953 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/35' 2024-03-09T08:57:01.953 INFO:tasks.workunit.client.0.smithi067.stdout:removed './122/37' 2024-03-09T08:57:01.953 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './122' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/19' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/15' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/31' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/36' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/20' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/12' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/39' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/10' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/34' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/38' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/11' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/16' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/32' 2024-03-09T08:57:01.954 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/30' 2024-03-09T08:57:01.955 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/14' 2024-03-09T08:57:01.955 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/18' 2024-03-09T08:57:01.955 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/13' 2024-03-09T08:57:01.955 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/33' 2024-03-09T08:57:01.955 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/17' 2024-03-09T08:57:01.955 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/35' 2024-03-09T08:57:01.955 INFO:tasks.workunit.client.0.smithi067.stdout:removed './123/37' 2024-03-09T08:57:01.955 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './123' 2024-03-09T08:57:01.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/19' 2024-03-09T08:57:01.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/15' 2024-03-09T08:57:01.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/31' 2024-03-09T08:57:01.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/36' 2024-03-09T08:57:01.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/20' 2024-03-09T08:57:01.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/12' 2024-03-09T08:57:01.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/39' 2024-03-09T08:57:01.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/10' 2024-03-09T08:57:01.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/34' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/38' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/11' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/16' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/32' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/30' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/14' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/18' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/13' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/33' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/17' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/35' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './124/37' 2024-03-09T08:57:01.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './124' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/19' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/15' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/31' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/36' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/20' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/12' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/39' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/10' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/34' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/38' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/11' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/16' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/32' 2024-03-09T08:57:01.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/30' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/14' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/18' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/13' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/33' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/17' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/35' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './125/37' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './125' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/19' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/15' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/31' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/36' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/20' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/12' 2024-03-09T08:57:01.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/39' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/10' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/34' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/38' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/11' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/16' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/32' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/30' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/14' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/18' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/13' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/33' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/17' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/35' 2024-03-09T08:57:01.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './126/37' 2024-03-09T08:57:01.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './126' 2024-03-09T08:57:01.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/19' 2024-03-09T08:57:01.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/15' 2024-03-09T08:57:01.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/31' 2024-03-09T08:57:01.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/36' 2024-03-09T08:57:01.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/20' 2024-03-09T08:57:01.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/12' 2024-03-09T08:57:01.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/39' 2024-03-09T08:57:01.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/10' 2024-03-09T08:57:01.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/34' 2024-03-09T08:57:01.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/38' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/11' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/16' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/32' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/30' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/14' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/18' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/13' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/33' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/17' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/35' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './127/37' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './127' 2024-03-09T08:57:01.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/19' 2024-03-09T08:57:01.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/15' 2024-03-09T08:57:01.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/31' 2024-03-09T08:57:01.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/36' 2024-03-09T08:57:01.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/20' 2024-03-09T08:57:01.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/12' 2024-03-09T08:57:01.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/39' 2024-03-09T08:57:01.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/10' 2024-03-09T08:57:01.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/34' 2024-03-09T08:57:01.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/38' 2024-03-09T08:57:01.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/11' 2024-03-09T08:57:01.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/16' 2024-03-09T08:57:01.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/32' 2024-03-09T08:57:01.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/30' 2024-03-09T08:57:01.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/14' 2024-03-09T08:57:01.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/18' 2024-03-09T08:57:01.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/13' 2024-03-09T08:57:01.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/33' 2024-03-09T08:57:01.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/17' 2024-03-09T08:57:01.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/35' 2024-03-09T08:57:01.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.998+0000 7f8e26400640 1 Processor -- start 2024-03-09T08:57:01.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:01.998+0000 7f8e26400640 1 -- start start 2024-03-09T08:57:02.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.000+0000 7f8e26400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e2006e8d0 0x7f8e2006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:02.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.000+0000 7f8e26400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e20155dc0 con 0x7f8e20159570 2024-03-09T08:57:02.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.000+0000 7f8e26400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e20156080 con 0x7f8e20150080 2024-03-09T08:57:02.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.000+0000 7f8e26400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e20156c90 con 0x7f8e2006e8d0 2024-03-09T08:57:02.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.000+0000 7f8e17e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e2006e8d0 msgr2=0x7f8e2006bab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:02.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.000+0000 7f8e17e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e2006e8d0 0x7f8e2006bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:02.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.000+0000 7f8e1fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8e20159570 0x7f8e201599d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35228/0 (socket says 172.21.15.67:35228) 2024-03-09T08:57:02.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.000+0000 7f8e1fe00640 1 -- 172.21.15.67:0/624110986 learned_addr learned my addr 172.21.15.67:0/624110986 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:02.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765479499 0 0) 0x7f8e20155dc0 con 0x7f8e20159570 2024-03-09T08:57:02.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e04003520 con 0x7f8e20159570 2024-03-09T08:57:02.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 64032625 0 0) 0x7f8e04003520 con 0x7f8e20159570 2024-03-09T08:57:02.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e2006e8d0 msgr2=0x7f8e2006bab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:57:02.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 --2- 172.21.15.67:0/624110986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e2006e8d0 0x7f8e2006bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:02.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 >> v1:172.21.15.134:6789/0 conn(0x7f8e20150080 legacy=0x7f8e200c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:02.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e20156360 con 0x7f8e20159570 2024-03-09T08:57:02.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f8e180019f0 con 0x7f8e20159570 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 >> v1:172.21.15.67:6789/0 conn(0x7f8e20159570 legacy=0x7f8e201599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 --2- 172.21.15.67:0/624110986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e2006e8d0 0x7f8e040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 --2- 172.21.15.67:0/624110986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 0x7f8e04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 --2- 172.21.15.67:0/624110986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e04004d40 0x7f8e040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e20156c90 con 0x7f8e040040d0 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0400bc20 con 0x7f8e04004d40 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.001+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0400be20 con 0x7f8e2006e8d0 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.002+0000 7f8e1f400640 1 -- 172.21.15.67:0/624110986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e2006e8d0 msgr2=0x7f8e040039e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.002+0000 7f8e1f400640 1 --2- 172.21.15.67:0/624110986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e2006e8d0 0x7f8e040039e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.002+0000 7f8e17e00640 1 --2- 172.21.15.67:0/624110986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 0x7f8e04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.002+0000 7f8e1fe00640 1 --2- 172.21.15.67:0/624110986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e04004d40 0x7f8e040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.002+0000 7f8e17e00640 1 -- 172.21.15.67:0/624110986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e2006e8d0 msgr2=0x7f8e040039e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.002+0000 7f8e17e00640 1 --2- 172.21.15.67:0/624110986 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e2006e8d0 0x7f8e040039e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:02.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.002+0000 7f8e17e00640 1 -- 172.21.15.67:0/624110986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e04004d40 msgr2=0x7f8e040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:02.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.002+0000 7f8e17e00640 1 --2- 172.21.15.67:0/624110986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e04004d40 0x7f8e040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:02.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.002+0000 7f8e17e00640 1 -- 172.21.15.67:0/624110986 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e04003520 con 0x7f8e040040d0 2024-03-09T08:57:02.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.003+0000 7f8e17e00640 1 --2- 172.21.15.67:0/624110986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 0x7f8e04004800 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009900 tx=0x7f8e04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c05600399b1949e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:02.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.003+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08009500 con 0x7f8e040040d0 2024-03-09T08:57:02.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.003+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0802f070 con 0x7f8e040040d0 2024-03-09T08:57:02.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.003+0000 7f8e1ea00640 1 -- 172.21.15.67:0/624110986 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08003df0 con 0x7f8e040040d0 2024-03-09T08:57:02.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.004+0000 7f8e26400640 1 -- 172.21.15.67:0/624110986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 msgr2=0x7f8e04004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:02.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.004+0000 7f8e26400640 1 --2- 172.21.15.67:0/624110986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 0x7f8e04004800 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009900 tx=0x7f8e04005830 comp rx=0 tx=0).stop 2024-03-09T08:57:02.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.004+0000 7f8e1fe00640 1 -- 172.21.15.67:0/624110986 reap_dead start 2024-03-09T08:57:02.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.004+0000 7f8e26400640 1 -- 172.21.15.67:0/624110986 shutdown_connections 2024-03-09T08:57:02.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.004+0000 7f8e26400640 1 -- 172.21.15.67:0/624110986 >> 172.21.15.67:0/624110986 conn(0x7f8e200aa890 msgr2=0x7f8e200ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:02.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.004+0000 7f8e26400640 1 -- 172.21.15.67:0/624110986 shutdown_connections 2024-03-09T08:57:02.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.004+0000 7f8e26400640 1 -- 172.21.15.67:0/624110986 wait complete. 2024-03-09T08:57:02.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.006+0000 7f8e26400640 1 Processor -- start 2024-03-09T08:57:02.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.006+0000 7f8e26400640 1 -- start start 2024-03-09T08:57:02.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.007+0000 7f8e26400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 0x7f8e201df810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:02.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.007+0000 7f8e26400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e200c8490 0x7f8e20153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:02.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.007+0000 7f8e26400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e201e0aa0 0x7f8e2014d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:02.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.007+0000 7f8e26400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e20150460 con 0x7f8e040040d0 2024-03-09T08:57:02.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.007+0000 7f8e26400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e20156920 con 0x7f8e201e0aa0 2024-03-09T08:57:02.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.007+0000 7f8e26400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e200c8b00 con 0x7f8e200c8490 2024-03-09T08:57:02.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.007+0000 7f8e17e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e200c8490 msgr2=0x7f8e20153c30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:02.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.007+0000 7f8e1f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 0x7f8e201df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:02.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.007+0000 7f8e17e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e200c8490 0x7f8e20153c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:02.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.007+0000 7f8e1f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 0x7f8e201df810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33774/0 (socket says 172.21.15.67:33774) 2024-03-09T08:57:02.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.007+0000 7f8e1f400640 1 -- 172.21.15.67:0/1801997954 learned_addr learned my addr 172.21.15.67:0/1801997954 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:02.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.008+0000 7f8e1fe00640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e201e0aa0 0x7f8e2014d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:02.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.008+0000 7f8e1f400640 1 -- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e200c8490 msgr2=0x7f8e20153c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:57:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.008+0000 7f8e1f400640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e200c8490 0x7f8e20153c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.008+0000 7f8e1f400640 1 -- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e201e0aa0 msgr2=0x7f8e2014d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.008+0000 7f8e1f400640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e201e0aa0 0x7f8e2014d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.008+0000 7f8e1f400640 1 -- 172.21.15.67:0/1801997954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e201dfd50 con 0x7f8e040040d0 2024-03-09T08:57:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.008+0000 7f8e1f400640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 0x7f8e201df810 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c780 tx=0x7f8e1000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.008+0000 7f8e17400640 1 -- 172.21.15.67:0/1801997954 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10007270 con 0x7f8e040040d0 2024-03-09T08:57:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.009+0000 7f8e17400640 1 -- 172.21.15.67:0/1801997954 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e10007430 con 0x7f8e040040d0 2024-03-09T08:57:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.009+0000 7f8e17400640 1 -- 172.21.15.67:0/1801997954 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100057c0 con 0x7f8e040040d0 2024-03-09T08:57:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.009+0000 7f8e26400640 1 -- 172.21.15.67:0/1801997954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e201dea70 con 0x7f8e040040d0 2024-03-09T08:57:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.009+0000 7f8e26400640 1 -- 172.21.15.67:0/1801997954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e201ded30 con 0x7f8e040040d0 2024-03-09T08:57:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.010+0000 7f8e26400640 1 -- 172.21.15.67:0/1801997954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e20075a00 con 0x7f8e040040d0 2024-03-09T08:57:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.012+0000 7f8e17400640 1 -- 172.21.15.67:0/1801997954 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8e10005960 con 0x7f8e040040d0 2024-03-09T08:57:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.014+0000 7f8e17400640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dfc06b4b0 0x7f8dfc06d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.014+0000 7f8e17400640 1 -- 172.21.15.67:0/1801997954 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 8361+0+0 (secure 0 0 0) 0x7f8e1008a810 con 0x7f8e040040d0 2024-03-09T08:57:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.014+0000 7f8e17e00640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dfc06b4b0 0x7f8dfc06d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:02.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.015+0000 7f8e17e00640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dfc06b4b0 0x7f8dfc06d970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8e08033000 tx=0x7f8e08038040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.018+0000 7f8e17400640 1 -- 172.21.15.67:0/1801997954 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8e10058cd0 con 0x7f8e040040d0 2024-03-09T08:57:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.363+0000 7f8e26400640 1 -- 172.21.15.67:0/1801997954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8e20154170 con 0x7f8e040040d0 2024-03-09T08:57:02.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.364+0000 7f8e17400640 1 -- 172.21.15.67:0/1801997954 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f8e100585d0 con 0x7f8e040040d0 2024-03-09T08:57:02.363 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:57:02.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.366+0000 7f8e26400640 1 -- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dfc06b4b0 msgr2=0x7f8dfc06d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:02.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.367+0000 7f8e26400640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dfc06b4b0 0x7f8dfc06d970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8e08033000 tx=0x7f8e08038040 comp rx=0 tx=0).stop 2024-03-09T08:57:02.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.367+0000 7f8e26400640 1 -- 172.21.15.67:0/1801997954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 msgr2=0x7f8e201df810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:02.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.367+0000 7f8e26400640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 0x7f8e201df810 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c780 tx=0x7f8e1000cc40 comp rx=0 tx=0).stop 2024-03-09T08:57:02.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.367+0000 7f8e26400640 1 -- 172.21.15.67:0/1801997954 shutdown_connections 2024-03-09T08:57:02.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.367+0000 7f8e26400640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dfc06b4b0 0x7f8dfc06d970 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:02.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.367+0000 7f8e26400640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e201e0aa0 0x7f8e2014d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.367+0000 7f8e26400640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e200c8490 0x7f8e20153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.367+0000 7f8e26400640 1 --2- 172.21.15.67:0/1801997954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e040040d0 0x7f8e201df810 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.367+0000 7f8e26400640 1 -- 172.21.15.67:0/1801997954 >> 172.21.15.67:0/1801997954 conn(0x7f8e200aa890 msgr2=0x7f8e2014ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.367+0000 7f8e26400640 1 -- 172.21.15.67:0/1801997954 shutdown_connections 2024-03-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:02.367+0000 7f8e26400640 1 -- 172.21.15.67:0/1801997954 wait complete. 2024-03-09T08:57:02.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './128/37' 2024-03-09T08:57:02.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './128' 2024-03-09T08:57:02.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/19' 2024-03-09T08:57:02.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/15' 2024-03-09T08:57:02.956 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/31' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/36' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/20' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/12' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/39' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/10' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/34' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/38' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/11' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/16' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/32' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/30' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/14' 2024-03-09T08:57:02.957 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/18' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/13' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/33' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/17' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/35' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './129/37' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './129' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/19' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/15' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/31' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/36' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/20' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/12' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/39' 2024-03-09T08:57:02.958 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/10' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/34' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/38' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/11' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/16' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/32' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/30' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/14' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/18' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/13' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/33' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/17' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/35' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './130/37' 2024-03-09T08:57:02.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './130' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/19' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/15' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/31' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/36' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/20' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/12' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/39' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/10' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/34' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/38' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/11' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/16' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/32' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/30' 2024-03-09T08:57:02.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/14' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/18' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/13' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/33' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/17' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/35' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './131/37' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './131' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/19' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/15' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/31' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/36' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/20' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/12' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/39' 2024-03-09T08:57:02.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/10' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/34' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/38' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/11' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/16' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/32' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/30' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/14' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/18' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/13' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/33' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/17' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/35' 2024-03-09T08:57:02.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './132/37' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './132' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/19' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/15' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/31' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/36' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/20' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/12' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/39' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/10' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/34' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/38' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/11' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/16' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/32' 2024-03-09T08:57:02.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/30' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/14' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/18' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/13' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/33' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/17' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/35' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './133/37' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './133' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/19' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/15' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/31' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/36' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/20' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/12' 2024-03-09T08:57:02.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/39' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/10' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/34' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/38' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/11' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/16' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/32' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/30' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/14' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/18' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/13' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/33' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/17' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/35' 2024-03-09T08:57:02.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './134/37' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './134' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/19' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/15' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/31' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/36' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/20' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/12' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/39' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/10' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/34' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/38' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/11' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/16' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/32' 2024-03-09T08:57:02.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/30' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/14' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/18' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/13' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/33' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/17' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/35' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './135/37' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './135' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/19' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/15' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/31' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/36' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/20' 2024-03-09T08:57:02.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/12' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/39' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/10' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/34' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/38' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/11' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/16' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/32' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/30' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/14' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/18' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/13' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/33' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/17' 2024-03-09T08:57:02.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/35' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './136/37' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './136' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/19' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/15' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/31' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/36' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/20' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/12' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/39' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/10' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/34' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/38' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/11' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/16' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/32' 2024-03-09T08:57:02.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/30' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/14' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/18' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/13' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/33' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/17' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/35' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './137/37' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './137' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/19' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/15' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/31' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/36' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/20' 2024-03-09T08:57:02.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/12' 2024-03-09T08:57:02.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/39' 2024-03-09T08:57:02.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/10' 2024-03-09T08:57:02.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/34' 2024-03-09T08:57:02.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/38' 2024-03-09T08:57:02.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/11' 2024-03-09T08:57:02.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/16' 2024-03-09T08:57:04.254 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/39' 2024-03-09T08:57:04.254 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/10' 2024-03-09T08:57:04.254 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/34' 2024-03-09T08:57:04.254 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/38' 2024-03-09T08:57:04.254 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/11' 2024-03-09T08:57:04.254 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/16' 2024-03-09T08:57:04.254 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/32' 2024-03-09T08:57:04.254 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/30' 2024-03-09T08:57:04.254 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/14' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/18' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/13' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/33' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/17' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/35' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed './119/37' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './119' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/19' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/15' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/31' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/36' 2024-03-09T08:57:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/20' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/12' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/39' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/10' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/34' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/38' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/11' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/16' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/32' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/30' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/14' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/18' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/13' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/33' 2024-03-09T08:57:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/17' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/35' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './120/37' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './120' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/19' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/15' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/31' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/36' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/20' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/12' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/39' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/10' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/34' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/38' 2024-03-09T08:57:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/11' 2024-03-09T08:57:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/16' 2024-03-09T08:57:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/32' 2024-03-09T08:57:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/30' 2024-03-09T08:57:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/14' 2024-03-09T08:57:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/18' 2024-03-09T08:57:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/13' 2024-03-09T08:57:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/33' 2024-03-09T08:57:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/17' 2024-03-09T08:57:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/35' 2024-03-09T08:57:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:removed './121/37' 2024-03-09T08:57:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './121' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/19' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/15' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/31' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/36' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/20' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/12' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/39' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/10' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/34' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/38' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/11' 2024-03-09T08:57:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/16' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/32' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/30' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/14' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/18' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/13' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/33' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/17' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/35' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './122/37' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './122' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/19' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/15' 2024-03-09T08:57:04.260 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/31' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/36' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/20' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/12' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/39' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/10' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/34' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/38' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/11' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/16' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/32' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/30' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/14' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/18' 2024-03-09T08:57:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/13' 2024-03-09T08:57:04.262 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/33' 2024-03-09T08:57:04.262 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/17' 2024-03-09T08:57:04.262 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/35' 2024-03-09T08:57:04.262 INFO:tasks.workunit.client.1.smithi134.stdout:removed './123/37' 2024-03-09T08:57:04.262 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './123' 2024-03-09T08:57:04.262 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/19' 2024-03-09T08:57:04.262 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/15' 2024-03-09T08:57:04.262 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/31' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/36' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/20' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/12' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/39' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/10' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/34' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/38' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/11' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/16' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/32' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/30' 2024-03-09T08:57:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/14' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/18' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/13' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/33' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/17' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/35' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './124/37' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './124' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/19' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/15' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/31' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/36' 2024-03-09T08:57:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/20' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/12' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/39' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/10' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/34' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/38' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/11' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/16' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/32' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/30' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/14' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/18' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/13' 2024-03-09T08:57:04.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/33' 2024-03-09T08:57:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/17' 2024-03-09T08:57:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/35' 2024-03-09T08:57:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './125/37' 2024-03-09T08:57:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './125' 2024-03-09T08:57:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/19' 2024-03-09T08:57:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/15' 2024-03-09T08:57:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/31' 2024-03-09T08:57:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/36' 2024-03-09T08:57:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/20' 2024-03-09T08:57:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/12' 2024-03-09T08:57:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/39' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/10' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/34' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/38' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/11' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/16' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/32' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/30' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/14' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/18' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/13' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/33' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/17' 2024-03-09T08:57:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/35' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './126/37' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './126' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/19' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/15' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/31' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/36' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/20' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/12' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/39' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/10' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/34' 2024-03-09T08:57:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/38' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/11' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/16' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/32' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/30' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/14' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/18' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/13' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/33' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/17' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/35' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './127/37' 2024-03-09T08:57:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './127' 2024-03-09T08:57:04.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/19' 2024-03-09T08:57:04.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/15' 2024-03-09T08:57:04.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/31' 2024-03-09T08:57:04.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/36' 2024-03-09T08:57:04.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/20' 2024-03-09T08:57:04.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/12' 2024-03-09T08:57:04.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/39' 2024-03-09T08:57:04.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/10' 2024-03-09T08:57:04.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/34' 2024-03-09T08:57:04.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/38' 2024-03-09T08:57:04.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/11' 2024-03-09T08:57:04.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/16' 2024-03-09T08:57:04.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/32' 2024-03-09T08:57:04.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/30' 2024-03-09T08:57:04.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/14' 2024-03-09T08:57:04.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/18' 2024-03-09T08:57:04.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/13' 2024-03-09T08:57:04.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/33' 2024-03-09T08:57:04.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/17' 2024-03-09T08:57:04.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/35' 2024-03-09T08:57:05.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/32' 2024-03-09T08:57:05.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/30' 2024-03-09T08:57:05.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/14' 2024-03-09T08:57:05.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/18' 2024-03-09T08:57:05.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/13' 2024-03-09T08:57:05.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/33' 2024-03-09T08:57:05.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/17' 2024-03-09T08:57:05.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/35' 2024-03-09T08:57:05.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './138/37' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './138' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/19' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/15' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/31' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/36' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/20' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/12' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/39' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/10' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/34' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/38' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/11' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/16' 2024-03-09T08:57:05.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/32' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/30' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/14' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/18' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/13' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/33' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/17' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/35' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './139/37' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './139' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/19' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/15' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/31' 2024-03-09T08:57:05.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/36' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/20' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/12' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/39' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/10' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/34' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/38' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/11' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/16' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/32' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/30' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/14' 2024-03-09T08:57:05.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/18' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/13' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/33' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/17' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/35' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './140/37' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './140' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/19' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/15' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/31' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/36' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/20' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/12' 2024-03-09T08:57:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/39' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/10' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/34' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/38' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/11' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/16' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/32' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/30' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/14' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/18' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/13' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/33' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/17' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/35' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './141/37' 2024-03-09T08:57:05.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './141' 2024-03-09T08:57:05.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/19' 2024-03-09T08:57:05.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/15' 2024-03-09T08:57:05.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/31' 2024-03-09T08:57:05.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/36' 2024-03-09T08:57:05.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/20' 2024-03-09T08:57:05.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/12' 2024-03-09T08:57:05.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/39' 2024-03-09T08:57:05.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/10' 2024-03-09T08:57:05.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/34' 2024-03-09T08:57:05.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/38' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/11' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/16' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/32' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/30' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/14' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/18' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/13' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/33' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/17' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/35' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './142/37' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './142' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/19' 2024-03-09T08:57:05.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/15' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/31' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/36' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/20' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/12' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/39' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/10' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/34' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/38' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/11' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/16' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/32' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/30' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/14' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/18' 2024-03-09T08:57:05.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/13' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/33' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/17' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/35' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './143/37' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './143' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/19' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/15' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/31' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/36' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/20' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/12' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/39' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/10' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/34' 2024-03-09T08:57:05.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/38' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/11' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/16' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/32' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/30' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/14' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/18' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/13' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/33' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/17' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/35' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './144/37' 2024-03-09T08:57:05.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './144' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/19' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/15' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/31' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/36' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/20' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/12' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/39' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/10' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/34' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/38' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/11' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/16' 2024-03-09T08:57:05.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/32' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/30' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/14' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/18' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/13' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/33' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/17' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/35' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './145/37' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './145' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/19' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/15' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/31' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/36' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/20' 2024-03-09T08:57:05.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/12' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/39' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/10' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/34' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/38' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/11' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/16' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/32' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/30' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/14' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/18' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/13' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/33' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/17' 2024-03-09T08:57:05.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/35' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './146/37' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './146' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/19' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/15' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/31' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/36' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/20' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/12' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/39' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/10' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/34' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/38' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/11' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/16' 2024-03-09T08:57:05.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/32' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/30' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/14' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/18' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/13' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/33' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/17' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/35' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './147/37' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './147' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/19' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/15' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/31' 2024-03-09T08:57:05.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/36' 2024-03-09T08:57:05.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:57:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.585+0000 7fddb0e00640 1 Processor -- start 2024-03-09T08:57:05.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.585+0000 7fddb0e00640 1 -- start start 2024-03-09T08:57:05.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.587+0000 7fddb0e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fddac153370 0x7fddac0c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:05.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.587+0000 7fddb0e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddac0c4010 con 0x7fddac06dc00 2024-03-09T08:57:05.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.587+0000 7fddb0e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddac0c42f0 con 0x7fddac153750 2024-03-09T08:57:05.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.587+0000 7fddb0e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddac156dc0 con 0x7fddac153370 2024-03-09T08:57:05.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.587+0000 7fdda9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fddac153370 0x7fddac0c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:05.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.587+0000 7fdda9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fddac153370 0x7fddac0c29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33776/0 (socket says 172.21.15.67:33776) 2024-03-09T08:57:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.587+0000 7fdda9e00640 1 -- 172.21.15.67:0/500823060 learned_addr learned my addr 172.21.15.67:0/500823060 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.588+0000 7fdda9e00640 1 -- 172.21.15.67:0/500823060 >> v1:172.21.15.134:6789/0 conn(0x7fddac153750 legacy=0x7fddac0c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.588+0000 7fdda9e00640 1 -- 172.21.15.67:0/500823060 >> v1:172.21.15.67:6789/0 conn(0x7fddac06dc00 legacy=0x7fddac0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.588+0000 7fdda9e00640 1 -- 172.21.15.67:0/500823060 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac0c5a50 con 0x7fddac153370 2024-03-09T08:57:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.589+0000 7fdda9e00640 1 --2- 172.21.15.67:0/500823060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddac153370 0x7fddac0c29a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009870 tx=0x7fdd9c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34f7a9380b6b4d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.589+0000 7fdda8a00640 1 -- 172.21.15.67:0/500823060 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c035020 con 0x7fddac153370 2024-03-09T08:57:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.589+0000 7fdda8a00640 1 -- 172.21.15.67:0/500823060 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9c02d040 con 0x7fddac153370 2024-03-09T08:57:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.589+0000 7fdda8a00640 1 -- 172.21.15.67:0/500823060 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c031a80 con 0x7fddac153370 2024-03-09T08:57:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.590+0000 7fddb0e00640 1 -- 172.21.15.67:0/500823060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddac153370 msgr2=0x7fddac0c29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.590+0000 7fddb0e00640 1 --2- 172.21.15.67:0/500823060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddac153370 0x7fddac0c29a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009870 tx=0x7fdd9c02b900 comp rx=0 tx=0).stop 2024-03-09T08:57:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.590+0000 7fddb0e00640 1 -- 172.21.15.67:0/500823060 shutdown_connections 2024-03-09T08:57:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.590+0000 7fddb0e00640 1 --2- 172.21.15.67:0/500823060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddac153370 0x7fddac0c29a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.590+0000 7fddb0e00640 1 -- 172.21.15.67:0/500823060 >> 172.21.15.67:0/500823060 conn(0x7fddac0ab320 msgr2=0x7fddac0ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.590+0000 7fddb0e00640 1 -- 172.21.15.67:0/500823060 shutdown_connections 2024-03-09T08:57:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.590+0000 7fddb0e00640 1 -- 172.21.15.67:0/500823060 wait complete. 2024-03-09T08:57:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.591+0000 7fddb0e00640 1 Processor -- start 2024-03-09T08:57:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.592+0000 7fddb0e00640 1 -- start start 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.592+0000 7fddb0e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddac06dc00 0x7fddac06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.592+0000 7fddb0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddac153370 0x7fddac14ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.592+0000 7fddb0e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac153750 0x7fddac1504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.592+0000 7fddb0e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddac0c5d30 con 0x7fddac153370 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.592+0000 7fddb0e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddac073dc0 con 0x7fddac06dc00 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.592+0000 7fddb0e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddac0730d0 con 0x7fddac153750 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.593+0000 7fddaa800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac153750 msgr2=0x7fddac1504e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.593+0000 7fddaa800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac153750 0x7fddac1504e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.593+0000 7fdda9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddac153370 0x7fddac14ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.593+0000 7fdda9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddac153370 0x7fddac14ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33792/0 (socket says 172.21.15.67:33792) 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.593+0000 7fdda9400640 1 -- 172.21.15.67:0/124445041 learned_addr learned my addr 172.21.15.67:0/124445041 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.593+0000 7fdda9e00640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddac06dc00 0x7fddac06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.593+0000 7fdda9400640 1 -- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac153750 msgr2=0x7fddac1504e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.593+0000 7fdda9400640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac153750 0x7fddac1504e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.593+0000 7fdda9400640 1 -- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddac06dc00 msgr2=0x7fddac06c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.593+0000 7fdda9400640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddac06dc00 0x7fddac06c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.593+0000 7fdda9400640 1 -- 172.21.15.67:0/124445041 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac14cf40 con 0x7fddac153370 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.594+0000 7fdda9400640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddac153370 0x7fddac14ffa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c430 tx=0x7fdd9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.594+0000 7fdda2000640 1 -- 172.21.15.67:0/124445041 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98011790 con 0x7fddac153370 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.594+0000 7fdda2000640 1 -- 172.21.15.67:0/124445041 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd98017070 con 0x7fddac153370 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.594+0000 7fddb0e00640 1 -- 172.21.15.67:0/124445041 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddac1efb90 con 0x7fddac153370 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.594+0000 7fdda2000640 1 -- 172.21.15.67:0/124445041 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98011b70 con 0x7fddac153370 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.594+0000 7fddb0e00640 1 -- 172.21.15.67:0/124445041 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddac06ce60 con 0x7fddac153370 2024-03-09T08:57:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.595+0000 7fddb0e00640 1 -- 172.21.15.67:0/124445041 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddac14d210 con 0x7fddac153370 2024-03-09T08:57:05.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.596+0000 7fdda2000640 1 -- 172.21.15.67:0/124445041 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdd98011d10 con 0x7fddac153370 2024-03-09T08:57:05.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.598+0000 7fdda2000640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd8c06b4b0 0x7fdd8c06d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:05.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.598+0000 7fdda2000640 1 -- 172.21.15.67:0/124445041 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 8361+0+0 (secure 0 0 0) 0x7fdd9808aaf0 con 0x7fddac153370 2024-03-09T08:57:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.599+0000 7fdda9e00640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd8c06b4b0 0x7fdd8c06d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.600+0000 7fdda9e00640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd8c06b4b0 0x7fdd8c06d970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c036000 tx=0x7fdd9c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.603+0000 7fdda2000640 1 -- 172.21.15.67:0/124445041 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdd98058ec0 con 0x7fddac153370 2024-03-09T08:57:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:removed './128/37' 2024-03-09T08:57:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './128' 2024-03-09T08:57:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/19' 2024-03-09T08:57:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/15' 2024-03-09T08:57:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/31' 2024-03-09T08:57:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/36' 2024-03-09T08:57:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/20' 2024-03-09T08:57:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/12' 2024-03-09T08:57:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/39' 2024-03-09T08:57:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/10' 2024-03-09T08:57:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/34' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/38' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/11' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/16' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/32' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/30' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/14' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/18' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/13' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/33' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/17' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/35' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './129/37' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './129' 2024-03-09T08:57:05.722 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/19' 2024-03-09T08:57:05.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/15' 2024-03-09T08:57:05.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/31' 2024-03-09T08:57:05.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/36' 2024-03-09T08:57:05.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/20' 2024-03-09T08:57:05.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/12' 2024-03-09T08:57:05.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/39' 2024-03-09T08:57:05.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/10' 2024-03-09T08:57:05.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/34' 2024-03-09T08:57:05.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/38' 2024-03-09T08:57:05.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/11' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/16' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/32' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/30' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/14' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/18' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/13' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/33' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/17' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/35' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './130/37' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './130' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/19' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/15' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/31' 2024-03-09T08:57:05.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/36' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/20' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/12' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/39' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/10' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/34' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/38' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/11' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/16' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/32' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/30' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/14' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/18' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/13' 2024-03-09T08:57:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/33' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/17' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/35' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './131/37' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './131' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/19' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/15' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/31' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/36' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/20' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/12' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/39' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/10' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/34' 2024-03-09T08:57:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/38' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/11' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/16' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/32' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/30' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/14' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/18' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/13' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/33' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/17' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/35' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './132/37' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './132' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/19' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/15' 2024-03-09T08:57:05.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/31' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/36' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/20' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/12' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/39' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/10' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/34' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/38' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/11' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/16' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/32' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/30' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/14' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/18' 2024-03-09T08:57:05.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/13' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/33' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/17' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/35' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './133/37' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './133' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/19' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/15' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/31' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/36' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/20' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/12' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/39' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/10' 2024-03-09T08:57:05.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/34' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/38' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/11' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/16' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/32' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/30' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/14' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/18' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/13' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/33' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/17' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/35' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed './134/37' 2024-03-09T08:57:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './134' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/19' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/15' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/31' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/36' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/20' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/12' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/39' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/10' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/34' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/38' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/11' 2024-03-09T08:57:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/16' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/32' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/30' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/14' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/18' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/13' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/33' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/17' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/35' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './135/37' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './135' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/19' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/15' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/31' 2024-03-09T08:57:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/36' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/20' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/12' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/39' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/10' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/34' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/38' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/11' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/16' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/32' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/30' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/14' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/18' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/13' 2024-03-09T08:57:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/33' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/17' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/35' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './136/37' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './136' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/19' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/15' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/31' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/36' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/20' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/12' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/39' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/10' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/34' 2024-03-09T08:57:05.734 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/38' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/11' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/16' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/32' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/30' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/14' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/18' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/13' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/33' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/17' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/35' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './137/37' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './137' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/19' 2024-03-09T08:57:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/15' 2024-03-09T08:57:05.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/31' 2024-03-09T08:57:05.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/36' 2024-03-09T08:57:05.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/20' 2024-03-09T08:57:05.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/12' 2024-03-09T08:57:05.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/39' 2024-03-09T08:57:05.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/10' 2024-03-09T08:57:05.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/34' 2024-03-09T08:57:05.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/38' 2024-03-09T08:57:05.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/11' 2024-03-09T08:57:05.736 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/16' 2024-03-09T08:57:05.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.984+0000 7fddb0e00640 1 -- 172.21.15.67:0/124445041 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fddac075a00 con 0x7fddac153370 2024-03-09T08:57:05.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.985+0000 7fdda2000640 1 -- 172.21.15.67:0/124445041 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fdd980587c0 con 0x7fddac153370 2024-03-09T08:57:05.984 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:57:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 -- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd8c06b4b0 msgr2=0x7fdd8c06d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd8c06b4b0 0x7fdd8c06d970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c036000 tx=0x7fdd9c03c040 comp rx=0 tx=0).stop 2024-03-09T08:57:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 -- 172.21.15.67:0/124445041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddac153370 msgr2=0x7fddac14ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddac153370 0x7fddac14ffa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800c430 tx=0x7fdd9800c8f0 comp rx=0 tx=0).stop 2024-03-09T08:57:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 -- 172.21.15.67:0/124445041 shutdown_connections 2024-03-09T08:57:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd8c06b4b0 0x7fdd8c06d970 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddac153750 0x7fddac1504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddac153370 0x7fddac14ffa0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 --2- 172.21.15.67:0/124445041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddac06dc00 0x7fddac06c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 -- 172.21.15.67:0/124445041 >> 172.21.15.67:0/124445041 conn(0x7fddac0ab320 msgr2=0x7fddac0a9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 -- 172.21.15.67:0/124445041 shutdown_connections 2024-03-09T08:57:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:05.988+0000 7fddb0e00640 1 -- 172.21.15.67:0/124445041 wait complete. 2024-03-09T08:57:07.243 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/20' 2024-03-09T08:57:07.244 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/12' 2024-03-09T08:57:07.244 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/39' 2024-03-09T08:57:07.244 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/10' 2024-03-09T08:57:07.244 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/34' 2024-03-09T08:57:07.244 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/38' 2024-03-09T08:57:07.244 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/11' 2024-03-09T08:57:07.244 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/16' 2024-03-09T08:57:07.244 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/32' 2024-03-09T08:57:07.244 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/30' 2024-03-09T08:57:07.244 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/14' 2024-03-09T08:57:07.244 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/18' 2024-03-09T08:57:07.245 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/13' 2024-03-09T08:57:07.245 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/33' 2024-03-09T08:57:07.245 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/17' 2024-03-09T08:57:07.245 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/35' 2024-03-09T08:57:07.245 INFO:tasks.workunit.client.0.smithi067.stdout:removed './148/37' 2024-03-09T08:57:07.245 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './148' 2024-03-09T08:57:07.245 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/19' 2024-03-09T08:57:07.245 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/15' 2024-03-09T08:57:07.245 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/31' 2024-03-09T08:57:07.245 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/36' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/20' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/12' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/39' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/10' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/34' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/38' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/11' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/16' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/32' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/30' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/14' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/18' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/13' 2024-03-09T08:57:07.246 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/33' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/17' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/35' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './149/37' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './149' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/19' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/15' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/31' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/36' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/20' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/12' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/39' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/10' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/34' 2024-03-09T08:57:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/38' 2024-03-09T08:57:07.248 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/11' 2024-03-09T08:57:07.248 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/16' 2024-03-09T08:57:07.248 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/32' 2024-03-09T08:57:07.249 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/30' 2024-03-09T08:57:07.249 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/14' 2024-03-09T08:57:07.249 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/18' 2024-03-09T08:57:07.249 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/13' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/33' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/17' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/35' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './150/37' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './150' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/19' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/15' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/31' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/36' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/20' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/12' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/39' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/10' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/34' 2024-03-09T08:57:07.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/38' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/11' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/16' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/32' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/30' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/14' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/18' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/13' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/33' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/17' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/35' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './151/37' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './151' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/19' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/15' 2024-03-09T08:57:07.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/31' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/36' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/20' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/12' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/39' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/10' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/34' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/38' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/11' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/16' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/32' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/30' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/14' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/18' 2024-03-09T08:57:07.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/13' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/33' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/17' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/35' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './152/37' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './152' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/19' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/15' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/31' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/36' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/20' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/12' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/39' 2024-03-09T08:57:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/10' 2024-03-09T08:57:07.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/34' 2024-03-09T08:57:07.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/38' 2024-03-09T08:57:07.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/11' 2024-03-09T08:57:07.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/16' 2024-03-09T08:57:07.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/32' 2024-03-09T08:57:07.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/30' 2024-03-09T08:57:07.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/14' 2024-03-09T08:57:07.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/18' 2024-03-09T08:57:07.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/13' 2024-03-09T08:57:07.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/33' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/17' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/35' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './153/37' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './153' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/19' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/15' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/31' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/36' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/20' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/12' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/39' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/10' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/34' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/38' 2024-03-09T08:57:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/11' 2024-03-09T08:57:07.256 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/16' 2024-03-09T08:57:07.256 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/32' 2024-03-09T08:57:07.256 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/30' 2024-03-09T08:57:07.256 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/14' 2024-03-09T08:57:07.256 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/18' 2024-03-09T08:57:07.256 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/13' 2024-03-09T08:57:07.256 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/33' 2024-03-09T08:57:07.256 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/17' 2024-03-09T08:57:07.256 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/35' 2024-03-09T08:57:07.256 INFO:tasks.workunit.client.0.smithi067.stdout:removed './154/37' 2024-03-09T08:57:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './154' 2024-03-09T08:57:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/19' 2024-03-09T08:57:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/15' 2024-03-09T08:57:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/31' 2024-03-09T08:57:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/36' 2024-03-09T08:57:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/20' 2024-03-09T08:57:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/12' 2024-03-09T08:57:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/39' 2024-03-09T08:57:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/10' 2024-03-09T08:57:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/34' 2024-03-09T08:57:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/38' 2024-03-09T08:57:07.258 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/11' 2024-03-09T08:57:07.258 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/16' 2024-03-09T08:57:07.258 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/32' 2024-03-09T08:57:07.258 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/30' 2024-03-09T08:57:07.258 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/14' 2024-03-09T08:57:07.258 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/18' 2024-03-09T08:57:07.258 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/13' 2024-03-09T08:57:07.258 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/33' 2024-03-09T08:57:07.258 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/17' 2024-03-09T08:57:07.258 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/35' 2024-03-09T08:57:07.258 INFO:tasks.workunit.client.0.smithi067.stdout:removed './155/37' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './155' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/19' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/15' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/31' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/36' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/20' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/12' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/39' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/10' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/34' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/38' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/11' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/16' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/32' 2024-03-09T08:57:07.259 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/30' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/14' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/18' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/13' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/33' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/17' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/35' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './156/37' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './156' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/19' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/15' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/31' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/36' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/20' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/12' 2024-03-09T08:57:07.260 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/39' 2024-03-09T08:57:07.261 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/10' 2024-03-09T08:57:07.261 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/34' 2024-03-09T08:57:07.261 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/38' 2024-03-09T08:57:07.261 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/11' 2024-03-09T08:57:07.261 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/16' 2024-03-09T08:57:07.261 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/32' 2024-03-09T08:57:07.261 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/30' 2024-03-09T08:57:07.261 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/14' 2024-03-09T08:57:07.261 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/18' 2024-03-09T08:57:07.261 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/13' 2024-03-09T08:57:07.261 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/33' 2024-03-09T08:57:07.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/32' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/30' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/14' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/18' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/13' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/33' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/17' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/35' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './138/37' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './138' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/19' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/15' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/31' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/36' 2024-03-09T08:57:07.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/20' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/12' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/39' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/10' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/34' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/38' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/11' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/16' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/32' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/30' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/14' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/18' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/13' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/33' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/17' 2024-03-09T08:57:07.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/35' 2024-03-09T08:57:07.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './139/37' 2024-03-09T08:57:07.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './139' 2024-03-09T08:57:07.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/19' 2024-03-09T08:57:07.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/15' 2024-03-09T08:57:07.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/31' 2024-03-09T08:57:07.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/36' 2024-03-09T08:57:07.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/20' 2024-03-09T08:57:07.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/12' 2024-03-09T08:57:07.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/39' 2024-03-09T08:57:07.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/10' 2024-03-09T08:57:07.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/34' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/38' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/11' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/16' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/32' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/30' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/14' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/18' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/13' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/33' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/17' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/35' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './140/37' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './140' 2024-03-09T08:57:07.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/19' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/15' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/31' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/36' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/20' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/12' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/39' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/10' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/34' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/38' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/11' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/16' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/32' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/30' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/14' 2024-03-09T08:57:07.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/18' 2024-03-09T08:57:07.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/13' 2024-03-09T08:57:07.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/33' 2024-03-09T08:57:07.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/17' 2024-03-09T08:57:07.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/35' 2024-03-09T08:57:07.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './141/37' 2024-03-09T08:57:07.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './141' 2024-03-09T08:57:07.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/19' 2024-03-09T08:57:07.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/15' 2024-03-09T08:57:07.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/31' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/36' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/20' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/12' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/39' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/10' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/34' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/38' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/11' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/16' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/32' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/30' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/14' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/18' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/13' 2024-03-09T08:57:07.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/33' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/17' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/35' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './142/37' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './142' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/19' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/15' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/31' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/36' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/20' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/12' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/39' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/10' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/34' 2024-03-09T08:57:07.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/38' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/11' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/16' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/32' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/30' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/14' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/18' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/13' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/33' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/17' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/35' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './143/37' 2024-03-09T08:57:07.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './143' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/19' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/15' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/31' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/36' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/20' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/12' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/39' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/10' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/34' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/38' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/11' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/16' 2024-03-09T08:57:07.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/32' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/30' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/14' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/18' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/13' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/33' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/17' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/35' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './144/37' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './144' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/19' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/15' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/31' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/36' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/20' 2024-03-09T08:57:07.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/12' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/39' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/10' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/34' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/38' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/11' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/16' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/32' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/30' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/14' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/18' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/13' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/33' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/17' 2024-03-09T08:57:07.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/35' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './145/37' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './145' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/19' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/15' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/31' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/36' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/20' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/12' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/39' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/10' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/34' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/38' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/11' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/16' 2024-03-09T08:57:07.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/32' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/30' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/14' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/18' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/13' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/33' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/17' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/35' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './146/37' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './146' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/19' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/15' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/31' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/36' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/20' 2024-03-09T08:57:07.324 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/12' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/39' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/10' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/34' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/38' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/11' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/16' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/32' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/30' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/14' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/18' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/13' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/33' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/17' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/35' 2024-03-09T08:57:07.325 INFO:tasks.workunit.client.1.smithi134.stdout:removed './147/37' 2024-03-09T08:57:07.326 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './147' 2024-03-09T08:57:07.326 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/19' 2024-03-09T08:57:07.326 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/15' 2024-03-09T08:57:07.326 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/31' 2024-03-09T08:57:07.326 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/36' 2024-03-09T08:57:08.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/17' 2024-03-09T08:57:08.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/35' 2024-03-09T08:57:08.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './157/37' 2024-03-09T08:57:08.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './157' 2024-03-09T08:57:08.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/19' 2024-03-09T08:57:08.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/15' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/31' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/36' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/20' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/12' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/39' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/10' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/34' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/38' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/11' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/16' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/32' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/30' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/14' 2024-03-09T08:57:08.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/18' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/13' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/33' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/17' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/35' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './158/37' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './158' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/19' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/15' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/31' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/36' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/20' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/12' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/39' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/10' 2024-03-09T08:57:08.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/34' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/38' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/11' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/16' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/32' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/30' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/14' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/18' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/13' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/33' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/17' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/35' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './159/37' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './159' 2024-03-09T08:57:08.210 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/19' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/15' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/31' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/36' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/20' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/12' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/39' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/10' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/34' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/38' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/11' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/16' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/32' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/30' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/14' 2024-03-09T08:57:08.211 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/18' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/13' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/33' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/17' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/35' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './160/37' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './160' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/19' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/15' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/31' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/36' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/20' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/12' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/39' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/10' 2024-03-09T08:57:08.212 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/34' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/38' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/11' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/16' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/32' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/30' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/14' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/18' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/13' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/33' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/17' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/35' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './161/37' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './161' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/19' 2024-03-09T08:57:08.213 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/15' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/31' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/36' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/20' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/12' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/39' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/10' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/34' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/38' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/11' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/16' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/32' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/30' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/14' 2024-03-09T08:57:08.214 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/18' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/13' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/33' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/17' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/35' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './162/37' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './162' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/19' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/15' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/31' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/36' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/20' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/12' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/39' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/10' 2024-03-09T08:57:08.215 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/34' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/38' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/11' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/16' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/32' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/30' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/14' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/18' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/13' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/33' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/17' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/35' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './163/37' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './163' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/19' 2024-03-09T08:57:08.216 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/15' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/31' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/36' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/20' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/12' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/39' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/10' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/34' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/38' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/11' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/16' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/32' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/30' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/14' 2024-03-09T08:57:08.217 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/18' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/13' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/33' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/17' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/35' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './164/37' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './164' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/19' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/15' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/31' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/36' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/20' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/12' 2024-03-09T08:57:08.218 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/39' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/10' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/34' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/38' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/11' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/16' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/32' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/30' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/14' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/18' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/13' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/33' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/17' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/35' 2024-03-09T08:57:08.219 INFO:tasks.workunit.client.0.smithi067.stdout:removed './165/37' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './165' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/19' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/15' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/31' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/36' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/20' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/12' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/39' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/10' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/34' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/38' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/11' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/16' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/32' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/30' 2024-03-09T08:57:08.220 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/14' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/18' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/13' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/33' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/17' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/35' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './166/37' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './166' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/19' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/15' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/31' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/36' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/20' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/12' 2024-03-09T08:57:08.221 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/39' 2024-03-09T08:57:08.222 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/10' 2024-03-09T08:57:08.222 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/34' 2024-03-09T08:57:08.222 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/38' 2024-03-09T08:57:08.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/20' 2024-03-09T08:57:08.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/12' 2024-03-09T08:57:08.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/39' 2024-03-09T08:57:08.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/10' 2024-03-09T08:57:08.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/34' 2024-03-09T08:57:08.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/38' 2024-03-09T08:57:08.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/11' 2024-03-09T08:57:08.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/16' 2024-03-09T08:57:08.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/32' 2024-03-09T08:57:08.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/30' 2024-03-09T08:57:08.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/14' 2024-03-09T08:57:08.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/18' 2024-03-09T08:57:08.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/13' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/33' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/17' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/35' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './148/37' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './148' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/19' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/15' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/31' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/36' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/20' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/12' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/39' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/10' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/34' 2024-03-09T08:57:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/38' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/11' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/16' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/32' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/30' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/14' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/18' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/13' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/33' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/17' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/35' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './149/37' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './149' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/19' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/15' 2024-03-09T08:57:08.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/31' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/36' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/20' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/12' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/39' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/10' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/34' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/38' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/11' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/16' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/32' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/30' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/14' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/18' 2024-03-09T08:57:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/13' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/33' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/17' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/35' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './150/37' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './150' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/19' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/15' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/31' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/36' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/20' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/12' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/39' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/10' 2024-03-09T08:57:08.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/34' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/38' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/11' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/16' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/32' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/30' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/14' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/18' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/13' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/33' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/17' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/35' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './151/37' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './151' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/19' 2024-03-09T08:57:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/15' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/31' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/36' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/20' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/12' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/39' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/10' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/34' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/38' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/11' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/16' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/32' 2024-03-09T08:57:08.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/30' 2024-03-09T08:57:08.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/14' 2024-03-09T08:57:08.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/18' 2024-03-09T08:57:08.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/13' 2024-03-09T08:57:08.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/33' 2024-03-09T08:57:08.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/17' 2024-03-09T08:57:08.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/35' 2024-03-09T08:57:08.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './152/37' 2024-03-09T08:57:08.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './152' 2024-03-09T08:57:08.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/19' 2024-03-09T08:57:08.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/15' 2024-03-09T08:57:08.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/31' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/36' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/20' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/12' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/39' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/10' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/34' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/38' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/11' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/16' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/32' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/30' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/14' 2024-03-09T08:57:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/18' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/13' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/33' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/17' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/35' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './153/37' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './153' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/19' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/15' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/31' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/36' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/20' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/12' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/39' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/10' 2024-03-09T08:57:08.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/34' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/38' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/11' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/16' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/32' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/30' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/14' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/18' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/13' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/33' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/17' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/35' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './154/37' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './154' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/19' 2024-03-09T08:57:08.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/15' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/31' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/36' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/20' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/12' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/39' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/10' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/34' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/38' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/11' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/16' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/32' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/30' 2024-03-09T08:57:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/14' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/18' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/13' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/33' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/17' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/35' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './155/37' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './155' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/19' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/15' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/31' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/36' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/20' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/12' 2024-03-09T08:57:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/39' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/10' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/34' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/38' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/11' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/16' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/32' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/30' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/14' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/18' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/13' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/33' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/17' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/35' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './156/37' 2024-03-09T08:57:08.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './156' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/19' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/15' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/31' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/36' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/20' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/12' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/39' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/10' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/34' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/38' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/11' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/16' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/32' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/30' 2024-03-09T08:57:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/14' 2024-03-09T08:57:08.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/18' 2024-03-09T08:57:08.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/13' 2024-03-09T08:57:08.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/33' 2024-03-09T08:57:09.007 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:57:09.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.261+0000 7f8e5a600640 1 Processor -- start 2024-03-09T08:57:09.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.261+0000 7f8e5a600640 1 -- start start 2024-03-09T08:57:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.262+0000 7f8e5a600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8e540c22a0 0x7f8e54071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.262+0000 7f8e5a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e54157d60 0x7f8e540acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:09.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.262+0000 7f8e5a600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e541581a0 con 0x7f8e540ac730 2024-03-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.262+0000 7f8e5a600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e54070ee0 con 0x7f8e54157d60 2024-03-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.262+0000 7f8e5a600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e54154d00 con 0x7f8e540c22a0 2024-03-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.262+0000 7f8e53e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e54157d60 0x7f8e540acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.263+0000 7f8e53e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e54157d60 0x7f8e540acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57078/0 (socket says 172.21.15.67:57078) 2024-03-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.263+0000 7f8e53400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8e540ac730 0x7f8e54157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48192/0 (socket says 172.21.15.67:48192) 2024-03-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.263+0000 7f8e53e00640 1 -- 172.21.15.67:0/4139644270 learned_addr learned my addr 172.21.15.67:0/4139644270 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.263+0000 7f8e52a00640 1 --2- 172.21.15.67:0/4139644270 >> v2:172.21.15.134:3300/0 conn(0x7f8e540c22a0 0x7f8e54071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.263+0000 7f8e53e00640 1 -- 172.21.15.67:0/4139644270 >> v2:172.21.15.134:3300/0 conn(0x7f8e540c22a0 msgr2=0x7f8e54071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.263+0000 7f8e53e00640 1 --2- 172.21.15.67:0/4139644270 >> v2:172.21.15.134:3300/0 conn(0x7f8e540c22a0 0x7f8e54071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.263+0000 7f8e53e00640 1 -- 172.21.15.67:0/4139644270 >> v1:172.21.15.67:6789/0 conn(0x7f8e540ac730 legacy=0x7f8e54157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.263+0000 7f8e53e00640 1 -- 172.21.15.67:0/4139644270 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e54153d40 con 0x7f8e54157d60 2024-03-09T08:57:09.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.265+0000 7f8e53e00640 1 --2- 172.21.15.67:0/4139644270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e54157d60 0x7f8e540acb10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8e540c1a30 tx=0x7f8e4c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338d7dd94e4d658d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:09.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.265+0000 7f8e52000640 1 -- 172.21.15.67:0/4139644270 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c02ca80 con 0x7f8e54157d60 2024-03-09T08:57:09.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.265+0000 7f8e52000640 1 -- 172.21.15.67:0/4139644270 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c02cc20 con 0x7f8e54157d60 2024-03-09T08:57:09.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.265+0000 7f8e52000640 1 -- 172.21.15.67:0/4139644270 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c0343f0 con 0x7f8e54157d60 2024-03-09T08:57:09.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.266+0000 7f8e5a600640 1 -- 172.21.15.67:0/4139644270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e54157d60 msgr2=0x7f8e540acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.266+0000 7f8e5a600640 1 --2- 172.21.15.67:0/4139644270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e54157d60 0x7f8e540acb10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8e540c1a30 tx=0x7f8e4c02bee0 comp rx=0 tx=0).stop 2024-03-09T08:57:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.266+0000 7f8e5a600640 1 -- 172.21.15.67:0/4139644270 shutdown_connections 2024-03-09T08:57:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.266+0000 7f8e5a600640 1 --2- 172.21.15.67:0/4139644270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e54157d60 0x7f8e540acb10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.266+0000 7f8e5a600640 1 --2- 172.21.15.67:0/4139644270 >> v2:172.21.15.134:3300/0 conn(0x7f8e540c22a0 0x7f8e54071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.266+0000 7f8e5a600640 1 -- 172.21.15.67:0/4139644270 >> 172.21.15.67:0/4139644270 conn(0x7f8e540c7370 msgr2=0x7f8e540c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.266+0000 7f8e5a600640 1 -- 172.21.15.67:0/4139644270 shutdown_connections 2024-03-09T08:57:09.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.266+0000 7f8e5a600640 1 -- 172.21.15.67:0/4139644270 wait complete. 2024-03-09T08:57:09.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.268+0000 7f8e5a600640 1 Processor -- start 2024-03-09T08:57:09.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.268+0000 7f8e5a600640 1 -- start start 2024-03-09T08:57:09.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.269+0000 7f8e5a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e540ac730 0x7f8e541e2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:09.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.269+0000 7f8e5a600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e540c22a0 0x7f8e540ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:09.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.269+0000 7f8e5a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e54157d60 0x7f8e5414e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:09.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.269+0000 7f8e5a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e54070c00 con 0x7f8e54157d60 2024-03-09T08:57:09.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.269+0000 7f8e5a600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e54070920 con 0x7f8e540ac730 2024-03-09T08:57:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.269+0000 7f8e5a600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e54151610 con 0x7f8e540c22a0 2024-03-09T08:57:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.269+0000 7f8e52a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e540c22a0 msgr2=0x7f8e540ab320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.270+0000 7f8e52a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e540c22a0 0x7f8e540ab320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.270+0000 7f8e53e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e54157d60 0x7f8e5414e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.270+0000 7f8e53e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e54157d60 0x7f8e5414e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57086/0 (socket says 172.21.15.67:57086) 2024-03-09T08:57:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.270+0000 7f8e53e00640 1 -- 172.21.15.67:0/727802097 learned_addr learned my addr 172.21.15.67:0/727802097 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.270+0000 7f8e53400640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e540ac730 0x7f8e541e2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.270+0000 7f8e53e00640 1 -- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e540c22a0 msgr2=0x7f8e540ab320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:57:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.270+0000 7f8e53e00640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e540c22a0 0x7f8e540ab320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.270+0000 7f8e53e00640 1 -- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e540ac730 msgr2=0x7f8e541e2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:09.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.270+0000 7f8e53e00640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e540ac730 0x7f8e541e2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.270+0000 7f8e53e00640 1 -- 172.21.15.67:0/727802097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e540abb70 con 0x7f8e54157d60 2024-03-09T08:57:09.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.270+0000 7f8e53e00640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e54157d60 0x7f8e5414e140 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c036040 tx=0x7f8e4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:09.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.271+0000 7f8e4b400640 1 -- 172.21.15.67:0/727802097 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c043020 con 0x7f8e54157d60 2024-03-09T08:57:09.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.271+0000 7f8e4b400640 1 -- 172.21.15.67:0/727802097 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c03e450 con 0x7f8e54157d60 2024-03-09T08:57:09.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.271+0000 7f8e4b400640 1 -- 172.21.15.67:0/727802097 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c009510 con 0x7f8e54157d60 2024-03-09T08:57:09.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.271+0000 7f8e5a600640 1 -- 172.21.15.67:0/727802097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e541de420 con 0x7f8e54157d60 2024-03-09T08:57:09.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.271+0000 7f8e5a600640 1 -- 172.21.15.67:0/727802097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e541de9e0 con 0x7f8e54157d60 2024-03-09T08:57:09.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.272+0000 7f8e5a600640 1 -- 172.21.15.67:0/727802097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e54075a00 con 0x7f8e54157d60 2024-03-09T08:57:09.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.273+0000 7f8e4b400640 1 -- 172.21.15.67:0/727802097 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8e4c046020 con 0x7f8e54157d60 2024-03-09T08:57:09.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.274+0000 7f8e4b400640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e3406b4b0 0x7f8e3406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:09.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.274+0000 7f8e4b400640 1 -- 172.21.15.67:0/727802097 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 8361+0+0 (secure 0 0 0) 0x7f8e4c0ad1f0 con 0x7f8e54157d60 2024-03-09T08:57:09.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.275+0000 7f8e53400640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e3406b4b0 0x7f8e3406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:09.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.276+0000 7f8e53400640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e3406b4b0 0x7f8e3406d970 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8e44000f60 tx=0x7f8e44007cc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:09.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.283+0000 7f8e4b400640 1 -- 172.21.15.67:0/727802097 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8e4c04d070 con 0x7f8e54157d60 2024-03-09T08:57:09.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.560+0000 7f8e5a600640 1 -- 172.21.15.67:0/727802097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8e540c1d70 con 0x7f8e54157d60 2024-03-09T08:57:09.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.561+0000 7f8e4b400640 1 -- 172.21.15.67:0/727802097 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f8e4c07b5c0 con 0x7f8e54157d60 2024-03-09T08:57:09.560 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:57:09.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.563+0000 7f8e5a600640 1 -- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e3406b4b0 msgr2=0x7f8e3406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:09.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.563+0000 7f8e5a600640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e3406b4b0 0x7f8e3406d970 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8e44000f60 tx=0x7f8e44007cc0 comp rx=0 tx=0).stop 2024-03-09T08:57:09.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.563+0000 7f8e5a600640 1 -- 172.21.15.67:0/727802097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e54157d60 msgr2=0x7f8e5414e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:09.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.563+0000 7f8e5a600640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e54157d60 0x7f8e5414e140 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c036040 tx=0x7f8e4c004a40 comp rx=0 tx=0).stop 2024-03-09T08:57:09.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.563+0000 7f8e5a600640 1 -- 172.21.15.67:0/727802097 shutdown_connections 2024-03-09T08:57:09.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.563+0000 7f8e5a600640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e3406b4b0 0x7f8e3406d970 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:09.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.563+0000 7f8e5a600640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e54157d60 0x7f8e5414e140 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:09.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.563+0000 7f8e5a600640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e540c22a0 0x7f8e540ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:09.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.563+0000 7f8e5a600640 1 --2- 172.21.15.67:0/727802097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e540ac730 0x7f8e541e2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:09.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.563+0000 7f8e5a600640 1 -- 172.21.15.67:0/727802097 >> 172.21.15.67:0/727802097 conn(0x7f8e540c7370 msgr2=0x7f8e540c2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:09.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.563+0000 7f8e5a600640 1 -- 172.21.15.67:0/727802097 shutdown_connections 2024-03-09T08:57:09.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:09.564+0000 7f8e5a600640 1 -- 172.21.15.67:0/727802097 wait complete. 2024-03-09T08:57:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/11' 2024-03-09T08:57:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/16' 2024-03-09T08:57:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/32' 2024-03-09T08:57:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/30' 2024-03-09T08:57:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/14' 2024-03-09T08:57:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/18' 2024-03-09T08:57:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/13' 2024-03-09T08:57:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/33' 2024-03-09T08:57:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/17' 2024-03-09T08:57:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/35' 2024-03-09T08:57:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:removed './167/37' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './167' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/19' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/15' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/31' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/36' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/20' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/12' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/39' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/10' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/34' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/38' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/11' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/16' 2024-03-09T08:57:09.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/32' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/30' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/14' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/18' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/13' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/33' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/17' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/35' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './168/37' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './168' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/19' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/15' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/31' 2024-03-09T08:57:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/36' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/20' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/12' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/39' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/10' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/34' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/38' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/11' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/16' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/32' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/30' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/14' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/18' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/13' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/33' 2024-03-09T08:57:09.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/17' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/35' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './169/37' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './169' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/19' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/15' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/31' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/36' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/20' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/12' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/39' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/10' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/34' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/38' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/11' 2024-03-09T08:57:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/16' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/32' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/30' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/14' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/18' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/13' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/33' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/17' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/35' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './170/37' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './170' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/19' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/15' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/31' 2024-03-09T08:57:09.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/36' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/20' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/12' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/39' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/10' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/34' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/38' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/11' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/16' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/32' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/30' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/14' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/18' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/13' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/33' 2024-03-09T08:57:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/17' 2024-03-09T08:57:09.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/35' 2024-03-09T08:57:09.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './171/37' 2024-03-09T08:57:09.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './171' 2024-03-09T08:57:09.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/19' 2024-03-09T08:57:09.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/15' 2024-03-09T08:57:09.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/31' 2024-03-09T08:57:09.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/36' 2024-03-09T08:57:09.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/20' 2024-03-09T08:57:09.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/12' 2024-03-09T08:57:09.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/39' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/10' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/34' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/38' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/11' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/16' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/32' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/30' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/14' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/18' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/13' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/33' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/17' 2024-03-09T08:57:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/35' 2024-03-09T08:57:09.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './172/37' 2024-03-09T08:57:09.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './172' 2024-03-09T08:57:09.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/19' 2024-03-09T08:57:09.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/15' 2024-03-09T08:57:09.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/31' 2024-03-09T08:57:09.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/36' 2024-03-09T08:57:09.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/20' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/12' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/39' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/10' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/34' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/38' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/11' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/16' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/32' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/30' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/14' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/18' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/13' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/33' 2024-03-09T08:57:09.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/17' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/35' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './173/37' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './173' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/19' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/15' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/31' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/36' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/20' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/12' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/39' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/10' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/34' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/38' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/11' 2024-03-09T08:57:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/16' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/32' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/30' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/14' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/18' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/13' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/33' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/17' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/35' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './174/37' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './174' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/19' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/15' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/31' 2024-03-09T08:57:09.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/36' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/20' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/12' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/39' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/10' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/34' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/38' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/11' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/16' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/32' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/30' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/14' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/18' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/13' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/33' 2024-03-09T08:57:09.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/17' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/35' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './175/37' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './175' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/19' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/15' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/31' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/36' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/20' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/12' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/39' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/10' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/34' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/38' 2024-03-09T08:57:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/11' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/16' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/32' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/30' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/14' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/18' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/13' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/33' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/17' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/35' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './176/37' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './176' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/19' 2024-03-09T08:57:09.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/15' 2024-03-09T08:57:10.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/17' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/35' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './157/37' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './157' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/19' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/15' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/31' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/36' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/20' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/12' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/39' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/10' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/34' 2024-03-09T08:57:10.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/38' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/11' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/16' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/32' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/30' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/14' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/18' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/13' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/33' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/17' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/35' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './158/37' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './158' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/19' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/15' 2024-03-09T08:57:10.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/31' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/36' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/20' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/12' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/39' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/10' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/34' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/38' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/11' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/16' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/32' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/30' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/14' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/18' 2024-03-09T08:57:10.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/13' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/33' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/17' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/35' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './159/37' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './159' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/19' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/15' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/31' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/36' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/20' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/12' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/39' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/10' 2024-03-09T08:57:10.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/34' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/38' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/11' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/16' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/32' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/30' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/14' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/18' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/13' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/33' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/17' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/35' 2024-03-09T08:57:10.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './160/37' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './160' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/19' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/15' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/31' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/36' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/20' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/12' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/39' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/10' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/34' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/38' 2024-03-09T08:57:10.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/11' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/16' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/32' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/30' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/14' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/18' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/13' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/33' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/17' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/35' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './161/37' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './161' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/19' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/15' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/31' 2024-03-09T08:57:10.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/36' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/20' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/12' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/39' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/10' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/34' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/38' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/11' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/16' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/32' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/30' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/14' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/18' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/13' 2024-03-09T08:57:10.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/33' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/17' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/35' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './162/37' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './162' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/19' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/15' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/31' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/36' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/20' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/12' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/39' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/10' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/34' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/38' 2024-03-09T08:57:10.927 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/11' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/16' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/32' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/30' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/14' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/18' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/13' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/33' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/17' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/35' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './163/37' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './163' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/19' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/15' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/31' 2024-03-09T08:57:10.928 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/36' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/20' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/12' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/39' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/10' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/34' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/38' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/11' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/16' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/32' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/30' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/14' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/18' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/13' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/33' 2024-03-09T08:57:10.929 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/17' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/35' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './164/37' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './164' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/19' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/15' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/31' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/36' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/20' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/12' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/39' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/10' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/34' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/38' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/11' 2024-03-09T08:57:10.930 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/16' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/32' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/30' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/14' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/18' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/13' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/33' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/17' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/35' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './165/37' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './165' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/19' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/15' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/31' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/36' 2024-03-09T08:57:10.931 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/20' 2024-03-09T08:57:10.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/12' 2024-03-09T08:57:10.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/39' 2024-03-09T08:57:10.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/10' 2024-03-09T08:57:10.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/34' 2024-03-09T08:57:10.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/38' 2024-03-09T08:57:10.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/11' 2024-03-09T08:57:10.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/16' 2024-03-09T08:57:10.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/32' 2024-03-09T08:57:10.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/30' 2024-03-09T08:57:10.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/14' 2024-03-09T08:57:10.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/18' 2024-03-09T08:57:10.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/13' 2024-03-09T08:57:10.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/33' 2024-03-09T08:57:10.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/17' 2024-03-09T08:57:10.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/35' 2024-03-09T08:57:10.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './166/37' 2024-03-09T08:57:10.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './166' 2024-03-09T08:57:10.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/19' 2024-03-09T08:57:10.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/15' 2024-03-09T08:57:10.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/31' 2024-03-09T08:57:10.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/36' 2024-03-09T08:57:10.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/20' 2024-03-09T08:57:10.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/12' 2024-03-09T08:57:10.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/39' 2024-03-09T08:57:10.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/10' 2024-03-09T08:57:10.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/34' 2024-03-09T08:57:10.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/38' 2024-03-09T08:57:11.138 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:57:11.146 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:57:11.350 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/31' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/36' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/20' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/12' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/39' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/10' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/34' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/38' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/11' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/16' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/32' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/30' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/14' 2024-03-09T08:57:11.351 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/18' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/13' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/33' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/17' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/35' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './177/37' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './177' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/19' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/15' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/31' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/36' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/20' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/12' 2024-03-09T08:57:11.352 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/39' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/10' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/34' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/38' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/11' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/16' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/32' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/30' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/14' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/18' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/13' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/33' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/17' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/35' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './178/37' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './178' 2024-03-09T08:57:11.353 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/19' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/15' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/31' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/36' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/20' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/12' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/39' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/10' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/34' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/38' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/11' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/16' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/32' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/30' 2024-03-09T08:57:11.354 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/14' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/18' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/13' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/33' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/17' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/35' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './179/37' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './179' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/19' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/15' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/31' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/36' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/20' 2024-03-09T08:57:11.355 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/12' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/39' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/10' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/34' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/38' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/11' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/16' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/32' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/30' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/14' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/18' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/13' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/33' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/17' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/35' 2024-03-09T08:57:11.356 INFO:tasks.workunit.client.0.smithi067.stdout:removed './180/37' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './180' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/19' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/15' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/31' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/36' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/20' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/12' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/39' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/10' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/34' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/38' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/11' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/16' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/32' 2024-03-09T08:57:11.357 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/30' 2024-03-09T08:57:11.358 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/14' 2024-03-09T08:57:11.358 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/18' 2024-03-09T08:57:11.358 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/13' 2024-03-09T08:57:11.358 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/33' 2024-03-09T08:57:11.358 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/17' 2024-03-09T08:57:11.358 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/35' 2024-03-09T08:57:11.358 INFO:tasks.workunit.client.0.smithi067.stdout:removed './181/37' 2024-03-09T08:57:11.358 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './181' 2024-03-09T08:57:11.358 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/19' 2024-03-09T08:57:11.358 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/15' 2024-03-09T08:57:11.359 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/31' 2024-03-09T08:57:11.359 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/36' 2024-03-09T08:57:11.359 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/20' 2024-03-09T08:57:11.359 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/12' 2024-03-09T08:57:11.359 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/39' 2024-03-09T08:57:11.359 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/10' 2024-03-09T08:57:11.359 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/34' 2024-03-09T08:57:11.359 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/38' 2024-03-09T08:57:11.359 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/11' 2024-03-09T08:57:11.359 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/16' 2024-03-09T08:57:11.359 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/32' 2024-03-09T08:57:11.360 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/30' 2024-03-09T08:57:11.360 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/14' 2024-03-09T08:57:11.360 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/18' 2024-03-09T08:57:11.360 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/13' 2024-03-09T08:57:11.360 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/33' 2024-03-09T08:57:11.360 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/17' 2024-03-09T08:57:11.360 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/35' 2024-03-09T08:57:11.360 INFO:tasks.workunit.client.0.smithi067.stdout:removed './182/37' 2024-03-09T08:57:11.360 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './182' 2024-03-09T08:57:11.361 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/19' 2024-03-09T08:57:11.361 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/15' 2024-03-09T08:57:11.361 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/31' 2024-03-09T08:57:11.361 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/36' 2024-03-09T08:57:11.361 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/20' 2024-03-09T08:57:11.361 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/12' 2024-03-09T08:57:11.361 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/39' 2024-03-09T08:57:11.361 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/10' 2024-03-09T08:57:11.361 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/34' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/38' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/11' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/16' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/32' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/30' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/14' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/18' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/13' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/33' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/17' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/35' 2024-03-09T08:57:11.362 INFO:tasks.workunit.client.0.smithi067.stdout:removed './183/37' 2024-03-09T08:57:11.363 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './183' 2024-03-09T08:57:11.363 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/19' 2024-03-09T08:57:11.363 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/15' 2024-03-09T08:57:11.363 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/31' 2024-03-09T08:57:11.363 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/36' 2024-03-09T08:57:11.363 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/20' 2024-03-09T08:57:11.363 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/12' 2024-03-09T08:57:11.363 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/39' 2024-03-09T08:57:11.363 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/10' 2024-03-09T08:57:11.363 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/34' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/38' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/11' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/16' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/32' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/30' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/14' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/18' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/13' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/33' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/17' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/35' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './184/37' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './184' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/19' 2024-03-09T08:57:11.364 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/15' 2024-03-09T08:57:11.365 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/31' 2024-03-09T08:57:11.365 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/36' 2024-03-09T08:57:11.365 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/20' 2024-03-09T08:57:11.365 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/12' 2024-03-09T08:57:11.365 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/39' 2024-03-09T08:57:11.365 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/10' 2024-03-09T08:57:11.365 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/34' 2024-03-09T08:57:11.365 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/38' 2024-03-09T08:57:11.365 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/11' 2024-03-09T08:57:11.365 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/16' 2024-03-09T08:57:11.365 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/32' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/30' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/14' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/18' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/13' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/33' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/17' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/35' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed './185/37' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './185' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/19' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/15' 2024-03-09T08:57:11.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/31' 2024-03-09T08:57:11.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/36' 2024-03-09T08:57:11.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/20' 2024-03-09T08:57:11.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/12' 2024-03-09T08:57:11.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/39' 2024-03-09T08:57:11.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/10' 2024-03-09T08:57:11.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/34' 2024-03-09T08:57:11.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/38' 2024-03-09T08:57:11.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/11' 2024-03-09T08:57:11.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/16' 2024-03-09T08:57:11.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/32' 2024-03-09T08:57:11.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/30' 2024-03-09T08:57:11.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/14' 2024-03-09T08:57:11.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/18' 2024-03-09T08:57:12.306 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/11' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/16' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/32' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/30' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/14' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/18' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/13' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/33' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/17' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/35' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './167/37' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './167' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/19' 2024-03-09T08:57:12.307 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/15' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/31' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/36' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/20' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/12' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/39' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/10' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/34' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/38' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/11' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/16' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/32' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/30' 2024-03-09T08:57:12.308 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/14' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/18' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/13' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/33' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/17' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/35' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './168/37' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './168' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/19' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/15' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/31' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/36' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/20' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/12' 2024-03-09T08:57:12.309 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/39' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/10' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/34' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/38' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/11' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/16' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/32' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/30' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/14' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/18' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/13' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/33' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/17' 2024-03-09T08:57:12.310 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/35' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './169/37' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './169' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/19' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/15' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/31' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/36' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/20' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/12' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/39' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/10' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/34' 2024-03-09T08:57:12.311 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/38' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/11' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/16' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/32' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/30' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/14' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/18' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/13' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/33' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/17' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/35' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './170/37' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './170' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/19' 2024-03-09T08:57:12.312 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/15' 2024-03-09T08:57:12.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/31' 2024-03-09T08:57:12.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/36' 2024-03-09T08:57:12.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/20' 2024-03-09T08:57:12.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/12' 2024-03-09T08:57:12.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/39' 2024-03-09T08:57:12.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/10' 2024-03-09T08:57:12.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/34' 2024-03-09T08:57:12.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/38' 2024-03-09T08:57:12.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/11' 2024-03-09T08:57:12.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/16' 2024-03-09T08:57:12.313 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/32' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/30' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/14' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/18' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/13' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/33' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/17' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/35' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './171/37' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './171' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/19' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/15' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/31' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/36' 2024-03-09T08:57:12.314 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/20' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/12' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/39' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/10' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/34' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/38' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/11' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/16' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/32' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/30' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/14' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/18' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/13' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/33' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/17' 2024-03-09T08:57:12.315 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/35' 2024-03-09T08:57:12.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './172/37' 2024-03-09T08:57:12.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './172' 2024-03-09T08:57:12.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/19' 2024-03-09T08:57:12.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/15' 2024-03-09T08:57:12.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/31' 2024-03-09T08:57:12.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/36' 2024-03-09T08:57:12.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/20' 2024-03-09T08:57:12.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/12' 2024-03-09T08:57:12.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/39' 2024-03-09T08:57:12.316 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/10' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/34' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/38' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/11' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/16' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/32' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/30' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/14' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/18' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/13' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/33' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/17' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/35' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './173/37' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './173' 2024-03-09T08:57:12.317 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/19' 2024-03-09T08:57:12.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/15' 2024-03-09T08:57:12.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/31' 2024-03-09T08:57:12.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/36' 2024-03-09T08:57:12.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/20' 2024-03-09T08:57:12.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/12' 2024-03-09T08:57:12.318 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/39' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/10' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/34' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/38' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/11' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/16' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/32' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/30' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/14' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/18' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/13' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/33' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/17' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/35' 2024-03-09T08:57:12.319 INFO:tasks.workunit.client.1.smithi134.stdout:removed './174/37' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './174' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/19' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/15' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/31' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/36' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/20' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/12' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/39' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/10' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/34' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/38' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/11' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/16' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/32' 2024-03-09T08:57:12.320 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/30' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/14' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/18' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/13' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/33' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/17' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/35' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './175/37' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './175' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/19' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/15' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/31' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/36' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/20' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/12' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/39' 2024-03-09T08:57:12.321 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/10' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/34' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/38' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/11' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/16' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/32' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/30' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/14' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/18' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/13' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/33' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/17' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/35' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './176/37' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './176' 2024-03-09T08:57:12.322 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/19' 2024-03-09T08:57:12.323 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/15' 2024-03-09T08:57:12.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:57:12.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.790+0000 7f930e800640 1 Processor -- start 2024-03-09T08:57:12.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.791+0000 7f930e800640 1 -- start start 2024-03-09T08:57:12.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.793+0000 7f930e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:12.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.793+0000 7f930e800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.793+0000 7f930e800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9308156a90 con 0x7f9308153300 2024-03-09T08:57:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.793+0000 7f930e800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9308157000 con 0x7f93080c5e40 2024-03-09T08:57:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.793+0000 7f930e800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93080ae430 con 0x7f93080cadb0 2024-03-09T08:57:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.793+0000 7f930d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9308153300 0x7f93080b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48202/0 (socket says 172.21.15.67:48202) 2024-03-09T08:57:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.793+0000 7f930d400640 1 -- 172.21.15.67:0/1780703072 learned_addr learned my addr 172.21.15.67:0/1780703072 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.793+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.794+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:12.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.794+0000 7f930ca00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 msgr2=0x7f9308156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:57:12.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.794+0000 7f930ca00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 msgr2=0x7f9308156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:57:12.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.794+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:12.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.794+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:12.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.794+0000 7f930de00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 msgr2=0x7f9308156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:57:12.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.794+0000 7f930de00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 msgr2=0x7f9308156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:57:12.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.794+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:12.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.794+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.996+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.998+0000 7f930de00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 msgr2=0x7f9308156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:57:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.998+0000 7f930de00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 msgr2=0x7f9308156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:57:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.998+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.998+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:57:12.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.998+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.998+0000 7f930ca00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 msgr2=0x7f9308156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:57:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.998+0000 7f930ca00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 msgr2=0x7f9308156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:57:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.999+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:12.999+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:57:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:13.399+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:13.400+0000 7f930de00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 msgr2=0x7f9308156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:57:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:13.400+0000 7f930de00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 msgr2=0x7f9308156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:57:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:13.400+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:13.400+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:57:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:13.400+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:13.400+0000 7f930ca00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 msgr2=0x7f9308156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:57:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:13.400+0000 7f930ca00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 msgr2=0x7f9308156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:57:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:13.400+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:13.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:13.400+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:57:13.599 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/31' 2024-03-09T08:57:13.599 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/36' 2024-03-09T08:57:13.599 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/20' 2024-03-09T08:57:13.599 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/12' 2024-03-09T08:57:13.599 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/39' 2024-03-09T08:57:13.599 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/10' 2024-03-09T08:57:13.599 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/34' 2024-03-09T08:57:13.599 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/38' 2024-03-09T08:57:13.599 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/11' 2024-03-09T08:57:13.599 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/16' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/32' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/30' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/14' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/18' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/13' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/33' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/17' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/35' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './177/37' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './177' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/19' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/15' 2024-03-09T08:57:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/31' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/36' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/20' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/12' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/39' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/10' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/34' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/38' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/11' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/16' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/32' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/30' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/14' 2024-03-09T08:57:13.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/18' 2024-03-09T08:57:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/13' 2024-03-09T08:57:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/33' 2024-03-09T08:57:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/17' 2024-03-09T08:57:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/35' 2024-03-09T08:57:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './178/37' 2024-03-09T08:57:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './178' 2024-03-09T08:57:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/19' 2024-03-09T08:57:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/15' 2024-03-09T08:57:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/31' 2024-03-09T08:57:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/36' 2024-03-09T08:57:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/20' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/12' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/39' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/10' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/34' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/38' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/11' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/16' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/32' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/30' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/14' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/18' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/13' 2024-03-09T08:57:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/33' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/17' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/35' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './179/37' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './179' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/19' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/15' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/31' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/36' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/20' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/12' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/39' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/10' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/34' 2024-03-09T08:57:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/38' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/11' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/16' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/32' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/30' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/14' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/18' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/13' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/33' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/17' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/35' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './180/37' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './180' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/19' 2024-03-09T08:57:13.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/15' 2024-03-09T08:57:13.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/31' 2024-03-09T08:57:13.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/36' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/20' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/12' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/39' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/10' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/34' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/38' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/11' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/16' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/32' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/30' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/14' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/18' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/13' 2024-03-09T08:57:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/33' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/17' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/35' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './181/37' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './181' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/19' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/15' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/31' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/36' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/20' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/12' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/39' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/10' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/34' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/38' 2024-03-09T08:57:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/11' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/16' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/32' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/30' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/14' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/18' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/13' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/33' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/17' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/35' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './182/37' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './182' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/19' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/15' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/31' 2024-03-09T08:57:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/36' 2024-03-09T08:57:13.610 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/20' 2024-03-09T08:57:13.610 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/12' 2024-03-09T08:57:13.610 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/39' 2024-03-09T08:57:13.610 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/10' 2024-03-09T08:57:13.610 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/34' 2024-03-09T08:57:13.610 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/38' 2024-03-09T08:57:13.610 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/11' 2024-03-09T08:57:13.610 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/16' 2024-03-09T08:57:13.610 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/32' 2024-03-09T08:57:13.610 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/30' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/14' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/18' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/13' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/33' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/17' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/35' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './183/37' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './183' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/19' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/15' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/31' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/36' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/20' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/12' 2024-03-09T08:57:13.611 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/39' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/10' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/34' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/38' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/11' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/16' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/32' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/30' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/14' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/18' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/13' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/33' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/17' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/35' 2024-03-09T08:57:13.612 INFO:tasks.workunit.client.1.smithi134.stdout:removed './184/37' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './184' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/19' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/15' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/31' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/36' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/20' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/12' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/39' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/10' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/34' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/38' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/11' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/16' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/32' 2024-03-09T08:57:13.613 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/30' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/14' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/18' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/13' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/33' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/17' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/35' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './185/37' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './185' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/19' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/15' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/31' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/36' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/20' 2024-03-09T08:57:13.614 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/12' 2024-03-09T08:57:13.615 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/39' 2024-03-09T08:57:13.615 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/10' 2024-03-09T08:57:13.615 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/34' 2024-03-09T08:57:13.615 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/38' 2024-03-09T08:57:13.615 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/11' 2024-03-09T08:57:13.615 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/16' 2024-03-09T08:57:13.615 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/32' 2024-03-09T08:57:13.615 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/30' 2024-03-09T08:57:13.615 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/14' 2024-03-09T08:57:13.615 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/18' 2024-03-09T08:57:14.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/13' 2024-03-09T08:57:14.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/33' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/17' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/35' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './186/37' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './186' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/19' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/15' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/31' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/36' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/20' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/12' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/39' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/10' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/34' 2024-03-09T08:57:14.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/38' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/11' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/16' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/32' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/30' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/14' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/18' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/13' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/33' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/17' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/35' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './187/37' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './187' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/19' 2024-03-09T08:57:14.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/15' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/31' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/36' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/20' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/12' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/39' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/10' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/34' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/38' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/11' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/16' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/32' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/30' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/14' 2024-03-09T08:57:14.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/18' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/13' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/33' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/17' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/35' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './188/37' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './188' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/19' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/15' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/31' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/36' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/20' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/12' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/39' 2024-03-09T08:57:14.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/10' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/34' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/38' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/11' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/16' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/32' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/30' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/14' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/18' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/13' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/33' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/17' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/35' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './189/37' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './189' 2024-03-09T08:57:14.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/19' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/15' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/31' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/36' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/20' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/12' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/39' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/10' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/34' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/38' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/11' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/16' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/32' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/30' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/14' 2024-03-09T08:57:14.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/18' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/13' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/33' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/17' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/35' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './190/37' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './190' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/19' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/15' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/31' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/36' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/20' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/12' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/39' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/10' 2024-03-09T08:57:14.107 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/34' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/38' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/11' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/16' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/32' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/30' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/14' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/18' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/13' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/33' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/17' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/35' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './191/37' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './191' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/19' 2024-03-09T08:57:14.108 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/15' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/31' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/36' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/20' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/12' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/39' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/10' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/34' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/38' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/11' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/16' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/32' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/30' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/14' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/18' 2024-03-09T08:57:14.109 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/13' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/33' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/17' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/35' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './192/37' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './192' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/19' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/15' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/31' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/36' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/20' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/12' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/39' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/10' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/34' 2024-03-09T08:57:14.110 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/38' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/11' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/16' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/32' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/30' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/14' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/18' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/13' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/33' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/17' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/35' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './193/37' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './193' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/19' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/15' 2024-03-09T08:57:14.111 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/31' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/36' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/20' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/12' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/39' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/10' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/34' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/38' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/11' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/16' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/32' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/30' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/14' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/18' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/13' 2024-03-09T08:57:14.112 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/33' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/17' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/35' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './194/37' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './194' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/19' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/15' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/31' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/36' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/20' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/12' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/39' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/10' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/34' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/38' 2024-03-09T08:57:14.113 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/11' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/16' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/32' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/30' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/14' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/18' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/13' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/33' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/17' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/35' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './195/37' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './195' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/19' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/15' 2024-03-09T08:57:14.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/31' 2024-03-09T08:57:14.115 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/36' 2024-03-09T08:57:14.115 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/20' 2024-03-09T08:57:14.115 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/12' 2024-03-09T08:57:14.115 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/39' 2024-03-09T08:57:14.115 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/10' 2024-03-09T08:57:14.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:14.201+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:14.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:14.201+0000 7f930ca00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 msgr2=0x7f9308156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:57:14.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:14.201+0000 7f930ca00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 msgr2=0x7f9308156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:57:14.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:14.201+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:14.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:14.201+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:14.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:14.202+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:57:14.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:14.202+0000 7f930de00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 msgr2=0x7f9308156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:57:14.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:14.202+0000 7f930de00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 msgr2=0x7f9308156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:57:14.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:14.202+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:14.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:14.202+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:57:15.703 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/34' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/38' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/11' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/16' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/32' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/30' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/14' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/18' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/13' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/33' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/17' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/35' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './196/37' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './196' 2024-03-09T08:57:15.704 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/19' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/15' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/31' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/36' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/20' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/12' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/39' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/10' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/34' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/38' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/11' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/16' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/32' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/30' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/14' 2024-03-09T08:57:15.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/18' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/13' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/33' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/17' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/35' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './197/37' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './197' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/19' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/15' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/31' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/36' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/20' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/12' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/39' 2024-03-09T08:57:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/10' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/34' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/38' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/11' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/16' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/32' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/30' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/14' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/18' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/13' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/33' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/17' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/35' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './198/37' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './198' 2024-03-09T08:57:15.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/19' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/15' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/31' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/36' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/20' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/12' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/39' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/10' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/34' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/38' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/11' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/16' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/32' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/30' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/14' 2024-03-09T08:57:15.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/18' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/13' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/33' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/17' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/35' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './199/37' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './199' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/21' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/27' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/31' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/36' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/20' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/39' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/34' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/38' 2024-03-09T08:57:15.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/28' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/24' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/29' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/32' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/22' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/25' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/30' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/33' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/23' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/35' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/26' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './200/37' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './200' 2024-03-09T08:57:15.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/21' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/27' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/31' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/36' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/20' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/39' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/34' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/38' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/28' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/24' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/29' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/32' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/22' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/25' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/30' 2024-03-09T08:57:15.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/33' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/23' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/35' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/26' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './201/37' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './201' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/21' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/27' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/31' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/36' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/20' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/39' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/34' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/38' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/28' 2024-03-09T08:57:15.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/24' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/29' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/32' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/22' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/25' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/30' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/33' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/23' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/35' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/26' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './202/37' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './202' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/21' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/27' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/31' 2024-03-09T08:57:15.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/36' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/20' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/39' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/34' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/38' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/28' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/24' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/29' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/32' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/22' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/25' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/30' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/33' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/23' 2024-03-09T08:57:15.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/35' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/26' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './203/37' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './203' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/21' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/27' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/31' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/36' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/20' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/39' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/34' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/38' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/28' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/24' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/29' 2024-03-09T08:57:15.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/32' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/22' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/25' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/30' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/33' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/23' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/35' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/26' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './204/37' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './204' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/21' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/27' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/31' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/36' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/20' 2024-03-09T08:57:15.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/39' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/34' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/38' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/28' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/24' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/29' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/32' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/22' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/25' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/30' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/33' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/23' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/35' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/26' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed './205/37' 2024-03-09T08:57:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './205' 2024-03-09T08:57:15.718 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/21' 2024-03-09T08:57:15.718 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/27' 2024-03-09T08:57:15.718 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/31' 2024-03-09T08:57:15.718 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/36' 2024-03-09T08:57:15.718 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/20' 2024-03-09T08:57:15.718 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/39' 2024-03-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.793+0000 7f9306a00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 msgr2=0x7f9308156550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.793+0000 7f9306a00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.134:3300/0 conn(0x7f93080cadb0 0x7f9308156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.793+0000 7f9306a00640 1 -- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 msgr2=0x7f9308156010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.793+0000 7f9306a00640 1 --2- 172.21.15.67:0/1780703072 >> v2:172.21.15.67:3300/0 conn(0x7f93080c5e40 0x7f9308156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.793+0000 7f9306a00640 1 -- 172.21.15.67:0/1780703072 >> v1:172.21.15.67:6789/0 conn(0x7f9308153300 legacy=0x7f93080b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.793+0000 7f9306a00640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e8003690 0x7f92e8005ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.793+0000 7f9306a00640 1 -- 172.21.15.67:0/1780703072 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93080ae6f0 con 0x7f92e8006150 2024-03-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.793+0000 7f9306a00640 1 -- 172.21.15.67:0/1780703072 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9308157000 con 0x7f92e8003690 2024-03-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.793+0000 7f930d400640 1 -- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e8003690 msgr2=0x7f92e8005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.794+0000 7f930d400640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e8003690 0x7f92e8005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.995+0000 7f930d400640 1 -- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e8003690 msgr2=0x7f92e8005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:15.995+0000 7f930d400640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e8003690 0x7f92e8005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:57:16.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/13' 2024-03-09T08:57:16.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/33' 2024-03-09T08:57:16.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/17' 2024-03-09T08:57:16.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/35' 2024-03-09T08:57:16.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './186/37' 2024-03-09T08:57:16.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './186' 2024-03-09T08:57:16.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/19' 2024-03-09T08:57:16.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/15' 2024-03-09T08:57:16.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/31' 2024-03-09T08:57:16.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/36' 2024-03-09T08:57:16.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/20' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/12' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/39' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/10' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/34' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/38' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/11' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/16' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/32' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/30' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/14' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/18' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/13' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/33' 2024-03-09T08:57:16.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/17' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/35' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './187/37' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './187' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/19' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/15' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/31' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/36' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/20' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/12' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/39' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/10' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/34' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/38' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/11' 2024-03-09T08:57:16.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/16' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/32' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/30' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/14' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/18' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/13' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/33' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/17' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/35' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './188/37' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './188' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/19' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/15' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/31' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/36' 2024-03-09T08:57:16.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/20' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/12' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/39' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/10' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/34' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/38' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/11' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/16' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/32' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/30' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/14' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/18' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/13' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/33' 2024-03-09T08:57:16.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/17' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/35' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './189/37' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './189' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/19' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/15' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/31' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/36' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/20' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/12' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/39' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/10' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/34' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/38' 2024-03-09T08:57:16.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/11' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/16' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/32' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/30' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/14' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/18' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/13' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/33' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/17' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/35' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './190/37' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './190' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/19' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/15' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/31' 2024-03-09T08:57:16.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/36' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/20' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/12' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/39' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/10' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/34' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/38' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/11' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/16' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/32' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/30' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/14' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/18' 2024-03-09T08:57:16.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/13' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/33' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/17' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/35' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './191/37' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './191' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/19' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/15' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/31' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/36' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/20' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/12' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/39' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/10' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/34' 2024-03-09T08:57:16.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/38' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/11' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/16' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/32' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/30' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/14' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/18' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/13' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/33' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/17' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/35' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './192/37' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './192' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/19' 2024-03-09T08:57:16.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/15' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/31' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/36' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/20' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/12' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/39' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/10' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/34' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/38' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/11' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/16' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/32' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/30' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/14' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/18' 2024-03-09T08:57:16.115 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/13' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/33' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/17' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/35' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './193/37' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './193' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/19' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/15' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/31' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/36' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/20' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/12' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/39' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/10' 2024-03-09T08:57:16.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/34' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/38' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/11' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/16' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/32' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/30' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/14' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/18' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/13' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/33' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/17' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/35' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './194/37' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './194' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/19' 2024-03-09T08:57:16.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/15' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/31' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/36' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/20' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/12' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/39' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/10' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/34' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/38' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/11' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/16' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/32' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/30' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/14' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/18' 2024-03-09T08:57:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/13' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/33' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/17' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/35' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './195/37' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './195' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/19' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/15' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/31' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/36' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/20' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/12' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/39' 2024-03-09T08:57:16.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/10' 2024-03-09T08:57:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.396+0000 7f930d400640 1 -- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e8003690 msgr2=0x7f92e8005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.397+0000 7f930d400640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e8003690 0x7f92e8005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:57:16.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.532+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616124111 0 0) 0x7f93080ae6f0 con 0x7f92e8006150 2024-03-09T08:57:16.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.532+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92f0003520 con 0x7f92e8006150 2024-03-09T08:57:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.546+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2278051570 0 0) 0x7f92f0003520 con 0x7f92e8006150 2024-03-09T08:57:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.546+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e8003690 msgr2=0x7f92e8005ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:57:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.546+0000 7f9307e00640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e8003690 0x7f92e8005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.546+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e800afa0 con 0x7f92e8006150 2024-03-09T08:57:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.546+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f92f400d580 con 0x7f92e8006150 2024-03-09T08:57:16.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.546+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 >> v1:172.21.15.134:6789/0 conn(0x7f92e8006150 legacy=0x7f92e8008610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.547+0000 7f930de00640 1 -- 172.21.15.67:0/1780703072 reap_dead start 2024-03-09T08:57:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.547+0000 7f9307e00640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f0003a20 0x7f92f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.547+0000 7f9307e00640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0004800 0x7f92f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.547+0000 7f9307e00640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f00053c0 0x7f92f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.547+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9308157000 con 0x7f92f00053c0 2024-03-09T08:57:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.547+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f000c280 con 0x7f92f0004800 2024-03-09T08:57:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.547+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f000c480 con 0x7f92f0003a20 2024-03-09T08:57:16.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.547+0000 7f930d400640 1 -- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f0003a20 msgr2=0x7f92f0003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.547+0000 7f930d400640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f0003a20 0x7f92f0003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.548+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0004800 0x7f92f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.548+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f00053c0 0x7f92f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.549+0000 7f930ca00640 1 -- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f0003a20 msgr2=0x7f92f0003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:57:16.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.549+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f0003a20 0x7f92f0003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.549+0000 7f930ca00640 1 -- 172.21.15.67:0/1780703072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f00053c0 msgr2=0x7f92f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.549+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f00053c0 0x7f92f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.549+0000 7f930ca00640 1 -- 172.21.15.67:0/1780703072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f0003520 con 0x7f92f0004800 2024-03-09T08:57:16.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.549+0000 7f930de00640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f00053c0 0x7f92f000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:57:16.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.552+0000 7f930ca00640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0004800 0x7f92f0004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f92f400e010 tx=0x7f92f0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=241b780d7e6044bd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:16.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.553+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4033070 con 0x7f92f0004800 2024-03-09T08:57:16.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.553+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f402d5c0 con 0x7f92f0004800 2024-03-09T08:57:16.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.553+0000 7f9307e00640 1 -- 172.21.15.67:0/1780703072 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400dd90 con 0x7f92f0004800 2024-03-09T08:57:16.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.554+0000 7f930e800640 1 -- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0004800 msgr2=0x7f92f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:16.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.554+0000 7f930e800640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0004800 0x7f92f0004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f92f400e010 tx=0x7f92f0005830 comp rx=0 tx=0).stop 2024-03-09T08:57:16.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.555+0000 7f930e800640 1 -- 172.21.15.67:0/1780703072 shutdown_connections 2024-03-09T08:57:16.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.555+0000 7f930e800640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f00053c0 0x7f92f000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.555+0000 7f930e800640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0004800 0x7f92f0004c80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.555+0000 7f930e800640 1 --2- 172.21.15.67:0/1780703072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f0003a20 0x7f92f0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.555+0000 7f930e800640 1 -- 172.21.15.67:0/1780703072 >> 172.21.15.67:0/1780703072 conn(0x7f93080732d0 msgr2=0x7f93080ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:16.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.555+0000 7f930e800640 1 -- 172.21.15.67:0/1780703072 shutdown_connections 2024-03-09T08:57:16.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.555+0000 7f930e800640 1 -- 172.21.15.67:0/1780703072 wait complete. 2024-03-09T08:57:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.558+0000 7f930e800640 1 Processor -- start 2024-03-09T08:57:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.559+0000 7f930e800640 1 -- start start 2024-03-09T08:57:16.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.560+0000 7f930e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f0003a20 0x7f93081ee890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:16.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.560+0000 7f930e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0004800 0x7f930814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:16.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.560+0000 7f930e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f00053c0 0x7f930814c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:16.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.560+0000 7f930e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9308159e90 con 0x7f92f0003a20 2024-03-09T08:57:16.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.560+0000 7f930e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9308159bb0 con 0x7f92f0004800 2024-03-09T08:57:16.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.560+0000 7f930e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93080c59e0 con 0x7f92f00053c0 2024-03-09T08:57:16.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.560+0000 7f930de00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f00053c0 msgr2=0x7f930814c310 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:16.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.560+0000 7f930de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f00053c0 0x7f930814c310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:16.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.560+0000 7f930d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f0003a20 0x7f93081ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:16.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.561+0000 7f930d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f0003a20 0x7f93081ee890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57144/0 (socket says 172.21.15.67:57144) 2024-03-09T08:57:16.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.561+0000 7f930d400640 1 -- 172.21.15.67:0/2449391737 learned_addr learned my addr 172.21.15.67:0/2449391737 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:16.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.561+0000 7f930ca00640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0004800 0x7f930814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.561+0000 7f930d400640 1 -- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f00053c0 msgr2=0x7f930814c310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:57:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.561+0000 7f930d400640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f00053c0 0x7f930814c310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.561+0000 7f930d400640 1 -- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0004800 msgr2=0x7f930814bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.561+0000 7f930d400640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0004800 0x7f930814bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.561+0000 7f930d400640 1 -- 172.21.15.67:0/2449391737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93080c8210 con 0x7f92f0003a20 2024-03-09T08:57:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.562+0000 7f930d400640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f0003a20 0x7f93081ee890 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00c760 tx=0x7f92fc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.562+0000 7f9305600640 1 -- 172.21.15.67:0/2449391737 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc017070 con 0x7f92f0003a20 2024-03-09T08:57:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.562+0000 7f9305600640 1 -- 172.21.15.67:0/2449391737 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92fc005ab0 con 0x7f92f0003a20 2024-03-09T08:57:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.562+0000 7f930e800640 1 -- 172.21.15.67:0/2449391737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93080c7f30 con 0x7f92f0003a20 2024-03-09T08:57:16.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.562+0000 7f9305600640 1 -- 172.21.15.67:0/2449391737 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc007070 con 0x7f92f0003a20 2024-03-09T08:57:16.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.562+0000 7f930e800640 1 -- 172.21.15.67:0/2449391737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93081ef3b0 con 0x7f92f0003a20 2024-03-09T08:57:16.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.564+0000 7f930e800640 1 -- 172.21.15.67:0/2449391737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930804f3c0 con 0x7f92f0003a20 2024-03-09T08:57:16.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.565+0000 7f9305600640 1 -- 172.21.15.67:0/2449391737 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f92fc01a070 con 0x7f92f0003a20 2024-03-09T08:57:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.566+0000 7f9305600640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92e806bd40 0x7f92e806e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.566+0000 7f9305600640 1 -- 172.21.15.67:0/2449391737 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 8361+0+0 (secure 0 0 0) 0x7f92fc089fe0 con 0x7f92f0003a20 2024-03-09T08:57:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.567+0000 7f930ca00640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92e806bd40 0x7f92e806e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.568+0000 7f930ca00640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92e806bd40 0x7f92e806e200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f92f4038000 tx=0x7f92f4033000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.571+0000 7f9305600640 1 -- 172.21.15.67:0/2449391737 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f92fc057df0 con 0x7f92f0003a20 2024-03-09T08:57:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.855+0000 7f930e800640 1 -- 172.21.15.67:0/2449391737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f93080ae6f0 con 0x7f92f0003a20 2024-03-09T08:57:16.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.855+0000 7f9305600640 1 -- 172.21.15.67:0/2449391737 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7f92fc05be90 con 0x7f92f0003a20 2024-03-09T08:57:16.854 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":40,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:57:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 -- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92e806bd40 msgr2=0x7f92e806e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92e806bd40 0x7f92e806e200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f92f4038000 tx=0x7f92f4033000 comp rx=0 tx=0).stop 2024-03-09T08:57:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 -- 172.21.15.67:0/2449391737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f0003a20 msgr2=0x7f93081ee890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:16.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f0003a20 0x7f93081ee890 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00c760 tx=0x7f92fc00cc20 comp rx=0 tx=0).stop 2024-03-09T08:57:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 -- 172.21.15.67:0/2449391737 shutdown_connections 2024-03-09T08:57:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92e806bd40 0x7f92e806e200 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92f00053c0 0x7f930814c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92f0004800 0x7f930814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 --2- 172.21.15.67:0/2449391737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92f0003a20 0x7f93081ee890 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 -- 172.21.15.67:0/2449391737 >> 172.21.15.67:0/2449391737 conn(0x7f93080732d0 msgr2=0x7f9308150360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 -- 172.21.15.67:0/2449391737 shutdown_connections 2024-03-09T08:57:16.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:16.858+0000 7f930e800640 1 -- 172.21.15.67:0/2449391737 wait complete. 2024-03-09T08:57:16.875 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T08:57:16.875 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T08:57:16.875 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.114+0000 7fdafc400640 1 Processor -- start 2024-03-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.115+0000 7fdafc400640 1 -- start start 2024-03-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.117+0000 7fdafc400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdaf40c1240 0x7fdaf4159e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.117+0000 7fdafc400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf40670b0 con 0x7fdaf415a340 2024-03-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.117+0000 7fdafc400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf40c26d0 con 0x7fdaf40c6ab0 2024-03-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.117+0000 7fdafc400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf40c74b0 con 0x7fdaf40c1240 2024-03-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.117+0000 7fdaf9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdaf40c1240 0x7fdaf4159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.117+0000 7fdaf9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdaf40c1240 0x7fdaf4159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57152/0 (socket says 172.21.15.67:57152) 2024-03-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.117+0000 7fdaf9a00640 1 -- 172.21.15.67:0/3899644337 learned_addr learned my addr 172.21.15.67:0/3899644337 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.118+0000 7fdaf9a00640 1 -- 172.21.15.67:0/3899644337 >> v1:172.21.15.134:6789/0 conn(0x7fdaf40c6ab0 legacy=0x7fdaf40aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.118+0000 7fdaf9a00640 1 -- 172.21.15.67:0/3899644337 >> v1:172.21.15.67:6789/0 conn(0x7fdaf415a340 legacy=0x7fdaf40ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.118+0000 7fdaf9a00640 1 -- 172.21.15.67:0/3899644337 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf40c7700 con 0x7fdaf40c1240 2024-03-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.118+0000 7fdaf3e00640 1 -- 172.21.15.67:0/3899644337 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640059678 0 0) 0x7fdaf40670b0 con 0x7fdaf415a340 2024-03-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.119+0000 7fdaf9a00640 1 --2- 172.21.15.67:0/3899644337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf40c1240 0x7fdaf4159e00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009870 tx=0x7fdae802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a38da03ec00d6c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.119+0000 7fdaf3e00640 1 -- 172.21.15.67:0/3899644337 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8035020 con 0x7fdaf40c1240 2024-03-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.119+0000 7fdaf3e00640 1 -- 172.21.15.67:0/3899644337 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae8031870 con 0x7fdaf40c1240 2024-03-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.119+0000 7fdaf3e00640 1 -- 172.21.15.67:0/3899644337 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8031c40 con 0x7fdaf40c1240 2024-03-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.120+0000 7fdafc400640 1 -- 172.21.15.67:0/3899644337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf40c1240 msgr2=0x7fdaf4159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.120+0000 7fdafc400640 1 --2- 172.21.15.67:0/3899644337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf40c1240 0x7fdaf4159e00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009870 tx=0x7fdae802b910 comp rx=0 tx=0).stop 2024-03-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.120+0000 7fdafc400640 1 -- 172.21.15.67:0/3899644337 shutdown_connections 2024-03-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.120+0000 7fdafc400640 1 --2- 172.21.15.67:0/3899644337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf40c1240 0x7fdaf4159e00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.120+0000 7fdafc400640 1 -- 172.21.15.67:0/3899644337 >> 172.21.15.67:0/3899644337 conn(0x7fdaf40a9950 msgr2=0x7fdaf40a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:17.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.120+0000 7fdafc400640 1 -- 172.21.15.67:0/3899644337 shutdown_connections 2024-03-09T08:57:17.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.120+0000 7fdafc400640 1 -- 172.21.15.67:0/3899644337 wait complete. 2024-03-09T08:57:17.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.122+0000 7fdafc400640 1 Processor -- start 2024-03-09T08:57:17.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.122+0000 7fdafc400640 1 -- start start 2024-03-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdafc400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf40c1240 0x7fdaf4157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdafc400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf40c6ab0 0x7fdaf41ef210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdafc400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf415a340 0x7fdaf41ef750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdafc400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40aade0 con 0x7fdaf415a340 2024-03-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdafc400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf415a7f0 con 0x7fdaf40c6ab0 2024-03-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdafc400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf406f990 con 0x7fdaf40c1240 2024-03-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdaf9a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf40c1240 msgr2=0x7fdaf4157a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdaf9a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf40c1240 0x7fdaf4157a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdafa400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf415a340 0x7fdaf41ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdafa400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf415a340 0x7fdaf41ef750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57158/0 (socket says 172.21.15.67:57158) 2024-03-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdafa400640 1 -- 172.21.15.67:0/3401892363 learned_addr learned my addr 172.21.15.67:0/3401892363 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdaf9000640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf40c6ab0 0x7fdaf41ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.123+0000 7fdafa400640 1 -- 172.21.15.67:0/3401892363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf40c1240 msgr2=0x7fdaf4157a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.124+0000 7fdafa400640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdaf40c1240 0x7fdaf4157a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.124+0000 7fdafa400640 1 -- 172.21.15.67:0/3401892363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf40c6ab0 msgr2=0x7fdaf41ef210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.124+0000 7fdafa400640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf40c6ab0 0x7fdaf41ef210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.124+0000 7fdafa400640 1 -- 172.21.15.67:0/3401892363 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf41eed30 con 0x7fdaf415a340 2024-03-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.124+0000 7fdaf9000640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdaf40c6ab0 0x7fdaf41ef210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.124+0000 7fdafa400640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf415a340 0x7fdaf41ef750 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c7d0 tx=0x7fdaec00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:17.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.124+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec003b70 con 0x7fdaf415a340 2024-03-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.124+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaec003d30 con 0x7fdaf415a340 2024-03-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.124+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00aa90 con 0x7fdaf415a340 2024-03-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.124+0000 7fdafc400640 1 -- 172.21.15.67:0/3401892363 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf41ebc90 con 0x7fdaf415a340 2024-03-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.124+0000 7fdafc400640 1 -- 172.21.15.67:0/3401892363 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf41ec250 con 0x7fdaf415a340 2024-03-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.125+0000 7fdafc400640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41ec420 0x7fdaf40a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.125+0000 7fdafc400640 1 -- 172.21.15.67:0/3401892363 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf404f3c0 con 0x7fdaf41ec420 2024-03-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.126+0000 7fdaf9a00640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41ec420 0x7fdaf40a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.126+0000 7fdaf9a00640 1 -- 172.21.15.67:0/3401892363 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdaf406bfd0 con 0x7fdaf41ec420 2024-03-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.126+0000 7fdaf9a00640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41ec420 0x7fdaf40a5da0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40c5710 tx=0x7fdae80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.127+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8041070 con 0x7fdaf41ec420 2024-03-09T08:57:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.127+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdaec00ac30 con 0x7fdaf415a340 2024-03-09T08:57:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.128+0000 7fdaf1600640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdad406b3d0 0x7fdad406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.128+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 8361+0+0 (secure 0 0 0) 0x7fdaec08abe0 con 0x7fdaf415a340 2024-03-09T08:57:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.128+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fdae8009490 con 0x7fdaf41ec420 2024-03-09T08:57:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.128+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41ec420 msgr2=0x7fdaf40a5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.128+0000 7fdaf1600640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41ec420 0x7fdaf40a5da0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40c5710 tx=0x7fdae80046c0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.128+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41ec420 msgr2=0x7fdaf40a5da0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.128+0000 7fdaf1600640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41ec420 0x7fdaf40a5da0 secure :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40c5710 tx=0x7fdae80046c0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.128+0000 7fdaf9000640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdad406b3d0 0x7fdad406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.129+0000 7fdaf9000640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdad406b3d0 0x7fdad406d890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4073100 tx=0x7fdae0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.135+0000 7fdafc400640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41eb740 0x7fdaf40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.135+0000 7fdafc400640 1 -- 172.21.15.67:0/3401892363 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40a6820 con 0x7fdaf41eb740 2024-03-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.136+0000 7fdaf9a00640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41eb740 0x7fdaf40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.136+0000 7fdaf9a00640 1 -- 172.21.15.67:0/3401892363 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdaf4075a00 con 0x7fdaf41eb740 2024-03-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.136+0000 7fdaf9a00640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41eb740 0x7fdaf40a62e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdae8000c00 tx=0x7fdae8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.137+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8035040 con 0x7fdaf41eb740 2024-03-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.137+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5177 (secure 0 0 0) 0x7fdae8045400 con 0x7fdaf41eb740 2024-03-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.137+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41eb740 msgr2=0x7fdaf40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.137+0000 7fdaf1600640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41eb740 0x7fdaf40a62e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdae8000c00 tx=0x7fdae8009240 comp rx=0 tx=0).stop 2024-03-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.137+0000 7fdaf1600640 1 -- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41eb740 msgr2=0x7fdaf40a62e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.137+0000 7fdaf1600640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf41eb740 0x7fdaf40a62e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 40, 2024-03-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.138+0000 7fdafc400640 1 -- 172.21.15.67:0/3401892363 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdad406b3d0 msgr2=0x7fdad406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.138+0000 7fdafc400640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdad406b3d0 0x7fdad406d890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4073100 tx=0x7fdae0009040 comp rx=0 tx=0).stop 2024-03-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.138+0000 7fdafc400640 1 -- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf415a340 msgr2=0x7fdaf41ef750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.138+0000 7fdafc400640 1 --2- 172.21.15.67:0/3401892363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdaf415a340 0x7fdaf41ef750 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c7d0 tx=0x7fdaec00cc90 comp rx=0 tx=0).stop 2024-03-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.138+0000 7fdafa400640 1 -- 172.21.15.67:0/3401892363 reap_dead start 2024-03-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.138+0000 7fdafa400640 1 -- 172.21.15.67:0/3401892363 reap_dead start 2024-03-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.138+0000 7fdafc400640 1 -- 172.21.15.67:0/3401892363 shutdown_connections 2024-03-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.138+0000 7fdafc400640 1 -- 172.21.15.67:0/3401892363 >> 172.21.15.67:0/3401892363 conn(0x7fdaf40a9950 msgr2=0x7fdaf40a7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.138+0000 7fdafc400640 1 -- 172.21.15.67:0/3401892363 shutdown_connections 2024-03-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.138+0000 7fdafc400640 1 -- 172.21.15.67:0/3401892363 wait complete. 2024-03-09T08:57:17.153 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:57:17.153 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T08:57:17.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:57:17.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.346+0000 7f6b59200640 1 Processor -- start 2024-03-09T08:57:17.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.346+0000 7f6b59200640 1 -- start start 2024-03-09T08:57:17.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.348+0000 7f6b59200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b540c8c60 0x7f6b540c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.348+0000 7f6b59200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6b540c9d50 0x7f6b540caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.348+0000 7f6b59200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5406b040 con 0x7f6b54153460 2024-03-09T08:57:17.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.348+0000 7f6b59200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b540ca590 con 0x7f6b540c9d50 2024-03-09T08:57:17.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.348+0000 7f6b59200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b54156710 con 0x7f6b540c8c60 2024-03-09T08:57:17.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.348+0000 7f6b51400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b540c8c60 msgr2=0x7f6b540c9810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:17.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.348+0000 7f6b51400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b540c8c60 0x7f6b540c9810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:17.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.348+0000 7f6b51e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6b54153460 0x7f6b54155ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48212/0 (socket says 172.21.15.67:48212) 2024-03-09T08:57:17.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.348+0000 7f6b51e00640 1 -- 172.21.15.67:0/2678469857 learned_addr learned my addr 172.21.15.67:0/2678469857 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:17.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.349+0000 7f6b52800640 1 --2- 172.21.15.67:0/2678469857 >> v2:172.21.15.134:3300/0 conn(0x7f6b540c9d50 0x7f6b540caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.349+0000 7f6b50a00640 1 -- 172.21.15.67:0/2678469857 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728943987 0 0) 0x7f6b44001110 con 0x7f6b54153460 2024-03-09T08:57:17.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.349+0000 7f6b50a00640 1 -- 172.21.15.67:0/2678469857 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b38003520 con 0x7f6b54153460 2024-03-09T08:57:17.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.349+0000 7f6b52800640 1 -- 172.21.15.67:0/2678469857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b540c8c60 msgr2=0x7f6b540c9810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:57:17.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.349+0000 7f6b52800640 1 --2- 172.21.15.67:0/2678469857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b540c8c60 0x7f6b540c9810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.349+0000 7f6b52800640 1 -- 172.21.15.67:0/2678469857 >> v1:172.21.15.67:6789/0 conn(0x7f6b54153460 legacy=0x7f6b54155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.349+0000 7f6b50a00640 1 -- 172.21.15.67:0/2678469857 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2836386034 0 0) 0x7f6b38003520 con 0x7f6b54153460 2024-03-09T08:57:17.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.349+0000 7f6b52800640 1 -- 172.21.15.67:0/2678469857 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b541569f0 con 0x7f6b540c9d50 2024-03-09T08:57:17.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.351+0000 7f6b52800640 1 --2- 172.21.15.67:0/2678469857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b540c9d50 0x7f6b540caa20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c009d90 tx=0x7f6b4c02bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff2b5b22722ed759 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:17.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.351+0000 7f6b50a00640 1 -- 172.21.15.67:0/2678469857 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c039070 con 0x7f6b540c9d50 2024-03-09T08:57:17.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.352+0000 7f6b50a00640 1 -- 172.21.15.67:0/2678469857 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c02c830 con 0x7f6b540c9d50 2024-03-09T08:57:17.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.352+0000 7f6b50a00640 1 -- 172.21.15.67:0/2678469857 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44001840 con 0x7f6b540c9d50 2024-03-09T08:57:17.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.352+0000 7f6b59200640 1 -- 172.21.15.67:0/2678469857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b540c9d50 msgr2=0x7f6b540caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.352+0000 7f6b59200640 1 --2- 172.21.15.67:0/2678469857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b540c9d50 0x7f6b540caa20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c009d90 tx=0x7f6b4c02bd00 comp rx=0 tx=0).stop 2024-03-09T08:57:17.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.352+0000 7f6b59200640 1 -- 172.21.15.67:0/2678469857 shutdown_connections 2024-03-09T08:57:17.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.352+0000 7f6b59200640 1 --2- 172.21.15.67:0/2678469857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b540c8c60 0x7f6b540c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.352+0000 7f6b59200640 1 --2- 172.21.15.67:0/2678469857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b540c9d50 0x7f6b540caa20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.352+0000 7f6b59200640 1 -- 172.21.15.67:0/2678469857 >> 172.21.15.67:0/2678469857 conn(0x7f6b540c8170 msgr2=0x7f6b540c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:17.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.352+0000 7f6b59200640 1 -- 172.21.15.67:0/2678469857 shutdown_connections 2024-03-09T08:57:17.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.353+0000 7f6b59200640 1 -- 172.21.15.67:0/2678469857 wait complete. 2024-03-09T08:57:17.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.355+0000 7f6b59200640 1 Processor -- start 2024-03-09T08:57:17.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.355+0000 7f6b59200640 1 -- start start 2024-03-09T08:57:17.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.356+0000 7f6b59200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b540c8c60 0x7f6b540c5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.356+0000 7f6b59200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b540c9d50 0x7f6b54154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.356+0000 7f6b59200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b54153460 0x7f6b540c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.356+0000 7f6b59200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b540caf60 con 0x7f6b540c9d50 2024-03-09T08:57:17.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.356+0000 7f6b59200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b54153840 con 0x7f6b540c8c60 2024-03-09T08:57:17.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.356+0000 7f6b59200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b540cb0e0 con 0x7f6b54153460 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.356+0000 7f6b51400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b540c9d50 0x7f6b54154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.356+0000 7f6b52800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b54153460 msgr2=0x7f6b540c2930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.356+0000 7f6b52800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b54153460 0x7f6b540c2930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.356+0000 7f6b51400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b540c9d50 0x7f6b54154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57176/0 (socket says 172.21.15.67:57176) 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.356+0000 7f6b51400640 1 -- 172.21.15.67:0/1744725440 learned_addr learned my addr 172.21.15.67:0/1744725440 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.357+0000 7f6b51e00640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b540c8c60 0x7f6b540c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.357+0000 7f6b51400640 1 -- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b54153460 msgr2=0x7f6b540c2930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.357+0000 7f6b51400640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b54153460 0x7f6b540c2930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.357+0000 7f6b51400640 1 -- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b540c8c60 msgr2=0x7f6b540c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.357+0000 7f6b51400640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b540c8c60 0x7f6b540c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.357+0000 7f6b51400640 1 -- 172.21.15.67:0/1744725440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5414e7b0 con 0x7f6b540c9d50 2024-03-09T08:57:17.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.357+0000 7f6b51400640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b540c9d50 0x7f6b54154ff0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000b580 tx=0x7f6b4000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:17.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.357+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40013020 con 0x7f6b540c9d50 2024-03-09T08:57:17.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.358+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b40003de0 con 0x7f6b540c9d50 2024-03-09T08:57:17.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.358+0000 7f6b59200640 1 -- 172.21.15.67:0/1744725440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b541ef860 con 0x7f6b540c9d50 2024-03-09T08:57:17.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.358+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b400052b0 con 0x7f6b540c9d50 2024-03-09T08:57:17.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.358+0000 7f6b59200640 1 -- 172.21.15.67:0/1744725440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5414ed50 con 0x7f6b540c9d50 2024-03-09T08:57:17.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.359+0000 7f6b59200640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b541487b0 0x7f6b540c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.359+0000 7f6b59200640 1 -- 172.21.15.67:0/1744725440 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b540752b0 con 0x7f6b541487b0 2024-03-09T08:57:17.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.359+0000 7f6b51e00640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b541487b0 0x7f6b540c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.360+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6b40016020 con 0x7f6b540c9d50 2024-03-09T08:57:17.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.360+0000 7f6b51e00640 1 -- 172.21.15.67:0/1744725440 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b54075a00 con 0x7f6b541487b0 2024-03-09T08:57:17.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.361+0000 7f6b51e00640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b541487b0 0x7f6b540c2e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b44005880 tx=0x7f6b4400eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:17.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.361+0000 7f6b4a000640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b3406b480 0x7f6b3406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.361+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 8361+0+0 (secure 0 0 0) 0x7f6b4008aa30 con 0x7f6b540c9d50 2024-03-09T08:57:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.362+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b440058d0 con 0x7f6b541487b0 2024-03-09T08:57:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.362+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f6b44018070 con 0x7f6b541487b0 2024-03-09T08:57:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.362+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b541487b0 msgr2=0x7f6b540c2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.362+0000 7f6b4a000640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b541487b0 0x7f6b540c2e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b44005880 tx=0x7f6b4400eb90 comp rx=0 tx=0).stop 2024-03-09T08:57:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.362+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b541487b0 msgr2=0x7f6b540c2e70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.362+0000 7f6b4a000640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b541487b0 0x7f6b540c2e70 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b44005880 tx=0x7f6b4400eb90 comp rx=0 tx=0).stop 2024-03-09T08:57:17.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.362+0000 7f6b52800640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b3406b480 0x7f6b3406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.363+0000 7f6b52800640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b3406b480 0x7f6b3406d940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c036000 tx=0x7f6b4c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:17.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.370+0000 7f6b59200640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b54147b40 0x7f6b54147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:17.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.370+0000 7f6b59200640 1 -- 172.21.15.67:0/1744725440 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b540c95f0 con 0x7f6b54147b40 2024-03-09T08:57:17.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.370+0000 7f6b51e00640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b54147b40 0x7f6b54147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.371+0000 7f6b51e00640 1 -- 172.21.15.67:0/1744725440 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6b54155530 con 0x7f6b54147b40 2024-03-09T08:57:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.371+0000 7f6b51e00640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b54147b40 0x7f6b54147f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b44017000 tx=0x7f6b440057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.372+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b440075b0 con 0x7f6b54147b40 2024-03-09T08:57:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.372+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4862 (secure 0 0 0) 0x7f6b440075b0 con 0x7f6b54147b40 2024-03-09T08:57:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.372+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b54147b40 msgr2=0x7f6b54147f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.372+0000 7f6b4a000640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b54147b40 0x7f6b54147f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b44017000 tx=0x7f6b440057f0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.372+0000 7f6b4a000640 1 -- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b54147b40 msgr2=0x7f6b54147f50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.372+0000 7f6b4a000640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b54147b40 0x7f6b54147f50 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b44017000 tx=0x7f6b440057f0 comp rx=0 tx=0).stop 2024-03-09T08:57:17.371 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:57:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:57:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 40, 2024-03-09T08:57:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:57:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:57:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:17.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:17.384 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:57:17.384 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:57:17.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.373+0000 7f6b59200640 1 -- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b3406b480 msgr2=0x7f6b3406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.373+0000 7f6b59200640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b3406b480 0x7f6b3406d940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c036000 tx=0x7f6b4c030000 comp rx=0 tx=0).stop 2024-03-09T08:57:17.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.373+0000 7f6b59200640 1 -- 172.21.15.67:0/1744725440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b540c9d50 msgr2=0x7f6b54154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:17.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.373+0000 7f6b59200640 1 --2- 172.21.15.67:0/1744725440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b540c9d50 0x7f6b54154ff0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000b580 tx=0x7f6b4000ba40 comp rx=0 tx=0).stop 2024-03-09T08:57:17.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.373+0000 7f6b52800640 1 -- 172.21.15.67:0/1744725440 reap_dead start 2024-03-09T08:57:17.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.373+0000 7f6b52800640 1 -- 172.21.15.67:0/1744725440 reap_dead start 2024-03-09T08:57:17.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.373+0000 7f6b59200640 1 -- 172.21.15.67:0/1744725440 shutdown_connections 2024-03-09T08:57:17.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.373+0000 7f6b59200640 1 -- 172.21.15.67:0/1744725440 >> 172.21.15.67:0/1744725440 conn(0x7f6b540c8170 msgr2=0x7f6b540ab190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:17.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.373+0000 7f6b59200640 1 -- 172.21.15.67:0/1744725440 shutdown_connections 2024-03-09T08:57:17.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:17.373+0000 7f6b59200640 1 -- 172.21.15.67:0/1744725440 wait complete. 2024-03-09T08:57:17.389 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:57:17.389 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T08:57:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/34' 2024-03-09T08:57:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/38' 2024-03-09T08:57:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/28' 2024-03-09T08:57:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/24' 2024-03-09T08:57:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/29' 2024-03-09T08:57:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/32' 2024-03-09T08:57:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/22' 2024-03-09T08:57:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/25' 2024-03-09T08:57:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/30' 2024-03-09T08:57:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/33' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/23' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/35' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/26' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './206/37' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './206' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/21' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/27' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/31' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/36' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/20' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/39' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/34' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/38' 2024-03-09T08:57:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/28' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/24' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/29' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/32' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/22' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/25' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/30' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/33' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/23' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/35' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/26' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './207/37' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './207' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/21' 2024-03-09T08:57:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/27' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/31' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/36' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/20' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/39' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/34' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/38' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/28' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/24' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/29' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/32' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/22' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/25' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/30' 2024-03-09T08:57:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/33' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/23' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/35' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/26' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './208/37' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './208' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/21' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/27' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/31' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/36' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/20' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/39' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/34' 2024-03-09T08:57:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/38' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/28' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/24' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/29' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/32' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/22' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/25' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/30' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/33' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/23' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/35' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/26' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './209/37' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './209' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/21' 2024-03-09T08:57:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/27' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/31' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/36' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/20' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/39' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/34' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/38' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/28' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/24' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/29' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/32' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/22' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/25' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/30' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/33' 2024-03-09T08:57:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/23' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/35' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/26' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './210/37' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './210' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/21' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/27' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/31' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/36' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/20' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/39' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/34' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/38' 2024-03-09T08:57:17.966 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/28' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/24' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/29' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/32' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/22' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/25' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/30' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/33' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/23' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/35' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/26' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './211/37' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './211' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/21' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/27' 2024-03-09T08:57:17.967 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/31' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/36' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/20' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/39' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/34' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/38' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/28' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/24' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/29' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/32' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/22' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/25' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/30' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/33' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/23' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/35' 2024-03-09T08:57:17.968 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/26' 2024-03-09T08:57:17.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './212/37' 2024-03-09T08:57:17.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './212' 2024-03-09T08:57:17.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/21' 2024-03-09T08:57:17.969 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/27' 2024-03-09T08:57:17.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/31' 2024-03-09T08:57:17.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/36' 2024-03-09T08:57:17.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/20' 2024-03-09T08:57:17.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/39' 2024-03-09T08:57:17.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/34' 2024-03-09T08:57:17.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/38' 2024-03-09T08:57:17.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/28' 2024-03-09T08:57:17.970 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/24' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/29' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/32' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/22' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/25' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/30' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/33' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/23' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/35' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/26' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './213/37' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './213' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/21' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/27' 2024-03-09T08:57:17.971 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/31' 2024-03-09T08:57:17.972 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/36' 2024-03-09T08:57:17.972 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/20' 2024-03-09T08:57:17.972 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/39' 2024-03-09T08:57:17.972 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/34' 2024-03-09T08:57:17.972 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/38' 2024-03-09T08:57:17.972 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/28' 2024-03-09T08:57:17.972 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/24' 2024-03-09T08:57:17.972 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/29' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/32' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/22' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/25' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/30' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/33' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/23' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/35' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/26' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './214/37' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './214' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/21' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/27' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/31' 2024-03-09T08:57:17.973 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/36' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/20' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/39' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/34' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/38' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/28' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/24' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/29' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/32' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/22' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/25' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/30' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/33' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/23' 2024-03-09T08:57:17.974 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/35' 2024-03-09T08:57:17.975 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/26' 2024-03-09T08:57:17.975 INFO:tasks.workunit.client.0.smithi067.stdout:removed './215/37' 2024-03-09T08:57:17.975 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './215' 2024-03-09T08:57:17.975 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/21' 2024-03-09T08:57:17.975 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/27' 2024-03-09T08:57:17.975 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/31' 2024-03-09T08:57:17.975 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/36' 2024-03-09T08:57:17.975 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/20' 2024-03-09T08:57:17.975 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/39' 2024-03-09T08:57:17.975 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/34' 2024-03-09T08:57:17.975 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/38' 2024-03-09T08:57:18.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/34' 2024-03-09T08:57:18.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/38' 2024-03-09T08:57:18.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/11' 2024-03-09T08:57:18.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/16' 2024-03-09T08:57:18.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/32' 2024-03-09T08:57:18.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/30' 2024-03-09T08:57:18.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/14' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/18' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/13' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/33' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/17' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/35' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './196/37' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './196' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/19' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/15' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/31' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/36' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/20' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/12' 2024-03-09T08:57:18.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/39' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/10' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/34' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/38' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/11' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/16' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/32' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/30' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/14' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/18' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/13' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/33' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/17' 2024-03-09T08:57:18.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/35' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './197/37' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './197' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/19' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/15' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/31' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/36' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/20' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/12' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/39' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/10' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/34' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/38' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/11' 2024-03-09T08:57:18.979 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/16' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/32' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/30' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/14' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/18' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/13' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/33' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/17' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/35' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './198/37' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './198' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/19' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/15' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/31' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/36' 2024-03-09T08:57:18.980 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/20' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/12' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/39' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/10' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/34' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/38' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/11' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/16' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/32' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/30' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/14' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/18' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/13' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/33' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/17' 2024-03-09T08:57:18.981 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/35' 2024-03-09T08:57:18.982 INFO:tasks.workunit.client.1.smithi134.stdout:removed './199/37' 2024-03-09T08:57:18.982 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './199' 2024-03-09T08:57:18.982 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/21' 2024-03-09T08:57:18.982 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/27' 2024-03-09T08:57:18.982 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/31' 2024-03-09T08:57:18.982 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/36' 2024-03-09T08:57:18.982 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/20' 2024-03-09T08:57:18.983 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/39' 2024-03-09T08:57:18.983 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/34' 2024-03-09T08:57:18.983 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/38' 2024-03-09T08:57:18.983 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/28' 2024-03-09T08:57:18.983 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/24' 2024-03-09T08:57:18.983 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/29' 2024-03-09T08:57:18.983 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/32' 2024-03-09T08:57:18.983 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/22' 2024-03-09T08:57:18.983 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/25' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/30' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/33' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/23' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/35' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/26' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './200/37' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './200' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/21' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/27' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/31' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/36' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/20' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/39' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/34' 2024-03-09T08:57:18.984 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/38' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/28' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/24' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/29' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/32' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/22' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/25' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/30' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/33' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/23' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/35' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/26' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './201/37' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './201' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/21' 2024-03-09T08:57:18.985 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/27' 2024-03-09T08:57:18.986 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/31' 2024-03-09T08:57:18.986 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/36' 2024-03-09T08:57:18.986 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/20' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/39' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/34' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/38' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/28' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/24' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/29' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/32' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/22' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/25' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/30' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/33' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/23' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/35' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/26' 2024-03-09T08:57:18.987 INFO:tasks.workunit.client.1.smithi134.stdout:removed './202/37' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './202' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/21' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/27' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/31' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/36' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/20' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/39' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/34' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/38' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/28' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/24' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/29' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/32' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/22' 2024-03-09T08:57:18.988 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/25' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/30' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/33' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/23' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/35' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/26' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './203/37' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './203' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/21' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/27' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/31' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/36' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/20' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/39' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/34' 2024-03-09T08:57:18.989 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/38' 2024-03-09T08:57:18.990 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/28' 2024-03-09T08:57:18.990 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/24' 2024-03-09T08:57:18.990 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/29' 2024-03-09T08:57:18.990 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/32' 2024-03-09T08:57:18.990 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/22' 2024-03-09T08:57:18.990 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/25' 2024-03-09T08:57:18.990 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/30' 2024-03-09T08:57:18.990 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/33' 2024-03-09T08:57:18.991 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/23' 2024-03-09T08:57:18.991 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/35' 2024-03-09T08:57:18.991 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/26' 2024-03-09T08:57:18.991 INFO:tasks.workunit.client.1.smithi134.stdout:removed './204/37' 2024-03-09T08:57:18.991 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './204' 2024-03-09T08:57:18.991 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/21' 2024-03-09T08:57:18.991 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/27' 2024-03-09T08:57:18.991 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/31' 2024-03-09T08:57:18.992 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/36' 2024-03-09T08:57:18.992 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/20' 2024-03-09T08:57:18.992 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/39' 2024-03-09T08:57:18.992 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/34' 2024-03-09T08:57:18.992 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/38' 2024-03-09T08:57:18.992 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/28' 2024-03-09T08:57:18.992 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/24' 2024-03-09T08:57:18.992 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/29' 2024-03-09T08:57:18.992 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/32' 2024-03-09T08:57:18.992 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/22' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/25' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/30' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/33' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/23' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/35' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/26' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './205/37' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './205' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/21' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/27' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/31' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/36' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/20' 2024-03-09T08:57:18.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/39' 2024-03-09T08:57:20.264 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/34' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/38' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/28' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/24' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/29' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/32' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/22' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/25' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/30' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/33' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/23' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/35' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/26' 2024-03-09T08:57:20.265 INFO:tasks.workunit.client.1.smithi134.stdout:removed './206/37' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './206' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/21' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/27' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/31' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/36' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/20' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/39' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/34' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/38' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/28' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/24' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/29' 2024-03-09T08:57:20.266 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/32' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/22' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/25' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/30' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/33' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/23' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/35' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/26' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './207/37' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './207' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/21' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/27' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/31' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/36' 2024-03-09T08:57:20.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/20' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/39' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/34' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/38' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/28' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/24' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/29' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/32' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/22' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/25' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/30' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/33' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/23' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/35' 2024-03-09T08:57:20.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/26' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './208/37' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './208' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/21' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/27' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/31' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/36' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/20' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/39' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/34' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/38' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/28' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/24' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/29' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/32' 2024-03-09T08:57:20.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/22' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/25' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/30' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/33' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/23' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/35' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/26' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './209/37' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './209' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/21' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/27' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/31' 2024-03-09T08:57:20.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/36' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/20' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/39' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/34' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/38' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/28' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/24' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/29' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/32' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/22' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/25' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/30' 2024-03-09T08:57:20.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/33' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/23' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/35' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/26' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './210/37' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './210' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/21' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/27' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/31' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/36' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/20' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/39' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/34' 2024-03-09T08:57:20.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/38' 2024-03-09T08:57:20.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/28' 2024-03-09T08:57:20.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/24' 2024-03-09T08:57:20.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/29' 2024-03-09T08:57:20.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/32' 2024-03-09T08:57:20.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/22' 2024-03-09T08:57:20.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/25' 2024-03-09T08:57:20.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/30' 2024-03-09T08:57:20.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/33' 2024-03-09T08:57:20.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/23' 2024-03-09T08:57:20.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/35' 2024-03-09T08:57:20.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/26' 2024-03-09T08:57:20.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './211/37' 2024-03-09T08:57:20.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './211' 2024-03-09T08:57:20.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/21' 2024-03-09T08:57:20.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/27' 2024-03-09T08:57:20.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/31' 2024-03-09T08:57:20.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/36' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/20' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/39' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/34' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/38' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/28' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/24' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/29' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/32' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/22' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/25' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/30' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/33' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/23' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/35' 2024-03-09T08:57:20.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/26' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './212/37' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './212' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/21' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/27' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/31' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/36' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/20' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/39' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/34' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/38' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/28' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/24' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/29' 2024-03-09T08:57:20.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/32' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/22' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/25' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/30' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/33' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/23' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/35' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/26' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed './213/37' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './213' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/21' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/27' 2024-03-09T08:57:20.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/31' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/36' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/20' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/39' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/34' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/38' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/28' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/24' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/29' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/32' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/22' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/25' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/30' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/33' 2024-03-09T08:57:20.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/23' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/35' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/26' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './214/37' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './214' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/21' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/27' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/31' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/36' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/20' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/39' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/34' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/38' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/28' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/24' 2024-03-09T08:57:20.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/29' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/32' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/22' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/25' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/30' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/33' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/23' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/35' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/26' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './215/37' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './215' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/21' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/27' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/31' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/36' 2024-03-09T08:57:20.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/20' 2024-03-09T08:57:20.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/39' 2024-03-09T08:57:20.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/34' 2024-03-09T08:57:20.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/38' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/28' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/24' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/29' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/32' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/22' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/25' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/30' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/33' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/23' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/35' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/26' 2024-03-09T08:57:22.363 INFO:tasks.workunit.client.1.smithi134.stdout:removed './216/37' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './216' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/21' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/27' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/31' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/36' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/20' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/39' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/34' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/38' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/28' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/24' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/29' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/32' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/22' 2024-03-09T08:57:22.364 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/25' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/30' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/33' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/23' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/35' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/26' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './217/37' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './217' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/21' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/27' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/31' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/36' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/20' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/39' 2024-03-09T08:57:22.365 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/34' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/38' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/28' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/24' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/29' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/32' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/22' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/25' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/30' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/33' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/23' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/35' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/26' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed './218/37' 2024-03-09T08:57:22.366 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './218' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/21' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/27' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/31' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/36' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/20' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/39' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/34' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/38' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/28' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/24' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/29' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/32' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/22' 2024-03-09T08:57:22.367 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/25' 2024-03-09T08:57:22.368 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/30' 2024-03-09T08:57:22.368 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/33' 2024-03-09T08:57:22.368 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/23' 2024-03-09T08:57:22.368 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/35' 2024-03-09T08:57:22.368 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/26' 2024-03-09T08:57:22.368 INFO:tasks.workunit.client.1.smithi134.stdout:removed './219/37' 2024-03-09T08:57:22.368 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './219' 2024-03-09T08:57:22.368 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/21' 2024-03-09T08:57:22.368 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/27' 2024-03-09T08:57:22.368 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/31' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/36' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/20' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/39' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/34' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/38' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/28' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/24' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/29' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/32' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/22' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/25' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/30' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/33' 2024-03-09T08:57:22.369 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/23' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/35' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/26' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed './220/37' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './220' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/21' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/27' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/31' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/36' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/20' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/39' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/34' 2024-03-09T08:57:22.370 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/38' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/28' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/24' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/29' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/32' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/22' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/25' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/30' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/33' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/23' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/35' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/26' 2024-03-09T08:57:22.371 INFO:tasks.workunit.client.1.smithi134.stdout:removed './221/37' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './221' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/21' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/27' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/31' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/36' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/20' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/39' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/34' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/38' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/28' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/24' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/29' 2024-03-09T08:57:22.372 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/32' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/22' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/25' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/30' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/33' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/23' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/35' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/26' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed './222/37' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './222' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/21' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/27' 2024-03-09T08:57:22.373 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/31' 2024-03-09T08:57:22.374 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/36' 2024-03-09T08:57:22.374 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/20' 2024-03-09T08:57:22.374 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/39' 2024-03-09T08:57:22.374 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/34' 2024-03-09T08:57:22.374 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/38' 2024-03-09T08:57:22.374 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/28' 2024-03-09T08:57:22.374 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/24' 2024-03-09T08:57:22.374 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/29' 2024-03-09T08:57:22.374 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/32' 2024-03-09T08:57:22.374 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/22' 2024-03-09T08:57:22.374 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/25' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/30' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/33' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/23' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/35' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/26' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './223/37' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './223' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/21' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/27' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/31' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/36' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/20' 2024-03-09T08:57:22.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/39' 2024-03-09T08:57:22.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/34' 2024-03-09T08:57:22.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/38' 2024-03-09T08:57:22.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/28' 2024-03-09T08:57:22.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/24' 2024-03-09T08:57:22.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/29' 2024-03-09T08:57:22.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/32' 2024-03-09T08:57:22.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/22' 2024-03-09T08:57:22.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/25' 2024-03-09T08:57:22.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/30' 2024-03-09T08:57:22.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/33' 2024-03-09T08:57:22.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/23' 2024-03-09T08:57:22.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/35' 2024-03-09T08:57:22.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/26' 2024-03-09T08:57:22.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed './224/37' 2024-03-09T08:57:22.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './224' 2024-03-09T08:57:22.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/21' 2024-03-09T08:57:22.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/27' 2024-03-09T08:57:22.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/31' 2024-03-09T08:57:22.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/36' 2024-03-09T08:57:22.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/20' 2024-03-09T08:57:22.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/39' 2024-03-09T08:57:22.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/34' 2024-03-09T08:57:22.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/38' 2024-03-09T08:57:22.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/28' 2024-03-09T08:57:22.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/24' 2024-03-09T08:57:22.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/29' 2024-03-09T08:57:22.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/32' 2024-03-09T08:57:22.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/22' 2024-03-09T08:57:22.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/25' 2024-03-09T08:57:22.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/30' 2024-03-09T08:57:22.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/33' 2024-03-09T08:57:22.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/23' 2024-03-09T08:57:22.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/35' 2024-03-09T08:57:22.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/26' 2024-03-09T08:57:22.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed './225/37' 2024-03-09T08:57:22.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './225' 2024-03-09T08:57:22.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/21' 2024-03-09T08:57:22.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/27' 2024-03-09T08:57:22.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/31' 2024-03-09T08:57:22.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/36' 2024-03-09T08:57:22.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/20' 2024-03-09T08:57:22.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/39' 2024-03-09T08:57:22.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/34' 2024-03-09T08:57:22.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/38' 2024-03-09T08:57:22.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/28' 2024-03-09T08:57:22.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/24' 2024-03-09T08:57:22.665 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/28' 2024-03-09T08:57:22.665 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/24' 2024-03-09T08:57:22.665 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/29' 2024-03-09T08:57:22.665 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/32' 2024-03-09T08:57:22.666 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/22' 2024-03-09T08:57:22.666 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/25' 2024-03-09T08:57:22.666 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/30' 2024-03-09T08:57:22.666 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/33' 2024-03-09T08:57:22.666 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/23' 2024-03-09T08:57:22.666 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/35' 2024-03-09T08:57:22.666 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/26' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './216/37' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './216' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/21' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/27' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/31' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/36' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/20' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/39' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/34' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/38' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/28' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/24' 2024-03-09T08:57:22.669 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/29' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/32' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/22' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/25' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/30' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/33' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/23' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/35' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/26' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './217/37' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './217' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/21' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/27' 2024-03-09T08:57:22.670 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/31' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/36' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/20' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/39' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/34' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/38' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/28' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/24' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/29' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/32' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/22' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/25' 2024-03-09T08:57:22.671 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/30' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/33' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/23' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/35' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/26' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './218/37' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './218' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/21' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/27' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/31' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/36' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/20' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/39' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/34' 2024-03-09T08:57:22.672 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/38' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/28' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/24' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/29' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/32' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/22' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/25' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/30' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/33' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/23' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/35' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/26' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './219/37' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './219' 2024-03-09T08:57:22.673 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/21' 2024-03-09T08:57:22.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/27' 2024-03-09T08:57:22.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/31' 2024-03-09T08:57:22.674 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/36' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/20' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/39' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/34' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/38' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/28' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/24' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/29' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/32' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/22' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/25' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/30' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/33' 2024-03-09T08:57:22.675 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/23' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/35' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/26' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './220/37' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './220' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/21' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/27' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/31' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/36' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/20' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/39' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/34' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/38' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/28' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/24' 2024-03-09T08:57:22.676 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/29' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/32' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/22' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/25' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/30' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/33' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/23' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/35' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/26' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './221/37' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './221' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/21' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/27' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/31' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/36' 2024-03-09T08:57:22.677 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/20' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/39' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/34' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/38' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/28' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/24' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/29' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/32' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/22' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/25' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/30' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/33' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/23' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/35' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/26' 2024-03-09T08:57:22.678 INFO:tasks.workunit.client.0.smithi067.stdout:removed './222/37' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './222' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/21' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/27' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/31' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/36' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/20' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/39' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/34' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/38' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/28' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/24' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/29' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/32' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/22' 2024-03-09T08:57:22.679 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/25' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/30' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/33' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/23' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/35' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/26' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './223/37' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './223' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/21' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/27' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/31' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/36' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/20' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/39' 2024-03-09T08:57:22.680 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/34' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/38' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/28' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/24' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/29' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/32' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/22' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/25' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/30' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/33' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/23' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/35' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/26' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './224/37' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './224' 2024-03-09T08:57:22.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/21' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/27' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/31' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/36' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/20' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/39' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/34' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/38' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/28' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/24' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/29' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/32' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/22' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/25' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/30' 2024-03-09T08:57:22.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/33' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/23' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/35' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/26' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './225/37' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './225' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/21' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/27' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/31' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/36' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/20' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/39' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/34' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/38' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/28' 2024-03-09T08:57:22.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/24' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/29' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/32' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/22' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/25' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/30' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/33' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/23' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/35' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/26' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed './226/37' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './226' 2024-03-09T08:57:25.544 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/21' 2024-03-09T08:57:25.545 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/27' 2024-03-09T08:57:25.545 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/31' 2024-03-09T08:57:25.545 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/36' 2024-03-09T08:57:25.545 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/20' 2024-03-09T08:57:25.545 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/39' 2024-03-09T08:57:25.545 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/34' 2024-03-09T08:57:25.545 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/38' 2024-03-09T08:57:25.545 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/28' 2024-03-09T08:57:25.545 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/24' 2024-03-09T08:57:25.545 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/29' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/32' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/22' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/25' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/30' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/33' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/23' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/35' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/26' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './227/37' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './227' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/21' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/27' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/31' 2024-03-09T08:57:25.546 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/36' 2024-03-09T08:57:25.547 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/20' 2024-03-09T08:57:25.547 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/39' 2024-03-09T08:57:25.547 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/34' 2024-03-09T08:57:25.547 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/38' 2024-03-09T08:57:25.547 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/28' 2024-03-09T08:57:25.547 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/24' 2024-03-09T08:57:25.547 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/29' 2024-03-09T08:57:25.547 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/32' 2024-03-09T08:57:25.547 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/22' 2024-03-09T08:57:25.547 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/25' 2024-03-09T08:57:25.547 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/30' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/33' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/23' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/35' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/26' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './228/37' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './228' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/21' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/27' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/31' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/36' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/20' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/39' 2024-03-09T08:57:25.548 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/34' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/38' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/28' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/24' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/29' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/32' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/22' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/25' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/30' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/33' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/23' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/35' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/26' 2024-03-09T08:57:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:removed './229/37' 2024-03-09T08:57:25.550 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './229' 2024-03-09T08:57:25.550 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/21' 2024-03-09T08:57:25.550 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/27' 2024-03-09T08:57:25.550 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/31' 2024-03-09T08:57:25.550 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/36' 2024-03-09T08:57:25.550 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/20' 2024-03-09T08:57:25.550 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/39' 2024-03-09T08:57:25.550 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/34' 2024-03-09T08:57:25.550 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/38' 2024-03-09T08:57:25.550 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/28' 2024-03-09T08:57:25.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/24' 2024-03-09T08:57:25.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/29' 2024-03-09T08:57:25.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/32' 2024-03-09T08:57:25.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/22' 2024-03-09T08:57:25.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/25' 2024-03-09T08:57:25.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/30' 2024-03-09T08:57:25.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/33' 2024-03-09T08:57:25.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/23' 2024-03-09T08:57:25.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/35' 2024-03-09T08:57:25.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/26' 2024-03-09T08:57:25.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './230/37' 2024-03-09T08:57:25.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './230' 2024-03-09T08:57:25.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/21' 2024-03-09T08:57:25.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/27' 2024-03-09T08:57:25.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/31' 2024-03-09T08:57:25.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/36' 2024-03-09T08:57:25.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/20' 2024-03-09T08:57:25.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/39' 2024-03-09T08:57:25.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/34' 2024-03-09T08:57:25.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/38' 2024-03-09T08:57:25.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/28' 2024-03-09T08:57:25.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/24' 2024-03-09T08:57:25.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/29' 2024-03-09T08:57:25.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/32' 2024-03-09T08:57:25.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/22' 2024-03-09T08:57:25.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/25' 2024-03-09T08:57:25.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/30' 2024-03-09T08:57:25.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/33' 2024-03-09T08:57:25.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/23' 2024-03-09T08:57:25.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/35' 2024-03-09T08:57:25.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/26' 2024-03-09T08:57:25.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './231/37' 2024-03-09T08:57:25.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './231' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/21' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/27' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/31' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/36' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/20' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/39' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/34' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/38' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/28' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/24' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/29' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/32' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/22' 2024-03-09T08:57:25.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/25' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/30' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/33' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/23' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/35' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/26' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './232/37' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './232' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/21' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/27' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/31' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/36' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/20' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/39' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/34' 2024-03-09T08:57:25.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/38' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/28' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/24' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/29' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/32' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/22' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/25' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/30' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/33' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/23' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/35' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/26' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './233/37' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './233' 2024-03-09T08:57:25.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/21' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/27' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/31' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/36' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/20' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/39' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/34' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/38' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/28' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/24' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/29' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/32' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/22' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/25' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/30' 2024-03-09T08:57:25.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/33' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/23' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/35' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/26' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './234/37' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './234' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/21' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/27' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/31' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/36' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/20' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/39' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/34' 2024-03-09T08:57:25.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/38' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/28' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/24' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/29' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/32' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/22' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/25' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/30' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/33' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/23' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/35' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/26' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './235/37' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './235' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/21' 2024-03-09T08:57:25.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/27' 2024-03-09T08:57:25.560 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/31' 2024-03-09T08:57:25.560 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/36' 2024-03-09T08:57:25.560 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/20' 2024-03-09T08:57:25.560 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/39' 2024-03-09T08:57:25.560 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/34' 2024-03-09T08:57:25.560 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/38' 2024-03-09T08:57:25.560 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/28' 2024-03-09T08:57:25.560 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/24' 2024-03-09T08:57:25.560 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/29' 2024-03-09T08:57:25.560 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/32' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/29' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/32' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/22' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/25' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/30' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/33' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/23' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/35' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/26' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed './226/37' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './226' 2024-03-09T08:57:26.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/21' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/27' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/31' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/36' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/20' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/39' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/34' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/38' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/28' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/24' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/29' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/32' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/22' 2024-03-09T08:57:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/25' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/30' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/33' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/23' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/35' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/26' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './227/37' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './227' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/21' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/27' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/31' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/36' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/20' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/39' 2024-03-09T08:57:26.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/34' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/38' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/28' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/24' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/29' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/32' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/22' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/25' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/30' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/33' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/23' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/35' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/26' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed './228/37' 2024-03-09T08:57:26.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './228' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/21' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/27' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/31' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/36' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/20' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/39' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/34' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/38' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/28' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/24' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/29' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/32' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/22' 2024-03-09T08:57:26.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/25' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/30' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/33' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/23' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/35' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/26' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './229/37' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './229' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/21' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/27' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/31' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/36' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/20' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/39' 2024-03-09T08:57:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/34' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/38' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/28' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/24' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/29' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/32' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/22' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/25' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/30' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/33' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/23' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/35' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/26' 2024-03-09T08:57:26.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed './230/37' 2024-03-09T08:57:26.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './230' 2024-03-09T08:57:26.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/21' 2024-03-09T08:57:26.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/27' 2024-03-09T08:57:26.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/31' 2024-03-09T08:57:26.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/36' 2024-03-09T08:57:26.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/20' 2024-03-09T08:57:26.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/39' 2024-03-09T08:57:26.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/34' 2024-03-09T08:57:26.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/38' 2024-03-09T08:57:26.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/28' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/24' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/29' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/32' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/22' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/25' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/30' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/33' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/23' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/35' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/26' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './231/37' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './231' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/21' 2024-03-09T08:57:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/27' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/31' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/36' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/20' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/39' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/34' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/38' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/28' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/24' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/29' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/32' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/22' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/25' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/30' 2024-03-09T08:57:26.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/33' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/23' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/35' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/26' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './232/37' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './232' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/21' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/27' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/31' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/36' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/20' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/39' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/34' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/38' 2024-03-09T08:57:26.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/28' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/24' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/29' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/32' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/22' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/25' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/30' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/33' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/23' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/35' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/26' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './233/37' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './233' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/21' 2024-03-09T08:57:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/27' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/31' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/36' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/20' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/39' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/34' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/38' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/28' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/24' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/29' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/32' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/22' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/25' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/30' 2024-03-09T08:57:26.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/33' 2024-03-09T08:57:26.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/23' 2024-03-09T08:57:26.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/35' 2024-03-09T08:57:26.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/26' 2024-03-09T08:57:26.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed './234/37' 2024-03-09T08:57:26.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './234' 2024-03-09T08:57:26.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/21' 2024-03-09T08:57:26.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/27' 2024-03-09T08:57:26.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/31' 2024-03-09T08:57:26.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/36' 2024-03-09T08:57:26.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/20' 2024-03-09T08:57:26.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/39' 2024-03-09T08:57:26.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/34' 2024-03-09T08:57:26.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/38' 2024-03-09T08:57:26.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/28' 2024-03-09T08:57:26.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/24' 2024-03-09T08:57:26.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/29' 2024-03-09T08:57:26.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/32' 2024-03-09T08:57:26.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/22' 2024-03-09T08:57:26.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/25' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/30' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/33' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/23' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/35' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/26' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed './235/37' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './235' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/21' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/27' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/31' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/36' 2024-03-09T08:57:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/20' 2024-03-09T08:57:26.442 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/39' 2024-03-09T08:57:26.442 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/34' 2024-03-09T08:57:26.442 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/38' 2024-03-09T08:57:26.442 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/28' 2024-03-09T08:57:26.442 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/24' 2024-03-09T08:57:26.442 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/29' 2024-03-09T08:57:26.442 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/32' 2024-03-09T08:57:27.842 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/22' 2024-03-09T08:57:27.842 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/25' 2024-03-09T08:57:27.843 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/30' 2024-03-09T08:57:27.843 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/33' 2024-03-09T08:57:27.843 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/23' 2024-03-09T08:57:27.843 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/35' 2024-03-09T08:57:27.843 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/26' 2024-03-09T08:57:27.843 INFO:tasks.workunit.client.0.smithi067.stdout:removed './236/37' 2024-03-09T08:57:27.843 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './236' 2024-03-09T08:57:27.843 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/21' 2024-03-09T08:57:27.843 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/27' 2024-03-09T08:57:27.844 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/31' 2024-03-09T08:57:27.844 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/36' 2024-03-09T08:57:27.844 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/20' 2024-03-09T08:57:27.844 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/39' 2024-03-09T08:57:27.844 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/34' 2024-03-09T08:57:27.844 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/38' 2024-03-09T08:57:27.844 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/28' 2024-03-09T08:57:27.844 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/24' 2024-03-09T08:57:27.845 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/29' 2024-03-09T08:57:27.845 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/32' 2024-03-09T08:57:27.845 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/22' 2024-03-09T08:57:27.845 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/25' 2024-03-09T08:57:27.845 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/30' 2024-03-09T08:57:27.845 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/33' 2024-03-09T08:57:27.845 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/23' 2024-03-09T08:57:27.845 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/35' 2024-03-09T08:57:27.845 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/26' 2024-03-09T08:57:27.845 INFO:tasks.workunit.client.0.smithi067.stdout:removed './237/37' 2024-03-09T08:57:27.846 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './237' 2024-03-09T08:57:27.846 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/21' 2024-03-09T08:57:27.846 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/27' 2024-03-09T08:57:27.846 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/31' 2024-03-09T08:57:27.846 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/36' 2024-03-09T08:57:27.846 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/20' 2024-03-09T08:57:27.847 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/39' 2024-03-09T08:57:27.847 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/34' 2024-03-09T08:57:27.847 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/38' 2024-03-09T08:57:27.847 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/28' 2024-03-09T08:57:27.847 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/24' 2024-03-09T08:57:27.848 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/29' 2024-03-09T08:57:27.848 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/32' 2024-03-09T08:57:27.848 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/22' 2024-03-09T08:57:27.848 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/25' 2024-03-09T08:57:27.848 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/30' 2024-03-09T08:57:27.848 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/33' 2024-03-09T08:57:27.848 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/23' 2024-03-09T08:57:27.849 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/35' 2024-03-09T08:57:27.849 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/26' 2024-03-09T08:57:27.849 INFO:tasks.workunit.client.0.smithi067.stdout:removed './238/37' 2024-03-09T08:57:27.849 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './238' 2024-03-09T08:57:27.849 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/21' 2024-03-09T08:57:27.849 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/27' 2024-03-09T08:57:27.849 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/31' 2024-03-09T08:57:27.849 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/36' 2024-03-09T08:57:27.849 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/20' 2024-03-09T08:57:27.849 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/39' 2024-03-09T08:57:27.849 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/34' 2024-03-09T08:57:27.850 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/38' 2024-03-09T08:57:27.850 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/28' 2024-03-09T08:57:27.850 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/24' 2024-03-09T08:57:27.850 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/29' 2024-03-09T08:57:27.850 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/32' 2024-03-09T08:57:27.850 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/22' 2024-03-09T08:57:27.850 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/25' 2024-03-09T08:57:27.850 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/30' 2024-03-09T08:57:27.850 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/33' 2024-03-09T08:57:27.850 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/23' 2024-03-09T08:57:27.851 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/35' 2024-03-09T08:57:27.851 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/26' 2024-03-09T08:57:27.851 INFO:tasks.workunit.client.0.smithi067.stdout:removed './239/37' 2024-03-09T08:57:27.851 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './239' 2024-03-09T08:57:27.851 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/21' 2024-03-09T08:57:27.851 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/27' 2024-03-09T08:57:27.851 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/31' 2024-03-09T08:57:27.851 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/36' 2024-03-09T08:57:27.851 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/20' 2024-03-09T08:57:27.851 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/39' 2024-03-09T08:57:27.852 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/34' 2024-03-09T08:57:27.852 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/38' 2024-03-09T08:57:27.852 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/28' 2024-03-09T08:57:27.852 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/24' 2024-03-09T08:57:27.852 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/29' 2024-03-09T08:57:27.852 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/32' 2024-03-09T08:57:27.852 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/22' 2024-03-09T08:57:27.852 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/25' 2024-03-09T08:57:27.852 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/30' 2024-03-09T08:57:27.853 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/33' 2024-03-09T08:57:27.853 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/23' 2024-03-09T08:57:27.853 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/35' 2024-03-09T08:57:27.853 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/26' 2024-03-09T08:57:27.853 INFO:tasks.workunit.client.0.smithi067.stdout:removed './240/37' 2024-03-09T08:57:27.853 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './240' 2024-03-09T08:57:27.853 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/21' 2024-03-09T08:57:27.854 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/27' 2024-03-09T08:57:27.854 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/31' 2024-03-09T08:57:27.854 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/36' 2024-03-09T08:57:27.854 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/20' 2024-03-09T08:57:27.854 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/39' 2024-03-09T08:57:27.854 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/34' 2024-03-09T08:57:27.855 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/38' 2024-03-09T08:57:27.855 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/28' 2024-03-09T08:57:27.855 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/24' 2024-03-09T08:57:27.855 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/29' 2024-03-09T08:57:27.855 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/32' 2024-03-09T08:57:27.855 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/22' 2024-03-09T08:57:27.855 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/25' 2024-03-09T08:57:27.855 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/30' 2024-03-09T08:57:27.856 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/33' 2024-03-09T08:57:27.856 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/23' 2024-03-09T08:57:27.856 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/35' 2024-03-09T08:57:27.856 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/26' 2024-03-09T08:57:27.856 INFO:tasks.workunit.client.0.smithi067.stdout:removed './241/37' 2024-03-09T08:57:27.856 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './241' 2024-03-09T08:57:27.856 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/21' 2024-03-09T08:57:27.857 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/27' 2024-03-09T08:57:27.857 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/31' 2024-03-09T08:57:27.857 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/36' 2024-03-09T08:57:27.857 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/20' 2024-03-09T08:57:27.857 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/39' 2024-03-09T08:57:27.857 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/34' 2024-03-09T08:57:27.857 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/38' 2024-03-09T08:57:27.858 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/28' 2024-03-09T08:57:27.858 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/24' 2024-03-09T08:57:27.858 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/29' 2024-03-09T08:57:27.858 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/32' 2024-03-09T08:57:27.858 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/22' 2024-03-09T08:57:27.858 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/25' 2024-03-09T08:57:27.859 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/30' 2024-03-09T08:57:27.859 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/33' 2024-03-09T08:57:27.859 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/23' 2024-03-09T08:57:27.859 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/35' 2024-03-09T08:57:27.859 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/26' 2024-03-09T08:57:27.859 INFO:tasks.workunit.client.0.smithi067.stdout:removed './242/37' 2024-03-09T08:57:27.860 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './242' 2024-03-09T08:57:27.860 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/21' 2024-03-09T08:57:27.860 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/27' 2024-03-09T08:57:27.860 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/31' 2024-03-09T08:57:27.860 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/36' 2024-03-09T08:57:27.860 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/20' 2024-03-09T08:57:27.861 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/39' 2024-03-09T08:57:27.861 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/34' 2024-03-09T08:57:27.861 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/38' 2024-03-09T08:57:27.861 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/28' 2024-03-09T08:57:27.862 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/24' 2024-03-09T08:57:27.862 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/29' 2024-03-09T08:57:27.862 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/32' 2024-03-09T08:57:27.862 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/22' 2024-03-09T08:57:27.862 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/25' 2024-03-09T08:57:27.863 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/30' 2024-03-09T08:57:27.863 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/33' 2024-03-09T08:57:27.863 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/23' 2024-03-09T08:57:27.863 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/35' 2024-03-09T08:57:27.863 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/26' 2024-03-09T08:57:27.863 INFO:tasks.workunit.client.0.smithi067.stdout:removed './243/37' 2024-03-09T08:57:27.864 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './243' 2024-03-09T08:57:27.864 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/21' 2024-03-09T08:57:27.864 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/27' 2024-03-09T08:57:27.864 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/31' 2024-03-09T08:57:27.864 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/36' 2024-03-09T08:57:27.865 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/20' 2024-03-09T08:57:27.865 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/39' 2024-03-09T08:57:27.865 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/34' 2024-03-09T08:57:27.865 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/38' 2024-03-09T08:57:27.865 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/28' 2024-03-09T08:57:27.865 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/24' 2024-03-09T08:57:27.865 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/29' 2024-03-09T08:57:27.866 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/32' 2024-03-09T08:57:27.866 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/22' 2024-03-09T08:57:27.866 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/25' 2024-03-09T08:57:27.866 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/30' 2024-03-09T08:57:27.866 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/33' 2024-03-09T08:57:27.867 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/23' 2024-03-09T08:57:27.867 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/35' 2024-03-09T08:57:27.867 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/26' 2024-03-09T08:57:27.867 INFO:tasks.workunit.client.0.smithi067.stdout:removed './244/37' 2024-03-09T08:57:27.867 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './244' 2024-03-09T08:57:27.867 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/21' 2024-03-09T08:57:27.867 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/27' 2024-03-09T08:57:27.868 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/31' 2024-03-09T08:57:27.868 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/36' 2024-03-09T08:57:27.868 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/20' 2024-03-09T08:57:27.868 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/39' 2024-03-09T08:57:27.868 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/34' 2024-03-09T08:57:27.868 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/38' 2024-03-09T08:57:27.868 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/28' 2024-03-09T08:57:27.869 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/24' 2024-03-09T08:57:27.869 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/29' 2024-03-09T08:57:27.869 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/32' 2024-03-09T08:57:27.869 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/22' 2024-03-09T08:57:27.869 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/25' 2024-03-09T08:57:27.869 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/30' 2024-03-09T08:57:27.869 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/33' 2024-03-09T08:57:27.870 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/23' 2024-03-09T08:57:27.870 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/35' 2024-03-09T08:57:27.870 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/26' 2024-03-09T08:57:27.870 INFO:tasks.workunit.client.0.smithi067.stdout:removed './245/37' 2024-03-09T08:57:27.870 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './245' 2024-03-09T08:57:27.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/21' 2024-03-09T08:57:27.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/27' 2024-03-09T08:57:27.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/31' 2024-03-09T08:57:27.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/36' 2024-03-09T08:57:27.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/20' 2024-03-09T08:57:27.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/39' 2024-03-09T08:57:27.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/34' 2024-03-09T08:57:27.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/38' 2024-03-09T08:57:27.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/28' 2024-03-09T08:57:27.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/24' 2024-03-09T08:57:27.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/29' 2024-03-09T08:57:27.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/32' 2024-03-09T08:57:27.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/22' 2024-03-09T08:57:28.289 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/22' 2024-03-09T08:57:28.289 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/25' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/30' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/33' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/23' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/35' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/26' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './236/37' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './236' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/21' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/27' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/31' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/36' 2024-03-09T08:57:28.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/20' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/39' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/34' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/38' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/28' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/24' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/29' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/32' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/22' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/25' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/30' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/33' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/23' 2024-03-09T08:57:28.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/35' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/26' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './237/37' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './237' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/21' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/27' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/31' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/36' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/20' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/39' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/34' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/38' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/28' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/24' 2024-03-09T08:57:28.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/29' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/32' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/22' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/25' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/30' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/33' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/23' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/35' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/26' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './238/37' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './238' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/21' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/27' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/31' 2024-03-09T08:57:28.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/36' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/20' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/39' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/34' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/38' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/28' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/24' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/29' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/32' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/22' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/25' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/30' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/33' 2024-03-09T08:57:28.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/23' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/35' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/26' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './239/37' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './239' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/21' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/27' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/31' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/36' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/20' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/39' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/34' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/38' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/28' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/24' 2024-03-09T08:57:28.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/29' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/32' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/22' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/25' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/30' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/33' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/23' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/35' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/26' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './240/37' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './240' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/21' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/27' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/31' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/36' 2024-03-09T08:57:28.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/20' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/39' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/34' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/38' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/28' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/24' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/29' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/32' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/22' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/25' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/30' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/33' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/23' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/35' 2024-03-09T08:57:28.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/26' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './241/37' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './241' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/21' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/27' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/31' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/36' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/20' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/39' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/34' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/38' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/28' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/24' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/29' 2024-03-09T08:57:28.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/32' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/22' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/25' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/30' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/33' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/23' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/35' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/26' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './242/37' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './242' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/21' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/27' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/31' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/36' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/20' 2024-03-09T08:57:28.299 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/39' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/34' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/38' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/28' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/24' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/29' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/32' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/22' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/25' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/30' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/33' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/23' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/35' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/26' 2024-03-09T08:57:28.300 INFO:tasks.workunit.client.1.smithi134.stdout:removed './243/37' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './243' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/21' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/27' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/31' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/36' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/20' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/39' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/34' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/38' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/28' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/24' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/29' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/32' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/22' 2024-03-09T08:57:28.301 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/25' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/30' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/33' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/23' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/35' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/26' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './244/37' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './244' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/21' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/27' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/31' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/36' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/20' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/39' 2024-03-09T08:57:28.302 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/34' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/38' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/28' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/24' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/29' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/32' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/22' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/25' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/30' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/33' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/23' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/35' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/26' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './245/37' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './245' 2024-03-09T08:57:28.303 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/21' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/27' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/31' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/36' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/20' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/39' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/34' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/38' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/28' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/24' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/29' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/32' 2024-03-09T08:57:28.304 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/22' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/25' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/30' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/33' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/23' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/35' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/26' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './246/37' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './246' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/21' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/27' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/31' 2024-03-09T08:57:28.871 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/36' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/20' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/39' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/34' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/38' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/28' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/24' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/29' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/32' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/22' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/25' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/30' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/33' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/23' 2024-03-09T08:57:28.872 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/35' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/26' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './247/37' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './247' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/21' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/27' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/31' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/36' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/20' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/39' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/34' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/38' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/28' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/24' 2024-03-09T08:57:28.873 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/29' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/32' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/22' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/25' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/30' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/33' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/23' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/35' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/26' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './248/37' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './248' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/21' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/27' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/31' 2024-03-09T08:57:28.874 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/36' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/20' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/39' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/34' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/38' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/28' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/24' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/29' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/32' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/22' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/25' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/30' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/33' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/23' 2024-03-09T08:57:28.875 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/35' 2024-03-09T08:57:28.876 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/26' 2024-03-09T08:57:28.876 INFO:tasks.workunit.client.0.smithi067.stdout:removed './249/37' 2024-03-09T08:57:28.876 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './249' 2024-03-09T08:57:28.876 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/21' 2024-03-09T08:57:28.876 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/27' 2024-03-09T08:57:28.876 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/31' 2024-03-09T08:57:28.876 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/36' 2024-03-09T08:57:28.876 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/20' 2024-03-09T08:57:28.876 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/39' 2024-03-09T08:57:28.876 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/34' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/38' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/28' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/24' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/29' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/32' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/22' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/25' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/30' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/33' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/23' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/35' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/26' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed './250/37' 2024-03-09T08:57:28.877 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './250' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/21' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/27' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/31' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/36' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/20' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/39' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/34' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/38' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/28' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/24' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/29' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/32' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/22' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/25' 2024-03-09T08:57:28.878 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/30' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/33' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/23' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/35' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/26' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './251/37' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './251' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/21' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/27' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/31' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/36' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/20' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/39' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/34' 2024-03-09T08:57:28.879 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/38' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/28' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/24' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/29' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/32' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/22' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/25' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/30' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/33' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/23' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/35' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/26' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './252/37' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './252' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/21' 2024-03-09T08:57:28.880 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/27' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/31' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/36' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/20' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/39' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/34' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/38' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/28' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/24' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/29' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/32' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/22' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/25' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/30' 2024-03-09T08:57:28.881 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/33' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/23' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/35' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/26' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './253/37' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './253' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/21' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/27' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/31' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/36' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/20' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/39' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/34' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/38' 2024-03-09T08:57:28.882 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/28' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/24' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/29' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/32' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/22' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/25' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/30' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/33' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/23' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/35' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/26' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './254/37' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './254' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/21' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/27' 2024-03-09T08:57:28.883 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/31' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/36' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/20' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/39' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/34' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/38' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/28' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/24' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/29' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/32' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/22' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/25' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/30' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/33' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/23' 2024-03-09T08:57:28.884 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/35' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/26' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './255/37' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './255' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/21' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/27' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/31' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/36' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/20' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/39' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/34' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/38' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/28' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/24' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/29' 2024-03-09T08:57:28.885 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/32' 2024-03-09T08:57:28.886 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/22' 2024-03-09T08:57:28.886 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/25' 2024-03-09T08:57:28.886 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/30' 2024-03-09T08:57:29.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/25' 2024-03-09T08:57:29.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/30' 2024-03-09T08:57:29.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/33' 2024-03-09T08:57:29.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/23' 2024-03-09T08:57:29.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/35' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/26' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './246/37' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './246' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/21' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/27' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/31' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/36' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/20' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/39' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/34' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/38' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/28' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/24' 2024-03-09T08:57:29.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/29' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/32' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/22' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/25' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/30' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/33' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/23' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/35' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/26' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './247/37' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './247' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/21' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/27' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/31' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/36' 2024-03-09T08:57:29.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/20' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/39' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/34' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/38' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/28' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/24' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/29' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/32' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/22' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/25' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/30' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/33' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/23' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/35' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/26' 2024-03-09T08:57:29.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './248/37' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './248' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/21' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/27' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/31' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/36' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/20' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/39' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/34' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/38' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/28' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/24' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/29' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/32' 2024-03-09T08:57:29.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/22' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/25' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/30' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/33' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/23' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/35' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/26' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './249/37' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './249' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/21' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/27' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/31' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/36' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/20' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/39' 2024-03-09T08:57:29.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/34' 2024-03-09T08:57:29.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/38' 2024-03-09T08:57:29.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/28' 2024-03-09T08:57:29.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/24' 2024-03-09T08:57:29.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/29' 2024-03-09T08:57:29.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/32' 2024-03-09T08:57:29.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/22' 2024-03-09T08:57:29.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/25' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/30' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/33' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/23' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/35' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/26' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './250/37' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './250' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/21' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/27' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/31' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/36' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/20' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/39' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/34' 2024-03-09T08:57:29.917 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/38' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/28' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/24' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/29' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/32' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/22' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/25' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/30' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/33' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/23' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/35' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/26' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './251/37' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './251' 2024-03-09T08:57:29.918 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/21' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/27' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/31' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/36' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/20' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/39' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/34' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/38' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/28' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/24' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/29' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/32' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/22' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/25' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/30' 2024-03-09T08:57:29.919 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/33' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/23' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/35' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/26' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './252/37' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './252' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/21' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/27' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/31' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/36' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/20' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/39' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/34' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/38' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/28' 2024-03-09T08:57:29.920 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/24' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/29' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/32' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/22' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/25' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/30' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/33' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/23' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/35' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/26' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './253/37' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './253' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/21' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/27' 2024-03-09T08:57:29.921 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/31' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/36' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/20' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/39' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/34' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/38' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/28' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/24' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/29' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/32' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/22' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/25' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/30' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/33' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/23' 2024-03-09T08:57:29.922 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/35' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/26' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './254/37' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './254' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/21' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/27' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/31' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/36' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/20' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/39' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/34' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/38' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/28' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/24' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/29' 2024-03-09T08:57:29.923 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/32' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/22' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/25' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/30' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/33' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/23' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/35' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/26' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './255/37' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './255' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/21' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/27' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/31' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/36' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/20' 2024-03-09T08:57:29.924 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/39' 2024-03-09T08:57:29.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/34' 2024-03-09T08:57:29.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/38' 2024-03-09T08:57:29.925 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/28' 2024-03-09T08:57:29.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/24' 2024-03-09T08:57:29.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/29' 2024-03-09T08:57:29.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/32' 2024-03-09T08:57:29.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/22' 2024-03-09T08:57:29.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/25' 2024-03-09T08:57:29.926 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/30' 2024-03-09T08:57:30.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/33' 2024-03-09T08:57:30.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/23' 2024-03-09T08:57:30.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/35' 2024-03-09T08:57:30.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/26' 2024-03-09T08:57:30.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './256/37' 2024-03-09T08:57:30.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './256' 2024-03-09T08:57:30.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/21' 2024-03-09T08:57:30.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/27' 2024-03-09T08:57:30.080 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/31' 2024-03-09T08:57:30.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/36' 2024-03-09T08:57:30.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/20' 2024-03-09T08:57:30.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/39' 2024-03-09T08:57:30.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/34' 2024-03-09T08:57:30.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/38' 2024-03-09T08:57:30.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/28' 2024-03-09T08:57:30.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/24' 2024-03-09T08:57:30.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/29' 2024-03-09T08:57:30.081 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/32' 2024-03-09T08:57:30.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/22' 2024-03-09T08:57:30.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/25' 2024-03-09T08:57:30.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/30' 2024-03-09T08:57:30.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/33' 2024-03-09T08:57:30.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/23' 2024-03-09T08:57:30.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/35' 2024-03-09T08:57:30.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/26' 2024-03-09T08:57:30.082 INFO:tasks.workunit.client.0.smithi067.stdout:removed './257/37' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './257' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/21' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/27' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/31' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/36' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/20' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/39' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/34' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/38' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/28' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/24' 2024-03-09T08:57:30.083 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/29' 2024-03-09T08:57:30.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/32' 2024-03-09T08:57:30.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/22' 2024-03-09T08:57:30.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/25' 2024-03-09T08:57:30.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/30' 2024-03-09T08:57:30.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/33' 2024-03-09T08:57:30.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/23' 2024-03-09T08:57:30.084 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/35' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/26' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './258/37' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './258' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/21' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/27' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/31' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/36' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/20' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/39' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/34' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/38' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/28' 2024-03-09T08:57:30.085 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/24' 2024-03-09T08:57:30.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/29' 2024-03-09T08:57:30.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/32' 2024-03-09T08:57:30.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/22' 2024-03-09T08:57:30.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/25' 2024-03-09T08:57:30.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/30' 2024-03-09T08:57:30.086 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/33' 2024-03-09T08:57:30.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/23' 2024-03-09T08:57:30.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/35' 2024-03-09T08:57:30.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/26' 2024-03-09T08:57:30.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './259/37' 2024-03-09T08:57:30.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './259' 2024-03-09T08:57:30.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/21' 2024-03-09T08:57:30.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/27' 2024-03-09T08:57:30.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/31' 2024-03-09T08:57:30.087 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/36' 2024-03-09T08:57:30.088 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/20' 2024-03-09T08:57:30.088 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/39' 2024-03-09T08:57:30.088 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/34' 2024-03-09T08:57:30.088 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/38' 2024-03-09T08:57:30.088 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/28' 2024-03-09T08:57:30.088 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/24' 2024-03-09T08:57:30.088 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/29' 2024-03-09T08:57:30.088 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/32' 2024-03-09T08:57:30.088 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/22' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/25' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/30' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/33' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/23' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/35' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/26' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './260/37' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './260' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/21' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/27' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/31' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/36' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/20' 2024-03-09T08:57:30.089 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/39' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/34' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/38' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/28' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/24' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/29' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/32' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/22' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/25' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/30' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/33' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/23' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/35' 2024-03-09T08:57:30.090 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/26' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './261/37' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './261' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/21' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/27' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/31' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/36' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/20' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/39' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/34' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/38' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/28' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/24' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/29' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/32' 2024-03-09T08:57:30.091 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/22' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/25' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/30' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/33' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/23' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/35' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/26' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './262/37' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './262' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/21' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/27' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/31' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/36' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/20' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/39' 2024-03-09T08:57:30.092 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/34' 2024-03-09T08:57:30.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/38' 2024-03-09T08:57:30.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/28' 2024-03-09T08:57:30.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/24' 2024-03-09T08:57:30.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/29' 2024-03-09T08:57:30.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/32' 2024-03-09T08:57:30.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/22' 2024-03-09T08:57:30.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/25' 2024-03-09T08:57:30.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/30' 2024-03-09T08:57:30.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/33' 2024-03-09T08:57:30.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/23' 2024-03-09T08:57:30.093 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/35' 2024-03-09T08:57:30.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/26' 2024-03-09T08:57:30.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './263/37' 2024-03-09T08:57:30.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './263' 2024-03-09T08:57:30.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/21' 2024-03-09T08:57:30.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/27' 2024-03-09T08:57:30.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/31' 2024-03-09T08:57:30.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/36' 2024-03-09T08:57:30.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/20' 2024-03-09T08:57:30.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/39' 2024-03-09T08:57:30.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/34' 2024-03-09T08:57:30.094 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/38' 2024-03-09T08:57:30.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/28' 2024-03-09T08:57:30.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/24' 2024-03-09T08:57:30.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/29' 2024-03-09T08:57:30.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/32' 2024-03-09T08:57:30.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/22' 2024-03-09T08:57:30.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/25' 2024-03-09T08:57:30.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/30' 2024-03-09T08:57:30.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/33' 2024-03-09T08:57:30.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/23' 2024-03-09T08:57:30.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/35' 2024-03-09T08:57:30.095 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/26' 2024-03-09T08:57:30.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './264/37' 2024-03-09T08:57:30.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './264' 2024-03-09T08:57:30.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/21' 2024-03-09T08:57:30.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/27' 2024-03-09T08:57:30.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/31' 2024-03-09T08:57:30.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/36' 2024-03-09T08:57:30.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/20' 2024-03-09T08:57:30.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/39' 2024-03-09T08:57:30.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/34' 2024-03-09T08:57:30.096 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/38' 2024-03-09T08:57:30.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/28' 2024-03-09T08:57:30.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/24' 2024-03-09T08:57:30.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/29' 2024-03-09T08:57:30.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/32' 2024-03-09T08:57:30.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/22' 2024-03-09T08:57:30.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/25' 2024-03-09T08:57:30.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/30' 2024-03-09T08:57:30.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/33' 2024-03-09T08:57:30.097 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/23' 2024-03-09T08:57:30.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/35' 2024-03-09T08:57:30.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/26' 2024-03-09T08:57:30.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './265/37' 2024-03-09T08:57:30.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './265' 2024-03-09T08:57:30.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/21' 2024-03-09T08:57:30.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/27' 2024-03-09T08:57:30.098 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/31' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/36' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/20' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/39' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/34' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/38' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/28' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/24' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/29' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/32' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/22' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/25' 2024-03-09T08:57:30.099 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/30' 2024-03-09T08:57:30.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/33' 2024-03-09T08:57:30.100 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/23' 2024-03-09T08:57:31.615 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/35' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/26' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './266/37' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './266' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/21' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/27' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/31' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/36' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/20' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/39' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/34' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/38' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/28' 2024-03-09T08:57:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/24' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/29' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/32' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/22' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/25' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/30' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/33' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/23' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/35' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/26' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './267/37' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './267' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/21' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/27' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/31' 2024-03-09T08:57:31.617 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/36' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/20' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/39' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/34' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/38' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/28' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/24' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/29' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/32' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/22' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/25' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/30' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/33' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/23' 2024-03-09T08:57:31.618 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/35' 2024-03-09T08:57:31.619 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/26' 2024-03-09T08:57:31.619 INFO:tasks.workunit.client.0.smithi067.stdout:removed './268/37' 2024-03-09T08:57:31.619 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './268' 2024-03-09T08:57:31.619 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/21' 2024-03-09T08:57:31.620 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/27' 2024-03-09T08:57:31.620 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/31' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/36' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/20' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/39' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/34' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/38' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/28' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/24' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/29' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/32' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/22' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/25' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/30' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/33' 2024-03-09T08:57:31.621 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/23' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/35' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/26' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './269/37' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './269' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/21' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/27' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/31' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/36' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/20' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/39' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/34' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/38' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/28' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/24' 2024-03-09T08:57:31.622 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/29' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/32' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/22' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/25' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/30' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/33' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/23' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/35' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/26' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './270/37' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './270' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/21' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/27' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/31' 2024-03-09T08:57:31.623 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/36' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/20' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/39' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/34' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/38' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/28' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/24' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/29' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/32' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/22' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/25' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/30' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/33' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/23' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/35' 2024-03-09T08:57:31.624 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/26' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './271/37' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './271' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/21' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/27' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/31' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/36' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/20' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/39' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/34' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/38' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/28' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/24' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/29' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/32' 2024-03-09T08:57:31.625 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/22' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/25' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/30' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/33' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/23' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/35' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/26' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './272/37' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './272' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/21' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/27' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/31' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/36' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/20' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/39' 2024-03-09T08:57:31.626 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/34' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/38' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/28' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/24' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/29' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/32' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/22' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/25' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/30' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/33' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/23' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/35' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/26' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed './273/37' 2024-03-09T08:57:31.627 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './273' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/21' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/27' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/31' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/36' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/20' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/39' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/34' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/38' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/28' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/24' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/29' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/32' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/22' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/25' 2024-03-09T08:57:31.628 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/30' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/33' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/23' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/35' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/26' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './274/37' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './274' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/21' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/27' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/31' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/36' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/20' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/39' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/34' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/38' 2024-03-09T08:57:31.629 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/28' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/24' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/29' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/32' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/22' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/25' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/30' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/33' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/23' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/35' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/26' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './275/37' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './275' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/21' 2024-03-09T08:57:31.630 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/27' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/31' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/36' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/20' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/39' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/34' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/38' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/28' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/24' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/29' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/32' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/22' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/25' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/30' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/33' 2024-03-09T08:57:31.631 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/23' 2024-03-09T08:57:31.632 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/35' 2024-03-09T08:57:31.632 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/26' 2024-03-09T08:57:31.898 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/33' 2024-03-09T08:57:31.898 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/23' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/35' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/26' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './256/37' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './256' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/21' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/27' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/31' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/36' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/20' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/39' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/34' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/38' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/28' 2024-03-09T08:57:31.899 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/24' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/29' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/32' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/22' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/25' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/30' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/33' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/23' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/35' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/26' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './257/37' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './257' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/21' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/27' 2024-03-09T08:57:31.900 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/31' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/36' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/20' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/39' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/34' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/38' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/28' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/24' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/29' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/32' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/22' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/25' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/30' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/33' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/23' 2024-03-09T08:57:31.901 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/35' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/26' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed './258/37' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './258' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/21' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/27' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/31' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/36' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/20' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/39' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/34' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/38' 2024-03-09T08:57:31.902 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/28' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/24' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/29' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/32' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/22' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/25' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/30' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/33' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/23' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/35' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/26' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './259/37' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './259' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/21' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/27' 2024-03-09T08:57:31.903 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/31' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/36' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/20' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/39' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/34' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/38' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/28' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/24' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/29' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/32' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/22' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/25' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/30' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/33' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/23' 2024-03-09T08:57:31.904 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/35' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/26' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './260/37' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './260' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/21' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/27' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/31' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/36' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/20' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/39' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/34' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/38' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/28' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/24' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/29' 2024-03-09T08:57:31.905 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/32' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/22' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/25' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/30' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/33' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/23' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/35' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/26' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './261/37' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './261' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/21' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/27' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/31' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/36' 2024-03-09T08:57:31.906 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/20' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/39' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/34' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/38' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/28' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/24' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/29' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/32' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/22' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/25' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/30' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/33' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/23' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/35' 2024-03-09T08:57:31.907 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/26' 2024-03-09T08:57:31.908 INFO:tasks.workunit.client.1.smithi134.stdout:removed './262/37' 2024-03-09T08:57:31.908 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './262' 2024-03-09T08:57:31.908 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/21' 2024-03-09T08:57:31.908 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/27' 2024-03-09T08:57:31.908 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/31' 2024-03-09T08:57:31.908 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/36' 2024-03-09T08:57:31.908 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/20' 2024-03-09T08:57:31.908 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/39' 2024-03-09T08:57:31.908 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/34' 2024-03-09T08:57:31.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/38' 2024-03-09T08:57:31.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/28' 2024-03-09T08:57:31.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/24' 2024-03-09T08:57:31.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/29' 2024-03-09T08:57:31.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/32' 2024-03-09T08:57:31.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/22' 2024-03-09T08:57:31.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/25' 2024-03-09T08:57:31.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/30' 2024-03-09T08:57:31.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/33' 2024-03-09T08:57:31.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/23' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/35' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/26' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './263/37' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './263' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/21' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/27' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/31' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/36' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/20' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/39' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/34' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/38' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/28' 2024-03-09T08:57:31.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/24' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/29' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/32' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/22' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/25' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/30' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/33' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/23' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/35' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/26' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './264/37' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './264' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/21' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/27' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/31' 2024-03-09T08:57:31.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/36' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/20' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/39' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/34' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/38' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/28' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/24' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/29' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/32' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/22' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/25' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/30' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/33' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/23' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/35' 2024-03-09T08:57:31.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/26' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './265/37' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './265' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/21' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/27' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/31' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/36' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/20' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/39' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/34' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/38' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/28' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/24' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/29' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/32' 2024-03-09T08:57:31.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/22' 2024-03-09T08:57:31.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/25' 2024-03-09T08:57:31.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/30' 2024-03-09T08:57:31.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/33' 2024-03-09T08:57:31.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/23' 2024-03-09T08:57:33.822 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/35' 2024-03-09T08:57:33.822 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/26' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './266/37' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './266' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/21' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/27' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/31' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/36' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/20' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/39' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/34' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/38' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/28' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/24' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/29' 2024-03-09T08:57:33.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/32' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/22' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/25' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/30' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/33' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/23' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/35' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/26' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './267/37' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './267' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/21' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/27' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/31' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/36' 2024-03-09T08:57:33.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/20' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/39' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/34' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/38' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/28' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/24' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/29' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/32' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/22' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/25' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/30' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/33' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/23' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/35' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/26' 2024-03-09T08:57:33.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './268/37' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './268' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/21' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/27' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/31' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/36' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/20' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/39' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/34' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/38' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/28' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/24' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/29' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/32' 2024-03-09T08:57:33.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/22' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/25' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/30' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/33' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/23' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/35' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/26' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './269/37' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './269' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/21' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/27' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/31' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/36' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/20' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/39' 2024-03-09T08:57:33.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/34' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/38' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/28' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/24' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/29' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/32' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/22' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/25' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/30' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/33' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/23' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/35' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/26' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './270/37' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './270' 2024-03-09T08:57:33.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/21' 2024-03-09T08:57:33.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/27' 2024-03-09T08:57:33.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/31' 2024-03-09T08:57:33.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/36' 2024-03-09T08:57:33.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/20' 2024-03-09T08:57:33.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/39' 2024-03-09T08:57:33.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/34' 2024-03-09T08:57:33.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/38' 2024-03-09T08:57:33.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/28' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/24' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/29' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/32' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/22' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/25' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/30' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/33' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/23' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/35' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/26' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './271/37' 2024-03-09T08:57:33.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './271' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/21' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/27' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/31' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/36' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/20' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/39' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/34' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/38' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/28' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/24' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/29' 2024-03-09T08:57:33.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/32' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/22' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/25' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/30' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/33' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/23' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/35' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/26' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './272/37' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './272' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/21' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/27' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/31' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/36' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/20' 2024-03-09T08:57:33.832 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/39' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/34' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/38' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/28' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/24' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/29' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/32' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/22' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/25' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/30' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/33' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/23' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/35' 2024-03-09T08:57:33.833 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/26' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './273/37' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './273' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/21' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/27' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/31' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/36' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/20' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/39' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/34' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/38' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/28' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/24' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/29' 2024-03-09T08:57:33.834 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/32' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/22' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/25' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/30' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/33' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/23' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/35' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/26' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed './274/37' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './274' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/21' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/27' 2024-03-09T08:57:33.835 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/31' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/36' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/20' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/39' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/34' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/38' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/28' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/24' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/29' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/32' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/22' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/25' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/30' 2024-03-09T08:57:33.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/33' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/23' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/35' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/26' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './275/37' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './275' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/21' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/27' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/31' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/36' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/20' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/39' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/34' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/38' 2024-03-09T08:57:33.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/28' 2024-03-09T08:57:33.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/24' 2024-03-09T08:57:33.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/29' 2024-03-09T08:57:33.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/32' 2024-03-09T08:57:33.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/22' 2024-03-09T08:57:33.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/25' 2024-03-09T08:57:33.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/30' 2024-03-09T08:57:33.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/33' 2024-03-09T08:57:33.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/23' 2024-03-09T08:57:33.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/35' 2024-03-09T08:57:33.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/26' 2024-03-09T08:57:34.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './276/37' 2024-03-09T08:57:34.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './276' 2024-03-09T08:57:34.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/21' 2024-03-09T08:57:34.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/27' 2024-03-09T08:57:34.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/31' 2024-03-09T08:57:34.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/36' 2024-03-09T08:57:34.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/20' 2024-03-09T08:57:34.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/39' 2024-03-09T08:57:34.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/34' 2024-03-09T08:57:34.114 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/38' 2024-03-09T08:57:34.115 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/28' 2024-03-09T08:57:34.115 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/24' 2024-03-09T08:57:34.115 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/29' 2024-03-09T08:57:34.115 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/32' 2024-03-09T08:57:34.115 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/22' 2024-03-09T08:57:34.115 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/25' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/30' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/33' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/23' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/35' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/26' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './277/37' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './277' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/21' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/27' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/31' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/36' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/20' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/39' 2024-03-09T08:57:34.116 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/34' 2024-03-09T08:57:34.117 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/38' 2024-03-09T08:57:34.117 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/28' 2024-03-09T08:57:34.117 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/24' 2024-03-09T08:57:34.117 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/29' 2024-03-09T08:57:34.117 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/32' 2024-03-09T08:57:34.117 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/22' 2024-03-09T08:57:34.117 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/25' 2024-03-09T08:57:34.117 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/30' 2024-03-09T08:57:34.117 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/33' 2024-03-09T08:57:34.118 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/23' 2024-03-09T08:57:34.118 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/35' 2024-03-09T08:57:34.118 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/26' 2024-03-09T08:57:34.118 INFO:tasks.workunit.client.0.smithi067.stdout:removed './278/37' 2024-03-09T08:57:34.118 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './278' 2024-03-09T08:57:34.118 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/21' 2024-03-09T08:57:34.118 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/27' 2024-03-09T08:57:34.118 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/31' 2024-03-09T08:57:34.118 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/36' 2024-03-09T08:57:34.119 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/20' 2024-03-09T08:57:34.119 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/39' 2024-03-09T08:57:34.119 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/34' 2024-03-09T08:57:34.119 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/38' 2024-03-09T08:57:34.119 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/28' 2024-03-09T08:57:34.119 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/24' 2024-03-09T08:57:34.119 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/29' 2024-03-09T08:57:34.119 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/32' 2024-03-09T08:57:34.119 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/22' 2024-03-09T08:57:34.119 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/25' 2024-03-09T08:57:34.119 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/30' 2024-03-09T08:57:34.120 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/33' 2024-03-09T08:57:34.120 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/23' 2024-03-09T08:57:34.120 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/35' 2024-03-09T08:57:34.120 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/26' 2024-03-09T08:57:34.120 INFO:tasks.workunit.client.0.smithi067.stdout:removed './279/37' 2024-03-09T08:57:34.121 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './279' 2024-03-09T08:57:34.121 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/21' 2024-03-09T08:57:34.121 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/27' 2024-03-09T08:57:34.121 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/31' 2024-03-09T08:57:34.121 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/36' 2024-03-09T08:57:34.121 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/20' 2024-03-09T08:57:34.121 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/39' 2024-03-09T08:57:34.121 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/34' 2024-03-09T08:57:34.122 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/38' 2024-03-09T08:57:34.122 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/28' 2024-03-09T08:57:34.122 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/24' 2024-03-09T08:57:34.123 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/29' 2024-03-09T08:57:34.123 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/32' 2024-03-09T08:57:34.123 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/22' 2024-03-09T08:57:34.123 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/25' 2024-03-09T08:57:34.123 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/30' 2024-03-09T08:57:34.123 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/33' 2024-03-09T08:57:34.123 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/23' 2024-03-09T08:57:34.123 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/35' 2024-03-09T08:57:34.123 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/26' 2024-03-09T08:57:34.124 INFO:tasks.workunit.client.0.smithi067.stdout:removed './280/37' 2024-03-09T08:57:34.124 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './280' 2024-03-09T08:57:34.124 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/21' 2024-03-09T08:57:34.124 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/27' 2024-03-09T08:57:34.124 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/31' 2024-03-09T08:57:34.124 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/36' 2024-03-09T08:57:34.124 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/20' 2024-03-09T08:57:34.124 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/39' 2024-03-09T08:57:34.125 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/34' 2024-03-09T08:57:34.125 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/38' 2024-03-09T08:57:34.125 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/28' 2024-03-09T08:57:34.125 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/24' 2024-03-09T08:57:34.125 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/29' 2024-03-09T08:57:34.125 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/32' 2024-03-09T08:57:34.126 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/22' 2024-03-09T08:57:34.126 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/25' 2024-03-09T08:57:34.126 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/30' 2024-03-09T08:57:34.126 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/33' 2024-03-09T08:57:34.126 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/23' 2024-03-09T08:57:34.126 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/35' 2024-03-09T08:57:34.126 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/26' 2024-03-09T08:57:34.127 INFO:tasks.workunit.client.0.smithi067.stdout:removed './281/37' 2024-03-09T08:57:34.127 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './281' 2024-03-09T08:57:34.127 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/21' 2024-03-09T08:57:34.127 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/27' 2024-03-09T08:57:34.127 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/31' 2024-03-09T08:57:34.128 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/36' 2024-03-09T08:57:34.128 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/20' 2024-03-09T08:57:34.128 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/39' 2024-03-09T08:57:34.128 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/34' 2024-03-09T08:57:34.128 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/38' 2024-03-09T08:57:34.128 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/28' 2024-03-09T08:57:34.128 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/24' 2024-03-09T08:57:34.129 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/29' 2024-03-09T08:57:34.129 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/32' 2024-03-09T08:57:34.129 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/22' 2024-03-09T08:57:34.129 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/25' 2024-03-09T08:57:34.129 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/30' 2024-03-09T08:57:34.129 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/33' 2024-03-09T08:57:34.129 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/23' 2024-03-09T08:57:34.129 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/35' 2024-03-09T08:57:34.130 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/26' 2024-03-09T08:57:34.130 INFO:tasks.workunit.client.0.smithi067.stdout:removed './282/37' 2024-03-09T08:57:34.130 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './282' 2024-03-09T08:57:34.130 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/21' 2024-03-09T08:57:34.130 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/27' 2024-03-09T08:57:34.130 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/31' 2024-03-09T08:57:34.130 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/36' 2024-03-09T08:57:34.130 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/20' 2024-03-09T08:57:34.130 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/39' 2024-03-09T08:57:34.130 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/34' 2024-03-09T08:57:34.130 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/38' 2024-03-09T08:57:34.131 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/28' 2024-03-09T08:57:34.131 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/24' 2024-03-09T08:57:34.131 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/29' 2024-03-09T08:57:34.131 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/32' 2024-03-09T08:57:34.131 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/22' 2024-03-09T08:57:34.131 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/25' 2024-03-09T08:57:34.131 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/30' 2024-03-09T08:57:34.131 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/33' 2024-03-09T08:57:34.131 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/23' 2024-03-09T08:57:34.131 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/35' 2024-03-09T08:57:34.131 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/26' 2024-03-09T08:57:34.132 INFO:tasks.workunit.client.0.smithi067.stdout:removed './283/37' 2024-03-09T08:57:34.132 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './283' 2024-03-09T08:57:34.132 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/21' 2024-03-09T08:57:34.132 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/27' 2024-03-09T08:57:34.132 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/31' 2024-03-09T08:57:34.132 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/36' 2024-03-09T08:57:34.132 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/20' 2024-03-09T08:57:34.132 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/39' 2024-03-09T08:57:34.132 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/34' 2024-03-09T08:57:34.132 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/38' 2024-03-09T08:57:34.132 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/28' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/24' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/29' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/32' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/22' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/25' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/30' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/33' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/23' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/35' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/26' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './284/37' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './284' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/21' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/27' 2024-03-09T08:57:34.133 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/31' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/36' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/20' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/39' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/34' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/38' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/28' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/24' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/29' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/32' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/22' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/25' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/30' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/33' 2024-03-09T08:57:34.134 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/23' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/35' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/26' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './285/37' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './285' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/21' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/27' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/31' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/36' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/20' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/39' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/34' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/38' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/28' 2024-03-09T08:57:34.135 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/24' 2024-03-09T08:57:34.136 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/29' 2024-03-09T08:57:34.136 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/32' 2024-03-09T08:57:34.136 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/22' 2024-03-09T08:57:34.136 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/25' 2024-03-09T08:57:34.136 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/30' 2024-03-09T08:57:34.136 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/33' 2024-03-09T08:57:34.136 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/23' 2024-03-09T08:57:34.136 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/35' 2024-03-09T08:57:34.136 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/26' 2024-03-09T08:57:34.136 INFO:tasks.workunit.client.0.smithi067.stdout:removed './286/37' 2024-03-09T08:57:34.136 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './286' 2024-03-09T08:57:37.390 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T08:57:37.390 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T08:57:37.390 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T08:57:37.391 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T08:57:37.400 INFO:tasks.ceph.mon.c:Started 2024-03-09T08:57:37.400 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:57:37.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:57:37.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.612+0000 7f8fe8c00640 1 Processor -- start 2024-03-09T08:57:37.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.612+0000 7f8fe8c00640 1 -- start start 2024-03-09T08:57:37.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.615+0000 7f8fe8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8fe40c5e40 0x7f8fe4156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:37.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.615+0000 7f8fe8c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe4157000 con 0x7f8fe4153300 2024-03-09T08:57:37.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.615+0000 7f8fe8c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe40ae430 con 0x7f8fe40cadb0 2024-03-09T08:57:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.615+0000 7f8fe8c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe40ae6f0 con 0x7f8fe40c5e40 2024-03-09T08:57:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.615+0000 7f8fe2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8fe40c5e40 0x7f8fe4156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.615+0000 7f8fe3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8fe4153300 0x7f8fe40b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36696/0 (socket says 172.21.15.67:36696) 2024-03-09T08:57:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.615+0000 7f8fe3400640 1 -- 172.21.15.67:0/114581528 learned_addr learned my addr 172.21.15.67:0/114581528 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.616+0000 7f8fe2a00640 1 -- 172.21.15.67:0/114581528 >> v1:172.21.15.134:6789/0 conn(0x7f8fe40cadb0 legacy=0x7f8fe4156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.616+0000 7f8fe2a00640 1 -- 172.21.15.67:0/114581528 >> v1:172.21.15.67:6789/0 conn(0x7f8fe4153300 legacy=0x7f8fe40b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.616+0000 7f8fe2a00640 1 -- 172.21.15.67:0/114581528 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe4155b10 con 0x7f8fe40c5e40 2024-03-09T08:57:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.617+0000 7f8fe2a00640 1 --2- 172.21.15.67:0/114581528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fe40c5e40 0x7f8fe4156010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00097a0 tx=0x7f8fd002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a686bba3173a5c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.617+0000 7f8fe2000640 1 -- 172.21.15.67:0/114581528 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0034070 con 0x7f8fe40c5e40 2024-03-09T08:57:37.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.617+0000 7f8fe2000640 1 -- 172.21.15.67:0/114581528 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd0030610 con 0x7f8fe40c5e40 2024-03-09T08:57:37.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.617+0000 7f8fe2000640 1 -- 172.21.15.67:0/114581528 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd00309e0 con 0x7f8fe40c5e40 2024-03-09T08:57:37.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.618+0000 7f8fe8c00640 1 -- 172.21.15.67:0/114581528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fe40c5e40 msgr2=0x7f8fe4156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:37.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.618+0000 7f8fe8c00640 1 --2- 172.21.15.67:0/114581528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fe40c5e40 0x7f8fe4156010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00097a0 tx=0x7f8fd002b960 comp rx=0 tx=0).stop 2024-03-09T08:57:37.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.618+0000 7f8fe8c00640 1 -- 172.21.15.67:0/114581528 shutdown_connections 2024-03-09T08:57:37.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.618+0000 7f8fe8c00640 1 --2- 172.21.15.67:0/114581528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fe40c5e40 0x7f8fe4156010 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:37.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.618+0000 7f8fe8c00640 1 -- 172.21.15.67:0/114581528 >> 172.21.15.67:0/114581528 conn(0x7f8fe40732d0 msgr2=0x7f8fe40ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:37.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.618+0000 7f8fe8c00640 1 -- 172.21.15.67:0/114581528 shutdown_connections 2024-03-09T08:57:37.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.618+0000 7f8fe8c00640 1 -- 172.21.15.67:0/114581528 wait complete. 2024-03-09T08:57:37.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.620+0000 7f8fe8c00640 1 Processor -- start 2024-03-09T08:57:37.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.620+0000 7f8fe8c00640 1 -- start start 2024-03-09T08:57:37.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.621+0000 7f8fe8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fe40c5e40 0x7f8fe41ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:37.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.621+0000 7f8fe8c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fe40cadb0 0x7f8fe40ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.621+0000 7f8fe8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fe4153300 0x7f8fe414bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.621+0000 7f8fe8c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe40c59e0 con 0x7f8fe4153300 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.621+0000 7f8fe8c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe4159e90 con 0x7f8fe40c5e40 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.621+0000 7f8fe8c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe40b0710 con 0x7f8fe40cadb0 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.621+0000 7f8fe3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fe4153300 0x7f8fe414bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.621+0000 7f8fe2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fe40cadb0 0x7f8fe40ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.621+0000 7f8fe3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fe40c5e40 0x7f8fe41ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.621+0000 7f8fe2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fe40cadb0 0x7f8fe40ab210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:49992/0 (socket says 172.21.15.67:49992) 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.621+0000 7f8fe2a00640 1 -- 172.21.15.67:0/4154138958 learned_addr learned my addr 172.21.15.67:0/4154138958 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.622+0000 7f8fe2a00640 1 -- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fe40cadb0 msgr2=0x7f8fe40ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.622+0000 7f8fe2a00640 1 -- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fe40cadb0 msgr2=0x7f8fe40ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:57:37.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.622+0000 7f8fe2a00640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fe40cadb0 0x7f8fe40ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:37.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.622+0000 7f8fe2a00640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fe40cadb0 0x7f8fe40ab210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:37.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.622+0000 7f8fe3e00640 1 -- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fe40cadb0 msgr2=0x7f8fe40ab210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:57:37.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.622+0000 7f8fe3e00640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fe40cadb0 0x7f8fe40ab210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:37.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.622+0000 7f8fe3e00640 1 -- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fe40c5e40 msgr2=0x7f8fe41ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:37.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.622+0000 7f8fe3e00640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fe40c5e40 0x7f8fe41ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:37.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.622+0000 7f8fe3e00640 1 -- 172.21.15.67:0/4154138958 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe41ef640 con 0x7f8fe4153300 2024-03-09T08:57:37.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.622+0000 7f8fe3400640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fe40c5e40 0x7f8fe41ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:57:37.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.622+0000 7f8fe3e00640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fe4153300 0x7f8fe414bf90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00c310 tx=0x7f8fdc00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:37.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.623+0000 7f8fdb400640 1 -- 172.21.15.67:0/4154138958 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc017070 con 0x7f8fe4153300 2024-03-09T08:57:37.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.623+0000 7f8fdb400640 1 -- 172.21.15.67:0/4154138958 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fdc0052d0 con 0x7f8fe4153300 2024-03-09T08:57:37.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.623+0000 7f8fe8c00640 1 -- 172.21.15.67:0/4154138958 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe41eeda0 con 0x7f8fe4153300 2024-03-09T08:57:37.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.623+0000 7f8fdb400640 1 -- 172.21.15.67:0/4154138958 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc003830 con 0x7f8fe4153300 2024-03-09T08:57:37.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.623+0000 7f8fe8c00640 1 -- 172.21.15.67:0/4154138958 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe41ef360 con 0x7f8fe4153300 2024-03-09T08:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.624+0000 7f8fe8c00640 1 -- 172.21.15.67:0/4154138958 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe404f3c0 con 0x7f8fe4153300 2024-03-09T08:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.625+0000 7f8fdb400640 1 -- 172.21.15.67:0/4154138958 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8fdc01a070 con 0x7f8fe4153300 2024-03-09T08:57:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.626+0000 7f8fdb400640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8fc406b500 0x7f8fc406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:37.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.626+0000 7f8fdb400640 1 -- 172.21.15.67:0/4154138958 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f8fdc089860 con 0x7f8fe4153300 2024-03-09T08:57:37.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.626+0000 7f8fe3400640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8fc406b500 0x7f8fc406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:37.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.627+0000 7f8fe3400640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8fc406b500 0x7f8fc406d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4005c00 tx=0x7f8fd4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:37.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:37.632+0000 7f8fdb400640 1 -- 172.21.15.67:0/4154138958 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8fdc057d00 con 0x7f8fe4153300 2024-03-09T08:57:38.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.000+0000 7f8fe8c00640 1 -- 172.21.15.67:0/4154138958 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8fe414fe80 con 0x7f8fe4153300 2024-03-09T08:57:38.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.001+0000 7f8fdb400640 1 -- 172.21.15.67:0/4154138958 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f8fdc05bda0 con 0x7f8fe4153300 2024-03-09T08:57:38.000 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":40,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:57:38.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.003+0000 7f8fe8c00640 1 -- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8fc406b500 msgr2=0x7f8fc406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:38.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.003+0000 7f8fe8c00640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8fc406b500 0x7f8fc406d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4005c00 tx=0x7f8fd4009040 comp rx=0 tx=0).stop 2024-03-09T08:57:38.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.003+0000 7f8fe8c00640 1 -- 172.21.15.67:0/4154138958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fe4153300 msgr2=0x7f8fe414bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:38.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.003+0000 7f8fe8c00640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fe4153300 0x7f8fe414bf90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00c310 tx=0x7f8fdc00c7d0 comp rx=0 tx=0).stop 2024-03-09T08:57:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.003+0000 7f8fe8c00640 1 -- 172.21.15.67:0/4154138958 shutdown_connections 2024-03-09T08:57:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.003+0000 7f8fe8c00640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8fc406b500 0x7f8fc406d9c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.003+0000 7f8fe8c00640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fe4153300 0x7f8fe414bf90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.003+0000 7f8fe8c00640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fe40cadb0 0x7f8fe40ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.003+0000 7f8fe8c00640 1 --2- 172.21.15.67:0/4154138958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fe40c5e40 0x7f8fe41ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.003+0000 7f8fe8c00640 1 -- 172.21.15.67:0/4154138958 >> 172.21.15.67:0/4154138958 conn(0x7f8fe40732d0 msgr2=0x7f8fe40ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.004+0000 7f8fe8c00640 1 -- 172.21.15.67:0/4154138958 shutdown_connections 2024-03-09T08:57:38.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:38.004+0000 7f8fe8c00640 1 -- 172.21.15.67:0/4154138958 wait complete. 2024-03-09T08:57:38.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed './276/37' 2024-03-09T08:57:38.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './276' 2024-03-09T08:57:38.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/21' 2024-03-09T08:57:38.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/27' 2024-03-09T08:57:38.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/31' 2024-03-09T08:57:38.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/36' 2024-03-09T08:57:38.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/20' 2024-03-09T08:57:38.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/39' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/34' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/38' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/28' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/24' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/29' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/32' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/22' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/25' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/30' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/33' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/23' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/35' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/26' 2024-03-09T08:57:38.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed './277/37' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './277' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/21' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/27' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/31' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/36' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/20' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/39' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/34' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/38' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/28' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/24' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/29' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/32' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/22' 2024-03-09T08:57:38.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/25' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/30' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/33' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/23' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/35' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/26' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './278/37' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './278' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/21' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/27' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/31' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/36' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/20' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/39' 2024-03-09T08:57:38.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/34' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/38' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/28' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/24' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/29' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/32' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/22' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/25' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/30' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/33' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/23' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/35' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/26' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './279/37' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './279' 2024-03-09T08:57:38.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/21' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/27' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/31' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/36' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/20' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/39' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/34' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/38' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/28' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/24' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/29' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/32' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/22' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/25' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/30' 2024-03-09T08:57:38.099 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/33' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/23' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/35' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/26' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './280/37' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './280' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/21' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/27' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/31' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/36' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/20' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/39' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/34' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/38' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/28' 2024-03-09T08:57:38.100 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/24' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/29' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/32' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/22' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/25' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/30' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/33' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/23' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/35' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/26' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './281/37' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './281' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/21' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/27' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/31' 2024-03-09T08:57:38.101 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/36' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/20' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/39' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/34' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/38' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/28' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/24' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/29' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/32' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/22' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/25' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/30' 2024-03-09T08:57:38.102 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/33' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/23' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/35' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/26' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './282/37' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './282' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/21' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/27' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/31' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/36' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/20' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/39' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/34' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/38' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/28' 2024-03-09T08:57:38.103 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/24' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/29' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/32' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/22' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/25' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/30' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/33' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/23' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/35' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/26' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './283/37' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './283' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/21' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/27' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/31' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/36' 2024-03-09T08:57:38.104 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/20' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/39' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/34' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/38' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/28' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/24' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/29' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/32' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/22' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/25' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/30' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/33' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/23' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/35' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/26' 2024-03-09T08:57:38.105 INFO:tasks.workunit.client.1.smithi134.stdout:removed './284/37' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './284' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/21' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/27' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/31' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/36' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/20' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/39' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/34' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/38' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/28' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/24' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/29' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/32' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/22' 2024-03-09T08:57:38.106 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/25' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/30' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/33' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/23' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/35' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/26' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './285/37' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './285' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/21' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/27' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/31' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/36' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/20' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/39' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/34' 2024-03-09T08:57:38.107 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/38' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/28' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/24' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/29' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/32' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/22' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/25' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/30' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/33' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/23' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/35' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/26' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed './286/37' 2024-03-09T08:57:38.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './286' 2024-03-09T08:57:39.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/21' 2024-03-09T08:57:39.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/27' 2024-03-09T08:57:39.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/31' 2024-03-09T08:57:39.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/36' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/20' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/39' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/34' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/38' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/28' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/24' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/29' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/32' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/22' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/25' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/30' 2024-03-09T08:57:39.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/33' 2024-03-09T08:57:39.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/23' 2024-03-09T08:57:39.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/35' 2024-03-09T08:57:39.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/26' 2024-03-09T08:57:39.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed './287/37' 2024-03-09T08:57:39.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './287' 2024-03-09T08:57:39.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/21' 2024-03-09T08:57:39.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/27' 2024-03-09T08:57:39.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/31' 2024-03-09T08:57:39.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/36' 2024-03-09T08:57:39.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/20' 2024-03-09T08:57:39.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/39' 2024-03-09T08:57:39.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/34' 2024-03-09T08:57:39.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/38' 2024-03-09T08:57:39.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/28' 2024-03-09T08:57:39.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/24' 2024-03-09T08:57:39.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/29' 2024-03-09T08:57:39.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/32' 2024-03-09T08:57:39.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/22' 2024-03-09T08:57:39.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/25' 2024-03-09T08:57:39.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/30' 2024-03-09T08:57:39.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/33' 2024-03-09T08:57:39.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/23' 2024-03-09T08:57:39.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/35' 2024-03-09T08:57:39.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/26' 2024-03-09T08:57:39.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed './288/37' 2024-03-09T08:57:39.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './288' 2024-03-09T08:57:39.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/21' 2024-03-09T08:57:39.853 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/27' 2024-03-09T08:57:39.853 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/31' 2024-03-09T08:57:39.853 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/36' 2024-03-09T08:57:39.853 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/20' 2024-03-09T08:57:39.853 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/39' 2024-03-09T08:57:39.853 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/34' 2024-03-09T08:57:39.853 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/38' 2024-03-09T08:57:39.853 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/28' 2024-03-09T08:57:39.854 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/24' 2024-03-09T08:57:39.854 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/29' 2024-03-09T08:57:39.854 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/32' 2024-03-09T08:57:39.854 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/22' 2024-03-09T08:57:39.854 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/25' 2024-03-09T08:57:39.854 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/30' 2024-03-09T08:57:39.854 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/33' 2024-03-09T08:57:39.854 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/23' 2024-03-09T08:57:39.854 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/35' 2024-03-09T08:57:39.854 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/26' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './289/37' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './289' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/21' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/27' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/31' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/36' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/20' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/39' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/34' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/38' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/28' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/24' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/29' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/32' 2024-03-09T08:57:39.855 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/22' 2024-03-09T08:57:39.856 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/25' 2024-03-09T08:57:39.856 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/30' 2024-03-09T08:57:39.856 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/33' 2024-03-09T08:57:39.856 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/23' 2024-03-09T08:57:39.856 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/35' 2024-03-09T08:57:39.856 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/26' 2024-03-09T08:57:39.856 INFO:tasks.workunit.client.1.smithi134.stdout:removed './290/37' 2024-03-09T08:57:39.856 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './290' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/21' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/27' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/31' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/36' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/20' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/39' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/34' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/38' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/28' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/24' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/29' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/32' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/22' 2024-03-09T08:57:39.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/25' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/30' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/33' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/23' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/35' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/26' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './291/37' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './291' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/21' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/27' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/31' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/36' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/20' 2024-03-09T08:57:39.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/39' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/34' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/38' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/28' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/24' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/29' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/32' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/22' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/25' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/30' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/33' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/23' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/35' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/26' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed './292/37' 2024-03-09T08:57:39.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './292' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/21' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/27' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/31' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/36' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/20' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/39' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/34' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/38' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/28' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/24' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/29' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/32' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/22' 2024-03-09T08:57:39.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/25' 2024-03-09T08:57:39.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/30' 2024-03-09T08:57:39.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/33' 2024-03-09T08:57:39.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/23' 2024-03-09T08:57:39.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/35' 2024-03-09T08:57:39.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/26' 2024-03-09T08:57:39.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed './293/37' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './293' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/21' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/27' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/31' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/36' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/20' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/39' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/34' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/38' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/28' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/24' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/29' 2024-03-09T08:57:39.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/32' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/22' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/25' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/30' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/33' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/23' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/35' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/26' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './294/37' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './294' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/21' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/27' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/31' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/36' 2024-03-09T08:57:39.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/20' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/39' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/34' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/38' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/28' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/24' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/29' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/32' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/22' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/25' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/30' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/33' 2024-03-09T08:57:39.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/23' 2024-03-09T08:57:39.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/35' 2024-03-09T08:57:39.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/26' 2024-03-09T08:57:39.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed './295/37' 2024-03-09T08:57:39.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './295' 2024-03-09T08:57:39.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/21' 2024-03-09T08:57:39.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/27' 2024-03-09T08:57:39.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/31' 2024-03-09T08:57:39.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/36' 2024-03-09T08:57:39.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/20' 2024-03-09T08:57:39.866 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/39' 2024-03-09T08:57:39.866 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/34' 2024-03-09T08:57:39.866 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/38' 2024-03-09T08:57:39.866 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/28' 2024-03-09T08:57:39.866 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/24' 2024-03-09T08:57:39.866 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/29' 2024-03-09T08:57:39.867 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/32' 2024-03-09T08:57:39.867 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/22' 2024-03-09T08:57:39.867 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/25' 2024-03-09T08:57:39.867 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/30' 2024-03-09T08:57:39.867 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/33' 2024-03-09T08:57:39.867 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/23' 2024-03-09T08:57:39.867 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/35' 2024-03-09T08:57:39.867 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/26' 2024-03-09T08:57:39.867 INFO:tasks.workunit.client.1.smithi134.stdout:removed './296/37' 2024-03-09T08:57:39.868 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './296' 2024-03-09T08:57:39.868 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/21' 2024-03-09T08:57:39.868 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/27' 2024-03-09T08:57:40.498 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/21' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/27' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/31' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/36' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/20' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/39' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/34' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/38' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/28' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/24' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/29' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/32' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/22' 2024-03-09T08:57:40.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/25' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/30' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/33' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/23' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/35' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/26' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './287/37' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './287' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/21' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/27' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/31' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/36' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/20' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/39' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/34' 2024-03-09T08:57:40.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/38' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/28' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/24' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/29' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/32' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/22' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/25' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/30' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/33' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/23' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/35' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/26' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed './288/37' 2024-03-09T08:57:40.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './288' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/21' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/27' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/31' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/36' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/20' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/39' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/34' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/38' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/28' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/24' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/29' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/32' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/22' 2024-03-09T08:57:40.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/25' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/30' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/33' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/23' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/35' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/26' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed './289/37' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './289' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/21' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/27' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/31' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/36' 2024-03-09T08:57:40.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/20' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/39' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/34' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/38' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/28' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/24' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/29' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/32' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/22' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/25' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/30' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/33' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/23' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/35' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/26' 2024-03-09T08:57:40.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed './290/37' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './290' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/21' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/27' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/31' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/36' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/20' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/39' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/34' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/38' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/28' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/24' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/29' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/32' 2024-03-09T08:57:40.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/22' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/25' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/30' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/33' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/23' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/35' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/26' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './291/37' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './291' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/21' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/27' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/31' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/36' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/20' 2024-03-09T08:57:40.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/39' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/34' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/38' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/28' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/24' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/29' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/32' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/22' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/25' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/30' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/33' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/23' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/35' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/26' 2024-03-09T08:57:40.507 INFO:tasks.workunit.client.0.smithi067.stdout:removed './292/37' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './292' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/21' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/27' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/31' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/36' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/20' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/39' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/34' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/38' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/28' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/24' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/29' 2024-03-09T08:57:40.508 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/32' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/22' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/25' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/30' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/33' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/23' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/35' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/26' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './293/37' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './293' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/21' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/27' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/31' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/36' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/20' 2024-03-09T08:57:40.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/39' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/34' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/38' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/28' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/24' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/29' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/32' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/22' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/25' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/30' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/33' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/23' 2024-03-09T08:57:40.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/35' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/26' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './294/37' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './294' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/21' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/27' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/31' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/36' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/20' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/39' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/34' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/38' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/28' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/24' 2024-03-09T08:57:40.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/29' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/32' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/22' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/25' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/30' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/33' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/23' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/35' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/26' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './295/37' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './295' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/21' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/27' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/31' 2024-03-09T08:57:40.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/36' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/20' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/39' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/34' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/38' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/28' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/24' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/29' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/32' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/22' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/25' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/30' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/33' 2024-03-09T08:57:40.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/23' 2024-03-09T08:57:40.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/35' 2024-03-09T08:57:40.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/26' 2024-03-09T08:57:40.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './296/37' 2024-03-09T08:57:40.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './296' 2024-03-09T08:57:40.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/21' 2024-03-09T08:57:40.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/27' 2024-03-09T08:57:41.022 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:57:41.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/31' 2024-03-09T08:57:41.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/36' 2024-03-09T08:57:41.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/20' 2024-03-09T08:57:41.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/39' 2024-03-09T08:57:41.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/34' 2024-03-09T08:57:41.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/38' 2024-03-09T08:57:41.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/28' 2024-03-09T08:57:41.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/24' 2024-03-09T08:57:41.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/29' 2024-03-09T08:57:41.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/32' 2024-03-09T08:57:41.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/22' 2024-03-09T08:57:41.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/25' 2024-03-09T08:57:41.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/30' 2024-03-09T08:57:41.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/33' 2024-03-09T08:57:41.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/23' 2024-03-09T08:57:41.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/35' 2024-03-09T08:57:41.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/26' 2024-03-09T08:57:41.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './297/37' 2024-03-09T08:57:41.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './297' 2024-03-09T08:57:41.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/21' 2024-03-09T08:57:41.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/27' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/31' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/36' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/20' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/39' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/34' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/38' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/28' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/24' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/29' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/32' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/22' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/25' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/30' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/33' 2024-03-09T08:57:41.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/23' 2024-03-09T08:57:41.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/35' 2024-03-09T08:57:41.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/26' 2024-03-09T08:57:41.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './298/37' 2024-03-09T08:57:41.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './298' 2024-03-09T08:57:41.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/21' 2024-03-09T08:57:41.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/27' 2024-03-09T08:57:41.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/31' 2024-03-09T08:57:41.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/36' 2024-03-09T08:57:41.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/20' 2024-03-09T08:57:41.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/39' 2024-03-09T08:57:41.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/34' 2024-03-09T08:57:41.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/38' 2024-03-09T08:57:41.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/28' 2024-03-09T08:57:41.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/24' 2024-03-09T08:57:41.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/29' 2024-03-09T08:57:41.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/32' 2024-03-09T08:57:41.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/22' 2024-03-09T08:57:41.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/25' 2024-03-09T08:57:41.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/30' 2024-03-09T08:57:41.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/33' 2024-03-09T08:57:41.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/23' 2024-03-09T08:57:41.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/35' 2024-03-09T08:57:41.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/26' 2024-03-09T08:57:41.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './299/37' 2024-03-09T08:57:41.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './299' 2024-03-09T08:57:41.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/166/19' 2024-03-09T08:57:41.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/166/15' 2024-03-09T08:57:41.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/166/20' 2024-03-09T08:57:41.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/166/12' 2024-03-09T08:57:41.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/166/10' 2024-03-09T08:57:41.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/166/11' 2024-03-09T08:57:41.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/166/16' 2024-03-09T08:57:41.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/166/14' 2024-03-09T08:57:41.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/166/18' 2024-03-09T08:57:41.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/166/13' 2024-03-09T08:57:41.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/166/17' 2024-03-09T08:57:41.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/166' 2024-03-09T08:57:41.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/147/19' 2024-03-09T08:57:41.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/147/15' 2024-03-09T08:57:41.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/147/20' 2024-03-09T08:57:41.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/147/12' 2024-03-09T08:57:41.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/147/10' 2024-03-09T08:57:41.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/147/11' 2024-03-09T08:57:41.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/147/16' 2024-03-09T08:57:41.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/147/14' 2024-03-09T08:57:41.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/147/18' 2024-03-09T08:57:41.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/147/13' 2024-03-09T08:57:41.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/147/17' 2024-03-09T08:57:41.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/147' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/151/19' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/151/15' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/151/20' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/151/12' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/151/10' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/151/11' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/151/16' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/151/14' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/151/18' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/151/13' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/151/17' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/151' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/170/19' 2024-03-09T08:57:41.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/170/15' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/170/20' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/170/12' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/170/10' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/170/11' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/170/16' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/170/14' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/170/18' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/170/13' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/170/17' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/170' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/167/19' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/167/15' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/167/20' 2024-03-09T08:57:41.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/167/12' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/167/10' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/167/11' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/167/16' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/167/14' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/167/18' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/167/13' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/167/17' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/167' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/140/19' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/140/15' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/140/20' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/140/12' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/140/10' 2024-03-09T08:57:41.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/140/11' 2024-03-09T08:57:41.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/140/16' 2024-03-09T08:57:41.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/140/14' 2024-03-09T08:57:41.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/140/18' 2024-03-09T08:57:41.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/140/13' 2024-03-09T08:57:41.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/140/17' 2024-03-09T08:57:41.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/140' 2024-03-09T08:57:41.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/142/19' 2024-03-09T08:57:41.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/142/15' 2024-03-09T08:57:41.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/142/20' 2024-03-09T08:57:41.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/142/12' 2024-03-09T08:57:41.056 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/142/10' 2024-03-09T08:57:41.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.259+0000 7faf51400640 1 Processor -- start 2024-03-09T08:57:41.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.260+0000 7faf51400640 1 -- start start 2024-03-09T08:57:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.262+0000 7faf51400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faf4c0c7bc0 0x7faf4c161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.262+0000 7faf51400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c15a2a0 con 0x7faf4c153500 2024-03-09T08:57:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.262+0000 7faf51400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c06bb30 con 0x7faf4c0c6630 2024-03-09T08:57:41.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.262+0000 7faf51400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c159d70 con 0x7faf4c0c7bc0 2024-03-09T08:57:41.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.262+0000 7faf4a800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faf4c153500 0x7faf4c06b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44782/0 (socket says 172.21.15.67:44782) 2024-03-09T08:57:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.262+0000 7faf4a800640 1 -- 172.21.15.67:0/4283768022 learned_addr learned my addr 172.21.15.67:0/4283768022 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.262+0000 7faf4b200640 1 --2- 172.21.15.67:0/4283768022 >> v2:172.21.15.134:3300/0 conn(0x7faf4c0c7bc0 0x7faf4c161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.263+0000 7faf49400640 1 -- 172.21.15.67:0/4283768022 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505479938 0 0) 0x7faf4c15a2a0 con 0x7faf4c153500 2024-03-09T08:57:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.263+0000 7faf49400640 1 -- 172.21.15.67:0/4283768022 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf30003520 con 0x7faf4c153500 2024-03-09T08:57:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.263+0000 7faf4b200640 1 -- 172.21.15.67:0/4283768022 >> v1:172.21.15.134:6789/0 conn(0x7faf4c0c6630 legacy=0x7faf4c0c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.263+0000 7faf4b200640 1 -- 172.21.15.67:0/4283768022 >> v1:172.21.15.67:6789/0 conn(0x7faf4c153500 legacy=0x7faf4c06b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.263+0000 7faf49400640 1 -- 172.21.15.67:0/4283768022 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2340193570 0 0) 0x7faf30003520 con 0x7faf4c153500 2024-03-09T08:57:41.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.263+0000 7faf4b200640 1 -- 172.21.15.67:0/4283768022 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c159fc0 con 0x7faf4c0c7bc0 2024-03-09T08:57:41.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.265+0000 7faf4b200640 1 --2- 172.21.15.67:0/4283768022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf4c0c7bc0 0x7faf4c161700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faf44009d90 tx=0x7faf4402bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6801b45f6fccb509 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:41.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.265+0000 7faf49400640 1 -- 172.21.15.67:0/4283768022 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44039070 con 0x7faf4c0c7bc0 2024-03-09T08:57:41.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.265+0000 7faf49400640 1 -- 172.21.15.67:0/4283768022 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4402c820 con 0x7faf4c0c7bc0 2024-03-09T08:57:41.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.265+0000 7faf49400640 1 -- 172.21.15.67:0/4283768022 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c001670 con 0x7faf4c0c7bc0 2024-03-09T08:57:41.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.266+0000 7faf51400640 1 -- 172.21.15.67:0/4283768022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf4c0c7bc0 msgr2=0x7faf4c161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.266+0000 7faf51400640 1 --2- 172.21.15.67:0/4283768022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf4c0c7bc0 0x7faf4c161700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faf44009d90 tx=0x7faf4402bcf0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.266+0000 7faf51400640 1 -- 172.21.15.67:0/4283768022 shutdown_connections 2024-03-09T08:57:41.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.266+0000 7faf51400640 1 --2- 172.21.15.67:0/4283768022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf4c0c7bc0 0x7faf4c161700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.266+0000 7faf51400640 1 -- 172.21.15.67:0/4283768022 >> 172.21.15.67:0/4283768022 conn(0x7faf4c0c8060 msgr2=0x7faf4c0c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:41.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.266+0000 7faf51400640 1 -- 172.21.15.67:0/4283768022 shutdown_connections 2024-03-09T08:57:41.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.266+0000 7faf51400640 1 -- 172.21.15.67:0/4283768022 wait complete. 2024-03-09T08:57:41.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.268+0000 7faf51400640 1 Processor -- start 2024-03-09T08:57:41.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.268+0000 7faf51400640 1 -- start start 2024-03-09T08:57:41.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.269+0000 7faf51400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf4c0c7bc0 0x7faf4c0ac250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.269+0000 7faf51400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf4c153500 0x7faf4c0bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.269+0000 7faf51400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf4c157e90 0x7faf4c0c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.269+0000 7faf51400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0bed90 con 0x7faf4c157e90 2024-03-09T08:57:41.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.269+0000 7faf51400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0c6f90 con 0x7faf4c153500 2024-03-09T08:57:41.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.269+0000 7faf51400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c06b750 con 0x7faf4c0c7bc0 2024-03-09T08:57:41.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.270+0000 7faf4a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf4c0c7bc0 0x7faf4c0ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.270+0000 7faf4a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf4c0c7bc0 0x7faf4c0ac250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:38016/0 (socket says 172.21.15.67:38016) 2024-03-09T08:57:41.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.270+0000 7faf4a800640 1 -- 172.21.15.67:0/869357915 learned_addr learned my addr 172.21.15.67:0/869357915 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:41.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.270+0000 7faf49e00640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf4c153500 0x7faf4c0bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.270+0000 7faf4a800640 1 -- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf4c0c7bc0 msgr2=0x7faf4c0ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:57:41.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.270+0000 7faf4a800640 1 -- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf4c0c7bc0 msgr2=0x7faf4c0ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:57:41.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.270+0000 7faf4a800640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf4c0c7bc0 0x7faf4c0ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:41.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.270+0000 7faf4a800640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf4c0c7bc0 0x7faf4c0ac250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.270+0000 7faf4b200640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf4c157e90 0x7faf4c0c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.271+0000 7faf49e00640 1 -- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf4c0c7bc0 msgr2=0x7faf4c0ac250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:57:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.271+0000 7faf49e00640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf4c0c7bc0 0x7faf4c0ac250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.271+0000 7faf49e00640 1 -- 172.21.15.67:0/869357915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf4c157e90 msgr2=0x7faf4c0c00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.271+0000 7faf49e00640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf4c157e90 0x7faf4c0c00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.271+0000 7faf49e00640 1 -- 172.21.15.67:0/869357915 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c0c1ec0 con 0x7faf4c153500 2024-03-09T08:57:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.271+0000 7faf4b200640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf4c157e90 0x7faf4c0c00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:57:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.271+0000 7faf49e00640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf4c153500 0x7faf4c0bfb80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c3f0 tx=0x7faf3800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.272+0000 7faf42a00640 1 -- 172.21.15.67:0/869357915 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38017070 con 0x7faf4c153500 2024-03-09T08:57:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.272+0000 7faf51400640 1 -- 172.21.15.67:0/869357915 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c1573b0 con 0x7faf4c153500 2024-03-09T08:57:41.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.272+0000 7faf51400640 1 -- 172.21.15.67:0/869357915 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c157970 con 0x7faf4c153500 2024-03-09T08:57:41.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.272+0000 7faf42a00640 1 -- 172.21.15.67:0/869357915 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf380052d0 con 0x7faf4c153500 2024-03-09T08:57:41.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.272+0000 7faf42a00640 1 -- 172.21.15.67:0/869357915 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf38003830 con 0x7faf4c153500 2024-03-09T08:57:41.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.273+0000 7faf51400640 1 -- 172.21.15.67:0/869357915 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf24005290 con 0x7faf4c153500 2024-03-09T08:57:41.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.274+0000 7faf42a00640 1 -- 172.21.15.67:0/869357915 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faf3801a070 con 0x7faf4c153500 2024-03-09T08:57:41.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.275+0000 7faf42a00640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf2c06b4d0 0x7faf2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.275+0000 7faf42a00640 1 -- 172.21.15.67:0/869357915 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7faf38089a40 con 0x7faf4c153500 2024-03-09T08:57:41.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.276+0000 7faf4a800640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf2c06b4d0 0x7faf2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.278+0000 7faf4a800640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf2c06b4d0 0x7faf2c06d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faf3c001500 tx=0x7faf3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:41.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/31' 2024-03-09T08:57:41.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/36' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/20' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/39' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/34' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/38' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/28' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/24' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/29' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/32' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/22' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/25' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/30' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/33' 2024-03-09T08:57:41.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/23' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/35' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/26' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './297/37' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './297' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/21' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/27' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/31' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/36' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/20' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/39' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/34' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/38' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/28' 2024-03-09T08:57:41.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/24' 2024-03-09T08:57:41.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/29' 2024-03-09T08:57:41.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/32' 2024-03-09T08:57:41.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/22' 2024-03-09T08:57:41.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/25' 2024-03-09T08:57:41.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/30' 2024-03-09T08:57:41.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/33' 2024-03-09T08:57:41.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/23' 2024-03-09T08:57:41.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/35' 2024-03-09T08:57:41.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/26' 2024-03-09T08:57:41.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed './298/37' 2024-03-09T08:57:41.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './298' 2024-03-09T08:57:41.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/21' 2024-03-09T08:57:41.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/27' 2024-03-09T08:57:41.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/31' 2024-03-09T08:57:41.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/36' 2024-03-09T08:57:41.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/20' 2024-03-09T08:57:41.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/39' 2024-03-09T08:57:41.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/34' 2024-03-09T08:57:41.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/38' 2024-03-09T08:57:41.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/28' 2024-03-09T08:57:41.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/24' 2024-03-09T08:57:41.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/29' 2024-03-09T08:57:41.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/32' 2024-03-09T08:57:41.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/22' 2024-03-09T08:57:41.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/25' 2024-03-09T08:57:41.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/30' 2024-03-09T08:57:41.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/33' 2024-03-09T08:57:41.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/23' 2024-03-09T08:57:41.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/35' 2024-03-09T08:57:41.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/26' 2024-03-09T08:57:41.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed './299/37' 2024-03-09T08:57:41.285 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './299' 2024-03-09T08:57:41.285 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/166/19' 2024-03-09T08:57:41.285 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/166/15' 2024-03-09T08:57:41.285 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/166/20' 2024-03-09T08:57:41.285 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/166/12' 2024-03-09T08:57:41.285 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/166/10' 2024-03-09T08:57:41.286 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/166/11' 2024-03-09T08:57:41.286 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/166/16' 2024-03-09T08:57:41.286 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/166/14' 2024-03-09T08:57:41.287 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/166/18' 2024-03-09T08:57:41.287 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/166/13' 2024-03-09T08:57:41.287 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/166/17' 2024-03-09T08:57:41.287 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/166' 2024-03-09T08:57:41.287 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/147/19' 2024-03-09T08:57:41.287 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/147/15' 2024-03-09T08:57:41.288 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/147/20' 2024-03-09T08:57:41.288 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/147/12' 2024-03-09T08:57:41.288 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/147/10' 2024-03-09T08:57:41.288 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/147/11' 2024-03-09T08:57:41.288 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/147/16' 2024-03-09T08:57:41.289 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/147/14' 2024-03-09T08:57:41.289 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/147/18' 2024-03-09T08:57:41.289 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/147/13' 2024-03-09T08:57:41.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/147/17' 2024-03-09T08:57:41.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/147' 2024-03-09T08:57:41.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/151/19' 2024-03-09T08:57:41.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/151/15' 2024-03-09T08:57:41.290 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/151/20' 2024-03-09T08:57:41.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/151/12' 2024-03-09T08:57:41.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/151/10' 2024-03-09T08:57:41.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/151/11' 2024-03-09T08:57:41.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/151/16' 2024-03-09T08:57:41.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/151/14' 2024-03-09T08:57:41.291 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/151/18' 2024-03-09T08:57:41.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/151/13' 2024-03-09T08:57:41.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/151/17' 2024-03-09T08:57:41.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/151' 2024-03-09T08:57:41.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/170/19' 2024-03-09T08:57:41.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/170/15' 2024-03-09T08:57:41.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/170/20' 2024-03-09T08:57:41.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/170/12' 2024-03-09T08:57:41.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/170/10' 2024-03-09T08:57:41.292 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/170/11' 2024-03-09T08:57:41.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/170/16' 2024-03-09T08:57:41.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/170/14' 2024-03-09T08:57:41.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/170/18' 2024-03-09T08:57:41.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/170/13' 2024-03-09T08:57:41.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/170/17' 2024-03-09T08:57:41.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/170' 2024-03-09T08:57:41.293 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/167/19' 2024-03-09T08:57:41.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/167/15' 2024-03-09T08:57:41.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/167/20' 2024-03-09T08:57:41.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/167/12' 2024-03-09T08:57:41.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/167/10' 2024-03-09T08:57:41.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/167/11' 2024-03-09T08:57:41.294 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/167/16' 2024-03-09T08:57:41.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/167/14' 2024-03-09T08:57:41.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/167/18' 2024-03-09T08:57:41.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/167/13' 2024-03-09T08:57:41.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/167/17' 2024-03-09T08:57:41.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/167' 2024-03-09T08:57:41.295 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/140/19' 2024-03-09T08:57:41.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/140/15' 2024-03-09T08:57:41.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/140/20' 2024-03-09T08:57:41.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/140/12' 2024-03-09T08:57:41.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/140/10' 2024-03-09T08:57:41.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/140/11' 2024-03-09T08:57:41.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/140/16' 2024-03-09T08:57:41.296 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/140/14' 2024-03-09T08:57:41.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/140/18' 2024-03-09T08:57:41.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/140/13' 2024-03-09T08:57:41.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/140/17' 2024-03-09T08:57:41.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/140' 2024-03-09T08:57:41.297 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/142/19' 2024-03-09T08:57:41.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/142/15' 2024-03-09T08:57:41.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/142/20' 2024-03-09T08:57:41.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/142/12' 2024-03-09T08:57:41.298 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/142/10' 2024-03-09T08:57:41.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:57:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.289+0000 7faf42a00640 1 -- 172.21.15.67:0/869357915 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf38057e00 con 0x7faf4c153500 2024-03-09T08:57:41.306 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.643+0000 7faf51400640 1 -- 172.21.15.67:0/869357915 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7faf24005580 con 0x7faf4c153500 2024-03-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.645+0000 7faf42a00640 1 -- 172.21.15.67:0/869357915 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7faf3805bea0 con 0x7faf4c153500 2024-03-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":42,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.647+0000 7faf51400640 1 -- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf2c06b4d0 msgr2=0x7faf2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.647+0000 7faf51400640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf2c06b4d0 0x7faf2c06d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faf3c001500 tx=0x7faf3c009040 comp rx=0 tx=0).stop 2024-03-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.647+0000 7faf51400640 1 -- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf4c153500 msgr2=0x7faf4c0bfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.647+0000 7faf51400640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf4c153500 0x7faf4c0bfb80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf3800c3f0 tx=0x7faf3800c8b0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.647+0000 7faf51400640 1 -- 172.21.15.67:0/869357915 shutdown_connections 2024-03-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.647+0000 7faf51400640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf2c06b4d0 0x7faf2c06d990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.647+0000 7faf51400640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf4c157e90 0x7faf4c0c00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.647+0000 7faf51400640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf4c153500 0x7faf4c0bfb80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.647+0000 7faf51400640 1 --2- 172.21.15.67:0/869357915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf4c0c7bc0 0x7faf4c0ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.647+0000 7faf51400640 1 -- 172.21.15.67:0/869357915 >> 172.21.15.67:0/869357915 conn(0x7faf4c0c8060 msgr2=0x7faf4c0c5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.648+0000 7faf51400640 1 -- 172.21.15.67:0/869357915 shutdown_connections 2024-03-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.648+0000 7faf51400640 1 -- 172.21.15.67:0/869357915 wait complete. 2024-03-09T08:57:41.665 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:57:41.665 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:57:41.666 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:57:41.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.870+0000 7f671e000640 1 Processor -- start 2024-03-09T08:57:41.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.871+0000 7f671e000640 1 -- start start 2024-03-09T08:57:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.872+0000 7f671e000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718151360 0x7f67180c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.872+0000 7f671e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f67180c30a0 0x7f67181539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.872+0000 7f671e000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67180c2780 con 0x7f67180ac930 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.872+0000 7f671e000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6718070670 con 0x7f67180c30a0 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.872+0000 7f671e000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67180708c0 con 0x7f6718151360 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.872+0000 7f671d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f67180c30a0 0x7f67181539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.872+0000 7f671d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f67180c30a0 0x7f67181539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40316/0 (socket says 172.21.15.67:40316) 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.872+0000 7f671d600640 1 -- 172.21.15.67:0/779746334 learned_addr learned my addr 172.21.15.67:0/779746334 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.872+0000 7f671cc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f67180ac930 0x7f6718150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44792/0 (socket says 172.21.15.67:44792) 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.873+0000 7f6717e00640 1 --2- 172.21.15.67:0/779746334 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718151360 0x7f67180c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.873+0000 7f671d600640 1 -- 172.21.15.67:0/779746334 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718151360 msgr2=0x7f67180c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.873+0000 7f671d600640 1 --2- 172.21.15.67:0/779746334 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718151360 0x7f67180c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.873+0000 7f671d600640 1 -- 172.21.15.67:0/779746334 >> v1:172.21.15.67:6789/0 conn(0x7f67180ac930 legacy=0x7f6718150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.873+0000 7f671d600640 1 -- 172.21.15.67:0/779746334 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6718070ba0 con 0x7f67180c30a0 2024-03-09T08:57:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.873+0000 7f6717400640 1 -- 172.21.15.67:0/779746334 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428284286 0 0) 0x7f67180c2780 con 0x7f67180ac930 2024-03-09T08:57:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.874+0000 7f671d600640 1 --2- 172.21.15.67:0/779746334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67180c30a0 0x7f67181539e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6710009e50 tx=0x7f671002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c417916e3e3ead6d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.874+0000 7f6717400640 1 -- 172.21.15.67:0/779746334 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671002cb00 con 0x7f67180c30a0 2024-03-09T08:57:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.874+0000 7f6717400640 1 -- 172.21.15.67:0/779746334 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671002cca0 con 0x7f67180c30a0 2024-03-09T08:57:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.874+0000 7f6717400640 1 -- 172.21.15.67:0/779746334 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710034610 con 0x7f67180c30a0 2024-03-09T08:57:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.875+0000 7f671e000640 1 -- 172.21.15.67:0/779746334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67180c30a0 msgr2=0x7f67181539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.875+0000 7f671e000640 1 --2- 172.21.15.67:0/779746334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67180c30a0 0x7f67181539e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6710009e50 tx=0x7f671002bf60 comp rx=0 tx=0).stop 2024-03-09T08:57:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.875+0000 7f671e000640 1 -- 172.21.15.67:0/779746334 shutdown_connections 2024-03-09T08:57:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.875+0000 7f671e000640 1 --2- 172.21.15.67:0/779746334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67180c30a0 0x7f67181539e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.875+0000 7f671e000640 1 --2- 172.21.15.67:0/779746334 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6718151360 0x7f67180c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.875+0000 7f671e000640 1 -- 172.21.15.67:0/779746334 >> 172.21.15.67:0/779746334 conn(0x7f67180c7220 msgr2=0x7f67180c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:41.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.875+0000 7f671e000640 1 -- 172.21.15.67:0/779746334 shutdown_connections 2024-03-09T08:57:41.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.875+0000 7f671e000640 1 -- 172.21.15.67:0/779746334 wait complete. 2024-03-09T08:57:41.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.877+0000 7f671e000640 1 Processor -- start 2024-03-09T08:57:41.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.877+0000 7f671e000640 1 -- start start 2024-03-09T08:57:41.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671e000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67180ac930 0x7f67181ef210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67180c30a0 0x7f6718147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6718151360 0x7f671814f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671e000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67180ace90 con 0x7f6718151360 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671e000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67180acd10 con 0x7f67180c30a0 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671e000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6718153f20 con 0x7f67180ac930 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6718151360 0x7f671814f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6718151360 0x7f671814f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40318/0 (socket says 172.21.15.67:40318) 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671d600640 1 -- 172.21.15.67:0/2527289050 learned_addr learned my addr 172.21.15.67:0/2527289050 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671cc00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67180ac930 0x7f67181ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f6717e00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67180c30a0 0x7f6718147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671d600640 1 -- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67180ac930 msgr2=0x7f67181ef210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671d600640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67180ac930 0x7f67181ef210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671d600640 1 -- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67180c30a0 msgr2=0x7f6718147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671d600640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67180c30a0 0x7f6718147f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.878+0000 7f671d600640 1 -- 172.21.15.67:0/2527289050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67180c66a0 con 0x7f6718151360 2024-03-09T08:57:41.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.879+0000 7f671cc00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67180ac930 0x7f67181ef210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:57:41.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.879+0000 7f671d600640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6718151360 0x7f671814f730 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6710036040 tx=0x7f6710004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:41.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.879+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710043030 con 0x7f6718151360 2024-03-09T08:57:41.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.879+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671002ce50 con 0x7f6718151360 2024-03-09T08:57:41.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.879+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710009510 con 0x7f6718151360 2024-03-09T08:57:41.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.879+0000 7f671e000640 1 -- 172.21.15.67:0/2527289050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67180c60e0 con 0x7f6718151360 2024-03-09T08:57:41.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.879+0000 7f671e000640 1 -- 172.21.15.67:0/2527289050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67181eb3a0 con 0x7f6718151360 2024-03-09T08:57:41.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.881+0000 7f671e000640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67181484d0 0x7f67181eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.881+0000 7f671e000640 1 -- 172.21.15.67:0/2527289050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67181eb5e0 con 0x7f67181484d0 2024-03-09T08:57:41.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.881+0000 7f671cc00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67181484d0 0x7f67181eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.881+0000 7f671cc00640 1 -- 172.21.15.67:0/2527289050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67180c26f0 con 0x7f67181484d0 2024-03-09T08:57:41.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.882+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6710046070 con 0x7f6718151360 2024-03-09T08:57:41.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.882+0000 7f671cc00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67181484d0 0x7f67181eb8c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f670c000f60 tx=0x7f670c00db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:41.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.883+0000 7f6714c00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f66f806b4d0 0x7f66f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.883+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f67100ac860 con 0x7f6718151360 2024-03-09T08:57:41.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.884+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c018070 con 0x7f67181484d0 2024-03-09T08:57:41.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.884+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f670c0072d0 con 0x7f67181484d0 2024-03-09T08:57:41.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.884+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67181484d0 msgr2=0x7f67181eb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.884+0000 7f6714c00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67181484d0 0x7f67181eb8c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f670c000f60 tx=0x7f670c00db80 comp rx=0 tx=0).stop 2024-03-09T08:57:41.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.884+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67181484d0 msgr2=0x7f67181eb8c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:41.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.884+0000 7f6714c00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67181484d0 0x7f67181eb8c0 secure :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f670c000f60 tx=0x7f670c00db80 comp rx=0 tx=0).stop 2024-03-09T08:57:41.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.884+0000 7f6717e00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f66f806b4d0 0x7f66f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.885+0000 7f6717e00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f66f806b4d0 0x7f66f806d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6704006fd0 tx=0x7f6704008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:41.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.898+0000 7f671e000640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f671814fc70 0x7f67181ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:41.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.898+0000 7f671e000640 1 -- 172.21.15.67:0/2527289050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67180c0ec0 con 0x7f671814fc70 2024-03-09T08:57:41.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.898+0000 7f671cc00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f671814fc70 0x7f67181ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:41.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.899+0000 7f671cc00640 1 -- 172.21.15.67:0/2527289050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f67180c3890 con 0x7f671814fc70 2024-03-09T08:57:41.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.899+0000 7f671cc00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f671814fc70 0x7f67181ebe00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f670c000f60 tx=0x7f670c001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:41.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.899+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c018050 con 0x7f671814fc70 2024-03-09T08:57:41.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.900+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f670c019040 con 0x7f671814fc70 2024-03-09T08:57:41.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.900+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f671814fc70 msgr2=0x7f67181ebe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.900+0000 7f6714c00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f671814fc70 0x7f67181ebe00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f670c000f60 tx=0x7f670c001080 comp rx=0 tx=0).stop 2024-03-09T08:57:41.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.900+0000 7f6714c00640 1 -- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f671814fc70 msgr2=0x7f67181ebe00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:41.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.900+0000 7f6714c00640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f671814fc70 0x7f67181ebe00 secure :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f670c000f60 tx=0x7f670c001080 comp rx=0 tx=0).stop 2024-03-09T08:57:41.900 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:57:41.900 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:41.900 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:41.900 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:57:41.900 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 42, 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:41.901 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:41.902 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:57:41.903 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:41.904 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:57:41.905 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:41.905 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:41.905 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:41.905 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:41.905 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:41.905 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:41.905 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:41.905 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:41.905 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:41.905 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:41.905 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:57:41.906 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:41.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:41.908 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:41.909 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:41.910 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:41.910 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.910 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:41.910 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:57:41.910 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:41.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.910 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.910 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:41.910 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:57:41.911 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:41.912 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.913 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:57:41.914 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:57:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.900+0000 7f671e000640 1 -- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f66f806b4d0 msgr2=0x7f66f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.900+0000 7f671e000640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f66f806b4d0 0x7f66f806d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6704006fd0 tx=0x7f6704008040 comp rx=0 tx=0).stop 2024-03-09T08:57:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.900+0000 7f671e000640 1 -- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6718151360 msgr2=0x7f671814f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.900+0000 7f671e000640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6718151360 0x7f671814f730 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6710036040 tx=0x7f6710004a40 comp rx=0 tx=0).stop 2024-03-09T08:57:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.900+0000 7f671d600640 1 -- 172.21.15.67:0/2527289050 reap_dead start 2024-03-09T08:57:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.900+0000 7f671e000640 1 -- 172.21.15.67:0/2527289050 shutdown_connections 2024-03-09T08:57:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.901+0000 7f671e000640 1 --2- 172.21.15.67:0/2527289050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6718151360 0x7f671814f730 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.901+0000 7f671e000640 1 -- 172.21.15.67:0/2527289050 >> 172.21.15.67:0/2527289050 conn(0x7f67180c7220 msgr2=0x7f6718154300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.901+0000 7f671e000640 1 -- 172.21.15.67:0/2527289050 shutdown_connections 2024-03-09T08:57:41.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:41.901+0000 7f671e000640 1 -- 172.21.15.67:0/2527289050 wait complete. 2024-03-09T08:57:41.919 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:57:41.919 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:57:41.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.123+0000 7f5106a00640 1 Processor -- start 2024-03-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.123+0000 7f5106a00640 1 -- start start 2024-03-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.125+0000 7f5106a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 0x7f510006e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.125+0000 7f5106a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5100153a20 con 0x7f51000c46a0 2024-03-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.125+0000 7f5106a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5100153d00 con 0x7f5100150db0 2024-03-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.125+0000 7f5106a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5100153fe0 con 0x7f510006dcd0 2024-03-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.125+0000 7f5104c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f51000c46a0 0x7f510006b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44802/0 (socket says 172.21.15.67:44802) 2024-03-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.125+0000 7f5104c00640 1 -- 172.21.15.67:0/2631426741 learned_addr learned my addr 172.21.15.67:0/2631426741 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f5106000640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 0x7f510006e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725970422 0 0) 0x7f5100153a20 con 0x7f51000c46a0 2024-03-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50e8003520 con 0x7f51000c46a0 2024-03-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f5106000640 1 -- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 msgr2=0x7f510006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f5106000640 1 -- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 msgr2=0x7f510006e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f5106000640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 0x7f510006e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f5106000640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 0x7f510006e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1456414139 0 0) 0x7f50e8003520 con 0x7f51000c46a0 2024-03-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 msgr2=0x7f510006e140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 0x7f510006e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 >> v1:172.21.15.134:6789/0 conn(0x7f5100150db0 legacy=0x7f51001578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51001542c0 con 0x7f51000c46a0 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f50ec001670 con 0x7f51000c46a0 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 >> v1:172.21.15.67:6789/0 conn(0x7f51000c46a0 legacy=0x7f510006b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e8003a50 0x7f50e8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f50e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 0x7f50e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5100153fe0 con 0x7f50e8004800 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e8005630 con 0x7f50e8003a50 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.126+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e800bf00 con 0x7f510006dcd0 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.127+0000 7f5104c00640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f50e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.127+0000 7f5105600640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e8003a50 0x7f50e8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.127+0000 7f5106000640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 0x7f50e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.127+0000 7f5104c00640 1 -- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 msgr2=0x7f50e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.127+0000 7f5104c00640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f510006dcd0 0x7f50e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.127+0000 7f5104c00640 1 -- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e8003a50 msgr2=0x7f50e8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.127+0000 7f5104c00640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e8003a50 0x7f50e8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.127+0000 7f5104c00640 1 -- 172.21.15.67:0/2631426741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e8003520 con 0x7f50e8004800 2024-03-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.128+0000 7f5104c00640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f50e8004c80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00c0e0 tx=0x7f50e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad721865858e828 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.129+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec007c20 con 0x7f50e8004800 2024-03-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.129+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec030070 con 0x7f50e8004800 2024-03-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.129+0000 7f50ffe00640 1 -- 172.21.15.67:0/2631426741 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec005d30 con 0x7f50e8004800 2024-03-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.129+0000 7f5106a00640 1 -- 172.21.15.67:0/2631426741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 msgr2=0x7f50e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.129+0000 7f5106a00640 1 --2- 172.21.15.67:0/2631426741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50e8004800 0x7f50e8004c80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00c0e0 tx=0x7f50e8005830 comp rx=0 tx=0).stop 2024-03-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.129+0000 7f5106000640 1 -- 172.21.15.67:0/2631426741 reap_dead start 2024-03-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.129+0000 7f5106a00640 1 -- 172.21.15.67:0/2631426741 shutdown_connections 2024-03-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.129+0000 7f5106a00640 1 -- 172.21.15.67:0/2631426741 >> 172.21.15.67:0/2631426741 conn(0x7f51000c82a0 msgr2=0x7f51000c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.129+0000 7f5106a00640 1 -- 172.21.15.67:0/2631426741 shutdown_connections 2024-03-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.130+0000 7f5106a00640 1 -- 172.21.15.67:0/2631426741 wait complete. 2024-03-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.132+0000 7f5106a00640 1 Processor -- start 2024-03-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.132+0000 7f5106a00640 1 -- start start 2024-03-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.134+0000 7f5106a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e8004800 0x7f51001ebd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.134+0000 7f5106a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51000c7820 0x7f5100146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.134+0000 7f5106a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51001471c0 0x7f51001ee910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.134+0000 7f5106a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510006b8d0 con 0x7f51001471c0 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.134+0000 7f5106a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510006b750 con 0x7f50e8004800 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.134+0000 7f5106a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5100151190 con 0x7f51000c7820 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.134+0000 7f5106000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51001471c0 0x7f51001ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.134+0000 7f5106000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51001471c0 0x7f51001ee910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40360/0 (socket says 172.21.15.67:40360) 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.134+0000 7f5106000640 1 -- 172.21.15.67:0/2413790468 learned_addr learned my addr 172.21.15.67:0/2413790468 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.134+0000 7f5105600640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e8004800 0x7f51001ebd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f5104c00640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51000c7820 0x7f5100146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f5106000640 1 -- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51000c7820 msgr2=0x7f5100146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f5106000640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51000c7820 0x7f5100146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f5106000640 1 -- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e8004800 msgr2=0x7f51001ebd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f5106000640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e8004800 0x7f51001ebd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f5106000640 1 -- 172.21.15.67:0/2413790468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5100155840 con 0x7f51001471c0 2024-03-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f5104c00640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51000c7820 0x7f5100146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f5105600640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50e8004800 0x7f51001ebd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f5106000640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51001471c0 0x7f51001ee910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009610 tx=0x7f50f8009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8007d80 con 0x7f51001471c0 2024-03-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f8019070 con 0x7f51001471c0 2024-03-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f5106a00640 1 -- 172.21.15.67:0/2413790468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5100073390 con 0x7f51001471c0 2024-03-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.135+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8014650 con 0x7f51001471c0 2024-03-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.136+0000 7f5106a00640 1 -- 172.21.15.67:0/2413790468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5100072dd0 con 0x7f51001471c0 2024-03-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.137+0000 7f5106a00640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51001eee50 0x7f51000c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.137+0000 7f5106a00640 1 -- 172.21.15.67:0/2413790468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510004f3c0 con 0x7f51001eee50 2024-03-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.137+0000 7f5105600640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51001eee50 0x7f51000c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.138+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f50f80147f0 con 0x7f51001471c0 2024-03-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.138+0000 7f5105600640 1 -- 172.21.15.67:0/2413790468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51000ad490 con 0x7f51001eee50 2024-03-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.138+0000 7f5105600640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51001eee50 0x7f51000c16d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f50f4003ea0 tx=0x7f50f4003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.139+0000 7f50fd600640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f50e006b500 0x7f50e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.139+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f50f808a9b0 con 0x7f51001471c0 2024-03-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.139+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4013570 con 0x7f51001eee50 2024-03-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.139+0000 7f5104c00640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f50e006b500 0x7f50e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.140+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f50f4019070 con 0x7f51001eee50 2024-03-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.140+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51001eee50 msgr2=0x7f51000c16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.140+0000 7f50fd600640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51001eee50 0x7f51000c16d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f50f4003ea0 tx=0x7f50f4003f90 comp rx=0 tx=0).stop 2024-03-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.140+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51001eee50 msgr2=0x7f51000c16d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.140+0000 7f50fd600640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51001eee50 0x7f51000c16d0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f50f4003ea0 tx=0x7f50f4003f90 comp rx=0 tx=0).stop 2024-03-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.141+0000 7f5104c00640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f50e006b500 0x7f50e006d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f50ec034000 tx=0x7f50ec007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.147+0000 7f5106a00640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5100072fa0 0x7f51000c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.147+0000 7f5106a00640 1 -- 172.21.15.67:0/2413790468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510014bc00 con 0x7f5100072fa0 2024-03-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.147+0000 7f5105600640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5100072fa0 0x7f51000c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.148+0000 7f5105600640 1 -- 172.21.15.67:0/2413790468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f51000c8820 con 0x7f5100072fa0 2024-03-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.149+0000 7f5105600640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5100072fa0 0x7f51000c1c10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009010 tx=0x7f50f4001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.149+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4013d70 con 0x7f5100072fa0 2024-03-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.150+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4873 (secure 0 0 0) 0x7f50f4013d70 con 0x7f5100072fa0 2024-03-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.150+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5100072fa0 msgr2=0x7f51000c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.150+0000 7f50fd600640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5100072fa0 0x7f51000c1c10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009010 tx=0x7f50f4001120 comp rx=0 tx=0).stop 2024-03-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.150+0000 7f50fd600640 1 -- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5100072fa0 msgr2=0x7f51000c1c10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.150+0000 7f50fd600640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5100072fa0 0x7f51000c1c10 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009010 tx=0x7f50f4001120 comp rx=0 tx=0).stop 2024-03-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 42, 2024-03-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.151+0000 7f5106a00640 1 -- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f50e006b500 msgr2=0x7f50e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.151+0000 7f5106a00640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f50e006b500 0x7f50e006d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f50ec034000 tx=0x7f50ec007000 comp rx=0 tx=0).stop 2024-03-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.151+0000 7f5106a00640 1 -- 172.21.15.67:0/2413790468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51001471c0 msgr2=0x7f51001ee910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.151+0000 7f5106a00640 1 --2- 172.21.15.67:0/2413790468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51001471c0 0x7f51001ee910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009610 tx=0x7f50f8009dd0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.151+0000 7f5106000640 1 -- 172.21.15.67:0/2413790468 reap_dead start 2024-03-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.151+0000 7f5106000640 1 -- 172.21.15.67:0/2413790468 reap_dead start 2024-03-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.151+0000 7f5106a00640 1 -- 172.21.15.67:0/2413790468 shutdown_connections 2024-03-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.151+0000 7f5106a00640 1 -- 172.21.15.67:0/2413790468 >> 172.21.15.67:0/2413790468 conn(0x7f51000c82a0 msgr2=0x7f510014fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.151+0000 7f5106a00640 1 -- 172.21.15.67:0/2413790468 shutdown_connections 2024-03-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.151+0000 7f5106a00640 1 -- 172.21.15.67:0/2413790468 wait complete. 2024-03-09T08:57:42.169 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:57:42.170 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:57:42.170 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:57:42.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.404+0000 7f83a6e00640 1 Processor -- start 2024-03-09T08:57:42.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.405+0000 7f83a6e00640 1 -- start start 2024-03-09T08:57:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.406+0000 7f83a6e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f83a0157890 0x7f83a006d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.406+0000 7f83a6e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f83a006dad0 0x7f83a015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.406+0000 7f83a6e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a015a370 con 0x7f83a006e630 2024-03-09T08:57:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.406+0000 7f83a6e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a015b130 con 0x7f83a0157890 2024-03-09T08:57:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.406+0000 7f83a6e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a015a650 con 0x7f83a006dad0 2024-03-09T08:57:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.407+0000 7f839fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f83a0157890 0x7f83a006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.407+0000 7f839fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f83a0157890 0x7f83a006d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40372/0 (socket says 172.21.15.67:40372) 2024-03-09T08:57:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.407+0000 7f839fe00640 1 -- 172.21.15.67:0/1002341617 learned_addr learned my addr 172.21.15.67:0/1002341617 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.407+0000 7f83a4e00640 1 --2- 172.21.15.67:0/1002341617 >> v2:172.21.15.134:3300/0 conn(0x7f83a006dad0 0x7f83a015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.407+0000 7f839fe00640 1 -- 172.21.15.67:0/1002341617 >> v2:172.21.15.134:3300/0 conn(0x7f83a006dad0 msgr2=0x7f83a015bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.407+0000 7f839fe00640 1 --2- 172.21.15.67:0/1002341617 >> v2:172.21.15.134:3300/0 conn(0x7f83a006dad0 0x7f83a015bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.407+0000 7f839fe00640 1 -- 172.21.15.67:0/1002341617 >> v1:172.21.15.134:6789/0 conn(0x7f83a006e630 legacy=0x7f83a015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.407+0000 7f839fe00640 1 -- 172.21.15.67:0/1002341617 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a015a930 con 0x7f83a0157890 2024-03-09T08:57:42.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.409+0000 7f839fe00640 1 --2- 172.21.15.67:0/1002341617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83a0157890 0x7f83a006d590 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f83a00c4f10 tx=0x7f839002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d10ea29eeb98a62 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.409+0000 7f839ea00640 1 -- 172.21.15.67:0/1002341617 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390034070 con 0x7f83a0157890 2024-03-09T08:57:42.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.409+0000 7f839ea00640 1 -- 172.21.15.67:0/1002341617 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8390030860 con 0x7f83a0157890 2024-03-09T08:57:42.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.409+0000 7f839ea00640 1 -- 172.21.15.67:0/1002341617 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390030c30 con 0x7f83a0157890 2024-03-09T08:57:42.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.410+0000 7f83a6e00640 1 -- 172.21.15.67:0/1002341617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83a0157890 msgr2=0x7f83a006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.410+0000 7f83a6e00640 1 --2- 172.21.15.67:0/1002341617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83a0157890 0x7f83a006d590 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f83a00c4f10 tx=0x7f839002b9a0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.410+0000 7f83a6e00640 1 -- 172.21.15.67:0/1002341617 shutdown_connections 2024-03-09T08:57:42.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.410+0000 7f83a6e00640 1 --2- 172.21.15.67:0/1002341617 >> v2:172.21.15.134:3300/0 conn(0x7f83a006dad0 0x7f83a015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.410+0000 7f83a6e00640 1 --2- 172.21.15.67:0/1002341617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83a0157890 0x7f83a006d590 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.410+0000 7f83a6e00640 1 -- 172.21.15.67:0/1002341617 >> 172.21.15.67:0/1002341617 conn(0x7f83a00c7370 msgr2=0x7f83a00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:42.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.410+0000 7f83a6e00640 1 -- 172.21.15.67:0/1002341617 shutdown_connections 2024-03-09T08:57:42.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.410+0000 7f83a6e00640 1 -- 172.21.15.67:0/1002341617 wait complete. 2024-03-09T08:57:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.412+0000 7f83a6e00640 1 Processor -- start 2024-03-09T08:57:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.412+0000 7f83a6e00640 1 -- start start 2024-03-09T08:57:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.413+0000 7f83a6e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83a006dad0 0x7f83a01551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.413+0000 7f83a6e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83a006e630 0x7f83a00713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.413+0000 7f83a6e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0157890 0x7f83a0071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.413+0000 7f83a6e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00c1ef0 con 0x7f83a006dad0 2024-03-09T08:57:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.413+0000 7f83a6e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00c1d70 con 0x7f83a006e630 2024-03-09T08:57:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.413+0000 7f83a6e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a0157c70 con 0x7f83a0157890 2024-03-09T08:57:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.413+0000 7f839fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83a006dad0 0x7f83a01551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.413+0000 7f839fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83a006dad0 0x7f83a01551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40380/0 (socket says 172.21.15.67:40380) 2024-03-09T08:57:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.413+0000 7f839fe00640 1 -- 172.21.15.67:0/4142804481 learned_addr learned my addr 172.21.15.67:0/4142804481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:42.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.414+0000 7f83a4e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0157890 0x7f83a0071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.414+0000 7f839f400640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83a006e630 0x7f83a00713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.414+0000 7f839fe00640 1 -- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0157890 msgr2=0x7f83a0071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.414+0000 7f839fe00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0157890 0x7f83a0071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.414+0000 7f839fe00640 1 -- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83a006e630 msgr2=0x7f83a00713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.414+0000 7f839fe00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83a006e630 0x7f83a00713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.414+0000 7f839fe00640 1 -- 172.21.15.67:0/4142804481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0070f30 con 0x7f83a006dad0 2024-03-09T08:57:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.414+0000 7f839fe00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83a006dad0 0x7f83a01551b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8390000c00 tx=0x7f83900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.414+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390034040 con 0x7f83a006dad0 2024-03-09T08:57:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.414+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8390039d90 con 0x7f83a006dad0 2024-03-09T08:57:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.415+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390043610 con 0x7f83a006dad0 2024-03-09T08:57:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.415+0000 7f83a6e00640 1 -- 172.21.15.67:0/4142804481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a0072070 con 0x7f83a006dad0 2024-03-09T08:57:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.415+0000 7f83a6e00640 1 -- 172.21.15.67:0/4142804481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a01e1480 con 0x7f83a006dad0 2024-03-09T08:57:42.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.416+0000 7f83a6e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 0x7f83a01f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.416+0000 7f83a6e00640 1 -- 172.21.15.67:0/4142804481 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a0071100 con 0x7f83a0075a00 2024-03-09T08:57:42.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.417+0000 7f839f400640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 0x7f83a01f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.418+0000 7f839f400640 1 -- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 msgr2=0x7f83a01f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-03-09T08:57:42.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.418+0000 7f839f400640 1 -- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 msgr2=0x7f83a01f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:57:42.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.418+0000 7f839f400640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 0x7f83a01f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:57:42.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.418+0000 7f839f400640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 0x7f83a01f36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:57:42.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.418+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f839004a060 con 0x7f83a006dad0 2024-03-09T08:57:42.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.419+0000 7f8397e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f838006b4d0 0x7f838006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.419+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f83900ac660 con 0x7f83a006dad0 2024-03-09T08:57:42.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.420+0000 7f83a4e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f838006b4d0 0x7f838006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.421+0000 7f83a4e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f838006b4d0 0x7f838006d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8398009f80 tx=0x7f839800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.619+0000 7f839f400640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 0x7f83a01f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.621+0000 7f839f400640 1 -- 172.21.15.67:0/4142804481 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83a0073000 con 0x7f83a0075a00 2024-03-09T08:57:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.622+0000 7f839f400640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 0x7f83a01f36d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f83a0072a30 tx=0x7f838c0108a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.622+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c01a070 con 0x7f83a0075a00 2024-03-09T08:57:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.624+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f838c007230 con 0x7f83a0075a00 2024-03-09T08:57:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.624+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 msgr2=0x7f83a01f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.624+0000 7f8397e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 0x7f83a01f36d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f83a0072a30 tx=0x7f838c0108a0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.624+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 msgr2=0x7f83a01f36d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.624+0000 7f8397e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0075a00 0x7f83a01f36d0 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f83a0072a30 tx=0x7f838c0108a0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.634+0000 7f83a6e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0155a20 0x7f83a01f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.634+0000 7f83a6e00640 1 -- 172.21.15.67:0/4142804481 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00683f0 con 0x7f83a0155a20 2024-03-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.634+0000 7f839f400640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0155a20 0x7f83a01f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.635+0000 7f839f400640 1 -- 172.21.15.67:0/4142804481 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f83a006d630 con 0x7f83a0155a20 2024-03-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.636+0000 7f839f400640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0155a20 0x7f83a01f3c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83a0073350 tx=0x7f838c0036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.637+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c01a040 con 0x7f83a0155a20 2024-03-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.637+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4533 (secure 0 0 0) 0x7f838c006db0 con 0x7f83a0155a20 2024-03-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.637+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0155a20 msgr2=0x7f83a01f3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.637+0000 7f8397e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0155a20 0x7f83a01f3c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83a0073350 tx=0x7f838c0036f0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.638+0000 7f8397e00640 1 -- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0155a20 msgr2=0x7f83a01f3c10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.638+0000 7f8397e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83a0155a20 0x7f83a01f3c10 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83a0073350 tx=0x7f838c0036f0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 42, 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.638+0000 7f83a6e00640 1 -- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f838006b4d0 msgr2=0x7f838006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.638+0000 7f83a6e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f838006b4d0 0x7f838006d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8398009f80 tx=0x7f839800b040 comp rx=0 tx=0).stop 2024-03-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.638+0000 7f83a6e00640 1 -- 172.21.15.67:0/4142804481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83a006dad0 msgr2=0x7f83a01551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.638+0000 7f83a6e00640 1 --2- 172.21.15.67:0/4142804481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83a006dad0 0x7f83a01551b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8390000c00 tx=0x7f83900046c0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.638+0000 7f83a4e00640 1 -- 172.21.15.67:0/4142804481 reap_dead start 2024-03-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.638+0000 7f83a4e00640 1 -- 172.21.15.67:0/4142804481 reap_dead start 2024-03-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.638+0000 7f83a6e00640 1 -- 172.21.15.67:0/4142804481 shutdown_connections 2024-03-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.638+0000 7f83a6e00640 1 -- 172.21.15.67:0/4142804481 >> 172.21.15.67:0/4142804481 conn(0x7f83a00c7370 msgr2=0x7f83a00c6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.639+0000 7f83a6e00640 1 -- 172.21.15.67:0/4142804481 shutdown_connections 2024-03-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.639+0000 7f83a6e00640 1 -- 172.21.15.67:0/4142804481 wait complete. 2024-03-09T08:57:42.656 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:57:42.656 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T08:57:42.657 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T08:57:42.804 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/142/11' 2024-03-09T08:57:42.804 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/142/16' 2024-03-09T08:57:42.804 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/142/14' 2024-03-09T08:57:42.804 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/142/18' 2024-03-09T08:57:42.804 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/142/13' 2024-03-09T08:57:42.804 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/142/17' 2024-03-09T08:57:42.804 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/142' 2024-03-09T08:57:42.804 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/150/19' 2024-03-09T08:57:42.804 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/150/15' 2024-03-09T08:57:42.804 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/150/20' 2024-03-09T08:57:42.804 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/150/12' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/150/10' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/150/11' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/150/16' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/150/14' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/150/18' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/150/13' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/150/17' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/150' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/105/19' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/105/15' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/105/20' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/105/12' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/105/10' 2024-03-09T08:57:42.805 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/105/11' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/105/16' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/105/14' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/105/18' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/105/13' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/105/17' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/105' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/149/19' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/149/15' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/149/20' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/149/12' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/149/10' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/149/11' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/149/16' 2024-03-09T08:57:42.806 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/149/14' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/149/18' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/149/13' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/149/17' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/149' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/109/19' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/109/15' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/109/20' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/109/12' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/109/10' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/109/11' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/109/16' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/109/14' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/109/18' 2024-03-09T08:57:42.807 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/109/13' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/109/17' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/109' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/158/19' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/158/15' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/158/20' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/158/12' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/158/10' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/158/11' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/158/16' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/158/14' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/158/18' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/158/13' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/158/17' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/158' 2024-03-09T08:57:42.808 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/106/19' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/106/15' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/106/20' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/106/12' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/106/10' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/106/11' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/106/16' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/106/14' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/106/18' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/106/13' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/106/17' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/106' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/189/19' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/189/15' 2024-03-09T08:57:42.809 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/189/20' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/189/12' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/189/10' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/189/11' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/189/16' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/189/14' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/189/18' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/189/13' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/189/17' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/189' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/133/19' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/133/15' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/133/20' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/133/12' 2024-03-09T08:57:42.810 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/133/10' 2024-03-09T08:57:42.811 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/133/11' 2024-03-09T08:57:42.811 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/133/16' 2024-03-09T08:57:42.811 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/133/14' 2024-03-09T08:57:42.811 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/133/18' 2024-03-09T08:57:42.811 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/133/13' 2024-03-09T08:57:42.811 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/133/17' 2024-03-09T08:57:42.811 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/133' 2024-03-09T08:57:42.811 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/173/19' 2024-03-09T08:57:42.812 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/173/15' 2024-03-09T08:57:42.812 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/173/20' 2024-03-09T08:57:42.812 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/173/12' 2024-03-09T08:57:42.812 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/173/10' 2024-03-09T08:57:42.812 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/173/11' 2024-03-09T08:57:42.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.873+0000 7f150ca00640 1 Processor -- start 2024-03-09T08:57:42.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.874+0000 7f150ca00640 1 -- start start 2024-03-09T08:57:42.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.875+0000 7f150ca00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f15080c22a0 0x7f1508071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.875+0000 7f150ca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15081581a0 con 0x7f1508157d60 2024-03-09T08:57:42.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.875+0000 7f150ca00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1508153d40 con 0x7f15080ac730 2024-03-09T08:57:42.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.875+0000 7f150ca00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1508070ee0 con 0x7f15080c22a0 2024-03-09T08:57:42.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.875+0000 7f1506800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1508157d60 0x7f150815e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44816/0 (socket says 172.21.15.67:44816) 2024-03-09T08:57:42.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.876+0000 7f1506800640 1 -- 172.21.15.67:0/3010124553 learned_addr learned my addr 172.21.15.67:0/3010124553 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:42.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.876+0000 7f14ffe00640 1 --2- 172.21.15.67:0/3010124553 >> v2:172.21.15.134:3300/0 conn(0x7f15080c22a0 0x7f1508071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.876+0000 7f1505400640 1 -- 172.21.15.67:0/3010124553 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255918362 0 0) 0x7f15081581a0 con 0x7f1508157d60 2024-03-09T08:57:42.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.876+0000 7f1505400640 1 -- 172.21.15.67:0/3010124553 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14ec003520 con 0x7f1508157d60 2024-03-09T08:57:42.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.876+0000 7f14ffe00640 1 -- 172.21.15.67:0/3010124553 >> v1:172.21.15.134:6789/0 conn(0x7f15080ac730 legacy=0x7f1508157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:42.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.876+0000 7f14ffe00640 1 -- 172.21.15.67:0/3010124553 >> v1:172.21.15.67:6789/0 conn(0x7f1508157d60 legacy=0x7f150815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.876+0000 7f14ffe00640 1 -- 172.21.15.67:0/3010124553 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15080c2790 con 0x7f15080c22a0 2024-03-09T08:57:42.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.878+0000 7f14ffe00640 1 --2- 172.21.15.67:0/3010124553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c22a0 0x7f1508071190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14f00097a0 tx=0x7f14f002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a5f2bd352f148ef server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.878+0000 7f1505400640 1 -- 172.21.15.67:0/3010124553 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0035020 con 0x7f15080c22a0 2024-03-09T08:57:42.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.878+0000 7f1505400640 1 -- 172.21.15.67:0/3010124553 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f0031830 con 0x7f15080c22a0 2024-03-09T08:57:42.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.878+0000 7f1505400640 1 -- 172.21.15.67:0/3010124553 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0031be0 con 0x7f15080c22a0 2024-03-09T08:57:42.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.879+0000 7f150ca00640 1 -- 172.21.15.67:0/3010124553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c22a0 msgr2=0x7f1508071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.879+0000 7f150ca00640 1 --2- 172.21.15.67:0/3010124553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c22a0 0x7f1508071190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14f00097a0 tx=0x7f14f002b8d0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.879+0000 7f150ca00640 1 -- 172.21.15.67:0/3010124553 shutdown_connections 2024-03-09T08:57:42.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.879+0000 7f150ca00640 1 --2- 172.21.15.67:0/3010124553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c22a0 0x7f1508071190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.879+0000 7f150ca00640 1 -- 172.21.15.67:0/3010124553 >> 172.21.15.67:0/3010124553 conn(0x7f15080c7370 msgr2=0x7f15080c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:42.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.879+0000 7f150ca00640 1 -- 172.21.15.67:0/3010124553 shutdown_connections 2024-03-09T08:57:42.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.879+0000 7f150ca00640 1 -- 172.21.15.67:0/3010124553 wait complete. 2024-03-09T08:57:42.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.882+0000 7f150ca00640 1 Processor -- start 2024-03-09T08:57:42.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.882+0000 7f150ca00640 1 -- start start 2024-03-09T08:57:42.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.884+0000 7f150ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15080ac730 0x7f150814c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.884+0000 7f150ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c22a0 0x7f15080aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.884+0000 7f150ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1508157d60 0x7f15080c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.884+0000 7f150ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1508070c00 con 0x7f15080ac730 2024-03-09T08:57:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.884+0000 7f150ca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1508154d00 con 0x7f15080c22a0 2024-03-09T08:57:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.884+0000 7f150ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1508151610 con 0x7f1508157d60 2024-03-09T08:57:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.884+0000 7f1505e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15080ac730 0x7f150814c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.884+0000 7f1505e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15080ac730 0x7f150814c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40382/0 (socket says 172.21.15.67:40382) 2024-03-09T08:57:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.884+0000 7f1505e00640 1 -- 172.21.15.67:0/2969251636 learned_addr learned my addr 172.21.15.67:0/2969251636 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.884+0000 7f1506800640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1508157d60 0x7f15080c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.884+0000 7f14ffe00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c22a0 0x7f15080aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.885+0000 7f1505e00640 1 -- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1508157d60 msgr2=0x7f15080c0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.885+0000 7f1505e00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1508157d60 0x7f15080c0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.885+0000 7f1505e00640 1 -- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c22a0 msgr2=0x7f15080aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.885+0000 7f1505e00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c22a0 0x7f15080aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.885+0000 7f1505e00640 1 -- 172.21.15.67:0/2969251636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15081ed5a0 con 0x7f15080ac730 2024-03-09T08:57:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.885+0000 7f1506800640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1508157d60 0x7f15080c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.885+0000 7f1505e00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15080ac730 0x7f150814c2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c400 tx=0x7f14f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.886+0000 7f14fe000640 1 -- 172.21.15.67:0/2969251636 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8017070 con 0x7f15080ac730 2024-03-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.886+0000 7f14fe000640 1 -- 172.21.15.67:0/2969251636 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f80052d0 con 0x7f15080ac730 2024-03-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.886+0000 7f150ca00640 1 -- 172.21.15.67:0/2969251636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150814c830 con 0x7f15080ac730 2024-03-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.886+0000 7f14fe000640 1 -- 172.21.15.67:0/2969251636 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8003830 con 0x7f15080ac730 2024-03-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.886+0000 7f150ca00640 1 -- 172.21.15.67:0/2969251636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15081f0430 con 0x7f15080ac730 2024-03-09T08:57:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.887+0000 7f150ca00640 1 -- 172.21.15.67:0/2969251636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1508075a00 con 0x7f15080ac730 2024-03-09T08:57:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.888+0000 7f14fe000640 1 -- 172.21.15.67:0/2969251636 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f14f801a070 con 0x7f15080ac730 2024-03-09T08:57:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.889+0000 7f14fe000640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f14e406b4d0 0x7f14e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.889+0000 7f14fe000640 1 -- 172.21.15.67:0/2969251636 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f14f8089930 con 0x7f15080ac730 2024-03-09T08:57:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.890+0000 7f14ffe00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f14e406b4d0 0x7f14e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.891+0000 7f14ffe00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f14e406b4d0 0x7f14e406d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f14f0036000 tx=0x7f14f0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:42.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:42.894+0000 7f14fe000640 1 -- 172.21.15.67:0/2969251636 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f15081ed5a0 con 0x7f15080ac730 2024-03-09T08:57:42.908 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/142/11' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/142/16' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/142/14' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/142/18' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/142/13' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/142/17' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/142' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/150/19' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/150/15' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/150/20' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/150/12' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/150/10' 2024-03-09T08:57:42.909 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/150/11' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/150/16' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/150/14' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/150/18' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/150/13' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/150/17' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/150' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/105/19' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/105/15' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/105/20' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/105/12' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/105/10' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/105/11' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/105/16' 2024-03-09T08:57:42.910 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/105/14' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/105/18' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/105/13' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/105/17' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/105' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/149/19' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/149/15' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/149/20' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/149/12' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/149/10' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/149/11' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/149/16' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/149/14' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/149/18' 2024-03-09T08:57:42.911 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/149/13' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/149/17' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/149' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/109/19' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/109/15' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/109/20' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/109/12' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/109/10' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/109/11' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/109/16' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/109/14' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/109/18' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/109/13' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/109/17' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/109' 2024-03-09T08:57:42.912 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/158/19' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/158/15' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/158/20' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/158/12' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/158/10' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/158/11' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/158/16' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/158/14' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/158/18' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/158/13' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/158/17' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/158' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/106/19' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/106/15' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/106/20' 2024-03-09T08:57:42.913 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/106/12' 2024-03-09T08:57:42.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/106/10' 2024-03-09T08:57:42.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/106/11' 2024-03-09T08:57:42.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/106/16' 2024-03-09T08:57:42.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/106/14' 2024-03-09T08:57:42.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/106/18' 2024-03-09T08:57:42.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/106/13' 2024-03-09T08:57:42.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/106/17' 2024-03-09T08:57:42.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/106' 2024-03-09T08:57:42.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/189/19' 2024-03-09T08:57:42.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/189/15' 2024-03-09T08:57:42.914 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/189/20' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/189/12' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/189/10' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/189/11' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/189/16' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/189/14' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/189/18' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/189/13' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/189/17' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/189' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/133/19' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/133/15' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/133/20' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/133/12' 2024-03-09T08:57:42.915 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/133/10' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/133/11' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/133/16' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/133/14' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/133/18' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/133/13' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/133/17' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/133' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/173/19' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/173/15' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/173/20' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/173/12' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/173/10' 2024-03-09T08:57:42.916 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/173/11' 2024-03-09T08:57:43.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.263+0000 7f150ca00640 1 -- 172.21.15.67:0/2969251636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f15080c10b0 con 0x7f15080ac730 2024-03-09T08:57:43.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.264+0000 7f14fe000640 1 -- 172.21.15.67:0/2969251636 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f14f8087e20 con 0x7f15080ac730 2024-03-09T08:57:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 -- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f14e406b4d0 msgr2=0x7f14e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f14e406b4d0 0x7f14e406d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f14f0036000 tx=0x7f14f0008030 comp rx=0 tx=0).stop 2024-03-09T08:57:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 -- 172.21.15.67:0/2969251636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15080ac730 msgr2=0x7f150814c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15080ac730 0x7f150814c2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c400 tx=0x7f14f800c8c0 comp rx=0 tx=0).stop 2024-03-09T08:57:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 -- 172.21.15.67:0/2969251636 shutdown_connections 2024-03-09T08:57:43.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f14e406b4d0 0x7f14e406d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:43.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1508157d60 0x7f15080c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:43.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15080c22a0 0x7f15080aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:43.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 --2- 172.21.15.67:0/2969251636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15080ac730 0x7f150814c2f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:43.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 -- 172.21.15.67:0/2969251636 >> 172.21.15.67:0/2969251636 conn(0x7f15080c7370 msgr2=0x7f15080c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:43.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 -- 172.21.15.67:0/2969251636 shutdown_connections 2024-03-09T08:57:43.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:43.268+0000 7f150ca00640 1 -- 172.21.15.67:0/2969251636 wait complete. 2024-03-09T08:57:43.289 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T08:57:44.551 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/173/16' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/173/14' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/173/18' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/173/13' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/173/17' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/173' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/174/19' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/174/15' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/174/20' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/174/12' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/174/10' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/174/11' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/174/16' 2024-03-09T08:57:44.552 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/174/14' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/174/18' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/174/13' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/174/17' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/174' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/155/19' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/155/15' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/155/20' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/155/12' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/155/10' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/155/11' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/155/16' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/155/14' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/155/18' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/155/13' 2024-03-09T08:57:44.553 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/155/17' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/155' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/114/19' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/114/15' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/114/20' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/114/12' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/114/10' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/114/11' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/114/16' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/114/14' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/114/18' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/114/13' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/114/17' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/114' 2024-03-09T08:57:44.554 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/159/19' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/159/15' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/159/20' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/159/12' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/159/10' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/159/11' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/159/16' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/159/14' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/159/18' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/159/13' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/159/17' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/159' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/187/19' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/187/15' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/187/20' 2024-03-09T08:57:44.555 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/187/12' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/187/10' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/187/11' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/187/16' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/187/14' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/187/18' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/187/13' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/187/17' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/187' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/185/19' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/185/15' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/185/20' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/185/12' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/185/10' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/185/11' 2024-03-09T08:57:44.556 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/185/16' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/185/14' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/185/18' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/185/13' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/185/17' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/185' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/120/19' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/120/15' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/120/20' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/120/12' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/120/10' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/120/11' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/120/16' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/120/14' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/120/18' 2024-03-09T08:57:44.557 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/120/13' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/120/17' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/120' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/110/19' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/110/15' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/110/20' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/110/12' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/110/10' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/110/11' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/110/16' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/110/14' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/110/18' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/110/13' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/110/17' 2024-03-09T08:57:44.558 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/110' 2024-03-09T08:57:44.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/137/19' 2024-03-09T08:57:44.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/137/15' 2024-03-09T08:57:44.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/137/20' 2024-03-09T08:57:44.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/137/12' 2024-03-09T08:57:44.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/137/10' 2024-03-09T08:57:44.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/137/11' 2024-03-09T08:57:44.559 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/137/16' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/173/16' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/173/14' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/173/18' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/173/13' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/173/17' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/173' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/174/19' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/174/15' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/174/20' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/174/12' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/174/10' 2024-03-09T08:57:44.601 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/174/11' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/174/16' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/174/14' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/174/18' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/174/13' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/174/17' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/174' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/155/19' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/155/15' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/155/20' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/155/12' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/155/10' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/155/11' 2024-03-09T08:57:44.602 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/155/16' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/155/14' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/155/18' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/155/13' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/155/17' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/155' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/114/19' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/114/15' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/114/20' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/114/12' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/114/10' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/114/11' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/114/16' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/114/14' 2024-03-09T08:57:44.603 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/114/18' 2024-03-09T08:57:44.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/114/13' 2024-03-09T08:57:44.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/114/17' 2024-03-09T08:57:44.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/114' 2024-03-09T08:57:44.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/159/19' 2024-03-09T08:57:44.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/159/15' 2024-03-09T08:57:44.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/159/20' 2024-03-09T08:57:44.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/159/12' 2024-03-09T08:57:44.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/159/10' 2024-03-09T08:57:44.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/159/11' 2024-03-09T08:57:44.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/159/16' 2024-03-09T08:57:44.604 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/159/14' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/159/18' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/159/13' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/159/17' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/159' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/187/19' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/187/15' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/187/20' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/187/12' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/187/10' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/187/11' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/187/16' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/187/14' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/187/18' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/187/13' 2024-03-09T08:57:44.605 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/187/17' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/187' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/185/19' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/185/15' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/185/20' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/185/12' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/185/10' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/185/11' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/185/16' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/185/14' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/185/18' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/185/13' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/185/17' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/185' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/120/19' 2024-03-09T08:57:44.606 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/120/15' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/120/20' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/120/12' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/120/10' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/120/11' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/120/16' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/120/14' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/120/18' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/120/13' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/120/17' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/120' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/110/19' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/110/15' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/110/20' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/110/12' 2024-03-09T08:57:44.607 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/110/10' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/110/11' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/110/16' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/110/14' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/110/18' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/110/13' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/110/17' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/110' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/137/19' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/137/15' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/137/20' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/137/12' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/137/10' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/137/11' 2024-03-09T08:57:44.608 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/137/16' 2024-03-09T08:57:44.976 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/137/14' 2024-03-09T08:57:44.976 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/137/18' 2024-03-09T08:57:44.976 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/137/13' 2024-03-09T08:57:44.976 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/137/17' 2024-03-09T08:57:44.976 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/137' 2024-03-09T08:57:44.976 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/198/19' 2024-03-09T08:57:44.976 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/198/15' 2024-03-09T08:57:44.976 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/198/20' 2024-03-09T08:57:44.976 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/198/12' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/198/10' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/198/11' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/198/16' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/198/14' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/198/18' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/198/13' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/198/17' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/198' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/136/19' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/136/15' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/136/20' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/136/12' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/136/10' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/136/11' 2024-03-09T08:57:44.977 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/136/16' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/136/14' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/136/18' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/136/13' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/136/17' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/136' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/125/19' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/125/15' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/125/20' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/125/12' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/125/10' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/125/11' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/125/16' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/125/14' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/125/18' 2024-03-09T08:57:44.978 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/125/13' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/125/17' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/125' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/194/19' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/194/15' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/194/20' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/194/12' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/194/10' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/194/11' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/194/16' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/194/14' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/194/18' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/194/13' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/194/17' 2024-03-09T08:57:44.979 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/194' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/119/19' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/119/15' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/119/20' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/119/12' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/119/10' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/119/11' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/119/16' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/119/14' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/119/18' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/119/13' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/119/17' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/119' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/157/19' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/157/15' 2024-03-09T08:57:44.980 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/157/20' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/157/12' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/157/10' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/157/11' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/157/16' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/157/14' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/157/18' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/157/13' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/157/17' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/157' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/172/19' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/172/15' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/172/20' 2024-03-09T08:57:44.981 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/172/12' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/172/10' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/172/11' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/172/16' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/172/14' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/172/18' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/172/13' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/172/17' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/172' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/186/19' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/186/15' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/186/20' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/186/12' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/186/10' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/186/11' 2024-03-09T08:57:44.982 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/186/16' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/186/14' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/186/18' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/186/13' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/186/17' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/186' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/190/19' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/190/15' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/190/20' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/190/12' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/190/10' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/190/11' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/190/16' 2024-03-09T08:57:44.983 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/190/14' 2024-03-09T08:57:45.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/137/14' 2024-03-09T08:57:45.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/137/18' 2024-03-09T08:57:45.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/137/13' 2024-03-09T08:57:45.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/137/17' 2024-03-09T08:57:45.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/137' 2024-03-09T08:57:45.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/198/19' 2024-03-09T08:57:45.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/198/15' 2024-03-09T08:57:45.267 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/198/20' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/198/12' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/198/10' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/198/11' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/198/16' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/198/14' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/198/18' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/198/13' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/198/17' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/198' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/136/19' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/136/15' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/136/20' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/136/12' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/136/10' 2024-03-09T08:57:45.268 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/136/11' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/136/16' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/136/14' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/136/18' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/136/13' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/136/17' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/136' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/125/19' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/125/15' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/125/20' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/125/12' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/125/10' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/125/11' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/125/16' 2024-03-09T08:57:45.269 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/125/14' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/125/18' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/125/13' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/125/17' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/125' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/194/19' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/194/15' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/194/20' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/194/12' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/194/10' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/194/11' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/194/16' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/194/14' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/194/18' 2024-03-09T08:57:45.270 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/194/13' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/194/17' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/194' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/119/19' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/119/15' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/119/20' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/119/12' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/119/10' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/119/11' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/119/16' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/119/14' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/119/18' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/119/13' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/119/17' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/119' 2024-03-09T08:57:45.271 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/157/19' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/157/15' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/157/20' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/157/12' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/157/10' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/157/11' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/157/16' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/157/14' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/157/18' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/157/13' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/157/17' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/157' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/172/19' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/172/15' 2024-03-09T08:57:45.272 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/172/20' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/172/12' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/172/10' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/172/11' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/172/16' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/172/14' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/172/18' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/172/13' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/172/17' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/172' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/186/19' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/186/15' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/186/20' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/186/12' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/186/10' 2024-03-09T08:57:45.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/186/11' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/186/16' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/186/14' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/186/18' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/186/13' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/186/17' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/186' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/190/19' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/190/15' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/190/20' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/190/12' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/190/10' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/190/11' 2024-03-09T08:57:45.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/190/16' 2024-03-09T08:57:45.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/190/14' 2024-03-09T08:57:46.429 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/190/18' 2024-03-09T08:57:46.429 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/190/13' 2024-03-09T08:57:46.429 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/190/17' 2024-03-09T08:57:46.429 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/190' 2024-03-09T08:57:46.429 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/104/19' 2024-03-09T08:57:46.429 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/104/15' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/104/20' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/104/12' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/104/10' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/104/11' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/104/16' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/104/14' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/104/18' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/104/13' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/104/17' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/104' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/164/19' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/164/15' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/164/20' 2024-03-09T08:57:46.430 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/164/12' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/164/10' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/164/11' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/164/16' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/164/14' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/164/18' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/164/13' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/164/17' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/164' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/112/19' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/112/15' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/112/20' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/112/12' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/112/10' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/112/11' 2024-03-09T08:57:46.431 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/112/16' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/112/14' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/112/18' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/112/13' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/112/17' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/112' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/100/19' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/100/15' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/100/20' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/100/12' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/100/10' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/100/11' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/100/16' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/100/14' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/100/18' 2024-03-09T08:57:46.432 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/100/13' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/100/17' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/100' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/134/19' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/134/15' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/134/20' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/134/12' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/134/10' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/134/11' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/134/16' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/134/14' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/134/18' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/134/13' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/134/17' 2024-03-09T08:57:46.433 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/134' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/115/19' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/115/15' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/115/20' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/115/12' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/115/10' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/115/11' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/115/16' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/115/14' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/115/18' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/115/13' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/115/17' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/115' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/116/19' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/116/15' 2024-03-09T08:57:46.434 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/116/20' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/116/12' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/116/10' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/116/11' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/116/16' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/116/14' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/116/18' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/116/13' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/116/17' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/116' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/195/19' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/195/15' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/195/20' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/195/12' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/195/10' 2024-03-09T08:57:46.435 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/195/11' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/195/16' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/195/14' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/195/18' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/195/13' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/195/17' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/195' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/138/19' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/138/15' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/138/20' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/138/12' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/138/10' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/138/11' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/138/16' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/138/14' 2024-03-09T08:57:46.436 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/138/18' 2024-03-09T08:57:47.071 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/190/18' 2024-03-09T08:57:47.071 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/190/13' 2024-03-09T08:57:47.071 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/190/17' 2024-03-09T08:57:47.071 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/190' 2024-03-09T08:57:47.071 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/104/19' 2024-03-09T08:57:47.071 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/104/15' 2024-03-09T08:57:47.071 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/104/20' 2024-03-09T08:57:47.071 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/104/12' 2024-03-09T08:57:47.071 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/104/10' 2024-03-09T08:57:47.071 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/104/11' 2024-03-09T08:57:47.071 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/104/16' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/104/14' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/104/18' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/104/13' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/104/17' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/104' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/164/19' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/164/15' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/164/20' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/164/12' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/164/10' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/164/11' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/164/16' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/164/14' 2024-03-09T08:57:47.072 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/164/18' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/164/13' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/164/17' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/164' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/112/19' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/112/15' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/112/20' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/112/12' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/112/10' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/112/11' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/112/16' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/112/14' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/112/18' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/112/13' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/112/17' 2024-03-09T08:57:47.073 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/112' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/100/19' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/100/15' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/100/20' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/100/12' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/100/10' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/100/11' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/100/16' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/100/14' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/100/18' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/100/13' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/100/17' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/100' 2024-03-09T08:57:47.074 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/134/19' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/134/15' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/134/20' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/134/12' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/134/10' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/134/11' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/134/16' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/134/14' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/134/18' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/134/13' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/134/17' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/134' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/115/19' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/115/15' 2024-03-09T08:57:47.075 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/115/20' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/115/12' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/115/10' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/115/11' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/115/16' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/115/14' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/115/18' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/115/13' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/115/17' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/115' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/116/19' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/116/15' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/116/20' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/116/12' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/116/10' 2024-03-09T08:57:47.076 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/116/11' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/116/16' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/116/14' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/116/18' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/116/13' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/116/17' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/116' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/195/19' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/195/15' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/195/20' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/195/12' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/195/10' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/195/11' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/195/16' 2024-03-09T08:57:47.077 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/195/14' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/195/18' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/195/13' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/195/17' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/195' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/138/19' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/138/15' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/138/20' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/138/12' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/138/10' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/138/11' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/138/16' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/138/14' 2024-03-09T08:57:47.078 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/138/18' 2024-03-09T08:57:47.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/138/13' 2024-03-09T08:57:47.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/138/17' 2024-03-09T08:57:47.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/138' 2024-03-09T08:57:47.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/135/19' 2024-03-09T08:57:47.932 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/135/15' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/135/20' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/135/12' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/135/10' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/135/11' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/135/16' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/135/14' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/135/18' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/135/13' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/135/17' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/135' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/117/19' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/117/15' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/117/20' 2024-03-09T08:57:47.933 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/117/12' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/117/10' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/117/11' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/117/16' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/117/14' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/117/18' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/117/13' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/117/17' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/117' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/143/19' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/143/15' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/143/20' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/143/12' 2024-03-09T08:57:47.934 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/143/10' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/143/11' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/143/16' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/143/14' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/143/18' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/143/13' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/143/17' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/143' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/180/19' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/180/15' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/180/20' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/180/12' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/180/10' 2024-03-09T08:57:47.935 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/180/11' 2024-03-09T08:57:47.936 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/180/16' 2024-03-09T08:57:47.936 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/180/14' 2024-03-09T08:57:47.936 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/180/18' 2024-03-09T08:57:47.936 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/180/13' 2024-03-09T08:57:47.936 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/180/17' 2024-03-09T08:57:47.936 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/180' 2024-03-09T08:57:47.936 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/126/19' 2024-03-09T08:57:47.936 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/126/15' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/126/20' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/126/12' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/126/10' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/126/11' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/126/16' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/126/14' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/126/18' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/126/13' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/126/17' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/126' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/161/19' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/161/15' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/161/20' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/161/12' 2024-03-09T08:57:47.937 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/161/10' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/161/11' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/161/16' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/161/14' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/161/18' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/161/13' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/161/17' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/161' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/176/19' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/176/15' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/176/20' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/176/12' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/176/10' 2024-03-09T08:57:47.938 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/176/11' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/176/16' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/176/14' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/176/18' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/176/13' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/176/17' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/176' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/111/19' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/111/15' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/111/20' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/111/12' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/111/10' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/111/11' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/111/16' 2024-03-09T08:57:47.939 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/111/14' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/111/18' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/111/13' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/111/17' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/111' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/191/19' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/191/15' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/191/20' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/191/12' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/191/10' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/191/11' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/191/16' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/191/14' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/191/18' 2024-03-09T08:57:47.940 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/191/13' 2024-03-09T08:57:48.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/138/13' 2024-03-09T08:57:48.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/138/17' 2024-03-09T08:57:48.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/138' 2024-03-09T08:57:48.041 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/135/19' 2024-03-09T08:57:48.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/135/15' 2024-03-09T08:57:48.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/135/20' 2024-03-09T08:57:48.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/135/12' 2024-03-09T08:57:48.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/135/10' 2024-03-09T08:57:48.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/135/11' 2024-03-09T08:57:48.042 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/135/16' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/135/14' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/135/18' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/135/13' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/135/17' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/135' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/117/19' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/117/15' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/117/20' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/117/12' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/117/10' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/117/11' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/117/16' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/117/14' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/117/18' 2024-03-09T08:57:48.043 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/117/13' 2024-03-09T08:57:48.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/117/17' 2024-03-09T08:57:48.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/117' 2024-03-09T08:57:48.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/143/19' 2024-03-09T08:57:48.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/143/15' 2024-03-09T08:57:48.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/143/20' 2024-03-09T08:57:48.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/143/12' 2024-03-09T08:57:48.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/143/10' 2024-03-09T08:57:48.044 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/143/11' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/143/16' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/143/14' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/143/18' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/143/13' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/143/17' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/143' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/180/19' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/180/15' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/180/20' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/180/12' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/180/10' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/180/11' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/180/16' 2024-03-09T08:57:48.045 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/180/14' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/180/18' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/180/13' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/180/17' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/180' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/126/19' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/126/15' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/126/20' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/126/12' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/126/10' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/126/11' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/126/16' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/126/14' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/126/18' 2024-03-09T08:57:48.046 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/126/13' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/126/17' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/126' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/161/19' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/161/15' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/161/20' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/161/12' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/161/10' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/161/11' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/161/16' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/161/14' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/161/18' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/161/13' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/161/17' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/161' 2024-03-09T08:57:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/176/19' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/176/15' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/176/20' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/176/12' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/176/10' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/176/11' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/176/16' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/176/14' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/176/18' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/176/13' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/176/17' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/176' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/111/19' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/111/15' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/111/20' 2024-03-09T08:57:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/111/12' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/111/10' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/111/11' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/111/16' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/111/14' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/111/18' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/111/13' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/111/17' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/111' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/191/19' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/191/15' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/191/20' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/191/12' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/191/10' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/191/11' 2024-03-09T08:57:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/191/16' 2024-03-09T08:57:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/191/14' 2024-03-09T08:57:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/191/18' 2024-03-09T08:57:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/191/13' 2024-03-09T08:57:48.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/191/17' 2024-03-09T08:57:48.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/191' 2024-03-09T08:57:48.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/184/19' 2024-03-09T08:57:48.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/184/15' 2024-03-09T08:57:48.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/184/20' 2024-03-09T08:57:48.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/184/12' 2024-03-09T08:57:48.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/184/10' 2024-03-09T08:57:48.823 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/184/11' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/184/16' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/184/14' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/184/18' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/184/13' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/184/17' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/184' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/192/19' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/192/15' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/192/20' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/192/12' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/192/10' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/192/11' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/192/16' 2024-03-09T08:57:48.824 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/192/14' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/192/18' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/192/13' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/192/17' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/192' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/103/19' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/103/15' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/103/20' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/103/12' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/103/10' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/103/11' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/103/16' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/103/14' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/103/18' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/103/13' 2024-03-09T08:57:48.825 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/103/17' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/103' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/128/19' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/128/15' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/128/20' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/128/12' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/128/10' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/128/11' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/128/16' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/128/14' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/128/18' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/128/13' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/128/17' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/128' 2024-03-09T08:57:48.826 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/144/19' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/144/15' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/144/20' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/144/12' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/144/10' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/144/11' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/144/16' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/144/14' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/144/18' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/144/13' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/144/17' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/144' 2024-03-09T08:57:48.827 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/146/19' 2024-03-09T08:57:48.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/146/15' 2024-03-09T08:57:48.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/146/20' 2024-03-09T08:57:48.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/146/12' 2024-03-09T08:57:48.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/146/10' 2024-03-09T08:57:48.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/146/11' 2024-03-09T08:57:48.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/146/16' 2024-03-09T08:57:48.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/146/14' 2024-03-09T08:57:48.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/146/18' 2024-03-09T08:57:48.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/146/13' 2024-03-09T08:57:48.828 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/146/17' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/146' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/141/19' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/141/15' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/141/20' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/141/12' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/141/10' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/141/11' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/141/16' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/141/14' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/141/18' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/141/13' 2024-03-09T08:57:48.829 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/141/17' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/141' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/182/19' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/182/15' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/182/20' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/182/12' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/182/10' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/182/11' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/182/16' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/182/14' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/182/18' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/182/13' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/182/17' 2024-03-09T08:57:48.830 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/182' 2024-03-09T08:57:48.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/107/19' 2024-03-09T08:57:48.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/107/15' 2024-03-09T08:57:48.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/107/20' 2024-03-09T08:57:48.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/107/12' 2024-03-09T08:57:48.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/107/10' 2024-03-09T08:57:48.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/107/11' 2024-03-09T08:57:48.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/107/16' 2024-03-09T08:57:48.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/107/14' 2024-03-09T08:57:48.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/107/18' 2024-03-09T08:57:48.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/107/13' 2024-03-09T08:57:48.831 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/107/17' 2024-03-09T08:57:50.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/191/17' 2024-03-09T08:57:50.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/191' 2024-03-09T08:57:50.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/184/19' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/184/15' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/184/20' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/184/12' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/184/10' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/184/11' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/184/16' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/184/14' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/184/18' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/184/13' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/184/17' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/184' 2024-03-09T08:57:50.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/192/19' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/192/15' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/192/20' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/192/12' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/192/10' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/192/11' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/192/16' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/192/14' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/192/18' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/192/13' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/192/17' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/192' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/103/19' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/103/15' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/103/20' 2024-03-09T08:57:50.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/103/12' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/103/10' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/103/11' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/103/16' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/103/14' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/103/18' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/103/13' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/103/17' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/103' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/128/19' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/128/15' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/128/20' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/128/12' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/128/10' 2024-03-09T08:57:50.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/128/11' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/128/16' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/128/14' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/128/18' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/128/13' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/128/17' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/128' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/144/19' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/144/15' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/144/20' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/144/12' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/144/10' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/144/11' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/144/16' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/144/14' 2024-03-09T08:57:50.320 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/144/18' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/144/13' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/144/17' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/144' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/146/19' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/146/15' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/146/20' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/146/12' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/146/10' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/146/11' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/146/16' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/146/14' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/146/18' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/146/13' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/146/17' 2024-03-09T08:57:50.321 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/146' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/141/19' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/141/15' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/141/20' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/141/12' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/141/10' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/141/11' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/141/16' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/141/14' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/141/18' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/141/13' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/141/17' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/141' 2024-03-09T08:57:50.322 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/182/19' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/182/15' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/182/20' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/182/12' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/182/10' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/182/11' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/182/16' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/182/14' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/182/18' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/182/13' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/182/17' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/182' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/107/19' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/107/15' 2024-03-09T08:57:50.323 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/107/20' 2024-03-09T08:57:50.324 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/107/12' 2024-03-09T08:57:50.324 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/107/10' 2024-03-09T08:57:50.324 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/107/11' 2024-03-09T08:57:50.324 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/107/16' 2024-03-09T08:57:50.324 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/107/14' 2024-03-09T08:57:50.324 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/107/18' 2024-03-09T08:57:50.324 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/107/13' 2024-03-09T08:57:50.324 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/107/17' 2024-03-09T08:57:50.633 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/107' 2024-03-09T08:57:50.633 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/178/19' 2024-03-09T08:57:50.633 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/178/15' 2024-03-09T08:57:50.633 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/178/20' 2024-03-09T08:57:50.633 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/178/12' 2024-03-09T08:57:50.633 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/178/10' 2024-03-09T08:57:50.633 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/178/11' 2024-03-09T08:57:50.633 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/178/16' 2024-03-09T08:57:50.633 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/178/14' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/178/18' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/178/13' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/178/17' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/178' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/156/19' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/156/15' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/156/20' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/156/12' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/156/10' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/156/11' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/156/16' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/156/14' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/156/18' 2024-03-09T08:57:50.634 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/156/13' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/156/17' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/156' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/131/19' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/131/15' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/131/20' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/131/12' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/131/10' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/131/11' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/131/16' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/131/14' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/131/18' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/131/13' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/131/17' 2024-03-09T08:57:50.635 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/131' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/123/19' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/123/15' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/123/20' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/123/12' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/123/10' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/123/11' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/123/16' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/123/14' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/123/18' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/123/13' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/123/17' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/123' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/160/19' 2024-03-09T08:57:50.636 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/160/15' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/160/20' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/160/12' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/160/10' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/160/11' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/160/16' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/160/14' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/160/18' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/160/13' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/160/17' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/160' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/165/19' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/165/15' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/165/20' 2024-03-09T08:57:50.637 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/165/12' 2024-03-09T08:57:50.638 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/165/10' 2024-03-09T08:57:50.638 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/165/11' 2024-03-09T08:57:50.638 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/165/16' 2024-03-09T08:57:50.638 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/165/14' 2024-03-09T08:57:50.638 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/165/18' 2024-03-09T08:57:50.639 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/165/13' 2024-03-09T08:57:50.639 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/165/17' 2024-03-09T08:57:50.640 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/165' 2024-03-09T08:57:50.640 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/101/19' 2024-03-09T08:57:50.640 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/101/15' 2024-03-09T08:57:50.640 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/101/20' 2024-03-09T08:57:50.640 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/101/12' 2024-03-09T08:57:50.640 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/101/10' 2024-03-09T08:57:50.640 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/101/11' 2024-03-09T08:57:50.640 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/101/16' 2024-03-09T08:57:50.640 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/101/14' 2024-03-09T08:57:50.640 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/101/18' 2024-03-09T08:57:50.640 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/101/13' 2024-03-09T08:57:50.641 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/101/17' 2024-03-09T08:57:50.641 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/101' 2024-03-09T08:57:50.641 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/148/19' 2024-03-09T08:57:50.641 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/148/15' 2024-03-09T08:57:50.641 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/148/20' 2024-03-09T08:57:50.641 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/148/12' 2024-03-09T08:57:50.641 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/148/10' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/148/11' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/148/16' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/148/14' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/148/18' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/148/13' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/148/17' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/148' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/168/19' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/168/15' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/168/20' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/168/12' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/168/10' 2024-03-09T08:57:50.642 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/168/11' 2024-03-09T08:57:50.643 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/168/16' 2024-03-09T08:57:50.643 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/168/14' 2024-03-09T08:57:50.643 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/168/18' 2024-03-09T08:57:50.643 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/168/13' 2024-03-09T08:57:50.643 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/168/17' 2024-03-09T08:57:50.643 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/168' 2024-03-09T08:57:52.327 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/107' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/178/19' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/178/15' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/178/20' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/178/12' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/178/10' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/178/11' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/178/16' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/178/14' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/178/18' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/178/13' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/178/17' 2024-03-09T08:57:52.328 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/178' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/156/19' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/156/15' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/156/20' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/156/12' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/156/10' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/156/11' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/156/16' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/156/14' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/156/18' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/156/13' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/156/17' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/156' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/131/19' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/131/15' 2024-03-09T08:57:52.329 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/131/20' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/131/12' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/131/10' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/131/11' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/131/16' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/131/14' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/131/18' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/131/13' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/131/17' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/131' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/123/19' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/123/15' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/123/20' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/123/12' 2024-03-09T08:57:52.330 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/123/10' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/123/11' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/123/16' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/123/14' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/123/18' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/123/13' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/123/17' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/123' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/160/19' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/160/15' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/160/20' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/160/12' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/160/10' 2024-03-09T08:57:52.331 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/160/11' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/160/16' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/160/14' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/160/18' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/160/13' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/160/17' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/160' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/165/19' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/165/15' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/165/20' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/165/12' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/165/10' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/165/11' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/165/16' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/165/14' 2024-03-09T08:57:52.332 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/165/18' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/165/13' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/165/17' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/165' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/101/19' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/101/15' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/101/20' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/101/12' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/101/10' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/101/11' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/101/16' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/101/14' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/101/18' 2024-03-09T08:57:52.333 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/101/13' 2024-03-09T08:57:52.334 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/101/17' 2024-03-09T08:57:52.334 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/101' 2024-03-09T08:57:52.334 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/148/19' 2024-03-09T08:57:52.334 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/148/15' 2024-03-09T08:57:52.334 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/148/20' 2024-03-09T08:57:52.334 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/148/12' 2024-03-09T08:57:52.334 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/148/10' 2024-03-09T08:57:52.334 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/148/11' 2024-03-09T08:57:52.334 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/148/16' 2024-03-09T08:57:52.334 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/148/14' 2024-03-09T08:57:52.335 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/148/18' 2024-03-09T08:57:52.335 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/148/13' 2024-03-09T08:57:52.335 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/148/17' 2024-03-09T08:57:52.335 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/148' 2024-03-09T08:57:52.335 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/168/19' 2024-03-09T08:57:52.335 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/168/15' 2024-03-09T08:57:52.335 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/168/20' 2024-03-09T08:57:52.335 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/168/12' 2024-03-09T08:57:52.335 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/168/10' 2024-03-09T08:57:52.335 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/168/11' 2024-03-09T08:57:52.336 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/168/16' 2024-03-09T08:57:52.336 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/168/14' 2024-03-09T08:57:52.336 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/168/18' 2024-03-09T08:57:52.336 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/168/13' 2024-03-09T08:57:52.336 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/168/17' 2024-03-09T08:57:52.336 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/168' 2024-03-09T08:57:52.766 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/179/19' 2024-03-09T08:57:52.766 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/179/15' 2024-03-09T08:57:52.766 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/179/20' 2024-03-09T08:57:52.766 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/179/12' 2024-03-09T08:57:52.766 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/179/10' 2024-03-09T08:57:52.766 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/179/11' 2024-03-09T08:57:52.766 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/179/16' 2024-03-09T08:57:52.766 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/179/14' 2024-03-09T08:57:52.766 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/179/18' 2024-03-09T08:57:52.766 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/179/13' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/179/17' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/179' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/132/19' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/132/15' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/132/20' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/132/12' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/132/10' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/132/11' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/132/16' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/132/14' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/132/18' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/132/13' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/132/17' 2024-03-09T08:57:52.767 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/132' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/199/19' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/199/15' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/199/20' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/199/12' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/199/10' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/199/11' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/199/16' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/199/14' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/199/18' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/199/13' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/199/17' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/199' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/163/19' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/163/15' 2024-03-09T08:57:52.768 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/163/20' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/163/12' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/163/10' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/163/11' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/163/16' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/163/14' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/163/18' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/163/13' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/163/17' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/163' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/153/19' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/153/15' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/153/20' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/153/12' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/153/10' 2024-03-09T08:57:52.769 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/153/11' 2024-03-09T08:57:52.770 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/153/16' 2024-03-09T08:57:52.770 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/153/14' 2024-03-09T08:57:52.770 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/153/18' 2024-03-09T08:57:52.770 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/153/13' 2024-03-09T08:57:52.770 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/153/17' 2024-03-09T08:57:52.770 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/153' 2024-03-09T08:57:52.770 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/122/19' 2024-03-09T08:57:52.770 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/122/15' 2024-03-09T08:57:52.770 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/122/20' 2024-03-09T08:57:52.770 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/122/12' 2024-03-09T08:57:52.770 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/122/10' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/122/11' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/122/16' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/122/14' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/122/18' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/122/13' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/122/17' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/122' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/162/19' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/162/15' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/162/20' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/162/12' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/162/10' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/162/11' 2024-03-09T08:57:52.771 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/162/16' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/162/14' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/162/18' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/162/13' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/162/17' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/162' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/197/19' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/197/15' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/197/20' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/197/12' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/197/10' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/197/11' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/197/16' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/197/14' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/197/18' 2024-03-09T08:57:52.772 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/197/13' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/197/17' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/197' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/113/19' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/113/15' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/113/20' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/113/12' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/113/10' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/113/11' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/113/16' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/113/14' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/113/18' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/113/13' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/113/17' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/113' 2024-03-09T08:57:52.773 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/152/19' 2024-03-09T08:57:53.290 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:57:53.290 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:57:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.504+0000 7f390d800640 1 Processor -- start 2024-03-09T08:57:53.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.504+0000 7f390d800640 1 -- start start 2024-03-09T08:57:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.506+0000 7f390d800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3908071190 0x7f3908157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.506+0000 7f390d800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3908153d40 con 0x7f39080c22a0 2024-03-09T08:57:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.506+0000 7f390d800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39080c2790 con 0x7f39080ac730 2024-03-09T08:57:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.506+0000 7f390d800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3908070ee0 con 0x7f3908071190 2024-03-09T08:57:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.506+0000 7f38fe000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f39080c22a0 0x7f390815c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60454/0 (socket says 172.21.15.67:60454) 2024-03-09T08:57:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.506+0000 7f38fe000640 1 -- 172.21.15.67:0/121609972 learned_addr learned my addr 172.21.15.67:0/121609972 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.506+0000 7f3907400640 1 --2- 172.21.15.67:0/121609972 >> v2:172.21.15.67:3300/0 conn(0x7f3908071190 0x7f3908157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.507+0000 7f3906000640 1 -- 172.21.15.67:0/121609972 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594820849 0 0) 0x7f3908153d40 con 0x7f39080c22a0 2024-03-09T08:57:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.507+0000 7f3906000640 1 -- 172.21.15.67:0/121609972 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38ec003520 con 0x7f39080c22a0 2024-03-09T08:57:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.507+0000 7f3907400640 1 -- 172.21.15.67:0/121609972 >> v1:172.21.15.134:6789/0 conn(0x7f39080ac730 legacy=0x7f3908157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.507+0000 7f3907400640 1 -- 172.21.15.67:0/121609972 >> v1:172.21.15.67:6789/0 conn(0x7f39080c22a0 legacy=0x7f390815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.507+0000 7f3907400640 1 -- 172.21.15.67:0/121609972 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3908070920 con 0x7f3908071190 2024-03-09T08:57:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.508+0000 7f3907400640 1 --2- 172.21.15.67:0/121609972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3908071190 0x7f3908157d60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f39000097a0 tx=0x7f390002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbe3c07c5b73d97 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.508+0000 7f3906000640 1 -- 172.21.15.67:0/121609972 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39000315a0 con 0x7f3908071190 2024-03-09T08:57:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.508+0000 7f3906000640 1 -- 172.21.15.67:0/121609972 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900037070 con 0x7f3908071190 2024-03-09T08:57:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.508+0000 7f3906000640 1 -- 172.21.15.67:0/121609972 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900031940 con 0x7f3908071190 2024-03-09T08:57:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.509+0000 7f390d800640 1 -- 172.21.15.67:0/121609972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3908071190 msgr2=0x7f3908157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.509+0000 7f390d800640 1 --2- 172.21.15.67:0/121609972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3908071190 0x7f3908157d60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f39000097a0 tx=0x7f390002b840 comp rx=0 tx=0).stop 2024-03-09T08:57:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.509+0000 7f390d800640 1 -- 172.21.15.67:0/121609972 shutdown_connections 2024-03-09T08:57:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.509+0000 7f390d800640 1 --2- 172.21.15.67:0/121609972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3908071190 0x7f3908157d60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.509+0000 7f390d800640 1 -- 172.21.15.67:0/121609972 >> 172.21.15.67:0/121609972 conn(0x7f39080c7370 msgr2=0x7f39080c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.509+0000 7f390d800640 1 -- 172.21.15.67:0/121609972 shutdown_connections 2024-03-09T08:57:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.509+0000 7f390d800640 1 -- 172.21.15.67:0/121609972 wait complete. 2024-03-09T08:57:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.511+0000 7f390d800640 1 Processor -- start 2024-03-09T08:57:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.511+0000 7f390d800640 1 -- start start 2024-03-09T08:57:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.512+0000 7f390d800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3908071190 0x7f39081e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.512+0000 7f390d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39080ac730 0x7f39081eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.512+0000 7f390d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39080c22a0 0x7f39080c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.512+0000 7f390d800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3908151610 con 0x7f39080c22a0 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.512+0000 7f390d800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3908154d00 con 0x7f39080ac730 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.512+0000 7f390d800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3908151790 con 0x7f3908071190 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.512+0000 7f3907400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39080c22a0 0x7f39080c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.512+0000 7f3907400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39080c22a0 0x7f39080c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40208/0 (socket says 172.21.15.67:40208) 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.512+0000 7f3907400640 1 -- 172.21.15.67:0/2737485612 learned_addr learned my addr 172.21.15.67:0/2737485612 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.512+0000 7f38fe000640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39080ac730 0x7f39081eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.512+0000 7f3906a00640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3908071190 0x7f39081e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.513+0000 7f3907400640 1 -- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3908071190 msgr2=0x7f39081e1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.513+0000 7f3907400640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3908071190 0x7f39081e1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.513+0000 7f3907400640 1 -- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39080ac730 msgr2=0x7f39081eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.513+0000 7f3907400640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39080ac730 0x7f39081eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:53.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.513+0000 7f3907400640 1 -- 172.21.15.67:0/2737485612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39081e23f0 con 0x7f39080c22a0 2024-03-09T08:57:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.513+0000 7f3907400640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39080c22a0 0x7f39080c6a10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f390002b880 tx=0x7f39000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.513+0000 7f38ff400640 1 -- 172.21.15.67:0/2737485612 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900032630 con 0x7f39080c22a0 2024-03-09T08:57:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.513+0000 7f38ff400640 1 -- 172.21.15.67:0/2737485612 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39000327d0 con 0x7f39080c22a0 2024-03-09T08:57:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.513+0000 7f38ff400640 1 -- 172.21.15.67:0/2737485612 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900032bc0 con 0x7f39080c22a0 2024-03-09T08:57:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.513+0000 7f390d800640 1 -- 172.21.15.67:0/2737485612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39081dd6a0 con 0x7f39080c22a0 2024-03-09T08:57:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.513+0000 7f390d800640 1 -- 172.21.15.67:0/2737485612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39081e16d0 con 0x7f39080c22a0 2024-03-09T08:57:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.515+0000 7f390d800640 1 -- 172.21.15.67:0/2737485612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3908153050 con 0x7f39080c22a0 2024-03-09T08:57:53.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.516+0000 7f38ff400640 1 -- 172.21.15.67:0/2737485612 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f390004c020 con 0x7f39080c22a0 2024-03-09T08:57:53.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.517+0000 7f38ff400640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38e406b520 0x7f38e406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:53.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.517+0000 7f38ff400640 1 -- 172.21.15.67:0/2737485612 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f390002f020 con 0x7f39080c22a0 2024-03-09T08:57:53.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.518+0000 7f3906a00640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38e406b520 0x7f38e406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:53.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.518+0000 7f3906a00640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38e406b520 0x7f38e406d9e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38f8005520 tx=0x7f38f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:53.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.522+0000 7f38ff400640 1 -- 172.21.15.67:0/2737485612 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3908153050 con 0x7f39080c22a0 2024-03-09T08:57:53.661 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/179/19' 2024-03-09T08:57:53.661 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/179/15' 2024-03-09T08:57:53.661 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/179/20' 2024-03-09T08:57:53.661 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/179/12' 2024-03-09T08:57:53.661 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/179/10' 2024-03-09T08:57:53.661 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/179/11' 2024-03-09T08:57:53.661 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/179/16' 2024-03-09T08:57:53.661 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/179/14' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/179/18' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/179/13' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/179/17' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/179' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/132/19' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/132/15' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/132/20' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/132/12' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/132/10' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/132/11' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/132/16' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/132/14' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/132/18' 2024-03-09T08:57:53.662 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/132/13' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/132/17' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/132' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/199/19' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/199/15' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/199/20' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/199/12' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/199/10' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/199/11' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/199/16' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/199/14' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/199/18' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/199/13' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/199/17' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/199' 2024-03-09T08:57:53.663 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/163/19' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/163/15' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/163/20' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/163/12' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/163/10' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/163/11' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/163/16' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/163/14' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/163/18' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/163/13' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/163/17' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/163' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/153/19' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/153/15' 2024-03-09T08:57:53.664 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/153/20' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/153/12' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/153/10' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/153/11' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/153/16' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/153/14' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/153/18' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/153/13' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/153/17' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/153' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/122/19' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/122/15' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/122/20' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/122/12' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/122/10' 2024-03-09T08:57:53.665 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/122/11' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/122/16' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/122/14' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/122/18' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/122/13' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/122/17' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/122' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/162/19' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/162/15' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/162/20' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/162/12' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/162/10' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/162/11' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/162/16' 2024-03-09T08:57:53.666 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/162/14' 2024-03-09T08:57:53.667 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/162/18' 2024-03-09T08:57:53.667 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/162/13' 2024-03-09T08:57:53.667 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/162/17' 2024-03-09T08:57:53.667 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/162' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/197/19' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/197/15' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/197/20' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/197/12' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/197/10' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/197/11' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/197/16' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/197/14' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/197/18' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/197/13' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/197/17' 2024-03-09T08:57:53.668 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/197' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/113/19' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/113/15' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/113/20' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/113/12' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/113/10' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/113/11' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/113/16' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/113/14' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/113/18' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/113/13' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/113/17' 2024-03-09T08:57:53.669 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/113' 2024-03-09T08:57:53.670 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/152/19' 2024-03-09T08:57:53.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.815+0000 7f390d800640 1 -- 172.21.15.67:0/2737485612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3908075a00 con 0x7f39080c22a0 2024-03-09T08:57:53.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.816+0000 7f38ff400640 1 -- 172.21.15.67:0/2737485612 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f3900040070 con 0x7f39080c22a0 2024-03-09T08:57:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":42,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:57:53.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 -- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38e406b520 msgr2=0x7f38e406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:53.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38e406b520 0x7f38e406d9e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38f8005520 tx=0x7f38f8009040 comp rx=0 tx=0).stop 2024-03-09T08:57:53.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 -- 172.21.15.67:0/2737485612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39080c22a0 msgr2=0x7f39080c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:53.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39080c22a0 0x7f39080c6a10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f390002b880 tx=0x7f39000046c0 comp rx=0 tx=0).stop 2024-03-09T08:57:53.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 -- 172.21.15.67:0/2737485612 shutdown_connections 2024-03-09T08:57:53.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f38e406b520 0x7f38e406d9e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:53.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39080c22a0 0x7f39080c6a10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:53.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39080ac730 0x7f39081eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:53.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 --2- 172.21.15.67:0/2737485612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3908071190 0x7f39081e1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:53.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 -- 172.21.15.67:0/2737485612 >> 172.21.15.67:0/2737485612 conn(0x7f39080c7370 msgr2=0x7f39080c4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:53.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 -- 172.21.15.67:0/2737485612 shutdown_connections 2024-03-09T08:57:53.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:53.819+0000 7f390d800640 1 -- 172.21.15.67:0/2737485612 wait complete. 2024-03-09T08:57:53.839 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:57:53.839 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T08:57:53.839 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:57:53.839 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:57:54.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.044+0000 7f72d1c00640 1 Processor -- start 2024-03-09T08:57:54.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.044+0000 7f72d1c00640 1 -- start start 2024-03-09T08:57:54.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.046+0000 7f72d1c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72cc06c1d0 0x7f72cc06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.046+0000 7f72d1c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc0c9890 con 0x7f72cc14fd90 2024-03-09T08:57:54.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.046+0000 7f72d1c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc0c5f80 con 0x7f72cc0c9380 2024-03-09T08:57:54.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.046+0000 7f72d1c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc06ba60 con 0x7f72cc06c1d0 2024-03-09T08:57:54.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.046+0000 7f72cbe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f72cc14fd90 0x7f72cc156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60466/0 (socket says 172.21.15.67:60466) 2024-03-09T08:57:54.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.046+0000 7f72cbe00640 1 -- 172.21.15.67:0/1259212922 learned_addr learned my addr 172.21.15.67:0/1259212922 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:54.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.047+0000 7f72d1200640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72cc06c1d0 0x7f72cc06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.047+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004358275 0 0) 0x7f72cc0c9890 con 0x7f72cc14fd90 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.047+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72b0003520 con 0x7f72cc14fd90 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.047+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693520801 0 0) 0x7f72cc0c5f80 con 0x7f72cc0c9380 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72cc0c9890 con 0x7f72cc0c9380 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1203281612 0 0) 0x7f72b0003520 con 0x7f72cc14fd90 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72cc06c1d0 msgr2=0x7f72cc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72cc06c1d0 0x7f72cc06c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 >> v1:172.21.15.134:6789/0 conn(0x7f72cc0c9380 legacy=0x7f72cc0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0c6240 con 0x7f72cc14fd90 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f72bc001670 con 0x7f72cc14fd90 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 >> v1:172.21.15.67:6789/0 conn(0x7f72cc14fd90 legacy=0x7f72cc156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b0003be0 0x7f72b0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72b0004800 0x7f72b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72b0005230 0x7f72b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc06ba60 con 0x7f72b0005230 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b000c140 con 0x7f72b0004800 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b000c340 con 0x7f72b0003be0 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72d1200640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72cc06c1d0 0x7f72cc06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.048+0000 7f72d1200640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72b0005230 0x7f72b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.049+0000 7f72cbe00640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b0003be0 0x7f72b0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.049+0000 7f72cb400640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72b0004800 0x7f72b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.049+0000 7f72d1200640 1 -- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b0003be0 msgr2=0x7f72b0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.049+0000 7f72d1200640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b0003be0 0x7f72b0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.049+0000 7f72d1200640 1 -- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72b0004800 msgr2=0x7f72b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.049+0000 7f72d1200640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72b0004800 0x7f72b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.049+0000 7f72d1200640 1 -- 172.21.15.67:0/1259212922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b0003520 con 0x7f72b0005230 2024-03-09T08:57:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.050+0000 7f72d1200640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72b0005230 0x7f72b000b9c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e1f0 tx=0x7f72b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab81554ec2106a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.051+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4009d40 con 0x7f72b0005230 2024-03-09T08:57:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.051+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c402cce0 con 0x7f72b0005230 2024-03-09T08:57:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.051+0000 7f72caa00640 1 -- 172.21.15.67:0/1259212922 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4003bb0 con 0x7f72b0005230 2024-03-09T08:57:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.051+0000 7f72d1c00640 1 -- 172.21.15.67:0/1259212922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72b0005230 msgr2=0x7f72b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.051+0000 7f72d1c00640 1 --2- 172.21.15.67:0/1259212922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72b0005230 0x7f72b000b9c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e1f0 tx=0x7f72b0005830 comp rx=0 tx=0).stop 2024-03-09T08:57:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.051+0000 7f72d1200640 1 -- 172.21.15.67:0/1259212922 reap_dead start 2024-03-09T08:57:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.051+0000 7f72d1c00640 1 -- 172.21.15.67:0/1259212922 shutdown_connections 2024-03-09T08:57:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.051+0000 7f72d1c00640 1 -- 172.21.15.67:0/1259212922 >> 172.21.15.67:0/1259212922 conn(0x7f72cc0cb480 msgr2=0x7f72cc0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.051+0000 7f72d1c00640 1 -- 172.21.15.67:0/1259212922 shutdown_connections 2024-03-09T08:57:54.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.052+0000 7f72d1c00640 1 -- 172.21.15.67:0/1259212922 wait complete. 2024-03-09T08:57:54.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.053+0000 7f72d1c00640 1 Processor -- start 2024-03-09T08:57:54.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.053+0000 7f72d1c00640 1 -- start start 2024-03-09T08:57:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.054+0000 7f72d1c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b0005230 0x7f72cc0b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.054+0000 7f72d1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72cc14cbf0 0x7f72cc14c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.054+0000 7f72d1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72cc14c790 0x7f72cc1e2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.054+0000 7f72d1c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0ad610 con 0x7f72cc14c790 2024-03-09T08:57:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.054+0000 7f72d1c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc06bcb0 con 0x7f72cc14cbf0 2024-03-09T08:57:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.054+0000 7f72d1c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0ad790 con 0x7f72b0005230 2024-03-09T08:57:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.055+0000 7f72d1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72cc14c790 0x7f72cc1e2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.055+0000 7f72d1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72cc14c790 0x7f72cc1e2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40232/0 (socket says 172.21.15.67:40232) 2024-03-09T08:57:54.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.055+0000 7f72d1200640 1 -- 172.21.15.67:0/1948209432 learned_addr learned my addr 172.21.15.67:0/1948209432 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.055+0000 7f72cbe00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b0005230 0x7f72cc0b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.055+0000 7f72cb400640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72cc14cbf0 0x7f72cc14c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.055+0000 7f72d1200640 1 -- 172.21.15.67:0/1948209432 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b0005230 msgr2=0x7f72cc0b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.055+0000 7f72d1200640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b0005230 0x7f72cc0b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.055+0000 7f72d1200640 1 -- 172.21.15.67:0/1948209432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72cc14cbf0 msgr2=0x7f72cc14c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.055+0000 7f72d1200640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72cc14cbf0 0x7f72cc14c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.055+0000 7f72d1200640 1 -- 172.21.15.67:0/1948209432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc158d00 con 0x7f72cc14c790 2024-03-09T08:57:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.056+0000 7f72d1200640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72cc14c790 0x7f72cc1e2bb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e310 tx=0x7f72c4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.056+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4009d40 con 0x7f72cc14c790 2024-03-09T08:57:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.056+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c4045070 con 0x7f72cc14c790 2024-03-09T08:57:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.056+0000 7f72d1c00640 1 -- 172.21.15.67:0/1948209432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc1ef110 con 0x7f72cc14c790 2024-03-09T08:57:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.056+0000 7f72d1c00640 1 -- 172.21.15.67:0/1948209432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc1ef690 con 0x7f72cc14c790 2024-03-09T08:57:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.056+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4003e10 con 0x7f72cc14c790 2024-03-09T08:57:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.057+0000 7f72d1c00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4002a70 0x7f72a4004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.057+0000 7f72d1c00640 1 -- 172.21.15.67:0/1948209432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a4007ec0 con 0x7f72a4002a70 2024-03-09T08:57:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.058+0000 7f72cbe00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4002a70 0x7f72a4004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.058+0000 7f72cbe00640 1 -- 172.21.15.67:0/1948209432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72a40080c0 con 0x7f72a4002a70 2024-03-09T08:57:54.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.059+0000 7f72cbe00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4002a70 0x7f72a4004f30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f72bc000c00 tx=0x7f72bc010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.059+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc012790 con 0x7f72a4002a70 2024-03-09T08:57:54.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.059+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f72c403d040 con 0x7f72cc14c790 2024-03-09T08:57:54.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.060+0000 7f72c3e00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f72ac06b3d0 0x7f72ac06d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.060+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f72c40acc60 con 0x7f72cc14c790 2024-03-09T08:57:54.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.060+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f72bc019070 con 0x7f72a4002a70 2024-03-09T08:57:54.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.060+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4002a70 msgr2=0x7f72a4004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.060+0000 7f72c3e00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4002a70 0x7f72a4004f30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f72bc000c00 tx=0x7f72bc010910 comp rx=0 tx=0).stop 2024-03-09T08:57:54.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.061+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4002a70 msgr2=0x7f72a4004f30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:54.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.061+0000 7f72c3e00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4002a70 0x7f72a4004f30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.061+0000 7f72cb400640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f72ac06b3d0 0x7f72ac06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.062+0000 7f72cb400640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f72ac06b3d0 0x7f72ac06d890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f72cc14d030 tx=0x7f72b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.069+0000 7f72d1c00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4006f70 0x7f72a4007400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.069+0000 7f72d1c00640 1 -- 172.21.15.67:0/1948209432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a400a4a0 con 0x7f72a4006f70 2024-03-09T08:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.069+0000 7f72cbe00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4006f70 0x7f72a4007400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.069+0000 7f72cbe00640 1 -- 172.21.15.67:0/1948209432 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f72a4007940 con 0x7f72a4006f70 2024-03-09T08:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.070+0000 7f72cbe00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4006f70 0x7f72a4007400 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f72bc018000 tx=0x7f72cc155c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.070+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc01d6e0 con 0x7f72a4006f70 2024-03-09T08:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.070+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f72bc01d6e0 con 0x7f72a4006f70 2024-03-09T08:57:54.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.070+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4006f70 msgr2=0x7f72a4007400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.070+0000 7f72c3e00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4006f70 0x7f72a4007400 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f72bc018000 tx=0x7f72cc155c00 comp rx=0 tx=0).stop 2024-03-09T08:57:54.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.070+0000 7f72c3e00640 1 -- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4006f70 msgr2=0x7f72a4007400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:54.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.070+0000 7f72c3e00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72a4006f70 0x7f72a4007400 secure :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f72bc018000 tx=0x7f72cc155c00 comp rx=0 tx=0).stop 2024-03-09T08:57:54.071 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:57:54.071 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:54.071 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:54.071 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:57:54.071 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 42, 2024-03-09T08:57:54.071 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:54.072 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:54.073 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:57:54.074 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:54.075 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:54.076 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:57:54.077 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:57:54.078 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:54.078 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:54.078 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:54.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.078 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:54.078 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:54.079 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:54.079 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:54.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.079 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:54.079 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:57:54.079 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:54.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.081 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:54.081 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:54.081 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:54.081 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:54.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.081 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:54.081 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:57:54.081 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.082 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.083 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:57:54.084 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.084 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.084 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:57:54.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.084 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:54.084 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.084 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T08:57:54.085 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.085 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.085 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:57:54.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.085 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:57:54.085 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.085 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:57:54.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.085 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:54.085 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:57:54.086 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:57:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.071+0000 7f72d1c00640 1 -- 172.21.15.67:0/1948209432 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f72ac06b3d0 msgr2=0x7f72ac06d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.071+0000 7f72d1c00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f72ac06b3d0 0x7f72ac06d890 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f72cc14d030 tx=0x7f72b8009040 comp rx=0 tx=0).stop 2024-03-09T08:57:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.071+0000 7f72d1c00640 1 -- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72cc14c790 msgr2=0x7f72cc1e2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.071+0000 7f72d1c00640 1 --2- 172.21.15.67:0/1948209432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72cc14c790 0x7f72cc1e2bb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e310 tx=0x7f72c4009750 comp rx=0 tx=0).stop 2024-03-09T08:57:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.071+0000 7f72d1200640 1 -- 172.21.15.67:0/1948209432 reap_dead start 2024-03-09T08:57:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.071+0000 7f72d1200640 1 -- 172.21.15.67:0/1948209432 reap_dead start 2024-03-09T08:57:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.071+0000 7f72d1c00640 1 -- 172.21.15.67:0/1948209432 shutdown_connections 2024-03-09T08:57:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.071+0000 7f72d1c00640 1 -- 172.21.15.67:0/1948209432 >> 172.21.15.67:0/1948209432 conn(0x7f72cc0cb480 msgr2=0x7f72cc1553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.071+0000 7f72d1c00640 1 -- 172.21.15.67:0/1948209432 shutdown_connections 2024-03-09T08:57:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.071+0000 7f72d1c00640 1 -- 172.21.15.67:0/1948209432 wait complete. 2024-03-09T08:57:54.089 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:57:54.089 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:57:54.089 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/152/15' 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/152/20' 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/152/12' 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/152/10' 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/152/11' 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/152/16' 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/152/14' 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/152/18' 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/152/13' 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/152/17' 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/152' 2024-03-09T08:57:54.141 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/121/19' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/121/15' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/121/20' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/121/12' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/121/10' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/121/11' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/121/16' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/121/14' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/121/18' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/121/13' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/121/17' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/121' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/118/19' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/118/15' 2024-03-09T08:57:54.142 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/118/20' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/118/12' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/118/10' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/118/11' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/118/16' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/118/14' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/118/18' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/118/13' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/118/17' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/118' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/196/19' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/196/15' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/196/20' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/196/12' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/196/10' 2024-03-09T08:57:54.143 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/196/11' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/196/16' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/196/14' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/196/18' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/196/13' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/196/17' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/196' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/124/19' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/124/15' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/124/20' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/124/12' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/124/10' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/124/11' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/124/16' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/124/14' 2024-03-09T08:57:54.144 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/124/18' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/124/13' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/124/17' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/124' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/188/19' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/188/15' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/188/20' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/188/12' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/188/10' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/188/11' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/188/16' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/188/14' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/188/18' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/188/13' 2024-03-09T08:57:54.145 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/188/17' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/188' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/145/19' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/145/15' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/145/20' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/145/12' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/145/10' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/145/11' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/145/16' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/145/14' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/145/18' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/145/13' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/145/17' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/145' 2024-03-09T08:57:54.146 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/130/19' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/130/15' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/130/20' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/130/12' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/130/10' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/130/11' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/130/16' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/130/14' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/130/18' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/130/13' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/130/17' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/130' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/193/19' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/193/15' 2024-03-09T08:57:54.147 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/193/20' 2024-03-09T08:57:54.148 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/193/12' 2024-03-09T08:57:54.148 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/193/10' 2024-03-09T08:57:54.148 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/193/11' 2024-03-09T08:57:54.148 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/193/16' 2024-03-09T08:57:54.148 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/193/14' 2024-03-09T08:57:54.148 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/193/18' 2024-03-09T08:57:54.148 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/193/13' 2024-03-09T08:57:54.148 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/193/17' 2024-03-09T08:57:54.148 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/193' 2024-03-09T08:57:54.148 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/175/19' 2024-03-09T08:57:54.148 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/175/15' 2024-03-09T08:57:54.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.303+0000 7f44ffe00640 1 Processor -- start 2024-03-09T08:57:54.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.303+0000 7f44ffe00640 1 -- start start 2024-03-09T08:57:54.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.305+0000 7f44ffe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000ad8d0 0x7f4500159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.305+0000 7f44ffe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f450006b880 0x7f4500153080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.305+0000 7f44ffe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450006c040 con 0x7f45000c6650 2024-03-09T08:57:54.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.305+0000 7f44ffe00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45000c61f0 con 0x7f450006b880 2024-03-09T08:57:54.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.305+0000 7f44ffe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450006bd60 con 0x7f45000ad8d0 2024-03-09T08:57:54.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.305+0000 7f44ff400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f45000c6650 0x7f4500155e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60470/0 (socket says 172.21.15.67:60470) 2024-03-09T08:57:54.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.305+0000 7f44ff400640 1 -- 172.21.15.67:0/3354624089 learned_addr learned my addr 172.21.15.67:0/3354624089 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:54.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.305+0000 7f44f7e00640 1 --2- 172.21.15.67:0/3354624089 >> v2:172.21.15.134:3300/0 conn(0x7f450006b880 0x7f4500153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.305+0000 7f44fea00640 1 --2- 172.21.15.67:0/3354624089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000ad8d0 0x7f4500159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.306+0000 7f44fe000640 1 -- 172.21.15.67:0/3354624089 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073490323 0 0) 0x7f450006c040 con 0x7f45000c6650 2024-03-09T08:57:54.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.306+0000 7f44fe000640 1 -- 172.21.15.67:0/3354624089 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e4003520 con 0x7f45000c6650 2024-03-09T08:57:54.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.306+0000 7f44f7e00640 1 -- 172.21.15.67:0/3354624089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000ad8d0 msgr2=0x7f4500159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:54.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.306+0000 7f44fe000640 1 -- 172.21.15.67:0/3354624089 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3193947604 0 0) 0x7f44e4003520 con 0x7f45000c6650 2024-03-09T08:57:54.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.306+0000 7f44f7e00640 1 --2- 172.21.15.67:0/3354624089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000ad8d0 0x7f4500159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.306+0000 7f44f7e00640 1 -- 172.21.15.67:0/3354624089 >> v1:172.21.15.67:6789/0 conn(0x7f45000c6650 legacy=0x7f4500155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.306+0000 7f44f7e00640 1 -- 172.21.15.67:0/3354624089 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45000c5e80 con 0x7f450006b880 2024-03-09T08:57:54.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.307+0000 7f44f7e00640 1 --2- 172.21.15.67:0/3354624089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f450006b880 0x7f4500153080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44e80097a0 tx=0x7f44e802b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f8db5a9e301464 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.308+0000 7f44fe000640 1 -- 172.21.15.67:0/3354624089 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8037070 con 0x7f450006b880 2024-03-09T08:57:54.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.308+0000 7f44fe000640 1 -- 172.21.15.67:0/3354624089 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e8030510 con 0x7f450006b880 2024-03-09T08:57:54.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.308+0000 7f44fe000640 1 -- 172.21.15.67:0/3354624089 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f80019f0 con 0x7f450006b880 2024-03-09T08:57:54.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.309+0000 7f44ffe00640 1 -- 172.21.15.67:0/3354624089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f450006b880 msgr2=0x7f4500153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.309+0000 7f44ffe00640 1 --2- 172.21.15.67:0/3354624089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f450006b880 0x7f4500153080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44e80097a0 tx=0x7f44e802b710 comp rx=0 tx=0).stop 2024-03-09T08:57:54.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.309+0000 7f44ffe00640 1 -- 172.21.15.67:0/3354624089 shutdown_connections 2024-03-09T08:57:54.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.309+0000 7f44ffe00640 1 --2- 172.21.15.67:0/3354624089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f450006b880 0x7f4500153080 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.309+0000 7f44ffe00640 1 --2- 172.21.15.67:0/3354624089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000ad8d0 0x7f4500159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.309+0000 7f44ffe00640 1 -- 172.21.15.67:0/3354624089 >> 172.21.15.67:0/3354624089 conn(0x7f45000c8510 msgr2=0x7f45000acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:54.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.309+0000 7f44ffe00640 1 -- 172.21.15.67:0/3354624089 shutdown_connections 2024-03-09T08:57:54.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.309+0000 7f44ffe00640 1 -- 172.21.15.67:0/3354624089 wait complete. 2024-03-09T08:57:54.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.312+0000 7f44ffe00640 1 Processor -- start 2024-03-09T08:57:54.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.312+0000 7f44ffe00640 1 -- start start 2024-03-09T08:57:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.313+0000 7f44ffe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f450006b880 0x7f4500159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.313+0000 7f44ffe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45000ad8d0 0x7f4500157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.313+0000 7f44ffe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000c6650 0x7f4500157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44ffe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45001535c0 con 0x7f45000ad8d0 2024-03-09T08:57:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44ffe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45000adec0 con 0x7f450006b880 2024-03-09T08:57:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44ffe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4500153740 con 0x7f45000c6650 2024-03-09T08:57:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44f7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45000ad8d0 0x7f4500157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44f7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45000ad8d0 0x7f4500157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40260/0 (socket says 172.21.15.67:40260) 2024-03-09T08:57:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44f7e00640 1 -- 172.21.15.67:0/1218100501 learned_addr learned my addr 172.21.15.67:0/1218100501 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44ff400640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000c6650 0x7f4500157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44fea00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f450006b880 0x7f4500159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44f7e00640 1 -- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000c6650 msgr2=0x7f4500157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44f7e00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45000c6650 0x7f4500157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44f7e00640 1 -- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f450006b880 msgr2=0x7f4500159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44f7e00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f450006b880 0x7f4500159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.314+0000 7f44f7e00640 1 -- 172.21.15.67:0/1218100501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45001eb7b0 con 0x7f45000ad8d0 2024-03-09T08:57:54.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.315+0000 7f44f7e00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45000ad8d0 0x7f4500157ee0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f44e802b750 tx=0x7f44e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.315+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8040020 con 0x7f45000ad8d0 2024-03-09T08:57:54.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.315+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e8038040 con 0x7f45000ad8d0 2024-03-09T08:57:54.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.315+0000 7f44ffe00640 1 -- 172.21.15.67:0/1218100501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4500155ad0 con 0x7f45000ad8d0 2024-03-09T08:57:54.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.315+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e8037070 con 0x7f45000ad8d0 2024-03-09T08:57:54.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.315+0000 7f44ffe00640 1 -- 172.21.15.67:0/1218100501 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45001ebd50 con 0x7f45000ad8d0 2024-03-09T08:57:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.316+0000 7f44ffe00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45000b0ea0 0x7f4500157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.316+0000 7f44ffe00640 1 -- 172.21.15.67:0/1218100501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4500157ac0 con 0x7f45000b0ea0 2024-03-09T08:57:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.317+0000 7f44fea00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45000b0ea0 0x7f4500157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.317+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f44e8043020 con 0x7f45000ad8d0 2024-03-09T08:57:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.318+0000 7f44fea00640 1 -- 172.21.15.67:0/1218100501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f450004f3c0 con 0x7f45000b0ea0 2024-03-09T08:57:54.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.318+0000 7f44f6a00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f44dc06b4d0 0x7f44dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.318+0000 7f44fea00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45000b0ea0 0x7f4500157580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44f0010750 tx=0x7f44f0010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.318+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f44e80aba60 con 0x7f45000ad8d0 2024-03-09T08:57:54.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.319+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f000f040 con 0x7f45000b0ea0 2024-03-09T08:57:54.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.319+0000 7f44ff400640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f44dc06b4d0 0x7f44dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.320+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f44f0010e60 con 0x7f45000b0ea0 2024-03-09T08:57:54.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.320+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45000b0ea0 msgr2=0x7f4500157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.320+0000 7f44f6a00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45000b0ea0 0x7f4500157580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44f0010750 tx=0x7f44f0010c10 comp rx=0 tx=0).stop 2024-03-09T08:57:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.320+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45000b0ea0 msgr2=0x7f4500157580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.320+0000 7f44f6a00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45000b0ea0 0x7f4500157580 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44f0010750 tx=0x7f44f0010c10 comp rx=0 tx=0).stop 2024-03-09T08:57:54.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.320+0000 7f44ff400640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f44dc06b4d0 0x7f44dc06d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f44f8001b60 tx=0x7f44f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.327+0000 7f44ffe00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45001ebf90 0x7f45000ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.327+0000 7f44ffe00640 1 -- 172.21.15.67:0/1218100501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4500151ee0 con 0x7f45001ebf90 2024-03-09T08:57:54.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.328+0000 7f44fea00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45001ebf90 0x7f45000ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.328+0000 7f44fea00640 1 -- 172.21.15.67:0/1218100501 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f45001530c0 con 0x7f45001ebf90 2024-03-09T08:57:54.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.329+0000 7f44fea00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45001ebf90 0x7f45000ab7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f44f0010750 tx=0x7f44f00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.329+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0004250 con 0x7f45001ebf90 2024-03-09T08:57:54.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.330+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4874 (secure 0 0 0) 0x7f44f0025070 con 0x7f45001ebf90 2024-03-09T08:57:54.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.330+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45001ebf90 msgr2=0x7f45000ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.330+0000 7f44f6a00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45001ebf90 0x7f45000ab7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f44f0010750 tx=0x7f44f00046c0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.330+0000 7f44f6a00640 1 -- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45001ebf90 msgr2=0x7f45000ab7b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:54.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.330+0000 7f44f6a00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45001ebf90 0x7f45000ab7b0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f44f0010750 tx=0x7f44f00046c0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.331 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:57:54.331 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:54.331 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:54.331 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:57:54.331 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 42, 2024-03-09T08:57:54.331 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:57:54.331 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:57:54.331 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:54.332 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:54.333 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:54.334 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:54.335 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:54.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:57:54.337 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:54.339 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:57:54.339 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.339 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.339 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.339 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:57:54.339 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:57:54.339 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:54.339 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:54.339 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.340 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.342 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:54.343 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.344 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.344 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:57:54.344 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:57:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.331+0000 7f44ffe00640 1 -- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f44dc06b4d0 msgr2=0x7f44dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.331+0000 7f44ffe00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f44dc06b4d0 0x7f44dc06d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f44f8001b60 tx=0x7f44f8009040 comp rx=0 tx=0).stop 2024-03-09T08:57:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.331+0000 7f44ffe00640 1 -- 172.21.15.67:0/1218100501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45000ad8d0 msgr2=0x7f4500157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.331+0000 7f44ffe00640 1 --2- 172.21.15.67:0/1218100501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f45000ad8d0 0x7f4500157ee0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f44e802b750 tx=0x7f44e80046c0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.331+0000 7f44ff400640 1 -- 172.21.15.67:0/1218100501 reap_dead start 2024-03-09T08:57:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.331+0000 7f44ff400640 1 -- 172.21.15.67:0/1218100501 reap_dead start 2024-03-09T08:57:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.331+0000 7f44ffe00640 1 -- 172.21.15.67:0/1218100501 shutdown_connections 2024-03-09T08:57:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.331+0000 7f44ffe00640 1 -- 172.21.15.67:0/1218100501 >> 172.21.15.67:0/1218100501 conn(0x7f45000c8510 msgr2=0x7f4500072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.331+0000 7f44ffe00640 1 -- 172.21.15.67:0/1218100501 shutdown_connections 2024-03-09T08:57:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.331+0000 7f44ffe00640 1 -- 172.21.15.67:0/1218100501 wait complete. 2024-03-09T08:57:54.347 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:57:54.347 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:57:54.348 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:57:54.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/152/15' 2024-03-09T08:57:54.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/152/20' 2024-03-09T08:57:54.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/152/12' 2024-03-09T08:57:54.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/152/10' 2024-03-09T08:57:54.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/152/11' 2024-03-09T08:57:54.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/152/16' 2024-03-09T08:57:54.509 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/152/14' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/152/18' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/152/13' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/152/17' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/152' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/121/19' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/121/15' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/121/20' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/121/12' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/121/10' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/121/11' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/121/16' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/121/14' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/121/18' 2024-03-09T08:57:54.510 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/121/13' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/121/17' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/121' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/118/19' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/118/15' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/118/20' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/118/12' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/118/10' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/118/11' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/118/16' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/118/14' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/118/18' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/118/13' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/118/17' 2024-03-09T08:57:54.511 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/118' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/196/19' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/196/15' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/196/20' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/196/12' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/196/10' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/196/11' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/196/16' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/196/14' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/196/18' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/196/13' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/196/17' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/196' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/124/19' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/124/15' 2024-03-09T08:57:54.512 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/124/20' 2024-03-09T08:57:54.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/124/12' 2024-03-09T08:57:54.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/124/10' 2024-03-09T08:57:54.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/124/11' 2024-03-09T08:57:54.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/124/16' 2024-03-09T08:57:54.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/124/14' 2024-03-09T08:57:54.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/124/18' 2024-03-09T08:57:54.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/124/13' 2024-03-09T08:57:54.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/124/17' 2024-03-09T08:57:54.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/124' 2024-03-09T08:57:54.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/188/19' 2024-03-09T08:57:54.513 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/188/15' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/188/20' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/188/12' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/188/10' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/188/11' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/188/16' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/188/14' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/188/18' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/188/13' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/188/17' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/188' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/145/19' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/145/15' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/145/20' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/145/12' 2024-03-09T08:57:54.514 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/145/10' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/145/11' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/145/16' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/145/14' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/145/18' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/145/13' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/145/17' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/145' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/130/19' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/130/15' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/130/20' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/130/12' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/130/10' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/130/11' 2024-03-09T08:57:54.515 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/130/16' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/130/14' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/130/18' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/130/13' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/130/17' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/130' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/193/19' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/193/15' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/193/20' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/193/12' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/193/10' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/193/11' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/193/16' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/193/14' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/193/18' 2024-03-09T08:57:54.516 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/193/13' 2024-03-09T08:57:54.517 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/193/17' 2024-03-09T08:57:54.517 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/193' 2024-03-09T08:57:54.517 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/175/19' 2024-03-09T08:57:54.517 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/175/15' 2024-03-09T08:57:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.564+0000 7fb414c00640 1 Processor -- start 2024-03-09T08:57:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.565+0000 7fb414c00640 1 -- start start 2024-03-09T08:57:54.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.566+0000 7fb414c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100730c0 0x7fb4100aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.566+0000 7fb414c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb410152e00 con 0x7fb41006d830 2024-03-09T08:57:54.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.566+0000 7fb414c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4101530e0 con 0x7fb4100734a0 2024-03-09T08:57:54.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.566+0000 7fb414c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100aa960 con 0x7fb4100730c0 2024-03-09T08:57:54.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.567+0000 7fb40e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb41006d830 0x7fb41006dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60484/0 (socket says 172.21.15.67:60484) 2024-03-09T08:57:54.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.567+0000 7fb40e800640 1 -- 172.21.15.67:0/2089010072 learned_addr learned my addr 172.21.15.67:0/2089010072 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:57:54.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.567+0000 7fb40de00640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100730c0 0x7fb4100aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.567+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225152462 0 0) 0x7fb410152e00 con 0x7fb41006d830 2024-03-09T08:57:54.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.567+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f4005730 con 0x7fb41006d830 2024-03-09T08:57:54.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.567+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3552107401 0 0) 0x7fb3f4005730 con 0x7fb41006d830 2024-03-09T08:57:54.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.567+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100730c0 msgr2=0x7fb4100aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:54.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.567+0000 7fb40d400640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100730c0 0x7fb4100aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.567+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 >> v1:172.21.15.134:6789/0 conn(0x7fb4100734a0 legacy=0x7fb4100c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.567+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4101533c0 con 0x7fb41006d830 2024-03-09T08:57:54.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.567+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb4080019f0 con 0x7fb41006d830 2024-03-09T08:57:54.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 >> v1:172.21.15.67:6789/0 conn(0x7fb41006d830 legacy=0x7fb41006dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40d400640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f4005bf0 0x7fb3f4006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40d400640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100730c0 0x7fb3f4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40d400640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3f4006f50 0x7fb3f40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100aa960 con 0x7fb3f4005bf0 2024-03-09T08:57:54.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f400dd00 con 0x7fb3f4006f50 2024-03-09T08:57:54.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f400df00 con 0x7fb4100730c0 2024-03-09T08:57:54.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40de00640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f4005bf0 0x7fb3f4006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb407e00640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100730c0 0x7fb3f4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40de00640 1 -- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100730c0 msgr2=0x7fb3f4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40e800640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3f4006f50 0x7fb3f40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40de00640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100730c0 0x7fb3f4006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40de00640 1 -- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3f4006f50 msgr2=0x7fb3f40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40de00640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3f4006f50 0x7fb3f40073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb40de00640 1 -- 172.21.15.67:0/2089010072 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4005730 con 0x7fb3f4005bf0 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.568+0000 7fb407e00640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100730c0 0x7fb3f4006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.570+0000 7fb40de00640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f4005bf0 0x7fb3f4006010 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb400009ac0 tx=0x7fb3f4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e24620ff4be008 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.570+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400039070 con 0x7fb3f4005bf0 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.570+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40002c600 con 0x7fb3f4005bf0 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.570+0000 7fb40d400640 1 -- 172.21.15.67:0/2089010072 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400009690 con 0x7fb3f4005bf0 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.570+0000 7fb414c00640 1 -- 172.21.15.67:0/2089010072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f4005bf0 msgr2=0x7fb3f4006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.570+0000 7fb414c00640 1 --2- 172.21.15.67:0/2089010072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f4005bf0 0x7fb3f4006010 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb400009ac0 tx=0x7fb3f4007a40 comp rx=0 tx=0).stop 2024-03-09T08:57:54.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.570+0000 7fb40e800640 1 -- 172.21.15.67:0/2089010072 reap_dead start 2024-03-09T08:57:54.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.570+0000 7fb414c00640 1 -- 172.21.15.67:0/2089010072 shutdown_connections 2024-03-09T08:57:54.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.570+0000 7fb414c00640 1 -- 172.21.15.67:0/2089010072 >> 172.21.15.67:0/2089010072 conn(0x7fb4100c7370 msgr2=0x7fb4100c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:54.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.571+0000 7fb414c00640 1 -- 172.21.15.67:0/2089010072 shutdown_connections 2024-03-09T08:57:54.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.571+0000 7fb414c00640 1 -- 172.21.15.67:0/2089010072 wait complete. 2024-03-09T08:57:54.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.572+0000 7fb414c00640 1 Processor -- start 2024-03-09T08:57:54.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.572+0000 7fb414c00640 1 -- start start 2024-03-09T08:57:54.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.573+0000 7fb414c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f4005bf0 0x7fb410143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.573+0000 7fb414c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb410143cb0 0x7fb4101516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.573+0000 7fb414c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb410151c00 0x7fb4100abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.573+0000 7fb414c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100c3590 con 0x7fb3f4005bf0 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.573+0000 7fb414c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410153980 con 0x7fb410143cb0 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.573+0000 7fb414c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100c3710 con 0x7fb410151c00 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.573+0000 7fb40de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f4005bf0 0x7fb410143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.573+0000 7fb40de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f4005bf0 0x7fb410143770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40272/0 (socket says 172.21.15.67:40272) 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.574+0000 7fb40de00640 1 -- 172.21.15.67:0/2122997240 learned_addr learned my addr 172.21.15.67:0/2122997240 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.574+0000 7fb407e00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb410143cb0 0x7fb4101516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.574+0000 7fb40e800640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb410151c00 0x7fb4100abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.574+0000 7fb40de00640 1 -- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb410151c00 msgr2=0x7fb4100abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.574+0000 7fb40de00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb410151c00 0x7fb4100abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.574+0000 7fb40de00640 1 -- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb410143cb0 msgr2=0x7fb4101516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.574+0000 7fb40de00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb410143cb0 0x7fb4101516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.574+0000 7fb40de00640 1 -- 172.21.15.67:0/2122997240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4101e73d0 con 0x7fb3f4005bf0 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.574+0000 7fb40de00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f4005bf0 0x7fb410143770 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb400009ac0 tx=0x7fb400004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.575+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400039050 con 0x7fb3f4005bf0 2024-03-09T08:57:54.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.575+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb400045070 con 0x7fb3f4005bf0 2024-03-09T08:57:54.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.575+0000 7fb414c00640 1 -- 172.21.15.67:0/2122997240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb410152610 con 0x7fb3f4005bf0 2024-03-09T08:57:54.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.575+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400003850 con 0x7fb3f4005bf0 2024-03-09T08:57:54.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.575+0000 7fb414c00640 1 -- 172.21.15.67:0/2122997240 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4101e7970 con 0x7fb3f4005bf0 2024-03-09T08:57:54.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.576+0000 7fb414c00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100ac170 0x7fb410150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.576+0000 7fb414c00640 1 -- 172.21.15.67:0/2122997240 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb410075a00 con 0x7fb4100ac170 2024-03-09T08:57:54.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.577+0000 7fb407e00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100ac170 0x7fb410150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.577+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb4000039f0 con 0x7fb3f4005bf0 2024-03-09T08:57:54.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.577+0000 7fb407e00640 1 -- 172.21.15.67:0/2122997240 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb410152b20 con 0x7fb4100ac170 2024-03-09T08:57:54.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.578+0000 7fb407e00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100ac170 0x7fb410150890 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb4100c2750 tx=0x7fb3f800fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.579+0000 7fb406000640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb3ec06b4d0 0x7fb3ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.579+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fb4000ac4d0 con 0x7fb3f4005bf0 2024-03-09T08:57:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.579+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8017020 con 0x7fb4100ac170 2024-03-09T08:57:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.579+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb3f80050d0 con 0x7fb4100ac170 2024-03-09T08:57:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.579+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100ac170 msgr2=0x7fb410150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.579+0000 7fb406000640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100ac170 0x7fb410150890 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb4100c2750 tx=0x7fb3f800fc00 comp rx=0 tx=0).stop 2024-03-09T08:57:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.579+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100ac170 msgr2=0x7fb410150890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:54.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.579+0000 7fb406000640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4100ac170 0x7fb410150890 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb4100c2750 tx=0x7fb3f800fc00 comp rx=0 tx=0).stop 2024-03-09T08:57:54.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.579+0000 7fb40e800640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb3ec06b4d0 0x7fb3ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.581+0000 7fb40e800640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb3ec06b4d0 0x7fb3ec06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb41006c090 tx=0x7fb408009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.587+0000 7fb414c00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb410150dd0 0x7fb4101ea940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:57:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.587+0000 7fb414c00640 1 -- 172.21.15.67:0/2122997240 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101511f0 con 0x7fb410150dd0 2024-03-09T08:57:54.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.587+0000 7fb407e00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb410150dd0 0x7fb4101ea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:57:54.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.588+0000 7fb407e00640 1 -- 172.21.15.67:0/2122997240 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb41014fe00 con 0x7fb410150dd0 2024-03-09T08:57:54.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.588+0000 7fb407e00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb410150dd0 0x7fb4101ea940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8018000 tx=0x7fb3f8001190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:57:54.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.589+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8017070 con 0x7fb410150dd0 2024-03-09T08:57:54.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.589+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fb3f800d920 con 0x7fb410150dd0 2024-03-09T08:57:54.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.589+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb410150dd0 msgr2=0x7fb4101ea940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.589+0000 7fb406000640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb410150dd0 0x7fb4101ea940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8018000 tx=0x7fb3f8001190 comp rx=0 tx=0).stop 2024-03-09T08:57:54.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.589+0000 7fb406000640 1 -- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb410150dd0 msgr2=0x7fb4101ea940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:57:54.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.589+0000 7fb406000640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb410150dd0 0x7fb4101ea940 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8018000 tx=0x7fb3f8001190 comp rx=0 tx=0).stop 2024-03-09T08:57:54.589 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:57:54.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:54.589 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:54.590 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:57:54.590 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 42, 2024-03-09T08:57:54.590 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:57:54.590 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:57:54.590 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:57:54.590 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:57:54.590 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.590 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T08:57:54.590 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:54.590 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:57:54.591 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:57:54.591 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:54.591 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:54.591 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:54.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:54.592 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:57:54.593 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:57:54.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:57:54.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.597 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:54.598 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:57:54.599 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:57:54.600 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:57:54.602 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:57:54.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.590+0000 7fb414c00640 1 -- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb3ec06b4d0 msgr2=0x7fb3ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.590+0000 7fb414c00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb3ec06b4d0 0x7fb3ec06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb41006c090 tx=0x7fb408009040 comp rx=0 tx=0).stop 2024-03-09T08:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.590+0000 7fb414c00640 1 -- 172.21.15.67:0/2122997240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f4005bf0 msgr2=0x7fb410143770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.590+0000 7fb414c00640 1 --2- 172.21.15.67:0/2122997240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3f4005bf0 0x7fb410143770 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb400009ac0 tx=0x7fb400004730 comp rx=0 tx=0).stop 2024-03-09T08:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.590+0000 7fb40e800640 1 -- 172.21.15.67:0/2122997240 reap_dead start 2024-03-09T08:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.590+0000 7fb40e800640 1 -- 172.21.15.67:0/2122997240 reap_dead start 2024-03-09T08:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.590+0000 7fb414c00640 1 -- 172.21.15.67:0/2122997240 shutdown_connections 2024-03-09T08:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.590+0000 7fb414c00640 1 -- 172.21.15.67:0/2122997240 >> 172.21.15.67:0/2122997240 conn(0x7fb4100c7370 msgr2=0x7fb4100c1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.590+0000 7fb414c00640 1 -- 172.21.15.67:0/2122997240 shutdown_connections 2024-03-09T08:57:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:57:54.590+0000 7fb414c00640 1 -- 172.21.15.67:0/2122997240 wait complete. 2024-03-09T08:57:54.606 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:57:54.607 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-03-09T08:57:54.607 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T08:57:54.607 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-03-09T08:57:54.607 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T08:57:54.607 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-09T08:57:54.607 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T08:57:55.336 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/175/20' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/175/12' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/175/10' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/175/11' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/175/16' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/175/14' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/175/18' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/175/13' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/175/17' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/175' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/181/19' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/181/15' 2024-03-09T08:57:55.337 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/181/20' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/181/12' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/181/10' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/181/11' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/181/16' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/181/14' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/181/18' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/181/13' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/181/17' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/181' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/127/19' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/127/15' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/127/20' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/127/12' 2024-03-09T08:57:55.338 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/127/10' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/127/11' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/127/16' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/127/14' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/127/18' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/127/13' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/127/17' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/127' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/108/19' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/108/15' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/108/20' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/108/12' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/108/10' 2024-03-09T08:57:55.339 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/108/11' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/108/16' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/108/14' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/108/18' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/108/13' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/108/17' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/108' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/183/19' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/183/15' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/183/20' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/183/12' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/183/10' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/183/11' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/183/16' 2024-03-09T08:57:55.340 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/183/14' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/183/18' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/183/13' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/183/17' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/183' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/177/19' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/177/15' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/177/20' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/177/12' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/177/10' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/177/11' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/177/16' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/177/14' 2024-03-09T08:57:55.341 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/177/18' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/177/13' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/177/17' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/177' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/169/19' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/169/15' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/169/20' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/169/12' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/169/10' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/169/11' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/169/16' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/169/14' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/169/18' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/169/13' 2024-03-09T08:57:55.342 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/169/17' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/169' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/154/19' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/154/15' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/154/20' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/154/12' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/154/10' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/154/11' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/154/16' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/154/14' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/154/18' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/154/13' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/154/17' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/154' 2024-03-09T08:57:55.343 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/171/19' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/171/15' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/171/20' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/171/12' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/171/10' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/171/11' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/171/16' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/171/14' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/171/18' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/171/13' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/171/17' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/171' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/139/19' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/139/15' 2024-03-09T08:57:55.344 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/139/20' 2024-03-09T08:57:55.559 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/139/12' 2024-03-09T08:57:56.150 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/139/10' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/139/11' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/139/16' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/139/14' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/139/18' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/139/13' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/139/17' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/139' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/102/19' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/102/15' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/102/20' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/102/12' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/102/10' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/102/11' 2024-03-09T08:57:56.151 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/102/16' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/102/14' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/102/18' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/102/13' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/102/17' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/102' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/129/19' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/129/15' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/129/20' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/129/12' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/129/10' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/129/11' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/129/16' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/129/14' 2024-03-09T08:57:56.152 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/129/18' 2024-03-09T08:57:56.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/129/13' 2024-03-09T08:57:56.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed './copyofsnap1/test1/129/17' 2024-03-09T08:57:56.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1/129' 2024-03-09T08:57:56.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1/test1' 2024-03-09T08:57:56.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory './copyofsnap1' 2024-03-09T08:57:56.153 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T08:57:56.153 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:57:56.153 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T08:57:56.200 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/175/20' 2024-03-09T08:57:56.200 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/175/12' 2024-03-09T08:57:56.200 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/175/10' 2024-03-09T08:57:56.200 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/175/11' 2024-03-09T08:57:56.200 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/175/16' 2024-03-09T08:57:56.200 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/175/14' 2024-03-09T08:57:56.200 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/175/18' 2024-03-09T08:57:56.201 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/175/13' 2024-03-09T08:57:56.201 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/175/17' 2024-03-09T08:57:56.201 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/175' 2024-03-09T08:57:56.201 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/181/19' 2024-03-09T08:57:56.201 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/181/15' 2024-03-09T08:57:56.201 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/181/20' 2024-03-09T08:57:56.201 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/181/12' 2024-03-09T08:57:56.201 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/181/10' 2024-03-09T08:57:56.201 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/181/11' 2024-03-09T08:57:56.202 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/181/16' 2024-03-09T08:57:56.202 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/181/14' 2024-03-09T08:57:56.202 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/181/18' 2024-03-09T08:57:56.202 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/181/13' 2024-03-09T08:57:56.202 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/181/17' 2024-03-09T08:57:56.202 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/181' 2024-03-09T08:57:56.202 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/127/19' 2024-03-09T08:57:56.202 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/127/15' 2024-03-09T08:57:56.202 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/127/20' 2024-03-09T08:57:56.202 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/127/12' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/127/10' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/127/11' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/127/16' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/127/14' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/127/18' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/127/13' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/127/17' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/127' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/108/19' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/108/15' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/108/20' 2024-03-09T08:57:56.203 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/108/12' 2024-03-09T08:57:56.204 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/108/10' 2024-03-09T08:57:56.204 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/108/11' 2024-03-09T08:57:56.204 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/108/16' 2024-03-09T08:57:56.204 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/108/14' 2024-03-09T08:57:56.204 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/108/18' 2024-03-09T08:57:56.204 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/108/13' 2024-03-09T08:57:56.204 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/108/17' 2024-03-09T08:57:56.204 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/108' 2024-03-09T08:57:56.204 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/183/19' 2024-03-09T08:57:56.204 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/183/15' 2024-03-09T08:57:56.204 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/183/20' 2024-03-09T08:57:56.205 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/183/12' 2024-03-09T08:57:56.205 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/183/10' 2024-03-09T08:57:56.205 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/183/11' 2024-03-09T08:57:56.205 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/183/16' 2024-03-09T08:57:56.205 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/183/14' 2024-03-09T08:57:56.205 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/183/18' 2024-03-09T08:57:56.205 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/183/13' 2024-03-09T08:57:56.205 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/183/17' 2024-03-09T08:57:56.205 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/183' 2024-03-09T08:57:56.205 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/177/19' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/177/15' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/177/20' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/177/12' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/177/10' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/177/11' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/177/16' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/177/14' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/177/18' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/177/13' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/177/17' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/177' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/169/19' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/169/15' 2024-03-09T08:57:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/169/20' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/169/12' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/169/10' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/169/11' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/169/16' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/169/14' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/169/18' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/169/13' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/169/17' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/169' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/154/19' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/154/15' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/154/20' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/154/12' 2024-03-09T08:57:56.207 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/154/10' 2024-03-09T08:57:56.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/154/11' 2024-03-09T08:57:56.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/154/16' 2024-03-09T08:57:56.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/154/14' 2024-03-09T08:57:56.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/154/18' 2024-03-09T08:57:56.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/154/13' 2024-03-09T08:57:56.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/154/17' 2024-03-09T08:57:56.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/154' 2024-03-09T08:57:56.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/171/19' 2024-03-09T08:57:56.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/171/15' 2024-03-09T08:57:56.208 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/171/20' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/171/12' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/171/10' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/171/11' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/171/16' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/171/14' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/171/18' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/171/13' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/171/17' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/171' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/139/19' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/139/15' 2024-03-09T08:57:56.209 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/139/20' 2024-03-09T08:57:56.273 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2024-03-09T08:57:56.273 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-authwb.sh 2024-03-09T08:57:56.341 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T08:57:56.341 INFO:tasks.workunit.client.1.smithi134.stderr:+ touch foo 2024-03-09T08:57:56.346 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod +x foo 2024-03-09T08:57:56.350 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T08:57:56.396 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/139/12' 2024-03-09T08:57:56.396 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/139/10' 2024-03-09T08:57:56.396 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/139/11' 2024-03-09T08:57:56.396 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/139/16' 2024-03-09T08:57:56.396 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/139/14' 2024-03-09T08:57:56.396 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/139/18' 2024-03-09T08:57:56.396 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/139/13' 2024-03-09T08:57:56.396 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/139/17' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/139' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/102/19' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/102/15' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/102/20' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/102/12' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/102/10' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/102/11' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/102/16' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/102/14' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/102/18' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/102/13' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/102/17' 2024-03-09T08:57:56.397 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/102' 2024-03-09T08:57:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/129/19' 2024-03-09T08:57:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/129/15' 2024-03-09T08:57:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/129/20' 2024-03-09T08:57:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/129/12' 2024-03-09T08:57:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/129/10' 2024-03-09T08:57:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/129/11' 2024-03-09T08:57:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/129/16' 2024-03-09T08:57:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/129/14' 2024-03-09T08:57:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/129/18' 2024-03-09T08:57:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/129/13' 2024-03-09T08:57:56.398 INFO:tasks.workunit.client.0.smithi067.stdout:removed './copyofsnap1/test1/129/17' 2024-03-09T08:57:56.399 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1/129' 2024-03-09T08:57:56.399 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1/test1' 2024-03-09T08:57:56.399 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory './copyofsnap1' 2024-03-09T08:57:56.400 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T08:57:56.400 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:57:56.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T08:57:56.486 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2024-03-09T08:57:56.487 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-authwb.sh 2024-03-09T08:57:56.546 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T08:57:56.546 INFO:tasks.workunit.client.0.smithi067.stderr:+ touch foo 2024-03-09T08:57:56.550 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod +x foo 2024-03-09T08:57:56.552 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T08:57:56.599 INFO:tasks.workunit.client.0.smithi067.stderr:+ find .snap/s/foo -executable 2024-03-09T08:57:56.600 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep foo 2024-03-09T08:57:56.602 INFO:tasks.workunit.client.0.smithi067.stdout:.snap/s/foo 2024-03-09T08:57:56.602 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T08:57:56.615 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm foo 2024-03-09T08:57:56.621 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T08:57:56.621 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T08:57:56.622 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:57:56.622 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T08:57:56.664 INFO:tasks.workunit.client.1.smithi134.stderr:+ find .snap/s/foo -executable 2024-03-09T08:57:56.664 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep foo 2024-03-09T08:57:56.669 INFO:tasks.workunit.client.1.smithi134.stdout:.snap/s/foo 2024-03-09T08:57:56.669 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T08:57:56.679 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm foo 2024-03-09T08:57:56.684 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T08:57:56.684 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T08:57:56.685 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:57:56.685 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T08:57:56.725 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2024-03-09T08:57:56.726 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-capwb.sh 2024-03-09T08:57:56.799 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2024-03-09T08:57:56.799 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-capwb.sh 2024-03-09T08:57:56.806 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T08:57:56.806 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir foo 2024-03-09T08:57:56.809 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo x 2024-03-09T08:57:56.810 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:57:56.851 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T08:57:56.852 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir foo 2024-03-09T08:57:56.854 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo x 2024-03-09T08:57:56.855 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:57:57.262 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-03-09T08:57:58.466 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir foo/.snap/ss 2024-03-09T08:57:58.509 INFO:tasks.workunit.client.0.smithi067.stderr:+ ln foo/x foo/xx 2024-03-09T08:57:58.517 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat foo/.snap/ss/x 2024-03-09T08:57:58.534 INFO:tasks.workunit.client.0.smithi067.stdout:x 2024-03-09T08:57:58.535 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir foo/.snap/ss 2024-03-09T08:57:58.549 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo a 2024-03-09T08:57:58.553 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo b 2024-03-09T08:57:58.554 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir foo/.snap/s 2024-03-09T08:57:58.557 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir foo/.snap/ss 2024-03-09T08:57:58.596 INFO:tasks.workunit.client.1.smithi134.stderr:+ ln foo/x foo/xx 2024-03-09T08:57:58.597 INFO:tasks.workunit.client.0.smithi067.stderr:++ cat foo/.snap/s/a 2024-03-09T08:57:58.602 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat foo/.snap/ss/x 2024-03-09T08:57:58.614 INFO:tasks.workunit.client.1.smithi134.stdout:x 2024-03-09T08:57:58.615 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir foo/.snap/ss 2024-03-09T08:57:58.615 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=a 2024-03-09T08:57:58.615 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z a ']' 2024-03-09T08:57:58.615 INFO:tasks.workunit.client.0.smithi067.stderr:+ ln foo/b foo/b2 2024-03-09T08:57:58.622 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat foo/.snap/s/b 2024-03-09T08:57:58.627 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo a 2024-03-09T08:57:58.630 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo b 2024-03-09T08:57:58.630 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir foo/.snap/s 2024-03-09T08:57:58.642 INFO:tasks.workunit.client.0.smithi067.stdout:b 2024-03-09T08:57:58.643 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'this used to hang:' 2024-03-09T08:57:58.643 INFO:tasks.workunit.client.0.smithi067.stdout:this used to hang: 2024-03-09T08:57:58.643 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo more 2024-03-09T08:57:58.643 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'oh, it didn'\''t hang! good job.' 2024-03-09T08:57:58.644 INFO:tasks.workunit.client.0.smithi067.stdout:oh, it didn't hang! good job. 2024-03-09T08:57:58.644 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat foo/b 2024-03-09T08:57:58.645 INFO:tasks.workunit.client.0.smithi067.stdout:b 2024-03-09T08:57:58.645 INFO:tasks.workunit.client.0.smithi067.stdout:more 2024-03-09T08:57:58.646 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir foo/.snap/s 2024-03-09T08:57:58.659 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -r foo 2024-03-09T08:57:58.673 INFO:tasks.workunit.client.1.smithi134.stderr:++ cat foo/.snap/s/a 2024-03-09T08:57:58.687 INFO:tasks.workunit.client.1.smithi134.stderr:+ r=a 2024-03-09T08:57:58.688 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' -z a ']' 2024-03-09T08:57:58.688 INFO:tasks.workunit.client.1.smithi134.stderr:+ ln foo/b foo/b2 2024-03-09T08:57:58.693 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat foo/.snap/s/b 2024-03-09T08:57:58.713 INFO:tasks.workunit.client.1.smithi134.stdout:b 2024-03-09T08:57:58.713 INFO:tasks.workunit.client.1.smithi134.stdout:this used to hang: 2024-03-09T08:57:58.714 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'this used to hang:' 2024-03-09T08:57:58.714 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo more 2024-03-09T08:57:58.714 INFO:tasks.workunit.client.1.smithi134.stdout:oh, it didn't hang! good job. 2024-03-09T08:57:58.714 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'oh, it didn'\''t hang! good job.' 2024-03-09T08:57:58.714 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat foo/b 2024-03-09T08:57:58.717 INFO:tasks.workunit.client.1.smithi134.stdout:b 2024-03-09T08:57:58.717 INFO:tasks.workunit.client.1.smithi134.stdout:more 2024-03-09T08:57:58.718 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir foo/.snap/s 2024-03-09T08:57:58.731 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm -r foo 2024-03-09T08:58:00.634 INFO:tasks.ceph.mon.a:Stopped 2024-03-09T08:58:00.635 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T08:58:00.635 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:58:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.859+0000 7f465de00640 1 Processor -- start 2024-03-09T08:58:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.859+0000 7f465de00640 1 -- start start 2024-03-09T08:58:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.861+0000 7f465de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806ed50 0x7f46581592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.861+0000 7f465de00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4658155b30 con 0x7f4658073180 2024-03-09T08:58:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.861+0000 7f465de00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4658155df0 con 0x7f465806f280 2024-03-09T08:58:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.861+0000 7f465de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46581560d0 con 0x7f465806ed50 2024-03-09T08:58:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.861+0000 7f465ca00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f4658073180 legacy=0x7f46580c4de0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T08:58:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.861+0000 7f4657e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806ed50 0x7f46581592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:00.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.861+0000 7f465d400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f465806f280 0x7f4658159810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:58080/0 (socket says 172.21.15.67:58080) 2024-03-09T08:58:00.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.861+0000 7f465d400640 1 -- 172.21.15.67:0/1008282399 learned_addr learned my addr 172.21.15.67:0/1008282399 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:58:00.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.862+0000 7f4657e00640 1 -- 172.21.15.67:0/1008282399 >> v1:172.21.15.134:6789/0 conn(0x7f465806f280 legacy=0x7f4658159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:00.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.862+0000 7f4657e00640 1 -- 172.21.15.67:0/1008282399 >> v1:172.21.15.67:6789/0 conn(0x7f4658073180 legacy=0x7f46580c4de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:00.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.862+0000 7f4657e00640 1 -- 172.21.15.67:0/1008282399 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46581563b0 con 0x7f465806ed50 2024-03-09T08:58:00.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.863+0000 7f4657e00640 1 --2- 172.21.15.67:0/1008282399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806ed50 0x7f46581592d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4644009800 tx=0x7f464402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7dfa7ecb997ed5f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:00.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.864+0000 7f4657400640 1 -- 172.21.15.67:0/1008282399 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644035020 con 0x7f465806ed50 2024-03-09T08:58:00.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.864+0000 7f4657400640 1 -- 172.21.15.67:0/1008282399 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644031800 con 0x7f465806ed50 2024-03-09T08:58:00.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.864+0000 7f4657400640 1 -- 172.21.15.67:0/1008282399 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644031bd0 con 0x7f465806ed50 2024-03-09T08:58:00.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.864+0000 7f465de00640 1 -- 172.21.15.67:0/1008282399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806ed50 msgr2=0x7f46581592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:00.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.864+0000 7f465de00640 1 --2- 172.21.15.67:0/1008282399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806ed50 0x7f46581592d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4644009800 tx=0x7f464402b8a0 comp rx=0 tx=0).stop 2024-03-09T08:58:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.864+0000 7f465de00640 1 -- 172.21.15.67:0/1008282399 shutdown_connections 2024-03-09T08:58:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.864+0000 7f465de00640 1 --2- 172.21.15.67:0/1008282399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806ed50 0x7f46581592d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.864+0000 7f465de00640 1 -- 172.21.15.67:0/1008282399 >> 172.21.15.67:0/1008282399 conn(0x7f465806c150 msgr2=0x7f465806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.864+0000 7f465de00640 1 -- 172.21.15.67:0/1008282399 shutdown_connections 2024-03-09T08:58:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.864+0000 7f465de00640 1 -- 172.21.15.67:0/1008282399 wait complete. 2024-03-09T08:58:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.866+0000 7f465de00640 1 Processor -- start 2024-03-09T08:58:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.866+0000 7f465de00640 1 -- start start 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.867+0000 7f465de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465806ed50 0x7f46581e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.867+0000 7f465de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806f280 0x7f465814a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.867+0000 7f465de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4658073180 0x7f46581eef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.867+0000 7f465de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4658159f20 con 0x7f4658073180 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.867+0000 7f465de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46580c3920 con 0x7f465806ed50 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.867+0000 7f465de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4658073560 con 0x7f465806f280 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.867+0000 7f465d400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4658073180 msgr2=0x7f46581eef20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.867+0000 7f465d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4658073180 0x7f46581eef20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.868+0000 7f465ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465806ed50 0x7f46581e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.868+0000 7f4657e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806f280 0x7f465814a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.868+0000 7f465ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465806ed50 0x7f46581e2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:59100/0 (socket says 172.21.15.67:59100) 2024-03-09T08:58:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.868+0000 7f465ca00640 1 -- 172.21.15.67:0/688419800 learned_addr learned my addr 172.21.15.67:0/688419800 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.869+0000 7f4657e00640 1 -- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465806ed50 msgr2=0x7f46581e2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.869+0000 7f4657e00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465806ed50 0x7f46581e2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.869+0000 7f4657e00640 1 -- 172.21.15.67:0/688419800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4658073180 msgr2=0x7f46581eef20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:58:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.869+0000 7f4657e00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4658073180 0x7f46581eef20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.869+0000 7f4657e00640 1 -- 172.21.15.67:0/688419800 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4658158a00 con 0x7f465806f280 2024-03-09T08:58:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.869+0000 7f4657e00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806f280 0x7f465814a7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4644033000 tx=0x7f46440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.870+0000 7f4654c00640 1 -- 172.21.15.67:0/688419800 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464403ad60 con 0x7f465806f280 2024-03-09T08:58:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.870+0000 7f4654c00640 1 -- 172.21.15.67:0/688419800 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46440443f0 con 0x7f465806f280 2024-03-09T08:58:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.870+0000 7f465de00640 1 -- 172.21.15.67:0/688419800 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46580adf80 con 0x7f465806f280 2024-03-09T08:58:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.870+0000 7f4654c00640 1 -- 172.21.15.67:0/688419800 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46440447e0 con 0x7f465806f280 2024-03-09T08:58:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.870+0000 7f465de00640 1 -- 172.21.15.67:0/688419800 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465814ad00 con 0x7f465806f280 2024-03-09T08:58:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.872+0000 7f465de00640 1 -- 172.21.15.67:0/688419800 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46580ae460 con 0x7f465806f280 2024-03-09T08:58:00.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.875+0000 7f4654c00640 1 -- 172.21.15.67:0/688419800 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f464404b060 con 0x7f465806f280 2024-03-09T08:58:00.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.875+0000 7f4654c00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f463806b480 0x7f463806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:00.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.875+0000 7f4654c00640 1 -- 172.21.15.67:0/688419800 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f46440ac7c0 con 0x7f465806f280 2024-03-09T08:58:00.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.876+0000 7f465ca00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f463806b480 0x7f463806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:00.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.878+0000 7f465ca00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f463806b480 0x7f463806d940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4648009450 tx=0x7f46480093e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:00.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:00.889+0000 7f4654c00640 1 -- 172.21.15.67:0/688419800 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f46580ae460 con 0x7f465806f280 2024-03-09T08:58:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.293+0000 7f465de00640 1 -- 172.21.15.67:0/688419800 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f465804f3c0 con 0x7f465806f280 2024-03-09T08:58:01.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.294+0000 7f4654c00640 1 -- 172.21.15.67:0/688419800 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f464403f070 con 0x7f465806f280 2024-03-09T08:58:01.298 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":42,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:58:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 -- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f463806b480 msgr2=0x7f463806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f463806b480 0x7f463806d940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4648009450 tx=0x7f46480093e0 comp rx=0 tx=0).stop 2024-03-09T08:58:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 -- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806f280 msgr2=0x7f465814a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806f280 0x7f465814a7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4644033000 tx=0x7f46440046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 -- 172.21.15.67:0/688419800 shutdown_connections 2024-03-09T08:58:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f463806b480 0x7f463806d940 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4658073180 0x7f46581eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f465806f280 0x7f465814a7c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 --2- 172.21.15.67:0/688419800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f465806ed50 0x7f46581e2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:01.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 -- 172.21.15.67:0/688419800 >> 172.21.15.67:0/688419800 conn(0x7f465806c150 msgr2=0x7f46581592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:01.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 -- 172.21.15.67:0/688419800 shutdown_connections 2024-03-09T08:58:01.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:01.296+0000 7f465de00640 1 -- 172.21.15.67:0/688419800 wait complete. 2024-03-09T08:58:03.876 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-03-09T08:58:04.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:58:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.580+0000 7fe36b200640 1 Processor -- start 2024-03-09T08:58:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.580+0000 7fe36b200640 1 -- start start 2024-03-09T08:58:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.581+0000 7fe36b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.581+0000 7fe36b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe3640cadb0 0x7fe364156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.581+0000 7fe36b200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe364156a90 con 0x7fe364153300 2024-03-09T08:58:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.581+0000 7fe36b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe364157000 con 0x7fe3640cadb0 2024-03-09T08:58:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.581+0000 7fe36b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640ae430 con 0x7fe3640c5e40 2024-03-09T08:58:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.582+0000 7fe36a800640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fe3640cadb0 msgr2=0x7fe364156550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T08:58:04.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.582+0000 7fe36a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe3640cadb0 0x7fe364156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:04.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.582+0000 7fe369e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe364153300 0x7fe3640b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:58086/0 (socket says 172.21.15.67:58086) 2024-03-09T08:58:04.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.582+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 learned_addr learned my addr 172.21.15.67:0/3029611253 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:58:04.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.582+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:04.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.583+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe364156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:58:04.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.583+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe364156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:04.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.583+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:04.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.583+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:04.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.783+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe3640cadb0 msgr2=0x7fe364156550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T08:58:04.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.783+0000 7fe36a800640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe3640cadb0 0x7fe364156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:58:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.784+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.785+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe364156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:58:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.785+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe364156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.785+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:04.785+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:58:05.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.184+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe3640cadb0 msgr2=0x7fe364156550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T08:58:05.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.184+0000 7fe36a800640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe3640cadb0 0x7fe364156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:58:05.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.186+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:05.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.187+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe364156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:58:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.187+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe364156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.187+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:05.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.187+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:58:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.985+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe3640cadb0 msgr2=0x7fe364156550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T08:58:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.985+0000 7fe36a800640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe3640cadb0 0x7fe364156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:58:05.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.989+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:05.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.989+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe364156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:58:05.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.989+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe364156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:05.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.989+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:05.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:05.989+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:58:07.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.582+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe364156010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:07.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.582+0000 7fe35f400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe364156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:07.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.582+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe3640cadb0 msgr2=0x7fe364156550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:07.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.582+0000 7fe35f400640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe3640cadb0 0x7fe364156550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:07.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.582+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.134:6789/0 conn(0x7fe364153300 legacy=0x7fe3640b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:07.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.582+0000 7fe35f400640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:07.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.582+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3640ae6f0 con 0x7fe348006170 2024-03-09T08:58:07.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.582+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe364157000 con 0x7fe3480036c0 2024-03-09T08:58:07.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.583+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.67:6789/0 conn(0x7fe348006170 legacy=0x7fe348008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T08:58:07.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.583+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:07.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.583+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:58:07.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.583+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:07.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.583+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:07.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.583+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:07.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.784+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.67:6789/0 conn(0x7fe348006170 legacy=0x7fe348008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T08:58:07.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.784+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:07.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.785+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:58:07.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.785+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:07.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.785+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:07.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:07.785+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:58:08.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.185+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.67:6789/0 conn(0x7fe348006170 legacy=0x7fe348008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T08:58:08.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.187+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:08.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.187+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:58:08.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.187+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:08.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.187+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:08.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.187+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:58:08.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.986+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.67:6789/0 conn(0x7fe348006170 legacy=0x7fe348008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T08:58:08.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.989+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:08.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.989+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:58:08.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.989+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:08.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.989+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:08.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:08.989+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:58:10.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.582+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:10.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.582+0000 7fe35f400640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe3480036c0 0x7fe348005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:10.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.582+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.67:6789/0 conn(0x7fe348006170 legacy=0x7fe348008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:10.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.582+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 reap_dead start 2024-03-09T08:58:10.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.582+0000 7fe35f400640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:10.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.583+0000 7fe35f400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:10.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.583+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34800af70 con 0x7fe348006170 2024-03-09T08:58:10.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.583+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe364157000 con 0x7fe348009eb0 2024-03-09T08:58:10.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.583+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640ae430 con 0x7fe3640c5e40 2024-03-09T08:58:10.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.583+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.67:6789/0 conn(0x7fe348006170 legacy=0x7fe348008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T08:58:10.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.583+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:10.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.584+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:10.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.584+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 msgr2=0x7fe34800a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:58:10.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.584+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 msgr2=0x7fe34800a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:10.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.584+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:10.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.584+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.585+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:58:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.585+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.585+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.585+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:10.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.784+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.67:6789/0 conn(0x7fe348006170 legacy=0x7fe348008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T08:58:10.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.785+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:10.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.786+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 msgr2=0x7fe34800a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:58:10.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.786+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 msgr2=0x7fe34800a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:10.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.786+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:10.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.786+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:58:10.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.786+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:10.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.787+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:58:10.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.787+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:10.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.787+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:10.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:10.787+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:58:11.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.185+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.67:6789/0 conn(0x7fe348006170 legacy=0x7fe348008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T08:58:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.188+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:11.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.188+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 msgr2=0x7fe34800a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:58:11.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.188+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 msgr2=0x7fe34800a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:11.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.188+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:11.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.188+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:58:11.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.188+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:11.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.189+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:58:11.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.189+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:11.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.189+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.189+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:58:11.458 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:58:11.465 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:58:11.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.986+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.67:6789/0 conn(0x7fe348006170 legacy=0x7fe348008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T08:58:11.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.990+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.990+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 msgr2=0x7fe34800a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:58:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.990+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 msgr2=0x7fe34800a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.990+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:11.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.990+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:58:11.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.991+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:11.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.991+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:58:11.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.991+0000 7fe369400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:58:11.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.991+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:11.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:11.991+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:58:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.583+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 msgr2=0x7fe348005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.583+0000 7fe35f400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3640c5e40 0x7fe348005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.583+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 msgr2=0x7fe34800a290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.583+0000 7fe35f400640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.134:3300/0 conn(0x7fe348009eb0 0x7fe34800a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.583+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.67:6789/0 conn(0x7fe348006170 legacy=0x7fe348008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.583+0000 7fe35f400640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe34800d3b0 0x7fe34800f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.583+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3640ae6f0 con 0x7fe34800fce0 2024-03-09T08:58:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.583+0000 7fe35f400640 1 -- 172.21.15.67:0/3029611253 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe364157000 con 0x7fe34800d3b0 2024-03-09T08:58:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.583+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe34800d3b0 msgr2=0x7fe34800f7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T08:58:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.584+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe34800d3b0 0x7fe34800f7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:13.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.714+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672847036 0 0) 0x7fe3640ae6f0 con 0x7fe34800fce0 2024-03-09T08:58:13.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.714+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe34c003520 con 0x7fe34800fce0 2024-03-09T08:58:13.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.721+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4056450443 0 0) 0x7fe34c003520 con 0x7fe34800fce0 2024-03-09T08:58:13.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe34800d3b0 msgr2=0x7fe34800f7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe368a00640 1 --2- 172.21.15.67:0/3029611253 >> v2:172.21.15.67:3300/0 conn(0x7fe34800d3b0 0x7fe34800f7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34800af70 con 0x7fe34800fce0 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fe3540044e0 con 0x7fe34800fce0 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 >> v1:172.21.15.134:6789/0 conn(0x7fe34800fce0 legacy=0x7fe3480120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 reap_dead start 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe368a00640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c003a20 0x7fe34c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe368a00640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c004800 0x7fe34c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe368a00640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c0053c0 0x7fe34c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe364157000 con 0x7fe34c003a20 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c00c280 con 0x7fe34c0053c0 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c00c480 con 0x7fe34c004800 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe369e00640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c003a20 msgr2=0x7fe34c003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T08:58:13.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.722+0000 7fe369e00640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c003a20 0x7fe34c003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:13.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.723+0000 7fe36a800640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c0053c0 0x7fe34c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:13.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.723+0000 7fe369400640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c004800 0x7fe34c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:13.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.723+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c004800 msgr2=0x7fe34c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:13.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.723+0000 7fe36a800640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c004800 0x7fe34c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:13.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.723+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c003a20 msgr2=0x7fe34c003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:13.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.723+0000 7fe36a800640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c003a20 0x7fe34c003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:13.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.724+0000 7fe36a800640 1 -- 172.21.15.67:0/3029611253 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c003520 con 0x7fe34c0053c0 2024-03-09T08:58:13.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.726+0000 7fe36a800640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c0053c0 0x7fe34c00b9c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3640adf30 tx=0x7fe34c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=703dfd01aa64a24f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.729+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360007a10 con 0x7fe34c0053c0 2024-03-09T08:58:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.729+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe360030070 con 0x7fe34c0053c0 2024-03-09T08:58:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.729+0000 7fe368a00640 1 -- 172.21.15.67:0/3029611253 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36002ce60 con 0x7fe34c0053c0 2024-03-09T08:58:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.731+0000 7fe36b200640 1 -- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c0053c0 msgr2=0x7fe34c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.731+0000 7fe36b200640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c0053c0 0x7fe34c00b9c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3640adf30 tx=0x7fe34c005830 comp rx=0 tx=0).stop 2024-03-09T08:58:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.731+0000 7fe36b200640 1 -- 172.21.15.67:0/3029611253 shutdown_connections 2024-03-09T08:58:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.731+0000 7fe36b200640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c0053c0 0x7fe34c00b9c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.731+0000 7fe36b200640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c004800 0x7fe34c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.731+0000 7fe36b200640 1 --2- 172.21.15.67:0/3029611253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c003a20 0x7fe34c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.731+0000 7fe36b200640 1 -- 172.21.15.67:0/3029611253 >> 172.21.15.67:0/3029611253 conn(0x7fe3640732d0 msgr2=0x7fe3640ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.731+0000 7fe36b200640 1 -- 172.21.15.67:0/3029611253 shutdown_connections 2024-03-09T08:58:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.731+0000 7fe36b200640 1 -- 172.21.15.67:0/3029611253 wait complete. 2024-03-09T08:58:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.737+0000 7fe36b200640 1 Processor -- start 2024-03-09T08:58:13.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.738+0000 7fe36b200640 1 -- start start 2024-03-09T08:58:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.741+0000 7fe36b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c003a20 0x7fe36414dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.741+0000 7fe36b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c004800 0x7fe36406bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.741+0000 7fe36b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c0053c0 0x7fe364152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.741+0000 7fe36b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe364159e90 con 0x7fe34c0053c0 2024-03-09T08:58:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.741+0000 7fe36b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe364159bb0 con 0x7fe34c004800 2024-03-09T08:58:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.741+0000 7fe36b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640c59e0 con 0x7fe34c003a20 2024-03-09T08:58:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.741+0000 7fe36a800640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c0053c0 msgr2=0x7fe364152240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T08:58:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.741+0000 7fe36a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c0053c0 0x7fe364152240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.741+0000 7fe369400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c004800 0x7fe36406bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.742+0000 7fe369e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c003a20 0x7fe36414dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:13.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.742+0000 7fe369400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c004800 0x7fe36406bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:51264/0 (socket says 172.21.15.67:51264) 2024-03-09T08:58:13.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.742+0000 7fe369400640 1 -- 172.21.15.67:0/3611954456 learned_addr learned my addr 172.21.15.67:0/3611954456 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:13.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.742+0000 7fe369400640 1 -- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c003a20 msgr2=0x7fe36414dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:13.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.742+0000 7fe369400640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c003a20 0x7fe36414dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:13.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.742+0000 7fe369400640 1 -- 172.21.15.67:0/3611954456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c0053c0 msgr2=0x7fe364152240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:58:13.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.742+0000 7fe369400640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c0053c0 0x7fe364152240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:13.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.742+0000 7fe369400640 1 -- 172.21.15.67:0/3611954456 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3641e06b0 con 0x7fe34c004800 2024-03-09T08:58:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.743+0000 7fe369400640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c004800 0x7fe36406bac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe3540186b0 tx=0x7fe354018b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.743+0000 7fe35e000640 1 -- 172.21.15.67:0/3611954456 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354017dd0 con 0x7fe34c004800 2024-03-09T08:58:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.743+0000 7fe35e000640 1 -- 172.21.15.67:0/3611954456 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe354028070 con 0x7fe34c004800 2024-03-09T08:58:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.743+0000 7fe35e000640 1 -- 172.21.15.67:0/3611954456 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3540236a0 con 0x7fe34c004800 2024-03-09T08:58:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.743+0000 7fe36b200640 1 -- 172.21.15.67:0/3611954456 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3641df660 con 0x7fe34c004800 2024-03-09T08:58:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.744+0000 7fe36b200640 1 -- 172.21.15.67:0/3611954456 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36414d9b0 con 0x7fe34c004800 2024-03-09T08:58:13.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.745+0000 7fe36b200640 1 -- 172.21.15.67:0/3611954456 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36406c110 con 0x7fe34c004800 2024-03-09T08:58:13.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.753+0000 7fe35e000640 1 -- 172.21.15.67:0/3611954456 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe354021030 con 0x7fe34c004800 2024-03-09T08:58:13.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.755+0000 7fe35e000640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe34806cb30 0x7fe34806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:13.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.755+0000 7fe35e000640 1 -- 172.21.15.67:0/3611954456 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fe35408d070 con 0x7fe34c004800 2024-03-09T08:58:13.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.755+0000 7fe369e00640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe34806cb30 0x7fe34806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:13.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.756+0000 7fe369e00640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe34806cb30 0x7fe34806eff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe358009010 tx=0x7fe358009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:13.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:13.768+0000 7fe35e000640 1 -- 172.21.15.67:0/3611954456 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe35401f040 con 0x7fe34c004800 2024-03-09T08:58:14.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.080+0000 7fe36b200640 1 -- 172.21.15.67:0/3611954456 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe36404f3c0 con 0x7fe34c004800 2024-03-09T08:58:14.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.081+0000 7fe35e000640 1 -- 172.21.15.67:0/3611954456 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7fe354018e20 con 0x7fe34c004800 2024-03-09T08:58:14.081 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":46,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:58:14.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 -- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe34806cb30 msgr2=0x7fe34806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe34806cb30 0x7fe34806eff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe358009010 tx=0x7fe358009040 comp rx=0 tx=0).stop 2024-03-09T08:58:14.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 -- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c004800 msgr2=0x7fe36406bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c004800 0x7fe36406bac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe3540186b0 tx=0x7fe354018b70 comp rx=0 tx=0).stop 2024-03-09T08:58:14.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 -- 172.21.15.67:0/3611954456 shutdown_connections 2024-03-09T08:58:14.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe34806cb30 0x7fe34806eff0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe34c0053c0 0x7fe364152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe34c004800 0x7fe36406bac0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 --2- 172.21.15.67:0/3611954456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe34c003a20 0x7fe36414dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 -- 172.21.15.67:0/3611954456 >> 172.21.15.67:0/3611954456 conn(0x7fe3640732d0 msgr2=0x7fe364158340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:14.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 -- 172.21.15.67:0/3611954456 shutdown_connections 2024-03-09T08:58:14.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.084+0000 7fe36b200640 1 -- 172.21.15.67:0/3611954456 wait complete. 2024-03-09T08:58:14.102 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T08:58:14.103 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T08:58:14.103 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:58:14.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.304+0000 7f3062600640 1 Processor -- start 2024-03-09T08:58:14.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.304+0000 7f3062600640 1 -- start start 2024-03-09T08:58:14.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.305+0000 7f3062600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c0ac9a0 0x7f305c1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.305+0000 7f3062600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f305c071190 0x7f305c151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.305+0000 7f3062600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c0c8200 con 0x7f305c0c2780 2024-03-09T08:58:14.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.305+0000 7f3062600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305c06d590 con 0x7f305c071190 2024-03-09T08:58:14.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.305+0000 7f3062600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c0c3740 con 0x7f305c0ac9a0 2024-03-09T08:58:14.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.306+0000 7f305b400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c0ac9a0 0x7f305c1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.306+0000 7f305aa00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f305c0c2780 0x7f305c154190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:41864/0 (socket says 172.21.15.67:41864) 2024-03-09T08:58:14.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.306+0000 7f305aa00640 1 -- 172.21.15.67:0/1277913939 learned_addr learned my addr 172.21.15.67:0/1277913939 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:58:14.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.306+0000 7f305be00640 1 --2- 172.21.15.67:0/1277913939 >> v2:172.21.15.134:3300/0 conn(0x7f305c071190 0x7f305c151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.307+0000 7f305a000640 1 -- 172.21.15.67:0/1277913939 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542951333 0 0) 0x7f305c0c8200 con 0x7f305c0c2780 2024-03-09T08:58:14.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.307+0000 7f305a000640 1 -- 172.21.15.67:0/1277913939 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3040003520 con 0x7f305c0c2780 2024-03-09T08:58:14.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.307+0000 7f305b400640 1 -- 172.21.15.67:0/1277913939 >> v2:172.21.15.134:3300/0 conn(0x7f305c071190 msgr2=0x7f305c151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:14.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.307+0000 7f305b400640 1 --2- 172.21.15.67:0/1277913939 >> v2:172.21.15.134:3300/0 conn(0x7f305c071190 0x7f305c151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.307+0000 7f305b400640 1 -- 172.21.15.67:0/1277913939 >> v1:172.21.15.134:6789/0 conn(0x7f305c0c2780 legacy=0x7f305c154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.307+0000 7f305b400640 1 -- 172.21.15.67:0/1277913939 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c0c3990 con 0x7f305c0ac9a0 2024-03-09T08:58:14.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.307+0000 7f305be00640 1 --2- 172.21.15.67:0/1277913939 >> v2:172.21.15.134:3300/0 conn(0x7f305c071190 0x7f305c151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:58:14.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.310+0000 7f305b400640 1 --2- 172.21.15.67:0/1277913939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c0ac9a0 0x7f305c1510e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f304c0097a0 tx=0x7f304c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=477089a33cf781fe server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:14.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.310+0000 7f305a000640 1 -- 172.21.15.67:0/1277913939 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c031780 con 0x7f305c0ac9a0 2024-03-09T08:58:14.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.311+0000 7f305a000640 1 -- 172.21.15.67:0/1277913939 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304c037070 con 0x7f305c0ac9a0 2024-03-09T08:58:14.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.311+0000 7f305a000640 1 -- 172.21.15.67:0/1277913939 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c031b20 con 0x7f305c0ac9a0 2024-03-09T08:58:14.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.311+0000 7f3062600640 1 -- 172.21.15.67:0/1277913939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c0ac9a0 msgr2=0x7f305c1510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.311+0000 7f3062600640 1 --2- 172.21.15.67:0/1277913939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c0ac9a0 0x7f305c1510e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f304c0097a0 tx=0x7f304c02b820 comp rx=0 tx=0).stop 2024-03-09T08:58:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.311+0000 7f3062600640 1 -- 172.21.15.67:0/1277913939 shutdown_connections 2024-03-09T08:58:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.312+0000 7f3062600640 1 --2- 172.21.15.67:0/1277913939 >> v2:172.21.15.134:3300/0 conn(0x7f305c071190 0x7f305c151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.312+0000 7f3062600640 1 --2- 172.21.15.67:0/1277913939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c0ac9a0 0x7f305c1510e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.312+0000 7f3062600640 1 -- 172.21.15.67:0/1277913939 >> 172.21.15.67:0/1277913939 conn(0x7f305c0c7370 msgr2=0x7f305c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.312+0000 7f3062600640 1 -- 172.21.15.67:0/1277913939 shutdown_connections 2024-03-09T08:58:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.312+0000 7f3062600640 1 -- 172.21.15.67:0/1277913939 wait complete. 2024-03-09T08:58:14.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.315+0000 7f3062600640 1 Processor -- start 2024-03-09T08:58:14.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.315+0000 7f3062600640 1 -- start start 2024-03-09T08:58:14.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.316+0000 7f3062600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c071190 0x7f305c0abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.316+0000 7f3062600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c0ac9a0 0x7f305c151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.316+0000 7f3062600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c0c2780 0x7f305c147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.316+0000 7f3062600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c0acd80 con 0x7f305c071190 2024-03-09T08:58:14.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.316+0000 7f3062600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c070c00 con 0x7f305c0c2780 2024-03-09T08:58:14.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.316+0000 7f3062600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c0acf00 con 0x7f305c0ac9a0 2024-03-09T08:58:14.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.317+0000 7f305b400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c071190 msgr2=0x7f305c0abd20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T08:58:14.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.317+0000 7f305b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c071190 0x7f305c0abd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:14.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.317+0000 7f305be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c0c2780 0x7f305c147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.317+0000 7f305be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c0c2780 0x7f305c147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:51284/0 (socket says 172.21.15.67:51284) 2024-03-09T08:58:14.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.317+0000 7f305be00640 1 -- 172.21.15.67:0/224215327 learned_addr learned my addr 172.21.15.67:0/224215327 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:14.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.318+0000 7f305aa00640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c0ac9a0 0x7f305c151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.318+0000 7f305be00640 1 -- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c0ac9a0 msgr2=0x7f305c151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.318+0000 7f305be00640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c0ac9a0 0x7f305c151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.318+0000 7f305be00640 1 -- 172.21.15.67:0/224215327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c071190 msgr2=0x7f305c0abd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:58:14.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.318+0000 7f305be00640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c071190 0x7f305c0abd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.318+0000 7f305be00640 1 -- 172.21.15.67:0/224215327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c0ab490 con 0x7f305c0c2780 2024-03-09T08:58:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.319+0000 7f305be00640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c0c2780 0x7f305c147750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f305400efc0 tx=0x7f3054009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.320+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054018030 con 0x7f305c0c2780 2024-03-09T08:58:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.320+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305400cde0 con 0x7f305c0c2780 2024-03-09T08:58:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.320+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054003bb0 con 0x7f305c0c2780 2024-03-09T08:58:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.320+0000 7f3062600640 1 -- 172.21.15.67:0/224215327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305c1ebcd0 con 0x7f305c0c2780 2024-03-09T08:58:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.320+0000 7f3062600640 1 -- 172.21.15.67:0/224215327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c1ec290 con 0x7f305c0c2780 2024-03-09T08:58:14.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.322+0000 7f3062600640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1ec7b0 0x7f305c147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.322+0000 7f3062600640 1 -- 172.21.15.67:0/224215327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c14e900 con 0x7f305c1ec7b0 2024-03-09T08:58:14.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.322+0000 7f305b400640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1ec7b0 0x7f305c147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.323+0000 7f305b400640 1 -- 172.21.15.67:0/224215327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f305c1481d0 con 0x7f305c1ec7b0 2024-03-09T08:58:14.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.324+0000 7f305b400640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1ec7b0 0x7f305c147c90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f305c06ea30 tx=0x7f304c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:14.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.324+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c006e10 con 0x7f305c1ec7b0 2024-03-09T08:58:14.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.325+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f304c037040 con 0x7f305c1ec7b0 2024-03-09T08:58:14.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.325+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1ec7b0 msgr2=0x7f305c147c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.325+0000 7f3053400640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1ec7b0 0x7f305c147c90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f305c06ea30 tx=0x7f304c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.325+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1ec7b0 msgr2=0x7f305c147c90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:14.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.325+0000 7f3053400640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1ec7b0 0x7f305c147c90 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f305c06ea30 tx=0x7f304c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.327+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f305401b070 con 0x7f305c0c2780 2024-03-09T08:58:14.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.328+0000 7f3053400640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f303c06afb0 0x7f303c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.328+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f305408b060 con 0x7f305c0c2780 2024-03-09T08:58:14.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.329+0000 7f305b400640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f303c06afb0 0x7f303c06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.330+0000 7f305b400640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f303c06afb0 0x7f303c06d470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f305c0680b0 tx=0x7f304c041000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:14.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.335+0000 7f3062600640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c14ff20 0x7f305c1ef050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.335+0000 7f3062600640 1 -- 172.21.15.67:0/224215327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c1ec4d0 con 0x7f305c14ff20 2024-03-09T08:58:14.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.336+0000 7f305aa00640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c14ff20 0x7f305c1ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.336+0000 7f305aa00640 1 -- 172.21.15.67:0/224215327 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f305c1ebae0 con 0x7f305c14ff20 2024-03-09T08:58:14.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.337+0000 7f305aa00640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c14ff20 0x7f305c1ef050 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f305c0c6bb0 tx=0x7f304800e890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:14.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.337+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048005ac0 con 0x7f305c14ff20 2024-03-09T08:58:14.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5034 (secure 0 0 0) 0x7f3048005ac0 con 0x7f305c14ff20 2024-03-09T08:58:14.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c14ff20 msgr2=0x7f305c1ef050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3053400640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c14ff20 0x7f305c1ef050 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f305c0c6bb0 tx=0x7f304800e890 comp rx=0 tx=0).stop 2024-03-09T08:58:14.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3053400640 1 -- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c14ff20 msgr2=0x7f305c1ef050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:14.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3053400640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c14ff20 0x7f305c1ef050 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f305c0c6bb0 tx=0x7f304800e890 comp rx=0 tx=0).stop 2024-03-09T08:58:14.338 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:58:14.338 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:58:14.338 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:58:14.338 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:58:14.339 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 46, 2024-03-09T08:58:14.339 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:58:14.339 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:58:14.339 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:58:14.339 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.339 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:58:14.339 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:14.339 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:58:14.339 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:58:14.340 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:58:14.342 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:14.343 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.345 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.346 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:14.347 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:14.348 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:58:14.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:58:14.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.351 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:58:14.352 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:58:14.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3062600640 1 -- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f303c06afb0 msgr2=0x7f303c06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3062600640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f303c06afb0 0x7f303c06d470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f305c0680b0 tx=0x7f304c041000 comp rx=0 tx=0).stop 2024-03-09T08:58:14.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3062600640 1 -- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c0c2780 msgr2=0x7f305c147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3062600640 1 --2- 172.21.15.67:0/224215327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c0c2780 0x7f305c147750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f305400efc0 tx=0x7f3054009f90 comp rx=0 tx=0).stop 2024-03-09T08:58:14.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f305be00640 1 -- 172.21.15.67:0/224215327 reap_dead start 2024-03-09T08:58:14.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f305be00640 1 -- 172.21.15.67:0/224215327 reap_dead start 2024-03-09T08:58:14.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3062600640 1 -- 172.21.15.67:0/224215327 shutdown_connections 2024-03-09T08:58:14.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3062600640 1 -- 172.21.15.67:0/224215327 >> 172.21.15.67:0/224215327 conn(0x7f305c0c7370 msgr2=0x7f305c0c07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:14.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3062600640 1 -- 172.21.15.67:0/224215327 shutdown_connections 2024-03-09T08:58:14.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.338+0000 7f3062600640 1 -- 172.21.15.67:0/224215327 wait complete. 2024-03-09T08:58:14.354 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:58:14.354 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T08:58:14.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:58:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.564+0000 7f13ab200640 1 Processor -- start 2024-03-09T08:58:14.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.564+0000 7f13ab200640 1 -- start start 2024-03-09T08:58:14.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.566+0000 7f13ab200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f13a406b040 0x7f13a40c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.566+0000 7f13ab200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f13a40717f0 0x7f13a406b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.566+0000 7f13ab200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a4159240 0x7f13a41596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.566+0000 7f13ab200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a4071d40 con 0x7f13a406b040 2024-03-09T08:58:14.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.566+0000 7f13ab200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a4072ba0 con 0x7f13a40717f0 2024-03-09T08:58:14.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.566+0000 7f13ab200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a414ff60 con 0x7f13a4159240 2024-03-09T08:58:14.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.566+0000 7f13a9e00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f13a406b040 msgr2=0x7f13a40c3630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T08:58:14.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.566+0000 7f13a9e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f13a406b040 0x7f13a40c3630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.567+0000 7f13a9400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f13a40717f0 0x7f13a406b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.567+0000 7f13a9400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f13a40717f0 0x7f13a406b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:51312/0 (socket says 172.21.15.67:51312) 2024-03-09T08:58:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.567+0000 7f13a9400640 1 -- 172.21.15.67:0/3477606523 learned_addr learned my addr 172.21.15.67:0/3477606523 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.567+0000 7f13aa800640 1 --2- 172.21.15.67:0/3477606523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a4159240 0x7f13a41596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.568+0000 7f13a9400640 1 -- 172.21.15.67:0/3477606523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a4159240 msgr2=0x7f13a41596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.568+0000 7f13a9400640 1 --2- 172.21.15.67:0/3477606523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a4159240 0x7f13a41596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.568+0000 7f13a9400640 1 -- 172.21.15.67:0/3477606523 >> v2:172.21.15.67:3300/0 conn(0x7f13a406b040 msgr2=0x7f13a40c3630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.568+0000 7f13a9400640 1 --2- 172.21.15.67:0/3477606523 >> v2:172.21.15.67:3300/0 conn(0x7f13a406b040 0x7f13a40c3630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.568+0000 7f13a9400640 1 -- 172.21.15.67:0/3477606523 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a4071f90 con 0x7f13a40717f0 2024-03-09T08:58:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.569+0000 7f13a9400640 1 --2- 172.21.15.67:0/3477606523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a40717f0 0x7f13a406b420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f13a40ae610 tx=0x7f139402b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc64575ecdb9ace5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.570+0000 7f13a8a00640 1 -- 172.21.15.67:0/3477606523 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394035020 con 0x7f13a40717f0 2024-03-09T08:58:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.570+0000 7f13a8a00640 1 -- 172.21.15.67:0/3477606523 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1394031800 con 0x7f13a40717f0 2024-03-09T08:58:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.570+0000 7f13a8a00640 1 -- 172.21.15.67:0/3477606523 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394031bd0 con 0x7f13a40717f0 2024-03-09T08:58:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.571+0000 7f13ab200640 1 -- 172.21.15.67:0/3477606523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a40717f0 msgr2=0x7f13a406b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.571+0000 7f13ab200640 1 --2- 172.21.15.67:0/3477606523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a40717f0 0x7f13a406b420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f13a40ae610 tx=0x7f139402b8a0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.571+0000 7f13ab200640 1 -- 172.21.15.67:0/3477606523 shutdown_connections 2024-03-09T08:58:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.571+0000 7f13ab200640 1 --2- 172.21.15.67:0/3477606523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a4159240 0x7f13a41596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.571+0000 7f13ab200640 1 --2- 172.21.15.67:0/3477606523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a40717f0 0x7f13a406b420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.571+0000 7f13ab200640 1 --2- 172.21.15.67:0/3477606523 >> v2:172.21.15.67:3300/0 conn(0x7f13a406b040 0x7f13a40c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.571+0000 7f13ab200640 1 -- 172.21.15.67:0/3477606523 >> 172.21.15.67:0/3477606523 conn(0x7f13a40c8480 msgr2=0x7f13a40c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.571+0000 7f13ab200640 1 -- 172.21.15.67:0/3477606523 shutdown_connections 2024-03-09T08:58:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.571+0000 7f13ab200640 1 -- 172.21.15.67:0/3477606523 wait complete. 2024-03-09T08:58:14.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.573+0000 7f13ab200640 1 Processor -- start 2024-03-09T08:58:14.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.574+0000 7f13ab200640 1 -- start start 2024-03-09T08:58:14.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13ab200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a406b040 0x7f13a414b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13ab200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a40717f0 0x7f13a41eb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13ab200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13a4159240 0x7f13a41ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13ab200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a4072830 con 0x7f13a4159240 2024-03-09T08:58:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13ab200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a4072550 con 0x7f13a40717f0 2024-03-09T08:58:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13ab200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a4159c10 con 0x7f13a406b040 2024-03-09T08:58:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13aa800640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13a4159240 msgr2=0x7f13a41ebe90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T08:58:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13aa800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13a4159240 0x7f13a41ebe90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13a9e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a406b040 0x7f13a414b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13a9400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a40717f0 0x7f13a41eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13a9400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a40717f0 0x7f13a41eb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:51328/0 (socket says 172.21.15.67:51328) 2024-03-09T08:58:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13a9e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a406b040 0x7f13a414b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:37830/0 (socket says 172.21.15.67:37830) 2024-03-09T08:58:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.575+0000 7f13a9400640 1 -- 172.21.15.67:0/2125422498 learned_addr learned my addr 172.21.15.67:0/2125422498 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:14.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.576+0000 7f13a9e00640 1 -- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a40717f0 msgr2=0x7f13a41eb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.576+0000 7f13a9e00640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13a40717f0 0x7f13a41eb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.576+0000 7f13a9e00640 1 -- 172.21.15.67:0/2125422498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13a4159240 msgr2=0x7f13a41ebe90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:58:14.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.576+0000 7f13a9e00640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13a4159240 0x7f13a41ebe90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.576+0000 7f13a9e00640 1 -- 172.21.15.67:0/2125422498 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a414e9d0 con 0x7f13a406b040 2024-03-09T08:58:14.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.577+0000 7f13a9e00640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a406b040 0x7f13a414b770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f139800b430 tx=0x7f139800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:14.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.578+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398016070 con 0x7f13a406b040 2024-03-09T08:58:14.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.578+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398003de0 con 0x7f13a406b040 2024-03-09T08:58:14.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.578+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13980052b0 con 0x7f13a406b040 2024-03-09T08:58:14.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.578+0000 7f13ab200640 1 -- 172.21.15.67:0/2125422498 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a41e1780 con 0x7f13a406b040 2024-03-09T08:58:14.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.578+0000 7f13ab200640 1 -- 172.21.15.67:0/2125422498 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a414c240 con 0x7f13a406b040 2024-03-09T08:58:14.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.579+0000 7f13ab200640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41df2b0 0x7f13a4143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.579+0000 7f13ab200640 1 -- 172.21.15.67:0/2125422498 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a414c0a0 con 0x7f13a41df2b0 2024-03-09T08:58:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.580+0000 7f13a9400640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41df2b0 0x7f13a4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.581+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1398019070 con 0x7f13a406b040 2024-03-09T08:58:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.582+0000 7f13a9400640 1 -- 172.21.15.67:0/2125422498 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13a404f3c0 con 0x7f13a41df2b0 2024-03-09T08:58:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.582+0000 7f139e000640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f138806b480 0x7f138806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.582+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f1398089880 con 0x7f13a406b040 2024-03-09T08:58:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.582+0000 7f13aa800640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f138806b480 0x7f138806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.583+0000 7f13aa800640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f138806b480 0x7f138806d940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009ac0 tx=0x7f13a000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.586+0000 7f13a9400640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41df2b0 0x7f13a4143640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f13940090b0 tx=0x7f1394004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.586+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394049070 con 0x7f13a41df2b0 2024-03-09T08:58:14.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.587+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f139403ac00 con 0x7f13a41df2b0 2024-03-09T08:58:14.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.587+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41df2b0 msgr2=0x7f13a4143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.587+0000 7f139e000640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41df2b0 0x7f13a4143640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f13940090b0 tx=0x7f1394004730 comp rx=0 tx=0).stop 2024-03-09T08:58:14.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.587+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41df2b0 msgr2=0x7f13a4143640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:14.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.587+0000 7f139e000640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41df2b0 0x7f13a4143640 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f13940090b0 tx=0x7f1394004730 comp rx=0 tx=0).stop 2024-03-09T08:58:14.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.598+0000 7f13ab200640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41ec3d0 0x7f13a4143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:14.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.598+0000 7f13ab200640 1 -- 172.21.15.67:0/2125422498 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a40c3410 con 0x7f13a41ec3d0 2024-03-09T08:58:14.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.598+0000 7f13a9400640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41ec3d0 0x7f13a4143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.599+0000 7f13a9400640 1 -- 172.21.15.67:0/2125422498 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f13a4070020 con 0x7f13a41ec3d0 2024-03-09T08:58:14.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.600+0000 7f13a9400640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41ec3d0 0x7f13a4143b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13a414d4a0 tx=0x7f1394031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:14.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.600+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394051070 con 0x7f13a41ec3d0 2024-03-09T08:58:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.600+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5175 (secure 0 0 0) 0x7f1394009490 con 0x7f13a41ec3d0 2024-03-09T08:58:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.600+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41ec3d0 msgr2=0x7f13a4143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.600+0000 7f139e000640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41ec3d0 0x7f13a4143b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13a414d4a0 tx=0x7f1394031000 comp rx=0 tx=0).stop 2024-03-09T08:58:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.600+0000 7f139e000640 1 -- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41ec3d0 msgr2=0x7f13a4143b80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.600+0000 7f139e000640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a41ec3d0 0x7f13a4143b80 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13a414d4a0 tx=0x7f1394031000 comp rx=0 tx=0).stop 2024-03-09T08:58:14.601 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:58:14.602 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:58:14.602 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:58:14.602 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:58:14.602 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 46, 2024-03-09T08:58:14.602 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:58:14.602 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:58:14.602 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:58:14.602 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.606 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:58:14.606 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:14.606 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:58:14.606 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:58:14.606 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:14.606 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:14.607 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:58:14.608 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:14.609 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:14.610 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:14.610 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:14.610 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:14.610 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:14.610 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:14.610 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:14.610 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:14.610 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:14.610 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:58:14.611 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.612 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.612 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:14.612 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:58:14.612 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.612 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.612 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.612 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:58:14.612 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.614 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:14.615 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.616 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:14.617 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:58:14.618 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.619 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:14.619 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:58:14.619 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:14.619 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:14.619 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:14.619 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:14.619 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:14.619 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:14.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:14.619 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:58:14.620 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:58:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.601+0000 7f13ab200640 1 -- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f138806b480 msgr2=0x7f138806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.601+0000 7f13ab200640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f138806b480 0x7f138806d940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009ac0 tx=0x7f13a000b040 comp rx=0 tx=0).stop 2024-03-09T08:58:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.601+0000 7f13ab200640 1 -- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a406b040 msgr2=0x7f13a414b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.601+0000 7f13ab200640 1 --2- 172.21.15.67:0/2125422498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13a406b040 0x7f13a414b770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f139800b430 tx=0x7f139800b8f0 comp rx=0 tx=0).stop 2024-03-09T08:58:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.601+0000 7f13aa800640 1 -- 172.21.15.67:0/2125422498 reap_dead start 2024-03-09T08:58:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.601+0000 7f13aa800640 1 -- 172.21.15.67:0/2125422498 reap_dead start 2024-03-09T08:58:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.601+0000 7f13ab200640 1 -- 172.21.15.67:0/2125422498 shutdown_connections 2024-03-09T08:58:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.601+0000 7f13ab200640 1 -- 172.21.15.67:0/2125422498 >> 172.21.15.67:0/2125422498 conn(0x7f13a40c8480 msgr2=0x7f13a40c55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.601+0000 7f13ab200640 1 -- 172.21.15.67:0/2125422498 shutdown_connections 2024-03-09T08:58:14.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:14.601+0000 7f13ab200640 1 -- 172.21.15.67:0/2125422498 wait complete. 2024-03-09T08:58:14.623 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:58:14.623 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T08:58:18.078 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T08:58:18.078 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T08:58:18.079 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:58:18.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T08:58:18.086 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T08:58:18.087 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T08:58:18.087 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:58:18.087 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T08:58:18.182 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2024-03-09T08:58:18.183 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2024-03-09T08:58:18.201 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2024-03-09T08:58:18.201 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2024-03-09T08:58:18.251 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T08:58:18.251 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:18.256 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/bar 2024-03-09T08:58:18.272 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T08:58:18.272 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:18.276 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/bar 2024-03-09T08:58:18.295 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/foo 2024-03-09T08:58:18.321 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/foo 2024-03-09T08:58:18.322 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/foo 2024-03-09T08:58:18.331 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:18.336 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/bar/a 2024-03-09T08:58:18.341 INFO:tasks.workunit.client.0.smithi067.stdout: File: .snap/bar/a 2024-03-09T08:58:18.342 INFO:tasks.workunit.client.0.smithi067.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-09T08:58:18.342 INFO:tasks.workunit.client.0.smithi067.stdout:Device: fch/252d Inode: 1099511633621 Links: 2 2024-03-09T08:58:18.342 INFO:tasks.workunit.client.0.smithi067.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-09T08:58:18.342 INFO:tasks.workunit.client.0.smithi067.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-09T08:58:18.342 INFO:tasks.workunit.client.0.smithi067.stdout:Access: 2024-03-09 08:58:18.255677569 +0000 2024-03-09T08:58:18.342 INFO:tasks.workunit.client.0.smithi067.stdout:Modify: 2024-03-09 08:58:18.255677569 +0000 2024-03-09T08:58:18.342 INFO:tasks.workunit.client.0.smithi067.stdout:Change: 2024-03-09 08:58:18.255677569 +0000 2024-03-09T08:58:18.342 INFO:tasks.workunit.client.0.smithi067.stdout: Birth: 2024-03-09 08:58:18.255677569 +0000 2024-03-09T08:58:18.343 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/bar 2024-03-09T08:58:18.352 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/foo 2024-03-09T08:58:18.364 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:18.632 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/bar/a 2024-03-09T08:58:18.641 INFO:tasks.workunit.client.1.smithi134.stdout: File: .snap/bar/a 2024-03-09T08:58:18.641 INFO:tasks.workunit.client.1.smithi134.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-09T08:58:18.641 INFO:tasks.workunit.client.1.smithi134.stdout:Device: fch/252d Inode: 1099511633618 Links: 2 2024-03-09T08:58:18.641 INFO:tasks.workunit.client.1.smithi134.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-09T08:58:18.641 INFO:tasks.workunit.client.1.smithi134.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-09T08:58:18.641 INFO:tasks.workunit.client.1.smithi134.stdout:Access: 2024-03-09 08:58:18.273818008 +0000 2024-03-09T08:58:18.641 INFO:tasks.workunit.client.1.smithi134.stdout:Modify: 2024-03-09 08:58:18.273818008 +0000 2024-03-09T08:58:18.642 INFO:tasks.workunit.client.1.smithi134.stdout:Change: 2024-03-09 08:58:18.273818008 +0000 2024-03-09T08:58:18.642 INFO:tasks.workunit.client.1.smithi134.stdout: Birth: 2024-03-09 08:58:18.273818008 +0000 2024-03-09T08:58:18.642 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/bar 2024-03-09T08:58:18.663 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T08:58:18.663 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T08:58:18.665 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:58:18.665 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T08:58:18.762 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2024-03-09T08:58:18.763 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-double-null.sh 2024-03-09T08:58:18.826 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T08:58:18.827 INFO:tasks.workunit.client.1.smithi134.stderr:++ seq 1 20 2024-03-09T08:58:18.830 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:18.830 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:18.835 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:18.835 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:18.881 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:18.918 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:18.921 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:18.994 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T08:58:18.994 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T08:58:18.995 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:58:18.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T08:58:19.111 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2024-03-09T08:58:19.111 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-double-null.sh 2024-03-09T08:58:19.181 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T08:58:19.182 INFO:tasks.workunit.client.0.smithi067.stderr:++ seq 1 20 2024-03-09T08:58:19.184 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:19.184 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:19.185 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:19.188 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:19.188 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:19.211 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:19.211 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:19.211 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:19.212 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:19.212 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:19.212 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:19.212 INFO:tasks.workunit.client.1.smithi134.stdout: 45416 ? 00:00:00 ps 2024-03-09T08:58:19.213 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:19.227 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:19.237 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:19.243 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:19.255 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:19.255 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:19.256 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:19.258 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:19.263 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:19.263 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:19.298 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:19.332 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:19.334 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:19.420 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:19.439 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:19.439 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:19.439 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:19.439 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:19.439 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:19.439 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:19.439 INFO:tasks.workunit.client.1.smithi134.stdout: 45426 ? 00:00:00 ps 2024-03-09T08:58:19.441 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:19.467 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:19.467 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:19.474 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:19.486 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:19.486 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:19.493 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:19.493 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:19.495 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:19.495 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:19.495 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:19.496 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:19.496 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:19.496 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:19.496 INFO:tasks.workunit.client.0.smithi067.stdout: 52796 ? 00:00:00 ps 2024-03-09T08:58:19.497 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:19.522 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:19.528 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:19.533 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:19.541 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:19.541 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:19.549 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:19.550 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:19.568 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:19.572 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:19.589 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:19.620 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:19.620 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:19.624 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:19.640 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:19.641 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:19.641 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:19.641 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:19.641 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:19.641 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:19.641 INFO:tasks.workunit.client.1.smithi134.stdout: 45436 ? 00:00:00 ps 2024-03-09T08:58:19.642 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:19.670 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:19.676 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:19.692 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:19.692 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:19.698 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:19.700 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:19.700 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:19.723 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:19.723 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:19.723 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:19.724 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:19.724 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:19.724 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:19.724 INFO:tasks.workunit.client.0.smithi067.stdout: 52807 ? 00:00:00 ps 2024-03-09T08:58:19.724 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:19.744 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:19.748 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:19.756 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:19.771 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:19.771 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:19.778 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:19.778 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:19.779 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:19.781 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:19.816 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:19.852 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:19.856 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:19.872 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:19.892 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:19.892 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:19.892 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:19.893 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:19.893 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:19.893 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:19.893 INFO:tasks.workunit.client.1.smithi134.stdout: 45447 ? 00:00:00 ps 2024-03-09T08:58:19.894 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:19.924 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:19.931 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:19.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:19.947 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:19.948 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:19.954 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:19.954 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:19.958 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:19.959 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:19.959 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:19.959 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:19.959 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:19.959 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:19.959 INFO:tasks.workunit.client.0.smithi067.stdout: 52817 ? 00:00:00 ps 2024-03-09T08:58:19.961 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:19.983 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:19.986 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:19.990 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:20.007 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:20.012 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:20.062 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:20.083 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:20.083 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:20.083 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:20.083 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:20.083 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:20.083 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:20.083 INFO:tasks.workunit.client.1.smithi134.stdout: 45457 ? 00:00:00 ps 2024-03-09T08:58:20.084 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:20.111 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:20.119 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:20.136 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:20.137 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:20.144 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:20.144 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:20.200 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:20.219 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:20.222 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:20.335 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:20.350 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:20.351 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:20.351 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:20.351 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:20.351 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:20.351 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:20.351 INFO:tasks.workunit.client.1.smithi134.stdout: 45467 ? 00:00:00 ps 2024-03-09T08:58:20.352 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:20.376 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:20.382 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:20.398 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:20.399 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:20.405 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:20.405 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:20.440 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:20.447 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:20.447 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:20.454 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:20.454 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:20.464 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:20.469 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:20.499 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:20.532 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:20.537 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:20.549 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:20.567 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:20.567 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:20.567 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:20.567 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:20.567 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:20.568 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:20.568 INFO:tasks.workunit.client.1.smithi134.stdout: 45477 ? 00:00:00 ps 2024-03-09T08:58:20.569 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:20.591 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:20.599 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:20.611 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:20.611 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:20.617 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:20.617 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:20.640 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:20.649 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:20.663 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:20.663 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:20.663 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:20.663 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:20.663 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:20.663 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:20.663 INFO:tasks.workunit.client.0.smithi067.stdout: 52827 ? 00:00:00 ps 2024-03-09T08:58:20.665 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:20.676 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:20.676 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:20.680 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:20.684 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:20.714 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:20.714 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:20.721 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:20.721 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:20.751 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:20.765 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:20.774 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:20.777 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:20.783 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:20.783 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:20.783 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:20.783 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:20.783 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:20.783 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:20.784 INFO:tasks.workunit.client.1.smithi134.stdout: 45487 ? 00:00:00 ps 2024-03-09T08:58:20.785 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:20.809 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:20.815 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:20.831 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:20.832 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:20.841 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:20.842 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:20.857 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:20.870 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:20.883 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:20.883 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:20.883 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:20.883 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:20.883 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:20.883 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:20.883 INFO:tasks.workunit.client.0.smithi067.stdout: 52837 ? 00:00:00 ps 2024-03-09T08:58:20.886 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:20.894 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:20.896 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:20.904 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:20.912 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:20.929 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:20.929 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:20.938 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:20.938 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:20.968 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:20.981 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:20.990 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:20.995 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:20.999 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:20.999 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:20.999 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:20.999 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:20.999 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:20.999 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:20.999 INFO:tasks.workunit.client.1.smithi134.stdout: 45497 ? 00:00:00 ps 2024-03-09T08:58:21.000 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:21.025 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:21.031 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:21.044 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:21.044 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:21.051 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:21.051 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:21.064 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:21.082 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:21.083 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:21.084 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:21.084 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:21.084 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:21.084 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:21.084 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:21.084 INFO:tasks.workunit.client.0.smithi067.stdout: 52847 ? 00:00:00 ps 2024-03-09T08:58:21.086 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:21.105 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:21.108 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:21.111 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:21.120 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:21.136 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:21.136 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:21.146 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:21.146 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:21.164 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:21.192 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:21.193 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:21.193 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:21.193 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:21.194 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:21.194 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:21.194 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:21.194 INFO:tasks.workunit.client.1.smithi134.stdout: 45507 ? 00:00:00 ps 2024-03-09T08:58:21.194 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:21.228 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:21.233 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:21.258 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:21.262 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:21.281 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:21.281 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:21.286 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:21.287 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:21.325 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:21.357 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:21.361 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:21.398 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:21.412 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:21.412 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:21.412 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:21.412 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:21.412 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:21.413 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:21.413 INFO:tasks.workunit.client.0.smithi067.stdout: 52857 ? 00:00:00 ps 2024-03-09T08:58:21.413 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:21.431 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:21.436 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:21.446 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:21.446 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:21.451 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:21.451 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:21.477 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:21.494 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:21.496 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:21.524 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:21.542 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:21.542 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:21.543 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:21.543 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:21.543 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:21.543 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:21.543 INFO:tasks.workunit.client.1.smithi134.stdout: 45517 ? 00:00:00 ps 2024-03-09T08:58:21.544 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:21.615 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:21.638 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:21.638 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:21.638 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:21.638 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:21.638 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:21.638 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:21.639 INFO:tasks.workunit.client.0.smithi067.stdout: 52867 ? 00:00:00 ps 2024-03-09T08:58:21.640 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:21.662 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:21.667 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:21.680 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:21.680 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:21.686 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:21.686 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:21.715 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:21.734 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:21.736 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:21.807 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:21.824 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:21.824 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:21.824 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:21.824 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:21.824 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:21.824 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:21.824 INFO:tasks.workunit.client.0.smithi067.stdout: 52877 ? 00:00:00 ps 2024-03-09T08:58:21.826 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:21.846 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:21.856 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:21.874 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:21.874 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:21.882 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:21.883 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:21.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:21.936 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:21.940 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:22.099 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:22.118 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:22.119 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:22.119 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:22.119 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:22.119 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:22.119 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:22.119 INFO:tasks.workunit.client.0.smithi067.stdout: 52888 ? 00:00:00 ps 2024-03-09T08:58:22.121 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:22.289 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:22.300 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:22.327 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:22.328 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:22.335 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:22.335 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:22.376 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:22.396 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:22.400 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:22.733 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:22.753 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:22.753 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:22.753 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:22.753 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:22.753 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:22.753 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:22.753 INFO:tasks.workunit.client.1.smithi134.stdout: 45527 ? 00:00:00 ps 2024-03-09T08:58:22.754 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:22.782 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:22.786 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:22.791 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:22.793 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:22.809 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:22.809 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:22.815 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:22.815 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:22.846 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:22.870 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:22.873 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:23.083 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:23.099 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:23.100 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:23.100 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:23.100 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:23.100 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:23.100 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:23.100 INFO:tasks.workunit.client.0.smithi067.stdout: 52898 ? 00:00:00 ps 2024-03-09T08:58:23.101 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:23.129 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:23.135 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:23.150 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:23.150 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:23.158 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:23.158 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:23.214 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:23.257 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:23.262 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:23.441 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:23.456 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:23.456 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:23.456 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:23.456 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:23.456 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:23.456 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:23.456 INFO:tasks.workunit.client.0.smithi067.stdout: 52908 ? 00:00:00 ps 2024-03-09T08:58:23.457 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:23.471 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:23.475 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:23.483 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:23.483 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:23.487 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:23.488 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:23.510 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:23.510 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:23.516 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:23.516 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:23.526 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:23.558 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:23.559 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:23.561 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:23.589 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:23.593 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:23.658 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:23.677 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:23.677 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:23.677 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:23.678 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:23.678 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:23.678 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:23.678 INFO:tasks.workunit.client.0.smithi067.stdout: 52918 ? 00:00:00 ps 2024-03-09T08:58:23.679 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:23.702 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:23.708 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:23.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:23.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:23.729 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:23.730 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:23.774 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:23.811 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:23.815 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:23.815 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:23.835 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:23.835 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:23.835 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:23.836 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:23.836 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:23.836 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:23.836 INFO:tasks.workunit.client.1.smithi134.stdout: 45537 ? 00:00:00 ps 2024-03-09T08:58:23.837 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:23.863 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:23.867 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:23.870 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:23.882 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:23.882 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:23.888 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:23.888 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:23.888 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:23.888 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:23.888 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:23.888 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:23.888 INFO:tasks.workunit.client.0.smithi067.stdout: 52928 ? 00:00:00 ps 2024-03-09T08:58:23.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:23.890 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:23.890 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:23.916 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:23.921 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:23.931 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:23.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:23.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:23.949 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:23.949 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:23.966 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:23.969 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:23.991 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:24.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:24.033 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:24.056 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:24.076 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:24.076 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:24.076 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:24.077 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:24.077 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:24.077 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:24.077 INFO:tasks.workunit.client.1.smithi134.stdout: 45548 ? 00:00:00 ps 2024-03-09T08:58:24.078 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:24.106 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:24.109 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:24.113 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:24.130 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:24.130 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:24.131 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:24.131 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:24.131 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:24.131 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:24.131 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:24.131 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:24.131 INFO:tasks.workunit.client.0.smithi067.stdout: 52938 ? 00:00:00 ps 2024-03-09T08:58:24.132 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:24.138 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:24.138 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:24.180 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:24.186 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:24.195 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:24.195 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:24.198 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:24.200 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:24.200 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:24.231 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:24.231 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:24.325 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:24.344 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:24.344 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:24.344 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:24.344 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:24.344 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:24.344 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:24.344 INFO:tasks.workunit.client.1.smithi134.stdout: 45558 ? 00:00:00 ps 2024-03-09T08:58:24.346 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:24.375 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:24.382 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:24.399 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:24.399 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:24.408 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:24.408 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:24.450 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:24.484 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:24.488 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:24.567 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:24.585 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:24.585 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:24.585 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:24.585 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:24.585 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:24.585 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:24.585 INFO:tasks.workunit.client.1.smithi134.stdout: 45568 ? 00:00:00 ps 2024-03-09T08:58:24.587 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:24.612 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:24.618 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:24.632 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:24.633 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:24.642 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:24.642 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:24.687 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:24.723 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:24.725 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:24.783 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:24.810 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:24.810 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:24.810 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:24.811 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:24.811 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:24.811 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:24.811 INFO:tasks.workunit.client.1.smithi134.stdout: 45578 ? 00:00:00 ps 2024-03-09T08:58:24.812 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:25.035 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:25.071 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:25.074 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:25.210 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:25.227 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:25.228 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:25.228 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:25.228 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:25.228 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:25.228 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:25.228 INFO:tasks.workunit.client.0.smithi067.stdout: 52949 ? 00:00:00 ps 2024-03-09T08:58:25.229 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:25.292 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:25.297 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:25.306 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:25.306 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:25.310 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:25.310 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:25.343 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:25.375 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:25.378 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:25.420 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:25.426 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:25.438 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:25.438 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:25.442 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:25.443 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:25.464 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:25.482 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:25.484 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:25.569 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:25.584 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:25.584 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:25.584 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:25.584 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:25.584 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:25.584 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:25.584 INFO:tasks.workunit.client.0.smithi067.stdout: 52959 ? 00:00:00 ps 2024-03-09T08:58:25.586 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:25.612 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:25.618 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:25.630 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:25.630 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:25.637 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:25.637 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:25.668 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:25.668 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:25.686 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:25.686 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:25.686 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:25.686 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:25.686 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:25.686 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:25.686 INFO:tasks.workunit.client.1.smithi134.stdout: 45588 ? 00:00:00 ps 2024-03-09T08:58:25.688 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:25.691 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:25.697 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:25.713 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:25.719 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:25.735 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:25.735 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:25.743 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:25.743 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:25.777 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:25.777 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:25.798 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:25.798 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:25.798 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:25.798 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:25.798 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:25.799 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:25.799 INFO:tasks.workunit.client.0.smithi067.stdout: 52969 ? 00:00:00 ps 2024-03-09T08:58:25.800 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:25.801 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:25.805 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:25.824 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:25.832 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:25.849 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:25.849 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:25.857 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:25.857 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:25.891 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:25.908 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:25.927 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:25.928 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:25.928 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:25.928 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:25.928 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:25.928 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:25.928 INFO:tasks.workunit.client.1.smithi134.stdout: 45598 ? 00:00:00 ps 2024-03-09T08:58:25.929 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:25.958 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:25.965 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:25.982 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:25.982 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T08:58:25.989 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:25.989 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat 2024-03-09T08:58:26.028 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:26.056 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a/foo 2024-03-09T08:58:26.059 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T08:58:26.134 INFO:tasks.workunit.client.1.smithi134.stderr:+ ps 2024-03-09T08:58:26.156 INFO:tasks.workunit.client.1.smithi134.stdout: PID TTY TIME CMD 2024-03-09T08:58:26.156 INFO:tasks.workunit.client.1.smithi134.stdout: 1134 ? 00:00:00 systemd 2024-03-09T08:58:26.156 INFO:tasks.workunit.client.1.smithi134.stdout: 1148 ? 00:00:00 (sd-pam) 2024-03-09T08:58:26.157 INFO:tasks.workunit.client.1.smithi134.stdout: 22845 ? 00:00:01 sshd 2024-03-09T08:58:26.157 INFO:tasks.workunit.client.1.smithi134.stdout: 45389 ? 00:00:00 timeout 2024-03-09T08:58:26.157 INFO:tasks.workunit.client.1.smithi134.stdout: 45407 ? 00:00:00 snaptest-double 2024-03-09T08:58:26.157 INFO:tasks.workunit.client.1.smithi134.stdout: 45608 ? 00:00:00 ps 2024-03-09T08:58:26.160 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:26.204 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:26.209 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:26.232 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a/foo 2024-03-09T08:58:26.240 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a 2024-03-09T08:58:26.257 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T08:58:26.257 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T08:58:26.259 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:58:26.259 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T08:58:26.342 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2024-03-09T08:58:26.342 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-estale.sh 2024-03-09T08:58:26.378 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:26.391 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/foo 2024-03-09T08:58:26.394 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:26.394 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:26.394 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:26.395 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:26.395 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:26.395 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:26.395 INFO:tasks.workunit.client.0.smithi067.stdout: 52979 ? 00:00:00 ps 2024-03-09T08:58:26.396 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:26.415 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:26.418 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'We want ENOENT, not ESTALE, here.' 2024-03-09T08:58:26.419 INFO:tasks.workunit.client.1.smithi134.stdout:We want ENOENT, not ESTALE, here. 2024-03-09T08:58:26.419 INFO:tasks.workunit.client.1.smithi134.stderr:++ seq 1 100 2024-03-09T08:58:26.421 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.421 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:26.422 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.422 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/1 2024-03-09T08:58:26.426 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/1': No such file or directory 2024-03-09T08:58:26.427 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.427 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/2 2024-03-09T08:58:26.428 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.430 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T08:58:26.430 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T08:58:26.430 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/2': No such file or directory 2024-03-09T08:58:26.431 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.431 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/3 2024-03-09T08:58:26.431 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.434 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/3': No such file or directory 2024-03-09T08:58:26.435 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.435 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/one 2024-03-09T08:58:26.435 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat 2024-03-09T08:58:26.435 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/4 2024-03-09T08:58:26.435 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.437 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/4': No such file or directory 2024-03-09T08:58:26.438 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.438 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/5 2024-03-09T08:58:26.439 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.441 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/5': No such file or directory 2024-03-09T08:58:26.441 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.442 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/6 2024-03-09T08:58:26.443 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.445 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/6': No such file or directory 2024-03-09T08:58:26.445 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.446 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/7 2024-03-09T08:58:26.447 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.449 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/7': No such file or directory 2024-03-09T08:58:26.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.450 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/8 2024-03-09T08:58:26.450 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.453 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/8': No such file or directory 2024-03-09T08:58:26.453 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.454 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/9 2024-03-09T08:58:26.455 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.457 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/9': No such file or directory 2024-03-09T08:58:26.457 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.457 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/two 2024-03-09T08:58:26.458 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/10 2024-03-09T08:58:26.458 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.461 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/10': No such file or directory 2024-03-09T08:58:26.461 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.462 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/11 2024-03-09T08:58:26.462 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.465 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/11': No such file or directory 2024-03-09T08:58:26.465 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.466 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/12 2024-03-09T08:58:26.467 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.469 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/12': No such file or directory 2024-03-09T08:58:26.469 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.470 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/13 2024-03-09T08:58:26.470 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.472 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/13': No such file or directory 2024-03-09T08:58:26.473 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.473 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/14 2024-03-09T08:58:26.474 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.476 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/14': No such file or directory 2024-03-09T08:58:26.476 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.477 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/15 2024-03-09T08:58:26.478 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.478 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a/foo 2024-03-09T08:58:26.483 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/15': No such file or directory 2024-03-09T08:58:26.483 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.483 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/16 2024-03-09T08:58:26.483 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.484 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/16': No such file or directory 2024-03-09T08:58:26.484 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.484 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/17 2024-03-09T08:58:26.484 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.485 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T08:58:26.486 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/17': No such file or directory 2024-03-09T08:58:26.486 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.487 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/18 2024-03-09T08:58:26.487 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.489 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/18': No such file or directory 2024-03-09T08:58:26.490 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.491 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/19 2024-03-09T08:58:26.491 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.493 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/19': No such file or directory 2024-03-09T08:58:26.494 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.494 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/20 2024-03-09T08:58:26.495 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.497 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/20': No such file or directory 2024-03-09T08:58:26.497 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.498 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/21 2024-03-09T08:58:26.499 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.501 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/21': No such file or directory 2024-03-09T08:58:26.501 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.502 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/22 2024-03-09T08:58:26.502 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.504 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/22': No such file or directory 2024-03-09T08:58:26.505 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.505 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/23 2024-03-09T08:58:26.506 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.508 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/23': No such file or directory 2024-03-09T08:58:26.508 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.509 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/24 2024-03-09T08:58:26.510 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.512 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/24': No such file or directory 2024-03-09T08:58:26.512 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.513 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/25 2024-03-09T08:58:26.513 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.515 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/25': No such file or directory 2024-03-09T08:58:26.516 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.516 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/26 2024-03-09T08:58:26.517 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.519 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/26': No such file or directory 2024-03-09T08:58:26.519 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.520 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/27 2024-03-09T08:58:26.521 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.543 INFO:tasks.workunit.client.0.smithi067.stderr:+ ps 2024-03-09T08:58:26.562 INFO:tasks.workunit.client.0.smithi067.stdout: PID TTY TIME CMD 2024-03-09T08:58:26.562 INFO:tasks.workunit.client.0.smithi067.stdout: 1709 ? 00:00:00 systemd 2024-03-09T08:58:26.562 INFO:tasks.workunit.client.0.smithi067.stdout: 1723 ? 00:00:00 (sd-pam) 2024-03-09T08:58:26.563 INFO:tasks.workunit.client.0.smithi067.stdout: 22817 ? 00:00:02 sshd 2024-03-09T08:58:26.563 INFO:tasks.workunit.client.0.smithi067.stdout: 52770 ? 00:00:00 timeout 2024-03-09T08:58:26.563 INFO:tasks.workunit.client.0.smithi067.stdout: 52788 ? 00:00:00 snaptest-double 2024-03-09T08:58:26.563 INFO:tasks.workunit.client.0.smithi067.stdout: 52989 ? 00:00:00 ps 2024-03-09T08:58:26.564 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-09T08:58:26.591 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a/foo 2024-03-09T08:58:26.599 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a 2024-03-09T08:58:26.613 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T08:58:26.614 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T08:58:26.615 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:58:26.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T08:58:26.729 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2024-03-09T08:58:26.836 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-estale.sh 2024-03-09T08:58:26.837 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/27': No such file or directory 2024-03-09T08:58:26.838 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.838 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/28 2024-03-09T08:58:26.838 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/foo 2024-03-09T08:58:26.927 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/28': No such file or directory 2024-03-09T08:58:26.928 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.929 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/29 2024-03-09T08:58:26.931 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.935 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/29': No such file or directory 2024-03-09T08:58:26.936 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.938 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/30 2024-03-09T08:58:26.939 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.940 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/30': No such file or directory 2024-03-09T08:58:26.959 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.959 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/31 2024-03-09T08:58:26.960 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'We want ENOENT, not ESTALE, here.' 2024-03-09T08:58:26.961 INFO:tasks.workunit.client.0.smithi067.stdout:We want ENOENT, not ESTALE, here. 2024-03-09T08:58:26.961 INFO:tasks.workunit.client.0.smithi067.stderr:++ seq 1 100 2024-03-09T08:58:26.962 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.963 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.964 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/31': No such file or directory 2024-03-09T08:58:26.965 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/1 2024-03-09T08:58:26.965 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:26.966 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/1': No such file or directory 2024-03-09T08:58:26.966 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.966 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/32 2024-03-09T08:58:26.966 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.967 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/32': No such file or directory 2024-03-09T08:58:26.968 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.968 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/2 2024-03-09T08:58:26.968 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:26.969 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/2': No such file or directory 2024-03-09T08:58:26.969 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.969 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/33 2024-03-09T08:58:26.969 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.970 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/33': No such file or directory 2024-03-09T08:58:26.971 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.971 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/3 2024-03-09T08:58:26.971 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:26.972 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/3': No such file or directory 2024-03-09T08:58:26.972 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.972 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/34 2024-03-09T08:58:26.972 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.972 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/34': No such file or directory 2024-03-09T08:58:26.974 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.974 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/4 2024-03-09T08:58:26.974 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:26.975 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/4': No such file or directory 2024-03-09T08:58:26.975 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.975 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/35 2024-03-09T08:58:26.975 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.975 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/35': No such file or directory 2024-03-09T08:58:26.976 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.976 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/5 2024-03-09T08:58:26.977 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:26.977 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/5': No such file or directory 2024-03-09T08:58:26.977 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.977 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/36 2024-03-09T08:58:26.977 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.978 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/36': No such file or directory 2024-03-09T08:58:26.978 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.978 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/6 2024-03-09T08:58:26.979 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.979 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/37 2024-03-09T08:58:26.979 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.979 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:26.980 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/37': No such file or directory 2024-03-09T08:58:26.980 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.981 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/38 2024-03-09T08:58:26.982 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.982 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/38': No such file or directory 2024-03-09T08:58:26.982 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/6': No such file or directory 2024-03-09T08:58:26.983 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.983 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/7 2024-03-09T08:58:26.983 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:26.984 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/7': No such file or directory 2024-03-09T08:58:26.984 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.984 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/8 2024-03-09T08:58:26.985 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.985 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/39 2024-03-09T08:58:26.985 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.986 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/39': No such file or directory 2024-03-09T08:58:26.987 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:26.988 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.991 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/8': No such file or directory 2024-03-09T08:58:26.991 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/40 2024-03-09T08:58:26.991 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.992 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.992 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/9 2024-03-09T08:58:26.992 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:26.992 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/9': No such file or directory 2024-03-09T08:58:26.992 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/40': No such file or directory 2024-03-09T08:58:26.993 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.993 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/41 2024-03-09T08:58:26.993 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.994 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.994 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/41': No such file or directory 2024-03-09T08:58:26.995 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.995 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/10 2024-03-09T08:58:26.995 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:26.996 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/42 2024-03-09T08:58:26.996 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:26.997 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/10': No such file or directory 2024-03-09T08:58:26.998 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:26.999 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/42': No such file or directory 2024-03-09T08:58:26.999 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.000 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/43 2024-03-09T08:58:27.001 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.003 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/43': No such file or directory 2024-03-09T08:58:27.004 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.005 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/44 2024-03-09T08:58:27.006 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.008 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/44': No such file or directory 2024-03-09T08:58:27.009 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.009 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/11 2024-03-09T08:58:27.010 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.010 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/45 2024-03-09T08:58:27.011 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.012 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/11': No such file or directory 2024-03-09T08:58:27.013 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.014 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/45': No such file or directory 2024-03-09T08:58:27.014 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.014 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/12 2024-03-09T08:58:27.015 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.015 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/12': No such file or directory 2024-03-09T08:58:27.016 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.016 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/13 2024-03-09T08:58:27.016 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.016 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/13': No such file or directory 2024-03-09T08:58:27.017 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.017 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/14 2024-03-09T08:58:27.017 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.017 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/46 2024-03-09T08:58:27.017 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.018 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/46': No such file or directory 2024-03-09T08:58:27.018 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.019 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/47 2024-03-09T08:58:27.020 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/14': No such file or directory 2024-03-09T08:58:27.021 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.021 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.022 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/15 2024-03-09T08:58:27.023 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/47': No such file or directory 2024-03-09T08:58:27.024 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.024 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.024 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/15': No such file or directory 2024-03-09T08:58:27.025 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.025 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/48 2024-03-09T08:58:27.025 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/16 2024-03-09T08:58:27.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.029 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/16': No such file or directory 2024-03-09T08:58:27.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/17 2024-03-09T08:58:27.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.030 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/48': No such file or directory 2024-03-09T08:58:27.030 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.032 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/17': No such file or directory 2024-03-09T08:58:27.033 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.033 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/49 2024-03-09T08:58:27.034 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.035 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/49': No such file or directory 2024-03-09T08:58:27.035 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/18 2024-03-09T08:58:27.035 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.035 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.036 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/18': No such file or directory 2024-03-09T08:58:27.036 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.037 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/50 2024-03-09T08:58:27.037 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.037 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/19 2024-03-09T08:58:27.038 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.040 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/19': No such file or directory 2024-03-09T08:58:27.042 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/50': No such file or directory 2024-03-09T08:58:27.043 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.046 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/51 2024-03-09T08:58:27.046 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.046 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.046 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/20 2024-03-09T08:58:27.046 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.047 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/20': No such file or directory 2024-03-09T08:58:27.047 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/51': No such file or directory 2024-03-09T08:58:27.047 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.048 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/52 2024-03-09T08:58:27.050 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.050 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.050 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/21 2024-03-09T08:58:27.050 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.051 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/21': No such file or directory 2024-03-09T08:58:27.051 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/52': No such file or directory 2024-03-09T08:58:27.051 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.052 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.052 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/53 2024-03-09T08:58:27.052 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.053 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/22 2024-03-09T08:58:27.053 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.060 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/53': No such file or directory 2024-03-09T08:58:27.061 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/22': No such file or directory 2024-03-09T08:58:27.062 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.062 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/54 2024-03-09T08:58:27.062 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.062 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.062 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/23 2024-03-09T08:58:27.062 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.062 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/54': No such file or directory 2024-03-09T08:58:27.063 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/23': No such file or directory 2024-03-09T08:58:27.064 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.064 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/55 2024-03-09T08:58:27.064 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.065 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.065 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/24 2024-03-09T08:58:27.065 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.065 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/24': No such file or directory 2024-03-09T08:58:27.066 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.066 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/25 2024-03-09T08:58:27.066 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.066 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/55': No such file or directory 2024-03-09T08:58:27.066 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.067 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/56 2024-03-09T08:58:27.068 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/25': No such file or directory 2024-03-09T08:58:27.068 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.068 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/26 2024-03-09T08:58:27.069 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.069 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.071 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/56': No such file or directory 2024-03-09T08:58:27.071 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.071 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/26': No such file or directory 2024-03-09T08:58:27.072 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.072 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/57 2024-03-09T08:58:27.073 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/27 2024-03-09T08:58:27.073 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.073 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.075 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/27': No such file or directory 2024-03-09T08:58:27.076 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/57': No such file or directory 2024-03-09T08:58:27.076 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.076 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.077 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/28 2024-03-09T08:58:27.077 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.077 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/58 2024-03-09T08:58:27.078 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.079 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/28': No such file or directory 2024-03-09T08:58:27.080 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.081 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/58': No such file or directory 2024-03-09T08:58:27.082 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/29 2024-03-09T08:58:27.082 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.082 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.082 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/59 2024-03-09T08:58:27.082 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.083 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/29': No such file or directory 2024-03-09T08:58:27.084 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.084 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/30 2024-03-09T08:58:27.085 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/59': No such file or directory 2024-03-09T08:58:27.085 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.085 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.086 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/60 2024-03-09T08:58:27.086 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.088 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/30': No such file or directory 2024-03-09T08:58:27.088 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.089 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/60': No such file or directory 2024-03-09T08:58:27.089 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/31 2024-03-09T08:58:27.089 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.090 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.090 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/61 2024-03-09T08:58:27.094 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.095 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/61': No such file or directory 2024-03-09T08:58:27.095 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.095 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/62 2024-03-09T08:58:27.096 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/31': No such file or directory 2024-03-09T08:58:27.096 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.096 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/32 2024-03-09T08:58:27.097 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.097 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.098 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/32': No such file or directory 2024-03-09T08:58:27.099 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/62': No such file or directory 2024-03-09T08:58:27.099 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.099 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.100 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/33 2024-03-09T08:58:27.100 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/63 2024-03-09T08:58:27.100 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.101 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.103 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/33': No such file or directory 2024-03-09T08:58:27.103 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.104 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/34 2024-03-09T08:58:27.104 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/63': No such file or directory 2024-03-09T08:58:27.105 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.105 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.106 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/64 2024-03-09T08:58:27.106 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.107 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/34': No such file or directory 2024-03-09T08:58:27.108 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.109 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/35 2024-03-09T08:58:27.109 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/64': No such file or directory 2024-03-09T08:58:27.110 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.110 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.110 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/65 2024-03-09T08:58:27.111 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.112 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/35': No such file or directory 2024-03-09T08:58:27.113 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.114 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/36 2024-03-09T08:58:27.114 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/65': No such file or directory 2024-03-09T08:58:27.114 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.115 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.115 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/66 2024-03-09T08:58:27.116 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.116 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/36': No such file or directory 2024-03-09T08:58:27.117 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.118 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/37 2024-03-09T08:58:27.118 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.121 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/37': No such file or directory 2024-03-09T08:58:27.122 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.123 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/38 2024-03-09T08:58:27.123 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.126 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/38': No such file or directory 2024-03-09T08:58:27.127 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.127 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/39 2024-03-09T08:58:27.128 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.131 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/39': No such file or directory 2024-03-09T08:58:27.132 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.133 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/40 2024-03-09T08:58:27.133 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.135 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/40': No such file or directory 2024-03-09T08:58:27.136 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.136 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/41 2024-03-09T08:58:27.137 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.140 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/41': No such file or directory 2024-03-09T08:58:27.140 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.141 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/42 2024-03-09T08:58:27.141 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.144 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/42': No such file or directory 2024-03-09T08:58:27.144 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.145 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/43 2024-03-09T08:58:27.146 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.149 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/43': No such file or directory 2024-03-09T08:58:27.149 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.150 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/44 2024-03-09T08:58:27.150 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.153 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/44': No such file or directory 2024-03-09T08:58:27.153 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.154 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/45 2024-03-09T08:58:27.155 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.158 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/45': No such file or directory 2024-03-09T08:58:27.158 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.159 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/46 2024-03-09T08:58:27.159 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.162 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/46': No such file or directory 2024-03-09T08:58:27.163 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.164 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/47 2024-03-09T08:58:27.165 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.169 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/47': No such file or directory 2024-03-09T08:58:27.171 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.172 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/48 2024-03-09T08:58:27.173 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.176 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/48': No such file or directory 2024-03-09T08:58:27.176 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.178 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/49 2024-03-09T08:58:27.181 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.182 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/49': No such file or directory 2024-03-09T08:58:27.182 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.183 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/50 2024-03-09T08:58:27.184 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.187 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/50': No such file or directory 2024-03-09T08:58:27.189 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.189 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/51 2024-03-09T08:58:27.193 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.193 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/51': No such file or directory 2024-03-09T08:58:27.194 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.195 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/52 2024-03-09T08:58:27.196 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.201 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/52': No such file or directory 2024-03-09T08:58:27.203 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.203 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/53 2024-03-09T08:58:27.204 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.207 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/53': No such file or directory 2024-03-09T08:58:27.209 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.214 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/54 2024-03-09T08:58:27.214 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.216 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/54': No such file or directory 2024-03-09T08:58:27.217 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.219 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/55 2024-03-09T08:58:27.220 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.381 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/66': No such file or directory 2024-03-09T08:58:27.382 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.383 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/67 2024-03-09T08:58:27.384 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.386 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/67': No such file or directory 2024-03-09T08:58:27.387 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.388 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/68 2024-03-09T08:58:27.389 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.391 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/68': No such file or directory 2024-03-09T08:58:27.391 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.392 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/69 2024-03-09T08:58:27.392 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.395 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/69': No such file or directory 2024-03-09T08:58:27.395 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.396 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/70 2024-03-09T08:58:27.397 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.399 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/70': No such file or directory 2024-03-09T08:58:27.399 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.401 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/71 2024-03-09T08:58:27.401 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.403 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/71': No such file or directory 2024-03-09T08:58:27.404 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.405 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/72 2024-03-09T08:58:27.405 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.408 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/72': No such file or directory 2024-03-09T08:58:27.408 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.408 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/73 2024-03-09T08:58:27.409 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.411 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/73': No such file or directory 2024-03-09T08:58:27.411 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.412 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/74 2024-03-09T08:58:27.413 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.415 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/74': No such file or directory 2024-03-09T08:58:27.416 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.417 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/75 2024-03-09T08:58:27.417 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.420 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/75': No such file or directory 2024-03-09T08:58:27.421 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.421 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/76 2024-03-09T08:58:27.422 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.423 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/76': No such file or directory 2024-03-09T08:58:27.424 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.425 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/77 2024-03-09T08:58:27.425 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.428 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/77': No such file or directory 2024-03-09T08:58:27.429 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.429 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/78 2024-03-09T08:58:27.429 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.432 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/78': No such file or directory 2024-03-09T08:58:27.432 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.433 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/79 2024-03-09T08:58:27.433 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.437 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/79': No such file or directory 2024-03-09T08:58:27.438 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.439 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/80 2024-03-09T08:58:27.443 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.444 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/80': No such file or directory 2024-03-09T08:58:27.445 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.446 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.447 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/81 2024-03-09T08:58:27.451 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/81': No such file or directory 2024-03-09T08:58:27.452 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.453 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.455 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/82 2024-03-09T08:58:27.458 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/82': No such file or directory 2024-03-09T08:58:27.459 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.463 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/83 2024-03-09T08:58:27.463 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.464 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/83': No such file or directory 2024-03-09T08:58:27.465 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.466 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/84 2024-03-09T08:58:27.467 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.470 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/84': No such file or directory 2024-03-09T08:58:27.471 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.473 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/85 2024-03-09T08:58:27.473 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.478 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/85': No such file or directory 2024-03-09T08:58:27.478 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.482 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/86 2024-03-09T08:58:27.482 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.484 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/86': No such file or directory 2024-03-09T08:58:27.485 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.487 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/55': No such file or directory 2024-03-09T08:58:27.488 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.489 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/87 2024-03-09T08:58:27.489 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.489 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/56 2024-03-09T08:58:27.490 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.491 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/87': No such file or directory 2024-03-09T08:58:27.492 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.493 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/56': No such file or directory 2024-03-09T08:58:27.493 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.493 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/88 2024-03-09T08:58:27.494 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.494 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/57 2024-03-09T08:58:27.495 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.498 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/57': No such file or directory 2024-03-09T08:58:27.498 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.498 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/88': No such file or directory 2024-03-09T08:58:27.502 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.504 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.505 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/89 2024-03-09T08:58:27.505 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.505 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/58 2024-03-09T08:58:27.505 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/89': No such file or directory 2024-03-09T08:58:27.506 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/58': No such file or directory 2024-03-09T08:58:27.507 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.507 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/90 2024-03-09T08:58:27.507 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.507 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.508 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/59 2024-03-09T08:58:27.508 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.508 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/59': No such file or directory 2024-03-09T08:58:27.509 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.509 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/60 2024-03-09T08:58:27.509 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.510 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/90': No such file or directory 2024-03-09T08:58:27.510 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.511 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/91 2024-03-09T08:58:27.511 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/60': No such file or directory 2024-03-09T08:58:27.512 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.512 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.512 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/61 2024-03-09T08:58:27.513 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.515 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/91': No such file or directory 2024-03-09T08:58:27.516 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/61': No such file or directory 2024-03-09T08:58:27.517 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.517 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.517 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/62 2024-03-09T08:58:27.517 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.518 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/92 2024-03-09T08:58:27.518 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.520 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/62': No such file or directory 2024-03-09T08:58:27.520 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.521 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/92': No such file or directory 2024-03-09T08:58:27.521 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/63 2024-03-09T08:58:27.521 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.522 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.523 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/93 2024-03-09T08:58:27.523 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.524 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/63': No such file or directory 2024-03-09T08:58:27.524 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.525 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/64 2024-03-09T08:58:27.525 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.527 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/93': No such file or directory 2024-03-09T08:58:27.528 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.528 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/94 2024-03-09T08:58:27.529 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/64': No such file or directory 2024-03-09T08:58:27.529 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.529 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.530 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/65 2024-03-09T08:58:27.530 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.532 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/65': No such file or directory 2024-03-09T08:58:27.532 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/94': No such file or directory 2024-03-09T08:58:27.533 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.533 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.533 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/66 2024-03-09T08:58:27.533 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.535 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/95 2024-03-09T08:58:27.535 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.538 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/66': No such file or directory 2024-03-09T08:58:27.538 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/95': No such file or directory 2024-03-09T08:58:27.539 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.539 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/67 2024-03-09T08:58:27.539 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.540 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.540 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/96 2024-03-09T08:58:27.540 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.541 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/67': No such file or directory 2024-03-09T08:58:27.541 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.542 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/68 2024-03-09T08:58:27.543 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/96': No such file or directory 2024-03-09T08:58:27.543 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.544 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.544 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/97 2024-03-09T08:58:27.544 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.545 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/68': No such file or directory 2024-03-09T08:58:27.545 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.546 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/97': No such file or directory 2024-03-09T08:58:27.547 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/69 2024-03-09T08:58:27.547 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.547 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.547 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/98 2024-03-09T08:58:27.547 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.548 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/69': No such file or directory 2024-03-09T08:58:27.549 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.549 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/98': No such file or directory 2024-03-09T08:58:27.550 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.550 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/99 2024-03-09T08:58:27.551 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.551 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/70 2024-03-09T08:58:27.551 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.552 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/70': No such file or directory 2024-03-09T08:58:27.553 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/99': No such file or directory 2024-03-09T08:58:27.553 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.554 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.554 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/foo/100 2024-03-09T08:58:27.554 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'No such file' 2024-03-09T08:58:27.555 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/71 2024-03-09T08:58:27.555 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.556 INFO:tasks.workunit.client.1.smithi134.stdout:stat: cannot statx '.snap/foo/100': No such file or directory 2024-03-09T08:58:27.556 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/71': No such file or directory 2024-03-09T08:58:27.557 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/foo 2024-03-09T08:58:27.557 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.557 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/72 2024-03-09T08:58:27.558 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.560 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/72': No such file or directory 2024-03-09T08:58:27.561 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.561 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/73 2024-03-09T08:58:27.562 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.564 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/73': No such file or directory 2024-03-09T08:58:27.565 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.566 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T08:58:27.566 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/74 2024-03-09T08:58:27.567 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.567 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T08:58:27.568 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:58:27.568 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T08:58:27.569 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/74': No such file or directory 2024-03-09T08:58:27.569 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.569 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/75 2024-03-09T08:58:27.569 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.571 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/75': No such file or directory 2024-03-09T08:58:27.572 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.572 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/76 2024-03-09T08:58:27.572 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.574 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/76': No such file or directory 2024-03-09T08:58:27.575 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.575 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/77 2024-03-09T08:58:27.576 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.578 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/77': No such file or directory 2024-03-09T08:58:27.578 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.579 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/78 2024-03-09T08:58:27.579 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.581 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/78': No such file or directory 2024-03-09T08:58:27.581 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.582 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/79 2024-03-09T08:58:27.582 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.584 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/79': No such file or directory 2024-03-09T08:58:27.585 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.585 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/80 2024-03-09T08:58:27.586 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.588 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/80': No such file or directory 2024-03-09T08:58:27.588 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.588 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/81 2024-03-09T08:58:27.589 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.591 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/81': No such file or directory 2024-03-09T08:58:27.591 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.592 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/82 2024-03-09T08:58:27.592 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.595 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/82': No such file or directory 2024-03-09T08:58:27.595 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.595 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/83 2024-03-09T08:58:27.596 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.598 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/83': No such file or directory 2024-03-09T08:58:27.598 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.599 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/84 2024-03-09T08:58:27.599 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.603 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/84': No such file or directory 2024-03-09T08:58:27.604 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.604 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/85 2024-03-09T08:58:27.604 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.605 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/85': No such file or directory 2024-03-09T08:58:27.606 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.607 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/86 2024-03-09T08:58:27.607 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.609 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/86': No such file or directory 2024-03-09T08:58:27.610 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.611 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/87 2024-03-09T08:58:27.611 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.613 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/87': No such file or directory 2024-03-09T08:58:27.613 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.614 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/88 2024-03-09T08:58:27.614 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.617 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/88': No such file or directory 2024-03-09T08:58:27.617 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.618 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/89 2024-03-09T08:58:27.618 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.620 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/89': No such file or directory 2024-03-09T08:58:27.621 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.621 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/90 2024-03-09T08:58:27.622 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.624 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/90': No such file or directory 2024-03-09T08:58:27.625 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.625 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/91 2024-03-09T08:58:27.625 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.628 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/91': No such file or directory 2024-03-09T08:58:27.628 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.629 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/92 2024-03-09T08:58:27.629 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.632 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/92': No such file or directory 2024-03-09T08:58:27.632 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.633 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/93 2024-03-09T08:58:27.633 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.640 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2024-03-09T08:58:27.641 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2024-03-09T08:58:27.694 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T08:58:27.694 INFO:tasks.workunit.client.1.smithi134.stderr:+ sudo git config --global http.sslVerify false 2024-03-09T08:58:27.734 INFO:tasks.workunit.client.1.smithi134.stderr:+ sudo git config --global http.postBuffer 1048576000 2024-03-09T08:58:27.773 INFO:tasks.workunit.client.1.smithi134.stderr:+ retried=false 2024-03-09T08:58:27.773 INFO:tasks.workunit.client.1.smithi134.stderr:+ trap -- retry EXIT 2024-03-09T08:58:27.773 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm -rf ceph 2024-03-09T08:58:27.776 INFO:tasks.workunit.client.1.smithi134.stderr:+ timeout 1800 git clone https://git.ceph.com/ceph.git 2024-03-09T08:58:27.784 INFO:tasks.workunit.client.1.smithi134.stderr:Cloning into 'ceph'... 2024-03-09T08:58:27.898 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/93': No such file or directory 2024-03-09T08:58:27.898 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.900 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/94 2024-03-09T08:58:27.901 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.903 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/94': No such file or directory 2024-03-09T08:58:27.904 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.905 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/95 2024-03-09T08:58:27.906 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.911 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/95': No such file or directory 2024-03-09T08:58:27.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/96 2024-03-09T08:58:27.912 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.915 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/96': No such file or directory 2024-03-09T08:58:27.916 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.917 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/97 2024-03-09T08:58:27.918 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.921 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/97': No such file or directory 2024-03-09T08:58:27.921 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.923 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/98 2024-03-09T08:58:27.923 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.926 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/98': No such file or directory 2024-03-09T08:58:27.927 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.928 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/99 2024-03-09T08:58:27.929 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.932 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/99': No such file or directory 2024-03-09T08:58:27.933 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 100` 2024-03-09T08:58:27.934 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/foo/100 2024-03-09T08:58:27.935 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'No such file' 2024-03-09T08:58:27.939 INFO:tasks.workunit.client.0.smithi067.stdout:stat: cannot statx '.snap/foo/100': No such file or directory 2024-03-09T08:58:27.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/foo 2024-03-09T08:58:27.956 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T08:58:27.957 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T08:58:27.957 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T08:58:27.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T08:58:28.057 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2024-03-09T08:58:28.057 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2024-03-09T08:58:28.119 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T08:58:28.119 INFO:tasks.workunit.client.0.smithi067.stderr:+ sudo git config --global http.sslVerify false 2024-03-09T08:58:28.165 INFO:tasks.workunit.client.0.smithi067.stderr:+ sudo git config --global http.postBuffer 1048576000 2024-03-09T08:58:28.213 INFO:tasks.workunit.client.0.smithi067.stderr:+ retried=false 2024-03-09T08:58:28.213 INFO:tasks.workunit.client.0.smithi067.stderr:+ trap -- retry EXIT 2024-03-09T08:58:28.213 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -rf ceph 2024-03-09T08:58:28.215 INFO:tasks.workunit.client.0.smithi067.stderr:+ timeout 1800 git clone https://git.ceph.com/ceph.git 2024-03-09T08:58:28.222 INFO:tasks.workunit.client.0.smithi067.stderr:Cloning into 'ceph'... 2024-03-09T08:58:34.624 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T08:58:34.625 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-03-09T08:58:34.625 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-09T08:58:34.625 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-09T08:58:34.634 INFO:tasks.ceph.mon.a:Started 2024-03-09T08:58:34.634 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:58:34.634 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:58:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.836+0000 7f2986400640 1 Processor -- start 2024-03-09T08:58:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.836+0000 7f2986400640 1 -- start start 2024-03-09T08:58:34.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.837+0000 7f2986400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 0x7f2980157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.837+0000 7f2986400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2980153d40 con 0x7f29800c22a0 2024-03-09T08:58:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.837+0000 7f2986400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29800c2790 con 0x7f29800ac730 2024-03-09T08:58:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.837+0000 7f2986400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2980070ee0 con 0x7f2980071190 2024-03-09T08:58:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.838+0000 7f2977e00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f29800c22a0 legacy=0x7f298015c5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T08:58:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.838+0000 7f297f400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f29800ac730 0x7f2980157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:42044/0 (socket says 172.21.15.67:42044) 2024-03-09T08:58:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.838+0000 7f297f400640 1 -- 172.21.15.67:0/1244482368 learned_addr learned my addr 172.21.15.67:0/1244482368 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:58:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.838+0000 7f297fe00640 1 --2- 172.21.15.67:0/1244482368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 0x7f2980157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:34.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.839+0000 7f297ea00640 1 -- 172.21.15.67:0/1244482368 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614130800 0 0) 0x7f29800c2790 con 0x7f29800ac730 2024-03-09T08:58:34.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.839+0000 7f297ea00640 1 -- 172.21.15.67:0/1244482368 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2964003520 con 0x7f29800ac730 2024-03-09T08:58:34.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.839+0000 7f297fe00640 1 -- 172.21.15.67:0/1244482368 >> v1:172.21.15.134:6789/0 conn(0x7f29800ac730 legacy=0x7f2980157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:34.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.839+0000 7f297fe00640 1 -- 172.21.15.67:0/1244482368 >> v1:172.21.15.67:6789/0 conn(0x7f29800c22a0 legacy=0x7f298015c5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:34.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.839+0000 7f297fe00640 1 -- 172.21.15.67:0/1244482368 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980070920 con 0x7f2980071190 2024-03-09T08:58:34.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.841+0000 7f297fe00640 1 --2- 172.21.15.67:0/1244482368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 0x7f2980157d60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f29800ad380 tx=0x7f297802be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43bf5f91033a1f01 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.841+0000 7f297ea00640 1 -- 172.21.15.67:0/1244482368 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978039070 con 0x7f2980071190 2024-03-09T08:58:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.841+0000 7f297ea00640 1 -- 172.21.15.67:0/1244482368 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297802cbb0 con 0x7f2980071190 2024-03-09T08:58:34.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.841+0000 7f297ea00640 1 -- 172.21.15.67:0/1244482368 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978034440 con 0x7f2980071190 2024-03-09T08:58:34.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.842+0000 7f2986400640 1 -- 172.21.15.67:0/1244482368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 msgr2=0x7f2980157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:34.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.842+0000 7f2986400640 1 --2- 172.21.15.67:0/1244482368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 0x7f2980157d60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f29800ad380 tx=0x7f297802be00 comp rx=0 tx=0).stop 2024-03-09T08:58:34.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.843+0000 7f2986400640 1 -- 172.21.15.67:0/1244482368 shutdown_connections 2024-03-09T08:58:34.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.843+0000 7f2986400640 1 --2- 172.21.15.67:0/1244482368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 0x7f2980157d60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:34.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.843+0000 7f2986400640 1 -- 172.21.15.67:0/1244482368 >> 172.21.15.67:0/1244482368 conn(0x7f29800c7370 msgr2=0x7f29800c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:34.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.843+0000 7f2986400640 1 -- 172.21.15.67:0/1244482368 shutdown_connections 2024-03-09T08:58:34.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.843+0000 7f2986400640 1 -- 172.21.15.67:0/1244482368 wait complete. 2024-03-09T08:58:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.846+0000 7f2986400640 1 Processor -- start 2024-03-09T08:58:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.846+0000 7f2986400640 1 -- start start 2024-03-09T08:58:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.847+0000 7f2986400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 0x7f29801e7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.847+0000 7f2986400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29800ac730 0x7f298014fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.847+0000 7f2986400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29800c22a0 0x7f29801eaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:34.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.847+0000 7f2986400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2980151610 con 0x7f29800ac730 2024-03-09T08:58:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.847+0000 7f2986400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2980154d00 con 0x7f29800c22a0 2024-03-09T08:58:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.847+0000 7f2986400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2980151790 con 0x7f2980071190 2024-03-09T08:58:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.848+0000 7f2977e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29800ac730 0x7f298014fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.848+0000 7f297f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 0x7f29801e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.848+0000 7f297f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 0x7f29801e7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:47420/0 (socket says 172.21.15.67:47420) 2024-03-09T08:58:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.848+0000 7f297f400640 1 -- 172.21.15.67:0/2170915100 learned_addr learned my addr 172.21.15.67:0/2170915100 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.848+0000 7f297fe00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29800c22a0 0x7f29801eaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.848+0000 7f2977e00640 1 -- 172.21.15.67:0/2170915100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29800ac730 msgr2=0x7f298014fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T08:58:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.848+0000 7f2977e00640 1 -- 172.21.15.67:0/2170915100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29800ac730 msgr2=0x7f298014fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:58:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.848+0000 7f2977e00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29800ac730 0x7f298014fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:58:34.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.848+0000 7f2977e00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29800ac730 0x7f298014fc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.849+0000 7f297fe00640 1 -- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 msgr2=0x7f29801e7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.849+0000 7f297fe00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 0x7f29801e7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.849+0000 7f297fe00640 1 -- 172.21.15.67:0/2170915100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29800ac730 msgr2=0x7f298014fc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:58:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.849+0000 7f297fe00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29800ac730 0x7f298014fc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.849+0000 7f297fe00640 1 -- 172.21.15.67:0/2170915100 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29801ea790 con 0x7f29800c22a0 2024-03-09T08:58:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.849+0000 7f297f400640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 0x7f29801e7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:58:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.849+0000 7f297fe00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29800c22a0 0x7f29801eaf50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2978033000 tx=0x7f2978004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.850+0000 7f2977400640 1 -- 172.21.15.67:0/2170915100 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978044020 con 0x7f29800c22a0 2024-03-09T08:58:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.850+0000 7f2986400640 1 -- 172.21.15.67:0/2170915100 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29801552b0 con 0x7f29800c22a0 2024-03-09T08:58:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.850+0000 7f2977400640 1 -- 172.21.15.67:0/2170915100 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2978009410 con 0x7f29800c22a0 2024-03-09T08:58:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.850+0000 7f2977400640 1 -- 172.21.15.67:0/2170915100 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978039040 con 0x7f29800c22a0 2024-03-09T08:58:34.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.850+0000 7f2986400640 1 -- 172.21.15.67:0/2170915100 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2980156f80 con 0x7f29800c22a0 2024-03-09T08:58:34.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.852+0000 7f2974c00640 1 -- 172.21.15.67:0/2170915100 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2950005290 con 0x7f29800c22a0 2024-03-09T08:58:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.853+0000 7f2977400640 1 -- 172.21.15.67:0/2170915100 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2978047020 con 0x7f29800c22a0 2024-03-09T08:58:34.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.854+0000 7f2977400640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f295c06b4d0 0x7f295c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.854+0000 7f2977400640 1 -- 172.21.15.67:0/2170915100 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f29780acfe0 con 0x7f29800c22a0 2024-03-09T08:58:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.854+0000 7f297f400640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f295c06b4d0 0x7f295c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.856+0000 7f297f400640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f295c06b4d0 0x7f295c06d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f29700011e0 tx=0x7f2970009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:34.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:34.869+0000 7f2977400640 1 -- 172.21.15.67:0/2170915100 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f297804e030 con 0x7f29800c22a0 2024-03-09T08:58:35.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.162+0000 7f2974c00640 1 -- 172.21.15.67:0/2170915100 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2950005580 con 0x7f29800c22a0 2024-03-09T08:58:35.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.163+0000 7f2977400640 1 -- 172.21.15.67:0/2170915100 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f297807b9e0 con 0x7f29800c22a0 2024-03-09T08:58:35.163 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":46,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:58:35.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.165+0000 7f2974c00640 1 -- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f295c06b4d0 msgr2=0x7f295c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:35.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.165+0000 7f2974c00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f295c06b4d0 0x7f295c06d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f29700011e0 tx=0x7f2970009040 comp rx=0 tx=0).stop 2024-03-09T08:58:35.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.166+0000 7f2974c00640 1 -- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29800c22a0 msgr2=0x7f29801eaf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:35.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.166+0000 7f2974c00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29800c22a0 0x7f29801eaf50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2978033000 tx=0x7f2978004a40 comp rx=0 tx=0).stop 2024-03-09T08:58:35.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.166+0000 7f2974c00640 1 -- 172.21.15.67:0/2170915100 shutdown_connections 2024-03-09T08:58:35.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.166+0000 7f2974c00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f295c06b4d0 0x7f295c06d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:35.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.166+0000 7f2974c00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29800c22a0 0x7f29801eaf50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:35.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.166+0000 7f2974c00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29800ac730 0x7f298014fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:35.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.166+0000 7f2974c00640 1 --2- 172.21.15.67:0/2170915100 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2980071190 0x7f29801e7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:35.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.166+0000 7f2974c00640 1 -- 172.21.15.67:0/2170915100 >> 172.21.15.67:0/2170915100 conn(0x7f29800c7370 msgr2=0x7f29800c3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:35.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.166+0000 7f2974c00640 1 -- 172.21.15.67:0/2170915100 shutdown_connections 2024-03-09T08:58:35.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:35.166+0000 7f2974c00640 1 -- 172.21.15.67:0/2170915100 wait complete. 2024-03-09T08:58:38.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:58:38.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.413+0000 7f8152a00640 1 Processor -- start 2024-03-09T08:58:38.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.413+0000 7f8152a00640 1 -- start start 2024-03-09T08:58:38.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.415+0000 7f8152a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f814c157600 0x7f814c15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.415+0000 7f8152a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f814c0aa800 0x7f814c0aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.415+0000 7f8152a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c15af40 con 0x7f814c0c3600 2024-03-09T08:58:38.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.415+0000 7f8152a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c1579e0 con 0x7f814c0aa800 2024-03-09T08:58:38.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.415+0000 7f8152a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c06d4c0 con 0x7f814c157600 2024-03-09T08:58:38.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.415+0000 7f8150c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f814c0c3600 0x7f814c0ab760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47152/0 (socket says 172.21.15.67:47152) 2024-03-09T08:58:38.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.415+0000 7f8150c00640 1 -- 172.21.15.67:0/1995380175 learned_addr learned my addr 172.21.15.67:0/1995380175 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:58:38.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.415+0000 7f8152000640 1 --2- 172.21.15.67:0/1995380175 >> v2:172.21.15.67:3300/0 conn(0x7f814c0aa800 0x7f814c0aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.415+0000 7f8151600640 1 --2- 172.21.15.67:0/1995380175 >> v2:172.21.15.134:3300/0 conn(0x7f814c157600 0x7f814c15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.416+0000 7f814be00640 1 -- 172.21.15.67:0/1995380175 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187655485 0 0) 0x7f8138001110 con 0x7f814c0c3600 2024-03-09T08:58:38.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.416+0000 7f8152000640 1 -- 172.21.15.67:0/1995380175 >> v2:172.21.15.134:3300/0 conn(0x7f814c157600 msgr2=0x7f814c15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:38.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.416+0000 7f8152000640 1 --2- 172.21.15.67:0/1995380175 >> v2:172.21.15.134:3300/0 conn(0x7f814c157600 0x7f814c15aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.416+0000 7f8152000640 1 -- 172.21.15.67:0/1995380175 >> v1:172.21.15.67:6789/0 conn(0x7f814c0c3600 legacy=0x7f814c0ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.416+0000 7f8152000640 1 -- 172.21.15.67:0/1995380175 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c06d7a0 con 0x7f814c0aa800 2024-03-09T08:58:38.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.416+0000 7f8151600640 1 --2- 172.21.15.67:0/1995380175 >> v2:172.21.15.134:3300/0 conn(0x7f814c157600 0x7f814c15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:58:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.417+0000 7f8152000640 1 --2- 172.21.15.67:0/1995380175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c0aa800 0x7f814c0aac90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f814c0c3d60 tx=0x7f814402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416824a8b2de1be5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.417+0000 7f814be00640 1 -- 172.21.15.67:0/1995380175 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144035020 con 0x7f814c0aa800 2024-03-09T08:58:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.417+0000 7f814be00640 1 -- 172.21.15.67:0/1995380175 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8144031830 con 0x7f814c0aa800 2024-03-09T08:58:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.417+0000 7f814be00640 1 -- 172.21.15.67:0/1995380175 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144031c00 con 0x7f814c0aa800 2024-03-09T08:58:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.418+0000 7f8152a00640 1 -- 172.21.15.67:0/1995380175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c0aa800 msgr2=0x7f814c0aac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.418+0000 7f8152a00640 1 --2- 172.21.15.67:0/1995380175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c0aa800 0x7f814c0aac90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f814c0c3d60 tx=0x7f814402b8d0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.418+0000 7f8152a00640 1 -- 172.21.15.67:0/1995380175 shutdown_connections 2024-03-09T08:58:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.418+0000 7f8152a00640 1 --2- 172.21.15.67:0/1995380175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c0aa800 0x7f814c0aac90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.418+0000 7f8152a00640 1 --2- 172.21.15.67:0/1995380175 >> v2:172.21.15.134:3300/0 conn(0x7f814c157600 0x7f814c15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.418+0000 7f8152a00640 1 -- 172.21.15.67:0/1995380175 >> 172.21.15.67:0/1995380175 conn(0x7f814c0c7300 msgr2=0x7f814c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.418+0000 7f8152a00640 1 -- 172.21.15.67:0/1995380175 shutdown_connections 2024-03-09T08:58:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.418+0000 7f8152a00640 1 -- 172.21.15.67:0/1995380175 wait complete. 2024-03-09T08:58:38.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.420+0000 7f8152a00640 1 Processor -- start 2024-03-09T08:58:38.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.420+0000 7f8152a00640 1 -- start start 2024-03-09T08:58:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.422+0000 7f8152a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f814c0aa800 0x7f814c1f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.422+0000 7f8152a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c0c3600 0x7f814c1ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.422+0000 7f8152a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f814c157600 0x7f814c151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.422+0000 7f8152a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0abe70 con 0x7f814c0c3600 2024-03-09T08:58:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.422+0000 7f8152a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c06dd60 con 0x7f814c157600 2024-03-09T08:58:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.422+0000 7f8152a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0abff0 con 0x7f814c0aa800 2024-03-09T08:58:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.422+0000 7f8150c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c0c3600 0x7f814c1ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.422+0000 7f8150c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c0c3600 0x7f814c1ef9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60290/0 (socket says 172.21.15.67:60290) 2024-03-09T08:58:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.422+0000 7f8150c00640 1 -- 172.21.15.67:0/3647236738 learned_addr learned my addr 172.21.15.67:0/3647236738 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.422+0000 7f8152000640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f814c157600 0x7f814c151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.422+0000 7f8151600640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f814c0aa800 0x7f814c1f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.423+0000 7f8150c00640 1 -- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f814c0aa800 msgr2=0x7f814c1f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.423+0000 7f8150c00640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f814c0aa800 0x7f814c1f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.423+0000 7f8150c00640 1 -- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f814c157600 msgr2=0x7f814c151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.423+0000 7f8150c00640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f814c157600 0x7f814c151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.423+0000 7f8150c00640 1 -- 172.21.15.67:0/3647236738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c152660 con 0x7f814c0c3600 2024-03-09T08:58:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.423+0000 7f8151600640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f814c0aa800 0x7f814c1f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:58:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.423+0000 7f8150c00640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c0c3600 0x7f814c1ef9b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f813800c780 tx=0x7f813800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.424+0000 7f8149600640 1 -- 172.21.15.67:0/3647236738 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8138018070 con 0x7f814c0c3600 2024-03-09T08:58:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.424+0000 7f8152a00640 1 -- 172.21.15.67:0/3647236738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c073360 con 0x7f814c0c3600 2024-03-09T08:58:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.424+0000 7f8149600640 1 -- 172.21.15.67:0/3647236738 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81380072d0 con 0x7f814c0c3600 2024-03-09T08:58:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.424+0000 7f8149600640 1 -- 172.21.15.67:0/3647236738 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380047e0 con 0x7f814c0c3600 2024-03-09T08:58:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.424+0000 7f8152a00640 1 -- 172.21.15.67:0/3647236738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c072da0 con 0x7f814c0c3600 2024-03-09T08:58:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.425+0000 7f8152a00640 1 -- 172.21.15.67:0/3647236738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814c04f3c0 con 0x7f814c0c3600 2024-03-09T08:58:38.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.427+0000 7f8149600640 1 -- 172.21.15.67:0/3647236738 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f813801b070 con 0x7f814c0c3600 2024-03-09T08:58:38.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.428+0000 7f8149600640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f812c06b4d0 0x7f812c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.428+0000 7f8149600640 1 -- 172.21.15.67:0/3647236738 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8138089ce0 con 0x7f814c0c3600 2024-03-09T08:58:38.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.429+0000 7f8151600640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f812c06b4d0 0x7f812c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.430+0000 7f8151600640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f812c06b4d0 0x7f812c06d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f81400044f0 tx=0x7f8140003a30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.432+0000 7f8149600640 1 -- 172.21.15.67:0/3647236738 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f814c15af40 con 0x7f814c0c3600 2024-03-09T08:58:38.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.710+0000 7f8152a00640 1 -- 172.21.15.67:0/3647236738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f814c0c76e0 con 0x7f814c0c3600 2024-03-09T08:58:38.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.711+0000 7f8149600640 1 -- 172.21.15.67:0/3647236738 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7f8138058070 con 0x7f814c0c3600 2024-03-09T08:58:38.711 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.713+0000 7f8152a00640 1 -- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f812c06b4d0 msgr2=0x7f812c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.713+0000 7f8152a00640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f812c06b4d0 0x7f812c06d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f81400044f0 tx=0x7f8140003a30 comp rx=0 tx=0).stop 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.713+0000 7f8152a00640 1 -- 172.21.15.67:0/3647236738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c0c3600 msgr2=0x7f814c1ef9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.713+0000 7f8152a00640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c0c3600 0x7f814c1ef9b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f813800c780 tx=0x7f813800cc40 comp rx=0 tx=0).stop 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.713+0000 7f8152a00640 1 -- 172.21.15.67:0/3647236738 shutdown_connections 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.713+0000 7f8152a00640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f812c06b4d0 0x7f812c06d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.713+0000 7f8152a00640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f814c157600 0x7f814c151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.714+0000 7f8152a00640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f814c0c3600 0x7f814c1ef9b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.714+0000 7f8152a00640 1 --2- 172.21.15.67:0/3647236738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f814c0aa800 0x7f814c1f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.714+0000 7f8152a00640 1 -- 172.21.15.67:0/3647236738 >> 172.21.15.67:0/3647236738 conn(0x7f814c0c7300 msgr2=0x7f814c0ab760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.714+0000 7f8152a00640 1 -- 172.21.15.67:0/3647236738 shutdown_connections 2024-03-09T08:58:38.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.714+0000 7f8152a00640 1 -- 172.21.15.67:0/3647236738 wait complete. 2024-03-09T08:58:38.733 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:58:38.733 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:58:38.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:58:38.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.955+0000 7f6a47e00640 1 Processor -- start 2024-03-09T08:58:38.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.956+0000 7f6a47e00640 1 -- start start 2024-03-09T08:58:38.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.959+0000 7f6a47e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6a4806e480 0x7f6a48156600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.959+0000 7f6a47e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a480c4360 con 0x7f6a480c17f0 2024-03-09T08:58:38.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.959+0000 7f6a47e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a481594a0 con 0x7f6a48156220 2024-03-09T08:58:38.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.959+0000 7f6a47e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a480c4ba0 con 0x7f6a4806e480 2024-03-09T08:58:38.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.959+0000 7f6a47400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6a480c17f0 0x7f6a48159980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51106/0 (socket says 172.21.15.67:51106) 2024-03-09T08:58:38.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.959+0000 7f6a47400640 1 -- 172.21.15.67:0/3369010311 learned_addr learned my addr 172.21.15.67:0/3369010311 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:58:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.959+0000 7f6a3e000640 1 --2- 172.21.15.67:0/3369010311 >> v2:172.21.15.134:3300/0 conn(0x7f6a4806e480 0x7f6a48156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736223794 0 0) 0x7f6a40001490 con 0x7f6a480c17f0 2024-03-09T08:58:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a2c003520 con 0x7f6a480c17f0 2024-03-09T08:58:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 267148500 0 0) 0x7f6a2c003520 con 0x7f6a480c17f0 2024-03-09T08:58:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 >> v2:172.21.15.134:3300/0 conn(0x7f6a4806e480 msgr2=0x7f6a48156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 --2- 172.21.15.67:0/3369010311 >> v2:172.21.15.134:3300/0 conn(0x7f6a4806e480 0x7f6a48156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 >> v1:172.21.15.134:6789/0 conn(0x7f6a48156220 legacy=0x7f6a480c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480c45e0 con 0x7f6a480c17f0 2024-03-09T08:58:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f6a40001bc0 con 0x7f6a480c17f0 2024-03-09T08:58:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 >> v1:172.21.15.67:6789/0 conn(0x7f6a480c17f0 legacy=0x7f6a48159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 --2- 172.21.15.67:0/3369010311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a2c003a70 0x7f6a2c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 --2- 172.21.15.67:0/3369010311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 0x7f6a2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 --2- 172.21.15.67:0/3369010311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a2c005210 0x7f6a2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c4ba0 con 0x7f6a2c004800 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c00c140 con 0x7f6a2c005210 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c00c340 con 0x7f6a2c003a70 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.960+0000 7f6a3e000640 1 --2- 172.21.15.67:0/3369010311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 0x7f6a2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.961+0000 7f6a46a00640 1 --2- 172.21.15.67:0/3369010311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a2c003a70 0x7f6a2c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.961+0000 7f6a3e000640 1 -- 172.21.15.67:0/3369010311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a2c003a70 msgr2=0x7f6a2c003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.961+0000 7f6a3e000640 1 --2- 172.21.15.67:0/3369010311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a2c003a70 0x7f6a2c003e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.961+0000 7f6a3e000640 1 -- 172.21.15.67:0/3369010311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a2c005210 msgr2=0x7f6a2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.961+0000 7f6a3e000640 1 --2- 172.21.15.67:0/3369010311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a2c005210 0x7f6a2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.961+0000 7f6a3e000640 1 -- 172.21.15.67:0/3369010311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c003520 con 0x7f6a2c004800 2024-03-09T08:58:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.961+0000 7f6a47400640 1 -- 172.21.15.67:0/3369010311 reap_dead start 2024-03-09T08:58:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.962+0000 7f6a3e000640 1 --2- 172.21.15.67:0/3369010311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 0x7f6a2c004c80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009a90 tx=0x7f6a2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbf004f42602a30 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.962+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30039070 con 0x7f6a2c004800 2024-03-09T08:58:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.962+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3002c680 con 0x7f6a2c004800 2024-03-09T08:58:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.962+0000 7f6a46000640 1 -- 172.21.15.67:0/3369010311 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30009690 con 0x7f6a2c004800 2024-03-09T08:58:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.963+0000 7f6a47e00640 1 -- 172.21.15.67:0/3369010311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 msgr2=0x7f6a2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.963+0000 7f6a47e00640 1 --2- 172.21.15.67:0/3369010311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 0x7f6a2c004c80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009a90 tx=0x7f6a2c005830 comp rx=0 tx=0).stop 2024-03-09T08:58:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.963+0000 7f6a47e00640 1 -- 172.21.15.67:0/3369010311 shutdown_connections 2024-03-09T08:58:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.963+0000 7f6a47e00640 1 --2- 172.21.15.67:0/3369010311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 0x7f6a2c004c80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.963+0000 7f6a47e00640 1 -- 172.21.15.67:0/3369010311 >> 172.21.15.67:0/3369010311 conn(0x7f6a480c8310 msgr2=0x7f6a480c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:38.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.963+0000 7f6a47e00640 1 -- 172.21.15.67:0/3369010311 shutdown_connections 2024-03-09T08:58:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.963+0000 7f6a47e00640 1 -- 172.21.15.67:0/3369010311 wait complete. 2024-03-09T08:58:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.965+0000 7f6a47e00640 1 Processor -- start 2024-03-09T08:58:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.965+0000 7f6a47e00640 1 -- start start 2024-03-09T08:58:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.966+0000 7f6a47e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 0x7f6a4814f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.966+0000 7f6a47e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a481e0c30 0x7f6a4814ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.966+0000 7f6a47e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a480ac7a0 0x7f6a480ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.966+0000 7f6a47e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a48156600 con 0x7f6a2c004800 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.966+0000 7f6a47e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c5820 con 0x7f6a480ac7a0 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.966+0000 7f6a47e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a48156780 con 0x7f6a481e0c30 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.966+0000 7f6a46a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 0x7f6a4814f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.966+0000 7f6a46a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 0x7f6a4814f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47926/0 (socket says 172.21.15.67:47926) 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.966+0000 7f6a46a00640 1 -- 172.21.15.67:0/457574005 learned_addr learned my addr 172.21.15.67:0/457574005 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.967+0000 7f6a46a00640 1 -- 172.21.15.67:0/457574005 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a481e0c30 msgr2=0x7f6a4814ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.967+0000 7f6a46a00640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a481e0c30 0x7f6a4814ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.967+0000 7f6a46a00640 1 -- 172.21.15.67:0/457574005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a480ac7a0 msgr2=0x7f6a480ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.967+0000 7f6a46a00640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a480ac7a0 0x7f6a480ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.967+0000 7f6a46a00640 1 -- 172.21.15.67:0/457574005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a481ec2d0 con 0x7f6a2c004800 2024-03-09T08:58:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.967+0000 7f6a46a00640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 0x7f6a4814f930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800d430 tx=0x7f6a3800d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.967+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38018070 con 0x7f6a2c004800 2024-03-09T08:58:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.967+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a38003610 con 0x7f6a2c004800 2024-03-09T08:58:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.967+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38003a00 con 0x7f6a2c004800 2024-03-09T08:58:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.967+0000 7f6a47e00640 1 -- 172.21.15.67:0/457574005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a481ec5b0 con 0x7f6a2c004800 2024-03-09T08:58:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.968+0000 7f6a47e00640 1 -- 172.21.15.67:0/457574005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a48150a20 con 0x7f6a2c004800 2024-03-09T08:58:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.969+0000 7f6a47e00640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a481504e0 0x7f6a480c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.969+0000 7f6a47e00640 1 -- 172.21.15.67:0/457574005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480ac060 con 0x7f6a481504e0 2024-03-09T08:58:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.969+0000 7f6a3e000640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a481504e0 0x7f6a480c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.970+0000 7f6a3e000640 1 -- 172.21.15.67:0/457574005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a4804f3c0 con 0x7f6a481504e0 2024-03-09T08:58:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.970+0000 7f6a3e000640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a481504e0 0x7f6a480c7590 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6a4806e8a0 tx=0x7f6a30004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.970+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30044020 con 0x7f6a481504e0 2024-03-09T08:58:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.970+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6a3801b070 con 0x7f6a2c004800 2024-03-09T08:58:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.972+0000 7f6a3f400640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a2406b400 0x7f6a2406d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.972+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6a38089c60 con 0x7f6a2c004800 2024-03-09T08:58:38.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.973+0000 7f6a47400640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a2406b400 0x7f6a2406d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.973+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f6a3003a040 con 0x7f6a481504e0 2024-03-09T08:58:38.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.973+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a481504e0 msgr2=0x7f6a480c7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.973+0000 7f6a3f400640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a481504e0 0x7f6a480c7590 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6a4806e8a0 tx=0x7f6a30004500 comp rx=0 tx=0).stop 2024-03-09T08:58:38.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.973+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a481504e0 msgr2=0x7f6a480c7590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:38.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.973+0000 7f6a3f400640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a481504e0 0x7f6a480c7590 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6a4806e8a0 tx=0x7f6a30004500 comp rx=0 tx=0).stop 2024-03-09T08:58:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.974+0000 7f6a47400640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a2406b400 0x7f6a2406d8c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6a48071680 tx=0x7f6a40001540 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.980+0000 7f6a47e00640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a480c7ad0 0x7f6a481eed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.980+0000 7f6a47e00640 1 -- 172.21.15.67:0/457574005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c1c90 con 0x7f6a480c7ad0 2024-03-09T08:58:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.980+0000 7f6a3e000640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a480c7ad0 0x7f6a481eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.981+0000 7f6a3e000640 1 -- 172.21.15.67:0/457574005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6a480c60a0 con 0x7f6a480c7ad0 2024-03-09T08:58:38.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.981+0000 7f6a3e000640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a480c7ad0 0x7f6a481eed20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6a30006f80 tx=0x7f6a30008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:38.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.981+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30039050 con 0x7f6a480c7ad0 2024-03-09T08:58:38.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f6a30048e20 con 0x7f6a480c7ad0 2024-03-09T08:58:38.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a480c7ad0 msgr2=0x7f6a481eed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a3f400640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a480c7ad0 0x7f6a481eed20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6a30006f80 tx=0x7f6a30008030 comp rx=0 tx=0).stop 2024-03-09T08:58:38.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a3f400640 1 -- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a480c7ad0 msgr2=0x7f6a481eed20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:38.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a3f400640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a480c7ad0 0x7f6a481eed20 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6a30006f80 tx=0x7f6a30008030 comp rx=0 tx=0).stop 2024-03-09T08:58:38.982 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:58:38.982 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:58:38.982 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 50, 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:38.984 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:38.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:58:38.990 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:38.994 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:58:38.994 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:58:38.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a47e00640 1 -- 172.21.15.67:0/457574005 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a2406b400 msgr2=0x7f6a2406d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a47e00640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a2406b400 0x7f6a2406d8c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6a48071680 tx=0x7f6a40001540 comp rx=0 tx=0).stop 2024-03-09T08:58:38.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a47e00640 1 -- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 msgr2=0x7f6a4814f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:38.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a47e00640 1 --2- 172.21.15.67:0/457574005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a2c004800 0x7f6a4814f930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800d430 tx=0x7f6a3800d8f0 comp rx=0 tx=0).stop 2024-03-09T08:58:38.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a47400640 1 -- 172.21.15.67:0/457574005 reap_dead start 2024-03-09T08:58:38.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a47400640 1 -- 172.21.15.67:0/457574005 reap_dead start 2024-03-09T08:58:38.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a47e00640 1 -- 172.21.15.67:0/457574005 shutdown_connections 2024-03-09T08:58:38.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a47e00640 1 -- 172.21.15.67:0/457574005 >> 172.21.15.67:0/457574005 conn(0x7f6a480c8310 msgr2=0x7f6a480c21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:38.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a47e00640 1 -- 172.21.15.67:0/457574005 shutdown_connections 2024-03-09T08:58:38.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:38.982+0000 7f6a47e00640 1 -- 172.21.15.67:0/457574005 wait complete. 2024-03-09T08:58:38.999 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:58:38.999 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:58:38.999 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:58:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.208+0000 7f232be00640 1 Processor -- start 2024-03-09T08:58:39.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.208+0000 7f232be00640 1 -- start start 2024-03-09T08:58:39.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.209+0000 7f232be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f232c0c5e00 0x7f232c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.209+0000 7f232be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f232c0c61e0 0x7f232c0c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.209+0000 7f232be00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232c0ad7c0 con 0x7f232c156660 2024-03-09T08:58:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.209+0000 7f232be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f232c06d740 con 0x7f232c0c61e0 2024-03-09T08:58:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.209+0000 7f232be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232c0c6660 con 0x7f232c0c5e00 2024-03-09T08:58:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.210+0000 7f232a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f232c0c61e0 0x7f232c0c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.210+0000 7f232a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f232c0c61e0 0x7f232c0c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47942/0 (socket says 172.21.15.67:47942) 2024-03-09T08:58:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.210+0000 7f232a000640 1 -- 172.21.15.67:0/2927468028 learned_addr learned my addr 172.21.15.67:0/2927468028 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:39.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.210+0000 7f232aa00640 1 --2- 172.21.15.67:0/2927468028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f232c0c5e00 0x7f232c0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.210+0000 7f232a000640 1 -- 172.21.15.67:0/2927468028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f232c0c5e00 msgr2=0x7f232c0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.210+0000 7f232a000640 1 --2- 172.21.15.67:0/2927468028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f232c0c5e00 0x7f232c0acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.210+0000 7f232a000640 1 -- 172.21.15.67:0/2927468028 >> v1:172.21.15.134:6789/0 conn(0x7f232c156660 legacy=0x7f232c06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.210+0000 7f232a000640 1 -- 172.21.15.67:0/2927468028 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c156af0 con 0x7f232c0c61e0 2024-03-09T08:58:39.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.211+0000 7f232a000640 1 --2- 172.21.15.67:0/2927468028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f232c0c61e0 0x7f232c0c9380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2318009870 tx=0x7f231802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774c849fa3fdf8de server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.212+0000 7f2329600640 1 -- 172.21.15.67:0/2927468028 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318034070 con 0x7f232c0c61e0 2024-03-09T08:58:39.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.212+0000 7f2329600640 1 -- 172.21.15.67:0/2927468028 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2318030840 con 0x7f232c0c61e0 2024-03-09T08:58:39.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.212+0000 7f2329600640 1 -- 172.21.15.67:0/2927468028 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318030c10 con 0x7f232c0c61e0 2024-03-09T08:58:39.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.212+0000 7f232be00640 1 -- 172.21.15.67:0/2927468028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f232c0c61e0 msgr2=0x7f232c0c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.212+0000 7f232be00640 1 --2- 172.21.15.67:0/2927468028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f232c0c61e0 0x7f232c0c9380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2318009870 tx=0x7f231802b980 comp rx=0 tx=0).stop 2024-03-09T08:58:39.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.212+0000 7f232be00640 1 -- 172.21.15.67:0/2927468028 shutdown_connections 2024-03-09T08:58:39.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.212+0000 7f232be00640 1 --2- 172.21.15.67:0/2927468028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f232c0c61e0 0x7f232c0c9380 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.212+0000 7f232be00640 1 --2- 172.21.15.67:0/2927468028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f232c0c5e00 0x7f232c0acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.212+0000 7f232be00640 1 -- 172.21.15.67:0/2927468028 >> 172.21.15.67:0/2927468028 conn(0x7f232c0cb480 msgr2=0x7f232c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:39.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.212+0000 7f232be00640 1 -- 172.21.15.67:0/2927468028 shutdown_connections 2024-03-09T08:58:39.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.212+0000 7f232be00640 1 -- 172.21.15.67:0/2927468028 wait complete. 2024-03-09T08:58:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.214+0000 7f232be00640 1 Processor -- start 2024-03-09T08:58:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.214+0000 7f232be00640 1 -- start start 2024-03-09T08:58:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.215+0000 7f232be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f232c0c5e00 0x7f232c150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.215+0000 7f232be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f232c0c61e0 0x7f232c150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.215+0000 7f232be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c156660 0x7f232c1ef330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.215+0000 7f232be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c06bb00 con 0x7f232c0c5e00 2024-03-09T08:58:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.215+0000 7f232be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c153680 con 0x7f232c156660 2024-03-09T08:58:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.215+0000 7f232be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232c06bde0 con 0x7f232c0c61e0 2024-03-09T08:58:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.215+0000 7f232aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f232c0c5e00 0x7f232c150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.215+0000 7f232aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f232c0c5e00 0x7f232c150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47956/0 (socket says 172.21.15.67:47956) 2024-03-09T08:58:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.215+0000 7f232aa00640 1 -- 172.21.15.67:0/3151981970 learned_addr learned my addr 172.21.15.67:0/3151981970 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.216+0000 7f232a000640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f232c0c61e0 0x7f232c150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.216+0000 7f232aa00640 1 -- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f232c0c61e0 msgr2=0x7f232c150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.216+0000 7f232aa00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f232c0c61e0 0x7f232c150a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.216+0000 7f232aa00640 1 -- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c156660 msgr2=0x7f232c1ef330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.216+0000 7f232aa00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c156660 0x7f232c1ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.216+0000 7f232aa00640 1 -- 172.21.15.67:0/3151981970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c0c3fb0 con 0x7f232c0c5e00 2024-03-09T08:58:39.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.216+0000 7f232aa00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f232c0c5e00 0x7f232c150550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f231c00d830 tx=0x7f231c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.216+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c00b660 con 0x7f232c0c5e00 2024-03-09T08:58:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.216+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231c019070 con 0x7f232c0c5e00 2024-03-09T08:58:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.216+0000 7f232be00640 1 -- 172.21.15.67:0/3151981970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232c0c2710 con 0x7f232c0c5e00 2024-03-09T08:58:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.216+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c00ba20 con 0x7f232c0c5e00 2024-03-09T08:58:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.217+0000 7f232be00640 1 -- 172.21.15.67:0/3151981970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232c157aa0 con 0x7f232c0c5e00 2024-03-09T08:58:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.218+0000 7f232be00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c157ce0 0x7f232c1ef870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.218+0000 7f232be00640 1 -- 172.21.15.67:0/3151981970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c04f3c0 con 0x7f232c157ce0 2024-03-09T08:58:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.218+0000 7f232a000640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c157ce0 0x7f232c1ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.219+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f231c01e460 con 0x7f232c0c5e00 2024-03-09T08:58:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.219+0000 7f232a000640 1 -- 172.21.15.67:0/3151981970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f232c06d2f0 con 0x7f232c157ce0 2024-03-09T08:58:39.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.219+0000 7f232a000640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c157ce0 0x7f232c1ef870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f232c06b1f0 tx=0x7f2318004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.220+0000 7f2322a00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f230c06b4d0 0x7f230c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.220+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f231c08ac30 con 0x7f232c0c5e00 2024-03-09T08:58:39.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.221+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318034040 con 0x7f232c157ce0 2024-03-09T08:58:39.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.221+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f2318043400 con 0x7f232c157ce0 2024-03-09T08:58:39.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.221+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c157ce0 msgr2=0x7f232c1ef870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.221+0000 7f2322a00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c157ce0 0x7f232c1ef870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f232c06b1f0 tx=0x7f2318004730 comp rx=0 tx=0).stop 2024-03-09T08:58:39.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.221+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c157ce0 msgr2=0x7f232c1ef870 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:39.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.221+0000 7f2322a00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c157ce0 0x7f232c1ef870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.221+0000 7f232b400640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f230c06b4d0 0x7f230c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.222+0000 7f232b400640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f230c06b4d0 0x7f230c06d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f23240012e0 tx=0x7f2324007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.228+0000 7f232be00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c0c38d0 0x7f232c159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.228+0000 7f232be00640 1 -- 172.21.15.67:0/3151981970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c1efdb0 con 0x7f232c0c38d0 2024-03-09T08:58:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.229+0000 7f232a000640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c0c38d0 0x7f232c159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.230+0000 7f232a000640 1 -- 172.21.15.67:0/3151981970 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f232c0c7b50 con 0x7f232c0c38d0 2024-03-09T08:58:39.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.230+0000 7f232a000640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c0c38d0 0x7f232c159410 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2318009870 tx=0x7f23180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.231+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318034050 con 0x7f232c0c38d0 2024-03-09T08:58:39.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.231+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5043 (secure 0 0 0) 0x7f2318009490 con 0x7f232c0c38d0 2024-03-09T08:58:39.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.231+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c0c38d0 msgr2=0x7f232c159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.231+0000 7f2322a00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c0c38d0 0x7f232c159410 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2318009870 tx=0x7f23180046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.231+0000 7f2322a00640 1 -- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c0c38d0 msgr2=0x7f232c159410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:39.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.231+0000 7f2322a00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f232c0c38d0 0x7f232c159410 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2318009870 tx=0x7f23180046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.232 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:58:39.232 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:58:39.232 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:58:39.232 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:58:39.232 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 50, 2024-03-09T08:58:39.232 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:58:39.232 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:58:39.232 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:58:39.232 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:39.233 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:39.234 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:39.235 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:39.236 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:58:39.238 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.240 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.241 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:39.242 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:39.243 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:39.244 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:39.245 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:39.245 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.245 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.245 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:58:39.245 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:58:39.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.232+0000 7f232be00640 1 -- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f230c06b4d0 msgr2=0x7f230c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.232+0000 7f232be00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f230c06b4d0 0x7f230c06d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f23240012e0 tx=0x7f2324007d30 comp rx=0 tx=0).stop 2024-03-09T08:58:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.232+0000 7f232be00640 1 -- 172.21.15.67:0/3151981970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f232c0c5e00 msgr2=0x7f232c150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.232+0000 7f232be00640 1 --2- 172.21.15.67:0/3151981970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f232c0c5e00 0x7f232c150550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f231c00d830 tx=0x7f231c00dcf0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.232+0000 7f232b400640 1 -- 172.21.15.67:0/3151981970 reap_dead start 2024-03-09T08:58:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.232+0000 7f232b400640 1 -- 172.21.15.67:0/3151981970 reap_dead start 2024-03-09T08:58:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.232+0000 7f232be00640 1 -- 172.21.15.67:0/3151981970 shutdown_connections 2024-03-09T08:58:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.232+0000 7f232be00640 1 -- 172.21.15.67:0/3151981970 >> 172.21.15.67:0/3151981970 conn(0x7f232c0cb480 msgr2=0x7f232c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.232+0000 7f232be00640 1 -- 172.21.15.67:0/3151981970 shutdown_connections 2024-03-09T08:58:39.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.232+0000 7f232be00640 1 -- 172.21.15.67:0/3151981970 wait complete. 2024-03-09T08:58:39.248 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:58:39.248 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:58:39.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:58:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.445+0000 7f30dce00640 1 Processor -- start 2024-03-09T08:58:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.445+0000 7f30dce00640 1 -- start start 2024-03-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.446+0000 7f30dce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f30d80ac9a0 0x7f30d81510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.446+0000 7f30dce00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f30d80c2780 0x7f30d8151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.446+0000 7f30dce00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d80c8200 con 0x7f30d8071190 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.447+0000 7f30dce00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d806d590 con 0x7f30d80ac9a0 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.447+0000 7f30dce00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d80c3740 con 0x7f30d80c2780 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.447+0000 7f30d5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f30d80ac9a0 0x7f30d81510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.447+0000 7f30d5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f30d80ac9a0 0x7f30d81510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47964/0 (socket says 172.21.15.67:47964) 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.447+0000 7f30d5e00640 1 -- 172.21.15.67:0/2755848537 learned_addr learned my addr 172.21.15.67:0/2755848537 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.447+0000 7f30d6800640 1 --1- 172.21.15.67:0/2755848537 >> v1:172.21.15.67:6789/0 conn(0x7f30d8071190 0x7f30d8154190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51116/0 (socket says 172.21.15.67:51116) 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.447+0000 7f30d5400640 1 --2- 172.21.15.67:0/2755848537 >> v2:172.21.15.134:3300/0 conn(0x7f30d80c2780 0x7f30d8151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.447+0000 7f30d5e00640 1 -- 172.21.15.67:0/2755848537 >> v2:172.21.15.134:3300/0 conn(0x7f30d80c2780 msgr2=0x7f30d8151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.447+0000 7f30d5e00640 1 --2- 172.21.15.67:0/2755848537 >> v2:172.21.15.134:3300/0 conn(0x7f30d80c2780 0x7f30d8151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.447+0000 7f30d5e00640 1 -- 172.21.15.67:0/2755848537 >> v1:172.21.15.67:6789/0 conn(0x7f30d8071190 legacy=0x7f30d8154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.447+0000 7f30d5e00640 1 -- 172.21.15.67:0/2755848537 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d80c3990 con 0x7f30d80ac9a0 2024-03-09T08:58:39.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.449+0000 7f30d5e00640 1 --2- 172.21.15.67:0/2755848537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d80ac9a0 0x7f30d81510e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f30d8070320 tx=0x7f30c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72f8b35a7fd56a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.449+0000 7f30d4a00640 1 -- 172.21.15.67:0/2755848537 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8035020 con 0x7f30d80ac9a0 2024-03-09T08:58:39.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.449+0000 7f30d4a00640 1 -- 172.21.15.67:0/2755848537 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c8031870 con 0x7f30d80ac9a0 2024-03-09T08:58:39.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.449+0000 7f30d4a00640 1 -- 172.21.15.67:0/2755848537 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8031c40 con 0x7f30d80ac9a0 2024-03-09T08:58:39.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.449+0000 7f30dce00640 1 -- 172.21.15.67:0/2755848537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d80ac9a0 msgr2=0x7f30d81510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.449+0000 7f30dce00640 1 --2- 172.21.15.67:0/2755848537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d80ac9a0 0x7f30d81510e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f30d8070320 tx=0x7f30c802b910 comp rx=0 tx=0).stop 2024-03-09T08:58:39.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.450+0000 7f30dce00640 1 -- 172.21.15.67:0/2755848537 shutdown_connections 2024-03-09T08:58:39.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.450+0000 7f30dce00640 1 --2- 172.21.15.67:0/2755848537 >> v2:172.21.15.134:3300/0 conn(0x7f30d80c2780 0x7f30d8151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.450+0000 7f30dce00640 1 --2- 172.21.15.67:0/2755848537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d80ac9a0 0x7f30d81510e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.450+0000 7f30dce00640 1 -- 172.21.15.67:0/2755848537 >> 172.21.15.67:0/2755848537 conn(0x7f30d80c7370 msgr2=0x7f30d80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.450+0000 7f30dce00640 1 -- 172.21.15.67:0/2755848537 shutdown_connections 2024-03-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.450+0000 7f30dce00640 1 -- 172.21.15.67:0/2755848537 wait complete. 2024-03-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.451+0000 7f30dce00640 1 Processor -- start 2024-03-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.452+0000 7f30dce00640 1 -- start start 2024-03-09T08:58:39.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.452+0000 7f30dce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d8071190 0x7f30d8149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30dce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d80ac9a0 0x7f30d81ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30dce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30d80c2780 0x7f30d81480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30dce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d80acd80 con 0x7f30d8071190 2024-03-09T08:58:39.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30dce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d8070c00 con 0x7f30d80c2780 2024-03-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30dce00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d80acf00 con 0x7f30d80ac9a0 2024-03-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30d5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d8071190 0x7f30d8149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30d5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d8071190 0x7f30d8149970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47974/0 (socket says 172.21.15.67:47974) 2024-03-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30d5e00640 1 -- 172.21.15.67:0/2264494773 learned_addr learned my addr 172.21.15.67:0/2264494773 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30d5e00640 1 -- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d80ac9a0 msgr2=0x7f30d81ebe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30d5e00640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d80ac9a0 0x7f30d81ebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30d5e00640 1 -- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30d80c2780 msgr2=0x7f30d81480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30d5e00640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30d80c2780 0x7f30d81480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.453+0000 7f30d5e00640 1 -- 172.21.15.67:0/2264494773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d814cf60 con 0x7f30d8071190 2024-03-09T08:58:39.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.454+0000 7f30d5e00640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d8071190 0x7f30d8149970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f30c8000c00 tx=0x7f30c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.454+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8035040 con 0x7f30d8071190 2024-03-09T08:58:39.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.454+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c8030d00 con 0x7f30d8071190 2024-03-09T08:58:39.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.454+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c80456a0 con 0x7f30d8071190 2024-03-09T08:58:39.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.454+0000 7f30dce00640 1 -- 172.21.15.67:0/2264494773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d814d240 con 0x7f30d8071190 2024-03-09T08:58:39.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.454+0000 7f30dce00640 1 -- 172.21.15.67:0/2264494773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d80c67a0 con 0x7f30d8071190 2024-03-09T08:58:39.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.456+0000 7f30dce00640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d814eb70 0x7f30d8148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.456+0000 7f30dce00640 1 -- 172.21.15.67:0/2264494773 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d814d760 con 0x7f30d814eb70 2024-03-09T08:58:39.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.456+0000 7f30d5400640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d814eb70 0x7f30d8148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.456+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f30c8038020 con 0x7f30d8071190 2024-03-09T08:58:39.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.457+0000 7f30d5400640 1 -- 172.21.15.67:0/2264494773 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30d81eb760 con 0x7f30d814eb70 2024-03-09T08:58:39.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.457+0000 7f30ce000640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f30b806b4d0 0x7f30b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.457+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f30c80ac9e0 con 0x7f30d8071190 2024-03-09T08:58:39.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.458+0000 7f30d5400640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d814eb70 0x7f30d8148620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30d80c2bc0 tx=0x7f30c400ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.458+0000 7f30d6800640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f30b806b4d0 0x7f30b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.458+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4017070 con 0x7f30d814eb70 2024-03-09T08:58:39.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.459+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f30c400c720 con 0x7f30d814eb70 2024-03-09T08:58:39.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.459+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d814eb70 msgr2=0x7f30d8148620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.459+0000 7f30ce000640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d814eb70 0x7f30d8148620 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30d80c2bc0 tx=0x7f30c400ec00 comp rx=0 tx=0).stop 2024-03-09T08:58:39.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.459+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d814eb70 msgr2=0x7f30d8148620 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:39.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.459+0000 7f30ce000640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d814eb70 0x7f30d8148620 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30d80c2bc0 tx=0x7f30c400ec00 comp rx=0 tx=0).stop 2024-03-09T08:58:39.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.460+0000 7f30d6800640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f30b806b4d0 0x7f30b806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f30d814c900 tx=0x7f30d0007d40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.467+0000 7f30dce00640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d8148b60 0x7f30d81eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.467+0000 7f30dce00640 1 -- 172.21.15.67:0/2264494773 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d8075a00 con 0x7f30d8148b60 2024-03-09T08:58:39.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.467+0000 7f30d5400640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d8148b60 0x7f30d81eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.468+0000 7f30d5400640 1 -- 172.21.15.67:0/2264494773 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f30d8072ef0 con 0x7f30d8148b60 2024-03-09T08:58:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.469+0000 7f30d5400640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d8148b60 0x7f30d81eecd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f30d80c2bc0 tx=0x7f30c40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.469+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4017070 con 0x7f30d8148b60 2024-03-09T08:58:39.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.470+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5186 (secure 0 0 0) 0x7f30c40063f0 con 0x7f30d8148b60 2024-03-09T08:58:39.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.470+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d8148b60 msgr2=0x7f30d81eecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.470+0000 7f30ce000640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d8148b60 0x7f30d81eecd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f30d80c2bc0 tx=0x7f30c40046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.470+0000 7f30ce000640 1 -- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d8148b60 msgr2=0x7f30d81eecd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:39.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.470+0000 7f30ce000640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30d8148b60 0x7f30d81eecd0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f30d80c2bc0 tx=0x7f30c40046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.471 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:58:39.471 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:58:39.471 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 50, 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:39.472 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:39.473 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:39.474 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:39.474 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:39.474 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:39.474 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:39.474 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:39.474 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:39.474 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:39.474 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:39.474 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:39.474 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.475 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:58:39.475 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:58:39.475 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:58:39.475 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:58:39.475 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:58:39.475 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:58:39.475 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:39.475 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:39.475 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:58:39.475 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:39.476 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:39.478 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:58:39.478 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.478 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:39.478 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:58:39.478 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.478 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.478 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:39.480 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:58:39.481 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.482 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.483 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:39.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.485 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:39.485 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:58:39.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:39.485 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:39.485 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:39.485 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:39.485 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:39.485 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:39.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:39.485 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:58:39.485 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:58:39.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.470+0000 7f30dce00640 1 -- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f30b806b4d0 msgr2=0x7f30b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.470+0000 7f30dce00640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f30b806b4d0 0x7f30b806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f30d814c900 tx=0x7f30d0007d40 comp rx=0 tx=0).stop 2024-03-09T08:58:39.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.470+0000 7f30dce00640 1 -- 172.21.15.67:0/2264494773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d8071190 msgr2=0x7f30d8149970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.470+0000 7f30dce00640 1 --2- 172.21.15.67:0/2264494773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30d8071190 0x7f30d8149970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f30c8000c00 tx=0x7f30c80046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.470+0000 7f30d6800640 1 -- 172.21.15.67:0/2264494773 reap_dead start 2024-03-09T08:58:39.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.471+0000 7f30d6800640 1 -- 172.21.15.67:0/2264494773 reap_dead start 2024-03-09T08:58:39.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.471+0000 7f30dce00640 1 -- 172.21.15.67:0/2264494773 shutdown_connections 2024-03-09T08:58:39.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.471+0000 7f30dce00640 1 -- 172.21.15.67:0/2264494773 >> 172.21.15.67:0/2264494773 conn(0x7f30d80c7370 msgr2=0x7f30d814d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:39.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.471+0000 7f30dce00640 1 -- 172.21.15.67:0/2264494773 shutdown_connections 2024-03-09T08:58:39.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.471+0000 7f30dce00640 1 -- 172.21.15.67:0/2264494773 wait complete. 2024-03-09T08:58:39.490 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:58:39.490 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T08:58:39.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T08:58:39.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.704+0000 7fb4c5c00640 1 Processor -- start 2024-03-09T08:58:39.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.704+0000 7fb4c5c00640 1 -- start start 2024-03-09T08:58:39.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.706+0000 7fb4c5c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4c0071190 0x7fb4c0157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.706+0000 7fb4c5c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c0153d40 con 0x7fb4c00c22a0 2024-03-09T08:58:39.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.706+0000 7fb4c5c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c00c2790 con 0x7fb4c00ac730 2024-03-09T08:58:39.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.706+0000 7fb4c5c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c0070ee0 con 0x7fb4c0071190 2024-03-09T08:58:39.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.706+0000 7fb4bf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4c0071190 0x7fb4c0157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.706+0000 7fb4b6000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb4c00c22a0 0x7fb4c015c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51120/0 (socket says 172.21.15.67:51120) 2024-03-09T08:58:39.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.706+0000 7fb4bf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4c0071190 0x7fb4c0157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47988/0 (socket says 172.21.15.67:47988) 2024-03-09T08:58:39.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.706+0000 7fb4b6000640 1 -- 172.21.15.67:0/1309753089 learned_addr learned my addr 172.21.15.67:0/1309753089 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:58:39.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.707+0000 7fb4bf400640 1 -- 172.21.15.67:0/1309753089 >> v1:172.21.15.134:6789/0 conn(0x7fb4c00ac730 legacy=0x7fb4c0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:39.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.707+0000 7fb4bf400640 1 -- 172.21.15.67:0/1309753089 >> v1:172.21.15.67:6789/0 conn(0x7fb4c00c22a0 legacy=0x7fb4c015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.707+0000 7fb4bf400640 1 -- 172.21.15.67:0/1309753089 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c0070920 con 0x7fb4c0071190 2024-03-09T08:58:39.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.707+0000 7fb4be000640 1 -- 172.21.15.67:0/1309753089 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175897357 0 0) 0x7fb4c0153d40 con 0x7fb4c00c22a0 2024-03-09T08:58:39.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.708+0000 7fb4bf400640 1 --2- 172.21.15.67:0/1309753089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4c0071190 0x7fb4c0157d60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80097a0 tx=0x7fb4b802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5ecb51ab758faf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.708+0000 7fb4be000640 1 -- 172.21.15.67:0/1309753089 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80315a0 con 0x7fb4c0071190 2024-03-09T08:58:39.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.708+0000 7fb4be000640 1 -- 172.21.15.67:0/1309753089 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b8037070 con 0x7fb4c0071190 2024-03-09T08:58:39.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.708+0000 7fb4be000640 1 -- 172.21.15.67:0/1309753089 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8031960 con 0x7fb4c0071190 2024-03-09T08:58:39.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.709+0000 7fb4c5c00640 1 -- 172.21.15.67:0/1309753089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4c0071190 msgr2=0x7fb4c0157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.709+0000 7fb4c5c00640 1 --2- 172.21.15.67:0/1309753089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4c0071190 0x7fb4c0157d60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80097a0 tx=0x7fb4b802b840 comp rx=0 tx=0).stop 2024-03-09T08:58:39.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.709+0000 7fb4c5c00640 1 -- 172.21.15.67:0/1309753089 shutdown_connections 2024-03-09T08:58:39.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.709+0000 7fb4c5c00640 1 --2- 172.21.15.67:0/1309753089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4c0071190 0x7fb4c0157d60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.709+0000 7fb4c5c00640 1 -- 172.21.15.67:0/1309753089 >> 172.21.15.67:0/1309753089 conn(0x7fb4c00c7370 msgr2=0x7fb4c00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:39.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.709+0000 7fb4c5c00640 1 -- 172.21.15.67:0/1309753089 shutdown_connections 2024-03-09T08:58:39.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.709+0000 7fb4c5c00640 1 -- 172.21.15.67:0/1309753089 wait complete. 2024-03-09T08:58:39.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.711+0000 7fb4c5c00640 1 Processor -- start 2024-03-09T08:58:39.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.711+0000 7fb4c5c00640 1 -- start start 2024-03-09T08:58:39.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4c5c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4c0071190 0x7fb4c0152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4c5c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4c00ac730 0x7fb4c01dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4c5c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4c00c22a0 0x7fb4c01dd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4c5c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c0151610 con 0x7fb4c00c22a0 2024-03-09T08:58:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4c5c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c0154d00 con 0x7fb4c0071190 2024-03-09T08:58:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4c5c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c0151790 con 0x7fb4c00ac730 2024-03-09T08:58:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4bf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4c00c22a0 0x7fb4c01dd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4bf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4c00c22a0 0x7fb4c01dd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47994/0 (socket says 172.21.15.67:47994) 2024-03-09T08:58:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4bf400640 1 -- 172.21.15.67:0/2692368567 learned_addr learned my addr 172.21.15.67:0/2692368567 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4bea00640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4c0071190 0x7fb4c0152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4bf400640 1 -- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4c00ac730 msgr2=0x7fb4c01dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4bf400640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4c00ac730 0x7fb4c01dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4bf400640 1 -- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4c0071190 msgr2=0x7fb4c0152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4bf400640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4c0071190 0x7fb4c0152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:39.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.712+0000 7fb4bf400640 1 -- 172.21.15.67:0/2692368567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c01df080 con 0x7fb4c00c22a0 2024-03-09T08:58:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.713+0000 7fb4bf400640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4c00c22a0 0x7fb4c01dd800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8034040 tx=0x7fb4b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.713+0000 7fb4b7400640 1 -- 172.21.15.67:0/2692368567 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8004230 con 0x7fb4c00c22a0 2024-03-09T08:58:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.713+0000 7fb4b7400640 1 -- 172.21.15.67:0/2692368567 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b80043d0 con 0x7fb4c00c22a0 2024-03-09T08:58:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.713+0000 7fb4c5c00640 1 -- 172.21.15.67:0/2692368567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c01e9ab0 con 0x7fb4c00c22a0 2024-03-09T08:58:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.713+0000 7fb4b7400640 1 -- 172.21.15.67:0/2692368567 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8032880 con 0x7fb4c00c22a0 2024-03-09T08:58:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.713+0000 7fb4c5c00640 1 -- 172.21.15.67:0/2692368567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c01534e0 con 0x7fb4c00c22a0 2024-03-09T08:58:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.715+0000 7fb4c5c00640 1 -- 172.21.15.67:0/2692368567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c01e98a0 con 0x7fb4c00c22a0 2024-03-09T08:58:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.715+0000 7fb4b7400640 1 -- 172.21.15.67:0/2692368567 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb4b8032a20 con 0x7fb4c00c22a0 2024-03-09T08:58:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.716+0000 7fb4b7400640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb49c06b4d0 0x7fb49c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.716+0000 7fb4b7400640 1 -- 172.21.15.67:0/2692368567 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb4b802f030 con 0x7fb4c00c22a0 2024-03-09T08:58:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.717+0000 7fb4bea00640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb49c06b4d0 0x7fb49c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:39.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.718+0000 7fb4bea00640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb49c06b4d0 0x7fb49c06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0007fd0 tx=0x7fb4b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:39.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:39.721+0000 7fb4b7400640 1 -- 172.21.15.67:0/2692368567 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb4b8041070 con 0x7fb4c00c22a0 2024-03-09T08:58:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.010+0000 7fb4c5c00640 1 -- 172.21.15.67:0/2692368567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fb4c00c6180 con 0x7fb4c00c22a0 2024-03-09T08:58:40.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.012+0000 7fb4b7400640 1 -- 172.21.15.67:0/2692368567 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fb4b807b160 con 0x7fb4c00c22a0 2024-03-09T08:58:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.016+0000 7fb4c5c00640 1 -- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb49c06b4d0 msgr2=0x7fb49c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:40.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.016+0000 7fb4c5c00640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb49c06b4d0 0x7fb49c06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0007fd0 tx=0x7fb4b0009040 comp rx=0 tx=0).stop 2024-03-09T08:58:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.016+0000 7fb4c5c00640 1 -- 172.21.15.67:0/2692368567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4c00c22a0 msgr2=0x7fb4c01dd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.016+0000 7fb4c5c00640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4c00c22a0 0x7fb4c01dd800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8034040 tx=0x7fb4b80046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.016+0000 7fb4c5c00640 1 -- 172.21.15.67:0/2692368567 shutdown_connections 2024-03-09T08:58:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.016+0000 7fb4c5c00640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb49c06b4d0 0x7fb49c06d990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.016+0000 7fb4c5c00640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4c00c22a0 0x7fb4c01dd800 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.016+0000 7fb4c5c00640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4c00ac730 0x7fb4c01dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.016+0000 7fb4c5c00640 1 --2- 172.21.15.67:0/2692368567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4c0071190 0x7fb4c0152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.017+0000 7fb4c5c00640 1 -- 172.21.15.67:0/2692368567 >> 172.21.15.67:0/2692368567 conn(0x7fb4c00c7370 msgr2=0x7fb4c0157650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.017+0000 7fb4c5c00640 1 -- 172.21.15.67:0/2692368567 shutdown_connections 2024-03-09T08:58:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:40.017+0000 7fb4c5c00640 1 -- 172.21.15.67:0/2692368567 wait complete. 2024-03-09T08:58:40.041 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T08:58:41.601 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:58:41.609 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:58:50.042 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:58:50.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:58:50.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.272+0000 7fdb6bc00640 1 Processor -- start 2024-03-09T08:58:50.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.272+0000 7fdb6bc00640 1 -- start start 2024-03-09T08:58:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.273+0000 7fdb6bc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdb6413f3b0 0x7fdb640737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.274+0000 7fdb6bc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb641597b0 0x7fdb64159be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.274+0000 7fdb6bc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb64156300 con 0x7fdb64155d60 2024-03-09T08:58:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.274+0000 7fdb6bc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb640700a0 con 0x7fdb6413f3b0 2024-03-09T08:58:50.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.274+0000 7fdb6bc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb641565e0 con 0x7fdb641597b0 2024-03-09T08:58:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.274+0000 7fdb69c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb641597b0 0x7fdb64159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.274+0000 7fdb69c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb641597b0 0x7fdb64159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:41018/0 (socket says 172.21.15.67:41018) 2024-03-09T08:58:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.274+0000 7fdb69c00640 1 -- 172.21.15.67:0/4238942040 learned_addr learned my addr 172.21.15.67:0/4238942040 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.274+0000 7fdb69200640 1 --2- 172.21.15.67:0/4238942040 >> v2:172.21.15.134:3300/0 conn(0x7fdb6413f3b0 0x7fdb640737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.275+0000 7fdb69c00640 1 -- 172.21.15.67:0/4238942040 >> v2:172.21.15.134:3300/0 conn(0x7fdb6413f3b0 msgr2=0x7fdb640737b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.275+0000 7fdb69c00640 1 --2- 172.21.15.67:0/4238942040 >> v2:172.21.15.134:3300/0 conn(0x7fdb6413f3b0 0x7fdb640737b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.275+0000 7fdb69c00640 1 -- 172.21.15.67:0/4238942040 >> v1:172.21.15.134:6789/0 conn(0x7fdb64155d60 legacy=0x7fdb640c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.275+0000 7fdb69c00640 1 -- 172.21.15.67:0/4238942040 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb641568c0 con 0x7fdb641597b0 2024-03-09T08:58:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.275+0000 7fdb69200640 1 --2- 172.21.15.67:0/4238942040 >> v2:172.21.15.134:3300/0 conn(0x7fdb6413f3b0 0x7fdb640737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:58:50.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.276+0000 7fdb69c00640 1 --2- 172.21.15.67:0/4238942040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb641597b0 0x7fdb64159be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009e50 tx=0x7fdb5c02be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b4727ba8be7b21b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.277+0000 7fdb63e00640 1 -- 172.21.15.67:0/4238942040 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c02c9e0 con 0x7fdb641597b0 2024-03-09T08:58:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.277+0000 7fdb63e00640 1 -- 172.21.15.67:0/4238942040 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c02cb80 con 0x7fdb641597b0 2024-03-09T08:58:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.277+0000 7fdb63e00640 1 -- 172.21.15.67:0/4238942040 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c0343f0 con 0x7fdb641597b0 2024-03-09T08:58:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.278+0000 7fdb6bc00640 1 -- 172.21.15.67:0/4238942040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb641597b0 msgr2=0x7fdb64159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.278+0000 7fdb6bc00640 1 --2- 172.21.15.67:0/4238942040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb641597b0 0x7fdb64159be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009e50 tx=0x7fdb5c02be40 comp rx=0 tx=0).stop 2024-03-09T08:58:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.278+0000 7fdb6bc00640 1 -- 172.21.15.67:0/4238942040 shutdown_connections 2024-03-09T08:58:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.278+0000 7fdb6bc00640 1 --2- 172.21.15.67:0/4238942040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb641597b0 0x7fdb64159be0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.278+0000 7fdb6bc00640 1 --2- 172.21.15.67:0/4238942040 >> v2:172.21.15.134:3300/0 conn(0x7fdb6413f3b0 0x7fdb640737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.278+0000 7fdb6bc00640 1 -- 172.21.15.67:0/4238942040 >> 172.21.15.67:0/4238942040 conn(0x7fdb64066f70 msgr2=0x7fdb64067350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.278+0000 7fdb6bc00640 1 -- 172.21.15.67:0/4238942040 shutdown_connections 2024-03-09T08:58:50.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.278+0000 7fdb6bc00640 1 -- 172.21.15.67:0/4238942040 wait complete. 2024-03-09T08:58:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.279+0000 7fdb6bc00640 1 Processor -- start 2024-03-09T08:58:50.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.280+0000 7fdb6bc00640 1 -- start start 2024-03-09T08:58:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.280+0000 7fdb6bc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb6413f3b0 0x7fdb64136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.280+0000 7fdb6bc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb64155d60 0x7fdb640c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.280+0000 7fdb6bc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb641597b0 0x7fdb640c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.280+0000 7fdb6bc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb640c6cc0 con 0x7fdb641597b0 2024-03-09T08:58:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.280+0000 7fdb6bc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb640c6b40 con 0x7fdb6413f3b0 2024-03-09T08:58:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.280+0000 7fdb6bc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb64073cf0 con 0x7fdb64155d60 2024-03-09T08:58:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.281+0000 7fdb69c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb641597b0 0x7fdb640c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.281+0000 7fdb69c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb641597b0 0x7fdb640c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36904/0 (socket says 172.21.15.67:36904) 2024-03-09T08:58:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.281+0000 7fdb69c00640 1 -- 172.21.15.67:0/1837019474 learned_addr learned my addr 172.21.15.67:0/1837019474 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:50.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.281+0000 7fdb69200640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb6413f3b0 0x7fdb64136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.281+0000 7fdb5be00640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb64155d60 0x7fdb640c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.281+0000 7fdb69c00640 1 -- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb64155d60 msgr2=0x7fdb640c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.281+0000 7fdb69c00640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb64155d60 0x7fdb640c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.281+0000 7fdb69c00640 1 -- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb6413f3b0 msgr2=0x7fdb64136ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.281+0000 7fdb69c00640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb6413f3b0 0x7fdb64136ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.281+0000 7fdb69c00640 1 -- 172.21.15.67:0/1837019474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb640c5de0 con 0x7fdb641597b0 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.282+0000 7fdb69c00640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb641597b0 0x7fdb640c4450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009f70 tx=0x7fdb5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.282+0000 7fdb61600640 1 -- 172.21.15.67:0/1837019474 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c044020 con 0x7fdb641597b0 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.282+0000 7fdb61600640 1 -- 172.21.15.67:0/1837019474 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c0406b0 con 0x7fdb641597b0 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.282+0000 7fdb61600640 1 -- 172.21.15.67:0/1837019474 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c009510 con 0x7fdb641597b0 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.282+0000 7fdb6bc00640 1 -- 172.21.15.67:0/1837019474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb641ef630 con 0x7fdb641597b0 2024-03-09T08:58:50.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.282+0000 7fdb6bc00640 1 -- 172.21.15.67:0/1837019474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb64137010 con 0x7fdb641597b0 2024-03-09T08:58:50.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.284+0000 7fdb6bc00640 1 -- 172.21.15.67:0/1837019474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb64075a00 con 0x7fdb641597b0 2024-03-09T08:58:50.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.284+0000 7fdb61600640 1 -- 172.21.15.67:0/1837019474 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdb5c047020 con 0x7fdb641597b0 2024-03-09T08:58:50.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.285+0000 7fdb61600640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb4006b4d0 0x7fdb4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.285+0000 7fdb61600640 1 -- 172.21.15.67:0/1837019474 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdb5c0acff0 con 0x7fdb641597b0 2024-03-09T08:58:50.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.286+0000 7fdb69200640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb4006b4d0 0x7fdb4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.287+0000 7fdb69200640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb4006b4d0 0x7fdb4006d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb64155bb0 tx=0x7fdb4c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:50.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.291+0000 7fdb61600640 1 -- 172.21.15.67:0/1837019474 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdb5c04e030 con 0x7fdb641597b0 2024-03-09T08:58:50.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.573+0000 7fdb6bc00640 1 -- 172.21.15.67:0/1837019474 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdb641efb50 con 0x7fdb641597b0 2024-03-09T08:58:50.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.574+0000 7fdb61600640 1 -- 172.21.15.67:0/1837019474 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fdb5c07bbd0 con 0x7fdb641597b0 2024-03-09T08:58:50.574 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:58:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 -- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb4006b4d0 msgr2=0x7fdb4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb4006b4d0 0x7fdb4006d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb64155bb0 tx=0x7fdb4c008040 comp rx=0 tx=0).stop 2024-03-09T08:58:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 -- 172.21.15.67:0/1837019474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb641597b0 msgr2=0x7fdb640c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb641597b0 0x7fdb640c4450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009f70 tx=0x7fdb5c004a40 comp rx=0 tx=0).stop 2024-03-09T08:58:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 -- 172.21.15.67:0/1837019474 shutdown_connections 2024-03-09T08:58:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb4006b4d0 0x7fdb4006d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb641597b0 0x7fdb640c4450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb64155d60 0x7fdb640c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 --2- 172.21.15.67:0/1837019474 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb6413f3b0 0x7fdb64136ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 -- 172.21.15.67:0/1837019474 >> 172.21.15.67:0/1837019474 conn(0x7fdb64066f70 msgr2=0x7fdb6413bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 -- 172.21.15.67:0/1837019474 shutdown_connections 2024-03-09T08:58:50.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.576+0000 7fdb6bc00640 1 -- 172.21.15.67:0/1837019474 wait complete. 2024-03-09T08:58:50.596 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:58:50.597 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T08:58:50.597 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:58:50.597 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:58:50.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.815+0000 7f8dcdc00640 1 Processor -- start 2024-03-09T08:58:50.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.816+0000 7f8dcdc00640 1 -- start start 2024-03-09T08:58:50.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.818+0000 7f8dcdc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8dc80c2fa0 0x7f8dc806d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.818+0000 7f8dcdc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8dc8152790 0x7f8dc814fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.818+0000 7f8dcdc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc8150400 con 0x7f8dc814ff70 2024-03-09T08:58:50.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.818+0000 7f8dcdc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc8071b20 con 0x7f8dc8152790 2024-03-09T08:58:50.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.818+0000 7f8dcdc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc806d940 con 0x7f8dc80c2fa0 2024-03-09T08:58:50.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.819+0000 7f8dbfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8dc8152790 0x7f8dc814fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.819+0000 7f8dbfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8dc8152790 0x7f8dc814fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36906/0 (socket says 172.21.15.67:36906) 2024-03-09T08:58:50.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.819+0000 7f8dbfe00640 1 -- 172.21.15.67:0/2648574242 learned_addr learned my addr 172.21.15.67:0/2648574242 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:50.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.819+0000 7f8dc7e00640 1 --2- 172.21.15.67:0/2648574242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8dc80c2fa0 0x7f8dc806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.819+0000 7f8dbfe00640 1 -- 172.21.15.67:0/2648574242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8dc80c2fa0 msgr2=0x7f8dc806d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:50.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.819+0000 7f8dbfe00640 1 --2- 172.21.15.67:0/2648574242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8dc80c2fa0 0x7f8dc806d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.819+0000 7f8dbfe00640 1 -- 172.21.15.67:0/2648574242 >> v1:172.21.15.134:6789/0 conn(0x7f8dc814ff70 legacy=0x7f8dc8071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.819+0000 7f8dbfe00640 1 -- 172.21.15.67:0/2648574242 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc806dc20 con 0x7f8dc8152790 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.820+0000 7f8dbfe00640 1 --2- 172.21.15.67:0/2648574242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc8152790 0x7f8dc814fa30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8db00098a0 tx=0x7f8db002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790dc9108be154af server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.821+0000 7f8dc7400640 1 -- 172.21.15.67:0/2648574242 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00317c0 con 0x7f8dc8152790 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.821+0000 7f8dc7400640 1 -- 172.21.15.67:0/2648574242 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db0037070 con 0x7f8dc8152790 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.821+0000 7f8dc7400640 1 -- 172.21.15.67:0/2648574242 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0031b80 con 0x7f8dc8152790 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.821+0000 7f8dcdc00640 1 -- 172.21.15.67:0/2648574242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc8152790 msgr2=0x7f8dc814fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.821+0000 7f8dcdc00640 1 --2- 172.21.15.67:0/2648574242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc8152790 0x7f8dc814fa30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8db00098a0 tx=0x7f8db002b860 comp rx=0 tx=0).stop 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.822+0000 7f8dcdc00640 1 -- 172.21.15.67:0/2648574242 shutdown_connections 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.822+0000 7f8dcdc00640 1 --2- 172.21.15.67:0/2648574242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc8152790 0x7f8dc814fa30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.822+0000 7f8dcdc00640 1 --2- 172.21.15.67:0/2648574242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8dc80c2fa0 0x7f8dc806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.822+0000 7f8dcdc00640 1 -- 172.21.15.67:0/2648574242 >> 172.21.15.67:0/2648574242 conn(0x7f8dc80c7220 msgr2=0x7f8dc80c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.822+0000 7f8dcdc00640 1 -- 172.21.15.67:0/2648574242 shutdown_connections 2024-03-09T08:58:50.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.822+0000 7f8dcdc00640 1 -- 172.21.15.67:0/2648574242 wait complete. 2024-03-09T08:58:50.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.823+0000 7f8dcdc00640 1 Processor -- start 2024-03-09T08:58:50.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.824+0000 7f8dcdc00640 1 -- start start 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.824+0000 7f8dcdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc80c2fa0 0x7f8dc81518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.824+0000 7f8dcdc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc814ff70 0x7f8dc814eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.824+0000 7f8dcdc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8dc8152790 0x7f8dc80a9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.824+0000 7f8dcdc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80711c0 con 0x7f8dc80c2fa0 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.824+0000 7f8dcdc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc806e1e0 con 0x7f8dc814ff70 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.824+0000 7f8dcdc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc8071340 con 0x7f8dc8152790 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.825+0000 7f8dc7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc80c2fa0 0x7f8dc81518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.825+0000 7f8dc7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc80c2fa0 0x7f8dc81518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36912/0 (socket says 172.21.15.67:36912) 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.825+0000 7f8dc7e00640 1 -- 172.21.15.67:0/4091818349 learned_addr learned my addr 172.21.15.67:0/4091818349 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.825+0000 7f8dc7e00640 1 -- 172.21.15.67:0/4091818349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8dc8152790 msgr2=0x7f8dc80a9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.825+0000 7f8dc7e00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8dc8152790 0x7f8dc80a9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.825+0000 7f8dc7e00640 1 -- 172.21.15.67:0/4091818349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc814ff70 msgr2=0x7f8dc814eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.825+0000 7f8dc7e00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc814ff70 0x7f8dc814eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.825+0000 7f8dc7e00640 1 -- 172.21.15.67:0/4091818349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc814f3f0 con 0x7f8dc80c2fa0 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.826+0000 7f8dc7e00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc80c2fa0 0x7f8dc81518c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c820 tx=0x7f8db800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.826+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8015070 con 0x7f8dc80c2fa0 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.826+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db8004240 con 0x7f8dc80c2fa0 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.826+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db800ce10 con 0x7f8dc80c2fa0 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.826+0000 7f8dcdc00640 1 -- 172.21.15.67:0/4091818349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc814d4c0 con 0x7f8dc80c2fa0 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.826+0000 7f8dcdc00640 1 -- 172.21.15.67:0/4091818349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc814da80 con 0x7f8dc80c2fa0 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.827+0000 7f8dcdc00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc81e7e00 0x7f8dc80a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.827+0000 7f8dcdc00640 1 -- 172.21.15.67:0/4091818349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc804f3c0 con 0x7f8dc81e7e00 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.827+0000 7f8dbfe00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc81e7e00 0x7f8dc80a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.828+0000 7f8dbfe00640 1 -- 172.21.15.67:0/4091818349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8dc8144210 con 0x7f8dc81e7e00 2024-03-09T08:58:50.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.828+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8db8007830 con 0x7f8dc80c2fa0 2024-03-09T08:58:50.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.829+0000 7f8dbfe00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc81e7e00 0x7f8dc80a9740 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009870 tx=0x7f8db0031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:50.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.829+0000 7f8dc4c00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8da406b4d0 0x7f8da406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.829+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8db808a420 con 0x7f8dc80c2fa0 2024-03-09T08:58:50.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.830+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0041070 con 0x7f8dc81e7e00 2024-03-09T08:58:50.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.830+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f8db0037040 con 0x7f8dc81e7e00 2024-03-09T08:58:50.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.830+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc81e7e00 msgr2=0x7f8dc80a9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.830+0000 7f8dc4c00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc81e7e00 0x7f8dc80a9740 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009870 tx=0x7f8db0031bc0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.830+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc81e7e00 msgr2=0x7f8dc80a9740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:50.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.830+0000 7f8dc4c00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc81e7e00 0x7f8dc80a9740 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009870 tx=0x7f8db0031bc0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.830+0000 7f8dcd200640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8da406b4d0 0x7f8da406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.832+0000 7f8dcd200640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8da406b4d0 0x7f8da406d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0004340 tx=0x7f8dc0007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:50.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.844+0000 7f8dcdc00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc814dcc0 0x7f8dc814e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:50.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.844+0000 7f8dcdc00640 1 -- 172.21.15.67:0/4091818349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80c7a70 con 0x7f8dc814dcc0 2024-03-09T08:58:50.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.845+0000 7f8dbfe00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc814dcc0 0x7f8dc814e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:50.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.845+0000 7f8dbfe00640 1 -- 172.21.15.67:0/4091818349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8dc8151e00 con 0x7f8dc814dcc0 2024-03-09T08:58:50.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.845+0000 7f8dbfe00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc814dcc0 0x7f8dc814e280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8db00098a0 tx=0x7f8db0031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:50.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.846+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0050070 con 0x7f8dc814dcc0 2024-03-09T08:58:50.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.846+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4536 (secure 0 0 0) 0x7f8db004ca80 con 0x7f8dc814dcc0 2024-03-09T08:58:50.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.846+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc814dcc0 msgr2=0x7f8dc814e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.846+0000 7f8dc4c00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc814dcc0 0x7f8dc814e280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8db00098a0 tx=0x7f8db0031000 comp rx=0 tx=0).stop 2024-03-09T08:58:50.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.846+0000 7f8dc4c00640 1 -- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc814dcc0 msgr2=0x7f8dc814e280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:50.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.846+0000 7f8dc4c00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc814dcc0 0x7f8dc814e280 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8db00098a0 tx=0x7f8db0031000 comp rx=0 tx=0).stop 2024-03-09T08:58:50.847 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:58:50.847 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:58:50.847 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:58:50.847 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:58:50.847 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 50, 2024-03-09T08:58:50.847 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:58:50.847 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:58:50.847 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:58:50.848 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:58:50.848 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:50.848 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T08:58:50.848 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:50.848 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:58:50.848 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:58:50.848 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:50.848 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:50.848 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:50.849 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:58:50.850 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:58:50.851 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:50.852 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:50.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:50.854 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:58:50.854 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:50.854 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:50.854 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:50.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.854 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:58:50.854 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:58:50.854 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:50.854 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:50.854 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:50.855 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:58:50.856 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:50.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:50.858 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:50.860 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:50.860 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:50.860 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:58:50.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:50.860 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:50.861 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:50.861 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:58:50.861 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:50.861 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:50.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:50.861 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:58:50.861 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:58:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.847+0000 7f8dcdc00640 1 -- 172.21.15.67:0/4091818349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8da406b4d0 msgr2=0x7f8da406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.847+0000 7f8dcdc00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8da406b4d0 0x7f8da406d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0004340 tx=0x7f8dc0007d30 comp rx=0 tx=0).stop 2024-03-09T08:58:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.847+0000 7f8dcdc00640 1 -- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc80c2fa0 msgr2=0x7f8dc81518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.847+0000 7f8dcdc00640 1 --2- 172.21.15.67:0/4091818349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8dc80c2fa0 0x7f8dc81518c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c820 tx=0x7f8db800cce0 comp rx=0 tx=0).stop 2024-03-09T08:58:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.847+0000 7f8dcd200640 1 -- 172.21.15.67:0/4091818349 reap_dead start 2024-03-09T08:58:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.847+0000 7f8dcd200640 1 -- 172.21.15.67:0/4091818349 reap_dead start 2024-03-09T08:58:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.847+0000 7f8dcdc00640 1 -- 172.21.15.67:0/4091818349 shutdown_connections 2024-03-09T08:58:50.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.847+0000 7f8dcdc00640 1 -- 172.21.15.67:0/4091818349 >> 172.21.15.67:0/4091818349 conn(0x7f8dc80c7220 msgr2=0x7f8dc80c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:50.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.847+0000 7f8dcdc00640 1 -- 172.21.15.67:0/4091818349 shutdown_connections 2024-03-09T08:58:50.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:50.847+0000 7f8dcdc00640 1 -- 172.21.15.67:0/4091818349 wait complete. 2024-03-09T08:58:50.865 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:58:50.866 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:58:50.866 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:58:51.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.075+0000 7f99dae00640 1 Processor -- start 2024-03-09T08:58:51.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.076+0000 7f99dae00640 1 -- start start 2024-03-09T08:58:51.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.077+0000 7f99dae00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99d40c5e00 0x7f99d40acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.077+0000 7f99dae00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f99d40c9380 0x7f99d4156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.077+0000 7f99dae00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d40c97d0 con 0x7f99d40c61e0 2024-03-09T08:58:51.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.077+0000 7f99dae00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d40c6690 con 0x7f99d40c5e00 2024-03-09T08:58:51.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.077+0000 7f99dae00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d406d740 con 0x7f99d40c9380 2024-03-09T08:58:51.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.077+0000 7f99d9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99d40c5e00 0x7f99d40acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.078+0000 7f99d9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99d40c5e00 0x7f99d40acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36946/0 (socket says 172.21.15.67:36946) 2024-03-09T08:58:51.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.078+0000 7f99d9a00640 1 -- 172.21.15.67:0/3005996251 learned_addr learned my addr 172.21.15.67:0/3005996251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.078+0000 7f99da400640 1 --2- 172.21.15.67:0/3005996251 >> v2:172.21.15.134:3300/0 conn(0x7f99d40c9380 0x7f99d4156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.078+0000 7f99d9a00640 1 -- 172.21.15.67:0/3005996251 >> v2:172.21.15.134:3300/0 conn(0x7f99d40c9380 msgr2=0x7f99d4156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.078+0000 7f99d9a00640 1 --2- 172.21.15.67:0/3005996251 >> v2:172.21.15.134:3300/0 conn(0x7f99d40c9380 0x7f99d4156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.078+0000 7f99d9a00640 1 -- 172.21.15.67:0/3005996251 >> v1:172.21.15.134:6789/0 conn(0x7f99d40c61e0 legacy=0x7f99d406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.078+0000 7f99d9a00640 1 -- 172.21.15.67:0/3005996251 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d40ad7c0 con 0x7f99d40c5e00 2024-03-09T08:58:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.080+0000 7f99d9a00640 1 --2- 172.21.15.67:0/3005996251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d40c5e00 0x7f99d40acf00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f99c80098a0 tx=0x7f99c802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b7cc6beabd66a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.080+0000 7f99d3e00640 1 -- 172.21.15.67:0/3005996251 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8035020 con 0x7f99d40c5e00 2024-03-09T08:58:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.080+0000 7f99d3e00640 1 -- 172.21.15.67:0/3005996251 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c8031640 con 0x7f99d40c5e00 2024-03-09T08:58:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.080+0000 7f99d3e00640 1 -- 172.21.15.67:0/3005996251 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c802d040 con 0x7f99d40c5e00 2024-03-09T08:58:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.081+0000 7f99dae00640 1 -- 172.21.15.67:0/3005996251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d40c5e00 msgr2=0x7f99d40acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.081+0000 7f99dae00640 1 --2- 172.21.15.67:0/3005996251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d40c5e00 0x7f99d40acf00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f99c80098a0 tx=0x7f99c802b8f0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.081+0000 7f99dae00640 1 -- 172.21.15.67:0/3005996251 shutdown_connections 2024-03-09T08:58:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.081+0000 7f99dae00640 1 --2- 172.21.15.67:0/3005996251 >> v2:172.21.15.134:3300/0 conn(0x7f99d40c9380 0x7f99d4156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.081+0000 7f99dae00640 1 --2- 172.21.15.67:0/3005996251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d40c5e00 0x7f99d40acf00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.081+0000 7f99dae00640 1 -- 172.21.15.67:0/3005996251 >> 172.21.15.67:0/3005996251 conn(0x7f99d40cb480 msgr2=0x7f99d40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.081+0000 7f99dae00640 1 -- 172.21.15.67:0/3005996251 shutdown_connections 2024-03-09T08:58:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.081+0000 7f99dae00640 1 -- 172.21.15.67:0/3005996251 wait complete. 2024-03-09T08:58:51.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.083+0000 7f99dae00640 1 Processor -- start 2024-03-09T08:58:51.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.083+0000 7f99dae00640 1 -- start start 2024-03-09T08:58:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.084+0000 7f99dae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99d40c5e00 0x7f99d40ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.084+0000 7f99dae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d40c61e0 0x7f99d40c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.084+0000 7f99dae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c9380 0x7f99d40c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.084+0000 7f99dae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d406bb00 con 0x7f99d40c61e0 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.084+0000 7f99dae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d4153680 con 0x7f99d40c9380 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.084+0000 7f99dae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d406bde0 con 0x7f99d40c5e00 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.084+0000 7f99d9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d40c61e0 0x7f99d40c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.085+0000 7f99d9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d40c61e0 0x7f99d40c29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36962/0 (socket says 172.21.15.67:36962) 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.085+0000 7f99d9000640 1 -- 172.21.15.67:0/3670209321 learned_addr learned my addr 172.21.15.67:0/3670209321 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.085+0000 7f99d9a00640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99d40c5e00 0x7f99d40ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.085+0000 7f99da400640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c9380 0x7f99d40c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.085+0000 7f99d9000640 1 -- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99d40c5e00 msgr2=0x7f99d40ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.085+0000 7f99d9000640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99d40c5e00 0x7f99d40ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.085+0000 7f99d9000640 1 -- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c9380 msgr2=0x7f99d40c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.085+0000 7f99d9000640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c9380 0x7f99d40c2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.085+0000 7f99d9000640 1 -- 172.21.15.67:0/3670209321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d41eed70 con 0x7f99d40c61e0 2024-03-09T08:58:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.085+0000 7f99d9000640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d40c61e0 0x7f99d40c29e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f99c000c430 tx=0x7f99c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.086+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0011790 con 0x7f99d40c61e0 2024-03-09T08:58:51.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.086+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c0017070 con 0x7f99d40c61e0 2024-03-09T08:58:51.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.086+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0011bb0 con 0x7f99d40c61e0 2024-03-09T08:58:51.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.086+0000 7f99dae00640 1 -- 172.21.15.67:0/3670209321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d41ef050 con 0x7f99d40c61e0 2024-03-09T08:58:51.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.086+0000 7f99dae00640 1 -- 172.21.15.67:0/3670209321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d41ef610 con 0x7f99d40c61e0 2024-03-09T08:58:51.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.087+0000 7f99dae00640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c24c0 0x7f99d414e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.087+0000 7f99dae00640 1 -- 172.21.15.67:0/3670209321 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d404f3c0 con 0x7f99d40c24c0 2024-03-09T08:58:51.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.087+0000 7f99d9a00640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c24c0 0x7f99d414e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.088+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f99c001c460 con 0x7f99d40c61e0 2024-03-09T08:58:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.088+0000 7f99d9a00640 1 -- 172.21.15.67:0/3670209321 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99d414f260 con 0x7f99d40c24c0 2024-03-09T08:58:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.089+0000 7f99d9a00640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c24c0 0x7f99d414e4c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f99d408d2c0 tx=0x7f99c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.089+0000 7f99d1600640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99b406b4d0 0x7f99b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.089+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f99c008a4b0 con 0x7f99d40c61e0 2024-03-09T08:58:51.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.089+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8049070 con 0x7f99d40c24c0 2024-03-09T08:58:51.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.090+0000 7f99da400640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99b406b4d0 0x7f99b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.090+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f99c803a630 con 0x7f99d40c24c0 2024-03-09T08:58:51.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.090+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c24c0 msgr2=0x7f99d414e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.090+0000 7f99d1600640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c24c0 0x7f99d414e4c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f99d408d2c0 tx=0x7f99c8004730 comp rx=0 tx=0).stop 2024-03-09T08:58:51.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.090+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c24c0 msgr2=0x7f99d414e4c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:51.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.090+0000 7f99d1600640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c24c0 0x7f99d414e4c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.091+0000 7f99da400640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99b406b4d0 0x7f99b406d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f99d414f110 tx=0x7f99cc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.097+0000 7f99dae00640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c3460 0x7f99d4156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.097+0000 7f99dae00640 1 -- 172.21.15.67:0/3670209321 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d40accb0 con 0x7f99d40c3460 2024-03-09T08:58:51.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.098+0000 7f99d9a00640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c3460 0x7f99d4156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.098+0000 7f99d9a00640 1 -- 172.21.15.67:0/3670209321 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f99d4073a30 con 0x7f99d40c3460 2024-03-09T08:58:51.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.099+0000 7f99d9a00640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c3460 0x7f99d4156fd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f99c802b930 tx=0x7f99c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.099+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8049040 con 0x7f99d40c3460 2024-03-09T08:58:51.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5044 (secure 0 0 0) 0x7f99c804b620 con 0x7f99d40c3460 2024-03-09T08:58:51.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c3460 msgr2=0x7f99d4156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99d1600640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c3460 0x7f99d4156fd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f99c802b930 tx=0x7f99c80046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99d1600640 1 -- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c3460 msgr2=0x7f99d4156fd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:51.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99d1600640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99d40c3460 0x7f99d4156fd0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f99c802b930 tx=0x7f99c80046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.101 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:58:51.101 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:58:51.101 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:58:51.101 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:58:51.101 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 50, 2024-03-09T08:58:51.101 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:58:51.101 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:58:51.101 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:58:51.101 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:58:51.101 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:51.102 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:51.103 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:51.103 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.103 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:58:51.103 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:58:51.103 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:51.103 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:51.103 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:51.103 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:51.103 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:58:51.104 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:51.105 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:58:51.106 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:51.108 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:51.109 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:58:51.111 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:58:51.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:58:51.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.114 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:51.114 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:51.114 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:51.114 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.114 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.114 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:58:51.114 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:58:51.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99dae00640 1 -- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99b406b4d0 msgr2=0x7f99b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99dae00640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99b406b4d0 0x7f99b406d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f99d414f110 tx=0x7f99cc00b040 comp rx=0 tx=0).stop 2024-03-09T08:58:51.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99dae00640 1 -- 172.21.15.67:0/3670209321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d40c61e0 msgr2=0x7f99d40c29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99dae00640 1 --2- 172.21.15.67:0/3670209321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99d40c61e0 0x7f99d40c29e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f99c000c430 tx=0x7f99c000c8f0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99da400640 1 -- 172.21.15.67:0/3670209321 reap_dead start 2024-03-09T08:58:51.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99da400640 1 -- 172.21.15.67:0/3670209321 reap_dead start 2024-03-09T08:58:51.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99dae00640 1 -- 172.21.15.67:0/3670209321 shutdown_connections 2024-03-09T08:58:51.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.100+0000 7f99dae00640 1 -- 172.21.15.67:0/3670209321 >> 172.21.15.67:0/3670209321 conn(0x7f99d40cb480 msgr2=0x7f99d40c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:51.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.101+0000 7f99dae00640 1 -- 172.21.15.67:0/3670209321 shutdown_connections 2024-03-09T08:58:51.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.101+0000 7f99dae00640 1 -- 172.21.15.67:0/3670209321 wait complete. 2024-03-09T08:58:51.118 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:58:51.118 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:58:51.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:58:51.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.321+0000 7f182ee00640 1 Processor -- start 2024-03-09T08:58:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.321+0000 7f182ee00640 1 -- start start 2024-03-09T08:58:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.322+0000 7f182ee00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f182814fd90 0x7f1828156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.322+0000 7f182ee00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18280c9380 0x7f18280acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.323+0000 7f182ee00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1828156bc0 con 0x7f18280ad440 2024-03-09T08:58:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.323+0000 7f182ee00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182806ba60 con 0x7f182814fd90 2024-03-09T08:58:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.323+0000 7f182ee00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18280c9890 con 0x7f18280c9380 2024-03-09T08:58:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.323+0000 7f182da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f182814fd90 0x7f1828156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.323+0000 7f182da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f182814fd90 0x7f1828156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36970/0 (socket says 172.21.15.67:36970) 2024-03-09T08:58:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.323+0000 7f182e400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f18280ad440 0x7f182806c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46586/0 (socket says 172.21.15.67:46586) 2024-03-09T08:58:51.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.323+0000 7f182da00640 1 -- 172.21.15.67:0/2484831883 learned_addr learned my addr 172.21.15.67:0/2484831883 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:51.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.323+0000 7f182d000640 1 --2- 172.21.15.67:0/2484831883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18280c9380 0x7f18280acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.323+0000 7f182da00640 1 -- 172.21.15.67:0/2484831883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18280c9380 msgr2=0x7f18280acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:51.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.323+0000 7f182da00640 1 --2- 172.21.15.67:0/2484831883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18280c9380 0x7f18280acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.324+0000 7f182da00640 1 -- 172.21.15.67:0/2484831883 >> v1:172.21.15.67:6789/0 conn(0x7f18280ad440 legacy=0x7f182806c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.324+0000 7f182da00640 1 -- 172.21.15.67:0/2484831883 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18280c5f80 con 0x7f182814fd90 2024-03-09T08:58:51.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.324+0000 7f1827e00640 1 -- 172.21.15.67:0/2484831883 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162212215 0 0) 0x7f1820001490 con 0x7f18280ad440 2024-03-09T08:58:51.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.325+0000 7f182da00640 1 --2- 172.21.15.67:0/2484831883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f182814fd90 0x7f1828156680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad362c984a7d6acc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.325+0000 7f1827e00640 1 -- 172.21.15.67:0/2484831883 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c0317c0 con 0x7f182814fd90 2024-03-09T08:58:51.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.325+0000 7f1827e00640 1 -- 172.21.15.67:0/2484831883 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c037070 con 0x7f182814fd90 2024-03-09T08:58:51.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.325+0000 7f1827e00640 1 -- 172.21.15.67:0/2484831883 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c031b80 con 0x7f182814fd90 2024-03-09T08:58:51.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.326+0000 7f182ee00640 1 -- 172.21.15.67:0/2484831883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f182814fd90 msgr2=0x7f1828156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.326+0000 7f182ee00640 1 --2- 172.21.15.67:0/2484831883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f182814fd90 0x7f1828156680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c02b860 comp rx=0 tx=0).stop 2024-03-09T08:58:51.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.326+0000 7f182ee00640 1 -- 172.21.15.67:0/2484831883 shutdown_connections 2024-03-09T08:58:51.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.326+0000 7f182ee00640 1 --2- 172.21.15.67:0/2484831883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18280c9380 0x7f18280acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.326+0000 7f182ee00640 1 --2- 172.21.15.67:0/2484831883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f182814fd90 0x7f1828156680 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.326+0000 7f182ee00640 1 -- 172.21.15.67:0/2484831883 >> 172.21.15.67:0/2484831883 conn(0x7f18280cb480 msgr2=0x7f18280cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.326+0000 7f182ee00640 1 -- 172.21.15.67:0/2484831883 shutdown_connections 2024-03-09T08:58:51.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.326+0000 7f182ee00640 1 -- 172.21.15.67:0/2484831883 wait complete. 2024-03-09T08:58:51.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.328+0000 7f182ee00640 1 Processor -- start 2024-03-09T08:58:51.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.328+0000 7f182ee00640 1 -- start start 2024-03-09T08:58:51.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.329+0000 7f182ee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18280ad440 0x7f1828155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.329+0000 7f182ee00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18280c9380 0x7f18281e2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.329+0000 7f182ee00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f182814fd90 0x7f1828146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.329+0000 7f182ee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182806bcb0 con 0x7f18280ad440 2024-03-09T08:58:51.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.329+0000 7f182ee00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18280c6520 con 0x7f182814fd90 2024-03-09T08:58:51.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.329+0000 7f182ee00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182806c8e0 con 0x7f18280c9380 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.329+0000 7f182da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18280ad440 0x7f1828155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.329+0000 7f182da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18280ad440 0x7f1828155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36980/0 (socket says 172.21.15.67:36980) 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.329+0000 7f182da00640 1 -- 172.21.15.67:0/618239383 learned_addr learned my addr 172.21.15.67:0/618239383 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.330+0000 7f182d000640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18280c9380 0x7f18281e2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.330+0000 7f182da00640 1 -- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18280c9380 msgr2=0x7f18281e2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.330+0000 7f182e400640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f182814fd90 0x7f1828146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.330+0000 7f182da00640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18280c9380 0x7f18281e2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.330+0000 7f182da00640 1 -- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f182814fd90 msgr2=0x7f1828146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.330+0000 7f182da00640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f182814fd90 0x7f1828146540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.330+0000 7f182da00640 1 -- 172.21.15.67:0/618239383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18281efab0 con 0x7f18280ad440 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.330+0000 7f182e400640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f182814fd90 0x7f1828146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:58:51.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.330+0000 7f182da00640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18280ad440 0x7f1828155750 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.331+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c004230 con 0x7f18280ad440 2024-03-09T08:58:51.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.331+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c0043f0 con 0x7f18280ad440 2024-03-09T08:58:51.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.331+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c003850 con 0x7f18280ad440 2024-03-09T08:58:51.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.331+0000 7f182ee00640 1 -- 172.21.15.67:0/618239383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18281efd50 con 0x7f18280ad440 2024-03-09T08:58:51.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.331+0000 7f182ee00640 1 -- 172.21.15.67:0/618239383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1828153fa0 con 0x7f18280ad440 2024-03-09T08:58:51.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.332+0000 7f182ee00640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f182814b520 0x7f1828146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.332+0000 7f182ee00640 1 -- 172.21.15.67:0/618239383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182804f3c0 con 0x7f182814b520 2024-03-09T08:58:51.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.332+0000 7f182d000640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f182814b520 0x7f1828146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.333+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f181c04c060 con 0x7f18280ad440 2024-03-09T08:58:51.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.333+0000 7f182d000640 1 -- 172.21.15.67:0/618239383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18281541e0 con 0x7f182814b520 2024-03-09T08:58:51.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.333+0000 7f1825600640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f180806b500 0x7f180806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.334+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f181c0abee0 con 0x7f18280ad440 2024-03-09T08:58:51.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.334+0000 7f182d000640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f182814b520 0x7f1828146a80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1814006fd0 tx=0x7f181400eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.334+0000 7f182e400640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f180806b500 0x7f180806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.334+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814017070 con 0x7f182814b520 2024-03-09T08:58:51.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.335+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f181400c710 con 0x7f182814b520 2024-03-09T08:58:51.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.335+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f182814b520 msgr2=0x7f1828146a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.335+0000 7f1825600640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f182814b520 0x7f1828146a80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1814006fd0 tx=0x7f181400eba0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.335+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f182814b520 msgr2=0x7f1828146a80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:51.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.335+0000 7f1825600640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f182814b520 0x7f1828146a80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.335+0000 7f182e400640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f180806b500 0x7f180806d9c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1820001730 tx=0x7f1820009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.342+0000 7f182ee00640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1828147580 0x7f18281479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.342+0000 7f182ee00640 1 -- 172.21.15.67:0/618239383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182814f5b0 con 0x7f1828147580 2024-03-09T08:58:51.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.343+0000 7f182d000640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1828147580 0x7f18281479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.344+0000 7f182d000640 1 -- 172.21.15.67:0/618239383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f182806d2d0 con 0x7f1828147580 2024-03-09T08:58:51.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.344+0000 7f182d000640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1828147580 0x7f18281479a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f182814b940 tx=0x7f1814004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.345+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181400f040 con 0x7f1828147580 2024-03-09T08:58:51.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.345+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7f1814013610 con 0x7f1828147580 2024-03-09T08:58:51.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.345+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1828147580 msgr2=0x7f18281479a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.345+0000 7f1825600640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1828147580 0x7f18281479a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f182814b940 tx=0x7f1814004730 comp rx=0 tx=0).stop 2024-03-09T08:58:51.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.345+0000 7f1825600640 1 -- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1828147580 msgr2=0x7f18281479a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:58:51.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.345+0000 7f1825600640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1828147580 0x7f18281479a0 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f182814b940 tx=0x7f1814004730 comp rx=0 tx=0).stop 2024-03-09T08:58:51.346 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:58:51.346 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:58:51.346 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:58:51.346 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:58:51.346 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 50, 2024-03-09T08:58:51.346 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:58:51.346 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:51.347 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:51.348 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:51.349 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:58:51.350 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:58:51.351 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.352 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:58:51.353 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:51.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.355 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:58:51.355 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:58:51.355 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:58:51.355 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:58:51.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.355 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:58:51.355 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:58:51.355 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.357 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:58:51.357 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:58:51.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.357 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:51.357 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:51.357 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:51.357 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.357 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.357 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:58:51.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.357 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:58:51.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:58:51.359 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:58:51.360 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:58:51.360 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:58:51.360 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:58:51.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:58:51.360 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:58:51.360 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:58:51.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.346+0000 7f182ee00640 1 -- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f180806b500 msgr2=0x7f180806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.346+0000 7f182ee00640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f180806b500 0x7f180806d9c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1820001730 tx=0x7f1820009040 comp rx=0 tx=0).stop 2024-03-09T08:58:51.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.346+0000 7f182ee00640 1 -- 172.21.15.67:0/618239383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18280ad440 msgr2=0x7f1828155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.346+0000 7f182ee00640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18280ad440 0x7f1828155750 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.346+0000 7f182e400640 1 -- 172.21.15.67:0/618239383 reap_dead start 2024-03-09T08:58:51.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.346+0000 7f182ee00640 1 -- 172.21.15.67:0/618239383 shutdown_connections 2024-03-09T08:58:51.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.346+0000 7f182ee00640 1 --2- 172.21.15.67:0/618239383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18280ad440 0x7f1828155750 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.346+0000 7f182ee00640 1 -- 172.21.15.67:0/618239383 >> 172.21.15.67:0/618239383 conn(0x7f18280cb480 msgr2=0x7f18280aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:51.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.346+0000 7f182ee00640 1 -- 172.21.15.67:0/618239383 shutdown_connections 2024-03-09T08:58:51.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.346+0000 7f182ee00640 1 -- 172.21.15.67:0/618239383 wait complete. 2024-03-09T08:58:51.362 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:58:51.363 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T08:58:51.363 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T08:58:51.363 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T08:58:51.363 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T08:58:51.363 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T08:58:51.363 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T08:58:51.392 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T08:58:51.392 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T08:58:51.393 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:58:51.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.595+0000 7fc985600640 1 Processor -- start 2024-03-09T08:58:51.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.595+0000 7fc985600640 1 -- start start 2024-03-09T08:58:51.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.596+0000 7fc985600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800ac730 0x7fc980071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc985600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc980157d60 0x7fc9800acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc985600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc980153d40 con 0x7fc9800c22a0 2024-03-09T08:58:51.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc985600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc980070ee0 con 0x7fc980157d60 2024-03-09T08:58:51.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc985600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc980154d00 con 0x7fc9800ac730 2024-03-09T08:58:51.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc97ea00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800ac730 msgr2=0x7fc980071190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:58:51.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc97ea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800ac730 0x7fc980071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:51.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc97f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc980157d60 0x7fc9800acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc97f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc980157d60 0x7fc9800acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37052/0 (socket says 172.21.15.67:37052) 2024-03-09T08:58:51.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc97f400640 1 -- 172.21.15.67:0/1885914586 learned_addr learned my addr 172.21.15.67:0/1885914586 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:51.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc97f400640 1 -- 172.21.15.67:0/1885914586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800ac730 msgr2=0x7fc980071190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:58:51.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc97f400640 1 --2- 172.21.15.67:0/1885914586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800ac730 0x7fc980071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc97f400640 1 -- 172.21.15.67:0/1885914586 >> v1:172.21.15.134:6789/0 conn(0x7fc9800c22a0 legacy=0x7fc980157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:51.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.597+0000 7fc97f400640 1 -- 172.21.15.67:0/1885914586 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800c2790 con 0x7fc980157d60 2024-03-09T08:58:51.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.599+0000 7fc97f400640 1 --2- 172.21.15.67:0/1885914586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc980157d60 0x7fc9800acb10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc978009e50 tx=0x7fc97802bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f948092d6b88c17d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.599+0000 7fc97e000640 1 -- 172.21.15.67:0/1885914586 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97802caa0 con 0x7fc980157d60 2024-03-09T08:58:51.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.599+0000 7fc97e000640 1 -- 172.21.15.67:0/1885914586 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97802cc40 con 0x7fc980157d60 2024-03-09T08:58:51.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.599+0000 7fc97e000640 1 -- 172.21.15.67:0/1885914586 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9780343f0 con 0x7fc980157d60 2024-03-09T08:58:51.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.600+0000 7fc985600640 1 -- 172.21.15.67:0/1885914586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc980157d60 msgr2=0x7fc9800acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.600+0000 7fc985600640 1 --2- 172.21.15.67:0/1885914586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc980157d60 0x7fc9800acb10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc978009e50 tx=0x7fc97802bf00 comp rx=0 tx=0).stop 2024-03-09T08:58:51.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.600+0000 7fc985600640 1 -- 172.21.15.67:0/1885914586 shutdown_connections 2024-03-09T08:58:51.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.600+0000 7fc985600640 1 --2- 172.21.15.67:0/1885914586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc980157d60 0x7fc9800acb10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.600+0000 7fc985600640 1 --2- 172.21.15.67:0/1885914586 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800ac730 0x7fc980071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.600+0000 7fc985600640 1 -- 172.21.15.67:0/1885914586 >> 172.21.15.67:0/1885914586 conn(0x7fc9800c7370 msgr2=0x7fc9800c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:51.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.600+0000 7fc985600640 1 -- 172.21.15.67:0/1885914586 shutdown_connections 2024-03-09T08:58:51.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.600+0000 7fc985600640 1 -- 172.21.15.67:0/1885914586 wait complete. 2024-03-09T08:58:51.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.602+0000 7fc985600640 1 Processor -- start 2024-03-09T08:58:51.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.602+0000 7fc985600640 1 -- start start 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc985600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9800ac730 0x7fc9800c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc985600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800c22a0 0x7fc9801dfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc985600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc980157d60 0x7fc980150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc985600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc980151610 con 0x7fc980157d60 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc985600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc980070c00 con 0x7fc9800ac730 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc985600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc980151790 con 0x7fc9800c22a0 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc977e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800c22a0 msgr2=0x7fc9801dfc00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc977e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800c22a0 0x7fc9801dfc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc97f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc980157d60 0x7fc980150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc97f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc980157d60 0x7fc980150400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37064/0 (socket says 172.21.15.67:37064) 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc97f400640 1 -- 172.21.15.67:0/3503055608 learned_addr learned my addr 172.21.15.67:0/3503055608 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.603+0000 7fc97ea00640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9800ac730 0x7fc9800c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.604+0000 7fc97f400640 1 -- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800c22a0 msgr2=0x7fc9801dfc00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:58:51.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.604+0000 7fc97f400640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800c22a0 0x7fc9801dfc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.604+0000 7fc97f400640 1 -- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9800ac730 msgr2=0x7fc9800c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.604+0000 7fc97f400640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9800ac730 0x7fc9800c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.604+0000 7fc97f400640 1 -- 172.21.15.67:0/3503055608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9801e0980 con 0x7fc980157d60 2024-03-09T08:58:51.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.604+0000 7fc97ea00640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9800ac730 0x7fc9800c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:58:51.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.604+0000 7fc97f400640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc980157d60 0x7fc980150400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc978036040 tx=0x7fc978004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.604+0000 7fc976a00640 1 -- 172.21.15.67:0/3503055608 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978043020 con 0x7fc980157d60 2024-03-09T08:58:51.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.604+0000 7fc976a00640 1 -- 172.21.15.67:0/3503055608 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc978007e10 con 0x7fc980157d60 2024-03-09T08:58:51.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.604+0000 7fc985600640 1 -- 172.21.15.67:0/3503055608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9801ec2e0 con 0x7fc980157d60 2024-03-09T08:58:51.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.604+0000 7fc976a00640 1 -- 172.21.15.67:0/3503055608 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978009510 con 0x7fc980157d60 2024-03-09T08:58:51.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.605+0000 7fc985600640 1 -- 172.21.15.67:0/3503055608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9800c1b10 con 0x7fc980157d60 2024-03-09T08:58:51.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.606+0000 7fc985600640 1 -- 172.21.15.67:0/3503055608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc980075a00 con 0x7fc980157d60 2024-03-09T08:58:51.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.607+0000 7fc976a00640 1 -- 172.21.15.67:0/3503055608 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc978046020 con 0x7fc980157d60 2024-03-09T08:58:51.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.608+0000 7fc976a00640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc95c06b480 0x7fc95c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:51.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.608+0000 7fc976a00640 1 -- 172.21.15.67:0/3503055608 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc9780aca20 con 0x7fc980157d60 2024-03-09T08:58:51.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.608+0000 7fc97ea00640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc95c06b480 0x7fc95c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:51.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.609+0000 7fc97ea00640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc95c06b480 0x7fc95c06d940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc970003930 tx=0x7fc970008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:51.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.617+0000 7fc976a00640 1 -- 172.21.15.67:0/3503055608 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc97804d070 con 0x7fc980157d60 2024-03-09T08:58:51.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.895+0000 7fc985600640 1 -- 172.21.15.67:0/3503055608 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc980151de0 con 0x7fc980157d60 2024-03-09T08:58:51.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.896+0000 7fc976a00640 1 -- 172.21.15.67:0/3503055608 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fc97807b4a0 con 0x7fc980157d60 2024-03-09T08:58:51.896 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:58:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 -- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc95c06b480 msgr2=0x7fc95c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc95c06b480 0x7fc95c06d940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc970003930 tx=0x7fc970008040 comp rx=0 tx=0).stop 2024-03-09T08:58:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 -- 172.21.15.67:0/3503055608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc980157d60 msgr2=0x7fc980150400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc980157d60 0x7fc980150400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc978036040 tx=0x7fc978004a40 comp rx=0 tx=0).stop 2024-03-09T08:58:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 -- 172.21.15.67:0/3503055608 shutdown_connections 2024-03-09T08:58:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc95c06b480 0x7fc95c06d940 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc980157d60 0x7fc980150400 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9800c22a0 0x7fc9801dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 --2- 172.21.15.67:0/3503055608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9800ac730 0x7fc9800c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 -- 172.21.15.67:0/3503055608 >> 172.21.15.67:0/3503055608 conn(0x7fc9800c7370 msgr2=0x7fc9801531a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 -- 172.21.15.67:0/3503055608 shutdown_connections 2024-03-09T08:58:51.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:51.899+0000 7fc985600640 1 -- 172.21.15.67:0/3503055608 wait complete. 2024-03-09T08:58:54.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:58:55.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.140+0000 7f9c4b200640 1 Processor -- start 2024-03-09T08:58:55.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.140+0000 7f9c4b200640 1 -- start start 2024-03-09T08:58:55.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.141+0000 7f9c4b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9c4c0c1f70 0x7f9c4c150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:55.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.141+0000 7f9c4b200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9c4c154080 0x7f9c4c151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:55.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.141+0000 7f9c4b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4c0c81e0 con 0x7f9c4c0710a0 2024-03-09T08:58:55.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.141+0000 7f9c4b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4c0c32b0 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.141+0000 7f9c4b200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4c0c2fd0 con 0x7f9c4c154080 2024-03-09T08:58:55.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.142+0000 7f9c49e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9c4c0c1f70 0x7f9c4c150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:55.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.142+0000 7f9c41400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9c4c0710a0 0x7f9c4c153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46602/0 (socket says 172.21.15.67:46602) 2024-03-09T08:58:55.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.142+0000 7f9c41400640 1 -- 172.21.15.67:0/1244305017 learned_addr learned my addr 172.21.15.67:0/1244305017 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.142+0000 7f9c4a800640 1 --2- 172.21.15.67:0/1244305017 >> v2:172.21.15.134:3300/0 conn(0x7f9c4c154080 0x7f9c4c151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.142+0000 7f9c49e00640 1 -- 172.21.15.67:0/1244305017 >> v2:172.21.15.134:3300/0 conn(0x7f9c4c154080 msgr2=0x7f9c4c151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.142+0000 7f9c49e00640 1 --2- 172.21.15.67:0/1244305017 >> v2:172.21.15.134:3300/0 conn(0x7f9c4c154080 0x7f9c4c151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.142+0000 7f9c49e00640 1 -- 172.21.15.67:0/1244305017 >> v1:172.21.15.67:6789/0 conn(0x7f9c4c0710a0 legacy=0x7f9c4c153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:55.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.142+0000 7f9c49e00640 1 -- 172.21.15.67:0/1244305017 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0c3700 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.143+0000 7f9c49e00640 1 --2- 172.21.15.67:0/1244305017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c0c1f70 0x7f9c4c150c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c073ef0 tx=0x7f9c3c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a84407d3758238 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.144+0000 7f9c49400640 1 -- 172.21.15.67:0/1244305017 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c0315a0 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.144+0000 7f9c49400640 1 -- 172.21.15.67:0/1244305017 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c037070 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.144+0000 7f9c49400640 1 -- 172.21.15.67:0/1244305017 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c031960 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.145+0000 7f9c4b200640 1 -- 172.21.15.67:0/1244305017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c0c1f70 msgr2=0x7f9c4c150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:55.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.145+0000 7f9c4b200640 1 --2- 172.21.15.67:0/1244305017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c0c1f70 0x7f9c4c150c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c073ef0 tx=0x7f9c3c02b840 comp rx=0 tx=0).stop 2024-03-09T08:58:55.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.145+0000 7f9c4b200640 1 -- 172.21.15.67:0/1244305017 shutdown_connections 2024-03-09T08:58:55.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.145+0000 7f9c4b200640 1 --2- 172.21.15.67:0/1244305017 >> v2:172.21.15.134:3300/0 conn(0x7f9c4c154080 0x7f9c4c151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:55.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.145+0000 7f9c4b200640 1 --2- 172.21.15.67:0/1244305017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c0c1f70 0x7f9c4c150c50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:55.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.145+0000 7f9c4b200640 1 -- 172.21.15.67:0/1244305017 >> 172.21.15.67:0/1244305017 conn(0x7f9c4c0c7300 msgr2=0x7f9c4c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:55.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.145+0000 7f9c4b200640 1 -- 172.21.15.67:0/1244305017 shutdown_connections 2024-03-09T08:58:55.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.145+0000 7f9c4b200640 1 -- 172.21.15.67:0/1244305017 wait complete. 2024-03-09T08:58:55.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.147+0000 7f9c4b200640 1 Processor -- start 2024-03-09T08:58:55.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.147+0000 7f9c4b200640 1 -- start start 2024-03-09T08:58:55.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c4b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c4c0710a0 0x7f9c4c0ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c4b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c0c1f70 0x7f9c4c147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c4b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c4c154080 0x7f9c4c148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c4b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c070d50 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c4b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c070830 con 0x7f9c4c0710a0 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c4b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c070ed0 con 0x7f9c4c154080 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c4a800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c4c154080 msgr2=0x7f9c4c148520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c4a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c4c154080 0x7f9c4c148520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c41400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c0c1f70 0x7f9c4c147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c41400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c0c1f70 0x7f9c4c147fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37068/0 (socket says 172.21.15.67:37068) 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c41400640 1 -- 172.21.15.67:0/3748538722 learned_addr learned my addr 172.21.15.67:0/3748538722 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.148+0000 7f9c49e00640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c4c0710a0 0x7f9c4c0ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c41400640 1 -- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c4c154080 msgr2=0x7f9c4c148520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:58:55.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c41400640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c4c154080 0x7f9c4c148520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:55.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c41400640 1 -- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c4c0710a0 msgr2=0x7f9c4c0ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:55.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c41400640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c4c0710a0 0x7f9c4c0ab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:55.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c41400640 1 -- 172.21.15.67:0/3748538722 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0ac7f0 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c49e00640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c4c0710a0 0x7f9c4c0ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:58:55.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c41400640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c0c1f70 0x7f9c4c147fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400c400 tx=0x7f9c3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:55.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c42a00640 1 -- 172.21.15.67:0/3748538722 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34017070 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c42a00640 1 -- 172.21.15.67:0/3748538722 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c34005ce0 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c42a00640 1 -- 172.21.15.67:0/3748538722 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c340072f0 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c4b200640 1 -- 172.21.15.67:0/3748538722 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4c1eedd0 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.149+0000 7f9c4b200640 1 -- 172.21.15.67:0/3748538722 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4c1ef390 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.150+0000 7f9c4b200640 1 -- 172.21.15.67:0/3748538722 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4c04f3c0 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.151+0000 7f9c42a00640 1 -- 172.21.15.67:0/3748538722 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9c3401a070 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.152+0000 7f9c42a00640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c2806b480 0x7f9c2806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:55.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.152+0000 7f9c42a00640 1 -- 172.21.15.67:0/3748538722 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9c34089950 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.153+0000 7f9c49e00640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c2806b480 0x7f9c2806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:55.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.154+0000 7f9c49e00640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c2806b480 0x7f9c2806d940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c036000 tx=0x7f9c3c03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:55.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.157+0000 7f9c42a00640 1 -- 172.21.15.67:0/3748538722 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9c34058470 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.430+0000 7f9c4b200640 1 -- 172.21.15.67:0/3748538722 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9c4c14cea0 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.430+0000 7f9c42a00640 1 -- 172.21.15.67:0/3748538722 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f9c34057d70 con 0x7f9c4c0c1f70 2024-03-09T08:58:55.431 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:58:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 -- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c2806b480 msgr2=0x7f9c2806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c2806b480 0x7f9c2806d940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c036000 tx=0x7f9c3c03d040 comp rx=0 tx=0).stop 2024-03-09T08:58:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 -- 172.21.15.67:0/3748538722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c0c1f70 msgr2=0x7f9c4c147fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c0c1f70 0x7f9c4c147fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400c400 tx=0x7f9c3400c8c0 comp rx=0 tx=0).stop 2024-03-09T08:58:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 -- 172.21.15.67:0/3748538722 shutdown_connections 2024-03-09T08:58:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c2806b480 0x7f9c2806d940 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c4c154080 0x7f9c4c148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9c4c0c1f70 0x7f9c4c147fe0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 --2- 172.21.15.67:0/3748538722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c4c0710a0 0x7f9c4c0ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 -- 172.21.15.67:0/3748538722 >> 172.21.15.67:0/3748538722 conn(0x7f9c4c0c7300 msgr2=0x7f9c4c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:55.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 -- 172.21.15.67:0/3748538722 shutdown_connections 2024-03-09T08:58:55.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:55.433+0000 7f9c4b200640 1 -- 172.21.15.67:0/3748538722 wait complete. 2024-03-09T08:58:58.451 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:58:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.683+0000 7f3073e00640 1 Processor -- start 2024-03-09T08:58:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.683+0000 7f3073e00640 1 -- start start 2024-03-09T08:58:58.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.684+0000 7f3073e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3074072770 0x7f3074072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:58.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.684+0000 7f3073e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f30740bebc0 0x7f3074156650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:58.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.684+0000 7f3073e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307406cd30 con 0x7f307406b690 2024-03-09T08:58:58.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.684+0000 7f3073e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307406b200 con 0x7f3074072770 2024-03-09T08:58:58.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.684+0000 7f3073e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30740c5eb0 con 0x7f30740bebc0 2024-03-09T08:58:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.684+0000 7f3072a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3074072770 0x7f3074072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.684+0000 7f3072a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3074072770 0x7f3074072b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37080/0 (socket says 172.21.15.67:37080) 2024-03-09T08:58:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.684+0000 7f3072a00640 1 -- 172.21.15.67:0/2378798398 learned_addr learned my addr 172.21.15.67:0/2378798398 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.685+0000 7f3072000640 1 --2- 172.21.15.67:0/2378798398 >> v2:172.21.15.134:3300/0 conn(0x7f30740bebc0 0x7f3074156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.685+0000 7f3072a00640 1 -- 172.21.15.67:0/2378798398 >> v2:172.21.15.134:3300/0 conn(0x7f30740bebc0 msgr2=0x7f3074156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.685+0000 7f3072a00640 1 --2- 172.21.15.67:0/2378798398 >> v2:172.21.15.134:3300/0 conn(0x7f30740bebc0 0x7f3074156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.685+0000 7f3072a00640 1 -- 172.21.15.67:0/2378798398 >> v1:172.21.15.134:6789/0 conn(0x7f307406b690 legacy=0x7f307406bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:58:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.685+0000 7f3072a00640 1 -- 172.21.15.67:0/2378798398 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740c6150 con 0x7f3074072770 2024-03-09T08:58:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.686+0000 7f3072a00640 1 --2- 172.21.15.67:0/2378798398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3074072770 0x7f3074072b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30740cc430 tx=0x7f306c02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d296e95c990d32bd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.686+0000 7f3071600640 1 -- 172.21.15.67:0/2378798398 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c02c960 con 0x7f3074072770 2024-03-09T08:58:58.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.686+0000 7f3071600640 1 -- 172.21.15.67:0/2378798398 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c02cb00 con 0x7f3074072770 2024-03-09T08:58:58.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.686+0000 7f3071600640 1 -- 172.21.15.67:0/2378798398 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0343f0 con 0x7f3074072770 2024-03-09T08:58:58.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.687+0000 7f3073e00640 1 -- 172.21.15.67:0/2378798398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3074072770 msgr2=0x7f3074072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:58.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.687+0000 7f3073e00640 1 --2- 172.21.15.67:0/2378798398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3074072770 0x7f3074072b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f30740cc430 tx=0x7f306c02bdc0 comp rx=0 tx=0).stop 2024-03-09T08:58:58.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.687+0000 7f3073e00640 1 -- 172.21.15.67:0/2378798398 shutdown_connections 2024-03-09T08:58:58.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.687+0000 7f3073e00640 1 --2- 172.21.15.67:0/2378798398 >> v2:172.21.15.134:3300/0 conn(0x7f30740bebc0 0x7f3074156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:58.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.687+0000 7f3073e00640 1 --2- 172.21.15.67:0/2378798398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3074072770 0x7f3074072b50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:58.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.687+0000 7f3073e00640 1 -- 172.21.15.67:0/2378798398 >> 172.21.15.67:0/2378798398 conn(0x7f30740c7fb0 msgr2=0x7f30740c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:58.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.687+0000 7f3073e00640 1 -- 172.21.15.67:0/2378798398 shutdown_connections 2024-03-09T08:58:58.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.687+0000 7f3073e00640 1 -- 172.21.15.67:0/2378798398 wait complete. 2024-03-09T08:58:58.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.689+0000 7f3073e00640 1 Processor -- start 2024-03-09T08:58:58.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.689+0000 7f3073e00640 1 -- start start 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3073e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307406b690 0x7f30741ec1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3073e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3074072770 0x7f3074158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3073e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30740bebc0 0x7f30741576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3073e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740c9430 con 0x7f30740bebc0 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3073e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740c6710 con 0x7f307406b690 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3073e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30740c2320 con 0x7f3074072770 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3072000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3074072770 msgr2=0x7f3074158d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3072000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3074072770 0x7f3074158d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3073400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30740bebc0 0x7f30741576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3073400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30740bebc0 0x7f30741576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37090/0 (socket says 172.21.15.67:37090) 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3073400640 1 -- 172.21.15.67:0/866030171 learned_addr learned my addr 172.21.15.67:0/866030171 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:58:58.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.690+0000 7f3072a00640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307406b690 0x7f30741ec1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.691+0000 7f3073400640 1 -- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3074072770 msgr2=0x7f3074158d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.691+0000 7f3073400640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3074072770 0x7f3074158d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.691+0000 7f3073400640 1 -- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307406b690 msgr2=0x7f30741ec1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.691+0000 7f3073400640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307406b690 0x7f30741ec1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.691+0000 7f3073400640 1 -- 172.21.15.67:0/866030171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740c4d50 con 0x7f30740bebc0 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.691+0000 7f3072a00640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307406b690 0x7f30741ec1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.691+0000 7f3073400640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30740bebc0 0x7f30741576c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f306400b440 tx=0x7f306400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.691+0000 7f306aa00640 1 -- 172.21.15.67:0/866030171 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064015020 con 0x7f30740bebc0 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.691+0000 7f306aa00640 1 -- 172.21.15.67:0/866030171 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3064011830 con 0x7f30740bebc0 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.691+0000 7f306aa00640 1 -- 172.21.15.67:0/866030171 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064011c20 con 0x7f30740bebc0 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.691+0000 7f3073e00640 1 -- 172.21.15.67:0/866030171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30741eb710 con 0x7f30740bebc0 2024-03-09T08:58:58.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.692+0000 7f3073e00640 1 -- 172.21.15.67:0/866030171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30741eb9d0 con 0x7f30740bebc0 2024-03-09T08:58:58.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.693+0000 7f3073e00640 1 -- 172.21.15.67:0/866030171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307404f3c0 con 0x7f30740bebc0 2024-03-09T08:58:58.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.694+0000 7f306aa00640 1 -- 172.21.15.67:0/866030171 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3064018020 con 0x7f30740bebc0 2024-03-09T08:58:58.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.695+0000 7f306aa00640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f305406b480 0x7f305406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:58:58.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.695+0000 7f306aa00640 1 -- 172.21.15.67:0/866030171 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f306408a180 con 0x7f30740bebc0 2024-03-09T08:58:58.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.696+0000 7f3072a00640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f305406b480 0x7f305406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:58:58.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.697+0000 7f3072a00640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f305406b480 0x7f305406d940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f306c035000 tx=0x7f306c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:58:58.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:58.700+0000 7f306aa00640 1 -- 172.21.15.67:0/866030171 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f306401f030 con 0x7f30740bebc0 2024-03-09T08:58:59.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.061+0000 7f3073e00640 1 -- 172.21.15.67:0/866030171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f30740b0440 con 0x7f30740bebc0 2024-03-09T08:58:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.061+0000 7f306aa00640 1 -- 172.21.15.67:0/866030171 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f3064058c00 con 0x7f30740bebc0 2024-03-09T08:58:59.062 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:58:59.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.065+0000 7f3073e00640 1 -- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f305406b480 msgr2=0x7f305406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:59.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.065+0000 7f3073e00640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f305406b480 0x7f305406d940 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f306c035000 tx=0x7f306c030000 comp rx=0 tx=0).stop 2024-03-09T08:58:59.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.065+0000 7f3073e00640 1 -- 172.21.15.67:0/866030171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30740bebc0 msgr2=0x7f30741576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:58:59.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.065+0000 7f3073e00640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30740bebc0 0x7f30741576c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f306400b440 tx=0x7f306400b900 comp rx=0 tx=0).stop 2024-03-09T08:58:59.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.065+0000 7f3073e00640 1 -- 172.21.15.67:0/866030171 shutdown_connections 2024-03-09T08:58:59.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.065+0000 7f3073e00640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f305406b480 0x7f305406d940 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:59.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.065+0000 7f3073e00640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f30740bebc0 0x7f30741576c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:59.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.065+0000 7f3073e00640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3074072770 0x7f3074158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:59.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.065+0000 7f3073e00640 1 --2- 172.21.15.67:0/866030171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f307406b690 0x7f30741ec1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:58:59.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.065+0000 7f3073e00640 1 -- 172.21.15.67:0/866030171 >> 172.21.15.67:0/866030171 conn(0x7f30740c7fb0 msgr2=0x7f30740ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:58:59.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.066+0000 7f3073e00640 1 -- 172.21.15.67:0/866030171 shutdown_connections 2024-03-09T08:58:59.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:58:59.066+0000 7f3073e00640 1 -- 172.21.15.67:0/866030171 wait complete. 2024-03-09T08:59:02.089 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:59:02.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.359+0000 7fa5d4a00640 1 Processor -- start 2024-03-09T08:59:02.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.359+0000 7fa5d4a00640 1 -- start start 2024-03-09T08:59:02.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5d4a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 0x7fa5d01527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:02.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5d4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:02.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5d4a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d006d920 con 0x7fa5d006d420 2024-03-09T08:59:02.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5d4a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d0155ba0 con 0x7fa5d014c9f0 2024-03-09T08:59:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5d4a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d006dc00 con 0x7fa5d00c3080 2024-03-09T08:59:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5ce800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 msgr2=0x7fa5d01527d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 0x7fa5d01527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5cf200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5cf200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47540/0 (socket says 172.21.15.67:47540) 2024-03-09T08:59:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5cf200640 1 -- 172.21.15.67:0/4239162864 learned_addr learned my addr 172.21.15.67:0/4239162864 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5cde00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa5d006d420 0x7fa5d014c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51958/0 (socket says 172.21.15.67:51958) 2024-03-09T08:59:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5cf200640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 msgr2=0x7fa5d01540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:59:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5cf200640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 msgr2=0x7fa5d01540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:02.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.361+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.562+0000 7fa5ce800640 1 -- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 msgr2=0x7fa5d01527d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.562+0000 7fa5ce800640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 0x7fa5d01527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:59:02.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.562+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:02.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.563+0000 7fa5cf200640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 msgr2=0x7fa5d01540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:59:02.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.563+0000 7fa5cf200640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 msgr2=0x7fa5d01540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:02.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.563+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:02.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.563+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:59:02.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.963+0000 7fa5ce800640 1 -- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 msgr2=0x7fa5d01527d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:02.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.963+0000 7fa5ce800640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 0x7fa5d01527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:59:02.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.964+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:02.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.964+0000 7fa5cf200640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 msgr2=0x7fa5d01540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:59:02.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.964+0000 7fa5cf200640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 msgr2=0x7fa5d01540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:02.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.964+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:02.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:02.965+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:59:03.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:03.764+0000 7fa5ce800640 1 -- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 msgr2=0x7fa5d01527d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:03.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:03.764+0000 7fa5ce800640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 0x7fa5d01527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:59:03.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:03.766+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:03.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:03.766+0000 7fa5cf200640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 msgr2=0x7fa5d01540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T08:59:03.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:03.766+0000 7fa5cf200640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 msgr2=0x7fa5d01540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:03.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:03.766+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:03.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:03.766+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:59:05.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.361+0000 7fa5c7e00640 1 -- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 msgr2=0x7fa5d01527d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:05.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.361+0000 7fa5c7e00640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 0x7fa5d01527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:05.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.361+0000 7fa5c7e00640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 msgr2=0x7fa5d01540a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:05.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.361+0000 7fa5c7e00640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.67:3300/0 conn(0x7fa5d014c9f0 0x7fa5d01540a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:05.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.361+0000 7fa5c7e00640 1 -- 172.21.15.67:0/4239162864 >> v1:172.21.15.67:6789/0 conn(0x7fa5d006d420 legacy=0x7fa5d014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:05.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.361+0000 7fa5c7e00640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 0x7fa5b0009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:05.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.361+0000 7fa5c7e00640 1 -- 172.21.15.67:0/4239162864 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d006dee0 con 0x7fa5b00036e0 2024-03-09T08:59:05.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.361+0000 7fa5c7e00640 1 -- 172.21.15.67:0/4239162864 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d0155ba0 con 0x7fa5b0007370 2024-03-09T08:59:05.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.362+0000 7fa5cde00640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 0x7fa5b0009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:05.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.362+0000 7fa5cde00640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 msgr2=0x7fa5b0009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:59:05.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.362+0000 7fa5cde00640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 msgr2=0x7fa5b0009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:05.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.362+0000 7fa5cde00640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 0x7fa5b0009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:05.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.362+0000 7fa5cde00640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 0x7fa5b0009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:05.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.563+0000 7fa5cde00640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 0x7fa5b0009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:05.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.564+0000 7fa5cde00640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 msgr2=0x7fa5b0009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:59:05.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.564+0000 7fa5cde00640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 msgr2=0x7fa5b0009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:05.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.564+0000 7fa5cde00640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 0x7fa5b0009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:05.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.564+0000 7fa5cde00640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 0x7fa5b0009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:59:05.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.647+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256035379 0 0) 0x7fa5d006dee0 con 0x7fa5b00036e0 2024-03-09T08:59:05.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.647+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b4003520 con 0x7fa5b00036e0 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.677+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 646195310 0 0) 0x7fa5b4003520 con 0x7fa5b00036e0 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.677+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 msgr2=0x7fa5b0009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.677+0000 7fa5cd400640 1 --2- 172.21.15.67:0/4239162864 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0007370 0x7fa5b0009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.677+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b000d190 con 0x7fa5b00036e0 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.677+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fa5c0004340 con 0x7fa5b00036e0 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.677+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 >> v1:172.21.15.134:6789/0 conn(0x7fa5b00036e0 legacy=0x7fa5b0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.678+0000 7fa5cd400640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b4003a20 0x7fa5b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.678+0000 7fa5cd400640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 0x7fa5b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.678+0000 7fa5cf200640 1 -- 172.21.15.67:0/4239162864 reap_dead start 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.678+0000 7fa5cd400640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4004d40 0x7fa5b40051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.678+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d0155ba0 con 0x7fa5b4004d40 2024-03-09T08:59:05.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.678+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b400bbd0 con 0x7fa5b4003a20 2024-03-09T08:59:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.678+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b400bdd0 con 0x7fa5d00c3080 2024-03-09T08:59:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.678+0000 7fa5cde00640 1 -- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 msgr2=0x7fa5b4004800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.678+0000 7fa5cde00640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 0x7fa5b4004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.678+0000 7fa5ce800640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b4003a20 0x7fa5b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.679+0000 7fa5cf200640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4004d40 0x7fa5b40051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.679+0000 7fa5ce800640 1 -- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 msgr2=0x7fa5b4004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.679+0000 7fa5ce800640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 0x7fa5b4004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.679+0000 7fa5ce800640 1 -- 172.21.15.67:0/4239162864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4004d40 msgr2=0x7fa5b40051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.679+0000 7fa5ce800640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4004d40 0x7fa5b40051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.679+0000 7fa5ce800640 1 -- 172.21.15.67:0/4239162864 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b4003520 con 0x7fa5b4003a20 2024-03-09T08:59:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.681+0000 7fa5ce800640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b4003a20 0x7fa5b4003e20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000c490 tx=0x7fa5b4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afecfa392182d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:05.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.681+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000c0e0 con 0x7fa5b4003a20 2024-03-09T08:59:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.681+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c0030070 con 0x7fa5b4003a20 2024-03-09T08:59:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.681+0000 7fa5cd400640 1 -- 172.21.15.67:0/4239162864 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0035470 con 0x7fa5b4003a20 2024-03-09T08:59:05.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.682+0000 7fa5d4a00640 1 -- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b4003a20 msgr2=0x7fa5b4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.682+0000 7fa5d4a00640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b4003a20 0x7fa5b4003e20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000c490 tx=0x7fa5b4005830 comp rx=0 tx=0).stop 2024-03-09T08:59:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.682+0000 7fa5d4a00640 1 -- 172.21.15.67:0/4239162864 shutdown_connections 2024-03-09T08:59:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.682+0000 7fa5d4a00640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4004d40 0x7fa5b40051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.682+0000 7fa5d4a00640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5d00c3080 0x7fa5b4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.682+0000 7fa5d4a00640 1 --2- 172.21.15.67:0/4239162864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b4003a20 0x7fa5b4003e20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.682+0000 7fa5d4a00640 1 -- 172.21.15.67:0/4239162864 >> 172.21.15.67:0/4239162864 conn(0x7fa5d00c7300 msgr2=0x7fa5d00c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.682+0000 7fa5d4a00640 1 -- 172.21.15.67:0/4239162864 shutdown_connections 2024-03-09T08:59:05.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.682+0000 7fa5d4a00640 1 -- 172.21.15.67:0/4239162864 wait complete. 2024-03-09T08:59:05.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.686+0000 7fa5d4a00640 1 Processor -- start 2024-03-09T08:59:05.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.686+0000 7fa5d4a00640 1 -- start start 2024-03-09T08:59:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.687+0000 7fa5d4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4003a20 0x7fa5d01e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.687+0000 7fa5d4a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b4004d40 0x7fa5d00aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.687+0000 7fa5d4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d00c3080 0x7fa5d01e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.687+0000 7fa5d4a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00c8030 con 0x7fa5b4003a20 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.687+0000 7fa5d4a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d00c3460 con 0x7fa5d00c3080 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5d4a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d014d8a0 con 0x7fa5b4004d40 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5cde00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b4004d40 msgr2=0x7fa5d00aca90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5cde00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b4004d40 0x7fa5d00aca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4003a20 0x7fa5d01e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4003a20 0x7fa5d01e7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47578/0 (socket says 172.21.15.67:47578) 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5ce800640 1 -- 172.21.15.67:0/764203520 learned_addr learned my addr 172.21.15.67:0/764203520 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5cf200640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d00c3080 0x7fa5d01e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5ce800640 1 -- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b4004d40 msgr2=0x7fa5d00aca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5ce800640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b4004d40 0x7fa5d00aca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5ce800640 1 -- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d00c3080 msgr2=0x7fa5d01e6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5ce800640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d00c3080 0x7fa5d01e6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:05.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.688+0000 7fa5ce800640 1 -- 172.21.15.67:0/764203520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d0072620 con 0x7fa5b4003a20 2024-03-09T08:59:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.689+0000 7fa5ce800640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4003a20 0x7fa5d01e7b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0000ea0 tx=0x7fa5c002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.689+0000 7fa5c6a00640 1 -- 172.21.15.67:0/764203520 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0035880 con 0x7fa5b4003a20 2024-03-09T08:59:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.689+0000 7fa5c6a00640 1 -- 172.21.15.67:0/764203520 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c0035a20 con 0x7fa5b4003a20 2024-03-09T08:59:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.689+0000 7fa5d4a00640 1 -- 172.21.15.67:0/764203520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d014dad0 con 0x7fa5b4003a20 2024-03-09T08:59:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.689+0000 7fa5c6a00640 1 -- 172.21.15.67:0/764203520 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0035e10 con 0x7fa5b4003a20 2024-03-09T08:59:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.689+0000 7fa5d4a00640 1 -- 172.21.15.67:0/764203520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d014ff10 con 0x7fa5b4003a20 2024-03-09T08:59:05.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.691+0000 7fa5bbe00640 1 -- 172.21.15.67:0/764203520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d004f3c0 con 0x7fa5b4003a20 2024-03-09T08:59:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.693+0000 7fa5c6a00640 1 -- 172.21.15.67:0/764203520 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa5c0041430 con 0x7fa5b4003a20 2024-03-09T08:59:05.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.694+0000 7fa5c6a00640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa5b006de60 0x7fa5b0070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:05.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.694+0000 7fa5c6a00640 1 -- 172.21.15.67:0/764203520 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa5c00abbf0 con 0x7fa5b4003a20 2024-03-09T08:59:05.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.694+0000 7fa5cde00640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa5b006de60 0x7fa5b0070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:05.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.695+0000 7fa5cde00640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa5b006de60 0x7fa5b0070320 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00c1b60 tx=0x7fa5bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:05.701+0000 7fa5c6a00640 1 -- 172.21.15.67:0/764203520 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa5c007a060 con 0x7fa5b4003a20 2024-03-09T08:59:06.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.053+0000 7fa5bbe00640 1 -- 172.21.15.67:0/764203520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa5d0151e50 con 0x7fa5b4003a20 2024-03-09T08:59:06.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.053+0000 7fa5c6a00640 1 -- 172.21.15.67:0/764203520 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7fa5c007e100 con 0x7fa5b4003a20 2024-03-09T08:59:06.054 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":54,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:59:06.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 -- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa5b006de60 msgr2=0x7fa5b0070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa5b006de60 0x7fa5b0070320 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00c1b60 tx=0x7fa5bc009040 comp rx=0 tx=0).stop 2024-03-09T08:59:06.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 -- 172.21.15.67:0/764203520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4003a20 msgr2=0x7fa5d01e7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4003a20 0x7fa5d01e7b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0000ea0 tx=0x7fa5c002e070 comp rx=0 tx=0).stop 2024-03-09T08:59:06.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 -- 172.21.15.67:0/764203520 shutdown_connections 2024-03-09T08:59:06.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa5b006de60 0x7fa5b0070320 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5d00c3080 0x7fa5d01e6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b4004d40 0x7fa5d00aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 --2- 172.21.15.67:0/764203520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b4003a20 0x7fa5d01e7b80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 -- 172.21.15.67:0/764203520 >> 172.21.15.67:0/764203520 conn(0x7fa5d00c7300 msgr2=0x7fa5d0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:06.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 -- 172.21.15.67:0/764203520 shutdown_connections 2024-03-09T08:59:06.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.056+0000 7fa5bbe00640 1 -- 172.21.15.67:0/764203520 wait complete. 2024-03-09T08:59:06.081 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T08:59:06.081 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T08:59:06.081 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:59:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.280+0000 7f723e400640 1 Processor -- start 2024-03-09T08:59:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.280+0000 7f723e400640 1 -- start start 2024-03-09T08:59:06.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.281+0000 7f723e400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f72380734a0 0x7f72380aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.281+0000 7f723e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f723806d830 0x7f723806dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.281+0000 7f723e400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7238152e00 con 0x7f72380730c0 2024-03-09T08:59:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.281+0000 7f723e400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72380aa960 con 0x7f72380734a0 2024-03-09T08:59:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.281+0000 7f723e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72381530e0 con 0x7f723806d830 2024-03-09T08:59:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.281+0000 7f7236a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f72380734a0 0x7f72380aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.281+0000 7f7237e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f723806d830 msgr2=0x7f723806dca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.281+0000 7f7237e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f723806d830 0x7f723806dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.281+0000 7f7236a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f72380734a0 0x7f72380aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47590/0 (socket says 172.21.15.67:47590) 2024-03-09T08:59:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.281+0000 7f7236a00640 1 -- 172.21.15.67:0/1175208815 learned_addr learned my addr 172.21.15.67:0/1175208815 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.282+0000 7f7236a00640 1 -- 172.21.15.67:0/1175208815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f723806d830 msgr2=0x7f723806dca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.282+0000 7f7236a00640 1 --2- 172.21.15.67:0/1175208815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f723806d830 0x7f723806dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.282+0000 7f7236a00640 1 -- 172.21.15.67:0/1175208815 >> v1:172.21.15.134:6789/0 conn(0x7f72380730c0 legacy=0x7f72380c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.282+0000 7f7236a00640 1 -- 172.21.15.67:0/1175208815 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72381533c0 con 0x7f72380734a0 2024-03-09T08:59:06.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.283+0000 7f7236a00640 1 --2- 172.21.15.67:0/1175208815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72380734a0 0x7f72380aa420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7224009870 tx=0x7f722402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b68a2b257bbe94 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:06.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.284+0000 7f7236000640 1 -- 172.21.15.67:0/1175208815 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224035020 con 0x7f72380734a0 2024-03-09T08:59:06.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.284+0000 7f7236000640 1 -- 172.21.15.67:0/1175208815 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7224031870 con 0x7f72380734a0 2024-03-09T08:59:06.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.284+0000 7f7236000640 1 -- 172.21.15.67:0/1175208815 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224031c40 con 0x7f72380734a0 2024-03-09T08:59:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.285+0000 7f723e400640 1 -- 172.21.15.67:0/1175208815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72380734a0 msgr2=0x7f72380aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.285+0000 7f723e400640 1 --2- 172.21.15.67:0/1175208815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72380734a0 0x7f72380aa420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7224009870 tx=0x7f722402b910 comp rx=0 tx=0).stop 2024-03-09T08:59:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.285+0000 7f723e400640 1 -- 172.21.15.67:0/1175208815 shutdown_connections 2024-03-09T08:59:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.285+0000 7f723e400640 1 --2- 172.21.15.67:0/1175208815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f723806d830 0x7f723806dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.285+0000 7f723e400640 1 --2- 172.21.15.67:0/1175208815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72380734a0 0x7f72380aa420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.285+0000 7f723e400640 1 -- 172.21.15.67:0/1175208815 >> 172.21.15.67:0/1175208815 conn(0x7f72380c7370 msgr2=0x7f72380c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.285+0000 7f723e400640 1 -- 172.21.15.67:0/1175208815 shutdown_connections 2024-03-09T08:59:06.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.285+0000 7f723e400640 1 -- 172.21.15.67:0/1175208815 wait complete. 2024-03-09T08:59:06.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.287+0000 7f723e400640 1 Processor -- start 2024-03-09T08:59:06.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.288+0000 7f723e400640 1 -- start start 2024-03-09T08:59:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.289+0000 7f723e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f723806d830 0x7f72380c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.289+0000 7f723e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72380730c0 0x7f7238148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.289+0000 7f723e400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72380734a0 0x7f7238149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.289+0000 7f723e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c3590 con 0x7f723806d830 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.289+0000 7f723e400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7238153980 con 0x7f72380734a0 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.289+0000 7f723e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72380c3710 con 0x7f72380730c0 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.289+0000 7f7237400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f723806d830 0x7f72380c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.289+0000 7f7236a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72380730c0 msgr2=0x7f7238148b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.289+0000 7f7237400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f723806d830 0x7f72380c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47592/0 (socket says 172.21.15.67:47592) 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.289+0000 7f7237400640 1 -- 172.21.15.67:0/3754799955 learned_addr learned my addr 172.21.15.67:0/3754799955 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.289+0000 7f7236a00640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72380730c0 0x7f7238148b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.290+0000 7f7237e00640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72380734a0 0x7f7238149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.290+0000 7f7237400640 1 -- 172.21.15.67:0/3754799955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72380730c0 msgr2=0x7f7238148b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:59:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.290+0000 7f7237400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72380730c0 0x7f7238148b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.290+0000 7f7237400640 1 -- 172.21.15.67:0/3754799955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72380734a0 msgr2=0x7f7238149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.290+0000 7f7237400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72380734a0 0x7f7238149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.290+0000 7f7237400640 1 -- 172.21.15.67:0/3754799955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72381eede0 con 0x7f723806d830 2024-03-09T08:59:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.290+0000 7f7237400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f723806d830 0x7f72380c0a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f722800c400 tx=0x7f722800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.290+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72280052d0 con 0x7f723806d830 2024-03-09T08:59:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.291+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7228005470 con 0x7f723806d830 2024-03-09T08:59:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.291+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228007260 con 0x7f723806d830 2024-03-09T08:59:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.291+0000 7f723e400640 1 -- 172.21.15.67:0/3754799955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72381ef0c0 con 0x7f723806d830 2024-03-09T08:59:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.291+0000 7f723e400640 1 -- 172.21.15.67:0/3754799955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72380c0f50 con 0x7f723806d830 2024-03-09T08:59:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.292+0000 7f723e400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7238150890 0x7f72381eb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.292+0000 7f723e400640 1 -- 172.21.15.67:0/3754799955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72381eeb90 con 0x7f7238150890 2024-03-09T08:59:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.292+0000 7f7236a00640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7238150890 0x7f72381eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.293+0000 7f7236a00640 1 -- 172.21.15.67:0/3754799955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7238075a00 con 0x7f7238150890 2024-03-09T08:59:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.293+0000 7f7236a00640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7238150890 0x7f72381eb620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f723806c090 tx=0x7f72240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.293+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7228007400 con 0x7f723806d830 2024-03-09T08:59:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.294+0000 7f722f400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f721806b480 0x7f721806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.294+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f7228089ab0 con 0x7f723806d830 2024-03-09T08:59:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.295+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224031c20 con 0x7f7238150890 2024-03-09T08:59:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.295+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f7224004020 con 0x7f7238150890 2024-03-09T08:59:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.295+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7238150890 msgr2=0x7f72381eb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.295+0000 7f722f400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7238150890 0x7f72381eb620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f723806c090 tx=0x7f72240046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.295+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7238150890 msgr2=0x7f72381eb620 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:06.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.295+0000 7f722f400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7238150890 0x7f72381eb620 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f723806c090 tx=0x7f72240046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.295+0000 7f7237e00640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f721806b480 0x7f721806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.296+0000 7f7237e00640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f721806b480 0x7f721806d940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7230003bc0 tx=0x7f7230009450 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:06.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.309+0000 7f723e400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72380c2a60 0x7f72381ebb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.309+0000 7f723e400640 1 -- 172.21.15.67:0/3754799955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380acff0 con 0x7f72380c2a60 2024-03-09T08:59:06.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.309+0000 7f7236a00640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72380c2a60 0x7f72381ebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.310+0000 7f7236a00640 1 -- 172.21.15.67:0/3754799955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f72380683f0 con 0x7f72380c2a60 2024-03-09T08:59:06.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.310+0000 7f7236a00640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72380c2a60 0x7f72381ebb60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7224000c00 tx=0x7f7224009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:06.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.311+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722404f070 con 0x7f72380c2a60 2024-03-09T08:59:06.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.311+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5007 (secure 0 0 0) 0x7f7224003c30 con 0x7f72380c2a60 2024-03-09T08:59:06.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.311+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72380c2a60 msgr2=0x7f72381ebb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.311+0000 7f722f400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72380c2a60 0x7f72381ebb60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7224000c00 tx=0x7f7224009240 comp rx=0 tx=0).stop 2024-03-09T08:59:06.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.311+0000 7f722f400640 1 -- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72380c2a60 msgr2=0x7f72381ebb60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:06.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.311+0000 7f722f400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72380c2a60 0x7f72381ebb60 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7224000c00 tx=0x7f7224009240 comp rx=0 tx=0).stop 2024-03-09T08:59:06.314 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:59:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:59:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:59:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:59:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 54, 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:59:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:59:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:06.325 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.326 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.326 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:59:06.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.326 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:59:06.326 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:06.326 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:06.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.326 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:06.326 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:06.326 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:06.327 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.327 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.327 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:59:06.327 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:59:06.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.312+0000 7f723e400640 1 -- 172.21.15.67:0/3754799955 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f721806b480 msgr2=0x7f721806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.312+0000 7f723e400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f721806b480 0x7f721806d940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7230003bc0 tx=0x7f7230009450 comp rx=0 tx=0).stop 2024-03-09T08:59:06.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.312+0000 7f723e400640 1 -- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f723806d830 msgr2=0x7f72380c0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.312+0000 7f723e400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f723806d830 0x7f72380c0a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f722800c400 tx=0x7f722800c8c0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.312+0000 7f7237e00640 1 -- 172.21.15.67:0/3754799955 reap_dead start 2024-03-09T08:59:06.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.312+0000 7f723e400640 1 -- 172.21.15.67:0/3754799955 shutdown_connections 2024-03-09T08:59:06.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.312+0000 7f723e400640 1 --2- 172.21.15.67:0/3754799955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f723806d830 0x7f72380c0a10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.312+0000 7f723e400640 1 -- 172.21.15.67:0/3754799955 >> 172.21.15.67:0/3754799955 conn(0x7f72380c7370 msgr2=0x7f72380c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:06.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.312+0000 7f723e400640 1 -- 172.21.15.67:0/3754799955 shutdown_connections 2024-03-09T08:59:06.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.312+0000 7f723e400640 1 -- 172.21.15.67:0/3754799955 wait complete. 2024-03-09T08:59:06.334 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:59:06.334 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T08:59:06.334 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:59:06.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.540+0000 7f139ae00640 1 Processor -- start 2024-03-09T08:59:06.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.540+0000 7f139ae00640 1 -- start start 2024-03-09T08:59:06.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.542+0000 7f139ae00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f139406c1d0 0x7f139406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.542+0000 7f139ae00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13940c9890 con 0x7f139414fd90 2024-03-09T08:59:06.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.542+0000 7f139ae00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13940c5f80 con 0x7f13940c9380 2024-03-09T08:59:06.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.542+0000 7f139ae00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f139406ba60 con 0x7f139406c1d0 2024-03-09T08:59:06.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.542+0000 7f1399a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f139414fd90 0x7f1394156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51974/0 (socket says 172.21.15.67:51974) 2024-03-09T08:59:06.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.542+0000 7f1399a00640 1 -- 172.21.15.67:0/4047858673 learned_addr learned my addr 172.21.15.67:0/4047858673 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f139a400640 1 --2- 172.21.15.67:0/4047858673 >> v2:172.21.15.134:3300/0 conn(0x7f139406c1d0 0x7f139406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048673020 0 0) 0x7f13940c9890 con 0x7f139414fd90 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f137c003520 con 0x7f139414fd90 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286228952 0 0) 0x7f13940c5f80 con 0x7f13940c9380 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13940c9890 con 0x7f13940c9380 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3936495914 0 0) 0x7f137c003520 con 0x7f139414fd90 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 >> v2:172.21.15.134:3300/0 conn(0x7f139406c1d0 msgr2=0x7f139406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 --2- 172.21.15.67:0/4047858673 >> v2:172.21.15.134:3300/0 conn(0x7f139406c1d0 0x7f139406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 >> v1:172.21.15.134:6789/0 conn(0x7f13940c9380 legacy=0x7f13940acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13940c6240 con 0x7f139414fd90 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f1388001670 con 0x7f139414fd90 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 >> v1:172.21.15.67:6789/0 conn(0x7f139414fd90 legacy=0x7f1394156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c003c00 0x7f137c004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f137c004800 0x7f137c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c005390 0x7f137c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139406ba60 con 0x7f137c004800 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c00c260 con 0x7f137c005390 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.543+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c00c460 con 0x7f137c003c00 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.544+0000 7f1399a00640 1 -- 172.21.15.67:0/4047858673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c003c00 msgr2=0x7f137c004020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.544+0000 7f1399000640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f137c004800 0x7f137c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.544+0000 7f1399a00640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c003c00 0x7f137c004020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.544+0000 7f139a400640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c005390 0x7f137c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.544+0000 7f1399000640 1 -- 172.21.15.67:0/4047858673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c003c00 msgr2=0x7f137c004020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.544+0000 7f1399000640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f137c003c00 0x7f137c004020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.544+0000 7f1399000640 1 -- 172.21.15.67:0/4047858673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c005390 msgr2=0x7f137c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.544+0000 7f1399000640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c005390 0x7f137c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.544+0000 7f1399000640 1 -- 172.21.15.67:0/4047858673 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c003520 con 0x7f137c004800 2024-03-09T08:59:06.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.544+0000 7f139a400640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c005390 0x7f137c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T08:59:06.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.544+0000 7f139a400640 1 -- 172.21.15.67:0/4047858673 reap_dead start 2024-03-09T08:59:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.545+0000 7f1399000640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f137c004800 0x7f137c004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f13800018d0 tx=0x7f137c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949ab60a9fe26282 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.546+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13800014f0 con 0x7f137c004800 2024-03-09T08:59:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.546+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13800052d0 con 0x7f137c004800 2024-03-09T08:59:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.546+0000 7f1393e00640 1 -- 172.21.15.67:0/4047858673 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380003810 con 0x7f137c004800 2024-03-09T08:59:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.546+0000 7f139ae00640 1 -- 172.21.15.67:0/4047858673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f137c004800 msgr2=0x7f137c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.546+0000 7f139ae00640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f137c004800 0x7f137c004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f13800018d0 tx=0x7f137c005830 comp rx=0 tx=0).stop 2024-03-09T08:59:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.547+0000 7f139ae00640 1 -- 172.21.15.67:0/4047858673 shutdown_connections 2024-03-09T08:59:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.547+0000 7f139ae00640 1 --2- 172.21.15.67:0/4047858673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f137c004800 0x7f137c004c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.547+0000 7f139ae00640 1 -- 172.21.15.67:0/4047858673 >> 172.21.15.67:0/4047858673 conn(0x7f13940cb480 msgr2=0x7f13940cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.547+0000 7f139ae00640 1 -- 172.21.15.67:0/4047858673 shutdown_connections 2024-03-09T08:59:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.547+0000 7f139ae00640 1 -- 172.21.15.67:0/4047858673 wait complete. 2024-03-09T08:59:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.549+0000 7f139ae00640 1 Processor -- start 2024-03-09T08:59:06.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.549+0000 7f139ae00640 1 -- start start 2024-03-09T08:59:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f139ae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c004800 0x7f139414c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f139ae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13941595c0 0x7f13940ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f139ae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139414a2a0 0x7f13940acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f139ae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394156680 con 0x7f139414a2a0 2024-03-09T08:59:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f139ae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139406bcb0 con 0x7f137c004800 2024-03-09T08:59:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f139ae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1394156800 con 0x7f13941595c0 2024-03-09T08:59:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f1399000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13941595c0 msgr2=0x7f13940ac9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f1399000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13941595c0 0x7f13940ac9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f139a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139414a2a0 0x7f13940acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f1399a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c004800 0x7f139414c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f139a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139414a2a0 0x7f13940acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47640/0 (socket says 172.21.15.67:47640) 2024-03-09T08:59:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.551+0000 7f139a400640 1 -- 172.21.15.67:0/3174077077 learned_addr learned my addr 172.21.15.67:0/3174077077 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.552+0000 7f139a400640 1 -- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13941595c0 msgr2=0x7f13940ac9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:59:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.552+0000 7f139a400640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13941595c0 0x7f13940ac9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.552+0000 7f139a400640 1 -- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c004800 msgr2=0x7f139414c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.552+0000 7f139a400640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f137c004800 0x7f139414c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.552+0000 7f139a400640 1 -- 172.21.15.67:0/3174077077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394149fa0 con 0x7f139414a2a0 2024-03-09T08:59:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.552+0000 7f139a400640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139414a2a0 0x7f13940acee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f13941537d0 tx=0x7f138c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.552+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c016620 con 0x7f139414a2a0 2024-03-09T08:59:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.552+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138c01b070 con 0x7f139414a2a0 2024-03-09T08:59:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.552+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c0169c0 con 0x7f139414a2a0 2024-03-09T08:59:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.553+0000 7f139ae00640 1 -- 172.21.15.67:0/3174077077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13941ef690 con 0x7f139414a2a0 2024-03-09T08:59:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.553+0000 7f139ae00640 1 -- 172.21.15.67:0/3174077077 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139414c5d0 con 0x7f139414a2a0 2024-03-09T08:59:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.554+0000 7f139ae00640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ad420 0x7f13940aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.554+0000 7f139ae00640 1 -- 172.21.15.67:0/3174077077 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940ac300 con 0x7f13940ad420 2024-03-09T08:59:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.554+0000 7f1399a00640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ad420 0x7f13940aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.555+0000 7f1399a00640 1 -- 172.21.15.67:0/3174077077 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1394149d50 con 0x7f13940ad420 2024-03-09T08:59:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.555+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f138c005d10 con 0x7f139414a2a0 2024-03-09T08:59:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.555+0000 7f1399a00640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ad420 0x7f13940aade0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1388001860 tx=0x7f138800fb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.556+0000 7f1391600640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f137406b480 0x7f137406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.556+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f138c08b040 con 0x7f139414a2a0 2024-03-09T08:59:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.556+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388019070 con 0x7f13940ad420 2024-03-09T08:59:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.556+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f138800d770 con 0x7f13940ad420 2024-03-09T08:59:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.556+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ad420 msgr2=0x7f13940aade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.556+0000 7f1391600640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ad420 0x7f13940aade0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1388001860 tx=0x7f138800fb50 comp rx=0 tx=0).stop 2024-03-09T08:59:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.556+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ad420 msgr2=0x7f13940aade0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.556+0000 7f1391600640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ad420 0x7f13940aade0 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1388001860 tx=0x7f138800fb50 comp rx=0 tx=0).stop 2024-03-09T08:59:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.556+0000 7f1399000640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f137406b480 0x7f137406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.558+0000 7f1399000640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f137406b480 0x7f137406d940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1380034000 tx=0x7f1380031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.571+0000 7f139ae00640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ab320 0x7f13940ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.571+0000 7f139ae00640 1 -- 172.21.15.67:0/3174077077 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940abc70 con 0x7f13940ab320 2024-03-09T08:59:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.571+0000 7f1399a00640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ab320 0x7f13940ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:06.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.572+0000 7f1399a00640 1 -- 172.21.15.67:0/3174077077 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f139404f3c0 con 0x7f13940ab320 2024-03-09T08:59:06.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.572+0000 7f1399a00640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ab320 0x7f13940ab730 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1388004fd0 tx=0x7f1388000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:06.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.573+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388019040 con 0x7f13940ab320 2024-03-09T08:59:06.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.573+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5175 (secure 0 0 0) 0x7f1388008070 con 0x7f13940ab320 2024-03-09T08:59:06.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.573+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ab320 msgr2=0x7f13940ab730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.573+0000 7f1391600640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ab320 0x7f13940ab730 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1388004fd0 tx=0x7f1388000ed0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.573+0000 7f1391600640 1 -- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ab320 msgr2=0x7f13940ab730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:06.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.573+0000 7f1391600640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13940ab320 0x7f13940ab730 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1388004fd0 tx=0x7f1388000ed0 comp rx=0 tx=0).stop 2024-03-09T08:59:06.575 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:59:06.575 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:59:06.575 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 54, 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:06.576 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:06.577 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:59:06.578 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:06.579 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:59:06.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:59:06.581 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:59:06.582 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.583 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:59:06.584 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:06.585 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:59:06.585 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:06.585 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.585 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.585 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:59:06.586 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:59:06.586 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:06.586 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:06.586 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:06.587 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.587 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.587 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:59:06.587 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:59:06.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.587 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:06.587 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:06.587 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:06.587 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.587 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.588 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:59:06.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.588 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:06.588 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:06.588 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:06.588 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.588 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:59:06.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.589 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:06.589 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:06.589 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:59:06.589 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.589 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.589 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:59:06.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.589 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:59:06.590 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:06.590 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:06.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.590 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:06.590 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:06.590 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T08:59:06.590 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.591 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:06.591 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:59:06.591 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:06.591 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:06.596 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:06.596 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:06.596 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:06.596 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:06.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:06.596 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:59:06.596 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:59:06.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.574+0000 7f139ae00640 1 -- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f137406b480 msgr2=0x7f137406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.574+0000 7f139ae00640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f137406b480 0x7f137406d940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1380034000 tx=0x7f1380031000 comp rx=0 tx=0).stop 2024-03-09T08:59:06.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.574+0000 7f139ae00640 1 -- 172.21.15.67:0/3174077077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139414a2a0 msgr2=0x7f13940acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:06.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.574+0000 7f139ae00640 1 --2- 172.21.15.67:0/3174077077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f139414a2a0 0x7f13940acee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f13941537d0 tx=0x7f138c00f520 comp rx=0 tx=0).stop 2024-03-09T08:59:06.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.574+0000 7f139a400640 1 -- 172.21.15.67:0/3174077077 reap_dead start 2024-03-09T08:59:06.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.574+0000 7f139a400640 1 -- 172.21.15.67:0/3174077077 reap_dead start 2024-03-09T08:59:06.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.574+0000 7f139ae00640 1 -- 172.21.15.67:0/3174077077 shutdown_connections 2024-03-09T08:59:06.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.574+0000 7f139ae00640 1 -- 172.21.15.67:0/3174077077 >> 172.21.15.67:0/3174077077 conn(0x7f13940cb480 msgr2=0x7f139414ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:06.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.574+0000 7f139ae00640 1 -- 172.21.15.67:0/3174077077 shutdown_connections 2024-03-09T08:59:06.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:06.574+0000 7f139ae00640 1 -- 172.21.15.67:0/3174077077 wait complete. 2024-03-09T08:59:06.599 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T08:59:06.599 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T08:59:11.745 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:59:11.754 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:59:26.600 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T08:59:26.600 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T08:59:26.600 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T08:59:26.600 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T08:59:26.606 INFO:tasks.ceph.mon.c:Started 2024-03-09T08:59:26.606 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:59:26.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:59:26.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.826+0000 7f8d2fe00640 1 Processor -- start 2024-03-09T08:59:27.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.827+0000 7f8d2fe00640 1 -- start start 2024-03-09T08:59:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.828+0000 7f8d2fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d300bebc0 0x7f8d30072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.828+0000 7f8d2fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3006cd30 con 0x7f8d3006bda0 2024-03-09T08:59:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.828+0000 7f8d2fe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d300c5eb0 con 0x7f8d30072770 2024-03-09T08:59:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.828+0000 7f8d2fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3006b200 con 0x7f8d300bebc0 2024-03-09T08:59:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.828+0000 7f8d2e000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8d3006bda0 0x7f8d3015e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41884/0 (socket says 172.21.15.67:41884) 2024-03-09T08:59:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.828+0000 7f8d2e000640 1 -- 172.21.15.67:0/3047466652 learned_addr learned my addr 172.21.15.67:0/3047466652 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:59:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.828+0000 7f8d2f400640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d300bebc0 0x7f8d30072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2f400640 1 -- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d300bebc0 msgr2=0x7f8d30072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:59:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2f400640 1 -- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d300bebc0 msgr2=0x7f8d30072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2f400640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d300bebc0 0x7f8d30072b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2f400640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d300bebc0 0x7f8d30072b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731145918 0 0) 0x7f8d300c5eb0 con 0x7f8d30072770 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d14003520 con 0x7f8d30072770 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667010469 0 0) 0x7f8d3006cd30 con 0x7f8d3006bda0 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d300c5eb0 con 0x7f8d3006bda0 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3248553254 0 0) 0x7f8d300c5eb0 con 0x7f8d3006bda0 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d300bebc0 msgr2=0x7f8d30072b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2d600640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d300bebc0 0x7f8d30072b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.829+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 >> v1:172.21.15.134:6789/0 conn(0x7f8d30072770 legacy=0x7f8d3006b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300c6150 con 0x7f8d3006bda0 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f8d1c0015f0 con 0x7f8d3006bda0 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 >> v1:172.21.15.67:6789/0 conn(0x7f8d3006bda0 legacy=0x7f8d3015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:27.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2d600640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d14004c70 0x7f8d14005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:27.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2d600640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 0x7f8d14003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:27.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2d600640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d140042d0 0x7f8d1400ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:27.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3006b200 con 0x7f8d140042d0 2024-03-09T08:59:27.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1400c280 con 0x7f8d14004c70 2024-03-09T08:59:27.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1400c480 con 0x7f8d140037a0 2024-03-09T08:59:27.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2ea00640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d14004c70 0x7f8d14005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:27.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2e000640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 0x7f8d14003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:27.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.830+0000 7f8d2f400640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d140042d0 0x7f8d1400ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:27.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.831+0000 7f8d2e000640 1 -- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d14004c70 msgr2=0x7f8d14005050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:27.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.831+0000 7f8d2e000640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d14004c70 0x7f8d14005050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.831+0000 7f8d2e000640 1 -- 172.21.15.67:0/3047466652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d140042d0 msgr2=0x7f8d1400ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.831+0000 7f8d2e000640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d140042d0 0x7f8d1400ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.831+0000 7f8d2e000640 1 -- 172.21.15.67:0/3047466652 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300c6150 con 0x7f8d140037a0 2024-03-09T08:59:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.831+0000 7f8d2f400640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d140042d0 0x7f8d1400ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:59:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.832+0000 7f8d2e000640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 0x7f8d14003c20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c1b0 tx=0x7f8d14005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e4b2482f5865af server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.832+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c007ca0 con 0x7f8d140037a0 2024-03-09T08:59:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.832+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c030070 con 0x7f8d140037a0 2024-03-09T08:59:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.832+0000 7f8d2d600640 1 -- 172.21.15.67:0/3047466652 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c005d10 con 0x7f8d140037a0 2024-03-09T08:59:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.833+0000 7f8d2fe00640 1 -- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 msgr2=0x7f8d14003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:27.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.833+0000 7f8d2fe00640 1 --2- 172.21.15.67:0/3047466652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 0x7f8d14003c20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c1b0 tx=0x7f8d14005830 comp rx=0 tx=0).stop 2024-03-09T08:59:27.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.833+0000 7f8d2f400640 1 -- 172.21.15.67:0/3047466652 reap_dead start 2024-03-09T08:59:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.833+0000 7f8d2fe00640 1 -- 172.21.15.67:0/3047466652 shutdown_connections 2024-03-09T08:59:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.833+0000 7f8d2fe00640 1 -- 172.21.15.67:0/3047466652 >> 172.21.15.67:0/3047466652 conn(0x7f8d300c7fb0 msgr2=0x7f8d300c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.833+0000 7f8d2fe00640 1 -- 172.21.15.67:0/3047466652 shutdown_connections 2024-03-09T08:59:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.834+0000 7f8d2fe00640 1 -- 172.21.15.67:0/3047466652 wait complete. 2024-03-09T08:59:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.835+0000 7f8d2fe00640 1 Processor -- start 2024-03-09T08:59:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.835+0000 7f8d2fe00640 1 -- start start 2024-03-09T08:59:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.837+0000 7f8d2fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 0x7f8d30154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.837+0000 7f8d2fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d30158470 0x7f8d301eedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.837+0000 7f8d2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d301ef2f0 0x7f8d301eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:27.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.837+0000 7f8d2fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300c9430 con 0x7f8d301ef2f0 2024-03-09T08:59:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.837+0000 7f8d2fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300c6710 con 0x7f8d30158470 2024-03-09T08:59:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.837+0000 7f8d2fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300c4e60 con 0x7f8d140037a0 2024-03-09T08:59:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.837+0000 7f8d2e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d30158470 0x7f8d301eedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.837+0000 7f8d2e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d30158470 0x7f8d301eedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:48158/0 (socket says 172.21.15.67:48158) 2024-03-09T08:59:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.837+0000 7f8d2e000640 1 -- 172.21.15.67:0/373322734 learned_addr learned my addr 172.21.15.67:0/373322734 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.837+0000 7f8d2ea00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 0x7f8d30154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.837+0000 7f8d2f400640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d301ef2f0 0x7f8d301eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.838+0000 7f8d2ea00640 1 -- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 msgr2=0x7f8d30154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T08:59:27.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.838+0000 7f8d2ea00640 1 -- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 msgr2=0x7f8d30154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.838+0000 7f8d2ea00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 0x7f8d30154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.838+0000 7f8d2ea00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 0x7f8d30154300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.838+0000 7f8d2e000640 1 -- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 msgr2=0x7f8d30154300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.838+0000 7f8d2e000640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 0x7f8d30154300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.838+0000 7f8d2e000640 1 -- 172.21.15.67:0/373322734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d301ef2f0 msgr2=0x7f8d301eb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.838+0000 7f8d2e000640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d301ef2f0 0x7f8d301eb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.838+0000 7f8d2e000640 1 -- 172.21.15.67:0/373322734 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d30155f60 con 0x7f8d30158470 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.838+0000 7f8d2f400640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d301ef2f0 0x7f8d301eb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.838+0000 7f8d2e000640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d30158470 0x7f8d301eedb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c2a0 tx=0x7f8d1c02e070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.839+0000 7f8d26a00640 1 -- 172.21.15.67:0/373322734 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c007ca0 con 0x7f8d30158470 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.839+0000 7f8d26a00640 1 -- 172.21.15.67:0/373322734 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c004bf0 con 0x7f8d30158470 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.839+0000 7f8d26a00640 1 -- 172.21.15.67:0/373322734 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c007070 con 0x7f8d30158470 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.839+0000 7f8d2fe00640 1 -- 172.21.15.67:0/373322734 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d30156240 con 0x7f8d30158470 2024-03-09T08:59:27.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.840+0000 7f8d2fe00640 1 -- 172.21.15.67:0/373322734 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d30156800 con 0x7f8d30158470 2024-03-09T08:59:27.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.841+0000 7f8d2fe00640 1 -- 172.21.15.67:0/373322734 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3004f3c0 con 0x7f8d30158470 2024-03-09T08:59:27.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.842+0000 7f8d26a00640 1 -- 172.21.15.67:0/373322734 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8d1c047020 con 0x7f8d30158470 2024-03-09T08:59:27.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.843+0000 7f8d26a00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d1006b520 0x7f8d1006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:27.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.843+0000 7f8d26a00640 1 -- 172.21.15.67:0/373322734 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8d1c0abe60 con 0x7f8d30158470 2024-03-09T08:59:27.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.843+0000 7f8d2ea00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d1006b520 0x7f8d1006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:27.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.844+0000 7f8d2ea00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d1006b520 0x7f8d1006d9e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8d20001010 tx=0x7f8d20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:27.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:26.849+0000 7f8d26a00640 1 -- 172.21.15.67:0/373322734 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8d1c07a9d0 con 0x7f8d30158470 2024-03-09T08:59:27.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.149+0000 7f8d2fe00640 1 -- 172.21.15.67:0/373322734 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8d3006c980 con 0x7f8d30158470 2024-03-09T08:59:27.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.149+0000 7f8d26a00640 1 -- 172.21.15.67:0/373322734 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f8d1c07a2d0 con 0x7f8d30158470 2024-03-09T08:59:27.150 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":54,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:59:27.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.152+0000 7f8d2fe00640 1 -- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d1006b520 msgr2=0x7f8d1006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:27.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.152+0000 7f8d2fe00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d1006b520 0x7f8d1006d9e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8d20001010 tx=0x7f8d20009040 comp rx=0 tx=0).stop 2024-03-09T08:59:27.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.152+0000 7f8d2fe00640 1 -- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d30158470 msgr2=0x7f8d301eedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:27.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.152+0000 7f8d2fe00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d30158470 0x7f8d301eedb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c2a0 tx=0x7f8d1c02e070 comp rx=0 tx=0).stop 2024-03-09T08:59:27.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.152+0000 7f8d2fe00640 1 -- 172.21.15.67:0/373322734 shutdown_connections 2024-03-09T08:59:27.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.152+0000 7f8d2fe00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d1006b520 0x7f8d1006d9e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:27.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.152+0000 7f8d2fe00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d301ef2f0 0x7f8d301eb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:27.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.152+0000 7f8d2fe00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d30158470 0x7f8d301eedb0 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c2a0 tx=0x7f8d1c02e070 comp rx=0 tx=0).stop 2024-03-09T08:59:27.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.152+0000 7f8d2fe00640 1 --2- 172.21.15.67:0/373322734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d140037a0 0x7f8d30154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:27.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.152+0000 7f8d2fe00640 1 -- 172.21.15.67:0/373322734 >> 172.21.15.67:0/373322734 conn(0x7f8d300c7fb0 msgr2=0x7f8d300ca5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:27.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.153+0000 7f8d2fe00640 1 -- 172.21.15.67:0/373322734 shutdown_connections 2024-03-09T08:59:27.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:27.153+0000 7f8d2fe00640 1 -- 172.21.15.67:0/373322734 wait complete. 2024-03-09T08:59:30.178 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:59:30.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.412+0000 7f6a43200640 1 Processor -- start 2024-03-09T08:59:30.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.412+0000 7f6a43200640 1 -- start start 2024-03-09T08:59:30.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.414+0000 7f6a43200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:30.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.414+0000 7f6a43200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:30.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.414+0000 7f6a43200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c06d920 con 0x7f6a3c0c3080 2024-03-09T08:59:30.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.414+0000 7f6a43200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3c155ba0 con 0x7f6a3c06d420 2024-03-09T08:59:30.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.414+0000 7f6a43200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c06dc00 con 0x7f6a3c14c9f0 2024-03-09T08:59:30.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.414+0000 7f6a41400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:30.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.414+0000 7f6a41400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52794/0 (socket says 172.21.15.67:52794) 2024-03-09T08:59:30.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.414+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 learned_addr learned my addr 172.21.15.67:0/4103652000 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:30.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.414+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:30.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.415+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 msgr2=0x7f6a3c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:59:30.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.415+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 msgr2=0x7f6a3c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:30.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.415+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:30.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.415+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:30.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.415+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:59:30.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.415+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:30.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.415+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:30.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.415+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:30.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.616+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:30.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.616+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:30.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.616+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 msgr2=0x7f6a3c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:59:30.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.616+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 msgr2=0x7f6a3c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:30.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.616+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:30.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.616+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:59:30.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.616+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:59:30.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.616+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:30.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.616+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:30.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:30.616+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:59:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.017+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.017+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.017+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 msgr2=0x7f6a3c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:59:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.017+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 msgr2=0x7f6a3c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.017+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:59:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.017+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.017+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:31.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.017+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.017+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:59:31.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.017+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:59:31.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.818+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:31.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.818+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:31.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.818+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:59:31.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.818+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:31.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.818+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:31.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.818+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:59:31.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.818+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 msgr2=0x7f6a3c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T08:59:31.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.818+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 msgr2=0x7f6a3c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:31.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.818+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:31.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:31.818+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T08:59:33.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.415+0000 7f6a37400640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1540a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:33.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.415+0000 7f6a37400640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1540a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:33.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.415+0000 7f6a37400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 msgr2=0x7f6a3c1527d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.415+0000 7f6a37400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.134:3300/0 conn(0x7f6a3c06d420 0x7f6a3c1527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.415+0000 7f6a37400640 1 -- 172.21.15.67:0/4103652000 >> v1:172.21.15.134:6789/0 conn(0x7f6a3c0c3080 legacy=0x7f6a3c14c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.415+0000 7f6a37400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 0x7f6a20009850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.415+0000 7f6a37400640 1 -- 172.21.15.67:0/4103652000 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c06dee0 con 0x7f6a200036e0 2024-03-09T08:59:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.415+0000 7f6a37400640 1 -- 172.21.15.67:0/4103652000 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3c155ba0 con 0x7f6a20007390 2024-03-09T08:59:33.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.416+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 0x7f6a20009850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:33.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.416+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 msgr2=0x7f6a20009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:59:33.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.416+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 msgr2=0x7f6a20009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:33.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.416+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 0x7f6a20009850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:33.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.416+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 0x7f6a20009850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.617+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 0x7f6a20009850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.617+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 msgr2=0x7f6a20009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:59:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.617+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 msgr2=0x7f6a20009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.617+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 0x7f6a20009850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:33.617+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 0x7f6a20009850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:59:34.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.018+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 0x7f6a20009850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:34.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.018+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 msgr2=0x7f6a20009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T08:59:34.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.018+0000 7f6a41400640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 msgr2=0x7f6a20009850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T08:59:34.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.018+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 0x7f6a20009850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:34.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.018+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 0x7f6a20009850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T08:59:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.815+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178473757 0 0) 0x7f6a3c06dee0 con 0x7f6a200036e0 2024-03-09T08:59:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.815+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a24003520 con 0x7f6a200036e0 2024-03-09T08:59:34.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 197165107 0 0) 0x7f6a24003520 con 0x7f6a200036e0 2024-03-09T08:59:34.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 msgr2=0x7f6a20009850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:34.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 --2- 172.21.15.67:0/4103652000 >> v2:172.21.15.67:3300/0 conn(0x7f6a20007390 0x7f6a20009850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:34.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2000afc0 con 0x7f6a200036e0 2024-03-09T08:59:34.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f6a300016f0 con 0x7f6a200036e0 2024-03-09T08:59:34.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 >> v1:172.21.15.67:6789/0 conn(0x7f6a200036e0 legacy=0x7f6a20005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24003a20 0x7f6a24003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a24004800 0x7f6a24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c155ba0 con 0x7f6a24004800 2024-03-09T08:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24005600 con 0x7f6a24003a20 2024-03-09T08:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.817+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2400bf00 con 0x7f6a3c14c9f0 2024-03-09T08:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.818+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 reap_dead start 2024-03-09T08:59:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.818+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.818+0000 7f6a41e00640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24003a20 0x7f6a24003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.819+0000 7f6a41400640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a24004800 0x7f6a24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.819+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24003a20 msgr2=0x7f6a24003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.819+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24003a20 0x7f6a24003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.819+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a24004800 msgr2=0x7f6a24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.819+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a24004800 0x7f6a24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.819+0000 7f6a42800640 1 -- 172.21.15.67:0/4103652000 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2000afc0 con 0x7f6a3c14c9f0 2024-03-09T08:59:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.821+0000 7f6a42800640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a2400b9c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0c1bd0 tx=0x7f6a24005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a255304004536634 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.821+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3803c070 con 0x7f6a3c14c9f0 2024-03-09T08:59:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.821+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3800e450 con 0x7f6a3c14c9f0 2024-03-09T08:59:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.821+0000 7f6a40a00640 1 -- 172.21.15.67:0/4103652000 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38004040 con 0x7f6a3c14c9f0 2024-03-09T08:59:34.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.822+0000 7f6a43200640 1 -- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a2400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:34.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.822+0000 7f6a43200640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a2400b9c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0c1bd0 tx=0x7f6a24005830 comp rx=0 tx=0).stop 2024-03-09T08:59:34.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.822+0000 7f6a43200640 1 -- 172.21.15.67:0/4103652000 shutdown_connections 2024-03-09T08:59:34.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.822+0000 7f6a43200640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a2400b9c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:34.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.822+0000 7f6a43200640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a24004800 0x7f6a24004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:34.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.822+0000 7f6a43200640 1 --2- 172.21.15.67:0/4103652000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24003a20 0x7f6a24003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:34.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.822+0000 7f6a43200640 1 -- 172.21.15.67:0/4103652000 >> 172.21.15.67:0/4103652000 conn(0x7f6a3c0c7300 msgr2=0x7f6a3c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:34.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.823+0000 7f6a43200640 1 -- 172.21.15.67:0/4103652000 shutdown_connections 2024-03-09T08:59:34.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.823+0000 7f6a43200640 1 -- 172.21.15.67:0/4103652000 wait complete. 2024-03-09T08:59:34.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.826+0000 7f6a43200640 1 Processor -- start 2024-03-09T08:59:34.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.826+0000 7f6a43200640 1 -- start start 2024-03-09T08:59:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.828+0000 7f6a43200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a24003a20 0x7f6a3c147060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.828+0000 7f6a43200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24004800 0x7f6a3c0ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.828+0000 7f6a43200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.828+0000 7f6a43200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c0c8030 con 0x7f6a24003a20 2024-03-09T08:59:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.828+0000 7f6a43200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c0c3460 con 0x7f6a24004800 2024-03-09T08:59:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.828+0000 7f6a41400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24004800 0x7f6a3c0ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.828+0000 7f6a41400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24004800 0x7f6a3c0ab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52852/0 (socket says 172.21.15.67:52852) 2024-03-09T08:59:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.828+0000 7f6a41400640 1 -- 172.21.15.67:0/2844629754 learned_addr learned my addr 172.21.15.67:0/2844629754 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.828+0000 7f6a43200640 1 -- 172.21.15.67:0/2844629754 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c1e70c0 con 0x7f6a3c14c9f0 2024-03-09T08:59:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.829+0000 7f6a41e00640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a24003a20 0x7f6a3c147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.829+0000 7f6a42800640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.829+0000 7f6a42800640 1 -- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T08:59:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.829+0000 7f6a42800640 1 -- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T08:59:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.829+0000 7f6a41400640 1 -- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 msgr2=0x7f6a3c1e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.829+0000 7f6a42800640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1e75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T08:59:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.830+0000 7f6a42800640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1e75c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.830+0000 7f6a41400640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1e75c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.830+0000 7f6a41400640 1 -- 172.21.15.67:0/2844629754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a24003a20 msgr2=0x7f6a3c147060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.830+0000 7f6a41400640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a24003a20 0x7f6a3c147060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.830+0000 7f6a41400640 1 -- 172.21.15.67:0/2844629754 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c0acf00 con 0x7f6a24004800 2024-03-09T08:59:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.830+0000 7f6a41400640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24004800 0x7f6a3c0ab740 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c004650 tx=0x7f6a2c017e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.832+0000 7f6a36000640 1 -- 172.21.15.67:0/2844629754 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c027070 con 0x7f6a24004800 2024-03-09T08:59:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.832+0000 7f6a36000640 1 -- 172.21.15.67:0/2844629754 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c016dc0 con 0x7f6a24004800 2024-03-09T08:59:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.832+0000 7f6a36000640 1 -- 172.21.15.67:0/2844629754 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00f040 con 0x7f6a24004800 2024-03-09T08:59:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.832+0000 7f6a43200640 1 -- 172.21.15.67:0/2844629754 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c0c63f0 con 0x7f6a24004800 2024-03-09T08:59:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.832+0000 7f6a43200640 1 -- 172.21.15.67:0/2844629754 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c0aa3b0 con 0x7f6a24004800 2024-03-09T08:59:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.834+0000 7f6a43200640 1 -- 172.21.15.67:0/2844629754 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3c04f3c0 con 0x7f6a24004800 2024-03-09T08:59:34.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.837+0000 7f6a36000640 1 -- 172.21.15.67:0/2844629754 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6a2c022630 con 0x7f6a24004800 2024-03-09T08:59:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.838+0000 7f6a36000640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a2006c000 0x7f6a2006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.838+0000 7f6a36000640 1 -- 172.21.15.67:0/2844629754 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6a2c08ba30 con 0x7f6a24004800 2024-03-09T08:59:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.839+0000 7f6a41e00640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a2006c000 0x7f6a2006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:34.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.840+0000 7f6a41e00640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a2006c000 0x7f6a2006e4c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6a30007bf0 tx=0x7f6a30009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:34.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:34.846+0000 7f6a36000640 1 -- 172.21.15.67:0/2844629754 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6a2c01f070 con 0x7f6a24004800 2024-03-09T08:59:35.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.377+0000 7f6a43200640 1 -- 172.21.15.67:0/2844629754 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6a3c0c1ae0 con 0x7f6a24004800 2024-03-09T08:59:35.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.378+0000 7f6a36000640 1 -- 172.21.15.67:0/2844629754 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7f6a2c059e50 con 0x7f6a24004800 2024-03-09T08:59:35.378 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":58,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:59:35.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 -- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a2006c000 msgr2=0x7f6a2006e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:35.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a2006c000 0x7f6a2006e4c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6a30007bf0 tx=0x7f6a30009040 comp rx=0 tx=0).stop 2024-03-09T08:59:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 -- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24004800 msgr2=0x7f6a3c0ab740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24004800 0x7f6a3c0ab740 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c004650 tx=0x7f6a2c017e20 comp rx=0 tx=0).stop 2024-03-09T08:59:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 -- 172.21.15.67:0/2844629754 shutdown_connections 2024-03-09T08:59:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a2006c000 0x7f6a2006e4c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a3c14c9f0 0x7f6a3c1e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a24004800 0x7f6a3c0ab740 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 --2- 172.21.15.67:0/2844629754 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a24003a20 0x7f6a3c147060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 -- 172.21.15.67:0/2844629754 >> 172.21.15.67:0/2844629754 conn(0x7f6a3c0c7300 msgr2=0x7f6a3c152650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 -- 172.21.15.67:0/2844629754 shutdown_connections 2024-03-09T08:59:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:35.381+0000 7f6a43200640 1 -- 172.21.15.67:0/2844629754 wait complete. 2024-03-09T08:59:38.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:59:38.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.692+0000 7fd512400640 1 Processor -- start 2024-03-09T08:59:38.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.692+0000 7fd512400640 1 -- start start 2024-03-09T08:59:38.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.693+0000 7fd512400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0acd90 0x7fd50c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.693+0000 7fd512400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd50c0c1d00 0x7fd50c0aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.693+0000 7fd512400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c156b30 con 0x7fd50c0aa8f0 2024-03-09T08:59:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.693+0000 7fd512400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c1592e0 con 0x7fd50c0c1d00 2024-03-09T08:59:38.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.693+0000 7fd512400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c06de60 con 0x7fd50c0acd90 2024-03-09T08:59:38.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.694+0000 7fd511000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0acd90 0x7fd50c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:38.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.694+0000 7fd511a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd50c0aa8f0 0x7fd50c159770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39250/0 (socket says 172.21.15.67:39250) 2024-03-09T08:59:38.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.694+0000 7fd511a00640 1 -- 172.21.15.67:0/2672587864 learned_addr learned my addr 172.21.15.67:0/2672587864 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:59:38.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.694+0000 7fd50be00640 1 --2- 172.21.15.67:0/2672587864 >> v2:172.21.15.134:3300/0 conn(0x7fd50c0c1d00 0x7fd50c0aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.694+0000 7fd511000640 1 -- 172.21.15.67:0/2672587864 >> v2:172.21.15.134:3300/0 conn(0x7fd50c0c1d00 msgr2=0x7fd50c0aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.694+0000 7fd511000640 1 --2- 172.21.15.67:0/2672587864 >> v2:172.21.15.134:3300/0 conn(0x7fd50c0c1d00 0x7fd50c0aa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.694+0000 7fd511000640 1 -- 172.21.15.67:0/2672587864 >> v1:172.21.15.67:6789/0 conn(0x7fd50c0aa8f0 legacy=0x7fd50c159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.694+0000 7fd511000640 1 -- 172.21.15.67:0/2672587864 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c06d5c0 con 0x7fd50c0acd90 2024-03-09T08:59:38.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.696+0000 7fd511000640 1 --2- 172.21.15.67:0/2672587864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0acd90 0x7fd50c1565f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5000097a0 tx=0x7fd50002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce0dabae8ed14fca server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:38.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.696+0000 7fd50b400640 1 -- 172.21.15.67:0/2672587864 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500035020 con 0x7fd50c0acd90 2024-03-09T08:59:38.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.696+0000 7fd50b400640 1 -- 172.21.15.67:0/2672587864 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd500031810 con 0x7fd50c0acd90 2024-03-09T08:59:38.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.696+0000 7fd50b400640 1 -- 172.21.15.67:0/2672587864 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500031be0 con 0x7fd50c0acd90 2024-03-09T08:59:38.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.697+0000 7fd512400640 1 -- 172.21.15.67:0/2672587864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0acd90 msgr2=0x7fd50c1565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:38.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.697+0000 7fd512400640 1 --2- 172.21.15.67:0/2672587864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0acd90 0x7fd50c1565f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5000097a0 tx=0x7fd50002b8b0 comp rx=0 tx=0).stop 2024-03-09T08:59:38.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.697+0000 7fd512400640 1 -- 172.21.15.67:0/2672587864 shutdown_connections 2024-03-09T08:59:38.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.697+0000 7fd512400640 1 --2- 172.21.15.67:0/2672587864 >> v2:172.21.15.134:3300/0 conn(0x7fd50c0c1d00 0x7fd50c0aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:38.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.697+0000 7fd512400640 1 --2- 172.21.15.67:0/2672587864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0acd90 0x7fd50c1565f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:38.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.697+0000 7fd512400640 1 -- 172.21.15.67:0/2672587864 >> 172.21.15.67:0/2672587864 conn(0x7fd50c0c7300 msgr2=0x7fd50c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:38.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.698+0000 7fd512400640 1 -- 172.21.15.67:0/2672587864 shutdown_connections 2024-03-09T08:59:38.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.698+0000 7fd512400640 1 -- 172.21.15.67:0/2672587864 wait complete. 2024-03-09T08:59:38.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.700+0000 7fd512400640 1 Processor -- start 2024-03-09T08:59:38.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.700+0000 7fd512400640 1 -- start start 2024-03-09T08:59:38.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.701+0000 7fd512400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd50c0aa8f0 0x7fd50c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:38.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.701+0000 7fd512400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd50c0acd90 0x7fd50c14c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:38.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.701+0000 7fd512400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0c1d00 0x7fd50c148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:38.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.701+0000 7fd512400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0ad170 con 0x7fd50c0aa8f0 2024-03-09T08:59:38.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.701+0000 7fd512400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c06db80 con 0x7fd50c0acd90 2024-03-09T08:59:38.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.701+0000 7fd512400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0ad2f0 con 0x7fd50c0c1d00 2024-03-09T08:59:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd511000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd50c0aa8f0 0x7fd50c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd50be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd50c0acd90 0x7fd50c14c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd50be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd50c0acd90 0x7fd50c14c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52860/0 (socket says 172.21.15.67:52860) 2024-03-09T08:59:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd50be00640 1 -- 172.21.15.67:0/3352318611 learned_addr learned my addr 172.21.15.67:0/3352318611 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd511a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0c1d00 0x7fd50c148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd511000640 1 -- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0c1d00 msgr2=0x7fd50c148980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd511000640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0c1d00 0x7fd50c148980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd511000640 1 -- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd50c0acd90 msgr2=0x7fd50c14c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd511000640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd50c0acd90 0x7fd50c14c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:38.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd511000640 1 -- 172.21.15.67:0/3352318611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c071a90 con 0x7fd50c0aa8f0 2024-03-09T08:59:38.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd50be00640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd50c0acd90 0x7fd50c14c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:59:38.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.702+0000 7fd511a00640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0c1d00 0x7fd50c148980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:59:38.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.703+0000 7fd511000640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd50c0aa8f0 0x7fd50c0c6830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd500033000 tx=0x7fd5000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:38.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.703+0000 7fd508c00640 1 -- 172.21.15.67:0/3352318611 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500041070 con 0x7fd50c0aa8f0 2024-03-09T08:59:38.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.703+0000 7fd508c00640 1 -- 172.21.15.67:0/3352318611 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50003ad00 con 0x7fd50c0aa8f0 2024-03-09T08:59:38.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.703+0000 7fd508c00640 1 -- 172.21.15.67:0/3352318611 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500035040 con 0x7fd50c0aa8f0 2024-03-09T08:59:38.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.704+0000 7fd512400640 1 -- 172.21.15.67:0/3352318611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c0c36d0 con 0x7fd50c0aa8f0 2024-03-09T08:59:38.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.704+0000 7fd512400640 1 -- 172.21.15.67:0/3352318611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c071fe0 con 0x7fd50c0aa8f0 2024-03-09T08:59:38.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.706+0000 7fd512400640 1 -- 172.21.15.67:0/3352318611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50c04f3c0 con 0x7fd50c0aa8f0 2024-03-09T08:59:38.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.707+0000 7fd508c00640 1 -- 172.21.15.67:0/3352318611 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd50004c060 con 0x7fd50c0aa8f0 2024-03-09T08:59:38.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.709+0000 7fd508c00640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd4ec06b520 0x7fd4ec06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:38.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.709+0000 7fd508c00640 1 -- 172.21.15.67:0/3352318611 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd5000ac0a0 con 0x7fd50c0aa8f0 2024-03-09T08:59:38.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.709+0000 7fd50be00640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd4ec06b520 0x7fd4ec06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:38.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.710+0000 7fd50be00640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd4ec06b520 0x7fd4ec06d9e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0c2680 tx=0x7fd4f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:38.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:38.714+0000 7fd508c00640 1 -- 172.21.15.67:0/3352318611 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd50007a4c0 con 0x7fd50c0aa8f0 2024-03-09T08:59:39.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.168+0000 7fd512400640 1 -- 172.21.15.67:0/3352318611 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd50c0c23d0 con 0x7fd50c0aa8f0 2024-03-09T08:59:39.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.168+0000 7fd508c00640 1 -- 172.21.15.67:0/3352318611 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7fd50007e560 con 0x7fd50c0aa8f0 2024-03-09T08:59:39.170 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:59:39.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.171+0000 7fd512400640 1 -- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd4ec06b520 msgr2=0x7fd4ec06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.171+0000 7fd512400640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd4ec06b520 0x7fd4ec06d9e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0c2680 tx=0x7fd4f8008040 comp rx=0 tx=0).stop 2024-03-09T08:59:39.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.171+0000 7fd512400640 1 -- 172.21.15.67:0/3352318611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd50c0aa8f0 msgr2=0x7fd50c0c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.172+0000 7fd512400640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd50c0aa8f0 0x7fd50c0c6830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd500033000 tx=0x7fd5000046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.172+0000 7fd512400640 1 -- 172.21.15.67:0/3352318611 shutdown_connections 2024-03-09T08:59:39.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.172+0000 7fd512400640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd4ec06b520 0x7fd4ec06d9e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.172+0000 7fd512400640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd50c0c1d00 0x7fd50c148980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.172+0000 7fd512400640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd50c0acd90 0x7fd50c14c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.172+0000 7fd512400640 1 --2- 172.21.15.67:0/3352318611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd50c0aa8f0 0x7fd50c0c6830 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.172+0000 7fd512400640 1 -- 172.21.15.67:0/3352318611 >> 172.21.15.67:0/3352318611 conn(0x7fd50c0c7300 msgr2=0x7fd50c159770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.172+0000 7fd512400640 1 -- 172.21.15.67:0/3352318611 shutdown_connections 2024-03-09T08:59:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.172+0000 7fd512400640 1 -- 172.21.15.67:0/3352318611 wait complete. 2024-03-09T08:59:39.197 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:59:39.197 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:59:39.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:59:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.394+0000 7f5eb7e00640 1 Processor -- start 2024-03-09T08:59:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.395+0000 7f5eb7e00640 1 -- start start 2024-03-09T08:59:39.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.396+0000 7f5eb7e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5eb8153350 0x7f5eb8070030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.396+0000 7f5eb7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5eb80c3590 0x7f5eb80c9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.396+0000 7f5eb7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb80c5470 con 0x7f5eb80c9400 2024-03-09T08:59:39.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.396+0000 7f5eb7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb80709f0 con 0x7f5eb80c3590 2024-03-09T08:59:39.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.396+0000 7f5eb7e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb80c4fe0 con 0x7f5eb8153350 2024-03-09T08:59:39.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.396+0000 7f5eb7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5eb80c3590 0x7f5eb80c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.396+0000 7f5eb7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5eb80c3590 0x7f5eb80c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38682/0 (socket says 172.21.15.67:38682) 2024-03-09T08:59:39.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.397+0000 7f5eb7400640 1 -- 172.21.15.67:0/771103234 learned_addr learned my addr 172.21.15.67:0/771103234 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:39.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.397+0000 7f5eb6a00640 1 --2- 172.21.15.67:0/771103234 >> v2:172.21.15.134:3300/0 conn(0x7f5eb8153350 0x7f5eb8070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.397+0000 7f5eb7400640 1 -- 172.21.15.67:0/771103234 >> v2:172.21.15.134:3300/0 conn(0x7f5eb8153350 msgr2=0x7f5eb8070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.397+0000 7f5eb7400640 1 --2- 172.21.15.67:0/771103234 >> v2:172.21.15.134:3300/0 conn(0x7f5eb8153350 0x7f5eb8070030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.397+0000 7f5eb7400640 1 -- 172.21.15.67:0/771103234 >> v1:172.21.15.67:6789/0 conn(0x7f5eb80c9400 legacy=0x7f5eb80c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.397+0000 7f5eb7400640 1 -- 172.21.15.67:0/771103234 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb8070710 con 0x7f5eb80c3590 2024-03-09T08:59:39.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.399+0000 7f5eb7400640 1 --2- 172.21.15.67:0/771103234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c3590 0x7f5eb80c9880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80c9f70 tx=0x7f5eb002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7103f3a5b0d3da9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.399+0000 7f5eb5600640 1 -- 172.21.15.67:0/771103234 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb002c8d0 con 0x7f5eb80c3590 2024-03-09T08:59:39.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.399+0000 7f5eb5600640 1 -- 172.21.15.67:0/771103234 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb002ca70 con 0x7f5eb80c3590 2024-03-09T08:59:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.399+0000 7f5eb5600640 1 -- 172.21.15.67:0/771103234 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb002ce40 con 0x7f5eb80c3590 2024-03-09T08:59:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.400+0000 7f5eb7e00640 1 -- 172.21.15.67:0/771103234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c3590 msgr2=0x7f5eb80c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.400+0000 7f5eb7e00640 1 --2- 172.21.15.67:0/771103234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c3590 0x7f5eb80c9880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80c9f70 tx=0x7f5eb002bdb0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.400+0000 7f5eb7e00640 1 -- 172.21.15.67:0/771103234 shutdown_connections 2024-03-09T08:59:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.400+0000 7f5eb7e00640 1 --2- 172.21.15.67:0/771103234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c3590 0x7f5eb80c9880 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.400+0000 7f5eb7e00640 1 --2- 172.21.15.67:0/771103234 >> v2:172.21.15.134:3300/0 conn(0x7f5eb8153350 0x7f5eb8070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.400+0000 7f5eb7e00640 1 -- 172.21.15.67:0/771103234 >> 172.21.15.67:0/771103234 conn(0x7f5eb80aa890 msgr2=0x7f5eb8066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.400+0000 7f5eb7e00640 1 -- 172.21.15.67:0/771103234 shutdown_connections 2024-03-09T08:59:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.400+0000 7f5eb7e00640 1 -- 172.21.15.67:0/771103234 wait complete. 2024-03-09T08:59:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.403+0000 7f5eb7e00640 1 Processor -- start 2024-03-09T08:59:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.403+0000 7f5eb7e00640 1 -- start start 2024-03-09T08:59:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.404+0000 7f5eb7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c3590 0x7f5eb81ef000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.404+0000 7f5eb7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eb80c9400 0x7f5eb814fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.404+0000 7f5eb7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5eb8153350 0x7f5eb8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.404+0000 7f5eb7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb8073270 con 0x7f5eb80c3590 2024-03-09T08:59:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.404+0000 7f5eb7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80c4c70 con 0x7f5eb80c9400 2024-03-09T08:59:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.404+0000 7f5eb7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80c28a0 con 0x7f5eb8153350 2024-03-09T08:59:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.405+0000 7f5eb6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eb80c9400 0x7f5eb814fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.405+0000 7f5eb6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c3590 0x7f5eb81ef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.405+0000 7f5eb6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eb80c9400 0x7f5eb814fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:44918/0 (socket says 172.21.15.67:44918) 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.405+0000 7f5eb6000640 1 -- 172.21.15.67:0/130978167 learned_addr learned my addr 172.21.15.67:0/130978167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.405+0000 7f5eb7400640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5eb8153350 0x7f5eb8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.405+0000 7f5eb7400640 1 -- 172.21.15.67:0/130978167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eb80c9400 msgr2=0x7f5eb814fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.405+0000 7f5eb7400640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5eb80c9400 0x7f5eb814fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.405+0000 7f5eb7400640 1 -- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c3590 msgr2=0x7f5eb81ef000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.405+0000 7f5eb7400640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c3590 0x7f5eb81ef000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.405+0000 7f5eb7400640 1 -- 172.21.15.67:0/130978167 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb81499e0 con 0x7f5eb8153350 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.405+0000 7f5eb6a00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c3590 0x7f5eb81ef000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.406+0000 7f5eb7400640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5eb8153350 0x7f5eb8150280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0032000 tx=0x7f5eb0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.406+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0050070 con 0x7f5eb8153350 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.406+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb00079a0 con 0x7f5eb8153350 2024-03-09T08:59:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.406+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0009510 con 0x7f5eb8153350 2024-03-09T08:59:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.406+0000 7f5eb7e00640 1 -- 172.21.15.67:0/130978167 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb81ef940 con 0x7f5eb8153350 2024-03-09T08:59:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.407+0000 7f5eb7e00640 1 -- 172.21.15.67:0/130978167 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb814d5f0 con 0x7f5eb8153350 2024-03-09T08:59:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.408+0000 7f5eb7e00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb81ef540 0x7f5eb8145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.408+0000 7f5eb7e00640 1 -- 172.21.15.67:0/130978167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb804f3c0 con 0x7f5eb81ef540 2024-03-09T08:59:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.408+0000 7f5eb6a00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb81ef540 0x7f5eb8145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.409+0000 7f5eb6a00640 1 -- 172.21.15.67:0/130978167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5eb814c7e0 con 0x7f5eb81ef540 2024-03-09T08:59:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.409+0000 7f5eb6a00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb81ef540 0x7f5eb8145b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8011d20 tx=0x7f5ea800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.410+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5eb00343f0 con 0x7f5eb8153350 2024-03-09T08:59:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.411+0000 7f5eaea00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5e9806b4d0 0x7f5e9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.411+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8019070 con 0x7f5eb81ef540 2024-03-09T08:59:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.411+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f5eb00b4670 con 0x7f5eb8153350 2024-03-09T08:59:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.411+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f5ea8004030 con 0x7f5eb81ef540 2024-03-09T08:59:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.411+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb81ef540 msgr2=0x7f5eb8145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.411+0000 7f5eaea00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb81ef540 0x7f5eb8145b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8011d20 tx=0x7f5ea800f610 comp rx=0 tx=0).stop 2024-03-09T08:59:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.411+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb81ef540 msgr2=0x7f5eb8145b70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.411+0000 7f5eaea00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb81ef540 0x7f5eb8145b70 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8011d20 tx=0x7f5ea800f610 comp rx=0 tx=0).stop 2024-03-09T08:59:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.411+0000 7f5eb6000640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5e9806b4d0 0x7f5e9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.412+0000 7f5eb6000640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5e9806b4d0 0x7f5e9806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80af560 tx=0x7f5ea4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.418+0000 7f5eb7e00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c7060 0x7f5eb81460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.418+0000 7f5eb7e00640 1 -- 172.21.15.67:0/130978167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb8067560 con 0x7f5eb80c7060 2024-03-09T08:59:39.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.419+0000 7f5eb6a00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c7060 0x7f5eb81460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.419+0000 7f5eb6a00640 1 -- 172.21.15.67:0/130978167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5eb8067280 con 0x7f5eb80c7060 2024-03-09T08:59:39.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.420+0000 7f5eb6a00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c7060 0x7f5eb81460b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8011d20 tx=0x7f5ea80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.421+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8019040 con 0x7f5eb80c7060 2024-03-09T08:59:39.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.421+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5018 (secure 0 0 0) 0x7f5ea801e660 con 0x7f5eb80c7060 2024-03-09T08:59:39.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.421+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c7060 msgr2=0x7f5eb81460b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.421+0000 7f5eaea00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c7060 0x7f5eb81460b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8011d20 tx=0x7f5ea80046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.421+0000 7f5eaea00640 1 -- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c7060 msgr2=0x7f5eb81460b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:39.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.421+0000 7f5eaea00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eb80c7060 0x7f5eb81460b0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8011d20 tx=0x7f5ea80046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.422 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:59:39.422 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:59:39.422 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:59:39.422 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:59:39.422 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 60, 2024-03-09T08:59:39.423 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:59:39.423 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:59:39.423 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:59:39.423 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:59:39.423 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.423 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T08:59:39.423 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:39.423 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:59:39.423 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.424 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:59:39.425 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:59:39.425 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:39.425 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:39.425 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:39.425 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:39.425 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:39.425 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:39.425 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:39.425 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:39.425 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:39.425 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:59:39.426 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:59:39.427 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:59:39.427 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:59:39.427 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:59:39.427 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:59:39.427 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:59:39.427 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:39.427 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:59:39.427 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:39.427 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:59:39.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.429 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:39.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:59:39.431 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:59:39.432 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.433 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:39.433 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:59:39.433 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.433 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.433 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.433 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:59:39.433 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:59:39.433 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:39.434 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:39.434 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:39.434 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.434 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.434 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:59:39.434 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:59:39.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.434 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:39.434 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:39.435 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.436 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.437 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:59:39.437 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:59:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.422+0000 7f5eb7e00640 1 -- 172.21.15.67:0/130978167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5e9806b4d0 msgr2=0x7f5e9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.422+0000 7f5eb7e00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5e9806b4d0 0x7f5e9806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80af560 tx=0x7f5ea4009040 comp rx=0 tx=0).stop 2024-03-09T08:59:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.422+0000 7f5eb7e00640 1 -- 172.21.15.67:0/130978167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5eb8153350 msgr2=0x7f5eb8150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.422+0000 7f5eb7e00640 1 --2- 172.21.15.67:0/130978167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5eb8153350 0x7f5eb8150280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0032000 tx=0x7f5eb0004a40 comp rx=0 tx=0).stop 2024-03-09T08:59:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.422+0000 7f5eb7400640 1 -- 172.21.15.67:0/130978167 reap_dead start 2024-03-09T08:59:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.422+0000 7f5eb7400640 1 -- 172.21.15.67:0/130978167 reap_dead start 2024-03-09T08:59:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.422+0000 7f5eb7e00640 1 -- 172.21.15.67:0/130978167 shutdown_connections 2024-03-09T08:59:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.422+0000 7f5eb7e00640 1 -- 172.21.15.67:0/130978167 >> 172.21.15.67:0/130978167 conn(0x7f5eb80aa890 msgr2=0x7f5eb8072810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.422+0000 7f5eb7e00640 1 -- 172.21.15.67:0/130978167 shutdown_connections 2024-03-09T08:59:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.422+0000 7f5eb7e00640 1 -- 172.21.15.67:0/130978167 wait complete. 2024-03-09T08:59:39.439 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:59:39.439 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:59:39.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:59:39.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.611+0000 7fa77d200640 1 Processor -- start 2024-03-09T08:59:39.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.611+0000 7fa77d200640 1 -- start start 2024-03-09T08:59:39.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.613+0000 7fa77d200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa778151360 0x7fa7780c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.613+0000 7fa77d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7780c30a0 0x7fa7781539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.613+0000 7fa77d200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7780c2780 con 0x7fa7780ac930 2024-03-09T08:59:39.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.613+0000 7fa77d200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa778070670 con 0x7fa7780c30a0 2024-03-09T08:59:39.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.613+0000 7fa77d200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7780708c0 con 0x7fa778151360 2024-03-09T08:59:39.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.613+0000 7fa777e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7780c30a0 0x7fa7781539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.614+0000 7fa777400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fa7780ac930 0x7fa778150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:39716/0 (socket says 172.21.15.67:39716) 2024-03-09T08:59:39.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.614+0000 7fa777400640 1 -- 172.21.15.67:0/2909416496 learned_addr learned my addr 172.21.15.67:0/2909416496 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:59:39.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.614+0000 7fa776a00640 1 --2- 172.21.15.67:0/2909416496 >> v2:172.21.15.134:3300/0 conn(0x7fa778151360 0x7fa7780c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.614+0000 7fa776000640 1 -- 172.21.15.67:0/2909416496 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737932602 0 0) 0x7fa7780c2780 con 0x7fa7780ac930 2024-03-09T08:59:39.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.614+0000 7fa776000640 1 -- 172.21.15.67:0/2909416496 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa75c003520 con 0x7fa7780ac930 2024-03-09T08:59:39.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.614+0000 7fa777e00640 1 -- 172.21.15.67:0/2909416496 >> v2:172.21.15.134:3300/0 conn(0x7fa778151360 msgr2=0x7fa7780c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:39.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.614+0000 7fa777e00640 1 --2- 172.21.15.67:0/2909416496 >> v2:172.21.15.134:3300/0 conn(0x7fa778151360 0x7fa7780c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.614+0000 7fa777e00640 1 -- 172.21.15.67:0/2909416496 >> v1:172.21.15.134:6789/0 conn(0x7fa7780ac930 legacy=0x7fa778150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.614+0000 7fa777e00640 1 -- 172.21.15.67:0/2909416496 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778070ba0 con 0x7fa7780c30a0 2024-03-09T08:59:39.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.616+0000 7fa777e00640 1 --2- 172.21.15.67:0/2909416496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7780c30a0 0x7fa7781539e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa7780c8040 tx=0x7fa77002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6468fea2c35109 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.617+0000 7fa776000640 1 -- 172.21.15.67:0/2909416496 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa770039070 con 0x7fa7780c30a0 2024-03-09T08:59:39.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.617+0000 7fa776000640 1 -- 172.21.15.67:0/2909416496 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77002cc70 con 0x7fa7780c30a0 2024-03-09T08:59:39.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.617+0000 7fa776000640 1 -- 172.21.15.67:0/2909416496 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa770034660 con 0x7fa7780c30a0 2024-03-09T08:59:39.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.618+0000 7fa77d200640 1 -- 172.21.15.67:0/2909416496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7780c30a0 msgr2=0x7fa7781539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.618+0000 7fa77d200640 1 --2- 172.21.15.67:0/2909416496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7780c30a0 0x7fa7781539e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa7780c8040 tx=0x7fa77002bec0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.618+0000 7fa77d200640 1 -- 172.21.15.67:0/2909416496 shutdown_connections 2024-03-09T08:59:39.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.618+0000 7fa77d200640 1 --2- 172.21.15.67:0/2909416496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7780c30a0 0x7fa7781539e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.618+0000 7fa77d200640 1 --2- 172.21.15.67:0/2909416496 >> v2:172.21.15.134:3300/0 conn(0x7fa778151360 0x7fa7780c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.618+0000 7fa77d200640 1 -- 172.21.15.67:0/2909416496 >> 172.21.15.67:0/2909416496 conn(0x7fa7780c7220 msgr2=0x7fa7780c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:39.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.618+0000 7fa77d200640 1 -- 172.21.15.67:0/2909416496 shutdown_connections 2024-03-09T08:59:39.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.618+0000 7fa77d200640 1 -- 172.21.15.67:0/2909416496 wait complete. 2024-03-09T08:59:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.620+0000 7fa77d200640 1 Processor -- start 2024-03-09T08:59:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.621+0000 7fa77d200640 1 -- start start 2024-03-09T08:59:39.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.622+0000 7fa77d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7780ac930 0x7fa7781469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.622+0000 7fa77d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7780c30a0 0x7fa778149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.622+0000 7fa77d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa778151360 0x7fa7781ea290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.622+0000 7fa77d200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780ace90 con 0x7fa7780ac930 2024-03-09T08:59:39.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.622+0000 7fa77d200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780acd10 con 0x7fa7780c30a0 2024-03-09T08:59:39.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.622+0000 7fa77d200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa778153f20 con 0x7fa778151360 2024-03-09T08:59:39.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.623+0000 7fa777400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7780ac930 0x7fa7781469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.623+0000 7fa776a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7780c30a0 0x7fa778149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.623+0000 7fa777e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa778151360 0x7fa7781ea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.623+0000 7fa776a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7780c30a0 0x7fa778149d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:44928/0 (socket says 172.21.15.67:44928) 2024-03-09T08:59:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.623+0000 7fa776a00640 1 -- 172.21.15.67:0/3800916019 learned_addr learned my addr 172.21.15.67:0/3800916019 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.623+0000 7fa777400640 1 -- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa778151360 msgr2=0x7fa7781ea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.623+0000 7fa777400640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa778151360 0x7fa7781ea290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.623+0000 7fa777400640 1 -- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7780c30a0 msgr2=0x7fa778149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.623+0000 7fa777400640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7780c30a0 0x7fa778149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.623+0000 7fa777400640 1 -- 172.21.15.67:0/3800916019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77814de40 con 0x7fa7780ac930 2024-03-09T08:59:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.623+0000 7fa776a00640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7780c30a0 0x7fa778149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:59:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.624+0000 7fa777400640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7780ac930 0x7fa7781469f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa76800c430 tx=0x7fa76800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.625+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768017070 con 0x7fa7780ac930 2024-03-09T08:59:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.625+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa768005ce0 con 0x7fa7780ac930 2024-03-09T08:59:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.625+0000 7fa77d200640 1 -- 172.21.15.67:0/3800916019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77814e120 con 0x7fa7780ac930 2024-03-09T08:59:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.625+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680072d0 con 0x7fa7780ac930 2024-03-09T08:59:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.625+0000 7fa77d200640 1 -- 172.21.15.67:0/3800916019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77814fc00 con 0x7fa7780ac930 2024-03-09T08:59:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.627+0000 7fa77d200640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa77814a290 0x7fa7780c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.627+0000 7fa77d200640 1 -- 172.21.15.67:0/3800916019 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7781e9fb0 con 0x7fa77814a290 2024-03-09T08:59:39.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.628+0000 7fa776a00640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa77814a290 0x7fa7780c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.628+0000 7fa776a00640 1 -- 172.21.15.67:0/3800916019 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa77804f3c0 con 0x7fa77814a290 2024-03-09T08:59:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.628+0000 7fa776a00640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa77814a290 0x7fa7780c6920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7781e4880 tx=0x7fa764011c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.629+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa76801a070 con 0x7fa7780ac930 2024-03-09T08:59:39.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.630+0000 7fa76f400640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa75806b4d0 0x7fa75806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.630+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa7680898b0 con 0x7fa7780ac930 2024-03-09T08:59:39.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.631+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764004220 con 0x7fa77814a290 2024-03-09T08:59:39.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.631+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa7640043c0 con 0x7fa77814a290 2024-03-09T08:59:39.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.631+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa77814a290 msgr2=0x7fa7780c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.631+0000 7fa76f400640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa77814a290 0x7fa7780c6920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7781e4880 tx=0x7fa764011c90 comp rx=0 tx=0).stop 2024-03-09T08:59:39.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.631+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa77814a290 msgr2=0x7fa7780c6920 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:39.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.631+0000 7fa76f400640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa77814a290 0x7fa7780c6920 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7781e4880 tx=0x7fa764011c90 comp rx=0 tx=0).stop 2024-03-09T08:59:39.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.631+0000 7fa777e00640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa75806b4d0 0x7fa75806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.632+0000 7fa777e00640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa75806b4d0 0x7fa75806d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa770036000 tx=0x7fa770030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.645+0000 7fa77d200640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7781ea7d0 0x7fa77814e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.645+0000 7fa77d200640 1 -- 172.21.15.67:0/3800916019 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa778072e70 con 0x7fa7781ea7d0 2024-03-09T08:59:39.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.645+0000 7fa776a00640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7781ea7d0 0x7fa77814e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.646+0000 7fa776a00640 1 -- 172.21.15.67:0/3800916019 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa778067b30 con 0x7fa7781ea7d0 2024-03-09T08:59:39.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.646+0000 7fa776a00640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7781ea7d0 0x7fa77814e6e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa778147050 tx=0x7fa7640046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.646+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764004220 con 0x7fa7781ea7d0 2024-03-09T08:59:39.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.647+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5186 (secure 0 0 0) 0x7fa764003610 con 0x7fa7781ea7d0 2024-03-09T08:59:39.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.647+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7781ea7d0 msgr2=0x7fa77814e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.647+0000 7fa76f400640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7781ea7d0 0x7fa77814e6e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa778147050 tx=0x7fa7640046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.647+0000 7fa76f400640 1 -- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7781ea7d0 msgr2=0x7fa77814e6e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:39.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.647+0000 7fa76f400640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7781ea7d0 0x7fa77814e6e0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa778147050 tx=0x7fa7640046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.648 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:59:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:59:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:59:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:59:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 60, 2024-03-09T08:59:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:59:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:59:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:59:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:59:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T08:59:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:59:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:59:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:59:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:59:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:59:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:59:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:59:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:59:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:59:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:59:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:59:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:59:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:59:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:59:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:59:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:59:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:59:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:59:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:59:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:59:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:59:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:59:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:59:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:59:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:59:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:59:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:59:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:59:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:59:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.665 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.666 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:59:39.666 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:59:39.666 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.666 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:39.666 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.666 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:39.667 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.667 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.667 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:59:39.668 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.668 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:39.668 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.668 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:39.668 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.668 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.668 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:59:39.669 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.669 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:39.669 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.669 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:59:39.670 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.670 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.670 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:59:39.670 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.670 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:59:39.670 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.670 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:39.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.670 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.670 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.672 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:59:39.672 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:59:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.648+0000 7fa77d200640 1 -- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa75806b4d0 msgr2=0x7fa75806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.648+0000 7fa77d200640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa75806b4d0 0x7fa75806d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa770036000 tx=0x7fa770030000 comp rx=0 tx=0).stop 2024-03-09T08:59:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.648+0000 7fa77d200640 1 -- 172.21.15.67:0/3800916019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7780ac930 msgr2=0x7fa7781469f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.648+0000 7fa77d200640 1 --2- 172.21.15.67:0/3800916019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7780ac930 0x7fa7781469f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa76800c430 tx=0x7fa76800c8f0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.648+0000 7fa777e00640 1 -- 172.21.15.67:0/3800916019 reap_dead start 2024-03-09T08:59:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.648+0000 7fa777e00640 1 -- 172.21.15.67:0/3800916019 reap_dead start 2024-03-09T08:59:39.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.648+0000 7fa77d200640 1 -- 172.21.15.67:0/3800916019 shutdown_connections 2024-03-09T08:59:39.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.648+0000 7fa77d200640 1 -- 172.21.15.67:0/3800916019 >> 172.21.15.67:0/3800916019 conn(0x7fa7780c7220 msgr2=0x7fa778150a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:39.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.648+0000 7fa77d200640 1 -- 172.21.15.67:0/3800916019 shutdown_connections 2024-03-09T08:59:39.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.648+0000 7fa77d200640 1 -- 172.21.15.67:0/3800916019 wait complete. 2024-03-09T08:59:39.674 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:59:39.674 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:59:39.674 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:59:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.846+0000 7f4771600640 1 Processor -- start 2024-03-09T08:59:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.846+0000 7f4771600640 1 -- start start 2024-03-09T08:59:39.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.847+0000 7f4771600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f476c0b0db0 0x7f476c1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.847+0000 7f4771600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f476c0bec10 0x7f476c072770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.847+0000 7f4771600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f476c0c6010 con 0x7f476c0c9410 2024-03-09T08:59:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.847+0000 7f4771600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476c0c62f0 con 0x7f476c0bec10 2024-03-09T08:59:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.847+0000 7f4771600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476c0c65d0 con 0x7f476c0b0db0 2024-03-09T08:59:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.847+0000 7f476b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f476c0bec10 0x7f476c072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.847+0000 7f476be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f476c0b0db0 0x7f476c1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.847+0000 7f476b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f476c0bec10 0x7f476c072770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38724/0 (socket says 172.21.15.67:38724) 2024-03-09T08:59:39.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.847+0000 7f476b400640 1 -- 172.21.15.67:0/3607761816 learned_addr learned my addr 172.21.15.67:0/3607761816 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:39.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.848+0000 7f476be00640 1 -- 172.21.15.67:0/3607761816 >> v2:172.21.15.67:3300/0 conn(0x7f476c0bec10 msgr2=0x7f476c072770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:39.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.848+0000 7f476be00640 1 --2- 172.21.15.67:0/3607761816 >> v2:172.21.15.67:3300/0 conn(0x7f476c0bec10 0x7f476c072770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.848+0000 7f476be00640 1 -- 172.21.15.67:0/3607761816 >> v1:172.21.15.134:6789/0 conn(0x7f476c0c9410 legacy=0x7f476c06b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.848+0000 7f476be00640 1 -- 172.21.15.67:0/3607761816 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c06b040 con 0x7f476c0b0db0 2024-03-09T08:59:39.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.850+0000 7f476be00640 1 --2- 172.21.15.67:0/3607761816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f476c0b0db0 0x7f476c1565a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f476c0cb470 tx=0x7f475c02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6711d7d4ec9c148a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.850+0000 7f476aa00640 1 -- 172.21.15.67:0/3607761816 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0317c0 con 0x7f476c0b0db0 2024-03-09T08:59:39.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.850+0000 7f476aa00640 1 -- 172.21.15.67:0/3607761816 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c037070 con 0x7f476c0b0db0 2024-03-09T08:59:39.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.850+0000 7f476aa00640 1 -- 172.21.15.67:0/3607761816 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c031bc0 con 0x7f476c0b0db0 2024-03-09T08:59:39.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.851+0000 7f4771600640 1 -- 172.21.15.67:0/3607761816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f476c0b0db0 msgr2=0x7f476c1565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.851+0000 7f4771600640 1 --2- 172.21.15.67:0/3607761816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f476c0b0db0 0x7f476c1565a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f476c0cb470 tx=0x7f475c02b860 comp rx=0 tx=0).stop 2024-03-09T08:59:39.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.851+0000 7f4771600640 1 -- 172.21.15.67:0/3607761816 shutdown_connections 2024-03-09T08:59:39.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.851+0000 7f4771600640 1 --2- 172.21.15.67:0/3607761816 >> v2:172.21.15.67:3300/0 conn(0x7f476c0bec10 0x7f476c072770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.851+0000 7f4771600640 1 --2- 172.21.15.67:0/3607761816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f476c0b0db0 0x7f476c1565a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.851+0000 7f4771600640 1 -- 172.21.15.67:0/3607761816 >> 172.21.15.67:0/3607761816 conn(0x7f476c0c7f90 msgr2=0x7f476c0c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:39.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.851+0000 7f4771600640 1 -- 172.21.15.67:0/3607761816 shutdown_connections 2024-03-09T08:59:39.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.852+0000 7f4771600640 1 -- 172.21.15.67:0/3607761816 wait complete. 2024-03-09T08:59:39.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.853+0000 7f4771600640 1 Processor -- start 2024-03-09T08:59:39.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.853+0000 7f4771600640 1 -- start start 2024-03-09T08:59:39.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.854+0000 7f4771600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f476c0b0db0 0x7f476c0ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.854+0000 7f4771600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f476c0bec10 0x7f476c0c1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.854+0000 7f4771600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c9410 0x7f476c1f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.854+0000 7f4771600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c06bf50 con 0x7f476c0b0db0 2024-03-09T08:59:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.854+0000 7f4771600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c156ae0 con 0x7f476c0bec10 2024-03-09T08:59:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.854+0000 7f4771600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c06c0d0 con 0x7f476c0c9410 2024-03-09T08:59:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.855+0000 7f4770c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c9410 0x7f476c1f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.855+0000 7f4770c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c9410 0x7f476c1f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46646/0 (socket says 172.21.15.67:46646) 2024-03-09T08:59:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.855+0000 7f4770c00640 1 -- 172.21.15.67:0/246535889 learned_addr learned my addr 172.21.15.67:0/246535889 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:39.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.855+0000 7f476be00640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f476c0b0db0 0x7f476c0ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.856+0000 7f476be00640 1 -- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c9410 msgr2=0x7f476c1f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.856+0000 7f476be00640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c9410 0x7f476c1f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.856+0000 7f476be00640 1 -- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f476c0bec10 msgr2=0x7f476c0c1140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T08:59:39.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.856+0000 7f476be00640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f476c0bec10 0x7f476c0c1140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.856+0000 7f476be00640 1 -- 172.21.15.67:0/246535889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0c2fe0 con 0x7f476c0b0db0 2024-03-09T08:59:39.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.856+0000 7f476be00640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f476c0b0db0 0x7f476c0ae8d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f475c033000 tx=0x7f475c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.856+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c032650 con 0x7f476c0b0db0 2024-03-09T08:59:39.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.856+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c032810 con 0x7f476c0b0db0 2024-03-09T08:59:39.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.857+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c032c00 con 0x7f476c0b0db0 2024-03-09T08:59:39.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.857+0000 7f4771600640 1 -- 172.21.15.67:0/246535889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476c1f3a00 con 0x7f476c0b0db0 2024-03-09T08:59:39.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.857+0000 7f4771600640 1 -- 172.21.15.67:0/246535889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c0c1de0 con 0x7f476c0b0db0 2024-03-09T08:59:39.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.858+0000 7f4771600640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c1680 0x7f476c1f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.859+0000 7f4771600640 1 -- 172.21.15.67:0/246535889 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0c0aa0 con 0x7f476c0c1680 2024-03-09T08:59:39.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.860+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f475c03b3f0 con 0x7f476c0b0db0 2024-03-09T08:59:39.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.860+0000 7f476b400640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c1680 0x7f476c1f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.860+0000 7f4763e00640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f474c06b520 0x7f474c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.861+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f475c0ac440 con 0x7f476c0b0db0 2024-03-09T08:59:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.861+0000 7f4770c00640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f474c06b520 0x7f474c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.862+0000 7f476b400640 1 -- 172.21.15.67:0/246535889 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f476c04f3c0 con 0x7f476c0c1680 2024-03-09T08:59:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.862+0000 7f4770c00640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f474c06b520 0x7f474c06d9e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f476c0c1b70 tx=0x7f4764009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.865+0000 7f476b400640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c1680 0x7f476c1f34c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f476c0cb010 tx=0x7f475800e9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.867+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758017070 con 0x7f476c0c1680 2024-03-09T08:59:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.867+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4758013680 con 0x7f476c0c1680 2024-03-09T08:59:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.867+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c1680 msgr2=0x7f476c1f34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.867+0000 7f4763e00640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c1680 0x7f476c1f34c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f476c0cb010 tx=0x7f475800e9d0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.867+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c1680 msgr2=0x7f476c1f34c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.867+0000 7f4763e00640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0c1680 0x7f476c1f34c0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f476c0cb010 tx=0x7f475800e9d0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.874+0000 7f4771600640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0ad080 0x7f476c152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.874+0000 7f4771600640 1 -- 172.21.15.67:0/246535889 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0ae2b0 con 0x7f476c0ad080 2024-03-09T08:59:39.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.874+0000 7f476b400640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0ad080 0x7f476c152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:39.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.875+0000 7f476b400640 1 -- 172.21.15.67:0/246535889 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f476c06c960 con 0x7f476c0ad080 2024-03-09T08:59:39.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.875+0000 7f476b400640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0ad080 0x7f476c152690 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f476c0cb010 tx=0x7f4758004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:39.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.875+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758017070 con 0x7f476c0ad080 2024-03-09T08:59:39.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.876+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4533 (secure 0 0 0) 0x7f4758006250 con 0x7f476c0ad080 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 60, 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 4, 2024-03-09T08:59:39.877 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:39.878 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:39.879 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:39.880 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.882 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.883 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:39.884 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:39.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:59:39.886 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.887 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:39.888 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:39.889 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:39.889 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:39.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:39.889 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:59:39.889 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:59:39.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.876+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0ad080 msgr2=0x7f476c152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.876+0000 7f4763e00640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0ad080 0x7f476c152690 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f476c0cb010 tx=0x7f4758004730 comp rx=0 tx=0).stop 2024-03-09T08:59:39.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.876+0000 7f4763e00640 1 -- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0ad080 msgr2=0x7f476c152690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:39.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.876+0000 7f4763e00640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f476c0ad080 0x7f476c152690 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f476c0cb010 tx=0x7f4758004730 comp rx=0 tx=0).stop 2024-03-09T08:59:39.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.876+0000 7f4771600640 1 -- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f474c06b520 msgr2=0x7f474c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.876+0000 7f4771600640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f474c06b520 0x7f474c06d9e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f476c0c1b70 tx=0x7f4764009040 comp rx=0 tx=0).stop 2024-03-09T08:59:39.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.876+0000 7f4771600640 1 -- 172.21.15.67:0/246535889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f476c0b0db0 msgr2=0x7f476c0ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:39.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.876+0000 7f4771600640 1 --2- 172.21.15.67:0/246535889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f476c0b0db0 0x7f476c0ae8d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f475c033000 tx=0x7f475c0046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:39.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.876+0000 7f4770c00640 1 -- 172.21.15.67:0/246535889 reap_dead start 2024-03-09T08:59:39.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.877+0000 7f4770c00640 1 -- 172.21.15.67:0/246535889 reap_dead start 2024-03-09T08:59:39.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.877+0000 7f4771600640 1 -- 172.21.15.67:0/246535889 shutdown_connections 2024-03-09T08:59:39.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.877+0000 7f4771600640 1 -- 172.21.15.67:0/246535889 >> 172.21.15.67:0/246535889 conn(0x7f476c0c7f90 msgr2=0x7f476c0c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:39.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.877+0000 7f4771600640 1 -- 172.21.15.67:0/246535889 shutdown_connections 2024-03-09T08:59:39.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:39.877+0000 7f4771600640 1 -- 172.21.15.67:0/246535889 wait complete. 2024-03-09T08:59:39.893 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:59:39.893 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T08:59:39.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T08:59:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.137+0000 7f5de2600640 1 Processor -- start 2024-03-09T08:59:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.137+0000 7f5de2600640 1 -- start start 2024-03-09T08:59:40.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.139+0000 7f5de2600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5ddc0acd90 0x7f5ddc1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:40.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.139+0000 7f5de2600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5ddc0c1d00 0x7f5ddc0aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:40.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.139+0000 7f5de2600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0aa8f0 0x7f5ddc159770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:40.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.139+0000 7f5de2600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ddc1592e0 con 0x7f5ddc0acd90 2024-03-09T08:59:40.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.139+0000 7f5de2600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ddc156b30 con 0x7f5ddc0c1d00 2024-03-09T08:59:40.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.139+0000 7f5de2600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc06de60 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.139+0000 7f5de1200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5ddc0acd90 0x7f5ddc1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:40.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.139+0000 7f5de1200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5ddc0acd90 0x7f5ddc1565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38740/0 (socket says 172.21.15.67:38740) 2024-03-09T08:59:40.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.139+0000 7f5de1200640 1 -- 172.21.15.67:0/1232518628 learned_addr learned my addr 172.21.15.67:0/1232518628 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:40.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.139+0000 7f5ddbe00640 1 --2- 172.21.15.67:0/1232518628 >> v2:172.21.15.134:3300/0 conn(0x7f5ddc0c1d00 0x7f5ddc0aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:40.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.139+0000 7f5de1c00640 1 --2- 172.21.15.67:0/1232518628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0aa8f0 0x7f5ddc159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:40.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.140+0000 7f5de1200640 1 -- 172.21.15.67:0/1232518628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0aa8f0 msgr2=0x7f5ddc159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:40.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.140+0000 7f5de1200640 1 --2- 172.21.15.67:0/1232518628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0aa8f0 0x7f5ddc159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.140+0000 7f5de1200640 1 -- 172.21.15.67:0/1232518628 >> v2:172.21.15.134:3300/0 conn(0x7f5ddc0c1d00 msgr2=0x7f5ddc0aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:40.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.140+0000 7f5de1200640 1 --2- 172.21.15.67:0/1232518628 >> v2:172.21.15.134:3300/0 conn(0x7f5ddc0c1d00 0x7f5ddc0aa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.140+0000 7f5de1200640 1 -- 172.21.15.67:0/1232518628 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc06d5c0 con 0x7f5ddc0acd90 2024-03-09T08:59:40.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.141+0000 7f5de1200640 1 --2- 172.21.15.67:0/1232518628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ddc0acd90 0x7f5ddc1565f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0c8220 tx=0x7f5dd002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d0e3c600f1e533 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.143+0000 7f5ddb400640 1 -- 172.21.15.67:0/1232518628 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0035020 con 0x7f5ddc0acd90 2024-03-09T08:59:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.143+0000 7f5ddb400640 1 -- 172.21.15.67:0/1232518628 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd0031870 con 0x7f5ddc0acd90 2024-03-09T08:59:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.143+0000 7f5ddb400640 1 -- 172.21.15.67:0/1232518628 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0031c40 con 0x7f5ddc0acd90 2024-03-09T08:59:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.143+0000 7f5de2600640 1 -- 172.21.15.67:0/1232518628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ddc0acd90 msgr2=0x7f5ddc1565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.143+0000 7f5de2600640 1 --2- 172.21.15.67:0/1232518628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ddc0acd90 0x7f5ddc1565f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0c8220 tx=0x7f5dd002b910 comp rx=0 tx=0).stop 2024-03-09T08:59:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.144+0000 7f5de2600640 1 -- 172.21.15.67:0/1232518628 shutdown_connections 2024-03-09T08:59:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.144+0000 7f5de2600640 1 --2- 172.21.15.67:0/1232518628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0aa8f0 0x7f5ddc159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.144+0000 7f5de2600640 1 --2- 172.21.15.67:0/1232518628 >> v2:172.21.15.134:3300/0 conn(0x7f5ddc0c1d00 0x7f5ddc0aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.144+0000 7f5de2600640 1 --2- 172.21.15.67:0/1232518628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ddc0acd90 0x7f5ddc1565f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.144+0000 7f5de2600640 1 -- 172.21.15.67:0/1232518628 >> 172.21.15.67:0/1232518628 conn(0x7f5ddc0c7300 msgr2=0x7f5ddc0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.144+0000 7f5de2600640 1 -- 172.21.15.67:0/1232518628 shutdown_connections 2024-03-09T08:59:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.144+0000 7f5de2600640 1 -- 172.21.15.67:0/1232518628 wait complete. 2024-03-09T08:59:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.146+0000 7f5de2600640 1 Processor -- start 2024-03-09T08:59:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.146+0000 7f5de2600640 1 -- start start 2024-03-09T08:59:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.147+0000 7f5de2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ddc0aa8f0 0x7f5ddc1ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.147+0000 7f5de2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc0acd90 0x7f5ddc13d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.147+0000 7f5de2600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0c1d00 0x7f5ddc13d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.147+0000 7f5de2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0ad170 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.147+0000 7f5de2600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc06db80 con 0x7f5ddc0acd90 2024-03-09T08:59:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.147+0000 7f5de2600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0ad2f0 con 0x7f5ddc0c1d00 2024-03-09T08:59:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5de1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ddc0aa8f0 0x7f5ddc1ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5ddbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc0acd90 0x7f5ddc13d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5ddbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc0acd90 0x7f5ddc13d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:44980/0 (socket says 172.21.15.67:44980) 2024-03-09T08:59:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5de1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ddc0aa8f0 0x7f5ddc1ef1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38742/0 (socket says 172.21.15.67:38742) 2024-03-09T08:59:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5ddbe00640 1 -- 172.21.15.67:0/3098487752 learned_addr learned my addr 172.21.15.67:0/3098487752 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5de1c00640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0c1d00 0x7f5ddc13d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5de1200640 1 -- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0c1d00 msgr2=0x7f5ddc13d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5de1200640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0c1d00 0x7f5ddc13d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5de1200640 1 -- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc0acd90 msgr2=0x7f5ddc13d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5de1200640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc0acd90 0x7f5ddc13d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5de1200640 1 -- 172.21.15.67:0/3098487752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc13de00 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.148+0000 7f5ddbe00640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc0acd90 0x7f5ddc13d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:59:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.149+0000 7f5de1200640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ddc0aa8f0 0x7f5ddc1ef1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0033000 tx=0x7f5dd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.149+0000 7f5dd8c00640 1 -- 172.21.15.67:0/3098487752 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0040070 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.149+0000 7f5dd8c00640 1 -- 172.21.15.67:0/3098487752 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd0031e30 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.149+0000 7f5dd8c00640 1 -- 172.21.15.67:0/3098487752 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0035040 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.150+0000 7f5de2600640 1 -- 172.21.15.67:0/3098487752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ddc0721b0 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.150+0000 7f5de2600640 1 -- 172.21.15.67:0/3098487752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ddc14b770 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.152+0000 7f5dd8c00640 1 -- 172.21.15.67:0/3098487752 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5dd004b060 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.153+0000 7f5dd8c00640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5dbc06b520 0x7f5dbc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.153+0000 7f5dd8c00640 1 -- 172.21.15.67:0/3098487752 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f5dd00ac070 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.153+0000 7f5ddbe00640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5dbc06b520 0x7f5dbc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.154+0000 7f5de2600640 1 -- 172.21.15.67:0/3098487752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ddc04f3c0 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.154+0000 7f5ddbe00640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5dbc06b520 0x7f5dbc06d9e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc1e2c70 tx=0x7f5dc8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.164+0000 7f5dd8c00640 1 -- 172.21.15.67:0/3098487752 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5dd007a490 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.430+0000 7f5de2600640 1 -- 172.21.15.67:0/3098487752 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f5ddc06e320 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.432+0000 7f5dd8c00640 1 -- 172.21.15.67:0/3098487752 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f5dd007e530 con 0x7f5ddc0aa8f0 2024-03-09T08:59:40.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.434+0000 7f5de2600640 1 -- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5dbc06b520 msgr2=0x7f5dbc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:40.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.434+0000 7f5de2600640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5dbc06b520 0x7f5dbc06d9e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc1e2c70 tx=0x7f5dc8009190 comp rx=0 tx=0).stop 2024-03-09T08:59:40.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.434+0000 7f5de2600640 1 -- 172.21.15.67:0/3098487752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ddc0aa8f0 msgr2=0x7f5ddc1ef1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:40.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.434+0000 7f5de2600640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ddc0aa8f0 0x7f5ddc1ef1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0033000 tx=0x7f5dd00046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.434+0000 7f5de2600640 1 -- 172.21.15.67:0/3098487752 shutdown_connections 2024-03-09T08:59:40.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.434+0000 7f5de2600640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5dbc06b520 0x7f5dbc06d9e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.434+0000 7f5de2600640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ddc0c1d00 0x7f5ddc13d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.434+0000 7f5de2600640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ddc0acd90 0x7f5ddc13d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.434+0000 7f5de2600640 1 --2- 172.21.15.67:0/3098487752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ddc0aa8f0 0x7f5ddc1ef1a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:40.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.434+0000 7f5de2600640 1 -- 172.21.15.67:0/3098487752 >> 172.21.15.67:0/3098487752 conn(0x7f5ddc0c7300 msgr2=0x7f5ddc14e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:40.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.435+0000 7f5de2600640 1 -- 172.21.15.67:0/3098487752 shutdown_connections 2024-03-09T08:59:40.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:40.435+0000 7f5de2600640 1 -- 172.21.15.67:0/3098487752 wait complete. 2024-03-09T08:59:40.461 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T08:59:41.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:59:41.892 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T08:59:50.462 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T08:59:50.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:59:50.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.649+0000 7fdc1e200640 1 Processor -- start 2024-03-09T08:59:50.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.650+0000 7fdc1e200640 1 -- start start 2024-03-09T08:59:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.652+0000 7fdc1e200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdc180c3780 0x7fdc18152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.652+0000 7fdc1e200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdc180aad00 0x7fdc180ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.652+0000 7fdc1e200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc180c1d70 con 0x7fdc1814c5c0 2024-03-09T08:59:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.652+0000 7fdc1e200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc180aaae0 con 0x7fdc180aad00 2024-03-09T08:59:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.652+0000 7fdc1e200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1814c9a0 con 0x7fdc180c3780 2024-03-09T08:59:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.652+0000 7fdc17e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdc180aad00 0x7fdc180ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.652+0000 7fdc17e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdc180aad00 0x7fdc180ab110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50912/0 (socket says 172.21.15.67:50912) 2024-03-09T08:59:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.652+0000 7fdc17e00640 1 -- 172.21.15.67:0/3434049952 learned_addr learned my addr 172.21.15.67:0/3434049952 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:50.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.653+0000 7fdc16a00640 1 --2- 172.21.15.67:0/3434049952 >> v2:172.21.15.134:3300/0 conn(0x7fdc180c3780 0x7fdc18152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:50.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.653+0000 7fdc17e00640 1 -- 172.21.15.67:0/3434049952 >> v2:172.21.15.134:3300/0 conn(0x7fdc180c3780 msgr2=0x7fdc18152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:50.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.653+0000 7fdc17e00640 1 --2- 172.21.15.67:0/3434049952 >> v2:172.21.15.134:3300/0 conn(0x7fdc180c3780 0x7fdc18152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:50.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.653+0000 7fdc17e00640 1 -- 172.21.15.67:0/3434049952 >> v1:172.21.15.134:6789/0 conn(0x7fdc1814c5c0 legacy=0x7fdc1806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:50.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.653+0000 7fdc17e00640 1 -- 172.21.15.67:0/3434049952 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc180c2010 con 0x7fdc180aad00 2024-03-09T08:59:50.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.654+0000 7fdc17e00640 1 --2- 172.21.15.67:0/3434049952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc180aad00 0x7fdc180ab110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc1806eee0 tx=0x7fdc1002cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45aa6508553118d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.654+0000 7fdc16000640 1 -- 172.21.15.67:0/3434049952 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1002db00 con 0x7fdc180aad00 2024-03-09T08:59:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.654+0000 7fdc16000640 1 -- 172.21.15.67:0/3434049952 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1002dca0 con 0x7fdc180aad00 2024-03-09T08:59:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.654+0000 7fdc16000640 1 -- 172.21.15.67:0/3434049952 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10035610 con 0x7fdc180aad00 2024-03-09T08:59:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.655+0000 7fdc1e200640 1 -- 172.21.15.67:0/3434049952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc180aad00 msgr2=0x7fdc180ab110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.655+0000 7fdc1e200640 1 --2- 172.21.15.67:0/3434049952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc180aad00 0x7fdc180ab110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc1806eee0 tx=0x7fdc1002cf60 comp rx=0 tx=0).stop 2024-03-09T08:59:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.655+0000 7fdc1e200640 1 -- 172.21.15.67:0/3434049952 shutdown_connections 2024-03-09T08:59:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.655+0000 7fdc1e200640 1 --2- 172.21.15.67:0/3434049952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc180aad00 0x7fdc180ab110 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.655+0000 7fdc1e200640 1 --2- 172.21.15.67:0/3434049952 >> v2:172.21.15.134:3300/0 conn(0x7fdc180c3780 0x7fdc18152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.655+0000 7fdc1e200640 1 -- 172.21.15.67:0/3434049952 >> 172.21.15.67:0/3434049952 conn(0x7fdc180c7370 msgr2=0x7fdc180c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.655+0000 7fdc1e200640 1 -- 172.21.15.67:0/3434049952 shutdown_connections 2024-03-09T08:59:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.655+0000 7fdc1e200640 1 -- 172.21.15.67:0/3434049952 wait complete. 2024-03-09T08:59:50.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.657+0000 7fdc1e200640 1 Processor -- start 2024-03-09T08:59:50.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.657+0000 7fdc1e200640 1 -- start start 2024-03-09T08:59:50.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc1e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc180aad00 0x7fdc181ef2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:50.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc1e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc180c3780 0x7fdc181de970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:50.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc1e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc1814c5c0 0x7fdc1814a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:50.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc1e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180c28b0 con 0x7fdc180c3780 2024-03-09T08:59:50.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc1e200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180c25d0 con 0x7fdc1814c5c0 2024-03-09T08:59:50.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc1e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1806e6a0 con 0x7fdc180aad00 2024-03-09T08:59:50.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc16a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc180c3780 0x7fdc181de970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:50.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc16a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc180c3780 0x7fdc181de970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50928/0 (socket says 172.21.15.67:50928) 2024-03-09T08:59:50.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc16a00640 1 -- 172.21.15.67:0/842623407 learned_addr learned my addr 172.21.15.67:0/842623407 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:50.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc17400640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc180aad00 0x7fdc181ef2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:50.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc16a00640 1 -- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc180aad00 msgr2=0x7fdc181ef2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:50.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc16a00640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc180aad00 0x7fdc181ef2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc17e00640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc1814c5c0 0x7fdc1814a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc16a00640 1 -- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc1814c5c0 msgr2=0x7fdc1814a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.658+0000 7fdc16a00640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc1814c5c0 0x7fdc1814a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.659+0000 7fdc16a00640 1 -- 172.21.15.67:0/842623407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc181efb80 con 0x7fdc180c3780 2024-03-09T08:59:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.659+0000 7fdc16a00640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc180c3780 0x7fdc181de970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400d830 tx=0x7fdc0400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.659+0000 7fdc0f400640 1 -- 172.21.15.67:0/842623407 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0400b6c0 con 0x7fdc180c3780 2024-03-09T08:59:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.659+0000 7fdc0f400640 1 -- 172.21.15.67:0/842623407 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc04019070 con 0x7fdc180c3780 2024-03-09T08:59:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.659+0000 7fdc0f400640 1 -- 172.21.15.67:0/842623407 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0400ba80 con 0x7fdc180c3780 2024-03-09T08:59:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.659+0000 7fdc1e200640 1 -- 172.21.15.67:0/842623407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc181efe60 con 0x7fdc180c3780 2024-03-09T08:59:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.659+0000 7fdc1e200640 1 -- 172.21.15.67:0/842623407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc18149540 con 0x7fdc180c3780 2024-03-09T08:59:50.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.661+0000 7fdc1e200640 1 -- 172.21.15.67:0/842623407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc18075a00 con 0x7fdc180c3780 2024-03-09T08:59:50.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.661+0000 7fdc0f400640 1 -- 172.21.15.67:0/842623407 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdc0401e460 con 0x7fdc180c3780 2024-03-09T08:59:50.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.662+0000 7fdc0f400640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdbf806b4d0 0x7fdbf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:50.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.662+0000 7fdc0f400640 1 -- 172.21.15.67:0/842623407 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdc0408ad30 con 0x7fdc180c3780 2024-03-09T08:59:50.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.663+0000 7fdc17400640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdbf806b4d0 0x7fdbf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:50.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.664+0000 7fdc17400640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdbf806b4d0 0x7fdbf806d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800a750 tx=0x7fdc08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:50.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:50.667+0000 7fdc0f400640 1 -- 172.21.15.67:0/842623407 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdc04059730 con 0x7fdc180c3780 2024-03-09T08:59:51.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.028+0000 7fdc1e200640 1 -- 172.21.15.67:0/842623407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdc1814fc80 con 0x7fdc180c3780 2024-03-09T08:59:51.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.029+0000 7fdc0f400640 1 -- 172.21.15.67:0/842623407 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fdc04059030 con 0x7fdc180c3780 2024-03-09T08:59:51.030 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:59:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 -- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdbf806b4d0 msgr2=0x7fdbf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdbf806b4d0 0x7fdbf806d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800a750 tx=0x7fdc08008040 comp rx=0 tx=0).stop 2024-03-09T08:59:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 -- 172.21.15.67:0/842623407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc180c3780 msgr2=0x7fdc181de970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc180c3780 0x7fdc181de970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400d830 tx=0x7fdc0400dcf0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 -- 172.21.15.67:0/842623407 shutdown_connections 2024-03-09T08:59:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdbf806b4d0 0x7fdbf806d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc1814c5c0 0x7fdc1814a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc180c3780 0x7fdc181de970 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 --2- 172.21.15.67:0/842623407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc180aad00 0x7fdc181ef2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 -- 172.21.15.67:0/842623407 >> 172.21.15.67:0/842623407 conn(0x7fdc180c7370 msgr2=0x7fdc18150080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:51.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 -- 172.21.15.67:0/842623407 shutdown_connections 2024-03-09T08:59:51.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.032+0000 7fdc1e200640 1 -- 172.21.15.67:0/842623407 wait complete. 2024-03-09T08:59:51.053 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T08:59:51.053 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T08:59:51.053 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T08:59:51.054 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T08:59:51.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.293+0000 7f52e6c00640 1 Processor -- start 2024-03-09T08:59:51.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.293+0000 7f52e6c00640 1 -- start start 2024-03-09T08:59:51.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.294+0000 7f52e6c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52e00ac9a0 0x7f52e01510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.294+0000 7f52e6c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f52e0071190 0x7f52e0151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.294+0000 7f52e6c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e00c8200 con 0x7f52e00c2780 2024-03-09T08:59:51.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.294+0000 7f52e6c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e006d590 con 0x7f52e0071190 2024-03-09T08:59:51.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.294+0000 7f52e6c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00c3740 con 0x7f52e00ac9a0 2024-03-09T08:59:51.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.295+0000 7f52dfe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52e00ac9a0 0x7f52e01510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.295+0000 7f52df400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f52e00c2780 0x7f52e0154190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:36196/0 (socket says 172.21.15.67:36196) 2024-03-09T08:59:51.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.295+0000 7f52df400640 1 -- 172.21.15.67:0/1179469722 learned_addr learned my addr 172.21.15.67:0/1179469722 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:59:51.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.295+0000 7f52e4c00640 1 --2- 172.21.15.67:0/1179469722 >> v2:172.21.15.134:3300/0 conn(0x7f52e0071190 0x7f52e0151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.296+0000 7f52e4c00640 1 -- 172.21.15.67:0/1179469722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52e00ac9a0 msgr2=0x7f52e01510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:51.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.296+0000 7f52e4c00640 1 --2- 172.21.15.67:0/1179469722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52e00ac9a0 0x7f52e01510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.296+0000 7f52e4c00640 1 -- 172.21.15.67:0/1179469722 >> v1:172.21.15.134:6789/0 conn(0x7f52e00c2780 legacy=0x7f52e0154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.296+0000 7f52e4c00640 1 -- 172.21.15.67:0/1179469722 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e00c3990 con 0x7f52e0071190 2024-03-09T08:59:51.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.297+0000 7f52e4c00640 1 --2- 172.21.15.67:0/1179469722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e0071190 0x7f52e0151620 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009d90 tx=0x7f52d802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fd6ffe2f918ebfe server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.297+0000 7f52dea00640 1 -- 172.21.15.67:0/1179469722 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8039070 con 0x7f52e0071190 2024-03-09T08:59:51.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.297+0000 7f52dea00640 1 -- 172.21.15.67:0/1179469722 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d802cbd0 con 0x7f52e0071190 2024-03-09T08:59:51.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.297+0000 7f52dea00640 1 -- 172.21.15.67:0/1179469722 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8034440 con 0x7f52e0071190 2024-03-09T08:59:51.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.298+0000 7f52e6c00640 1 -- 172.21.15.67:0/1179469722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e0071190 msgr2=0x7f52e0151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.298+0000 7f52e6c00640 1 --2- 172.21.15.67:0/1179469722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e0071190 0x7f52e0151620 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009d90 tx=0x7f52d802be20 comp rx=0 tx=0).stop 2024-03-09T08:59:51.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.298+0000 7f52e6c00640 1 -- 172.21.15.67:0/1179469722 shutdown_connections 2024-03-09T08:59:51.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.298+0000 7f52e6c00640 1 --2- 172.21.15.67:0/1179469722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e0071190 0x7f52e0151620 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.298+0000 7f52e6c00640 1 --2- 172.21.15.67:0/1179469722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52e00ac9a0 0x7f52e01510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.298+0000 7f52e6c00640 1 -- 172.21.15.67:0/1179469722 >> 172.21.15.67:0/1179469722 conn(0x7f52e00c7370 msgr2=0x7f52e00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:51.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.298+0000 7f52e6c00640 1 -- 172.21.15.67:0/1179469722 shutdown_connections 2024-03-09T08:59:51.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.298+0000 7f52e6c00640 1 -- 172.21.15.67:0/1179469722 wait complete. 2024-03-09T08:59:51.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.300+0000 7f52e6c00640 1 Processor -- start 2024-03-09T08:59:51.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.300+0000 7f52e6c00640 1 -- start start 2024-03-09T08:59:51.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.301+0000 7f52e6c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e0071190 0x7f52e01ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.301+0000 7f52e6c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ac9a0 0x7f52e00c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.301+0000 7f52e6c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52e00c2780 0x7f52e01ef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.301+0000 7f52e6c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00acd80 con 0x7f52e00ac9a0 2024-03-09T08:59:51.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.301+0000 7f52e6c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0070c00 con 0x7f52e0071190 2024-03-09T08:59:51.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.301+0000 7f52e6c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e00acf00 con 0x7f52e00c2780 2024-03-09T08:59:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.301+0000 7f52df400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ac9a0 0x7f52e00c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.301+0000 7f52df400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ac9a0 0x7f52e00c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50936/0 (socket says 172.21.15.67:50936) 2024-03-09T08:59:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.301+0000 7f52df400640 1 -- 172.21.15.67:0/3817075609 learned_addr learned my addr 172.21.15.67:0/3817075609 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.301+0000 7f52dfe00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e0071190 0x7f52e01ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.302+0000 7f52e4c00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52e00c2780 0x7f52e01ef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.302+0000 7f52df400640 1 -- 172.21.15.67:0/3817075609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52e00c2780 msgr2=0x7f52e01ef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.302+0000 7f52df400640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52e00c2780 0x7f52e01ef9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.302+0000 7f52df400640 1 -- 172.21.15.67:0/3817075609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e0071190 msgr2=0x7f52e01ebb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.302+0000 7f52df400640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52e0071190 0x7f52e01ebb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.302+0000 7f52df400640 1 -- 172.21.15.67:0/3817075609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e01ec070 con 0x7f52e00ac9a0 2024-03-09T08:59:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.302+0000 7f52df400640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ac9a0 0x7f52e00c6560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00c440 tx=0x7f52cc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.303+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc017070 con 0x7f52e00ac9a0 2024-03-09T08:59:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.303+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc005ce0 con 0x7f52e00ac9a0 2024-03-09T08:59:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.303+0000 7f52e6c00640 1 -- 172.21.15.67:0/3817075609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e00abef0 con 0x7f52e00ac9a0 2024-03-09T08:59:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.303+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0072f0 con 0x7f52e00ac9a0 2024-03-09T08:59:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.303+0000 7f52e6c00640 1 -- 172.21.15.67:0/3817075609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e01ef1d0 con 0x7f52e00ac9a0 2024-03-09T08:59:51.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.304+0000 7f52e6c00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab9a0 0x7f52e00a9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.304+0000 7f52e6c00640 1 -- 172.21.15.67:0/3817075609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0075a00 con 0x7f52e00ab9a0 2024-03-09T08:59:51.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.305+0000 7f52dfe00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab9a0 0x7f52e00a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.305+0000 7f52dfe00640 1 -- 172.21.15.67:0/3817075609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52e01510b0 con 0x7f52e00ab9a0 2024-03-09T08:59:51.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.305+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f52cc01a070 con 0x7f52e00ac9a0 2024-03-09T08:59:51.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.306+0000 7f52dfe00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab9a0 0x7f52e00a9060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f52d00118e0 tx=0x7f52d0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.307+0000 7f52d7e00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f52c006b520 0x7f52c006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.307+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f52cc089710 con 0x7f52e00ac9a0 2024-03-09T08:59:51.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.307+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0019070 con 0x7f52e00ab9a0 2024-03-09T08:59:51.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.307+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f52d0004300 con 0x7f52e00ab9a0 2024-03-09T08:59:51.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.307+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab9a0 msgr2=0x7f52e00a9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.307+0000 7f52d7e00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab9a0 0x7f52e00a9060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f52d00118e0 tx=0x7f52d0011da0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.307+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab9a0 msgr2=0x7f52e00a9060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:51.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.307+0000 7f52d7e00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab9a0 0x7f52e00a9060 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f52d00118e0 tx=0x7f52d0011da0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.307+0000 7f52e4c00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f52c006b520 0x7f52c006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.308+0000 7f52e4c00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f52c006b520 0x7f52c006d9e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f52d8036000 tx=0x7f52d8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.321+0000 7f52e6c00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab450 0x7f52e00a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.321+0000 7f52e6c00640 1 -- 172.21.15.67:0/3817075609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0071d70 con 0x7f52e00ab450 2024-03-09T08:59:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.321+0000 7f52dfe00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab450 0x7f52e00a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.322+0000 7f52dfe00640 1 -- 172.21.15.67:0/3817075609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f52e00a9f30 con 0x7f52e00ab450 2024-03-09T08:59:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.322+0000 7f52dfe00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab450 0x7f52e00a95a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f52d00118e0 tx=0x7f52d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0019040 con 0x7f52e00ab450 2024-03-09T08:59:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5019 (secure 0 0 0) 0x7f52d0003da0 con 0x7f52e00ab450 2024-03-09T08:59:51.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab450 msgr2=0x7f52e00a95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52d7e00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab450 0x7f52e00a95a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f52d00118e0 tx=0x7f52d00046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52d7e00640 1 -- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab450 msgr2=0x7f52e00a95a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:51.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52d7e00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ab450 0x7f52e00a95a0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f52d00118e0 tx=0x7f52d00046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.324 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:59:51.324 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:59:51.324 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:59:51.324 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 60, 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:51.325 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:51.326 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:59:51.327 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:51.328 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:59:51.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.330 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.331 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:51.332 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:51.334 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:51.335 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.336 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:51.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.337 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:51.337 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.337 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:51.337 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.337 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.337 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:59:51.337 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:59:51.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52e6c00640 1 -- 172.21.15.67:0/3817075609 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f52c006b520 msgr2=0x7f52c006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52e6c00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f52c006b520 0x7f52c006d9e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f52d8036000 tx=0x7f52d8030000 comp rx=0 tx=0).stop 2024-03-09T08:59:51.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52e6c00640 1 -- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ac9a0 msgr2=0x7f52e00c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52e6c00640 1 --2- 172.21.15.67:0/3817075609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52e00ac9a0 0x7f52e00c6560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00c440 tx=0x7f52cc00c900 comp rx=0 tx=0).stop 2024-03-09T08:59:51.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52e4c00640 1 -- 172.21.15.67:0/3817075609 reap_dead start 2024-03-09T08:59:51.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52e4c00640 1 -- 172.21.15.67:0/3817075609 reap_dead start 2024-03-09T08:59:51.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52e6c00640 1 -- 172.21.15.67:0/3817075609 shutdown_connections 2024-03-09T08:59:51.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.323+0000 7f52e6c00640 1 -- 172.21.15.67:0/3817075609 >> 172.21.15.67:0/3817075609 conn(0x7f52e00c7370 msgr2=0x7f52e006dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:51.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.324+0000 7f52e6c00640 1 -- 172.21.15.67:0/3817075609 shutdown_connections 2024-03-09T08:59:51.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.324+0000 7f52e6c00640 1 -- 172.21.15.67:0/3817075609 wait complete. 2024-03-09T08:59:51.342 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:59:51.342 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T08:59:51.343 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T08:59:51.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.555+0000 7fdc0da00640 1 Processor -- start 2024-03-09T08:59:51.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.555+0000 7fdc0da00640 1 -- start start 2024-03-09T08:59:51.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.557+0000 7fdc0da00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdc080acf00 0x7fdc080c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.557+0000 7fdc0da00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc080c9380 0x7fdc08156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.557+0000 7fdc0da00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc080c9810 con 0x7fdc080c5e00 2024-03-09T08:59:51.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.557+0000 7fdc0da00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0806d740 con 0x7fdc080acf00 2024-03-09T08:59:51.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.557+0000 7fdc0da00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc08153680 con 0x7fdc080c9380 2024-03-09T08:59:51.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.557+0000 7fdbff400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdc080acf00 0x7fdc080c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.558+0000 7fdbff400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdc080acf00 0x7fdc080c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:50248/0 (socket says 172.21.15.67:50248) 2024-03-09T08:59:51.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.558+0000 7fdbff400640 1 -- 172.21.15.67:0/158551041 learned_addr learned my addr 172.21.15.67:0/158551041 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:51.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.558+0000 7fdc0d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc080c9380 0x7fdc08156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.558+0000 7fdbff400640 1 -- 172.21.15.67:0/158551041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc080c9380 msgr2=0x7fdc08156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:51.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.558+0000 7fdbff400640 1 --2- 172.21.15.67:0/158551041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc080c9380 0x7fdc08156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.558+0000 7fdbff400640 1 -- 172.21.15.67:0/158551041 >> v1:172.21.15.134:6789/0 conn(0x7fdc080c5e00 legacy=0x7fdc0806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.558+0000 7fdbff400640 1 -- 172.21.15.67:0/158551041 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc080ad7c0 con 0x7fdc080acf00 2024-03-09T08:59:51.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.558+0000 7fdc0d000640 1 --2- 172.21.15.67:0/158551041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc080c9380 0x7fdc08156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T08:59:51.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.560+0000 7fdbff400640 1 --2- 172.21.15.67:0/158551041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080acf00 0x7fdc080c61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00098a0 tx=0x7fdbf002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e599b5f806b25aa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.560+0000 7fdc07400640 1 -- 172.21.15.67:0/158551041 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0035020 con 0x7fdc080acf00 2024-03-09T08:59:51.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.560+0000 7fdc07400640 1 -- 172.21.15.67:0/158551041 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf0031640 con 0x7fdc080acf00 2024-03-09T08:59:51.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.560+0000 7fdc07400640 1 -- 172.21.15.67:0/158551041 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf002d040 con 0x7fdc080acf00 2024-03-09T08:59:51.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.561+0000 7fdc0da00640 1 -- 172.21.15.67:0/158551041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080acf00 msgr2=0x7fdc080c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.561+0000 7fdc0da00640 1 --2- 172.21.15.67:0/158551041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080acf00 0x7fdc080c61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00098a0 tx=0x7fdbf002b8f0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.561+0000 7fdc0da00640 1 -- 172.21.15.67:0/158551041 shutdown_connections 2024-03-09T08:59:51.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.561+0000 7fdc0da00640 1 --2- 172.21.15.67:0/158551041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc080c9380 0x7fdc08156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.561+0000 7fdc0da00640 1 --2- 172.21.15.67:0/158551041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080acf00 0x7fdc080c61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.561+0000 7fdc0da00640 1 -- 172.21.15.67:0/158551041 >> 172.21.15.67:0/158551041 conn(0x7fdc080cb480 msgr2=0x7fdc080cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:51.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.561+0000 7fdc0da00640 1 -- 172.21.15.67:0/158551041 shutdown_connections 2024-03-09T08:59:51.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.561+0000 7fdc0da00640 1 -- 172.21.15.67:0/158551041 wait complete. 2024-03-09T08:59:51.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.563+0000 7fdc0da00640 1 Processor -- start 2024-03-09T08:59:51.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.563+0000 7fdc0da00640 1 -- start start 2024-03-09T08:59:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.564+0000 7fdc0da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080acf00 0x7fdc0814e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.564+0000 7fdc0da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc080c5e00 0x7fdc080c1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.564+0000 7fdc0da00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc080c9380 0x7fdc080c1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.564+0000 7fdc0da00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0806bde0 con 0x7fdc080c5e00 2024-03-09T08:59:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.564+0000 7fdc0da00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0806bb00 con 0x7fdc080acf00 2024-03-09T08:59:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.564+0000 7fdc0da00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0806c0c0 con 0x7fdc080c9380 2024-03-09T08:59:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.564+0000 7fdbff400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc080c5e00 0x7fdc080c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.564+0000 7fdbff400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc080c5e00 0x7fdc080c1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50962/0 (socket says 172.21.15.67:50962) 2024-03-09T08:59:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.564+0000 7fdbff400640 1 -- 172.21.15.67:0/1424979111 learned_addr learned my addr 172.21.15.67:0/1424979111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.564+0000 7fdc0d000640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc080c9380 0x7fdc080c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.564+0000 7fdc07e00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080acf00 0x7fdc0814e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.565+0000 7fdbff400640 1 -- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc080c9380 msgr2=0x7fdc080c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.565+0000 7fdbff400640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc080c9380 0x7fdc080c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.565+0000 7fdbff400640 1 -- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080acf00 msgr2=0x7fdc0814e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.565+0000 7fdbff400640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080acf00 0x7fdc0814e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.565+0000 7fdbff400640 1 -- 172.21.15.67:0/1424979111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc08158660 con 0x7fdc080c5e00 2024-03-09T08:59:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.565+0000 7fdc07e00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080acf00 0x7fdc0814e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T08:59:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.565+0000 7fdbff400640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc080c5e00 0x7fdc080c1420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0000c00 tx=0x7fdbf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.565+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0040070 con 0x7fdc080c5e00 2024-03-09T08:59:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.566+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf00399f0 con 0x7fdc080c5e00 2024-03-09T08:59:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.566+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0035040 con 0x7fdc080c5e00 2024-03-09T08:59:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.566+0000 7fdc0da00640 1 -- 172.21.15.67:0/1424979111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc080bf7b0 con 0x7fdc080c5e00 2024-03-09T08:59:51.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.566+0000 7fdc0da00640 1 -- 172.21.15.67:0/1424979111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc080bfd70 con 0x7fdc080c5e00 2024-03-09T08:59:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.567+0000 7fdc0da00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080c0d90 0x7fdc081ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.567+0000 7fdc0da00640 1 -- 172.21.15.67:0/1424979111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0804f3c0 con 0x7fdc080c0d90 2024-03-09T08:59:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.567+0000 7fdc07e00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080c0d90 0x7fdc081ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.568+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdbf004b060 con 0x7fdc080c5e00 2024-03-09T08:59:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.568+0000 7fdc07e00640 1 -- 172.21.15.67:0/1424979111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc080c6620 con 0x7fdc080c0d90 2024-03-09T08:59:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.569+0000 7fdc04c00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdbe406b4d0 0x7fdbe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.569+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdbf00abcc0 con 0x7fdc080c5e00 2024-03-09T08:59:51.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.569+0000 7fdc07e00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080c0d90 0x7fdc081ebc70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80012e0 tx=0x7fdbf800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.569+0000 7fdc0d000640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdbe406b4d0 0x7fdbe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.570+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8003b80 con 0x7fdc080c0d90 2024-03-09T08:59:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.570+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fdbf8018070 con 0x7fdc080c0d90 2024-03-09T08:59:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.570+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080c0d90 msgr2=0x7fdc081ebc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.570+0000 7fdc04c00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080c0d90 0x7fdc081ebc70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80012e0 tx=0x7fdbf800ecd0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.570+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080c0d90 msgr2=0x7fdc081ebc70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.570+0000 7fdc04c00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc080c0d90 0x7fdc081ebc70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.570+0000 7fdc0d000640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdbe406b4d0 0x7fdbe406d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009b60 tx=0x7fdc00009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.578+0000 7fdc0da00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc081ec1b0 0x7fdc08157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.578+0000 7fdc0da00640 1 -- 172.21.15.67:0/1424979111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0806d2f0 con 0x7fdc081ec1b0 2024-03-09T08:59:51.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.579+0000 7fdc07e00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc081ec1b0 0x7fdc08157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.580+0000 7fdc07e00640 1 -- 172.21.15.67:0/1424979111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdc080cbbe0 con 0x7fdc081ec1b0 2024-03-09T08:59:51.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.580+0000 7fdc07e00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc081ec1b0 0x7fdc08157810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8017000 tx=0x7fdbf8001400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.581+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8006e50 con 0x7fdc081ec1b0 2024-03-09T08:59:51.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.581+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7fdbf80050d0 con 0x7fdc081ec1b0 2024-03-09T08:59:51.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.581+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc081ec1b0 msgr2=0x7fdc08157810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.581+0000 7fdc04c00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc081ec1b0 0x7fdc08157810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8017000 tx=0x7fdbf8001400 comp rx=0 tx=0).stop 2024-03-09T08:59:51.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.581+0000 7fdc04c00640 1 -- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc081ec1b0 msgr2=0x7fdc08157810 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:51.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.581+0000 7fdc04c00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc081ec1b0 0x7fdc08157810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.586 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:59:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:59:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:59:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:59:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 60, 2024-03-09T08:59:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:59:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:59:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:59:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:59:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T08:59:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:59:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:59:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:59:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:59:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:59:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:59:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:59:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:59:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:59:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:59:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:59:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:59:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:59:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:59:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:59:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:59:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T08:59:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:59:51.606 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:59:51.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.582+0000 7fdc0da00640 1 -- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdbe406b4d0 msgr2=0x7fdbe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.582+0000 7fdc0da00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdbe406b4d0 0x7fdbe406d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009b60 tx=0x7fdc00009190 comp rx=0 tx=0).stop 2024-03-09T08:59:51.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.582+0000 7fdc0da00640 1 -- 172.21.15.67:0/1424979111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc080c5e00 msgr2=0x7fdc080c1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.582+0000 7fdc0da00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc080c5e00 0x7fdc080c1420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0000c00 tx=0x7fdbf00046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.582+0000 7fdc0d000640 1 -- 172.21.15.67:0/1424979111 reap_dead start 2024-03-09T08:59:51.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.582+0000 7fdc0da00640 1 -- 172.21.15.67:0/1424979111 shutdown_connections 2024-03-09T08:59:51.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.582+0000 7fdc0da00640 1 --2- 172.21.15.67:0/1424979111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc080c5e00 0x7fdc080c1420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.582+0000 7fdc0da00640 1 -- 172.21.15.67:0/1424979111 >> 172.21.15.67:0/1424979111 conn(0x7fdc080cb480 msgr2=0x7fdc08151640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:51.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.582+0000 7fdc0da00640 1 -- 172.21.15.67:0/1424979111 shutdown_connections 2024-03-09T08:59:51.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.582+0000 7fdc0da00640 1 -- 172.21.15.67:0/1424979111 wait complete. 2024-03-09T08:59:51.608 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:59:51.608 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T08:59:51.609 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T08:59:51.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.826+0000 7f9622a00640 1 Processor -- start 2024-03-09T08:59:51.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.826+0000 7f9622a00640 1 -- start start 2024-03-09T08:59:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.828+0000 7f9622a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0ac930 0x7f961c0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.828+0000 7f9622a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f961c0c30a0 0x7f961c150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.828+0000 7f9622a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f961c151190 0x7f961c1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.828+0000 7f9622a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961c070670 con 0x7f961c0c30a0 2024-03-09T08:59:51.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.828+0000 7f9622a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961c0c2780 con 0x7f961c151190 2024-03-09T08:59:51.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.828+0000 7f9622a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961c0708c0 con 0x7f961c0ac930 2024-03-09T08:59:51.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.828+0000 7f9620c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f961c0c30a0 0x7f961c150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.828+0000 7f9620c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f961c0c30a0 0x7f961c150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50976/0 (socket says 172.21.15.67:50976) 2024-03-09T08:59:51.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.828+0000 7f9620c00640 1 -- 172.21.15.67:0/1495591805 learned_addr learned my addr 172.21.15.67:0/1495591805 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:51.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.828+0000 7f9621600640 1 --2- 172.21.15.67:0/1495591805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0ac930 0x7f961c0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.828+0000 7f9622000640 1 --2- 172.21.15.67:0/1495591805 >> v2:172.21.15.134:3300/0 conn(0x7f961c151190 0x7f961c1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.829+0000 7f9620c00640 1 -- 172.21.15.67:0/1495591805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0ac930 msgr2=0x7f961c0c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:51.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.829+0000 7f9620c00640 1 --2- 172.21.15.67:0/1495591805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0ac930 0x7f961c0c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.829+0000 7f9620c00640 1 -- 172.21.15.67:0/1495591805 >> v2:172.21.15.134:3300/0 conn(0x7f961c151190 msgr2=0x7f961c1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:51.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.829+0000 7f9620c00640 1 --2- 172.21.15.67:0/1495591805 >> v2:172.21.15.134:3300/0 conn(0x7f961c151190 0x7f961c1539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.829+0000 7f9620c00640 1 -- 172.21.15.67:0/1495591805 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c070ba0 con 0x7f961c0c30a0 2024-03-09T08:59:51.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.830+0000 7f9620c00640 1 --2- 172.21.15.67:0/1495591805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f961c0c30a0 0x7f961c150c50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f961c0c1590 tx=0x7f960802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a68ed1d53ac065d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.830+0000 7f961be00640 1 -- 172.21.15.67:0/1495591805 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608035020 con 0x7f961c0c30a0 2024-03-09T08:59:51.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.830+0000 7f961be00640 1 -- 172.21.15.67:0/1495591805 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9608031870 con 0x7f961c0c30a0 2024-03-09T08:59:51.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.830+0000 7f961be00640 1 -- 172.21.15.67:0/1495591805 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608031c40 con 0x7f961c0c30a0 2024-03-09T08:59:51.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.831+0000 7f9622a00640 1 -- 172.21.15.67:0/1495591805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f961c0c30a0 msgr2=0x7f961c150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.831+0000 7f9622a00640 1 --2- 172.21.15.67:0/1495591805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f961c0c30a0 0x7f961c150c50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f961c0c1590 tx=0x7f960802b910 comp rx=0 tx=0).stop 2024-03-09T08:59:51.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.831+0000 7f9622a00640 1 -- 172.21.15.67:0/1495591805 shutdown_connections 2024-03-09T08:59:51.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.831+0000 7f9622a00640 1 --2- 172.21.15.67:0/1495591805 >> v2:172.21.15.134:3300/0 conn(0x7f961c151190 0x7f961c1539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.831+0000 7f9622a00640 1 --2- 172.21.15.67:0/1495591805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f961c0c30a0 0x7f961c150c50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.831+0000 7f9622a00640 1 --2- 172.21.15.67:0/1495591805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0ac930 0x7f961c0c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.831+0000 7f9622a00640 1 -- 172.21.15.67:0/1495591805 >> 172.21.15.67:0/1495591805 conn(0x7f961c0c7220 msgr2=0x7f961c0c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:51.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.831+0000 7f9622a00640 1 -- 172.21.15.67:0/1495591805 shutdown_connections 2024-03-09T08:59:51.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.831+0000 7f9622a00640 1 -- 172.21.15.67:0/1495591805 wait complete. 2024-03-09T08:59:51.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.832+0000 7f9622a00640 1 Processor -- start 2024-03-09T08:59:51.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.833+0000 7f9622a00640 1 -- start start 2024-03-09T08:59:51.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.833+0000 7f9622a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0ac930 0x7f961c1489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.833+0000 7f9622a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f961c0c30a0 0x7f961c1ec410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.833+0000 7f9622a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f961c151190 0x7f961c1e21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.833+0000 7f9622a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c0acd10 con 0x7f961c0c30a0 2024-03-09T08:59:51.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.833+0000 7f9622a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c1515e0 con 0x7f961c151190 2024-03-09T08:59:51.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.833+0000 7f9622a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961c0ace90 con 0x7f961c0ac930 2024-03-09T08:59:51.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.834+0000 7f9620c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f961c0c30a0 0x7f961c1ec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.834+0000 7f9620c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f961c0c30a0 0x7f961c1ec410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50982/0 (socket says 172.21.15.67:50982) 2024-03-09T08:59:51.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.834+0000 7f9620c00640 1 -- 172.21.15.67:0/907226684 learned_addr learned my addr 172.21.15.67:0/907226684 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:51.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.834+0000 7f9621600640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0ac930 0x7f961c1489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.834+0000 7f9620c00640 1 -- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0ac930 msgr2=0x7f961c1489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.834+0000 7f9620c00640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0ac930 0x7f961c1489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.834+0000 7f9620c00640 1 -- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f961c151190 msgr2=0x7f961c1e21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.834+0000 7f9622000640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f961c151190 0x7f961c1e21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.834+0000 7f9620c00640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f961c151190 0x7f961c1e21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.834+0000 7f9620c00640 1 -- 172.21.15.67:0/907226684 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c0c4f90 con 0x7f961c0c30a0 2024-03-09T08:59:51.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.835+0000 7f9620c00640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f961c0c30a0 0x7f961c1ec410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f960802b950 tx=0x7f96080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.835+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608035040 con 0x7f961c0c30a0 2024-03-09T08:59:51.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.835+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9608030d00 con 0x7f961c0c30a0 2024-03-09T08:59:51.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.835+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96080456a0 con 0x7f961c0c30a0 2024-03-09T08:59:51.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.835+0000 7f9622a00640 1 -- 172.21.15.67:0/907226684 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961c0c5270 con 0x7f961c0c30a0 2024-03-09T08:59:51.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.835+0000 7f9622a00640 1 -- 172.21.15.67:0/907226684 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961c1e2700 con 0x7f961c0c30a0 2024-03-09T08:59:51.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.836+0000 7f9622a00640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0c58c0 0x7f961c1524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.836+0000 7f9622a00640 1 -- 172.21.15.67:0/907226684 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961c04f3c0 con 0x7f961c0c58c0 2024-03-09T08:59:51.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.837+0000 7f9621600640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0c58c0 0x7f961c1524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.837+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9608038020 con 0x7f961c0c30a0 2024-03-09T08:59:51.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.837+0000 7f9621600640 1 -- 172.21.15.67:0/907226684 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f961c0722a0 con 0x7f961c0c58c0 2024-03-09T08:59:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.838+0000 7f9621600640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0c58c0 0x7f961c1524f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f961c0c2bf0 tx=0x7f961000f910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.839+0000 7f9619600640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f95fc06b4d0 0x7f95fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.839+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f96080ac920 con 0x7f961c0c30a0 2024-03-09T08:59:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.839+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610018030 con 0x7f961c0c58c0 2024-03-09T08:59:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.839+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f9610004020 con 0x7f961c0c58c0 2024-03-09T08:59:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.839+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0c58c0 msgr2=0x7f961c1524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.839+0000 7f9619600640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0c58c0 0x7f961c1524f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f961c0c2bf0 tx=0x7f961000f910 comp rx=0 tx=0).stop 2024-03-09T08:59:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.839+0000 7f9622000640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f95fc06b4d0 0x7f95fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.839+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0c58c0 msgr2=0x7f961c1524f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.839+0000 7f9619600640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c0c58c0 0x7f961c1524f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.840+0000 7f9622000640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f95fc06b4d0 0x7f95fc06d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9614008e70 tx=0x7f96140083a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.846+0000 7f9622a00640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c152a30 0x7f961c147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:51.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.846+0000 7f9622a00640 1 -- 172.21.15.67:0/907226684 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961c074540 con 0x7f961c152a30 2024-03-09T08:59:51.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.847+0000 7f9621600640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c152a30 0x7f961c147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:51.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.848+0000 7f9621600640 1 -- 172.21.15.67:0/907226684 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f961c074ab0 con 0x7f961c152a30 2024-03-09T08:59:51.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.848+0000 7f9621600640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c152a30 0x7f961c147f90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f961c0c2bf0 tx=0x7f96100046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.849+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610014dd0 con 0x7f961c152a30 2024-03-09T08:59:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.849+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f9610014dd0 con 0x7f961c152a30 2024-03-09T08:59:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.849+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c152a30 msgr2=0x7f961c147f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.849+0000 7f9619600640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c152a30 0x7f961c147f90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f961c0c2bf0 tx=0x7f96100046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.849+0000 7f9619600640 1 -- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c152a30 msgr2=0x7f961c147f90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T08:59:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.849+0000 7f9619600640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f961c152a30 0x7f961c147f90 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f961c0c2bf0 tx=0x7f96100046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.852 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T08:59:51.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:59:51.852 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:59:51.853 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T08:59:51.853 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 60, 2024-03-09T08:59:51.853 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T08:59:51.853 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T08:59:51.853 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T08:59:51.853 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T08:59:51.854 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.854 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T08:59:51.854 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:51.854 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T08:59:51.854 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T08:59:51.855 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:51.855 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:51.855 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:51.855 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:51.855 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:51.855 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:51.856 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:51.856 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:51.856 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:51.856 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:51.856 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:51.856 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.856 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T08:59:51.857 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T08:59:51.857 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:51.857 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:51.857 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:51.857 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:51.857 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:51.858 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:51.858 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:51.858 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:51.858 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:51.858 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:51.858 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:51.859 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.859 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T08:59:51.859 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T08:59:51.859 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T08:59:51.859 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T08:59:51.860 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T08:59:51.860 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T08:59:51.860 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:51.860 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T08:59:51.860 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T08:59:51.860 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T08:59:51.861 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T08:59:51.862 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T08:59:51.862 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T08:59:51.862 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T08:59:51.862 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T08:59:51.862 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T08:59:51.862 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T08:59:51.863 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T08:59:51.863 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T08:59:51.863 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T08:59:51.863 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T08:59:51.863 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T08:59:51.863 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T08:59:51.864 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.864 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T08:59:51.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.864 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T08:59:51.864 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.864 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T08:59:51.865 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T08:59:51.865 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:51.865 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:51.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.865 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:51.865 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T08:59:51.866 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.866 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:51.866 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T08:59:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T08:59:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T08:59:51.867 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T08:59:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T08:59:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:51.868 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T08:59:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.869 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T08:59:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T08:59:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T08:59:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:51.870 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T08:59:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T08:59:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T08:59:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T08:59:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T08:59:51.871 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T08:59:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T08:59:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T08:59:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T08:59:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T08:59:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T08:59:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T08:59:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T08:59:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T08:59:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T08:59:51.873 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T08:59:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.874 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T08:59:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T08:59:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T08:59:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T08:59:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T08:59:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T08:59:51.878 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T08:59:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T08:59:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T08:59:51.879 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T08:59:51.879 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T08:59:51.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.850+0000 7f9622a00640 1 -- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f95fc06b4d0 msgr2=0x7f95fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.850+0000 7f9622a00640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f95fc06b4d0 0x7f95fc06d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9614008e70 tx=0x7f96140083a0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.850+0000 7f9622a00640 1 -- 172.21.15.67:0/907226684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f961c0c30a0 msgr2=0x7f961c1ec410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:51.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.850+0000 7f9622a00640 1 --2- 172.21.15.67:0/907226684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f961c0c30a0 0x7f961c1ec410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f960802b950 tx=0x7f96080046c0 comp rx=0 tx=0).stop 2024-03-09T08:59:51.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.850+0000 7f9622000640 1 -- 172.21.15.67:0/907226684 reap_dead start 2024-03-09T08:59:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.850+0000 7f9622000640 1 -- 172.21.15.67:0/907226684 reap_dead start 2024-03-09T08:59:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.850+0000 7f9622a00640 1 -- 172.21.15.67:0/907226684 shutdown_connections 2024-03-09T08:59:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.850+0000 7f9622a00640 1 -- 172.21.15.67:0/907226684 >> 172.21.15.67:0/907226684 conn(0x7f961c0c7220 msgr2=0x7f961c072940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.850+0000 7f9622a00640 1 -- 172.21.15.67:0/907226684 shutdown_connections 2024-03-09T08:59:51.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:51.850+0000 7f9622a00640 1 -- 172.21.15.67:0/907226684 wait complete. 2024-03-09T08:59:51.881 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T08:59:51.881 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T08:59:51.881 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T08:59:51.882 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T08:59:51.882 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T08:59:51.882 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T08:59:51.882 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T08:59:51.927 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T08:59:51.927 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T08:59:51.927 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:59:52.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.154+0000 7f4756a00640 1 Processor -- start 2024-03-09T08:59:52.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.154+0000 7f4756a00640 1 -- start start 2024-03-09T08:59:52.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.156+0000 7f4756a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47500c5e40 0x7f4750156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:52.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.156+0000 7f4756a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f47500cadb0 0x7f4750156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:52.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.156+0000 7f4756a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4750156a90 con 0x7f4750153300 2024-03-09T08:59:52.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.156+0000 7f4756a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4750157000 con 0x7f47500cadb0 2024-03-09T08:59:52.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.156+0000 7f4756a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47500ae430 con 0x7f47500c5e40 2024-03-09T08:59:52.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.156+0000 7f4754c00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47500c5e40 msgr2=0x7f4750156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:52.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.156+0000 7f4754c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47500c5e40 0x7f4750156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:52.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.156+0000 7f4755600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4750153300 0x7f47500b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42218/0 (socket says 172.21.15.67:42218) 2024-03-09T08:59:52.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.156+0000 7f4755600640 1 -- 172.21.15.67:0/3221593150 learned_addr learned my addr 172.21.15.67:0/3221593150 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:59:52.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f4756000640 1 --2- 172.21.15.67:0/3221593150 >> v2:172.21.15.134:3300/0 conn(0x7f47500cadb0 0x7f4750156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:52.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593664878 0 0) 0x7f4750156a90 con 0x7f4750153300 2024-03-09T08:59:52.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4738003520 con 0x7f4750153300 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 688917185 0 0) 0x7f4738003520 con 0x7f4750153300 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47500c5e40 msgr2=0x7f4750156010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 --2- 172.21.15.67:0/3221593150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47500c5e40 0x7f4750156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 >> v2:172.21.15.134:3300/0 conn(0x7f47500cadb0 msgr2=0x7f4750156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 --2- 172.21.15.67:0/3221593150 >> v2:172.21.15.134:3300/0 conn(0x7f47500cadb0 0x7f4750156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47500ae6f0 con 0x7f4750153300 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f4744001670 con 0x7f4750153300 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 >> v1:172.21.15.67:6789/0 conn(0x7f4750153300 legacy=0x7f47500b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 --2- 172.21.15.67:0/3221593150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4738003a20 0x7f4738003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.157+0000 7f474fe00640 1 --2- 172.21.15.67:0/3221593150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4738004800 0x7f4738004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f474fe00640 1 --2- 172.21.15.67:0/3221593150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47500c5e40 0x7f47380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750157000 con 0x7f4738004800 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47500ae430 con 0x7f4738003a20 2024-03-09T08:59:52.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473800bb40 con 0x7f47500c5e40 2024-03-09T08:59:52.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f4756000640 1 -- 172.21.15.67:0/3221593150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47500c5e40 msgr2=0x7f47380051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:52.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f4754c00640 1 --2- 172.21.15.67:0/3221593150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4738004800 0x7f4738004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:52.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f4756000640 1 --2- 172.21.15.67:0/3221593150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47500c5e40 0x7f47380051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:52.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f4755600640 1 --2- 172.21.15.67:0/3221593150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4738003a20 0x7f4738003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:52.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f4754c00640 1 -- 172.21.15.67:0/3221593150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47500c5e40 msgr2=0x7f47380051c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T08:59:52.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f4754c00640 1 --2- 172.21.15.67:0/3221593150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47500c5e40 0x7f47380051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:52.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f4754c00640 1 -- 172.21.15.67:0/3221593150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4738003a20 msgr2=0x7f4738003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:52.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f4754c00640 1 --2- 172.21.15.67:0/3221593150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4738003a20 0x7f4738003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:52.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.158+0000 7f4754c00640 1 -- 172.21.15.67:0/3221593150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4738003520 con 0x7f4738004800 2024-03-09T08:59:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.160+0000 7f4754c00640 1 --2- 172.21.15.67:0/3221593150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4738004800 0x7f4738004c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f473c009900 tx=0x7f4738005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b35ba68dbee081a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.160+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c009500 con 0x7f4738004800 2024-03-09T08:59:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.160+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c003bb0 con 0x7f4738004800 2024-03-09T08:59:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.160+0000 7f474fe00640 1 -- 172.21.15.67:0/3221593150 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c005070 con 0x7f4738004800 2024-03-09T08:59:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.161+0000 7f4756a00640 1 -- 172.21.15.67:0/3221593150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4738004800 msgr2=0x7f4738004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.161+0000 7f4756a00640 1 --2- 172.21.15.67:0/3221593150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4738004800 0x7f4738004c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f473c009900 tx=0x7f4738005830 comp rx=0 tx=0).stop 2024-03-09T08:59:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.161+0000 7f4756000640 1 -- 172.21.15.67:0/3221593150 reap_dead start 2024-03-09T08:59:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.161+0000 7f4756a00640 1 -- 172.21.15.67:0/3221593150 shutdown_connections 2024-03-09T08:59:52.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.161+0000 7f4756a00640 1 -- 172.21.15.67:0/3221593150 >> 172.21.15.67:0/3221593150 conn(0x7f47500732d0 msgr2=0x7f47500ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.161+0000 7f4756a00640 1 -- 172.21.15.67:0/3221593150 shutdown_connections 2024-03-09T08:59:52.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.161+0000 7f4756a00640 1 -- 172.21.15.67:0/3221593150 wait complete. 2024-03-09T08:59:52.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.164+0000 7f4756a00640 1 Processor -- start 2024-03-09T08:59:52.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.165+0000 7f4756a00640 1 -- start start 2024-03-09T08:59:52.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4738004800 0x7f475014e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:52.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475014c260 0x7f47501ebc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:52.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47501ec1c0 0x7f47501ef010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:52.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750159e90 con 0x7f47501ec1c0 2024-03-09T08:59:52.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750159bb0 con 0x7f475014c260 2024-03-09T08:59:52.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47500c59e0 con 0x7f4738004800 2024-03-09T08:59:52.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47501ec1c0 0x7f47501ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:52.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47501ec1c0 0x7f47501ef010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50994/0 (socket says 172.21.15.67:50994) 2024-03-09T08:59:52.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756000640 1 -- 172.21.15.67:0/2141850490 learned_addr learned my addr 172.21.15.67:0/2141850490 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:52.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4754c00640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475014c260 0x7f47501ebc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:52.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756000640 1 -- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4738004800 msgr2=0x7f475014e0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T08:59:52.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756000640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4738004800 0x7f475014e0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:52.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756000640 1 -- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475014c260 msgr2=0x7f47501ebc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:52.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756000640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475014c260 0x7f47501ebc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:52.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.166+0000 7f4756000640 1 -- 172.21.15.67:0/2141850490 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47501eb6c0 con 0x7f47501ec1c0 2024-03-09T08:59:52.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.167+0000 7f4756000640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47501ec1c0 0x7f47501ef010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4748009b10 tx=0x7f474800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:52.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.167+0000 7f474d600640 1 -- 172.21.15.67:0/2141850490 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748003990 con 0x7f47501ec1c0 2024-03-09T08:59:52.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.167+0000 7f474d600640 1 -- 172.21.15.67:0/2141850490 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4748019070 con 0x7f47501ec1c0 2024-03-09T08:59:52.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.167+0000 7f474d600640 1 -- 172.21.15.67:0/2141850490 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748003d70 con 0x7f47501ec1c0 2024-03-09T08:59:52.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.168+0000 7f4756a00640 1 -- 172.21.15.67:0/2141850490 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475014b6d0 con 0x7f47501ec1c0 2024-03-09T08:59:52.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.168+0000 7f4756a00640 1 -- 172.21.15.67:0/2141850490 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4750153bd0 con 0x7f47501ec1c0 2024-03-09T08:59:52.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.169+0000 7f4756a00640 1 -- 172.21.15.67:0/2141850490 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475004f3c0 con 0x7f47501ec1c0 2024-03-09T08:59:52.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.170+0000 7f474d600640 1 -- 172.21.15.67:0/2141850490 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4748012050 con 0x7f47501ec1c0 2024-03-09T08:59:52.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.171+0000 7f474d600640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473006b500 0x7f473006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:52.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.171+0000 7f474d600640 1 -- 172.21.15.67:0/2141850490 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f474808aa10 con 0x7f47501ec1c0 2024-03-09T08:59:52.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.172+0000 7f4755600640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473006b500 0x7f473006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:52.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.173+0000 7f4755600640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473006b500 0x7f473006d9c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4744000c00 tx=0x7f4744007ef0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:52.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.179+0000 7f474d600640 1 -- 172.21.15.67:0/2141850490 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4748010040 con 0x7f47501ec1c0 2024-03-09T08:59:52.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.471+0000 7f4756a00640 1 -- 172.21.15.67:0/2141850490 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f47500cab90 con 0x7f47501ec1c0 2024-03-09T08:59:52.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.472+0000 7f474d600640 1 -- 172.21.15.67:0/2141850490 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f4748059530 con 0x7f47501ec1c0 2024-03-09T08:59:52.473 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:59:52.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 -- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473006b500 msgr2=0x7f473006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:52.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473006b500 0x7f473006d9c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4744000c00 tx=0x7f4744007ef0 comp rx=0 tx=0).stop 2024-03-09T08:59:52.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 -- 172.21.15.67:0/2141850490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47501ec1c0 msgr2=0x7f47501ef010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:52.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47501ec1c0 0x7f47501ef010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4748009b10 tx=0x7f474800c490 comp rx=0 tx=0).stop 2024-03-09T08:59:52.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 -- 172.21.15.67:0/2141850490 shutdown_connections 2024-03-09T08:59:52.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473006b500 0x7f473006d9c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:52.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47501ec1c0 0x7f47501ef010 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:52.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f475014c260 0x7f47501ebc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:52.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 --2- 172.21.15.67:0/2141850490 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4738004800 0x7f475014e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:52.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 -- 172.21.15.67:0/2141850490 >> 172.21.15.67:0/2141850490 conn(0x7f47500732d0 msgr2=0x7f4750156010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:52.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 -- 172.21.15.67:0/2141850490 shutdown_connections 2024-03-09T08:59:52.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:52.475+0000 7f4756a00640 1 -- 172.21.15.67:0/2141850490 wait complete. 2024-03-09T08:59:55.496 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:59:55.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.712+0000 7ffb82400640 1 Processor -- start 2024-03-09T08:59:55.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.712+0000 7ffb82400640 1 -- start start 2024-03-09T08:59:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.714+0000 7ffb82400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffb7c0c22a0 0x7ffb7c071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.714+0000 7ffb82400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffb7c157d60 0x7ffb7c0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.714+0000 7ffb82400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb7c1581a0 con 0x7ffb7c0ac730 2024-03-09T08:59:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.714+0000 7ffb82400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7c070ee0 con 0x7ffb7c0c22a0 2024-03-09T08:59:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.714+0000 7ffb82400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7c154d00 con 0x7ffb7c157d60 2024-03-09T08:59:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.714+0000 7ffb73e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffb7c0c22a0 0x7ffb7c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.714+0000 7ffb7b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ffb7c0ac730 0x7ffb7c157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42220/0 (socket says 172.21.15.67:42220) 2024-03-09T08:59:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.714+0000 7ffb7b400640 1 -- 172.21.15.67:0/2264532958 learned_addr learned my addr 172.21.15.67:0/2264532958 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:59:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.714+0000 7ffb73e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffb7c0c22a0 0x7ffb7c071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51002/0 (socket says 172.21.15.67:51002) 2024-03-09T08:59:55.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.714+0000 7ffb7be00640 1 --2- 172.21.15.67:0/2264532958 >> v2:172.21.15.134:3300/0 conn(0x7ffb7c157d60 0x7ffb7c0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:55.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.715+0000 7ffb73e00640 1 -- 172.21.15.67:0/2264532958 >> v2:172.21.15.134:3300/0 conn(0x7ffb7c157d60 msgr2=0x7ffb7c0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T08:59:55.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.715+0000 7ffb73e00640 1 --2- 172.21.15.67:0/2264532958 >> v2:172.21.15.134:3300/0 conn(0x7ffb7c157d60 0x7ffb7c0acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:55.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.715+0000 7ffb73e00640 1 -- 172.21.15.67:0/2264532958 >> v1:172.21.15.67:6789/0 conn(0x7ffb7c0ac730 legacy=0x7ffb7c157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:55.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.715+0000 7ffb73e00640 1 -- 172.21.15.67:0/2264532958 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c153d40 con 0x7ffb7c0c22a0 2024-03-09T08:59:55.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.715+0000 7ffb7aa00640 1 -- 172.21.15.67:0/2264532958 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456292638 0 0) 0x7ffb7c1581a0 con 0x7ffb7c0ac730 2024-03-09T08:59:55.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.716+0000 7ffb73e00640 1 --2- 172.21.15.67:0/2264532958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7c0c22a0 0x7ffb7c071190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c073af0 tx=0x7ffb6402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c839c8cc72780e7c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:55.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.716+0000 7ffb7aa00640 1 -- 172.21.15.67:0/2264532958 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb640315a0 con 0x7ffb7c0c22a0 2024-03-09T08:59:55.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.716+0000 7ffb7aa00640 1 -- 172.21.15.67:0/2264532958 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb64037070 con 0x7ffb7c0c22a0 2024-03-09T08:59:55.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.716+0000 7ffb7aa00640 1 -- 172.21.15.67:0/2264532958 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64031960 con 0x7ffb7c0c22a0 2024-03-09T08:59:55.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.717+0000 7ffb82400640 1 -- 172.21.15.67:0/2264532958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7c0c22a0 msgr2=0x7ffb7c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:55.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.717+0000 7ffb82400640 1 --2- 172.21.15.67:0/2264532958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7c0c22a0 0x7ffb7c071190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c073af0 tx=0x7ffb6402b840 comp rx=0 tx=0).stop 2024-03-09T08:59:55.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.717+0000 7ffb82400640 1 -- 172.21.15.67:0/2264532958 shutdown_connections 2024-03-09T08:59:55.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.717+0000 7ffb82400640 1 --2- 172.21.15.67:0/2264532958 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7c0c22a0 0x7ffb7c071190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:55.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.717+0000 7ffb82400640 1 --2- 172.21.15.67:0/2264532958 >> v2:172.21.15.134:3300/0 conn(0x7ffb7c157d60 0x7ffb7c0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:55.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.717+0000 7ffb82400640 1 -- 172.21.15.67:0/2264532958 >> 172.21.15.67:0/2264532958 conn(0x7ffb7c0c7370 msgr2=0x7ffb7c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:55.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.717+0000 7ffb82400640 1 -- 172.21.15.67:0/2264532958 shutdown_connections 2024-03-09T08:59:55.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.717+0000 7ffb82400640 1 -- 172.21.15.67:0/2264532958 wait complete. 2024-03-09T08:59:55.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.719+0000 7ffb82400640 1 Processor -- start 2024-03-09T08:59:55.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.719+0000 7ffb82400640 1 -- start start 2024-03-09T08:59:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.720+0000 7ffb82400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7c0ac730 0x7ffb7c1ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:55.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.720+0000 7ffb82400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb7c0c22a0 0x7ffb7c1dd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.720+0000 7ffb82400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb7c157d60 0x7ffb7c1e2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.720+0000 7ffb82400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c070c00 con 0x7ffb7c0ac730 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.720+0000 7ffb82400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c070920 con 0x7ffb7c157d60 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.720+0000 7ffb82400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c151610 con 0x7ffb7c0c22a0 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb73e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb7c0c22a0 msgr2=0x7ffb7c1dd720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb7b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7c0ac730 0x7ffb7c1ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb7c0c22a0 0x7ffb7c1dd720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb7b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7c0ac730 0x7ffb7c1ee8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51016/0 (socket says 172.21.15.67:51016) 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb7b400640 1 -- 172.21.15.67:0/2327970385 learned_addr learned my addr 172.21.15.67:0/2327970385 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb7be00640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb7c157d60 0x7ffb7c1e2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb7b400640 1 -- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb7c0c22a0 msgr2=0x7ffb7c1dd720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T08:59:55.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb7b400640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb7c0c22a0 0x7ffb7c1dd720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:55.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb7b400640 1 -- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb7c157d60 msgr2=0x7ffb7c1e2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:55.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb7b400640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb7c157d60 0x7ffb7c1e2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:55.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb7b400640 1 -- 172.21.15.67:0/2327970385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c1eede0 con 0x7ffb7c0ac730 2024-03-09T08:59:55.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.721+0000 7ffb7b400640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7c0ac730 0x7ffb7c1ee8a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00c4d0 tx=0x7ffb6c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:55.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.722+0000 7ffb73400640 1 -- 172.21.15.67:0/2327970385 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c018070 con 0x7ffb7c0ac730 2024-03-09T08:59:55.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.722+0000 7ffb73400640 1 -- 172.21.15.67:0/2327970385 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c007500 con 0x7ffb7c0ac730 2024-03-09T08:59:55.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.722+0000 7ffb82400640 1 -- 172.21.15.67:0/2327970385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7c1ef080 con 0x7ffb7c0ac730 2024-03-09T08:59:55.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.722+0000 7ffb73400640 1 -- 172.21.15.67:0/2327970385 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c004a20 con 0x7ffb7c0ac730 2024-03-09T08:59:55.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.722+0000 7ffb82400640 1 -- 172.21.15.67:0/2327970385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7c1ef640 con 0x7ffb7c0ac730 2024-03-09T08:59:55.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.723+0000 7ffb82400640 1 -- 172.21.15.67:0/2327970385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7c075a00 con 0x7ffb7c0ac730 2024-03-09T08:59:55.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.724+0000 7ffb73400640 1 -- 172.21.15.67:0/2327970385 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ffb6c01b070 con 0x7ffb7c0ac730 2024-03-09T08:59:55.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.725+0000 7ffb73400640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb5806b480 0x7ffb5806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:55.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.725+0000 7ffb73400640 1 -- 172.21.15.67:0/2327970385 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ffb6c089970 con 0x7ffb7c0ac730 2024-03-09T08:59:55.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.725+0000 7ffb73e00640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb5806b480 0x7ffb5806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T08:59:55.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.726+0000 7ffb73e00640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb5806b480 0x7ffb5806d940 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffb64036000 tx=0x7ffb64008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T08:59:55.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:55.729+0000 7ffb73400640 1 -- 172.21.15.67:0/2327970385 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffb6c058360 con 0x7ffb7c0ac730 2024-03-09T08:59:56.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.113+0000 7ffb82400640 1 -- 172.21.15.67:0/2327970385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ffb7c14efa0 con 0x7ffb7c0ac730 2024-03-09T08:59:56.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.114+0000 7ffb73400640 1 -- 172.21.15.67:0/2327970385 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7ffb6c057c60 con 0x7ffb7c0ac730 2024-03-09T08:59:56.115 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":60,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T08:59:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.116+0000 7ffb82400640 1 -- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb5806b480 msgr2=0x7ffb5806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.116+0000 7ffb82400640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb5806b480 0x7ffb5806d940 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffb64036000 tx=0x7ffb64008030 comp rx=0 tx=0).stop 2024-03-09T08:59:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.116+0000 7ffb82400640 1 -- 172.21.15.67:0/2327970385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7c0ac730 msgr2=0x7ffb7c1ee8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T08:59:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.116+0000 7ffb82400640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7c0ac730 0x7ffb7c1ee8a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00c4d0 tx=0x7ffb6c00c990 comp rx=0 tx=0).stop 2024-03-09T08:59:56.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.116+0000 7ffb82400640 1 -- 172.21.15.67:0/2327970385 shutdown_connections 2024-03-09T08:59:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.116+0000 7ffb82400640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb5806b480 0x7ffb5806d940 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.116+0000 7ffb82400640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb7c157d60 0x7ffb7c1e2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.116+0000 7ffb82400640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb7c0c22a0 0x7ffb7c1dd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.116+0000 7ffb82400640 1 --2- 172.21.15.67:0/2327970385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb7c0ac730 0x7ffb7c1ee8a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T08:59:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.116+0000 7ffb82400640 1 -- 172.21.15.67:0/2327970385 >> 172.21.15.67:0/2327970385 conn(0x7ffb7c0c7370 msgr2=0x7ffb7c072930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T08:59:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.116+0000 7ffb82400640 1 -- 172.21.15.67:0/2327970385 shutdown_connections 2024-03-09T08:59:56.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:56.117+0000 7ffb82400640 1 -- 172.21.15.67:0/2327970385 wait complete. 2024-03-09T08:59:59.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T08:59:59.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.347+0000 7fdea5200640 1 Processor -- start 2024-03-09T08:59:59.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.348+0000 7fdea5200640 1 -- start start 2024-03-09T08:59:59.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.349+0000 7fdea5200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdea0151510 0x7fdea00729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T08:59:59.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.349+0000 7fdea5200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea0153ff0 con 0x7fdea006ffd0 2024-03-09T08:59:59.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.349+0000 7fdea5200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea01542d0 con 0x7fdea00c4330 2024-03-09T08:59:59.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.349+0000 7fdea5200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea01545b0 con 0x7fdea0151510 2024-03-09T08:59:59.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.349+0000 7fde9de00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdea0151510 msgr2=0x7fdea00729c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:59.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.349+0000 7fde9de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdea0151510 0x7fdea00729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T08:59:59.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.350+0000 7fde9e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdea006ffd0 0x7fdea0157920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36982/0 (socket says 172.21.15.67:36982) 2024-03-09T08:59:59.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.350+0000 7fde9e800640 1 -- 172.21.15.67:0/1034196721 learned_addr learned my addr 172.21.15.67:0/1034196721 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T08:59:59.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.550+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdea0151510 msgr2=0x7fdea00729c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:59.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.550+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdea0151510 0x7fdea00729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T08:59:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.951+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdea0151510 msgr2=0x7fdea00729c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T08:59:59.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T08:59:59.951+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdea0151510 0x7fdea00729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:00:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:00.752+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdea0151510 msgr2=0x7fdea00729c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:00:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:00.752+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdea0151510 0x7fdea00729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:00:02.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.349+0000 7fde97400640 1 -- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdea0151510 msgr2=0x7fdea00729c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:00:02.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.349+0000 7fde97400640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdea0151510 0x7fdea00729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:02.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.349+0000 7fde97400640 1 -- 172.21.15.67:0/1034196721 >> v1:172.21.15.134:6789/0 conn(0x7fdea00c4330 legacy=0x7fdea0157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:02.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.349+0000 7fde97400640 1 -- 172.21.15.67:0/1034196721 >> v1:172.21.15.67:6789/0 conn(0x7fdea006ffd0 legacy=0x7fdea0157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:02.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.350+0000 7fde97400640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:02.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.350+0000 7fde97400640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:02.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.350+0000 7fde97400640 1 -- 172.21.15.67:0/1034196721 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea01545b0 con 0x7fde7c003710 2024-03-09T09:00:02.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.350+0000 7fde97400640 1 -- 172.21.15.67:0/1034196721 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde7c008d40 con 0x7fde7c0061c0 2024-03-09T09:00:02.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.350+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:02.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.351+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:02.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.351+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:00:02.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.351+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:02.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.351+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:02.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.351+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:02.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.351+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:00:02.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.351+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:02.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.351+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:02.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.351+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:02.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.552+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:02.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.552+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:02.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.553+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:00:02.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.553+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:02.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.553+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:02.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.553+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:00:02.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.553+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:02.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.553+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:02.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.553+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:00:02.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.553+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:00:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.954+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.954+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.955+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:00:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.955+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.955+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.955+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:00:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.955+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:00:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.955+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.955+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:02.955+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:00:03.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:03.756+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:03.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:03.757+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:03.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:03.757+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:00:03.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:03.757+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:03.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:03.757+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:03.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:03.757+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:00:03.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:03.757+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:03.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:03.757+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:03.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:03.757+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:00:03.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:03.757+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:00:05.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.350+0000 7fde97400640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:00:05.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.350+0000 7fde97400640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:05.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.350+0000 7fde97400640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c005b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:00:05.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.350+0000 7fde97400640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:05.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.350+0000 7fde9e800640 1 -- 172.21.15.67:0/1034196721 reap_dead start 2024-03-09T09:00:05.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.350+0000 7fde97400640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:05.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.350+0000 7fde97400640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 0x7fde7c00b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.350+0000 7fde97400640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.350+0000 7fde97400640 1 -- 172.21.15.67:0/1034196721 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea01545b0 con 0x7fde7c003710 2024-03-09T09:00:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.350+0000 7fde97400640 1 -- 172.21.15.67:0/1034196721 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde7c008d40 con 0x7fde7c0061c0 2024-03-09T09:00:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.350+0000 7fde97400640 1 -- 172.21.15.67:0/1034196721 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c005c90 con 0x7fde7c009190 2024-03-09T09:00:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.351+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 msgr2=0x7fde7c00b580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:00:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.351+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 0x7fde7c00b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:05.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.351+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:05.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.351+0000 7fde9e800640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:05.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.352+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:00:05.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.352+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:05.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.352+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:05.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.352+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:05.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.352+0000 7fde9e800640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:00:05.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.352+0000 7fde9e800640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:05.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.352+0000 7fde9e800640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:05.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.352+0000 7fde9e800640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:05.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.552+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 msgr2=0x7fde7c00b580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:00:05.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.552+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 0x7fde7c00b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:00:05.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.553+0000 7fde9e800640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:05.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.553+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.554+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:00:05.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.554+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:05.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.554+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:05.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.554+0000 7fde9e800640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:00:05.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.554+0000 7fde9e800640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:05.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.554+0000 7fde9e800640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:05.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.554+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:00:05.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.554+0000 7fde9e800640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:00:05.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.953+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 msgr2=0x7fde7c00b580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:00:05.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.953+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 0x7fde7c00b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:00:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.955+0000 7fde9e800640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.956+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.956+0000 7fde9e800640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:00:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.956+0000 7fde9e800640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.956+0000 7fde9e800640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.956+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:00:05.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.956+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 msgr2=0x7fde7c008600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:05.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.956+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:05.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.956+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:00:05.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:05.956+0000 7fde9e800640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:00:06.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.754+0000 7fde97e00640 1 -- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 msgr2=0x7fde7c00b580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:00:06.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.754+0000 7fde97e00640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 0x7fde7c00b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:00:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.757+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.134:3300/0 conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:06.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.758+0000 7fde9e800640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:06.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.758+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 msgr2=0x7fde7c00b580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:00:06.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.758+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 0x7fde7c00b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:06.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.758+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 msgr2=0x7fde7c00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:06.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.758+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:06.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.758+0000 7fde9de00640 1 -- 172.21.15.67:0/1034196721 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea0154890 con 0x7fde7c0061c0 2024-03-09T09:00:06.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.761+0000 7fde9de00640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c0061c0 0x7fde7c008600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdea00721c0 tx=0x7fde90005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=923c80eabb77e67a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:06.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.762+0000 7fde9d400640 1 -- 172.21.15.67:0/1034196721 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9002f720 con 0x7fde7c0061c0 2024-03-09T09:00:06.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.762+0000 7fde9d400640 1 -- 172.21.15.67:0/1034196721 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9002f8c0 con 0x7fde7c0061c0 2024-03-09T09:00:06.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.762+0000 7fde9d400640 1 -- 172.21.15.67:0/1034196721 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9002fcb0 con 0x7fde7c0061c0 2024-03-09T09:00:06.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.763+0000 7fdea5200640 1 -- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c0061c0 msgr2=0x7fde7c008600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:06.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.763+0000 7fdea5200640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c0061c0 0x7fde7c008600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdea00721c0 tx=0x7fde90005040 comp rx=0 tx=0).stop 2024-03-09T09:00:06.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.763+0000 7fdea5200640 1 -- 172.21.15.67:0/1034196721 shutdown_connections 2024-03-09T09:00:06.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.763+0000 7fdea5200640 1 --2- 172.21.15.67:0/1034196721 >> v2:172.21.15.67:3300/0 conn(0x7fde7c003710 0x7fde7c00bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:06.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.763+0000 7fdea5200640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c009190 0x7fde7c00b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:06.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.763+0000 7fdea5200640 1 --2- 172.21.15.67:0/1034196721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c0061c0 0x7fde7c008600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:06.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.763+0000 7fdea5200640 1 -- 172.21.15.67:0/1034196721 >> 172.21.15.67:0/1034196721 conn(0x7fdea00ab320 msgr2=0x7fdea00ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:06.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.764+0000 7fdea5200640 1 -- 172.21.15.67:0/1034196721 shutdown_connections 2024-03-09T09:00:06.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.764+0000 7fdea5200640 1 -- 172.21.15.67:0/1034196721 wait complete. 2024-03-09T09:00:06.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.766+0000 7fdea5200640 1 Processor -- start 2024-03-09T09:00:06.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.766+0000 7fdea5200640 1 -- start start 2024-03-09T09:00:06.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fdea5200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c003710 0x7fdea01ee990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:06.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fdea5200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c0061c0 0x7fdea00ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:06.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fdea5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde7c009190 0x7fdea014bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fdea5200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00c40f0 con 0x7fde7c009190 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fdea5200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00c3f70 con 0x7fde7c003710 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fdea5200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00729c0 con 0x7fde7c0061c0 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fde97e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c0061c0 msgr2=0x7fdea00ae8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fde97e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c0061c0 0x7fdea00ae8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fde9e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde7c009190 0x7fdea014bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fde9e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde7c009190 0x7fdea014bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38262/0 (socket says 172.21.15.67:38262) 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fde9e800640 1 -- 172.21.15.67:0/12406825 learned_addr learned my addr 172.21.15.67:0/12406825 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.768+0000 7fde9de00640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c003710 0x7fdea01ee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.769+0000 7fde9e800640 1 -- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c0061c0 msgr2=0x7fdea00ae8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.769+0000 7fde9e800640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c0061c0 0x7fdea00ae8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.769+0000 7fde9e800640 1 -- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c003710 msgr2=0x7fdea01ee990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.769+0000 7fde9e800640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c003710 0x7fdea01ee990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.769+0000 7fde9e800640 1 -- 172.21.15.67:0/12406825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea014b850 con 0x7fde7c009190 2024-03-09T09:00:06.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.769+0000 7fde9e800640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde7c009190 0x7fdea014bd00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdea0070460 tx=0x7fde980077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:06.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.769+0000 7fde96000640 1 -- 172.21.15.67:0/12406825 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9801b070 con 0x7fde7c009190 2024-03-09T09:00:06.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.769+0000 7fde96000640 1 -- 172.21.15.67:0/12406825 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde98004b60 con 0x7fde7c009190 2024-03-09T09:00:06.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.769+0000 7fdea5200640 1 -- 172.21.15.67:0/12406825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea006db80 con 0x7fde7c009190 2024-03-09T09:00:06.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.770+0000 7fde96000640 1 -- 172.21.15.67:0/12406825 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde980148c0 con 0x7fde7c009190 2024-03-09T09:00:06.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.770+0000 7fdea5200640 1 -- 172.21.15.67:0/12406825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea006cf40 con 0x7fde7c009190 2024-03-09T09:00:06.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.771+0000 7fdea5200640 1 -- 172.21.15.67:0/12406825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea0075a00 con 0x7fde7c009190 2024-03-09T09:00:06.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.773+0000 7fde96000640 1 -- 172.21.15.67:0/12406825 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fde9801e070 con 0x7fde7c009190 2024-03-09T09:00:06.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.774+0000 7fde96000640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fde7c06c720 0x7fde7c06ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:06.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.774+0000 7fde96000640 1 -- 172.21.15.67:0/12406825 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fde9808afc0 con 0x7fde7c009190 2024-03-09T09:00:06.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.774+0000 7fde9de00640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fde7c06c720 0x7fde7c06ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:06.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.776+0000 7fde9de00640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fde7c06c720 0x7fde7c06ebe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fde90039000 tx=0x7fde90032000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:06.779+0000 7fde96000640 1 -- 172.21.15.67:0/12406825 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fde980593f0 con 0x7fde7c009190 2024-03-09T09:00:07.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.155+0000 7fdea5200640 1 -- 172.21.15.67:0/12406825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdea0070b10 con 0x7fde7c009190 2024-03-09T09:00:07.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.156+0000 7fde96000640 1 -- 172.21.15.67:0/12406825 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7fde9805d490 con 0x7fde7c009190 2024-03-09T09:00:07.157 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":64,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:00:07.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 -- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fde7c06c720 msgr2=0x7fde7c06ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fde7c06c720 0x7fde7c06ebe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fde90039000 tx=0x7fde90032000 comp rx=0 tx=0).stop 2024-03-09T09:00:07.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 -- 172.21.15.67:0/12406825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde7c009190 msgr2=0x7fdea014bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde7c009190 0x7fdea014bd00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdea0070460 tx=0x7fde980077b0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 -- 172.21.15.67:0/12406825 shutdown_connections 2024-03-09T09:00:07.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fde7c06c720 0x7fde7c06ebe0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde7c009190 0x7fdea014bd00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde7c0061c0 0x7fdea00ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 --2- 172.21.15.67:0/12406825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde7c003710 0x7fdea01ee990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 -- 172.21.15.67:0/12406825 >> 172.21.15.67:0/12406825 conn(0x7fdea00ab320 msgr2=0x7fdea0153270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:07.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 -- 172.21.15.67:0/12406825 shutdown_connections 2024-03-09T09:00:07.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.160+0000 7fdea5200640 1 -- 172.21.15.67:0/12406825 wait complete. 2024-03-09T09:00:07.181 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:00:07.181 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:00:07.182 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:00:07.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.469+0000 7ff3dd200640 1 Processor -- start 2024-03-09T09:00:07.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.469+0000 7ff3dd200640 1 -- start start 2024-03-09T09:00:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.472+0000 7ff3dd200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3d80acf00 0x7ff3d80c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.472+0000 7ff3dd200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff3d80c9380 0x7ff3d8156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.472+0000 7ff3dd200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d80c9810 con 0x7ff3d80c5e00 2024-03-09T09:00:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.472+0000 7ff3dd200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d806d740 con 0x7ff3d80c9380 2024-03-09T09:00:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.472+0000 7ff3dd200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d8153680 con 0x7ff3d80acf00 2024-03-09T09:00:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.472+0000 7ff3d6a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3d80acf00 msgr2=0x7ff3d80c61e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:00:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.472+0000 7ff3d6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3d80acf00 0x7ff3d80c61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.472+0000 7ff3d7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff3d80c5e00 0x7ff3d806c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36988/0 (socket says 172.21.15.67:36988) 2024-03-09T09:00:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.472+0000 7ff3d7400640 1 -- 172.21.15.67:0/914928161 learned_addr learned my addr 172.21.15.67:0/914928161 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:00:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.472+0000 7ff3d7e00640 1 --2- 172.21.15.67:0/914928161 >> v2:172.21.15.67:3300/0 conn(0x7ff3d80c9380 0x7ff3d8156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.473+0000 7ff3d7e00640 1 -- 172.21.15.67:0/914928161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3d80acf00 msgr2=0x7ff3d80c61e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:00:07.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.473+0000 7ff3d7e00640 1 --2- 172.21.15.67:0/914928161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3d80acf00 0x7ff3d80c61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.473+0000 7ff3d7e00640 1 -- 172.21.15.67:0/914928161 >> v1:172.21.15.67:6789/0 conn(0x7ff3d80c5e00 legacy=0x7ff3d806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.473+0000 7ff3d7e00640 1 -- 172.21.15.67:0/914928161 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d80ad7c0 con 0x7ff3d80c9380 2024-03-09T09:00:07.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.473+0000 7ff3d6000640 1 -- 172.21.15.67:0/914928161 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906348878 0 0) 0x7ff3d80c9810 con 0x7ff3d80c5e00 2024-03-09T09:00:07.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.475+0000 7ff3d7e00640 1 --2- 172.21.15.67:0/914928161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80c9380 0x7ff3d8156660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009dc0 tx=0x7ff3d002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1790f757ffe3b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:07.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.475+0000 7ff3d6000640 1 -- 172.21.15.67:0/914928161 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0039070 con 0x7ff3d80c9380 2024-03-09T09:00:07.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.475+0000 7ff3d6000640 1 -- 172.21.15.67:0/914928161 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d002cc50 con 0x7ff3d80c9380 2024-03-09T09:00:07.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.475+0000 7ff3d6000640 1 -- 172.21.15.67:0/914928161 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0034440 con 0x7ff3d80c9380 2024-03-09T09:00:07.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.476+0000 7ff3dd200640 1 -- 172.21.15.67:0/914928161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80c9380 msgr2=0x7ff3d8156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.476+0000 7ff3dd200640 1 --2- 172.21.15.67:0/914928161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80c9380 0x7ff3d8156660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009dc0 tx=0x7ff3d002bea0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.476+0000 7ff3dd200640 1 -- 172.21.15.67:0/914928161 shutdown_connections 2024-03-09T09:00:07.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.476+0000 7ff3dd200640 1 --2- 172.21.15.67:0/914928161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80c9380 0x7ff3d8156660 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.476+0000 7ff3dd200640 1 --2- 172.21.15.67:0/914928161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3d80acf00 0x7ff3d80c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.476+0000 7ff3dd200640 1 -- 172.21.15.67:0/914928161 >> 172.21.15.67:0/914928161 conn(0x7ff3d80cb480 msgr2=0x7ff3d80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.476+0000 7ff3dd200640 1 -- 172.21.15.67:0/914928161 shutdown_connections 2024-03-09T09:00:07.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.477+0000 7ff3dd200640 1 -- 172.21.15.67:0/914928161 wait complete. 2024-03-09T09:00:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.478+0000 7ff3dd200640 1 Processor -- start 2024-03-09T09:00:07.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.478+0000 7ff3dd200640 1 -- start start 2024-03-09T09:00:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.479+0000 7ff3dd200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d80acf00 0x7ff3d81eeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.479+0000 7ff3dd200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80c5e00 0x7ff3d80bf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.479+0000 7ff3dd200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3d80c9380 0x7ff3d80bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.479+0000 7ff3dd200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d806bde0 con 0x7ff3d80c5e00 2024-03-09T09:00:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.479+0000 7ff3dd200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d806bb00 con 0x7ff3d80acf00 2024-03-09T09:00:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.479+0000 7ff3dd200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d806c0c0 con 0x7ff3d80c9380 2024-03-09T09:00:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.480+0000 7ff3d7e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3d80c9380 msgr2=0x7ff3d80bfee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:00:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.480+0000 7ff3d6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80c5e00 0x7ff3d80bf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.480+0000 7ff3d7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3d80c9380 0x7ff3d80bfee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.480+0000 7ff3d6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80c5e00 0x7ff3d80bf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38272/0 (socket says 172.21.15.67:38272) 2024-03-09T09:00:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.480+0000 7ff3d6a00640 1 -- 172.21.15.67:0/262226900 learned_addr learned my addr 172.21.15.67:0/262226900 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:07.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.480+0000 7ff3d7400640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d80acf00 0x7ff3d81eeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.480+0000 7ff3d6a00640 1 -- 172.21.15.67:0/262226900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3d80c9380 msgr2=0x7ff3d80bfee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:00:07.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.480+0000 7ff3d6a00640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3d80c9380 0x7ff3d80bfee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.480+0000 7ff3d6a00640 1 -- 172.21.15.67:0/262226900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d80acf00 msgr2=0x7ff3d81eeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.480+0000 7ff3d6a00640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d80acf00 0x7ff3d81eeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.480+0000 7ff3d6a00640 1 -- 172.21.15.67:0/262226900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d814fce0 con 0x7ff3d80c5e00 2024-03-09T09:00:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.481+0000 7ff3d6a00640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80c5e00 0x7ff3d80bf9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400b550 tx=0x7ff3c400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.481+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4013020 con 0x7ff3d80c5e00 2024-03-09T09:00:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.481+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c4003de0 con 0x7ff3d80c5e00 2024-03-09T09:00:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.481+0000 7ff3dd200640 1 -- 172.21.15.67:0/262226900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d81ef4c0 con 0x7ff3d80c5e00 2024-03-09T09:00:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.481+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40052b0 con 0x7ff3d80c5e00 2024-03-09T09:00:07.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.481+0000 7ff3dd200640 1 -- 172.21.15.67:0/262226900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d80c20e0 con 0x7ff3d80c5e00 2024-03-09T09:00:07.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.482+0000 7ff3dd200640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d81ef0e0 0x7ff3d814eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.482+0000 7ff3dd200640 1 -- 172.21.15.67:0/262226900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80c2680 con 0x7ff3d81ef0e0 2024-03-09T09:00:07.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.483+0000 7ff3d7400640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d81ef0e0 0x7ff3d814eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.483+0000 7ff3d7400640 1 -- 172.21.15.67:0/262226900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3d804f3c0 con 0x7ff3d81ef0e0 2024-03-09T09:00:07.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.483+0000 7ff3d7400640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d81ef0e0 0x7ff3d814eed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80011e0 tx=0x7ff3c80018b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:07.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.484+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8019070 con 0x7ff3d81ef0e0 2024-03-09T09:00:07.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.484+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff3c8010560 con 0x7ff3d81ef0e0 2024-03-09T09:00:07.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.484+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d81ef0e0 msgr2=0x7ff3d814eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.484+0000 7ff3cf400640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d81ef0e0 0x7ff3d814eed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80011e0 tx=0x7ff3c80018b0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.484+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d81ef0e0 msgr2=0x7ff3d814eed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:07.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.484+0000 7ff3cf400640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d81ef0e0 0x7ff3d814eed0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80011e0 tx=0x7ff3c80018b0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.484+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff3c4016020 con 0x7ff3d80c5e00 2024-03-09T09:00:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.485+0000 7ff3cf400640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff3b806afb0 0x7ff3b806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.486+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff3c408a350 con 0x7ff3d80c5e00 2024-03-09T09:00:07.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.486+0000 7ff3d7400640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff3b806afb0 0x7ff3b806d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.487+0000 7ff3d7400640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff3b806afb0 0x7ff3b806d470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80011e0 tx=0x7ff3c8009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:07.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.492+0000 7ff3dd200640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80bea20 0x7ff3d814f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.492+0000 7ff3dd200640 1 -- 172.21.15.67:0/262226900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80affb0 con 0x7ff3d80bea20 2024-03-09T09:00:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.492+0000 7ff3d7e00640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80bea20 0x7ff3d814f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.493+0000 7ff3d7e00640 1 -- 172.21.15.67:0/262226900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff3d80afcd0 con 0x7ff3d80bea20 2024-03-09T09:00:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.493+0000 7ff3d7e00640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80bea20 0x7ff3d814f410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff3d808d290 tx=0x7ff3d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:07.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.493+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0044070 con 0x7ff3d80bea20 2024-03-09T09:00:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5007 (secure 0 0 0) 0x7ff3d0034d80 con 0x7ff3d80bea20 2024-03-09T09:00:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80bea20 msgr2=0x7ff3d814f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3cf400640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80bea20 0x7ff3d814f410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff3d808d290 tx=0x7ff3d0004a40 comp rx=0 tx=0).stop 2024-03-09T09:00:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3cf400640 1 -- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80bea20 msgr2=0x7ff3d814f410 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:07.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3cf400640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80bea20 0x7ff3d814f410 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.495 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:00:07.495 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:00:07.495 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:00:07.495 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 64, 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:07.496 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:07.497 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:07.498 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:07.499 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:00:07.500 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:00:07.501 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.503 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.504 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:07.505 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:07.506 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.507 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.508 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:00:07.508 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:00:07.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3dd200640 1 -- 172.21.15.67:0/262226900 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff3b806afb0 msgr2=0x7ff3b806d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3dd200640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff3b806afb0 0x7ff3b806d470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80011e0 tx=0x7ff3c8009030 comp rx=0 tx=0).stop 2024-03-09T09:00:07.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3dd200640 1 -- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80c5e00 msgr2=0x7ff3d80bf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3dd200640 1 --2- 172.21.15.67:0/262226900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3d80c5e00 0x7ff3d80bf9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400b550 tx=0x7ff3c400ba10 comp rx=0 tx=0).stop 2024-03-09T09:00:07.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3d7e00640 1 -- 172.21.15.67:0/262226900 reap_dead start 2024-03-09T09:00:07.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3dd200640 1 -- 172.21.15.67:0/262226900 shutdown_connections 2024-03-09T09:00:07.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.494+0000 7ff3dd200640 1 -- 172.21.15.67:0/262226900 >> 172.21.15.67:0/262226900 conn(0x7ff3d80cb480 msgr2=0x7ff3d80ab6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:07.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.495+0000 7ff3dd200640 1 -- 172.21.15.67:0/262226900 shutdown_connections 2024-03-09T09:00:07.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.495+0000 7ff3dd200640 1 -- 172.21.15.67:0/262226900 wait complete. 2024-03-09T09:00:07.512 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:00:07.512 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:00:07.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:00:07.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.717+0000 7f12d5000640 1 Processor -- start 2024-03-09T09:00:07.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.717+0000 7f12d5000640 1 -- start start 2024-03-09T09:00:07.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.719+0000 7f12d5000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12d0151360 0x7f12d00c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.719+0000 7f12d5000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d00c30a0 0x7f12d01539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.719+0000 7f12d5000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d00c2780 con 0x7f12d00ac930 2024-03-09T09:00:07.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.719+0000 7f12d5000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d0070670 con 0x7f12d0151360 2024-03-09T09:00:07.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.719+0000 7f12d5000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00708c0 con 0x7f12d00c30a0 2024-03-09T09:00:07.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.719+0000 7f12cfe00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d00c30a0 msgr2=0x7f12d01539e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:00:07.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.719+0000 7f12cfe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d00c30a0 0x7f12d01539e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:07.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.719+0000 7f12cea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12d0151360 0x7f12d00c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.719+0000 7f12cea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12d0151360 0x7f12d00c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38306/0 (socket says 172.21.15.67:38306) 2024-03-09T09:00:07.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.719+0000 7f12cea00640 1 -- 172.21.15.67:0/1331737612 learned_addr learned my addr 172.21.15.67:0/1331737612 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:07.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.720+0000 7f12cea00640 1 -- 172.21.15.67:0/1331737612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d00c30a0 msgr2=0x7f12d01539e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:00:07.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.720+0000 7f12cea00640 1 --2- 172.21.15.67:0/1331737612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d00c30a0 0x7f12d01539e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.720+0000 7f12cea00640 1 -- 172.21.15.67:0/1331737612 >> v1:172.21.15.134:6789/0 conn(0x7f12d00ac930 legacy=0x7f12d0150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.720+0000 7f12cea00640 1 -- 172.21.15.67:0/1331737612 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d0070ba0 con 0x7f12d0151360 2024-03-09T09:00:07.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.721+0000 7f12cea00640 1 --2- 172.21.15.67:0/1331737612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d0151360 0x7f12d00c3660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009870 tx=0x7f12bc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d13156a8b94278 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:07.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.721+0000 7f12ce000640 1 -- 172.21.15.67:0/1331737612 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc034070 con 0x7f12d0151360 2024-03-09T09:00:07.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.721+0000 7f12ce000640 1 -- 172.21.15.67:0/1331737612 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc030860 con 0x7f12d0151360 2024-03-09T09:00:07.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.721+0000 7f12ce000640 1 -- 172.21.15.67:0/1331737612 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc030c30 con 0x7f12d0151360 2024-03-09T09:00:07.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.722+0000 7f12d5000640 1 -- 172.21.15.67:0/1331737612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d0151360 msgr2=0x7f12d00c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.722+0000 7f12d5000640 1 --2- 172.21.15.67:0/1331737612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d0151360 0x7f12d00c3660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009870 tx=0x7f12bc02b9a0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.722+0000 7f12d5000640 1 -- 172.21.15.67:0/1331737612 shutdown_connections 2024-03-09T09:00:07.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.722+0000 7f12d5000640 1 --2- 172.21.15.67:0/1331737612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d00c30a0 0x7f12d01539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.722+0000 7f12d5000640 1 --2- 172.21.15.67:0/1331737612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d0151360 0x7f12d00c3660 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.722+0000 7f12d5000640 1 -- 172.21.15.67:0/1331737612 >> 172.21.15.67:0/1331737612 conn(0x7f12d00c7220 msgr2=0x7f12d00c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:07.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.722+0000 7f12d5000640 1 -- 172.21.15.67:0/1331737612 shutdown_connections 2024-03-09T09:00:07.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.722+0000 7f12d5000640 1 -- 172.21.15.67:0/1331737612 wait complete. 2024-03-09T09:00:07.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.724+0000 7f12d5000640 1 Processor -- start 2024-03-09T09:00:07.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.724+0000 7f12d5000640 1 -- start start 2024-03-09T09:00:07.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.726+0000 7f12d5000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d00ac930 0x7f12d00a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.726+0000 7f12d5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d00c30a0 0x7f12d014d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.726+0000 7f12d5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d0151360 0x7f12d00aa890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.726+0000 7f12d5000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00ace90 con 0x7f12d00c30a0 2024-03-09T09:00:07.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.726+0000 7f12d5000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00acd10 con 0x7f12d0151360 2024-03-09T09:00:07.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.726+0000 7f12d5000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0153f20 con 0x7f12d00ac930 2024-03-09T09:00:07.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.726+0000 7f12cf400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d00ac930 msgr2=0x7f12d00a94d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:00:07.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.726+0000 7f12cf400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d00ac930 0x7f12d00a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:07.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.726+0000 7f12cea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d00c30a0 0x7f12d014d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.726+0000 7f12cea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d00c30a0 0x7f12d014d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38318/0 (socket says 172.21.15.67:38318) 2024-03-09T09:00:07.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.726+0000 7f12cea00640 1 -- 172.21.15.67:0/1275726089 learned_addr learned my addr 172.21.15.67:0/1275726089 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:07.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.727+0000 7f12cfe00640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d0151360 0x7f12d00aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.727+0000 7f12cea00640 1 -- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d00ac930 msgr2=0x7f12d00a94d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:00:07.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.727+0000 7f12cea00640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12d00ac930 0x7f12d00a94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.727+0000 7f12cea00640 1 -- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d0151360 msgr2=0x7f12d00aa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.727+0000 7f12cea00640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d0151360 0x7f12d00aa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.727+0000 7f12cea00640 1 -- 172.21.15.67:0/1275726089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d01f09f0 con 0x7f12d00c30a0 2024-03-09T09:00:07.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.727+0000 7f12cfe00640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d0151360 0x7f12d00aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:00:07.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.727+0000 7f12cea00640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d00c30a0 0x7f12d014d4c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000c00 tx=0x7f12bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:07.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.728+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc034040 con 0x7f12d00c30a0 2024-03-09T09:00:07.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.728+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc0443f0 con 0x7f12d00c30a0 2024-03-09T09:00:07.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.728+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc0447e0 con 0x7f12d00c30a0 2024-03-09T09:00:07.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.728+0000 7f12d5000640 1 -- 172.21.15.67:0/1275726089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d00abf10 con 0x7f12d00c30a0 2024-03-09T09:00:07.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.728+0000 7f12d5000640 1 -- 172.21.15.67:0/1275726089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d01f06c0 con 0x7f12d00c30a0 2024-03-09T09:00:07.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.730+0000 7f12d5000640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d00a9a10 0x7f12d00aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.730+0000 7f12d5000640 1 -- 172.21.15.67:0/1275726089 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d014e150 con 0x7f12d00a9a10 2024-03-09T09:00:07.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.730+0000 7f12cf400640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d00a9a10 0x7f12d00aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.731+0000 7f12cf400640 1 -- 172.21.15.67:0/1275726089 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12d01557d0 con 0x7f12d00a9a10 2024-03-09T09:00:07.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.731+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f12bc04b060 con 0x7f12d00c30a0 2024-03-09T09:00:07.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.731+0000 7f12cf400640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d00a9a10 0x7f12d00aadd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12c0001030 tx=0x7f12c000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:07.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.733+0000 7f12c7400640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12b006b4d0 0x7f12b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.733+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f12bc0ac7d0 con 0x7f12d00c30a0 2024-03-09T09:00:07.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.733+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c00074e0 con 0x7f12d00a9a10 2024-03-09T09:00:07.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.733+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f12c0004020 con 0x7f12d00a9a10 2024-03-09T09:00:07.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.733+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d00a9a10 msgr2=0x7f12d00aadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.733+0000 7f12c7400640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d00a9a10 0x7f12d00aadd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12c0001030 tx=0x7f12c000c810 comp rx=0 tx=0).stop 2024-03-09T09:00:07.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.733+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d00a9a10 msgr2=0x7f12d00aadd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:07.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.733+0000 7f12c7400640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d00a9a10 0x7f12d00aadd0 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12c0001030 tx=0x7f12c000c810 comp rx=0 tx=0).stop 2024-03-09T09:00:07.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.734+0000 7f12cfe00640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12b006b4d0 0x7f12b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.735+0000 7f12cfe00640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12b006b4d0 0x7f12b006d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f12c8003bc0 tx=0x7f12c8009510 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:07.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.741+0000 7f12d5000640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d014da00 0x7f12d01f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:07.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.741+0000 7f12d5000640 1 -- 172.21.15.67:0/1275726089 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00abd00 con 0x7f12d014da00 2024-03-09T09:00:07.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.742+0000 7f12cf400640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d014da00 0x7f12d01f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:07.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.742+0000 7f12cf400640 1 -- 172.21.15.67:0/1275726089 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f12d004f3c0 con 0x7f12d014da00 2024-03-09T09:00:07.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.743+0000 7f12cf400640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d014da00 0x7f12d01f31b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f12c0001eb0 tx=0x7f12c0000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:07.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.743+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0005cd0 con 0x7f12d014da00 2024-03-09T09:00:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.744+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5175 (secure 0 0 0) 0x7f12c0005cd0 con 0x7f12d014da00 2024-03-09T09:00:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.744+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d014da00 msgr2=0x7f12d01f31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.744+0000 7f12c7400640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d014da00 0x7f12d01f31b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f12c0001eb0 tx=0x7f12c0000ed0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.744+0000 7f12c7400640 1 -- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d014da00 msgr2=0x7f12d01f31b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.744+0000 7f12c7400640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12d014da00 0x7f12d01f31b0 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f12c0001eb0 tx=0x7f12c0000ed0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.745 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 64, 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:00:07.746 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:07.747 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:00:07.748 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:00:07.749 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:07.750 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:07.750 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:07.750 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:07.750 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:07.750 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:07.750 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:07.750 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:07.750 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:07.750 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:07.750 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:07.751 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.751 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:00:07.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.751 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:00:07.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.751 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:00:07.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:00:07.752 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:07.752 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:07.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.752 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:07.752 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:00:07.752 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:00:07.753 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.754 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:07.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:07.756 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:00:07.757 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.758 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:07.758 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:07.758 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:07.758 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.758 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.758 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:00:07.758 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.758 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:07.758 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:07.758 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:00:07.758 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:07.759 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:00:07.760 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:07.760 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:07.760 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:07.760 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:07.760 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:07.760 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:07.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:07.760 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:00:07.760 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:00:07.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.745+0000 7f12d5000640 1 -- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12b006b4d0 msgr2=0x7f12b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.745+0000 7f12d5000640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12b006b4d0 0x7f12b006d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f12c8003bc0 tx=0x7f12c8009510 comp rx=0 tx=0).stop 2024-03-09T09:00:07.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.745+0000 7f12d5000640 1 -- 172.21.15.67:0/1275726089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d00c30a0 msgr2=0x7f12d014d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:07.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.745+0000 7f12d5000640 1 --2- 172.21.15.67:0/1275726089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12d00c30a0 0x7f12d014d4c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000c00 tx=0x7f12bc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:00:07.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.745+0000 7f12cfe00640 1 -- 172.21.15.67:0/1275726089 reap_dead start 2024-03-09T09:00:07.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.745+0000 7f12cfe00640 1 -- 172.21.15.67:0/1275726089 reap_dead start 2024-03-09T09:00:07.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.745+0000 7f12d5000640 1 -- 172.21.15.67:0/1275726089 shutdown_connections 2024-03-09T09:00:07.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.745+0000 7f12d5000640 1 -- 172.21.15.67:0/1275726089 >> 172.21.15.67:0/1275726089 conn(0x7f12d00c7220 msgr2=0x7f12d01521d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:07.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.745+0000 7f12d5000640 1 -- 172.21.15.67:0/1275726089 shutdown_connections 2024-03-09T09:00:07.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:07.745+0000 7f12d5000640 1 -- 172.21.15.67:0/1275726089 wait complete. 2024-03-09T09:00:07.765 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:00:07.765 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:00:12.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:00:12.086 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:00:27.766 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:00:27.766 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:00:27.766 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:00:27.767 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:00:27.775 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:00:27.775 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:00:27.776 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:00:27.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.981+0000 7fbb75c00640 1 Processor -- start 2024-03-09T09:00:27.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.981+0000 7fbb75c00640 1 -- start start 2024-03-09T09:00:27.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.982+0000 7fbb75c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbb700c5e40 0x7fbb70156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:27.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.982+0000 7fbb75c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb70157000 con 0x7fbb70153300 2024-03-09T09:00:27.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.983+0000 7fbb75c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb700ae430 con 0x7fbb700cadb0 2024-03-09T09:00:27.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.983+0000 7fbb75c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb700ae6f0 con 0x7fbb700c5e40 2024-03-09T09:00:27.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.983+0000 7fbb6f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbb700c5e40 0x7fbb70156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:27.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.983+0000 7fbb6fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbb70153300 0x7fbb700b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58782/0 (socket says 172.21.15.67:58782) 2024-03-09T09:00:27.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.983+0000 7fbb6fe00640 1 -- 172.21.15.67:0/1754841766 learned_addr learned my addr 172.21.15.67:0/1754841766 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:00:27.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.983+0000 7fbb6f400640 1 -- 172.21.15.67:0/1754841766 >> v1:172.21.15.134:6789/0 conn(0x7fbb700cadb0 legacy=0x7fbb70156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:27.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.983+0000 7fbb6f400640 1 -- 172.21.15.67:0/1754841766 >> v1:172.21.15.67:6789/0 conn(0x7fbb70153300 legacy=0x7fbb700b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:27.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.983+0000 7fbb6f400640 1 -- 172.21.15.67:0/1754841766 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb70155b10 con 0x7fbb700c5e40 2024-03-09T09:00:27.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.985+0000 7fbb6f400640 1 --2- 172.21.15.67:0/1754841766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb700c5e40 0x7fbb70156010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0097a0 tx=0x7fbb5c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328d1cecd299615 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.985+0000 7fbb6ea00640 1 -- 172.21.15.67:0/1754841766 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c034070 con 0x7fbb700c5e40 2024-03-09T09:00:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.985+0000 7fbb6ea00640 1 -- 172.21.15.67:0/1754841766 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c030610 con 0x7fbb700c5e40 2024-03-09T09:00:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.985+0000 7fbb6ea00640 1 -- 172.21.15.67:0/1754841766 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c0309e0 con 0x7fbb700c5e40 2024-03-09T09:00:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.986+0000 7fbb75c00640 1 -- 172.21.15.67:0/1754841766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb700c5e40 msgr2=0x7fbb70156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.986+0000 7fbb75c00640 1 --2- 172.21.15.67:0/1754841766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb700c5e40 0x7fbb70156010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0097a0 tx=0x7fbb5c02b960 comp rx=0 tx=0).stop 2024-03-09T09:00:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.986+0000 7fbb75c00640 1 -- 172.21.15.67:0/1754841766 shutdown_connections 2024-03-09T09:00:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.986+0000 7fbb75c00640 1 --2- 172.21.15.67:0/1754841766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb700c5e40 0x7fbb70156010 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.986+0000 7fbb75c00640 1 -- 172.21.15.67:0/1754841766 >> 172.21.15.67:0/1754841766 conn(0x7fbb700732d0 msgr2=0x7fbb700ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.986+0000 7fbb75c00640 1 -- 172.21.15.67:0/1754841766 shutdown_connections 2024-03-09T09:00:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.986+0000 7fbb75c00640 1 -- 172.21.15.67:0/1754841766 wait complete. 2024-03-09T09:00:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.988+0000 7fbb75c00640 1 Processor -- start 2024-03-09T09:00:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.988+0000 7fbb75c00640 1 -- start start 2024-03-09T09:00:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb75c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb700c5e40 0x7fbb701544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb75c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb700cadb0 0x7fbb7014be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb75c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb70153300 0x7fbb7014c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb75c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700c59e0 con 0x7fbb700c5e40 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb75c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb70159e90 con 0x7fbb700cadb0 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb75c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700b0710 con 0x7fbb70153300 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb6fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb700c5e40 0x7fbb701544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb6fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb700c5e40 0x7fbb701544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44288/0 (socket says 172.21.15.67:44288) 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb6fe00640 1 -- 172.21.15.67:0/1429475227 learned_addr learned my addr 172.21.15.67:0/1429475227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb6f400640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb700cadb0 0x7fbb7014be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb75200640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb70153300 0x7fbb7014c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb75200640 1 -- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb70153300 msgr2=0x7fbb7014c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb75200640 1 -- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb70153300 msgr2=0x7fbb7014c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb75200640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb70153300 0x7fbb7014c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb6fe00640 1 -- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb70153300 msgr2=0x7fbb7014c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb75200640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb70153300 0x7fbb7014c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.989+0000 7fbb6fe00640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb70153300 0x7fbb7014c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.990+0000 7fbb6fe00640 1 -- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb700cadb0 msgr2=0x7fbb7014be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.990+0000 7fbb6fe00640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb700cadb0 0x7fbb7014be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:27.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.990+0000 7fbb6fe00640 1 -- 172.21.15.67:0/1429475227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb700c7d00 con 0x7fbb700c5e40 2024-03-09T09:00:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.990+0000 7fbb6fe00640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb700c5e40 0x7fbb701544a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000c4b0 tx=0x7fbb6000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.990+0000 7fbb67e00640 1 -- 172.21.15.67:0/1429475227 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60017070 con 0x7fbb700c5e40 2024-03-09T09:00:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.990+0000 7fbb67e00640 1 -- 172.21.15.67:0/1429475227 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb60005ce0 con 0x7fbb700c5e40 2024-03-09T09:00:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.990+0000 7fbb75c00640 1 -- 172.21.15.67:0/1429475227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb701ec6b0 con 0x7fbb700c5e40 2024-03-09T09:00:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.990+0000 7fbb67e00640 1 -- 172.21.15.67:0/1429475227 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60007270 con 0x7fbb700c5e40 2024-03-09T09:00:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.990+0000 7fbb75c00640 1 -- 172.21.15.67:0/1429475227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb701549e0 con 0x7fbb700c5e40 2024-03-09T09:00:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.992+0000 7fbb75c00640 1 -- 172.21.15.67:0/1429475227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7004f3c0 con 0x7fbb700c5e40 2024-03-09T09:00:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.993+0000 7fbb67e00640 1 -- 172.21.15.67:0/1429475227 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbb6001a070 con 0x7fbb700c5e40 2024-03-09T09:00:27.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.994+0000 7fbb67e00640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb5006b500 0x7fbb5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:27.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.994+0000 7fbb67e00640 1 -- 172.21.15.67:0/1429475227 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbb600897e0 con 0x7fbb700c5e40 2024-03-09T09:00:27.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.994+0000 7fbb6f400640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb5006b500 0x7fbb5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:27.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.995+0000 7fbb6f400640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb5006b500 0x7fbb5006d9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c036000 tx=0x7fbb5c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:27.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:27.999+0000 7fbb67e00640 1 -- 172.21.15.67:0/1429475227 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbb60057c10 con 0x7fbb700c5e40 2024-03-09T09:00:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.337+0000 7fbb75c00640 1 -- 172.21.15.67:0/1429475227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbb700af5b0 con 0x7fbb700c5e40 2024-03-09T09:00:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.337+0000 7fbb67e00640 1 -- 172.21.15.67:0/1429475227 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fbb6005bcb0 con 0x7fbb700c5e40 2024-03-09T09:00:28.339 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":64,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 -- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb5006b500 msgr2=0x7fbb5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb5006b500 0x7fbb5006d9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c036000 tx=0x7fbb5c03b040 comp rx=0 tx=0).stop 2024-03-09T09:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 -- 172.21.15.67:0/1429475227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb700c5e40 msgr2=0x7fbb701544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb700c5e40 0x7fbb701544a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000c4b0 tx=0x7fbb6000c970 comp rx=0 tx=0).stop 2024-03-09T09:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 -- 172.21.15.67:0/1429475227 shutdown_connections 2024-03-09T09:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb5006b500 0x7fbb5006d9c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:28.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb70153300 0x7fbb7014c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb700cadb0 0x7fbb7014be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 --2- 172.21.15.67:0/1429475227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb700c5e40 0x7fbb701544a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 -- 172.21.15.67:0/1429475227 >> 172.21.15.67:0/1429475227 conn(0x7fbb700732d0 msgr2=0x7fbb701520b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 -- 172.21.15.67:0/1429475227 shutdown_connections 2024-03-09T09:00:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:28.340+0000 7fbb75c00640 1 -- 172.21.15.67:0/1429475227 wait complete. 2024-03-09T09:00:31.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:00:31.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.603+0000 7f99c6c00640 1 Processor -- start 2024-03-09T09:00:31.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.604+0000 7f99c6c00640 1 -- start start 2024-03-09T09:00:31.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.605+0000 7f99c6c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c0153300 0x7f99c00b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:31.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.605+0000 7f99c6c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f99c00c5e40 0x7f99c00b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:31.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.605+0000 7f99c6c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c0157000 con 0x7f99c00cadb0 2024-03-09T09:00:31.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.605+0000 7f99c6c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c00ae430 con 0x7f99c00c5e40 2024-03-09T09:00:31.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.605+0000 7f99c6c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00ae6f0 con 0x7f99c0153300 2024-03-09T09:00:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.606+0000 7f99c4e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f99c00c5e40 0x7f99c00b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.606+0000 7f99c5800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c0153300 0x7f99c00b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.606+0000 7f99c4e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f99c00c5e40 0x7f99c00b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:41826/0 (socket says 172.21.15.67:41826) 2024-03-09T09:00:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.606+0000 7f99c4e00640 1 -- 172.21.15.67:0/1948906768 learned_addr learned my addr 172.21.15.67:0/1948906768 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.606+0000 7f99c6200640 1 --1- 172.21.15.67:0/1948906768 >> v1:172.21.15.134:6789/0 conn(0x7f99c00cadb0 0x7f99c0156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:54886/0 (socket says 172.21.15.67:54886) 2024-03-09T09:00:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.607+0000 7f99c5800640 1 -- 172.21.15.67:0/1948906768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c0153300 msgr2=0x7f99c00b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:00:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.607+0000 7f99c5800640 1 -- 172.21.15.67:0/1948906768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c0153300 msgr2=0x7f99c00b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.607+0000 7f99c5800640 1 --2- 172.21.15.67:0/1948906768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c0153300 0x7f99c00b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.607+0000 7f99c5800640 1 --2- 172.21.15.67:0/1948906768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c0153300 0x7f99c00b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.607+0000 7f99c4e00640 1 -- 172.21.15.67:0/1948906768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c0153300 msgr2=0x7f99c00b0000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:00:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.607+0000 7f99c4e00640 1 --2- 172.21.15.67:0/1948906768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c0153300 0x7f99c00b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.607+0000 7f99c4e00640 1 -- 172.21.15.67:0/1948906768 >> v1:172.21.15.134:6789/0 conn(0x7f99c00cadb0 legacy=0x7f99c0156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.607+0000 7f99c4e00640 1 -- 172.21.15.67:0/1948906768 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0155b10 con 0x7f99c00c5e40 2024-03-09T09:00:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.609+0000 7f99c4e00640 1 --2- 172.21.15.67:0/1948906768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c00c5e40 0x7f99c00b0540 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009870 tx=0x7f99b002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4d94788166c9fb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:31.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.609+0000 7f99bfe00640 1 -- 172.21.15.67:0/1948906768 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0034070 con 0x7f99c00c5e40 2024-03-09T09:00:31.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.609+0000 7f99bfe00640 1 -- 172.21.15.67:0/1948906768 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b0030840 con 0x7f99c00c5e40 2024-03-09T09:00:31.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.609+0000 7f99bfe00640 1 -- 172.21.15.67:0/1948906768 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0030c10 con 0x7f99c00c5e40 2024-03-09T09:00:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.610+0000 7f99c6c00640 1 -- 172.21.15.67:0/1948906768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c00c5e40 msgr2=0x7f99c00b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.610+0000 7f99c6c00640 1 --2- 172.21.15.67:0/1948906768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c00c5e40 0x7f99c00b0540 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009870 tx=0x7f99b002b980 comp rx=0 tx=0).stop 2024-03-09T09:00:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.610+0000 7f99c6c00640 1 -- 172.21.15.67:0/1948906768 shutdown_connections 2024-03-09T09:00:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.610+0000 7f99c6c00640 1 --2- 172.21.15.67:0/1948906768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c00c5e40 0x7f99c00b0540 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:31.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.610+0000 7f99c6c00640 1 --2- 172.21.15.67:0/1948906768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c0153300 0x7f99c00b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:31.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.610+0000 7f99c6c00640 1 -- 172.21.15.67:0/1948906768 >> 172.21.15.67:0/1948906768 conn(0x7f99c00732d0 msgr2=0x7f99c00ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:31.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.610+0000 7f99c6c00640 1 -- 172.21.15.67:0/1948906768 shutdown_connections 2024-03-09T09:00:31.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.610+0000 7f99c6c00640 1 -- 172.21.15.67:0/1948906768 wait complete. 2024-03-09T09:00:31.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.612+0000 7f99c6c00640 1 Processor -- start 2024-03-09T09:00:31.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.612+0000 7f99c6c00640 1 -- start start 2024-03-09T09:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.613+0000 7f99c6c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c00c5e40 0x7f99c01ee940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.613+0000 7f99c6c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c00cadb0 0x7f99c0154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.613+0000 7f99c6c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c0153300 0x7f99c006b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.613+0000 7f99c6c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00c59e0 con 0x7f99c0153300 2024-03-09T09:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.613+0000 7f99c6c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0159e90 con 0x7f99c00cadb0 2024-03-09T09:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.613+0000 7f99c6c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0156720 con 0x7f99c00c5e40 2024-03-09T09:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c0153300 0x7f99c006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c0153300 0x7f99c006b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38044/0 (socket says 172.21.15.67:38044) 2024-03-09T09:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c6200640 1 -- 172.21.15.67:0/1732305603 learned_addr learned my addr 172.21.15.67:0/1732305603 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c5800640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c00c5e40 0x7f99c01ee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c4e00640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c00cadb0 0x7f99c0154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:31.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c5800640 1 -- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c00c5e40 msgr2=0x7f99c01ee940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:00:31.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c5800640 1 -- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c00c5e40 msgr2=0x7f99c01ee940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:00:31.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c5800640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c00c5e40 0x7f99c01ee940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:31.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c6200640 1 -- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c00c5e40 msgr2=0x7f99c01ee940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:31.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c5800640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c00c5e40 0x7f99c01ee940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c6200640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c00c5e40 0x7f99c01ee940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c6200640 1 -- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c00cadb0 msgr2=0x7f99c0154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c6200640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c00cadb0 0x7f99c0154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.614+0000 7f99c6200640 1 -- 172.21.15.67:0/1732305603 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c01eee80 con 0x7f99c0153300 2024-03-09T09:00:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.615+0000 7f99c6200640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c0153300 0x7f99c006b120 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c780 tx=0x7f99b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.615+0000 7f99bd600640 1 -- 172.21.15.67:0/1732305603 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8017070 con 0x7f99c0153300 2024-03-09T09:00:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.615+0000 7f99bd600640 1 -- 172.21.15.67:0/1732305603 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b800a640 con 0x7f99c0153300 2024-03-09T09:00:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.615+0000 7f99bd600640 1 -- 172.21.15.67:0/1732305603 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b800aa30 con 0x7f99c0153300 2024-03-09T09:00:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.615+0000 7f99c6c00640 1 -- 172.21.15.67:0/1732305603 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c00c64e0 con 0x7f99c0153300 2024-03-09T09:00:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.615+0000 7f99c6c00640 1 -- 172.21.15.67:0/1732305603 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c01553e0 con 0x7f99c0153300 2024-03-09T09:00:31.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.616+0000 7f99c6c00640 1 -- 172.21.15.67:0/1732305603 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c004f3c0 con 0x7f99c0153300 2024-03-09T09:00:31.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.619+0000 7f99bd600640 1 -- 172.21.15.67:0/1732305603 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f99b801a070 con 0x7f99c0153300 2024-03-09T09:00:31.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.620+0000 7f99bd600640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99a406b500 0x7f99a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:31.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.620+0000 7f99bd600640 1 -- 172.21.15.67:0/1732305603 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f99b8089a20 con 0x7f99c0153300 2024-03-09T09:00:31.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.620+0000 7f99c5800640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99a406b500 0x7f99a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:31.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.622+0000 7f99c5800640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99a406b500 0x7f99a406d9c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f99c00c6ba0 tx=0x7f99b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:31.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.624+0000 7f99bd600640 1 -- 172.21.15.67:0/1732305603 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f99b8057ef0 con 0x7f99c0153300 2024-03-09T09:00:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.939+0000 7f99c6c00640 1 -- 172.21.15.67:0/1732305603 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f99c00cab90 con 0x7f99c0153300 2024-03-09T09:00:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.940+0000 7f99bd600640 1 -- 172.21.15.67:0/1732305603 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f99b805bf90 con 0x7f99c0153300 2024-03-09T09:00:31.941 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":64,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:00:31.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 -- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99a406b500 msgr2=0x7f99a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99a406b500 0x7f99a406d9c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f99c00c6ba0 tx=0x7f99b4008040 comp rx=0 tx=0).stop 2024-03-09T09:00:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 -- 172.21.15.67:0/1732305603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c0153300 msgr2=0x7f99c006b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c0153300 0x7f99c006b120 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c780 tx=0x7f99b800cc40 comp rx=0 tx=0).stop 2024-03-09T09:00:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 -- 172.21.15.67:0/1732305603 shutdown_connections 2024-03-09T09:00:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99a406b500 0x7f99a406d9c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c0153300 0x7f99c006b120 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c00cadb0 0x7f99c0154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 --2- 172.21.15.67:0/1732305603 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c00c5e40 0x7f99c01ee940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 -- 172.21.15.67:0/1732305603 >> 172.21.15.67:0/1732305603 conn(0x7f99c00732d0 msgr2=0x7f99c00aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 -- 172.21.15.67:0/1732305603 shutdown_connections 2024-03-09T09:00:31.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:31.943+0000 7f99c6c00640 1 -- 172.21.15.67:0/1732305603 wait complete. 2024-03-09T09:00:34.965 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:00:35.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.177+0000 7fe9be600640 1 Processor -- start 2024-03-09T09:00:35.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.177+0000 7fe9be600640 1 -- start start 2024-03-09T09:00:35.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.178+0000 7fe9be600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe9b81531c0 0x7fe9b81570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.178+0000 7fe9be600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9b80cae50 0x7fe9b80ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.178+0000 7fe9be600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b8156700 con 0x7fe9b80c9440 2024-03-09T09:00:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.178+0000 7fe9be600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b80afba0 con 0x7fe9b80cae50 2024-03-09T09:00:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.178+0000 7fe9be600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b80b01c0 con 0x7fe9b81531c0 2024-03-09T09:00:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.179+0000 7fe9b7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9b80cae50 0x7fe9b80ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.179+0000 7fe9b7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9b80cae50 0x7fe9b80ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38058/0 (socket says 172.21.15.67:38058) 2024-03-09T09:00:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.179+0000 7fe9b7e00640 1 -- 172.21.15.67:0/3977072248 learned_addr learned my addr 172.21.15.67:0/3977072248 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.179+0000 7fe9bd200640 1 --2- 172.21.15.67:0/3977072248 >> v2:172.21.15.134:3300/0 conn(0x7fe9b81531c0 0x7fe9b81570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.179+0000 7fe9b7e00640 1 -- 172.21.15.67:0/3977072248 >> v2:172.21.15.134:3300/0 conn(0x7fe9b81531c0 msgr2=0x7fe9b81570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.179+0000 7fe9b7e00640 1 --2- 172.21.15.67:0/3977072248 >> v2:172.21.15.134:3300/0 conn(0x7fe9b81531c0 0x7fe9b81570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.179+0000 7fe9b7e00640 1 -- 172.21.15.67:0/3977072248 >> v1:172.21.15.134:6789/0 conn(0x7fe9b80c9440 legacy=0x7fe9b80c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.179+0000 7fe9b7e00640 1 -- 172.21.15.67:0/3977072248 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b81569a0 con 0x7fe9b80cae50 2024-03-09T09:00:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.179+0000 7fe9bd200640 1 --2- 172.21.15.67:0/3977072248 >> v2:172.21.15.134:3300/0 conn(0x7fe9b81531c0 0x7fe9b81570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:00:35.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.180+0000 7fe9b7e00640 1 --2- 172.21.15.67:0/3977072248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9b80cae50 0x7fe9b80ae590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80b0340 tx=0x7fe9a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8041fb32d7889c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:35.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.181+0000 7fe9b7400640 1 -- 172.21.15.67:0/3977072248 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a402e070 con 0x7fe9b80cae50 2024-03-09T09:00:35.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.181+0000 7fe9b7400640 1 -- 172.21.15.67:0/3977072248 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a4037070 con 0x7fe9b80cae50 2024-03-09T09:00:35.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.181+0000 7fe9b7400640 1 -- 172.21.15.67:0/3977072248 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4031a00 con 0x7fe9b80cae50 2024-03-09T09:00:35.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.182+0000 7fe9be600640 1 -- 172.21.15.67:0/3977072248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9b80cae50 msgr2=0x7fe9b80ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.182+0000 7fe9be600640 1 --2- 172.21.15.67:0/3977072248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9b80cae50 0x7fe9b80ae590 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80b0340 tx=0x7fe9a402b880 comp rx=0 tx=0).stop 2024-03-09T09:00:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.182+0000 7fe9be600640 1 -- 172.21.15.67:0/3977072248 shutdown_connections 2024-03-09T09:00:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.182+0000 7fe9be600640 1 --2- 172.21.15.67:0/3977072248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9b80cae50 0x7fe9b80ae590 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.182+0000 7fe9be600640 1 --2- 172.21.15.67:0/3977072248 >> v2:172.21.15.134:3300/0 conn(0x7fe9b81531c0 0x7fe9b81570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.182+0000 7fe9be600640 1 -- 172.21.15.67:0/3977072248 >> 172.21.15.67:0/3977072248 conn(0x7fe9b80ad190 msgr2=0x7fe9b80ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.182+0000 7fe9be600640 1 -- 172.21.15.67:0/3977072248 shutdown_connections 2024-03-09T09:00:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.182+0000 7fe9be600640 1 -- 172.21.15.67:0/3977072248 wait complete. 2024-03-09T09:00:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.186+0000 7fe9be600640 1 Processor -- start 2024-03-09T09:00:35.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.186+0000 7fe9be600640 1 -- start start 2024-03-09T09:00:35.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.188+0000 7fe9be600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b80c9440 0x7fe9b81df600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.188+0000 7fe9be600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9b80cae50 0x7fe9b8141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.188+0000 7fe9be600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9b81531c0 0x7fe9b8142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.188+0000 7fe9be600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b81535a0 con 0x7fe9b80cae50 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.188+0000 7fe9be600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b80b06f0 con 0x7fe9b81531c0 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.188+0000 7fe9be600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b80cb2b0 con 0x7fe9b80c9440 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.188+0000 7fe9b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9b80cae50 0x7fe9b8141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.188+0000 7fe9b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9b80cae50 0x7fe9b8141b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38072/0 (socket says 172.21.15.67:38072) 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.188+0000 7fe9b7e00640 1 -- 172.21.15.67:0/2166564600 learned_addr learned my addr 172.21.15.67:0/2166564600 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.189+0000 7fe9bd200640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b80c9440 0x7fe9b81df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.189+0000 7fe9bdc00640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9b81531c0 0x7fe9b8142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.189+0000 7fe9b7e00640 1 -- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b80c9440 msgr2=0x7fe9b81df600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.189+0000 7fe9b7e00640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b80c9440 0x7fe9b81df600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.189+0000 7fe9b7e00640 1 -- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9b81531c0 msgr2=0x7fe9b8142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.189+0000 7fe9b7e00640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9b81531c0 0x7fe9b8142060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.189+0000 7fe9b7e00640 1 -- 172.21.15.67:0/2166564600 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b81425a0 con 0x7fe9b80cae50 2024-03-09T09:00:35.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.189+0000 7fe9bdc00640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9b81531c0 0x7fe9b8142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:00:35.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.189+0000 7fe9b7e00640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9b80cae50 0x7fe9b8141b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4000c00 tx=0x7fe9a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:35.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.190+0000 7fe9b4c00640 1 -- 172.21.15.67:0/2166564600 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4032880 con 0x7fe9b80cae50 2024-03-09T09:00:35.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.190+0000 7fe9b4c00640 1 -- 172.21.15.67:0/2166564600 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a4032a20 con 0x7fe9b80cae50 2024-03-09T09:00:35.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.190+0000 7fe9b4c00640 1 -- 172.21.15.67:0/2166564600 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4032e10 con 0x7fe9b80cae50 2024-03-09T09:00:35.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.190+0000 7fe9be600640 1 -- 172.21.15.67:0/2166564600 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b806b0e0 con 0x7fe9b80cae50 2024-03-09T09:00:35.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.190+0000 7fe9be600640 1 -- 172.21.15.67:0/2166564600 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b81ec790 con 0x7fe9b80cae50 2024-03-09T09:00:35.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.191+0000 7fe9be600640 1 -- 172.21.15.67:0/2166564600 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b80af990 con 0x7fe9b80cae50 2024-03-09T09:00:35.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.192+0000 7fe9b4c00640 1 -- 172.21.15.67:0/2166564600 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe9a404c020 con 0x7fe9b80cae50 2024-03-09T09:00:35.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.193+0000 7fe9b4c00640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe99806b4d0 0x7fe99806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.193+0000 7fe9b4c00640 1 -- 172.21.15.67:0/2166564600 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe9a40ac530 con 0x7fe9b80cae50 2024-03-09T09:00:35.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.194+0000 7fe9bd200640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe99806b4d0 0x7fe99806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.199+0000 7fe9b4c00640 1 -- 172.21.15.67:0/2166564600 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe9b80af990 con 0x7fe9b80cae50 2024-03-09T09:00:35.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.201+0000 7fe9bd200640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe99806b4d0 0x7fe99806d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00a950 tx=0x7fe9ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:35.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.476+0000 7fe9be600640 1 -- 172.21.15.67:0/2166564600 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe9b8075a00 con 0x7fe9b80cae50 2024-03-09T09:00:35.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.477+0000 7fe9b4c00640 1 -- 172.21.15.67:0/2166564600 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7fe9b8075a00 con 0x7fe9b80cae50 2024-03-09T09:00:35.478 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:00:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.479+0000 7fe9be600640 1 -- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe99806b4d0 msgr2=0x7fe99806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.479+0000 7fe9be600640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe99806b4d0 0x7fe99806d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00a950 tx=0x7fe9ac008040 comp rx=0 tx=0).stop 2024-03-09T09:00:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.479+0000 7fe9be600640 1 -- 172.21.15.67:0/2166564600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9b80cae50 msgr2=0x7fe9b8141b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.479+0000 7fe9be600640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9b80cae50 0x7fe9b8141b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4000c00 tx=0x7fe9a40046c0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.479+0000 7fe9be600640 1 -- 172.21.15.67:0/2166564600 shutdown_connections 2024-03-09T09:00:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.479+0000 7fe9be600640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe99806b4d0 0x7fe99806d990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.479+0000 7fe9be600640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9b81531c0 0x7fe9b8142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.479+0000 7fe9be600640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9b80cae50 0x7fe9b8141b20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.479+0000 7fe9be600640 1 --2- 172.21.15.67:0/2166564600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9b80c9440 0x7fe9b81df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.479+0000 7fe9be600640 1 -- 172.21.15.67:0/2166564600 >> 172.21.15.67:0/2166564600 conn(0x7fe9b80ad190 msgr2=0x7fe9b8151f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:35.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.480+0000 7fe9be600640 1 -- 172.21.15.67:0/2166564600 shutdown_connections 2024-03-09T09:00:35.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.480+0000 7fe9be600640 1 -- 172.21.15.67:0/2166564600 wait complete. 2024-03-09T09:00:35.502 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:00:35.502 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:00:35.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:00:35.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.722+0000 7f4a12200640 1 Processor -- start 2024-03-09T09:00:35.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.723+0000 7f4a12200640 1 -- start start 2024-03-09T09:00:35.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.724+0000 7f4a12200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0c0acf30 0x7f4a0c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.724+0000 7f4a12200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4a0c06cbf0 0x7f4a0c0c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.724+0000 7f4a12200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a0c06d270 con 0x7f4a0c0c6ee0 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.724+0000 7f4a12200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a0c0c3e70 con 0x7f4a0c06cbf0 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.724+0000 7f4a12200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c0c35c0 con 0x7f4a0c0acf30 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.724+0000 7f4a0be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4a0c06cbf0 0x7f4a0c0c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.724+0000 7f4a0be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4a0c06cbf0 0x7f4a0c0c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38084/0 (socket says 172.21.15.67:38084) 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.725+0000 7f4a0be00640 1 -- 172.21.15.67:0/2652986328 learned_addr learned my addr 172.21.15.67:0/2652986328 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.725+0000 7f4a10e00640 1 --2- 172.21.15.67:0/2652986328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0c0acf30 0x7f4a0c06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.725+0000 7f4a0be00640 1 -- 172.21.15.67:0/2652986328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0c0acf30 msgr2=0x7f4a0c06d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.725+0000 7f4a0be00640 1 --2- 172.21.15.67:0/2652986328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0c0acf30 0x7f4a0c06d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.725+0000 7f4a0be00640 1 -- 172.21.15.67:0/2652986328 >> v1:172.21.15.134:6789/0 conn(0x7f4a0c0c6ee0 legacy=0x7f4a0c1560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:35.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.725+0000 7f4a0be00640 1 -- 172.21.15.67:0/2652986328 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c067360 con 0x7f4a0c06cbf0 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.725+0000 7f4a10e00640 1 --2- 172.21.15.67:0/2652986328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0c0acf30 0x7f4a0c06d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.726+0000 7f4a0be00640 1 --2- 172.21.15.67:0/2652986328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c06cbf0 0x7f4a0c0c69a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009870 tx=0x7f49f802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104764b8bd2dd92e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.727+0000 7f4a0b400640 1 -- 172.21.15.67:0/2652986328 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f80317c0 con 0x7f4a0c06cbf0 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.727+0000 7f4a0b400640 1 -- 172.21.15.67:0/2652986328 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f8037070 con 0x7f4a0c06cbf0 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.727+0000 7f4a0b400640 1 -- 172.21.15.67:0/2652986328 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8031b80 con 0x7f4a0c06cbf0 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.727+0000 7f4a12200640 1 -- 172.21.15.67:0/2652986328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c06cbf0 msgr2=0x7f4a0c0c69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.728+0000 7f4a12200640 1 --2- 172.21.15.67:0/2652986328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c06cbf0 0x7f4a0c0c69a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009870 tx=0x7f49f802b860 comp rx=0 tx=0).stop 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.728+0000 7f4a12200640 1 -- 172.21.15.67:0/2652986328 shutdown_connections 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.728+0000 7f4a12200640 1 --2- 172.21.15.67:0/2652986328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c06cbf0 0x7f4a0c0c69a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.728+0000 7f4a12200640 1 --2- 172.21.15.67:0/2652986328 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0c0acf30 0x7f4a0c06d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.728+0000 7f4a12200640 1 -- 172.21.15.67:0/2652986328 >> 172.21.15.67:0/2652986328 conn(0x7f4a0c0a9740 msgr2=0x7f4a0c0a9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.728+0000 7f4a12200640 1 -- 172.21.15.67:0/2652986328 shutdown_connections 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.728+0000 7f4a12200640 1 -- 172.21.15.67:0/2652986328 wait complete. 2024-03-09T09:00:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.729+0000 7f4a12200640 1 Processor -- start 2024-03-09T09:00:35.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.730+0000 7f4a12200640 1 -- start start 2024-03-09T09:00:35.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.730+0000 7f4a12200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0c06cbf0 0x7f4a0c0c1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.730+0000 7f4a12200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c0acf30 0x7f4a0c1eed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.730+0000 7f4a12200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c0c6ee0 0x7f4a0c1ef260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.730+0000 7f4a12200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c0ad310 con 0x7f4a0c0c6ee0 2024-03-09T09:00:35.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.730+0000 7f4a12200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c0c38a0 con 0x7f4a0c0acf30 2024-03-09T09:00:35.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.730+0000 7f4a12200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c1567c0 con 0x7f4a0c06cbf0 2024-03-09T09:00:35.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.731+0000 7f4a11800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c0c6ee0 0x7f4a0c1ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.731+0000 7f4a11800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c0c6ee0 0x7f4a0c1ef260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38094/0 (socket says 172.21.15.67:38094) 2024-03-09T09:00:35.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.731+0000 7f4a11800640 1 -- 172.21.15.67:0/1366331023 learned_addr learned my addr 172.21.15.67:0/1366331023 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:35.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.731+0000 7f4a0be00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c0acf30 0x7f4a0c1eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.731+0000 7f4a10e00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0c06cbf0 0x7f4a0c0c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.731+0000 7f4a11800640 1 -- 172.21.15.67:0/1366331023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0c06cbf0 msgr2=0x7f4a0c0c1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.731+0000 7f4a11800640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4a0c06cbf0 0x7f4a0c0c1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.731+0000 7f4a11800640 1 -- 172.21.15.67:0/1366331023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c0acf30 msgr2=0x7f4a0c1eed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.731+0000 7f4a11800640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a0c0acf30 0x7f4a0c1eed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:35.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.731+0000 7f4a11800640 1 -- 172.21.15.67:0/1366331023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c149870 con 0x7f4a0c0c6ee0 2024-03-09T09:00:35.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.732+0000 7f4a11800640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c0c6ee0 0x7f4a0c1ef260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400b970 tx=0x7f4a0400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:35.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.732+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04019070 con 0x7f4a0c0c6ee0 2024-03-09T09:00:35.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.732+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0400cb40 con 0x7f4a0c0c6ee0 2024-03-09T09:00:35.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.732+0000 7f4a12200640 1 -- 172.21.15.67:0/1366331023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c149590 con 0x7f4a0c0c6ee0 2024-03-09T09:00:35.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.732+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04014440 con 0x7f4a0c0c6ee0 2024-03-09T09:00:35.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.732+0000 7f4a12200640 1 -- 172.21.15.67:0/1366331023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0c072120 con 0x7f4a0c0c6ee0 2024-03-09T09:00:35.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.733+0000 7f4a12200640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c146ff0 0x7f4a0c145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.733+0000 7f4a12200640 1 -- 172.21.15.67:0/1366331023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c04f3c0 con 0x7f4a0c146ff0 2024-03-09T09:00:35.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.734+0000 7f4a10e00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c146ff0 0x7f4a0c145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.734+0000 7f4a10e00640 1 -- 172.21.15.67:0/1366331023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a0c066fb0 con 0x7f4a0c146ff0 2024-03-09T09:00:35.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.734+0000 7f4a10e00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c146ff0 0x7f4a0c145f70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000f480 tx=0x7f4a0000f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:35.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.735+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00004230 con 0x7f4a0c146ff0 2024-03-09T09:00:35.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.735+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4a000043d0 con 0x7f4a0c146ff0 2024-03-09T09:00:35.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.735+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c146ff0 msgr2=0x7f4a0c145f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.735+0000 7f4a08c00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c146ff0 0x7f4a0c145f70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000f480 tx=0x7f4a0000f940 comp rx=0 tx=0).stop 2024-03-09T09:00:35.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.735+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c146ff0 msgr2=0x7f4a0c145f70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:35.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.735+0000 7f4a08c00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c146ff0 0x7f4a0c145f70 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000f480 tx=0x7f4a0000f940 comp rx=0 tx=0).stop 2024-03-09T09:00:35.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.735+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4a0401c070 con 0x7f4a0c0c6ee0 2024-03-09T09:00:35.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.736+0000 7f4a08c00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f49ec06b0d0 0x7f49ec06d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.736+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4a0408a120 con 0x7f4a0c0c6ee0 2024-03-09T09:00:35.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.737+0000 7f4a10e00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f49ec06b0d0 0x7f49ec06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.738+0000 7f4a10e00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f49ec06b0d0 0x7f49ec06d590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000c00 tx=0x7f4a00015000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:35.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.743+0000 7f4a12200640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c1464b0 0x7f4a0c146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:35.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.743+0000 7f4a12200640 1 -- 172.21.15.67:0/1366331023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c0c33a0 con 0x7f4a0c1464b0 2024-03-09T09:00:35.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.743+0000 7f4a0be00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c1464b0 0x7f4a0c146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:35.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.743+0000 7f4a0be00640 1 -- 172.21.15.67:0/1366331023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4a0c06de80 con 0x7f4a0c1464b0 2024-03-09T09:00:35.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.744+0000 7f4a0be00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c1464b0 0x7f4a0c146890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0a7620 tx=0x7f49f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:35.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.744+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8032450 con 0x7f4a0c1464b0 2024-03-09T09:00:35.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.744+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5018 (secure 0 0 0) 0x7f49f8037040 con 0x7f4a0c1464b0 2024-03-09T09:00:35.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.744+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c1464b0 msgr2=0x7f4a0c146890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.744+0000 7f4a08c00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c1464b0 0x7f4a0c146890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0a7620 tx=0x7f49f8004730 comp rx=0 tx=0).stop 2024-03-09T09:00:35.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.744+0000 7f4a08c00640 1 -- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c1464b0 msgr2=0x7f4a0c146890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:35.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.744+0000 7f4a08c00640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c1464b0 0x7f4a0c146890 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0a7620 tx=0x7f49f8004730 comp rx=0 tx=0).stop 2024-03-09T09:00:35.751 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:00:35.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:00:35.752 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:00:35.752 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:00:35.752 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 66, 2024-03-09T09:00:35.752 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:00:35.752 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:00:35.752 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:00:35.753 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:00:35.753 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:35.753 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:00:35.753 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:35.753 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:00:35.753 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:00:35.754 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:35.754 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:35.754 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:35.754 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:35.754 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:35.754 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:35.755 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:35.755 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:35.755 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:35.755 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:35.755 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:35.755 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:35.756 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:00:35.756 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:00:35.756 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:35.756 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:35.756 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:35.756 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:35.756 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:35.757 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:35.757 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:35.757 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:35.757 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:35.757 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:35.758 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:35.758 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:35.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.758 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:00:35.758 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:00:35.758 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:00:35.759 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:00:35.759 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:00:35.760 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:00:35.760 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:35.760 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:35.760 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:00:35.760 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:00:35.760 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:00:35.761 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:00:35.761 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:00:35.761 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:00:35.761 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:00:35.761 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:35.761 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:00:35.762 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:35.762 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:35.762 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:35.763 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:35.763 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:35.763 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:35.763 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:35.763 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:35.763 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:35.764 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:35.764 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:35.764 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:35.764 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:00:35.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.764 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:00:35.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.765 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:00:35.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:00:35.765 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:35.765 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:35.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:35.766 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:00:35.766 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:35.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.766 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.767 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:35.767 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:00:35.767 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:35.767 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:35.767 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:35.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.768 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:00:35.768 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:00:35.768 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:35.768 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:35.768 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:35.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.768 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.769 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:00:35.769 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:00:35.769 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:35.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:35.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:35.770 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:00:35.770 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:35.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.770 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.770 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:35.771 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:00:35.771 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:35.771 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:35.771 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:35.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.771 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:00:35.772 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:00:35.772 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:35.772 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:35.772 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:35.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.773 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:00:35.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:00:35.773 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:35.773 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:35.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:35.774 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:00:35.774 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:35.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.774 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:35.774 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:00:35.775 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:35.775 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:35.775 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:35.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.775 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:00:35.775 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:00:35.776 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:35.776 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:35.776 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:35.776 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:35.776 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:35.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.777 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:00:35.777 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:00:35.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.777 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:35.777 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:35.777 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:35.777 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:35.778 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:35.778 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:00:35.778 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.778 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:35.778 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:35.778 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:35.779 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:35.779 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:35.779 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:00:35.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.779 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:35.779 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:35.779 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:35.780 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:35.780 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:35.780 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:00:35.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.780 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:00:35.781 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:35.781 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:35.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:35.782 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:35.782 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:35.782 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:35.782 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:35.782 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:35.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:35.782 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:00:35.782 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:00:35.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.745+0000 7f4a12200640 1 -- 172.21.15.67:0/1366331023 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f49ec06b0d0 msgr2=0x7f49ec06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.745+0000 7f4a12200640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f49ec06b0d0 0x7f49ec06d590 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4a00000c00 tx=0x7f4a00015000 comp rx=0 tx=0).stop 2024-03-09T09:00:35.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.745+0000 7f4a12200640 1 -- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c0c6ee0 msgr2=0x7f4a0c1ef260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:35.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.745+0000 7f4a12200640 1 --2- 172.21.15.67:0/1366331023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0c0c6ee0 0x7f4a0c1ef260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400b970 tx=0x7f4a0400be30 comp rx=0 tx=0).stop 2024-03-09T09:00:35.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.745+0000 7f4a11800640 1 -- 172.21.15.67:0/1366331023 reap_dead start 2024-03-09T09:00:35.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.745+0000 7f4a11800640 1 -- 172.21.15.67:0/1366331023 reap_dead start 2024-03-09T09:00:35.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.745+0000 7f4a12200640 1 -- 172.21.15.67:0/1366331023 shutdown_connections 2024-03-09T09:00:35.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.745+0000 7f4a12200640 1 -- 172.21.15.67:0/1366331023 >> 172.21.15.67:0/1366331023 conn(0x7f4a0c0a9740 msgr2=0x7f4a0c0abb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:35.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.745+0000 7f4a12200640 1 -- 172.21.15.67:0/1366331023 shutdown_connections 2024-03-09T09:00:35.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:35.745+0000 7f4a12200640 1 -- 172.21.15.67:0/1366331023 wait complete. 2024-03-09T09:00:35.787 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:00:35.787 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:00:35.787 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:00:36.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.079+0000 7fb590a00640 1 Processor -- start 2024-03-09T09:00:36.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.079+0000 7fb590a00640 1 -- start start 2024-03-09T09:00:36.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.080+0000 7fb590a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c0c9380 0x7fb58c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.080+0000 7fb590a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58c0c9890 con 0x7fb58c14fd90 2024-03-09T09:00:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.080+0000 7fb590a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58c0c5f80 con 0x7fb58c0ad440 2024-03-09T09:00:36.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.080+0000 7fb590a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c06ba60 con 0x7fb58c0c9380 2024-03-09T09:00:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.080+0000 7fb58b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb58c14fd90 0x7fb58c156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54946/0 (socket says 172.21.15.67:54946) 2024-03-09T09:00:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.080+0000 7fb58b400640 1 -- 172.21.15.67:0/1057275634 learned_addr learned my addr 172.21.15.67:0/1057275634 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:00:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.081+0000 7fb58aa00640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c0c9380 0x7fb58c0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.081+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570715590 0 0) 0x7fb58c0c9890 con 0x7fb58c14fd90 2024-03-09T09:00:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.081+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb570003520 con 0x7fb58c14fd90 2024-03-09T09:00:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1318286958 0 0) 0x7fb570003520 con 0x7fb58c14fd90 2024-03-09T09:00:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c0c9380 msgr2=0x7fb58c0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c0c9380 0x7fb58c0acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 >> v1:172.21.15.134:6789/0 conn(0x7fb58c0ad440 legacy=0x7fb58c06c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c0c6240 con 0x7fb58c14fd90 2024-03-09T09:00:36.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb57c001670 con 0x7fb58c14fd90 2024-03-09T09:00:36.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 >> v1:172.21.15.67:6789/0 conn(0x7fb58c14fd90 legacy=0x7fb58c156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb570003a50 0x7fb570003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb570004800 0x7fb570004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb570005230 0x7fb57000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c06ba60 con 0x7fb570005230 2024-03-09T09:00:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57000c140 con 0x7fb570003a50 2024-03-09T09:00:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.082+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57000c340 con 0x7fb570004800 2024-03-09T09:00:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.083+0000 7fb58be00640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb570005230 0x7fb57000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.083+0000 7fb58b400640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb570003a50 0x7fb570003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.083+0000 7fb58aa00640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb570004800 0x7fb570004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.083+0000 7fb58be00640 1 -- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb570004800 msgr2=0x7fb570004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.083+0000 7fb58be00640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb570004800 0x7fb570004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.083+0000 7fb58be00640 1 -- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb570003a50 msgr2=0x7fb570003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.083+0000 7fb58be00640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb570003a50 0x7fb570003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.083+0000 7fb58be00640 1 -- 172.21.15.67:0/1057275634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb570003520 con 0x7fb570005230 2024-03-09T09:00:36.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.083+0000 7fb58b400640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb570003a50 0x7fb570003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:00:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.084+0000 7fb58be00640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb570005230 0x7fb57000b9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb584001f60 tx=0x7fb570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a849435022b59c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.085+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584001b30 con 0x7fb570005230 2024-03-09T09:00:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.085+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58402d710 con 0x7fb570005230 2024-03-09T09:00:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.085+0000 7fb58a000640 1 -- 172.21.15.67:0/1057275634 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58402db00 con 0x7fb570005230 2024-03-09T09:00:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.085+0000 7fb590a00640 1 -- 172.21.15.67:0/1057275634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb570005230 msgr2=0x7fb57000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.085+0000 7fb590a00640 1 --2- 172.21.15.67:0/1057275634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb570005230 0x7fb57000b9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb584001f60 tx=0x7fb570005830 comp rx=0 tx=0).stop 2024-03-09T09:00:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.085+0000 7fb58be00640 1 -- 172.21.15.67:0/1057275634 reap_dead start 2024-03-09T09:00:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.086+0000 7fb590a00640 1 -- 172.21.15.67:0/1057275634 shutdown_connections 2024-03-09T09:00:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.086+0000 7fb590a00640 1 -- 172.21.15.67:0/1057275634 >> 172.21.15.67:0/1057275634 conn(0x7fb58c0cb480 msgr2=0x7fb58c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.086+0000 7fb590a00640 1 -- 172.21.15.67:0/1057275634 shutdown_connections 2024-03-09T09:00:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.086+0000 7fb590a00640 1 -- 172.21.15.67:0/1057275634 wait complete. 2024-03-09T09:00:36.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.088+0000 7fb590a00640 1 Processor -- start 2024-03-09T09:00:36.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.088+0000 7fb590a00640 1 -- start start 2024-03-09T09:00:36.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.089+0000 7fb590a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb570005230 0x7fb58c0ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.089+0000 7fb590a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c152d40 0x7fb58c0b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.089+0000 7fb590a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c0b0550 0x7fb58c14b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.089+0000 7fb590a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0acf00 con 0x7fb570005230 2024-03-09T09:00:36.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.089+0000 7fb590a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c06bcb0 con 0x7fb58c0b0550 2024-03-09T09:00:36.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.089+0000 7fb590a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0ad080 con 0x7fb58c152d40 2024-03-09T09:00:36.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.089+0000 7fb58b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb570005230 0x7fb58c0ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.089+0000 7fb58b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb570005230 0x7fb58c0ac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38114/0 (socket says 172.21.15.67:38114) 2024-03-09T09:00:36.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.089+0000 7fb58b400640 1 -- 172.21.15.67:0/1216157320 learned_addr learned my addr 172.21.15.67:0/1216157320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:36.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.089+0000 7fb58aa00640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c152d40 0x7fb58c0b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.090+0000 7fb58b400640 1 -- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c152d40 msgr2=0x7fb58c0b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.090+0000 7fb58b400640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb58c152d40 0x7fb58c0b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.090+0000 7fb58b400640 1 -- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c0b0550 msgr2=0x7fb58c14b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.090+0000 7fb58b400640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c0b0550 0x7fb58c14b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.090+0000 7fb58b400640 1 -- 172.21.15.67:0/1216157320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c14c410 con 0x7fb570005230 2024-03-09T09:00:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.090+0000 7fb58b400640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb570005230 0x7fb58c0ac370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb57c001200 tx=0x7fb57c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.090+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00ee60 con 0x7fb570005230 2024-03-09T09:00:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.090+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c018070 con 0x7fb570005230 2024-03-09T09:00:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.090+0000 7fb590a00640 1 -- 172.21.15.67:0/1216157320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58c1ef360 con 0x7fb570005230 2024-03-09T09:00:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.090+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00c9c0 con 0x7fb570005230 2024-03-09T09:00:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.090+0000 7fb590a00640 1 -- 172.21.15.67:0/1216157320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58c0ac8b0 con 0x7fb570005230 2024-03-09T09:00:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.092+0000 7fb590a00640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c150170 0x7fb58c14b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.092+0000 7fb590a00640 1 -- 172.21.15.67:0/1216157320 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0b0ba0 con 0x7fb58c150170 2024-03-09T09:00:36.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.092+0000 7fb58aa00640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c150170 0x7fb58c14b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.093+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb57c00cb60 con 0x7fb570005230 2024-03-09T09:00:36.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.093+0000 7fb58aa00640 1 -- 172.21.15.67:0/1216157320 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb58c14c1c0 con 0x7fb58c150170 2024-03-09T09:00:36.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.093+0000 7fb58aa00640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c150170 0x7fb58c14b9b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb58c1531a0 tx=0x7fb578014d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.094+0000 7fb583400640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb56c06b4d0 0x7fb56c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.094+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb57c08a330 con 0x7fb570005230 2024-03-09T09:00:36.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.094+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578004440 con 0x7fb58c150170 2024-03-09T09:00:36.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.094+0000 7fb58be00640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb56c06b4d0 0x7fb56c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.094+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb578004440 con 0x7fb58c150170 2024-03-09T09:00:36.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.095+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c150170 msgr2=0x7fb58c14b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.095+0000 7fb583400640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c150170 0x7fb58c14b9b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb58c1531a0 tx=0x7fb578014d80 comp rx=0 tx=0).stop 2024-03-09T09:00:36.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.095+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c150170 msgr2=0x7fb58c14b9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:36.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.095+0000 7fb583400640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c150170 0x7fb58c14b9b0 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb58c1531a0 tx=0x7fb578014d80 comp rx=0 tx=0).stop 2024-03-09T09:00:36.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.096+0000 7fb58be00640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb56c06b4d0 0x7fb56c06d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb584036000 tx=0x7fb584009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.102+0000 7fb590a00640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c14c6f0 0x7fb58c0ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.102+0000 7fb590a00640 1 -- 172.21.15.67:0/1216157320 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c04f3c0 con 0x7fb58c14c6f0 2024-03-09T09:00:36.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.102+0000 7fb58aa00640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c14c6f0 0x7fb58c0ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.103+0000 7fb58aa00640 1 -- 172.21.15.67:0/1216157320 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb58c06d2d0 con 0x7fb58c14c6f0 2024-03-09T09:00:36.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.104+0000 7fb58aa00640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c14c6f0 0x7fb58c0ae060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb57801d000 tx=0x7fb578004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.104+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57802d020 con 0x7fb58c14c6f0 2024-03-09T09:00:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5186 (secure 0 0 0) 0x7fb578003db0 con 0x7fb58c14c6f0 2024-03-09T09:00:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c14c6f0 msgr2=0x7fb58c0ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb583400640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c14c6f0 0x7fb58c0ae060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb57801d000 tx=0x7fb578004730 comp rx=0 tx=0).stop 2024-03-09T09:00:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb583400640 1 -- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c14c6f0 msgr2=0x7fb58c0ae060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb583400640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb58c14c6f0 0x7fb58c0ae060 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb57801d000 tx=0x7fb578004730 comp rx=0 tx=0).stop 2024-03-09T09:00:36.107 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:00:36.107 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:00:36.107 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:00:36.107 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:00:36.107 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 66, 2024-03-09T09:00:36.107 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:00:36.107 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:00:36.107 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:00:36.107 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:00:36.107 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:36.108 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:36.109 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:36.111 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:36.111 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.111 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:00:36.111 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:00:36.111 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:00:36.111 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:00:36.111 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:00:36.111 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:00:36.111 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:36.111 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:36.112 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:36.113 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.114 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:00:36.115 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:36.116 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:00:36.116 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.116 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.116 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.116 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:00:36.116 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:00:36.116 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:36.117 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:36.117 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:36.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.117 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:00:36.117 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:00:36.117 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:36.117 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:36.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.117 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:00:36.118 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:36.119 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.120 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:36.121 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:00:36.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.121 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:00:36.121 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:36.121 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:00:36.123 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:00:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb590a00640 1 -- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb56c06b4d0 msgr2=0x7fb56c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb590a00640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb56c06b4d0 0x7fb56c06d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb584036000 tx=0x7fb584009030 comp rx=0 tx=0).stop 2024-03-09T09:00:36.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb590a00640 1 -- 172.21.15.67:0/1216157320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb570005230 msgr2=0x7fb58c0ac370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb590a00640 1 --2- 172.21.15.67:0/1216157320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb570005230 0x7fb58c0ac370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb57c001200 tx=0x7fb57c00ec10 comp rx=0 tx=0).stop 2024-03-09T09:00:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb58be00640 1 -- 172.21.15.67:0/1216157320 reap_dead start 2024-03-09T09:00:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb58be00640 1 -- 172.21.15.67:0/1216157320 reap_dead start 2024-03-09T09:00:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb590a00640 1 -- 172.21.15.67:0/1216157320 shutdown_connections 2024-03-09T09:00:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.105+0000 7fb590a00640 1 -- 172.21.15.67:0/1216157320 >> 172.21.15.67:0/1216157320 conn(0x7fb58c0cb480 msgr2=0x7fb58c14dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.106+0000 7fb590a00640 1 -- 172.21.15.67:0/1216157320 shutdown_connections 2024-03-09T09:00:36.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.106+0000 7fb590a00640 1 -- 172.21.15.67:0/1216157320 wait complete. 2024-03-09T09:00:36.128 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:00:36.128 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:00:36.128 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:00:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.338+0000 7fdddde00640 1 Processor -- start 2024-03-09T09:00:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.338+0000 7fdddde00640 1 -- start start 2024-03-09T09:00:36.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.339+0000 7fdddde00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80c5e00 0x7fddd80acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.339+0000 7fdddde00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fddd80c61e0 0x7fddd80c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.340+0000 7fdddde00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fddd8156660 0x7fddd806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.340+0000 7fdddde00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd806d740 con 0x7fddd8156660 2024-03-09T09:00:36.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.340+0000 7fdddde00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd80c6660 con 0x7fddd80c61e0 2024-03-09T09:00:36.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.340+0000 7fdddde00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd8153680 con 0x7fddd80c5e00 2024-03-09T09:00:36.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.340+0000 7fdddd400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fddd8156660 0x7fddd806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.340+0000 7fdddd400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fddd8156660 0x7fddd806c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38120/0 (socket says 172.21.15.67:38120) 2024-03-09T09:00:36.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.340+0000 7fdddd400640 1 -- 172.21.15.67:0/4273501320 learned_addr learned my addr 172.21.15.67:0/4273501320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:36.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.340+0000 7fdddca00640 1 --2- 172.21.15.67:0/4273501320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80c5e00 0x7fddd80acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.340+0000 7fddd7e00640 1 --2- 172.21.15.67:0/4273501320 >> v2:172.21.15.134:3300/0 conn(0x7fddd80c61e0 0x7fddd80c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.341+0000 7fdddd400640 1 -- 172.21.15.67:0/4273501320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80c5e00 msgr2=0x7fddd80acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:36.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.341+0000 7fdddd400640 1 --2- 172.21.15.67:0/4273501320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80c5e00 0x7fddd80acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.341+0000 7fdddd400640 1 -- 172.21.15.67:0/4273501320 >> v2:172.21.15.134:3300/0 conn(0x7fddd80c61e0 msgr2=0x7fddd80c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:36.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.341+0000 7fdddd400640 1 --2- 172.21.15.67:0/4273501320 >> v2:172.21.15.134:3300/0 conn(0x7fddd80c61e0 0x7fddd80c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.341+0000 7fdddd400640 1 -- 172.21.15.67:0/4273501320 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd80ad7c0 con 0x7fddd8156660 2024-03-09T09:00:36.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.342+0000 7fdddd400640 1 --2- 172.21.15.67:0/4273501320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd8156660 0x7fddd806c5e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fddd808d1a0 tx=0x7fddd002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1fe0c4053ba535 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.343+0000 7fddd7400640 1 -- 172.21.15.67:0/4273501320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd002ca70 con 0x7fddd8156660 2024-03-09T09:00:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.343+0000 7fddd7400640 1 -- 172.21.15.67:0/4273501320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd002cc10 con 0x7fddd8156660 2024-03-09T09:00:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.343+0000 7fddd7400640 1 -- 172.21.15.67:0/4273501320 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd00343f0 con 0x7fddd8156660 2024-03-09T09:00:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.343+0000 7fdddde00640 1 -- 172.21.15.67:0/4273501320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd8156660 msgr2=0x7fddd806c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.343+0000 7fdddde00640 1 --2- 172.21.15.67:0/4273501320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd8156660 0x7fddd806c5e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fddd808d1a0 tx=0x7fddd002bed0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.344+0000 7fdddde00640 1 -- 172.21.15.67:0/4273501320 shutdown_connections 2024-03-09T09:00:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.344+0000 7fdddde00640 1 --2- 172.21.15.67:0/4273501320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd8156660 0x7fddd806c5e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.344+0000 7fdddde00640 1 --2- 172.21.15.67:0/4273501320 >> v2:172.21.15.134:3300/0 conn(0x7fddd80c61e0 0x7fddd80c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.344+0000 7fdddde00640 1 --2- 172.21.15.67:0/4273501320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80c5e00 0x7fddd80acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.344+0000 7fdddde00640 1 -- 172.21.15.67:0/4273501320 >> 172.21.15.67:0/4273501320 conn(0x7fddd80cb480 msgr2=0x7fddd80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.344+0000 7fdddde00640 1 -- 172.21.15.67:0/4273501320 shutdown_connections 2024-03-09T09:00:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.344+0000 7fdddde00640 1 -- 172.21.15.67:0/4273501320 wait complete. 2024-03-09T09:00:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.345+0000 7fdddde00640 1 Processor -- start 2024-03-09T09:00:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.346+0000 7fdddde00640 1 -- start start 2024-03-09T09:00:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.346+0000 7fdddde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd80c5e00 0x7fddd80ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.346+0000 7fdddde00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80c61e0 0x7fddd81eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.346+0000 7fdddde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd8156660 0x7fddd81ef130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddde00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd806bb00 con 0x7fddd80c5e00 2024-03-09T09:00:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddde00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd80ad4e0 con 0x7fddd8156660 2024-03-09T09:00:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddde00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd806bde0 con 0x7fddd80c61e0 2024-03-09T09:00:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd80c5e00 0x7fddd80ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd80c5e00 0x7fddd80ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38124/0 (socket says 172.21.15.67:38124) 2024-03-09T09:00:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddca00640 1 -- 172.21.15.67:0/3768097689 learned_addr learned my addr 172.21.15.67:0/3768097689 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddd400640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd8156660 0x7fddd81ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fddd7e00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80c61e0 0x7fddd81eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddca00640 1 -- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80c61e0 msgr2=0x7fddd81eebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddca00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80c61e0 0x7fddd81eebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddca00640 1 -- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd8156660 msgr2=0x7fddd81ef130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddca00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd8156660 0x7fddd81ef130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddca00640 1 -- 172.21.15.67:0/3768097689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd80c1f30 con 0x7fddd80c5e00 2024-03-09T09:00:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.347+0000 7fdddd400640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd8156660 0x7fddd81ef130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:00:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.348+0000 7fdddca00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd80c5e00 0x7fddd80ac050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00e870 tx=0x7fddcc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.348+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc017070 con 0x7fddd80c5e00 2024-03-09T09:00:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.348+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc00ee60 con 0x7fddd80c5e00 2024-03-09T09:00:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.348+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc00cb80 con 0x7fddd80c5e00 2024-03-09T09:00:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.348+0000 7fdddde00640 1 -- 172.21.15.67:0/3768097689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd80c1c50 con 0x7fddd80c5e00 2024-03-09T09:00:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.348+0000 7fdddde00640 1 -- 172.21.15.67:0/3768097689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd8159980 con 0x7fddd80c5e00 2024-03-09T09:00:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.350+0000 7fdddde00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bef20 0x7fddd80bf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.350+0000 7fdddde00640 1 -- 172.21.15.67:0/3768097689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd804f3c0 con 0x7fddd80bef20 2024-03-09T09:00:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.350+0000 7fddd7e00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bef20 0x7fddd80bf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.351+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fddcc01a070 con 0x7fddd80c5e00 2024-03-09T09:00:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.351+0000 7fddd7e00640 1 -- 172.21.15.67:0/3768097689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fddd80c1460 con 0x7fddd80bef20 2024-03-09T09:00:36.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.352+0000 7fddd4c00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fddb806b4d0 0x7fddb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.352+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fddcc08a480 con 0x7fddd80c5e00 2024-03-09T09:00:36.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.352+0000 7fddd7e00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bef20 0x7fddd80bf300 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fddd80c01b0 tx=0x7fddc4010c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.352+0000 7fdddd400640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fddb806b4d0 0x7fddb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.352+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc400f040 con 0x7fddd80bef20 2024-03-09T09:00:36.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.353+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fddc401c400 con 0x7fddd80bef20 2024-03-09T09:00:36.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.353+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bef20 msgr2=0x7fddd80bf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.353+0000 7fddd4c00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bef20 0x7fddd80bf300 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fddd80c01b0 tx=0x7fddc4010c20 comp rx=0 tx=0).stop 2024-03-09T09:00:36.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.353+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bef20 msgr2=0x7fddd80bf300 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:36.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.353+0000 7fddd4c00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bef20 0x7fddd80bf300 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fddd80c01b0 tx=0x7fddc4010c20 comp rx=0 tx=0).stop 2024-03-09T09:00:36.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.354+0000 7fdddd400640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fddb806b4d0 0x7fddb806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fddd0035000 tx=0x7fddd0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.362+0000 7fdddde00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bf840 0x7fddd814df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.362+0000 7fdddde00640 1 -- 172.21.15.67:0/3768097689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd806c020 con 0x7fddd80bf840 2024-03-09T09:00:36.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.362+0000 7fddd7e00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bf840 0x7fddd814df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.363+0000 7fddd7e00640 1 -- 172.21.15.67:0/3768097689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fddd806d2f0 con 0x7fddd80bf840 2024-03-09T09:00:36.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.364+0000 7fddd7e00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bf840 0x7fddd814df50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fddd80c01b0 tx=0x7fddc40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.364+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc4005b90 con 0x7fddd80bf840 2024-03-09T09:00:36.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.365+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4533 (secure 0 0 0) 0x7fddc4025070 con 0x7fddd80bf840 2024-03-09T09:00:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.365+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bf840 msgr2=0x7fddd814df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.365+0000 7fddd4c00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bf840 0x7fddd814df50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fddd80c01b0 tx=0x7fddc40046c0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.365+0000 7fddd4c00640 1 -- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bf840 msgr2=0x7fddd814df50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.365+0000 7fddd4c00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd80bf840 0x7fddd814df50 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fddd80c01b0 tx=0x7fddc40046c0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.368 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:00:36.368 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:00:36.368 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:00:36.368 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:00:36.369 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 66, 2024-03-09T09:00:36.369 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:00:36.369 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:00:36.369 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:00:36.369 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:00:36.369 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:36.370 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 4, 2024-03-09T09:00:36.370 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:36.370 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:00:36.370 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:00:36.370 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:36.371 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:36.371 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:36.371 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:36.371 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:36.371 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:36.371 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:36.371 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:36.371 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:36.371 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:36.372 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:36.372 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:36.372 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:00:36.372 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:00:36.372 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:36.372 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:36.372 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:36.372 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:36.372 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:36.372 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:36.373 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:36.373 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:36.373 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:36.373 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:36.373 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:36.373 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.373 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:00:36.374 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:00:36.374 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:00:36.374 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:00:36.374 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:00:36.374 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:00:36.374 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:36.374 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:36.374 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:00:36.374 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:36.375 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:36.376 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:36.376 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:36.376 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:36.376 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:36.376 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:36.376 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:36.376 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:36.377 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:00:36.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.377 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:00:36.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.377 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:00:36.377 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:00:36.377 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:36.377 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:36.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.377 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:36.378 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:00:36.378 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:36.378 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:00:36.378 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.378 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.379 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.379 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:00:36.379 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:00:36.379 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:36.379 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:36.379 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:36.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.380 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:00:36.380 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:00:36.380 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:36.380 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:36.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:36.380 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:00:36.380 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.381 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:36.381 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:00:36.381 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.381 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.381 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.381 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:00:36.381 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:00:36.381 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:36.382 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:36.382 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:36.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.382 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:00:36.382 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:00:36.382 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:36.382 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:36.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.383 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:36.383 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:00:36.383 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.383 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:36.383 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:00:36.383 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:36.383 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:00:36.384 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:00:36.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.385 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:36.385 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:36.385 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:36.385 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.385 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:36.386 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:00:36.386 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:36.386 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:36.386 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:36.386 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:36.386 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:36.386 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:36.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:36.386 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:00:36.386 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:00:36.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.365+0000 7fdddde00640 1 -- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fddb806b4d0 msgr2=0x7fddb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.365+0000 7fdddde00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fddb806b4d0 0x7fddb806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fddd0035000 tx=0x7fddd0030000 comp rx=0 tx=0).stop 2024-03-09T09:00:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.365+0000 7fdddde00640 1 -- 172.21.15.67:0/3768097689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd80c5e00 msgr2=0x7fddd80ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.365+0000 7fdddde00640 1 --2- 172.21.15.67:0/3768097689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddd80c5e00 0x7fddd80ac050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00e870 tx=0x7fddcc00ed30 comp rx=0 tx=0).stop 2024-03-09T09:00:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.366+0000 7fdddd400640 1 -- 172.21.15.67:0/3768097689 reap_dead start 2024-03-09T09:00:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.366+0000 7fdddd400640 1 -- 172.21.15.67:0/3768097689 reap_dead start 2024-03-09T09:00:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.366+0000 7fdddde00640 1 -- 172.21.15.67:0/3768097689 shutdown_connections 2024-03-09T09:00:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.366+0000 7fdddde00640 1 -- 172.21.15.67:0/3768097689 >> 172.21.15.67:0/3768097689 conn(0x7fddd80cb480 msgr2=0x7fddd80aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:36.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.366+0000 7fdddde00640 1 -- 172.21.15.67:0/3768097689 shutdown_connections 2024-03-09T09:00:36.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.366+0000 7fdddde00640 1 -- 172.21.15.67:0/3768097689 wait complete. 2024-03-09T09:00:36.390 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:00:36.390 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:00:36.390 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:00:36.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.610+0000 7fcb4f400640 1 Processor -- start 2024-03-09T09:00:36.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.610+0000 7fcb4f400640 1 -- start start 2024-03-09T09:00:36.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.611+0000 7fcb4f400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcb480c30f0 0x7fcb48152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.611+0000 7fcb4f400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb480ab3c0 con 0x7fcb480aa420 2024-03-09T09:00:36.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.611+0000 7fcb4f400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb480aa8a0 con 0x7fcb4806dbe0 2024-03-09T09:00:36.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.611+0000 7fcb4f400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb480c82d0 con 0x7fcb480c30f0 2024-03-09T09:00:36.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.612+0000 7fcb47e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcb480aa420 0x7fcb481490f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54954/0 (socket says 172.21.15.67:54954) 2024-03-09T09:00:36.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.612+0000 7fcb47e00640 1 -- 172.21.15.67:0/1460872163 learned_addr learned my addr 172.21.15.67:0/1460872163 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:00:36.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.612+0000 7fcb4ca00640 1 --2- 172.21.15.67:0/1460872163 >> v2:172.21.15.134:3300/0 conn(0x7fcb480c30f0 0x7fcb48152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.612+0000 7fcb47400640 1 -- 172.21.15.67:0/1460872163 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730873692 0 0) 0x7fcb34001110 con 0x7fcb480aa420 2024-03-09T09:00:36.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.612+0000 7fcb47400640 1 -- 172.21.15.67:0/1460872163 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb30003520 con 0x7fcb480aa420 2024-03-09T09:00:36.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.613+0000 7fcb4ca00640 1 -- 172.21.15.67:0/1460872163 >> v1:172.21.15.134:6789/0 conn(0x7fcb4806dbe0 legacy=0x7fcb48072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.613+0000 7fcb47400640 1 -- 172.21.15.67:0/1460872163 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3885659074 0 0) 0x7fcb30003520 con 0x7fcb480aa420 2024-03-09T09:00:36.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.613+0000 7fcb4ca00640 1 -- 172.21.15.67:0/1460872163 >> v1:172.21.15.67:6789/0 conn(0x7fcb480aa420 legacy=0x7fcb481490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.613+0000 7fcb4ca00640 1 -- 172.21.15.67:0/1460872163 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480aab20 con 0x7fcb480c30f0 2024-03-09T09:00:36.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.614+0000 7fcb4ca00640 1 --2- 172.21.15.67:0/1460872163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb480c30f0 0x7fcb48152a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0097a0 tx=0x7fcb3c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a2ef5eb196af516 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.615+0000 7fcb47400640 1 -- 172.21.15.67:0/1460872163 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c037070 con 0x7fcb480c30f0 2024-03-09T09:00:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.615+0000 7fcb47400640 1 -- 172.21.15.67:0/1460872163 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3c030510 con 0x7fcb480c30f0 2024-03-09T09:00:36.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.615+0000 7fcb47400640 1 -- 172.21.15.67:0/1460872163 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34001840 con 0x7fcb480c30f0 2024-03-09T09:00:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.615+0000 7fcb4f400640 1 -- 172.21.15.67:0/1460872163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb480c30f0 msgr2=0x7fcb48152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.615+0000 7fcb4f400640 1 --2- 172.21.15.67:0/1460872163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb480c30f0 0x7fcb48152a20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0097a0 tx=0x7fcb3c02b790 comp rx=0 tx=0).stop 2024-03-09T09:00:36.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.615+0000 7fcb4f400640 1 -- 172.21.15.67:0/1460872163 shutdown_connections 2024-03-09T09:00:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.615+0000 7fcb4f400640 1 --2- 172.21.15.67:0/1460872163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb480c30f0 0x7fcb48152a20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.616+0000 7fcb4f400640 1 -- 172.21.15.67:0/1460872163 >> 172.21.15.67:0/1460872163 conn(0x7fcb480c7370 msgr2=0x7fcb480c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.616+0000 7fcb4f400640 1 -- 172.21.15.67:0/1460872163 shutdown_connections 2024-03-09T09:00:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.616+0000 7fcb4f400640 1 -- 172.21.15.67:0/1460872163 wait complete. 2024-03-09T09:00:36.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.617+0000 7fcb4f400640 1 Processor -- start 2024-03-09T09:00:36.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.618+0000 7fcb4f400640 1 -- start start 2024-03-09T09:00:36.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.619+0000 7fcb4f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb4806dbe0 0x7fcb481ec990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.619+0000 7fcb4f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb480aa420 0x7fcb48147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.619+0000 7fcb4f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb480c30f0 0x7fcb48147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.619+0000 7fcb4f400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4806d6c0 con 0x7fcb480aa420 2024-03-09T09:00:36.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.619+0000 7fcb4f400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb480ab0e0 con 0x7fcb4806dbe0 2024-03-09T09:00:36.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.619+0000 7fcb4f400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb48149800 con 0x7fcb480c30f0 2024-03-09T09:00:36.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.619+0000 7fcb47e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb480aa420 0x7fcb48147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.619+0000 7fcb47e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb480aa420 0x7fcb48147520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38136/0 (socket says 172.21.15.67:38136) 2024-03-09T09:00:36.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.619+0000 7fcb47e00640 1 -- 172.21.15.67:0/2309168268 learned_addr learned my addr 172.21.15.67:0/2309168268 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:36.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.619+0000 7fcb4ca00640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb4806dbe0 0x7fcb481ec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.619+0000 7fcb4d400640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb480c30f0 0x7fcb48147a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb47e00640 1 -- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb480c30f0 msgr2=0x7fcb48147a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb47e00640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb480c30f0 0x7fcb48147a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb47e00640 1 -- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb4806dbe0 msgr2=0x7fcb481ec990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb47e00640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb4806dbe0 0x7fcb481ec990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb47e00640 1 -- 172.21.15.67:0/2309168268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480bc430 con 0x7fcb480aa420 2024-03-09T09:00:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb4d400640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb480c30f0 0x7fcb48147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:00:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb4ca00640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb4806dbe0 0x7fcb481ec990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:00:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb47e00640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb480aa420 0x7fcb48147520 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb340019b0 tx=0x7fcb3400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb44c00640 1 -- 172.21.15.67:0/2309168268 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34005ad0 con 0x7fcb480aa420 2024-03-09T09:00:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb44c00640 1 -- 172.21.15.67:0/2309168268 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb34005c70 con 0x7fcb480aa420 2024-03-09T09:00:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb44c00640 1 -- 172.21.15.67:0/2309168268 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34007290 con 0x7fcb480aa420 2024-03-09T09:00:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.620+0000 7fcb4f400640 1 -- 172.21.15.67:0/2309168268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb481ed2d0 con 0x7fcb480aa420 2024-03-09T09:00:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.621+0000 7fcb4f400640 1 -- 172.21.15.67:0/2309168268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb481ed5b0 con 0x7fcb480aa420 2024-03-09T09:00:36.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.622+0000 7fcb4f400640 1 -- 172.21.15.67:0/2309168268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb48075a00 con 0x7fcb480aa420 2024-03-09T09:00:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.623+0000 7fcb44c00640 1 -- 172.21.15.67:0/2309168268 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcb34007430 con 0x7fcb480aa420 2024-03-09T09:00:36.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.624+0000 7fcb44c00640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb2806b500 0x7fcb2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.624+0000 7fcb44c00640 1 -- 172.21.15.67:0/2309168268 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcb3408a3f0 con 0x7fcb480aa420 2024-03-09T09:00:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.624+0000 7fcb4ca00640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb2806b500 0x7fcb2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.625+0000 7fcb4ca00640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb2806b500 0x7fcb2806d9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c034000 tx=0x7fcb3c02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:36.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.629+0000 7fcb44c00640 1 -- 172.21.15.67:0/2309168268 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcb480ab3c0 con 0x7fcb480aa420 2024-03-09T09:00:36.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.954+0000 7fcb4f400640 1 -- 172.21.15.67:0/2309168268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fcb480bc910 con 0x7fcb480aa420 2024-03-09T09:00:36.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.955+0000 7fcb44c00640 1 -- 172.21.15.67:0/2309168268 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fcb34010300 con 0x7fcb480aa420 2024-03-09T09:00:36.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.958+0000 7fcb4f400640 1 -- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb2806b500 msgr2=0x7fcb2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.958+0000 7fcb4f400640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb2806b500 0x7fcb2806d9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c034000 tx=0x7fcb3c02f000 comp rx=0 tx=0).stop 2024-03-09T09:00:36.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.958+0000 7fcb4f400640 1 -- 172.21.15.67:0/2309168268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb480aa420 msgr2=0x7fcb48147520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:36.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.958+0000 7fcb4f400640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb480aa420 0x7fcb48147520 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcb340019b0 tx=0x7fcb3400cbe0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.958+0000 7fcb4f400640 1 -- 172.21.15.67:0/2309168268 shutdown_connections 2024-03-09T09:00:36.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.958+0000 7fcb4f400640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb2806b500 0x7fcb2806d9c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.958+0000 7fcb4f400640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb480c30f0 0x7fcb48147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.958+0000 7fcb4f400640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb480aa420 0x7fcb48147520 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.958+0000 7fcb4f400640 1 --2- 172.21.15.67:0/2309168268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb4806dbe0 0x7fcb481ec990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.958+0000 7fcb4f400640 1 -- 172.21.15.67:0/2309168268 >> 172.21.15.67:0/2309168268 conn(0x7fcb480c7370 msgr2=0x7fcb4814bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:36.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.959+0000 7fcb4f400640 1 -- 172.21.15.67:0/2309168268 shutdown_connections 2024-03-09T09:00:36.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:36.959+0000 7fcb4f400640 1 -- 172.21.15.67:0/2309168268 wait complete. 2024-03-09T09:00:36.984 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:00:42.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:00:42.219 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:00:46.985 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:00:46.985 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:00:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.245+0000 7eff0e800640 1 Processor -- start 2024-03-09T09:00:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.245+0000 7eff0e800640 1 -- start start 2024-03-09T09:00:47.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.246+0000 7eff0e800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7eff08153370 0x7eff080c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.246+0000 7eff0e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7eff0806dc00 0x7eff080729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.246+0000 7eff0e800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff080c4010 con 0x7eff08153750 2024-03-09T09:00:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.246+0000 7eff0e800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff08156dc0 con 0x7eff0806dc00 2024-03-09T09:00:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.246+0000 7eff0e800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff080c42f0 con 0x7eff08153370 2024-03-09T09:00:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.246+0000 7eff06a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7eff08153750 0x7eff080c2ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39920/0 (socket says 172.21.15.67:39920) 2024-03-09T09:00:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.246+0000 7eff06a00640 1 -- 172.21.15.67:0/3324048951 learned_addr learned my addr 172.21.15.67:0/3324048951 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:00:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.247+0000 7eff07e00640 1 --2- 172.21.15.67:0/3324048951 >> v2:172.21.15.67:3300/0 conn(0x7eff0806dc00 0x7eff080729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.247+0000 7eff07400640 1 --2- 172.21.15.67:0/3324048951 >> v2:172.21.15.134:3300/0 conn(0x7eff08153370 0x7eff080c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.247+0000 7eff06000640 1 -- 172.21.15.67:0/3324048951 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884601241 0 0) 0x7eff080c4010 con 0x7eff08153750 2024-03-09T09:00:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.247+0000 7eff06000640 1 -- 172.21.15.67:0/3324048951 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeec003520 con 0x7eff08153750 2024-03-09T09:00:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.247+0000 7eff07e00640 1 -- 172.21.15.67:0/3324048951 >> v2:172.21.15.134:3300/0 conn(0x7eff08153370 msgr2=0x7eff080c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.247+0000 7eff07e00640 1 --2- 172.21.15.67:0/3324048951 >> v2:172.21.15.134:3300/0 conn(0x7eff08153370 0x7eff080c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.247+0000 7eff07e00640 1 -- 172.21.15.67:0/3324048951 >> v1:172.21.15.67:6789/0 conn(0x7eff08153750 legacy=0x7eff080c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.247+0000 7eff07e00640 1 -- 172.21.15.67:0/3324048951 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff080c5a50 con 0x7eff0806dc00 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.248+0000 7eff07e00640 1 --2- 172.21.15.67:0/3324048951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff0806dc00 0x7eff080729c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7eff080c5fd0 tx=0x7eff0002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5848cbb0c547d552 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.249+0000 7eff06000640 1 -- 172.21.15.67:0/3324048951 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00039070 con 0x7eff0806dc00 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.249+0000 7eff06000640 1 -- 172.21.15.67:0/3324048951 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0002cbf0 con 0x7eff0806dc00 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.249+0000 7eff06000640 1 -- 172.21.15.67:0/3324048951 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00034440 con 0x7eff0806dc00 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.249+0000 7eff0e800640 1 -- 172.21.15.67:0/3324048951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff0806dc00 msgr2=0x7eff080729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.249+0000 7eff0e800640 1 --2- 172.21.15.67:0/3324048951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff0806dc00 0x7eff080729c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7eff080c5fd0 tx=0x7eff0002be40 comp rx=0 tx=0).stop 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.249+0000 7eff0e800640 1 -- 172.21.15.67:0/3324048951 shutdown_connections 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.249+0000 7eff0e800640 1 --2- 172.21.15.67:0/3324048951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff0806dc00 0x7eff080729c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.249+0000 7eff0e800640 1 --2- 172.21.15.67:0/3324048951 >> v2:172.21.15.134:3300/0 conn(0x7eff08153370 0x7eff080c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.249+0000 7eff0e800640 1 -- 172.21.15.67:0/3324048951 >> 172.21.15.67:0/3324048951 conn(0x7eff080ab320 msgr2=0x7eff080ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.249+0000 7eff0e800640 1 -- 172.21.15.67:0/3324048951 shutdown_connections 2024-03-09T09:00:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.249+0000 7eff0e800640 1 -- 172.21.15.67:0/3324048951 wait complete. 2024-03-09T09:00:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.251+0000 7eff0e800640 1 Processor -- start 2024-03-09T09:00:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.251+0000 7eff0e800640 1 -- start start 2024-03-09T09:00:47.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.252+0000 7eff0e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff0806dc00 0x7eff0814dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.252+0000 7eff0e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff08153370 0x7eff0806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.252+0000 7eff0e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff08153750 0x7eff0806ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.252+0000 7eff0e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff080c5d30 con 0x7eff08153750 2024-03-09T09:00:47.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.252+0000 7eff0e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff08073dc0 con 0x7eff08153370 2024-03-09T09:00:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.252+0000 7eff0e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff08072f00 con 0x7eff0806dc00 2024-03-09T09:00:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.252+0000 7eff07e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff08153750 0x7eff0806ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.252+0000 7eff07e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff08153750 0x7eff0806ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40806/0 (socket says 172.21.15.67:40806) 2024-03-09T09:00:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.252+0000 7eff07e00640 1 -- 172.21.15.67:0/3555332034 learned_addr learned my addr 172.21.15.67:0/3555332034 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7eff06a00640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff08153370 0x7eff0806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7eff07400640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff0806dc00 0x7eff0814dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7eff07e00640 1 -- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff0806dc00 msgr2=0x7eff0814dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7eff07e00640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff0806dc00 0x7eff0814dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7eff07e00640 1 -- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff08153370 msgr2=0x7eff0806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7eff07e00640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff08153370 0x7eff0806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7eff07e00640 1 -- 172.21.15.67:0/3555332034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff081eb7c0 con 0x7eff08153750 2024-03-09T09:00:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7eff06a00640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff08153370 0x7eff0806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:00:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7eff07e00640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff08153750 0x7eff0806ef80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7eff00009e80 tx=0x7eff00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7efeff400640 1 -- 172.21.15.67:0/3555332034 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00039050 con 0x7eff08153750 2024-03-09T09:00:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7efeff400640 1 -- 172.21.15.67:0/3555332034 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff00045070 con 0x7eff08153750 2024-03-09T09:00:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7eff0e800640 1 -- 172.21.15.67:0/3555332034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff081ebaa0 con 0x7eff08153750 2024-03-09T09:00:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.253+0000 7efeff400640 1 -- 172.21.15.67:0/3555332034 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00009450 con 0x7eff08153750 2024-03-09T09:00:47.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.254+0000 7eff0e800640 1 -- 172.21.15.67:0/3555332034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff081ec060 con 0x7eff08153750 2024-03-09T09:00:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.255+0000 7eff0e800640 1 -- 172.21.15.67:0/3555332034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff08075a00 con 0x7eff08153750 2024-03-09T09:00:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.256+0000 7efeff400640 1 -- 172.21.15.67:0/3555332034 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7eff0003e850 con 0x7eff08153750 2024-03-09T09:00:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.257+0000 7efeff400640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efee806b4d0 0x7efee806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.257+0000 7efeff400640 1 -- 172.21.15.67:0/3555332034 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7eff000ad670 con 0x7eff08153750 2024-03-09T09:00:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.257+0000 7eff07400640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efee806b4d0 0x7efee806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.258+0000 7eff07400640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efee806b4d0 0x7efee806d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efef8009a10 tx=0x7efef8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:47.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.264+0000 7efeff400640 1 -- 172.21.15.67:0/3555332034 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7eff00032e20 con 0x7eff08153750 2024-03-09T09:00:47.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.543+0000 7eff0e800640 1 -- 172.21.15.67:0/3555332034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7eff080c6690 con 0x7eff08153750 2024-03-09T09:00:47.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.544+0000 7efeff400640 1 -- 172.21.15.67:0/3555332034 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7eff0007c210 con 0x7eff08153750 2024-03-09T09:00:47.545 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:00:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.547+0000 7eff0e800640 1 -- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efee806b4d0 msgr2=0x7efee806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.547+0000 7eff0e800640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efee806b4d0 0x7efee806d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efef8009a10 tx=0x7efef8008040 comp rx=0 tx=0).stop 2024-03-09T09:00:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.547+0000 7eff0e800640 1 -- 172.21.15.67:0/3555332034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff08153750 msgr2=0x7eff0806ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.547+0000 7eff0e800640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff08153750 0x7eff0806ef80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7eff00009e80 tx=0x7eff00004a40 comp rx=0 tx=0).stop 2024-03-09T09:00:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.547+0000 7eff0e800640 1 -- 172.21.15.67:0/3555332034 shutdown_connections 2024-03-09T09:00:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.547+0000 7eff0e800640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efee806b4d0 0x7efee806d990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.547+0000 7eff0e800640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff08153750 0x7eff0806ef80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.547+0000 7eff0e800640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff08153370 0x7eff0806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.547+0000 7eff0e800640 1 --2- 172.21.15.67:0/3555332034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff0806dc00 0x7eff0814dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.547+0000 7eff0e800640 1 -- 172.21.15.67:0/3555332034 >> 172.21.15.67:0/3555332034 conn(0x7eff080ab320 msgr2=0x7eff080c7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:47.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.548+0000 7eff0e800640 1 -- 172.21.15.67:0/3555332034 shutdown_connections 2024-03-09T09:00:47.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.548+0000 7eff0e800640 1 -- 172.21.15.67:0/3555332034 wait complete. 2024-03-09T09:00:47.567 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:00:47.567 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:00:47.567 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:00:47.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:00:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.786+0000 7f6caea00640 1 Processor -- start 2024-03-09T09:00:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.786+0000 7f6caea00640 1 -- start start 2024-03-09T09:00:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.789+0000 7f6caea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca814fd90 0x7f6ca8156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.789+0000 7f6caea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca8156bc0 con 0x7f6ca80c9380 2024-03-09T09:00:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.789+0000 7f6caea00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca80c9890 con 0x7f6ca806c1d0 2024-03-09T09:00:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.789+0000 7f6caea00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca806cd10 con 0x7f6ca814fd90 2024-03-09T09:00:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.789+0000 7f6cacc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6ca80c9380 0x7f6ca80acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39922/0 (socket says 172.21.15.67:39922) 2024-03-09T09:00:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.789+0000 7f6cacc00640 1 -- 172.21.15.67:0/3317726284 learned_addr learned my addr 172.21.15.67:0/3317726284 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:00:47.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6cad600640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca814fd90 0x7f6ca8156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485622730 0 0) 0x7f6c94001110 con 0x7f6ca80c9380 2024-03-09T09:00:47.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c90003520 con 0x7f6ca80c9380 2024-03-09T09:00:47.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2810327651 0 0) 0x7f6c90003520 con 0x7f6ca80c9380 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca814fd90 msgr2=0x7f6ca8156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca814fd90 0x7f6ca8156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 >> v1:172.21.15.134:6789/0 conn(0x7f6ca806c1d0 legacy=0x7f6ca806c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca80c5f80 con 0x7f6ca80c9380 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f6c94001840 con 0x7f6ca80c9380 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 >> v1:172.21.15.67:6789/0 conn(0x7f6ca80c9380 legacy=0x7f6ca80acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c90003a50 0x7f6c90003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca814fd90 0x7f6c90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c90004d90 0x7f6c90005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9000bc10 con 0x7f6c90004d90 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9000bdc0 con 0x7f6c90003a50 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.790+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9000bfc0 con 0x7f6ca814fd90 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.791+0000 7f6cae000640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c90004d90 0x7f6c90005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.791+0000 7f6cad600640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c90003a50 0x7f6c90003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.791+0000 7f6cacc00640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca814fd90 0x7f6c90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.791+0000 7f6cae000640 1 -- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca814fd90 msgr2=0x7f6c90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.791+0000 7f6cae000640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca814fd90 0x7f6c90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.791+0000 7f6cae000640 1 -- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c90003a50 msgr2=0x7f6c90003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.791+0000 7f6cae000640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c90003a50 0x7f6c90003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.791+0000 7f6cae000640 1 -- 172.21.15.67:0/3317726284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90003520 con 0x7f6c90004d90 2024-03-09T09:00:47.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.791+0000 7f6cacc00640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca814fd90 0x7f6c90004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:00:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.792+0000 7f6cae000640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c90004d90 0x7f6c90005210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0001e50 tx=0x7f6c90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54668432db49acea server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.793+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0001a70 con 0x7f6c90004d90 2024-03-09T09:00:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.793+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca0003b70 con 0x7f6c90004d90 2024-03-09T09:00:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.793+0000 7f6ca7e00640 1 -- 172.21.15.67:0/3317726284 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00050d0 con 0x7f6c90004d90 2024-03-09T09:00:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.793+0000 7f6caea00640 1 -- 172.21.15.67:0/3317726284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c90004d90 msgr2=0x7f6c90005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.793+0000 7f6caea00640 1 --2- 172.21.15.67:0/3317726284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c90004d90 0x7f6c90005210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0001e50 tx=0x7f6c90005830 comp rx=0 tx=0).stop 2024-03-09T09:00:47.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.793+0000 7f6cae000640 1 -- 172.21.15.67:0/3317726284 reap_dead start 2024-03-09T09:00:47.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.794+0000 7f6caea00640 1 -- 172.21.15.67:0/3317726284 shutdown_connections 2024-03-09T09:00:47.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.794+0000 7f6caea00640 1 -- 172.21.15.67:0/3317726284 >> 172.21.15.67:0/3317726284 conn(0x7f6ca80cb480 msgr2=0x7f6ca80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:47.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.794+0000 7f6caea00640 1 -- 172.21.15.67:0/3317726284 shutdown_connections 2024-03-09T09:00:47.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.794+0000 7f6caea00640 1 -- 172.21.15.67:0/3317726284 wait complete. 2024-03-09T09:00:47.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.795+0000 7f6caea00640 1 Processor -- start 2024-03-09T09:00:47.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.795+0000 7f6caea00640 1 -- start start 2024-03-09T09:00:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.796+0000 7f6caea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c90004d90 0x7f6ca81551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.796+0000 7f6caea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca80b0c40 0x7f6ca81ef140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.796+0000 7f6caea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ef680 0x7f6ca80ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.796+0000 7f6caea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca80c6520 con 0x7f6ca81ef680 2024-03-09T09:00:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.796+0000 7f6caea00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca806ba60 con 0x7f6c90004d90 2024-03-09T09:00:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.796+0000 7f6caea00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca806bcb0 con 0x7f6ca80b0c40 2024-03-09T09:00:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.797+0000 7f6cae000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ef680 0x7f6ca80ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.797+0000 7f6cae000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ef680 0x7f6ca80ae0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40830/0 (socket says 172.21.15.67:40830) 2024-03-09T09:00:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.797+0000 7f6cae000640 1 -- 172.21.15.67:0/3242559318 learned_addr learned my addr 172.21.15.67:0/3242559318 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:47.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.797+0000 7f6cad600640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c90004d90 0x7f6ca81551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.797+0000 7f6cacc00640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca80b0c40 0x7f6ca81ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.797+0000 7f6cae000640 1 -- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca80b0c40 msgr2=0x7f6ca81ef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.797+0000 7f6cae000640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca80b0c40 0x7f6ca81ef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.797+0000 7f6cae000640 1 -- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c90004d90 msgr2=0x7f6ca81551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.797+0000 7f6cae000640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c90004d90 0x7f6ca81551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.797+0000 7f6cae000640 1 -- 172.21.15.67:0/3242559318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca80c1d10 con 0x7f6ca81ef680 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.797+0000 7f6cad600640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c90004d90 0x7f6ca81551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.798+0000 7f6cae000640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ef680 0x7f6ca80ae0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ca002c060 tx=0x7f6ca00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.798+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0007db0 con 0x7f6ca81ef680 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.798+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca0003970 con 0x7f6ca81ef680 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.798+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0006d70 con 0x7f6ca81ef680 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.798+0000 7f6caea00640 1 -- 172.21.15.67:0/3242559318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca81ebc10 con 0x7f6ca81ef680 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.798+0000 7f6caea00640 1 -- 172.21.15.67:0/3242559318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca81ec1d0 con 0x7f6ca81ef680 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.799+0000 7f6caea00640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ec3a0 0x7f6ca80ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.799+0000 7f6caea00640 1 -- 172.21.15.67:0/3242559318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca804f3c0 con 0x7f6ca81ec3a0 2024-03-09T09:00:47.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.799+0000 7f6cad600640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ec3a0 0x7f6ca80ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.800+0000 7f6cad600640 1 -- 172.21.15.67:0/3242559318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ca814def0 con 0x7f6ca81ec3a0 2024-03-09T09:00:47.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.800+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6ca0007030 con 0x7f6ca81ef680 2024-03-09T09:00:47.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.800+0000 7f6cad600640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ec3a0 0x7f6ca80ae5e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c013610 tx=0x7f6c9c013ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:47.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.801+0000 7f6ca5600640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c8806b4d0 0x7f6c8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.801+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6ca00ab7b0 con 0x7f6ca81ef680 2024-03-09T09:00:47.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.802+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c01c070 con 0x7f6ca81ec3a0 2024-03-09T09:00:47.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.802+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f6c9c0119e0 con 0x7f6ca81ec3a0 2024-03-09T09:00:47.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.802+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ec3a0 msgr2=0x7f6ca80ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.802+0000 7f6ca5600640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ec3a0 0x7f6ca80ae5e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c013610 tx=0x7f6c9c013ad0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.802+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ec3a0 msgr2=0x7f6ca80ae5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:47.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.802+0000 7f6ca5600640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ec3a0 0x7f6ca80ae5e0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c013610 tx=0x7f6c9c013ad0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.802+0000 7f6cacc00640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c8806b4d0 0x7f6c8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.803+0000 7f6cacc00640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c8806b4d0 0x7f6c8806d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80ad920 tx=0x7f6c94009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.809+0000 7f6caea00640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81efb00 0x7f6ca80aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.809+0000 7f6caea00640 1 -- 172.21.15.67:0/3242559318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca806d2d0 con 0x7f6ca81efb00 2024-03-09T09:00:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.809+0000 7f6cad600640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81efb00 0x7f6ca80aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:47.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.810+0000 7f6cad600640 1 -- 172.21.15.67:0/3242559318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6ca814d850 con 0x7f6ca81efb00 2024-03-09T09:00:47.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.810+0000 7f6cad600640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81efb00 0x7f6ca80aeb20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c013610 tx=0x7f6c9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:47.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c006270 con 0x7f6ca81efb00 2024-03-09T09:00:47.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5019 (secure 0 0 0) 0x7f6c9c006270 con 0x7f6ca81efb00 2024-03-09T09:00:47.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81efb00 msgr2=0x7f6ca80aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6ca5600640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81efb00 0x7f6ca80aeb20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c013610 tx=0x7f6c9c004730 comp rx=0 tx=0).stop 2024-03-09T09:00:47.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6ca5600640 1 -- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81efb00 msgr2=0x7f6ca80aeb20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:47.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6ca5600640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81efb00 0x7f6ca80aeb20 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c013610 tx=0x7f6c9c004730 comp rx=0 tx=0).stop 2024-03-09T09:00:47.813 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:00:47.813 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:00:47.813 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:00:47.813 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:00:47.813 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 66, 2024-03-09T09:00:47.813 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:00:47.813 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:00:47.813 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:00:47.813 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:00:47.813 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:47.813 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:47.814 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:47.815 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:00:47.816 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:47.817 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:00:47.818 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:47.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:47.820 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:00:47.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:47.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:47.823 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:47.824 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:47.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:47.826 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:00:47.826 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:00:47.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6caea00640 1 -- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c8806b4d0 msgr2=0x7f6c8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6caea00640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c8806b4d0 0x7f6c8806d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80ad920 tx=0x7f6c94009040 comp rx=0 tx=0).stop 2024-03-09T09:00:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6caea00640 1 -- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ef680 msgr2=0x7f6ca80ae0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6caea00640 1 --2- 172.21.15.67:0/3242559318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca81ef680 0x7f6ca80ae0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ca002c060 tx=0x7f6ca00014a0 comp rx=0 tx=0).stop 2024-03-09T09:00:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6cae000640 1 -- 172.21.15.67:0/3242559318 reap_dead start 2024-03-09T09:00:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6cae000640 1 -- 172.21.15.67:0/3242559318 reap_dead start 2024-03-09T09:00:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6caea00640 1 -- 172.21.15.67:0/3242559318 shutdown_connections 2024-03-09T09:00:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.811+0000 7f6caea00640 1 -- 172.21.15.67:0/3242559318 >> 172.21.15.67:0/3242559318 conn(0x7f6ca80cb480 msgr2=0x7f6ca80ca610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.812+0000 7f6caea00640 1 -- 172.21.15.67:0/3242559318 shutdown_connections 2024-03-09T09:00:47.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:47.812+0000 7f6caea00640 1 -- 172.21.15.67:0/3242559318 wait complete. 2024-03-09T09:00:47.829 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:00:47.829 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:00:47.829 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:00:48.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.038+0000 7fdd10e00640 1 Processor -- start 2024-03-09T09:00:48.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.038+0000 7fdd10e00640 1 -- start start 2024-03-09T09:00:48.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.039+0000 7fdd10e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdd0c0730c0 0x7fdd0c0aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.039+0000 7fdd10e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd0c0734a0 0x7fdd0c0c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.039+0000 7fdd10e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c152e00 con 0x7fdd0c0c33c0 2024-03-09T09:00:48.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.039+0000 7fdd10e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0c0aa960 con 0x7fdd0c0734a0 2024-03-09T09:00:48.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.039+0000 7fdd10e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0c1530e0 con 0x7fdd0c0730c0 2024-03-09T09:00:48.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.039+0000 7fdd03e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd0c0734a0 0x7fdd0c0c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.039+0000 7fdd03e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd0c0734a0 0x7fdd0c0c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40850/0 (socket says 172.21.15.67:40850) 2024-03-09T09:00:48.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.039+0000 7fdd03e00640 1 -- 172.21.15.67:0/1651967947 learned_addr learned my addr 172.21.15.67:0/1651967947 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:48.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.040+0000 7fdd09e00640 1 --2- 172.21.15.67:0/1651967947 >> v2:172.21.15.134:3300/0 conn(0x7fdd0c0730c0 0x7fdd0c0aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.040+0000 7fdd03e00640 1 -- 172.21.15.67:0/1651967947 >> v2:172.21.15.134:3300/0 conn(0x7fdd0c0730c0 msgr2=0x7fdd0c0aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:48.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.040+0000 7fdd03e00640 1 --2- 172.21.15.67:0/1651967947 >> v2:172.21.15.134:3300/0 conn(0x7fdd0c0730c0 0x7fdd0c0aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.040+0000 7fdd03e00640 1 -- 172.21.15.67:0/1651967947 >> v1:172.21.15.134:6789/0 conn(0x7fdd0c0c33c0 legacy=0x7fdd0c06d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:48.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.040+0000 7fdd03e00640 1 -- 172.21.15.67:0/1651967947 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c1533c0 con 0x7fdd0c0734a0 2024-03-09T09:00:48.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.041+0000 7fdd03e00640 1 --2- 172.21.15.67:0/1651967947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd0c0734a0 0x7fdd0c0c2e80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c06c090 tx=0x7fdcf402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8b4320ed9bfc8e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.042+0000 7fdd09400640 1 -- 172.21.15.67:0/1651967947 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf402e070 con 0x7fdd0c0734a0 2024-03-09T09:00:48.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.042+0000 7fdd09400640 1 -- 172.21.15.67:0/1651967947 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf4037070 con 0x7fdd0c0734a0 2024-03-09T09:00:48.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.042+0000 7fdd09400640 1 -- 172.21.15.67:0/1651967947 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4031a00 con 0x7fdd0c0734a0 2024-03-09T09:00:48.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.042+0000 7fdd10e00640 1 -- 172.21.15.67:0/1651967947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd0c0734a0 msgr2=0x7fdd0c0c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.042+0000 7fdd10e00640 1 --2- 172.21.15.67:0/1651967947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd0c0734a0 0x7fdd0c0c2e80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c06c090 tx=0x7fdcf402b880 comp rx=0 tx=0).stop 2024-03-09T09:00:48.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.042+0000 7fdd10e00640 1 -- 172.21.15.67:0/1651967947 shutdown_connections 2024-03-09T09:00:48.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.042+0000 7fdd10e00640 1 --2- 172.21.15.67:0/1651967947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd0c0734a0 0x7fdd0c0c2e80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.042+0000 7fdd10e00640 1 --2- 172.21.15.67:0/1651967947 >> v2:172.21.15.134:3300/0 conn(0x7fdd0c0730c0 0x7fdd0c0aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.042+0000 7fdd10e00640 1 -- 172.21.15.67:0/1651967947 >> 172.21.15.67:0/1651967947 conn(0x7fdd0c0c7370 msgr2=0x7fdd0c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:48.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.042+0000 7fdd10e00640 1 -- 172.21.15.67:0/1651967947 shutdown_connections 2024-03-09T09:00:48.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.042+0000 7fdd10e00640 1 -- 172.21.15.67:0/1651967947 wait complete. 2024-03-09T09:00:48.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.044+0000 7fdd10e00640 1 Processor -- start 2024-03-09T09:00:48.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.044+0000 7fdd10e00640 1 -- start start 2024-03-09T09:00:48.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.045+0000 7fdd10e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd0c0730c0 0x7fdd0c0c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.045+0000 7fdd10e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd0c0734a0 0x7fdd0c152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.045+0000 7fdd10e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c0c33c0 0x7fdd0c147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.045+0000 7fdd10e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0aaef0 con 0x7fdd0c0730c0 2024-03-09T09:00:48.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.045+0000 7fdd10e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c153980 con 0x7fdd0c0c33c0 2024-03-09T09:00:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.045+0000 7fdd10e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0ab070 con 0x7fdd0c0734a0 2024-03-09T09:00:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.045+0000 7fdd09e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd0c0730c0 0x7fdd0c0c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.045+0000 7fdd09e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd0c0730c0 0x7fdd0c0c5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40858/0 (socket says 172.21.15.67:40858) 2024-03-09T09:00:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.045+0000 7fdd09e00640 1 -- 172.21.15.67:0/2872456809 learned_addr learned my addr 172.21.15.67:0/2872456809 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.046+0000 7fdd09e00640 1 -- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd0c0734a0 msgr2=0x7fdd0c152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.046+0000 7fdd03e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd0c0734a0 0x7fdd0c152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.046+0000 7fdd0a800640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c0c33c0 0x7fdd0c147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.046+0000 7fdd09e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd0c0734a0 0x7fdd0c152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.046+0000 7fdd09e00640 1 -- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c0c33c0 msgr2=0x7fdd0c147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.046+0000 7fdd09e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c0c33c0 0x7fdd0c147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.046+0000 7fdd09e00640 1 -- 172.21.15.67:0/2872456809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c0a8e60 con 0x7fdd0c0730c0 2024-03-09T09:00:48.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.046+0000 7fdd03e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd0c0734a0 0x7fdd0c152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:00:48.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.046+0000 7fdd0a800640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c0c33c0 0x7fdd0c147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:00:48.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.046+0000 7fdd09e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd0c0730c0 0x7fdd0c0c5c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00d830 tx=0x7fdcfc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.047+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc00b6c0 con 0x7fdd0c0730c0 2024-03-09T09:00:48.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.047+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc019070 con 0x7fdd0c0730c0 2024-03-09T09:00:48.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.047+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc00ba80 con 0x7fdd0c0730c0 2024-03-09T09:00:48.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.047+0000 7fdd10e00640 1 -- 172.21.15.67:0/2872456809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0c0c6590 con 0x7fdd0c0730c0 2024-03-09T09:00:48.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.047+0000 7fdd10e00640 1 -- 172.21.15.67:0/2872456809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0c0c6b50 con 0x7fdd0c0730c0 2024-03-09T09:00:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.048+0000 7fdd10e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147960 0x7fdd0c1ed350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.048+0000 7fdd10e00640 1 -- 172.21.15.67:0/2872456809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0c6d20 con 0x7fdd0c147960 2024-03-09T09:00:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.049+0000 7fdd03e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147960 0x7fdd0c1ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.049+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdcfc01e460 con 0x7fdd0c0730c0 2024-03-09T09:00:48.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.050+0000 7fdd03e00640 1 -- 172.21.15.67:0/2872456809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd0c068d20 con 0x7fdd0c147960 2024-03-09T09:00:48.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.050+0000 7fdd02000640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdce806b4d0 0x7fdce806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.050+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdcfc08acb0 con 0x7fdd0c0730c0 2024-03-09T09:00:48.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.050+0000 7fdd03e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147960 0x7fdd0c1ed350 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0ac8a0 tx=0x7fdcf40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.051+0000 7fdd0a800640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdce806b4d0 0x7fdce806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.052+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4032670 con 0x7fdd0c147960 2024-03-09T09:00:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.053+0000 7fdd0a800640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdce806b4d0 0x7fdce806d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdd04007e30 tx=0x7fdd04007340 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.054+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fdcf4037040 con 0x7fdd0c147960 2024-03-09T09:00:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.054+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147960 msgr2=0x7fdd0c1ed350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.054+0000 7fdd02000640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147960 0x7fdd0c1ed350 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0ac8a0 tx=0x7fdcf40046c0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.054+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147960 msgr2=0x7fdd0c1ed350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.054+0000 7fdd02000640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147960 0x7fdd0c1ed350 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0ac8a0 tx=0x7fdcf40046c0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.068+0000 7fdd10e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147d40 0x7fdd0c1ed890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.068+0000 7fdd10e00640 1 -- 172.21.15.67:0/2872456809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0c0c6ec0 con 0x7fdd0c147d40 2024-03-09T09:00:48.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.069+0000 7fdd03e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147d40 0x7fdd0c1ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.070+0000 7fdd03e00640 1 -- 172.21.15.67:0/2872456809 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdd0c075a00 con 0x7fdd0c147d40 2024-03-09T09:00:48.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.071+0000 7fdd03e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147d40 0x7fdd0c1ed890 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4000c00 tx=0x7fdcf4031000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.072+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf404c870 con 0x7fdd0c147d40 2024-03-09T09:00:48.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.072+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7fdcf4031ec0 con 0x7fdd0c147d40 2024-03-09T09:00:48.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.072+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147d40 msgr2=0x7fdd0c1ed890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.072+0000 7fdd02000640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147d40 0x7fdd0c1ed890 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4000c00 tx=0x7fdcf4031000 comp rx=0 tx=0).stop 2024-03-09T09:00:48.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.072+0000 7fdd02000640 1 -- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147d40 msgr2=0x7fdd0c1ed890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:48.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.072+0000 7fdd02000640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd0c147d40 0x7fdd0c1ed890 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4000c00 tx=0x7fdcf4031000 comp rx=0 tx=0).stop 2024-03-09T09:00:48.074 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:00:48.074 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:00:48.074 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:00:48.074 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:00:48.074 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 66, 2024-03-09T09:00:48.074 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:00:48.075 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:00:48.075 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:00:48.078 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:00:48.078 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:48.078 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:00:48.078 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:48.078 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:00:48.078 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:00:48.078 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:00:48.079 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:48.080 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:48.080 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:48.080 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:48.080 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:48.080 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:48.080 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:48.080 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:48.080 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:48.080 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:48.080 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:00:48.081 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:48.082 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:00:48.083 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:00:48.084 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:48.084 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:48.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:48.084 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:00:48.084 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:48.084 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:00:48.084 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:00:48.085 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.086 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.087 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:00:48.087 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:00:48.087 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:48.087 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:48.087 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:48.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.087 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:00:48.087 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:00:48.088 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:48.088 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:48.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:48.088 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:00:48.088 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:00:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:00:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:00:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:48.089 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.090 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.090 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:00:48.090 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:00:48.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.090 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:48.090 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:48.090 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:48.090 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.090 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:48.091 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:00:48.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.091 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:48.091 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:48.091 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:48.091 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.091 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:48.091 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:00:48.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.092 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:48.092 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:48.092 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:00:48.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.092 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:48.092 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:00:48.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.092 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:00:48.092 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:48.093 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:48.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.093 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:48.093 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:48.093 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:00:48.093 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.093 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:48.093 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:00:48.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.093 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:48.094 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:48.094 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:00:48.094 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.094 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.094 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:00:48.094 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:00:48.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.073+0000 7fdd10e00640 1 -- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdce806b4d0 msgr2=0x7fdce806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.073+0000 7fdd10e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdce806b4d0 0x7fdce806d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdd04007e30 tx=0x7fdd04007340 comp rx=0 tx=0).stop 2024-03-09T09:00:48.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.073+0000 7fdd10e00640 1 -- 172.21.15.67:0/2872456809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd0c0730c0 msgr2=0x7fdd0c0c5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.073+0000 7fdd10e00640 1 --2- 172.21.15.67:0/2872456809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd0c0730c0 0x7fdd0c0c5c50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00d830 tx=0x7fdcfc00dcf0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.073+0000 7fdd0a800640 1 -- 172.21.15.67:0/2872456809 reap_dead start 2024-03-09T09:00:48.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.073+0000 7fdd0a800640 1 -- 172.21.15.67:0/2872456809 reap_dead start 2024-03-09T09:00:48.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.073+0000 7fdd10e00640 1 -- 172.21.15.67:0/2872456809 shutdown_connections 2024-03-09T09:00:48.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.073+0000 7fdd10e00640 1 -- 172.21.15.67:0/2872456809 >> 172.21.15.67:0/2872456809 conn(0x7fdd0c0c7370 msgr2=0x7fdd0c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:48.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.073+0000 7fdd10e00640 1 -- 172.21.15.67:0/2872456809 shutdown_connections 2024-03-09T09:00:48.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.073+0000 7fdd10e00640 1 -- 172.21.15.67:0/2872456809 wait complete. 2024-03-09T09:00:48.096 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:00:48.097 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:00:48.097 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:00:48.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.361+0000 7fb2c1c00640 1 Processor -- start 2024-03-09T09:00:48.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.361+0000 7fb2c1c00640 1 -- start start 2024-03-09T09:00:48.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.363+0000 7fb2c1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb2bc153460 0x7fb2bc155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.364+0000 7fb2c1c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb2bc0c8c60 0x7fb2bc0c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.364+0000 7fb2c1c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc06b040 con 0x7fb2bc0c9d50 2024-03-09T09:00:48.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.364+0000 7fb2c1c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc0ca590 con 0x7fb2bc153460 2024-03-09T09:00:48.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.364+0000 7fb2c1c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc156710 con 0x7fb2bc0c8c60 2024-03-09T09:00:48.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.364+0000 7fb2baa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb2bc153460 0x7fb2bc155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.364+0000 7fb2baa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb2bc153460 0x7fb2bc155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40866/0 (socket says 172.21.15.67:40866) 2024-03-09T09:00:48.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.364+0000 7fb2baa00640 1 -- 172.21.15.67:0/1811528775 learned_addr learned my addr 172.21.15.67:0/1811528775 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:48.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.364+0000 7fb2ba000640 1 --2- 172.21.15.67:0/1811528775 >> v2:172.21.15.134:3300/0 conn(0x7fb2bc0c8c60 0x7fb2bc0c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.365+0000 7fb2baa00640 1 -- 172.21.15.67:0/1811528775 >> v2:172.21.15.134:3300/0 conn(0x7fb2bc0c8c60 msgr2=0x7fb2bc0c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:48.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.365+0000 7fb2baa00640 1 --2- 172.21.15.67:0/1811528775 >> v2:172.21.15.134:3300/0 conn(0x7fb2bc0c8c60 0x7fb2bc0c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.365+0000 7fb2baa00640 1 -- 172.21.15.67:0/1811528775 >> v1:172.21.15.134:6789/0 conn(0x7fb2bc0c9d50 legacy=0x7fb2bc0caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:48.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.365+0000 7fb2baa00640 1 -- 172.21.15.67:0/1811528775 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc1569f0 con 0x7fb2bc153460 2024-03-09T09:00:48.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.366+0000 7fb2baa00640 1 --2- 172.21.15.67:0/1811528775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2bc153460 0x7fb2bc155ec0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc06dac0 tx=0x7fb2ac02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973f5bbb2443dd63 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.366+0000 7fb2b9600640 1 -- 172.21.15.67:0/1811528775 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac02e070 con 0x7fb2bc153460 2024-03-09T09:00:48.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.367+0000 7fb2b9600640 1 -- 172.21.15.67:0/1811528775 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac037070 con 0x7fb2bc153460 2024-03-09T09:00:48.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.367+0000 7fb2b9600640 1 -- 172.21.15.67:0/1811528775 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac031a00 con 0x7fb2bc153460 2024-03-09T09:00:48.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.367+0000 7fb2c1c00640 1 -- 172.21.15.67:0/1811528775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2bc153460 msgr2=0x7fb2bc155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.367+0000 7fb2c1c00640 1 --2- 172.21.15.67:0/1811528775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2bc153460 0x7fb2bc155ec0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc06dac0 tx=0x7fb2ac02b880 comp rx=0 tx=0).stop 2024-03-09T09:00:48.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.367+0000 7fb2c1c00640 1 -- 172.21.15.67:0/1811528775 shutdown_connections 2024-03-09T09:00:48.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.367+0000 7fb2c1c00640 1 --2- 172.21.15.67:0/1811528775 >> v2:172.21.15.134:3300/0 conn(0x7fb2bc0c8c60 0x7fb2bc0c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.367+0000 7fb2c1c00640 1 --2- 172.21.15.67:0/1811528775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2bc153460 0x7fb2bc155ec0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.367+0000 7fb2c1c00640 1 -- 172.21.15.67:0/1811528775 >> 172.21.15.67:0/1811528775 conn(0x7fb2bc0c8170 msgr2=0x7fb2bc0c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:48.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.367+0000 7fb2c1c00640 1 -- 172.21.15.67:0/1811528775 shutdown_connections 2024-03-09T09:00:48.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.367+0000 7fb2c1c00640 1 -- 172.21.15.67:0/1811528775 wait complete. 2024-03-09T09:00:48.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.369+0000 7fb2c1c00640 1 Processor -- start 2024-03-09T09:00:48.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.369+0000 7fb2c1c00640 1 -- start start 2024-03-09T09:00:48.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.370+0000 7fb2c1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2bc0c8c60 0x7fb2bc1489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.370+0000 7fb2c1c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2bc0c9d50 0x7fb2bc148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.370+0000 7fb2c1c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc153460 0x7fb2bc1ef290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.370+0000 7fb2c1c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc153840 con 0x7fb2bc0c9d50 2024-03-09T09:00:48.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.370+0000 7fb2c1c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc156400 con 0x7fb2bc0c8c60 2024-03-09T09:00:48.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.370+0000 7fb2c1c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0cb130 con 0x7fb2bc153460 2024-03-09T09:00:48.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.370+0000 7fb2ba000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2bc0c9d50 0x7fb2bc148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.371+0000 7fb2ba000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2bc0c9d50 0x7fb2bc148f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40878/0 (socket says 172.21.15.67:40878) 2024-03-09T09:00:48.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.371+0000 7fb2ba000640 1 -- 172.21.15.67:0/2551722129 learned_addr learned my addr 172.21.15.67:0/2551722129 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:48.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.371+0000 7fb2baa00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2bc0c8c60 0x7fb2bc1489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.371+0000 7fb2bb400640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc153460 0x7fb2bc1ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.371+0000 7fb2ba000640 1 -- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc153460 msgr2=0x7fb2bc1ef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.371+0000 7fb2ba000640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc153460 0x7fb2bc1ef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.371+0000 7fb2ba000640 1 -- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2bc0c8c60 msgr2=0x7fb2bc1489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.371+0000 7fb2ba000640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2bc0c8c60 0x7fb2bc1489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.371+0000 7fb2ba000640 1 -- 172.21.15.67:0/2551722129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc1585a0 con 0x7fb2bc0c9d50 2024-03-09T09:00:48.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.371+0000 7fb2baa00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2bc0c8c60 0x7fb2bc1489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:00:48.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.371+0000 7fb2ba000640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2bc0c9d50 0x7fb2bc148f10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400e9e0 tx=0x7fb2a400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.372+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4017070 con 0x7fb2bc0c9d50 2024-03-09T09:00:48.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.372+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a400c9d0 con 0x7fb2bc0c9d50 2024-03-09T09:00:48.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.372+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a400cdc0 con 0x7fb2bc0c9d50 2024-03-09T09:00:48.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.372+0000 7fb2c1c00640 1 -- 172.21.15.67:0/2551722129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc158880 con 0x7fb2bc0c9d50 2024-03-09T09:00:48.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.372+0000 7fb2c1c00640 1 -- 172.21.15.67:0/2551722129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc158e40 con 0x7fb2bc0c9d50 2024-03-09T09:00:48.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.373+0000 7fb2c1c00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc159040 0x7fb2bc1ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.373+0000 7fb2c1c00640 1 -- 172.21.15.67:0/2551722129 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0752b0 con 0x7fb2bc159040 2024-03-09T09:00:48.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.373+0000 7fb2baa00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc159040 0x7fb2bc1ef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.374+0000 7fb2baa00640 1 -- 172.21.15.67:0/2551722129 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2bc1efd10 con 0x7fb2bc159040 2024-03-09T09:00:48.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.374+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb2a401a070 con 0x7fb2bc0c9d50 2024-03-09T09:00:48.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.374+0000 7fb2baa00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc159040 0x7fb2bc1ef7d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0ab140 tx=0x7fb2ac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.375+0000 7fb2b2a00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb29806b500 0x7fb29806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.375+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb2a408a960 con 0x7fb2bc0c9d50 2024-03-09T09:00:48.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.376+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac04a070 con 0x7fb2bc159040 2024-03-09T09:00:48.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.376+0000 7fb2bb400640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb29806b500 0x7fb29806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.376+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb2ac037040 con 0x7fb2bc159040 2024-03-09T09:00:48.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.376+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc159040 msgr2=0x7fb2bc1ef7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.376+0000 7fb2b2a00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc159040 0x7fb2bc1ef7d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0ab140 tx=0x7fb2ac004730 comp rx=0 tx=0).stop 2024-03-09T09:00:48.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.376+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc159040 msgr2=0x7fb2bc1ef7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:48.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.376+0000 7fb2b2a00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc159040 0x7fb2bc1ef7d0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0ab140 tx=0x7fb2ac004730 comp rx=0 tx=0).stop 2024-03-09T09:00:48.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.376+0000 7fb2bb400640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb29806b500 0x7fb29806d9c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4005520 tx=0x7fb2b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.383+0000 7fb2c1c00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc0beb70 0x7fb2bc159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.383+0000 7fb2c1c00640 1 -- 172.21.15.67:0/2551722129 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0c7520 con 0x7fb2bc0beb70 2024-03-09T09:00:48.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.383+0000 7fb2baa00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc0beb70 0x7fb2bc159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.384+0000 7fb2baa00640 1 -- 172.21.15.67:0/2551722129 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb2bc075a00 con 0x7fb2bc0beb70 2024-03-09T09:00:48.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.384+0000 7fb2baa00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc0beb70 0x7fb2bc159640 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac000c00 tx=0x7fb2ac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.384+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac04a040 con 0x7fb2bc0beb70 2024-03-09T09:00:48.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fb2ac009490 con 0x7fb2bc0beb70 2024-03-09T09:00:48.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc0beb70 msgr2=0x7fb2bc159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2b2a00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc0beb70 0x7fb2bc159640 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac000c00 tx=0x7fb2ac0046c0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2b2a00640 1 -- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc0beb70 msgr2=0x7fb2bc159640 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:00:48.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2b2a00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2bc0beb70 0x7fb2bc159640 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.386 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:00:48.386 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 66, 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:48.387 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:48.388 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:00:48.389 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:00:48.390 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:00:48.391 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.393 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.394 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:00:48.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:00:48.396 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.397 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:00:48.398 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:00:48.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:00:48.398 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:00:48.398 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:00:48.398 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:00:48.398 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:00:48.398 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:00:48.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:00:48.398 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:00:48.398 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:00:48.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2c1c00640 1 -- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb29806b500 msgr2=0x7fb29806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2c1c00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb29806b500 0x7fb29806d9c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4005520 tx=0x7fb2b4009040 comp rx=0 tx=0).stop 2024-03-09T09:00:48.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2c1c00640 1 -- 172.21.15.67:0/2551722129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2bc0c9d50 msgr2=0x7fb2bc148f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2c1c00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2bc0c9d50 0x7fb2bc148f10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400e9e0 tx=0x7fb2a400eea0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2bb400640 1 -- 172.21.15.67:0/2551722129 reap_dead start 2024-03-09T09:00:48.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2c1c00640 1 -- 172.21.15.67:0/2551722129 shutdown_connections 2024-03-09T09:00:48.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2c1c00640 1 --2- 172.21.15.67:0/2551722129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2bc0c9d50 0x7fb2bc148f10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2c1c00640 1 -- 172.21.15.67:0/2551722129 >> 172.21.15.67:0/2551722129 conn(0x7fb2bc0c8170 msgr2=0x7fb2bc0ab190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:48.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2c1c00640 1 -- 172.21.15.67:0/2551722129 shutdown_connections 2024-03-09T09:00:48.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.385+0000 7fb2c1c00640 1 -- 172.21.15.67:0/2551722129 wait complete. 2024-03-09T09:00:48.405 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:00:48.405 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-03-09T09:00:48.405 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:00:48.405 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-03-09T09:00:48.405 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:00:48.405 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-09T09:00:48.406 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:00:48.440 INFO:tasks.ceph.mon.b:Stopped 2024-03-09T09:00:48.440 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:00:48.440 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:00:48.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.641+0000 7f8f5ce00640 1 Processor -- start 2024-03-09T09:00:48.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.641+0000 7f8f5ce00640 1 -- start start 2024-03-09T09:00:48.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.643+0000 7f8f5ce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f58153300 0x7f8f580b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.643+0000 7f8f5ce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f580cadb0 0x7f8f580b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.643+0000 7f8f5ce00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f58157000 con 0x7f8f580c5e40 2024-03-09T09:00:48.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.643+0000 7f8f5ce00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f580ae430 con 0x7f8f58153300 2024-03-09T09:00:48.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.643+0000 7f8f5ce00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f580ae6f0 con 0x7f8f580cadb0 2024-03-09T09:00:48.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.643+0000 7f8f57400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f58153300 0x7f8f580b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.643+0000 7f8f57400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f58153300 0x7f8f580b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40918/0 (socket says 172.21.15.67:40918) 2024-03-09T09:00:48.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.643+0000 7f8f57400640 1 -- 172.21.15.67:0/1732034622 learned_addr learned my addr 172.21.15.67:0/1732034622 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:48.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.643+0000 7f8f57e00640 1 --2- 172.21.15.67:0/1732034622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f580cadb0 0x7f8f580b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.644+0000 7f8f57400640 1 -- 172.21.15.67:0/1732034622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f580cadb0 msgr2=0x7f8f580b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:48.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.644+0000 7f8f57400640 1 --2- 172.21.15.67:0/1732034622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f580cadb0 0x7f8f580b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.644+0000 7f8f57400640 1 -- 172.21.15.67:0/1732034622 >> v1:172.21.15.67:6789/0 conn(0x7f8f580c5e40 legacy=0x7f8f58156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.644+0000 7f8f56a00640 1 -- 172.21.15.67:0/1732034622 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734809257 0 0) 0x7f8f58157000 con 0x7f8f580c5e40 2024-03-09T09:00:48.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.644+0000 7f8f57400640 1 -- 172.21.15.67:0/1732034622 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58155b10 con 0x7f8f58153300 2024-03-09T09:00:48.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.645+0000 7f8f57400640 1 --2- 172.21.15.67:0/1732034622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f58153300 0x7f8f580b0000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009870 tx=0x7f8f4802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e551a86b12dd553 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.645+0000 7f8f56a00640 1 -- 172.21.15.67:0/1732034622 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48034070 con 0x7f8f58153300 2024-03-09T09:00:48.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.646+0000 7f8f56a00640 1 -- 172.21.15.67:0/1732034622 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f48030840 con 0x7f8f58153300 2024-03-09T09:00:48.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.646+0000 7f8f56a00640 1 -- 172.21.15.67:0/1732034622 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48030c10 con 0x7f8f58153300 2024-03-09T09:00:48.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.646+0000 7f8f5ce00640 1 -- 172.21.15.67:0/1732034622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f58153300 msgr2=0x7f8f580b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.646+0000 7f8f5ce00640 1 --2- 172.21.15.67:0/1732034622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f58153300 0x7f8f580b0000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009870 tx=0x7f8f4802b980 comp rx=0 tx=0).stop 2024-03-09T09:00:48.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.646+0000 7f8f5ce00640 1 -- 172.21.15.67:0/1732034622 shutdown_connections 2024-03-09T09:00:48.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.646+0000 7f8f5ce00640 1 --2- 172.21.15.67:0/1732034622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f580cadb0 0x7f8f580b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.646+0000 7f8f5ce00640 1 --2- 172.21.15.67:0/1732034622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f58153300 0x7f8f580b0000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.646+0000 7f8f5ce00640 1 -- 172.21.15.67:0/1732034622 >> 172.21.15.67:0/1732034622 conn(0x7f8f580732d0 msgr2=0x7f8f580ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:48.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.646+0000 7f8f5ce00640 1 -- 172.21.15.67:0/1732034622 shutdown_connections 2024-03-09T09:00:48.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.646+0000 7f8f5ce00640 1 -- 172.21.15.67:0/1732034622 wait complete. 2024-03-09T09:00:48.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.648+0000 7f8f5ce00640 1 Processor -- start 2024-03-09T09:00:48.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.648+0000 7f8f5ce00640 1 -- start start 2024-03-09T09:00:48.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.649+0000 7f8f5ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f580c5e40 0x7f8f581539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.649+0000 7f8f5ce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f580cadb0 0x7f8f581ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.649+0000 7f8f5ce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f58153300 0x7f8f5814d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.649+0000 7f8f5ce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f580c59e0 con 0x7f8f580c5e40 2024-03-09T09:00:48.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.649+0000 7f8f5ce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f58159e90 con 0x7f8f58153300 2024-03-09T09:00:48.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.649+0000 7f8f5ce00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f58156720 con 0x7f8f580cadb0 2024-03-09T09:00:48.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.649+0000 7f8f57e00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f58153300 msgr2=0x7f8f5814d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:00:48.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.649+0000 7f8f57400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f580c5e40 0x7f8f581539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.649+0000 7f8f57e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f58153300 0x7f8f5814d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:48.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.649+0000 7f8f57400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f580c5e40 0x7f8f581539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40930/0 (socket says 172.21.15.67:40930) 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.649+0000 7f8f57400640 1 -- 172.21.15.67:0/2291083879 learned_addr learned my addr 172.21.15.67:0/2291083879 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.650+0000 7f8f4fe00640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f580cadb0 0x7f8f581ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.650+0000 7f8f57400640 1 -- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f580cadb0 msgr2=0x7f8f581ef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.650+0000 7f8f57400640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f580cadb0 0x7f8f581ef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.650+0000 7f8f57400640 1 -- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f58153300 msgr2=0x7f8f5814d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.650+0000 7f8f57400640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f58153300 0x7f8f5814d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.650+0000 7f8f57400640 1 -- 172.21.15.67:0/2291083879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f580acd40 con 0x7f8f580c5e40 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.650+0000 7f8f57400640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f580c5e40 0x7f8f581539c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8f480090b0 tx=0x7f8f480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.651+0000 7f8f4fe00640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f580cadb0 0x7f8f581ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.651+0000 7f8f4f400640 1 -- 172.21.15.67:0/2291083879 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48034040 con 0x7f8f580c5e40 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.651+0000 7f8f4f400640 1 -- 172.21.15.67:0/2291083879 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4803ab10 con 0x7f8f580c5e40 2024-03-09T09:00:48.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.651+0000 7f8f4f400640 1 -- 172.21.15.67:0/2291083879 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48004030 con 0x7f8f580c5e40 2024-03-09T09:00:48.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.651+0000 7f8f5ce00640 1 -- 172.21.15.67:0/2291083879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f581ef730 con 0x7f8f580c5e40 2024-03-09T09:00:48.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.651+0000 7f8f5ce00640 1 -- 172.21.15.67:0/2291083879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f580acfe0 con 0x7f8f580c5e40 2024-03-09T09:00:48.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.653+0000 7f8f5ce00640 1 -- 172.21.15.67:0/2291083879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5804f3c0 con 0x7f8f580c5e40 2024-03-09T09:00:48.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.654+0000 7f8f4f400640 1 -- 172.21.15.67:0/2291083879 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8f4804b060 con 0x7f8f580c5e40 2024-03-09T09:00:48.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.656+0000 7f8f4f400640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f3406b480 0x7f8f3406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:48.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.656+0000 7f8f4f400640 1 -- 172.21.15.67:0/2291083879 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8f480ac220 con 0x7f8f580c5e40 2024-03-09T09:00:48.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.656+0000 7f8f4fe00640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f3406b480 0x7f8f3406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:48.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.657+0000 7f8f4fe00640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f3406b480 0x7f8f3406d940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8f580ac990 tx=0x7f8f40009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:48.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.662+0000 7f8f4f400640 1 -- 172.21.15.67:0/2291083879 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8f4803f070 con 0x7f8f580c5e40 2024-03-09T09:00:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.947+0000 7f8f5ce00640 1 -- 172.21.15.67:0/2291083879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8f580cab90 con 0x7f8f580c5e40 2024-03-09T09:00:48.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.948+0000 7f8f4f400640 1 -- 172.21.15.67:0/2291083879 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f8f4807ae60 con 0x7f8f580c5e40 2024-03-09T09:00:48.949 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:00:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.951+0000 7f8f5ce00640 1 -- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f3406b480 msgr2=0x7f8f3406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.951+0000 7f8f5ce00640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f3406b480 0x7f8f3406d940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8f580ac990 tx=0x7f8f40009040 comp rx=0 tx=0).stop 2024-03-09T09:00:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.951+0000 7f8f5ce00640 1 -- 172.21.15.67:0/2291083879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f580c5e40 msgr2=0x7f8f581539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.951+0000 7f8f5ce00640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f580c5e40 0x7f8f581539c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8f480090b0 tx=0x7f8f480046c0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.951+0000 7f8f5ce00640 1 -- 172.21.15.67:0/2291083879 shutdown_connections 2024-03-09T09:00:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.951+0000 7f8f5ce00640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f3406b480 0x7f8f3406d940 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.951+0000 7f8f5ce00640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f58153300 0x7f8f5814d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.951+0000 7f8f5ce00640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f580cadb0 0x7f8f581ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.951+0000 7f8f5ce00640 1 --2- 172.21.15.67:0/2291083879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f580c5e40 0x7f8f581539c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:48.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.951+0000 7f8f5ce00640 1 -- 172.21.15.67:0/2291083879 >> 172.21.15.67:0/2291083879 conn(0x7f8f580732d0 msgr2=0x7f8f580c92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:48.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.952+0000 7f8f5ce00640 1 -- 172.21.15.67:0/2291083879 shutdown_connections 2024-03-09T09:00:48.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:48.952+0000 7f8f5ce00640 1 -- 172.21.15.67:0/2291083879 wait complete. 2024-03-09T09:00:51.979 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:00:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.188+0000 7f53c9a00640 1 Processor -- start 2024-03-09T09:00:52.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.188+0000 7f53c9a00640 1 -- start start 2024-03-09T09:00:52.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.190+0000 7f53c9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f53c40c1d00 0x7f53c40aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:52.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.190+0000 7f53c9a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53c40aa8f0 0x7f53c4159770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:52.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.190+0000 7f53c9a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c406de60 con 0x7f53c40acd90 2024-03-09T09:00:52.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.190+0000 7f53c9a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c41592e0 con 0x7f53c40c1d00 2024-03-09T09:00:52.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.190+0000 7f53c9a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c406d5c0 con 0x7f53c40aa8f0 2024-03-09T09:00:52.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.190+0000 7f53c3e00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f53c40acd90 legacy=0x7f53c41565f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:00:52.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.190+0000 7f53bbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f53c40c1d00 0x7f53c40aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:52.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.190+0000 7f53bbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f53c40c1d00 0x7f53c40aa3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53170/0 (socket says 172.21.15.67:53170) 2024-03-09T09:00:52.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.190+0000 7f53bbe00640 1 -- 172.21.15.67:0/2190714249 learned_addr learned my addr 172.21.15.67:0/2190714249 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:52.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.191+0000 7f53c9000640 1 --2- 172.21.15.67:0/2190714249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53c40aa8f0 0x7f53c4159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:52.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.191+0000 7f53bbe00640 1 -- 172.21.15.67:0/2190714249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53c40aa8f0 msgr2=0x7f53c4159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:00:52.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.191+0000 7f53bbe00640 1 --2- 172.21.15.67:0/2190714249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53c40aa8f0 0x7f53c4159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:52.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.191+0000 7f53bbe00640 1 -- 172.21.15.67:0/2190714249 >> v1:172.21.15.134:6789/0 conn(0x7f53c40acd90 legacy=0x7f53c41565f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:00:52.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.191+0000 7f53bbe00640 1 -- 172.21.15.67:0/2190714249 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c406d8a0 con 0x7f53c40c1d00 2024-03-09T09:00:52.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.191+0000 7f53c9000640 1 --2- 172.21.15.67:0/2190714249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53c40aa8f0 0x7f53c4159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:00:52.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.192+0000 7f53bbe00640 1 --2- 172.21.15.67:0/2190714249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40c1d00 0x7f53c40aa3b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009870 tx=0x7f53ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a43fa39c0ed8bc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:52.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.192+0000 7f53c3400640 1 -- 172.21.15.67:0/2190714249 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac035020 con 0x7f53c40c1d00 2024-03-09T09:00:52.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.192+0000 7f53c3400640 1 -- 172.21.15.67:0/2190714249 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac031640 con 0x7f53c40c1d00 2024-03-09T09:00:52.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.192+0000 7f53c3400640 1 -- 172.21.15.67:0/2190714249 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac02d040 con 0x7f53c40c1d00 2024-03-09T09:00:52.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.193+0000 7f53c9a00640 1 -- 172.21.15.67:0/2190714249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40c1d00 msgr2=0x7f53c40aa3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:52.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.193+0000 7f53c9a00640 1 --2- 172.21.15.67:0/2190714249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40c1d00 0x7f53c40aa3b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009870 tx=0x7f53ac02b8f0 comp rx=0 tx=0).stop 2024-03-09T09:00:52.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.193+0000 7f53c9a00640 1 -- 172.21.15.67:0/2190714249 shutdown_connections 2024-03-09T09:00:52.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.193+0000 7f53c9a00640 1 --2- 172.21.15.67:0/2190714249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53c40aa8f0 0x7f53c4159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:52.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.193+0000 7f53c9a00640 1 --2- 172.21.15.67:0/2190714249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40c1d00 0x7f53c40aa3b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:52.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.193+0000 7f53c9a00640 1 -- 172.21.15.67:0/2190714249 >> 172.21.15.67:0/2190714249 conn(0x7f53c40c7300 msgr2=0x7f53c40c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:52.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.193+0000 7f53c9a00640 1 -- 172.21.15.67:0/2190714249 shutdown_connections 2024-03-09T09:00:52.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.193+0000 7f53c9a00640 1 -- 172.21.15.67:0/2190714249 wait complete. 2024-03-09T09:00:52.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.195+0000 7f53c9a00640 1 Processor -- start 2024-03-09T09:00:52.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.195+0000 7f53c9a00640 1 -- start start 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.195+0000 7f53c9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40aa8f0 0x7f53c41482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c9a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c40acd90 0x7f53c4148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c9a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53c40c1d00 0x7f53c4153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c9a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40ad2f0 con 0x7f53c40aa8f0 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c9a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40ad170 con 0x7f53c40acd90 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c9a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40ac200 con 0x7f53c40c1d00 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53bbe00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c40acd90 msgr2=0x7f53c4148bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53bbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c40acd90 0x7f53c4148bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40aa8f0 0x7f53c41482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40aa8f0 0x7f53c41482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53174/0 (socket says 172.21.15.67:53174) 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c3e00640 1 -- 172.21.15.67:0/3095994116 learned_addr learned my addr 172.21.15.67:0/3095994116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:52.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c9000640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53c40c1d00 0x7f53c4153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:52.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c3e00640 1 -- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53c40c1d00 msgr2=0x7f53c4153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:52.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c3e00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53c40c1d00 0x7f53c4153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:52.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c3e00640 1 -- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c40acd90 msgr2=0x7f53c4148bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:00:52.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c3e00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c40acd90 0x7f53c4148bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:52.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.196+0000 7f53c3e00640 1 -- 172.21.15.67:0/3095994116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c40c6ea0 con 0x7f53c40aa8f0 2024-03-09T09:00:52.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.197+0000 7f53c3e00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40aa8f0 0x7f53c41482e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f53b400b550 tx=0x7f53b400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:52.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.197+0000 7f53c0c00640 1 -- 172.21.15.67:0/3095994116 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4013020 con 0x7f53c40aa8f0 2024-03-09T09:00:52.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.197+0000 7f53c0c00640 1 -- 172.21.15.67:0/3095994116 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b4003de0 con 0x7f53c40aa8f0 2024-03-09T09:00:52.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.197+0000 7f53c9a00640 1 -- 172.21.15.67:0/3095994116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c41488e0 con 0x7f53c40aa8f0 2024-03-09T09:00:52.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.197+0000 7f53c0c00640 1 -- 172.21.15.67:0/3095994116 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b40052b0 con 0x7f53c40aa8f0 2024-03-09T09:00:52.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.197+0000 7f53c9a00640 1 -- 172.21.15.67:0/3095994116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c41ec3f0 con 0x7f53c40aa8f0 2024-03-09T09:00:52.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.199+0000 7f53c9a00640 1 -- 172.21.15.67:0/3095994116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c404f3c0 con 0x7f53c40aa8f0 2024-03-09T09:00:52.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.199+0000 7f53c0c00640 1 -- 172.21.15.67:0/3095994116 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f53b4016020 con 0x7f53c40aa8f0 2024-03-09T09:00:52.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.200+0000 7f53c0c00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f53a006b480 0x7f53a006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:52.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.200+0000 7f53c0c00640 1 -- 172.21.15.67:0/3095994116 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f53b408a340 con 0x7f53c40aa8f0 2024-03-09T09:00:52.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.201+0000 7f53bbe00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f53a006b480 0x7f53a006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:52.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.202+0000 7f53bbe00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f53a006b480 0x7f53a006d940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f53ac036000 tx=0x7f53ac008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:52.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.205+0000 7f53c0c00640 1 -- 172.21.15.67:0/3095994116 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f53b401d020 con 0x7f53c40aa8f0 2024-03-09T09:00:52.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.504+0000 7f53c9a00640 1 -- 172.21.15.67:0/3095994116 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f53c41532a0 con 0x7f53c40aa8f0 2024-03-09T09:00:52.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.505+0000 7f53c0c00640 1 -- 172.21.15.67:0/3095994116 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f53b4058dc0 con 0x7f53c40aa8f0 2024-03-09T09:00:52.506 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:00:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.507+0000 7f53c9a00640 1 -- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f53a006b480 msgr2=0x7f53a006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.507+0000 7f53c9a00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f53a006b480 0x7f53a006d940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f53ac036000 tx=0x7f53ac008030 comp rx=0 tx=0).stop 2024-03-09T09:00:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.507+0000 7f53c9a00640 1 -- 172.21.15.67:0/3095994116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40aa8f0 msgr2=0x7f53c41482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.507+0000 7f53c9a00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40aa8f0 0x7f53c41482e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f53b400b550 tx=0x7f53b400ba10 comp rx=0 tx=0).stop 2024-03-09T09:00:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.508+0000 7f53c9a00640 1 -- 172.21.15.67:0/3095994116 shutdown_connections 2024-03-09T09:00:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.508+0000 7f53c9a00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f53a006b480 0x7f53a006d940 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.508+0000 7f53c9a00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53c40c1d00 0x7f53c4153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.508+0000 7f53c9a00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53c40acd90 0x7f53c4148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.508+0000 7f53c9a00640 1 --2- 172.21.15.67:0/3095994116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53c40aa8f0 0x7f53c41482e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.508+0000 7f53c9a00640 1 -- 172.21.15.67:0/3095994116 >> 172.21.15.67:0/3095994116 conn(0x7f53c40c7300 msgr2=0x7f53c40c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.508+0000 7f53c9a00640 1 -- 172.21.15.67:0/3095994116 shutdown_connections 2024-03-09T09:00:52.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:52.508+0000 7f53c9a00640 1 -- 172.21.15.67:0/3095994116 wait complete. 2024-03-09T09:00:55.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:00:55.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.757+0000 7f98aaa00640 1 Processor -- start 2024-03-09T09:00:55.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.757+0000 7f98aaa00640 1 -- start start 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.758+0000 7f98aaa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f98a41570d0 0x7f98a40cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.759+0000 7f98aaa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f98a40c9440 0x7f98a40c9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.759+0000 7f98aaa00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a4156700 con 0x7f98a41531c0 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.759+0000 7f98aaa00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a40afba0 con 0x7f98a41570d0 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.759+0000 7f98aaa00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a40b01c0 con 0x7f98a40c9440 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.759+0000 7f98a9600640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f98a41531c0 legacy=0x7f98a40ae590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.759+0000 7f98aa000640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f98a40c9440 msgr2=0x7f98a40c9870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.759+0000 7f98aa000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f98a40c9440 0x7f98a40c9870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.759+0000 7f98a8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f98a41570d0 0x7f98a40cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.759+0000 7f98a8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f98a41570d0 0x7f98a40cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53178/0 (socket says 172.21.15.67:53178) 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.759+0000 7f98a8c00640 1 -- 172.21.15.67:0/244717252 learned_addr learned my addr 172.21.15.67:0/244717252 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.760+0000 7f98a8c00640 1 -- 172.21.15.67:0/244717252 >> v2:172.21.15.134:3300/0 conn(0x7f98a40c9440 msgr2=0x7f98a40c9870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.760+0000 7f98a8c00640 1 --2- 172.21.15.67:0/244717252 >> v2:172.21.15.134:3300/0 conn(0x7f98a40c9440 0x7f98a40c9870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.760+0000 7f98a8c00640 1 -- 172.21.15.67:0/244717252 >> v1:172.21.15.134:6789/0 conn(0x7f98a41531c0 legacy=0x7f98a40ae590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:00:55.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.760+0000 7f98a8c00640 1 -- 172.21.15.67:0/244717252 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a4157530 con 0x7f98a41570d0 2024-03-09T09:00:55.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.761+0000 7f98a8c00640 1 --2- 172.21.15.67:0/244717252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98a41570d0 0x7f98a40cae50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f98a40ca770 tx=0x7f989002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46a3a0262678702 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:55.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.761+0000 7f98a3e00640 1 -- 172.21.15.67:0/244717252 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890035020 con 0x7f98a41570d0 2024-03-09T09:00:55.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.761+0000 7f98a3e00640 1 -- 172.21.15.67:0/244717252 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9890031800 con 0x7f98a41570d0 2024-03-09T09:00:55.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.761+0000 7f98a3e00640 1 -- 172.21.15.67:0/244717252 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9890031bd0 con 0x7f98a41570d0 2024-03-09T09:00:55.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.762+0000 7f98aaa00640 1 -- 172.21.15.67:0/244717252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98a41570d0 msgr2=0x7f98a40cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:55.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.762+0000 7f98aaa00640 1 --2- 172.21.15.67:0/244717252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98a41570d0 0x7f98a40cae50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f98a40ca770 tx=0x7f989002b8a0 comp rx=0 tx=0).stop 2024-03-09T09:00:55.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.762+0000 7f98aaa00640 1 -- 172.21.15.67:0/244717252 shutdown_connections 2024-03-09T09:00:55.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.762+0000 7f98aaa00640 1 --2- 172.21.15.67:0/244717252 >> v2:172.21.15.134:3300/0 conn(0x7f98a40c9440 0x7f98a40c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:55.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.762+0000 7f98aaa00640 1 --2- 172.21.15.67:0/244717252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98a41570d0 0x7f98a40cae50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:55.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.762+0000 7f98aaa00640 1 -- 172.21.15.67:0/244717252 >> 172.21.15.67:0/244717252 conn(0x7f98a40ad190 msgr2=0x7f98a40ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:55.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.762+0000 7f98aaa00640 1 -- 172.21.15.67:0/244717252 shutdown_connections 2024-03-09T09:00:55.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.762+0000 7f98aaa00640 1 -- 172.21.15.67:0/244717252 wait complete. 2024-03-09T09:00:55.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.763+0000 7f98aaa00640 1 Processor -- start 2024-03-09T09:00:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.764+0000 7f98aaa00640 1 -- start start 2024-03-09T09:00:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.764+0000 7f98aaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98a40c9440 0x7f98a40ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.764+0000 7f98aaa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a41531c0 0x7f98a41eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98aaa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a41570d0 0x7f98a41ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98aaa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40b0410 con 0x7f98a40c9440 2024-03-09T09:00:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98aaa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40cb390 con 0x7f98a41531c0 2024-03-09T09:00:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98aaa00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40b06f0 con 0x7f98a41570d0 2024-03-09T09:00:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98a8c00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a41531c0 msgr2=0x7f98a41eb5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:00:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98a9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98a40c9440 0x7f98a40ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98a8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a41531c0 0x7f98a41eb5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98a9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98a40c9440 0x7f98a40ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53188/0 (socket says 172.21.15.67:53188) 2024-03-09T09:00:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98a9600640 1 -- 172.21.15.67:0/523331548 learned_addr learned my addr 172.21.15.67:0/523331548 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:00:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98aa000640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a41570d0 0x7f98a41ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98a9600640 1 -- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a41570d0 msgr2=0x7f98a41ebb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98a9600640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a41570d0 0x7f98a41ebb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98a9600640 1 -- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a41531c0 msgr2=0x7f98a41eb5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:00:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98a9600640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a41531c0 0x7f98a41eb5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.765+0000 7f98a9600640 1 -- 172.21.15.67:0/523331548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a41e2b00 con 0x7f98a40c9440 2024-03-09T09:00:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.766+0000 7f98a9600640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98a40c9440 0x7f98a40ac470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f989800b9e0 tx=0x7f989800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.766+0000 7f98a1600640 1 -- 172.21.15.67:0/523331548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800cba0 con 0x7f98a40c9440 2024-03-09T09:00:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.766+0000 7f98a1600640 1 -- 172.21.15.67:0/523331548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9898010070 con 0x7f98a40c9440 2024-03-09T09:00:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.766+0000 7f98aaa00640 1 -- 172.21.15.67:0/523331548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a41eeb60 con 0x7f98a40c9440 2024-03-09T09:00:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.766+0000 7f98a1600640 1 -- 172.21.15.67:0/523331548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898015470 con 0x7f98a40c9440 2024-03-09T09:00:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.766+0000 7f98aaa00640 1 -- 172.21.15.67:0/523331548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a41eee20 con 0x7f98a40c9440 2024-03-09T09:00:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.767+0000 7f98aaa00640 1 -- 172.21.15.67:0/523331548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a404f3c0 con 0x7f98a40c9440 2024-03-09T09:00:55.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.768+0000 7f98a1600640 1 -- 172.21.15.67:0/523331548 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f989801e3f0 con 0x7f98a40c9440 2024-03-09T09:00:55.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.770+0000 7f98a1600640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f988406b480 0x7f988406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:55.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.770+0000 7f98a1600640 1 -- 172.21.15.67:0/523331548 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f989808a8d0 con 0x7f98a40c9440 2024-03-09T09:00:55.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.771+0000 7f98a8c00640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f988406b480 0x7f988406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:55.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.772+0000 7f98a8c00640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f988406b480 0x7f988406d940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9890036000 tx=0x7f9890008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:00:55.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:55.774+0000 7f98a1600640 1 -- 172.21.15.67:0/523331548 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f98980592c0 con 0x7f98a40c9440 2024-03-09T09:00:56.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.103+0000 7f98aaa00640 1 -- 172.21.15.67:0/523331548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f98a40cac30 con 0x7f98a40c9440 2024-03-09T09:00:56.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.104+0000 7f98a1600640 1 -- 172.21.15.67:0/523331548 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f9898058bc0 con 0x7f98a40c9440 2024-03-09T09:00:56.106 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":66,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:00:56.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 -- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f988406b480 msgr2=0x7f988406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:56.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f988406b480 0x7f988406d940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9890036000 tx=0x7f9890008030 comp rx=0 tx=0).stop 2024-03-09T09:00:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 -- 172.21.15.67:0/523331548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98a40c9440 msgr2=0x7f98a40ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:00:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98a40c9440 0x7f98a40ac470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f989800b9e0 tx=0x7f989800bea0 comp rx=0 tx=0).stop 2024-03-09T09:00:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 -- 172.21.15.67:0/523331548 shutdown_connections 2024-03-09T09:00:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f988406b480 0x7f988406d940 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98a41570d0 0x7f98a41ebb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98a41531c0 0x7f98a41eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 --2- 172.21.15.67:0/523331548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98a40c9440 0x7f98a40ac470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:00:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 -- 172.21.15.67:0/523331548 >> 172.21.15.67:0/523331548 conn(0x7f98a40ad190 msgr2=0x7f98a4158f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:00:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 -- 172.21.15.67:0/523331548 shutdown_connections 2024-03-09T09:00:56.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:56.108+0000 7f98aaa00640 1 -- 172.21.15.67:0/523331548 wait complete. 2024-03-09T09:00:59.132 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:00:59.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.386+0000 7faac7e00640 1 Processor -- start 2024-03-09T09:00:59.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.386+0000 7faac7e00640 1 -- start start 2024-03-09T09:00:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.389+0000 7faac7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.389+0000 7faac7e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faac81511f0 0x7faac8070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:00:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.389+0000 7faac7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac80c3ba0 con 0x7faac80c6b70 2024-03-09T09:00:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.389+0000 7faac7e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac8154370 con 0x7faac81511f0 2024-03-09T09:00:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.389+0000 7faac7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac81548b0 con 0x7faac80c5da0 2024-03-09T09:00:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.390+0000 7faac7400640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7faac81511f0 msgr2=0x7faac8070170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:00:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.390+0000 7faac7400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faac81511f0 0x7faac8070170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.390+0000 7faac6000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faac80c6b70 0x7faac8150ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53356/0 (socket says 172.21.15.67:53356) 2024-03-09T09:00:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.390+0000 7faac6000640 1 -- 172.21.15.67:0/1460258789 learned_addr learned my addr 172.21.15.67:0/1460258789 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:00:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.390+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.390+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 msgr2=0x7faac80c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:00:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.390+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 msgr2=0x7faac80c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.390+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.390+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:00:59.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.591+0000 7faac7400640 1 -- 172.21.15.67:0/1460258789 >> v2:172.21.15.134:3300/0 conn(0x7faac81511f0 msgr2=0x7faac8070170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:00:59.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.591+0000 7faac7400640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.134:3300/0 conn(0x7faac81511f0 0x7faac8070170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:00:59.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.591+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:59.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.592+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 msgr2=0x7faac80c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:00:59.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.592+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 msgr2=0x7faac80c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:59.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.592+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:59.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.592+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:00:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.992+0000 7faac7400640 1 -- 172.21.15.67:0/1460258789 >> v2:172.21.15.134:3300/0 conn(0x7faac81511f0 msgr2=0x7faac8070170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:00:59.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.992+0000 7faac7400640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.134:3300/0 conn(0x7faac81511f0 0x7faac8070170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:00:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.994+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:00:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.994+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 msgr2=0x7faac80c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:00:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.994+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 msgr2=0x7faac80c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:00:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.994+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:00:59.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:00:59.994+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:01:00.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:00.793+0000 7faac7400640 1 -- 172.21.15.67:0/1460258789 >> v2:172.21.15.134:3300/0 conn(0x7faac81511f0 msgr2=0x7faac8070170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:01:00.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:00.794+0000 7faac7400640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.134:3300/0 conn(0x7faac81511f0 0x7faac8070170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:01:00.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:00.796+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:00.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:00.796+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 msgr2=0x7faac80c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:01:00.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:00.796+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 msgr2=0x7faac80c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:00.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:00.796+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:00.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:00.796+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:01:02.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.389+0000 7faabfe00640 1 -- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 msgr2=0x7faac80c6180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:02.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.389+0000 7faabfe00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faac80c5da0 0x7faac80c6180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:02.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.389+0000 7faabfe00640 1 -- 172.21.15.67:0/1460258789 >> v2:172.21.15.134:3300/0 conn(0x7faac81511f0 msgr2=0x7faac8070170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:02.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.390+0000 7faabfe00640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.134:3300/0 conn(0x7faac81511f0 0x7faac8070170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:02.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.390+0000 7faabfe00640 1 -- 172.21.15.67:0/1460258789 >> v1:172.21.15.67:6789/0 conn(0x7faac80c6b70 legacy=0x7faac8150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:02.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.390+0000 7faabfe00640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 0x7faaa8005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:02.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.390+0000 7faabfe00640 1 -- 172.21.15.67:0/1460258789 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac81545d0 con 0x7faaa8006190 2024-03-09T09:01:02.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.390+0000 7faabfe00640 1 -- 172.21.15.67:0/1460258789 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac8154370 con 0x7faaa80036c0 2024-03-09T09:01:02.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.390+0000 7faac6000640 1 -- 172.21.15.67:0/1460258789 >> v1:172.21.15.134:6789/0 conn(0x7faaa8006190 legacy=0x7faaa8008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:01:02.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.390+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 0x7faaa8005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:02.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.391+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 msgr2=0x7faaa8005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:01:02.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.391+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 msgr2=0x7faaa8005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:02.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.391+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 0x7faaa8005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:02.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.391+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 0x7faaa8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:02.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.591+0000 7faac6000640 1 -- 172.21.15.67:0/1460258789 >> v1:172.21.15.134:6789/0 conn(0x7faaa8006190 legacy=0x7faaa8008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:01:02.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.592+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 0x7faaa8005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:02.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.593+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 msgr2=0x7faaa8005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:01:02.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.593+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 msgr2=0x7faaa8005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:02.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.593+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 0x7faaa8005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:02.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.593+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 0x7faaa8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:01:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.992+0000 7faac6000640 1 -- 172.21.15.67:0/1460258789 >> v1:172.21.15.134:6789/0 conn(0x7faaa8006190 legacy=0x7faaa8008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:01:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.994+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> v2:172.21.15.67:3300/0 conn(0x7faaa80036c0 0x7faaa8005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.995+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 >> v1:172.21.15.134:6789/0 conn(0x7faaa8006190 legacy=0x7faaa8008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.995+0000 7faac6a00640 1 -- 172.21.15.67:0/1460258789 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa800af70 con 0x7faaa80036c0 2024-03-09T09:01:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.998+0000 7faac6a00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa80036c0 0x7faaa8005b40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faab8006fd0 tx=0x7faab802d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9a0d59b95e6003 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.999+0000 7faac5600640 1 -- 172.21.15.67:0/1460258789 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8033070 con 0x7faaa80036c0 2024-03-09T09:01:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.999+0000 7faac5600640 1 -- 172.21.15.67:0/1460258789 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab8031070 con 0x7faaa80036c0 2024-03-09T09:01:03.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:02.999+0000 7faac5600640 1 -- 172.21.15.67:0/1460258789 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8038460 con 0x7faaa80036c0 2024-03-09T09:01:03.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.001+0000 7faac7e00640 1 -- 172.21.15.67:0/1460258789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa80036c0 msgr2=0x7faaa8005b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.001+0000 7faac7e00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa80036c0 0x7faaa8005b40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faab8006fd0 tx=0x7faab802d330 comp rx=0 tx=0).stop 2024-03-09T09:01:03.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.001+0000 7faac7400640 1 -- 172.21.15.67:0/1460258789 reap_dead start 2024-03-09T09:01:03.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.001+0000 7faac7e00640 1 -- 172.21.15.67:0/1460258789 shutdown_connections 2024-03-09T09:01:03.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.001+0000 7faac7e00640 1 --2- 172.21.15.67:0/1460258789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaa80036c0 0x7faaa8005b40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.001+0000 7faac7e00640 1 -- 172.21.15.67:0/1460258789 >> 172.21.15.67:0/1460258789 conn(0x7faac80c9470 msgr2=0x7faac80c9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:03.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.001+0000 7faac7e00640 1 -- 172.21.15.67:0/1460258789 shutdown_connections 2024-03-09T09:01:03.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.001+0000 7faac7e00640 1 -- 172.21.15.67:0/1460258789 wait complete. 2024-03-09T09:01:03.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.005+0000 7faac7e00640 1 Processor -- start 2024-03-09T09:01:03.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.005+0000 7faac7e00640 1 -- start start 2024-03-09T09:01:03.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.006+0000 7faac7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa80036c0 0x7faac806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.006+0000 7faac7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faac80c2b10 0x7faac80c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.006+0000 7faac7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac80c85f0 0x7faac81ee900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.006+0000 7faac7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac80c4560 con 0x7faac80c2b10 2024-03-09T09:01:03.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.006+0000 7faac7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac80c41f0 con 0x7faac80c85f0 2024-03-09T09:01:03.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.006+0000 7faac7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac80c66c0 con 0x7faaa80036c0 2024-03-09T09:01:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.007+0000 7faac6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faac80c2b10 0x7faac80c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.007+0000 7faac6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa80036c0 0x7faac806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.007+0000 7faac6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa80036c0 0x7faac806dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:52826/0 (socket says 172.21.15.67:52826) 2024-03-09T09:01:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.007+0000 7faac6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faac80c2b10 0x7faac80c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45586/0 (socket says 172.21.15.67:45586) 2024-03-09T09:01:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.007+0000 7faac6a00640 1 -- 172.21.15.67:0/465664897 learned_addr learned my addr 172.21.15.67:0/465664897 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.007+0000 7faac7400640 1 -- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac80c85f0 msgr2=0x7faac81ee900 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:01:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.007+0000 7faac7400640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac80c85f0 0x7faac81ee900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.008+0000 7faac6a00640 1 -- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac80c85f0 msgr2=0x7faac81ee900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:01:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.008+0000 7faac6a00640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac80c85f0 0x7faac81ee900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.008+0000 7faac6a00640 1 -- 172.21.15.67:0/465664897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faac80c2b10 msgr2=0x7faac80c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.008+0000 7faac6a00640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faac80c2b10 0x7faac80c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.008+0000 7faac6a00640 1 -- 172.21.15.67:0/465664897 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac81eb6e0 con 0x7faaa80036c0 2024-03-09T09:01:03.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.008+0000 7faac6000640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faac80c2b10 0x7faac80c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:01:03.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.008+0000 7faac6a00640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa80036c0 0x7faac806dbe0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faab8015a40 tx=0x7faab80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.009+0000 7faabea00640 1 -- 172.21.15.67:0/465664897 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8004020 con 0x7faaa80036c0 2024-03-09T09:01:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.009+0000 7faabea00640 1 -- 172.21.15.67:0/465664897 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab80041c0 con 0x7faaa80036c0 2024-03-09T09:01:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.009+0000 7faabea00640 1 -- 172.21.15.67:0/465664897 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8003610 con 0x7faaa80036c0 2024-03-09T09:01:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.009+0000 7faac7e00640 1 -- 172.21.15.67:0/465664897 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac81eb9c0 con 0x7faaa80036c0 2024-03-09T09:01:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.009+0000 7faac7e00640 1 -- 172.21.15.67:0/465664897 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac81479d0 con 0x7faaa80036c0 2024-03-09T09:01:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.011+0000 7faac7e00640 1 -- 172.21.15.67:0/465664897 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac8071210 con 0x7faaa80036c0 2024-03-09T09:01:03.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.012+0000 7faabea00640 1 -- 172.21.15.67:0/465664897 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faab804a060 con 0x7faaa80036c0 2024-03-09T09:01:03.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.013+0000 7faabea00640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faaa806c230 0x7faaa806e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.013+0000 7faabea00640 1 -- 172.21.15.67:0/465664897 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7faab80ad210 con 0x7faaa80036c0 2024-03-09T09:01:03.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.014+0000 7faac6000640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faaa806c230 0x7faaa806e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.015+0000 7faac6000640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faaa806c230 0x7faaa806e6f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faab4000f60 tx=0x7faab4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.031+0000 7faabea00640 1 -- 172.21.15.67:0/465664897 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faab807bd30 con 0x7faaa80036c0 2024-03-09T09:01:03.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.382+0000 7faac7e00640 1 -- 172.21.15.67:0/465664897 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7faac804f3c0 con 0x7faaa80036c0 2024-03-09T09:01:03.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.383+0000 7faabea00640 1 -- 172.21.15.67:0/465664897 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7faab807b630 con 0x7faaa80036c0 2024-03-09T09:01:03.384 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":70,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:01:03.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 -- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faaa806c230 msgr2=0x7faaa806e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faaa806c230 0x7faaa806e6f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faab4000f60 tx=0x7faab4009040 comp rx=0 tx=0).stop 2024-03-09T09:01:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 -- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa80036c0 msgr2=0x7faac806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa80036c0 0x7faac806dbe0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faab8015a40 tx=0x7faab80046c0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 -- 172.21.15.67:0/465664897 shutdown_connections 2024-03-09T09:01:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faaa806c230 0x7faaa806e6f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faac80c85f0 0x7faac81ee900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faac80c2b10 0x7faac80c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 --2- 172.21.15.67:0/465664897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa80036c0 0x7faac806dbe0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 -- 172.21.15.67:0/465664897 >> 172.21.15.67:0/465664897 conn(0x7faac80c9470 msgr2=0x7faac8073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 -- 172.21.15.67:0/465664897 shutdown_connections 2024-03-09T09:01:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.386+0000 7faac7e00640 1 -- 172.21.15.67:0/465664897 wait complete. 2024-03-09T09:01:03.409 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:01:03.409 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:01:03.409 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:01:03.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.601+0000 7fd9dee00640 1 Processor -- start 2024-03-09T09:01:03.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.602+0000 7fd9dee00640 1 -- start start 2024-03-09T09:01:03.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.603+0000 7fd9dee00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9d80ad8d0 0x7fd9d8159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.603+0000 7fd9dee00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd9d806b880 0x7fd9d8153080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.603+0000 7fd9dee00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d806c040 con 0x7fd9d80c6650 2024-03-09T09:01:03.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.603+0000 7fd9dee00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d80c61f0 con 0x7fd9d806b880 2024-03-09T09:01:03.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.603+0000 7fd9dee00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d806bd60 con 0x7fd9d80ad8d0 2024-03-09T09:01:03.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.603+0000 7fd9dd000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd9d806b880 0x7fd9d8153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.603+0000 7fd9dd000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd9d806b880 0x7fd9d8153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45600/0 (socket says 172.21.15.67:45600) 2024-03-09T09:01:03.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.603+0000 7fd9dd000640 1 -- 172.21.15.67:0/1794648345 learned_addr learned my addr 172.21.15.67:0/1794648345 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:03.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.604+0000 7fd9dda00640 1 --2- 172.21.15.67:0/1794648345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9d80ad8d0 0x7fd9d8159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.604+0000 7fd9dd000640 1 -- 172.21.15.67:0/1794648345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9d80ad8d0 msgr2=0x7fd9d8159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:03.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.604+0000 7fd9dd000640 1 --2- 172.21.15.67:0/1794648345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9d80ad8d0 0x7fd9d8159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.604+0000 7fd9dd000640 1 -- 172.21.15.67:0/1794648345 >> v1:172.21.15.67:6789/0 conn(0x7fd9d80c6650 legacy=0x7fd9d8155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.604+0000 7fd9dd000640 1 -- 172.21.15.67:0/1794648345 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80c5e80 con 0x7fd9d806b880 2024-03-09T09:01:03.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.604+0000 7fd9dda00640 1 --2- 172.21.15.67:0/1794648345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9d80ad8d0 0x7fd9d8159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:01:03.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.605+0000 7fd9dd000640 1 --2- 172.21.15.67:0/1794648345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d806b880 0x7fd9d8153080 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009870 tx=0x7fd9c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9340c3f4350124c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.605+0000 7fd9d7e00640 1 -- 172.21.15.67:0/1794648345 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4035020 con 0x7fd9d806b880 2024-03-09T09:01:03.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.605+0000 7fd9d7e00640 1 -- 172.21.15.67:0/1794648345 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4031870 con 0x7fd9d806b880 2024-03-09T09:01:03.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.605+0000 7fd9d7e00640 1 -- 172.21.15.67:0/1794648345 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4031c40 con 0x7fd9d806b880 2024-03-09T09:01:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.606+0000 7fd9dee00640 1 -- 172.21.15.67:0/1794648345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d806b880 msgr2=0x7fd9d8153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.606+0000 7fd9dee00640 1 --2- 172.21.15.67:0/1794648345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d806b880 0x7fd9d8153080 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009870 tx=0x7fd9c402b910 comp rx=0 tx=0).stop 2024-03-09T09:01:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.606+0000 7fd9dee00640 1 -- 172.21.15.67:0/1794648345 shutdown_connections 2024-03-09T09:01:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.606+0000 7fd9dee00640 1 --2- 172.21.15.67:0/1794648345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d806b880 0x7fd9d8153080 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.606+0000 7fd9dee00640 1 --2- 172.21.15.67:0/1794648345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9d80ad8d0 0x7fd9d8159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.606+0000 7fd9dee00640 1 -- 172.21.15.67:0/1794648345 >> 172.21.15.67:0/1794648345 conn(0x7fd9d80c8510 msgr2=0x7fd9d80acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.606+0000 7fd9dee00640 1 -- 172.21.15.67:0/1794648345 shutdown_connections 2024-03-09T09:01:03.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.606+0000 7fd9dee00640 1 -- 172.21.15.67:0/1794648345 wait complete. 2024-03-09T09:01:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.608+0000 7fd9dee00640 1 Processor -- start 2024-03-09T09:01:03.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.608+0000 7fd9dee00640 1 -- start start 2024-03-09T09:01:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.609+0000 7fd9dee00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9d806b880 0x7fd9d8157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.609+0000 7fd9dee00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d80ad8d0 0x7fd9d80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.609+0000 7fd9dee00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9d80c6650 0x7fd9d81ef230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.609+0000 7fd9dee00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81535c0 con 0x7fd9d80ad8d0 2024-03-09T09:01:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.609+0000 7fd9dee00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80adec0 con 0x7fd9d80c6650 2024-03-09T09:01:03.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.609+0000 7fd9dee00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d8153740 con 0x7fd9d806b880 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.609+0000 7fd9dd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d80ad8d0 0x7fd9d80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.609+0000 7fd9dd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d80ad8d0 0x7fd9d80ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45604/0 (socket says 172.21.15.67:45604) 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.609+0000 7fd9dd000640 1 -- 172.21.15.67:0/2994569154 learned_addr learned my addr 172.21.15.67:0/2994569154 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.609+0000 7fd9de400640 1 -- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9d80c6650 msgr2=0x7fd9d81ef230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.609+0000 7fd9de400640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9d80c6650 0x7fd9d81ef230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.610+0000 7fd9dda00640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9d806b880 0x7fd9d8157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.610+0000 7fd9dd000640 1 -- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9d806b880 msgr2=0x7fd9d8157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.610+0000 7fd9dd000640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9d806b880 0x7fd9d8157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.610+0000 7fd9dd000640 1 -- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9d80c6650 msgr2=0x7fd9d81ef230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.610+0000 7fd9dd000640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9d80c6650 0x7fd9d81ef230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.610+0000 7fd9dd000640 1 -- 172.21.15.67:0/2994569154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80acf60 con 0x7fd9d80ad8d0 2024-03-09T09:01:03.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.610+0000 7fd9dd000640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d80ad8d0 0x7fd9d80ac8d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd9c402b950 tx=0x7fd9c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.610+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4035040 con 0x7fd9d80ad8d0 2024-03-09T09:01:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.611+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4045480 con 0x7fd9d80ad8d0 2024-03-09T09:01:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.611+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4045870 con 0x7fd9d80ad8d0 2024-03-09T09:01:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.611+0000 7fd9dee00640 1 -- 172.21.15.67:0/2994569154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d80aae60 con 0x7fd9d80ad8d0 2024-03-09T09:01:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.611+0000 7fd9dee00640 1 -- 172.21.15.67:0/2994569154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d80ad240 con 0x7fd9d80ad8d0 2024-03-09T09:01:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.612+0000 7fd9dee00640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8154e70 0x7fd9d80ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.612+0000 7fd9dee00640 1 -- 172.21.15.67:0/2994569154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d804f3c0 con 0x7fd9d8154e70 2024-03-09T09:01:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.612+0000 7fd9dda00640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8154e70 0x7fd9d80ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.612+0000 7fd9dda00640 1 -- 172.21.15.67:0/2994569154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9d8151730 con 0x7fd9d8154e70 2024-03-09T09:01:03.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.613+0000 7fd9dda00640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8154e70 0x7fd9d80ab940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc007fa0 tx=0x7fd9cc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.613+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc003b90 con 0x7fd9d8154e70 2024-03-09T09:01:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.613+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd9cc003d30 con 0x7fd9d8154e70 2024-03-09T09:01:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.613+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8154e70 msgr2=0x7fd9d80ab940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.613+0000 7fd9d5600640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8154e70 0x7fd9d80ab940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc007fa0 tx=0x7fd9cc00c850 comp rx=0 tx=0).stop 2024-03-09T09:01:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.613+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8154e70 msgr2=0x7fd9d80ab940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.613+0000 7fd9d5600640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8154e70 0x7fd9d80ab940 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc007fa0 tx=0x7fd9cc00c850 comp rx=0 tx=0).stop 2024-03-09T09:01:03.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.614+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd9c4038020 con 0x7fd9d80ad8d0 2024-03-09T09:01:03.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.615+0000 7fd9d5600640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd9b806aef0 0x7fd9b806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.615+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd9c40acaf0 con 0x7fd9d80ad8d0 2024-03-09T09:01:03.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.616+0000 7fd9dda00640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd9b806aef0 0x7fd9b806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.617+0000 7fd9dda00640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd9b806aef0 0x7fd9b806d3b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc000f80 tx=0x7fd9cc015000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.621+0000 7fd9dee00640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8149490 0x7fd9d81498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.621+0000 7fd9dee00640 1 -- 172.21.15.67:0/2994569154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d8150d20 con 0x7fd9d8149490 2024-03-09T09:01:03.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.621+0000 7fd9de400640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8149490 0x7fd9d81498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.622+0000 7fd9de400640 1 -- 172.21.15.67:0/2994569154 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd9d8073a60 con 0x7fd9d8149490 2024-03-09T09:01:03.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.622+0000 7fd9de400640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8149490 0x7fd9d81498a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0001030 tx=0x7fd9d000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.622+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00074e0 con 0x7fd9d8149490 2024-03-09T09:01:03.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.622+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5177 (secure 0 0 0) 0x7fd9d00074e0 con 0x7fd9d8149490 2024-03-09T09:01:03.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.622+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8149490 msgr2=0x7fd9d81498a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.622+0000 7fd9d5600640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8149490 0x7fd9d81498a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0001030 tx=0x7fd9d000c7a0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.622+0000 7fd9d5600640 1 -- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8149490 msgr2=0x7fd9d81498a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:03.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.622+0000 7fd9d5600640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d8149490 0x7fd9d81498a0 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0001030 tx=0x7fd9d000c7a0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.623 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:01:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:01:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 70, 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:01:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:01:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:01:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:01:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:01:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:01:03.636 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:01:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.623+0000 7fd9dee00640 1 -- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd9b806aef0 msgr2=0x7fd9b806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.623+0000 7fd9dee00640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd9b806aef0 0x7fd9b806d3b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc000f80 tx=0x7fd9cc015000 comp rx=0 tx=0).stop 2024-03-09T09:01:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.623+0000 7fd9dee00640 1 -- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d80ad8d0 msgr2=0x7fd9d80ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.623+0000 7fd9dee00640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d80ad8d0 0x7fd9d80ac8d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd9c402b950 tx=0x7fd9c40046c0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.623+0000 7fd9de400640 1 -- 172.21.15.67:0/2994569154 reap_dead start 2024-03-09T09:01:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.623+0000 7fd9dee00640 1 -- 172.21.15.67:0/2994569154 shutdown_connections 2024-03-09T09:01:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.623+0000 7fd9dee00640 1 --2- 172.21.15.67:0/2994569154 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9d80ad8d0 0x7fd9d80ac8d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.623+0000 7fd9dee00640 1 -- 172.21.15.67:0/2994569154 >> 172.21.15.67:0/2994569154 conn(0x7fd9d80c8510 msgr2=0x7fd9d81566d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.623+0000 7fd9dee00640 1 -- 172.21.15.67:0/2994569154 shutdown_connections 2024-03-09T09:01:03.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.623+0000 7fd9dee00640 1 -- 172.21.15.67:0/2994569154 wait complete. 2024-03-09T09:01:03.640 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:01:03.640 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:01:03.640 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:01:03.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.838+0000 7f92b0c00640 1 Processor -- start 2024-03-09T09:01:03.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.839+0000 7f92b0c00640 1 -- start start 2024-03-09T09:01:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.840+0000 7f92b0c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac0c5e30 0x7f92ac0c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.840+0000 7f92b0c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac159d60 con 0x7f92ac153220 2024-03-09T09:01:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.841+0000 7f92b0c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac15a040 con 0x7f92ac0c95d0 2024-03-09T09:01:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.841+0000 7f92b0c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0c8060 con 0x7f92ac0c5e30 2024-03-09T09:01:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.841+0000 7f92a3e00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f92ac0c95d0 legacy=0x7f92ac159310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:01:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.841+0000 7f92ab400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f92ac153220 0x7f92ac0c8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53366/0 (socket says 172.21.15.67:53366) 2024-03-09T09:01:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.841+0000 7f92ab400640 1 -- 172.21.15.67:0/766863719 learned_addr learned my addr 172.21.15.67:0/766863719 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:01:03.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.841+0000 7f92abe00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac0c5e30 0x7f92ac0c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.841+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778450472 0 0) 0x7f92ac159d60 con 0x7f92ac153220 2024-03-09T09:01:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.841+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9290003520 con 0x7f92ac153220 2024-03-09T09:01:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 272546848 0 0) 0x7f9290003520 con 0x7f92ac153220 2024-03-09T09:01:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac0c5e30 msgr2=0x7f92ac0c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac0c5e30 0x7f92ac0c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 >> v1:172.21.15.134:6789/0 conn(0x7f92ac0c95d0 legacy=0x7f92ac159310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0c8340 con 0x7f92ac153220 2024-03-09T09:01:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f929c001670 con 0x7f92ac153220 2024-03-09T09:01:03.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 >> v1:172.21.15.67:6789/0 conn(0x7f92ac153220 legacy=0x7f92ac0c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9290003a50 0x7f9290003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 0x7f9290004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac0c5e30 0x7f9290005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0c8060 con 0x7f9290004800 2024-03-09T09:01:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929000bb80 con 0x7f9290003a50 2024-03-09T09:01:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929000bd80 con 0x7f92ac0c5e30 2024-03-09T09:01:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92ab400640 1 -- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9290003a50 msgr2=0x7f9290003e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:01:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.842+0000 7f92ab400640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9290003a50 0x7f9290003e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:03.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.843+0000 7f92a3e00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 0x7f9290004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.843+0000 7f92abe00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac0c5e30 0x7f9290005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.843+0000 7f92a3e00640 1 -- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac0c5e30 msgr2=0x7f9290005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:03.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.843+0000 7f92a3e00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac0c5e30 0x7f9290005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.843+0000 7f92a3e00640 1 -- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9290003a50 msgr2=0x7f9290003e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:03.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.843+0000 7f92a3e00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9290003a50 0x7f9290003e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.843+0000 7f92a3e00640 1 -- 172.21.15.67:0/766863719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9290003520 con 0x7f9290004800 2024-03-09T09:01:03.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.843+0000 7f92abe00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac0c5e30 0x7f9290005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:01:03.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.845+0000 7f92a3e00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 0x7f9290004c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9294009900 tx=0x7f9290005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9d545dfbaa4411 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.845+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294009500 con 0x7f9290004800 2024-03-09T09:01:03.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.845+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929402f070 con 0x7f9290004800 2024-03-09T09:01:03.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.846+0000 7f92aaa00640 1 -- 172.21.15.67:0/766863719 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294003df0 con 0x7f9290004800 2024-03-09T09:01:03.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.846+0000 7f92b0c00640 1 -- 172.21.15.67:0/766863719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 msgr2=0x7f9290004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.846+0000 7f92b0c00640 1 --2- 172.21.15.67:0/766863719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 0x7f9290004c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9294009900 tx=0x7f9290005830 comp rx=0 tx=0).stop 2024-03-09T09:01:03.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.846+0000 7f92abe00640 1 -- 172.21.15.67:0/766863719 reap_dead start 2024-03-09T09:01:03.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.847+0000 7f92b0c00640 1 -- 172.21.15.67:0/766863719 shutdown_connections 2024-03-09T09:01:03.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.847+0000 7f92b0c00640 1 -- 172.21.15.67:0/766863719 >> 172.21.15.67:0/766863719 conn(0x7f92ac0ad060 msgr2=0x7f92ac0ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:03.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.847+0000 7f92b0c00640 1 -- 172.21.15.67:0/766863719 shutdown_connections 2024-03-09T09:01:03.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.847+0000 7f92b0c00640 1 -- 172.21.15.67:0/766863719 wait complete. 2024-03-09T09:01:03.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.849+0000 7f92b0c00640 1 Processor -- start 2024-03-09T09:01:03.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.849+0000 7f92b0c00640 1 -- start start 2024-03-09T09:01:03.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92b0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 0x7f92ac1ef360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92b0c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92ac1595d0 0x7f92ac157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92b0c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157f80 0x7f92ac146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92b0c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac155f20 con 0x7f9290004800 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92b0c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac159a20 con 0x7f92ac1595d0 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92b0c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac1560a0 con 0x7f92ac157f80 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92a3e00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92ac1595d0 msgr2=0x7f92ac157a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92a3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92ac1595d0 0x7f92ac157a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92ab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 0x7f92ac1ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92ab400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 0x7f92ac1ef360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45632/0 (socket says 172.21.15.67:45632) 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92ab400640 1 -- 172.21.15.67:0/2449412534 learned_addr learned my addr 172.21.15.67:0/2449412534 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92abe00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157f80 0x7f92ac146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92ab400640 1 -- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157f80 msgr2=0x7f92ac146220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92ab400640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157f80 0x7f92ac146220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92ab400640 1 -- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92ac1595d0 msgr2=0x7f92ac157a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:01:03.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.851+0000 7f92ab400640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92ac1595d0 0x7f92ac157a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.852+0000 7f92ab400640 1 -- 172.21.15.67:0/2449412534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac1ef8a0 con 0x7f9290004800 2024-03-09T09:01:03.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.852+0000 7f92abe00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157f80 0x7f92ac146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:01:03.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.852+0000 7f92ab400640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 0x7f92ac1ef360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f929c00c750 tx=0x7f929c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.852+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c018070 con 0x7f9290004800 2024-03-09T09:01:03.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.852+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c007500 con 0x7f9290004800 2024-03-09T09:01:03.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.852+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c00ce60 con 0x7f9290004800 2024-03-09T09:01:03.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.853+0000 7f92b0c00640 1 -- 172.21.15.67:0/2449412534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac1586b0 con 0x7f9290004800 2024-03-09T09:01:03.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.853+0000 7f92b0c00640 1 -- 172.21.15.67:0/2449412534 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac157240 con 0x7f9290004800 2024-03-09T09:01:03.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.854+0000 7f92b0c00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac146760 0x7f92ac0affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.854+0000 7f92b0c00640 1 -- 172.21.15.67:0/2449412534 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac158460 con 0x7f92ac146760 2024-03-09T09:01:03.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.855+0000 7f92a3e00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac146760 0x7f92ac0affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.855+0000 7f92a3e00640 1 -- 172.21.15.67:0/2449412534 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92ac04f3c0 con 0x7f92ac146760 2024-03-09T09:01:03.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.856+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f929c01b070 con 0x7f9290004800 2024-03-09T09:01:03.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.856+0000 7f92a3e00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac146760 0x7f92ac0affa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9294003930 tx=0x7f9294005a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.857+0000 7f92a3400640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f928806b500 0x7f928806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.858+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f929c089bd0 con 0x7f9290004800 2024-03-09T09:01:03.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.858+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294006dd0 con 0x7f92ac146760 2024-03-09T09:01:03.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.858+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f929402f040 con 0x7f92ac146760 2024-03-09T09:01:03.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.858+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac146760 msgr2=0x7f92ac0affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.858+0000 7f92a3400640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac146760 0x7f92ac0affa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9294003930 tx=0x7f9294005a80 comp rx=0 tx=0).stop 2024-03-09T09:01:03.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.858+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac146760 msgr2=0x7f92ac0affa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:03.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.858+0000 7f92a3400640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac146760 0x7f92ac0affa0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9294003930 tx=0x7f9294005a80 comp rx=0 tx=0).stop 2024-03-09T09:01:03.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.858+0000 7f92abe00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f928806b500 0x7f928806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.859+0000 7f92abe00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f928806b500 0x7f928806d9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f92ac08d7a0 tx=0x7f92a400d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.866+0000 7f92b0c00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157480 0x7f92ac0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:03.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.866+0000 7f92b0c00640 1 -- 172.21.15.67:0/2449412534 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0c93b0 con 0x7f92ac157480 2024-03-09T09:01:03.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.867+0000 7f92a3e00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157480 0x7f92ac0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:03.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.868+0000 7f92a3e00640 1 -- 172.21.15.67:0/2449412534 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f92ac0b11e0 con 0x7f92ac157480 2024-03-09T09:01:03.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.868+0000 7f92a3e00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157480 0x7f92ac0b04e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9294009900 tx=0x7f9294005a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:03.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.869+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294045830 con 0x7f92ac157480 2024-03-09T09:01:03.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.869+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5032 (secure 0 0 0) 0x7f9294040480 con 0x7f92ac157480 2024-03-09T09:01:03.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.869+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157480 msgr2=0x7f92ac0b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.869+0000 7f92a3400640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157480 0x7f92ac0b04e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9294009900 tx=0x7f9294005a10 comp rx=0 tx=0).stop 2024-03-09T09:01:03.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.869+0000 7f92a3400640 1 -- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157480 msgr2=0x7f92ac0b04e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:03.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.869+0000 7f92a3400640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92ac157480 0x7f92ac0b04e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.870 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:01:03.870 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:01:03.870 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:01:03.870 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:01:03.870 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 70, 2024-03-09T09:01:03.870 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:01:03.870 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:03.872 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:01:03.873 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:03.874 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:03.874 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:01:03.874 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:01:03.874 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:01:03.874 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:01:03.874 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:01:03.874 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:01:03.874 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:01:03.874 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:03.874 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:01:03.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:03.876 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:01:03.877 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:03.879 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:03.880 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:03.881 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:03.882 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:01:03.883 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:01:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.870+0000 7f92b0c00640 1 -- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f928806b500 msgr2=0x7f928806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.870+0000 7f92b0c00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f928806b500 0x7f928806d9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f92ac08d7a0 tx=0x7f92a400d040 comp rx=0 tx=0).stop 2024-03-09T09:01:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.870+0000 7f92b0c00640 1 -- 172.21.15.67:0/2449412534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 msgr2=0x7f92ac1ef360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.870+0000 7f92b0c00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 0x7f92ac1ef360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f929c00c750 tx=0x7f929c00cc10 comp rx=0 tx=0).stop 2024-03-09T09:01:03.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.870+0000 7f92abe00640 1 -- 172.21.15.67:0/2449412534 reap_dead start 2024-03-09T09:01:03.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.870+0000 7f92b0c00640 1 -- 172.21.15.67:0/2449412534 shutdown_connections 2024-03-09T09:01:03.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.870+0000 7f92b0c00640 1 --2- 172.21.15.67:0/2449412534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9290004800 0x7f92ac1ef360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:03.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.870+0000 7f92b0c00640 1 -- 172.21.15.67:0/2449412534 >> 172.21.15.67:0/2449412534 conn(0x7f92ac0ad060 msgr2=0x7f92ac0ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:03.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.870+0000 7f92b0c00640 1 -- 172.21.15.67:0/2449412534 shutdown_connections 2024-03-09T09:01:03.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:03.870+0000 7f92b0c00640 1 -- 172.21.15.67:0/2449412534 wait complete. 2024-03-09T09:01:03.891 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:01:03.891 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:01:12.468 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:01:12.478 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:01:23.892 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:01:23.892 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-03-09T09:01:23.892 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-09T09:01:23.892 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-09T09:01:23.900 INFO:tasks.ceph.mon.b:Started 2024-03-09T09:01:23.900 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:01:23.901 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:01:24.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.113+0000 7f6e3f200640 1 Processor -- start 2024-03-09T09:01:24.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.113+0000 7f6e3f200640 1 -- start start 2024-03-09T09:01:24.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.115+0000 7f6e3f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6e38071190 0x7f6e38157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:24.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.115+0000 7f6e3f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e38153d40 con 0x7f6e380c22a0 2024-03-09T09:01:24.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.115+0000 7f6e3f200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e380c2790 con 0x7f6e380ac730 2024-03-09T09:01:24.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.115+0000 7f6e3f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e38070ee0 con 0x7f6e38071190 2024-03-09T09:01:24.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.115+0000 7f6e3d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6e38071190 0x7f6e38157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:24.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.115+0000 7f6e3d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6e38071190 0x7f6e38157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55276/0 (socket says 172.21.15.67:55276) 2024-03-09T09:01:24.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.115+0000 7f6e3d200640 1 -- 172.21.15.67:0/2061152377 learned_addr learned my addr 172.21.15.67:0/2061152377 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:24.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.115+0000 7f6e37400640 1 --1- 172.21.15.67:0/2061152377 >> v1:172.21.15.67:6789/0 conn(0x7f6e380c22a0 0x7f6e3815c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49638/0 (socket says 172.21.15.67:49638) 2024-03-09T09:01:24.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.116+0000 7f6e3d200640 1 -- 172.21.15.67:0/2061152377 >> v1:172.21.15.134:6789/0 conn(0x7f6e380ac730 legacy=0x7f6e38157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:24.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.116+0000 7f6e3d200640 1 -- 172.21.15.67:0/2061152377 >> v1:172.21.15.67:6789/0 conn(0x7f6e380c22a0 legacy=0x7f6e3815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:24.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.116+0000 7f6e3d200640 1 -- 172.21.15.67:0/2061152377 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e38070920 con 0x7f6e38071190 2024-03-09T09:01:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.117+0000 7f6e3d200640 1 --2- 172.21.15.67:0/2061152377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e38071190 0x7f6e38157d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009e50 tx=0x7f6e3002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae38c0b4e7071338 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.117+0000 7f6e36a00640 1 -- 172.21.15.67:0/2061152377 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3002cb10 con 0x7f6e38071190 2024-03-09T09:01:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.117+0000 7f6e36a00640 1 -- 172.21.15.67:0/2061152377 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3002ccb0 con 0x7f6e38071190 2024-03-09T09:01:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.117+0000 7f6e36a00640 1 -- 172.21.15.67:0/2061152377 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3002ce60 con 0x7f6e38071190 2024-03-09T09:01:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.118+0000 7f6e3f200640 1 -- 172.21.15.67:0/2061152377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e38071190 msgr2=0x7f6e38157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.118+0000 7f6e3f200640 1 --2- 172.21.15.67:0/2061152377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e38071190 0x7f6e38157d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009e50 tx=0x7f6e3002bf70 comp rx=0 tx=0).stop 2024-03-09T09:01:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.118+0000 7f6e3f200640 1 -- 172.21.15.67:0/2061152377 shutdown_connections 2024-03-09T09:01:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.118+0000 7f6e3f200640 1 --2- 172.21.15.67:0/2061152377 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e38071190 0x7f6e38157d60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.118+0000 7f6e3f200640 1 -- 172.21.15.67:0/2061152377 >> 172.21.15.67:0/2061152377 conn(0x7f6e380c7370 msgr2=0x7f6e380c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.118+0000 7f6e3f200640 1 -- 172.21.15.67:0/2061152377 shutdown_connections 2024-03-09T09:01:24.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.118+0000 7f6e3f200640 1 -- 172.21.15.67:0/2061152377 wait complete. 2024-03-09T09:01:24.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.120+0000 7f6e3f200640 1 Processor -- start 2024-03-09T09:01:24.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.121+0000 7f6e3f200640 1 -- start start 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.122+0000 7f6e3f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e38071190 0x7f6e381ecd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.122+0000 7f6e3f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e380ac730 0x7f6e380c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.122+0000 7f6e3f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e380c22a0 0x7f6e380c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.122+0000 7f6e3f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e38151610 con 0x7f6e380ac730 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.122+0000 7f6e3f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e38154d00 con 0x7f6e38071190 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.122+0000 7f6e3f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e38151790 con 0x7f6e380c22a0 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.122+0000 7f6e37400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e380ac730 0x7f6e380c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e380ac730 0x7f6e380c5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55278/0 (socket says 172.21.15.67:55278) 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37400640 1 -- 172.21.15.67:0/3017070537 learned_addr learned my addr 172.21.15.67:0/3017070537 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e38071190 0x7f6e381ecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e3d200640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e380c22a0 0x7f6e380c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37e00640 1 -- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e38071190 msgr2=0x7f6e381ecd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:01:24.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37e00640 1 -- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e38071190 msgr2=0x7f6e381ecd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:01:24.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37e00640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e38071190 0x7f6e381ecd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:24.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37e00640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e38071190 0x7f6e381ecd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:24.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37400640 1 -- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e380c22a0 msgr2=0x7f6e380c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:24.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37400640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e380c22a0 0x7f6e380c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:24.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37400640 1 -- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e38071190 msgr2=0x7f6e381ecd70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:01:24.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37400640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e38071190 0x7f6e381ecd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:24.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37400640 1 -- 172.21.15.67:0/3017070537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e381ed760 con 0x7f6e380ac730 2024-03-09T09:01:24.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.123+0000 7f6e37400640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e380ac730 0x7f6e380c5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400c440 tx=0x7f6e2400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:24.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.124+0000 7f6e2fe00640 1 -- 172.21.15.67:0/3017070537 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240074e0 con 0x7f6e380ac730 2024-03-09T09:01:24.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.124+0000 7f6e2fe00640 1 -- 172.21.15.67:0/3017070537 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e24007680 con 0x7f6e380ac730 2024-03-09T09:01:24.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.124+0000 7f6e2fe00640 1 -- 172.21.15.67:0/3017070537 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24004a20 con 0x7f6e380ac730 2024-03-09T09:01:24.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.124+0000 7f6e3f200640 1 -- 172.21.15.67:0/3017070537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e381dd810 con 0x7f6e380ac730 2024-03-09T09:01:24.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.124+0000 7f6e3f200640 1 -- 172.21.15.67:0/3017070537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e380bc390 con 0x7f6e380ac730 2024-03-09T09:01:24.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.125+0000 7f6e3f200640 1 -- 172.21.15.67:0/3017070537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e38075a00 con 0x7f6e380ac730 2024-03-09T09:01:24.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.126+0000 7f6e2fe00640 1 -- 172.21.15.67:0/3017070537 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6e2401a020 con 0x7f6e380ac730 2024-03-09T09:01:24.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.128+0000 7f6e2fe00640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6e1806b4d0 0x7f6e1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:24.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.128+0000 7f6e2fe00640 1 -- 172.21.15.67:0/3017070537 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6e24089cb0 con 0x7f6e380ac730 2024-03-09T09:01:24.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.128+0000 7f6e37e00640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6e1806b4d0 0x7f6e1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:24.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.129+0000 7f6e37e00640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6e1806b4d0 0x7f6e1806d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6e3814e770 tx=0x7f6e28009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:24.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.133+0000 7f6e2fe00640 1 -- 172.21.15.67:0/3017070537 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6e24058730 con 0x7f6e380ac730 2024-03-09T09:01:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.444+0000 7f6e3f200640 1 -- 172.21.15.67:0/3017070537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6e38152310 con 0x7f6e380ac730 2024-03-09T09:01:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.445+0000 7f6e2fe00640 1 -- 172.21.15.67:0/3017070537 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f6e24058030 con 0x7f6e380ac730 2024-03-09T09:01:24.446 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":70,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:01:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.447+0000 7f6e3f200640 1 -- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6e1806b4d0 msgr2=0x7f6e1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.448+0000 7f6e3f200640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6e1806b4d0 0x7f6e1806d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6e3814e770 tx=0x7f6e28009040 comp rx=0 tx=0).stop 2024-03-09T09:01:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.448+0000 7f6e3f200640 1 -- 172.21.15.67:0/3017070537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e380ac730 msgr2=0x7f6e380c5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.448+0000 7f6e3f200640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e380ac730 0x7f6e380c5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400c440 tx=0x7f6e2400c900 comp rx=0 tx=0).stop 2024-03-09T09:01:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.448+0000 7f6e3f200640 1 -- 172.21.15.67:0/3017070537 shutdown_connections 2024-03-09T09:01:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.448+0000 7f6e3f200640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6e1806b4d0 0x7f6e1806d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.448+0000 7f6e3f200640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6e380c22a0 0x7f6e380c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.448+0000 7f6e3f200640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e380ac730 0x7f6e380c5170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.448+0000 7f6e3f200640 1 --2- 172.21.15.67:0/3017070537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6e38071190 0x7f6e381ecd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.448+0000 7f6e3f200640 1 -- 172.21.15.67:0/3017070537 >> 172.21.15.67:0/3017070537 conn(0x7f6e380c7370 msgr2=0x7f6e380c4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.448+0000 7f6e3f200640 1 -- 172.21.15.67:0/3017070537 shutdown_connections 2024-03-09T09:01:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:24.448+0000 7f6e3f200640 1 -- 172.21.15.67:0/3017070537 wait complete. 2024-03-09T09:01:27.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:01:27.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.694+0000 7f486e200640 1 Processor -- start 2024-03-09T09:01:27.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.694+0000 7f486e200640 1 -- start start 2024-03-09T09:01:27.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.696+0000 7f486e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4868072770 0x7f4868072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:27.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.696+0000 7f486e200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f48680bebc0 0x7f4868156650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:27.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.696+0000 7f486e200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486806cd30 con 0x7f486806b690 2024-03-09T09:01:27.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.696+0000 7f486e200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486806b200 con 0x7f48680bebc0 2024-03-09T09:01:27.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.696+0000 7f486e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48680c5eb0 con 0x7f4868072770 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.696+0000 7f486d800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f486806b690 0x7f486806bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49652/0 (socket says 172.21.15.67:49652) 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.696+0000 7f486d800640 1 -- 172.21.15.67:0/762194036 learned_addr learned my addr 172.21.15.67:0/762194036 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.696+0000 7f486ce00640 1 --2- 172.21.15.67:0/762194036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4868072770 0x7f4868072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f4867e00640 1 --2- 172.21.15.67:0/762194036 >> v2:172.21.15.134:3300/0 conn(0x7f48680bebc0 0x7f4868156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f4867e00640 1 -- 172.21.15.67:0/762194036 >> v2:172.21.15.134:3300/0 conn(0x7f48680bebc0 msgr2=0x7f4868156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f4867e00640 1 -- 172.21.15.67:0/762194036 >> v2:172.21.15.134:3300/0 conn(0x7f48680bebc0 msgr2=0x7f4868156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f4867e00640 1 --2- 172.21.15.67:0/762194036 >> v2:172.21.15.134:3300/0 conn(0x7f48680bebc0 0x7f4868156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f4867e00640 1 --2- 172.21.15.67:0/762194036 >> v2:172.21.15.134:3300/0 conn(0x7f48680bebc0 0x7f4868156650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f4867400640 1 -- 172.21.15.67:0/762194036 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033848322 0 0) 0x7f486806cd30 con 0x7f486806b690 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f4867400640 1 -- 172.21.15.67:0/762194036 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4848003520 con 0x7f486806b690 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f486ce00640 1 -- 172.21.15.67:0/762194036 >> v2:172.21.15.134:3300/0 conn(0x7f48680bebc0 msgr2=0x7f4868156650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f486ce00640 1 --2- 172.21.15.67:0/762194036 >> v2:172.21.15.134:3300/0 conn(0x7f48680bebc0 0x7f4868156650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:27.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f4867400640 1 -- 172.21.15.67:0/762194036 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2989104535 0 0) 0x7f4848003520 con 0x7f486806b690 2024-03-09T09:01:27.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f486ce00640 1 -- 172.21.15.67:0/762194036 >> v1:172.21.15.67:6789/0 conn(0x7f486806b690 legacy=0x7f486806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:27.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.697+0000 7f486ce00640 1 -- 172.21.15.67:0/762194036 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48680c6150 con 0x7f4868072770 2024-03-09T09:01:27.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.699+0000 7f486ce00640 1 --2- 172.21.15.67:0/762194036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4868072770 0x7f4868072b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f48580097a0 tx=0x7f485802b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a4290976767265 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:27.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.699+0000 7f4867400640 1 -- 172.21.15.67:0/762194036 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858037070 con 0x7f4868072770 2024-03-09T09:01:27.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.699+0000 7f4867400640 1 -- 172.21.15.67:0/762194036 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4858030510 con 0x7f4868072770 2024-03-09T09:01:27.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.700+0000 7f4867400640 1 -- 172.21.15.67:0/762194036 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485802be60 con 0x7f4868072770 2024-03-09T09:01:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.700+0000 7f486e200640 1 -- 172.21.15.67:0/762194036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4868072770 msgr2=0x7f4868072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.700+0000 7f486e200640 1 --2- 172.21.15.67:0/762194036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4868072770 0x7f4868072b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f48580097a0 tx=0x7f485802b660 comp rx=0 tx=0).stop 2024-03-09T09:01:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.700+0000 7f486e200640 1 -- 172.21.15.67:0/762194036 shutdown_connections 2024-03-09T09:01:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.700+0000 7f486e200640 1 --2- 172.21.15.67:0/762194036 >> v2:172.21.15.134:3300/0 conn(0x7f48680bebc0 0x7f4868156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.700+0000 7f486e200640 1 --2- 172.21.15.67:0/762194036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4868072770 0x7f4868072b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.700+0000 7f486e200640 1 -- 172.21.15.67:0/762194036 >> 172.21.15.67:0/762194036 conn(0x7f48680c7fb0 msgr2=0x7f48680c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.700+0000 7f486e200640 1 -- 172.21.15.67:0/762194036 shutdown_connections 2024-03-09T09:01:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.700+0000 7f486e200640 1 -- 172.21.15.67:0/762194036 wait complete. 2024-03-09T09:01:27.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.702+0000 7f486e200640 1 Processor -- start 2024-03-09T09:01:27.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.702+0000 7f486e200640 1 -- start start 2024-03-09T09:01:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.703+0000 7f486e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f486806b690 0x7f48680bf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.703+0000 7f486e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4868072770 0x7f48681eedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.703+0000 7f486e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48680bebc0 0x7f486814fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.703+0000 7f486e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48680c9430 con 0x7f48680bebc0 2024-03-09T09:01:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.703+0000 7f486e200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48680c6710 con 0x7f486806b690 2024-03-09T09:01:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.703+0000 7f486e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48680c1f30 con 0x7f4868072770 2024-03-09T09:01:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.704+0000 7f486d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48680bebc0 0x7f486814fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.704+0000 7f486d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48680bebc0 0x7f486814fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55284/0 (socket says 172.21.15.67:55284) 2024-03-09T09:01:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.704+0000 7f486d800640 1 -- 172.21.15.67:0/12406803 learned_addr learned my addr 172.21.15.67:0/12406803 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:27.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.704+0000 7f486ce00640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f486806b690 0x7f48680bf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.704+0000 7f4867e00640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4868072770 0x7f48681eedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.704+0000 7f486d800640 1 -- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4868072770 msgr2=0x7f48681eedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.704+0000 7f486d800640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4868072770 0x7f48681eedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.704+0000 7f486d800640 1 -- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f486806b690 msgr2=0x7f48680bf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.704+0000 7f486d800640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f486806b690 0x7f48680bf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.704+0000 7f486d800640 1 -- 172.21.15.67:0/12406803 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4868159d60 con 0x7f48680bebc0 2024-03-09T09:01:27.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.705+0000 7f486d800640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48680bebc0 0x7f486814fb40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4860002030 tx=0x7f486000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.705+0000 7f4864c00640 1 -- 172.21.15.67:0/12406803 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860012400 con 0x7f48680bebc0 2024-03-09T09:01:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.705+0000 7f4864c00640 1 -- 172.21.15.67:0/12406803 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48600125a0 con 0x7f48680bebc0 2024-03-09T09:01:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.705+0000 7f486e200640 1 -- 172.21.15.67:0/12406803 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48681ef6e0 con 0x7f48680bebc0 2024-03-09T09:01:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.705+0000 7f4864c00640 1 -- 172.21.15.67:0/12406803 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860012a10 con 0x7f48680bebc0 2024-03-09T09:01:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.705+0000 7f486e200640 1 -- 172.21.15.67:0/12406803 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48681ef300 con 0x7f48680bebc0 2024-03-09T09:01:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.707+0000 7f486e200640 1 -- 172.21.15.67:0/12406803 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4868159b50 con 0x7f48680bebc0 2024-03-09T09:01:27.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.707+0000 7f4864c00640 1 -- 172.21.15.67:0/12406803 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4860012bb0 con 0x7f48680bebc0 2024-03-09T09:01:27.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.709+0000 7f4864c00640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484406b4d0 0x7f484406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:27.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.709+0000 7f4864c00640 1 -- 172.21.15.67:0/12406803 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f486008afe0 con 0x7f48680bebc0 2024-03-09T09:01:27.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.709+0000 7f486ce00640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484406b4d0 0x7f484406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.711+0000 7f486ce00640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484406b4d0 0x7f484406d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4858034000 tx=0x7f4858008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:27.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.714+0000 7f4864c00640 1 -- 172.21.15.67:0/12406803 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4860059470 con 0x7f48680bebc0 2024-03-09T09:01:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.991+0000 7f486e200640 1 -- 172.21.15.67:0/12406803 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f486806c980 con 0x7f48680bebc0 2024-03-09T09:01:27.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.992+0000 7f4864c00640 1 -- 172.21.15.67:0/12406803 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f486005d510 con 0x7f48680bebc0 2024-03-09T09:01:27.994 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":70,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:01:27.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 -- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484406b4d0 msgr2=0x7f484406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:27.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484406b4d0 0x7f484406d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4858034000 tx=0x7f4858008030 comp rx=0 tx=0).stop 2024-03-09T09:01:27.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 -- 172.21.15.67:0/12406803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48680bebc0 msgr2=0x7f486814fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48680bebc0 0x7f486814fb40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4860002030 tx=0x7f486000a430 comp rx=0 tx=0).stop 2024-03-09T09:01:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 -- 172.21.15.67:0/12406803 shutdown_connections 2024-03-09T09:01:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484406b4d0 0x7f484406d990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48680bebc0 0x7f486814fb40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4868072770 0x7f48681eedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 --2- 172.21.15.67:0/12406803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f486806b690 0x7f48680bf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 -- 172.21.15.67:0/12406803 >> 172.21.15.67:0/12406803 conn(0x7f48680c7fb0 msgr2=0x7f48681537c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 -- 172.21.15.67:0/12406803 shutdown_connections 2024-03-09T09:01:27.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:27.995+0000 7f486e200640 1 -- 172.21.15.67:0/12406803 wait complete. 2024-03-09T09:01:31.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:01:31.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.260+0000 7f745e600640 1 Processor -- start 2024-03-09T09:01:31.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.260+0000 7f745e600640 1 -- start start 2024-03-09T09:01:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.262+0000 7f745e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7458153300 0x7f74580b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.262+0000 7f745e600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f74580cadb0 0x7f74580b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.262+0000 7f745e600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7458157000 con 0x7f74580c5e40 2024-03-09T09:01:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.262+0000 7f745e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74580ae430 con 0x7f7458153300 2024-03-09T09:01:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.262+0000 7f745e600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74580ae6f0 con 0x7f74580cadb0 2024-03-09T09:01:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.262+0000 7f745d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7458153300 0x7f74580b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.262+0000 7f745d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7458153300 0x7f74580b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58390/0 (socket says 172.21.15.67:58390) 2024-03-09T09:01:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.262+0000 7f745d200640 1 -- 172.21.15.67:0/3369711710 learned_addr learned my addr 172.21.15.67:0/3369711710 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.262+0000 7f7457e00640 1 --1- 172.21.15.67:0/3369711710 >> v1:172.21.15.67:6789/0 conn(0x7f74580c5e40 0x7f7458156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43754/0 (socket says 172.21.15.67:43754) 2024-03-09T09:01:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.263+0000 7f745dc00640 1 --2- 172.21.15.67:0/3369711710 >> v2:172.21.15.134:3300/0 conn(0x7f74580cadb0 0x7f74580b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.263+0000 7f745d200640 1 -- 172.21.15.67:0/3369711710 >> v2:172.21.15.134:3300/0 conn(0x7f74580cadb0 msgr2=0x7f74580b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.263+0000 7f745d200640 1 --2- 172.21.15.67:0/3369711710 >> v2:172.21.15.134:3300/0 conn(0x7f74580cadb0 0x7f74580b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.263+0000 7f745d200640 1 -- 172.21.15.67:0/3369711710 >> v1:172.21.15.67:6789/0 conn(0x7f74580c5e40 legacy=0x7f7458156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.263+0000 7f7457400640 1 -- 172.21.15.67:0/3369711710 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191715956 0 0) 0x7f7458157000 con 0x7f74580c5e40 2024-03-09T09:01:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.263+0000 7f745d200640 1 -- 172.21.15.67:0/3369711710 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7458155b10 con 0x7f7458153300 2024-03-09T09:01:31.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.265+0000 7f745d200640 1 --2- 172.21.15.67:0/3369711710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7458153300 0x7f74580b0000 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f74580adc60 tx=0x7f744c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bcb3444e4aa5e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:31.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.266+0000 7f7457400640 1 -- 172.21.15.67:0/3369711710 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c034070 con 0x7f7458153300 2024-03-09T09:01:31.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.266+0000 7f7457400640 1 -- 172.21.15.67:0/3369711710 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c030860 con 0x7f7458153300 2024-03-09T09:01:31.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.266+0000 7f7457400640 1 -- 172.21.15.67:0/3369711710 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c030c30 con 0x7f7458153300 2024-03-09T09:01:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.266+0000 7f745e600640 1 -- 172.21.15.67:0/3369711710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7458153300 msgr2=0x7f74580b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.266+0000 7f745e600640 1 --2- 172.21.15.67:0/3369711710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7458153300 0x7f74580b0000 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f74580adc60 tx=0x7f744c02b9a0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.266+0000 7f745e600640 1 -- 172.21.15.67:0/3369711710 shutdown_connections 2024-03-09T09:01:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.266+0000 7f745e600640 1 --2- 172.21.15.67:0/3369711710 >> v2:172.21.15.134:3300/0 conn(0x7f74580cadb0 0x7f74580b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.266+0000 7f745e600640 1 --2- 172.21.15.67:0/3369711710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7458153300 0x7f74580b0000 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.266+0000 7f745e600640 1 -- 172.21.15.67:0/3369711710 >> 172.21.15.67:0/3369711710 conn(0x7f74580732d0 msgr2=0x7f74580ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.266+0000 7f745e600640 1 -- 172.21.15.67:0/3369711710 shutdown_connections 2024-03-09T09:01:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.266+0000 7f745e600640 1 -- 172.21.15.67:0/3369711710 wait complete. 2024-03-09T09:01:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.268+0000 7f745e600640 1 Processor -- start 2024-03-09T09:01:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.268+0000 7f745e600640 1 -- start start 2024-03-09T09:01:31.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.269+0000 7f745e600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74580c5e40 0x7f745806b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.269+0000 7f745e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74580cadb0 0x7f74581ebba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.269+0000 7f745e600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7458153300 0x7f745814bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.269+0000 7f745e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580c59e0 con 0x7f74580cadb0 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.269+0000 7f745e600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7458159e90 con 0x7f7458153300 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.269+0000 7f745e600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7458156720 con 0x7f74580c5e40 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.269+0000 7f7457e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74580cadb0 0x7f74581ebba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.270+0000 7f7457e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74580cadb0 0x7f74581ebba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58392/0 (socket says 172.21.15.67:58392) 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.270+0000 7f7457e00640 1 -- 172.21.15.67:0/3480272473 learned_addr learned my addr 172.21.15.67:0/3480272473 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.270+0000 7f745d200640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74580c5e40 0x7f745806b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.270+0000 7f745dc00640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7458153300 0x7f745814bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.270+0000 7f7457e00640 1 -- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74580c5e40 msgr2=0x7f745806b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.270+0000 7f7457e00640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74580c5e40 0x7f745806b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.270+0000 7f7457e00640 1 -- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7458153300 msgr2=0x7f745814bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.270+0000 7f7457e00640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7458153300 0x7f745814bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.270+0000 7f7457e00640 1 -- 172.21.15.67:0/3480272473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74581e9090 con 0x7f74580cadb0 2024-03-09T09:01:31.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.270+0000 7f7457e00640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74580cadb0 0x7f74581ebba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f744400c500 tx=0x7f744400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:31.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.271+0000 7f7454c00640 1 -- 172.21.15.67:0/3480272473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444018070 con 0x7f74580cadb0 2024-03-09T09:01:31.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.271+0000 7f7454c00640 1 -- 172.21.15.67:0/3480272473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7444007580 con 0x7f74580cadb0 2024-03-09T09:01:31.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.271+0000 7f7454c00640 1 -- 172.21.15.67:0/3480272473 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444004a20 con 0x7f74580cadb0 2024-03-09T09:01:31.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.271+0000 7f745e600640 1 -- 172.21.15.67:0/3480272473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74580c6e10 con 0x7f74580cadb0 2024-03-09T09:01:31.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.271+0000 7f745e600640 1 -- 172.21.15.67:0/3480272473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745806b8a0 con 0x7f74580cadb0 2024-03-09T09:01:31.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.272+0000 7f745e600640 1 -- 172.21.15.67:0/3480272473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74581eb470 con 0x7f74580cadb0 2024-03-09T09:01:31.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.273+0000 7f7454c00640 1 -- 172.21.15.67:0/3480272473 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f744401b070 con 0x7f74580cadb0 2024-03-09T09:01:31.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.274+0000 7f7454c00640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f743806b4d0 0x7f743806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.274+0000 7f7454c00640 1 -- 172.21.15.67:0/3480272473 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f7444089890 con 0x7f74580cadb0 2024-03-09T09:01:31.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.274+0000 7f745d200640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f743806b4d0 0x7f743806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.276+0000 7f745d200640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f743806b4d0 0x7f743806d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f744c036000 tx=0x7f744c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:31.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.280+0000 7f7454c00640 1 -- 172.21.15.67:0/3480272473 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7444058390 con 0x7f74580cadb0 2024-03-09T09:01:31.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.556+0000 7f745e600640 1 -- 172.21.15.67:0/3480272473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f745804f3c0 con 0x7f74580cadb0 2024-03-09T09:01:31.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.557+0000 7f7454c00640 1 -- 172.21.15.67:0/3480272473 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7f7444057c90 con 0x7f74580cadb0 2024-03-09T09:01:31.558 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":72,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:01:31.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.559+0000 7f745e600640 1 -- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f743806b4d0 msgr2=0x7f743806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.559+0000 7f745e600640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f743806b4d0 0x7f743806d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f744c036000 tx=0x7f744c008030 comp rx=0 tx=0).stop 2024-03-09T09:01:31.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.559+0000 7f745e600640 1 -- 172.21.15.67:0/3480272473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74580cadb0 msgr2=0x7f74581ebba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.559+0000 7f745e600640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74580cadb0 0x7f74581ebba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f744400c500 tx=0x7f744400c9c0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.559+0000 7f745e600640 1 -- 172.21.15.67:0/3480272473 shutdown_connections 2024-03-09T09:01:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.559+0000 7f745e600640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f743806b4d0 0x7f743806d990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.559+0000 7f745e600640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7458153300 0x7f745814bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.559+0000 7f745e600640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74580cadb0 0x7f74581ebba0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.559+0000 7f745e600640 1 --2- 172.21.15.67:0/3480272473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74580c5e40 0x7f745806b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.559+0000 7f745e600640 1 -- 172.21.15.67:0/3480272473 >> 172.21.15.67:0/3480272473 conn(0x7f74580732d0 msgr2=0x7f745814f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.560+0000 7f745e600640 1 -- 172.21.15.67:0/3480272473 shutdown_connections 2024-03-09T09:01:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.560+0000 7f745e600640 1 -- 172.21.15.67:0/3480272473 wait complete. 2024-03-09T09:01:31.579 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:01:31.579 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:01:31.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:01:31.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.789+0000 7fa9a4a00640 1 Processor -- start 2024-03-09T09:01:31.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.790+0000 7fa9a4a00640 1 -- start start 2024-03-09T09:01:31.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.791+0000 7fa9a4a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a014fd90 0x7fa9a0156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.791+0000 7fa9a4a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa9a00c9380 0x7fa9a00acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.791+0000 7fa9a4a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a0156bc0 con 0x7fa9a00ad440 2024-03-09T09:01:31.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.791+0000 7fa9a4a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a006ba60 con 0x7fa9a00c9380 2024-03-09T09:01:31.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.791+0000 7fa9a4a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00c9890 con 0x7fa9a014fd90 2024-03-09T09:01:31.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.792+0000 7fa99e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a014fd90 0x7fa9a0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.792+0000 7fa99e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a014fd90 0x7fa9a0156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:55640/0 (socket says 172.21.15.67:55640) 2024-03-09T09:01:31.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.792+0000 7fa99e800640 1 -- 172.21.15.67:0/2185280909 learned_addr learned my addr 172.21.15.67:0/2185280909 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:31.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.792+0000 7fa99de00640 1 --2- 172.21.15.67:0/2185280909 >> v2:172.21.15.134:3300/0 conn(0x7fa9a00c9380 0x7fa9a00acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.792+0000 7fa99e800640 1 -- 172.21.15.67:0/2185280909 >> v2:172.21.15.134:3300/0 conn(0x7fa9a00c9380 msgr2=0x7fa9a00acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:31.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.792+0000 7fa99e800640 1 --2- 172.21.15.67:0/2185280909 >> v2:172.21.15.134:3300/0 conn(0x7fa9a00c9380 0x7fa9a00acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.792+0000 7fa99e800640 1 -- 172.21.15.67:0/2185280909 >> v1:172.21.15.134:6789/0 conn(0x7fa9a00ad440 legacy=0x7fa9a006c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:31.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.792+0000 7fa99e800640 1 -- 172.21.15.67:0/2185280909 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a00c5f80 con 0x7fa9a014fd90 2024-03-09T09:01:31.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.794+0000 7fa99e800640 1 --2- 172.21.15.67:0/2185280909 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a014fd90 0x7fa9a0156680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa990009870 tx=0x7fa99002b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22cc16f8fec0b5f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:31.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.795+0000 7fa99d400640 1 -- 172.21.15.67:0/2185280909 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990038070 con 0x7fa9a014fd90 2024-03-09T09:01:31.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.795+0000 7fa99d400640 1 -- 172.21.15.67:0/2185280909 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa990033620 con 0x7fa9a014fd90 2024-03-09T09:01:31.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.795+0000 7fa99d400640 1 -- 172.21.15.67:0/2185280909 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9900339f0 con 0x7fa9a014fd90 2024-03-09T09:01:31.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.795+0000 7fa9a4a00640 1 -- 172.21.15.67:0/2185280909 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a014fd90 msgr2=0x7fa9a0156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.795+0000 7fa9a4a00640 1 --2- 172.21.15.67:0/2185280909 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a014fd90 0x7fa9a0156680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa990009870 tx=0x7fa99002b7d0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.796+0000 7fa9a4a00640 1 -- 172.21.15.67:0/2185280909 shutdown_connections 2024-03-09T09:01:31.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.796+0000 7fa9a4a00640 1 --2- 172.21.15.67:0/2185280909 >> v2:172.21.15.134:3300/0 conn(0x7fa9a00c9380 0x7fa9a00acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.796+0000 7fa9a4a00640 1 --2- 172.21.15.67:0/2185280909 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a014fd90 0x7fa9a0156680 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.796+0000 7fa9a4a00640 1 -- 172.21.15.67:0/2185280909 >> 172.21.15.67:0/2185280909 conn(0x7fa9a00cb480 msgr2=0x7fa9a00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:31.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.796+0000 7fa9a4a00640 1 -- 172.21.15.67:0/2185280909 shutdown_connections 2024-03-09T09:01:31.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.796+0000 7fa9a4a00640 1 -- 172.21.15.67:0/2185280909 wait complete. 2024-03-09T09:01:31.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.799+0000 7fa9a4a00640 1 Processor -- start 2024-03-09T09:01:31.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.799+0000 7fa9a4a00640 1 -- start start 2024-03-09T09:01:31.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.800+0000 7fa9a4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a00ad440 0x7fa9a01efa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.801+0000 7fa9a4a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a00c9380 0x7fa9a00c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.801+0000 7fa9a4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a014fd90 0x7fa9a01eba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.801+0000 7fa9a4a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a006bcb0 con 0x7fa9a014fd90 2024-03-09T09:01:31.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.801+0000 7fa9a4a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00c6520 con 0x7fa9a00ad440 2024-03-09T09:01:31.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.801+0000 7fa9a4a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a006c8e0 con 0x7fa9a00c9380 2024-03-09T09:01:31.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.801+0000 7fa99f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a014fd90 0x7fa9a01eba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.801+0000 7fa99f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a014fd90 0x7fa9a01eba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58404/0 (socket says 172.21.15.67:58404) 2024-03-09T09:01:31.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.801+0000 7fa99f200640 1 -- 172.21.15.67:0/1761772916 learned_addr learned my addr 172.21.15.67:0/1761772916 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:31.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.801+0000 7fa99e800640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a00ad440 0x7fa9a01efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.802+0000 7fa99f200640 1 -- 172.21.15.67:0/1761772916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a00c9380 msgr2=0x7fa9a00c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.802+0000 7fa99f200640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9a00c9380 0x7fa9a00c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.802+0000 7fa99f200640 1 -- 172.21.15.67:0/1761772916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a00ad440 msgr2=0x7fa9a01efa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.802+0000 7fa99f200640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9a00ad440 0x7fa9a01efa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.802+0000 7fa99f200640 1 -- 172.21.15.67:0/1761772916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a01494e0 con 0x7fa9a014fd90 2024-03-09T09:01:31.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.802+0000 7fa99f200640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a014fd90 0x7fa9a01eba00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa99800c7b0 tx=0x7fa99800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:31.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.802+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998004b60 con 0x7fa9a014fd90 2024-03-09T09:01:31.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.802+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa998004d00 con 0x7fa9a014fd90 2024-03-09T09:01:31.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.802+0000 7fa9a4a00640 1 -- 172.21.15.67:0/1761772916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a01eef70 con 0x7fa9a014fd90 2024-03-09T09:01:31.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.802+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99800aa90 con 0x7fa9a014fd90 2024-03-09T09:01:31.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.803+0000 7fa9a4a00640 1 -- 172.21.15.67:0/1761772916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a01559e0 con 0x7fa9a014fd90 2024-03-09T09:01:31.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.804+0000 7fa9a4a00640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0153a70 0x7fa9a0146420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.804+0000 7fa9a4a00640 1 -- 172.21.15.67:0/1761772916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a004f3c0 con 0x7fa9a0153a70 2024-03-09T09:01:31.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.804+0000 7fa99e800640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0153a70 0x7fa9a0146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.805+0000 7fa99e800640 1 -- 172.21.15.67:0/1761772916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9a006d2d0 con 0x7fa9a0153a70 2024-03-09T09:01:31.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.805+0000 7fa99e800640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0153a70 0x7fa9a0146420 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa9a01537d0 tx=0x7fa9900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:31.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.806+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa99800ac30 con 0x7fa9a014fd90 2024-03-09T09:01:31.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.807+0000 7fa996a00640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa98006b4d0 0x7fa98006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.807+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa99808a900 con 0x7fa9a014fd90 2024-03-09T09:01:31.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.807+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990044070 con 0x7fa9a0153a70 2024-03-09T09:01:31.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.807+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa99003d640 con 0x7fa9a0153a70 2024-03-09T09:01:31.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.807+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0153a70 msgr2=0x7fa9a0146420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.807+0000 7fa996a00640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0153a70 0x7fa9a0146420 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa9a01537d0 tx=0x7fa9900046c0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.807+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0153a70 msgr2=0x7fa9a0146420 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:31.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.807+0000 7fa996a00640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0153a70 0x7fa9a0146420 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:31.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.807+0000 7fa99de00640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa98006b4d0 0x7fa98006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.809+0000 7fa99de00640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa98006b4d0 0x7fa98006d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00cb210 tx=0x7fa98c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:31.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.817+0000 7fa9a4a00640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0146960 0x7fa9a0149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:31.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.817+0000 7fa9a4a00640 1 -- 172.21.15.67:0/1761772916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a014c820 con 0x7fa9a0146960 2024-03-09T09:01:31.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.817+0000 7fa99e800640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0146960 0x7fa9a0149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:31.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.818+0000 7fa99e800640 1 -- 172.21.15.67:0/1761772916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa9a006c420 con 0x7fa9a0146960 2024-03-09T09:01:31.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.818+0000 7fa99e800640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0146960 0x7fa9a0149bc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa99002b810 tx=0x7fa990009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:31.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.819+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990038040 con 0x7fa9a0146960 2024-03-09T09:01:31.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.819+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7fa990052430 con 0x7fa9a0146960 2024-03-09T09:01:31.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.819+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0146960 msgr2=0x7fa9a0149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.819+0000 7fa996a00640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0146960 0x7fa9a0149bc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa99002b810 tx=0x7fa990009240 comp rx=0 tx=0).stop 2024-03-09T09:01:31.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.819+0000 7fa996a00640 1 -- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0146960 msgr2=0x7fa9a0149bc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:31.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.819+0000 7fa996a00640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a0146960 0x7fa9a0149bc0 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa99002b810 tx=0x7fa990009240 comp rx=0 tx=0).stop 2024-03-09T09:01:31.821 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:01:31.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:01:31.821 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:01:31.821 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:01:31.821 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 72, 2024-03-09T09:01:31.821 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:01:31.822 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:01:31.822 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:01:31.822 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:01:31.822 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:31.822 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:01:31.822 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:31.822 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:01:31.822 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:01:31.822 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:31.822 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:31.822 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:31.823 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.824 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:01:31.825 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:01:31.825 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:01:31.825 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:01:31.825 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:01:31.825 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:01:31.825 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:31.825 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:31.825 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:01:31.825 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:01:31.825 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:01:31.825 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:31.826 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:31.827 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:31.827 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:31.827 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:31.827 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:31.827 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:31.827 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:01:31.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.827 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.830 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:01:31.830 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:01:31.830 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:31.830 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:31.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:31.830 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:01:31.830 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:31.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:31.832 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:31.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:01:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:01:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:01:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.834 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:31.835 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:31.836 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:01:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:31.837 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:01:31.837 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:01:31.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.820+0000 7fa9a4a00640 1 -- 172.21.15.67:0/1761772916 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa98006b4d0 msgr2=0x7fa98006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.820+0000 7fa9a4a00640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa98006b4d0 0x7fa98006d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00cb210 tx=0x7fa98c008040 comp rx=0 tx=0).stop 2024-03-09T09:01:31.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.820+0000 7fa9a4a00640 1 -- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a014fd90 msgr2=0x7fa9a01eba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:31.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.820+0000 7fa9a4a00640 1 --2- 172.21.15.67:0/1761772916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9a014fd90 0x7fa9a01eba00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa99800c7b0 tx=0x7fa99800cc70 comp rx=0 tx=0).stop 2024-03-09T09:01:31.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.820+0000 7fa99f200640 1 -- 172.21.15.67:0/1761772916 reap_dead start 2024-03-09T09:01:31.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.820+0000 7fa99f200640 1 -- 172.21.15.67:0/1761772916 reap_dead start 2024-03-09T09:01:31.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.820+0000 7fa9a4a00640 1 -- 172.21.15.67:0/1761772916 shutdown_connections 2024-03-09T09:01:31.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.820+0000 7fa9a4a00640 1 -- 172.21.15.67:0/1761772916 >> 172.21.15.67:0/1761772916 conn(0x7fa9a00cb480 msgr2=0x7fa9a00ab060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:31.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.820+0000 7fa9a4a00640 1 -- 172.21.15.67:0/1761772916 shutdown_connections 2024-03-09T09:01:31.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:31.820+0000 7fa9a4a00640 1 -- 172.21.15.67:0/1761772916 wait complete. 2024-03-09T09:01:31.840 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:01:31.840 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:01:31.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:01:32.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.082+0000 7ff603400640 1 Processor -- start 2024-03-09T09:01:32.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.083+0000 7ff603400640 1 -- start start 2024-03-09T09:01:32.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.084+0000 7ff603400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5fc0ac9a0 0x7ff5fc1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.084+0000 7ff603400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff5fc0c2780 0x7ff5fc151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.084+0000 7ff603400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5fc0c8200 con 0x7ff5fc071190 2024-03-09T09:01:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.084+0000 7ff603400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5fc06d590 con 0x7ff5fc0c2780 2024-03-09T09:01:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.084+0000 7ff603400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0c3740 con 0x7ff5fc0ac9a0 2024-03-09T09:01:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.084+0000 7ff601400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff5fc071190 0x7ff5fc154190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43760/0 (socket says 172.21.15.67:43760) 2024-03-09T09:01:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.084+0000 7ff601400640 1 -- 172.21.15.67:0/1468155007 learned_addr learned my addr 172.21.15.67:0/1468155007 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:01:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.085+0000 7ff600a00640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5fc0ac9a0 0x7ff5fc1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.085+0000 7ff5fbe00640 1 --2- 172.21.15.67:0/1468155007 >> v2:172.21.15.67:3300/0 conn(0x7ff5fc0c2780 0x7ff5fc151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.085+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872411710 0 0) 0x7ff5fc0c8200 con 0x7ff5fc071190 2024-03-09T09:01:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.085+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5e4003520 con 0x7ff5fc071190 2024-03-09T09:01:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.085+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2648828100 0 0) 0x7ff5e4003520 con 0x7ff5fc071190 2024-03-09T09:01:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.085+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5fc0ac9a0 msgr2=0x7ff5fc1510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5fc0ac9a0 0x7ff5fc1510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 >> v2:172.21.15.67:3300/0 conn(0x7ff5fc0c2780 msgr2=0x7ff5fc151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 --2- 172.21.15.67:0/1468155007 >> v2:172.21.15.67:3300/0 conn(0x7ff5fc0c2780 0x7ff5fc151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc0c3990 con 0x7ff5fc071190 2024-03-09T09:01:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7ff5f40019f0 con 0x7ff5fc071190 2024-03-09T09:01:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 >> v1:172.21.15.67:6789/0 conn(0x7ff5fc071190 legacy=0x7ff5fc154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5e40039b0 0x7ff5e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5e4004800 0x7ff5e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e40051c0 0x7ff5e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0c3740 con 0x7ff5e40051c0 2024-03-09T09:01:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e400bf00 con 0x7ff5e4004800 2024-03-09T09:01:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e400c100 con 0x7ff5e40039b0 2024-03-09T09:01:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff600a00640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5fc0ac9a0 0x7ff5fc1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:01:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff5fbe00640 1 --2- 172.21.15.67:0/1468155007 >> v2:172.21.15.67:3300/0 conn(0x7ff5fc0c2780 0x7ff5fc151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:01:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.086+0000 7ff601400640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e40051c0 0x7ff5e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.087+0000 7ff600a00640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5e40039b0 0x7ff5e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.087+0000 7ff601400640 1 -- 172.21.15.67:0/1468155007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5e40039b0 msgr2=0x7ff5e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:32.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.087+0000 7ff601400640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5e40039b0 0x7ff5e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.087+0000 7ff601400640 1 -- 172.21.15.67:0/1468155007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5e4004800 msgr2=0x7ff5e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:32.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.087+0000 7ff601400640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5e4004800 0x7ff5e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.087+0000 7ff601400640 1 -- 172.21.15.67:0/1468155007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e4003520 con 0x7ff5e40051c0 2024-03-09T09:01:32.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.089+0000 7ff601400640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e40051c0 0x7ff5e400b9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400a7d0 tx=0x7ff5e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a864a1f82c03a86 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.089+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400a3f0 con 0x7ff5e40051c0 2024-03-09T09:01:32.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.089+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f402ce50 con 0x7ff5e40051c0 2024-03-09T09:01:32.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.089+0000 7ff5fb400640 1 -- 172.21.15.67:0/1468155007 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4034890 con 0x7ff5e40051c0 2024-03-09T09:01:32.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.090+0000 7ff603400640 1 -- 172.21.15.67:0/1468155007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e40051c0 msgr2=0x7ff5e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.090+0000 7ff603400640 1 --2- 172.21.15.67:0/1468155007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5e40051c0 0x7ff5e400b9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400a7d0 tx=0x7ff5e4005830 comp rx=0 tx=0).stop 2024-03-09T09:01:32.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.090+0000 7ff601400640 1 -- 172.21.15.67:0/1468155007 reap_dead start 2024-03-09T09:01:32.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.090+0000 7ff603400640 1 -- 172.21.15.67:0/1468155007 shutdown_connections 2024-03-09T09:01:32.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.090+0000 7ff603400640 1 -- 172.21.15.67:0/1468155007 >> 172.21.15.67:0/1468155007 conn(0x7ff5fc0c7370 msgr2=0x7ff5fc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:32.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.090+0000 7ff603400640 1 -- 172.21.15.67:0/1468155007 shutdown_connections 2024-03-09T09:01:32.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.090+0000 7ff603400640 1 -- 172.21.15.67:0/1468155007 wait complete. 2024-03-09T09:01:32.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.092+0000 7ff603400640 1 Processor -- start 2024-03-09T09:01:32.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.092+0000 7ff603400640 1 -- start start 2024-03-09T09:01:32.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.093+0000 7ff603400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5e40051c0 0x7ff5fc146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.093+0000 7ff603400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc146130 0x7ff5fc1eed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.093+0000 7ff603400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5fc1ef240 0x7ff5fc0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.093+0000 7ff603400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0acd80 con 0x7ff5fc1ef240 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.093+0000 7ff603400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc070c00 con 0x7ff5fc146130 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.093+0000 7ff603400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0acf00 con 0x7ff5e40051c0 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.094+0000 7ff601400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5fc1ef240 0x7ff5fc0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.094+0000 7ff601400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5fc1ef240 0x7ff5fc0c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58464/0 (socket says 172.21.15.67:58464) 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.094+0000 7ff601400640 1 -- 172.21.15.67:0/3520727988 learned_addr learned my addr 172.21.15.67:0/3520727988 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.094+0000 7ff600a00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5e40051c0 0x7ff5fc146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.094+0000 7ff5fbe00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc146130 0x7ff5fc1eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.094+0000 7ff601400640 1 -- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5e40051c0 msgr2=0x7ff5fc146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.094+0000 7ff601400640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5e40051c0 0x7ff5fc146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.094+0000 7ff601400640 1 -- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc146130 msgr2=0x7ff5fc1eed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.094+0000 7ff601400640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc146130 0x7ff5fc1eed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.094+0000 7ff601400640 1 -- 172.21.15.67:0/3520727988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc0c3ea0 con 0x7ff5fc1ef240 2024-03-09T09:01:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.094+0000 7ff601400640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5fc1ef240 0x7ff5fc0c62f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff5f402cbe0 tx=0x7ff5f4001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.095+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400a3f0 con 0x7ff5fc1ef240 2024-03-09T09:01:32.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.095+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f4044070 con 0x7ff5fc1ef240 2024-03-09T09:01:32.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.095+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4001f40 con 0x7ff5fc1ef240 2024-03-09T09:01:32.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.095+0000 7ff603400640 1 -- 172.21.15.67:0/3520727988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5fc147280 con 0x7ff5fc1ef240 2024-03-09T09:01:32.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.095+0000 7ff603400640 1 -- 172.21.15.67:0/3520727988 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5fc0c4140 con 0x7ff5fc1ef240 2024-03-09T09:01:32.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.096+0000 7ff603400640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4340 0x7ff5fc152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.096+0000 7ff603400640 1 -- 172.21.15.67:0/3520727988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc075a00 con 0x7ff5fc0c4340 2024-03-09T09:01:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.097+0000 7ff600a00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4340 0x7ff5fc152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.097+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff5f4034a60 con 0x7ff5fc1ef240 2024-03-09T09:01:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.098+0000 7ff600a00640 1 -- 172.21.15.67:0/3520727988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5fc0c8080 con 0x7ff5fc0c4340 2024-03-09T09:01:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.098+0000 7ff5f8c00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff5dc06b4d0 0x7ff5dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.098+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff5f40acc30 con 0x7ff5fc1ef240 2024-03-09T09:01:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.098+0000 7ff600a00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4340 0x7ff5fc152cc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0ad660 tx=0x7ff5f0013d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.099+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f001c070 con 0x7ff5fc0c4340 2024-03-09T09:01:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.099+0000 7ff5fbe00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff5dc06b4d0 0x7ff5dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.100+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff5f00119f0 con 0x7ff5fc0c4340 2024-03-09T09:01:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.100+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4340 msgr2=0x7ff5fc152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.100+0000 7ff5f8c00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4340 0x7ff5fc152cc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0ad660 tx=0x7ff5f0013d10 comp rx=0 tx=0).stop 2024-03-09T09:01:32.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.100+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4340 msgr2=0x7ff5fc152cc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:32.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.100+0000 7ff5f8c00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4340 0x7ff5fc152cc0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0ad660 tx=0x7ff5f0013d10 comp rx=0 tx=0).stop 2024-03-09T09:01:32.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.100+0000 7ff5fbe00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff5dc06b4d0 0x7ff5dc06d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8004790 tx=0x7ff5e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.114+0000 7ff603400640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4940 0x7ff5fc153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.115+0000 7ff603400640 1 -- 172.21.15.67:0/3520727988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc153740 con 0x7ff5fc0c4940 2024-03-09T09:01:32.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.115+0000 7ff600a00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4940 0x7ff5fc153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.116+0000 7ff600a00640 1 -- 172.21.15.67:0/3520727988 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff5fc0c7c20 con 0x7ff5fc0c4940 2024-03-09T09:01:32.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.117+0000 7ff600a00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4940 0x7ff5fc153200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff5f001e000 tx=0x7ff5f0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.117+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0006460 con 0x7ff5fc0c4940 2024-03-09T09:01:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.118+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4533 (secure 0 0 0) 0x7ff5f0006460 con 0x7ff5fc0c4940 2024-03-09T09:01:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.118+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4940 msgr2=0x7ff5fc153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.118+0000 7ff5f8c00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4940 0x7ff5fc153200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff5f001e000 tx=0x7ff5f0004730 comp rx=0 tx=0).stop 2024-03-09T09:01:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.118+0000 7ff5f8c00640 1 -- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4940 msgr2=0x7ff5fc153200 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.118+0000 7ff5f8c00640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5fc0c4940 0x7ff5fc153200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.120 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:01:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:01:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:01:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:01:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 72, 2024-03-09T09:01:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:01:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:01:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:01:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:01:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:01:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:01:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:01:32.131 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:01:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.118+0000 7ff603400640 1 -- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff5dc06b4d0 msgr2=0x7ff5dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.118+0000 7ff603400640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff5dc06b4d0 0x7ff5dc06d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8004790 tx=0x7ff5e8008040 comp rx=0 tx=0).stop 2024-03-09T09:01:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.119+0000 7ff603400640 1 -- 172.21.15.67:0/3520727988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5fc1ef240 msgr2=0x7ff5fc0c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.119+0000 7ff603400640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5fc1ef240 0x7ff5fc0c62f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff5f402cbe0 tx=0x7ff5f4001cf0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.119+0000 7ff601400640 1 -- 172.21.15.67:0/3520727988 reap_dead start 2024-03-09T09:01:32.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.119+0000 7ff603400640 1 -- 172.21.15.67:0/3520727988 shutdown_connections 2024-03-09T09:01:32.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.119+0000 7ff603400640 1 --2- 172.21.15.67:0/3520727988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5fc1ef240 0x7ff5fc0c62f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.119+0000 7ff603400640 1 -- 172.21.15.67:0/3520727988 >> 172.21.15.67:0/3520727988 conn(0x7ff5fc0c7370 msgr2=0x7ff5fc14da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:32.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.119+0000 7ff603400640 1 -- 172.21.15.67:0/3520727988 shutdown_connections 2024-03-09T09:01:32.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.119+0000 7ff603400640 1 -- 172.21.15.67:0/3520727988 wait complete. 2024-03-09T09:01:32.137 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:01:32.137 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:01:32.137 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:01:32.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.395+0000 7f5911c00640 1 Processor -- start 2024-03-09T09:01:32.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.396+0000 7f5911c00640 1 -- start start 2024-03-09T09:01:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.397+0000 7f5911c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f590c06c1d0 0x7f590c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.397+0000 7f5911c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c0c9890 con 0x7f590c0c9380 2024-03-09T09:01:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.397+0000 7f5911c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c0c5f80 con 0x7f590c14fd90 2024-03-09T09:01:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.397+0000 7f5911c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590c06ba60 con 0x7f590c06c1d0 2024-03-09T09:01:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.397+0000 7f590b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f590c0c9380 0x7f590c0acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43770/0 (socket says 172.21.15.67:43770) 2024-03-09T09:01:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.397+0000 7f590b400640 1 -- 172.21.15.67:0/2087065089 learned_addr learned my addr 172.21.15.67:0/2087065089 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:01:32.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.398+0000 7f5911200640 1 --2- 172.21.15.67:0/2087065089 >> v2:172.21.15.67:3300/0 conn(0x7f590c06c1d0 0x7f590c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.398+0000 7f590aa00640 1 -- 172.21.15.67:0/2087065089 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792778592 0 0) 0x7f590c0c9890 con 0x7f590c0c9380 2024-03-09T09:01:32.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.398+0000 7f590aa00640 1 -- 172.21.15.67:0/2087065089 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58f0003520 con 0x7f590c0c9380 2024-03-09T09:01:32.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.398+0000 7f5911200640 1 -- 172.21.15.67:0/2087065089 >> v1:172.21.15.134:6789/0 conn(0x7f590c14fd90 legacy=0x7f590c156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:32.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.398+0000 7f5911200640 1 -- 172.21.15.67:0/2087065089 >> v1:172.21.15.67:6789/0 conn(0x7f590c0c9380 legacy=0x7f590c0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.398+0000 7f5911200640 1 -- 172.21.15.67:0/2087065089 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c0c6240 con 0x7f590c06c1d0 2024-03-09T09:01:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.400+0000 7f5911200640 1 --2- 172.21.15.67:0/2087065089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c06c1d0 0x7f590c06c5f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f590c06bbe0 tx=0x7f590402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95331285c3405152 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.400+0000 7f590aa00640 1 -- 172.21.15.67:0/2087065089 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904038070 con 0x7f590c06c1d0 2024-03-09T09:01:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.400+0000 7f590aa00640 1 -- 172.21.15.67:0/2087065089 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5904031700 con 0x7f590c06c1d0 2024-03-09T09:01:32.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.400+0000 7f590aa00640 1 -- 172.21.15.67:0/2087065089 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904031ad0 con 0x7f590c06c1d0 2024-03-09T09:01:32.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.401+0000 7f5911c00640 1 -- 172.21.15.67:0/2087065089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c06c1d0 msgr2=0x7f590c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.401+0000 7f5911c00640 1 --2- 172.21.15.67:0/2087065089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c06c1d0 0x7f590c06c5f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f590c06bbe0 tx=0x7f590402b7a0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.401+0000 7f5911c00640 1 -- 172.21.15.67:0/2087065089 shutdown_connections 2024-03-09T09:01:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.401+0000 7f5911c00640 1 --2- 172.21.15.67:0/2087065089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c06c1d0 0x7f590c06c5f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.401+0000 7f5911c00640 1 -- 172.21.15.67:0/2087065089 >> 172.21.15.67:0/2087065089 conn(0x7f590c0cb480 msgr2=0x7f590c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.401+0000 7f5911c00640 1 -- 172.21.15.67:0/2087065089 shutdown_connections 2024-03-09T09:01:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.401+0000 7f5911c00640 1 -- 172.21.15.67:0/2087065089 wait complete. 2024-03-09T09:01:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.404+0000 7f5911c00640 1 Processor -- start 2024-03-09T09:01:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.404+0000 7f5911c00640 1 -- start start 2024-03-09T09:01:32.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f5911c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c06c1d0 0x7f590c150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f5911c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0c9380 0x7f590c1ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f5911c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c14fd90 0x7f590c1ed020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f5911c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c0ad610 con 0x7f590c06c1d0 2024-03-09T09:01:32.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f5911c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c06bcb0 con 0x7f590c0c9380 2024-03-09T09:01:32.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f5911c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c0ad790 con 0x7f590c14fd90 2024-03-09T09:01:32.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f590be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c06c1d0 0x7f590c150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f590be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c06c1d0 0x7f590c150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58480/0 (socket says 172.21.15.67:58480) 2024-03-09T09:01:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f590be00640 1 -- 172.21.15.67:0/3086741860 learned_addr learned my addr 172.21.15.67:0/3086741860 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f590b400640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0c9380 0x7f590c1ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f590be00640 1 -- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c14fd90 msgr2=0x7f590c1ed020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.405+0000 7f590be00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c14fd90 0x7f590c1ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.406+0000 7f590be00640 1 -- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0c9380 msgr2=0x7f590c1ecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.406+0000 7f590be00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0c9380 0x7f590c1ecae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.406+0000 7f590be00640 1 -- 172.21.15.67:0/3086741860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c152810 con 0x7f590c06c1d0 2024-03-09T09:01:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.406+0000 7f590b400640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0c9380 0x7f590c1ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:01:32.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.406+0000 7f590be00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c06c1d0 0x7f590c150a40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00c7f0 tx=0x7f58fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.406+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc016020 con 0x7f590c06c1d0 2024-03-09T09:01:32.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.406+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc00aa50 con 0x7f590c06c1d0 2024-03-09T09:01:32.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.406+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc00ae40 con 0x7f590c06c1d0 2024-03-09T09:01:32.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.406+0000 7f5911c00640 1 -- 172.21.15.67:0/3086741860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c152af0 con 0x7f590c06c1d0 2024-03-09T09:01:32.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.407+0000 7f5911c00640 1 -- 172.21.15.67:0/3086741860 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c1530b0 con 0x7f590c06c1d0 2024-03-09T09:01:32.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.408+0000 7f5911c00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0acd00 0x7f590c1ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.408+0000 7f5911c00640 1 -- 172.21.15.67:0/3086741860 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c1ec720 con 0x7f590c0acd00 2024-03-09T09:01:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.408+0000 7f590b400640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0acd00 0x7f590c1ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.408+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f58fc019020 con 0x7f590c06c1d0 2024-03-09T09:01:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.409+0000 7f590b400640 1 -- 172.21.15.67:0/3086741860 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f590c04f3c0 con 0x7f590c0acd00 2024-03-09T09:01:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.409+0000 7f5903e00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58e806b4d0 0x7f58e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.409+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f58fc08a6c0 con 0x7f590c06c1d0 2024-03-09T09:01:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.410+0000 7f590b400640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0acd00 0x7f590c1ee7c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f590c0c56f0 tx=0x7f58f400d950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.410+0000 7f5911200640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58e806b4d0 0x7f58e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.410+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4005cc0 con 0x7f590c0acd00 2024-03-09T09:01:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.412+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f58f4017070 con 0x7f590c0acd00 2024-03-09T09:01:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.412+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0acd00 msgr2=0x7f590c1ee7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.412+0000 7f5903e00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0acd00 0x7f590c1ee7c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f590c0c56f0 tx=0x7f58f400d950 comp rx=0 tx=0).stop 2024-03-09T09:01:32.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.412+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0acd00 msgr2=0x7f590c1ee7c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:32.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.412+0000 7f5903e00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0acd00 0x7f590c1ee7c0 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f590c0c56f0 tx=0x7f58f400d950 comp rx=0 tx=0).stop 2024-03-09T09:01:32.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.412+0000 7f5911200640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58e806b4d0 0x7f58e806d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5904035000 tx=0x7f5904030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.420+0000 7f5911c00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c1ed560 0x7f590c1eed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.420+0000 7f5911c00640 1 -- 172.21.15.67:0/3086741860 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c06d2d0 con 0x7f590c1ed560 2024-03-09T09:01:32.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.420+0000 7f590b400640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c1ed560 0x7f590c1eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.421+0000 7f590b400640 1 -- 172.21.15.67:0/3086741860 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f590c150f80 con 0x7f590c1ed560 2024-03-09T09:01:32.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.421+0000 7f590b400640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c1ed560 0x7f590c1eed00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f590c14f740 tx=0x7f58f4000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.422+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4007410 con 0x7f590c1ed560 2024-03-09T09:01:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.422+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5043 (secure 0 0 0) 0x7f58f4007410 con 0x7f590c1ed560 2024-03-09T09:01:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.422+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c1ed560 msgr2=0x7f590c1eed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.422+0000 7f5903e00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c1ed560 0x7f590c1eed00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f590c14f740 tx=0x7f58f4000ed0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5903e00640 1 -- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c1ed560 msgr2=0x7f590c1eed00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5903e00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c1ed560 0x7f590c1eed00 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f590c14f740 tx=0x7f58f4000ed0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.425 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:01:32.425 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:01:32.425 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:01:32.425 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:01:32.425 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 72, 2024-03-09T09:01:32.425 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:01:32.425 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:01:32.425 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:01:32.425 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:32.426 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:32.427 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:01:32.428 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:01:32.429 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:01:32.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.431 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.432 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:32.433 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:32.434 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:01:32.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:01:32.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:32.437 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:32.438 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:32.438 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:32.438 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:32.438 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:32.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:32.438 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:01:32.438 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:01:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5911c00640 1 -- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58e806b4d0 msgr2=0x7f58e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5911c00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58e806b4d0 0x7f58e806d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5904035000 tx=0x7f5904030000 comp rx=0 tx=0).stop 2024-03-09T09:01:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5911c00640 1 -- 172.21.15.67:0/3086741860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c06c1d0 msgr2=0x7f590c150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5911c00640 1 --2- 172.21.15.67:0/3086741860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c06c1d0 0x7f590c150a40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00c7f0 tx=0x7f58fc00ccb0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5911200640 1 -- 172.21.15.67:0/3086741860 reap_dead start 2024-03-09T09:01:32.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5911200640 1 -- 172.21.15.67:0/3086741860 reap_dead start 2024-03-09T09:01:32.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5911c00640 1 -- 172.21.15.67:0/3086741860 shutdown_connections 2024-03-09T09:01:32.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5911c00640 1 -- 172.21.15.67:0/3086741860 >> 172.21.15.67:0/3086741860 conn(0x7f590c0cb480 msgr2=0x7f590c0ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:32.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5911c00640 1 -- 172.21.15.67:0/3086741860 shutdown_connections 2024-03-09T09:01:32.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.423+0000 7f5911c00640 1 -- 172.21.15.67:0/3086741860 wait complete. 2024-03-09T09:01:32.442 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:01:32.442 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:01:32.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:01:32.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.633+0000 7fd7d4200640 1 Processor -- start 2024-03-09T09:01:32.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.633+0000 7fd7d4200640 1 -- start start 2024-03-09T09:01:32.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.634+0000 7fd7d4200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd7cc0ac730 0x7fd7cc071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.634+0000 7fd7d4200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd7cc157d60 0x7fd7cc0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.634+0000 7fd7d4200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc1581a0 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.634+0000 7fd7d4200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc070ee0 con 0x7fd7cc157d60 2024-03-09T09:01:32.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.634+0000 7fd7d4200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc154d00 con 0x7fd7cc0ac730 2024-03-09T09:01:32.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.634+0000 7fd7d0e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd7cc0c22a0 0x7fd7cc157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43772/0 (socket says 172.21.15.67:43772) 2024-03-09T09:01:32.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.634+0000 7fd7d0e00640 1 -- 172.21.15.67:0/762322875 learned_addr learned my addr 172.21.15.67:0/762322875 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:01:32.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.634+0000 7fd7d2200640 1 --2- 172.21.15.67:0/762322875 >> v2:172.21.15.67:3300/0 conn(0x7fd7cc157d60 0x7fd7cc0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.635+0000 7fd7d1800640 1 --2- 172.21.15.67:0/762322875 >> v2:172.21.15.134:3300/0 conn(0x7fd7cc0ac730 0x7fd7cc071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.635+0000 7fd7cbe00640 1 -- 172.21.15.67:0/762322875 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144624908 0 0) 0x7fd7cc1581a0 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.635+0000 7fd7cbe00640 1 -- 172.21.15.67:0/762322875 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7b4003520 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.635+0000 7fd7d2200640 1 -- 172.21.15.67:0/762322875 >> v2:172.21.15.134:3300/0 conn(0x7fd7cc0ac730 msgr2=0x7fd7cc071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:32.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.635+0000 7fd7d2200640 1 --2- 172.21.15.67:0/762322875 >> v2:172.21.15.134:3300/0 conn(0x7fd7cc0ac730 0x7fd7cc071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.635+0000 7fd7d2200640 1 -- 172.21.15.67:0/762322875 >> v1:172.21.15.67:6789/0 conn(0x7fd7cc0c22a0 legacy=0x7fd7cc157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.635+0000 7fd7d2200640 1 -- 172.21.15.67:0/762322875 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc153d40 con 0x7fd7cc157d60 2024-03-09T09:01:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.636+0000 7fd7d2200640 1 --2- 172.21.15.67:0/762322875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7cc157d60 0x7fd7cc0acb10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0ad380 tx=0x7fd7c402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d189016dad58d992 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.636+0000 7fd7cbe00640 1 -- 172.21.15.67:0/762322875 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4039070 con 0x7fd7cc157d60 2024-03-09T09:01:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.637+0000 7fd7cbe00640 1 -- 172.21.15.67:0/762322875 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c402cbf0 con 0x7fd7cc157d60 2024-03-09T09:01:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.637+0000 7fd7cbe00640 1 -- 172.21.15.67:0/762322875 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4034440 con 0x7fd7cc157d60 2024-03-09T09:01:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.637+0000 7fd7d4200640 1 -- 172.21.15.67:0/762322875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7cc157d60 msgr2=0x7fd7cc0acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.637+0000 7fd7d4200640 1 --2- 172.21.15.67:0/762322875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7cc157d60 0x7fd7cc0acb10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0ad380 tx=0x7fd7c402be40 comp rx=0 tx=0).stop 2024-03-09T09:01:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.637+0000 7fd7d4200640 1 -- 172.21.15.67:0/762322875 shutdown_connections 2024-03-09T09:01:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.637+0000 7fd7d4200640 1 --2- 172.21.15.67:0/762322875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7cc157d60 0x7fd7cc0acb10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.637+0000 7fd7d4200640 1 --2- 172.21.15.67:0/762322875 >> v2:172.21.15.134:3300/0 conn(0x7fd7cc0ac730 0x7fd7cc071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.637+0000 7fd7d4200640 1 -- 172.21.15.67:0/762322875 >> 172.21.15.67:0/762322875 conn(0x7fd7cc0c7370 msgr2=0x7fd7cc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.637+0000 7fd7d4200640 1 -- 172.21.15.67:0/762322875 shutdown_connections 2024-03-09T09:01:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.637+0000 7fd7d4200640 1 -- 172.21.15.67:0/762322875 wait complete. 2024-03-09T09:01:32.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.639+0000 7fd7d4200640 1 Processor -- start 2024-03-09T09:01:32.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.639+0000 7fd7d4200640 1 -- start start 2024-03-09T09:01:32.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.640+0000 7fd7d4200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7cc0ac730 0x7fd7cc0c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.640+0000 7fd7d4200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7cc0c22a0 0x7fd7cc14cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.640+0000 7fd7d4200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7cc157d60 0x7fd7cc14d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.640+0000 7fd7d4200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc070c00 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.640+0000 7fd7d4200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc070920 con 0x7fd7cc157d60 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.640+0000 7fd7d4200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc151610 con 0x7fd7cc0ac730 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.640+0000 7fd7d0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7cc0c22a0 0x7fd7cc14cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.640+0000 7fd7d0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7cc0c22a0 0x7fd7cc14cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58506/0 (socket says 172.21.15.67:58506) 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.640+0000 7fd7d0e00640 1 -- 172.21.15.67:0/2677781571 learned_addr learned my addr 172.21.15.67:0/2677781571 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.641+0000 7fd7d2200640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7cc157d60 0x7fd7cc14d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.641+0000 7fd7d1800640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7cc0ac730 0x7fd7cc0c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.641+0000 7fd7d0e00640 1 -- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7cc0ac730 msgr2=0x7fd7cc0c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.641+0000 7fd7d0e00640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7cc0ac730 0x7fd7cc0c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.641+0000 7fd7d0e00640 1 -- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7cc157d60 msgr2=0x7fd7cc14d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.641+0000 7fd7d0e00640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7cc157d60 0x7fd7cc14d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.641+0000 7fd7d0e00640 1 -- 172.21.15.67:0/2677781571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc1efb70 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.641+0000 7fd7d2200640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7cc157d60 0x7fd7cc14d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:01:32.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.641+0000 7fd7d0e00640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7cc0c22a0 0x7fd7cc14cac0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00c400 tx=0x7fd7bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.642+0000 7fd7c9600640 1 -- 172.21.15.67:0/2677781571 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc017070 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.642+0000 7fd7c9600640 1 -- 172.21.15.67:0/2677781571 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc005d00 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.642+0000 7fd7c9600640 1 -- 172.21.15.67:0/2677781571 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc007310 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.642+0000 7fd7d4200640 1 -- 172.21.15.67:0/2677781571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc1efe50 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.642+0000 7fd7d4200640 1 -- 172.21.15.67:0/2677781571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc1dd870 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.643+0000 7fd7d4200640 1 -- 172.21.15.67:0/2677781571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc1dd3c0 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.644+0000 7fd7c9600640 1 -- 172.21.15.67:0/2677781571 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd7bc01a070 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.645+0000 7fd7c9600640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd7b006b4d0 0x7fd7b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:32.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.645+0000 7fd7c9600640 1 -- 172.21.15.67:0/2677781571 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd7bc0899d0 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.646+0000 7fd7d1800640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd7b006b4d0 0x7fd7b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:32.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.647+0000 7fd7d1800640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd7b006b4d0 0x7fd7b006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0006fd0 tx=0x7fd7c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:32.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.650+0000 7fd7c9600640 1 -- 172.21.15.67:0/2677781571 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd7bc0584f0 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.927+0000 7fd7d4200640 1 -- 172.21.15.67:0/2677781571 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fd7cc1f0020 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.930+0000 7fd7c9600640 1 -- 172.21.15.67:0/2677781571 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fd7bc057df0 con 0x7fd7cc0c22a0 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 -- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd7b006b4d0 msgr2=0x7fd7b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd7b006b4d0 0x7fd7b006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0006fd0 tx=0x7fd7c0008040 comp rx=0 tx=0).stop 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 -- 172.21.15.67:0/2677781571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7cc0c22a0 msgr2=0x7fd7cc14cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7cc0c22a0 0x7fd7cc14cac0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00c400 tx=0x7fd7bc00c8c0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 -- 172.21.15.67:0/2677781571 shutdown_connections 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd7b006b4d0 0x7fd7b006d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7cc157d60 0x7fd7cc14d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7cc0c22a0 0x7fd7cc14cac0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 --2- 172.21.15.67:0/2677781571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7cc0ac730 0x7fd7cc0c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 -- 172.21.15.67:0/2677781571 >> 172.21.15.67:0/2677781571 conn(0x7fd7cc0c7370 msgr2=0x7fd7cc074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 -- 172.21.15.67:0/2677781571 shutdown_connections 2024-03-09T09:01:32.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:32.933+0000 7fd7d4200640 1 -- 172.21.15.67:0/2677781571 wait complete. 2024-03-09T09:01:32.960 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:01:42.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:01:42.628 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:01:42.960 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:01:42.961 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:01:43.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.204+0000 7f1575000640 1 Processor -- start 2024-03-09T09:01:43.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.204+0000 7f1575000640 1 -- start start 2024-03-09T09:01:43.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.206+0000 7f1575000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f157006d650 0x7f1570163490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:43.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.206+0000 7f1575000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f15700c2e40 0x7f15700c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:43.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.206+0000 7f1575000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15700c1da0 con 0x7f157008c730 2024-03-09T09:01:43.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.206+0000 7f1575000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15700c2080 con 0x7f15700c2e40 2024-03-09T09:01:43.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.206+0000 7f1575000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15700c2360 con 0x7f157006d650 2024-03-09T09:01:43.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.207+0000 7f156ea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f157008c730 0x7f157008bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37816/0 (socket says 172.21.15.67:37816) 2024-03-09T09:01:43.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.207+0000 7f156ea00640 1 -- 172.21.15.67:0/1297756924 learned_addr learned my addr 172.21.15.67:0/1297756924 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:01:43.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.207+0000 7f156fe00640 1 --2- 172.21.15.67:0/1297756924 >> v2:172.21.15.67:3300/0 conn(0x7f15700c2e40 0x7f15700c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:43.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.207+0000 7f156f400640 1 --2- 172.21.15.67:0/1297756924 >> v2:172.21.15.134:3300/0 conn(0x7f157006d650 0x7f1570163490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:43.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.207+0000 7f156e000640 1 -- 172.21.15.67:0/1297756924 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867374280 0 0) 0x7f15700c1da0 con 0x7f157008c730 2024-03-09T09:01:43.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.416+0000 7f156e000640 1 -- 172.21.15.67:0/1297756924 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1554003500 con 0x7f157008c730 2024-03-09T09:01:43.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.416+0000 7f156fe00640 1 -- 172.21.15.67:0/1297756924 >> v2:172.21.15.134:3300/0 conn(0x7f157006d650 msgr2=0x7f1570163490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:43.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.416+0000 7f156fe00640 1 --2- 172.21.15.67:0/1297756924 >> v2:172.21.15.134:3300/0 conn(0x7f157006d650 0x7f1570163490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:43.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.416+0000 7f156fe00640 1 -- 172.21.15.67:0/1297756924 >> v1:172.21.15.67:6789/0 conn(0x7f157008c730 legacy=0x7f157008bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:43.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.416+0000 7f156fe00640 1 -- 172.21.15.67:0/1297756924 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15700c2640 con 0x7f15700c2e40 2024-03-09T09:01:43.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.419+0000 7f156fe00640 1 --2- 172.21.15.67:0/1297756924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700c2e40 0x7f15700c32f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f157008cc30 tx=0x7f15680346c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175af254ca4be52e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:43.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.419+0000 7f156e000640 1 -- 172.21.15.67:0/1297756924 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568040070 con 0x7f15700c2e40 2024-03-09T09:01:43.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.419+0000 7f156e000640 1 -- 172.21.15.67:0/1297756924 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1568009540 con 0x7f15700c2e40 2024-03-09T09:01:43.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.419+0000 7f156e000640 1 -- 172.21.15.67:0/1297756924 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15680098f0 con 0x7f15700c2e40 2024-03-09T09:01:43.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.420+0000 7f1575000640 1 -- 172.21.15.67:0/1297756924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700c2e40 msgr2=0x7f15700c32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:43.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.420+0000 7f1575000640 1 --2- 172.21.15.67:0/1297756924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700c2e40 0x7f15700c32f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f157008cc30 tx=0x7f15680346c0 comp rx=0 tx=0).stop 2024-03-09T09:01:43.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.421+0000 7f1575000640 1 -- 172.21.15.67:0/1297756924 shutdown_connections 2024-03-09T09:01:43.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.421+0000 7f1575000640 1 --2- 172.21.15.67:0/1297756924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15700c2e40 0x7f15700c32f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:43.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.421+0000 7f1575000640 1 --2- 172.21.15.67:0/1297756924 >> v2:172.21.15.134:3300/0 conn(0x7f157006d650 0x7f1570163490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:43.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.421+0000 7f1575000640 1 -- 172.21.15.67:0/1297756924 >> 172.21.15.67:0/1297756924 conn(0x7f15700c7300 msgr2=0x7f15700c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:43.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.421+0000 7f1575000640 1 -- 172.21.15.67:0/1297756924 shutdown_connections 2024-03-09T09:01:43.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.421+0000 7f1575000640 1 -- 172.21.15.67:0/1297756924 wait complete. 2024-03-09T09:01:43.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.424+0000 7f1575000640 1 Processor -- start 2024-03-09T09:01:43.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.424+0000 7f1575000640 1 -- start start 2024-03-09T09:01:43.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.426+0000 7f1575000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f157006d650 0x7f1570165fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:43.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.426+0000 7f1575000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f157008c730 0x7f15701faae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:43.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.426+0000 7f1575000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15700c2e40 0x7f157015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:43.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.426+0000 7f1575000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157008c510 con 0x7f157006d650 2024-03-09T09:01:43.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.426+0000 7f1575000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157008c390 con 0x7f15700c2e40 2024-03-09T09:01:43.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.426+0000 7f1575000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157006da30 con 0x7f157008c730 2024-03-09T09:01:43.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.426+0000 7f156f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f157006d650 0x7f1570165fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:43.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.426+0000 7f156f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f157006d650 0x7f1570165fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59186/0 (socket says 172.21.15.67:59186) 2024-03-09T09:01:43.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.426+0000 7f156f400640 1 -- 172.21.15.67:0/2482970677 learned_addr learned my addr 172.21.15.67:0/2482970677 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.427+0000 7f156fe00640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15700c2e40 0x7f157015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.427+0000 7f156ea00640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f157008c730 0x7f15701faae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.427+0000 7f156f400640 1 -- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f157008c730 msgr2=0x7f15701faae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.427+0000 7f156f400640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f157008c730 0x7f15701faae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.427+0000 7f156f400640 1 -- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15700c2e40 msgr2=0x7f157015f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.427+0000 7f156f400640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15700c2e40 0x7f157015f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.427+0000 7f156f400640 1 -- 172.21.15.67:0/2482970677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570166990 con 0x7f157006d650 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.427+0000 7f156ea00640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f157008c730 0x7f15701faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.427+0000 7f156f400640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f157006d650 0x7f1570165fc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f156000ec80 tx=0x7f156000c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.427+0000 7f1567400640 1 -- 172.21.15.67:0/2482970677 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156000b040 con 0x7f157006d650 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.428+0000 7f1567400640 1 -- 172.21.15.67:0/2482970677 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156000c040 con 0x7f157006d650 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.428+0000 7f1567400640 1 -- 172.21.15.67:0/2482970677 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156001a640 con 0x7f157006d650 2024-03-09T09:01:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.428+0000 7f1575000640 1 -- 172.21.15.67:0/2482970677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15701fa000 con 0x7f157006d650 2024-03-09T09:01:43.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.428+0000 7f1575000640 1 -- 172.21.15.67:0/2482970677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15701fa5c0 con 0x7f157006d650 2024-03-09T09:01:43.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.429+0000 7f1575000640 1 -- 172.21.15.67:0/2482970677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15700c7c10 con 0x7f157006d650 2024-03-09T09:01:43.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.430+0000 7f1567400640 1 -- 172.21.15.67:0/2482970677 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f156001a7e0 con 0x7f157006d650 2024-03-09T09:01:43.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.431+0000 7f1567400640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f155006b500 0x7f155006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:43.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.431+0000 7f1567400640 1 -- 172.21.15.67:0/2482970677 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f156008abe0 con 0x7f157006d650 2024-03-09T09:01:43.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.431+0000 7f156ea00640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f155006b500 0x7f155006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:43.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.432+0000 7f156ea00640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f155006b500 0x7f155006d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1570074010 tx=0x7f155c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:43.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.436+0000 7f1567400640 1 -- 172.21.15.67:0/2482970677 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f15700c7c10 con 0x7f157006d650 2024-03-09T09:01:43.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.770+0000 7f1575000640 1 -- 172.21.15.67:0/2482970677 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f157004f3c0 con 0x7f157006d650 2024-03-09T09:01:43.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.771+0000 7f1567400640 1 -- 172.21.15.67:0/2482970677 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f1560016030 con 0x7f157006d650 2024-03-09T09:01:43.772 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":72,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:01:43.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.773+0000 7f1575000640 1 -- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f155006b500 msgr2=0x7f155006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:43.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.773+0000 7f1575000640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f155006b500 0x7f155006d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1570074010 tx=0x7f155c009040 comp rx=0 tx=0).stop 2024-03-09T09:01:43.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.773+0000 7f1575000640 1 -- 172.21.15.67:0/2482970677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f157006d650 msgr2=0x7f1570165fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:43.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.773+0000 7f1575000640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f157006d650 0x7f1570165fc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f156000ec80 tx=0x7f156000c5c0 comp rx=0 tx=0).stop 2024-03-09T09:01:43.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.773+0000 7f1575000640 1 -- 172.21.15.67:0/2482970677 shutdown_connections 2024-03-09T09:01:43.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.773+0000 7f1575000640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f155006b500 0x7f155006d9c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:43.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.773+0000 7f1575000640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15700c2e40 0x7f157015f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:43.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.773+0000 7f1575000640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f157008c730 0x7f15701faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:43.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.773+0000 7f1575000640 1 --2- 172.21.15.67:0/2482970677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f157006d650 0x7f1570165fc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:43.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.774+0000 7f1575000640 1 -- 172.21.15.67:0/2482970677 >> 172.21.15.67:0/2482970677 conn(0x7f15700c7300 msgr2=0x7f15700c4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:43.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.774+0000 7f1575000640 1 -- 172.21.15.67:0/2482970677 shutdown_connections 2024-03-09T09:01:43.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:43.774+0000 7f1575000640 1 -- 172.21.15.67:0/2482970677 wait complete. 2024-03-09T09:01:43.794 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:01:43.794 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:01:43.794 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:01:43.794 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:01:44.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.058+0000 7fc6e0a00640 1 Processor -- start 2024-03-09T09:01:44.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.058+0000 7fc6e0a00640 1 -- start start 2024-03-09T09:01:44.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.060+0000 7fc6e0a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6dc0b0db0 0x7fc6dc1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.060+0000 7fc6e0a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc6dc0c9410 0x7fc6dc06b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.060+0000 7fc6e0a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc0c6010 con 0x7fc6dc0bec10 2024-03-09T09:01:44.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.060+0000 7fc6e0a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc0c62f0 con 0x7fc6dc0c9410 2024-03-09T09:01:44.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.060+0000 7fc6e0a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0c65d0 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.060+0000 7fc6d9e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc6dc0bec10 0x7fc6dc072770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37832/0 (socket says 172.21.15.67:37832) 2024-03-09T09:01:44.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.060+0000 7fc6d9e00640 1 -- 172.21.15.67:0/2913704738 learned_addr learned my addr 172.21.15.67:0/2913704738 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:01:44.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.060+0000 7fc6da800640 1 --2- 172.21.15.67:0/2913704738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6dc0b0db0 0x7fc6dc1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.060+0000 7fc6db200640 1 --2- 172.21.15.67:0/2913704738 >> v2:172.21.15.134:3300/0 conn(0x7fc6dc0c9410 0x7fc6dc06b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.061+0000 7fc6d9400640 1 -- 172.21.15.67:0/2913704738 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224447932 0 0) 0x7fc6dc0c6010 con 0x7fc6dc0bec10 2024-03-09T09:01:44.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.061+0000 7fc6d9400640 1 -- 172.21.15.67:0/2913704738 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6c0003520 con 0x7fc6dc0bec10 2024-03-09T09:01:44.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.061+0000 7fc6d9400640 1 -- 172.21.15.67:0/2913704738 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3608717020 0 0) 0x7fc6c0003520 con 0x7fc6dc0bec10 2024-03-09T09:01:44.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.061+0000 7fc6da800640 1 -- 172.21.15.67:0/2913704738 >> v2:172.21.15.134:3300/0 conn(0x7fc6dc0c9410 msgr2=0x7fc6dc06b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:44.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.061+0000 7fc6da800640 1 --2- 172.21.15.67:0/2913704738 >> v2:172.21.15.134:3300/0 conn(0x7fc6dc0c9410 0x7fc6dc06b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.061+0000 7fc6da800640 1 -- 172.21.15.67:0/2913704738 >> v1:172.21.15.67:6789/0 conn(0x7fc6dc0bec10 legacy=0x7fc6dc072770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.061+0000 7fc6da800640 1 -- 172.21.15.67:0/2913704738 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc06b040 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.063+0000 7fc6da800640 1 --2- 172.21.15.67:0/2913704738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6dc0b0db0 0x7fc6dc1565a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0097a0 tx=0x7fc6cc02b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f6ecef2ca28401 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.063+0000 7fc6d9400640 1 -- 172.21.15.67:0/2913704738 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc037070 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.063+0000 7fc6d9400640 1 -- 172.21.15.67:0/2913704738 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc030510 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.063+0000 7fc6d9400640 1 -- 172.21.15.67:0/2913704738 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8001670 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.064+0000 7fc6e0a00640 1 -- 172.21.15.67:0/2913704738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6dc0b0db0 msgr2=0x7fc6dc1565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.064+0000 7fc6e0a00640 1 --2- 172.21.15.67:0/2913704738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6dc0b0db0 0x7fc6dc1565a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0097a0 tx=0x7fc6cc02b710 comp rx=0 tx=0).stop 2024-03-09T09:01:44.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.064+0000 7fc6e0a00640 1 -- 172.21.15.67:0/2913704738 shutdown_connections 2024-03-09T09:01:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.064+0000 7fc6e0a00640 1 --2- 172.21.15.67:0/2913704738 >> v2:172.21.15.134:3300/0 conn(0x7fc6dc0c9410 0x7fc6dc06b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.064+0000 7fc6e0a00640 1 --2- 172.21.15.67:0/2913704738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6dc0b0db0 0x7fc6dc1565a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.064+0000 7fc6e0a00640 1 -- 172.21.15.67:0/2913704738 >> 172.21.15.67:0/2913704738 conn(0x7fc6dc0c7f90 msgr2=0x7fc6dc0c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.064+0000 7fc6e0a00640 1 -- 172.21.15.67:0/2913704738 shutdown_connections 2024-03-09T09:01:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.064+0000 7fc6e0a00640 1 -- 172.21.15.67:0/2913704738 wait complete. 2024-03-09T09:01:44.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.067+0000 7fc6e0a00640 1 Processor -- start 2024-03-09T09:01:44.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.067+0000 7fc6e0a00640 1 -- start start 2024-03-09T09:01:44.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6e0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc0b0db0 0x7fc6dc1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6e0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6dc0bec10 0x7fc6dc14f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6e0a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6dc0c9410 0x7fc6dc14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6e0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc072e80 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6e0a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc156ae0 con 0x7fc6dc0bec10 2024-03-09T09:01:44.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6e0a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc073000 con 0x7fc6dc0c9410 2024-03-09T09:01:44.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6da800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc0b0db0 0x7fc6dc1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6da800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc0b0db0 0x7fc6dc1598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59198/0 (socket says 172.21.15.67:59198) 2024-03-09T09:01:44.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6da800640 1 -- 172.21.15.67:0/677023396 learned_addr learned my addr 172.21.15.67:0/677023396 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:44.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6d9e00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6dc0bec10 0x7fc6dc14f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6db200640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6dc0c9410 0x7fc6dc14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6da800640 1 -- 172.21.15.67:0/677023396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6dc0c9410 msgr2=0x7fc6dc14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6da800640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6dc0c9410 0x7fc6dc14f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.069+0000 7fc6da800640 1 -- 172.21.15.67:0/677023396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6dc0bec10 msgr2=0x7fc6dc14f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.070+0000 7fc6da800640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6dc0bec10 0x7fc6dc14f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.070+0000 7fc6da800640 1 -- 172.21.15.67:0/677023396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc0c16f0 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.070+0000 7fc6da800640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc0b0db0 0x7fc6dc1598a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc02b750 tx=0x7fc6cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.070+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc040020 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.070+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc038040 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.070+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc037070 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.070+0000 7fc6e0a00640 1 -- 172.21.15.67:0/677023396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc1ebb20 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.071+0000 7fc6e0a00640 1 -- 172.21.15.67:0/677023396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc1ec0e0 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.072+0000 7fc6e0a00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc14fe40 0x7fc6dc156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.072+0000 7fc6e0a00640 1 -- 172.21.15.67:0/677023396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0c72f0 con 0x7fc6dc14fe40 2024-03-09T09:01:44.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.072+0000 7fc6d9e00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc14fe40 0x7fc6dc156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.072+0000 7fc6d9e00640 1 -- 172.21.15.67:0/677023396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6dc158af0 con 0x7fc6dc14fe40 2024-03-09T09:01:44.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.073+0000 7fc6d9e00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc14fe40 0x7fc6dc156f60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8005880 tx=0x7fc6c800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.073+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c80058d0 con 0x7fc6dc14fe40 2024-03-09T09:01:44.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.073+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc6c8018070 con 0x7fc6dc14fe40 2024-03-09T09:01:44.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.073+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc14fe40 msgr2=0x7fc6dc156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.073+0000 7fc6d2a00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc14fe40 0x7fc6dc156f60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8005880 tx=0x7fc6c800eb90 comp rx=0 tx=0).stop 2024-03-09T09:01:44.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.073+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc14fe40 msgr2=0x7fc6dc156f60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:44.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.073+0000 7fc6d2a00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc14fe40 0x7fc6dc156f60 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8005880 tx=0x7fc6c800eb90 comp rx=0 tx=0).stop 2024-03-09T09:01:44.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.074+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc6cc043020 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.075+0000 7fc6d2a00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc6bc06afb0 0x7fc6bc06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.075+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc6cc0ab700 con 0x7fc6dc0b0db0 2024-03-09T09:01:44.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.075+0000 7fc6d9e00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc6bc06afb0 0x7fc6bc06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.077+0000 7fc6d9e00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc6bc06afb0 0x7fc6bc06d470 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80011e0 tx=0x7fc6c8012000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.081+0000 7fc6e0a00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc1574a0 0x7fc6dc157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.081+0000 7fc6e0a00640 1 -- 172.21.15.67:0/677023396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0ae320 con 0x7fc6dc1574a0 2024-03-09T09:01:44.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.082+0000 7fc6db200640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc1574a0 0x7fc6dc157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.082+0000 7fc6db200640 1 -- 172.21.15.67:0/677023396 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc6dc04f3c0 con 0x7fc6dc1574a0 2024-03-09T09:01:44.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.082+0000 7fc6db200640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc1574a0 0x7fc6dc157880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0cc560 tx=0x7fc6d400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4003990 con 0x7fc6dc1574a0 2024-03-09T09:01:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7fc6d4003990 con 0x7fc6dc1574a0 2024-03-09T09:01:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc1574a0 msgr2=0x7fc6dc157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6d2a00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc1574a0 0x7fc6dc157880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0cc560 tx=0x7fc6d400f810 comp rx=0 tx=0).stop 2024-03-09T09:01:44.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6d2a00640 1 -- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc1574a0 msgr2=0x7fc6dc157880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:44.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6d2a00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc1574a0 0x7fc6dc157880 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0cc560 tx=0x7fc6d400f810 comp rx=0 tx=0).stop 2024-03-09T09:01:44.086 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:01:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:01:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:01:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 72, 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:01:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:01:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:01:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:01:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:01:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:01:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:01:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:01:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.101 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:01:44.101 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:01:44.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6e0a00640 1 -- 172.21.15.67:0/677023396 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc6bc06afb0 msgr2=0x7fc6bc06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6e0a00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc6bc06afb0 0x7fc6bc06d470 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80011e0 tx=0x7fc6c8012000 comp rx=0 tx=0).stop 2024-03-09T09:01:44.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6e0a00640 1 -- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc0b0db0 msgr2=0x7fc6dc1598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6e0a00640 1 --2- 172.21.15.67:0/677023396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6dc0b0db0 0x7fc6dc1598a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc02b750 tx=0x7fc6cc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6db200640 1 -- 172.21.15.67:0/677023396 reap_dead start 2024-03-09T09:01:44.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6db200640 1 -- 172.21.15.67:0/677023396 reap_dead start 2024-03-09T09:01:44.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6e0a00640 1 -- 172.21.15.67:0/677023396 shutdown_connections 2024-03-09T09:01:44.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.083+0000 7fc6e0a00640 1 -- 172.21.15.67:0/677023396 >> 172.21.15.67:0/677023396 conn(0x7fc6dc0c7f90 msgr2=0x7fc6dc0ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:44.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.084+0000 7fc6e0a00640 1 -- 172.21.15.67:0/677023396 shutdown_connections 2024-03-09T09:01:44.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.084+0000 7fc6e0a00640 1 -- 172.21.15.67:0/677023396 wait complete. 2024-03-09T09:01:44.104 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:01:44.104 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:01:44.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:01:44.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.320+0000 7fa0e5000640 1 Processor -- start 2024-03-09T09:01:44.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.321+0000 7fa0e5000640 1 -- start start 2024-03-09T09:01:44.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.322+0000 7fa0e5000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa0e0151360 0x7fa0e00c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.322+0000 7fa0e5000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e00c2780 con 0x7fa0e00ac930 2024-03-09T09:01:44.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.322+0000 7fa0e5000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e00708c0 con 0x7fa0e00c30a0 2024-03-09T09:01:44.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.322+0000 7fa0e5000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e0070670 con 0x7fa0e0151360 2024-03-09T09:01:44.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.322+0000 7fa0df400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa0e00ac930 0x7fa0e0150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37838/0 (socket says 172.21.15.67:37838) 2024-03-09T09:01:44.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.322+0000 7fa0df400640 1 -- 172.21.15.67:0/1000631507 learned_addr learned my addr 172.21.15.67:0/1000631507 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:01:44.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0dea00640 1 --2- 172.21.15.67:0/1000631507 >> v2:172.21.15.134:3300/0 conn(0x7fa0e0151360 0x7fa0e00c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421442794 0 0) 0x7fa0e00c2780 con 0x7fa0e00ac930 2024-03-09T09:01:44.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c4003520 con 0x7fa0e00ac930 2024-03-09T09:01:44.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3456992307 0 0) 0x7fa0c4003520 con 0x7fa0e00ac930 2024-03-09T09:01:44.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 >> v2:172.21.15.134:3300/0 conn(0x7fa0e0151360 msgr2=0x7fa0e00c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 --2- 172.21.15.67:0/1000631507 >> v2:172.21.15.134:3300/0 conn(0x7fa0e0151360 0x7fa0e00c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 >> v1:172.21.15.134:6789/0 conn(0x7fa0e00c30a0 legacy=0x7fa0e01539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e0070ba0 con 0x7fa0e00ac930 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fa0d80019f0 con 0x7fa0e00ac930 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 >> v1:172.21.15.67:6789/0 conn(0x7fa0e00ac930 legacy=0x7fa0e0150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 --2- 172.21.15.67:0/1000631507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0c4003a20 0x7fa0c4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 --2- 172.21.15.67:0/1000631507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c4004800 0x7fa0c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 --2- 172.21.15.67:0/1000631507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0c40051c0 0x7fa0c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e0070670 con 0x7fa0c4003a20 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.323+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c400c0f0 con 0x7fa0c4004800 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.324+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4010700 con 0x7fa0c40051c0 2024-03-09T09:01:44.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.324+0000 7fa0df400640 1 --2- 172.21.15.67:0/1000631507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0c4003a20 0x7fa0c4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.324+0000 7fa0df400640 1 -- 172.21.15.67:0/1000631507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0c40051c0 msgr2=0x7fa0c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:44.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.324+0000 7fa0df400640 1 --2- 172.21.15.67:0/1000631507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0c40051c0 0x7fa0c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.324+0000 7fa0df400640 1 -- 172.21.15.67:0/1000631507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c4004800 msgr2=0x7fa0c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:01:44.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.324+0000 7fa0dea00640 1 --2- 172.21.15.67:0/1000631507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c4004800 0x7fa0c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.324+0000 7fa0df400640 1 --2- 172.21.15.67:0/1000631507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0c4004800 0x7fa0c4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.324+0000 7fa0df400640 1 -- 172.21.15.67:0/1000631507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4003520 con 0x7fa0c4003a20 2024-03-09T09:01:44.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.324+0000 7fa0dfe00640 1 -- 172.21.15.67:0/1000631507 reap_dead start 2024-03-09T09:01:44.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.326+0000 7fa0df400640 1 --2- 172.21.15.67:0/1000631507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0c4003a20 0x7fa0c4003e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800a860 tx=0x7fa0c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6936d16bf7048163 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.327+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d800a480 con 0x7fa0c4003a20 2024-03-09T09:01:44.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.327+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d8034630 con 0x7fa0c4003a20 2024-03-09T09:01:44.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.327+0000 7fa0de000640 1 -- 172.21.15.67:0/1000631507 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8034ae0 con 0x7fa0c4003a20 2024-03-09T09:01:44.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.327+0000 7fa0e5000640 1 -- 172.21.15.67:0/1000631507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0c4003a20 msgr2=0x7fa0c4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.327+0000 7fa0e5000640 1 --2- 172.21.15.67:0/1000631507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0c4003a20 0x7fa0c4003e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800a860 tx=0x7fa0c4005830 comp rx=0 tx=0).stop 2024-03-09T09:01:44.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.327+0000 7fa0e5000640 1 -- 172.21.15.67:0/1000631507 shutdown_connections 2024-03-09T09:01:44.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.327+0000 7fa0e5000640 1 --2- 172.21.15.67:0/1000631507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0c4003a20 0x7fa0c4003e40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.327+0000 7fa0e5000640 1 -- 172.21.15.67:0/1000631507 >> 172.21.15.67:0/1000631507 conn(0x7fa0e00c7220 msgr2=0x7fa0e00c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:44.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.328+0000 7fa0e5000640 1 -- 172.21.15.67:0/1000631507 shutdown_connections 2024-03-09T09:01:44.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.328+0000 7fa0e5000640 1 -- 172.21.15.67:0/1000631507 wait complete. 2024-03-09T09:01:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.329+0000 7fa0e5000640 1 Processor -- start 2024-03-09T09:01:44.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.330+0000 7fa0e5000640 1 -- start start 2024-03-09T09:01:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.331+0000 7fa0e5000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0c4003a20 0x7fa0e00aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.331+0000 7fa0e5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0e01ead90 0x7fa0e0145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.331+0000 7fa0e5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0e014bd20 0x7fa0e014c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.331+0000 7fa0e5000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00ace90 con 0x7fa0e01ead90 2024-03-09T09:01:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.331+0000 7fa0e5000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00acd10 con 0x7fa0e014bd20 2024-03-09T09:01:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.331+0000 7fa0e5000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00c3660 con 0x7fa0c4003a20 2024-03-09T09:01:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.331+0000 7fa0dea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0e01ead90 0x7fa0e0145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.331+0000 7fa0dea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0e01ead90 0x7fa0e0145f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59246/0 (socket says 172.21.15.67:59246) 2024-03-09T09:01:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.331+0000 7fa0dea00640 1 -- 172.21.15.67:0/3723792497 learned_addr learned my addr 172.21.15.67:0/3723792497 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:44.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.331+0000 7fa0df400640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0c4003a20 0x7fa0e00aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.332+0000 7fa0dfe00640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0e014bd20 0x7fa0e014c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.332+0000 7fa0dea00640 1 -- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0c4003a20 msgr2=0x7fa0e00aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.332+0000 7fa0dea00640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0c4003a20 0x7fa0e00aa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.332+0000 7fa0dea00640 1 -- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0e014bd20 msgr2=0x7fa0e014c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.332+0000 7fa0dea00640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0e014bd20 0x7fa0e014c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.332+0000 7fa0dea00640 1 -- 172.21.15.67:0/3723792497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e00a9cb0 con 0x7fa0e01ead90 2024-03-09T09:01:44.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.332+0000 7fa0dea00640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0e01ead90 0x7fa0e0145f70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80107f0 tx=0x7fa0c8010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.333+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800f040 con 0x7fa0e01ead90 2024-03-09T09:01:44.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.333+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c800eb60 con 0x7fa0e01ead90 2024-03-09T09:01:44.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.333+0000 7fa0e5000640 1 -- 172.21.15.67:0/3723792497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e014b560 con 0x7fa0e01ead90 2024-03-09T09:01:44.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.333+0000 7fa0e5000640 1 -- 172.21.15.67:0/3723792497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e014de90 con 0x7fa0e01ead90 2024-03-09T09:01:44.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.333+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c801c400 con 0x7fa0e01ead90 2024-03-09T09:01:44.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.334+0000 7fa0e5000640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8002a70 0x7fa0b8004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.334+0000 7fa0e5000640 1 -- 172.21.15.67:0/3723792497 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b8007ec0 con 0x7fa0b8002a70 2024-03-09T09:01:44.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.335+0000 7fa0df400640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8002a70 0x7fa0b8004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.336+0000 7fa0df400640 1 -- 172.21.15.67:0/3723792497 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0b80080c0 con 0x7fa0b8002a70 2024-03-09T09:01:44.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.336+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa0c801c5a0 con 0x7fa0e01ead90 2024-03-09T09:01:44.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.336+0000 7fa0df400640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8002a70 0x7fa0b8004f30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800a830 tx=0x7fa0e00729f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.338+0000 7fa0d7400640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0bc06b4d0 0x7fa0bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.338+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa0c808a9f0 con 0x7fa0e01ead90 2024-03-09T09:01:44.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.338+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d800a540 con 0x7fa0b8002a70 2024-03-09T09:01:44.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.338+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa0d803f450 con 0x7fa0b8002a70 2024-03-09T09:01:44.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.338+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8002a70 msgr2=0x7fa0b8004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.338+0000 7fa0d7400640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8002a70 0x7fa0b8004f30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800a830 tx=0x7fa0e00729f0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.338+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8002a70 msgr2=0x7fa0b8004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:44.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.338+0000 7fa0d7400640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8002a70 0x7fa0b8004f30 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800a830 tx=0x7fa0e00729f0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.338+0000 7fa0dfe00640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0bc06b4d0 0x7fa0bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.340+0000 7fa0dfe00640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0bc06b4d0 0x7fa0bc06d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa0e006ec40 tx=0x7fa0d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.347+0000 7fa0e5000640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8006f00 0x7fa0b8007390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.347+0000 7fa0e5000640 1 -- 172.21.15.67:0/3723792497 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b800a4a0 con 0x7fa0b8006f00 2024-03-09T09:01:44.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.348+0000 7fa0df400640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8006f00 0x7fa0b8007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.349+0000 7fa0df400640 1 -- 172.21.15.67:0/3723792497 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa0b80078d0 con 0x7fa0b8006f00 2024-03-09T09:01:44.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.349+0000 7fa0df400640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8006f00 0x7fa0b8007390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80012e0 tx=0x7fa0d8030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.350+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d803fc40 con 0x7fa0b8006f00 2024-03-09T09:01:44.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.350+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fa0d803fc40 con 0x7fa0b8006f00 2024-03-09T09:01:44.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.350+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8006f00 msgr2=0x7fa0b8007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.350+0000 7fa0d7400640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8006f00 0x7fa0b8007390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80012e0 tx=0x7fa0d8030000 comp rx=0 tx=0).stop 2024-03-09T09:01:44.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.350+0000 7fa0d7400640 1 -- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8006f00 msgr2=0x7fa0b8007390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:44.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.350+0000 7fa0d7400640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8006f00 0x7fa0b8007390 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80012e0 tx=0x7fa0d8030000 comp rx=0 tx=0).stop 2024-03-09T09:01:44.352 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 72, 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:44.353 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:01:44.354 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:01:44.355 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:01:44.356 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:01:44.356 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:01:44.356 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:01:44.356 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:44.356 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:44.356 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:01:44.356 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:01:44.356 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:01:44.356 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:44.357 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:44.358 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:44.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:01:44.360 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.361 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.361 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.361 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:01:44.361 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:01:44.361 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:44.361 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:44.361 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:44.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.362 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:01:44.362 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:01:44.362 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:44.362 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:44.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.362 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:44.362 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:01:44.362 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.362 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:01:44.363 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.365 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:01:44.365 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:01:44.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.351+0000 7fa0e5000640 1 -- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0bc06b4d0 msgr2=0x7fa0bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.351+0000 7fa0e5000640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0bc06b4d0 0x7fa0bc06d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa0e006ec40 tx=0x7fa0d0009040 comp rx=0 tx=0).stop 2024-03-09T09:01:44.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.351+0000 7fa0e5000640 1 -- 172.21.15.67:0/3723792497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0e01ead90 msgr2=0x7fa0e0145f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.351+0000 7fa0e5000640 1 --2- 172.21.15.67:0/3723792497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa0e01ead90 0x7fa0e0145f70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80107f0 tx=0x7fa0c8010cb0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.351+0000 7fa0dfe00640 1 -- 172.21.15.67:0/3723792497 reap_dead start 2024-03-09T09:01:44.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.351+0000 7fa0dfe00640 1 -- 172.21.15.67:0/3723792497 reap_dead start 2024-03-09T09:01:44.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.351+0000 7fa0e5000640 1 -- 172.21.15.67:0/3723792497 shutdown_connections 2024-03-09T09:01:44.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.351+0000 7fa0e5000640 1 -- 172.21.15.67:0/3723792497 >> 172.21.15.67:0/3723792497 conn(0x7fa0e00c7220 msgr2=0x7fa0e014e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:44.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.352+0000 7fa0e5000640 1 -- 172.21.15.67:0/3723792497 shutdown_connections 2024-03-09T09:01:44.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.352+0000 7fa0e5000640 1 -- 172.21.15.67:0/3723792497 wait complete. 2024-03-09T09:01:44.372 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:01:44.372 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:01:44.373 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:01:44.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.699+0000 7fcdfa600640 1 Processor -- start 2024-03-09T09:01:44.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.699+0000 7fcdfa600640 1 -- start start 2024-03-09T09:01:44.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.702+0000 7fcdfa600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcdec0cda50 0x7fcdec0d79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.702+0000 7fcdfa600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdec0d7ef0 con 0x7fcdec0db1c0 2024-03-09T09:01:44.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.702+0000 7fcdfa600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdec0cde30 con 0x7fcdec006ea0 2024-03-09T09:01:44.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.702+0000 7fcdfa600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdec04ac10 con 0x7fcdec0cda50 2024-03-09T09:01:44.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.705+0000 7fcdf3e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fcdec006ea0 0x7fcdec0daab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:39472/0 (socket says 172.21.15.67:39472) 2024-03-09T09:01:44.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.705+0000 7fcdf3e00640 1 -- 172.21.15.67:0/3427825428 learned_addr learned my addr 172.21.15.67:0/3427825428 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:01:44.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.705+0000 7fcdf9200640 1 --2- 172.21.15.67:0/3427825428 >> v2:172.21.15.134:3300/0 conn(0x7fcdec0cda50 0x7fcdec0d79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.706+0000 7fcdf9200640 1 -- 172.21.15.67:0/3427825428 >> v1:172.21.15.134:6789/0 conn(0x7fcdec006ea0 legacy=0x7fcdec0daab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.706+0000 7fcdf9200640 1 -- 172.21.15.67:0/3427825428 >> v1:172.21.15.67:6789/0 conn(0x7fcdec0db1c0 legacy=0x7fcdec0e1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.708+0000 7fcdf9200640 1 -- 172.21.15.67:0/3427825428 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec00b6a0 con 0x7fcdec0cda50 2024-03-09T09:01:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.712+0000 7fcdf9200640 1 --2- 172.21.15.67:0/3427825428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdec0cda50 0x7fcdec0d79b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0ce000 tx=0x7fcde002e110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0cb46b4dff475fb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.713+0000 7fcdf3400640 1 -- 172.21.15.67:0/3427825428 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde003b070 con 0x7fcdec0cda50 2024-03-09T09:01:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.713+0000 7fcdf3400640 1 -- 172.21.15.67:0/3427825428 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde0036440 con 0x7fcdec0cda50 2024-03-09T09:01:44.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.713+0000 7fcdf3400640 1 -- 172.21.15.67:0/3427825428 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0036810 con 0x7fcdec0cda50 2024-03-09T09:01:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.714+0000 7fcdfa600640 1 -- 172.21.15.67:0/3427825428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdec0cda50 msgr2=0x7fcdec0d79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.714+0000 7fcdfa600640 1 --2- 172.21.15.67:0/3427825428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdec0cda50 0x7fcdec0d79b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0ce000 tx=0x7fcde002e110 comp rx=0 tx=0).stop 2024-03-09T09:01:44.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.717+0000 7fcdfa600640 1 -- 172.21.15.67:0/3427825428 shutdown_connections 2024-03-09T09:01:44.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.717+0000 7fcdfa600640 1 --2- 172.21.15.67:0/3427825428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdec0cda50 0x7fcdec0d79b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.717+0000 7fcdfa600640 1 -- 172.21.15.67:0/3427825428 >> 172.21.15.67:0/3427825428 conn(0x7fcdec048250 msgr2=0x7fcdec048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.718+0000 7fcdfa600640 1 -- 172.21.15.67:0/3427825428 shutdown_connections 2024-03-09T09:01:44.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.718+0000 7fcdfa600640 1 -- 172.21.15.67:0/3427825428 wait complete. 2024-03-09T09:01:44.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.723+0000 7fcdfa600640 1 Processor -- start 2024-03-09T09:01:44.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.724+0000 7fcdfa600640 1 -- start start 2024-03-09T09:01:44.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.726+0000 7fcdfa600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdec006ea0 0x7fcdec16ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.726+0000 7fcdfa600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec0cda50 0x7fcdec046520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.726+0000 7fcdfa600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0db1c0 0x7fcdec16d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.726+0000 7fcdfa600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec04b490 con 0x7fcdec0cda50 2024-03-09T09:01:44.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.726+0000 7fcdfa600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec04b1b0 con 0x7fcdec006ea0 2024-03-09T09:01:44.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.726+0000 7fcdfa600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec0da590 con 0x7fcdec0db1c0 2024-03-09T09:01:44.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.730+0000 7fcdf9200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdec006ea0 0x7fcdec16ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.731+0000 7fcdf3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec0cda50 0x7fcdec046520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.731+0000 7fcdf9c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0db1c0 0x7fcdec16d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.731+0000 7fcdf9c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0db1c0 0x7fcdec16d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:39228/0 (socket says 172.21.15.67:39228) 2024-03-09T09:01:44.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.731+0000 7fcdf9c00640 1 -- 172.21.15.67:0/1471996330 learned_addr learned my addr 172.21.15.67:0/1471996330 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.734+0000 7fcdf3e00640 1 -- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0db1c0 msgr2=0x7fcdec16d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.734+0000 7fcdf3e00640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0db1c0 0x7fcdec16d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.734+0000 7fcdf3e00640 1 -- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdec006ea0 msgr2=0x7fcdec16ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.734+0000 7fcdf3e00640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdec006ea0 0x7fcdec16ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.734+0000 7fcdf3e00640 1 -- 172.21.15.67:0/1471996330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec041cc0 con 0x7fcdec0cda50 2024-03-09T09:01:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.735+0000 7fcdf3e00640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec0cda50 0x7fcdec046520 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcde8000f60 tx=0x7fcde800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.736+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8017070 con 0x7fcdec0cda50 2024-03-09T09:01:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.736+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde80058b0 con 0x7fcdec0cda50 2024-03-09T09:01:44.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.736+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8005ca0 con 0x7fcdec0cda50 2024-03-09T09:01:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.737+0000 7fcdfa600640 1 -- 172.21.15.67:0/1471996330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec041fa0 con 0x7fcdec0cda50 2024-03-09T09:01:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.737+0000 7fcdfa600640 1 -- 172.21.15.67:0/1471996330 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec16f790 con 0x7fcdec0cda50 2024-03-09T09:01:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.750+0000 7fcdfa600640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0412e0 0x7fcdec0416c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.750+0000 7fcdfa600640 1 -- 172.21.15.67:0/1471996330 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec005390 con 0x7fcdec0412e0 2024-03-09T09:01:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.751+0000 7fcdf9200640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0412e0 0x7fcdec0416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.752+0000 7fcdf9200640 1 -- 172.21.15.67:0/1471996330 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdec00b900 con 0x7fcdec0412e0 2024-03-09T09:01:44.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.754+0000 7fcdf9200640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0412e0 0x7fcdec0416c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcdec047df0 tx=0x7fcde0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.755+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcde801a070 con 0x7fcdec0cda50 2024-03-09T09:01:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.757+0000 7fcdf0c00640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcdd406b4d0 0x7fcdd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.757+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0046030 con 0x7fcdec0412e0 2024-03-09T09:01:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.757+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcde80897e0 con 0x7fcdec0cda50 2024-03-09T09:01:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.757+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fcde0041470 con 0x7fcdec0412e0 2024-03-09T09:01:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.757+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0412e0 msgr2=0x7fcdec0416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.757+0000 7fcdf0c00640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0412e0 0x7fcdec0416c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcdec047df0 tx=0x7fcde0004730 comp rx=0 tx=0).stop 2024-03-09T09:01:44.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.757+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0412e0 msgr2=0x7fcdec0416c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:44.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.757+0000 7fcdf0c00640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0412e0 0x7fcdec0416c0 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcdec047df0 tx=0x7fcde0004730 comp rx=0 tx=0).stop 2024-03-09T09:01:44.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.760+0000 7fcdf9c00640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcdd406b4d0 0x7fcdd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.762+0000 7fcdf9c00640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcdd406b4d0 0x7fcdd406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcdf406a1a0 tx=0x7fcdf4076980 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.779+0000 7fcdfa600640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0d5e60 0x7fcdec0d6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.779+0000 7fcdfa600640 1 -- 172.21.15.67:0/1471996330 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec04a6e0 con 0x7fcdec0d5e60 2024-03-09T09:01:44.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.779+0000 7fcdf9200640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0d5e60 0x7fcdec0d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:44.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.780+0000 7fcdf9200640 1 -- 172.21.15.67:0/1471996330 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcdec0d7ae0 con 0x7fcdec0d5e60 2024-03-09T09:01:44.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.781+0000 7fcdf9200640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0d5e60 0x7fcdec0d6270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcde002e150 tx=0x7fcde00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.783+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0046040 con 0x7fcdec0d5e60 2024-03-09T09:01:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.783+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5044 (secure 0 0 0) 0x7fcde0004440 con 0x7fcdec0d5e60 2024-03-09T09:01:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.783+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0d5e60 msgr2=0x7fcdec0d6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.783+0000 7fcdf0c00640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0d5e60 0x7fcdec0d6270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcde002e150 tx=0x7fcde00046c0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.783+0000 7fcdf0c00640 1 -- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0d5e60 msgr2=0x7fcdec0d6270 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:01:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.783+0000 7fcdf0c00640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdec0d5e60 0x7fcdec0d6270 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcde002e150 tx=0x7fcde00046c0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.785 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 72, 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:44.786 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.787 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:01:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:01:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:01:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:01:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:01:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:01:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:01:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:01:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:01:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:01:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:01:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:01:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:01:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:01:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:01:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:01:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:01:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:01:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:01:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:01:44.799 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:01:44.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.784+0000 7fcdfa600640 1 -- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcdd406b4d0 msgr2=0x7fcdd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.785+0000 7fcdfa600640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcdd406b4d0 0x7fcdd406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcdf406a1a0 tx=0x7fcdf4076980 comp rx=0 tx=0).stop 2024-03-09T09:01:44.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.785+0000 7fcdf9c00640 1 -- 172.21.15.67:0/1471996330 reap_dead start 2024-03-09T09:01:44.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.785+0000 7fcdfa600640 1 -- 172.21.15.67:0/1471996330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec0cda50 msgr2=0x7fcdec046520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:44.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.785+0000 7fcdfa600640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec0cda50 0x7fcdec046520 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcde8000f60 tx=0x7fcde800cbe0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.785+0000 7fcdfa600640 1 -- 172.21.15.67:0/1471996330 shutdown_connections 2024-03-09T09:01:44.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.785+0000 7fcdfa600640 1 --2- 172.21.15.67:0/1471996330 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcdec0cda50 0x7fcdec046520 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcde8000f60 tx=0x7fcde800cbe0 comp rx=0 tx=0).stop 2024-03-09T09:01:44.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.785+0000 7fcdfa600640 1 -- 172.21.15.67:0/1471996330 >> 172.21.15.67:0/1471996330 conn(0x7fcdec048250 msgr2=0x7fcdec048630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:44.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.787+0000 7fcdfa600640 1 -- 172.21.15.67:0/1471996330 shutdown_connections 2024-03-09T09:01:44.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:44.787+0000 7fcdfa600640 1 -- 172.21.15.67:0/1471996330 wait complete. 2024-03-09T09:01:44.814 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:01:44.815 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-03-09T09:01:44.815 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:01:44.815 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-03-09T09:01:44.815 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:01:44.815 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-09T09:01:44.815 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:01:44.853 INFO:tasks.ceph.mon.a:Stopped 2024-03-09T09:01:44.854 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:01:44.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:01:45.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.174+0000 7f1dd3a00640 1 Processor -- start 2024-03-09T09:01:45.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.174+0000 7f1dd3a00640 1 -- start start 2024-03-09T09:01:45.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.176+0000 7f1dd3a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1dcc153500 0x7f1dcc06b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:45.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.176+0000 7f1dd3a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:45.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.176+0000 7f1dd3a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dcc15a2a0 con 0x7f1dcc0c6630 2024-03-09T09:01:45.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.176+0000 7f1dd3a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dcc159d70 con 0x7f1dcc153500 2024-03-09T09:01:45.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.176+0000 7f1dd3a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc06bb30 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.176+0000 7f1dcbe00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f1dcc0c6630 legacy=0x7f1dcc0c74b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:01:45.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.176+0000 7f1dd1000640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f1dcc153500 msgr2=0x7f1dcc06b040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:01:45.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.176+0000 7f1dd1000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1dcc153500 0x7f1dcc06b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:45.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.177+0000 7f1dd1a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.177+0000 7f1dd1a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:39312/0 (socket says 172.21.15.67:39312) 2024-03-09T09:01:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.177+0000 7f1dd1a00640 1 -- 172.21.15.67:0/2881786291 learned_addr learned my addr 172.21.15.67:0/2881786291 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.177+0000 7f1dd1a00640 1 -- 172.21.15.67:0/2881786291 >> v2:172.21.15.67:3300/0 conn(0x7f1dcc153500 msgr2=0x7f1dcc06b040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.177+0000 7f1dd1a00640 1 --2- 172.21.15.67:0/2881786291 >> v2:172.21.15.67:3300/0 conn(0x7f1dcc153500 0x7f1dcc06b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.177+0000 7f1dd1a00640 1 -- 172.21.15.67:0/2881786291 >> v1:172.21.15.67:6789/0 conn(0x7f1dcc0c6630 legacy=0x7f1dcc0c74b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:45.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.177+0000 7f1dd1a00640 1 -- 172.21.15.67:0/2881786291 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc159fc0 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.179+0000 7f1dd1a00640 1 --2- 172.21.15.67:0/2881786291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1606f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009e50 tx=0x7f1dc402bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a5d2c214cae514 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:45.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.179+0000 7f1dcb400640 1 -- 172.21.15.67:0/2881786291 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc402c980 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.179+0000 7f1dcb400640 1 -- 172.21.15.67:0/2881786291 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc402cb20 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.179+0000 7f1dcb400640 1 -- 172.21.15.67:0/2881786291 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40343f0 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.180+0000 7f1dd3a00640 1 -- 172.21.15.67:0/2881786291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 msgr2=0x7f1dcc1606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:45.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.180+0000 7f1dd3a00640 1 --2- 172.21.15.67:0/2881786291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1606f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009e50 tx=0x7f1dc402bde0 comp rx=0 tx=0).stop 2024-03-09T09:01:45.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.180+0000 7f1dd3a00640 1 -- 172.21.15.67:0/2881786291 shutdown_connections 2024-03-09T09:01:45.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.180+0000 7f1dd3a00640 1 --2- 172.21.15.67:0/2881786291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1606f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:45.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.180+0000 7f1dd3a00640 1 --2- 172.21.15.67:0/2881786291 >> v2:172.21.15.67:3300/0 conn(0x7f1dcc153500 0x7f1dcc06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:45.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.180+0000 7f1dd3a00640 1 -- 172.21.15.67:0/2881786291 >> 172.21.15.67:0/2881786291 conn(0x7f1dcc0c8060 msgr2=0x7f1dcc0c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:45.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.180+0000 7f1dd3a00640 1 -- 172.21.15.67:0/2881786291 shutdown_connections 2024-03-09T09:01:45.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.180+0000 7f1dd3a00640 1 -- 172.21.15.67:0/2881786291 wait complete. 2024-03-09T09:01:45.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.183+0000 7f1dd3a00640 1 Processor -- start 2024-03-09T09:01:45.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.183+0000 7f1dd3a00640 1 -- start start 2024-03-09T09:01:45.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.184+0000 7f1dd3a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1efa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:45.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.184+0000 7f1dd3a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1dcc153500 0x7f1dcc1ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:45.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.184+0000 7f1dd3a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dcc14aaf0 0x7f1dcc14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:45.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.184+0000 7f1dd3a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc0bed90 con 0x7f1dcc14aaf0 2024-03-09T09:01:45.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.184+0000 7f1dd3a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc0c6f90 con 0x7f1dcc153500 2024-03-09T09:01:45.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dd3a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc0c94e0 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dd1a00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dcc14aaf0 msgr2=0x7f1dcc14bd60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:01:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dd1a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dcc14aaf0 0x7f1dcc14bd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dd1000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dd1000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:39320/0 (socket says 172.21.15.67:39320) 2024-03-09T09:01:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dd1000640 1 -- 172.21.15.67:0/804560872 learned_addr learned my addr 172.21.15.67:0/804560872 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dcbe00640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1dcc153500 0x7f1dcc1ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dd1000640 1 -- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1dcc153500 msgr2=0x7f1dcc1ebb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dd1000640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1dcc153500 0x7f1dcc1ebb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dd1000640 1 -- 172.21.15.67:0/804560872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dcc14aaf0 msgr2=0x7f1dcc14bd60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:01:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dd1000640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dcc14aaf0 0x7f1dcc14bd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.185+0000 7f1dd1000640 1 -- 172.21.15.67:0/804560872 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc1eff50 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.186+0000 7f1dd1000640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1efa10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000b550 tx=0x7f1dc000b950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:45.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.186+0000 7f1dc8c00640 1 -- 172.21.15.67:0/804560872 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0016070 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.186+0000 7f1dc8c00640 1 -- 172.21.15.67:0/804560872 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc0003d10 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.186+0000 7f1dd3a00640 1 -- 172.21.15.67:0/804560872 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc1489a0 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.186+0000 7f1dc8c00640 1 -- 172.21.15.67:0/804560872 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00052b0 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.186+0000 7f1dd3a00640 1 -- 172.21.15.67:0/804560872 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc1ef210 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.188+0000 7f1dd3a00640 1 -- 172.21.15.67:0/804560872 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dcc075a00 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.189+0000 7f1dc8c00640 1 -- 172.21.15.67:0/804560872 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1dc0019070 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.190+0000 7f1dc8c00640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dac06b480 0x7f1dac06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.190+0000 7f1dc8c00640 1 -- 172.21.15.67:0/804560872 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1dc0089880 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.191+0000 7f1dcbe00640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dac06b480 0x7f1dac06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:45.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.191+0000 7f1dcbe00640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dac06b480 0x7f1dac06d940 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0ca850 tx=0x7f1db8009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:45.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.195+0000 7f1dc8c00640 1 -- 172.21.15.67:0/804560872 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1dc0058300 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.583+0000 7f1dd3a00640 1 -- 172.21.15.67:0/804560872 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1dcc0cafb0 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.584+0000 7f1dc8c00640 1 -- 172.21.15.67:0/804560872 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f1dc0057c00 con 0x7f1dcc0c7bc0 2024-03-09T09:01:45.586 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":72,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:01:45.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 -- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dac06b480 msgr2=0x7f1dac06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:45.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dac06b480 0x7f1dac06d940 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0ca850 tx=0x7f1db8009110 comp rx=0 tx=0).stop 2024-03-09T09:01:45.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 -- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 msgr2=0x7f1dcc1efa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:45.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1efa10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000b550 tx=0x7f1dc000b950 comp rx=0 tx=0).stop 2024-03-09T09:01:45.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 -- 172.21.15.67:0/804560872 shutdown_connections 2024-03-09T09:01:45.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dac06b480 0x7f1dac06d940 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:45.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dcc14aaf0 0x7f1dcc14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:45.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1dcc153500 0x7f1dcc1ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:45.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 --2- 172.21.15.67:0/804560872 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dcc0c7bc0 0x7f1dcc1efa10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:45.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 -- 172.21.15.67:0/804560872 >> 172.21.15.67:0/804560872 conn(0x7f1dcc0c8060 msgr2=0x7f1dcc152500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:45.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 -- 172.21.15.67:0/804560872 shutdown_connections 2024-03-09T09:01:45.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:45.587+0000 7f1dd3a00640 1 -- 172.21.15.67:0/804560872 wait complete. 2024-03-09T09:01:48.608 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:01:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.823+0000 7f9d40200640 1 Processor -- start 2024-03-09T09:01:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.823+0000 7f9d40200640 1 -- start start 2024-03-09T09:01:48.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.824+0000 7f9d40200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d380c30f0 0x7f9d38152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:48.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.824+0000 7f9d40200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d380ab3c0 con 0x7f9d380aa420 2024-03-09T09:01:48.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.824+0000 7f9d40200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d380aa8a0 con 0x7f9d3806dbe0 2024-03-09T09:01:48.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.824+0000 7f9d40200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380c82d0 con 0x7f9d380c30f0 2024-03-09T09:01:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.824+0000 7f9d3ce00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f9d380aa420 legacy=0x7f9d381490f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:01:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.825+0000 7f9d3d800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d380c30f0 0x7f9d38152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.825+0000 7f9d3d800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d380c30f0 0x7f9d38152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:39330/0 (socket says 172.21.15.67:39330) 2024-03-09T09:01:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.825+0000 7f9d3d800640 1 -- 172.21.15.67:0/1028909056 learned_addr learned my addr 172.21.15.67:0/1028909056 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.825+0000 7f9d3d800640 1 -- 172.21.15.67:0/1028909056 >> v1:172.21.15.134:6789/0 conn(0x7f9d3806dbe0 legacy=0x7f9d38072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.825+0000 7f9d3d800640 1 -- 172.21.15.67:0/1028909056 >> v1:172.21.15.67:6789/0 conn(0x7f9d380aa420 legacy=0x7f9d381490f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:48.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.825+0000 7f9d3d800640 1 -- 172.21.15.67:0/1028909056 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380aab20 con 0x7f9d380c30f0 2024-03-09T09:01:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.827+0000 7f9d3d800640 1 --2- 172.21.15.67:0/1028909056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d380c30f0 0x7f9d38152a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0098a0 tx=0x7f9d2c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=135f9bd0065ecfa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.827+0000 7f9d37e00640 1 -- 172.21.15.67:0/1028909056 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c035020 con 0x7f9d380c30f0 2024-03-09T09:01:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.827+0000 7f9d37e00640 1 -- 172.21.15.67:0/1028909056 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c031640 con 0x7f9d380c30f0 2024-03-09T09:01:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.827+0000 7f9d37e00640 1 -- 172.21.15.67:0/1028909056 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c02d040 con 0x7f9d380c30f0 2024-03-09T09:01:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.828+0000 7f9d40200640 1 -- 172.21.15.67:0/1028909056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d380c30f0 msgr2=0x7f9d38152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.828+0000 7f9d40200640 1 --2- 172.21.15.67:0/1028909056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d380c30f0 0x7f9d38152a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0098a0 tx=0x7f9d2c02b8f0 comp rx=0 tx=0).stop 2024-03-09T09:01:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.828+0000 7f9d40200640 1 -- 172.21.15.67:0/1028909056 shutdown_connections 2024-03-09T09:01:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.828+0000 7f9d40200640 1 --2- 172.21.15.67:0/1028909056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d380c30f0 0x7f9d38152a20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.828+0000 7f9d40200640 1 -- 172.21.15.67:0/1028909056 >> 172.21.15.67:0/1028909056 conn(0x7f9d380c7370 msgr2=0x7f9d380c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.828+0000 7f9d40200640 1 -- 172.21.15.67:0/1028909056 shutdown_connections 2024-03-09T09:01:48.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.828+0000 7f9d40200640 1 -- 172.21.15.67:0/1028909056 wait complete. 2024-03-09T09:01:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.830+0000 7f9d40200640 1 Processor -- start 2024-03-09T09:01:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.830+0000 7f9d40200640 1 -- start start 2024-03-09T09:01:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d40200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d3806dbe0 0x7f9d380ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d40200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d380aa420 0x7f9d38146480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:48.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d40200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380c30f0 0x7f9d381469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:48.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d40200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3806d6c0 con 0x7f9d380aa420 2024-03-09T09:01:48.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d40200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380ab0e0 con 0x7f9d380c30f0 2024-03-09T09:01:48.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d40200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38149800 con 0x7f9d3806dbe0 2024-03-09T09:01:48.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d3ce00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d380aa420 msgr2=0x7f9d38146480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:01:48.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d3ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d380aa420 0x7f9d38146480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d3d800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d3806dbe0 0x7f9d380ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d3e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380c30f0 0x7f9d381469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d3d800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d3806dbe0 0x7f9d380ac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:39346/0 (socket says 172.21.15.67:39346) 2024-03-09T09:01:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.832+0000 7f9d3d800640 1 -- 172.21.15.67:0/2555060960 learned_addr learned my addr 172.21.15.67:0/2555060960 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.833+0000 7f9d3e200640 1 -- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d3806dbe0 msgr2=0x7f9d380ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.833+0000 7f9d3e200640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d3806dbe0 0x7f9d380ac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.833+0000 7f9d3e200640 1 -- 172.21.15.67:0/2555060960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d380aa420 msgr2=0x7f9d38146480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:01:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.833+0000 7f9d3e200640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d380aa420 0x7f9d38146480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:48.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.833+0000 7f9d3e200640 1 -- 172.21.15.67:0/2555060960 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d38146f00 con 0x7f9d380c30f0 2024-03-09T09:01:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.834+0000 7f9d3e200640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380c30f0 0x7f9d381469c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000c700 tx=0x7f9d3000cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.834+0000 7f9d35600640 1 -- 172.21.15.67:0/2555060960 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30003b70 con 0x7f9d380c30f0 2024-03-09T09:01:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.834+0000 7f9d35600640 1 -- 172.21.15.67:0/2555060960 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d30003d10 con 0x7f9d380c30f0 2024-03-09T09:01:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.834+0000 7f9d35600640 1 -- 172.21.15.67:0/2555060960 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30005310 con 0x7f9d380c30f0 2024-03-09T09:01:48.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.834+0000 7f9d40200640 1 -- 172.21.15.67:0/2555060960 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d381ef910 con 0x7f9d380c30f0 2024-03-09T09:01:48.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.834+0000 7f9d40200640 1 -- 172.21.15.67:0/2555060960 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3814c5c0 con 0x7f9d380c30f0 2024-03-09T09:01:48.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.835+0000 7f9d40200640 1 -- 172.21.15.67:0/2555060960 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d38075a00 con 0x7f9d380c30f0 2024-03-09T09:01:48.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.837+0000 7f9d35600640 1 -- 172.21.15.67:0/2555060960 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9d300054b0 con 0x7f9d380c30f0 2024-03-09T09:01:48.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.838+0000 7f9d35600640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d1806b480 0x7f9d1806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:48.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.838+0000 7f9d35600640 1 -- 172.21.15.67:0/2555060960 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9d3008ad20 con 0x7f9d380c30f0 2024-03-09T09:01:48.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.838+0000 7f9d3d800640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d1806b480 0x7f9d1806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:48.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.840+0000 7f9d3d800640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d1806b480 0x7f9d1806d940 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0097f0 tx=0x7f9d2c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:48.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:48.854+0000 7f9d35600640 1 -- 172.21.15.67:0/2555060960 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9d30039030 con 0x7f9d380c30f0 2024-03-09T09:01:49.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.131+0000 7f9d40200640 1 -- 172.21.15.67:0/2555060960 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9d380c6cf0 con 0x7f9d380c30f0 2024-03-09T09:01:49.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.132+0000 7f9d35600640 1 -- 172.21.15.67:0/2555060960 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f9d3005f070 con 0x7f9d380c30f0 2024-03-09T09:01:49.133 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":72,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:01:49.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.134+0000 7f9d40200640 1 -- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d1806b480 msgr2=0x7f9d1806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:49.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.134+0000 7f9d40200640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d1806b480 0x7f9d1806d940 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0097f0 tx=0x7f9d2c03c040 comp rx=0 tx=0).stop 2024-03-09T09:01:49.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.134+0000 7f9d40200640 1 -- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380c30f0 msgr2=0x7f9d381469c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:49.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.134+0000 7f9d40200640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380c30f0 0x7f9d381469c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000c700 tx=0x7f9d3000cbc0 comp rx=0 tx=0).stop 2024-03-09T09:01:49.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.135+0000 7f9d40200640 1 -- 172.21.15.67:0/2555060960 shutdown_connections 2024-03-09T09:01:49.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.135+0000 7f9d40200640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d1806b480 0x7f9d1806d940 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:49.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.135+0000 7f9d40200640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d380c30f0 0x7f9d381469c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:49.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.135+0000 7f9d40200640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d380aa420 0x7f9d38146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:49.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.135+0000 7f9d40200640 1 --2- 172.21.15.67:0/2555060960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d3806dbe0 0x7f9d380ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:49.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.135+0000 7f9d40200640 1 -- 172.21.15.67:0/2555060960 >> 172.21.15.67:0/2555060960 conn(0x7f9d380c7370 msgr2=0x7f9d380ab620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:49.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.135+0000 7f9d40200640 1 -- 172.21.15.67:0/2555060960 shutdown_connections 2024-03-09T09:01:49.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:49.135+0000 7f9d40200640 1 -- 172.21.15.67:0/2555060960 wait complete. 2024-03-09T09:01:52.158 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:01:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.397+0000 7f12ca000640 1 Processor -- start 2024-03-09T09:01:52.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.397+0000 7f12ca000640 1 -- start start 2024-03-09T09:01:52.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.399+0000 7f12ca000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f12c4153370 0x7f12c40c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:52.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.399+0000 7f12ca000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12c40c2400 0x7f12c40acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:52.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.399+0000 7f12ca000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c4156d50 con 0x7f12c40c1d70 2024-03-09T09:01:52.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.399+0000 7f12ca000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c40aadf0 con 0x7f12c40c2400 2024-03-09T09:01:52.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.399+0000 7f12ca000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c40aa4c0 con 0x7f12c4153370 2024-03-09T09:01:52.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.399+0000 7f12c3e00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f12c40c2400 msgr2=0x7f12c40acd20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:01:52.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.399+0000 7f12c3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12c40c2400 0x7f12c40acd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:52.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.400+0000 7f12c3400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f12c4153370 0x7f12c40c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:52.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.400+0000 7f12c3400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f12c4153370 0x7f12c40c07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:37192/0 (socket says 172.21.15.67:37192) 2024-03-09T09:01:52.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.400+0000 7f12c3400640 1 -- 172.21.15.67:0/1799460993 learned_addr learned my addr 172.21.15.67:0/1799460993 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:52.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.400+0000 7f12c3400640 1 -- 172.21.15.67:0/1799460993 >> v2:172.21.15.67:3300/0 conn(0x7f12c40c2400 msgr2=0x7f12c40acd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:52.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.400+0000 7f12c3400640 1 --2- 172.21.15.67:0/1799460993 >> v2:172.21.15.67:3300/0 conn(0x7f12c40c2400 0x7f12c40acd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:52.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.400+0000 7f12c3400640 1 -- 172.21.15.67:0/1799460993 >> v1:172.21.15.134:6789/0 conn(0x7f12c40c1d70 legacy=0x7f12c40ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:52.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.400+0000 7f12c3400640 1 -- 172.21.15.67:0/1799460993 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c40aa7a0 con 0x7f12c4153370 2024-03-09T09:01:52.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.402+0000 7f12c3400640 1 --2- 172.21.15.67:0/1799460993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c4153370 0x7f12c40c07a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f12c4070340 tx=0x7f12b402bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b03ddb4d991ccedf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:52.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.402+0000 7f12c2a00640 1 -- 172.21.15.67:0/1799460993 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b402caa0 con 0x7f12c4153370 2024-03-09T09:01:52.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.402+0000 7f12c2a00640 1 -- 172.21.15.67:0/1799460993 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b402cc40 con 0x7f12c4153370 2024-03-09T09:01:52.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.403+0000 7f12c2a00640 1 -- 172.21.15.67:0/1799460993 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b40343f0 con 0x7f12c4153370 2024-03-09T09:01:52.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.403+0000 7f12ca000640 1 -- 172.21.15.67:0/1799460993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c4153370 msgr2=0x7f12c40c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:52.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.403+0000 7f12ca000640 1 --2- 172.21.15.67:0/1799460993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c4153370 0x7f12c40c07a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f12c4070340 tx=0x7f12b402bf00 comp rx=0 tx=0).stop 2024-03-09T09:01:52.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.403+0000 7f12ca000640 1 -- 172.21.15.67:0/1799460993 shutdown_connections 2024-03-09T09:01:52.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.403+0000 7f12ca000640 1 --2- 172.21.15.67:0/1799460993 >> v2:172.21.15.67:3300/0 conn(0x7f12c40c2400 0x7f12c40acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:52.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.403+0000 7f12ca000640 1 --2- 172.21.15.67:0/1799460993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c4153370 0x7f12c40c07a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:52.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.403+0000 7f12ca000640 1 -- 172.21.15.67:0/1799460993 >> 172.21.15.67:0/1799460993 conn(0x7f12c40c7370 msgr2=0x7f12c40c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:52.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.403+0000 7f12ca000640 1 -- 172.21.15.67:0/1799460993 shutdown_connections 2024-03-09T09:01:52.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.403+0000 7f12ca000640 1 -- 172.21.15.67:0/1799460993 wait complete. 2024-03-09T09:01:52.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.405+0000 7f12ca000640 1 Processor -- start 2024-03-09T09:01:52.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.405+0000 7f12ca000640 1 -- start start 2024-03-09T09:01:52.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.406+0000 7f12ca000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c40c1d70 0x7f12c4156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:52.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.406+0000 7f12ca000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12c40c2400 0x7f12c41ed980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:52.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.406+0000 7f12ca000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4153370 0x7f12c41edec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:52.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.406+0000 7f12ca000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4153750 con 0x7f12c4153370 2024-03-09T09:01:52.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.406+0000 7f12ca000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c40ad260 con 0x7f12c40c1d70 2024-03-09T09:01:52.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.406+0000 7f12ca000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c41538d0 con 0x7f12c40c2400 2024-03-09T09:01:52.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.406+0000 7f12c3e00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4153370 msgr2=0x7f12c41edec0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:01:52.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.406+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4153370 0x7f12c41edec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:52.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.407+0000 7f12c3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c40c1d70 0x7f12c4156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:52.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.407+0000 7f12c3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c40c1d70 0x7f12c4156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:37208/0 (socket says 172.21.15.67:37208) 2024-03-09T09:01:52.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.407+0000 7f12c3400640 1 -- 172.21.15.67:0/2274356131 learned_addr learned my addr 172.21.15.67:0/2274356131 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:52.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.407+0000 7f12baa00640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12c40c2400 0x7f12c41ed980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:52.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.407+0000 7f12c3400640 1 -- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12c40c2400 msgr2=0x7f12c41ed980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:52.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.407+0000 7f12c3400640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12c40c2400 0x7f12c41ed980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:52.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.407+0000 7f12c3400640 1 -- 172.21.15.67:0/2274356131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4153370 msgr2=0x7f12c41edec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:01:52.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.407+0000 7f12c3400640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4153370 0x7f12c41edec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:52.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.407+0000 7f12c3400640 1 -- 172.21.15.67:0/2274356131 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4151910 con 0x7f12c40c1d70 2024-03-09T09:01:52.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.408+0000 7f12c3400640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c40c1d70 0x7f12c4156140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12b402c510 tx=0x7f12b4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:52.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.408+0000 7f12bbe00640 1 -- 172.21.15.67:0/2274356131 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4043030 con 0x7f12c40c1d70 2024-03-09T09:01:52.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.408+0000 7f12ca000640 1 -- 172.21.15.67:0/2274356131 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c41ecea0 con 0x7f12c40c1d70 2024-03-09T09:01:52.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.408+0000 7f12bbe00640 1 -- 172.21.15.67:0/2274356131 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b403e450 con 0x7f12c40c1d70 2024-03-09T09:01:52.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.408+0000 7f12bbe00640 1 -- 172.21.15.67:0/2274356131 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4009510 con 0x7f12c40c1d70 2024-03-09T09:01:52.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.408+0000 7f12ca000640 1 -- 172.21.15.67:0/2274356131 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c4156680 con 0x7f12c40c1d70 2024-03-09T09:01:52.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.409+0000 7f12ca000640 1 -- 172.21.15.67:0/2274356131 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c4075a00 con 0x7f12c40c1d70 2024-03-09T09:01:52.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.412+0000 7f12bbe00640 1 -- 172.21.15.67:0/2274356131 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f12b4046070 con 0x7f12c40c1d70 2024-03-09T09:01:52.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.412+0000 7f12bbe00640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a006b480 0x7f12a006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:52.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.413+0000 7f12bbe00640 1 -- 172.21.15.67:0/2274356131 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f12b40ac900 con 0x7f12c40c1d70 2024-03-09T09:01:52.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.413+0000 7f12baa00640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a006b480 0x7f12a006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:52.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.414+0000 7f12baa00640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a006b480 0x7f12a006d940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f12c40c36e0 tx=0x7f12ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:01:52.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.433+0000 7f12bbe00640 1 -- 172.21.15.67:0/2274356131 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f12b407b2f0 con 0x7f12c40c1d70 2024-03-09T09:01:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.872+0000 7f12ca000640 1 -- 172.21.15.67:0/2274356131 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f12c40c1530 con 0x7f12c40c1d70 2024-03-09T09:01:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.873+0000 7f12bbe00640 1 -- 172.21.15.67:0/2274356131 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f12b407abf0 con 0x7f12c40c1d70 2024-03-09T09:01:52.875 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":72,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:01:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 -- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a006b480 msgr2=0x7f12a006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a006b480 0x7f12a006d940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f12c40c36e0 tx=0x7f12ac009040 comp rx=0 tx=0).stop 2024-03-09T09:01:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 -- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c40c1d70 msgr2=0x7f12c4156140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:01:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c40c1d70 0x7f12c4156140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12b402c510 tx=0x7f12b4004a40 comp rx=0 tx=0).stop 2024-03-09T09:01:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 -- 172.21.15.67:0/2274356131 shutdown_connections 2024-03-09T09:01:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a006b480 0x7f12a006d940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4153370 0x7f12c41edec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12c40c2400 0x7f12c41ed980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 --2- 172.21.15.67:0/2274356131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c40c1d70 0x7f12c4156140 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 -- 172.21.15.67:0/2274356131 >> 172.21.15.67:0/2274356131 conn(0x7f12c40c7370 msgr2=0x7f12c40ac530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:01:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 -- 172.21.15.67:0/2274356131 shutdown_connections 2024-03-09T09:01:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:52.876+0000 7f12ca000640 1 -- 172.21.15.67:0/2274356131 wait complete. 2024-03-09T09:01:55.896 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:01:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.121+0000 7f8f17e00640 1 Processor -- start 2024-03-09T09:01:56.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.121+0000 7f8f17e00640 1 -- start start 2024-03-09T09:01:56.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.122+0000 7f8f17e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:56.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.122+0000 7f8f17e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:56.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.122+0000 7f8f17e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f18150280 con 0x7f8f180c2df0 2024-03-09T09:01:56.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.122+0000 7f8f17e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f18156080 con 0x7f8f1814fea0 2024-03-09T09:01:56.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.122+0000 7f8f17e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1806f350 con 0x7f8f18152860 2024-03-09T09:01:56.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.123+0000 7f8f16000640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f8f180c2df0 legacy=0x7f8f18156510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:01:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.123+0000 7f8f17400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.123+0000 7f8f17400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:50256/0 (socket says 172.21.15.67:50256) 2024-03-09T09:01:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.123+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 learned_addr learned my addr 172.21.15.67:0/3703807856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:01:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.123+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.124+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8f1815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:01:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.124+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8f1815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:56.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.124+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:56.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.124+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:56.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.124+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8f180c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:01:56.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.124+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8f180c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:56.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.124+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:56.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.124+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:56.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.324+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v1:172.21.15.67:6789/0 conn(0x7f8f180c2df0 legacy=0x7f8f18156510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:01:56.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.325+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:56.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.326+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:56.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.326+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8f1815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:01:56.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.326+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8f1815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:56.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.326+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:56.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.326+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:01:56.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.326+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8f180c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:01:56.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.326+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8f180c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:56.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.326+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:56.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.326+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:01:56.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.725+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v1:172.21.15.67:6789/0 conn(0x7f8f180c2df0 legacy=0x7f8f18156510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:01:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.727+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.728+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8f1815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:01:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.728+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8f1815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.728+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.728+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:01:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.728+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:56.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.729+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8f180c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:01:56.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.729+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8f180c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:56.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.729+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:56.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:56.729+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:01:57.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:57.526+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v1:172.21.15.67:6789/0 conn(0x7f8f180c2df0 legacy=0x7f8f18156510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:01:57.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:57.530+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:57.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:57.530+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8f1815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:01:57.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:57.530+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8f1815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:57.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:57.530+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:57.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:57.530+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:01:57.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:57.531+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:01:57.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:57.531+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8f180c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:01:57.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:57.531+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8f180c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:01:57.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:57.531+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:01:57.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:57.531+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:01:59.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.123+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8f1815d1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:59.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.123+0000 7f8f0fe00640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8f1815d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:59.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.123+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8f180c8380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:59.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.123+0000 7f8f0fe00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8f180c8380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:01:59.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.123+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 >> v1:172.21.15.67:6789/0 conn(0x7f8f180c2df0 legacy=0x7f8f18156510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:01:59.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.123+0000 7f8f0fe00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8007320 0x7f8ef80097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:01:59.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.123+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1806f630 con 0x7f8ef8003690 2024-03-09T09:01:59.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.123+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f18156080 con 0x7f8ef8007320 2024-03-09T09:01:59.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.123+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8007320 msgr2=0x7f8ef80097e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:01:59.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.123+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8007320 0x7f8ef80097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:01:59.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.324+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8007320 msgr2=0x7f8ef80097e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:01:59.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.324+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8007320 0x7f8ef80097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:01:59.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.725+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8007320 msgr2=0x7f8ef80097e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:01:59.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:01:59.725+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8007320 0x7f8ef80097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:02:00.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:00.526+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8007320 msgr2=0x7f8ef80097e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:02:00.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:00.526+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8007320 0x7f8ef80097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:02:02.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.123+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8007320 msgr2=0x7f8ef80097e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:02.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.123+0000 7f8f0fe00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8007320 0x7f8ef80097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:02.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.123+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 >> v1:172.21.15.134:6789/0 conn(0x7f8ef8003690 legacy=0x7f8ef8005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:02.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.123+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 reap_dead start 2024-03-09T09:02:02.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.123+0000 7f8f0fe00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:02.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.123+0000 7f8f0fe00640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:02.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.123+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f18150280 con 0x7f8ef8009fb0 2024-03-09T09:02:02.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.124+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f18156080 con 0x7f8f1814fea0 2024-03-09T09:02:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.124+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1806f350 con 0x7f8f18152860 2024-03-09T09:02:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.124+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v1:172.21.15.67:6789/0 conn(0x7f8ef8009fb0 legacy=0x7f8ef800a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:02:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.124+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:02.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.124+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:02.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.125+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8ef8010040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:02:02.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.125+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8ef8010040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:02:02.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.125+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:02.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.125+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8ef8012590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:02:02.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.125+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8ef8012590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:02:02.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.125+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:02.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.125+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:02.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.125+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:02.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.325+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v1:172.21.15.67:6789/0 conn(0x7f8ef8009fb0 legacy=0x7f8ef800a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:02:02.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.327+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:02.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.327+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:02.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.327+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8ef8010040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:02:02.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.327+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8ef8010040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:02:02.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.327+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8ef8012590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:02:02.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.327+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:02.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.327+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8ef8012590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:02:02.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.327+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:02.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.327+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:02:02.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.327+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:02:02.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.726+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v1:172.21.15.67:6789/0 conn(0x7f8ef8009fb0 legacy=0x7f8ef800a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:02:02.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.729+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:02.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.729+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:02.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.729+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8ef8012590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:02:02.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.729+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8ef8012590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:02:02.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.729+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:02.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.730+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:02:02.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.730+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8ef8010040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:02:02.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.730+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8ef8010040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:02:02.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.730+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:02.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:02.730+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:02:03.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:03.527+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v1:172.21.15.67:6789/0 conn(0x7f8ef8009fb0 legacy=0x7f8ef800a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:02:03.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:03.531+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:03.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:03.531+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:03.532+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8ef8010040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:02:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:03.532+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8ef8010040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:02:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:03.532+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:03.532+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8ef8012590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:02:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:03.532+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8ef8012590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:02:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:03.532+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:03.532+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:02:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:03.532+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:02:05.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.124+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8ef8012590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:05.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.124+0000 7f8f0fe00640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8ef8012590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.124+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 msgr2=0x7f8ef8010040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:05.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.124+0000 7f8f0fe00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.134:3300/0 conn(0x7f8f1814fea0 0x7f8ef8010040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.124+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 >> v1:172.21.15.67:6789/0 conn(0x7f8ef8009fb0 legacy=0x7f8ef800a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:05.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.124+0000 7f8f0fe00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8006220 0x7f8ef80097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.124+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef800ac00 con 0x7f8ef8006700 2024-03-09T09:02:05.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.124+0000 7f8f0fe00640 1 -- 172.21.15.67:0/3703807856 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f18156080 con 0x7f8ef8006220 2024-03-09T09:02:05.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.124+0000 7f8f16a00640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8006220 msgr2=0x7f8ef80097e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:02:05.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.124+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8006220 0x7f8ef80097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:05.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106454162 0 0) 0x7f8ef800ac00 con 0x7f8ef8006700 2024-03-09T09:02:05.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8efc003520 con 0x7f8ef8006700 2024-03-09T09:02:05.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3147387781 0 0) 0x7f8efc003520 con 0x7f8ef8006700 2024-03-09T09:02:05.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8006220 msgr2=0x7f8ef80097e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:05.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 --2- 172.21.15.67:0/3703807856 >> v2:172.21.15.67:3300/0 conn(0x7f8ef8006220 0x7f8ef80097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f18150280 con 0x7f8ef8006700 2024-03-09T09:02:05.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f8f04007860 con 0x7f8ef8006700 2024-03-09T09:02:05.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 >> v1:172.21.15.134:6789/0 conn(0x7f8ef8006700 legacy=0x7f8ef8005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 reap_dead start 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc003a20 0x7f8efc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8efc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8efc004e60 0x7f8efc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f18156080 con 0x7f8efc004e60 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc005620 con 0x7f8efc003a20 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.125+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc00bf00 con 0x7f8f18152860 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.126+0000 7f8f17400640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8efc004e60 msgr2=0x7f8efc00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.126+0000 7f8f17400640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8efc004e60 0x7f8efc00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.126+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8efc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.126+0000 7f8f16a00640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc003a20 0x7f8efc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.126+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc003a20 msgr2=0x7f8efc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:05.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.126+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc003a20 0x7f8efc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.126+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8efc004e60 msgr2=0x7f8efc00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:05.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.126+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8efc004e60 0x7f8efc00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.126+0000 7f8f16000640 1 -- 172.21.15.67:0/3703807856 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc003520 con 0x7f8f18152860 2024-03-09T09:02:05.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.127+0000 7f8f16000640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8efc004800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f04003980 tx=0x7f8efc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c15f67bbb93e1d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.128+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04010510 con 0x7f8f18152860 2024-03-09T09:02:05.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.128+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0400d040 con 0x7f8f18152860 2024-03-09T09:02:05.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.128+0000 7f8f15600640 1 -- 172.21.15.67:0/3703807856 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04030040 con 0x7f8f18152860 2024-03-09T09:02:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.128+0000 7f8f17e00640 1 -- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 msgr2=0x7f8efc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.128+0000 7f8f17e00640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8efc004800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f04003980 tx=0x7f8efc005830 comp rx=0 tx=0).stop 2024-03-09T09:02:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.129+0000 7f8f17e00640 1 -- 172.21.15.67:0/3703807856 shutdown_connections 2024-03-09T09:02:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.129+0000 7f8f17e00640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8efc004e60 0x7f8efc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.129+0000 7f8f17e00640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f18152860 0x7f8efc004800 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.129+0000 7f8f17e00640 1 --2- 172.21.15.67:0/3703807856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc003a20 0x7f8efc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.129+0000 7f8f17e00640 1 -- 172.21.15.67:0/3703807856 >> 172.21.15.67:0/3703807856 conn(0x7f8f1806d670 msgr2=0x7f8f1806da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.129+0000 7f8f17e00640 1 -- 172.21.15.67:0/3703807856 shutdown_connections 2024-03-09T09:02:05.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.129+0000 7f8f17e00640 1 -- 172.21.15.67:0/3703807856 wait complete. 2024-03-09T09:02:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.131+0000 7f8f17e00640 1 Processor -- start 2024-03-09T09:02:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.131+0000 7f8f17e00640 1 -- start start 2024-03-09T09:02:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f17e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8efc003a20 0x7f8f181542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f17e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc004e60 0x7f8f181eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f17e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f18152860 0x7f8f180c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f17e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180ab620 con 0x7f8f18152860 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f17e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180ab4a0 con 0x7f8efc004e60 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f17e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180ab7a0 con 0x7f8efc003a20 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f17400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f18152860 msgr2=0x7f8f180c7ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f17400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f18152860 0x7f8f180c7ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f16a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8efc003a20 0x7f8f181542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f16a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8efc003a20 0x7f8f181542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:48286/0 (socket says 172.21.15.67:48286) 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f16a00640 1 -- 172.21.15.67:0/674549032 learned_addr learned my addr 172.21.15.67:0/674549032 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.132+0000 7f8f16000640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc004e60 0x7f8f181eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.133+0000 7f8f16a00640 1 -- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc004e60 msgr2=0x7f8f181eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.133+0000 7f8f16a00640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc004e60 0x7f8f181eb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.133+0000 7f8f16a00640 1 -- 172.21.15.67:0/674549032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f18152860 msgr2=0x7f8f180c7ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:02:05.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.133+0000 7f8f16a00640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f18152860 0x7f8f180c7ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.133+0000 7f8f16a00640 1 -- 172.21.15.67:0/674549032 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f181ebc10 con 0x7f8efc003a20 2024-03-09T09:02:05.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.133+0000 7f8f16a00640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8efc003a20 0x7f8f181542b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f08011ee0 tx=0x7f8f080039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.133+0000 7f8f0ea00640 1 -- 172.21.15.67:0/674549032 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f08015770 con 0x7f8efc003a20 2024-03-09T09:02:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.133+0000 7f8f0ea00640 1 -- 172.21.15.67:0/674549032 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f08015930 con 0x7f8efc003a20 2024-03-09T09:02:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.133+0000 7f8f17e00640 1 -- 172.21.15.67:0/674549032 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f180c1f00 con 0x7f8efc003a20 2024-03-09T09:02:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.133+0000 7f8f0ea00640 1 -- 172.21.15.67:0/674549032 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f08015d20 con 0x7f8efc003a20 2024-03-09T09:02:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.133+0000 7f8f17e00640 1 -- 172.21.15.67:0/674549032 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f180c24c0 con 0x7f8efc003a20 2024-03-09T09:02:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.135+0000 7f8f17e00640 1 -- 172.21.15.67:0/674549032 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1804f3c0 con 0x7f8efc003a20 2024-03-09T09:02:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.135+0000 7f8f0ea00640 1 -- 172.21.15.67:0/674549032 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8f0801f020 con 0x7f8efc003a20 2024-03-09T09:02:05.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.137+0000 7f8f0ea00640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ef806c6e0 0x7f8ef806eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.137+0000 7f8f0ea00640 1 -- 172.21.15.67:0/674549032 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8f0808af10 con 0x7f8efc003a20 2024-03-09T09:02:05.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.137+0000 7f8f16000640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ef806c6e0 0x7f8ef806eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.138+0000 7f8f16000640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ef806c6e0 0x7f8ef806eba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8f04039000 tx=0x7f8f0400e040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.141+0000 7f8f0ea00640 1 -- 172.21.15.67:0/674549032 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8f08026060 con 0x7f8efc003a20 2024-03-09T09:02:05.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.420+0000 7f8f17e00640 1 -- 172.21.15.67:0/674549032 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8f1806ef90 con 0x7f8efc003a20 2024-03-09T09:02:05.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.420+0000 7f8f0ea00640 1 -- 172.21.15.67:0/674549032 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7f8f08059340 con 0x7f8efc003a20 2024-03-09T09:02:05.422 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":76,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 -- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ef806c6e0 msgr2=0x7f8ef806eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ef806c6e0 0x7f8ef806eba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8f04039000 tx=0x7f8f0400e040 comp rx=0 tx=0).stop 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 -- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8efc003a20 msgr2=0x7f8f181542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8efc003a20 0x7f8f181542b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f08011ee0 tx=0x7f8f080039e0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 -- 172.21.15.67:0/674549032 shutdown_connections 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ef806c6e0 0x7f8ef806eba0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f18152860 0x7f8f180c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8efc004e60 0x7f8f181eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 --2- 172.21.15.67:0/674549032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8efc003a20 0x7f8f181542b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 -- 172.21.15.67:0/674549032 >> 172.21.15.67:0/674549032 conn(0x7f8f1806d670 msgr2=0x7f8f180729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 -- 172.21.15.67:0/674549032 shutdown_connections 2024-03-09T09:02:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.423+0000 7f8f17e00640 1 -- 172.21.15.67:0/674549032 wait complete. 2024-03-09T09:02:05.441 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:02:05.441 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:02:05.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:02:05.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.652+0000 7f76fbe00640 1 Processor -- start 2024-03-09T09:02:05.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.652+0000 7f76fbe00640 1 -- start start 2024-03-09T09:02:05.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.653+0000 7f76fbe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76fc0c1c20 0x7f76fc155d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.653+0000 7f76fbe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f76fc0738b0 0x7f76fc156290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.653+0000 7f76fbe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76fc06de50 con 0x7f76fc153200 2024-03-09T09:02:05.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.653+0000 7f76fbe00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76fc1535e0 con 0x7f76fc0738b0 2024-03-09T09:02:05.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.653+0000 7f76fbe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc159120 con 0x7f76fc0c1c20 2024-03-09T09:02:05.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.654+0000 7f76fa000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76fc0c1c20 0x7f76fc155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.654+0000 7f76fa000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76fc0c1c20 0x7f76fc155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:48292/0 (socket says 172.21.15.67:48292) 2024-03-09T09:02:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.654+0000 7f76fa000640 1 -- 172.21.15.67:0/1051509807 learned_addr learned my addr 172.21.15.67:0/1051509807 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.654+0000 7f76fb400640 1 --2- 172.21.15.67:0/1051509807 >> v2:172.21.15.134:3300/0 conn(0x7f76fc0738b0 0x7f76fc156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.654+0000 7f76fb400640 1 -- 172.21.15.67:0/1051509807 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76fc0c1c20 msgr2=0x7f76fc155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.654+0000 7f76fb400640 1 --2- 172.21.15.67:0/1051509807 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76fc0c1c20 0x7f76fc155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.654+0000 7f76fb400640 1 -- 172.21.15.67:0/1051509807 >> v1:172.21.15.134:6789/0 conn(0x7f76fc153200 legacy=0x7f76fc0aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.654+0000 7f76fb400640 1 -- 172.21.15.67:0/1051509807 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc159370 con 0x7f76fc0738b0 2024-03-09T09:02:05.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.655+0000 7f76fb400640 1 --2- 172.21.15.67:0/1051509807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0738b0 0x7f76fc156290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009d90 tx=0x7f76f402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6998182d5a57fa36 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.656+0000 7f76f9600640 1 -- 172.21.15.67:0/1051509807 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4039070 con 0x7f76fc0738b0 2024-03-09T09:02:05.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.656+0000 7f76f9600640 1 -- 172.21.15.67:0/1051509807 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f402cbc0 con 0x7f76fc0738b0 2024-03-09T09:02:05.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.656+0000 7f76f9600640 1 -- 172.21.15.67:0/1051509807 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f4034440 con 0x7f76fc0738b0 2024-03-09T09:02:05.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.656+0000 7f76fbe00640 1 -- 172.21.15.67:0/1051509807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0738b0 msgr2=0x7f76fc156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.656+0000 7f76fbe00640 1 --2- 172.21.15.67:0/1051509807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0738b0 0x7f76fc156290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009d90 tx=0x7f76f402be10 comp rx=0 tx=0).stop 2024-03-09T09:02:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.657+0000 7f76fbe00640 1 -- 172.21.15.67:0/1051509807 shutdown_connections 2024-03-09T09:02:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.657+0000 7f76fbe00640 1 --2- 172.21.15.67:0/1051509807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0738b0 0x7f76fc156290 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.657+0000 7f76fbe00640 1 --2- 172.21.15.67:0/1051509807 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76fc0c1c20 0x7f76fc155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.657+0000 7f76fbe00640 1 -- 172.21.15.67:0/1051509807 >> 172.21.15.67:0/1051509807 conn(0x7f76fc0c7220 msgr2=0x7f76fc0c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.657+0000 7f76fbe00640 1 -- 172.21.15.67:0/1051509807 shutdown_connections 2024-03-09T09:02:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.657+0000 7f76fbe00640 1 -- 172.21.15.67:0/1051509807 wait complete. 2024-03-09T09:02:05.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.658+0000 7f76fbe00640 1 Processor -- start 2024-03-09T09:02:05.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.658+0000 7f76fbe00640 1 -- start start 2024-03-09T09:02:05.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.659+0000 7f76fbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0738b0 0x7f76fc158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.659+0000 7f76fbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76fc0c1c20 0x7f76fc1517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.659+0000 7f76fbe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76fc153200 0x7f76fc1464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.659+0000 7f76fbe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc159930 con 0x7f76fc0c1c20 2024-03-09T09:02:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.659+0000 7f76fbe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc159f80 con 0x7f76fc0738b0 2024-03-09T09:02:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.659+0000 7f76fbe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc159c10 con 0x7f76fc153200 2024-03-09T09:02:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.659+0000 7f76fa000640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76fc0c1c20 msgr2=0x7f76fc1517b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:02:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.660+0000 7f76fa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76fc0c1c20 0x7f76fc1517b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.660+0000 7f76faa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0738b0 0x7f76fc158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.660+0000 7f76faa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0738b0 0x7f76fc158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:47016/0 (socket says 172.21.15.67:47016) 2024-03-09T09:02:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.660+0000 7f76faa00640 1 -- 172.21.15.67:0/1522618115 learned_addr learned my addr 172.21.15.67:0/1522618115 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.660+0000 7f76fb400640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76fc153200 0x7f76fc1464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.660+0000 7f76faa00640 1 -- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76fc153200 msgr2=0x7f76fc1464c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.660+0000 7f76faa00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76fc153200 0x7f76fc1464c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.660+0000 7f76faa00640 1 -- 172.21.15.67:0/1522618115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76fc0c1c20 msgr2=0x7f76fc1517b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:02:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.660+0000 7f76faa00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76fc0c1c20 0x7f76fc1517b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.660+0000 7f76faa00640 1 -- 172.21.15.67:0/1522618115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc157c10 con 0x7f76fc0738b0 2024-03-09T09:02:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.661+0000 7f76faa00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0738b0 0x7f76fc158810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00c3f0 tx=0x7f76ec00c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.661+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec0052d0 con 0x7f76fc0738b0 2024-03-09T09:02:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.661+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec005490 con 0x7f76fc0738b0 2024-03-09T09:02:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.661+0000 7f76fbe00640 1 -- 172.21.15.67:0/1522618115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76fc150990 con 0x7f76fc0738b0 2024-03-09T09:02:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.661+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec007260 con 0x7f76fc0738b0 2024-03-09T09:02:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.661+0000 7f76fbe00640 1 -- 172.21.15.67:0/1522618115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76fc1576f0 con 0x7f76fc0738b0 2024-03-09T09:02:05.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.662+0000 7f76fbe00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc1eca20 0x7f76fc146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.662+0000 7f76fbe00640 1 -- 172.21.15.67:0/1522618115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc0aace0 con 0x7f76fc1eca20 2024-03-09T09:02:05.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.663+0000 7f76fa000640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc1eca20 0x7f76fc146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.663+0000 7f76fa000640 1 -- 172.21.15.67:0/1522618115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76fc04f3c0 con 0x7f76fc1eca20 2024-03-09T09:02:05.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.663+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f76ec007400 con 0x7f76fc0738b0 2024-03-09T09:02:05.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.663+0000 7f76fa000640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc1eca20 0x7f76fc146a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76fc153d50 tx=0x7f76e800ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.664+0000 7f76f2a00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f76dc06b480 0x7f76dc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.664+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f76ec089a30 con 0x7f76fc0738b0 2024-03-09T09:02:05.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.665+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e8017070 con 0x7f76fc1eca20 2024-03-09T09:02:05.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.665+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f76e8004020 con 0x7f76fc1eca20 2024-03-09T09:02:05.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.665+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc1eca20 msgr2=0x7f76fc146a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.665+0000 7f76f2a00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc1eca20 0x7f76fc146a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76fc153d50 tx=0x7f76e800ec70 comp rx=0 tx=0).stop 2024-03-09T09:02:05.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.665+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc1eca20 msgr2=0x7f76fc146a00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:05.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.665+0000 7f76f2a00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc1eca20 0x7f76fc146a00 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76fc153d50 tx=0x7f76e800ec70 comp rx=0 tx=0).stop 2024-03-09T09:02:05.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.665+0000 7f76fb400640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f76dc06b480 0x7f76dc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.666+0000 7f76fb400640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f76dc06b480 0x7f76dc06d940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f76f4036000 tx=0x7f76f4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.672+0000 7f76fbe00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0c5580 0x7f76fc0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.672+0000 7f76fbe00640 1 -- 172.21.15.67:0/1522618115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc14c750 con 0x7f76fc0c5580 2024-03-09T09:02:05.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.672+0000 7f76fa000640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0c5580 0x7f76fc0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.673+0000 7f76fa000640 1 -- 172.21.15.67:0/1522618115 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f76fc0c3b00 con 0x7f76fc0c5580 2024-03-09T09:02:05.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.673+0000 7f76fa000640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0c5580 0x7f76fc0c5980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0c1aa0 tx=0x7f76e8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.674+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e8017040 con 0x7f76fc0c5580 2024-03-09T09:02:05.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.674+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5177 (secure 0 0 0) 0x7f76e8003820 con 0x7f76fc0c5580 2024-03-09T09:02:05.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.674+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0c5580 msgr2=0x7f76fc0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.674+0000 7f76f2a00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0c5580 0x7f76fc0c5980 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0c1aa0 tx=0x7f76e8004730 comp rx=0 tx=0).stop 2024-03-09T09:02:05.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.674+0000 7f76f2a00640 1 -- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0c5580 msgr2=0x7f76fc0c5980 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:05.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.674+0000 7f76f2a00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0c5580 0x7f76fc0c5980 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0c1aa0 tx=0x7f76e8004730 comp rx=0 tx=0).stop 2024-03-09T09:02:05.676 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 76, 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:02:05.677 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.678 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:02:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:05.681 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.682 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:02:05.682 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.682 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:02:05.682 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.682 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:02:05.682 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:02:05.682 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:05.682 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:05.682 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.683 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:05.683 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:02:05.683 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.683 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.683 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:05.683 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:02:05.683 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.683 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.683 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.684 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:02:05.684 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:02:05.684 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:05.684 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:05.684 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:05.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.684 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.684 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:02:05.684 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:02:05.684 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:05.685 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:05.685 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.685 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:05.685 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:02:05.685 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.685 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.686 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.687 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.688 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.689 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:05.690 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:02:05.691 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:02:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.675+0000 7f76fbe00640 1 -- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f76dc06b480 msgr2=0x7f76dc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.675+0000 7f76fbe00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f76dc06b480 0x7f76dc06d940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f76f4036000 tx=0x7f76f4030000 comp rx=0 tx=0).stop 2024-03-09T09:02:05.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.675+0000 7f76fbe00640 1 -- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0738b0 msgr2=0x7f76fc158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.675+0000 7f76fbe00640 1 --2- 172.21.15.67:0/1522618115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76fc0738b0 0x7f76fc158810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00c3f0 tx=0x7f76ec00c8b0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.675+0000 7f76fb400640 1 -- 172.21.15.67:0/1522618115 reap_dead start 2024-03-09T09:02:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.675+0000 7f76fb400640 1 -- 172.21.15.67:0/1522618115 reap_dead start 2024-03-09T09:02:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.675+0000 7f76fbe00640 1 -- 172.21.15.67:0/1522618115 shutdown_connections 2024-03-09T09:02:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.675+0000 7f76fbe00640 1 -- 172.21.15.67:0/1522618115 >> 172.21.15.67:0/1522618115 conn(0x7f76fc0c7220 msgr2=0x7f76fc0c27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.675+0000 7f76fbe00640 1 -- 172.21.15.67:0/1522618115 shutdown_connections 2024-03-09T09:02:05.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.675+0000 7f76fbe00640 1 -- 172.21.15.67:0/1522618115 wait complete. 2024-03-09T09:02:05.700 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:02:05.700 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:02:05.701 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:02:05.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.917+0000 7f8d7dc00640 1 Processor -- start 2024-03-09T09:02:05.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.917+0000 7f8d7dc00640 1 -- start start 2024-03-09T09:02:05.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.918+0000 7f8d7dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8d780c2fa0 0x7f8d7806d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.918+0000 7f8d7dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78152790 0x7f8d7814fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.919+0000 7f8d7dc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d78150400 con 0x7f8d7814ff70 2024-03-09T09:02:05.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.919+0000 7f8d7dc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d78071b20 con 0x7f8d780c2fa0 2024-03-09T09:02:05.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.919+0000 7f8d7dc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7806d940 con 0x7f8d78152790 2024-03-09T09:02:05.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.919+0000 7f8d77e00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f8d780c2fa0 msgr2=0x7f8d7806d340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:02:05.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.919+0000 7f8d77e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8d780c2fa0 0x7f8d7806d340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:05.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.919+0000 7f8d77400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78152790 0x7f8d7814fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.919+0000 7f8d77400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78152790 0x7f8d7814fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:48308/0 (socket says 172.21.15.67:48308) 2024-03-09T09:02:05.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.919+0000 7f8d77400640 1 -- 172.21.15.67:0/101153155 learned_addr learned my addr 172.21.15.67:0/101153155 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.920+0000 7f8d77400640 1 -- 172.21.15.67:0/101153155 >> v2:172.21.15.67:3300/0 conn(0x7f8d780c2fa0 msgr2=0x7f8d7806d340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.920+0000 7f8d77400640 1 --2- 172.21.15.67:0/101153155 >> v2:172.21.15.67:3300/0 conn(0x7f8d780c2fa0 0x7f8d7806d340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.920+0000 7f8d77400640 1 -- 172.21.15.67:0/101153155 >> v1:172.21.15.134:6789/0 conn(0x7f8d7814ff70 legacy=0x7f8d78071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.920+0000 7f8d77400640 1 -- 172.21.15.67:0/101153155 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7806dc20 con 0x7f8d78152790 2024-03-09T09:02:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.920+0000 7f8d76a00640 1 -- 172.21.15.67:0/101153155 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606851995 0 0) 0x7f8d78150400 con 0x7f8d7814ff70 2024-03-09T09:02:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.921+0000 7f8d77400640 1 --2- 172.21.15.67:0/101153155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78152790 0x7f8d7814fa30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009e80 tx=0x7f8d6802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beb459b96a0eaced server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.921+0000 7f8d76a00640 1 -- 172.21.15.67:0/101153155 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6802ca70 con 0x7f8d78152790 2024-03-09T09:02:05.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.921+0000 7f8d76a00640 1 -- 172.21.15.67:0/101153155 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6802cc10 con 0x7f8d78152790 2024-03-09T09:02:05.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.921+0000 7f8d76a00640 1 -- 172.21.15.67:0/101153155 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d680343f0 con 0x7f8d78152790 2024-03-09T09:02:05.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.922+0000 7f8d7dc00640 1 -- 172.21.15.67:0/101153155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78152790 msgr2=0x7f8d7814fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.922+0000 7f8d7dc00640 1 --2- 172.21.15.67:0/101153155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78152790 0x7f8d7814fa30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009e80 tx=0x7f8d6802bed0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.922+0000 7f8d7dc00640 1 -- 172.21.15.67:0/101153155 shutdown_connections 2024-03-09T09:02:05.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.922+0000 7f8d7dc00640 1 --2- 172.21.15.67:0/101153155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78152790 0x7f8d7814fa30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.922+0000 7f8d7dc00640 1 --2- 172.21.15.67:0/101153155 >> v2:172.21.15.67:3300/0 conn(0x7f8d780c2fa0 0x7f8d7806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.922+0000 7f8d7dc00640 1 -- 172.21.15.67:0/101153155 >> 172.21.15.67:0/101153155 conn(0x7f8d780c7220 msgr2=0x7f8d780c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:05.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.922+0000 7f8d7dc00640 1 -- 172.21.15.67:0/101153155 shutdown_connections 2024-03-09T09:02:05.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.922+0000 7f8d7dc00640 1 -- 172.21.15.67:0/101153155 wait complete. 2024-03-09T09:02:05.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.924+0000 7f8d7dc00640 1 Processor -- start 2024-03-09T09:02:05.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.924+0000 7f8d7dc00640 1 -- start start 2024-03-09T09:02:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.925+0000 7f8d7dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780c2fa0 0x7f8d780aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.925+0000 7f8d7dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d7814ff70 0x7f8d780ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.925+0000 7f8d7dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d78152790 0x7f8d780acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.925+0000 7f8d7dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780711c0 con 0x7f8d7814ff70 2024-03-09T09:02:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.925+0000 7f8d7dc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7806e1e0 con 0x7f8d78152790 2024-03-09T09:02:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.925+0000 7f8d7dc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78071340 con 0x7f8d780c2fa0 2024-03-09T09:02:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.925+0000 7f8d77400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d7814ff70 msgr2=0x7f8d780ac9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:02:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.925+0000 7f8d77400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d7814ff70 0x7f8d780ac9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.926+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780c2fa0 0x7f8d780aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.926+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780c2fa0 0x7f8d780aa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:48310/0 (socket says 172.21.15.67:48310) 2024-03-09T09:02:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.926+0000 7f8d77e00640 1 -- 172.21.15.67:0/2469541376 learned_addr learned my addr 172.21.15.67:0/2469541376 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.926+0000 7f8d7d200640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d78152790 0x7f8d780acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.926+0000 7f8d77e00640 1 -- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d78152790 msgr2=0x7f8d780acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.926+0000 7f8d77e00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d78152790 0x7f8d780acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.926+0000 7f8d77e00640 1 -- 172.21.15.67:0/2469541376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d7814ff70 msgr2=0x7f8d780ac9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.926+0000 7f8d77e00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d7814ff70 0x7f8d780ac9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.926+0000 7f8d77e00640 1 -- 172.21.15.67:0/2469541376 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d78154a10 con 0x7f8d780c2fa0 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.926+0000 7f8d7d200640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d78152790 0x7f8d780acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.927+0000 7f8d77e00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780c2fa0 0x7f8d780aa500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000b550 tx=0x7f8d6000ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.927+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60003dc0 con 0x7f8d780c2fa0 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.927+0000 7f8d7dc00640 1 -- 172.21.15.67:0/2469541376 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d781ef260 con 0x7f8d780c2fa0 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.927+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d60015070 con 0x7f8d780c2fa0 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.927+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600052b0 con 0x7f8d780c2fa0 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.927+0000 7f8d7dc00640 1 -- 172.21.15.67:0/2469541376 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d780c4eb0 con 0x7f8d780c2fa0 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.928+0000 7f8d7dc00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780aaa40 0x7f8d780c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.928+0000 7f8d7dc00640 1 -- 172.21.15.67:0/2469541376 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7804f3c0 con 0x7f8d780aaa40 2024-03-09T09:02:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.929+0000 7f8d77400640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780aaa40 0x7f8d780c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.930+0000 7f8d77400640 1 -- 172.21.15.67:0/2469541376 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d780c7970 con 0x7f8d780aaa40 2024-03-09T09:02:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.930+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8d60005450 con 0x7f8d780c2fa0 2024-03-09T09:02:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.930+0000 7f8d77400640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780aaa40 0x7f8d780c5250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009e50 tx=0x7f8d68004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.931+0000 7f8d6fe00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d5406b4b0 0x7f8d5406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.931+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8d6008a3b0 con 0x7f8d780c2fa0 2024-03-09T09:02:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.931+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68043070 con 0x7f8d780aaa40 2024-03-09T09:02:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.931+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f8d68009af0 con 0x7f8d780aaa40 2024-03-09T09:02:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.931+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780aaa40 msgr2=0x7f8d780c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.931+0000 7f8d6fe00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780aaa40 0x7f8d780c5250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009e50 tx=0x7f8d68004a40 comp rx=0 tx=0).stop 2024-03-09T09:02:05.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.931+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780aaa40 msgr2=0x7f8d780c5250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:05.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.931+0000 7f8d6fe00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780aaa40 0x7f8d780c5250 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009e50 tx=0x7f8d68004a40 comp rx=0 tx=0).stop 2024-03-09T09:02:05.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.931+0000 7f8d7d200640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d5406b4b0 0x7f8d5406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.933+0000 7f8d7d200640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d5406b4b0 0x7f8d5406d970 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8d7814c230 tx=0x7f8d70009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.938+0000 7f8d7dc00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78148ac0 0x7f8d780c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:05.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.938+0000 7f8d7dc00640 1 -- 172.21.15.67:0/2469541376 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78067b30 con 0x7f8d78148ac0 2024-03-09T09:02:05.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.939+0000 7f8d77400640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78148ac0 0x7f8d780c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:05.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.940+0000 7f8d77400640 1 -- 172.21.15.67:0/2469541376 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8d780c7c50 con 0x7f8d78148ac0 2024-03-09T09:02:05.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.940+0000 7f8d77400640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78148ac0 0x7f8d780c5990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8d680093e0 tx=0x7f8d68009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:05.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.941+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68004750 con 0x7f8d78148ac0 2024-03-09T09:02:05.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.941+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5175 (secure 0 0 0) 0x7f8d68004750 con 0x7f8d78148ac0 2024-03-09T09:02:05.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.941+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78148ac0 msgr2=0x7f8d780c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.941+0000 7f8d6fe00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78148ac0 0x7f8d780c5990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8d680093e0 tx=0x7f8d68009810 comp rx=0 tx=0).stop 2024-03-09T09:02:05.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.941+0000 7f8d6fe00640 1 -- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78148ac0 msgr2=0x7f8d780c5990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:05.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.941+0000 7f8d6fe00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78148ac0 0x7f8d780c5990 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8d680093e0 tx=0x7f8d68009810 comp rx=0 tx=0).stop 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 76, 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:02:05.944 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:02:05.945 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:02:05.947 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:05.948 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:05.950 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:05.951 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:02:05.952 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.953 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:02:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:02:05.957 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:02:05.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.942+0000 7f8d7dc00640 1 -- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d5406b4b0 msgr2=0x7f8d5406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.942+0000 7f8d7dc00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d5406b4b0 0x7f8d5406d970 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8d7814c230 tx=0x7f8d70009040 comp rx=0 tx=0).stop 2024-03-09T09:02:05.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.942+0000 7f8d7dc00640 1 -- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780c2fa0 msgr2=0x7f8d780aa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:05.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.942+0000 7f8d7dc00640 1 --2- 172.21.15.67:0/2469541376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d780c2fa0 0x7f8d780aa500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000b550 tx=0x7f8d6000ba10 comp rx=0 tx=0).stop 2024-03-09T09:02:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.942+0000 7f8d7d200640 1 -- 172.21.15.67:0/2469541376 reap_dead start 2024-03-09T09:02:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.942+0000 7f8d7d200640 1 -- 172.21.15.67:0/2469541376 reap_dead start 2024-03-09T09:02:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.942+0000 7f8d7dc00640 1 -- 172.21.15.67:0/2469541376 shutdown_connections 2024-03-09T09:02:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.942+0000 7f8d7dc00640 1 -- 172.21.15.67:0/2469541376 >> 172.21.15.67:0/2469541376 conn(0x7f8d780c7220 msgr2=0x7f8d780c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.942+0000 7f8d7dc00640 1 -- 172.21.15.67:0/2469541376 shutdown_connections 2024-03-09T09:02:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:05.942+0000 7f8d7dc00640 1 -- 172.21.15.67:0/2469541376 wait complete. 2024-03-09T09:02:05.960 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:02:05.960 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:02:06.401 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 1% (242/12340) Updating files: 2% (247/12340) Updating files: 2% (292/12340) Updating files: 2% (359/12340) Updating files: 3% (371/12340) Updating files: 4% (494/12340) Updating files: 4% (552/12340) Updating files: 5% (617/12340) Updating files: 5% (637/12340) Updating files: 6% (741/12340) Updating files: 6% (761/12340) Updating files: 6% (843/12340) Updating files: 7% (864/12340) Updating files: 7% (912/12340) Updating files: 8% (988/12340) Updating files: 8% (1010/12340) Updating files: 9% (1111/12340) Updating files: 9% (1134/12340) Updating files: 9% (1224/12340) Updating files: 10% (1234/12340) Updating files: 11% (1358/12340) Updating files: 11% (1376/12340) Updating files: 11% (1454/12340) Updating files: 11% (1463/12340) Updating files: 11% (1470/12340) Updating files: 12% (1481/12340) Updating files: 12% (1556/12340) Updating files: 13% (1605/12340) Updating files: 13% (1639/12340) Updating files: 13% (1668/12340) Updating files: 14% (1728/12340) Updating files: 14% (1736/12340) Updating files: 14% (1846/12340) Updating files: 15% (1851/12340) Updating files: 16% (1975/12340) Updating files: 16% (2093/12340) Updating files: 17% (2098/12340) Updating files: 17% (2137/12340) Updating files: 18% (2222/12340) Updating files: 18% (2225/12340) Updating files: 18% (2243/12340) Updating files: 19% (2345/12340) Updating files: 19% (2396/12340) Updating files: 19% (2449/12340) Updating files: 20% (2468/12340) Updating files: 20% (2516/12340) Updating files: 21% (2592/12340) Updating files: 21% (2670/12340) Updating files: 21% (2683/12340) Updating files: 22% (2715/12340) Updating files: 22% (2734/12340) Updating files: 22% (2784/12340) Updating files: 23% (2839/12340) Updating files: 23% (2857/12340) Updating files: 23% (2922/12340) Updating files: 24% (2962/12340) Updating files: 24% (3012/12340) Updating files: 25% (3085/12340) Updating files: 25% (3178/12340) Updating files: 26% (3209/12340) Updating files: 26% (3261/12340) Updating files: 27% (3332/12340) Updating files: 28% (3456/12340) Updating files: 28% (3554/12340) Updating files: 29% (3579/12340) Updating files: 29% (3615/12340) Updating files: 30% (3702/12340) Updating files: 30% (3810/12340) Updating files: 31% (3826/12340) Updating files: 31% (3883/12340) Updating files: 31% (3936/12340) Updating files: 32% (3949/12340) Updating files: 32% (4036/12340) Updating files: 33% (4073/12340) Updating files: 34% (4196/12340) Updating files: 34% (4230/12340) Updating files: 34% (4280/12340) Updating files: 35% (4319/12340) Updating files: 35% (4346/12340) Updating files: 36% (4443/12340) Updating files: 37% (4566/12340) Updating files: 37% (4609/12340) Updating files: 38% (4690/12340) Updating files: 38% (4768/12340) Updating files: 39% (4813/12340) Updating files: 40% (4936/12340) Updating files: 40% (4943/12340) Updating files: 40% (4956/12340) Updating files: 40% (4957/12340) Updating files: 40% (4972/12340) Updating files: 41% (5060/12340) Updating files: 41% (5133/12340) Updating files: 42% (5183/12340) Updating files: 42% (5211/12340) Updating files: 42% (5218/12340) Updating files: 43% (5307/12340) Updating files: 43% (5349/12340) Updating files: 44% (5430/12340) Updating files: 45% (5553/12340) Updating files: 45% (5575/12340) Updating files: 45% (5594/12340) Updating files: 46% (5677/12340) Updating files: 47% (5800/12340) Updating files: 47% (5810/12340) Updating files: 47% (5873/12340) Updating files: 48% (5924/12340) Updating files: 49% (6047/12340) Updating files: 49% (6064/12340) Updating files: 50% (6170/12340) Updating files: 50% (6278/12340) Updating files: 51% (6294/12340) Updating files: 52% (6417/12340) Updating files: 52% (6425/12340) Updating files: 53% (6541/12340) Updating files: 54% (6664/12340) Updating files: 54% (6690/12340) Updating files: 55% (6787/12340) Updating files: 55% (6825/12340) Updating files: 56% (6911/12340) Updating files: 56% (6951/12340) Updating files: 57% (7034/12340) Updating files: 57% (7040/12340) Updating files: 57% (7152/12340) Updating files: 58% (7158/12340) Updating files: 58% (7221/12340) Updating files: 59% (7281/12340) Updating files: 59% (7344/12340) Updating files: 60% (7404/12340) Updating files: 61% (7528/12340) Updating files: 61% (7577/12340) Updating files: 62% (7651/12340) Updating files: 62% (7678/12340) Updating files: 62% (7728/12340) Updating files: 63% (7775/12340) Updating files: 63% (7789/12340) Updating files: 64% (7898/12340) Updating files: 64% (7930/12340) Updating files: 64% (7958/12340) Updating files: 64% (7973/12340) Updating files: 65% (8021/12340) Updating files: 65% (8076/12340) Updating files: 66% (8145/12340) Updating files: 67% (8268/12340) Updating files: 67% (8295/12340) Updating files: 68% (8392/12340) Updating files: 68% (8437/12340) Updating files: 68% (8512/12340) Updating files: 69% (8515/12340) Updating files: 69% (8615/12340) Updating files: 70% (8638/12340) Updating files: 70% (8761/12340) Updating files: 71% (8762/12340) Updating files: 71% (8797/12340) Updating files: 71% (8801/12340) Updating files: 71% (8820/12340) Updating files: 71% (8881/12340) Updating files: 72% (8885/12340) Updating files: 73% (9009/12340) Updating files: 73% (9128/12340) Updating files: 74% (9132/12340) Updating files: 75% (9255/12340) Updating files: 75% (9295/12340) Updating files: 76% (9379/12340) Updating files: 76% (9455/12340) Updating files: 77% (9502/12340) Updating files: 77% (9610/12340) Updating files: 78% (9626/12340) Updating files: 78% (9687/12340) Updating files: 79% (9749/12340) Updating files: 79% (9835/12340) Updating files: 80% (9872/12340) Updating files: 80% (9913/12340) Updating files: 81% (9996/12340) Updating files: 82% (10119/12340) Updating files: 82% (10127/12340) Updating files: 83% (10243/12340) Updating files: 84% (10366/12340) Updating files: 85% (10489/12340) Updating files: 85% (10587/12340) Updating files: 86% (10613/12340) Updating files: 86% (10642/12340) Updating files: 86% (10651/12340) Updating files: 87% (10736/12340) Updating files: 87% (10858/12340) Updating files: 88% (10860/12340) Updating files: 88% (10884/12340) Updating files: 89% (10983/12340) Updating files: 90% (11106/12340) Updating files: 90% (11109/12340) Updating files: 90% (11227/12340) Updating files: 91% (11230/12340) Updating files: 91% (11308/12340) Updating files: 92% (11353/12340) Updating files: 93% (11477/12340) Updating files: 93% (11565/12340) Updating files: 94% (11600/12340) Updating files: 94% (11626/12340) Updating files: 94% (11662/12340) Updating files: 94% (11700/12340) Updating files: 95% (11723/12340) Updating files: 96% (11847/12340) Updating files: 96% (11866/12340) Updating files: 96% (11892/12340) Updating files: 96% (11904/12340) Updating files: 96% (11963/12340) Updating files: 97% (11970/12340) Updating files: 98% (12094/12340) Updating files: 98% (12154/12340) Updating files: 98% (12175/12340) Updating files: 98% (12200/12340) Updating files: 99% (12217/12340) Updating files: 100% (12340/12340) Updating files: 100% (12340/12340), done. 2024-03-09T09:02:06.490 INFO:tasks.workunit.client.0.smithi067.stderr:+ trap - EXIT 2024-03-09T09:02:06.490 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ceph 2024-03-09T09:02:06.491 INFO:tasks.workunit.client.0.smithi067.stderr:++ seq 1 90 2024-03-09T09:02:06.493 INFO:tasks.workunit.client.0.smithi067.stderr:+ versions='1 2024-03-09T09:02:06.493 INFO:tasks.workunit.client.0.smithi067.stderr:2 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:3 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:4 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:5 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:6 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:7 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:8 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:9 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:10 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:11 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:12 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:13 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:14 2024-03-09T09:02:06.494 INFO:tasks.workunit.client.0.smithi067.stderr:15 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:16 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:17 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:18 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:19 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:20 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:21 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:22 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:23 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:24 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:25 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:26 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:27 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:28 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:29 2024-03-09T09:02:06.495 INFO:tasks.workunit.client.0.smithi067.stderr:30 2024-03-09T09:02:06.496 INFO:tasks.workunit.client.0.smithi067.stderr:31 2024-03-09T09:02:06.496 INFO:tasks.workunit.client.0.smithi067.stderr:32 2024-03-09T09:02:06.496 INFO:tasks.workunit.client.0.smithi067.stderr:33 2024-03-09T09:02:06.496 INFO:tasks.workunit.client.0.smithi067.stderr:34 2024-03-09T09:02:06.496 INFO:tasks.workunit.client.0.smithi067.stderr:35 2024-03-09T09:02:06.496 INFO:tasks.workunit.client.0.smithi067.stderr:36 2024-03-09T09:02:06.496 INFO:tasks.workunit.client.0.smithi067.stderr:37 2024-03-09T09:02:06.496 INFO:tasks.workunit.client.0.smithi067.stderr:38 2024-03-09T09:02:06.496 INFO:tasks.workunit.client.0.smithi067.stderr:39 2024-03-09T09:02:06.496 INFO:tasks.workunit.client.0.smithi067.stderr:40 2024-03-09T09:02:06.496 INFO:tasks.workunit.client.0.smithi067.stderr:41 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:42 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:43 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:44 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:45 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:46 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:47 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:48 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:49 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:50 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:51 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:52 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:53 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:54 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:55 2024-03-09T09:02:06.497 INFO:tasks.workunit.client.0.smithi067.stderr:56 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:57 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:58 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:59 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:60 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:61 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:62 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:63 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:64 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:65 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:66 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:67 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:68 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:69 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:70 2024-03-09T09:02:06.498 INFO:tasks.workunit.client.0.smithi067.stderr:71 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:72 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:73 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:74 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:75 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:76 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:77 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:78 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:79 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:80 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:81 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:82 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:83 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:84 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:85 2024-03-09T09:02:06.499 INFO:tasks.workunit.client.0.smithi067.stderr:86 2024-03-09T09:02:06.500 INFO:tasks.workunit.client.0.smithi067.stderr:87 2024-03-09T09:02:06.500 INFO:tasks.workunit.client.0.smithi067.stderr:88 2024-03-09T09:02:06.500 INFO:tasks.workunit.client.0.smithi067.stderr:89 2024-03-09T09:02:06.500 INFO:tasks.workunit.client.0.smithi067.stderr:90' 2024-03-09T09:02:06.500 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:02:06.500 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 48 ']' 2024-03-09T09:02:06.500 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.1 2024-03-09T09:02:06.500 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.1 2024-03-09T09:02:06.500 INFO:tasks.workunit.client.0.smithi067.stdout:v0.1 2024-03-09T09:02:06.501 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.1 2024-03-09T09:02:07.292 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 1% (242/12340) Updating files: 2% (247/12340) Updating files: 2% (292/12340) Updating files: 3% (371/12340) Updating files: 3% (466/12340) Updating files: 4% (494/12340) Updating files: 5% (617/12340) Updating files: 5% (693/12340) Updating files: 5% (728/12340) Updating files: 6% (741/12340) Updating files: 6% (760/12340) Updating files: 7% (864/12340) Updating files: 7% (870/12340) Updating files: 8% (988/12340) Updating files: 8% (1017/12340) Updating files: 8% (1031/12340) Updating files: 8% (1078/12340) Updating files: 9% (1111/12340) Updating files: 9% (1159/12340) Updating files: 9% (1204/12340) Updating files: 9% (1230/12340) Updating files: 10% (1234/12340) Updating files: 11% (1358/12340) Updating files: 11% (1390/12340) Updating files: 11% (1472/12340) Updating files: 12% (1481/12340) Updating files: 12% (1538/12340) Updating files: 13% (1605/12340) Updating files: 13% (1613/12340) Updating files: 14% (1728/12340) Updating files: 14% (1742/12340) Updating files: 14% (1771/12340) Updating files: 15% (1851/12340) Updating files: 15% (1894/12340) Updating files: 15% (1902/12340) Updating files: 16% (1975/12340) Updating files: 16% (1977/12340) Updating files: 16% (2060/12340) Updating files: 17% (2098/12340) Updating files: 17% (2132/12340) Updating files: 17% (2171/12340) Updating files: 17% (2180/12340) Updating files: 18% (2222/12340) Updating files: 18% (2232/12340) Updating files: 19% (2345/12340) Updating files: 19% (2434/12340) Updating files: 20% (2468/12340) Updating files: 20% (2516/12340) Updating files: 20% (2541/12340) Updating files: 20% (2580/12340) Updating files: 21% (2592/12340) Updating files: 21% (2713/12340) Updating files: 22% (2715/12340) Updating files: 22% (2771/12340) Updating files: 22% (2815/12340) Updating files: 23% (2839/12340) Updating files: 23% (2842/12340) Updating files: 24% (2962/12340) Updating files: 24% (3012/12340) Updating files: 25% (3085/12340) Updating files: 26% (3209/12340) Updating files: 26% (3232/12340) Updating files: 26% (3302/12340) Updating files: 26% (3307/12340) Updating files: 27% (3332/12340) Updating files: 27% (3428/12340) Updating files: 28% (3456/12340) Updating files: 28% (3507/12340) Updating files: 28% (3569/12340) Updating files: 29% (3579/12340) Updating files: 30% (3702/12340) Updating files: 30% (3777/12340) Updating files: 31% (3826/12340) Updating files: 31% (3883/12340) Updating files: 32% (3949/12340) Updating files: 32% (3962/12340) Updating files: 33% (4073/12340) Updating files: 33% (4115/12340) Updating files: 34% (4196/12340) Updating files: 34% (4210/12340) Updating files: 34% (4267/12340) Updating files: 35% (4319/12340) Updating files: 35% (4411/12340) Updating files: 36% (4443/12340) Updating files: 36% (4483/12340) Updating files: 37% (4566/12340) Updating files: 38% (4690/12340) Updating files: 38% (4698/12340) Updating files: 38% (4768/12340) Updating files: 38% (4778/12340) Updating files: 39% (4813/12340) Updating files: 39% (4916/12340) Updating files: 40% (4936/12340) Updating files: 40% (5023/12340) Updating files: 40% (5041/12340) Updating files: 40% (5057/12340) Updating files: 41% (5060/12340) Updating files: 41% (5107/12340) Updating files: 41% (5179/12340) Updating files: 42% (5183/12340) Updating files: 42% (5229/12340) Updating files: 43% (5307/12340) Updating files: 43% (5400/12340) Updating files: 44% (5430/12340) Updating files: 45% (5553/12340) Updating files: 45% (5589/12340) Updating files: 45% (5616/12340) Updating files: 46% (5677/12340) Updating files: 47% (5800/12340) Updating files: 47% (5810/12340) Updating files: 47% (5873/12340) Updating files: 48% (5924/12340) Updating files: 48% (5963/12340) Updating files: 48% (6000/12340) Updating files: 49% (6047/12340) Updating files: 49% (6132/12340) Updating files: 49% (6155/12340) Updating files: 50% (6170/12340) Updating files: 50% (6193/12340) Updating files: 51% (6294/12340) Updating files: 52% (6417/12340) Updating files: 52% (6425/12340) Updating files: 52% (6455/12340) Updating files: 53% (6541/12340) Updating files: 53% (6611/12340) Updating files: 53% (6657/12340) Updating files: 54% (6664/12340) Updating files: 55% (6787/12340) Updating files: 55% (6825/12340) Updating files: 56% (6911/12340) Updating files: 56% (6951/12340) Updating files: 57% (7034/12340) Updating files: 57% (7109/12340) Updating files: 58% (7158/12340) Updating files: 59% (7281/12340) Updating files: 59% (7344/12340) Updating files: 59% (7379/12340) Updating files: 60% (7404/12340) Updating files: 61% (7528/12340) Updating files: 61% (7549/12340) Updating files: 61% (7616/12340) Updating files: 62% (7651/12340) Updating files: 62% (7699/12340) Updating files: 62% (7770/12340) Updating files: 63% (7775/12340) Updating files: 63% (7793/12340) Updating files: 64% (7898/12340) Updating files: 65% (8021/12340) Updating files: 66% (8145/12340) Updating files: 66% (8155/12340) Updating files: 67% (8268/12340) Updating files: 67% (8327/12340) Updating files: 68% (8392/12340) Updating files: 68% (8453/12340) Updating files: 68% (8490/12340) Updating files: 69% (8515/12340) Updating files: 69% (8544/12340) Updating files: 69% (8587/12340) Updating files: 70% (8638/12340) Updating files: 70% (8651/12340) Updating files: 70% (8700/12340) Updating files: 70% (8734/12340) Updating files: 71% (8762/12340) Updating files: 71% (8844/12340) Updating files: 72% (8885/12340) Updating files: 72% (8923/12340) Updating files: 73% (9009/12340) Updating files: 73% (9128/12340) Updating files: 74% (9132/12340) Updating files: 74% (9191/12340) Updating files: 74% (9215/12340) Updating files: 75% (9255/12340) Updating files: 75% (9314/12340) Updating files: 76% (9379/12340) Updating files: 76% (9400/12340) Updating files: 76% (9455/12340) Updating files: 77% (9502/12340) Updating files: 77% (9529/12340) Updating files: 77% (9610/12340) Updating files: 78% (9626/12340) Updating files: 78% (9687/12340) Updating files: 79% (9749/12340) Updating files: 79% (9756/12340) Updating files: 80% (9872/12340) Updating files: 80% (9898/12340) Updating files: 81% (9996/12340) Updating files: 81% (10037/12340) Updating files: 82% (10119/12340) Updating files: 82% (10127/12340) Updating files: 83% (10243/12340) Updating files: 83% (10284/12340) Updating files: 84% (10366/12340) Updating files: 85% (10489/12340) Updating files: 85% (10587/12340) Updating files: 86% (10613/12340) Updating files: 87% (10736/12340) Updating files: 87% (10745/12340) Updating files: 87% (10749/12340) Updating files: 88% (10860/12340) Updating files: 88% (10927/12340) Updating files: 89% (10983/12340) Updating files: 90% (11106/12340) Updating files: 90% (11165/12340) Updating files: 91% (11230/12340) Updating files: 92% (11353/12340) Updating files: 92% (11357/12340) Updating files: 92% (11472/12340) Updating files: 93% (11477/12340) Updating files: 93% (11490/12340) Updating files: 94% (11600/12340) Updating files: 94% (11680/12340) Updating files: 95% (11723/12340) Updating files: 95% (11817/12340) Updating files: 96% (11847/12340) Updating files: 96% (11866/12340) Updating files: 96% (11967/12340) Updating files: 97% (11970/12340) Updating files: 97% (12029/12340) Updating files: 97% (12050/12340) Updating files: 98% (12094/12340) Updating files: 98% (12170/12340) Updating files: 99% (12217/12340) Updating files: 99% (12338/12340) Updating files: 100% (12340/12340) Updating files: 100% (12340/12340), done. 2024-03-09T09:02:07.388 INFO:tasks.workunit.client.1.smithi134.stderr:+ trap - EXIT 2024-03-09T09:02:07.389 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ceph 2024-03-09T09:02:07.390 INFO:tasks.workunit.client.1.smithi134.stderr:++ seq 1 90 2024-03-09T09:02:07.394 INFO:tasks.workunit.client.1.smithi134.stderr:+ versions='1 2024-03-09T09:02:07.394 INFO:tasks.workunit.client.1.smithi134.stderr:2 2024-03-09T09:02:07.394 INFO:tasks.workunit.client.1.smithi134.stderr:3 2024-03-09T09:02:07.394 INFO:tasks.workunit.client.1.smithi134.stderr:4 2024-03-09T09:02:07.394 INFO:tasks.workunit.client.1.smithi134.stderr:5 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:6 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:7 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:8 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:9 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:10 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:11 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:12 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:13 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:14 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:15 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:16 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:17 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:18 2024-03-09T09:02:07.395 INFO:tasks.workunit.client.1.smithi134.stderr:19 2024-03-09T09:02:07.396 INFO:tasks.workunit.client.1.smithi134.stderr:20 2024-03-09T09:02:07.396 INFO:tasks.workunit.client.1.smithi134.stderr:21 2024-03-09T09:02:07.396 INFO:tasks.workunit.client.1.smithi134.stderr:22 2024-03-09T09:02:07.396 INFO:tasks.workunit.client.1.smithi134.stderr:23 2024-03-09T09:02:07.396 INFO:tasks.workunit.client.1.smithi134.stderr:24 2024-03-09T09:02:07.396 INFO:tasks.workunit.client.1.smithi134.stderr:25 2024-03-09T09:02:07.396 INFO:tasks.workunit.client.1.smithi134.stderr:26 2024-03-09T09:02:07.396 INFO:tasks.workunit.client.1.smithi134.stderr:27 2024-03-09T09:02:07.396 INFO:tasks.workunit.client.1.smithi134.stderr:28 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:29 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:30 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:31 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:32 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:33 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:34 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:35 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:36 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:37 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:38 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:39 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:40 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:41 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:42 2024-03-09T09:02:07.397 INFO:tasks.workunit.client.1.smithi134.stderr:43 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:44 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:45 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:46 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:47 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:48 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:49 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:50 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:51 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:52 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:53 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:54 2024-03-09T09:02:07.398 INFO:tasks.workunit.client.1.smithi134.stderr:55 2024-03-09T09:02:07.399 INFO:tasks.workunit.client.1.smithi134.stderr:56 2024-03-09T09:02:07.399 INFO:tasks.workunit.client.1.smithi134.stderr:57 2024-03-09T09:02:07.399 INFO:tasks.workunit.client.1.smithi134.stderr:58 2024-03-09T09:02:07.399 INFO:tasks.workunit.client.1.smithi134.stderr:59 2024-03-09T09:02:07.399 INFO:tasks.workunit.client.1.smithi134.stderr:60 2024-03-09T09:02:07.399 INFO:tasks.workunit.client.1.smithi134.stderr:61 2024-03-09T09:02:07.399 INFO:tasks.workunit.client.1.smithi134.stderr:62 2024-03-09T09:02:07.399 INFO:tasks.workunit.client.1.smithi134.stderr:63 2024-03-09T09:02:07.399 INFO:tasks.workunit.client.1.smithi134.stderr:64 2024-03-09T09:02:07.399 INFO:tasks.workunit.client.1.smithi134.stderr:65 2024-03-09T09:02:07.399 INFO:tasks.workunit.client.1.smithi134.stderr:66 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:67 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:68 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:69 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:70 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:71 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:72 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:73 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:74 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:75 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:76 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:77 2024-03-09T09:02:07.400 INFO:tasks.workunit.client.1.smithi134.stderr:78 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:79 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:80 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:81 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:82 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:83 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:84 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:85 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:86 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:87 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:88 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:89 2024-03-09T09:02:07.401 INFO:tasks.workunit.client.1.smithi134.stderr:90' 2024-03-09T09:02:07.402 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:02:07.402 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 1 -eq 48 ']' 2024-03-09T09:02:07.402 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.1 2024-03-09T09:02:07.402 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.1 2024-03-09T09:02:07.402 INFO:tasks.workunit.client.1.smithi134.stdout:v0.1 2024-03-09T09:02:07.403 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.1 2024-03-09T09:02:13.896 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:02:13.903 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:02:25.961 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:02:25.961 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-03-09T09:02:25.961 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-09T09:02:25.961 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-09T09:02:25.970 INFO:tasks.ceph.mon.a:Started 2024-03-09T09:02:25.970 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:02:25.971 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:02:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.165+0000 7f9812800640 1 Processor -- start 2024-03-09T09:02:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.165+0000 7f9812800640 1 -- start start 2024-03-09T09:02:26.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.166+0000 7f9812800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f980c0ac730 0x7f980c071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:26.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.166+0000 7f9812800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f980c0c22a0 0x7f980c0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:26.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.166+0000 7f9812800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980c1581a0 con 0x7f980c157650 2024-03-09T09:02:26.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.166+0000 7f9812800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f980c070ee0 con 0x7f980c0ac730 2024-03-09T09:02:26.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.166+0000 7f9812800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f980c154d00 con 0x7f980c0c22a0 2024-03-09T09:02:26.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.166+0000 7f980be00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f980c157650 legacy=0x7f980c157a90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:02:26.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.166+0000 7f980b400640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f980c0ac730 msgr2=0x7f980c071190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:02:26.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.167+0000 7f980b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f980c0ac730 0x7f980c071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:26.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.167+0000 7f9803e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f980c0c22a0 0x7f980c0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:26.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.167+0000 7f9803e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f980c0c22a0 0x7f980c0acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:38242/0 (socket says 172.21.15.67:38242) 2024-03-09T09:02:26.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.167+0000 7f9803e00640 1 -- 172.21.15.67:0/1855988256 learned_addr learned my addr 172.21.15.67:0/1855988256 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:26.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.168+0000 7f9803e00640 1 -- 172.21.15.67:0/1855988256 >> v2:172.21.15.67:3300/0 conn(0x7f980c0ac730 msgr2=0x7f980c071190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:26.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.168+0000 7f9803e00640 1 --2- 172.21.15.67:0/1855988256 >> v2:172.21.15.67:3300/0 conn(0x7f980c0ac730 0x7f980c071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:26.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.168+0000 7f9803e00640 1 -- 172.21.15.67:0/1855988256 >> v1:172.21.15.67:6789/0 conn(0x7f980c157650 legacy=0x7f980c157a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:26.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.168+0000 7f9803e00640 1 -- 172.21.15.67:0/1855988256 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c153d40 con 0x7f980c0c22a0 2024-03-09T09:02:26.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.169+0000 7f9803e00640 1 --2- 172.21.15.67:0/1855988256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f980c0c22a0 0x7f980c0acb10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f980c0c1a30 tx=0x7f97f402b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd3191e44e362c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:26.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.169+0000 7f980aa00640 1 -- 172.21.15.67:0/1855988256 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f4035020 con 0x7f980c0c22a0 2024-03-09T09:02:26.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.170+0000 7f980aa00640 1 -- 172.21.15.67:0/1855988256 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f4031890 con 0x7f980c0c22a0 2024-03-09T09:02:26.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.170+0000 7f980aa00640 1 -- 172.21.15.67:0/1855988256 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f4031c60 con 0x7f980c0c22a0 2024-03-09T09:02:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.170+0000 7f9812800640 1 -- 172.21.15.67:0/1855988256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f980c0c22a0 msgr2=0x7f980c0acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.170+0000 7f9812800640 1 --2- 172.21.15.67:0/1855988256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f980c0c22a0 0x7f980c0acb10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f980c0c1a30 tx=0x7f97f402b930 comp rx=0 tx=0).stop 2024-03-09T09:02:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.170+0000 7f9812800640 1 -- 172.21.15.67:0/1855988256 shutdown_connections 2024-03-09T09:02:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.170+0000 7f9812800640 1 --2- 172.21.15.67:0/1855988256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f980c0c22a0 0x7f980c0acb10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.170+0000 7f9812800640 1 --2- 172.21.15.67:0/1855988256 >> v2:172.21.15.67:3300/0 conn(0x7f980c0ac730 0x7f980c071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.170+0000 7f9812800640 1 -- 172.21.15.67:0/1855988256 >> 172.21.15.67:0/1855988256 conn(0x7f980c0c7370 msgr2=0x7f980c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.171+0000 7f9812800640 1 -- 172.21.15.67:0/1855988256 shutdown_connections 2024-03-09T09:02:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.171+0000 7f9812800640 1 -- 172.21.15.67:0/1855988256 wait complete. 2024-03-09T09:02:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.172+0000 7f9812800640 1 Processor -- start 2024-03-09T09:02:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.173+0000 7f9812800640 1 -- start start 2024-03-09T09:02:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.174+0000 7f9812800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f980c0ac730 0x7f980c0c0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.174+0000 7f9812800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f980c0c22a0 0x7f980c150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.174+0000 7f9812800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f980c157650 0x7f980c0c1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.174+0000 7f9812800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c070c00 con 0x7f980c0c22a0 2024-03-09T09:02:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.174+0000 7f9812800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c070920 con 0x7f980c0ac730 2024-03-09T09:02:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.174+0000 7f9812800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c151610 con 0x7f980c157650 2024-03-09T09:02:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.174+0000 7f9803e00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f980c0c22a0 msgr2=0x7f980c150960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:02:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.174+0000 7f9803e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f980c0c22a0 0x7f980c150960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.174+0000 7f980be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f980c157650 0x7f980c0c1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.174+0000 7f980b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f980c0ac730 0x7f980c0c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.174+0000 7f980be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f980c157650 0x7f980c0c1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:53240/0 (socket says 172.21.15.67:53240) 2024-03-09T09:02:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.175+0000 7f980be00640 1 -- 172.21.15.67:0/191510967 learned_addr learned my addr 172.21.15.67:0/191510967 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:26.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.175+0000 7f980be00640 1 -- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f980c0ac730 msgr2=0x7f980c0c0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.175+0000 7f980be00640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f980c0ac730 0x7f980c0c0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.175+0000 7f980be00640 1 -- 172.21.15.67:0/191510967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f980c0c22a0 msgr2=0x7f980c150960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:02:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.175+0000 7f980be00640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f980c0c22a0 0x7f980c150960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.175+0000 7f980be00640 1 -- 172.21.15.67:0/191510967 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c1ea610 con 0x7f980c157650 2024-03-09T09:02:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.176+0000 7f980be00640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f980c157650 0x7f980c0c1100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f980400be50 tx=0x7f9804009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.176+0000 7f9803400640 1 -- 172.21.15.67:0/191510967 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804016430 con 0x7f980c157650 2024-03-09T09:02:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.176+0000 7f9803400640 1 -- 172.21.15.67:0/191510967 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9804010030 con 0x7f980c157650 2024-03-09T09:02:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.176+0000 7f9803400640 1 -- 172.21.15.67:0/191510967 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804016810 con 0x7f980c157650 2024-03-09T09:02:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.176+0000 7f9812800640 1 -- 172.21.15.67:0/191510967 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c0bcb30 con 0x7f980c157650 2024-03-09T09:02:26.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.176+0000 7f9812800640 1 -- 172.21.15.67:0/191510967 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c1ea8f0 con 0x7f980c157650 2024-03-09T09:02:26.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.179+0000 7f9812800640 1 -- 172.21.15.67:0/191510967 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980c151960 con 0x7f980c157650 2024-03-09T09:02:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.180+0000 7f9803400640 1 -- 172.21.15.67:0/191510967 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f98040153e0 con 0x7f980c157650 2024-03-09T09:02:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.181+0000 7f9803400640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f97e806f890 0x7f97e8071d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.181+0000 7f9803400640 1 -- 172.21.15.67:0/191510967 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f980408a9a0 con 0x7f980c157650 2024-03-09T09:02:26.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.181+0000 7f980b400640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f97e806f890 0x7f97e8071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:26.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.183+0000 7f980b400640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f97e806f890 0x7f97e8071d50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f980c1e03a0 tx=0x7f97fc009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:26.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.194+0000 7f9803400640 1 -- 172.21.15.67:0/191510967 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f980c151960 con 0x7f980c157650 2024-03-09T09:02:26.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.532+0000 7f9812800640 1 -- 172.21.15.67:0/191510967 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f980c075a00 con 0x7f980c157650 2024-03-09T09:02:26.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.533+0000 7f9803400640 1 -- 172.21.15.67:0/191510967 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f98040594c0 con 0x7f980c157650 2024-03-09T09:02:26.535 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":76,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:02:26.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 -- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f97e806f890 msgr2=0x7f97e8071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:26.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f97e806f890 0x7f97e8071d50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f980c1e03a0 tx=0x7f97fc009110 comp rx=0 tx=0).stop 2024-03-09T09:02:26.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 -- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f980c157650 msgr2=0x7f980c0c1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:26.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f980c157650 0x7f980c0c1100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f980400be50 tx=0x7f9804009f40 comp rx=0 tx=0).stop 2024-03-09T09:02:26.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 -- 172.21.15.67:0/191510967 shutdown_connections 2024-03-09T09:02:26.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f97e806f890 0x7f97e8071d50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:26.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f980c157650 0x7f980c0c1100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:26.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f980c0c22a0 0x7f980c150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:26.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 --2- 172.21.15.67:0/191510967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f980c0ac730 0x7f980c0c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:26.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 -- 172.21.15.67:0/191510967 >> 172.21.15.67:0/191510967 conn(0x7f980c0c7370 msgr2=0x7f980c157a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:26.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 -- 172.21.15.67:0/191510967 shutdown_connections 2024-03-09T09:02:26.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:26.535+0000 7f9812800640 1 -- 172.21.15.67:0/191510967 wait complete. 2024-03-09T09:02:29.556 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:02:29.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.786+0000 7f38c5600640 1 Processor -- start 2024-03-09T09:02:29.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.787+0000 7f38c5600640 1 -- start start 2024-03-09T09:02:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.789+0000 7f38c5600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38c00c22a0 0x7f38c0071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.789+0000 7f38c5600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c0153d40 con 0x7f38c00ac730 2024-03-09T09:02:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.789+0000 7f38c5600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c00c2790 con 0x7f38c0157d60 2024-03-09T09:02:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.789+0000 7f38c5600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0070ee0 con 0x7f38c00c22a0 2024-03-09T09:02:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.789+0000 7f38bea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f38c00ac730 0x7f38c0157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39272/0 (socket says 172.21.15.67:39272) 2024-03-09T09:02:29.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.789+0000 7f38bea00640 1 -- 172.21.15.67:0/1826631042 learned_addr learned my addr 172.21.15.67:0/1826631042 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:02:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.789+0000 7f38b7e00640 1 --2- 172.21.15.67:0/1826631042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38c00c22a0 0x7f38c0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.790+0000 7f38be000640 1 -- 172.21.15.67:0/1826631042 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3695139087 0 0) 0x7f38c00c2790 con 0x7f38c0157d60 2024-03-09T09:02:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.790+0000 7f38be000640 1 -- 172.21.15.67:0/1826631042 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38a4003520 con 0x7f38c0157d60 2024-03-09T09:02:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.790+0000 7f38b7e00640 1 -- 172.21.15.67:0/1826631042 >> v1:172.21.15.134:6789/0 conn(0x7f38c0157d60 legacy=0x7f38c015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.790+0000 7f38b7e00640 1 -- 172.21.15.67:0/1826631042 >> v1:172.21.15.67:6789/0 conn(0x7f38c00ac730 legacy=0x7f38c0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:29.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.790+0000 7f38b7e00640 1 -- 172.21.15.67:0/1826631042 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c0070920 con 0x7f38c00c22a0 2024-03-09T09:02:29.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.791+0000 7f38b7e00640 1 --2- 172.21.15.67:0/1826631042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38c00c22a0 0x7f38c0071190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38a80097d0 tx=0x7f38a802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9975e10baffe17 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:29.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.792+0000 7f38be000640 1 -- 172.21.15.67:0/1826631042 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8035020 con 0x7f38c00c22a0 2024-03-09T09:02:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.792+0000 7f38be000640 1 -- 172.21.15.67:0/1826631042 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a8031800 con 0x7f38c00c22a0 2024-03-09T09:02:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.792+0000 7f38be000640 1 -- 172.21.15.67:0/1826631042 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8031bb0 con 0x7f38c00c22a0 2024-03-09T09:02:29.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.793+0000 7f38c5600640 1 -- 172.21.15.67:0/1826631042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38c00c22a0 msgr2=0x7f38c0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.793+0000 7f38c5600640 1 --2- 172.21.15.67:0/1826631042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38c00c22a0 0x7f38c0071190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38a80097d0 tx=0x7f38a802b8a0 comp rx=0 tx=0).stop 2024-03-09T09:02:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.793+0000 7f38c5600640 1 -- 172.21.15.67:0/1826631042 shutdown_connections 2024-03-09T09:02:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.793+0000 7f38c5600640 1 --2- 172.21.15.67:0/1826631042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38c00c22a0 0x7f38c0071190 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.793+0000 7f38c5600640 1 -- 172.21.15.67:0/1826631042 >> 172.21.15.67:0/1826631042 conn(0x7f38c00c7370 msgr2=0x7f38c00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.793+0000 7f38c5600640 1 -- 172.21.15.67:0/1826631042 shutdown_connections 2024-03-09T09:02:29.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.793+0000 7f38c5600640 1 -- 172.21.15.67:0/1826631042 wait complete. 2024-03-09T09:02:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.794+0000 7f38c5600640 1 Processor -- start 2024-03-09T09:02:29.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.795+0000 7f38c5600640 1 -- start start 2024-03-09T09:02:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.795+0000 7f38c5600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c00ac730 0x7f38c01ef590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.795+0000 7f38c5600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38c00c22a0 0x7f38c00c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38c5600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38c0157d60 0x7f38c01ebd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38c5600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0151610 con 0x7f38c00c22a0 2024-03-09T09:02:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38c5600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0154d00 con 0x7f38c00ac730 2024-03-09T09:02:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38c5600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0151790 con 0x7f38c0157d60 2024-03-09T09:02:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38c00c22a0 0x7f38c00c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38c00c22a0 0x7f38c00c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44816/0 (socket says 172.21.15.67:44816) 2024-03-09T09:02:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38b7e00640 1 -- 172.21.15.67:0/3768926808 learned_addr learned my addr 172.21.15.67:0/3768926808 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38bea00640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c00ac730 0x7f38c01ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:29.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38b7e00640 1 -- 172.21.15.67:0/3768926808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38c00c22a0 msgr2=0x7f38c00c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:02:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38b7e00640 1 -- 172.21.15.67:0/3768926808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38c00c22a0 msgr2=0x7f38c00c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:02:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38b7e00640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38c00c22a0 0x7f38c00c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38b7e00640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38c00c22a0 0x7f38c00c0a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.796+0000 7f38bf400640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38c0157d60 0x7f38c01ebd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.797+0000 7f38bea00640 1 -- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38c0157d60 msgr2=0x7f38c01ebd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.797+0000 7f38bea00640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38c0157d60 0x7f38c01ebd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.797+0000 7f38bea00640 1 -- 172.21.15.67:0/3768926808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38c00c22a0 msgr2=0x7f38c00c0a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:02:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.797+0000 7f38bea00640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38c00c22a0 0x7f38c00c0a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.797+0000 7f38bea00640 1 -- 172.21.15.67:0/3768926808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c01efad0 con 0x7f38c00ac730 2024-03-09T09:02:29.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.797+0000 7f38bea00640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c00ac730 0x7f38c01ef590 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38b000c440 tx=0x7f38b000c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.797+0000 7f38b6a00640 1 -- 172.21.15.67:0/3768926808 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b0017070 con 0x7f38c00ac730 2024-03-09T09:02:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.797+0000 7f38b6a00640 1 -- 172.21.15.67:0/3768926808 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b0005ce0 con 0x7f38c00ac730 2024-03-09T09:02:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.797+0000 7f38b6a00640 1 -- 172.21.15.67:0/3768926808 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00072d0 con 0x7f38c00ac730 2024-03-09T09:02:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.798+0000 7f38c5600640 1 -- 172.21.15.67:0/3768926808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c01eb590 con 0x7f38c00ac730 2024-03-09T09:02:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.798+0000 7f38c5600640 1 -- 172.21.15.67:0/3768926808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c009c820 con 0x7f38c00ac730 2024-03-09T09:02:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.799+0000 7f38c5600640 1 -- 172.21.15.67:0/3768926808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c0075a00 con 0x7f38c00ac730 2024-03-09T09:02:29.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.800+0000 7f38b6a00640 1 -- 172.21.15.67:0/3768926808 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f38b001a070 con 0x7f38c00ac730 2024-03-09T09:02:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.802+0000 7f38b6a00640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f389c06b4d0 0x7f389c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.802+0000 7f38b6a00640 1 -- 172.21.15.67:0/3768926808 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f38b00899f0 con 0x7f38c00ac730 2024-03-09T09:02:29.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.802+0000 7f38b7e00640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f389c06b4d0 0x7f389c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:29.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.803+0000 7f38b7e00640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f389c06b4d0 0x7f389c06d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f38c00c3e90 tx=0x7f38a8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:29.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:29.815+0000 7f38b6a00640 1 -- 172.21.15.67:0/3768926808 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f38b0057ea0 con 0x7f38c00ac730 2024-03-09T09:02:30.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.098+0000 7f38c5600640 1 -- 172.21.15.67:0/3768926808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f38c00c1d70 con 0x7f38c00ac730 2024-03-09T09:02:30.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.099+0000 7f38b6a00640 1 -- 172.21.15.67:0/3768926808 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f38b005bf40 con 0x7f38c00ac730 2024-03-09T09:02:30.102 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":76,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:02:30.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.102+0000 7f38c5600640 1 -- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f389c06b4d0 msgr2=0x7f389c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:30.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.102+0000 7f38c5600640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f389c06b4d0 0x7f389c06d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f38c00c3e90 tx=0x7f38a8008030 comp rx=0 tx=0).stop 2024-03-09T09:02:30.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.102+0000 7f38c5600640 1 -- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c00ac730 msgr2=0x7f38c01ef590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:30.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.102+0000 7f38c5600640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c00ac730 0x7f38c01ef590 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38b000c440 tx=0x7f38b000c900 comp rx=0 tx=0).stop 2024-03-09T09:02:30.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.102+0000 7f38c5600640 1 -- 172.21.15.67:0/3768926808 shutdown_connections 2024-03-09T09:02:30.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.102+0000 7f38c5600640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f389c06b4d0 0x7f389c06d990 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:30.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.102+0000 7f38c5600640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38c0157d60 0x7f38c01ebd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:30.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.102+0000 7f38c5600640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38c00c22a0 0x7f38c00c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:30.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.102+0000 7f38c5600640 1 --2- 172.21.15.67:0/3768926808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c00ac730 0x7f38c01ef590 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:30.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.102+0000 7f38c5600640 1 -- 172.21.15.67:0/3768926808 >> 172.21.15.67:0/3768926808 conn(0x7f38c00c7370 msgr2=0x7f38c0153590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:30.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.103+0000 7f38c5600640 1 -- 172.21.15.67:0/3768926808 shutdown_connections 2024-03-09T09:02:30.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:30.103+0000 7f38c5600640 1 -- 172.21.15.67:0/3768926808 wait complete. 2024-03-09T09:02:33.132 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:02:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.345+0000 7fba8f200640 1 Processor -- start 2024-03-09T09:02:33.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.345+0000 7fba8f200640 1 -- start start 2024-03-09T09:02:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.347+0000 7fba8f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fba88153630 0x7fba880c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.347+0000 7fba8f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba88072d30 con 0x7fba880738c0 2024-03-09T09:02:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.347+0000 7fba8f200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba880c8630 con 0x7fba88070810 2024-03-09T09:02:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.347+0000 7fba8f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba88155df0 con 0x7fba88153630 2024-03-09T09:02:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.347+0000 7fba87400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fba880738c0 0x7fba881562f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39274/0 (socket says 172.21.15.67:39274) 2024-03-09T09:02:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.347+0000 7fba87400640 1 -- 172.21.15.67:0/2016170942 learned_addr learned my addr 172.21.15.67:0/2016170942 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:02:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.347+0000 7fba87e00640 1 --2- 172.21.15.67:0/2016170942 >> v2:172.21.15.67:3300/0 conn(0x7fba88153630 0x7fba880c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.348+0000 7fba87e00640 1 -- 172.21.15.67:0/2016170942 >> v2:172.21.15.67:3300/0 conn(0x7fba88153630 msgr2=0x7fba880c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:02:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.348+0000 7fba87e00640 1 -- 172.21.15.67:0/2016170942 >> v2:172.21.15.67:3300/0 conn(0x7fba88153630 msgr2=0x7fba880c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:02:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.348+0000 7fba87e00640 1 --2- 172.21.15.67:0/2016170942 >> v2:172.21.15.67:3300/0 conn(0x7fba88153630 0x7fba880c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.348+0000 7fba87e00640 1 --2- 172.21.15.67:0/2016170942 >> v2:172.21.15.67:3300/0 conn(0x7fba88153630 0x7fba880c6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.348+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661053970 0 0) 0x7fba880c8630 con 0x7fba88070810 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.348+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba6c003520 con 0x7fba88070810 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3242481847 0 0) 0x7fba6c003520 con 0x7fba88070810 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 >> v2:172.21.15.67:3300/0 conn(0x7fba88153630 msgr2=0x7fba880c6160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 --2- 172.21.15.67:0/2016170942 >> v2:172.21.15.67:3300/0 conn(0x7fba88153630 0x7fba880c6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 >> v1:172.21.15.67:6789/0 conn(0x7fba880738c0 legacy=0x7fba881562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba88072fa0 con 0x7fba88070810 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fba800019f0 con 0x7fba88070810 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 >> v1:172.21.15.134:6789/0 conn(0x7fba88070810 legacy=0x7fba881606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 --2- 172.21.15.67:0/2016170942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba6c0039b0 0x7fba6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 --2- 172.21.15.67:0/2016170942 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba6c004800 0x7fba6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 --2- 172.21.15.67:0/2016170942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6c0053c0 0x7fba6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba88155df0 con 0x7fba6c0039b0 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c00c250 con 0x7fba6c0053c0 2024-03-09T09:02:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c00c450 con 0x7fba6c004800 2024-03-09T09:02:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.349+0000 7fba87e00640 1 --2- 172.21.15.67:0/2016170942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba6c0039b0 0x7fba6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.350+0000 7fba87400640 1 --2- 172.21.15.67:0/2016170942 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba6c004800 0x7fba6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.350+0000 7fba8d200640 1 --2- 172.21.15.67:0/2016170942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6c0053c0 0x7fba6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.350+0000 7fba87e00640 1 -- 172.21.15.67:0/2016170942 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba6c004800 msgr2=0x7fba6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.350+0000 7fba87e00640 1 --2- 172.21.15.67:0/2016170942 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba6c004800 0x7fba6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.350+0000 7fba87e00640 1 -- 172.21.15.67:0/2016170942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6c0053c0 msgr2=0x7fba6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:33.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.350+0000 7fba87e00640 1 --2- 172.21.15.67:0/2016170942 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6c0053c0 0x7fba6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.350+0000 7fba87e00640 1 -- 172.21.15.67:0/2016170942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c003520 con 0x7fba6c0039b0 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.350+0000 7fba8d200640 1 -- 172.21.15.67:0/2016170942 reap_dead start 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.351+0000 7fba87e00640 1 --2- 172.21.15.67:0/2016170942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba6c0039b0 0x7fba6c003dd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fba7800b9e0 tx=0x7fba6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28077d3767421db server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.351+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78036020 con 0x7fba6c0039b0 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.351+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba78032410 con 0x7fba6c0039b0 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.352+0000 7fba86a00640 1 -- 172.21.15.67:0/2016170942 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7800b5e0 con 0x7fba6c0039b0 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.352+0000 7fba8f200640 1 -- 172.21.15.67:0/2016170942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba6c0039b0 msgr2=0x7fba6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.352+0000 7fba8f200640 1 --2- 172.21.15.67:0/2016170942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba6c0039b0 0x7fba6c003dd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fba7800b9e0 tx=0x7fba6c005830 comp rx=0 tx=0).stop 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.352+0000 7fba8f200640 1 -- 172.21.15.67:0/2016170942 shutdown_connections 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.352+0000 7fba8f200640 1 --2- 172.21.15.67:0/2016170942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba6c0039b0 0x7fba6c003dd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.352+0000 7fba8f200640 1 -- 172.21.15.67:0/2016170942 >> 172.21.15.67:0/2016170942 conn(0x7fba88066d90 msgr2=0x7fba880671a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.352+0000 7fba8f200640 1 -- 172.21.15.67:0/2016170942 shutdown_connections 2024-03-09T09:02:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.353+0000 7fba8f200640 1 -- 172.21.15.67:0/2016170942 wait complete. 2024-03-09T09:02:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.355+0000 7fba8f200640 1 Processor -- start 2024-03-09T09:02:33.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.356+0000 7fba8f200640 1 -- start start 2024-03-09T09:02:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.357+0000 7fba8f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6c0039b0 0x7fba88158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.357+0000 7fba8f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba88157490 0x7fba88158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.357+0000 7fba8f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba88152170 0x7fba88152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.357+0000 7fba8f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba880c8a50 con 0x7fba88157490 2024-03-09T09:02:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.357+0000 7fba8f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba880c88d0 con 0x7fba6c0039b0 2024-03-09T09:02:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.357+0000 7fba8f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba880c7630 con 0x7fba88152170 2024-03-09T09:02:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.357+0000 7fba87400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba88157490 0x7fba88158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.357+0000 7fba87400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba88157490 0x7fba88158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44832/0 (socket says 172.21.15.67:44832) 2024-03-09T09:02:33.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.357+0000 7fba87400640 1 -- 172.21.15.67:0/3546996897 learned_addr learned my addr 172.21.15.67:0/3546996897 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.357+0000 7fba8d200640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba88152170 0x7fba88152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.357+0000 7fba87e00640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6c0039b0 0x7fba88158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.358+0000 7fba87400640 1 -- 172.21.15.67:0/3546996897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba88157490 msgr2=0x7fba88158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.358+0000 7fba87400640 1 -- 172.21.15.67:0/3546996897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba88157490 msgr2=0x7fba88158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.358+0000 7fba87400640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba88157490 0x7fba88158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.358+0000 7fba87400640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba88157490 0x7fba88158590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.358+0000 7fba87e00640 1 -- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba88152170 msgr2=0x7fba88152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.358+0000 7fba87e00640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba88152170 0x7fba88152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.358+0000 7fba87e00640 1 -- 172.21.15.67:0/3546996897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba88157490 msgr2=0x7fba88158590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.358+0000 7fba87e00640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba88157490 0x7fba88158590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.358+0000 7fba87e00640 1 -- 172.21.15.67:0/3546996897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba881ec740 con 0x7fba6c0039b0 2024-03-09T09:02:33.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.358+0000 7fba8d200640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba88152170 0x7fba88152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:02:33.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.359+0000 7fba87e00640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6c0039b0 0x7fba88158050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fba78034000 tx=0x7fba78004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:33.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.359+0000 7fba7fe00640 1 -- 172.21.15.67:0/3546996897 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7803b8b0 con 0x7fba6c0039b0 2024-03-09T09:02:33.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.359+0000 7fba7fe00640 1 -- 172.21.15.67:0/3546996897 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7803ba50 con 0x7fba6c0039b0 2024-03-09T09:02:33.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.359+0000 7fba8f200640 1 -- 172.21.15.67:0/3546996897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba881eca20 con 0x7fba6c0039b0 2024-03-09T09:02:33.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.359+0000 7fba7fe00640 1 -- 172.21.15.67:0/3546996897 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7803be00 con 0x7fba6c0039b0 2024-03-09T09:02:33.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.359+0000 7fba8f200640 1 -- 172.21.15.67:0/3546996897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba88155ba0 con 0x7fba6c0039b0 2024-03-09T09:02:33.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.360+0000 7fba8f200640 1 -- 172.21.15.67:0/3546996897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba88075a00 con 0x7fba6c0039b0 2024-03-09T09:02:33.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.362+0000 7fba7fe00640 1 -- 172.21.15.67:0/3546996897 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fba78039020 con 0x7fba6c0039b0 2024-03-09T09:02:33.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.363+0000 7fba7fe00640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba6806b4d0 0x7fba6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:33.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.363+0000 7fba7fe00640 1 -- 172.21.15.67:0/3546996897 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fba780acaf0 con 0x7fba6c0039b0 2024-03-09T09:02:33.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.364+0000 7fba87400640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba6806b4d0 0x7fba6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:33.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.365+0000 7fba87400640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba6806b4d0 0x7fba6806d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fba88157890 tx=0x7fba74009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.373+0000 7fba7fe00640 1 -- 172.21.15.67:0/3546996897 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fba78053020 con 0x7fba6c0039b0 2024-03-09T09:02:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.776+0000 7fba8f200640 1 -- 172.21.15.67:0/3546996897 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fba880705f0 con 0x7fba6c0039b0 2024-03-09T09:02:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.777+0000 7fba7fe00640 1 -- 172.21.15.67:0/3546996897 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fba78040070 con 0x7fba6c0039b0 2024-03-09T09:02:33.778 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":76,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":29,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.779+0000 7fba8f200640 1 -- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba6806b4d0 msgr2=0x7fba6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.779+0000 7fba8f200640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba6806b4d0 0x7fba6806d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fba88157890 tx=0x7fba74009040 comp rx=0 tx=0).stop 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.779+0000 7fba8f200640 1 -- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6c0039b0 msgr2=0x7fba88158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.779+0000 7fba8f200640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6c0039b0 0x7fba88158050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fba78034000 tx=0x7fba78004730 comp rx=0 tx=0).stop 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.779+0000 7fba8f200640 1 -- 172.21.15.67:0/3546996897 shutdown_connections 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.780+0000 7fba8f200640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba6806b4d0 0x7fba6806d990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.780+0000 7fba8f200640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba88152170 0x7fba88152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.780+0000 7fba8f200640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba88157490 0x7fba88158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.780+0000 7fba8f200640 1 --2- 172.21.15.67:0/3546996897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba6c0039b0 0x7fba88158050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.780+0000 7fba8f200640 1 -- 172.21.15.67:0/3546996897 >> 172.21.15.67:0/3546996897 conn(0x7fba88066d90 msgr2=0x7fba8814fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.780+0000 7fba8f200640 1 -- 172.21.15.67:0/3546996897 shutdown_connections 2024-03-09T09:02:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:33.780+0000 7fba8f200640 1 -- 172.21.15.67:0/3546996897 wait complete. 2024-03-09T09:02:36.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:02:37.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.010+0000 7f346b400640 1 Processor -- start 2024-03-09T09:02:37.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.010+0000 7f346b400640 1 -- start start 2024-03-09T09:02:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.011+0000 7f346b400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f34640ac730 0x7f3464071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.011+0000 7f346b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3464157d60 0x7f34640acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.011+0000 7f346b400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34641581a0 con 0x7f34640c22a0 2024-03-09T09:02:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.011+0000 7f346b400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3464070ee0 con 0x7f3464157d60 2024-03-09T09:02:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.011+0000 7f346b400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3464154d00 con 0x7f34640ac730 2024-03-09T09:02:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.011+0000 7f3463e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f34640c22a0 0x7f3464157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39290/0 (socket says 172.21.15.67:39290) 2024-03-09T09:02:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.011+0000 7f3463e00640 1 -- 172.21.15.67:0/969446383 learned_addr learned my addr 172.21.15.67:0/969446383 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:02:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.011+0000 7f3469400640 1 --2- 172.21.15.67:0/969446383 >> v2:172.21.15.67:3300/0 conn(0x7f3464157d60 0x7f34640acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.012+0000 7f3468a00640 1 --2- 172.21.15.67:0/969446383 >> v2:172.21.15.134:3300/0 conn(0x7f34640ac730 0x7f3464071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.012+0000 7f3469400640 1 -- 172.21.15.67:0/969446383 >> v2:172.21.15.67:3300/0 conn(0x7f3464157d60 msgr2=0x7f34640acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:02:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.012+0000 7f3469400640 1 -- 172.21.15.67:0/969446383 >> v2:172.21.15.67:3300/0 conn(0x7f3464157d60 msgr2=0x7f34640acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:02:37.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.012+0000 7f3469400640 1 --2- 172.21.15.67:0/969446383 >> v2:172.21.15.67:3300/0 conn(0x7f3464157d60 0x7f34640acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:37.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.012+0000 7f3469400640 1 --2- 172.21.15.67:0/969446383 >> v2:172.21.15.67:3300/0 conn(0x7f3464157d60 0x7f34640acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:37.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.012+0000 7f3468a00640 1 -- 172.21.15.67:0/969446383 >> v2:172.21.15.67:3300/0 conn(0x7f3464157d60 msgr2=0x7f34640acb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:37.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.012+0000 7f3468a00640 1 --2- 172.21.15.67:0/969446383 >> v2:172.21.15.67:3300/0 conn(0x7f3464157d60 0x7f34640acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:37.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.012+0000 7f3468a00640 1 -- 172.21.15.67:0/969446383 >> v1:172.21.15.67:6789/0 conn(0x7f34640c22a0 legacy=0x7f3464157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:37.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.013+0000 7f3468a00640 1 -- 172.21.15.67:0/969446383 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3464153d40 con 0x7f34640ac730 2024-03-09T09:02:37.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.014+0000 7f3468a00640 1 --2- 172.21.15.67:0/969446383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34640ac730 0x7f3464071190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f34640ad380 tx=0x7f345802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c415da41dd03f50e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:37.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.014+0000 7f3463400640 1 -- 172.21.15.67:0/969446383 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34580315a0 con 0x7f34640ac730 2024-03-09T09:02:37.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.014+0000 7f3463400640 1 -- 172.21.15.67:0/969446383 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3458037070 con 0x7f34640ac730 2024-03-09T09:02:37.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.014+0000 7f3463400640 1 -- 172.21.15.67:0/969446383 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458031960 con 0x7f34640ac730 2024-03-09T09:02:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.015+0000 7f346b400640 1 -- 172.21.15.67:0/969446383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34640ac730 msgr2=0x7f3464071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.015+0000 7f346b400640 1 --2- 172.21.15.67:0/969446383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34640ac730 0x7f3464071190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f34640ad380 tx=0x7f345802b840 comp rx=0 tx=0).stop 2024-03-09T09:02:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.015+0000 7f346b400640 1 -- 172.21.15.67:0/969446383 shutdown_connections 2024-03-09T09:02:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.015+0000 7f346b400640 1 --2- 172.21.15.67:0/969446383 >> v2:172.21.15.67:3300/0 conn(0x7f3464157d60 0x7f34640acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.015+0000 7f346b400640 1 --2- 172.21.15.67:0/969446383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34640ac730 0x7f3464071190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.015+0000 7f346b400640 1 -- 172.21.15.67:0/969446383 >> 172.21.15.67:0/969446383 conn(0x7f34640c7370 msgr2=0x7f34640c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.015+0000 7f346b400640 1 -- 172.21.15.67:0/969446383 shutdown_connections 2024-03-09T09:02:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.015+0000 7f346b400640 1 -- 172.21.15.67:0/969446383 wait complete. 2024-03-09T09:02:37.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.017+0000 7f346b400640 1 Processor -- start 2024-03-09T09:02:37.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.017+0000 7f346b400640 1 -- start start 2024-03-09T09:02:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.018+0000 7f346b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640ac730 0x7f34640c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.018+0000 7f346b400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34640c22a0 0x7f34641ec120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.018+0000 7f346b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464157d60 0x7f34641ec660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.018+0000 7f346b400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3464070c00 con 0x7f34640ac730 2024-03-09T09:02:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.018+0000 7f346b400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3464070920 con 0x7f3464157d60 2024-03-09T09:02:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.018+0000 7f346b400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3464151610 con 0x7f34640c22a0 2024-03-09T09:02:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.018+0000 7f3468a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640ac730 0x7f34640c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.018+0000 7f3468a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640ac730 0x7f34640c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44864/0 (socket says 172.21.15.67:44864) 2024-03-09T09:02:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.018+0000 7f3468a00640 1 -- 172.21.15.67:0/1765058182 learned_addr learned my addr 172.21.15.67:0/1765058182 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.019+0000 7f3469400640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464157d60 0x7f34641ec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.019+0000 7f3468a00640 1 -- 172.21.15.67:0/1765058182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640ac730 msgr2=0x7f34640c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:02:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.019+0000 7f3468a00640 1 -- 172.21.15.67:0/1765058182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640ac730 msgr2=0x7f34640c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:02:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.019+0000 7f3468a00640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640ac730 0x7f34640c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:02:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.019+0000 7f3463e00640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34640c22a0 0x7f34641ec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.019+0000 7f3468a00640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640ac730 0x7f34640c0a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.019+0000 7f3469400640 1 -- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34640c22a0 msgr2=0x7f34641ec120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.019+0000 7f3469400640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34640c22a0 0x7f34641ec120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.019+0000 7f3469400640 1 -- 172.21.15.67:0/1765058182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640ac730 msgr2=0x7f34640c0a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:02:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.019+0000 7f3469400640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640ac730 0x7f34640c0a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.019+0000 7f3469400640 1 -- 172.21.15.67:0/1765058182 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34641de8c0 con 0x7f3464157d60 2024-03-09T09:02:37.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.020+0000 7f3469400640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464157d60 0x7f34641ec660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f345c007cf0 tx=0x7f345c00e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:37.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.020+0000 7f3460c00640 1 -- 172.21.15.67:0/1765058182 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c00ed70 con 0x7f3464157d60 2024-03-09T09:02:37.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.020+0000 7f3460c00640 1 -- 172.21.15.67:0/1765058182 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345c003990 con 0x7f3464157d60 2024-03-09T09:02:37.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.020+0000 7f346b400640 1 -- 172.21.15.67:0/1765058182 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34641ecba0 con 0x7f3464157d60 2024-03-09T09:02:37.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.020+0000 7f3460c00640 1 -- 172.21.15.67:0/1765058182 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c003d80 con 0x7f3464157d60 2024-03-09T09:02:37.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.020+0000 7f346b400640 1 -- 172.21.15.67:0/1765058182 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34640c15a0 con 0x7f3464157d60 2024-03-09T09:02:37.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.022+0000 7f346b400640 1 -- 172.21.15.67:0/1765058182 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34641ebe00 con 0x7f3464157d60 2024-03-09T09:02:37.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.022+0000 7f3460c00640 1 -- 172.21.15.67:0/1765058182 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f345c01b020 con 0x7f3464157d60 2024-03-09T09:02:37.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.023+0000 7f3460c00640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f344406b4d0 0x7f344406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:37.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.023+0000 7f3460c00640 1 -- 172.21.15.67:0/1765058182 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f345c08a620 con 0x7f3464157d60 2024-03-09T09:02:37.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.024+0000 7f3468a00640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f344406b4d0 0x7f344406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:37.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.025+0000 7f3468a00640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f344406b4d0 0x7f344406d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f345802b880 tx=0x7f345803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:37.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.031+0000 7f3460c00640 1 -- 172.21.15.67:0/1765058182 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f34641ecba0 con 0x7f3464157d60 2024-03-09T09:02:37.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.375+0000 7f346b400640 1 -- 172.21.15.67:0/1765058182 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3464075a00 con 0x7f3464157d60 2024-03-09T09:02:37.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.376+0000 7f3460c00640 1 -- 172.21.15.67:0/1765058182 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f345c0590a0 con 0x7f3464157d60 2024-03-09T09:02:37.378 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":76,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":33,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:02:37.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 -- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f344406b4d0 msgr2=0x7f344406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:37.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f344406b4d0 0x7f344406d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f345802b880 tx=0x7f345803d040 comp rx=0 tx=0).stop 2024-03-09T09:02:37.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 -- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464157d60 msgr2=0x7f34641ec660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:37.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464157d60 0x7f34641ec660 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f345c007cf0 tx=0x7f345c00e4b0 comp rx=0 tx=0).stop 2024-03-09T09:02:37.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 -- 172.21.15.67:0/1765058182 shutdown_connections 2024-03-09T09:02:37.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f344406b4d0 0x7f344406d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:37.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3464157d60 0x7f34641ec660 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:37.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34640c22a0 0x7f34641ec120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:37.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 --2- 172.21.15.67:0/1765058182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34640ac730 0x7f34640c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:37.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 -- 172.21.15.67:0/1765058182 >> 172.21.15.67:0/1765058182 conn(0x7f34640c7370 msgr2=0x7f34640c47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:37.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 -- 172.21.15.67:0/1765058182 shutdown_connections 2024-03-09T09:02:37.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:37.379+0000 7f346b400640 1 -- 172.21.15.67:0/1765058182 wait complete. 2024-03-09T09:02:40.401 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:02:40.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.622+0000 7f40cea00640 1 Processor -- start 2024-03-09T09:02:40.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.622+0000 7f40cea00640 1 -- start start 2024-03-09T09:02:40.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.623+0000 7f40cea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c814c5c0 0x7f40c8152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:40.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.623+0000 7f40cea00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f40c80c3780 0x7f40c806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:40.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.623+0000 7f40cea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c80c1d70 con 0x7f40c806e4d0 2024-03-09T09:02:40.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.623+0000 7f40cea00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c80aaae0 con 0x7f40c80c3780 2024-03-09T09:02:40.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.623+0000 7f40cea00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c814c9a0 con 0x7f40c814c5c0 2024-03-09T09:02:40.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.624+0000 7f40cca00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f40c806e4d0 0x7f40c80aad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46564/0 (socket says 172.21.15.67:46564) 2024-03-09T09:02:40.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.624+0000 7f40cca00640 1 -- 172.21.15.67:0/980021943 learned_addr learned my addr 172.21.15.67:0/980021943 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:02:40.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.624+0000 7f40c7400640 1 --2- 172.21.15.67:0/980021943 >> v2:172.21.15.134:3300/0 conn(0x7f40c80c3780 0x7f40c806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:40.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.624+0000 7f40c7e00640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c814c5c0 0x7f40c8152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:40.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.624+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406976171 0 0) 0x7f40c80c1d70 con 0x7f40c806e4d0 2024-03-09T09:02:40.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40ac003520 con 0x7f40c806e4d0 2024-03-09T09:02:40.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 278077710 0 0) 0x7f40ac003520 con 0x7f40c806e4d0 2024-03-09T09:02:40.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c814c5c0 msgr2=0x7f40c8152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:40.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c814c5c0 0x7f40c8152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:40.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 >> v2:172.21.15.134:3300/0 conn(0x7f40c80c3780 msgr2=0x7f40c806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 --2- 172.21.15.67:0/980021943 >> v2:172.21.15.134:3300/0 conn(0x7f40c80c3780 0x7f40c806df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c80c2010 con 0x7f40c806e4d0 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f40c00019f0 con 0x7f40c806e4d0 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 >> v1:172.21.15.67:6789/0 conn(0x7f40c806e4d0 legacy=0x7f40c80aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c814c5c0 0x7f40ac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40ac003f30 0x7f40ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40ac004d40 0x7f40ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80aaae0 con 0x7f40ac003f30 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c814c9a0 con 0x7f40ac004d40 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.625+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac00ba10 con 0x7f40c814c5c0 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.626+0000 7f40c7400640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40ac003f30 0x7f40ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.626+0000 7f40cca00640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40ac004d40 0x7f40ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:40.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.626+0000 7f40c7e00640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c814c5c0 0x7f40ac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:40.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.626+0000 7f40c7400640 1 -- 172.21.15.67:0/980021943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c814c5c0 msgr2=0x7f40ac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:40.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.626+0000 7f40c7400640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c814c5c0 0x7f40ac0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:40.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.626+0000 7f40c7400640 1 -- 172.21.15.67:0/980021943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40ac004d40 msgr2=0x7f40ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:40.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.626+0000 7f40c7400640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40ac004d40 0x7f40ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:40.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.626+0000 7f40c7400640 1 -- 172.21.15.67:0/980021943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c80c2010 con 0x7f40ac003f30 2024-03-09T09:02:40.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.628+0000 7f40c7400640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40ac003f30 0x7f40ac004800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f40b400bc60 tx=0x7f40ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8e27895d5432dc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:40.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.628+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4032070 con 0x7f40ac003f30 2024-03-09T09:02:40.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.628+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b402cb00 con 0x7f40ac003f30 2024-03-09T09:02:40.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.628+0000 7f40c6a00640 1 -- 172.21.15.67:0/980021943 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b400b800 con 0x7f40ac003f30 2024-03-09T09:02:40.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.629+0000 7f40cea00640 1 -- 172.21.15.67:0/980021943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40ac003f30 msgr2=0x7f40ac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:40.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.629+0000 7f40cea00640 1 --2- 172.21.15.67:0/980021943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40ac003f30 0x7f40ac004800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f40b400bc60 tx=0x7f40ac005830 comp rx=0 tx=0).stop 2024-03-09T09:02:40.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.629+0000 7f40cca00640 1 -- 172.21.15.67:0/980021943 reap_dead start 2024-03-09T09:02:40.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.629+0000 7f40cea00640 1 -- 172.21.15.67:0/980021943 shutdown_connections 2024-03-09T09:02:40.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.629+0000 7f40cea00640 1 -- 172.21.15.67:0/980021943 >> 172.21.15.67:0/980021943 conn(0x7f40c80c7370 msgr2=0x7f40c80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:40.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.629+0000 7f40cea00640 1 -- 172.21.15.67:0/980021943 shutdown_connections 2024-03-09T09:02:40.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.629+0000 7f40cea00640 1 -- 172.21.15.67:0/980021943 wait complete. 2024-03-09T09:02:40.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.631+0000 7f40cea00640 1 Processor -- start 2024-03-09T09:02:40.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.631+0000 7f40cea00640 1 -- start start 2024-03-09T09:02:40.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40ac003f30 0x7f40c80729d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:40.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c81eb460 0x7f40c81462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c814cb50 0x7f40c8146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80c28b0 con 0x7f40c814cb50 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cea00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c80c25d0 con 0x7f40c81eb460 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cea00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8152ce0 con 0x7f40ac003f30 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c814cb50 0x7f40c8146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c814cb50 0x7f40c8146830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54798/0 (socket says 172.21.15.67:54798) 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cca00640 1 -- 172.21.15.67:0/998763638 learned_addr learned my addr 172.21.15.67:0/998763638 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40c7400640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c81eb460 0x7f40c81462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40c7e00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40ac003f30 0x7f40c80729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cca00640 1 -- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40ac003f30 msgr2=0x7f40c80729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cca00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40ac003f30 0x7f40c80729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cca00640 1 -- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c81eb460 msgr2=0x7f40c81462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.632+0000 7f40cca00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c81eb460 0x7f40c81462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.633+0000 7f40cca00640 1 -- 172.21.15.67:0/998763638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8151dc0 con 0x7f40c814cb50 2024-03-09T09:02:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.633+0000 7f40cca00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c814cb50 0x7f40c8146830 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40c8070470 tx=0x7f40c000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.633+0000 7f40bfe00640 1 -- 172.21.15.67:0/998763638 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0007ad0 con 0x7f40c814cb50 2024-03-09T09:02:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.633+0000 7f40bfe00640 1 -- 172.21.15.67:0/998763638 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c000ee20 con 0x7f40c814cb50 2024-03-09T09:02:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.633+0000 7f40bfe00640 1 -- 172.21.15.67:0/998763638 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0014650 con 0x7f40c814cb50 2024-03-09T09:02:40.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.633+0000 7f40cea00640 1 -- 172.21.15.67:0/998763638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c81520a0 con 0x7f40c814cb50 2024-03-09T09:02:40.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.633+0000 7f40cea00640 1 -- 172.21.15.67:0/998763638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c8072f10 con 0x7f40c814cb50 2024-03-09T09:02:40.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.634+0000 7f40cea00640 1 -- 172.21.15.67:0/998763638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c8068d50 con 0x7f40c814cb50 2024-03-09T09:02:40.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.635+0000 7f40bfe00640 1 -- 172.21.15.67:0/998763638 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f40c00147f0 con 0x7f40c814cb50 2024-03-09T09:02:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.636+0000 7f40bfe00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40a806b4d0 0x7f40a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.636+0000 7f40bfe00640 1 -- 172.21.15.67:0/998763638 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f40c008a540 con 0x7f40c814cb50 2024-03-09T09:02:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.637+0000 7f40c7e00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40a806b4d0 0x7f40a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:40.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.638+0000 7f40c7e00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40a806b4d0 0x7f40a806d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009930 tx=0x7f40b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:40.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.645+0000 7f40bfe00640 1 -- 172.21.15.67:0/998763638 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f40c8068d50 con 0x7f40c814cb50 2024-03-09T09:02:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.979+0000 7f40cea00640 1 -- 172.21.15.67:0/998763638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f40c8075a00 con 0x7f40c814cb50 2024-03-09T09:02:40.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.979+0000 7f40bfe00640 1 -- 172.21.15.67:0/998763638 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7f40c0058fc0 con 0x7f40c814cb50 2024-03-09T09:02:40.981 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":80,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 -- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40a806b4d0 msgr2=0x7f40a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40a806b4d0 0x7f40a806d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009930 tx=0x7f40b8008040 comp rx=0 tx=0).stop 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 -- 172.21.15.67:0/998763638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c814cb50 msgr2=0x7f40c8146830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c814cb50 0x7f40c8146830 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f40c8070470 tx=0x7f40c000e530 comp rx=0 tx=0).stop 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 -- 172.21.15.67:0/998763638 shutdown_connections 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40a806b4d0 0x7f40a806d990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40c814cb50 0x7f40c8146830 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c81eb460 0x7f40c81462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 --2- 172.21.15.67:0/998763638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40ac003f30 0x7f40c80729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 -- 172.21.15.67:0/998763638 >> 172.21.15.67:0/998763638 conn(0x7f40c80c7370 msgr2=0x7f40c814a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 -- 172.21.15.67:0/998763638 shutdown_connections 2024-03-09T09:02:40.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:40.982+0000 7f40cea00640 1 -- 172.21.15.67:0/998763638 wait complete. 2024-03-09T09:02:41.001 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:02:41.001 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:02:41.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:02:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.220+0000 7f9ce7c00640 1 Processor -- start 2024-03-09T09:02:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.220+0000 7f9ce7c00640 1 -- start start 2024-03-09T09:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.221+0000 7f9ce7c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9ce00ac9a0 0x7f9ce01510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.221+0000 7f9ce7c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ce0071190 0x7f9ce0151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.221+0000 7f9ce7c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce00c8200 con 0x7f9ce00c2780 2024-03-09T09:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.221+0000 7f9ce7c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce006d590 con 0x7f9ce00ac9a0 2024-03-09T09:02:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.221+0000 7f9ce7c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce00c3740 con 0x7f9ce0071190 2024-03-09T09:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.222+0000 7f9cdfe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9ce00c2780 0x7f9ce0154190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46572/0 (socket says 172.21.15.67:46572) 2024-03-09T09:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.222+0000 7f9cdfe00640 1 -- 172.21.15.67:0/3658028400 learned_addr learned my addr 172.21.15.67:0/3658028400 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.222+0000 7f9ce5c00640 1 --2- 172.21.15.67:0/3658028400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ce0071190 0x7f9ce0151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.222+0000 7f9ce5200640 1 --2- 172.21.15.67:0/3658028400 >> v2:172.21.15.134:3300/0 conn(0x7f9ce00ac9a0 0x7f9ce01510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.222+0000 7f9cdf400640 1 -- 172.21.15.67:0/3658028400 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938904295 0 0) 0x7f9ce00c8200 con 0x7f9ce00c2780 2024-03-09T09:02:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.222+0000 7f9cdf400640 1 -- 172.21.15.67:0/3658028400 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cc8003520 con 0x7f9ce00c2780 2024-03-09T09:02:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.223+0000 7f9ce5c00640 1 -- 172.21.15.67:0/3658028400 >> v2:172.21.15.134:3300/0 conn(0x7f9ce00ac9a0 msgr2=0x7f9ce01510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.223+0000 7f9ce5c00640 1 --2- 172.21.15.67:0/3658028400 >> v2:172.21.15.134:3300/0 conn(0x7f9ce00ac9a0 0x7f9ce01510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.223+0000 7f9ce5c00640 1 -- 172.21.15.67:0/3658028400 >> v1:172.21.15.67:6789/0 conn(0x7f9ce00c2780 legacy=0x7f9ce0154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.223+0000 7f9cdf400640 1 -- 172.21.15.67:0/3658028400 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 93240505 0 0) 0x7f9cc8003520 con 0x7f9ce00c2780 2024-03-09T09:02:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.223+0000 7f9ce5c00640 1 -- 172.21.15.67:0/3658028400 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce00c3990 con 0x7f9ce0071190 2024-03-09T09:02:41.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.224+0000 7f9ce5c00640 1 --2- 172.21.15.67:0/3658028400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ce0071190 0x7f9ce0151620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009d90 tx=0x7f9cd802bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f121d024023681 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.225+0000 7f9cdf400640 1 -- 172.21.15.67:0/3658028400 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8039070 con 0x7f9ce0071190 2024-03-09T09:02:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.225+0000 7f9cdf400640 1 -- 172.21.15.67:0/3658028400 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd802c810 con 0x7f9ce0071190 2024-03-09T09:02:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.225+0000 7f9cdf400640 1 -- 172.21.15.67:0/3658028400 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc001670 con 0x7f9ce0071190 2024-03-09T09:02:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.225+0000 7f9ce7c00640 1 -- 172.21.15.67:0/3658028400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ce0071190 msgr2=0x7f9ce0151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.225+0000 7f9ce7c00640 1 --2- 172.21.15.67:0/3658028400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ce0071190 0x7f9ce0151620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009d90 tx=0x7f9cd802bce0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.225+0000 7f9ce7c00640 1 -- 172.21.15.67:0/3658028400 shutdown_connections 2024-03-09T09:02:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.225+0000 7f9ce7c00640 1 --2- 172.21.15.67:0/3658028400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ce0071190 0x7f9ce0151620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.225+0000 7f9ce7c00640 1 --2- 172.21.15.67:0/3658028400 >> v2:172.21.15.134:3300/0 conn(0x7f9ce00ac9a0 0x7f9ce01510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.225+0000 7f9ce7c00640 1 -- 172.21.15.67:0/3658028400 >> 172.21.15.67:0/3658028400 conn(0x7f9ce00c7370 msgr2=0x7f9ce00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.226+0000 7f9ce7c00640 1 -- 172.21.15.67:0/3658028400 shutdown_connections 2024-03-09T09:02:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.226+0000 7f9ce7c00640 1 -- 172.21.15.67:0/3658028400 wait complete. 2024-03-09T09:02:41.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.228+0000 7f9ce7c00640 1 Processor -- start 2024-03-09T09:02:41.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.228+0000 7f9ce7c00640 1 -- start start 2024-03-09T09:02:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.229+0000 7f9ce7c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0071190 0x7f9ce014c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.229+0000 7f9ce7c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ce00ac9a0 0x7f9ce01523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.229+0000 7f9ce7c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ce00c2780 0x7f9ce01528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.229+0000 7f9ce7c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce00acd80 con 0x7f9ce0071190 2024-03-09T09:02:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.229+0000 7f9ce7c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce0070c00 con 0x7f9ce00c2780 2024-03-09T09:02:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.229+0000 7f9ce7c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce00acf00 con 0x7f9ce00ac9a0 2024-03-09T09:02:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.229+0000 7f9ce5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0071190 0x7f9ce014c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.229+0000 7f9ce5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0071190 0x7f9ce014c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54814/0 (socket says 172.21.15.67:54814) 2024-03-09T09:02:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.229+0000 7f9ce5200640 1 -- 172.21.15.67:0/955535320 learned_addr learned my addr 172.21.15.67:0/955535320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.230+0000 7f9cdfe00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ce00ac9a0 0x7f9ce01523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.230+0000 7f9ce5c00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ce00c2780 0x7f9ce01528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.230+0000 7f9ce5200640 1 -- 172.21.15.67:0/955535320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ce00ac9a0 msgr2=0x7f9ce01523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.230+0000 7f9ce5200640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ce00ac9a0 0x7f9ce01523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.230+0000 7f9ce5200640 1 -- 172.21.15.67:0/955535320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ce00c2780 msgr2=0x7f9ce01528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.230+0000 7f9ce5200640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ce00c2780 0x7f9ce01528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.230+0000 7f9ce5200640 1 -- 172.21.15.67:0/955535320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce014e630 con 0x7f9ce0071190 2024-03-09T09:02:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.231+0000 7f9ce5200640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0071190 0x7f9ce014c490 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400e9e0 tx=0x7f9cd400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.231+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4017070 con 0x7f9ce0071190 2024-03-09T09:02:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.231+0000 7f9ce7c00640 1 -- 172.21.15.67:0/955535320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce014b660 con 0x7f9ce0071190 2024-03-09T09:02:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.231+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd400c9d0 con 0x7f9ce0071190 2024-03-09T09:02:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.231+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd400cdc0 con 0x7f9ce0071190 2024-03-09T09:02:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.231+0000 7f9ce7c00640 1 -- 172.21.15.67:0/955535320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce014e090 con 0x7f9ce0071190 2024-03-09T09:02:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.232+0000 7f9ce7c00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce01517d0 0x7f9ce0144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.232+0000 7f9ce7c00640 1 -- 172.21.15.67:0/955535320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce0075a00 con 0x7f9ce01517d0 2024-03-09T09:02:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.232+0000 7f9cdfe00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce01517d0 0x7f9ce0144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.233+0000 7f9cdfe00640 1 -- 172.21.15.67:0/955535320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ce00c1100 con 0x7f9ce01517d0 2024-03-09T09:02:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.233+0000 7f9cdfe00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce01517d0 0x7f9ce0144770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ce01509a0 tx=0x7f9ccc00ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.234+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc017020 con 0x7f9ce01517d0 2024-03-09T09:02:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.234+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9cd401a070 con 0x7f9ce0071190 2024-03-09T09:02:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.235+0000 7f9cdcc00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cc006b3d0 0x7f9cc006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.235+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9cd408a530 con 0x7f9ce0071190 2024-03-09T09:02:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.235+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f9ccc00d820 con 0x7f9ce01517d0 2024-03-09T09:02:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.235+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce01517d0 msgr2=0x7f9ce0144770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.235+0000 7f9cdcc00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce01517d0 0x7f9ce0144770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ce01509a0 tx=0x7f9ccc00ff70 comp rx=0 tx=0).stop 2024-03-09T09:02:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.235+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce01517d0 msgr2=0x7f9ce0144770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.235+0000 7f9cdcc00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce01517d0 0x7f9ce0144770 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ce01509a0 tx=0x7f9ccc00ff70 comp rx=0 tx=0).stop 2024-03-09T09:02:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.235+0000 7f9ce5c00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cc006b3d0 0x7f9cc006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.236+0000 7f9ce5c00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cc006b3d0 0x7f9cc006d890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8036000 tx=0x7f9cd8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.242+0000 7f9ce7c00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0152ee0 0x7f9ce0144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.242+0000 7f9ce7c00640 1 -- 172.21.15.67:0/955535320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce01535b0 con 0x7f9ce0152ee0 2024-03-09T09:02:41.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.243+0000 7f9cdfe00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0152ee0 0x7f9ce0144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.243+0000 7f9cdfe00640 1 -- 172.21.15.67:0/955535320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9ce006e410 con 0x7f9ce0152ee0 2024-03-09T09:02:41.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.243+0000 7f9cdfe00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0152ee0 0x7f9ce0144cb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9ce006eab0 tx=0x7f9ccc001730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc01c620 con 0x7f9ce0152ee0 2024-03-09T09:02:41.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f9ccc01c620 con 0x7f9ce0152ee0 2024-03-09T09:02:41.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0152ee0 msgr2=0x7f9ce0144cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9cdcc00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0152ee0 0x7f9ce0144cb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9ce006eab0 tx=0x7f9ccc001730 comp rx=0 tx=0).stop 2024-03-09T09:02:41.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9cdcc00640 1 -- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0152ee0 msgr2=0x7f9ce0144cb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:41.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9cdcc00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0152ee0 0x7f9ce0144cb0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9ce006eab0 tx=0x7f9ccc001730 comp rx=0 tx=0).stop 2024-03-09T09:02:41.246 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:02:41.246 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 80, 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:41.247 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:41.248 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:02:41.249 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:41.250 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.251 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:41.252 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:02:41.253 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.253 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.253 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.253 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:02:41.253 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:02:41.253 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:41.253 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:41.254 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:41.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.254 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:02:41.254 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:02:41.254 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:41.254 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:41.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.254 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:41.254 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:02:41.254 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.256 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:41.257 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:02:41.258 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:02:41.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9ce7c00640 1 -- 172.21.15.67:0/955535320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cc006b3d0 msgr2=0x7f9cc006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9ce7c00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cc006b3d0 0x7f9cc006d890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8036000 tx=0x7f9cd8030000 comp rx=0 tx=0).stop 2024-03-09T09:02:41.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9ce7c00640 1 -- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0071190 msgr2=0x7f9ce014c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9ce7c00640 1 --2- 172.21.15.67:0/955535320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ce0071190 0x7f9ce014c490 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400e9e0 tx=0x7f9cd400eea0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9ce5c00640 1 -- 172.21.15.67:0/955535320 reap_dead start 2024-03-09T09:02:41.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9ce5c00640 1 -- 172.21.15.67:0/955535320 reap_dead start 2024-03-09T09:02:41.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9ce7c00640 1 -- 172.21.15.67:0/955535320 shutdown_connections 2024-03-09T09:02:41.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.244+0000 7f9ce7c00640 1 -- 172.21.15.67:0/955535320 >> 172.21.15.67:0/955535320 conn(0x7f9ce00c7370 msgr2=0x7f9ce006d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:41.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.245+0000 7f9ce7c00640 1 -- 172.21.15.67:0/955535320 shutdown_connections 2024-03-09T09:02:41.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.245+0000 7f9ce7c00640 1 -- 172.21.15.67:0/955535320 wait complete. 2024-03-09T09:02:41.264 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:02:41.264 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:02:41.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:02:41.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.474+0000 7fbdd9400640 1 Processor -- start 2024-03-09T09:02:41.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.474+0000 7fbdd9400640 1 -- start start 2024-03-09T09:02:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.475+0000 7fbdd9400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbdd414fd90 0x7fbdd4156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.475+0000 7fbdd9400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbdd40c9380 0x7fbdd40acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.475+0000 7fbdd9400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd4156bc0 con 0x7fbdd40ad440 2024-03-09T09:02:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.475+0000 7fbdd9400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd406ba60 con 0x7fbdd414fd90 2024-03-09T09:02:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.475+0000 7fbdd9400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40c9890 con 0x7fbdd40c9380 2024-03-09T09:02:41.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.475+0000 7fbdd3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbdd414fd90 0x7fbdd4156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.476+0000 7fbdd3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbdd414fd90 0x7fbdd4156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54836/0 (socket says 172.21.15.67:54836) 2024-03-09T09:02:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.476+0000 7fbdd3e00640 1 -- 172.21.15.67:0/375865960 learned_addr learned my addr 172.21.15.67:0/375865960 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.476+0000 7fbdcb400640 1 --2- 172.21.15.67:0/375865960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbdd40c9380 0x7fbdd40acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.476+0000 7fbdd3e00640 1 -- 172.21.15.67:0/375865960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbdd40c9380 msgr2=0x7fbdd40acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.476+0000 7fbdd3e00640 1 --2- 172.21.15.67:0/375865960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbdd40c9380 0x7fbdd40acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.476+0000 7fbdd3e00640 1 -- 172.21.15.67:0/375865960 >> v1:172.21.15.134:6789/0 conn(0x7fbdd40ad440 legacy=0x7fbdd406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.476+0000 7fbdd3e00640 1 -- 172.21.15.67:0/375865960 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd40c5f80 con 0x7fbdd414fd90 2024-03-09T09:02:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.478+0000 7fbdd3e00640 1 --2- 172.21.15.67:0/375865960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd414fd90 0x7fbdd4156680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009870 tx=0x7fbdc402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af947d3e3d05af36 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.478+0000 7fbdd3400640 1 -- 172.21.15.67:0/375865960 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc40317c0 con 0x7fbdd414fd90 2024-03-09T09:02:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.478+0000 7fbdd3400640 1 -- 172.21.15.67:0/375865960 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc4037070 con 0x7fbdd414fd90 2024-03-09T09:02:41.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.478+0000 7fbdd3400640 1 -- 172.21.15.67:0/375865960 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4031b80 con 0x7fbdd414fd90 2024-03-09T09:02:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.479+0000 7fbdd9400640 1 -- 172.21.15.67:0/375865960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd414fd90 msgr2=0x7fbdd4156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.479+0000 7fbdd9400640 1 --2- 172.21.15.67:0/375865960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd414fd90 0x7fbdd4156680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009870 tx=0x7fbdc402b860 comp rx=0 tx=0).stop 2024-03-09T09:02:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.479+0000 7fbdd9400640 1 -- 172.21.15.67:0/375865960 shutdown_connections 2024-03-09T09:02:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.479+0000 7fbdd9400640 1 --2- 172.21.15.67:0/375865960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbdd40c9380 0x7fbdd40acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.479+0000 7fbdd9400640 1 --2- 172.21.15.67:0/375865960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd414fd90 0x7fbdd4156680 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.479+0000 7fbdd9400640 1 -- 172.21.15.67:0/375865960 >> 172.21.15.67:0/375865960 conn(0x7fbdd40cb480 msgr2=0x7fbdd40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.479+0000 7fbdd9400640 1 -- 172.21.15.67:0/375865960 shutdown_connections 2024-03-09T09:02:41.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.479+0000 7fbdd9400640 1 -- 172.21.15.67:0/375865960 wait complete. 2024-03-09T09:02:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.481+0000 7fbdd9400640 1 Processor -- start 2024-03-09T09:02:41.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.481+0000 7fbdd9400640 1 -- start start 2024-03-09T09:02:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.482+0000 7fbdd9400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd40ad440 0x7fbdd40af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.482+0000 7fbdd9400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbdd40c9380 0x7fbdd4154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.482+0000 7fbdd9400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd414fd90 0x7fbdd41ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.482+0000 7fbdd9400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd406bcb0 con 0x7fbdd414fd90 2024-03-09T09:02:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.482+0000 7fbdd9400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40c6520 con 0x7fbdd40ad440 2024-03-09T09:02:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.482+0000 7fbdd9400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd406c8e0 con 0x7fbdd40c9380 2024-03-09T09:02:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.482+0000 7fbdd8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd414fd90 0x7fbdd41ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.482+0000 7fbdd8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd414fd90 0x7fbdd41ebfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54842/0 (socket says 172.21.15.67:54842) 2024-03-09T09:02:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.482+0000 7fbdd8a00640 1 -- 172.21.15.67:0/589083843 learned_addr learned my addr 172.21.15.67:0/589083843 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.483+0000 7fbdd3e00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd40ad440 0x7fbdd40af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.483+0000 7fbdcb400640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbdd40c9380 0x7fbdd4154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.483+0000 7fbdd8a00640 1 -- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbdd40c9380 msgr2=0x7fbdd4154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.483+0000 7fbdd8a00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbdd40c9380 0x7fbdd4154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.483+0000 7fbdd8a00640 1 -- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd40ad440 msgr2=0x7fbdd40af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.483+0000 7fbdd8a00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd40ad440 0x7fbdd40af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.483+0000 7fbdd8a00640 1 -- 172.21.15.67:0/589083843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd41efb10 con 0x7fbdd414fd90 2024-03-09T09:02:41.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.483+0000 7fbdcb400640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbdd40c9380 0x7fbdd4154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:02:41.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.483+0000 7fbdd8a00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd414fd90 0x7fbdd41ebfc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00c7d0 tx=0x7fbdcc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.484+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc003b70 con 0x7fbdd414fd90 2024-03-09T09:02:41.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.484+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc003d30 con 0x7fbdd414fd90 2024-03-09T09:02:41.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.484+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00aa90 con 0x7fbdd414fd90 2024-03-09T09:02:41.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.484+0000 7fbdd9400640 1 -- 172.21.15.67:0/589083843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd4148bb0 con 0x7fbdd414fd90 2024-03-09T09:02:41.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.484+0000 7fbdd9400640 1 -- 172.21.15.67:0/589083843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd40c5540 con 0x7fbdd414fd90 2024-03-09T09:02:41.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.485+0000 7fbdd9400640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd41551f0 0x7fbdd40b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.485+0000 7fbdd9400640 1 -- 172.21.15.67:0/589083843 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd4158cb0 con 0x7fbdd41551f0 2024-03-09T09:02:41.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.486+0000 7fbdd3e00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd41551f0 0x7fbdd40b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.486+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbdcc00ac30 con 0x7fbdd414fd90 2024-03-09T09:02:41.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.486+0000 7fbdd3e00640 1 -- 172.21.15.67:0/589083843 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbdd4148840 con 0x7fbdd41551f0 2024-03-09T09:02:41.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.487+0000 7fbdd0c00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbdb006b4d0 0x7fbdb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.487+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbdcc08a490 con 0x7fbdd414fd90 2024-03-09T09:02:41.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.487+0000 7fbdd3e00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd41551f0 0x7fbdd40b1010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbdd41537d0 tx=0x7fbdc4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.487+0000 7fbdcb400640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbdb006b4d0 0x7fbdb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.487+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc404a070 con 0x7fbdd41551f0 2024-03-09T09:02:41.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.488+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fbdc4037040 con 0x7fbdd41551f0 2024-03-09T09:02:41.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.488+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd41551f0 msgr2=0x7fbdd40b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.488+0000 7fbdd0c00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd41551f0 0x7fbdd40b1010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbdd41537d0 tx=0x7fbdc4004730 comp rx=0 tx=0).stop 2024-03-09T09:02:41.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.488+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd41551f0 msgr2=0x7fbdd40b1010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:41.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.488+0000 7fbdd0c00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd41551f0 0x7fbdd40b1010 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbdd41537d0 tx=0x7fbdc4004730 comp rx=0 tx=0).stop 2024-03-09T09:02:41.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.488+0000 7fbdcb400640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbdb006b4d0 0x7fbdb006d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbdd41e9a20 tx=0x7fbdbc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.496+0000 7fbdd9400640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd40b0650 0x7fbdd41ec500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.496+0000 7fbdd9400640 1 -- 172.21.15.67:0/589083843 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd404f3c0 con 0x7fbdd40b0650 2024-03-09T09:02:41.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.496+0000 7fbdd3e00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd40b0650 0x7fbdd41ec500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.497+0000 7fbdd3e00640 1 -- 172.21.15.67:0/589083843 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbdd406d080 con 0x7fbdd40b0650 2024-03-09T09:02:41.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.497+0000 7fbdd3e00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd40b0650 0x7fbdd41ec500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009870 tx=0x7fbdc4031000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.498+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4052070 con 0x7fbdd40b0650 2024-03-09T09:02:41.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.498+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5016 (secure 0 0 0) 0x7fbdc4003e10 con 0x7fbdd40b0650 2024-03-09T09:02:41.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.498+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd40b0650 msgr2=0x7fbdd41ec500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.498+0000 7fbdd0c00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd40b0650 0x7fbdd41ec500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009870 tx=0x7fbdc4031000 comp rx=0 tx=0).stop 2024-03-09T09:02:41.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.498+0000 7fbdd0c00640 1 -- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd40b0650 msgr2=0x7fbdd41ec500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:41.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.498+0000 7fbdd0c00640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdd40b0650 0x7fbdd41ec500 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009870 tx=0x7fbdc4031000 comp rx=0 tx=0).stop 2024-03-09T09:02:41.501 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:02:41.501 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:02:41.501 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:02:41.501 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:02:41.501 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 80, 2024-03-09T09:02:41.501 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:02:41.501 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:02:41.501 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:02:41.501 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:02:41.501 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.501 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:41.502 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:41.503 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.503 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:02:41.503 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:02:41.503 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:41.503 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:41.503 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:41.503 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:41.503 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:41.503 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:41.503 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:41.503 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:41.504 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:41.505 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:41.506 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:41.506 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:41.506 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:41.506 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:41.506 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:41.506 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.506 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:02:41.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.507 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.508 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:02:41.509 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:02:41.509 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:41.509 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:41.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.509 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:41.509 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:02:41.509 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.509 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:41.510 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:02:41.510 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.510 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.510 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.510 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:02:41.510 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:02:41.510 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:41.511 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:41.511 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:41.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.512 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:02:41.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:02:41.512 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:41.512 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:41.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.512 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:41.512 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:02:41.512 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:41.513 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:41.514 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.514 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.514 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:02:41.514 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:02:41.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.514 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:41.514 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.514 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:41.514 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.515 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.515 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:02:41.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.515 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:41.515 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.515 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:41.515 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.515 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.515 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:02:41.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.515 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.516 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:41.517 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.517 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:41.517 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.517 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.517 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:02:41.517 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:02:41.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.499+0000 7fbdd9400640 1 -- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbdb006b4d0 msgr2=0x7fbdb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.499+0000 7fbdd9400640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbdb006b4d0 0x7fbdb006d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbdd41e9a20 tx=0x7fbdbc008040 comp rx=0 tx=0).stop 2024-03-09T09:02:41.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.499+0000 7fbdd9400640 1 -- 172.21.15.67:0/589083843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd414fd90 msgr2=0x7fbdd41ebfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.499+0000 7fbdd9400640 1 --2- 172.21.15.67:0/589083843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbdd414fd90 0x7fbdd41ebfc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00c7d0 tx=0x7fbdcc00cc90 comp rx=0 tx=0).stop 2024-03-09T09:02:41.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.499+0000 7fbdd8a00640 1 -- 172.21.15.67:0/589083843 reap_dead start 2024-03-09T09:02:41.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.499+0000 7fbdd8a00640 1 -- 172.21.15.67:0/589083843 reap_dead start 2024-03-09T09:02:41.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.499+0000 7fbdd9400640 1 -- 172.21.15.67:0/589083843 shutdown_connections 2024-03-09T09:02:41.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.499+0000 7fbdd9400640 1 -- 172.21.15.67:0/589083843 >> 172.21.15.67:0/589083843 conn(0x7fbdd40cb480 msgr2=0x7fbdd414d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:41.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.499+0000 7fbdd9400640 1 -- 172.21.15.67:0/589083843 shutdown_connections 2024-03-09T09:02:41.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.499+0000 7fbdd9400640 1 -- 172.21.15.67:0/589083843 wait complete. 2024-03-09T09:02:41.520 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:02:41.520 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:02:41.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:02:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.731+0000 7f5470200640 1 Processor -- start 2024-03-09T09:02:41.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.731+0000 7f5470200640 1 -- start start 2024-03-09T09:02:41.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.732+0000 7f5470200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5468070080 0x7f5468157b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.732+0000 7f5470200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f54680c5100 0x7f54680c5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.732+0000 7f5470200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5468072d90 con 0x7f5468071140 2024-03-09T09:02:41.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.732+0000 7f5470200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5468158040 con 0x7f5468070080 2024-03-09T09:02:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.733+0000 7f5470200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5468073070 con 0x7f54680c5100 2024-03-09T09:02:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.733+0000 7f546ce00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5468070080 0x7f5468157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.733+0000 7f546d800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5468071140 0x7f54680c4810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46586/0 (socket says 172.21.15.67:46586) 2024-03-09T09:02:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.733+0000 7f546d800640 1 -- 172.21.15.67:0/3901029917 learned_addr learned my addr 172.21.15.67:0/3901029917 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:02:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.733+0000 7f546e200640 1 --2- 172.21.15.67:0/3901029917 >> v2:172.21.15.134:3300/0 conn(0x7f54680c5100 0x7f54680c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.733+0000 7f546ce00640 1 -- 172.21.15.67:0/3901029917 >> v2:172.21.15.134:3300/0 conn(0x7f54680c5100 msgr2=0x7f54680c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.734+0000 7f546ce00640 1 --2- 172.21.15.67:0/3901029917 >> v2:172.21.15.134:3300/0 conn(0x7f54680c5100 0x7f54680c5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.734+0000 7f546ce00640 1 -- 172.21.15.67:0/3901029917 >> v1:172.21.15.67:6789/0 conn(0x7f5468071140 legacy=0x7f54680c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.734+0000 7f546ce00640 1 -- 172.21.15.67:0/3901029917 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5468073350 con 0x7f5468070080 2024-03-09T09:02:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.734+0000 7f5467e00640 1 -- 172.21.15.67:0/3901029917 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424506489 0 0) 0x7f5468072d90 con 0x7f5468071140 2024-03-09T09:02:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.735+0000 7f546ce00640 1 --2- 172.21.15.67:0/3901029917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5468070080 0x7f5468157b00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f54680ac100 tx=0x7f545402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47e27a61ac5247f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.735+0000 7f5467e00640 1 -- 172.21.15.67:0/3901029917 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454035020 con 0x7f5468070080 2024-03-09T09:02:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.735+0000 7f5467e00640 1 -- 172.21.15.67:0/3901029917 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5454031830 con 0x7f5468070080 2024-03-09T09:02:41.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.735+0000 7f5467e00640 1 -- 172.21.15.67:0/3901029917 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454031c00 con 0x7f5468070080 2024-03-09T09:02:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.736+0000 7f5470200640 1 -- 172.21.15.67:0/3901029917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5468070080 msgr2=0x7f5468157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.736+0000 7f5470200640 1 --2- 172.21.15.67:0/3901029917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5468070080 0x7f5468157b00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f54680ac100 tx=0x7f545402b8d0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.736+0000 7f5470200640 1 -- 172.21.15.67:0/3901029917 shutdown_connections 2024-03-09T09:02:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.736+0000 7f5470200640 1 --2- 172.21.15.67:0/3901029917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5468070080 0x7f5468157b00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.736+0000 7f5470200640 1 --2- 172.21.15.67:0/3901029917 >> v2:172.21.15.134:3300/0 conn(0x7f54680c5100 0x7f54680c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.736+0000 7f5470200640 1 -- 172.21.15.67:0/3901029917 >> 172.21.15.67:0/3901029917 conn(0x7f54680ab320 msgr2=0x7f54680ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.736+0000 7f5470200640 1 -- 172.21.15.67:0/3901029917 shutdown_connections 2024-03-09T09:02:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.736+0000 7f5470200640 1 -- 172.21.15.67:0/3901029917 wait complete. 2024-03-09T09:02:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.738+0000 7f5470200640 1 Processor -- start 2024-03-09T09:02:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.738+0000 7f5470200640 1 -- start start 2024-03-09T09:02:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.739+0000 7f5470200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5468070080 0x7f54681ebd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.739+0000 7f5470200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5468071140 0x7f546806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.739+0000 7f5470200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54680c5100 0x7f546814f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.739+0000 7f5470200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54680704e0 con 0x7f54680c5100 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.739+0000 7f5470200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54680c42f0 con 0x7f5468071140 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.739+0000 7f5470200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54680a8fd0 con 0x7f5468070080 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.739+0000 7f546e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54680c5100 0x7f546814f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.739+0000 7f546e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54680c5100 0x7f546814f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54862/0 (socket says 172.21.15.67:54862) 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.739+0000 7f546e200640 1 -- 172.21.15.67:0/33479679 learned_addr learned my addr 172.21.15.67:0/33479679 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.739+0000 7f546ce00640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5468071140 0x7f546806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.739+0000 7f546d800640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5468070080 0x7f54681ebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.740+0000 7f546e200640 1 -- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5468070080 msgr2=0x7f54681ebd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.740+0000 7f546e200640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5468070080 0x7f54681ebd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.740+0000 7f546e200640 1 -- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5468071140 msgr2=0x7f546806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.740+0000 7f546e200640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5468071140 0x7f546806d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.740+0000 7f546e200640 1 -- 172.21.15.67:0/33479679 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54680c2bf0 con 0x7f54680c5100 2024-03-09T09:02:41.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.740+0000 7f546ce00640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5468071140 0x7f546806d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:02:41.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.740+0000 7f546e200640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54680c5100 0x7f546814f560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5460007e60 tx=0x7f546000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.740+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460014420 con 0x7f54680c5100 2024-03-09T09:02:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.740+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460019070 con 0x7f54680c5100 2024-03-09T09:02:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.740+0000 7f5470200640 1 -- 172.21.15.67:0/33479679 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54680c2ed0 con 0x7f54680c5100 2024-03-09T09:02:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.740+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460014820 con 0x7f54680c5100 2024-03-09T09:02:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.741+0000 7f5470200640 1 -- 172.21.15.67:0/33479679 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546806daa0 con 0x7f54680c5100 2024-03-09T09:02:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.742+0000 7f5470200640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546814faa0 0x7f546806c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.742+0000 7f5470200640 1 -- 172.21.15.67:0/33479679 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54680ad530 con 0x7f546814faa0 2024-03-09T09:02:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.742+0000 7f546d800640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546814faa0 0x7f546806c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.743+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f54600149c0 con 0x7f54680c5100 2024-03-09T09:02:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.743+0000 7f546d800640 1 -- 172.21.15.67:0/33479679 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54680c29a0 con 0x7f546814faa0 2024-03-09T09:02:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.744+0000 7f5465600640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f544806b500 0x7f544806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.744+0000 7f546d800640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546814faa0 0x7f546806c530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f545c0011e0 tx=0x7f545c0018b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.744+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f546008ae30 con 0x7f54680c5100 2024-03-09T09:02:41.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.744+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c019070 con 0x7f546814faa0 2024-03-09T09:02:41.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.744+0000 7f546ce00640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f544806b500 0x7f544806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.745+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f545c010560 con 0x7f546814faa0 2024-03-09T09:02:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.745+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546814faa0 msgr2=0x7f546806c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.745+0000 7f5465600640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546814faa0 0x7f546806c530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f545c0011e0 tx=0x7f545c0018b0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.745+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546814faa0 msgr2=0x7f546806c530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.745+0000 7f5465600640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546814faa0 0x7f546806c530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.745+0000 7f546ce00640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f544806b500 0x7f544806d9c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5454036000 tx=0x7f5454008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.752+0000 7f5470200640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546806ca70 0x7f54681ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.752+0000 7f5470200640 1 -- 172.21.15.67:0/33479679 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546809c280 con 0x7f546806ca70 2024-03-09T09:02:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.753+0000 7f546d800640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546806ca70 0x7f54681ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.754+0000 7f546d800640 1 -- 172.21.15.67:0/33479679 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f546806c200 con 0x7f546806ca70 2024-03-09T09:02:41.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.754+0000 7f546d800640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546806ca70 0x7f54681ee8d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f545c005fd0 tx=0x7f545c001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:41.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.755+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c019070 con 0x7f546806ca70 2024-03-09T09:02:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.755+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5186 (secure 0 0 0) 0x7f545c01a040 con 0x7f546806ca70 2024-03-09T09:02:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.755+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546806ca70 msgr2=0x7f54681ee8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.755+0000 7f5465600640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546806ca70 0x7f54681ee8d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f545c005fd0 tx=0x7f545c001080 comp rx=0 tx=0).stop 2024-03-09T09:02:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.755+0000 7f5465600640 1 -- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546806ca70 msgr2=0x7f54681ee8d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:41.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.755+0000 7f5465600640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f546806ca70 0x7f54681ee8d0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f545c005fd0 tx=0x7f545c001080 comp rx=0 tx=0).stop 2024-03-09T09:02:41.759 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:02:41.759 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:02:41.759 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:02:41.759 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:02:41.759 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 80, 2024-03-09T09:02:41.759 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:02:41.759 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:02:41.759 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:02:41.759 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:02:41.759 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.759 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:41.760 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:41.761 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:02:41.762 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:02:41.763 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:02:41.763 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:02:41.763 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:02:41.763 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:41.763 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:02:41.763 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:41.763 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:41.763 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:41.763 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:41.764 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:41.764 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:41.764 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:41.764 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:41.764 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:41.764 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:41.764 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:41.764 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.764 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:02:41.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.765 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:02:41.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.765 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:02:41.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:02:41.765 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:41.765 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:41.766 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:41.766 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:02:41.766 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.766 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:41.767 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:02:41.767 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.767 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.767 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.767 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:02:41.767 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:02:41.768 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:41.768 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:41.768 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:41.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.768 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.768 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:02:41.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:02:41.768 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:02:41.769 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:02:41.770 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:41.770 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:41.770 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:41.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.770 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.770 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:02:41.770 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:02:41.770 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:41.770 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:41.770 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:02:41.771 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:02:41.772 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:41.772 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:41.772 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:41.772 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.772 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.772 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:02:41.772 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:02:41.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.772 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:02:41.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.774 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:41.774 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.774 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:02:41.774 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.774 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.774 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:02:41.774 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.774 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:02:41.774 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.774 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:41.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.775 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.775 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:41.775 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.775 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:02:41.775 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.775 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:41.775 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:02:41.775 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:41.775 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:41.776 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:41.776 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:41.776 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:41.776 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:41.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:41.776 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:02:41.776 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:02:41.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.756+0000 7f5470200640 1 -- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f544806b500 msgr2=0x7f544806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.756+0000 7f5470200640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f544806b500 0x7f544806d9c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5454036000 tx=0x7f5454008030 comp rx=0 tx=0).stop 2024-03-09T09:02:41.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.756+0000 7f5470200640 1 -- 172.21.15.67:0/33479679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54680c5100 msgr2=0x7f546814f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:41.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.756+0000 7f5470200640 1 --2- 172.21.15.67:0/33479679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54680c5100 0x7f546814f560 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5460007e60 tx=0x7f546000e4b0 comp rx=0 tx=0).stop 2024-03-09T09:02:41.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.756+0000 7f546e200640 1 -- 172.21.15.67:0/33479679 reap_dead start 2024-03-09T09:02:41.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.756+0000 7f546e200640 1 -- 172.21.15.67:0/33479679 reap_dead start 2024-03-09T09:02:41.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.756+0000 7f5470200640 1 -- 172.21.15.67:0/33479679 shutdown_connections 2024-03-09T09:02:41.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.756+0000 7f5470200640 1 -- 172.21.15.67:0/33479679 >> 172.21.15.67:0/33479679 conn(0x7f54680ab320 msgr2=0x7f5468156350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:41.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.756+0000 7f5470200640 1 -- 172.21.15.67:0/33479679 shutdown_connections 2024-03-09T09:02:41.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:41.756+0000 7f5470200640 1 -- 172.21.15.67:0/33479679 wait complete. 2024-03-09T09:02:41.778 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:02:41.779 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:02:41.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:02:42.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.001+0000 7f2cf2a00640 1 Processor -- start 2024-03-09T09:02:42.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.001+0000 7f2cf2a00640 1 -- start start 2024-03-09T09:02:42.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.003+0000 7f2cf2a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2cec06e2f0 0x7f2cec0aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:42.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.003+0000 7f2cf2a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec0ab3c0 con 0x7f2cec152a20 2024-03-09T09:02:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.003+0000 7f2cf2a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec0aab20 con 0x7f2cec0c30f0 2024-03-09T09:02:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.003+0000 7f2cf2a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cec0c82d0 con 0x7f2cec06e2f0 2024-03-09T09:02:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.004+0000 7f2ce3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2cec152a20 0x7f2cec06dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46602/0 (socket says 172.21.15.67:46602) 2024-03-09T09:02:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.004+0000 7f2ce3e00640 1 -- 172.21.15.67:0/3533576329 learned_addr learned my addr 172.21.15.67:0/3533576329 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:02:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.004+0000 7f2cf0a00640 1 --2- 172.21.15.67:0/3533576329 >> v2:172.21.15.134:3300/0 conn(0x7f2cec06e2f0 0x7f2cec0aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.004+0000 7f2ceb400640 1 -- 172.21.15.67:0/3533576329 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321336219 0 0) 0x7f2cd4001110 con 0x7f2cec152a20 2024-03-09T09:02:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.004+0000 7f2ceb400640 1 -- 172.21.15.67:0/3533576329 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd0003520 con 0x7f2cec152a20 2024-03-09T09:02:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.005+0000 7f2cf0a00640 1 -- 172.21.15.67:0/3533576329 >> v1:172.21.15.134:6789/0 conn(0x7f2cec0c30f0 legacy=0x7f2cec1490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.005+0000 7f2cf0a00640 1 -- 172.21.15.67:0/3533576329 >> v1:172.21.15.67:6789/0 conn(0x7f2cec152a20 legacy=0x7f2cec06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:42.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.005+0000 7f2cf0a00640 1 -- 172.21.15.67:0/3533576329 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0aae00 con 0x7f2cec06e2f0 2024-03-09T09:02:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.006+0000 7f2cf0a00640 1 --2- 172.21.15.67:0/3533576329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec06e2f0 0x7f2cec0aa420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009d90 tx=0x7f2ce402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940cead1e7e7ba1c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.006+0000 7f2ceb400640 1 -- 172.21.15.67:0/3533576329 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4039070 con 0x7f2cec06e2f0 2024-03-09T09:02:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.007+0000 7f2ceb400640 1 -- 172.21.15.67:0/3533576329 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce402cc60 con 0x7f2cec06e2f0 2024-03-09T09:02:42.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.007+0000 7f2ceb400640 1 -- 172.21.15.67:0/3533576329 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4034640 con 0x7f2cec06e2f0 2024-03-09T09:02:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.007+0000 7f2cf2a00640 1 -- 172.21.15.67:0/3533576329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec06e2f0 msgr2=0x7f2cec0aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.007+0000 7f2cf2a00640 1 --2- 172.21.15.67:0/3533576329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec06e2f0 0x7f2cec0aa420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009d90 tx=0x7f2ce402beb0 comp rx=0 tx=0).stop 2024-03-09T09:02:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.007+0000 7f2cf2a00640 1 -- 172.21.15.67:0/3533576329 shutdown_connections 2024-03-09T09:02:42.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.007+0000 7f2cf2a00640 1 --2- 172.21.15.67:0/3533576329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec06e2f0 0x7f2cec0aa420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:42.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.007+0000 7f2cf2a00640 1 -- 172.21.15.67:0/3533576329 >> 172.21.15.67:0/3533576329 conn(0x7f2cec0c7370 msgr2=0x7f2cec0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:42.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.007+0000 7f2cf2a00640 1 -- 172.21.15.67:0/3533576329 shutdown_connections 2024-03-09T09:02:42.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.007+0000 7f2cf2a00640 1 -- 172.21.15.67:0/3533576329 wait complete. 2024-03-09T09:02:42.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.009+0000 7f2cf2a00640 1 Processor -- start 2024-03-09T09:02:42.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.009+0000 7f2cf2a00640 1 -- start start 2024-03-09T09:02:42.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.010+0000 7f2cf2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06e2f0 0x7f2cec1dd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:42.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.010+0000 7f2cf2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0c30f0 0x7f2cec14ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:42.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.010+0000 7f2cf2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec152a20 0x7f2cec14b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:42.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.010+0000 7f2cf2a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec149800 con 0x7f2cec06e2f0 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.010+0000 7f2cf2a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec06d6c0 con 0x7f2cec152a20 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.010+0000 7f2cf2a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec149980 con 0x7f2cec0c30f0 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.011+0000 7f2cebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06e2f0 0x7f2cec1dd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.011+0000 7f2cebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06e2f0 0x7f2cec1dd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54868/0 (socket says 172.21.15.67:54868) 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.011+0000 7f2cebe00640 1 -- 172.21.15.67:0/2712549280 learned_addr learned my addr 172.21.15.67:0/2712549280 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.011+0000 7f2ce3e00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0c30f0 0x7f2cec14ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.011+0000 7f2cf0a00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec152a20 0x7f2cec14b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.011+0000 7f2cebe00640 1 -- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0c30f0 msgr2=0x7f2cec14ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.011+0000 7f2cebe00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0c30f0 0x7f2cec14ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.011+0000 7f2cebe00640 1 -- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec152a20 msgr2=0x7f2cec14b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.011+0000 7f2cebe00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec152a20 0x7f2cec14b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.011+0000 7f2cebe00640 1 -- 172.21.15.67:0/2712549280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec14d540 con 0x7f2cec06e2f0 2024-03-09T09:02:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.012+0000 7f2cebe00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06e2f0 0x7f2cec1dd030 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00c480 tx=0x7f2cdc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:42.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.012+0000 7f2ce8c00640 1 -- 172.21.15.67:0/2712549280 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0052d0 con 0x7f2cec06e2f0 2024-03-09T09:02:42.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.012+0000 7f2ce8c00640 1 -- 172.21.15.67:0/2712549280 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc005470 con 0x7f2cec06e2f0 2024-03-09T09:02:42.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.012+0000 7f2ce8c00640 1 -- 172.21.15.67:0/2712549280 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc012870 con 0x7f2cec06e2f0 2024-03-09T09:02:42.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.012+0000 7f2cf2a00640 1 -- 172.21.15.67:0/2712549280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec14c800 con 0x7f2cec06e2f0 2024-03-09T09:02:42.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.012+0000 7f2cf2a00640 1 -- 172.21.15.67:0/2712549280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec0ac7a0 con 0x7f2cec06e2f0 2024-03-09T09:02:42.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.013+0000 7f2cf2a00640 1 -- 172.21.15.67:0/2712549280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec1ebd40 con 0x7f2cec06e2f0 2024-03-09T09:02:42.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.014+0000 7f2ce8c00640 1 -- 172.21.15.67:0/2712549280 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2cdc012a10 con 0x7f2cec06e2f0 2024-03-09T09:02:42.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.016+0000 7f2ce8c00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2cc806b500 0x7f2cc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:42.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.016+0000 7f2ce8c00640 1 -- 172.21.15.67:0/2712549280 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f2cdc08a1f0 con 0x7f2cec06e2f0 2024-03-09T09:02:42.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.016+0000 7f2ce3e00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2cc806b500 0x7f2cc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:42.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.017+0000 7f2ce3e00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2cc806b500 0x7f2cc806d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40012b0 tx=0x7f2cd4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:42.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.022+0000 7f2ce8c00640 1 -- 172.21.15.67:0/2712549280 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2cdc058d10 con 0x7f2cec06e2f0 2024-03-09T09:02:42.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.363+0000 7f2cf2a00640 1 -- 172.21.15.67:0/2712549280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f2cec075a00 con 0x7f2cec06e2f0 2024-03-09T09:02:42.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.365+0000 7f2ce8c00640 1 -- 172.21.15.67:0/2712549280 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f2cdc058610 con 0x7f2cec06e2f0 2024-03-09T09:02:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.368+0000 7f2cf2a00640 1 -- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2cc806b500 msgr2=0x7f2cc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.368+0000 7f2cf2a00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2cc806b500 0x7f2cc806d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40012b0 tx=0x7f2cd4009040 comp rx=0 tx=0).stop 2024-03-09T09:02:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.368+0000 7f2cf2a00640 1 -- 172.21.15.67:0/2712549280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06e2f0 msgr2=0x7f2cec1dd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:42.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.368+0000 7f2cf2a00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06e2f0 0x7f2cec1dd030 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00c480 tx=0x7f2cdc00c940 comp rx=0 tx=0).stop 2024-03-09T09:02:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.369+0000 7f2cf2a00640 1 -- 172.21.15.67:0/2712549280 shutdown_connections 2024-03-09T09:02:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.369+0000 7f2cf2a00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2cc806b500 0x7f2cc806d9c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.369+0000 7f2cf2a00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2cec152a20 0x7f2cec14b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.369+0000 7f2cf2a00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2cec0c30f0 0x7f2cec14ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.369+0000 7f2cf2a00640 1 --2- 172.21.15.67:0/2712549280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2cec06e2f0 0x7f2cec1dd030 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.369+0000 7f2cf2a00640 1 -- 172.21.15.67:0/2712549280 >> 172.21.15.67:0/2712549280 conn(0x7f2cec0c7370 msgr2=0x7f2cec0c5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.369+0000 7f2cf2a00640 1 -- 172.21.15.67:0/2712549280 shutdown_connections 2024-03-09T09:02:42.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:42.369+0000 7f2cf2a00640 1 -- 172.21.15.67:0/2712549280 wait complete. 2024-03-09T09:02:42.389 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:02:44.034 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:02:44.041 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:02:52.390 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:02:52.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:02:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.606+0000 7f9883e00640 1 Processor -- start 2024-03-09T09:02:52.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.606+0000 7f9883e00640 1 -- start start 2024-03-09T09:02:52.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.608+0000 7f9883e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f98840c1f70 0x7f9884150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.608+0000 7f9883e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98840c81e0 con 0x7f98840710a0 2024-03-09T09:02:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.608+0000 7f9883e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98840c32b0 con 0x7f9884154080 2024-03-09T09:02:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.608+0000 7f9883e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98840c2fd0 con 0x7f98840c1f70 2024-03-09T09:02:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.608+0000 7f9882a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f98840c1f70 0x7f9884150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.608+0000 7f9882000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f98840710a0 0x7f9884153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53340/0 (socket says 172.21.15.67:53340) 2024-03-09T09:02:52.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.608+0000 7f9882000640 1 -- 172.21.15.67:0/581476492 learned_addr learned my addr 172.21.15.67:0/581476492 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:02:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.609+0000 7f9882a00640 1 -- 172.21.15.67:0/581476492 >> v1:172.21.15.134:6789/0 conn(0x7f9884154080 legacy=0x7f988415e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.609+0000 7f9882a00640 1 -- 172.21.15.67:0/581476492 >> v1:172.21.15.67:6789/0 conn(0x7f98840710a0 legacy=0x7f9884153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:52.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.609+0000 7f9882a00640 1 -- 172.21.15.67:0/581476492 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840c3700 con 0x7f98840c1f70 2024-03-09T09:02:52.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.611+0000 7f9882a00640 1 --2- 172.21.15.67:0/581476492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98840c1f70 0x7f9884150c50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f98740097a0 tx=0x7f987402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bff20afa579e13f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:52.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.611+0000 7f9881600640 1 -- 172.21.15.67:0/581476492 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874035020 con 0x7f98840c1f70 2024-03-09T09:02:52.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.611+0000 7f9881600640 1 -- 172.21.15.67:0/581476492 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9874031830 con 0x7f98840c1f70 2024-03-09T09:02:52.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.611+0000 7f9881600640 1 -- 172.21.15.67:0/581476492 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874031c00 con 0x7f98840c1f70 2024-03-09T09:02:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.612+0000 7f9883e00640 1 -- 172.21.15.67:0/581476492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98840c1f70 msgr2=0x7f9884150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.612+0000 7f9883e00640 1 --2- 172.21.15.67:0/581476492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98840c1f70 0x7f9884150c50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f98740097a0 tx=0x7f987402b8d0 comp rx=0 tx=0).stop 2024-03-09T09:02:52.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.612+0000 7f9883e00640 1 -- 172.21.15.67:0/581476492 shutdown_connections 2024-03-09T09:02:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.612+0000 7f9883e00640 1 --2- 172.21.15.67:0/581476492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98840c1f70 0x7f9884150c50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.612+0000 7f9883e00640 1 -- 172.21.15.67:0/581476492 >> 172.21.15.67:0/581476492 conn(0x7f98840c7300 msgr2=0x7f98840c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.613+0000 7f9883e00640 1 -- 172.21.15.67:0/581476492 shutdown_connections 2024-03-09T09:02:52.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.613+0000 7f9883e00640 1 -- 172.21.15.67:0/581476492 wait complete. 2024-03-09T09:02:52.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.615+0000 7f9883e00640 1 Processor -- start 2024-03-09T09:02:52.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.615+0000 7f9883e00640 1 -- start start 2024-03-09T09:02:52.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.616+0000 7f9883e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98840710a0 0x7f98840c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:52.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.616+0000 7f9883e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98840c1f70 0x7f98840ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:52.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.616+0000 7f9883e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9884154080 0x7f9884152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:52.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.616+0000 7f9883e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9884070d50 con 0x7f9884154080 2024-03-09T09:02:52.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.616+0000 7f9883e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9884070830 con 0x7f98840c1f70 2024-03-09T09:02:52.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.616+0000 7f9883e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9884070ed0 con 0x7f98840710a0 2024-03-09T09:02:52.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.617+0000 7f9882000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98840c1f70 0x7f98840ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:52.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.617+0000 7f9883400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9884154080 0x7f9884152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:52.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.617+0000 7f9882a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98840710a0 0x7f98840c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:52.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.617+0000 7f9882000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98840c1f70 0x7f98840ab770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52450/0 (socket says 172.21.15.67:52450) 2024-03-09T09:02:52.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.617+0000 7f9882000640 1 -- 172.21.15.67:0/1663954339 learned_addr learned my addr 172.21.15.67:0/1663954339 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.617+0000 7f9882000640 1 -- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98840710a0 msgr2=0x7f98840c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.617+0000 7f9882000640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98840710a0 0x7f98840c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.617+0000 7f9882000640 1 -- 172.21.15.67:0/1663954339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9884154080 msgr2=0x7f9884152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.617+0000 7f9882000640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9884154080 0x7f9884152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.617+0000 7f9882000640 1 -- 172.21.15.67:0/1663954339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840c69b0 con 0x7f98840c1f70 2024-03-09T09:02:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.618+0000 7f9882000640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98840c1f70 0x7f98840ab770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c3c0 tx=0x7f986c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.618+0000 7f987b400640 1 -- 172.21.15.67:0/1663954339 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c017070 con 0x7f98840c1f70 2024-03-09T09:02:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.618+0000 7f987b400640 1 -- 172.21.15.67:0/1663954339 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c005ce0 con 0x7f98840c1f70 2024-03-09T09:02:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.618+0000 7f9883e00640 1 -- 172.21.15.67:0/1663954339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9884151c40 con 0x7f98840c1f70 2024-03-09T09:02:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.618+0000 7f987b400640 1 -- 172.21.15.67:0/1663954339 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c0072f0 con 0x7f98840c1f70 2024-03-09T09:02:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.619+0000 7f9883e00640 1 -- 172.21.15.67:0/1663954339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9884152200 con 0x7f98840c1f70 2024-03-09T09:02:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.620+0000 7f9883e00640 1 -- 172.21.15.67:0/1663954339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988404f3c0 con 0x7f98840c1f70 2024-03-09T09:02:52.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.622+0000 7f987b400640 1 -- 172.21.15.67:0/1663954339 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f986c01a070 con 0x7f98840c1f70 2024-03-09T09:02:52.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.623+0000 7f987b400640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f986006b500 0x7f986006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:52.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.623+0000 7f987b400640 1 -- 172.21.15.67:0/1663954339 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f986c0898e0 con 0x7f98840c1f70 2024-03-09T09:02:52.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.623+0000 7f9882a00640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f986006b500 0x7f986006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:52.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.624+0000 7f9882a00640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f986006b500 0x7f986006d9c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9874036000 tx=0x7f9874008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:52.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.630+0000 7f987b400640 1 -- 172.21.15.67:0/1663954339 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f986c057d20 con 0x7f98840c1f70 2024-03-09T09:02:52.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.976+0000 7f9883e00640 1 -- 172.21.15.67:0/1663954339 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f98840c4840 con 0x7f98840c1f70 2024-03-09T09:02:52.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.977+0000 7f987b400640 1 -- 172.21.15.67:0/1663954339 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f986c05bdc0 con 0x7f98840c1f70 2024-03-09T09:02:52.980 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":80,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:02:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 -- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f986006b500 msgr2=0x7f986006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f986006b500 0x7f986006d9c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9874036000 tx=0x7f9874008030 comp rx=0 tx=0).stop 2024-03-09T09:02:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 -- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98840c1f70 msgr2=0x7f98840ab770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98840c1f70 0x7f98840ab770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c3c0 tx=0x7f986c00c880 comp rx=0 tx=0).stop 2024-03-09T09:02:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 -- 172.21.15.67:0/1663954339 shutdown_connections 2024-03-09T09:02:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f986006b500 0x7f986006d9c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9884154080 0x7f9884152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:52.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98840c1f70 0x7f98840ab770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 --2- 172.21.15.67:0/1663954339 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98840710a0 0x7f98840c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 -- 172.21.15.67:0/1663954339 >> 172.21.15.67:0/1663954339 conn(0x7f98840c7300 msgr2=0x7f98840c3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 -- 172.21.15.67:0/1663954339 shutdown_connections 2024-03-09T09:02:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:52.981+0000 7f9883e00640 1 -- 172.21.15.67:0/1663954339 wait complete. 2024-03-09T09:02:53.000 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:02:53.001 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:02:53.001 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:02:53.001 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:02:53.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.225+0000 7f88cf200640 1 Processor -- start 2024-03-09T09:02:53.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.225+0000 7f88cf200640 1 -- start start 2024-03-09T09:02:53.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.227+0000 7f88cf200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 0x7f88c80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.227+0000 7f88cf200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f88c8151360 0x7f88c81539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.227+0000 7f88cf200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c80c2780 con 0x7f88c80c30a0 2024-03-09T09:02:53.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.227+0000 7f88cf200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c8070670 con 0x7f88c8151360 2024-03-09T09:02:53.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.227+0000 7f88cf200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80708c0 con 0x7f88c80ac930 2024-03-09T09:02:53.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.228+0000 7f88ce800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f88c8151360 0x7f88c81539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.228+0000 7f88cd400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f88c80c30a0 0x7f88c8150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53346/0 (socket says 172.21.15.67:53346) 2024-03-09T09:02:53.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.228+0000 7f88cd400640 1 -- 172.21.15.67:0/2927993538 learned_addr learned my addr 172.21.15.67:0/2927993538 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:02:53.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.228+0000 7f88cde00640 1 --2- 172.21.15.67:0/2927993538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 0x7f88c80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.228+0000 7f88ce800640 1 -- 172.21.15.67:0/2927993538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 msgr2=0x7f88c80c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:53.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.228+0000 7f88ce800640 1 --2- 172.21.15.67:0/2927993538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 0x7f88c80c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.228+0000 7f88ce800640 1 -- 172.21.15.67:0/2927993538 >> v1:172.21.15.67:6789/0 conn(0x7f88c80c30a0 legacy=0x7f88c8150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.228+0000 7f88ce800640 1 -- 172.21.15.67:0/2927993538 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8070ba0 con 0x7f88c8151360 2024-03-09T09:02:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.229+0000 7f88ce800640 1 --2- 172.21.15.67:0/2927993538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c8151360 0x7f88c81539e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009d90 tx=0x7f88c402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c4b3e51eedfb60 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.230+0000 7f88cca00640 1 -- 172.21.15.67:0/2927993538 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4039070 con 0x7f88c8151360 2024-03-09T09:02:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.230+0000 7f88cca00640 1 -- 172.21.15.67:0/2927993538 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c402cc50 con 0x7f88c8151360 2024-03-09T09:02:53.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.230+0000 7f88cca00640 1 -- 172.21.15.67:0/2927993538 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4034440 con 0x7f88c8151360 2024-03-09T09:02:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.230+0000 7f88cf200640 1 -- 172.21.15.67:0/2927993538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c8151360 msgr2=0x7f88c81539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.230+0000 7f88cf200640 1 --2- 172.21.15.67:0/2927993538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c8151360 0x7f88c81539e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009d90 tx=0x7f88c402bea0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.231+0000 7f88cf200640 1 -- 172.21.15.67:0/2927993538 shutdown_connections 2024-03-09T09:02:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.231+0000 7f88cf200640 1 --2- 172.21.15.67:0/2927993538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c8151360 0x7f88c81539e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.231+0000 7f88cf200640 1 --2- 172.21.15.67:0/2927993538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 0x7f88c80c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.231+0000 7f88cf200640 1 -- 172.21.15.67:0/2927993538 >> 172.21.15.67:0/2927993538 conn(0x7f88c80c7220 msgr2=0x7f88c80c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.231+0000 7f88cf200640 1 -- 172.21.15.67:0/2927993538 shutdown_connections 2024-03-09T09:02:53.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.231+0000 7f88cf200640 1 -- 172.21.15.67:0/2927993538 wait complete. 2024-03-09T09:02:53.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.232+0000 7f88cf200640 1 Processor -- start 2024-03-09T09:02:53.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.233+0000 7f88cf200640 1 -- start start 2024-03-09T09:02:53.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.234+0000 7f88cf200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 0x7f88c81e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.234+0000 7f88cf200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c80c30a0 0x7f88c8151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.234+0000 7f88cf200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c8151360 0x7f88c8152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.234+0000 7f88cf200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80ace90 con 0x7f88c80c30a0 2024-03-09T09:02:53.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.234+0000 7f88cf200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80acd10 con 0x7f88c8151360 2024-03-09T09:02:53.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.234+0000 7f88cf200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c8153f20 con 0x7f88c80ac930 2024-03-09T09:02:53.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.235+0000 7f88cde00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 0x7f88c81e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.235+0000 7f88cde00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 0x7f88c81e86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:38042/0 (socket says 172.21.15.67:38042) 2024-03-09T09:02:53.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.235+0000 7f88cde00640 1 -- 172.21.15.67:0/3432111383 learned_addr learned my addr 172.21.15.67:0/3432111383 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:53.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.236+0000 7f88ce800640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c8151360 0x7f88c8152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.236+0000 7f88cd400640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c80c30a0 0x7f88c8151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.236+0000 7f88cde00640 1 -- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c8151360 msgr2=0x7f88c8152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.236+0000 7f88cde00640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88c8151360 0x7f88c8152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.236+0000 7f88cde00640 1 -- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c80c30a0 msgr2=0x7f88c8151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.236+0000 7f88cde00640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c80c30a0 0x7f88c8151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.236+0000 7f88cde00640 1 -- 172.21.15.67:0/3432111383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c81e8fc0 con 0x7f88c80ac930 2024-03-09T09:02:53.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.238+0000 7f88cde00640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 0x7f88c81e86a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00e9b0 tx=0x7f88bc00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.239+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc017070 con 0x7f88c80ac930 2024-03-09T09:02:53.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.239+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc00c9a0 con 0x7f88c80ac930 2024-03-09T09:02:53.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.239+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc00cd90 con 0x7f88c80ac930 2024-03-09T09:02:53.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.239+0000 7f88cf200640 1 -- 172.21.15.67:0/3432111383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c81ec180 con 0x7f88c80ac930 2024-03-09T09:02:53.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.239+0000 7f88cf200640 1 -- 172.21.15.67:0/3432111383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c814b670 con 0x7f88c80ac930 2024-03-09T09:02:53.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.241+0000 7f88cf200640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c8152570 0x7f88c81eead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.241+0000 7f88cf200640 1 -- 172.21.15.67:0/3432111383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c81ebfe0 con 0x7f88c8152570 2024-03-09T09:02:53.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.241+0000 7f88cd400640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c8152570 0x7f88c81eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.242+0000 7f88cd400640 1 -- 172.21.15.67:0/3432111383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88c804f3c0 con 0x7f88c8152570 2024-03-09T09:02:53.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.243+0000 7f88cd400640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c8152570 0x7f88c81eead0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f88c8144670 tx=0x7f88b800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.243+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8019070 con 0x7f88c8152570 2024-03-09T09:02:53.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.244+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f88b8005ac0 con 0x7f88c8152570 2024-03-09T09:02:53.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.244+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c8152570 msgr2=0x7f88c81eead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.244+0000 7f88c2000640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c8152570 0x7f88c81eead0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f88c8144670 tx=0x7f88b800e960 comp rx=0 tx=0).stop 2024-03-09T09:02:53.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.244+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c8152570 msgr2=0x7f88c81eead0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:53.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.244+0000 7f88c2000640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c8152570 0x7f88c81eead0 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f88c8144670 tx=0x7f88b800e960 comp rx=0 tx=0).stop 2024-03-09T09:02:53.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.256+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f88bc01a070 con 0x7f88c80ac930 2024-03-09T09:02:53.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.258+0000 7f88c2000640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f88ac06b030 0x7f88ac06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.258+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f88bc08a920 con 0x7f88c80ac930 2024-03-09T09:02:53.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.260+0000 7f88cd400640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f88ac06b030 0x7f88ac06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.264+0000 7f88cd400640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f88ac06b030 0x7f88ac06d4f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f88c8144670 tx=0x7f88b8007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.270+0000 7f88cf200640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c81ef010 0x7f88c81ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.270+0000 7f88cf200640 1 -- 172.21.15.67:0/3432111383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c81ef930 con 0x7f88c81ef010 2024-03-09T09:02:53.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.271+0000 7f88ce800640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c81ef010 0x7f88c81ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.272+0000 7f88ce800640 1 -- 172.21.15.67:0/3432111383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f88c8151850 con 0x7f88c81ef010 2024-03-09T09:02:53.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.273+0000 7f88ce800640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c81ef010 0x7f88c81ef3f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88c806eb80 tx=0x7f88c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.273+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4044070 con 0x7f88c81ef010 2024-03-09T09:02:53.277 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 80, 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:53.278 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:02:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:02:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:53.279 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:02:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:02:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:53.280 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:02:53.281 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:02:53.282 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.283 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:53.285 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.287 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:53.287 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:02:53.287 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.287 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.287 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.287 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:02:53.287 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:02:53.287 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:53.287 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.289 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.289 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:53.289 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:02:53.289 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.289 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.289 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.289 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:02:53.289 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:02:53.289 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:53.289 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.290 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.291 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:02:53.291 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.291 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:53.291 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.291 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:53.291 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.291 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.291 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:02:53.291 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:02:53.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.274+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4536 (secure 0 0 0) 0x7f88c403f440 con 0x7f88c81ef010 2024-03-09T09:02:53.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.274+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c81ef010 msgr2=0x7f88c81ef3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.274+0000 7f88c2000640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c81ef010 0x7f88c81ef3f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88c806eb80 tx=0x7f88c4004a40 comp rx=0 tx=0).stop 2024-03-09T09:02:53.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.274+0000 7f88c2000640 1 -- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c81ef010 msgr2=0x7f88c81ef3f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:53.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.274+0000 7f88c2000640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88c81ef010 0x7f88c81ef3f0 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88c806eb80 tx=0x7f88c4004a40 comp rx=0 tx=0).stop 2024-03-09T09:02:53.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.276+0000 7f88b7400640 1 -- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f88ac06b030 msgr2=0x7f88ac06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.276+0000 7f88b7400640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f88ac06b030 0x7f88ac06d4f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f88c8144670 tx=0x7f88b8007000 comp rx=0 tx=0).stop 2024-03-09T09:02:53.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.276+0000 7f88b7400640 1 -- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 msgr2=0x7f88c81e86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.276+0000 7f88b7400640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 0x7f88c81e86a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00e9b0 tx=0x7f88bc00ee70 comp rx=0 tx=0).stop 2024-03-09T09:02:53.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.276+0000 7f88ce800640 1 -- 172.21.15.67:0/3432111383 reap_dead start 2024-03-09T09:02:53.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.276+0000 7f88b7400640 1 -- 172.21.15.67:0/3432111383 shutdown_connections 2024-03-09T09:02:53.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.276+0000 7f88b7400640 1 --2- 172.21.15.67:0/3432111383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88c80ac930 0x7f88c81e86a0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00e9b0 tx=0x7f88bc00ee70 comp rx=0 tx=0).stop 2024-03-09T09:02:53.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.276+0000 7f88b7400640 1 -- 172.21.15.67:0/3432111383 >> 172.21.15.67:0/3432111383 conn(0x7f88c80c7220 msgr2=0x7f88c814e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:53.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.278+0000 7f88b7400640 1 -- 172.21.15.67:0/3432111383 shutdown_connections 2024-03-09T09:02:53.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.279+0000 7f88b7400640 1 -- 172.21.15.67:0/3432111383 wait complete. 2024-03-09T09:02:53.304 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:02:53.305 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:02:53.305 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:02:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.518+0000 7f26bda00640 1 Processor -- start 2024-03-09T09:02:53.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.518+0000 7f26bda00640 1 -- start start 2024-03-09T09:02:53.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.520+0000 7f26bda00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26b8071190 0x7f26b81548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.520+0000 7f26bda00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b80c8200 con 0x7f26b80c2780 2024-03-09T09:02:53.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.520+0000 7f26bda00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b80c3990 con 0x7f26b80ac9a0 2024-03-09T09:02:53.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.520+0000 7f26bda00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b806d590 con 0x7f26b8071190 2024-03-09T09:02:53.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.520+0000 7f26b6000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f26b80c2780 0x7f26b8154190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53350/0 (socket says 172.21.15.67:53350) 2024-03-09T09:02:53.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.520+0000 7f26b6000640 1 -- 172.21.15.67:0/4223582034 learned_addr learned my addr 172.21.15.67:0/4223582034 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:02:53.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.521+0000 7f26b7400640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26b8071190 0x7f26b81548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.521+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634097706 0 0) 0x7f26b80c8200 con 0x7f26b80c2780 2024-03-09T09:02:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.521+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f269c003520 con 0x7f26b80c2780 2024-03-09T09:02:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.521+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041955502 0 0) 0x7f26b80c3990 con 0x7f26b80ac9a0 2024-03-09T09:02:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.521+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b80c8200 con 0x7f26b80ac9a0 2024-03-09T09:02:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.521+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 28532115 0 0) 0x7f269c003520 con 0x7f26b80c2780 2024-03-09T09:02:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.521+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26b8071190 msgr2=0x7f26b81548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.521+0000 7f26b5600640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26b8071190 0x7f26b81548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.521+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 >> v1:172.21.15.134:6789/0 conn(0x7f26b80ac9a0 legacy=0x7f26b81510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.521+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b8070920 con 0x7f26b80c2780 2024-03-09T09:02:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.521+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f26a4001670 con 0x7f26b80c2780 2024-03-09T09:02:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 >> v1:172.21.15.67:6789/0 conn(0x7f26b80c2780 legacy=0x7f26b8154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b5600640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f269c003b70 0x7f269c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b5600640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c004800 0x7f269c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b5600640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c0051c0 0x7f269c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c00c000 con 0x7f269c003b70 2024-03-09T09:02:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c00c1b0 con 0x7f269c004800 2024-03-09T09:02:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c00c3b0 con 0x7f269c0051c0 2024-03-09T09:02:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b7400640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26b8071190 0x7f26b81548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:02:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b6a00640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f269c003b70 0x7f269c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b6000640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c004800 0x7f269c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b7400640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c0051c0 0x7f269c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b6a00640 1 -- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c0051c0 msgr2=0x7f269c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b6a00640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f269c0051c0 0x7f269c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b6a00640 1 -- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c004800 msgr2=0x7f269c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b6a00640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c004800 0x7f269c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.522+0000 7f26b6a00640 1 -- 172.21.15.67:0/4223582034 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c003520 con 0x7f269c003b70 2024-03-09T09:02:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.524+0000 7f26b6a00640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f269c003b70 0x7f269c003f70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f26a80018c0 tx=0x7f269c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a401a776c12ec12c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.524+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8037070 con 0x7f269c003b70 2024-03-09T09:02:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.524+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a80052f0 con 0x7f269c003b70 2024-03-09T09:02:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.524+0000 7f26b5600640 1 -- 172.21.15.67:0/4223582034 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a80014e0 con 0x7f269c003b70 2024-03-09T09:02:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.525+0000 7f26bda00640 1 -- 172.21.15.67:0/4223582034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f269c003b70 msgr2=0x7f269c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.525+0000 7f26bda00640 1 --2- 172.21.15.67:0/4223582034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f269c003b70 0x7f269c003f70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f26a80018c0 tx=0x7f269c005830 comp rx=0 tx=0).stop 2024-03-09T09:02:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.525+0000 7f26b7400640 1 -- 172.21.15.67:0/4223582034 reap_dead start 2024-03-09T09:02:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.526+0000 7f26bda00640 1 -- 172.21.15.67:0/4223582034 shutdown_connections 2024-03-09T09:02:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.526+0000 7f26bda00640 1 -- 172.21.15.67:0/4223582034 >> 172.21.15.67:0/4223582034 conn(0x7f26b80c7370 msgr2=0x7f26b80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.526+0000 7f26bda00640 1 -- 172.21.15.67:0/4223582034 shutdown_connections 2024-03-09T09:02:53.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.526+0000 7f26bda00640 1 -- 172.21.15.67:0/4223582034 wait complete. 2024-03-09T09:02:53.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.528+0000 7f26bda00640 1 Processor -- start 2024-03-09T09:02:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.528+0000 7f26bda00640 1 -- start start 2024-03-09T09:02:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.529+0000 7f26bda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c003b70 0x7f26b814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.529+0000 7f26bda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b81530d0 0x7f26b81476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.529+0000 7f26bda00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26b8147c00 0x7f26b814bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.530+0000 7f26bda00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80acd80 con 0x7f26b81530d0 2024-03-09T09:02:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.530+0000 7f26bda00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80c3740 con 0x7f269c003b70 2024-03-09T09:02:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.530+0000 7f26bda00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80acf00 con 0x7f26b8147c00 2024-03-09T09:02:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.530+0000 7f26b6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b81530d0 0x7f26b81476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.530+0000 7f26b6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b81530d0 0x7f26b81476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53480/0 (socket says 172.21.15.67:53480) 2024-03-09T09:02:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.530+0000 7f26b6000640 1 -- 172.21.15.67:0/1230253032 learned_addr learned my addr 172.21.15.67:0/1230253032 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.530+0000 7f26b6a00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c003b70 0x7f26b814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.530+0000 7f26b7400640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26b8147c00 0x7f26b814bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.531+0000 7f26b6000640 1 -- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26b8147c00 msgr2=0x7f26b814bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.531+0000 7f26b6000640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26b8147c00 0x7f26b814bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.531+0000 7f26b6000640 1 -- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c003b70 msgr2=0x7f26b814ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.531+0000 7f26b6000640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f269c003b70 0x7f26b814ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.531+0000 7f26b6000640 1 -- 172.21.15.67:0/1230253032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b8148080 con 0x7f26b81530d0 2024-03-09T09:02:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.531+0000 7f26b6000640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b81530d0 0x7f26b81476c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26a4000f60 tx=0x7f26a400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.532+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4012590 con 0x7f26b81530d0 2024-03-09T09:02:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.532+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a4018070 con 0x7f26b81530d0 2024-03-09T09:02:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.532+0000 7f26bda00640 1 -- 172.21.15.67:0/1230253032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b8150250 con 0x7f26b81530d0 2024-03-09T09:02:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.532+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4012970 con 0x7f26b81530d0 2024-03-09T09:02:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.532+0000 7f26bda00640 1 -- 172.21.15.67:0/1230253032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b814f770 con 0x7f26b81530d0 2024-03-09T09:02:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.533+0000 7f26bda00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b814f970 0x7f26b814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.533+0000 7f26bda00640 1 -- 172.21.15.67:0/1230253032 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b814f5f0 con 0x7f26b814f970 2024-03-09T09:02:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.534+0000 7f26b6a00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b814f970 0x7f26b814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.535+0000 7f26b6a00640 1 -- 172.21.15.67:0/1230253032 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26b814c9b0 con 0x7f26b814f970 2024-03-09T09:02:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.535+0000 7f26b6a00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b814f970 0x7f26b814c470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f26a8001890 tx=0x7f26a80012d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.536+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8043030 con 0x7f26b814f970 2024-03-09T09:02:53.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.536+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f26a4012b10 con 0x7f26b81530d0 2024-03-09T09:02:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.537+0000 7f26aea00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f269806b3d0 0x7f269806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.538+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f26a8038040 con 0x7f26b814f970 2024-03-09T09:02:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.538+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b814f970 msgr2=0x7f26b814c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.538+0000 7f26aea00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b814f970 0x7f26b814c470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f26a8001890 tx=0x7f26a80012d0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.538+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b814f970 msgr2=0x7f26b814c470 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.538+0000 7f26aea00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b814f970 0x7f26b814c470 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f26a8001890 tx=0x7f26a80012d0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.538+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f26a408a150 con 0x7f26b81530d0 2024-03-09T09:02:53.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.538+0000 7f26b7400640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f269806b3d0 0x7f269806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.539+0000 7f26b7400640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f269806b3d0 0x7f269806d890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f26b8070370 tx=0x7f26b000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.549+0000 7f26bda00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b81ee810 0x7f26b81eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.549+0000 7f26bda00640 1 -- 172.21.15.67:0/1230253032 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8075a00 con 0x7f26b81ee810 2024-03-09T09:02:53.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.549+0000 7f26b6a00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b81ee810 0x7f26b81eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.550+0000 7f26b6a00640 1 -- 172.21.15.67:0/1230253032 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f26b8073730 con 0x7f26b81ee810 2024-03-09T09:02:53.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.551+0000 7f26b6a00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b81ee810 0x7f26b81eebf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26b8068060 tx=0x7f26a8031000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.551+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8037070 con 0x7f26b81ee810 2024-03-09T09:02:53.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.552+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5017 (secure 0 0 0) 0x7f26a80038c0 con 0x7f26b81ee810 2024-03-09T09:02:53.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.552+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b81ee810 msgr2=0x7f26b81eebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.552+0000 7f26aea00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b81ee810 0x7f26b81eebf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26b8068060 tx=0x7f26a8031000 comp rx=0 tx=0).stop 2024-03-09T09:02:53.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.552+0000 7f26aea00640 1 -- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b81ee810 msgr2=0x7f26b81eebf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:53.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.552+0000 7f26aea00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26b81ee810 0x7f26b81eebf0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26b8068060 tx=0x7f26a8031000 comp rx=0 tx=0).stop 2024-03-09T09:02:53.554 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:02:53.554 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 80, 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:53.555 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:53.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:53.556 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:53.556 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:53.556 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:53.556 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:53.556 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:53.556 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:53.556 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:53.556 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:53.556 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:53.557 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:02:53.558 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:53.559 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:53.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:02:53.561 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:02:53.562 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:53.562 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:53.562 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:53.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.562 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:02:53.562 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:02:53.562 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.563 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:53.565 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.566 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:53.567 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:02:53.568 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:02:53.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.552+0000 7f26bda00640 1 -- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f269806b3d0 msgr2=0x7f269806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.552+0000 7f26bda00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f269806b3d0 0x7f269806d890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f26b8070370 tx=0x7f26b000b040 comp rx=0 tx=0).stop 2024-03-09T09:02:53.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.552+0000 7f26bda00640 1 -- 172.21.15.67:0/1230253032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b81530d0 msgr2=0x7f26b81476c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.552+0000 7f26bda00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b81530d0 0x7f26b81476c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26a4000f60 tx=0x7f26a400ead0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.552+0000 7f26b7400640 1 -- 172.21.15.67:0/1230253032 reap_dead start 2024-03-09T09:02:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.553+0000 7f26bda00640 1 -- 172.21.15.67:0/1230253032 shutdown_connections 2024-03-09T09:02:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.553+0000 7f26bda00640 1 --2- 172.21.15.67:0/1230253032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26b81530d0 0x7f26b81476c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.553+0000 7f26bda00640 1 -- 172.21.15.67:0/1230253032 >> 172.21.15.67:0/1230253032 conn(0x7f26b80c7370 msgr2=0x7f26b80c1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.553+0000 7f26bda00640 1 -- 172.21.15.67:0/1230253032 shutdown_connections 2024-03-09T09:02:53.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.553+0000 7f26bda00640 1 -- 172.21.15.67:0/1230253032 wait complete. 2024-03-09T09:02:53.573 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:02:53.573 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:02:53.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:02:53.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.766+0000 7f9b8da00640 1 Processor -- start 2024-03-09T09:02:53.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.766+0000 7f9b8da00640 1 -- start start 2024-03-09T09:02:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.767+0000 7f9b8da00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b880734a0 0x7f9b880aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.767+0000 7f9b8da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9b8806d830 0x7f9b8806dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.767+0000 7f9b8da00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b88152e00 con 0x7f9b880730c0 2024-03-09T09:02:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.767+0000 7f9b8da00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b880aa960 con 0x7f9b8806d830 2024-03-09T09:02:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.767+0000 7f9b8da00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881530e0 con 0x7f9b880734a0 2024-03-09T09:02:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.768+0000 7f9b86a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9b880730c0 0x7f9b880c2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53362/0 (socket says 172.21.15.67:53362) 2024-03-09T09:02:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.768+0000 7f9b86a00640 1 -- 172.21.15.67:0/1457908763 learned_addr learned my addr 172.21.15.67:0/1457908763 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:02:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.768+0000 7f9b87400640 1 --2- 172.21.15.67:0/1457908763 >> v2:172.21.15.67:3300/0 conn(0x7f9b8806d830 0x7f9b8806dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.768+0000 7f9b7e000640 1 --2- 172.21.15.67:0/1457908763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b880734a0 0x7f9b880aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.768+0000 7f9b87400640 1 -- 172.21.15.67:0/1457908763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b880734a0 msgr2=0x7f9b880aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.768+0000 7f9b87400640 1 --2- 172.21.15.67:0/1457908763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b880734a0 0x7f9b880aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.768+0000 7f9b87400640 1 -- 172.21.15.67:0/1457908763 >> v1:172.21.15.67:6789/0 conn(0x7f9b880730c0 legacy=0x7f9b880c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.768+0000 7f9b86000640 1 -- 172.21.15.67:0/1457908763 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417655537 0 0) 0x7f9b88152e00 con 0x7f9b880730c0 2024-03-09T09:02:53.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.768+0000 7f9b87400640 1 -- 172.21.15.67:0/1457908763 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b881533c0 con 0x7f9b8806d830 2024-03-09T09:02:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.770+0000 7f9b87400640 1 --2- 172.21.15.67:0/1457908763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8806d830 0x7f9b8806dcc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097a0 tx=0x7f9b8002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a96ec40aaab7c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.770+0000 7f9b86000640 1 -- 172.21.15.67:0/1457908763 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b800315a0 con 0x7f9b8806d830 2024-03-09T09:02:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.770+0000 7f9b86000640 1 -- 172.21.15.67:0/1457908763 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b80037070 con 0x7f9b8806d830 2024-03-09T09:02:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.770+0000 7f9b86000640 1 -- 172.21.15.67:0/1457908763 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80031960 con 0x7f9b8806d830 2024-03-09T09:02:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.770+0000 7f9b8da00640 1 -- 172.21.15.67:0/1457908763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8806d830 msgr2=0x7f9b8806dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.770+0000 7f9b8da00640 1 --2- 172.21.15.67:0/1457908763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8806d830 0x7f9b8806dcc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b800097a0 tx=0x7f9b8002b840 comp rx=0 tx=0).stop 2024-03-09T09:02:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.771+0000 7f9b8da00640 1 -- 172.21.15.67:0/1457908763 shutdown_connections 2024-03-09T09:02:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.771+0000 7f9b8da00640 1 --2- 172.21.15.67:0/1457908763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b8806d830 0x7f9b8806dcc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.771+0000 7f9b8da00640 1 --2- 172.21.15.67:0/1457908763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b880734a0 0x7f9b880aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.771+0000 7f9b8da00640 1 -- 172.21.15.67:0/1457908763 >> 172.21.15.67:0/1457908763 conn(0x7f9b880c7370 msgr2=0x7f9b880c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.771+0000 7f9b8da00640 1 -- 172.21.15.67:0/1457908763 shutdown_connections 2024-03-09T09:02:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.771+0000 7f9b8da00640 1 -- 172.21.15.67:0/1457908763 wait complete. 2024-03-09T09:02:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.773+0000 7f9b8da00640 1 Processor -- start 2024-03-09T09:02:53.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.773+0000 7f9b8da00640 1 -- start start 2024-03-09T09:02:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.774+0000 7f9b8da00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b8806d830 0x7f9b88156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.774+0000 7f9b8da00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b880730c0 0x7f9b8814d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.774+0000 7f9b8da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b880734a0 0x7f9b881dc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.774+0000 7f9b8da00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b880c3590 con 0x7f9b880734a0 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.774+0000 7f9b8da00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b88153980 con 0x7f9b880730c0 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.774+0000 7f9b8da00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b880c3710 con 0x7f9b8806d830 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.774+0000 7f9b87400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b880734a0 0x7f9b881dc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.774+0000 7f9b87400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b880734a0 0x7f9b881dc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53500/0 (socket says 172.21.15.67:53500) 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.774+0000 7f9b87400640 1 -- 172.21.15.67:0/710304637 learned_addr learned my addr 172.21.15.67:0/710304637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.775+0000 7f9b86a00640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b8806d830 0x7f9b88156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.775+0000 7f9b7e000640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b880730c0 0x7f9b8814d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.775+0000 7f9b87400640 1 -- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b8806d830 msgr2=0x7f9b88156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.775+0000 7f9b87400640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b8806d830 0x7f9b88156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.775+0000 7f9b87400640 1 -- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b880730c0 msgr2=0x7f9b8814d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.775+0000 7f9b87400640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b880730c0 0x7f9b8814d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.775+0000 7f9b87400640 1 -- 172.21.15.67:0/710304637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8814df40 con 0x7f9b880734a0 2024-03-09T09:02:53.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.775+0000 7f9b87400640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b880734a0 0x7f9b881dc2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9b80034040 tx=0x7f9b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.776+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80004230 con 0x7f9b880734a0 2024-03-09T09:02:53.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.776+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b800043f0 con 0x7f9b880734a0 2024-03-09T09:02:53.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.776+0000 7f9b8da00640 1 -- 172.21.15.67:0/710304637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b881547f0 con 0x7f9b880734a0 2024-03-09T09:02:53.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.776+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b80003850 con 0x7f9b880734a0 2024-03-09T09:02:53.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.776+0000 7f9b8da00640 1 -- 172.21.15.67:0/710304637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b88154db0 con 0x7f9b880734a0 2024-03-09T09:02:53.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.777+0000 7f9b8da00640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b88156c50 0x7f9b881dc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.777+0000 7f9b8da00640 1 -- 172.21.15.67:0/710304637 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8814dcf0 con 0x7f9b88156c50 2024-03-09T09:02:53.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.778+0000 7f9b86a00640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b88156c50 0x7f9b881dc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.778+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9b800039f0 con 0x7f9b880734a0 2024-03-09T09:02:53.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.778+0000 7f9b86a00640 1 -- 172.21.15.67:0/710304637 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b88154fb0 con 0x7f9b88156c50 2024-03-09T09:02:53.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.779+0000 7f9b7f400640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b6406b4d0 0x7f9b6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.779+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9b8002f030 con 0x7f9b880734a0 2024-03-09T09:02:53.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.779+0000 7f9b86a00640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b88156c50 0x7f9b881dc830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001560 tx=0x7f9b78007a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.779+0000 7f9b7e000640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b6406b4d0 0x7f9b6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.779+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7800fa00 con 0x7f9b88156c50 2024-03-09T09:02:53.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.780+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f9b78018070 con 0x7f9b88156c50 2024-03-09T09:02:53.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.780+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b88156c50 msgr2=0x7f9b881dc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.780+0000 7f9b7f400640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b88156c50 0x7f9b881dc830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001560 tx=0x7f9b78007a90 comp rx=0 tx=0).stop 2024-03-09T09:02:53.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.780+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b88156c50 msgr2=0x7f9b881dc830 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:53.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.780+0000 7f9b7f400640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b88156c50 0x7f9b881dc830 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.781+0000 7f9b7e000640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b6406b4d0 0x7f9b6406d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9b70006fd0 tx=0x7f9b70008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.787+0000 7f9b8da00640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b8814d7f0 0x7f9b881ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:53.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.787+0000 7f9b8da00640 1 -- 172.21.15.67:0/710304637 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881dcd70 con 0x7f9b8814d7f0 2024-03-09T09:02:53.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.788+0000 7f9b86a00640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b8814d7f0 0x7f9b881ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.789+0000 7f9b86a00640 1 -- 172.21.15.67:0/710304637 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9b88075a00 con 0x7f9b8814d7f0 2024-03-09T09:02:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.789+0000 7f9b86a00640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b8814d7f0 0x7f9b881ee880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001560 tx=0x7f9b78001400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:53.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.790+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78013780 con 0x7f9b8814d7f0 2024-03-09T09:02:53.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7f9b78013780 con 0x7f9b8814d7f0 2024-03-09T09:02:53.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b8814d7f0 msgr2=0x7f9b881ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b7f400640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b8814d7f0 0x7f9b881ee880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001560 tx=0x7f9b78001400 comp rx=0 tx=0).stop 2024-03-09T09:02:53.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b7f400640 1 -- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b8814d7f0 msgr2=0x7f9b881ee880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:02:53.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b7f400640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b8814d7f0 0x7f9b881ee880 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001560 tx=0x7f9b78001400 comp rx=0 tx=0).stop 2024-03-09T09:02:53.793 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:02:53.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:02:53.793 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:02:53.794 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:02:53.794 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 80, 2024-03-09T09:02:53.794 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:02:53.794 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:02:53.794 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:02:53.794 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:02:53.794 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.794 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:02:53.794 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:53.794 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:02:53.794 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:02:53.795 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:53.795 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:53.795 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:53.795 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:53.795 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:53.795 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:53.795 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:53.795 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:53.795 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:53.795 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:53.795 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:53.796 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.796 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:02:53.796 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:02:53.796 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:53.796 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:53.796 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:53.796 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:53.796 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:53.796 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:53.796 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:53.796 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:02:53.797 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:02:53.798 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.799 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:53.800 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:02:53.801 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.802 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:02:53.803 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:02:53.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.804 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:02:53.804 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:02:53.804 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.804 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:02:53.804 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:02:53.804 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.805 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:02:53.806 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:02:53.806 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.806 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:53.806 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.806 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:53.806 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.806 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.806 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:02:53.806 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.806 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:53.806 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:02:53.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:02:53.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:02:53.809 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:02:53.809 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:02:53.809 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:02:53.809 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:02:53.809 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:02:53.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:02:53.809 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:02:53.809 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:02:53.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b8da00640 1 -- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b6406b4d0 msgr2=0x7f9b6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b8da00640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b6406b4d0 0x7f9b6406d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9b70006fd0 tx=0x7f9b70008040 comp rx=0 tx=0).stop 2024-03-09T09:02:53.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b8da00640 1 -- 172.21.15.67:0/710304637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b880734a0 msgr2=0x7f9b881dc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:53.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b8da00640 1 --2- 172.21.15.67:0/710304637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b880734a0 0x7f9b881dc2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9b80034040 tx=0x7f9b800046c0 comp rx=0 tx=0).stop 2024-03-09T09:02:53.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b87400640 1 -- 172.21.15.67:0/710304637 reap_dead start 2024-03-09T09:02:53.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b87400640 1 -- 172.21.15.67:0/710304637 reap_dead start 2024-03-09T09:02:53.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b8da00640 1 -- 172.21.15.67:0/710304637 shutdown_connections 2024-03-09T09:02:53.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b8da00640 1 -- 172.21.15.67:0/710304637 >> 172.21.15.67:0/710304637 conn(0x7f9b880c7370 msgr2=0x7f9b88071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:53.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b8da00640 1 -- 172.21.15.67:0/710304637 shutdown_connections 2024-03-09T09:02:53.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:53.791+0000 7f9b8da00640 1 -- 172.21.15.67:0/710304637 wait complete. 2024-03-09T09:02:53.811 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:02:53.812 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-03-09T09:02:53.812 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:02:53.812 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-03-09T09:02:53.812 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:02:53.812 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-09T09:02:53.812 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:02:53.831 INFO:tasks.ceph.mon.a:Stopped 2024-03-09T09:02:53.831 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:02:53.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:02:54.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.085+0000 7f85c9800640 1 Processor -- start 2024-03-09T09:02:54.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.085+0000 7f85c9800640 1 -- start start 2024-03-09T09:02:54.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.086+0000 7f85c9800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f85c4072770 0x7f85c4072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:54.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.087+0000 7f85c9800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85c406bda0 0x7f85c4156650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:54.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.087+0000 7f85c9800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c406cd30 con 0x7f85c40bebc0 2024-03-09T09:02:54.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.087+0000 7f85c9800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c406b200 con 0x7f85c4072770 2024-03-09T09:02:54.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.087+0000 7f85c9800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40c5eb0 con 0x7f85c406bda0 2024-03-09T09:02:54.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.087+0000 7f85c3400640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f85c40bebc0 legacy=0x7f85c406b690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:02:54.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.087+0000 7f85c3e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f85c4072770 0x7f85c4072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:54.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.087+0000 7f85c3e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f85c4072770 0x7f85c4072b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52502/0 (socket says 172.21.15.67:52502) 2024-03-09T09:02:54.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.087+0000 7f85c3e00640 1 -- 172.21.15.67:0/2876621778 learned_addr learned my addr 172.21.15.67:0/2876621778 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.087+0000 7f85c8e00640 1 --2- 172.21.15.67:0/2876621778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85c406bda0 0x7f85c4156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.088+0000 7f85c3e00640 1 -- 172.21.15.67:0/2876621778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85c406bda0 msgr2=0x7f85c4156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:02:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.088+0000 7f85c3e00640 1 --2- 172.21.15.67:0/2876621778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85c406bda0 0x7f85c4156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.088+0000 7f85c3e00640 1 -- 172.21.15.67:0/2876621778 >> v1:172.21.15.67:6789/0 conn(0x7f85c40bebc0 legacy=0x7f85c406b690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.088+0000 7f85c3e00640 1 -- 172.21.15.67:0/2876621778 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40c6150 con 0x7f85c4072770 2024-03-09T09:02:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.088+0000 7f85c8e00640 1 --2- 172.21.15.67:0/2876621778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85c406bda0 0x7f85c4156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:02:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.089+0000 7f85c3e00640 1 --2- 172.21.15.67:0/2876621778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c4072770 0x7f85c4072b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009870 tx=0x7f85b402b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16e50848d172193d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.090+0000 7f85c2a00640 1 -- 172.21.15.67:0/2876621778 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4038070 con 0x7f85c4072770 2024-03-09T09:02:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.090+0000 7f85c2a00640 1 -- 172.21.15.67:0/2876621778 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b4031740 con 0x7f85c4072770 2024-03-09T09:02:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.090+0000 7f85c2a00640 1 -- 172.21.15.67:0/2876621778 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b402f050 con 0x7f85c4072770 2024-03-09T09:02:54.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.091+0000 7f85c9800640 1 -- 172.21.15.67:0/2876621778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c4072770 msgr2=0x7f85c4072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:54.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.091+0000 7f85c9800640 1 --2- 172.21.15.67:0/2876621778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c4072770 0x7f85c4072b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009870 tx=0x7f85b402b7e0 comp rx=0 tx=0).stop 2024-03-09T09:02:54.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.091+0000 7f85c9800640 1 -- 172.21.15.67:0/2876621778 shutdown_connections 2024-03-09T09:02:54.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.091+0000 7f85c9800640 1 --2- 172.21.15.67:0/2876621778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85c406bda0 0x7f85c4156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:54.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.091+0000 7f85c9800640 1 --2- 172.21.15.67:0/2876621778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c4072770 0x7f85c4072b50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:54.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.091+0000 7f85c9800640 1 -- 172.21.15.67:0/2876621778 >> 172.21.15.67:0/2876621778 conn(0x7f85c40c7fb0 msgr2=0x7f85c40c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:54.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.091+0000 7f85c9800640 1 -- 172.21.15.67:0/2876621778 shutdown_connections 2024-03-09T09:02:54.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.091+0000 7f85c9800640 1 -- 172.21.15.67:0/2876621778 wait complete. 2024-03-09T09:02:54.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.093+0000 7f85c9800640 1 Processor -- start 2024-03-09T09:02:54.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.093+0000 7f85c9800640 1 -- start start 2024-03-09T09:02:54.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.094+0000 7f85c9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85c406bda0 0x7f85c40c1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.094+0000 7f85c9800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85c4072770 0x7f85c41eece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.094+0000 7f85c9800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c40bebc0 0x7f85c40bfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.094+0000 7f85c9800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40c9430 con 0x7f85c406bda0 2024-03-09T09:02:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.094+0000 7f85c9800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40c6710 con 0x7f85c40bebc0 2024-03-09T09:02:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.094+0000 7f85c9800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c41ef600 con 0x7f85c4072770 2024-03-09T09:02:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.094+0000 7f85c3e00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85c406bda0 msgr2=0x7f85c40c1250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:02:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.094+0000 7f85c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85c406bda0 0x7f85c40c1250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.094+0000 7f85c8e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c40bebc0 0x7f85c40bfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.095+0000 7f85c8e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c40bebc0 0x7f85c40bfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52514/0 (socket says 172.21.15.67:52514) 2024-03-09T09:02:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.095+0000 7f85c8e00640 1 -- 172.21.15.67:0/3957567819 learned_addr learned my addr 172.21.15.67:0/3957567819 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.095+0000 7f85c3400640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85c4072770 0x7f85c41eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.095+0000 7f85c8e00640 1 -- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85c4072770 msgr2=0x7f85c41eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.095+0000 7f85c8e00640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85c4072770 0x7f85c41eece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.095+0000 7f85c8e00640 1 -- 172.21.15.67:0/3957567819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85c406bda0 msgr2=0x7f85c40c1250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:02:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.095+0000 7f85c8e00640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85c406bda0 0x7f85c40c1250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.095+0000 7f85c8e00640 1 -- 172.21.15.67:0/3957567819 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40c2160 con 0x7f85c40bebc0 2024-03-09T09:02:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.096+0000 7f85c8e00640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c40bebc0 0x7f85c40bfd90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00efc0 tx=0x7f85bc00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.096+0000 7f85bbe00640 1 -- 172.21.15.67:0/3957567819 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc018030 con 0x7f85c40bebc0 2024-03-09T09:02:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.096+0000 7f85bbe00640 1 -- 172.21.15.67:0/3957567819 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc00cdf0 con 0x7f85c40bebc0 2024-03-09T09:02:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.096+0000 7f85c9800640 1 -- 172.21.15.67:0/3957567819 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c40bf320 con 0x7f85c40bebc0 2024-03-09T09:02:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.096+0000 7f85bbe00640 1 -- 172.21.15.67:0/3957567819 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc003bb0 con 0x7f85c40bebc0 2024-03-09T09:02:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.096+0000 7f85c9800640 1 -- 172.21.15.67:0/3957567819 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c40c1c40 con 0x7f85c40bebc0 2024-03-09T09:02:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.098+0000 7f85c9800640 1 -- 172.21.15.67:0/3957567819 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c40c1e40 con 0x7f85c40bebc0 2024-03-09T09:02:54.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.099+0000 7f85bbe00640 1 -- 172.21.15.67:0/3957567819 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f85bc01b070 con 0x7f85c40bebc0 2024-03-09T09:02:54.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.101+0000 7f85bbe00640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85a406b4b0 0x7f85a406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:54.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.101+0000 7f85bbe00640 1 -- 172.21.15.67:0/3957567819 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f85bc08af00 con 0x7f85c40bebc0 2024-03-09T09:02:54.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.102+0000 7f85c3e00640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85a406b4b0 0x7f85a406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:54.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.103+0000 7f85c3e00640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85a406b4b0 0x7f85a406d970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f85b4035000 tx=0x7f85b4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:54.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.119+0000 7f85bbe00640 1 -- 172.21.15.67:0/3957567819 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f85c40c1e40 con 0x7f85c40bebc0 2024-03-09T09:02:54.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.517+0000 7f85c9800640 1 -- 172.21.15.67:0/3957567819 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f85c404f3c0 con 0x7f85c40bebc0 2024-03-09T09:02:54.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.518+0000 7f85bbe00640 1 -- 172.21.15.67:0/3957567819 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f85bc010040 con 0x7f85c40bebc0 2024-03-09T09:02:54.520 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":80,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.520+0000 7f85c9800640 1 -- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85a406b4b0 msgr2=0x7f85a406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.520+0000 7f85c9800640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85a406b4b0 0x7f85a406d970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f85b4035000 tx=0x7f85b4008030 comp rx=0 tx=0).stop 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.520+0000 7f85c9800640 1 -- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c40bebc0 msgr2=0x7f85c40bfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.520+0000 7f85c9800640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c40bebc0 0x7f85c40bfd90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00efc0 tx=0x7f85bc00c490 comp rx=0 tx=0).stop 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.520+0000 7f85c9800640 1 -- 172.21.15.67:0/3957567819 shutdown_connections 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.520+0000 7f85c9800640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f85a406b4b0 0x7f85a406d970 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.520+0000 7f85c9800640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85c40bebc0 0x7f85c40bfd90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.520+0000 7f85c9800640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85c4072770 0x7f85c41eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.520+0000 7f85c9800640 1 --2- 172.21.15.67:0/3957567819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85c406bda0 0x7f85c40c1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.520+0000 7f85c9800640 1 -- 172.21.15.67:0/3957567819 >> 172.21.15.67:0/3957567819 conn(0x7f85c40c7fb0 msgr2=0x7f85c4155480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.521+0000 7f85c9800640 1 -- 172.21.15.67:0/3957567819 shutdown_connections 2024-03-09T09:02:54.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:54.521+0000 7f85c9800640 1 -- 172.21.15.67:0/3957567819 wait complete. 2024-03-09T09:02:57.545 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:02:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.748+0000 7fe503800640 1 Processor -- start 2024-03-09T09:02:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.748+0000 7fe503800640 1 -- start start 2024-03-09T09:02:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.749+0000 7fe503800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe4fc06b040 0x7fe4fc155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.749+0000 7fe503800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc0c6a40 con 0x7fe4fc06c650 2024-03-09T09:02:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.749+0000 7fe503800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc06b5b0 con 0x7fe4fc0c5f50 2024-03-09T09:02:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.749+0000 7fe503800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4fc0c9e00 con 0x7fe4fc06b040 2024-03-09T09:02:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.749+0000 7fe501800640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7fe4fc06c650 legacy=0x7fe4fc0c6330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:02:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.750+0000 7fe4f3e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe4fc06b040 0x7fe4fc155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.750+0000 7fe4f3e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe4fc06b040 0x7fe4fc155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52530/0 (socket says 172.21.15.67:52530) 2024-03-09T09:02:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.750+0000 7fe4f3e00640 1 -- 172.21.15.67:0/2252271537 learned_addr learned my addr 172.21.15.67:0/2252271537 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.750+0000 7fe4f3e00640 1 -- 172.21.15.67:0/2252271537 >> v1:172.21.15.134:6789/0 conn(0x7fe4fc0c5f50 legacy=0x7fe4fc0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.750+0000 7fe4f3e00640 1 -- 172.21.15.67:0/2252271537 >> v1:172.21.15.67:6789/0 conn(0x7fe4fc06c650 legacy=0x7fe4fc0c6330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:02:57.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.750+0000 7fe4f3e00640 1 -- 172.21.15.67:0/2252271537 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc06b890 con 0x7fe4fc06b040 2024-03-09T09:02:57.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.752+0000 7fe4f3e00640 1 --2- 172.21.15.67:0/2252271537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4fc06b040 0x7fe4fc155e20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009870 tx=0x7fe4e402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b490afd83c9b36 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.753+0000 7fe4fbe00640 1 -- 172.21.15.67:0/2252271537 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4034070 con 0x7fe4fc06b040 2024-03-09T09:02:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.753+0000 7fe4fbe00640 1 -- 172.21.15.67:0/2252271537 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e4030860 con 0x7fe4fc06b040 2024-03-09T09:02:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.753+0000 7fe4fbe00640 1 -- 172.21.15.67:0/2252271537 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4030c30 con 0x7fe4fc06b040 2024-03-09T09:02:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.753+0000 7fe503800640 1 -- 172.21.15.67:0/2252271537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4fc06b040 msgr2=0x7fe4fc155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.753+0000 7fe503800640 1 --2- 172.21.15.67:0/2252271537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4fc06b040 0x7fe4fc155e20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009870 tx=0x7fe4e402b9a0 comp rx=0 tx=0).stop 2024-03-09T09:02:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.754+0000 7fe503800640 1 -- 172.21.15.67:0/2252271537 shutdown_connections 2024-03-09T09:02:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.754+0000 7fe503800640 1 --2- 172.21.15.67:0/2252271537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4fc06b040 0x7fe4fc155e20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.754+0000 7fe503800640 1 -- 172.21.15.67:0/2252271537 >> 172.21.15.67:0/2252271537 conn(0x7fe4fc0c8050 msgr2=0x7fe4fc0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.754+0000 7fe503800640 1 -- 172.21.15.67:0/2252271537 shutdown_connections 2024-03-09T09:02:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.754+0000 7fe503800640 1 -- 172.21.15.67:0/2252271537 wait complete. 2024-03-09T09:02:57.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.755+0000 7fe503800640 1 Processor -- start 2024-03-09T09:02:57.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.756+0000 7fe503800640 1 -- start start 2024-03-09T09:02:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.756+0000 7fe503800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4fc06b040 0x7fe4fc1ef290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.756+0000 7fe503800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc06c650 0x7fe4fc14b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.756+0000 7fe503800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4fc0c5f50 0x7fe4fc0bfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.756+0000 7fe503800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc06be50 con 0x7fe4fc06c650 2024-03-09T09:02:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.756+0000 7fe503800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc06cac0 con 0x7fe4fc06b040 2024-03-09T09:02:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.756+0000 7fe503800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc06c130 con 0x7fe4fc0c5f50 2024-03-09T09:02:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.757+0000 7fe4f3e00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc06c650 msgr2=0x7fe4fc14b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:02:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.757+0000 7fe4f3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc06c650 0x7fe4fc14b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:02:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.757+0000 7fe500e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4fc06b040 0x7fe4fc1ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.757+0000 7fe500e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4fc06b040 0x7fe4fc1ef290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52534/0 (socket says 172.21.15.67:52534) 2024-03-09T09:02:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.757+0000 7fe500e00640 1 -- 172.21.15.67:0/2881660858 learned_addr learned my addr 172.21.15.67:0/2881660858 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:02:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.757+0000 7fe501800640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4fc0c5f50 0x7fe4fc0bfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:57.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.758+0000 7fe500e00640 1 -- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4fc0c5f50 msgr2=0x7fe4fc0bfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:57.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.758+0000 7fe500e00640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4fc0c5f50 0x7fe4fc0bfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:57.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.758+0000 7fe500e00640 1 -- 172.21.15.67:0/2881660858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc06c650 msgr2=0x7fe4fc14b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:02:57.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.758+0000 7fe500e00640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc06c650 0x7fe4fc14b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:57.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.758+0000 7fe500e00640 1 -- 172.21.15.67:0/2881660858 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc1efd40 con 0x7fe4fc06b040 2024-03-09T09:02:57.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.758+0000 7fe500e00640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4fc06b040 0x7fe4fc1ef290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00c3f0 tx=0x7fe4ec00c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:57.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.759+0000 7fe4f9600640 1 -- 172.21.15.67:0/2881660858 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0052d0 con 0x7fe4fc06b040 2024-03-09T09:02:57.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.759+0000 7fe4f9600640 1 -- 172.21.15.67:0/2881660858 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec005490 con 0x7fe4fc06b040 2024-03-09T09:02:57.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.759+0000 7fe503800640 1 -- 172.21.15.67:0/2881660858 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc1f0020 con 0x7fe4fc06b040 2024-03-09T09:02:57.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.759+0000 7fe4f9600640 1 -- 172.21.15.67:0/2881660858 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec007260 con 0x7fe4fc06b040 2024-03-09T09:02:57.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.759+0000 7fe503800640 1 -- 172.21.15.67:0/2881660858 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc0c2610 con 0x7fe4fc06b040 2024-03-09T09:02:57.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.760+0000 7fe503800640 1 -- 172.21.15.67:0/2881660858 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc075a00 con 0x7fe4fc06b040 2024-03-09T09:02:57.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.763+0000 7fe4f9600640 1 -- 172.21.15.67:0/2881660858 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe4ec007400 con 0x7fe4fc06b040 2024-03-09T09:02:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.764+0000 7fe4f9600640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4d806b480 0x7fe4d806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:02:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.764+0000 7fe4f9600640 1 -- 172.21.15.67:0/2881660858 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe4ec089ee0 con 0x7fe4fc06b040 2024-03-09T09:02:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.765+0000 7fe4f3e00640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4d806b480 0x7fe4d806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:02:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.766+0000 7fe4f3e00640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4d806b480 0x7fe4d806d940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4036000 tx=0x7fe4e4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:02:57.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:57.775+0000 7fe4f9600640 1 -- 172.21.15.67:0/2881660858 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe4ec058390 con 0x7fe4fc06b040 2024-03-09T09:02:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.052+0000 7fe503800640 1 -- 172.21.15.67:0/2881660858 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe4fc06cca0 con 0x7fe4fc06b040 2024-03-09T09:02:58.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.053+0000 7fe4f9600640 1 -- 172.21.15.67:0/2881660858 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fe4ec05c430 con 0x7fe4fc06b040 2024-03-09T09:02:58.055 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":80,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:02:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 -- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4d806b480 msgr2=0x7fe4d806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4d806b480 0x7fe4d806d940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4036000 tx=0x7fe4e4008030 comp rx=0 tx=0).stop 2024-03-09T09:02:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 -- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4fc06b040 msgr2=0x7fe4fc1ef290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:02:58.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4fc06b040 0x7fe4fc1ef290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00c3f0 tx=0x7fe4ec00c8b0 comp rx=0 tx=0).stop 2024-03-09T09:02:58.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 -- 172.21.15.67:0/2881660858 shutdown_connections 2024-03-09T09:02:58.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4d806b480 0x7fe4d806d940 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:58.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4fc0c5f50 0x7fe4fc0bfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:58.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe4fc06c650 0x7fe4fc14b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:58.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 --2- 172.21.15.67:0/2881660858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4fc06b040 0x7fe4fc1ef290 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:02:58.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 -- 172.21.15.67:0/2881660858 >> 172.21.15.67:0/2881660858 conn(0x7fe4fc0c8050 msgr2=0x7fe4fc0abbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:02:58.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 -- 172.21.15.67:0/2881660858 shutdown_connections 2024-03-09T09:02:58.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:02:58.056+0000 7fe503800640 1 -- 172.21.15.67:0/2881660858 wait complete. 2024-03-09T09:03:01.081 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:03:01.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.301+0000 7fcbd2a00640 1 Processor -- start 2024-03-09T09:03:01.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.301+0000 7fcbd2a00640 1 -- start start 2024-03-09T09:03:01.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.302+0000 7fcbd2a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcbcc0c30f0 0x7fcbcc152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:01.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.302+0000 7fcbd2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcbcc0aa420 0x7fcbcc1490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:01.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.302+0000 7fcbd2a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbcc0ab3c0 con 0x7fcbcc149630 2024-03-09T09:03:01.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.302+0000 7fcbd2a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc0c82d0 con 0x7fcbcc0aa420 2024-03-09T09:03:01.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.302+0000 7fcbd2a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc0aa8a0 con 0x7fcbcc0c30f0 2024-03-09T09:03:01.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.303+0000 7fcbcb400640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fcbcc0aa420 msgr2=0x7fcbcc1490f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:03:01.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.303+0000 7fcbcb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcbcc0aa420 0x7fcbcc1490f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:01.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.303+0000 7fcbcbe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcbcc0c30f0 0x7fcbcc152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:01.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.303+0000 7fcbcbe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcbcc0c30f0 0x7fcbcc152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:55636/0 (socket says 172.21.15.67:55636) 2024-03-09T09:03:01.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.303+0000 7fcbcbe00640 1 -- 172.21.15.67:0/563495046 learned_addr learned my addr 172.21.15.67:0/563495046 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:01.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.304+0000 7fcbcbe00640 1 -- 172.21.15.67:0/563495046 >> v2:172.21.15.67:3300/0 conn(0x7fcbcc0aa420 msgr2=0x7fcbcc1490f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:03:01.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.304+0000 7fcbcbe00640 1 --2- 172.21.15.67:0/563495046 >> v2:172.21.15.67:3300/0 conn(0x7fcbcc0aa420 0x7fcbcc1490f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:01.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.304+0000 7fcbcbe00640 1 -- 172.21.15.67:0/563495046 >> v1:172.21.15.134:6789/0 conn(0x7fcbcc149630 legacy=0x7fcbcc06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:01.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.304+0000 7fcbcbe00640 1 -- 172.21.15.67:0/563495046 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0aab20 con 0x7fcbcc0c30f0 2024-03-09T09:03:01.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.305+0000 7fcbcbe00640 1 --2- 172.21.15.67:0/563495046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc0c30f0 0x7fcbcc152a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0702f0 tx=0x7fcbbc02db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=394810b16f6065dd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:01.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.306+0000 7fcbcaa00640 1 -- 172.21.15.67:0/563495046 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc02f900 con 0x7fcbcc0c30f0 2024-03-09T09:03:01.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.306+0000 7fcbcaa00640 1 -- 172.21.15.67:0/563495046 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc02faa0 con 0x7fcbcc0c30f0 2024-03-09T09:03:01.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.306+0000 7fcbcaa00640 1 -- 172.21.15.67:0/563495046 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc037420 con 0x7fcbcc0c30f0 2024-03-09T09:03:01.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.307+0000 7fcbd2a00640 1 -- 172.21.15.67:0/563495046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc0c30f0 msgr2=0x7fcbcc152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:01.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.307+0000 7fcbd2a00640 1 --2- 172.21.15.67:0/563495046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc0c30f0 0x7fcbcc152a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0702f0 tx=0x7fcbbc02db40 comp rx=0 tx=0).stop 2024-03-09T09:03:01.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.307+0000 7fcbd2a00640 1 -- 172.21.15.67:0/563495046 shutdown_connections 2024-03-09T09:03:01.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.307+0000 7fcbd2a00640 1 --2- 172.21.15.67:0/563495046 >> v2:172.21.15.67:3300/0 conn(0x7fcbcc0aa420 0x7fcbcc1490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:01.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.307+0000 7fcbd2a00640 1 --2- 172.21.15.67:0/563495046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc0c30f0 0x7fcbcc152a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:01.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.307+0000 7fcbd2a00640 1 -- 172.21.15.67:0/563495046 >> 172.21.15.67:0/563495046 conn(0x7fcbcc0c7370 msgr2=0x7fcbcc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:01.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.307+0000 7fcbd2a00640 1 -- 172.21.15.67:0/563495046 shutdown_connections 2024-03-09T09:03:01.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.307+0000 7fcbd2a00640 1 -- 172.21.15.67:0/563495046 wait complete. 2024-03-09T09:03:01.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.309+0000 7fcbd2a00640 1 Processor -- start 2024-03-09T09:03:01.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.309+0000 7fcbd2a00640 1 -- start start 2024-03-09T09:03:01.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.310+0000 7fcbd2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc0aa420 0x7fcbcc149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:01.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.310+0000 7fcbd2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbcc0c30f0 0x7fcbcc1dd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:01.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.310+0000 7fcbd2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbcc149630 0x7fcbcc14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:01.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.310+0000 7fcbd2a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc06d6c0 con 0x7fcbcc149630 2024-03-09T09:03:01.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.310+0000 7fcbd2a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0ab0e0 con 0x7fcbcc0aa420 2024-03-09T09:03:01.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.310+0000 7fcbd2a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0ab620 con 0x7fcbcc0c30f0 2024-03-09T09:03:01.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.310+0000 7fcbd0a00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbcc149630 msgr2=0x7fcbcc14bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:03:01.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.310+0000 7fcbd0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbcc149630 0x7fcbcc14bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:01.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.310+0000 7fcbcb400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbcc0c30f0 0x7fcbcc1dd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:01.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.311+0000 7fcbcbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc0aa420 0x7fcbcc149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:01.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.311+0000 7fcbcb400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbcc0c30f0 0x7fcbcc1dd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:41340/0 (socket says 172.21.15.67:41340) 2024-03-09T09:03:01.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.311+0000 7fcbcb400640 1 -- 172.21.15.67:0/4220376357 learned_addr learned my addr 172.21.15.67:0/4220376357 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.311+0000 7fcbcbe00640 1 -- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbcc0c30f0 msgr2=0x7fcbcc1dd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.311+0000 7fcbcbe00640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbcc0c30f0 0x7fcbcc1dd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.311+0000 7fcbcbe00640 1 -- 172.21.15.67:0/4220376357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbcc149630 msgr2=0x7fcbcc14bbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:03:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.311+0000 7fcbcbe00640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbcc149630 0x7fcbcc14bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.311+0000 7fcbcbe00640 1 -- 172.21.15.67:0/4220376357 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc14a080 con 0x7fcbcc0aa420 2024-03-09T09:03:01.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.312+0000 7fcbcbe00640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc0aa420 0x7fcbcc149b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc035000 tx=0x7fcbbc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:01.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.312+0000 7fcbc3e00640 1 -- 172.21.15.67:0/4220376357 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc046020 con 0x7fcbcc0aa420 2024-03-09T09:03:01.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.312+0000 7fcbc3e00640 1 -- 172.21.15.67:0/4220376357 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc037e30 con 0x7fcbcc0aa420 2024-03-09T09:03:01.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.312+0000 7fcbc3e00640 1 -- 172.21.15.67:0/4220376357 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc042690 con 0x7fcbcc0aa420 2024-03-09T09:03:01.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.312+0000 7fcbd2a00640 1 -- 172.21.15.67:0/4220376357 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbcc1ef0b0 con 0x7fcbcc0aa420 2024-03-09T09:03:01.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.313+0000 7fcbd2a00640 1 -- 172.21.15.67:0/4220376357 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbcc1ef670 con 0x7fcbcc0aa420 2024-03-09T09:03:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.314+0000 7fcbd2a00640 1 -- 172.21.15.67:0/4220376357 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbcc075a00 con 0x7fcbcc0aa420 2024-03-09T09:03:01.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.316+0000 7fcbc3e00640 1 -- 172.21.15.67:0/4220376357 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcbbc049020 con 0x7fcbcc0aa420 2024-03-09T09:03:01.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.316+0000 7fcbc3e00640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbac06b480 0x7fcbac06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:01.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.317+0000 7fcbc3e00640 1 -- 172.21.15.67:0/4220376357 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcbbc0ad600 con 0x7fcbcc0aa420 2024-03-09T09:03:01.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.317+0000 7fcbcb400640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbac06b480 0x7fcbac06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:01.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.318+0000 7fcbcb400640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbac06b480 0x7fcbac06d940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009420 tx=0x7fcbb80093b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:01.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.326+0000 7fcbc3e00640 1 -- 172.21.15.67:0/4220376357 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcbbc07c100 con 0x7fcbcc0aa420 2024-03-09T09:03:01.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.641+0000 7fcbd2a00640 1 -- 172.21.15.67:0/4220376357 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcbcc0c5170 con 0x7fcbcc0aa420 2024-03-09T09:03:01.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.642+0000 7fcbc3e00640 1 -- 172.21.15.67:0/4220376357 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fcbbc07ba00 con 0x7fcbcc0aa420 2024-03-09T09:03:01.644 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":80,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:03:01.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 -- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbac06b480 msgr2=0x7fcbac06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:01.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbac06b480 0x7fcbac06d940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009420 tx=0x7fcbb80093b0 comp rx=0 tx=0).stop 2024-03-09T09:03:01.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 -- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc0aa420 msgr2=0x7fcbcc149b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:01.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc0aa420 0x7fcbcc149b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc035000 tx=0x7fcbbc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:03:01.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 -- 172.21.15.67:0/4220376357 shutdown_connections 2024-03-09T09:03:01.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbac06b480 0x7fcbac06d940 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:01.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbcc149630 0x7fcbcc14bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:01.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbcc0c30f0 0x7fcbcc1dd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:01.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 --2- 172.21.15.67:0/4220376357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbcc0aa420 0x7fcbcc149b40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:01.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 -- 172.21.15.67:0/4220376357 >> 172.21.15.67:0/4220376357 conn(0x7fcbcc0c7370 msgr2=0x7fcbcc0ab9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:01.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 -- 172.21.15.67:0/4220376357 shutdown_connections 2024-03-09T09:03:01.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:01.646+0000 7fcbd2a00640 1 -- 172.21.15.67:0/4220376357 wait complete. 2024-03-09T09:03:04.667 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:03:04.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:04.887+0000 7fed7b200640 1 Processor -- start 2024-03-09T09:03:04.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:04.888+0000 7fed7b200640 1 -- start start 2024-03-09T09:03:04.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:04.889+0000 7fed7b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fed7c0cae50 0x7fed7c0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:04.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:04.889+0000 7fed7b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c156700 con 0x7fed7c1531c0 2024-03-09T09:03:04.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:04.889+0000 7fed79e00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7fed7c1531c0 legacy=0x7fed7c0ae590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:03:04.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:04.889+0000 7fed7b200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c157530 con 0x7fed7c1570d0 2024-03-09T09:03:04.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:04.889+0000 7fed7b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c0afba0 con 0x7fed7c0cae50 2024-03-09T09:03:04.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:04.889+0000 7fed7a800640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fed7c0cae50 msgr2=0x7fed7c0c9b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:03:04.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:04.890+0000 7fed7a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fed7c0cae50 0x7fed7c0c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:04.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:04.890+0000 7fed79400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fed7c1570d0 0x7fed7c0c9440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:55406/0 (socket says 172.21.15.67:55406) 2024-03-09T09:03:04.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:04.890+0000 7fed79400640 1 -- 172.21.15.67:0/1512561280 learned_addr learned my addr 172.21.15.67:0/1512561280 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:03:05.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:05.090+0000 7fed79e00640 1 -- 172.21.15.67:0/1512561280 >> v1:172.21.15.67:6789/0 conn(0x7fed7c1531c0 legacy=0x7fed7c0ae590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:03:05.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:05.091+0000 7fed7a800640 1 -- 172.21.15.67:0/1512561280 >> v2:172.21.15.67:3300/0 conn(0x7fed7c0cae50 msgr2=0x7fed7c0c9b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:03:05.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:05.091+0000 7fed7a800640 1 --2- 172.21.15.67:0/1512561280 >> v2:172.21.15.67:3300/0 conn(0x7fed7c0cae50 0x7fed7c0c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:03:05.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:05.491+0000 7fed79e00640 1 -- 172.21.15.67:0/1512561280 >> v1:172.21.15.67:6789/0 conn(0x7fed7c1531c0 legacy=0x7fed7c0ae590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:03:05.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:05.492+0000 7fed7a800640 1 -- 172.21.15.67:0/1512561280 >> v2:172.21.15.67:3300/0 conn(0x7fed7c0cae50 msgr2=0x7fed7c0c9b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:03:05.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:05.492+0000 7fed7a800640 1 --2- 172.21.15.67:0/1512561280 >> v2:172.21.15.67:3300/0 conn(0x7fed7c0cae50 0x7fed7c0c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:03:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:06.292+0000 7fed79e00640 1 -- 172.21.15.67:0/1512561280 >> v1:172.21.15.67:6789/0 conn(0x7fed7c1531c0 legacy=0x7fed7c0ae590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:03:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:06.293+0000 7fed7a800640 1 -- 172.21.15.67:0/1512561280 >> v2:172.21.15.67:3300/0 conn(0x7fed7c0cae50 msgr2=0x7fed7c0c9b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:03:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:06.293+0000 7fed7a800640 1 --2- 172.21.15.67:0/1512561280 >> v2:172.21.15.67:3300/0 conn(0x7fed7c0cae50 0x7fed7c0c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:03:07.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.889+0000 7fed73400640 1 -- 172.21.15.67:0/1512561280 >> v2:172.21.15.67:3300/0 conn(0x7fed7c0cae50 msgr2=0x7fed7c0c9b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:03:07.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.889+0000 7fed73400640 1 --2- 172.21.15.67:0/1512561280 >> v2:172.21.15.67:3300/0 conn(0x7fed7c0cae50 0x7fed7c0c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:07.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.889+0000 7fed73400640 1 -- 172.21.15.67:0/1512561280 >> v1:172.21.15.134:6789/0 conn(0x7fed7c1570d0 legacy=0x7fed7c0c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:07.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.889+0000 7fed73400640 1 -- 172.21.15.67:0/1512561280 >> v1:172.21.15.67:6789/0 conn(0x7fed7c1531c0 legacy=0x7fed7c0ae590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:03:07.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.890+0000 7fed73400640 1 --2- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed580036c0 0x7fed58005b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:07.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.890+0000 7fed73400640 1 --2- 172.21.15.67:0/1512561280 >> v2:172.21.15.134:3300/0 conn(0x7fed580061a0 0x7fed58008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:07.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.890+0000 7fed73400640 1 -- 172.21.15.67:0/1512561280 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed58008cd0 con 0x7fed580061a0 2024-03-09T09:03:07.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.890+0000 7fed73400640 1 -- 172.21.15.67:0/1512561280 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed58008e80 con 0x7fed580036c0 2024-03-09T09:03:07.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.891+0000 7fed79e00640 1 --2- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed580036c0 0x7fed58005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:07.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.891+0000 7fed79400640 1 --2- 172.21.15.67:0/1512561280 >> v2:172.21.15.134:3300/0 conn(0x7fed580061a0 0x7fed58008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:07.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.891+0000 7fed79e00640 1 -- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed580036c0 msgr2=0x7fed58005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:03:07.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.891+0000 7fed79e00640 1 -- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed580036c0 msgr2=0x7fed58005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:03:07.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.891+0000 7fed79e00640 1 --2- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed580036c0 0x7fed58005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:03:07.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.891+0000 7fed79400640 1 -- 172.21.15.67:0/1512561280 >> v2:172.21.15.134:3300/0 conn(0x7fed580061a0 msgr2=0x7fed58008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:03:07.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.891+0000 7fed79400640 1 -- 172.21.15.67:0/1512561280 >> v2:172.21.15.134:3300/0 conn(0x7fed580061a0 msgr2=0x7fed58008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:03:07.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.891+0000 7fed79400640 1 --2- 172.21.15.67:0/1512561280 >> v2:172.21.15.134:3300/0 conn(0x7fed580061a0 0x7fed58008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:03:07.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.891+0000 7fed79e00640 1 --2- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed580036c0 0x7fed58005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:07.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:07.891+0000 7fed79400640 1 --2- 172.21.15.67:0/1512561280 >> v2:172.21.15.134:3300/0 conn(0x7fed580061a0 0x7fed58008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:08.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.092+0000 7fed79400640 1 --2- 172.21.15.67:0/1512561280 >> v2:172.21.15.134:3300/0 conn(0x7fed580061a0 0x7fed58008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:08.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.092+0000 7fed79e00640 1 --2- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed580036c0 0x7fed58005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:08.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.093+0000 7fed79400640 1 -- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed580036c0 msgr2=0x7fed58005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:08.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.093+0000 7fed79400640 1 --2- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed580036c0 0x7fed58005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.093+0000 7fed79400640 1 -- 172.21.15.67:0/1512561280 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c1569a0 con 0x7fed580061a0 2024-03-09T09:03:08.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.094+0000 7fed79e00640 1 --2- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed580036c0 0x7fed58005b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:03:08.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.097+0000 7fed79400640 1 --2- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed580061a0 0x7fed58008640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed7c157700 tx=0x7fed6402c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c2bef67c035db06 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:08.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.106+0000 7fed78a00640 1 -- 172.21.15.67:0/1512561280 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64039070 con 0x7fed580061a0 2024-03-09T09:03:08.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.109+0000 7fed78a00640 1 -- 172.21.15.67:0/1512561280 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6402cdb0 con 0x7fed580061a0 2024-03-09T09:03:08.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.109+0000 7fed78a00640 1 -- 172.21.15.67:0/1512561280 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64034660 con 0x7fed580061a0 2024-03-09T09:03:08.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.110+0000 7fed7b200640 1 -- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed580061a0 msgr2=0x7fed58008640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:08.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.110+0000 7fed7b200640 1 --2- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed580061a0 0x7fed58008640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed7c157700 tx=0x7fed6402c000 comp rx=0 tx=0).stop 2024-03-09T09:03:08.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.110+0000 7fed7a800640 1 -- 172.21.15.67:0/1512561280 reap_dead start 2024-03-09T09:03:08.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.110+0000 7fed7b200640 1 -- 172.21.15.67:0/1512561280 shutdown_connections 2024-03-09T09:03:08.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.110+0000 7fed7b200640 1 --2- 172.21.15.67:0/1512561280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed580061a0 0x7fed58008640 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.110+0000 7fed7b200640 1 -- 172.21.15.67:0/1512561280 >> 172.21.15.67:0/1512561280 conn(0x7fed7c0ad190 msgr2=0x7fed7c0ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:08.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.111+0000 7fed7b200640 1 -- 172.21.15.67:0/1512561280 shutdown_connections 2024-03-09T09:03:08.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.111+0000 7fed7b200640 1 -- 172.21.15.67:0/1512561280 wait complete. 2024-03-09T09:03:08.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.118+0000 7fed7b200640 1 Processor -- start 2024-03-09T09:03:08.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.118+0000 7fed7b200640 1 -- start start 2024-03-09T09:03:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.122+0000 7fed7b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed7c0c5010 0x7fed7c06b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.122+0000 7fed7b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed7c1dfd70 0x7fed7c06b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.123+0000 7fed7b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed7c06bac0 0x7fed7c1eeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.123+0000 7fed7b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0b06f0 con 0x7fed7c06bac0 2024-03-09T09:03:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.123+0000 7fed7b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0b01c0 con 0x7fed7c0c5010 2024-03-09T09:03:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.123+0000 7fed7b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c1535a0 con 0x7fed7c1dfd70 2024-03-09T09:03:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.123+0000 7fed7a800640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed7c06bac0 msgr2=0x7fed7c1eeaf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:03:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.123+0000 7fed7a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed7c06bac0 0x7fed7c1eeaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.123+0000 7fed79e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed7c0c5010 0x7fed7c06b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.123+0000 7fed79e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed7c0c5010 0x7fed7c06b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:55672/0 (socket says 172.21.15.67:55672) 2024-03-09T09:03:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.123+0000 7fed79e00640 1 -- 172.21.15.67:0/1225581811 learned_addr learned my addr 172.21.15.67:0/1225581811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:08.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.123+0000 7fed79400640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed7c1dfd70 0x7fed7c06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:08.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.124+0000 7fed79e00640 1 -- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed7c1dfd70 msgr2=0x7fed7c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:08.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.124+0000 7fed79e00640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed7c1dfd70 0x7fed7c06b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.124+0000 7fed79e00640 1 -- 172.21.15.67:0/1225581811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed7c06bac0 msgr2=0x7fed7c1eeaf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:03:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.124+0000 7fed79e00640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed7c06bac0 0x7fed7c1eeaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.124+0000 7fed79e00640 1 -- 172.21.15.67:0/1225581811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c1eb690 con 0x7fed7c0c5010 2024-03-09T09:03:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.124+0000 7fed79e00640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed7c0c5010 0x7fed7c06b040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00db00 tx=0x7fed6c00dbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.125+0000 7fed72000640 1 -- 172.21.15.67:0/1225581811 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c01b070 con 0x7fed7c0c5010 2024-03-09T09:03:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.125+0000 7fed72000640 1 -- 172.21.15.67:0/1225581811 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c00bda0 con 0x7fed7c0c5010 2024-03-09T09:03:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.125+0000 7fed72000640 1 -- 172.21.15.67:0/1225581811 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0052c0 con 0x7fed7c0c5010 2024-03-09T09:03:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.125+0000 7fed7b200640 1 -- 172.21.15.67:0/1225581811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c1eb930 con 0x7fed7c0c5010 2024-03-09T09:03:08.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.125+0000 7fed7b200640 1 -- 172.21.15.67:0/1225581811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c1ebef0 con 0x7fed7c0c5010 2024-03-09T09:03:08.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.127+0000 7fed7b200640 1 -- 172.21.15.67:0/1225581811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c04f3c0 con 0x7fed7c0c5010 2024-03-09T09:03:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.128+0000 7fed72000640 1 -- 172.21.15.67:0/1225581811 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fed6c01e070 con 0x7fed7c0c5010 2024-03-09T09:03:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.129+0000 7fed72000640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed5806c3c0 0x7fed5806e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.129+0000 7fed72000640 1 -- 172.21.15.67:0/1225581811 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fed6c08a960 con 0x7fed7c0c5010 2024-03-09T09:03:08.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.130+0000 7fed79400640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed5806c3c0 0x7fed5806e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:08.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.131+0000 7fed79400640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed5806c3c0 0x7fed5806e880 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fed64036000 tx=0x7fed64030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:08.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.145+0000 7fed72000640 1 -- 172.21.15.67:0/1225581811 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fed6c058d90 con 0x7fed7c0c5010 2024-03-09T09:03:08.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.515+0000 7fed7b200640 1 -- 172.21.15.67:0/1225581811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fed7c0cac30 con 0x7fed7c0c5010 2024-03-09T09:03:08.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.517+0000 7fed72000640 1 -- 172.21.15.67:0/1225581811 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7fed6c088e20 con 0x7fed7c0c5010 2024-03-09T09:03:08.519 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":82,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:03:08.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 -- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed5806c3c0 msgr2=0x7fed5806e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:08.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed5806c3c0 0x7fed5806e880 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fed64036000 tx=0x7fed64030000 comp rx=0 tx=0).stop 2024-03-09T09:03:08.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 -- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed7c0c5010 msgr2=0x7fed7c06b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:08.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed7c0c5010 0x7fed7c06b040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00db00 tx=0x7fed6c00dbf0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 -- 172.21.15.67:0/1225581811 shutdown_connections 2024-03-09T09:03:08.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fed5806c3c0 0x7fed5806e880 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed7c06bac0 0x7fed7c1eeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed7c1dfd70 0x7fed7c06b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 --2- 172.21.15.67:0/1225581811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed7c0c5010 0x7fed7c06b040 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 -- 172.21.15.67:0/1225581811 >> 172.21.15.67:0/1225581811 conn(0x7fed7c0ad190 msgr2=0x7fed7c0ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:08.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 -- 172.21.15.67:0/1225581811 shutdown_connections 2024-03-09T09:03:08.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.520+0000 7fed7b200640 1 -- 172.21.15.67:0/1225581811 wait complete. 2024-03-09T09:03:08.542 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:03:08.542 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:03:08.543 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:03:08.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.759+0000 7fcb3d600640 1 Processor -- start 2024-03-09T09:03:08.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.760+0000 7fcb3d600640 1 -- start start 2024-03-09T09:03:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.761+0000 7fcb3d600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcb380729c0 0x7fcb38072da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.761+0000 7fcb3d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcb380c3f70 0x7fcb380c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.761+0000 7fcb3d600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb38073af0 con 0x7fcb380c4450 2024-03-09T09:03:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.761+0000 7fcb3d600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb380c5780 con 0x7fcb380c3f70 2024-03-09T09:03:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.761+0000 7fcb3d600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb380c9020 con 0x7fcb380729c0 2024-03-09T09:03:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.761+0000 7fcb2fe00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fcb380c3f70 msgr2=0x7fcb380c5240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:03:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.761+0000 7fcb2fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcb380c3f70 0x7fcb380c5240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.761+0000 7fcb37400640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7fcb380c4450 legacy=0x7fcb38153d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:03:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.762+0000 7fcb36a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcb380729c0 0x7fcb38072da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.762+0000 7fcb36a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcb380729c0 0x7fcb38072da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:55682/0 (socket says 172.21.15.67:55682) 2024-03-09T09:03:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.762+0000 7fcb36a00640 1 -- 172.21.15.67:0/2028694673 learned_addr learned my addr 172.21.15.67:0/2028694673 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.762+0000 7fcb36a00640 1 -- 172.21.15.67:0/2028694673 >> v2:172.21.15.67:3300/0 conn(0x7fcb380c3f70 msgr2=0x7fcb380c5240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:03:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.762+0000 7fcb36a00640 1 --2- 172.21.15.67:0/2028694673 >> v2:172.21.15.67:3300/0 conn(0x7fcb380c3f70 0x7fcb380c5240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.762+0000 7fcb36a00640 1 -- 172.21.15.67:0/2028694673 >> v1:172.21.15.67:6789/0 conn(0x7fcb380c4450 legacy=0x7fcb38153d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:03:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.762+0000 7fcb36a00640 1 -- 172.21.15.67:0/2028694673 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380c4a40 con 0x7fcb380729c0 2024-03-09T09:03:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.764+0000 7fcb36a00640 1 --2- 172.21.15.67:0/2028694673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380729c0 0x7fcb38072da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb38071ff0 tx=0x7fcb2802b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b73ff9a5b540e525 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.764+0000 7fcb36000640 1 -- 172.21.15.67:0/2028694673 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28035020 con 0x7fcb380729c0 2024-03-09T09:03:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.764+0000 7fcb36000640 1 -- 172.21.15.67:0/2028694673 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb28031800 con 0x7fcb380729c0 2024-03-09T09:03:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.764+0000 7fcb36000640 1 -- 172.21.15.67:0/2028694673 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28031bd0 con 0x7fcb380729c0 2024-03-09T09:03:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.765+0000 7fcb3d600640 1 -- 172.21.15.67:0/2028694673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380729c0 msgr2=0x7fcb38072da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.765+0000 7fcb3d600640 1 --2- 172.21.15.67:0/2028694673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380729c0 0x7fcb38072da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb38071ff0 tx=0x7fcb2802b8a0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.765+0000 7fcb3d600640 1 -- 172.21.15.67:0/2028694673 shutdown_connections 2024-03-09T09:03:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.765+0000 7fcb3d600640 1 --2- 172.21.15.67:0/2028694673 >> v2:172.21.15.67:3300/0 conn(0x7fcb380c3f70 0x7fcb380c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.765+0000 7fcb3d600640 1 --2- 172.21.15.67:0/2028694673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380729c0 0x7fcb38072da0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.765+0000 7fcb3d600640 1 -- 172.21.15.67:0/2028694673 >> 172.21.15.67:0/2028694673 conn(0x7fcb380ab320 msgr2=0x7fcb380ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.765+0000 7fcb3d600640 1 -- 172.21.15.67:0/2028694673 shutdown_connections 2024-03-09T09:03:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.765+0000 7fcb3d600640 1 -- 172.21.15.67:0/2028694673 wait complete. 2024-03-09T09:03:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.768+0000 7fcb3d600640 1 Processor -- start 2024-03-09T09:03:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.768+0000 7fcb3d600640 1 -- start start 2024-03-09T09:03:08.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.769+0000 7fcb3d600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb380729c0 0x7fcb381567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.770+0000 7fcb3d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb380c3f70 0x7fcb381ec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.770+0000 7fcb3d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380c4450 0x7fcb3814e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.770+0000 7fcb3d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380c9300 con 0x7fcb380c3f70 2024-03-09T09:03:08.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.770+0000 7fcb3d600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb38073dd0 con 0x7fcb380c4450 2024-03-09T09:03:08.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.770+0000 7fcb3d600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb38070250 con 0x7fcb380729c0 2024-03-09T09:03:08.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.770+0000 7fcb2fe00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb380c3f70 msgr2=0x7fcb381ec100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:03:08.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.770+0000 7fcb2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb380c3f70 0x7fcb381ec100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:08.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.770+0000 7fcb36a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb380729c0 0x7fcb381567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:08.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.770+0000 7fcb37400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380c4450 0x7fcb3814e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:08.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.770+0000 7fcb36a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb380729c0 0x7fcb381567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:41388/0 (socket says 172.21.15.67:41388) 2024-03-09T09:03:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.770+0000 7fcb36a00640 1 -- 172.21.15.67:0/1012451257 learned_addr learned my addr 172.21.15.67:0/1012451257 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.771+0000 7fcb37400640 1 -- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb380729c0 msgr2=0x7fcb381567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.771+0000 7fcb37400640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb380729c0 0x7fcb381567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.771+0000 7fcb37400640 1 -- 172.21.15.67:0/1012451257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb380c3f70 msgr2=0x7fcb381ec100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:03:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.771+0000 7fcb37400640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb380c3f70 0x7fcb381ec100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.771+0000 7fcb37400640 1 -- 172.21.15.67:0/1012451257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb381ef7a0 con 0x7fcb380c4450 2024-03-09T09:03:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.771+0000 7fcb37400640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380c4450 0x7fcb3814e6e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb30003d00 tx=0x7fcb30009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.772+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000cd90 con 0x7fcb380c4450 2024-03-09T09:03:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.772+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb30010030 con 0x7fcb380c4450 2024-03-09T09:03:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.772+0000 7fcb3d600640 1 -- 172.21.15.67:0/1012451257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb380ad8a0 con 0x7fcb380c4450 2024-03-09T09:03:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.772+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30016670 con 0x7fcb380c4450 2024-03-09T09:03:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.772+0000 7fcb3d600640 1 -- 172.21.15.67:0/1012451257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb38156cf0 con 0x7fcb380c4450 2024-03-09T09:03:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.773+0000 7fcb3d600640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb381ec640 0x7fcb381f0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.773+0000 7fcb3d600640 1 -- 172.21.15.67:0/1012451257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb38075a00 con 0x7fcb381ec640 2024-03-09T09:03:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.774+0000 7fcb36a00640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb381ec640 0x7fcb381f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.775+0000 7fcb36a00640 1 -- 172.21.15.67:0/1012451257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb380738d0 con 0x7fcb381ec640 2024-03-09T09:03:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.775+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcb300153e0 con 0x7fcb380c4450 2024-03-09T09:03:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.775+0000 7fcb36a00640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb381ec640 0x7fcb381f0220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcb28000c00 tx=0x7fcb28004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.776+0000 7fcb2ea00640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb1406b480 0x7fcb1406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.776+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcb3008a7c0 con 0x7fcb380c4450 2024-03-09T09:03:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.776+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2803ac00 con 0x7fcb381ec640 2024-03-09T09:03:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.777+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fcb2804a070 con 0x7fcb381ec640 2024-03-09T09:03:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.777+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb381ec640 msgr2=0x7fcb381f0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.777+0000 7fcb2ea00640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb381ec640 0x7fcb381f0220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcb28000c00 tx=0x7fcb28004730 comp rx=0 tx=0).stop 2024-03-09T09:03:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.777+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb381ec640 msgr2=0x7fcb381f0220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.777+0000 7fcb2ea00640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb381ec640 0x7fcb381f0220 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.777+0000 7fcb2fe00640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb1406b480 0x7fcb1406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.778+0000 7fcb2fe00640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb1406b480 0x7fcb1406d940 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcb20006fd0 tx=0x7fcb20006d80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.784+0000 7fcb3d600640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380ace20 0x7fcb3814ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.784+0000 7fcb3d600640 1 -- 172.21.15.67:0/1012451257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3806c4a0 con 0x7fcb380ace20 2024-03-09T09:03:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.784+0000 7fcb36a00640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380ace20 0x7fcb3814ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.785+0000 7fcb36a00640 1 -- 172.21.15.67:0/1012451257 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcb380c52e0 con 0x7fcb380ace20 2024-03-09T09:03:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.786+0000 7fcb36a00640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380ace20 0x7fcb3814ec20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb280090b0 tx=0x7fcb280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.786+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2804f230 con 0x7fcb380ace20 2024-03-09T09:03:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.787+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5007 (secure 0 0 0) 0x7fcb28031e50 con 0x7fcb380ace20 2024-03-09T09:03:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.787+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380ace20 msgr2=0x7fcb3814ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.787+0000 7fcb2ea00640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380ace20 0x7fcb3814ec20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb280090b0 tx=0x7fcb280046c0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.787+0000 7fcb2ea00640 1 -- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380ace20 msgr2=0x7fcb3814ec20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.787+0000 7fcb2ea00640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380ace20 0x7fcb3814ec20 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb280090b0 tx=0x7fcb280046c0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.789 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:03:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:03:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:03:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:03:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 82, 2024-03-09T09:03:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:03:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:03:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:03:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:03:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:03:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:03:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:03:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:03:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:03:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:03:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:03:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:03:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:03:08.804 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:03:08.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.787+0000 7fcb3d600640 1 -- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb1406b480 msgr2=0x7fcb1406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:08.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.787+0000 7fcb3d600640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb1406b480 0x7fcb1406d940 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcb20006fd0 tx=0x7fcb20006d80 comp rx=0 tx=0).stop 2024-03-09T09:03:08.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.787+0000 7fcb3d600640 1 -- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380c4450 msgr2=0x7fcb3814e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:08.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.787+0000 7fcb3d600640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380c4450 0x7fcb3814e6e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb30003d00 tx=0x7fcb30009ed0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.787+0000 7fcb37400640 1 -- 172.21.15.67:0/1012451257 reap_dead start 2024-03-09T09:03:08.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.788+0000 7fcb3d600640 1 -- 172.21.15.67:0/1012451257 shutdown_connections 2024-03-09T09:03:08.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.788+0000 7fcb3d600640 1 --2- 172.21.15.67:0/1012451257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb380c4450 0x7fcb3814e6e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:08.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.788+0000 7fcb3d600640 1 -- 172.21.15.67:0/1012451257 >> 172.21.15.67:0/1012451257 conn(0x7fcb380ab320 msgr2=0x7fcb3806c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:08.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.788+0000 7fcb3d600640 1 -- 172.21.15.67:0/1012451257 shutdown_connections 2024-03-09T09:03:08.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:08.788+0000 7fcb3d600640 1 -- 172.21.15.67:0/1012451257 wait complete. 2024-03-09T09:03:08.817 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:03:08.817 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:03:08.817 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:03:09.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.040+0000 7f66bdc00640 1 Processor -- start 2024-03-09T09:03:09.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.041+0000 7f66bdc00640 1 -- start start 2024-03-09T09:03:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66bdc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f66b814c9a0 0x7f66b81567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66bdc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b80c1d70 0x7f66b80c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66bdc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f66b80c0ce0 0x7f66b80ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66bdc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b806d590 con 0x7f66b814c9a0 2024-03-09T09:03:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66bdc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b80ac4e0 con 0x7f66b80c0ce0 2024-03-09T09:03:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66bdc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b8156d00 con 0x7f66b80c1d70 2024-03-09T09:03:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66b6a00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f66b814c9a0 msgr2=0x7f66b81567c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:03:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66b6a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f66b814c9a0 0x7f66b81567c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:09.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66b7400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f66b80c0ce0 0x7f66b80ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66b7400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f66b80c0ce0 0x7f66b80ac9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52556/0 (socket says 172.21.15.67:52556) 2024-03-09T09:03:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66b7400640 1 -- 172.21.15.67:0/4139308335 learned_addr learned my addr 172.21.15.67:0/4139308335 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.042+0000 7f66b6000640 1 --2- 172.21.15.67:0/4139308335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b80c1d70 0x7f66b80c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.043+0000 7f66b7400640 1 -- 172.21.15.67:0/4139308335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b80c1d70 msgr2=0x7f66b80c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.043+0000 7f66b7400640 1 --2- 172.21.15.67:0/4139308335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b80c1d70 0x7f66b80c07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.043+0000 7f66b7400640 1 -- 172.21.15.67:0/4139308335 >> v2:172.21.15.67:3300/0 conn(0x7f66b814c9a0 msgr2=0x7f66b81567c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:03:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.043+0000 7f66b7400640 1 --2- 172.21.15.67:0/4139308335 >> v2:172.21.15.67:3300/0 conn(0x7f66b814c9a0 0x7f66b81567c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.043+0000 7f66b7400640 1 -- 172.21.15.67:0/4139308335 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b80aa550 con 0x7f66b80c0ce0 2024-03-09T09:03:09.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.044+0000 7f66b7400640 1 --2- 172.21.15.67:0/4139308335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b80c0ce0 0x7f66b80ac9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66b80c12d0 tx=0x7f66b002be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75fca503637a3ec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:09.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.044+0000 7f66b5600640 1 -- 172.21.15.67:0/4139308335 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b002ca10 con 0x7f66b80c0ce0 2024-03-09T09:03:09.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.044+0000 7f66b5600640 1 -- 172.21.15.67:0/4139308335 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b002cbb0 con 0x7f66b80c0ce0 2024-03-09T09:03:09.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.044+0000 7f66b5600640 1 -- 172.21.15.67:0/4139308335 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b00343f0 con 0x7f66b80c0ce0 2024-03-09T09:03:09.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.045+0000 7f66bdc00640 1 -- 172.21.15.67:0/4139308335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b80c0ce0 msgr2=0x7f66b80ac9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:09.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.045+0000 7f66bdc00640 1 --2- 172.21.15.67:0/4139308335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b80c0ce0 0x7f66b80ac9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66b80c12d0 tx=0x7f66b002be70 comp rx=0 tx=0).stop 2024-03-09T09:03:09.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.045+0000 7f66bdc00640 1 -- 172.21.15.67:0/4139308335 shutdown_connections 2024-03-09T09:03:09.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.045+0000 7f66bdc00640 1 --2- 172.21.15.67:0/4139308335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b80c0ce0 0x7f66b80ac9c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:09.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.045+0000 7f66bdc00640 1 --2- 172.21.15.67:0/4139308335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b80c1d70 0x7f66b80c07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:09.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.045+0000 7f66bdc00640 1 --2- 172.21.15.67:0/4139308335 >> v2:172.21.15.67:3300/0 conn(0x7f66b814c9a0 0x7f66b81567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:09.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.045+0000 7f66bdc00640 1 -- 172.21.15.67:0/4139308335 >> 172.21.15.67:0/4139308335 conn(0x7f66b80c7370 msgr2=0x7f66b80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:09.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.045+0000 7f66bdc00640 1 -- 172.21.15.67:0/4139308335 shutdown_connections 2024-03-09T09:03:09.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.045+0000 7f66bdc00640 1 -- 172.21.15.67:0/4139308335 wait complete. 2024-03-09T09:03:09.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.047+0000 7f66bdc00640 1 Processor -- start 2024-03-09T09:03:09.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.047+0000 7f66bdc00640 1 -- start start 2024-03-09T09:03:09.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.047+0000 7f66bdc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b80c0ce0 0x7f66b8155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:09.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66bdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66b80c1d70 0x7f66b81480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:09.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66bdc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b814c9a0 0x7f66b8148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:09.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66bdc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b80aadf0 con 0x7f66b80c1d70 2024-03-09T09:03:09.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66bdc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b80aab10 con 0x7f66b814c9a0 2024-03-09T09:03:09.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66bdc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b80acf00 con 0x7f66b80c0ce0 2024-03-09T09:03:09.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66b6000640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66b80c1d70 msgr2=0x7f66b81480e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:03:09.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66b6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66b80c1d70 0x7f66b81480e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:09.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66b6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b80c0ce0 0x7f66b8155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:09.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66b7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b814c9a0 0x7f66b8148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:09.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66b6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b80c0ce0 0x7f66b8155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46286/0 (socket says 172.21.15.67:46286) 2024-03-09T09:03:09.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66b6a00640 1 -- 172.21.15.67:0/2238915728 learned_addr learned my addr 172.21.15.67:0/2238915728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:09.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66b7400640 1 -- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b80c0ce0 msgr2=0x7f66b8155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66b7400640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b80c0ce0 0x7f66b8155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.048+0000 7f66b7400640 1 -- 172.21.15.67:0/2238915728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66b80c1d70 msgr2=0x7f66b81480e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:03:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.049+0000 7f66b7400640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66b80c1d70 0x7f66b81480e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.049+0000 7f66b7400640 1 -- 172.21.15.67:0/2238915728 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b81ef500 con 0x7f66b814c9a0 2024-03-09T09:03:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.049+0000 7f66b7400640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b814c9a0 0x7f66b8148620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f66b0032000 tx=0x7f66b0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.049+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0043020 con 0x7f66b814c9a0 2024-03-09T09:03:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.049+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b00094d0 con 0x7f66b814c9a0 2024-03-09T09:03:09.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.049+0000 7f66bdc00640 1 -- 172.21.15.67:0/2238915728 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b8070900 con 0x7f66b814c9a0 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.049+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b003e670 con 0x7f66b814c9a0 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.049+0000 7f66bdc00640 1 -- 172.21.15.67:0/2238915728 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b8070be0 con 0x7f66b814c9a0 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.050+0000 7f66bdc00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b8155d40 0x7f66b81ece30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.050+0000 7f66bdc00640 1 -- 172.21.15.67:0/2238915728 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b8075a00 con 0x7f66b8155d40 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.051+0000 7f66b6a00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b8155d40 0x7f66b81ece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.051+0000 7f66b6a00640 1 -- 172.21.15.67:0/2238915728 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66b80c15e0 con 0x7f66b8155d40 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.052+0000 7f66b6a00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b8155d40 0x7f66b81ece30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009690 tx=0x7f66a8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.052+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f66b0046020 con 0x7f66b814c9a0 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.052+0000 7f66aea00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f669806b480 0x7f669806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.053+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f66b00acbd0 con 0x7f66b814c9a0 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.053+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8016070 con 0x7f66b8155d40 2024-03-09T09:03:09.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.053+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f66a8005a40 con 0x7f66b8155d40 2024-03-09T09:03:09.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.053+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b8155d40 msgr2=0x7f66b81ece30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:09.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.053+0000 7f66aea00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b8155d40 0x7f66b81ece30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009690 tx=0x7f66a8009b50 comp rx=0 tx=0).stop 2024-03-09T09:03:09.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.053+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b8155d40 msgr2=0x7f66b81ece30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:09.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.053+0000 7f66aea00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b8155d40 0x7f66b81ece30 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009690 tx=0x7f66a8009b50 comp rx=0 tx=0).stop 2024-03-09T09:03:09.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.053+0000 7f66b6000640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f669806b480 0x7f669806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:09.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.054+0000 7f66b6000640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f669806b480 0x7f669806d940 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f66b81efb10 tx=0x7f66a4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:09.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.060+0000 7f66bdc00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b81ed370 0x7f66b81ed780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:09.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.060+0000 7f66bdc00640 1 -- 172.21.15.67:0/2238915728 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b8153790 con 0x7f66b81ed370 2024-03-09T09:03:09.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.061+0000 7f66b6a00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b81ed370 0x7f66b81ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:09.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.062+0000 7f66b6a00640 1 -- 172.21.15.67:0/2238915728 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f66b81592f0 con 0x7f66b81ed370 2024-03-09T09:03:09.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.062+0000 7f66b6a00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b81ed370 0x7f66b81ed780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009690 tx=0x7f66a80059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:09.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.062+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8016050 con 0x7f66b81ed370 2024-03-09T09:03:09.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5175 (secure 0 0 0) 0x7f66a8006430 con 0x7f66b81ed370 2024-03-09T09:03:09.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b81ed370 msgr2=0x7f66b81ed780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:09.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66aea00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b81ed370 0x7f66b81ed780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009690 tx=0x7f66a80059d0 comp rx=0 tx=0).stop 2024-03-09T09:03:09.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66aea00640 1 -- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b81ed370 msgr2=0x7f66b81ed780 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:09.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66aea00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66b81ed370 0x7f66b81ed780 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:09.068 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:03:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:03:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:03:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:03:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 82, 2024-03-09T09:03:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:03:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:03:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:03:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:03:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:03:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:03:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:03:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:03:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:03:09.082 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:03:09.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66bdc00640 1 -- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f669806b480 msgr2=0x7f669806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:09.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66bdc00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f669806b480 0x7f669806d940 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f66b81efb10 tx=0x7f66a4009190 comp rx=0 tx=0).stop 2024-03-09T09:03:09.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66bdc00640 1 -- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b814c9a0 msgr2=0x7f66b8148620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:09.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66bdc00640 1 --2- 172.21.15.67:0/2238915728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66b814c9a0 0x7f66b8148620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f66b0032000 tx=0x7f66b0004a40 comp rx=0 tx=0).stop 2024-03-09T09:03:09.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66b7400640 1 -- 172.21.15.67:0/2238915728 reap_dead start 2024-03-09T09:03:09.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66b7400640 1 -- 172.21.15.67:0/2238915728 reap_dead start 2024-03-09T09:03:09.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66bdc00640 1 -- 172.21.15.67:0/2238915728 shutdown_connections 2024-03-09T09:03:09.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66bdc00640 1 -- 172.21.15.67:0/2238915728 >> 172.21.15.67:0/2238915728 conn(0x7f66b80c7370 msgr2=0x7f66b80c5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:09.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66bdc00640 1 -- 172.21.15.67:0/2238915728 shutdown_connections 2024-03-09T09:03:09.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:09.063+0000 7f66bdc00640 1 -- 172.21.15.67:0/2238915728 wait complete. 2024-03-09T09:03:09.085 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:03:09.085 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:03:14.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:03:14.676 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:03:29.086 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:03:29.086 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-03-09T09:03:29.086 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-09T09:03:29.087 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-09T09:03:29.097 INFO:tasks.ceph.mon.a:Started 2024-03-09T09:03:29.098 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:03:29.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:03:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.406+0000 7f803e800640 1 Processor -- start 2024-03-09T09:03:29.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.406+0000 7f803e800640 1 -- start start 2024-03-09T09:03:29.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.407+0000 7f803e800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f80380710a0 0x7f8038153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:29.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.407+0000 7f803e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8038151360 0x7f8038153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:29.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.407+0000 7f803e800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80380c81e0 con 0x7f80380c1f70 2024-03-09T09:03:29.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.407+0000 7f803e800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380c32b0 con 0x7f80380710a0 2024-03-09T09:03:29.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.407+0000 7f803e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80380c2fd0 con 0x7f8038151360 2024-03-09T09:03:29.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.407+0000 7f803d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f80380c1f70 0x7f8038150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45140/0 (socket says 172.21.15.67:45140) 2024-03-09T09:03:29.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.407+0000 7f803d400640 1 -- 172.21.15.67:0/1697343369 learned_addr learned my addr 172.21.15.67:0/1697343369 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:03:29.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.407+0000 7f803de00640 1 --2- 172.21.15.67:0/1697343369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8038151360 0x7f8038153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:29.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.408+0000 7f803ca00640 1 --2- 172.21.15.67:0/1697343369 >> v2:172.21.15.134:3300/0 conn(0x7f80380710a0 0x7f8038153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:29.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.408+0000 7f803de00640 1 -- 172.21.15.67:0/1697343369 >> v2:172.21.15.134:3300/0 conn(0x7f80380710a0 msgr2=0x7f8038153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:29.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.408+0000 7f803de00640 1 --2- 172.21.15.67:0/1697343369 >> v2:172.21.15.134:3300/0 conn(0x7f80380710a0 0x7f8038153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:29.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.408+0000 7f803de00640 1 -- 172.21.15.67:0/1697343369 >> v1:172.21.15.67:6789/0 conn(0x7f80380c1f70 legacy=0x7f8038150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:29.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.408+0000 7f803de00640 1 -- 172.21.15.67:0/1697343369 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380c3700 con 0x7f8038151360 2024-03-09T09:03:29.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.409+0000 7f803de00640 1 --2- 172.21.15.67:0/1697343369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8038151360 0x7f8038153eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8030009d90 tx=0x7f803002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7ee6a576b58f1c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.410+0000 7f8037e00640 1 -- 172.21.15.67:0/1697343369 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030039070 con 0x7f8038151360 2024-03-09T09:03:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.410+0000 7f8037e00640 1 -- 172.21.15.67:0/1697343369 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803002cbc0 con 0x7f8038151360 2024-03-09T09:03:29.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.410+0000 7f8037e00640 1 -- 172.21.15.67:0/1697343369 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030034440 con 0x7f8038151360 2024-03-09T09:03:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.410+0000 7f803e800640 1 -- 172.21.15.67:0/1697343369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8038151360 msgr2=0x7f8038153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.410+0000 7f803e800640 1 --2- 172.21.15.67:0/1697343369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8038151360 0x7f8038153eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8030009d90 tx=0x7f803002be10 comp rx=0 tx=0).stop 2024-03-09T09:03:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.411+0000 7f803e800640 1 -- 172.21.15.67:0/1697343369 shutdown_connections 2024-03-09T09:03:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.411+0000 7f803e800640 1 --2- 172.21.15.67:0/1697343369 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8038151360 0x7f8038153eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.411+0000 7f803e800640 1 --2- 172.21.15.67:0/1697343369 >> v2:172.21.15.134:3300/0 conn(0x7f80380710a0 0x7f8038153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.411+0000 7f803e800640 1 -- 172.21.15.67:0/1697343369 >> 172.21.15.67:0/1697343369 conn(0x7f80380c7300 msgr2=0x7f80380c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:29.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.411+0000 7f803e800640 1 -- 172.21.15.67:0/1697343369 shutdown_connections 2024-03-09T09:03:29.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.411+0000 7f803e800640 1 -- 172.21.15.67:0/1697343369 wait complete. 2024-03-09T09:03:29.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.413+0000 7f803e800640 1 Processor -- start 2024-03-09T09:03:29.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.414+0000 7f803e800640 1 -- start start 2024-03-09T09:03:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.415+0000 7f803e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80380710a0 0x7f80380c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.415+0000 7f803e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80380c1f70 0x7f80381491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.415+0000 7f803e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8038151360 0x7f80381ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.415+0000 7f803e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8038070d50 con 0x7f8038151360 2024-03-09T09:03:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.415+0000 7f803e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8038070830 con 0x7f80380710a0 2024-03-09T09:03:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.415+0000 7f803e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8038070ed0 con 0x7f80380c1f70 2024-03-09T09:03:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8038151360 0x7f80381ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:29.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8038151360 0x7f80381ebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47062/0 (socket says 172.21.15.67:47062) 2024-03-09T09:03:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803de00640 1 -- 172.21.15.67:0/3597499494 learned_addr learned my addr 172.21.15.67:0/3597499494 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803ca00640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80380c1f70 0x7f80381491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803d400640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80380710a0 0x7f80380c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803de00640 1 -- 172.21.15.67:0/3597499494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8038151360 msgr2=0x7f80381ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:03:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803de00640 1 -- 172.21.15.67:0/3597499494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8038151360 msgr2=0x7f80381ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:03:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803de00640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8038151360 0x7f80381ebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:03:29.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803de00640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8038151360 0x7f80381ebda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803ca00640 1 -- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80380710a0 msgr2=0x7f80380c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803ca00640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80380710a0 0x7f80380c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803ca00640 1 -- 172.21.15.67:0/3597499494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8038151360 msgr2=0x7f80381ebda0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:03:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803ca00640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8038151360 0x7f80381ebda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.416+0000 7f803ca00640 1 -- 172.21.15.67:0/3597499494 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380c5510 con 0x7f80380c1f70 2024-03-09T09:03:29.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.417+0000 7f803ca00640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80380c1f70 0x7f80381491e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f802400e870 tx=0x7f802400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.417+0000 7f8035600640 1 -- 172.21.15.67:0/3597499494 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024017070 con 0x7f80380c1f70 2024-03-09T09:03:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.417+0000 7f8035600640 1 -- 172.21.15.67:0/3597499494 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802400ee60 con 0x7f80380c1f70 2024-03-09T09:03:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.417+0000 7f8035600640 1 -- 172.21.15.67:0/3597499494 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802400cb60 con 0x7f80380c1f70 2024-03-09T09:03:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.418+0000 7f803e800640 1 -- 172.21.15.67:0/3597499494 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80381ef540 con 0x7f80380c1f70 2024-03-09T09:03:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.418+0000 7f803e800640 1 -- 172.21.15.67:0/3597499494 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80381eb880 con 0x7f80380c1f70 2024-03-09T09:03:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.419+0000 7f803e800640 1 -- 172.21.15.67:0/3597499494 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803804f3c0 con 0x7f80380c1f70 2024-03-09T09:03:29.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.420+0000 7f8035600640 1 -- 172.21.15.67:0/3597499494 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f802401a070 con 0x7f80380c1f70 2024-03-09T09:03:29.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.421+0000 7f8035600640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f801806b4d0 0x7f801806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:29.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.421+0000 7f8035600640 1 -- 172.21.15.67:0/3597499494 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f802408a7a0 con 0x7f80380c1f70 2024-03-09T09:03:29.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.421+0000 7f803d400640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f801806b4d0 0x7f801806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:29.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.422+0000 7f803d400640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f801806b4d0 0x7f801806d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f802c000f60 tx=0x7f802c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:29.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.427+0000 7f8035600640 1 -- 172.21.15.67:0/3597499494 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8024059220 con 0x7f80380c1f70 2024-03-09T09:03:29.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.817+0000 7f803e800640 1 -- 172.21.15.67:0/3597499494 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f803806d7c0 con 0x7f80380c1f70 2024-03-09T09:03:29.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.818+0000 7f8035600640 1 -- 172.21.15.67:0/3597499494 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f8024058b20 con 0x7f80380c1f70 2024-03-09T09:03:29.821 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":82,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:03:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.821+0000 7f803e800640 1 -- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f801806b4d0 msgr2=0x7f801806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.821+0000 7f803e800640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f801806b4d0 0x7f801806d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f802c000f60 tx=0x7f802c009040 comp rx=0 tx=0).stop 2024-03-09T09:03:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.821+0000 7f803e800640 1 -- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80380c1f70 msgr2=0x7f80381491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.821+0000 7f803e800640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80380c1f70 0x7f80381491e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f802400e870 tx=0x7f802400ed30 comp rx=0 tx=0).stop 2024-03-09T09:03:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.822+0000 7f803e800640 1 -- 172.21.15.67:0/3597499494 shutdown_connections 2024-03-09T09:03:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.822+0000 7f803e800640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f801806b4d0 0x7f801806d990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.822+0000 7f803e800640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8038151360 0x7f80381ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.822+0000 7f803e800640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80380c1f70 0x7f80381491e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.822+0000 7f803e800640 1 --2- 172.21.15.67:0/3597499494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80380710a0 0x7f80380c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.822+0000 7f803e800640 1 -- 172.21.15.67:0/3597499494 >> 172.21.15.67:0/3597499494 conn(0x7f80380c7300 msgr2=0x7f8038150a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.822+0000 7f803e800640 1 -- 172.21.15.67:0/3597499494 shutdown_connections 2024-03-09T09:03:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:29.822+0000 7f803e800640 1 -- 172.21.15.67:0/3597499494 wait complete. 2024-03-09T09:03:32.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:03:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.067+0000 7f2c7e400640 1 Processor -- start 2024-03-09T09:03:33.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.067+0000 7f2c7e400640 1 -- start start 2024-03-09T09:03:33.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.068+0000 7f2c7e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c78153300 0x7f2c78073580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:33.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.068+0000 7f2c7e400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c78155b00 0x7f2c781596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:33.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.068+0000 7f2c7e400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2c78073ac0 0x7f2c78159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:33.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.068+0000 7f2c7e400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c780aadb0 con 0x7f2c78155b00 2024-03-09T09:03:33.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.068+0000 7f2c7e400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c780ab090 con 0x7f2c78073ac0 2024-03-09T09:03:33.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.068+0000 7f2c7e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780ab370 con 0x7f2c78153300 2024-03-09T09:03:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.069+0000 7f2c77e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c78155b00 0x7f2c781596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.069+0000 7f2c7d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c78153300 0x7f2c78073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.069+0000 7f2c77e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c78155b00 0x7f2c781596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47068/0 (socket says 172.21.15.67:47068) 2024-03-09T09:03:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.069+0000 7f2c77e00640 1 -- 172.21.15.67:0/1896935939 learned_addr learned my addr 172.21.15.67:0/1896935939 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.069+0000 7f2c7da00640 1 --2- 172.21.15.67:0/1896935939 >> v2:172.21.15.134:3300/0 conn(0x7f2c78073ac0 0x7f2c78159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.069+0000 7f2c77e00640 1 -- 172.21.15.67:0/1896935939 >> v2:172.21.15.67:3300/0 conn(0x7f2c78155b00 msgr2=0x7f2c781596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:03:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.069+0000 7f2c77e00640 1 -- 172.21.15.67:0/1896935939 >> v2:172.21.15.67:3300/0 conn(0x7f2c78155b00 msgr2=0x7f2c781596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:03:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.069+0000 7f2c77e00640 1 --2- 172.21.15.67:0/1896935939 >> v2:172.21.15.67:3300/0 conn(0x7f2c78155b00 0x7f2c781596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:03:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.069+0000 7f2c77e00640 1 --2- 172.21.15.67:0/1896935939 >> v2:172.21.15.67:3300/0 conn(0x7f2c78155b00 0x7f2c781596b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:33.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.070+0000 7f2c7d000640 1 -- 172.21.15.67:0/1896935939 >> v2:172.21.15.134:3300/0 conn(0x7f2c78073ac0 msgr2=0x7f2c78159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.070+0000 7f2c7d000640 1 --2- 172.21.15.67:0/1896935939 >> v2:172.21.15.134:3300/0 conn(0x7f2c78073ac0 0x7f2c78159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.070+0000 7f2c7d000640 1 -- 172.21.15.67:0/1896935939 >> v2:172.21.15.67:3300/0 conn(0x7f2c78155b00 msgr2=0x7f2c781596b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:03:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.070+0000 7f2c7d000640 1 --2- 172.21.15.67:0/1896935939 >> v2:172.21.15.67:3300/0 conn(0x7f2c78155b00 0x7f2c781596b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.070+0000 7f2c7d000640 1 -- 172.21.15.67:0/1896935939 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c78159190 con 0x7f2c78153300 2024-03-09T09:03:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.070+0000 7f2c7da00640 1 --2- 172.21.15.67:0/1896935939 >> v2:172.21.15.134:3300/0 conn(0x7f2c78073ac0 0x7f2c78159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:03:33.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.071+0000 7f2c7d000640 1 --2- 172.21.15.67:0/1896935939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c78153300 0x7f2c78073580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2c78072590 tx=0x7f2c6c02b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4b42e5a6fd182cd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.072+0000 7f2c77400640 1 -- 172.21.15.67:0/1896935939 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c037070 con 0x7f2c78153300 2024-03-09T09:03:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.072+0000 7f2c77400640 1 -- 172.21.15.67:0/1896935939 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6c030710 con 0x7f2c78153300 2024-03-09T09:03:33.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.072+0000 7f2c77400640 1 -- 172.21.15.67:0/1896935939 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c030ae0 con 0x7f2c78153300 2024-03-09T09:03:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.073+0000 7f2c7e400640 1 -- 172.21.15.67:0/1896935939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c78153300 msgr2=0x7f2c78073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.073+0000 7f2c7e400640 1 --2- 172.21.15.67:0/1896935939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c78153300 0x7f2c78073580 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2c78072590 tx=0x7f2c6c02b710 comp rx=0 tx=0).stop 2024-03-09T09:03:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.073+0000 7f2c7e400640 1 -- 172.21.15.67:0/1896935939 shutdown_connections 2024-03-09T09:03:33.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.073+0000 7f2c7e400640 1 --2- 172.21.15.67:0/1896935939 >> v2:172.21.15.134:3300/0 conn(0x7f2c78073ac0 0x7f2c78159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.073+0000 7f2c7e400640 1 --2- 172.21.15.67:0/1896935939 >> v2:172.21.15.67:3300/0 conn(0x7f2c78155b00 0x7f2c781596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.073+0000 7f2c7e400640 1 --2- 172.21.15.67:0/1896935939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c78153300 0x7f2c78073580 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.073+0000 7f2c7e400640 1 -- 172.21.15.67:0/1896935939 >> 172.21.15.67:0/1896935939 conn(0x7f2c780c9400 msgr2=0x7f2c780c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:33.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.073+0000 7f2c7e400640 1 -- 172.21.15.67:0/1896935939 shutdown_connections 2024-03-09T09:03:33.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.073+0000 7f2c7e400640 1 -- 172.21.15.67:0/1896935939 wait complete. 2024-03-09T09:03:33.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.076+0000 7f2c7e400640 1 Processor -- start 2024-03-09T09:03:33.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.076+0000 7f2c7e400640 1 -- start start 2024-03-09T09:03:33.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.077+0000 7f2c7e400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c78073ac0 0x7f2c78149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:33.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.077+0000 7f2c7e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c78153300 0x7f2c78158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:33.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c7e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c78155b00 0x7f2c780c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:33.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c7e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780aa840 con 0x7f2c78153300 2024-03-09T09:03:33.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c7e400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c78070810 con 0x7f2c78073ac0 2024-03-09T09:03:33.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c7e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780aa9c0 con 0x7f2c78155b00 2024-03-09T09:03:33.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c78153300 0x7f2c78158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:33.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c78153300 0x7f2c78158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47070/0 (socket says 172.21.15.67:47070) 2024-03-09T09:03:33.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c77e00640 1 -- 172.21.15.67:0/3898498179 learned_addr learned my addr 172.21.15.67:0/3898498179 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:33.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c7d000640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c78073ac0 0x7f2c78149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:33.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c7da00640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c78155b00 0x7f2c780c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:33.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c77e00640 1 -- 172.21.15.67:0/3898498179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c78153300 msgr2=0x7f2c78158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:03:33.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c77e00640 1 -- 172.21.15.67:0/3898498179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c78153300 msgr2=0x7f2c78158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:03:33.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c77e00640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c78153300 0x7f2c78158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:03:33.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.078+0000 7f2c77e00640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c78153300 0x7f2c78158a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:33.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.079+0000 7f2c7d000640 1 -- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c78155b00 msgr2=0x7f2c780c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:33.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.079+0000 7f2c7d000640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c78155b00 0x7f2c780c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.079+0000 7f2c7d000640 1 -- 172.21.15.67:0/3898498179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c78153300 msgr2=0x7f2c78158a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:03:33.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.079+0000 7f2c7d000640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c78153300 0x7f2c78158a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.079+0000 7f2c7d000640 1 -- 172.21.15.67:0/3898498179 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c78158fa0 con 0x7f2c78073ac0 2024-03-09T09:03:33.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.079+0000 7f2c7d000640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c78073ac0 0x7f2c78149d00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c032000 tx=0x7f2c6c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:33.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.080+0000 7f2c74c00640 1 -- 172.21.15.67:0/3898498179 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c041030 con 0x7f2c78073ac0 2024-03-09T09:03:33.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.080+0000 7f2c74c00640 1 -- 172.21.15.67:0/3898498179 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6c030510 con 0x7f2c78073ac0 2024-03-09T09:03:33.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.080+0000 7f2c74c00640 1 -- 172.21.15.67:0/3898498179 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c037040 con 0x7f2c78073ac0 2024-03-09T09:03:33.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.080+0000 7f2c7e400640 1 -- 172.21.15.67:0/3898498179 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c78149a20 con 0x7f2c78073ac0 2024-03-09T09:03:33.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.080+0000 7f2c7e400640 1 -- 172.21.15.67:0/3898498179 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c780c7530 con 0x7f2c78073ac0 2024-03-09T09:03:33.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.082+0000 7f2c7e400640 1 -- 172.21.15.67:0/3898498179 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c78157280 con 0x7f2c78073ac0 2024-03-09T09:03:33.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.083+0000 7f2c74c00640 1 -- 172.21.15.67:0/3898498179 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2c6c044070 con 0x7f2c78073ac0 2024-03-09T09:03:33.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.084+0000 7f2c74c00640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c5806b4d0 0x7f2c5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:33.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.084+0000 7f2c74c00640 1 -- 172.21.15.67:0/3898498179 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f2c6c0abe00 con 0x7f2c78073ac0 2024-03-09T09:03:33.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.085+0000 7f2c77e00640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c5806b4d0 0x7f2c5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:33.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.086+0000 7f2c77e00640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c5806b4d0 0x7f2c5806d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2c7806d770 tx=0x7f2c64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:33.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.096+0000 7f2c74c00640 1 -- 172.21.15.67:0/3898498179 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2c78157280 con 0x7f2c78073ac0 2024-03-09T09:03:33.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.448+0000 7f2c7e400640 1 -- 172.21.15.67:0/3898498179 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2c78156e30 con 0x7f2c78073ac0 2024-03-09T09:03:33.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.450+0000 7f2c74c00640 1 -- 172.21.15.67:0/3898498179 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f2c6c07a880 con 0x7f2c78073ac0 2024-03-09T09:03:33.452 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":82,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:03:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 -- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c5806b4d0 msgr2=0x7f2c5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c5806b4d0 0x7f2c5806d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2c7806d770 tx=0x7f2c64008040 comp rx=0 tx=0).stop 2024-03-09T09:03:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 -- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c78073ac0 msgr2=0x7f2c78149d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c78073ac0 0x7f2c78149d00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c032000 tx=0x7f2c6c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 -- 172.21.15.67:0/3898498179 shutdown_connections 2024-03-09T09:03:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c5806b4d0 0x7f2c5806d990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c78155b00 0x7f2c780c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c78153300 0x7f2c78158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 --2- 172.21.15.67:0/3898498179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c78073ac0 0x7f2c78149d00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 -- 172.21.15.67:0/3898498179 >> 172.21.15.67:0/3898498179 conn(0x7f2c780c9400 msgr2=0x7f2c7814f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:33.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 -- 172.21.15.67:0/3898498179 shutdown_connections 2024-03-09T09:03:33.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:33.452+0000 7f2c7e400640 1 -- 172.21.15.67:0/3898498179 wait complete. 2024-03-09T09:03:36.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:03:36.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.739+0000 7f9e3fe00640 1 Processor -- start 2024-03-09T09:03:36.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.740+0000 7f9e3fe00640 1 -- start start 2024-03-09T09:03:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.741+0000 7f9e3fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9e401570d0 0x7f9e400cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.741+0000 7f9e3fe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9e400c9440 0x7f9e400c9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.741+0000 7f9e3fe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e40156700 con 0x7f9e401531c0 2024-03-09T09:03:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.741+0000 7f9e3fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e400afba0 con 0x7f9e401570d0 2024-03-09T09:03:36.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.741+0000 7f9e3fe00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e400b01c0 con 0x7f9e400c9440 2024-03-09T09:03:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.741+0000 7f9e3e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9e401570d0 0x7f9e400cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.741+0000 7f9e3e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9e401570d0 0x7f9e400cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47072/0 (socket says 172.21.15.67:47072) 2024-03-09T09:03:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.741+0000 7f9e3e000640 1 -- 172.21.15.67:0/3658600196 learned_addr learned my addr 172.21.15.67:0/3658600196 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.742+0000 7f9e3e000640 1 -- 172.21.15.67:0/3658600196 >> v2:172.21.15.67:3300/0 conn(0x7f9e401570d0 msgr2=0x7f9e400cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:03:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.742+0000 7f9e3e000640 1 -- 172.21.15.67:0/3658600196 >> v2:172.21.15.67:3300/0 conn(0x7f9e401570d0 msgr2=0x7f9e400cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:03:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.742+0000 7f9e3e000640 1 --2- 172.21.15.67:0/3658600196 >> v2:172.21.15.67:3300/0 conn(0x7f9e401570d0 0x7f9e400cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:03:36.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.742+0000 7f9e3f400640 1 --2- 172.21.15.67:0/3658600196 >> v2:172.21.15.134:3300/0 conn(0x7f9e400c9440 0x7f9e400c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.742+0000 7f9e3e000640 1 --2- 172.21.15.67:0/3658600196 >> v2:172.21.15.67:3300/0 conn(0x7f9e401570d0 0x7f9e400cae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.742+0000 7f9e3f400640 1 -- 172.21.15.67:0/3658600196 >> v2:172.21.15.67:3300/0 conn(0x7f9e401570d0 msgr2=0x7f9e400cae50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:03:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.742+0000 7f9e3f400640 1 --2- 172.21.15.67:0/3658600196 >> v2:172.21.15.67:3300/0 conn(0x7f9e401570d0 0x7f9e400cae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.742+0000 7f9e3f400640 1 -- 172.21.15.67:0/3658600196 >> v1:172.21.15.134:6789/0 conn(0x7f9e401531c0 legacy=0x7f9e400ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:36.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.742+0000 7f9e3f400640 1 -- 172.21.15.67:0/3658600196 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e40157530 con 0x7f9e400c9440 2024-03-09T09:03:36.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.744+0000 7f9e3f400640 1 --2- 172.21.15.67:0/3658600196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e400c9440 0x7f9e400c9870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9e400ca770 tx=0x7f9e3802be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=423d4ed88512dea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.745+0000 7f9e3d600640 1 -- 172.21.15.67:0/3658600196 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38039070 con 0x7f9e400c9440 2024-03-09T09:03:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.745+0000 7f9e3d600640 1 -- 172.21.15.67:0/3658600196 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3802cbf0 con 0x7f9e400c9440 2024-03-09T09:03:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.745+0000 7f9e3d600640 1 -- 172.21.15.67:0/3658600196 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38034440 con 0x7f9e400c9440 2024-03-09T09:03:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.745+0000 7f9e3fe00640 1 -- 172.21.15.67:0/3658600196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e400c9440 msgr2=0x7f9e400c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.745+0000 7f9e3fe00640 1 --2- 172.21.15.67:0/3658600196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e400c9440 0x7f9e400c9870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9e400ca770 tx=0x7f9e3802be40 comp rx=0 tx=0).stop 2024-03-09T09:03:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.746+0000 7f9e3fe00640 1 -- 172.21.15.67:0/3658600196 shutdown_connections 2024-03-09T09:03:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.746+0000 7f9e3fe00640 1 --2- 172.21.15.67:0/3658600196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e400c9440 0x7f9e400c9870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.746+0000 7f9e3fe00640 1 --2- 172.21.15.67:0/3658600196 >> v2:172.21.15.67:3300/0 conn(0x7f9e401570d0 0x7f9e400cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.746+0000 7f9e3fe00640 1 -- 172.21.15.67:0/3658600196 >> 172.21.15.67:0/3658600196 conn(0x7f9e400ad190 msgr2=0x7f9e400ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:36.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.746+0000 7f9e3fe00640 1 -- 172.21.15.67:0/3658600196 shutdown_connections 2024-03-09T09:03:36.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.746+0000 7f9e3fe00640 1 -- 172.21.15.67:0/3658600196 wait complete. 2024-03-09T09:03:36.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.747+0000 7f9e3fe00640 1 Processor -- start 2024-03-09T09:03:36.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.748+0000 7f9e3fe00640 1 -- start start 2024-03-09T09:03:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e400c9440 0x7f9e40154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e401531c0 0x7f9e4006b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e401570d0 0x7f9e400ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400b0410 con 0x7f9e401531c0 2024-03-09T09:03:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400cb390 con 0x7f9e400c9440 2024-03-09T09:03:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400b06f0 con 0x7f9e401570d0 2024-03-09T09:03:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e401531c0 0x7f9e4006b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e401531c0 0x7f9e4006b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47080/0 (socket says 172.21.15.67:47080) 2024-03-09T09:03:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3e000640 1 -- 172.21.15.67:0/1175343322 learned_addr learned my addr 172.21.15.67:0/1175343322 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3e000640 1 -- 172.21.15.67:0/1175343322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e401531c0 msgr2=0x7f9e4006b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:03:36.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3e000640 1 -- 172.21.15.67:0/1175343322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e401531c0 msgr2=0x7f9e4006b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:03:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3e000640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e401531c0 0x7f9e4006b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:03:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.749+0000 7f9e3ea00640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e400c9440 0x7f9e40154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.750+0000 7f9e3e000640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e401531c0 0x7f9e4006b590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:36.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.750+0000 7f9e3f400640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e401570d0 0x7f9e400ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.750+0000 7f9e3ea00640 1 -- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e401570d0 msgr2=0x7f9e400ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.750+0000 7f9e3ea00640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e401570d0 0x7f9e400ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.750+0000 7f9e3ea00640 1 -- 172.21.15.67:0/1175343322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e401531c0 msgr2=0x7f9e4006b590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:03:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.750+0000 7f9e3ea00640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e401531c0 0x7f9e4006b590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.750+0000 7f9e3ea00640 1 -- 172.21.15.67:0/1175343322 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e401ee9b0 con 0x7f9e400c9440 2024-03-09T09:03:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.751+0000 7f9e3ea00640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e400c9440 0x7f9e40154480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c340 tx=0x7f9e3000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.751+0000 7f9e36a00640 1 -- 172.21.15.67:0/1175343322 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e300052d0 con 0x7f9e400c9440 2024-03-09T09:03:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.751+0000 7f9e36a00640 1 -- 172.21.15.67:0/1175343322 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e30005490 con 0x7f9e400c9440 2024-03-09T09:03:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.751+0000 7f9e3fe00640 1 -- 172.21.15.67:0/1175343322 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e400c5990 con 0x7f9e400c9440 2024-03-09T09:03:36.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.751+0000 7f9e36a00640 1 -- 172.21.15.67:0/1175343322 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30007260 con 0x7f9e400c9440 2024-03-09T09:03:36.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.752+0000 7f9e3fe00640 1 -- 172.21.15.67:0/1175343322 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e401eef50 con 0x7f9e400c9440 2024-03-09T09:03:36.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.753+0000 7f9e3fe00640 1 -- 172.21.15.67:0/1175343322 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4004f3c0 con 0x7f9e400c9440 2024-03-09T09:03:36.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.754+0000 7f9e36a00640 1 -- 172.21.15.67:0/1175343322 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9e30007400 con 0x7f9e400c9440 2024-03-09T09:03:36.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.756+0000 7f9e36a00640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9e2006b4d0 0x7f9e2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:36.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.756+0000 7f9e36a00640 1 -- 172.21.15.67:0/1175343322 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9e30089900 con 0x7f9e400c9440 2024-03-09T09:03:36.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.757+0000 7f9e3e000640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9e2006b4d0 0x7f9e2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:36.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.758+0000 7f9e3e000640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9e2006b4d0 0x7f9e2006d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9e400ad0f0 tx=0x7f9e2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:36.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:36.769+0000 7f9e36a00640 1 -- 172.21.15.67:0/1175343322 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e30057dd0 con 0x7f9e400c9440 2024-03-09T09:03:37.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.049+0000 7f9e3fe00640 1 -- 172.21.15.67:0/1175343322 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9e400cac30 con 0x7f9e400c9440 2024-03-09T09:03:37.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.050+0000 7f9e36a00640 1 -- 172.21.15.67:0/1175343322 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f9e3005be70 con 0x7f9e400c9440 2024-03-09T09:03:37.052 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":82,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":28,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:03:37.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.052+0000 7f9e3fe00640 1 -- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9e2006b4d0 msgr2=0x7f9e2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:37.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.052+0000 7f9e3fe00640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9e2006b4d0 0x7f9e2006d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9e400ad0f0 tx=0x7f9e2c008040 comp rx=0 tx=0).stop 2024-03-09T09:03:37.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.053+0000 7f9e3fe00640 1 -- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e400c9440 msgr2=0x7f9e40154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:37.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.053+0000 7f9e3fe00640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e400c9440 0x7f9e40154480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c340 tx=0x7f9e3000c800 comp rx=0 tx=0).stop 2024-03-09T09:03:37.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.053+0000 7f9e3fe00640 1 -- 172.21.15.67:0/1175343322 shutdown_connections 2024-03-09T09:03:37.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.053+0000 7f9e3fe00640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9e2006b4d0 0x7f9e2006d990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:37.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.053+0000 7f9e3fe00640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e401570d0 0x7f9e400ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:37.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.053+0000 7f9e3fe00640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e401531c0 0x7f9e4006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:37.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.053+0000 7f9e3fe00640 1 --2- 172.21.15.67:0/1175343322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e400c9440 0x7f9e40154480 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:37.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.053+0000 7f9e3fe00640 1 -- 172.21.15.67:0/1175343322 >> 172.21.15.67:0/1175343322 conn(0x7f9e400ad190 msgr2=0x7f9e40151ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:37.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.053+0000 7f9e3fe00640 1 -- 172.21.15.67:0/1175343322 shutdown_connections 2024-03-09T09:03:37.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:37.053+0000 7f9e3fe00640 1 -- 172.21.15.67:0/1175343322 wait complete. 2024-03-09T09:03:40.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:03:40.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.288+0000 7fbcf1600640 1 Processor -- start 2024-03-09T09:03:40.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.288+0000 7fbcf1600640 1 -- start start 2024-03-09T09:03:40.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbcf1600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 0x7fbcec0cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:40.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbcf1600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbcec0cb790 0x7fbcec156190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:40.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbcf1600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcec0c7340 con 0x7fbcec153290 2024-03-09T09:03:40.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbcf1600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcec155d00 con 0x7fbcec0cb790 2024-03-09T09:03:40.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbcf1600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0c7620 con 0x7fbcec06c940 2024-03-09T09:03:40.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbcebe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbcec153290 0x7fbcec159b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34214/0 (socket says 172.21.15.67:34214) 2024-03-09T09:03:40.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbcebe00640 1 -- 172.21.15.67:0/4138831252 learned_addr learned my addr 172.21.15.67:0/4138831252 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:03:40.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbcf0c00640 1 --2- 172.21.15.67:0/4138831252 >> v2:172.21.15.67:3300/0 conn(0x7fbcec0cb790 0x7fbcec156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:40.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbceb400640 1 --2- 172.21.15.67:0/4138831252 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 0x7fbcec0cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:40.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbcf0c00640 1 -- 172.21.15.67:0/4138831252 >> v2:172.21.15.67:3300/0 conn(0x7fbcec0cb790 msgr2=0x7fbcec156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:03:40.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbcf0c00640 1 -- 172.21.15.67:0/4138831252 >> v2:172.21.15.67:3300/0 conn(0x7fbcec0cb790 msgr2=0x7fbcec156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:03:40.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.290+0000 7fbcf0c00640 1 --2- 172.21.15.67:0/4138831252 >> v2:172.21.15.67:3300/0 conn(0x7fbcec0cb790 0x7fbcec156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:03:40.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.291+0000 7fbcf0c00640 1 --2- 172.21.15.67:0/4138831252 >> v2:172.21.15.67:3300/0 conn(0x7fbcec0cb790 0x7fbcec156190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:40.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.291+0000 7fbceb400640 1 -- 172.21.15.67:0/4138831252 >> v2:172.21.15.67:3300/0 conn(0x7fbcec0cb790 msgr2=0x7fbcec156190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:03:40.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.291+0000 7fbceb400640 1 --2- 172.21.15.67:0/4138831252 >> v2:172.21.15.67:3300/0 conn(0x7fbcec0cb790 0x7fbcec156190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:40.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.291+0000 7fbceb400640 1 -- 172.21.15.67:0/4138831252 >> v1:172.21.15.67:6789/0 conn(0x7fbcec153290 legacy=0x7fbcec159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:40.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.291+0000 7fbceb400640 1 -- 172.21.15.67:0/4138831252 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0c7900 con 0x7fbcec06c940 2024-03-09T09:03:40.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.292+0000 7fbceb400640 1 --2- 172.21.15.67:0/4138831252 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 0x7fbcec0cb250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80097a0 tx=0x7fbcd802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a37547824380e66f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:40.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.293+0000 7fbceaa00640 1 -- 172.21.15.67:0/4138831252 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8035020 con 0x7fbcec06c940 2024-03-09T09:03:40.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.293+0000 7fbceaa00640 1 -- 172.21.15.67:0/4138831252 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd8031810 con 0x7fbcec06c940 2024-03-09T09:03:40.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.293+0000 7fbceaa00640 1 -- 172.21.15.67:0/4138831252 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8031be0 con 0x7fbcec06c940 2024-03-09T09:03:40.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.294+0000 7fbcf1600640 1 -- 172.21.15.67:0/4138831252 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 msgr2=0x7fbcec0cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:40.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.294+0000 7fbcf1600640 1 --2- 172.21.15.67:0/4138831252 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 0x7fbcec0cb250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80097a0 tx=0x7fbcd802b8b0 comp rx=0 tx=0).stop 2024-03-09T09:03:40.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.294+0000 7fbcf1600640 1 -- 172.21.15.67:0/4138831252 shutdown_connections 2024-03-09T09:03:40.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.294+0000 7fbcf1600640 1 --2- 172.21.15.67:0/4138831252 >> v2:172.21.15.67:3300/0 conn(0x7fbcec0cb790 0x7fbcec156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:40.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.294+0000 7fbcf1600640 1 --2- 172.21.15.67:0/4138831252 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 0x7fbcec0cb250 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:40.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.294+0000 7fbcf1600640 1 -- 172.21.15.67:0/4138831252 >> 172.21.15.67:0/4138831252 conn(0x7fbcec0ad170 msgr2=0x7fbcec0ad580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:40.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.294+0000 7fbcf1600640 1 -- 172.21.15.67:0/4138831252 shutdown_connections 2024-03-09T09:03:40.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.294+0000 7fbcf1600640 1 -- 172.21.15.67:0/4138831252 wait complete. 2024-03-09T09:03:40.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.296+0000 7fbcf1600640 1 Processor -- start 2024-03-09T09:03:40.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.296+0000 7fbcf1600640 1 -- start start 2024-03-09T09:03:40.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.298+0000 7fbcf1600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 0x7fbcec14bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:40.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.298+0000 7fbcf1600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcec0cb790 0x7fbcec1e28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:40.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.298+0000 7fbcf1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec153290 0x7fbcec1eebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:40.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.298+0000 7fbcf1600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0c93e0 con 0x7fbcec153290 2024-03-09T09:03:40.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.298+0000 7fbcf1600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0c7010 con 0x7fbcec0cb790 2024-03-09T09:03:40.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.298+0000 7fbcf1600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0c9560 con 0x7fbcec06c940 2024-03-09T09:03:40.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.298+0000 7fbcf0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec153290 0x7fbcec1eebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:40.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.298+0000 7fbcf0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec153290 0x7fbcec1eebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52914/0 (socket says 172.21.15.67:52914) 2024-03-09T09:03:40.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.298+0000 7fbcf0c00640 1 -- 172.21.15.67:0/1264111107 learned_addr learned my addr 172.21.15.67:0/1264111107 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:40.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.299+0000 7fbcebe00640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 0x7fbcec14bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:40.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.299+0000 7fbcf0c00640 1 -- 172.21.15.67:0/1264111107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec153290 msgr2=0x7fbcec1eebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:03:40.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.299+0000 7fbcf0c00640 1 -- 172.21.15.67:0/1264111107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec153290 msgr2=0x7fbcec1eebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:03:40.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.299+0000 7fbcf0c00640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec153290 0x7fbcec1eebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:03:40.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.299+0000 7fbceb400640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcec0cb790 0x7fbcec1e28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:40.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.299+0000 7fbcf0c00640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec153290 0x7fbcec1eebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:40.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.299+0000 7fbcebe00640 1 -- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcec0cb790 msgr2=0x7fbcec1e28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:40.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.299+0000 7fbcebe00640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcec0cb790 0x7fbcec1e28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:40.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.299+0000 7fbcebe00640 1 -- 172.21.15.67:0/1264111107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec153290 msgr2=0x7fbcec1eebe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:03:40.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.299+0000 7fbcebe00640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec153290 0x7fbcec1eebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:40.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.299+0000 7fbcebe00640 1 -- 172.21.15.67:0/1264111107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec14c4a0 con 0x7fbcec06c940 2024-03-09T09:03:40.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.300+0000 7fbcebe00640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 0x7fbcec14bf60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00caa0 tx=0x7fbcdc00cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:40.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.300+0000 7fbce3e00640 1 -- 172.21.15.67:0/1264111107 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc017070 con 0x7fbcec06c940 2024-03-09T09:03:40.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.300+0000 7fbcf1600640 1 -- 172.21.15.67:0/1264111107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec1ef8d0 con 0x7fbcec06c940 2024-03-09T09:03:40.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.300+0000 7fbce3e00640 1 -- 172.21.15.67:0/1264111107 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc00ab40 con 0x7fbcec06c940 2024-03-09T09:03:40.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.300+0000 7fbce3e00640 1 -- 172.21.15.67:0/1264111107 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc012400 con 0x7fbcec06c940 2024-03-09T09:03:40.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.300+0000 7fbcf1600640 1 -- 172.21.15.67:0/1264111107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec14c860 con 0x7fbcec06c940 2024-03-09T09:03:40.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.302+0000 7fbcf1600640 1 -- 172.21.15.67:0/1264111107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec04f3c0 con 0x7fbcec06c940 2024-03-09T09:03:40.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.303+0000 7fbce3e00640 1 -- 172.21.15.67:0/1264111107 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbcdc01a070 con 0x7fbcec06c940 2024-03-09T09:03:40.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.304+0000 7fbce3e00640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbccc06b4d0 0x7fbccc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:40.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.304+0000 7fbce3e00640 1 -- 172.21.15.67:0/1264111107 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbcdc08a220 con 0x7fbcec06c940 2024-03-09T09:03:40.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.305+0000 7fbceb400640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbccc06b4d0 0x7fbccc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:40.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.307+0000 7fbceb400640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbccc06b4d0 0x7fbccc06d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8036000 tx=0x7fbcd8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.321+0000 7fbce3e00640 1 -- 172.21.15.67:0/1264111107 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbcdc0585d0 con 0x7fbcec06c940 2024-03-09T09:03:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.724+0000 7fbcf1600640 1 -- 172.21.15.67:0/1264111107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbcec06c690 con 0x7fbcec06c940 2024-03-09T09:03:40.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.725+0000 7fbce3e00640 1 -- 172.21.15.67:0/1264111107 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fbcdc05c670 con 0x7fbcec06c940 2024-03-09T09:03:40.727 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":82,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":32,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:03:40.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 -- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbccc06b4d0 msgr2=0x7fbccc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbccc06b4d0 0x7fbccc06d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8036000 tx=0x7fbcd8008030 comp rx=0 tx=0).stop 2024-03-09T09:03:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 -- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 msgr2=0x7fbcec14bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 0x7fbcec14bf60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00caa0 tx=0x7fbcdc00cea0 comp rx=0 tx=0).stop 2024-03-09T09:03:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 -- 172.21.15.67:0/1264111107 shutdown_connections 2024-03-09T09:03:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbccc06b4d0 0x7fbccc06d990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec153290 0x7fbcec1eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcec0cb790 0x7fbcec1e28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 --2- 172.21.15.67:0/1264111107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcec06c940 0x7fbcec14bf60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 -- 172.21.15.67:0/1264111107 >> 172.21.15.67:0/1264111107 conn(0x7fbcec0ad170 msgr2=0x7fbcec0ad550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 -- 172.21.15.67:0/1264111107 shutdown_connections 2024-03-09T09:03:40.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:40.728+0000 7fbcf1600640 1 -- 172.21.15.67:0/1264111107 wait complete. 2024-03-09T09:03:43.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:03:43.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.949+0000 7f5b5fc00640 1 Processor -- start 2024-03-09T09:03:43.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.949+0000 7f5b5fc00640 1 -- start start 2024-03-09T09:03:43.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.951+0000 7f5b5fc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b58154380 0x7f5b580c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:43.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.951+0000 7f5b5fc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b580c6d00 0x7f5b5815a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:43.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.951+0000 7f5b5fc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b58073950 con 0x7f5b580c67b0 2024-03-09T09:03:43.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.951+0000 7f5b5fc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b580706b0 con 0x7f5b58154380 2024-03-09T09:03:43.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.951+0000 7f5b5fc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b58070b40 con 0x7f5b580c6d00 2024-03-09T09:03:43.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.951+0000 7f5b5d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b58154380 0x7f5b580c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:43.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.951+0000 7f5b57e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5b580c67b0 0x7f5b5815a170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34226/0 (socket says 172.21.15.67:34226) 2024-03-09T09:03:43.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.951+0000 7f5b5d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b58154380 0x7f5b580c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52922/0 (socket says 172.21.15.67:52922) 2024-03-09T09:03:43.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.951+0000 7f5b57e00640 1 -- 172.21.15.67:0/162069908 learned_addr learned my addr 172.21.15.67:0/162069908 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:03:43.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.952+0000 7f5b5dc00640 1 --2- 172.21.15.67:0/162069908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b580c6d00 0x7f5b5815a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:43.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.952+0000 7f5b5d200640 1 -- 172.21.15.67:0/162069908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b580c6d00 msgr2=0x7f5b5815a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:43.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.952+0000 7f5b5d200640 1 --2- 172.21.15.67:0/162069908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b580c6d00 0x7f5b5815a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:43.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.952+0000 7f5b5d200640 1 -- 172.21.15.67:0/162069908 >> v1:172.21.15.67:6789/0 conn(0x7f5b580c67b0 legacy=0x7f5b5815a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:43.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.952+0000 7f5b5d200640 1 -- 172.21.15.67:0/162069908 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b58073670 con 0x7f5b58154380 2024-03-09T09:03:43.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.953+0000 7f5b5d200640 1 --2- 172.21.15.67:0/162069908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b58154380 0x7f5b580c9610 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0097a0 tx=0x7f5b4c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5329d009ef8c9ccb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:43.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.954+0000 7f5b57400640 1 -- 172.21.15.67:0/162069908 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c031780 con 0x7f5b58154380 2024-03-09T09:03:43.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.954+0000 7f5b57400640 1 -- 172.21.15.67:0/162069908 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c037070 con 0x7f5b58154380 2024-03-09T09:03:43.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.954+0000 7f5b57400640 1 -- 172.21.15.67:0/162069908 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c031b40 con 0x7f5b58154380 2024-03-09T09:03:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.954+0000 7f5b5fc00640 1 -- 172.21.15.67:0/162069908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b58154380 msgr2=0x7f5b580c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.954+0000 7f5b5fc00640 1 --2- 172.21.15.67:0/162069908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b58154380 0x7f5b580c9610 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0097a0 tx=0x7f5b4c02b820 comp rx=0 tx=0).stop 2024-03-09T09:03:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.954+0000 7f5b5fc00640 1 -- 172.21.15.67:0/162069908 shutdown_connections 2024-03-09T09:03:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.954+0000 7f5b5fc00640 1 --2- 172.21.15.67:0/162069908 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b580c6d00 0x7f5b5815a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.954+0000 7f5b5fc00640 1 --2- 172.21.15.67:0/162069908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b58154380 0x7f5b580c9610 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.954+0000 7f5b5fc00640 1 -- 172.21.15.67:0/162069908 >> 172.21.15.67:0/162069908 conn(0x7f5b5806b220 msgr2=0x7f5b580aa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.954+0000 7f5b5fc00640 1 -- 172.21.15.67:0/162069908 shutdown_connections 2024-03-09T09:03:43.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.955+0000 7f5b5fc00640 1 -- 172.21.15.67:0/162069908 wait complete. 2024-03-09T09:03:43.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.956+0000 7f5b5fc00640 1 Processor -- start 2024-03-09T09:03:43.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.956+0000 7f5b5fc00640 1 -- start start 2024-03-09T09:03:43.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.957+0000 7f5b5fc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b580c67b0 0x7f5b580ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:43.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.957+0000 7f5b5fc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b580c6d00 0x7f5b581481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:43.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.957+0000 7f5b5fc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b58154380 0x7f5b58159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:43.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.957+0000 7f5b5fc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5806fb80 con 0x7f5b580c6d00 2024-03-09T09:03:43.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.957+0000 7f5b5fc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b58070340 con 0x7f5b580c67b0 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.957+0000 7f5b5fc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580c9b50 con 0x7f5b58154380 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.957+0000 7f5b57e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b580c6d00 0x7f5b581481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.957+0000 7f5b57e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b580c6d00 0x7f5b581481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52930/0 (socket says 172.21.15.67:52930) 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.958+0000 7f5b57e00640 1 -- 172.21.15.67:0/3834640070 learned_addr learned my addr 172.21.15.67:0/3834640070 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.958+0000 7f5b5dc00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b58154380 0x7f5b58159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.958+0000 7f5b5d200640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b580c67b0 0x7f5b580ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.958+0000 7f5b57e00640 1 -- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b58154380 msgr2=0x7f5b58159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.958+0000 7f5b57e00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b58154380 0x7f5b58159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.958+0000 7f5b57e00640 1 -- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b580c67b0 msgr2=0x7f5b580ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.958+0000 7f5b57e00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b580c67b0 0x7f5b580ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.958+0000 7f5b57e00640 1 -- 172.21.15.67:0/3834640070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b58148b80 con 0x7f5b580c6d00 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.958+0000 7f5b5dc00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b58154380 0x7f5b58159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:03:43.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.958+0000 7f5b5d200640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b580c67b0 0x7f5b580ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:03:43.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.958+0000 7f5b57e00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b580c6d00 0x7f5b581481e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400c440 tx=0x7f5b44001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:43.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.959+0000 7f5b54c00640 1 -- 172.21.15.67:0/3834640070 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44017070 con 0x7f5b580c6d00 2024-03-09T09:03:43.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.959+0000 7f5b54c00640 1 -- 172.21.15.67:0/3834640070 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b44005ce0 con 0x7f5b580c6d00 2024-03-09T09:03:43.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.959+0000 7f5b54c00640 1 -- 172.21.15.67:0/3834640070 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b440072f0 con 0x7f5b580c6d00 2024-03-09T09:03:43.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.959+0000 7f5b5fc00640 1 -- 172.21.15.67:0/3834640070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b580c8570 con 0x7f5b580c6d00 2024-03-09T09:03:43.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.959+0000 7f5b5fc00640 1 -- 172.21.15.67:0/3834640070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b580c8b30 con 0x7f5b580c6d00 2024-03-09T09:03:43.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.960+0000 7f5b5fc00640 1 -- 172.21.15.67:0/3834640070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b58075a00 con 0x7f5b580c6d00 2024-03-09T09:03:43.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.962+0000 7f5b54c00640 1 -- 172.21.15.67:0/3834640070 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5b4401a070 con 0x7f5b580c6d00 2024-03-09T09:03:43.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.963+0000 7f5b54c00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b3806b520 0x7f5b3806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:43.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.963+0000 7f5b54c00640 1 -- 172.21.15.67:0/3834640070 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f5b440899f0 con 0x7f5b580c6d00 2024-03-09T09:03:43.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.964+0000 7f5b5d200640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b3806b520 0x7f5b3806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:43.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.965+0000 7f5b5d200640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b3806b520 0x7f5b3806d9e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c036000 tx=0x7f5b4c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:43.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:43.969+0000 7f5b54c00640 1 -- 172.21.15.67:0/3834640070 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5b58148b80 con 0x7f5b580c6d00 2024-03-09T09:03:44.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.245+0000 7f5b5fc00640 1 -- 172.21.15.67:0/3834640070 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5b580c37f0 con 0x7f5b580c6d00 2024-03-09T09:03:44.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.245+0000 7f5b54c00640 1 -- 172.21.15.67:0/3834640070 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7f5b440584f0 con 0x7f5b580c6d00 2024-03-09T09:03:44.248 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":86,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:03:44.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.249+0000 7f5b5fc00640 1 -- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b3806b520 msgr2=0x7f5b3806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.250+0000 7f5b5fc00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b3806b520 0x7f5b3806d9e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c036000 tx=0x7f5b4c008030 comp rx=0 tx=0).stop 2024-03-09T09:03:44.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.250+0000 7f5b5fc00640 1 -- 172.21.15.67:0/3834640070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b580c6d00 msgr2=0x7f5b581481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.250+0000 7f5b5fc00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b580c6d00 0x7f5b581481e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400c440 tx=0x7f5b44001e70 comp rx=0 tx=0).stop 2024-03-09T09:03:44.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.250+0000 7f5b5fc00640 1 -- 172.21.15.67:0/3834640070 shutdown_connections 2024-03-09T09:03:44.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.250+0000 7f5b5fc00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b3806b520 0x7f5b3806d9e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.250+0000 7f5b5fc00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b58154380 0x7f5b58159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.250+0000 7f5b5fc00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b580c6d00 0x7f5b581481e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.250+0000 7f5b5fc00640 1 --2- 172.21.15.67:0/3834640070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b580c67b0 0x7f5b580ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.250+0000 7f5b5fc00640 1 -- 172.21.15.67:0/3834640070 >> 172.21.15.67:0/3834640070 conn(0x7f5b5806b220 msgr2=0x7f5b58067620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:44.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.250+0000 7f5b5fc00640 1 -- 172.21.15.67:0/3834640070 shutdown_connections 2024-03-09T09:03:44.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.250+0000 7f5b5fc00640 1 -- 172.21.15.67:0/3834640070 wait complete. 2024-03-09T09:03:44.270 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:03:44.270 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:03:44.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:03:44.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.526+0000 7f702aa00640 1 Processor -- start 2024-03-09T09:03:44.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.526+0000 7f702aa00640 1 -- start start 2024-03-09T09:03:44.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.527+0000 7f702aa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f70240c1cf0 0x7f70241561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.527+0000 7f702aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f70240c26c0 0x7f7024160440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.527+0000 7f702aa00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70240c4ac0 con 0x7f7024153340 2024-03-09T09:03:44.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.527+0000 7f702aa00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7024155d10 con 0x7f70240c26c0 2024-03-09T09:03:44.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.527+0000 7f702aa00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70240ab460 con 0x7f70240c1cf0 2024-03-09T09:03:44.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.527+0000 7f702a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f70240c26c0 0x7f7024160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.527+0000 7f702a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f70240c26c0 0x7f7024160440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52938/0 (socket says 172.21.15.67:52938) 2024-03-09T09:03:44.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.527+0000 7f702a000640 1 -- 172.21.15.67:0/4025580361 learned_addr learned my addr 172.21.15.67:0/4025580361 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:44.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.528+0000 7f7028c00640 1 --2- 172.21.15.67:0/4025580361 >> v2:172.21.15.134:3300/0 conn(0x7f70240c1cf0 0x7f70241561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.528+0000 7f702a000640 1 -- 172.21.15.67:0/4025580361 >> v2:172.21.15.134:3300/0 conn(0x7f70240c1cf0 msgr2=0x7f70241561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:44.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.528+0000 7f702a000640 1 --2- 172.21.15.67:0/4025580361 >> v2:172.21.15.134:3300/0 conn(0x7f70240c1cf0 0x7f70241561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.528+0000 7f702a000640 1 -- 172.21.15.67:0/4025580361 >> v1:172.21.15.134:6789/0 conn(0x7f7024153340 legacy=0x7f70240c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:44.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.528+0000 7f702a000640 1 -- 172.21.15.67:0/4025580361 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70240c47e0 con 0x7f70240c26c0 2024-03-09T09:03:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.529+0000 7f702a000640 1 --2- 172.21.15.67:0/4025580361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70240c26c0 0x7f7024160440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f702406c0f0 tx=0x7f701c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632b44774e8c5d5d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.529+0000 7f7023e00640 1 -- 172.21.15.67:0/4025580361 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c02cb20 con 0x7f70240c26c0 2024-03-09T09:03:44.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.529+0000 7f7023e00640 1 -- 172.21.15.67:0/4025580361 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701c02ccc0 con 0x7f70240c26c0 2024-03-09T09:03:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.529+0000 7f7023e00640 1 -- 172.21.15.67:0/4025580361 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c034610 con 0x7f70240c26c0 2024-03-09T09:03:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.530+0000 7f702aa00640 1 -- 172.21.15.67:0/4025580361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70240c26c0 msgr2=0x7f7024160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.530+0000 7f702aa00640 1 --2- 172.21.15.67:0/4025580361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70240c26c0 0x7f7024160440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f702406c0f0 tx=0x7f701c02bf80 comp rx=0 tx=0).stop 2024-03-09T09:03:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.530+0000 7f702aa00640 1 -- 172.21.15.67:0/4025580361 shutdown_connections 2024-03-09T09:03:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.530+0000 7f702aa00640 1 --2- 172.21.15.67:0/4025580361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f70240c26c0 0x7f7024160440 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.530+0000 7f702aa00640 1 --2- 172.21.15.67:0/4025580361 >> v2:172.21.15.134:3300/0 conn(0x7f70240c1cf0 0x7f70241561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.530+0000 7f702aa00640 1 -- 172.21.15.67:0/4025580361 >> 172.21.15.67:0/4025580361 conn(0x7f702406c150 msgr2=0x7f702406c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.530+0000 7f702aa00640 1 -- 172.21.15.67:0/4025580361 shutdown_connections 2024-03-09T09:03:44.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.530+0000 7f702aa00640 1 -- 172.21.15.67:0/4025580361 wait complete. 2024-03-09T09:03:44.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.532+0000 7f702aa00640 1 Processor -- start 2024-03-09T09:03:44.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.532+0000 7f702aa00640 1 -- start start 2024-03-09T09:03:44.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.533+0000 7f702aa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70240c1cf0 0x7f7024153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.533+0000 7f702aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70240c26c0 0x7f7024158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.533+0000 7f702aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024153340 0x7f7024072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.533+0000 7f702aa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70240c5580 con 0x7f7024153340 2024-03-09T09:03:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.533+0000 7f702aa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70240c5400 con 0x7f70240c26c0 2024-03-09T09:03:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.533+0000 7f702aa00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70241590b0 con 0x7f70240c1cf0 2024-03-09T09:03:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.533+0000 7f702a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024153340 0x7f7024072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.533+0000 7f702a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024153340 0x7f7024072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52946/0 (socket says 172.21.15.67:52946) 2024-03-09T09:03:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.533+0000 7f702a000640 1 -- 172.21.15.67:0/920384015 learned_addr learned my addr 172.21.15.67:0/920384015 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.533+0000 7f7028c00640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70240c26c0 0x7f7024158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.533+0000 7f7029600640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70240c1cf0 0x7f7024153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.534+0000 7f702a000640 1 -- 172.21.15.67:0/920384015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70240c1cf0 msgr2=0x7f7024153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.534+0000 7f702a000640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70240c1cf0 0x7f7024153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.534+0000 7f702a000640 1 -- 172.21.15.67:0/920384015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70240c26c0 msgr2=0x7f7024158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.534+0000 7f702a000640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70240c26c0 0x7f7024158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.534+0000 7f702a000640 1 -- 172.21.15.67:0/920384015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7024148630 con 0x7f7024153340 2024-03-09T09:03:44.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.534+0000 7f702a000640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024153340 0x7f7024072130 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f701c009f40 tx=0x7f701c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:44.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.534+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c043020 con 0x7f7024153340 2024-03-09T09:03:44.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.534+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701c03e450 con 0x7f7024153340 2024-03-09T09:03:44.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.535+0000 7f702aa00640 1 -- 172.21.15.67:0/920384015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7024156e50 con 0x7f7024153340 2024-03-09T09:03:44.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.535+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c009510 con 0x7f7024153340 2024-03-09T09:03:44.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.535+0000 7f702aa00640 1 -- 172.21.15.67:0/920384015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7024158590 con 0x7f7024153340 2024-03-09T09:03:44.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.536+0000 7f702aa00640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024155090 0x7f70241478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.536+0000 7f702aa00640 1 -- 172.21.15.67:0/920384015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70240735b0 con 0x7f7024155090 2024-03-09T09:03:44.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.536+0000 7f7029600640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024155090 0x7f70241478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.537+0000 7f7029600640 1 -- 172.21.15.67:0/920384015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f702404f3c0 con 0x7f7024155090 2024-03-09T09:03:44.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.537+0000 7f7029600640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024155090 0x7f70241478b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7018001eb0 tx=0x7f701800e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:44.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.537+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f701c046020 con 0x7f7024153340 2024-03-09T09:03:44.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.539+0000 7f7021600640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f700406b4d0 0x7f700406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.539+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70180052e0 con 0x7f7024155090 2024-03-09T09:03:44.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.539+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f701c0ace10 con 0x7f7024153340 2024-03-09T09:03:44.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.540+0000 7f7028c00640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f700406b4d0 0x7f700406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.540+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f7018005480 con 0x7f7024155090 2024-03-09T09:03:44.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.540+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024155090 msgr2=0x7f70241478b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.540+0000 7f7021600640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024155090 0x7f70241478b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7018001eb0 tx=0x7f701800e890 comp rx=0 tx=0).stop 2024-03-09T09:03:44.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.540+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024155090 msgr2=0x7f70241478b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:44.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.540+0000 7f7021600640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024155090 0x7f70241478b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.541+0000 7f7028c00640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f700406b4d0 0x7f700406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f70240c81b0 tx=0x7f7010008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:44.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.547+0000 7f702aa00640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024072670 0x7f7024147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.547+0000 7f702aa00640 1 -- 172.21.15.67:0/920384015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7024148330 con 0x7f7024072670 2024-03-09T09:03:44.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.547+0000 7f7029600640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024072670 0x7f7024147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.548+0000 7f7029600640 1 -- 172.21.15.67:0/920384015 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7024156240 con 0x7f7024072670 2024-03-09T09:03:44.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.548+0000 7f7029600640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024072670 0x7f7024147df0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7018001eb0 tx=0x7f7018001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:44.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7018007620 con 0x7f7024072670 2024-03-09T09:03:44.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f7018007620 con 0x7f7024072670 2024-03-09T09:03:44.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024072670 msgr2=0x7f7024147df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f7021600640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024072670 0x7f7024147df0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7018001eb0 tx=0x7f7018001080 comp rx=0 tx=0).stop 2024-03-09T09:03:44.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f7021600640 1 -- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024072670 msgr2=0x7f7024147df0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:44.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f7021600640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024072670 0x7f7024147df0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7018001eb0 tx=0x7f7018001080 comp rx=0 tx=0).stop 2024-03-09T09:03:44.552 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:03:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:03:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:03:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:03:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 86, 2024-03-09T09:03:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:03:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:03:44.552 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:44.553 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:44.554 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:03:44.555 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:44.556 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:44.557 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:44.558 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:44.559 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:44.560 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:03:44.561 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.562 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:44.563 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:03:44.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.563 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:44.563 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:44.563 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:03:44.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.563 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.563 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:03:44.563 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:03:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f702aa00640 1 -- 172.21.15.67:0/920384015 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f700406b4d0 msgr2=0x7f700406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f702aa00640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f700406b4d0 0x7f700406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f70240c81b0 tx=0x7f7010008040 comp rx=0 tx=0).stop 2024-03-09T09:03:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f702aa00640 1 -- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024153340 msgr2=0x7f7024072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f702aa00640 1 --2- 172.21.15.67:0/920384015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7024153340 0x7f7024072130 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f701c009f40 tx=0x7f701c004a40 comp rx=0 tx=0).stop 2024-03-09T09:03:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f702a000640 1 -- 172.21.15.67:0/920384015 reap_dead start 2024-03-09T09:03:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f702a000640 1 -- 172.21.15.67:0/920384015 reap_dead start 2024-03-09T09:03:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f702aa00640 1 -- 172.21.15.67:0/920384015 shutdown_connections 2024-03-09T09:03:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.549+0000 7f702aa00640 1 -- 172.21.15.67:0/920384015 >> 172.21.15.67:0/920384015 conn(0x7f702406c150 msgr2=0x7f702406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.550+0000 7f702aa00640 1 -- 172.21.15.67:0/920384015 shutdown_connections 2024-03-09T09:03:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.550+0000 7f702aa00640 1 -- 172.21.15.67:0/920384015 wait complete. 2024-03-09T09:03:44.568 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:03:44.568 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:03:44.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:03:44.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.764+0000 7fc373e00640 1 Processor -- start 2024-03-09T09:03:44.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.764+0000 7fc373e00640 1 -- start start 2024-03-09T09:03:44.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.765+0000 7fc373e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc37414fd90 0x7fc374156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.765+0000 7fc373e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc37406c1d0 0x7fc37406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.765+0000 7fc373e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc374156bc0 con 0x7fc3740c9380 2024-03-09T09:03:44.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.765+0000 7fc373e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37406ba60 con 0x7fc37406c1d0 2024-03-09T09:03:44.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.765+0000 7fc373e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3740c9890 con 0x7fc37414fd90 2024-03-09T09:03:44.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.766+0000 7fc373400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc37406c1d0 0x7fc37406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.766+0000 7fc372000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc3740c9380 0x7fc3740acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34240/0 (socket says 172.21.15.67:34240) 2024-03-09T09:03:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.766+0000 7fc372000640 1 -- 172.21.15.67:0/685836126 learned_addr learned my addr 172.21.15.67:0/685836126 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:03:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.766+0000 7fc372a00640 1 --2- 172.21.15.67:0/685836126 >> v2:172.21.15.134:3300/0 conn(0x7fc37414fd90 0x7fc374156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.766+0000 7fc373400640 1 -- 172.21.15.67:0/685836126 >> v2:172.21.15.134:3300/0 conn(0x7fc37414fd90 msgr2=0x7fc374156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.766+0000 7fc373400640 1 --2- 172.21.15.67:0/685836126 >> v2:172.21.15.134:3300/0 conn(0x7fc37414fd90 0x7fc374156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.766+0000 7fc373400640 1 -- 172.21.15.67:0/685836126 >> v1:172.21.15.67:6789/0 conn(0x7fc3740c9380 legacy=0x7fc3740acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.766+0000 7fc373400640 1 -- 172.21.15.67:0/685836126 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3740c5f80 con 0x7fc37406c1d0 2024-03-09T09:03:44.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.767+0000 7fc373400640 1 --2- 172.21.15.67:0/685836126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37406c1d0 0x7fc37406c5f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc3740ada40 tx=0x7fc36c02bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfa061fb1693929 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:44.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.768+0000 7fc371600640 1 -- 172.21.15.67:0/685836126 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c039070 con 0x7fc37406c1d0 2024-03-09T09:03:44.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.768+0000 7fc371600640 1 -- 172.21.15.67:0/685836126 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c02cb50 con 0x7fc37406c1d0 2024-03-09T09:03:44.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.768+0000 7fc371600640 1 -- 172.21.15.67:0/685836126 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c034440 con 0x7fc37406c1d0 2024-03-09T09:03:44.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.768+0000 7fc373e00640 1 -- 172.21.15.67:0/685836126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37406c1d0 msgr2=0x7fc37406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.768+0000 7fc373e00640 1 --2- 172.21.15.67:0/685836126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37406c1d0 0x7fc37406c5f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc3740ada40 tx=0x7fc36c02bda0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.768+0000 7fc373e00640 1 -- 172.21.15.67:0/685836126 shutdown_connections 2024-03-09T09:03:44.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.768+0000 7fc373e00640 1 --2- 172.21.15.67:0/685836126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37406c1d0 0x7fc37406c5f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.768+0000 7fc373e00640 1 --2- 172.21.15.67:0/685836126 >> v2:172.21.15.134:3300/0 conn(0x7fc37414fd90 0x7fc374156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.768+0000 7fc373e00640 1 -- 172.21.15.67:0/685836126 >> 172.21.15.67:0/685836126 conn(0x7fc3740cb480 msgr2=0x7fc3740cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:44.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.769+0000 7fc373e00640 1 -- 172.21.15.67:0/685836126 shutdown_connections 2024-03-09T09:03:44.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.769+0000 7fc373e00640 1 -- 172.21.15.67:0/685836126 wait complete. 2024-03-09T09:03:44.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.770+0000 7fc373e00640 1 Processor -- start 2024-03-09T09:03:44.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.770+0000 7fc373e00640 1 -- start start 2024-03-09T09:03:44.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.771+0000 7fc373e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc37406c1d0 0x7fc3740c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.771+0000 7fc373e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3740c9380 0x7fc3740ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.771+0000 7fc373e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37414fd90 0x7fc374151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.771+0000 7fc373e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37406bcb0 con 0x7fc37414fd90 2024-03-09T09:03:44.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.771+0000 7fc373e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740c6520 con 0x7fc37406c1d0 2024-03-09T09:03:44.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.771+0000 7fc373e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740ad610 con 0x7fc3740c9380 2024-03-09T09:03:44.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.772+0000 7fc373400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37414fd90 0x7fc374151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.772+0000 7fc373400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37414fd90 0x7fc374151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52978/0 (socket says 172.21.15.67:52978) 2024-03-09T09:03:44.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.772+0000 7fc373400640 1 -- 172.21.15.67:0/1438784314 learned_addr learned my addr 172.21.15.67:0/1438784314 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:44.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.772+0000 7fc372a00640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc37406c1d0 0x7fc3740c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.772+0000 7fc373400640 1 -- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3740c9380 msgr2=0x7fc3740ac1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.772+0000 7fc373400640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3740c9380 0x7fc3740ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.772+0000 7fc373400640 1 -- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc37406c1d0 msgr2=0x7fc3740c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.772+0000 7fc373400640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc37406c1d0 0x7fc3740c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.772+0000 7fc373400640 1 -- 172.21.15.67:0/1438784314 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc374151300 con 0x7fc37414fd90 2024-03-09T09:03:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.772+0000 7fc373400640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37414fd90 0x7fc374151ae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc36c02c3b0 tx=0x7fc36c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.773+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c039050 con 0x7fc37414fd90 2024-03-09T09:03:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.773+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c045070 con 0x7fc37414fd90 2024-03-09T09:03:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.773+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c009450 con 0x7fc37414fd90 2024-03-09T09:03:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.773+0000 7fc373e00640 1 -- 172.21.15.67:0/1438784314 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc374152ca0 con 0x7fc37414fd90 2024-03-09T09:03:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.773+0000 7fc373e00640 1 -- 172.21.15.67:0/1438784314 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3740ac710 con 0x7fc37414fd90 2024-03-09T09:03:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.774+0000 7fc373e00640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3740c3b70 0x7fc37414d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.774+0000 7fc373e00640 1 -- 172.21.15.67:0/1438784314 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc374152a50 con 0x7fc3740c3b70 2024-03-09T09:03:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.775+0000 7fc372a00640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3740c3b70 0x7fc37414d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.775+0000 7fc372a00640 1 -- 172.21.15.67:0/1438784314 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc37404f3c0 con 0x7fc3740c3b70 2024-03-09T09:03:44.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.775+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc36c03e850 con 0x7fc37414fd90 2024-03-09T09:03:44.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.776+0000 7fc372a00640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3740c3b70 0x7fc37414d5b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3640118e0 tx=0x7fc364011da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:44.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.776+0000 7fc36b400640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc35406b4d0 0x7fc35406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.776+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc36c0ad5e0 con 0x7fc37414fd90 2024-03-09T09:03:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.777+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364019070 con 0x7fc3740c3b70 2024-03-09T09:03:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.777+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc364004300 con 0x7fc3740c3b70 2024-03-09T09:03:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.777+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3740c3b70 msgr2=0x7fc37414d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.777+0000 7fc36b400640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3740c3b70 0x7fc37414d5b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3640118e0 tx=0x7fc364011da0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.777+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3740c3b70 msgr2=0x7fc37414d5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.777+0000 7fc36b400640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3740c3b70 0x7fc37414d5b0 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3640118e0 tx=0x7fc364011da0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.777+0000 7fc372000640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc35406b4d0 0x7fc35406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.778+0000 7fc372000640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc35406b4d0 0x7fc35406d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc360000f60 tx=0x7fc360007a80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:44.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.784+0000 7fc373e00640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc374150cb0 0x7fc37414daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:44.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.784+0000 7fc373e00640 1 -- 172.21.15.67:0/1438784314 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37406d2d0 con 0x7fc374150cb0 2024-03-09T09:03:44.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.785+0000 7fc372a00640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc374150cb0 0x7fc37414daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:44.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.785+0000 7fc372a00640 1 -- 172.21.15.67:0/1438784314 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc3740ca660 con 0x7fc374150cb0 2024-03-09T09:03:44.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.785+0000 7fc372a00640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc374150cb0 0x7fc37414daf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc3640118e0 tx=0x7fc3640046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:44.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.786+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364019040 con 0x7fc374150cb0 2024-03-09T09:03:44.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.786+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5016 (secure 0 0 0) 0x7fc364003df0 con 0x7fc374150cb0 2024-03-09T09:03:44.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.786+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc374150cb0 msgr2=0x7fc37414daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.786+0000 7fc36b400640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc374150cb0 0x7fc37414daf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc3640118e0 tx=0x7fc3640046c0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.786+0000 7fc36b400640 1 -- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc374150cb0 msgr2=0x7fc37414daf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:44.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.786+0000 7fc36b400640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc374150cb0 0x7fc37414daf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.789 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:03:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:03:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:03:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:03:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 86, 2024-03-09T09:03:44.789 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:44.790 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:44.791 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:03:44.792 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:03:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:03:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:03:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:03:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:03:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:03:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:03:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:03:44.793 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:03:44.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:03:44.795 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:03:44.796 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.797 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:44.799 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.800 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:03:44.801 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.802 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:44.802 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:03:44.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:44.802 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:44.802 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:44.802 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:44.802 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:44.802 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:44.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:44.802 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:03:44.802 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:03:44.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.787+0000 7fc373e00640 1 -- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc35406b4d0 msgr2=0x7fc35406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.787+0000 7fc373e00640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc35406b4d0 0x7fc35406d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc360000f60 tx=0x7fc360007a80 comp rx=0 tx=0).stop 2024-03-09T09:03:44.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.787+0000 7fc373e00640 1 -- 172.21.15.67:0/1438784314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37414fd90 msgr2=0x7fc374151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:44.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.787+0000 7fc373e00640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37414fd90 0x7fc374151ae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc36c02c3b0 tx=0x7fc36c004a40 comp rx=0 tx=0).stop 2024-03-09T09:03:44.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.787+0000 7fc373400640 1 -- 172.21.15.67:0/1438784314 reap_dead start 2024-03-09T09:03:44.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.787+0000 7fc373e00640 1 -- 172.21.15.67:0/1438784314 shutdown_connections 2024-03-09T09:03:44.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.787+0000 7fc373e00640 1 --2- 172.21.15.67:0/1438784314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc37414fd90 0x7fc374151ae0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:44.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.787+0000 7fc373e00640 1 -- 172.21.15.67:0/1438784314 >> 172.21.15.67:0/1438784314 conn(0x7fc3740cb480 msgr2=0x7fc3740cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:44.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.787+0000 7fc373e00640 1 -- 172.21.15.67:0/1438784314 shutdown_connections 2024-03-09T09:03:44.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:44.787+0000 7fc373e00640 1 -- 172.21.15.67:0/1438784314 wait complete. 2024-03-09T09:03:44.805 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:03:44.807 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:03:44.807 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:03:44.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:03:44.854 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:03:45.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.064+0000 7f8466200640 1 Processor -- start 2024-03-09T09:03:45.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.065+0000 7f8466200640 1 -- start start 2024-03-09T09:03:45.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.067+0000 7f8466200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f84600c6070 0x7f846006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.067+0000 7f8466200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f846015aba0 0x7f846015b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.067+0000 7f8466200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84600ad370 con 0x7f8460157300 2024-03-09T09:03:45.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.067+0000 7f8466200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846006c9f0 con 0x7f846015aba0 2024-03-09T09:03:45.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.067+0000 7f8466200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846006ba60 con 0x7f84600c6070 2024-03-09T09:03:45.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.067+0000 7f8465800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8460157300 0x7f84600be080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34250/0 (socket says 172.21.15.67:34250) 2024-03-09T09:03:45.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.067+0000 7f8465800640 1 -- 172.21.15.67:0/2361770111 learned_addr learned my addr 172.21.15.67:0/2361770111 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:03:45.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.067+0000 7f845fe00640 1 --2- 172.21.15.67:0/2361770111 >> v2:172.21.15.67:3300/0 conn(0x7f846015aba0 0x7f846015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.068+0000 7f8464e00640 1 --2- 172.21.15.67:0/2361770111 >> v2:172.21.15.134:3300/0 conn(0x7f84600c6070 0x7f846006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.068+0000 7f845f400640 1 -- 172.21.15.67:0/2361770111 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528187932 0 0) 0x7f84600ad370 con 0x7f8460157300 2024-03-09T09:03:45.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.068+0000 7f845fe00640 1 -- 172.21.15.67:0/2361770111 >> v2:172.21.15.134:3300/0 conn(0x7f84600c6070 msgr2=0x7f846006c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:45.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.068+0000 7f845fe00640 1 --2- 172.21.15.67:0/2361770111 >> v2:172.21.15.134:3300/0 conn(0x7f84600c6070 0x7f846006c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.068+0000 7f845fe00640 1 -- 172.21.15.67:0/2361770111 >> v1:172.21.15.67:6789/0 conn(0x7f8460157300 legacy=0x7f84600be080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.068+0000 7f845fe00640 1 -- 172.21.15.67:0/2361770111 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84600ad650 con 0x7f846015aba0 2024-03-09T09:03:45.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.070+0000 7f845fe00640 1 --2- 172.21.15.67:0/2361770111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f846015aba0 0x7f846015b590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8460073720 tx=0x7f844c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc491059707fe868 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:45.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.070+0000 7f845f400640 1 -- 172.21.15.67:0/2361770111 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0315a0 con 0x7f846015aba0 2024-03-09T09:03:45.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.070+0000 7f845f400640 1 -- 172.21.15.67:0/2361770111 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c037070 con 0x7f846015aba0 2024-03-09T09:03:45.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.070+0000 7f845f400640 1 -- 172.21.15.67:0/2361770111 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c031960 con 0x7f846015aba0 2024-03-09T09:03:45.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.071+0000 7f8466200640 1 -- 172.21.15.67:0/2361770111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f846015aba0 msgr2=0x7f846015b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.071+0000 7f8466200640 1 --2- 172.21.15.67:0/2361770111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f846015aba0 0x7f846015b590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8460073720 tx=0x7f844c02b840 comp rx=0 tx=0).stop 2024-03-09T09:03:45.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.071+0000 7f8466200640 1 -- 172.21.15.67:0/2361770111 shutdown_connections 2024-03-09T09:03:45.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.071+0000 7f8466200640 1 --2- 172.21.15.67:0/2361770111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f846015aba0 0x7f846015b590 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.071+0000 7f8466200640 1 --2- 172.21.15.67:0/2361770111 >> v2:172.21.15.134:3300/0 conn(0x7f84600c6070 0x7f846006c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.071+0000 7f8466200640 1 -- 172.21.15.67:0/2361770111 >> 172.21.15.67:0/2361770111 conn(0x7f84600cb480 msgr2=0x7f84600cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:45.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.072+0000 7f8466200640 1 -- 172.21.15.67:0/2361770111 shutdown_connections 2024-03-09T09:03:45.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.072+0000 7f8466200640 1 -- 172.21.15.67:0/2361770111 wait complete. 2024-03-09T09:03:45.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.073+0000 7f8466200640 1 Processor -- start 2024-03-09T09:03:45.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.074+0000 7f8466200640 1 -- start start 2024-03-09T09:03:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.075+0000 7f8466200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84600c6070 0x7f84600c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.075+0000 7f8466200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8460157300 0x7f84600ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.075+0000 7f8466200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846015aba0 0x7f84600ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.075+0000 7f8466200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846006d690 con 0x7f84600c6070 2024-03-09T09:03:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.075+0000 7f8466200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846006c020 con 0x7f8460157300 2024-03-09T09:03:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.075+0000 7f8466200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846015b000 con 0x7f846015aba0 2024-03-09T09:03:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.075+0000 7f8464e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84600c6070 0x7f84600c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.075+0000 7f8464e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84600c6070 0x7f84600c3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52996/0 (socket says 172.21.15.67:52996) 2024-03-09T09:03:45.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.075+0000 7f8464e00640 1 -- 172.21.15.67:0/2446246859 learned_addr learned my addr 172.21.15.67:0/2446246859 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:45.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.076+0000 7f8465800640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846015aba0 0x7f84600ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.076+0000 7f845fe00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8460157300 0x7f84600ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.076+0000 7f8464e00640 1 -- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846015aba0 msgr2=0x7f84600ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.076+0000 7f8464e00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f846015aba0 0x7f84600ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.076+0000 7f8464e00640 1 -- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8460157300 msgr2=0x7f84600ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.076+0000 7f8464e00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8460157300 0x7f84600ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.076+0000 7f8464e00640 1 -- 172.21.15.67:0/2446246859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84600c4420 con 0x7f84600c6070 2024-03-09T09:03:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.076+0000 7f8464e00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84600c6070 0x7f84600c3ee0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f845400e870 tx=0x7f845400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.077+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454017070 con 0x7f84600c6070 2024-03-09T09:03:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.077+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845400c940 con 0x7f84600c6070 2024-03-09T09:03:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.077+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845400cd30 con 0x7f84600c6070 2024-03-09T09:03:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.077+0000 7f8466200640 1 -- 172.21.15.67:0/2446246859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84600c4ba0 con 0x7f84600c6070 2024-03-09T09:03:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.077+0000 7f8466200640 1 -- 172.21.15.67:0/2446246859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84600c5160 con 0x7f84600c6070 2024-03-09T09:03:45.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.078+0000 7f8466200640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460151b10 0x7f84600c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.078+0000 7f8466200640 1 -- 172.21.15.67:0/2446246859 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846004f3c0 con 0x7f8460151b10 2024-03-09T09:03:45.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.079+0000 7f845fe00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460151b10 0x7f84600c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.079+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f845401a070 con 0x7f84600c6070 2024-03-09T09:03:45.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.080+0000 7f845fe00640 1 -- 172.21.15.67:0/2446246859 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84600c54f0 con 0x7f8460151b10 2024-03-09T09:03:45.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.080+0000 7f845fe00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460151b10 0x7f84600c36e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f846008d2e0 tx=0x7f844c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:45.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.081+0000 7f845cc00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f844006b500 0x7f844006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.081+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f845408a500 con 0x7f84600c6070 2024-03-09T09:03:45.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.081+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c031e80 con 0x7f8460151b10 2024-03-09T09:03:45.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.081+0000 7f8465800640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f844006b500 0x7f844006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.082+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f844c037040 con 0x7f8460151b10 2024-03-09T09:03:45.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.082+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460151b10 msgr2=0x7f84600c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.082+0000 7f845cc00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460151b10 0x7f84600c36e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f846008d2e0 tx=0x7f844c004730 comp rx=0 tx=0).stop 2024-03-09T09:03:45.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.082+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460151b10 msgr2=0x7f84600c36e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:45.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.082+0000 7f845cc00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460151b10 0x7f84600c36e0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f846008d2e0 tx=0x7f844c004730 comp rx=0 tx=0).stop 2024-03-09T09:03:45.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.083+0000 7f8465800640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f844006b500 0x7f844006d9c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8458005fd0 tx=0x7f8458007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:45.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.090+0000 7f8466200640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460152060 0x7f8460150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.090+0000 7f8466200640 1 -- 172.21.15.67:0/2446246859 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84600ad0b0 con 0x7f8460152060 2024-03-09T09:03:45.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.092+0000 7f845fe00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460152060 0x7f8460150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.092+0000 7f845fe00640 1 -- 172.21.15.67:0/2446246859 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f846006d2e0 con 0x7f8460152060 2024-03-09T09:03:45.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.093+0000 7f845fe00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460152060 0x7f8460150ff0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f844c02b880 tx=0x7f844c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:45.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.093+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c050030 con 0x7f8460152060 2024-03-09T09:03:45.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.094+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5186 (secure 0 0 0) 0x7f844c04c670 con 0x7f8460152060 2024-03-09T09:03:45.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.094+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460152060 msgr2=0x7f8460150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.094+0000 7f845cc00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460152060 0x7f8460150ff0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f844c02b880 tx=0x7f844c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.094+0000 7f845cc00640 1 -- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460152060 msgr2=0x7f8460150ff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:45.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.094+0000 7f845cc00640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8460152060 0x7f8460150ff0 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f844c02b880 tx=0x7f844c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.097 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:03:45.097 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:03:45.097 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:03:45.097 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:03:45.097 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 86, 2024-03-09T09:03:45.097 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:03:45.097 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:03:45.097 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:03:45.097 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:45.099 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:03:45.100 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:45.101 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:03:45.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:45.103 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.104 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:45.105 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:45.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:45.108 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:45.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:45.110 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:03:45.111 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:03:45.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.094+0000 7f8466200640 1 -- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f844006b500 msgr2=0x7f844006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.094+0000 7f8466200640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f844006b500 0x7f844006d9c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8458005fd0 tx=0x7f8458007e80 comp rx=0 tx=0).stop 2024-03-09T09:03:45.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.095+0000 7f8466200640 1 -- 172.21.15.67:0/2446246859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84600c6070 msgr2=0x7f84600c3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.095+0000 7f8466200640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84600c6070 0x7f84600c3ee0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f845400e870 tx=0x7f845400ed30 comp rx=0 tx=0).stop 2024-03-09T09:03:45.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.095+0000 7f8465800640 1 -- 172.21.15.67:0/2446246859 reap_dead start 2024-03-09T09:03:45.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.095+0000 7f8466200640 1 -- 172.21.15.67:0/2446246859 shutdown_connections 2024-03-09T09:03:45.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.095+0000 7f8466200640 1 --2- 172.21.15.67:0/2446246859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84600c6070 0x7f84600c3ee0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.095+0000 7f8466200640 1 -- 172.21.15.67:0/2446246859 >> 172.21.15.67:0/2446246859 conn(0x7f84600cb480 msgr2=0x7f84600ca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:45.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.095+0000 7f8466200640 1 -- 172.21.15.67:0/2446246859 shutdown_connections 2024-03-09T09:03:45.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.095+0000 7f8466200640 1 -- 172.21.15.67:0/2446246859 wait complete. 2024-03-09T09:03:45.113 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:03:45.114 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:03:45.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:03:45.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.313+0000 7f5baca00640 1 Processor -- start 2024-03-09T09:03:45.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.314+0000 7f5baca00640 1 -- start start 2024-03-09T09:03:45.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.315+0000 7f5baca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ba8152a20 0x7f5ba80aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.315+0000 7f5baca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba80ab3c0 con 0x7f5ba806dbe0 2024-03-09T09:03:45.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.315+0000 7f5baca00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba80aab20 con 0x7f5ba80c30f0 2024-03-09T09:03:45.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.315+0000 7f5baca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80c82d0 con 0x7f5ba8152a20 2024-03-09T09:03:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.316+0000 7f5ba5e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f5ba80c30f0 0x7f5ba81490f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:50858/0 (socket says 172.21.15.67:50858) 2024-03-09T09:03:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.316+0000 7f5ba6800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5ba806dbe0 0x7f5ba8072940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34254/0 (socket says 172.21.15.67:34254) 2024-03-09T09:03:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.316+0000 7f5ba5e00640 1 -- 172.21.15.67:0/1900352600 learned_addr learned my addr 172.21.15.67:0/1900352600 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:03:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.316+0000 7f5b9fe00640 1 --2- 172.21.15.67:0/1900352600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ba8152a20 0x7f5ba80aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.316+0000 7f5ba5400640 1 -- 172.21.15.67:0/1900352600 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448731727 0 0) 0x7f5ba80aab20 con 0x7f5ba80c30f0 2024-03-09T09:03:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.316+0000 7f5ba5400640 1 -- 172.21.15.67:0/1900352600 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b8c003520 con 0x7f5ba80c30f0 2024-03-09T09:03:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.316+0000 7f5ba5400640 1 -- 172.21.15.67:0/1900352600 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502128783 0 0) 0x7f5ba0001490 con 0x7f5ba806dbe0 2024-03-09T09:03:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.316+0000 7f5ba5400640 1 -- 172.21.15.67:0/1900352600 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba80aab20 con 0x7f5ba806dbe0 2024-03-09T09:03:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.316+0000 7f5b9fe00640 1 -- 172.21.15.67:0/1900352600 >> v1:172.21.15.134:6789/0 conn(0x7f5ba80c30f0 legacy=0x7f5ba81490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.317+0000 7f5b9fe00640 1 -- 172.21.15.67:0/1900352600 >> v1:172.21.15.67:6789/0 conn(0x7f5ba806dbe0 legacy=0x7f5ba8072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.317+0000 7f5b9fe00640 1 -- 172.21.15.67:0/1900352600 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80aae00 con 0x7f5ba8152a20 2024-03-09T09:03:45.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.318+0000 7f5b9fe00640 1 --2- 172.21.15.67:0/1900352600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ba8152a20 0x7f5ba80aa420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5b900097d0 tx=0x7f5b9002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d10087a7af9553ca server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:45.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.319+0000 7f5ba5400640 1 -- 172.21.15.67:0/1900352600 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90035020 con 0x7f5ba8152a20 2024-03-09T09:03:45.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.319+0000 7f5ba5400640 1 -- 172.21.15.67:0/1900352600 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b90031800 con 0x7f5ba8152a20 2024-03-09T09:03:45.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.319+0000 7f5ba5400640 1 -- 172.21.15.67:0/1900352600 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90031bd0 con 0x7f5ba8152a20 2024-03-09T09:03:45.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.320+0000 7f5baca00640 1 -- 172.21.15.67:0/1900352600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ba8152a20 msgr2=0x7f5ba80aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.320+0000 7f5baca00640 1 --2- 172.21.15.67:0/1900352600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ba8152a20 0x7f5ba80aa420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5b900097d0 tx=0x7f5b9002b8a0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.320+0000 7f5baca00640 1 -- 172.21.15.67:0/1900352600 shutdown_connections 2024-03-09T09:03:45.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.320+0000 7f5baca00640 1 --2- 172.21.15.67:0/1900352600 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ba8152a20 0x7f5ba80aa420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.320+0000 7f5baca00640 1 -- 172.21.15.67:0/1900352600 >> 172.21.15.67:0/1900352600 conn(0x7f5ba80c7370 msgr2=0x7f5ba80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.320+0000 7f5baca00640 1 -- 172.21.15.67:0/1900352600 shutdown_connections 2024-03-09T09:03:45.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.320+0000 7f5baca00640 1 -- 172.21.15.67:0/1900352600 wait complete. 2024-03-09T09:03:45.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.322+0000 7f5baca00640 1 Processor -- start 2024-03-09T09:03:45.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.322+0000 7f5baca00640 1 -- start start 2024-03-09T09:03:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.323+0000 7f5baca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ba806dbe0 0x7f5ba81eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.323+0000 7f5baca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ba80c30f0 0x7f5ba814b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.323+0000 7f5baca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ba8152a20 0x7f5ba814b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.323+0000 7f5baca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8149800 con 0x7f5ba8152a20 2024-03-09T09:03:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.323+0000 7f5baca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba806d6c0 con 0x7f5ba80c30f0 2024-03-09T09:03:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.323+0000 7f5baca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8149980 con 0x7f5ba806dbe0 2024-03-09T09:03:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.323+0000 7f5ba6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ba8152a20 0x7f5ba814b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.323+0000 7f5ba6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ba8152a20 0x7f5ba814b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53010/0 (socket says 172.21.15.67:53010) 2024-03-09T09:03:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.323+0000 7f5ba6800640 1 -- 172.21.15.67:0/2305403525 learned_addr learned my addr 172.21.15.67:0/2305403525 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.323+0000 7f5b9fe00640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ba80c30f0 0x7f5ba814b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.323+0000 7f5ba5e00640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ba806dbe0 0x7f5ba81eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.324+0000 7f5ba6800640 1 -- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ba806dbe0 msgr2=0x7f5ba81eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.324+0000 7f5ba6800640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ba806dbe0 0x7f5ba81eeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.324+0000 7f5ba6800640 1 -- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ba80c30f0 msgr2=0x7f5ba814b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.324+0000 7f5ba6800640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ba80c30f0 0x7f5ba814b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.324+0000 7f5ba6800640 1 -- 172.21.15.67:0/2305403525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba8152780 con 0x7f5ba8152a20 2024-03-09T09:03:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.324+0000 7f5ba6800640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ba8152a20 0x7f5ba814b6a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000c7a0 tx=0x7f5ba000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.325+0000 7f5b9e000640 1 -- 172.21.15.67:0/2305403525 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0017070 con 0x7f5ba8152a20 2024-03-09T09:03:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.325+0000 7f5b9e000640 1 -- 172.21.15.67:0/2305403525 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba000a640 con 0x7f5ba8152a20 2024-03-09T09:03:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.325+0000 7f5b9e000640 1 -- 172.21.15.67:0/2305403525 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba000aa30 con 0x7f5ba8152a20 2024-03-09T09:03:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.325+0000 7f5baca00640 1 -- 172.21.15.67:0/2305403525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba8148d60 con 0x7f5ba8152a20 2024-03-09T09:03:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.325+0000 7f5baca00640 1 -- 172.21.15.67:0/2305403525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba8148a80 con 0x7f5ba8152a20 2024-03-09T09:03:45.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.326+0000 7f5baca00640 1 -- 172.21.15.67:0/2305403525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba8075a00 con 0x7f5ba8152a20 2024-03-09T09:03:45.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.327+0000 7f5b9e000640 1 -- 172.21.15.67:0/2305403525 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5ba001a070 con 0x7f5ba8152a20 2024-03-09T09:03:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.328+0000 7f5b9e000640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b8406b4d0 0x7f5b8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:45.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.328+0000 7f5b9e000640 1 -- 172.21.15.67:0/2305403525 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f5ba0089b70 con 0x7f5ba8152a20 2024-03-09T09:03:45.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.329+0000 7f5ba5e00640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b8406b4d0 0x7f5b8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:45.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.330+0000 7f5ba5e00640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b8406b4d0 0x7f5b8406d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000f60 tx=0x7f5b98007d70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:45.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.334+0000 7f5b9e000640 1 -- 172.21.15.67:0/2305403525 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5ba80ab3c0 con 0x7f5ba8152a20 2024-03-09T09:03:45.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.709+0000 7f5baca00640 1 -- 172.21.15.67:0/2305403525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f5ba80ace10 con 0x7f5ba8152a20 2024-03-09T09:03:45.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.710+0000 7f5b9e000640 1 -- 172.21.15.67:0/2305403525 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f5ba0057fa0 con 0x7f5ba8152a20 2024-03-09T09:03:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.712+0000 7f5baca00640 1 -- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b8406b4d0 msgr2=0x7f5b8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.712+0000 7f5baca00640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b8406b4d0 0x7f5b8406d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000f60 tx=0x7f5b98007d70 comp rx=0 tx=0).stop 2024-03-09T09:03:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.712+0000 7f5baca00640 1 -- 172.21.15.67:0/2305403525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ba8152a20 msgr2=0x7f5ba814b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.712+0000 7f5baca00640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ba8152a20 0x7f5ba814b6a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000c7a0 tx=0x7f5ba000cc60 comp rx=0 tx=0).stop 2024-03-09T09:03:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.712+0000 7f5baca00640 1 -- 172.21.15.67:0/2305403525 shutdown_connections 2024-03-09T09:03:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.712+0000 7f5baca00640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b8406b4d0 0x7f5b8406d990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.712+0000 7f5baca00640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ba8152a20 0x7f5ba814b6a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.712+0000 7f5baca00640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ba80c30f0 0x7f5ba814b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.712+0000 7f5baca00640 1 --2- 172.21.15.67:0/2305403525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ba806dbe0 0x7f5ba81eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:45.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.712+0000 7f5baca00640 1 -- 172.21.15.67:0/2305403525 >> 172.21.15.67:0/2305403525 conn(0x7f5ba80c7370 msgr2=0x7f5ba8074640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:45.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.713+0000 7f5baca00640 1 -- 172.21.15.67:0/2305403525 shutdown_connections 2024-03-09T09:03:45.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:45.713+0000 7f5baca00640 1 -- 172.21.15.67:0/2305403525 wait complete. 2024-03-09T09:03:45.732 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:03:55.733 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:03:55.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:03:55.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.943+0000 7f281e400640 1 Processor -- start 2024-03-09T09:03:55.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.944+0000 7f281e400640 1 -- start start 2024-03-09T09:03:55.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.946+0000 7f281e400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2818153250 0x7f281806fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:55.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.946+0000 7f281e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818073010 0x7f2818070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:55.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.946+0000 7f281e400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2818153630 con 0x7f2818070890 2024-03-09T09:03:55.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.946+0000 7f281e400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28180aa5f0 con 0x7f2818153250 2024-03-09T09:03:55.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.946+0000 7f281e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28180abfe0 con 0x7f2818073010 2024-03-09T09:03:55.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.947+0000 7f2817e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818073010 0x7f2818070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:55.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.947+0000 7f2817e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818073010 0x7f2818070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:33534/0 (socket says 172.21.15.67:33534) 2024-03-09T09:03:55.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.947+0000 7f2817e00640 1 -- 172.21.15.67:0/3762233366 learned_addr learned my addr 172.21.15.67:0/3762233366 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:55.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.947+0000 7f281da00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f2818070890 0x7f28180c28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:56980/0 (socket says 172.21.15.67:56980) 2024-03-09T09:03:55.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.947+0000 7f281d000640 1 --2- 172.21.15.67:0/3762233366 >> v2:172.21.15.134:3300/0 conn(0x7f2818153250 0x7f281806fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:55.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.947+0000 7f2817400640 1 -- 172.21.15.67:0/3762233366 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382316091 0 0) 0x7f2818153630 con 0x7f2818070890 2024-03-09T09:03:55.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.947+0000 7f2817400640 1 -- 172.21.15.67:0/3762233366 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2800003520 con 0x7f2818070890 2024-03-09T09:03:55.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.948+0000 7f2817e00640 1 -- 172.21.15.67:0/3762233366 >> v2:172.21.15.134:3300/0 conn(0x7f2818153250 msgr2=0x7f281806fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:55.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.948+0000 7f2817e00640 1 --2- 172.21.15.67:0/3762233366 >> v2:172.21.15.134:3300/0 conn(0x7f2818153250 0x7f281806fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:55.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.948+0000 7f2817e00640 1 -- 172.21.15.67:0/3762233366 >> v1:172.21.15.134:6789/0 conn(0x7f2818070890 legacy=0x7f28180c28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:55.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.948+0000 7f2817e00640 1 -- 172.21.15.67:0/3762233366 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28180abd00 con 0x7f2818073010 2024-03-09T09:03:55.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.949+0000 7f2817e00640 1 --2- 172.21.15.67:0/3762233366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818073010 0x7f2818070350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f28040098a0 tx=0x7f280402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83685d1763cffffb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:55.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.950+0000 7f2817400640 1 -- 172.21.15.67:0/3762233366 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804034070 con 0x7f2818073010 2024-03-09T09:03:55.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.950+0000 7f2817400640 1 -- 172.21.15.67:0/3762233366 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804030840 con 0x7f2818073010 2024-03-09T09:03:55.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.950+0000 7f2817400640 1 -- 172.21.15.67:0/3762233366 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804030bf0 con 0x7f2818073010 2024-03-09T09:03:55.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.951+0000 7f281e400640 1 -- 172.21.15.67:0/3762233366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818073010 msgr2=0x7f2818070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:55.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.951+0000 7f281e400640 1 --2- 172.21.15.67:0/3762233366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818073010 0x7f2818070350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f28040098a0 tx=0x7f280402b980 comp rx=0 tx=0).stop 2024-03-09T09:03:55.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.951+0000 7f281e400640 1 -- 172.21.15.67:0/3762233366 shutdown_connections 2024-03-09T09:03:55.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.951+0000 7f281e400640 1 --2- 172.21.15.67:0/3762233366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818073010 0x7f2818070350 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:55.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.951+0000 7f281e400640 1 --2- 172.21.15.67:0/3762233366 >> v2:172.21.15.134:3300/0 conn(0x7f2818153250 0x7f281806fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:55.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.951+0000 7f281e400640 1 -- 172.21.15.67:0/3762233366 >> 172.21.15.67:0/3762233366 conn(0x7f2818066d90 msgr2=0x7f2818067170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:55.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.951+0000 7f281e400640 1 -- 172.21.15.67:0/3762233366 shutdown_connections 2024-03-09T09:03:55.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.951+0000 7f281e400640 1 -- 172.21.15.67:0/3762233366 wait complete. 2024-03-09T09:03:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.953+0000 7f281e400640 1 Processor -- start 2024-03-09T09:03:55.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.953+0000 7f281e400640 1 -- start start 2024-03-09T09:03:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.954+0000 7f281e400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2818070890 0x7f281806fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.954+0000 7f281e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2818073010 0x7f28181e1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.954+0000 7f281e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818153250 0x7f28181e1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.954+0000 7f281e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281806f8f0 con 0x7f2818073010 2024-03-09T09:03:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.954+0000 7f281e400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2818156aa0 con 0x7f2818070890 2024-03-09T09:03:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.954+0000 7f281e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28180c2fb0 con 0x7f2818153250 2024-03-09T09:03:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.954+0000 7f2817e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2818073010 0x7f28181e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.954+0000 7f2817e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2818073010 0x7f28181e1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40550/0 (socket says 172.21.15.67:40550) 2024-03-09T09:03:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.954+0000 7f2817e00640 1 -- 172.21.15.67:0/3243773385 learned_addr learned my addr 172.21.15.67:0/3243773385 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.954+0000 7f281da00640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818153250 0x7f28181e1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.954+0000 7f281d000640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2818070890 0x7f281806fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.955+0000 7f2817e00640 1 -- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818153250 msgr2=0x7f28181e1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.955+0000 7f2817e00640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818153250 0x7f28181e1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.955+0000 7f2817e00640 1 -- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2818070890 msgr2=0x7f281806fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.955+0000 7f2817e00640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2818070890 0x7f281806fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.955+0000 7f2817e00640 1 -- 172.21.15.67:0/3243773385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28181ee8c0 con 0x7f2818073010 2024-03-09T09:03:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.955+0000 7f2817e00640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2818073010 0x7f28181e1340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f280402b9c0 tx=0x7f2804004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.956+0000 7f2814c00640 1 -- 172.21.15.67:0/3243773385 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804048070 con 0x7f2818073010 2024-03-09T09:03:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.956+0000 7f2814c00640 1 -- 172.21.15.67:0/3243773385 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804039a90 con 0x7f2818073010 2024-03-09T09:03:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.956+0000 7f2814c00640 1 -- 172.21.15.67:0/3243773385 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804034040 con 0x7f2818073010 2024-03-09T09:03:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.956+0000 7f281e400640 1 -- 172.21.15.67:0/3243773385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28181eeba0 con 0x7f2818073010 2024-03-09T09:03:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.956+0000 7f281e400640 1 -- 172.21.15.67:0/3243773385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28181ef160 con 0x7f2818073010 2024-03-09T09:03:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.958+0000 7f281e400640 1 -- 172.21.15.67:0/3243773385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281804f3c0 con 0x7f2818073010 2024-03-09T09:03:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.959+0000 7f2814c00640 1 -- 172.21.15.67:0/3243773385 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2804037070 con 0x7f2818073010 2024-03-09T09:03:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.960+0000 7f2814c00640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f27f806b500 0x7f27f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:55.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.960+0000 7f2814c00640 1 -- 172.21.15.67:0/3243773385 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f28040abf90 con 0x7f2818073010 2024-03-09T09:03:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.960+0000 7f281d000640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f27f806b500 0x7f27f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:55.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.961+0000 7f281d000640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f27f806b500 0x7f27f806d9c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f28181ef980 tx=0x7f280c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:55.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:55.966+0000 7f2814c00640 1 -- 172.21.15.67:0/3243773385 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f280407a3c0 con 0x7f2818073010 2024-03-09T09:03:56.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.289+0000 7f281e400640 1 -- 172.21.15.67:0/3243773385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f28180c6430 con 0x7f2818073010 2024-03-09T09:03:56.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.290+0000 7f2814c00640 1 -- 172.21.15.67:0/3243773385 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f280407e460 con 0x7f2818073010 2024-03-09T09:03:56.292 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":86,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:03:56.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.292+0000 7f281e400640 1 -- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f27f806b500 msgr2=0x7f27f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.292+0000 7f281e400640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f27f806b500 0x7f27f806d9c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f28181ef980 tx=0x7f280c009190 comp rx=0 tx=0).stop 2024-03-09T09:03:56.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.292+0000 7f281e400640 1 -- 172.21.15.67:0/3243773385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2818073010 msgr2=0x7f28181e1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.292+0000 7f281e400640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2818073010 0x7f28181e1340 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f280402b9c0 tx=0x7f2804004730 comp rx=0 tx=0).stop 2024-03-09T09:03:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.293+0000 7f281e400640 1 -- 172.21.15.67:0/3243773385 shutdown_connections 2024-03-09T09:03:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.293+0000 7f281e400640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f27f806b500 0x7f27f806d9c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.293+0000 7f281e400640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2818153250 0x7f28181e1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.293+0000 7f281e400640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2818073010 0x7f28181e1340 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.293+0000 7f281e400640 1 --2- 172.21.15.67:0/3243773385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2818070890 0x7f281806fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.293+0000 7f281e400640 1 -- 172.21.15.67:0/3243773385 >> 172.21.15.67:0/3243773385 conn(0x7f2818066d90 msgr2=0x7f281806d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.293+0000 7f281e400640 1 -- 172.21.15.67:0/3243773385 shutdown_connections 2024-03-09T09:03:56.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.293+0000 7f281e400640 1 -- 172.21.15.67:0/3243773385 wait complete. 2024-03-09T09:03:56.313 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:03:56.313 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:03:56.313 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:03:56.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:03:56.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.539+0000 7f3b2cc00640 1 Processor -- start 2024-03-09T09:03:56.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.539+0000 7f3b2cc00640 1 -- start start 2024-03-09T09:03:56.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.541+0000 7f3b2cc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3b280ac9a0 0x7f3b281510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.541+0000 7f3b2cc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b280c2780 0x7f3b28151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.541+0000 7f3b2cc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3b28071190 0x7f3b28154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.541+0000 7f3b2cc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b2806d590 con 0x7f3b28071190 2024-03-09T09:03:56.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.541+0000 7f3b2cc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b280c3740 con 0x7f3b280ac9a0 2024-03-09T09:03:56.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.541+0000 7f3b2cc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280c8200 con 0x7f3b280c2780 2024-03-09T09:03:56.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.541+0000 7f3b26800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3b28071190 0x7f3b28154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.541+0000 7f3b25400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b280c2780 0x7f3b28151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.541+0000 7f3b26800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3b28071190 0x7f3b28154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40554/0 (socket says 172.21.15.67:40554) 2024-03-09T09:03:56.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.541+0000 7f3b26800640 1 -- 172.21.15.67:0/3061451525 learned_addr learned my addr 172.21.15.67:0/3061451525 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:56.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.541+0000 7f3b25e00640 1 --2- 172.21.15.67:0/3061451525 >> v2:172.21.15.134:3300/0 conn(0x7f3b280ac9a0 0x7f3b281510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.542+0000 7f3b26800640 1 -- 172.21.15.67:0/3061451525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b280c2780 msgr2=0x7f3b28151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:56.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.542+0000 7f3b26800640 1 --2- 172.21.15.67:0/3061451525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b280c2780 0x7f3b28151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.542+0000 7f3b26800640 1 -- 172.21.15.67:0/3061451525 >> v2:172.21.15.134:3300/0 conn(0x7f3b280ac9a0 msgr2=0x7f3b281510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:56.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.542+0000 7f3b26800640 1 --2- 172.21.15.67:0/3061451525 >> v2:172.21.15.134:3300/0 conn(0x7f3b280ac9a0 0x7f3b281510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.542+0000 7f3b26800640 1 -- 172.21.15.67:0/3061451525 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280c3990 con 0x7f3b28071190 2024-03-09T09:03:56.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.543+0000 7f3b26800640 1 --2- 172.21.15.67:0/3061451525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28071190 0x7f3b28154190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b2806ed40 tx=0x7f3b2002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd7b9e4707f1b8b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:56.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.543+0000 7f3b24a00640 1 -- 172.21.15.67:0/3061451525 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2002c9f0 con 0x7f3b28071190 2024-03-09T09:03:56.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.544+0000 7f3b24a00640 1 -- 172.21.15.67:0/3061451525 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2002cb90 con 0x7f3b28071190 2024-03-09T09:03:56.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.544+0000 7f3b24a00640 1 -- 172.21.15.67:0/3061451525 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200343f0 con 0x7f3b28071190 2024-03-09T09:03:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.544+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3061451525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28071190 msgr2=0x7f3b28154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.544+0000 7f3b2cc00640 1 --2- 172.21.15.67:0/3061451525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28071190 0x7f3b28154190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b2806ed40 tx=0x7f3b2002be50 comp rx=0 tx=0).stop 2024-03-09T09:03:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.544+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3061451525 shutdown_connections 2024-03-09T09:03:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.544+0000 7f3b2cc00640 1 --2- 172.21.15.67:0/3061451525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28071190 0x7f3b28154190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.544+0000 7f3b2cc00640 1 --2- 172.21.15.67:0/3061451525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b280c2780 0x7f3b28151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.544+0000 7f3b2cc00640 1 --2- 172.21.15.67:0/3061451525 >> v2:172.21.15.134:3300/0 conn(0x7f3b280ac9a0 0x7f3b281510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.544+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3061451525 >> 172.21.15.67:0/3061451525 conn(0x7f3b280c7370 msgr2=0x7f3b280c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.544+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3061451525 shutdown_connections 2024-03-09T09:03:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.545+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3061451525 wait complete. 2024-03-09T09:03:56.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.546+0000 7f3b2cc00640 1 Processor -- start 2024-03-09T09:03:56.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.546+0000 7f3b2cc00640 1 -- start start 2024-03-09T09:03:56.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.548+0000 7f3b2cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28071190 0x7f3b28148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.548+0000 7f3b2cc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b280ac9a0 0x7f3b281e9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.548+0000 7f3b2cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b280c2780 0x7f3b281ea080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.548+0000 7f3b2cc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280acd80 con 0x7f3b28071190 2024-03-09T09:03:56.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.548+0000 7f3b2cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b28070c00 con 0x7f3b280c2780 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.548+0000 7f3b2cc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280acf00 con 0x7f3b280ac9a0 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.548+0000 7f3b25e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28071190 0x7f3b28148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.548+0000 7f3b25e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28071190 0x7f3b28148830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40562/0 (socket says 172.21.15.67:40562) 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.548+0000 7f3b25e00640 1 -- 172.21.15.67:0/3802357282 learned_addr learned my addr 172.21.15.67:0/3802357282 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.548+0000 7f3b25400640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b280ac9a0 0x7f3b281e9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.548+0000 7f3b26800640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b280c2780 0x7f3b281ea080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.549+0000 7f3b25e00640 1 -- 172.21.15.67:0/3802357282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b280ac9a0 msgr2=0x7f3b281e9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.549+0000 7f3b25e00640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b280ac9a0 0x7f3b281e9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.549+0000 7f3b25e00640 1 -- 172.21.15.67:0/3802357282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b280c2780 msgr2=0x7f3b281ea080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.549+0000 7f3b25e00640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b280c2780 0x7f3b281ea080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.549+0000 7f3b25e00640 1 -- 172.21.15.67:0/3802357282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2814bd30 con 0x7f3b28071190 2024-03-09T09:03:56.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.549+0000 7f3b25e00640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28071190 0x7f3b28148830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800e8c0 tx=0x7f3b1800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:56.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.549+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18017070 con 0x7f3b28071190 2024-03-09T09:03:56.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.549+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1800c940 con 0x7f3b28071190 2024-03-09T09:03:56.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.549+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800cd10 con 0x7f3b28071190 2024-03-09T09:03:56.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.549+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3802357282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b28153270 con 0x7f3b28071190 2024-03-09T09:03:56.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.550+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3802357282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b28153830 con 0x7f3b28071190 2024-03-09T09:03:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.551+0000 7f3b2cc00640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28146d00 0x7f3b2813c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.551+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3802357282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b281eda30 con 0x7f3b28146d00 2024-03-09T09:03:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.551+0000 7f3b25400640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28146d00 0x7f3b2813c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.552+0000 7f3b25400640 1 -- 172.21.15.67:0/3802357282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b280a9c40 con 0x7f3b28146d00 2024-03-09T09:03:56.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.552+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3b1801a070 con 0x7f3b28071190 2024-03-09T09:03:56.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.552+0000 7f3b25400640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28146d00 0x7f3b2813c250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b2814da50 tx=0x7f3b14009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:56.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.553+0000 7f3b1e000640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3b0806b520 0x7f3b0806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.553+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f3b1808a860 con 0x7f3b28071190 2024-03-09T09:03:56.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.553+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400ea00 con 0x7f3b28146d00 2024-03-09T09:03:56.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.553+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f3b1400eba0 con 0x7f3b28146d00 2024-03-09T09:03:56.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.553+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28146d00 msgr2=0x7f3b2813c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.553+0000 7f3b1e000640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28146d00 0x7f3b2813c250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b2814da50 tx=0x7f3b14009ca0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.553+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28146d00 msgr2=0x7f3b2813c250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:56.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.553+0000 7f3b1e000640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28146d00 0x7f3b2813c250 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b2814da50 tx=0x7f3b14009ca0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.553+0000 7f3b26800640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3b0806b520 0x7f3b0806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.554+0000 7f3b26800640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3b0806b520 0x7f3b0806d9e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3b20035000 tx=0x7f3b20043040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:56.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.568+0000 7f3b2cc00640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b281ed4f0 0x7f3b2813c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.568+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3802357282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2813ccd0 con 0x7f3b281ed4f0 2024-03-09T09:03:56.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.568+0000 7f3b25400640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b281ed4f0 0x7f3b2813c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.569+0000 7f3b25400640 1 -- 172.21.15.67:0/3802357282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3b280ac530 con 0x7f3b281ed4f0 2024-03-09T09:03:56.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.569+0000 7f3b25400640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b281ed4f0 0x7f3b2813c790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3b280678c0 tx=0x7f3b140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:56.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.569+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14004360 con 0x7f3b281ed4f0 2024-03-09T09:03:56.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4536 (secure 0 0 0) 0x7f3b14004360 con 0x7f3b281ed4f0 2024-03-09T09:03:56.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b281ed4f0 msgr2=0x7f3b2813c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b1e000640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b281ed4f0 0x7f3b2813c790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3b280678c0 tx=0x7f3b140046c0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b1e000640 1 -- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b281ed4f0 msgr2=0x7f3b2813c790 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:56.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b1e000640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b281ed4f0 0x7f3b2813c790 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3b280678c0 tx=0x7f3b140046c0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 86, 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:56.573 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:56.574 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:03:56.576 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:56.577 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:03:56.578 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.580 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:56.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:03:56.582 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.583 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:56.584 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:03:56.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.584 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:56.584 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:56.584 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:03:56.584 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.584 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.584 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:03:56.584 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:03:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3802357282 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3b0806b520 msgr2=0x7f3b0806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b2cc00640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3b0806b520 0x7f3b0806d9e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3b20035000 tx=0x7f3b20043040 comp rx=0 tx=0).stop 2024-03-09T09:03:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28071190 msgr2=0x7f3b28148830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b2cc00640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28071190 0x7f3b28148830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800e8c0 tx=0x7f3b1800ed80 comp rx=0 tx=0).stop 2024-03-09T09:03:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b26800640 1 -- 172.21.15.67:0/3802357282 reap_dead start 2024-03-09T09:03:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3802357282 shutdown_connections 2024-03-09T09:03:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b2cc00640 1 --2- 172.21.15.67:0/3802357282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3b28071190 0x7f3b28148830 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3802357282 >> 172.21.15.67:0/3802357282 conn(0x7f3b280c7370 msgr2=0x7f3b280c1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.570+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3802357282 shutdown_connections 2024-03-09T09:03:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.571+0000 7f3b2cc00640 1 -- 172.21.15.67:0/3802357282 wait complete. 2024-03-09T09:03:56.592 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:03:56.592 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:03:56.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:03:56.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.792+0000 7f2248800640 1 Processor -- start 2024-03-09T09:03:56.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.793+0000 7f2248800640 1 -- start start 2024-03-09T09:03:56.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.795+0000 7f2248800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f22400c4a20 0x7f2240153420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.795+0000 7f2248800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2240071880 0x7f224014fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.795+0000 7f2248800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22400c2f90 con 0x7f224006eac0 2024-03-09T09:03:56.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.795+0000 7f2248800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2240071cf0 con 0x7f22400c4a20 2024-03-09T09:03:56.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.795+0000 7f2248800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22400c4250 con 0x7f2240071880 2024-03-09T09:03:56.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.795+0000 7f2246800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2240071880 0x7f224014fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.795+0000 7f2245400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f22400c4a20 0x7f2240153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.795+0000 7f2246800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2240071880 0x7f224014fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:33566/0 (socket says 172.21.15.67:33566) 2024-03-09T09:03:56.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.795+0000 7f2246800640 1 -- 172.21.15.67:0/2956829856 learned_addr learned my addr 172.21.15.67:0/2956829856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:56.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.796+0000 7f2246800640 1 -- 172.21.15.67:0/2956829856 >> v2:172.21.15.134:3300/0 conn(0x7f22400c4a20 msgr2=0x7f2240153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:56.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.796+0000 7f2246800640 1 --2- 172.21.15.67:0/2956829856 >> v2:172.21.15.134:3300/0 conn(0x7f22400c4a20 0x7f2240153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.796+0000 7f2246800640 1 -- 172.21.15.67:0/2956829856 >> v1:172.21.15.134:6789/0 conn(0x7f224006eac0 legacy=0x7f22400c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.796+0000 7f2246800640 1 -- 172.21.15.67:0/2956829856 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22400c44a0 con 0x7f2240071880 2024-03-09T09:03:56.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.797+0000 7f2246800640 1 --2- 172.21.15.67:0/2956829856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2240071880 0x7f224014fcd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22400c43d0 tx=0x7f223c02be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42f4362c527e9a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:56.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.797+0000 7f2244a00640 1 -- 172.21.15.67:0/2956829856 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c02c9d0 con 0x7f2240071880 2024-03-09T09:03:56.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.797+0000 7f2244a00640 1 -- 172.21.15.67:0/2956829856 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223c02cb70 con 0x7f2240071880 2024-03-09T09:03:56.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.797+0000 7f2244a00640 1 -- 172.21.15.67:0/2956829856 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c0343f0 con 0x7f2240071880 2024-03-09T09:03:56.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.798+0000 7f2248800640 1 -- 172.21.15.67:0/2956829856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2240071880 msgr2=0x7f224014fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.798+0000 7f2248800640 1 --2- 172.21.15.67:0/2956829856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2240071880 0x7f224014fcd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22400c43d0 tx=0x7f223c02be30 comp rx=0 tx=0).stop 2024-03-09T09:03:56.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.798+0000 7f2248800640 1 -- 172.21.15.67:0/2956829856 shutdown_connections 2024-03-09T09:03:56.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.798+0000 7f2248800640 1 --2- 172.21.15.67:0/2956829856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2240071880 0x7f224014fcd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.798+0000 7f2248800640 1 --2- 172.21.15.67:0/2956829856 >> v2:172.21.15.134:3300/0 conn(0x7f22400c4a20 0x7f2240153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.798+0000 7f2248800640 1 -- 172.21.15.67:0/2956829856 >> 172.21.15.67:0/2956829856 conn(0x7f22400c8b20 msgr2=0x7f22400c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:56.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.798+0000 7f2248800640 1 -- 172.21.15.67:0/2956829856 shutdown_connections 2024-03-09T09:03:56.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.798+0000 7f2248800640 1 -- 172.21.15.67:0/2956829856 wait complete. 2024-03-09T09:03:56.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.799+0000 7f2248800640 1 Processor -- start 2024-03-09T09:03:56.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.800+0000 7f2248800640 1 -- start start 2024-03-09T09:03:56.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.800+0000 7f2248800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f224006eac0 0x7f224014ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.800+0000 7f2248800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2240071880 0x7f224014b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.800+0000 7f2248800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22400c4a20 0x7f22401484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.800+0000 7f2248800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22400c58b0 con 0x7f22400c4a20 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.800+0000 7f2248800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22400c3e60 con 0x7f2240071880 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.800+0000 7f2248800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22400c4780 con 0x7f224006eac0 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.801+0000 7f2246800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22400c4a20 0x7f22401484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.801+0000 7f2245e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f224006eac0 0x7f224014ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.801+0000 7f2246800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22400c4a20 0x7f22401484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40598/0 (socket says 172.21.15.67:40598) 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.801+0000 7f2246800640 1 -- 172.21.15.67:0/1995202466 learned_addr learned my addr 172.21.15.67:0/1995202466 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.801+0000 7f2245400640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2240071880 0x7f224014b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.801+0000 7f2246800640 1 -- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f224006eac0 msgr2=0x7f224014ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.801+0000 7f2246800640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f224006eac0 0x7f224014ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.801+0000 7f2246800640 1 -- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2240071880 msgr2=0x7f224014b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.801+0000 7f2246800640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2240071880 0x7f224014b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:56.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.801+0000 7f2246800640 1 -- 172.21.15.67:0/1995202466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22401524a0 con 0x7f22400c4a20 2024-03-09T09:03:56.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.801+0000 7f2246800640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22400c4a20 0x7f22401484f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f223c02c440 tx=0x7f223c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:56.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.802+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c043020 con 0x7f22400c4a20 2024-03-09T09:03:56.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.802+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223c03e450 con 0x7f22400c4a20 2024-03-09T09:03:56.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.802+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c009510 con 0x7f22400c4a20 2024-03-09T09:03:56.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.802+0000 7f2248800640 1 -- 172.21.15.67:0/1995202466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2240151b10 con 0x7f22400c4a20 2024-03-09T09:03:56.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.802+0000 7f2248800640 1 -- 172.21.15.67:0/1995202466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22400c66f0 con 0x7f22400c4a20 2024-03-09T09:03:56.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.803+0000 7f2248800640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2240150810 0x7f2240148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.803+0000 7f2248800640 1 -- 172.21.15.67:0/1995202466 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22400c6570 con 0x7f2240150810 2024-03-09T09:03:56.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.803+0000 7f2245e00640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2240150810 0x7f2240148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.804+0000 7f2245e00640 1 -- 172.21.15.67:0/1995202466 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2240152030 con 0x7f2240150810 2024-03-09T09:03:56.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.804+0000 7f2245e00640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2240150810 0x7f2240148a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f223400f540 tx=0x7f223400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:56.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.804+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340052e0 con 0x7f2240150810 2024-03-09T09:03:56.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.804+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f223c046020 con 0x7f22400c4a20 2024-03-09T09:03:56.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.805+0000 7f223a000640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f222406b3d0 0x7f222406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.805+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f223c0acc30 con 0x7f22400c4a20 2024-03-09T09:03:56.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.805+0000 7f2245400640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f222406b3d0 0x7f222406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.805+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f2234018070 con 0x7f2240150810 2024-03-09T09:03:56.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.805+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2240150810 msgr2=0x7f2240148a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.805+0000 7f223a000640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2240150810 0x7f2240148a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f223400f540 tx=0x7f223400fa00 comp rx=0 tx=0).stop 2024-03-09T09:03:56.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.805+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2240150810 msgr2=0x7f2240148a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:56.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.805+0000 7f223a000640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2240150810 0x7f2240148a30 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f223400f540 tx=0x7f223400fa00 comp rx=0 tx=0).stop 2024-03-09T09:03:56.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.806+0000 7f2245400640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f222406b3d0 0x7f222406d890 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f224014f670 tx=0x7f2230009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:56.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.812+0000 7f2248800640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22400c6a60 0x7f22400c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:56.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.812+0000 7f2248800640 1 -- 172.21.15.67:0/1995202466 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2240075a00 con 0x7f22400c6a60 2024-03-09T09:03:56.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.813+0000 7f2245e00640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22400c6a60 0x7f22400c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:56.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.813+0000 7f2245e00640 1 -- 172.21.15.67:0/1995202466 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f224006b7a0 con 0x7f22400c6a60 2024-03-09T09:03:56.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.814+0000 7f2245e00640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22400c6a60 0x7f22400c6ed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f223400f540 tx=0x7f2234001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:56.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.814+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223401de00 con 0x7f22400c6a60 2024-03-09T09:03:56.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.814+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5017 (secure 0 0 0) 0x7f2234003dd0 con 0x7f22400c6a60 2024-03-09T09:03:56.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.814+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22400c6a60 msgr2=0x7f22400c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.814+0000 7f223a000640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22400c6a60 0x7f22400c6ed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f223400f540 tx=0x7f2234001080 comp rx=0 tx=0).stop 2024-03-09T09:03:56.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.814+0000 7f223a000640 1 -- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22400c6a60 msgr2=0x7f22400c6ed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:56.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.814+0000 7f223a000640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22400c6a60 0x7f22400c6ed0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f223400f540 tx=0x7f2234001080 comp rx=0 tx=0).stop 2024-03-09T09:03:56.817 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:03:56.817 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:03:56.817 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:03:56.817 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:03:56.817 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 86, 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:56.818 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:56.819 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:03:56.820 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:56.821 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.822 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:03:56.823 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:56.824 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:03:56.825 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:03:56.826 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.827 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:56.828 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:03:56.829 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.830 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:56.830 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:03:56.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:56.830 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:56.830 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:56.830 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:56.830 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:56.830 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:56.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:56.830 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:03:56.830 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:03:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.815+0000 7f2248800640 1 -- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f222406b3d0 msgr2=0x7f222406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.815+0000 7f2248800640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f222406b3d0 0x7f222406d890 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f224014f670 tx=0x7f2230009190 comp rx=0 tx=0).stop 2024-03-09T09:03:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.815+0000 7f2248800640 1 -- 172.21.15.67:0/1995202466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22400c4a20 msgr2=0x7f22401484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.815+0000 7f2248800640 1 --2- 172.21.15.67:0/1995202466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22400c4a20 0x7f22401484f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f223c02c440 tx=0x7f223c004a40 comp rx=0 tx=0).stop 2024-03-09T09:03:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.815+0000 7f2246800640 1 -- 172.21.15.67:0/1995202466 reap_dead start 2024-03-09T09:03:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.815+0000 7f2246800640 1 -- 172.21.15.67:0/1995202466 reap_dead start 2024-03-09T09:03:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.815+0000 7f2248800640 1 -- 172.21.15.67:0/1995202466 shutdown_connections 2024-03-09T09:03:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.815+0000 7f2248800640 1 -- 172.21.15.67:0/1995202466 >> 172.21.15.67:0/1995202466 conn(0x7f22400c8b20 msgr2=0x7f22400721f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.815+0000 7f2248800640 1 -- 172.21.15.67:0/1995202466 shutdown_connections 2024-03-09T09:03:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:56.815+0000 7f2248800640 1 -- 172.21.15.67:0/1995202466 wait complete. 2024-03-09T09:03:56.834 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:03:56.834 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:03:56.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:03:57.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.027+0000 7f911f000640 1 Processor -- start 2024-03-09T09:03:57.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.027+0000 7f911f000640 1 -- start start 2024-03-09T09:03:57.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.028+0000 7f911f000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f91180ac930 0x7f91180c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.028+0000 7f911f000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f91180c30a0 0x7f9118150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.028+0000 7f911f000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118151190 0x7f91181539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.028+0000 7f911f000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9118070670 con 0x7f91180c30a0 2024-03-09T09:03:57.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.028+0000 7f911f000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91180c2780 con 0x7f91180ac930 2024-03-09T09:03:57.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.028+0000 7f911f000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91180708c0 con 0x7f9118151190 2024-03-09T09:03:57.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.029+0000 7f911d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f91180c30a0 0x7f9118150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.029+0000 7f911d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f91180c30a0 0x7f9118150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40610/0 (socket says 172.21.15.67:40610) 2024-03-09T09:03:57.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.029+0000 7f911d200640 1 -- 172.21.15.67:0/3498297052 learned_addr learned my addr 172.21.15.67:0/3498297052 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:57.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.029+0000 7f911dc00640 1 --2- 172.21.15.67:0/3498297052 >> v2:172.21.15.134:3300/0 conn(0x7f91180ac930 0x7f91180c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.029+0000 7f911e600640 1 --2- 172.21.15.67:0/3498297052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118151190 0x7f91181539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.029+0000 7f911d200640 1 -- 172.21.15.67:0/3498297052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118151190 msgr2=0x7f91181539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:57.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.029+0000 7f911d200640 1 --2- 172.21.15.67:0/3498297052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118151190 0x7f91181539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.029+0000 7f911d200640 1 -- 172.21.15.67:0/3498297052 >> v2:172.21.15.134:3300/0 conn(0x7f91180ac930 msgr2=0x7f91180c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:57.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.029+0000 7f911d200640 1 --2- 172.21.15.67:0/3498297052 >> v2:172.21.15.134:3300/0 conn(0x7f91180ac930 0x7f91180c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.029+0000 7f911d200640 1 -- 172.21.15.67:0/3498297052 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118070ba0 con 0x7f91180c30a0 2024-03-09T09:03:57.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.030+0000 7f911d200640 1 --2- 172.21.15.67:0/3498297052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91180c30a0 0x7f9118150c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f911806edb0 tx=0x7f910402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871039e15851341f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:57.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.031+0000 7f9117e00640 1 -- 172.21.15.67:0/3498297052 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104035020 con 0x7f91180c30a0 2024-03-09T09:03:57.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.031+0000 7f9117e00640 1 -- 172.21.15.67:0/3498297052 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910402d040 con 0x7f91180c30a0 2024-03-09T09:03:57.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.031+0000 7f9117e00640 1 -- 172.21.15.67:0/3498297052 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104031a80 con 0x7f91180c30a0 2024-03-09T09:03:57.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.032+0000 7f911f000640 1 -- 172.21.15.67:0/3498297052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91180c30a0 msgr2=0x7f9118150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.032+0000 7f911f000640 1 --2- 172.21.15.67:0/3498297052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91180c30a0 0x7f9118150c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f911806edb0 tx=0x7f910402b900 comp rx=0 tx=0).stop 2024-03-09T09:03:57.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.032+0000 7f911f000640 1 -- 172.21.15.67:0/3498297052 shutdown_connections 2024-03-09T09:03:57.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.032+0000 7f911f000640 1 --2- 172.21.15.67:0/3498297052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118151190 0x7f91181539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.032+0000 7f911f000640 1 --2- 172.21.15.67:0/3498297052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91180c30a0 0x7f9118150c50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.032+0000 7f911f000640 1 --2- 172.21.15.67:0/3498297052 >> v2:172.21.15.134:3300/0 conn(0x7f91180ac930 0x7f91180c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.032+0000 7f911f000640 1 -- 172.21.15.67:0/3498297052 >> 172.21.15.67:0/3498297052 conn(0x7f91180c7220 msgr2=0x7f91180c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:57.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.032+0000 7f911f000640 1 -- 172.21.15.67:0/3498297052 shutdown_connections 2024-03-09T09:03:57.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.032+0000 7f911f000640 1 -- 172.21.15.67:0/3498297052 wait complete. 2024-03-09T09:03:57.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.034+0000 7f911f000640 1 Processor -- start 2024-03-09T09:03:57.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.034+0000 7f911f000640 1 -- start start 2024-03-09T09:03:57.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.035+0000 7f911f000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91180ac930 0x7f91181390a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.035+0000 7f911f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91180c30a0 0x7f91181ed8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.035+0000 7f911f000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118151190 0x7f91181ede10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.035+0000 7f911f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91180acd10 con 0x7f91180c30a0 2024-03-09T09:03:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.035+0000 7f911f000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91181515a0 con 0x7f91180ac930 2024-03-09T09:03:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.035+0000 7f911f000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91180ace90 con 0x7f9118151190 2024-03-09T09:03:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.035+0000 7f911d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91180c30a0 0x7f91181ed8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.035+0000 7f911d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91180c30a0 0x7f91181ed8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40612/0 (socket says 172.21.15.67:40612) 2024-03-09T09:03:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.035+0000 7f911d200640 1 -- 172.21.15.67:0/1763422644 learned_addr learned my addr 172.21.15.67:0/1763422644 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.035+0000 7f911e600640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118151190 0x7f91181ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.035+0000 7f911dc00640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91180ac930 0x7f91181390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.036+0000 7f911d200640 1 -- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118151190 msgr2=0x7f91181ede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.036+0000 7f911d200640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118151190 0x7f91181ede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.036+0000 7f911d200640 1 -- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91180ac930 msgr2=0x7f91181390a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.036+0000 7f911d200640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91180ac930 0x7f91181390a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.036+0000 7f911d200640 1 -- 172.21.15.67:0/1763422644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91180ab4b0 con 0x7f91180c30a0 2024-03-09T09:03:57.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.036+0000 7f911d200640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91180c30a0 0x7f91181ed8d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f910402b940 tx=0x7f91040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:57.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.037+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104040070 con 0x7f91180c30a0 2024-03-09T09:03:57.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.037+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9104039c10 con 0x7f91180c30a0 2024-03-09T09:03:57.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.037+0000 7f911f000640 1 -- 172.21.15.67:0/1763422644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91180ab790 con 0x7f91180c30a0 2024-03-09T09:03:57.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.037+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104035040 con 0x7f91180c30a0 2024-03-09T09:03:57.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.037+0000 7f911f000640 1 -- 172.21.15.67:0/1763422644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91180abd50 con 0x7f91180c30a0 2024-03-09T09:03:57.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.038+0000 7f911f000640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118134060 0x7f91181ee600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.038+0000 7f911f000640 1 -- 172.21.15.67:0/1763422644 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911804f3c0 con 0x7f9118134060 2024-03-09T09:03:57.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.039+0000 7f911dc00640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118134060 0x7f91181ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.039+0000 7f911dc00640 1 -- 172.21.15.67:0/1763422644 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91181ee350 con 0x7f9118134060 2024-03-09T09:03:57.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.040+0000 7f911dc00640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118134060 0x7f91181ee600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f910c00b4d0 tx=0x7f910c00b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:57.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.040+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f910404b060 con 0x7f91180c30a0 2024-03-09T09:03:57.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.041+0000 7f9115600640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90f806b4d0 0x7f90f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.041+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f91040ac070 con 0x7f91180c30a0 2024-03-09T09:03:57.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.042+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c018070 con 0x7f9118134060 2024-03-09T09:03:57.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.042+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f910c0106e0 con 0x7f9118134060 2024-03-09T09:03:57.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.042+0000 7f911e600640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90f806b4d0 0x7f90f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.042+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118134060 msgr2=0x7f91181ee600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.042+0000 7f9115600640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118134060 0x7f91181ee600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f910c00b4d0 tx=0x7f910c00b990 comp rx=0 tx=0).stop 2024-03-09T09:03:57.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.042+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118134060 msgr2=0x7f91181ee600 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:57.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.042+0000 7f9115600640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9118134060 0x7f91181ee600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.043+0000 7f911e600640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90f806b4d0 0x7f90f806d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9118145090 tx=0x7f9110009820 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:57.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.049+0000 7f911f000640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91181eeb40 0x7f91181eef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.049+0000 7f911f000640 1 -- 172.21.15.67:0/1763422644 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91181ef490 con 0x7f91181eeb40 2024-03-09T09:03:57.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.050+0000 7f911dc00640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91181eeb40 0x7f91181eef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.050+0000 7f911dc00640 1 -- 172.21.15.67:0/1763422644 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9118067b30 con 0x7f91181eeb40 2024-03-09T09:03:57.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.051+0000 7f911dc00640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91181eeb40 0x7f91181eef50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f910c00bf30 tx=0x7f910c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:57.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.051+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c014430 con 0x7f91181eeb40 2024-03-09T09:03:57.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.052+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7f910c014430 con 0x7f91181eeb40 2024-03-09T09:03:57.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.052+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91181eeb40 msgr2=0x7f91181eef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.052+0000 7f9115600640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91181eeb40 0x7f91181eef50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f910c00bf30 tx=0x7f910c004730 comp rx=0 tx=0).stop 2024-03-09T09:03:57.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.052+0000 7f9115600640 1 -- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91181eeb40 msgr2=0x7f91181eef50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:03:57.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.052+0000 7f9115600640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91181eeb40 0x7f91181eef50 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f910c00bf30 tx=0x7f910c004730 comp rx=0 tx=0).stop 2024-03-09T09:03:57.055 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:03:57.055 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:03:57.055 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:03:57.055 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:03:57.055 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 86, 2024-03-09T09:03:57.055 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:57.056 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:57.057 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:57.057 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:57.057 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:57.057 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:57.057 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:57.057 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:57.057 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:57.057 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:57.057 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:03:57.057 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:03:57.057 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:57.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.059 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:03:57.059 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:03:57.059 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:03:57.059 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:03:57.059 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:03:57.059 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:03:57.059 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:57.059 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:03:57.059 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:03:57.059 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:03:57.060 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:03:57.061 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:03:57.061 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:03:57.061 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:03:57.061 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:03:57.061 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:57.061 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:03:57.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.061 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:03:57.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.061 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:03:57.062 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:57.063 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:57.063 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:57.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.063 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:03:57.063 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:03:57.063 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:57.063 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:57.063 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:57.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.063 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:57.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.065 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:03:57.066 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:57.067 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:03:57.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.068 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:57.068 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:57.068 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:03:57.068 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:57.068 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:57.068 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:03:57.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.068 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:57.068 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:57.069 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:03:57.069 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:57.069 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:57.069 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:03:57.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.069 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:03:57.069 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:57.069 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:57.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.070 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:57.070 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:57.070 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:03:57.070 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:57.070 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:03:57.070 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:03:57.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:03:57.070 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:03:57.070 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:03:57.070 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:03:57.071 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:03:57.071 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:03:57.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:03:57.071 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:03:57.071 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:03:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.053+0000 7f911f000640 1 -- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90f806b4d0 msgr2=0x7f90f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.053+0000 7f911f000640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90f806b4d0 0x7f90f806d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9118145090 tx=0x7f9110009820 comp rx=0 tx=0).stop 2024-03-09T09:03:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.053+0000 7f911f000640 1 -- 172.21.15.67:0/1763422644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91180c30a0 msgr2=0x7f91181ed8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.053+0000 7f911f000640 1 --2- 172.21.15.67:0/1763422644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91180c30a0 0x7f91181ed8d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f910402b940 tx=0x7f91040046c0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.053+0000 7f911e600640 1 -- 172.21.15.67:0/1763422644 reap_dead start 2024-03-09T09:03:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.053+0000 7f911e600640 1 -- 172.21.15.67:0/1763422644 reap_dead start 2024-03-09T09:03:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.053+0000 7f911f000640 1 -- 172.21.15.67:0/1763422644 shutdown_connections 2024-03-09T09:03:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.053+0000 7f911f000640 1 -- 172.21.15.67:0/1763422644 >> 172.21.15.67:0/1763422644 conn(0x7f91180c7220 msgr2=0x7f9118072940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.053+0000 7f911f000640 1 -- 172.21.15.67:0/1763422644 shutdown_connections 2024-03-09T09:03:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.053+0000 7f911f000640 1 -- 172.21.15.67:0/1763422644 wait complete. 2024-03-09T09:03:57.074 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:03:57.074 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T09:03:57.074 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:03:57.074 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T09:03:57.074 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:03:57.074 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:03:57.074 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:03:57.104 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:03:57.104 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:03:57.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:03:57.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.326+0000 7fbbb5400640 1 Processor -- start 2024-03-09T09:03:57.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.326+0000 7fbbb5400640 1 -- start start 2024-03-09T09:03:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.328+0000 7fbbb5400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb0153300 0x7fbbb00b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.328+0000 7fbbb5400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb0157000 con 0x7fbbb00c5e40 2024-03-09T09:03:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.328+0000 7fbbb5400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb00ae430 con 0x7fbbb00cadb0 2024-03-09T09:03:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.328+0000 7fbbb5400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00ae6f0 con 0x7fbbb0153300 2024-03-09T09:03:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.328+0000 7fbbafe00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb0153300 msgr2=0x7fbbb00b0000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:03:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.329+0000 7fbbafe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb0153300 0x7fbbb00b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.329+0000 7fbbaf400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbbb00c5e40 0x7fbbb0156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36660/0 (socket says 172.21.15.67:36660) 2024-03-09T09:03:57.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.329+0000 7fbbaf400640 1 -- 172.21.15.67:0/3303392228 learned_addr learned my addr 172.21.15.67:0/3303392228 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:03:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.329+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101937084 0 0) 0x7fbbb0157000 con 0x7fbbb00c5e40 2024-03-09T09:03:57.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.329+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb94003520 con 0x7fbbb00c5e40 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.329+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966332404 0 0) 0x7fbbb00ae430 con 0x7fbbb00cadb0 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.329+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbb0157000 con 0x7fbbb00cadb0 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707348120 0 0) 0x7fbb94003520 con 0x7fbbb00c5e40 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb0153300 msgr2=0x7fbbb00b0000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 --2- 172.21.15.67:0/3303392228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb0153300 0x7fbbb00b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 >> v1:172.21.15.134:6789/0 conn(0x7fbbb00cadb0 legacy=0x7fbbb0160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb0155b10 con 0x7fbbb00c5e40 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fbb9c001670 con 0x7fbbb00c5e40 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 >> v1:172.21.15.67:6789/0 conn(0x7fbbb00c5e40 legacy=0x7fbbb0156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 --2- 172.21.15.67:0/3303392228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 0x7fbb94003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 --2- 172.21.15.67:0/3303392228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb94004800 0x7fbb94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 --2- 172.21.15.67:0/3303392228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb0153300 0x7fbb9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00ae6f0 con 0x7fbb94003be0 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94005560 con 0x7fbb94004800 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9400bf00 con 0x7fbbb0153300 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbb4a00640 1 -- 172.21.15.67:0/3303392228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb0153300 msgr2=0x7fbb9400b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbb4a00640 1 --2- 172.21.15.67:0/3303392228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb0153300 0x7fbb9400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbafe00640 1 --2- 172.21.15.67:0/3303392228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 0x7fbb94003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.330+0000 7fbbaf400640 1 --2- 172.21.15.67:0/3303392228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb94004800 0x7fbb94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.331+0000 7fbbafe00640 1 -- 172.21.15.67:0/3303392228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb0153300 msgr2=0x7fbb9400b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.331+0000 7fbbafe00640 1 --2- 172.21.15.67:0/3303392228 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb0153300 0x7fbb9400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.331+0000 7fbbafe00640 1 -- 172.21.15.67:0/3303392228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb94004800 msgr2=0x7fbb94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.331+0000 7fbbafe00640 1 --2- 172.21.15.67:0/3303392228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb94004800 0x7fbb94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.331+0000 7fbbafe00640 1 -- 172.21.15.67:0/3303392228 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94003520 con 0x7fbb94003be0 2024-03-09T09:03:57.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.332+0000 7fbbafe00640 1 --2- 172.21.15.67:0/3303392228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 0x7fbb94003fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009900 tx=0x7fbb94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179a6813d5e4aba5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:57.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.332+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0009500 con 0x7fbb94003be0 2024-03-09T09:03:57.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.332+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba002f070 con 0x7fbb94003be0 2024-03-09T09:03:57.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.332+0000 7fbbaea00640 1 -- 172.21.15.67:0/3303392228 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0003df0 con 0x7fbb94003be0 2024-03-09T09:03:57.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.333+0000 7fbbb5400640 1 -- 172.21.15.67:0/3303392228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 msgr2=0x7fbb94003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.333+0000 7fbbb5400640 1 --2- 172.21.15.67:0/3303392228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 0x7fbb94003fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009900 tx=0x7fbb94005830 comp rx=0 tx=0).stop 2024-03-09T09:03:57.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.333+0000 7fbbb4a00640 1 -- 172.21.15.67:0/3303392228 reap_dead start 2024-03-09T09:03:57.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.333+0000 7fbbb5400640 1 -- 172.21.15.67:0/3303392228 shutdown_connections 2024-03-09T09:03:57.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.333+0000 7fbbb5400640 1 -- 172.21.15.67:0/3303392228 >> 172.21.15.67:0/3303392228 conn(0x7fbbb00732d0 msgr2=0x7fbbb00ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:57.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.333+0000 7fbbb5400640 1 -- 172.21.15.67:0/3303392228 shutdown_connections 2024-03-09T09:03:57.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.333+0000 7fbbb5400640 1 -- 172.21.15.67:0/3303392228 wait complete. 2024-03-09T09:03:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.335+0000 7fbbb5400640 1 Processor -- start 2024-03-09T09:03:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.335+0000 7fbbb5400640 1 -- start start 2024-03-09T09:03:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbb5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 0x7fbbb014f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbb5400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb00ab0c0 0x7fbbb014d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbb5400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbb014d880 0x7fbbb0148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbb5400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00c59e0 con 0x7fbb94003be0 2024-03-09T09:03:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbb5400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb0159e90 con 0x7fbbb014d880 2024-03-09T09:03:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbb5400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb0156720 con 0x7fbbb00ab0c0 2024-03-09T09:03:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbaf400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb00ab0c0 msgr2=0x7fbbb014d340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:03:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbaf400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb00ab0c0 0x7fbbb014d340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:03:57.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbafe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 0x7fbbb014f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbafe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 0x7fbbb014f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40664/0 (socket says 172.21.15.67:40664) 2024-03-09T09:03:57.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbafe00640 1 -- 172.21.15.67:0/420980150 learned_addr learned my addr 172.21.15.67:0/420980150 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:03:57.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.336+0000 7fbbb4a00640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbb014d880 0x7fbbb0148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.337+0000 7fbbafe00640 1 -- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb00ab0c0 msgr2=0x7fbbb014d340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:03:57.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.337+0000 7fbbafe00640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb00ab0c0 0x7fbbb014d340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.337+0000 7fbbafe00640 1 -- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbb014d880 msgr2=0x7fbbb0148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.337+0000 7fbbafe00640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbb014d880 0x7fbbb0148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.337+0000 7fbbafe00640 1 -- 172.21.15.67:0/420980150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb01eec50 con 0x7fbb94003be0 2024-03-09T09:03:57.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.337+0000 7fbbb4a00640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbb014d880 0x7fbbb0148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:03:57.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.337+0000 7fbbafe00640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 0x7fbbb014f530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbba0000c00 tx=0x7fbba0005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:57.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.337+0000 7fbba7e00640 1 -- 172.21.15.67:0/420980150 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0006dd0 con 0x7fbb94003be0 2024-03-09T09:03:57.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.337+0000 7fbba7e00640 1 -- 172.21.15.67:0/420980150 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba0003e50 con 0x7fbb94003be0 2024-03-09T09:03:57.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.337+0000 7fbbb5400640 1 -- 172.21.15.67:0/420980150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb006d270 con 0x7fbb94003be0 2024-03-09T09:03:57.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.337+0000 7fbba7e00640 1 -- 172.21.15.67:0/420980150 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba00348b0 con 0x7fbb94003be0 2024-03-09T09:03:57.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.338+0000 7fbbb5400640 1 -- 172.21.15.67:0/420980150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb01eeef0 con 0x7fbb94003be0 2024-03-09T09:03:57.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.339+0000 7fbbb5400640 1 -- 172.21.15.67:0/420980150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb014dc60 con 0x7fbb94003be0 2024-03-09T09:03:57.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.340+0000 7fbba7e00640 1 -- 172.21.15.67:0/420980150 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbba0034a50 con 0x7fbb94003be0 2024-03-09T09:03:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.341+0000 7fbba7e00640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb9006b480 0x7fbb9006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:03:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.342+0000 7fbba7e00640 1 -- 172.21.15.67:0/420980150 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbba00abd90 con 0x7fbb94003be0 2024-03-09T09:03:57.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.342+0000 7fbbaf400640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb9006b480 0x7fbb9006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:03:57.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.343+0000 7fbbaf400640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb9006b480 0x7fbb9006d940 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c001400 tx=0x7fbb9c001390 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:03:57.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.346+0000 7fbba7e00640 1 -- 172.21.15.67:0/420980150 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbbb014dc60 con 0x7fbb94003be0 2024-03-09T09:03:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.673+0000 7fbbb5400640 1 -- 172.21.15.67:0/420980150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbbb004f3c0 con 0x7fbb94003be0 2024-03-09T09:03:57.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.674+0000 7fbba7e00640 1 -- 172.21.15.67:0/420980150 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fbba004d060 con 0x7fbb94003be0 2024-03-09T09:03:57.677 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":86,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:03:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 -- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb9006b480 msgr2=0x7fbb9006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb9006b480 0x7fbb9006d940 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c001400 tx=0x7fbb9c001390 comp rx=0 tx=0).stop 2024-03-09T09:03:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 -- 172.21.15.67:0/420980150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 msgr2=0x7fbbb014f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:03:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 0x7fbbb014f530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbba0000c00 tx=0x7fbba0005a10 comp rx=0 tx=0).stop 2024-03-09T09:03:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 -- 172.21.15.67:0/420980150 shutdown_connections 2024-03-09T09:03:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb9006b480 0x7fbb9006d940 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbb014d880 0x7fbbb0148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbbb00ab0c0 0x7fbbb014d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 --2- 172.21.15.67:0/420980150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb94003be0 0x7fbbb014f530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:03:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 -- 172.21.15.67:0/420980150 >> 172.21.15.67:0/420980150 conn(0x7fbbb00732d0 msgr2=0x7fbbb0158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:03:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 -- 172.21.15.67:0/420980150 shutdown_connections 2024-03-09T09:03:57.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:03:57.677+0000 7fbbb5400640 1 -- 172.21.15.67:0/420980150 wait complete. 2024-03-09T09:04:00.700 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:04:00.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.938+0000 7fbedca00640 1 Processor -- start 2024-03-09T09:04:00.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.938+0000 7fbedca00640 1 -- start start 2024-03-09T09:04:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.939+0000 7fbedca00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbed8071190 0x7fbed8157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.939+0000 7fbedca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed8153d40 con 0x7fbed80c22a0 2024-03-09T09:04:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.939+0000 7fbedca00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed80c2790 con 0x7fbed80ac730 2024-03-09T09:04:00.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.939+0000 7fbedca00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed8070ee0 con 0x7fbed8071190 2024-03-09T09:04:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.940+0000 7fbed5400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbed80c22a0 0x7fbed815c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52344/0 (socket says 172.21.15.67:52344) 2024-03-09T09:04:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.940+0000 7fbed5400640 1 -- 172.21.15.67:0/903202939 learned_addr learned my addr 172.21.15.67:0/903202939 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:04:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.940+0000 7fbed6800640 1 --2- 172.21.15.67:0/903202939 >> v2:172.21.15.134:3300/0 conn(0x7fbed8071190 0x7fbed8157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.940+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065319195 0 0) 0x7fbed8153d40 con 0x7fbed80c22a0 2024-03-09T09:04:00.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbebc003520 con 0x7fbed80c22a0 2024-03-09T09:04:00.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 873560951 0 0) 0x7fbebc003520 con 0x7fbed80c22a0 2024-03-09T09:04:00.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 >> v2:172.21.15.134:3300/0 conn(0x7fbed8071190 msgr2=0x7fbed8157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:00.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 --2- 172.21.15.67:0/903202939 >> v2:172.21.15.134:3300/0 conn(0x7fbed8071190 0x7fbed8157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 >> v1:172.21.15.134:6789/0 conn(0x7fbed80ac730 legacy=0x7fbed8157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8070920 con 0x7fbed80c22a0 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fbec4001670 con 0x7fbed80c22a0 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 >> v1:172.21.15.67:6789/0 conn(0x7fbed80c22a0 legacy=0x7fbed815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbebc0039b0 0x7fbebc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc004800 0x7fbebc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 0x7fbebc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8070ee0 con 0x7fbebc0051c0 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc00c160 con 0x7fbebc004800 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc00c360 con 0x7fbebc0039b0 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed5e00640 1 -- 172.21.15.67:0/903202939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbebc0039b0 msgr2=0x7fbebc003dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.941+0000 7fbed5e00640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbebc0039b0 0x7fbebc003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.942+0000 7fbed6800640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 0x7fbebc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:00.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.942+0000 7fbed5400640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc004800 0x7fbebc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:00.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.942+0000 7fbed6800640 1 -- 172.21.15.67:0/903202939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbebc0039b0 msgr2=0x7fbebc003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:04:00.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.942+0000 7fbed6800640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbebc0039b0 0x7fbebc003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:00.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.942+0000 7fbed6800640 1 -- 172.21.15.67:0/903202939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc004800 msgr2=0x7fbebc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:00.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.942+0000 7fbed6800640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc004800 0x7fbebc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:00.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.942+0000 7fbed6800640 1 -- 172.21.15.67:0/903202939 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc003520 con 0x7fbebc0051c0 2024-03-09T09:04:00.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.942+0000 7fbed5400640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbebc004800 0x7fbebc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:04:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.943+0000 7fbed6800640 1 -- 172.21.15.67:0/903202939 reap_dead start 2024-03-09T09:04:00.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.944+0000 7fbed6800640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 0x7fbebc00b9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009fc0 tx=0x7fbebc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd48aa5b2631846c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:00.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.944+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0009c10 con 0x7fbebc0051c0 2024-03-09T09:04:00.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.944+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed002cca0 con 0x7fbebc0051c0 2024-03-09T09:04:00.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.944+0000 7fbed4a00640 1 -- 172.21.15.67:0/903202939 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0003bd0 con 0x7fbebc0051c0 2024-03-09T09:04:00.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.945+0000 7fbedca00640 1 -- 172.21.15.67:0/903202939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 msgr2=0x7fbebc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:00.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.945+0000 7fbedca00640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 0x7fbebc00b9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009fc0 tx=0x7fbebc005830 comp rx=0 tx=0).stop 2024-03-09T09:04:00.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.945+0000 7fbedca00640 1 -- 172.21.15.67:0/903202939 shutdown_connections 2024-03-09T09:04:00.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.945+0000 7fbedca00640 1 --2- 172.21.15.67:0/903202939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 0x7fbebc00b9c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:00.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.945+0000 7fbedca00640 1 -- 172.21.15.67:0/903202939 >> 172.21.15.67:0/903202939 conn(0x7fbed80c7370 msgr2=0x7fbed80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:00.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.945+0000 7fbedca00640 1 -- 172.21.15.67:0/903202939 shutdown_connections 2024-03-09T09:04:00.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.945+0000 7fbedca00640 1 -- 172.21.15.67:0/903202939 wait complete. 2024-03-09T09:04:00.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.947+0000 7fbedca00640 1 Processor -- start 2024-03-09T09:04:00.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.947+0000 7fbedca00640 1 -- start start 2024-03-09T09:04:00.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.948+0000 7fbedca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 0x7fbed81560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:00.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.948+0000 7fbedca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbed81ec440 0x7fbed81dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:00.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.948+0000 7fbedca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbed8156620 0x7fbed80abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.948+0000 7fbedca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8151610 con 0x7fbebc0051c0 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.948+0000 7fbedca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8154d00 con 0x7fbed81ec440 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.948+0000 7fbedca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8151790 con 0x7fbed8156620 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.948+0000 7fbed6800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbed8156620 msgr2=0x7fbed80abb40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.948+0000 7fbed5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 0x7fbed81560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.948+0000 7fbed6800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbed8156620 0x7fbed80abb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.948+0000 7fbed5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 0x7fbed81560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57524/0 (socket says 172.21.15.67:57524) 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.948+0000 7fbed5e00640 1 -- 172.21.15.67:0/2522311298 learned_addr learned my addr 172.21.15.67:0/2522311298 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.949+0000 7fbed5400640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbed81ec440 0x7fbed81dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.949+0000 7fbed5e00640 1 -- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbed8156620 msgr2=0x7fbed80abb40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.949+0000 7fbed5e00640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbed8156620 0x7fbed80abb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.949+0000 7fbed5e00640 1 -- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbed81ec440 msgr2=0x7fbed81dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.949+0000 7fbed5e00640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbed81ec440 0x7fbed81dd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:00.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.949+0000 7fbed5e00640 1 -- 172.21.15.67:0/2522311298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed81df450 con 0x7fbebc0051c0 2024-03-09T09:04:00.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.949+0000 7fbed5400640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbed81ec440 0x7fbed81dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:04:00.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.949+0000 7fbed5e00640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 0x7fbed81560e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbec800c4a0 tx=0x7fbec800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:00.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.950+0000 7fbece000640 1 -- 172.21.15.67:0/2522311298 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8017070 con 0x7fbebc0051c0 2024-03-09T09:04:00.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.950+0000 7fbece000640 1 -- 172.21.15.67:0/2522311298 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec8007480 con 0x7fbebc0051c0 2024-03-09T09:04:00.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.950+0000 7fbedca00640 1 -- 172.21.15.67:0/2522311298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed81eb960 con 0x7fbebc0051c0 2024-03-09T09:04:00.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.950+0000 7fbece000640 1 -- 172.21.15.67:0/2522311298 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8005a00 con 0x7fbebc0051c0 2024-03-09T09:04:00.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.950+0000 7fbedca00640 1 -- 172.21.15.67:0/2522311298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed81ebf20 con 0x7fbebc0051c0 2024-03-09T09:04:00.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.951+0000 7fbedca00640 1 -- 172.21.15.67:0/2522311298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed8075a00 con 0x7fbebc0051c0 2024-03-09T09:04:00.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.953+0000 7fbece000640 1 -- 172.21.15.67:0/2522311298 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbec801a070 con 0x7fbebc0051c0 2024-03-09T09:04:00.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.954+0000 7fbece000640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbeb806b480 0x7fbeb806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:00.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.954+0000 7fbece000640 1 -- 172.21.15.67:0/2522311298 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbec8089950 con 0x7fbebc0051c0 2024-03-09T09:04:00.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.955+0000 7fbed5400640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbeb806b480 0x7fbeb806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:00.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.956+0000 7fbed5400640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbeb806b480 0x7fbeb806d940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbec4000ff0 tx=0x7fbec4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:00.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:00.959+0000 7fbece000640 1 -- 172.21.15.67:0/2522311298 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbec8058340 con 0x7fbebc0051c0 2024-03-09T09:04:01.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.309+0000 7fbedca00640 1 -- 172.21.15.67:0/2522311298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbed80c1d70 con 0x7fbebc0051c0 2024-03-09T09:04:01.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.310+0000 7fbece000640 1 -- 172.21.15.67:0/2522311298 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fbec8001150 con 0x7fbebc0051c0 2024-03-09T09:04:01.312 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":86,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:04:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.313+0000 7fbedca00640 1 -- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbeb806b480 msgr2=0x7fbeb806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.313+0000 7fbedca00640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbeb806b480 0x7fbeb806d940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbec4000ff0 tx=0x7fbec4009040 comp rx=0 tx=0).stop 2024-03-09T09:04:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.313+0000 7fbedca00640 1 -- 172.21.15.67:0/2522311298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 msgr2=0x7fbed81560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.313+0000 7fbedca00640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 0x7fbed81560e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbec800c4a0 tx=0x7fbec800c960 comp rx=0 tx=0).stop 2024-03-09T09:04:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.313+0000 7fbedca00640 1 -- 172.21.15.67:0/2522311298 shutdown_connections 2024-03-09T09:04:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.313+0000 7fbedca00640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbeb806b480 0x7fbeb806d940 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.313+0000 7fbedca00640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbed8156620 0x7fbed80abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.313+0000 7fbedca00640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbed81ec440 0x7fbed81dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.313+0000 7fbedca00640 1 --2- 172.21.15.67:0/2522311298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbebc0051c0 0x7fbed81560e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.313+0000 7fbedca00640 1 -- 172.21.15.67:0/2522311298 >> 172.21.15.67:0/2522311298 conn(0x7fbed80c7370 msgr2=0x7fbed80c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:01.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.314+0000 7fbedca00640 1 -- 172.21.15.67:0/2522311298 shutdown_connections 2024-03-09T09:04:01.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:01.314+0000 7fbedca00640 1 -- 172.21.15.67:0/2522311298 wait complete. 2024-03-09T09:04:04.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:04:04.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.541+0000 7fdd8e000640 1 Processor -- start 2024-03-09T09:04:04.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.541+0000 7fdd8e000640 1 -- start start 2024-03-09T09:04:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.542+0000 7fdd8e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd880710a0 0x7fdd88153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.543+0000 7fdd8e000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdd88151360 0x7fdd88153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.543+0000 7fdd8e000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd880c81e0 con 0x7fdd880c1f70 2024-03-09T09:04:04.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.543+0000 7fdd8e000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd880c32b0 con 0x7fdd880710a0 2024-03-09T09:04:04.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.543+0000 7fdd8e000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd880c2fd0 con 0x7fdd88151360 2024-03-09T09:04:04.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.543+0000 7fdd87e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd880710a0 0x7fdd88153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:04.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.543+0000 7fdd87e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd880710a0 0x7fdd88153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57528/0 (socket says 172.21.15.67:57528) 2024-03-09T09:04:04.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.543+0000 7fdd87e00640 1 -- 172.21.15.67:0/679615004 learned_addr learned my addr 172.21.15.67:0/679615004 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:04.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.543+0000 7fdd87e00640 1 -- 172.21.15.67:0/679615004 >> v2:172.21.15.134:3300/0 conn(0x7fdd88151360 msgr2=0x7fdd88153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:04.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.543+0000 7fdd87e00640 1 --2- 172.21.15.67:0/679615004 >> v2:172.21.15.134:3300/0 conn(0x7fdd88151360 0x7fdd88153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:04.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.543+0000 7fdd87e00640 1 -- 172.21.15.67:0/679615004 >> v1:172.21.15.134:6789/0 conn(0x7fdd880c1f70 legacy=0x7fdd88150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:04.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.543+0000 7fdd87e00640 1 -- 172.21.15.67:0/679615004 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd880c3700 con 0x7fdd880710a0 2024-03-09T09:04:04.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.545+0000 7fdd87e00640 1 --2- 172.21.15.67:0/679615004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd880710a0 0x7fdd88153970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdd88073ef0 tx=0x7fdd7402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced0a94c0ef63899 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:04.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.545+0000 7fdd87400640 1 -- 172.21.15.67:0/679615004 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7402e070 con 0x7fdd880710a0 2024-03-09T09:04:04.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.545+0000 7fdd87400640 1 -- 172.21.15.67:0/679615004 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd74037070 con 0x7fdd880710a0 2024-03-09T09:04:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.545+0000 7fdd87400640 1 -- 172.21.15.67:0/679615004 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd74031a00 con 0x7fdd880710a0 2024-03-09T09:04:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.546+0000 7fdd8e000640 1 -- 172.21.15.67:0/679615004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd880710a0 msgr2=0x7fdd88153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:04.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.546+0000 7fdd8e000640 1 --2- 172.21.15.67:0/679615004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd880710a0 0x7fdd88153970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdd88073ef0 tx=0x7fdd7402b880 comp rx=0 tx=0).stop 2024-03-09T09:04:04.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.546+0000 7fdd8e000640 1 -- 172.21.15.67:0/679615004 shutdown_connections 2024-03-09T09:04:04.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.546+0000 7fdd8e000640 1 --2- 172.21.15.67:0/679615004 >> v2:172.21.15.134:3300/0 conn(0x7fdd88151360 0x7fdd88153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:04.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.546+0000 7fdd8e000640 1 --2- 172.21.15.67:0/679615004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd880710a0 0x7fdd88153970 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:04.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.546+0000 7fdd8e000640 1 -- 172.21.15.67:0/679615004 >> 172.21.15.67:0/679615004 conn(0x7fdd880c7300 msgr2=0x7fdd880c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:04.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.546+0000 7fdd8e000640 1 -- 172.21.15.67:0/679615004 shutdown_connections 2024-03-09T09:04:04.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.546+0000 7fdd8e000640 1 -- 172.21.15.67:0/679615004 wait complete. 2024-03-09T09:04:04.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.548+0000 7fdd8e000640 1 Processor -- start 2024-03-09T09:04:04.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.548+0000 7fdd8e000640 1 -- start start 2024-03-09T09:04:04.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.549+0000 7fdd8e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd880710a0 0x7fdd881ef730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:04.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.549+0000 7fdd8e000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd880c1f70 0x7fdd880c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:04.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.549+0000 7fdd8e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88151360 0x7fdd880aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:04.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.549+0000 7fdd8e000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd88070d50 con 0x7fdd880710a0 2024-03-09T09:04:04.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.549+0000 7fdd8e000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd88070830 con 0x7fdd88151360 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.549+0000 7fdd8e000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd88070ed0 con 0x7fdd880c1f70 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.549+0000 7fdd8cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd880710a0 0x7fdd881ef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.549+0000 7fdd87e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd880c1f70 msgr2=0x7fdd880c09a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.549+0000 7fdd87e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd880c1f70 0x7fdd880c09a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.549+0000 7fdd8cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd880710a0 0x7fdd881ef730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57536/0 (socket says 172.21.15.67:57536) 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.549+0000 7fdd8cc00640 1 -- 172.21.15.67:0/1046950434 learned_addr learned my addr 172.21.15.67:0/1046950434 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.550+0000 7fdd8d600640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88151360 0x7fdd880aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.550+0000 7fdd8cc00640 1 -- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd880c1f70 msgr2=0x7fdd880c09a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.550+0000 7fdd8cc00640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd880c1f70 0x7fdd880c09a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.550+0000 7fdd8cc00640 1 -- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88151360 msgr2=0x7fdd880aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.550+0000 7fdd8cc00640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88151360 0x7fdd880aab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.550+0000 7fdd8cc00640 1 -- 172.21.15.67:0/1046950434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd881efc70 con 0x7fdd880710a0 2024-03-09T09:04:04.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.550+0000 7fdd8d600640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88151360 0x7fdd880aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:04:04.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.550+0000 7fdd8cc00640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd880710a0 0x7fdd881ef730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00c430 tx=0x7fdd7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:04.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.551+0000 7fdd84c00640 1 -- 172.21.15.67:0/1046950434 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c011790 con 0x7fdd880710a0 2024-03-09T09:04:04.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.551+0000 7fdd84c00640 1 -- 172.21.15.67:0/1046950434 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c017070 con 0x7fdd880710a0 2024-03-09T09:04:04.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.551+0000 7fdd84c00640 1 -- 172.21.15.67:0/1046950434 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c011bb0 con 0x7fdd880710a0 2024-03-09T09:04:04.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.551+0000 7fdd8e000640 1 -- 172.21.15.67:0/1046950434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8814af10 con 0x7fdd880710a0 2024-03-09T09:04:04.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.551+0000 7fdd8e000640 1 -- 172.21.15.67:0/1046950434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8814b4d0 con 0x7fdd880710a0 2024-03-09T09:04:04.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.553+0000 7fdd8e000640 1 -- 172.21.15.67:0/1046950434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8814b710 con 0x7fdd880710a0 2024-03-09T09:04:04.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.554+0000 7fdd84c00640 1 -- 172.21.15.67:0/1046950434 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdd7c01c460 con 0x7fdd880710a0 2024-03-09T09:04:04.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.556+0000 7fdd84c00640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd6806b500 0x7fdd6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:04.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.557+0000 7fdd84c00640 1 -- 172.21.15.67:0/1046950434 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdd7c08a390 con 0x7fdd880710a0 2024-03-09T09:04:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.561+0000 7fdd87e00640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd6806b500 0x7fdd6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:04.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.564+0000 7fdd84c00640 1 -- 172.21.15.67:0/1046950434 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdd8814b710 con 0x7fdd880710a0 2024-03-09T09:04:04.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.564+0000 7fdd87e00640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd6806b500 0x7fdd6806d9c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdd74036000 tx=0x7fdd74008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:04.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.867+0000 7fdd8e000640 1 -- 172.21.15.67:0/1046950434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdd880737e0 con 0x7fdd880710a0 2024-03-09T09:04:04.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.868+0000 7fdd84c00640 1 -- 172.21.15.67:0/1046950434 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fdd880737e0 con 0x7fdd880710a0 2024-03-09T09:04:04.869 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":86,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 -- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd6806b500 msgr2=0x7fdd6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd6806b500 0x7fdd6806d9c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdd74036000 tx=0x7fdd74008030 comp rx=0 tx=0).stop 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 -- 172.21.15.67:0/1046950434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd880710a0 msgr2=0x7fdd881ef730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd880710a0 0x7fdd881ef730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00c430 tx=0x7fdd7c00c8f0 comp rx=0 tx=0).stop 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 -- 172.21.15.67:0/1046950434 shutdown_connections 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd6806b500 0x7fdd6806d9c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88151360 0x7fdd880aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd880c1f70 0x7fdd880c09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 --2- 172.21.15.67:0/1046950434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd880710a0 0x7fdd881ef730 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 -- 172.21.15.67:0/1046950434 >> 172.21.15.67:0/1046950434 conn(0x7fdd880c7300 msgr2=0x7fdd8814f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 -- 172.21.15.67:0/1046950434 shutdown_connections 2024-03-09T09:04:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:04.871+0000 7fdd8e000640 1 -- 172.21.15.67:0/1046950434 wait complete. 2024-03-09T09:04:07.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:04:08.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.099+0000 7f475b600640 1 Processor -- start 2024-03-09T09:04:08.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.099+0000 7f475b600640 1 -- start start 2024-03-09T09:04:08.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.100+0000 7f475b600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:08.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.100+0000 7f475b600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47540c22a0 0x7f47540acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:08.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.100+0000 7f475b600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4754153d40 con 0x7f4754157650 2024-03-09T09:04:08.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.100+0000 7f475b600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4754070ee0 con 0x7f47540ac730 2024-03-09T09:04:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.100+0000 7f475b600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4754154d00 con 0x7f47540c22a0 2024-03-09T09:04:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.101+0000 7f4753e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47540c22a0 msgr2=0x7f47540acb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.101+0000 7f4753e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47540c22a0 0x7f47540acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.101+0000 7f4758c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.101+0000 7f4758c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57546/0 (socket says 172.21.15.67:57546) 2024-03-09T09:04:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.101+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 learned_addr learned my addr 172.21.15.67:0/3337298642 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.101+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 msgr2=0x7f4754071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:04:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.101+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 msgr2=0x7f4754071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:04:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.101+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.101+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:08.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.302+0000 7f4753e00640 1 -- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47540c22a0 msgr2=0x7f47540acb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:08.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.302+0000 7f4753e00640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47540c22a0 0x7f47540acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:04:08.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.302+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:08.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.303+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 msgr2=0x7f4754071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:04:08.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.303+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 msgr2=0x7f4754071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:04:08.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.303+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:08.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.303+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:04:08.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.703+0000 7f4753e00640 1 -- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47540c22a0 msgr2=0x7f47540acb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:08.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.703+0000 7f4753e00640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47540c22a0 0x7f47540acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:04:08.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.704+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:08.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.705+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 msgr2=0x7f4754071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:04:08.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.705+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 msgr2=0x7f4754071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:04:08.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.705+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:08.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:08.705+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:04:09.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:09.504+0000 7f4753e00640 1 -- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47540c22a0 msgr2=0x7f47540acb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:09.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:09.504+0000 7f4753e00640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47540c22a0 0x7f47540acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:04:09.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:09.506+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:09.507+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 msgr2=0x7f4754071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:04:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:09.507+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 msgr2=0x7f4754071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:04:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:09.507+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:09.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:09.507+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:04:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.101+0000 7f4752000640 1 -- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47540c22a0 msgr2=0x7f47540acb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:04:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.101+0000 7f4752000640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47540c22a0 0x7f47540acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.101+0000 7f4752000640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 msgr2=0x7f4754071190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:04:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.101+0000 7f4752000640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.67:3300/0 conn(0x7f47540ac730 0x7f4754071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.101+0000 7f4752000640 1 -- 172.21.15.67:0/3337298642 >> v1:172.21.15.67:6789/0 conn(0x7f4754157650 legacy=0x7f4754157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.101+0000 7f4752000640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 0x7f4734005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.101+0000 7f4752000640 1 -- 172.21.15.67:0/3337298642 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47540c2790 con 0x7f4734006170 2024-03-09T09:04:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.101+0000 7f4752000640 1 -- 172.21.15.67:0/3337298642 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4754070ee0 con 0x7f47340036c0 2024-03-09T09:04:11.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.102+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 0x7f4734005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:11.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.102+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 msgr2=0x7f4734005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:04:11.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.102+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 msgr2=0x7f4734005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:04:11.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.102+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 0x7f4734005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:11.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.102+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 0x7f4734005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:11.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.303+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 0x7f4734005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:11.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.304+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 msgr2=0x7f4734005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:04:11.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.304+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 msgr2=0x7f4734005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:04:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.304+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 0x7f4734005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.304+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 0x7f4734005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:04:11.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.395+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288391195 0 0) 0x7f47540c2790 con 0x7f4734006170 2024-03-09T09:04:11.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.395+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f473c003520 con 0x7f4734006170 2024-03-09T09:04:11.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.396+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2983065759 0 0) 0x7f473c003520 con 0x7f4734006170 2024-03-09T09:04:11.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.396+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 msgr2=0x7f4734005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:04:11.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.396+0000 7f4753400640 1 --2- 172.21.15.67:0/3337298642 >> v2:172.21.15.134:3300/0 conn(0x7f47340036c0 0x7f4734005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.396+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473400af80 con 0x7f4734006170 2024-03-09T09:04:11.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.396+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f4740004340 con 0x7f4734006170 2024-03-09T09:04:11.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.396+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 >> v1:172.21.15.134:6789/0 conn(0x7f4734006170 legacy=0x7f4734008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:11.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.396+0000 7f4759600640 1 -- 172.21.15.67:0/3337298642 reap_dead start 2024-03-09T09:04:11.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.396+0000 7f4753400640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 0x7f473c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:11.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.397+0000 7f4753400640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c004800 0x7f473c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:11.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.397+0000 7f4753400640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c005210 0x7f473c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:11.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.397+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4754070ee0 con 0x7f473c004800 2024-03-09T09:04:11.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.397+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c00c140 con 0x7f473c005210 2024-03-09T09:04:11.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.397+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c00c340 con 0x7f473c0039b0 2024-03-09T09:04:11.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.397+0000 7f4758c00640 1 -- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 msgr2=0x7f473c003db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:11.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.397+0000 7f4758c00640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 0x7f473c003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:11.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.398+0000 7f4759600640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c005210 0x7f473c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:11.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.399+0000 7f4759600640 1 -- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 msgr2=0x7f473c003db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:04:11.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.399+0000 7f4759600640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 0x7f473c003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.399+0000 7f4759600640 1 -- 172.21.15.67:0/3337298642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c004800 msgr2=0x7f473c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:11.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.399+0000 7f4759600640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c004800 0x7f473c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.399+0000 7f4759600640 1 -- 172.21.15.67:0/3337298642 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c003520 con 0x7f473c005210 2024-03-09T09:04:11.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.403+0000 7f4759600640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c005210 0x7f473c00b9c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4754157a30 tx=0x7f473c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89cf4e4d6b00a3ce server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:11.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.405+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c001960 con 0x7f473c005210 2024-03-09T09:04:11.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.405+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474c005d40 con 0x7f473c005210 2024-03-09T09:04:11.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.405+0000 7f4753400640 1 -- 172.21.15.67:0/3337298642 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c0072b0 con 0x7f473c005210 2024-03-09T09:04:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.406+0000 7f475b600640 1 -- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c005210 msgr2=0x7f473c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.406+0000 7f475b600640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c005210 0x7f473c00b9c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4754157a30 tx=0x7f473c005830 comp rx=0 tx=0).stop 2024-03-09T09:04:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.406+0000 7f475b600640 1 -- 172.21.15.67:0/3337298642 shutdown_connections 2024-03-09T09:04:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.406+0000 7f475b600640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c005210 0x7f473c00b9c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.406+0000 7f475b600640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c004800 0x7f473c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.406+0000 7f475b600640 1 --2- 172.21.15.67:0/3337298642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 0x7f473c003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.406+0000 7f475b600640 1 -- 172.21.15.67:0/3337298642 >> 172.21.15.67:0/3337298642 conn(0x7f47540c7370 msgr2=0x7f47540c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.406+0000 7f475b600640 1 -- 172.21.15.67:0/3337298642 shutdown_connections 2024-03-09T09:04:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.406+0000 7f475b600640 1 -- 172.21.15.67:0/3337298642 wait complete. 2024-03-09T09:04:11.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.411+0000 7f475b600640 1 Processor -- start 2024-03-09T09:04:11.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.411+0000 7f475b600640 1 -- start start 2024-03-09T09:04:11.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.414+0000 7f475b600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 0x7f4754148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:11.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.414+0000 7f475b600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c004800 0x7f475414f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:11.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.414+0000 7f475b600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c005210 0x7f47541ea5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:11.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.414+0000 7f475b600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4754151610 con 0x7f473c005210 2024-03-09T09:04:11.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.414+0000 7f475b600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4754070c00 con 0x7f473c004800 2024-03-09T09:04:11.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.414+0000 7f475b600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4754151790 con 0x7f473c0039b0 2024-03-09T09:04:11.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.414+0000 7f4758c00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 msgr2=0x7f4754148c60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:11.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.414+0000 7f4758c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 0x7f4754148c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:11.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.414+0000 7f4759600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c005210 0x7f47541ea5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:11.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.415+0000 7f4759600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c005210 0x7f47541ea5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45686/0 (socket says 172.21.15.67:45686) 2024-03-09T09:04:11.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.415+0000 7f4759600640 1 -- 172.21.15.67:0/2396215765 learned_addr learned my addr 172.21.15.67:0/2396215765 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:11.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.415+0000 7f4753e00640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c004800 0x7f475414f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.415+0000 7f4759600640 1 -- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 msgr2=0x7f4754148c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:04:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.415+0000 7f4759600640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 0x7f4754148c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.415+0000 7f4759600640 1 -- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c004800 msgr2=0x7f475414f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.415+0000 7f4759600640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c004800 0x7f475414f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.415+0000 7f4759600640 1 -- 172.21.15.67:0/2396215765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4754155950 con 0x7f473c005210 2024-03-09T09:04:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.416+0000 7f4759600640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c005210 0x7f47541ea5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f474c02bc80 tx=0x7f474c0014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.416+0000 7f4750c00640 1 -- 172.21.15.67:0/2396215765 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c005b40 con 0x7f473c005210 2024-03-09T09:04:11.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.416+0000 7f4750c00640 1 -- 172.21.15.67:0/2396215765 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474c039040 con 0x7f473c005210 2024-03-09T09:04:11.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.416+0000 7f475b600640 1 -- 172.21.15.67:0/2396215765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47540c40b0 con 0x7f473c005210 2024-03-09T09:04:11.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.416+0000 7f4750c00640 1 -- 172.21.15.67:0/2396215765 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c007830 con 0x7f473c005210 2024-03-09T09:04:11.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.417+0000 7f475b600640 1 -- 172.21.15.67:0/2396215765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4754155300 con 0x7f473c005210 2024-03-09T09:04:11.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.418+0000 7f475b600640 1 -- 172.21.15.67:0/2396215765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4754075a00 con 0x7f473c005210 2024-03-09T09:04:11.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.420+0000 7f4750c00640 1 -- 172.21.15.67:0/2396215765 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f474c046020 con 0x7f473c005210 2024-03-09T09:04:11.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.421+0000 7f4750c00640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473406bcb0 0x7f473406e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:11.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.421+0000 7f4750c00640 1 -- 172.21.15.67:0/2396215765 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f474c0ab790 con 0x7f473c005210 2024-03-09T09:04:11.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.421+0000 7f4758c00640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473406bcb0 0x7f473406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:11.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.422+0000 7f4758c00640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473406bcb0 0x7f473406e170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4748009100 tx=0x7f4748008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:11.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.427+0000 7f4750c00640 1 -- 172.21.15.67:0/2396215765 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f474c07a170 con 0x7f473c005210 2024-03-09T09:04:11.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.786+0000 7f475b600640 1 -- 172.21.15.67:0/2396215765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f47541ea210 con 0x7f473c005210 2024-03-09T09:04:11.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.786+0000 7f4750c00640 1 -- 172.21.15.67:0/2396215765 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7f474c079a70 con 0x7f473c005210 2024-03-09T09:04:11.789 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":90,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:04:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 -- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473406bcb0 msgr2=0x7f473406e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473406bcb0 0x7f473406e170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4748009100 tx=0x7f4748008040 comp rx=0 tx=0).stop 2024-03-09T09:04:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 -- 172.21.15.67:0/2396215765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c005210 msgr2=0x7f47541ea5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c005210 0x7f47541ea5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f474c02bc80 tx=0x7f474c0014d0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 -- 172.21.15.67:0/2396215765 shutdown_connections 2024-03-09T09:04:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f473406bcb0 0x7f473406e170 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f473c005210 0x7f47541ea5f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f473c004800 0x7f475414f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 --2- 172.21.15.67:0/2396215765 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f473c0039b0 0x7f4754148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:11.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 -- 172.21.15.67:0/2396215765 >> 172.21.15.67:0/2396215765 conn(0x7f47540c7370 msgr2=0x7f4754157a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 -- 172.21.15.67:0/2396215765 shutdown_connections 2024-03-09T09:04:11.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:11.789+0000 7f475b600640 1 -- 172.21.15.67:0/2396215765 wait complete. 2024-03-09T09:04:11.808 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:04:11.809 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:04:11.809 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:04:12.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.004+0000 7fcd5ec00640 1 Processor -- start 2024-03-09T09:04:12.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.004+0000 7fcd5ec00640 1 -- start start 2024-03-09T09:04:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.005+0000 7fcd5ec00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcd580c5e00 0x7fcd580acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.005+0000 7fcd5ec00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd580c9380 0x7fcd58156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.005+0000 7fcd5ec00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd580c97d0 con 0x7fcd580c61e0 2024-03-09T09:04:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.005+0000 7fcd5ec00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd580c6690 con 0x7fcd580c9380 2024-03-09T09:04:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.005+0000 7fcd5ec00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd5806d740 con 0x7fcd580c5e00 2024-03-09T09:04:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.005+0000 7fcd5e200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd580c9380 0x7fcd58156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.005+0000 7fcd5e200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd580c9380 0x7fcd58156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45688/0 (socket says 172.21.15.67:45688) 2024-03-09T09:04:12.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.005+0000 7fcd5e200640 1 -- 172.21.15.67:0/863338009 learned_addr learned my addr 172.21.15.67:0/863338009 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:12.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.006+0000 7fcd5e200640 1 -- 172.21.15.67:0/863338009 >> v2:172.21.15.134:3300/0 conn(0x7fcd580c5e00 msgr2=0x7fcd580acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:12.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.006+0000 7fcd5e200640 1 --2- 172.21.15.67:0/863338009 >> v2:172.21.15.134:3300/0 conn(0x7fcd580c5e00 0x7fcd580acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.006+0000 7fcd5e200640 1 -- 172.21.15.67:0/863338009 >> v1:172.21.15.134:6789/0 conn(0x7fcd580c61e0 legacy=0x7fcd5806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:12.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.006+0000 7fcd5e200640 1 -- 172.21.15.67:0/863338009 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580ad7c0 con 0x7fcd580c9380 2024-03-09T09:04:12.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.007+0000 7fcd5e200640 1 --2- 172.21.15.67:0/863338009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd580c9380 0x7fcd58156660 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0098a0 tx=0x7fcd4c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46ceb98c40ee006 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:12.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.008+0000 7fcd57e00640 1 -- 172.21.15.67:0/863338009 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c035020 con 0x7fcd580c9380 2024-03-09T09:04:12.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.008+0000 7fcd57e00640 1 -- 172.21.15.67:0/863338009 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c031640 con 0x7fcd580c9380 2024-03-09T09:04:12.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.008+0000 7fcd57e00640 1 -- 172.21.15.67:0/863338009 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c02d040 con 0x7fcd580c9380 2024-03-09T09:04:12.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.008+0000 7fcd5ec00640 1 -- 172.21.15.67:0/863338009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd580c9380 msgr2=0x7fcd58156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.008+0000 7fcd5ec00640 1 --2- 172.21.15.67:0/863338009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd580c9380 0x7fcd58156660 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0098a0 tx=0x7fcd4c02b8f0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.008+0000 7fcd5ec00640 1 -- 172.21.15.67:0/863338009 shutdown_connections 2024-03-09T09:04:12.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.008+0000 7fcd5ec00640 1 --2- 172.21.15.67:0/863338009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd580c9380 0x7fcd58156660 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.008+0000 7fcd5ec00640 1 --2- 172.21.15.67:0/863338009 >> v2:172.21.15.134:3300/0 conn(0x7fcd580c5e00 0x7fcd580acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.008+0000 7fcd5ec00640 1 -- 172.21.15.67:0/863338009 >> 172.21.15.67:0/863338009 conn(0x7fcd580cb480 msgr2=0x7fcd580cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:12.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.008+0000 7fcd5ec00640 1 -- 172.21.15.67:0/863338009 shutdown_connections 2024-03-09T09:04:12.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.008+0000 7fcd5ec00640 1 -- 172.21.15.67:0/863338009 wait complete. 2024-03-09T09:04:12.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.010+0000 7fcd5ec00640 1 Processor -- start 2024-03-09T09:04:12.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.010+0000 7fcd5ec00640 1 -- start start 2024-03-09T09:04:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.011+0000 7fcd5ec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd580c5e00 0x7fcd581eed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.011+0000 7fcd5ec00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd580c61e0 0x7fcd58157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.011+0000 7fcd5ec00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd580c9380 0x7fcd58157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.011+0000 7fcd5ec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5806bb00 con 0x7fcd580c5e00 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.011+0000 7fcd5ec00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58153680 con 0x7fcd580c9380 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.011+0000 7fcd5ec00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5806bde0 con 0x7fcd580c61e0 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.011+0000 7fcd5ce00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd580c61e0 msgr2=0x7fcd58157920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.011+0000 7fcd5d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd580c5e00 0x7fcd581eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.012+0000 7fcd5ce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd580c61e0 0x7fcd58157920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.012+0000 7fcd5d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd580c5e00 0x7fcd581eed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45698/0 (socket says 172.21.15.67:45698) 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.012+0000 7fcd5d800640 1 -- 172.21.15.67:0/4123236394 learned_addr learned my addr 172.21.15.67:0/4123236394 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.012+0000 7fcd5d800640 1 -- 172.21.15.67:0/4123236394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd580c61e0 msgr2=0x7fcd58157920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.012+0000 7fcd5d800640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd580c61e0 0x7fcd58157920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.012+0000 7fcd5d800640 1 -- 172.21.15.67:0/4123236394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd580c9380 msgr2=0x7fcd58157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.012+0000 7fcd5d800640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd580c9380 0x7fcd58157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.012+0000 7fcd5d800640 1 -- 172.21.15.67:0/4123236394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd58159620 con 0x7fcd580c5e00 2024-03-09T09:04:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.012+0000 7fcd5d800640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd580c5e00 0x7fcd581eed60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000ba50 tx=0x7fcd5000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.013+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50019070 con 0x7fcd580c5e00 2024-03-09T09:04:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.013+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5000cc00 con 0x7fcd580c5e00 2024-03-09T09:04:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.013+0000 7fcd5ec00640 1 -- 172.21.15.67:0/4123236394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd58159f30 con 0x7fcd580c5e00 2024-03-09T09:04:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.013+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50014440 con 0x7fcd580c5e00 2024-03-09T09:04:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.013+0000 7fcd5ec00640 1 -- 172.21.15.67:0/4123236394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5814d2f0 con 0x7fcd580c5e00 2024-03-09T09:04:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.014+0000 7fcd5ec00640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581ef2a0 0x7fcd581eb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.014+0000 7fcd5ec00640 1 -- 172.21.15.67:0/4123236394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58159ce0 con 0x7fcd581ef2a0 2024-03-09T09:04:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.014+0000 7fcd5ce00640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581ef2a0 0x7fcd581eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.014+0000 7fcd5ce00640 1 -- 172.21.15.67:0/4123236394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd58159b00 con 0x7fcd581ef2a0 2024-03-09T09:04:12.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.014+0000 7fcd5ce00640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581ef2a0 0x7fcd581eb580 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd48000fe0 tx=0x7fcd4800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:12.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.015+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd480074e0 con 0x7fcd581ef2a0 2024-03-09T09:04:12.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.015+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcd5001c070 con 0x7fcd580c5e00 2024-03-09T09:04:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.016+0000 7fcd55600640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd3c06b3d0 0x7fcd3c06d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.016+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcd5008a310 con 0x7fcd580c5e00 2024-03-09T09:04:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.016+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fcd48004770 con 0x7fcd581ef2a0 2024-03-09T09:04:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.016+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581ef2a0 msgr2=0x7fcd581eb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.016+0000 7fcd55600640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581ef2a0 0x7fcd581eb580 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd48000fe0 tx=0x7fcd4800c7a0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.016+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581ef2a0 msgr2=0x7fcd581eb580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.016+0000 7fcd55600640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581ef2a0 0x7fcd581eb580 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd48000fe0 tx=0x7fcd4800c7a0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.016+0000 7fcd5e200640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd3c06b3d0 0x7fcd3c06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.018+0000 7fcd5e200640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd3c06b3d0 0x7fcd3c06d890 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c036000 tx=0x7fcd4c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:12.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.024+0000 7fcd5ec00640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581583a0 0x7fcd581ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.024+0000 7fcd5ec00640 1 -- 172.21.15.67:0/4123236394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5806d2f0 con 0x7fcd581583a0 2024-03-09T09:04:12.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.025+0000 7fcd5ce00640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581583a0 0x7fcd581ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.025+0000 7fcd5ce00640 1 -- 172.21.15.67:0/4123236394 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcd580c21b0 con 0x7fcd581583a0 2024-03-09T09:04:12.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.025+0000 7fcd5ce00640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581583a0 0x7fcd581ebac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcd48000fe0 tx=0x7fcd480010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:12.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.026+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48018040 con 0x7fcd581583a0 2024-03-09T09:04:12.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.026+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5177 (secure 0 0 0) 0x7fcd48013410 con 0x7fcd581583a0 2024-03-09T09:04:12.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.026+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581583a0 msgr2=0x7fcd581ebac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.026+0000 7fcd55600640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581583a0 0x7fcd581ebac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcd48000fe0 tx=0x7fcd480010a0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.026+0000 7fcd55600640 1 -- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581583a0 msgr2=0x7fcd581ebac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:12.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.026+0000 7fcd55600640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd581583a0 0x7fcd581ebac0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcd48000fe0 tx=0x7fcd480010a0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.029 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:04:12.029 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:04:12.029 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:04:12.029 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:04:12.029 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 90, 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:12.030 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:12.031 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:04:12.032 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:12.033 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.034 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:04:12.035 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:04:12.036 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.037 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:04:12.038 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:12.039 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.040 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.041 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:12.042 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:12.043 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:12.043 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.043 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.043 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:04:12.043 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:04:12.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.026+0000 7fcd5ec00640 1 -- 172.21.15.67:0/4123236394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd3c06b3d0 msgr2=0x7fcd3c06d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.026+0000 7fcd5ec00640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd3c06b3d0 0x7fcd3c06d890 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c036000 tx=0x7fcd4c030000 comp rx=0 tx=0).stop 2024-03-09T09:04:12.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.027+0000 7fcd5ec00640 1 -- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd580c5e00 msgr2=0x7fcd581eed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.027+0000 7fcd5ec00640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd580c5e00 0x7fcd581eed60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd5000ba50 tx=0x7fcd5000bf10 comp rx=0 tx=0).stop 2024-03-09T09:04:12.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.027+0000 7fcd5e200640 1 -- 172.21.15.67:0/4123236394 reap_dead start 2024-03-09T09:04:12.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.027+0000 7fcd5ec00640 1 -- 172.21.15.67:0/4123236394 shutdown_connections 2024-03-09T09:04:12.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.027+0000 7fcd5ec00640 1 --2- 172.21.15.67:0/4123236394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd580c5e00 0x7fcd581eed60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.027+0000 7fcd5ec00640 1 -- 172.21.15.67:0/4123236394 >> 172.21.15.67:0/4123236394 conn(0x7fcd580cb480 msgr2=0x7fcd58156660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:12.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.027+0000 7fcd5ec00640 1 -- 172.21.15.67:0/4123236394 shutdown_connections 2024-03-09T09:04:12.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.027+0000 7fcd5ec00640 1 -- 172.21.15.67:0/4123236394 wait complete. 2024-03-09T09:04:12.045 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:04:12.045 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:04:12.045 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:04:12.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.262+0000 7fcf63000640 1 Processor -- start 2024-03-09T09:04:12.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.262+0000 7fcf63000640 1 -- start start 2024-03-09T09:04:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.264+0000 7fcf63000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcf5c153560 0x7fcf5c06b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.264+0000 7fcf63000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcf5c0c5f50 0x7fcf5c0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.265+0000 7fcf63000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5c06cf30 con 0x7fcf5c0c96f0 2024-03-09T09:04:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.265+0000 7fcf63000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5c159d80 con 0x7fcf5c0c5f50 2024-03-09T09:04:12.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.265+0000 7fcf63000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5c15a060 con 0x7fcf5c153560 2024-03-09T09:04:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.265+0000 7fcf61000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcf5c0c5f50 0x7fcf5c0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.265+0000 7fcf61000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcf5c0c5f50 0x7fcf5c0c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45732/0 (socket says 172.21.15.67:45732) 2024-03-09T09:04:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.265+0000 7fcf61000640 1 -- 172.21.15.67:0/3499447208 learned_addr learned my addr 172.21.15.67:0/3499447208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.265+0000 7fcf5be00640 1 --2- 172.21.15.67:0/3499447208 >> v2:172.21.15.134:3300/0 conn(0x7fcf5c153560 0x7fcf5c06b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.265+0000 7fcf61000640 1 -- 172.21.15.67:0/3499447208 >> v2:172.21.15.134:3300/0 conn(0x7fcf5c153560 msgr2=0x7fcf5c06b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:12.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.265+0000 7fcf61000640 1 --2- 172.21.15.67:0/3499447208 >> v2:172.21.15.134:3300/0 conn(0x7fcf5c153560 0x7fcf5c06b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.266+0000 7fcf61000640 1 -- 172.21.15.67:0/3499447208 >> v1:172.21.15.134:6789/0 conn(0x7fcf5c0c96f0 legacy=0x7fcf5c0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:12.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.266+0000 7fcf61000640 1 -- 172.21.15.67:0/3499447208 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c0ca440 con 0x7fcf5c0c5f50 2024-03-09T09:04:12.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.267+0000 7fcf61000640 1 --2- 172.21.15.67:0/3499447208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf5c0c5f50 0x7fcf5c0c6360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c073c90 tx=0x7fcf5402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e3ec308dc3652 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:12.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.268+0000 7fcf5aa00640 1 -- 172.21.15.67:0/3499447208 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5402ca80 con 0x7fcf5c0c5f50 2024-03-09T09:04:12.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.268+0000 7fcf5aa00640 1 -- 172.21.15.67:0/3499447208 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5402cc20 con 0x7fcf5c0c5f50 2024-03-09T09:04:12.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.268+0000 7fcf5aa00640 1 -- 172.21.15.67:0/3499447208 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf540343f0 con 0x7fcf5c0c5f50 2024-03-09T09:04:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.268+0000 7fcf63000640 1 -- 172.21.15.67:0/3499447208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf5c0c5f50 msgr2=0x7fcf5c0c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.268+0000 7fcf63000640 1 --2- 172.21.15.67:0/3499447208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf5c0c5f50 0x7fcf5c0c6360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c073c90 tx=0x7fcf5402bee0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.268+0000 7fcf63000640 1 -- 172.21.15.67:0/3499447208 shutdown_connections 2024-03-09T09:04:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.268+0000 7fcf63000640 1 --2- 172.21.15.67:0/3499447208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf5c0c5f50 0x7fcf5c0c6360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.268+0000 7fcf63000640 1 --2- 172.21.15.67:0/3499447208 >> v2:172.21.15.134:3300/0 conn(0x7fcf5c153560 0x7fcf5c06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.268+0000 7fcf63000640 1 -- 172.21.15.67:0/3499447208 >> 172.21.15.67:0/3499447208 conn(0x7fcf5c0c8050 msgr2=0x7fcf5c0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:12.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.269+0000 7fcf63000640 1 -- 172.21.15.67:0/3499447208 shutdown_connections 2024-03-09T09:04:12.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.269+0000 7fcf63000640 1 -- 172.21.15.67:0/3499447208 wait complete. 2024-03-09T09:04:12.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.270+0000 7fcf63000640 1 Processor -- start 2024-03-09T09:04:12.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.270+0000 7fcf63000640 1 -- start start 2024-03-09T09:04:12.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.271+0000 7fcf63000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c0c5f50 0x7fcf5c157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.271+0000 7fcf63000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf5c0c96f0 0x7fcf5c148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.271+0000 7fcf63000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf5c153560 0x7fcf5c148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.271+0000 7fcf63000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0c68a0 con 0x7fcf5c153560 2024-03-09T09:04:12.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.271+0000 7fcf63000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0c9ff0 con 0x7fcf5c0c5f50 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.271+0000 7fcf63000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0c6a20 con 0x7fcf5c0c96f0 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.271+0000 7fcf5b400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf5c0c96f0 msgr2=0x7fcf5c148720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.271+0000 7fcf5b400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf5c0c96f0 0x7fcf5c148720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.271+0000 7fcf61000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf5c153560 0x7fcf5c148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.272+0000 7fcf61000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf5c153560 0x7fcf5c148c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45748/0 (socket says 172.21.15.67:45748) 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.272+0000 7fcf61000640 1 -- 172.21.15.67:0/3375237694 learned_addr learned my addr 172.21.15.67:0/3375237694 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.272+0000 7fcf5be00640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c0c5f50 0x7fcf5c157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.272+0000 7fcf61000640 1 -- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf5c0c96f0 msgr2=0x7fcf5c148720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.272+0000 7fcf61000640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf5c0c96f0 0x7fcf5c148720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.272+0000 7fcf61000640 1 -- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c0c5f50 msgr2=0x7fcf5c157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.272+0000 7fcf61000640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c0c5f50 0x7fcf5c157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.272+0000 7fcf61000640 1 -- 172.21.15.67:0/3375237694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c0abbf0 con 0x7fcf5c153560 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.272+0000 7fcf61000640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf5c153560 0x7fcf5c148c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009f40 tx=0x7fcf54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:12.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.272+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54043020 con 0x7fcf5c153560 2024-03-09T09:04:12.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.273+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5403e450 con 0x7fcf5c153560 2024-03-09T09:04:12.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.273+0000 7fcf63000640 1 -- 172.21.15.67:0/3375237694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c157ed0 con 0x7fcf5c153560 2024-03-09T09:04:12.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.273+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54009510 con 0x7fcf5c153560 2024-03-09T09:04:12.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.273+0000 7fcf63000640 1 -- 172.21.15.67:0/3375237694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c0ab3f0 con 0x7fcf5c153560 2024-03-09T09:04:12.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.274+0000 7fcf63000640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c0adbc0 0x7fcf5c158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.274+0000 7fcf63000640 1 -- 172.21.15.67:0/3375237694 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c068d20 con 0x7fcf5c0adbc0 2024-03-09T09:04:12.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.274+0000 7fcf5be00640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c0adbc0 0x7fcf5c158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.275+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcf54046020 con 0x7fcf5c153560 2024-03-09T09:04:12.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.275+0000 7fcf5be00640 1 -- 172.21.15.67:0/3375237694 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf5c075a00 con 0x7fcf5c0adbc0 2024-03-09T09:04:12.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.275+0000 7fcf5be00640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c0adbc0 0x7fcf5c158540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00bfd0 tx=0x7fcf4c00f8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:12.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.276+0000 7fcf53e00640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcf3c06b480 0x7fcf3c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.276+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcf540acaf0 con 0x7fcf5c153560 2024-03-09T09:04:12.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.277+0000 7fcf5b400640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcf3c06b480 0x7fcf3c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.277+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c019030 con 0x7fcf5c0adbc0 2024-03-09T09:04:12.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.277+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fcf4c004020 con 0x7fcf5c0adbc0 2024-03-09T09:04:12.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.277+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c0adbc0 msgr2=0x7fcf5c158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.277+0000 7fcf53e00640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c0adbc0 0x7fcf5c158540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00bfd0 tx=0x7fcf4c00f8a0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.277+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c0adbc0 msgr2=0x7fcf5c158540 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:12.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.277+0000 7fcf53e00640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c0adbc0 0x7fcf5c158540 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00bfd0 tx=0x7fcf4c00f8a0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.278+0000 7fcf5b400640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcf3c06b480 0x7fcf3c06d940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcf48000f60 tx=0x7fcf48007a30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:12.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.284+0000 7fcf63000640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c158a80 0x7fcf5c1ef080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:12.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.284+0000 7fcf63000640 1 -- 172.21.15.67:0/3375237694 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0c94d0 con 0x7fcf5c158a80 2024-03-09T09:04:12.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.284+0000 7fcf5be00640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c158a80 0x7fcf5c1ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:12.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.285+0000 7fcf5be00640 1 -- 172.21.15.67:0/3375237694 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcf5c1491a0 con 0x7fcf5c158a80 2024-03-09T09:04:12.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.286+0000 7fcf5be00640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c158a80 0x7fcf5c1ef080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00bfd0 tx=0x7fcf4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:12.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.286+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c014bc0 con 0x7fcf5c158a80 2024-03-09T09:04:12.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.287+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5005 (secure 0 0 0) 0x7fcf4c014bc0 con 0x7fcf5c158a80 2024-03-09T09:04:12.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.287+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c158a80 msgr2=0x7fcf5c1ef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.290 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:04:12.290 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:04:12.290 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:04:12.290 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:04:12.290 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 90, 2024-03-09T09:04:12.290 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:04:12.290 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:04:12.290 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:12.291 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:12.292 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:04:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:12.294 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:12.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:04:12.296 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:12.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:12.299 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.300 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:04:12.301 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:12.302 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:12.303 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:12.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:12.303 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:04:12.303 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.287+0000 7fcf53e00640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c158a80 0x7fcf5c1ef080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00bfd0 tx=0x7fcf4c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.287+0000 7fcf53e00640 1 -- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c158a80 msgr2=0x7fcf5c1ef080 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.287+0000 7fcf53e00640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c158a80 0x7fcf5c1ef080 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00bfd0 tx=0x7fcf4c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.288+0000 7fcf63000640 1 -- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcf3c06b480 msgr2=0x7fcf3c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.288+0000 7fcf63000640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcf3c06b480 0x7fcf3c06d940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcf48000f60 tx=0x7fcf48007a30 comp rx=0 tx=0).stop 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.288+0000 7fcf63000640 1 -- 172.21.15.67:0/3375237694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf5c153560 msgr2=0x7fcf5c148c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.288+0000 7fcf63000640 1 --2- 172.21.15.67:0/3375237694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf5c153560 0x7fcf5c148c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009f40 tx=0x7fcf54004a40 comp rx=0 tx=0).stop 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.288+0000 7fcf61000640 1 -- 172.21.15.67:0/3375237694 reap_dead start 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.288+0000 7fcf61000640 1 -- 172.21.15.67:0/3375237694 reap_dead start 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.288+0000 7fcf63000640 1 -- 172.21.15.67:0/3375237694 shutdown_connections 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.288+0000 7fcf63000640 1 -- 172.21.15.67:0/3375237694 >> 172.21.15.67:0/3375237694 conn(0x7fcf5c0c8050 msgr2=0x7fcf5c151890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:12.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.288+0000 7fcf63000640 1 -- 172.21.15.67:0/3375237694 shutdown_connections 2024-03-09T09:04:12.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:12.288+0000 7fcf63000640 1 -- 172.21.15.67:0/3375237694 wait complete. 2024-03-09T09:04:12.307 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:04:12.307 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:04:14.970 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:04:14.979 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:04:32.307 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:04:32.308 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:04:32.308 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:04:32.308 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:04:32.316 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:04:32.316 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:04:32.316 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:04:32.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.581+0000 7f2597200640 1 Processor -- start 2024-03-09T09:04:32.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.581+0000 7f2597200640 1 -- start start 2024-03-09T09:04:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.582+0000 7f2597200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f25980710a0 0x7f2598153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:32.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2597200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2598151360 0x7f2598153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2597200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25980c81e0 con 0x7f25980c1f70 2024-03-09T09:04:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2597200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25980c32b0 con 0x7f25980710a0 2024-03-09T09:04:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2597200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25980c2fd0 con 0x7f2598151360 2024-03-09T09:04:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2595400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f25980710a0 0x7f2598153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2595e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f25980c1f70 0x7f2598150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:45864/0 (socket says 172.21.15.67:45864) 2024-03-09T09:04:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2595e00640 1 -- 172.21.15.67:0/3642514378 learned_addr learned my addr 172.21.15.67:0/3642514378 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:04:32.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2596800640 1 --2- 172.21.15.67:0/3642514378 >> v2:172.21.15.134:3300/0 conn(0x7f2598151360 0x7f2598153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2595400640 1 -- 172.21.15.67:0/3642514378 >> v2:172.21.15.134:3300/0 conn(0x7f2598151360 msgr2=0x7f2598153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2595400640 1 --2- 172.21.15.67:0/3642514378 >> v2:172.21.15.134:3300/0 conn(0x7f2598151360 0x7f2598153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2595400640 1 -- 172.21.15.67:0/3642514378 >> v1:172.21.15.134:6789/0 conn(0x7f25980c1f70 legacy=0x7f2598150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.583+0000 7f2595400640 1 -- 172.21.15.67:0/3642514378 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25980c3700 con 0x7f25980710a0 2024-03-09T09:04:32.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.585+0000 7f2595400640 1 --2- 172.21.15.67:0/3642514378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25980710a0 0x7f2598153970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2598073ef0 tx=0x7f258402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ea9f48675634a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:32.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.585+0000 7f2594a00640 1 -- 172.21.15.67:0/3642514378 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840315a0 con 0x7f25980710a0 2024-03-09T09:04:32.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.585+0000 7f2594a00640 1 -- 172.21.15.67:0/3642514378 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2584037070 con 0x7f25980710a0 2024-03-09T09:04:32.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.585+0000 7f2594a00640 1 -- 172.21.15.67:0/3642514378 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584031960 con 0x7f25980710a0 2024-03-09T09:04:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.586+0000 7f2597200640 1 -- 172.21.15.67:0/3642514378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25980710a0 msgr2=0x7f2598153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.586+0000 7f2597200640 1 --2- 172.21.15.67:0/3642514378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25980710a0 0x7f2598153970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2598073ef0 tx=0x7f258402b840 comp rx=0 tx=0).stop 2024-03-09T09:04:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.586+0000 7f2597200640 1 -- 172.21.15.67:0/3642514378 shutdown_connections 2024-03-09T09:04:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.586+0000 7f2597200640 1 --2- 172.21.15.67:0/3642514378 >> v2:172.21.15.134:3300/0 conn(0x7f2598151360 0x7f2598153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.586+0000 7f2597200640 1 --2- 172.21.15.67:0/3642514378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25980710a0 0x7f2598153970 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.586+0000 7f2597200640 1 -- 172.21.15.67:0/3642514378 >> 172.21.15.67:0/3642514378 conn(0x7f25980c7300 msgr2=0x7f25980c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.586+0000 7f2597200640 1 -- 172.21.15.67:0/3642514378 shutdown_connections 2024-03-09T09:04:32.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.586+0000 7f2597200640 1 -- 172.21.15.67:0/3642514378 wait complete. 2024-03-09T09:04:32.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.588+0000 7f2597200640 1 Processor -- start 2024-03-09T09:04:32.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.588+0000 7f2597200640 1 -- start start 2024-03-09T09:04:32.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.589+0000 7f2597200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25980710a0 0x7f25981de820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:32.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.589+0000 7f2597200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25980c1f70 0x7f25981dd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:32.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.589+0000 7f2597200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2598151360 0x7f25981df3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:32.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.589+0000 7f2597200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2598070d50 con 0x7f2598151360 2024-03-09T09:04:32.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.589+0000 7f2597200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2598070830 con 0x7f25980710a0 2024-03-09T09:04:32.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.589+0000 7f2597200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2598070ed0 con 0x7f25980c1f70 2024-03-09T09:04:32.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.589+0000 7f2596800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2598151360 0x7f25981df3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:32.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2596800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2598151360 0x7f25981df3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34590/0 (socket says 172.21.15.67:34590) 2024-03-09T09:04:32.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2596800640 1 -- 172.21.15.67:0/1454963997 learned_addr learned my addr 172.21.15.67:0/1454963997 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:32.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2595400640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25980c1f70 0x7f25981dd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:32.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2595e00640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25980710a0 0x7f25981de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:32.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2595400640 1 -- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25980c1f70 msgr2=0x7f25981dd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:04:32.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2595400640 1 -- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25980c1f70 msgr2=0x7f25981dd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:04:32.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2595400640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25980c1f70 0x7f25981dd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:32.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2595400640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25980c1f70 0x7f25981dd900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:32.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2596800640 1 -- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25980c1f70 msgr2=0x7f25981dd900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:04:32.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2596800640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25980c1f70 0x7f25981dd900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:32.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2596800640 1 -- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25980710a0 msgr2=0x7f25981de820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:32.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2596800640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25980710a0 0x7f25981de820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:32.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2596800640 1 -- 172.21.15.67:0/1454963997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25981ec370 con 0x7f2598151360 2024-03-09T09:04:32.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.590+0000 7f2596800640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2598151360 0x7f25981df3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f259000efc0 tx=0x7f259000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:32.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.591+0000 7f258e000640 1 -- 172.21.15.67:0/1454963997 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590018020 con 0x7f2598151360 2024-03-09T09:04:32.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.591+0000 7f258e000640 1 -- 172.21.15.67:0/1454963997 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590014480 con 0x7f2598151360 2024-03-09T09:04:32.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.591+0000 7f258e000640 1 -- 172.21.15.67:0/1454963997 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590014870 con 0x7f2598151360 2024-03-09T09:04:32.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.591+0000 7f2597200640 1 -- 172.21.15.67:0/1454963997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25981dce20 con 0x7f2598151360 2024-03-09T09:04:32.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.591+0000 7f2597200640 1 -- 172.21.15.67:0/1454963997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2598157e20 con 0x7f2598151360 2024-03-09T09:04:32.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.592+0000 7f2597200640 1 -- 172.21.15.67:0/1454963997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259804f3c0 con 0x7f2598151360 2024-03-09T09:04:32.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.594+0000 7f258e000640 1 -- 172.21.15.67:0/1454963997 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2590005d10 con 0x7f2598151360 2024-03-09T09:04:32.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.595+0000 7f258e000640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f257806b500 0x7f257806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:32.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.595+0000 7f258e000640 1 -- 172.21.15.67:0/1454963997 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f259008b3c0 con 0x7f2598151360 2024-03-09T09:04:32.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.595+0000 7f2595e00640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f257806b500 0x7f257806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:32.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.596+0000 7f2595e00640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f257806b500 0x7f257806d9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2588007fd0 tx=0x7f2588007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:32.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.600+0000 7f258e000640 1 -- 172.21.15.67:0/1454963997 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2590022030 con 0x7f2598151360 2024-03-09T09:04:32.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.923+0000 7f2597200640 1 -- 172.21.15.67:0/1454963997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f25980acda0 con 0x7f2598151360 2024-03-09T09:04:32.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.924+0000 7f258e000640 1 -- 172.21.15.67:0/1454963997 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f2590010040 con 0x7f2598151360 2024-03-09T09:04:32.927 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":90,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:04:32.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 -- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f257806b500 msgr2=0x7f257806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:32.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f257806b500 0x7f257806d9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2588007fd0 tx=0x7f2588007e80 comp rx=0 tx=0).stop 2024-03-09T09:04:32.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 -- 172.21.15.67:0/1454963997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2598151360 msgr2=0x7f25981df3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:32.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2598151360 0x7f25981df3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f259000efc0 tx=0x7f259000c490 comp rx=0 tx=0).stop 2024-03-09T09:04:32.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 -- 172.21.15.67:0/1454963997 shutdown_connections 2024-03-09T09:04:32.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f257806b500 0x7f257806d9c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:32.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2598151360 0x7f25981df3d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:32.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25980c1f70 0x7f25981dd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:32.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 --2- 172.21.15.67:0/1454963997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25980710a0 0x7f25981de820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:32.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 -- 172.21.15.67:0/1454963997 >> 172.21.15.67:0/1454963997 conn(0x7f25980c7300 msgr2=0x7f25980731a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:32.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 -- 172.21.15.67:0/1454963997 shutdown_connections 2024-03-09T09:04:32.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:32.927+0000 7f2597200640 1 -- 172.21.15.67:0/1454963997 wait complete. 2024-03-09T09:04:35.949 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:04:36.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.189+0000 7f043fe00640 1 Processor -- start 2024-03-09T09:04:36.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.189+0000 7f043fe00640 1 -- start start 2024-03-09T09:04:36.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.191+0000 7f043fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f04400710a0 0x7f0440153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.191+0000 7f043fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0440151360 0x7f0440153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.191+0000 7f043fe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04400c81e0 con 0x7f04400c1f70 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.191+0000 7f043fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04400c32b0 con 0x7f04400710a0 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.191+0000 7f043fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04400c2fd0 con 0x7f0440151360 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.191+0000 7f0437e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f04400710a0 0x7f0440153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.191+0000 7f0437e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f04400710a0 0x7f0440153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34600/0 (socket says 172.21.15.67:34600) 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.191+0000 7f0437e00640 1 -- 172.21.15.67:0/3295587431 learned_addr learned my addr 172.21.15.67:0/3295587431 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.191+0000 7f043f400640 1 --2- 172.21.15.67:0/3295587431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0440151360 0x7f0440153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.192+0000 7f043f400640 1 -- 172.21.15.67:0/3295587431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0440151360 msgr2=0x7f0440153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.192+0000 7f0437e00640 1 -- 172.21.15.67:0/3295587431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0440151360 msgr2=0x7f0440153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.192+0000 7f043f400640 1 -- 172.21.15.67:0/3295587431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0440151360 msgr2=0x7f0440153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.192+0000 7f043f400640 1 --2- 172.21.15.67:0/3295587431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0440151360 0x7f0440153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:36.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.192+0000 7f043f400640 1 --2- 172.21.15.67:0/3295587431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0440151360 0x7f0440153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:36.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.192+0000 7f0437e00640 1 --2- 172.21.15.67:0/3295587431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0440151360 0x7f0440153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:36.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.192+0000 7f0437e00640 1 -- 172.21.15.67:0/3295587431 >> v1:172.21.15.134:6789/0 conn(0x7f04400c1f70 legacy=0x7f0440150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:36.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.192+0000 7f0437e00640 1 -- 172.21.15.67:0/3295587431 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04400c3700 con 0x7f04400710a0 2024-03-09T09:04:36.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.193+0000 7f0437e00640 1 --2- 172.21.15.67:0/3295587431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04400710a0 0x7f0440153970 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0428009870 tx=0x7f042802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ef84b5a6289b90 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:36.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.193+0000 7f043e000640 1 -- 172.21.15.67:0/3295587431 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428035020 con 0x7f04400710a0 2024-03-09T09:04:36.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.193+0000 7f043e000640 1 -- 172.21.15.67:0/3295587431 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0428031640 con 0x7f04400710a0 2024-03-09T09:04:36.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.193+0000 7f043e000640 1 -- 172.21.15.67:0/3295587431 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042802d040 con 0x7f04400710a0 2024-03-09T09:04:36.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.194+0000 7f043fe00640 1 -- 172.21.15.67:0/3295587431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04400710a0 msgr2=0x7f0440153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:36.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.194+0000 7f043fe00640 1 --2- 172.21.15.67:0/3295587431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04400710a0 0x7f0440153970 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0428009870 tx=0x7f042802b8f0 comp rx=0 tx=0).stop 2024-03-09T09:04:36.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.194+0000 7f043fe00640 1 -- 172.21.15.67:0/3295587431 shutdown_connections 2024-03-09T09:04:36.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.194+0000 7f043fe00640 1 --2- 172.21.15.67:0/3295587431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0440151360 0x7f0440153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:36.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.194+0000 7f043fe00640 1 --2- 172.21.15.67:0/3295587431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04400710a0 0x7f0440153970 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:36.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.194+0000 7f043fe00640 1 -- 172.21.15.67:0/3295587431 >> 172.21.15.67:0/3295587431 conn(0x7f04400c7300 msgr2=0x7f04400c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:36.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.194+0000 7f043fe00640 1 -- 172.21.15.67:0/3295587431 shutdown_connections 2024-03-09T09:04:36.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.194+0000 7f043fe00640 1 -- 172.21.15.67:0/3295587431 wait complete. 2024-03-09T09:04:36.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.196+0000 7f043fe00640 1 Processor -- start 2024-03-09T09:04:36.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.196+0000 7f043fe00640 1 -- start start 2024-03-09T09:04:36.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f043fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04400710a0 0x7f04401df6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:36.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f043fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04400c1f70 0x7f044014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:36.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f043fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0440151360 0x7f0440148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:36.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f043fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0440070d50 con 0x7f04400710a0 2024-03-09T09:04:36.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f043fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0440070830 con 0x7f0440151360 2024-03-09T09:04:36.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f043fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0440070ed0 con 0x7f04400c1f70 2024-03-09T09:04:36.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f043ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04400710a0 0x7f04401df6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:36.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f043ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04400710a0 0x7f04401df6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34606/0 (socket says 172.21.15.67:34606) 2024-03-09T09:04:36.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f043ea00640 1 -- 172.21.15.67:0/3292031623 learned_addr learned my addr 172.21.15.67:0/3292031623 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:36.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f043f400640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0440151360 0x7f0440148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:36.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f0437e00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04400c1f70 0x7f044014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f0437e00640 1 -- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04400c1f70 msgr2=0x7f044014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f0437e00640 1 -- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04400c1f70 msgr2=0x7f044014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f0437e00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04400c1f70 0x7f044014ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.198+0000 7f0437e00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04400c1f70 0x7f044014ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.199+0000 7f043ea00640 1 -- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04400c1f70 msgr2=0x7f044014ab60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.199+0000 7f043ea00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04400c1f70 0x7f044014ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.199+0000 7f043ea00640 1 -- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0440151360 msgr2=0x7f0440148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.199+0000 7f043ea00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0440151360 0x7f0440148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.199+0000 7f043ea00640 1 -- 172.21.15.67:0/3292031623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04400ac7c0 con 0x7f04400710a0 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.199+0000 7f043ea00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04400710a0 0x7f04401df6d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f043000c430 tx=0x7f043000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.199+0000 7f0436a00640 1 -- 172.21.15.67:0/3292031623 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430017070 con 0x7f04400710a0 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.199+0000 7f0436a00640 1 -- 172.21.15.67:0/3292031623 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04300052d0 con 0x7f04400710a0 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.199+0000 7f0436a00640 1 -- 172.21.15.67:0/3292031623 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430003830 con 0x7f04400710a0 2024-03-09T09:04:36.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.199+0000 7f043fe00640 1 -- 172.21.15.67:0/3292031623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04400c67f0 con 0x7f04400710a0 2024-03-09T09:04:36.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.199+0000 7f043fe00640 1 -- 172.21.15.67:0/3292031623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04401ef230 con 0x7f04400710a0 2024-03-09T09:04:36.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.201+0000 7f043fe00640 1 -- 172.21.15.67:0/3292031623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044004f3c0 con 0x7f04400710a0 2024-03-09T09:04:36.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.202+0000 7f0436a00640 1 -- 172.21.15.67:0/3292031623 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f043001a070 con 0x7f04400710a0 2024-03-09T09:04:36.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.203+0000 7f0436a00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f041c06b4d0 0x7f041c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:36.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.203+0000 7f0436a00640 1 -- 172.21.15.67:0/3292031623 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f04300898d0 con 0x7f04400710a0 2024-03-09T09:04:36.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.203+0000 7f0437e00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f041c06b4d0 0x7f041c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:36.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.204+0000 7f0437e00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f041c06b4d0 0x7f041c06d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0428036000 tx=0x7f0428008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:36.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.207+0000 7f0436a00640 1 -- 172.21.15.67:0/3292031623 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0430057d00 con 0x7f04400710a0 2024-03-09T09:04:36.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.720+0000 7f043fe00640 1 -- 172.21.15.67:0/3292031623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f044006dfe0 con 0x7f04400710a0 2024-03-09T09:04:36.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.721+0000 7f0436a00640 1 -- 172.21.15.67:0/3292031623 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f043005bda0 con 0x7f04400710a0 2024-03-09T09:04:36.724 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":90,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:04:36.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.724+0000 7f043fe00640 1 -- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f041c06b4d0 msgr2=0x7f041c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:36.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.724+0000 7f043fe00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f041c06b4d0 0x7f041c06d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0428036000 tx=0x7f0428008030 comp rx=0 tx=0).stop 2024-03-09T09:04:36.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.724+0000 7f043fe00640 1 -- 172.21.15.67:0/3292031623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04400710a0 msgr2=0x7f04401df6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:36.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.724+0000 7f043fe00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04400710a0 0x7f04401df6d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f043000c430 tx=0x7f043000c8f0 comp rx=0 tx=0).stop 2024-03-09T09:04:36.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.725+0000 7f043fe00640 1 -- 172.21.15.67:0/3292031623 shutdown_connections 2024-03-09T09:04:36.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.725+0000 7f043fe00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f041c06b4d0 0x7f041c06d990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:36.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.725+0000 7f043fe00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0440151360 0x7f0440148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:36.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.725+0000 7f043fe00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04400c1f70 0x7f044014ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:36.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.725+0000 7f043fe00640 1 --2- 172.21.15.67:0/3292031623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04400710a0 0x7f04401df6d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:36.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.725+0000 7f043fe00640 1 -- 172.21.15.67:0/3292031623 >> 172.21.15.67:0/3292031623 conn(0x7f04400c7300 msgr2=0x7f04400c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:36.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.725+0000 7f043fe00640 1 -- 172.21.15.67:0/3292031623 shutdown_connections 2024-03-09T09:04:36.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:36.725+0000 7f043fe00640 1 -- 172.21.15.67:0/3292031623 wait complete. 2024-03-09T09:04:39.749 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:04:40.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.005+0000 7f9993c00640 1 Processor -- start 2024-03-09T09:04:40.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.005+0000 7f9993c00640 1 -- start start 2024-03-09T09:04:40.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.007+0000 7f9993c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f998c0c22a0 0x7f998c071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.007+0000 7f9993c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c157d60 0x7f998c0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.007+0000 7f9993c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998c153d40 con 0x7f998c0ac730 2024-03-09T09:04:40.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.007+0000 7f9993c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998c070ee0 con 0x7f998c0c22a0 2024-03-09T09:04:40.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.007+0000 7f9993c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c1581f0 con 0x7f998c157d60 2024-03-09T09:04:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.007+0000 7f9991c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c157d60 0x7f998c0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.007+0000 7f9991c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c157d60 0x7f998c0acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:36218/0 (socket says 172.21.15.67:36218) 2024-03-09T09:04:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.007+0000 7f9991c00640 1 -- 172.21.15.67:0/3402192393 learned_addr learned my addr 172.21.15.67:0/3402192393 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.008+0000 7f998be00640 1 --2- 172.21.15.67:0/3402192393 >> v2:172.21.15.134:3300/0 conn(0x7f998c0c22a0 0x7f998c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.008+0000 7f9991c00640 1 -- 172.21.15.67:0/3402192393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c157d60 msgr2=0x7f998c0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:04:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.008+0000 7f9991c00640 1 -- 172.21.15.67:0/3402192393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c157d60 msgr2=0x7f998c0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:04:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.008+0000 7f9991c00640 1 --2- 172.21.15.67:0/3402192393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c157d60 0x7f998c0acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.008+0000 7f9991c00640 1 --2- 172.21.15.67:0/3402192393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c157d60 0x7f998c0acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.008+0000 7f998be00640 1 -- 172.21.15.67:0/3402192393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c157d60 msgr2=0x7f998c0acb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:04:40.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.008+0000 7f998be00640 1 --2- 172.21.15.67:0/3402192393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c157d60 0x7f998c0acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.008+0000 7f998be00640 1 -- 172.21.15.67:0/3402192393 >> v1:172.21.15.134:6789/0 conn(0x7f998c0ac730 legacy=0x7f998c157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.008+0000 7f998be00640 1 -- 172.21.15.67:0/3402192393 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c0c2790 con 0x7f998c0c22a0 2024-03-09T09:04:40.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.008+0000 7f998b400640 1 -- 172.21.15.67:0/3402192393 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947757903 0 0) 0x7f998c153d40 con 0x7f998c0ac730 2024-03-09T09:04:40.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.010+0000 7f998be00640 1 --2- 172.21.15.67:0/3402192393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f998c0c22a0 0x7f998c071190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f99780097a0 tx=0x7f997802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=512d3ac01c24955d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.010+0000 7f998b400640 1 -- 172.21.15.67:0/3402192393 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99780315a0 con 0x7f998c0c22a0 2024-03-09T09:04:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.010+0000 7f998b400640 1 -- 172.21.15.67:0/3402192393 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9978037070 con 0x7f998c0c22a0 2024-03-09T09:04:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.010+0000 7f998b400640 1 -- 172.21.15.67:0/3402192393 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978031960 con 0x7f998c0c22a0 2024-03-09T09:04:40.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.011+0000 7f9993c00640 1 -- 172.21.15.67:0/3402192393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f998c0c22a0 msgr2=0x7f998c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.011+0000 7f9993c00640 1 --2- 172.21.15.67:0/3402192393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f998c0c22a0 0x7f998c071190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f99780097a0 tx=0x7f997802b840 comp rx=0 tx=0).stop 2024-03-09T09:04:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.011+0000 7f9993c00640 1 -- 172.21.15.67:0/3402192393 shutdown_connections 2024-03-09T09:04:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.011+0000 7f9993c00640 1 --2- 172.21.15.67:0/3402192393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c157d60 0x7f998c0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.011+0000 7f9993c00640 1 --2- 172.21.15.67:0/3402192393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f998c0c22a0 0x7f998c071190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.011+0000 7f9993c00640 1 -- 172.21.15.67:0/3402192393 >> 172.21.15.67:0/3402192393 conn(0x7f998c0c7370 msgr2=0x7f998c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.011+0000 7f9993c00640 1 -- 172.21.15.67:0/3402192393 shutdown_connections 2024-03-09T09:04:40.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.011+0000 7f9993c00640 1 -- 172.21.15.67:0/3402192393 wait complete. 2024-03-09T09:04:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.012+0000 7f9993c00640 1 Processor -- start 2024-03-09T09:04:40.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.013+0000 7f9993c00640 1 -- start start 2024-03-09T09:04:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.013+0000 7f9993c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c0ac730 0x7f998c1e27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.013+0000 7f9993c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f998c0c22a0 0x7f998c1e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f9993c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f998c157d60 0x7f998c1ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f9993c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c070c00 con 0x7f998c0c22a0 2024-03-09T09:04:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f9993c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c154d00 con 0x7f998c157d60 2024-03-09T09:04:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f9993c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c151610 con 0x7f998c0ac730 2024-03-09T09:04:40.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f998be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f998c0c22a0 0x7f998c1e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f9991200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c0ac730 0x7f998c1e27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f998be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f998c0c22a0 0x7f998c1e2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47512/0 (socket says 172.21.15.67:47512) 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f998be00640 1 -- 172.21.15.67:0/4231826050 learned_addr learned my addr 172.21.15.67:0/4231826050 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f9991c00640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f998c157d60 0x7f998c1ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f9991200640 1 -- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c0ac730 msgr2=0x7f998c1e27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f9991200640 1 -- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c0ac730 msgr2=0x7f998c1e27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f9991200640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c0ac730 0x7f998c1e27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.014+0000 7f9991200640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c0ac730 0x7f998c1e27d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.015+0000 7f998be00640 1 -- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c0ac730 msgr2=0x7f998c1e27d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.015+0000 7f998be00640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c0ac730 0x7f998c1e27d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.015+0000 7f998be00640 1 -- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f998c157d60 msgr2=0x7f998c1ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.015+0000 7f998be00640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f998c157d60 0x7f998c1ef3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.015+0000 7f998be00640 1 -- 172.21.15.67:0/4231826050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c14c6c0 con 0x7f998c0c22a0 2024-03-09T09:04:40.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.015+0000 7f998be00640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f998c0c22a0 0x7f998c1e2d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9978034040 tx=0x7f99780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.015+0000 7f9988c00640 1 -- 172.21.15.67:0/4231826050 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978004230 con 0x7f998c0c22a0 2024-03-09T09:04:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.015+0000 7f9988c00640 1 -- 172.21.15.67:0/4231826050 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99780043f0 con 0x7f998c0c22a0 2024-03-09T09:04:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.015+0000 7f9988c00640 1 -- 172.21.15.67:0/4231826050 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978003850 con 0x7f998c0c22a0 2024-03-09T09:04:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.015+0000 7f9993c00640 1 -- 172.21.15.67:0/4231826050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998c151d50 con 0x7f998c0c22a0 2024-03-09T09:04:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.016+0000 7f9993c00640 1 -- 172.21.15.67:0/4231826050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c152310 con 0x7f998c0c22a0 2024-03-09T09:04:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.017+0000 7f9993c00640 1 -- 172.21.15.67:0/4231826050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998c075a00 con 0x7f998c0c22a0 2024-03-09T09:04:40.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.018+0000 7f9988c00640 1 -- 172.21.15.67:0/4231826050 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f99780039f0 con 0x7f998c0c22a0 2024-03-09T09:04:40.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.018+0000 7f9988c00640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f996c06b4d0 0x7f996c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.019+0000 7f9988c00640 1 -- 172.21.15.67:0/4231826050 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f997802f030 con 0x7f998c0c22a0 2024-03-09T09:04:40.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.019+0000 7f9991200640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f996c06b4d0 0x7f996c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.020+0000 7f9991200640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f996c06b4d0 0x7f996c06d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9980000f60 tx=0x7f9980009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.026+0000 7f9988c00640 1 -- 172.21.15.67:0/4231826050 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9978041070 con 0x7f998c0c22a0 2024-03-09T09:04:40.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.359+0000 7f9993c00640 1 -- 172.21.15.67:0/4231826050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f998c1572a0 con 0x7f998c0c22a0 2024-03-09T09:04:40.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.360+0000 7f9988c00640 1 -- 172.21.15.67:0/4231826050 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7f9978044340 con 0x7f998c0c22a0 2024-03-09T09:04:40.362 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":92,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:04:40.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.362+0000 7f9993c00640 1 -- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f996c06b4d0 msgr2=0x7f996c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.362+0000 7f9993c00640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f996c06b4d0 0x7f996c06d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9980000f60 tx=0x7f9980009040 comp rx=0 tx=0).stop 2024-03-09T09:04:40.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.362+0000 7f9993c00640 1 -- 172.21.15.67:0/4231826050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f998c0c22a0 msgr2=0x7f998c1e2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.362+0000 7f9993c00640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f998c0c22a0 0x7f998c1e2d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9978034040 tx=0x7f99780046c0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.363+0000 7f9993c00640 1 -- 172.21.15.67:0/4231826050 shutdown_connections 2024-03-09T09:04:40.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.363+0000 7f9993c00640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f996c06b4d0 0x7f996c06d990 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.363+0000 7f9993c00640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f998c157d60 0x7f998c1ef3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.363+0000 7f9993c00640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f998c0c22a0 0x7f998c1e2d10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.363+0000 7f9993c00640 1 --2- 172.21.15.67:0/4231826050 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f998c0ac730 0x7f998c1e27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.363+0000 7f9993c00640 1 -- 172.21.15.67:0/4231826050 >> 172.21.15.67:0/4231826050 conn(0x7f998c0c7370 msgr2=0x7f998c072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:40.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.363+0000 7f9993c00640 1 -- 172.21.15.67:0/4231826050 shutdown_connections 2024-03-09T09:04:40.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.363+0000 7f9993c00640 1 -- 172.21.15.67:0/4231826050 wait complete. 2024-03-09T09:04:40.385 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:04:40.385 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:04:40.385 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:04:40.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.587+0000 7f48bfe00640 1 Processor -- start 2024-03-09T09:04:40.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.587+0000 7f48bfe00640 1 -- start start 2024-03-09T09:04:40.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.590+0000 7f48bfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f48c0150fb0 0x7f48c00c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.590+0000 7f48bfe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f48c0154410 0x7f48c006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.590+0000 7f48bfe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c00c51b0 con 0x7f48c00c6360 2024-03-09T09:04:40.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.590+0000 7f48bfe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c00c5aa0 con 0x7f48c0150fb0 2024-03-09T09:04:40.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.590+0000 7f48bfe00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c0151390 con 0x7f48c0154410 2024-03-09T09:04:40.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.590+0000 7f48bea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f48c0150fb0 0x7f48c00c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.590+0000 7f48bea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f48c0150fb0 0x7f48c00c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47524/0 (socket says 172.21.15.67:47524) 2024-03-09T09:04:40.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.590+0000 7f48bea00640 1 -- 172.21.15.67:0/198275409 learned_addr learned my addr 172.21.15.67:0/198275409 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:40.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.591+0000 7f48be000640 1 --2- 172.21.15.67:0/198275409 >> v2:172.21.15.134:3300/0 conn(0x7f48c0154410 0x7f48c006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.591+0000 7f48bea00640 1 -- 172.21.15.67:0/198275409 >> v2:172.21.15.134:3300/0 conn(0x7f48c0154410 msgr2=0x7f48c006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:40.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.591+0000 7f48bea00640 1 --2- 172.21.15.67:0/198275409 >> v2:172.21.15.134:3300/0 conn(0x7f48c0154410 0x7f48c006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.591+0000 7f48bea00640 1 -- 172.21.15.67:0/198275409 >> v1:172.21.15.67:6789/0 conn(0x7f48c00c6360 legacy=0x7f48c006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.591+0000 7f48bea00640 1 -- 172.21.15.67:0/198275409 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00c5450 con 0x7f48c0150fb0 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.592+0000 7f48bea00640 1 --2- 172.21.15.67:0/198275409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0150fb0 0x7f48c00c5e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48c006f530 tx=0x7f48b002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2f066569e089b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.592+0000 7f48bd600640 1 -- 172.21.15.67:0/198275409 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b002e070 con 0x7f48c0150fb0 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.592+0000 7f48bd600640 1 -- 172.21.15.67:0/198275409 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b0037070 con 0x7f48c0150fb0 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.592+0000 7f48bd600640 1 -- 172.21.15.67:0/198275409 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0031a00 con 0x7f48c0150fb0 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.593+0000 7f48bfe00640 1 -- 172.21.15.67:0/198275409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0150fb0 msgr2=0x7f48c00c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.593+0000 7f48bfe00640 1 --2- 172.21.15.67:0/198275409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0150fb0 0x7f48c00c5e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48c006f530 tx=0x7f48b002b880 comp rx=0 tx=0).stop 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.593+0000 7f48bfe00640 1 -- 172.21.15.67:0/198275409 shutdown_connections 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.593+0000 7f48bfe00640 1 --2- 172.21.15.67:0/198275409 >> v2:172.21.15.134:3300/0 conn(0x7f48c0154410 0x7f48c006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.593+0000 7f48bfe00640 1 --2- 172.21.15.67:0/198275409 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0150fb0 0x7f48c00c5e20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.593+0000 7f48bfe00640 1 -- 172.21.15.67:0/198275409 >> 172.21.15.67:0/198275409 conn(0x7f48c006f060 msgr2=0x7f48c0066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.593+0000 7f48bfe00640 1 -- 172.21.15.67:0/198275409 shutdown_connections 2024-03-09T09:04:40.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.593+0000 7f48bfe00640 1 -- 172.21.15.67:0/198275409 wait complete. 2024-03-09T09:04:40.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.595+0000 7f48bfe00640 1 Processor -- start 2024-03-09T09:04:40.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.595+0000 7f48bfe00640 1 -- start start 2024-03-09T09:04:40.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.597+0000 7f48bfe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48c00c6360 0x7f48c00adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.597+0000 7f48bfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48c0150fb0 0x7f48c014b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.597+0000 7f48bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154410 0x7f48c006c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.597+0000 7f48bfe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c01548f0 con 0x7f48c0154410 2024-03-09T09:04:40.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.597+0000 7f48bfe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0072a50 con 0x7f48c0150fb0 2024-03-09T09:04:40.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.597+0000 7f48bfe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00703e0 con 0x7f48c00c6360 2024-03-09T09:04:40.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.597+0000 7f48bf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154410 0x7f48c006c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.597+0000 7f48bf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154410 0x7f48c006c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47538/0 (socket says 172.21.15.67:47538) 2024-03-09T09:04:40.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.597+0000 7f48bf400640 1 -- 172.21.15.67:0/225040870 learned_addr learned my addr 172.21.15.67:0/225040870 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:40.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.597+0000 7f48bea00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48c00c6360 0x7f48c00adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.597+0000 7f48be000640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48c0150fb0 0x7f48c014b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.598+0000 7f48bf400640 1 -- 172.21.15.67:0/225040870 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48c00c6360 msgr2=0x7f48c00adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.598+0000 7f48bf400640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48c00c6360 0x7f48c00adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.598+0000 7f48bf400640 1 -- 172.21.15.67:0/225040870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48c0150fb0 msgr2=0x7f48c014b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.598+0000 7f48bf400640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48c0150fb0 0x7f48c014b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.598+0000 7f48bf400640 1 -- 172.21.15.67:0/225040870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c0145180 con 0x7f48c0154410 2024-03-09T09:04:40.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.598+0000 7f48bf400640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154410 0x7f48c006c150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f48b800c6e0 tx=0x7f48b800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.598+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8003b70 con 0x7f48c0154410 2024-03-09T09:04:40.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.598+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b8003d30 con 0x7f48c0154410 2024-03-09T09:04:40.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.598+0000 7f48bfe00640 1 -- 172.21.15.67:0/225040870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c0144ea0 con 0x7f48c0154410 2024-03-09T09:04:40.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.598+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8005310 con 0x7f48c0154410 2024-03-09T09:04:40.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.598+0000 7f48bfe00640 1 -- 172.21.15.67:0/225040870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c01ec540 con 0x7f48c0154410 2024-03-09T09:04:40.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.599+0000 7f48bfe00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154d80 0x7f48c006c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.599+0000 7f48bfe00640 1 -- 172.21.15.67:0/225040870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c004f3c0 con 0x7f48c0154d80 2024-03-09T09:04:40.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.600+0000 7f48bea00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154d80 0x7f48c006c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.600+0000 7f48bea00640 1 -- 172.21.15.67:0/225040870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48c0073790 con 0x7f48c0154d80 2024-03-09T09:04:40.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.600+0000 7f48bea00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154d80 0x7f48c006c690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f48c0067a60 tx=0x7f48b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.601+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b004a070 con 0x7f48c0154d80 2024-03-09T09:04:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.601+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f48b80054b0 con 0x7f48c0154410 2024-03-09T09:04:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.602+0000 7f48b6a00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f48a006b3d0 0x7f48a006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.602+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f48b808a400 con 0x7f48c0154410 2024-03-09T09:04:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.602+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f48b0037040 con 0x7f48c0154d80 2024-03-09T09:04:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.602+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154d80 msgr2=0x7f48c006c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.602+0000 7f48b6a00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154d80 0x7f48c006c690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f48c0067a60 tx=0x7f48b0004730 comp rx=0 tx=0).stop 2024-03-09T09:04:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.602+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154d80 msgr2=0x7f48c006c690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.602+0000 7f48b6a00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154d80 0x7f48c006c690 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f48c0067a60 tx=0x7f48b0004730 comp rx=0 tx=0).stop 2024-03-09T09:04:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.602+0000 7f48be000640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f48a006b3d0 0x7f48a006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.603+0000 7f48be000640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f48a006b3d0 0x7f48a006d890 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f48c014c480 tx=0x7f48ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.610+0000 7f48bfe00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c014b870 0x7f48c01eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.610+0000 7f48bfe00640 1 -- 172.21.15.67:0/225040870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0072cb0 con 0x7f48c014b870 2024-03-09T09:04:40.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.610+0000 7f48bea00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c014b870 0x7f48c01eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.611+0000 7f48bea00640 1 -- 172.21.15.67:0/225040870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f48c00c5fc0 con 0x7f48c014b870 2024-03-09T09:04:40.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.611+0000 7f48bea00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c014b870 0x7f48c01eecb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f48c0073b90 tx=0x7f48b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.611+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b004d8c0 con 0x7f48c014b870 2024-03-09T09:04:40.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.611+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f48b0003e20 con 0x7f48c014b870 2024-03-09T09:04:40.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.611+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c014b870 msgr2=0x7f48c01eecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.611+0000 7f48b6a00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c014b870 0x7f48c01eecb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f48c0073b90 tx=0x7f48b00046c0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.611+0000 7f48b6a00640 1 -- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c014b870 msgr2=0x7f48c01eecb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:40.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.611+0000 7f48b6a00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c014b870 0x7f48c01eecb0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f48c0073b90 tx=0x7f48b00046c0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.615 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:04:40.615 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:04:40.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 92, 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:40.616 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:40.617 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:04:40.618 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:40.619 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:04:40.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:04:40.621 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.622 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.622 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.622 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:04:40.622 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:04:40.622 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:40.622 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:40.622 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:40.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.623 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.624 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:40.625 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:04:40.626 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:04:40.627 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.628 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:04:40.628 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:40.628 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:40.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.628 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.628 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:40.628 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:40.629 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:04:40.629 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.629 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.629 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:04:40.629 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.629 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:40.629 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:40.629 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:40.629 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.629 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.630 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:04:40.630 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:04:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.612+0000 7f48bfe00640 1 -- 172.21.15.67:0/225040870 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f48a006b3d0 msgr2=0x7f48a006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.612+0000 7f48bfe00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f48a006b3d0 0x7f48a006d890 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f48c014c480 tx=0x7f48ac008040 comp rx=0 tx=0).stop 2024-03-09T09:04:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.612+0000 7f48bfe00640 1 -- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154410 msgr2=0x7f48c006c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.612+0000 7f48bfe00640 1 --2- 172.21.15.67:0/225040870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48c0154410 0x7f48c006c150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f48b800c6e0 tx=0x7f48b800cba0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.612+0000 7f48bf400640 1 -- 172.21.15.67:0/225040870 reap_dead start 2024-03-09T09:04:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.612+0000 7f48bf400640 1 -- 172.21.15.67:0/225040870 reap_dead start 2024-03-09T09:04:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.612+0000 7f48bfe00640 1 -- 172.21.15.67:0/225040870 shutdown_connections 2024-03-09T09:04:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.612+0000 7f48bfe00640 1 -- 172.21.15.67:0/225040870 >> 172.21.15.67:0/225040870 conn(0x7f48c006f060 msgr2=0x7f48c0066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.612+0000 7f48bfe00640 1 -- 172.21.15.67:0/225040870 shutdown_connections 2024-03-09T09:04:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.612+0000 7f48bfe00640 1 -- 172.21.15.67:0/225040870 wait complete. 2024-03-09T09:04:40.637 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:04:40.637 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:04:40.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:04:40.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.830+0000 7fd866400640 1 Processor -- start 2024-03-09T09:04:40.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.830+0000 7fd866400640 1 -- start start 2024-03-09T09:04:40.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.832+0000 7fd866400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 0x7fd860154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.832+0000 7fd866400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd860071190 0x7fd8601546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.832+0000 7fd866400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8600c8200 con 0x7fd8600ac9a0 2024-03-09T09:04:40.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.832+0000 7fd866400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86006d590 con 0x7fd860071190 2024-03-09T09:04:40.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.832+0000 7fd866400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600c3740 con 0x7fd8600c2780 2024-03-09T09:04:40.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.832+0000 7fd857e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 0x7fd860154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.832+0000 7fd857e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 0x7fd860154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:36246/0 (socket says 172.21.15.67:36246) 2024-03-09T09:04:40.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.832+0000 7fd857e00640 1 -- 172.21.15.67:0/665087648 learned_addr learned my addr 172.21.15.67:0/665087648 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:40.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.832+0000 7fd85fe00640 1 --2- 172.21.15.67:0/665087648 >> v2:172.21.15.134:3300/0 conn(0x7fd860071190 0x7fd8601546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.833+0000 7fd857e00640 1 -- 172.21.15.67:0/665087648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 msgr2=0x7fd860154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.833+0000 7fd857e00640 1 -- 172.21.15.67:0/665087648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 msgr2=0x7fd860154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.833+0000 7fd857e00640 1 --2- 172.21.15.67:0/665087648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 0x7fd860154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.833+0000 7fd857e00640 1 --2- 172.21.15.67:0/665087648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 0x7fd860154190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.833+0000 7fd85fe00640 1 -- 172.21.15.67:0/665087648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 msgr2=0x7fd860154190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.833+0000 7fd85fe00640 1 --2- 172.21.15.67:0/665087648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 0x7fd860154190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.833+0000 7fd85fe00640 1 -- 172.21.15.67:0/665087648 >> v1:172.21.15.134:6789/0 conn(0x7fd8600ac9a0 legacy=0x7fd8601510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.833+0000 7fd85fe00640 1 -- 172.21.15.67:0/665087648 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600c3990 con 0x7fd860071190 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.835+0000 7fd85fe00640 1 --2- 172.21.15.67:0/665087648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860071190 0x7fd8601546d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd858009d90 tx=0x7fd85802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1681989052886402 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.835+0000 7fd85ea00640 1 -- 172.21.15.67:0/665087648 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858039070 con 0x7fd860071190 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.835+0000 7fd85ea00640 1 -- 172.21.15.67:0/665087648 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85802cbc0 con 0x7fd860071190 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.835+0000 7fd85ea00640 1 -- 172.21.15.67:0/665087648 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858034440 con 0x7fd860071190 2024-03-09T09:04:40.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.835+0000 7fd866400640 1 -- 172.21.15.67:0/665087648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860071190 msgr2=0x7fd8601546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.835+0000 7fd866400640 1 --2- 172.21.15.67:0/665087648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860071190 0x7fd8601546d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd858009d90 tx=0x7fd85802be10 comp rx=0 tx=0).stop 2024-03-09T09:04:40.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.836+0000 7fd866400640 1 -- 172.21.15.67:0/665087648 shutdown_connections 2024-03-09T09:04:40.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.836+0000 7fd866400640 1 --2- 172.21.15.67:0/665087648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860071190 0x7fd8601546d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.836+0000 7fd866400640 1 --2- 172.21.15.67:0/665087648 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 0x7fd860154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.836+0000 7fd866400640 1 -- 172.21.15.67:0/665087648 >> 172.21.15.67:0/665087648 conn(0x7fd8600c7370 msgr2=0x7fd8600c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:40.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.836+0000 7fd866400640 1 -- 172.21.15.67:0/665087648 shutdown_connections 2024-03-09T09:04:40.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.836+0000 7fd866400640 1 -- 172.21.15.67:0/665087648 wait complete. 2024-03-09T09:04:40.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.837+0000 7fd866400640 1 Processor -- start 2024-03-09T09:04:40.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.838+0000 7fd866400640 1 -- start start 2024-03-09T09:04:40.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.838+0000 7fd866400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860071190 0x7fd8600ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.838+0000 7fd866400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8600ac9a0 0x7fd8600c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.838+0000 7fd866400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 0x7fd860149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.838+0000 7fd866400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd860154c10 con 0x7fd8600ac9a0 2024-03-09T09:04:40.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.838+0000 7fd866400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd860070c00 con 0x7fd860071190 2024-03-09T09:04:40.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.838+0000 7fd866400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600acd80 con 0x7fd8600c2780 2024-03-09T09:04:40.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.839+0000 7fd857e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8600ac9a0 0x7fd8600c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.839+0000 7fd857e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8600ac9a0 0x7fd8600c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47562/0 (socket says 172.21.15.67:47562) 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.839+0000 7fd857e00640 1 -- 172.21.15.67:0/4157406262 learned_addr learned my addr 172.21.15.67:0/4157406262 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.839+0000 7fd85f400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860071190 0x7fd8600ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.839+0000 7fd85fe00640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 0x7fd860149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.839+0000 7fd857e00640 1 -- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 msgr2=0x7fd860149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.839+0000 7fd857e00640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8600c2780 0x7fd860149300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.839+0000 7fd857e00640 1 -- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860071190 msgr2=0x7fd8600ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.839+0000 7fd857e00640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860071190 0x7fd8600ab7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.839+0000 7fd857e00640 1 -- 172.21.15.67:0/4157406262 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600c5310 con 0x7fd8600ac9a0 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.840+0000 7fd857e00640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8600ac9a0 0x7fd8600c65d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd84800e810 tx=0x7fd84800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.840+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848017070 con 0x7fd8600ac9a0 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.840+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd848004020 con 0x7fd8600ac9a0 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.840+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84800f040 con 0x7fd8600ac9a0 2024-03-09T09:04:40.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.840+0000 7fd866400640 1 -- 172.21.15.67:0/4157406262 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8600c55f0 con 0x7fd8600ac9a0 2024-03-09T09:04:40.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.840+0000 7fd866400640 1 -- 172.21.15.67:0/4157406262 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8600c5bb0 con 0x7fd8600ac9a0 2024-03-09T09:04:40.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.842+0000 7fd866400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149840 0x7fd8601ebc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.842+0000 7fd866400640 1 -- 172.21.15.67:0/4157406262 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd860071800 con 0x7fd860149840 2024-03-09T09:04:40.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.842+0000 7fd85f400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149840 0x7fd8601ebc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.843+0000 7fd85f400640 1 -- 172.21.15.67:0/4157406262 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8600c5db0 con 0x7fd860149840 2024-03-09T09:04:40.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.843+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd84800c6d0 con 0x7fd8600ac9a0 2024-03-09T09:04:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.843+0000 7fd85f400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149840 0x7fd8601ebc80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd860073660 tx=0x7fd85000e8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.844+0000 7fd857400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd83c06b4d0 0x7fd83c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.844+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd84808a0e0 con 0x7fd8600ac9a0 2024-03-09T09:04:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.845+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850019070 con 0x7fd860149840 2024-03-09T09:04:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.845+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd8500052e0 con 0x7fd860149840 2024-03-09T09:04:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.845+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149840 msgr2=0x7fd8601ebc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.845+0000 7fd857400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149840 0x7fd8601ebc80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd860073660 tx=0x7fd85000e8b0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.845+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149840 msgr2=0x7fd8601ebc80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.845+0000 7fd857400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149840 0x7fd8601ebc80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.845+0000 7fd85fe00640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd83c06b4d0 0x7fd83c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.846+0000 7fd85fe00640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd83c06b4d0 0x7fd83c06d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd858036000 tx=0x7fd858030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.853+0000 7fd866400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149c20 0x7fd8601ec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:40.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.853+0000 7fd866400640 1 -- 172.21.15.67:0/4157406262 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd860075a00 con 0x7fd860149c20 2024-03-09T09:04:40.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.853+0000 7fd85f400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149c20 0x7fd8601ec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:40.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.854+0000 7fd85f400640 1 -- 172.21.15.67:0/4157406262 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd8600c4440 con 0x7fd860149c20 2024-03-09T09:04:40.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.854+0000 7fd85f400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149c20 0x7fd8601ec1c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd860073660 tx=0x7fd850000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:40.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.855+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850019040 con 0x7fd860149c20 2024-03-09T09:04:40.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5016 (secure 0 0 0) 0x7fd8500035f0 con 0x7fd860149c20 2024-03-09T09:04:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149c20 msgr2=0x7fd8601ec1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd857400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149c20 0x7fd8601ec1c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd860073660 tx=0x7fd850000ed0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd857400640 1 -- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149c20 msgr2=0x7fd8601ec1c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd857400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860149c20 0x7fd8601ec1c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.859 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:04:40.859 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:04:40.859 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:04:40.859 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:04:40.859 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 92, 2024-03-09T09:04:40.860 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:04:40.860 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:04:40.860 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:04:40.860 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:04:40.860 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.860 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:04:40.860 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:40.860 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:04:40.860 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.861 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:04:40.862 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:04:40.862 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:40.862 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:40.862 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:40.862 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:40.862 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:40.862 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:40.862 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:40.862 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:40.862 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:40.863 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:40.864 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:40.865 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:40.865 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:40.865 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:40.865 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:40.865 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:40.865 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:40.865 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.865 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:04:40.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.865 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.866 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.868 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:04:40.868 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:04:40.868 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:40.868 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:40.868 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.868 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:40.868 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:04:40.868 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.868 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.868 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.869 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:04:40.870 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:04:40.870 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:40.870 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:40.870 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.870 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:40.870 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:04:40.870 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.870 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.871 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:40.871 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:04:40.871 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:40.871 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.871 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.871 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:04:40.871 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:04:40.871 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:40.872 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:40.872 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:40.872 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.872 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.872 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:04:40.872 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:04:40.872 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.872 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:40.872 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:04:40.873 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:40.874 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:40.875 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:40.875 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:40.875 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:40.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:40.875 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:04:40.875 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:04:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd866400640 1 -- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd83c06b4d0 msgr2=0x7fd83c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd866400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd83c06b4d0 0x7fd83c06d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd858036000 tx=0x7fd858030000 comp rx=0 tx=0).stop 2024-03-09T09:04:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd866400640 1 -- 172.21.15.67:0/4157406262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8600ac9a0 msgr2=0x7fd8600c65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd866400640 1 --2- 172.21.15.67:0/4157406262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8600ac9a0 0x7fd8600c65d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd84800e810 tx=0x7fd84800ecd0 comp rx=0 tx=0).stop 2024-03-09T09:04:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd85fe00640 1 -- 172.21.15.67:0/4157406262 reap_dead start 2024-03-09T09:04:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd85fe00640 1 -- 172.21.15.67:0/4157406262 reap_dead start 2024-03-09T09:04:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd866400640 1 -- 172.21.15.67:0/4157406262 shutdown_connections 2024-03-09T09:04:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.856+0000 7fd866400640 1 -- 172.21.15.67:0/4157406262 >> 172.21.15.67:0/4157406262 conn(0x7fd8600c7370 msgr2=0x7fd860154190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.857+0000 7fd866400640 1 -- 172.21.15.67:0/4157406262 shutdown_connections 2024-03-09T09:04:40.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:40.857+0000 7fd866400640 1 -- 172.21.15.67:0/4157406262 wait complete. 2024-03-09T09:04:40.877 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:04:40.877 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:04:40.877 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:04:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.067+0000 7f6ff7200640 1 Processor -- start 2024-03-09T09:04:41.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.067+0000 7f6ff7200640 1 -- start start 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.070+0000 7f6ff7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff814d930 0x7f6ff81542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.070+0000 7f6ff7200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6ff80c4790 0x7f6ff806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.070+0000 7f6ff7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6ff806b580 0x7f6ff8154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.070+0000 7f6ff7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff8157c80 con 0x7f6ff806b580 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.070+0000 7f6ff7200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff8156e20 con 0x7f6ff80c4790 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.070+0000 7f6ff7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff814dd10 con 0x7f6ff814d930 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.070+0000 7f6ff6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6ff806b580 0x7f6ff8154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.071+0000 7f6ff6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6ff806b580 0x7f6ff8154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47570/0 (socket says 172.21.15.67:47570) 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.071+0000 7f6ff6800640 1 -- 172.21.15.67:0/4117161547 learned_addr learned my addr 172.21.15.67:0/4117161547 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.071+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4117161547 >> v2:172.21.15.134:3300/0 conn(0x7f6ff80c4790 0x7f6ff806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.071+0000 7f6ff5e00640 1 --2- 172.21.15.67:0/4117161547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff814d930 0x7f6ff81542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:41.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.071+0000 7f6ff6800640 1 -- 172.21.15.67:0/4117161547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff814d930 msgr2=0x7f6ff81542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.071+0000 7f6ff6800640 1 --2- 172.21.15.67:0/4117161547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff814d930 0x7f6ff81542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.071+0000 7f6ff6800640 1 -- 172.21.15.67:0/4117161547 >> v2:172.21.15.134:3300/0 conn(0x7f6ff80c4790 msgr2=0x7f6ff806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.071+0000 7f6ff6800640 1 --2- 172.21.15.67:0/4117161547 >> v2:172.21.15.134:3300/0 conn(0x7f6ff80c4790 0x7f6ff806b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:41.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.071+0000 7f6ff6800640 1 -- 172.21.15.67:0/4117161547 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff81547f0 con 0x7f6ff806b580 2024-03-09T09:04:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.072+0000 7f6ff6800640 1 --2- 172.21.15.67:0/4117161547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff806b580 0x7f6ff8154ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ff814dec0 tx=0x7f6ff002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ede25bad9865cd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.072+0000 7f6ff4a00640 1 -- 172.21.15.67:0/4117161547 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff002c9e0 con 0x7f6ff806b580 2024-03-09T09:04:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.073+0000 7f6ff4a00640 1 -- 172.21.15.67:0/4117161547 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff002cb80 con 0x7f6ff806b580 2024-03-09T09:04:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.073+0000 7f6ff4a00640 1 -- 172.21.15.67:0/4117161547 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00343f0 con 0x7f6ff806b580 2024-03-09T09:04:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.073+0000 7f6ff7200640 1 -- 172.21.15.67:0/4117161547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff806b580 msgr2=0x7f6ff8154ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.073+0000 7f6ff7200640 1 --2- 172.21.15.67:0/4117161547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff806b580 0x7f6ff8154ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ff814dec0 tx=0x7f6ff002be40 comp rx=0 tx=0).stop 2024-03-09T09:04:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.073+0000 7f6ff7200640 1 -- 172.21.15.67:0/4117161547 shutdown_connections 2024-03-09T09:04:41.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.073+0000 7f6ff7200640 1 --2- 172.21.15.67:0/4117161547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff806b580 0x7f6ff8154ed0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:41.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.073+0000 7f6ff7200640 1 --2- 172.21.15.67:0/4117161547 >> v2:172.21.15.134:3300/0 conn(0x7f6ff80c4790 0x7f6ff806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:41.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.073+0000 7f6ff7200640 1 --2- 172.21.15.67:0/4117161547 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff814d930 0x7f6ff81542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:41.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.073+0000 7f6ff7200640 1 -- 172.21.15.67:0/4117161547 >> 172.21.15.67:0/4117161547 conn(0x7f6ff80c8480 msgr2=0x7f6ff80c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:41.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.073+0000 7f6ff7200640 1 -- 172.21.15.67:0/4117161547 shutdown_connections 2024-03-09T09:04:41.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.073+0000 7f6ff7200640 1 -- 172.21.15.67:0/4117161547 wait complete. 2024-03-09T09:04:41.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.075+0000 7f6ff7200640 1 Processor -- start 2024-03-09T09:04:41.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.075+0000 7f6ff7200640 1 -- start start 2024-03-09T09:04:41.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.076+0000 7f6ff7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff806b580 0x7f6ff8146970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.076+0000 7f6ff7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff80c4790 0x7f6ff8148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.076+0000 7f6ff7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff814d930 0x7f6ff8153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.076+0000 7f6ff7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff8157630 con 0x7f6ff806b580 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.076+0000 7f6ff7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff8157350 con 0x7f6ff80c4790 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.076+0000 7f6ff7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff8157910 con 0x7f6ff814d930 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.076+0000 7f6ff5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff806b580 0x7f6ff8146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.076+0000 7f6ff5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff806b580 0x7f6ff8146970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47584/0 (socket says 172.21.15.67:47584) 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.076+0000 7f6ff5e00640 1 -- 172.21.15.67:0/4210569434 learned_addr learned my addr 172.21.15.67:0/4210569434 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.077+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff80c4790 0x7f6ff8148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.077+0000 7f6ff5e00640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff814d930 msgr2=0x7f6ff8153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.077+0000 7f6ff5e00640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff814d930 0x7f6ff8153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.077+0000 7f6ff5e00640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff80c4790 msgr2=0x7f6ff8148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:41.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.077+0000 7f6ff5e00640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff80c4790 0x7f6ff8148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:41.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.077+0000 7f6ff5e00640 1 -- 172.21.15.67:0/4210569434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff806f6b0 con 0x7f6ff806b580 2024-03-09T09:04:41.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.077+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff80c4790 0x7f6ff8148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:04:41.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.077+0000 7f6ff5e00640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff806b580 0x7f6ff8146970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800d830 tx=0x7f6fe800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:41.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.077+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe800b6c0 con 0x7f6ff806b580 2024-03-09T09:04:41.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.078+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe8019070 con 0x7f6ff806b580 2024-03-09T09:04:41.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.078+0000 7f6ff7200640 1 -- 172.21.15.67:0/4210569434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff806f990 con 0x7f6ff806b580 2024-03-09T09:04:41.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.078+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe800ba80 con 0x7f6ff806b580 2024-03-09T09:04:41.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.078+0000 7f6ff7200640 1 -- 172.21.15.67:0/4210569434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff81536f0 con 0x7f6ff806b580 2024-03-09T09:04:41.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.079+0000 7f6ff7200640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:41.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.079+0000 7f6ff7200640 1 -- 172.21.15.67:0/4210569434 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff804f3c0 con 0x7f6ff806f1f0 2024-03-09T09:04:41.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.079+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:41.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.080+0000 7f6ff5400640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 msgr2=0x7f6ff80c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-03-09T09:04:41.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.080+0000 7f6ff5400640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 msgr2=0x7f6ff80c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:04:41.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.080+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:41.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.080+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:41.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.081+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6fe801e460 con 0x7f6ff806b580 2024-03-09T09:04:41.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.082+0000 7f6fee000640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fd806b4d0 0x7f6fd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:41.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.082+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6fe808ad40 con 0x7f6ff806b580 2024-03-09T09:04:41.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.083+0000 7f6ff6800640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fd806b4d0 0x7f6fd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:41.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.083+0000 7f6ff6800640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fd806b4d0 0x7f6fd806d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0035000 tx=0x7f6ff002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:41.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.281+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:41.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.282+0000 7f6ff5400640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 msgr2=0x7f6ff80c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:04:41.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.282+0000 7f6ff5400640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 msgr2=0x7f6ff80c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:04:41.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.282+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:41.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.282+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:04:41.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.683+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:41.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.684+0000 7f6ff5400640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 msgr2=0x7f6ff80c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:04:41.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.684+0000 7f6ff5400640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 msgr2=0x7f6ff80c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:04:41.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.684+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:41.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:41.684+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:04:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:42.485+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:42.486+0000 7f6ff5400640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 msgr2=0x7f6ff80c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:04:42.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:42.486+0000 7f6ff5400640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 msgr2=0x7f6ff80c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:04:42.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:42.486+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:04:42.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:42.486+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:04:44.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.089+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:44.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.090+0000 7f6ff5400640 1 -- 172.21.15.67:0/4210569434 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ff80c4490 con 0x7f6ff806f1f0 2024-03-09T09:04:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.091+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4006fd0 tx=0x7f6fe4016da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:44.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.092+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400f040 con 0x7f6ff806f1f0 2024-03-09T09:04:44.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.093+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f6fe4012d00 con 0x7f6ff806f1f0 2024-03-09T09:04:44.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.093+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 msgr2=0x7f6ff80c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:44.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.093+0000 7f6fee000640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4006fd0 tx=0x7f6fe4016da0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.093+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 msgr2=0x7f6ff80c6ee0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:44.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.093+0000 7f6fee000640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff806f1f0 0x7f6ff80c6ee0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.103+0000 7f6ff7200640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff81538c0 0x7f6ff80c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:44.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.103+0000 7f6ff7200640 1 -- 172.21.15.67:0/4210569434 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff80c33f0 con 0x7f6ff81538c0 2024-03-09T09:04:44.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.103+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff81538c0 0x7f6ff80c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:44.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.104+0000 7f6ff5400640 1 -- 172.21.15.67:0/4210569434 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6ff8155980 con 0x7f6ff81538c0 2024-03-09T09:04:44.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.105+0000 7f6ff5400640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff81538c0 0x7f6ff80c7420 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4006fd0 tx=0x7f6fe40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:44.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.105+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4027070 con 0x7f6ff81538c0 2024-03-09T09:04:44.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.106+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4533 (secure 0 0 0) 0x7f6fe4027570 con 0x7f6ff81538c0 2024-03-09T09:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 92, 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 6, 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:44.122 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:04:44.122 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.106+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff81538c0 msgr2=0x7f6ff80c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.106+0000 7f6fee000640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff81538c0 0x7f6ff80c7420 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4006fd0 tx=0x7f6fe40046c0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.106+0000 7f6fee000640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff81538c0 msgr2=0x7f6ff80c7420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.106+0000 7f6fee000640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff81538c0 0x7f6ff80c7420 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4006fd0 tx=0x7f6fe40046c0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.107+0000 7f6ff7200640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fd806b4d0 msgr2=0x7f6fd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.107+0000 7f6ff7200640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fd806b4d0 0x7f6fd806d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0035000 tx=0x7f6ff002e000 comp rx=0 tx=0).stop 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.107+0000 7f6ff7200640 1 -- 172.21.15.67:0/4210569434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff806b580 msgr2=0x7f6ff8146970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.107+0000 7f6ff7200640 1 --2- 172.21.15.67:0/4210569434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ff806b580 0x7f6ff8146970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800d830 tx=0x7f6fe800dcf0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.107+0000 7f6ff6800640 1 -- 172.21.15.67:0/4210569434 reap_dead start 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.107+0000 7f6ff6800640 1 -- 172.21.15.67:0/4210569434 reap_dead start 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.108+0000 7f6ff7200640 1 -- 172.21.15.67:0/4210569434 shutdown_connections 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.108+0000 7f6ff7200640 1 -- 172.21.15.67:0/4210569434 >> 172.21.15.67:0/4210569434 conn(0x7f6ff80c8480 msgr2=0x7f6ff8154ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.108+0000 7f6ff7200640 1 -- 172.21.15.67:0/4210569434 shutdown_connections 2024-03-09T09:04:44.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.108+0000 7f6ff7200640 1 -- 172.21.15.67:0/4210569434 wait complete. 2024-03-09T09:04:44.130 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:04:44.130 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:04:44.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:04:44.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.348+0000 7f6ff5000640 1 Processor -- start 2024-03-09T09:04:44.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.348+0000 7f6ff5000640 1 -- start start 2024-03-09T09:04:44.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.349+0000 7f6ff5000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6ff00c22a0 0x7f6ff0071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:44.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.349+0000 7f6ff5000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff0157d60 0x7f6ff00acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:44.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.349+0000 7f6ff5000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff0153d40 con 0x7f6ff00ac730 2024-03-09T09:04:44.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.349+0000 7f6ff5000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff0070ee0 con 0x7f6ff00c22a0 2024-03-09T09:04:44.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.349+0000 7f6ff5000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff01581f0 con 0x7f6ff0157d60 2024-03-09T09:04:44.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.349+0000 7f6fede00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6ff00ac730 0x7f6ff0157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60918/0 (socket says 172.21.15.67:60918) 2024-03-09T09:04:44.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.349+0000 7f6fede00640 1 -- 172.21.15.67:0/1072280498 learned_addr learned my addr 172.21.15.67:0/1072280498 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:04:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6fee800640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff0157d60 0x7f6ff00acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6fed400640 1 --2- 172.21.15.67:0/1072280498 >> v2:172.21.15.134:3300/0 conn(0x7f6ff00c22a0 0x7f6ff0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:44.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928477545 0 0) 0x7f6ff0153d40 con 0x7f6ff00ac730 2024-03-09T09:04:44.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd4003520 con 0x7f6ff00ac730 2024-03-09T09:04:44.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1730735636 0 0) 0x7f6fd4003520 con 0x7f6ff00ac730 2024-03-09T09:04:44.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff0157d60 msgr2=0x7f6ff00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:44.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff0157d60 0x7f6ff00acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 >> v2:172.21.15.134:3300/0 conn(0x7f6ff00c22a0 msgr2=0x7f6ff0071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:44.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 --2- 172.21.15.67:0/1072280498 >> v2:172.21.15.134:3300/0 conn(0x7f6ff00c22a0 0x7f6ff0071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff00c2790 con 0x7f6ff00ac730 2024-03-09T09:04:44.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f6fe0001690 con 0x7f6ff00ac730 2024-03-09T09:04:44.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 >> v1:172.21.15.67:6789/0 conn(0x7f6ff00ac730 legacy=0x7f6ff0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 0x7f6fd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd4004800 0x7f6fd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.350+0000 7f6feca00640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff0157d60 0x7f6fd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.351+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0070ee0 con 0x7f6fd40039b0 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.351+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff01581f0 con 0x7f6fd4004800 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.351+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd400b9c0 con 0x7f6ff0157d60 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.351+0000 7f6fede00640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 0x7f6fd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.351+0000 7f6fee800640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff0157d60 0x7f6fd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.351+0000 7f6fede00640 1 -- 172.21.15.67:0/1072280498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff0157d60 msgr2=0x7f6fd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.351+0000 7f6fede00640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff0157d60 0x7f6fd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.351+0000 7f6fede00640 1 -- 172.21.15.67:0/1072280498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd4004800 msgr2=0x7f6fd4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.351+0000 7f6fede00640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fd4004800 0x7f6fd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.351+0000 7f6fede00640 1 -- 172.21.15.67:0/1072280498 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd4003520 con 0x7f6fd40039b0 2024-03-09T09:04:44.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.351+0000 7f6fee800640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff0157d60 0x7f6fd40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:04:44.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.352+0000 7f6fede00640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 0x7f6fd4003dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000c240 tx=0x7f6fd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac0f12f7734771b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:44.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.353+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0007d60 con 0x7f6fd40039b0 2024-03-09T09:04:44.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.353+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0030070 con 0x7f6fd40039b0 2024-03-09T09:04:44.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.353+0000 7f6feca00640 1 -- 172.21.15.67:0/1072280498 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe002cd20 con 0x7f6fd40039b0 2024-03-09T09:04:44.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.353+0000 7f6ff5000640 1 -- 172.21.15.67:0/1072280498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 msgr2=0x7f6fd4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:44.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.353+0000 7f6ff5000640 1 --2- 172.21.15.67:0/1072280498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 0x7f6fd4003dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000c240 tx=0x7f6fd4005830 comp rx=0 tx=0).stop 2024-03-09T09:04:44.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.353+0000 7f6fee800640 1 -- 172.21.15.67:0/1072280498 reap_dead start 2024-03-09T09:04:44.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.353+0000 7f6ff5000640 1 -- 172.21.15.67:0/1072280498 shutdown_connections 2024-03-09T09:04:44.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.353+0000 7f6ff5000640 1 -- 172.21.15.67:0/1072280498 >> 172.21.15.67:0/1072280498 conn(0x7f6ff00c7370 msgr2=0x7f6ff00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.353+0000 7f6ff5000640 1 -- 172.21.15.67:0/1072280498 shutdown_connections 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.353+0000 7f6ff5000640 1 -- 172.21.15.67:0/1072280498 wait complete. 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.355+0000 7f6ff5000640 1 Processor -- start 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.355+0000 7f6ff5000640 1 -- start start 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.356+0000 7f6ff5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 0x7f6ff014dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.356+0000 7f6ff5000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff00abea0 0x7f6ff00aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.356+0000 7f6ff5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff00aaa60 0x7f6ff01ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.356+0000 7f6ff5000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0070c00 con 0x7f6fd40039b0 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.356+0000 7f6ff5000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0154d00 con 0x7f6ff00aaa60 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.356+0000 7f6ff5000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0151610 con 0x7f6ff00abea0 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.356+0000 7f6fede00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 0x7f6ff014dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.356+0000 7f6fede00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 0x7f6ff014dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47588/0 (socket says 172.21.15.67:47588) 2024-03-09T09:04:44.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.356+0000 7f6fede00640 1 -- 172.21.15.67:0/3524254216 learned_addr learned my addr 172.21.15.67:0/3524254216 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.357+0000 7f6fed400640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff00abea0 0x7f6ff00aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.357+0000 7f6fee800640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff00aaa60 0x7f6ff01ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.357+0000 7f6fede00640 1 -- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff00abea0 msgr2=0x7f6ff00aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.357+0000 7f6fede00640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff00abea0 0x7f6ff00aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.357+0000 7f6fede00640 1 -- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff00aaa60 msgr2=0x7f6ff01ee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.357+0000 7f6fede00640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff00aaa60 0x7f6ff01ee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.357+0000 7f6fede00640 1 -- 172.21.15.67:0/3524254216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff00ab660 con 0x7f6fd40039b0 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.357+0000 7f6fede00640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 0x7f6ff014dbb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6fe002ddf0 tx=0x7f6ff00c2960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.357+0000 7f6fe6000640 1 -- 172.21.15.67:0/3524254216 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0001ca0 con 0x7f6fd40039b0 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.357+0000 7f6fe6000640 1 -- 172.21.15.67:0/3524254216 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0004bf0 con 0x7f6fd40039b0 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.358+0000 7f6fe6000640 1 -- 172.21.15.67:0/3524254216 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0007070 con 0x7f6fd40039b0 2024-03-09T09:04:44.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.358+0000 7f6ff5000640 1 -- 172.21.15.67:0/3524254216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff009be10 con 0x7f6fd40039b0 2024-03-09T09:04:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.358+0000 7f6ff5000640 1 -- 172.21.15.67:0/3524254216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff00ab380 con 0x7f6fd40039b0 2024-03-09T09:04:44.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.359+0000 7f6ff5000640 1 -- 172.21.15.67:0/3524254216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff0075a00 con 0x7f6fd40039b0 2024-03-09T09:04:44.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.360+0000 7f6fe6000640 1 -- 172.21.15.67:0/3524254216 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6fe003f070 con 0x7f6fd40039b0 2024-03-09T09:04:44.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.361+0000 7f6fe6000640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fd006b500 0x7f6fd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:44.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.361+0000 7f6fe6000640 1 -- 172.21.15.67:0/3524254216 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6fe00abaa0 con 0x7f6fd40039b0 2024-03-09T09:04:44.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.362+0000 7f6fed400640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fd006b500 0x7f6fd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:44.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.363+0000 7f6fed400640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fd006b500 0x7f6fd006d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00c26a0 tx=0x7f6fdc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:44.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.366+0000 7f6fe6000640 1 -- 172.21.15.67:0/3524254216 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6fe0007e00 con 0x7f6fd40039b0 2024-03-09T09:04:44.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.649+0000 7f6ff5000640 1 -- 172.21.15.67:0/3524254216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f6ff00c1d70 con 0x7f6fd40039b0 2024-03-09T09:04:44.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.650+0000 7f6fe6000640 1 -- 172.21.15.67:0/3524254216 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f6fe007a5a0 con 0x7f6fd40039b0 2024-03-09T09:04:44.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.653+0000 7f6ff5000640 1 -- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fd006b500 msgr2=0x7f6fd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:44.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.653+0000 7f6ff5000640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fd006b500 0x7f6fd006d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00c26a0 tx=0x7f6fdc008040 comp rx=0 tx=0).stop 2024-03-09T09:04:44.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.653+0000 7f6ff5000640 1 -- 172.21.15.67:0/3524254216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 msgr2=0x7f6ff014dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:44.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.653+0000 7f6ff5000640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 0x7f6ff014dbb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6fe002ddf0 tx=0x7f6ff00c2960 comp rx=0 tx=0).stop 2024-03-09T09:04:44.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.654+0000 7f6ff5000640 1 -- 172.21.15.67:0/3524254216 shutdown_connections 2024-03-09T09:04:44.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.654+0000 7f6ff5000640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6fd006b500 0x7f6fd006d9c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.654+0000 7f6ff5000640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff00aaa60 0x7f6ff01ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.654+0000 7f6ff5000640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ff00abea0 0x7f6ff00aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.654+0000 7f6ff5000640 1 --2- 172.21.15.67:0/3524254216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fd40039b0 0x7f6ff014dbb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:44.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.654+0000 7f6ff5000640 1 -- 172.21.15.67:0/3524254216 >> 172.21.15.67:0/3524254216 conn(0x7f6ff00c7370 msgr2=0x7f6ff0156d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:44.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.654+0000 7f6ff5000640 1 -- 172.21.15.67:0/3524254216 shutdown_connections 2024-03-09T09:04:44.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:44.654+0000 7f6ff5000640 1 -- 172.21.15.67:0/3524254216 wait complete. 2024-03-09T09:04:44.676 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:04:45.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:04:45.115 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:04:54.677 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:04:54.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:04:54.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.894+0000 7f99e6200640 1 Processor -- start 2024-03-09T09:04:54.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.894+0000 7f99e6200640 1 -- start start 2024-03-09T09:04:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.895+0000 7f99e6200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99e0155e20 0x7f99e006c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.895+0000 7f99e6200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e00c6a40 con 0x7f99e00c5f50 2024-03-09T09:04:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.895+0000 7f99e6200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e006b5b0 con 0x7f99e006b040 2024-03-09T09:04:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.895+0000 7f99e6200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e00c9e00 con 0x7f99e0155e20 2024-03-09T09:04:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.896+0000 7f99df400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f99e00c5f50 0x7f99e00c96f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54872/0 (socket says 172.21.15.67:54872) 2024-03-09T09:04:54.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.896+0000 7f99df400640 1 -- 172.21.15.67:0/2043621089 learned_addr learned my addr 172.21.15.67:0/2043621089 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:04:54.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.896+0000 7f99dfe00640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99e0155e20 0x7f99e006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:54.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.896+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843989091 0 0) 0x7f99e00c6a40 con 0x7f99e00c5f50 2024-03-09T09:04:54.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.896+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99c4003520 con 0x7f99e00c5f50 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.896+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 839468126 0 0) 0x7f99c4003520 con 0x7f99e00c5f50 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.896+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99e0155e20 msgr2=0x7f99e006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.896+0000 7f99de000640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99e0155e20 0x7f99e006c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 >> v1:172.21.15.134:6789/0 conn(0x7f99e006b040 legacy=0x7f99e00c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e006b890 con 0x7f99e00c5f50 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f99d0001670 con 0x7f99e00c5f50 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 >> v1:172.21.15.67:6789/0 conn(0x7f99e00c5f50 legacy=0x7f99e00c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99de000640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c40039e0 0x7f99c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99de000640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c4004800 0x7f99c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99de000640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c4005210 0x7f99c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e00c9e00 con 0x7f99c40039e0 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c400c0c0 con 0x7f99c4004800 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c400c2c0 con 0x7f99c4005210 2024-03-09T09:04:54.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99dfe00640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99e0155e20 0x7f99e006c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:04:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99df400640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c40039e0 0x7f99c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.897+0000 7f99dea00640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c4004800 0x7f99c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.898+0000 7f99df400640 1 -- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c4005210 msgr2=0x7f99c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.898+0000 7f99dfe00640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c4005210 0x7f99c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.898+0000 7f99df400640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c4005210 0x7f99c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.898+0000 7f99df400640 1 -- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c4004800 msgr2=0x7f99c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.898+0000 7f99df400640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c4004800 0x7f99c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.898+0000 7f99df400640 1 -- 172.21.15.67:0/2043621089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4003520 con 0x7f99c40039e0 2024-03-09T09:04:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.898+0000 7f99dea00640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c4004800 0x7f99c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:04:54.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.898+0000 7f99dfe00640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99c4005210 0x7f99c400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:04:54.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.899+0000 7f99df400640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c40039e0 0x7f99c4003e00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f99d000c110 tx=0x7f99c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd331da05920a9e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:54.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.899+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0007c40 con 0x7f99c40039e0 2024-03-09T09:04:54.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.899+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d0030070 con 0x7f99c40039e0 2024-03-09T09:04:54.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.899+0000 7f99de000640 1 -- 172.21.15.67:0/2043621089 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0005d30 con 0x7f99c40039e0 2024-03-09T09:04:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.899+0000 7f99e6200640 1 -- 172.21.15.67:0/2043621089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c40039e0 msgr2=0x7f99c4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.899+0000 7f99e6200640 1 --2- 172.21.15.67:0/2043621089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99c40039e0 0x7f99c4003e00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f99d000c110 tx=0x7f99c4005830 comp rx=0 tx=0).stop 2024-03-09T09:04:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.900+0000 7f99dfe00640 1 -- 172.21.15.67:0/2043621089 reap_dead start 2024-03-09T09:04:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.900+0000 7f99e6200640 1 -- 172.21.15.67:0/2043621089 shutdown_connections 2024-03-09T09:04:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.900+0000 7f99e6200640 1 -- 172.21.15.67:0/2043621089 >> 172.21.15.67:0/2043621089 conn(0x7f99e00c8050 msgr2=0x7f99e00c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.900+0000 7f99e6200640 1 -- 172.21.15.67:0/2043621089 shutdown_connections 2024-03-09T09:04:54.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.900+0000 7f99e6200640 1 -- 172.21.15.67:0/2043621089 wait complete. 2024-03-09T09:04:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.901+0000 7f99e6200640 1 Processor -- start 2024-03-09T09:04:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.902+0000 7f99e6200640 1 -- start start 2024-03-09T09:04:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.902+0000 7f99e6200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c40039e0 0x7f99e01e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.902+0000 7f99e6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99e01e3030 0x7f99e014e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99e6200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99e014bb50 0x7f99e014eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99e6200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e006c130 con 0x7f99e01e3030 2024-03-09T09:04:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99e6200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e006be50 con 0x7f99c40039e0 2024-03-09T09:04:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99e6200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e014cc00 con 0x7f99e014bb50 2024-03-09T09:04:54.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99dea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99e01e3030 0x7f99e014e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:54.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99dea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99e01e3030 0x7f99e014e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40708/0 (socket says 172.21.15.67:40708) 2024-03-09T09:04:54.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99dea00640 1 -- 172.21.15.67:0/2240553865 learned_addr learned my addr 172.21.15.67:0/2240553865 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:54.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99df400640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c40039e0 0x7f99e01e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:54.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99dfe00640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99e014bb50 0x7f99e014eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:54.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99dea00640 1 -- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99e014bb50 msgr2=0x7f99e014eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:54.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99dea00640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99e014bb50 0x7f99e014eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:54.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99dea00640 1 -- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c40039e0 msgr2=0x7f99e01e2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:54.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99dea00640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c40039e0 0x7f99e01e2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:54.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99dea00640 1 -- 172.21.15.67:0/2240553865 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e014b900 con 0x7f99e01e3030 2024-03-09T09:04:54.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.903+0000 7f99dfe00640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99e014bb50 0x7f99e014eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:04:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.904+0000 7f99dea00640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99e01e3030 0x7f99e014e970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d770 tx=0x7f99cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.904+0000 7f99d7400640 1 -- 172.21.15.67:0/2240553865 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc005270 con 0x7f99e01e3030 2024-03-09T09:04:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.904+0000 7f99d7400640 1 -- 172.21.15.67:0/2240553865 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc017070 con 0x7f99e01e3030 2024-03-09T09:04:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.904+0000 7f99e6200640 1 -- 172.21.15.67:0/2240553865 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e014b620 con 0x7f99e01e3030 2024-03-09T09:04:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.904+0000 7f99d7400640 1 -- 172.21.15.67:0/2240553865 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00b650 con 0x7f99e01e3030 2024-03-09T09:04:54.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.904+0000 7f99e6200640 1 -- 172.21.15.67:0/2240553865 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e01ec830 con 0x7f99e01e3030 2024-03-09T09:04:54.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.906+0000 7f99e6200640 1 -- 172.21.15.67:0/2240553865 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e0075a00 con 0x7f99e01e3030 2024-03-09T09:04:54.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.908+0000 7f99d7400640 1 -- 172.21.15.67:0/2240553865 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f99cc00b7f0 con 0x7f99e01e3030 2024-03-09T09:04:54.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.909+0000 7f99d7400640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99c006b4d0 0x7f99c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:54.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.909+0000 7f99d7400640 1 -- 172.21.15.67:0/2240553865 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f99cc08a400 con 0x7f99e01e3030 2024-03-09T09:04:54.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.910+0000 7f99df400640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99c006b4d0 0x7f99c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:54.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.911+0000 7f99df400640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99c006b4d0 0x7f99c006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f99d0034000 tx=0x7f99d0009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:54.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:54.915+0000 7f99d7400640 1 -- 172.21.15.67:0/2240553865 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f99cc058e80 con 0x7f99e01e3030 2024-03-09T09:04:55.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.237+0000 7f99e6200640 1 -- 172.21.15.67:0/2240553865 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f99e0156530 con 0x7f99e01e3030 2024-03-09T09:04:55.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.237+0000 7f99d7400640 1 -- 172.21.15.67:0/2240553865 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f99cc01ae40 con 0x7f99e01e3030 2024-03-09T09:04:55.240 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":92,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:04:55.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.240+0000 7f99e6200640 1 -- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99c006b4d0 msgr2=0x7f99c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.240+0000 7f99e6200640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99c006b4d0 0x7f99c006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f99d0034000 tx=0x7f99d0009030 comp rx=0 tx=0).stop 2024-03-09T09:04:55.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.240+0000 7f99e6200640 1 -- 172.21.15.67:0/2240553865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99e01e3030 msgr2=0x7f99e014e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.240+0000 7f99e6200640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99e01e3030 0x7f99e014e970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d770 tx=0x7f99cc00dc30 comp rx=0 tx=0).stop 2024-03-09T09:04:55.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.240+0000 7f99e6200640 1 -- 172.21.15.67:0/2240553865 shutdown_connections 2024-03-09T09:04:55.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.240+0000 7f99e6200640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f99c006b4d0 0x7f99c006d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.240+0000 7f99e6200640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99e014bb50 0x7f99e014eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.240+0000 7f99e6200640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99e01e3030 0x7f99e014e970 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.240+0000 7f99e6200640 1 --2- 172.21.15.67:0/2240553865 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c40039e0 0x7f99e01e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.240+0000 7f99e6200640 1 -- 172.21.15.67:0/2240553865 >> 172.21.15.67:0/2240553865 conn(0x7f99e00c8050 msgr2=0x7f99e00ab030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:55.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.240+0000 7f99e6200640 1 -- 172.21.15.67:0/2240553865 shutdown_connections 2024-03-09T09:04:55.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.241+0000 7f99e6200640 1 -- 172.21.15.67:0/2240553865 wait complete. 2024-03-09T09:04:55.261 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:04:55.261 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:04:55.261 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:04:55.261 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:04:55.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.478+0000 7fde35e00640 1 Processor -- start 2024-03-09T09:04:55.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.478+0000 7fde35e00640 1 -- start start 2024-03-09T09:04:55.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.479+0000 7fde35e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde300ac930 0x7fde300c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.480+0000 7fde35e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fde300c30a0 0x7fde30150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.480+0000 7fde35e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde300c2780 con 0x7fde30151190 2024-03-09T09:04:55.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.480+0000 7fde35e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde30070670 con 0x7fde300c30a0 2024-03-09T09:04:55.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.480+0000 7fde35e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde300708c0 con 0x7fde300ac930 2024-03-09T09:04:55.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.480+0000 7fde34a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde300ac930 0x7fde300c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.480+0000 7fde34a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde300ac930 0x7fde300c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:34472/0 (socket says 172.21.15.67:34472) 2024-03-09T09:04:55.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.480+0000 7fde34a00640 1 -- 172.21.15.67:0/886422614 learned_addr learned my addr 172.21.15.67:0/886422614 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:55.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.481+0000 7fde2fe00640 1 --2- 172.21.15.67:0/886422614 >> v2:172.21.15.134:3300/0 conn(0x7fde300c30a0 0x7fde30150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.481+0000 7fde2f400640 1 -- 172.21.15.67:0/886422614 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127470578 0 0) 0x7fde300c2780 con 0x7fde30151190 2024-03-09T09:04:55.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.481+0000 7fde2f400640 1 -- 172.21.15.67:0/886422614 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde18003520 con 0x7fde30151190 2024-03-09T09:04:55.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.481+0000 7fde34a00640 1 -- 172.21.15.67:0/886422614 >> v2:172.21.15.134:3300/0 conn(0x7fde300c30a0 msgr2=0x7fde30150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:55.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.481+0000 7fde34a00640 1 --2- 172.21.15.67:0/886422614 >> v2:172.21.15.134:3300/0 conn(0x7fde300c30a0 0x7fde30150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.481+0000 7fde34a00640 1 -- 172.21.15.67:0/886422614 >> v1:172.21.15.134:6789/0 conn(0x7fde30151190 legacy=0x7fde301539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.481+0000 7fde34a00640 1 -- 172.21.15.67:0/886422614 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30070ba0 con 0x7fde300ac930 2024-03-09T09:04:55.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.483+0000 7fde34a00640 1 --2- 172.21.15.67:0/886422614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde300ac930 0x7fde300c3660 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fde24009870 tx=0x7fde2402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b694e405f7bbc435 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:55.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.483+0000 7fde2f400640 1 -- 172.21.15.67:0/886422614 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24034070 con 0x7fde300ac930 2024-03-09T09:04:55.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.483+0000 7fde2f400640 1 -- 172.21.15.67:0/886422614 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde24030840 con 0x7fde300ac930 2024-03-09T09:04:55.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.483+0000 7fde2f400640 1 -- 172.21.15.67:0/886422614 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24030bf0 con 0x7fde300ac930 2024-03-09T09:04:55.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.484+0000 7fde35e00640 1 -- 172.21.15.67:0/886422614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde300ac930 msgr2=0x7fde300c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.484+0000 7fde35e00640 1 --2- 172.21.15.67:0/886422614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde300ac930 0x7fde300c3660 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fde24009870 tx=0x7fde2402b980 comp rx=0 tx=0).stop 2024-03-09T09:04:55.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.484+0000 7fde35e00640 1 -- 172.21.15.67:0/886422614 shutdown_connections 2024-03-09T09:04:55.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.484+0000 7fde35e00640 1 --2- 172.21.15.67:0/886422614 >> v2:172.21.15.134:3300/0 conn(0x7fde300c30a0 0x7fde30150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.484+0000 7fde35e00640 1 --2- 172.21.15.67:0/886422614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde300ac930 0x7fde300c3660 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.484+0000 7fde35e00640 1 -- 172.21.15.67:0/886422614 >> 172.21.15.67:0/886422614 conn(0x7fde300c7220 msgr2=0x7fde300c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:55.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.484+0000 7fde35e00640 1 -- 172.21.15.67:0/886422614 shutdown_connections 2024-03-09T09:04:55.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.484+0000 7fde35e00640 1 -- 172.21.15.67:0/886422614 wait complete. 2024-03-09T09:04:55.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.486+0000 7fde35e00640 1 Processor -- start 2024-03-09T09:04:55.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.486+0000 7fde35e00640 1 -- start start 2024-03-09T09:04:55.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.487+0000 7fde35e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde300ac930 0x7fde30143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.487+0000 7fde35e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde300c30a0 0x7fde300c0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.487+0000 7fde35e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30151190 0x7fde301ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.487+0000 7fde35e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300acd10 con 0x7fde300ac930 2024-03-09T09:04:55.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.487+0000 7fde35e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde301515e0 con 0x7fde30151190 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.487+0000 7fde35e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde300ace90 con 0x7fde300c30a0 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.487+0000 7fde34a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde300ac930 0x7fde30143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.487+0000 7fde34a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde300ac930 0x7fde30143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40716/0 (socket says 172.21.15.67:40716) 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.487+0000 7fde34a00640 1 -- 172.21.15.67:0/3104291767 learned_addr learned my addr 172.21.15.67:0/3104291767 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.487+0000 7fde2fe00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde300c30a0 0x7fde300c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde35400640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30151190 0x7fde301ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde34a00640 1 -- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde300c30a0 msgr2=0x7fde300c0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde34a00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde300c30a0 0x7fde300c0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde34a00640 1 -- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30151190 msgr2=0x7fde301ee8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde34a00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30151190 0x7fde301ee8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde34a00640 1 -- 172.21.15.67:0/3104291767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30151d50 con 0x7fde300ac930 2024-03-09T09:04:55.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde35400640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde30151190 0x7fde301ee8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:04:55.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde34a00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde300ac930 0x7fde30143e80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde2402b9c0 tx=0x7fde240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:55.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24046070 con 0x7fde300ac930 2024-03-09T09:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde24003820 con 0x7fde300ac930 2024-03-09T09:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde35e00640 1 -- 172.21.15.67:0/3104291767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde300aa650 con 0x7fde300ac930 2024-03-09T09:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.488+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24034040 con 0x7fde300ac930 2024-03-09T09:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.489+0000 7fde35e00640 1 -- 172.21.15.67:0/3104291767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde300aac10 con 0x7fde300ac930 2024-03-09T09:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.489+0000 7fde35e00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30153ff0 0x7fde301eede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.489+0000 7fde35e00640 1 -- 172.21.15.67:0/3104291767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde30152030 con 0x7fde30153ff0 2024-03-09T09:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.490+0000 7fde2fe00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30153ff0 0x7fde301eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.490+0000 7fde2fe00640 1 -- 172.21.15.67:0/3104291767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde3004f3c0 con 0x7fde30153ff0 2024-03-09T09:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.491+0000 7fde2fe00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30153ff0 0x7fde301eede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fde300c6e80 tx=0x7fde1c0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.491+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fde24049070 con 0x7fde300ac930 2024-03-09T09:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.492+0000 7fde2cc00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fde1006b4d0 0x7fde1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.492+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fde240ac180 con 0x7fde300ac930 2024-03-09T09:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.493+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c004230 con 0x7fde30153ff0 2024-03-09T09:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.493+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fde1c0043d0 con 0x7fde30153ff0 2024-03-09T09:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.493+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30153ff0 msgr2=0x7fde301eede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.493+0000 7fde2cc00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30153ff0 0x7fde301eede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fde300c6e80 tx=0x7fde1c0119f0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.493+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30153ff0 msgr2=0x7fde301eede0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.493+0000 7fde2cc00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30153ff0 0x7fde301eede0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fde300c6e80 tx=0x7fde1c0119f0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.493+0000 7fde35400640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fde1006b4d0 0x7fde1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.494+0000 7fde35400640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fde1006b4d0 0x7fde1006d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fde28007fd0 tx=0x7fde28009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:55.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.500+0000 7fde35e00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30151850 0x7fde301eb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.500+0000 7fde35e00640 1 -- 172.21.15.67:0/3104291767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300aa4b0 con 0x7fde30151850 2024-03-09T09:04:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.500+0000 7fde2fe00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30151850 0x7fde301eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.501+0000 7fde2fe00640 1 -- 172.21.15.67:0/3104291767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fde300733f0 con 0x7fde30151850 2024-03-09T09:04:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.501+0000 7fde2fe00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30151850 0x7fde301eb050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fde300c6e80 tx=0x7fde1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.501+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c015db0 con 0x7fde30151850 2024-03-09T09:04:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.501+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7fde1c015db0 con 0x7fde30151850 2024-03-09T09:04:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.501+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30151850 msgr2=0x7fde301eb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.501+0000 7fde2cc00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30151850 0x7fde301eb050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fde300c6e80 tx=0x7fde1c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.501+0000 7fde2cc00640 1 -- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30151850 msgr2=0x7fde301eb050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.501+0000 7fde2cc00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde30151850 0x7fde301eb050 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fde300c6e80 tx=0x7fde1c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.505 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:04:55.505 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:04:55.505 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:04:55.505 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:04:55.505 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 92, 2024-03-09T09:04:55.505 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:04:55.505 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:04:55.505 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 17, 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:55.506 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:55.507 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:55.508 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:04:55.510 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.511 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.513 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.514 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:55.515 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:55.516 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.517 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:04:55.518 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.518 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:55.518 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:55.518 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:55.518 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.518 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.518 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:04:55.518 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:04:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.502+0000 7fde35e00640 1 -- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fde1006b4d0 msgr2=0x7fde1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.502+0000 7fde35e00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fde1006b4d0 0x7fde1006d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fde28007fd0 tx=0x7fde28009040 comp rx=0 tx=0).stop 2024-03-09T09:04:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.502+0000 7fde35e00640 1 -- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde300ac930 msgr2=0x7fde30143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.502+0000 7fde35e00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde300ac930 0x7fde30143e80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde2402b9c0 tx=0x7fde240046c0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.502+0000 7fde35400640 1 -- 172.21.15.67:0/3104291767 reap_dead start 2024-03-09T09:04:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.502+0000 7fde35e00640 1 -- 172.21.15.67:0/3104291767 shutdown_connections 2024-03-09T09:04:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.502+0000 7fde35e00640 1 --2- 172.21.15.67:0/3104291767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde300ac930 0x7fde30143e80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.502+0000 7fde35e00640 1 -- 172.21.15.67:0/3104291767 >> 172.21.15.67:0/3104291767 conn(0x7fde300c7220 msgr2=0x7fde30072940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.502+0000 7fde35e00640 1 -- 172.21.15.67:0/3104291767 shutdown_connections 2024-03-09T09:04:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.502+0000 7fde35e00640 1 -- 172.21.15.67:0/3104291767 wait complete. 2024-03-09T09:04:55.521 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:04:55.521 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:04:55.521 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:04:55.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.751+0000 7fc00fe00640 1 Processor -- start 2024-03-09T09:04:55.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.751+0000 7fc00fe00640 1 -- start start 2024-03-09T09:04:55.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.753+0000 7fc00fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc0100c9380 0x7fc0100acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.753+0000 7fc00fe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc0100ad440 0x7fc01006c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.753+0000 7fc00fe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0100c9890 con 0x7fc01014fd90 2024-03-09T09:04:55.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.753+0000 7fc00fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01006ba60 con 0x7fc0100c9380 2024-03-09T09:04:55.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.753+0000 7fc00fe00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0100c5f80 con 0x7fc0100ad440 2024-03-09T09:04:55.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.753+0000 7fc00e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc0100c9380 0x7fc0100acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.754+0000 7fc00e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc0100c9380 0x7fc0100acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40744/0 (socket says 172.21.15.67:40744) 2024-03-09T09:04:55.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.754+0000 7fc00e000640 1 -- 172.21.15.67:0/1725657521 learned_addr learned my addr 172.21.15.67:0/1725657521 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:55.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.754+0000 7fc00f400640 1 --2- 172.21.15.67:0/1725657521 >> v2:172.21.15.134:3300/0 conn(0x7fc0100ad440 0x7fc01006c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.754+0000 7fc00e000640 1 -- 172.21.15.67:0/1725657521 >> v2:172.21.15.134:3300/0 conn(0x7fc0100ad440 msgr2=0x7fc01006c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:55.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.754+0000 7fc00e000640 1 --2- 172.21.15.67:0/1725657521 >> v2:172.21.15.134:3300/0 conn(0x7fc0100ad440 0x7fc01006c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.754+0000 7fc00e000640 1 -- 172.21.15.67:0/1725657521 >> v1:172.21.15.134:6789/0 conn(0x7fc01014fd90 legacy=0x7fc010156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.754+0000 7fc00e000640 1 -- 172.21.15.67:0/1725657521 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0100c6240 con 0x7fc0100c9380 2024-03-09T09:04:55.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.756+0000 7fc00e000640 1 --2- 172.21.15.67:0/1725657521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0100c9380 0x7fc0100acf00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc01006d270 tx=0x7fbffc02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ce00b7d7edd966 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:55.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.756+0000 7fc00d600640 1 -- 172.21.15.67:0/1725657521 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc035020 con 0x7fc0100c9380 2024-03-09T09:04:55.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.756+0000 7fc00d600640 1 -- 172.21.15.67:0/1725657521 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc031800 con 0x7fc0100c9380 2024-03-09T09:04:55.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.756+0000 7fc00d600640 1 -- 172.21.15.67:0/1725657521 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc031bd0 con 0x7fc0100c9380 2024-03-09T09:04:55.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.757+0000 7fc00fe00640 1 -- 172.21.15.67:0/1725657521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0100c9380 msgr2=0x7fc0100acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.757+0000 7fc00fe00640 1 --2- 172.21.15.67:0/1725657521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0100c9380 0x7fc0100acf00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc01006d270 tx=0x7fbffc02b8a0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.757+0000 7fc00fe00640 1 -- 172.21.15.67:0/1725657521 shutdown_connections 2024-03-09T09:04:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.757+0000 7fc00fe00640 1 --2- 172.21.15.67:0/1725657521 >> v2:172.21.15.134:3300/0 conn(0x7fc0100ad440 0x7fc01006c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.757+0000 7fc00fe00640 1 --2- 172.21.15.67:0/1725657521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0100c9380 0x7fc0100acf00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.757+0000 7fc00fe00640 1 -- 172.21.15.67:0/1725657521 >> 172.21.15.67:0/1725657521 conn(0x7fc0100cb480 msgr2=0x7fc0100cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.757+0000 7fc00fe00640 1 -- 172.21.15.67:0/1725657521 shutdown_connections 2024-03-09T09:04:55.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.757+0000 7fc00fe00640 1 -- 172.21.15.67:0/1725657521 wait complete. 2024-03-09T09:04:55.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.760+0000 7fc00fe00640 1 Processor -- start 2024-03-09T09:04:55.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.760+0000 7fc00fe00640 1 -- start start 2024-03-09T09:04:55.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.761+0000 7fc00fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0100ad440 0x7fc0101ec330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.761+0000 7fc00fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0100c9380 0x7fc010159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.761+0000 7fc00fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014fd90 0x7fc01014bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.761+0000 7fc00fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01006c710 con 0x7fc0100c9380 2024-03-09T09:04:55.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.761+0000 7fc00fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01006bcb0 con 0x7fc01014fd90 2024-03-09T09:04:55.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.761+0000 7fc00fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01006c890 con 0x7fc0100ad440 2024-03-09T09:04:55.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.762+0000 7fc00e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0100c9380 0x7fc010159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.762+0000 7fc00e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0100c9380 0x7fc010159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40750/0 (socket says 172.21.15.67:40750) 2024-03-09T09:04:55.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.762+0000 7fc00e000640 1 -- 172.21.15.67:0/4003446536 learned_addr learned my addr 172.21.15.67:0/4003446536 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:55.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.762+0000 7fc00ea00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0100ad440 0x7fc0101ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.762+0000 7fc00f400640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014fd90 0x7fc01014bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.762+0000 7fc00e000640 1 -- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0100ad440 msgr2=0x7fc0101ec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.762+0000 7fc00e000640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0100ad440 0x7fc0101ec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.762+0000 7fc00e000640 1 -- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014fd90 msgr2=0x7fc01014bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.762+0000 7fc00e000640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014fd90 0x7fc01014bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.762+0000 7fc00e000640 1 -- 172.21.15.67:0/4003446536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0100ac6f0 con 0x7fc0100c9380 2024-03-09T09:04:55.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.762+0000 7fc00f400640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014fd90 0x7fc01014bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:04:55.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.763+0000 7fc00e000640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0100c9380 0x7fc010159690 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbffc02b8e0 tx=0x7fbffc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:55.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.763+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc040070 con 0x7fc0100c9380 2024-03-09T09:04:55.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.763+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc031e50 con 0x7fc0100c9380 2024-03-09T09:04:55.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.763+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc035040 con 0x7fc0100c9380 2024-03-09T09:04:55.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.763+0000 7fc00fe00640 1 -- 172.21.15.67:0/4003446536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc010156ff0 con 0x7fc0100c9380 2024-03-09T09:04:55.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.764+0000 7fc00fe00640 1 -- 172.21.15.67:0/4003446536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0101575b0 con 0x7fc0100c9380 2024-03-09T09:04:55.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.765+0000 7fc00fe00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014cc30 0x7fc01014c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.765+0000 7fc00fe00640 1 -- 172.21.15.67:0/4003446536 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100ae3f0 con 0x7fc01014cc30 2024-03-09T09:04:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.766+0000 7fc00ea00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014cc30 0x7fc01014c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.767+0000 7fc00ea00640 1 -- 172.21.15.67:0/4003446536 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc01014ca20 con 0x7fc01014cc30 2024-03-09T09:04:55.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.767+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbffc04b060 con 0x7fc0100c9380 2024-03-09T09:04:55.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.767+0000 7fc00ea00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014cc30 0x7fc01014c4e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc000001eb0 tx=0x7fc00000e890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.768+0000 7fc006a00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbff006b4d0 0x7fbff006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.768+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbffc0ac070 con 0x7fc0100c9380 2024-03-09T09:04:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.769+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0000052e0 con 0x7fc01014cc30 2024-03-09T09:04:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.769+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc000005480 con 0x7fc01014cc30 2024-03-09T09:04:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.769+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014cc30 msgr2=0x7fc01014c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.769+0000 7fc006a00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014cc30 0x7fc01014c4e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc000001eb0 tx=0x7fc00000e890 comp rx=0 tx=0).stop 2024-03-09T09:04:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.769+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014cc30 msgr2=0x7fc01014c4e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.769+0000 7fc006a00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc01014cc30 0x7fc01014c4e0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc000001eb0 tx=0x7fc00000e890 comp rx=0 tx=0).stop 2024-03-09T09:04:55.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.769+0000 7fc00f400640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbff006b4d0 0x7fbff006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.770+0000 7fc00f400640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbff006b4d0 0x7fbff006d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc008007520 tx=0x7fc00800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.782+0000 7fc00fe00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0101577f0 0x7fc010158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:55.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.782+0000 7fc00fe00640 1 -- 172.21.15.67:0/4003446536 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01004f3c0 con 0x7fc0101577f0 2024-03-09T09:04:55.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.787+0000 7fc00ea00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0101577f0 0x7fc010158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:55.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.787+0000 7fc00ea00640 1 -- 172.21.15.67:0/4003446536 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc01006d2d0 con 0x7fc0101577f0 2024-03-09T09:04:55.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.788+0000 7fc00ea00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0101577f0 0x7fc010158090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc000001eb0 tx=0x7fc000001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.788+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000007660 con 0x7fc0101577f0 2024-03-09T09:04:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.789+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5017 (secure 0 0 0) 0x7fc000007660 con 0x7fc0101577f0 2024-03-09T09:04:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.789+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0101577f0 msgr2=0x7fc010158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.789+0000 7fc006a00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0101577f0 0x7fc010158090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc000001eb0 tx=0x7fc000001080 comp rx=0 tx=0).stop 2024-03-09T09:04:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.789+0000 7fc006a00640 1 -- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0101577f0 msgr2=0x7fc010158090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:55.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.789+0000 7fc006a00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0101577f0 0x7fc010158090 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc000001eb0 tx=0x7fc000001080 comp rx=0 tx=0).stop 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 92, 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 17, 2024-03-09T09:04:55.793 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:55.794 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:55.795 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:04:55.796 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:55.797 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:55.798 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:55.799 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:55.800 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:04:55.801 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:55.803 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:04:55.804 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:55.805 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:55.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:55.806 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:04:55.806 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:04:55.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.790+0000 7fc00fe00640 1 -- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbff006b4d0 msgr2=0x7fbff006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.790+0000 7fc00fe00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbff006b4d0 0x7fbff006d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc008007520 tx=0x7fc00800b040 comp rx=0 tx=0).stop 2024-03-09T09:04:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.790+0000 7fc00fe00640 1 -- 172.21.15.67:0/4003446536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0100c9380 msgr2=0x7fc010159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.790+0000 7fc00fe00640 1 --2- 172.21.15.67:0/4003446536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0100c9380 0x7fc010159690 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbffc02b8e0 tx=0x7fbffc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:04:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.790+0000 7fc00f400640 1 -- 172.21.15.67:0/4003446536 reap_dead start 2024-03-09T09:04:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.790+0000 7fc00f400640 1 -- 172.21.15.67:0/4003446536 reap_dead start 2024-03-09T09:04:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.790+0000 7fc00fe00640 1 -- 172.21.15.67:0/4003446536 shutdown_connections 2024-03-09T09:04:55.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.790+0000 7fc00fe00640 1 -- 172.21.15.67:0/4003446536 >> 172.21.15.67:0/4003446536 conn(0x7fc0100cb480 msgr2=0x7fc0100cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.790+0000 7fc00fe00640 1 -- 172.21.15.67:0/4003446536 shutdown_connections 2024-03-09T09:04:55.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:55.790+0000 7fc00fe00640 1 -- 172.21.15.67:0/4003446536 wait complete. 2024-03-09T09:04:55.812 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:04:55.812 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:04:55.812 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:04:56.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.020+0000 7efe0b800640 1 Processor -- start 2024-03-09T09:04:56.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.020+0000 7efe0b800640 1 -- start start 2024-03-09T09:04:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.021+0000 7efe0b800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe040ac9a0 0x7efe041510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.021+0000 7efe0b800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efe04071190 0x7efe04151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.021+0000 7efe0b800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe040c8200 con 0x7efe040c2780 2024-03-09T09:04:56.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.021+0000 7efe0b800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0406d590 con 0x7efe04071190 2024-03-09T09:04:56.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.021+0000 7efe0b800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe040c3740 con 0x7efe040ac9a0 2024-03-09T09:04:56.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.022+0000 7efe09800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efe04071190 0x7efe04151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.022+0000 7efe09800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efe04071190 0x7efe04151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40758/0 (socket says 172.21.15.67:40758) 2024-03-09T09:04:56.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.022+0000 7efe09800640 1 -- 172.21.15.67:0/2920330397 learned_addr learned my addr 172.21.15.67:0/2920330397 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:56.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.022+0000 7efe08e00640 1 --2- 172.21.15.67:0/2920330397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe040ac9a0 0x7efe041510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.022+0000 7efe09800640 1 -- 172.21.15.67:0/2920330397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe040ac9a0 msgr2=0x7efe041510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.022+0000 7efe09800640 1 --2- 172.21.15.67:0/2920330397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe040ac9a0 0x7efe041510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.022+0000 7efe09800640 1 -- 172.21.15.67:0/2920330397 >> v1:172.21.15.67:6789/0 conn(0x7efe040c2780 legacy=0x7efe04154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.022+0000 7efe09800640 1 -- 172.21.15.67:0/2920330397 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe040c3990 con 0x7efe04071190 2024-03-09T09:04:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.022+0000 7efe08e00640 1 --2- 172.21.15.67:0/2920330397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe040ac9a0 0x7efe041510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:04:56.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.022+0000 7efe03400640 1 -- 172.21.15.67:0/2920330397 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665687585 0 0) 0x7efe040c8200 con 0x7efe040c2780 2024-03-09T09:04:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.024+0000 7efe09800640 1 --2- 172.21.15.67:0/2920330397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe04071190 0x7efe04151620 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009e50 tx=0x7efdfc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f64145349894cd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.024+0000 7efe03400640 1 -- 172.21.15.67:0/2920330397 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc02ca70 con 0x7efe04071190 2024-03-09T09:04:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.024+0000 7efe03400640 1 -- 172.21.15.67:0/2920330397 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc02cc10 con 0x7efe04071190 2024-03-09T09:04:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.024+0000 7efe03400640 1 -- 172.21.15.67:0/2920330397 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0343f0 con 0x7efe04071190 2024-03-09T09:04:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.024+0000 7efe0b800640 1 -- 172.21.15.67:0/2920330397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe04071190 msgr2=0x7efe04151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.024+0000 7efe0b800640 1 --2- 172.21.15.67:0/2920330397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe04071190 0x7efe04151620 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009e50 tx=0x7efdfc02bed0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.024+0000 7efe0b800640 1 -- 172.21.15.67:0/2920330397 shutdown_connections 2024-03-09T09:04:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.024+0000 7efe0b800640 1 --2- 172.21.15.67:0/2920330397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe04071190 0x7efe04151620 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.024+0000 7efe0b800640 1 --2- 172.21.15.67:0/2920330397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe040ac9a0 0x7efe041510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.024+0000 7efe0b800640 1 -- 172.21.15.67:0/2920330397 >> 172.21.15.67:0/2920330397 conn(0x7efe040c7370 msgr2=0x7efe040c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.025+0000 7efe0b800640 1 -- 172.21.15.67:0/2920330397 shutdown_connections 2024-03-09T09:04:56.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.025+0000 7efe0b800640 1 -- 172.21.15.67:0/2920330397 wait complete. 2024-03-09T09:04:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.026+0000 7efe0b800640 1 Processor -- start 2024-03-09T09:04:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.027+0000 7efe0b800640 1 -- start start 2024-03-09T09:04:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.027+0000 7efe0b800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe04071190 0x7efe041ec9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.027+0000 7efe0b800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040ac9a0 0x7efe04145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.027+0000 7efe0b800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe040c2780 0x7efe04146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.027+0000 7efe0b800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe040acd80 con 0x7efe04071190 2024-03-09T09:04:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.027+0000 7efe0b800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe04070c00 con 0x7efe040ac9a0 2024-03-09T09:04:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.027+0000 7efe0b800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe040acf00 con 0x7efe040c2780 2024-03-09T09:04:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.028+0000 7efe08e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe04071190 0x7efe041ec9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.028+0000 7efe08e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe04071190 0x7efe041ec9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40766/0 (socket says 172.21.15.67:40766) 2024-03-09T09:04:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.028+0000 7efe08e00640 1 -- 172.21.15.67:0/2600823320 learned_addr learned my addr 172.21.15.67:0/2600823320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:56.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.028+0000 7efe09800640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe040c2780 0x7efe04146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.028+0000 7efe03e00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040ac9a0 0x7efe04145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.028+0000 7efe08e00640 1 -- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe040c2780 msgr2=0x7efe04146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.028+0000 7efe08e00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe040c2780 0x7efe04146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.028+0000 7efe08e00640 1 -- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040ac9a0 msgr2=0x7efe04145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.028+0000 7efe08e00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efe040ac9a0 0x7efe04145bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.028+0000 7efe08e00640 1 -- 172.21.15.67:0/2600823320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe04153b10 con 0x7efe04071190 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.029+0000 7efe08e00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe04071190 0x7efe041ec9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdf800e7e0 tx=0x7efdf800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.029+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf800f040 con 0x7efe04071190 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.029+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf8004030 con 0x7efe04071190 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.029+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8004400 con 0x7efe04071190 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.029+0000 7efe0b800640 1 -- 172.21.15.67:0/2600823320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe04146a00 con 0x7efe04071190 2024-03-09T09:04:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.029+0000 7efe0b800640 1 -- 172.21.15.67:0/2600823320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0414b690 con 0x7efe04071190 2024-03-09T09:04:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.030+0000 7efe0b800640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04149710 0x7efe041ecee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.030+0000 7efe0b800640 1 -- 172.21.15.67:0/2600823320 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe04075a00 con 0x7efe04149710 2024-03-09T09:04:56.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.031+0000 7efe03e00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04149710 0x7efe041ecee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.031+0000 7efe03e00640 1 -- 172.21.15.67:0/2600823320 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe040008d0 con 0x7efe04149710 2024-03-09T09:04:56.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.031+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7efdf800c6d0 con 0x7efe04071190 2024-03-09T09:04:56.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.032+0000 7efe03e00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04149710 0x7efe041ecee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efe041473c0 tx=0x7efdf000e960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:56.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.032+0000 7efe00c00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efde406b4d0 0x7efde406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.032+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7efdf808a3c0 con 0x7efe04071190 2024-03-09T09:04:56.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.033+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00052e0 con 0x7efe04149710 2024-03-09T09:04:56.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.033+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7efdf0018070 con 0x7efe04149710 2024-03-09T09:04:56.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.033+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04149710 msgr2=0x7efe041ecee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.033+0000 7efe00c00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04149710 0x7efe041ecee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efe041473c0 tx=0x7efdf000e960 comp rx=0 tx=0).stop 2024-03-09T09:04:56.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.033+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04149710 msgr2=0x7efe041ecee0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:56.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.033+0000 7efe00c00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04149710 0x7efe041ecee0 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efe041473c0 tx=0x7efdf000e960 comp rx=0 tx=0).stop 2024-03-09T09:04:56.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.033+0000 7efe09800640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efde406b4d0 0x7efde406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.034+0000 7efe09800640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efde406b4d0 0x7efde406d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efdfc035000 tx=0x7efdfc030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:56.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.040+0000 7efe0b800640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04146c70 0x7efe041488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.040+0000 7efe0b800640 1 -- 172.21.15.67:0/2600823320 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe04148e10 con 0x7efe04146c70 2024-03-09T09:04:56.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.040+0000 7efe03e00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04146c70 0x7efe041488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.041+0000 7efe03e00640 1 -- 172.21.15.67:0/2600823320 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efe04067dd0 con 0x7efe04146c70 2024-03-09T09:04:56.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.041+0000 7efe03e00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04146c70 0x7efe041488d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efdf0014000 tx=0x7efdf0001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:56.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.042+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00037f0 con 0x7efe04146c70 2024-03-09T09:04:56.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.042+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7efdf00037f0 con 0x7efe04146c70 2024-03-09T09:04:56.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.042+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04146c70 msgr2=0x7efe041488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.042+0000 7efe00c00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04146c70 0x7efe041488d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efdf0014000 tx=0x7efdf0001080 comp rx=0 tx=0).stop 2024-03-09T09:04:56.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.042+0000 7efe00c00640 1 -- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04146c70 msgr2=0x7efe041488d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:04:56.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.042+0000 7efe00c00640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efe04146c70 0x7efe041488d0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efdf0014000 tx=0x7efdf0001080 comp rx=0 tx=0).stop 2024-03-09T09:04:56.046 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:04:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:04:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:04:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:04:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 92, 2024-03-09T09:04:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:04:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:04:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:04:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:04:56.046 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 18, 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:56.047 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:04:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:04:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:56.048 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:04:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:04:56.050 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:04:56.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:56.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:04:56.054 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:56.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:56.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:04:56.057 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:04:56.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:04:56.058 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:04:56.058 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:04:56.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.043+0000 7efe0b800640 1 -- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efde406b4d0 msgr2=0x7efde406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.043+0000 7efe0b800640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efde406b4d0 0x7efde406d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efdfc035000 tx=0x7efdfc030000 comp rx=0 tx=0).stop 2024-03-09T09:04:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.043+0000 7efe0b800640 1 -- 172.21.15.67:0/2600823320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe04071190 msgr2=0x7efe041ec9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.043+0000 7efe0b800640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe04071190 0x7efe041ec9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdf800e7e0 tx=0x7efdf800eca0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.043+0000 7efe09800640 1 -- 172.21.15.67:0/2600823320 reap_dead start 2024-03-09T09:04:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.043+0000 7efe0b800640 1 -- 172.21.15.67:0/2600823320 shutdown_connections 2024-03-09T09:04:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.043+0000 7efe0b800640 1 --2- 172.21.15.67:0/2600823320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe04071190 0x7efe041ec9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.043+0000 7efe0b800640 1 -- 172.21.15.67:0/2600823320 >> 172.21.15.67:0/2600823320 conn(0x7efe040c7370 msgr2=0x7efe040c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.043+0000 7efe0b800640 1 -- 172.21.15.67:0/2600823320 shutdown_connections 2024-03-09T09:04:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.043+0000 7efe0b800640 1 -- 172.21.15.67:0/2600823320 wait complete. 2024-03-09T09:04:56.062 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:04:56.062 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T09:04:56.062 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:04:56.062 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T09:04:56.063 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:04:56.063 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:04:56.063 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:04:56.081 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:04:56.081 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:04:56.081 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:04:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.287+0000 7f9fc9600640 1 Processor -- start 2024-03-09T09:04:56.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.287+0000 7f9fc9600640 1 -- start start 2024-03-09T09:04:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.288+0000 7f9fc9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9fc4153370 0x7f9fc40c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.288+0000 7f9fc9600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc40c4010 con 0x7f9fc4153750 2024-03-09T09:04:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.288+0000 7f9fc9600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc40c42f0 con 0x7f9fc406dc00 2024-03-09T09:04:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.288+0000 7f9fc9600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc4156dc0 con 0x7f9fc4153370 2024-03-09T09:04:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.288+0000 7f9fc2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9fc4153370 0x7f9fc40c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.289+0000 7f9fc2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9fc4153370 0x7f9fc40c29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40768/0 (socket says 172.21.15.67:40768) 2024-03-09T09:04:56.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.289+0000 7f9fc2a00640 1 -- 172.21.15.67:0/2289474809 learned_addr learned my addr 172.21.15.67:0/2289474809 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:56.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.289+0000 7f9fc2a00640 1 -- 172.21.15.67:0/2289474809 >> v1:172.21.15.134:6789/0 conn(0x7f9fc406dc00 legacy=0x7f9fc40729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.289+0000 7f9fc2a00640 1 -- 172.21.15.67:0/2289474809 >> v1:172.21.15.67:6789/0 conn(0x7f9fc4153750 legacy=0x7f9fc40c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.289+0000 7f9fc2a00640 1 -- 172.21.15.67:0/2289474809 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40c5a50 con 0x7f9fc4153370 2024-03-09T09:04:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.290+0000 7f9fc2a00640 1 --2- 172.21.15.67:0/2289474809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fc4153370 0x7f9fc40c29a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009870 tx=0x7f9fac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbb6845ca8a0f76 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.291+0000 7f9fc2000640 1 -- 172.21.15.67:0/2289474809 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac034070 con 0x7f9fc4153370 2024-03-09T09:04:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.291+0000 7f9fc2000640 1 -- 172.21.15.67:0/2289474809 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac030860 con 0x7f9fc4153370 2024-03-09T09:04:56.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.291+0000 7f9fc2000640 1 -- 172.21.15.67:0/2289474809 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac030c30 con 0x7f9fc4153370 2024-03-09T09:04:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.291+0000 7f9fc9600640 1 -- 172.21.15.67:0/2289474809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fc4153370 msgr2=0x7f9fc40c29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.291+0000 7f9fc9600640 1 --2- 172.21.15.67:0/2289474809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fc4153370 0x7f9fc40c29a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009870 tx=0x7f9fac02b9a0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.291+0000 7f9fc9600640 1 -- 172.21.15.67:0/2289474809 shutdown_connections 2024-03-09T09:04:56.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.291+0000 7f9fc9600640 1 --2- 172.21.15.67:0/2289474809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fc4153370 0x7f9fc40c29a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.291+0000 7f9fc9600640 1 -- 172.21.15.67:0/2289474809 >> 172.21.15.67:0/2289474809 conn(0x7f9fc40ab320 msgr2=0x7f9fc40ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:56.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.291+0000 7f9fc9600640 1 -- 172.21.15.67:0/2289474809 shutdown_connections 2024-03-09T09:04:56.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.291+0000 7f9fc9600640 1 -- 172.21.15.67:0/2289474809 wait complete. 2024-03-09T09:04:56.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.294+0000 7f9fc9600640 1 Processor -- start 2024-03-09T09:04:56.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.294+0000 7f9fc9600640 1 -- start start 2024-03-09T09:04:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fc406dc00 0x7f9fc4151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fc4153370 0x7f9fc41ef640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc9600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fc4153750 0x7f9fc41efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc9600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40c5d30 con 0x7f9fc406dc00 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc9600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc4073dc0 con 0x7f9fc4153370 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc9600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc40730d0 con 0x7f9fc4153750 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fc406dc00 0x7f9fc4151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fc406dc00 0x7f9fc4151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40776/0 (socket says 172.21.15.67:40776) 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc2a00640 1 -- 172.21.15.67:0/2433920705 learned_addr learned my addr 172.21.15.67:0/2433920705 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc3400640 1 -- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fc4153750 msgr2=0x7f9fc41efb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc3400640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fc4153750 0x7f9fc41efb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fba000640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fc4153370 0x7f9fc41ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc2a00640 1 -- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fc4153750 msgr2=0x7f9fc41efb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc2a00640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fc4153750 0x7f9fc41efb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc2a00640 1 -- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fc4153370 msgr2=0x7f9fc41ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc2a00640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fc4153370 0x7f9fc41ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fc2a00640 1 -- 172.21.15.67:0/2433920705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc4154000 con 0x7f9fc406dc00 2024-03-09T09:04:56.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.296+0000 7f9fba000640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fc4153370 0x7f9fc41ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:04:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.297+0000 7f9fc2a00640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fc406dc00 0x7f9fc4151680 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9fac02b9e0 tx=0x7f9fac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.297+0000 7f9fbb400640 1 -- 172.21.15.67:0/2433920705 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac048070 con 0x7f9fc406dc00 2024-03-09T09:04:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.297+0000 7f9fbb400640 1 -- 172.21.15.67:0/2433920705 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac039d90 con 0x7f9fc406dc00 2024-03-09T09:04:56.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.297+0000 7f9fbb400640 1 -- 172.21.15.67:0/2433920705 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac034040 con 0x7f9fc406dc00 2024-03-09T09:04:56.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.297+0000 7f9fc9600640 1 -- 172.21.15.67:0/2433920705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc4154270 con 0x7f9fc406dc00 2024-03-09T09:04:56.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.297+0000 7f9fc9600640 1 -- 172.21.15.67:0/2433920705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc4151bc0 con 0x7f9fc406dc00 2024-03-09T09:04:56.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.298+0000 7f9fc9600640 1 -- 172.21.15.67:0/2433920705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc4075a00 con 0x7f9fc406dc00 2024-03-09T09:04:56.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.299+0000 7f9fbb400640 1 -- 172.21.15.67:0/2433920705 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9fac04b070 con 0x7f9fc406dc00 2024-03-09T09:04:56.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.301+0000 7f9fbb400640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9fa006b480 0x7f9fa006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:56.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.301+0000 7f9fbb400640 1 -- 172.21.15.67:0/2433920705 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9fac0ac680 con 0x7f9fc406dc00 2024-03-09T09:04:56.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.301+0000 7f9fba000640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9fa006b480 0x7f9fa006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:56.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.302+0000 7f9fba000640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9fa006b480 0x7f9fa006d940 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40c9320 tx=0x7f9fb4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:56.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.305+0000 7f9fbb400640 1 -- 172.21.15.67:0/2433920705 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9fac07b1a0 con 0x7f9fc406dc00 2024-03-09T09:04:56.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.578+0000 7f9fc9600640 1 -- 172.21.15.67:0/2433920705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9fc406c320 con 0x7f9fc406dc00 2024-03-09T09:04:56.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.579+0000 7f9fbb400640 1 -- 172.21.15.67:0/2433920705 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f9fac07aaa0 con 0x7f9fc406dc00 2024-03-09T09:04:56.582 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":92,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:04:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 -- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9fa006b480 msgr2=0x7f9fa006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9fa006b480 0x7f9fa006d940 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40c9320 tx=0x7f9fb4009040 comp rx=0 tx=0).stop 2024-03-09T09:04:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 -- 172.21.15.67:0/2433920705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fc406dc00 msgr2=0x7f9fc4151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fc406dc00 0x7f9fc4151680 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9fac02b9e0 tx=0x7f9fac004730 comp rx=0 tx=0).stop 2024-03-09T09:04:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 -- 172.21.15.67:0/2433920705 shutdown_connections 2024-03-09T09:04:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9fa006b480 0x7f9fa006d940 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fc4153750 0x7f9fc41efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fc4153370 0x7f9fc41ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 --2- 172.21.15.67:0/2433920705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9fc406dc00 0x7f9fc4151680 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 -- 172.21.15.67:0/2433920705 >> 172.21.15.67:0/2433920705 conn(0x7f9fc40ab320 msgr2=0x7f9fc40ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:56.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 -- 172.21.15.67:0/2433920705 shutdown_connections 2024-03-09T09:04:56.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:56.582+0000 7f9fc9600640 1 -- 172.21.15.67:0/2433920705 wait complete. 2024-03-09T09:04:58.369 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 0% (43/13130) Updating files: 1% (132/13130) Updating files: 1% (152/13130) Updating files: 1% (248/13130) Updating files: 2% (263/13130) Updating files: 2% (278/13130) Updating files: 2% (352/13130) Updating files: 3% (394/13130) Updating files: 3% (432/13130) Updating files: 3% (513/13130) Updating files: 3% (514/13130) Updating files: 4% (526/13130) Updating files: 4% (602/13130) Updating files: 4% (637/13130) Updating files: 5% (657/13130) Updating files: 5% (787/13130) Updating files: 6% (788/13130) Updating files: 6% (892/13130) Updating files: 7% (920/13130) Updating files: 7% (975/13130) Updating files: 7% (1002/13130) Updating files: 8% (1051/13130) Updating files: 8% (1092/13130) Updating files: 8% (1156/13130) Updating files: 9% (1182/13130) Updating files: 9% (1189/13130) Updating files: 9% (1296/13130) Updating files: 10% (1313/13130) Updating files: 10% (1355/13130) Updating files: 11% (1445/13130) Updating files: 11% (1463/13130) Updating files: 11% (1539/13130) Updating files: 12% (1576/13130) Updating files: 12% (1641/13130) Updating files: 13% (1707/13130) Updating files: 13% (1744/13130) Updating files: 13% (1799/13130) Updating files: 14% (1839/13130) Updating files: 14% (1909/13130) Updating files: 14% (1928/13130) Updating files: 15% (1970/13130) Updating files: 15% (2015/13130) Updating files: 16% (2101/13130) Updating files: 16% (2121/13130) Updating files: 16% (2201/13130) Updating files: 17% (2233/13130) Updating files: 17% (2241/13130) Updating files: 17% (2331/13130) Updating files: 17% (2350/13130) Updating files: 18% (2364/13130) Updating files: 18% (2393/13130) Updating files: 18% (2436/13130) Updating files: 18% (2477/13130) Updating files: 19% (2495/13130) Updating files: 19% (2586/13130) Updating files: 19% (2601/13130) Updating files: 20% (2626/13130) Updating files: 20% (2636/13130) Updating files: 20% (2719/13130) Updating files: 21% (2758/13130) Updating files: 21% (2866/13130) Updating files: 22% (2889/13130) Updating files: 22% (2913/13130) Updating files: 23% (3020/13130) Updating files: 23% (3048/13130) Updating files: 23% (3066/13130) Updating files: 24% (3152/13130) Updating files: 24% (3164/13130) Updating files: 25% (3283/13130) Updating files: 25% (3338/13130) Updating files: 25% (3380/13130) Updating files: 26% (3414/13130) Updating files: 26% (3501/13130) Updating files: 27% (3546/13130) Updating files: 27% (3614/13130) Updating files: 27% (3655/13130) Updating files: 28% (3677/13130) Updating files: 28% (3690/13130) Updating files: 28% (3722/13130) Updating files: 29% (3808/13130) Updating files: 29% (3821/13130) Updating files: 29% (3891/13130) Updating files: 29% (3908/13130) Updating files: 30% (3939/13130) Updating files: 30% (3963/13130) Updating files: 30% (4016/13130) Updating files: 30% (4048/13130) Updating files: 31% (4071/13130) Updating files: 31% (4112/13130) Updating files: 31% (4175/13130) Updating files: 32% (4202/13130) Updating files: 32% (4216/13130) Updating files: 32% (4244/13130) Updating files: 32% (4246/13130) Updating files: 33% (4333/13130) Updating files: 33% (4344/13130) Updating files: 33% (4450/13130) Updating files: 34% (4465/13130) Updating files: 34% (4469/13130) Updating files: 34% (4571/13130) Updating files: 35% (4596/13130) Updating files: 35% (4666/13130) Updating files: 35% (4712/13130) Updating files: 36% (4727/13130) Updating files: 36% (4730/13130) Updating files: 36% (4846/13130) Updating files: 37% (4859/13130) Updating files: 37% (4945/13130) Updating files: 38% (4990/13130) Updating files: 39% (5121/13130) Updating files: 39% (5131/13130) Updating files: 40% (5252/13130) Updating files: 40% (5261/13130) Updating files: 40% (5318/13130) Updating files: 41% (5384/13130) Updating files: 41% (5469/13130) Updating files: 42% (5515/13130) Updating files: 42% (5547/13130) Updating files: 42% (5609/13130) Updating files: 43% (5646/13130) Updating files: 43% (5657/13130) Updating files: 44% (5778/13130) Updating files: 45% (5909/13130) Updating files: 45% (5997/13130) Updating files: 45% (6037/13130) Updating files: 46% (6040/13130) Updating files: 46% (6171/13130) Updating files: 47% (6172/13130) Updating files: 47% (6203/13130) Updating files: 47% (6217/13130) Updating files: 48% (6303/13130) Updating files: 48% (6330/13130) Updating files: 48% (6378/13130) Updating files: 49% (6434/13130) Updating files: 49% (6487/13130) Updating files: 49% (6545/13130) Updating files: 50% (6565/13130) Updating files: 50% (6595/13130) Updating files: 50% (6689/13130) Updating files: 51% (6697/13130) Updating files: 52% (6828/13130) Updating files: 52% (6894/13130) Updating files: 53% (6959/13130) Updating files: 53% (7046/13130) Updating files: 53% (7065/13130) Updating files: 54% (7091/13130) Updating files: 55% (7222/13130) Updating files: 55% (7276/13130) Updating files: 55% (7311/13130) Updating files: 56% (7353/13130) Updating files: 56% (7482/13130) Updating files: 57% (7485/13130) Updating files: 57% (7508/13130) Updating files: 57% (7595/13130) Updating files: 58% (7616/13130) Updating files: 59% (7747/13130) Updating files: 59% (7753/13130) Updating files: 59% (7845/13130) Updating files: 59% (7846/13130) Updating files: 60% (7878/13130) Updating files: 60% (7971/13130) Updating files: 61% (8010/13130) Updating files: 61% (8014/13130) Updating files: 61% (8062/13130) Updating files: 61% (8128/13130) Updating files: 62% (8141/13130) Updating files: 62% (8254/13130) Updating files: 63% (8272/13130) Updating files: 64% (8404/13130) Updating files: 64% (8414/13130) Updating files: 64% (8428/13130) Updating files: 64% (8478/13130) Updating files: 64% (8518/13130) Updating files: 65% (8535/13130) Updating files: 65% (8604/13130) Updating files: 66% (8666/13130) Updating files: 66% (8710/13130) Updating files: 66% (8732/13130) Updating files: 66% (8788/13130) Updating files: 67% (8798/13130) Updating files: 67% (8905/13130) Updating files: 68% (8929/13130) Updating files: 68% (8937/13130) Updating files: 68% (8963/13130) Updating files: 68% (8994/13130) Updating files: 68% (9020/13130) Updating files: 69% (9060/13130) Updating files: 69% (9141/13130) Updating files: 70% (9191/13130) Updating files: 71% (9323/13130) Updating files: 71% (9355/13130) Updating files: 71% (9428/13130) Updating files: 72% (9454/13130) Updating files: 72% (9473/13130) Updating files: 72% (9499/13130) Updating files: 72% (9565/13130) Updating files: 73% (9585/13130) Updating files: 73% (9665/13130) Updating files: 73% (9703/13130) Updating files: 74% (9717/13130) Updating files: 74% (9770/13130) Updating files: 74% (9822/13130) Updating files: 75% (9848/13130) Updating files: 75% (9872/13130) Updating files: 75% (9925/13130) Updating files: 75% (9944/13130) Updating files: 76% (9979/13130) Updating files: 76% (10052/13130) Updating files: 77% (10111/13130) Updating files: 77% (10220/13130) Updating files: 77% (10232/13130) Updating files: 78% (10242/13130) Updating files: 78% (10278/13130) Updating files: 79% (10373/13130) Updating files: 79% (10460/13130) Updating files: 79% (10502/13130) Updating files: 80% (10504/13130) Updating files: 80% (10518/13130) Updating files: 80% (10578/13130) Updating files: 81% (10636/13130) Updating files: 81% (10664/13130) Updating files: 81% (10736/13130) Updating files: 82% (10767/13130) Updating files: 82% (10801/13130) Updating files: 82% (10887/13130) Updating files: 83% (10898/13130) Updating files: 83% (10919/13130) Updating files: 84% (11030/13130) Updating files: 84% (11034/13130) Updating files: 84% (11122/13130) Updating files: 85% (11161/13130) Updating files: 85% (11223/13130) Updating files: 85% (11275/13130) Updating files: 86% (11292/13130) Updating files: 86% (11297/13130) Updating files: 86% (11377/13130) Updating files: 87% (11424/13130) Updating files: 87% (11436/13130) Updating files: 87% (11466/13130) Updating files: 87% (11486/13130) Updating files: 87% (11513/13130) Updating files: 88% (11555/13130) Updating files: 89% (11686/13130) Updating files: 89% (11704/13130) Updating files: 89% (11759/13130) Updating files: 90% (11817/13130) Updating files: 90% (11932/13130) Updating files: 91% (11949/13130) Updating files: 91% (12066/13130) Updating files: 92% (12080/13130) Updating files: 92% (12164/13130) Updating files: 93% (12211/13130) Updating files: 93% (12223/13130) Updating files: 94% (12343/13130) Updating files: 94% (12354/13130) Updating files: 95% (12474/13130) Updating files: 95% (12532/13130) Updating files: 96% (12605/13130) Updating files: 96% (12730/13130) Updating files: 97% (12737/13130) Updating files: 97% (12777/13130) Updating files: 97% (12858/13130) Updating files: 98% (12868/13130) Updating files: 98% (12987/13130) Updating files: 99% (12999/13130) Updating files: 100% (13130/13130) Updating files: 100% (13130/13130), done. 2024-03-09T09:04:58.863 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 204c175eea3 msg: blobhash on entity_name_t causes strange badness... should look into that, but avoid for now 2024-03-09T09:04:58.889 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.1 2024-03-09T09:04:59.359 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:04:59.359 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 2 -eq 48 ']' 2024-03-09T09:04:59.359 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.2 2024-03-09T09:04:59.359 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.2 2024-03-09T09:04:59.359 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.2 2024-03-09T09:04:59.359 INFO:tasks.workunit.client.1.smithi134.stdout:v0.2 2024-03-09T09:04:59.361 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 0% (63/13130) Updating files: 1% (132/13130) Updating files: 1% (251/13130) Updating files: 2% (263/13130) Updating files: 2% (299/13130) Updating files: 2% (336/13130) Updating files: 2% (352/13130) Updating files: 3% (394/13130) Updating files: 3% (422/13130) Updating files: 4% (526/13130) Updating files: 4% (529/13130) Updating files: 4% (615/13130) Updating files: 5% (657/13130) Updating files: 5% (734/13130) Updating files: 6% (788/13130) Updating files: 7% (920/13130) Updating files: 7% (941/13130) Updating files: 7% (977/13130) Updating files: 7% (1006/13130) Updating files: 7% (1046/13130) Updating files: 8% (1051/13130) Updating files: 8% (1150/13130) Updating files: 9% (1182/13130) Updating files: 9% (1214/13130) Updating files: 9% (1290/13130) Updating files: 10% (1313/13130) Updating files: 10% (1321/13130) Updating files: 10% (1331/13130) Updating files: 11% (1445/13130) Updating files: 11% (1461/13130) Updating files: 11% (1534/13130) Updating files: 12% (1576/13130) Updating files: 12% (1609/13130) Updating files: 12% (1649/13130) Updating files: 13% (1707/13130) Updating files: 13% (1733/13130) Updating files: 13% (1741/13130) Updating files: 13% (1765/13130) Updating files: 13% (1830/13130) Updating files: 14% (1839/13130) Updating files: 14% (1911/13130) Updating files: 14% (1924/13130) Updating files: 14% (1932/13130) Updating files: 15% (1970/13130) Updating files: 15% (2043/13130) Updating files: 16% (2101/13130) Updating files: 16% (2162/13130) Updating files: 16% (2216/13130) Updating files: 17% (2233/13130) Updating files: 17% (2248/13130) Updating files: 17% (2306/13130) Updating files: 18% (2364/13130) Updating files: 18% (2399/13130) Updating files: 18% (2412/13130) Updating files: 18% (2459/13130) Updating files: 18% (2477/13130) Updating files: 19% (2495/13130) Updating files: 19% (2548/13130) Updating files: 19% (2573/13130) Updating files: 19% (2601/13130) Updating files: 20% (2626/13130) Updating files: 20% (2670/13130) Updating files: 20% (2737/13130) Updating files: 21% (2758/13130) Updating files: 22% (2889/13130) Updating files: 22% (2891/13130) Updating files: 22% (2913/13130) Updating files: 22% (3013/13130) Updating files: 23% (3020/13130) Updating files: 23% (3067/13130) Updating files: 23% (3134/13130) Updating files: 24% (3152/13130) Updating files: 24% (3236/13130) Updating files: 25% (3283/13130) Updating files: 25% (3369/13130) Updating files: 26% (3414/13130) Updating files: 26% (3485/13130) Updating files: 27% (3546/13130) Updating files: 27% (3558/13130) Updating files: 27% (3565/13130) Updating files: 27% (3653/13130) Updating files: 27% (3676/13130) Updating files: 28% (3677/13130) Updating files: 28% (3754/13130) Updating files: 28% (3787/13130) Updating files: 29% (3808/13130) Updating files: 29% (3816/13130) Updating files: 29% (3852/13130) Updating files: 29% (3891/13130) Updating files: 30% (3939/13130) Updating files: 30% (3977/13130) Updating files: 30% (3990/13130) Updating files: 30% (4016/13130) Updating files: 30% (4068/13130) Updating files: 31% (4071/13130) Updating files: 31% (4175/13130) Updating files: 32% (4202/13130) Updating files: 33% (4333/13130) Updating files: 33% (4343/13130) Updating files: 33% (4437/13130) Updating files: 34% (4465/13130) Updating files: 34% (4541/13130) Updating files: 35% (4596/13130) Updating files: 35% (4711/13130) Updating files: 36% (4727/13130) Updating files: 36% (4734/13130) Updating files: 36% (4828/13130) Updating files: 37% (4859/13130) Updating files: 37% (4934/13130) Updating files: 38% (4990/13130) Updating files: 38% (5072/13130) Updating files: 39% (5121/13130) Updating files: 39% (5180/13130) Updating files: 39% (5250/13130) Updating files: 40% (5252/13130) Updating files: 40% (5290/13130) Updating files: 41% (5384/13130) Updating files: 41% (5433/13130) Updating files: 41% (5469/13130) Updating files: 41% (5497/13130) Updating files: 42% (5515/13130) Updating files: 42% (5528/13130) Updating files: 42% (5536/13130) Updating files: 43% (5646/13130) Updating files: 43% (5749/13130) Updating files: 44% (5778/13130) Updating files: 45% (5909/13130) Updating files: 45% (5941/13130) Updating files: 45% (5994/13130) Updating files: 46% (6040/13130) Updating files: 46% (6094/13130) Updating files: 47% (6172/13130) Updating files: 48% (6303/13130) Updating files: 48% (6330/13130) Updating files: 49% (6434/13130) Updating files: 49% (6455/13130) Updating files: 49% (6558/13130) Updating files: 50% (6565/13130) Updating files: 50% (6589/13130) Updating files: 50% (6599/13130) Updating files: 50% (6684/13130) Updating files: 51% (6697/13130) Updating files: 52% (6828/13130) Updating files: 52% (6833/13130) Updating files: 52% (6928/13130) Updating files: 53% (6959/13130) Updating files: 53% (6999/13130) Updating files: 53% (7089/13130) Updating files: 54% (7091/13130) Updating files: 54% (7118/13130) Updating files: 55% (7222/13130) Updating files: 55% (7348/13130) Updating files: 56% (7353/13130) Updating files: 56% (7395/13130) Updating files: 56% (7440/13130) Updating files: 57% (7485/13130) Updating files: 58% (7616/13130) Updating files: 58% (7666/13130) Updating files: 59% (7747/13130) Updating files: 59% (7783/13130) Updating files: 59% (7853/13130) Updating files: 60% (7878/13130) Updating files: 60% (7913/13130) Updating files: 61% (8010/13130) Updating files: 61% (8034/13130) Updating files: 61% (8100/13130) Updating files: 62% (8141/13130) Updating files: 62% (8181/13130) Updating files: 62% (8203/13130) Updating files: 63% (8272/13130) Updating files: 63% (8299/13130) Updating files: 63% (8352/13130) Updating files: 63% (8356/13130) Updating files: 63% (8382/13130) Updating files: 64% (8404/13130) Updating files: 64% (8482/13130) Updating files: 65% (8535/13130) Updating files: 65% (8637/13130) Updating files: 66% (8666/13130) Updating files: 66% (8673/13130) Updating files: 66% (8739/13130) Updating files: 66% (8765/13130) Updating files: 67% (8798/13130) Updating files: 67% (8808/13130) Updating files: 67% (8838/13130) Updating files: 68% (8929/13130) Updating files: 68% (8992/13130) Updating files: 68% (9037/13130) Updating files: 69% (9060/13130) Updating files: 69% (9071/13130) Updating files: 69% (9113/13130) Updating files: 70% (9191/13130) Updating files: 70% (9198/13130) Updating files: 70% (9268/13130) Updating files: 71% (9323/13130) Updating files: 71% (9420/13130) Updating files: 72% (9454/13130) Updating files: 72% (9545/13130) Updating files: 73% (9585/13130) Updating files: 73% (9652/13130) Updating files: 74% (9717/13130) Updating files: 74% (9719/13130) Updating files: 74% (9789/13130) Updating files: 74% (9846/13130) Updating files: 75% (9848/13130) Updating files: 75% (9889/13130) Updating files: 75% (9944/13130) Updating files: 75% (9975/13130) Updating files: 76% (9979/13130) Updating files: 76% (9994/13130) Updating files: 76% (10037/13130) Updating files: 76% (10093/13130) Updating files: 77% (10111/13130) Updating files: 78% (10242/13130) Updating files: 78% (10348/13130) Updating files: 79% (10373/13130) Updating files: 79% (10459/13130) Updating files: 80% (10504/13130) Updating files: 81% (10636/13130) Updating files: 81% (10655/13130) Updating files: 81% (10736/13130) Updating files: 82% (10767/13130) Updating files: 82% (10836/13130) Updating files: 83% (10898/13130) Updating files: 83% (10930/13130) Updating files: 84% (11030/13130) Updating files: 84% (11094/13130) Updating files: 84% (11151/13130) Updating files: 85% (11161/13130) Updating files: 85% (11237/13130) Updating files: 86% (11292/13130) Updating files: 86% (11316/13130) Updating files: 87% (11424/13130) Updating files: 87% (11439/13130) Updating files: 88% (11555/13130) Updating files: 88% (11573/13130) Updating files: 88% (11632/13130) Updating files: 89% (11686/13130) Updating files: 89% (11704/13130) Updating files: 89% (11791/13130) Updating files: 90% (11817/13130) Updating files: 90% (11846/13130) Updating files: 90% (11946/13130) Updating files: 91% (11949/13130) Updating files: 91% (11978/13130) Updating files: 91% (12052/13130) Updating files: 92% (12080/13130) Updating files: 92% (12144/13130) Updating files: 92% (12198/13130) Updating files: 93% (12211/13130) Updating files: 94% (12343/13130) Updating files: 95% (12474/13130) Updating files: 95% (12498/13130) Updating files: 96% (12605/13130) Updating files: 96% (12735/13130) Updating files: 97% (12737/13130) Updating files: 98% (12868/13130) Updating files: 98% (12912/13130) Updating files: 99% (12999/13130) Updating files: 100% (13130/13130) Updating files: 100% (13130/13130), done. 2024-03-09T09:04:59.603 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:04:59.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.814+0000 7f04a7e00640 1 Processor -- start 2024-03-09T09:04:59.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.814+0000 7f04a7e00640 1 -- start start 2024-03-09T09:04:59.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.816+0000 7f04a7e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f04a8153300 0x7f04a80b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:59.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.816+0000 7f04a7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a8157000 con 0x7f04a80c5e40 2024-03-09T09:04:59.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.816+0000 7f04a7e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a80ae430 con 0x7f04a80cadb0 2024-03-09T09:04:59.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.816+0000 7f04a7e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a80ae6f0 con 0x7f04a8153300 2024-03-09T09:04:59.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.816+0000 7f04a6000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f04a80c5e40 0x7f04a8156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:32936/0 (socket says 172.21.15.67:32936) 2024-03-09T09:04:59.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.816+0000 7f04a6000640 1 -- 172.21.15.67:0/119119113 learned_addr learned my addr 172.21.15.67:0/119119113 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:04:59.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.816+0000 7f04a6a00640 1 --2- 172.21.15.67:0/119119113 >> v2:172.21.15.134:3300/0 conn(0x7f04a8153300 0x7f04a80b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:59.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715760833 0 0) 0x7f04a8157000 con 0x7f04a80c5e40 2024-03-09T09:04:59.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f048c003520 con 0x7f04a80c5e40 2024-03-09T09:04:59.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 340297500 0 0) 0x7f048c003520 con 0x7f04a80c5e40 2024-03-09T09:04:59.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 >> v2:172.21.15.134:3300/0 conn(0x7f04a8153300 msgr2=0x7f04a80b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:59.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 --2- 172.21.15.67:0/119119113 >> v2:172.21.15.134:3300/0 conn(0x7f04a8153300 0x7f04a80b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:59.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 >> v1:172.21.15.134:6789/0 conn(0x7f04a80cadb0 legacy=0x7f04a8160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:59.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a8155b10 con 0x7f04a80c5e40 2024-03-09T09:04:59.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f04940019f0 con 0x7f04a80c5e40 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 >> v1:172.21.15.67:6789/0 conn(0x7f04a80c5e40 legacy=0x7f04a8156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 --2- 172.21.15.67:0/119119113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f048c003a70 0x7f048c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 --2- 172.21.15.67:0/119119113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 0x7f048c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 --2- 172.21.15.67:0/119119113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f048c005210 0x7f048c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80ae6f0 con 0x7f048c004800 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c00c140 con 0x7f048c005210 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.817+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c00c340 con 0x7f048c003a70 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.818+0000 7f04a6a00640 1 -- 172.21.15.67:0/119119113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f048c003a70 msgr2=0x7f048c003e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.818+0000 7f04a6a00640 1 --2- 172.21.15.67:0/119119113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f048c003a70 0x7f048c003e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.818+0000 7f04a6000640 1 --2- 172.21.15.67:0/119119113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 0x7f048c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.818+0000 7f04a7400640 1 --2- 172.21.15.67:0/119119113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f048c005210 0x7f048c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.818+0000 7f04a6000640 1 -- 172.21.15.67:0/119119113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f048c003a70 msgr2=0x7f048c003e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:04:59.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.818+0000 7f04a6000640 1 --2- 172.21.15.67:0/119119113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f048c003a70 0x7f048c003e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:59.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.818+0000 7f04a6000640 1 -- 172.21.15.67:0/119119113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f048c005210 msgr2=0x7f048c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:04:59.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.818+0000 7f04a6000640 1 --2- 172.21.15.67:0/119119113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f048c005210 0x7f048c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:59.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.818+0000 7f04a7400640 1 -- 172.21.15.67:0/119119113 reap_dead start 2024-03-09T09:04:59.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.818+0000 7f04a6000640 1 -- 172.21.15.67:0/119119113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c003520 con 0x7f048c004800 2024-03-09T09:04:59.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.820+0000 7f04a6000640 1 --2- 172.21.15.67:0/119119113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 0x7f048c004c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f049400a790 tx=0x7f048c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b062caa88a459053 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:59.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.820+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049400a3b0 con 0x7f048c004800 2024-03-09T09:04:59.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.820+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0494034650 con 0x7f048c004800 2024-03-09T09:04:59.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.820+0000 7f04a5600640 1 -- 172.21.15.67:0/119119113 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494034a40 con 0x7f048c004800 2024-03-09T09:04:59.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.821+0000 7f04a7e00640 1 -- 172.21.15.67:0/119119113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 msgr2=0x7f048c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:59.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.821+0000 7f04a7e00640 1 --2- 172.21.15.67:0/119119113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 0x7f048c004c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f049400a790 tx=0x7f048c005830 comp rx=0 tx=0).stop 2024-03-09T09:04:59.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.821+0000 7f04a7e00640 1 -- 172.21.15.67:0/119119113 shutdown_connections 2024-03-09T09:04:59.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.821+0000 7f04a7e00640 1 --2- 172.21.15.67:0/119119113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 0x7f048c004c80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:59.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.821+0000 7f04a7e00640 1 -- 172.21.15.67:0/119119113 >> 172.21.15.67:0/119119113 conn(0x7f04a80732d0 msgr2=0x7f04a80ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:04:59.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.821+0000 7f04a7e00640 1 -- 172.21.15.67:0/119119113 shutdown_connections 2024-03-09T09:04:59.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.821+0000 7f04a7e00640 1 -- 172.21.15.67:0/119119113 wait complete. 2024-03-09T09:04:59.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.823+0000 7f04a7e00640 1 Processor -- start 2024-03-09T09:04:59.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.823+0000 7f04a7e00640 1 -- start start 2024-03-09T09:04:59.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.824+0000 7f04a7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 0x7f04a806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:59.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.824+0000 7f04a7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a80aafa0 0x7f04a81eb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:59.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.824+0000 7f04a7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a806c070 0x7f04a81ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.824+0000 7f04a7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80c59e0 con 0x7f048c004800 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.824+0000 7f04a7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a8159e90 con 0x7f04a80aafa0 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.824+0000 7f04a7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80b0000 con 0x7f04a806c070 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.824+0000 7f04a7400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a806c070 msgr2=0x7f04a81ebe50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.824+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 0x7f04a806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.824+0000 7f04a7400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a806c070 0x7f04a81ebe50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.824+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 0x7f04a806bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60184/0 (socket says 172.21.15.67:60184) 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.824+0000 7f04a6a00640 1 -- 172.21.15.67:0/2677355768 learned_addr learned my addr 172.21.15.67:0/2677355768 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.825+0000 7f04a6a00640 1 -- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a806c070 msgr2=0x7f04a81ebe50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.825+0000 7f04a6a00640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a806c070 0x7f04a81ebe50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.825+0000 7f04a6000640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a80aafa0 0x7f04a81eb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.825+0000 7f04a6a00640 1 -- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a80aafa0 msgr2=0x7f04a81eb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:04:59.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.825+0000 7f04a6a00640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a80aafa0 0x7f04a81eb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:04:59.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.825+0000 7f04a6a00640 1 -- 172.21.15.67:0/2677355768 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a80c7fb0 con 0x7f048c004800 2024-03-09T09:04:59.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.825+0000 7f04a6a00640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 0x7f04a806bb30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f049800e9e0 tx=0x7f049800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:59.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.826+0000 7f049f400640 1 -- 172.21.15.67:0/2677355768 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498017070 con 0x7f048c004800 2024-03-09T09:04:59.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.826+0000 7f049f400640 1 -- 172.21.15.67:0/2677355768 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049800c9d0 con 0x7f048c004800 2024-03-09T09:04:59.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.826+0000 7f04a7e00640 1 -- 172.21.15.67:0/2677355768 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a80c6d60 con 0x7f048c004800 2024-03-09T09:04:59.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.826+0000 7f049f400640 1 -- 172.21.15.67:0/2677355768 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049800cdc0 con 0x7f048c004800 2024-03-09T09:04:59.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.826+0000 7f04a7e00640 1 -- 172.21.15.67:0/2677355768 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a80ace40 con 0x7f048c004800 2024-03-09T09:04:59.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.827+0000 7f04a7e00640 1 -- 172.21.15.67:0/2677355768 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a8150e20 con 0x7f048c004800 2024-03-09T09:04:59.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.828+0000 7f049f400640 1 -- 172.21.15.67:0/2677355768 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f049801a070 con 0x7f048c004800 2024-03-09T09:04:59.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.829+0000 7f049f400640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f048806b4b0 0x7f048806d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:04:59.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.829+0000 7f049f400640 1 -- 172.21.15.67:0/2677355768 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f049808a990 con 0x7f048c004800 2024-03-09T09:04:59.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.829+0000 7f04a6000640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f048806b4b0 0x7f048806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:04:59.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.830+0000 7f04a6000640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f048806b4b0 0x7f048806d970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0494034000 tx=0x7f049402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:04:59.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:04:59.839+0000 7f049f400640 1 -- 172.21.15.67:0/2677355768 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f04a8150e20 con 0x7f048c004800 2024-03-09T09:04:59.923 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 204c175eea3 msg: blobhash on entity_name_t causes strange badness... should look into that, but avoid for now 2024-03-09T09:04:59.948 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.1 2024-03-09T09:05:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.297+0000 7f04a7e00640 1 -- 172.21.15.67:0/2677355768 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f04a804f3c0 con 0x7f048c004800 2024-03-09T09:05:00.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.298+0000 7f049f400640 1 -- 172.21.15.67:0/2677355768 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f0498059410 con 0x7f048c004800 2024-03-09T09:05:00.301 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":92,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:05:00.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.300+0000 7f04a7e00640 1 -- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f048806b4b0 msgr2=0x7f048806d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.300+0000 7f04a7e00640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f048806b4b0 0x7f048806d970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0494034000 tx=0x7f049402f000 comp rx=0 tx=0).stop 2024-03-09T09:05:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.300+0000 7f04a7e00640 1 -- 172.21.15.67:0/2677355768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 msgr2=0x7f04a806bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.300+0000 7f04a7e00640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 0x7f04a806bb30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f049800e9e0 tx=0x7f049800eea0 comp rx=0 tx=0).stop 2024-03-09T09:05:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.300+0000 7f04a7e00640 1 -- 172.21.15.67:0/2677355768 shutdown_connections 2024-03-09T09:05:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.300+0000 7f04a7e00640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f048806b4b0 0x7f048806d970 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.300+0000 7f04a7e00640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04a806c070 0x7f04a81ebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.300+0000 7f04a7e00640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04a80aafa0 0x7f04a81eb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.300+0000 7f04a7e00640 1 --2- 172.21.15.67:0/2677355768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f048c004800 0x7f04a806bb30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.301+0000 7f04a7e00640 1 -- 172.21.15.67:0/2677355768 >> 172.21.15.67:0/2677355768 conn(0x7f04a80732d0 msgr2=0x7f04a81515f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.301+0000 7f04a7e00640 1 -- 172.21.15.67:0/2677355768 shutdown_connections 2024-03-09T09:05:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:00.301+0000 7f04a7e00640 1 -- 172.21.15.67:0/2677355768 wait complete. 2024-03-09T09:05:00.340 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:05:00.340 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 2 -eq 48 ']' 2024-03-09T09:05:00.341 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.2 2024-03-09T09:05:00.341 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.2 2024-03-09T09:05:00.341 INFO:tasks.workunit.client.0.smithi067.stdout:v0.2 2024-03-09T09:05:00.341 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.2 2024-03-09T09:05:02.806 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 77% (173/222) Updating files: 78% (174/222) Updating files: 79% (176/222) Updating files: 80% (178/222) Updating files: 81% (180/222) Updating files: 82% (183/222) Updating files: 83% (185/222) Updating files: 84% (187/222) Updating files: 85% (189/222) Updating files: 86% (191/222) Updating files: 87% (194/222) Updating files: 88% (196/222) Updating files: 89% (198/222) Updating files: 90% (200/222) Updating files: 91% (203/222) Updating files: 92% (205/222) Updating files: 93% (207/222) Updating files: 94% (209/222) Updating files: 95% (211/222) Updating files: 96% (214/222) Updating files: 97% (216/222) Updating files: 97% (217/222) Updating files: 98% (218/222) Updating files: 99% (220/222) Updating files: 100% (222/222) Updating files: 100% (222/222), done. 2024-03-09T09:05:02.857 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 2b5f76f48e9 readme 2024-03-09T09:05:02.874 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.2 2024-03-09T09:05:03.322 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:05:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.525+0000 7fd0a3e00640 1 Processor -- start 2024-03-09T09:05:03.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.526+0000 7fd0a3e00640 1 -- start start 2024-03-09T09:05:03.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.527+0000 7fd0a3e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd0a40c1f70 0x7fd0a4150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.527+0000 7fd0a3e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a40c81e0 con 0x7fd0a40710a0 2024-03-09T09:05:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.527+0000 7fd0a3e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a40c32b0 con 0x7fd0a4154080 2024-03-09T09:05:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.527+0000 7fd0a3e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a40c2fd0 con 0x7fd0a40c1f70 2024-03-09T09:05:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.527+0000 7fd0a2000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd0a40710a0 0x7fd0a4153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:32950/0 (socket says 172.21.15.67:32950) 2024-03-09T09:05:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.527+0000 7fd0a2000640 1 -- 172.21.15.67:0/1793207083 learned_addr learned my addr 172.21.15.67:0/1793207083 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:05:03.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.528+0000 7fd0a2a00640 1 --2- 172.21.15.67:0/1793207083 >> v2:172.21.15.134:3300/0 conn(0x7fd0a40c1f70 0x7fd0a4150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.528+0000 7fd0a2a00640 1 -- 172.21.15.67:0/1793207083 >> v1:172.21.15.134:6789/0 conn(0x7fd0a4154080 legacy=0x7fd0a415e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.528+0000 7fd0a2a00640 1 -- 172.21.15.67:0/1793207083 >> v1:172.21.15.67:6789/0 conn(0x7fd0a40710a0 legacy=0x7fd0a4153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:03.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.528+0000 7fd0a2a00640 1 -- 172.21.15.67:0/1793207083 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a40c3700 con 0x7fd0a40c1f70 2024-03-09T09:05:03.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.531+0000 7fd0a2a00640 1 --2- 172.21.15.67:0/1793207083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40c1f70 0x7fd0a4150c50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd0940097a0 tx=0x7fd09402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b26ce5f1cd28122a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.531+0000 7fd0a1600640 1 -- 172.21.15.67:0/1793207083 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094035020 con 0x7fd0a40c1f70 2024-03-09T09:05:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.531+0000 7fd0a1600640 1 -- 172.21.15.67:0/1793207083 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd094031830 con 0x7fd0a40c1f70 2024-03-09T09:05:03.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.531+0000 7fd0a1600640 1 -- 172.21.15.67:0/1793207083 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094031c00 con 0x7fd0a40c1f70 2024-03-09T09:05:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.532+0000 7fd0a3e00640 1 -- 172.21.15.67:0/1793207083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40c1f70 msgr2=0x7fd0a4150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.532+0000 7fd0a3e00640 1 --2- 172.21.15.67:0/1793207083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40c1f70 0x7fd0a4150c50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd0940097a0 tx=0x7fd09402b8d0 comp rx=0 tx=0).stop 2024-03-09T09:05:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.532+0000 7fd0a3e00640 1 -- 172.21.15.67:0/1793207083 shutdown_connections 2024-03-09T09:05:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.532+0000 7fd0a3e00640 1 --2- 172.21.15.67:0/1793207083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40c1f70 0x7fd0a4150c50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.532+0000 7fd0a3e00640 1 -- 172.21.15.67:0/1793207083 >> 172.21.15.67:0/1793207083 conn(0x7fd0a40c7300 msgr2=0x7fd0a40c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.532+0000 7fd0a3e00640 1 -- 172.21.15.67:0/1793207083 shutdown_connections 2024-03-09T09:05:03.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.532+0000 7fd0a3e00640 1 -- 172.21.15.67:0/1793207083 wait complete. 2024-03-09T09:05:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.534+0000 7fd0a3e00640 1 Processor -- start 2024-03-09T09:05:03.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.534+0000 7fd0a3e00640 1 -- start start 2024-03-09T09:05:03.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a3e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0a40710a0 0x7fd0a414a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:03.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40c1f70 0x7fd0a414c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:03.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a4154080 0x7fd0a41eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:03.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a4070d50 con 0x7fd0a4154080 2024-03-09T09:05:03.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a3e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a4070830 con 0x7fd0a40c1f70 2024-03-09T09:05:03.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a3e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a4070ed0 con 0x7fd0a40710a0 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a2a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0a40710a0 msgr2=0x7fd0a414a090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0a40710a0 0x7fd0a414a090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a4154080 0x7fd0a41eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a4154080 0x7fd0a41eb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60188/0 (socket says 172.21.15.67:60188) 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a3400640 1 -- 172.21.15.67:0/2483731211 learned_addr learned my addr 172.21.15.67:0/2483731211 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.535+0000 7fd0a2000640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40c1f70 0x7fd0a414c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.536+0000 7fd0a3400640 1 -- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0a40710a0 msgr2=0x7fd0a414a090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.536+0000 7fd0a3400640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0a40710a0 0x7fd0a414a090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.536+0000 7fd0a3400640 1 -- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40c1f70 msgr2=0x7fd0a414c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.536+0000 7fd0a3400640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40c1f70 0x7fd0a414c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.536+0000 7fd0a3400640 1 -- 172.21.15.67:0/2483731211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a414a5d0 con 0x7fd0a4154080 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.536+0000 7fd0a3400640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a4154080 0x7fd0a41eb7c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00c7a0 tx=0x7fd09c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.536+0000 7fd09aa00640 1 -- 172.21.15.67:0/2483731211 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c003b70 con 0x7fd0a4154080 2024-03-09T09:05:03.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.536+0000 7fd09aa00640 1 -- 172.21.15.67:0/2483731211 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c003d30 con 0x7fd0a4154080 2024-03-09T09:05:03.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.536+0000 7fd0a3e00640 1 -- 172.21.15.67:0/2483731211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a414b0d0 con 0x7fd0a4154080 2024-03-09T09:05:03.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.536+0000 7fd09aa00640 1 -- 172.21.15.67:0/2483731211 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00aa90 con 0x7fd0a4154080 2024-03-09T09:05:03.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.537+0000 7fd0a3e00640 1 -- 172.21.15.67:0/2483731211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a4149890 con 0x7fd0a4154080 2024-03-09T09:05:03.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.538+0000 7fd0a3e00640 1 -- 172.21.15.67:0/2483731211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a404f3c0 con 0x7fd0a4154080 2024-03-09T09:05:03.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.539+0000 7fd09aa00640 1 -- 172.21.15.67:0/2483731211 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd09c00ac30 con 0x7fd0a4154080 2024-03-09T09:05:03.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.540+0000 7fd09aa00640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08406b480 0x7fd08406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:03.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.540+0000 7fd09aa00640 1 -- 172.21.15.67:0/2483731211 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd09c08a530 con 0x7fd0a4154080 2024-03-09T09:05:03.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.541+0000 7fd0a2a00640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08406b480 0x7fd08406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:03.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.542+0000 7fd0a2a00640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08406b480 0x7fd08406d940 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd094036000 tx=0x7fd094008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:03.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.545+0000 7fd09aa00640 1 -- 172.21.15.67:0/2483731211 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0a414b0d0 con 0x7fd0a4154080 2024-03-09T09:05:03.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.839+0000 7fd0a3e00640 1 -- 172.21.15.67:0/2483731211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd0a40c7e30 con 0x7fd0a4154080 2024-03-09T09:05:03.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.839+0000 7fd09aa00640 1 -- 172.21.15.67:0/2483731211 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fd09c058fb0 con 0x7fd0a4154080 2024-03-09T09:05:03.842 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":92,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":26,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:05:03.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 -- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08406b480 msgr2=0x7fd08406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:03.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08406b480 0x7fd08406d940 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd094036000 tx=0x7fd094008030 comp rx=0 tx=0).stop 2024-03-09T09:05:03.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 -- 172.21.15.67:0/2483731211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a4154080 msgr2=0x7fd0a41eb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:03.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a4154080 0x7fd0a41eb7c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00c7a0 tx=0x7fd09c00cc60 comp rx=0 tx=0).stop 2024-03-09T09:05:03.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 -- 172.21.15.67:0/2483731211 shutdown_connections 2024-03-09T09:05:03.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd08406b480 0x7fd08406d940 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:03.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a4154080 0x7fd0a41eb7c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:03.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a40c1f70 0x7fd0a414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:03.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 --2- 172.21.15.67:0/2483731211 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0a40710a0 0x7fd0a414a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:03.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 -- 172.21.15.67:0/2483731211 >> 172.21.15.67:0/2483731211 conn(0x7fd0a40c7300 msgr2=0x7fd0a414da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:03.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 -- 172.21.15.67:0/2483731211 shutdown_connections 2024-03-09T09:05:03.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:03.843+0000 7fd0a3e00640 1 -- 172.21.15.67:0/2483731211 wait complete. 2024-03-09T09:05:03.921 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:05:03.921 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 3 -eq 48 ']' 2024-03-09T09:05:03.921 INFO:tasks.workunit.client.0.smithi067.stdout:v0.3 2024-03-09T09:05:03.922 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.3 2024-03-09T09:05:03.922 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.3 2024-03-09T09:05:03.922 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.3 2024-03-09T09:05:04.512 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 47% (105/222) Updating files: 48% (107/222) Updating files: 49% (109/222) Updating files: 50% (111/222) Updating files: 51% (114/222) Updating files: 52% (116/222) Updating files: 53% (118/222) Updating files: 54% (120/222) Updating files: 55% (123/222) Updating files: 56% (125/222) Updating files: 57% (127/222) Updating files: 58% (129/222) Updating files: 59% (131/222) Updating files: 60% (134/222) Updating files: 61% (136/222) Updating files: 62% (138/222) Updating files: 63% (140/222) Updating files: 64% (143/222) Updating files: 65% (145/222) Updating files: 66% (147/222) Updating files: 67% (149/222) Updating files: 68% (151/222) Updating files: 69% (154/222) Updating files: 70% (156/222) Updating files: 71% (158/222) Updating files: 72% (160/222) Updating files: 73% (163/222) Updating files: 74% (165/222) Updating files: 75% (167/222) Updating files: 76% (169/222) Updating files: 77% (171/222) Updating files: 78% (174/222) Updating files: 79% (176/222) Updating files: 80% (178/222) Updating files: 81% (180/222) Updating files: 82% (183/222) Updating files: 83% (185/222) Updating files: 84% (187/222) Updating files: 85% (189/222) Updating files: 86% (191/222) Updating files: 87% (194/222) Updating files: 88% (196/222) Updating files: 89% (198/222) Updating files: 90% (200/222) Updating files: 91% (203/222) Updating files: 92% (205/222) Updating files: 93% (207/222) Updating files: 94% (209/222) Updating files: 95% (211/222) Updating files: 96% (214/222) Updating files: 97% (216/222) Updating files: 98% (218/222) Updating files: 99% (220/222) Updating files: 100% (222/222) Updating files: 100% (222/222), done. 2024-03-09T09:05:04.585 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 2b5f76f48e9 readme 2024-03-09T09:05:04.601 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.2 2024-03-09T09:05:04.782 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:05:04.783 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 3 -eq 48 ']' 2024-03-09T09:05:04.783 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.3 2024-03-09T09:05:04.783 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.3 2024-03-09T09:05:04.783 INFO:tasks.workunit.client.1.smithi134.stdout:v0.3 2024-03-09T09:05:04.783 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.3 2024-03-09T09:05:06.870 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:05:07.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.069+0000 7f2bcb200640 1 Processor -- start 2024-03-09T09:05:07.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.069+0000 7f2bcb200640 1 -- start start 2024-03-09T09:05:07.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.071+0000 7f2bcb200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:07.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.071+0000 7f2bcb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bcc157000 con 0x7f2bcc0c5e40 2024-03-09T09:05:07.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.071+0000 7f2bcb200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bcc0ae430 con 0x7f2bcc0cadb0 2024-03-09T09:05:07.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.071+0000 7f2bcb200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bcc0ae6f0 con 0x7f2bcc153300 2024-03-09T09:05:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.072+0000 7f2bc9400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2bcc0c5e40 0x7f2bcc156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:32956/0 (socket says 172.21.15.67:32956) 2024-03-09T09:05:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.072+0000 7f2bc9e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.072+0000 7f2bc9400640 1 -- 172.21.15.67:0/4038734300 learned_addr learned my addr 172.21.15.67:0/4038734300 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:05:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.072+0000 7f2bc9e00640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 msgr2=0x7f2bcc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:05:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.072+0000 7f2bc9e00640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 msgr2=0x7f2bcc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:05:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.072+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.072+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:07.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.273+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:07.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.273+0000 7f2bc9e00640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 msgr2=0x7f2bcc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:05:07.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.273+0000 7f2bc9e00640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 msgr2=0x7f2bcc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:05:07.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.273+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:07.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.273+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:05:07.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.674+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:07.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.675+0000 7f2bc9e00640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 msgr2=0x7f2bcc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:05:07.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.675+0000 7f2bc9e00640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 msgr2=0x7f2bcc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:05:07.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.675+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:07.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:07.675+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:05:08.477 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 34% (102/295) Updating files: 35% (104/295) Updating files: 36% (107/295) Updating files: 37% (110/295) Updating files: 38% (113/295) Updating files: 39% (116/295) Updating files: 40% (118/295) Updating files: 41% (121/295) Updating files: 42% (124/295) Updating files: 43% (127/295) Updating files: 44% (130/295) Updating files: 45% (133/295) Updating files: 46% (136/295) Updating files: 47% (139/295) Updating files: 48% (142/295) Updating files: 49% (145/295) Updating files: 50% (148/295) Updating files: 51% (151/295) Updating files: 52% (154/295) Updating files: 53% (157/295) Updating files: 54% (160/295) Updating files: 55% (163/295) Updating files: 56% (166/295) Updating files: 57% (169/295) Updating files: 58% (172/295) Updating files: 59% (175/295) Updating files: 60% (177/295) Updating files: 61% (180/295) Updating files: 61% (182/295) Updating files: 62% (183/295) Updating files: 63% (186/295) Updating files: 64% (189/295) Updating files: 65% (192/295) Updating files: 66% (195/295) Updating files: 67% (198/295) Updating files: 68% (201/295) Updating files: 69% (204/295) Updating files: 70% (207/295) Updating files: 71% (210/295) Updating files: 72% (213/295) Updating files: 73% (216/295) Updating files: 74% (219/295) Updating files: 75% (222/295) Updating files: 76% (225/295) Updating files: 77% (228/295) Updating files: 78% (231/295) Updating files: 79% (234/295) Updating files: 80% (236/295) Updating files: 81% (239/295) Updating files: 82% (242/295) Updating files: 83% (245/295) Updating files: 84% (248/295) Updating files: 85% (251/295) Updating files: 86% (254/295) Updating files: 87% (257/295) Updating files: 88% (260/295) Updating files: 89% (263/295) Updating files: 90% (266/295) Updating files: 91% (269/295) Updating files: 92% (272/295) Updating files: 93% (275/295) Updating files: 94% (278/295) Updating files: 95% (281/295) Updating files: 96% (284/295) Updating files: 97% (287/295) Updating files: 98% (290/295) Updating files: 99% (293/295) Updating files: 100% (295/295) Updating files: 100% (295/295), done. 2024-03-09T09:05:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:08.476+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:08.477+0000 7f2bc9e00640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 msgr2=0x7f2bcc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:05:08.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:08.477+0000 7f2bc9e00640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 msgr2=0x7f2bcc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:05:08.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:08.477+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:08.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:08.477+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:05:08.508 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 2e986fe9ddf include ceph.spec in configure.ac 2024-03-09T09:05:08.522 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.3 2024-03-09T09:05:08.634 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:05:08.634 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 4 -eq 48 ']' 2024-03-09T09:05:08.634 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.4 2024-03-09T09:05:08.634 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.4 2024-03-09T09:05:08.634 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.4 2024-03-09T09:05:08.635 INFO:tasks.workunit.client.0.smithi067.stdout:v0.4 2024-03-09T09:05:08.643 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 36% (109/295) Updating files: 37% (110/295) Updating files: 38% (113/295) Updating files: 39% (116/295) Updating files: 40% (118/295) Updating files: 41% (121/295) Updating files: 42% (124/295) Updating files: 43% (127/295) Updating files: 44% (130/295) Updating files: 45% (133/295) Updating files: 46% (136/295) Updating files: 47% (139/295) Updating files: 48% (142/295) Updating files: 49% (145/295) Updating files: 50% (148/295) Updating files: 51% (151/295) Updating files: 52% (154/295) Updating files: 53% (157/295) Updating files: 54% (160/295) Updating files: 55% (163/295) Updating files: 56% (166/295) Updating files: 57% (169/295) Updating files: 58% (172/295) Updating files: 59% (175/295) Updating files: 60% (177/295) Updating files: 61% (180/295) Updating files: 62% (183/295) Updating files: 63% (186/295) Updating files: 64% (189/295) Updating files: 65% (192/295) Updating files: 66% (195/295) Updating files: 67% (198/295) Updating files: 68% (201/295) Updating files: 69% (204/295) Updating files: 70% (207/295) Updating files: 71% (210/295) Updating files: 72% (213/295) Updating files: 73% (216/295) Updating files: 74% (219/295) Updating files: 75% (222/295) Updating files: 76% (225/295) Updating files: 77% (228/295) Updating files: 78% (231/295) Updating files: 79% (234/295) Updating files: 80% (236/295) Updating files: 81% (239/295) Updating files: 82% (242/295) Updating files: 83% (245/295) Updating files: 84% (248/295) Updating files: 85% (251/295) Updating files: 86% (254/295) Updating files: 87% (257/295) Updating files: 88% (260/295) Updating files: 89% (263/295) Updating files: 90% (266/295) Updating files: 91% (269/295) Updating files: 92% (272/295) Updating files: 93% (275/295) Updating files: 94% (278/295) Updating files: 95% (281/295) Updating files: 96% (284/295) Updating files: 97% (287/295) Updating files: 98% (290/295) Updating files: 99% (293/295) Updating files: 100% (295/295) Updating files: 100% (295/295), done. 2024-03-09T09:05:08.667 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 2e986fe9ddf include ceph.spec in configure.ac 2024-03-09T09:05:08.682 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.3 2024-03-09T09:05:08.838 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:05:08.838 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 4 -eq 48 ']' 2024-03-09T09:05:08.838 INFO:tasks.workunit.client.1.smithi134.stdout:v0.4 2024-03-09T09:05:08.839 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.4 2024-03-09T09:05:08.839 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.4 2024-03-09T09:05:08.839 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.4 2024-03-09T09:05:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.071+0000 7f2bc3400640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 msgr2=0x7f2bcc0b0000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.071+0000 7f2bc3400640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.134:3300/0 conn(0x7f2bcc153300 0x7f2bcc0b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.071+0000 7f2bc3400640 1 -- 172.21.15.67:0/4038734300 >> v1:172.21.15.134:6789/0 conn(0x7f2bcc0cadb0 legacy=0x7f2bcc160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.071+0000 7f2bc3400640 1 -- 172.21.15.67:0/4038734300 >> v1:172.21.15.67:6789/0 conn(0x7f2bcc0c5e40 legacy=0x7f2bcc156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.071+0000 7f2bc3400640 1 --2- 172.21.15.67:0/4038734300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bac003730 0x7f2bac005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.071+0000 7f2bc3400640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 0x7f2bac0086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.071+0000 7f2bc3400640 1 -- 172.21.15.67:0/4038734300 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bcc0ae6f0 con 0x7f2bac0061f0 2024-03-09T09:05:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.071+0000 7f2bc3400640 1 -- 172.21.15.67:0/4038734300 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac008d60 con 0x7f2bac003730 2024-03-09T09:05:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.072+0000 7f2bc9e00640 1 -- 172.21.15.67:0/4038734300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bac003730 msgr2=0x7f2bac005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:05:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.072+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bac003730 0x7f2bac005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.072+0000 7f2bc9400640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 0x7f2bac0086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.073+0000 7f2bc9400640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 msgr2=0x7f2bac0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:05:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.073+0000 7f2bc9400640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 msgr2=0x7f2bac0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:05:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.073+0000 7f2bc9400640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 0x7f2bac0086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.073+0000 7f2bc9400640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 0x7f2bac0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:10.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.273+0000 7f2bc9e00640 1 -- 172.21.15.67:0/4038734300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bac003730 msgr2=0x7f2bac005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:05:10.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.273+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bac003730 0x7f2bac005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:05:10.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.274+0000 7f2bc9400640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 0x7f2bac0086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:10.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.275+0000 7f2bc9400640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 msgr2=0x7f2bac0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:05:10.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.275+0000 7f2bc9400640 1 -- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 msgr2=0x7f2bac0086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:05:10.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.275+0000 7f2bc9400640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 0x7f2bac0086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:10.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.275+0000 7f2bc9400640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 0x7f2bac0086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:05:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.674+0000 7f2bc9e00640 1 -- 172.21.15.67:0/4038734300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bac003730 msgr2=0x7f2bac005b60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:05:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.674+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/4038734300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bac003730 0x7f2bac005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:05:10.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.676+0000 7f2bc9400640 1 --2- 172.21.15.67:0/4038734300 >> v2:172.21.15.67:3300/0 conn(0x7f2bac0061f0 0x7f2bac0086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.677+0000 7f2bc9400640 1 -- 172.21.15.67:0/4038734300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bac003730 msgr2=0x7f2bac005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.677+0000 7f2bc9400640 1 --2- 172.21.15.67:0/4038734300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bac003730 0x7f2bac005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.677+0000 7f2bc9400640 1 -- 172.21.15.67:0/4038734300 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc155b10 con 0x7f2bac0061f0 2024-03-09T09:05:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.680+0000 7f2bc9400640 1 --2- 172.21.15.67:0/4038734300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0061f0 0x7f2bac0086b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8007d20 tx=0x7f2bb802bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebea6b5ea6b828f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.681+0000 7f2bc8a00640 1 -- 172.21.15.67:0/4038734300 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8032070 con 0x7f2bac0061f0 2024-03-09T09:05:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.681+0000 7f2bc8a00640 1 -- 172.21.15.67:0/4038734300 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb802c710 con 0x7f2bac0061f0 2024-03-09T09:05:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.681+0000 7f2bc8a00640 1 -- 172.21.15.67:0/4038734300 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb802cae0 con 0x7f2bac0061f0 2024-03-09T09:05:10.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.683+0000 7f2bcb200640 1 -- 172.21.15.67:0/4038734300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0061f0 msgr2=0x7f2bac0086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:10.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.683+0000 7f2bcb200640 1 --2- 172.21.15.67:0/4038734300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0061f0 0x7f2bac0086b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8007d20 tx=0x7f2bb802bb70 comp rx=0 tx=0).stop 2024-03-09T09:05:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.683+0000 7f2bca800640 1 -- 172.21.15.67:0/4038734300 reap_dead start 2024-03-09T09:05:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.683+0000 7f2bcb200640 1 -- 172.21.15.67:0/4038734300 shutdown_connections 2024-03-09T09:05:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.683+0000 7f2bcb200640 1 --2- 172.21.15.67:0/4038734300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bac0061f0 0x7f2bac0086b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.683+0000 7f2bcb200640 1 -- 172.21.15.67:0/4038734300 >> 172.21.15.67:0/4038734300 conn(0x7f2bcc0732d0 msgr2=0x7f2bcc0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.683+0000 7f2bcb200640 1 -- 172.21.15.67:0/4038734300 shutdown_connections 2024-03-09T09:05:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.683+0000 7f2bcb200640 1 -- 172.21.15.67:0/4038734300 wait complete. 2024-03-09T09:05:10.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.686+0000 7f2bcb200640 1 Processor -- start 2024-03-09T09:05:10.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.687+0000 7f2bcb200640 1 -- start start 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.688+0000 7f2bcb200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bac0061f0 0x7f2bcc1eef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.688+0000 7f2bcb200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bcc0c6e30 0x7f2bcc14d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.688+0000 7f2bcb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bcc1ef450 0x7f2bcc14df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.688+0000 7f2bcb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc0c59e0 con 0x7f2bcc1ef450 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.688+0000 7f2bcb200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc159e90 con 0x7f2bac0061f0 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.689+0000 7f2bcb200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc0b0540 con 0x7f2bcc0c6e30 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.689+0000 7f2bc9400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bcc0c6e30 msgr2=0x7f2bcc14d9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.689+0000 7f2bc9400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bcc0c6e30 0x7f2bcc14d9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.689+0000 7f2bca800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bcc1ef450 0x7f2bcc14df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.689+0000 7f2bca800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bcc1ef450 0x7f2bcc14df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53944/0 (socket says 172.21.15.67:53944) 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.689+0000 7f2bca800640 1 -- 172.21.15.67:0/2076164454 learned_addr learned my addr 172.21.15.67:0/2076164454 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:10.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.689+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bac0061f0 0x7f2bcc1eef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.690+0000 7f2bca800640 1 -- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bcc0c6e30 msgr2=0x7f2bcc14d9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:05:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.690+0000 7f2bca800640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bcc0c6e30 0x7f2bcc14d9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.690+0000 7f2bca800640 1 -- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bac0061f0 msgr2=0x7f2bcc1eef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.690+0000 7f2bca800640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bac0061f0 0x7f2bcc1eef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.690+0000 7f2bca800640 1 -- 172.21.15.67:0/2076164454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc0c6c20 con 0x7f2bcc1ef450 2024-03-09T09:05:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.690+0000 7f2bca800640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bcc1ef450 0x7f2bcc14df10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0af440 tx=0x7f2bc400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.690+0000 7f2bc2000640 1 -- 172.21.15.67:0/2076164454 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4017070 con 0x7f2bcc1ef450 2024-03-09T09:05:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.690+0000 7f2bc2000640 1 -- 172.21.15.67:0/2076164454 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc4005d40 con 0x7f2bcc1ef450 2024-03-09T09:05:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.691+0000 7f2bcb200640 1 -- 172.21.15.67:0/2076164454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bcc1ef990 con 0x7f2bcc1ef450 2024-03-09T09:05:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.691+0000 7f2bc2000640 1 -- 172.21.15.67:0/2076164454 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400a880 con 0x7f2bcc1ef450 2024-03-09T09:05:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.691+0000 7f2bcb200640 1 -- 172.21.15.67:0/2076164454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc0c7a80 con 0x7f2bcc1ef450 2024-03-09T09:05:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.692+0000 7f2bcb200640 1 -- 172.21.15.67:0/2076164454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bcc04f3c0 con 0x7f2bcc1ef450 2024-03-09T09:05:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.695+0000 7f2bc2000640 1 -- 172.21.15.67:0/2076164454 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2bc401a070 con 0x7f2bcc1ef450 2024-03-09T09:05:10.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.696+0000 7f2bc2000640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2bac06c360 0x7f2bac06e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:10.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.696+0000 7f2bc2000640 1 -- 172.21.15.67:0/2076164454 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f2bc40896b0 con 0x7f2bcc1ef450 2024-03-09T09:05:10.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.697+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2bac06c360 0x7f2bac06e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:10.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.698+0000 7f2bc9e00640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2bac06c360 0x7f2bac06e820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc004650 tx=0x7f2bbc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.702+0000 7f2bc2000640 1 -- 172.21.15.67:0/2076164454 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2bc4057ae0 con 0x7f2bcc1ef450 2024-03-09T09:05:10.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.992+0000 7f2bcb200640 1 -- 172.21.15.67:0/2076164454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2bcc0cab90 con 0x7f2bcc1ef450 2024-03-09T09:05:10.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.992+0000 7f2bc2000640 1 -- 172.21.15.67:0/2076164454 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1558 (secure 0 0 0) 0x7f2bc405bb80 con 0x7f2bcc1ef450 2024-03-09T09:05:10.995 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":96,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:05:10.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 -- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2bac06c360 msgr2=0x7f2bac06e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:10.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2bac06c360 0x7f2bac06e820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc004650 tx=0x7f2bbc008040 comp rx=0 tx=0).stop 2024-03-09T09:05:10.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 -- 172.21.15.67:0/2076164454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bcc1ef450 msgr2=0x7f2bcc14df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:10.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bcc1ef450 0x7f2bcc14df10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0af440 tx=0x7f2bc400cb90 comp rx=0 tx=0).stop 2024-03-09T09:05:10.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 -- 172.21.15.67:0/2076164454 shutdown_connections 2024-03-09T09:05:10.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2bac06c360 0x7f2bac06e820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:10.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2bcc1ef450 0x7f2bcc14df10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:10.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2bcc0c6e30 0x7f2bcc14d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:10.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 --2- 172.21.15.67:0/2076164454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bac0061f0 0x7f2bcc1eef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:10.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 -- 172.21.15.67:0/2076164454 >> 172.21.15.67:0/2076164454 conn(0x7f2bcc0732d0 msgr2=0x7f2bcc158460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:10.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 -- 172.21.15.67:0/2076164454 shutdown_connections 2024-03-09T09:05:10.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:10.995+0000 7f2bcb200640 1 -- 172.21.15.67:0/2076164454 wait complete. 2024-03-09T09:05:11.023 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:05:11.023 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:05:11.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:05:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.307+0000 7f6359800640 1 Processor -- start 2024-03-09T09:05:11.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.308+0000 7f6359800640 1 -- start start 2024-03-09T09:05:11.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.311+0000 7f6359800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63540acf00 0x7f63540c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.311+0000 7f6359800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f63540c9380 0x7f6354156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.311+0000 7f6359800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63540c9810 con 0x7f63540c5e00 2024-03-09T09:05:11.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.311+0000 7f6359800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f635406d740 con 0x7f63540c9380 2024-03-09T09:05:11.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.311+0000 7f6359800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6354153680 con 0x7f63540acf00 2024-03-09T09:05:11.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.311+0000 7f6353e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f63540c5e00 0x7f635406c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39626/0 (socket says 172.21.15.67:39626) 2024-03-09T09:05:11.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.311+0000 7f6353e00640 1 -- 172.21.15.67:0/1669144112 learned_addr learned my addr 172.21.15.67:0/1669144112 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:05:11.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.311+0000 7f6353400640 1 -- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63540acf00 msgr2=0x7f63540c61e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:05:11.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.311+0000 7f6353400640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63540acf00 0x7f63540c61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318482947 0 0) 0x7f63540c9810 con 0x7f63540c5e00 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6358e00640 1 --2- 172.21.15.67:0/1669144112 >> v2:172.21.15.134:3300/0 conn(0x7f63540c9380 0x7f6354156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6338003520 con 0x7f63540c5e00 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2318618258 0 0) 0x7f6338003520 con 0x7f63540c5e00 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63540acf00 msgr2=0x7f63540c61e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63540acf00 0x7f63540c61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 >> v2:172.21.15.134:3300/0 conn(0x7f63540c9380 msgr2=0x7f6354156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 --2- 172.21.15.67:0/1669144112 >> v2:172.21.15.134:3300/0 conn(0x7f63540c9380 0x7f6354156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63540ad7c0 con 0x7f63540c5e00 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f6344001670 con 0x7f63540c5e00 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 >> v1:172.21.15.67:6789/0 conn(0x7f63540c5e00 legacy=0x7f635406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63540acf00 0x7f6338003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63380040a0 0x7f6338004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338004d90 0x7f63380051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6354153680 con 0x7f63380040a0 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633800bc00 con 0x7f6338004d90 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.312+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633800be00 con 0x7f63540acf00 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.313+0000 7f6353e00640 1 -- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63540acf00 msgr2=0x7f6338003a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.313+0000 7f6353e00640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63540acf00 0x7f6338003a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.313+0000 7f6353400640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63380040a0 0x7f6338004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.313+0000 7f6358e00640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338004d90 0x7f63380051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.313+0000 7f6353400640 1 -- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63540acf00 msgr2=0x7f6338003a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.313+0000 7f6353400640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63540acf00 0x7f6338003a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.313+0000 7f6353400640 1 -- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338004d90 msgr2=0x7f63380051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.313+0000 7f6353400640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338004d90 0x7f63380051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.313+0000 7f6353400640 1 -- 172.21.15.67:0/1669144112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6338003520 con 0x7f63380040a0 2024-03-09T09:05:11.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.315+0000 7f6353400640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63380040a0 0x7f6338004800 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6340009900 tx=0x7f6338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966b7b9e7b563d27 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:11.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.315+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340009500 con 0x7f63380040a0 2024-03-09T09:05:11.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.315+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634002f070 con 0x7f63380040a0 2024-03-09T09:05:11.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.315+0000 7f6352a00640 1 -- 172.21.15.67:0/1669144112 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340003df0 con 0x7f63380040a0 2024-03-09T09:05:11.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.316+0000 7f6359800640 1 -- 172.21.15.67:0/1669144112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63380040a0 msgr2=0x7f6338004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.316+0000 7f6359800640 1 --2- 172.21.15.67:0/1669144112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63380040a0 0x7f6338004800 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6340009900 tx=0x7f6338005830 comp rx=0 tx=0).stop 2024-03-09T09:05:11.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.316+0000 7f6358e00640 1 -- 172.21.15.67:0/1669144112 reap_dead start 2024-03-09T09:05:11.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.316+0000 7f6359800640 1 -- 172.21.15.67:0/1669144112 shutdown_connections 2024-03-09T09:05:11.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.316+0000 7f6359800640 1 -- 172.21.15.67:0/1669144112 >> 172.21.15.67:0/1669144112 conn(0x7f63540cb480 msgr2=0x7f63540cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:11.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.316+0000 7f6359800640 1 -- 172.21.15.67:0/1669144112 shutdown_connections 2024-03-09T09:05:11.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.316+0000 7f6359800640 1 -- 172.21.15.67:0/1669144112 wait complete. 2024-03-09T09:05:11.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.318+0000 7f6359800640 1 Processor -- start 2024-03-09T09:05:11.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.318+0000 7f6359800640 1 -- start start 2024-03-09T09:05:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.319+0000 7f6359800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63380040a0 0x7f63541eb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6359800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c0330 0x7f63541ec670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6359800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63540c16a0 0x7f63540c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6359800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635406bde0 con 0x7f63540c0330 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6359800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635406bb00 con 0x7f63540c16a0 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6359800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635406c0c0 con 0x7f63380040a0 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6353e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63380040a0 msgr2=0x7f63541eb1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6353e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63380040a0 0x7f63541eb1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6353400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c0330 0x7f63541ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6353400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c0330 0x7f63541ec670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53956/0 (socket says 172.21.15.67:53956) 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6353400640 1 -- 172.21.15.67:0/3827024825 learned_addr learned my addr 172.21.15.67:0/3827024825 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6358e00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63540c16a0 0x7f63540c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6353400640 1 -- 172.21.15.67:0/3827024825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63380040a0 msgr2=0x7f63541eb1a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:05:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6353400640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63380040a0 0x7f63541eb1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6353400640 1 -- 172.21.15.67:0/3827024825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63540c16a0 msgr2=0x7f63540c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.320+0000 7f6353400640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f63540c16a0 0x7f63540c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.321+0000 7f6353400640 1 -- 172.21.15.67:0/3827024825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63540bf290 con 0x7f63540c0330 2024-03-09T09:05:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.321+0000 7f6353400640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c0330 0x7f63541ec670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63400090b0 tx=0x7f6340005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:11.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.321+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340006dd0 con 0x7f63540c0330 2024-03-09T09:05:11.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.321+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6340003e50 con 0x7f63540c0330 2024-03-09T09:05:11.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.321+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340034890 con 0x7f63540c0330 2024-03-09T09:05:11.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.321+0000 7f6359800640 1 -- 172.21.15.67:0/3827024825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63540bf570 con 0x7f63540c0330 2024-03-09T09:05:11.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.322+0000 7f6359800640 1 -- 172.21.15.67:0/3827024825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63540bfb30 con 0x7f63540c0330 2024-03-09T09:05:11.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.323+0000 7f6359800640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541e8500 0x7f63541e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.323+0000 7f6359800640 1 -- 172.21.15.67:0/3827024825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540bfd00 con 0x7f63541e8500 2024-03-09T09:05:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.323+0000 7f6353e00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541e8500 0x7f63541e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.323+0000 7f6353e00640 1 -- 172.21.15.67:0/3827024825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f63541e8e20 con 0x7f63541e8500 2024-03-09T09:05:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.324+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6340034a30 con 0x7f63540c0330 2024-03-09T09:05:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.324+0000 7f6353e00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541e8500 0x7f63541e88e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f634400c6f0 tx=0x7f634400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.325+0000 7f634be00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f633406b4b0 0x7f633406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.325+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f63400abba0 con 0x7f63540c0330 2024-03-09T09:05:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.325+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344017070 con 0x7f63541e8500 2024-03-09T09:05:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.325+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f63440072d0 con 0x7f63541e8500 2024-03-09T09:05:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.325+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541e8500 msgr2=0x7f63541e88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.325+0000 7f634be00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541e8500 0x7f63541e88e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f634400c6f0 tx=0x7f634400cbb0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.325+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541e8500 msgr2=0x7f63541e88e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.325+0000 7f634be00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541e8500 0x7f63541e88e0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f634400c6f0 tx=0x7f634400cbb0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.325+0000 7f6358e00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f633406b4b0 0x7f633406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.327+0000 7f6358e00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f633406b4b0 0x7f633406d970 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6354073710 tx=0x7f634c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.333+0000 7f6359800640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541ee770 0x7f63541eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.333+0000 7f6359800640 1 -- 172.21.15.67:0/3827024825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635404f3c0 con 0x7f63541ee770 2024-03-09T09:05:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.333+0000 7f6353e00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541ee770 0x7f63541eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.333+0000 7f6353e00640 1 -- 172.21.15.67:0/3827024825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f635406d2f0 con 0x7f63541ee770 2024-03-09T09:05:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.333+0000 7f6353e00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541ee770 0x7f63541eeb80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f634400d010 tx=0x7f6344001470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:11.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344017030 con 0x7f63541ee770 2024-03-09T09:05:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5177 (secure 0 0 0) 0x7f634401a8d0 con 0x7f63541ee770 2024-03-09T09:05:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541ee770 msgr2=0x7f63541eeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f634be00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541ee770 0x7f63541eeb80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f634400d010 tx=0x7f6344001470 comp rx=0 tx=0).stop 2024-03-09T09:05:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f634be00640 1 -- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541ee770 msgr2=0x7f63541eeb80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:11.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f634be00640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63541ee770 0x7f63541eeb80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.337 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 96, 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:05:11.338 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:05:11.339 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.340 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:05:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:11.342 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:05:11.344 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:11.347 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:05:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:05:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:05:11.351 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:05:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f6359800640 1 -- 172.21.15.67:0/3827024825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f633406b4b0 msgr2=0x7f633406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f6359800640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f633406b4b0 0x7f633406d970 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6354073710 tx=0x7f634c00b040 comp rx=0 tx=0).stop 2024-03-09T09:05:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f6359800640 1 -- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c0330 msgr2=0x7f63541ec670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f6359800640 1 --2- 172.21.15.67:0/3827024825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c0330 0x7f63541ec670 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f63400090b0 tx=0x7f6340005a10 comp rx=0 tx=0).stop 2024-03-09T09:05:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f6358e00640 1 -- 172.21.15.67:0/3827024825 reap_dead start 2024-03-09T09:05:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f6358e00640 1 -- 172.21.15.67:0/3827024825 reap_dead start 2024-03-09T09:05:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f6359800640 1 -- 172.21.15.67:0/3827024825 shutdown_connections 2024-03-09T09:05:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.334+0000 7f6359800640 1 -- 172.21.15.67:0/3827024825 >> 172.21.15.67:0/3827024825 conn(0x7f63540cb480 msgr2=0x7f63540ae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.335+0000 7f6359800640 1 -- 172.21.15.67:0/3827024825 shutdown_connections 2024-03-09T09:05:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.335+0000 7f6359800640 1 -- 172.21.15.67:0/3827024825 wait complete. 2024-03-09T09:05:11.354 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:05:11.354 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:05:11.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:05:11.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.557+0000 7fa42d800640 1 Processor -- start 2024-03-09T09:05:11.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.557+0000 7fa42d800640 1 -- start start 2024-03-09T09:05:11.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.558+0000 7fa42d800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42806c1d0 0x7fa42806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.558+0000 7fa42d800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4280c9890 con 0x7fa42814fd90 2024-03-09T09:05:11.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.558+0000 7fa42d800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4280c5f80 con 0x7fa4280c9380 2024-03-09T09:05:11.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.558+0000 7fa42d800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42806ba60 con 0x7fa42806c1d0 2024-03-09T09:05:11.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.558+0000 7fa42ce00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42806c1d0 msgr2=0x7fa42806c5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:05:11.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.559+0000 7fa42ce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42806c1d0 0x7fa42806c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:11.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.559+0000 7fa427e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa42814fd90 0x7fa428156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39628/0 (socket says 172.21.15.67:39628) 2024-03-09T09:05:11.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.559+0000 7fa427e00640 1 -- 172.21.15.67:0/484287827 learned_addr learned my addr 172.21.15.67:0/484287827 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:05:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.559+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374686960 0 0) 0x7fa4280c9890 con 0x7fa42814fd90 2024-03-09T09:05:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.559+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa40c003520 con 0x7fa42814fd90 2024-03-09T09:05:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.559+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2009887058 0 0) 0x7fa40c003520 con 0x7fa42814fd90 2024-03-09T09:05:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.559+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42806c1d0 msgr2=0x7fa42806c5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.559+0000 7fa426a00640 1 --2- 172.21.15.67:0/484287827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42806c1d0 0x7fa42806c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.559+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 >> v1:172.21.15.134:6789/0 conn(0x7fa4280c9380 legacy=0x7fa4280acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.559+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280c6240 con 0x7fa42814fd90 2024-03-09T09:05:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.559+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fa418001670 con 0x7fa42814fd90 2024-03-09T09:05:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 >> v1:172.21.15.67:6789/0 conn(0x7fa42814fd90 legacy=0x7fa428156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa426a00640 1 --2- 172.21.15.67:0/484287827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40c003a50 0x7fa40c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa426a00640 1 --2- 172.21.15.67:0/484287827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa40c004800 0x7fa40c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa426a00640 1 --2- 172.21.15.67:0/484287827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42806c1d0 0x7fa40c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42806ba60 con 0x7fa40c004800 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c00bf00 con 0x7fa40c003a50 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c00c080 con 0x7fa42806c1d0 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa427400640 1 --2- 172.21.15.67:0/484287827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa40c004800 0x7fa40c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa42ce00640 1 -- 172.21.15.67:0/484287827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42806c1d0 msgr2=0x7fa40c00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa42ce00640 1 --2- 172.21.15.67:0/484287827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42806c1d0 0x7fa40c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa427e00640 1 --2- 172.21.15.67:0/484287827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40c003a50 0x7fa40c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa427400640 1 -- 172.21.15.67:0/484287827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42806c1d0 msgr2=0x7fa40c00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa427400640 1 --2- 172.21.15.67:0/484287827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42806c1d0 0x7fa40c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa427400640 1 -- 172.21.15.67:0/484287827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40c003a50 msgr2=0x7fa40c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa427400640 1 --2- 172.21.15.67:0/484287827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40c003a50 0x7fa40c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.560+0000 7fa427400640 1 -- 172.21.15.67:0/484287827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c003520 con 0x7fa40c004800 2024-03-09T09:05:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.562+0000 7fa427400640 1 --2- 172.21.15.67:0/484287827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa40c004800 0x7fa40c004c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa414001750 tx=0x7fa40c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1fa3201b9f286c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.562+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414001370 con 0x7fa40c004800 2024-03-09T09:05:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.562+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4140052d0 con 0x7fa40c004800 2024-03-09T09:05:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.562+0000 7fa426a00640 1 -- 172.21.15.67:0/484287827 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa414003830 con 0x7fa40c004800 2024-03-09T09:05:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.563+0000 7fa42d800640 1 -- 172.21.15.67:0/484287827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa40c004800 msgr2=0x7fa40c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.563+0000 7fa42d800640 1 --2- 172.21.15.67:0/484287827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa40c004800 0x7fa40c004c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa414001750 tx=0x7fa40c005830 comp rx=0 tx=0).stop 2024-03-09T09:05:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.563+0000 7fa42ce00640 1 -- 172.21.15.67:0/484287827 reap_dead start 2024-03-09T09:05:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.563+0000 7fa42d800640 1 -- 172.21.15.67:0/484287827 shutdown_connections 2024-03-09T09:05:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.563+0000 7fa42d800640 1 -- 172.21.15.67:0/484287827 >> 172.21.15.67:0/484287827 conn(0x7fa4280cb480 msgr2=0x7fa4280cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.563+0000 7fa42d800640 1 -- 172.21.15.67:0/484287827 shutdown_connections 2024-03-09T09:05:11.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.563+0000 7fa42d800640 1 -- 172.21.15.67:0/484287827 wait complete. 2024-03-09T09:05:11.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.566+0000 7fa42d800640 1 Processor -- start 2024-03-09T09:05:11.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.566+0000 7fa42d800640 1 -- start start 2024-03-09T09:05:11.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.567+0000 7fa42d800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40c004800 0x7fa4281efa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.567+0000 7fa42d800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42814c8c0 0x7fa4281e2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa428157bf0 0x7fa42814bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42d800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280ad610 con 0x7fa428157bf0 2024-03-09T09:05:11.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42d800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42806bcb0 con 0x7fa40c004800 2024-03-09T09:05:11.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42d800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280ad790 con 0x7fa42814c8c0 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa427400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42814c8c0 msgr2=0x7fa4281e2e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa427400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42814c8c0 0x7fa4281e2e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa428157bf0 0x7fa42814bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa428157bf0 0x7fa42814bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53984/0 (socket says 172.21.15.67:53984) 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42ce00640 1 -- 172.21.15.67:0/2146225851 learned_addr learned my addr 172.21.15.67:0/2146225851 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa427e00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40c004800 0x7fa4281efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42ce00640 1 -- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42814c8c0 msgr2=0x7fa4281e2e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42ce00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa42814c8c0 0x7fa4281e2e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42ce00640 1 -- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40c004800 msgr2=0x7fa4281efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42ce00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa40c004800 0x7fa4281efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.568+0000 7fa42ce00640 1 -- 172.21.15.67:0/2146225851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280acd70 con 0x7fa428157bf0 2024-03-09T09:05:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.569+0000 7fa42ce00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa428157bf0 0x7fa42814bbe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa42000baf0 tx=0x7fa42000c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.569+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420005110 con 0x7fa428157bf0 2024-03-09T09:05:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.569+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4200052d0 con 0x7fa428157bf0 2024-03-09T09:05:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.569+0000 7fa42d800640 1 -- 172.21.15.67:0/2146225851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4280aed30 con 0x7fa428157bf0 2024-03-09T09:05:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.569+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4200056c0 con 0x7fa428157bf0 2024-03-09T09:05:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.569+0000 7fa42d800640 1 -- 172.21.15.67:0/2146225851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4281ef210 con 0x7fa428157bf0 2024-03-09T09:05:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.570+0000 7fa42d800640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa428158070 0x7fa42814c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.570+0000 7fa42d800640 1 -- 172.21.15.67:0/2146225851 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42804f3c0 con 0x7fa428158070 2024-03-09T09:05:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.571+0000 7fa427e00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa428158070 0x7fa42814c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.571+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa420005860 con 0x7fa428157bf0 2024-03-09T09:05:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.572+0000 7fa427e00640 1 -- 172.21.15.67:0/2146225851 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa42806d2d0 con 0x7fa428158070 2024-03-09T09:05:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.572+0000 7fa427e00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa428158070 0x7fa42814c120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa418010780 tx=0x7fa418010c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.572+0000 7fa41fe00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa40806b480 0x7fa40806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.572+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa42008a640 con 0x7fa428157bf0 2024-03-09T09:05:11.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.573+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418018020 con 0x7fa428158070 2024-03-09T09:05:11.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.573+0000 7fa427400640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa40806b480 0x7fa40806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.573+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa41800e760 con 0x7fa428158070 2024-03-09T09:05:11.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.573+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa428158070 msgr2=0x7fa42814c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.573+0000 7fa41fe00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa428158070 0x7fa42814c120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa418010780 tx=0x7fa418010c40 comp rx=0 tx=0).stop 2024-03-09T09:05:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.573+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa428158070 msgr2=0x7fa42814c120 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.573+0000 7fa41fe00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa428158070 0x7fa42814c120 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa418010780 tx=0x7fa418010c40 comp rx=0 tx=0).stop 2024-03-09T09:05:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.574+0000 7fa427400640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa40806b480 0x7fa40806d940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa414033000 tx=0x7fa414009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.581+0000 7fa42d800640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4281568f0 0x7fa428156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.581+0000 7fa42d800640 1 -- 172.21.15.67:0/2146225851 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa428155840 con 0x7fa4281568f0 2024-03-09T09:05:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.581+0000 7fa427e00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4281568f0 0x7fa428156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.582+0000 7fa427e00640 1 -- 172.21.15.67:0/2146225851 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa42814c660 con 0x7fa4281568f0 2024-03-09T09:05:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.583+0000 7fa427e00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4281568f0 0x7fa428156d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa418019000 tx=0x7fa418001b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:11.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.583+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418018030 con 0x7fa4281568f0 2024-03-09T09:05:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.584+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5005 (secure 0 0 0) 0x7fa41801d620 con 0x7fa4281568f0 2024-03-09T09:05:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.584+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4281568f0 msgr2=0x7fa428156d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.584+0000 7fa41fe00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4281568f0 0x7fa428156d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa418019000 tx=0x7fa418001b60 comp rx=0 tx=0).stop 2024-03-09T09:05:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.584+0000 7fa41fe00640 1 -- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4281568f0 msgr2=0x7fa428156d00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.584+0000 7fa41fe00640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4281568f0 0x7fa428156d00 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa418019000 tx=0x7fa418001b60 comp rx=0 tx=0).stop 2024-03-09T09:05:11.587 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 96, 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:05:11.588 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:05:11.589 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.590 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:05:11.591 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:11.592 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:11.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:05:11.595 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:11.596 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:05:11.597 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.598 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:11.600 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:05:11.601 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:05:11.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.585+0000 7fa42d800640 1 -- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa40806b480 msgr2=0x7fa40806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.585+0000 7fa42d800640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa40806b480 0x7fa40806d940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa414033000 tx=0x7fa414009030 comp rx=0 tx=0).stop 2024-03-09T09:05:11.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.585+0000 7fa42d800640 1 -- 172.21.15.67:0/2146225851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa428157bf0 msgr2=0x7fa42814bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:11.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.585+0000 7fa42d800640 1 --2- 172.21.15.67:0/2146225851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa428157bf0 0x7fa42814bbe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa42000baf0 tx=0x7fa42000c3e0 comp rx=0 tx=0).stop 2024-03-09T09:05:11.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.585+0000 7fa42ce00640 1 -- 172.21.15.67:0/2146225851 reap_dead start 2024-03-09T09:05:11.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.585+0000 7fa42ce00640 1 -- 172.21.15.67:0/2146225851 reap_dead start 2024-03-09T09:05:11.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.585+0000 7fa42d800640 1 -- 172.21.15.67:0/2146225851 shutdown_connections 2024-03-09T09:05:11.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.585+0000 7fa42d800640 1 -- 172.21.15.67:0/2146225851 >> 172.21.15.67:0/2146225851 conn(0x7fa4280cb480 msgr2=0x7fa42814d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:11.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.585+0000 7fa42d800640 1 -- 172.21.15.67:0/2146225851 shutdown_connections 2024-03-09T09:05:11.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:11.586+0000 7fa42d800640 1 -- 172.21.15.67:0/2146225851 wait complete. 2024-03-09T09:05:11.611 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:05:11.611 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:05:11.967 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 37% (93/247) Updating files: 38% (94/247) Updating files: 39% (97/247) Updating files: 40% (99/247) Updating files: 41% (102/247) Updating files: 42% (104/247) Updating files: 43% (107/247) Updating files: 44% (109/247) Updating files: 45% (112/247) Updating files: 46% (114/247) Updating files: 47% (117/247) Updating files: 48% (119/247) Updating files: 49% (122/247) Updating files: 50% (124/247) Updating files: 51% (126/247) Updating files: 52% (129/247) Updating files: 53% (131/247) Updating files: 54% (134/247) Updating files: 55% (136/247) Updating files: 56% (139/247) Updating files: 57% (141/247) Updating files: 58% (144/247) Updating files: 59% (146/247) Updating files: 60% (149/247) Updating files: 61% (151/247) Updating files: 62% (154/247) Updating files: 63% (156/247) Updating files: 64% (159/247) Updating files: 65% (161/247) Updating files: 66% (164/247) Updating files: 67% (166/247) Updating files: 68% (168/247) Updating files: 69% (171/247) Updating files: 70% (173/247) Updating files: 71% (176/247) Updating files: 72% (178/247) Updating files: 73% (181/247) Updating files: 74% (183/247) Updating files: 75% (186/247) Updating files: 76% (188/247) Updating files: 77% (191/247) Updating files: 78% (193/247) Updating files: 79% (196/247) Updating files: 80% (198/247) Updating files: 81% (201/247) Updating files: 82% (203/247) Updating files: 83% (206/247) Updating files: 84% (208/247) Updating files: 85% (210/247) Updating files: 86% (213/247) Updating files: 87% (215/247) Updating files: 88% (218/247) Updating files: 89% (220/247) Updating files: 90% (223/247) Updating files: 91% (225/247) Updating files: 92% (228/247) Updating files: 93% (230/247) Updating files: 94% (233/247) Updating files: 95% (235/247) Updating files: 95% (236/247) Updating files: 96% (238/247) Updating files: 97% (240/247) Updating files: 98% (243/247) Updating files: 99% (245/247) Updating files: 100% (247/247) Updating files: 100% (247/247), done. 2024-03-09T09:05:12.017 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 13b1bf7c492 kclient: initializing kaddr (merge fix) 2024-03-09T09:05:12.036 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.4 2024-03-09T09:05:12.832 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:05:12.832 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 5 -eq 48 ']' 2024-03-09T09:05:12.832 INFO:tasks.workunit.client.0.smithi067.stdout:v0.5 2024-03-09T09:05:12.833 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.5 2024-03-09T09:05:12.833 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.5 2024-03-09T09:05:12.833 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.5 2024-03-09T09:05:13.073 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 25% (62/247) Updating files: 26% (65/247) Updating files: 27% (67/247) Updating files: 28% (70/247) Updating files: 29% (72/247) Updating files: 30% (75/247) Updating files: 31% (77/247) Updating files: 32% (80/247) Updating files: 32% (81/247) Updating files: 33% (82/247) Updating files: 34% (84/247) Updating files: 35% (87/247) Updating files: 36% (89/247) Updating files: 37% (92/247) Updating files: 38% (94/247) Updating files: 39% (97/247) Updating files: 40% (99/247) Updating files: 41% (102/247) Updating files: 42% (104/247) Updating files: 43% (107/247) Updating files: 44% (109/247) Updating files: 45% (112/247) Updating files: 46% (114/247) Updating files: 47% (117/247) Updating files: 48% (119/247) Updating files: 49% (122/247) Updating files: 50% (124/247) Updating files: 51% (126/247) Updating files: 52% (129/247) Updating files: 53% (131/247) Updating files: 54% (134/247) Updating files: 55% (136/247) Updating files: 56% (139/247) Updating files: 57% (141/247) Updating files: 58% (144/247) Updating files: 59% (146/247) Updating files: 60% (149/247) Updating files: 61% (151/247) Updating files: 62% (154/247) Updating files: 63% (156/247) Updating files: 64% (159/247) Updating files: 65% (161/247) Updating files: 66% (164/247) Updating files: 67% (166/247) Updating files: 68% (168/247) Updating files: 69% (171/247) Updating files: 70% (173/247) Updating files: 71% (176/247) Updating files: 72% (178/247) Updating files: 73% (181/247) Updating files: 74% (183/247) Updating files: 75% (186/247) Updating files: 76% (188/247) Updating files: 77% (191/247) Updating files: 78% (193/247) Updating files: 78% (195/247) Updating files: 79% (196/247) Updating files: 80% (198/247) Updating files: 81% (201/247) Updating files: 82% (203/247) Updating files: 83% (206/247) Updating files: 84% (208/247) Updating files: 85% (210/247) Updating files: 86% (213/247) Updating files: 87% (215/247) Updating files: 88% (218/247) Updating files: 89% (220/247) Updating files: 90% (223/247) Updating files: 91% (225/247) Updating files: 92% (228/247) Updating files: 93% (230/247) Updating files: 94% (233/247) Updating files: 95% (235/247) Updating files: 96% (238/247) Updating files: 97% (240/247) Updating files: 98% (243/247) Updating files: 99% (245/247) Updating files: 100% (247/247) Updating files: 100% (247/247), done. 2024-03-09T09:05:13.131 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 13b1bf7c492 kclient: initializing kaddr (merge fix) 2024-03-09T09:05:13.142 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.4 2024-03-09T09:05:13.246 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:05:13.246 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 5 -eq 48 ']' 2024-03-09T09:05:13.246 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.5 2024-03-09T09:05:13.246 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.5 2024-03-09T09:05:13.246 INFO:tasks.workunit.client.1.smithi134.stdout:v0.5 2024-03-09T09:05:13.247 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.5 2024-03-09T09:05:15.238 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:05:15.245 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:05:15.780 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 27% (67/245) Updating files: 28% (69/245) Updating files: 29% (72/245) Updating files: 30% (74/245) Updating files: 31% (76/245) Updating files: 32% (79/245) Updating files: 33% (81/245) Updating files: 34% (84/245) Updating files: 35% (86/245) Updating files: 36% (89/245) Updating files: 37% (91/245) Updating files: 38% (94/245) Updating files: 39% (96/245) Updating files: 40% (98/245) Updating files: 41% (101/245) Updating files: 42% (103/245) Updating files: 43% (106/245) Updating files: 44% (108/245) Updating files: 45% (111/245) Updating files: 46% (113/245) Updating files: 47% (116/245) Updating files: 48% (118/245) Updating files: 49% (121/245) Updating files: 50% (123/245) Updating files: 51% (125/245) Updating files: 52% (128/245) Updating files: 53% (130/245) Updating files: 54% (133/245) Updating files: 55% (135/245) Updating files: 56% (138/245) Updating files: 57% (140/245) Updating files: 58% (143/245) Updating files: 59% (145/245) Updating files: 60% (147/245) Updating files: 61% (150/245) Updating files: 62% (152/245) Updating files: 63% (155/245) Updating files: 63% (156/245) Updating files: 64% (157/245) Updating files: 65% (160/245) Updating files: 66% (162/245) Updating files: 67% (165/245) Updating files: 68% (167/245) Updating files: 69% (170/245) Updating files: 70% (172/245) Updating files: 71% (174/245) Updating files: 72% (177/245) Updating files: 73% (179/245) Updating files: 74% (182/245) Updating files: 75% (184/245) Updating files: 76% (187/245) Updating files: 77% (189/245) Updating files: 78% (192/245) Updating files: 79% (194/245) Updating files: 80% (196/245) Updating files: 81% (199/245) Updating files: 82% (201/245) Updating files: 83% (204/245) Updating files: 84% (206/245) Updating files: 85% (209/245) Updating files: 86% (211/245) Updating files: 87% (214/245) Updating files: 88% (216/245) Updating files: 89% (219/245) Updating files: 90% (221/245) Updating files: 91% (223/245) Updating files: 92% (226/245) Updating files: 93% (228/245) Updating files: 94% (231/245) Updating files: 95% (233/245) Updating files: 96% (236/245) Updating files: 97% (238/245) Updating files: 98% (241/245) Updating files: 99% (243/245) Updating files: 100% (245/245) Updating files: 100% (245/245), done. 2024-03-09T09:05:16.841 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 1dd420957f9 journal: detect size of raw block devices properly 2024-03-09T09:05:16.855 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.5 2024-03-09T09:05:17.007 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:05:17.007 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 6 -eq 48 ']' 2024-03-09T09:05:17.007 INFO:tasks.workunit.client.1.smithi134.stdout:v0.6 2024-03-09T09:05:17.008 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.6 2024-03-09T09:05:17.008 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.6 2024-03-09T09:05:17.008 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.6 2024-03-09T09:05:17.890 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 31% (77/245) Updating files: 32% (79/245) Updating files: 33% (81/245) Updating files: 34% (84/245) Updating files: 35% (86/245) Updating files: 36% (89/245) Updating files: 37% (91/245) Updating files: 38% (94/245) Updating files: 39% (96/245) Updating files: 40% (98/245) Updating files: 41% (101/245) Updating files: 42% (103/245) Updating files: 42% (104/245) Updating files: 43% (106/245) Updating files: 44% (108/245) Updating files: 45% (111/245) Updating files: 46% (113/245) Updating files: 47% (116/245) Updating files: 48% (118/245) Updating files: 49% (121/245) Updating files: 50% (123/245) Updating files: 51% (125/245) Updating files: 52% (128/245) Updating files: 53% (130/245) Updating files: 54% (133/245) Updating files: 55% (135/245) Updating files: 56% (138/245) Updating files: 57% (140/245) Updating files: 58% (143/245) Updating files: 59% (145/245) Updating files: 60% (147/245) Updating files: 61% (150/245) Updating files: 62% (152/245) Updating files: 63% (155/245) Updating files: 64% (157/245) Updating files: 65% (160/245) Updating files: 66% (162/245) Updating files: 67% (165/245) Updating files: 67% (166/245) Updating files: 68% (167/245) Updating files: 69% (170/245) Updating files: 70% (172/245) Updating files: 71% (174/245) Updating files: 72% (177/245) Updating files: 73% (179/245) Updating files: 74% (182/245) Updating files: 75% (184/245) Updating files: 76% (187/245) Updating files: 77% (189/245) Updating files: 78% (192/245) Updating files: 79% (194/245) Updating files: 80% (196/245) Updating files: 81% (199/245) Updating files: 82% (201/245) Updating files: 83% (204/245) Updating files: 83% (205/245) Updating files: 84% (206/245) Updating files: 85% (209/245) Updating files: 86% (211/245) Updating files: 87% (214/245) Updating files: 88% (216/245) Updating files: 89% (219/245) Updating files: 90% (221/245) Updating files: 91% (223/245) Updating files: 92% (226/245) Updating files: 93% (228/245) Updating files: 94% (231/245) Updating files: 95% (233/245) Updating files: 96% (236/245) Updating files: 97% (238/245) Updating files: 98% (241/245) Updating files: 99% (243/245) Updating files: 100% (245/245) Updating files: 100% (245/245), done. 2024-03-09T09:05:17.948 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 1dd420957f9 journal: detect size of raw block devices properly 2024-03-09T09:05:17.967 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.5 2024-03-09T09:05:19.083 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:05:19.083 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 6 -eq 48 ']' 2024-03-09T09:05:19.083 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.6 2024-03-09T09:05:19.083 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.6 2024-03-09T09:05:19.083 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.6 2024-03-09T09:05:19.083 INFO:tasks.workunit.client.0.smithi067.stdout:v0.6 2024-03-09T09:05:19.874 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 28% (65/227) Updating files: 29% (66/227) Updating files: 30% (69/227) Updating files: 31% (71/227) Updating files: 32% (73/227) Updating files: 33% (75/227) Updating files: 34% (78/227) Updating files: 35% (80/227) Updating files: 36% (82/227) Updating files: 37% (84/227) Updating files: 38% (87/227) Updating files: 39% (89/227) Updating files: 40% (91/227) Updating files: 41% (94/227) Updating files: 42% (96/227) Updating files: 43% (98/227) Updating files: 44% (100/227) Updating files: 45% (103/227) Updating files: 46% (105/227) Updating files: 47% (107/227) Updating files: 48% (109/227) Updating files: 49% (112/227) Updating files: 50% (114/227) Updating files: 51% (116/227) Updating files: 52% (119/227) Updating files: 53% (121/227) Updating files: 54% (123/227) Updating files: 55% (125/227) Updating files: 56% (128/227) Updating files: 57% (130/227) Updating files: 58% (132/227) Updating files: 59% (134/227) Updating files: 60% (137/227) Updating files: 61% (139/227) Updating files: 62% (141/227) Updating files: 63% (144/227) Updating files: 64% (146/227) Updating files: 65% (148/227) Updating files: 66% (150/227) Updating files: 67% (153/227) Updating files: 68% (155/227) Updating files: 69% (157/227) Updating files: 70% (159/227) Updating files: 71% (162/227) Updating files: 72% (164/227) Updating files: 73% (166/227) Updating files: 74% (168/227) Updating files: 75% (171/227) Updating files: 76% (173/227) Updating files: 77% (175/227) Updating files: 78% (178/227) Updating files: 79% (180/227) Updating files: 80% (182/227) Updating files: 81% (184/227) Updating files: 82% (187/227) Updating files: 83% (189/227) Updating files: 84% (191/227) Updating files: 85% (193/227) Updating files: 86% (196/227) Updating files: 87% (198/227) Updating files: 88% (200/227) Updating files: 89% (203/227) Updating files: 90% (205/227) Updating files: 91% (207/227) Updating files: 92% (209/227) Updating files: 93% (212/227) Updating files: 94% (214/227) Updating files: 94% (215/227) Updating files: 95% (216/227) Updating files: 96% (218/227) Updating files: 97% (221/227) Updating files: 98% (223/227) Updating files: 99% (225/227) Updating files: 100% (227/227) Updating files: 100% (227/227), done. 2024-03-09T09:05:19.938 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at a4c752df713 v0.6 2024-03-09T09:05:19.952 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.6 2024-03-09T09:05:20.859 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:05:20.859 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 7 -eq 48 ']' 2024-03-09T09:05:20.860 INFO:tasks.workunit.client.1.smithi134.stdout:v0.7 2024-03-09T09:05:20.860 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.7 2024-03-09T09:05:20.860 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.7 2024-03-09T09:05:20.860 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.7 2024-03-09T09:05:31.612 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:05:31.613 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:05:31.613 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:05:31.613 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:05:31.621 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:05:31.621 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:05:31.621 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:05:31.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.835+0000 7fb683e00640 1 Processor -- start 2024-03-09T09:05:31.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.835+0000 7fb683e00640 1 -- start start 2024-03-09T09:05:31.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.836+0000 7fb683e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 0x7fb684150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:31.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.836+0000 7fb683e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6840c81e0 con 0x7fb6840710a0 2024-03-09T09:05:31.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.836+0000 7fb683e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6840c32b0 con 0x7fb684154080 2024-03-09T09:05:31.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.836+0000 7fb683e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840c2fd0 con 0x7fb6840c1f70 2024-03-09T09:05:31.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.837+0000 7fb682a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 0x7fb684150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.837+0000 7fb682a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 0x7fb684150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:60628/0 (socket says 172.21.15.67:60628) 2024-03-09T09:05:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.837+0000 7fb683400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb684154080 0x7fb68415e2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:49142/0 (socket says 172.21.15.67:49142) 2024-03-09T09:05:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.837+0000 7fb682a00640 1 -- 172.21.15.67:0/2369729644 learned_addr learned my addr 172.21.15.67:0/2369729644 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.837+0000 7fb67a000640 1 --1- 172.21.15.67:0/2369729644 >> v1:172.21.15.67:6789/0 conn(0x7fb6840710a0 0x7fb684153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42792/0 (socket says 172.21.15.67:42792) 2024-03-09T09:05:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.837+0000 7fb682a00640 1 -- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 msgr2=0x7fb684150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:05:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.837+0000 7fb682a00640 1 -- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 msgr2=0x7fb684150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:05:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.837+0000 7fb682a00640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 0x7fb684150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:31.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.837+0000 7fb682a00640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 0x7fb684150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:31.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.837+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286544206 0 0) 0x7fb6840c32b0 con 0x7fb684154080 2024-03-09T09:05:31.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.837+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb668003520 con 0x7fb684154080 2024-03-09T09:05:31.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608110005 0 0) 0x7fb6840c81e0 con 0x7fb6840710a0 2024-03-09T09:05:31.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6840c32b0 con 0x7fb6840710a0 2024-03-09T09:05:31.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1795423094 0 0) 0x7fb668003520 con 0x7fb684154080 2024-03-09T09:05:31.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 msgr2=0x7fb684150c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:31.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 0x7fb684150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:31.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 >> v1:172.21.15.67:6789/0 conn(0x7fb6840710a0 legacy=0x7fb684153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:31.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6840c3700 con 0x7fb684154080 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb67c001670 con 0x7fb684154080 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 >> v1:172.21.15.134:6789/0 conn(0x7fb684154080 legacy=0x7fb68415e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb668003c00 0x7fb668004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb668004800 0x7fb668004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 0x7fb66800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840c2fd0 con 0x7fb668004800 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb668005570 con 0x7fb668003c00 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66800bf00 con 0x7fb6840c1f70 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb67a000640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb668004800 0x7fb668004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.838+0000 7fb682a00640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb668003c00 0x7fb668004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.839+0000 7fb683400640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 0x7fb66800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.839+0000 7fb682a00640 1 -- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 msgr2=0x7fb66800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.839+0000 7fb682a00640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6840c1f70 0x7fb66800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:31.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.839+0000 7fb682a00640 1 -- 172.21.15.67:0/2369729644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb668004800 msgr2=0x7fb668004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.839+0000 7fb682a00640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb668004800 0x7fb668004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.839+0000 7fb682a00640 1 -- 172.21.15.67:0/2369729644 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb668003520 con 0x7fb668003c00 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.839+0000 7fb67a000640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb668004800 0x7fb668004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.840+0000 7fb682a00640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb668003c00 0x7fb668004000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb674009cc0 tx=0x7fb668005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b769f7ee3e8d81 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.840+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674032070 con 0x7fb668003c00 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.840+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67402cb10 con 0x7fb668003c00 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.840+0000 7fb682000640 1 -- 172.21.15.67:0/2369729644 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740098c0 con 0x7fb668003c00 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.841+0000 7fb683e00640 1 -- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb668003c00 msgr2=0x7fb668004000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.841+0000 7fb683e00640 1 --2- 172.21.15.67:0/2369729644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb668003c00 0x7fb668004000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb674009cc0 tx=0x7fb668005830 comp rx=0 tx=0).stop 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.841+0000 7fb683400640 1 -- 172.21.15.67:0/2369729644 reap_dead start 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.841+0000 7fb683e00640 1 -- 172.21.15.67:0/2369729644 shutdown_connections 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.841+0000 7fb683e00640 1 -- 172.21.15.67:0/2369729644 >> 172.21.15.67:0/2369729644 conn(0x7fb6840c7300 msgr2=0x7fb6840c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.841+0000 7fb683e00640 1 -- 172.21.15.67:0/2369729644 shutdown_connections 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.841+0000 7fb683e00640 1 -- 172.21.15.67:0/2369729644 wait complete. 2024-03-09T09:05:31.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.843+0000 7fb683e00640 1 Processor -- start 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.843+0000 7fb683e00640 1 -- start start 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.844+0000 7fb683e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb668003c00 0x7fb684154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.844+0000 7fb683e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6840c6110 0x7fb6841ebd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.844+0000 7fb683e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68414c2e0 0x7fb68414c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.844+0000 7fb683e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb684070d50 con 0x7fb6840c6110 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.844+0000 7fb683e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb684070830 con 0x7fb68414c2e0 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.844+0000 7fb683e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb684070ed0 con 0x7fb668003c00 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.844+0000 7fb67a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6840c6110 0x7fb6841ebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb67a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6840c6110 0x7fb6841ebd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51610/0 (socket says 172.21.15.67:51610) 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb67a000640 1 -- 172.21.15.67:0/1679479913 learned_addr learned my addr 172.21.15.67:0/1679479913 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb682a00640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb668003c00 0x7fb684154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb683400640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68414c2e0 0x7fb68414c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb682a00640 1 -- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb668003c00 msgr2=0x7fb684154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:05:31.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb682a00640 1 -- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb668003c00 msgr2=0x7fb684154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb682a00640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb668003c00 0x7fb684154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb682a00640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb668003c00 0x7fb684154d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb67a000640 1 -- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb668003c00 msgr2=0x7fb684154d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb67a000640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb668003c00 0x7fb684154d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb67a000640 1 -- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68414c2e0 msgr2=0x7fb68414c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb67a000640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68414c2e0 0x7fb68414c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb67a000640 1 -- 172.21.15.67:0/1679479913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6841eb5a0 con 0x7fb6840c6110 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.845+0000 7fb67a000640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6840c6110 0x7fb6841ebd60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb66c001820 tx=0x7fb66c001ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.846+0000 7fb67b400640 1 -- 172.21.15.67:0/1679479913 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c00f940 con 0x7fb6840c6110 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.846+0000 7fb67b400640 1 -- 172.21.15.67:0/1679479913 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c018070 con 0x7fb6840c6110 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.846+0000 7fb67b400640 1 -- 172.21.15.67:0/1679479913 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c00fce0 con 0x7fb6840c6110 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.846+0000 7fb683e00640 1 -- 172.21.15.67:0/1679479913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6841ef590 con 0x7fb6840c6110 2024-03-09T09:05:31.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.846+0000 7fb683e00640 1 -- 172.21.15.67:0/1679479913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6840c6ab0 con 0x7fb6840c6110 2024-03-09T09:05:31.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.847+0000 7fb683e00640 1 -- 172.21.15.67:0/1679479913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6840679a0 con 0x7fb6840c6110 2024-03-09T09:05:31.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.848+0000 7fb67b400640 1 -- 172.21.15.67:0/1679479913 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb66c01d460 con 0x7fb6840c6110 2024-03-09T09:05:31.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.849+0000 7fb67b400640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb66006b500 0x7fb66006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:31.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.849+0000 7fb67b400640 1 -- 172.21.15.67:0/1679479913 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb66c08a640 con 0x7fb6840c6110 2024-03-09T09:05:31.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.849+0000 7fb682a00640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb66006b500 0x7fb66006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:31.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.850+0000 7fb682a00640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb66006b500 0x7fb66006d9c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb674037000 tx=0x7fb674032000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:31.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:31.854+0000 7fb67b400640 1 -- 172.21.15.67:0/1679479913 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb6840679a0 con 0x7fb6840c6110 2024-03-09T09:05:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.127+0000 7fb683e00640 1 -- 172.21.15.67:0/1679479913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb684071dd0 con 0x7fb6840c6110 2024-03-09T09:05:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.128+0000 7fb67b400640 1 -- 172.21.15.67:0/1679479913 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fb66c059100 con 0x7fb6840c6110 2024-03-09T09:05:32.131 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":96,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:05:32.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 -- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb66006b500 msgr2=0x7fb66006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:32.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb66006b500 0x7fb66006d9c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb674037000 tx=0x7fb674032000 comp rx=0 tx=0).stop 2024-03-09T09:05:32.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 -- 172.21.15.67:0/1679479913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6840c6110 msgr2=0x7fb6841ebd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:32.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6840c6110 0x7fb6841ebd60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb66c001820 tx=0x7fb66c001ce0 comp rx=0 tx=0).stop 2024-03-09T09:05:32.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 -- 172.21.15.67:0/1679479913 shutdown_connections 2024-03-09T09:05:32.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb66006b500 0x7fb66006d9c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:32.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb68414c2e0 0x7fb68414c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:32.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6840c6110 0x7fb6841ebd60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:32.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 --2- 172.21.15.67:0/1679479913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb668003c00 0x7fb684154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:32.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 -- 172.21.15.67:0/1679479913 >> 172.21.15.67:0/1679479913 conn(0x7fb6840c7300 msgr2=0x7fb6840c07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:32.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 -- 172.21.15.67:0/1679479913 shutdown_connections 2024-03-09T09:05:32.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:32.131+0000 7fb683e00640 1 -- 172.21.15.67:0/1679479913 wait complete. 2024-03-09T09:05:35.156 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:05:35.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.380+0000 7ff8fbe00640 1 Processor -- start 2024-03-09T09:05:35.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.380+0000 7ff8fbe00640 1 -- start start 2024-03-09T09:05:35.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.381+0000 7ff8fbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff8fc14dad0 0x7ff8fc06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:35.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.382+0000 7ff8fbe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 0x7ff8fc154290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:35.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.382+0000 7ff8fbe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc0c30e0 con 0x7ff8fc06b940 2024-03-09T09:05:35.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.382+0000 7ff8fbe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c33c0 con 0x7ff8fc14dad0 2024-03-09T09:05:35.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.382+0000 7ff8fbe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c36a0 con 0x7ff8fc153e60 2024-03-09T09:05:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.382+0000 7ff8faa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff8fc14dad0 0x7ff8fc06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.382+0000 7ff8faa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff8fc14dad0 0x7ff8fc06e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51624/0 (socket says 172.21.15.67:51624) 2024-03-09T09:05:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.382+0000 7ff8faa00640 1 -- 172.21.15.67:0/367536428 learned_addr learned my addr 172.21.15.67:0/367536428 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.382+0000 7ff8fa000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff8fc06b940 0x7ff8fc06ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42800/0 (socket says 172.21.15.67:42800) 2024-03-09T09:05:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.383+0000 7ff8fb400640 1 --2- 172.21.15.67:0/367536428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 0x7ff8fc154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.383+0000 7ff8faa00640 1 -- 172.21.15.67:0/367536428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 msgr2=0x7ff8fc154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.383+0000 7ff8faa00640 1 --2- 172.21.15.67:0/367536428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 0x7ff8fc154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.383+0000 7ff8faa00640 1 -- 172.21.15.67:0/367536428 >> v1:172.21.15.67:6789/0 conn(0x7ff8fc06b940 legacy=0x7ff8fc06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.383+0000 7ff8faa00640 1 -- 172.21.15.67:0/367536428 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc0c3980 con 0x7ff8fc14dad0 2024-03-09T09:05:35.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.383+0000 7ff8fb400640 1 --2- 172.21.15.67:0/367536428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 0x7ff8fc154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:05:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.384+0000 7ff8faa00640 1 --2- 172.21.15.67:0/367536428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc14dad0 0x7ff8fc06e470 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009870 tx=0x7ff8ec02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bacc76f2a15ffc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.384+0000 7ff8f9600640 1 -- 172.21.15.67:0/367536428 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0317c0 con 0x7ff8fc14dad0 2024-03-09T09:05:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.384+0000 7ff8f9600640 1 -- 172.21.15.67:0/367536428 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec037070 con 0x7ff8fc14dad0 2024-03-09T09:05:35.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.384+0000 7ff8f9600640 1 -- 172.21.15.67:0/367536428 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec031b80 con 0x7ff8fc14dad0 2024-03-09T09:05:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.385+0000 7ff8fbe00640 1 -- 172.21.15.67:0/367536428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc14dad0 msgr2=0x7ff8fc06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.385+0000 7ff8fbe00640 1 --2- 172.21.15.67:0/367536428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc14dad0 0x7ff8fc06e470 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009870 tx=0x7ff8ec02b860 comp rx=0 tx=0).stop 2024-03-09T09:05:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.385+0000 7ff8fbe00640 1 -- 172.21.15.67:0/367536428 shutdown_connections 2024-03-09T09:05:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.385+0000 7ff8fbe00640 1 --2- 172.21.15.67:0/367536428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 0x7ff8fc154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:35.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.385+0000 7ff8fbe00640 1 --2- 172.21.15.67:0/367536428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc14dad0 0x7ff8fc06e470 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.385+0000 7ff8fbe00640 1 -- 172.21.15.67:0/367536428 >> 172.21.15.67:0/367536428 conn(0x7ff8fc06d650 msgr2=0x7ff8fc06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.385+0000 7ff8fbe00640 1 -- 172.21.15.67:0/367536428 shutdown_connections 2024-03-09T09:05:35.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.385+0000 7ff8fbe00640 1 -- 172.21.15.67:0/367536428 wait complete. 2024-03-09T09:05:35.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.387+0000 7ff8fbe00640 1 Processor -- start 2024-03-09T09:05:35.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.387+0000 7ff8fbe00640 1 -- start start 2024-03-09T09:05:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.388+0000 7ff8fbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8fc06b940 0x7ff8fc1496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.388+0000 7ff8fbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc14dad0 0x7ff8fc149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.388+0000 7ff8fbe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 0x7ff8fc14eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.388+0000 7ff8fbe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc06f670 con 0x7ff8fc14dad0 2024-03-09T09:05:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.388+0000 7ff8fbe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc06f4f0 con 0x7ff8fc06b940 2024-03-09T09:05:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.388+0000 7ff8fbe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc14deb0 con 0x7ff8fc153e60 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.388+0000 7ff8fa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc14dad0 0x7ff8fc149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.388+0000 7ff8fa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc14dad0 0x7ff8fc149c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51632/0 (socket says 172.21.15.67:51632) 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.388+0000 7ff8fa000640 1 -- 172.21.15.67:0/2736930207 learned_addr learned my addr 172.21.15.67:0/2736930207 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.389+0000 7ff8fb400640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 0x7ff8fc14eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.389+0000 7ff8faa00640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8fc06b940 0x7ff8fc1496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.389+0000 7ff8fb400640 1 -- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 msgr2=0x7ff8fc14eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.389+0000 7ff8fb400640 1 -- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 msgr2=0x7ff8fc14eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.389+0000 7ff8fb400640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 0x7ff8fc14eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.389+0000 7ff8fa000640 1 -- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 msgr2=0x7ff8fc14eaf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.389+0000 7ff8fb400640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 0x7ff8fc14eaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.389+0000 7ff8fa000640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 0x7ff8fc14eaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.389+0000 7ff8fa000640 1 -- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8fc06b940 msgr2=0x7ff8fc1496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.389+0000 7ff8fa000640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8fc06b940 0x7ff8fc1496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.389+0000 7ff8fa000640 1 -- 172.21.15.67:0/2736930207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc147c30 con 0x7ff8fc14dad0 2024-03-09T09:05:35.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.390+0000 7ff8fa000640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc14dad0 0x7ff8fc149c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800c440 tx=0x7ff8e800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:35.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.390+0000 7ff8f2a00640 1 -- 172.21.15.67:0/2736930207 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80074e0 con 0x7ff8fc14dad0 2024-03-09T09:05:35.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.390+0000 7ff8f2a00640 1 -- 172.21.15.67:0/2736930207 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e8007680 con 0x7ff8fc14dad0 2024-03-09T09:05:35.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.390+0000 7ff8f2a00640 1 -- 172.21.15.67:0/2736930207 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8004a20 con 0x7ff8fc14dad0 2024-03-09T09:05:35.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.390+0000 7ff8fbe00640 1 -- 172.21.15.67:0/2736930207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc147f10 con 0x7ff8fc14dad0 2024-03-09T09:05:35.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.390+0000 7ff8fbe00640 1 -- 172.21.15.67:0/2736930207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc14ad50 con 0x7ff8fc14dad0 2024-03-09T09:05:35.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.391+0000 7ff8fbe00640 1 -- 172.21.15.67:0/2736930207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc14d630 con 0x7ff8fc14dad0 2024-03-09T09:05:35.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.393+0000 7ff8f2a00640 1 -- 172.21.15.67:0/2736930207 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff8e801a020 con 0x7ff8fc14dad0 2024-03-09T09:05:35.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.394+0000 7ff8f2a00640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8dc06b4d0 0x7ff8dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:35.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.394+0000 7ff8f2a00640 1 -- 172.21.15.67:0/2736930207 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff8e8089d40 con 0x7ff8fc14dad0 2024-03-09T09:05:35.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.395+0000 7ff8faa00640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8dc06b4d0 0x7ff8dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:35.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.396+0000 7ff8faa00640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8dc06b4d0 0x7ff8dc06d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec036000 tx=0x7ff8ec03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:35.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.401+0000 7ff8f2a00640 1 -- 172.21.15.67:0/2736930207 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff8fc14d630 con 0x7ff8fc14dad0 2024-03-09T09:05:35.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.683+0000 7ff8fbe00640 1 -- 172.21.15.67:0/2736930207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff8fc075a00 con 0x7ff8fc14dad0 2024-03-09T09:05:35.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.684+0000 7ff8f2a00640 1 -- 172.21.15.67:0/2736930207 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7ff8fc075a00 con 0x7ff8fc14dad0 2024-03-09T09:05:35.687 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":96,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:05:35.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 -- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8dc06b4d0 msgr2=0x7ff8dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:35.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8dc06b4d0 0x7ff8dc06d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec036000 tx=0x7ff8ec03d040 comp rx=0 tx=0).stop 2024-03-09T09:05:35.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 -- 172.21.15.67:0/2736930207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc14dad0 msgr2=0x7ff8fc149c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:35.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc14dad0 0x7ff8fc149c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800c440 tx=0x7ff8e800c900 comp rx=0 tx=0).stop 2024-03-09T09:05:35.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 -- 172.21.15.67:0/2736930207 shutdown_connections 2024-03-09T09:05:35.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8dc06b4d0 0x7ff8dc06d990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:35.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8fc153e60 0x7ff8fc14eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:35.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8fc14dad0 0x7ff8fc149c30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:35.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 --2- 172.21.15.67:0/2736930207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8fc06b940 0x7ff8fc1496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:35.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 -- 172.21.15.67:0/2736930207 >> 172.21.15.67:0/2736930207 conn(0x7ff8fc06d650 msgr2=0x7ff8fc154290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:35.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 -- 172.21.15.67:0/2736930207 shutdown_connections 2024-03-09T09:05:35.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:35.687+0000 7ff8fbe00640 1 -- 172.21.15.67:0/2736930207 wait complete. 2024-03-09T09:05:37.253 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 8% (18/201) Updating files: 9% (19/201) Updating files: 10% (21/201) Updating files: 11% (23/201) Updating files: 12% (25/201) Updating files: 13% (27/201) Updating files: 14% (29/201) Updating files: 15% (31/201) Updating files: 16% (33/201) Updating files: 17% (35/201) Updating files: 18% (37/201) Updating files: 19% (39/201) Updating files: 20% (41/201) Updating files: 21% (43/201) Updating files: 22% (45/201) Updating files: 23% (47/201) Updating files: 24% (49/201) Updating files: 25% (51/201) Updating files: 26% (53/201) Updating files: 27% (55/201) Updating files: 28% (57/201) Updating files: 29% (59/201) Updating files: 30% (61/201) Updating files: 31% (63/201) Updating files: 32% (65/201) Updating files: 33% (67/201) Updating files: 34% (69/201) Updating files: 35% (71/201) Updating files: 36% (73/201) Updating files: 37% (75/201) Updating files: 38% (77/201) Updating files: 39% (79/201) Updating files: 40% (81/201) Updating files: 41% (83/201) Updating files: 42% (85/201) Updating files: 43% (87/201) Updating files: 44% (89/201) Updating files: 45% (91/201) Updating files: 46% (93/201) Updating files: 47% (95/201) Updating files: 48% (97/201) Updating files: 49% (99/201) Updating files: 50% (101/201) Updating files: 51% (103/201) Updating files: 52% (105/201) Updating files: 53% (107/201) Updating files: 54% (109/201) Updating files: 55% (111/201) Updating files: 56% (113/201) Updating files: 57% (115/201) Updating files: 58% (117/201) Updating files: 59% (119/201) Updating files: 60% (121/201) Updating files: 61% (123/201) Updating files: 62% (125/201) Updating files: 63% (127/201) Updating files: 64% (129/201) Updating files: 65% (131/201) Updating files: 66% (133/201) Updating files: 67% (135/201) Updating files: 68% (137/201) Updating files: 69% (139/201) Updating files: 70% (141/201) Updating files: 71% (143/201) Updating files: 72% (145/201) Updating files: 73% (147/201) Updating files: 74% (149/201) Updating files: 75% (151/201) Updating files: 76% (153/201) Updating files: 77% (155/201) Updating files: 78% (157/201) Updating files: 79% (159/201) Updating files: 80% (161/201) Updating files: 81% (163/201) Updating files: 82% (165/201) Updating files: 83% (167/201) Updating files: 84% (169/201) Updating files: 84% (170/201) Updating files: 85% (171/201) Updating files: 86% (173/201) Updating files: 87% (175/201) Updating files: 88% (177/201) Updating files: 89% (179/201) Updating files: 90% (181/201) Updating files: 91% (183/201) Updating files: 92% (185/201) Updating files: 93% (187/201) Updating files: 94% (189/201) Updating files: 95% (191/201) Updating files: 96% (193/201) Updating files: 97% (195/201) Updating files: 98% (197/201) Updating files: 99% (199/201) Updating files: 100% (201/201) Updating files: 100% (201/201), done. 2024-03-09T09:05:37.628 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 29% (68/227) Updating files: 30% (69/227) Updating files: 31% (71/227) Updating files: 32% (73/227) Updating files: 33% (75/227) Updating files: 34% (78/227) Updating files: 35% (80/227) Updating files: 36% (82/227) Updating files: 37% (84/227) Updating files: 37% (86/227) Updating files: 38% (87/227) Updating files: 39% (89/227) Updating files: 40% (91/227) Updating files: 41% (94/227) Updating files: 42% (96/227) Updating files: 43% (98/227) Updating files: 44% (100/227) Updating files: 45% (103/227) Updating files: 46% (105/227) Updating files: 47% (107/227) Updating files: 48% (109/227) Updating files: 48% (111/227) Updating files: 49% (112/227) Updating files: 50% (114/227) Updating files: 51% (116/227) Updating files: 52% (119/227) Updating files: 53% (121/227) Updating files: 54% (123/227) Updating files: 55% (125/227) Updating files: 56% (128/227) Updating files: 57% (130/227) Updating files: 58% (132/227) Updating files: 59% (134/227) Updating files: 60% (137/227) Updating files: 61% (139/227) Updating files: 62% (141/227) Updating files: 63% (144/227) Updating files: 64% (146/227) Updating files: 65% (148/227) Updating files: 66% (150/227) Updating files: 67% (153/227) Updating files: 68% (155/227) Updating files: 69% (157/227) Updating files: 70% (159/227) Updating files: 71% (162/227) Updating files: 72% (164/227) Updating files: 72% (165/227) Updating files: 73% (166/227) Updating files: 74% (168/227) Updating files: 75% (171/227) Updating files: 76% (173/227) Updating files: 77% (175/227) Updating files: 78% (178/227) Updating files: 79% (180/227) Updating files: 80% (182/227) Updating files: 81% (184/227) Updating files: 82% (187/227) Updating files: 83% (189/227) Updating files: 84% (191/227) Updating files: 85% (193/227) Updating files: 85% (195/227) Updating files: 86% (196/227) Updating files: 87% (198/227) Updating files: 88% (200/227) Updating files: 89% (203/227) Updating files: 90% (205/227) Updating files: 91% (207/227) Updating files: 92% (209/227) Updating files: 93% (212/227) Updating files: 94% (214/227) Updating files: 95% (216/227) Updating files: 96% (218/227) Updating files: 97% (221/227) Updating files: 98% (223/227) Updating files: 99% (225/227) Updating files: 100% (227/227) Updating files: 100% (227/227), done. 2024-03-09T09:05:37.666 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at a4c752df713 v0.6 2024-03-09T09:05:37.680 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.6 2024-03-09T09:05:37.938 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:05:37.938 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 7 -eq 48 ']' 2024-03-09T09:05:37.938 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.7 2024-03-09T09:05:37.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.7 2024-03-09T09:05:37.939 INFO:tasks.workunit.client.0.smithi067.stdout:v0.7 2024-03-09T09:05:37.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.7 2024-03-09T09:05:38.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:05:38.768 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at c46e1b320c6 mkcephfs: btrfsctl -a between mkfs and mount 2024-03-09T09:05:38.785 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.7 2024-03-09T09:05:38.859 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:05:38.860 INFO:tasks.workunit.client.1.smithi134.stdout:v0.8 2024-03-09T09:05:38.860 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 8 -eq 48 ']' 2024-03-09T09:05:38.860 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.8 2024-03-09T09:05:38.860 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.8 2024-03-09T09:05:38.860 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.8 2024-03-09T09:05:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.962+0000 7fa118a00640 1 Processor -- start 2024-03-09T09:05:38.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.962+0000 7fa118a00640 1 -- start start 2024-03-09T09:05:38.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.963+0000 7fa118a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa114153250 0x7fa11406fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.963+0000 7fa118a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa1140c28a0 0x7fa1140c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.964+0000 7fa118a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa114153630 con 0x7fa114073010 2024-03-09T09:05:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.964+0000 7fa118a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1140aa5f0 con 0x7fa1140c28a0 2024-03-09T09:05:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.964+0000 7fa118a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1140abfe0 con 0x7fa114153250 2024-03-09T09:05:38.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.964+0000 7fa113200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa1140c28a0 0x7fa1140c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:38.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.964+0000 7fa10be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa114073010 0x7fa114070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34110/0 (socket says 172.21.15.67:34110) 2024-03-09T09:05:38.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.964+0000 7fa10be00640 1 -- 172.21.15.67:0/351370028 learned_addr learned my addr 172.21.15.67:0/351370028 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:05:38.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.964+0000 7fa112800640 1 --2- 172.21.15.67:0/351370028 >> v2:172.21.15.134:3300/0 conn(0x7fa114153250 0x7fa11406fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:38.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.965+0000 7fa113200640 1 -- 172.21.15.67:0/351370028 >> v2:172.21.15.134:3300/0 conn(0x7fa114153250 msgr2=0x7fa11406fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:38.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.965+0000 7fa113200640 1 --2- 172.21.15.67:0/351370028 >> v2:172.21.15.134:3300/0 conn(0x7fa114153250 0x7fa11406fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:38.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.965+0000 7fa113200640 1 -- 172.21.15.67:0/351370028 >> v1:172.21.15.67:6789/0 conn(0x7fa114073010 legacy=0x7fa114070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:38.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.965+0000 7fa113200640 1 -- 172.21.15.67:0/351370028 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1140abd00 con 0x7fa1140c28a0 2024-03-09T09:05:38.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.964+0000 7fa111e00640 1 -- 172.21.15.67:0/351370028 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332557959 0 0) 0x7fa114153630 con 0x7fa114073010 2024-03-09T09:05:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.966+0000 7fa113200640 1 --2- 172.21.15.67:0/351370028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1140c28a0 0x7fa1140c2cf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa11406e760 tx=0x7fa10c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7964afa49cfdf976 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.966+0000 7fa111e00640 1 -- 172.21.15.67:0/351370028 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c039070 con 0x7fa1140c28a0 2024-03-09T09:05:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.966+0000 7fa111e00640 1 -- 172.21.15.67:0/351370028 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c02cc70 con 0x7fa1140c28a0 2024-03-09T09:05:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.966+0000 7fa111e00640 1 -- 172.21.15.67:0/351370028 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c034660 con 0x7fa1140c28a0 2024-03-09T09:05:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.967+0000 7fa118a00640 1 -- 172.21.15.67:0/351370028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1140c28a0 msgr2=0x7fa1140c2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.967+0000 7fa118a00640 1 --2- 172.21.15.67:0/351370028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1140c28a0 0x7fa1140c2cf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa11406e760 tx=0x7fa10c02bec0 comp rx=0 tx=0).stop 2024-03-09T09:05:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.967+0000 7fa118a00640 1 -- 172.21.15.67:0/351370028 shutdown_connections 2024-03-09T09:05:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.967+0000 7fa118a00640 1 --2- 172.21.15.67:0/351370028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1140c28a0 0x7fa1140c2cf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.967+0000 7fa118a00640 1 --2- 172.21.15.67:0/351370028 >> v2:172.21.15.134:3300/0 conn(0x7fa114153250 0x7fa11406fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.967+0000 7fa118a00640 1 -- 172.21.15.67:0/351370028 >> 172.21.15.67:0/351370028 conn(0x7fa114066d90 msgr2=0x7fa114067170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.967+0000 7fa118a00640 1 -- 172.21.15.67:0/351370028 shutdown_connections 2024-03-09T09:05:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.967+0000 7fa118a00640 1 -- 172.21.15.67:0/351370028 wait complete. 2024-03-09T09:05:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.969+0000 7fa118a00640 1 Processor -- start 2024-03-09T09:05:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.969+0000 7fa118a00640 1 -- start start 2024-03-09T09:05:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.970+0000 7fa118a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa114073010 0x7fa11414afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.970+0000 7fa118a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1140c28a0 0x7fa114155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.970+0000 7fa118a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114153250 0x7fa1141ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa118a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11406f8f0 con 0x7fa114073010 2024-03-09T09:05:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa118a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa114156aa0 con 0x7fa114153250 2024-03-09T09:05:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa118a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa114070a60 con 0x7fa1140c28a0 2024-03-09T09:05:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa112800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa114073010 0x7fa11414afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa112800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa114073010 0x7fa11414afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37498/0 (socket says 172.21.15.67:37498) 2024-03-09T09:05:38.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa112800640 1 -- 172.21.15.67:0/3662334093 learned_addr learned my addr 172.21.15.67:0/3662334093 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa113200640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114153250 0x7fa1141ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa10be00640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1140c28a0 0x7fa114155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa112800640 1 -- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1140c28a0 msgr2=0x7fa114155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa112800640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1140c28a0 0x7fa114155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa112800640 1 -- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114153250 msgr2=0x7fa1141ef3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa112800640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114153250 0x7fa1141ef3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa112800640 1 -- 172.21.15.67:0/3662334093 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11414bec0 con 0x7fa114073010 2024-03-09T09:05:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.971+0000 7fa10be00640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1140c28a0 0x7fa114155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:05:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.972+0000 7fa113200640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114153250 0x7fa1141ef3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:05:38.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.972+0000 7fa112800640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa114073010 0x7fa11414afa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e8a0 tx=0x7fa10400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.972+0000 7fa10aa00640 1 -- 172.21.15.67:0/3662334093 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104017070 con 0x7fa114073010 2024-03-09T09:05:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.972+0000 7fa10aa00640 1 -- 172.21.15.67:0/3662334093 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10400c940 con 0x7fa114073010 2024-03-09T09:05:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.972+0000 7fa10aa00640 1 -- 172.21.15.67:0/3662334093 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10400cd30 con 0x7fa114073010 2024-03-09T09:05:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.972+0000 7fa118a00640 1 -- 172.21.15.67:0/3662334093 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1141ebad0 con 0x7fa114073010 2024-03-09T09:05:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.973+0000 7fa118a00640 1 -- 172.21.15.67:0/3662334093 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1140c8990 con 0x7fa114073010 2024-03-09T09:05:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.974+0000 7fa118a00640 1 -- 172.21.15.67:0/3662334093 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11404f3c0 con 0x7fa114073010 2024-03-09T09:05:38.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.976+0000 7fa10aa00640 1 -- 172.21.15.67:0/3662334093 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa10401a070 con 0x7fa114073010 2024-03-09T09:05:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.977+0000 7fa10aa00640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0f006b4d0 0x7fa0f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.977+0000 7fa10aa00640 1 -- 172.21.15.67:0/3662334093 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa10408abc0 con 0x7fa114073010 2024-03-09T09:05:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.978+0000 7fa10be00640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0f006b4d0 0x7fa0f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:38.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.979+0000 7fa10be00640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0f006b4d0 0x7fa0f006d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa1140af4f0 tx=0x7fa0fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:38.980+0000 7fa10aa00640 1 -- 172.21.15.67:0/3662334093 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa11414bec0 con 0x7fa114073010 2024-03-09T09:05:39.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.359+0000 7fa118a00640 1 -- 172.21.15.67:0/3662334093 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa114153860 con 0x7fa114073010 2024-03-09T09:05:39.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.360+0000 7fa10aa00640 1 -- 172.21.15.67:0/3662334093 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1564 (secure 0 0 0) 0x7fa104059730 con 0x7fa114073010 2024-03-09T09:05:39.363 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:05:39.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.362+0000 7fa118a00640 1 -- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0f006b4d0 msgr2=0x7fa0f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.362+0000 7fa118a00640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0f006b4d0 0x7fa0f006d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa1140af4f0 tx=0x7fa0fc009040 comp rx=0 tx=0).stop 2024-03-09T09:05:39.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.362+0000 7fa118a00640 1 -- 172.21.15.67:0/3662334093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa114073010 msgr2=0x7fa11414afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.362+0000 7fa118a00640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa114073010 0x7fa11414afa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa10400e8a0 tx=0x7fa10400ed60 comp rx=0 tx=0).stop 2024-03-09T09:05:39.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.363+0000 7fa118a00640 1 -- 172.21.15.67:0/3662334093 shutdown_connections 2024-03-09T09:05:39.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.363+0000 7fa118a00640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa0f006b4d0 0x7fa0f006d990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.363+0000 7fa118a00640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114153250 0x7fa1141ef3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.363+0000 7fa118a00640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1140c28a0 0x7fa114155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.363+0000 7fa118a00640 1 --2- 172.21.15.67:0/3662334093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa114073010 0x7fa11414afa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.363+0000 7fa118a00640 1 -- 172.21.15.67:0/3662334093 >> 172.21.15.67:0/3662334093 conn(0x7fa114066d90 msgr2=0x7fa114152380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.363+0000 7fa118a00640 1 -- 172.21.15.67:0/3662334093 shutdown_connections 2024-03-09T09:05:39.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.363+0000 7fa118a00640 1 -- 172.21.15.67:0/3662334093 wait complete. 2024-03-09T09:05:39.386 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:05:39.386 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:05:39.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:05:39.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.615+0000 7f0b0e800640 1 Processor -- start 2024-03-09T09:05:39.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.615+0000 7f0b0e800640 1 -- start start 2024-03-09T09:05:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.618+0000 7f0b0e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b080ad0d0 0x7f0b081533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.618+0000 7f0b0e800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b080ad560 con 0x7f0b0806ee60 2024-03-09T09:05:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.618+0000 7f0b0e800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b08152900 con 0x7f0b080c51f0 2024-03-09T09:05:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.618+0000 7f0b0e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b080ad840 con 0x7f0b080ad0d0 2024-03-09T09:05:39.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.618+0000 7f0b0d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0b0806ee60 0x7f0b080c2ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34114/0 (socket says 172.21.15.67:34114) 2024-03-09T09:05:39.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.618+0000 7f0b0d400640 1 -- 172.21.15.67:0/4290919874 learned_addr learned my addr 172.21.15.67:0/4290919874 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:05:39.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.619+0000 7f0b0ca00640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b080ad0d0 0x7f0b081533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.619+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290102508 0 0) 0x7f0b080ad560 con 0x7f0b0806ee60 2024-03-09T09:05:39.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.619+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0af0003520 con 0x7f0b0806ee60 2024-03-09T09:05:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.619+0000 7f0b0ca00640 1 -- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b080ad0d0 msgr2=0x7f0b081533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:05:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.619+0000 7f0b0ca00640 1 -- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b080ad0d0 msgr2=0x7f0b081533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:05:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.619+0000 7f0b0ca00640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b080ad0d0 0x7f0b081533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.619+0000 7f0b0ca00640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b080ad0d0 0x7f0b081533e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753291983 0 0) 0x7f0b08152900 con 0x7f0b080c51f0 2024-03-09T09:05:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b080ad560 con 0x7f0b080c51f0 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1072813818 0 0) 0x7f0af0003520 con 0x7f0b0806ee60 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b080ad0d0 msgr2=0x7f0b081533e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b080ad0d0 0x7f0b081533e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 >> v1:172.21.15.134:6789/0 conn(0x7f0b080c51f0 legacy=0x7f0b0814f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b08152be0 con 0x7f0b0806ee60 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f0afc001670 con 0x7f0b0806ee60 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 >> v1:172.21.15.67:6789/0 conn(0x7f0b0806ee60 legacy=0x7f0b080c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af0003be0 0x7f0af0004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0af0004800 0x7f0af0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0af0005180 0x7f0af000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b080ad840 con 0x7f0af0003be0 2024-03-09T09:05:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af000c030 con 0x7f0af0005180 2024-03-09T09:05:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af000c230 con 0x7f0af0004800 2024-03-09T09:05:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.620+0000 7f0b0d400640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af0003be0 0x7f0af0004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.621+0000 7f0b0de00640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0af0005180 0x7f0af000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.621+0000 7f0b0ca00640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0af0004800 0x7f0af0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.621+0000 7f0b0d400640 1 -- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0af0004800 msgr2=0x7f0af0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:39.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.621+0000 7f0b0d400640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0af0004800 0x7f0af0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.621+0000 7f0b0d400640 1 -- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0af0005180 msgr2=0x7f0af000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:39.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.621+0000 7f0b0d400640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0af0005180 0x7f0af000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.621+0000 7f0b0d400640 1 -- 172.21.15.67:0/4290919874 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af0003520 con 0x7f0af0003be0 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.624+0000 7f0b0d400640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af0003be0 0x7f0af0004000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0afc007fa0 tx=0x7f0af0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d87298e935d2ac server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.624+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc007bc0 con 0x7f0af0003be0 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.624+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc030070 con 0x7f0af0003be0 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.624+0000 7f0b07e00640 1 -- 172.21.15.67:0/4290919874 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc005d30 con 0x7f0af0003be0 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.625+0000 7f0b0e800640 1 -- 172.21.15.67:0/4290919874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af0003be0 msgr2=0x7f0af0004000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.625+0000 7f0b0e800640 1 --2- 172.21.15.67:0/4290919874 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af0003be0 0x7f0af0004000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0afc007fa0 tx=0x7f0af0005830 comp rx=0 tx=0).stop 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.625+0000 7f0b0de00640 1 -- 172.21.15.67:0/4290919874 reap_dead start 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.625+0000 7f0b0e800640 1 -- 172.21.15.67:0/4290919874 shutdown_connections 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.625+0000 7f0b0e800640 1 -- 172.21.15.67:0/4290919874 >> 172.21.15.67:0/4290919874 conn(0x7f0b0806c150 msgr2=0x7f0b0806c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.625+0000 7f0b0e800640 1 -- 172.21.15.67:0/4290919874 shutdown_connections 2024-03-09T09:05:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.625+0000 7f0b0e800640 1 -- 172.21.15.67:0/4290919874 wait complete. 2024-03-09T09:05:39.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.627+0000 7f0b0e800640 1 Processor -- start 2024-03-09T09:05:39.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.627+0000 7f0b0e800640 1 -- start start 2024-03-09T09:05:39.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.628+0000 7f0b0e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af0003be0 0x7f0b081ea850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.628+0000 7f0b0e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b08149d80 0x7f0b08151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.628+0000 7f0b0e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b081524a0 0x7f0b0814e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.628+0000 7f0b0e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0806d4a0 con 0x7f0af0003be0 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.628+0000 7f0b0e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b080c33b0 con 0x7f0b081524a0 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.628+0000 7f0b0e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0806f240 con 0x7f0b08149d80 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.628+0000 7f0b0d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af0003be0 0x7f0b081ea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af0003be0 0x7f0b081ea850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37518/0 (socket says 172.21.15.67:37518) 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0d400640 1 -- 172.21.15.67:0/1899277694 learned_addr learned my addr 172.21.15.67:0/1899277694 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0ca00640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b08149d80 0x7f0b08151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0de00640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b081524a0 0x7f0b0814e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0ca00640 1 -- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b08149d80 msgr2=0x7f0b08151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0ca00640 1 -- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b08149d80 msgr2=0x7f0b08151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0ca00640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b08149d80 0x7f0b08151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:39.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0ca00640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b08149d80 0x7f0b08151f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:39.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0d400640 1 -- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b08149d80 msgr2=0x7f0b08151f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:05:39.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0d400640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b08149d80 0x7f0b08151f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0d400640 1 -- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b081524a0 msgr2=0x7f0b0814e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0d400640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b081524a0 0x7f0b0814e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.629+0000 7f0b0d400640 1 -- 172.21.15.67:0/1899277694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b081db180 con 0x7f0af0003be0 2024-03-09T09:05:39.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.630+0000 7f0b0d400640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af0003be0 0x7f0b081ea850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0afc02c400 tx=0x7f0b08152db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:39.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.630+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc045070 con 0x7f0af0003be0 2024-03-09T09:05:39.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.630+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc02c990 con 0x7f0af0003be0 2024-03-09T09:05:39.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.630+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc02cd60 con 0x7f0af0003be0 2024-03-09T09:05:39.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.630+0000 7f0b0e800640 1 -- 172.21.15.67:0/1899277694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0814f830 con 0x7f0af0003be0 2024-03-09T09:05:39.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.630+0000 7f0b0e800640 1 -- 172.21.15.67:0/1899277694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0814fdf0 con 0x7f0af0003be0 2024-03-09T09:05:39.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.631+0000 7f0b0e800640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b081db350 0x7f0b081db960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.631+0000 7f0b0e800640 1 -- 172.21.15.67:0/1899277694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0814cbc0 con 0x7f0b081db350 2024-03-09T09:05:39.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.632+0000 7f0b0ca00640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b081db350 0x7f0b081db960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.632+0000 7f0b0ca00640 1 -- 172.21.15.67:0/1899277694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b08073650 con 0x7f0b081db350 2024-03-09T09:05:39.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.633+0000 7f0b0ca00640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b081db350 0x7f0b081db960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006fd0 tx=0x7f0af4012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:39.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.633+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af401c070 con 0x7f0b081db350 2024-03-09T09:05:39.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.633+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0afc048070 con 0x7f0af0003be0 2024-03-09T09:05:39.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.634+0000 7f0b05600640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ae806b3a0 0x7f0ae806d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.634+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0afc0ab6e0 con 0x7f0af0003be0 2024-03-09T09:05:39.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.634+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f0af4010ab0 con 0x7f0b081db350 2024-03-09T09:05:39.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.634+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b081db350 msgr2=0x7f0b081db960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.634+0000 7f0b05600640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b081db350 0x7f0b081db960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006fd0 tx=0x7f0af4012d00 comp rx=0 tx=0).stop 2024-03-09T09:05:39.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.635+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b081db350 msgr2=0x7f0b081db960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:39.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.635+0000 7f0b05600640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b081db350 0x7f0b081db960 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006fd0 tx=0x7f0af4012d00 comp rx=0 tx=0).stop 2024-03-09T09:05:39.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.635+0000 7f0b0de00640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ae806b3a0 0x7f0ae806d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.636+0000 7f0b0de00640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ae806b3a0 0x7f0ae806d860 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0b080c5e60 tx=0x7f0b00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:39.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.642+0000 7f0b0e800640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0814e560 0x7f0b081dbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.642+0000 7f0b0e800640 1 -- 172.21.15.67:0/1899277694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0804f3c0 con 0x7f0b0814e560 2024-03-09T09:05:39.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.642+0000 7f0b0ca00640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0814e560 0x7f0b081dbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.643+0000 7f0b0ca00640 1 -- 172.21.15.67:0/1899277694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0b080c4570 con 0x7f0b0814e560 2024-03-09T09:05:39.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.643+0000 7f0b0ca00640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0814e560 0x7f0b081dbea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006fd0 tx=0x7f0af40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:39.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.643+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af402f020 con 0x7f0b0814e560 2024-03-09T09:05:39.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f0af4015650 con 0x7f0b0814e560 2024-03-09T09:05:39.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0814e560 msgr2=0x7f0b081dbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b05600640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0814e560 0x7f0b081dbea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006fd0 tx=0x7f0af40046c0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b05600640 1 -- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0814e560 msgr2=0x7f0b081dbea0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:39.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b05600640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b0814e560 0x7f0b081dbea0 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0af4006fd0 tx=0x7f0af40046c0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.647 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:05:39.648 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:05:39.648 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:05:39.648 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:05:39.648 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 98, 2024-03-09T09:05:39.648 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:05:39.648 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:05:39.648 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:39.649 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:39.650 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:05:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:05:39.651 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:05:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:05:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:05:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:05:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:05:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:05:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:05:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:05:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:05:39.652 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:39.653 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:05:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:05:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:05:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:05:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.654 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:05:39.655 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.656 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:05:39.657 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:05:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:05:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:05:39.658 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:05:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:05:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:05:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:05:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:39.660 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:05:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:39.661 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:05:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:05:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:05:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:05:39.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:39.663 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.664 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:05:39.664 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:05:39.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b0e800640 1 -- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ae806b3a0 msgr2=0x7f0ae806d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b0e800640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ae806b3a0 0x7f0ae806d860 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0b080c5e60 tx=0x7f0b00009040 comp rx=0 tx=0).stop 2024-03-09T09:05:39.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b0e800640 1 -- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af0003be0 msgr2=0x7f0b081ea850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b0e800640 1 --2- 172.21.15.67:0/1899277694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0af0003be0 0x7f0b081ea850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0afc02c400 tx=0x7f0b08152db0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b0de00640 1 -- 172.21.15.67:0/1899277694 reap_dead start 2024-03-09T09:05:39.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b0de00640 1 -- 172.21.15.67:0/1899277694 reap_dead start 2024-03-09T09:05:39.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b0e800640 1 -- 172.21.15.67:0/1899277694 shutdown_connections 2024-03-09T09:05:39.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.644+0000 7f0b0e800640 1 -- 172.21.15.67:0/1899277694 >> 172.21.15.67:0/1899277694 conn(0x7f0b0806c150 msgr2=0x7f0b0814eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:39.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.645+0000 7f0b0e800640 1 -- 172.21.15.67:0/1899277694 shutdown_connections 2024-03-09T09:05:39.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.645+0000 7f0b0e800640 1 -- 172.21.15.67:0/1899277694 wait complete. 2024-03-09T09:05:39.666 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:05:39.667 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:05:39.667 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:05:39.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.902+0000 7f62b0a00640 1 Processor -- start 2024-03-09T09:05:39.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.902+0000 7f62b0a00640 1 -- start start 2024-03-09T09:05:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.904+0000 7f62b0a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 0x7f62ac0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.904+0000 7f62b0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f62ac0ad440 0x7f62ac06c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.904+0000 7f62b0a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ac0c9890 con 0x7f62ac14fd90 2024-03-09T09:05:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.904+0000 7f62b0a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ac06ba60 con 0x7f62ac0ad440 2024-03-09T09:05:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.904+0000 7f62b0a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0c5f80 con 0x7f62ac0c9380 2024-03-09T09:05:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.904+0000 7f62aa800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f62ac14fd90 0x7f62ac156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34124/0 (socket says 172.21.15.67:34124) 2024-03-09T09:05:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.904+0000 7f62aa800640 1 -- 172.21.15.67:0/3204890448 learned_addr learned my addr 172.21.15.67:0/3204890448 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:05:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.904+0000 7f62ab200640 1 --2- 172.21.15.67:0/3204890448 >> v2:172.21.15.67:3300/0 conn(0x7f62ac0ad440 0x7f62ac06c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.904+0000 7f62a9e00640 1 --2- 172.21.15.67:0/3204890448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 0x7f62ac0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.905+0000 7f62a9400640 1 -- 172.21.15.67:0/3204890448 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553855698 0 0) 0x7f62ac0c9890 con 0x7f62ac14fd90 2024-03-09T09:05:39.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.905+0000 7f62a9400640 1 -- 172.21.15.67:0/3204890448 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6290003520 con 0x7f62ac14fd90 2024-03-09T09:05:39.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.905+0000 7f62a9e00640 1 -- 172.21.15.67:0/3204890448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 msgr2=0x7f62ac0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:05:39.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.905+0000 7f62a9e00640 1 -- 172.21.15.67:0/3204890448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 msgr2=0x7f62ac0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:05:39.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.905+0000 7f62a9e00640 1 --2- 172.21.15.67:0/3204890448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 0x7f62ac0acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.905+0000 7f62ab200640 1 -- 172.21.15.67:0/3204890448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 msgr2=0x7f62ac0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.905+0000 7f62a9e00640 1 --2- 172.21.15.67:0/3204890448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 0x7f62ac0acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.905+0000 7f62ab200640 1 --2- 172.21.15.67:0/3204890448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 0x7f62ac0acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.905+0000 7f62ab200640 1 -- 172.21.15.67:0/3204890448 >> v1:172.21.15.67:6789/0 conn(0x7f62ac14fd90 legacy=0x7f62ac156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.905+0000 7f62ab200640 1 -- 172.21.15.67:0/3204890448 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac0c6240 con 0x7f62ac0ad440 2024-03-09T09:05:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.906+0000 7f62ab200640 1 --2- 172.21.15.67:0/3204890448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ac0ad440 0x7f62ac06c1d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f62ac06bbe0 tx=0x7f62a402bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d8122e89c67ce2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.906+0000 7f62a9400640 1 -- 172.21.15.67:0/3204890448 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4039070 con 0x7f62ac0ad440 2024-03-09T09:05:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.906+0000 7f62a9400640 1 -- 172.21.15.67:0/3204890448 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a402cb20 con 0x7f62ac0ad440 2024-03-09T09:05:39.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.906+0000 7f62a9400640 1 -- 172.21.15.67:0/3204890448 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4034440 con 0x7f62ac0ad440 2024-03-09T09:05:39.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.907+0000 7f62b0a00640 1 -- 172.21.15.67:0/3204890448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ac0ad440 msgr2=0x7f62ac06c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.907+0000 7f62b0a00640 1 --2- 172.21.15.67:0/3204890448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ac0ad440 0x7f62ac06c1d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f62ac06bbe0 tx=0x7f62a402bd70 comp rx=0 tx=0).stop 2024-03-09T09:05:39.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.907+0000 7f62b0a00640 1 -- 172.21.15.67:0/3204890448 shutdown_connections 2024-03-09T09:05:39.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.907+0000 7f62b0a00640 1 --2- 172.21.15.67:0/3204890448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ac0ad440 0x7f62ac06c1d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.907+0000 7f62b0a00640 1 --2- 172.21.15.67:0/3204890448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 0x7f62ac0acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.907+0000 7f62b0a00640 1 -- 172.21.15.67:0/3204890448 >> 172.21.15.67:0/3204890448 conn(0x7f62ac0cb480 msgr2=0x7f62ac0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:39.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.907+0000 7f62b0a00640 1 -- 172.21.15.67:0/3204890448 shutdown_connections 2024-03-09T09:05:39.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.907+0000 7f62b0a00640 1 -- 172.21.15.67:0/3204890448 wait complete. 2024-03-09T09:05:39.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.910+0000 7f62b0a00640 1 Processor -- start 2024-03-09T09:05:39.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.910+0000 7f62b0a00640 1 -- start start 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.911+0000 7f62b0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ac0ad440 0x7f62ac159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.911+0000 7f62b0a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 0x7f62ac149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.911+0000 7f62b0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac14fd90 0x7f62ac149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.911+0000 7f62b0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac06c710 con 0x7f62ac0ad440 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.911+0000 7f62b0a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac06bcb0 con 0x7f62ac14fd90 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.911+0000 7f62b0a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac06c890 con 0x7f62ac0c9380 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.912+0000 7f62aa800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ac0ad440 0x7f62ac159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.912+0000 7f62aa800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ac0ad440 0x7f62ac159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37546/0 (socket says 172.21.15.67:37546) 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.912+0000 7f62aa800640 1 -- 172.21.15.67:0/58395344 learned_addr learned my addr 172.21.15.67:0/58395344 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.912+0000 7f62a9e00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 0x7f62ac149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.912+0000 7f62aa800640 1 -- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 msgr2=0x7f62ac149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.912+0000 7f62aa800640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62ac0c9380 0x7f62ac149220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.912+0000 7f62aa800640 1 -- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac14fd90 msgr2=0x7f62ac149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.912+0000 7f62aa800640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac14fd90 0x7f62ac149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.912+0000 7f62aa800640 1 -- 172.21.15.67:0/58395344 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac1ec620 con 0x7f62ac0ad440 2024-03-09T09:05:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.912+0000 7f62aa800640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ac0ad440 0x7f62ac159830 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c390 tx=0x7f629c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.913+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c017070 con 0x7f62ac0ad440 2024-03-09T09:05:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.913+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c005ce0 con 0x7f62ac0ad440 2024-03-09T09:05:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.913+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c0072f0 con 0x7f62ac0ad440 2024-03-09T09:05:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.913+0000 7f62b0a00640 1 -- 172.21.15.67:0/58395344 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ac149ca0 con 0x7f62ac0ad440 2024-03-09T09:05:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.913+0000 7f62b0a00640 1 -- 172.21.15.67:0/58395344 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac157400 con 0x7f62ac0ad440 2024-03-09T09:05:39.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.914+0000 7f62b0a00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac157600 0x7f62ac157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.914+0000 7f62b0a00640 1 -- 172.21.15.67:0/58395344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac04f3c0 con 0x7f62ac157600 2024-03-09T09:05:39.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.915+0000 7f62a9e00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac157600 0x7f62ac157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.915+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f629c01a070 con 0x7f62ac0ad440 2024-03-09T09:05:39.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.916+0000 7f62a2a00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f628c06b4d0 0x7f628c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.916+0000 7f62a9e00640 1 -- 172.21.15.67:0/58395344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62ac06d2d0 con 0x7f62ac157600 2024-03-09T09:05:39.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.916+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f629c089a20 con 0x7f62ac0ad440 2024-03-09T09:05:39.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.916+0000 7f62ab200640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f628c06b4d0 0x7f628c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.917+0000 7f62a9e00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac157600 0x7f62ac157ec0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0c18d0 tx=0x7f629800eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:39.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.917+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298017070 con 0x7f62ac157600 2024-03-09T09:05:39.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.918+0000 7f62ab200640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f628c06b4d0 0x7f628c06d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f62a4036000 tx=0x7f62a402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.918+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f6298004020 con 0x7f62ac157600 2024-03-09T09:05:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.918+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac157600 msgr2=0x7f62ac157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.918+0000 7f62a2a00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac157600 0x7f62ac157ec0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0c18d0 tx=0x7f629800eba0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.918+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac157600 msgr2=0x7f62ac157ec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:39.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.918+0000 7f62a2a00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac157600 0x7f62ac157ec0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0c18d0 tx=0x7f629800eba0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.925+0000 7f62b0a00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac1ec120 0x7f62ac158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:39.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.925+0000 7f62b0a00640 1 -- 172.21.15.67:0/58395344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0c6b20 con 0x7f62ac1ec120 2024-03-09T09:05:39.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.926+0000 7f62a9e00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac1ec120 0x7f62ac158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:39.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.927+0000 7f62a9e00640 1 -- 172.21.15.67:0/58395344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f62ac159d70 con 0x7f62ac1ec120 2024-03-09T09:05:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.927+0000 7f62a9e00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac1ec120 0x7f62ac158400 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6298016000 tx=0x7f62980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:39.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.928+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298029020 con 0x7f62ac1ec120 2024-03-09T09:05:39.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.928+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5016 (secure 0 0 0) 0x7f6298003a20 con 0x7f62ac1ec120 2024-03-09T09:05:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.928+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac1ec120 msgr2=0x7f62ac158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.928+0000 7f62a2a00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac1ec120 0x7f62ac158400 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6298016000 tx=0x7f62980046c0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.928+0000 7f62a2a00640 1 -- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac1ec120 msgr2=0x7f62ac158400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.928+0000 7f62a2a00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62ac1ec120 0x7f62ac158400 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6298016000 tx=0x7f62980046c0 comp rx=0 tx=0).stop 2024-03-09T09:05:39.932 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 98, 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:05:39.933 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:39.934 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:39.935 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:05:39.936 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:39.937 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.938 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:39.939 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:39.942 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:05:39.943 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:05:39.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:39.945 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:39.946 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:39.946 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:39.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:39.946 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:39.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:39.946 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:05:39.946 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:05:39.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.929+0000 7f62b0a00640 1 -- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f628c06b4d0 msgr2=0x7f628c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.929+0000 7f62b0a00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f628c06b4d0 0x7f628c06d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f62a4036000 tx=0x7f62a402e000 comp rx=0 tx=0).stop 2024-03-09T09:05:39.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.929+0000 7f62b0a00640 1 -- 172.21.15.67:0/58395344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ac0ad440 msgr2=0x7f62ac159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:39.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.929+0000 7f62b0a00640 1 --2- 172.21.15.67:0/58395344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62ac0ad440 0x7f62ac159830 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c390 tx=0x7f629c00c850 comp rx=0 tx=0).stop 2024-03-09T09:05:39.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.929+0000 7f62ab200640 1 -- 172.21.15.67:0/58395344 reap_dead start 2024-03-09T09:05:39.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.929+0000 7f62ab200640 1 -- 172.21.15.67:0/58395344 reap_dead start 2024-03-09T09:05:39.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.929+0000 7f62b0a00640 1 -- 172.21.15.67:0/58395344 shutdown_connections 2024-03-09T09:05:39.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.929+0000 7f62b0a00640 1 -- 172.21.15.67:0/58395344 >> 172.21.15.67:0/58395344 conn(0x7f62ac0cb480 msgr2=0x7f62ac0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:39.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.929+0000 7f62b0a00640 1 -- 172.21.15.67:0/58395344 shutdown_connections 2024-03-09T09:05:39.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:39.929+0000 7f62b0a00640 1 -- 172.21.15.67:0/58395344 wait complete. 2024-03-09T09:05:39.949 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:05:39.949 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:05:39.949 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:05:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.147+0000 7fa2c3e00640 1 Processor -- start 2024-03-09T09:05:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.147+0000 7fa2c3e00640 1 -- start start 2024-03-09T09:05:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.148+0000 7fa2c3e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c406b040 0x7fa2c40c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.149+0000 7fa2c3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa2c40717f0 0x7fa2c406b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.149+0000 7fa2c3e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa2c4159240 0x7fa2c41596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.149+0000 7fa2c3e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c4071d40 con 0x7fa2c40717f0 2024-03-09T09:05:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.149+0000 7fa2c3e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c4072ba0 con 0x7fa2c4159240 2024-03-09T09:05:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.149+0000 7fa2c3e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c414ff60 con 0x7fa2c406b040 2024-03-09T09:05:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.149+0000 7fa2c2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa2c40717f0 0x7fa2c406b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.149+0000 7fa2c2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa2c40717f0 0x7fa2c406b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37548/0 (socket says 172.21.15.67:37548) 2024-03-09T09:05:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.149+0000 7fa2c2000640 1 -- 172.21.15.67:0/3256961365 learned_addr learned my addr 172.21.15.67:0/3256961365 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.149+0000 7fa2c3400640 1 --2- 172.21.15.67:0/3256961365 >> v2:172.21.15.134:3300/0 conn(0x7fa2c4159240 0x7fa2c41596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.149+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/3256961365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c406b040 0x7fa2c40c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.150+0000 7fa2c2000640 1 -- 172.21.15.67:0/3256961365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c406b040 msgr2=0x7fa2c40c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.150+0000 7fa2c2000640 1 --2- 172.21.15.67:0/3256961365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c406b040 0x7fa2c40c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:40.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.150+0000 7fa2c2000640 1 -- 172.21.15.67:0/3256961365 >> v2:172.21.15.134:3300/0 conn(0x7fa2c4159240 msgr2=0x7fa2c41596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.150+0000 7fa2c2000640 1 --2- 172.21.15.67:0/3256961365 >> v2:172.21.15.134:3300/0 conn(0x7fa2c4159240 0x7fa2c41596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.150+0000 7fa2c2000640 1 -- 172.21.15.67:0/3256961365 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c4071f90 con 0x7fa2c40717f0 2024-03-09T09:05:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.150+0000 7fa2c3400640 1 --2- 172.21.15.67:0/3256961365 >> v2:172.21.15.134:3300/0 conn(0x7fa2c4159240 0x7fa2c41596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:05:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.152+0000 7fa2c2000640 1 --2- 172.21.15.67:0/3256961365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c40717f0 0x7fa2c406b420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4071780 tx=0x7fa2ac02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3bd7e2dcc5f148 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.152+0000 7fa2c1600640 1 -- 172.21.15.67:0/3256961365 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac02e070 con 0x7fa2c40717f0 2024-03-09T09:05:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.152+0000 7fa2c1600640 1 -- 172.21.15.67:0/3256961365 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ac037070 con 0x7fa2c40717f0 2024-03-09T09:05:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.152+0000 7fa2c1600640 1 -- 172.21.15.67:0/3256961365 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac031a00 con 0x7fa2c40717f0 2024-03-09T09:05:40.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.153+0000 7fa2c3e00640 1 -- 172.21.15.67:0/3256961365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c40717f0 msgr2=0x7fa2c406b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:40.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.153+0000 7fa2c3e00640 1 --2- 172.21.15.67:0/3256961365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c40717f0 0x7fa2c406b420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4071780 tx=0x7fa2ac02b880 comp rx=0 tx=0).stop 2024-03-09T09:05:40.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.153+0000 7fa2c3e00640 1 -- 172.21.15.67:0/3256961365 shutdown_connections 2024-03-09T09:05:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.153+0000 7fa2c3e00640 1 --2- 172.21.15.67:0/3256961365 >> v2:172.21.15.134:3300/0 conn(0x7fa2c4159240 0x7fa2c41596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.153+0000 7fa2c3e00640 1 --2- 172.21.15.67:0/3256961365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c40717f0 0x7fa2c406b420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.153+0000 7fa2c3e00640 1 --2- 172.21.15.67:0/3256961365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c406b040 0x7fa2c40c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.153+0000 7fa2c3e00640 1 -- 172.21.15.67:0/3256961365 >> 172.21.15.67:0/3256961365 conn(0x7fa2c40c8480 msgr2=0x7fa2c40c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.153+0000 7fa2c3e00640 1 -- 172.21.15.67:0/3256961365 shutdown_connections 2024-03-09T09:05:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.153+0000 7fa2c3e00640 1 -- 172.21.15.67:0/3256961365 wait complete. 2024-03-09T09:05:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.155+0000 7fa2c3e00640 1 Processor -- start 2024-03-09T09:05:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.156+0000 7fa2c3e00640 1 -- start start 2024-03-09T09:05:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.157+0000 7fa2c3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c406b040 0x7fa2c40c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.157+0000 7fa2c3e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c40717f0 0x7fa2c41ebb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.157+0000 7fa2c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c4159240 0x7fa2c4155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.157+0000 7fa2c3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4072830 con 0x7fa2c4159240 2024-03-09T09:05:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.157+0000 7fa2c3e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4072550 con 0x7fa2c406b040 2024-03-09T09:05:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.157+0000 7fa2c3e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4159c10 con 0x7fa2c40717f0 2024-03-09T09:05:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.157+0000 7fa2c3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c4159240 0x7fa2c4155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.157+0000 7fa2c3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c4159240 0x7fa2c4155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37562/0 (socket says 172.21.15.67:37562) 2024-03-09T09:05:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.157+0000 7fa2c3400640 1 -- 172.21.15.67:0/667404107 learned_addr learned my addr 172.21.15.67:0/667404107 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.157+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c406b040 0x7fa2c40c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.157+0000 7fa2c2000640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c40717f0 0x7fa2c41ebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.158+0000 7fa2c3400640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c40717f0 msgr2=0x7fa2c41ebb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.158+0000 7fa2c3400640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c40717f0 0x7fa2c41ebb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:40.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.158+0000 7fa2c3400640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c406b040 msgr2=0x7fa2c40c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:40.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.158+0000 7fa2c3400640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c406b040 0x7fa2c40c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:40.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.158+0000 7fa2c3400640 1 -- 172.21.15.67:0/667404107 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c41ec3b0 con 0x7fa2c4159240 2024-03-09T09:05:40.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.158+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2c406b040 0x7fa2c40c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:05:40.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.158+0000 7fa2c3400640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c4159240 0x7fa2c4155bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc007df0 tx=0x7fa2bc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:40.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.158+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc01a070 con 0x7fa2c4159240 2024-03-09T09:05:40.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.158+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc003990 con 0x7fa2c4159240 2024-03-09T09:05:40.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.158+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc003d80 con 0x7fa2c4159240 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.158+0000 7fa2c3e00640 1 -- 172.21.15.67:0/667404107 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c41ec690 con 0x7fa2c4159240 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.159+0000 7fa2c3e00640 1 -- 172.21.15.67:0/667404107 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c41e0ad0 con 0x7fa2c4159240 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.159+0000 7fa2c3e00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.159+0000 7fa2c3e00640 1 -- 172.21.15.67:0/667404107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41e0d10 con 0x7fa2c41ef0d0 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.160+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.160+0000 7fa2c2a00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 msgr2=0x7fa2c4156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.160+0000 7fa2c2a00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 msgr2=0x7fa2c4156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.160+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.160+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.161+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa2bc01d070 con 0x7fa2c4159240 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.162+0000 7fa2baa00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2a006b4d0 0x7fa2a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.162+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa2bc08a7e0 con 0x7fa2c4159240 2024-03-09T09:05:40.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.162+0000 7fa2c2000640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2a006b4d0 0x7fa2a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:40.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.163+0000 7fa2c2000640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2a006b4d0 0x7fa2a006d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac036000 tx=0x7fa2ac03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:40.333 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 37% (75/201) Updating files: 38% (77/201) Updating files: 39% (79/201) Updating files: 40% (81/201) Updating files: 41% (83/201) Updating files: 42% (85/201) Updating files: 43% (87/201) Updating files: 44% (89/201) Updating files: 45% (91/201) Updating files: 46% (93/201) Updating files: 47% (95/201) Updating files: 48% (97/201) Updating files: 49% (99/201) Updating files: 50% (101/201) Updating files: 51% (103/201) Updating files: 52% (105/201) Updating files: 53% (107/201) Updating files: 54% (109/201) Updating files: 55% (111/201) Updating files: 56% (113/201) Updating files: 57% (115/201) Updating files: 58% (117/201) Updating files: 59% (119/201) Updating files: 60% (121/201) Updating files: 61% (123/201) Updating files: 62% (125/201) Updating files: 63% (127/201) Updating files: 64% (129/201) Updating files: 65% (131/201) Updating files: 66% (133/201) Updating files: 67% (135/201) Updating files: 68% (137/201) Updating files: 69% (139/201) Updating files: 70% (141/201) Updating files: 71% (143/201) Updating files: 72% (145/201) Updating files: 73% (147/201) Updating files: 74% (149/201) Updating files: 75% (151/201) Updating files: 76% (153/201) Updating files: 77% (155/201) Updating files: 78% (157/201) Updating files: 79% (159/201) Updating files: 80% (161/201) Updating files: 81% (163/201) Updating files: 82% (165/201) Updating files: 83% (167/201) Updating files: 84% (169/201) Updating files: 85% (171/201) Updating files: 86% (173/201) Updating files: 86% (174/201) Updating files: 87% (175/201) Updating files: 88% (177/201) Updating files: 89% (179/201) Updating files: 90% (181/201) Updating files: 91% (183/201) Updating files: 92% (185/201) Updating files: 93% (187/201) Updating files: 94% (189/201) Updating files: 95% (191/201) Updating files: 96% (193/201) Updating files: 97% (195/201) Updating files: 98% (197/201) Updating files: 99% (199/201) Updating files: 100% (201/201) Updating files: 100% (201/201), done. 2024-03-09T09:05:40.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.361+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:40.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.362+0000 7fa2c2a00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 msgr2=0x7fa2c4156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:05:40.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.362+0000 7fa2c2a00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 msgr2=0x7fa2c4156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:05:40.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.362+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:40.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.362+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:05:40.386 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at c46e1b320c6 mkcephfs: btrfsctl -a between mkfs and mount 2024-03-09T09:05:40.400 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.7 2024-03-09T09:05:40.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.763+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:40.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.764+0000 7fa2c2a00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 msgr2=0x7fa2c4156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:05:40.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.764+0000 7fa2c2a00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 msgr2=0x7fa2c4156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:05:40.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.764+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:40.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:40.764+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:05:40.942 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:05:40.942 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 8 -eq 48 ']' 2024-03-09T09:05:40.942 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.8 2024-03-09T09:05:40.942 INFO:tasks.workunit.client.0.smithi067.stdout:v0.8 2024-03-09T09:05:40.942 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.8 2024-03-09T09:05:40.943 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.8 2024-03-09T09:05:41.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:41.565+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:41.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:41.566+0000 7fa2c2a00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 msgr2=0x7fa2c4156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:05:41.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:41.566+0000 7fa2c2a00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 msgr2=0x7fa2c4156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:05:41.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:41.566+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:05:41.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:41.566+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:05:43.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.168+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:43.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.170+0000 7fa2c2a00640 1 -- 172.21.15.67:0/667404107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2c4156670 con 0x7fa2c41ef0d0 2024-03-09T09:05:43.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.170+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4013a30 tx=0x7fa2b4013b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:43.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.171+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4022020 con 0x7fa2c41ef0d0 2024-03-09T09:05:43.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.172+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa2b4015020 con 0x7fa2c41ef0d0 2024-03-09T09:05:43.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.172+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 msgr2=0x7fa2c4156130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:43.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.172+0000 7fa2baa00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4013a30 tx=0x7fa2b4013b20 comp rx=0 tx=0).stop 2024-03-09T09:05:43.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.172+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 msgr2=0x7fa2c4156130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:43.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.172+0000 7fa2baa00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41ef0d0 0x7fa2c4156130 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4013a30 tx=0x7fa2b4013b20 comp rx=0 tx=0).stop 2024-03-09T09:05:43.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.181+0000 7fa2c3e00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41e0ff0 0x7fa2c41e1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:43.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.181+0000 7fa2c3e00640 1 -- 172.21.15.67:0/667404107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4156a40 con 0x7fa2c41e0ff0 2024-03-09T09:05:43.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.182+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41e0ff0 0x7fa2c41e1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:43.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.183+0000 7fa2c2a00640 1 -- 172.21.15.67:0/667404107 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa2c41534c0 con 0x7fa2c41e0ff0 2024-03-09T09:05:43.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.184+0000 7fa2c2a00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41e0ff0 0x7fa2c41e1400 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4013a30 tx=0x7fa2b40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:43.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.185+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4022030 con 0x7fa2c41e0ff0 2024-03-09T09:05:43.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.186+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4533 (secure 0 0 0) 0x7fa2b4014cf0 con 0x7fa2c41e0ff0 2024-03-09T09:05:43.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.186+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41e0ff0 msgr2=0x7fa2c41e1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:43.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.186+0000 7fa2baa00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41e0ff0 0x7fa2c41e1400 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4013a30 tx=0x7fa2b40046c0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.186+0000 7fa2baa00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41e0ff0 msgr2=0x7fa2c41e1400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:43.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.186+0000 7fa2baa00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2c41e0ff0 0x7fa2c41e1400 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4013a30 tx=0x7fa2b40046c0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.189 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 98, 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 6, 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:43.190 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:43.191 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:43.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:05:43.193 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:43.194 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:43.195 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:43.196 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:43.197 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:05:43.198 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:43.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:43.200 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:43.201 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:43.201 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:05:43.201 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:43.201 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:43.201 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:43.201 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:43.201 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:43.201 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:43.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:43.201 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:05:43.201 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:05:43.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.187+0000 7fa2c3e00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2a006b4d0 msgr2=0x7fa2a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:43.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.187+0000 7fa2c3e00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2a006b4d0 0x7fa2a006d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac036000 tx=0x7fa2ac03d040 comp rx=0 tx=0).stop 2024-03-09T09:05:43.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.187+0000 7fa2c3e00640 1 -- 172.21.15.67:0/667404107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c4159240 msgr2=0x7fa2c4155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:43.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.187+0000 7fa2c3e00640 1 --2- 172.21.15.67:0/667404107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2c4159240 0x7fa2c4155bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc007df0 tx=0x7fa2bc00d4a0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.187+0000 7fa2c3400640 1 -- 172.21.15.67:0/667404107 reap_dead start 2024-03-09T09:05:43.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.187+0000 7fa2c3400640 1 -- 172.21.15.67:0/667404107 reap_dead start 2024-03-09T09:05:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.187+0000 7fa2c3e00640 1 -- 172.21.15.67:0/667404107 shutdown_connections 2024-03-09T09:05:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.187+0000 7fa2c3e00640 1 -- 172.21.15.67:0/667404107 >> 172.21.15.67:0/667404107 conn(0x7fa2c40c8480 msgr2=0x7fa2c40c55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.187+0000 7fa2c3e00640 1 -- 172.21.15.67:0/667404107 shutdown_connections 2024-03-09T09:05:43.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.187+0000 7fa2c3e00640 1 -- 172.21.15.67:0/667404107 wait complete. 2024-03-09T09:05:43.217 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:05:43.218 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:05:43.218 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:05:43.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.451+0000 7fe9a1600640 1 Processor -- start 2024-03-09T09:05:43.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.451+0000 7fe9a1600640 1 -- start start 2024-03-09T09:05:43.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.453+0000 7fe9a1600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe99c0c30f0 0x7fe99c152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:43.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.453+0000 7fe9a1600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe99c06dbe0 0x7fe99c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:43.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.453+0000 7fe9a1600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99c0ab3c0 con 0x7fe99c0aa420 2024-03-09T09:05:43.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.453+0000 7fe9a1600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99c0c82d0 con 0x7fe99c06dbe0 2024-03-09T09:05:43.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.453+0000 7fe9a1600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99c0aa8a0 con 0x7fe99c0c30f0 2024-03-09T09:05:43.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.453+0000 7fe99b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe99c06dbe0 0x7fe99c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:43.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.454+0000 7fe99b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe99c06dbe0 0x7fe99c06e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37572/0 (socket says 172.21.15.67:37572) 2024-03-09T09:05:43.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.454+0000 7fe99b400640 1 -- 172.21.15.67:0/522301725 learned_addr learned my addr 172.21.15.67:0/522301725 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:43.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.454+0000 7fe99aa00640 1 --2- 172.21.15.67:0/522301725 >> v2:172.21.15.134:3300/0 conn(0x7fe99c0c30f0 0x7fe99c152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:43.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.455+0000 7fe99b400640 1 -- 172.21.15.67:0/522301725 >> v2:172.21.15.134:3300/0 conn(0x7fe99c0c30f0 msgr2=0x7fe99c152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:43.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.455+0000 7fe99b400640 1 --2- 172.21.15.67:0/522301725 >> v2:172.21.15.134:3300/0 conn(0x7fe99c0c30f0 0x7fe99c152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.455+0000 7fe99b400640 1 -- 172.21.15.67:0/522301725 >> v1:172.21.15.134:6789/0 conn(0x7fe99c0aa420 legacy=0x7fe99c1490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:43.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.455+0000 7fe99b400640 1 -- 172.21.15.67:0/522301725 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c0aab20 con 0x7fe99c06dbe0 2024-03-09T09:05:43.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.456+0000 7fe99b400640 1 --2- 172.21.15.67:0/522301725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c06dbe0 0x7fe99c06e090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0702f0 tx=0x7fe994034980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43808ceecb481447 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:43.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.456+0000 7fe999600640 1 -- 172.21.15.67:0/522301725 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99403d020 con 0x7fe99c06dbe0 2024-03-09T09:05:43.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.457+0000 7fe999600640 1 -- 172.21.15.67:0/522301725 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9940114e0 con 0x7fe99c06dbe0 2024-03-09T09:05:43.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.457+0000 7fe999600640 1 -- 172.21.15.67:0/522301725 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940118b0 con 0x7fe99c06dbe0 2024-03-09T09:05:43.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.457+0000 7fe9a1600640 1 -- 172.21.15.67:0/522301725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c06dbe0 msgr2=0x7fe99c06e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:43.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.457+0000 7fe9a1600640 1 --2- 172.21.15.67:0/522301725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c06dbe0 0x7fe99c06e090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0702f0 tx=0x7fe994034980 comp rx=0 tx=0).stop 2024-03-09T09:05:43.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.458+0000 7fe9a1600640 1 -- 172.21.15.67:0/522301725 shutdown_connections 2024-03-09T09:05:43.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.458+0000 7fe9a1600640 1 --2- 172.21.15.67:0/522301725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c06dbe0 0x7fe99c06e090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.458+0000 7fe9a1600640 1 --2- 172.21.15.67:0/522301725 >> v2:172.21.15.134:3300/0 conn(0x7fe99c0c30f0 0x7fe99c152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.458+0000 7fe9a1600640 1 -- 172.21.15.67:0/522301725 >> 172.21.15.67:0/522301725 conn(0x7fe99c0c7370 msgr2=0x7fe99c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:43.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.459+0000 7fe9a1600640 1 -- 172.21.15.67:0/522301725 shutdown_connections 2024-03-09T09:05:43.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.459+0000 7fe9a1600640 1 -- 172.21.15.67:0/522301725 wait complete. 2024-03-09T09:05:43.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.462+0000 7fe9a1600640 1 Processor -- start 2024-03-09T09:05:43.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.462+0000 7fe9a1600640 1 -- start start 2024-03-09T09:05:43.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.463+0000 7fe9a1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c06dbe0 0x7fe99c14a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:43.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.463+0000 7fe9a1600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe99c0aa420 0x7fe99c151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:43.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.463+0000 7fe9a1600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe99c0c30f0 0x7fe99c1db030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:43.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.463+0000 7fe9a1600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c06d6c0 con 0x7fe99c06dbe0 2024-03-09T09:05:43.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.463+0000 7fe9a1600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c0ab0e0 con 0x7fe99c0c30f0 2024-03-09T09:05:43.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.463+0000 7fe9a1600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c149800 con 0x7fe99c0aa420 2024-03-09T09:05:43.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.463+0000 7fe99aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c06dbe0 0x7fe99c14a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:43.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.463+0000 7fe99aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c06dbe0 0x7fe99c14a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37586/0 (socket says 172.21.15.67:37586) 2024-03-09T09:05:43.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.463+0000 7fe99aa00640 1 -- 172.21.15.67:0/3533884973 learned_addr learned my addr 172.21.15.67:0/3533884973 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:43.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.464+0000 7fe99a000640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe99c0aa420 0x7fe99c151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:43.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.464+0000 7fe99b400640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe99c0c30f0 0x7fe99c1db030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:43.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.464+0000 7fe99aa00640 1 -- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe99c0aa420 msgr2=0x7fe99c151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:43.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.464+0000 7fe99aa00640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe99c0aa420 0x7fe99c151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.464+0000 7fe99aa00640 1 -- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe99c0c30f0 msgr2=0x7fe99c1db030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:43.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.464+0000 7fe99aa00640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe99c0c30f0 0x7fe99c1db030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.464+0000 7fe99aa00640 1 -- 172.21.15.67:0/3533884973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c1d9b40 con 0x7fe99c06dbe0 2024-03-09T09:05:43.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.464+0000 7fe99aa00640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c06dbe0 0x7fe99c14a3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe98400e870 tx=0x7fe98400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:43.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.465+0000 7fe992a00640 1 -- 172.21.15.67:0/3533884973 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984017070 con 0x7fe99c06dbe0 2024-03-09T09:05:43.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.465+0000 7fe992a00640 1 -- 172.21.15.67:0/3533884973 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98400c940 con 0x7fe99c06dbe0 2024-03-09T09:05:43.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.465+0000 7fe992a00640 1 -- 172.21.15.67:0/3533884973 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98400cd30 con 0x7fe99c06dbe0 2024-03-09T09:05:43.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.465+0000 7fe9a1600640 1 -- 172.21.15.67:0/3533884973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99c1d8dc0 con 0x7fe99c06dbe0 2024-03-09T09:05:43.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.465+0000 7fe9a1600640 1 -- 172.21.15.67:0/3533884973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99c14aba0 con 0x7fe99c06dbe0 2024-03-09T09:05:43.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.467+0000 7fe9a1600640 1 -- 172.21.15.67:0/3533884973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99c075a00 con 0x7fe99c06dbe0 2024-03-09T09:05:43.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.467+0000 7fe992a00640 1 -- 172.21.15.67:0/3533884973 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe98401a070 con 0x7fe99c06dbe0 2024-03-09T09:05:43.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.468+0000 7fe992a00640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe97806b4d0 0x7fe97806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:43.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.468+0000 7fe992a00640 1 -- 172.21.15.67:0/3533884973 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe98408a3c0 con 0x7fe99c06dbe0 2024-03-09T09:05:43.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.469+0000 7fe99a000640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe97806b4d0 0x7fe97806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:43.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.469+0000 7fe99a000640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe97806b4d0 0x7fe97806d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe99c074b60 tx=0x7fe98c009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:43.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.473+0000 7fe992a00640 1 -- 172.21.15.67:0/3533884973 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe9840587f0 con 0x7fe99c06dbe0 2024-03-09T09:05:43.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.757+0000 7fe9a1600640 1 -- 172.21.15.67:0/3533884973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fe99c153bd0 con 0x7fe99c06dbe0 2024-03-09T09:05:43.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.758+0000 7fe992a00640 1 -- 172.21.15.67:0/3533884973 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fe98405c890 con 0x7fe99c06dbe0 2024-03-09T09:05:43.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 -- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe97806b4d0 msgr2=0x7fe97806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:43.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe97806b4d0 0x7fe97806d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe99c074b60 tx=0x7fe98c009060 comp rx=0 tx=0).stop 2024-03-09T09:05:43.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 -- 172.21.15.67:0/3533884973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c06dbe0 msgr2=0x7fe99c14a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:43.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c06dbe0 0x7fe99c14a3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe98400e870 tx=0x7fe98400ed30 comp rx=0 tx=0).stop 2024-03-09T09:05:43.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 -- 172.21.15.67:0/3533884973 shutdown_connections 2024-03-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe97806b4d0 0x7fe97806d990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe99c0c30f0 0x7fe99c1db030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe99c0aa420 0x7fe99c151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 --2- 172.21.15.67:0/3533884973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe99c06dbe0 0x7fe99c14a3c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 -- 172.21.15.67:0/3533884973 >> 172.21.15.67:0/3533884973 conn(0x7fe99c0c7370 msgr2=0x7fe99c0c3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 -- 172.21.15.67:0/3533884973 shutdown_connections 2024-03-09T09:05:43.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:43.762+0000 7fe9a1600640 1 -- 172.21.15.67:0/3533884973 wait complete. 2024-03-09T09:05:43.787 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:05:43.883 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 38% (86/225) Updating files: 39% (88/225) Updating files: 40% (90/225) Updating files: 41% (93/225) Updating files: 42% (95/225) Updating files: 43% (97/225) Updating files: 44% (99/225) Updating files: 45% (102/225) Updating files: 46% (104/225) Updating files: 47% (106/225) Updating files: 48% (108/225) Updating files: 49% (111/225) Updating files: 50% (113/225) Updating files: 51% (115/225) Updating files: 52% (117/225) Updating files: 53% (120/225) Updating files: 54% (122/225) Updating files: 55% (124/225) Updating files: 56% (126/225) Updating files: 57% (129/225) Updating files: 58% (131/225) Updating files: 59% (133/225) Updating files: 60% (135/225) Updating files: 61% (138/225) Updating files: 62% (140/225) Updating files: 63% (142/225) Updating files: 64% (144/225) Updating files: 65% (147/225) Updating files: 66% (149/225) Updating files: 67% (151/225) Updating files: 68% (153/225) Updating files: 69% (156/225) Updating files: 70% (158/225) Updating files: 71% (160/225) Updating files: 72% (162/225) Updating files: 73% (165/225) Updating files: 74% (167/225) Updating files: 75% (169/225) Updating files: 76% (171/225) Updating files: 77% (174/225) Updating files: 78% (176/225) Updating files: 79% (178/225) Updating files: 79% (179/225) Updating files: 80% (180/225) Updating files: 81% (183/225) Updating files: 82% (185/225) Updating files: 83% (187/225) Updating files: 84% (189/225) Updating files: 85% (192/225) Updating files: 86% (194/225) Updating files: 87% (196/225) Updating files: 88% (198/225) Updating files: 89% (201/225) Updating files: 90% (203/225) Updating files: 91% (205/225) Updating files: 92% (207/225) Updating files: 93% (210/225) Updating files: 94% (212/225) Updating files: 95% (214/225) Updating files: 96% (216/225) Updating files: 97% (219/225) Updating files: 98% (221/225) Updating files: 99% (223/225) Updating files: 100% (225/225) Updating files: 100% (225/225), done. 2024-03-09T09:05:43.924 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 284161c00ab kclient: kill dput/d_drop debug cruft 2024-03-09T09:05:43.940 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.8 2024-03-09T09:05:44.125 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:05:44.125 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 9 -eq 48 ']' 2024-03-09T09:05:44.126 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.9 2024-03-09T09:05:44.126 INFO:tasks.workunit.client.0.smithi067.stdout:v0.9 2024-03-09T09:05:44.126 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.9 2024-03-09T09:05:44.126 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.9 2024-03-09T09:05:45.372 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:05:45.380 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:05:53.788 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:05:53.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:05:54.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.004+0000 7fe9adc00640 1 Processor -- start 2024-03-09T09:05:54.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.004+0000 7fe9adc00640 1 -- start start 2024-03-09T09:05:54.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.005+0000 7fe9adc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9a8153300 0x7fe9a80b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9adc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9a80c5e40 0x7fe9a80b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9adc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe9a80cadb0 0x7fe9a8156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9adc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a8157000 con 0x7fe9a8153300 2024-03-09T09:05:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9adc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a80ae430 con 0x7fe9a80cadb0 2024-03-09T09:05:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9adc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80ae6f0 con 0x7fe9a80c5e40 2024-03-09T09:05:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9a7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9a8153300 0x7fe9a80b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9a7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe9a8153300 0x7fe9a80b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43978/0 (socket says 172.21.15.67:43978) 2024-03-09T09:05:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9a7e00640 1 -- 172.21.15.67:0/1269422037 learned_addr learned my addr 172.21.15.67:0/1269422037 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:54.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9a7400640 1 --2- 172.21.15.67:0/1269422037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9a80c5e40 0x7fe9a80b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9ad200640 1 --2- 172.21.15.67:0/1269422037 >> v2:172.21.15.134:3300/0 conn(0x7fe9a80cadb0 0x7fe9a8156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9a7e00640 1 -- 172.21.15.67:0/1269422037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9a80c5e40 msgr2=0x7fe9a80b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9a7e00640 1 --2- 172.21.15.67:0/1269422037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9a80c5e40 0x7fe9a80b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9a7e00640 1 -- 172.21.15.67:0/1269422037 >> v2:172.21.15.134:3300/0 conn(0x7fe9a80cadb0 msgr2=0x7fe9a8156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.006+0000 7fe9a7e00640 1 --2- 172.21.15.67:0/1269422037 >> v2:172.21.15.134:3300/0 conn(0x7fe9a80cadb0 0x7fe9a8156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.007+0000 7fe9a7e00640 1 -- 172.21.15.67:0/1269422037 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a8155b10 con 0x7fe9a8153300 2024-03-09T09:05:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.008+0000 7fe9a7e00640 1 --2- 172.21.15.67:0/1269422037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a8153300 0x7fe9a80b0000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe9a808d7a0 tx=0x7fe99802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc80311bd632f8c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.008+0000 7fe9a6a00640 1 -- 172.21.15.67:0/1269422037 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998035020 con 0x7fe9a8153300 2024-03-09T09:05:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.008+0000 7fe9a6a00640 1 -- 172.21.15.67:0/1269422037 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe998031870 con 0x7fe9a8153300 2024-03-09T09:05:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.008+0000 7fe9a6a00640 1 -- 172.21.15.67:0/1269422037 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998031c40 con 0x7fe9a8153300 2024-03-09T09:05:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.008+0000 7fe9adc00640 1 -- 172.21.15.67:0/1269422037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a8153300 msgr2=0x7fe9a80b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.008+0000 7fe9adc00640 1 --2- 172.21.15.67:0/1269422037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a8153300 0x7fe9a80b0000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe9a808d7a0 tx=0x7fe99802b910 comp rx=0 tx=0).stop 2024-03-09T09:05:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.009+0000 7fe9adc00640 1 -- 172.21.15.67:0/1269422037 shutdown_connections 2024-03-09T09:05:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.009+0000 7fe9adc00640 1 --2- 172.21.15.67:0/1269422037 >> v2:172.21.15.134:3300/0 conn(0x7fe9a80cadb0 0x7fe9a8156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.009+0000 7fe9adc00640 1 --2- 172.21.15.67:0/1269422037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9a80c5e40 0x7fe9a80b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.009+0000 7fe9adc00640 1 --2- 172.21.15.67:0/1269422037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a8153300 0x7fe9a80b0000 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.009+0000 7fe9adc00640 1 -- 172.21.15.67:0/1269422037 >> 172.21.15.67:0/1269422037 conn(0x7fe9a80732d0 msgr2=0x7fe9a80ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.009+0000 7fe9adc00640 1 -- 172.21.15.67:0/1269422037 shutdown_connections 2024-03-09T09:05:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.009+0000 7fe9adc00640 1 -- 172.21.15.67:0/1269422037 wait complete. 2024-03-09T09:05:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.010+0000 7fe9adc00640 1 Processor -- start 2024-03-09T09:05:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.011+0000 7fe9adc00640 1 -- start start 2024-03-09T09:05:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.011+0000 7fe9adc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9a80c5e40 0x7fe9a80c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9adc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a80cadb0 0x7fe9a81eee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9adc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a8153300 0x7fe9a80abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9adc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80c59e0 con 0x7fe9a80cadb0 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9adc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a8159e90 con 0x7fe9a8153300 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9adc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a8156550 con 0x7fe9a80c5e40 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9a7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a80cadb0 0x7fe9a81eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9a7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a80cadb0 0x7fe9a81eee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43982/0 (socket says 172.21.15.67:43982) 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9a7400640 1 -- 172.21.15.67:0/3081523270 learned_addr learned my addr 172.21.15.67:0/3081523270 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9a7e00640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9a80c5e40 0x7fe9a80c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9ad200640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a8153300 0x7fe9a80abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9a7400640 1 -- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9a80c5e40 msgr2=0x7fe9a80c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9a7400640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9a80c5e40 0x7fe9a80c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9a7400640 1 -- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a8153300 msgr2=0x7fe9a80abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9a7400640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a8153300 0x7fe9a80abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9a7400640 1 -- 172.21.15.67:0/3081523270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a81ef670 con 0x7fe9a80cadb0 2024-03-09T09:05:54.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.012+0000 7fe9ad200640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a8153300 0x7fe9a80abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:05:54.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.013+0000 7fe9a7400640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a80cadb0 0x7fe9a81eee30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe99400d800 tx=0x7fe99400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.013+0000 7fe99fe00640 1 -- 172.21.15.67:0/3081523270 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994018020 con 0x7fe9a80cadb0 2024-03-09T09:05:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.013+0000 7fe99fe00640 1 -- 172.21.15.67:0/3081523270 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9940096a0 con 0x7fe9a80cadb0 2024-03-09T09:05:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.013+0000 7fe99fe00640 1 -- 172.21.15.67:0/3081523270 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994009a70 con 0x7fe9a80cadb0 2024-03-09T09:05:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.013+0000 7fe9adc00640 1 -- 172.21.15.67:0/3081523270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a81ef950 con 0x7fe9a80cadb0 2024-03-09T09:05:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.013+0000 7fe9adc00640 1 -- 172.21.15.67:0/3081523270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a80c77c0 con 0x7fe9a80cadb0 2024-03-09T09:05:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.014+0000 7fe9adc00640 1 -- 172.21.15.67:0/3081523270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a804f3c0 con 0x7fe9a80cadb0 2024-03-09T09:05:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.015+0000 7fe99fe00640 1 -- 172.21.15.67:0/3081523270 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe99401b020 con 0x7fe9a80cadb0 2024-03-09T09:05:54.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.017+0000 7fe99fe00640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe98806b4d0 0x7fe98806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.017+0000 7fe99fe00640 1 -- 172.21.15.67:0/3081523270 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe99408a820 con 0x7fe9a80cadb0 2024-03-09T09:05:54.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.018+0000 7fe9a7e00640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe98806b4d0 0x7fe98806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.019+0000 7fe9a7e00640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe98806b4d0 0x7fe98806d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe998036000 tx=0x7fe99803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.021+0000 7fe99fe00640 1 -- 172.21.15.67:0/3081523270 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe994022030 con 0x7fe9a80cadb0 2024-03-09T09:05:54.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.342+0000 7fe9adc00640 1 -- 172.21.15.67:0/3081523270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe9a80cab90 con 0x7fe9a80cadb0 2024-03-09T09:05:54.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.343+0000 7fe99fe00640 1 -- 172.21.15.67:0/3081523270 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fe9940592a0 con 0x7fe9a80cadb0 2024-03-09T09:05:54.346 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:05:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 -- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe98806b4d0 msgr2=0x7fe98806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe98806b4d0 0x7fe98806d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe998036000 tx=0x7fe99803c040 comp rx=0 tx=0).stop 2024-03-09T09:05:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 -- 172.21.15.67:0/3081523270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a80cadb0 msgr2=0x7fe9a81eee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a80cadb0 0x7fe9a81eee30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe99400d800 tx=0x7fe99400dcc0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 -- 172.21.15.67:0/3081523270 shutdown_connections 2024-03-09T09:05:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe98806b4d0 0x7fe98806d990 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a8153300 0x7fe9a80abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a80cadb0 0x7fe9a81eee30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 --2- 172.21.15.67:0/3081523270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9a80c5e40 0x7fe9a80c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 -- 172.21.15.67:0/3081523270 >> 172.21.15.67:0/3081523270 conn(0x7fe9a80732d0 msgr2=0x7fe9a81568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 -- 172.21.15.67:0/3081523270 shutdown_connections 2024-03-09T09:05:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.346+0000 7fe9adc00640 1 -- 172.21.15.67:0/3081523270 wait complete. 2024-03-09T09:05:54.374 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:05:54.374 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:05:54.375 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:05:54.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:05:54.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.576+0000 7f52c3200640 1 Processor -- start 2024-03-09T09:05:54.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.576+0000 7f52c3200640 1 -- start start 2024-03-09T09:05:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.578+0000 7f52c3200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52c414fd90 0x7f52c4156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.578+0000 7f52c3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f52c406c1d0 0x7f52c406c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.578+0000 7f52c3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c4156bc0 con 0x7f52c40c9380 2024-03-09T09:05:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.578+0000 7f52c3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c406ba60 con 0x7f52c406c1d0 2024-03-09T09:05:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.578+0000 7f52c3200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40c9890 con 0x7f52c414fd90 2024-03-09T09:05:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.578+0000 7f52bbe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f52c40c9380 0x7f52c40acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36482/0 (socket says 172.21.15.67:36482) 2024-03-09T09:05:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.578+0000 7f52bbe00640 1 -- 172.21.15.67:0/3954490727 learned_addr learned my addr 172.21.15.67:0/3954490727 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:05:54.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.578+0000 7f52c2800640 1 --2- 172.21.15.67:0/3954490727 >> v2:172.21.15.67:3300/0 conn(0x7f52c406c1d0 0x7f52c406c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.578+0000 7f52c1e00640 1 --2- 172.21.15.67:0/3954490727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52c414fd90 0x7f52c4156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.578+0000 7f52c1400640 1 -- 172.21.15.67:0/3954490727 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084836956 0 0) 0x7f52ac001110 con 0x7f52c40c9380 2024-03-09T09:05:54.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.579+0000 7f52c1400640 1 -- 172.21.15.67:0/3954490727 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52a8003520 con 0x7f52c40c9380 2024-03-09T09:05:54.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.579+0000 7f52c2800640 1 -- 172.21.15.67:0/3954490727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52c414fd90 msgr2=0x7f52c4156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:54.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.579+0000 7f52c2800640 1 --2- 172.21.15.67:0/3954490727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52c414fd90 0x7f52c4156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.579+0000 7f52c2800640 1 -- 172.21.15.67:0/3954490727 >> v1:172.21.15.67:6789/0 conn(0x7f52c40c9380 legacy=0x7f52c40acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.579+0000 7f52c2800640 1 -- 172.21.15.67:0/3954490727 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40c5f80 con 0x7f52c406c1d0 2024-03-09T09:05:54.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.580+0000 7f52c2800640 1 --2- 172.21.15.67:0/3954490727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c406c1d0 0x7f52c406c600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009d90 tx=0x7f52bc02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52349fed310ddc6d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.581+0000 7f52c1400640 1 -- 172.21.15.67:0/3954490727 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc039070 con 0x7f52c406c1d0 2024-03-09T09:05:54.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.581+0000 7f52c1400640 1 -- 172.21.15.67:0/3954490727 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc02cb40 con 0x7f52c406c1d0 2024-03-09T09:05:54.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.581+0000 7f52c1400640 1 -- 172.21.15.67:0/3954490727 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc034440 con 0x7f52c406c1d0 2024-03-09T09:05:54.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.581+0000 7f52c3200640 1 -- 172.21.15.67:0/3954490727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c406c1d0 msgr2=0x7f52c406c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.581+0000 7f52c3200640 1 --2- 172.21.15.67:0/3954490727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c406c1d0 0x7f52c406c600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009d90 tx=0x7f52bc02bd90 comp rx=0 tx=0).stop 2024-03-09T09:05:54.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.581+0000 7f52c3200640 1 -- 172.21.15.67:0/3954490727 shutdown_connections 2024-03-09T09:05:54.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.581+0000 7f52c3200640 1 --2- 172.21.15.67:0/3954490727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c406c1d0 0x7f52c406c600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.581+0000 7f52c3200640 1 --2- 172.21.15.67:0/3954490727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52c414fd90 0x7f52c4156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.581+0000 7f52c3200640 1 -- 172.21.15.67:0/3954490727 >> 172.21.15.67:0/3954490727 conn(0x7f52c40cb480 msgr2=0x7f52c40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:54.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.581+0000 7f52c3200640 1 -- 172.21.15.67:0/3954490727 shutdown_connections 2024-03-09T09:05:54.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.581+0000 7f52c3200640 1 -- 172.21.15.67:0/3954490727 wait complete. 2024-03-09T09:05:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.583+0000 7f52c3200640 1 Processor -- start 2024-03-09T09:05:54.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.583+0000 7f52c3200640 1 -- start start 2024-03-09T09:05:54.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.584+0000 7f52c3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c406c1d0 0x7f52c414a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.584+0000 7f52c3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c40c9380 0x7f52c4148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.584+0000 7f52c3200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52c414fd90 0x7f52c4148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.584+0000 7f52c3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c406bcb0 con 0x7f52c406c1d0 2024-03-09T09:05:54.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.584+0000 7f52c3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40c6520 con 0x7f52c40c9380 2024-03-09T09:05:54.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.584+0000 7f52c3200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40ad610 con 0x7f52c414fd90 2024-03-09T09:05:54.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.584+0000 7f52c1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c406c1d0 0x7f52c414a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.584+0000 7f52c1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c406c1d0 0x7f52c414a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44010/0 (socket says 172.21.15.67:44010) 2024-03-09T09:05:54.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.584+0000 7f52c1e00640 1 -- 172.21.15.67:0/1246501817 learned_addr learned my addr 172.21.15.67:0/1246501817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:54.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.585+0000 7f52bbe00640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c40c9380 0x7f52c4148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.585+0000 7f52c2800640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52c414fd90 0x7f52c4148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.585+0000 7f52c1e00640 1 -- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52c414fd90 msgr2=0x7f52c4148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.585+0000 7f52c1e00640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52c414fd90 0x7f52c4148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.585+0000 7f52c1e00640 1 -- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c40c9380 msgr2=0x7f52c4148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.585+0000 7f52c1e00640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c40c9380 0x7f52c4148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.585+0000 7f52c1e00640 1 -- 172.21.15.67:0/1246501817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c414d5b0 con 0x7f52c406c1d0 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.585+0000 7f52bbe00640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52c40c9380 0x7f52c4148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.585+0000 7f52c2800640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52c414fd90 0x7f52c4148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.585+0000 7f52c1e00640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c406c1d0 0x7f52c414a9f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f52b400d830 tx=0x7f52b400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.585+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400b6c0 con 0x7f52c406c1d0 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.586+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4019070 con 0x7f52c406c1d0 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.586+0000 7f52c3200640 1 -- 172.21.15.67:0/1246501817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c40aaf80 con 0x7f52c406c1d0 2024-03-09T09:05:54.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.586+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400ba80 con 0x7f52c406c1d0 2024-03-09T09:05:54.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.586+0000 7f52c3200640 1 -- 172.21.15.67:0/1246501817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c4151a80 con 0x7f52c406c1d0 2024-03-09T09:05:54.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.587+0000 7f52c3200640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c4150170 0x7f52c4152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.587+0000 7f52c3200640 1 -- 172.21.15.67:0/1246501817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4151330 con 0x7f52c4150170 2024-03-09T09:05:54.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.587+0000 7f52bbe00640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c4150170 0x7f52c4152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.587+0000 7f52bbe00640 1 -- 172.21.15.67:0/1246501817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52c404f3c0 con 0x7f52c4150170 2024-03-09T09:05:54.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.588+0000 7f52bbe00640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c4150170 0x7f52c4152280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52c414b310 tx=0x7f52ac00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.588+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac005cc0 con 0x7f52c4150170 2024-03-09T09:05:54.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.588+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f52ac017070 con 0x7f52c4150170 2024-03-09T09:05:54.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.588+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c4150170 msgr2=0x7f52c4152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.588+0000 7f52ba000640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c4150170 0x7f52c4152280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52c414b310 tx=0x7f52ac00dac0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.588+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c4150170 msgr2=0x7f52c4152280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:54.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.588+0000 7f52ba000640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c4150170 0x7f52c4152280 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52c414b310 tx=0x7f52ac00dac0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.589+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f52b401e460 con 0x7f52c406c1d0 2024-03-09T09:05:54.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.590+0000 7f52ba000640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f52a006afb0 0x7f52a006d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.590+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f52b408abf0 con 0x7f52c406c1d0 2024-03-09T09:05:54.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.590+0000 7f52bbe00640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f52a006afb0 0x7f52a006d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.592+0000 7f52bbe00640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f52a006afb0 0x7f52a006d470 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f52ac013000 tx=0x7f52ac011000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.595+0000 7f52c3200640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c41527c0 0x7f52c4152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.595+0000 7f52c3200640 1 -- 172.21.15.67:0/1246501817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40adb80 con 0x7f52c41527c0 2024-03-09T09:05:54.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.596+0000 7f52c2800640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c41527c0 0x7f52c4152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.596+0000 7f52c2800640 1 -- 172.21.15.67:0/1246501817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f52c406d2d0 con 0x7f52c41527c0 2024-03-09T09:05:54.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.596+0000 7f52c2800640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c41527c0 0x7f52c4152ba0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52c40aad00 tx=0x7f52bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc044070 con 0x7f52c41527c0 2024-03-09T09:05:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f52bc034cf0 con 0x7f52c41527c0 2024-03-09T09:05:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c41527c0 msgr2=0x7f52c4152ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52ba000640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c41527c0 0x7f52c4152ba0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52c40aad00 tx=0x7f52bc004a40 comp rx=0 tx=0).stop 2024-03-09T09:05:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52ba000640 1 -- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c41527c0 msgr2=0x7f52c4152ba0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52ba000640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c41527c0 0x7f52c4152ba0 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52c40aad00 tx=0x7f52bc004a40 comp rx=0 tx=0).stop 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 98, 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 17, 2024-03-09T09:05:54.601 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:54.602 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:54.603 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:05:54.604 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:54.605 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:54.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:54.607 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:54.608 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:05:54.609 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:54.611 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:54.612 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:05:54.613 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.614 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:54.614 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:54.614 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:54.614 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.614 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.614 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:05:54.614 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:05:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52c3200640 1 -- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f52a006afb0 msgr2=0x7f52a006d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52c3200640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f52a006afb0 0x7f52a006d470 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f52ac013000 tx=0x7f52ac011000 comp rx=0 tx=0).stop 2024-03-09T09:05:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52c3200640 1 -- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c406c1d0 msgr2=0x7f52c414a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52c3200640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c406c1d0 0x7f52c414a9f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f52b400d830 tx=0x7f52b400dcf0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52c2800640 1 -- 172.21.15.67:0/1246501817 reap_dead start 2024-03-09T09:05:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52c3200640 1 -- 172.21.15.67:0/1246501817 shutdown_connections 2024-03-09T09:05:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52c3200640 1 --2- 172.21.15.67:0/1246501817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52c406c1d0 0x7f52c414a9f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.597+0000 7f52c3200640 1 -- 172.21.15.67:0/1246501817 >> 172.21.15.67:0/1246501817 conn(0x7f52c40cb480 msgr2=0x7f52c414b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.598+0000 7f52c3200640 1 -- 172.21.15.67:0/1246501817 shutdown_connections 2024-03-09T09:05:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.598+0000 7f52c3200640 1 -- 172.21.15.67:0/1246501817 wait complete. 2024-03-09T09:05:54.624 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:05:54.624 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:05:54.624 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:05:54.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.826+0000 7f219e400640 1 Processor -- start 2024-03-09T09:05:54.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.827+0000 7f219e400640 1 -- start start 2024-03-09T09:05:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.828+0000 7f219e400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f21980ad8d0 0x7f2198159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.828+0000 7f219e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f219806b880 0x7f2198153080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.828+0000 7f219e400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219806c040 con 0x7f21980c6650 2024-03-09T09:05:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.828+0000 7f219e400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21980c61f0 con 0x7f21980ad8d0 2024-03-09T09:05:54.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.828+0000 7f219e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219806bd60 con 0x7f219806b880 2024-03-09T09:05:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.829+0000 7f219da00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f21980c6650 0x7f2198155e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36488/0 (socket says 172.21.15.67:36488) 2024-03-09T09:05:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.829+0000 7f219da00640 1 -- 172.21.15.67:0/2653431552 learned_addr learned my addr 172.21.15.67:0/2653431552 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:05:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.829+0000 7f218fe00640 1 --2- 172.21.15.67:0/2653431552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f219806b880 0x7f2198153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.829+0000 7f219d000640 1 --2- 172.21.15.67:0/2653431552 >> v2:172.21.15.134:3300/0 conn(0x7f21980ad8d0 0x7f2198159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.829+0000 7f2197e00640 1 -- 172.21.15.67:0/2653431552 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957916902 0 0) 0x7f219806c040 con 0x7f21980c6650 2024-03-09T09:05:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.829+0000 7f2197e00640 1 -- 172.21.15.67:0/2653431552 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f217c003520 con 0x7f21980c6650 2024-03-09T09:05:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.830+0000 7f218fe00640 1 -- 172.21.15.67:0/2653431552 >> v2:172.21.15.134:3300/0 conn(0x7f21980ad8d0 msgr2=0x7f2198159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.830+0000 7f218fe00640 1 --2- 172.21.15.67:0/2653431552 >> v2:172.21.15.134:3300/0 conn(0x7f21980ad8d0 0x7f2198159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.830+0000 7f218fe00640 1 -- 172.21.15.67:0/2653431552 >> v1:172.21.15.67:6789/0 conn(0x7f21980c6650 legacy=0x7f2198155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.830+0000 7f218fe00640 1 -- 172.21.15.67:0/2653431552 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21980c5e80 con 0x7f219806b880 2024-03-09T09:05:54.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.831+0000 7f218fe00640 1 --2- 172.21.15.67:0/2653431552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f219806b880 0x7f2198153080 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f21800097a0 tx=0x7f218002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d9d25d239c96f71 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.831+0000 7f2197e00640 1 -- 172.21.15.67:0/2653431552 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180031780 con 0x7f219806b880 2024-03-09T09:05:54.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.831+0000 7f2197e00640 1 -- 172.21.15.67:0/2653431552 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2180037070 con 0x7f219806b880 2024-03-09T09:05:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.831+0000 7f2197e00640 1 -- 172.21.15.67:0/2653431552 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180031b20 con 0x7f219806b880 2024-03-09T09:05:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.832+0000 7f219e400640 1 -- 172.21.15.67:0/2653431552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f219806b880 msgr2=0x7f2198153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.832+0000 7f219e400640 1 --2- 172.21.15.67:0/2653431552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f219806b880 0x7f2198153080 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f21800097a0 tx=0x7f218002b820 comp rx=0 tx=0).stop 2024-03-09T09:05:54.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.832+0000 7f219e400640 1 -- 172.21.15.67:0/2653431552 shutdown_connections 2024-03-09T09:05:54.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.832+0000 7f219e400640 1 --2- 172.21.15.67:0/2653431552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f219806b880 0x7f2198153080 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.832+0000 7f219e400640 1 --2- 172.21.15.67:0/2653431552 >> v2:172.21.15.134:3300/0 conn(0x7f21980ad8d0 0x7f2198159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.832+0000 7f219e400640 1 -- 172.21.15.67:0/2653431552 >> 172.21.15.67:0/2653431552 conn(0x7f21980c8510 msgr2=0x7f21980acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:54.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.832+0000 7f219e400640 1 -- 172.21.15.67:0/2653431552 shutdown_connections 2024-03-09T09:05:54.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.832+0000 7f219e400640 1 -- 172.21.15.67:0/2653431552 wait complete. 2024-03-09T09:05:54.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.834+0000 7f219e400640 1 Processor -- start 2024-03-09T09:05:54.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.834+0000 7f219e400640 1 -- start start 2024-03-09T09:05:54.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.835+0000 7f219e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219806b880 0x7f21981ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.835+0000 7f219e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21980ad8d0 0x7f21981eb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.835+0000 7f219e400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21980c6650 0x7f219814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.835+0000 7f219e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21981535c0 con 0x7f219806b880 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.835+0000 7f219e400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21980adec0 con 0x7f21980c6650 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.835+0000 7f219e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2198153740 con 0x7f21980ad8d0 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.835+0000 7f219d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219806b880 0x7f21981ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.835+0000 7f219d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219806b880 0x7f21981ef3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44048/0 (socket says 172.21.15.67:44048) 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.835+0000 7f219d000640 1 -- 172.21.15.67:0/3086679682 learned_addr learned my addr 172.21.15.67:0/3086679682 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.835+0000 7f218fe00640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21980ad8d0 0x7f21981eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f219d000640 1 -- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21980ad8d0 msgr2=0x7f21981eb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f219d000640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21980ad8d0 0x7f21981eb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f219d000640 1 -- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21980c6650 msgr2=0x7f219814e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f219d000640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21980c6650 0x7f219814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f219d000640 1 -- 172.21.15.67:0/3086679682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21981eb990 con 0x7f219806b880 2024-03-09T09:05:54.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f218fe00640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21980ad8d0 0x7f21981eb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:05:54.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f219d000640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219806b880 0x7f21981ef3d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f218800e9b0 tx=0x7f218800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188017070 con 0x7f219806b880 2024-03-09T09:05:54.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218800c9a0 con 0x7f219806b880 2024-03-09T09:05:54.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218800cd90 con 0x7f219806b880 2024-03-09T09:05:54.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f219e400640 1 -- 172.21.15.67:0/3086679682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21981596c0 con 0x7f219806b880 2024-03-09T09:05:54.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.836+0000 7f219e400640 1 -- 172.21.15.67:0/3086679682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2198159c80 con 0x7f219806b880 2024-03-09T09:05:54.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.837+0000 7f219e400640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21980ae320 0x7f2198148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.837+0000 7f219e400640 1 -- 172.21.15.67:0/3086679682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219804f3c0 con 0x7f21980ae320 2024-03-09T09:05:54.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.838+0000 7f218fe00640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21980ae320 0x7f2198148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.838+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f218801a070 con 0x7f219806b880 2024-03-09T09:05:54.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.839+0000 7f218fe00640 1 -- 172.21.15.67:0/3086679682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21980b12c0 con 0x7f21980ae320 2024-03-09T09:05:54.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.839+0000 7f2195600640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f217406b4d0 0x7f217406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.839+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f218808a530 con 0x7f219806b880 2024-03-09T09:05:54.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.839+0000 7f218fe00640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21980ae320 0x7f2198148740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f21980c6ae0 tx=0x7f21800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.840+0000 7f219da00640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f217406b4d0 0x7f217406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.840+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180006e10 con 0x7f21980ae320 2024-03-09T09:05:54.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.841+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f2180037040 con 0x7f21980ae320 2024-03-09T09:05:54.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.841+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21980ae320 msgr2=0x7f2198148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.841+0000 7f2195600640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21980ae320 0x7f2198148740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f21980c6ae0 tx=0x7f21800046c0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.841+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21980ae320 msgr2=0x7f2198148740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:54.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.841+0000 7f2195600640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21980ae320 0x7f2198148740 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f21980c6ae0 tx=0x7f21800046c0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.841+0000 7f219da00640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f217406b4d0 0x7f217406d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2190001f80 tx=0x7f21900015f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.848+0000 7f219e400640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198148c80 0x7f2198157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.848+0000 7f219e400640 1 -- 172.21.15.67:0/3086679682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21980c7d40 con 0x7f2198148c80 2024-03-09T09:05:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.849+0000 7f218fe00640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198148c80 0x7f2198157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.850+0000 7f218fe00640 1 -- 172.21.15.67:0/3086679682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f21981490c0 con 0x7f2198148c80 2024-03-09T09:05:54.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.850+0000 7f218fe00640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198148c80 0x7f2198157250 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2180000c00 tx=0x7f2180031000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:54.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.850+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180031ec0 con 0x7f2198148c80 2024-03-09T09:05:54.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.851+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5017 (secure 0 0 0) 0x7f21800093c0 con 0x7f2198148c80 2024-03-09T09:05:54.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.851+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198148c80 msgr2=0x7f2198157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.851+0000 7f2195600640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198148c80 0x7f2198157250 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2180000c00 tx=0x7f2180031000 comp rx=0 tx=0).stop 2024-03-09T09:05:54.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.851+0000 7f2195600640 1 -- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198148c80 msgr2=0x7f2198157250 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:54.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.851+0000 7f2195600640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2198148c80 0x7f2198157250 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.855 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:05:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:05:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:05:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:05:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 98, 2024-03-09T09:05:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:05:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:05:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:05:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:05:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 17, 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:05:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:05:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:54.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.865 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.866 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:05:54.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:54.868 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:54.868 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:54.868 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:54.868 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:54.868 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:54.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:54.868 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:05:54.868 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:05:54.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.852+0000 7f219e400640 1 -- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f217406b4d0 msgr2=0x7f217406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.852+0000 7f219e400640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f217406b4d0 0x7f217406d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2190001f80 tx=0x7f21900015f0 comp rx=0 tx=0).stop 2024-03-09T09:05:54.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.852+0000 7f219e400640 1 -- 172.21.15.67:0/3086679682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219806b880 msgr2=0x7f21981ef3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:54.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.852+0000 7f219e400640 1 --2- 172.21.15.67:0/3086679682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f219806b880 0x7f21981ef3d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f218800e9b0 tx=0x7f218800ee70 comp rx=0 tx=0).stop 2024-03-09T09:05:54.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.852+0000 7f219da00640 1 -- 172.21.15.67:0/3086679682 reap_dead start 2024-03-09T09:05:54.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.852+0000 7f219da00640 1 -- 172.21.15.67:0/3086679682 reap_dead start 2024-03-09T09:05:54.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.852+0000 7f219e400640 1 -- 172.21.15.67:0/3086679682 shutdown_connections 2024-03-09T09:05:54.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.852+0000 7f219e400640 1 -- 172.21.15.67:0/3086679682 >> 172.21.15.67:0/3086679682 conn(0x7f21980c8510 msgr2=0x7f21980c6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:54.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.852+0000 7f219e400640 1 -- 172.21.15.67:0/3086679682 shutdown_connections 2024-03-09T09:05:54.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:54.852+0000 7f219e400640 1 -- 172.21.15.67:0/3086679682 wait complete. 2024-03-09T09:05:54.873 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:05:54.873 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:05:54.874 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:05:55.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.090+0000 7fedfe600640 1 Processor -- start 2024-03-09T09:05:55.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.091+0000 7fedfe600640 1 -- start start 2024-03-09T09:05:55.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.092+0000 7fedfe600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fedf814fd90 0x7fedf8156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.092+0000 7fedfe600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fedf806c1d0 0x7fedf806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.092+0000 7fedfe600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf8156bc0 con 0x7fedf80c9380 2024-03-09T09:05:55.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.092+0000 7fedfe600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf806ba60 con 0x7fedf814fd90 2024-03-09T09:05:55.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.092+0000 7fedfe600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf80c9890 con 0x7fedf806c1d0 2024-03-09T09:05:55.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.092+0000 7fedf7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fedf80c9380 0x7fedf80acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36490/0 (socket says 172.21.15.67:36490) 2024-03-09T09:05:55.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.092+0000 7fedf7e00640 1 -- 172.21.15.67:0/3942552549 learned_addr learned my addr 172.21.15.67:0/3942552549 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:05:55.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.092+0000 7fedfd200640 1 --2- 172.21.15.67:0/3942552549 >> v2:172.21.15.67:3300/0 conn(0x7fedf814fd90 0x7fedf8156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.093+0000 7fedfdc00640 1 --2- 172.21.15.67:0/3942552549 >> v2:172.21.15.134:3300/0 conn(0x7fedf806c1d0 0x7fedf806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.093+0000 7fedf7400640 1 -- 172.21.15.67:0/3942552549 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439972576 0 0) 0x7fede4001110 con 0x7fedf80c9380 2024-03-09T09:05:55.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.093+0000 7fedf7400640 1 -- 172.21.15.67:0/3942552549 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fede0003520 con 0x7fedf80c9380 2024-03-09T09:05:55.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.093+0000 7fedfd200640 1 -- 172.21.15.67:0/3942552549 >> v2:172.21.15.134:3300/0 conn(0x7fedf806c1d0 msgr2=0x7fedf806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:55.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.093+0000 7fedfd200640 1 --2- 172.21.15.67:0/3942552549 >> v2:172.21.15.134:3300/0 conn(0x7fedf806c1d0 0x7fedf806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.093+0000 7fedfd200640 1 -- 172.21.15.67:0/3942552549 >> v1:172.21.15.67:6789/0 conn(0x7fedf80c9380 legacy=0x7fedf80acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.093+0000 7fedfd200640 1 -- 172.21.15.67:0/3942552549 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf80c5f80 con 0x7fedf814fd90 2024-03-09T09:05:55.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.094+0000 7fedfd200640 1 --2- 172.21.15.67:0/3942552549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedf814fd90 0x7fedf8156680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fedf806d270 tx=0x7fedec02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a20a445a18165d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:55.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.094+0000 7fedf7400640 1 -- 172.21.15.67:0/3942552549 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec038070 con 0x7fedf814fd90 2024-03-09T09:05:55.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.095+0000 7fedf7400640 1 -- 172.21.15.67:0/3942552549 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec031710 con 0x7fedf814fd90 2024-03-09T09:05:55.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.095+0000 7fedf7400640 1 -- 172.21.15.67:0/3942552549 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec031ac0 con 0x7fedf814fd90 2024-03-09T09:05:55.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.095+0000 7fedfe600640 1 -- 172.21.15.67:0/3942552549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedf814fd90 msgr2=0x7fedf8156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.095+0000 7fedfe600640 1 --2- 172.21.15.67:0/3942552549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedf814fd90 0x7fedf8156680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fedf806d270 tx=0x7fedec02b7b0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.095+0000 7fedfe600640 1 -- 172.21.15.67:0/3942552549 shutdown_connections 2024-03-09T09:05:55.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.095+0000 7fedfe600640 1 --2- 172.21.15.67:0/3942552549 >> v2:172.21.15.134:3300/0 conn(0x7fedf806c1d0 0x7fedf806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.095+0000 7fedfe600640 1 --2- 172.21.15.67:0/3942552549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedf814fd90 0x7fedf8156680 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.095+0000 7fedfe600640 1 -- 172.21.15.67:0/3942552549 >> 172.21.15.67:0/3942552549 conn(0x7fedf80cb480 msgr2=0x7fedf80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.095+0000 7fedfe600640 1 -- 172.21.15.67:0/3942552549 shutdown_connections 2024-03-09T09:05:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.095+0000 7fedfe600640 1 -- 172.21.15.67:0/3942552549 wait complete. 2024-03-09T09:05:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.097+0000 7fedfe600640 1 Processor -- start 2024-03-09T09:05:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.097+0000 7fedfe600640 1 -- start start 2024-03-09T09:05:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.098+0000 7fedfe600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedf806c1d0 0x7fedf81596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.098+0000 7fedfe600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedf80c9380 0x7fedf8157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.098+0000 7fedfe600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf814fd90 0x7fedf81580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.098+0000 7fedfe600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf806bcb0 con 0x7fedf80c9380 2024-03-09T09:05:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.098+0000 7fedfe600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf80c6520 con 0x7fedf806c1d0 2024-03-09T09:05:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.098+0000 7fedfe600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf80ad610 con 0x7fedf814fd90 2024-03-09T09:05:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.098+0000 7fedf7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedf80c9380 0x7fedf8157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.098+0000 7fedf7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedf80c9380 0x7fedf8157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44066/0 (socket says 172.21.15.67:44066) 2024-03-09T09:05:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.098+0000 7fedf7e00640 1 -- 172.21.15.67:0/3750227972 learned_addr learned my addr 172.21.15.67:0/3750227972 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.099+0000 7fedfd200640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedf806c1d0 0x7fedf81596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.099+0000 7fedfdc00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf814fd90 0x7fedf81580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.099+0000 7fedf7e00640 1 -- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf814fd90 msgr2=0x7fedf81580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.099+0000 7fedf7e00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf814fd90 0x7fedf81580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.099+0000 7fedf7e00640 1 -- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedf806c1d0 msgr2=0x7fedf81596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.099+0000 7fedf7e00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fedf806c1d0 0x7fedf81596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.099+0000 7fedf7e00640 1 -- 172.21.15.67:0/3750227972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf81ec630 con 0x7fedf80c9380 2024-03-09T09:05:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.099+0000 7fedfdc00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf814fd90 0x7fedf81580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:05:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.099+0000 7fedf7e00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedf80c9380 0x7fedf8157b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fede400c3a0 tx=0x7fede400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.100+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4017070 con 0x7fedf80c9380 2024-03-09T09:05:55.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.100+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede4005ce0 con 0x7fedf80c9380 2024-03-09T09:05:55.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.100+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4007310 con 0x7fedf80c9380 2024-03-09T09:05:55.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.100+0000 7fedfe600640 1 -- 172.21.15.67:0/3750227972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf81570b0 con 0x7fedf80c9380 2024-03-09T09:05:55.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.100+0000 7fedfe600640 1 -- 172.21.15.67:0/3750227972 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf8159c20 con 0x7fedf80c9380 2024-03-09T09:05:55.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.101+0000 7fedfe600640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8157700 0x7fedf81eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.101+0000 7fedfe600640 1 -- 172.21.15.67:0/3750227972 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf81ec3e0 con 0x7fedf8157700 2024-03-09T09:05:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.101+0000 7fedfd200640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8157700 0x7fedf81eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.102+0000 7fedfd200640 1 -- 172.21.15.67:0/3750227972 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedf81ec910 con 0x7fedf8157700 2024-03-09T09:05:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.102+0000 7fedfd200640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8157700 0x7fedf81eeed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fedf80ae040 tx=0x7fedec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.103+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fede401a070 con 0x7fedf80c9380 2024-03-09T09:05:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.104+0000 7fedf4c00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fedd806b500 0x7fedd806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.104+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec038040 con 0x7fedf8157700 2024-03-09T09:05:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.104+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fede40898c0 con 0x7fedf80c9380 2024-03-09T09:05:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.104+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fedec03d620 con 0x7fedf8157700 2024-03-09T09:05:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.104+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8157700 msgr2=0x7fedf81eeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.104+0000 7fedf4c00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8157700 0x7fedf81eeed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fedf80ae040 tx=0x7fedec0046c0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.104+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8157700 msgr2=0x7fedf81eeed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.104+0000 7fedf4c00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8157700 0x7fedf81eeed0 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fedf80ae040 tx=0x7fedec0046c0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.104+0000 7fedfdc00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fedd806b500 0x7fedd806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.105+0000 7fedfdc00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fedd806b500 0x7fedd806d9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fedf0007440 tx=0x7fedf000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.111+0000 7fedfe600640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8158610 0x7fedf81ef410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.111+0000 7fedfe600640 1 -- 172.21.15.67:0/3750227972 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf804f3c0 con 0x7fedf8158610 2024-03-09T09:05:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.112+0000 7fedfd200640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8158610 0x7fedf81ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.113+0000 7fedfd200640 1 -- 172.21.15.67:0/3750227972 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fedf806d2d0 con 0x7fedf8158610 2024-03-09T09:05:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.113+0000 7fedfd200640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8158610 0x7fedf81ef410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fedf804fcf0 tx=0x7fedec031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:55.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.114+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec03daa0 con 0x7fedf8158610 2024-03-09T09:05:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.114+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fedec03daa0 con 0x7fedf8158610 2024-03-09T09:05:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.114+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8158610 msgr2=0x7fedf81ef410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.114+0000 7fedf4c00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8158610 0x7fedf81ef410 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fedf804fcf0 tx=0x7fedec031000 comp rx=0 tx=0).stop 2024-03-09T09:05:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.114+0000 7fedf4c00640 1 -- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8158610 msgr2=0x7fedf81ef410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:05:55.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.114+0000 7fedf4c00640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fedf8158610 0x7fedf81ef410 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fedf804fcf0 tx=0x7fedec031000 comp rx=0 tx=0).stop 2024-03-09T09:05:55.121 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:05:55.121 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:05:55.121 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:05:55.121 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:05:55.122 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 98, 2024-03-09T09:05:55.122 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:05:55.122 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:05:55.122 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:05:55.122 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:05:55.122 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:55.123 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 18, 2024-03-09T09:05:55.123 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:55.123 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:05:55.123 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:05:55.123 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:55.123 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:55.124 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:55.124 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:55.124 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:55.124 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:55.124 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:55.124 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:55.124 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:55.125 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:55.125 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:55.125 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:55.125 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:05:55.125 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:05:55.125 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:55.126 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:55.126 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:55.126 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:55.126 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:55.126 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:55.126 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:55.127 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:55.127 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:55.127 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:55.127 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:55.127 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:55.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.128 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:05:55.128 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:05:55.128 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:05:55.128 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:05:55.128 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:05:55.128 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:05:55.129 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:55.129 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:05:55.129 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:05:55.129 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:05:55.129 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:05:55.129 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:05:55.130 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:05:55.130 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:05:55.130 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:05:55.130 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:05:55.130 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:05:55.131 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:05:55.131 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:05:55.131 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:05:55.131 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:05:55.132 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:05:55.132 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:05:55.132 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:05:55.132 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:05:55.132 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:05:55.133 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:05:55.133 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:05:55.133 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:55.133 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:05:55.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.134 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:05:55.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:55.134 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:05:55.134 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:05:55.135 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:55.135 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:55.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:55.135 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:55.136 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:05:55.136 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:55.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:55.137 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:55.137 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:05:55.137 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:55.137 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:55.138 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:55.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.138 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:05:55.138 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:05:55.138 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:55.139 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:55.139 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:55.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:55.140 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:05:55.140 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:05:55.140 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:55.140 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:55.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:55.141 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:55.141 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:05:55.141 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:55.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:55.142 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:55.142 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:05:55.142 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:55.142 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:55.143 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:55.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.143 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:05:55.143 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:05:55.143 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:55.144 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:55.144 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:55.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:55.144 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:05:55.145 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:05:55.145 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:05:55.145 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:05:55.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:55.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:05:55.146 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:05:55.146 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:55.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:55.147 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:05:55.147 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:05:55.147 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:05:55.147 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:55.147 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:55.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.148 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:05:55.148 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:05:55.148 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:05:55.148 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:05:55.149 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:05:55.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:55.149 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:55.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.149 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:05:55.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:05:55.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:55.150 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:55.150 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:55.150 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:55.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:55.151 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:05:55.151 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:05:55.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:05:55.151 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:05:55.151 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:05:55.152 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:05:55.152 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:05:55.152 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:05:55.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:05:55.152 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:05:55.152 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:05:55.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.115+0000 7fedfe600640 1 -- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fedd806b500 msgr2=0x7fedd806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.115+0000 7fedfe600640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fedd806b500 0x7fedd806d9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fedf0007440 tx=0x7fedf000b040 comp rx=0 tx=0).stop 2024-03-09T09:05:55.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.115+0000 7fedfe600640 1 -- 172.21.15.67:0/3750227972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedf80c9380 msgr2=0x7fedf8157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.115+0000 7fedfe600640 1 --2- 172.21.15.67:0/3750227972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedf80c9380 0x7fedf8157b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fede400c3a0 tx=0x7fede400c860 comp rx=0 tx=0).stop 2024-03-09T09:05:55.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.115+0000 7fedfdc00640 1 -- 172.21.15.67:0/3750227972 reap_dead start 2024-03-09T09:05:55.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.115+0000 7fedfdc00640 1 -- 172.21.15.67:0/3750227972 reap_dead start 2024-03-09T09:05:55.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.115+0000 7fedfe600640 1 -- 172.21.15.67:0/3750227972 shutdown_connections 2024-03-09T09:05:55.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.115+0000 7fedfe600640 1 -- 172.21.15.67:0/3750227972 >> 172.21.15.67:0/3750227972 conn(0x7fedf80cb480 msgr2=0x7fedf806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:55.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.115+0000 7fedfe600640 1 -- 172.21.15.67:0/3750227972 shutdown_connections 2024-03-09T09:05:55.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.115+0000 7fedfe600640 1 -- 172.21.15.67:0/3750227972 wait complete. 2024-03-09T09:05:55.156 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:05:55.156 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-03-09T09:05:55.156 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:05:55.156 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-03-09T09:05:55.156 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:05:55.156 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-09T09:05:55.156 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:05:55.200 INFO:tasks.ceph.mon.b:Stopped 2024-03-09T09:05:55.200 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:05:55.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:05:55.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.412+0000 7f96cde00640 1 Processor -- start 2024-03-09T09:05:55.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.413+0000 7f96cde00640 1 -- start start 2024-03-09T09:05:55.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.414+0000 7f96cde00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac730 0x7f96c8071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.414+0000 7f96cde00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f96c80c22a0 0x7f96c80acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.414+0000 7f96cde00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c8153d40 con 0x7f96c8157650 2024-03-09T09:05:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.414+0000 7f96cde00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c8070ee0 con 0x7f96c80c22a0 2024-03-09T09:05:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.414+0000 7f96cde00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c8154d00 con 0x7f96c80ac730 2024-03-09T09:05:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.414+0000 7f96bfe00640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f96c80c22a0 msgr2=0x7f96c80acb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:05:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.414+0000 7f96bfe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f96c80c22a0 0x7f96c80acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.414+0000 7f96c7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f96c8157650 0x7f96c8157b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36500/0 (socket says 172.21.15.67:36500) 2024-03-09T09:05:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c7400640 1 -- 172.21.15.67:0/1318815331 learned_addr learned my addr 172.21.15.67:0/1318815331 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:05:55.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6a00640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac730 0x7f96c8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39962450 0 0) 0x7f96c8153d40 con 0x7f96c8157650 2024-03-09T09:05:55.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96ac003520 con 0x7f96c8157650 2024-03-09T09:05:55.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1635676233 0 0) 0x7f96ac003520 con 0x7f96c8157650 2024-03-09T09:05:55.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac730 msgr2=0x7f96c8071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:55.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6000640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac730 0x7f96c8071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 >> v2:172.21.15.134:3300/0 conn(0x7f96c80c22a0 msgr2=0x7f96c80acb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6000640 1 --2- 172.21.15.67:0/1318815331 >> v2:172.21.15.134:3300/0 conn(0x7f96c80c22a0 0x7f96c80acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c80c2790 con 0x7f96c8157650 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f96c00019f0 con 0x7f96c8157650 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 >> v1:172.21.15.67:6789/0 conn(0x7f96c8157650 legacy=0x7f96c8157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.415+0000 7f96c6000640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac730 0x7f96ac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c6000640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96ac003f30 0x7f96ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c6000640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96ac004d40 0x7f96ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c8070ee0 con 0x7f96ac004d40 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c8154d00 con 0x7f96ac003f30 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac00bba0 con 0x7f96c80ac730 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96bfe00640 1 -- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96ac003f30 msgr2=0x7f96ac004800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96bfe00640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96ac003f30 0x7f96ac004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c7400640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96ac004d40 0x7f96ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c6a00640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac730 0x7f96ac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c7400640 1 -- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac730 msgr2=0x7f96ac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:05:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c7400640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96c80ac730 0x7f96ac0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c7400640 1 -- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96ac003f30 msgr2=0x7f96ac004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c7400640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96ac003f30 0x7f96ac004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.416+0000 7f96c7400640 1 -- 172.21.15.67:0/1318815331 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac003520 con 0x7f96ac004d40 2024-03-09T09:05:55.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.417+0000 7f96c7400640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96ac004d40 0x7f96ac0051c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f96c000a7e0 tx=0x7f96ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1b0d0a637e80f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.418+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c000a3d0 con 0x7f96ac004d40 2024-03-09T09:05:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.418+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0034630 con 0x7f96ac004d40 2024-03-09T09:05:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.418+0000 7f96c6000640 1 -- 172.21.15.67:0/1318815331 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0034aa0 con 0x7f96ac004d40 2024-03-09T09:05:55.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.418+0000 7f96cde00640 1 -- 172.21.15.67:0/1318815331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96ac004d40 msgr2=0x7f96ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.418+0000 7f96cde00640 1 --2- 172.21.15.67:0/1318815331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96ac004d40 0x7f96ac0051c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f96c000a7e0 tx=0x7f96ac005830 comp rx=0 tx=0).stop 2024-03-09T09:05:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.418+0000 7f96c7400640 1 -- 172.21.15.67:0/1318815331 reap_dead start 2024-03-09T09:05:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.418+0000 7f96cde00640 1 -- 172.21.15.67:0/1318815331 shutdown_connections 2024-03-09T09:05:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.418+0000 7f96cde00640 1 -- 172.21.15.67:0/1318815331 >> 172.21.15.67:0/1318815331 conn(0x7f96c80c7370 msgr2=0x7f96c80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.418+0000 7f96cde00640 1 -- 172.21.15.67:0/1318815331 shutdown_connections 2024-03-09T09:05:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.418+0000 7f96cde00640 1 -- 172.21.15.67:0/1318815331 wait complete. 2024-03-09T09:05:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.420+0000 7f96cde00640 1 Processor -- start 2024-03-09T09:05:55.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.420+0000 7f96cde00640 1 -- start start 2024-03-09T09:05:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.421+0000 7f96cde00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96ac004d40 0x7f96c81ef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.421+0000 7f96cde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c8150040 0x7f96c81ef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.421+0000 7f96cde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c80aaa40 0x7f96c80aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.421+0000 7f96cde00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c8151610 con 0x7f96c8150040 2024-03-09T09:05:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.421+0000 7f96cde00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c8070c00 con 0x7f96c80aaa40 2024-03-09T09:05:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.421+0000 7f96cde00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c8151790 con 0x7f96ac004d40 2024-03-09T09:05:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.421+0000 7f96bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c8150040 0x7f96c81ef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.421+0000 7f96c7400640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c80aaa40 msgr2=0x7f96c80aaec0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:05:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.421+0000 7f96c7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c80aaa40 0x7f96c80aaec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:55.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.421+0000 7f96bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c8150040 0x7f96c81ef5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44104/0 (socket says 172.21.15.67:44104) 2024-03-09T09:05:55.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.421+0000 7f96bfe00640 1 -- 172.21.15.67:0/3880002884 learned_addr learned my addr 172.21.15.67:0/3880002884 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:55.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.422+0000 7f96c6a00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96ac004d40 0x7f96c81ef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.422+0000 7f96bfe00640 1 -- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96ac004d40 msgr2=0x7f96c81ef0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.422+0000 7f96bfe00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96ac004d40 0x7f96c81ef0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.422+0000 7f96bfe00640 1 -- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c80aaa40 msgr2=0x7f96c80aaec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:05:55.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.422+0000 7f96bfe00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c80aaa40 0x7f96c80aaec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.422+0000 7f96bfe00640 1 -- 172.21.15.67:0/3880002884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c81e2770 con 0x7f96c8150040 2024-03-09T09:05:55.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.422+0000 7f96bfe00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c8150040 0x7f96c81ef5e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f96b000b4e0 tx=0x7f96b000b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:55.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.422+0000 7f96bf400640 1 -- 172.21.15.67:0/3880002884 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0003dc0 con 0x7f96c8150040 2024-03-09T09:05:55.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.423+0000 7f96bf400640 1 -- 172.21.15.67:0/3880002884 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b0015070 con 0x7f96c8150040 2024-03-09T09:05:55.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.423+0000 7f96bf400640 1 -- 172.21.15.67:0/3880002884 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b00052b0 con 0x7f96c8150040 2024-03-09T09:05:55.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.423+0000 7f96cde00640 1 -- 172.21.15.67:0/3880002884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c81e2a50 con 0x7f96c8150040 2024-03-09T09:05:55.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.423+0000 7f96cde00640 1 -- 172.21.15.67:0/3880002884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c81efb20 con 0x7f96c8150040 2024-03-09T09:05:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.424+0000 7f96cde00640 1 -- 172.21.15.67:0/3880002884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c8075a00 con 0x7f96c8150040 2024-03-09T09:05:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.425+0000 7f96bf400640 1 -- 172.21.15.67:0/3880002884 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f96b0005450 con 0x7f96c8150040 2024-03-09T09:05:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.426+0000 7f96bf400640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a406b4b0 0x7f96a406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.426+0000 7f96bf400640 1 -- 172.21.15.67:0/3880002884 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f96b008a350 con 0x7f96c8150040 2024-03-09T09:05:55.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.426+0000 7f96c6a00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a406b4b0 0x7f96a406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:55.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.427+0000 7f96c6a00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a406b4b0 0x7f96a406d970 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009a10 tx=0x7f96b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:55.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.430+0000 7f96bf400640 1 -- 172.21.15.67:0/3880002884 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f96b000e340 con 0x7f96c8150040 2024-03-09T09:05:55.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.703+0000 7f96cde00640 1 -- 172.21.15.67:0/3880002884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f96c80c11c0 con 0x7f96c8150040 2024-03-09T09:05:55.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.704+0000 7f96bf400640 1 -- 172.21.15.67:0/3880002884 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f96b0058dd0 con 0x7f96c8150040 2024-03-09T09:05:55.707 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:05:55.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 -- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a406b4b0 msgr2=0x7f96a406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a406b4b0 0x7f96a406d970 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009a10 tx=0x7f96b8008040 comp rx=0 tx=0).stop 2024-03-09T09:05:55.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 -- 172.21.15.67:0/3880002884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c8150040 msgr2=0x7f96c81ef5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:55.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c8150040 0x7f96c81ef5e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f96b000b4e0 tx=0x7f96b000b9a0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 -- 172.21.15.67:0/3880002884 shutdown_connections 2024-03-09T09:05:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f96a406b4b0 0x7f96a406d970 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96c80aaa40 0x7f96c80aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c8150040 0x7f96c81ef5e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 --2- 172.21.15.67:0/3880002884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96ac004d40 0x7f96c81ef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 -- 172.21.15.67:0/3880002884 >> 172.21.15.67:0/3880002884 conn(0x7f96c80c7370 msgr2=0x7f96c80c5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 -- 172.21.15.67:0/3880002884 shutdown_connections 2024-03-09T09:05:55.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:55.707+0000 7f96cde00640 1 -- 172.21.15.67:0/3880002884 wait complete. 2024-03-09T09:05:58.733 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:05:58.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.939+0000 7f3d82a00640 1 Processor -- start 2024-03-09T09:05:58.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.939+0000 7f3d82a00640 1 -- start start 2024-03-09T09:05:58.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.941+0000 7f3d82a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c14c5c0 0x7f3d7c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:58.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.941+0000 7f3d82a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3d7c0aad00 0x7f3d7c0ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:58.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.941+0000 7f3d82a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c0c1d70 con 0x7f3d7c0c3780 2024-03-09T09:05:58.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.941+0000 7f3d82a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7c0aaae0 con 0x7f3d7c0aad00 2024-03-09T09:05:58.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.941+0000 7f3d82a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c14c9a0 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.941+0000 7f3d7b400640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f3d7c0c3780 legacy=0x7f3d7c06df90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:05:58.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.941+0000 7f3d80a00640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f3d7c0aad00 msgr2=0x7f3d7c0ab170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:05:58.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.941+0000 7f3d80a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3d7c0aad00 0x7f3d7c0ab170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:58.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.941+0000 7f3d7be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c14c5c0 0x7f3d7c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:58.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.941+0000 7f3d7be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c14c5c0 0x7f3d7c152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:44076/0 (socket says 172.21.15.67:44076) 2024-03-09T09:05:58.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.941+0000 7f3d7be00640 1 -- 172.21.15.67:0/401793902 learned_addr learned my addr 172.21.15.67:0/401793902 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:58.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.942+0000 7f3d7be00640 1 -- 172.21.15.67:0/401793902 >> v2:172.21.15.134:3300/0 conn(0x7f3d7c0aad00 msgr2=0x7f3d7c0ab170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:58.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.942+0000 7f3d7be00640 1 --2- 172.21.15.67:0/401793902 >> v2:172.21.15.134:3300/0 conn(0x7f3d7c0aad00 0x7f3d7c0ab170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:58.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.942+0000 7f3d7be00640 1 -- 172.21.15.67:0/401793902 >> v1:172.21.15.134:6789/0 conn(0x7f3d7c0c3780 legacy=0x7f3d7c06df90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:05:58.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.942+0000 7f3d7be00640 1 -- 172.21.15.67:0/401793902 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c0c2010 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.944+0000 7f3d7be00640 1 --2- 172.21.15.67:0/401793902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c14c5c0 0x7f3d7c152ce0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009870 tx=0x7f3d6c02b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd955e22bc7865c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:58.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.944+0000 7f3d7aa00640 1 -- 172.21.15.67:0/401793902 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c02e070 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.944+0000 7f3d7aa00640 1 -- 172.21.15.67:0/401793902 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c037070 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.944+0000 7f3d7aa00640 1 -- 172.21.15.67:0/401793902 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c031a00 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.945+0000 7f3d82a00640 1 -- 172.21.15.67:0/401793902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c14c5c0 msgr2=0x7f3d7c152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:58.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.945+0000 7f3d82a00640 1 --2- 172.21.15.67:0/401793902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c14c5c0 0x7f3d7c152ce0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009870 tx=0x7f3d6c02b880 comp rx=0 tx=0).stop 2024-03-09T09:05:58.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.945+0000 7f3d82a00640 1 -- 172.21.15.67:0/401793902 shutdown_connections 2024-03-09T09:05:58.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.945+0000 7f3d82a00640 1 --2- 172.21.15.67:0/401793902 >> v2:172.21.15.134:3300/0 conn(0x7f3d7c0aad00 0x7f3d7c0ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:58.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.945+0000 7f3d82a00640 1 --2- 172.21.15.67:0/401793902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c14c5c0 0x7f3d7c152ce0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:58.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.945+0000 7f3d82a00640 1 -- 172.21.15.67:0/401793902 >> 172.21.15.67:0/401793902 conn(0x7f3d7c0c7370 msgr2=0x7f3d7c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:58.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.945+0000 7f3d82a00640 1 -- 172.21.15.67:0/401793902 shutdown_connections 2024-03-09T09:05:58.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.945+0000 7f3d82a00640 1 -- 172.21.15.67:0/401793902 wait complete. 2024-03-09T09:05:58.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.948+0000 7f3d82a00640 1 Processor -- start 2024-03-09T09:05:58.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.948+0000 7f3d82a00640 1 -- start start 2024-03-09T09:05:58.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d82a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d7c0aad00 0x7f3d7c13faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:58.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d82a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c0c3780 0x7f3d7c14df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:58.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d82a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d7c14c5c0 0x7f3d7c1ef570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:58.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d82a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c28b0 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d82a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c25d0 con 0x7f3d7c0aad00 2024-03-09T09:05:58.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d82a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c06e6a0 con 0x7f3d7c0c3780 2024-03-09T09:05:58.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d7be00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d7c0aad00 msgr2=0x7f3d7c13faf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:05:58.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d7be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d7c0aad00 0x7f3d7c13faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:05:58.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d80a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d7c14c5c0 0x7f3d7c1ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:58.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d80a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d7c14c5c0 0x7f3d7c1ef570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58674/0 (socket says 172.21.15.67:58674) 2024-03-09T09:05:58.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d80a00640 1 -- 172.21.15.67:0/4292139058 learned_addr learned my addr 172.21.15.67:0/4292139058 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:05:58.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.950+0000 7f3d7b400640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c0c3780 0x7f3d7c14df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:58.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.951+0000 7f3d80a00640 1 -- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c0c3780 msgr2=0x7f3d7c14df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:58.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.951+0000 7f3d80a00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c0c3780 0x7f3d7c14df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:58.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.951+0000 7f3d80a00640 1 -- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d7c0aad00 msgr2=0x7f3d7c13faf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:05:58.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.951+0000 7f3d80a00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d7c0aad00 0x7f3d7c13faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:58.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.951+0000 7f3d80a00640 1 -- 172.21.15.67:0/4292139058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c1efce0 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.951+0000 7f3d80a00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d7c14c5c0 0x7f3d7c1ef570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400be50 tx=0x7f3d7400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:58.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.952+0000 7f3d73e00640 1 -- 172.21.15.67:0/4292139058 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74016430 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.952+0000 7f3d73e00640 1 -- 172.21.15.67:0/4292139058 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d74010030 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.952+0000 7f3d73e00640 1 -- 172.21.15.67:0/4292139058 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74016810 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.952+0000 7f3d82a00640 1 -- 172.21.15.67:0/4292139058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c1effc0 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.952+0000 7f3d82a00640 1 -- 172.21.15.67:0/4292139058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c1ec2c0 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.953+0000 7f3d82a00640 1 -- 172.21.15.67:0/4292139058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c075a00 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.954+0000 7f3d73e00640 1 -- 172.21.15.67:0/4292139058 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3d740169b0 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.955+0000 7f3d73e00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d5c06b4b0 0x7f3d5c06d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:05:58.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.955+0000 7f3d73e00640 1 -- 172.21.15.67:0/4292139058 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f3d7408ac30 con 0x7f3d7c14c5c0 2024-03-09T09:05:58.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.956+0000 7f3d7be00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d5c06b4b0 0x7f3d5c06d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:05:58.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.957+0000 7f3d7be00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d5c06b4b0 0x7f3d5c06d970 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c036000 tx=0x7f3d6c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:05:58.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:58.960+0000 7f3d73e00640 1 -- 172.21.15.67:0/4292139058 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3d7401b020 con 0x7f3d7c14c5c0 2024-03-09T09:05:59.017 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 36% (81/225) Updating files: 37% (84/225) Updating files: 38% (86/225) Updating files: 39% (88/225) Updating files: 40% (90/225) Updating files: 41% (93/225) Updating files: 42% (95/225) Updating files: 43% (97/225) Updating files: 44% (99/225) Updating files: 45% (102/225) Updating files: 46% (104/225) Updating files: 47% (106/225) Updating files: 48% (108/225) Updating files: 49% (111/225) Updating files: 50% (113/225) Updating files: 51% (115/225) Updating files: 52% (117/225) Updating files: 53% (120/225) Updating files: 54% (122/225) Updating files: 55% (124/225) Updating files: 56% (126/225) Updating files: 57% (129/225) Updating files: 58% (131/225) Updating files: 59% (133/225) Updating files: 60% (135/225) Updating files: 61% (138/225) Updating files: 62% (140/225) Updating files: 63% (142/225) Updating files: 64% (144/225) Updating files: 65% (147/225) Updating files: 66% (149/225) Updating files: 67% (151/225) Updating files: 68% (153/225) Updating files: 69% (156/225) Updating files: 70% (158/225) Updating files: 71% (160/225) Updating files: 72% (162/225) Updating files: 73% (165/225) Updating files: 74% (167/225) Updating files: 75% (169/225) Updating files: 76% (171/225) Updating files: 77% (174/225) Updating files: 78% (176/225) Updating files: 79% (178/225) Updating files: 80% (180/225) Updating files: 81% (183/225) Updating files: 82% (185/225) Updating files: 83% (187/225) Updating files: 84% (189/225) Updating files: 85% (192/225) Updating files: 86% (194/225) Updating files: 87% (196/225) Updating files: 88% (198/225) Updating files: 89% (201/225) Updating files: 90% (203/225) Updating files: 91% (205/225) Updating files: 92% (207/225) Updating files: 93% (210/225) Updating files: 94% (212/225) Updating files: 95% (214/225) Updating files: 96% (216/225) Updating files: 96% (218/225) Updating files: 97% (219/225) Updating files: 98% (221/225) Updating files: 99% (223/225) Updating files: 100% (225/225) Updating files: 100% (225/225), done. 2024-03-09T09:05:59.056 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 284161c00ab kclient: kill dput/d_drop debug cruft 2024-03-09T09:05:59.074 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.8 2024-03-09T09:05:59.252 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:05:59.252 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 9 -eq 48 ']' 2024-03-09T09:05:59.252 INFO:tasks.workunit.client.1.smithi134.stdout:v0.9 2024-03-09T09:05:59.252 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.9 2024-03-09T09:05:59.253 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.9 2024-03-09T09:05:59.253 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.9 2024-03-09T09:05:59.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.295+0000 7f3d82a00640 1 -- 172.21.15.67:0/4292139058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3d7c14fc80 con 0x7f3d7c14c5c0 2024-03-09T09:05:59.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.296+0000 7f3d73e00640 1 -- 172.21.15.67:0/4292139058 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f3d74059750 con 0x7f3d7c14c5c0 2024-03-09T09:05:59.299 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:05:59.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.298+0000 7f3d82a00640 1 -- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d5c06b4b0 msgr2=0x7f3d5c06d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:59.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.298+0000 7f3d82a00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d5c06b4b0 0x7f3d5c06d970 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c036000 tx=0x7f3d6c008030 comp rx=0 tx=0).stop 2024-03-09T09:05:59.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.298+0000 7f3d82a00640 1 -- 172.21.15.67:0/4292139058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d7c14c5c0 msgr2=0x7f3d7c1ef570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:05:59.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.298+0000 7f3d82a00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d7c14c5c0 0x7f3d7c1ef570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400be50 tx=0x7f3d7400be80 comp rx=0 tx=0).stop 2024-03-09T09:05:59.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.299+0000 7f3d82a00640 1 -- 172.21.15.67:0/4292139058 shutdown_connections 2024-03-09T09:05:59.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.299+0000 7f3d82a00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d5c06b4b0 0x7f3d5c06d970 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:59.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.299+0000 7f3d82a00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d7c14c5c0 0x7f3d7c1ef570 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:59.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.299+0000 7f3d82a00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d7c0c3780 0x7f3d7c14df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.299+0000 7f3d82a00640 1 --2- 172.21.15.67:0/4292139058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d7c0aad00 0x7f3d7c13faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:05:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.299+0000 7f3d82a00640 1 -- 172.21.15.67:0/4292139058 >> 172.21.15.67:0/4292139058 conn(0x7f3d7c0c7370 msgr2=0x7f3d7c1517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:05:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.299+0000 7f3d82a00640 1 -- 172.21.15.67:0/4292139058 shutdown_connections 2024-03-09T09:05:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:05:59.299+0000 7f3d82a00640 1 -- 172.21.15.67:0/4292139058 wait complete. 2024-03-09T09:06:01.834 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 46% (89/193) Updating files: 47% (91/193) Updating files: 48% (93/193) Updating files: 49% (95/193) Updating files: 50% (97/193) Updating files: 51% (99/193) Updating files: 52% (101/193) Updating files: 53% (103/193) Updating files: 54% (105/193) Updating files: 55% (107/193) Updating files: 56% (109/193) Updating files: 57% (111/193) Updating files: 58% (112/193) Updating files: 59% (114/193) Updating files: 60% (116/193) Updating files: 61% (118/193) Updating files: 62% (120/193) Updating files: 63% (122/193) Updating files: 64% (124/193) Updating files: 65% (126/193) Updating files: 66% (128/193) Updating files: 67% (130/193) Updating files: 68% (132/193) Updating files: 69% (134/193) Updating files: 70% (136/193) Updating files: 71% (138/193) Updating files: 72% (139/193) Updating files: 73% (141/193) Updating files: 74% (143/193) Updating files: 75% (145/193) Updating files: 76% (147/193) Updating files: 77% (149/193) Updating files: 78% (151/193) Updating files: 79% (153/193) Updating files: 80% (155/193) Updating files: 81% (157/193) Updating files: 82% (159/193) Updating files: 83% (161/193) Updating files: 84% (163/193) Updating files: 85% (165/193) Updating files: 86% (166/193) Updating files: 87% (168/193) Updating files: 88% (170/193) Updating files: 89% (172/193) Updating files: 90% (174/193) Updating files: 91% (176/193) Updating files: 92% (178/193) Updating files: 93% (180/193) Updating files: 94% (182/193) Updating files: 95% (184/193) Updating files: 96% (186/193) Updating files: 97% (188/193) Updating files: 98% (190/193) Updating files: 99% (192/193) Updating files: 100% (193/193) Updating files: 100% (193/193), done. 2024-03-09T09:06:01.884 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 41481853924 kclient: strip out kernel version compatibility cruft 2024-03-09T09:06:01.899 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.9 2024-03-09T09:06:02.188 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:06:02.188 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 10 -eq 48 ']' 2024-03-09T09:06:02.188 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.10 2024-03-09T09:06:02.189 INFO:tasks.workunit.client.1.smithi134.stdout:v0.10 2024-03-09T09:06:02.189 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.10 2024-03-09T09:06:02.189 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.10 2024-03-09T09:06:02.324 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:06:02.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.521+0000 7fdf2b200640 1 Processor -- start 2024-03-09T09:06:02.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.521+0000 7fdf2b200640 1 -- start start 2024-03-09T09:06:02.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.522+0000 7fdf2b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf2c153210 0x7fdf2c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:02.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.522+0000 7fdf2b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdf2c156600 0x7fdf2c06ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:02.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.522+0000 7fdf2b200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c1535f0 con 0x7fdf2c06f390 2024-03-09T09:06:02.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.522+0000 7fdf2b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c06f9b0 con 0x7fdf2c156600 2024-03-09T09:06:02.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.522+0000 7fdf2b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c06fc90 con 0x7fdf2c153210 2024-03-09T09:06:02.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.522+0000 7fdf2a800640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7fdf2c06f390 legacy=0x7fdf2c1591a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:06:02.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.523+0000 7fdf29400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdf2c156600 0x7fdf2c06ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:02.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.523+0000 7fdf29400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdf2c156600 0x7fdf2c06ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58680/0 (socket says 172.21.15.67:58680) 2024-03-09T09:06:02.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.523+0000 7fdf29400640 1 -- 172.21.15.67:0/4268963464 learned_addr learned my addr 172.21.15.67:0/4268963464 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:02.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.523+0000 7fdf29e00640 1 --2- 172.21.15.67:0/4268963464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf2c153210 0x7fdf2c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:02.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.523+0000 7fdf29400640 1 -- 172.21.15.67:0/4268963464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf2c153210 msgr2=0x7fdf2c06e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:02.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.523+0000 7fdf29400640 1 --2- 172.21.15.67:0/4268963464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf2c153210 0x7fdf2c06e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:02.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.523+0000 7fdf29400640 1 -- 172.21.15.67:0/4268963464 >> v1:172.21.15.134:6789/0 conn(0x7fdf2c06f390 legacy=0x7fdf2c1591a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:06:02.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.523+0000 7fdf29400640 1 -- 172.21.15.67:0/4268963464 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c06ff70 con 0x7fdf2c156600 2024-03-09T09:06:02.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.525+0000 7fdf29400640 1 --2- 172.21.15.67:0/4268963464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf2c156600 0x7fdf2c06ee50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009870 tx=0x7fdf1802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8443d82df33e5a83 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:02.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.525+0000 7fdf28a00640 1 -- 172.21.15.67:0/4268963464 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf180317c0 con 0x7fdf2c156600 2024-03-09T09:06:02.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.525+0000 7fdf28a00640 1 -- 172.21.15.67:0/4268963464 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf18037070 con 0x7fdf2c156600 2024-03-09T09:06:02.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.525+0000 7fdf28a00640 1 -- 172.21.15.67:0/4268963464 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18031b80 con 0x7fdf2c156600 2024-03-09T09:06:02.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.526+0000 7fdf2b200640 1 -- 172.21.15.67:0/4268963464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf2c156600 msgr2=0x7fdf2c06ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:02.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.526+0000 7fdf2b200640 1 --2- 172.21.15.67:0/4268963464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf2c156600 0x7fdf2c06ee50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009870 tx=0x7fdf1802b860 comp rx=0 tx=0).stop 2024-03-09T09:06:02.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.526+0000 7fdf2b200640 1 -- 172.21.15.67:0/4268963464 shutdown_connections 2024-03-09T09:06:02.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.526+0000 7fdf2b200640 1 --2- 172.21.15.67:0/4268963464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf2c156600 0x7fdf2c06ee50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:02.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.526+0000 7fdf2b200640 1 --2- 172.21.15.67:0/4268963464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf2c153210 0x7fdf2c06e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:02.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.526+0000 7fdf2b200640 1 -- 172.21.15.67:0/4268963464 >> 172.21.15.67:0/4268963464 conn(0x7fdf2c0c8310 msgr2=0x7fdf2c0c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:02.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.526+0000 7fdf2b200640 1 -- 172.21.15.67:0/4268963464 shutdown_connections 2024-03-09T09:06:02.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.526+0000 7fdf2b200640 1 -- 172.21.15.67:0/4268963464 wait complete. 2024-03-09T09:06:02.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.528+0000 7fdf2b200640 1 Processor -- start 2024-03-09T09:06:02.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.528+0000 7fdf2b200640 1 -- start start 2024-03-09T09:06:02.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.529+0000 7fdf2b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf2c06f390 0x7fdf2c0aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.529+0000 7fdf2b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf2c153210 0x7fdf2c1eed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.529+0000 7fdf2b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf2c156600 0x7fdf2c1ef290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.529+0000 7fdf2b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0c1f60 con 0x7fdf2c153210 2024-03-09T09:06:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.529+0000 7fdf2b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0c1de0 con 0x7fdf2c06f390 2024-03-09T09:06:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.529+0000 7fdf2b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c1598b0 con 0x7fdf2c156600 2024-03-09T09:06:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.529+0000 7fdf29e00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf2c06f390 msgr2=0x7fdf2c0aca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:06:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.529+0000 7fdf29e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf2c06f390 0x7fdf2c0aca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.529+0000 7fdf29400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf2c153210 0x7fdf2c1eed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.530+0000 7fdf29400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf2c153210 0x7fdf2c1eed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58688/0 (socket says 172.21.15.67:58688) 2024-03-09T09:06:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.530+0000 7fdf29400640 1 -- 172.21.15.67:0/3408660570 learned_addr learned my addr 172.21.15.67:0/3408660570 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:02.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.530+0000 7fdf2a800640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf2c156600 0x7fdf2c1ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:02.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.530+0000 7fdf29400640 1 -- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf2c156600 msgr2=0x7fdf2c1ef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:02.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.530+0000 7fdf29400640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf2c156600 0x7fdf2c1ef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.530+0000 7fdf29400640 1 -- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf2c06f390 msgr2=0x7fdf2c0aca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:06:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.530+0000 7fdf29400640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf2c06f390 0x7fdf2c0aca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.530+0000 7fdf29400640 1 -- 172.21.15.67:0/3408660570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c073940 con 0x7fdf2c153210 2024-03-09T09:06:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.530+0000 7fdf2a800640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf2c156600 0x7fdf2c1ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:06:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.531+0000 7fdf29400640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf2c153210 0x7fdf2c1eed50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009870 tx=0x7fdf180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.531+0000 7fdf22000640 1 -- 172.21.15.67:0/3408660570 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18032650 con 0x7fdf2c153210 2024-03-09T09:06:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.531+0000 7fdf22000640 1 -- 172.21.15.67:0/3408660570 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf180327f0 con 0x7fdf2c153210 2024-03-09T09:06:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.531+0000 7fdf2b200640 1 -- 172.21.15.67:0/3408660570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c0730b0 con 0x7fdf2c153210 2024-03-09T09:06:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.531+0000 7fdf22000640 1 -- 172.21.15.67:0/3408660570 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18032be0 con 0x7fdf2c153210 2024-03-09T09:06:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.531+0000 7fdf2b200640 1 -- 172.21.15.67:0/3408660570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c0acf50 con 0x7fdf2c153210 2024-03-09T09:06:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.532+0000 7fdf2b200640 1 -- 172.21.15.67:0/3408660570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c04f3c0 con 0x7fdf2c153210 2024-03-09T09:06:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.534+0000 7fdf22000640 1 -- 172.21.15.67:0/3408660570 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdf1804c060 con 0x7fdf2c153210 2024-03-09T09:06:02.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.535+0000 7fdf22000640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf0c06b480 0x7fdf0c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:02.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.535+0000 7fdf22000640 1 -- 172.21.15.67:0/3408660570 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdf180abd90 con 0x7fdf2c153210 2024-03-09T09:06:02.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.536+0000 7fdf29e00640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf0c06b480 0x7fdf0c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:02.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.537+0000 7fdf29e00640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf0c06b480 0x7fdf0c06d940 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c1592b0 tx=0x7fdf1c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:02.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.540+0000 7fdf22000640 1 -- 172.21.15.67:0/3408660570 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdf18040070 con 0x7fdf2c153210 2024-03-09T09:06:02.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.886+0000 7fdf2b200640 1 -- 172.21.15.67:0/3408660570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdf2c0c1900 con 0x7fdf2c153210 2024-03-09T09:06:02.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.887+0000 7fdf22000640 1 -- 172.21.15.67:0/3408660570 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fdf1807a890 con 0x7fdf2c153210 2024-03-09T09:06:02.891 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:06:02.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 -- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf0c06b480 msgr2=0x7fdf0c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:02.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf0c06b480 0x7fdf0c06d940 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c1592b0 tx=0x7fdf1c009190 comp rx=0 tx=0).stop 2024-03-09T09:06:02.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 -- 172.21.15.67:0/3408660570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf2c153210 msgr2=0x7fdf2c1eed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:02.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf2c153210 0x7fdf2c1eed50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009870 tx=0x7fdf180046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:02.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 -- 172.21.15.67:0/3408660570 shutdown_connections 2024-03-09T09:06:02.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf0c06b480 0x7fdf0c06d940 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:02.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf2c156600 0x7fdf2c1ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:02.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf2c153210 0x7fdf2c1eed50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:02.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 --2- 172.21.15.67:0/3408660570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf2c06f390 0x7fdf2c0aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:02.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 -- 172.21.15.67:0/3408660570 >> 172.21.15.67:0/3408660570 conn(0x7fdf2c0c8310 msgr2=0x7fdf2c0c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:02.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 -- 172.21.15.67:0/3408660570 shutdown_connections 2024-03-09T09:06:02.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:02.890+0000 7fdf2b200640 1 -- 172.21.15.67:0/3408660570 wait complete. 2024-03-09T09:06:03.226 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 84% (164/193) Updating files: 85% (165/193) Updating files: 86% (166/193) Updating files: 87% (168/193) Updating files: 88% (170/193) Updating files: 89% (172/193) Updating files: 90% (174/193) Updating files: 91% (176/193) Updating files: 92% (178/193) Updating files: 93% (180/193) Updating files: 94% (182/193) Updating files: 95% (184/193) Updating files: 96% (186/193) Updating files: 97% (188/193) Updating files: 98% (190/193) Updating files: 99% (192/193) Updating files: 100% (193/193) Updating files: 100% (193/193), done. 2024-03-09T09:06:03.526 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 41481853924 kclient: strip out kernel version compatibility cruft 2024-03-09T09:06:03.532 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.9 2024-03-09T09:06:03.584 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:03.584 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 10 -eq 48 ']' 2024-03-09T09:06:03.584 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.10 2024-03-09T09:06:03.584 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.10 2024-03-09T09:06:03.585 INFO:tasks.workunit.client.0.smithi067.stdout:v0.10 2024-03-09T09:06:03.585 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.10 2024-03-09T09:06:05.913 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:06:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.156+0000 7f1ad2c00640 1 Processor -- start 2024-03-09T09:06:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.156+0000 7f1ad2c00640 1 -- start start 2024-03-09T09:06:06.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.157+0000 7f1ad2c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:06.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.157+0000 7f1ad2c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1acc155cf0 con 0x7f1acc0c5770 2024-03-09T09:06:06.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.157+0000 7f1ad2c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1acc155fd0 con 0x7f1acc0c1820 2024-03-09T09:06:06.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.157+0000 7f1ad2c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc1562b0 con 0x7f1acc0c8280 2024-03-09T09:06:06.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.158+0000 7f1ad2200640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f1acc0c1820 legacy=0x7f1acc0c1cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:06:06.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.158+0000 7f1ad1800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1acc0c5770 0x7f1acc06b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41690/0 (socket says 172.21.15.67:41690) 2024-03-09T09:06:06.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.158+0000 7f1ad1800640 1 -- 172.21.15.67:0/1596372429 learned_addr learned my addr 172.21.15.67:0/1596372429 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:06:06.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.158+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:06.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.159+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 msgr2=0x7f1acc0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:06:06.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.159+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 msgr2=0x7f1acc0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:06:06.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.159+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:06:06.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.159+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:06.178 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 45% (75/164) Updating files: 46% (76/164) Updating files: 47% (78/164) Updating files: 48% (79/164) Updating files: 49% (81/164) Updating files: 50% (82/164) Updating files: 51% (84/164) Updating files: 52% (86/164) Updating files: 53% (87/164) Updating files: 54% (89/164) Updating files: 55% (91/164) Updating files: 56% (92/164) Updating files: 57% (94/164) Updating files: 58% (96/164) Updating files: 59% (97/164) Updating files: 60% (99/164) Updating files: 61% (101/164) Updating files: 62% (102/164) Updating files: 63% (104/164) Updating files: 64% (105/164) Updating files: 65% (107/164) Updating files: 66% (109/164) Updating files: 67% (110/164) Updating files: 68% (112/164) Updating files: 69% (114/164) Updating files: 70% (115/164) Updating files: 71% (117/164) Updating files: 72% (119/164) Updating files: 73% (120/164) Updating files: 74% (122/164) Updating files: 75% (123/164) Updating files: 76% (125/164) Updating files: 77% (127/164) Updating files: 78% (128/164) Updating files: 79% (130/164) Updating files: 80% (132/164) Updating files: 81% (133/164) Updating files: 82% (135/164) Updating files: 83% (137/164) Updating files: 84% (138/164) Updating files: 85% (140/164) Updating files: 86% (142/164) Updating files: 87% (143/164) Updating files: 88% (145/164) Updating files: 89% (146/164) Updating files: 90% (148/164) Updating files: 91% (150/164) Updating files: 92% (151/164) Updating files: 93% (153/164) Updating files: 94% (155/164) Updating files: 95% (156/164) Updating files: 96% (158/164) Updating files: 97% (160/164) Updating files: 98% (161/164) Updating files: 99% (163/164) Updating files: 100% (164/164) Updating files: 100% (164/164), done. 2024-03-09T09:06:06.214 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 007313fe844 v0.10 2024-03-09T09:06:06.220 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.10 2024-03-09T09:06:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.359+0000 7f1ad2200640 1 -- 172.21.15.67:0/1596372429 >> v1:172.21.15.134:6789/0 conn(0x7f1acc0c1820 legacy=0x7f1acc0c1cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:06:06.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.360+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:06.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.361+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 msgr2=0x7f1acc0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:06:06.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.361+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 msgr2=0x7f1acc0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:06:06.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.361+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:06:06.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.361+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:06:06.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.760+0000 7f1ad2200640 1 -- 172.21.15.67:0/1596372429 >> v1:172.21.15.134:6789/0 conn(0x7f1acc0c1820 legacy=0x7f1acc0c1cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:06:06.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.762+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:06.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.763+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 msgr2=0x7f1acc0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:06:06.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.763+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 msgr2=0x7f1acc0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:06:06.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.763+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:06:06.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:06.763+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:06:06.901 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:06.901 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 11 -eq 48 ']' 2024-03-09T09:06:06.901 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.11 2024-03-09T09:06:06.901 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.11 2024-03-09T09:06:06.902 INFO:tasks.workunit.client.0.smithi067.stdout:v0.11 2024-03-09T09:06:06.902 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.11 2024-03-09T09:06:07.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:07.561+0000 7f1ad2200640 1 -- 172.21.15.67:0/1596372429 >> v1:172.21.15.134:6789/0 conn(0x7f1acc0c1820 legacy=0x7f1acc0c1cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:06:07.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:07.564+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:07.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:07.565+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 msgr2=0x7f1acc0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:06:07.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:07.565+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 msgr2=0x7f1acc0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:06:07.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:07.565+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:06:07.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:07.565+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:06:07.887 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 9fa6f5e7d09 v0.11 2024-03-09T09:06:07.890 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.11 2024-03-09T09:06:08.005 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:08.005 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 12 -eq 48 ']' 2024-03-09T09:06:08.005 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.12 2024-03-09T09:06:08.005 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.12 2024-03-09T09:06:08.006 INFO:tasks.workunit.client.0.smithi067.stdout:v0.12 2024-03-09T09:06:08.006 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.12 2024-03-09T09:06:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.157+0000 7f1acaa00640 1 -- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 msgr2=0x7f1acc0c86e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:06:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.157+0000 7f1acaa00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc0c8280 0x7f1acc0c86e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.158+0000 7f1acaa00640 1 -- 172.21.15.67:0/1596372429 >> v1:172.21.15.134:6789/0 conn(0x7f1acc0c1820 legacy=0x7f1acc0c1cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:06:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.158+0000 7f1acaa00640 1 -- 172.21.15.67:0/1596372429 >> v1:172.21.15.67:6789/0 conn(0x7f1acc0c5770 legacy=0x7f1acc06b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.158+0000 7f1acaa00640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.134:3300/0 conn(0x7f1aac0036d0 0x7f1aac005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.158+0000 7f1acaa00640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 0x7f1aac008660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.158+0000 7f1acaa00640 1 -- 172.21.15.67:0/1596372429 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1acc1562b0 con 0x7f1aac0061a0 2024-03-09T09:06:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.158+0000 7f1acaa00640 1 -- 172.21.15.67:0/1596372429 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aac008d10 con 0x7f1aac0036d0 2024-03-09T09:06:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.158+0000 7f1ad1800640 1 -- 172.21.15.67:0/1596372429 >> v2:172.21.15.134:3300/0 conn(0x7f1aac0036d0 msgr2=0x7f1aac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:06:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.158+0000 7f1ad1800640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.134:3300/0 conn(0x7f1aac0036d0 0x7f1aac005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.158+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 0x7f1aac008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.159+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 msgr2=0x7f1aac008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:06:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.159+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 msgr2=0x7f1aac008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:06:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.159+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 0x7f1aac008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:06:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.159+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 0x7f1aac008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:09.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.359+0000 7f1ad1800640 1 -- 172.21.15.67:0/1596372429 >> v2:172.21.15.134:3300/0 conn(0x7f1aac0036d0 msgr2=0x7f1aac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:06:09.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.359+0000 7f1ad1800640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.134:3300/0 conn(0x7f1aac0036d0 0x7f1aac005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:06:09.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.360+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 0x7f1aac008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:09.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.361+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 msgr2=0x7f1aac008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:06:09.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.361+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 msgr2=0x7f1aac008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:06:09.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.361+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 0x7f1aac008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:06:09.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.361+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 0x7f1aac008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:06:09.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.760+0000 7f1ad1800640 1 -- 172.21.15.67:0/1596372429 >> v2:172.21.15.134:3300/0 conn(0x7f1aac0036d0 msgr2=0x7f1aac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:06:09.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.760+0000 7f1ad1800640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.134:3300/0 conn(0x7f1aac0036d0 0x7f1aac005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:06:09.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.762+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.67:3300/0 conn(0x7f1aac0061a0 0x7f1aac008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.763+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 >> v2:172.21.15.134:3300/0 conn(0x7f1aac0036d0 msgr2=0x7f1aac005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:06:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.763+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> v2:172.21.15.134:3300/0 conn(0x7f1aac0036d0 0x7f1aac005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.763+0000 7f1ad0e00640 1 -- 172.21.15.67:0/1596372429 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc156590 con 0x7f1aac0061a0 2024-03-09T09:06:09.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.766+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aac0061a0 0x7f1aac008660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1aac003680 tx=0x7f1ab802d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190abd6fbc0f468 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:09.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.769+0000 7f1acbe00640 1 -- 172.21.15.67:0/1596372429 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8033070 con 0x7f1aac0061a0 2024-03-09T09:06:09.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.769+0000 7f1acbe00640 1 -- 172.21.15.67:0/1596372429 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab8031070 con 0x7f1aac0061a0 2024-03-09T09:06:09.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.769+0000 7f1acbe00640 1 -- 172.21.15.67:0/1596372429 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8038460 con 0x7f1aac0061a0 2024-03-09T09:06:09.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.770+0000 7f1ad2c00640 1 -- 172.21.15.67:0/1596372429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aac0061a0 msgr2=0x7f1aac008660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:09.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.770+0000 7f1ad2c00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aac0061a0 0x7f1aac008660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1aac003680 tx=0x7f1ab802d330 comp rx=0 tx=0).stop 2024-03-09T09:06:09.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.771+0000 7f1ad2200640 1 -- 172.21.15.67:0/1596372429 reap_dead start 2024-03-09T09:06:09.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.771+0000 7f1ad2c00640 1 -- 172.21.15.67:0/1596372429 shutdown_connections 2024-03-09T09:06:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.771+0000 7f1ad2c00640 1 --2- 172.21.15.67:0/1596372429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aac0061a0 0x7f1aac008660 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.771+0000 7f1ad2c00640 1 -- 172.21.15.67:0/1596372429 >> 172.21.15.67:0/1596372429 conn(0x7f1acc06d650 msgr2=0x7f1acc06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.771+0000 7f1ad2c00640 1 -- 172.21.15.67:0/1596372429 shutdown_connections 2024-03-09T09:06:09.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.771+0000 7f1ad2c00640 1 -- 172.21.15.67:0/1596372429 wait complete. 2024-03-09T09:06:09.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.775+0000 7f1ad2c00640 1 Processor -- start 2024-03-09T09:06:09.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.776+0000 7f1ad2c00640 1 -- start start 2024-03-09T09:06:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.778+0000 7f1ad2c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aac0061a0 0x7f1acc150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.778+0000 7f1ad2c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1acc156f40 0x7f1acc158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.778+0000 7f1ad2c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc149220 0x7f1acc1ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.778+0000 7f1ad2c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0c23c0 con 0x7f1acc156f40 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.778+0000 7f1ad2c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0c5b50 con 0x7f1aac0061a0 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.778+0000 7f1ad2c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0ad960 con 0x7f1acc149220 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.778+0000 7f1ad1800640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aac0061a0 msgr2=0x7f1acc150700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.778+0000 7f1ad1800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aac0061a0 0x7f1acc150700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.778+0000 7f1ad0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1acc156f40 0x7f1acc158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.778+0000 7f1ad0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1acc156f40 0x7f1acc158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56532/0 (socket says 172.21.15.67:56532) 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.778+0000 7f1ad0e00640 1 -- 172.21.15.67:0/624910954 learned_addr learned my addr 172.21.15.67:0/624910954 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.779+0000 7f1ad2200640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc149220 0x7f1acc1ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.779+0000 7f1ad0e00640 1 -- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc149220 msgr2=0x7f1acc1ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.779+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc149220 0x7f1acc1ee7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.779+0000 7f1ad0e00640 1 -- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aac0061a0 msgr2=0x7f1acc150700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:06:09.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.779+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aac0061a0 0x7f1acc150700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:09.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.779+0000 7f1ad0e00640 1 -- 172.21.15.67:0/624910954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc148f40 con 0x7f1acc156f40 2024-03-09T09:06:09.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.779+0000 7f1ad0e00640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1acc156f40 0x7f1acc158630 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1ab802d940 tx=0x7f1ab80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.780+0000 7f1ac9600640 1 -- 172.21.15.67:0/624910954 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8004020 con 0x7f1acc156f40 2024-03-09T09:06:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.780+0000 7f1ac9600640 1 -- 172.21.15.67:0/624910954 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab80041c0 con 0x7f1acc156f40 2024-03-09T09:06:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.780+0000 7f1ad2c00640 1 -- 172.21.15.67:0/624910954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1acc1ef170 con 0x7f1acc156f40 2024-03-09T09:06:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.780+0000 7f1ac9600640 1 -- 172.21.15.67:0/624910954 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8003610 con 0x7f1acc156f40 2024-03-09T09:06:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.780+0000 7f1ad2c00640 1 -- 172.21.15.67:0/624910954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc149840 con 0x7f1acc156f40 2024-03-09T09:06:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.781+0000 7f1ad2c00640 1 -- 172.21.15.67:0/624910954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc04f3c0 con 0x7f1acc156f40 2024-03-09T09:06:09.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.783+0000 7f1ac9600640 1 -- 172.21.15.67:0/624910954 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1ab80037b0 con 0x7f1acc156f40 2024-03-09T09:06:09.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.784+0000 7f1ac9600640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1aac06c160 0x7f1aac06e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:09.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.784+0000 7f1ac9600640 1 -- 172.21.15.67:0/624910954 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1ab80add60 con 0x7f1acc156f40 2024-03-09T09:06:09.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.785+0000 7f1ad1800640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1aac06c160 0x7f1aac06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:09.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.786+0000 7f1ad1800640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1aac06c160 0x7f1aac06e620 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005880 tx=0x7f1ac0007a10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:09.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:09.792+0000 7f1ac9600640 1 -- 172.21.15.67:0/624910954 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1ab807c860 con 0x7f1acc156f40 2024-03-09T09:06:09.811 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 15% (26/164) Updating files: 16% (27/164) Updating files: 17% (28/164) Updating files: 18% (30/164) Updating files: 19% (32/164) Updating files: 20% (33/164) Updating files: 21% (35/164) Updating files: 22% (37/164) Updating files: 23% (38/164) Updating files: 24% (40/164) Updating files: 25% (41/164) Updating files: 26% (43/164) Updating files: 27% (45/164) Updating files: 28% (46/164) Updating files: 29% (48/164) Updating files: 30% (50/164) Updating files: 31% (51/164) Updating files: 32% (53/164) Updating files: 33% (55/164) Updating files: 34% (56/164) Updating files: 35% (58/164) Updating files: 36% (60/164) Updating files: 37% (61/164) Updating files: 38% (63/164) Updating files: 39% (64/164) Updating files: 40% (66/164) Updating files: 41% (68/164) Updating files: 42% (69/164) Updating files: 43% (71/164) Updating files: 44% (73/164) Updating files: 45% (74/164) Updating files: 46% (76/164) Updating files: 47% (78/164) Updating files: 48% (79/164) Updating files: 49% (81/164) Updating files: 50% (82/164) Updating files: 51% (84/164) Updating files: 52% (86/164) Updating files: 53% (87/164) Updating files: 54% (89/164) Updating files: 55% (91/164) Updating files: 56% (92/164) Updating files: 57% (94/164) Updating files: 58% (96/164) Updating files: 59% (97/164) Updating files: 60% (99/164) Updating files: 61% (101/164) Updating files: 62% (102/164) Updating files: 63% (104/164) Updating files: 64% (105/164) Updating files: 65% (107/164) Updating files: 66% (109/164) Updating files: 67% (110/164) Updating files: 68% (112/164) Updating files: 69% (114/164) Updating files: 70% (115/164) Updating files: 71% (117/164) Updating files: 72% (119/164) Updating files: 73% (120/164) Updating files: 74% (122/164) Updating files: 75% (123/164) Updating files: 76% (125/164) Updating files: 77% (127/164) Updating files: 78% (128/164) Updating files: 79% (130/164) Updating files: 80% (132/164) Updating files: 81% (133/164) Updating files: 82% (135/164) Updating files: 83% (137/164) Updating files: 84% (138/164) Updating files: 85% (140/164) Updating files: 86% (142/164) Updating files: 87% (143/164) Updating files: 88% (145/164) Updating files: 89% (146/164) Updating files: 90% (148/164) Updating files: 91% (150/164) Updating files: 92% (151/164) Updating files: 93% (153/164) Updating files: 94% (155/164) Updating files: 95% (156/164) Updating files: 96% (158/164) Updating files: 97% (160/164) Updating files: 98% (161/164) Updating files: 99% (163/164) Updating files: 100% (164/164) Updating files: 100% (164/164), done. 2024-03-09T09:06:09.905 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 007313fe844 v0.10 2024-03-09T09:06:09.922 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.10 2024-03-09T09:06:09.979 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:06:09.979 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 11 -eq 48 ']' 2024-03-09T09:06:09.980 INFO:tasks.workunit.client.1.smithi134.stdout:v0.11 2024-03-09T09:06:09.980 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.11 2024-03-09T09:06:09.980 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.11 2024-03-09T09:06:09.981 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.11 2024-03-09T09:06:10.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.088+0000 7f1ad2c00640 1 -- 172.21.15.67:0/624910954 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1acc0c79f0 con 0x7f1acc156f40 2024-03-09T09:06:10.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.089+0000 7f1ac9600640 1 -- 172.21.15.67:0/624910954 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f1ab807c160 con 0x7f1acc156f40 2024-03-09T09:06:10.092 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":102,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:06:10.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.091+0000 7f1ad2c00640 1 -- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1aac06c160 msgr2=0x7f1aac06e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.091+0000 7f1ad2c00640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1aac06c160 0x7f1aac06e620 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005880 tx=0x7f1ac0007a10 comp rx=0 tx=0).stop 2024-03-09T09:06:10.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.091+0000 7f1ad2c00640 1 -- 172.21.15.67:0/624910954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1acc156f40 msgr2=0x7f1acc158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.091+0000 7f1ad2c00640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1acc156f40 0x7f1acc158630 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1ab802d940 tx=0x7f1ab80046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.091+0000 7f1ad2c00640 1 -- 172.21.15.67:0/624910954 shutdown_connections 2024-03-09T09:06:10.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.091+0000 7f1ad2c00640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1aac06c160 0x7f1aac06e620 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.091+0000 7f1ad2c00640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1acc149220 0x7f1acc1ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.091+0000 7f1ad2c00640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1acc156f40 0x7f1acc158630 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.091+0000 7f1ad2c00640 1 --2- 172.21.15.67:0/624910954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aac0061a0 0x7f1acc150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.091+0000 7f1ad2c00640 1 -- 172.21.15.67:0/624910954 >> 172.21.15.67:0/624910954 conn(0x7f1acc06d650 msgr2=0x7f1acc06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:10.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.092+0000 7f1ad2c00640 1 -- 172.21.15.67:0/624910954 shutdown_connections 2024-03-09T09:06:10.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.092+0000 7f1ad2c00640 1 -- 172.21.15.67:0/624910954 wait complete. 2024-03-09T09:06:10.113 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:06:10.113 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:06:10.113 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:06:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.384+0000 7f9f28200640 1 Processor -- start 2024-03-09T09:06:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.384+0000 7f9f28200640 1 -- start start 2024-03-09T09:06:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.385+0000 7f9f28200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9f20153560 0x7f9f2006b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.386+0000 7f9f28200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f200c6a70 con 0x7f9f200c5f50 2024-03-09T09:06:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.386+0000 7f9f28200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2006cf30 con 0x7f9f200c96f0 2024-03-09T09:06:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.386+0000 7f9f28200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f20159d80 con 0x7f9f20153560 2024-03-09T09:06:10.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.386+0000 7f9f24e00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f9f200c96f0 legacy=0x7f9f200cac40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:06:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.386+0000 7f9f26200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9f200c5f50 0x7f9f200c6360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58628/0 (socket says 172.21.15.67:58628) 2024-03-09T09:06:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.386+0000 7f9f26200640 1 -- 172.21.15.67:0/3364779489 learned_addr learned my addr 172.21.15.67:0/3364779489 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:06:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.386+0000 7f9f25800640 1 --2- 172.21.15.67:0/3364779489 >> v2:172.21.15.67:3300/0 conn(0x7f9f20153560 0x7f9f2006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.387+0000 7f9f1fe00640 1 -- 172.21.15.67:0/3364779489 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6763230 0 0) 0x7f9f18001490 con 0x7f9f200c5f50 2024-03-09T09:06:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.387+0000 7f9f1fe00640 1 -- 172.21.15.67:0/3364779489 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f08003520 con 0x7f9f200c5f50 2024-03-09T09:06:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.387+0000 7f9f25800640 1 -- 172.21.15.67:0/3364779489 >> v1:172.21.15.134:6789/0 conn(0x7f9f200c96f0 legacy=0x7f9f200cac40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:06:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.387+0000 7f9f25800640 1 -- 172.21.15.67:0/3364779489 >> v1:172.21.15.67:6789/0 conn(0x7f9f200c5f50 legacy=0x7f9f200c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.387+0000 7f9f25800640 1 -- 172.21.15.67:0/3364779489 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2015a060 con 0x7f9f20153560 2024-03-09T09:06:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.389+0000 7f9f25800640 1 --2- 172.21.15.67:0/3364779489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f20153560 0x7f9f2006b040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f2006d100 tx=0x7f9f1402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b205d9527405442 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:10.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.389+0000 7f9f1fe00640 1 -- 172.21.15.67:0/3364779489 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14038070 con 0x7f9f20153560 2024-03-09T09:06:10.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.389+0000 7f9f1fe00640 1 -- 172.21.15.67:0/3364779489 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14031700 con 0x7f9f20153560 2024-03-09T09:06:10.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.389+0000 7f9f1fe00640 1 -- 172.21.15.67:0/3364779489 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14031ad0 con 0x7f9f20153560 2024-03-09T09:06:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.390+0000 7f9f28200640 1 -- 172.21.15.67:0/3364779489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f20153560 msgr2=0x7f9f2006b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.390+0000 7f9f28200640 1 --2- 172.21.15.67:0/3364779489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f20153560 0x7f9f2006b040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f2006d100 tx=0x7f9f1402b7a0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.390+0000 7f9f28200640 1 -- 172.21.15.67:0/3364779489 shutdown_connections 2024-03-09T09:06:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.390+0000 7f9f28200640 1 --2- 172.21.15.67:0/3364779489 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f20153560 0x7f9f2006b040 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.390+0000 7f9f28200640 1 -- 172.21.15.67:0/3364779489 >> 172.21.15.67:0/3364779489 conn(0x7f9f200c8050 msgr2=0x7f9f200c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.390+0000 7f9f28200640 1 -- 172.21.15.67:0/3364779489 shutdown_connections 2024-03-09T09:06:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.390+0000 7f9f28200640 1 -- 172.21.15.67:0/3364779489 wait complete. 2024-03-09T09:06:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.393+0000 7f9f28200640 1 Processor -- start 2024-03-09T09:06:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.393+0000 7f9f28200640 1 -- start start 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.394+0000 7f9f28200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f200c5f50 0x7f9f20148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.394+0000 7f9f28200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f200c96f0 0x7f9f201ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.394+0000 7f9f28200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f20153560 0x7f9f20158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.394+0000 7f9f28200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200c9ff0 con 0x7f9f20153560 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.394+0000 7f9f28200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200ca720 con 0x7f9f200c5f50 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.394+0000 7f9f28200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20161710 con 0x7f9f200c96f0 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.394+0000 7f9f25800640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f200c5f50 msgr2=0x7f9f20148c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.394+0000 7f9f25800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f200c5f50 0x7f9f20148c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.394+0000 7f9f26200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f20153560 0x7f9f20158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.395+0000 7f9f26200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f20153560 0x7f9f20158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56548/0 (socket says 172.21.15.67:56548) 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.395+0000 7f9f26200640 1 -- 172.21.15.67:0/3986248665 learned_addr learned my addr 172.21.15.67:0/3986248665 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:10.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.395+0000 7f9f24e00640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f200c96f0 0x7f9f201ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.395+0000 7f9f26200640 1 -- 172.21.15.67:0/3986248665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f200c96f0 msgr2=0x7f9f201ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.395+0000 7f9f26200640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9f200c96f0 0x7f9f201ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.395+0000 7f9f26200640 1 -- 172.21.15.67:0/3986248665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f200c5f50 msgr2=0x7f9f20148c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:06:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.395+0000 7f9f26200640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f200c5f50 0x7f9f20148c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.395+0000 7f9f26200640 1 -- 172.21.15.67:0/3986248665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f201491b0 con 0x7f9f20153560 2024-03-09T09:06:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.396+0000 7f9f26200640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f20153560 0x7f9f20158a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800c820 tx=0x7f9f1800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.396+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18018070 con 0x7f9f20153560 2024-03-09T09:06:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.396+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1800a640 con 0x7f9f20153560 2024-03-09T09:06:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.396+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800aa70 con 0x7f9f20153560 2024-03-09T09:06:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.396+0000 7f9f28200640 1 -- 172.21.15.67:0/3986248665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f201f0390 con 0x7f9f20153560 2024-03-09T09:06:10.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.396+0000 7f9f28200640 1 -- 172.21.15.67:0/3986248665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f201edac0 con 0x7f9f20153560 2024-03-09T09:06:10.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.397+0000 7f9f28200640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201efef0 0x7f9f201ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.397+0000 7f9f28200640 1 -- 172.21.15.67:0/3986248665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f201ed920 con 0x7f9f201efef0 2024-03-09T09:06:10.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.398+0000 7f9f25800640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201efef0 0x7f9f201ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.398+0000 7f9f25800640 1 -- 172.21.15.67:0/3986248665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f2014dc50 con 0x7f9f201efef0 2024-03-09T09:06:10.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.399+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9f1800ac10 con 0x7f9f20153560 2024-03-09T09:06:10.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.399+0000 7f9f25800640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201efef0 0x7f9f201ee2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f2008d1b0 tx=0x7f9f140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:10.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.400+0000 7f9f1d600640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9f0006b480 0x7f9f0006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.400+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9f18089c20 con 0x7f9f20153560 2024-03-09T09:06:10.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.400+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14043070 con 0x7f9f201efef0 2024-03-09T09:06:10.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.400+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f9f140093c0 con 0x7f9f201efef0 2024-03-09T09:06:10.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.400+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201efef0 msgr2=0x7f9f201ee2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.400+0000 7f9f1d600640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201efef0 0x7f9f201ee2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f2008d1b0 tx=0x7f9f140046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.400+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201efef0 msgr2=0x7f9f201ee2c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:10.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.400+0000 7f9f1d600640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201efef0 0x7f9f201ee2c0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f2008d1b0 tx=0x7f9f140046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.400+0000 7f9f24e00640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9f0006b480 0x7f9f0006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.402+0000 7f9f24e00640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9f0006b480 0x7f9f0006d940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9f200ab630 tx=0x7f9f0c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.408+0000 7f9f28200640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201428e0 0x7f9f201ee800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.408+0000 7f9f28200640 1 -- 172.21.15.67:0/3986248665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200bedf0 con 0x7f9f201428e0 2024-03-09T09:06:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.408+0000 7f9f25800640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201428e0 0x7f9f201ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.408+0000 7f9f25800640 1 -- 172.21.15.67:0/3986248665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9f201f0850 con 0x7f9f201428e0 2024-03-09T09:06:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.409+0000 7f9f25800640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201428e0 0x7f9f201ee800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9f14000c00 tx=0x7f9f14031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.409+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14038040 con 0x7f9f201428e0 2024-03-09T09:06:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.409+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7f9f14052430 con 0x7f9f201428e0 2024-03-09T09:06:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.409+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201428e0 msgr2=0x7f9f201ee800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.409+0000 7f9f1d600640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201428e0 0x7f9f201ee800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9f14000c00 tx=0x7f9f14031000 comp rx=0 tx=0).stop 2024-03-09T09:06:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.409+0000 7f9f1d600640 1 -- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201428e0 msgr2=0x7f9f201ee800 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.409+0000 7f9f1d600640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f201428e0 0x7f9f201ee800 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.413 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:06:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:06:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:06:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:06:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 102, 2024-03-09T09:06:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:06:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:06:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:06:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:06:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:06:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:06:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:06:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:06:10.426 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:06:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.410+0000 7f9f28200640 1 -- 172.21.15.67:0/3986248665 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9f0006b480 msgr2=0x7f9f0006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.410+0000 7f9f28200640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9f0006b480 0x7f9f0006d940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9f200ab630 tx=0x7f9f0c008040 comp rx=0 tx=0).stop 2024-03-09T09:06:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.410+0000 7f9f28200640 1 -- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f20153560 msgr2=0x7f9f20158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.410+0000 7f9f28200640 1 --2- 172.21.15.67:0/3986248665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f20153560 0x7f9f20158a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800c820 tx=0x7f9f1800cce0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.410+0000 7f9f26200640 1 -- 172.21.15.67:0/3986248665 reap_dead start 2024-03-09T09:06:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.410+0000 7f9f26200640 1 -- 172.21.15.67:0/3986248665 reap_dead start 2024-03-09T09:06:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.410+0000 7f9f28200640 1 -- 172.21.15.67:0/3986248665 shutdown_connections 2024-03-09T09:06:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.410+0000 7f9f28200640 1 -- 172.21.15.67:0/3986248665 >> 172.21.15.67:0/3986248665 conn(0x7f9f200c8050 msgr2=0x7f9f2014ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.410+0000 7f9f28200640 1 -- 172.21.15.67:0/3986248665 shutdown_connections 2024-03-09T09:06:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.410+0000 7f9f28200640 1 -- 172.21.15.67:0/3986248665 wait complete. 2024-03-09T09:06:10.431 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:06:10.431 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:06:10.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:06:10.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.652+0000 7f69edc00640 1 Processor -- start 2024-03-09T09:06:10.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.652+0000 7f69edc00640 1 -- start start 2024-03-09T09:06:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.654+0000 7f69edc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80734a0 0x7f69e80aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.654+0000 7f69edc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f69e806d830 0x7f69e806dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.654+0000 7f69edc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e8152e00 con 0x7f69e80730c0 2024-03-09T09:06:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.654+0000 7f69edc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e80aa960 con 0x7f69e806d830 2024-03-09T09:06:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.654+0000 7f69edc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e81530e0 con 0x7f69e80734a0 2024-03-09T09:06:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.655+0000 7f69e6a00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f69e80730c0 legacy=0x7f69e80c2e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:06:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.655+0000 7f69e7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f69e806d830 0x7f69e806dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.655+0000 7f69e7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f69e806d830 0x7f69e806dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56558/0 (socket says 172.21.15.67:56558) 2024-03-09T09:06:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.655+0000 7f69e7400640 1 -- 172.21.15.67:0/3960144555 learned_addr learned my addr 172.21.15.67:0/3960144555 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.655+0000 7f69de000640 1 --2- 172.21.15.67:0/3960144555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80734a0 0x7f69e80aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.655+0000 7f69e7400640 1 -- 172.21.15.67:0/3960144555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80734a0 msgr2=0x7f69e80aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.655+0000 7f69e7400640 1 --2- 172.21.15.67:0/3960144555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80734a0 0x7f69e80aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.655+0000 7f69e7400640 1 -- 172.21.15.67:0/3960144555 >> v1:172.21.15.134:6789/0 conn(0x7f69e80730c0 legacy=0x7f69e80c2e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:06:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.655+0000 7f69e7400640 1 -- 172.21.15.67:0/3960144555 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e81533c0 con 0x7f69e806d830 2024-03-09T09:06:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.656+0000 7f69e7400640 1 --2- 172.21.15.67:0/3960144555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e806d830 0x7f69e806dcc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009e50 tx=0x7f69e002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f83cf10af346a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.657+0000 7f69e6000640 1 -- 172.21.15.67:0/3960144555 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e002c9e0 con 0x7f69e806d830 2024-03-09T09:06:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.657+0000 7f69e6000640 1 -- 172.21.15.67:0/3960144555 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e002cb80 con 0x7f69e806d830 2024-03-09T09:06:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.657+0000 7f69e6000640 1 -- 172.21.15.67:0/3960144555 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e00343f0 con 0x7f69e806d830 2024-03-09T09:06:10.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.657+0000 7f69edc00640 1 -- 172.21.15.67:0/3960144555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e806d830 msgr2=0x7f69e806dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.657+0000 7f69edc00640 1 --2- 172.21.15.67:0/3960144555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e806d830 0x7f69e806dcc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009e50 tx=0x7f69e002be40 comp rx=0 tx=0).stop 2024-03-09T09:06:10.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.657+0000 7f69edc00640 1 -- 172.21.15.67:0/3960144555 shutdown_connections 2024-03-09T09:06:10.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.657+0000 7f69edc00640 1 --2- 172.21.15.67:0/3960144555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e806d830 0x7f69e806dcc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.657+0000 7f69edc00640 1 --2- 172.21.15.67:0/3960144555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80734a0 0x7f69e80aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.658+0000 7f69edc00640 1 -- 172.21.15.67:0/3960144555 >> 172.21.15.67:0/3960144555 conn(0x7f69e80c7370 msgr2=0x7f69e80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:10.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.658+0000 7f69edc00640 1 -- 172.21.15.67:0/3960144555 shutdown_connections 2024-03-09T09:06:10.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.658+0000 7f69edc00640 1 -- 172.21.15.67:0/3960144555 wait complete. 2024-03-09T09:06:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.660+0000 7f69edc00640 1 Processor -- start 2024-03-09T09:06:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.661+0000 7f69edc00640 1 -- start start 2024-03-09T09:06:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.662+0000 7f69edc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e806d830 0x7f69e81eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.662+0000 7f69edc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e80730c0 0x7f69e814a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.662+0000 7f69edc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69e80734a0 0x7f69e814a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.662+0000 7f69edc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80c3590 con 0x7f69e80730c0 2024-03-09T09:06:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.662+0000 7f69edc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e8153980 con 0x7f69e80734a0 2024-03-09T09:06:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.662+0000 7f69edc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80c3710 con 0x7f69e806d830 2024-03-09T09:06:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.662+0000 7f69e7400640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69e80734a0 msgr2=0x7f69e814a640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:06:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.662+0000 7f69de000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e80730c0 0x7f69e814a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.662+0000 7f69e7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69e80734a0 0x7f69e814a640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.663+0000 7f69de000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e80730c0 0x7f69e814a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56560/0 (socket says 172.21.15.67:56560) 2024-03-09T09:06:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.663+0000 7f69de000640 1 -- 172.21.15.67:0/756811069 learned_addr learned my addr 172.21.15.67:0/756811069 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.663+0000 7f69e6a00640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e806d830 0x7f69e81eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.663+0000 7f69de000640 1 -- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e806d830 msgr2=0x7f69e81eccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.663+0000 7f69de000640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e806d830 0x7f69e81eccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.663+0000 7f69de000640 1 -- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69e80734a0 msgr2=0x7f69e814a640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:06:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.663+0000 7f69de000640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69e80734a0 0x7f69e814a640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.663+0000 7f69de000640 1 -- 172.21.15.67:0/756811069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e8156940 con 0x7f69e80730c0 2024-03-09T09:06:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.663+0000 7f69e6a00640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e806d830 0x7f69e81eccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:06:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.663+0000 7f69de000640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e80730c0 0x7f69e814a100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f69d000d5f0 tx=0x7f69d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.664+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d00125f0 con 0x7f69e80730c0 2024-03-09T09:06:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.664+0000 7f69edc00640 1 -- 172.21.15.67:0/756811069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e81ed610 con 0x7f69e80730c0 2024-03-09T09:06:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.664+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d0019070 con 0x7f69e80730c0 2024-03-09T09:06:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.664+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d00129d0 con 0x7f69e80730c0 2024-03-09T09:06:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.664+0000 7f69edc00640 1 -- 172.21.15.67:0/756811069 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e814b2a0 con 0x7f69e80730c0 2024-03-09T09:06:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.665+0000 7f69edc00640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80bc390 0x7f69e80c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.665+0000 7f69edc00640 1 -- 172.21.15.67:0/756811069 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e814ab80 con 0x7f69e80bc390 2024-03-09T09:06:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.666+0000 7f69e6a00640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80bc390 0x7f69e80c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.666+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f69d0012b70 con 0x7f69e80730c0 2024-03-09T09:06:10.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.667+0000 7f69df400640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f69c406b480 0x7f69c406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.667+0000 7f69e6a00640 1 -- 172.21.15.67:0/756811069 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69e8075a00 con 0x7f69e80bc390 2024-03-09T09:06:10.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.667+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f69d008a900 con 0x7f69e80730c0 2024-03-09T09:06:10.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.668+0000 7f69e7400640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f69c406b480 0x7f69c406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.668+0000 7f69e6a00640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80bc390 0x7f69e80c0b60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f69d8003930 tx=0x7f69d800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.668+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8003960 con 0x7f69e80bc390 2024-03-09T09:06:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.669+0000 7f69e7400640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f69c406b480 0x7f69c406d940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f69e0035000 tx=0x7f69e0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.669+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f69d8017070 con 0x7f69e80bc390 2024-03-09T09:06:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.669+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80bc390 msgr2=0x7f69e80c0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.669+0000 7f69df400640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80bc390 0x7f69e80c0b60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f69d8003930 tx=0x7f69d800db90 comp rx=0 tx=0).stop 2024-03-09T09:06:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.669+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80bc390 msgr2=0x7f69e80c0b60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:10.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.669+0000 7f69df400640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80bc390 0x7f69e80c0b60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.676+0000 7f69edc00640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80c10a0 0x7f69e81e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:10.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.676+0000 7f69edc00640 1 -- 172.21.15.67:0/756811069 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e814d010 con 0x7f69e80c10a0 2024-03-09T09:06:10.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.677+0000 7f69e6a00640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80c10a0 0x7f69e81e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:10.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.678+0000 7f69e6a00640 1 -- 172.21.15.67:0/756811069 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f69e8150130 con 0x7f69e80c10a0 2024-03-09T09:06:10.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.679+0000 7f69e6a00640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80c10a0 0x7f69e81e94f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f69d8016000 tx=0x7f69d8005980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.679+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d801cd20 con 0x7f69e80c10a0 2024-03-09T09:06:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4836 (secure 0 0 0) 0x7f69d80050d0 con 0x7f69e80c10a0 2024-03-09T09:06:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80c10a0 msgr2=0x7f69e81e94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69df400640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80c10a0 0x7f69e81e94f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f69d8016000 tx=0x7f69d8005980 comp rx=0 tx=0).stop 2024-03-09T09:06:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69df400640 1 -- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80c10a0 msgr2=0x7f69e81e94f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69df400640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69e80c10a0 0x7f69e81e94f0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f69d8016000 tx=0x7f69d8005980 comp rx=0 tx=0).stop 2024-03-09T09:06:10.683 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 102, 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:06:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:06:10.695 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:06:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69edc00640 1 -- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f69c406b480 msgr2=0x7f69c406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69edc00640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f69c406b480 0x7f69c406d940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f69e0035000 tx=0x7f69e0030000 comp rx=0 tx=0).stop 2024-03-09T09:06:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69edc00640 1 -- 172.21.15.67:0/756811069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e80730c0 msgr2=0x7f69e814a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69edc00640 1 --2- 172.21.15.67:0/756811069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e80730c0 0x7f69e814a100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f69d000d5f0 tx=0x7f69d000dab0 comp rx=0 tx=0).stop 2024-03-09T09:06:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69e7400640 1 -- 172.21.15.67:0/756811069 reap_dead start 2024-03-09T09:06:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69e7400640 1 -- 172.21.15.67:0/756811069 reap_dead start 2024-03-09T09:06:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69edc00640 1 -- 172.21.15.67:0/756811069 shutdown_connections 2024-03-09T09:06:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69edc00640 1 -- 172.21.15.67:0/756811069 >> 172.21.15.67:0/756811069 conn(0x7f69e80c7370 msgr2=0x7f69e80c2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.680+0000 7f69edc00640 1 -- 172.21.15.67:0/756811069 shutdown_connections 2024-03-09T09:06:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:10.681+0000 7f69edc00640 1 -- 172.21.15.67:0/756811069 wait complete. 2024-03-09T09:06:10.706 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:06:10.706 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:06:10.727 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 9fa6f5e7d09 v0.11 2024-03-09T09:06:10.739 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.11 2024-03-09T09:06:11.003 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:06:11.003 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 12 -eq 48 ']' 2024-03-09T09:06:11.003 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.12 2024-03-09T09:06:11.003 INFO:tasks.workunit.client.1.smithi134.stdout:v0.12 2024-03-09T09:06:11.004 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.12 2024-03-09T09:06:11.004 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.12 2024-03-09T09:06:12.518 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at f524150822a v0.12 2024-03-09T09:06:12.525 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.12 2024-03-09T09:06:12.846 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:06:12.846 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 13 -eq 48 ']' 2024-03-09T09:06:12.846 INFO:tasks.workunit.client.1.smithi134.stdout:v0.13 2024-03-09T09:06:12.846 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.13 2024-03-09T09:06:12.846 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.13 2024-03-09T09:06:12.846 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.13 2024-03-09T09:06:14.469 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 29% (22/75) Updating files: 30% (23/75) Updating files: 32% (24/75) Updating files: 33% (25/75) Updating files: 34% (26/75) Updating files: 36% (27/75) Updating files: 37% (28/75) Updating files: 38% (29/75) Updating files: 40% (30/75) Updating files: 41% (31/75) Updating files: 42% (32/75) Updating files: 44% (33/75) Updating files: 45% (34/75) Updating files: 46% (35/75) Updating files: 48% (36/75) Updating files: 49% (37/75) Updating files: 50% (38/75) Updating files: 52% (39/75) Updating files: 53% (40/75) Updating files: 54% (41/75) Updating files: 56% (42/75) Updating files: 57% (43/75) Updating files: 58% (44/75) Updating files: 60% (45/75) Updating files: 61% (46/75) Updating files: 62% (47/75) Updating files: 64% (48/75) Updating files: 65% (49/75) Updating files: 66% (50/75) Updating files: 68% (51/75) Updating files: 69% (52/75) Updating files: 70% (53/75) Updating files: 72% (54/75) Updating files: 73% (55/75) Updating files: 74% (56/75) Updating files: 76% (57/75) Updating files: 77% (58/75) Updating files: 78% (59/75) Updating files: 80% (60/75) Updating files: 81% (61/75) Updating files: 82% (62/75) Updating files: 84% (63/75) Updating files: 85% (64/75) Updating files: 86% (65/75) Updating files: 88% (66/75) Updating files: 89% (67/75) Updating files: 90% (68/75) Updating files: 92% (69/75) Updating files: 93% (70/75) Updating files: 94% (71/75) Updating files: 96% (72/75) Updating files: 97% (73/75) Updating files: 98% (74/75) Updating files: 100% (75/75) Updating files: 100% (75/75), done. 2024-03-09T09:06:14.513 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at f524150822a v0.12 2024-03-09T09:06:14.525 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.12 2024-03-09T09:06:14.652 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:14.653 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 13 -eq 48 ']' 2024-03-09T09:06:14.653 INFO:tasks.workunit.client.0.smithi067.stdout:v0.13 2024-03-09T09:06:14.653 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.13 2024-03-09T09:06:14.653 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.13 2024-03-09T09:06:14.653 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.13 2024-03-09T09:06:14.858 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 75% (101/133) Updating files: 76% (102/133) Updating files: 77% (103/133) Updating files: 78% (104/133) Updating files: 79% (106/133) Updating files: 80% (107/133) Updating files: 81% (108/133) Updating files: 82% (110/133) Updating files: 83% (111/133) Updating files: 84% (112/133) Updating files: 85% (114/133) Updating files: 86% (115/133) Updating files: 87% (116/133) Updating files: 88% (118/133) Updating files: 89% (119/133) Updating files: 90% (120/133) Updating files: 91% (122/133) Updating files: 92% (123/133) Updating files: 93% (124/133) Updating files: 94% (126/133) Updating files: 95% (127/133) Updating files: 96% (128/133) Updating files: 97% (130/133) Updating files: 98% (131/133) Updating files: 99% (132/133) Updating files: 100% (133/133) Updating files: 100% (133/133), done. 2024-03-09T09:06:14.952 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 2b32297267d v0.13 2024-03-09T09:06:14.965 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.13 2024-03-09T09:06:15.099 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:06:15.099 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 14 -eq 48 ']' 2024-03-09T09:06:15.100 INFO:tasks.workunit.client.1.smithi134.stdout:v0.14 2024-03-09T09:06:15.100 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.14 2024-03-09T09:06:15.100 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.14 2024-03-09T09:06:15.100 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.14 2024-03-09T09:06:15.491 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 2b32297267d v0.13 2024-03-09T09:06:15.500 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.13 2024-03-09T09:06:15.511 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:06:15.516 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:06:15.925 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:15.925 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 14 -eq 48 ']' 2024-03-09T09:06:15.925 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.14 2024-03-09T09:06:15.925 INFO:tasks.workunit.client.0.smithi067.stdout:v0.14 2024-03-09T09:06:15.926 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.14 2024-03-09T09:06:15.926 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.14 2024-03-09T09:06:17.481 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 33% (51/151) Updating files: 34% (52/151) Updating files: 35% (53/151) Updating files: 36% (55/151) Updating files: 37% (56/151) Updating files: 38% (58/151) Updating files: 39% (59/151) Updating files: 40% (61/151) Updating files: 41% (62/151) Updating files: 42% (64/151) Updating files: 43% (65/151) Updating files: 44% (67/151) Updating files: 45% (68/151) Updating files: 46% (70/151) Updating files: 47% (71/151) Updating files: 48% (73/151) Updating files: 49% (74/151) Updating files: 50% (76/151) Updating files: 51% (78/151) Updating files: 52% (79/151) Updating files: 53% (81/151) Updating files: 54% (82/151) Updating files: 55% (84/151) Updating files: 56% (85/151) Updating files: 57% (87/151) Updating files: 58% (88/151) Updating files: 59% (90/151) Updating files: 60% (91/151) Updating files: 61% (93/151) Updating files: 62% (94/151) Updating files: 63% (96/151) Updating files: 64% (97/151) Updating files: 65% (99/151) Updating files: 66% (100/151) Updating files: 67% (102/151) Updating files: 68% (103/151) Updating files: 69% (105/151) Updating files: 70% (106/151) Updating files: 71% (108/151) Updating files: 72% (109/151) Updating files: 73% (111/151) Updating files: 74% (112/151) Updating files: 75% (114/151) Updating files: 76% (115/151) Updating files: 77% (117/151) Updating files: 78% (118/151) Updating files: 79% (120/151) Updating files: 80% (121/151) Updating files: 81% (123/151) Updating files: 82% (124/151) Updating files: 83% (126/151) Updating files: 84% (127/151) Updating files: 85% (129/151) Updating files: 86% (130/151) Updating files: 87% (132/151) Updating files: 88% (133/151) Updating files: 89% (135/151) Updating files: 90% (136/151) Updating files: 91% (138/151) Updating files: 92% (139/151) Updating files: 93% (141/151) Updating files: 94% (142/151) Updating files: 95% (144/151) Updating files: 96% (145/151) Updating files: 97% (147/151) Updating files: 98% (148/151) Updating files: 99% (150/151) Updating files: 100% (151/151) Updating files: 100% (151/151), done. 2024-03-09T09:06:17.535 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 1ec1e8d6fb6 v0.14 2024-03-09T09:06:17.548 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.14 2024-03-09T09:06:17.658 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 54% (82/151) Updating files: 55% (84/151) Updating files: 56% (85/151) Updating files: 57% (87/151) Updating files: 58% (88/151) Updating files: 59% (90/151) Updating files: 60% (91/151) Updating files: 61% (93/151) Updating files: 62% (94/151) Updating files: 63% (96/151) Updating files: 64% (97/151) Updating files: 65% (99/151) Updating files: 66% (100/151) Updating files: 67% (102/151) Updating files: 68% (103/151) Updating files: 69% (105/151) Updating files: 70% (106/151) Updating files: 71% (108/151) Updating files: 72% (109/151) Updating files: 73% (111/151) Updating files: 74% (112/151) Updating files: 75% (114/151) Updating files: 76% (115/151) Updating files: 77% (117/151) Updating files: 78% (118/151) Updating files: 79% (120/151) Updating files: 80% (121/151) Updating files: 81% (123/151) Updating files: 82% (124/151) Updating files: 83% (126/151) Updating files: 84% (127/151) Updating files: 85% (129/151) Updating files: 86% (130/151) Updating files: 87% (132/151) Updating files: 88% (133/151) Updating files: 89% (135/151) Updating files: 90% (136/151) Updating files: 91% (138/151) Updating files: 92% (139/151) Updating files: 93% (141/151) Updating files: 94% (142/151) Updating files: 95% (144/151) Updating files: 96% (145/151) Updating files: 97% (147/151) Updating files: 98% (148/151) Updating files: 99% (150/151) Updating files: 100% (151/151) Updating files: 100% (151/151), done. 2024-03-09T09:06:17.721 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 1ec1e8d6fb6 v0.14 2024-03-09T09:06:17.735 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.14 2024-03-09T09:06:17.746 INFO:tasks.workunit.client.1.smithi134.stdout:v0.15 2024-03-09T09:06:17.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:06:17.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 15 -eq 48 ']' 2024-03-09T09:06:17.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.15 2024-03-09T09:06:17.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.15 2024-03-09T09:06:17.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.15 2024-03-09T09:06:17.851 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:17.851 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 15 -eq 48 ']' 2024-03-09T09:06:17.852 INFO:tasks.workunit.client.0.smithi067.stdout:v0.15 2024-03-09T09:06:17.852 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.15 2024-03-09T09:06:17.852 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.15 2024-03-09T09:06:17.852 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.15 2024-03-09T09:06:19.029 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 88% (99/112) Updating files: 89% (100/112) Updating files: 90% (101/112) Updating files: 91% (102/112) Updating files: 92% (104/112) Updating files: 93% (105/112) Updating files: 94% (106/112) Updating files: 95% (107/112) Updating files: 96% (108/112) Updating files: 97% (109/112) Updating files: 98% (110/112) Updating files: 99% (111/112) Updating files: 100% (112/112) Updating files: 100% (112/112), done. 2024-03-09T09:06:19.090 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at ab3c421eb95 v0.15 2024-03-09T09:06:19.099 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.15 2024-03-09T09:06:19.251 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:19.251 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 16 -eq 48 ']' 2024-03-09T09:06:19.252 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.16 2024-03-09T09:06:19.252 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.16 2024-03-09T09:06:19.252 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.16 2024-03-09T09:06:19.252 INFO:tasks.workunit.client.0.smithi067.stdout:v0.16 2024-03-09T09:06:19.527 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 53% (60/112) Updating files: 54% (61/112) Updating files: 55% (62/112) Updating files: 56% (63/112) Updating files: 57% (64/112) Updating files: 58% (65/112) Updating files: 59% (67/112) Updating files: 60% (68/112) Updating files: 61% (69/112) Updating files: 62% (70/112) Updating files: 63% (71/112) Updating files: 64% (72/112) Updating files: 65% (73/112) Updating files: 66% (74/112) Updating files: 67% (76/112) Updating files: 68% (77/112) Updating files: 69% (78/112) Updating files: 70% (79/112) Updating files: 71% (80/112) Updating files: 72% (81/112) Updating files: 73% (82/112) Updating files: 74% (83/112) Updating files: 75% (84/112) Updating files: 76% (86/112) Updating files: 77% (87/112) Updating files: 78% (88/112) Updating files: 79% (89/112) Updating files: 80% (90/112) Updating files: 81% (91/112) Updating files: 82% (92/112) Updating files: 83% (93/112) Updating files: 84% (95/112) Updating files: 85% (96/112) Updating files: 86% (97/112) Updating files: 87% (98/112) Updating files: 88% (99/112) Updating files: 89% (100/112) Updating files: 90% (101/112) Updating files: 91% (102/112) Updating files: 92% (104/112) Updating files: 93% (105/112) Updating files: 94% (106/112) Updating files: 95% (107/112) Updating files: 96% (108/112) Updating files: 97% (109/112) Updating files: 98% (110/112) Updating files: 99% (111/112) Updating files: 100% (112/112) Updating files: 100% (112/112), done. 2024-03-09T09:06:19.590 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at ab3c421eb95 v0.15 2024-03-09T09:06:19.611 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.15 2024-03-09T09:06:19.865 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:06:19.865 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 16 -eq 48 ']' 2024-03-09T09:06:19.865 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.16 2024-03-09T09:06:19.865 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.16 2024-03-09T09:06:19.866 INFO:tasks.workunit.client.1.smithi134.stdout:v0.16 2024-03-09T09:06:19.866 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.16 2024-03-09T09:06:20.602 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 76% (82/107) Updating files: 77% (83/107) Updating files: 78% (84/107) Updating files: 79% (85/107) Updating files: 80% (86/107) Updating files: 81% (87/107) Updating files: 82% (88/107) Updating files: 83% (89/107) Updating files: 84% (90/107) Updating files: 85% (91/107) Updating files: 86% (93/107) Updating files: 87% (94/107) Updating files: 88% (95/107) Updating files: 89% (96/107) Updating files: 90% (97/107) Updating files: 91% (98/107) Updating files: 92% (99/107) Updating files: 93% (100/107) Updating files: 94% (101/107) Updating files: 95% (102/107) Updating files: 96% (103/107) Updating files: 97% (104/107) Updating files: 98% (105/107) Updating files: 99% (106/107) Updating files: 100% (107/107) Updating files: 100% (107/107), done. 2024-03-09T09:06:20.639 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 38dbaa5175c v0.16 2024-03-09T09:06:20.644 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.16 2024-03-09T09:06:20.763 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:20.763 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 17 -eq 48 ']' 2024-03-09T09:06:20.763 INFO:tasks.workunit.client.0.smithi067.stdout:v0.17 2024-03-09T09:06:20.764 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.17 2024-03-09T09:06:20.764 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.17 2024-03-09T09:06:20.764 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.17 2024-03-09T09:06:21.439 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 93% (100/107) Updating files: 94% (101/107) Updating files: 95% (102/107) Updating files: 96% (103/107) Updating files: 97% (104/107) Updating files: 98% (105/107) Updating files: 99% (106/107) Updating files: 100% (107/107) Updating files: 100% (107/107), done. 2024-03-09T09:06:21.511 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 38dbaa5175c v0.16 2024-03-09T09:06:21.523 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.16 2024-03-09T09:06:22.872 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:06:22.872 INFO:tasks.workunit.client.1.smithi134.stdout:v0.17 2024-03-09T09:06:22.872 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 17 -eq 48 ']' 2024-03-09T09:06:22.873 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.17 2024-03-09T09:06:22.873 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.17 2024-03-09T09:06:22.873 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.17 2024-03-09T09:06:30.706 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:06:30.707 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-03-09T09:06:30.707 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-09T09:06:30.707 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-09T09:06:30.722 INFO:tasks.ceph.mon.b:Started 2024-03-09T09:06:30.722 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:06:30.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:06:30.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.967+0000 7fc629600640 1 Processor -- start 2024-03-09T09:06:30.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.967+0000 7fc629600640 1 -- start start 2024-03-09T09:06:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.968+0000 7fc629600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6240c5f50 0x7fc624155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.968+0000 7fc629600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc62406b040 0x7fc62406c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc629600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc6240c96f0 0x7fc6240c6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc629600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62406b5b0 con 0x7fc6240c5f50 2024-03-09T09:06:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc629600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62406b890 con 0x7fc6240c96f0 2024-03-09T09:06:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc629600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62406bb70 con 0x7fc62406b040 2024-03-09T09:06:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc622a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6240c5f50 0x7fc624155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc622a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6240c5f50 0x7fc624155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44354/0 (socket says 172.21.15.67:44354) 2024-03-09T09:06:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc622a00640 1 -- 172.21.15.67:0/3855472176 learned_addr learned my addr 172.21.15.67:0/3855472176 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc61be00640 1 --2- 172.21.15.67:0/3855472176 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc62406b040 0x7fc62406c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc623400640 1 --2- 172.21.15.67:0/3855472176 >> v2:172.21.15.134:3300/0 conn(0x7fc6240c96f0 0x7fc6240c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc622a00640 1 -- 172.21.15.67:0/3855472176 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc62406b040 msgr2=0x7fc62406c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc622a00640 1 --2- 172.21.15.67:0/3855472176 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc62406b040 0x7fc62406c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc622a00640 1 -- 172.21.15.67:0/3855472176 >> v2:172.21.15.134:3300/0 conn(0x7fc6240c96f0 msgr2=0x7fc6240c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.969+0000 7fc622a00640 1 --2- 172.21.15.67:0/3855472176 >> v2:172.21.15.134:3300/0 conn(0x7fc6240c96f0 0x7fc6240c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.970+0000 7fc622a00640 1 -- 172.21.15.67:0/3855472176 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62406be50 con 0x7fc6240c5f50 2024-03-09T09:06:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.971+0000 7fc622a00640 1 --2- 172.21.15.67:0/3855472176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6240c5f50 0x7fc624155e20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6240c57d0 tx=0x7fc61402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b9c49a43514dd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.971+0000 7fc622000640 1 -- 172.21.15.67:0/3855472176 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614035020 con 0x7fc6240c5f50 2024-03-09T09:06:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.971+0000 7fc622000640 1 -- 172.21.15.67:0/3855472176 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc614031870 con 0x7fc6240c5f50 2024-03-09T09:06:30.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.971+0000 7fc622000640 1 -- 172.21.15.67:0/3855472176 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614031c40 con 0x7fc6240c5f50 2024-03-09T09:06:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.972+0000 7fc629600640 1 -- 172.21.15.67:0/3855472176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6240c5f50 msgr2=0x7fc624155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.972+0000 7fc629600640 1 --2- 172.21.15.67:0/3855472176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6240c5f50 0x7fc624155e20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6240c57d0 tx=0x7fc61402b910 comp rx=0 tx=0).stop 2024-03-09T09:06:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.972+0000 7fc629600640 1 -- 172.21.15.67:0/3855472176 shutdown_connections 2024-03-09T09:06:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.972+0000 7fc629600640 1 --2- 172.21.15.67:0/3855472176 >> v2:172.21.15.134:3300/0 conn(0x7fc6240c96f0 0x7fc6240c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.972+0000 7fc629600640 1 --2- 172.21.15.67:0/3855472176 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc62406b040 0x7fc62406c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.972+0000 7fc629600640 1 --2- 172.21.15.67:0/3855472176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6240c5f50 0x7fc624155e20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.972+0000 7fc629600640 1 -- 172.21.15.67:0/3855472176 >> 172.21.15.67:0/3855472176 conn(0x7fc6240c8050 msgr2=0x7fc6240c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.972+0000 7fc629600640 1 -- 172.21.15.67:0/3855472176 shutdown_connections 2024-03-09T09:06:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.972+0000 7fc629600640 1 -- 172.21.15.67:0/3855472176 wait complete. 2024-03-09T09:06:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.974+0000 7fc629600640 1 Processor -- start 2024-03-09T09:06:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.974+0000 7fc629600640 1 -- start start 2024-03-09T09:06:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.975+0000 7fc629600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc62406b040 0x7fc6240c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.975+0000 7fc629600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6240c5f50 0x7fc6241dfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.975+0000 7fc629600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6240c96f0 0x7fc62414b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.975+0000 7fc629600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6240c6870 con 0x7fc62406b040 2024-03-09T09:06:30.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.975+0000 7fc629600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62414cc00 con 0x7fc6240c5f50 2024-03-09T09:06:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.975+0000 7fc629600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6240c69f0 con 0x7fc6240c96f0 2024-03-09T09:06:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.975+0000 7fc622a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc62406b040 0x7fc6240c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.975+0000 7fc622a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc62406b040 0x7fc6240c4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44360/0 (socket says 172.21.15.67:44360) 2024-03-09T09:06:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.975+0000 7fc622a00640 1 -- 172.21.15.67:0/3732422291 learned_addr learned my addr 172.21.15.67:0/3732422291 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.975+0000 7fc623400640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6240c96f0 0x7fc62414b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.975+0000 7fc61be00640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6240c5f50 0x7fc6241dfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.976+0000 7fc622a00640 1 -- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6240c96f0 msgr2=0x7fc62414b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.976+0000 7fc622a00640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6240c96f0 0x7fc62414b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.976+0000 7fc622a00640 1 -- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6240c5f50 msgr2=0x7fc6241dfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.976+0000 7fc622a00640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6240c5f50 0x7fc6241dfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.976+0000 7fc622a00640 1 -- 172.21.15.67:0/3732422291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6241ebce0 con 0x7fc62406b040 2024-03-09T09:06:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.976+0000 7fc622a00640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc62406b040 0x7fc6240c4a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc61402b950 tx=0x7fc6140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.976+0000 7fc61aa00640 1 -- 172.21.15.67:0/3732422291 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61403ad60 con 0x7fc62406b040 2024-03-09T09:06:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.976+0000 7fc61aa00640 1 -- 172.21.15.67:0/3732422291 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6140443f0 con 0x7fc62406b040 2024-03-09T09:06:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.976+0000 7fc629600640 1 -- 172.21.15.67:0/3732422291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6241ebfc0 con 0x7fc62406b040 2024-03-09T09:06:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.976+0000 7fc61aa00640 1 -- 172.21.15.67:0/3732422291 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6140447e0 con 0x7fc62406b040 2024-03-09T09:06:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.976+0000 7fc629600640 1 -- 172.21.15.67:0/3732422291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6241ec580 con 0x7fc62406b040 2024-03-09T09:06:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.978+0000 7fc629600640 1 -- 172.21.15.67:0/3732422291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc624075a00 con 0x7fc62406b040 2024-03-09T09:06:30.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.979+0000 7fc61aa00640 1 -- 172.21.15.67:0/3732422291 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc614038020 con 0x7fc62406b040 2024-03-09T09:06:30.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.979+0000 7fc61aa00640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60006b4d0 0x7fc60006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:30.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.980+0000 7fc61aa00640 1 -- 172.21.15.67:0/3732422291 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc6140ac5c0 con 0x7fc62406b040 2024-03-09T09:06:30.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.980+0000 7fc61be00640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60006b4d0 0x7fc60006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:30.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.981+0000 7fc61be00640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60006b4d0 0x7fc60006d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc6241e1550 tx=0x7fc60c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:30.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:30.984+0000 7fc61aa00640 1 -- 172.21.15.67:0/3732422291 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc61403f070 con 0x7fc62406b040 2024-03-09T09:06:31.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.259+0000 7fc629600640 1 -- 172.21.15.67:0/3732422291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc62406cca0 con 0x7fc62406b040 2024-03-09T09:06:31.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.260+0000 7fc61aa00640 1 -- 172.21.15.67:0/3732422291 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fc61407aa70 con 0x7fc62406b040 2024-03-09T09:06:31.264 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":102,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:06:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.263+0000 7fc629600640 1 -- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60006b4d0 msgr2=0x7fc60006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.263+0000 7fc629600640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60006b4d0 0x7fc60006d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc6241e1550 tx=0x7fc60c008040 comp rx=0 tx=0).stop 2024-03-09T09:06:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.263+0000 7fc629600640 1 -- 172.21.15.67:0/3732422291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc62406b040 msgr2=0x7fc6240c4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.263+0000 7fc629600640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc62406b040 0x7fc6240c4a50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc61402b950 tx=0x7fc6140046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.263+0000 7fc629600640 1 -- 172.21.15.67:0/3732422291 shutdown_connections 2024-03-09T09:06:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.263+0000 7fc629600640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60006b4d0 0x7fc60006d990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.263+0000 7fc629600640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6240c96f0 0x7fc62414b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.263+0000 7fc629600640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6240c5f50 0x7fc6241dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.263+0000 7fc629600640 1 --2- 172.21.15.67:0/3732422291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc62406b040 0x7fc6240c4a50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.263+0000 7fc629600640 1 -- 172.21.15.67:0/3732422291 >> 172.21.15.67:0/3732422291 conn(0x7fc6240c8050 msgr2=0x7fc624158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.264+0000 7fc629600640 1 -- 172.21.15.67:0/3732422291 shutdown_connections 2024-03-09T09:06:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:31.264+0000 7fc629600640 1 -- 172.21.15.67:0/3732422291 wait complete. 2024-03-09T09:06:34.287 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:06:34.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.511+0000 7f5972800640 1 Processor -- start 2024-03-09T09:06:34.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.512+0000 7f5972800640 1 -- start start 2024-03-09T09:06:34.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.514+0000 7f5972800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f596c15a1b0 0x7f596c0aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:34.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.514+0000 7f5972800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c0ab550 con 0x7f596c0c37c0 2024-03-09T09:06:34.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.514+0000 7f5972800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c15a700 con 0x7f596c0a9ab0 2024-03-09T09:06:34.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.514+0000 7f5972800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c15a9e0 con 0x7f596c15a1b0 2024-03-09T09:06:34.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.515+0000 7f596be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f596c0c37c0 0x7f596c0aae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56092/0 (socket says 172.21.15.67:56092) 2024-03-09T09:06:34.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.515+0000 7f596be00640 1 -- 172.21.15.67:0/1060026778 learned_addr learned my addr 172.21.15.67:0/1060026778 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:06:34.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.515+0000 7f596aa00640 1 --2- 172.21.15.67:0/1060026778 >> v2:172.21.15.134:3300/0 conn(0x7f596c15a1b0 0x7f596c0aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:34.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.516+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166939364 0 0) 0x7f596c0ab550 con 0x7f596c0c37c0 2024-03-09T09:06:34.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.516+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5950003520 con 0x7f596c0c37c0 2024-03-09T09:06:34.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.516+0000 7f596aa00640 1 -- 172.21.15.67:0/1060026778 >> v2:172.21.15.134:3300/0 conn(0x7f596c15a1b0 msgr2=0x7f596c0aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:06:34.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.516+0000 7f596aa00640 1 -- 172.21.15.67:0/1060026778 >> v2:172.21.15.134:3300/0 conn(0x7f596c15a1b0 msgr2=0x7f596c0aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:06:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.516+0000 7f596aa00640 1 --2- 172.21.15.67:0/1060026778 >> v2:172.21.15.134:3300/0 conn(0x7f596c15a1b0 0x7f596c0aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:06:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.516+0000 7f596aa00640 1 --2- 172.21.15.67:0/1060026778 >> v2:172.21.15.134:3300/0 conn(0x7f596c15a1b0 0x7f596c0aa900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:34.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 655815435 0 0) 0x7f5950003520 con 0x7f596c0c37c0 2024-03-09T09:06:34.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 >> v2:172.21.15.134:3300/0 conn(0x7f596c15a1b0 msgr2=0x7f596c0aa900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:06:34.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 --2- 172.21.15.67:0/1060026778 >> v2:172.21.15.134:3300/0 conn(0x7f596c15a1b0 0x7f596c0aa900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:34.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 >> v1:172.21.15.134:6789/0 conn(0x7f596c0a9ab0 legacy=0x7f596c0ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:34.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c15acc0 con 0x7f596c0c37c0 2024-03-09T09:06:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f59640019f0 con 0x7f596c0c37c0 2024-03-09T09:06:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 >> v1:172.21.15.67:6789/0 conn(0x7f596c0c37c0 legacy=0x7f596c0aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 --2- 172.21.15.67:0/1060026778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59500039b0 0x7f5950003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 --2- 172.21.15.67:0/1060026778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5950004800 0x7f5950004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 --2- 172.21.15.67:0/1060026778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59500051c0 0x7f595000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c15a9e0 con 0x7f59500051c0 2024-03-09T09:06:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595000c110 con 0x7f5950004800 2024-03-09T09:06:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.518+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595000c310 con 0x7f59500039b0 2024-03-09T09:06:34.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.519+0000 7f596aa00640 1 --2- 172.21.15.67:0/1060026778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5950004800 0x7f5950004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.519+0000 7f596be00640 1 --2- 172.21.15.67:0/1060026778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59500051c0 0x7f595000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.519+0000 7f596b400640 1 --2- 172.21.15.67:0/1060026778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59500039b0 0x7f5950003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.520+0000 7f596be00640 1 -- 172.21.15.67:0/1060026778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59500039b0 msgr2=0x7f5950003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.520+0000 7f596be00640 1 --2- 172.21.15.67:0/1060026778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59500039b0 0x7f5950003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.520+0000 7f596be00640 1 -- 172.21.15.67:0/1060026778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5950004800 msgr2=0x7f5950004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.520+0000 7f596be00640 1 --2- 172.21.15.67:0/1060026778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5950004800 0x7f5950004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.520+0000 7f596be00640 1 -- 172.21.15.67:0/1060026778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c15acc0 con 0x7f59500051c0 2024-03-09T09:06:34.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.522+0000 7f596be00640 1 -- 172.21.15.67:0/1060026778 reap_dead start 2024-03-09T09:06:34.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.522+0000 7f596be00640 1 --2- 172.21.15.67:0/1060026778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59500051c0 0x7f595000b9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f596c0c3c70 tx=0x7f5950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ad4d4248a1c3ff server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:34.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.523+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596400a480 con 0x7f59500051c0 2024-03-09T09:06:34.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.523+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5964032600 con 0x7f59500051c0 2024-03-09T09:06:34.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.523+0000 7f596a000640 1 -- 172.21.15.67:0/1060026778 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59640329d0 con 0x7f59500051c0 2024-03-09T09:06:34.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.524+0000 7f5972800640 1 -- 172.21.15.67:0/1060026778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59500051c0 msgr2=0x7f595000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:34.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.524+0000 7f5972800640 1 --2- 172.21.15.67:0/1060026778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59500051c0 0x7f595000b9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f596c0c3c70 tx=0x7f5950005830 comp rx=0 tx=0).stop 2024-03-09T09:06:34.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.524+0000 7f5972800640 1 -- 172.21.15.67:0/1060026778 shutdown_connections 2024-03-09T09:06:34.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.524+0000 7f5972800640 1 --2- 172.21.15.67:0/1060026778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59500051c0 0x7f595000b9c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:34.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.524+0000 7f5972800640 1 -- 172.21.15.67:0/1060026778 >> 172.21.15.67:0/1060026778 conn(0x7f596c0c7370 msgr2=0x7f596c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:34.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.525+0000 7f5972800640 1 -- 172.21.15.67:0/1060026778 shutdown_connections 2024-03-09T09:06:34.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.525+0000 7f5972800640 1 -- 172.21.15.67:0/1060026778 wait complete. 2024-03-09T09:06:34.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.527+0000 7f5972800640 1 Processor -- start 2024-03-09T09:06:34.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.527+0000 7f5972800640 1 -- start start 2024-03-09T09:06:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.528+0000 7f5972800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59500051c0 0x7f596c157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.529+0000 7f5972800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c157db0 0x7f596c1f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.529+0000 7f5972800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f596c1f3300 0x7f596c0720e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.529+0000 7f5972800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c06dfa0 con 0x7f596c157db0 2024-03-09T09:06:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.529+0000 7f5972800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0a9e90 con 0x7f596c1f3300 2024-03-09T09:06:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.529+0000 7f5972800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c06e120 con 0x7f59500051c0 2024-03-09T09:06:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.529+0000 7f596aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c157db0 0x7f596c1f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.529+0000 7f596aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c157db0 0x7f596c1f2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44384/0 (socket says 172.21.15.67:44384) 2024-03-09T09:06:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.529+0000 7f596aa00640 1 -- 172.21.15.67:0/1700749349 learned_addr learned my addr 172.21.15.67:0/1700749349 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.530+0000 7f596be00640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f596c1f3300 0x7f596c0720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.530+0000 7f596aa00640 1 -- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59500051c0 msgr2=0x7f596c157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.530+0000 7f596aa00640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59500051c0 0x7f596c157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.530+0000 7f596aa00640 1 -- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f596c1f3300 msgr2=0x7f596c0720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.530+0000 7f596aa00640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f596c1f3300 0x7f596c0720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.530+0000 7f596aa00640 1 -- 172.21.15.67:0/1700749349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c15b4d0 con 0x7f596c157db0 2024-03-09T09:06:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.531+0000 7f596aa00640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c157db0 0x7f596c1f2dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5954010920 tx=0x7f5954010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.532+0000 7f5963400640 1 -- 172.21.15.67:0/1700749349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595401a020 con 0x7f596c157db0 2024-03-09T09:06:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.532+0000 7f5963400640 1 -- 172.21.15.67:0/1700749349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595400e9c0 con 0x7f596c157db0 2024-03-09T09:06:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.532+0000 7f5963400640 1 -- 172.21.15.67:0/1700749349 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595400edb0 con 0x7f596c157db0 2024-03-09T09:06:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.532+0000 7f5972800640 1 -- 172.21.15.67:0/1700749349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c15c380 con 0x7f596c157db0 2024-03-09T09:06:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.533+0000 7f5972800640 1 -- 172.21.15.67:0/1700749349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c158210 con 0x7f596c157db0 2024-03-09T09:06:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.536+0000 7f5972800640 1 -- 172.21.15.67:0/1700749349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c075a00 con 0x7f596c157db0 2024-03-09T09:06:34.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.538+0000 7f5963400640 1 -- 172.21.15.67:0/1700749349 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f595401d020 con 0x7f596c157db0 2024-03-09T09:06:34.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.540+0000 7f5963400640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f594806b4d0 0x7f594806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:34.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.540+0000 7f5963400640 1 -- 172.21.15.67:0/1700749349 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f595408add0 con 0x7f596c157db0 2024-03-09T09:06:34.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.540+0000 7f596b400640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f594806b4d0 0x7f594806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.542+0000 7f596b400640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f594806b4d0 0x7f594806d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f595c0014f0 tx=0x7f595c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:34.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.546+0000 7f5963400640 1 -- 172.21.15.67:0/1700749349 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f59540598f0 con 0x7f596c157db0 2024-03-09T09:06:34.638 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 29% (63/217) Updating files: 30% (66/217) Updating files: 31% (68/217) Updating files: 32% (70/217) Updating files: 33% (72/217) Updating files: 34% (74/217) Updating files: 35% (76/217) Updating files: 36% (79/217) Updating files: 37% (81/217) Updating files: 38% (83/217) Updating files: 39% (85/217) Updating files: 40% (87/217) Updating files: 41% (89/217) Updating files: 42% (92/217) Updating files: 43% (94/217) Updating files: 44% (96/217) Updating files: 45% (98/217) Updating files: 46% (100/217) Updating files: 47% (102/217) Updating files: 48% (105/217) Updating files: 49% (107/217) Updating files: 50% (109/217) Updating files: 51% (111/217) Updating files: 52% (113/217) Updating files: 53% (116/217) Updating files: 54% (118/217) Updating files: 55% (120/217) Updating files: 56% (122/217) Updating files: 57% (124/217) Updating files: 58% (126/217) Updating files: 59% (129/217) Updating files: 60% (131/217) Updating files: 61% (133/217) Updating files: 62% (135/217) Updating files: 63% (137/217) Updating files: 64% (139/217) Updating files: 65% (142/217) Updating files: 66% (144/217) Updating files: 67% (146/217) Updating files: 68% (148/217) Updating files: 69% (150/217) Updating files: 70% (152/217) Updating files: 71% (155/217) Updating files: 72% (157/217) Updating files: 73% (159/217) Updating files: 74% (161/217) Updating files: 75% (163/217) Updating files: 76% (165/217) Updating files: 77% (168/217) Updating files: 78% (170/217) Updating files: 79% (172/217) Updating files: 80% (174/217) Updating files: 81% (176/217) Updating files: 82% (178/217) Updating files: 83% (181/217) Updating files: 84% (183/217) Updating files: 85% (185/217) Updating files: 86% (187/217) Updating files: 87% (189/217) Updating files: 88% (191/217) Updating files: 89% (194/217) Updating files: 90% (196/217) Updating files: 91% (198/217) Updating files: 92% (200/217) Updating files: 93% (202/217) Updating files: 94% (204/217) Updating files: 95% (207/217) Updating files: 96% (209/217) Updating files: 97% (211/217) Updating files: 98% (213/217) Updating files: 99% (215/217) Updating files: 100% (217/217) Updating files: 100% (217/217), done. 2024-03-09T09:06:34.713 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at dac3a8c35fb v0.17 2024-03-09T09:06:34.719 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.17 2024-03-09T09:06:34.871 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:06:34.871 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 18 -eq 48 ']' 2024-03-09T09:06:34.871 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.18 2024-03-09T09:06:34.871 INFO:tasks.workunit.client.1.smithi134.stdout:v0.18 2024-03-09T09:06:34.872 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.18 2024-03-09T09:06:34.872 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.18 2024-03-09T09:06:34.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.945+0000 7f5972800640 1 -- 172.21.15.67:0/1700749349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f596c15c8a0 con 0x7f596c157db0 2024-03-09T09:06:34.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.946+0000 7f5963400640 1 -- 172.21.15.67:0/1700749349 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f59540591f0 con 0x7f596c157db0 2024-03-09T09:06:34.950 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":102,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.949+0000 7f5972800640 1 -- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f594806b4d0 msgr2=0x7f594806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.949+0000 7f5972800640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f594806b4d0 0x7f594806d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f595c0014f0 tx=0x7f595c009040 comp rx=0 tx=0).stop 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.949+0000 7f5972800640 1 -- 172.21.15.67:0/1700749349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c157db0 msgr2=0x7f596c1f2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.949+0000 7f5972800640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c157db0 0x7f596c1f2dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5954010920 tx=0x7f5954010de0 comp rx=0 tx=0).stop 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.949+0000 7f5972800640 1 -- 172.21.15.67:0/1700749349 shutdown_connections 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.949+0000 7f5972800640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f594806b4d0 0x7f594806d990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.949+0000 7f5972800640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f596c1f3300 0x7f596c0720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.949+0000 7f5972800640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f596c157db0 0x7f596c1f2dc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.949+0000 7f5972800640 1 --2- 172.21.15.67:0/1700749349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59500051c0 0x7f596c157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.949+0000 7f5972800640 1 -- 172.21.15.67:0/1700749349 >> 172.21.15.67:0/1700749349 conn(0x7f596c0c7370 msgr2=0x7f596c0c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.950+0000 7f5972800640 1 -- 172.21.15.67:0/1700749349 shutdown_connections 2024-03-09T09:06:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:34.950+0000 7f5972800640 1 -- 172.21.15.67:0/1700749349 wait complete. 2024-03-09T09:06:35.206 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 40% (88/217) Updating files: 41% (89/217) Updating files: 42% (92/217) Updating files: 43% (94/217) Updating files: 44% (96/217) Updating files: 45% (98/217) Updating files: 46% (100/217) Updating files: 47% (102/217) Updating files: 48% (105/217) Updating files: 49% (107/217) Updating files: 50% (109/217) Updating files: 51% (111/217) Updating files: 52% (113/217) Updating files: 53% (116/217) Updating files: 54% (118/217) Updating files: 55% (120/217) Updating files: 56% (122/217) Updating files: 57% (124/217) Updating files: 58% (126/217) Updating files: 59% (129/217) Updating files: 60% (131/217) Updating files: 61% (133/217) Updating files: 62% (135/217) Updating files: 63% (137/217) Updating files: 64% (139/217) Updating files: 65% (142/217) Updating files: 66% (144/217) Updating files: 67% (146/217) Updating files: 68% (148/217) Updating files: 69% (150/217) Updating files: 70% (152/217) Updating files: 71% (155/217) Updating files: 72% (157/217) Updating files: 73% (159/217) Updating files: 74% (161/217) Updating files: 75% (163/217) Updating files: 76% (165/217) Updating files: 77% (168/217) Updating files: 78% (170/217) Updating files: 79% (172/217) Updating files: 80% (174/217) Updating files: 81% (176/217) Updating files: 82% (178/217) Updating files: 82% (179/217) Updating files: 83% (181/217) Updating files: 84% (183/217) Updating files: 85% (185/217) Updating files: 86% (187/217) Updating files: 87% (189/217) Updating files: 88% (191/217) Updating files: 89% (194/217) Updating files: 90% (196/217) Updating files: 91% (198/217) Updating files: 92% (200/217) Updating files: 93% (202/217) Updating files: 94% (204/217) Updating files: 95% (207/217) Updating files: 96% (209/217) Updating files: 97% (211/217) Updating files: 98% (213/217) Updating files: 99% (215/217) Updating files: 100% (217/217) Updating files: 100% (217/217), done. 2024-03-09T09:06:35.271 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at dac3a8c35fb v0.17 2024-03-09T09:06:35.280 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.17 2024-03-09T09:06:35.704 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:35.705 INFO:tasks.workunit.client.0.smithi067.stdout:v0.18 2024-03-09T09:06:35.705 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 18 -eq 48 ']' 2024-03-09T09:06:35.705 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.18 2024-03-09T09:06:35.705 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.18 2024-03-09T09:06:35.705 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.18 2024-03-09T09:06:37.971 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:06:38.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.197+0000 7f167f000640 1 Processor -- start 2024-03-09T09:06:38.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.197+0000 7f167f000640 1 -- start start 2024-03-09T09:06:38.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.198+0000 7f167f000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f16780cadb0 0x7f1678156720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.198+0000 7f167f000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1678157000 con 0x7f1678153300 2024-03-09T09:06:38.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.198+0000 7f167f000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780ae430 con 0x7f16780c5e40 2024-03-09T09:06:38.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.198+0000 7f167f000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16780ae6f0 con 0x7f16780cadb0 2024-03-09T09:06:38.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.199+0000 7f167d200640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f16780c5e40 0x7f1678156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:43438/0 (socket says 172.21.15.67:43438) 2024-03-09T09:06:38.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.199+0000 7f167d200640 1 -- 172.21.15.67:0/1431219164 learned_addr learned my addr 172.21.15.67:0/1431219164 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:06:38.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.199+0000 7f167dc00640 1 --1- 172.21.15.67:0/1431219164 >> v1:172.21.15.67:6789/0 conn(0x7f1678153300 0x7f16780b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56104/0 (socket says 172.21.15.67:56104) 2024-03-09T09:06:38.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.199+0000 7f167e600640 1 --2- 172.21.15.67:0/1431219164 >> v2:172.21.15.134:3300/0 conn(0x7f16780cadb0 0x7f1678156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.199+0000 7f167e600640 1 -- 172.21.15.67:0/1431219164 >> v2:172.21.15.134:3300/0 conn(0x7f16780cadb0 msgr2=0x7f1678156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:06:38.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.199+0000 7f167e600640 1 -- 172.21.15.67:0/1431219164 >> v2:172.21.15.134:3300/0 conn(0x7f16780cadb0 msgr2=0x7f1678156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:06:38.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.199+0000 7f167e600640 1 --2- 172.21.15.67:0/1431219164 >> v2:172.21.15.134:3300/0 conn(0x7f16780cadb0 0x7f1678156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:06:38.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.199+0000 7f167e600640 1 --2- 172.21.15.67:0/1431219164 >> v2:172.21.15.134:3300/0 conn(0x7f16780cadb0 0x7f1678156720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:38.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.199+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487292604 0 0) 0x7f1678157000 con 0x7f1678153300 2024-03-09T09:06:38.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.199+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1660003520 con 0x7f1678153300 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2178946558 0 0) 0x7f1660003520 con 0x7f1678153300 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 >> v2:172.21.15.134:3300/0 conn(0x7f16780cadb0 msgr2=0x7f1678156720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 --2- 172.21.15.67:0/1431219164 >> v2:172.21.15.134:3300/0 conn(0x7f16780cadb0 0x7f1678156720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 >> v1:172.21.15.134:6789/0 conn(0x7f16780c5e40 legacy=0x7f1678156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678155b10 con 0x7f1678153300 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f166c001670 con 0x7f1678153300 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 >> v1:172.21.15.67:6789/0 conn(0x7f1678153300 legacy=0x7f16780b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660003a20 0x7f1660003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 0x7f1660004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1660005230 0x7f166000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780ae6f0 con 0x7f1660003a20 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166000c190 con 0x7f1660004800 2024-03-09T09:06:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166000c390 con 0x7f1660005230 2024-03-09T09:06:38.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f167dc00640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660003a20 0x7f1660003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f167d200640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 0x7f1660004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.201+0000 7f167e600640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1660005230 0x7f166000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.202+0000 7f167d200640 1 -- 172.21.15.67:0/1431219164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1660005230 msgr2=0x7f166000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.202+0000 7f167d200640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1660005230 0x7f166000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.202+0000 7f167d200640 1 -- 172.21.15.67:0/1431219164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660003a20 msgr2=0x7f1660003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.202+0000 7f167d200640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660003a20 0x7f1660003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.202+0000 7f167d200640 1 -- 172.21.15.67:0/1431219164 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1660003520 con 0x7f1660004800 2024-03-09T09:06:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.202+0000 7f167e600640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1660005230 0x7f166000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:06:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.202+0000 7f167dc00640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1660003a20 0x7f1660003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:06:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.202+0000 7f167e600640 1 -- 172.21.15.67:0/1431219164 reap_dead start 2024-03-09T09:06:38.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.204+0000 7f167d200640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 0x7f1660004c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1664001810 tx=0x7f1660005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e20a0214f5f8d16b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.207+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664001460 con 0x7f1660004800 2024-03-09T09:06:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.207+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1664004aa0 con 0x7f1660004800 2024-03-09T09:06:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.207+0000 7f1677e00640 1 -- 172.21.15.67:0/1431219164 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664007070 con 0x7f1660004800 2024-03-09T09:06:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.208+0000 7f167f000640 1 -- 172.21.15.67:0/1431219164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 msgr2=0x7f1660004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.208+0000 7f167f000640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 0x7f1660004c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1664001810 tx=0x7f1660005830 comp rx=0 tx=0).stop 2024-03-09T09:06:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.208+0000 7f167f000640 1 -- 172.21.15.67:0/1431219164 shutdown_connections 2024-03-09T09:06:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.208+0000 7f167f000640 1 --2- 172.21.15.67:0/1431219164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 0x7f1660004c80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.208+0000 7f167f000640 1 -- 172.21.15.67:0/1431219164 >> 172.21.15.67:0/1431219164 conn(0x7f16780732d0 msgr2=0x7f16780ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.209+0000 7f167f000640 1 -- 172.21.15.67:0/1431219164 shutdown_connections 2024-03-09T09:06:38.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.209+0000 7f167f000640 1 -- 172.21.15.67:0/1431219164 wait complete. 2024-03-09T09:06:38.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.211+0000 7f167f000640 1 Processor -- start 2024-03-09T09:06:38.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.212+0000 7f167f000640 1 -- start start 2024-03-09T09:06:38.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.213+0000 7f167f000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 0x7f16780c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.213+0000 7f167f000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16781519a0 0x7f167806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.213+0000 7f167f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16780c6a00 0x7f167806b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.213+0000 7f167f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780c59e0 con 0x7f16780c6a00 2024-03-09T09:06:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.213+0000 7f167f000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678159e90 con 0x7f1660004800 2024-03-09T09:06:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.213+0000 7f167f000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16780b0000 con 0x7f16781519a0 2024-03-09T09:06:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.214+0000 7f167dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 0x7f16780c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.214+0000 7f167d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16781519a0 0x7f167806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.214+0000 7f167e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16780c6a00 0x7f167806b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.214+0000 7f167dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 0x7f16780c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:42708/0 (socket says 172.21.15.67:42708) 2024-03-09T09:06:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.214+0000 7f167d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16781519a0 0x7f167806b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:60990/0 (socket says 172.21.15.67:60990) 2024-03-09T09:06:38.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.214+0000 7f167dc00640 1 -- 172.21.15.67:0/599384458 learned_addr learned my addr 172.21.15.67:0/599384458 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:38.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.215+0000 7f167dc00640 1 -- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16781519a0 msgr2=0x7f167806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.215+0000 7f167dc00640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16781519a0 0x7f167806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.215+0000 7f167dc00640 1 -- 172.21.15.67:0/599384458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16780c6a00 msgr2=0x7f167806b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.215+0000 7f167dc00640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16780c6a00 0x7f167806b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.215+0000 7f167dc00640 1 -- 172.21.15.67:0/599384458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16781ec2e0 con 0x7f1660004800 2024-03-09T09:06:38.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.215+0000 7f167e600640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16780c6a00 0x7f167806b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:06:38.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.215+0000 7f167dc00640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 0x7f16780c64c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f166c0017e0 tx=0x7f166c004ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:38.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.216+0000 7f1675600640 1 -- 172.21.15.67:0/599384458 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c007c50 con 0x7f1660004800 2024-03-09T09:06:38.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.216+0000 7f1675600640 1 -- 172.21.15.67:0/599384458 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166c018070 con 0x7f1660004800 2024-03-09T09:06:38.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.216+0000 7f1675600640 1 -- 172.21.15.67:0/599384458 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c00fbd0 con 0x7f1660004800 2024-03-09T09:06:38.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.216+0000 7f167f000640 1 -- 172.21.15.67:0/599384458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16781ec5c0 con 0x7f1660004800 2024-03-09T09:06:38.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.216+0000 7f167f000640 1 -- 172.21.15.67:0/599384458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167806cdb0 con 0x7f1660004800 2024-03-09T09:06:38.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.218+0000 7f167f000640 1 -- 172.21.15.67:0/599384458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167804f3c0 con 0x7f1660004800 2024-03-09T09:06:38.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.219+0000 7f1675600640 1 -- 172.21.15.67:0/599384458 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f166c01d460 con 0x7f1660004800 2024-03-09T09:06:38.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.220+0000 7f1675600640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f165806b4d0 0x7f165806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.220+0000 7f1675600640 1 -- 172.21.15.67:0/599384458 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f166c08a4c0 con 0x7f1660004800 2024-03-09T09:06:38.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.221+0000 7f167d200640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f165806b4d0 0x7f165806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.225+0000 7f167d200640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f165806b4d0 0x7f165806d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1664033000 tx=0x7f1664009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:38.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.229+0000 7f1675600640 1 -- 172.21.15.67:0/599384458 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f166c0588f0 con 0x7f1660004800 2024-03-09T09:06:38.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.511+0000 7f167f000640 1 -- 172.21.15.67:0/599384458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f16780cab90 con 0x7f1660004800 2024-03-09T09:06:38.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.512+0000 7f1675600640 1 -- 172.21.15.67:0/599384458 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f166c05c990 con 0x7f1660004800 2024-03-09T09:06:38.515 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":104,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:06:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.515+0000 7f167f000640 1 -- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f165806b4d0 msgr2=0x7f165806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.515+0000 7f167f000640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f165806b4d0 0x7f165806d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1664033000 tx=0x7f1664009030 comp rx=0 tx=0).stop 2024-03-09T09:06:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.515+0000 7f167f000640 1 -- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 msgr2=0x7f16780c64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.515+0000 7f167f000640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 0x7f16780c64c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f166c0017e0 tx=0x7f166c004ea0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.515+0000 7f167f000640 1 -- 172.21.15.67:0/599384458 shutdown_connections 2024-03-09T09:06:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.515+0000 7f167f000640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f165806b4d0 0x7f165806d990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.515+0000 7f167f000640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16780c6a00 0x7f167806b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.515+0000 7f167f000640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16781519a0 0x7f167806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.515+0000 7f167f000640 1 --2- 172.21.15.67:0/599384458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1660004800 0x7f16780c64c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.515+0000 7f167f000640 1 -- 172.21.15.67:0/599384458 >> 172.21.15.67:0/599384458 conn(0x7f16780732d0 msgr2=0x7f1678159090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.515+0000 7f167f000640 1 -- 172.21.15.67:0/599384458 shutdown_connections 2024-03-09T09:06:38.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.516+0000 7f167f000640 1 -- 172.21.15.67:0/599384458 wait complete. 2024-03-09T09:06:38.544 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:06:38.544 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:06:38.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:06:38.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.828+0000 7f22ae000640 1 Processor -- start 2024-03-09T09:06:38.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.828+0000 7f22ae000640 1 -- start start 2024-03-09T09:06:38.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.829+0000 7f22ae000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22a80730c0 0x7f22a80aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.829+0000 7f22ae000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22a806d830 0x7f22a806dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.829+0000 7f22ae000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a8152e00 con 0x7f22a80734a0 2024-03-09T09:06:38.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.829+0000 7f22ae000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a80aa960 con 0x7f22a80730c0 2024-03-09T09:06:38.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.829+0000 7f22ae000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a81530e0 con 0x7f22a806d830 2024-03-09T09:06:38.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.830+0000 7f22a7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22a80730c0 0x7f22a80aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.830+0000 7f22a7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f22a80730c0 0x7f22a80aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44420/0 (socket says 172.21.15.67:44420) 2024-03-09T09:06:38.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.830+0000 7f22a7400640 1 -- 172.21.15.67:0/1538386092 learned_addr learned my addr 172.21.15.67:0/1538386092 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:38.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.830+0000 7f22a7e00640 1 --2- 172.21.15.67:0/1538386092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22a806d830 0x7f22a806dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.830+0000 7f22a6a00640 1 --1- 172.21.15.67:0/1538386092 >> v1:172.21.15.134:6789/0 conn(0x7f22a80734a0 0x7f22a80c2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:43452/0 (socket says 172.21.15.67:43452) 2024-03-09T09:06:38.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.830+0000 7f22a7400640 1 -- 172.21.15.67:0/1538386092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22a806d830 msgr2=0x7f22a806dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:38.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.830+0000 7f22a7400640 1 --2- 172.21.15.67:0/1538386092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22a806d830 0x7f22a806dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.830+0000 7f22a7400640 1 -- 172.21.15.67:0/1538386092 >> v1:172.21.15.134:6789/0 conn(0x7f22a80734a0 legacy=0x7f22a80c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.830+0000 7f22a7400640 1 -- 172.21.15.67:0/1538386092 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a81533c0 con 0x7f22a80730c0 2024-03-09T09:06:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.830+0000 7f22a6000640 1 -- 172.21.15.67:0/1538386092 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766059110 0 0) 0x7f22a8152e00 con 0x7f22a80734a0 2024-03-09T09:06:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.831+0000 7f22a7400640 1 --2- 172.21.15.67:0/1538386092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a80730c0 0x7f22a80aa420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2298009870 tx=0x7f229802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dd09cfd9ea9c27 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.832+0000 7f22a6000640 1 -- 172.21.15.67:0/1538386092 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298035020 con 0x7f22a80730c0 2024-03-09T09:06:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.832+0000 7f22a6000640 1 -- 172.21.15.67:0/1538386092 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2298031640 con 0x7f22a80730c0 2024-03-09T09:06:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.832+0000 7f22a6000640 1 -- 172.21.15.67:0/1538386092 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229802d040 con 0x7f22a80730c0 2024-03-09T09:06:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.832+0000 7f22ae000640 1 -- 172.21.15.67:0/1538386092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a80730c0 msgr2=0x7f22a80aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.832+0000 7f22ae000640 1 --2- 172.21.15.67:0/1538386092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a80730c0 0x7f22a80aa420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2298009870 tx=0x7f229802b8f0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.832+0000 7f22ae000640 1 -- 172.21.15.67:0/1538386092 shutdown_connections 2024-03-09T09:06:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.832+0000 7f22ae000640 1 --2- 172.21.15.67:0/1538386092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22a806d830 0x7f22a806dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.832+0000 7f22ae000640 1 --2- 172.21.15.67:0/1538386092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a80730c0 0x7f22a80aa420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.832+0000 7f22ae000640 1 -- 172.21.15.67:0/1538386092 >> 172.21.15.67:0/1538386092 conn(0x7f22a80c7370 msgr2=0x7f22a80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.832+0000 7f22ae000640 1 -- 172.21.15.67:0/1538386092 shutdown_connections 2024-03-09T09:06:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.832+0000 7f22ae000640 1 -- 172.21.15.67:0/1538386092 wait complete. 2024-03-09T09:06:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.834+0000 7f22ae000640 1 Processor -- start 2024-03-09T09:06:38.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.834+0000 7f22ae000640 1 -- start start 2024-03-09T09:06:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.835+0000 7f22ae000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a806d830 0x7f22a80c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.835+0000 7f22ae000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22a80730c0 0x7f22a813e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.835+0000 7f22ae000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22a80734a0 0x7f22a813e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.835+0000 7f22ae000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80c3590 con 0x7f22a806d830 2024-03-09T09:06:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.835+0000 7f22ae000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a8153980 con 0x7f22a80730c0 2024-03-09T09:06:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.835+0000 7f22ae000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80c3710 con 0x7f22a80734a0 2024-03-09T09:06:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.835+0000 7f22a7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a806d830 0x7f22a80c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.835+0000 7f22a7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a806d830 0x7f22a80c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44428/0 (socket says 172.21.15.67:44428) 2024-03-09T09:06:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.835+0000 7f22a7400640 1 -- 172.21.15.67:0/895029900 learned_addr learned my addr 172.21.15.67:0/895029900 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.835+0000 7f22a6a00640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22a80730c0 0x7f22a813e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.835+0000 7f22a7e00640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22a80734a0 0x7f22a813e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.836+0000 7f22a7400640 1 -- 172.21.15.67:0/895029900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22a80734a0 msgr2=0x7f22a813e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.836+0000 7f22a7400640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22a80734a0 0x7f22a813e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.836+0000 7f22a7400640 1 -- 172.21.15.67:0/895029900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22a80730c0 msgr2=0x7f22a813e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.836+0000 7f22a7400640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22a80730c0 0x7f22a813e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.836+0000 7f22a7400640 1 -- 172.21.15.67:0/895029900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a8154810 con 0x7f22a806d830 2024-03-09T09:06:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.836+0000 7f22a7400640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a806d830 0x7f22a80c69d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2298000c00 tx=0x7f22980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.836+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298040070 con 0x7f22a806d830 2024-03-09T09:06:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.836+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22980399f0 con 0x7f22a806d830 2024-03-09T09:06:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.836+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298035040 con 0x7f22a806d830 2024-03-09T09:06:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.836+0000 7f22ae000640 1 -- 172.21.15.67:0/895029900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a8152300 con 0x7f22a806d830 2024-03-09T09:06:38.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.837+0000 7f22ae000640 1 -- 172.21.15.67:0/895029900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a81ef190 con 0x7f22a806d830 2024-03-09T09:06:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.838+0000 7f22ae000640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a8149390 0x7f22a81ec650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.838+0000 7f22ae000640 1 -- 172.21.15.67:0/895029900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a813eb60 con 0x7f22a8149390 2024-03-09T09:06:38.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.838+0000 7f22a6a00640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a8149390 0x7f22a81ec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.838+0000 7f22a6a00640 1 -- 172.21.15.67:0/895029900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22a8075a00 con 0x7f22a8149390 2024-03-09T09:06:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.839+0000 7f22a6a00640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a8149390 0x7f22a81ec650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f22a80c72b0 tx=0x7f22940018d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:38.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.839+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2298038020 con 0x7f22a806d830 2024-03-09T09:06:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.840+0000 7f229f400640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f228806b4d0 0x7f228806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.840+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f22980abe80 con 0x7f22a806d830 2024-03-09T09:06:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.840+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294011580 con 0x7f22a8149390 2024-03-09T09:06:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.840+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f2294018070 con 0x7f22a8149390 2024-03-09T09:06:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.840+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a8149390 msgr2=0x7f22a81ec650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.840+0000 7f229f400640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a8149390 0x7f22a81ec650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f22a80c72b0 tx=0x7f22940018d0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.840+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a8149390 msgr2=0x7f22a81ec650 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.840+0000 7f229f400640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a8149390 0x7f22a81ec650 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f22a80c72b0 tx=0x7f22940018d0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.840+0000 7f22a7e00640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f228806b4d0 0x7f228806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.841+0000 7f22a7e00640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f228806b4d0 0x7f228806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f22a00113e0 tx=0x7f22a000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.848+0000 7f22ae000640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a81ecb90 0x7f22a8150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.848+0000 7f22ae000640 1 -- 172.21.15.67:0/895029900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a8149770 con 0x7f22a81ecb90 2024-03-09T09:06:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.848+0000 7f22a6a00640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a81ecb90 0x7f22a8150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.849+0000 7f22a6a00640 1 -- 172.21.15.67:0/895029900 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f22a80c6f10 con 0x7f22a81ecb90 2024-03-09T09:06:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.849+0000 7f22a6a00640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a81ecb90 0x7f22a8150890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2294014000 tx=0x7f2294001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.849+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294011b80 con 0x7f22a81ecb90 2024-03-09T09:06:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.849+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f2294011b80 con 0x7f22a81ecb90 2024-03-09T09:06:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.849+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a81ecb90 msgr2=0x7f22a8150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.849+0000 7f229f400640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a81ecb90 0x7f22a8150890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2294014000 tx=0x7f2294001080 comp rx=0 tx=0).stop 2024-03-09T09:06:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.849+0000 7f229f400640 1 -- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a81ecb90 msgr2=0x7f22a8150890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:38.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.849+0000 7f229f400640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a81ecb90 0x7f22a8150890 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2294014000 tx=0x7f2294001080 comp rx=0 tx=0).stop 2024-03-09T09:06:38.854 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:06:38.854 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:06:38.854 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:06:38.854 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:06:38.854 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 104, 2024-03-09T09:06:38.854 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:06:38.854 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:06:38.854 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:06:38.854 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:06:38.854 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:38.854 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:38.855 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:38.856 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:06:38.857 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:38.858 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:38.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:38.861 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:38.862 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:38.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.862 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:06:38.862 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:06:38.862 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:38.862 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:38.862 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:38.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:06:38.863 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:06:38.864 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:06:38.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:38.866 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:06:38.867 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:06:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.850+0000 7f22ae000640 1 -- 172.21.15.67:0/895029900 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f228806b4d0 msgr2=0x7f228806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.850+0000 7f22ae000640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f228806b4d0 0x7f228806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f22a00113e0 tx=0x7f22a000b040 comp rx=0 tx=0).stop 2024-03-09T09:06:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.850+0000 7f22ae000640 1 -- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a806d830 msgr2=0x7f22a80c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.850+0000 7f22ae000640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a806d830 0x7f22a80c69d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2298000c00 tx=0x7f22980046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.850+0000 7f22a7e00640 1 -- 172.21.15.67:0/895029900 reap_dead start 2024-03-09T09:06:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.850+0000 7f22ae000640 1 -- 172.21.15.67:0/895029900 shutdown_connections 2024-03-09T09:06:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.850+0000 7f22ae000640 1 --2- 172.21.15.67:0/895029900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a806d830 0x7f22a80c69d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.850+0000 7f22ae000640 1 -- 172.21.15.67:0/895029900 >> 172.21.15.67:0/895029900 conn(0x7f22a80c7370 msgr2=0x7f22a80aaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.850+0000 7f22ae000640 1 -- 172.21.15.67:0/895029900 shutdown_connections 2024-03-09T09:06:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:38.850+0000 7f22ae000640 1 -- 172.21.15.67:0/895029900 wait complete. 2024-03-09T09:06:38.869 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:06:38.870 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:06:38.870 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:06:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.049+0000 7f47c9800640 1 Processor -- start 2024-03-09T09:06:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.050+0000 7f47c9800640 1 -- start start 2024-03-09T09:06:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.051+0000 7f47c9800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47c40ad8d0 0x7f47c4159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.051+0000 7f47c9800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c406c040 con 0x7f47c40c6650 2024-03-09T09:06:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.051+0000 7f47c9800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c406bd60 con 0x7f47c406b880 2024-03-09T09:06:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.051+0000 7f47c9800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c40c61f0 con 0x7f47c40ad8d0 2024-03-09T09:06:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.051+0000 7f47c3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47c40ad8d0 0x7f47c4159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.052+0000 7f47c3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47c40ad8d0 0x7f47c4159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60518/0 (socket says 172.21.15.67:60518) 2024-03-09T09:06:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.052+0000 7f47c3e00640 1 -- 172.21.15.67:0/2341968887 learned_addr learned my addr 172.21.15.67:0/2341968887 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.052+0000 7f47c3e00640 1 -- 172.21.15.67:0/2341968887 >> v1:172.21.15.134:6789/0 conn(0x7f47c406b880 legacy=0x7f47c4153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.052+0000 7f47c3e00640 1 -- 172.21.15.67:0/2341968887 >> v1:172.21.15.67:6789/0 conn(0x7f47c40c6650 legacy=0x7f47c4155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.052+0000 7f47c3e00640 1 -- 172.21.15.67:0/2341968887 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40c5e80 con 0x7f47c40ad8d0 2024-03-09T09:06:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.054+0000 7f47c3e00640 1 --2- 172.21.15.67:0/2341968887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c40ad8d0 0x7f47c4159c50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f47c40c6370 tx=0x7f47b402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b6248c9b5c26b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.054+0000 7f47c2a00640 1 -- 172.21.15.67:0/2341968887 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b402caf0 con 0x7f47c40ad8d0 2024-03-09T09:06:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.054+0000 7f47c2a00640 1 -- 172.21.15.67:0/2341968887 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b402cc90 con 0x7f47c40ad8d0 2024-03-09T09:06:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.054+0000 7f47c2a00640 1 -- 172.21.15.67:0/2341968887 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4034610 con 0x7f47c40ad8d0 2024-03-09T09:06:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.055+0000 7f47c9800640 1 -- 172.21.15.67:0/2341968887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c40ad8d0 msgr2=0x7f47c4159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.055+0000 7f47c9800640 1 --2- 172.21.15.67:0/2341968887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c40ad8d0 0x7f47c4159c50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f47c40c6370 tx=0x7f47b402bf50 comp rx=0 tx=0).stop 2024-03-09T09:06:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.055+0000 7f47c9800640 1 -- 172.21.15.67:0/2341968887 shutdown_connections 2024-03-09T09:06:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.055+0000 7f47c9800640 1 --2- 172.21.15.67:0/2341968887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c40ad8d0 0x7f47c4159c50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.055+0000 7f47c9800640 1 -- 172.21.15.67:0/2341968887 >> 172.21.15.67:0/2341968887 conn(0x7f47c40c8510 msgr2=0x7f47c40acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.055+0000 7f47c9800640 1 -- 172.21.15.67:0/2341968887 shutdown_connections 2024-03-09T09:06:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.055+0000 7f47c9800640 1 -- 172.21.15.67:0/2341968887 wait complete. 2024-03-09T09:06:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.058+0000 7f47c9800640 1 Processor -- start 2024-03-09T09:06:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.058+0000 7f47c9800640 1 -- start start 2024-03-09T09:06:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.059+0000 7f47c9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c406b880 0x7f47c40b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.059+0000 7f47c9800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40ad8d0 0x7f47c4148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c9800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c40c6650 0x7f47c4148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c9800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40ade30 con 0x7f47c406b880 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c9800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40adcb0 con 0x7f47c40ad8d0 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c9800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4156550 con 0x7f47c40c6650 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c406b880 0x7f47c40b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c406b880 0x7f47c40b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60530/0 (socket says 172.21.15.67:60530) 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c3e00640 1 -- 172.21.15.67:0/1881893926 learned_addr learned my addr 172.21.15.67:0/1881893926 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c3e00640 1 -- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c40c6650 msgr2=0x7f47c4148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c3e00640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c40c6650 0x7f47c4148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c3e00640 1 -- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40ad8d0 msgr2=0x7f47c4148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c3400640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40ad8d0 0x7f47c4148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c3e00640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40ad8d0 0x7f47c4148a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c3e00640 1 -- 172.21.15.67:0/1881893926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c41ec4f0 con 0x7f47c406b880 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.060+0000 7f47c3400640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40ad8d0 0x7f47c4148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.061+0000 7f47c3e00640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c406b880 0x7f47c40b0bd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f47b4036040 tx=0x7f47b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.061+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4043020 con 0x7f47c406b880 2024-03-09T09:06:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.061+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b403e450 con 0x7f47c406b880 2024-03-09T09:06:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.061+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b403e840 con 0x7f47c406b880 2024-03-09T09:06:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.061+0000 7f47c9800640 1 -- 172.21.15.67:0/1881893926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c40ae3c0 con 0x7f47c406b880 2024-03-09T09:06:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.061+0000 7f47c9800640 1 -- 172.21.15.67:0/1881893926 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c40b1110 con 0x7f47c406b880 2024-03-09T09:06:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.062+0000 7f47c9800640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c414ca30 0x7f47c40aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.062+0000 7f47c9800640 1 -- 172.21.15.67:0/1881893926 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41ec2a0 con 0x7f47c414ca30 2024-03-09T09:06:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.063+0000 7f47c3400640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c414ca30 0x7f47c40aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.063+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f47b4046020 con 0x7f47c406b880 2024-03-09T09:06:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.063+0000 7f47c3400640 1 -- 172.21.15.67:0/1881893926 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47c41ec7d0 con 0x7f47c414ca30 2024-03-09T09:06:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.064+0000 7f47bbe00640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f47a006b520 0x7f47a006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.064+0000 7f47c3400640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c414ca30 0x7f47c40aee80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f47c41514f0 tx=0x7f47ac00d950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.064+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f47b40acf90 con 0x7f47c406b880 2024-03-09T09:06:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.064+0000 7f47c8e00640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f47a006b520 0x7f47a006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.064+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac005280 con 0x7f47c414ca30 2024-03-09T09:06:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.066+0000 7f47c8e00640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f47a006b520 0x7f47a006d9e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f47c408d7a0 tx=0x7f47bc007950 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.066+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f47ac005280 con 0x7f47c414ca30 2024-03-09T09:06:39.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.066+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c414ca30 msgr2=0x7f47c40aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.066+0000 7f47bbe00640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c414ca30 0x7f47c40aee80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f47c41514f0 tx=0x7f47ac00d950 comp rx=0 tx=0).stop 2024-03-09T09:06:39.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.066+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c414ca30 msgr2=0x7f47c40aee80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:39.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.066+0000 7f47bbe00640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c414ca30 0x7f47c40aee80 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f47c41514f0 tx=0x7f47ac00d950 comp rx=0 tx=0).stop 2024-03-09T09:06:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.073+0000 7f47c9800640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40af3c0 0x7f47c41eed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.073+0000 7f47c9800640 1 -- 172.21.15.67:0/1881893926 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c404f3c0 con 0x7f47c40af3c0 2024-03-09T09:06:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.074+0000 7f47c3400640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40af3c0 0x7f47c41eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.074+0000 7f47c3400640 1 -- 172.21.15.67:0/1881893926 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f47c4151840 con 0x7f47c40af3c0 2024-03-09T09:06:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.075+0000 7f47c3400640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40af3c0 0x7f47c41eed90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f47c40b0820 tx=0x7f47ac001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.075+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac006bc0 con 0x7f47c40af3c0 2024-03-09T09:06:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f47ac006bc0 con 0x7f47c40af3c0 2024-03-09T09:06:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40af3c0 msgr2=0x7f47c41eed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47bbe00640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40af3c0 0x7f47c41eed90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f47c40b0820 tx=0x7f47ac001120 comp rx=0 tx=0).stop 2024-03-09T09:06:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47bbe00640 1 -- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40af3c0 msgr2=0x7f47c41eed90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47bbe00640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c40af3c0 0x7f47c41eed90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 104, 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:06:39.080 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:06:39.081 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.082 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:06:39.083 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:39.084 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:06:39.086 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:06:39.088 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:39.089 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:39.090 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:06:39.091 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47c9800640 1 -- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f47a006b520 msgr2=0x7f47a006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47c9800640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f47a006b520 0x7f47a006d9e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f47c408d7a0 tx=0x7f47bc007950 comp rx=0 tx=0).stop 2024-03-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47c9800640 1 -- 172.21.15.67:0/1881893926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c406b880 msgr2=0x7f47c40b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47c9800640 1 --2- 172.21.15.67:0/1881893926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c406b880 0x7f47c40b0bd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f47b4036040 tx=0x7f47b4004a40 comp rx=0 tx=0).stop 2024-03-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47c8e00640 1 -- 172.21.15.67:0/1881893926 reap_dead start 2024-03-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47c8e00640 1 -- 172.21.15.67:0/1881893926 reap_dead start 2024-03-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47c9800640 1 -- 172.21.15.67:0/1881893926 shutdown_connections 2024-03-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47c9800640 1 -- 172.21.15.67:0/1881893926 >> 172.21.15.67:0/1881893926 conn(0x7f47c40c8510 msgr2=0x7f47c414f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.076+0000 7f47c9800640 1 -- 172.21.15.67:0/1881893926 shutdown_connections 2024-03-09T09:06:39.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.077+0000 7f47c9800640 1 -- 172.21.15.67:0/1881893926 wait complete. 2024-03-09T09:06:39.094 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:06:39.094 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:06:39.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:06:39.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.296+0000 7f1e35200640 1 Processor -- start 2024-03-09T09:06:39.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.296+0000 7f1e35200640 1 -- start start 2024-03-09T09:06:39.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.298+0000 7f1e35200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1e300ac930 0x7f1e300c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.298+0000 7f1e35200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e300c30a0 0x7f1e30150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.298+0000 7f1e35200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e300c2780 con 0x7f1e30151190 2024-03-09T09:06:39.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.298+0000 7f1e35200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e30070670 con 0x7f1e300ac930 2024-03-09T09:06:39.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.298+0000 7f1e35200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e300708c0 con 0x7f1e300c30a0 2024-03-09T09:06:39.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.298+0000 7f1e2fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1e30151190 0x7f1e301539e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33286/0 (socket says 172.21.15.67:33286) 2024-03-09T09:06:39.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.298+0000 7f1e2fe00640 1 -- 172.21.15.67:0/2896098468 learned_addr learned my addr 172.21.15.67:0/2896098468 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:06:39.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.298+0000 7f1e2ea00640 1 --2- 172.21.15.67:0/2896098468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e300c30a0 0x7f1e30150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2f400640 1 --2- 172.21.15.67:0/2896098468 >> v2:172.21.15.134:3300/0 conn(0x7f1e300ac930 0x7f1e300c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518125031 0 0) 0x7f1e300c2780 con 0x7f1e30151190 2024-03-09T09:06:39.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e14003520 con 0x7f1e30151190 2024-03-09T09:06:39.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2712083531 0 0) 0x7f1e14003520 con 0x7f1e30151190 2024-03-09T09:06:39.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e300c30a0 msgr2=0x7f1e30150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 --2- 172.21.15.67:0/2896098468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e300c30a0 0x7f1e30150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 >> v2:172.21.15.134:3300/0 conn(0x7f1e300ac930 msgr2=0x7f1e300c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 --2- 172.21.15.67:0/2896098468 >> v2:172.21.15.134:3300/0 conn(0x7f1e300ac930 0x7f1e300c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e30070ba0 con 0x7f1e30151190 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f1e280019f0 con 0x7f1e30151190 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 >> v1:172.21.15.67:6789/0 conn(0x7f1e30151190 legacy=0x7f1e301539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 --2- 172.21.15.67:0/2896098468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e300c30a0 0x7f1e14003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 --2- 172.21.15.67:0/2896098468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e14003ff0 0x7f1e14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 --2- 172.21.15.67:0/2896098468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e14004d40 0x7f1e140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e30070670 con 0x7f1e14003ff0 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e300708c0 con 0x7f1e14004d40 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1400baf0 con 0x7f1e300c30a0 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2f400640 1 --2- 172.21.15.67:0/2896098468 >> v2:172.21.15.134:3300/0 conn(0x7f1e300ac930 0x7f1e300c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:06:39.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.299+0000 7f1e2ea00640 1 --2- 172.21.15.67:0/2896098468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e14003ff0 0x7f1e14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.300+0000 7f1e2ea00640 1 -- 172.21.15.67:0/2896098468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e300c30a0 msgr2=0x7f1e14003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:39.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.300+0000 7f1e2ea00640 1 --2- 172.21.15.67:0/2896098468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e300c30a0 0x7f1e14003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.300+0000 7f1e2ea00640 1 -- 172.21.15.67:0/2896098468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e14004d40 msgr2=0x7f1e140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:39.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.300+0000 7f1e2ea00640 1 --2- 172.21.15.67:0/2896098468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e14004d40 0x7f1e140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.300+0000 7f1e2ea00640 1 -- 172.21.15.67:0/2896098468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e14003520 con 0x7f1e14003ff0 2024-03-09T09:06:39.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.301+0000 7f1e2ea00640 1 --2- 172.21.15.67:0/2896098468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e14003ff0 0x7f1e14004800 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009ac0 tx=0x7f1e14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad84aacf0bc40c03 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.301+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c039070 con 0x7f1e14003ff0 2024-03-09T09:06:39.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.301+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1c02c710 con 0x7f1e14003ff0 2024-03-09T09:06:39.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.301+0000 7f1e2e000640 1 -- 172.21.15.67:0/2896098468 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c009690 con 0x7f1e14003ff0 2024-03-09T09:06:39.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.302+0000 7f1e35200640 1 -- 172.21.15.67:0/2896098468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e14003ff0 msgr2=0x7f1e14004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.302+0000 7f1e35200640 1 --2- 172.21.15.67:0/2896098468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e14003ff0 0x7f1e14004800 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009ac0 tx=0x7f1e14005830 comp rx=0 tx=0).stop 2024-03-09T09:06:39.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.302+0000 7f1e2fe00640 1 -- 172.21.15.67:0/2896098468 reap_dead start 2024-03-09T09:06:39.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.302+0000 7f1e35200640 1 -- 172.21.15.67:0/2896098468 shutdown_connections 2024-03-09T09:06:39.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.302+0000 7f1e35200640 1 -- 172.21.15.67:0/2896098468 >> 172.21.15.67:0/2896098468 conn(0x7f1e300c7220 msgr2=0x7f1e300c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:39.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.302+0000 7f1e35200640 1 -- 172.21.15.67:0/2896098468 shutdown_connections 2024-03-09T09:06:39.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.302+0000 7f1e35200640 1 -- 172.21.15.67:0/2896098468 wait complete. 2024-03-09T09:06:39.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.304+0000 7f1e35200640 1 Processor -- start 2024-03-09T09:06:39.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.304+0000 7f1e35200640 1 -- start start 2024-03-09T09:06:39.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.305+0000 7f1e35200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e14003ff0 0x7f1e301ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.305+0000 7f1e35200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e30147150 0x7f1e300c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.305+0000 7f1e35200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e300c63a0 0x7f1e301479e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.305+0000 7f1e35200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e300acd10 con 0x7f1e300c63a0 2024-03-09T09:06:39.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.305+0000 7f1e35200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e301515e0 con 0x7f1e14003ff0 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.305+0000 7f1e35200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e300ace90 con 0x7f1e30147150 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e300c63a0 0x7f1e301479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e300c63a0 0x7f1e301479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60554/0 (socket says 172.21.15.67:60554) 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2fe00640 1 -- 172.21.15.67:0/1048079815 learned_addr learned my addr 172.21.15.67:0/1048079815 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2ea00640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e30147150 0x7f1e300c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2f400640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e14003ff0 0x7f1e301ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2fe00640 1 -- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e30147150 msgr2=0x7f1e300c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2fe00640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e30147150 0x7f1e300c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2fe00640 1 -- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e14003ff0 msgr2=0x7f1e301ebb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2fe00640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e14003ff0 0x7f1e301ebb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2fe00640 1 -- 172.21.15.67:0/1048079815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e301ec080 con 0x7f1e300c63a0 2024-03-09T09:06:39.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2ea00640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e30147150 0x7f1e300c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:06:39.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.306+0000 7f1e2fe00640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e300c63a0 0x7f1e301479e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1e28001370 tx=0x7f1e2800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.307+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28004d60 con 0x7f1e300c63a0 2024-03-09T09:06:39.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.307+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2800a610 con 0x7f1e300c63a0 2024-03-09T09:06:39.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.307+0000 7f1e35200640 1 -- 172.21.15.67:0/1048079815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e301eb0f0 con 0x7f1e300c63a0 2024-03-09T09:06:39.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.307+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2800aa00 con 0x7f1e300c63a0 2024-03-09T09:06:39.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.307+0000 7f1e35200640 1 -- 172.21.15.67:0/1048079815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3014d110 con 0x7f1e300c63a0 2024-03-09T09:06:39.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.308+0000 7f1e35200640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e3014d350 0x7f1e30147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.308+0000 7f1e35200640 1 -- 172.21.15.67:0/1048079815 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3004f3c0 con 0x7f1e3014d350 2024-03-09T09:06:39.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.308+0000 7f1e2f400640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e3014d350 0x7f1e30147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.309+0000 7f1e2f400640 1 -- 172.21.15.67:0/1048079815 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e300c3760 con 0x7f1e3014d350 2024-03-09T09:06:39.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.309+0000 7f1e2f400640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e3014d350 0x7f1e30147f20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e20006fd0 tx=0x7f1e20010c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.310+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2001a020 con 0x7f1e3014d350 2024-03-09T09:06:39.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.310+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1e28011070 con 0x7f1e300c63a0 2024-03-09T09:06:39.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.311+0000 7f1e27400640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e1006b3d0 0x7f1e1006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.311+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1e28089f70 con 0x7f1e300c63a0 2024-03-09T09:06:39.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.311+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f1e20010e70 con 0x7f1e3014d350 2024-03-09T09:06:39.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.311+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e3014d350 msgr2=0x7f1e30147f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.311+0000 7f1e27400640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e3014d350 0x7f1e30147f20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e20006fd0 tx=0x7f1e20010c20 comp rx=0 tx=0).stop 2024-03-09T09:06:39.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.311+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e3014d350 msgr2=0x7f1e30147f20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:39.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.311+0000 7f1e27400640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e3014d350 0x7f1e30147f20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.312+0000 7f1e2ea00640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e1006b3d0 0x7f1e1006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.313+0000 7f1e2ea00640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e1006b3d0 0x7f1e1006d890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c036000 tx=0x7f1e1c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.319+0000 7f1e35200640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e301ee660 0x7f1e301eea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.319+0000 7f1e35200640 1 -- 172.21.15.67:0/1048079815 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e301454b0 con 0x7f1e301ee660 2024-03-09T09:06:39.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.319+0000 7f1e2f400640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e301ee660 0x7f1e301eea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.320+0000 7f1e2f400640 1 -- 172.21.15.67:0/1048079815 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1e30148460 con 0x7f1e301ee660 2024-03-09T09:06:39.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.320+0000 7f1e2f400640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e301ee660 0x7f1e301eea40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e20006fd0 tx=0x7f1e200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.321+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20027070 con 0x7f1e301ee660 2024-03-09T09:06:39.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.321+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4847 (secure 0 0 0) 0x7f1e200157c0 con 0x7f1e301ee660 2024-03-09T09:06:39.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.321+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e301ee660 msgr2=0x7f1e301eea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.321+0000 7f1e27400640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e301ee660 0x7f1e301eea40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e20006fd0 tx=0x7f1e200046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.321+0000 7f1e27400640 1 -- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e301ee660 msgr2=0x7f1e301eea40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:39.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.321+0000 7f1e27400640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e301ee660 0x7f1e301eea40 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1e20006fd0 tx=0x7f1e200046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.325 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:06:39.325 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:06:39.325 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 104, 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:39.326 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:39.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:06:39.328 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:06:39.329 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:06:39.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.331 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:39.332 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:39.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:39.334 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:06:39.335 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:39.336 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:06:39.337 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:06:39.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.322+0000 7f1e35200640 1 -- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e1006b3d0 msgr2=0x7f1e1006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.322+0000 7f1e35200640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e1006b3d0 0x7f1e1006d890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c036000 tx=0x7f1e1c030000 comp rx=0 tx=0).stop 2024-03-09T09:06:39.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.322+0000 7f1e35200640 1 -- 172.21.15.67:0/1048079815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e300c63a0 msgr2=0x7f1e301479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.322+0000 7f1e35200640 1 --2- 172.21.15.67:0/1048079815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e300c63a0 0x7f1e301479e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1e28001370 tx=0x7f1e2800cb90 comp rx=0 tx=0).stop 2024-03-09T09:06:39.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.322+0000 7f1e2fe00640 1 -- 172.21.15.67:0/1048079815 reap_dead start 2024-03-09T09:06:39.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.322+0000 7f1e2fe00640 1 -- 172.21.15.67:0/1048079815 reap_dead start 2024-03-09T09:06:39.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.322+0000 7f1e35200640 1 -- 172.21.15.67:0/1048079815 shutdown_connections 2024-03-09T09:06:39.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.322+0000 7f1e35200640 1 -- 172.21.15.67:0/1048079815 >> 172.21.15.67:0/1048079815 conn(0x7f1e300c7220 msgr2=0x7f1e300c0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:39.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.322+0000 7f1e35200640 1 -- 172.21.15.67:0/1048079815 shutdown_connections 2024-03-09T09:06:39.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.322+0000 7f1e35200640 1 -- 172.21.15.67:0/1048079815 wait complete. 2024-03-09T09:06:39.340 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:06:39.340 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:06:39.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:06:39.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.532+0000 7f6353e00640 1 Processor -- start 2024-03-09T09:06:39.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.532+0000 7f6353e00640 1 -- start start 2024-03-09T09:06:39.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.534+0000 7f6353e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6354153300 0x7f63540b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.534+0000 7f6353e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6354157000 con 0x7f63540c5e40 2024-03-09T09:06:39.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.534+0000 7f6353e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63540ae430 con 0x7f63540cadb0 2024-03-09T09:06:39.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.534+0000 7f6353e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63540ae6f0 con 0x7f6354153300 2024-03-09T09:06:39.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.534+0000 7f6352000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f63540c5e40 0x7f6354156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33288/0 (socket says 172.21.15.67:33288) 2024-03-09T09:06:39.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.534+0000 7f6352000640 1 -- 172.21.15.67:0/783487887 learned_addr learned my addr 172.21.15.67:0/783487887 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:06:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.534+0000 7f6352a00640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6354153300 0x7f63540b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389715262 0 0) 0x7f6354157000 con 0x7f63540c5e40 2024-03-09T09:06:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6338003520 con 0x7f63540c5e40 2024-03-09T09:06:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3187339344 0 0) 0x7f6338003520 con 0x7f63540c5e40 2024-03-09T09:06:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6354153300 msgr2=0x7f63540b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6354153300 0x7f63540b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 >> v1:172.21.15.134:6789/0 conn(0x7f63540cadb0 legacy=0x7f6354160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354155b10 con 0x7f63540c5e40 2024-03-09T09:06:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f6340001670 con 0x7f63540c5e40 2024-03-09T09:06:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 >> v1:172.21.15.67:6789/0 conn(0x7f63540c5e40 legacy=0x7f6354156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6338003a50 0x7f6338003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338004800 0x7f6338004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6354153300 0x7f63380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540ae6f0 con 0x7f6338003a50 2024-03-09T09:06:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633800bac0 con 0x7f6338004800 2024-03-09T09:06:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633800bcc0 con 0x7f6354153300 2024-03-09T09:06:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.535+0000 7f6352a00640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6338003a50 0x7f6338003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.536+0000 7f6352000640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338004800 0x7f6338004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.536+0000 7f6353400640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6354153300 0x7f63380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.536+0000 7f6352a00640 1 -- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6354153300 msgr2=0x7f63380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.536+0000 7f6352a00640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6354153300 0x7f63380051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.536+0000 7f6352a00640 1 -- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338004800 msgr2=0x7f6338004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.536+0000 7f6352a00640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338004800 0x7f6338004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.536+0000 7f6352a00640 1 -- 172.21.15.67:0/783487887 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6338003520 con 0x7f6338003a50 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.536+0000 7f6352000640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338004800 0x7f6338004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.537+0000 7f6352a00640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6338003a50 0x7f6338003e70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6344009ac0 tx=0x7f6338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5df44073265b50 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.537+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344039070 con 0x7f6338003a50 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.537+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634402c6b0 con 0x7f6338003a50 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.537+0000 7f6351600640 1 -- 172.21.15.67:0/783487887 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6344009690 con 0x7f6338003a50 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.538+0000 7f6353e00640 1 -- 172.21.15.67:0/783487887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6338003a50 msgr2=0x7f6338003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.538+0000 7f6353e00640 1 --2- 172.21.15.67:0/783487887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6338003a50 0x7f6338003e70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6344009ac0 tx=0x7f6338005830 comp rx=0 tx=0).stop 2024-03-09T09:06:39.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.538+0000 7f6353400640 1 -- 172.21.15.67:0/783487887 reap_dead start 2024-03-09T09:06:39.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.538+0000 7f6353e00640 1 -- 172.21.15.67:0/783487887 shutdown_connections 2024-03-09T09:06:39.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.538+0000 7f6353e00640 1 -- 172.21.15.67:0/783487887 >> 172.21.15.67:0/783487887 conn(0x7f63540732d0 msgr2=0x7f63540ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:39.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.538+0000 7f6353e00640 1 -- 172.21.15.67:0/783487887 shutdown_connections 2024-03-09T09:06:39.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.538+0000 7f6353e00640 1 -- 172.21.15.67:0/783487887 wait complete. 2024-03-09T09:06:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.541+0000 7f6353e00640 1 Processor -- start 2024-03-09T09:06:39.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.541+0000 7f6353e00640 1 -- start start 2024-03-09T09:06:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.542+0000 7f6353e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338003a50 0x7f63540c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.542+0000 7f6353e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63541ef6f0 0x7f635414bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.543+0000 7f6353e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c8080 0x7f635414c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.543+0000 7f6353e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540c59e0 con 0x7f63540c8080 2024-03-09T09:06:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.543+0000 7f6353e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6354159e90 con 0x7f6338003a50 2024-03-09T09:06:39.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.543+0000 7f6353e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6354156720 con 0x7f63541ef6f0 2024-03-09T09:06:39.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.543+0000 7f6353400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c8080 0x7f635414c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.543+0000 7f6353400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c8080 0x7f635414c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60574/0 (socket says 172.21.15.67:60574) 2024-03-09T09:06:39.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.543+0000 7f6353400640 1 -- 172.21.15.67:0/4253912227 learned_addr learned my addr 172.21.15.67:0/4253912227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:39.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.543+0000 7f6352a00640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338003a50 0x7f63540c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.543+0000 7f6352000640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63541ef6f0 0x7f635414bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f6353400640 1 -- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63541ef6f0 msgr2=0x7f635414bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f6353400640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63541ef6f0 0x7f635414bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f6353400640 1 -- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338003a50 msgr2=0x7f63540c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f6353400640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338003a50 0x7f63540c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f6353400640 1 -- 172.21.15.67:0/4253912227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635406b7d0 con 0x7f63540c8080 2024-03-09T09:06:39.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f6352000640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63541ef6f0 0x7f635414bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:06:39.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f6352a00640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338003a50 0x7f63540c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:06:39.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f6353400640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c8080 0x7f635414c060 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f63540ade40 tx=0x7f634c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f634aa00640 1 -- 172.21.15.67:0/4253912227 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c018070 con 0x7f63540c8080 2024-03-09T09:06:39.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f634aa00640 1 -- 172.21.15.67:0/4253912227 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c003b70 con 0x7f63540c8080 2024-03-09T09:06:39.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f634aa00640 1 -- 172.21.15.67:0/4253912227 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c005070 con 0x7f63540c8080 2024-03-09T09:06:39.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.544+0000 7f6353e00640 1 -- 172.21.15.67:0/4253912227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635406bab0 con 0x7f63540c8080 2024-03-09T09:06:39.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.545+0000 7f6353e00640 1 -- 172.21.15.67:0/4253912227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63540c9a70 con 0x7f63540c8080 2024-03-09T09:06:39.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.546+0000 7f6353e00640 1 -- 172.21.15.67:0/4253912227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63541efbd0 con 0x7f63540c8080 2024-03-09T09:06:39.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.547+0000 7f634aa00640 1 -- 172.21.15.67:0/4253912227 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f634c01b070 con 0x7f63540c8080 2024-03-09T09:06:39.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.548+0000 7f634aa00640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f633406b520 0x7f633406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:39.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.549+0000 7f634aa00640 1 -- 172.21.15.67:0/4253912227 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f634c089c60 con 0x7f63540c8080 2024-03-09T09:06:39.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.549+0000 7f6352a00640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f633406b520 0x7f633406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:39.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.551+0000 7f6352a00640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f633406b520 0x7f633406d9e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6344036000 tx=0x7f6344030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:39.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.553+0000 7f634aa00640 1 -- 172.21.15.67:0/4253912227 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f634c058760 con 0x7f63540c8080 2024-03-09T09:06:39.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.875+0000 7f6353e00640 1 -- 172.21.15.67:0/4253912227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f63540c57c0 con 0x7f63540c8080 2024-03-09T09:06:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.877+0000 7f634aa00640 1 -- 172.21.15.67:0/4253912227 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f634c058060 con 0x7f63540c8080 2024-03-09T09:06:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 -- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f633406b520 msgr2=0x7f633406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f633406b520 0x7f633406d9e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6344036000 tx=0x7f6344030000 comp rx=0 tx=0).stop 2024-03-09T09:06:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 -- 172.21.15.67:0/4253912227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c8080 msgr2=0x7f635414c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c8080 0x7f635414c060 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f63540ade40 tx=0x7f634c00dc00 comp rx=0 tx=0).stop 2024-03-09T09:06:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 -- 172.21.15.67:0/4253912227 shutdown_connections 2024-03-09T09:06:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f633406b520 0x7f633406d9e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63540c8080 0x7f635414c060 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63541ef6f0 0x7f635414bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 --2- 172.21.15.67:0/4253912227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6338003a50 0x7f63540c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 -- 172.21.15.67:0/4253912227 >> 172.21.15.67:0/4253912227 conn(0x7f63540732d0 msgr2=0x7f6354157220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 -- 172.21.15.67:0/4253912227 shutdown_connections 2024-03-09T09:06:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:39.881+0000 7f6353e00640 1 -- 172.21.15.67:0/4253912227 wait complete. 2024-03-09T09:06:39.902 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:06:45.646 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:06:45.651 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:06:45.817 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 15% (117/779) Updating files: 16% (125/779) Updating files: 17% (133/779) Updating files: 18% (141/779) Updating files: 19% (149/779) Updating files: 20% (156/779) Updating files: 20% (161/779) Updating files: 21% (164/779) Updating files: 22% (172/779) Updating files: 23% (180/779) Updating files: 24% (187/779) Updating files: 25% (195/779) Updating files: 26% (203/779) Updating files: 27% (211/779) Updating files: 28% (219/779) Updating files: 29% (226/779) Updating files: 30% (234/779) Updating files: 30% (236/779) Updating files: 31% (242/779) Updating files: 32% (250/779) Updating files: 33% (258/779) Updating files: 33% (263/779) Updating files: 34% (265/779) Updating files: 35% (273/779) Updating files: 36% (281/779) Updating files: 37% (289/779) Updating files: 38% (297/779) Updating files: 39% (304/779) Updating files: 40% (312/779) Updating files: 41% (320/779) Updating files: 42% (328/779) Updating files: 42% (332/779) Updating files: 43% (335/779) Updating files: 44% (343/779) Updating files: 45% (351/779) Updating files: 46% (359/779) Updating files: 47% (367/779) Updating files: 48% (374/779) Updating files: 49% (382/779) Updating files: 50% (390/779) Updating files: 50% (396/779) Updating files: 51% (398/779) Updating files: 52% (406/779) Updating files: 53% (413/779) Updating files: 54% (421/779) Updating files: 55% (429/779) Updating files: 56% (437/779) Updating files: 56% (438/779) Updating files: 57% (445/779) Updating files: 58% (452/779) Updating files: 59% (460/779) Updating files: 60% (468/779) Updating files: 61% (476/779) Updating files: 62% (483/779) Updating files: 63% (491/779) Updating files: 64% (499/779) Updating files: 65% (507/779) Updating files: 66% (515/779) Updating files: 67% (522/779) Updating files: 68% (530/779) Updating files: 69% (538/779) Updating files: 70% (546/779) Updating files: 71% (554/779) Updating files: 72% (561/779) Updating files: 73% (569/779) Updating files: 74% (577/779) Updating files: 75% (585/779) Updating files: 76% (593/779) Updating files: 77% (600/779) Updating files: 78% (608/779) Updating files: 79% (616/779) Updating files: 80% (624/779) Updating files: 81% (631/779) Updating files: 82% (639/779) Updating files: 83% (647/779) Updating files: 83% (651/779) Updating files: 84% (655/779) Updating files: 85% (663/779) Updating files: 86% (670/779) Updating files: 87% (678/779) Updating files: 88% (686/779) Updating files: 89% (694/779) Updating files: 90% (702/779) Updating files: 91% (709/779) Updating files: 92% (717/779) Updating files: 92% (723/779) Updating files: 93% (725/779) Updating files: 94% (733/779) Updating files: 95% (741/779) Updating files: 96% (748/779) Updating files: 97% (756/779) Updating files: 98% (764/779) Updating files: 99% (772/779) Updating files: 100% (779/779) Updating files: 100% (779/779), done. 2024-03-09T09:06:45.842 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 2f10137986a v0.18 2024-03-09T09:06:45.846 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.18 2024-03-09T09:06:46.006 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:06:46.006 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 19 -eq 48 ']' 2024-03-09T09:06:46.006 INFO:tasks.workunit.client.1.smithi134.stdout:v0.19 2024-03-09T09:06:46.006 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.19 2024-03-09T09:06:46.006 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.19 2024-03-09T09:06:46.006 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.19 2024-03-09T09:06:47.167 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 3% (30/779) Updating files: 4% (32/779) Updating files: 5% (39/779) Updating files: 6% (47/779) Updating files: 7% (55/779) Updating files: 8% (63/779) Updating files: 9% (71/779) Updating files: 10% (78/779) Updating files: 11% (86/779) Updating files: 12% (94/779) Updating files: 13% (102/779) Updating files: 13% (104/779) Updating files: 14% (110/779) Updating files: 15% (117/779) Updating files: 16% (125/779) Updating files: 17% (133/779) Updating files: 18% (141/779) Updating files: 19% (149/779) Updating files: 20% (156/779) Updating files: 21% (164/779) Updating files: 22% (172/779) Updating files: 23% (180/779) Updating files: 24% (187/779) Updating files: 25% (195/779) Updating files: 25% (202/779) Updating files: 26% (203/779) Updating files: 27% (211/779) Updating files: 27% (216/779) Updating files: 28% (219/779) Updating files: 29% (226/779) Updating files: 30% (234/779) Updating files: 31% (242/779) Updating files: 32% (250/779) Updating files: 33% (258/779) Updating files: 34% (265/779) Updating files: 35% (273/779) Updating files: 36% (281/779) Updating files: 37% (289/779) Updating files: 38% (297/779) Updating files: 39% (304/779) Updating files: 40% (312/779) Updating files: 41% (320/779) Updating files: 42% (328/779) Updating files: 43% (335/779) Updating files: 44% (343/779) Updating files: 45% (351/779) Updating files: 46% (359/779) Updating files: 47% (367/779) Updating files: 48% (374/779) Updating files: 49% (382/779) Updating files: 50% (390/779) Updating files: 51% (398/779) Updating files: 52% (406/779) Updating files: 53% (413/779) Updating files: 54% (421/779) Updating files: 55% (429/779) Updating files: 55% (434/779) Updating files: 56% (437/779) Updating files: 57% (445/779) Updating files: 58% (452/779) Updating files: 59% (460/779) Updating files: 60% (468/779) Updating files: 61% (476/779) Updating files: 62% (483/779) Updating files: 63% (491/779) Updating files: 64% (499/779) Updating files: 65% (507/779) Updating files: 66% (515/779) Updating files: 67% (522/779) Updating files: 68% (530/779) Updating files: 69% (538/779) Updating files: 70% (546/779) Updating files: 71% (554/779) Updating files: 72% (561/779) Updating files: 73% (569/779) Updating files: 74% (577/779) Updating files: 75% (585/779) Updating files: 76% (593/779) Updating files: 77% (600/779) Updating files: 78% (608/779) Updating files: 79% (616/779) Updating files: 80% (624/779) Updating files: 81% (631/779) Updating files: 82% (639/779) Updating files: 83% (647/779) Updating files: 84% (655/779) Updating files: 85% (663/779) Updating files: 86% (670/779) Updating files: 87% (678/779) Updating files: 88% (686/779) Updating files: 89% (694/779) Updating files: 90% (702/779) Updating files: 91% (709/779) Updating files: 92% (717/779) Updating files: 93% (725/779) Updating files: 94% (733/779) Updating files: 94% (740/779) Updating files: 95% (741/779) Updating files: 96% (748/779) Updating files: 96% (749/779) Updating files: 96% (752/779) Updating files: 97% (756/779) Updating files: 98% (764/779) Updating files: 99% (772/779) Updating files: 100% (779/779) Updating files: 100% (779/779), done. 2024-03-09T09:06:47.292 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 2f10137986a v0.18 2024-03-09T09:06:47.301 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.18 2024-03-09T09:06:47.359 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:47.359 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 19 -eq 48 ']' 2024-03-09T09:06:47.359 INFO:tasks.workunit.client.0.smithi067.stdout:v0.19 2024-03-09T09:06:47.360 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.19 2024-03-09T09:06:47.360 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.19 2024-03-09T09:06:47.360 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.19 2024-03-09T09:06:49.314 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 87% (189/217) Updating files: 88% (191/217) Updating files: 89% (194/217) Updating files: 90% (196/217) Updating files: 91% (198/217) Updating files: 92% (200/217) Updating files: 93% (202/217) Updating files: 94% (204/217) Updating files: 95% (207/217) Updating files: 96% (209/217) Updating files: 97% (211/217) Updating files: 97% (212/217) Updating files: 98% (213/217) Updating files: 99% (215/217) Updating files: 100% (217/217) Updating files: 100% (217/217), done. 2024-03-09T09:06:49.381 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 98f5be53f85 v0.19 2024-03-09T09:06:49.391 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.19 2024-03-09T09:06:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:v0.20 2024-03-09T09:06:49.727 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:06:49.727 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 20 -eq 48 ']' 2024-03-09T09:06:49.727 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.20 2024-03-09T09:06:49.727 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.20 2024-03-09T09:06:49.727 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.20 2024-03-09T09:06:49.903 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:06:49.903 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:06:50.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.131+0000 7f26cb600640 1 Processor -- start 2024-03-09T09:06:50.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.131+0000 7f26cb600640 1 -- start start 2024-03-09T09:06:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.133+0000 7f26cb600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f26c4153370 0x7f26c40c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.133+0000 7f26cb600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c4156d50 con 0x7f26c40c1d70 2024-03-09T09:06:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.133+0000 7f26cb600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c40aa4c0 con 0x7f26c40c2400 2024-03-09T09:06:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.133+0000 7f26cb600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c40aadf0 con 0x7f26c4153370 2024-03-09T09:06:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.133+0000 7f26c3e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f26c40c1d70 0x7f26c40ac610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59680/0 (socket says 172.21.15.67:59680) 2024-03-09T09:06:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.133+0000 7f26c3e00640 1 -- 172.21.15.67:0/976836551 learned_addr learned my addr 172.21.15.67:0/976836551 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:06:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.133+0000 7f26c8c00640 1 --2- 172.21.15.67:0/976836551 >> v2:172.21.15.134:3300/0 conn(0x7f26c4153370 0x7f26c40c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.134+0000 7f26c3400640 1 -- 172.21.15.67:0/976836551 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774730784 0 0) 0x7f26c4156d50 con 0x7f26c40c1d70 2024-03-09T09:06:50.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.134+0000 7f26c3400640 1 -- 172.21.15.67:0/976836551 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26ac003520 con 0x7f26c40c1d70 2024-03-09T09:06:50.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.134+0000 7f26c8c00640 1 -- 172.21.15.67:0/976836551 >> v1:172.21.15.134:6789/0 conn(0x7f26c40c2400 legacy=0x7f26c40acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.134+0000 7f26c8c00640 1 -- 172.21.15.67:0/976836551 >> v1:172.21.15.67:6789/0 conn(0x7f26c40c1d70 legacy=0x7f26c40ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.134+0000 7f26c8c00640 1 -- 172.21.15.67:0/976836551 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c40aa7a0 con 0x7f26c4153370 2024-03-09T09:06:50.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.135+0000 7f26c8c00640 1 --2- 172.21.15.67:0/976836551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c4153370 0x7f26c40c07a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f26c40aaf70 tx=0x7f26b802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9bb5514ccb3c6e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.136+0000 7f26c3400640 1 -- 172.21.15.67:0/976836551 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8031780 con 0x7f26c4153370 2024-03-09T09:06:50.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.136+0000 7f26c3400640 1 -- 172.21.15.67:0/976836551 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b8037070 con 0x7f26c4153370 2024-03-09T09:06:50.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.136+0000 7f26c3400640 1 -- 172.21.15.67:0/976836551 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8031b40 con 0x7f26c4153370 2024-03-09T09:06:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.136+0000 7f26cb600640 1 -- 172.21.15.67:0/976836551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c4153370 msgr2=0x7f26c40c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.136+0000 7f26cb600640 1 --2- 172.21.15.67:0/976836551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c4153370 0x7f26c40c07a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f26c40aaf70 tx=0x7f26b802b820 comp rx=0 tx=0).stop 2024-03-09T09:06:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.136+0000 7f26cb600640 1 -- 172.21.15.67:0/976836551 shutdown_connections 2024-03-09T09:06:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.136+0000 7f26cb600640 1 --2- 172.21.15.67:0/976836551 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c4153370 0x7f26c40c07a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.136+0000 7f26cb600640 1 -- 172.21.15.67:0/976836551 >> 172.21.15.67:0/976836551 conn(0x7f26c40c7370 msgr2=0x7f26c40c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:50.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.137+0000 7f26cb600640 1 -- 172.21.15.67:0/976836551 shutdown_connections 2024-03-09T09:06:50.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.137+0000 7f26cb600640 1 -- 172.21.15.67:0/976836551 wait complete. 2024-03-09T09:06:50.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.139+0000 7f26cb600640 1 Processor -- start 2024-03-09T09:06:50.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.139+0000 7f26cb600640 1 -- start start 2024-03-09T09:06:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.141+0000 7f26cb600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c40c1d70 0x7f26c41459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.141+0000 7f26cb600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26c40c2400 0x7f26c4145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.141+0000 7f26cb600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c4153370 0x7f26c40c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.141+0000 7f26cb600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c41538d0 con 0x7f26c40c1d70 2024-03-09T09:06:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.141+0000 7f26cb600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c4153750 con 0x7f26c4153370 2024-03-09T09:06:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.141+0000 7f26cb600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c41568e0 con 0x7f26c40c2400 2024-03-09T09:06:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.142+0000 7f26c8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c40c1d70 0x7f26c41459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.142+0000 7f26c8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c40c1d70 0x7f26c41459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34644/0 (socket says 172.21.15.67:34644) 2024-03-09T09:06:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.142+0000 7f26c8c00640 1 -- 172.21.15.67:0/155077089 learned_addr learned my addr 172.21.15.67:0/155077089 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.142+0000 7f26c3e00640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26c40c2400 0x7f26c4145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.142+0000 7f26c9600640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c4153370 0x7f26c40c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.143+0000 7f26c8c00640 1 -- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26c40c2400 msgr2=0x7f26c4145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.143+0000 7f26c8c00640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26c40c2400 0x7f26c4145f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.143+0000 7f26c8c00640 1 -- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c4153370 msgr2=0x7f26c40c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.143+0000 7f26c8c00640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c4153370 0x7f26c40c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.143+0000 7f26c8c00640 1 -- 172.21.15.67:0/155077089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c40c2d80 con 0x7f26c40c1d70 2024-03-09T09:06:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.143+0000 7f26c8c00640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c40c1d70 0x7f26c41459e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009890 tx=0x7f26b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.144+0000 7f26c0c00640 1 -- 172.21.15.67:0/155077089 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8032610 con 0x7f26c40c1d70 2024-03-09T09:06:50.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.144+0000 7f26c0c00640 1 -- 172.21.15.67:0/155077089 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b80327d0 con 0x7f26c40c1d70 2024-03-09T09:06:50.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.144+0000 7f26c0c00640 1 -- 172.21.15.67:0/155077089 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8032bc0 con 0x7f26c40c1d70 2024-03-09T09:06:50.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.144+0000 7f26cb600640 1 -- 172.21.15.67:0/155077089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c41eba90 con 0x7f26c40c1d70 2024-03-09T09:06:50.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.144+0000 7f26cb600640 1 -- 172.21.15.67:0/155077089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c40c54f0 con 0x7f26c40c1d70 2024-03-09T09:06:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.146+0000 7f26cb600640 1 -- 172.21.15.67:0/155077089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c4075a00 con 0x7f26c40c1d70 2024-03-09T09:06:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.148+0000 7f26c0c00640 1 -- 172.21.15.67:0/155077089 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f26b80453f0 con 0x7f26c40c1d70 2024-03-09T09:06:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.149+0000 7f26c0c00640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26a806b5a0 0x7f26a806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.149+0000 7f26c0c00640 1 -- 172.21.15.67:0/155077089 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f26b80ac2e0 con 0x7f26c40c1d70 2024-03-09T09:06:50.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.150+0000 7f26c3e00640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26a806b5a0 0x7f26a806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.151+0000 7f26c3e00640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26a806b5a0 0x7f26a806da60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f26c41ec8d0 tx=0x7f26b4007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.158+0000 7f26c0c00640 1 -- 172.21.15.67:0/155077089 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f26b8040070 con 0x7f26c40c1d70 2024-03-09T09:06:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.482+0000 7f26cb600640 1 -- 172.21.15.67:0/155077089 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f26c4149160 con 0x7f26c40c1d70 2024-03-09T09:06:50.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.483+0000 7f26c0c00640 1 -- 172.21.15.67:0/155077089 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f26b802f050 con 0x7f26c40c1d70 2024-03-09T09:06:50.487 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":104,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:06:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 -- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26a806b5a0 msgr2=0x7f26a806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26a806b5a0 0x7f26a806da60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f26c41ec8d0 tx=0x7f26b4007d30 comp rx=0 tx=0).stop 2024-03-09T09:06:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 -- 172.21.15.67:0/155077089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c40c1d70 msgr2=0x7f26c41459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c40c1d70 0x7f26c41459e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009890 tx=0x7f26b80046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 -- 172.21.15.67:0/155077089 shutdown_connections 2024-03-09T09:06:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26a806b5a0 0x7f26a806da60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26c4153370 0x7f26c40c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26c40c2400 0x7f26c4145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 --2- 172.21.15.67:0/155077089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26c40c1d70 0x7f26c41459e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 -- 172.21.15.67:0/155077089 >> 172.21.15.67:0/155077089 conn(0x7f26c40c7370 msgr2=0x7f26c406d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 -- 172.21.15.67:0/155077089 shutdown_connections 2024-03-09T09:06:50.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.486+0000 7f26cb600640 1 -- 172.21.15.67:0/155077089 wait complete. 2024-03-09T09:06:50.507 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:06:50.507 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:06:50.507 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:06:50.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:06:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.708+0000 7f87d2000640 1 Processor -- start 2024-03-09T09:06:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.709+0000 7f87d2000640 1 -- start start 2024-03-09T09:06:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.710+0000 7f87d2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f87cc0ac9a0 0x7f87cc1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.710+0000 7f87d2000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87cc0c2780 0x7f87cc151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.710+0000 7f87d2000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc0c8200 con 0x7f87cc071190 2024-03-09T09:06:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.711+0000 7f87d2000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc06d590 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.711+0000 7f87d2000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0c3740 con 0x7f87cc0c2780 2024-03-09T09:06:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.711+0000 7f87cb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f87cc0ac9a0 0x7f87cc1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.711+0000 7f87cb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f87cc0ac9a0 0x7f87cc1510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34650/0 (socket says 172.21.15.67:34650) 2024-03-09T09:06:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.711+0000 7f87cb400640 1 -- 172.21.15.67:0/980580370 learned_addr learned my addr 172.21.15.67:0/980580370 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.711+0000 7f87caa00640 1 --2- 172.21.15.67:0/980580370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87cc0c2780 0x7f87cc151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.712+0000 7f87cb400640 1 -- 172.21.15.67:0/980580370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87cc0c2780 msgr2=0x7f87cc151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.712+0000 7f87cb400640 1 --2- 172.21.15.67:0/980580370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87cc0c2780 0x7f87cc151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.712+0000 7f87cb400640 1 -- 172.21.15.67:0/980580370 >> v1:172.21.15.67:6789/0 conn(0x7f87cc071190 legacy=0x7f87cc154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.712+0000 7f87cb400640 1 -- 172.21.15.67:0/980580370 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc0c3990 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.714+0000 7f87cb400640 1 --2- 172.21.15.67:0/980580370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0ac9a0 0x7f87cc1510e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009870 tx=0x7f87bc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66131db349f2857 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.715+0000 7f87ca000640 1 -- 172.21.15.67:0/980580370 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc035020 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.715+0000 7f87ca000640 1 -- 172.21.15.67:0/980580370 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc031640 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.715+0000 7f87ca000640 1 -- 172.21.15.67:0/980580370 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc02d040 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.716+0000 7f87d2000640 1 -- 172.21.15.67:0/980580370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0ac9a0 msgr2=0x7f87cc1510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.716+0000 7f87d2000640 1 --2- 172.21.15.67:0/980580370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0ac9a0 0x7f87cc1510e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009870 tx=0x7f87bc02b8f0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.717+0000 7f87d2000640 1 -- 172.21.15.67:0/980580370 shutdown_connections 2024-03-09T09:06:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.717+0000 7f87d2000640 1 --2- 172.21.15.67:0/980580370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87cc0c2780 0x7f87cc151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.717+0000 7f87d2000640 1 --2- 172.21.15.67:0/980580370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0ac9a0 0x7f87cc1510e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.717+0000 7f87d2000640 1 -- 172.21.15.67:0/980580370 >> 172.21.15.67:0/980580370 conn(0x7f87cc0c7370 msgr2=0x7f87cc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:50.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.717+0000 7f87d2000640 1 -- 172.21.15.67:0/980580370 shutdown_connections 2024-03-09T09:06:50.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.717+0000 7f87d2000640 1 -- 172.21.15.67:0/980580370 wait complete. 2024-03-09T09:06:50.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.719+0000 7f87d2000640 1 Processor -- start 2024-03-09T09:06:50.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.719+0000 7f87d2000640 1 -- start start 2024-03-09T09:06:50.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.721+0000 7f87d2000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87cc071190 0x7f87cc1ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.722+0000 7f87d2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0ac9a0 0x7f87cc06dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.722+0000 7f87d2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87cc0c2780 0x7f87cc152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.722+0000 7f87d2000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0acd80 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.722+0000 7f87d2000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc070c00 con 0x7f87cc0c2780 2024-03-09T09:06:50.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.722+0000 7f87d2000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0acf00 con 0x7f87cc071190 2024-03-09T09:06:50.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.722+0000 7f87caa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0ac9a0 0x7f87cc06dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.722+0000 7f87cb400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87cc071190 0x7f87cc1ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.722+0000 7f87caa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0ac9a0 0x7f87cc06dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34652/0 (socket says 172.21.15.67:34652) 2024-03-09T09:06:50.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.722+0000 7f87caa00640 1 -- 172.21.15.67:0/1546120083 learned_addr learned my addr 172.21.15.67:0/1546120083 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:50.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.723+0000 7f87cbe00640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87cc0c2780 0x7f87cc152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.723+0000 7f87caa00640 1 -- 172.21.15.67:0/1546120083 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87cc071190 msgr2=0x7f87cc1ecc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.723+0000 7f87caa00640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87cc071190 0x7f87cc1ecc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.723+0000 7f87caa00640 1 -- 172.21.15.67:0/1546120083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87cc0c2780 msgr2=0x7f87cc152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.723+0000 7f87caa00640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87cc0c2780 0x7f87cc152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.723+0000 7f87caa00640 1 -- 172.21.15.67:0/1546120083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc0c6060 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.724+0000 7f87caa00640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0ac9a0 0x7f87cc06dea0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f87b800d600 tx=0x7f87b800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.724+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8004240 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.724+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b80043e0 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.724+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8003830 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.724+0000 7f87d2000640 1 -- 172.21.15.67:0/1546120083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc0c5dc0 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.725+0000 7f87d2000640 1 -- 172.21.15.67:0/1546120083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc0c6ee0 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.727+0000 7f87d2000640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc152740 0x7f87cc143550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.727+0000 7f87d2000640 1 -- 172.21.15.67:0/1546120083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc075a00 con 0x7f87cc152740 2024-03-09T09:06:50.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.728+0000 7f87cb400640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc152740 0x7f87cc143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.729+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f87b80039d0 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.729+0000 7f87cb400640 1 -- 172.21.15.67:0/1546120083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87cc0c4b90 con 0x7f87cc152740 2024-03-09T09:06:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.729+0000 7f87cb400640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc152740 0x7f87cc143550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f87cc158350 tx=0x7f87bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.730+0000 7f87c3400640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f87ac06b520 0x7f87ac06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.730+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f87b8089e70 con 0x7f87cc0ac9a0 2024-03-09T09:06:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.730+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc035040 con 0x7f87cc152740 2024-03-09T09:06:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.730+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f87bc03a840 con 0x7f87cc152740 2024-03-09T09:06:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.730+0000 7f87cbe00640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f87ac06b520 0x7f87ac06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.730+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc152740 msgr2=0x7f87cc143550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.730+0000 7f87c3400640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc152740 0x7f87cc143550 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f87cc158350 tx=0x7f87bc004730 comp rx=0 tx=0).stop 2024-03-09T09:06:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.730+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc152740 msgr2=0x7f87cc143550 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.730+0000 7f87c3400640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc152740 0x7f87cc143550 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.731+0000 7f87cbe00640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f87ac06b520 0x7f87ac06d9e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f87c4001c30 tx=0x7f87c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.738+0000 7f87d2000640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0c6910 0x7f87cc143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.738+0000 7f87d2000640 1 -- 172.21.15.67:0/1546120083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0746e0 con 0x7f87cc0c6910 2024-03-09T09:06:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.738+0000 7f87cb400640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0c6910 0x7f87cc143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.738+0000 7f87cb400640 1 -- 172.21.15.67:0/1546120083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f87cc0c7120 con 0x7f87cc0c6910 2024-03-09T09:06:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.739+0000 7f87cb400640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0c6910 0x7f87cc143a90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009870 tx=0x7f87bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.739+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc035030 con 0x7f87cc0c6910 2024-03-09T09:06:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.739+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7f87bc009490 con 0x7f87cc0c6910 2024-03-09T09:06:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.739+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0c6910 msgr2=0x7f87cc143a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.739+0000 7f87c3400640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0c6910 0x7f87cc143a90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009870 tx=0x7f87bc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.739+0000 7f87c3400640 1 -- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0c6910 msgr2=0x7f87cc143a90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.739+0000 7f87c3400640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0c6910 0x7f87cc143a90 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009870 tx=0x7f87bc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.743 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:06:50.743 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 104, 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:06:50.744 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:06:50.745 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.746 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:06:50.747 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:50.748 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.750 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.751 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.752 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:50.753 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:06:50.754 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:06:50.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.756 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:06:50.756 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:50.756 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:50.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.756 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.756 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:50.756 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:50.756 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:06:50.756 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.756 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:50.757 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:06:50.757 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.757 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:50.757 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:50.757 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:06:50.757 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.757 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.757 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:06:50.757 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:06:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.740+0000 7f87d2000640 1 -- 172.21.15.67:0/1546120083 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f87ac06b520 msgr2=0x7f87ac06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.740+0000 7f87d2000640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f87ac06b520 0x7f87ac06d9e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f87c4001c30 tx=0x7f87c4009040 comp rx=0 tx=0).stop 2024-03-09T09:06:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.740+0000 7f87d2000640 1 -- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0ac9a0 msgr2=0x7f87cc06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.740+0000 7f87d2000640 1 --2- 172.21.15.67:0/1546120083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87cc0ac9a0 0x7f87cc06dea0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f87b800d600 tx=0x7f87b800dac0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.740+0000 7f87cbe00640 1 -- 172.21.15.67:0/1546120083 reap_dead start 2024-03-09T09:06:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.740+0000 7f87cbe00640 1 -- 172.21.15.67:0/1546120083 reap_dead start 2024-03-09T09:06:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.740+0000 7f87d2000640 1 -- 172.21.15.67:0/1546120083 shutdown_connections 2024-03-09T09:06:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.740+0000 7f87d2000640 1 -- 172.21.15.67:0/1546120083 >> 172.21.15.67:0/1546120083 conn(0x7f87cc0c7370 msgr2=0x7f87cc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.740+0000 7f87d2000640 1 -- 172.21.15.67:0/1546120083 shutdown_connections 2024-03-09T09:06:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.740+0000 7f87d2000640 1 -- 172.21.15.67:0/1546120083 wait complete. 2024-03-09T09:06:50.760 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:06:50.760 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:06:50.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:06:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.941+0000 7fb543200640 1 Processor -- start 2024-03-09T09:06:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.941+0000 7fb543200640 1 -- start start 2024-03-09T09:06:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.943+0000 7fb543200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5440c9380 0x7fb5440acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.943+0000 7fb543200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5440c9890 con 0x7fb5440ad440 2024-03-09T09:06:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.943+0000 7fb543200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5440c5f80 con 0x7fb54414fd90 2024-03-09T09:06:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.943+0000 7fb543200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54406ba60 con 0x7fb5440c9380 2024-03-09T09:06:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.943+0000 7fb542800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb5440ad440 0x7fb54406c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59700/0 (socket says 172.21.15.67:59700) 2024-03-09T09:06:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.944+0000 7fb542800640 1 -- 172.21.15.67:0/1073404249 learned_addr learned my addr 172.21.15.67:0/1073404249 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:06:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.944+0000 7fb541400640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5440c9380 0x7fb5440acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.944+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551852081 0 0) 0x7fb5440c9890 con 0x7fb5440ad440 2024-03-09T09:06:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.944+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb528003520 con 0x7fb5440ad440 2024-03-09T09:06:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.944+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470123730 0 0) 0x7fb5440c5f80 con 0x7fb54414fd90 2024-03-09T09:06:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.944+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5440c9890 con 0x7fb54414fd90 2024-03-09T09:06:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.944+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1418139470 0 0) 0x7fb528003520 con 0x7fb5440ad440 2024-03-09T09:06:50.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.944+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5440c9380 msgr2=0x7fb5440acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.944+0000 7fb540a00640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5440c9380 0x7fb5440acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.944+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 >> v1:172.21.15.134:6789/0 conn(0x7fb54414fd90 legacy=0x7fb544156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.944+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5440c6240 con 0x7fb5440ad440 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb53c0019f0 con 0x7fb5440ad440 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 >> v1:172.21.15.67:6789/0 conn(0x7fb5440ad440 legacy=0x7fb54406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb540a00640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb528003be0 0x7fb528004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb540a00640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5440c9380 0x7fb528004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb540a00640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb528004d40 0x7fb5280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54406ba60 con 0x7fb528003be0 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52800bc20 con 0x7fb528004d40 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52800be20 con 0x7fb5440c9380 2024-03-09T09:06:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb541e00640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb528003be0 0x7fb528004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb542800640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb528004d40 0x7fb5280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb541e00640 1 -- 172.21.15.67:0/1073404249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5440c9380 msgr2=0x7fb528004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb541e00640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5440c9380 0x7fb528004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb541e00640 1 -- 172.21.15.67:0/1073404249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb528004d40 msgr2=0x7fb5280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb541e00640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb528004d40 0x7fb5280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb541e00640 1 -- 172.21.15.67:0/1073404249 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528003520 con 0x7fb528003be0 2024-03-09T09:06:50.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.945+0000 7fb542800640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb528004d40 0x7fb5280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:06:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.947+0000 7fb541e00640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb528003be0 0x7fb528004000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb5340018d0 tx=0x7fb528005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67813cce8d7af19b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.947+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534037070 con 0x7fb528003be0 2024-03-09T09:06:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.947+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5340052d0 con 0x7fb528003be0 2024-03-09T09:06:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.947+0000 7fb540a00640 1 -- 172.21.15.67:0/1073404249 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340014f0 con 0x7fb528003be0 2024-03-09T09:06:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.947+0000 7fb543200640 1 -- 172.21.15.67:0/1073404249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb528003be0 msgr2=0x7fb528004000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.947+0000 7fb543200640 1 --2- 172.21.15.67:0/1073404249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb528003be0 0x7fb528004000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb5340018d0 tx=0x7fb528005830 comp rx=0 tx=0).stop 2024-03-09T09:06:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.947+0000 7fb542800640 1 -- 172.21.15.67:0/1073404249 reap_dead start 2024-03-09T09:06:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.947+0000 7fb543200640 1 -- 172.21.15.67:0/1073404249 shutdown_connections 2024-03-09T09:06:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.947+0000 7fb543200640 1 -- 172.21.15.67:0/1073404249 >> 172.21.15.67:0/1073404249 conn(0x7fb5440cb480 msgr2=0x7fb5440cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.947+0000 7fb543200640 1 -- 172.21.15.67:0/1073404249 shutdown_connections 2024-03-09T09:06:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.948+0000 7fb543200640 1 -- 172.21.15.67:0/1073404249 wait complete. 2024-03-09T09:06:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.949+0000 7fb543200640 1 Processor -- start 2024-03-09T09:06:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.949+0000 7fb543200640 1 -- start start 2024-03-09T09:06:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.950+0000 7fb543200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb528003be0 0x7fb5440b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.950+0000 7fb543200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5440afe90 0x7fb544154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.950+0000 7fb543200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544151530 0x7fb5441eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.950+0000 7fb543200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5440acf00 con 0x7fb528003be0 2024-03-09T09:06:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.950+0000 7fb543200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54406bcb0 con 0x7fb544151530 2024-03-09T09:06:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.950+0000 7fb543200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5440ad080 con 0x7fb5440afe90 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.950+0000 7fb541e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb528003be0 0x7fb5440b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.950+0000 7fb541e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb528003be0 0x7fb5440b0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34688/0 (socket says 172.21.15.67:34688) 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.950+0000 7fb541e00640 1 -- 172.21.15.67:0/3958058193 learned_addr learned my addr 172.21.15.67:0/3958058193 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.951+0000 7fb542800640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544151530 0x7fb5441eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.951+0000 7fb541400640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5440afe90 0x7fb544154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.951+0000 7fb541e00640 1 -- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5440afe90 msgr2=0x7fb544154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.951+0000 7fb541e00640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5440afe90 0x7fb544154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.951+0000 7fb541e00640 1 -- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544151530 msgr2=0x7fb5441eeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.951+0000 7fb541e00640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544151530 0x7fb5441eeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.951+0000 7fb541e00640 1 -- 172.21.15.67:0/3958058193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5440b0800 con 0x7fb528003be0 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.951+0000 7fb541e00640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb528003be0 0x7fb5440b0cb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb534005040 tx=0x7fb5340012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.951+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534037070 con 0x7fb528003be0 2024-03-09T09:06:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.952+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb534038040 con 0x7fb528003be0 2024-03-09T09:06:50.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.952+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340014f0 con 0x7fb528003be0 2024-03-09T09:06:50.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.952+0000 7fb543200640 1 -- 172.21.15.67:0/3958058193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb544159bd0 con 0x7fb528003be0 2024-03-09T09:06:50.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.952+0000 7fb543200640 1 -- 172.21.15.67:0/3958058193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb544150f00 con 0x7fb528003be0 2024-03-09T09:06:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.953+0000 7fb543200640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544151910 0x7fb5441eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.953+0000 7fb543200640 1 -- 172.21.15.67:0/3958058193 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb544159980 con 0x7fb544151910 2024-03-09T09:06:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.953+0000 7fb541400640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544151910 0x7fb5441eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.954+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb5340052d0 con 0x7fb528003be0 2024-03-09T09:06:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.954+0000 7fb541400640 1 -- 172.21.15.67:0/3958058193 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb54404f3c0 con 0x7fb544151910 2024-03-09T09:06:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.954+0000 7fb53a000640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb52406b4d0 0x7fb52406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.955+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb5340abac0 con 0x7fb528003be0 2024-03-09T09:06:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.955+0000 7fb541400640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544151910 0x7fb5441eefe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb5440c97e0 tx=0x7fb530011a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.955+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530015870 con 0x7fb544151910 2024-03-09T09:06:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.955+0000 7fb542800640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb52406b4d0 0x7fb52406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.956+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb530015870 con 0x7fb544151910 2024-03-09T09:06:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.956+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544151910 msgr2=0x7fb5441eefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.956+0000 7fb53a000640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544151910 0x7fb5441eefe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb5440c97e0 tx=0x7fb530011a60 comp rx=0 tx=0).stop 2024-03-09T09:06:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.956+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544151910 msgr2=0x7fb5441eefe0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.956+0000 7fb53a000640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb544151910 0x7fb5441eefe0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb5440c97e0 tx=0x7fb530011a60 comp rx=0 tx=0).stop 2024-03-09T09:06:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.956+0000 7fb542800640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb52406b4d0 0x7fb52406d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb54408d730 tx=0x7fb53c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.963+0000 7fb543200640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5441ef520 0x7fb544148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:50.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.963+0000 7fb543200640 1 -- 172.21.15.67:0/3958058193 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441510d0 con 0x7fb5441ef520 2024-03-09T09:06:50.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.964+0000 7fb541400640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5441ef520 0x7fb544148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:50.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.965+0000 7fb541400640 1 -- 172.21.15.67:0/3958058193 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb54406d2d0 con 0x7fb5441ef520 2024-03-09T09:06:50.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.965+0000 7fb541400640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5441ef520 0x7fb544148210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb5440c97e0 tx=0x7fb530009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:50.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.966+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530006300 con 0x7fb5441ef520 2024-03-09T09:06:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.966+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7fb530006300 con 0x7fb5441ef520 2024-03-09T09:06:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.966+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5441ef520 msgr2=0x7fb544148210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.966+0000 7fb53a000640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5441ef520 0x7fb544148210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb5440c97e0 tx=0x7fb530009190 comp rx=0 tx=0).stop 2024-03-09T09:06:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.966+0000 7fb53a000640 1 -- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5441ef520 msgr2=0x7fb544148210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:50.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.966+0000 7fb53a000640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5441ef520 0x7fb544148210 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb5440c97e0 tx=0x7fb530009190 comp rx=0 tx=0).stop 2024-03-09T09:06:50.970 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:06:50.970 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:06:50.970 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:06:50.970 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 104, 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:50.971 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:50.972 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:06:50.973 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:06:50.974 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.975 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.976 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:50.977 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.978 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.979 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:50.980 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:06:50.980 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.980 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.980 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:50.980 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:06:50.980 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:50.980 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.980 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.980 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:06:50.982 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:06:50.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.967+0000 7fb543200640 1 -- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb52406b4d0 msgr2=0x7fb52406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.967+0000 7fb543200640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb52406b4d0 0x7fb52406d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb54408d730 tx=0x7fb53c009040 comp rx=0 tx=0).stop 2024-03-09T09:06:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.967+0000 7fb543200640 1 -- 172.21.15.67:0/3958058193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb528003be0 msgr2=0x7fb5440b0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.967+0000 7fb543200640 1 --2- 172.21.15.67:0/3958058193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb528003be0 0x7fb5440b0cb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb534005040 tx=0x7fb5340012d0 comp rx=0 tx=0).stop 2024-03-09T09:06:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.967+0000 7fb542800640 1 -- 172.21.15.67:0/3958058193 reap_dead start 2024-03-09T09:06:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.967+0000 7fb542800640 1 -- 172.21.15.67:0/3958058193 reap_dead start 2024-03-09T09:06:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.967+0000 7fb543200640 1 -- 172.21.15.67:0/3958058193 shutdown_connections 2024-03-09T09:06:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.967+0000 7fb543200640 1 -- 172.21.15.67:0/3958058193 >> 172.21.15.67:0/3958058193 conn(0x7fb5440cb480 msgr2=0x7fb5440ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.967+0000 7fb543200640 1 -- 172.21.15.67:0/3958058193 shutdown_connections 2024-03-09T09:06:50.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:50.967+0000 7fb543200640 1 -- 172.21.15.67:0/3958058193 wait complete. 2024-03-09T09:06:50.986 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:06:50.986 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:06:50.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:06:51.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.189+0000 7f0e92200640 1 Processor -- start 2024-03-09T09:06:51.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.189+0000 7f0e92200640 1 -- start start 2024-03-09T09:06:51.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.191+0000 7f0e92200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0e8c153560 0x7f0e8c06b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.191+0000 7f0e92200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c5f50 0x7f0e8c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.191+0000 7f0e92200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8c06cf30 con 0x7f0e8c0c96f0 2024-03-09T09:06:51.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.191+0000 7f0e92200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8c159d80 con 0x7f0e8c153560 2024-03-09T09:06:51.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.191+0000 7f0e92200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c15a060 con 0x7f0e8c0c5f50 2024-03-09T09:06:51.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.192+0000 7f0e8aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0e8c0c96f0 0x7f0e8c0cac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59708/0 (socket says 172.21.15.67:59708) 2024-03-09T09:06:51.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.192+0000 7f0e8aa00640 1 -- 172.21.15.67:0/4163094042 learned_addr learned my addr 172.21.15.67:0/4163094042 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:06:51.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.192+0000 7f0e8b400640 1 --2- 172.21.15.67:0/4163094042 >> v2:172.21.15.134:3300/0 conn(0x7f0e8c153560 0x7f0e8c06b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.192+0000 7f0e8be00640 1 --2- 172.21.15.67:0/4163094042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c5f50 0x7f0e8c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.192+0000 7f0e8a000640 1 -- 172.21.15.67:0/4163094042 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008495239 0 0) 0x7f0e8c06cf30 con 0x7f0e8c0c96f0 2024-03-09T09:06:51.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.192+0000 7f0e8a000640 1 -- 172.21.15.67:0/4163094042 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e70003520 con 0x7f0e8c0c96f0 2024-03-09T09:06:51.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.193+0000 7f0e8be00640 1 -- 172.21.15.67:0/4163094042 >> v2:172.21.15.134:3300/0 conn(0x7f0e8c153560 msgr2=0x7f0e8c06b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:51.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.193+0000 7f0e8be00640 1 --2- 172.21.15.67:0/4163094042 >> v2:172.21.15.134:3300/0 conn(0x7f0e8c153560 0x7f0e8c06b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.193+0000 7f0e8be00640 1 -- 172.21.15.67:0/4163094042 >> v1:172.21.15.67:6789/0 conn(0x7f0e8c0c96f0 legacy=0x7f0e8c0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.193+0000 7f0e8be00640 1 -- 172.21.15.67:0/4163094042 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c0ca440 con 0x7f0e8c0c5f50 2024-03-09T09:06:51.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.193+0000 7f0e8b400640 1 --2- 172.21.15.67:0/4163094042 >> v2:172.21.15.134:3300/0 conn(0x7f0e8c153560 0x7f0e8c06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:06:51.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.194+0000 7f0e8be00640 1 --2- 172.21.15.67:0/4163094042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c5f50 0x7f0e8c0c63a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009d90 tx=0x7f0e8402be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9c2375d747c9b71 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:51.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.194+0000 7f0e8a000640 1 -- 172.21.15.67:0/4163094042 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84039070 con 0x7f0e8c0c5f50 2024-03-09T09:06:51.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.194+0000 7f0e8a000640 1 -- 172.21.15.67:0/4163094042 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8402cbd0 con 0x7f0e8c0c5f50 2024-03-09T09:06:51.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.194+0000 7f0e8a000640 1 -- 172.21.15.67:0/4163094042 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84034440 con 0x7f0e8c0c5f50 2024-03-09T09:06:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.195+0000 7f0e92200640 1 -- 172.21.15.67:0/4163094042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c5f50 msgr2=0x7f0e8c0c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.195+0000 7f0e92200640 1 --2- 172.21.15.67:0/4163094042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c5f50 0x7f0e8c0c63a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009d90 tx=0x7f0e8402be20 comp rx=0 tx=0).stop 2024-03-09T09:06:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.195+0000 7f0e92200640 1 -- 172.21.15.67:0/4163094042 shutdown_connections 2024-03-09T09:06:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.195+0000 7f0e92200640 1 --2- 172.21.15.67:0/4163094042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c5f50 0x7f0e8c0c63a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.195+0000 7f0e92200640 1 --2- 172.21.15.67:0/4163094042 >> v2:172.21.15.134:3300/0 conn(0x7f0e8c153560 0x7f0e8c06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.195+0000 7f0e92200640 1 -- 172.21.15.67:0/4163094042 >> 172.21.15.67:0/4163094042 conn(0x7f0e8c0c8050 msgr2=0x7f0e8c0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.195+0000 7f0e92200640 1 -- 172.21.15.67:0/4163094042 shutdown_connections 2024-03-09T09:06:51.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.195+0000 7f0e92200640 1 -- 172.21.15.67:0/4163094042 wait complete. 2024-03-09T09:06:51.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.197+0000 7f0e92200640 1 Processor -- start 2024-03-09T09:06:51.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.197+0000 7f0e92200640 1 -- start start 2024-03-09T09:06:51.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.198+0000 7f0e92200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e8c0c5f50 0x7f0e8c1e2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.198+0000 7f0e92200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c96f0 0x7f0e8c154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.198+0000 7f0e92200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8c153560 0x7f0e8c155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e92200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0c68e0 con 0x7f0e8c153560 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e92200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0c9ff0 con 0x7f0e8c0c5f50 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e92200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0c6a60 con 0x7f0e8c0c96f0 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e8be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8c153560 0x7f0e8c155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e8be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8c153560 0x7f0e8c155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34704/0 (socket says 172.21.15.67:34704) 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e8be00640 1 -- 172.21.15.67:0/4194493006 learned_addr learned my addr 172.21.15.67:0/4194493006 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e8aa00640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c96f0 0x7f0e8c154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e8b400640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e8c0c5f50 0x7f0e8c1e2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e8be00640 1 -- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c96f0 msgr2=0x7f0e8c154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e8be00640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c96f0 0x7f0e8c154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e8be00640 1 -- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e8c0c5f50 msgr2=0x7f0e8c1e2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e8be00640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e8c0c5f50 0x7f0e8c1e2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.199+0000 7f0e8be00640 1 -- 172.21.15.67:0/4194493006 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c154210 con 0x7f0e8c153560 2024-03-09T09:06:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.200+0000 7f0e8be00640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8c153560 0x7f0e8c155010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e84037040 tx=0x7f0e84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.200+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84039050 con 0x7f0e8c153560 2024-03-09T09:06:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.200+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e84045070 con 0x7f0e8c153560 2024-03-09T09:06:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.200+0000 7f0e92200640 1 -- 172.21.15.67:0/4194493006 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c153f30 con 0x7f0e8c153560 2024-03-09T09:06:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.200+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84009450 con 0x7f0e8c153560 2024-03-09T09:06:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.200+0000 7f0e92200640 1 -- 172.21.15.67:0/4194493006 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c1ec880 con 0x7f0e8c153560 2024-03-09T09:06:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.201+0000 7f0e92200640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c21a0 0x7f0e8c14c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.201+0000 7f0e92200640 1 -- 172.21.15.67:0/4194493006 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0b0e00 con 0x7f0e8c0c21a0 2024-03-09T09:06:51.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.202+0000 7f0e8b400640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c21a0 0x7f0e8c14c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.202+0000 7f0e8b400640 1 -- 172.21.15.67:0/4194493006 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e8c068d20 con 0x7f0e8c0c21a0 2024-03-09T09:06:51.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.203+0000 7f0e8b400640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c21a0 0x7f0e8c14c780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c011860 tx=0x7f0e7c011d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:51.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.203+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0e8404a460 con 0x7f0e8c153560 2024-03-09T09:06:51.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.204+0000 7f0e83400640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e6c06b4d0 0x7f0e6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.205+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0e840ace80 con 0x7f0e8c153560 2024-03-09T09:06:51.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.205+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c019070 con 0x7f0e8c0c21a0 2024-03-09T09:06:51.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.205+0000 7f0e8aa00640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e6c06b4d0 0x7f0e6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.205+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f0e7c004240 con 0x7f0e8c0c21a0 2024-03-09T09:06:51.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.205+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c21a0 msgr2=0x7f0e8c14c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.205+0000 7f0e83400640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c21a0 0x7f0e8c14c780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c011860 tx=0x7f0e7c011d20 comp rx=0 tx=0).stop 2024-03-09T09:06:51.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.205+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c21a0 msgr2=0x7f0e8c14c780 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:51.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.205+0000 7f0e83400640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c0c21a0 0x7f0e8c14c780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.206+0000 7f0e8aa00640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e6c06b4d0 0x7f0e6c06d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c1e2450 tx=0x7f0e78009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:51.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.212+0000 7f0e92200640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c154460 0x7f0e8c14ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.213+0000 7f0e92200640 1 -- 172.21.15.67:0/4194493006 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0b0fb0 con 0x7f0e8c154460 2024-03-09T09:06:51.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.213+0000 7f0e8b400640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c154460 0x7f0e8c14ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.213+0000 7f0e8b400640 1 -- 172.21.15.67:0/4194493006 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0e8c075a00 con 0x7f0e8c154460 2024-03-09T09:06:51.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.214+0000 7f0e8b400640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c154460 0x7f0e8c14ccc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c011860 tx=0x7f0e7c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:51.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.214+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c019040 con 0x7f0e8c154460 2024-03-09T09:06:51.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4848 (secure 0 0 0) 0x7f0e7c003e10 con 0x7f0e8c154460 2024-03-09T09:06:51.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c154460 msgr2=0x7f0e8c14ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e83400640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c154460 0x7f0e8c14ccc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c011860 tx=0x7f0e7c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e83400640 1 -- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c154460 msgr2=0x7f0e8c14ccc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:06:51.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e83400640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e8c154460 0x7f0e8c14ccc0 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c011860 tx=0x7f0e7c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.219 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:06:51.219 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:06:51.219 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:06:51.219 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:06:51.219 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 104, 2024-03-09T09:06:51.219 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:06:51.219 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:06:51.219 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:06:51.219 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:06:51.219 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:51.220 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:51.221 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:06:51.222 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:06:51.223 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.224 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:51.225 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:51.226 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:06:51.227 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:06:51.228 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:51.229 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:06:51.231 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:06:51.231 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:06:51.231 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:06:51.231 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:06:51.231 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:06:51.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:06:51.231 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:06:51.231 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:06:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e92200640 1 -- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e6c06b4d0 msgr2=0x7f0e6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e92200640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e6c06b4d0 0x7f0e6c06d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c1e2450 tx=0x7f0e78009040 comp rx=0 tx=0).stop 2024-03-09T09:06:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e92200640 1 -- 172.21.15.67:0/4194493006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8c153560 msgr2=0x7f0e8c155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e92200640 1 --2- 172.21.15.67:0/4194493006 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8c153560 0x7f0e8c155010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e84037040 tx=0x7f0e84004a40 comp rx=0 tx=0).stop 2024-03-09T09:06:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e8be00640 1 -- 172.21.15.67:0/4194493006 reap_dead start 2024-03-09T09:06:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e8be00640 1 -- 172.21.15.67:0/4194493006 reap_dead start 2024-03-09T09:06:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e92200640 1 -- 172.21.15.67:0/4194493006 shutdown_connections 2024-03-09T09:06:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e92200640 1 -- 172.21.15.67:0/4194493006 >> 172.21.15.67:0/4194493006 conn(0x7f0e8c0c8050 msgr2=0x7f0e8c14d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.215+0000 7f0e92200640 1 -- 172.21.15.67:0/4194493006 shutdown_connections 2024-03-09T09:06:51.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.216+0000 7f0e92200640 1 -- 172.21.15.67:0/4194493006 wait complete. 2024-03-09T09:06:51.236 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:06:51.236 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T09:06:51.237 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:06:51.237 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T09:06:51.237 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:06:51.237 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:06:51.237 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:06:51.274 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:06:51.274 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:06:51.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:06:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.494+0000 7fb209800640 1 Processor -- start 2024-03-09T09:06:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.495+0000 7fb209800640 1 -- start start 2024-03-09T09:06:51.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.496+0000 7fb209800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb2040aa8d0 0x7fb204159b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.496+0000 7fb209800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb204153630 con 0x7fb204153250 2024-03-09T09:06:51.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.496+0000 7fb209800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb204156b20 con 0x7fb204072830 2024-03-09T09:06:51.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.496+0000 7fb209800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20415a0c0 con 0x7fb2040aa8d0 2024-03-09T09:06:51.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.496+0000 7fb203e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb204153250 0x7fb2040c6410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59716/0 (socket says 172.21.15.67:59716) 2024-03-09T09:06:51.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.496+0000 7fb203e00640 1 -- 172.21.15.67:0/1825591442 learned_addr learned my addr 172.21.15.67:0/1825591442 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:06:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.496+0000 7fb208e00640 1 --2- 172.21.15.67:0/1825591442 >> v2:172.21.15.67:3300/0 conn(0x7fb2040aa8d0 0x7fb204159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.497+0000 7fb203400640 1 -- 172.21.15.67:0/1825591442 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432856099 0 0) 0x7fb204153630 con 0x7fb204153250 2024-03-09T09:06:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.497+0000 7fb203400640 1 -- 172.21.15.67:0/1825591442 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1e8003520 con 0x7fb204153250 2024-03-09T09:06:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.497+0000 7fb208e00640 1 -- 172.21.15.67:0/1825591442 >> v1:172.21.15.134:6789/0 conn(0x7fb204072830 legacy=0x7fb204159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.497+0000 7fb208e00640 1 -- 172.21.15.67:0/1825591442 >> v1:172.21.15.67:6789/0 conn(0x7fb204153250 legacy=0x7fb2040c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.497+0000 7fb208e00640 1 -- 172.21.15.67:0/1825591442 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb204155fa0 con 0x7fb2040aa8d0 2024-03-09T09:06:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.498+0000 7fb208e00640 1 --2- 172.21.15.67:0/1825591442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2040aa8d0 0x7fb204159b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0097a0 tx=0x7fb1fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5b96b56e99d960 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:51.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.499+0000 7fb203400640 1 -- 172.21.15.67:0/1825591442 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc035020 con 0x7fb2040aa8d0 2024-03-09T09:06:51.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.499+0000 7fb203400640 1 -- 172.21.15.67:0/1825591442 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1fc031830 con 0x7fb2040aa8d0 2024-03-09T09:06:51.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.499+0000 7fb203400640 1 -- 172.21.15.67:0/1825591442 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc031c00 con 0x7fb2040aa8d0 2024-03-09T09:06:51.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.500+0000 7fb209800640 1 -- 172.21.15.67:0/1825591442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2040aa8d0 msgr2=0x7fb204159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.500+0000 7fb209800640 1 --2- 172.21.15.67:0/1825591442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2040aa8d0 0x7fb204159b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0097a0 tx=0x7fb1fc02b8d0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.500+0000 7fb209800640 1 -- 172.21.15.67:0/1825591442 shutdown_connections 2024-03-09T09:06:51.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.500+0000 7fb209800640 1 --2- 172.21.15.67:0/1825591442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2040aa8d0 0x7fb204159b80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.500+0000 7fb209800640 1 -- 172.21.15.67:0/1825591442 >> 172.21.15.67:0/1825591442 conn(0x7fb204067180 msgr2=0x7fb204067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:51.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.500+0000 7fb209800640 1 -- 172.21.15.67:0/1825591442 shutdown_connections 2024-03-09T09:06:51.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.500+0000 7fb209800640 1 -- 172.21.15.67:0/1825591442 wait complete. 2024-03-09T09:06:51.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.503+0000 7fb209800640 1 Processor -- start 2024-03-09T09:06:51.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.503+0000 7fb209800640 1 -- start start 2024-03-09T09:06:51.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb209800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb204072830 0x7fb204148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb209800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2040aa8d0 0x7fb2041e0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb209800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb204153250 0x7fb2041ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb209800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb204156840 con 0x7fb2040aa8d0 2024-03-09T09:06:51.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb209800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb204156560 con 0x7fb204153250 2024-03-09T09:06:51.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb209800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040c6b20 con 0x7fb204072830 2024-03-09T09:06:51.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb203e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb204072830 msgr2=0x7fb204148e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:06:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb203e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb204072830 0x7fb204148e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb1fb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2040aa8d0 0x7fb2041e0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb1fb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2040aa8d0 0x7fb2041e0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34736/0 (socket says 172.21.15.67:34736) 2024-03-09T09:06:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb1fb400640 1 -- 172.21.15.67:0/283690925 learned_addr learned my addr 172.21.15.67:0/283690925 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.505+0000 7fb208e00640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb204153250 0x7fb2041ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.506+0000 7fb1fb400640 1 -- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb204072830 msgr2=0x7fb204148e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:06:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.506+0000 7fb1fb400640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb204072830 0x7fb204148e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.506+0000 7fb1fb400640 1 -- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb204153250 msgr2=0x7fb2041ebfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.506+0000 7fb1fb400640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb204153250 0x7fb2041ebfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.506+0000 7fb1fb400640 1 -- 172.21.15.67:0/283690925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20414fe80 con 0x7fb2040aa8d0 2024-03-09T09:06:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.506+0000 7fb1fb400640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2040aa8d0 0x7fb2041e0dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00c400 tx=0x7fb1ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.506+0000 7fb200c00640 1 -- 172.21.15.67:0/283690925 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec0052d0 con 0x7fb2040aa8d0 2024-03-09T09:06:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.506+0000 7fb200c00640 1 -- 172.21.15.67:0/283690925 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec005490 con 0x7fb2040aa8d0 2024-03-09T09:06:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.506+0000 7fb200c00640 1 -- 172.21.15.67:0/283690925 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec007260 con 0x7fb2040aa8d0 2024-03-09T09:06:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.507+0000 7fb209800640 1 -- 172.21.15.67:0/283690925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb204150160 con 0x7fb2040aa8d0 2024-03-09T09:06:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.507+0000 7fb209800640 1 -- 172.21.15.67:0/283690925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2041eb7a0 con 0x7fb2040aa8d0 2024-03-09T09:06:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.508+0000 7fb209800640 1 -- 172.21.15.67:0/283690925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb204152190 con 0x7fb2040aa8d0 2024-03-09T09:06:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.509+0000 7fb200c00640 1 -- 172.21.15.67:0/283690925 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb1ec007400 con 0x7fb2040aa8d0 2024-03-09T09:06:51.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.510+0000 7fb200c00640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb1e006b480 0x7fb1e006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:51.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.510+0000 7fb200c00640 1 -- 172.21.15.67:0/283690925 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb1ec089ac0 con 0x7fb2040aa8d0 2024-03-09T09:06:51.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.510+0000 7fb203e00640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb1e006b480 0x7fb1e006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:51.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.511+0000 7fb203e00640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb1e006b480 0x7fb1e006d940 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb204154390 tx=0x7fb1f4001630 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:51.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.514+0000 7fb200c00640 1 -- 172.21.15.67:0/283690925 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb204152190 con 0x7fb2040aa8d0 2024-03-09T09:06:51.750 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 21% (46/217) Updating files: 22% (48/217) Updating files: 23% (50/217) Updating files: 24% (53/217) Updating files: 25% (55/217) Updating files: 26% (57/217) Updating files: 27% (59/217) Updating files: 28% (61/217) Updating files: 29% (63/217) Updating files: 30% (66/217) Updating files: 31% (68/217) Updating files: 32% (70/217) Updating files: 33% (72/217) Updating files: 34% (74/217) Updating files: 35% (76/217) Updating files: 36% (79/217) Updating files: 37% (81/217) Updating files: 38% (83/217) Updating files: 39% (85/217) Updating files: 40% (87/217) Updating files: 41% (89/217) Updating files: 42% (92/217) Updating files: 43% (94/217) Updating files: 44% (96/217) Updating files: 45% (98/217) Updating files: 46% (100/217) Updating files: 47% (102/217) Updating files: 47% (103/217) Updating files: 48% (105/217) Updating files: 49% (107/217) Updating files: 50% (109/217) Updating files: 51% (111/217) Updating files: 52% (113/217) Updating files: 53% (116/217) Updating files: 54% (118/217) Updating files: 55% (120/217) Updating files: 56% (122/217) Updating files: 57% (124/217) Updating files: 58% (126/217) Updating files: 59% (129/217) Updating files: 60% (131/217) Updating files: 61% (133/217) Updating files: 62% (135/217) Updating files: 63% (137/217) Updating files: 64% (139/217) Updating files: 65% (142/217) Updating files: 66% (144/217) Updating files: 67% (146/217) Updating files: 68% (148/217) Updating files: 69% (150/217) Updating files: 70% (152/217) Updating files: 71% (155/217) Updating files: 72% (157/217) Updating files: 73% (159/217) Updating files: 74% (161/217) Updating files: 75% (163/217) Updating files: 76% (165/217) Updating files: 77% (168/217) Updating files: 78% (170/217) Updating files: 79% (172/217) Updating files: 80% (174/217) Updating files: 81% (176/217) Updating files: 82% (178/217) Updating files: 83% (181/217) Updating files: 84% (183/217) Updating files: 85% (185/217) Updating files: 86% (187/217) Updating files: 87% (189/217) Updating files: 88% (191/217) Updating files: 89% (194/217) Updating files: 90% (196/217) Updating files: 91% (198/217) Updating files: 92% (200/217) Updating files: 93% (202/217) Updating files: 94% (204/217) Updating files: 95% (207/217) Updating files: 96% (209/217) Updating files: 96% (210/217) Updating files: 97% (211/217) Updating files: 98% (213/217) Updating files: 99% (215/217) Updating files: 100% (217/217) Updating files: 100% (217/217), done. 2024-03-09T09:06:51.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.793+0000 7fb209800640 1 -- 172.21.15.67:0/283690925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb20404f3c0 con 0x7fb2040aa8d0 2024-03-09T09:06:51.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.794+0000 7fb200c00640 1 -- 172.21.15.67:0/283690925 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fb1ec05ce20 con 0x7fb2040aa8d0 2024-03-09T09:06:51.797 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":104,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:06:51.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 -- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb1e006b480 msgr2=0x7fb1e006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb1e006b480 0x7fb1e006d940 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb204154390 tx=0x7fb1f4001630 comp rx=0 tx=0).stop 2024-03-09T09:06:51.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 -- 172.21.15.67:0/283690925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2040aa8d0 msgr2=0x7fb2041e0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:51.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2040aa8d0 0x7fb2041e0dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00c400 tx=0x7fb1ec00c8c0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 -- 172.21.15.67:0/283690925 shutdown_connections 2024-03-09T09:06:51.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb1e006b480 0x7fb1e006d940 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb204153250 0x7fb2041ebfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb2040aa8d0 0x7fb2041e0dd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 --2- 172.21.15.67:0/283690925 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb204072830 0x7fb204148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:51.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 -- 172.21.15.67:0/283690925 >> 172.21.15.67:0/283690925 conn(0x7fb204067180 msgr2=0x7fb204154630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:51.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 -- 172.21.15.67:0/283690925 shutdown_connections 2024-03-09T09:06:51.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:51.797+0000 7fb209800640 1 -- 172.21.15.67:0/283690925 wait complete. 2024-03-09T09:06:51.804 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 98f5be53f85 v0.19 2024-03-09T09:06:51.814 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.19 2024-03-09T09:06:51.963 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:51.963 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 20 -eq 48 ']' 2024-03-09T09:06:51.964 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.20 2024-03-09T09:06:51.964 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.20 2024-03-09T09:06:51.964 INFO:tasks.workunit.client.0.smithi067.stdout:v0.20 2024-03-09T09:06:51.964 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.20 2024-03-09T09:06:54.363 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 43% (106/245) Updating files: 44% (108/245) Updating files: 45% (111/245) Updating files: 46% (113/245) Updating files: 47% (116/245) Updating files: 48% (118/245) Updating files: 49% (121/245) Updating files: 50% (123/245) Updating files: 51% (125/245) Updating files: 52% (128/245) Updating files: 53% (130/245) Updating files: 54% (133/245) Updating files: 55% (135/245) Updating files: 56% (138/245) Updating files: 57% (140/245) Updating files: 58% (143/245) Updating files: 59% (145/245) Updating files: 60% (147/245) Updating files: 61% (150/245) Updating files: 62% (152/245) Updating files: 63% (155/245) Updating files: 64% (157/245) Updating files: 65% (160/245) Updating files: 66% (162/245) Updating files: 67% (165/245) Updating files: 68% (167/245) Updating files: 69% (170/245) Updating files: 70% (172/245) Updating files: 71% (174/245) Updating files: 72% (177/245) Updating files: 73% (179/245) Updating files: 74% (182/245) Updating files: 75% (184/245) Updating files: 76% (187/245) Updating files: 77% (189/245) Updating files: 78% (192/245) Updating files: 79% (194/245) Updating files: 80% (196/245) Updating files: 81% (199/245) Updating files: 82% (201/245) Updating files: 83% (204/245) Updating files: 84% (206/245) Updating files: 85% (209/245) Updating files: 86% (211/245) Updating files: 87% (214/245) Updating files: 88% (216/245) Updating files: 89% (219/245) Updating files: 90% (221/245) Updating files: 91% (223/245) Updating files: 92% (226/245) Updating files: 93% (228/245) Updating files: 94% (231/245) Updating files: 95% (233/245) Updating files: 96% (236/245) Updating files: 97% (238/245) Updating files: 98% (241/245) Updating files: 99% (243/245) Updating files: 100% (245/245) Updating files: 100% (245/245), done. 2024-03-09T09:06:54.409 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 266d2076b6c debian: fix up debian scripts 2024-03-09T09:06:54.417 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.20 2024-03-09T09:06:54.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:06:54.902 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:06:54.902 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 21 -eq 48 ']' 2024-03-09T09:06:54.903 INFO:tasks.workunit.client.0.smithi067.stdout:v0.21 2024-03-09T09:06:54.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.21 2024-03-09T09:06:54.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.21 2024-03-09T09:06:54.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.21 2024-03-09T09:06:55.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.029+0000 7f335ae00640 1 Processor -- start 2024-03-09T09:06:55.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.029+0000 7f335ae00640 1 -- start start 2024-03-09T09:06:55.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.030+0000 7f335ae00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f33540c30f0 0x7f3354152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:55.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.030+0000 7f335ae00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f33540aa420 0x7f33541490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:55.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f335ae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 0x7f335406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:55.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f335ae00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33540c82d0 con 0x7f33540c30f0 2024-03-09T09:06:55.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f335ae00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33540ab3c0 con 0x7f33540aa420 2024-03-09T09:06:55.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f335ae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33540aa8a0 con 0x7f3354149630 2024-03-09T09:06:55.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f3358e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 msgr2=0x7f335406dbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:06:55.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f3358e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 0x7f335406dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:55.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f3353e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f33540c30f0 0x7f3354152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:55.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f3353e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f33540c30f0 0x7f3354152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34752/0 (socket says 172.21.15.67:34752) 2024-03-09T09:06:55.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f3353e00640 1 -- 172.21.15.67:0/1614513408 learned_addr learned my addr 172.21.15.67:0/1614513408 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:55.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f3353400640 1 --2- 172.21.15.67:0/1614513408 >> v2:172.21.15.134:3300/0 conn(0x7f33540aa420 0x7f33541490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:55.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f3353e00640 1 -- 172.21.15.67:0/1614513408 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 msgr2=0x7f335406dbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:06:55.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f3353e00640 1 --2- 172.21.15.67:0/1614513408 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 0x7f335406dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:55.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f3353e00640 1 -- 172.21.15.67:0/1614513408 >> v2:172.21.15.134:3300/0 conn(0x7f33540aa420 msgr2=0x7f33541490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:06:55.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f3353e00640 1 --2- 172.21.15.67:0/1614513408 >> v2:172.21.15.134:3300/0 conn(0x7f33540aa420 0x7f33541490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:55.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.031+0000 7f3353e00640 1 -- 172.21.15.67:0/1614513408 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33540aab20 con 0x7f33540c30f0 2024-03-09T09:06:55.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.033+0000 7f3353e00640 1 --2- 172.21.15.67:0/1614513408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33540c30f0 0x7f3354152a20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f33540ad6e0 tx=0x7f334402b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d69ef5c167f56 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:55.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.033+0000 7f3352a00640 1 -- 172.21.15.67:0/1614513408 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344031760 con 0x7f33540c30f0 2024-03-09T09:06:55.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.033+0000 7f3352a00640 1 -- 172.21.15.67:0/1614513408 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3344037070 con 0x7f33540c30f0 2024-03-09T09:06:55.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.033+0000 7f3352a00640 1 -- 172.21.15.67:0/1614513408 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344031b20 con 0x7f33540c30f0 2024-03-09T09:06:55.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.034+0000 7f335ae00640 1 -- 172.21.15.67:0/1614513408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33540c30f0 msgr2=0x7f3354152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:55.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.034+0000 7f335ae00640 1 --2- 172.21.15.67:0/1614513408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33540c30f0 0x7f3354152a20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f33540ad6e0 tx=0x7f334402b800 comp rx=0 tx=0).stop 2024-03-09T09:06:55.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.034+0000 7f335ae00640 1 -- 172.21.15.67:0/1614513408 shutdown_connections 2024-03-09T09:06:55.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.034+0000 7f335ae00640 1 --2- 172.21.15.67:0/1614513408 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 0x7f335406dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:55.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.034+0000 7f335ae00640 1 --2- 172.21.15.67:0/1614513408 >> v2:172.21.15.134:3300/0 conn(0x7f33540aa420 0x7f33541490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:55.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.034+0000 7f335ae00640 1 --2- 172.21.15.67:0/1614513408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33540c30f0 0x7f3354152a20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:55.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.034+0000 7f335ae00640 1 -- 172.21.15.67:0/1614513408 >> 172.21.15.67:0/1614513408 conn(0x7f33540c7370 msgr2=0x7f33540c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:55.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.034+0000 7f335ae00640 1 -- 172.21.15.67:0/1614513408 shutdown_connections 2024-03-09T09:06:55.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.034+0000 7f335ae00640 1 -- 172.21.15.67:0/1614513408 wait complete. 2024-03-09T09:06:55.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.036+0000 7f335ae00640 1 Processor -- start 2024-03-09T09:06:55.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.036+0000 7f335ae00640 1 -- start start 2024-03-09T09:06:55.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.037+0000 7f335ae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33540aa420 0x7f335414aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:55.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.037+0000 7f335ae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33540c30f0 0x7f33541ef460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:55.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.037+0000 7f335ae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 0x7f33541eb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:55.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.037+0000 7f335ae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335406d6c0 con 0x7f33540c30f0 2024-03-09T09:06:55.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.037+0000 7f335ae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33540ab0e0 con 0x7f33540aa420 2024-03-09T09:06:55.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.037+0000 7f335ae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33540ab620 con 0x7f3354149630 2024-03-09T09:06:55.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.037+0000 7f3358e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 msgr2=0x7f33541eb630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:06:55.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.037+0000 7f3358e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 0x7f33541eb630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:55.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.038+0000 7f3353400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33540c30f0 0x7f33541ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:55.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.038+0000 7f3353400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33540c30f0 0x7f33541ef460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34764/0 (socket says 172.21.15.67:34764) 2024-03-09T09:06:55.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.038+0000 7f3353400640 1 -- 172.21.15.67:0/2853641766 learned_addr learned my addr 172.21.15.67:0/2853641766 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.038+0000 7f3353e00640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33540aa420 0x7f335414aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.038+0000 7f3353400640 1 -- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 msgr2=0x7f33541eb630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.038+0000 7f3353400640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 0x7f33541eb630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.038+0000 7f3353400640 1 -- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33540aa420 msgr2=0x7f335414aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.038+0000 7f3353400640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33540aa420 0x7f335414aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.038+0000 7f3353400640 1 -- 172.21.15.67:0/2853641766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335414a510 con 0x7f33540c30f0 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.038+0000 7f3353400640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33540c30f0 0x7f33541ef460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f334000d5d0 tx=0x7f334000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.039+0000 7f334be00640 1 -- 172.21.15.67:0/2853641766 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340004240 con 0x7f33540c30f0 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.039+0000 7f334be00640 1 -- 172.21.15.67:0/2853641766 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33400043e0 con 0x7f33540c30f0 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.039+0000 7f335ae00640 1 -- 172.21.15.67:0/2853641766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3354073710 con 0x7f33540c30f0 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.039+0000 7f334be00640 1 -- 172.21.15.67:0/2853641766 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340003830 con 0x7f33540c30f0 2024-03-09T09:06:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.039+0000 7f335ae00640 1 -- 172.21.15.67:0/2853641766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33541eef40 con 0x7f33540c30f0 2024-03-09T09:06:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.040+0000 7f335ae00640 1 -- 172.21.15.67:0/2853641766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335414afd0 con 0x7f33540c30f0 2024-03-09T09:06:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.041+0000 7f334be00640 1 -- 172.21.15.67:0/2853641766 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f33400039d0 con 0x7f33540c30f0 2024-03-09T09:06:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.043+0000 7f334be00640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f333406b4b0 0x7f333406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.043+0000 7f334be00640 1 -- 172.21.15.67:0/2853641766 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f334008a6c0 con 0x7f33540c30f0 2024-03-09T09:06:55.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.043+0000 7f3353e00640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f333406b4b0 0x7f333406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:55.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.045+0000 7f3353e00640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f333406b4b0 0x7f333406d970 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3344036000 tx=0x7f334403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:55.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.047+0000 7f334be00640 1 -- 172.21.15.67:0/2853641766 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f33400591e0 con 0x7f33540c30f0 2024-03-09T09:06:55.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.355+0000 7f335ae00640 1 -- 172.21.15.67:0/2853641766 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3354075a00 con 0x7f33540c30f0 2024-03-09T09:06:55.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.355+0000 7f334be00640 1 -- 172.21.15.67:0/2853641766 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f3340058ae0 con 0x7f33540c30f0 2024-03-09T09:06:55.359 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":104,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:06:55.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 -- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f333406b4b0 msgr2=0x7f333406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:55.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f333406b4b0 0x7f333406d970 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3344036000 tx=0x7f334403d040 comp rx=0 tx=0).stop 2024-03-09T09:06:55.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 -- 172.21.15.67:0/2853641766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33540c30f0 msgr2=0x7f33541ef460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:55.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33540c30f0 0x7f33541ef460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f334000d5d0 tx=0x7f334000da90 comp rx=0 tx=0).stop 2024-03-09T09:06:55.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 -- 172.21.15.67:0/2853641766 shutdown_connections 2024-03-09T09:06:55.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f333406b4b0 0x7f333406d970 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:55.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3354149630 0x7f33541eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:55.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33540c30f0 0x7f33541ef460 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:55.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 --2- 172.21.15.67:0/2853641766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33540aa420 0x7f335414aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:55.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 -- 172.21.15.67:0/2853641766 >> 172.21.15.67:0/2853641766 conn(0x7f33540c7370 msgr2=0x7f33540c57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:55.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 -- 172.21.15.67:0/2853641766 shutdown_connections 2024-03-09T09:06:55.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:55.358+0000 7f335ae00640 1 -- 172.21.15.67:0/2853641766 wait complete. 2024-03-09T09:06:58.379 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:06:58.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.575+0000 7fdf5e800640 1 Processor -- start 2024-03-09T09:06:58.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.575+0000 7fdf5e800640 1 -- start start 2024-03-09T09:06:58.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.576+0000 7fdf5e800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdf5806bda0 0x7fdf58072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:58.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.576+0000 7fdf5e800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5806cd30 con 0x7fdf580bebc0 2024-03-09T09:06:58.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.576+0000 7fdf5e800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf580c5eb0 con 0x7fdf58072770 2024-03-09T09:06:58.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.576+0000 7fdf5e800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5806b200 con 0x7fdf5806bda0 2024-03-09T09:06:58.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.577+0000 7fdf5de00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdf580bebc0 0x7fdf58160550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59730/0 (socket says 172.21.15.67:59730) 2024-03-09T09:06:58.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.577+0000 7fdf5de00640 1 -- 172.21.15.67:0/1780507633 learned_addr learned my addr 172.21.15.67:0/1780507633 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:06:58.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.577+0000 7fdf5ca00640 1 --2- 172.21.15.67:0/1780507633 >> v2:172.21.15.134:3300/0 conn(0x7fdf5806bda0 0x7fdf58072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:58.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.578+0000 7fdf5ca00640 1 -- 172.21.15.67:0/1780507633 >> v1:172.21.15.134:6789/0 conn(0x7fdf58072770 legacy=0x7fdf5806b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:58.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.578+0000 7fdf5ca00640 1 -- 172.21.15.67:0/1780507633 >> v1:172.21.15.67:6789/0 conn(0x7fdf580bebc0 legacy=0x7fdf58160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:58.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.578+0000 7fdf5ca00640 1 -- 172.21.15.67:0/1780507633 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf580c6150 con 0x7fdf5806bda0 2024-03-09T09:06:58.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.579+0000 7fdf5ca00640 1 --2- 172.21.15.67:0/1780507633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf5806bda0 0x7fdf58072b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf440097a0 tx=0x7fdf4402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324cb9b34d353f48 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:58.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.580+0000 7fdf57e00640 1 -- 172.21.15.67:0/1780507633 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf440315a0 con 0x7fdf5806bda0 2024-03-09T09:06:58.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.580+0000 7fdf57e00640 1 -- 172.21.15.67:0/1780507633 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf44037070 con 0x7fdf5806bda0 2024-03-09T09:06:58.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.580+0000 7fdf57e00640 1 -- 172.21.15.67:0/1780507633 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44031960 con 0x7fdf5806bda0 2024-03-09T09:06:58.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.580+0000 7fdf5e800640 1 -- 172.21.15.67:0/1780507633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf5806bda0 msgr2=0x7fdf58072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:58.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.580+0000 7fdf5e800640 1 --2- 172.21.15.67:0/1780507633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf5806bda0 0x7fdf58072b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf440097a0 tx=0x7fdf4402b840 comp rx=0 tx=0).stop 2024-03-09T09:06:58.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.580+0000 7fdf5e800640 1 -- 172.21.15.67:0/1780507633 shutdown_connections 2024-03-09T09:06:58.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.580+0000 7fdf5e800640 1 --2- 172.21.15.67:0/1780507633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf5806bda0 0x7fdf58072b50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:58.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.580+0000 7fdf5e800640 1 -- 172.21.15.67:0/1780507633 >> 172.21.15.67:0/1780507633 conn(0x7fdf580c7fb0 msgr2=0x7fdf580c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:58.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.581+0000 7fdf5e800640 1 -- 172.21.15.67:0/1780507633 shutdown_connections 2024-03-09T09:06:58.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.581+0000 7fdf5e800640 1 -- 172.21.15.67:0/1780507633 wait complete. 2024-03-09T09:06:58.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.583+0000 7fdf5e800640 1 Processor -- start 2024-03-09T09:06:58.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.584+0000 7fdf5e800640 1 -- start start 2024-03-09T09:06:58.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.585+0000 7fdf5e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5806bda0 0x7fdf581553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:58.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.585+0000 7fdf5e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf58072770 0x7fdf5814c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:58.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.585+0000 7fdf5e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf580bebc0 0x7fdf581efa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:58.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.585+0000 7fdf5e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580c9430 con 0x7fdf5806bda0 2024-03-09T09:06:58.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.585+0000 7fdf5e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580c6710 con 0x7fdf58072770 2024-03-09T09:06:58.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.585+0000 7fdf5e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf581ef5c0 con 0x7fdf580bebc0 2024-03-09T09:06:58.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.585+0000 7fdf5de00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf580bebc0 msgr2=0x7fdf581efa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:06:58.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.585+0000 7fdf5d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5806bda0 0x7fdf581553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:58.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.586+0000 7fdf5de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf580bebc0 0x7fdf581efa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:06:58.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.586+0000 7fdf5d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5806bda0 0x7fdf581553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34774/0 (socket says 172.21.15.67:34774) 2024-03-09T09:06:58.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.586+0000 7fdf5d400640 1 -- 172.21.15.67:0/890011126 learned_addr learned my addr 172.21.15.67:0/890011126 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:06:58.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.586+0000 7fdf5ca00640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf58072770 0x7fdf5814c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:58.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.586+0000 7fdf5d400640 1 -- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf580bebc0 msgr2=0x7fdf581efa50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:06:58.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.586+0000 7fdf5d400640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf580bebc0 0x7fdf581efa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:58.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.586+0000 7fdf5d400640 1 -- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf58072770 msgr2=0x7fdf5814c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:58.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.586+0000 7fdf5d400640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf58072770 0x7fdf5814c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:58.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.586+0000 7fdf5d400640 1 -- 172.21.15.67:0/890011126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5814e570 con 0x7fdf5806bda0 2024-03-09T09:06:58.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.586+0000 7fdf5d400640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5806bda0 0x7fdf581553d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c4d0 tx=0x7fdf4c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:58.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.587+0000 7fdf55600640 1 -- 172.21.15.67:0/890011126 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0052d0 con 0x7fdf5806bda0 2024-03-09T09:06:58.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.587+0000 7fdf55600640 1 -- 172.21.15.67:0/890011126 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c005470 con 0x7fdf5806bda0 2024-03-09T09:06:58.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.587+0000 7fdf5e800640 1 -- 172.21.15.67:0/890011126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5814e850 con 0x7fdf5806bda0 2024-03-09T09:06:58.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.587+0000 7fdf55600640 1 -- 172.21.15.67:0/890011126 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c012850 con 0x7fdf5806bda0 2024-03-09T09:06:58.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.587+0000 7fdf5e800640 1 -- 172.21.15.67:0/890011126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5814c870 con 0x7fdf5806bda0 2024-03-09T09:06:58.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.589+0000 7fdf5e800640 1 -- 172.21.15.67:0/890011126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5814ed70 con 0x7fdf5806bda0 2024-03-09T09:06:58.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.590+0000 7fdf55600640 1 -- 172.21.15.67:0/890011126 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdf4c0129f0 con 0x7fdf5806bda0 2024-03-09T09:06:58.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.591+0000 7fdf55600640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf3806b480 0x7fdf3806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:06:58.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.591+0000 7fdf55600640 1 -- 172.21.15.67:0/890011126 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdf4c08a240 con 0x7fdf5806bda0 2024-03-09T09:06:58.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.592+0000 7fdf5ca00640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf3806b480 0x7fdf3806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:06:58.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.593+0000 7fdf5ca00640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf3806b480 0x7fdf3806d940 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdf44036000 tx=0x7fdf4403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:06:58.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.595+0000 7fdf55600640 1 -- 172.21.15.67:0/890011126 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdf5814ed70 con 0x7fdf5806bda0 2024-03-09T09:06:58.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.974+0000 7fdf5e800640 1 -- 172.21.15.67:0/890011126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdf580684c0 con 0x7fdf5806bda0 2024-03-09T09:06:58.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.974+0000 7fdf55600640 1 -- 172.21.15.67:0/890011126 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fdf580684c0 con 0x7fdf5806bda0 2024-03-09T09:06:58.978 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":104,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:06:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 -- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf3806b480 msgr2=0x7fdf3806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf3806b480 0x7fdf3806d940 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdf44036000 tx=0x7fdf4403d040 comp rx=0 tx=0).stop 2024-03-09T09:06:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 -- 172.21.15.67:0/890011126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5806bda0 msgr2=0x7fdf581553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:06:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5806bda0 0x7fdf581553d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c4d0 tx=0x7fdf4c00c990 comp rx=0 tx=0).stop 2024-03-09T09:06:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 -- 172.21.15.67:0/890011126 shutdown_connections 2024-03-09T09:06:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf3806b480 0x7fdf3806d940 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf580bebc0 0x7fdf581efa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:58.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf58072770 0x7fdf5814c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:58.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 --2- 172.21.15.67:0/890011126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf5806bda0 0x7fdf581553d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:06:58.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 -- 172.21.15.67:0/890011126 >> 172.21.15.67:0/890011126 conn(0x7fdf580c7fb0 msgr2=0x7fdf58151660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:06:58.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 -- 172.21.15.67:0/890011126 shutdown_connections 2024-03-09T09:06:58.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:06:58.977+0000 7fdf5e800640 1 -- 172.21.15.67:0/890011126 wait complete. 2024-03-09T09:07:01.999 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:07:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.216+0000 7f6cd8e00640 1 Processor -- start 2024-03-09T09:07:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.216+0000 7f6cd8e00640 1 -- start start 2024-03-09T09:07:02.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.218+0000 7f6cd8e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd40c5e40 0x7f6cd4156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:02.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.218+0000 7f6cd8e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd4157000 con 0x7f6cd4153300 2024-03-09T09:07:02.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.218+0000 7f6cd8e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd40ae430 con 0x7f6cd40cadb0 2024-03-09T09:07:02.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.218+0000 7f6cd8e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd40ae6f0 con 0x7f6cd40c5e40 2024-03-09T09:07:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.218+0000 7f6cd2a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd40c5e40 msgr2=0x7f6cd4156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:07:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.218+0000 7f6cd3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6cd4153300 0x7f6cd40b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48954/0 (socket says 172.21.15.67:48954) 2024-03-09T09:07:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.218+0000 7f6cd3400640 1 -- 172.21.15.67:0/3324954295 learned_addr learned my addr 172.21.15.67:0/3324954295 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:07:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.218+0000 7f6cd2a00640 1 --2- 172.21.15.67:0/3324954295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd40c5e40 0x7f6cd4156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:02.334 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 5% (13/245) Updating files: 6% (15/245) Updating files: 7% (18/245) Updating files: 8% (20/245) Updating files: 9% (23/245) Updating files: 10% (25/245) Updating files: 11% (27/245) Updating files: 12% (30/245) Updating files: 13% (32/245) Updating files: 14% (35/245) Updating files: 15% (37/245) Updating files: 16% (40/245) Updating files: 17% (42/245) Updating files: 18% (45/245) Updating files: 19% (47/245) Updating files: 20% (49/245) Updating files: 21% (52/245) Updating files: 22% (54/245) Updating files: 23% (57/245) Updating files: 24% (59/245) Updating files: 25% (62/245) Updating files: 26% (64/245) Updating files: 27% (67/245) Updating files: 28% (69/245) Updating files: 28% (71/245) Updating files: 29% (72/245) Updating files: 30% (74/245) Updating files: 31% (76/245) Updating files: 32% (79/245) Updating files: 33% (81/245) Updating files: 34% (84/245) Updating files: 35% (86/245) Updating files: 36% (89/245) Updating files: 37% (91/245) Updating files: 38% (94/245) Updating files: 39% (96/245) Updating files: 40% (98/245) Updating files: 40% (100/245) Updating files: 41% (101/245) Updating files: 42% (103/245) Updating files: 43% (106/245) Updating files: 44% (108/245) Updating files: 45% (111/245) Updating files: 46% (113/245) Updating files: 47% (116/245) Updating files: 48% (118/245) Updating files: 49% (121/245) Updating files: 50% (123/245) Updating files: 51% (125/245) Updating files: 52% (128/245) Updating files: 53% (130/245) Updating files: 54% (133/245) Updating files: 55% (135/245) Updating files: 56% (138/245) Updating files: 57% (140/245) Updating files: 58% (143/245) Updating files: 59% (145/245) Updating files: 60% (147/245) Updating files: 61% (150/245) Updating files: 62% (152/245) Updating files: 63% (155/245) Updating files: 64% (157/245) Updating files: 65% (160/245) Updating files: 66% (162/245) Updating files: 67% (165/245) Updating files: 68% (167/245) Updating files: 69% (170/245) Updating files: 70% (172/245) Updating files: 71% (174/245) Updating files: 72% (177/245) Updating files: 73% (179/245) Updating files: 74% (182/245) Updating files: 75% (184/245) Updating files: 76% (187/245) Updating files: 77% (189/245) Updating files: 77% (191/245) Updating files: 78% (192/245) Updating files: 79% (194/245) Updating files: 80% (196/245) Updating files: 81% (199/245) Updating files: 82% (201/245) Updating files: 83% (204/245) Updating files: 84% (206/245) Updating files: 85% (209/245) Updating files: 86% (211/245) Updating files: 87% (214/245) Updating files: 88% (216/245) Updating files: 89% (219/245) Updating files: 90% (221/245) Updating files: 91% (223/245) Updating files: 91% (224/245) Updating files: 92% (226/245) Updating files: 93% (228/245) Updating files: 94% (231/245) Updating files: 95% (233/245) Updating files: 96% (236/245) Updating files: 97% (238/245) Updating files: 98% (241/245) Updating files: 99% (243/245) Updating files: 100% (245/245) Updating files: 100% (245/245), done. 2024-03-09T09:07:02.387 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 266d2076b6c debian: fix up debian scripts 2024-03-09T09:07:02.394 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.20 2024-03-09T09:07:02.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.419+0000 7f6cd2a00640 1 -- 172.21.15.67:0/3324954295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd40c5e40 msgr2=0x7f6cd4156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:07:02.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.419+0000 7f6cd2a00640 1 --2- 172.21.15.67:0/3324954295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd40c5e40 0x7f6cd4156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:07:02.571 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:07:02.572 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 21 -eq 48 ']' 2024-03-09T09:07:02.572 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.21 2024-03-09T09:07:02.572 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.21 2024-03-09T09:07:02.572 INFO:tasks.workunit.client.1.smithi134.stdout:v0.21 2024-03-09T09:07:02.572 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.21 2024-03-09T09:07:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.820+0000 7f6cd2a00640 1 -- 172.21.15.67:0/3324954295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd40c5e40 msgr2=0x7f6cd4156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:07:02.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:02.820+0000 7f6cd2a00640 1 --2- 172.21.15.67:0/3324954295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd40c5e40 0x7f6cd4156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:07:03.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:03.621+0000 7f6cd2a00640 1 -- 172.21.15.67:0/3324954295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd40c5e40 msgr2=0x7f6cd4156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:07:03.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:03.621+0000 7f6cd2a00640 1 --2- 172.21.15.67:0/3324954295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd40c5e40 0x7f6cd4156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:07:05.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.217+0000 7f6cd0c00640 1 -- 172.21.15.67:0/3324954295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd40c5e40 msgr2=0x7f6cd4156010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:07:05.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.217+0000 7f6cd0c00640 1 --2- 172.21.15.67:0/3324954295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd40c5e40 0x7f6cd4156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:05.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.217+0000 7f6cd0c00640 1 -- 172.21.15.67:0/3324954295 >> v1:172.21.15.134:6789/0 conn(0x7f6cd40cadb0 legacy=0x7f6cd4156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:05.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.217+0000 7f6cd0c00640 1 -- 172.21.15.67:0/3324954295 >> v1:172.21.15.67:6789/0 conn(0x7f6cd4153300 legacy=0x7f6cd40b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.217+0000 7f6cd0c00640 1 --2- 172.21.15.67:0/3324954295 >> v2:172.21.15.67:3300/0 conn(0x7f6cb4003710 0x7f6cb4005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.217+0000 7f6cd0c00640 1 --2- 172.21.15.67:0/3324954295 >> v2:172.21.15.134:3300/0 conn(0x7f6cb40061e0 0x7f6cb4008680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.217+0000 7f6cd0c00640 1 -- 172.21.15.67:0/3324954295 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd40ae6f0 con 0x7f6cb4003710 2024-03-09T09:07:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.217+0000 7f6cd0c00640 1 -- 172.21.15.67:0/3324954295 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb4008d10 con 0x7f6cb40061e0 2024-03-09T09:07:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.218+0000 7f6cd3400640 1 --2- 172.21.15.67:0/3324954295 >> v2:172.21.15.67:3300/0 conn(0x7f6cb4003710 0x7f6cb4005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.218+0000 7f6cd2a00640 1 --2- 172.21.15.67:0/3324954295 >> v2:172.21.15.134:3300/0 conn(0x7f6cb40061e0 0x7f6cb4008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.219+0000 7f6cd2a00640 1 -- 172.21.15.67:0/3324954295 >> v2:172.21.15.134:3300/0 conn(0x7f6cb40061e0 msgr2=0x7f6cb4008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:07:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.219+0000 7f6cd2a00640 1 -- 172.21.15.67:0/3324954295 >> v2:172.21.15.134:3300/0 conn(0x7f6cb40061e0 msgr2=0x7f6cb4008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:07:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.219+0000 7f6cd2a00640 1 --2- 172.21.15.67:0/3324954295 >> v2:172.21.15.134:3300/0 conn(0x7f6cb40061e0 0x7f6cb4008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.219+0000 7f6cd3400640 1 -- 172.21.15.67:0/3324954295 >> v2:172.21.15.67:3300/0 conn(0x7f6cb4003710 msgr2=0x7f6cb4005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:07:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.219+0000 7f6cd2a00640 1 --2- 172.21.15.67:0/3324954295 >> v2:172.21.15.134:3300/0 conn(0x7f6cb40061e0 0x7f6cb4008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.219+0000 7f6cd3400640 1 -- 172.21.15.67:0/3324954295 >> v2:172.21.15.67:3300/0 conn(0x7f6cb4003710 msgr2=0x7f6cb4005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:07:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.219+0000 7f6cd3400640 1 --2- 172.21.15.67:0/3324954295 >> v2:172.21.15.67:3300/0 conn(0x7f6cb4003710 0x7f6cb4005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.219+0000 7f6cd3400640 1 --2- 172.21.15.67:0/3324954295 >> v2:172.21.15.67:3300/0 conn(0x7f6cb4003710 0x7f6cb4005b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:05.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.420+0000 7f6cd3400640 1 --2- 172.21.15.67:0/3324954295 >> v2:172.21.15.67:3300/0 conn(0x7f6cb4003710 0x7f6cb4005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:05.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.420+0000 7f6cd2a00640 1 --2- 172.21.15.67:0/3324954295 >> v2:172.21.15.134:3300/0 conn(0x7f6cb40061e0 0x7f6cb4008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.421+0000 7f6cd3400640 1 -- 172.21.15.67:0/3324954295 >> v2:172.21.15.134:3300/0 conn(0x7f6cb40061e0 msgr2=0x7f6cb4008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.421+0000 7f6cd3400640 1 --2- 172.21.15.67:0/3324954295 >> v2:172.21.15.134:3300/0 conn(0x7f6cb40061e0 0x7f6cb4008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.421+0000 7f6cd3400640 1 -- 172.21.15.67:0/3324954295 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4155b10 con 0x7f6cb4003710 2024-03-09T09:07:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.422+0000 7f6cd3400640 1 --2- 172.21.15.67:0/3324954295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cb4003710 0x7f6cb4005b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40061b0 tx=0x7f6cc402c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9569e58061118ab server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:05.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.423+0000 7f6cd2000640 1 -- 172.21.15.67:0/3324954295 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4039070 con 0x7f6cb4003710 2024-03-09T09:07:05.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.423+0000 7f6cd2000640 1 -- 172.21.15.67:0/3324954295 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc402ce10 con 0x7f6cb4003710 2024-03-09T09:07:05.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.423+0000 7f6cd2000640 1 -- 172.21.15.67:0/3324954295 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4034680 con 0x7f6cb4003710 2024-03-09T09:07:05.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.424+0000 7f6cd8e00640 1 -- 172.21.15.67:0/3324954295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cb4003710 msgr2=0x7f6cb4005b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:05.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.424+0000 7f6cd8e00640 1 --2- 172.21.15.67:0/3324954295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cb4003710 0x7f6cb4005b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40061b0 tx=0x7f6cc402c060 comp rx=0 tx=0).stop 2024-03-09T09:07:05.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.424+0000 7f6cd3e00640 1 -- 172.21.15.67:0/3324954295 reap_dead start 2024-03-09T09:07:05.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.424+0000 7f6cd8e00640 1 -- 172.21.15.67:0/3324954295 shutdown_connections 2024-03-09T09:07:05.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.424+0000 7f6cd8e00640 1 --2- 172.21.15.67:0/3324954295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cb4003710 0x7f6cb4005b90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:05.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.424+0000 7f6cd8e00640 1 -- 172.21.15.67:0/3324954295 >> 172.21.15.67:0/3324954295 conn(0x7f6cd40732d0 msgr2=0x7f6cd40ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:05.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.424+0000 7f6cd8e00640 1 -- 172.21.15.67:0/3324954295 shutdown_connections 2024-03-09T09:07:05.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.424+0000 7f6cd8e00640 1 -- 172.21.15.67:0/3324954295 wait complete. 2024-03-09T09:07:05.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.426+0000 7f6cd8e00640 1 Processor -- start 2024-03-09T09:07:05.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.426+0000 7f6cd8e00640 1 -- start start 2024-03-09T09:07:05.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.427+0000 7f6cd8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cb4003710 0x7f6cd40cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:05.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.427+0000 7f6cd8e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd406d2e0 0x7f6cd41ec250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:05.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.427+0000 7f6cd8e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cd40ac8d0 0x7f6cd40aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:05.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.427+0000 7f6cd8e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd40c59e0 con 0x7f6cb4003710 2024-03-09T09:07:05.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.427+0000 7f6cd8e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4159e90 con 0x7f6cd40ac8d0 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.427+0000 7f6cd8e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd40b0710 con 0x7f6cd406d2e0 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.427+0000 7f6cd2a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd406d2e0 msgr2=0x7f6cd41ec250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.427+0000 7f6cd3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cb4003710 0x7f6cd40cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.427+0000 7f6cd2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd406d2e0 0x7f6cd41ec250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.427+0000 7f6cd3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cb4003710 0x7f6cd40cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50960/0 (socket says 172.21.15.67:50960) 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.427+0000 7f6cd3400640 1 -- 172.21.15.67:0/2293506750 learned_addr learned my addr 172.21.15.67:0/2293506750 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.428+0000 7f6cd3e00640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cd40ac8d0 0x7f6cd40aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.428+0000 7f6cd3400640 1 -- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd406d2e0 msgr2=0x7f6cd41ec250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.428+0000 7f6cd3400640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd406d2e0 0x7f6cd41ec250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.428+0000 7f6cd3400640 1 -- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cd40ac8d0 msgr2=0x7f6cd40aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.428+0000 7f6cd3400640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cd40ac8d0 0x7f6cd40aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.428+0000 7f6cd3400640 1 -- 172.21.15.67:0/2293506750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd406b300 con 0x7f6cb4003710 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.428+0000 7f6cd3400640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cb4003710 0x7f6cd40cb4c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6cc402c670 tx=0x7f6cc4001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:05.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.428+0000 7f6ccb400640 1 -- 172.21.15.67:0/2293506750 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4044030 con 0x7f6cb4003710 2024-03-09T09:07:05.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.429+0000 7f6ccb400640 1 -- 172.21.15.67:0/2293506750 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc402cc10 con 0x7f6cb4003710 2024-03-09T09:07:05.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.429+0000 7f6ccb400640 1 -- 172.21.15.67:0/2293506750 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4039040 con 0x7f6cb4003710 2024-03-09T09:07:05.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.429+0000 7f6cd8e00640 1 -- 172.21.15.67:0/2293506750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd41eb790 con 0x7f6cb4003710 2024-03-09T09:07:05.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.429+0000 7f6cd8e00640 1 -- 172.21.15.67:0/2293506750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd40cbd80 con 0x7f6cb4003710 2024-03-09T09:07:05.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.431+0000 7f6cd8e00640 1 -- 172.21.15.67:0/2293506750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd404f3c0 con 0x7f6cb4003710 2024-03-09T09:07:05.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.432+0000 7f6ccb400640 1 -- 172.21.15.67:0/2293506750 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6cc4047070 con 0x7f6cb4003710 2024-03-09T09:07:05.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.433+0000 7f6ccb400640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb406c260 0x7f6cb406e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:05.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.434+0000 7f6ccb400640 1 -- 172.21.15.67:0/2293506750 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6cc40ac6e0 con 0x7f6cb4003710 2024-03-09T09:07:05.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.434+0000 7f6cd2a00640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb406c260 0x7f6cb406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:05.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.435+0000 7f6cd2a00640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb406c260 0x7f6cb406e720 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6cd414f6a0 tx=0x7f6cc000d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:05.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.442+0000 7f6ccb400640 1 -- 172.21.15.67:0/2293506750 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6cc404e070 con 0x7f6cb4003710 2024-03-09T09:07:05.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.773+0000 7f6cd8e00640 1 -- 172.21.15.67:0/2293506750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6cd40cab90 con 0x7f6cb4003710 2024-03-09T09:07:05.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.774+0000 7f6ccb400640 1 -- 172.21.15.67:0/2293506750 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f6cc403d370 con 0x7f6cb4003710 2024-03-09T09:07:05.791 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":108,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:07:05.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.777+0000 7f6cd8e00640 1 -- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb406c260 msgr2=0x7f6cb406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:05.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.777+0000 7f6cd8e00640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb406c260 0x7f6cb406e720 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6cd414f6a0 tx=0x7f6cc000d040 comp rx=0 tx=0).stop 2024-03-09T09:07:05.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.777+0000 7f6cd8e00640 1 -- 172.21.15.67:0/2293506750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cb4003710 msgr2=0x7f6cd40cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:05.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.777+0000 7f6cd8e00640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cb4003710 0x7f6cd40cb4c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6cc402c670 tx=0x7f6cc4001510 comp rx=0 tx=0).stop 2024-03-09T09:07:05.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.777+0000 7f6cd8e00640 1 -- 172.21.15.67:0/2293506750 shutdown_connections 2024-03-09T09:07:05.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.777+0000 7f6cd8e00640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cb406c260 0x7f6cb406e720 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:05.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.777+0000 7f6cd8e00640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cd40ac8d0 0x7f6cd40aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:05.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.777+0000 7f6cd8e00640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cd406d2e0 0x7f6cd41ec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:05.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.777+0000 7f6cd8e00640 1 --2- 172.21.15.67:0/2293506750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cb4003710 0x7f6cd40cb4c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:05.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.777+0000 7f6cd8e00640 1 -- 172.21.15.67:0/2293506750 >> 172.21.15.67:0/2293506750 conn(0x7f6cd40732d0 msgr2=0x7f6cd40ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:05.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.777+0000 7f6cd8e00640 1 -- 172.21.15.67:0/2293506750 shutdown_connections 2024-03-09T09:07:05.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:05.778+0000 7f6cd8e00640 1 -- 172.21.15.67:0/2293506750 wait complete. 2024-03-09T09:07:05.799 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:07:05.799 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:07:05.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:07:06.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.023+0000 7f5d0dc00640 1 Processor -- start 2024-03-09T09:07:06.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.023+0000 7f5d0dc00640 1 -- start start 2024-03-09T09:07:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.025+0000 7f5d0dc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5d080c68a0 0x7f5d08072810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.025+0000 7f5d0dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d080c2e80 0x7f5d08155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.025+0000 7f5d0dc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d08072d50 con 0x7f5d080c64c0 2024-03-09T09:07:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.025+0000 7f5d0dc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0806f600 con 0x7f5d080c68a0 2024-03-09T09:07:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.025+0000 7f5d0dc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080aa640 con 0x7f5d080c2e80 2024-03-09T09:07:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.025+0000 7f5d0d200640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d080c2e80 msgr2=0x7f5d08155cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:07:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.025+0000 7f5d0d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d080c2e80 0x7f5d08155cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.026+0000 7f5d07400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5d080c68a0 0x7f5d08072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.026+0000 7f5d07e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f5d080c64c0 0x7f5d08070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:45584/0 (socket says 172.21.15.67:45584) 2024-03-09T09:07:06.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.026+0000 7f5d07e00640 1 -- 172.21.15.67:0/1576498388 learned_addr learned my addr 172.21.15.67:0/1576498388 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:07:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.026+0000 7f5d07400640 1 -- 172.21.15.67:0/1576498388 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d080c2e80 msgr2=0x7f5d08155cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:07:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.026+0000 7f5d07400640 1 --2- 172.21.15.67:0/1576498388 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d080c2e80 0x7f5d08155cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.026+0000 7f5d07400640 1 -- 172.21.15.67:0/1576498388 >> v1:172.21.15.134:6789/0 conn(0x7f5d080c64c0 legacy=0x7f5d08070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.027+0000 7f5d07400640 1 -- 172.21.15.67:0/1576498388 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d080aa920 con 0x7f5d080c68a0 2024-03-09T09:07:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.028+0000 7f5d07400640 1 --2- 172.21.15.67:0/1576498388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d080c68a0 0x7f5d08072810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40097a0 tx=0x7f5cf402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0778980aded42d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.028+0000 7f5d06a00640 1 -- 172.21.15.67:0/1576498388 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40315a0 con 0x7f5d080c68a0 2024-03-09T09:07:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.028+0000 7f5d06a00640 1 -- 172.21.15.67:0/1576498388 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf4037070 con 0x7f5d080c68a0 2024-03-09T09:07:06.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.028+0000 7f5d06a00640 1 -- 172.21.15.67:0/1576498388 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4031960 con 0x7f5d080c68a0 2024-03-09T09:07:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.029+0000 7f5d0dc00640 1 -- 172.21.15.67:0/1576498388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d080c68a0 msgr2=0x7f5d08072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.029+0000 7f5d0dc00640 1 --2- 172.21.15.67:0/1576498388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d080c68a0 0x7f5d08072810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40097a0 tx=0x7f5cf402b840 comp rx=0 tx=0).stop 2024-03-09T09:07:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.029+0000 7f5d0dc00640 1 -- 172.21.15.67:0/1576498388 shutdown_connections 2024-03-09T09:07:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.029+0000 7f5d0dc00640 1 --2- 172.21.15.67:0/1576498388 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d080c2e80 0x7f5d08155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.029+0000 7f5d0dc00640 1 --2- 172.21.15.67:0/1576498388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d080c68a0 0x7f5d08072810 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.029+0000 7f5d0dc00640 1 -- 172.21.15.67:0/1576498388 >> 172.21.15.67:0/1576498388 conn(0x7f5d0806f060 msgr2=0x7f5d08066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.029+0000 7f5d0dc00640 1 -- 172.21.15.67:0/1576498388 shutdown_connections 2024-03-09T09:07:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.029+0000 7f5d0dc00640 1 -- 172.21.15.67:0/1576498388 wait complete. 2024-03-09T09:07:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.031+0000 7f5d0dc00640 1 Processor -- start 2024-03-09T09:07:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.031+0000 7f5d0dc00640 1 -- start start 2024-03-09T09:07:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d0dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d080c2e80 0x7f5d080adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d0dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d080c64c0 0x7f5d080ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d0dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d080c68a0 0x7f5d0813ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d0dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d08156210 con 0x7f5d080c64c0 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d0dc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0806fb30 con 0x7f5d080c68a0 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d0dc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d08156390 con 0x7f5d080c2e80 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d07e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d080c2e80 msgr2=0x7f5d080adf60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d07e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d080c2e80 0x7f5d080adf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d07400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d080c64c0 0x7f5d080ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d07400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d080c64c0 0x7f5d080ad380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50962/0 (socket says 172.21.15.67:50962) 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d07400640 1 -- 172.21.15.67:0/2395417622 learned_addr learned my addr 172.21.15.67:0/2395417622 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.032+0000 7f5d0d200640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d080c68a0 0x7f5d0813ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.033+0000 7f5d07400640 1 -- 172.21.15.67:0/2395417622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d080c2e80 msgr2=0x7f5d080adf60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.033+0000 7f5d07400640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d080c2e80 0x7f5d080adf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.033+0000 7f5d07400640 1 -- 172.21.15.67:0/2395417622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d080c68a0 msgr2=0x7f5d0813ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.033+0000 7f5d07400640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d080c68a0 0x7f5d0813ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.033+0000 7f5d07400640 1 -- 172.21.15.67:0/2395417622 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d081e12c0 con 0x7f5d080c64c0 2024-03-09T09:07:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.033+0000 7f5d07400640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d080c64c0 0x7f5d080ad380 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40098c0 tx=0x7f5cf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.033+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4049070 con 0x7f5d080c64c0 2024-03-09T09:07:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.033+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf4032640 con 0x7f5d080c64c0 2024-03-09T09:07:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.033+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4032a10 con 0x7f5d080c64c0 2024-03-09T09:07:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.033+0000 7f5d0dc00640 1 -- 172.21.15.67:0/2395417622 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0813ddf0 con 0x7f5d080c64c0 2024-03-09T09:07:06.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.034+0000 7f5d0dc00640 1 -- 172.21.15.67:0/2395417622 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d081df580 con 0x7f5d080c64c0 2024-03-09T09:07:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.034+0000 7f5d0dc00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d081df750 0x7f5d0813d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.035+0000 7f5d0dc00640 1 -- 172.21.15.67:0/2395417622 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0804f3c0 con 0x7f5d081df750 2024-03-09T09:07:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.035+0000 7f5d07e00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d081df750 0x7f5d0813d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.035+0000 7f5d07e00640 1 -- 172.21.15.67:0/2395417622 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d080c2c60 con 0x7f5d081df750 2024-03-09T09:07:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.036+0000 7f5d07e00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d081df750 0x7f5d0813d370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8004810 tx=0x7f5cf800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.036+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5cf4032bb0 con 0x7f5d080c64c0 2024-03-09T09:07:06.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.037+0000 7f5cffe00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5ce806b480 0x7f5ce806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.037+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f5cf402f020 con 0x7f5d080c64c0 2024-03-09T09:07:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.037+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8017070 con 0x7f5d081df750 2024-03-09T09:07:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.037+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f5cf8005ce0 con 0x7f5d081df750 2024-03-09T09:07:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.037+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d081df750 msgr2=0x7f5d0813d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.037+0000 7f5cffe00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d081df750 0x7f5d0813d370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8004810 tx=0x7f5cf800c8e0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.037+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d081df750 msgr2=0x7f5d0813d370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.037+0000 7f5cffe00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d081df750 0x7f5d0813d370 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8004810 tx=0x7f5cf800c8e0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.037+0000 7f5d0d200640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5ce806b480 0x7f5ce806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.038+0000 7f5d0d200640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5ce806b480 0x7f5ce806d940 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5d080ab470 tx=0x7f5d00009450 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.044+0000 7f5d0dc00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d0813e060 0x7f5d081ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.044+0000 7f5d0dc00640 1 -- 172.21.15.67:0/2395417622 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080c5bb0 con 0x7f5d0813e060 2024-03-09T09:07:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.045+0000 7f5d07e00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d0813e060 0x7f5d081ebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.045+0000 7f5d07e00640 1 -- 172.21.15.67:0/2395417622 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5d0813d8b0 con 0x7f5d0813e060 2024-03-09T09:07:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.046+0000 7f5d07e00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d0813e060 0x7f5d081ebc10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8007fd0 tx=0x7f5cf8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.046+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8017040 con 0x7f5d0813e060 2024-03-09T09:07:06.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.046+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7f5cf801a8c0 con 0x7f5d0813e060 2024-03-09T09:07:06.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.046+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d0813e060 msgr2=0x7f5d081ebc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.046+0000 7f5cffe00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d0813e060 0x7f5d081ebc10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8007fd0 tx=0x7f5cf8000bc0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.046+0000 7f5cffe00640 1 -- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d0813e060 msgr2=0x7f5d081ebc10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:06.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.046+0000 7f5cffe00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d0813e060 0x7f5d081ebc10 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8007fd0 tx=0x7f5cf8000bc0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 108, 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:06.051 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:07:06.052 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:07:06.052 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:06.052 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:06.052 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:06.052 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:06.052 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:06.052 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:06.052 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:06.052 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:06.052 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:06.052 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:06.053 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:06.054 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:06.054 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:06.054 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:06.054 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.054 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:07:06.054 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:07:06.054 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:07:06.054 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:07:06.054 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:06.055 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:07:06.056 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:06.056 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:06.056 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:06.056 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:06.056 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:06.056 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:06.056 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:06.056 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:06.056 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:06.056 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.057 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:06.058 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:07:06.058 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.058 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:06.058 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:07:06.058 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.058 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.058 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:06.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.060 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.061 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:07:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:07:06.063 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:06.063 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:06.063 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:06.063 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.063 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.063 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:07:06.064 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:07:06.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.064 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:06.064 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:06.064 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:07:06.064 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.064 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:06.064 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:07:06.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.065 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:06.065 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:06.065 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:07:06.065 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.065 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:06.065 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:07:06.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.065 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:06.066 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:06.066 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:07:06.066 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.066 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:06.066 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:07:06.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.066 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:07:06.066 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:06.067 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:06.068 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:07:06.068 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.068 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.068 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:07:06.068 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:07:06.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.047+0000 7f5d0dc00640 1 -- 172.21.15.67:0/2395417622 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5ce806b480 msgr2=0x7f5ce806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.047+0000 7f5d0dc00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5ce806b480 0x7f5ce806d940 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5d080ab470 tx=0x7f5d00009450 comp rx=0 tx=0).stop 2024-03-09T09:07:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.047+0000 7f5d0dc00640 1 -- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d080c64c0 msgr2=0x7f5d080ad380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.047+0000 7f5d0dc00640 1 --2- 172.21.15.67:0/2395417622 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d080c64c0 0x7f5d080ad380 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40098c0 tx=0x7f5cf40046c0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.047+0000 7f5d0d200640 1 -- 172.21.15.67:0/2395417622 reap_dead start 2024-03-09T09:07:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.047+0000 7f5d0d200640 1 -- 172.21.15.67:0/2395417622 reap_dead start 2024-03-09T09:07:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.047+0000 7f5d0dc00640 1 -- 172.21.15.67:0/2395417622 shutdown_connections 2024-03-09T09:07:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.047+0000 7f5d0dc00640 1 -- 172.21.15.67:0/2395417622 >> 172.21.15.67:0/2395417622 conn(0x7f5d0806f060 msgr2=0x7f5d08154770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.047+0000 7f5d0dc00640 1 -- 172.21.15.67:0/2395417622 shutdown_connections 2024-03-09T09:07:06.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.047+0000 7f5d0dc00640 1 -- 172.21.15.67:0/2395417622 wait complete. 2024-03-09T09:07:06.071 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:07:06.071 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:07:06.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:07:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.281+0000 7f5a83200640 1 Processor -- start 2024-03-09T09:07:06.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.281+0000 7f5a83200640 1 -- start start 2024-03-09T09:07:06.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.283+0000 7f5a83200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a7c0c9380 0x7f5a7c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.283+0000 7f5a83200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7c0c9890 con 0x7f5a7c0ad440 2024-03-09T09:07:06.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.283+0000 7f5a83200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7c0c5f80 con 0x7f5a7c14fd90 2024-03-09T09:07:06.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.283+0000 7f5a83200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7c06ba60 con 0x7f5a7c0c9380 2024-03-09T09:07:06.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.283+0000 7f5a81400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a7c0c9380 0x7f5a7c0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.283+0000 7f5a81e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f5a7c14fd90 0x7f5a7c156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:45588/0 (socket says 172.21.15.67:45588) 2024-03-09T09:07:06.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.283+0000 7f5a81e00640 1 -- 172.21.15.67:0/226906705 learned_addr learned my addr 172.21.15.67:0/226906705 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:07:06.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.284+0000 7f5a81400640 1 -- 172.21.15.67:0/226906705 >> v1:172.21.15.134:6789/0 conn(0x7f5a7c14fd90 legacy=0x7f5a7c156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.284+0000 7f5a81400640 1 -- 172.21.15.67:0/226906705 >> v1:172.21.15.67:6789/0 conn(0x7f5a7c0ad440 legacy=0x7f5a7c06c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.284+0000 7f5a81400640 1 -- 172.21.15.67:0/226906705 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c0c6240 con 0x7f5a7c0c9380 2024-03-09T09:07:06.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.284+0000 7f5a80a00640 1 -- 172.21.15.67:0/226906705 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647523012 0 0) 0x7f5a7c0c5f80 con 0x7f5a7c14fd90 2024-03-09T09:07:06.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.285+0000 7f5a81400640 1 --2- 172.21.15.67:0/226906705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0c9380 0x7f5a7c0acf00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c06bbe0 tx=0x7f5a6c02b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65bf47344af92ba server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:06.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.286+0000 7f5a80a00640 1 -- 172.21.15.67:0/226906705 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c037070 con 0x7f5a7c0c9380 2024-03-09T09:07:06.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.286+0000 7f5a80a00640 1 -- 172.21.15.67:0/226906705 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c030710 con 0x7f5a7c0c9380 2024-03-09T09:07:06.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.286+0000 7f5a80a00640 1 -- 172.21.15.67:0/226906705 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c02be60 con 0x7f5a7c0c9380 2024-03-09T09:07:06.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.286+0000 7f5a83200640 1 -- 172.21.15.67:0/226906705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0c9380 msgr2=0x7f5a7c0acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.286+0000 7f5a83200640 1 --2- 172.21.15.67:0/226906705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0c9380 0x7f5a7c0acf00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c06bbe0 tx=0x7f5a6c02b660 comp rx=0 tx=0).stop 2024-03-09T09:07:06.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.287+0000 7f5a83200640 1 -- 172.21.15.67:0/226906705 shutdown_connections 2024-03-09T09:07:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.287+0000 7f5a83200640 1 --2- 172.21.15.67:0/226906705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c0c9380 0x7f5a7c0acf00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.287+0000 7f5a83200640 1 -- 172.21.15.67:0/226906705 >> 172.21.15.67:0/226906705 conn(0x7f5a7c0cb480 msgr2=0x7f5a7c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.287+0000 7f5a83200640 1 -- 172.21.15.67:0/226906705 shutdown_connections 2024-03-09T09:07:06.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.287+0000 7f5a83200640 1 -- 172.21.15.67:0/226906705 wait complete. 2024-03-09T09:07:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.288+0000 7f5a83200640 1 Processor -- start 2024-03-09T09:07:06.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.289+0000 7f5a83200640 1 -- start start 2024-03-09T09:07:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.289+0000 7f5a83200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0ad440 0x7f5a7c0c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.289+0000 7f5a83200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a7c0c9380 0x7f5a7c0c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.289+0000 7f5a83200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c14fd90 0x7f5a7c1eee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.289+0000 7f5a83200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c06c8e0 con 0x7f5a7c14fd90 2024-03-09T09:07:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.289+0000 7f5a83200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c06bcb0 con 0x7f5a7c0ad440 2024-03-09T09:07:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a83200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c06ca60 con 0x7f5a7c0c9380 2024-03-09T09:07:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a81400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a7c0c9380 msgr2=0x7f5a7c0c3930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:07:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a81400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a7c0c9380 0x7f5a7c0c3930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a82800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c14fd90 0x7f5a7c1eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a82800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c14fd90 0x7f5a7c1eee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50990/0 (socket says 172.21.15.67:50990) 2024-03-09T09:07:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a82800640 1 -- 172.21.15.67:0/250974634 learned_addr learned my addr 172.21.15.67:0/250974634 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a81e00640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0ad440 0x7f5a7c0c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a82800640 1 -- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a7c0c9380 msgr2=0x7f5a7c0c3930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:07:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a82800640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a7c0c9380 0x7f5a7c0c3930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a82800640 1 -- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0ad440 msgr2=0x7f5a7c0c33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a82800640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0ad440 0x7f5a7c0c33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.290+0000 7f5a82800640 1 -- 172.21.15.67:0/250974634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c0ab090 con 0x7f5a7c14fd90 2024-03-09T09:07:06.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.291+0000 7f5a82800640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c14fd90 0x7f5a7c1eee60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800cb20 tx=0x7f5a780020d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:06.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.291+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78017070 con 0x7f5a7c14fd90 2024-03-09T09:07:06.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.291+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a78003da0 con 0x7f5a7c14fd90 2024-03-09T09:07:06.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.291+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800a900 con 0x7f5a7c14fd90 2024-03-09T09:07:06.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.291+0000 7f5a83200640 1 -- 172.21.15.67:0/250974634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7c159b20 con 0x7f5a7c14fd90 2024-03-09T09:07:06.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.291+0000 7f5a83200640 1 -- 172.21.15.67:0/250974634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7c1ec5c0 con 0x7f5a7c14fd90 2024-03-09T09:07:06.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.292+0000 7f5a83200640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c5290 0x7f5a7c1ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.292+0000 7f5a83200640 1 -- 172.21.15.67:0/250974634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0c2550 con 0x7f5a7c0c5290 2024-03-09T09:07:06.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.293+0000 7f5a81e00640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c5290 0x7f5a7c1ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.293+0000 7f5a81e00640 1 -- 172.21.15.67:0/250974634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a7c0c5820 con 0x7f5a7c0c5290 2024-03-09T09:07:06.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.294+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5a7801a070 con 0x7f5a7c14fd90 2024-03-09T09:07:06.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.294+0000 7f5a81e00640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c5290 0x7f5a7c1ef3a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c147500 tx=0x7f5a700017e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:06.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.295+0000 7f5a76000640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5a6006b480 0x7f5a6006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.296+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f5a78089cb0 con 0x7f5a7c14fd90 2024-03-09T09:07:06.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.296+0000 7f5a81400640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5a6006b480 0x7f5a6006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.296+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a70019070 con 0x7f5a7c0c5290 2024-03-09T09:07:06.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.296+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f5a70011580 con 0x7f5a7c0c5290 2024-03-09T09:07:06.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.296+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c5290 msgr2=0x7f5a7c1ef3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.296+0000 7f5a76000640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c5290 0x7f5a7c1ef3a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c147500 tx=0x7f5a700017e0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.296+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c5290 msgr2=0x7f5a7c1ef3a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:06.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.296+0000 7f5a76000640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c5290 0x7f5a7c1ef3a0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c147500 tx=0x7f5a700017e0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.297+0000 7f5a81400640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5a6006b480 0x7f5a6006d940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c034000 tx=0x7f5a6c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:06.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.303+0000 7f5a83200640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c2c10 0x7f5a7c1ef8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:06.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.303+0000 7f5a83200640 1 -- 172.21.15.67:0/250974634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c04f3c0 con 0x7f5a7c0c2c10 2024-03-09T09:07:06.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.304+0000 7f5a81e00640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c2c10 0x7f5a7c1ef8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:06.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.305+0000 7f5a81e00640 1 -- 172.21.15.67:0/250974634 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5a7c06d2d0 con 0x7f5a7c0c2c10 2024-03-09T09:07:06.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.305+0000 7f5a81e00640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c2c10 0x7f5a7c1ef8e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5a70001eb0 tx=0x7f5a70007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:06.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.306+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a70019030 con 0x7f5a7c0c2c10 2024-03-09T09:07:06.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.306+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4523 (secure 0 0 0) 0x7f5a700263f0 con 0x7f5a7c0c2c10 2024-03-09T09:07:06.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.306+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c2c10 msgr2=0x7f5a7c1ef8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.306+0000 7f5a76000640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c2c10 0x7f5a7c1ef8e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5a70001eb0 tx=0x7f5a70007f90 comp rx=0 tx=0).stop 2024-03-09T09:07:06.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.306+0000 7f5a76000640 1 -- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c2c10 msgr2=0x7f5a7c1ef8e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:06.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.306+0000 7f5a76000640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a7c0c2c10 0x7f5a7c1ef8e0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5a70001eb0 tx=0x7f5a70007f90 comp rx=0 tx=0).stop 2024-03-09T09:07:06.310 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:07:06.310 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:07:06.310 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 108, 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:06.311 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:06.312 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:06.313 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:06.313 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:06.313 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:06.313 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:06.313 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:06.313 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:06.313 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:06.313 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.313 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:07:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:07:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:07:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:07:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:07:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:07:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:07:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:07:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:07:06.314 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:06.315 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:07:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:07:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:07:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:07:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:07:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:06.317 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.318 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:07:06.319 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:07:06.320 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:06.321 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:07:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:07:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:06.323 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:07:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:06.324 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:07:06.324 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:07:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.307+0000 7f5a83200640 1 -- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5a6006b480 msgr2=0x7f5a6006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.307+0000 7f5a83200640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5a6006b480 0x7f5a6006d940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c034000 tx=0x7f5a6c008030 comp rx=0 tx=0).stop 2024-03-09T09:07:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.307+0000 7f5a83200640 1 -- 172.21.15.67:0/250974634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c14fd90 msgr2=0x7f5a7c1eee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.307+0000 7f5a83200640 1 --2- 172.21.15.67:0/250974634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a7c14fd90 0x7f5a7c1eee60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800cb20 tx=0x7f5a780020d0 comp rx=0 tx=0).stop 2024-03-09T09:07:06.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.307+0000 7f5a82800640 1 -- 172.21.15.67:0/250974634 reap_dead start 2024-03-09T09:07:06.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.307+0000 7f5a82800640 1 -- 172.21.15.67:0/250974634 reap_dead start 2024-03-09T09:07:06.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.307+0000 7f5a83200640 1 -- 172.21.15.67:0/250974634 shutdown_connections 2024-03-09T09:07:06.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.307+0000 7f5a83200640 1 -- 172.21.15.67:0/250974634 >> 172.21.15.67:0/250974634 conn(0x7f5a7c0cb480 msgr2=0x7f5a7c1df260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:06.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.307+0000 7f5a83200640 1 -- 172.21.15.67:0/250974634 shutdown_connections 2024-03-09T09:07:06.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:06.307+0000 7f5a83200640 1 -- 172.21.15.67:0/250974634 wait complete. 2024-03-09T09:07:06.329 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:07:06.329 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:07:16.107 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:07:16.115 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:07:18.006 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 10% (58/537) Updating files: 11% (60/537) Updating files: 12% (65/537) Updating files: 13% (70/537) Updating files: 14% (76/537) Updating files: 15% (81/537) Updating files: 16% (86/537) Updating files: 17% (92/537) Updating files: 18% (97/537) Updating files: 19% (103/537) Updating files: 20% (108/537) Updating files: 21% (113/537) Updating files: 22% (119/537) Updating files: 22% (120/537) Updating files: 23% (124/537) Updating files: 24% (129/537) Updating files: 24% (132/537) Updating files: 25% (135/537) Updating files: 26% (140/537) Updating files: 27% (145/537) Updating files: 28% (151/537) Updating files: 29% (156/537) Updating files: 30% (162/537) Updating files: 31% (167/537) Updating files: 32% (172/537) Updating files: 33% (178/537) Updating files: 34% (183/537) Updating files: 35% (188/537) Updating files: 35% (192/537) Updating files: 36% (194/537) Updating files: 37% (199/537) Updating files: 38% (205/537) Updating files: 39% (210/537) Updating files: 40% (215/537) Updating files: 41% (221/537) Updating files: 42% (226/537) Updating files: 43% (231/537) Updating files: 44% (237/537) Updating files: 45% (242/537) Updating files: 46% (248/537) Updating files: 47% (253/537) Updating files: 48% (258/537) Updating files: 49% (264/537) Updating files: 50% (269/537) Updating files: 51% (274/537) Updating files: 52% (280/537) Updating files: 53% (285/537) Updating files: 54% (290/537) Updating files: 55% (296/537) Updating files: 56% (301/537) Updating files: 57% (307/537) Updating files: 58% (312/537) Updating files: 59% (317/537) Updating files: 59% (319/537) Updating files: 60% (323/537) Updating files: 61% (328/537) Updating files: 62% (333/537) Updating files: 63% (339/537) Updating files: 64% (344/537) Updating files: 65% (350/537) Updating files: 66% (355/537) Updating files: 67% (360/537) Updating files: 68% (366/537) Updating files: 69% (371/537) Updating files: 70% (376/537) Updating files: 71% (382/537) Updating files: 72% (387/537) Updating files: 73% (393/537) Updating files: 74% (398/537) Updating files: 75% (403/537) Updating files: 76% (409/537) Updating files: 77% (414/537) Updating files: 78% (419/537) Updating files: 79% (425/537) Updating files: 80% (430/537) Updating files: 81% (435/537) Updating files: 82% (441/537) Updating files: 83% (446/537) Updating files: 84% (452/537) Updating files: 85% (457/537) Updating files: 86% (462/537) Updating files: 87% (468/537) Updating files: 88% (473/537) Updating files: 89% (478/537) Updating files: 90% (484/537) Updating files: 91% (489/537) Updating files: 92% (495/537) Updating files: 93% (500/537) Updating files: 94% (505/537) Updating files: 95% (511/537) Updating files: 96% (516/537) Updating files: 97% (521/537) Updating files: 98% (527/537) Updating files: 99% (532/537) Updating files: 100% (537/537) Updating files: 100% (537/537), done. 2024-03-09T09:07:18.340 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 090436f56ac v0.21 2024-03-09T09:07:18.353 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.21 2024-03-09T09:07:18.419 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:07:18.419 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 22 -eq 48 ']' 2024-03-09T09:07:18.419 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.22 2024-03-09T09:07:18.419 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.22 2024-03-09T09:07:18.419 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.22 2024-03-09T09:07:18.420 INFO:tasks.workunit.client.0.smithi067.stdout:v0.22 2024-03-09T09:07:19.178 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 14% (76/537) Updating files: 15% (81/537) Updating files: 16% (86/537) Updating files: 17% (92/537) Updating files: 18% (97/537) Updating files: 19% (103/537) Updating files: 20% (108/537) Updating files: 21% (113/537) Updating files: 22% (119/537) Updating files: 23% (124/537) Updating files: 24% (129/537) Updating files: 25% (135/537) Updating files: 26% (140/537) Updating files: 27% (145/537) Updating files: 28% (151/537) Updating files: 29% (156/537) Updating files: 30% (162/537) Updating files: 30% (165/537) Updating files: 31% (167/537) Updating files: 32% (172/537) Updating files: 33% (178/537) Updating files: 34% (183/537) Updating files: 35% (188/537) Updating files: 36% (194/537) Updating files: 37% (199/537) Updating files: 38% (205/537) Updating files: 39% (210/537) Updating files: 40% (215/537) Updating files: 41% (221/537) Updating files: 42% (226/537) Updating files: 43% (231/537) Updating files: 44% (237/537) Updating files: 45% (242/537) Updating files: 46% (248/537) Updating files: 47% (253/537) Updating files: 48% (258/537) Updating files: 49% (264/537) Updating files: 50% (269/537) Updating files: 51% (274/537) Updating files: 52% (280/537) Updating files: 52% (283/537) Updating files: 53% (285/537) Updating files: 54% (290/537) Updating files: 55% (296/537) Updating files: 56% (301/537) Updating files: 57% (307/537) Updating files: 58% (312/537) Updating files: 59% (317/537) Updating files: 60% (323/537) Updating files: 61% (328/537) Updating files: 61% (330/537) Updating files: 62% (333/537) Updating files: 63% (339/537) Updating files: 64% (344/537) Updating files: 65% (350/537) Updating files: 66% (355/537) Updating files: 67% (360/537) Updating files: 68% (366/537) Updating files: 68% (370/537) Updating files: 69% (371/537) Updating files: 70% (376/537) Updating files: 71% (382/537) Updating files: 72% (387/537) Updating files: 73% (393/537) Updating files: 74% (398/537) Updating files: 75% (403/537) Updating files: 76% (409/537) Updating files: 77% (414/537) Updating files: 78% (419/537) Updating files: 78% (421/537) Updating files: 79% (425/537) Updating files: 80% (430/537) Updating files: 81% (435/537) Updating files: 82% (441/537) Updating files: 83% (446/537) Updating files: 83% (449/537) Updating files: 84% (452/537) Updating files: 85% (457/537) Updating files: 86% (462/537) Updating files: 87% (468/537) Updating files: 88% (473/537) Updating files: 89% (478/537) Updating files: 90% (484/537) Updating files: 91% (489/537) Updating files: 92% (495/537) Updating files: 93% (500/537) Updating files: 94% (505/537) Updating files: 95% (511/537) Updating files: 96% (516/537) Updating files: 97% (521/537) Updating files: 98% (527/537) Updating files: 99% (532/537) Updating files: 100% (537/537) Updating files: 100% (537/537), done. 2024-03-09T09:07:19.231 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 090436f56ac v0.21 2024-03-09T09:07:19.249 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.21 2024-03-09T09:07:19.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:07:19.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 22 -eq 48 ']' 2024-03-09T09:07:19.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.22 2024-03-09T09:07:19.833 INFO:tasks.workunit.client.1.smithi134.stdout:v0.22 2024-03-09T09:07:19.834 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.22 2024-03-09T09:07:19.834 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.22 2024-03-09T09:07:22.893 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 63% (129/204) Updating files: 64% (131/204) Updating files: 65% (133/204) Updating files: 66% (135/204) Updating files: 67% (137/204) Updating files: 68% (139/204) Updating files: 69% (141/204) Updating files: 70% (143/204) Updating files: 71% (145/204) Updating files: 72% (147/204) Updating files: 73% (149/204) Updating files: 74% (151/204) Updating files: 75% (153/204) Updating files: 76% (156/204) Updating files: 77% (158/204) Updating files: 78% (160/204) Updating files: 79% (162/204) Updating files: 80% (164/204) Updating files: 81% (166/204) Updating files: 82% (168/204) Updating files: 83% (170/204) Updating files: 84% (172/204) Updating files: 85% (174/204) Updating files: 86% (176/204) Updating files: 87% (178/204) Updating files: 88% (180/204) Updating files: 89% (182/204) Updating files: 90% (184/204) Updating files: 91% (186/204) Updating files: 92% (188/204) Updating files: 93% (190/204) Updating files: 94% (192/204) Updating files: 94% (193/204) Updating files: 95% (194/204) Updating files: 96% (196/204) Updating files: 97% (198/204) Updating files: 98% (200/204) Updating files: 99% (202/204) Updating files: 100% (204/204) Updating files: 100% (204/204), done. 2024-03-09T09:07:22.944 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 8a7c95f60ad v0.22 2024-03-09T09:07:22.959 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.22 2024-03-09T09:07:23.067 INFO:tasks.workunit.client.1.smithi134.stdout:v0.23 2024-03-09T09:07:23.067 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:07:23.068 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 23 -eq 48 ']' 2024-03-09T09:07:23.068 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.23 2024-03-09T09:07:23.068 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.23 2024-03-09T09:07:23.068 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.23 2024-03-09T09:07:24.660 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 11% (23/204) Updating files: 12% (25/204) Updating files: 13% (27/204) Updating files: 14% (29/204) Updating files: 15% (31/204) Updating files: 16% (33/204) Updating files: 17% (35/204) Updating files: 18% (37/204) Updating files: 19% (39/204) Updating files: 20% (41/204) Updating files: 21% (43/204) Updating files: 22% (45/204) Updating files: 23% (47/204) Updating files: 23% (48/204) Updating files: 24% (49/204) Updating files: 25% (51/204) Updating files: 26% (54/204) Updating files: 27% (56/204) Updating files: 28% (58/204) Updating files: 29% (60/204) Updating files: 30% (62/204) Updating files: 31% (64/204) Updating files: 32% (66/204) Updating files: 33% (68/204) Updating files: 34% (70/204) Updating files: 35% (72/204) Updating files: 36% (74/204) Updating files: 37% (76/204) Updating files: 38% (78/204) Updating files: 39% (80/204) Updating files: 40% (82/204) Updating files: 41% (84/204) Updating files: 42% (86/204) Updating files: 43% (88/204) Updating files: 44% (90/204) Updating files: 45% (92/204) Updating files: 46% (94/204) Updating files: 47% (96/204) Updating files: 48% (98/204) Updating files: 49% (100/204) Updating files: 50% (102/204) Updating files: 51% (105/204) Updating files: 52% (107/204) Updating files: 53% (109/204) Updating files: 54% (111/204) Updating files: 55% (113/204) Updating files: 56% (115/204) Updating files: 57% (117/204) Updating files: 58% (119/204) Updating files: 59% (121/204) Updating files: 60% (123/204) Updating files: 61% (125/204) Updating files: 62% (127/204) Updating files: 63% (129/204) Updating files: 64% (131/204) Updating files: 65% (133/204) Updating files: 66% (135/204) Updating files: 67% (137/204) Updating files: 68% (139/204) Updating files: 69% (141/204) Updating files: 70% (143/204) Updating files: 71% (145/204) Updating files: 72% (147/204) Updating files: 73% (149/204) Updating files: 74% (151/204) Updating files: 75% (153/204) Updating files: 76% (156/204) Updating files: 77% (158/204) Updating files: 78% (160/204) Updating files: 79% (162/204) Updating files: 80% (164/204) Updating files: 81% (166/204) Updating files: 82% (168/204) Updating files: 83% (170/204) Updating files: 83% (171/204) Updating files: 84% (172/204) Updating files: 85% (174/204) Updating files: 86% (176/204) Updating files: 87% (178/204) Updating files: 88% (180/204) Updating files: 89% (182/204) Updating files: 90% (184/204) Updating files: 91% (186/204) Updating files: 92% (188/204) Updating files: 93% (190/204) Updating files: 94% (192/204) Updating files: 95% (194/204) Updating files: 96% (196/204) Updating files: 97% (198/204) Updating files: 98% (200/204) Updating files: 99% (202/204) Updating files: 100% (204/204) Updating files: 100% (204/204), done. 2024-03-09T09:07:24.696 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 8a7c95f60ad v0.22 2024-03-09T09:07:24.711 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.22 2024-03-09T09:07:25.812 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:07:25.813 INFO:tasks.workunit.client.0.smithi067.stdout:v0.23 2024-03-09T09:07:25.813 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 23 -eq 48 ']' 2024-03-09T09:07:25.813 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.23 2024-03-09T09:07:25.813 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.23 2024-03-09T09:07:25.813 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.23 2024-03-09T09:07:26.330 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:07:26.330 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:07:26.330 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:07:26.330 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:07:26.337 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:07:26.338 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:07:26.338 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:07:26.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.569+0000 7f1799200640 1 Processor -- start 2024-03-09T09:07:26.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.569+0000 7f1799200640 1 -- start start 2024-03-09T09:07:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.571+0000 7f1799200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940ac730 0x7f1794071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.571+0000 7f1799200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f17940c22a0 0x7f17940acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.571+0000 7f1799200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1794153d40 con 0x7f1794157650 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.571+0000 7f1799200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1794070ee0 con 0x7f17940c22a0 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.571+0000 7f1799200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1794154d00 con 0x7f17940ac730 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1791e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940ac730 0x7f1794071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1791e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940ac730 0x7f1794071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:45926/0 (socket says 172.21.15.67:45926) 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1791e00640 1 -- 172.21.15.67:0/4199535222 learned_addr learned my addr 172.21.15.67:0/4199535222 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1789400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f17940c22a0 0x7f17940acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1792800640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f1794157650 0x7f1794157b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:34558/0 (socket says 172.21.15.67:34558) 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1791e00640 1 -- 172.21.15.67:0/4199535222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940ac730 msgr2=0x7f1794071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1791e00640 1 -- 172.21.15.67:0/4199535222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940ac730 msgr2=0x7f1794071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1791e00640 1 --2- 172.21.15.67:0/4199535222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940ac730 0x7f1794071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1791e00640 1 --2- 172.21.15.67:0/4199535222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940ac730 0x7f1794071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1789400640 1 -- 172.21.15.67:0/4199535222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940ac730 msgr2=0x7f1794071190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:07:26.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1789400640 1 --2- 172.21.15.67:0/4199535222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940ac730 0x7f1794071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1789400640 1 -- 172.21.15.67:0/4199535222 >> v1:172.21.15.134:6789/0 conn(0x7f1794157650 legacy=0x7f1794157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1791400640 1 -- 172.21.15.67:0/4199535222 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004268999 0 0) 0x7f1794153d40 con 0x7f1794157650 2024-03-09T09:07:26.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.572+0000 7f1789400640 1 -- 172.21.15.67:0/4199535222 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17940c2790 con 0x7f17940c22a0 2024-03-09T09:07:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.574+0000 7f1789400640 1 --2- 172.21.15.67:0/4199535222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940c22a0 0x7f17940acb10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f177c0097a0 tx=0x7f177c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91253d4d8324c154 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.574+0000 7f1791400640 1 -- 172.21.15.67:0/4199535222 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c0315a0 con 0x7f17940c22a0 2024-03-09T09:07:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.574+0000 7f1791400640 1 -- 172.21.15.67:0/4199535222 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c037070 con 0x7f17940c22a0 2024-03-09T09:07:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.574+0000 7f1791400640 1 -- 172.21.15.67:0/4199535222 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c031960 con 0x7f17940c22a0 2024-03-09T09:07:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.575+0000 7f1799200640 1 -- 172.21.15.67:0/4199535222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940c22a0 msgr2=0x7f17940acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.575+0000 7f1799200640 1 --2- 172.21.15.67:0/4199535222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940c22a0 0x7f17940acb10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f177c0097a0 tx=0x7f177c02b840 comp rx=0 tx=0).stop 2024-03-09T09:07:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.575+0000 7f1799200640 1 -- 172.21.15.67:0/4199535222 shutdown_connections 2024-03-09T09:07:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.575+0000 7f1799200640 1 --2- 172.21.15.67:0/4199535222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940c22a0 0x7f17940acb10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.575+0000 7f1799200640 1 --2- 172.21.15.67:0/4199535222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940ac730 0x7f1794071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:26.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.575+0000 7f1799200640 1 -- 172.21.15.67:0/4199535222 >> 172.21.15.67:0/4199535222 conn(0x7f17940c7370 msgr2=0x7f17940c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.575+0000 7f1799200640 1 -- 172.21.15.67:0/4199535222 shutdown_connections 2024-03-09T09:07:26.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.575+0000 7f1799200640 1 -- 172.21.15.67:0/4199535222 wait complete. 2024-03-09T09:07:26.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.577+0000 7f1799200640 1 Processor -- start 2024-03-09T09:07:26.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.578+0000 7f1799200640 1 -- start start 2024-03-09T09:07:26.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.579+0000 7f1799200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940ac730 0x7f17940c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:26.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.579+0000 7f1799200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940c22a0 0x7f17941ec010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.579+0000 7f1799200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1794157650 0x7f17940be810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.579+0000 7f1799200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1794151610 con 0x7f1794157650 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.579+0000 7f1799200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1794070c00 con 0x7f17940ac730 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.579+0000 7f1799200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1794151790 con 0x7f17940c22a0 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1792800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1794157650 0x7f17940be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1789400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940c22a0 0x7f17941ec010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1792800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1794157650 0x7f17940be810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38298/0 (socket says 172.21.15.67:38298) 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1792800640 1 -- 172.21.15.67:0/380194481 learned_addr learned my addr 172.21.15.67:0/380194481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1791e00640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940ac730 0x7f17940c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1789400640 1 -- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940c22a0 msgr2=0x7f17941ec010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1789400640 1 -- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940c22a0 msgr2=0x7f17941ec010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1789400640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940c22a0 0x7f17941ec010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:26.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1789400640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940c22a0 0x7f17941ec010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1792800640 1 -- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940c22a0 msgr2=0x7f17941ec010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:07:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1792800640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940c22a0 0x7f17941ec010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1792800640 1 -- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940ac730 msgr2=0x7f17940c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1792800640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940ac730 0x7f17940c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.580+0000 7f1792800640 1 -- 172.21.15.67:0/380194481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17940c71a0 con 0x7f1794157650 2024-03-09T09:07:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.581+0000 7f1792800640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1794157650 0x7f17940be810 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f178c00c450 tx=0x7f178c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.581+0000 7f178aa00640 1 -- 172.21.15.67:0/380194481 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c0074e0 con 0x7f1794157650 2024-03-09T09:07:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.581+0000 7f178aa00640 1 -- 172.21.15.67:0/380194481 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c0047e0 con 0x7f1794157650 2024-03-09T09:07:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.581+0000 7f1799200640 1 -- 172.21.15.67:0/380194481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17941ec550 con 0x7f1794157650 2024-03-09T09:07:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.581+0000 7f178aa00640 1 -- 172.21.15.67:0/380194481 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c004bd0 con 0x7f1794157650 2024-03-09T09:07:26.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.581+0000 7f1799200640 1 -- 172.21.15.67:0/380194481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17941eec70 con 0x7f1794157650 2024-03-09T09:07:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.582+0000 7f1799200640 1 -- 172.21.15.67:0/380194481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1794075a00 con 0x7f1794157650 2024-03-09T09:07:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.583+0000 7f178aa00640 1 -- 172.21.15.67:0/380194481 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f178c011070 con 0x7f1794157650 2024-03-09T09:07:26.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.584+0000 7f178aa00640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f177006b4d0 0x7f177006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.584+0000 7f178aa00640 1 -- 172.21.15.67:0/380194481 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f178c08a030 con 0x7f1794157650 2024-03-09T09:07:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.585+0000 7f1791e00640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f177006b4d0 0x7f177006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:26.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.586+0000 7f1791e00640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f177006b4d0 0x7f177006d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f17840096d0 tx=0x7f17840091b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:26.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.589+0000 7f178aa00640 1 -- 172.21.15.67:0/380194481 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f17941ec550 con 0x7f1794157650 2024-03-09T09:07:26.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.880+0000 7f1799200640 1 -- 172.21.15.67:0/380194481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1794068550 con 0x7f1794157650 2024-03-09T09:07:26.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.881+0000 7f178aa00640 1 -- 172.21.15.67:0/380194481 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f178c058b50 con 0x7f1794157650 2024-03-09T09:07:26.885 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":108,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 -- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f177006b4d0 msgr2=0x7f177006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f177006b4d0 0x7f177006d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f17840096d0 tx=0x7f17840091b0 comp rx=0 tx=0).stop 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 -- 172.21.15.67:0/380194481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1794157650 msgr2=0x7f17940be810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1794157650 0x7f17940be810 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f178c00c450 tx=0x7f178c00c910 comp rx=0 tx=0).stop 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 -- 172.21.15.67:0/380194481 shutdown_connections 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f177006b4d0 0x7f177006d990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1794157650 0x7f17940be810 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940c22a0 0x7f17941ec010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 --2- 172.21.15.67:0/380194481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940ac730 0x7f17940c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 -- 172.21.15.67:0/380194481 >> 172.21.15.67:0/380194481 conn(0x7f17940c7370 msgr2=0x7f1794156cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 -- 172.21.15.67:0/380194481 shutdown_connections 2024-03-09T09:07:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:26.884+0000 7f1799200640 1 -- 172.21.15.67:0/380194481 wait complete. 2024-03-09T09:07:28.858 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 49% (81/163) Updating files: 50% (82/163) Updating files: 51% (84/163) Updating files: 52% (85/163) Updating files: 53% (87/163) Updating files: 54% (89/163) Updating files: 55% (90/163) Updating files: 56% (92/163) Updating files: 57% (93/163) Updating files: 58% (95/163) Updating files: 59% (97/163) Updating files: 60% (98/163) Updating files: 61% (100/163) Updating files: 62% (102/163) Updating files: 63% (103/163) Updating files: 64% (105/163) Updating files: 65% (106/163) Updating files: 66% (108/163) Updating files: 67% (110/163) Updating files: 68% (111/163) Updating files: 69% (113/163) Updating files: 70% (115/163) Updating files: 71% (116/163) Updating files: 72% (118/163) Updating files: 73% (119/163) Updating files: 74% (121/163) Updating files: 75% (123/163) Updating files: 76% (124/163) Updating files: 77% (126/163) Updating files: 78% (128/163) Updating files: 79% (129/163) Updating files: 80% (131/163) Updating files: 81% (133/163) Updating files: 82% (134/163) Updating files: 83% (136/163) Updating files: 84% (137/163) Updating files: 85% (139/163) Updating files: 85% (140/163) Updating files: 86% (141/163) Updating files: 87% (142/163) Updating files: 88% (144/163) Updating files: 89% (146/163) Updating files: 90% (147/163) Updating files: 91% (149/163) Updating files: 92% (150/163) Updating files: 93% (152/163) Updating files: 94% (154/163) Updating files: 95% (155/163) Updating files: 96% (157/163) Updating files: 97% (159/163) Updating files: 98% (160/163) Updating files: 99% (162/163) Updating files: 100% (163/163) Updating files: 100% (163/163), done. 2024-03-09T09:07:28.912 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 5d1d8d0c460 v0.23 2024-03-09T09:07:28.925 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.23 2024-03-09T09:07:29.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:07:29.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 24 -eq 48 ']' 2024-03-09T09:07:29.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.24 2024-03-09T09:07:29.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.24 2024-03-09T09:07:29.011 INFO:tasks.workunit.client.0.smithi067.stdout:v0.24 2024-03-09T09:07:29.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.24 2024-03-09T09:07:29.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:07:30.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.130+0000 7f0a70a00640 1 Processor -- start 2024-03-09T09:07:30.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.130+0000 7f0a70a00640 1 -- start start 2024-03-09T09:07:30.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.131+0000 7f0a70a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c153750 0x7f0a6c0c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:30.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.131+0000 7f0a70a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0a6c06dc00 0x7f0a6c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:30.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.131+0000 7f0a70a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c0c4010 con 0x7f0a6c153370 2024-03-09T09:07:30.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.131+0000 7f0a70a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6c156dc0 con 0x7f0a6c06dc00 2024-03-09T09:07:30.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.131+0000 7f0a70a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c0c42f0 con 0x7f0a6c153750 2024-03-09T09:07:30.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.132+0000 7f0a69400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c153750 0x7f0a6c0c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:30.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.132+0000 7f0a69400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c153750 0x7f0a6c0c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:53652/0 (socket says 172.21.15.67:53652) 2024-03-09T09:07:30.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.132+0000 7f0a69400640 1 -- 172.21.15.67:0/422530446 learned_addr learned my addr 172.21.15.67:0/422530446 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:30.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.133+0000 7f0a69400640 1 -- 172.21.15.67:0/422530446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c153750 msgr2=0x7f0a6c0c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:07:30.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.133+0000 7f0a69400640 1 -- 172.21.15.67:0/422530446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c153750 msgr2=0x7f0a6c0c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:07:30.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.133+0000 7f0a69400640 1 --2- 172.21.15.67:0/422530446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c153750 0x7f0a6c0c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:30.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.133+0000 7f0a69400640 1 --2- 172.21.15.67:0/422530446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c153750 0x7f0a6c0c30b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:30.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.133+0000 7f0a6a800640 1 --2- 172.21.15.67:0/422530446 >> v2:172.21.15.134:3300/0 conn(0x7f0a6c06dc00 0x7f0a6c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:30.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.133+0000 7f0a68a00640 1 -- 172.21.15.67:0/422530446 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984337698 0 0) 0x7f0a6c0c4010 con 0x7f0a6c153370 2024-03-09T09:07:30.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.133+0000 7f0a68a00640 1 -- 172.21.15.67:0/422530446 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a50003520 con 0x7f0a6c153370 2024-03-09T09:07:30.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.134+0000 7f0a6a800640 1 -- 172.21.15.67:0/422530446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c153750 msgr2=0x7f0a6c0c30b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:07:30.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.134+0000 7f0a6a800640 1 --2- 172.21.15.67:0/422530446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c153750 0x7f0a6c0c30b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:30.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.134+0000 7f0a6a800640 1 -- 172.21.15.67:0/422530446 >> v1:172.21.15.134:6789/0 conn(0x7f0a6c153370 legacy=0x7f0a6c0c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:30.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.134+0000 7f0a6a800640 1 -- 172.21.15.67:0/422530446 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0c5a50 con 0x7f0a6c06dc00 2024-03-09T09:07:30.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.135+0000 7f0a6a800640 1 --2- 172.21.15.67:0/422530446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a6c06dc00 0x7f0a6c0729c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009d90 tx=0x7f0a6402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f65bd6a2358af55 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:30.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.135+0000 7f0a68a00640 1 -- 172.21.15.67:0/422530446 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a64039070 con 0x7f0a6c06dc00 2024-03-09T09:07:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.135+0000 7f0a68a00640 1 -- 172.21.15.67:0/422530446 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6402cbc0 con 0x7f0a6c06dc00 2024-03-09T09:07:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.135+0000 7f0a68a00640 1 -- 172.21.15.67:0/422530446 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a64034440 con 0x7f0a6c06dc00 2024-03-09T09:07:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.136+0000 7f0a70a00640 1 -- 172.21.15.67:0/422530446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a6c06dc00 msgr2=0x7f0a6c0729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.136+0000 7f0a70a00640 1 --2- 172.21.15.67:0/422530446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a6c06dc00 0x7f0a6c0729c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009d90 tx=0x7f0a6402be10 comp rx=0 tx=0).stop 2024-03-09T09:07:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.136+0000 7f0a70a00640 1 -- 172.21.15.67:0/422530446 shutdown_connections 2024-03-09T09:07:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.136+0000 7f0a70a00640 1 --2- 172.21.15.67:0/422530446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a6c06dc00 0x7f0a6c0729c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.136+0000 7f0a70a00640 1 --2- 172.21.15.67:0/422530446 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c153750 0x7f0a6c0c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.136+0000 7f0a70a00640 1 -- 172.21.15.67:0/422530446 >> 172.21.15.67:0/422530446 conn(0x7f0a6c0ab320 msgr2=0x7f0a6c0ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.137+0000 7f0a70a00640 1 -- 172.21.15.67:0/422530446 shutdown_connections 2024-03-09T09:07:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.137+0000 7f0a70a00640 1 -- 172.21.15.67:0/422530446 wait complete. 2024-03-09T09:07:30.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.139+0000 7f0a70a00640 1 Processor -- start 2024-03-09T09:07:30.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.139+0000 7f0a70a00640 1 -- start start 2024-03-09T09:07:30.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.140+0000 7f0a70a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c06dc00 0x7f0a6c154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:30.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.140+0000 7f0a70a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a6c153370 0x7f0a6c14bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:30.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.140+0000 7f0a70a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a6c153750 0x7f0a6c153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.140+0000 7f0a70a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c0c5d30 con 0x7f0a6c153370 2024-03-09T09:07:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.140+0000 7f0a70a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c073dc0 con 0x7f0a6c153750 2024-03-09T09:07:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.140+0000 7f0a70a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c072f00 con 0x7f0a6c06dc00 2024-03-09T09:07:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a6c153370 0x7f0a6c14bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a6c153370 0x7f0a6c14bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59818/0 (socket says 172.21.15.67:59818) 2024-03-09T09:07:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69400640 1 -- 172.21.15.67:0/4218027047 learned_addr learned my addr 172.21.15.67:0/4218027047 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69e00640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c06dc00 0x7f0a6c154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a6a800640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a6c153750 0x7f0a6c153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:30.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69e00640 1 -- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c06dc00 msgr2=0x7f0a6c154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:07:30.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69e00640 1 -- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c06dc00 msgr2=0x7f0a6c154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69e00640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c06dc00 0x7f0a6c154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69400640 1 -- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c06dc00 msgr2=0x7f0a6c154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69e00640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c06dc00 0x7f0a6c154690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69400640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c06dc00 0x7f0a6c154690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69400640 1 -- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a6c153750 msgr2=0x7f0a6c153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69400640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a6c153750 0x7f0a6c153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.141+0000 7f0a69400640 1 -- 172.21.15.67:0/4218027047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c14c5c0 con 0x7f0a6c153370 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.142+0000 7f0a69400640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a6c153370 0x7f0a6c14bd70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800e840 tx=0x7f0a5800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.142+0000 7f0a62000640 1 -- 172.21.15.67:0/4218027047 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a58017070 con 0x7f0a6c153370 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.142+0000 7f0a62000640 1 -- 172.21.15.67:0/4218027047 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a58004020 con 0x7f0a6c153370 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.142+0000 7f0a62000640 1 -- 172.21.15.67:0/4218027047 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5800f040 con 0x7f0a6c153370 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.142+0000 7f0a70a00640 1 -- 172.21.15.67:0/4218027047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c14c860 con 0x7f0a6c153370 2024-03-09T09:07:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.142+0000 7f0a70a00640 1 -- 172.21.15.67:0/4218027047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c155c40 con 0x7f0a6c153370 2024-03-09T09:07:30.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.143+0000 7f0a70a00640 1 -- 172.21.15.67:0/4218027047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c075a00 con 0x7f0a6c153370 2024-03-09T09:07:30.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.144+0000 7f0a62000640 1 -- 172.21.15.67:0/4218027047 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0a5800c6d0 con 0x7f0a6c153370 2024-03-09T09:07:30.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.145+0000 7f0a62000640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:30.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.146+0000 7f0a62000640 1 -- 172.21.15.67:0/4218027047 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0a5808a100 con 0x7f0a6c153370 2024-03-09T09:07:30.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.146+0000 7f0a69e00640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:30.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.147+0000 7f0a69e00640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c007ae0 tx=0x7f0a5c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:30.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.152+0000 7f0a62000640 1 -- 172.21.15.67:0/4218027047 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0a580584a0 con 0x7f0a6c153370 2024-03-09T09:07:30.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.438+0000 7f0a70a00640 1 -- 172.21.15.67:0/4218027047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0a6c0c4010 con 0x7f0a6c153370 2024-03-09T09:07:30.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.439+0000 7f0a62000640 1 -- 172.21.15.67:0/4218027047 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f0a5805c540 con 0x7f0a6c153370 2024-03-09T09:07:30.443 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":108,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:07:30.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.441+0000 7f0a70a00640 1 -- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a4c06b4d0 msgr2=0x7f0a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.441+0000 7f0a70a00640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c007ae0 tx=0x7f0a5c009040 comp rx=0 tx=0).stop 2024-03-09T09:07:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.441+0000 7f0a70a00640 1 -- 172.21.15.67:0/4218027047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a6c153370 msgr2=0x7f0a6c14bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.441+0000 7f0a70a00640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a6c153370 0x7f0a6c14bd70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800e840 tx=0x7f0a5800ed00 comp rx=0 tx=0).stop 2024-03-09T09:07:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.441+0000 7f0a70a00640 1 -- 172.21.15.67:0/4218027047 shutdown_connections 2024-03-09T09:07:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.441+0000 7f0a70a00640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.441+0000 7f0a70a00640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a6c153750 0x7f0a6c153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.441+0000 7f0a70a00640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a6c153370 0x7f0a6c14bd70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.441+0000 7f0a70a00640 1 --2- 172.21.15.67:0/4218027047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a6c06dc00 0x7f0a6c154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.441+0000 7f0a70a00640 1 -- 172.21.15.67:0/4218027047 >> 172.21.15.67:0/4218027047 conn(0x7f0a6c0ab320 msgr2=0x7f0a6c06c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.442+0000 7f0a70a00640 1 -- 172.21.15.67:0/4218027047 shutdown_connections 2024-03-09T09:07:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:30.442+0000 7f0a70a00640 1 -- 172.21.15.67:0/4218027047 wait complete. 2024-03-09T09:07:33.471 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:07:33.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.664+0000 7fc7fcc00640 1 Processor -- start 2024-03-09T09:07:33.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.665+0000 7fc7fcc00640 1 -- start start 2024-03-09T09:07:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.666+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7f8152a20 0x7fc7f80aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.666+0000 7fc7fcc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc7f806dbe0 0x7fc7f806e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.666+0000 7fc7fcc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f80ab3c0 con 0x7fc7f80c30f0 2024-03-09T09:07:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.666+0000 7fc7fcc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f80c82d0 con 0x7fc7f806dbe0 2024-03-09T09:07:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.666+0000 7fc7fcc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80aab20 con 0x7fc7f8152a20 2024-03-09T09:07:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.667+0000 7fc7f6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc7f806dbe0 0x7fc7f806e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:33.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.667+0000 7fc7f5e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc7f80c30f0 0x7fc7f81490f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:46024/0 (socket says 172.21.15.67:46024) 2024-03-09T09:07:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.667+0000 7fc7f5e00640 1 -- 172.21.15.67:0/650741283 learned_addr learned my addr 172.21.15.67:0/650741283 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:07:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.667+0000 7fc7f5400640 1 --2- 172.21.15.67:0/650741283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7f8152a20 0x7fc7f80aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.667+0000 7fc7f6800640 1 -- 172.21.15.67:0/650741283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7f8152a20 msgr2=0x7fc7f80aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:33.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.667+0000 7fc7f6800640 1 --2- 172.21.15.67:0/650741283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7f8152a20 0x7fc7f80aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:33.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.667+0000 7fc7f6800640 1 -- 172.21.15.67:0/650741283 >> v1:172.21.15.134:6789/0 conn(0x7fc7f80c30f0 legacy=0x7fc7f81490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:33.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.667+0000 7fc7f6800640 1 -- 172.21.15.67:0/650741283 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f80aae00 con 0x7fc7f806dbe0 2024-03-09T09:07:33.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.668+0000 7fc7f6800640 1 --2- 172.21.15.67:0/650741283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f806dbe0 0x7fc7f806e060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009d90 tx=0x7fc7f002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e0722b97510208 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:33.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.669+0000 7fc7f4a00640 1 -- 172.21.15.67:0/650741283 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0039070 con 0x7fc7f806dbe0 2024-03-09T09:07:33.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.669+0000 7fc7f4a00640 1 -- 172.21.15.67:0/650741283 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f002cbd0 con 0x7fc7f806dbe0 2024-03-09T09:07:33.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.669+0000 7fc7f4a00640 1 -- 172.21.15.67:0/650741283 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0034440 con 0x7fc7f806dbe0 2024-03-09T09:07:33.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.670+0000 7fc7fcc00640 1 -- 172.21.15.67:0/650741283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f806dbe0 msgr2=0x7fc7f806e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:33.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.670+0000 7fc7fcc00640 1 --2- 172.21.15.67:0/650741283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f806dbe0 0x7fc7f806e060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009d90 tx=0x7fc7f002be20 comp rx=0 tx=0).stop 2024-03-09T09:07:33.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.670+0000 7fc7fcc00640 1 -- 172.21.15.67:0/650741283 shutdown_connections 2024-03-09T09:07:33.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.670+0000 7fc7fcc00640 1 --2- 172.21.15.67:0/650741283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f806dbe0 0x7fc7f806e060 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:33.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.670+0000 7fc7fcc00640 1 --2- 172.21.15.67:0/650741283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7f8152a20 0x7fc7f80aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:33.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.670+0000 7fc7fcc00640 1 -- 172.21.15.67:0/650741283 >> 172.21.15.67:0/650741283 conn(0x7fc7f80c7370 msgr2=0x7fc7f80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:33.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.670+0000 7fc7fcc00640 1 -- 172.21.15.67:0/650741283 shutdown_connections 2024-03-09T09:07:33.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.670+0000 7fc7fcc00640 1 -- 172.21.15.67:0/650741283 wait complete. 2024-03-09T09:07:33.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.672+0000 7fc7fcc00640 1 Processor -- start 2024-03-09T09:07:33.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.672+0000 7fc7fcc00640 1 -- start start 2024-03-09T09:07:33.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7f806dbe0 0x7fc7f81dd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:33.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f80c30f0 0x7fc7f81486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:33.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8152a20 0x7fc7f8148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:33.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7fcc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8149800 con 0x7fc7f8152a20 2024-03-09T09:07:33.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7fcc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f806d6c0 con 0x7fc7f80c30f0 2024-03-09T09:07:33.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7fcc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8149980 con 0x7fc7f806dbe0 2024-03-09T09:07:33.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8152a20 0x7fc7f8148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8152a20 0x7fc7f8148c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59836/0 (socket says 172.21.15.67:59836) 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f6800640 1 -- 172.21.15.67:0/2113027518 learned_addr learned my addr 172.21.15.67:0/2113027518 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f5400640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f80c30f0 0x7fc7f81486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f5e00640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7f806dbe0 0x7fc7f81dd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f6800640 1 -- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7f806dbe0 msgr2=0x7fc7f81dd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f6800640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7f806dbe0 0x7fc7f81dd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f6800640 1 -- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f80c30f0 msgr2=0x7fc7f81486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f6800640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f80c30f0 0x7fc7f81486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f6800640 1 -- 172.21.15.67:0/2113027518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f81eb4b0 con 0x7fc7f8152a20 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f5e00640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7f806dbe0 0x7fc7f81dd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.673+0000 7fc7f5400640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f80c30f0 0x7fc7f81486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:07:33.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.674+0000 7fc7f6800640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8152a20 0x7fc7f8148c30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009d90 tx=0x7fc7f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:33.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.674+0000 7fc7ee000640 1 -- 172.21.15.67:0/2113027518 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0039050 con 0x7fc7f8152a20 2024-03-09T09:07:33.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.674+0000 7fc7ee000640 1 -- 172.21.15.67:0/2113027518 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f0045070 con 0x7fc7f8152a20 2024-03-09T09:07:33.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.674+0000 7fc7ee000640 1 -- 172.21.15.67:0/2113027518 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f003e830 con 0x7fc7f8152a20 2024-03-09T09:07:33.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.674+0000 7fc7fcc00640 1 -- 172.21.15.67:0/2113027518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f8150120 con 0x7fc7f8152a20 2024-03-09T09:07:33.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.674+0000 7fc7fcc00640 1 -- 172.21.15.67:0/2113027518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f814d490 con 0x7fc7f8152a20 2024-03-09T09:07:33.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.676+0000 7fc7fcc00640 1 -- 172.21.15.67:0/2113027518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f8075a00 con 0x7fc7f8152a20 2024-03-09T09:07:33.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.676+0000 7fc7ee000640 1 -- 172.21.15.67:0/2113027518 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc7f003e9d0 con 0x7fc7f8152a20 2024-03-09T09:07:33.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.677+0000 7fc7ee000640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc7d806b4d0 0x7fc7d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:33.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.677+0000 7fc7ee000640 1 -- 172.21.15.67:0/2113027518 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc7f00ad4e0 con 0x7fc7f8152a20 2024-03-09T09:07:33.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.678+0000 7fc7f5e00640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc7d806b4d0 0x7fc7d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:33.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.679+0000 7fc7f5e00640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc7d806b4d0 0x7fc7d806d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8005880 tx=0x7fc7e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:33.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:33.688+0000 7fc7ee000640 1 -- 172.21.15.67:0/2113027518 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc7f007c080 con 0x7fc7f8152a20 2024-03-09T09:07:34.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.042+0000 7fc7fcc00640 1 -- 172.21.15.67:0/2113027518 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc7f80746e0 con 0x7fc7f8152a20 2024-03-09T09:07:34.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.043+0000 7fc7ee000640 1 -- 172.21.15.67:0/2113027518 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fc7f007b980 con 0x7fc7f8152a20 2024-03-09T09:07:34.046 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:07:34.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 -- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc7d806b4d0 msgr2=0x7fc7d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc7d806b4d0 0x7fc7d806d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8005880 tx=0x7fc7e8009040 comp rx=0 tx=0).stop 2024-03-09T09:07:34.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 -- 172.21.15.67:0/2113027518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8152a20 msgr2=0x7fc7f8148c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8152a20 0x7fc7f8148c30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009d90 tx=0x7fc7f0004a40 comp rx=0 tx=0).stop 2024-03-09T09:07:34.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 -- 172.21.15.67:0/2113027518 shutdown_connections 2024-03-09T09:07:34.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc7d806b4d0 0x7fc7d806d990 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7f8152a20 0x7fc7f8148c30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7f80c30f0 0x7fc7f81486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 --2- 172.21.15.67:0/2113027518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc7f806dbe0 0x7fc7f81dd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 -- 172.21.15.67:0/2113027518 >> 172.21.15.67:0/2113027518 conn(0x7fc7f80c7370 msgr2=0x7fc7f8151920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:34.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 -- 172.21.15.67:0/2113027518 shutdown_connections 2024-03-09T09:07:34.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.045+0000 7fc7fcc00640 1 -- 172.21.15.67:0/2113027518 wait complete. 2024-03-09T09:07:34.064 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:07:34.064 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:07:34.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:07:34.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.281+0000 7f0a9e600640 1 Processor -- start 2024-03-09T09:07:34.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.282+0000 7f0a9e600640 1 -- start start 2024-03-09T09:07:34.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.283+0000 7f0a9e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0a98151360 0x7f0a980c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.283+0000 7f0a9e600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980c30a0 0x7f0a981539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.283+0000 7f0a9e600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a980c2780 con 0x7f0a980ac930 2024-03-09T09:07:34.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.283+0000 7f0a9e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a98070670 con 0x7f0a98151360 2024-03-09T09:07:34.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.283+0000 7f0a9e600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980708c0 con 0x7f0a980c30a0 2024-03-09T09:07:34.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.283+0000 7f0a97e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0a98151360 0x7f0a980c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.284+0000 7f0a97e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0a98151360 0x7f0a980c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59848/0 (socket says 172.21.15.67:59848) 2024-03-09T09:07:34.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.284+0000 7f0a97e00640 1 -- 172.21.15.67:0/2850663769 learned_addr learned my addr 172.21.15.67:0/2850663769 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:34.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.284+0000 7f0a9dc00640 1 --2- 172.21.15.67:0/2850663769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980c30a0 0x7f0a981539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.284+0000 7f0a97e00640 1 -- 172.21.15.67:0/2850663769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980c30a0 msgr2=0x7f0a981539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:34.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.284+0000 7f0a97e00640 1 --2- 172.21.15.67:0/2850663769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980c30a0 0x7f0a981539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.284+0000 7f0a97e00640 1 -- 172.21.15.67:0/2850663769 >> v1:172.21.15.134:6789/0 conn(0x7f0a980ac930 legacy=0x7f0a98150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.284+0000 7f0a97e00640 1 -- 172.21.15.67:0/2850663769 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a98070ba0 con 0x7f0a98151360 2024-03-09T09:07:34.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.286+0000 7f0a97e00640 1 --2- 172.21.15.67:0/2850663769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a98151360 0x7f0a980c3660 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009870 tx=0x7f0a8402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab708c8026aa4b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.286+0000 7f0a97400640 1 -- 172.21.15.67:0/2850663769 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84034070 con 0x7f0a98151360 2024-03-09T09:07:34.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.286+0000 7f0a97400640 1 -- 172.21.15.67:0/2850663769 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a84030840 con 0x7f0a98151360 2024-03-09T09:07:34.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.286+0000 7f0a97400640 1 -- 172.21.15.67:0/2850663769 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84030c10 con 0x7f0a98151360 2024-03-09T09:07:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.287+0000 7f0a9e600640 1 -- 172.21.15.67:0/2850663769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a98151360 msgr2=0x7f0a980c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.287+0000 7f0a9e600640 1 --2- 172.21.15.67:0/2850663769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a98151360 0x7f0a980c3660 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009870 tx=0x7f0a8402b980 comp rx=0 tx=0).stop 2024-03-09T09:07:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.287+0000 7f0a9e600640 1 -- 172.21.15.67:0/2850663769 shutdown_connections 2024-03-09T09:07:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.287+0000 7f0a9e600640 1 --2- 172.21.15.67:0/2850663769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980c30a0 0x7f0a981539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.287+0000 7f0a9e600640 1 --2- 172.21.15.67:0/2850663769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a98151360 0x7f0a980c3660 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.287+0000 7f0a9e600640 1 -- 172.21.15.67:0/2850663769 >> 172.21.15.67:0/2850663769 conn(0x7f0a980c7220 msgr2=0x7f0a980c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.287+0000 7f0a9e600640 1 -- 172.21.15.67:0/2850663769 shutdown_connections 2024-03-09T09:07:34.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.287+0000 7f0a9e600640 1 -- 172.21.15.67:0/2850663769 wait complete. 2024-03-09T09:07:34.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.289+0000 7f0a9e600640 1 Processor -- start 2024-03-09T09:07:34.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.289+0000 7f0a9e600640 1 -- start start 2024-03-09T09:07:34.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.290+0000 7f0a9e600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980ac930 0x7f0a981443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.290+0000 7f0a9e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c30a0 0x7f0a9814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.290+0000 7f0a9e600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a98151360 0x7f0a980c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.290+0000 7f0a9e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980ace90 con 0x7f0a980c30a0 2024-03-09T09:07:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.290+0000 7f0a9e600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980acd10 con 0x7f0a98151360 2024-03-09T09:07:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.290+0000 7f0a9e600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98153f20 con 0x7f0a980ac930 2024-03-09T09:07:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.290+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c30a0 0x7f0a9814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.290+0000 7f0a97e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c30a0 0x7f0a9814b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59850/0 (socket says 172.21.15.67:59850) 2024-03-09T09:07:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.290+0000 7f0a97e00640 1 -- 172.21.15.67:0/2959526268 learned_addr learned my addr 172.21.15.67:0/2959526268 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.291+0000 7f0a9d200640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980ac930 0x7f0a981443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.291+0000 7f0a9dc00640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a98151360 0x7f0a980c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.291+0000 7f0a97e00640 1 -- 172.21.15.67:0/2959526268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980ac930 msgr2=0x7f0a981443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.291+0000 7f0a97e00640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980ac930 0x7f0a981443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.291+0000 7f0a97e00640 1 -- 172.21.15.67:0/2959526268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a98151360 msgr2=0x7f0a980c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.291+0000 7f0a97e00640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a98151360 0x7f0a980c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.291+0000 7f0a97e00640 1 -- 172.21.15.67:0/2959526268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9814b730 con 0x7f0a980c30a0 2024-03-09T09:07:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.291+0000 7f0a97e00640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c30a0 0x7f0a9814b1f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0a840090b0 tx=0x7f0a840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.291+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84034040 con 0x7f0a980c30a0 2024-03-09T09:07:34.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.291+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8403ada0 con 0x7f0a980c30a0 2024-03-09T09:07:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.291+0000 7f0a9e600640 1 -- 172.21.15.67:0/2959526268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a98153cb0 con 0x7f0a980c30a0 2024-03-09T09:07:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.292+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84044610 con 0x7f0a980c30a0 2024-03-09T09:07:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.292+0000 7f0a9e600640 1 -- 172.21.15.67:0/2959526268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a980c5ef0 con 0x7f0a980c30a0 2024-03-09T09:07:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.293+0000 7f0a9e600640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c6260 0x7f0a981eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.293+0000 7f0a9e600640 1 -- 172.21.15.67:0/2959526268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980c5d70 con 0x7f0a980c6260 2024-03-09T09:07:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.293+0000 7f0a9d200640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c6260 0x7f0a981eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.293+0000 7f0a9d200640 1 -- 172.21.15.67:0/2959526268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a98144930 con 0x7f0a980c6260 2024-03-09T09:07:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.293+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0a84037070 con 0x7f0a980c30a0 2024-03-09T09:07:34.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.294+0000 7f0a9d200640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c6260 0x7f0a981eec60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e4d0 tx=0x7f0a8c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.294+0000 7f0a94c00640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a7806b4d0 0x7f0a7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.295+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0a840ac740 con 0x7f0a980c30a0 2024-03-09T09:07:34.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.295+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c018070 con 0x7f0a980c6260 2024-03-09T09:07:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.295+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f0a8c0052e0 con 0x7f0a980c6260 2024-03-09T09:07:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.295+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c6260 msgr2=0x7f0a981eec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.295+0000 7f0a94c00640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c6260 0x7f0a981eec60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e4d0 tx=0x7f0a8c00e990 comp rx=0 tx=0).stop 2024-03-09T09:07:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.295+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c6260 msgr2=0x7f0a981eec60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.295+0000 7f0a94c00640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c6260 0x7f0a981eec60 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e4d0 tx=0x7f0a8c00e990 comp rx=0 tx=0).stop 2024-03-09T09:07:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.295+0000 7f0a9dc00640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a7806b4d0 0x7f0a7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.296+0000 7f0a9dc00640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a7806b4d0 0x7f0a7806d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009f80 tx=0x7f0a9000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.302+0000 7f0a9e600640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a98144b10 0x7f0a981ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.302+0000 7f0a9e600640 1 -- 172.21.15.67:0/2959526268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9804f3c0 con 0x7f0a98144b10 2024-03-09T09:07:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.302+0000 7f0a9d200640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a98144b10 0x7f0a981ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.302+0000 7f0a9d200640 1 -- 172.21.15.67:0/2959526268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0a9814af80 con 0x7f0a98144b10 2024-03-09T09:07:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.303+0000 7f0a9d200640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a98144b10 0x7f0a981ef1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e4d0 tx=0x7f0a8c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.303+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c018040 con 0x7f0a98144b10 2024-03-09T09:07:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.303+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f0a8c007630 con 0x7f0a98144b10 2024-03-09T09:07:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.303+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a98144b10 msgr2=0x7f0a981ef1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.303+0000 7f0a94c00640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a98144b10 0x7f0a981ef1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e4d0 tx=0x7f0a8c001120 comp rx=0 tx=0).stop 2024-03-09T09:07:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.303+0000 7f0a94c00640 1 -- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a98144b10 msgr2=0x7f0a981ef1a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.303+0000 7f0a94c00640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a98144b10 0x7f0a981ef1a0 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e4d0 tx=0x7f0a8c001120 comp rx=0 tx=0).stop 2024-03-09T09:07:34.308 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:07:34.308 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:07:34.308 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:07:34.308 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:07:34.308 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 110, 2024-03-09T09:07:34.308 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:07:34.308 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:07:34.308 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:34.309 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:34.310 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:34.310 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:34.310 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:34.310 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:34.310 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:34.310 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:34.310 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:07:34.310 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:07:34.310 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:34.310 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:34.311 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:34.311 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:34.311 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:34.311 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:34.311 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:34.311 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:34.311 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:34.311 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:34.312 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:34.312 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.312 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:07:34.312 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:07:34.312 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:07:34.312 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:07:34.312 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:07:34.312 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:07:34.313 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:34.313 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:34.313 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:07:34.313 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:07:34.313 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:07:34.313 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:07:34.313 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:07:34.313 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:07:34.313 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:07:34.313 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:07:34.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:34.315 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:07:34.316 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.317 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:07:34.318 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:34.319 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.320 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.321 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:34.322 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:07:34.323 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.323 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.323 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:07:34.323 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:07:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.304+0000 7f0a9e600640 1 -- 172.21.15.67:0/2959526268 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a7806b4d0 msgr2=0x7f0a7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.304+0000 7f0a9e600640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a7806b4d0 0x7f0a7806d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009f80 tx=0x7f0a9000b040 comp rx=0 tx=0).stop 2024-03-09T09:07:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.304+0000 7f0a9e600640 1 -- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c30a0 msgr2=0x7f0a9814b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.304+0000 7f0a9e600640 1 --2- 172.21.15.67:0/2959526268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a980c30a0 0x7f0a9814b1f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0a840090b0 tx=0x7f0a840046c0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.304+0000 7f0a9dc00640 1 -- 172.21.15.67:0/2959526268 reap_dead start 2024-03-09T09:07:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.304+0000 7f0a9dc00640 1 -- 172.21.15.67:0/2959526268 reap_dead start 2024-03-09T09:07:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.304+0000 7f0a9e600640 1 -- 172.21.15.67:0/2959526268 shutdown_connections 2024-03-09T09:07:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.304+0000 7f0a9e600640 1 -- 172.21.15.67:0/2959526268 >> 172.21.15.67:0/2959526268 conn(0x7f0a980c7220 msgr2=0x7f0a980c0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.304+0000 7f0a9e600640 1 -- 172.21.15.67:0/2959526268 shutdown_connections 2024-03-09T09:07:34.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.304+0000 7f0a9e600640 1 -- 172.21.15.67:0/2959526268 wait complete. 2024-03-09T09:07:34.325 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:07:34.325 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:07:34.325 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:07:34.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.528+0000 7f1ecca00640 1 Processor -- start 2024-03-09T09:07:34.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.528+0000 7f1ecca00640 1 -- start start 2024-03-09T09:07:34.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.530+0000 7f1ecca00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1ec80c5e00 0x7f1ec80acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.530+0000 7f1ecca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 0x7f1ec80c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.530+0000 7f1ecca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec80ad7c0 con 0x7f1ec8156660 2024-03-09T09:07:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.530+0000 7f1ecca00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec806d740 con 0x7f1ec80c5e00 2024-03-09T09:07:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.530+0000 7f1ecca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c6660 con 0x7f1ec80c61e0 2024-03-09T09:07:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.530+0000 7f1ec7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1ec8156660 0x7f1ec806c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52828/0 (socket says 172.21.15.67:52828) 2024-03-09T09:07:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.530+0000 7f1ec7e00640 1 -- 172.21.15.67:0/3711704429 learned_addr learned my addr 172.21.15.67:0/3711704429 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:07:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.531+0000 7f1ec6a00640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 0x7f1ec80c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.531+0000 7f1ec7400640 1 --2- 172.21.15.67:0/3711704429 >> v2:172.21.15.134:3300/0 conn(0x7f1ec80c5e00 0x7f1ec80acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.531+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955209155 0 0) 0x7f1ec80ad7c0 con 0x7f1ec8156660 2024-03-09T09:07:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.531+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eac003520 con 0x7f1ec8156660 2024-03-09T09:07:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.531+0000 7f1ec6a00640 1 -- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 msgr2=0x7f1ec80c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:07:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.531+0000 7f1ec6a00640 1 -- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 msgr2=0x7f1ec80c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:07:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.531+0000 7f1ec6a00640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 0x7f1ec80c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.531+0000 7f1ec6a00640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 0x7f1ec80c9380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3212899339 0 0) 0x7f1eac003520 con 0x7f1ec8156660 2024-03-09T09:07:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 msgr2=0x7f1ec80c9380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:07:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 0x7f1ec80c9380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 >> v2:172.21.15.134:3300/0 conn(0x7f1ec80c5e00 msgr2=0x7f1ec80acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 --2- 172.21.15.67:0/3711704429 >> v2:172.21.15.134:3300/0 conn(0x7f1ec80c5e00 0x7f1ec80acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec8156af0 con 0x7f1ec8156660 2024-03-09T09:07:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f1ec00019f0 con 0x7f1ec8156660 2024-03-09T09:07:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 >> v1:172.21.15.67:6789/0 conn(0x7f1ec8156660 legacy=0x7f1ec806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac003a20 0x7f1eac003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac004800 0x7f1eac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 0x7f1eac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c6660 con 0x7f1eac003a20 2024-03-09T09:07:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac00b9f0 con 0x7f1eac004800 2024-03-09T09:07:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eac00bbf0 con 0x7f1ec80c61e0 2024-03-09T09:07:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec7400640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac003a20 0x7f1eac003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.532+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 0x7f1eac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.533+0000 7f1ec7400640 1 -- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 msgr2=0x7f1eac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.533+0000 7f1ec7400640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80c61e0 0x7f1eac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.533+0000 7f1ec7400640 1 -- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac004800 msgr2=0x7f1eac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.533+0000 7f1ec7400640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1eac004800 0x7f1eac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.533+0000 7f1ec7400640 1 -- 172.21.15.67:0/3711704429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac003520 con 0x7f1eac003a20 2024-03-09T09:07:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.534+0000 7f1ec7400640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac003a20 0x7f1eac003e40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009ac0 tx=0x7f1eac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7704c8c8da22a76e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.534+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8039070 con 0x7f1eac003a20 2024-03-09T09:07:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.534+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb802c710 con 0x7f1eac003a20 2024-03-09T09:07:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.534+0000 7f1ec6000640 1 -- 172.21.15.67:0/3711704429 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8009690 con 0x7f1eac003a20 2024-03-09T09:07:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.535+0000 7f1ecca00640 1 -- 172.21.15.67:0/3711704429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac003a20 msgr2=0x7f1eac003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.535+0000 7f1ecca00640 1 --2- 172.21.15.67:0/3711704429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1eac003a20 0x7f1eac003e40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009ac0 tx=0x7f1eac005830 comp rx=0 tx=0).stop 2024-03-09T09:07:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.535+0000 7f1ec7e00640 1 -- 172.21.15.67:0/3711704429 reap_dead start 2024-03-09T09:07:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.535+0000 7f1ecca00640 1 -- 172.21.15.67:0/3711704429 shutdown_connections 2024-03-09T09:07:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.535+0000 7f1ecca00640 1 -- 172.21.15.67:0/3711704429 >> 172.21.15.67:0/3711704429 conn(0x7f1ec80cb480 msgr2=0x7f1ec80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:34.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.535+0000 7f1ecca00640 1 -- 172.21.15.67:0/3711704429 shutdown_connections 2024-03-09T09:07:34.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.535+0000 7f1ecca00640 1 -- 172.21.15.67:0/3711704429 wait complete. 2024-03-09T09:07:34.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.538+0000 7f1ecca00640 1 Processor -- start 2024-03-09T09:07:34.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.538+0000 7f1ecca00640 1 -- start start 2024-03-09T09:07:34.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.539+0000 7f1ecca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eac003a20 0x7f1ec80c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.539+0000 7f1ecca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80c0f10 0x7f1ec81ee600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.539+0000 7f1ecca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80ab210 0x7f1ec81eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.539+0000 7f1ecca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec806bb00 con 0x7f1ec80c0f10 2024-03-09T09:07:34.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.539+0000 7f1ecca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec8153680 con 0x7f1ec80ab210 2024-03-09T09:07:34.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.539+0000 7f1ecca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec806bde0 con 0x7f1eac003a20 2024-03-09T09:07:34.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.540+0000 7f1ec6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80c0f10 0x7f1ec81ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.540+0000 7f1ec6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80c0f10 0x7f1ec81ee600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59890/0 (socket says 172.21.15.67:59890) 2024-03-09T09:07:34.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.540+0000 7f1ec6a00640 1 -- 172.21.15.67:0/3651711910 learned_addr learned my addr 172.21.15.67:0/3651711910 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.540+0000 7f1ec7400640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eac003a20 0x7f1ec80c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.540+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80ab210 0x7f1ec81eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.540+0000 7f1ec6a00640 1 -- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eac003a20 msgr2=0x7f1ec80c05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.540+0000 7f1ec6a00640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1eac003a20 0x7f1ec80c05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.540+0000 7f1ec6a00640 1 -- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80ab210 msgr2=0x7f1ec81eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.540+0000 7f1ec6a00640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80ab210 0x7f1ec81eeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.540+0000 7f1ec6a00640 1 -- 172.21.15.67:0/3651711910 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec81ef080 con 0x7f1ec80c0f10 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.540+0000 7f1ec6a00640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80c0f10 0x7f1ec81ee600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d830 tx=0x7f1eb400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.541+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4018020 con 0x7f1ec80c0f10 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.541+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb400b660 con 0x7f1ec80c0f10 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.541+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400ba50 con 0x7f1ec80c0f10 2024-03-09T09:07:34.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.541+0000 7f1ecca00640 1 -- 172.21.15.67:0/3651711910 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec80becb0 con 0x7f1ec80c0f10 2024-03-09T09:07:34.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.541+0000 7f1ecca00640 1 -- 172.21.15.67:0/3651711910 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec80c19c0 con 0x7f1ec80c0f10 2024-03-09T09:07:34.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.542+0000 7f1ecca00640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2910 0x7f1ec80c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.542+0000 7f1ecca00640 1 -- 172.21.15.67:0/3651711910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec804f3c0 con 0x7f1ec80c2910 2024-03-09T09:07:34.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.543+0000 7f1ec7400640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2910 0x7f1ec80c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.544+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1eb401b020 con 0x7f1ec80c0f10 2024-03-09T09:07:34.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.544+0000 7f1ec7400640 1 -- 172.21.15.67:0/3651711910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ec806d2f0 con 0x7f1ec80c2910 2024-03-09T09:07:34.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.544+0000 7f1ec7400640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2910 0x7f1ec80c1400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009a90 tx=0x7f1eb8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.544+0000 7f1ebf400640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ea806b500 0x7f1ea806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.545+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1eb408adc0 con 0x7f1ec80c0f10 2024-03-09T09:07:34.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.545+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8039070 con 0x7f1ec80c2910 2024-03-09T09:07:34.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.545+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ea806b500 0x7f1ea806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.546+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f1eb803a040 con 0x7f1ec80c2910 2024-03-09T09:07:34.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.546+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2910 msgr2=0x7f1ec80c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.546+0000 7f1ebf400640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2910 0x7f1ec80c1400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009a90 tx=0x7f1eb8004730 comp rx=0 tx=0).stop 2024-03-09T09:07:34.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.546+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2910 msgr2=0x7f1ec80c1400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:34.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.546+0000 7f1ebf400640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2910 0x7f1ec80c1400 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.546+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ea806b500 0x7f1ea806d9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80c7dc0 tx=0x7f1ec0001360 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.550 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 65% (107/163) Updating files: 66% (108/163) Updating files: 67% (110/163) Updating files: 68% (111/163) Updating files: 69% (113/163) Updating files: 70% (115/163) Updating files: 71% (116/163) Updating files: 72% (118/163) Updating files: 73% (119/163) Updating files: 74% (121/163) Updating files: 75% (123/163) Updating files: 76% (124/163) Updating files: 77% (126/163) Updating files: 78% (128/163) Updating files: 79% (129/163) Updating files: 80% (131/163) Updating files: 81% (133/163) Updating files: 82% (134/163) Updating files: 83% (136/163) Updating files: 84% (137/163) Updating files: 85% (139/163) Updating files: 86% (141/163) Updating files: 87% (142/163) Updating files: 88% (144/163) Updating files: 89% (146/163) Updating files: 90% (147/163) Updating files: 91% (149/163) Updating files: 92% (150/163) Updating files: 93% (152/163) Updating files: 94% (154/163) Updating files: 95% (155/163) Updating files: 96% (157/163) Updating files: 97% (159/163) Updating files: 98% (160/163) Updating files: 99% (162/163) Updating files: 100% (163/163) Updating files: 100% (163/163), done. 2024-03-09T09:07:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.553+0000 7f1ecca00640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2ed0 0x7f1ec80c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.553+0000 7f1ecca00640 1 -- 172.21.15.67:0/3651711910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec81ef2f0 con 0x7f1ec80c2ed0 2024-03-09T09:07:34.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.553+0000 7f1ec7400640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2ed0 0x7f1ec80c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.554+0000 7f1ec7400640 1 -- 172.21.15.67:0/3651711910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1ec806c780 con 0x7f1ec80c2ed0 2024-03-09T09:07:34.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.554+0000 7f1ec7400640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2ed0 0x7f1ec80c32b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009ac0 tx=0x7f1eb8008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.555+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8048950 con 0x7f1ec80c2ed0 2024-03-09T09:07:34.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f1eb804c070 con 0x7f1ec80c2ed0 2024-03-09T09:07:34.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2ed0 msgr2=0x7f1ec80c32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ebf400640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2ed0 0x7f1ec80c32b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009ac0 tx=0x7f1eb8008030 comp rx=0 tx=0).stop 2024-03-09T09:07:34.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ebf400640 1 -- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2ed0 msgr2=0x7f1ec80c32b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:34.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ebf400640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec80c2ed0 0x7f1ec80c32b0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009ac0 tx=0x7f1eb8008030 comp rx=0 tx=0).stop 2024-03-09T09:07:34.560 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:07:34.560 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:07:34.560 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:07:34.560 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:07:34.560 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 110, 2024-03-09T09:07:34.560 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:34.561 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:34.562 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:34.562 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:34.562 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:34.562 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:34.562 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:34.562 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:34.562 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:34.562 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:34.562 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:07:34.562 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:07:34.562 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.564 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:07:34.564 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:07:34.564 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:07:34.564 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:07:34.564 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:07:34.564 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:07:34.564 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:34.564 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:34.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:07:34.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:07:34.564 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:07:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.567 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:07:34.568 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:34.569 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:34.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:34.569 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:07:34.569 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:34.569 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:07:34.570 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.570 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.570 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.570 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:07:34.570 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:07:34.570 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:34.570 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:34.570 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:34.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:07:34.571 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:34.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.572 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.572 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:07:34.572 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:07:34.572 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:34.572 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:34.572 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:34.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.572 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.573 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:07:34.573 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:07:34.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.573 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:34.573 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:34.573 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:07:34.573 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.573 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:34.573 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:07:34.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:34.573 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:34.574 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:34.574 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:07:34.574 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:34.574 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:34.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:34.574 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:07:34.574 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:07:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ecca00640 1 -- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ea806b500 msgr2=0x7f1ea806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ecca00640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ea806b500 0x7f1ea806d9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80c7dc0 tx=0x7f1ec0001360 comp rx=0 tx=0).stop 2024-03-09T09:07:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ecca00640 1 -- 172.21.15.67:0/3651711910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80c0f10 msgr2=0x7f1ec81ee600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ecca00640 1 --2- 172.21.15.67:0/3651711910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec80c0f10 0x7f1ec81ee600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d830 tx=0x7f1eb400dcf0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ec7e00640 1 -- 172.21.15.67:0/3651711910 reap_dead start 2024-03-09T09:07:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ec7e00640 1 -- 172.21.15.67:0/3651711910 reap_dead start 2024-03-09T09:07:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ecca00640 1 -- 172.21.15.67:0/3651711910 shutdown_connections 2024-03-09T09:07:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ecca00640 1 -- 172.21.15.67:0/3651711910 >> 172.21.15.67:0/3651711910 conn(0x7f1ec80cb480 msgr2=0x7f1ec80ca0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ecca00640 1 -- 172.21.15.67:0/3651711910 shutdown_connections 2024-03-09T09:07:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.556+0000 7f1ecca00640 1 -- 172.21.15.67:0/3651711910 wait complete. 2024-03-09T09:07:34.576 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:07:34.577 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:07:34.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:07:34.599 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 5d1d8d0c460 v0.23 2024-03-09T09:07:34.613 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.23 2024-03-09T09:07:34.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.786+0000 7fe6edc00640 1 Processor -- start 2024-03-09T09:07:34.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.786+0000 7fe6edc00640 1 -- start start 2024-03-09T09:07:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.788+0000 7fe6edc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6e80c2780 0x7fe6e8154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.788+0000 7fe6edc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e80c8200 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.788+0000 7fe6edc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e80c3990 con 0x7fe6e8071190 2024-03-09T09:07:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.788+0000 7fe6edc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e806d590 con 0x7fe6e80c2780 2024-03-09T09:07:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.788+0000 7fe6e6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6e80c2780 0x7fe6e8154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.788+0000 7fe6e6a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe6e80ac9a0 0x7fe6e81510e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52844/0 (socket says 172.21.15.67:52844) 2024-03-09T09:07:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.788+0000 7fe6e6a00640 1 -- 172.21.15.67:0/2087224183 learned_addr learned my addr 172.21.15.67:0/2087224183 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:07:34.794 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:07:34.794 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 24 -eq 48 ']' 2024-03-09T09:07:34.794 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.24 2024-03-09T09:07:34.794 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.24 2024-03-09T09:07:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.789+0000 7fe6e5600640 1 -- 172.21.15.67:0/2087224183 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597210498 0 0) 0x7fe6e80c8200 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.789+0000 7fe6e5600640 1 -- 172.21.15.67:0/2087224183 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6cc003520 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.789+0000 7fe6e6000640 1 -- 172.21.15.67:0/2087224183 >> v1:172.21.15.134:6789/0 conn(0x7fe6e8071190 legacy=0x7fe6e81546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.789+0000 7fe6e6000640 1 -- 172.21.15.67:0/2087224183 >> v1:172.21.15.67:6789/0 conn(0x7fe6e80ac9a0 legacy=0x7fe6e81510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.789+0000 7fe6e6000640 1 -- 172.21.15.67:0/2087224183 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8070920 con 0x7fe6e80c2780 2024-03-09T09:07:34.795 INFO:tasks.workunit.client.1.smithi134.stdout:v0.24 2024-03-09T09:07:34.795 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.24 2024-03-09T09:07:34.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.791+0000 7fe6e6000640 1 --2- 172.21.15.67:0/2087224183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80c2780 0x7fe6e8154190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40097a0 tx=0x7fe6d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57fb391afd01382 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.791+0000 7fe6e5600640 1 -- 172.21.15.67:0/2087224183 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4035020 con 0x7fe6e80c2780 2024-03-09T09:07:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.791+0000 7fe6e5600640 1 -- 172.21.15.67:0/2087224183 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d4031830 con 0x7fe6e80c2780 2024-03-09T09:07:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.791+0000 7fe6e5600640 1 -- 172.21.15.67:0/2087224183 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4031c00 con 0x7fe6e80c2780 2024-03-09T09:07:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.792+0000 7fe6edc00640 1 -- 172.21.15.67:0/2087224183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80c2780 msgr2=0x7fe6e8154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.792+0000 7fe6edc00640 1 --2- 172.21.15.67:0/2087224183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80c2780 0x7fe6e8154190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40097a0 tx=0x7fe6d402b8d0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.792+0000 7fe6edc00640 1 -- 172.21.15.67:0/2087224183 shutdown_connections 2024-03-09T09:07:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.792+0000 7fe6edc00640 1 --2- 172.21.15.67:0/2087224183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80c2780 0x7fe6e8154190 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.792+0000 7fe6edc00640 1 -- 172.21.15.67:0/2087224183 >> 172.21.15.67:0/2087224183 conn(0x7fe6e80c7370 msgr2=0x7fe6e80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.793+0000 7fe6edc00640 1 -- 172.21.15.67:0/2087224183 shutdown_connections 2024-03-09T09:07:34.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.793+0000 7fe6edc00640 1 -- 172.21.15.67:0/2087224183 wait complete. 2024-03-09T09:07:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.794+0000 7fe6edc00640 1 Processor -- start 2024-03-09T09:07:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.795+0000 7fe6edc00640 1 -- start start 2024-03-09T09:07:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.795+0000 7fe6edc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8071190 0x7fe6e81ea110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.795+0000 7fe6edc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ac9a0 0x7fe6e814b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.795+0000 7fe6edc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80c2780 0x7fe6e8152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6edc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80acd80 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6edc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80c3740 con 0x7fe6e80c2780 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6edc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80acf00 con 0x7fe6e8071190 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ac9a0 0x7fe6e814b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ac9a0 0x7fe6e814b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59910/0 (socket says 172.21.15.67:59910) 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6000640 1 -- 172.21.15.67:0/3721431913 learned_addr learned my addr 172.21.15.67:0/3721431913 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8071190 0x7fe6e81ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e7400640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80c2780 0x7fe6e8152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6000640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8071190 msgr2=0x7fe6e81ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8071190 msgr2=0x7fe6e81ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8071190 msgr2=0x7fe6e81ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:07:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8071190 0x7fe6e81ea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8071190 0x7fe6e81ea110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6000640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8071190 0x7fe6e81ea110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6000640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80c2780 msgr2=0x7fe6e8152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6000640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6e80c2780 0x7fe6e8152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.796+0000 7fe6e6000640 1 -- 172.21.15.67:0/3721431913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8151d50 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.797+0000 7fe6e6000640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ac9a0 0x7fe6e814b860 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40098c0 tx=0x7fe6d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.797+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4040070 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.797+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d403e070 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.797+0000 7fe6edc00640 1 -- 172.21.15.67:0/3721431913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e8152030 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.797+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4035040 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.798+0000 7fe6edc00640 1 -- 172.21.15.67:0/3721431913 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e81525f0 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.799+0000 7fe6edc00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.799+0000 7fe6edc00640 1 -- 172.21.15.67:0/3721431913 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e8075a00 con 0x7fe6e8153330 2024-03-09T09:07:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.799+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.800+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 msgr2=0x7fe6e8145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-03-09T09:07:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.800+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 msgr2=0x7fe6e8145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:07:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.800+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.800+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:07:34.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.801+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe6d4038020 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.802+0000 7fe6dea00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6c806b500 0x7fe6c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:34.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.802+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe6d40ab9e0 con 0x7fe6e80ac9a0 2024-03-09T09:07:34.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.803+0000 7fe6e7400640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6c806b500 0x7fe6c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:34.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:34.804+0000 7fe6e7400640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6c806b500 0x7fe6c806d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0007fd0 tx=0x7fe6e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:35.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:35.001+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:35.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:35.002+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 msgr2=0x7fe6e8145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:07:35.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:35.002+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 msgr2=0x7fe6e8145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:07:35.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:35.002+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:35.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:35.002+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:07:35.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:35.403+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:35.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:35.404+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 msgr2=0x7fe6e8145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:07:35.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:35.404+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 msgr2=0x7fe6e8145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:07:35.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:35.404+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:35.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:35.404+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:07:36.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:36.207+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:36.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:36.208+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 msgr2=0x7fe6e8145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:07:36.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:36.208+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 msgr2=0x7fe6e8145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:07:36.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:36.208+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:07:36.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:36.208+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:07:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.811+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.812+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe6e8151790 con 0x7fe6e8153330 2024-03-09T09:07:37.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.813+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80011e0 tx=0x7fe6d8014d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:37.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.814+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d801a070 con 0x7fe6e8153330 2024-03-09T09:07:37.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.815+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fe6d8001d00 con 0x7fe6e8153330 2024-03-09T09:07:37.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.815+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 msgr2=0x7fe6e8145d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:37.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.815+0000 7fe6dea00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80011e0 tx=0x7fe6d8014d70 comp rx=0 tx=0).stop 2024-03-09T09:07:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.815+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 msgr2=0x7fe6e8145d80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.815+0000 7fe6dea00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8153330 0x7fe6e8145d80 secure :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80011e0 tx=0x7fe6d8014d70 comp rx=0 tx=0).stop 2024-03-09T09:07:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.823+0000 7fe6edc00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8146aa0 0x7fe6e81462c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.823+0000 7fe6edc00640 1 -- 172.21.15.67:0/3721431913 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e8154bc0 con 0x7fe6e8146aa0 2024-03-09T09:07:37.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.824+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8146aa0 0x7fe6e81462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.825+0000 7fe6e6a00640 1 -- 172.21.15.67:0/3721431913 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe6e8068460 con 0x7fe6e8146aa0 2024-03-09T09:07:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.826+0000 7fe6e6a00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8146aa0 0x7fe6e81462c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80011e0 tx=0x7fe6d8001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.827+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8012ca0 con 0x7fe6e8146aa0 2024-03-09T09:07:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.828+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fe6d8012ca0 con 0x7fe6e8146aa0 2024-03-09T09:07:37.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.828+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8146aa0 msgr2=0x7fe6e81462c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.828+0000 7fe6dea00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8146aa0 0x7fe6e81462c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80011e0 tx=0x7fe6d8001150 comp rx=0 tx=0).stop 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.828+0000 7fe6dea00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8146aa0 msgr2=0x7fe6e81462c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.828+0000 7fe6dea00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6e8146aa0 0x7fe6e81462c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 110, 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:07:37.832 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 6, 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:37.833 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:37.834 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:07:37.835 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:37.836 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:37.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:07:37.838 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.839 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:37.840 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:37.841 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:37.842 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:07:37.843 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:07:37.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.829+0000 7fe6edc00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6c806b500 msgr2=0x7fe6c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.829+0000 7fe6edc00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6c806b500 0x7fe6c806d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0007fd0 tx=0x7fe6e0009040 comp rx=0 tx=0).stop 2024-03-09T09:07:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.829+0000 7fe6edc00640 1 -- 172.21.15.67:0/3721431913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ac9a0 msgr2=0x7fe6e814b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.829+0000 7fe6edc00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ac9a0 0x7fe6e814b860 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40098c0 tx=0x7fe6d40046c0 comp rx=0 tx=0).stop 2024-03-09T09:07:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.829+0000 7fe6e7400640 1 -- 172.21.15.67:0/3721431913 reap_dead start 2024-03-09T09:07:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.829+0000 7fe6edc00640 1 -- 172.21.15.67:0/3721431913 shutdown_connections 2024-03-09T09:07:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.829+0000 7fe6edc00640 1 --2- 172.21.15.67:0/3721431913 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6e80ac9a0 0x7fe6e814b860 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.829+0000 7fe6edc00640 1 -- 172.21.15.67:0/3721431913 >> 172.21.15.67:0/3721431913 conn(0x7fe6e80c7370 msgr2=0x7fe6e8154090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.829+0000 7fe6edc00640 1 -- 172.21.15.67:0/3721431913 shutdown_connections 2024-03-09T09:07:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:37.829+0000 7fe6edc00640 1 -- 172.21.15.67:0/3721431913 wait complete. 2024-03-09T09:07:37.860 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:07:37.860 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:07:37.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:07:38.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.128+0000 7f59cc400640 1 Processor -- start 2024-03-09T09:07:38.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.128+0000 7f59cc400640 1 -- start start 2024-03-09T09:07:38.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.136+0000 7f59cc400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f59c40c1d70 0x7f59c40ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:38.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.136+0000 7f59cc400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f59c40c2400 0x7f59c40acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:38.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.136+0000 7f59cc400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c4156000 con 0x7f59c4153350 2024-03-09T09:07:38.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.136+0000 7f59cc400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c4156c10 con 0x7f59c40c1d70 2024-03-09T09:07:38.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.136+0000 7f59cc400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c41562e0 con 0x7f59c40c2400 2024-03-09T09:07:38.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.140+0000 7f59ca400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f59c40c2400 0x7f59c40acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:38.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.140+0000 7f59ca400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f59c40c2400 0x7f59c40acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:42678/0 (socket says 172.21.15.67:42678) 2024-03-09T09:07:38.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.140+0000 7f59ca400640 1 -- 172.21.15.67:0/1216651117 learned_addr learned my addr 172.21.15.67:0/1216651117 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:38.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.141+0000 7f59c9000640 1 --2- 172.21.15.67:0/1216651117 >> v2:172.21.15.67:3300/0 conn(0x7f59c40c1d70 0x7f59c40ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:38.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.144+0000 7f59ca400640 1 -- 172.21.15.67:0/1216651117 >> v2:172.21.15.67:3300/0 conn(0x7f59c40c1d70 msgr2=0x7f59c40ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:38.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.144+0000 7f59ca400640 1 --2- 172.21.15.67:0/1216651117 >> v2:172.21.15.67:3300/0 conn(0x7f59c40c1d70 0x7f59c40ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:38.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.144+0000 7f59ca400640 1 -- 172.21.15.67:0/1216651117 >> v1:172.21.15.67:6789/0 conn(0x7f59c4153350 legacy=0x7f59c40c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:38.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.147+0000 7f59ca400640 1 -- 172.21.15.67:0/1216651117 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c41565c0 con 0x7f59c40c2400 2024-03-09T09:07:38.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.153+0000 7f59ca400640 1 --2- 172.21.15.67:0/1216651117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c40c2400 0x7f59c40acbe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59c4070010 tx=0x7f59bc02bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=923a1f9c02fc58a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:38.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.153+0000 7f59c3e00640 1 -- 172.21.15.67:0/1216651117 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc02ca80 con 0x7f59c40c2400 2024-03-09T09:07:38.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.154+0000 7f59c3e00640 1 -- 172.21.15.67:0/1216651117 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59bc02cc20 con 0x7f59c40c2400 2024-03-09T09:07:38.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.154+0000 7f59c3e00640 1 -- 172.21.15.67:0/1216651117 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0343f0 con 0x7f59c40c2400 2024-03-09T09:07:38.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.157+0000 7f59cc400640 1 -- 172.21.15.67:0/1216651117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c40c2400 msgr2=0x7f59c40acbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:38.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.157+0000 7f59cc400640 1 --2- 172.21.15.67:0/1216651117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c40c2400 0x7f59c40acbe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59c4070010 tx=0x7f59bc02bee0 comp rx=0 tx=0).stop 2024-03-09T09:07:38.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.157+0000 7f59cc400640 1 -- 172.21.15.67:0/1216651117 shutdown_connections 2024-03-09T09:07:38.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.157+0000 7f59cc400640 1 --2- 172.21.15.67:0/1216651117 >> v2:172.21.15.67:3300/0 conn(0x7f59c40c1d70 0x7f59c40ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:38.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.157+0000 7f59cc400640 1 --2- 172.21.15.67:0/1216651117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c40c2400 0x7f59c40acbe0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:38.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.158+0000 7f59cc400640 1 -- 172.21.15.67:0/1216651117 >> 172.21.15.67:0/1216651117 conn(0x7f59c40c7370 msgr2=0x7f59c40c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:38.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.158+0000 7f59cc400640 1 -- 172.21.15.67:0/1216651117 shutdown_connections 2024-03-09T09:07:38.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.158+0000 7f59cc400640 1 -- 172.21.15.67:0/1216651117 wait complete. 2024-03-09T09:07:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.161+0000 7f59cc400640 1 Processor -- start 2024-03-09T09:07:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.161+0000 7f59cc400640 1 -- start start 2024-03-09T09:07:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.162+0000 7f59cc400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c40c1d70 0x7f59c41ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:38.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.163+0000 7f59cc400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c40c2400 0x7f59c41efaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.163+0000 7f59cc400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c4153350 0x7f59c4148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.163+0000 7f59cc400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c41538b0 con 0x7f59c4153350 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.163+0000 7f59cc400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c4153730 con 0x7f59c40c1d70 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.163+0000 7f59cc400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40aae50 con 0x7f59c40c2400 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.163+0000 7f59c9a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c40c1d70 0x7f59c41ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.163+0000 7f59c9a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c40c1d70 0x7f59c41ef5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:42690/0 (socket says 172.21.15.67:42690) 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.163+0000 7f59c9a00640 1 -- 172.21.15.67:0/4231605394 learned_addr learned my addr 172.21.15.67:0/4231605394 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.163+0000 7f59c9000640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c40c2400 0x7f59c41efaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.163+0000 7f59ca400640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c4153350 0x7f59c4148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.164+0000 7f59c9a00640 1 -- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c40c2400 msgr2=0x7f59c41efaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.164+0000 7f59c9a00640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c40c2400 0x7f59c41efaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.164+0000 7f59c9a00640 1 -- 172.21.15.67:0/4231605394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c4153350 msgr2=0x7f59c4148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.164+0000 7f59c9a00640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c4153350 0x7f59c4148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.164+0000 7f59c9a00640 1 -- 172.21.15.67:0/4231605394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c41f0030 con 0x7f59c40c1d70 2024-03-09T09:07:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.164+0000 7f59c9a00640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c40c1d70 0x7f59c41ef5b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59b800c3c0 tx=0x7f59b800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.164+0000 7f59c1600640 1 -- 172.21.15.67:0/4231605394 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b80052d0 con 0x7f59c40c1d70 2024-03-09T09:07:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.164+0000 7f59c1600640 1 -- 172.21.15.67:0/4231605394 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b8005490 con 0x7f59c40c1d70 2024-03-09T09:07:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.164+0000 7f59cc400640 1 -- 172.21.15.67:0/4231605394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c4159ec0 con 0x7f59c40c1d70 2024-03-09T09:07:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.165+0000 7f59cc400640 1 -- 172.21.15.67:0/4231605394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c4157620 con 0x7f59c40c1d70 2024-03-09T09:07:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.165+0000 7f59c1600640 1 -- 172.21.15.67:0/4231605394 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8007260 con 0x7f59c40c1d70 2024-03-09T09:07:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.166+0000 7f59cc400640 1 -- 172.21.15.67:0/4231605394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c4075a00 con 0x7f59c40c1d70 2024-03-09T09:07:38.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.168+0000 7f59c1600640 1 -- 172.21.15.67:0/4231605394 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f59b8007400 con 0x7f59c40c1d70 2024-03-09T09:07:38.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.169+0000 7f59c1600640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59a406b4d0 0x7f59a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.169+0000 7f59c1600640 1 -- 172.21.15.67:0/4231605394 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f59b8089b00 con 0x7f59c40c1d70 2024-03-09T09:07:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.169+0000 7f59c9000640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59a406b4d0 0x7f59a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:38.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.170+0000 7f59c9000640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59a406b4d0 0x7f59a406d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f59c4152cf0 tx=0x7f59b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:38.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.175+0000 7f59c1600640 1 -- 172.21.15.67:0/4231605394 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f59b8058580 con 0x7f59c40c1d70 2024-03-09T09:07:38.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.659+0000 7f59cc400640 1 -- 172.21.15.67:0/4231605394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f59c406e430 con 0x7f59c40c1d70 2024-03-09T09:07:38.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.662+0000 7f59c1600640 1 -- 172.21.15.67:0/4231605394 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f59b8057e80 con 0x7f59c40c1d70 2024-03-09T09:07:38.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 -- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59a406b4d0 msgr2=0x7f59a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:38.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59a406b4d0 0x7f59a406d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f59c4152cf0 tx=0x7f59b0008040 comp rx=0 tx=0).stop 2024-03-09T09:07:38.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 -- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c40c1d70 msgr2=0x7f59c41ef5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:38.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c40c1d70 0x7f59c41ef5b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59b800c3c0 tx=0x7f59b800c880 comp rx=0 tx=0).stop 2024-03-09T09:07:38.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 -- 172.21.15.67:0/4231605394 shutdown_connections 2024-03-09T09:07:38.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59a406b4d0 0x7f59a406d990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:38.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c4153350 0x7f59c4148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:38.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c40c2400 0x7f59c41efaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:38.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 --2- 172.21.15.67:0/4231605394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c40c1d70 0x7f59c41ef5b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:38.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 -- 172.21.15.67:0/4231605394 >> 172.21.15.67:0/4231605394 conn(0x7f59c40c7370 msgr2=0x7f59c40c2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:38.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 -- 172.21.15.67:0/4231605394 shutdown_connections 2024-03-09T09:07:38.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:38.665+0000 7f59cc400640 1 -- 172.21.15.67:0/4231605394 wait complete. 2024-03-09T09:07:38.692 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:07:42.785 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 36% (67/185) Updating files: 37% (69/185) Updating files: 38% (71/185) Updating files: 39% (73/185) Updating files: 40% (74/185) Updating files: 41% (76/185) Updating files: 42% (78/185) Updating files: 43% (80/185) Updating files: 44% (82/185) Updating files: 45% (84/185) Updating files: 46% (86/185) Updating files: 47% (87/185) Updating files: 48% (89/185) Updating files: 49% (91/185) Updating files: 50% (93/185) Updating files: 51% (95/185) Updating files: 52% (97/185) Updating files: 53% (99/185) Updating files: 54% (100/185) Updating files: 55% (102/185) Updating files: 56% (104/185) Updating files: 57% (106/185) Updating files: 58% (108/185) Updating files: 59% (110/185) Updating files: 60% (111/185) Updating files: 61% (113/185) Updating files: 62% (115/185) Updating files: 63% (117/185) Updating files: 64% (119/185) Updating files: 65% (121/185) Updating files: 66% (123/185) Updating files: 67% (124/185) Updating files: 68% (126/185) Updating files: 69% (128/185) Updating files: 70% (130/185) Updating files: 71% (132/185) Updating files: 72% (134/185) Updating files: 73% (136/185) Updating files: 74% (137/185) Updating files: 75% (139/185) Updating files: 76% (141/185) Updating files: 77% (143/185) Updating files: 78% (145/185) Updating files: 79% (147/185) Updating files: 80% (148/185) Updating files: 81% (150/185) Updating files: 82% (152/185) Updating files: 83% (154/185) Updating files: 84% (156/185) Updating files: 85% (158/185) Updating files: 86% (160/185) Updating files: 87% (161/185) Updating files: 88% (163/185) Updating files: 89% (165/185) Updating files: 90% (167/185) Updating files: 91% (169/185) Updating files: 92% (171/185) Updating files: 93% (173/185) Updating files: 94% (174/185) Updating files: 95% (176/185) Updating files: 96% (178/185) Updating files: 97% (180/185) Updating files: 98% (182/185) Updating files: 99% (184/185) Updating files: 100% (185/185) Updating files: 100% (185/185), done. 2024-03-09T09:07:42.864 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 180a4176035 v0.24 2024-03-09T09:07:42.881 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.24 2024-03-09T09:07:43.136 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:07:43.136 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 25 -eq 48 ']' 2024-03-09T09:07:43.137 INFO:tasks.workunit.client.1.smithi134.stdout:v0.25 2024-03-09T09:07:43.137 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.25 2024-03-09T09:07:43.137 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.25 2024-03-09T09:07:43.137 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.25 2024-03-09T09:07:45.382 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 10% (20/185) Updating files: 11% (21/185) Updating files: 12% (23/185) Updating files: 13% (25/185) Updating files: 14% (26/185) Updating files: 15% (28/185) Updating files: 16% (30/185) Updating files: 17% (32/185) Updating files: 18% (34/185) Updating files: 19% (36/185) Updating files: 20% (37/185) Updating files: 21% (39/185) Updating files: 22% (41/185) Updating files: 23% (43/185) Updating files: 24% (45/185) Updating files: 25% (47/185) Updating files: 26% (49/185) Updating files: 27% (50/185) Updating files: 28% (52/185) Updating files: 29% (54/185) Updating files: 30% (56/185) Updating files: 31% (58/185) Updating files: 32% (60/185) Updating files: 33% (62/185) Updating files: 34% (63/185) Updating files: 35% (65/185) Updating files: 36% (67/185) Updating files: 37% (69/185) Updating files: 38% (71/185) Updating files: 39% (73/185) Updating files: 40% (74/185) Updating files: 41% (76/185) Updating files: 42% (78/185) Updating files: 43% (80/185) Updating files: 44% (82/185) Updating files: 45% (84/185) Updating files: 46% (86/185) Updating files: 47% (87/185) Updating files: 48% (89/185) Updating files: 49% (91/185) Updating files: 50% (93/185) Updating files: 51% (95/185) Updating files: 52% (97/185) Updating files: 53% (99/185) Updating files: 54% (100/185) Updating files: 55% (102/185) Updating files: 56% (104/185) Updating files: 57% (106/185) Updating files: 58% (108/185) Updating files: 59% (110/185) Updating files: 60% (111/185) Updating files: 61% (113/185) Updating files: 62% (115/185) Updating files: 63% (117/185) Updating files: 64% (119/185) Updating files: 65% (121/185) Updating files: 66% (123/185) Updating files: 67% (124/185) Updating files: 68% (126/185) Updating files: 69% (128/185) Updating files: 70% (130/185) Updating files: 71% (132/185) Updating files: 72% (134/185) Updating files: 73% (136/185) Updating files: 74% (137/185) Updating files: 75% (139/185) Updating files: 76% (141/185) Updating files: 77% (143/185) Updating files: 78% (145/185) Updating files: 79% (147/185) Updating files: 80% (148/185) Updating files: 81% (150/185) Updating files: 82% (152/185) Updating files: 83% (154/185) Updating files: 84% (156/185) Updating files: 85% (158/185) Updating files: 86% (160/185) Updating files: 87% (161/185) Updating files: 88% (163/185) Updating files: 89% (165/185) Updating files: 90% (167/185) Updating files: 91% (169/185) Updating files: 92% (171/185) Updating files: 93% (173/185) Updating files: 94% (174/185) Updating files: 95% (176/185) Updating files: 96% (178/185) Updating files: 97% (180/185) Updating files: 97% (181/185) Updating files: 98% (182/185) Updating files: 99% (184/185) Updating files: 100% (185/185) Updating files: 100% (185/185), done. 2024-03-09T09:07:45.462 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 180a4176035 v0.24 2024-03-09T09:07:45.475 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.24 2024-03-09T09:07:45.667 INFO:tasks.workunit.client.0.smithi067.stdout:v0.25 2024-03-09T09:07:45.667 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:07:45.668 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 25 -eq 48 ']' 2024-03-09T09:07:45.668 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.25 2024-03-09T09:07:45.668 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.25 2024-03-09T09:07:45.668 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.25 2024-03-09T09:07:46.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:07:46.261 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:07:48.693 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:07:48.694 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:07:48.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.923+0000 7f4e5fc00640 1 Processor -- start 2024-03-09T09:07:48.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.923+0000 7f4e5fc00640 1 -- start start 2024-03-09T09:07:48.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.926+0000 7f4e5fc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4e580a9ab0 0x7f4e580ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:48.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.926+0000 7f4e5fc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e580abcd0 0x7f4e580ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:48.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.926+0000 7f4e5fc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e580ab550 con 0x7f4e5815a1b0 2024-03-09T09:07:48.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.926+0000 7f4e5fc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5815a700 con 0x7f4e580abcd0 2024-03-09T09:07:48.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.926+0000 7f4e5fc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5815a9e0 con 0x7f4e580a9ab0 2024-03-09T09:07:48.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.927+0000 7f4e5dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e580abcd0 0x7f4e580ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:48.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.927+0000 7f4e5dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e580abcd0 0x7f4e580ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53784/0 (socket says 172.21.15.67:53784) 2024-03-09T09:07:48.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.927+0000 7f4e5dc00640 1 -- 172.21.15.67:0/2767610212 learned_addr learned my addr 172.21.15.67:0/2767610212 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:48.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.927+0000 7f4e5d200640 1 --2- 172.21.15.67:0/2767610212 >> v2:172.21.15.134:3300/0 conn(0x7f4e580a9ab0 0x7f4e580ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:48.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.928+0000 7f4e5dc00640 1 -- 172.21.15.67:0/2767610212 >> v2:172.21.15.134:3300/0 conn(0x7f4e580a9ab0 msgr2=0x7f4e580ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:48.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.928+0000 7f4e5dc00640 1 --2- 172.21.15.67:0/2767610212 >> v2:172.21.15.134:3300/0 conn(0x7f4e580a9ab0 0x7f4e580ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:48.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.928+0000 7f4e5dc00640 1 -- 172.21.15.67:0/2767610212 >> v1:172.21.15.134:6789/0 conn(0x7f4e5815a1b0 legacy=0x7f4e580aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:48.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.928+0000 7f4e5dc00640 1 -- 172.21.15.67:0/2767610212 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5815acc0 con 0x7f4e580abcd0 2024-03-09T09:07:48.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.930+0000 7f4e5dc00640 1 --2- 172.21.15.67:0/2767610212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580abcd0 0x7f4e580ab010 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4e58070370 tx=0x7f4e5002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f4097e7769d8af server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:48.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.931+0000 7f4e57400640 1 -- 172.21.15.67:0/2767610212 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5002cb10 con 0x7f4e580abcd0 2024-03-09T09:07:48.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.931+0000 7f4e57400640 1 -- 172.21.15.67:0/2767610212 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5002ccb0 con 0x7f4e580abcd0 2024-03-09T09:07:48.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.931+0000 7f4e57400640 1 -- 172.21.15.67:0/2767610212 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5002ce60 con 0x7f4e580abcd0 2024-03-09T09:07:48.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.932+0000 7f4e5fc00640 1 -- 172.21.15.67:0/2767610212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580abcd0 msgr2=0x7f4e580ab010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:48.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.932+0000 7f4e5fc00640 1 --2- 172.21.15.67:0/2767610212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580abcd0 0x7f4e580ab010 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4e58070370 tx=0x7f4e5002bf70 comp rx=0 tx=0).stop 2024-03-09T09:07:48.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.936+0000 7f4e5fc00640 1 -- 172.21.15.67:0/2767610212 shutdown_connections 2024-03-09T09:07:48.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.936+0000 7f4e5fc00640 1 --2- 172.21.15.67:0/2767610212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580abcd0 0x7f4e580ab010 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:48.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.936+0000 7f4e5fc00640 1 --2- 172.21.15.67:0/2767610212 >> v2:172.21.15.134:3300/0 conn(0x7f4e580a9ab0 0x7f4e580ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:48.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.936+0000 7f4e5fc00640 1 -- 172.21.15.67:0/2767610212 >> 172.21.15.67:0/2767610212 conn(0x7f4e580c7370 msgr2=0x7f4e580c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:48.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.937+0000 7f4e5fc00640 1 -- 172.21.15.67:0/2767610212 shutdown_connections 2024-03-09T09:07:48.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.937+0000 7f4e5fc00640 1 -- 172.21.15.67:0/2767610212 wait complete. 2024-03-09T09:07:48.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.940+0000 7f4e5fc00640 1 Processor -- start 2024-03-09T09:07:48.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.940+0000 7f4e5fc00640 1 -- start start 2024-03-09T09:07:48.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.942+0000 7f4e5fc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e580a9ab0 0x7f4e58155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:48.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.943+0000 7f4e5fc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580abcd0 0x7f4e58152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:48.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.943+0000 7f4e5fc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e5815a1b0 0x7f4e58152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:48.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.943+0000 7f4e5fc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5806dfa0 con 0x7f4e580abcd0 2024-03-09T09:07:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.944+0000 7f4e5fc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580a9e90 con 0x7f4e5815a1b0 2024-03-09T09:07:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.944+0000 7f4e5fc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5806e120 con 0x7f4e580a9ab0 2024-03-09T09:07:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.944+0000 7f4e57e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580abcd0 0x7f4e58152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.944+0000 7f4e57e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580abcd0 0x7f4e58152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53794/0 (socket says 172.21.15.67:53794) 2024-03-09T09:07:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.944+0000 7f4e57e00640 1 -- 172.21.15.67:0/890974392 learned_addr learned my addr 172.21.15.67:0/890974392 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:48.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.945+0000 7f4e57e00640 1 -- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e580a9ab0 msgr2=0x7f4e58155bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T09:07:48.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.945+0000 7f4e57e00640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e580a9ab0 0x7f4e58155bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:48.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.945+0000 7f4e57e00640 1 -- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e5815a1b0 msgr2=0x7f4e58152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.945+0000 7f4e57e00640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e5815a1b0 0x7f4e58152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.945+0000 7f4e57e00640 1 -- 172.21.15.67:0/890974392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58156840 con 0x7f4e580abcd0 2024-03-09T09:07:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.947+0000 7f4e57e00640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580abcd0 0x7f4e58152420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400c430 tx=0x7f4e4400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:48.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.948+0000 7f4e54c00640 1 -- 172.21.15.67:0/890974392 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44017070 con 0x7f4e580abcd0 2024-03-09T09:07:48.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.949+0000 7f4e54c00640 1 -- 172.21.15.67:0/890974392 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e440052d0 con 0x7f4e580abcd0 2024-03-09T09:07:48.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.949+0000 7f4e54c00640 1 -- 172.21.15.67:0/890974392 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44003830 con 0x7f4e580abcd0 2024-03-09T09:07:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.949+0000 7f4e5fc00640 1 -- 172.21.15.67:0/890974392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e581e3940 con 0x7f4e580abcd0 2024-03-09T09:07:48.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.950+0000 7f4e5fc00640 1 -- 172.21.15.67:0/890974392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e580c28d0 con 0x7f4e580abcd0 2024-03-09T09:07:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.953+0000 7f4e5fc00640 1 -- 172.21.15.67:0/890974392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e58075a00 con 0x7f4e580abcd0 2024-03-09T09:07:48.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.954+0000 7f4e54c00640 1 -- 172.21.15.67:0/890974392 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4e4401a070 con 0x7f4e580abcd0 2024-03-09T09:07:48.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.955+0000 7f4e54c00640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e3806b550 0x7f4e3806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:48.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.955+0000 7f4e54c00640 1 -- 172.21.15.67:0/890974392 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4e440898e0 con 0x7f4e580abcd0 2024-03-09T09:07:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.957+0000 7f4e5d200640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e3806b550 0x7f4e3806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:48.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.958+0000 7f4e5d200640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e3806b550 0x7f4e3806da10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00b7f0 tx=0x7f4e4c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:48.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:48.967+0000 7f4e54c00640 1 -- 172.21.15.67:0/890974392 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4e440584e0 con 0x7f4e580abcd0 2024-03-09T09:07:49.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.376+0000 7f4e5fc00640 1 -- 172.21.15.67:0/890974392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4e5806e790 con 0x7f4e580abcd0 2024-03-09T09:07:49.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.376+0000 7f4e54c00640 1 -- 172.21.15.67:0/890974392 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f4e44057de0 con 0x7f4e580abcd0 2024-03-09T09:07:49.380 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 -- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e3806b550 msgr2=0x7f4e3806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e3806b550 0x7f4e3806da10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00b7f0 tx=0x7f4e4c008040 comp rx=0 tx=0).stop 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 -- 172.21.15.67:0/890974392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580abcd0 msgr2=0x7f4e58152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580abcd0 0x7f4e58152420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400c430 tx=0x7f4e4400c8f0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 -- 172.21.15.67:0/890974392 shutdown_connections 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e3806b550 0x7f4e3806da10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e5815a1b0 0x7f4e58152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580abcd0 0x7f4e58152420 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 --2- 172.21.15.67:0/890974392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e580a9ab0 0x7f4e58155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 -- 172.21.15.67:0/890974392 >> 172.21.15.67:0/890974392 conn(0x7f4e580c7370 msgr2=0x7f4e581593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 -- 172.21.15.67:0/890974392 shutdown_connections 2024-03-09T09:07:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.379+0000 7f4e5fc00640 1 -- 172.21.15.67:0/890974392 wait complete. 2024-03-09T09:07:49.408 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:07:49.408 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:07:49.408 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:07:49.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:07:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.619+0000 7fa343e00640 1 Processor -- start 2024-03-09T09:07:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.619+0000 7fa343e00640 1 -- start start 2024-03-09T09:07:49.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.621+0000 7fa343e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa34406c1d0 0x7fa34406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.621+0000 7fa343e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3440c9890 con 0x7fa3440c9380 2024-03-09T09:07:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.622+0000 7fa343e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3440c5f80 con 0x7fa34414fd90 2024-03-09T09:07:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.622+0000 7fa342000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa3440c9380 0x7fa3440acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49968/0 (socket says 172.21.15.67:49968) 2024-03-09T09:07:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.622+0000 7fa342000640 1 -- 172.21.15.67:0/2809415782 learned_addr learned my addr 172.21.15.67:0/2809415782 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:07:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.622+0000 7fa343e00640 1 -- 172.21.15.67:0/2809415782 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa34406ba60 con 0x7fa34406c1d0 2024-03-09T09:07:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.622+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330656363 0 0) 0x7fa3440c9890 con 0x7fa3440c9380 2024-03-09T09:07:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.622+0000 7fa343400640 1 --2- 172.21.15.67:0/2809415782 >> v2:172.21.15.134:3300/0 conn(0x7fa34406c1d0 0x7fa34406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.622+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa328003520 con 0x7fa3440c9380 2024-03-09T09:07:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3713596390 0 0) 0x7fa328003520 con 0x7fa3440c9380 2024-03-09T09:07:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 >> v2:172.21.15.134:3300/0 conn(0x7fa34406c1d0 msgr2=0x7fa34406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 --2- 172.21.15.67:0/2809415782 >> v2:172.21.15.134:3300/0 conn(0x7fa34406c1d0 0x7fa34406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 >> v1:172.21.15.134:6789/0 conn(0x7fa34414fd90 legacy=0x7fa344156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3440c6240 con 0x7fa3440c9380 2024-03-09T09:07:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fa330001670 con 0x7fa3440c9380 2024-03-09T09:07:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 >> v1:172.21.15.67:6789/0 conn(0x7fa3440c9380 legacy=0x7fa3440acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 --2- 172.21.15.67:0/2809415782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa328003a20 0x7fa328003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 --2- 172.21.15.67:0/2809415782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 0x7fa328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 --2- 172.21.15.67:0/2809415782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3280052e0 0x7fa32800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34406ba60 con 0x7fa328004800 2024-03-09T09:07:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32800c1e0 con 0x7fa3280052e0 2024-03-09T09:07:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32800c3e0 con 0x7fa328003a20 2024-03-09T09:07:49.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.623+0000 7fa342000640 1 --2- 172.21.15.67:0/2809415782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 0x7fa328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.624+0000 7fa342a00640 1 --2- 172.21.15.67:0/2809415782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa328003a20 0x7fa328003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.624+0000 7fa343400640 1 --2- 172.21.15.67:0/2809415782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3280052e0 0x7fa32800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.624+0000 7fa342000640 1 -- 172.21.15.67:0/2809415782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa328003a20 msgr2=0x7fa328003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:49.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.624+0000 7fa342000640 1 --2- 172.21.15.67:0/2809415782 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa328003a20 0x7fa328003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.624+0000 7fa342000640 1 -- 172.21.15.67:0/2809415782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3280052e0 msgr2=0x7fa32800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:49.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.624+0000 7fa342000640 1 --2- 172.21.15.67:0/2809415782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3280052e0 0x7fa32800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.624+0000 7fa342000640 1 -- 172.21.15.67:0/2809415782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa328003520 con 0x7fa328004800 2024-03-09T09:07:49.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.624+0000 7fa343400640 1 -- 172.21.15.67:0/2809415782 reap_dead start 2024-03-09T09:07:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.625+0000 7fa342000640 1 --2- 172.21.15.67:0/2809415782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 0x7fa328004c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa33000c190 tx=0x7fa328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e63f590af8ff8ce server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.626+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330007c60 con 0x7fa328004800 2024-03-09T09:07:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.626+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa330030070 con 0x7fa328004800 2024-03-09T09:07:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.626+0000 7fa341600640 1 -- 172.21.15.67:0/2809415782 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330005d50 con 0x7fa328004800 2024-03-09T09:07:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.627+0000 7fa343e00640 1 -- 172.21.15.67:0/2809415782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 msgr2=0x7fa328004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.627+0000 7fa343e00640 1 --2- 172.21.15.67:0/2809415782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 0x7fa328004c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa33000c190 tx=0x7fa328005830 comp rx=0 tx=0).stop 2024-03-09T09:07:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.627+0000 7fa343e00640 1 -- 172.21.15.67:0/2809415782 shutdown_connections 2024-03-09T09:07:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.627+0000 7fa343e00640 1 --2- 172.21.15.67:0/2809415782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 0x7fa328004c80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.627+0000 7fa343e00640 1 -- 172.21.15.67:0/2809415782 >> 172.21.15.67:0/2809415782 conn(0x7fa3440cb480 msgr2=0x7fa3440cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:49.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.627+0000 7fa343e00640 1 -- 172.21.15.67:0/2809415782 shutdown_connections 2024-03-09T09:07:49.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.627+0000 7fa343e00640 1 -- 172.21.15.67:0/2809415782 wait complete. 2024-03-09T09:07:49.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.630+0000 7fa343e00640 1 Processor -- start 2024-03-09T09:07:49.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.630+0000 7fa343e00640 1 -- start start 2024-03-09T09:07:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.631+0000 7fa343e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 0x7fa3441581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.631+0000 7fa343e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34414c410 0x7fa3440c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa343e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3440c1930 0x7fa344159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa343e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa344156680 con 0x7fa328004800 2024-03-09T09:07:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa343e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34406bcb0 con 0x7fa34414c410 2024-03-09T09:07:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa343e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa344156800 con 0x7fa3440c1930 2024-03-09T09:07:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa342a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 0x7fa3441581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa342a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 0x7fa3441581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53812/0 (socket says 172.21.15.67:53812) 2024-03-09T09:07:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa342a00640 1 -- 172.21.15.67:0/3445185038 learned_addr learned my addr 172.21.15.67:0/3445185038 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:49.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa342000640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34414c410 0x7fa3440c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa343400640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3440c1930 0x7fa344159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa342a00640 1 -- 172.21.15.67:0/3445185038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3440c1930 msgr2=0x7fa344159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa342a00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3440c1930 0x7fa344159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa342a00640 1 -- 172.21.15.67:0/3445185038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34414c410 msgr2=0x7fa3440c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa342a00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34414c410 0x7fa3440c28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.632+0000 7fa342a00640 1 -- 172.21.15.67:0/3445185038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa344158720 con 0x7fa328004800 2024-03-09T09:07:49.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.633+0000 7fa342a00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 0x7fa3441581e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa33400e550 tx=0x7fa33400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:49.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.633+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3340052e0 con 0x7fa328004800 2024-03-09T09:07:49.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.633+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa334018070 con 0x7fa328004800 2024-03-09T09:07:49.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.633+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334003830 con 0x7fa328004800 2024-03-09T09:07:49.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.633+0000 7fa343e00640 1 -- 172.21.15.67:0/3445185038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34414c8d0 con 0x7fa328004800 2024-03-09T09:07:49.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.634+0000 7fa343e00640 1 -- 172.21.15.67:0/3445185038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3441579e0 con 0x7fa328004800 2024-03-09T09:07:49.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.635+0000 7fa343e00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa344159d70 0x7fa34414b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.636+0000 7fa343e00640 1 -- 172.21.15.67:0/3445185038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3440cbba0 con 0x7fa344159d70 2024-03-09T09:07:49.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.636+0000 7fa342000640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa344159d70 0x7fa34414b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.636+0000 7fa342000640 1 -- 172.21.15.67:0/3445185038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa34404f3c0 con 0x7fa344159d70 2024-03-09T09:07:49.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.636+0000 7fa342000640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa344159d70 0x7fa34414b5e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa33000c160 tx=0x7fa3440c6410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:49.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.637+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa3340039d0 con 0x7fa328004800 2024-03-09T09:07:49.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.638+0000 7fa33aa00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa32406b520 0x7fa32406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.638+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa33408a2c0 con 0x7fa328004800 2024-03-09T09:07:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.639+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33002c7e0 con 0x7fa344159d70 2024-03-09T09:07:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.639+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa330045070 con 0x7fa344159d70 2024-03-09T09:07:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.639+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa344159d70 msgr2=0x7fa34414b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.639+0000 7fa33aa00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa344159d70 0x7fa34414b5e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa33000c160 tx=0x7fa3440c6410 comp rx=0 tx=0).stop 2024-03-09T09:07:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.639+0000 7fa343400640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa32406b520 0x7fa32406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.639+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa344159d70 msgr2=0x7fa34414b5e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:49.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.639+0000 7fa33aa00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa344159d70 0x7fa34414b5e0 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa33000c160 tx=0x7fa3440c6410 comp rx=0 tx=0).stop 2024-03-09T09:07:49.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.640+0000 7fa343400640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa32406b520 0x7fa32406d9e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa3440c9760 tx=0x7fa33c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.647+0000 7fa343e00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa34414bb20 0x7fa3441ee740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.647+0000 7fa343e00640 1 -- 172.21.15.67:0/3445185038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34414bf00 con 0x7fa34414bb20 2024-03-09T09:07:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.648+0000 7fa342000640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa34414bb20 0x7fa3441ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.648+0000 7fa342000640 1 -- 172.21.15.67:0/3445185038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa34406d2d0 con 0x7fa34414bb20 2024-03-09T09:07:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.648+0000 7fa342000640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa34414bb20 0x7fa3441ee740 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa3440abfe0 tx=0x7fa330001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.649+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33004ae00 con 0x7fa34414bb20 2024-03-09T09:07:49.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.649+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7fa330030040 con 0x7fa34414bb20 2024-03-09T09:07:49.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.649+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa34414bb20 msgr2=0x7fa3441ee740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.649+0000 7fa33aa00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa34414bb20 0x7fa3441ee740 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa3440abfe0 tx=0x7fa330001df0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.649+0000 7fa33aa00640 1 -- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa34414bb20 msgr2=0x7fa3441ee740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:49.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.649+0000 7fa33aa00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa34414bb20 0x7fa3441ee740 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa3440abfe0 tx=0x7fa330001df0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.653 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:07:49.654 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:07:49.654 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:07:49.654 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:07:49.654 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 110, 2024-03-09T09:07:49.654 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:07:49.654 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:07:49.654 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:07:49.654 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:07:49.654 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:49.654 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 17, 2024-03-09T09:07:49.654 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:49.655 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:49.656 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:07:49.657 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:49.658 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:07:49.659 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.660 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.660 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:49.660 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:07:49.660 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.660 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.660 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.660 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:07:49.660 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:07:49.660 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.661 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:07:49.663 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:07:49.664 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.665 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:49.666 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:07:49.667 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:07:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.650+0000 7fa343e00640 1 -- 172.21.15.67:0/3445185038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa32406b520 msgr2=0x7fa32406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.650+0000 7fa343e00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa32406b520 0x7fa32406d9e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa3440c9760 tx=0x7fa33c00b040 comp rx=0 tx=0).stop 2024-03-09T09:07:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.650+0000 7fa343e00640 1 -- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 msgr2=0x7fa3441581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.650+0000 7fa343e00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 0x7fa3441581e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa33400e550 tx=0x7fa33400ea10 comp rx=0 tx=0).stop 2024-03-09T09:07:49.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.650+0000 7fa343400640 1 -- 172.21.15.67:0/3445185038 reap_dead start 2024-03-09T09:07:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.650+0000 7fa343e00640 1 -- 172.21.15.67:0/3445185038 shutdown_connections 2024-03-09T09:07:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.650+0000 7fa343e00640 1 --2- 172.21.15.67:0/3445185038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa328004800 0x7fa3441581e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.650+0000 7fa343e00640 1 -- 172.21.15.67:0/3445185038 >> 172.21.15.67:0/3445185038 conn(0x7fa3440cb480 msgr2=0x7fa3440c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.650+0000 7fa343e00640 1 -- 172.21.15.67:0/3445185038 shutdown_connections 2024-03-09T09:07:49.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.650+0000 7fa343e00640 1 -- 172.21.15.67:0/3445185038 wait complete. 2024-03-09T09:07:49.670 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:07:49.671 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:07:49.671 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:07:49.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.877+0000 7f7e06200640 1 Processor -- start 2024-03-09T09:07:49.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.877+0000 7f7e06200640 1 -- start start 2024-03-09T09:07:49.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.879+0000 7f7e06200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7e000c5ad0 0x7f7e000c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.879+0000 7f7e06200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e000c4110 con 0x7f7e000c29a0 2024-03-09T09:07:49.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.879+0000 7f7e06200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e00159ce0 con 0x7f7e000c7680 2024-03-09T09:07:49.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.879+0000 7f7e06200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0015a2a0 con 0x7f7e000c5ad0 2024-03-09T09:07:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.880+0000 7f7dffe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7e000c5ad0 0x7f7e000c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.880+0000 7f7dfea00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f7e000c7680 0x7f7e00073290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:38786/0 (socket says 172.21.15.67:38786) 2024-03-09T09:07:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.880+0000 7f7dfea00640 1 -- 172.21.15.67:0/2123106318 learned_addr learned my addr 172.21.15.67:0/2123106318 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:07:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.882+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806999886 0 0) 0x7f7e00159ce0 con 0x7f7e000c7680 2024-03-09T09:07:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.882+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de4003520 con 0x7f7e000c7680 2024-03-09T09:07:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1554466473 0 0) 0x7f7de4003520 con 0x7f7e000c7680 2024-03-09T09:07:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 >> v2:172.21.15.67:3300/0 conn(0x7f7e000c5ad0 msgr2=0x7f7e000c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 --2- 172.21.15.67:0/2123106318 >> v2:172.21.15.67:3300/0 conn(0x7f7e000c5ad0 0x7f7e000c2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 >> v1:172.21.15.67:6789/0 conn(0x7f7e000c29a0 legacy=0x7f7e00072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e00159fc0 con 0x7f7e000c7680 2024-03-09T09:07:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f7dec001690 con 0x7f7e000c7680 2024-03-09T09:07:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 >> v1:172.21.15.134:6789/0 conn(0x7f7e000c7680 legacy=0x7f7e00073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 --2- 172.21.15.67:0/2123106318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7de40039b0 0x7f7de4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 --2- 172.21.15.67:0/2123106318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7de4004800 0x7f7de4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 --2- 172.21.15.67:0/2123106318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de40052e0 0x7f7de400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0015a2a0 con 0x7f7de40039b0 2024-03-09T09:07:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de400c1b0 con 0x7f7de40052e0 2024-03-09T09:07:49.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.883+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de400c3b0 con 0x7f7de4004800 2024-03-09T09:07:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.886+0000 7f7dfea00640 1 --2- 172.21.15.67:0/2123106318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7de4004800 0x7f7de4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.886+0000 7f7dffe00640 1 --2- 172.21.15.67:0/2123106318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de40052e0 0x7f7de400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.886+0000 7f7dff400640 1 --2- 172.21.15.67:0/2123106318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7de40039b0 0x7f7de4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.886+0000 7f7dffe00640 1 -- 172.21.15.67:0/2123106318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7de4004800 msgr2=0x7f7de4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.887+0000 7f7dffe00640 1 --2- 172.21.15.67:0/2123106318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7de4004800 0x7f7de4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.887+0000 7f7dffe00640 1 -- 172.21.15.67:0/2123106318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7de40039b0 msgr2=0x7f7de4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.887+0000 7f7dffe00640 1 --2- 172.21.15.67:0/2123106318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7de40039b0 0x7f7de4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.887+0000 7f7dffe00640 1 -- 172.21.15.67:0/2123106318 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e00159fc0 con 0x7f7de40052e0 2024-03-09T09:07:49.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.887+0000 7f7dffe00640 1 -- 172.21.15.67:0/2123106318 reap_dead start 2024-03-09T09:07:49.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.888+0000 7f7dffe00640 1 --2- 172.21.15.67:0/2123106318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de40052e0 0x7f7de400b9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009f90 tx=0x7f7de4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ffe6798eec9a17 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.888+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8009c10 con 0x7f7de40052e0 2024-03-09T09:07:49.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.889+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df802cc20 con 0x7f7de40052e0 2024-03-09T09:07:49.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.889+0000 7f7dfe000640 1 -- 172.21.15.67:0/2123106318 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8003990 con 0x7f7de40052e0 2024-03-09T09:07:49.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.890+0000 7f7e06200640 1 -- 172.21.15.67:0/2123106318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de40052e0 msgr2=0x7f7de400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.890+0000 7f7e06200640 1 --2- 172.21.15.67:0/2123106318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de40052e0 0x7f7de400b9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009f90 tx=0x7f7de4005830 comp rx=0 tx=0).stop 2024-03-09T09:07:49.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.890+0000 7f7e06200640 1 -- 172.21.15.67:0/2123106318 shutdown_connections 2024-03-09T09:07:49.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.890+0000 7f7e06200640 1 --2- 172.21.15.67:0/2123106318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de40052e0 0x7f7de400b9c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.890+0000 7f7e06200640 1 -- 172.21.15.67:0/2123106318 >> 172.21.15.67:0/2123106318 conn(0x7f7e000ab320 msgr2=0x7f7e000ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:49.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.890+0000 7f7e06200640 1 -- 172.21.15.67:0/2123106318 shutdown_connections 2024-03-09T09:07:49.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.890+0000 7f7e06200640 1 -- 172.21.15.67:0/2123106318 wait complete. 2024-03-09T09:07:49.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.893+0000 7f7e06200640 1 Processor -- start 2024-03-09T09:07:49.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.893+0000 7f7e06200640 1 -- start start 2024-03-09T09:07:49.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.894+0000 7f7e06200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de40052e0 0x7f7e0014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.894+0000 7f7e06200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e000ac0b0 0x7f7e00151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.894+0000 7f7e06200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e00153f60 0x7f7e001543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.894+0000 7f7e06200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000c3850 con 0x7f7e000ac0b0 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.894+0000 7f7e06200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000c4440 con 0x7f7de40052e0 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.894+0000 7f7e06200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000c39d0 con 0x7f7e00153f60 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.894+0000 7f7dfea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e000ac0b0 0x7f7e00151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.894+0000 7f7dfea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e000ac0b0 0x7f7e00151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53850/0 (socket says 172.21.15.67:53850) 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.894+0000 7f7dfea00640 1 -- 172.21.15.67:0/675257162 learned_addr learned my addr 172.21.15.67:0/675257162 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.894+0000 7f7dff400640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de40052e0 0x7f7e0014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.895+0000 7f7dffe00640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e00153f60 0x7f7e001543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.895+0000 7f7dfea00640 1 -- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e00153f60 msgr2=0x7f7e001543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.895+0000 7f7dfea00640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7e00153f60 0x7f7e001543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.895+0000 7f7dfea00640 1 -- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de40052e0 msgr2=0x7f7e0014fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.895+0000 7f7dfea00640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7de40052e0 0x7f7e0014fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.895+0000 7f7dfea00640 1 -- 172.21.15.67:0/675257162 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e000ae1c0 con 0x7f7e000ac0b0 2024-03-09T09:07:49.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.895+0000 7f7dfea00640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e000ac0b0 0x7f7e00151d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7dec007310 tx=0x7f7dec0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:49.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.895+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec01a070 con 0x7f7e000ac0b0 2024-03-09T09:07:49.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.896+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec00f960 con 0x7f7e000ac0b0 2024-03-09T09:07:49.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.896+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec00fd50 con 0x7f7e000ac0b0 2024-03-09T09:07:49.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.896+0000 7f7e06200640 1 -- 172.21.15.67:0/675257162 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0014f8f0 con 0x7f7e000ac0b0 2024-03-09T09:07:49.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.896+0000 7f7e06200640 1 -- 172.21.15.67:0/675257162 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e001528f0 con 0x7f7e000ac0b0 2024-03-09T09:07:49.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.897+0000 7f7e06200640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e00153ad0 0x7f7e001ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.897+0000 7f7e06200640 1 -- 172.21.15.67:0/675257162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e00075a00 con 0x7f7e00153ad0 2024-03-09T09:07:49.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.898+0000 7f7dff400640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e00153ad0 0x7f7e001ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.898+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7dec01d070 con 0x7f7e000ac0b0 2024-03-09T09:07:49.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.898+0000 7f7dff400640 1 -- 172.21.15.67:0/675257162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e000c3630 con 0x7f7e00153ad0 2024-03-09T09:07:49.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.898+0000 7f7dff400640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e00153ad0 0x7f7e001ee9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7df0012630 tx=0x7f7df0012af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:49.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.899+0000 7f7df7400640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7de006b520 0x7f7de006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.899+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f7dec089f90 con 0x7f7e000ac0b0 2024-03-09T09:07:49.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.900+0000 7f7dffe00640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7de006b520 0x7f7de006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.900+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df00053e0 con 0x7f7e00153ad0 2024-03-09T09:07:49.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.900+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f7df001a070 con 0x7f7e00153ad0 2024-03-09T09:07:49.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.900+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e00153ad0 msgr2=0x7f7e001ee9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.900+0000 7f7df7400640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e00153ad0 0x7f7e001ee9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7df0012630 tx=0x7f7df0012af0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.900+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e00153ad0 msgr2=0x7f7e001ee9c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:49.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.900+0000 7f7df7400640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e00153ad0 0x7f7e001ee9c0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7df0012630 tx=0x7f7df0012af0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.900+0000 7f7dffe00640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7de006b520 0x7f7de006d9e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7de400c380 tx=0x7f7df802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:49.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.911+0000 7f7e06200640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e001eef00 0x7f7e001ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:49.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.911+0000 7f7e06200640 1 -- 172.21.15.67:0/675257162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001ef820 con 0x7f7e001eef00 2024-03-09T09:07:49.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.911+0000 7f7dff400640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e001eef00 0x7f7e001ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:49.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.912+0000 7f7dff400640 1 -- 172.21.15.67:0/675257162 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7e00159af0 con 0x7f7e001eef00 2024-03-09T09:07:49.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.912+0000 7f7dff400640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e001eef00 0x7f7e001ef2e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7df0012630 tx=0x7f7df0001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:49.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.912+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df0020c60 con 0x7f7e001eef00 2024-03-09T09:07:49.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f7df00067d0 con 0x7f7e001eef00 2024-03-09T09:07:49.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e001eef00 msgr2=0x7f7e001ef2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7df7400640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e001eef00 0x7f7e001ef2e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7df0012630 tx=0x7f7df0001120 comp rx=0 tx=0).stop 2024-03-09T09:07:49.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7df7400640 1 -- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e001eef00 msgr2=0x7f7e001ef2e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:49.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7df7400640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7e001eef00 0x7f7e001ef2e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.917 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:07:49.917 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:07:49.917 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:07:49.917 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:07:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 110, 2024-03-09T09:07:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:07:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:07:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:07:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:07:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 18, 2024-03-09T09:07:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:07:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:07:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:49.920 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:49.920 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:07:49.920 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:07:49.920 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:49.920 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:49.920 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:49.920 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:49.920 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:49.920 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:07:49.921 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:49.922 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:07:49.923 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:49.925 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:49.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:07:49.927 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.928 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:07:49.929 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:07:49.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7e06200640 1 -- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7de006b520 msgr2=0x7f7de006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7e06200640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7de006b520 0x7f7de006d9e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7de400c380 tx=0x7f7df802e000 comp rx=0 tx=0).stop 2024-03-09T09:07:49.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7e06200640 1 -- 172.21.15.67:0/675257162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e000ac0b0 msgr2=0x7f7e00151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:49.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7e06200640 1 --2- 172.21.15.67:0/675257162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e000ac0b0 0x7f7e00151d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7dec007310 tx=0x7f7dec0077d0 comp rx=0 tx=0).stop 2024-03-09T09:07:49.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7dffe00640 1 -- 172.21.15.67:0/675257162 reap_dead start 2024-03-09T09:07:49.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7dffe00640 1 -- 172.21.15.67:0/675257162 reap_dead start 2024-03-09T09:07:49.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7e06200640 1 -- 172.21.15.67:0/675257162 shutdown_connections 2024-03-09T09:07:49.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7e06200640 1 -- 172.21.15.67:0/675257162 >> 172.21.15.67:0/675257162 conn(0x7f7e000ab320 msgr2=0x7f7e000ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:49.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7e06200640 1 -- 172.21.15.67:0/675257162 shutdown_connections 2024-03-09T09:07:49.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:49.913+0000 7f7e06200640 1 -- 172.21.15.67:0/675257162 wait complete. 2024-03-09T09:07:49.933 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:07:49.933 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:07:49.933 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:07:50.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.139+0000 7f1e53e00640 1 Processor -- start 2024-03-09T09:07:50.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.139+0000 7f1e53e00640 1 -- start start 2024-03-09T09:07:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.142+0000 7f1e53e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1e5414fd90 0x7f1e54156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.142+0000 7f1e53e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e54156bc0 con 0x7f1e5406c1d0 2024-03-09T09:07:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.142+0000 7f1e53e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e540c9890 con 0x7f1e540c9380 2024-03-09T09:07:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.142+0000 7f1e53e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e5406cd00 con 0x7f1e5414fd90 2024-03-09T09:07:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.142+0000 7f1e53400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1e5406c1d0 0x7f1e5406c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49992/0 (socket says 172.21.15.67:49992) 2024-03-09T09:07:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.142+0000 7f1e53400640 1 -- 172.21.15.67:0/4179359731 learned_addr learned my addr 172.21.15.67:0/4179359731 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:07:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.143+0000 7f1e52a00640 1 --2- 172.21.15.67:0/4179359731 >> v2:172.21.15.134:3300/0 conn(0x7f1e5414fd90 0x7f1e54156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.143+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080994447 0 0) 0x7f1e4c001490 con 0x7f1e5406c1d0 2024-03-09T09:07:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.143+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e38003520 con 0x7f1e5406c1d0 2024-03-09T09:07:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.143+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385290868 0 0) 0x7f1e540c9890 con 0x7f1e540c9380 2024-03-09T09:07:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e4c001490 con 0x7f1e540c9380 2024-03-09T09:07:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3509657649 0 0) 0x7f1e38003520 con 0x7f1e5406c1d0 2024-03-09T09:07:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 >> v2:172.21.15.134:3300/0 conn(0x7f1e5414fd90 msgr2=0x7f1e54156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 --2- 172.21.15.67:0/4179359731 >> v2:172.21.15.134:3300/0 conn(0x7f1e5414fd90 0x7f1e54156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 >> v1:172.21.15.134:6789/0 conn(0x7f1e540c9380 legacy=0x7f1e540acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540c5f80 con 0x7f1e5406c1d0 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f1e4c001bc0 con 0x7f1e5406c1d0 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 >> v1:172.21.15.67:6789/0 conn(0x7f1e5406c1d0 legacy=0x7f1e5406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 --2- 172.21.15.67:0/4179359731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e38003c00 0x7f1e38004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 --2- 172.21.15.67:0/4179359731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 0x7f1e38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 --2- 172.21.15.67:0/4179359731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e380052f0 0x7f1e3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5406cd00 con 0x7f1e38004800 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3800c230 con 0x7f1e38003c00 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3800c430 con 0x7f1e380052f0 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.144+0000 7f1e52000640 1 --2- 172.21.15.67:0/4179359731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 0x7f1e38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.145+0000 7f1e52a00640 1 --2- 172.21.15.67:0/4179359731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e38003c00 0x7f1e38004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.145+0000 7f1e53400640 1 --2- 172.21.15.67:0/4179359731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e380052f0 0x7f1e3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.145+0000 7f1e52000640 1 -- 172.21.15.67:0/4179359731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e380052f0 msgr2=0x7f1e3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.145+0000 7f1e52000640 1 --2- 172.21.15.67:0/4179359731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e380052f0 0x7f1e3800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.145+0000 7f1e52000640 1 -- 172.21.15.67:0/4179359731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e38003c00 msgr2=0x7f1e38004000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:07:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.145+0000 7f1e52000640 1 --2- 172.21.15.67:0/4179359731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e38003c00 0x7f1e38004000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.145+0000 7f1e52000640 1 -- 172.21.15.67:0/4179359731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e38003520 con 0x7f1e38004800 2024-03-09T09:07:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.145+0000 7f1e53400640 1 -- 172.21.15.67:0/4179359731 reap_dead start 2024-03-09T09:07:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.146+0000 7f1e52000640 1 --2- 172.21.15.67:0/4179359731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 0x7f1e38004c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e400018d0 tx=0x7f1e38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1b854541290d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.146+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40037070 con 0x7f1e38004800 2024-03-09T09:07:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.146+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e400052d0 con 0x7f1e38004800 2024-03-09T09:07:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.146+0000 7f1e51600640 1 -- 172.21.15.67:0/4179359731 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400014f0 con 0x7f1e38004800 2024-03-09T09:07:50.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.147+0000 7f1e53e00640 1 -- 172.21.15.67:0/4179359731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 msgr2=0x7f1e38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:50.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.147+0000 7f1e53e00640 1 --2- 172.21.15.67:0/4179359731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 0x7f1e38004c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e400018d0 tx=0x7f1e38005830 comp rx=0 tx=0).stop 2024-03-09T09:07:50.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.147+0000 7f1e53e00640 1 -- 172.21.15.67:0/4179359731 shutdown_connections 2024-03-09T09:07:50.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.147+0000 7f1e53e00640 1 --2- 172.21.15.67:0/4179359731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 0x7f1e38004c80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:50.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.147+0000 7f1e53e00640 1 -- 172.21.15.67:0/4179359731 >> 172.21.15.67:0/4179359731 conn(0x7f1e540cb480 msgr2=0x7f1e540cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:50.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.148+0000 7f1e53e00640 1 -- 172.21.15.67:0/4179359731 shutdown_connections 2024-03-09T09:07:50.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.148+0000 7f1e53e00640 1 -- 172.21.15.67:0/4179359731 wait complete. 2024-03-09T09:07:50.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.150+0000 7f1e53e00640 1 Processor -- start 2024-03-09T09:07:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.150+0000 7f1e53e00640 1 -- start start 2024-03-09T09:07:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.151+0000 7f1e53e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 0x7f1e54149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.151+0000 7f1e53e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150170 0x7f1e541579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.151+0000 7f1e53e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e54157f10 0x7f1e541eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.151+0000 7f1e53e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540c6520 con 0x7f1e38004800 2024-03-09T09:07:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.151+0000 7f1e53e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5406ba60 con 0x7f1e54157f10 2024-03-09T09:07:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.151+0000 7f1e53e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5406bcb0 con 0x7f1e54150170 2024-03-09T09:07:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.151+0000 7f1e52a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 0x7f1e54149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.151+0000 7f1e52a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 0x7f1e54149220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53858/0 (socket says 172.21.15.67:53858) 2024-03-09T09:07:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.151+0000 7f1e52a00640 1 -- 172.21.15.67:0/3912640351 learned_addr learned my addr 172.21.15.67:0/3912640351 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.151+0000 7f1e52000640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150170 0x7f1e541579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.151+0000 7f1e53400640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e54157f10 0x7f1e541eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e52a00640 1 -- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150170 msgr2=0x7f1e541579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e52a00640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150170 0x7f1e541579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e52a00640 1 -- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e54157f10 msgr2=0x7f1e541eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e52a00640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e54157f10 0x7f1e541eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e52a00640 1 -- 172.21.15.67:0/3912640351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e54158bb0 con 0x7f1e38004800 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e52000640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150170 0x7f1e541579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e53400640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e54157f10 0x7f1e541eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e52a00640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 0x7f1e54149220 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1e44011890 tx=0x7f1e44011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44019070 con 0x7f1e38004800 2024-03-09T09:07:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e44004240 con 0x7f1e38004800 2024-03-09T09:07:50.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400f820 con 0x7f1e38004800 2024-03-09T09:07:50.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e53e00640 1 -- 172.21.15.67:0/3912640351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e54149bd0 con 0x7f1e38004800 2024-03-09T09:07:50.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.152+0000 7f1e53e00640 1 -- 172.21.15.67:0/3912640351 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5414a2d0 con 0x7f1e38004800 2024-03-09T09:07:50.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.154+0000 7f1e53e00640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54149da0 0x7f1e541ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:50.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.154+0000 7f1e53e00640 1 -- 172.21.15.67:0/3912640351 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5404f3c0 con 0x7f1e54149da0 2024-03-09T09:07:50.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.154+0000 7f1e52000640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54149da0 0x7f1e541ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:50.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.155+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1e4401c070 con 0x7f1e38004800 2024-03-09T09:07:50.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.155+0000 7f1e52000640 1 -- 172.21.15.67:0/3912640351 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e5406d2d0 con 0x7f1e54149da0 2024-03-09T09:07:50.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.156+0000 7f1e4b400640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e3406b500 0x7f1e3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:50.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.156+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1e4408a930 con 0x7f1e38004800 2024-03-09T09:07:50.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.156+0000 7f1e52000640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54149da0 0x7f1e541ef1a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1e400018a0 tx=0x7f1e400012d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:50.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.156+0000 7f1e53400640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e3406b500 0x7f1e3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:50.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.157+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40041070 con 0x7f1e54149da0 2024-03-09T09:07:50.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.158+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f1e400035f0 con 0x7f1e54149da0 2024-03-09T09:07:50.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.158+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54149da0 msgr2=0x7f1e541ef1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:50.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.158+0000 7f1e4b400640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54149da0 0x7f1e541ef1a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1e400018a0 tx=0x7f1e400012d0 comp rx=0 tx=0).stop 2024-03-09T09:07:50.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.158+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54149da0 msgr2=0x7f1e541ef1a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:50.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.158+0000 7f1e4b400640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54149da0 0x7f1e541ef1a0 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1e400018a0 tx=0x7f1e400012d0 comp rx=0 tx=0).stop 2024-03-09T09:07:50.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.158+0000 7f1e53400640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e3406b500 0x7f1e3406d9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1e540c9760 tx=0x7f1e4c0018e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:50.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.172+0000 7f1e53e00640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150840 0x7f1e541eb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:07:50.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.172+0000 7f1e53e00640 1 -- 172.21.15.67:0/3912640351 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5414e4a0 con 0x7f1e54150840 2024-03-09T09:07:50.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.173+0000 7f1e52000640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150840 0x7f1e541eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:07:50.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.174+0000 7f1e52000640 1 -- 172.21.15.67:0/3912640351 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1e540746e0 con 0x7f1e54150840 2024-03-09T09:07:50.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.174+0000 7f1e52000640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150840 0x7f1e541eb740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005040 tx=0x7f1e40032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:07:50.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.175+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e40037040 con 0x7f1e54150840 2024-03-09T09:07:50.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.176+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f1e40003da0 con 0x7f1e54150840 2024-03-09T09:07:50.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.176+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150840 msgr2=0x7f1e541eb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:50.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.176+0000 7f1e4b400640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150840 0x7f1e541eb740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005040 tx=0x7f1e40032000 comp rx=0 tx=0).stop 2024-03-09T09:07:50.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.176+0000 7f1e4b400640 1 -- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150840 msgr2=0x7f1e541eb740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:07:50.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.176+0000 7f1e4b400640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e54150840 0x7f1e541eb740 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1e40005040 tx=0x7f1e40032000 comp rx=0 tx=0).stop 2024-03-09T09:07:50.180 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:07:50.180 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:07:50.180 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:07:50.180 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:07:50.180 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 110, 2024-03-09T09:07:50.180 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:07:50.180 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:07:50.180 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 18, 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:50.181 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:50.182 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:07:50.183 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:07:50.184 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:07:50.185 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:50.186 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:50.187 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:50.188 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:50.189 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:50.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:07:50.191 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:07:50.192 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:07:50.192 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:07:50.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:07:50.192 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:07:50.192 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:07:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.176+0000 7f1e53e00640 1 -- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e3406b500 msgr2=0x7f1e3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.176+0000 7f1e53e00640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e3406b500 0x7f1e3406d9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1e540c9760 tx=0x7f1e4c0018e0 comp rx=0 tx=0).stop 2024-03-09T09:07:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.176+0000 7f1e53e00640 1 -- 172.21.15.67:0/3912640351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 msgr2=0x7f1e54149220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:07:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.176+0000 7f1e53e00640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 0x7f1e54149220 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1e44011890 tx=0x7f1e44011d50 comp rx=0 tx=0).stop 2024-03-09T09:07:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.176+0000 7f1e53400640 1 -- 172.21.15.67:0/3912640351 reap_dead start 2024-03-09T09:07:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.176+0000 7f1e53e00640 1 -- 172.21.15.67:0/3912640351 shutdown_connections 2024-03-09T09:07:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.177+0000 7f1e53e00640 1 --2- 172.21.15.67:0/3912640351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e38004800 0x7f1e54149220 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:07:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.177+0000 7f1e53e00640 1 -- 172.21.15.67:0/3912640351 >> 172.21.15.67:0/3912640351 conn(0x7f1e540cb480 msgr2=0x7f1e5414dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:07:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.177+0000 7f1e53e00640 1 -- 172.21.15.67:0/3912640351 shutdown_connections 2024-03-09T09:07:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:07:50.177+0000 7f1e53e00640 1 -- 172.21.15.67:0/3912640351 wait complete. 2024-03-09T09:07:50.201 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:07:50.201 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T09:07:50.201 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-03-09T09:07:50.201 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T09:07:50.201 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:07:50.201 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:07:50.202 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:07:50.226 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:07:50.227 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-03-09T09:07:50.227 INFO:tasks.ceph.mon.b:Sent signal 19 2024-03-09T09:07:50.227 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-03-09T09:07:52.102 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 8% (55/622) Updating files: 9% (56/622) Updating files: 10% (63/622) Updating files: 11% (69/622) Updating files: 12% (75/622) Updating files: 13% (81/622) Updating files: 14% (88/622) Updating files: 15% (94/622) Updating files: 16% (100/622) Updating files: 17% (106/622) Updating files: 18% (112/622) Updating files: 19% (119/622) Updating files: 20% (125/622) Updating files: 21% (131/622) Updating files: 22% (137/622) Updating files: 23% (144/622) Updating files: 24% (150/622) Updating files: 25% (156/622) Updating files: 26% (162/622) Updating files: 27% (168/622) Updating files: 27% (170/622) Updating files: 28% (175/622) Updating files: 29% (181/622) Updating files: 30% (187/622) Updating files: 31% (193/622) Updating files: 32% (200/622) Updating files: 33% (206/622) Updating files: 34% (212/622) Updating files: 35% (218/622) Updating files: 36% (224/622) Updating files: 37% (231/622) Updating files: 38% (237/622) Updating files: 39% (243/622) Updating files: 40% (249/622) Updating files: 41% (256/622) Updating files: 42% (262/622) Updating files: 43% (268/622) Updating files: 44% (274/622) Updating files: 45% (280/622) Updating files: 46% (287/622) Updating files: 47% (293/622) Updating files: 48% (299/622) Updating files: 49% (305/622) Updating files: 50% (311/622) Updating files: 51% (318/622) Updating files: 52% (324/622) Updating files: 53% (330/622) Updating files: 54% (336/622) Updating files: 55% (343/622) Updating files: 56% (349/622) Updating files: 57% (355/622) Updating files: 58% (361/622) Updating files: 59% (367/622) Updating files: 60% (374/622) Updating files: 61% (380/622) Updating files: 62% (386/622) Updating files: 63% (392/622) Updating files: 64% (399/622) Updating files: 65% (405/622) Updating files: 66% (411/622) Updating files: 67% (417/622) Updating files: 68% (423/622) Updating files: 68% (427/622) Updating files: 69% (430/622) Updating files: 70% (436/622) Updating files: 71% (442/622) Updating files: 72% (448/622) Updating files: 73% (455/622) Updating files: 74% (461/622) Updating files: 75% (467/622) Updating files: 76% (473/622) Updating files: 77% (479/622) Updating files: 78% (486/622) Updating files: 79% (492/622) Updating files: 79% (497/622) Updating files: 80% (498/622) Updating files: 81% (504/622) Updating files: 82% (511/622) Updating files: 83% (517/622) Updating files: 84% (523/622) Updating files: 85% (529/622) Updating files: 85% (530/622) Updating files: 86% (535/622) Updating files: 87% (542/622) Updating files: 88% (548/622) Updating files: 89% (554/622) Updating files: 90% (560/622) Updating files: 91% (567/622) Updating files: 92% (573/622) Updating files: 93% (579/622) Updating files: 94% (585/622) Updating files: 95% (591/622) Updating files: 95% (593/622) Updating files: 96% (598/622) Updating files: 97% (604/622) Updating files: 98% (610/622) Updating files: 99% (616/622) Updating files: 100% (622/622) Updating files: 100% (622/622), done. 2024-03-09T09:07:52.756 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 3fa121bf0a8 v0.25 2024-03-09T09:07:52.770 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.25 2024-03-09T09:07:53.153 INFO:tasks.workunit.client.1.smithi134.stdout:v0.26 2024-03-09T09:07:53.153 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:07:53.153 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 26 -eq 48 ']' 2024-03-09T09:07:53.154 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.26 2024-03-09T09:07:53.154 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.26 2024-03-09T09:07:53.154 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.26 2024-03-09T09:07:54.917 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 1% (10/622) Updating files: 2% (13/622) Updating files: 3% (19/622) Updating files: 4% (25/622) Updating files: 5% (32/622) Updating files: 6% (38/622) Updating files: 7% (44/622) Updating files: 8% (50/622) Updating files: 9% (56/622) Updating files: 10% (63/622) Updating files: 11% (69/622) Updating files: 12% (75/622) Updating files: 13% (81/622) Updating files: 14% (88/622) Updating files: 15% (94/622) Updating files: 16% (100/622) Updating files: 17% (106/622) Updating files: 18% (112/622) Updating files: 19% (119/622) Updating files: 20% (125/622) Updating files: 21% (131/622) Updating files: 22% (137/622) Updating files: 23% (144/622) Updating files: 24% (150/622) Updating files: 25% (156/622) Updating files: 26% (162/622) Updating files: 27% (168/622) Updating files: 28% (175/622) Updating files: 29% (181/622) Updating files: 30% (187/622) Updating files: 30% (188/622) Updating files: 31% (193/622) Updating files: 32% (200/622) Updating files: 33% (206/622) Updating files: 34% (212/622) Updating files: 35% (218/622) Updating files: 36% (224/622) Updating files: 37% (231/622) Updating files: 38% (237/622) Updating files: 39% (243/622) Updating files: 40% (249/622) Updating files: 41% (256/622) Updating files: 42% (262/622) Updating files: 43% (268/622) Updating files: 44% (274/622) Updating files: 45% (280/622) Updating files: 46% (287/622) Updating files: 47% (293/622) Updating files: 48% (299/622) Updating files: 49% (305/622) Updating files: 50% (311/622) Updating files: 51% (318/622) Updating files: 52% (324/622) Updating files: 53% (330/622) Updating files: 54% (336/622) Updating files: 55% (343/622) Updating files: 56% (349/622) Updating files: 57% (355/622) Updating files: 58% (361/622) Updating files: 59% (367/622) Updating files: 60% (374/622) Updating files: 61% (380/622) Updating files: 62% (386/622) Updating files: 63% (392/622) Updating files: 64% (399/622) Updating files: 64% (403/622) Updating files: 65% (405/622) Updating files: 66% (411/622) Updating files: 67% (417/622) Updating files: 68% (423/622) Updating files: 69% (430/622) Updating files: 70% (436/622) Updating files: 71% (442/622) Updating files: 72% (448/622) Updating files: 73% (455/622) Updating files: 74% (461/622) Updating files: 75% (467/622) Updating files: 76% (473/622) Updating files: 77% (479/622) Updating files: 77% (483/622) Updating files: 78% (486/622) Updating files: 79% (492/622) Updating files: 80% (498/622) Updating files: 80% (499/622) Updating files: 81% (504/622) Updating files: 82% (511/622) Updating files: 83% (517/622) Updating files: 84% (523/622) Updating files: 85% (529/622) Updating files: 85% (530/622) Updating files: 86% (535/622) Updating files: 87% (542/622) Updating files: 88% (548/622) Updating files: 89% (554/622) Updating files: 90% (560/622) Updating files: 91% (567/622) Updating files: 92% (573/622) Updating files: 93% (579/622) Updating files: 94% (585/622) Updating files: 95% (591/622) Updating files: 96% (598/622) Updating files: 97% (604/622) Updating files: 98% (610/622) Updating files: 99% (616/622) Updating files: 99% (620/622) Updating files: 100% (622/622) Updating files: 100% (622/622), done. 2024-03-09T09:07:56.104 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 3fa121bf0a8 v0.25 2024-03-09T09:07:56.128 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.25 2024-03-09T09:07:56.259 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:07:56.260 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 26 -eq 48 ']' 2024-03-09T09:07:56.260 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.26 2024-03-09T09:07:56.260 INFO:tasks.workunit.client.0.smithi067.stdout:v0.26 2024-03-09T09:07:56.261 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.26 2024-03-09T09:07:56.261 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.26 2024-03-09T09:07:59.303 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 10% (26/239) Updating files: 11% (27/239) Updating files: 12% (29/239) Updating files: 13% (32/239) Updating files: 14% (34/239) Updating files: 15% (36/239) Updating files: 16% (39/239) Updating files: 17% (41/239) Updating files: 18% (44/239) Updating files: 19% (46/239) Updating files: 20% (48/239) Updating files: 21% (51/239) Updating files: 22% (53/239) Updating files: 23% (55/239) Updating files: 24% (58/239) Updating files: 25% (60/239) Updating files: 26% (63/239) Updating files: 27% (65/239) Updating files: 28% (67/239) Updating files: 29% (70/239) Updating files: 30% (72/239) Updating files: 30% (73/239) Updating files: 31% (75/239) Updating files: 32% (77/239) Updating files: 33% (79/239) Updating files: 34% (82/239) Updating files: 35% (84/239) Updating files: 36% (87/239) Updating files: 37% (89/239) Updating files: 38% (91/239) Updating files: 39% (94/239) Updating files: 40% (96/239) Updating files: 41% (98/239) Updating files: 42% (101/239) Updating files: 43% (103/239) Updating files: 44% (106/239) Updating files: 45% (108/239) Updating files: 46% (110/239) Updating files: 47% (113/239) Updating files: 48% (115/239) Updating files: 49% (118/239) Updating files: 50% (120/239) Updating files: 51% (122/239) Updating files: 52% (125/239) Updating files: 53% (127/239) Updating files: 54% (130/239) Updating files: 55% (132/239) Updating files: 56% (134/239) Updating files: 57% (137/239) Updating files: 58% (139/239) Updating files: 59% (142/239) Updating files: 60% (144/239) Updating files: 61% (146/239) Updating files: 62% (149/239) Updating files: 63% (151/239) Updating files: 64% (153/239) Updating files: 65% (156/239) Updating files: 66% (158/239) Updating files: 67% (161/239) Updating files: 68% (163/239) Updating files: 69% (165/239) Updating files: 70% (168/239) Updating files: 71% (170/239) Updating files: 71% (171/239) Updating files: 72% (173/239) Updating files: 73% (175/239) Updating files: 74% (177/239) Updating files: 75% (180/239) Updating files: 76% (182/239) Updating files: 77% (185/239) Updating files: 78% (187/239) Updating files: 79% (189/239) Updating files: 80% (192/239) Updating files: 81% (194/239) Updating files: 82% (196/239) Updating files: 83% (199/239) Updating files: 84% (201/239) Updating files: 85% (204/239) Updating files: 86% (206/239) Updating files: 87% (208/239) Updating files: 88% (211/239) Updating files: 89% (213/239) Updating files: 90% (216/239) Updating files: 91% (218/239) Updating files: 92% (220/239) Updating files: 93% (223/239) Updating files: 94% (225/239) Updating files: 95% (228/239) Updating files: 96% (230/239) Updating files: 97% (232/239) Updating files: 98% (235/239) Updating files: 99% (237/239) Updating files: 100% (239/239) Updating files: 100% (239/239), done. 2024-03-09T09:07:59.381 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 9981ff90968 v0.26 2024-03-09T09:07:59.388 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.26 2024-03-09T09:07:59.501 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:07:59.501 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 27 -eq 48 ']' 2024-03-09T09:07:59.501 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.27 2024-03-09T09:07:59.501 INFO:tasks.workunit.client.1.smithi134.stdout:v0.27 2024-03-09T09:07:59.502 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.27 2024-03-09T09:07:59.502 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.27 2024-03-09T09:08:00.081 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 42% (102/239) Updating files: 43% (103/239) Updating files: 44% (106/239) Updating files: 45% (108/239) Updating files: 46% (110/239) Updating files: 47% (113/239) Updating files: 48% (115/239) Updating files: 49% (118/239) Updating files: 50% (120/239) Updating files: 51% (122/239) Updating files: 52% (125/239) Updating files: 53% (127/239) Updating files: 54% (130/239) Updating files: 55% (132/239) Updating files: 56% (134/239) Updating files: 57% (137/239) Updating files: 58% (139/239) Updating files: 59% (142/239) Updating files: 60% (144/239) Updating files: 61% (146/239) Updating files: 62% (149/239) Updating files: 63% (151/239) Updating files: 64% (153/239) Updating files: 65% (156/239) Updating files: 66% (158/239) Updating files: 67% (161/239) Updating files: 68% (163/239) Updating files: 69% (165/239) Updating files: 70% (168/239) Updating files: 71% (170/239) Updating files: 72% (173/239) Updating files: 72% (174/239) Updating files: 73% (175/239) Updating files: 74% (177/239) Updating files: 75% (180/239) Updating files: 76% (182/239) Updating files: 77% (185/239) Updating files: 78% (187/239) Updating files: 79% (189/239) Updating files: 80% (192/239) Updating files: 81% (194/239) Updating files: 82% (196/239) Updating files: 83% (199/239) Updating files: 84% (201/239) Updating files: 85% (204/239) Updating files: 86% (206/239) Updating files: 87% (208/239) Updating files: 88% (211/239) Updating files: 89% (213/239) Updating files: 90% (216/239) Updating files: 91% (218/239) Updating files: 92% (220/239) Updating files: 93% (223/239) Updating files: 94% (225/239) Updating files: 95% (228/239) Updating files: 96% (230/239) Updating files: 97% (232/239) Updating files: 98% (235/239) Updating files: 99% (237/239) Updating files: 100% (239/239) Updating files: 100% (239/239), done. 2024-03-09T09:08:00.135 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 9981ff90968 v0.26 2024-03-09T09:08:00.151 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.26 2024-03-09T09:08:00.883 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:08:00.883 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 27 -eq 48 ']' 2024-03-09T09:08:00.883 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.27 2024-03-09T09:08:00.883 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.27 2024-03-09T09:08:00.883 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.27 2024-03-09T09:08:00.883 INFO:tasks.workunit.client.0.smithi067.stdout:v0.27 2024-03-09T09:08:04.658 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 38% (97/253) Updating files: 39% (99/253) Updating files: 40% (102/253) Updating files: 41% (104/253) Updating files: 42% (107/253) Updating files: 43% (109/253) Updating files: 44% (112/253) Updating files: 45% (114/253) Updating files: 46% (117/253) Updating files: 47% (119/253) Updating files: 48% (122/253) Updating files: 49% (124/253) Updating files: 50% (127/253) Updating files: 51% (130/253) Updating files: 52% (132/253) Updating files: 53% (135/253) Updating files: 54% (137/253) Updating files: 55% (140/253) Updating files: 56% (142/253) Updating files: 57% (145/253) Updating files: 58% (147/253) Updating files: 59% (150/253) Updating files: 60% (152/253) Updating files: 60% (153/253) Updating files: 61% (155/253) Updating files: 62% (157/253) Updating files: 63% (160/253) Updating files: 64% (162/253) Updating files: 65% (165/253) Updating files: 66% (167/253) Updating files: 67% (170/253) Updating files: 68% (173/253) Updating files: 69% (175/253) Updating files: 70% (178/253) Updating files: 71% (180/253) Updating files: 72% (183/253) Updating files: 73% (185/253) Updating files: 74% (188/253) Updating files: 75% (190/253) Updating files: 76% (193/253) Updating files: 77% (195/253) Updating files: 78% (198/253) Updating files: 79% (200/253) Updating files: 80% (203/253) Updating files: 81% (205/253) Updating files: 82% (208/253) Updating files: 83% (210/253) Updating files: 84% (213/253) Updating files: 85% (216/253) Updating files: 86% (218/253) Updating files: 87% (221/253) Updating files: 88% (223/253) Updating files: 89% (226/253) Updating files: 90% (228/253) Updating files: 91% (231/253) Updating files: 92% (233/253) Updating files: 93% (236/253) Updating files: 94% (238/253) Updating files: 95% (241/253) Updating files: 96% (243/253) Updating files: 96% (244/253) Updating files: 97% (246/253) Updating files: 98% (248/253) Updating files: 99% (251/253) Updating files: 100% (253/253) Updating files: 100% (253/253), done. 2024-03-09T09:08:04.818 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 793034c62c8 v0.27 2024-03-09T09:08:04.832 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.27 2024-03-09T09:08:04.973 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:08:04.973 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 28 -eq 48 ']' 2024-03-09T09:08:04.973 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.28 2024-03-09T09:08:04.973 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.28 2024-03-09T09:08:04.973 INFO:tasks.workunit.client.0.smithi067.stdout:v0.28 2024-03-09T09:08:04.973 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.28 2024-03-09T09:08:05.228 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-03-09T09:08:05.228 INFO:tasks.ceph.mon.b:Sent signal 18 2024-03-09T09:08:05.229 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:08:05.229 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:08:05.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.437+0000 7fb9daa00640 1 Processor -- start 2024-03-09T09:08:05.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.438+0000 7fb9daa00640 1 -- start start 2024-03-09T09:08:05.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9daa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:05.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9daa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:05.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9daa00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d406cd30 con 0x7fb9d4072770 2024-03-09T09:08:05.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9daa00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d406b200 con 0x7fb9d40bebc0 2024-03-09T09:08:05.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9daa00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d40c5eb0 con 0x7fb9d406bda0 2024-03-09T09:08:05.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9da000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:05.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9da000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44332/0 (socket says 172.21.15.67:44332) 2024-03-09T09:08:05.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9da000640 1 -- 172.21.15.67:0/3377968950 learned_addr learned my addr 172.21.15.67:0/3377968950 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:08:05.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9d9600640 1 --1- 172.21.15.67:0/3377968950 >> v1:172.21.15.67:6789/0 conn(0x7fb9d4072770 0x7fb9d406b690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42348/0 (socket says 172.21.15.67:42348) 2024-03-09T09:08:05.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9da000640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 msgr2=0x7fb9d4156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:08:05.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9da000640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 msgr2=0x7fb9d4156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:05.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.439+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:05.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.440+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:05.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.440+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:05.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.440+0000 7fb9d8c00640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 msgr2=0x7fb9d4072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:08:05.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.440+0000 7fb9d8c00640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 msgr2=0x7fb9d4072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:05.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.440+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:05.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.440+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.641+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.641+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.642+0000 7fb9da000640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 msgr2=0x7fb9d4156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:08:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.642+0000 7fb9da000640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 msgr2=0x7fb9d4156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.642+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.642+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:08:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.642+0000 7fb9d8c00640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 msgr2=0x7fb9d4072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:08:05.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.642+0000 7fb9d8c00640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 msgr2=0x7fb9d4072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:05.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.642+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:05.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:05.642+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:08:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.043+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.043+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.043+0000 7fb9da000640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 msgr2=0x7fb9d4156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:08:06.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.043+0000 7fb9da000640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 msgr2=0x7fb9d4156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.043+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.043+0000 7fb9d8c00640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 msgr2=0x7fb9d4072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:08:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.043+0000 7fb9d8c00640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 msgr2=0x7fb9d4072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.043+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.043+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:08:06.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.043+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:08:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.845+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.845+0000 7fb9d8c00640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 msgr2=0x7fb9d4072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:08:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.845+0000 7fb9d8c00640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 msgr2=0x7fb9d4072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.845+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.845+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.845+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:08:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.846+0000 7fb9da000640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 msgr2=0x7fb9d4156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:08:06.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.846+0000 7fb9da000640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 msgr2=0x7fb9d4156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:06.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.846+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:06.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:06.846+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:08:08.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.439+0000 7fb9d2a00640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 msgr2=0x7fb9d4072b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:08:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.439+0000 7fb9d2a00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.134:3300/0 conn(0x7fb9d406bda0 0x7fb9d4072b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.439+0000 7fb9d2a00640 1 -- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 msgr2=0x7fb9d4156650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:08:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.439+0000 7fb9d2a00640 1 --2- 172.21.15.67:0/3377968950 >> v2:172.21.15.67:3300/0 conn(0x7fb9d40bebc0 0x7fb9d4156650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.439+0000 7fb9d2a00640 1 -- 172.21.15.67:0/3377968950 >> v1:172.21.15.67:6789/0 conn(0x7fb9d4072770 legacy=0x7fb9d406b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.439+0000 7fb9d2a00640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4003670 0x7fb9b4005ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.439+0000 7fb9d2a00640 1 -- 172.21.15.67:0/3377968950 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d40c6150 con 0x7fb9b40061d0 2024-03-09T09:08:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.440+0000 7fb9d2a00640 1 -- 172.21.15.67:0/3377968950 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d406b200 con 0x7fb9b4003670 2024-03-09T09:08:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.440+0000 7fb9d9600640 1 -- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4003670 msgr2=0x7fb9b4005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:08:08.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.440+0000 7fb9d9600640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4003670 0x7fb9b4005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:08.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.641+0000 7fb9d9600640 1 -- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4003670 msgr2=0x7fb9b4005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:08:08.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:08.641+0000 7fb9d9600640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4003670 0x7fb9b4005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:08:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:09.042+0000 7fb9d9600640 1 -- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4003670 msgr2=0x7fb9b4005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:08:09.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:09.042+0000 7fb9d9600640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4003670 0x7fb9b4005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:08:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:09.843+0000 7fb9d9600640 1 -- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4003670 msgr2=0x7fb9b4005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:08:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:09.843+0000 7fb9d9600640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4003670 0x7fb9b4005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:08:10.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.245+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764517768 0 0) 0x7fb9d40c6150 con 0x7fb9b40061d0 2024-03-09T09:08:10.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.246+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9bc003520 con 0x7fb9b40061d0 2024-03-09T09:08:10.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3082709069 0 0) 0x7fb9bc003520 con 0x7fb9b40061d0 2024-03-09T09:08:10.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4003670 msgr2=0x7fb9b4005ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:08:10.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9b4003670 0x7fb9b4005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b400afd0 con 0x7fb9b40061d0 2024-03-09T09:08:10.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb9c000d580 con 0x7fb9b40061d0 2024-03-09T09:08:10.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 >> v1:172.21.15.134:6789/0 conn(0x7fb9b40061d0 legacy=0x7fb9b4008690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9da000640 1 -- 172.21.15.67:0/3377968950 reap_dead start 2024-03-09T09:08:10.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc003a20 0x7fb9bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 0x7fb9bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9bc0053c0 0x7fb9bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d406b200 con 0x7fb9bc003a20 2024-03-09T09:08:10.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc00c280 con 0x7fb9bc0053c0 2024-03-09T09:08:10.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.247+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc00c480 con 0x7fb9bc004800 2024-03-09T09:08:10.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.248+0000 7fb9d8c00640 1 -- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 msgr2=0x7fb9bc004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:08:10.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.248+0000 7fb9d8c00640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 0x7fb9bc004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:10.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.248+0000 7fb9d9600640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc003a20 0x7fb9bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.248+0000 7fb9da000640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9bc0053c0 0x7fb9bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.249+0000 7fb9d9600640 1 -- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 msgr2=0x7fb9bc004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:08:10.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.249+0000 7fb9d9600640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 0x7fb9bc004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.249+0000 7fb9d9600640 1 -- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9bc0053c0 msgr2=0x7fb9bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:10.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.249+0000 7fb9d9600640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9bc0053c0 0x7fb9bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.249+0000 7fb9d9600640 1 -- 172.21.15.67:0/3377968950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc003520 con 0x7fb9bc003a20 2024-03-09T09:08:10.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.252+0000 7fb9d9600640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc003a20 0x7fb9bc003e20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8001b20 tx=0x7fb9bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deee01a8bba96b03 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.262+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8001720 con 0x7fb9bc003a20 2024-03-09T09:08:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.262+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c8007290 con 0x7fb9bc003a20 2024-03-09T09:08:10.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.263+0000 7fb9d3e00640 1 -- 172.21.15.67:0/3377968950 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8007680 con 0x7fb9bc003a20 2024-03-09T09:08:10.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.265+0000 7fb9daa00640 1 -- 172.21.15.67:0/3377968950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc003a20 msgr2=0x7fb9bc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.265+0000 7fb9daa00640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc003a20 0x7fb9bc003e20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8001b20 tx=0x7fb9bc005830 comp rx=0 tx=0).stop 2024-03-09T09:08:10.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.265+0000 7fb9daa00640 1 -- 172.21.15.67:0/3377968950 shutdown_connections 2024-03-09T09:08:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.265+0000 7fb9daa00640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9bc0053c0 0x7fb9bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.265+0000 7fb9daa00640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 0x7fb9bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.265+0000 7fb9daa00640 1 --2- 172.21.15.67:0/3377968950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc003a20 0x7fb9bc003e20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.265+0000 7fb9daa00640 1 -- 172.21.15.67:0/3377968950 >> 172.21.15.67:0/3377968950 conn(0x7fb9d40c7fb0 msgr2=0x7fb9d40c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.265+0000 7fb9daa00640 1 -- 172.21.15.67:0/3377968950 shutdown_connections 2024-03-09T09:08:10.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.266+0000 7fb9daa00640 1 -- 172.21.15.67:0/3377968950 wait complete. 2024-03-09T09:08:10.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.272+0000 7fb9daa00640 1 Processor -- start 2024-03-09T09:08:10.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.273+0000 7fb9daa00640 1 -- start start 2024-03-09T09:08:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.275+0000 7fb9daa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9bc003a20 0x7fb9d40c1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.275+0000 7fb9daa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 0x7fb9d41ec370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.275+0000 7fb9daa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc0053c0 0x7fb9d414d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.276+0000 7fb9daa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40c9430 con 0x7fb9bc0053c0 2024-03-09T09:08:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.276+0000 7fb9daa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40c6710 con 0x7fb9bc003a20 2024-03-09T09:08:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.276+0000 7fb9daa00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40add50 con 0x7fb9bc004800 2024-03-09T09:08:10.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.276+0000 7fb9d8c00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 msgr2=0x7fb9d41ec370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:08:10.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.276+0000 7fb9d8c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 0x7fb9d41ec370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:10.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.276+0000 7fb9da000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc0053c0 0x7fb9d414d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.276+0000 7fb9da000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc0053c0 0x7fb9d414d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34004/0 (socket says 172.21.15.67:34004) 2024-03-09T09:08:10.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.276+0000 7fb9da000640 1 -- 172.21.15.67:0/1211347037 learned_addr learned my addr 172.21.15.67:0/1211347037 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:08:10.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.276+0000 7fb9d9600640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9bc003a20 0x7fb9d40c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.277+0000 7fb9da000640 1 -- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 msgr2=0x7fb9d41ec370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:08:10.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.277+0000 7fb9da000640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 0x7fb9d41ec370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.277+0000 7fb9da000640 1 -- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9bc003a20 msgr2=0x7fb9d40c1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.277+0000 7fb9da000640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9bc003a20 0x7fb9d40c1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.277+0000 7fb9da000640 1 -- 172.21.15.67:0/1211347037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d40c0bd0 con 0x7fb9bc0053c0 2024-03-09T09:08:10.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.278+0000 7fb9da000640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc0053c0 0x7fb9d414d7f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40c5a10 tx=0x7fb9cc015d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:10.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.278+0000 7fb9d1600640 1 -- 172.21.15.67:0/1211347037 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc01c070 con 0x7fb9bc0053c0 2024-03-09T09:08:10.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.279+0000 7fb9d1600640 1 -- 172.21.15.67:0/1211347037 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9cc003ba0 con 0x7fb9bc0053c0 2024-03-09T09:08:10.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.279+0000 7fb9d1600640 1 -- 172.21.15.67:0/1211347037 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc013860 con 0x7fb9bc0053c0 2024-03-09T09:08:10.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.279+0000 7fb9daa00640 1 -- 172.21.15.67:0/1211347037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d40ad790 con 0x7fb9bc0053c0 2024-03-09T09:08:10.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.279+0000 7fb9daa00640 1 -- 172.21.15.67:0/1211347037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d40ad110 con 0x7fb9bc0053c0 2024-03-09T09:08:10.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.280+0000 7fb9daa00640 1 -- 172.21.15.67:0/1211347037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d40c09e0 con 0x7fb9bc0053c0 2024-03-09T09:08:10.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.282+0000 7fb9d1600640 1 -- 172.21.15.67:0/1211347037 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb9cc01f070 con 0x7fb9bc0053c0 2024-03-09T09:08:10.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.283+0000 7fb9d1600640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb9b406be00 0x7fb9b406e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.284+0000 7fb9d1600640 1 -- 172.21.15.67:0/1211347037 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb9cc08b530 con 0x7fb9bc0053c0 2024-03-09T09:08:10.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.284+0000 7fb9d9600640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb9b406be00 0x7fb9b406e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.285+0000 7fb9d9600640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb9b406be00 0x7fb9b406e2c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8034000 tx=0x7fb9c8039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:10.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.289+0000 7fb9d1600640 1 -- 172.21.15.67:0/1211347037 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb9cc026020 con 0x7fb9bc0053c0 2024-03-09T09:08:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.586+0000 7fb9daa00640 1 -- 172.21.15.67:0/1211347037 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb9d404f3c0 con 0x7fb9bc0053c0 2024-03-09T09:08:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.586+0000 7fb9d1600640 1 -- 172.21.15.67:0/1211347037 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fb9cc05a050 con 0x7fb9bc0053c0 2024-03-09T09:08:10.600 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":112,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:08:10.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 -- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb9b406be00 msgr2=0x7fb9b406e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb9b406be00 0x7fb9b406e2c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8034000 tx=0x7fb9c8039040 comp rx=0 tx=0).stop 2024-03-09T09:08:10.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 -- 172.21.15.67:0/1211347037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc0053c0 msgr2=0x7fb9d414d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc0053c0 0x7fb9d414d7f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40c5a10 tx=0x7fb9cc015d90 comp rx=0 tx=0).stop 2024-03-09T09:08:10.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 -- 172.21.15.67:0/1211347037 shutdown_connections 2024-03-09T09:08:10.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb9b406be00 0x7fb9b406e2c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9bc0053c0 0x7fb9d414d7f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9bc004800 0x7fb9d41ec370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 --2- 172.21.15.67:0/1211347037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9bc003a20 0x7fb9d40c1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 -- 172.21.15.67:0/1211347037 >> 172.21.15.67:0/1211347037 conn(0x7fb9d40c7fb0 msgr2=0x7fb9d40c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 -- 172.21.15.67:0/1211347037 shutdown_connections 2024-03-09T09:08:10.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.590+0000 7fb9daa00640 1 -- 172.21.15.67:0/1211347037 wait complete. 2024-03-09T09:08:10.619 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:08:10.619 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:08:10.619 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:08:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.817+0000 7fcc63c00640 1 Processor -- start 2024-03-09T09:08:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.817+0000 7fcc63c00640 1 -- start start 2024-03-09T09:08:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.819+0000 7fcc63c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcc5c073af0 0x7fcc5c0c2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.819+0000 7fcc63c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5c0c3c30 con 0x7fcc5c1594b0 2024-03-09T09:08:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.819+0000 7fcc63c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5c06f500 con 0x7fcc5c0c56c0 2024-03-09T09:08:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.819+0000 7fcc63c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5c06f2b0 con 0x7fcc5c073af0 2024-03-09T09:08:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.819+0000 7fcc61c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcc5c1594b0 0x7fcc5c159920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49978/0 (socket says 172.21.15.67:49978) 2024-03-09T09:08:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.819+0000 7fcc61c00640 1 -- 172.21.15.67:0/1166867990 learned_addr learned my addr 172.21.15.67:0/1166867990 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:08:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.819+0000 7fcc5be00640 1 --2- 172.21.15.67:0/1166867990 >> v2:172.21.15.134:3300/0 conn(0x7fcc5c073af0 0x7fcc5c0c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369578153 0 0) 0x7fcc5c0c3c30 con 0x7fcc5c1594b0 2024-03-09T09:08:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc44003520 con 0x7fcc5c1594b0 2024-03-09T09:08:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1922465396 0 0) 0x7fcc44003520 con 0x7fcc5c1594b0 2024-03-09T09:08:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 >> v2:172.21.15.134:3300/0 conn(0x7fcc5c073af0 msgr2=0x7fcc5c0c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:10.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 --2- 172.21.15.67:0/1166867990 >> v2:172.21.15.134:3300/0 conn(0x7fcc5c073af0 0x7fcc5c0c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 >> v1:172.21.15.134:6789/0 conn(0x7fcc5c0c56c0 legacy=0x7fcc5c1532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c06f7e0 con 0x7fcc5c1594b0 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fcc540019f0 con 0x7fcc5c1594b0 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 >> v1:172.21.15.67:6789/0 conn(0x7fcc5c1594b0 legacy=0x7fcc5c159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 --2- 172.21.15.67:0/1166867990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc440039b0 0x7fcc44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 --2- 172.21.15.67:0/1166867990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc44004800 0x7fcc44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 --2- 172.21.15.67:0/1166867990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440051c0 0x7fcc4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c06f2b0 con 0x7fcc44004800 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4400c0c0 con 0x7fcc440039b0 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.820+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4400c2c0 con 0x7fcc440051c0 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.821+0000 7fcc61c00640 1 -- 172.21.15.67:0/1166867990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440051c0 msgr2=0x7fcc4400b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.821+0000 7fcc5be00640 1 --2- 172.21.15.67:0/1166867990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc44004800 0x7fcc44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.821+0000 7fcc61c00640 1 --2- 172.21.15.67:0/1166867990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440051c0 0x7fcc4400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.821+0000 7fcc61200640 1 --2- 172.21.15.67:0/1166867990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc440039b0 0x7fcc44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.821+0000 7fcc5be00640 1 -- 172.21.15.67:0/1166867990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440051c0 msgr2=0x7fcc4400b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:08:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.821+0000 7fcc5be00640 1 --2- 172.21.15.67:0/1166867990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc440051c0 0x7fcc4400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.821+0000 7fcc5be00640 1 -- 172.21.15.67:0/1166867990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc440039b0 msgr2=0x7fcc44003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.821+0000 7fcc5be00640 1 --2- 172.21.15.67:0/1166867990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc440039b0 0x7fcc44003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.821+0000 7fcc61c00640 1 -- 172.21.15.67:0/1166867990 reap_dead start 2024-03-09T09:08:10.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.821+0000 7fcc5be00640 1 -- 172.21.15.67:0/1166867990 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc44003520 con 0x7fcc44004800 2024-03-09T09:08:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.822+0000 7fcc5be00640 1 --2- 172.21.15.67:0/1166867990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc44004800 0x7fcc44004c80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009ac0 tx=0x7fcc44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c1bdc2041c6d34 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.822+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48039070 con 0x7fcc44004800 2024-03-09T09:08:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.823+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4802c6b0 con 0x7fcc44004800 2024-03-09T09:08:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.823+0000 7fcc5b400640 1 -- 172.21.15.67:0/1166867990 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48009690 con 0x7fcc44004800 2024-03-09T09:08:10.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.823+0000 7fcc63c00640 1 -- 172.21.15.67:0/1166867990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc44004800 msgr2=0x7fcc44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.823+0000 7fcc63c00640 1 --2- 172.21.15.67:0/1166867990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc44004800 0x7fcc44004c80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009ac0 tx=0x7fcc44005830 comp rx=0 tx=0).stop 2024-03-09T09:08:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.823+0000 7fcc63c00640 1 -- 172.21.15.67:0/1166867990 shutdown_connections 2024-03-09T09:08:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.823+0000 7fcc63c00640 1 --2- 172.21.15.67:0/1166867990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc44004800 0x7fcc44004c80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.823+0000 7fcc63c00640 1 -- 172.21.15.67:0/1166867990 >> 172.21.15.67:0/1166867990 conn(0x7fcc5c06c540 msgr2=0x7fcc5c06c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.823+0000 7fcc63c00640 1 -- 172.21.15.67:0/1166867990 shutdown_connections 2024-03-09T09:08:10.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.823+0000 7fcc63c00640 1 -- 172.21.15.67:0/1166867990 wait complete. 2024-03-09T09:08:10.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.825+0000 7fcc63c00640 1 Processor -- start 2024-03-09T09:08:10.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.825+0000 7fcc63c00640 1 -- start start 2024-03-09T09:08:10.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.826+0000 7fcc63c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc44004800 0x7fcc5c14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.826+0000 7fcc63c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1ee880 0x7fcc5c155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.826+0000 7fcc63c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c1eb990 0x7fcc5c1ebe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.826+0000 7fcc63c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0c5aa0 con 0x7fcc5c1ee880 2024-03-09T09:08:10.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.826+0000 7fcc63c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c06fda0 con 0x7fcc44004800 2024-03-09T09:08:10.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.826+0000 7fcc63c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0c5c20 con 0x7fcc5c1eb990 2024-03-09T09:08:10.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.826+0000 7fcc61c00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c1eb990 msgr2=0x7fcc5c1ebe40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:08:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.826+0000 7fcc61c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c1eb990 0x7fcc5c1ebe40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.826+0000 7fcc5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1ee880 0x7fcc5c155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.826+0000 7fcc5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1ee880 0x7fcc5c155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34022/0 (socket says 172.21.15.67:34022) 2024-03-09T09:08:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.826+0000 7fcc5be00640 1 -- 172.21.15.67:0/2692479694 learned_addr learned my addr 172.21.15.67:0/2692479694 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:08:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.827+0000 7fcc61200640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc44004800 0x7fcc5c14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.827+0000 7fcc5be00640 1 -- 172.21.15.67:0/2692479694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c1eb990 msgr2=0x7fcc5c1ebe40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:08:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.827+0000 7fcc5be00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc5c1eb990 0x7fcc5c1ebe40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.827+0000 7fcc5be00640 1 -- 172.21.15.67:0/2692479694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc44004800 msgr2=0x7fcc5c14a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.827+0000 7fcc5be00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc44004800 0x7fcc5c14a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.827+0000 7fcc5be00640 1 -- 172.21.15.67:0/2692479694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c0737d0 con 0x7fcc5c1ee880 2024-03-09T09:08:10.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.827+0000 7fcc5be00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1ee880 0x7fcc5c155060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009be0 tx=0x7fcc48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.828+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48039050 con 0x7fcc5c1ee880 2024-03-09T09:08:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.828+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48045070 con 0x7fcc5c1ee880 2024-03-09T09:08:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.828+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48003a50 con 0x7fcc5c1ee880 2024-03-09T09:08:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.828+0000 7fcc63c00640 1 -- 172.21.15.67:0/2692479694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c073210 con 0x7fcc5c1ee880 2024-03-09T09:08:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.828+0000 7fcc63c00640 1 -- 172.21.15.67:0/2692479694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5c14ae20 con 0x7fcc5c1ee880 2024-03-09T09:08:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.829+0000 7fcc63c00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1eb500 0x7fcc5c1e8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.829+0000 7fcc63c00640 1 -- 172.21.15.67:0/2692479694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c1eb2d0 con 0x7fcc5c1eb500 2024-03-09T09:08:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.829+0000 7fcc61200640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1eb500 0x7fcc5c1e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.829+0000 7fcc61200640 1 -- 172.21.15.67:0/2692479694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc5c072fc0 con 0x7fcc5c1eb500 2024-03-09T09:08:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.830+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcc48003bf0 con 0x7fcc5c1ee880 2024-03-09T09:08:10.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.830+0000 7fcc61200640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1eb500 0x7fcc5c1e8570 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c06ecc0 tx=0x7fcc5000f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:10.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.831+0000 7fcc58c00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc3c06b4b0 0x7fcc3c06d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.831+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcc480ac200 con 0x7fcc5c1ee880 2024-03-09T09:08:10.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.832+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50017020 con 0x7fcc5c1eb500 2024-03-09T09:08:10.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.832+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fcc500053a0 con 0x7fcc5c1eb500 2024-03-09T09:08:10.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.832+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1eb500 msgr2=0x7fcc5c1e8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.832+0000 7fcc58c00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1eb500 0x7fcc5c1e8570 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c06ecc0 tx=0x7fcc5000f9d0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.832+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1eb500 msgr2=0x7fcc5c1e8570 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:08:10.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.832+0000 7fcc58c00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1eb500 0x7fcc5c1e8570 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c06ecc0 tx=0x7fcc5000f9d0 comp rx=0 tx=0).stop 2024-03-09T09:08:10.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.832+0000 7fcc61c00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc3c06b4b0 0x7fcc3c06d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.833+0000 7fcc61c00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc3c06b4b0 0x7fcc3c06d970 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c070eb0 tx=0x7fcc54001480 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:10.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.839+0000 7fcc63c00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1e8ab0 0x7fcc5c153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:10.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.839+0000 7fcc63c00640 1 -- 172.21.15.67:0/2692479694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c075a00 con 0x7fcc5c1e8ab0 2024-03-09T09:08:10.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.839+0000 7fcc61200640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1e8ab0 0x7fcc5c153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:10.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.839+0000 7fcc61200640 1 -- 172.21.15.67:0/2692479694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcc5c06ef00 con 0x7fcc5c1e8ab0 2024-03-09T09:08:10.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.840+0000 7fcc61200640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1e8ab0 0x7fcc5c153db0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcc50018000 tx=0x7fcc50001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:10.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.840+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50017030 con 0x7fcc5c1e8ab0 2024-03-09T09:08:10.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.840+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7fcc50003d70 con 0x7fcc5c1e8ab0 2024-03-09T09:08:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.840+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1e8ab0 msgr2=0x7fcc5c153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.840+0000 7fcc58c00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1e8ab0 0x7fcc5c153db0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcc50018000 tx=0x7fcc50001230 comp rx=0 tx=0).stop 2024-03-09T09:08:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.840+0000 7fcc58c00640 1 -- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1e8ab0 msgr2=0x7fcc5c153db0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:08:10.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.840+0000 7fcc58c00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1e8ab0 0x7fcc5c153db0 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcc50018000 tx=0x7fcc50001230 comp rx=0 tx=0).stop 2024-03-09T09:08:10.845 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:08:10.845 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:08:10.845 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:08:10.846 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:08:10.846 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 112, 2024-03-09T09:08:10.846 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:08:10.846 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:08:10.846 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:08:10.846 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:10.846 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:10.847 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:10.848 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:08:10.849 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:08:10.850 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:08:10.850 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:08:10.850 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:08:10.850 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:08:10.851 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:08:10.851 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:08:10.851 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:08:10.851 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:08:10.851 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:10.851 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:10.851 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:10.851 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:10.851 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:10.851 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:10.852 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:10.852 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:10.852 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:10.852 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:10.852 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:10.852 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:10.852 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:08:10.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.852 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:08:10.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.853 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:08:10.853 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:08:10.853 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:10.853 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:10.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:10.853 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:08:10.854 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:10.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.854 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:10.854 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:08:10.854 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:10.854 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:10.854 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:10.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.855 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:08:10.855 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:08:10.855 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:10.855 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:10.855 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:10.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.855 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:08:10.855 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:08:10.855 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:10.856 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:10.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:10.856 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:08:10.856 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:10.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:10.856 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:08:10.856 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:10.856 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:10.857 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:10.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.857 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:08:10.857 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:08:10.857 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:10.857 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:10.857 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:10.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.857 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:08:10.857 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:10.858 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.859 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:10.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:08:10.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.862 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:10.862 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:10.862 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:08:10.862 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:10.862 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:10.862 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:08:10.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:10.862 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:10.862 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:10.862 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:08:10.863 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:10.863 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:10.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:10.863 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:08:10.863 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:08:10.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.841+0000 7fcc63c00640 1 -- 172.21.15.67:0/2692479694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc3c06b4b0 msgr2=0x7fcc3c06d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.841+0000 7fcc63c00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc3c06b4b0 0x7fcc3c06d970 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c070eb0 tx=0x7fcc54001480 comp rx=0 tx=0).stop 2024-03-09T09:08:10.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.841+0000 7fcc63c00640 1 -- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1ee880 msgr2=0x7fcc5c155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:10.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.841+0000 7fcc63c00640 1 --2- 172.21.15.67:0/2692479694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5c1ee880 0x7fcc5c155060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009be0 tx=0x7fcc48004730 comp rx=0 tx=0).stop 2024-03-09T09:08:10.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.841+0000 7fcc61c00640 1 -- 172.21.15.67:0/2692479694 reap_dead start 2024-03-09T09:08:10.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.841+0000 7fcc61c00640 1 -- 172.21.15.67:0/2692479694 reap_dead start 2024-03-09T09:08:10.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.841+0000 7fcc63c00640 1 -- 172.21.15.67:0/2692479694 shutdown_connections 2024-03-09T09:08:10.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.841+0000 7fcc63c00640 1 -- 172.21.15.67:0/2692479694 >> 172.21.15.67:0/2692479694 conn(0x7fcc5c06c540 msgr2=0x7fcc5c14cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:10.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.841+0000 7fcc63c00640 1 -- 172.21.15.67:0/2692479694 shutdown_connections 2024-03-09T09:08:10.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:10.841+0000 7fcc63c00640 1 -- 172.21.15.67:0/2692479694 wait complete. 2024-03-09T09:08:10.867 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:08:10.867 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:08:10.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:08:11.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.069+0000 7fd4d2800640 1 Processor -- start 2024-03-09T09:08:11.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.070+0000 7fd4d2800640 1 -- start start 2024-03-09T09:08:11.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.071+0000 7fd4d2800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd4cc0c9380 0x7fd4cc0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:11.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.071+0000 7fd4d2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd4cc0ad440 0x7fd4cc06c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:11.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.071+0000 7fd4d2800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4cc0c9890 con 0x7fd4cc14fd90 2024-03-09T09:08:11.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.071+0000 7fd4d2800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4cc06ba60 con 0x7fd4cc0ad440 2024-03-09T09:08:11.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.071+0000 7fd4d2800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4cc0c5f80 con 0x7fd4cc0c9380 2024-03-09T09:08:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.071+0000 7fd4d1400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd4cc14fd90 0x7fd4cc156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49982/0 (socket says 172.21.15.67:49982) 2024-03-09T09:08:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.071+0000 7fd4d1400640 1 -- 172.21.15.67:0/3782336268 learned_addr learned my addr 172.21.15.67:0/3782336268 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:08:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.071+0000 7fd4d1e00640 1 --2- 172.21.15.67:0/3782336268 >> v2:172.21.15.67:3300/0 conn(0x7fd4cc0ad440 0x7fd4cc06c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.071+0000 7fd4d0a00640 1 --2- 172.21.15.67:0/3782336268 >> v2:172.21.15.134:3300/0 conn(0x7fd4cc0c9380 0x7fd4cc0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.072+0000 7fd4cbe00640 1 -- 172.21.15.67:0/3782336268 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385342840 0 0) 0x7fd4cc0c9890 con 0x7fd4cc14fd90 2024-03-09T09:08:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.072+0000 7fd4cbe00640 1 -- 172.21.15.67:0/3782336268 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4b4003520 con 0x7fd4cc14fd90 2024-03-09T09:08:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.072+0000 7fd4d1e00640 1 -- 172.21.15.67:0/3782336268 >> v2:172.21.15.134:3300/0 conn(0x7fd4cc0c9380 msgr2=0x7fd4cc0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.072+0000 7fd4d1e00640 1 --2- 172.21.15.67:0/3782336268 >> v2:172.21.15.134:3300/0 conn(0x7fd4cc0c9380 0x7fd4cc0acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.072+0000 7fd4d1e00640 1 -- 172.21.15.67:0/3782336268 >> v1:172.21.15.67:6789/0 conn(0x7fd4cc14fd90 legacy=0x7fd4cc156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.072+0000 7fd4d1e00640 1 -- 172.21.15.67:0/3782336268 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0c6240 con 0x7fd4cc0ad440 2024-03-09T09:08:11.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.073+0000 7fd4d1e00640 1 --2- 172.21.15.67:0/3782336268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc0ad440 0x7fd4cc06c1d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0ae040 tx=0x7fd4c402bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8e5d04b0c39216 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:11.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.074+0000 7fd4cbe00640 1 -- 172.21.15.67:0/3782336268 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4039070 con 0x7fd4cc0ad440 2024-03-09T09:08:11.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.074+0000 7fd4cbe00640 1 -- 172.21.15.67:0/3782336268 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c402cb50 con 0x7fd4cc0ad440 2024-03-09T09:08:11.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.074+0000 7fd4cbe00640 1 -- 172.21.15.67:0/3782336268 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4034440 con 0x7fd4cc0ad440 2024-03-09T09:08:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.074+0000 7fd4d2800640 1 -- 172.21.15.67:0/3782336268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc0ad440 msgr2=0x7fd4cc06c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.074+0000 7fd4d2800640 1 --2- 172.21.15.67:0/3782336268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc0ad440 0x7fd4cc06c1d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0ae040 tx=0x7fd4c402bda0 comp rx=0 tx=0).stop 2024-03-09T09:08:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.075+0000 7fd4d2800640 1 -- 172.21.15.67:0/3782336268 shutdown_connections 2024-03-09T09:08:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.075+0000 7fd4d2800640 1 --2- 172.21.15.67:0/3782336268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc0ad440 0x7fd4cc06c1d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.075+0000 7fd4d2800640 1 --2- 172.21.15.67:0/3782336268 >> v2:172.21.15.134:3300/0 conn(0x7fd4cc0c9380 0x7fd4cc0acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.075+0000 7fd4d2800640 1 -- 172.21.15.67:0/3782336268 >> 172.21.15.67:0/3782336268 conn(0x7fd4cc0cb480 msgr2=0x7fd4cc0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.075+0000 7fd4d2800640 1 -- 172.21.15.67:0/3782336268 shutdown_connections 2024-03-09T09:08:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.075+0000 7fd4d2800640 1 -- 172.21.15.67:0/3782336268 wait complete. 2024-03-09T09:08:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.077+0000 7fd4d2800640 1 Processor -- start 2024-03-09T09:08:11.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.077+0000 7fd4d2800640 1 -- start start 2024-03-09T09:08:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.078+0000 7fd4d2800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4cc0ad440 0x7fd4cc1ee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d2800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc0c9380 0x7fd4cc1ec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc14fd90 0x7fd4cc1ecc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc06c710 con 0x7fd4cc14fd90 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d2800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc06bcb0 con 0x7fd4cc0c9380 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d2800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc06c890 con 0x7fd4cc0ad440 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d1400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4cc0ad440 msgr2=0x7fd4cc1ee9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d1400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4cc0ad440 0x7fd4cc1ee9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc14fd90 0x7fd4cc1ecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc14fd90 0x7fd4cc1ecc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34064/0 (socket says 172.21.15.67:34064) 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d1e00640 1 -- 172.21.15.67:0/2882812960 learned_addr learned my addr 172.21.15.67:0/2882812960 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d1e00640 1 -- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4cc0ad440 msgr2=0x7fd4cc1ee9b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d1e00640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4cc0ad440 0x7fd4cc1ee9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d0a00640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc0c9380 0x7fd4cc1ec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d1e00640 1 -- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc0c9380 msgr2=0x7fd4cc1ec6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d1e00640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc0c9380 0x7fd4cc1ec6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.079+0000 7fd4d1e00640 1 -- 172.21.15.67:0/2882812960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0b12c0 con 0x7fd4cc14fd90 2024-03-09T09:08:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.080+0000 7fd4d1e00640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc14fd90 0x7fd4cc1ecc10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4037040 tx=0x7fd4c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.080+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4039050 con 0x7fd4cc14fd90 2024-03-09T09:08:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.080+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c4045070 con 0x7fd4cc14fd90 2024-03-09T09:08:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.080+0000 7fd4d2800640 1 -- 172.21.15.67:0/2882812960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc149e10 con 0x7fd4cc14fd90 2024-03-09T09:08:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.080+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4003e10 con 0x7fd4cc14fd90 2024-03-09T09:08:11.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.080+0000 7fd4d2800640 1 -- 172.21.15.67:0/2882812960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc159d40 con 0x7fd4cc14fd90 2024-03-09T09:08:11.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.081+0000 7fd4d2800640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1533b0 0x7fd4cc1eeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:11.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.081+0000 7fd4d2800640 1 -- 172.21.15.67:0/2882812960 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc04f3c0 con 0x7fd4cc1533b0 2024-03-09T09:08:11.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.082+0000 7fd4d1400640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1533b0 0x7fd4cc1eeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:11.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.082+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd4c404a460 con 0x7fd4cc14fd90 2024-03-09T09:08:11.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.083+0000 7fd4d1400640 1 -- 172.21.15.67:0/2882812960 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4cc06d2d0 con 0x7fd4cc1533b0 2024-03-09T09:08:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.083+0000 7fd4c9600640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd4ac06b4b0 0x7fd4ac06d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.083+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd4c40acd40 con 0x7fd4cc14fd90 2024-03-09T09:08:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.083+0000 7fd4d1400640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1533b0 0x7fd4cc1eeef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc159130 tx=0x7fd4c000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.084+0000 7fd4d0a00640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd4ac06b4b0 0x7fd4ac06d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.084+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0017070 con 0x7fd4cc1533b0 2024-03-09T09:08:11.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.085+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd4c0005da0 con 0x7fd4cc1533b0 2024-03-09T09:08:11.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.085+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1533b0 msgr2=0x7fd4cc1eeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:11.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.085+0000 7fd4c9600640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1533b0 0x7fd4cc1eeef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc159130 tx=0x7fd4c000c7a0 comp rx=0 tx=0).stop 2024-03-09T09:08:11.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.085+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1533b0 msgr2=0x7fd4cc1eeef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:08:11.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.085+0000 7fd4c9600640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1533b0 0x7fd4cc1eeef0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc159130 tx=0x7fd4c000c7a0 comp rx=0 tx=0).stop 2024-03-09T09:08:11.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.085+0000 7fd4d0a00640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd4ac06b4b0 0x7fd4ac06d970 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc158a80 tx=0x7fd4b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:11.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.092+0000 7fd4d2800640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1ed150 0x7fd4cc156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:11.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.092+0000 7fd4d2800640 1 -- 172.21.15.67:0/2882812960 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc073a30 con 0x7fd4cc1ed150 2024-03-09T09:08:11.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.092+0000 7fd4d1400640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1ed150 0x7fd4cc156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:11.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.093+0000 7fd4d1400640 1 -- 172.21.15.67:0/2882812960 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd4cc06d630 con 0x7fd4cc1ed150 2024-03-09T09:08:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.094+0000 7fd4d1400640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1ed150 0x7fd4cc156ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0000c00 tx=0x7fd4c0000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.094+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0017040 con 0x7fd4cc1ed150 2024-03-09T09:08:11.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.095+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4523 (secure 0 0 0) 0x7fd4c0019920 con 0x7fd4cc1ed150 2024-03-09T09:08:11.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.095+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1ed150 msgr2=0x7fd4cc156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:11.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.095+0000 7fd4c9600640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1ed150 0x7fd4cc156ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0000c00 tx=0x7fd4c0000ed0 comp rx=0 tx=0).stop 2024-03-09T09:08:11.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.095+0000 7fd4c9600640 1 -- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1ed150 msgr2=0x7fd4cc156ff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:08:11.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.095+0000 7fd4c9600640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4cc1ed150 0x7fd4cc156ff0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0000c00 tx=0x7fd4c0000ed0 comp rx=0 tx=0).stop 2024-03-09T09:08:11.100 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:08:11.100 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:08:11.100 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:08:11.100 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 112, 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:11.101 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:11.102 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:11.102 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:11.102 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:11.102 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:11.102 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:11.102 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:11.102 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:11.102 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:11.102 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:11.102 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:08:11.103 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:08:11.103 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:11.103 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:11.103 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:11.103 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:11.103 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:11.103 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:11.103 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:11.103 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:11.103 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:11.103 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:11.104 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:11.104 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:11.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.104 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:08:11.104 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:08:11.105 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:08:11.105 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:08:11.105 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:08:11.105 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:08:11.105 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:08:11.105 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:08:11.105 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:08:11.105 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:08:11.106 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:08:11.106 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:08:11.106 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:08:11.106 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:08:11.106 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:08:11.106 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:08:11.106 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:08:11.106 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:11.106 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:11.106 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:11.107 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:11.107 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:11.107 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:11.107 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:11.107 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:11.107 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:11.107 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:11.107 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:11.107 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:11.107 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:08:11.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.108 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:08:11.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:11.108 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:08:11.108 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:08:11.108 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:11.108 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:11.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:11.108 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:11.108 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:08:11.108 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:11.109 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:11.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:11.111 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:11.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:11.112 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:08:11.112 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:08:11.112 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:11.112 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:11.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:11.112 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:11.112 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:08:11.112 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:11.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:11.113 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:11.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.114 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:08:11.114 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:08:11.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:11.114 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:11.114 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:11.114 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:08:11.114 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:11.114 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:11.115 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:08:11.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:11.115 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:11.115 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:11.115 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:08:11.115 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:11.115 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:11.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:11.115 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:08:11.115 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:08:11.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.095+0000 7fd4d2800640 1 -- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd4ac06b4b0 msgr2=0x7fd4ac06d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:11.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.095+0000 7fd4d2800640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd4ac06b4b0 0x7fd4ac06d970 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc158a80 tx=0x7fd4b8008040 comp rx=0 tx=0).stop 2024-03-09T09:08:11.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.095+0000 7fd4d2800640 1 -- 172.21.15.67:0/2882812960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc14fd90 msgr2=0x7fd4cc1ecc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:11.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.095+0000 7fd4d2800640 1 --2- 172.21.15.67:0/2882812960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd4cc14fd90 0x7fd4cc1ecc10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4037040 tx=0x7fd4c4004a40 comp rx=0 tx=0).stop 2024-03-09T09:08:11.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.095+0000 7fd4d1e00640 1 -- 172.21.15.67:0/2882812960 reap_dead start 2024-03-09T09:08:11.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.095+0000 7fd4d1e00640 1 -- 172.21.15.67:0/2882812960 reap_dead start 2024-03-09T09:08:11.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.096+0000 7fd4d2800640 1 -- 172.21.15.67:0/2882812960 shutdown_connections 2024-03-09T09:08:11.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.096+0000 7fd4d2800640 1 -- 172.21.15.67:0/2882812960 >> 172.21.15.67:0/2882812960 conn(0x7fd4cc0cb480 msgr2=0x7fd4cc0c6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:11.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.096+0000 7fd4d2800640 1 -- 172.21.15.67:0/2882812960 shutdown_connections 2024-03-09T09:08:11.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:11.096+0000 7fd4d2800640 1 -- 172.21.15.67:0/2882812960 wait complete. 2024-03-09T09:08:11.118 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:08:11.118 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:08:14.640 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 26% (67/253) Updating files: 27% (69/253) Updating files: 28% (71/253) Updating files: 29% (74/253) Updating files: 30% (76/253) Updating files: 31% (79/253) Updating files: 32% (81/253) Updating files: 33% (84/253) Updating files: 34% (87/253) Updating files: 35% (89/253) Updating files: 36% (92/253) Updating files: 37% (94/253) Updating files: 38% (97/253) Updating files: 39% (99/253) Updating files: 40% (102/253) Updating files: 41% (104/253) Updating files: 42% (107/253) Updating files: 43% (109/253) Updating files: 44% (112/253) Updating files: 45% (114/253) Updating files: 46% (117/253) Updating files: 47% (119/253) Updating files: 48% (122/253) Updating files: 49% (124/253) Updating files: 50% (127/253) Updating files: 51% (130/253) Updating files: 52% (132/253) Updating files: 53% (135/253) Updating files: 54% (137/253) Updating files: 55% (140/253) Updating files: 56% (142/253) Updating files: 57% (145/253) Updating files: 58% (147/253) Updating files: 59% (150/253) Updating files: 59% (151/253) Updating files: 60% (152/253) Updating files: 61% (155/253) Updating files: 62% (157/253) Updating files: 63% (160/253) Updating files: 64% (162/253) Updating files: 65% (165/253) Updating files: 66% (167/253) Updating files: 67% (170/253) Updating files: 68% (173/253) Updating files: 69% (175/253) Updating files: 70% (178/253) Updating files: 71% (180/253) Updating files: 72% (183/253) Updating files: 73% (185/253) Updating files: 74% (188/253) Updating files: 75% (190/253) Updating files: 76% (193/253) Updating files: 77% (195/253) Updating files: 78% (198/253) Updating files: 79% (200/253) Updating files: 80% (203/253) Updating files: 81% (205/253) Updating files: 82% (208/253) Updating files: 83% (210/253) Updating files: 84% (213/253) Updating files: 85% (216/253) Updating files: 86% (218/253) Updating files: 87% (221/253) Updating files: 88% (223/253) Updating files: 89% (226/253) Updating files: 90% (228/253) Updating files: 91% (231/253) Updating files: 92% (233/253) Updating files: 93% (236/253) Updating files: 94% (238/253) Updating files: 95% (241/253) Updating files: 96% (243/253) Updating files: 97% (246/253) Updating files: 98% (248/253) Updating files: 99% (251/253) Updating files: 100% (253/253) Updating files: 100% (253/253), done. 2024-03-09T09:08:14.683 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 793034c62c8 v0.27 2024-03-09T09:08:14.697 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.27 2024-03-09T09:08:14.945 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:08:14.946 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 28 -eq 48 ']' 2024-03-09T09:08:14.946 INFO:tasks.workunit.client.1.smithi134.stdout:v0.28 2024-03-09T09:08:14.946 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.28 2024-03-09T09:08:14.946 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.28 2024-03-09T09:08:14.946 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.28 2024-03-09T09:08:16.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:08:16.394 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:08:19.624 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 34% (78/227) Updating files: 35% (80/227) Updating files: 36% (82/227) Updating files: 37% (84/227) Updating files: 38% (87/227) Updating files: 39% (89/227) Updating files: 40% (91/227) Updating files: 41% (94/227) Updating files: 42% (96/227) Updating files: 43% (98/227) Updating files: 44% (100/227) Updating files: 45% (103/227) Updating files: 46% (105/227) Updating files: 47% (107/227) Updating files: 48% (109/227) Updating files: 49% (112/227) Updating files: 50% (114/227) Updating files: 51% (116/227) Updating files: 52% (119/227) Updating files: 53% (121/227) Updating files: 54% (123/227) Updating files: 55% (125/227) Updating files: 56% (128/227) Updating files: 57% (130/227) Updating files: 58% (132/227) Updating files: 59% (134/227) Updating files: 60% (137/227) Updating files: 61% (139/227) Updating files: 62% (141/227) Updating files: 63% (144/227) Updating files: 64% (146/227) Updating files: 65% (148/227) Updating files: 66% (150/227) Updating files: 67% (153/227) Updating files: 68% (155/227) Updating files: 69% (157/227) Updating files: 70% (159/227) Updating files: 71% (162/227) Updating files: 72% (164/227) Updating files: 73% (166/227) Updating files: 74% (168/227) Updating files: 75% (171/227) Updating files: 76% (173/227) Updating files: 77% (175/227) Updating files: 78% (178/227) Updating files: 79% (180/227) Updating files: 80% (182/227) Updating files: 81% (184/227) Updating files: 81% (185/227) Updating files: 82% (187/227) Updating files: 83% (189/227) Updating files: 84% (191/227) Updating files: 85% (193/227) Updating files: 86% (196/227) Updating files: 87% (198/227) Updating files: 88% (200/227) Updating files: 89% (203/227) Updating files: 90% (205/227) Updating files: 91% (207/227) Updating files: 92% (209/227) Updating files: 92% (211/227) Updating files: 93% (212/227) Updating files: 94% (214/227) Updating files: 95% (216/227) Updating files: 96% (218/227) Updating files: 97% (221/227) Updating files: 98% (223/227) Updating files: 99% (225/227) Updating files: 100% (227/227) Updating files: 100% (227/227), done. 2024-03-09T09:08:19.715 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 071881d7e55 v0.28 2024-03-09T09:08:19.727 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.28 2024-03-09T09:08:19.821 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:08:19.821 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 29 -eq 48 ']' 2024-03-09T09:08:19.821 INFO:tasks.workunit.client.1.smithi134.stdout:v0.29 2024-03-09T09:08:19.822 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.29 2024-03-09T09:08:19.822 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.29 2024-03-09T09:08:19.822 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.29 2024-03-09T09:08:24.374 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 38% (88/227) Updating files: 39% (89/227) Updating files: 40% (91/227) Updating files: 41% (94/227) Updating files: 42% (96/227) Updating files: 43% (98/227) Updating files: 44% (100/227) Updating files: 45% (103/227) Updating files: 46% (105/227) Updating files: 47% (107/227) Updating files: 48% (109/227) Updating files: 49% (112/227) Updating files: 50% (114/227) Updating files: 51% (116/227) Updating files: 52% (119/227) Updating files: 53% (121/227) Updating files: 54% (123/227) Updating files: 55% (125/227) Updating files: 56% (128/227) Updating files: 57% (130/227) Updating files: 58% (132/227) Updating files: 59% (134/227) Updating files: 60% (137/227) Updating files: 61% (139/227) Updating files: 62% (141/227) Updating files: 63% (144/227) Updating files: 64% (146/227) Updating files: 65% (148/227) Updating files: 66% (150/227) Updating files: 67% (153/227) Updating files: 68% (155/227) Updating files: 69% (157/227) Updating files: 70% (159/227) Updating files: 71% (162/227) Updating files: 72% (164/227) Updating files: 73% (166/227) Updating files: 74% (168/227) Updating files: 75% (171/227) Updating files: 76% (173/227) Updating files: 77% (175/227) Updating files: 78% (178/227) Updating files: 79% (180/227) Updating files: 80% (182/227) Updating files: 81% (184/227) Updating files: 82% (187/227) Updating files: 83% (189/227) Updating files: 84% (191/227) Updating files: 85% (193/227) Updating files: 86% (196/227) Updating files: 87% (198/227) Updating files: 88% (200/227) Updating files: 89% (203/227) Updating files: 90% (205/227) Updating files: 91% (207/227) Updating files: 92% (209/227) Updating files: 93% (212/227) Updating files: 94% (214/227) Updating files: 95% (216/227) Updating files: 96% (218/227) Updating files: 97% (221/227) Updating files: 98% (223/227) Updating files: 99% (225/227) Updating files: 100% (227/227) Updating files: 100% (227/227), done. 2024-03-09T09:08:24.425 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 071881d7e55 v0.28 2024-03-09T09:08:24.443 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.28 2024-03-09T09:08:24.660 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:08:24.661 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 29 -eq 48 ']' 2024-03-09T09:08:24.661 INFO:tasks.workunit.client.0.smithi067.stdout:v0.29 2024-03-09T09:08:24.662 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.29 2024-03-09T09:08:24.662 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.29 2024-03-09T09:08:24.662 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.29 2024-03-09T09:08:27.110 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 58% (93/159) Updating files: 59% (94/159) Updating files: 60% (96/159) Updating files: 61% (97/159) Updating files: 62% (99/159) Updating files: 63% (101/159) Updating files: 64% (102/159) Updating files: 65% (104/159) Updating files: 66% (105/159) Updating files: 67% (107/159) Updating files: 68% (109/159) Updating files: 69% (110/159) Updating files: 70% (112/159) Updating files: 71% (113/159) Updating files: 72% (115/159) Updating files: 73% (117/159) Updating files: 74% (118/159) Updating files: 75% (120/159) Updating files: 76% (121/159) Updating files: 77% (123/159) Updating files: 78% (125/159) Updating files: 79% (126/159) Updating files: 80% (128/159) Updating files: 81% (129/159) Updating files: 82% (131/159) Updating files: 83% (132/159) Updating files: 84% (134/159) Updating files: 85% (136/159) Updating files: 86% (137/159) Updating files: 87% (139/159) Updating files: 88% (140/159) Updating files: 89% (142/159) Updating files: 90% (144/159) Updating files: 91% (145/159) Updating files: 92% (147/159) Updating files: 93% (148/159) Updating files: 94% (150/159) Updating files: 95% (152/159) Updating files: 96% (153/159) Updating files: 97% (155/159) Updating files: 98% (156/159) Updating files: 99% (158/159) Updating files: 100% (159/159) Updating files: 100% (159/159), done. 2024-03-09T09:08:27.160 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 8e69c39f699 v0.29 2024-03-09T09:08:27.172 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.29 2024-03-09T09:08:27.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:08:27.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 30 -eq 48 ']' 2024-03-09T09:08:27.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.30 2024-03-09T09:08:27.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.30 2024-03-09T09:08:27.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.30 2024-03-09T09:08:27.909 INFO:tasks.workunit.client.0.smithi067.stdout:v0.30 2024-03-09T09:08:30.574 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 72% (206/285) Updating files: 73% (209/285) Updating files: 74% (211/285) Updating files: 75% (214/285) Updating files: 76% (217/285) Updating files: 77% (220/285) Updating files: 78% (223/285) Updating files: 79% (226/285) Updating files: 80% (228/285) Updating files: 81% (231/285) Updating files: 82% (234/285) Updating files: 83% (237/285) Updating files: 84% (240/285) Updating files: 85% (243/285) Updating files: 86% (246/285) Updating files: 87% (248/285) Updating files: 88% (251/285) Updating files: 89% (254/285) Updating files: 90% (257/285) Updating files: 91% (260/285) Updating files: 92% (263/285) Updating files: 93% (266/285) Updating files: 94% (268/285) Updating files: 94% (270/285) Updating files: 95% (271/285) Updating files: 96% (274/285) Updating files: 97% (277/285) Updating files: 98% (280/285) Updating files: 99% (283/285) Updating files: 100% (285/285) Updating files: 100% (285/285), done. 2024-03-09T09:08:30.622 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 64b1b2c70f0 v0.30 2024-03-09T09:08:30.637 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.30 2024-03-09T09:08:30.972 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:08:30.973 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 31 -eq 48 ']' 2024-03-09T09:08:30.973 INFO:tasks.workunit.client.0.smithi067.stdout:v0.31 2024-03-09T09:08:30.973 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.31 2024-03-09T09:08:30.973 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.31 2024-03-09T09:08:30.973 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.31 2024-03-09T09:08:31.119 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:08:31.119 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:08:31.119 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:08:31.119 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:08:31.127 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:08:31.128 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-03-09T09:08:31.128 INFO:tasks.ceph.mon.b:Sent signal 19 2024-03-09T09:08:31.128 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-03-09T09:08:36.218 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 5% (9/159) Updating files: 6% (10/159) Updating files: 7% (12/159) Updating files: 8% (13/159) Updating files: 9% (15/159) Updating files: 10% (16/159) Updating files: 11% (18/159) Updating files: 12% (20/159) Updating files: 13% (21/159) Updating files: 14% (23/159) Updating files: 15% (24/159) Updating files: 16% (26/159) Updating files: 17% (28/159) Updating files: 18% (29/159) Updating files: 19% (31/159) Updating files: 20% (32/159) Updating files: 21% (34/159) Updating files: 22% (35/159) Updating files: 23% (37/159) Updating files: 24% (39/159) Updating files: 25% (40/159) Updating files: 26% (42/159) Updating files: 27% (43/159) Updating files: 28% (45/159) Updating files: 29% (47/159) Updating files: 30% (48/159) Updating files: 31% (50/159) Updating files: 32% (51/159) Updating files: 33% (53/159) Updating files: 34% (55/159) Updating files: 35% (56/159) Updating files: 36% (58/159) Updating files: 37% (59/159) Updating files: 38% (61/159) Updating files: 39% (63/159) Updating files: 40% (64/159) Updating files: 41% (66/159) Updating files: 42% (67/159) Updating files: 43% (69/159) Updating files: 44% (70/159) Updating files: 45% (72/159) Updating files: 46% (74/159) Updating files: 47% (75/159) Updating files: 48% (77/159) Updating files: 49% (78/159) Updating files: 50% (80/159) Updating files: 51% (82/159) Updating files: 52% (83/159) Updating files: 53% (85/159) Updating files: 54% (86/159) Updating files: 55% (88/159) Updating files: 56% (90/159) Updating files: 57% (91/159) Updating files: 58% (93/159) Updating files: 59% (94/159) Updating files: 60% (96/159) Updating files: 61% (97/159) Updating files: 62% (99/159) Updating files: 63% (101/159) Updating files: 64% (102/159) Updating files: 65% (104/159) Updating files: 66% (105/159) Updating files: 67% (107/159) Updating files: 68% (109/159) Updating files: 69% (110/159) Updating files: 70% (112/159) Updating files: 71% (113/159) Updating files: 72% (115/159) Updating files: 73% (117/159) Updating files: 74% (118/159) Updating files: 75% (120/159) Updating files: 76% (121/159) Updating files: 77% (123/159) Updating files: 78% (125/159) Updating files: 79% (126/159) Updating files: 80% (128/159) Updating files: 81% (129/159) Updating files: 82% (131/159) Updating files: 83% (132/159) Updating files: 84% (134/159) Updating files: 85% (136/159) Updating files: 86% (137/159) Updating files: 87% (139/159) Updating files: 88% (140/159) Updating files: 89% (142/159) Updating files: 89% (143/159) Updating files: 90% (144/159) Updating files: 91% (145/159) Updating files: 92% (147/159) Updating files: 93% (148/159) Updating files: 94% (150/159) Updating files: 95% (152/159) Updating files: 96% (153/159) Updating files: 97% (155/159) Updating files: 98% (156/159) Updating files: 99% (158/159) Updating files: 100% (159/159) Updating files: 100% (159/159), done. 2024-03-09T09:08:36.343 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 8e69c39f699 v0.29 2024-03-09T09:08:36.358 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.29 2024-03-09T09:08:36.448 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:08:36.448 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 30 -eq 48 ']' 2024-03-09T09:08:36.448 INFO:tasks.workunit.client.1.smithi134.stdout:v0.30 2024-03-09T09:08:36.448 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.30 2024-03-09T09:08:36.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.30 2024-03-09T09:08:36.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.30 2024-03-09T09:08:42.272 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 24% (71/285) Updating files: 25% (72/285) Updating files: 26% (75/285) Updating files: 27% (77/285) Updating files: 28% (80/285) Updating files: 29% (83/285) Updating files: 30% (86/285) Updating files: 31% (89/285) Updating files: 32% (92/285) Updating files: 33% (95/285) Updating files: 34% (97/285) Updating files: 34% (99/285) Updating files: 35% (100/285) Updating files: 36% (103/285) Updating files: 37% (106/285) Updating files: 38% (109/285) Updating files: 39% (112/285) Updating files: 40% (114/285) Updating files: 41% (117/285) Updating files: 42% (120/285) Updating files: 43% (123/285) Updating files: 44% (126/285) Updating files: 45% (129/285) Updating files: 46% (132/285) Updating files: 47% (134/285) Updating files: 48% (137/285) Updating files: 49% (140/285) Updating files: 50% (143/285) Updating files: 51% (146/285) Updating files: 52% (149/285) Updating files: 53% (152/285) Updating files: 54% (154/285) Updating files: 55% (157/285) Updating files: 56% (160/285) Updating files: 57% (163/285) Updating files: 58% (166/285) Updating files: 59% (169/285) Updating files: 60% (171/285) Updating files: 61% (174/285) Updating files: 62% (177/285) Updating files: 63% (180/285) Updating files: 64% (183/285) Updating files: 65% (186/285) Updating files: 66% (189/285) Updating files: 67% (191/285) Updating files: 68% (194/285) Updating files: 69% (197/285) Updating files: 69% (198/285) Updating files: 70% (200/285) Updating files: 71% (203/285) Updating files: 72% (206/285) Updating files: 73% (209/285) Updating files: 74% (211/285) Updating files: 75% (214/285) Updating files: 76% (217/285) Updating files: 77% (220/285) Updating files: 78% (223/285) Updating files: 79% (226/285) Updating files: 80% (228/285) Updating files: 81% (231/285) Updating files: 82% (234/285) Updating files: 83% (237/285) Updating files: 84% (240/285) Updating files: 84% (241/285) Updating files: 85% (243/285) Updating files: 86% (246/285) Updating files: 87% (248/285) Updating files: 88% (251/285) Updating files: 89% (254/285) Updating files: 90% (257/285) Updating files: 91% (260/285) Updating files: 92% (263/285) Updating files: 93% (266/285) Updating files: 94% (268/285) Updating files: 95% (271/285) Updating files: 96% (274/285) Updating files: 97% (277/285) Updating files: 98% (280/285) Updating files: 99% (283/285) Updating files: 100% (285/285) Updating files: 100% (285/285), done. 2024-03-09T09:08:42.815 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 64b1b2c70f0 v0.30 2024-03-09T09:08:42.835 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.30 2024-03-09T09:08:43.877 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:08:43.877 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 31 -eq 48 ']' 2024-03-09T09:08:43.877 INFO:tasks.workunit.client.1.smithi134.stdout:v0.31 2024-03-09T09:08:43.877 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.31 2024-03-09T09:08:43.878 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.31 2024-03-09T09:08:43.878 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.31 2024-03-09T09:08:46.129 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-03-09T09:08:46.130 INFO:tasks.ceph.mon.b:Sent signal 18 2024-03-09T09:08:46.130 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:08:46.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:08:46.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.405+0000 7fe148c00640 1 Processor -- start 2024-03-09T09:08:46.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.405+0000 7fe148c00640 1 -- start start 2024-03-09T09:08:46.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.406+0000 7fe148c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:46.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.406+0000 7fe148c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:46.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.406+0000 7fe148c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.406+0000 7fe148c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1440c81e0 con 0x7fe1440c27a0 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.407+0000 7fe148c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1440c32b0 con 0x7fe1440710a0 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.407+0000 7fe148c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440c2fd0 con 0x7fe1440c1f70 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.407+0000 7fe143e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.407+0000 7fe143e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54342/0 (socket says 172.21.15.67:54342) 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.407+0000 7fe143e00640 1 -- 172.21.15.67:0/3402795861 learned_addr learned my addr 172.21.15.67:0/3402795861 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.407+0000 7fe143e00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 msgr2=0x7fe144153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.407+0000 7fe143e00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 msgr2=0x7fe144153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.407+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.407+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.407+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.407+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:46.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.408+0000 7fe143400640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 msgr2=0x7fe144150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:08:46.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.408+0000 7fe143400640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 msgr2=0x7fe144150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:46.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.408+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:46.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.408+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:46.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.408+0000 7fe142a00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 msgr2=0x7fe144151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:08:46.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.408+0000 7fe142a00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 msgr2=0x7fe144151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:46.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.408+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:46.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.408+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:46.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:08:46.529 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:08:46.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.608+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:46.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.608+0000 7fe143e00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 msgr2=0x7fe144153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:08:46.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.608+0000 7fe143e00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 msgr2=0x7fe144153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.608+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.609+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.609+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.609+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.609+0000 7fe142a00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 msgr2=0x7fe144151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.609+0000 7fe142a00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 msgr2=0x7fe144151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.609+0000 7fe143400640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 msgr2=0x7fe144150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.609+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.609+0000 7fe143400640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 msgr2=0x7fe144150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.609+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.609+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:08:46.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:46.609+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:08:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.010+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.010+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.010+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:47.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe142a00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 msgr2=0x7fe144151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:08:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe142a00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 msgr2=0x7fe144151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:08:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe143e00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 msgr2=0x7fe144153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:08:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe143e00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 msgr2=0x7fe144153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe143400640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 msgr2=0x7fe144150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:08:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe143400640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 msgr2=0x7fe144150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:08:47.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.011+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:08:47.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.812+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:47.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:47.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:47.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe143e00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 msgr2=0x7fe144153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:08:47.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe143e00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 msgr2=0x7fe144153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:47.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:47.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe143400640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 msgr2=0x7fe144150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:08:47.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe142a00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 msgr2=0x7fe144151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:08:47.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe143400640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 msgr2=0x7fe144150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:47.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe142a00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 msgr2=0x7fe144151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:08:47.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:47.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:47.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:08:47.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:08:47.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:47.813+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:08:48.702 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 5% (20/398) Updating files: 6% (24/398) Updating files: 7% (28/398) Updating files: 8% (32/398) Updating files: 9% (36/398) Updating files: 10% (40/398) Updating files: 11% (44/398) Updating files: 12% (48/398) Updating files: 13% (52/398) Updating files: 14% (56/398) Updating files: 15% (60/398) Updating files: 16% (64/398) Updating files: 17% (68/398) Updating files: 18% (72/398) Updating files: 19% (76/398) Updating files: 20% (80/398) Updating files: 21% (84/398) Updating files: 22% (88/398) Updating files: 23% (92/398) Updating files: 24% (96/398) Updating files: 25% (100/398) Updating files: 26% (104/398) Updating files: 26% (107/398) Updating files: 27% (108/398) Updating files: 28% (112/398) Updating files: 29% (116/398) Updating files: 30% (120/398) Updating files: 31% (124/398) Updating files: 32% (128/398) Updating files: 33% (132/398) Updating files: 34% (136/398) Updating files: 35% (140/398) Updating files: 36% (144/398) Updating files: 37% (148/398) Updating files: 38% (152/398) Updating files: 39% (156/398) Updating files: 40% (160/398) Updating files: 41% (164/398) Updating files: 42% (168/398) Updating files: 43% (172/398) Updating files: 44% (176/398) Updating files: 45% (180/398) Updating files: 46% (184/398) Updating files: 47% (188/398) Updating files: 48% (192/398) Updating files: 49% (196/398) Updating files: 50% (199/398) Updating files: 51% (203/398) Updating files: 52% (207/398) Updating files: 53% (211/398) Updating files: 54% (215/398) Updating files: 55% (219/398) Updating files: 56% (223/398) Updating files: 57% (227/398) Updating files: 58% (231/398) Updating files: 59% (235/398) Updating files: 60% (239/398) Updating files: 61% (243/398) Updating files: 62% (247/398) Updating files: 63% (251/398) Updating files: 64% (255/398) Updating files: 65% (259/398) Updating files: 66% (263/398) Updating files: 67% (267/398) Updating files: 68% (271/398) Updating files: 69% (275/398) Updating files: 70% (279/398) Updating files: 71% (283/398) Updating files: 72% (287/398) Updating files: 73% (291/398) Updating files: 74% (295/398) Updating files: 75% (299/398) Updating files: 76% (303/398) Updating files: 77% (307/398) Updating files: 78% (311/398) Updating files: 79% (315/398) Updating files: 80% (319/398) Updating files: 81% (323/398) Updating files: 82% (327/398) Updating files: 83% (331/398) Updating files: 84% (335/398) Updating files: 85% (339/398) Updating files: 86% (343/398) Updating files: 87% (347/398) Updating files: 88% (351/398) Updating files: 88% (354/398) Updating files: 89% (355/398) Updating files: 90% (359/398) Updating files: 91% (363/398) Updating files: 92% (367/398) Updating files: 93% (371/398) Updating files: 94% (375/398) Updating files: 95% (379/398) Updating files: 96% (383/398) Updating files: 97% (387/398) Updating files: 98% (391/398) Updating files: 98% (393/398) Updating files: 99% (395/398) Updating files: 100% (398/398) Updating files: 100% (398/398), done. 2024-03-09T09:08:49.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:49.407+0000 7fe140c00640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 msgr2=0x7fe144150c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:08:49.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:49.407+0000 7fe140c00640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe144150c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:49.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:49.407+0000 7fe140c00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 msgr2=0x7fe144151190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:08:49.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:49.407+0000 7fe140c00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.134:3300/0 conn(0x7fe1440710a0 0x7fe144151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:49.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:49.407+0000 7fe140c00640 1 -- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 msgr2=0x7fe144153970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:08:49.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:49.407+0000 7fe140c00640 1 --2- 172.21.15.67:0/3402795861 >> v2:172.21.15.67:3300/0 conn(0x7fe1440c27a0 0x7fe144153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:49.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:49.407+0000 7fe140c00640 1 -- 172.21.15.67:0/3402795861 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1440c3700 con 0x7fe128007300 2024-03-09T09:08:49.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:49.407+0000 7fe140c00640 1 -- 172.21.15.67:0/3402795861 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12800c080 con 0x7fe128003670 2024-03-09T09:08:50.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.359+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543985156 0 0) 0x7fe1440c3700 con 0x7fe128007300 2024-03-09T09:08:50.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.359+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe124003520 con 0x7fe128007300 2024-03-09T09:08:50.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 461544515 0 0) 0x7fe124003520 con 0x7fe128007300 2024-03-09T09:08:50.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 >> v1:172.21.15.134:6789/0 conn(0x7fe128003670 legacy=0x7fe128005ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:50.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12800d380 con 0x7fe128007300 2024-03-09T09:08:50.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fe13000d580 con 0x7fe128007300 2024-03-09T09:08:50.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 >> v1:172.21.15.67:6789/0 conn(0x7fe128007300 legacy=0x7fe1280097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:50.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe143e00640 1 -- 172.21.15.67:0/3402795861 reap_dead start 2024-03-09T09:08:50.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe142000640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe124003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:50.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe142000640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004230 0x7fe124004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:50.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe142000640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004fe0 0x7fe12400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:50.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12400bf00 con 0x7fe124004fe0 2024-03-09T09:08:50.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12400c080 con 0x7fe124004230 2024-03-09T09:08:50.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.362+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12400c230 con 0x7fe1440c1f70 2024-03-09T09:08:50.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.363+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004230 0x7fe124004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:50.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.363+0000 7fe143e00640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004fe0 0x7fe12400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:50.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.363+0000 7fe143400640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe124003af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:50.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.364+0000 7fe142a00640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 msgr2=0x7fe124003af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:50.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.364+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe124003af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.364+0000 7fe142a00640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004fe0 msgr2=0x7fe12400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:50.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.364+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004fe0 0x7fe12400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.364+0000 7fe142a00640 1 -- 172.21.15.67:0/3402795861 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe124003520 con 0x7fe124004230 2024-03-09T09:08:50.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.366+0000 7fe142a00640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004230 0x7fe124004800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe13000e010 tx=0x7fe124005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a65f17204054c718 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.367+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130033070 con 0x7fe124004230 2024-03-09T09:08:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.367+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13002d600 con 0x7fe124004230 2024-03-09T09:08:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.367+0000 7fe142000640 1 -- 172.21.15.67:0/3402795861 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13000dd90 con 0x7fe124004230 2024-03-09T09:08:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.369+0000 7fe148c00640 1 -- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004230 msgr2=0x7fe124004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.369+0000 7fe148c00640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004230 0x7fe124004800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe13000e010 tx=0x7fe124005830 comp rx=0 tx=0).stop 2024-03-09T09:08:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.369+0000 7fe148c00640 1 -- 172.21.15.67:0/3402795861 shutdown_connections 2024-03-09T09:08:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.369+0000 7fe148c00640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004fe0 0x7fe12400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.369+0000 7fe148c00640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004230 0x7fe124004800 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.369+0000 7fe148c00640 1 --2- 172.21.15.67:0/3402795861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe124003af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.369+0000 7fe148c00640 1 -- 172.21.15.67:0/3402795861 >> 172.21.15.67:0/3402795861 conn(0x7fe1440c7300 msgr2=0x7fe1440c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.369+0000 7fe148c00640 1 -- 172.21.15.67:0/3402795861 shutdown_connections 2024-03-09T09:08:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.369+0000 7fe148c00640 1 -- 172.21.15.67:0/3402795861 wait complete. 2024-03-09T09:08:50.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.374+0000 7fe148c00640 1 Processor -- start 2024-03-09T09:08:50.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.374+0000 7fe148c00640 1 -- start start 2024-03-09T09:08:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.377+0000 7fe148c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004230 0x7fe1441dea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.377+0000 7fe148c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004fe0 0x7fe1440c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:50.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.377+0000 7fe148c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe1441e0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.377+0000 7fe148c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe144070d50 con 0x7fe124004230 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.377+0000 7fe148c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe144070830 con 0x7fe124004fe0 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.377+0000 7fe148c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe144070ed0 con 0x7fe1440c1f70 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe143400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004230 0x7fe1441dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe143400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004230 0x7fe1441dea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55310/0 (socket says 172.21.15.67:55310) 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe143400640 1 -- 172.21.15.67:0/3490934918 learned_addr learned my addr 172.21.15.67:0/3490934918 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe142a00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004fe0 0x7fe1440c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe143e00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe1441e0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe142a00640 1 -- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004fe0 msgr2=0x7fe1440c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe142a00640 1 -- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004fe0 msgr2=0x7fe1440c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe142a00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004fe0 0x7fe1440c0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe142a00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004fe0 0x7fe1440c0a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:08:50.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe143400640 1 -- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 msgr2=0x7fe1441e0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:50.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe143400640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe1441e0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe143400640 1 -- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004fe0 msgr2=0x7fe1440c0a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:08:50.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe143400640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004fe0 0x7fe1440c0a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.378+0000 7fe143400640 1 -- 172.21.15.67:0/3490934918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440c1190 con 0x7fe124004230 2024-03-09T09:08:50.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.379+0000 7fe143400640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004230 0x7fe1441dea40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe134011fa0 tx=0x7fe134003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:50.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.379+0000 7fe13b400640 1 -- 172.21.15.67:0/3490934918 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1340165f0 con 0x7fe124004230 2024-03-09T09:08:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.379+0000 7fe13b400640 1 -- 172.21.15.67:0/3490934918 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1340167b0 con 0x7fe124004230 2024-03-09T09:08:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.379+0000 7fe13b400640 1 -- 172.21.15.67:0/3490934918 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe134016ba0 con 0x7fe124004230 2024-03-09T09:08:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.379+0000 7fe148c00640 1 -- 172.21.15.67:0/3490934918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1440c4b20 con 0x7fe124004230 2024-03-09T09:08:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.380+0000 7fe148c00640 1 -- 172.21.15.67:0/3490934918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1441e0750 con 0x7fe124004230 2024-03-09T09:08:50.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.381+0000 7fe148c00640 1 -- 172.21.15.67:0/3490934918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1440c4910 con 0x7fe124004230 2024-03-09T09:08:50.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.382+0000 7fe13b400640 1 -- 172.21.15.67:0/3490934918 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe134016d40 con 0x7fe124004230 2024-03-09T09:08:50.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.383+0000 7fe13b400640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12806bea0 0x7fe12806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:50.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.383+0000 7fe13b400640 1 -- 172.21.15.67:0/3490934918 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe13408b370 con 0x7fe124004230 2024-03-09T09:08:50.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.384+0000 7fe142a00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12806bea0 0x7fe12806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:50.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.385+0000 7fe142a00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12806bea0 0x7fe12806e360 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe130038000 tx=0x7fe130033000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:50.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.388+0000 7fe13b400640 1 -- 172.21.15.67:0/3490934918 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe134027340 con 0x7fe124004230 2024-03-09T09:08:50.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.661+0000 7fe148c00640 1 -- 172.21.15.67:0/3490934918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe14404f3c0 con 0x7fe124004230 2024-03-09T09:08:50.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.662+0000 7fe13b400640 1 -- 172.21.15.67:0/3490934918 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fe134059f50 con 0x7fe124004230 2024-03-09T09:08:50.666 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":114,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:08:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.664+0000 7fe148c00640 1 -- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12806bea0 msgr2=0x7fe12806e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.664+0000 7fe148c00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12806bea0 0x7fe12806e360 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe130038000 tx=0x7fe130033000 comp rx=0 tx=0).stop 2024-03-09T09:08:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.664+0000 7fe148c00640 1 -- 172.21.15.67:0/3490934918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004230 msgr2=0x7fe1441dea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.664+0000 7fe148c00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004230 0x7fe1441dea40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe134011fa0 tx=0x7fe134003aa0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.665+0000 7fe148c00640 1 -- 172.21.15.67:0/3490934918 shutdown_connections 2024-03-09T09:08:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.665+0000 7fe148c00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12806bea0 0x7fe12806e360 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.665+0000 7fe148c00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1440c1f70 0x7fe1441e0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.665+0000 7fe148c00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe124004fe0 0x7fe1440c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.665+0000 7fe148c00640 1 --2- 172.21.15.67:0/3490934918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe124004230 0x7fe1441dea40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.665+0000 7fe148c00640 1 -- 172.21.15.67:0/3490934918 >> 172.21.15.67:0/3490934918 conn(0x7fe1440c7300 msgr2=0x7fe144072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:50.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.665+0000 7fe148c00640 1 -- 172.21.15.67:0/3490934918 shutdown_connections 2024-03-09T09:08:50.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:50.665+0000 7fe148c00640 1 -- 172.21.15.67:0/3490934918 wait complete. 2024-03-09T09:08:53.088 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 9019c6ce640 v0.31 2024-03-09T09:08:53.103 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.31 2024-03-09T09:08:53.199 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:08:53.199 INFO:tasks.workunit.client.0.smithi067.stdout:v0.32 2024-03-09T09:08:53.200 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 32 -eq 48 ']' 2024-03-09T09:08:53.200 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.32 2024-03-09T09:08:53.200 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.32 2024-03-09T09:08:53.200 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.32 2024-03-09T09:08:53.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:08:53.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.932+0000 7f2e61a00640 1 Processor -- start 2024-03-09T09:08:53.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.932+0000 7f2e61a00640 1 -- start start 2024-03-09T09:08:53.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.933+0000 7f2e61a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2e5c150180 0x7f2e5c0c2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:53.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.933+0000 7f2e61a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2e5c0c4270 0x7f2e5c0c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:53.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.933+0000 7f2e61a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5c0c2ca0 con 0x7f2e5c0a7040 2024-03-09T09:08:53.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.933+0000 7f2e61a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5c152ce0 con 0x7f2e5c0c4270 2024-03-09T09:08:53.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.933+0000 7f2e61a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5c0c29d0 con 0x7f2e5c150180 2024-03-09T09:08:53.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.934+0000 7f2e5b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2e5c0c4270 0x7f2e5c0c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:53.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.934+0000 7f2e5a000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2e5c0a7040 0x7f2e5c06f270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55128/0 (socket says 172.21.15.67:55128) 2024-03-09T09:08:53.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.934+0000 7f2e5a000640 1 -- 172.21.15.67:0/1732461416 learned_addr learned my addr 172.21.15.67:0/1732461416 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:08:53.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.934+0000 7f2e5aa00640 1 --2- 172.21.15.67:0/1732461416 >> v2:172.21.15.134:3300/0 conn(0x7f2e5c150180 0x7f2e5c0c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:53.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.934+0000 7f2e5b400640 1 -- 172.21.15.67:0/1732461416 >> v2:172.21.15.134:3300/0 conn(0x7f2e5c150180 msgr2=0x7f2e5c0c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:53.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.934+0000 7f2e5b400640 1 --2- 172.21.15.67:0/1732461416 >> v2:172.21.15.134:3300/0 conn(0x7f2e5c150180 0x7f2e5c0c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:53.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.934+0000 7f2e5b400640 1 -- 172.21.15.67:0/1732461416 >> v1:172.21.15.67:6789/0 conn(0x7f2e5c0a7040 legacy=0x7f2e5c06f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:53.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.934+0000 7f2e5b400640 1 -- 172.21.15.67:0/1732461416 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c153a60 con 0x7f2e5c0c4270 2024-03-09T09:08:53.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.936+0000 7f2e5b400640 1 --2- 172.21.15.67:0/1732461416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e5c0c4270 0x7f2e5c0c46a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0670d0 tx=0x7f2e5402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44bd5e52c636d72 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:53.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.936+0000 7f2e59600640 1 -- 172.21.15.67:0/1732461416 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e540315a0 con 0x7f2e5c0c4270 2024-03-09T09:08:53.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.936+0000 7f2e59600640 1 -- 172.21.15.67:0/1732461416 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e54037070 con 0x7f2e5c0c4270 2024-03-09T09:08:53.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.936+0000 7f2e59600640 1 -- 172.21.15.67:0/1732461416 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54031960 con 0x7f2e5c0c4270 2024-03-09T09:08:53.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.937+0000 7f2e61a00640 1 -- 172.21.15.67:0/1732461416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e5c0c4270 msgr2=0x7f2e5c0c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:53.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.937+0000 7f2e61a00640 1 --2- 172.21.15.67:0/1732461416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e5c0c4270 0x7f2e5c0c46a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0670d0 tx=0x7f2e5402b840 comp rx=0 tx=0).stop 2024-03-09T09:08:53.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.937+0000 7f2e61a00640 1 -- 172.21.15.67:0/1732461416 shutdown_connections 2024-03-09T09:08:53.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.937+0000 7f2e61a00640 1 --2- 172.21.15.67:0/1732461416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e5c0c4270 0x7f2e5c0c46a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:53.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.937+0000 7f2e61a00640 1 --2- 172.21.15.67:0/1732461416 >> v2:172.21.15.134:3300/0 conn(0x7f2e5c150180 0x7f2e5c0c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:53.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.937+0000 7f2e61a00640 1 -- 172.21.15.67:0/1732461416 >> 172.21.15.67:0/1732461416 conn(0x7f2e5c0c7a90 msgr2=0x7f2e5c0c7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:53.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.937+0000 7f2e61a00640 1 -- 172.21.15.67:0/1732461416 shutdown_connections 2024-03-09T09:08:53.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.937+0000 7f2e61a00640 1 -- 172.21.15.67:0/1732461416 wait complete. 2024-03-09T09:08:53.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.939+0000 7f2e61a00640 1 Processor -- start 2024-03-09T09:08:53.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.940+0000 7f2e61a00640 1 -- start start 2024-03-09T09:08:53.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.940+0000 7f2e61a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e5c0a7040 0x7f2e5c1ecc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:53.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.940+0000 7f2e61a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e5c0c4270 0x7f2e5c1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:53.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e61a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e5c150180 0x7f2e5c0a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:53.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e61a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c153540 con 0x7f2e5c0a7040 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e61a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c153260 con 0x7f2e5c0c4270 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e61a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c06f980 con 0x7f2e5c150180 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e5aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e5c0a7040 0x7f2e5c1ecc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e5aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e5c0a7040 0x7f2e5c1ecc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55318/0 (socket says 172.21.15.67:55318) 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e5aa00640 1 -- 172.21.15.67:0/3135241062 learned_addr learned my addr 172.21.15.67:0/3135241062 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e5a000640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e5c0c4270 0x7f2e5c1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e5b400640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e5c150180 0x7f2e5c0a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e5aa00640 1 -- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e5c150180 msgr2=0x7f2e5c0a83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e5aa00640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e5c150180 0x7f2e5c0a83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e5aa00640 1 -- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e5c0c4270 msgr2=0x7f2e5c1521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e5aa00640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e5c0c4270 0x7f2e5c1521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e5aa00640 1 -- 172.21.15.67:0/3135241062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c1eede0 con 0x7f2e5c0a7040 2024-03-09T09:08:53.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.941+0000 7f2e5b400640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e5c150180 0x7f2e5c0a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:08:53.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.942+0000 7f2e5aa00640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e5c0a7040 0x7f2e5c1ecc40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c007f20 tx=0x7f2e4c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:53.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.942+0000 7f2e52a00640 1 -- 172.21.15.67:0/3135241062 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c014620 con 0x7f2e5c0a7040 2024-03-09T09:08:53.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.942+0000 7f2e52a00640 1 -- 172.21.15.67:0/3135241062 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c019070 con 0x7f2e5c0a7040 2024-03-09T09:08:53.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.942+0000 7f2e61a00640 1 -- 172.21.15.67:0/3135241062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5c148c30 con 0x7f2e5c0a7040 2024-03-09T09:08:53.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.942+0000 7f2e52a00640 1 -- 172.21.15.67:0/3135241062 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c014a00 con 0x7f2e5c0a7040 2024-03-09T09:08:53.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.942+0000 7f2e61a00640 1 -- 172.21.15.67:0/3135241062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5c1ef080 con 0x7f2e5c0a7040 2024-03-09T09:08:53.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.943+0000 7f2e61a00640 1 -- 172.21.15.67:0/3135241062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e5c075a00 con 0x7f2e5c0a7040 2024-03-09T09:08:53.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.944+0000 7f2e52a00640 1 -- 172.21.15.67:0/3135241062 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2e4c014ba0 con 0x7f2e5c0a7040 2024-03-09T09:08:53.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.945+0000 7f2e52a00640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2e3806b500 0x7f2e3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:53.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.945+0000 7f2e52a00640 1 -- 172.21.15.67:0/3135241062 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f2e4c08aff0 con 0x7f2e5c0a7040 2024-03-09T09:08:53.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.946+0000 7f2e5a000640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2e3806b500 0x7f2e3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:53.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.947+0000 7f2e5a000640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2e3806b500 0x7f2e3806d9c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c1e9610 tx=0x7f2e44009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:53.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:53.950+0000 7f2e52a00640 1 -- 172.21.15.67:0/3135241062 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2e4c0599a0 con 0x7f2e5c0a7040 2024-03-09T09:08:54.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.272+0000 7f2e61a00640 1 -- 172.21.15.67:0/3135241062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2e5c0c5f30 con 0x7f2e5c0a7040 2024-03-09T09:08:54.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.272+0000 7f2e52a00640 1 -- 172.21.15.67:0/3135241062 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f2e4c0592a0 con 0x7f2e5c0a7040 2024-03-09T09:08:54.277 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:08:54.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 -- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2e3806b500 msgr2=0x7f2e3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2e3806b500 0x7f2e3806d9c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c1e9610 tx=0x7f2e44009040 comp rx=0 tx=0).stop 2024-03-09T09:08:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 -- 172.21.15.67:0/3135241062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e5c0a7040 msgr2=0x7f2e5c1ecc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e5c0a7040 0x7f2e5c1ecc40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c007f20 tx=0x7f2e4c00e570 comp rx=0 tx=0).stop 2024-03-09T09:08:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 -- 172.21.15.67:0/3135241062 shutdown_connections 2024-03-09T09:08:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2e3806b500 0x7f2e3806d9c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e5c150180 0x7f2e5c0a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e5c0c4270 0x7f2e5c1521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 --2- 172.21.15.67:0/3135241062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e5c0a7040 0x7f2e5c1ecc40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 -- 172.21.15.67:0/3135241062 >> 172.21.15.67:0/3135241062 conn(0x7f2e5c0c7a90 msgr2=0x7f2e5c0c6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 -- 172.21.15.67:0/3135241062 shutdown_connections 2024-03-09T09:08:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.275+0000 7f2e61a00640 1 -- 172.21.15.67:0/3135241062 wait complete. 2024-03-09T09:08:54.299 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:08:54.299 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:08:54.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:08:54.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.525+0000 7f2609e00640 1 Processor -- start 2024-03-09T09:08:54.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.525+0000 7f2609e00640 1 -- start start 2024-03-09T09:08:54.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.526+0000 7f2609e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f260414fd90 0x7f2604156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.526+0000 7f2609e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f26040c9380 0x7f26040acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.526+0000 7f2609e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2604156bc0 con 0x7f26040ad440 2024-03-09T09:08:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.526+0000 7f2609e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260406ba60 con 0x7f26040c9380 2024-03-09T09:08:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.526+0000 7f2609e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26040c9890 con 0x7f260414fd90 2024-03-09T09:08:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.526+0000 7f2609400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f26040ad440 0x7f260406c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55140/0 (socket says 172.21.15.67:55140) 2024-03-09T09:08:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.526+0000 7f2609400640 1 -- 172.21.15.67:0/3931310901 learned_addr learned my addr 172.21.15.67:0/3931310901 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:08:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2603e00640 1 --2- 172.21.15.67:0/3931310901 >> v2:172.21.15.134:3300/0 conn(0x7f26040c9380 0x7f26040acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2608a00640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f260414fd90 0x7f2604156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711362530 0 0) 0x7f25fc001490 con 0x7f26040ad440 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ec003520 con 0x7f26040ad440 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2466928083 0 0) 0x7f25ec003520 con 0x7f26040ad440 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f260414fd90 msgr2=0x7f2604156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2603400640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f260414fd90 0x7f2604156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 >> v2:172.21.15.134:3300/0 conn(0x7f26040c9380 msgr2=0x7f26040acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2603400640 1 --2- 172.21.15.67:0/3931310901 >> v2:172.21.15.134:3300/0 conn(0x7f26040c9380 0x7f26040acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040c5f80 con 0x7f26040ad440 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f25fc001bc0 con 0x7f26040ad440 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.527+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 >> v1:172.21.15.67:6789/0 conn(0x7f26040ad440 legacy=0x7f260406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2603400640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 0x7f25ec003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2603400640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f260414fd90 0x7f25ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2603400640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25ec004d40 0x7f25ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260406ba60 con 0x7f25ec003a20 2024-03-09T09:08:54.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040c9890 con 0x7f25ec004d40 2024-03-09T09:08:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec00bac0 con 0x7f260414fd90 2024-03-09T09:08:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2608a00640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 0x7f25ec003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2603e00640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f260414fd90 0x7f25ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2608a00640 1 -- 172.21.15.67:0/3931310901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f260414fd90 msgr2=0x7f25ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2608a00640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f260414fd90 0x7f25ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2608a00640 1 -- 172.21.15.67:0/3931310901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25ec004d40 msgr2=0x7f25ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2608a00640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25ec004d40 0x7f25ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2608a00640 1 -- 172.21.15.67:0/3931310901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec003520 con 0x7f25ec003a20 2024-03-09T09:08:54.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.528+0000 7f2603e00640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f260414fd90 0x7f25ec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:08:54.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.530+0000 7f2608a00640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 0x7f25ec003e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009af0 tx=0x7f25ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a259ebacf4e818 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.530+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0039070 con 0x7f25ec003a20 2024-03-09T09:08:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.530+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f002c6b0 con 0x7f25ec003a20 2024-03-09T09:08:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.530+0000 7f2603400640 1 -- 172.21.15.67:0/3931310901 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0009690 con 0x7f25ec003a20 2024-03-09T09:08:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.531+0000 7f2609e00640 1 -- 172.21.15.67:0/3931310901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 msgr2=0x7f25ec003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.531+0000 7f2609e00640 1 --2- 172.21.15.67:0/3931310901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 0x7f25ec003e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009af0 tx=0x7f25ec005830 comp rx=0 tx=0).stop 2024-03-09T09:08:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.531+0000 7f2609400640 1 -- 172.21.15.67:0/3931310901 reap_dead start 2024-03-09T09:08:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.531+0000 7f2609e00640 1 -- 172.21.15.67:0/3931310901 shutdown_connections 2024-03-09T09:08:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.531+0000 7f2609e00640 1 -- 172.21.15.67:0/3931310901 >> 172.21.15.67:0/3931310901 conn(0x7f26040cb480 msgr2=0x7f26040cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.531+0000 7f2609e00640 1 -- 172.21.15.67:0/3931310901 shutdown_connections 2024-03-09T09:08:54.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.531+0000 7f2609e00640 1 -- 172.21.15.67:0/3931310901 wait complete. 2024-03-09T09:08:54.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.533+0000 7f2609e00640 1 Processor -- start 2024-03-09T09:08:54.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.533+0000 7f2609e00640 1 -- start start 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.534+0000 7f2609e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 0x7f26041ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.534+0000 7f2609e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26040c4fb0 0x7f26041e18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.534+0000 7f2609e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26041589e0 0x7f2604158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.534+0000 7f2609e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260406bcb0 con 0x7f25ec003a20 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.534+0000 7f2609e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040c6520 con 0x7f26040c4fb0 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.534+0000 7f2609e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2604156680 con 0x7f26041589e0 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.534+0000 7f2608a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 0x7f26041ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.534+0000 7f2608a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 0x7f26041ec7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55342/0 (socket says 172.21.15.67:55342) 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.534+0000 7f2608a00640 1 -- 172.21.15.67:0/3575284417 learned_addr learned my addr 172.21.15.67:0/3575284417 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.535+0000 7f2609400640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26041589e0 0x7f2604158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.535+0000 7f2603e00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26040c4fb0 0x7f26041e18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.535+0000 7f2608a00640 1 -- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26041589e0 msgr2=0x7f2604158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.535+0000 7f2608a00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26041589e0 0x7f2604158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.535+0000 7f2608a00640 1 -- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26040c4fb0 msgr2=0x7f26041e18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.535+0000 7f2608a00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26040c4fb0 0x7f26041e18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.535+0000 7f2608a00640 1 -- 172.21.15.67:0/3575284417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26041570f0 con 0x7f25ec003a20 2024-03-09T09:08:54.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.535+0000 7f2603e00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26040c4fb0 0x7f26041e18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:08:54.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.535+0000 7f2608a00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 0x7f26041ec7e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f25f0037040 tx=0x7f25f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:54.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.536+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0039050 con 0x7f25ec003a20 2024-03-09T09:08:54.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.536+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f0045070 con 0x7f25ec003a20 2024-03-09T09:08:54.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.536+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0003a50 con 0x7f25ec003a20 2024-03-09T09:08:54.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.536+0000 7f2609e00640 1 -- 172.21.15.67:0/3575284417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26041573d0 con 0x7f25ec003a20 2024-03-09T09:08:54.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.536+0000 7f2609e00640 1 -- 172.21.15.67:0/3575284417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26041e1e20 con 0x7f25ec003a20 2024-03-09T09:08:54.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.537+0000 7f2609e00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041e0150 0x7f26041ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.537+0000 7f2609e00640 1 -- 172.21.15.67:0/3575284417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2604075a00 con 0x7f26041e0150 2024-03-09T09:08:54.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.538+0000 7f2603e00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041e0150 0x7f26041ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.538+0000 7f2603e00640 1 -- 172.21.15.67:0/3575284417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26041df260 con 0x7f26041e0150 2024-03-09T09:08:54.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.538+0000 7f2603e00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041e0150 0x7f26041ef440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26040acb50 tx=0x7f25f8011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:54.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.539+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f25f0003bf0 con 0x7f25ec003a20 2024-03-09T09:08:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.540+0000 7f2600c00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f25e406b4d0 0x7f25e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.540+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f25f00ac7b0 con 0x7f25ec003a20 2024-03-09T09:08:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.541+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f800f8c0 con 0x7f26041e0150 2024-03-09T09:08:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.541+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f25f800fa60 con 0x7f26041e0150 2024-03-09T09:08:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.541+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041e0150 msgr2=0x7f26041ef440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.541+0000 7f2600c00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041e0150 0x7f26041ef440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26040acb50 tx=0x7f25f8011a40 comp rx=0 tx=0).stop 2024-03-09T09:08:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.541+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041e0150 msgr2=0x7f26041ef440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:08:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.541+0000 7f2600c00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041e0150 0x7f26041ef440 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26040acb50 tx=0x7f25f8011a40 comp rx=0 tx=0).stop 2024-03-09T09:08:54.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.541+0000 7f2609400640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f25e406b4d0 0x7f25e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.543+0000 7f2609400640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f25e406b4d0 0x7f25e406d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f260408d730 tx=0x7f25fc0021f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:54.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.555+0000 7f2609e00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041ef980 0x7f26041efd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.555+0000 7f2609e00640 1 -- 172.21.15.67:0/3575284417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26041f02a0 con 0x7f26041ef980 2024-03-09T09:08:54.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.555+0000 7f2603e00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041ef980 0x7f26041efd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.556+0000 7f2603e00640 1 -- 172.21.15.67:0/3575284417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f260404f3c0 con 0x7f26041ef980 2024-03-09T09:08:54.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.556+0000 7f2603e00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041ef980 0x7f26041efd60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f26040acb50 tx=0x7f25f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:54.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.556+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8028070 con 0x7f26041ef980 2024-03-09T09:08:54.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f25f8026070 con 0x7f26041ef980 2024-03-09T09:08:54.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041ef980 msgr2=0x7f26041efd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2600c00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041ef980 0x7f26041efd60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f26040acb50 tx=0x7f25f8004730 comp rx=0 tx=0).stop 2024-03-09T09:08:54.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2600c00640 1 -- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041ef980 msgr2=0x7f26041efd60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:08:54.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2600c00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26041ef980 0x7f26041efd60 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f26040acb50 tx=0x7f25f8004730 comp rx=0 tx=0).stop 2024-03-09T09:08:54.561 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:08:54.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:08:54.561 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 118, 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:54.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:54.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:08:54.564 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:08:54.565 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.567 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:54.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:08:54.569 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.572 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.574 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:08:54.575 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:08:54.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2609e00640 1 -- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f25e406b4d0 msgr2=0x7f25e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2609e00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f25e406b4d0 0x7f25e406d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f260408d730 tx=0x7f25fc0021f0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2609e00640 1 -- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 msgr2=0x7f26041ec7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2609e00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 0x7f26041ec7e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f25f0037040 tx=0x7f25f0004730 comp rx=0 tx=0).stop 2024-03-09T09:08:54.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2609400640 1 -- 172.21.15.67:0/3575284417 reap_dead start 2024-03-09T09:08:54.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2609e00640 1 -- 172.21.15.67:0/3575284417 shutdown_connections 2024-03-09T09:08:54.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2609e00640 1 --2- 172.21.15.67:0/3575284417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25ec003a20 0x7f26041ec7e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2609e00640 1 -- 172.21.15.67:0/3575284417 >> 172.21.15.67:0/3575284417 conn(0x7f26040cb480 msgr2=0x7f260414bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:54.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.557+0000 7f2609e00640 1 -- 172.21.15.67:0/3575284417 shutdown_connections 2024-03-09T09:08:54.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.558+0000 7f2609e00640 1 -- 172.21.15.67:0/3575284417 wait complete. 2024-03-09T09:08:54.579 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:08:54.579 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:08:54.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:08:54.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.813+0000 7f872ca00640 1 Processor -- start 2024-03-09T09:08:54.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.813+0000 7f872ca00640 1 -- start start 2024-03-09T09:08:54.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.814+0000 7f872ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f872814fd90 0x7f8728156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.814+0000 7f872ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f872806c1d0 0x7f872806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.814+0000 7f872ca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8728156bc0 con 0x7f87280c9380 2024-03-09T09:08:54.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.814+0000 7f872ca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f872806ba60 con 0x7f872814fd90 2024-03-09T09:08:54.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.814+0000 7f872ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87280c9890 con 0x7f872806c1d0 2024-03-09T09:08:54.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.815+0000 7f8726800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f872814fd90 0x7f8728156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.815+0000 7f871fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f87280c9380 0x7f87280acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55156/0 (socket says 172.21.15.67:55156) 2024-03-09T09:08:54.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.815+0000 7f871fe00640 1 -- 172.21.15.67:0/3182177507 learned_addr learned my addr 172.21.15.67:0/3182177507 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:08:54.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.815+0000 7f8727200640 1 --2- 172.21.15.67:0/3182177507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f872806c1d0 0x7f872806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.815+0000 7f8726800640 1 -- 172.21.15.67:0/3182177507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f872806c1d0 msgr2=0x7f872806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:54.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.815+0000 7f8726800640 1 --2- 172.21.15.67:0/3182177507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f872806c1d0 0x7f872806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.815+0000 7f8726800640 1 -- 172.21.15.67:0/3182177507 >> v1:172.21.15.67:6789/0 conn(0x7f87280c9380 legacy=0x7f87280acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.815+0000 7f8726800640 1 -- 172.21.15.67:0/3182177507 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87280c5f80 con 0x7f872814fd90 2024-03-09T09:08:54.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.815+0000 7f8727200640 1 --2- 172.21.15.67:0/3182177507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f872806c1d0 0x7f872806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:08:54.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.815+0000 7f8725e00640 1 -- 172.21.15.67:0/3182177507 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507164836 0 0) 0x7f8710001110 con 0x7f87280c9380 2024-03-09T09:08:54.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.816+0000 7f8726800640 1 --2- 172.21.15.67:0/3182177507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f872814fd90 0x7f8728156680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f87180097a0 tx=0x7f871802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1de890b8a77e796 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:54.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.817+0000 7f8725e00640 1 -- 172.21.15.67:0/3182177507 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718038070 con 0x7f872814fd90 2024-03-09T09:08:54.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.817+0000 7f8725e00640 1 -- 172.21.15.67:0/3182177507 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8718031700 con 0x7f872814fd90 2024-03-09T09:08:54.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.817+0000 7f8725e00640 1 -- 172.21.15.67:0/3182177507 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718031ad0 con 0x7f872814fd90 2024-03-09T09:08:54.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.817+0000 7f872ca00640 1 -- 172.21.15.67:0/3182177507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f872814fd90 msgr2=0x7f8728156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.817+0000 7f872ca00640 1 --2- 172.21.15.67:0/3182177507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f872814fd90 0x7f8728156680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f87180097a0 tx=0x7f871802b7a0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.818+0000 7f872ca00640 1 -- 172.21.15.67:0/3182177507 shutdown_connections 2024-03-09T09:08:54.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.818+0000 7f872ca00640 1 --2- 172.21.15.67:0/3182177507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f872806c1d0 0x7f872806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.818+0000 7f872ca00640 1 --2- 172.21.15.67:0/3182177507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f872814fd90 0x7f8728156680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.818+0000 7f872ca00640 1 -- 172.21.15.67:0/3182177507 >> 172.21.15.67:0/3182177507 conn(0x7f87280cb480 msgr2=0x7f87280cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:54.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.818+0000 7f872ca00640 1 -- 172.21.15.67:0/3182177507 shutdown_connections 2024-03-09T09:08:54.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.818+0000 7f872ca00640 1 -- 172.21.15.67:0/3182177507 wait complete. 2024-03-09T09:08:54.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.820+0000 7f872ca00640 1 Processor -- start 2024-03-09T09:08:54.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.820+0000 7f872ca00640 1 -- start start 2024-03-09T09:08:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.821+0000 7f872ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f872806c1d0 0x7f8728159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.821+0000 7f872ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87280c9380 0x7f87280acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.821+0000 7f872ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f872814fd90 0x7f87280ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.821+0000 7f872ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872806bcb0 con 0x7f872814fd90 2024-03-09T09:08:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.821+0000 7f872ca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87280c6520 con 0x7f872806c1d0 2024-03-09T09:08:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.821+0000 7f872ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87280ad610 con 0x7f87280c9380 2024-03-09T09:08:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.821+0000 7f8727200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f872814fd90 0x7f87280ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.821+0000 7f8727200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f872814fd90 0x7f87280ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55374/0 (socket says 172.21.15.67:55374) 2024-03-09T09:08:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.821+0000 7f8727200640 1 -- 172.21.15.67:0/209156075 learned_addr learned my addr 172.21.15.67:0/209156075 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:08:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.822+0000 7f871fe00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87280c9380 0x7f87280acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.822+0000 7f8726800640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f872806c1d0 0x7f8728159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.822+0000 7f8727200640 1 -- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87280c9380 msgr2=0x7f87280acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.822+0000 7f8727200640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87280c9380 0x7f87280acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.822+0000 7f8727200640 1 -- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f872806c1d0 msgr2=0x7f8728159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.822+0000 7f8727200640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f872806c1d0 0x7f8728159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.822+0000 7f8727200640 1 -- 172.21.15.67:0/209156075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87281ef970 con 0x7f872814fd90 2024-03-09T09:08:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.822+0000 7f8727200640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f872814fd90 0x7f87280ae070 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f872000cd50 tx=0x7f872000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.823+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000f680 con 0x7f872814fd90 2024-03-09T09:08:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.823+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872000f840 con 0x7f872814fd90 2024-03-09T09:08:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.823+0000 7f872ca00640 1 -- 172.21.15.67:0/209156075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87281590f0 con 0x7f872814fd90 2024-03-09T09:08:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.823+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000fc30 con 0x7f872814fd90 2024-03-09T09:08:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.823+0000 7f872ca00640 1 -- 172.21.15.67:0/209156075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87281596b0 con 0x7f872814fd90 2024-03-09T09:08:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.824+0000 7f872ca00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87280c1930 0x7f8728148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.824+0000 7f872ca00640 1 -- 172.21.15.67:0/209156075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8728158330 con 0x7f87280c1930 2024-03-09T09:08:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.825+0000 7f8726800640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87280c1930 0x7f8728148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.826+0000 7f8726800640 1 -- 172.21.15.67:0/209156075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87281598b0 con 0x7f87280c1930 2024-03-09T09:08:54.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.826+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f872000e3e0 con 0x7f872814fd90 2024-03-09T09:08:54.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.826+0000 7f8726800640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87280c1930 0x7f8728148890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87280ae040 tx=0x7f87180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.827+0000 7f871ea00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f870406b4d0 0x7f870406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.827+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f872008b410 con 0x7f872814fd90 2024-03-09T09:08:54.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.827+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718038040 con 0x7f87280c1930 2024-03-09T09:08:54.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.828+0000 7f871fe00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f870406b4d0 0x7f870406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.828+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f8718031eb0 con 0x7f87280c1930 2024-03-09T09:08:54.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.828+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87280c1930 msgr2=0x7f8728148890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.828+0000 7f871ea00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87280c1930 0x7f8728148890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87280ae040 tx=0x7f87180046c0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.828+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87280c1930 msgr2=0x7f8728148890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:08:54.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.828+0000 7f871ea00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87280c1930 0x7f8728148890 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87280ae040 tx=0x7f87180046c0 comp rx=0 tx=0).stop 2024-03-09T09:08:54.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.829+0000 7f871fe00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f870406b4d0 0x7f870406d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f87280ad4f0 tx=0x7f8710007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:54.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.842+0000 7f872ca00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8728148dd0 0x7f8728156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:54.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.842+0000 7f872ca00640 1 -- 172.21.15.67:0/209156075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872804f3c0 con 0x7f8728148dd0 2024-03-09T09:08:54.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.843+0000 7f8726800640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8728148dd0 0x7f8728156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:54.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.844+0000 7f8726800640 1 -- 172.21.15.67:0/209156075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f872806d2d0 con 0x7f8728148dd0 2024-03-09T09:08:54.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.844+0000 7f8726800640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8728148dd0 0x7f8728156fd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f871802b7e0 tx=0x7f8718031000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:54.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.844+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871804c070 con 0x7f8728148dd0 2024-03-09T09:08:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.845+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f871803de50 con 0x7f8728148dd0 2024-03-09T09:08:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.845+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8728148dd0 msgr2=0x7f8728156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.845+0000 7f871ea00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8728148dd0 0x7f8728156fd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f871802b7e0 tx=0x7f8718031000 comp rx=0 tx=0).stop 2024-03-09T09:08:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.845+0000 7f871ea00640 1 -- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8728148dd0 msgr2=0x7f8728156fd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:08:54.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.845+0000 7f871ea00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8728148dd0 0x7f8728156fd0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f871802b7e0 tx=0x7f8718031000 comp rx=0 tx=0).stop 2024-03-09T09:08:54.850 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:08:54.850 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:08:54.850 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:08:54.850 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:08:54.850 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 118, 2024-03-09T09:08:54.850 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:08:54.850 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:08:54.850 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:08:54.850 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:08:54.850 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:54.851 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:08:54.851 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:08:54.851 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:08:54.851 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:08:54.851 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:54.851 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:54.851 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:54.851 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:54.851 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:54.851 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:54.852 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:08:54.853 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:54.854 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:54.855 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:08:54.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.857 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:08:54.858 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.859 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:54.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:08:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:08:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:54.862 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:08:54.863 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:08:54.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.846+0000 7f872ca00640 1 -- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f870406b4d0 msgr2=0x7f870406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.846+0000 7f872ca00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f870406b4d0 0x7f870406d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f87280ad4f0 tx=0x7f8710007c00 comp rx=0 tx=0).stop 2024-03-09T09:08:54.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.846+0000 7f872ca00640 1 -- 172.21.15.67:0/209156075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f872814fd90 msgr2=0x7f87280ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:54.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.846+0000 7f872ca00640 1 --2- 172.21.15.67:0/209156075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f872814fd90 0x7f87280ae070 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f872000cd50 tx=0x7f872000a650 comp rx=0 tx=0).stop 2024-03-09T09:08:54.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.846+0000 7f8727200640 1 -- 172.21.15.67:0/209156075 reap_dead start 2024-03-09T09:08:54.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.846+0000 7f8727200640 1 -- 172.21.15.67:0/209156075 reap_dead start 2024-03-09T09:08:54.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.846+0000 7f872ca00640 1 -- 172.21.15.67:0/209156075 shutdown_connections 2024-03-09T09:08:54.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.846+0000 7f872ca00640 1 -- 172.21.15.67:0/209156075 >> 172.21.15.67:0/209156075 conn(0x7f87280cb480 msgr2=0x7f8728153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:54.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.846+0000 7f872ca00640 1 -- 172.21.15.67:0/209156075 shutdown_connections 2024-03-09T09:08:54.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:54.846+0000 7f872ca00640 1 -- 172.21.15.67:0/209156075 wait complete. 2024-03-09T09:08:54.866 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:08:54.866 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:08:54.866 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:08:55.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.064+0000 7fe23f400640 1 Processor -- start 2024-03-09T09:08:55.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.064+0000 7fe23f400640 1 -- start start 2024-03-09T09:08:55.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.065+0000 7fe23f400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe2380730c0 0x7fe2380aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.065+0000 7fe23f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2380734a0 0x7fe2380c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.066+0000 7fe23f400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe238152e00 con 0x7fe2380c33c0 2024-03-09T09:08:55.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.066+0000 7fe23f400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2380aa960 con 0x7fe2380730c0 2024-03-09T09:08:55.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.066+0000 7fe23f400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2381530e0 con 0x7fe2380734a0 2024-03-09T09:08:55.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.066+0000 7fe23d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe2380c33c0 0x7fe23806d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55168/0 (socket says 172.21.15.67:55168) 2024-03-09T09:08:55.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.066+0000 7fe23d400640 1 -- 172.21.15.67:0/517914410 learned_addr learned my addr 172.21.15.67:0/517914410 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:08:55.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.066+0000 7fe23ca00640 1 --2- 172.21.15.67:0/517914410 >> v2:172.21.15.134:3300/0 conn(0x7fe2380730c0 0x7fe2380aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:55.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.066+0000 7fe237e00640 1 --2- 172.21.15.67:0/517914410 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2380734a0 0x7fe2380c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:55.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.067+0000 7fe237400640 1 -- 172.21.15.67:0/517914410 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332540605 0 0) 0x7fe238152e00 con 0x7fe2380c33c0 2024-03-09T09:08:55.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.067+0000 7fe237400640 1 -- 172.21.15.67:0/517914410 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe220003520 con 0x7fe2380c33c0 2024-03-09T09:08:55.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.067+0000 7fe23ca00640 1 -- 172.21.15.67:0/517914410 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2380734a0 msgr2=0x7fe2380c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:55.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.067+0000 7fe23ca00640 1 --2- 172.21.15.67:0/517914410 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2380734a0 0x7fe2380c2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.067+0000 7fe237400640 1 -- 172.21.15.67:0/517914410 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3881563617 0 0) 0x7fe220003520 con 0x7fe2380c33c0 2024-03-09T09:08:55.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.067+0000 7fe23ca00640 1 -- 172.21.15.67:0/517914410 >> v1:172.21.15.67:6789/0 conn(0x7fe2380c33c0 legacy=0x7fe23806d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.067+0000 7fe23ca00640 1 -- 172.21.15.67:0/517914410 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2381533c0 con 0x7fe2380730c0 2024-03-09T09:08:55.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.068+0000 7fe23ca00640 1 --2- 172.21.15.67:0/517914410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2380730c0 0x7fe2380aa420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0097a0 tx=0x7fe22c02b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a16b948ab6453b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:55.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.069+0000 7fe237400640 1 -- 172.21.15.67:0/517914410 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c037070 con 0x7fe2380730c0 2024-03-09T09:08:55.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.069+0000 7fe237400640 1 -- 172.21.15.67:0/517914410 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c030510 con 0x7fe2380730c0 2024-03-09T09:08:55.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.069+0000 7fe237400640 1 -- 172.21.15.67:0/517914410 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2300019f0 con 0x7fe2380730c0 2024-03-09T09:08:55.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.069+0000 7fe23f400640 1 -- 172.21.15.67:0/517914410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2380730c0 msgr2=0x7fe2380aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.069+0000 7fe23f400640 1 --2- 172.21.15.67:0/517914410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2380730c0 0x7fe2380aa420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0097a0 tx=0x7fe22c02b710 comp rx=0 tx=0).stop 2024-03-09T09:08:55.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.069+0000 7fe23f400640 1 -- 172.21.15.67:0/517914410 shutdown_connections 2024-03-09T09:08:55.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.069+0000 7fe23f400640 1 --2- 172.21.15.67:0/517914410 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2380734a0 0x7fe2380c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.069+0000 7fe23f400640 1 --2- 172.21.15.67:0/517914410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2380730c0 0x7fe2380aa420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.069+0000 7fe23f400640 1 -- 172.21.15.67:0/517914410 >> 172.21.15.67:0/517914410 conn(0x7fe2380c7370 msgr2=0x7fe2380c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:55.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.069+0000 7fe23f400640 1 -- 172.21.15.67:0/517914410 shutdown_connections 2024-03-09T09:08:55.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.069+0000 7fe23f400640 1 -- 172.21.15.67:0/517914410 wait complete. 2024-03-09T09:08:55.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.071+0000 7fe23f400640 1 Processor -- start 2024-03-09T09:08:55.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.071+0000 7fe23f400640 1 -- start start 2024-03-09T09:08:55.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.072+0000 7fe23f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2380730c0 0x7fe2381ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.072+0000 7fe23f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2380734a0 0x7fe2380c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.072+0000 7fe23f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2380c33c0 0x7fe238146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.072+0000 7fe23f400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380aaef0 con 0x7fe2380734a0 2024-03-09T09:08:55.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.072+0000 7fe23f400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe238153980 con 0x7fe2380730c0 2024-03-09T09:08:55.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.072+0000 7fe23f400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380ab070 con 0x7fe2380c33c0 2024-03-09T09:08:55.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe237e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2380734a0 0x7fe2380c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe237e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2380734a0 0x7fe2380c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55388/0 (socket says 172.21.15.67:55388) 2024-03-09T09:08:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe237e00640 1 -- 172.21.15.67:0/1749407517 learned_addr learned my addr 172.21.15.67:0/1749407517 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:08:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe23d400640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2380c33c0 0x7fe238146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe23ca00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2380730c0 0x7fe2381ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe237e00640 1 -- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2380c33c0 msgr2=0x7fe238146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe237e00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2380c33c0 0x7fe238146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe237e00640 1 -- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2380730c0 msgr2=0x7fe2381ef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe237e00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2380730c0 0x7fe2381ef8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe237e00640 1 -- 172.21.15.67:0/1749407517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2381499c0 con 0x7fe2380734a0 2024-03-09T09:08:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe23ca00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2380730c0 0x7fe2381ef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:08:55.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.073+0000 7fe237e00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2380734a0 0x7fe2380c0a10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe22400e9b0 tx=0x7fe22400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:55.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.074+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224017070 con 0x7fe2380734a0 2024-03-09T09:08:55.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.074+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22400c9a0 con 0x7fe2380734a0 2024-03-09T09:08:55.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.074+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22400cd70 con 0x7fe2380734a0 2024-03-09T09:08:55.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.074+0000 7fe23f400640 1 -- 172.21.15.67:0/1749407517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe238156830 con 0x7fe2380734a0 2024-03-09T09:08:55.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.074+0000 7fe23f400640 1 -- 172.21.15.67:0/1749407517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe238156df0 con 0x7fe2380734a0 2024-03-09T09:08:55.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.075+0000 7fe23f400640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815fba0 0x7fe238146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.075+0000 7fe23f400640 1 -- 172.21.15.67:0/1749407517 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2381efe30 con 0x7fe23815fba0 2024-03-09T09:08:55.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.076+0000 7fe23ca00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815fba0 0x7fe238146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:55.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.076+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe22401a070 con 0x7fe2380734a0 2024-03-09T09:08:55.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.076+0000 7fe23ca00640 1 -- 172.21.15.67:0/1749407517 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe238075a00 con 0x7fe23815fba0 2024-03-09T09:08:55.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.077+0000 7fe234c00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe21806b4d0 0x7fe21806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.077+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe22408a640 con 0x7fe2380734a0 2024-03-09T09:08:55.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.077+0000 7fe23ca00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815fba0 0x7fe238146a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe238070710 tx=0x7fe22c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:55.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.078+0000 7fe23d400640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe21806b4d0 0x7fe21806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:55.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.078+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c042070 con 0x7fe23815fba0 2024-03-09T09:08:55.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.079+0000 7fe23d400640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe21806b4d0 0x7fe21806d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe230001b60 tx=0x7fe230009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:55.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.079+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fe22c03e660 con 0x7fe23815fba0 2024-03-09T09:08:55.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.079+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815fba0 msgr2=0x7fe238146a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.079+0000 7fe234c00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815fba0 0x7fe238146a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe238070710 tx=0x7fe22c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.079+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815fba0 msgr2=0x7fe238146a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:08:55.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.079+0000 7fe234c00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815fba0 0x7fe238146a50 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe238070710 tx=0x7fe22c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.086+0000 7fe23f400640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815eb50 0x7fe2381eb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.086+0000 7fe23f400640 1 -- 172.21.15.67:0/1749407517 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380ac570 con 0x7fe23815eb50 2024-03-09T09:08:55.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.087+0000 7fe23ca00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815eb50 0x7fe2381eb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:55.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.087+0000 7fe23ca00640 1 -- 172.21.15.67:0/1749407517 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe2380ac290 con 0x7fe23815eb50 2024-03-09T09:08:55.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.088+0000 7fe23ca00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815eb50 0x7fe2381eb970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0097a0 tx=0x7fe22c009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:55.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.088+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c037040 con 0x7fe23815eb50 2024-03-09T09:08:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.089+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fe22c03cb90 con 0x7fe23815eb50 2024-03-09T09:08:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.089+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815eb50 msgr2=0x7fe2381eb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.089+0000 7fe234c00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815eb50 0x7fe2381eb970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0097a0 tx=0x7fe22c009190 comp rx=0 tx=0).stop 2024-03-09T09:08:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.089+0000 7fe234c00640 1 -- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815eb50 msgr2=0x7fe2381eb970 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:08:55.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.089+0000 7fe234c00640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe23815eb50 0x7fe2381eb970 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0097a0 tx=0x7fe22c009190 comp rx=0 tx=0).stop 2024-03-09T09:08:55.094 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:08:55.094 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:08:55.094 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:08:55.094 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:08:55.094 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 118, 2024-03-09T09:08:55.094 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:08:55.094 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:08:55.094 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:08:55.094 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:08:55.094 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:55.094 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:55.095 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:55.096 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:08:55.097 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:08:55.098 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:08:55.098 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:08:55.098 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:08:55.098 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:08:55.098 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:08:55.098 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:08:55.098 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:08:55.098 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:08:55.098 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:08:55.098 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.099 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:08:55.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:55.100 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:08:55.100 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:08:55.100 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:55.100 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:55.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:55.100 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:55.100 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:55.101 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:55.102 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:55.103 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:08:55.104 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:08:55.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:08:55.106 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:08:55.107 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:08:55.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.090+0000 7fe23f400640 1 -- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe21806b4d0 msgr2=0x7fe21806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.090+0000 7fe23f400640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe21806b4d0 0x7fe21806d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe230001b60 tx=0x7fe230009040 comp rx=0 tx=0).stop 2024-03-09T09:08:55.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.090+0000 7fe23f400640 1 -- 172.21.15.67:0/1749407517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2380734a0 msgr2=0x7fe2380c0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.090+0000 7fe23f400640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2380734a0 0x7fe2380c0a10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe22400e9b0 tx=0x7fe22400ee70 comp rx=0 tx=0).stop 2024-03-09T09:08:55.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.090+0000 7fe23d400640 1 -- 172.21.15.67:0/1749407517 reap_dead start 2024-03-09T09:08:55.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.090+0000 7fe23f400640 1 -- 172.21.15.67:0/1749407517 shutdown_connections 2024-03-09T09:08:55.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.090+0000 7fe23f400640 1 --2- 172.21.15.67:0/1749407517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2380734a0 0x7fe2380c0a10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.090+0000 7fe23f400640 1 -- 172.21.15.67:0/1749407517 >> 172.21.15.67:0/1749407517 conn(0x7fe2380c7370 msgr2=0x7fe2380721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:55.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.090+0000 7fe23f400640 1 -- 172.21.15.67:0/1749407517 shutdown_connections 2024-03-09T09:08:55.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.090+0000 7fe23f400640 1 -- 172.21.15.67:0/1749407517 wait complete. 2024-03-09T09:08:55.110 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:08:55.110 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:08:55.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:08:55.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.327+0000 7f0dcc200640 1 Processor -- start 2024-03-09T09:08:55.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.327+0000 7f0dcc200640 1 -- start start 2024-03-09T09:08:55.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.328+0000 7f0dcc200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40c22a0 0x7f0dc4071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.328+0000 7f0dcc200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc4153d40 con 0x7f0dc40ac730 2024-03-09T09:08:55.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.328+0000 7f0dcc200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc40c2790 con 0x7f0dc4157d60 2024-03-09T09:08:55.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.328+0000 7f0dcc200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc4070ee0 con 0x7f0dc40c22a0 2024-03-09T09:08:55.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.328+0000 7f0dc9800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0dc40ac730 0x7f0dc4157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55172/0 (socket says 172.21.15.67:55172) 2024-03-09T09:08:55.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.328+0000 7f0dc9800640 1 -- 172.21.15.67:0/2477814374 learned_addr learned my addr 172.21.15.67:0/2477814374 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:08:55.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.329+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/2477814374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40c22a0 0x7f0dc4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:55.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.329+0000 7f0dc3e00640 1 -- 172.21.15.67:0/2477814374 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724757969 0 0) 0x7f0dc4153d40 con 0x7f0dc40ac730 2024-03-09T09:08:55.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.329+0000 7f0dc3e00640 1 -- 172.21.15.67:0/2477814374 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dac003520 con 0x7f0dc40ac730 2024-03-09T09:08:55.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.329+0000 7f0dc8e00640 1 -- 172.21.15.67:0/2477814374 >> v1:172.21.15.134:6789/0 conn(0x7f0dc4157d60 legacy=0x7f0dc415e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:08:55.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.329+0000 7f0dc8e00640 1 -- 172.21.15.67:0/2477814374 >> v1:172.21.15.67:6789/0 conn(0x7f0dc40ac730 legacy=0x7f0dc4157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.329+0000 7f0dc3e00640 1 -- 172.21.15.67:0/2477814374 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2453450899 0 0) 0x7f0dac003520 con 0x7f0dc40ac730 2024-03-09T09:08:55.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.330+0000 7f0dc8e00640 1 -- 172.21.15.67:0/2477814374 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc4070920 con 0x7f0dc40c22a0 2024-03-09T09:08:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.331+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/2477814374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40c22a0 0x7f0dc4071190 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0db00097a0 tx=0x7f0db002b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c6a0919b9596f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.331+0000 7f0dc3e00640 1 -- 172.21.15.67:0/2477814374 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0037070 con 0x7f0dc40c22a0 2024-03-09T09:08:55.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.331+0000 7f0dc3e00640 1 -- 172.21.15.67:0/2477814374 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db0030510 con 0x7f0dc40c22a0 2024-03-09T09:08:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.331+0000 7f0dc3e00640 1 -- 172.21.15.67:0/2477814374 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8001670 con 0x7f0dc40c22a0 2024-03-09T09:08:55.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.332+0000 7f0dcc200640 1 -- 172.21.15.67:0/2477814374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40c22a0 msgr2=0x7f0dc4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.332+0000 7f0dcc200640 1 --2- 172.21.15.67:0/2477814374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40c22a0 0x7f0dc4071190 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0db00097a0 tx=0x7f0db002b6d0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.332+0000 7f0dcc200640 1 -- 172.21.15.67:0/2477814374 shutdown_connections 2024-03-09T09:08:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.332+0000 7f0dcc200640 1 --2- 172.21.15.67:0/2477814374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40c22a0 0x7f0dc4071190 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.332+0000 7f0dcc200640 1 -- 172.21.15.67:0/2477814374 >> 172.21.15.67:0/2477814374 conn(0x7f0dc40c7370 msgr2=0x7f0dc40c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.333+0000 7f0dcc200640 1 -- 172.21.15.67:0/2477814374 shutdown_connections 2024-03-09T09:08:55.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.333+0000 7f0dcc200640 1 -- 172.21.15.67:0/2477814374 wait complete. 2024-03-09T09:08:55.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.334+0000 7f0dcc200640 1 Processor -- start 2024-03-09T09:08:55.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.334+0000 7f0dcc200640 1 -- start start 2024-03-09T09:08:55.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.335+0000 7f0dcc200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc40ac730 0x7f0dc40aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.335+0000 7f0dcc200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc40c22a0 0x7f0dc40c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.335+0000 7f0dcc200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc4157d60 0x7f0dc41def70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.335+0000 7f0dcc200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc4151610 con 0x7f0dc40c22a0 2024-03-09T09:08:55.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.335+0000 7f0dcc200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc4154d00 con 0x7f0dc40ac730 2024-03-09T09:08:55.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.335+0000 7f0dcc200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc4151790 con 0x7f0dc4157d60 2024-03-09T09:08:55.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.336+0000 7f0dc8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc40c22a0 0x7f0dc40c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:55.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.336+0000 7f0dc8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc40c22a0 0x7f0dc40c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55404/0 (socket says 172.21.15.67:55404) 2024-03-09T09:08:55.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.336+0000 7f0dc8e00640 1 -- 172.21.15.67:0/2397702659 learned_addr learned my addr 172.21.15.67:0/2397702659 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:08:55.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.336+0000 7f0dc8e00640 1 -- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc4157d60 msgr2=0x7f0dc41def70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.336+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc4157d60 0x7f0dc41def70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.336+0000 7f0dc8e00640 1 -- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc40ac730 msgr2=0x7f0dc40aaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.336+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc40ac730 0x7f0dc40aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.336+0000 7f0dc8e00640 1 -- 172.21.15.67:0/2397702659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc40ab470 con 0x7f0dc40c22a0 2024-03-09T09:08:55.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.337+0000 7f0dc8e00640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc40c22a0 0x7f0dc40c6560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0db002b710 tx=0x7f0db00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:55.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.337+0000 7f0dc1600640 1 -- 172.21.15.67:0/2397702659 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0042020 con 0x7f0dc40c22a0 2024-03-09T09:08:55.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.337+0000 7f0dc1600640 1 -- 172.21.15.67:0/2397702659 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db00044c0 con 0x7f0dc40c22a0 2024-03-09T09:08:55.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.337+0000 7f0dc1600640 1 -- 172.21.15.67:0/2397702659 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0037040 con 0x7f0dc40c22a0 2024-03-09T09:08:55.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.337+0000 7f0dcc200640 1 -- 172.21.15.67:0/2397702659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc41560f0 con 0x7f0dc40c22a0 2024-03-09T09:08:55.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.337+0000 7f0dcc200640 1 -- 172.21.15.67:0/2397702659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc40ac290 con 0x7f0dc40c22a0 2024-03-09T09:08:55.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.339+0000 7f0dcc200640 1 -- 172.21.15.67:0/2397702659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc4075a00 con 0x7f0dc40c22a0 2024-03-09T09:08:55.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.339+0000 7f0dc1600640 1 -- 172.21.15.67:0/2397702659 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0db0045020 con 0x7f0dc40c22a0 2024-03-09T09:08:55.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.341+0000 7f0dc1600640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0da406b4d0 0x7f0da406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:08:55.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.341+0000 7f0dc1600640 1 -- 172.21.15.67:0/2397702659 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0db00ac3f0 con 0x7f0dc40c22a0 2024-03-09T09:08:55.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.342+0000 7f0dc9800640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0da406b4d0 0x7f0da406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:08:55.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.343+0000 7f0dc9800640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0da406b4d0 0x7f0da406d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0db8001eb0 tx=0x7f0db8001a40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:08:55.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.345+0000 7f0dc1600640 1 -- 172.21.15.67:0/2397702659 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0db004c030 con 0x7f0dc40c22a0 2024-03-09T09:08:55.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.670+0000 7f0dcc200640 1 -- 172.21.15.67:0/2397702659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f0dc40c1d70 con 0x7f0dc40c22a0 2024-03-09T09:08:55.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.672+0000 7f0dc1600640 1 -- 172.21.15.67:0/2397702659 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f0dc40ab470 con 0x7f0dc40c22a0 2024-03-09T09:08:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.674+0000 7f0dcc200640 1 -- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0da406b4d0 msgr2=0x7f0da406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.674+0000 7f0dcc200640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0da406b4d0 0x7f0da406d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0db8001eb0 tx=0x7f0db8001a40 comp rx=0 tx=0).stop 2024-03-09T09:08:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.674+0000 7f0dcc200640 1 -- 172.21.15.67:0/2397702659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc40c22a0 msgr2=0x7f0dc40c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:08:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.674+0000 7f0dcc200640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc40c22a0 0x7f0dc40c6560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0db002b710 tx=0x7f0db00046c0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.674+0000 7f0dcc200640 1 -- 172.21.15.67:0/2397702659 shutdown_connections 2024-03-09T09:08:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.674+0000 7f0dcc200640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0da406b4d0 0x7f0da406d990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.674+0000 7f0dcc200640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc4157d60 0x7f0dc41def70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.674+0000 7f0dcc200640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0dc40c22a0 0x7f0dc40c6560 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.674+0000 7f0dcc200640 1 --2- 172.21.15.67:0/2397702659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc40ac730 0x7f0dc40aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:08:55.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.674+0000 7f0dcc200640 1 -- 172.21.15.67:0/2397702659 >> 172.21.15.67:0/2397702659 conn(0x7f0dc40c7370 msgr2=0x7f0dc4157650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:08:55.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.675+0000 7f0dcc200640 1 -- 172.21.15.67:0/2397702659 shutdown_connections 2024-03-09T09:08:55.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:08:55.675+0000 7f0dcc200640 1 -- 172.21.15.67:0/2397702659 wait complete. 2024-03-09T09:08:55.698 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:08:57.180 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 11% (22/195) Updating files: 12% (24/195) Updating files: 13% (26/195) Updating files: 14% (28/195) Updating files: 15% (30/195) Updating files: 16% (32/195) Updating files: 17% (34/195) Updating files: 18% (36/195) Updating files: 18% (37/195) Updating files: 19% (38/195) Updating files: 20% (39/195) Updating files: 21% (41/195) Updating files: 22% (43/195) Updating files: 23% (45/195) Updating files: 24% (47/195) Updating files: 25% (49/195) Updating files: 26% (51/195) Updating files: 27% (53/195) Updating files: 28% (55/195) Updating files: 29% (57/195) Updating files: 30% (59/195) Updating files: 31% (61/195) Updating files: 32% (63/195) Updating files: 33% (65/195) Updating files: 34% (67/195) Updating files: 35% (69/195) Updating files: 36% (71/195) Updating files: 37% (73/195) Updating files: 38% (75/195) Updating files: 39% (77/195) Updating files: 40% (78/195) Updating files: 41% (80/195) Updating files: 42% (82/195) Updating files: 43% (84/195) Updating files: 44% (86/195) Updating files: 45% (88/195) Updating files: 46% (90/195) Updating files: 47% (92/195) Updating files: 48% (94/195) Updating files: 49% (96/195) Updating files: 50% (98/195) Updating files: 51% (100/195) Updating files: 52% (102/195) Updating files: 53% (104/195) Updating files: 54% (106/195) Updating files: 55% (108/195) Updating files: 56% (110/195) Updating files: 57% (112/195) Updating files: 58% (114/195) Updating files: 59% (116/195) Updating files: 60% (117/195) Updating files: 61% (119/195) Updating files: 62% (121/195) Updating files: 63% (123/195) Updating files: 64% (125/195) Updating files: 65% (127/195) Updating files: 65% (128/195) Updating files: 66% (129/195) Updating files: 67% (131/195) Updating files: 68% (133/195) Updating files: 69% (135/195) Updating files: 70% (137/195) Updating files: 71% (139/195) Updating files: 72% (141/195) Updating files: 73% (143/195) Updating files: 74% (145/195) Updating files: 75% (147/195) Updating files: 76% (149/195) Updating files: 77% (151/195) Updating files: 78% (153/195) Updating files: 79% (155/195) Updating files: 80% (156/195) Updating files: 81% (158/195) Updating files: 82% (160/195) Updating files: 83% (162/195) Updating files: 84% (164/195) Updating files: 85% (166/195) Updating files: 86% (168/195) Updating files: 87% (170/195) Updating files: 88% (172/195) Updating files: 89% (174/195) Updating files: 90% (176/195) Updating files: 91% (178/195) Updating files: 92% (180/195) Updating files: 93% (182/195) Updating files: 94% (184/195) Updating files: 95% (186/195) Updating files: 96% (188/195) Updating files: 97% (190/195) Updating files: 98% (192/195) Updating files: 99% (194/195) Updating files: 100% (195/195) Updating files: 100% (195/195), done. 2024-03-09T09:08:57.928 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at c08d08baa6a v0.32 2024-03-09T09:08:57.941 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.32 2024-03-09T09:08:58.237 INFO:tasks.workunit.client.0.smithi067.stdout:v0.33 2024-03-09T09:08:58.238 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:08:58.238 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 33 -eq 48 ']' 2024-03-09T09:08:58.238 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.33 2024-03-09T09:08:58.238 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.33 2024-03-09T09:08:58.238 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.33 2024-03-09T09:08:58.942 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 5% (20/398) Updating files: 6% (24/398) Updating files: 7% (28/398) Updating files: 8% (32/398) Updating files: 8% (34/398) Updating files: 9% (36/398) Updating files: 10% (40/398) Updating files: 11% (44/398) Updating files: 12% (48/398) Updating files: 13% (52/398) Updating files: 14% (56/398) Updating files: 15% (60/398) Updating files: 16% (64/398) Updating files: 17% (68/398) Updating files: 18% (72/398) Updating files: 19% (76/398) Updating files: 20% (80/398) Updating files: 21% (84/398) Updating files: 22% (88/398) Updating files: 23% (92/398) Updating files: 24% (96/398) Updating files: 25% (100/398) Updating files: 26% (104/398) Updating files: 27% (108/398) Updating files: 28% (112/398) Updating files: 29% (116/398) Updating files: 30% (120/398) Updating files: 30% (122/398) Updating files: 31% (124/398) Updating files: 32% (128/398) Updating files: 33% (132/398) Updating files: 34% (136/398) Updating files: 35% (140/398) Updating files: 36% (144/398) Updating files: 37% (148/398) Updating files: 38% (152/398) Updating files: 39% (156/398) Updating files: 40% (160/398) Updating files: 41% (164/398) Updating files: 42% (168/398) Updating files: 43% (172/398) Updating files: 44% (176/398) Updating files: 45% (180/398) Updating files: 46% (184/398) Updating files: 47% (188/398) Updating files: 48% (192/398) Updating files: 49% (196/398) Updating files: 50% (199/398) Updating files: 51% (203/398) Updating files: 52% (207/398) Updating files: 53% (211/398) Updating files: 54% (215/398) Updating files: 55% (219/398) Updating files: 56% (223/398) Updating files: 57% (227/398) Updating files: 58% (231/398) Updating files: 59% (235/398) Updating files: 60% (239/398) Updating files: 61% (243/398) Updating files: 62% (247/398) Updating files: 63% (251/398) Updating files: 64% (255/398) Updating files: 65% (259/398) Updating files: 65% (261/398) Updating files: 66% (263/398) Updating files: 67% (267/398) Updating files: 68% (271/398) Updating files: 69% (275/398) Updating files: 70% (279/398) Updating files: 71% (283/398) Updating files: 72% (287/398) Updating files: 73% (291/398) Updating files: 74% (295/398) Updating files: 75% (299/398) Updating files: 76% (303/398) Updating files: 76% (306/398) Updating files: 77% (307/398) Updating files: 78% (311/398) Updating files: 79% (315/398) Updating files: 80% (319/398) Updating files: 81% (323/398) Updating files: 82% (327/398) Updating files: 83% (331/398) Updating files: 84% (335/398) Updating files: 85% (339/398) Updating files: 86% (343/398) Updating files: 87% (347/398) Updating files: 88% (351/398) Updating files: 89% (355/398) Updating files: 90% (359/398) Updating files: 91% (363/398) Updating files: 92% (367/398) Updating files: 93% (371/398) Updating files: 94% (375/398) Updating files: 95% (379/398) Updating files: 96% (383/398) Updating files: 97% (387/398) Updating files: 98% (391/398) Updating files: 99% (395/398) Updating files: 100% (398/398) Updating files: 100% (398/398), done. 2024-03-09T09:08:58.989 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 9019c6ce640 v0.31 2024-03-09T09:08:59.002 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.31 2024-03-09T09:08:59.212 INFO:tasks.workunit.client.1.smithi134.stdout:v0.32 2024-03-09T09:08:59.212 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:08:59.212 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 32 -eq 48 ']' 2024-03-09T09:08:59.212 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.32 2024-03-09T09:08:59.213 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.32 2024-03-09T09:08:59.213 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.32 2024-03-09T09:09:01.436 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 41% (81/195) Updating files: 42% (82/195) Updating files: 43% (84/195) Updating files: 44% (86/195) Updating files: 45% (88/195) Updating files: 46% (90/195) Updating files: 47% (92/195) Updating files: 48% (94/195) Updating files: 49% (96/195) Updating files: 50% (98/195) Updating files: 51% (100/195) Updating files: 52% (102/195) Updating files: 53% (104/195) Updating files: 54% (106/195) Updating files: 55% (108/195) Updating files: 56% (110/195) Updating files: 57% (112/195) Updating files: 58% (114/195) Updating files: 59% (116/195) Updating files: 60% (117/195) Updating files: 61% (119/195) Updating files: 62% (121/195) Updating files: 63% (123/195) Updating files: 64% (125/195) Updating files: 65% (127/195) Updating files: 66% (129/195) Updating files: 67% (131/195) Updating files: 68% (133/195) Updating files: 69% (135/195) Updating files: 70% (137/195) Updating files: 71% (139/195) Updating files: 72% (141/195) Updating files: 73% (143/195) Updating files: 74% (145/195) Updating files: 75% (147/195) Updating files: 76% (149/195) Updating files: 77% (151/195) Updating files: 78% (153/195) Updating files: 79% (155/195) Updating files: 80% (156/195) Updating files: 81% (158/195) Updating files: 82% (160/195) Updating files: 83% (162/195) Updating files: 84% (164/195) Updating files: 85% (166/195) Updating files: 86% (168/195) Updating files: 87% (170/195) Updating files: 88% (172/195) Updating files: 89% (174/195) Updating files: 90% (176/195) Updating files: 91% (178/195) Updating files: 92% (180/195) Updating files: 93% (182/195) Updating files: 94% (184/195) Updating files: 95% (186/195) Updating files: 96% (188/195) Updating files: 97% (190/195) Updating files: 98% (192/195) Updating files: 99% (194/195) Updating files: 100% (195/195) Updating files: 100% (195/195), done. 2024-03-09T09:09:01.784 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at c08d08baa6a v0.32 2024-03-09T09:09:01.795 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.32 2024-03-09T09:09:01.899 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:09:01.899 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 33 -eq 48 ']' 2024-03-09T09:09:01.899 INFO:tasks.workunit.client.1.smithi134.stdout:v0.33 2024-03-09T09:09:01.899 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.33 2024-03-09T09:09:01.900 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.33 2024-03-09T09:09:01.900 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.33 2024-03-09T09:09:05.459 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 22% (36/160) Updating files: 23% (37/160) Updating files: 24% (39/160) Updating files: 25% (40/160) Updating files: 26% (42/160) Updating files: 27% (44/160) Updating files: 28% (45/160) Updating files: 29% (47/160) Updating files: 30% (48/160) Updating files: 31% (50/160) Updating files: 32% (52/160) Updating files: 33% (53/160) Updating files: 34% (55/160) Updating files: 35% (56/160) Updating files: 36% (58/160) Updating files: 37% (60/160) Updating files: 38% (61/160) Updating files: 39% (63/160) Updating files: 40% (64/160) Updating files: 41% (66/160) Updating files: 42% (68/160) Updating files: 43% (69/160) Updating files: 44% (71/160) Updating files: 45% (72/160) Updating files: 46% (74/160) Updating files: 47% (76/160) Updating files: 48% (77/160) Updating files: 49% (79/160) Updating files: 50% (80/160) Updating files: 51% (82/160) Updating files: 52% (84/160) Updating files: 53% (85/160) Updating files: 54% (87/160) Updating files: 55% (88/160) Updating files: 56% (90/160) Updating files: 57% (92/160) Updating files: 58% (93/160) Updating files: 59% (95/160) Updating files: 60% (96/160) Updating files: 61% (98/160) Updating files: 62% (100/160) Updating files: 63% (101/160) Updating files: 64% (103/160) Updating files: 65% (104/160) Updating files: 66% (106/160) Updating files: 67% (108/160) Updating files: 68% (109/160) Updating files: 69% (111/160) Updating files: 70% (112/160) Updating files: 71% (114/160) Updating files: 72% (116/160) Updating files: 73% (117/160) Updating files: 74% (119/160) Updating files: 75% (120/160) Updating files: 76% (122/160) Updating files: 77% (124/160) Updating files: 78% (125/160) Updating files: 79% (127/160) Updating files: 80% (128/160) Updating files: 81% (130/160) Updating files: 82% (132/160) Updating files: 83% (133/160) Updating files: 84% (135/160) Updating files: 85% (136/160) Updating files: 86% (138/160) Updating files: 87% (140/160) Updating files: 88% (141/160) Updating files: 89% (143/160) Updating files: 90% (144/160) Updating files: 91% (146/160) Updating files: 92% (148/160) Updating files: 93% (149/160) Updating files: 94% (151/160) Updating files: 95% (152/160) Updating files: 96% (154/160) Updating files: 97% (156/160) Updating files: 98% (157/160) Updating files: 99% (159/160) Updating files: 100% (160/160) Updating files: 100% (160/160), done. 2024-03-09T09:09:05.513 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 340b4348bf6 v0.33 2024-03-09T09:09:05.525 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.33 2024-03-09T09:09:05.699 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:09:05.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:09:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.932+0000 7f92eae00640 1 Processor -- start 2024-03-09T09:09:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.932+0000 7f92eae00640 1 -- start start 2024-03-09T09:09:05.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.933+0000 7f92eae00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f92e40ac730 0x7f92e4071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:05.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.934+0000 7f92eae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e4157d60 0x7f92e40acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:05.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.934+0000 7f92eae00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e4153d40 con 0x7f92e40c22a0 2024-03-09T09:09:05.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.934+0000 7f92eae00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e4070ee0 con 0x7f92e40ac730 2024-03-09T09:09:05.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.934+0000 7f92eae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41581f0 con 0x7f92e4157d60 2024-03-09T09:09:05.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.934+0000 7f92e3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f92e40ac730 0x7f92e4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:05.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.934+0000 7f92e3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f92e40ac730 0x7f92e4071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53732/0 (socket says 172.21.15.67:53732) 2024-03-09T09:09:05.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.934+0000 7f92e3e00640 1 -- 172.21.15.67:0/3334130435 learned_addr learned my addr 172.21.15.67:0/3334130435 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:05.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.934+0000 7f92e8e00640 1 --2- 172.21.15.67:0/3334130435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e4157d60 0x7f92e40acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:05.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.935+0000 7f92e3e00640 1 -- 172.21.15.67:0/3334130435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e4157d60 msgr2=0x7f92e40acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:05.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.935+0000 7f92e3e00640 1 --2- 172.21.15.67:0/3334130435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e4157d60 0x7f92e40acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:05.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.935+0000 7f92e3e00640 1 -- 172.21.15.67:0/3334130435 >> v1:172.21.15.134:6789/0 conn(0x7f92e40c22a0 legacy=0x7f92e4157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:05.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.935+0000 7f92e3e00640 1 -- 172.21.15.67:0/3334130435 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40c2790 con 0x7f92e40ac730 2024-03-09T09:09:05.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.936+0000 7f92e3e00640 1 --2- 172.21.15.67:0/3334130435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ac730 0x7f92e4071190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009870 tx=0x7f92d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab908edc90f7065 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:05.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.936+0000 7f92e2a00640 1 -- 172.21.15.67:0/3334130435 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4035020 con 0x7f92e40ac730 2024-03-09T09:09:05.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.936+0000 7f92e2a00640 1 -- 172.21.15.67:0/3334130435 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d4031640 con 0x7f92e40ac730 2024-03-09T09:09:05.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.936+0000 7f92e2a00640 1 -- 172.21.15.67:0/3334130435 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d402d040 con 0x7f92e40ac730 2024-03-09T09:09:05.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.937+0000 7f92eae00640 1 -- 172.21.15.67:0/3334130435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ac730 msgr2=0x7f92e4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:05.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.937+0000 7f92eae00640 1 --2- 172.21.15.67:0/3334130435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ac730 0x7f92e4071190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009870 tx=0x7f92d402b8f0 comp rx=0 tx=0).stop 2024-03-09T09:09:05.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.937+0000 7f92eae00640 1 -- 172.21.15.67:0/3334130435 shutdown_connections 2024-03-09T09:09:05.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.937+0000 7f92eae00640 1 --2- 172.21.15.67:0/3334130435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e4157d60 0x7f92e40acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:05.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.937+0000 7f92eae00640 1 --2- 172.21.15.67:0/3334130435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ac730 0x7f92e4071190 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:05.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.937+0000 7f92eae00640 1 -- 172.21.15.67:0/3334130435 >> 172.21.15.67:0/3334130435 conn(0x7f92e40c7370 msgr2=0x7f92e40c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:05.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.937+0000 7f92eae00640 1 -- 172.21.15.67:0/3334130435 shutdown_connections 2024-03-09T09:09:05.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.937+0000 7f92eae00640 1 -- 172.21.15.67:0/3334130435 wait complete. 2024-03-09T09:09:05.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.939+0000 7f92eae00640 1 Processor -- start 2024-03-09T09:09:05.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.939+0000 7f92eae00640 1 -- start start 2024-03-09T09:09:05.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.940+0000 7f92eae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ac730 0x7f92e41de310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:05.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.940+0000 7f92eae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92e40c22a0 0x7f92e40c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:05.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.940+0000 7f92eae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e4157d60 0x7f92e41dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:05.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.940+0000 7f92eae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4070c00 con 0x7f92e40ac730 2024-03-09T09:09:05.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.940+0000 7f92eae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4154d00 con 0x7f92e40c22a0 2024-03-09T09:09:05.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.940+0000 7f92eae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4151610 con 0x7f92e4157d60 2024-03-09T09:09:05.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.940+0000 7f92e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ac730 0x7f92e41de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.940+0000 7f92e3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ac730 0x7f92e41de310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53740/0 (socket says 172.21.15.67:53740) 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.940+0000 7f92e3e00640 1 -- 172.21.15.67:0/2243711012 learned_addr learned my addr 172.21.15.67:0/2243711012 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.941+0000 7f92e8e00640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e4157d60 0x7f92e41dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.941+0000 7f92e3e00640 1 -- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e4157d60 msgr2=0x7f92e41dfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.941+0000 7f92e3e00640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e4157d60 0x7f92e41dfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.941+0000 7f92e3e00640 1 -- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92e40c22a0 msgr2=0x7f92e40c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.941+0000 7f92e3e00640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92e40c22a0 0x7f92e40c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.941+0000 7f92e3e00640 1 -- 172.21.15.67:0/2243711012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e41efb70 con 0x7f92e40ac730 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.941+0000 7f92e3e00640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ac730 0x7f92e41de310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f92d4000c00 tx=0x7f92d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.941+0000 7f92dbe00640 1 -- 172.21.15.67:0/2243711012 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4040070 con 0x7f92e40ac730 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.941+0000 7f92dbe00640 1 -- 172.21.15.67:0/2243711012 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d40399f0 con 0x7f92e40ac730 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.941+0000 7f92dbe00640 1 -- 172.21.15.67:0/2243711012 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4035040 con 0x7f92e40ac730 2024-03-09T09:09:05.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.941+0000 7f92eae00640 1 -- 172.21.15.67:0/2243711012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e41df430 con 0x7f92e40ac730 2024-03-09T09:09:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.942+0000 7f92eae00640 1 -- 172.21.15.67:0/2243711012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e40c0f50 con 0x7f92e40ac730 2024-03-09T09:09:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.943+0000 7f92eae00640 1 -- 172.21.15.67:0/2243711012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e4075a00 con 0x7f92e40ac730 2024-03-09T09:09:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.944+0000 7f92dbe00640 1 -- 172.21.15.67:0/2243711012 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f92d404b060 con 0x7f92e40ac730 2024-03-09T09:09:05.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.945+0000 7f92dbe00640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92c406b500 0x7f92c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:05.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.945+0000 7f92dbe00640 1 -- 172.21.15.67:0/2243711012 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f92d40abe90 con 0x7f92e40ac730 2024-03-09T09:09:05.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.945+0000 7f92e3400640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92c406b500 0x7f92c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:05.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.946+0000 7f92e3400640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92c406b500 0x7f92c406d9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f92e41eca60 tx=0x7f92d0007970 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:05.949+0000 7f92dbe00640 1 -- 172.21.15.67:0/2243711012 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f92d407a9b0 con 0x7f92e40ac730 2024-03-09T09:09:06.088 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:09:06.088 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 34 -eq 48 ']' 2024-03-09T09:09:06.088 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.34 2024-03-09T09:09:06.088 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.34 2024-03-09T09:09:06.088 INFO:tasks.workunit.client.1.smithi134.stdout:v0.34 2024-03-09T09:09:06.088 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.34 2024-03-09T09:09:06.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.285+0000 7f92eae00640 1 -- 172.21.15.67:0/2243711012 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f92e40c1d70 con 0x7f92e40ac730 2024-03-09T09:09:06.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.286+0000 7f92dbe00640 1 -- 172.21.15.67:0/2243711012 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f92d407a2b0 con 0x7f92e40ac730 2024-03-09T09:09:06.290 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:09:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 -- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92c406b500 msgr2=0x7f92c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92c406b500 0x7f92c406d9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f92e41eca60 tx=0x7f92d0007970 comp rx=0 tx=0).stop 2024-03-09T09:09:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 -- 172.21.15.67:0/2243711012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ac730 msgr2=0x7f92e41de310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ac730 0x7f92e41de310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f92d4000c00 tx=0x7f92d40046c0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 -- 172.21.15.67:0/2243711012 shutdown_connections 2024-03-09T09:09:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92c406b500 0x7f92c406d9c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92e4157d60 0x7f92e41dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92e40c22a0 0x7f92e40c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 --2- 172.21.15.67:0/2243711012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ac730 0x7f92e41de310 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 -- 172.21.15.67:0/2243711012 >> 172.21.15.67:0/2243711012 conn(0x7f92e40c7370 msgr2=0x7f92e40c7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 -- 172.21.15.67:0/2243711012 shutdown_connections 2024-03-09T09:09:06.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.289+0000 7f92eae00640 1 -- 172.21.15.67:0/2243711012 wait complete. 2024-03-09T09:09:06.310 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:09:06.311 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:09:06.311 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:09:06.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:09:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.519+0000 7fc1cb000640 1 Processor -- start 2024-03-09T09:09:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.519+0000 7fc1cb000640 1 -- start start 2024-03-09T09:09:06.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.520+0000 7fc1cb000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc1c4150140 0x7fc1c406d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.520+0000 7fc1cb000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc1c4152790 0x7fc1c4071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.520+0000 7fc1cb000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c406d940 con 0x7fc1c40c2fa0 2024-03-09T09:09:06.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.520+0000 7fc1cb000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c4071b20 con 0x7fc1c4150140 2024-03-09T09:09:06.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.520+0000 7fc1cb000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c406dc20 con 0x7fc1c4152790 2024-03-09T09:09:06.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.521+0000 7fc1c9200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc1c4150140 0x7fc1c406d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.521+0000 7fc1c9200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc1c4150140 0x7fc1c406d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53750/0 (socket says 172.21.15.67:53750) 2024-03-09T09:09:06.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.521+0000 7fc1c9200640 1 -- 172.21.15.67:0/1153660775 learned_addr learned my addr 172.21.15.67:0/1153660775 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.521+0000 7fc1c9200640 1 -- 172.21.15.67:0/1153660775 >> v2:172.21.15.134:3300/0 conn(0x7fc1c4152790 msgr2=0x7fc1c4071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.521+0000 7fc1c9200640 1 --2- 172.21.15.67:0/1153660775 >> v2:172.21.15.134:3300/0 conn(0x7fc1c4152790 0x7fc1c4071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.521+0000 7fc1c9200640 1 -- 172.21.15.67:0/1153660775 >> v1:172.21.15.134:6789/0 conn(0x7fc1c40c2fa0 legacy=0x7fc1c414fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.521+0000 7fc1c9200640 1 -- 172.21.15.67:0/1153660775 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c406df00 con 0x7fc1c4150140 2024-03-09T09:09:06.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.522+0000 7fc1c9200640 1 --2- 172.21.15.67:0/1153660775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c4150140 0x7fc1c406d340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40ad570 tx=0x7fc1b002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686781e3256467f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:06.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.523+0000 7fc1c3e00640 1 -- 172.21.15.67:0/1153660775 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b002e070 con 0x7fc1c4150140 2024-03-09T09:09:06.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.523+0000 7fc1c3e00640 1 -- 172.21.15.67:0/1153660775 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b0037070 con 0x7fc1c4150140 2024-03-09T09:09:06.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.523+0000 7fc1c3e00640 1 -- 172.21.15.67:0/1153660775 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0031a00 con 0x7fc1c4150140 2024-03-09T09:09:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.524+0000 7fc1cb000640 1 -- 172.21.15.67:0/1153660775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c4150140 msgr2=0x7fc1c406d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.524+0000 7fc1cb000640 1 --2- 172.21.15.67:0/1153660775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c4150140 0x7fc1c406d340 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40ad570 tx=0x7fc1b002b880 comp rx=0 tx=0).stop 2024-03-09T09:09:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.524+0000 7fc1cb000640 1 -- 172.21.15.67:0/1153660775 shutdown_connections 2024-03-09T09:09:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.524+0000 7fc1cb000640 1 --2- 172.21.15.67:0/1153660775 >> v2:172.21.15.134:3300/0 conn(0x7fc1c4152790 0x7fc1c4071fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.524+0000 7fc1cb000640 1 --2- 172.21.15.67:0/1153660775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c4150140 0x7fc1c406d340 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.524+0000 7fc1cb000640 1 -- 172.21.15.67:0/1153660775 >> 172.21.15.67:0/1153660775 conn(0x7fc1c40c7220 msgr2=0x7fc1c40c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:06.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.524+0000 7fc1cb000640 1 -- 172.21.15.67:0/1153660775 shutdown_connections 2024-03-09T09:09:06.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.524+0000 7fc1cb000640 1 -- 172.21.15.67:0/1153660775 wait complete. 2024-03-09T09:09:06.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.526+0000 7fc1cb000640 1 Processor -- start 2024-03-09T09:09:06.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.526+0000 7fc1cb000640 1 -- start start 2024-03-09T09:09:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.527+0000 7fc1cb000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c40c2fa0 0x7fc1c415e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.527+0000 7fc1cb000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c4150140 0x7fc1c414e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.527+0000 7fc1cb000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c4152790 0x7fc1c413c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.527+0000 7fc1cb000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c4071340 con 0x7fc1c4152790 2024-03-09T09:09:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.527+0000 7fc1cb000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40711c0 con 0x7fc1c4150140 2024-03-09T09:09:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.527+0000 7fc1cb000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c3380 con 0x7fc1c40c2fa0 2024-03-09T09:09:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.527+0000 7fc1ca600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c4152790 0x7fc1c413c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.527+0000 7fc1ca600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c4152790 0x7fc1c413c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53764/0 (socket says 172.21.15.67:53764) 2024-03-09T09:09:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.527+0000 7fc1ca600640 1 -- 172.21.15.67:0/1808270167 learned_addr learned my addr 172.21.15.67:0/1808270167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.527+0000 7fc1c9200640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c4150140 0x7fc1c414e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.527+0000 7fc1c9c00640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c40c2fa0 0x7fc1c415e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.528+0000 7fc1ca600640 1 -- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c40c2fa0 msgr2=0x7fc1c415e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.528+0000 7fc1ca600640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c40c2fa0 0x7fc1c415e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.528+0000 7fc1ca600640 1 -- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c4150140 msgr2=0x7fc1c414e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.528+0000 7fc1ca600640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1c4150140 0x7fc1c414e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.528+0000 7fc1ca600640 1 -- 172.21.15.67:0/1808270167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c4150b80 con 0x7fc1c4152790 2024-03-09T09:09:06.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.528+0000 7fc1c9c00640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1c40c2fa0 0x7fc1c415e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:09:06.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.528+0000 7fc1ca600640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c4152790 0x7fc1c413c250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00b970 tx=0x7fc1bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:06.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.529+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc019070 con 0x7fc1c4152790 2024-03-09T09:09:06.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.529+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc00cb60 con 0x7fc1c4152790 2024-03-09T09:09:06.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.529+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc014440 con 0x7fc1c4152790 2024-03-09T09:09:06.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.530+0000 7fc1cb000640 1 -- 172.21.15.67:0/1808270167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c4150e60 con 0x7fc1c4152790 2024-03-09T09:09:06.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.530+0000 7fc1cb000640 1 -- 172.21.15.67:0/1808270167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c40c07f0 con 0x7fc1c4152790 2024-03-09T09:09:06.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.531+0000 7fc1cb000640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c40c0cf0 0x7fc1c40c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.531+0000 7fc1cb000640 1 -- 172.21.15.67:0/1808270167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c404f3c0 con 0x7fc1c40c0cf0 2024-03-09T09:09:06.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.531+0000 7fc1c9c00640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c40c0cf0 0x7fc1c40c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.532+0000 7fc1c9c00640 1 -- 172.21.15.67:0/1808270167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1c4152570 con 0x7fc1c40c0cf0 2024-03-09T09:09:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.532+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc1bc01c070 con 0x7fc1c4152790 2024-03-09T09:09:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.532+0000 7fc1c9c00640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c40c0cf0 0x7fc1c40c10d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800d730 tx=0x7fc1b800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.534+0000 7fc1c1600640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc1a406b4d0 0x7fc1a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.534+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc1bc08a370 con 0x7fc1c4152790 2024-03-09T09:09:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.534+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8018030 con 0x7fc1c40c0cf0 2024-03-09T09:09:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.534+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc1b8004020 con 0x7fc1c40c0cf0 2024-03-09T09:09:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.534+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c40c0cf0 msgr2=0x7fc1c40c10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.534+0000 7fc1c1600640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c40c0cf0 0x7fc1c40c10d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800d730 tx=0x7fc1b800dbf0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.534+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c40c0cf0 msgr2=0x7fc1c40c10d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:06.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.534+0000 7fc1c1600640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c40c0cf0 0x7fc1c40c10d0 secure :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800d730 tx=0x7fc1b800dbf0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.534+0000 7fc1c9200640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc1a406b4d0 0x7fc1a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.536+0000 7fc1c9200640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc1a406b4d0 0x7fc1a406d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0036000 tx=0x7fc1b003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.549+0000 7fc1cb000640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c41ef2c0 0x7fc1c41ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.549+0000 7fc1cb000640 1 -- 172.21.15.67:0/1808270167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c09c0 con 0x7fc1c41ef2c0 2024-03-09T09:09:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.549+0000 7fc1c9c00640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c41ef2c0 0x7fc1c41ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.550+0000 7fc1c9c00640 1 -- 172.21.15.67:0/1808270167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc1c4072850 con 0x7fc1c41ef2c0 2024-03-09T09:09:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.550+0000 7fc1c9c00640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c41ef2c0 0x7fc1c41ef6d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8018000 tx=0x7fc1b80046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:06.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.550+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b801db60 con 0x7fc1c41ef2c0 2024-03-09T09:09:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.551+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7fc1b8009710 con 0x7fc1c41ef2c0 2024-03-09T09:09:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.551+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c41ef2c0 msgr2=0x7fc1c41ef6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.551+0000 7fc1c1600640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c41ef2c0 0x7fc1c41ef6d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8018000 tx=0x7fc1b80046f0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.551+0000 7fc1c1600640 1 -- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c41ef2c0 msgr2=0x7fc1c41ef6d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.551+0000 7fc1c1600640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c41ef2c0 0x7fc1c41ef6d0 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8018000 tx=0x7fc1b80046f0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.555 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:09:06.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 118, 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:09:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:09:06.557 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:09:06.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:06.560 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:09:06.561 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:06.562 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:09:06.564 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.565 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:06.566 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:06.567 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:06.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:09:06.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.569 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:06.569 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:06.569 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:09:06.569 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.569 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.569 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:09:06.569 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:09:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.551+0000 7fc1cb000640 1 -- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc1a406b4d0 msgr2=0x7fc1a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.551+0000 7fc1cb000640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc1a406b4d0 0x7fc1a406d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0036000 tx=0x7fc1b003d040 comp rx=0 tx=0).stop 2024-03-09T09:09:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.551+0000 7fc1cb000640 1 -- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c4152790 msgr2=0x7fc1c413c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.551+0000 7fc1cb000640 1 --2- 172.21.15.67:0/1808270167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1c4152790 0x7fc1c413c250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00b970 tx=0x7fc1bc00be30 comp rx=0 tx=0).stop 2024-03-09T09:09:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.551+0000 7fc1ca600640 1 -- 172.21.15.67:0/1808270167 reap_dead start 2024-03-09T09:09:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.551+0000 7fc1ca600640 1 -- 172.21.15.67:0/1808270167 reap_dead start 2024-03-09T09:09:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.552+0000 7fc1cb000640 1 -- 172.21.15.67:0/1808270167 shutdown_connections 2024-03-09T09:09:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.552+0000 7fc1cb000640 1 -- 172.21.15.67:0/1808270167 >> 172.21.15.67:0/1808270167 conn(0x7fc1c40c7220 msgr2=0x7fc1c40c1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.552+0000 7fc1cb000640 1 -- 172.21.15.67:0/1808270167 shutdown_connections 2024-03-09T09:09:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.552+0000 7fc1cb000640 1 -- 172.21.15.67:0/1808270167 wait complete. 2024-03-09T09:09:06.579 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:09:06.579 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:09:06.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:09:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.776+0000 7f8e99a00640 1 Processor -- start 2024-03-09T09:09:06.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.776+0000 7f8e99a00640 1 -- start start 2024-03-09T09:09:06.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.777+0000 7f8e99a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e940ac9a0 0x7f8e941510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.777+0000 7f8e99a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8e940c2780 0x7f8e94151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.778+0000 7f8e99a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e940c8200 con 0x7f8e94071190 2024-03-09T09:09:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.778+0000 7f8e99a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e9406d590 con 0x7f8e940c2780 2024-03-09T09:09:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.778+0000 7f8e99a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940c3740 con 0x7f8e940ac9a0 2024-03-09T09:09:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.778+0000 7f8e93400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8e94071190 0x7f8e94154190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46766/0 (socket says 172.21.15.67:46766) 2024-03-09T09:09:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.778+0000 7f8e93400640 1 -- 172.21.15.67:0/1379399936 learned_addr learned my addr 172.21.15.67:0/1379399936 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.778+0000 7f8e92a00640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e940ac9a0 0x7f8e941510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.778+0000 7f8e92000640 1 --2- 172.21.15.67:0/1379399936 >> v2:172.21.15.134:3300/0 conn(0x7f8e940c2780 0x7f8e94151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.778+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408377428 0 0) 0x7f8e940c8200 con 0x7f8e94071190 2024-03-09T09:09:06.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.778+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e78003520 con 0x7f8e94071190 2024-03-09T09:09:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3657643448 0 0) 0x7f8e78003520 con 0x7f8e94071190 2024-03-09T09:09:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e940ac9a0 msgr2=0x7f8e941510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e940ac9a0 0x7f8e941510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 >> v2:172.21.15.134:3300/0 conn(0x7f8e940c2780 msgr2=0x7f8e94151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 --2- 172.21.15.67:0/1379399936 >> v2:172.21.15.134:3300/0 conn(0x7f8e940c2780 0x7f8e94151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e940c3990 con 0x7f8e94071190 2024-03-09T09:09:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f8e8c0019f0 con 0x7f8e94071190 2024-03-09T09:09:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 >> v1:172.21.15.67:6789/0 conn(0x7f8e94071190 legacy=0x7f8e94154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e780039b0 0x7f8e78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e78004800 0x7f8e78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e940ac9a0 0x7f8e780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940c3740 con 0x7f8e78004800 2024-03-09T09:09:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7800b9c0 con 0x7f8e780039b0 2024-03-09T09:09:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7800bb40 con 0x7f8e940ac9a0 2024-03-09T09:09:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.779+0000 7f8e92000640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e78004800 0x7f8e78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.780+0000 7f8e92a00640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e780039b0 0x7f8e78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.780+0000 7f8e93400640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e940ac9a0 0x7f8e780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.780+0000 7f8e92000640 1 -- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e940ac9a0 msgr2=0x7f8e780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.780+0000 7f8e92000640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e940ac9a0 0x7f8e780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.780+0000 7f8e92000640 1 -- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e780039b0 msgr2=0x7f8e78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:06.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.780+0000 7f8e92000640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e780039b0 0x7f8e78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.780+0000 7f8e92000640 1 -- 172.21.15.67:0/1379399936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e78003520 con 0x7f8e78004800 2024-03-09T09:09:06.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.780+0000 7f8e93400640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e940ac9a0 0x7f8e780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:09:06.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.781+0000 7f8e92000640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e78004800 0x7f8e78004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009ac0 tx=0x7f8e78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfec72e6b93b58b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:06.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.781+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80039070 con 0x7f8e78004800 2024-03-09T09:09:06.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.781+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8002c6b0 con 0x7f8e78004800 2024-03-09T09:09:06.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.781+0000 7f8e91600640 1 -- 172.21.15.67:0/1379399936 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80009690 con 0x7f8e78004800 2024-03-09T09:09:06.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.782+0000 7f8e99a00640 1 -- 172.21.15.67:0/1379399936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e78004800 msgr2=0x7f8e78004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.782+0000 7f8e99a00640 1 --2- 172.21.15.67:0/1379399936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e78004800 0x7f8e78004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009ac0 tx=0x7f8e78005830 comp rx=0 tx=0).stop 2024-03-09T09:09:06.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.782+0000 7f8e93400640 1 -- 172.21.15.67:0/1379399936 reap_dead start 2024-03-09T09:09:06.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.782+0000 7f8e99a00640 1 -- 172.21.15.67:0/1379399936 shutdown_connections 2024-03-09T09:09:06.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.782+0000 7f8e99a00640 1 -- 172.21.15.67:0/1379399936 >> 172.21.15.67:0/1379399936 conn(0x7f8e940c7370 msgr2=0x7f8e940c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:06.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.782+0000 7f8e99a00640 1 -- 172.21.15.67:0/1379399936 shutdown_connections 2024-03-09T09:09:06.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.782+0000 7f8e99a00640 1 -- 172.21.15.67:0/1379399936 wait complete. 2024-03-09T09:09:06.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.784+0000 7f8e99a00640 1 Processor -- start 2024-03-09T09:09:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.784+0000 7f8e99a00640 1 -- start start 2024-03-09T09:09:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.785+0000 7f8e99a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e78004800 0x7f8e940ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.785+0000 7f8e99a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9414e420 0x7f8e941eec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.785+0000 7f8e99a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9414bab0 0x7f8e940aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.785+0000 7f8e99a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940acd80 con 0x7f8e9414bab0 2024-03-09T09:09:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.785+0000 7f8e99a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94070c00 con 0x7f8e9414e420 2024-03-09T09:09:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.785+0000 7f8e99a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940acf00 con 0x7f8e78004800 2024-03-09T09:09:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e93400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9414bab0 0x7f8e940aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e93400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9414bab0 0x7f8e940aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53796/0 (socket says 172.21.15.67:53796) 2024-03-09T09:09:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e93400640 1 -- 172.21.15.67:0/1070764786 learned_addr learned my addr 172.21.15.67:0/1070764786 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e92000640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9414e420 0x7f8e941eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e92a00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e78004800 0x7f8e940ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e93400640 1 -- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e78004800 msgr2=0x7f8e940ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e93400640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e78004800 0x7f8e940ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e93400640 1 -- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9414e420 msgr2=0x7f8e941eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e93400640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9414e420 0x7f8e941eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e93400640 1 -- 172.21.15.67:0/1070764786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e94150580 con 0x7f8e9414bab0 2024-03-09T09:09:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e92000640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9414e420 0x7f8e941eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:09:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.786+0000 7f8e93400640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9414bab0 0x7f8e940aa520 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8e940c7fe0 tx=0x7f8e8c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.787+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c007a80 con 0x7f8e9414bab0 2024-03-09T09:09:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.787+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8c00eda0 con 0x7f8e9414bab0 2024-03-09T09:09:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.787+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c014670 con 0x7f8e9414bab0 2024-03-09T09:09:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.787+0000 7f8e99a00640 1 -- 172.21.15.67:0/1070764786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e941e2800 con 0x7f8e9414bab0 2024-03-09T09:09:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.787+0000 7f8e99a00640 1 -- 172.21.15.67:0/1070764786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e940abe00 con 0x7f8e9414bab0 2024-03-09T09:09:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.788+0000 7f8e99a00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e941ef190 0x7f8e940aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.788+0000 7f8e99a00640 1 -- 172.21.15.67:0/1070764786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e941e25b0 con 0x7f8e941ef190 2024-03-09T09:09:06.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.789+0000 7f8e92a00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e941ef190 0x7f8e940aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.789+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8e8c014810 con 0x7f8e9414bab0 2024-03-09T09:09:06.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.790+0000 7f8e92a00640 1 -- 172.21.15.67:0/1070764786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e941e2d20 con 0x7f8e941ef190 2024-03-09T09:09:06.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.790+0000 7f8e8aa00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e7406b500 0x7f8e7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.790+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8e8c08a580 con 0x7f8e9414bab0 2024-03-09T09:09:06.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.790+0000 7f8e92a00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e941ef190 0x7f8e940aaa60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8e84014920 tx=0x7f8e84014de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.791+0000 7f8e92000640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e7406b500 0x7f8e7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.791+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84012b30 con 0x7f8e941ef190 2024-03-09T09:09:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.792+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f8e84012b30 con 0x7f8e941ef190 2024-03-09T09:09:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.792+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e941ef190 msgr2=0x7f8e940aaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.792+0000 7f8e8aa00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e941ef190 0x7f8e940aaa60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8e84014920 tx=0x7f8e84014de0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.792+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e941ef190 msgr2=0x7f8e940aaa60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.792+0000 7f8e8aa00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e941ef190 0x7f8e940aaa60 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8e84014920 tx=0x7f8e84014de0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.792+0000 7f8e92000640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e7406b500 0x7f8e7406d9c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8e80036000 tx=0x7f8e80030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.799+0000 7f8e99a00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e940aafa0 0x7f8e941485a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.799+0000 7f8e99a00640 1 -- 172.21.15.67:0/1070764786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940ab380 con 0x7f8e940aafa0 2024-03-09T09:09:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.799+0000 7f8e92a00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e940aafa0 0x7f8e941485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:06.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.800+0000 7f8e92a00640 1 -- 172.21.15.67:0/1070764786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8e94075a00 con 0x7f8e940aafa0 2024-03-09T09:09:06.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.801+0000 7f8e92a00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e940aafa0 0x7f8e941485a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e8401f000 tx=0x7f8e84004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.801+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84016930 con 0x7f8e940aafa0 2024-03-09T09:09:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.802+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f8e84016930 con 0x7f8e940aafa0 2024-03-09T09:09:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.802+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e940aafa0 msgr2=0x7f8e941485a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.802+0000 7f8e8aa00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e940aafa0 0x7f8e941485a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e8401f000 tx=0x7f8e84004730 comp rx=0 tx=0).stop 2024-03-09T09:09:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.802+0000 7f8e8aa00640 1 -- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e940aafa0 msgr2=0x7f8e941485a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.802+0000 7f8e8aa00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e940aafa0 0x7f8e941485a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 118, 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:09:06.807 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:06.808 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:06.809 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:09:06.809 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:09:06.809 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:06.809 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:06.809 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:06.809 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:06.809 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:06.809 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:06.809 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:06.809 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:06.809 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:06.810 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:06.811 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:09:06.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:06.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:09:06.815 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:06.818 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:06.819 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:09:06.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:06.819 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:06.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:06.819 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:09:06.819 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:09:06.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.802+0000 7f8e99a00640 1 -- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e7406b500 msgr2=0x7f8e7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.802+0000 7f8e99a00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e7406b500 0x7f8e7406d9c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8e80036000 tx=0x7f8e80030000 comp rx=0 tx=0).stop 2024-03-09T09:09:06.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.802+0000 7f8e93400640 1 -- 172.21.15.67:0/1070764786 reap_dead start 2024-03-09T09:09:06.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.803+0000 7f8e99a00640 1 -- 172.21.15.67:0/1070764786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9414bab0 msgr2=0x7f8e940aa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:06.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.803+0000 7f8e99a00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9414bab0 0x7f8e940aa520 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8e940c7fe0 tx=0x7f8e8c00e530 comp rx=0 tx=0).stop 2024-03-09T09:09:06.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.803+0000 7f8e99a00640 1 -- 172.21.15.67:0/1070764786 shutdown_connections 2024-03-09T09:09:06.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.803+0000 7f8e99a00640 1 --2- 172.21.15.67:0/1070764786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e9414bab0 0x7f8e940aa520 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:06.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.803+0000 7f8e99a00640 1 -- 172.21.15.67:0/1070764786 >> 172.21.15.67:0/1070764786 conn(0x7f8e940c7370 msgr2=0x7f8e940c07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:06.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.803+0000 7f8e99a00640 1 -- 172.21.15.67:0/1070764786 shutdown_connections 2024-03-09T09:09:06.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:06.803+0000 7f8e99a00640 1 -- 172.21.15.67:0/1070764786 wait complete. 2024-03-09T09:09:06.825 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:09:06.825 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:09:06.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:09:07.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.042+0000 7f48b0a00640 1 Processor -- start 2024-03-09T09:09:07.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.043+0000 7f48b0a00640 1 -- start start 2024-03-09T09:09:07.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.044+0000 7f48b0a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac151360 0x7f48ac0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.044+0000 7f48b0a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ac0c2780 con 0x7f48ac0c30a0 2024-03-09T09:09:07.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.044+0000 7f48b0a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ac0708c0 con 0x7f48ac0ac930 2024-03-09T09:09:07.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.044+0000 7f48b0a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac070670 con 0x7f48ac151360 2024-03-09T09:09:07.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.044+0000 7f48abe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f48ac0c30a0 0x7f48ac1539e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46782/0 (socket says 172.21.15.67:46782) 2024-03-09T09:09:07.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.044+0000 7f48abe00640 1 -- 172.21.15.67:0/4224547883 learned_addr learned my addr 172.21.15.67:0/4224547883 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:07.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.045+0000 7f48a2a00640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac151360 0x7f48ac0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.045+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913271097 0 0) 0x7f48ac0c2780 con 0x7f48ac0c30a0 2024-03-09T09:09:07.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.045+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4890003520 con 0x7f48ac0c30a0 2024-03-09T09:09:07.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.045+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 499860852 0 0) 0x7f4890003520 con 0x7f48ac0c30a0 2024-03-09T09:09:07.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.045+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac151360 msgr2=0x7f48ac0c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.045+0000 7f48aaa00640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac151360 0x7f48ac0c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.045+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 >> v1:172.21.15.134:6789/0 conn(0x7f48ac0ac930 legacy=0x7f48ac150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.045+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac070ba0 con 0x7f48ac0c30a0 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.045+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f48a40019f0 con 0x7f48ac0c30a0 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.045+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 >> v1:172.21.15.67:6789/0 conn(0x7f48ac0c30a0 legacy=0x7f48ac1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48aaa00640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890003a50 0x7f4890003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48aaa00640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4890004800 0x7f4890004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48aaa00640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac151360 0x7f48900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac070670 con 0x7f4890003a50 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489000fed0 con 0x7f4890004800 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48900100d0 con 0x7f48ac151360 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48ab400640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890003a50 0x7f4890003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48abe00640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac151360 0x7f48900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48a2a00640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4890004800 0x7f4890004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48ab400640 1 -- 172.21.15.67:0/4224547883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac151360 msgr2=0x7f48900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:07.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48ab400640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac151360 0x7f48900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48ab400640 1 -- 172.21.15.67:0/4224547883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4890004800 msgr2=0x7f4890004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:07.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48ab400640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4890004800 0x7f4890004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.046+0000 7f48ab400640 1 -- 172.21.15.67:0/4224547883 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac070ba0 con 0x7f4890003a50 2024-03-09T09:09:07.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.048+0000 7f48ab400640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890003a50 0x7f4890003e70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f489c001750 tx=0x7f4890005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274e955b132568ec server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:07.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.048+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c001370 con 0x7f4890003a50 2024-03-09T09:09:07.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.049+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c0052d0 con 0x7f4890003a50 2024-03-09T09:09:07.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.049+0000 7f48aaa00640 1 -- 172.21.15.67:0/4224547883 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c003830 con 0x7f4890003a50 2024-03-09T09:09:07.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.049+0000 7f48b0a00640 1 -- 172.21.15.67:0/4224547883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890003a50 msgr2=0x7f4890003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.049+0000 7f48b0a00640 1 --2- 172.21.15.67:0/4224547883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4890003a50 0x7f4890003e70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f489c001750 tx=0x7f4890005830 comp rx=0 tx=0).stop 2024-03-09T09:09:07.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.049+0000 7f48abe00640 1 -- 172.21.15.67:0/4224547883 reap_dead start 2024-03-09T09:09:07.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.049+0000 7f48b0a00640 1 -- 172.21.15.67:0/4224547883 shutdown_connections 2024-03-09T09:09:07.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.049+0000 7f48b0a00640 1 -- 172.21.15.67:0/4224547883 >> 172.21.15.67:0/4224547883 conn(0x7f48ac0c7220 msgr2=0x7f48ac0c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:07.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.049+0000 7f48b0a00640 1 -- 172.21.15.67:0/4224547883 shutdown_connections 2024-03-09T09:09:07.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.050+0000 7f48b0a00640 1 -- 172.21.15.67:0/4224547883 wait complete. 2024-03-09T09:09:07.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.052+0000 7f48b0a00640 1 Processor -- start 2024-03-09T09:09:07.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.052+0000 7f48b0a00640 1 -- start start 2024-03-09T09:09:07.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48b0a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4890003a50 0x7f48ac0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48b0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48ac1545f0 0x7f48ac146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48b0a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48ac1520b0 0x7f48ac146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48b0a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0ace90 con 0x7f48ac1545f0 2024-03-09T09:09:07.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48b0a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0acd10 con 0x7f48ac1520b0 2024-03-09T09:09:07.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48b0a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0c3660 con 0x7f4890003a50 2024-03-09T09:09:07.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48a2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48ac1545f0 0x7f48ac146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48a2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48ac1545f0 0x7f48ac146450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53810/0 (socket says 172.21.15.67:53810) 2024-03-09T09:09:07.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48a2a00640 1 -- 172.21.15.67:0/2518730700 learned_addr learned my addr 172.21.15.67:0/2518730700 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:07.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48ab400640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4890003a50 0x7f48ac0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48abe00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48ac1520b0 0x7f48ac146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48a2a00640 1 -- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4890003a50 msgr2=0x7f48ac0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48a2a00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4890003a50 0x7f48ac0c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48a2a00640 1 -- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48ac1520b0 msgr2=0x7f48ac146990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48a2a00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48ac1520b0 0x7f48ac146990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48a2a00640 1 -- 172.21.15.67:0/2518730700 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac154fb0 con 0x7f48ac1545f0 2024-03-09T09:09:07.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.053+0000 7f48abe00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48ac1520b0 0x7f48ac146990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:09:07.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.054+0000 7f48a2a00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48ac1545f0 0x7f48ac146450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f48940108c0 tx=0x7f4894010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:07.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.054+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489400f040 con 0x7f48ac1545f0 2024-03-09T09:09:07.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.054+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489400eb60 con 0x7f48ac1545f0 2024-03-09T09:09:07.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.054+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489401c400 con 0x7f48ac1545f0 2024-03-09T09:09:07.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.054+0000 7f48b0a00640 1 -- 172.21.15.67:0/2518730700 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac1e6e10 con 0x7f48ac1545f0 2024-03-09T09:09:07.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.055+0000 7f48b0a00640 1 -- 172.21.15.67:0/2518730700 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac1e73d0 con 0x7f48ac1545f0 2024-03-09T09:09:07.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.056+0000 7f48b0a00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1e75d0 0x7f48ac1ea390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.056+0000 7f48b0a00640 1 -- 172.21.15.67:0/2518730700 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac146ed0 con 0x7f48ac1e75d0 2024-03-09T09:09:07.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.057+0000 7f48ab400640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1e75d0 0x7f48ac1ea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.057+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f489401c5a0 con 0x7f48ac1545f0 2024-03-09T09:09:07.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.057+0000 7f48ab400640 1 -- 172.21.15.67:0/2518730700 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48ac04f3c0 con 0x7f48ac1e75d0 2024-03-09T09:09:07.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.058+0000 7f48a3e00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f488806b550 0x7f488806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.058+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f489408aad0 con 0x7f48ac1545f0 2024-03-09T09:09:07.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.058+0000 7f48ab400640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1e75d0 0x7f48ac1ea390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f489c001720 tx=0x7f489c001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:07.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.059+0000 7f48abe00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f488806b550 0x7f488806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.059+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c041070 con 0x7f48ac1e75d0 2024-03-09T09:09:07.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.060+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f489c0067d0 con 0x7f48ac1e75d0 2024-03-09T09:09:07.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.060+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1e75d0 msgr2=0x7f48ac1ea390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.060+0000 7f48a3e00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1e75d0 0x7f48ac1ea390 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f489c001720 tx=0x7f489c001120 comp rx=0 tx=0).stop 2024-03-09T09:09:07.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.060+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1e75d0 msgr2=0x7f48ac1ea390 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:07.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.060+0000 7f48a3e00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1e75d0 0x7f48ac1ea390 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.060+0000 7f48abe00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f488806b550 0x7f488806da10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0c3480 tx=0x7f48a4001270 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:07.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.069+0000 7f48b0a00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1ea8d0 0x7f48ac0ab400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.069+0000 7f48b0a00640 1 -- 172.21.15.67:0/2518730700 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac147f10 con 0x7f48ac1ea8d0 2024-03-09T09:09:07.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.069+0000 7f48ab400640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1ea8d0 0x7f48ac0ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.070+0000 7f48ab400640 1 -- 172.21.15.67:0/2518730700 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f48ac1e79b0 con 0x7f48ac1ea8d0 2024-03-09T09:09:07.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.071+0000 7f48ab400640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1ea8d0 0x7f48ac0ab400 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f489c001750 tx=0x7f489c030000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.071+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c007a30 con 0x7f48ac1ea8d0 2024-03-09T09:09:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.072+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f489c047e50 con 0x7f48ac1ea8d0 2024-03-09T09:09:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.072+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1ea8d0 msgr2=0x7f48ac0ab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.072+0000 7f48a3e00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1ea8d0 0x7f48ac0ab400 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f489c001750 tx=0x7f489c030000 comp rx=0 tx=0).stop 2024-03-09T09:09:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.072+0000 7f48a3e00640 1 -- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1ea8d0 msgr2=0x7f48ac0ab400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.072+0000 7f48a3e00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48ac1ea8d0 0x7f48ac0ab400 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f489c001750 tx=0x7f489c030000 comp rx=0 tx=0).stop 2024-03-09T09:09:07.077 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:09:07.077 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:09:07.077 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:09:07.077 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:09:07.077 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 118, 2024-03-09T09:09:07.077 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:09:07.077 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:09:07.077 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:09:07.078 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:09:07.078 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:07.078 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:09:07.078 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:07.078 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:09:07.078 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:09:07.078 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:07.078 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:07.078 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:07.078 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:07.079 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:09:07.080 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:09:07.081 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:09:07.081 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:07.081 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:07.081 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:09:07.081 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:09:07.081 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:09:07.081 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:09:07.081 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:07.082 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:07.083 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:07.083 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:07.083 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:09:07.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.083 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:09:07.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:07.083 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:07.084 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:07.085 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:09:07.086 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:07.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:07.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:07.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:09:07.086 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:07.086 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:07.086 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:07.087 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:07.088 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:07.089 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:07.090 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:07.090 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:07.090 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:07.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:07.090 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:09:07.090 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:09:07.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.072+0000 7f48b0a00640 1 -- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f488806b550 msgr2=0x7f488806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.072+0000 7f48b0a00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f488806b550 0x7f488806da10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0c3480 tx=0x7f48a4001270 comp rx=0 tx=0).stop 2024-03-09T09:09:07.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.072+0000 7f48b0a00640 1 -- 172.21.15.67:0/2518730700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48ac1545f0 msgr2=0x7f48ac146450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.072+0000 7f48b0a00640 1 --2- 172.21.15.67:0/2518730700 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48ac1545f0 0x7f48ac146450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f48940108c0 tx=0x7f4894010d80 comp rx=0 tx=0).stop 2024-03-09T09:09:07.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.072+0000 7f48abe00640 1 -- 172.21.15.67:0/2518730700 reap_dead start 2024-03-09T09:09:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.073+0000 7f48abe00640 1 -- 172.21.15.67:0/2518730700 reap_dead start 2024-03-09T09:09:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.073+0000 7f48b0a00640 1 -- 172.21.15.67:0/2518730700 shutdown_connections 2024-03-09T09:09:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.073+0000 7f48b0a00640 1 -- 172.21.15.67:0/2518730700 >> 172.21.15.67:0/2518730700 conn(0x7f48ac0c7220 msgr2=0x7f48ac0c1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.073+0000 7f48b0a00640 1 -- 172.21.15.67:0/2518730700 shutdown_connections 2024-03-09T09:09:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.073+0000 7f48b0a00640 1 -- 172.21.15.67:0/2518730700 wait complete. 2024-03-09T09:09:07.094 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:09:07.095 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-03-09T09:09:07.095 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:09:07.095 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-03-09T09:09:07.095 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:09:07.095 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-09T09:09:07.095 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:09:07.124 INFO:tasks.ceph.mon.b:Stopped 2024-03-09T09:09:07.124 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:09:07.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:09:07.237 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 2f039eeeb74 v0.34 2024-03-09T09:09:07.247 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.34 2024-03-09T09:09:07.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.334+0000 7f184be00640 1 Processor -- start 2024-03-09T09:09:07.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.334+0000 7f184be00640 1 -- start start 2024-03-09T09:09:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.336+0000 7f184be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f184c0c5e40 0x7f184c156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.336+0000 7f184be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f184c157000 con 0x7f184c0cadb0 2024-03-09T09:09:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.336+0000 7f184aa00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f184c153300 legacy=0x7f184c0b0000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:09:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.336+0000 7f184be00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f184c0ae430 con 0x7f184c153300 2024-03-09T09:09:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.336+0000 7f184be00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184c0ae6f0 con 0x7f184c0c5e40 2024-03-09T09:09:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.336+0000 7f1843e00640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f184c0c5e40 msgr2=0x7f184c156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.336+0000 7f1843e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f184c0c5e40 0x7f184c156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.336+0000 7f184b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f184c0cadb0 0x7f184c156550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46786/0 (socket says 172.21.15.67:46786) 2024-03-09T09:09:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.336+0000 7f184b400640 1 -- 172.21.15.67:0/164603484 learned_addr learned my addr 172.21.15.67:0/164603484 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38087725 0 0) 0x7f184c157000 con 0x7f184c0cadb0 2024-03-09T09:09:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1830003520 con 0x7f184c0cadb0 2024-03-09T09:09:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3465709899 0 0) 0x7f1830003520 con 0x7f184c0cadb0 2024-03-09T09:09:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 >> v2:172.21.15.134:3300/0 conn(0x7f184c0c5e40 msgr2=0x7f184c156010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 --2- 172.21.15.67:0/164603484 >> v2:172.21.15.134:3300/0 conn(0x7f184c0c5e40 0x7f184c156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 >> v1:172.21.15.134:6789/0 conn(0x7f184c153300 legacy=0x7f184c0b0000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c155b10 con 0x7f184c0cadb0 2024-03-09T09:09:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f18440019f0 con 0x7f184c0cadb0 2024-03-09T09:09:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 >> v1:172.21.15.67:6789/0 conn(0x7f184c0cadb0 legacy=0x7f184c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 --2- 172.21.15.67:0/164603484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1830003a20 0x7f1830003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 --2- 172.21.15.67:0/164603484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1830004800 0x7f1830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.337+0000 7f184a000640 1 --2- 172.21.15.67:0/164603484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18300052e0 0x7f183000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.338+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0ae6f0 con 0x7f18300052e0 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.338+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183000c1b0 con 0x7f1830004800 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.338+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183000c3b0 con 0x7f1830003a20 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.338+0000 7f1843e00640 1 -- 172.21.15.67:0/164603484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1830004800 msgr2=0x7f1830004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.338+0000 7f1843e00640 1 --2- 172.21.15.67:0/164603484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1830004800 0x7f1830004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.338+0000 7f184b400640 1 --2- 172.21.15.67:0/164603484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18300052e0 0x7f183000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.338+0000 7f184b400640 1 -- 172.21.15.67:0/164603484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1830003a20 msgr2=0x7f1830003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.338+0000 7f184b400640 1 --2- 172.21.15.67:0/164603484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1830003a20 0x7f1830003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.338+0000 7f184b400640 1 -- 172.21.15.67:0/164603484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1830004800 msgr2=0x7f1830004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.338+0000 7f184b400640 1 --2- 172.21.15.67:0/164603484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1830004800 0x7f1830004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.338+0000 7f184b400640 1 -- 172.21.15.67:0/164603484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830003520 con 0x7f18300052e0 2024-03-09T09:09:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.340+0000 7f184b400640 1 -- 172.21.15.67:0/164603484 reap_dead start 2024-03-09T09:09:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.340+0000 7f184b400640 1 --2- 172.21.15.67:0/164603484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18300052e0 0x7f183000b9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1844002230 tx=0x7f1830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cbf6c4443d9f46 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.340+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184400a480 con 0x7f18300052e0 2024-03-09T09:09:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.340+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1844034630 con 0x7f18300052e0 2024-03-09T09:09:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.340+0000 7f184a000640 1 -- 172.21.15.67:0/164603484 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844034a00 con 0x7f18300052e0 2024-03-09T09:09:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.341+0000 7f184be00640 1 -- 172.21.15.67:0/164603484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18300052e0 msgr2=0x7f183000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.341+0000 7f184be00640 1 --2- 172.21.15.67:0/164603484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18300052e0 0x7f183000b9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1844002230 tx=0x7f1830005830 comp rx=0 tx=0).stop 2024-03-09T09:09:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.341+0000 7f184be00640 1 -- 172.21.15.67:0/164603484 shutdown_connections 2024-03-09T09:09:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.341+0000 7f184be00640 1 --2- 172.21.15.67:0/164603484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18300052e0 0x7f183000b9c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.341+0000 7f184be00640 1 -- 172.21.15.67:0/164603484 >> 172.21.15.67:0/164603484 conn(0x7f184c0732d0 msgr2=0x7f184c0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.341+0000 7f184be00640 1 -- 172.21.15.67:0/164603484 shutdown_connections 2024-03-09T09:09:07.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.341+0000 7f184be00640 1 -- 172.21.15.67:0/164603484 wait complete. 2024-03-09T09:09:07.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.344+0000 7f184be00640 1 Processor -- start 2024-03-09T09:09:07.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.344+0000 7f184be00640 1 -- start start 2024-03-09T09:09:07.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.345+0000 7f184be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18300052e0 0x7f184c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.345+0000 7f184be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f184c1e0820 0x7f184c0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.345+0000 7f184be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f184c06b160 0x7f184c06b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.345+0000 7f184be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0c59e0 con 0x7f184c06b160 2024-03-09T09:09:07.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.345+0000 7f184be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c159e90 con 0x7f18300052e0 2024-03-09T09:09:07.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.345+0000 7f184be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0b0000 con 0x7f184c1e0820 2024-03-09T09:09:07.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.346+0000 7f184aa00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18300052e0 msgr2=0x7f184c06d260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:07.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.346+0000 7f184aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18300052e0 0x7f184c06d260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:07.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.346+0000 7f184b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f184c06b160 0x7f184c06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.346+0000 7f184b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f184c06b160 0x7f184c06b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53838/0 (socket says 172.21.15.67:53838) 2024-03-09T09:09:07.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.346+0000 7f184b400640 1 -- 172.21.15.67:0/3735281120 learned_addr learned my addr 172.21.15.67:0/3735281120 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:07.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.346+0000 7f1843e00640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f184c1e0820 0x7f184c0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.347+0000 7f184b400640 1 -- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f184c1e0820 msgr2=0x7f184c0aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.347+0000 7f184b400640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f184c1e0820 0x7f184c0aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.347+0000 7f184b400640 1 -- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18300052e0 msgr2=0x7f184c06d260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:09:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.347+0000 7f184b400640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18300052e0 0x7f184c06d260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.347+0000 7f184b400640 1 -- 172.21.15.67:0/3735281120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c06caa0 con 0x7f184c06b160 2024-03-09T09:09:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.347+0000 7f184b400640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f184c06b160 0x7f184c06b610 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1844005c20 tx=0x7f184c1536e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.347+0000 7f1842a00640 1 -- 172.21.15.67:0/3735281120 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844001fc0 con 0x7f184c06b160 2024-03-09T09:09:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.347+0000 7f1842a00640 1 -- 172.21.15.67:0/3735281120 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184403a040 con 0x7f184c06b160 2024-03-09T09:09:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.348+0000 7f1842a00640 1 -- 172.21.15.67:0/3735281120 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184400a460 con 0x7f184c06b160 2024-03-09T09:09:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.348+0000 7f184be00640 1 -- 172.21.15.67:0/3735281120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184c1df3c0 con 0x7f184c06b160 2024-03-09T09:09:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.349+0000 7f184be00640 1 -- 172.21.15.67:0/3735281120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c157a40 con 0x7f184c06b160 2024-03-09T09:09:07.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.350+0000 7f184be00640 1 -- 172.21.15.67:0/3735281120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184c04f3c0 con 0x7f184c06b160 2024-03-09T09:09:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.352+0000 7f1842a00640 1 -- 172.21.15.67:0/3735281120 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1844047020 con 0x7f184c06b160 2024-03-09T09:09:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.353+0000 7f1842a00640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f182806b4b0 0x7f182806d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.353+0000 7f1842a00640 1 -- 172.21.15.67:0/3735281120 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f18440ac020 con 0x7f184c06b160 2024-03-09T09:09:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.353+0000 7f184aa00640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f182806b4b0 0x7f182806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:07.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.355+0000 7f184aa00640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f182806b4b0 0x7f182806d970 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f184c158510 tx=0x7f183c007c70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:07.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.358+0000 7f1842a00640 1 -- 172.21.15.67:0/3735281120 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f184407ac60 con 0x7f184c06b160 2024-03-09T09:09:07.407 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:09:07.408 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 35 -eq 48 ']' 2024-03-09T09:09:07.408 INFO:tasks.workunit.client.1.smithi134.stdout:v0.35 2024-03-09T09:09:07.408 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.35 2024-03-09T09:09:07.408 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.35 2024-03-09T09:09:07.409 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.35 2024-03-09T09:09:07.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.684+0000 7f184be00640 1 -- 172.21.15.67:0/3735281120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f184c0ae430 con 0x7f184c06b160 2024-03-09T09:09:07.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.685+0000 7f1842a00640 1 -- 172.21.15.67:0/3735281120 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f184407a560 con 0x7f184c06b160 2024-03-09T09:09:07.689 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:09:07.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.687+0000 7f184be00640 1 -- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f182806b4b0 msgr2=0x7f182806d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.688+0000 7f184be00640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f182806b4b0 0x7f182806d970 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f184c158510 tx=0x7f183c007c70 comp rx=0 tx=0).stop 2024-03-09T09:09:07.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.688+0000 7f184be00640 1 -- 172.21.15.67:0/3735281120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f184c06b160 msgr2=0x7f184c06b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:07.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.688+0000 7f184be00640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f184c06b160 0x7f184c06b610 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1844005c20 tx=0x7f184c1536e0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.688+0000 7f184be00640 1 -- 172.21.15.67:0/3735281120 shutdown_connections 2024-03-09T09:09:07.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.688+0000 7f184be00640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f182806b4b0 0x7f182806d970 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.688+0000 7f184be00640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f184c06b160 0x7f184c06b610 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.688+0000 7f184be00640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f184c1e0820 0x7f184c0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.688+0000 7f184be00640 1 --2- 172.21.15.67:0/3735281120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18300052e0 0x7f184c06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:07.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.688+0000 7f184be00640 1 -- 172.21.15.67:0/3735281120 >> 172.21.15.67:0/3735281120 conn(0x7f184c0732d0 msgr2=0x7f184c1513c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:07.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.688+0000 7f184be00640 1 -- 172.21.15.67:0/3735281120 shutdown_connections 2024-03-09T09:09:07.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:07.688+0000 7f184be00640 1 -- 172.21.15.67:0/3735281120 wait complete. 2024-03-09T09:09:10.711 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:09:10.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.943+0000 7f10c0000640 1 Processor -- start 2024-03-09T09:09:10.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.943+0000 7f10c0000640 1 -- start start 2024-03-09T09:09:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.944+0000 7f10c0000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f10b8153370 0x7f10b80c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.944+0000 7f10c0000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f10b806dc00 0x7f10b80729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.944+0000 7f10c0000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b80c4010 con 0x7f10b8153750 2024-03-09T09:09:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.944+0000 7f10c0000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b8156dc0 con 0x7f10b806dc00 2024-03-09T09:09:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.944+0000 7f10c0000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b80c42f0 con 0x7f10b8153370 2024-03-09T09:09:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.944+0000 7f10bd600640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f10b8153370 msgr2=0x7f10b80c29a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.944+0000 7f10bd600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f10b8153370 0x7f10b80c29a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.945+0000 7f10bcc00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f10b8153750 legacy=0x7f10b80c2ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:09:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.945+0000 7f10be000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f10b806dc00 0x7f10b80729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.945+0000 7f10be000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f10b806dc00 0x7f10b80729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44024/0 (socket says 172.21.15.67:44024) 2024-03-09T09:09:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.945+0000 7f10be000640 1 -- 172.21.15.67:0/1209962517 learned_addr learned my addr 172.21.15.67:0/1209962517 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.945+0000 7f10be000640 1 -- 172.21.15.67:0/1209962517 >> v2:172.21.15.134:3300/0 conn(0x7f10b8153370 msgr2=0x7f10b80c29a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.945+0000 7f10be000640 1 --2- 172.21.15.67:0/1209962517 >> v2:172.21.15.134:3300/0 conn(0x7f10b8153370 0x7f10b80c29a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.945+0000 7f10be000640 1 -- 172.21.15.67:0/1209962517 >> v1:172.21.15.134:6789/0 conn(0x7f10b8153750 legacy=0x7f10b80c2ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.945+0000 7f10be000640 1 -- 172.21.15.67:0/1209962517 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b80c5a50 con 0x7f10b806dc00 2024-03-09T09:09:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.946+0000 7f10be000640 1 --2- 172.21.15.67:0/1209962517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10b806dc00 0x7f10b80729c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f10b80c5fd0 tx=0x7f10b002be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12c3262b491aa7c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.947+0000 7f10b7e00640 1 -- 172.21.15.67:0/1209962517 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b002ca10 con 0x7f10b806dc00 2024-03-09T09:09:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.947+0000 7f10b7e00640 1 -- 172.21.15.67:0/1209962517 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b002cbb0 con 0x7f10b806dc00 2024-03-09T09:09:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.947+0000 7f10b7e00640 1 -- 172.21.15.67:0/1209962517 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b00343f0 con 0x7f10b806dc00 2024-03-09T09:09:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.948+0000 7f10c0000640 1 -- 172.21.15.67:0/1209962517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10b806dc00 msgr2=0x7f10b80729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.948+0000 7f10c0000640 1 --2- 172.21.15.67:0/1209962517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10b806dc00 0x7f10b80729c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f10b80c5fd0 tx=0x7f10b002be70 comp rx=0 tx=0).stop 2024-03-09T09:09:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.948+0000 7f10c0000640 1 -- 172.21.15.67:0/1209962517 shutdown_connections 2024-03-09T09:09:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.948+0000 7f10c0000640 1 --2- 172.21.15.67:0/1209962517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10b806dc00 0x7f10b80729c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.948+0000 7f10c0000640 1 --2- 172.21.15.67:0/1209962517 >> v2:172.21.15.134:3300/0 conn(0x7f10b8153370 0x7f10b80c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.948+0000 7f10c0000640 1 -- 172.21.15.67:0/1209962517 >> 172.21.15.67:0/1209962517 conn(0x7f10b80ab320 msgr2=0x7f10b80ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:10.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.948+0000 7f10c0000640 1 -- 172.21.15.67:0/1209962517 shutdown_connections 2024-03-09T09:09:10.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.948+0000 7f10c0000640 1 -- 172.21.15.67:0/1209962517 wait complete. 2024-03-09T09:09:10.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.950+0000 7f10c0000640 1 Processor -- start 2024-03-09T09:09:10.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.950+0000 7f10c0000640 1 -- start start 2024-03-09T09:09:10.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.951+0000 7f10c0000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10b806dc00 0x7f10b81df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:10.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.951+0000 7f10c0000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10b8153370 0x7f10b8159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:10.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.951+0000 7f10c0000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b8153750 0x7f10b81e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:10.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.951+0000 7f10c0000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80c5d30 con 0x7f10b806dc00 2024-03-09T09:09:10.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.951+0000 7f10c0000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8073dc0 con 0x7f10b8153750 2024-03-09T09:09:10.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.951+0000 7f10c0000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8072f00 con 0x7f10b8153370 2024-03-09T09:09:10.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.952+0000 7f10be000640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b8153750 msgr2=0x7f10b81e2c00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:10.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.952+0000 7f10be000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b8153750 0x7f10b81e2c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:10.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.952+0000 7f10bd600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10b806dc00 0x7f10b81df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:10.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.952+0000 7f10bd600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10b806dc00 0x7f10b81df3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44028/0 (socket says 172.21.15.67:44028) 2024-03-09T09:09:10.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.952+0000 7f10bd600640 1 -- 172.21.15.67:0/4178241138 learned_addr learned my addr 172.21.15.67:0/4178241138 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.952+0000 7f10bcc00640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10b8153370 0x7f10b8159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.952+0000 7f10bd600640 1 -- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10b8153370 msgr2=0x7f10b8159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.952+0000 7f10bd600640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10b8153370 0x7f10b8159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.952+0000 7f10bd600640 1 -- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b8153750 msgr2=0x7f10b81e2c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.952+0000 7f10bd600640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b8153750 0x7f10b81e2c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.953+0000 7f10bd600640 1 -- 172.21.15.67:0/4178241138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b81df920 con 0x7f10b806dc00 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.953+0000 7f10bd600640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10b806dc00 0x7f10b81df3e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00b550 tx=0x7f10ac00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.953+0000 7f10b5600640 1 -- 172.21.15.67:0/4178241138 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac013020 con 0x7f10b806dc00 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.953+0000 7f10b5600640 1 -- 172.21.15.67:0/4178241138 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac003de0 con 0x7f10b806dc00 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.953+0000 7f10b5600640 1 -- 172.21.15.67:0/4178241138 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0052b0 con 0x7f10b806dc00 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.953+0000 7f10c0000640 1 -- 172.21.15.67:0/4178241138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b8150910 con 0x7f10b806dc00 2024-03-09T09:09:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.954+0000 7f10c0000640 1 -- 172.21.15.67:0/4178241138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b8159420 con 0x7f10b806dc00 2024-03-09T09:09:10.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.955+0000 7f10c0000640 1 -- 172.21.15.67:0/4178241138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b8159620 con 0x7f10b806dc00 2024-03-09T09:09:10.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.956+0000 7f10b5600640 1 -- 172.21.15.67:0/4178241138 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f10ac016020 con 0x7f10b806dc00 2024-03-09T09:09:10.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.957+0000 7f10b5600640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f109806b480 0x7f109806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:10.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.957+0000 7f10b5600640 1 -- 172.21.15.67:0/4178241138 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f10ac08a3c0 con 0x7f10b806dc00 2024-03-09T09:09:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.958+0000 7f10bcc00640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f109806b480 0x7f109806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.959+0000 7f10bcc00640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f109806b480 0x7f109806d940 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f10b80a9bc0 tx=0x7f10a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:10.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:10.962+0000 7f10b5600640 1 -- 172.21.15.67:0/4178241138 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f10b8159620 con 0x7f10b806dc00 2024-03-09T09:09:11.213 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 28% (95/334) Updating files: 29% (97/334) Updating files: 30% (101/334) Updating files: 31% (104/334) Updating files: 32% (107/334) Updating files: 33% (111/334) Updating files: 34% (114/334) Updating files: 35% (117/334) Updating files: 36% (121/334) Updating files: 37% (124/334) Updating files: 38% (127/334) Updating files: 39% (131/334) Updating files: 40% (134/334) Updating files: 41% (137/334) Updating files: 42% (141/334) Updating files: 43% (144/334) Updating files: 44% (147/334) Updating files: 45% (151/334) Updating files: 46% (154/334) Updating files: 47% (157/334) Updating files: 48% (161/334) Updating files: 49% (164/334) Updating files: 50% (167/334) Updating files: 51% (171/334) Updating files: 52% (174/334) Updating files: 53% (178/334) Updating files: 54% (181/334) Updating files: 55% (184/334) Updating files: 55% (187/334) Updating files: 56% (188/334) Updating files: 57% (191/334) Updating files: 58% (194/334) Updating files: 59% (198/334) Updating files: 60% (201/334) Updating files: 61% (204/334) Updating files: 62% (208/334) Updating files: 63% (211/334) Updating files: 64% (214/334) Updating files: 65% (218/334) Updating files: 66% (221/334) Updating files: 67% (224/334) Updating files: 68% (228/334) Updating files: 69% (231/334) Updating files: 69% (233/334) Updating files: 70% (234/334) Updating files: 71% (238/334) Updating files: 72% (241/334) Updating files: 73% (244/334) Updating files: 74% (248/334) Updating files: 75% (251/334) Updating files: 76% (254/334) Updating files: 77% (258/334) Updating files: 78% (261/334) Updating files: 79% (264/334) Updating files: 80% (268/334) Updating files: 81% (271/334) Updating files: 82% (274/334) Updating files: 83% (278/334) Updating files: 84% (281/334) Updating files: 85% (284/334) Updating files: 86% (288/334) Updating files: 87% (291/334) Updating files: 88% (294/334) Updating files: 89% (298/334) Updating files: 90% (301/334) Updating files: 91% (304/334) Updating files: 92% (308/334) Updating files: 93% (311/334) Updating files: 94% (314/334) Updating files: 95% (318/334) Updating files: 96% (321/334) Updating files: 97% (324/334) Updating files: 98% (328/334) Updating files: 99% (331/334) Updating files: 100% (334/334) Updating files: 100% (334/334), done. 2024-03-09T09:09:11.279 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 0afda378124 v0.35 2024-03-09T09:09:11.286 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.35 2024-03-09T09:09:11.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.385+0000 7f10c0000640 1 -- 172.21.15.67:0/4178241138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f10b8075a00 con 0x7f10b806dc00 2024-03-09T09:09:11.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.385+0000 7f10b5600640 1 -- 172.21.15.67:0/4178241138 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f10ac01d020 con 0x7f10b806dc00 2024-03-09T09:09:11.390 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:09:11.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.389+0000 7f10c0000640 1 -- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f109806b480 msgr2=0x7f109806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:11.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.389+0000 7f10c0000640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f109806b480 0x7f109806d940 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f10b80a9bc0 tx=0x7f10a4008040 comp rx=0 tx=0).stop 2024-03-09T09:09:11.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.389+0000 7f10c0000640 1 -- 172.21.15.67:0/4178241138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10b806dc00 msgr2=0x7f10b81df3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:11.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.389+0000 7f10c0000640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10b806dc00 0x7f10b81df3e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00b550 tx=0x7f10ac00ba10 comp rx=0 tx=0).stop 2024-03-09T09:09:11.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.389+0000 7f10c0000640 1 -- 172.21.15.67:0/4178241138 shutdown_connections 2024-03-09T09:09:11.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.389+0000 7f10c0000640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f109806b480 0x7f109806d940 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:11.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.389+0000 7f10c0000640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10b8153750 0x7f10b81e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:11.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.389+0000 7f10c0000640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10b8153370 0x7f10b8159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:11.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.389+0000 7f10c0000640 1 --2- 172.21.15.67:0/4178241138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f10b806dc00 0x7f10b81df3e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:11.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.389+0000 7f10c0000640 1 -- 172.21.15.67:0/4178241138 >> 172.21.15.67:0/4178241138 conn(0x7f10b80ab320 msgr2=0x7f10b80c5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:11.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.390+0000 7f10c0000640 1 -- 172.21.15.67:0/4178241138 shutdown_connections 2024-03-09T09:09:11.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:11.390+0000 7f10c0000640 1 -- 172.21.15.67:0/4178241138 wait complete. 2024-03-09T09:09:11.818 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:09:11.818 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 36 -eq 48 ']' 2024-03-09T09:09:11.819 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.36 2024-03-09T09:09:11.819 INFO:tasks.workunit.client.1.smithi134.stdout:v0.36 2024-03-09T09:09:11.819 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.36 2024-03-09T09:09:11.819 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.36 2024-03-09T09:09:14.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:09:14.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.620+0000 7f8ae3200640 1 Processor -- start 2024-03-09T09:09:14.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.621+0000 7f8ae3200640 1 -- start start 2024-03-09T09:09:14.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.622+0000 7f8ae3200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8adc073180 0x7f8adc0c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:14.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.622+0000 7f8ae3200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8adc06ed50 0x7f8adc1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:14.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.622+0000 7f8ae3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8adc0c5320 0x7f8adc159810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:14.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.622+0000 7f8ae3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc155b30 con 0x7f8adc0c5320 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.622+0000 7f8ae3200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc155df0 con 0x7f8adc06ed50 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.622+0000 7f8ae3200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc1560d0 con 0x7f8adc073180 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.622+0000 7f8ae1400640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f8adc06ed50 msgr2=0x7f8adc1592d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.623+0000 7f8ae1400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8adc06ed50 0x7f8adc1592d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.623+0000 7f8ae2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8adc0c5320 0x7f8adc159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.623+0000 7f8ae2800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8adc0c5320 0x7f8adc159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44034/0 (socket says 172.21.15.67:44034) 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.623+0000 7f8ae2800640 1 -- 172.21.15.67:0/2299449502 learned_addr learned my addr 172.21.15.67:0/2299449502 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.623+0000 7f8ae1e00640 1 --2- 172.21.15.67:0/2299449502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8adc073180 0x7f8adc0c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.623+0000 7f8ae2800640 1 -- 172.21.15.67:0/2299449502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8adc073180 msgr2=0x7f8adc0c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.623+0000 7f8ae2800640 1 --2- 172.21.15.67:0/2299449502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8adc073180 0x7f8adc0c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.623+0000 7f8ae2800640 1 -- 172.21.15.67:0/2299449502 >> v2:172.21.15.134:3300/0 conn(0x7f8adc06ed50 msgr2=0x7f8adc1592d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:14.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.623+0000 7f8ae2800640 1 --2- 172.21.15.67:0/2299449502 >> v2:172.21.15.134:3300/0 conn(0x7f8adc06ed50 0x7f8adc1592d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.623+0000 7f8ae2800640 1 -- 172.21.15.67:0/2299449502 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc1563b0 con 0x7f8adc0c5320 2024-03-09T09:09:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.624+0000 7f8ae2800640 1 --2- 172.21.15.67:0/2299449502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8adc0c5320 0x7f8adc159810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009e50 tx=0x7f8ad802bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4b55a8449fd229 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.625+0000 7f8ae0a00640 1 -- 172.21.15.67:0/2299449502 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad802caa0 con 0x7f8adc0c5320 2024-03-09T09:09:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.625+0000 7f8ae0a00640 1 -- 172.21.15.67:0/2299449502 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad802cc40 con 0x7f8adc0c5320 2024-03-09T09:09:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.625+0000 7f8ae0a00640 1 -- 172.21.15.67:0/2299449502 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80343f0 con 0x7f8adc0c5320 2024-03-09T09:09:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.625+0000 7f8ae3200640 1 -- 172.21.15.67:0/2299449502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8adc0c5320 msgr2=0x7f8adc159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.625+0000 7f8ae3200640 1 --2- 172.21.15.67:0/2299449502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8adc0c5320 0x7f8adc159810 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009e50 tx=0x7f8ad802bf00 comp rx=0 tx=0).stop 2024-03-09T09:09:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.625+0000 7f8ae3200640 1 -- 172.21.15.67:0/2299449502 shutdown_connections 2024-03-09T09:09:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.625+0000 7f8ae3200640 1 --2- 172.21.15.67:0/2299449502 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8adc0c5320 0x7f8adc159810 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.625+0000 7f8ae3200640 1 --2- 172.21.15.67:0/2299449502 >> v2:172.21.15.134:3300/0 conn(0x7f8adc06ed50 0x7f8adc1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.625+0000 7f8ae3200640 1 --2- 172.21.15.67:0/2299449502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8adc073180 0x7f8adc0c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.625+0000 7f8ae3200640 1 -- 172.21.15.67:0/2299449502 >> 172.21.15.67:0/2299449502 conn(0x7f8adc06c150 msgr2=0x7f8adc06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.626+0000 7f8ae3200640 1 -- 172.21.15.67:0/2299449502 shutdown_connections 2024-03-09T09:09:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.626+0000 7f8ae3200640 1 -- 172.21.15.67:0/2299449502 wait complete. 2024-03-09T09:09:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.627+0000 7f8ae3200640 1 Processor -- start 2024-03-09T09:09:14.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.627+0000 7f8ae3200640 1 -- start start 2024-03-09T09:09:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.628+0000 7f8ae3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc06ed50 0x7f8adc0c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.628+0000 7f8ae3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8adc073180 0x7f8adc1e2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.628+0000 7f8ae3200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8adc0c5320 0x7f8adc1e2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.628+0000 7f8ae3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc073560 con 0x7f8adc073180 2024-03-09T09:09:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.628+0000 7f8ae3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0c3920 con 0x7f8adc06ed50 2024-03-09T09:09:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.628+0000 7f8ae3200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc159d50 con 0x7f8adc0c5320 2024-03-09T09:09:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.628+0000 7f8ae1e00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc06ed50 msgr2=0x7f8adc0c7dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:14.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.628+0000 7f8ae1e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc06ed50 0x7f8adc0c7dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.628+0000 7f8ae1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8adc073180 0x7f8adc1e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.629+0000 7f8ae1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8adc073180 0x7f8adc1e2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44036/0 (socket says 172.21.15.67:44036) 2024-03-09T09:09:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.629+0000 7f8ae1400640 1 -- 172.21.15.67:0/1161104573 learned_addr learned my addr 172.21.15.67:0/1161104573 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.629+0000 7f8ae2800640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8adc0c5320 0x7f8adc1e2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.629+0000 7f8ae1400640 1 -- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8adc0c5320 msgr2=0x7f8adc1e2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.629+0000 7f8ae1400640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8adc0c5320 0x7f8adc1e2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.629+0000 7f8ae1400640 1 -- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc06ed50 msgr2=0x7f8adc0c7dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:09:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.629+0000 7f8ae1400640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc06ed50 0x7f8adc0c7dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.629+0000 7f8ae1400640 1 -- 172.21.15.67:0/1161104573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc0ae440 con 0x7f8adc073180 2024-03-09T09:09:14.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.629+0000 7f8ae1400640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8adc073180 0x7f8adc1e2350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00b500 tx=0x7f8acc00b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:14.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.630+0000 7f8ad6000640 1 -- 172.21.15.67:0/1161104573 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc003dc0 con 0x7f8adc073180 2024-03-09T09:09:14.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.630+0000 7f8ad6000640 1 -- 172.21.15.67:0/1161104573 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc015070 con 0x7f8adc073180 2024-03-09T09:09:14.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.630+0000 7f8ad6000640 1 -- 172.21.15.67:0/1161104573 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc005290 con 0x7f8adc073180 2024-03-09T09:09:14.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.630+0000 7f8ae3200640 1 -- 172.21.15.67:0/1161104573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc1ec390 con 0x7f8adc073180 2024-03-09T09:09:14.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.630+0000 7f8ae3200640 1 -- 172.21.15.67:0/1161104573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc0c6010 con 0x7f8adc073180 2024-03-09T09:09:14.635 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 29% (47/160) Updating files: 30% (48/160) Updating files: 31% (50/160) Updating files: 32% (52/160) Updating files: 33% (53/160) Updating files: 34% (55/160) Updating files: 35% (56/160) Updating files: 36% (58/160) Updating files: 37% (60/160) Updating files: 38% (61/160) Updating files: 39% (63/160) Updating files: 40% (64/160) Updating files: 41% (66/160) Updating files: 42% (68/160) Updating files: 43% (69/160) Updating files: 44% (71/160) Updating files: 45% (72/160) Updating files: 46% (74/160) Updating files: 47% (76/160) Updating files: 48% (77/160) Updating files: 49% (79/160) Updating files: 50% (80/160) Updating files: 51% (82/160) Updating files: 51% (83/160) Updating files: 52% (84/160) Updating files: 53% (85/160) Updating files: 54% (87/160) Updating files: 55% (88/160) Updating files: 56% (90/160) Updating files: 57% (92/160) Updating files: 58% (93/160) Updating files: 59% (95/160) Updating files: 60% (96/160) Updating files: 61% (98/160) Updating files: 62% (100/160) Updating files: 63% (101/160) Updating files: 64% (103/160) Updating files: 65% (104/160) Updating files: 66% (106/160) Updating files: 67% (108/160) Updating files: 68% (109/160) Updating files: 69% (111/160) Updating files: 70% (112/160) Updating files: 71% (114/160) Updating files: 72% (116/160) Updating files: 73% (117/160) Updating files: 74% (119/160) Updating files: 75% (120/160) Updating files: 76% (122/160) Updating files: 77% (124/160) Updating files: 78% (125/160) Updating files: 79% (127/160) Updating files: 80% (128/160) Updating files: 81% (130/160) Updating files: 82% (132/160) Updating files: 83% (133/160) Updating files: 84% (135/160) Updating files: 85% (136/160) Updating files: 86% (138/160) Updating files: 87% (140/160) Updating files: 88% (141/160) Updating files: 89% (143/160) Updating files: 90% (144/160) Updating files: 91% (146/160) Updating files: 92% (148/160) Updating files: 93% (149/160) Updating files: 94% (151/160) Updating files: 95% (152/160) Updating files: 96% (154/160) Updating files: 97% (156/160) Updating files: 98% (157/160) Updating files: 99% (159/160) Updating files: 100% (160/160) Updating files: 100% (160/160), done. 2024-03-09T09:09:14.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.631+0000 7f8ae3200640 1 -- 172.21.15.67:0/1161104573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc04f3c0 con 0x7f8adc073180 2024-03-09T09:09:14.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.632+0000 7f8ad6000640 1 -- 172.21.15.67:0/1161104573 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8acc005430 con 0x7f8adc073180 2024-03-09T09:09:14.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.634+0000 7f8ad6000640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ac006b480 0x7f8ac006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:14.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.634+0000 7f8ad6000640 1 -- 172.21.15.67:0/1161104573 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8acc08a3a0 con 0x7f8adc073180 2024-03-09T09:09:14.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.634+0000 7f8ae1e00640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ac006b480 0x7f8ac006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:14.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.635+0000 7f8ae1e00640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ac006b480 0x7f8ac006d940 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8adc151890 tx=0x7f8ad0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:14.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.638+0000 7f8ad6000640 1 -- 172.21.15.67:0/1161104573 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8acc00e340 con 0x7f8adc073180 2024-03-09T09:09:14.846 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 340b4348bf6 v0.33 2024-03-09T09:09:14.857 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.33 2024-03-09T09:09:14.934 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:09:14.934 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 34 -eq 48 ']' 2024-03-09T09:09:14.934 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.34 2024-03-09T09:09:14.934 INFO:tasks.workunit.client.0.smithi067.stdout:v0.34 2024-03-09T09:09:14.935 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.34 2024-03-09T09:09:14.935 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.34 2024-03-09T09:09:14.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.949+0000 7f8ae3200640 1 -- 172.21.15.67:0/1161104573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8adc06eb30 con 0x7f8adc073180 2024-03-09T09:09:14.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.949+0000 7f8ad6000640 1 -- 172.21.15.67:0/1161104573 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f8acc058ec0 con 0x7f8adc073180 2024-03-09T09:09:14.953 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:09:14.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 -- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ac006b480 msgr2=0x7f8ac006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:14.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ac006b480 0x7f8ac006d940 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8adc151890 tx=0x7f8ad0008040 comp rx=0 tx=0).stop 2024-03-09T09:09:14.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 -- 172.21.15.67:0/1161104573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8adc073180 msgr2=0x7f8adc1e2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:14.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8adc073180 0x7f8adc1e2350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00b500 tx=0x7f8acc00b9c0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 -- 172.21.15.67:0/1161104573 shutdown_connections 2024-03-09T09:09:14.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ac006b480 0x7f8ac006d940 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8adc0c5320 0x7f8adc1e2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8adc073180 0x7f8adc1e2350 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 --2- 172.21.15.67:0/1161104573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8adc06ed50 0x7f8adc0c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:14.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 -- 172.21.15.67:0/1161104573 >> 172.21.15.67:0/1161104573 conn(0x7f8adc06c150 msgr2=0x7f8adc0c2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:14.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 -- 172.21.15.67:0/1161104573 shutdown_connections 2024-03-09T09:09:14.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:14.953+0000 7f8ae3200640 1 -- 172.21.15.67:0/1161104573 wait complete. 2024-03-09T09:09:15.982 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 52% (128/244) Updating files: 53% (130/244) Updating files: 54% (132/244) Updating files: 55% (135/244) Updating files: 56% (137/244) Updating files: 57% (140/244) Updating files: 58% (142/244) Updating files: 59% (144/244) Updating files: 60% (147/244) Updating files: 61% (149/244) Updating files: 62% (152/244) Updating files: 63% (154/244) Updating files: 64% (157/244) Updating files: 64% (158/244) Updating files: 65% (159/244) Updating files: 66% (162/244) Updating files: 67% (164/244) Updating files: 68% (166/244) Updating files: 69% (169/244) Updating files: 70% (171/244) Updating files: 71% (174/244) Updating files: 72% (176/244) Updating files: 73% (179/244) Updating files: 74% (181/244) Updating files: 75% (183/244) Updating files: 76% (186/244) Updating files: 77% (188/244) Updating files: 78% (191/244) Updating files: 78% (192/244) Updating files: 79% (193/244) Updating files: 80% (196/244) Updating files: 81% (198/244) Updating files: 82% (201/244) Updating files: 83% (203/244) Updating files: 84% (205/244) Updating files: 85% (208/244) Updating files: 86% (210/244) Updating files: 87% (213/244) Updating files: 88% (215/244) Updating files: 89% (218/244) Updating files: 90% (220/244) Updating files: 91% (223/244) Updating files: 92% (225/244) Updating files: 93% (227/244) Updating files: 93% (228/244) Updating files: 94% (230/244) Updating files: 95% (232/244) Updating files: 96% (235/244) Updating files: 97% (237/244) Updating files: 98% (240/244) Updating files: 99% (242/244) Updating files: 100% (244/244) Updating files: 100% (244/244), done. 2024-03-09T09:09:16.050 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 877cacbbdbb v0.36 2024-03-09T09:09:16.058 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.36 2024-03-09T09:09:16.321 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:09:16.322 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 37 -eq 48 ']' 2024-03-09T09:09:16.322 INFO:tasks.workunit.client.1.smithi134.stdout:v0.37 2024-03-09T09:09:16.322 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.37 2024-03-09T09:09:16.322 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.37 2024-03-09T09:09:16.322 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.37 2024-03-09T09:09:16.649 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:09:16.655 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:09:17.033 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 21% (28/129) Updating files: 22% (29/129) Updating files: 23% (30/129) Updating files: 24% (31/129) Updating files: 25% (33/129) Updating files: 26% (34/129) Updating files: 27% (35/129) Updating files: 28% (37/129) Updating files: 29% (38/129) Updating files: 30% (39/129) Updating files: 31% (40/129) Updating files: 32% (42/129) Updating files: 33% (43/129) Updating files: 34% (44/129) Updating files: 35% (46/129) Updating files: 36% (47/129) Updating files: 37% (48/129) Updating files: 38% (50/129) Updating files: 39% (51/129) Updating files: 40% (52/129) Updating files: 41% (53/129) Updating files: 42% (55/129) Updating files: 43% (56/129) Updating files: 44% (57/129) Updating files: 45% (59/129) Updating files: 46% (60/129) Updating files: 47% (61/129) Updating files: 48% (62/129) Updating files: 49% (64/129) Updating files: 50% (65/129) Updating files: 51% (66/129) Updating files: 52% (68/129) Updating files: 53% (69/129) Updating files: 54% (70/129) Updating files: 55% (71/129) Updating files: 56% (73/129) Updating files: 57% (74/129) Updating files: 58% (75/129) Updating files: 59% (77/129) Updating files: 60% (78/129) Updating files: 61% (79/129) Updating files: 62% (80/129) Updating files: 63% (82/129) Updating files: 64% (83/129) Updating files: 65% (84/129) Updating files: 66% (86/129) Updating files: 67% (87/129) Updating files: 68% (88/129) Updating files: 69% (90/129) Updating files: 70% (91/129) Updating files: 71% (92/129) Updating files: 72% (93/129) Updating files: 73% (95/129) Updating files: 74% (96/129) Updating files: 75% (97/129) Updating files: 76% (99/129) Updating files: 77% (100/129) Updating files: 78% (101/129) Updating files: 79% (102/129) Updating files: 80% (104/129) Updating files: 81% (105/129) Updating files: 82% (106/129) Updating files: 83% (108/129) Updating files: 84% (109/129) Updating files: 85% (110/129) Updating files: 86% (111/129) Updating files: 87% (113/129) Updating files: 88% (114/129) Updating files: 89% (115/129) Updating files: 90% (117/129) Updating files: 91% (118/129) Updating files: 92% (119/129) Updating files: 93% (120/129) Updating files: 94% (122/129) Updating files: 95% (123/129) Updating files: 96% (124/129) Updating files: 97% (126/129) Updating files: 98% (127/129) Updating files: 99% (128/129) Updating files: 100% (129/129) Updating files: 100% (129/129), done. 2024-03-09T09:09:17.101 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 2f039eeeb74 v0.34 2024-03-09T09:09:17.110 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.34 2024-03-09T09:09:17.871 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:09:17.871 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 35 -eq 48 ']' 2024-03-09T09:09:17.872 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.35 2024-03-09T09:09:17.872 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.35 2024-03-09T09:09:17.872 INFO:tasks.workunit.client.0.smithi067.stdout:v0.35 2024-03-09T09:09:17.872 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.35 2024-03-09T09:09:17.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:09:18.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.238+0000 7fdfa3000640 1 Processor -- start 2024-03-09T09:09:18.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.238+0000 7fdfa3000640 1 -- start start 2024-03-09T09:09:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.239+0000 7fdfa3000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.239+0000 7fdfa3000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9c0c42b0 con 0x7fdf9c150350 2024-03-09T09:09:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.239+0000 7fdfa3000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9c150730 con 0x7fdf9c071cb0 2024-03-09T09:09:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.239+0000 7fdfa3000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0c4e80 con 0x7fdf9c06e800 2024-03-09T09:09:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.240+0000 7fdfa1000640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7fdf9c071cb0 legacy=0x7fdf9c15d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:09:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.240+0000 7fdf9be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdf9c150350 0x7fdf9c072170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42086/0 (socket says 172.21.15.67:42086) 2024-03-09T09:09:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.240+0000 7fdf9be00640 1 -- 172.21.15.67:0/1153049841 learned_addr learned my addr 172.21.15.67:0/1153049841 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:18.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.240+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:18.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.240+0000 7fdf9b400640 1 -- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 msgr2=0x7fdf9c0c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:09:18.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.240+0000 7fdf9b400640 1 -- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 msgr2=0x7fdf9c0c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:09:18.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.240+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:09:18.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.240+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:18.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.441+0000 7fdfa1000640 1 -- 172.21.15.67:0/1153049841 >> v1:172.21.15.134:6789/0 conn(0x7fdf9c071cb0 legacy=0x7fdf9c15d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:09:18.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.441+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:18.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.442+0000 7fdf9b400640 1 -- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 msgr2=0x7fdf9c0c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:09:18.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.442+0000 7fdf9b400640 1 -- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 msgr2=0x7fdf9c0c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:09:18.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.442+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:09:18.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.442+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:09:18.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.842+0000 7fdfa1000640 1 -- 172.21.15.67:0/1153049841 >> v1:172.21.15.134:6789/0 conn(0x7fdf9c071cb0 legacy=0x7fdf9c15d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:09:18.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.843+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:18.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.844+0000 7fdf9b400640 1 -- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 msgr2=0x7fdf9c0c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:09:18.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.844+0000 7fdf9b400640 1 -- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 msgr2=0x7fdf9c0c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:09:18.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.844+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:09:18.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:18.844+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:09:19.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:19.643+0000 7fdfa1000640 1 -- 172.21.15.67:0/1153049841 >> v1:172.21.15.134:6789/0 conn(0x7fdf9c071cb0 legacy=0x7fdf9c15d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:09:19.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:19.645+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:19.646+0000 7fdf9b400640 1 -- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 msgr2=0x7fdf9c0c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:09:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:19.646+0000 7fdf9b400640 1 -- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 msgr2=0x7fdf9c0c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:09:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:19.646+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:09:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:19.646+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:09:21.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.240+0000 7fdf99600640 1 -- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 msgr2=0x7fdf9c0c5050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:21.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.240+0000 7fdf99600640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf9c06e800 0x7fdf9c0c5050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:21.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.240+0000 7fdf99600640 1 -- 172.21.15.67:0/1153049841 >> v1:172.21.15.134:6789/0 conn(0x7fdf9c071cb0 legacy=0x7fdf9c15d580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:21.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.240+0000 7fdf99600640 1 -- 172.21.15.67:0/1153049841 >> v1:172.21.15.67:6789/0 conn(0x7fdf9c150350 legacy=0x7fdf9c072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:21.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.240+0000 7fdf99600640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 0x7fdf7c005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:21.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.240+0000 7fdf99600640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.134:3300/0 conn(0x7fdf7c006240 0x7fdf7c008700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:21.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.240+0000 7fdf99600640 1 -- 172.21.15.67:0/1153049841 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9c0c4e80 con 0x7fdf7c0036d0 2024-03-09T09:09:21.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.240+0000 7fdf99600640 1 -- 172.21.15.67:0/1153049841 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7c008dc0 con 0x7fdf7c006240 2024-03-09T09:09:21.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.241+0000 7fdf9b400640 1 -- 172.21.15.67:0/1153049841 >> v2:172.21.15.134:3300/0 conn(0x7fdf7c006240 msgr2=0x7fdf7c008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:21.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.241+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.134:3300/0 conn(0x7fdf7c006240 0x7fdf7c008700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:21.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.241+0000 7fdf9be00640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 0x7fdf7c005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:21.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.241+0000 7fdf9be00640 1 -- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 msgr2=0x7fdf7c005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:09:21.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.241+0000 7fdf9be00640 1 -- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 msgr2=0x7fdf7c005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:09:21.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.241+0000 7fdf9be00640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 0x7fdf7c005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:09:21.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.241+0000 7fdf9be00640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 0x7fdf7c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.442+0000 7fdf9b400640 1 -- 172.21.15.67:0/1153049841 >> v2:172.21.15.134:3300/0 conn(0x7fdf7c006240 msgr2=0x7fdf7c008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.442+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.134:3300/0 conn(0x7fdf7c006240 0x7fdf7c008700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:09:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.442+0000 7fdf9be00640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 0x7fdf7c005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.443+0000 7fdf9be00640 1 -- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 msgr2=0x7fdf7c005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:09:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.443+0000 7fdf9be00640 1 -- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 msgr2=0x7fdf7c005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:09:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.443+0000 7fdf9be00640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 0x7fdf7c005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:09:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.443+0000 7fdf9be00640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 0x7fdf7c005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:09:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.843+0000 7fdf9b400640 1 -- 172.21.15.67:0/1153049841 >> v2:172.21.15.134:3300/0 conn(0x7fdf7c006240 msgr2=0x7fdf7c008700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.843+0000 7fdf9b400640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.134:3300/0 conn(0x7fdf7c006240 0x7fdf7c008700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:09:21.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.844+0000 7fdf9be00640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.67:3300/0 conn(0x7fdf7c0036d0 0x7fdf7c005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:21.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.845+0000 7fdf9be00640 1 -- 172.21.15.67:0/1153049841 >> v2:172.21.15.134:3300/0 conn(0x7fdf7c006240 msgr2=0x7fdf7c008700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:21.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.845+0000 7fdf9be00640 1 --2- 172.21.15.67:0/1153049841 >> v2:172.21.15.134:3300/0 conn(0x7fdf7c006240 0x7fdf7c008700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:21.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.845+0000 7fdf9be00640 1 -- 172.21.15.67:0/1153049841 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c0c4ba0 con 0x7fdf7c0036d0 2024-03-09T09:09:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.848+0000 7fdf9be00640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf7c0036d0 0x7fdf7c005b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c006210 tx=0x7fdf8c02bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb58653fc3608665 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.849+0000 7fdf9aa00640 1 -- 172.21.15.67:0/1153049841 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c032070 con 0x7fdf7c0036d0 2024-03-09T09:09:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.849+0000 7fdf9aa00640 1 -- 172.21.15.67:0/1153049841 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c02c760 con 0x7fdf7c0036d0 2024-03-09T09:09:21.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.849+0000 7fdf9aa00640 1 -- 172.21.15.67:0/1153049841 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c02cb30 con 0x7fdf7c0036d0 2024-03-09T09:09:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.851+0000 7fdfa3000640 1 -- 172.21.15.67:0/1153049841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf7c0036d0 msgr2=0x7fdf7c005b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.851+0000 7fdfa3000640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf7c0036d0 0x7fdf7c005b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c006210 tx=0x7fdf8c02bbc0 comp rx=0 tx=0).stop 2024-03-09T09:09:21.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.851+0000 7fdfa1000640 1 -- 172.21.15.67:0/1153049841 reap_dead start 2024-03-09T09:09:21.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.851+0000 7fdfa3000640 1 -- 172.21.15.67:0/1153049841 shutdown_connections 2024-03-09T09:09:21.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.851+0000 7fdfa3000640 1 --2- 172.21.15.67:0/1153049841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf7c0036d0 0x7fdf7c005b50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:21.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.851+0000 7fdfa3000640 1 -- 172.21.15.67:0/1153049841 >> 172.21.15.67:0/1153049841 conn(0x7fdf9c0aaad0 msgr2=0x7fdf9c06c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:21.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.852+0000 7fdfa3000640 1 -- 172.21.15.67:0/1153049841 shutdown_connections 2024-03-09T09:09:21.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.852+0000 7fdfa3000640 1 -- 172.21.15.67:0/1153049841 wait complete. 2024-03-09T09:09:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.858+0000 7fdfa3000640 1 Processor -- start 2024-03-09T09:09:21.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.859+0000 7fdfa3000640 1 -- start start 2024-03-09T09:09:21.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdfa3000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf7c0036d0 0x7fdf9c1ebe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:21.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdfa3000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c0ae180 0x7fdf9c14dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:21.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdfa3000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf9c14e120 0x7fdf9c14ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:21.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdfa3000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c153b70 con 0x7fdf9c14e120 2024-03-09T09:09:21.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdfa3000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0c3d90 con 0x7fdf9c0ae180 2024-03-09T09:09:21.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdfa3000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c06cf10 con 0x7fdf7c0036d0 2024-03-09T09:09:21.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdf9b400640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c0ae180 msgr2=0x7fdf9c14dbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:21.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdf9b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c0ae180 0x7fdf9c14dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdfa1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf9c14e120 0x7fdf9c14ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdfa1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf9c14e120 0x7fdf9c14ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44178/0 (socket says 172.21.15.67:44178) 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdfa1000640 1 -- 172.21.15.67:0/224468287 learned_addr learned my addr 172.21.15.67:0/224468287 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.862+0000 7fdf9be00640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf7c0036d0 0x7fdf9c1ebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.863+0000 7fdfa1000640 1 -- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf7c0036d0 msgr2=0x7fdf9c1ebe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.863+0000 7fdfa1000640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf7c0036d0 0x7fdf9c1ebe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.863+0000 7fdfa1000640 1 -- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c0ae180 msgr2=0x7fdf9c14dbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.863+0000 7fdfa1000640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c0ae180 0x7fdf9c14dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.863+0000 7fdfa1000640 1 -- 172.21.15.67:0/224468287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c14c600 con 0x7fdf9c14e120 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.863+0000 7fdfa1000640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf9c14e120 0x7fdf9c14ad50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdf94003cb0 tx=0x7fdf9400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.864+0000 7fdf93e00640 1 -- 172.21.15.67:0/224468287 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9400ce30 con 0x7fdf9c14e120 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.864+0000 7fdf93e00640 1 -- 172.21.15.67:0/224468287 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf94010030 con 0x7fdf9c14e120 2024-03-09T09:09:21.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.864+0000 7fdfa3000640 1 -- 172.21.15.67:0/224468287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c14c320 con 0x7fdf9c14e120 2024-03-09T09:09:21.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.864+0000 7fdf93e00640 1 -- 172.21.15.67:0/224468287 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94016690 con 0x7fdf9c14e120 2024-03-09T09:09:21.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.864+0000 7fdfa3000640 1 -- 172.21.15.67:0/224468287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c1eb610 con 0x7fdf9c14e120 2024-03-09T09:09:21.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.866+0000 7fdfa3000640 1 -- 172.21.15.67:0/224468287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9c075a00 con 0x7fdf9c14e120 2024-03-09T09:09:21.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.867+0000 7fdf93e00640 1 -- 172.21.15.67:0/224468287 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdf940153e0 con 0x7fdf9c14e120 2024-03-09T09:09:21.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.868+0000 7fdf93e00640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf7c06c290 0x7fdf7c06e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:21.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.868+0000 7fdf93e00640 1 -- 172.21.15.67:0/224468287 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdf9408a3c0 con 0x7fdf9c14e120 2024-03-09T09:09:21.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.869+0000 7fdf9be00640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf7c06c290 0x7fdf7c06e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:21.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.870+0000 7fdf9be00640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf7c06c290 0x7fdf7c06e750 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c037000 tx=0x7fdf8c010040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:21.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:21.874+0000 7fdf93e00640 1 -- 172.21.15.67:0/224468287 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdf94058970 con 0x7fdf9c14e120 2024-03-09T09:09:22.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.154+0000 7fdfa3000640 1 -- 172.21.15.67:0/224468287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdf9c0c31e0 con 0x7fdf9c14e120 2024-03-09T09:09:22.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.155+0000 7fdf93e00640 1 -- 172.21.15.67:0/224468287 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fdf9405ca10 con 0x7fdf9c14e120 2024-03-09T09:09:22.159 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":122,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:09:22.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 -- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf7c06c290 msgr2=0x7fdf7c06e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf7c06c290 0x7fdf7c06e750 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c037000 tx=0x7fdf8c010040 comp rx=0 tx=0).stop 2024-03-09T09:09:22.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 -- 172.21.15.67:0/224468287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf9c14e120 msgr2=0x7fdf9c14ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf9c14e120 0x7fdf9c14ad50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdf94003cb0 tx=0x7fdf9400be60 comp rx=0 tx=0).stop 2024-03-09T09:09:22.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 -- 172.21.15.67:0/224468287 shutdown_connections 2024-03-09T09:09:22.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf7c06c290 0x7fdf7c06e750 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf9c14e120 0x7fdf9c14ad50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf9c0ae180 0x7fdf9c14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 --2- 172.21.15.67:0/224468287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf7c0036d0 0x7fdf9c1ebe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 -- 172.21.15.67:0/224468287 >> 172.21.15.67:0/224468287 conn(0x7fdf9c0aaad0 msgr2=0x7fdf9c06c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:22.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 -- 172.21.15.67:0/224468287 shutdown_connections 2024-03-09T09:09:22.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.158+0000 7fdfa3000640 1 -- 172.21.15.67:0/224468287 wait complete. 2024-03-09T09:09:22.180 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:09:22.180 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:09:22.180 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:09:22.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.396+0000 7fd478c00640 1 Processor -- start 2024-03-09T09:09:22.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.396+0000 7fd478c00640 1 -- start start 2024-03-09T09:09:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.397+0000 7fd478c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c5ff0 0x7fd4740ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.397+0000 7fd478c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd474155e70 con 0x7fd4740bed00 2024-03-09T09:09:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.397+0000 7fd478c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4740c9710 con 0x7fd47414fef0 2024-03-09T09:09:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.397+0000 7fd478c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740c9d60 con 0x7fd4740c5ff0 2024-03-09T09:09:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.397+0000 7fd471e00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7fd47414fef0 legacy=0x7fd47406b040 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:09:22.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.397+0000 7fd472800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd4740bed00 0x7fd4740ca730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50246/0 (socket says 172.21.15.67:50246) 2024-03-09T09:09:22.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.397+0000 7fd472800640 1 -- 172.21.15.67:0/56671309 learned_addr learned my addr 172.21.15.67:0/56671309 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd471400640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c5ff0 0x7fd4740ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674357148 0 0) 0x7fd474155e70 con 0x7fd4740bed00 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd454003520 con 0x7fd4740bed00 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 599365164 0 0) 0x7fd454003520 con 0x7fd4740bed00 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c5ff0 msgr2=0x7fd4740ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c5ff0 0x7fd4740ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 >> v1:172.21.15.134:6789/0 conn(0x7fd47414fef0 legacy=0x7fd47406b040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4740c99f0 con 0x7fd4740bed00 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fd46c001670 con 0x7fd4740bed00 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 >> v1:172.21.15.67:6789/0 conn(0x7fd4740bed00 legacy=0x7fd4740ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c5ff0 0x7fd4540039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd454003f40 0x7fd454004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd454004d40 0x7fd4540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740c9d60 con 0x7fd454004d40 2024-03-09T09:09:22.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45400bad0 con 0x7fd454003f40 2024-03-09T09:09:22.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.398+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45400bcd0 con 0x7fd4740c5ff0 2024-03-09T09:09:22.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.399+0000 7fd471400640 1 -- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd454003f40 msgr2=0x7fd454004800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:22.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.399+0000 7fd471400640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd454003f40 0x7fd454004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:22.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.399+0000 7fd472800640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd454004d40 0x7fd4540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.399+0000 7fd471e00640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c5ff0 0x7fd4540039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.399+0000 7fd472800640 1 -- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c5ff0 msgr2=0x7fd4540039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:22.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.399+0000 7fd472800640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c5ff0 0x7fd4540039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.399+0000 7fd472800640 1 -- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd454003f40 msgr2=0x7fd454004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:22.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.399+0000 7fd472800640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd454003f40 0x7fd454004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.399+0000 7fd472800640 1 -- 172.21.15.67:0/56671309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454003520 con 0x7fd454004d40 2024-03-09T09:09:22.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.400+0000 7fd472800640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd454004d40 0x7fd4540051c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c210 tx=0x7fd454005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81992458d3e01b8f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:22.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.401+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c007d30 con 0x7fd454004d40 2024-03-09T09:09:22.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.401+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c030070 con 0x7fd454004d40 2024-03-09T09:09:22.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.401+0000 7fd470a00640 1 -- 172.21.15.67:0/56671309 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c005d30 con 0x7fd454004d40 2024-03-09T09:09:22.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.401+0000 7fd478c00640 1 -- 172.21.15.67:0/56671309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd454004d40 msgr2=0x7fd4540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.401+0000 7fd478c00640 1 --2- 172.21.15.67:0/56671309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd454004d40 0x7fd4540051c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c210 tx=0x7fd454005830 comp rx=0 tx=0).stop 2024-03-09T09:09:22.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.401+0000 7fd472800640 1 -- 172.21.15.67:0/56671309 reap_dead start 2024-03-09T09:09:22.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.401+0000 7fd478c00640 1 -- 172.21.15.67:0/56671309 shutdown_connections 2024-03-09T09:09:22.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.401+0000 7fd478c00640 1 -- 172.21.15.67:0/56671309 >> 172.21.15.67:0/56671309 conn(0x7fd4740cb670 msgr2=0x7fd4740cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:22.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.402+0000 7fd478c00640 1 -- 172.21.15.67:0/56671309 shutdown_connections 2024-03-09T09:09:22.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.402+0000 7fd478c00640 1 -- 172.21.15.67:0/56671309 wait complete. 2024-03-09T09:09:22.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.403+0000 7fd478c00640 1 Processor -- start 2024-03-09T09:09:22.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.403+0000 7fd478c00640 1 -- start start 2024-03-09T09:09:22.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.404+0000 7fd478c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd454004d40 0x7fd4741eb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.404+0000 7fd478c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c7a20 0x7fd4740c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.404+0000 7fd478c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4740c1b30 0x7fd4741592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.404+0000 7fd478c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47406b8d0 con 0x7fd454004d40 2024-03-09T09:09:22.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.404+0000 7fd478c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47406b750 con 0x7fd4740c1b30 2024-03-09T09:09:22.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.404+0000 7fd478c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740cb2f0 con 0x7fd4740c7a20 2024-03-09T09:09:22.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd472800640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4740c1b30 msgr2=0x7fd4741592c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd471e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd454004d40 0x7fd4741eb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd472800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4740c1b30 0x7fd4741592c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd471e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd454004d40 0x7fd4741eb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44210/0 (socket says 172.21.15.67:44210) 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd471e00640 1 -- 172.21.15.67:0/2021976686 learned_addr learned my addr 172.21.15.67:0/2021976686 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd471400640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c7a20 0x7fd4740c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd471e00640 1 -- 172.21.15.67:0/2021976686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c7a20 msgr2=0x7fd4740c15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd471e00640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c7a20 0x7fd4740c15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd471e00640 1 -- 172.21.15.67:0/2021976686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4740c1b30 msgr2=0x7fd4741592c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd471e00640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4740c1b30 0x7fd4741592c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd471e00640 1 -- 172.21.15.67:0/2021976686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4741eee20 con 0x7fd454004d40 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd471400640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd4740c7a20 0x7fd4740c15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.405+0000 7fd471e00640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd454004d40 0x7fd4741eb4f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd4640096c0 tx=0x7fd464009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.406+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464016070 con 0x7fd454004d40 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.406+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46400e750 con 0x7fd454004d40 2024-03-09T09:09:22.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.406+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46400eb40 con 0x7fd454004d40 2024-03-09T09:09:22.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.406+0000 7fd478c00640 1 -- 172.21.15.67:0/2021976686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4741ef100 con 0x7fd454004d40 2024-03-09T09:09:22.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.406+0000 7fd478c00640 1 -- 172.21.15.67:0/2021976686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4741ef6c0 con 0x7fd454004d40 2024-03-09T09:09:22.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.407+0000 7fd478c00640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474159800 0x7fd474159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.407+0000 7fd478c00640 1 -- 172.21.15.67:0/2021976686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd474158f70 con 0x7fd474159800 2024-03-09T09:09:22.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.407+0000 7fd471400640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474159800 0x7fd474159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.408+0000 7fd471400640 1 -- 172.21.15.67:0/2021976686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd474075a00 con 0x7fd474159800 2024-03-09T09:09:22.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.408+0000 7fd471400640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474159800 0x7fd474159be0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd4740c6450 tx=0x7fd45c00e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:22.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.408+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0137d0 con 0x7fd474159800 2024-03-09T09:09:22.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.409+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd464019070 con 0x7fd454004d40 2024-03-09T09:09:22.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.409+0000 7fd46a000640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd45006b400 0x7fd45006d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.410+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd46408a370 con 0x7fd454004d40 2024-03-09T09:09:22.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.410+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd45c01a070 con 0x7fd474159800 2024-03-09T09:09:22.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.410+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474159800 msgr2=0x7fd474159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.410+0000 7fd46a000640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474159800 0x7fd474159be0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd4740c6450 tx=0x7fd45c00e800 comp rx=0 tx=0).stop 2024-03-09T09:09:22.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.410+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474159800 msgr2=0x7fd474159be0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:22.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.410+0000 7fd46a000640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474159800 0x7fd474159be0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.410+0000 7fd472800640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd45006b400 0x7fd45006d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.411+0000 7fd472800640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd45006b400 0x7fd45006d8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd46c034000 tx=0x7fd46c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.417+0000 7fd478c00640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474157150 0x7fd474157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.417+0000 7fd478c00640 1 -- 172.21.15.67:0/2021976686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741ef890 con 0x7fd474157150 2024-03-09T09:09:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.417+0000 7fd471400640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474157150 0x7fd474157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.418+0000 7fd471400640 1 -- 172.21.15.67:0/2021976686 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd4740cb0d0 con 0x7fd474157150 2024-03-09T09:09:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.418+0000 7fd471400640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474157150 0x7fd474157550 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd45c019000 tx=0x7fd45c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c013e60 con 0x7fd474157150 2024-03-09T09:09:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7fd45c029430 con 0x7fd474157150 2024-03-09T09:09:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474157150 msgr2=0x7fd474157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd46a000640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474157150 0x7fd474157550 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd45c019000 tx=0x7fd45c004a40 comp rx=0 tx=0).stop 2024-03-09T09:09:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd46a000640 1 -- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474157150 msgr2=0x7fd474157550 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd46a000640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd474157150 0x7fd474157550 secure :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd45c019000 tx=0x7fd45c004a40 comp rx=0 tx=0).stop 2024-03-09T09:09:22.424 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:09:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:09:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:09:22.424 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:09:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 122, 2024-03-09T09:09:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:09:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:09:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:09:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:09:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:09:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:09:22.425 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:09:22.426 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:09:22.427 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:09:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:09:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:09:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:09:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:09:22.428 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:09:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:09:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:09:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:09:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:09:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:09:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:09:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:22.429 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.430 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.431 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:22.432 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:22.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:09:22.434 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.435 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:22.436 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:09:22.437 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:09:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:09:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.438 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:09:22.438 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:09:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd478c00640 1 -- 172.21.15.67:0/2021976686 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd45006b400 msgr2=0x7fd45006d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd478c00640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd45006b400 0x7fd45006d8c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd46c034000 tx=0x7fd46c009030 comp rx=0 tx=0).stop 2024-03-09T09:09:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd478c00640 1 -- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd454004d40 msgr2=0x7fd4741eb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd478c00640 1 --2- 172.21.15.67:0/2021976686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd454004d40 0x7fd4741eb4f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd4640096c0 tx=0x7fd464009b80 comp rx=0 tx=0).stop 2024-03-09T09:09:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd472800640 1 -- 172.21.15.67:0/2021976686 reap_dead start 2024-03-09T09:09:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd472800640 1 -- 172.21.15.67:0/2021976686 reap_dead start 2024-03-09T09:09:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd478c00640 1 -- 172.21.15.67:0/2021976686 shutdown_connections 2024-03-09T09:09:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.419+0000 7fd478c00640 1 -- 172.21.15.67:0/2021976686 >> 172.21.15.67:0/2021976686 conn(0x7fd4740cb670 msgr2=0x7fd4741560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.420+0000 7fd478c00640 1 -- 172.21.15.67:0/2021976686 shutdown_connections 2024-03-09T09:09:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.420+0000 7fd478c00640 1 -- 172.21.15.67:0/2021976686 wait complete. 2024-03-09T09:09:22.441 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:09:22.441 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:09:22.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:09:22.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.662+0000 7f1fe3200640 1 Processor -- start 2024-03-09T09:09:22.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.663+0000 7f1fe3200640 1 -- start start 2024-03-09T09:09:22.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.664+0000 7f1fe3200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe40c9380 0x7f1fe40acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.664+0000 7f1fe3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe40c9890 con 0x7f1fe40ad440 2024-03-09T09:09:22.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.664+0000 7f1fe3200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe40c5f80 con 0x7f1fe414fd90 2024-03-09T09:09:22.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.664+0000 7f1fe3200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe406ba60 con 0x7f1fe40c9380 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.664+0000 7f1fe1e00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f1fe414fd90 legacy=0x7f1fe4156680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.664+0000 7f1fe2800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1fe40ad440 0x7f1fe406c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50256/0 (socket says 172.21.15.67:50256) 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe2800640 1 -- 172.21.15.67:0/3298053283 learned_addr learned my addr 172.21.15.67:0/3298053283 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe1400640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe40c9380 0x7f1fe40acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264478353 0 0) 0x7f1fe40c9890 con 0x7f1fe40ad440 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc8003520 con 0x7f1fe40ad440 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 712568583 0 0) 0x7f1fc8003520 con 0x7f1fe40ad440 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe40c9380 msgr2=0x7f1fe40acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe0a00640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe40c9380 0x7f1fe40acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 >> v1:172.21.15.134:6789/0 conn(0x7f1fe414fd90 legacy=0x7f1fe4156680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe40c6240 con 0x7f1fe40ad440 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f1fdc0019f0 con 0x7f1fe40ad440 2024-03-09T09:09:22.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 >> v1:172.21.15.67:6789/0 conn(0x7f1fe40ad440 legacy=0x7f1fe406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.665+0000 7f1fe0a00640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe40c9380 0x7f1fc8003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe0a00640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fc8003fd0 0x7f1fc8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe0a00640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fc8004d40 0x7f1fc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe406ba60 con 0x7f1fc8003fd0 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc800bc50 con 0x7f1fc8004d40 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc800be50 con 0x7f1fe40c9380 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe1400640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fc8003fd0 0x7f1fc8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe2800640 1 -- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fc8004d40 msgr2=0x7f1fc80051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe2800640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fc8004d40 0x7f1fc80051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe1e00640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe40c9380 0x7f1fc8003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe1400640 1 -- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe40c9380 msgr2=0x7f1fc8003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe1400640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe40c9380 0x7f1fc8003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe1400640 1 -- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fc8004d40 msgr2=0x7f1fc80051c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:22.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe1400640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fc8004d40 0x7f1fc80051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.666+0000 7f1fe1400640 1 -- 172.21.15.67:0/3298053283 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc8003520 con 0x7f1fc8003fd0 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.667+0000 7f1fe1400640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fc8003fd0 0x7f1fc8004800 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009a90 tx=0x7f1fc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf3086bb192814 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.668+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc039070 con 0x7f1fc8003fd0 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.668+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc02c680 con 0x7f1fc8003fd0 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.668+0000 7f1fe0a00640 1 -- 172.21.15.67:0/3298053283 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc009690 con 0x7f1fc8003fd0 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.668+0000 7f1fe3200640 1 -- 172.21.15.67:0/3298053283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fc8003fd0 msgr2=0x7f1fc8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.668+0000 7f1fe3200640 1 --2- 172.21.15.67:0/3298053283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fc8003fd0 0x7f1fc8004800 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009a90 tx=0x7f1fc8005830 comp rx=0 tx=0).stop 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.668+0000 7f1fe2800640 1 -- 172.21.15.67:0/3298053283 reap_dead start 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.668+0000 7f1fe3200640 1 -- 172.21.15.67:0/3298053283 shutdown_connections 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.668+0000 7f1fe3200640 1 -- 172.21.15.67:0/3298053283 >> 172.21.15.67:0/3298053283 conn(0x7f1fe40cb480 msgr2=0x7f1fe40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.669+0000 7f1fe3200640 1 -- 172.21.15.67:0/3298053283 shutdown_connections 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.669+0000 7f1fe3200640 1 -- 172.21.15.67:0/3298053283 wait complete. 2024-03-09T09:09:22.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.670+0000 7f1fe3200640 1 Processor -- start 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.670+0000 7f1fe3200640 1 -- start start 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.671+0000 7f1fe3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fc8003fd0 0x7f1fe4155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.671+0000 7f1fe3200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe4148c20 0x7f1fe4158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.671+0000 7f1fe3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fe41588c0 0x7f1fe4147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.671+0000 7f1fe3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40acf00 con 0x7f1fe41588c0 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.671+0000 7f1fe3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe406bcb0 con 0x7f1fc8003fd0 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.671+0000 7f1fe3200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40ad080 con 0x7f1fe4148c20 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.671+0000 7f1fe1e00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fc8003fd0 msgr2=0x7f1fe4155030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.672+0000 7f1fe2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fe41588c0 0x7f1fe4147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.672+0000 7f1fe1e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fc8003fd0 0x7f1fe4155030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.672+0000 7f1fe2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fe41588c0 0x7f1fe4147e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44244/0 (socket says 172.21.15.67:44244) 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.672+0000 7f1fe2800640 1 -- 172.21.15.67:0/4164244793 learned_addr learned my addr 172.21.15.67:0/4164244793 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.672+0000 7f1fe1400640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe4148c20 0x7f1fe4158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.672+0000 7f1fe2800640 1 -- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe4148c20 msgr2=0x7f1fe4158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.672+0000 7f1fe2800640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe4148c20 0x7f1fe4158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.672+0000 7f1fe2800640 1 -- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fc8003fd0 msgr2=0x7f1fe4155030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:09:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.672+0000 7f1fe2800640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fc8003fd0 0x7f1fe4155030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.672+0000 7f1fe2800640 1 -- 172.21.15.67:0/4164244793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe41df790 con 0x7f1fe41588c0 2024-03-09T09:09:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.672+0000 7f1fe2800640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fe41588c0 0x7f1fe4147e60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc001460 tx=0x7f1fdc00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.673+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00a610 con 0x7f1fe41588c0 2024-03-09T09:09:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.673+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc00a7d0 con 0x7f1fe41588c0 2024-03-09T09:09:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.673+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00abc0 con 0x7f1fe41588c0 2024-03-09T09:09:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.673+0000 7f1fe3200640 1 -- 172.21.15.67:0/4164244793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe41df4b0 con 0x7f1fe41588c0 2024-03-09T09:09:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.673+0000 7f1fe3200640 1 -- 172.21.15.67:0/4164244793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe4154830 con 0x7f1fe41588c0 2024-03-09T09:09:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.674+0000 7f1fe3200640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe4154a70 0x7f1fe41483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.674+0000 7f1fe3200640 1 -- 172.21.15.67:0/4164244793 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe41df260 con 0x7f1fe4154a70 2024-03-09T09:09:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.675+0000 7f1fe1e00640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe4154a70 0x7f1fe41483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.675+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1fdc011070 con 0x7f1fe41588c0 2024-03-09T09:09:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.675+0000 7f1fe1e00640 1 -- 172.21.15.67:0/4164244793 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1fe404f3c0 con 0x7f1fe4154a70 2024-03-09T09:09:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.676+0000 7f1fe1e00640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe4154a70 0x7f1fe41483a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4003930 tx=0x7f1fd400d8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.676+0000 7f1fda000640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1fc006b480 0x7f1fc006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.676+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1fdc089ba0 con 0x7f1fe41588c0 2024-03-09T09:09:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.677+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4003bc0 con 0x7f1fe4154a70 2024-03-09T09:09:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.677+0000 7f1fe1400640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1fc006b480 0x7f1fc006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.677+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f1fd4017070 con 0x7f1fe4154a70 2024-03-09T09:09:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.677+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe4154a70 msgr2=0x7f1fe41483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.677+0000 7f1fda000640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe4154a70 0x7f1fe41483a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4003930 tx=0x7f1fd400d8b0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.677+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe4154a70 msgr2=0x7f1fe41483a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:22.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.677+0000 7f1fda000640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe4154a70 0x7f1fe41483a0 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4003930 tx=0x7f1fd400d8b0 comp rx=0 tx=0).stop 2024-03-09T09:09:22.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.678+0000 7f1fe1400640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1fc006b480 0x7f1fc006d940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc036000 tx=0x7f1fcc030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:22.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.684+0000 7f1fe3200640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe41ef070 0x7f1fe41ef450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:22.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.684+0000 7f1fe3200640 1 -- 172.21.15.67:0/4164244793 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40c6980 con 0x7f1fe41ef070 2024-03-09T09:09:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.685+0000 7f1fe1e00640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe41ef070 0x7f1fe41ef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.686+0000 7f1fe1e00640 1 -- 172.21.15.67:0/4164244793 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1fe406d2a0 con 0x7f1fe41ef070 2024-03-09T09:09:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.686+0000 7f1fe1e00640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe41ef070 0x7f1fe41ef450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4003930 tx=0x7f1fd4005a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.686+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd40052e0 con 0x7f1fe41ef070 2024-03-09T09:09:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4523 (secure 0 0 0) 0x7f1fd40052e0 con 0x7f1fe41ef070 2024-03-09T09:09:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe41ef070 msgr2=0x7f1fe41ef450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fda000640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe41ef070 0x7f1fe41ef450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4003930 tx=0x7f1fd4005a80 comp rx=0 tx=0).stop 2024-03-09T09:09:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fda000640 1 -- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe41ef070 msgr2=0x7f1fe41ef450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:22.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fda000640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fe41ef070 0x7f1fe41ef450 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4003930 tx=0x7f1fd4005a80 comp rx=0 tx=0).stop 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 122, 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:22.696 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:22.697 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:22.698 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:09:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:09:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:09:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:09:22.699 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:09:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:09:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:09:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:09:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:09:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:09:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:09:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:09:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:09:22.700 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:22.701 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.703 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.704 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.705 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:09:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:09:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.706 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:09:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:09:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:09:22.707 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:22.708 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:22.709 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:09:22.709 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:09:22.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fe3200640 1 -- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1fc006b480 msgr2=0x7f1fc006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fe3200640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1fc006b480 0x7f1fc006d940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc036000 tx=0x7f1fcc030000 comp rx=0 tx=0).stop 2024-03-09T09:09:22.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fe3200640 1 -- 172.21.15.67:0/4164244793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fe41588c0 msgr2=0x7f1fe4147e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:22.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fe3200640 1 --2- 172.21.15.67:0/4164244793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fe41588c0 0x7f1fe4147e60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc001460 tx=0x7f1fdc00cf00 comp rx=0 tx=0).stop 2024-03-09T09:09:22.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fe2800640 1 -- 172.21.15.67:0/4164244793 reap_dead start 2024-03-09T09:09:22.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fe2800640 1 -- 172.21.15.67:0/4164244793 reap_dead start 2024-03-09T09:09:22.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fe3200640 1 -- 172.21.15.67:0/4164244793 shutdown_connections 2024-03-09T09:09:22.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.687+0000 7f1fe3200640 1 -- 172.21.15.67:0/4164244793 >> 172.21.15.67:0/4164244793 conn(0x7f1fe40cb480 msgr2=0x7f1fe414c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:22.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.688+0000 7f1fe3200640 1 -- 172.21.15.67:0/4164244793 shutdown_connections 2024-03-09T09:09:22.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:22.688+0000 7f1fe3200640 1 -- 172.21.15.67:0/4164244793 wait complete. 2024-03-09T09:09:22.713 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:09:22.713 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:09:29.789 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 2% (5/181) Updating files: 3% (6/181) Updating files: 4% (8/181) Updating files: 5% (10/181) Updating files: 6% (11/181) Updating files: 6% (12/181) Updating files: 7% (13/181) Updating files: 8% (15/181) Updating files: 9% (17/181) Updating files: 10% (19/181) Updating files: 11% (20/181) Updating files: 12% (22/181) Updating files: 13% (24/181) Updating files: 14% (26/181) Updating files: 15% (28/181) Updating files: 16% (29/181) Updating files: 17% (31/181) Updating files: 18% (33/181) Updating files: 19% (35/181) Updating files: 20% (37/181) Updating files: 21% (39/181) Updating files: 22% (40/181) Updating files: 23% (42/181) Updating files: 24% (44/181) Updating files: 25% (46/181) Updating files: 26% (48/181) Updating files: 27% (49/181) Updating files: 28% (51/181) Updating files: 29% (53/181) Updating files: 30% (55/181) Updating files: 31% (57/181) Updating files: 32% (58/181) Updating files: 33% (60/181) Updating files: 34% (62/181) Updating files: 34% (63/181) Updating files: 35% (64/181) Updating files: 36% (66/181) Updating files: 37% (67/181) Updating files: 38% (69/181) Updating files: 39% (71/181) Updating files: 40% (73/181) Updating files: 41% (75/181) Updating files: 42% (77/181) Updating files: 43% (78/181) Updating files: 44% (80/181) Updating files: 45% (82/181) Updating files: 46% (84/181) Updating files: 47% (86/181) Updating files: 48% (87/181) Updating files: 49% (89/181) Updating files: 50% (91/181) Updating files: 51% (93/181) Updating files: 52% (95/181) Updating files: 53% (96/181) Updating files: 54% (98/181) Updating files: 55% (100/181) Updating files: 56% (102/181) Updating files: 57% (104/181) Updating files: 58% (105/181) Updating files: 59% (107/181) Updating files: 60% (109/181) Updating files: 61% (111/181) Updating files: 62% (113/181) Updating files: 63% (115/181) Updating files: 64% (116/181) Updating files: 65% (118/181) Updating files: 65% (119/181) Updating files: 66% (120/181) Updating files: 67% (122/181) Updating files: 68% (124/181) Updating files: 69% (125/181) Updating files: 70% (127/181) Updating files: 71% (129/181) Updating files: 72% (131/181) Updating files: 73% (133/181) Updating files: 74% (134/181) Updating files: 75% (136/181) Updating files: 76% (138/181) Updating files: 77% (140/181) Updating files: 78% (142/181) Updating files: 79% (143/181) Updating files: 80% (145/181) Updating files: 81% (147/181) Updating files: 82% (149/181) Updating files: 83% (151/181) Updating files: 84% (153/181) Updating files: 85% (154/181) Updating files: 86% (156/181) Updating files: 87% (158/181) Updating files: 88% (160/181) Updating files: 89% (162/181) Updating files: 90% (163/181) Updating files: 91% (165/181) Updating files: 92% (167/181) Updating files: 93% (169/181) Updating files: 94% (171/181) Updating files: 95% (172/181) Updating files: 96% (174/181) Updating files: 97% (176/181) Updating files: 97% (177/181) Updating files: 98% (178/181) Updating files: 99% (180/181) Updating files: 100% (181/181) Updating files: 100% (181/181), done. 2024-03-09T09:09:29.860 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at a6f3bbb744a v0.37 2024-03-09T09:09:29.868 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.37 2024-03-09T09:09:30.842 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:09:30.842 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 38 -eq 48 ']' 2024-03-09T09:09:30.842 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.38 2024-03-09T09:09:30.842 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.38 2024-03-09T09:09:30.842 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.38 2024-03-09T09:09:30.843 INFO:tasks.workunit.client.1.smithi134.stdout:v0.38 2024-03-09T09:09:32.598 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 26% (90/334) Updating files: 27% (91/334) Updating files: 28% (94/334) Updating files: 29% (97/334) Updating files: 30% (101/334) Updating files: 31% (104/334) Updating files: 32% (107/334) Updating files: 33% (111/334) Updating files: 34% (114/334) Updating files: 35% (117/334) Updating files: 36% (121/334) Updating files: 37% (124/334) Updating files: 38% (127/334) Updating files: 39% (131/334) Updating files: 40% (134/334) Updating files: 41% (137/334) Updating files: 42% (141/334) Updating files: 43% (144/334) Updating files: 44% (147/334) Updating files: 45% (151/334) Updating files: 46% (154/334) Updating files: 47% (157/334) Updating files: 48% (161/334) Updating files: 49% (164/334) Updating files: 50% (167/334) Updating files: 51% (171/334) Updating files: 52% (174/334) Updating files: 52% (175/334) Updating files: 53% (178/334) Updating files: 54% (181/334) Updating files: 55% (184/334) Updating files: 56% (188/334) Updating files: 57% (191/334) Updating files: 58% (194/334) Updating files: 59% (198/334) Updating files: 60% (201/334) Updating files: 60% (202/334) Updating files: 61% (204/334) Updating files: 62% (208/334) Updating files: 63% (211/334) Updating files: 64% (214/334) Updating files: 65% (218/334) Updating files: 66% (221/334) Updating files: 67% (224/334) Updating files: 68% (228/334) Updating files: 68% (230/334) Updating files: 69% (231/334) Updating files: 70% (234/334) Updating files: 71% (238/334) Updating files: 72% (241/334) Updating files: 73% (244/334) Updating files: 74% (248/334) Updating files: 75% (251/334) Updating files: 76% (254/334) Updating files: 77% (258/334) Updating files: 78% (261/334) Updating files: 79% (264/334) Updating files: 80% (268/334) Updating files: 81% (271/334) Updating files: 82% (274/334) Updating files: 83% (278/334) Updating files: 84% (281/334) Updating files: 85% (284/334) Updating files: 86% (288/334) Updating files: 87% (291/334) Updating files: 88% (294/334) Updating files: 89% (298/334) Updating files: 90% (301/334) Updating files: 91% (304/334) Updating files: 92% (308/334) Updating files: 93% (311/334) Updating files: 94% (314/334) Updating files: 95% (318/334) Updating files: 96% (321/334) Updating files: 97% (324/334) Updating files: 98% (328/334) Updating files: 99% (331/334) Updating files: 100% (334/334) Updating files: 100% (334/334), done. 2024-03-09T09:09:33.071 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 0afda378124 v0.35 2024-03-09T09:09:33.090 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.35 2024-03-09T09:09:33.514 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:09:33.514 INFO:tasks.workunit.client.0.smithi067.stdout:v0.36 2024-03-09T09:09:33.515 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 36 -eq 48 ']' 2024-03-09T09:09:33.515 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.36 2024-03-09T09:09:33.515 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.36 2024-03-09T09:09:33.515 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.36 2024-03-09T09:09:36.985 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 11% (29/244) Updating files: 12% (30/244) Updating files: 13% (32/244) Updating files: 14% (35/244) Updating files: 15% (37/244) Updating files: 16% (40/244) Updating files: 17% (42/244) Updating files: 18% (44/244) Updating files: 19% (47/244) Updating files: 20% (49/244) Updating files: 21% (52/244) Updating files: 22% (54/244) Updating files: 23% (57/244) Updating files: 24% (59/244) Updating files: 25% (61/244) Updating files: 26% (64/244) Updating files: 27% (66/244) Updating files: 28% (69/244) Updating files: 29% (71/244) Updating files: 30% (74/244) Updating files: 31% (76/244) Updating files: 32% (79/244) Updating files: 33% (81/244) Updating files: 34% (83/244) Updating files: 35% (86/244) Updating files: 36% (88/244) Updating files: 37% (91/244) Updating files: 38% (93/244) Updating files: 39% (96/244) Updating files: 40% (98/244) Updating files: 41% (101/244) Updating files: 42% (103/244) Updating files: 43% (105/244) Updating files: 44% (108/244) Updating files: 45% (110/244) Updating files: 46% (113/244) Updating files: 47% (115/244) Updating files: 48% (118/244) Updating files: 49% (120/244) Updating files: 50% (122/244) Updating files: 51% (125/244) Updating files: 52% (127/244) Updating files: 53% (130/244) Updating files: 54% (132/244) Updating files: 55% (135/244) Updating files: 56% (137/244) Updating files: 57% (140/244) Updating files: 58% (142/244) Updating files: 59% (144/244) Updating files: 60% (147/244) Updating files: 61% (149/244) Updating files: 62% (152/244) Updating files: 63% (154/244) Updating files: 64% (157/244) Updating files: 65% (159/244) Updating files: 66% (162/244) Updating files: 67% (164/244) Updating files: 68% (166/244) Updating files: 69% (169/244) Updating files: 70% (171/244) Updating files: 71% (174/244) Updating files: 72% (176/244) Updating files: 73% (179/244) Updating files: 74% (181/244) Updating files: 75% (183/244) Updating files: 76% (186/244) Updating files: 77% (188/244) Updating files: 77% (190/244) Updating files: 78% (191/244) Updating files: 79% (193/244) Updating files: 80% (196/244) Updating files: 81% (198/244) Updating files: 82% (201/244) Updating files: 83% (203/244) Updating files: 84% (205/244) Updating files: 85% (208/244) Updating files: 86% (210/244) Updating files: 87% (213/244) Updating files: 88% (215/244) Updating files: 89% (218/244) Updating files: 90% (220/244) Updating files: 91% (223/244) Updating files: 91% (224/244) Updating files: 92% (225/244) Updating files: 93% (227/244) Updating files: 94% (230/244) Updating files: 95% (232/244) Updating files: 96% (235/244) Updating files: 97% (237/244) Updating files: 98% (240/244) Updating files: 99% (242/244) Updating files: 100% (244/244) Updating files: 100% (244/244), done. 2024-03-09T09:09:37.058 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 877cacbbdbb v0.36 2024-03-09T09:09:37.072 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.36 2024-03-09T09:09:37.635 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:09:37.635 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 37 -eq 48 ']' 2024-03-09T09:09:37.636 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.37 2024-03-09T09:09:37.636 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.37 2024-03-09T09:09:37.636 INFO:tasks.workunit.client.0.smithi067.stdout:v0.37 2024-03-09T09:09:37.636 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.37 2024-03-09T09:09:40.078 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 35% (65/181) Updating files: 36% (66/181) Updating files: 37% (67/181) Updating files: 38% (69/181) Updating files: 39% (71/181) Updating files: 40% (73/181) Updating files: 41% (75/181) Updating files: 42% (77/181) Updating files: 43% (78/181) Updating files: 44% (80/181) Updating files: 45% (82/181) Updating files: 46% (84/181) Updating files: 47% (86/181) Updating files: 48% (87/181) Updating files: 49% (89/181) Updating files: 50% (91/181) Updating files: 51% (93/181) Updating files: 52% (95/181) Updating files: 53% (96/181) Updating files: 54% (98/181) Updating files: 55% (100/181) Updating files: 56% (102/181) Updating files: 57% (104/181) Updating files: 58% (105/181) Updating files: 59% (107/181) Updating files: 60% (109/181) Updating files: 61% (111/181) Updating files: 62% (113/181) Updating files: 63% (115/181) Updating files: 64% (116/181) Updating files: 64% (117/181) Updating files: 65% (118/181) Updating files: 66% (120/181) Updating files: 67% (122/181) Updating files: 68% (124/181) Updating files: 69% (125/181) Updating files: 70% (127/181) Updating files: 71% (129/181) Updating files: 72% (131/181) Updating files: 73% (133/181) Updating files: 74% (134/181) Updating files: 75% (136/181) Updating files: 76% (138/181) Updating files: 77% (140/181) Updating files: 78% (142/181) Updating files: 79% (143/181) Updating files: 80% (145/181) Updating files: 81% (147/181) Updating files: 82% (149/181) Updating files: 83% (151/181) Updating files: 84% (153/181) Updating files: 85% (154/181) Updating files: 86% (156/181) Updating files: 87% (158/181) Updating files: 88% (160/181) Updating files: 89% (162/181) Updating files: 90% (163/181) Updating files: 91% (165/181) Updating files: 92% (167/181) Updating files: 93% (169/181) Updating files: 94% (171/181) Updating files: 95% (172/181) Updating files: 96% (174/181) Updating files: 97% (176/181) Updating files: 98% (178/181) Updating files: 99% (180/181) Updating files: 100% (181/181) Updating files: 100% (181/181), done. 2024-03-09T09:09:40.152 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at a6f3bbb744a v0.37 2024-03-09T09:09:40.160 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.37 2024-03-09T09:09:40.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:09:40.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 38 -eq 48 ']' 2024-03-09T09:09:40.915 INFO:tasks.workunit.client.0.smithi067.stdout:v0.38 2024-03-09T09:09:40.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.38 2024-03-09T09:09:40.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.38 2024-03-09T09:09:40.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.38 2024-03-09T09:09:42.713 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:09:42.713 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-03-09T09:09:42.713 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-09T09:09:42.714 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-09T09:09:42.722 INFO:tasks.ceph.mon.b:Started 2024-03-09T09:09:42.722 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:09:42.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:09:42.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.945+0000 7f486e200640 1 Processor -- start 2024-03-09T09:09:42.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.945+0000 7f486e200640 1 -- start start 2024-03-09T09:09:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.946+0000 7f486e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48680c1f70 0x7f4868150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f486e200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4868154080 0x7f4868151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f486e200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48680c81e0 con 0x7f48680710a0 2024-03-09T09:09:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f486e200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48680c32b0 con 0x7f4868154080 2024-03-09T09:09:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f486e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48680c2fd0 con 0x7f48680c1f70 2024-03-09T09:09:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f4867e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f48680710a0 0x7f4868153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45440/0 (socket says 172.21.15.67:45440) 2024-03-09T09:09:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f4867e00640 1 -- 172.21.15.67:0/157439747 learned_addr learned my addr 172.21.15.67:0/157439747 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f486ce00640 1 --2- 172.21.15.67:0/157439747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48680c1f70 0x7f4868150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f486d800640 1 --2- 172.21.15.67:0/157439747 >> v2:172.21.15.134:3300/0 conn(0x7f4868154080 0x7f4868151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f4867400640 1 -- 172.21.15.67:0/157439747 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510139617 0 0) 0x7f48680c81e0 con 0x7f48680710a0 2024-03-09T09:09:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f486d800640 1 -- 172.21.15.67:0/157439747 >> v2:172.21.15.134:3300/0 conn(0x7f4868154080 msgr2=0x7f4868151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:09:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f486d800640 1 -- 172.21.15.67:0/157439747 >> v2:172.21.15.134:3300/0 conn(0x7f4868154080 msgr2=0x7f4868151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:09:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f4867400640 1 -- 172.21.15.67:0/157439747 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4850003520 con 0x7f48680710a0 2024-03-09T09:09:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.947+0000 7f486d800640 1 --2- 172.21.15.67:0/157439747 >> v2:172.21.15.134:3300/0 conn(0x7f4868154080 0x7f4868151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:09:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.948+0000 7f486d800640 1 --2- 172.21.15.67:0/157439747 >> v2:172.21.15.134:3300/0 conn(0x7f4868154080 0x7f4868151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.948+0000 7f486ce00640 1 -- 172.21.15.67:0/157439747 >> v2:172.21.15.134:3300/0 conn(0x7f4868154080 msgr2=0x7f4868151190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:09:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.948+0000 7f486ce00640 1 --2- 172.21.15.67:0/157439747 >> v2:172.21.15.134:3300/0 conn(0x7f4868154080 0x7f4868151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.948+0000 7f486ce00640 1 -- 172.21.15.67:0/157439747 >> v1:172.21.15.67:6789/0 conn(0x7f48680710a0 legacy=0x7f4868153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.948+0000 7f486ce00640 1 -- 172.21.15.67:0/157439747 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48680c3700 con 0x7f48680c1f70 2024-03-09T09:09:42.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.949+0000 7f486ce00640 1 --2- 172.21.15.67:0/157439747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48680c1f70 0x7f4868150c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f485c0097a0 tx=0x7f485c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5003050e2c76f8b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:42.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.949+0000 7f4867400640 1 -- 172.21.15.67:0/157439747 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c0315a0 con 0x7f48680c1f70 2024-03-09T09:09:42.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.949+0000 7f4867400640 1 -- 172.21.15.67:0/157439747 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485c037070 con 0x7f48680c1f70 2024-03-09T09:09:42.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.949+0000 7f4867400640 1 -- 172.21.15.67:0/157439747 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c031940 con 0x7f48680c1f70 2024-03-09T09:09:42.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.950+0000 7f486e200640 1 -- 172.21.15.67:0/157439747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48680c1f70 msgr2=0x7f4868150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:42.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.950+0000 7f486e200640 1 --2- 172.21.15.67:0/157439747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48680c1f70 0x7f4868150c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f485c0097a0 tx=0x7f485c02b840 comp rx=0 tx=0).stop 2024-03-09T09:09:42.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.950+0000 7f486e200640 1 -- 172.21.15.67:0/157439747 shutdown_connections 2024-03-09T09:09:42.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.950+0000 7f486e200640 1 --2- 172.21.15.67:0/157439747 >> v2:172.21.15.134:3300/0 conn(0x7f4868154080 0x7f4868151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:42.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.950+0000 7f486e200640 1 --2- 172.21.15.67:0/157439747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48680c1f70 0x7f4868150c50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:42.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.950+0000 7f486e200640 1 -- 172.21.15.67:0/157439747 >> 172.21.15.67:0/157439747 conn(0x7f48680c7300 msgr2=0x7f48680c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:42.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.950+0000 7f486e200640 1 -- 172.21.15.67:0/157439747 shutdown_connections 2024-03-09T09:09:42.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.950+0000 7f486e200640 1 -- 172.21.15.67:0/157439747 wait complete. 2024-03-09T09:09:42.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.952+0000 7f486e200640 1 Processor -- start 2024-03-09T09:09:42.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.952+0000 7f486e200640 1 -- start start 2024-03-09T09:09:42.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.953+0000 7f486e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48680710a0 0x7f486814f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:42.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.953+0000 7f486e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48680c1f70 0x7f48680c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:42.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.953+0000 7f486e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4868154080 0x7f48680c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:42.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.953+0000 7f486e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4868070d50 con 0x7f4868154080 2024-03-09T09:09:42.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.953+0000 7f486e200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4868070830 con 0x7f48680710a0 2024-03-09T09:09:42.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.953+0000 7f486e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4868070ed0 con 0x7f48680c1f70 2024-03-09T09:09:42.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.953+0000 7f486d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4868154080 0x7f48680c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:42.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.953+0000 7f486d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4868154080 0x7f48680c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33860/0 (socket says 172.21.15.67:33860) 2024-03-09T09:09:42.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.953+0000 7f486d800640 1 -- 172.21.15.67:0/2677974113 learned_addr learned my addr 172.21.15.67:0/2677974113 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:42.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.953+0000 7f4867e00640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48680c1f70 0x7f48680c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:42.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.953+0000 7f486ce00640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48680710a0 0x7f486814f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:42.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.954+0000 7f486d800640 1 -- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48680c1f70 msgr2=0x7f48680c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:42.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.954+0000 7f486d800640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48680c1f70 0x7f48680c4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.954+0000 7f486ce00640 1 -- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48680710a0 msgr2=0x7f486814f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.954+0000 7f486ce00640 1 -- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48680710a0 msgr2=0x7f486814f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.954+0000 7f486ce00640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48680710a0 0x7f486814f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.954+0000 7f486d800640 1 -- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48680710a0 msgr2=0x7f486814f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.954+0000 7f486ce00640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48680710a0 0x7f486814f9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.954+0000 7f486d800640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48680710a0 0x7f486814f9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.954+0000 7f486d800640 1 -- 172.21.15.67:0/2677974113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48680c65c0 con 0x7f4868154080 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.954+0000 7f486d800640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4868154080 0x7f48680c5430 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4860007e60 tx=0x7f486000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.954+0000 7f4864c00640 1 -- 172.21.15.67:0/2677974113 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860014620 con 0x7f4868154080 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.955+0000 7f4864c00640 1 -- 172.21.15.67:0/2677974113 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4860019070 con 0x7f4868154080 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.955+0000 7f4864c00640 1 -- 172.21.15.67:0/2677974113 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860014a00 con 0x7f4868154080 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.955+0000 7f486e200640 1 -- 172.21.15.67:0/2677974113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f486814c3e0 con 0x7f4868154080 2024-03-09T09:09:42.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.955+0000 7f486e200640 1 -- 172.21.15.67:0/2677974113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f486814fee0 con 0x7f4868154080 2024-03-09T09:09:42.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.956+0000 7f486e200640 1 -- 172.21.15.67:0/2677974113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486804f3c0 con 0x7f4868154080 2024-03-09T09:09:42.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.957+0000 7f4864c00640 1 -- 172.21.15.67:0/2677974113 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4860014ba0 con 0x7f4868154080 2024-03-09T09:09:42.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.958+0000 7f4864c00640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484806b4d0 0x7f484806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:42.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.958+0000 7f4864c00640 1 -- 172.21.15.67:0/2677974113 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f486008ade0 con 0x7f4868154080 2024-03-09T09:09:42.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.959+0000 7f486ce00640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484806b4d0 0x7f484806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:42.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.960+0000 7f486ce00640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484806b4d0 0x7f484806d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f485c036000 tx=0x7f485c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:42.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:42.963+0000 7f4864c00640 1 -- 172.21.15.67:0/2677974113 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4860059860 con 0x7f4868154080 2024-03-09T09:09:43.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.324+0000 7f486e200640 1 -- 172.21.15.67:0/2677974113 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f48681ebe40 con 0x7f4868154080 2024-03-09T09:09:43.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.324+0000 7f4864c00640 1 -- 172.21.15.67:0/2677974113 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f4860059160 con 0x7f4868154080 2024-03-09T09:09:43.329 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":122,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:09:43.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 -- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484806b4d0 msgr2=0x7f484806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:43.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484806b4d0 0x7f484806d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f485c036000 tx=0x7f485c008030 comp rx=0 tx=0).stop 2024-03-09T09:09:43.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 -- 172.21.15.67:0/2677974113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4868154080 msgr2=0x7f48680c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:43.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4868154080 0x7f48680c5430 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4860007e60 tx=0x7f486000e4b0 comp rx=0 tx=0).stop 2024-03-09T09:09:43.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 -- 172.21.15.67:0/2677974113 shutdown_connections 2024-03-09T09:09:43.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f484806b4d0 0x7f484806d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:43.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4868154080 0x7f48680c5430 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:43.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f48680c1f70 0x7f48680c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:43.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 --2- 172.21.15.67:0/2677974113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48680710a0 0x7f486814f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:43.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 -- 172.21.15.67:0/2677974113 >> 172.21.15.67:0/2677974113 conn(0x7f48680c7300 msgr2=0x7f48680ac080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:43.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 -- 172.21.15.67:0/2677974113 shutdown_connections 2024-03-09T09:09:43.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:43.327+0000 7f486e200640 1 -- 172.21.15.67:0/2677974113 wait complete. 2024-03-09T09:09:44.757 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 54% (93/171) Updating files: 55% (95/171) Updating files: 56% (96/171) Updating files: 57% (98/171) Updating files: 58% (100/171) Updating files: 59% (101/171) Updating files: 60% (103/171) Updating files: 61% (105/171) Updating files: 62% (107/171) Updating files: 63% (108/171) Updating files: 64% (110/171) Updating files: 65% (112/171) Updating files: 66% (113/171) Updating files: 67% (115/171) Updating files: 68% (117/171) Updating files: 69% (118/171) Updating files: 70% (120/171) Updating files: 71% (122/171) Updating files: 71% (123/171) Updating files: 72% (124/171) Updating files: 73% (125/171) Updating files: 74% (127/171) Updating files: 75% (129/171) Updating files: 76% (130/171) Updating files: 77% (132/171) Updating files: 78% (134/171) Updating files: 79% (136/171) Updating files: 80% (137/171) Updating files: 81% (139/171) Updating files: 82% (141/171) Updating files: 83% (142/171) Updating files: 84% (144/171) Updating files: 85% (146/171) Updating files: 86% (148/171) Updating files: 87% (149/171) Updating files: 88% (151/171) Updating files: 89% (153/171) Updating files: 90% (154/171) Updating files: 91% (156/171) Updating files: 92% (158/171) Updating files: 93% (160/171) Updating files: 94% (161/171) Updating files: 95% (163/171) Updating files: 96% (165/171) Updating files: 97% (166/171) Updating files: 98% (168/171) Updating files: 99% (170/171) Updating files: 100% (171/171) Updating files: 100% (171/171), done. 2024-03-09T09:09:44.982 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at b600ec2ac7c v0.38 2024-03-09T09:09:44.987 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.38 2024-03-09T09:09:45.055 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:09:45.055 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 39 -eq 48 ']' 2024-03-09T09:09:45.056 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.39 2024-03-09T09:09:45.056 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.39 2024-03-09T09:09:45.056 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.39 2024-03-09T09:09:45.056 INFO:tasks.workunit.client.1.smithi134.stdout:v0.39 2024-03-09T09:09:46.363 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:09:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.585+0000 7fbaddc00640 1 Processor -- start 2024-03-09T09:09:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.586+0000 7fbaddc00640 1 -- start start 2024-03-09T09:09:46.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.587+0000 7fbaddc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbad8153300 0x7fbad80b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:46.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.587+0000 7fbaddc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbad80cadb0 0x7fbad80b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:46.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.587+0000 7fbaddc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad8157000 con 0x7fbad80c5e40 2024-03-09T09:09:46.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.587+0000 7fbaddc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad80ae430 con 0x7fbad80cadb0 2024-03-09T09:09:46.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.587+0000 7fbaddc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80ae6f0 con 0x7fbad8153300 2024-03-09T09:09:46.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.587+0000 7fbad7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbad80c5e40 0x7fbad8156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45450/0 (socket says 172.21.15.67:45450) 2024-03-09T09:09:46.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.587+0000 7fbad7400640 1 -- 172.21.15.67:0/1231374796 learned_addr learned my addr 172.21.15.67:0/1231374796 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:46.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.587+0000 7fbadd200640 1 --2- 172.21.15.67:0/1231374796 >> v2:172.21.15.67:3300/0 conn(0x7fbad80cadb0 0x7fbad80b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:46.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.588+0000 7fbad7e00640 1 --2- 172.21.15.67:0/1231374796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbad8153300 0x7fbad80b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:46.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.588+0000 7fbadd200640 1 -- 172.21.15.67:0/1231374796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbad8153300 msgr2=0x7fbad80b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:46.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.588+0000 7fbadd200640 1 --2- 172.21.15.67:0/1231374796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbad8153300 0x7fbad80b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:46.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.588+0000 7fbad6a00640 1 -- 172.21.15.67:0/1231374796 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382967978 0 0) 0x7fbad8157000 con 0x7fbad80c5e40 2024-03-09T09:09:46.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.588+0000 7fbadd200640 1 -- 172.21.15.67:0/1231374796 >> v1:172.21.15.67:6789/0 conn(0x7fbad80c5e40 legacy=0x7fbad8156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:46.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.588+0000 7fbadd200640 1 -- 172.21.15.67:0/1231374796 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad8155b10 con 0x7fbad80cadb0 2024-03-09T09:09:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.589+0000 7fbadd200640 1 --2- 172.21.15.67:0/1231374796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbad80cadb0 0x7fbad80b0540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbad00097a0 tx=0x7fbad002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8f481a3d528243 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.590+0000 7fbad6a00640 1 -- 172.21.15.67:0/1231374796 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad0035020 con 0x7fbad80cadb0 2024-03-09T09:09:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.590+0000 7fbad6a00640 1 -- 172.21.15.67:0/1231374796 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad0031830 con 0x7fbad80cadb0 2024-03-09T09:09:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.590+0000 7fbad6a00640 1 -- 172.21.15.67:0/1231374796 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad0031c00 con 0x7fbad80cadb0 2024-03-09T09:09:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.590+0000 7fbaddc00640 1 -- 172.21.15.67:0/1231374796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbad80cadb0 msgr2=0x7fbad80b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.590+0000 7fbaddc00640 1 --2- 172.21.15.67:0/1231374796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbad80cadb0 0x7fbad80b0540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbad00097a0 tx=0x7fbad002b8d0 comp rx=0 tx=0).stop 2024-03-09T09:09:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.590+0000 7fbaddc00640 1 -- 172.21.15.67:0/1231374796 shutdown_connections 2024-03-09T09:09:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.590+0000 7fbaddc00640 1 --2- 172.21.15.67:0/1231374796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbad80cadb0 0x7fbad80b0540 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.590+0000 7fbaddc00640 1 --2- 172.21.15.67:0/1231374796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbad8153300 0x7fbad80b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.590+0000 7fbaddc00640 1 -- 172.21.15.67:0/1231374796 >> 172.21.15.67:0/1231374796 conn(0x7fbad80732d0 msgr2=0x7fbad80ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.591+0000 7fbaddc00640 1 -- 172.21.15.67:0/1231374796 shutdown_connections 2024-03-09T09:09:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.591+0000 7fbaddc00640 1 -- 172.21.15.67:0/1231374796 wait complete. 2024-03-09T09:09:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.593+0000 7fbaddc00640 1 Processor -- start 2024-03-09T09:09:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.593+0000 7fbaddc00640 1 -- start start 2024-03-09T09:09:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.594+0000 7fbaddc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbad80c5e40 0x7fbad81df450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.594+0000 7fbaddc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad80cadb0 0x7fbad81e0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.594+0000 7fbaddc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbad8153300 0x7fbad81eea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.594+0000 7fbaddc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80c59e0 con 0x7fbad80c5e40 2024-03-09T09:09:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.594+0000 7fbaddc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8159e90 con 0x7fbad80cadb0 2024-03-09T09:09:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.594+0000 7fbaddc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8156720 con 0x7fbad8153300 2024-03-09T09:09:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.594+0000 7fbad7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbad80c5e40 0x7fbad81df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.595+0000 7fbad7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbad80c5e40 0x7fbad81df450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33870/0 (socket says 172.21.15.67:33870) 2024-03-09T09:09:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.595+0000 7fbad7e00640 1 -- 172.21.15.67:0/3691133538 learned_addr learned my addr 172.21.15.67:0/3691133538 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.595+0000 7fbad7e00640 1 -- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbad8153300 msgr2=0x7fbad81eea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.595+0000 7fbad7400640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad80cadb0 0x7fbad81e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.595+0000 7fbad7e00640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbad8153300 0x7fbad81eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.595+0000 7fbad7e00640 1 -- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad80cadb0 msgr2=0x7fbad81e0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.595+0000 7fbad7e00640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad80cadb0 0x7fbad81e0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.595+0000 7fbad7e00640 1 -- 172.21.15.67:0/3691133538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad81ef2a0 con 0x7fbad80c5e40 2024-03-09T09:09:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.595+0000 7fbad7e00640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbad80c5e40 0x7fbad81df450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbac8007e70 tx=0x7fbac800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.596+0000 7fbacfe00640 1 -- 172.21.15.67:0/3691133538 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8014620 con 0x7fbad80c5e40 2024-03-09T09:09:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.596+0000 7fbacfe00640 1 -- 172.21.15.67:0/3691133538 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac8019070 con 0x7fbad80c5e40 2024-03-09T09:09:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.596+0000 7fbaddc00640 1 -- 172.21.15.67:0/3691133538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad81e0850 con 0x7fbad80c5e40 2024-03-09T09:09:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.596+0000 7fbacfe00640 1 -- 172.21.15.67:0/3691133538 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8014a00 con 0x7fbad80c5e40 2024-03-09T09:09:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.596+0000 7fbaddc00640 1 -- 172.21.15.67:0/3691133538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad81ef540 con 0x7fbad80c5e40 2024-03-09T09:09:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.597+0000 7fbaddc00640 1 -- 172.21.15.67:0/3691133538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad804f3c0 con 0x7fbad80c5e40 2024-03-09T09:09:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.598+0000 7fbacfe00640 1 -- 172.21.15.67:0/3691133538 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbac8014ba0 con 0x7fbad80c5e40 2024-03-09T09:09:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.599+0000 7fbacfe00640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbab406b4d0 0x7fbab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.599+0000 7fbacfe00640 1 -- 172.21.15.67:0/3691133538 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbac808add0 con 0x7fbad80c5e40 2024-03-09T09:09:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.600+0000 7fbad7400640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbab406b4d0 0x7fbab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.601+0000 7fbad7400640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbab406b4d0 0x7fbab406d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbac0007fd0 tx=0x7fbac0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:46.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.604+0000 7fbacfe00640 1 -- 172.21.15.67:0/3691133538 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbac80598f0 con 0x7fbad80c5e40 2024-03-09T09:09:46.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:09:46.812 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:09:46.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.882+0000 7fbaddc00640 1 -- 172.21.15.67:0/3691133538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbad80cab90 con 0x7fbad80c5e40 2024-03-09T09:09:46.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.882+0000 7fbacfe00640 1 -- 172.21.15.67:0/3691133538 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fbac808c080 con 0x7fbad80c5e40 2024-03-09T09:09:46.886 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":122,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:09:46.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 -- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbab406b4d0 msgr2=0x7fbab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbab406b4d0 0x7fbab406d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbac0007fd0 tx=0x7fbac0009040 comp rx=0 tx=0).stop 2024-03-09T09:09:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 -- 172.21.15.67:0/3691133538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbad80c5e40 msgr2=0x7fbad81df450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbad80c5e40 0x7fbad81df450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbac8007e70 tx=0x7fbac800e4b0 comp rx=0 tx=0).stop 2024-03-09T09:09:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 -- 172.21.15.67:0/3691133538 shutdown_connections 2024-03-09T09:09:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbab406b4d0 0x7fbab406d990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbad8153300 0x7fbad81eea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad80cadb0 0x7fbad81e0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 --2- 172.21.15.67:0/3691133538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbad80c5e40 0x7fbad81df450 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 -- 172.21.15.67:0/3691133538 >> 172.21.15.67:0/3691133538 conn(0x7fbad80732d0 msgr2=0x7fbad8150470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 -- 172.21.15.67:0/3691133538 shutdown_connections 2024-03-09T09:09:46.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:46.885+0000 7fbaddc00640 1 -- 172.21.15.67:0/3691133538 wait complete. 2024-03-09T09:09:48.910 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 42% (110/258) Updating files: 43% (111/258) Updating files: 44% (114/258) Updating files: 45% (117/258) Updating files: 46% (119/258) Updating files: 47% (122/258) Updating files: 48% (124/258) Updating files: 49% (127/258) Updating files: 50% (129/258) Updating files: 51% (132/258) Updating files: 52% (135/258) Updating files: 53% (137/258) Updating files: 54% (140/258) Updating files: 55% (142/258) Updating files: 56% (145/258) Updating files: 57% (148/258) Updating files: 58% (150/258) Updating files: 59% (153/258) Updating files: 60% (155/258) Updating files: 61% (158/258) Updating files: 62% (160/258) Updating files: 63% (163/258) Updating files: 63% (164/258) Updating files: 64% (166/258) Updating files: 65% (168/258) Updating files: 66% (171/258) Updating files: 67% (173/258) Updating files: 68% (176/258) Updating files: 69% (179/258) Updating files: 70% (181/258) Updating files: 71% (184/258) Updating files: 72% (186/258) Updating files: 73% (189/258) Updating files: 74% (191/258) Updating files: 75% (194/258) Updating files: 76% (197/258) Updating files: 77% (199/258) Updating files: 78% (202/258) Updating files: 79% (204/258) Updating files: 80% (207/258) Updating files: 81% (209/258) Updating files: 82% (212/258) Updating files: 83% (215/258) Updating files: 84% (217/258) Updating files: 85% (220/258) Updating files: 86% (222/258) Updating files: 87% (225/258) Updating files: 88% (228/258) Updating files: 89% (230/258) Updating files: 90% (233/258) Updating files: 91% (235/258) Updating files: 92% (238/258) Updating files: 93% (240/258) Updating files: 94% (243/258) Updating files: 95% (246/258) Updating files: 96% (248/258) Updating files: 97% (251/258) Updating files: 98% (253/258) Updating files: 99% (256/258) Updating files: 100% (258/258) Updating files: 100% (258/258), done. 2024-03-09T09:09:48.971 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 321ecdaba2c v0.39 2024-03-09T09:09:48.980 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.39 2024-03-09T09:09:49.378 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:09:49.379 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 40 -eq 48 ']' 2024-03-09T09:09:49.379 INFO:tasks.workunit.client.1.smithi134.stdout:v0.40 2024-03-09T09:09:49.379 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.40 2024-03-09T09:09:49.379 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.40 2024-03-09T09:09:49.379 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.40 2024-03-09T09:09:49.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:09:50.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.119+0000 7f7313200640 1 Processor -- start 2024-03-09T09:09:50.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.120+0000 7f7313200640 1 -- start start 2024-03-09T09:09:50.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.121+0000 7f7313200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f73140c5e40 0x7f7314156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.121+0000 7f7313200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f73140cadb0 0x7f7314156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.121+0000 7f7313200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7314156a90 con 0x7f7314153300 2024-03-09T09:09:50.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.121+0000 7f7313200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7314157000 con 0x7f73140c5e40 2024-03-09T09:09:50.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.121+0000 7f7313200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73140ae430 con 0x7f73140cadb0 2024-03-09T09:09:50.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.121+0000 7f730be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f73140c5e40 0x7f7314156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.121+0000 7f7311e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7314153300 0x7f73140b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60598/0 (socket says 172.21.15.67:60598) 2024-03-09T09:09:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.122+0000 7f7311e00640 1 -- 172.21.15.67:0/2672402790 learned_addr learned my addr 172.21.15.67:0/2672402790 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.122+0000 7f730be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f73140c5e40 0x7f7314156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52640/0 (socket says 172.21.15.67:52640) 2024-03-09T09:09:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.122+0000 7f7312800640 1 --2- 172.21.15.67:0/2672402790 >> v2:172.21.15.134:3300/0 conn(0x7f73140cadb0 0x7f7314156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.122+0000 7f730be00640 1 -- 172.21.15.67:0/2672402790 >> v2:172.21.15.134:3300/0 conn(0x7f73140cadb0 msgr2=0x7f7314156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.122+0000 7f730be00640 1 --2- 172.21.15.67:0/2672402790 >> v2:172.21.15.134:3300/0 conn(0x7f73140cadb0 0x7f7314156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.122+0000 7f730be00640 1 -- 172.21.15.67:0/2672402790 >> v1:172.21.15.67:6789/0 conn(0x7f7314153300 legacy=0x7f73140b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.122+0000 7f730be00640 1 -- 172.21.15.67:0/2672402790 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73140ae6f0 con 0x7f73140c5e40 2024-03-09T09:09:50.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.123+0000 7f730be00640 1 --2- 172.21.15.67:0/2672402790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73140c5e40 0x7f7314156010 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f73140adc60 tx=0x7f72fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7515532dc5bac2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.124+0000 7f7311400640 1 -- 172.21.15.67:0/2672402790 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc035020 con 0x7f73140c5e40 2024-03-09T09:09:50.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.124+0000 7f7311400640 1 -- 172.21.15.67:0/2672402790 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc031830 con 0x7f73140c5e40 2024-03-09T09:09:50.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.124+0000 7f7311400640 1 -- 172.21.15.67:0/2672402790 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc031c00 con 0x7f73140c5e40 2024-03-09T09:09:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.124+0000 7f7313200640 1 -- 172.21.15.67:0/2672402790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73140c5e40 msgr2=0x7f7314156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.124+0000 7f7313200640 1 --2- 172.21.15.67:0/2672402790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73140c5e40 0x7f7314156010 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f73140adc60 tx=0x7f72fc02b8d0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.124+0000 7f7313200640 1 -- 172.21.15.67:0/2672402790 shutdown_connections 2024-03-09T09:09:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.124+0000 7f7313200640 1 --2- 172.21.15.67:0/2672402790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73140c5e40 0x7f7314156010 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.124+0000 7f7313200640 1 --2- 172.21.15.67:0/2672402790 >> v2:172.21.15.134:3300/0 conn(0x7f73140cadb0 0x7f7314156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.124+0000 7f7313200640 1 -- 172.21.15.67:0/2672402790 >> 172.21.15.67:0/2672402790 conn(0x7f73140732d0 msgr2=0x7f73140ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.125+0000 7f7313200640 1 -- 172.21.15.67:0/2672402790 shutdown_connections 2024-03-09T09:09:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.125+0000 7f7313200640 1 -- 172.21.15.67:0/2672402790 wait complete. 2024-03-09T09:09:50.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.126+0000 7f7313200640 1 Processor -- start 2024-03-09T09:09:50.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.127+0000 7f7313200640 1 -- start start 2024-03-09T09:09:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.127+0000 7f7313200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73140c5e40 0x7f7314153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.127+0000 7f7313200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73140cadb0 0x7f731414cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7313200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7314153300 0x7f731414d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7313200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7314159e90 con 0x7f7314153300 2024-03-09T09:09:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7313200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7314159bb0 con 0x7f73140c5e40 2024-03-09T09:09:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7313200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73140c59e0 con 0x7f73140cadb0 2024-03-09T09:09:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7312800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7314153300 0x7f731414d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7312800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7314153300 0x7f731414d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52644/0 (socket says 172.21.15.67:52644) 2024-03-09T09:09:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7312800640 1 -- 172.21.15.67:0/1372722637 learned_addr learned my addr 172.21.15.67:0/1372722637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f730be00640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73140cadb0 0x7f731414cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7311e00640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73140c5e40 0x7f7314153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7312800640 1 -- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73140cadb0 msgr2=0x7f731414cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7312800640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73140cadb0 0x7f731414cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7312800640 1 -- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73140c5e40 msgr2=0x7f7314153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7312800640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73140c5e40 0x7f7314153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7312800640 1 -- 172.21.15.67:0/1372722637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314154b80 con 0x7f7314153300 2024-03-09T09:09:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.128+0000 7f7311e00640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73140c5e40 0x7f7314153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:09:50.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.129+0000 7f7312800640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7314153300 0x7f731414d2b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f730c007e70 tx=0x7f730c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.129+0000 7f730a000640 1 -- 172.21.15.67:0/1372722637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c014620 con 0x7f7314153300 2024-03-09T09:09:50.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.129+0000 7f730a000640 1 -- 172.21.15.67:0/1372722637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730c019070 con 0x7f7314153300 2024-03-09T09:09:50.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.129+0000 7f730a000640 1 -- 172.21.15.67:0/1372722637 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c014a20 con 0x7f7314153300 2024-03-09T09:09:50.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.129+0000 7f7313200640 1 -- 172.21.15.67:0/1372722637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73141ef3f0 con 0x7f7314153300 2024-03-09T09:09:50.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.129+0000 7f7313200640 1 -- 172.21.15.67:0/1372722637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7314154130 con 0x7f7314153300 2024-03-09T09:09:50.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.130+0000 7f7313200640 1 -- 172.21.15.67:0/1372722637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731404f3c0 con 0x7f7314153300 2024-03-09T09:09:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.131+0000 7f730a000640 1 -- 172.21.15.67:0/1372722637 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f730c014bc0 con 0x7f7314153300 2024-03-09T09:09:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.132+0000 7f730a000640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f72f006b500 0x7f72f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.132+0000 7f730a000640 1 -- 172.21.15.67:0/1372722637 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f730c08aea0 con 0x7f7314153300 2024-03-09T09:09:50.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.133+0000 7f7311e00640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f72f006b500 0x7f72f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.134+0000 7f7311e00640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f72f006b500 0x7f72f006d9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f73140c59b0 tx=0x7f7304009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.137+0000 7f730a000640 1 -- 172.21.15.67:0/1372722637 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f730c059930 con 0x7f7314153300 2024-03-09T09:09:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.472+0000 7f7313200640 1 -- 172.21.15.67:0/1372722637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f73140cab90 con 0x7f7314153300 2024-03-09T09:09:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.473+0000 7f730a000640 1 -- 172.21.15.67:0/1372722637 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f730c059230 con 0x7f7314153300 2024-03-09T09:09:50.477 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:09:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 -- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f72f006b500 msgr2=0x7f72f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f72f006b500 0x7f72f006d9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f73140c59b0 tx=0x7f7304009040 comp rx=0 tx=0).stop 2024-03-09T09:09:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 -- 172.21.15.67:0/1372722637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7314153300 msgr2=0x7f731414d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7314153300 0x7f731414d2b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f730c007e70 tx=0x7f730c00e4b0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 -- 172.21.15.67:0/1372722637 shutdown_connections 2024-03-09T09:09:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f72f006b500 0x7f72f006d9c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7314153300 0x7f731414d2b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73140cadb0 0x7f731414cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 --2- 172.21.15.67:0/1372722637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73140c5e40 0x7f7314153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 -- 172.21.15.67:0/1372722637 >> 172.21.15.67:0/1372722637 conn(0x7f73140732d0 msgr2=0x7f73141521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 -- 172.21.15.67:0/1372722637 shutdown_connections 2024-03-09T09:09:50.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.477+0000 7f7313200640 1 -- 172.21.15.67:0/1372722637 wait complete. 2024-03-09T09:09:50.503 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:09:50.503 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:09:50.503 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:09:50.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.688+0000 7f3bf6800640 1 Processor -- start 2024-03-09T09:09:50.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.688+0000 7f3bf6800640 1 -- start start 2024-03-09T09:09:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.690+0000 7f3bf6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3bf014fd90 0x7f3bf0156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.690+0000 7f3bf6800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf0156bc0 con 0x7f3bf00c9380 2024-03-09T09:09:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.690+0000 7f3bf6800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf00c9890 con 0x7f3bf006c1d0 2024-03-09T09:09:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.690+0000 7f3bf6800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf006cd00 con 0x7f3bf014fd90 2024-03-09T09:09:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.690+0000 7f3bf5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3bf014fd90 0x7f3bf0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.690+0000 7f3bf4a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3bf00c9380 0x7f3bf00acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60614/0 (socket says 172.21.15.67:60614) 2024-03-09T09:09:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.690+0000 7f3bf4a00640 1 -- 172.21.15.67:0/1242276998 learned_addr learned my addr 172.21.15.67:0/1242276998 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.690+0000 7f3befe00640 1 -- 172.21.15.67:0/1242276998 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700199603 0 0) 0x7f3bdc001110 con 0x7f3bf00c9380 2024-03-09T09:09:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.691+0000 7f3befe00640 1 -- 172.21.15.67:0/1242276998 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bd8003520 con 0x7f3bf00c9380 2024-03-09T09:09:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.691+0000 7f3bf5400640 1 -- 172.21.15.67:0/1242276998 >> v1:172.21.15.134:6789/0 conn(0x7f3bf006c1d0 legacy=0x7f3bf006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.691+0000 7f3bf5400640 1 -- 172.21.15.67:0/1242276998 >> v1:172.21.15.67:6789/0 conn(0x7f3bf00c9380 legacy=0x7f3bf00acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.691+0000 7f3bf5400640 1 -- 172.21.15.67:0/1242276998 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00c5f80 con 0x7f3bf014fd90 2024-03-09T09:09:50.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.693+0000 7f3bf5400640 1 --2- 172.21.15.67:0/1242276998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf014fd90 0x7f3bf0156680 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3be40097a0 tx=0x7f3be402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdc3ad9ffc8584f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.693+0000 7f3befe00640 1 -- 172.21.15.67:0/1242276998 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be40315a0 con 0x7f3bf014fd90 2024-03-09T09:09:50.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.693+0000 7f3befe00640 1 -- 172.21.15.67:0/1242276998 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be4037070 con 0x7f3bf014fd90 2024-03-09T09:09:50.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.693+0000 7f3befe00640 1 -- 172.21.15.67:0/1242276998 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4031940 con 0x7f3bf014fd90 2024-03-09T09:09:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.694+0000 7f3bf6800640 1 -- 172.21.15.67:0/1242276998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf014fd90 msgr2=0x7f3bf0156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.694+0000 7f3bf6800640 1 --2- 172.21.15.67:0/1242276998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf014fd90 0x7f3bf0156680 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3be40097a0 tx=0x7f3be402b840 comp rx=0 tx=0).stop 2024-03-09T09:09:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.694+0000 7f3bf6800640 1 -- 172.21.15.67:0/1242276998 shutdown_connections 2024-03-09T09:09:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.694+0000 7f3bf6800640 1 --2- 172.21.15.67:0/1242276998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf014fd90 0x7f3bf0156680 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.694+0000 7f3bf6800640 1 -- 172.21.15.67:0/1242276998 >> 172.21.15.67:0/1242276998 conn(0x7f3bf00cb480 msgr2=0x7f3bf00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:50.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.694+0000 7f3bf6800640 1 -- 172.21.15.67:0/1242276998 shutdown_connections 2024-03-09T09:09:50.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.694+0000 7f3bf6800640 1 -- 172.21.15.67:0/1242276998 wait complete. 2024-03-09T09:09:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.697+0000 7f3bf6800640 1 Processor -- start 2024-03-09T09:09:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.697+0000 7f3bf6800640 1 -- start start 2024-03-09T09:09:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.698+0000 7f3bf6800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf006c1d0 0x7f3bf00ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf6800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf00c9380 0x7f3bf00afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf014fd90 0x7f3bf00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf6800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00c6520 con 0x7f3bf014fd90 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf6800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf006ba60 con 0x7f3bf00c9380 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf6800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf006bcb0 con 0x7f3bf006c1d0 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf014fd90 0x7f3bf00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf014fd90 0x7f3bf00b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52666/0 (socket says 172.21.15.67:52666) 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf5e00640 1 -- 172.21.15.67:0/4008066799 learned_addr learned my addr 172.21.15.67:0/4008066799 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf5400640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf006c1d0 0x7f3bf00ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf4a00640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf00c9380 0x7f3bf00afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf5e00640 1 -- 172.21.15.67:0/4008066799 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf006c1d0 msgr2=0x7f3bf00ac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf5e00640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf006c1d0 0x7f3bf00ac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf5e00640 1 -- 172.21.15.67:0/4008066799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf00c9380 msgr2=0x7f3bf00afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf5e00640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf00c9380 0x7f3bf00afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.699+0000 7f3bf5e00640 1 -- 172.21.15.67:0/4008066799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00c1d80 con 0x7f3bf014fd90 2024-03-09T09:09:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.700+0000 7f3bf5e00640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf014fd90 0x7f3bf00b00c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3be800c6b0 tx=0x7f3be800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.700+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8003b70 con 0x7f3bf014fd90 2024-03-09T09:09:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.700+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be8003d30 con 0x7f3bf014fd90 2024-03-09T09:09:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.700+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8005310 con 0x7f3bf014fd90 2024-03-09T09:09:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.700+0000 7f3bf6800640 1 -- 172.21.15.67:0/4008066799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf01ebd50 con 0x7f3bf014fd90 2024-03-09T09:09:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.700+0000 7f3bf6800640 1 -- 172.21.15.67:0/4008066799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf00ae420 con 0x7f3bf014fd90 2024-03-09T09:09:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.702+0000 7f3bf6800640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00ae5f0 0x7f3bf00c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.702+0000 7f3bf6800640 1 -- 172.21.15.67:0/4008066799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00b0600 con 0x7f3bf00ae5f0 2024-03-09T09:09:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.702+0000 7f3bf5400640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00ae5f0 0x7f3bf00c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.703+0000 7f3bf5400640 1 -- 172.21.15.67:0/4008066799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3bf004f3c0 con 0x7f3bf00ae5f0 2024-03-09T09:09:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.703+0000 7f3bf5400640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00ae5f0 0x7f3bf00c4710 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00ae040 tx=0x7f3be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.703+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3be80054b0 con 0x7f3bf014fd90 2024-03-09T09:09:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.705+0000 7f3bed600640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3bd006b4d0 0x7f3bd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.705+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be404a070 con 0x7f3bf00ae5f0 2024-03-09T09:09:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.705+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f3be808a510 con 0x7f3bf014fd90 2024-03-09T09:09:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.705+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f3be4037050 con 0x7f3bf00ae5f0 2024-03-09T09:09:50.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.705+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00ae5f0 msgr2=0x7f3bf00c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.705+0000 7f3bed600640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00ae5f0 0x7f3bf00c4710 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00ae040 tx=0x7f3be40046c0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.705+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00ae5f0 msgr2=0x7f3bf00c4710 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:50.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.705+0000 7f3bed600640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00ae5f0 0x7f3bf00c4710 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.705+0000 7f3bf4a00640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3bd006b4d0 0x7f3bd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.706+0000 7f3bf4a00640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3bd006b4d0 0x7f3bd006d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc007fd0 tx=0x7f3bdc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.713+0000 7f3bf6800640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00af6e0 0x7f3bf00c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.713+0000 7f3bf6800640 1 -- 172.21.15.67:0/4008066799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00c5190 con 0x7f3bf00af6e0 2024-03-09T09:09:50.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.713+0000 7f3bf5400640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00af6e0 0x7f3bf00c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.714+0000 7f3bf5400640 1 -- 172.21.15.67:0/4008066799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3bf00ca8d0 con 0x7f3bf00af6e0 2024-03-09T09:09:50.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.714+0000 7f3bf5400640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00af6e0 0x7f3bf00c4c50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3be40097a0 tx=0x7f3be4031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.714+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be404a040 con 0x7f3bf00af6e0 2024-03-09T09:09:50.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f3be404d6c0 con 0x7f3bf00af6e0 2024-03-09T09:09:50.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00af6e0 msgr2=0x7f3bf00c4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bed600640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00af6e0 0x7f3bf00c4c50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3be40097a0 tx=0x7f3be4031000 comp rx=0 tx=0).stop 2024-03-09T09:09:50.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bed600640 1 -- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00af6e0 msgr2=0x7f3bf00c4c50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:50.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bed600640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf00af6e0 0x7f3bf00c4c50 secure :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3be40097a0 tx=0x7f3be4031000 comp rx=0 tx=0).stop 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 124, 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:09:50.720 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:50.721 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:50.722 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:09:50.723 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:50.724 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:09:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:50.728 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:09:50.729 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:50.730 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.731 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.732 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:50.733 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:50.733 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:09:50.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.733 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.733 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:09:50.733 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:09:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bf6800640 1 -- 172.21.15.67:0/4008066799 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3bd006b4d0 msgr2=0x7f3bd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bf6800640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3bd006b4d0 0x7f3bd006d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc007fd0 tx=0x7f3bdc009040 comp rx=0 tx=0).stop 2024-03-09T09:09:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bf6800640 1 -- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf014fd90 msgr2=0x7f3bf00b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bf6800640 1 --2- 172.21.15.67:0/4008066799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf014fd90 0x7f3bf00b00c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3be800c6b0 tx=0x7f3be800cb70 comp rx=0 tx=0).stop 2024-03-09T09:09:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bf5e00640 1 -- 172.21.15.67:0/4008066799 reap_dead start 2024-03-09T09:09:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bf5e00640 1 -- 172.21.15.67:0/4008066799 reap_dead start 2024-03-09T09:09:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bf6800640 1 -- 172.21.15.67:0/4008066799 shutdown_connections 2024-03-09T09:09:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.715+0000 7f3bf6800640 1 -- 172.21.15.67:0/4008066799 >> 172.21.15.67:0/4008066799 conn(0x7f3bf00cb480 msgr2=0x7f3bf0153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.716+0000 7f3bf6800640 1 -- 172.21.15.67:0/4008066799 shutdown_connections 2024-03-09T09:09:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.716+0000 7f3bf6800640 1 -- 172.21.15.67:0/4008066799 wait complete. 2024-03-09T09:09:50.736 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:09:50.737 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:09:50.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:09:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.945+0000 7f71a6a00640 1 Processor -- start 2024-03-09T09:09:50.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.945+0000 7f71a6a00640 1 -- start start 2024-03-09T09:09:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.947+0000 7f71a6a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f71a0071190 0x7f71a01548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.947+0000 7f71a6a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a00c8200 con 0x7f71a00ac9a0 2024-03-09T09:09:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.947+0000 7f71a6a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a00c3990 con 0x7f71a00c2780 2024-03-09T09:09:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.947+0000 7f71a6a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a006d590 con 0x7f71a0071190 2024-03-09T09:09:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.947+0000 7f71a4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f71a0071190 0x7f71a01548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.947+0000 7f719fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f71a00ac9a0 0x7f71a01510e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60624/0 (socket says 172.21.15.67:60624) 2024-03-09T09:09:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.947+0000 7f719fe00640 1 -- 172.21.15.67:0/2500630640 learned_addr learned my addr 172.21.15.67:0/2500630640 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.948+0000 7f71a4a00640 1 -- 172.21.15.67:0/2500630640 >> v1:172.21.15.134:6789/0 conn(0x7f71a00c2780 legacy=0x7f71a0154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.948+0000 7f71a4a00640 1 -- 172.21.15.67:0/2500630640 >> v1:172.21.15.67:6789/0 conn(0x7f71a00ac9a0 legacy=0x7f71a01510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.948+0000 7f71a4a00640 1 -- 172.21.15.67:0/2500630640 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0070920 con 0x7f71a0071190 2024-03-09T09:09:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.948+0000 7f719ea00640 1 -- 172.21.15.67:0/2500630640 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531034042 0 0) 0x7f71a00c8200 con 0x7f71a00ac9a0 2024-03-09T09:09:50.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.949+0000 7f71a4a00640 1 --2- 172.21.15.67:0/2500630640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a0071190 0x7f71a01548a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7198009d90 tx=0x7f719802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56c9670e263d22 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.950+0000 7f719ea00640 1 -- 172.21.15.67:0/2500630640 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198039070 con 0x7f71a0071190 2024-03-09T09:09:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.950+0000 7f719ea00640 1 -- 172.21.15.67:0/2500630640 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719802cbd0 con 0x7f71a0071190 2024-03-09T09:09:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.950+0000 7f719ea00640 1 -- 172.21.15.67:0/2500630640 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198034440 con 0x7f71a0071190 2024-03-09T09:09:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.950+0000 7f71a6a00640 1 -- 172.21.15.67:0/2500630640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a0071190 msgr2=0x7f71a01548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.950+0000 7f71a6a00640 1 --2- 172.21.15.67:0/2500630640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a0071190 0x7f71a01548a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7198009d90 tx=0x7f719802be20 comp rx=0 tx=0).stop 2024-03-09T09:09:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.950+0000 7f71a6a00640 1 -- 172.21.15.67:0/2500630640 shutdown_connections 2024-03-09T09:09:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.950+0000 7f71a6a00640 1 --2- 172.21.15.67:0/2500630640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a0071190 0x7f71a01548a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.950+0000 7f71a6a00640 1 -- 172.21.15.67:0/2500630640 >> 172.21.15.67:0/2500630640 conn(0x7f71a00c7370 msgr2=0x7f71a00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.951+0000 7f71a6a00640 1 -- 172.21.15.67:0/2500630640 shutdown_connections 2024-03-09T09:09:50.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.951+0000 7f71a6a00640 1 -- 172.21.15.67:0/2500630640 wait complete. 2024-03-09T09:09:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.952+0000 7f71a6a00640 1 Processor -- start 2024-03-09T09:09:50.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.952+0000 7f71a6a00640 1 -- start start 2024-03-09T09:09:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.953+0000 7f71a6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a0071190 0x7f71a01ef050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.953+0000 7f71a6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a00ac9a0 0x7f71a01ef590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.953+0000 7f71a6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71a00c2780 0x7f71a0146480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.953+0000 7f71a6a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00acd80 con 0x7f71a00ac9a0 2024-03-09T09:09:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.953+0000 7f71a6a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00c3740 con 0x7f71a0071190 2024-03-09T09:09:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.953+0000 7f71a6a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00acf00 con 0x7f71a00c2780 2024-03-09T09:09:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.954+0000 7f719f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a00ac9a0 0x7f71a01ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.954+0000 7f719f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a00ac9a0 0x7f71a01ef590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52680/0 (socket says 172.21.15.67:52680) 2024-03-09T09:09:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.954+0000 7f719f400640 1 -- 172.21.15.67:0/2711676901 learned_addr learned my addr 172.21.15.67:0/2711676901 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.954+0000 7f719fe00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a0071190 0x7f71a01ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.954+0000 7f719f400640 1 -- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71a00c2780 msgr2=0x7f71a0146480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.954+0000 7f719f400640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71a00c2780 0x7f71a0146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.954+0000 7f719f400640 1 -- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a0071190 msgr2=0x7f71a01ef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.954+0000 7f719f400640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a0071190 0x7f71a01ef050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.954+0000 7f719f400640 1 -- 172.21.15.67:0/2711676901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a01efad0 con 0x7f71a00ac9a0 2024-03-09T09:09:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.955+0000 7f719f400640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a00ac9a0 0x7f71a01ef590 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f718c00b3c0 tx=0x7f718c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.955+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c011830 con 0x7f71a00ac9a0 2024-03-09T09:09:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.955+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718c016070 con 0x7f71a00ac9a0 2024-03-09T09:09:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.955+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c011c10 con 0x7f71a00ac9a0 2024-03-09T09:09:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.955+0000 7f71a6a00640 1 -- 172.21.15.67:0/2711676901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a0151f80 con 0x7f71a00ac9a0 2024-03-09T09:09:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.955+0000 7f71a6a00640 1 -- 172.21.15.67:0/2711676901 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a00c64e0 con 0x7f71a00ac9a0 2024-03-09T09:09:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.957+0000 7f71a6a00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a0075a00 0x7f71a01521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.957+0000 7f71a6a00640 1 -- 172.21.15.67:0/2711676901 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a0154e70 con 0x7f71a0075a00 2024-03-09T09:09:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.957+0000 7f719fe00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a0075a00 0x7f71a01521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.958+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f718c01b460 con 0x7f71a00ac9a0 2024-03-09T09:09:50.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.958+0000 7f719fe00640 1 -- 172.21.15.67:0/2711676901 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71a0150550 con 0x7f71a0075a00 2024-03-09T09:09:50.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.959+0000 7f719fe00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a0075a00 0x7f71a01521b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f719000d410 tx=0x7f719000d8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.960+0000 7f7197e00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f718006b500 0x7f718006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.960+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f718c089ff0 con 0x7f71a00ac9a0 2024-03-09T09:09:50.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.960+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7190018070 con 0x7f71a0075a00 2024-03-09T09:09:50.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.960+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f71900075a0 con 0x7f71a0075a00 2024-03-09T09:09:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.960+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a0075a00 msgr2=0x7f71a01521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.960+0000 7f7197e00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a0075a00 0x7f71a01521b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f719000d410 tx=0x7f719000d8d0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.960+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a0075a00 msgr2=0x7f71a01521b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.960+0000 7f7197e00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a0075a00 0x7f71a01521b0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f719000d410 tx=0x7f719000d8d0 comp rx=0 tx=0).stop 2024-03-09T09:09:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.960+0000 7f71a4a00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f718006b500 0x7f718006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.961+0000 7f71a4a00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f718006b500 0x7f718006d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7198036000 tx=0x7f719802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.972+0000 7f71a6a00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a01469c0 0x7f71a0148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:50.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.972+0000 7f71a6a00640 1 -- 172.21.15.67:0/2711676901 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00c5200 con 0x7f71a01469c0 2024-03-09T09:09:50.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.973+0000 7f719fe00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a01469c0 0x7f71a0148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:50.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.974+0000 7f719fe00640 1 -- 172.21.15.67:0/2711676901 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f71a0067dd0 con 0x7f71a01469c0 2024-03-09T09:09:50.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.975+0000 7f719fe00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a01469c0 0x7f71a0148440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7190015000 tx=0x7f7190001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:50.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.975+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7190018050 con 0x7f71a01469c0 2024-03-09T09:09:50.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f7190019040 con 0x7f71a01469c0 2024-03-09T09:09:50.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a01469c0 msgr2=0x7f71a0148440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f7197e00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a01469c0 0x7f71a0148440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7190015000 tx=0x7f7190001080 comp rx=0 tx=0).stop 2024-03-09T09:09:50.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f7197e00640 1 -- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a01469c0 msgr2=0x7f71a0148440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:50.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f7197e00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a01469c0 0x7f71a0148440 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7190015000 tx=0x7f7190001080 comp rx=0 tx=0).stop 2024-03-09T09:09:50.980 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:09:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:09:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:09:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:09:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 124, 2024-03-09T09:09:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:09:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:09:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:09:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:09:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:50.981 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:09:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:09:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:09:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:50.982 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:09:50.983 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:50.984 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:50.984 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:09:50.985 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:09:50.986 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:50.987 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:50.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.990 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:09:50.991 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:50.993 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.994 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:50.994 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:09:50.994 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:50.994 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:50.994 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:50.994 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:50.994 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:50.994 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:50.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:50.994 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:09:50.994 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:09:50.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f71a6a00640 1 -- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f718006b500 msgr2=0x7f718006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f71a6a00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f718006b500 0x7f718006d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7198036000 tx=0x7f719802e000 comp rx=0 tx=0).stop 2024-03-09T09:09:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f71a6a00640 1 -- 172.21.15.67:0/2711676901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a00ac9a0 msgr2=0x7f71a01ef590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f71a6a00640 1 --2- 172.21.15.67:0/2711676901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71a00ac9a0 0x7f71a01ef590 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f718c00b3c0 tx=0x7f718c00b880 comp rx=0 tx=0).stop 2024-03-09T09:09:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f71a4a00640 1 -- 172.21.15.67:0/2711676901 reap_dead start 2024-03-09T09:09:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f71a4a00640 1 -- 172.21.15.67:0/2711676901 reap_dead start 2024-03-09T09:09:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.976+0000 7f71a6a00640 1 -- 172.21.15.67:0/2711676901 shutdown_connections 2024-03-09T09:09:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.977+0000 7f71a6a00640 1 -- 172.21.15.67:0/2711676901 >> 172.21.15.67:0/2711676901 conn(0x7f71a00c7370 msgr2=0x7f71a00c4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.977+0000 7f71a6a00640 1 -- 172.21.15.67:0/2711676901 shutdown_connections 2024-03-09T09:09:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:50.977+0000 7f71a6a00640 1 -- 172.21.15.67:0/2711676901 wait complete. 2024-03-09T09:09:51.005 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:09:51.005 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:09:51.005 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:09:51.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.236+0000 7fc8acc00640 1 Processor -- start 2024-03-09T09:09:51.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.236+0000 7fc8acc00640 1 -- start start 2024-03-09T09:09:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.238+0000 7fc8acc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc8a8150db0 0x7fc8a81538c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.238+0000 7fc8acc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a80c33e0 0x7fc8a80c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.238+0000 7fc8acc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a8154110 con 0x7fc8a806d5f0 2024-03-09T09:09:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.238+0000 7fc8acc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a81543f0 con 0x7fc8a8150db0 2024-03-09T09:09:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.238+0000 7fc8acc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81546d0 con 0x7fc8a80c33e0 2024-03-09T09:09:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.239+0000 7fc8a6a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc8a806d5f0 0x7fc8a81575e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:60640/0 (socket says 172.21.15.67:60640) 2024-03-09T09:09:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.239+0000 7fc8a6a00640 1 -- 172.21.15.67:0/2368140063 learned_addr learned my addr 172.21.15.67:0/2368140063 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:09:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.239+0000 7fc8a7e00640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a80c33e0 0x7fc8a80c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.239+0000 7fc8a7400640 1 --2- 172.21.15.67:0/2368140063 >> v2:172.21.15.134:3300/0 conn(0x7fc8a8150db0 0x7fc8a81538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.239+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912737401 0 0) 0x7fc8a8154110 con 0x7fc8a806d5f0 2024-03-09T09:09:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.239+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc88c003520 con 0x7fc8a806d5f0 2024-03-09T09:09:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.239+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3342785053 0 0) 0x7fc88c003520 con 0x7fc8a806d5f0 2024-03-09T09:09:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a80c33e0 msgr2=0x7fc8a80c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a80c33e0 0x7fc8a80c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 >> v2:172.21.15.134:3300/0 conn(0x7fc8a8150db0 msgr2=0x7fc8a81538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 --2- 172.21.15.67:0/2368140063 >> v2:172.21.15.134:3300/0 conn(0x7fc8a8150db0 0x7fc8a81538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a81570c0 con 0x7fc8a806d5f0 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fc894001670 con 0x7fc8a806d5f0 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 >> v1:172.21.15.67:6789/0 conn(0x7fc8a806d5f0 legacy=0x7fc8a81575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c003a20 0x7fc88c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc88c004800 0x7fc88c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a80c33e0 0x7fc88c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81543f0 con 0x7fc88c003a20 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81546d0 con 0x7fc88c004800 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c00b9c0 con 0x7fc8a80c33e0 2024-03-09T09:09:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.240+0000 7fc8a7400640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c003a20 0x7fc88c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.241+0000 7fc8a6a00640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc88c004800 0x7fc88c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.241+0000 7fc8a7400640 1 -- 172.21.15.67:0/2368140063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a80c33e0 msgr2=0x7fc88c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.241+0000 7fc8a7400640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a80c33e0 0x7fc88c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.241+0000 7fc8a7400640 1 -- 172.21.15.67:0/2368140063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc88c004800 msgr2=0x7fc88c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.241+0000 7fc8a7400640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc88c004800 0x7fc88c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.241+0000 7fc8a7400640 1 -- 172.21.15.67:0/2368140063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a81570c0 con 0x7fc88c003a20 2024-03-09T09:09:51.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.241+0000 7fc8a6a00640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc88c004800 0x7fc88c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:09:51.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.242+0000 7fc8a7400640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c003a20 0x7fc88c003e40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc898009af0 tx=0x7fc88c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5156d522f8c1c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:51.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.243+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898039070 con 0x7fc88c003a20 2024-03-09T09:09:51.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.243+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89802c710 con 0x7fc88c003a20 2024-03-09T09:09:51.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.243+0000 7fc8a6000640 1 -- 172.21.15.67:0/2368140063 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898009690 con 0x7fc88c003a20 2024-03-09T09:09:51.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.243+0000 7fc8acc00640 1 -- 172.21.15.67:0/2368140063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c003a20 msgr2=0x7fc88c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.243+0000 7fc8acc00640 1 --2- 172.21.15.67:0/2368140063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c003a20 0x7fc88c003e40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc898009af0 tx=0x7fc88c005830 comp rx=0 tx=0).stop 2024-03-09T09:09:51.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.244+0000 7fc8a7e00640 1 -- 172.21.15.67:0/2368140063 reap_dead start 2024-03-09T09:09:51.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.244+0000 7fc8acc00640 1 -- 172.21.15.67:0/2368140063 shutdown_connections 2024-03-09T09:09:51.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.244+0000 7fc8acc00640 1 -- 172.21.15.67:0/2368140063 >> 172.21.15.67:0/2368140063 conn(0x7fc8a806c150 msgr2=0x7fc8a806c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:51.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.244+0000 7fc8acc00640 1 -- 172.21.15.67:0/2368140063 shutdown_connections 2024-03-09T09:09:51.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.244+0000 7fc8acc00640 1 -- 172.21.15.67:0/2368140063 wait complete. 2024-03-09T09:09:51.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.246+0000 7fc8acc00640 1 Processor -- start 2024-03-09T09:09:51.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.246+0000 7fc8acc00640 1 -- start start 2024-03-09T09:09:51.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.247+0000 7fc8acc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc88c003a20 0x7fc8a80acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.247+0000 7fc8acc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a80c6c70 0x7fc8a80c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.247+0000 7fc8acc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8a80c88d0 0x7fc8a80c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.247+0000 7fc8acc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a8151190 con 0x7fc8a80c6c70 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.247+0000 7fc8acc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a8157cf0 con 0x7fc8a80c88d0 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.247+0000 7fc8acc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a8151310 con 0x7fc88c003a20 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.248+0000 7fc8a6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a80c6c70 0x7fc8a80c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.248+0000 7fc8a6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a80c6c70 0x7fc8a80c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52702/0 (socket says 172.21.15.67:52702) 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.248+0000 7fc8a6a00640 1 -- 172.21.15.67:0/1706971713 learned_addr learned my addr 172.21.15.67:0/1706971713 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.248+0000 7fc8a7400640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc88c003a20 0x7fc8a80acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.248+0000 7fc8a7e00640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8a80c88d0 0x7fc8a80c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.248+0000 7fc8a6a00640 1 -- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc88c003a20 msgr2=0x7fc8a80acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.248+0000 7fc8a6a00640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc88c003a20 0x7fc8a80acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.248+0000 7fc8a6a00640 1 -- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8a80c88d0 msgr2=0x7fc8a80c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.248+0000 7fc8a6a00640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8a80c88d0 0x7fc8a80c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.248+0000 7fc8a6a00640 1 -- 172.21.15.67:0/1706971713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80c8ec0 con 0x7fc8a80c6c70 2024-03-09T09:09:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.249+0000 7fc8a6a00640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a80c6c70 0x7fc8a80c78e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc89400d960 tx=0x7fc89400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.249+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894019070 con 0x7fc8a80c6c70 2024-03-09T09:09:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.249+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8940074e0 con 0x7fc8a80c6c70 2024-03-09T09:09:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.249+0000 7fc8acc00640 1 -- 172.21.15.67:0/1706971713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a80736f0 con 0x7fc8a80c6c70 2024-03-09T09:09:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.249+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894004a20 con 0x7fc8a80c6c70 2024-03-09T09:09:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.249+0000 7fc8acc00640 1 -- 172.21.15.67:0/1706971713 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a80ad330 con 0x7fc8a80c6c70 2024-03-09T09:09:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.251+0000 7fc8acc00640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81eeae0 0x7fc8a81eeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.251+0000 7fc8acc00640 1 -- 172.21.15.67:0/1706971713 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80c91a0 con 0x7fc8a81eeae0 2024-03-09T09:09:51.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.251+0000 7fc8a7400640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81eeae0 0x7fc8a81eeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.252+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc89401c070 con 0x7fc8a80c6c70 2024-03-09T09:09:51.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.252+0000 7fc8a7400640 1 -- 172.21.15.67:0/1706971713 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc8a804f3c0 con 0x7fc8a81eeae0 2024-03-09T09:09:51.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.253+0000 7fc89f400640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc88806b4d0 0x7fc88806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.253+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc89408a090 con 0x7fc8a80c6c70 2024-03-09T09:09:51.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.253+0000 7fc8a7400640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81eeae0 0x7fc8a81eeec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc898009ac0 tx=0x7fc898004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:51.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.253+0000 7fc8a7e00640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc88806b4d0 0x7fc88806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.254+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898039070 con 0x7fc8a81eeae0 2024-03-09T09:09:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.254+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc89803a040 con 0x7fc8a81eeae0 2024-03-09T09:09:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.254+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81eeae0 msgr2=0x7fc8a81eeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.254+0000 7fc89f400640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81eeae0 0x7fc8a81eeec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc898009ac0 tx=0x7fc898004730 comp rx=0 tx=0).stop 2024-03-09T09:09:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.254+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81eeae0 msgr2=0x7fc8a81eeec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.254+0000 7fc89f400640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81eeae0 0x7fc8a81eeec0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc898009ac0 tx=0x7fc898004730 comp rx=0 tx=0).stop 2024-03-09T09:09:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.255+0000 7fc8a7e00640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc88806b4d0 0x7fc88806d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80c4390 tx=0x7fc8a000d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.263+0000 7fc8acc00640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81ef400 0x7fc8a8147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.263+0000 7fc8acc00640 1 -- 172.21.15.67:0/1706971713 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80c53c0 con 0x7fc8a81ef400 2024-03-09T09:09:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.264+0000 7fc8a7400640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81ef400 0x7fc8a8147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.264+0000 7fc8a7400640 1 -- 172.21.15.67:0/1706971713 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc8a80c31c0 con 0x7fc8a81ef400 2024-03-09T09:09:51.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.265+0000 7fc8a7400640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81ef400 0x7fc8a8147660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc898000c00 tx=0x7fc898008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:51.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.266+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898048750 con 0x7fc8a81ef400 2024-03-09T09:09:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fc89804c070 con 0x7fc8a81ef400 2024-03-09T09:09:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81ef400 msgr2=0x7fc8a8147660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc89f400640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81ef400 0x7fc8a8147660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc898000c00 tx=0x7fc898008030 comp rx=0 tx=0).stop 2024-03-09T09:09:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc89f400640 1 -- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81ef400 msgr2=0x7fc8a8147660 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:09:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc89f400640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8a81ef400 0x7fc8a8147660 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc898000c00 tx=0x7fc898008030 comp rx=0 tx=0).stop 2024-03-09T09:09:51.271 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:09:51.272 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:09:51.272 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:09:51.272 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:09:51.272 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 124, 2024-03-09T09:09:51.272 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:09:51.272 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:09:51.272 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:09:51.272 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:09:51.272 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:51.272 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:09:51.272 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:51.273 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:51.274 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:51.274 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:09:51.274 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:09:51.274 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:51.274 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:51.274 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:51.274 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:51.274 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:51.274 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:51.274 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:51.275 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:51.275 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:51.275 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:51.275 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:51.275 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:51.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.275 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:09:51.275 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:09:51.275 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:09:51.275 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:09:51.276 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.277 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:51.278 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:09:51.279 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.280 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:09:51.281 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:09:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:09:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:09:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:09:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:09:51.282 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:09:51.283 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:09:51.284 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:09:51.284 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:09:51.284 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:09:51.284 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:09:51.284 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:09:51.284 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:09:51.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:09:51.284 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:09:51.284 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:09:51.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc8acc00640 1 -- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc88806b4d0 msgr2=0x7fc88806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc8acc00640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc88806b4d0 0x7fc88806d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80c4390 tx=0x7fc8a000d040 comp rx=0 tx=0).stop 2024-03-09T09:09:51.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc8acc00640 1 -- 172.21.15.67:0/1706971713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a80c6c70 msgr2=0x7fc8a80c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc8acc00640 1 --2- 172.21.15.67:0/1706971713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a80c6c70 0x7fc8a80c78e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc89400d960 tx=0x7fc89400de20 comp rx=0 tx=0).stop 2024-03-09T09:09:51.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc8a7e00640 1 -- 172.21.15.67:0/1706971713 reap_dead start 2024-03-09T09:09:51.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc8a7e00640 1 -- 172.21.15.67:0/1706971713 reap_dead start 2024-03-09T09:09:51.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc8acc00640 1 -- 172.21.15.67:0/1706971713 shutdown_connections 2024-03-09T09:09:51.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.267+0000 7fc8acc00640 1 -- 172.21.15.67:0/1706971713 >> 172.21.15.67:0/1706971713 conn(0x7fc8a806c150 msgr2=0x7fc8a81e2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:51.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.268+0000 7fc8acc00640 1 -- 172.21.15.67:0/1706971713 shutdown_connections 2024-03-09T09:09:51.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.268+0000 7fc8acc00640 1 -- 172.21.15.67:0/1706971713 wait complete. 2024-03-09T09:09:51.294 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:09:51.295 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:09:51.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:09:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.503+0000 7f2ee2200640 1 Processor -- start 2024-03-09T09:09:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.503+0000 7f2ee2200640 1 -- start start 2024-03-09T09:09:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.505+0000 7f2ee2200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2edc06b040 0x7f2edc155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.505+0000 7f2ee2200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2edc06c650 0x7f2edc0c6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.505+0000 7f2ee2200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc06b5b0 con 0x7f2edc0c5f50 2024-03-09T09:09:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.505+0000 7f2ee2200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2edc0c9e00 con 0x7f2edc06b040 2024-03-09T09:09:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.505+0000 7f2ee2200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc06cae0 con 0x7f2edc06c650 2024-03-09T09:09:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.505+0000 7f2edaa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2edc06b040 0x7f2edc155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.505+0000 7f2edaa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2edc06b040 0x7f2edc155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52712/0 (socket says 172.21.15.67:52712) 2024-03-09T09:09:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.505+0000 7f2edaa00640 1 -- 172.21.15.67:0/968700895 learned_addr learned my addr 172.21.15.67:0/968700895 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.506+0000 7f2edbe00640 1 --2- 172.21.15.67:0/968700895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2edc06c650 0x7f2edc0c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.506+0000 7f2edaa00640 1 -- 172.21.15.67:0/968700895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2edc06c650 msgr2=0x7f2edc0c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.506+0000 7f2edaa00640 1 --2- 172.21.15.67:0/968700895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2edc06c650 0x7f2edc0c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.506+0000 7f2edaa00640 1 -- 172.21.15.67:0/968700895 >> v1:172.21.15.134:6789/0 conn(0x7f2edc0c5f50 legacy=0x7f2edc0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:09:51.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.506+0000 7f2edaa00640 1 -- 172.21.15.67:0/968700895 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc06b890 con 0x7f2edc06b040 2024-03-09T09:09:51.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.508+0000 7f2edaa00640 1 --2- 172.21.15.67:0/968700895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc06b040 0x7f2edc155e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009870 tx=0x7f2ec802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90fab99035ab6d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:51.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.508+0000 7f2eda000640 1 -- 172.21.15.67:0/968700895 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8035020 con 0x7f2edc06b040 2024-03-09T09:09:51.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.508+0000 7f2eda000640 1 -- 172.21.15.67:0/968700895 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec8031640 con 0x7f2edc06b040 2024-03-09T09:09:51.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.508+0000 7f2eda000640 1 -- 172.21.15.67:0/968700895 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec802d040 con 0x7f2edc06b040 2024-03-09T09:09:51.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.509+0000 7f2ee2200640 1 -- 172.21.15.67:0/968700895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc06b040 msgr2=0x7f2edc155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.509+0000 7f2ee2200640 1 --2- 172.21.15.67:0/968700895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc06b040 0x7f2edc155e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009870 tx=0x7f2ec802b8f0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.509+0000 7f2ee2200640 1 -- 172.21.15.67:0/968700895 shutdown_connections 2024-03-09T09:09:51.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.509+0000 7f2ee2200640 1 --2- 172.21.15.67:0/968700895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2edc06c650 0x7f2edc0c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.509+0000 7f2ee2200640 1 --2- 172.21.15.67:0/968700895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc06b040 0x7f2edc155e20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.509+0000 7f2ee2200640 1 -- 172.21.15.67:0/968700895 >> 172.21.15.67:0/968700895 conn(0x7f2edc0c8050 msgr2=0x7f2edc0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:51.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.509+0000 7f2ee2200640 1 -- 172.21.15.67:0/968700895 shutdown_connections 2024-03-09T09:09:51.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.509+0000 7f2ee2200640 1 -- 172.21.15.67:0/968700895 wait complete. 2024-03-09T09:09:51.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.512+0000 7f2ee2200640 1 Processor -- start 2024-03-09T09:09:51.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.512+0000 7f2ee2200640 1 -- start start 2024-03-09T09:09:51.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.513+0000 7f2ee2200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc06b040 0x7f2edc14ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.513+0000 7f2ee2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc06c650 0x7f2edc0c2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.513+0000 7f2ee2200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2edc0c5f50 0x7f2edc14acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2ee2200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc06c130 con 0x7f2edc06c650 2024-03-09T09:09:51.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2ee2200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc06be50 con 0x7f2edc06b040 2024-03-09T09:09:51.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2ee2200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc14cc00 con 0x7f2edc0c5f50 2024-03-09T09:09:51.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2edaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc06c650 0x7f2edc0c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2edaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc06c650 0x7f2edc0c2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52720/0 (socket says 172.21.15.67:52720) 2024-03-09T09:09:51.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2edaa00640 1 -- 172.21.15.67:0/2523462028 learned_addr learned my addr 172.21.15.67:0/2523462028 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:09:51.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2edb400640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc06b040 0x7f2edc14ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2edaa00640 1 -- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2edc0c5f50 msgr2=0x7f2edc14acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2edaa00640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2edc0c5f50 0x7f2edc14acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2edaa00640 1 -- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc06b040 msgr2=0x7f2edc14ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2edaa00640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc06b040 0x7f2edc14ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2edaa00640 1 -- 172.21.15.67:0/2523462028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc0ad6f0 con 0x7f2edc06c650 2024-03-09T09:09:51.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.514+0000 7f2edb400640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc06b040 0x7f2edc14ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:09:51.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.515+0000 7f2edaa00640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc06c650 0x7f2edc0c2880 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8000c00 tx=0x7f2ec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:51.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.515+0000 7f2ed3400640 1 -- 172.21.15.67:0/2523462028 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8040070 con 0x7f2edc06c650 2024-03-09T09:09:51.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.515+0000 7f2ed3400640 1 -- 172.21.15.67:0/2523462028 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec80399f0 con 0x7f2edc06c650 2024-03-09T09:09:51.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.515+0000 7f2ed3400640 1 -- 172.21.15.67:0/2523462028 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8035040 con 0x7f2edc06c650 2024-03-09T09:09:51.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.515+0000 7f2ee2200640 1 -- 172.21.15.67:0/2523462028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc1585a0 con 0x7f2edc06c650 2024-03-09T09:09:51.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.515+0000 7f2ee2200640 1 -- 172.21.15.67:0/2523462028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc1e1580 con 0x7f2edc06c650 2024-03-09T09:09:51.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.517+0000 7f2ee2200640 1 -- 172.21.15.67:0/2523462028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2edc075a00 con 0x7f2edc06c650 2024-03-09T09:09:51.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.518+0000 7f2ed3400640 1 -- 172.21.15.67:0/2523462028 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2ec804b060 con 0x7f2edc06c650 2024-03-09T09:09:51.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.519+0000 7f2ed3400640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ebc06b500 0x7f2ebc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:09:51.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.519+0000 7f2ed3400640 1 -- 172.21.15.67:0/2523462028 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f2ec80abe90 con 0x7f2edc06c650 2024-03-09T09:09:51.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.520+0000 7f2edb400640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ebc06b500 0x7f2ebc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:09:51.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.521+0000 7f2edb400640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ebc06b500 0x7f2ebc06d9c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00a830 tx=0x7f2ecc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:09:51.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.523+0000 7f2ed3400640 1 -- 172.21.15.67:0/2523462028 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2ec807a9b0 con 0x7f2edc06c650 2024-03-09T09:09:51.832 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 36% (82/226) Updating files: 37% (84/226) Updating files: 38% (86/226) Updating files: 39% (89/226) Updating files: 40% (91/226) Updating files: 41% (93/226) Updating files: 42% (95/226) Updating files: 43% (98/226) Updating files: 44% (100/226) Updating files: 45% (102/226) Updating files: 46% (104/226) Updating files: 47% (107/226) Updating files: 48% (109/226) Updating files: 49% (111/226) Updating files: 50% (113/226) Updating files: 51% (116/226) Updating files: 52% (118/226) Updating files: 53% (120/226) Updating files: 54% (123/226) Updating files: 55% (125/226) Updating files: 56% (127/226) Updating files: 57% (129/226) Updating files: 58% (132/226) Updating files: 59% (134/226) Updating files: 60% (136/226) Updating files: 61% (138/226) Updating files: 62% (141/226) Updating files: 63% (143/226) Updating files: 64% (145/226) Updating files: 65% (147/226) Updating files: 66% (150/226) Updating files: 67% (152/226) Updating files: 68% (154/226) Updating files: 69% (156/226) Updating files: 70% (159/226) Updating files: 71% (161/226) Updating files: 72% (163/226) Updating files: 73% (165/226) Updating files: 74% (168/226) Updating files: 75% (170/226) Updating files: 76% (172/226) Updating files: 77% (175/226) Updating files: 78% (177/226) Updating files: 79% (179/226) Updating files: 80% (181/226) Updating files: 81% (184/226) Updating files: 82% (186/226) Updating files: 83% (188/226) Updating files: 84% (190/226) Updating files: 85% (193/226) Updating files: 86% (195/226) Updating files: 87% (197/226) Updating files: 88% (199/226) Updating files: 89% (202/226) Updating files: 90% (204/226) Updating files: 91% (206/226) Updating files: 92% (208/226) Updating files: 93% (211/226) Updating files: 94% (213/226) Updating files: 95% (215/226) Updating files: 96% (217/226) Updating files: 97% (220/226) Updating files: 98% (222/226) Updating files: 99% (224/226) Updating files: 100% (226/226) Updating files: 100% (226/226), done. 2024-03-09T09:09:51.891 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 7eea40ea37f v0.40 2024-03-09T09:09:51.899 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.40 2024-03-09T09:09:51.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.938+0000 7f2ee2200640 1 -- 172.21.15.67:0/2523462028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f2edc06cca0 con 0x7f2edc06c650 2024-03-09T09:09:51.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.940+0000 7f2ed3400640 1 -- 172.21.15.67:0/2523462028 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f2ec807a2b0 con 0x7f2edc06c650 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.943+0000 7f2ee2200640 1 -- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ebc06b500 msgr2=0x7f2ebc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.943+0000 7f2ee2200640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ebc06b500 0x7f2ebc06d9c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00a830 tx=0x7f2ecc008040 comp rx=0 tx=0).stop 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.943+0000 7f2ee2200640 1 -- 172.21.15.67:0/2523462028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc06c650 msgr2=0x7f2edc0c2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.943+0000 7f2ee2200640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc06c650 0x7f2edc0c2880 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8000c00 tx=0x7f2ec80046c0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.943+0000 7f2ee2200640 1 -- 172.21.15.67:0/2523462028 shutdown_connections 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.943+0000 7f2ee2200640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ebc06b500 0x7f2ebc06d9c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.943+0000 7f2ee2200640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2edc0c5f50 0x7f2edc14acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.943+0000 7f2ee2200640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2edc06c650 0x7f2edc0c2880 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.943+0000 7f2ee2200640 1 --2- 172.21.15.67:0/2523462028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2edc06b040 0x7f2edc14ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.943+0000 7f2ee2200640 1 -- 172.21.15.67:0/2523462028 >> 172.21.15.67:0/2523462028 conn(0x7f2edc0c8050 msgr2=0x7f2edc0c1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.944+0000 7f2ee2200640 1 -- 172.21.15.67:0/2523462028 shutdown_connections 2024-03-09T09:09:51.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:09:51.944+0000 7f2ee2200640 1 -- 172.21.15.67:0/2523462028 wait complete. 2024-03-09T09:09:51.969 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:09:52.111 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:09:52.111 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 41 -eq 48 ']' 2024-03-09T09:09:52.111 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.41 2024-03-09T09:09:52.112 INFO:tasks.workunit.client.1.smithi134.stdout:v0.41 2024-03-09T09:09:52.112 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.41 2024-03-09T09:09:52.112 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.41 2024-03-09T09:09:59.335 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 36% (62/171) Updating files: 37% (64/171) Updating files: 38% (65/171) Updating files: 39% (67/171) Updating files: 40% (69/171) Updating files: 41% (71/171) Updating files: 42% (72/171) Updating files: 43% (74/171) Updating files: 44% (76/171) Updating files: 45% (77/171) Updating files: 46% (79/171) Updating files: 47% (81/171) Updating files: 48% (83/171) Updating files: 49% (84/171) Updating files: 50% (86/171) Updating files: 51% (88/171) Updating files: 52% (89/171) Updating files: 53% (91/171) Updating files: 54% (93/171) Updating files: 55% (95/171) Updating files: 56% (96/171) Updating files: 57% (98/171) Updating files: 58% (100/171) Updating files: 59% (101/171) Updating files: 60% (103/171) Updating files: 61% (105/171) Updating files: 62% (107/171) Updating files: 63% (108/171) Updating files: 64% (110/171) Updating files: 65% (112/171) Updating files: 66% (113/171) Updating files: 67% (115/171) Updating files: 68% (117/171) Updating files: 69% (118/171) Updating files: 70% (120/171) Updating files: 71% (122/171) Updating files: 72% (124/171) Updating files: 73% (125/171) Updating files: 74% (127/171) Updating files: 75% (129/171) Updating files: 76% (130/171) Updating files: 77% (132/171) Updating files: 78% (134/171) Updating files: 79% (136/171) Updating files: 80% (137/171) Updating files: 81% (139/171) Updating files: 81% (140/171) Updating files: 82% (141/171) Updating files: 83% (142/171) Updating files: 84% (144/171) Updating files: 85% (146/171) Updating files: 86% (148/171) Updating files: 87% (149/171) Updating files: 88% (151/171) Updating files: 89% (153/171) Updating files: 90% (154/171) Updating files: 91% (156/171) Updating files: 92% (158/171) Updating files: 93% (160/171) Updating files: 94% (161/171) Updating files: 95% (163/171) Updating files: 96% (165/171) Updating files: 97% (166/171) Updating files: 98% (168/171) Updating files: 99% (170/171) Updating files: 100% (171/171) Updating files: 100% (171/171), done. 2024-03-09T09:09:59.571 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at b600ec2ac7c v0.38 2024-03-09T09:09:59.574 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.38 2024-03-09T09:09:59.623 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:09:59.623 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 39 -eq 48 ']' 2024-03-09T09:09:59.624 INFO:tasks.workunit.client.0.smithi067.stdout:v0.39 2024-03-09T09:09:59.624 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.39 2024-03-09T09:09:59.624 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.39 2024-03-09T09:09:59.624 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.39 2024-03-09T09:10:01.970 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:10:01.971 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:10:02.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.243+0000 7f2871800640 1 Processor -- start 2024-03-09T09:10:02.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.243+0000 7f2871800640 1 -- start start 2024-03-09T09:10:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.245+0000 7f2871800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f286c06d420 0x7f286c1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.245+0000 7f2871800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f286c14c9f0 0x7f286c1540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.245+0000 7f2871800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286c06d920 con 0x7f286c0c3080 2024-03-09T09:10:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.245+0000 7f2871800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286c155ba0 con 0x7f286c06d420 2024-03-09T09:10:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.245+0000 7f2871800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286c06dc00 con 0x7f286c14c9f0 2024-03-09T09:10:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.245+0000 7f286b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f286c06d420 0x7f286c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.245+0000 7f286be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f286c0c3080 0x7f286c14c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43412/0 (socket says 172.21.15.67:43412) 2024-03-09T09:10:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.245+0000 7f286be00640 1 -- 172.21.15.67:0/4008655925 learned_addr learned my addr 172.21.15.67:0/4008655925 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:10:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.246+0000 7f2870e00640 1 --2- 172.21.15.67:0/4008655925 >> v2:172.21.15.134:3300/0 conn(0x7f286c14c9f0 0x7f286c1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.246+0000 7f286b400640 1 -- 172.21.15.67:0/4008655925 >> v2:172.21.15.134:3300/0 conn(0x7f286c14c9f0 msgr2=0x7f286c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:10:02.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.246+0000 7f286b400640 1 --2- 172.21.15.67:0/4008655925 >> v2:172.21.15.134:3300/0 conn(0x7f286c14c9f0 0x7f286c1540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.246+0000 7f286b400640 1 -- 172.21.15.67:0/4008655925 >> v1:172.21.15.67:6789/0 conn(0x7f286c0c3080 legacy=0x7f286c14c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.246+0000 7f286b400640 1 -- 172.21.15.67:0/4008655925 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c06dee0 con 0x7f286c06d420 2024-03-09T09:10:02.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.246+0000 7f2870e00640 1 --2- 172.21.15.67:0/4008655925 >> v2:172.21.15.134:3300/0 conn(0x7f286c14c9f0 0x7f286c1540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:10:02.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.247+0000 7f286b400640 1 --2- 172.21.15.67:0/4008655925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286c06d420 0x7f286c1527d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f286c0712d0 tx=0x7f285802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdb5798ab3dbd2d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.247+0000 7f286aa00640 1 -- 172.21.15.67:0/4008655925 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28580315a0 con 0x7f286c06d420 2024-03-09T09:10:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.248+0000 7f286aa00640 1 -- 172.21.15.67:0/4008655925 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2858037070 con 0x7f286c06d420 2024-03-09T09:10:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.248+0000 7f286aa00640 1 -- 172.21.15.67:0/4008655925 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858031960 con 0x7f286c06d420 2024-03-09T09:10:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.248+0000 7f2871800640 1 -- 172.21.15.67:0/4008655925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286c06d420 msgr2=0x7f286c1527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.248+0000 7f2871800640 1 --2- 172.21.15.67:0/4008655925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286c06d420 0x7f286c1527d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f286c0712d0 tx=0x7f285802b840 comp rx=0 tx=0).stop 2024-03-09T09:10:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.248+0000 7f2871800640 1 -- 172.21.15.67:0/4008655925 shutdown_connections 2024-03-09T09:10:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.248+0000 7f2871800640 1 --2- 172.21.15.67:0/4008655925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286c06d420 0x7f286c1527d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.248+0000 7f2871800640 1 --2- 172.21.15.67:0/4008655925 >> v2:172.21.15.134:3300/0 conn(0x7f286c14c9f0 0x7f286c1540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.248+0000 7f2871800640 1 -- 172.21.15.67:0/4008655925 >> 172.21.15.67:0/4008655925 conn(0x7f286c0c7300 msgr2=0x7f286c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.248+0000 7f2871800640 1 -- 172.21.15.67:0/4008655925 shutdown_connections 2024-03-09T09:10:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.248+0000 7f2871800640 1 -- 172.21.15.67:0/4008655925 wait complete. 2024-03-09T09:10:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.251+0000 7f2871800640 1 Processor -- start 2024-03-09T09:10:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.251+0000 7f2871800640 1 -- start start 2024-03-09T09:10:02.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2871800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f286c06d420 0x7f286c0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2871800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f286c0c3080 0x7f286c143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2871800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286c14c9f0 0x7f286c143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2871800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c0c8030 con 0x7f286c14c9f0 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2871800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c0c3460 con 0x7f286c0c3080 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2871800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286c0ac450 con 0x7f286c06d420 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2870e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286c14c9f0 0x7f286c143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2870e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286c14c9f0 0x7f286c143f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45318/0 (socket says 172.21.15.67:45318) 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2870e00640 1 -- 172.21.15.67:0/954715660 learned_addr learned my addr 172.21.15.67:0/954715660 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f286b400640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f286c0c3080 0x7f286c143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f286be00640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f286c06d420 0x7f286c0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2870e00640 1 -- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f286c06d420 msgr2=0x7f286c0730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2870e00640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f286c06d420 0x7f286c0730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2870e00640 1 -- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f286c0c3080 msgr2=0x7f286c143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2870e00640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f286c0c3080 0x7f286c143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f2870e00640 1 -- 172.21.15.67:0/954715660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c1444b0 con 0x7f286c14c9f0 2024-03-09T09:10:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.253+0000 7f286b400640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f286c0c3080 0x7f286c143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:10:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.254+0000 7f2870e00640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286c14c9f0 0x7f286c143f70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2864007e40 tx=0x7f286400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.254+0000 7f2863e00640 1 -- 172.21.15.67:0/954715660 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286401a070 con 0x7f286c14c9f0 2024-03-09T09:10:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.254+0000 7f2863e00640 1 -- 172.21.15.67:0/954715660 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2864003bb0 con 0x7f286c14c9f0 2024-03-09T09:10:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.254+0000 7f2863e00640 1 -- 172.21.15.67:0/954715660 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864004030 con 0x7f286c14c9f0 2024-03-09T09:10:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.254+0000 7f2871800640 1 -- 172.21.15.67:0/954715660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c0ac6a0 con 0x7f286c14c9f0 2024-03-09T09:10:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.254+0000 7f2871800640 1 -- 172.21.15.67:0/954715660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286c0abac0 con 0x7f286c14c9f0 2024-03-09T09:10:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.255+0000 7f2871800640 1 -- 172.21.15.67:0/954715660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286c04f3c0 con 0x7f286c14c9f0 2024-03-09T09:10:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.256+0000 7f2863e00640 1 -- 172.21.15.67:0/954715660 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f286401d070 con 0x7f286c14c9f0 2024-03-09T09:10:02.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.258+0000 7f2863e00640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f284c06b520 0x7f284c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.258+0000 7f2863e00640 1 -- 172.21.15.67:0/954715660 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f286408a850 con 0x7f286c14c9f0 2024-03-09T09:10:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.259+0000 7f286be00640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f284c06b520 0x7f284c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.260+0000 7f286be00640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f284c06b520 0x7f284c06d9e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f285c000f60 tx=0x7f285c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:02.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.266+0000 7f2863e00640 1 -- 172.21.15.67:0/954715660 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2864010070 con 0x7f286c14c9f0 2024-03-09T09:10:02.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.568+0000 7f2871800640 1 -- 172.21.15.67:0/954715660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f286c0ab870 con 0x7f286c14c9f0 2024-03-09T09:10:02.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.569+0000 7f2863e00640 1 -- 172.21.15.67:0/954715660 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f2864059350 con 0x7f286c14c9f0 2024-03-09T09:10:02.573 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:10:02.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.571+0000 7f2871800640 1 -- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f284c06b520 msgr2=0x7f284c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.571+0000 7f2871800640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f284c06b520 0x7f284c06d9e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f285c000f60 tx=0x7f285c009040 comp rx=0 tx=0).stop 2024-03-09T09:10:02.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.571+0000 7f2871800640 1 -- 172.21.15.67:0/954715660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286c14c9f0 msgr2=0x7f286c143f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.571+0000 7f2871800640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286c14c9f0 0x7f286c143f70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2864007e40 tx=0x7f286400d4a0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.572+0000 7f2871800640 1 -- 172.21.15.67:0/954715660 shutdown_connections 2024-03-09T09:10:02.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.572+0000 7f2871800640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f284c06b520 0x7f284c06d9e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.572+0000 7f2871800640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f286c14c9f0 0x7f286c143f70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.572+0000 7f2871800640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f286c0c3080 0x7f286c143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.572+0000 7f2871800640 1 --2- 172.21.15.67:0/954715660 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f286c06d420 0x7f286c0730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.572+0000 7f2871800640 1 -- 172.21.15.67:0/954715660 >> 172.21.15.67:0/954715660 conn(0x7f286c0c7300 msgr2=0x7f286c0c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:02.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.572+0000 7f2871800640 1 -- 172.21.15.67:0/954715660 shutdown_connections 2024-03-09T09:10:02.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.572+0000 7f2871800640 1 -- 172.21.15.67:0/954715660 wait complete. 2024-03-09T09:10:02.595 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:10:02.595 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:10:02.595 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:10:02.595 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:10:02.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.795+0000 7ff07ae00640 1 Processor -- start 2024-03-09T09:10:02.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.795+0000 7ff07ae00640 1 -- start start 2024-03-09T09:10:02.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.796+0000 7ff07ae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0740c5e00 0x7ff0740acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.796+0000 7ff07ae00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff0740c9380 0x7ff074156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.796+0000 7ff07ae00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0740c9810 con 0x7ff0740c61e0 2024-03-09T09:10:02.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.796+0000 7ff07ae00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07406d740 con 0x7ff0740c9380 2024-03-09T09:10:02.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.796+0000 7ff07ae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740c6660 con 0x7ff0740c5e00 2024-03-09T09:10:02.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.797+0000 7ff079000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff0740c61e0 0x7ff07406c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43424/0 (socket says 172.21.15.67:43424) 2024-03-09T09:10:02.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.797+0000 7ff079000640 1 -- 172.21.15.67:0/3537177768 learned_addr learned my addr 172.21.15.67:0/3537177768 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:10:02.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.797+0000 7ff07a400640 1 --2- 172.21.15.67:0/3537177768 >> v2:172.21.15.134:3300/0 conn(0x7ff0740c9380 0x7ff074156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.797+0000 7ff079a00640 1 --2- 172.21.15.67:0/3537177768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0740c5e00 0x7ff0740acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.797+0000 7ff073e00640 1 -- 172.21.15.67:0/3537177768 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347635270 0 0) 0x7ff0740c9810 con 0x7ff0740c61e0 2024-03-09T09:10:02.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.797+0000 7ff073e00640 1 -- 172.21.15.67:0/3537177768 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff05c003520 con 0x7ff0740c61e0 2024-03-09T09:10:02.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.798+0000 7ff07a400640 1 -- 172.21.15.67:0/3537177768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0740c5e00 msgr2=0x7ff0740acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:10:02.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.798+0000 7ff07a400640 1 --2- 172.21.15.67:0/3537177768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0740c5e00 0x7ff0740acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.798+0000 7ff07a400640 1 -- 172.21.15.67:0/3537177768 >> v1:172.21.15.67:6789/0 conn(0x7ff0740c61e0 legacy=0x7ff07406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.798+0000 7ff07a400640 1 -- 172.21.15.67:0/3537177768 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0740ad7c0 con 0x7ff0740c9380 2024-03-09T09:10:02.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.799+0000 7ff07a400640 1 --2- 172.21.15.67:0/3537177768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0740c9380 0x7ff074156660 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0097a0 tx=0x7ff06c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9d7d7c94bd39aa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:02.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.799+0000 7ff073e00640 1 -- 172.21.15.67:0/3537177768 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c035020 con 0x7ff0740c9380 2024-03-09T09:10:02.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.799+0000 7ff073e00640 1 -- 172.21.15.67:0/3537177768 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c031810 con 0x7ff0740c9380 2024-03-09T09:10:02.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.799+0000 7ff073e00640 1 -- 172.21.15.67:0/3537177768 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c031bc0 con 0x7ff0740c9380 2024-03-09T09:10:02.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.800+0000 7ff07ae00640 1 -- 172.21.15.67:0/3537177768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0740c9380 msgr2=0x7ff074156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.800+0000 7ff07ae00640 1 --2- 172.21.15.67:0/3537177768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0740c9380 0x7ff074156660 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0097a0 tx=0x7ff06c02b8b0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.800+0000 7ff07ae00640 1 -- 172.21.15.67:0/3537177768 shutdown_connections 2024-03-09T09:10:02.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.800+0000 7ff07ae00640 1 --2- 172.21.15.67:0/3537177768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0740c9380 0x7ff074156660 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.800+0000 7ff07ae00640 1 --2- 172.21.15.67:0/3537177768 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0740c5e00 0x7ff0740acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.800+0000 7ff07ae00640 1 -- 172.21.15.67:0/3537177768 >> 172.21.15.67:0/3537177768 conn(0x7ff0740cb480 msgr2=0x7ff0740cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:02.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.800+0000 7ff07ae00640 1 -- 172.21.15.67:0/3537177768 shutdown_connections 2024-03-09T09:10:02.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.800+0000 7ff07ae00640 1 -- 172.21.15.67:0/3537177768 wait complete. 2024-03-09T09:10:02.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.802+0000 7ff07ae00640 1 Processor -- start 2024-03-09T09:10:02.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.802+0000 7ff07ae00640 1 -- start start 2024-03-09T09:10:02.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.803+0000 7ff07ae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c5e00 0x7ff0741563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.803+0000 7ff07ae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0740c61e0 0x7ff0740c1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.803+0000 7ff07ae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0740c9380 0x7ff0740c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.803+0000 7ff07ae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07406bb00 con 0x7ff0740c5e00 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.803+0000 7ff07ae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff074153680 con 0x7ff0740c61e0 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.803+0000 7ff07ae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07406bde0 con 0x7ff0740c9380 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.803+0000 7ff079a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c5e00 0x7ff0741563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.803+0000 7ff079a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c5e00 0x7ff0741563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45334/0 (socket says 172.21.15.67:45334) 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.803+0000 7ff079a00640 1 -- 172.21.15.67:0/3830965938 learned_addr learned my addr 172.21.15.67:0/3830965938 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.803+0000 7ff079000640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0740c61e0 0x7ff0740c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.803+0000 7ff07a400640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0740c9380 0x7ff0740c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.804+0000 7ff079a00640 1 -- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0740c9380 msgr2=0x7ff0740c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.804+0000 7ff079a00640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0740c9380 0x7ff0740c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.804+0000 7ff079a00640 1 -- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0740c61e0 msgr2=0x7ff0740c1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.804+0000 7ff079a00640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0740c61e0 0x7ff0740c1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.804+0000 7ff079a00640 1 -- 172.21.15.67:0/3830965938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0741eb0b0 con 0x7ff0740c5e00 2024-03-09T09:10:02.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.804+0000 7ff07a400640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0740c9380 0x7ff0740c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:10:02.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.804+0000 7ff079a00640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c5e00 0x7ff0741563e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff06800efc0 tx=0x7ff06800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:02.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.804+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068018020 con 0x7ff0740c5e00 2024-03-09T09:10:02.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.804+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068014480 con 0x7ff0740c5e00 2024-03-09T09:10:02.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.804+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068014850 con 0x7ff0740c5e00 2024-03-09T09:10:02.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.804+0000 7ff07ae00640 1 -- 172.21.15.67:0/3830965938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff074156920 con 0x7ff0740c5e00 2024-03-09T09:10:02.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.805+0000 7ff07ae00640 1 -- 172.21.15.67:0/3830965938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0741eb630 con 0x7ff0740c5e00 2024-03-09T09:10:02.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.805+0000 7ff07ae00640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c2ed0 0x7ff0740c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.806+0000 7ff07ae00640 1 -- 172.21.15.67:0/3830965938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740c51f0 con 0x7ff0740c2ed0 2024-03-09T09:10:02.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.806+0000 7ff079000640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c2ed0 0x7ff0740c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.806+0000 7ff079000640 1 -- 172.21.15.67:0/3830965938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff07404f3c0 con 0x7ff0740c2ed0 2024-03-09T09:10:02.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.807+0000 7ff079000640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c2ed0 0x7ff0740c4cb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff0740c5a10 tx=0x7ff06000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:02.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.807+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060005ac0 con 0x7ff0740c2ed0 2024-03-09T09:10:02.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.807+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff068005d10 con 0x7ff0740c5e00 2024-03-09T09:10:02.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.808+0000 7ff071600640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff05406b400 0x7ff05406d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.808+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff06808b0d0 con 0x7ff0740c5e00 2024-03-09T09:10:02.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.808+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff060018070 con 0x7ff0740c2ed0 2024-03-09T09:10:02.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.808+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c2ed0 msgr2=0x7ff0740c4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.808+0000 7ff071600640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c2ed0 0x7ff0740c4cb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff0740c5a10 tx=0x7ff06000e940 comp rx=0 tx=0).stop 2024-03-09T09:10:02.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.808+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c2ed0 msgr2=0x7ff0740c4cb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:10:02.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.808+0000 7ff071600640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c2ed0 0x7ff0740c4cb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.809+0000 7ff07a400640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff05406b400 0x7ff05406d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.810+0000 7ff07a400640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff05406b400 0x7ff05406d8c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff06c036000 tx=0x7ff06c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:02.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.815+0000 7ff07ae00640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff074151680 0x7ff074151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:02.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.816+0000 7ff07ae00640 1 -- 172.21.15.67:0/3830965938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07406d2f0 con 0x7ff074151680 2024-03-09T09:10:02.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.816+0000 7ff079000640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff074151680 0x7ff074151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:02.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.816+0000 7ff079000640 1 -- 172.21.15.67:0/3830965938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff0740aafa0 con 0x7ff074151680 2024-03-09T09:10:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.817+0000 7ff079000640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff074151680 0x7ff074151a80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff074156af0 tx=0x7ff060000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.817+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06001dc90 con 0x7ff074151680 2024-03-09T09:10:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.817+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7ff06001dc90 con 0x7ff074151680 2024-03-09T09:10:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.817+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff074151680 msgr2=0x7ff074151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.817+0000 7ff071600640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff074151680 0x7ff074151a80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff074156af0 tx=0x7ff060000ed0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.817+0000 7ff071600640 1 -- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff074151680 msgr2=0x7ff074151a80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:10:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.817+0000 7ff071600640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff074151680 0x7ff074151a80 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff074156af0 tx=0x7ff060000ed0 comp rx=0 tx=0).stop 2024-03-09T09:10:02.822 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 124, 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:10:02.823 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:10:02.824 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.825 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:10:02.826 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:10:02.826 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:10:02.826 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:10:02.826 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:10:02.826 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:10:02.826 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:10:02.826 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:10:02.826 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:10:02.826 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:10:02.826 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:10:02.826 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:10:02.827 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:02.829 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:02.830 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:02.831 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:10:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:02.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:02.835 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:02.836 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:02.837 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:02.838 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:10:02.838 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:02.838 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:02.838 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:02.838 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:10:02.838 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:02.838 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:02.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:02.838 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:10:02.838 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:10:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.818+0000 7ff07ae00640 1 -- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff05406b400 msgr2=0x7ff05406d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.818+0000 7ff07ae00640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff05406b400 0x7ff05406d8c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff06c036000 tx=0x7ff06c03c040 comp rx=0 tx=0).stop 2024-03-09T09:10:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.818+0000 7ff07ae00640 1 -- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c5e00 msgr2=0x7ff0741563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.818+0000 7ff07ae00640 1 --2- 172.21.15.67:0/3830965938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0740c5e00 0x7ff0741563e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff06800efc0 tx=0x7ff06800c490 comp rx=0 tx=0).stop 2024-03-09T09:10:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.818+0000 7ff07a400640 1 -- 172.21.15.67:0/3830965938 reap_dead start 2024-03-09T09:10:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.818+0000 7ff07a400640 1 -- 172.21.15.67:0/3830965938 reap_dead start 2024-03-09T09:10:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.818+0000 7ff07ae00640 1 -- 172.21.15.67:0/3830965938 shutdown_connections 2024-03-09T09:10:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.818+0000 7ff07ae00640 1 -- 172.21.15.67:0/3830965938 >> 172.21.15.67:0/3830965938 conn(0x7ff0740cb480 msgr2=0x7ff074152b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.818+0000 7ff07ae00640 1 -- 172.21.15.67:0/3830965938 shutdown_connections 2024-03-09T09:10:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:02.818+0000 7ff07ae00640 1 -- 172.21.15.67:0/3830965938 wait complete. 2024-03-09T09:10:02.841 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:10:02.841 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:10:02.841 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:10:03.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.044+0000 7f807d400640 1 Processor -- start 2024-03-09T09:10:03.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.045+0000 7f807d400640 1 -- start start 2024-03-09T09:10:03.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.046+0000 7f807d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f80780c34c0 0x7f80780c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.046+0000 7f807d400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8078157910 0x7f8078157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.046+0000 7f807d400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80780c2810 con 0x7f807806ec60 2024-03-09T09:10:03.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.046+0000 7f807d400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807806fa20 con 0x7f80780c34c0 2024-03-09T09:10:03.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.046+0000 7f807d400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80780c7270 con 0x7f8078157910 2024-03-09T09:10:03.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.046+0000 7f8076800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f80780c34c0 0x7f80780c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.047+0000 7f8076800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f80780c34c0 0x7f80780c2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45350/0 (socket says 172.21.15.67:45350) 2024-03-09T09:10:03.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.047+0000 7f8076800640 1 -- 172.21.15.67:0/1404411709 learned_addr learned my addr 172.21.15.67:0/1404411709 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:03.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.047+0000 7f8077200640 1 --2- 172.21.15.67:0/1404411709 >> v2:172.21.15.134:3300/0 conn(0x7f8078157910 0x7f8078157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.047+0000 7f8076800640 1 -- 172.21.15.67:0/1404411709 >> v2:172.21.15.134:3300/0 conn(0x7f8078157910 msgr2=0x7f8078157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:10:03.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.047+0000 7f8076800640 1 --2- 172.21.15.67:0/1404411709 >> v2:172.21.15.134:3300/0 conn(0x7f8078157910 0x7f8078157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.047+0000 7f8076800640 1 -- 172.21.15.67:0/1404411709 >> v1:172.21.15.134:6789/0 conn(0x7f807806ec60 legacy=0x7f807806f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:10:03.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.047+0000 7f8076800640 1 -- 172.21.15.67:0/1404411709 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80780c74c0 con 0x7f80780c34c0 2024-03-09T09:10:03.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.049+0000 7f8076800640 1 --2- 172.21.15.67:0/1404411709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80780c34c0 0x7f80780c2af0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f80780ad080 tx=0x7f806802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5cf99f729d85c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:03.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.049+0000 7f8075400640 1 -- 172.21.15.67:0/1404411709 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806802e070 con 0x7f80780c34c0 2024-03-09T09:10:03.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.049+0000 7f8075400640 1 -- 172.21.15.67:0/1404411709 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8068037070 con 0x7f80780c34c0 2024-03-09T09:10:03.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.049+0000 7f8075400640 1 -- 172.21.15.67:0/1404411709 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8068031a00 con 0x7f80780c34c0 2024-03-09T09:10:03.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.050+0000 7f807d400640 1 -- 172.21.15.67:0/1404411709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80780c34c0 msgr2=0x7f80780c2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.050+0000 7f807d400640 1 --2- 172.21.15.67:0/1404411709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80780c34c0 0x7f80780c2af0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f80780ad080 tx=0x7f806802b880 comp rx=0 tx=0).stop 2024-03-09T09:10:03.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.050+0000 7f807d400640 1 -- 172.21.15.67:0/1404411709 shutdown_connections 2024-03-09T09:10:03.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.050+0000 7f807d400640 1 --2- 172.21.15.67:0/1404411709 >> v2:172.21.15.134:3300/0 conn(0x7f8078157910 0x7f8078157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.050+0000 7f807d400640 1 --2- 172.21.15.67:0/1404411709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80780c34c0 0x7f80780c2af0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.050+0000 7f807d400640 1 -- 172.21.15.67:0/1404411709 >> 172.21.15.67:0/1404411709 conn(0x7f80780c7a90 msgr2=0x7f80780c7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:03.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.050+0000 7f807d400640 1 -- 172.21.15.67:0/1404411709 shutdown_connections 2024-03-09T09:10:03.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.050+0000 7f807d400640 1 -- 172.21.15.67:0/1404411709 wait complete. 2024-03-09T09:10:03.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.052+0000 7f807d400640 1 Processor -- start 2024-03-09T09:10:03.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.052+0000 7f807d400640 1 -- start start 2024-03-09T09:10:03.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.053+0000 7f807d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f807806ec60 0x7f80780c1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.053+0000 7f807d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80780c34c0 0x7f8078154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.053+0000 7f807d400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8078157910 0x7f80781551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.053+0000 7f807d400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8078073cf0 con 0x7f80780c34c0 2024-03-09T09:10:03.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.053+0000 7f807d400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8078066d90 con 0x7f807806ec60 2024-03-09T09:10:03.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.053+0000 7f807d400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80780672b0 con 0x7f8078157910 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.053+0000 7f8075e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80780c34c0 0x7f8078154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.053+0000 7f8075e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80780c34c0 0x7f8078154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45364/0 (socket says 172.21.15.67:45364) 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.053+0000 7f8075e00640 1 -- 172.21.15.67:0/2780879092 learned_addr learned my addr 172.21.15.67:0/2780879092 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.053+0000 7f8076800640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f807806ec60 0x7f80780c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.053+0000 7f8077200640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8078157910 0x7f80781551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f8075e00640 1 -- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8078157910 msgr2=0x7f80781551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f8075e00640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8078157910 0x7f80781551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f8075e00640 1 -- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f807806ec60 msgr2=0x7f80780c1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f8075e00640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f807806ec60 0x7f80780c1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f8075e00640 1 -- 172.21.15.67:0/2780879092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80780c1df0 con 0x7f80780c34c0 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f8077200640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8078157910 0x7f80781551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:10:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f8075e00640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80780c34c0 0x7f8078154ca0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f806400b3f0 tx=0x7f806400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8064015020 con 0x7f80780c34c0 2024-03-09T09:10:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8064011830 con 0x7f80780c34c0 2024-03-09T09:10:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f807d400640 1 -- 172.21.15.67:0/2780879092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80781ec010 con 0x7f80780c34c0 2024-03-09T09:10:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8064011c20 con 0x7f80780c34c0 2024-03-09T09:10:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.054+0000 7f807d400640 1 -- 172.21.15.67:0/2780879092 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80781ec8b0 con 0x7f80780c34c0 2024-03-09T09:10:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.055+0000 7f807d400640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80780a6610 0x7f80781ef4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.055+0000 7f807d400640 1 -- 172.21.15.67:0/2780879092 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8078075a00 con 0x7f80780a6610 2024-03-09T09:10:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.056+0000 7f8076800640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80780a6610 0x7f80781ef4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.056+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8064018020 con 0x7f80780c34c0 2024-03-09T09:10:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.057+0000 7f8076800640 1 -- 172.21.15.67:0/2780879092 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80781ef240 con 0x7f80780a6610 2024-03-09T09:10:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.057+0000 7f806ea00640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f805806b4d0 0x7f805806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.057+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f806408a220 con 0x7f80780c34c0 2024-03-09T09:10:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.057+0000 7f8076800640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80780a6610 0x7f80781ef4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f80780c8b30 tx=0x7f8068004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.058+0000 7f8077200640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f805806b4d0 0x7f805806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.058+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806804a070 con 0x7f80780a6610 2024-03-09T09:10:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.059+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f8068037040 con 0x7f80780a6610 2024-03-09T09:10:03.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.059+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80780a6610 msgr2=0x7f80781ef4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.059+0000 7f806ea00640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80780a6610 0x7f80781ef4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f80780c8b30 tx=0x7f8068004730 comp rx=0 tx=0).stop 2024-03-09T09:10:03.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.059+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80780a6610 msgr2=0x7f80781ef4e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:10:03.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.059+0000 7f806ea00640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80780a6610 0x7f80781ef4e0 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f80780c8b30 tx=0x7f8068004730 comp rx=0 tx=0).stop 2024-03-09T09:10:03.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.059+0000 7f8077200640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f805806b4d0 0x7f805806d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8070009f70 tx=0x7f807000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:03.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.068+0000 7f807d400640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80781efa20 0x7f80781efe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.068+0000 7f807d400640 1 -- 172.21.15.67:0/2780879092 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80780689d0 con 0x7f80781efa20 2024-03-09T09:10:03.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.068+0000 7f8076800640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80781efa20 0x7f80781efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.069+0000 7f8076800640 1 -- 172.21.15.67:0/2780879092 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8078155720 con 0x7f80781efa20 2024-03-09T09:10:03.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.070+0000 7f8076800640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80781efa20 0x7f80781efe30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8078075b80 tx=0x7f80680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:03.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.070+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8068050030 con 0x7f80781efa20 2024-03-09T09:10:03.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f8068003c90 con 0x7f80781efa20 2024-03-09T09:10:03.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80781efa20 msgr2=0x7f80781efe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f806ea00640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80781efa20 0x7f80781efe30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8078075b80 tx=0x7f80680046c0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f806ea00640 1 -- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80781efa20 msgr2=0x7f80781efe30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:10:03.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f806ea00640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80781efa20 0x7f80781efe30 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8078075b80 tx=0x7f80680046c0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.076 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:10:03.076 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:10:03.076 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 124, 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:03.077 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:03.078 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:10:03.079 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:10:03.080 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:03.081 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.084 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:03.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:10:03.086 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:03.087 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:10:03.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.088 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:03.088 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:03.088 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:10:03.088 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.088 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.088 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:10:03.088 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:10:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f807d400640 1 -- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f805806b4d0 msgr2=0x7f805806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f807d400640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f805806b4d0 0x7f805806d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8070009f70 tx=0x7f807000b040 comp rx=0 tx=0).stop 2024-03-09T09:10:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f807d400640 1 -- 172.21.15.67:0/2780879092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80780c34c0 msgr2=0x7f8078154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f807d400640 1 --2- 172.21.15.67:0/2780879092 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80780c34c0 0x7f8078154ca0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f806400b3f0 tx=0x7f806400b8b0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f8077200640 1 -- 172.21.15.67:0/2780879092 reap_dead start 2024-03-09T09:10:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f8077200640 1 -- 172.21.15.67:0/2780879092 reap_dead start 2024-03-09T09:10:03.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f807d400640 1 -- 172.21.15.67:0/2780879092 shutdown_connections 2024-03-09T09:10:03.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f807d400640 1 -- 172.21.15.67:0/2780879092 >> 172.21.15.67:0/2780879092 conn(0x7f80780c7a90 msgr2=0x7f80781570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:03.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f807d400640 1 -- 172.21.15.67:0/2780879092 shutdown_connections 2024-03-09T09:10:03.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.071+0000 7f807d400640 1 -- 172.21.15.67:0/2780879092 wait complete. 2024-03-09T09:10:03.091 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:10:03.091 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:10:03.091 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:10:03.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.399+0000 7faee7200640 1 Processor -- start 2024-03-09T09:10:03.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.400+0000 7faee7200640 1 -- start start 2024-03-09T09:10:03.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.401+0000 7faee7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee80ac930 0x7faee80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.401+0000 7faee7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faee8151360 0x7faee81539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.401+0000 7faee7200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee80c2780 con 0x7faee80c30a0 2024-03-09T09:10:03.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.401+0000 7faee7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee8070670 con 0x7faee8151360 2024-03-09T09:10:03.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.401+0000 7faee7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee80708c0 con 0x7faee80ac930 2024-03-09T09:10:03.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.401+0000 7faee6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faee8151360 0x7faee81539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.401+0000 7faee6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faee8151360 0x7faee81539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45370/0 (socket says 172.21.15.67:45370) 2024-03-09T09:10:03.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.401+0000 7faee6800640 1 -- 172.21.15.67:0/2053860882 learned_addr learned my addr 172.21.15.67:0/2053860882 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:03.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.401+0000 7faee5e00640 1 --2- 172.21.15.67:0/2053860882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee80ac930 0x7faee80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.402+0000 7faee6800640 1 -- 172.21.15.67:0/2053860882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee80ac930 msgr2=0x7faee80c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:10:03.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.402+0000 7faee6800640 1 --2- 172.21.15.67:0/2053860882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee80ac930 0x7faee80c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.402+0000 7faee6800640 1 -- 172.21.15.67:0/2053860882 >> v1:172.21.15.134:6789/0 conn(0x7faee80c30a0 legacy=0x7faee8150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:10:03.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.402+0000 7faee6800640 1 -- 172.21.15.67:0/2053860882 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee8070ba0 con 0x7faee8151360 2024-03-09T09:10:03.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.404+0000 7faee6800640 1 --2- 172.21.15.67:0/2053860882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee8151360 0x7faee81539e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faee0009e50 tx=0x7faee002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0def65498ba469 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.404+0000 7faee4a00640 1 -- 172.21.15.67:0/2053860882 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee002ca70 con 0x7faee8151360 2024-03-09T09:10:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.404+0000 7faee4a00640 1 -- 172.21.15.67:0/2053860882 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee002cc10 con 0x7faee8151360 2024-03-09T09:10:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.404+0000 7faee4a00640 1 -- 172.21.15.67:0/2053860882 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee00343f0 con 0x7faee8151360 2024-03-09T09:10:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.404+0000 7faee7200640 1 -- 172.21.15.67:0/2053860882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee8151360 msgr2=0x7faee81539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.404+0000 7faee7200640 1 --2- 172.21.15.67:0/2053860882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee8151360 0x7faee81539e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faee0009e50 tx=0x7faee002bed0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.405+0000 7faee7200640 1 -- 172.21.15.67:0/2053860882 shutdown_connections 2024-03-09T09:10:03.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.405+0000 7faee7200640 1 --2- 172.21.15.67:0/2053860882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee8151360 0x7faee81539e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.405+0000 7faee7200640 1 --2- 172.21.15.67:0/2053860882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee80ac930 0x7faee80c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.405+0000 7faee7200640 1 -- 172.21.15.67:0/2053860882 >> 172.21.15.67:0/2053860882 conn(0x7faee80c7220 msgr2=0x7faee80c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:03.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.405+0000 7faee7200640 1 -- 172.21.15.67:0/2053860882 shutdown_connections 2024-03-09T09:10:03.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.405+0000 7faee7200640 1 -- 172.21.15.67:0/2053860882 wait complete. 2024-03-09T09:10:03.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.407+0000 7faee7200640 1 Processor -- start 2024-03-09T09:10:03.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.407+0000 7faee7200640 1 -- start start 2024-03-09T09:10:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.408+0000 7faee7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee80ac930 0x7faee8148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.408+0000 7faee7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80c30a0 0x7faee81eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.408+0000 7faee7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee8151360 0x7faee80c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.408+0000 7faee7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee80ace90 con 0x7faee80c30a0 2024-03-09T09:10:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.408+0000 7faee7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee80acd10 con 0x7faee8151360 2024-03-09T09:10:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.408+0000 7faee7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee8153f20 con 0x7faee80ac930 2024-03-09T09:10:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.409+0000 7faee5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80c30a0 0x7faee81eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.409+0000 7faee5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80c30a0 0x7faee81eeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45376/0 (socket says 172.21.15.67:45376) 2024-03-09T09:10:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.409+0000 7faee5400640 1 -- 172.21.15.67:0/3293723582 learned_addr learned my addr 172.21.15.67:0/3293723582 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.409+0000 7faee5e00640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee80ac930 0x7faee8148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.409+0000 7faee6800640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee8151360 0x7faee80c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.409+0000 7faee5400640 1 -- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee80ac930 msgr2=0x7faee8148b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.409+0000 7faee5400640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee80ac930 0x7faee8148b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.409+0000 7faee5400640 1 -- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee8151360 msgr2=0x7faee80c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.409+0000 7faee5400640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee8151360 0x7faee80c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.409+0000 7faee5400640 1 -- 172.21.15.67:0/3293723582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee814e100 con 0x7faee80c30a0 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.409+0000 7faee5e00640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee80ac930 0x7faee8148b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.410+0000 7faee5400640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80c30a0 0x7faee81eeff0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faed400c430 tx=0x7faed400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.410+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4011790 con 0x7faee80c30a0 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.410+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed4017070 con 0x7faee80c30a0 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.410+0000 7faee7200640 1 -- 172.21.15.67:0/3293723582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee814d8a0 con 0x7faee80c30a0 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.410+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4011b70 con 0x7faee80c30a0 2024-03-09T09:10:03.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.410+0000 7faee7200640 1 -- 172.21.15.67:0/3293723582 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee81ef530 con 0x7faee80c30a0 2024-03-09T09:10:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.411+0000 7faee7200640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee814c650 0x7faee80c1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.411+0000 7faee7200640 1 -- 172.21.15.67:0/3293723582 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee81490a0 con 0x7faee814c650 2024-03-09T09:10:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.412+0000 7faee5e00640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee814c650 0x7faee80c1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.412+0000 7faee5e00640 1 -- 172.21.15.67:0/3293723582 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faee804f3c0 con 0x7faee814c650 2024-03-09T09:10:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.413+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faed4011d10 con 0x7faee80c30a0 2024-03-09T09:10:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.413+0000 7faee5e00640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee814c650 0x7faee80c1aa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faed8011830 tx=0x7faed8011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:03.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.415+0000 7faede000640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faec806b500 0x7faec806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.415+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7faed408a4a0 con 0x7faee80c30a0 2024-03-09T09:10:03.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.415+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed800fa30 con 0x7faee814c650 2024-03-09T09:10:03.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.415+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7faed800fbd0 con 0x7faee814c650 2024-03-09T09:10:03.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.416+0000 7faee6800640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faec806b500 0x7faec806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.416+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee814c650 msgr2=0x7faee80c1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.416+0000 7faede000640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee814c650 0x7faee80c1aa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faed8011830 tx=0x7faed8011cf0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.416+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee814c650 msgr2=0x7faee80c1aa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:10:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.416+0000 7faede000640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee814c650 0x7faee80c1aa0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faed8011830 tx=0x7faed8011cf0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.417+0000 7faee6800640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faec806b500 0x7faec806d9c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faee0035000 tx=0x7faee0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:03.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.423+0000 7faee7200640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee8146450 0x7faee8146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:03.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.423+0000 7faee7200640 1 -- 172.21.15.67:0/3293723582 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee80c6be0 con 0x7faee8146450 2024-03-09T09:10:03.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.423+0000 7faee5e00640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee8146450 0x7faee8146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.424+0000 7faee5e00640 1 -- 172.21.15.67:0/3293723582 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faee80c0e00 con 0x7faee8146450 2024-03-09T09:10:03.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.424+0000 7faee5e00640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee8146450 0x7faee8146860 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faed8011830 tx=0x7faed80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:03.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.425+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8006680 con 0x7faee8146450 2024-03-09T09:10:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.425+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7faed8006680 con 0x7faee8146450 2024-03-09T09:10:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.425+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee8146450 msgr2=0x7faee8146860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.425+0000 7faede000640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee8146450 0x7faee8146860 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faed8011830 tx=0x7faed80046c0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.425+0000 7faede000640 1 -- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee8146450 msgr2=0x7faee8146860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:10:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.425+0000 7faede000640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee8146450 0x7faee8146860 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faed8011830 tx=0x7faed80046c0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.432 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:10:03.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:10:03.432 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 124, 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:10:03.433 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:03.434 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:03.434 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:03.434 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:03.434 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:03.434 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:03.434 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:03.434 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:03.434 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:03.434 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:03.434 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:03.434 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:03.435 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.436 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:10:03.436 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:10:03.436 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:10:03.436 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:10:03.437 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:10:03.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:03.441 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:10:03.441 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.441 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.441 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.441 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:10:03.441 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:10:03.441 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:03.441 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:03.441 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:03.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.442 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:10:03.442 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:10:03.442 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:03.442 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:10:03.443 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:03.446 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:03.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:03.447 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:10:03.447 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:10:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.426+0000 7faee7200640 1 -- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faec806b500 msgr2=0x7faec806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.426+0000 7faee7200640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faec806b500 0x7faec806d9c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faee0035000 tx=0x7faee0030000 comp rx=0 tx=0).stop 2024-03-09T09:10:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.426+0000 7faee7200640 1 -- 172.21.15.67:0/3293723582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80c30a0 msgr2=0x7faee81eeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.426+0000 7faee7200640 1 --2- 172.21.15.67:0/3293723582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faee80c30a0 0x7faee81eeff0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faed400c430 tx=0x7faed400c8f0 comp rx=0 tx=0).stop 2024-03-09T09:10:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.426+0000 7faee6800640 1 -- 172.21.15.67:0/3293723582 reap_dead start 2024-03-09T09:10:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.426+0000 7faee6800640 1 -- 172.21.15.67:0/3293723582 reap_dead start 2024-03-09T09:10:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.426+0000 7faee7200640 1 -- 172.21.15.67:0/3293723582 shutdown_connections 2024-03-09T09:10:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.426+0000 7faee7200640 1 -- 172.21.15.67:0/3293723582 >> 172.21.15.67:0/3293723582 conn(0x7faee80c7220 msgr2=0x7faee81542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.426+0000 7faee7200640 1 -- 172.21.15.67:0/3293723582 shutdown_connections 2024-03-09T09:10:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:03.427+0000 7faee7200640 1 -- 172.21.15.67:0/3293723582 wait complete. 2024-03-09T09:10:03.449 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:10:03.449 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-03-09T09:10:03.449 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a'] 2024-03-09T09:10:03.449 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-03-09T09:10:03.450 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:10:03.450 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-09T09:10:03.450 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:10:03.480 INFO:tasks.ceph.mon.b:Stopped 2024-03-09T09:10:03.480 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-03-09T09:10:03.481 INFO:tasks.ceph.mon.a:Sent signal 19 2024-03-09T09:10:03.481 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-03-09T09:10:04.873 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 85% (74/87) Updating files: 86% (75/87) Updating files: 87% (76/87) Updating files: 88% (77/87) Updating files: 89% (78/87) Updating files: 90% (79/87) Updating files: 91% (80/87) Updating files: 93% (81/87) Updating files: 94% (82/87) Updating files: 95% (83/87) Updating files: 96% (84/87) Updating files: 97% (85/87) Updating files: 98% (86/87) Updating files: 100% (87/87) Updating files: 100% (87/87), done. 2024-03-09T09:10:04.958 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at c1345f7136a v0.41 2024-03-09T09:10:04.967 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.41 2024-03-09T09:10:05.026 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 24% (63/258) Updating files: 25% (65/258) Updating files: 26% (68/258) Updating files: 27% (70/258) Updating files: 28% (73/258) Updating files: 29% (75/258) Updating files: 30% (78/258) Updating files: 31% (80/258) Updating files: 32% (83/258) Updating files: 33% (86/258) Updating files: 34% (88/258) Updating files: 35% (91/258) Updating files: 36% (93/258) Updating files: 37% (96/258) Updating files: 38% (99/258) Updating files: 39% (101/258) Updating files: 40% (104/258) Updating files: 41% (106/258) Updating files: 42% (109/258) Updating files: 43% (111/258) Updating files: 44% (114/258) Updating files: 45% (117/258) Updating files: 46% (119/258) Updating files: 46% (121/258) Updating files: 47% (122/258) Updating files: 48% (124/258) Updating files: 49% (127/258) Updating files: 50% (129/258) Updating files: 51% (132/258) Updating files: 52% (135/258) Updating files: 53% (137/258) Updating files: 54% (140/258) Updating files: 55% (142/258) Updating files: 56% (145/258) Updating files: 57% (148/258) Updating files: 58% (150/258) Updating files: 59% (153/258) Updating files: 60% (155/258) Updating files: 61% (158/258) Updating files: 62% (160/258) Updating files: 63% (163/258) Updating files: 64% (166/258) Updating files: 65% (168/258) Updating files: 66% (171/258) Updating files: 66% (172/258) Updating files: 67% (173/258) Updating files: 68% (176/258) Updating files: 69% (179/258) Updating files: 70% (181/258) Updating files: 71% (184/258) Updating files: 72% (186/258) Updating files: 73% (189/258) Updating files: 74% (191/258) Updating files: 75% (194/258) Updating files: 76% (197/258) Updating files: 77% (199/258) Updating files: 78% (202/258) Updating files: 78% (203/258) Updating files: 79% (204/258) Updating files: 80% (207/258) Updating files: 81% (209/258) Updating files: 82% (212/258) Updating files: 83% (215/258) Updating files: 84% (217/258) Updating files: 85% (220/258) Updating files: 86% (222/258) Updating files: 87% (225/258) Updating files: 88% (228/258) Updating files: 88% (229/258) Updating files: 89% (230/258) Updating files: 90% (233/258) Updating files: 91% (235/258) Updating files: 92% (238/258) Updating files: 93% (240/258) Updating files: 94% (243/258) Updating files: 95% (246/258) Updating files: 96% (248/258) Updating files: 97% (251/258) Updating files: 98% (253/258) Updating files: 99% (256/258) Updating files: 100% (258/258) Updating files: 100% (258/258), done. 2024-03-09T09:10:05.081 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 321ecdaba2c v0.39 2024-03-09T09:10:05.089 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.39 2024-03-09T09:10:05.364 INFO:tasks.workunit.client.1.smithi134.stdout:v0.42 2024-03-09T09:10:05.364 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:10:05.364 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 42 -eq 48 ']' 2024-03-09T09:10:05.365 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.42 2024-03-09T09:10:05.365 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.42 2024-03-09T09:10:05.365 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.42 2024-03-09T09:10:06.108 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:10:06.108 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 40 -eq 48 ']' 2024-03-09T09:10:06.108 INFO:tasks.workunit.client.0.smithi067.stdout:v0.40 2024-03-09T09:10:06.108 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.40 2024-03-09T09:10:06.108 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.40 2024-03-09T09:10:06.109 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.40 2024-03-09T09:10:08.795 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 32% (74/226) Updating files: 33% (75/226) Updating files: 34% (77/226) Updating files: 35% (80/226) Updating files: 36% (82/226) Updating files: 37% (84/226) Updating files: 38% (86/226) Updating files: 39% (89/226) Updating files: 40% (91/226) Updating files: 41% (93/226) Updating files: 42% (95/226) Updating files: 43% (98/226) Updating files: 44% (100/226) Updating files: 45% (102/226) Updating files: 46% (104/226) Updating files: 47% (107/226) Updating files: 48% (109/226) Updating files: 49% (111/226) Updating files: 50% (113/226) Updating files: 51% (116/226) Updating files: 52% (118/226) Updating files: 53% (120/226) Updating files: 54% (123/226) Updating files: 55% (125/226) Updating files: 56% (127/226) Updating files: 57% (129/226) Updating files: 58% (132/226) Updating files: 59% (134/226) Updating files: 60% (136/226) Updating files: 61% (138/226) Updating files: 62% (141/226) Updating files: 63% (143/226) Updating files: 64% (145/226) Updating files: 65% (147/226) Updating files: 66% (150/226) Updating files: 67% (152/226) Updating files: 68% (154/226) Updating files: 69% (156/226) Updating files: 70% (159/226) Updating files: 71% (161/226) Updating files: 72% (163/226) Updating files: 73% (165/226) Updating files: 74% (168/226) Updating files: 75% (170/226) Updating files: 76% (172/226) Updating files: 77% (175/226) Updating files: 78% (177/226) Updating files: 79% (179/226) Updating files: 80% (181/226) Updating files: 81% (184/226) Updating files: 82% (186/226) Updating files: 83% (188/226) Updating files: 84% (190/226) Updating files: 85% (193/226) Updating files: 86% (195/226) Updating files: 87% (197/226) Updating files: 88% (199/226) Updating files: 89% (202/226) Updating files: 90% (204/226) Updating files: 91% (206/226) Updating files: 92% (208/226) Updating files: 93% (211/226) Updating files: 94% (213/226) Updating files: 95% (215/226) Updating files: 96% (217/226) Updating files: 97% (220/226) Updating files: 98% (222/226) Updating files: 99% (224/226) Updating files: 100% (226/226) Updating files: 100% (226/226), done. 2024-03-09T09:10:08.826 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 7eea40ea37f v0.40 2024-03-09T09:10:08.832 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.40 2024-03-09T09:10:09.010 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:10:09.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 41 -eq 48 ']' 2024-03-09T09:10:09.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.41 2024-03-09T09:10:09.011 INFO:tasks.workunit.client.0.smithi067.stdout:v0.41 2024-03-09T09:10:09.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.41 2024-03-09T09:10:09.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.41 2024-03-09T09:10:10.760 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 1% (4/250) Updating files: 2% (5/250) Updating files: 3% (8/250) Updating files: 4% (10/250) Updating files: 5% (13/250) Updating files: 6% (15/250) Updating files: 7% (18/250) Updating files: 8% (20/250) Updating files: 9% (23/250) Updating files: 10% (25/250) Updating files: 11% (28/250) Updating files: 12% (30/250) Updating files: 13% (33/250) Updating files: 14% (35/250) Updating files: 15% (38/250) Updating files: 16% (40/250) Updating files: 17% (43/250) Updating files: 18% (45/250) Updating files: 19% (48/250) Updating files: 20% (50/250) Updating files: 21% (53/250) Updating files: 22% (55/250) Updating files: 23% (58/250) Updating files: 24% (60/250) Updating files: 25% (63/250) Updating files: 26% (65/250) Updating files: 27% (68/250) Updating files: 28% (70/250) Updating files: 29% (73/250) Updating files: 30% (75/250) Updating files: 31% (78/250) Updating files: 32% (80/250) Updating files: 33% (83/250) Updating files: 34% (85/250) Updating files: 35% (88/250) Updating files: 36% (90/250) Updating files: 37% (93/250) Updating files: 38% (95/250) Updating files: 39% (98/250) Updating files: 40% (100/250) Updating files: 41% (103/250) Updating files: 42% (105/250) Updating files: 43% (108/250) Updating files: 44% (110/250) Updating files: 45% (113/250) Updating files: 46% (115/250) Updating files: 47% (118/250) Updating files: 48% (120/250) Updating files: 49% (123/250) Updating files: 50% (125/250) Updating files: 51% (128/250) Updating files: 52% (130/250) Updating files: 53% (133/250) Updating files: 54% (135/250) Updating files: 55% (138/250) Updating files: 56% (140/250) Updating files: 57% (143/250) Updating files: 58% (145/250) Updating files: 59% (148/250) Updating files: 60% (150/250) Updating files: 61% (153/250) Updating files: 62% (155/250) Updating files: 63% (158/250) Updating files: 64% (160/250) Updating files: 65% (163/250) Updating files: 66% (165/250) Updating files: 67% (168/250) Updating files: 68% (170/250) Updating files: 68% (172/250) Updating files: 69% (173/250) Updating files: 70% (175/250) Updating files: 71% (178/250) Updating files: 72% (180/250) Updating files: 73% (183/250) Updating files: 74% (185/250) Updating files: 74% (187/250) Updating files: 75% (188/250) Updating files: 76% (190/250) Updating files: 77% (193/250) Updating files: 78% (195/250) Updating files: 79% (198/250) Updating files: 80% (200/250) Updating files: 81% (203/250) Updating files: 82% (205/250) Updating files: 83% (208/250) Updating files: 84% (210/250) Updating files: 85% (213/250) Updating files: 86% (215/250) Updating files: 87% (218/250) Updating files: 88% (220/250) Updating files: 89% (223/250) Updating files: 90% (225/250) Updating files: 91% (228/250) Updating files: 92% (230/250) Updating files: 93% (233/250) Updating files: 94% (235/250) Updating files: 95% (238/250) Updating files: 96% (240/250) Updating files: 97% (243/250) Updating files: 98% (245/250) Updating files: 99% (248/250) Updating files: 100% (250/250) Updating files: 100% (250/250), done. 2024-03-09T09:10:10.809 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at b205c64c687 v0.42 2024-03-09T09:10:10.818 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.42 2024-03-09T09:10:11.151 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:10:11.151 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 43 -eq 48 ']' 2024-03-09T09:10:11.151 INFO:tasks.workunit.client.1.smithi134.stdout:v0.43 2024-03-09T09:10:11.151 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.43 2024-03-09T09:10:11.151 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.43 2024-03-09T09:10:11.152 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.43 2024-03-09T09:10:13.318 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 85% (74/87) Updating files: 86% (75/87) Updating files: 87% (76/87) Updating files: 88% (77/87) Updating files: 89% (78/87) Updating files: 90% (79/87) Updating files: 91% (80/87) Updating files: 93% (81/87) Updating files: 94% (82/87) Updating files: 95% (83/87) Updating files: 96% (84/87) Updating files: 97% (85/87) Updating files: 98% (86/87) Updating files: 100% (87/87) Updating files: 100% (87/87), done. 2024-03-09T09:10:13.422 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at c1345f7136a v0.41 2024-03-09T09:10:13.434 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.41 2024-03-09T09:10:13.477 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 18% (21/111) Updating files: 19% (22/111) Updating files: 20% (23/111) Updating files: 21% (24/111) Updating files: 22% (25/111) Updating files: 23% (26/111) Updating files: 24% (27/111) Updating files: 25% (28/111) Updating files: 26% (29/111) Updating files: 27% (30/111) Updating files: 28% (32/111) Updating files: 29% (33/111) Updating files: 30% (34/111) Updating files: 31% (35/111) Updating files: 32% (36/111) Updating files: 33% (37/111) Updating files: 34% (38/111) Updating files: 35% (39/111) Updating files: 36% (40/111) Updating files: 37% (42/111) Updating files: 38% (43/111) Updating files: 39% (44/111) Updating files: 40% (45/111) Updating files: 41% (46/111) Updating files: 42% (47/111) Updating files: 43% (48/111) Updating files: 44% (49/111) Updating files: 45% (50/111) Updating files: 46% (52/111) Updating files: 47% (53/111) Updating files: 48% (54/111) Updating files: 49% (55/111) Updating files: 50% (56/111) Updating files: 51% (57/111) Updating files: 52% (58/111) Updating files: 53% (59/111) Updating files: 54% (60/111) Updating files: 55% (62/111) Updating files: 56% (63/111) Updating files: 57% (64/111) Updating files: 58% (65/111) Updating files: 59% (66/111) Updating files: 60% (67/111) Updating files: 61% (68/111) Updating files: 62% (69/111) Updating files: 63% (70/111) Updating files: 64% (72/111) Updating files: 65% (73/111) Updating files: 66% (74/111) Updating files: 67% (75/111) Updating files: 68% (76/111) Updating files: 69% (77/111) Updating files: 70% (78/111) Updating files: 71% (79/111) Updating files: 72% (80/111) Updating files: 73% (82/111) Updating files: 74% (83/111) Updating files: 75% (84/111) Updating files: 76% (85/111) Updating files: 77% (86/111) Updating files: 78% (87/111) Updating files: 79% (88/111) Updating files: 80% (89/111) Updating files: 81% (90/111) Updating files: 82% (92/111) Updating files: 83% (93/111) Updating files: 84% (94/111) Updating files: 85% (95/111) Updating files: 86% (96/111) Updating files: 87% (97/111) Updating files: 88% (98/111) Updating files: 89% (99/111) Updating files: 90% (100/111) Updating files: 91% (102/111) Updating files: 92% (103/111) Updating files: 93% (104/111) Updating files: 94% (105/111) Updating files: 95% (106/111) Updating files: 96% (107/111) Updating files: 97% (108/111) Updating files: 98% (109/111) Updating files: 99% (110/111) Updating files: 100% (111/111) Updating files: 100% (111/111), done. 2024-03-09T09:10:13.550 INFO:tasks.workunit.client.0.smithi067.stdout:v0.42 2024-03-09T09:10:13.550 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:10:13.551 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 42 -eq 48 ']' 2024-03-09T09:10:13.551 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.42 2024-03-09T09:10:13.551 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.42 2024-03-09T09:10:13.551 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.42 2024-03-09T09:10:13.571 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 9fa8781c014 v0.43 2024-03-09T09:10:13.574 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.43 2024-03-09T09:10:14.109 INFO:tasks.workunit.client.1.smithi134.stdout:v0.44 2024-03-09T09:10:14.110 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:10:14.110 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 44 -eq 48 ']' 2024-03-09T09:10:14.110 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.44 2024-03-09T09:10:14.110 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.44 2024-03-09T09:10:14.110 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.44 2024-03-09T09:10:16.935 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:10:16.943 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:10:18.482 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-03-09T09:10:18.482 INFO:tasks.ceph.mon.a:Sent signal 18 2024-03-09T09:10:18.483 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:10:18.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:10:18.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.683+0000 7fe713000640 1 Processor -- start 2024-03-09T09:10:18.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.683+0000 7fe713000640 1 -- start start 2024-03-09T09:10:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.685+0000 7fe713000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.685+0000 7fe713000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c153d40 con 0x7fe70c0c22a0 2024-03-09T09:10:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.685+0000 7fe713000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c0c2790 con 0x7fe70c0ac730 2024-03-09T09:10:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.685+0000 7fe713000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70c070ee0 con 0x7fe70c071190 2024-03-09T09:10:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.685+0000 7fe70be00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7fe70c0ac730 legacy=0x7fe70c157650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:10:18.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.686+0000 7fe711000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.686+0000 7fe711000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41520/0 (socket says 172.21.15.67:41520) 2024-03-09T09:10:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.686+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 learned_addr learned my addr 172.21.15.67:0/1380189134 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.686+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 msgr2=0x7fe70c157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:10:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.686+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 msgr2=0x7fe70c157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:10:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.686+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:10:18.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.686+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:10:18.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.886+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> v1:172.21.15.134:6789/0 conn(0x7fe70c0ac730 legacy=0x7fe70c157650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:10:18.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.887+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.887+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 msgr2=0x7fe70c157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:10:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.887+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 msgr2=0x7fe70c157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:10:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.887+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:10:18.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:18.888+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:10:19.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:19.287+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> v1:172.21.15.134:6789/0 conn(0x7fe70c0ac730 legacy=0x7fe70c157650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:10:19.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:19.289+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:19.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:19.290+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 msgr2=0x7fe70c157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:10:19.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:19.290+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 msgr2=0x7fe70c157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:10:19.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:19.290+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:10:19.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:19.290+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:10:20.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:20.088+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> v1:172.21.15.134:6789/0 conn(0x7fe70c0ac730 legacy=0x7fe70c157650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:10:20.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:20.091+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:20.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:20.091+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 msgr2=0x7fe70c157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:10:20.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:20.091+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 msgr2=0x7fe70c157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:10:20.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:20.091+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:10:20.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:20.091+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:10:21.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.686+0000 7fe709600640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 msgr2=0x7fe70c157d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:10:21.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.686+0000 7fe709600640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.67:3300/0 conn(0x7fe70c071190 0x7fe70c157d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:21.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.686+0000 7fe709600640 1 -- 172.21.15.67:0/1380189134 >> v1:172.21.15.134:6789/0 conn(0x7fe70c0ac730 legacy=0x7fe70c157650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:10:21.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.686+0000 7fe709600640 1 -- 172.21.15.67:0/1380189134 >> v1:172.21.15.67:6789/0 conn(0x7fe70c0c22a0 legacy=0x7fe70c15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:21.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.686+0000 7fe709600640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:21.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.686+0000 7fe709600640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec006180 0x7fe6ec008620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:21.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.686+0000 7fe709600640 1 -- 172.21.15.67:0/1380189134 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70c070ee0 con 0x7fe6ec006180 2024-03-09T09:10:21.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.686+0000 7fe709600640 1 -- 172.21.15.67:0/1380189134 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec008ce0 con 0x7fe6ec0036a0 2024-03-09T09:10:21.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.687+0000 7fe70b400640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec006180 msgr2=0x7fe6ec008620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:10:21.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.687+0000 7fe70b400640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec006180 0x7fe6ec008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:10:21.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.687+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:21.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.687+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 msgr2=0x7fe6ec005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:10:21.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.687+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 msgr2=0x7fe6ec005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:10:21.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.687+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:10:21.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.687+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:10:21.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.888+0000 7fe70b400640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec006180 msgr2=0x7fe6ec008620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:10:21.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.888+0000 7fe70b400640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec006180 0x7fe6ec008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:10:21.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.888+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:21.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.889+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 msgr2=0x7fe6ec005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:10:21.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.889+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 msgr2=0x7fe6ec005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:10:21.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.889+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:10:21.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:21.889+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:10:22.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:22.289+0000 7fe70b400640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec006180 msgr2=0x7fe6ec008620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:10:22.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:22.289+0000 7fe70b400640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec006180 0x7fe6ec008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:10:22.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:22.290+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:22.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:22.291+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 msgr2=0x7fe6ec005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:10:22.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:22.291+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 msgr2=0x7fe6ec005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:10:22.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:22.291+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:10:22.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:22.291+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:10:23.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:23.090+0000 7fe70b400640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec006180 msgr2=0x7fe6ec008620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:10:23.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:23.090+0000 7fe70b400640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec006180 0x7fe6ec008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:10:23.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:23.093+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:23.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:23.093+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 msgr2=0x7fe6ec005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:10:23.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:23.093+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 msgr2=0x7fe6ec005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:10:23.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:23.093+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:10:23.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:23.093+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:10:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.686+0000 7fe709600640 1 -- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 msgr2=0x7fe6ec005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:10:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.686+0000 7fe709600640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.686+0000 7fe709600640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec006180 msgr2=0x7fe6ec008620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:10:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.686+0000 7fe709600640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec006180 0x7fe6ec008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.686+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 reap_dead start 2024-03-09T09:10:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.687+0000 7fe709600640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec0091e0 0x7fe6ec00b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.687+0000 7fe709600640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.687+0000 7fe709600640 1 -- 172.21.15.67:0/1380189134 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c070920 con 0x7fe70c0c22a0 2024-03-09T09:10:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.687+0000 7fe709600640 1 -- 172.21.15.67:0/1380189134 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70c070ee0 con 0x7fe6ec0091e0 2024-03-09T09:10:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.687+0000 7fe709600640 1 -- 172.21.15.67:0/1380189134 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec008ce0 con 0x7fe6ec0036a0 2024-03-09T09:10:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.687+0000 7fe70be00640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec0091e0 msgr2=0x7fe6ec00b5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:10:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.687+0000 7fe70be00640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec0091e0 0x7fe6ec00b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:10:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.688+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.689+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec0091e0 msgr2=0x7fe6ec00b5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:10:24.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.689+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec0091e0 0x7fe6ec00b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:24.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.689+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 >> v1:172.21.15.67:6789/0 conn(0x7fe70c0c22a0 legacy=0x7fe6ec00bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:24.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.689+0000 7fe711000640 1 -- 172.21.15.67:0/1380189134 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c0c2790 con 0x7fe6ec0036a0 2024-03-09T09:10:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.692+0000 7fe711000640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0c8400 tx=0x7fe70402c630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9a604279e33f48 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.693+0000 7fe70aa00640 1 -- 172.21.15.67:0/1380189134 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70403a070 con 0x7fe6ec0036a0 2024-03-09T09:10:24.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.693+0000 7fe70aa00640 1 -- 172.21.15.67:0/1380189134 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704032480 con 0x7fe6ec0036a0 2024-03-09T09:10:24.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.693+0000 7fe70aa00640 1 -- 172.21.15.67:0/1380189134 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704032650 con 0x7fe6ec0036a0 2024-03-09T09:10:24.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.694+0000 7fe713000640 1 -- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 msgr2=0x7fe6ec005af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:24.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.694+0000 7fe713000640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0c8400 tx=0x7fe70402c630 comp rx=0 tx=0).stop 2024-03-09T09:10:24.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.695+0000 7fe713000640 1 -- 172.21.15.67:0/1380189134 shutdown_connections 2024-03-09T09:10:24.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.695+0000 7fe713000640 1 --2- 172.21.15.67:0/1380189134 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe6ec005af0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:24.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.695+0000 7fe713000640 1 --2- 172.21.15.67:0/1380189134 >> v2:172.21.15.134:3300/0 conn(0x7fe6ec0091e0 0x7fe6ec00b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:24.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.695+0000 7fe713000640 1 -- 172.21.15.67:0/1380189134 >> 172.21.15.67:0/1380189134 conn(0x7fe70c0c7370 msgr2=0x7fe70c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.695+0000 7fe713000640 1 -- 172.21.15.67:0/1380189134 shutdown_connections 2024-03-09T09:10:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.695+0000 7fe713000640 1 -- 172.21.15.67:0/1380189134 wait complete. 2024-03-09T09:10:24.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.698+0000 7fe713000640 1 Processor -- start 2024-03-09T09:10:24.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.698+0000 7fe713000640 1 -- start start 2024-03-09T09:10:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.699+0000 7fe713000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe70c1efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.699+0000 7fe713000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6ec0091e0 0x7fe70c14d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.699+0000 7fe713000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70c0c22a0 0x7fe70c1ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.699+0000 7fe713000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c154ce0 con 0x7fe70c0c22a0 2024-03-09T09:10:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.699+0000 7fe713000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c071570 con 0x7fe6ec0091e0 2024-03-09T09:10:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.699+0000 7fe713000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c151610 con 0x7fe6ec0036a0 2024-03-09T09:10:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.700+0000 7fe70b400640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6ec0091e0 msgr2=0x7fe70c14d8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:10:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.700+0000 7fe70b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6ec0091e0 0x7fe70c14d8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:10:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.700+0000 7fe711000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70c0c22a0 0x7fe70c1ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.700+0000 7fe70be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe70c1efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.700+0000 7fe711000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70c0c22a0 0x7fe70c1ebfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59552/0 (socket says 172.21.15.67:59552) 2024-03-09T09:10:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.700+0000 7fe711000640 1 -- 172.21.15.67:0/4288345823 learned_addr learned my addr 172.21.15.67:0/4288345823 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.701+0000 7fe711000640 1 -- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 msgr2=0x7fe70c1efb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.701+0000 7fe711000640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe70c1efb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.701+0000 7fe711000640 1 -- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6ec0091e0 msgr2=0x7fe70c14d8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:10:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.701+0000 7fe711000640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6ec0091e0 0x7fe70c14d8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.701+0000 7fe711000640 1 -- 172.21.15.67:0/4288345823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c14fb20 con 0x7fe70c0c22a0 2024-03-09T09:10:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.702+0000 7fe711000640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70c0c22a0 0x7fe70c1ebfa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe704038040 tx=0x7fe704004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.703+0000 7fe703e00640 1 -- 172.21.15.67:0/4288345823 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704044020 con 0x7fe70c0c22a0 2024-03-09T09:10:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.703+0000 7fe703e00640 1 -- 172.21.15.67:0/4288345823 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7040357c0 con 0x7fe70c0c22a0 2024-03-09T09:10:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.703+0000 7fe713000640 1 -- 172.21.15.67:0/4288345823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70c0c1770 con 0x7fe70c0c22a0 2024-03-09T09:10:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.703+0000 7fe703e00640 1 -- 172.21.15.67:0/4288345823 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70403a040 con 0x7fe70c0c22a0 2024-03-09T09:10:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.703+0000 7fe713000640 1 -- 172.21.15.67:0/4288345823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c14fdc0 con 0x7fe70c0c22a0 2024-03-09T09:10:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.705+0000 7fe713000640 1 -- 172.21.15.67:0/4288345823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70c068d50 con 0x7fe70c0c22a0 2024-03-09T09:10:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.709+0000 7fe703e00640 1 -- 172.21.15.67:0/4288345823 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe704047020 con 0x7fe70c0c22a0 2024-03-09T09:10:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.710+0000 7fe703e00640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6ec06c810 0x7fe6ec06ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.711+0000 7fe703e00640 1 -- 172.21.15.67:0/4288345823 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe7040ace40 con 0x7fe70c0c22a0 2024-03-09T09:10:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.711+0000 7fe70be00640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6ec06c810 0x7fe6ec06ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.712+0000 7fe70be00640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6ec06c810 0x7fe6ec06ecd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0ac700 tx=0x7fe6fc00d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:24.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:24.717+0000 7fe703e00640 1 -- 172.21.15.67:0/4288345823 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe70c068d50 con 0x7fe70c0c22a0 2024-03-09T09:10:25.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.026+0000 7fe713000640 1 -- 172.21.15.67:0/4288345823 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe70c075a00 con 0x7fe70c0c22a0 2024-03-09T09:10:25.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.027+0000 7fe703e00640 1 -- 172.21.15.67:0/4288345823 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fe70404e030 con 0x7fe70c0c22a0 2024-03-09T09:10:25.031 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":128,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:10:25.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.029+0000 7fe713000640 1 -- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6ec06c810 msgr2=0x7fe6ec06ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.029+0000 7fe713000640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6ec06c810 0x7fe6ec06ecd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0ac700 tx=0x7fe6fc00d040 comp rx=0 tx=0).stop 2024-03-09T09:10:25.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.029+0000 7fe713000640 1 -- 172.21.15.67:0/4288345823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70c0c22a0 msgr2=0x7fe70c1ebfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.029+0000 7fe713000640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70c0c22a0 0x7fe70c1ebfa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe704038040 tx=0x7fe704004ab0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.029+0000 7fe713000640 1 -- 172.21.15.67:0/4288345823 shutdown_connections 2024-03-09T09:10:25.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.029+0000 7fe713000640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe6ec06c810 0x7fe6ec06ecd0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.029+0000 7fe713000640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe70c0c22a0 0x7fe70c1ebfa0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.029+0000 7fe713000640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6ec0091e0 0x7fe70c14d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.029+0000 7fe713000640 1 --2- 172.21.15.67:0/4288345823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6ec0036a0 0x7fe70c1efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.029+0000 7fe713000640 1 -- 172.21.15.67:0/4288345823 >> 172.21.15.67:0/4288345823 conn(0x7fe70c0c7370 msgr2=0x7fe70c1574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.030+0000 7fe713000640 1 -- 172.21.15.67:0/4288345823 shutdown_connections 2024-03-09T09:10:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.030+0000 7fe713000640 1 -- 172.21.15.67:0/4288345823 wait complete. 2024-03-09T09:10:25.053 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:10:25.054 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:10:25.054 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:10:25.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.306+0000 7f34f6c00640 1 Processor -- start 2024-03-09T09:10:25.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.306+0000 7f34f6c00640 1 -- start start 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.308+0000 7f34f6c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f34f00730c0 0x7f34f00aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.308+0000 7f34f6c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34f00734a0 0x7f34f00c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.308+0000 7f34f6c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34f00c33c0 0x7f34f006d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.308+0000 7f34f6c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f00aa960 con 0x7f34f00734a0 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.308+0000 7f34f6c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f0152e00 con 0x7f34f00730c0 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.308+0000 7f34f6c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f01530e0 con 0x7f34f00c33c0 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.308+0000 7f34efe00640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f34f00730c0 msgr2=0x7f34f00aa420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.309+0000 7f34efe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f34f00730c0 0x7f34f00aa420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.309+0000 7f34e7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34f00734a0 0x7f34f00c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.309+0000 7f34e7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f34f00734a0 0x7f34f00c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59560/0 (socket says 172.21.15.67:59560) 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.309+0000 7f34e7400640 1 -- 172.21.15.67:0/4164405873 learned_addr learned my addr 172.21.15.67:0/4164405873 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.309+0000 7f34f4c00640 1 --2- 172.21.15.67:0/4164405873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34f00c33c0 0x7f34f006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.309+0000 7f34e7400640 1 -- 172.21.15.67:0/4164405873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34f00c33c0 msgr2=0x7f34f006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:10:25.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.309+0000 7f34e7400640 1 --2- 172.21.15.67:0/4164405873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34f00c33c0 0x7f34f006d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.309+0000 7f34e7400640 1 -- 172.21.15.67:0/4164405873 >> v2:172.21.15.134:3300/0 conn(0x7f34f00730c0 msgr2=0x7f34f00aa420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:10:25.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.309+0000 7f34e7400640 1 --2- 172.21.15.67:0/4164405873 >> v2:172.21.15.134:3300/0 conn(0x7f34f00730c0 0x7f34f00aa420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.309+0000 7f34e7400640 1 -- 172.21.15.67:0/4164405873 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f01533c0 con 0x7f34f00734a0 2024-03-09T09:10:25.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.310+0000 7f34e7400640 1 --2- 172.21.15.67:0/4164405873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f00734a0 0x7f34f00c2e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f34f006c090 tx=0x7f34d802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81763316e886ef4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:25.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.311+0000 7f34ef400640 1 -- 172.21.15.67:0/4164405873 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8035020 con 0x7f34f00734a0 2024-03-09T09:10:25.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.311+0000 7f34ef400640 1 -- 172.21.15.67:0/4164405873 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d8031800 con 0x7f34f00734a0 2024-03-09T09:10:25.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.311+0000 7f34ef400640 1 -- 172.21.15.67:0/4164405873 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8031bd0 con 0x7f34f00734a0 2024-03-09T09:10:25.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.312+0000 7f34f6c00640 1 -- 172.21.15.67:0/4164405873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f00734a0 msgr2=0x7f34f00c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.312+0000 7f34f6c00640 1 --2- 172.21.15.67:0/4164405873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f00734a0 0x7f34f00c2e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f34f006c090 tx=0x7f34d802b8a0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.312+0000 7f34f6c00640 1 -- 172.21.15.67:0/4164405873 shutdown_connections 2024-03-09T09:10:25.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.312+0000 7f34f6c00640 1 --2- 172.21.15.67:0/4164405873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34f00c33c0 0x7f34f006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.312+0000 7f34f6c00640 1 --2- 172.21.15.67:0/4164405873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f00734a0 0x7f34f00c2e80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.312+0000 7f34f6c00640 1 --2- 172.21.15.67:0/4164405873 >> v2:172.21.15.134:3300/0 conn(0x7f34f00730c0 0x7f34f00aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.312+0000 7f34f6c00640 1 -- 172.21.15.67:0/4164405873 >> 172.21.15.67:0/4164405873 conn(0x7f34f00c7370 msgr2=0x7f34f00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:25.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.312+0000 7f34f6c00640 1 -- 172.21.15.67:0/4164405873 shutdown_connections 2024-03-09T09:10:25.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.312+0000 7f34f6c00640 1 -- 172.21.15.67:0/4164405873 wait complete. 2024-03-09T09:10:25.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.314+0000 7f34f6c00640 1 Processor -- start 2024-03-09T09:10:25.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.314+0000 7f34f6c00640 1 -- start start 2024-03-09T09:10:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.315+0000 7f34f6c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f00730c0 0x7f34f01e2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.315+0000 7f34f6c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f00734a0 0x7f34f014b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.315+0000 7f34f6c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34f00c33c0 0x7f34f014baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.315+0000 7f34f6c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00aaef0 con 0x7f34f00734a0 2024-03-09T09:10:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.315+0000 7f34f6c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0153980 con 0x7f34f00730c0 2024-03-09T09:10:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.315+0000 7f34f6c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00ab070 con 0x7f34f00c33c0 2024-03-09T09:10:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.315+0000 7f34efe00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f00730c0 msgr2=0x7f34f01e2140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:10:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.315+0000 7f34efe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f00730c0 0x7f34f01e2140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:10:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.315+0000 7f34e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f00734a0 0x7f34f014b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.316+0000 7f34e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f00734a0 0x7f34f014b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59566/0 (socket says 172.21.15.67:59566) 2024-03-09T09:10:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.316+0000 7f34e7400640 1 -- 172.21.15.67:0/278778528 learned_addr learned my addr 172.21.15.67:0/278778528 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.316+0000 7f34f4c00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34f00c33c0 0x7f34f014baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.316+0000 7f34f4c00640 1 -- 172.21.15.67:0/278778528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f00730c0 msgr2=0x7f34f01e2140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:10:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.316+0000 7f34f4c00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34f00730c0 0x7f34f01e2140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.316+0000 7f34f4c00640 1 -- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f00734a0 msgr2=0x7f34f014b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.316+0000 7f34f4c00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f00734a0 0x7f34f014b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.316+0000 7f34f4c00640 1 -- 172.21.15.67:0/278778528 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f01eb6c0 con 0x7f34f00c33c0 2024-03-09T09:10:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.316+0000 7f34e7400640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f00734a0 0x7f34f014b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:10:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.317+0000 7f34f4c00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34f00c33c0 0x7f34f014baa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34e800eba0 tx=0x7f34e800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.317+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e8018020 con 0x7f34f00c33c0 2024-03-09T09:10:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.317+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e800cd20 con 0x7f34f00c33c0 2024-03-09T09:10:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.317+0000 7f34f6c00640 1 -- 172.21.15.67:0/278778528 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f01eb420 con 0x7f34f00c33c0 2024-03-09T09:10:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.317+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e80146c0 con 0x7f34f00c33c0 2024-03-09T09:10:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.317+0000 7f34f6c00640 1 -- 172.21.15.67:0/278778528 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f0156b70 con 0x7f34f00c33c0 2024-03-09T09:10:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.319+0000 7f34f6c00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c3d0 0x7f34f014cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.319+0000 7f34f6c00640 1 -- 172.21.15.67:0/278778528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0153ba0 con 0x7f34f014c3d0 2024-03-09T09:10:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.319+0000 7f34efe00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c3d0 0x7f34f014cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.320+0000 7f34efe00640 1 -- 172.21.15.67:0/278778528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34f0075a00 con 0x7f34f014c3d0 2024-03-09T09:10:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.320+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f34e8005d10 con 0x7f34f00c33c0 2024-03-09T09:10:25.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.320+0000 7f34efe00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c3d0 0x7f34f014cce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f34e0003930 tx=0x7f34e000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:25.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.322+0000 7f34ecc00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f34cc06b480 0x7f34cc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.322+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f34e808b220 con 0x7f34f00c33c0 2024-03-09T09:10:25.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.322+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0003e10 con 0x7f34f014c3d0 2024-03-09T09:10:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.322+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f34e0015070 con 0x7f34f014c3d0 2024-03-09T09:10:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.322+0000 7f34e7400640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f34cc06b480 0x7f34cc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.322+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c3d0 msgr2=0x7f34f014cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.322+0000 7f34ecc00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c3d0 0x7f34f014cce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f34e0003930 tx=0x7f34e000b8c0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.322+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c3d0 msgr2=0x7f34f014cce0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:10:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.322+0000 7f34ecc00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c3d0 0x7f34f014cce0 secure :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f34e0003930 tx=0x7f34e000b8c0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.323+0000 7f34e7400640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f34cc06b480 0x7f34cc06d940 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f34f01e2ab0 tx=0x7f34d803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:25.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.330+0000 7f34f6c00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c7b0 0x7f34f014d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.330+0000 7f34f6c00640 1 -- 172.21.15.67:0/278778528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00c7ef0 con 0x7f34f014c7b0 2024-03-09T09:10:25.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.331+0000 7f34efe00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c7b0 0x7f34f014d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.331+0000 7f34efe00640 1 -- 172.21.15.67:0/278778528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f34f0156e50 con 0x7f34f014c7b0 2024-03-09T09:10:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.332+0000 7f34efe00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c7b0 0x7f34f014d220 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f34e0003930 tx=0x7f34e0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.332+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00055a0 con 0x7f34f014c7b0 2024-03-09T09:10:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.332+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7f34e00055a0 con 0x7f34f014c7b0 2024-03-09T09:10:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.332+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c7b0 msgr2=0x7f34f014d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.332+0000 7f34ecc00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c7b0 0x7f34f014d220 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f34e0003930 tx=0x7f34e0005a20 comp rx=0 tx=0).stop 2024-03-09T09:10:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.332+0000 7f34ecc00640 1 -- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c7b0 msgr2=0x7f34f014d220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:10:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.332+0000 7f34ecc00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34f014c7b0 0x7f34f014d220 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f34e0003930 tx=0x7f34e0005a20 comp rx=0 tx=0).stop 2024-03-09T09:10:25.338 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:10:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:10:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:10:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:10:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 128, 2024-03-09T09:10:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:10:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:10:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:10:25.338 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:25.339 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:25.340 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:10:25.341 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:25.342 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:10:25.343 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:25.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:10:25.345 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:25.346 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:10:25.347 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.348 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.349 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.350 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:10:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.351 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:10:25.351 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:10:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.333+0000 7f34f6c00640 1 -- 172.21.15.67:0/278778528 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f34cc06b480 msgr2=0x7f34cc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.333+0000 7f34f6c00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f34cc06b480 0x7f34cc06d940 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f34f01e2ab0 tx=0x7f34d803c040 comp rx=0 tx=0).stop 2024-03-09T09:10:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.333+0000 7f34f6c00640 1 -- 172.21.15.67:0/278778528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34f00c33c0 msgr2=0x7f34f014baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.333+0000 7f34f6c00640 1 --2- 172.21.15.67:0/278778528 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34f00c33c0 0x7f34f014baa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34e800eba0 tx=0x7f34e800c490 comp rx=0 tx=0).stop 2024-03-09T09:10:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.333+0000 7f34f4c00640 1 -- 172.21.15.67:0/278778528 reap_dead start 2024-03-09T09:10:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.333+0000 7f34f4c00640 1 -- 172.21.15.67:0/278778528 reap_dead start 2024-03-09T09:10:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.333+0000 7f34f6c00640 1 -- 172.21.15.67:0/278778528 shutdown_connections 2024-03-09T09:10:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.333+0000 7f34f6c00640 1 -- 172.21.15.67:0/278778528 >> 172.21.15.67:0/278778528 conn(0x7f34f00c7370 msgr2=0x7f34f00ab1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.333+0000 7f34f6c00640 1 -- 172.21.15.67:0/278778528 shutdown_connections 2024-03-09T09:10:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.333+0000 7f34f6c00640 1 -- 172.21.15.67:0/278778528 wait complete. 2024-03-09T09:10:25.354 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:10:25.354 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:10:25.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:10:25.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.550+0000 7f4fbd000640 1 Processor -- start 2024-03-09T09:10:25.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.551+0000 7f4fbd000640 1 -- start start 2024-03-09T09:10:25.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.552+0000 7f4fbd000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4fb80730c0 0x7f4fb80aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.552+0000 7f4fbd000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4fb80734a0 0x7f4fb80c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.553+0000 7f4fbd000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb8152e00 con 0x7f4fb80c33c0 2024-03-09T09:10:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.553+0000 7f4fbd000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb80aa960 con 0x7f4fb80730c0 2024-03-09T09:10:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.553+0000 7f4fbd000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb81530e0 con 0x7f4fb80734a0 2024-03-09T09:10:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.553+0000 7f4fb5400640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f4fb80734a0 msgr2=0x7f4fb80c2e80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:10:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.553+0000 7f4fb5400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4fb80734a0 0x7f4fb80c2e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:10:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.553+0000 7f4fb5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4fb80730c0 0x7f4fb80aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.553+0000 7f4fb5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4fb80730c0 0x7f4fb80aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59578/0 (socket says 172.21.15.67:59578) 2024-03-09T09:10:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.553+0000 7f4fb5e00640 1 -- 172.21.15.67:0/1509141106 learned_addr learned my addr 172.21.15.67:0/1509141106 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.554+0000 7f4fb5e00640 1 -- 172.21.15.67:0/1509141106 >> v2:172.21.15.134:3300/0 conn(0x7f4fb80734a0 msgr2=0x7f4fb80c2e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:10:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.554+0000 7f4fb5e00640 1 --2- 172.21.15.67:0/1509141106 >> v2:172.21.15.134:3300/0 conn(0x7f4fb80734a0 0x7f4fb80c2e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.554+0000 7f4fb5e00640 1 -- 172.21.15.67:0/1509141106 >> v1:172.21.15.67:6789/0 conn(0x7f4fb80c33c0 legacy=0x7f4fb806d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.554+0000 7f4fb5e00640 1 -- 172.21.15.67:0/1509141106 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb81533c0 con 0x7f4fb80730c0 2024-03-09T09:10:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.555+0000 7f4fb5e00640 1 --2- 172.21.15.67:0/1509141106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb80730c0 0x7f4fb80aa420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8070320 tx=0x7f4fa802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb11c1e8a99d660 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:25.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.555+0000 7f4fb4a00640 1 -- 172.21.15.67:0/1509141106 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8034070 con 0x7f4fb80730c0 2024-03-09T09:10:25.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.555+0000 7f4fb4a00640 1 -- 172.21.15.67:0/1509141106 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa8030800 con 0x7f4fb80730c0 2024-03-09T09:10:25.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.555+0000 7f4fb4a00640 1 -- 172.21.15.67:0/1509141106 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8030bd0 con 0x7f4fb80730c0 2024-03-09T09:10:25.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.556+0000 7f4fbd000640 1 -- 172.21.15.67:0/1509141106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb80730c0 msgr2=0x7f4fb80aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.556+0000 7f4fbd000640 1 --2- 172.21.15.67:0/1509141106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb80730c0 0x7f4fb80aa420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8070320 tx=0x7f4fa802b940 comp rx=0 tx=0).stop 2024-03-09T09:10:25.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.556+0000 7f4fbd000640 1 -- 172.21.15.67:0/1509141106 shutdown_connections 2024-03-09T09:10:25.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.556+0000 7f4fbd000640 1 --2- 172.21.15.67:0/1509141106 >> v2:172.21.15.134:3300/0 conn(0x7f4fb80734a0 0x7f4fb80c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.556+0000 7f4fbd000640 1 --2- 172.21.15.67:0/1509141106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb80730c0 0x7f4fb80aa420 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.556+0000 7f4fbd000640 1 -- 172.21.15.67:0/1509141106 >> 172.21.15.67:0/1509141106 conn(0x7f4fb80c7370 msgr2=0x7f4fb80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:25.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.557+0000 7f4fbd000640 1 -- 172.21.15.67:0/1509141106 shutdown_connections 2024-03-09T09:10:25.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.557+0000 7f4fbd000640 1 -- 172.21.15.67:0/1509141106 wait complete. 2024-03-09T09:10:25.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.558+0000 7f4fbd000640 1 Processor -- start 2024-03-09T09:10:25.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.558+0000 7f4fbd000640 1 -- start start 2024-03-09T09:10:25.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.559+0000 7f4fbd000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb80730c0 0x7f4fb814e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.559+0000 7f4fbd000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb80734a0 0x7f4fb81dd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.559+0000 7f4fbd000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb80c33c0 0x7f4fb81dbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fbd000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb80aaef0 con 0x7f4fb80730c0 2024-03-09T09:10:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fbd000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8153980 con 0x7f4fb80734a0 2024-03-09T09:10:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fbd000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb80ab070 con 0x7f4fb80c33c0 2024-03-09T09:10:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fb5400640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb80734a0 msgr2=0x7f4fb81dd3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:10:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fb5400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb80734a0 0x7f4fb81dd3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:10:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fb5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb80730c0 0x7f4fb814e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fb5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb80730c0 0x7f4fb814e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59588/0 (socket says 172.21.15.67:59588) 2024-03-09T09:10:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fb5e00640 1 -- 172.21.15.67:0/2894081115 learned_addr learned my addr 172.21.15.67:0/2894081115 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:10:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fb6800640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb80c33c0 0x7f4fb81dbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fb5e00640 1 -- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb80c33c0 msgr2=0x7f4fb81dbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fb5e00640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb80c33c0 0x7f4fb81dbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fb5e00640 1 -- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb80734a0 msgr2=0x7f4fb81dd3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:10:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.560+0000 7f4fb5e00640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fb80734a0 0x7f4fb81dd3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.561+0000 7f4fb5e00640 1 -- 172.21.15.67:0/2894081115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb814ee50 con 0x7f4fb80730c0 2024-03-09T09:10:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.561+0000 7f4fb5e00640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb80730c0 0x7f4fb814e4e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4fa802b980 tx=0x7f4fa8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.561+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8048070 con 0x7f4fb80730c0 2024-03-09T09:10:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.561+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa8030600 con 0x7f4fb80730c0 2024-03-09T09:10:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.561+0000 7f4fbd000640 1 -- 172.21.15.67:0/2894081115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb81efb10 con 0x7f4fb80730c0 2024-03-09T09:10:25.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.561+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8034040 con 0x7f4fb80730c0 2024-03-09T09:10:25.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.562+0000 7f4fbd000640 1 -- 172.21.15.67:0/2894081115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb81efdb0 con 0x7f4fb80730c0 2024-03-09T09:10:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.563+0000 7f4fbd000640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81dc390 0x7f4fb81e0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.563+0000 7f4fbd000640 1 -- 172.21.15.67:0/2894081115 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb81ef970 con 0x7f4fb81dc390 2024-03-09T09:10:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.564+0000 7f4fb5400640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81dc390 0x7f4fb81e0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.564+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4fa804b070 con 0x7f4fb80730c0 2024-03-09T09:10:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.564+0000 7f4fb5400640 1 -- 172.21.15.67:0/2894081115 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fb8075a00 con 0x7f4fb81dc390 2024-03-09T09:10:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.565+0000 7f4fb5400640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81dc390 0x7f4fb81e0050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80c2660 tx=0x7f4fa400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.565+0000 7f4fae000640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f9806b480 0x7f4f9806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.565+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4fa80ac840 con 0x7f4fb80730c0 2024-03-09T09:10:25.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.566+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4003e10 con 0x7f4fb81dc390 2024-03-09T09:10:25.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.566+0000 7f4fb6800640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f9806b480 0x7f4f9806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.566+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4fa4015070 con 0x7f4fb81dc390 2024-03-09T09:10:25.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.567+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81dc390 msgr2=0x7f4fb81e0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.567+0000 7f4fae000640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81dc390 0x7f4fb81e0050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80c2660 tx=0x7f4fa400b8c0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.567+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81dc390 msgr2=0x7f4fb81e0050 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:10:25.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.567+0000 7f4fae000640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81dc390 0x7f4fb81e0050 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:10:25.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.567+0000 7f4fb6800640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f9806b480 0x7f4f9806d940 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0007fd0 tx=0x7f4fb0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:25.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.574+0000 7f4fbd000640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81efff0 0x7f4fb81e0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:10:25.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.574+0000 7f4fbd000640 1 -- 172.21.15.67:0/2894081115 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb80ace30 con 0x7f4fb81efff0 2024-03-09T09:10:25.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.575+0000 7f4fb5400640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81efff0 0x7f4fb81e0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:10:25.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.576+0000 7f4fb5400640 1 -- 172.21.15.67:0/2894081115 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4fb8067dd0 con 0x7f4fb81efff0 2024-03-09T09:10:25.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.576+0000 7f4fb5400640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81efff0 0x7f4fb81e0590 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4006fd0 tx=0x7f4fa4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:10:25.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.577+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4023410 con 0x7f4fb81efff0 2024-03-09T09:10:25.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.577+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4693 (secure 0 0 0) 0x7f4fa4010400 con 0x7f4fb81efff0 2024-03-09T09:10:25.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.577+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81efff0 msgr2=0x7f4fb81e0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.577+0000 7f4fae000640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81efff0 0x7f4fb81e0590 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4006fd0 tx=0x7f4fa4005a20 comp rx=0 tx=0).stop 2024-03-09T09:10:25.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.577+0000 7f4fae000640 1 -- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81efff0 msgr2=0x7f4fb81e0590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:10:25.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.577+0000 7f4fae000640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fb81efff0 0x7f4fb81e0590 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4006fd0 tx=0x7f4fa4005a20 comp rx=0 tx=0).stop 2024-03-09T09:10:25.584 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:10:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:10:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:10:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:10:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 128, 2024-03-09T09:10:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:10:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:10:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:10:25.584 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:10:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:10:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:10:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:10:25.585 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:25.586 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:25.587 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:10:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:10:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:25.588 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:25.589 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:25.590 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.591 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:10:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:10:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:10:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:10:25.592 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:10:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:10:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:10:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:10:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:10:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:10:25.593 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:10:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:10:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:10:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:10:25.594 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:10:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:10:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:10:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:10:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:10:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:10:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:10:25.595 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:10:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:10:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:10:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:10:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:10:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:10:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:10:25.596 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:10:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:10:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.597 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:10:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:10:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:10:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:10:25.599 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:10:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:10:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:25.600 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:10:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:10:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:25.601 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:10:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.602 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:10:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.603 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:10:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:10:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:25.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:10:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:10:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:10:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:10:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:10:25.605 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:10:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:10:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:10:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:10:25.606 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:10:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:10:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:10:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:10:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:10:25.607 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:10:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:10:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:25.608 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:10:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:10:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:25.609 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:10:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:10:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:10:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:10:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:10:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:10:25.610 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:10:25.611 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:10:25.611 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:10:25.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:10:25.612 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:10:25.612 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:10:25.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.578+0000 7f4fbd000640 1 -- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f9806b480 msgr2=0x7f4f9806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.578+0000 7f4fbd000640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f9806b480 0x7f4f9806d940 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0007fd0 tx=0x7f4fb0009040 comp rx=0 tx=0).stop 2024-03-09T09:10:25.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.578+0000 7f4fbd000640 1 -- 172.21.15.67:0/2894081115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb80730c0 msgr2=0x7f4fb814e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:10:25.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.578+0000 7f4fbd000640 1 --2- 172.21.15.67:0/2894081115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fb80730c0 0x7f4fb814e4e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4fa802b980 tx=0x7f4fa8004730 comp rx=0 tx=0).stop 2024-03-09T09:10:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.578+0000 7f4fb6800640 1 -- 172.21.15.67:0/2894081115 reap_dead start 2024-03-09T09:10:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.578+0000 7f4fb6800640 1 -- 172.21.15.67:0/2894081115 reap_dead start 2024-03-09T09:10:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.578+0000 7f4fbd000640 1 -- 172.21.15.67:0/2894081115 shutdown_connections 2024-03-09T09:10:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.578+0000 7f4fbd000640 1 -- 172.21.15.67:0/2894081115 >> 172.21.15.67:0/2894081115 conn(0x7f4fb80c7370 msgr2=0x7f4fb80abb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:10:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.578+0000 7f4fbd000640 1 -- 172.21.15.67:0/2894081115 shutdown_connections 2024-03-09T09:10:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:10:25.578+0000 7f4fbd000640 1 -- 172.21.15.67:0/2894081115 wait complete. 2024-03-09T09:10:25.614 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:10:25.614 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:10:27.223 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 22% (42/186) Updating files: 23% (43/186) Updating files: 24% (45/186) Updating files: 25% (47/186) Updating files: 26% (49/186) Updating files: 27% (51/186) Updating files: 28% (53/186) Updating files: 29% (54/186) Updating files: 30% (56/186) Updating files: 31% (58/186) Updating files: 32% (60/186) Updating files: 33% (62/186) Updating files: 34% (64/186) Updating files: 35% (66/186) Updating files: 36% (67/186) Updating files: 37% (69/186) Updating files: 38% (71/186) Updating files: 39% (73/186) Updating files: 40% (75/186) Updating files: 41% (77/186) Updating files: 42% (79/186) Updating files: 43% (80/186) Updating files: 44% (82/186) Updating files: 45% (84/186) Updating files: 46% (86/186) Updating files: 47% (88/186) Updating files: 48% (90/186) Updating files: 49% (92/186) Updating files: 50% (93/186) Updating files: 51% (95/186) Updating files: 52% (97/186) Updating files: 53% (99/186) Updating files: 54% (101/186) Updating files: 55% (103/186) Updating files: 55% (104/186) Updating files: 56% (105/186) Updating files: 57% (107/186) Updating files: 58% (108/186) Updating files: 59% (110/186) Updating files: 60% (112/186) Updating files: 61% (114/186) Updating files: 62% (116/186) Updating files: 63% (118/186) Updating files: 64% (120/186) Updating files: 65% (121/186) Updating files: 66% (123/186) Updating files: 67% (125/186) Updating files: 68% (127/186) Updating files: 69% (129/186) Updating files: 70% (131/186) Updating files: 71% (133/186) Updating files: 72% (134/186) Updating files: 73% (136/186) Updating files: 74% (138/186) Updating files: 75% (140/186) Updating files: 76% (142/186) Updating files: 77% (144/186) Updating files: 78% (146/186) Updating files: 79% (147/186) Updating files: 80% (149/186) Updating files: 81% (151/186) Updating files: 81% (152/186) Updating files: 82% (153/186) Updating files: 83% (155/186) Updating files: 84% (157/186) Updating files: 85% (159/186) Updating files: 86% (160/186) Updating files: 87% (162/186) Updating files: 88% (164/186) Updating files: 89% (166/186) Updating files: 90% (168/186) Updating files: 91% (170/186) Updating files: 92% (172/186) Updating files: 93% (173/186) Updating files: 94% (175/186) Updating files: 95% (177/186) Updating files: 96% (179/186) Updating files: 97% (181/186) Updating files: 98% (183/186) Updating files: 99% (185/186) Updating files: 100% (186/186) Updating files: 100% (186/186), done. 2024-03-09T09:10:28.078 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at cdd5298d276 v0.44 2024-03-09T09:10:28.085 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.44 2024-03-09T09:10:28.759 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 28% (71/250) Updating files: 29% (73/250) Updating files: 30% (75/250) Updating files: 31% (78/250) Updating files: 32% (80/250) Updating files: 33% (83/250) Updating files: 34% (85/250) Updating files: 35% (88/250) Updating files: 36% (90/250) Updating files: 37% (93/250) Updating files: 38% (95/250) Updating files: 39% (98/250) Updating files: 40% (100/250) Updating files: 41% (103/250) Updating files: 42% (105/250) Updating files: 43% (108/250) Updating files: 44% (110/250) Updating files: 45% (113/250) Updating files: 46% (115/250) Updating files: 47% (118/250) Updating files: 48% (120/250) Updating files: 49% (123/250) Updating files: 50% (125/250) Updating files: 51% (128/250) Updating files: 52% (130/250) Updating files: 53% (133/250) Updating files: 54% (135/250) Updating files: 55% (138/250) Updating files: 56% (140/250) Updating files: 57% (143/250) Updating files: 58% (145/250) Updating files: 59% (148/250) Updating files: 60% (150/250) Updating files: 61% (153/250) Updating files: 62% (155/250) Updating files: 63% (158/250) Updating files: 64% (160/250) Updating files: 65% (163/250) Updating files: 66% (165/250) Updating files: 67% (168/250) Updating files: 68% (170/250) Updating files: 69% (173/250) Updating files: 70% (175/250) Updating files: 71% (178/250) Updating files: 72% (180/250) Updating files: 73% (183/250) Updating files: 74% (185/250) Updating files: 75% (188/250) Updating files: 76% (190/250) Updating files: 77% (193/250) Updating files: 78% (195/250) Updating files: 79% (198/250) Updating files: 80% (200/250) Updating files: 80% (202/250) Updating files: 81% (203/250) Updating files: 81% (204/250) Updating files: 82% (205/250) Updating files: 83% (208/250) Updating files: 84% (210/250) Updating files: 85% (213/250) Updating files: 86% (215/250) Updating files: 87% (218/250) Updating files: 88% (220/250) Updating files: 89% (223/250) Updating files: 90% (225/250) Updating files: 91% (228/250) Updating files: 92% (230/250) Updating files: 93% (233/250) Updating files: 94% (235/250) Updating files: 95% (238/250) Updating files: 96% (240/250) Updating files: 97% (243/250) Updating files: 98% (245/250) Updating files: 99% (248/250) Updating files: 100% (250/250) Updating files: 100% (250/250), done. 2024-03-09T09:10:28.786 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at b205c64c687 v0.42 2024-03-09T09:10:28.790 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.42 2024-03-09T09:10:28.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:10:28.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 43 -eq 48 ']' 2024-03-09T09:10:28.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.43 2024-03-09T09:10:28.916 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.43 2024-03-09T09:10:28.916 INFO:tasks.workunit.client.0.smithi067.stdout:v0.43 2024-03-09T09:10:28.916 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.43 2024-03-09T09:10:29.026 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:10:29.026 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 45 -eq 48 ']' 2024-03-09T09:10:29.026 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.45 2024-03-09T09:10:29.026 INFO:tasks.workunit.client.1.smithi134.stdout:v0.45 2024-03-09T09:10:29.026 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.45 2024-03-09T09:10:29.027 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.45 2024-03-09T09:10:30.183 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 100% (111/111) Updating files: 100% (111/111), done. 2024-03-09T09:10:30.240 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 9fa8781c014 v0.43 2024-03-09T09:10:30.246 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.43 2024-03-09T09:10:30.684 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:10:30.684 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 44 -eq 48 ']' 2024-03-09T09:10:30.684 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.44 2024-03-09T09:10:30.684 INFO:tasks.workunit.client.0.smithi067.stdout:v0.44 2024-03-09T09:10:30.684 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.44 2024-03-09T09:10:30.685 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.44 2024-03-09T09:10:32.158 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 39% (61/154) Updating files: 40% (62/154) Updating files: 41% (64/154) Updating files: 42% (65/154) Updating files: 43% (67/154) Updating files: 44% (68/154) Updating files: 45% (70/154) Updating files: 46% (71/154) Updating files: 47% (73/154) Updating files: 48% (74/154) Updating files: 49% (76/154) Updating files: 50% (77/154) Updating files: 51% (79/154) Updating files: 52% (81/154) Updating files: 53% (82/154) Updating files: 54% (84/154) Updating files: 55% (85/154) Updating files: 56% (87/154) Updating files: 57% (88/154) Updating files: 58% (90/154) Updating files: 59% (91/154) Updating files: 60% (93/154) Updating files: 61% (94/154) Updating files: 62% (96/154) Updating files: 63% (98/154) Updating files: 64% (99/154) Updating files: 65% (101/154) Updating files: 66% (102/154) Updating files: 67% (104/154) Updating files: 68% (105/154) Updating files: 69% (107/154) Updating files: 70% (108/154) Updating files: 71% (110/154) Updating files: 72% (111/154) Updating files: 73% (113/154) Updating files: 74% (114/154) Updating files: 75% (116/154) Updating files: 76% (118/154) Updating files: 77% (119/154) Updating files: 78% (121/154) Updating files: 79% (122/154) Updating files: 80% (124/154) Updating files: 81% (125/154) Updating files: 82% (127/154) Updating files: 83% (128/154) Updating files: 84% (130/154) Updating files: 85% (131/154) Updating files: 86% (133/154) Updating files: 87% (134/154) Updating files: 88% (136/154) Updating files: 89% (138/154) Updating files: 90% (139/154) Updating files: 91% (141/154) Updating files: 92% (142/154) Updating files: 93% (144/154) Updating files: 94% (145/154) Updating files: 95% (147/154) Updating files: 96% (148/154) Updating files: 97% (150/154) Updating files: 98% (151/154) Updating files: 99% (153/154) Updating files: 100% (154/154) Updating files: 100% (154/154), done. 2024-03-09T09:10:32.208 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 0aea1cb1df5 v0.45 2024-03-09T09:10:32.211 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.45 2024-03-09T09:10:32.868 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:10:32.868 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 46 -eq 48 ']' 2024-03-09T09:10:32.868 INFO:tasks.workunit.client.1.smithi134.stdout:v0.46 2024-03-09T09:10:32.868 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.46 2024-03-09T09:10:32.868 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.46 2024-03-09T09:10:32.868 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.46 2024-03-09T09:10:33.302 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 36% (67/186) Updating files: 37% (69/186) Updating files: 38% (71/186) Updating files: 39% (73/186) Updating files: 40% (75/186) Updating files: 41% (77/186) Updating files: 42% (79/186) Updating files: 43% (80/186) Updating files: 44% (82/186) Updating files: 45% (84/186) Updating files: 46% (86/186) Updating files: 47% (88/186) Updating files: 48% (90/186) Updating files: 49% (92/186) Updating files: 50% (93/186) Updating files: 51% (95/186) Updating files: 52% (97/186) Updating files: 53% (99/186) Updating files: 54% (101/186) Updating files: 55% (103/186) Updating files: 56% (105/186) Updating files: 57% (107/186) Updating files: 58% (108/186) Updating files: 59% (110/186) Updating files: 60% (112/186) Updating files: 61% (114/186) Updating files: 62% (116/186) Updating files: 63% (118/186) Updating files: 64% (120/186) Updating files: 65% (121/186) Updating files: 66% (123/186) Updating files: 67% (125/186) Updating files: 68% (127/186) Updating files: 69% (129/186) Updating files: 70% (131/186) Updating files: 71% (133/186) Updating files: 72% (134/186) Updating files: 73% (136/186) Updating files: 74% (138/186) Updating files: 75% (140/186) Updating files: 76% (142/186) Updating files: 77% (144/186) Updating files: 78% (146/186) Updating files: 79% (147/186) Updating files: 80% (149/186) Updating files: 81% (151/186) Updating files: 82% (153/186) Updating files: 83% (155/186) Updating files: 84% (157/186) Updating files: 85% (159/186) Updating files: 86% (160/186) Updating files: 87% (162/186) Updating files: 88% (164/186) Updating files: 89% (166/186) Updating files: 90% (168/186) Updating files: 91% (170/186) Updating files: 92% (172/186) Updating files: 93% (173/186) Updating files: 94% (175/186) Updating files: 95% (177/186) Updating files: 96% (179/186) Updating files: 97% (181/186) Updating files: 98% (183/186) Updating files: 99% (185/186) Updating files: 100% (186/186) Updating files: 100% (186/186), done. 2024-03-09T09:10:33.370 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at cdd5298d276 v0.44 2024-03-09T09:10:33.378 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.44 2024-03-09T09:10:33.504 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:10:33.504 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 45 -eq 48 ']' 2024-03-09T09:10:33.505 INFO:tasks.workunit.client.0.smithi067.stdout:v0.45 2024-03-09T09:10:33.505 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.45 2024-03-09T09:10:33.505 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.45 2024-03-09T09:10:33.506 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.45 2024-03-09T09:10:40.006 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 15% (24/154) Updating files: 16% (25/154) Updating files: 17% (27/154) Updating files: 18% (28/154) Updating files: 19% (30/154) Updating files: 20% (31/154) Updating files: 21% (33/154) Updating files: 22% (34/154) Updating files: 23% (36/154) Updating files: 24% (37/154) Updating files: 25% (39/154) Updating files: 26% (41/154) Updating files: 27% (42/154) Updating files: 28% (44/154) Updating files: 29% (45/154) Updating files: 30% (47/154) Updating files: 31% (48/154) Updating files: 32% (50/154) Updating files: 33% (51/154) Updating files: 34% (53/154) Updating files: 35% (54/154) Updating files: 36% (56/154) Updating files: 37% (57/154) Updating files: 38% (59/154) Updating files: 39% (61/154) Updating files: 40% (62/154) Updating files: 41% (64/154) Updating files: 42% (65/154) Updating files: 43% (67/154) Updating files: 44% (68/154) Updating files: 45% (70/154) Updating files: 46% (71/154) Updating files: 47% (73/154) Updating files: 48% (74/154) Updating files: 49% (76/154) Updating files: 50% (77/154) Updating files: 51% (79/154) Updating files: 52% (81/154) Updating files: 53% (82/154) Updating files: 54% (84/154) Updating files: 55% (85/154) Updating files: 56% (87/154) Updating files: 57% (88/154) Updating files: 58% (90/154) Updating files: 59% (91/154) Updating files: 60% (93/154) Updating files: 61% (94/154) Updating files: 62% (96/154) Updating files: 63% (98/154) Updating files: 64% (99/154) Updating files: 65% (101/154) Updating files: 66% (102/154) Updating files: 67% (104/154) Updating files: 68% (105/154) Updating files: 69% (107/154) Updating files: 70% (108/154) Updating files: 71% (110/154) Updating files: 72% (111/154) Updating files: 73% (113/154) Updating files: 74% (114/154) Updating files: 75% (116/154) Updating files: 76% (118/154) Updating files: 77% (119/154) Updating files: 78% (121/154) Updating files: 79% (122/154) Updating files: 80% (124/154) Updating files: 81% (125/154) Updating files: 82% (127/154) Updating files: 83% (128/154) Updating files: 84% (130/154) Updating files: 85% (131/154) Updating files: 86% (133/154) Updating files: 87% (134/154) Updating files: 88% (136/154) Updating files: 88% (137/154) Updating files: 89% (138/154) Updating files: 90% (139/154) Updating files: 91% (141/154) Updating files: 92% (142/154) Updating files: 93% (144/154) Updating files: 94% (145/154) Updating files: 95% (147/154) Updating files: 96% (148/154) Updating files: 97% (150/154) Updating files: 98% (151/154) Updating files: 99% (153/154) Updating files: 100% (154/154) Updating files: 100% (154/154), done. 2024-03-09T09:10:40.041 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 0aea1cb1df5 v0.45 2024-03-09T09:10:40.048 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.45 2024-03-09T09:10:40.876 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:10:40.876 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 46 -eq 48 ']' 2024-03-09T09:10:40.877 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.46 2024-03-09T09:10:40.877 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.46 2024-03-09T09:10:40.877 INFO:tasks.workunit.client.0.smithi067.stdout:v0.46 2024-03-09T09:10:40.877 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.46 2024-03-09T09:10:45.615 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:10:45.615 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-03-09T09:10:45.615 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-09T09:10:45.616 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-09T09:10:45.626 INFO:tasks.ceph.mon.b:Started 2024-03-09T09:10:45.626 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-03-09T09:10:45.627 INFO:tasks.ceph.mon.a:Sent signal 19 2024-03-09T09:10:45.627 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-03-09T09:10:46.126 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 16% (50/309) Updating files: 17% (53/309) Updating files: 18% (56/309) Updating files: 19% (59/309) Updating files: 20% (62/309) Updating files: 21% (65/309) Updating files: 22% (68/309) Updating files: 23% (72/309) Updating files: 24% (75/309) Updating files: 25% (78/309) Updating files: 26% (81/309) Updating files: 27% (84/309) Updating files: 28% (87/309) Updating files: 29% (90/309) Updating files: 30% (93/309) Updating files: 31% (96/309) Updating files: 32% (99/309) Updating files: 33% (102/309) Updating files: 34% (106/309) Updating files: 35% (109/309) Updating files: 36% (112/309) Updating files: 36% (113/309) Updating files: 37% (115/309) Updating files: 38% (118/309) Updating files: 39% (121/309) Updating files: 40% (124/309) Updating files: 41% (127/309) Updating files: 42% (130/309) Updating files: 43% (133/309) Updating files: 44% (136/309) Updating files: 45% (140/309) Updating files: 46% (143/309) Updating files: 47% (146/309) Updating files: 48% (149/309) Updating files: 49% (152/309) Updating files: 50% (155/309) Updating files: 51% (158/309) Updating files: 52% (161/309) Updating files: 53% (164/309) Updating files: 54% (167/309) Updating files: 55% (170/309) Updating files: 56% (174/309) Updating files: 57% (177/309) Updating files: 58% (180/309) Updating files: 59% (183/309) Updating files: 60% (186/309) Updating files: 60% (188/309) Updating files: 61% (189/309) Updating files: 62% (192/309) Updating files: 63% (195/309) Updating files: 64% (198/309) Updating files: 65% (201/309) Updating files: 66% (204/309) Updating files: 67% (208/309) Updating files: 68% (211/309) Updating files: 69% (214/309) Updating files: 70% (217/309) Updating files: 71% (220/309) Updating files: 72% (223/309) Updating files: 73% (226/309) Updating files: 74% (229/309) Updating files: 75% (232/309) Updating files: 76% (235/309) Updating files: 77% (238/309) Updating files: 78% (242/309) Updating files: 79% (245/309) Updating files: 80% (248/309) Updating files: 81% (251/309) Updating files: 82% (254/309) Updating files: 83% (257/309) Updating files: 84% (260/309) Updating files: 85% (263/309) Updating files: 86% (266/309) Updating files: 87% (269/309) Updating files: 88% (272/309) Updating files: 89% (276/309) Updating files: 90% (279/309) Updating files: 91% (282/309) Updating files: 92% (285/309) Updating files: 93% (288/309) Updating files: 94% (291/309) Updating files: 95% (294/309) Updating files: 96% (297/309) Updating files: 97% (300/309) Updating files: 98% (303/309) Updating files: 99% (306/309) Updating files: 100% (309/309) Updating files: 100% (309/309), done. 2024-03-09T09:10:46.179 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at cb7f1c9c752 v0.46 2024-03-09T09:10:46.184 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.46 2024-03-09T09:10:46.449 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:10:46.449 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 47 -eq 48 ']' 2024-03-09T09:10:46.449 INFO:tasks.workunit.client.0.smithi067.stdout:v0.47 2024-03-09T09:10:46.450 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.47 2024-03-09T09:10:46.450 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.47 2024-03-09T09:10:46.450 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.47 2024-03-09T09:10:47.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:10:47.092 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:10:49.410 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 49% (108/219) Updating files: 50% (110/219) Updating files: 51% (112/219) Updating files: 52% (114/219) Updating files: 53% (117/219) Updating files: 54% (119/219) Updating files: 55% (121/219) Updating files: 56% (123/219) Updating files: 57% (125/219) Updating files: 58% (128/219) Updating files: 59% (130/219) Updating files: 60% (132/219) Updating files: 61% (134/219) Updating files: 62% (136/219) Updating files: 63% (138/219) Updating files: 64% (141/219) Updating files: 65% (143/219) Updating files: 66% (145/219) Updating files: 67% (147/219) Updating files: 68% (149/219) Updating files: 69% (152/219) Updating files: 70% (154/219) Updating files: 71% (156/219) Updating files: 72% (158/219) Updating files: 73% (160/219) Updating files: 74% (163/219) Updating files: 75% (165/219) Updating files: 76% (167/219) Updating files: 77% (169/219) Updating files: 78% (171/219) Updating files: 79% (174/219) Updating files: 80% (176/219) Updating files: 81% (178/219) Updating files: 82% (180/219) Updating files: 83% (182/219) Updating files: 84% (184/219) Updating files: 85% (187/219) Updating files: 86% (189/219) Updating files: 87% (191/219) Updating files: 88% (193/219) Updating files: 89% (195/219) Updating files: 90% (198/219) Updating files: 91% (200/219) Updating files: 92% (202/219) Updating files: 93% (204/219) Updating files: 94% (206/219) Updating files: 95% (209/219) Updating files: 96% (211/219) Updating files: 97% (213/219) Updating files: 98% (215/219) Updating files: 99% (217/219) Updating files: 100% (219/219) Updating files: 100% (219/219), done. 2024-03-09T09:10:49.464 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at ad663d58c89 v0.47 2024-03-09T09:10:49.474 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.47 2024-03-09T09:10:49.799 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:10:49.799 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 48 -eq 48 ']' 2024-03-09T09:10:49.799 INFO:tasks.workunit.client.0.smithi067.stderr:+ continue 2024-03-09T09:10:49.799 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:10:49.799 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 49 -eq 48 ']' 2024-03-09T09:10:49.799 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.49 2024-03-09T09:10:49.800 INFO:tasks.workunit.client.0.smithi067.stdout:v0.49 2024-03-09T09:10:49.800 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.49 2024-03-09T09:10:49.800 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.49 2024-03-09T09:10:50.120 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 2% (7/309) Updating files: 3% (10/309) Updating files: 4% (13/309) Updating files: 5% (16/309) Updating files: 6% (19/309) Updating files: 7% (22/309) Updating files: 7% (23/309) Updating files: 8% (25/309) Updating files: 9% (28/309) Updating files: 10% (31/309) Updating files: 11% (34/309) Updating files: 11% (37/309) Updating files: 12% (38/309) Updating files: 13% (41/309) Updating files: 14% (44/309) Updating files: 15% (47/309) Updating files: 16% (50/309) Updating files: 17% (53/309) Updating files: 18% (56/309) Updating files: 19% (59/309) Updating files: 20% (62/309) Updating files: 21% (65/309) Updating files: 22% (68/309) Updating files: 23% (72/309) Updating files: 24% (75/309) Updating files: 25% (78/309) Updating files: 26% (81/309) Updating files: 27% (84/309) Updating files: 28% (87/309) Updating files: 29% (90/309) Updating files: 30% (93/309) Updating files: 31% (96/309) Updating files: 32% (99/309) Updating files: 33% (102/309) Updating files: 34% (106/309) Updating files: 35% (109/309) Updating files: 35% (111/309) Updating files: 36% (112/309) Updating files: 37% (115/309) Updating files: 38% (118/309) Updating files: 39% (121/309) Updating files: 40% (124/309) Updating files: 41% (127/309) Updating files: 42% (130/309) Updating files: 43% (133/309) Updating files: 44% (136/309) Updating files: 45% (140/309) Updating files: 46% (143/309) Updating files: 47% (146/309) Updating files: 48% (149/309) Updating files: 49% (152/309) Updating files: 50% (155/309) Updating files: 51% (158/309) Updating files: 52% (161/309) Updating files: 53% (164/309) Updating files: 54% (167/309) Updating files: 55% (170/309) Updating files: 56% (174/309) Updating files: 57% (177/309) Updating files: 57% (179/309) Updating files: 58% (180/309) Updating files: 59% (183/309) Updating files: 60% (186/309) Updating files: 61% (189/309) Updating files: 62% (192/309) Updating files: 63% (195/309) Updating files: 64% (198/309) Updating files: 65% (201/309) Updating files: 66% (204/309) Updating files: 67% (208/309) Updating files: 68% (211/309) Updating files: 69% (214/309) Updating files: 70% (217/309) Updating files: 71% (220/309) Updating files: 72% (223/309) Updating files: 72% (224/309) Updating files: 73% (226/309) Updating files: 74% (229/309) Updating files: 75% (232/309) Updating files: 76% (235/309) Updating files: 77% (238/309) Updating files: 78% (242/309) Updating files: 79% (245/309) Updating files: 80% (248/309) Updating files: 81% (251/309) Updating files: 82% (254/309) Updating files: 83% (257/309) Updating files: 84% (260/309) Updating files: 85% (263/309) Updating files: 86% (266/309) Updating files: 87% (269/309) Updating files: 88% (272/309) Updating files: 89% (276/309) Updating files: 90% (279/309) Updating files: 91% (282/309) Updating files: 92% (285/309) Updating files: 93% (288/309) Updating files: 94% (291/309) Updating files: 95% (294/309) Updating files: 96% (297/309) Updating files: 97% (300/309) Updating files: 98% (303/309) Updating files: 99% (306/309) Updating files: 100% (309/309) Updating files: 100% (309/309), done. 2024-03-09T09:10:50.180 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at cb7f1c9c752 v0.46 2024-03-09T09:10:50.190 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.46 2024-03-09T09:10:50.357 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:10:50.357 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 47 -eq 48 ']' 2024-03-09T09:10:50.358 INFO:tasks.workunit.client.1.smithi134.stdout:v0.47 2024-03-09T09:10:50.358 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.47 2024-03-09T09:10:50.358 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.47 2024-03-09T09:10:50.358 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.47 2024-03-09T09:11:00.627 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-03-09T09:11:00.628 INFO:tasks.ceph.mon.a:Sent signal 18 2024-03-09T09:11:00.628 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:11:00.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:11:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.831+0000 7fcbfb600640 1 Processor -- start 2024-03-09T09:11:00.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.832+0000 7fcbfb600640 1 -- start start 2024-03-09T09:11:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.833+0000 7fcbfb600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf406e8d0 0x7fcbf406f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.833+0000 7fcbfb600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcbf41593f0 0x7fcbf4159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.833+0000 7fcbfb600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf4155e00 con 0x7fcbf4150080 2024-03-09T09:11:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.833+0000 7fcbfb600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf41560c0 con 0x7fcbf41593f0 2024-03-09T09:11:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.833+0000 7fcbfb600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41563a0 con 0x7fcbf406e8d0 2024-03-09T09:11:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.833+0000 7fcbf8c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcbf4150080 0x7fcbf40c83d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38294/0 (socket says 172.21.15.67:38294) 2024-03-09T09:11:00.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.833+0000 7fcbf8c00640 1 -- 172.21.15.67:0/4043503748 learned_addr learned my addr 172.21.15.67:0/4043503748 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:11:00.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf9600640 1 --2- 172.21.15.67:0/4043503748 >> v2:172.21.15.134:3300/0 conn(0x7fcbf41593f0 0x7fcbf4159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:00.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3e00640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf406e8d0 0x7fcbf406f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:00.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817014091 0 0) 0x7fcbf4155e00 con 0x7fcbf4150080 2024-03-09T09:11:00.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbd8003520 con 0x7fcbf4150080 2024-03-09T09:11:00.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851741001 0 0) 0x7fcbd8003520 con 0x7fcbf4150080 2024-03-09T09:11:00.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf406e8d0 msgr2=0x7fcbf406f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:00.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf406e8d0 0x7fcbf406f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 >> v2:172.21.15.134:3300/0 conn(0x7fcbf41593f0 msgr2=0x7fcbf4159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 --2- 172.21.15.67:0/4043503748 >> v2:172.21.15.134:3300/0 conn(0x7fcbf41593f0 0x7fcbf4159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf4156680 con 0x7fcbf4150080 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fcbdc001670 con 0x7fcbf4150080 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 >> v1:172.21.15.67:6789/0 conn(0x7fcbf4150080 legacy=0x7fcbf40c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd80039b0 0x7fcbd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf406e8d0 0x7fcbd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.834+0000 7fcbf3400640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd8004d40 0x7fcbd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41560c0 con 0x7fcbd80039b0 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf41563a0 con 0x7fcbd8004d40 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd800ba40 con 0x7fcbf406e8d0 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf8c00640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd80039b0 0x7fcbd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf3e00640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf406e8d0 0x7fcbd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:00.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf9600640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd8004d40 0x7fcbd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:00.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf8c00640 1 -- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf406e8d0 msgr2=0x7fcbd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:00.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf8c00640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf406e8d0 0x7fcbd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:00.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf8c00640 1 -- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd8004d40 msgr2=0x7fcbd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:00.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf8c00640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd8004d40 0x7fcbd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:00.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf8c00640 1 -- 172.21.15.67:0/4043503748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd8003520 con 0x7fcbd80039b0 2024-03-09T09:11:00.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.835+0000 7fcbf3e00640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf406e8d0 0x7fcbd8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:11:00.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.836+0000 7fcbf8c00640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd80039b0 0x7fcbd8003dd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00c210 tx=0x7fcbd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f99e3c7e20d9f11 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:00.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.837+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc007d50 con 0x7fcbd80039b0 2024-03-09T09:11:00.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.837+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc030070 con 0x7fcbd80039b0 2024-03-09T09:11:00.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.837+0000 7fcbf3400640 1 -- 172.21.15.67:0/4043503748 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc02cd80 con 0x7fcbd80039b0 2024-03-09T09:11:00.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.837+0000 7fcbfb600640 1 -- 172.21.15.67:0/4043503748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd80039b0 msgr2=0x7fcbd8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:00.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.837+0000 7fcbfb600640 1 --2- 172.21.15.67:0/4043503748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbd80039b0 0x7fcbd8003dd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00c210 tx=0x7fcbd8005830 comp rx=0 tx=0).stop 2024-03-09T09:11:00.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.837+0000 7fcbf9600640 1 -- 172.21.15.67:0/4043503748 reap_dead start 2024-03-09T09:11:00.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.837+0000 7fcbfb600640 1 -- 172.21.15.67:0/4043503748 shutdown_connections 2024-03-09T09:11:00.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.837+0000 7fcbfb600640 1 -- 172.21.15.67:0/4043503748 >> 172.21.15.67:0/4043503748 conn(0x7fcbf40aa890 msgr2=0x7fcbf40ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:00.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.837+0000 7fcbfb600640 1 -- 172.21.15.67:0/4043503748 shutdown_connections 2024-03-09T09:11:00.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.838+0000 7fcbfb600640 1 -- 172.21.15.67:0/4043503748 wait complete. 2024-03-09T09:11:00.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.839+0000 7fcbfb600640 1 Processor -- start 2024-03-09T09:11:00.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.839+0000 7fcbfb600640 1 -- start start 2024-03-09T09:11:00.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.840+0000 7fcbfb600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd80039b0 0x7fcbf40721a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:00.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.840+0000 7fcbfb600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf41557f0 0x7fcbf414ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:00.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.840+0000 7fcbfb600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf41df4a0 0x7fcbf414b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:00.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.840+0000 7fcbfb600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40c8ae0 con 0x7fcbf41557f0 2024-03-09T09:11:00.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.840+0000 7fcbfb600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf4150460 con 0x7fcbd80039b0 2024-03-09T09:11:00.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.840+0000 7fcbfb600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf40c8c60 con 0x7fcbf41df4a0 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.840+0000 7fcbf3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf41557f0 0x7fcbf414ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.840+0000 7fcbf3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf41557f0 0x7fcbf414ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36950/0 (socket says 172.21.15.67:36950) 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.840+0000 7fcbf3e00640 1 -- 172.21.15.67:0/2131819694 learned_addr learned my addr 172.21.15.67:0/2131819694 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.841+0000 7fcbf8c00640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd80039b0 0x7fcbf40721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.841+0000 7fcbf9600640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf41df4a0 0x7fcbf414b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.841+0000 7fcbf3e00640 1 -- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf41df4a0 msgr2=0x7fcbf414b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.841+0000 7fcbf3e00640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf41df4a0 0x7fcbf414b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.841+0000 7fcbf3e00640 1 -- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd80039b0 msgr2=0x7fcbf40721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.841+0000 7fcbf3e00640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd80039b0 0x7fcbf40721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.841+0000 7fcbf3e00640 1 -- 172.21.15.67:0/2131819694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf4155cf0 con 0x7fcbf41557f0 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.841+0000 7fcbf3e00640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf41557f0 0x7fcbf414ad50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400f3b0 tx=0x7fcbe400f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.842+0000 7fcbf0c00640 1 -- 172.21.15.67:0/2131819694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4003b90 con 0x7fcbf41557f0 2024-03-09T09:11:00.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.842+0000 7fcbf0c00640 1 -- 172.21.15.67:0/2131819694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe4003d30 con 0x7fcbf41557f0 2024-03-09T09:11:00.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.842+0000 7fcbf0c00640 1 -- 172.21.15.67:0/2131819694 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4004220 con 0x7fcbf41557f0 2024-03-09T09:11:00.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.842+0000 7fcbfb600640 1 -- 172.21.15.67:0/2131819694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf41551c0 con 0x7fcbf41557f0 2024-03-09T09:11:00.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.842+0000 7fcbfb600640 1 -- 172.21.15.67:0/2131819694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf41def80 con 0x7fcbf41557f0 2024-03-09T09:11:00.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.843+0000 7fcbfb600640 1 -- 172.21.15.67:0/2131819694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf4075a00 con 0x7fcbf41557f0 2024-03-09T09:11:00.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.844+0000 7fcbf0c00640 1 -- 172.21.15.67:0/2131819694 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcbe4005d10 con 0x7fcbf41557f0 2024-03-09T09:11:00.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.845+0000 7fcbf0c00640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbd006b500 0x7fcbd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:00.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.845+0000 7fcbf0c00640 1 -- 172.21.15.67:0/2131819694 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcbe408acd0 con 0x7fcbf41557f0 2024-03-09T09:11:00.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.846+0000 7fcbf8c00640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbd006b500 0x7fcbd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:00.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.847+0000 7fcbf8c00640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbd006b500 0x7fcbd006d9c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc034000 tx=0x7fcbdc007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:00.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:00.854+0000 7fcbf0c00640 1 -- 172.21.15.67:0/2131819694 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcbe4012070 con 0x7fcbf41557f0 2024-03-09T09:11:01.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.185+0000 7fcbfb600640 1 -- 172.21.15.67:0/2131819694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcbf40ae1d0 con 0x7fcbf41557f0 2024-03-09T09:11:01.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.186+0000 7fcbf0c00640 1 -- 172.21.15.67:0/2131819694 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fcbe40596c0 con 0x7fcbf41557f0 2024-03-09T09:11:01.191 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":136,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:11:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 -- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbd006b500 msgr2=0x7fcbd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbd006b500 0x7fcbd006d9c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc034000 tx=0x7fcbdc007000 comp rx=0 tx=0).stop 2024-03-09T09:11:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 -- 172.21.15.67:0/2131819694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf41557f0 msgr2=0x7fcbf414ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf41557f0 0x7fcbf414ad50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400f3b0 tx=0x7fcbe400f870 comp rx=0 tx=0).stop 2024-03-09T09:11:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 -- 172.21.15.67:0/2131819694 shutdown_connections 2024-03-09T09:11:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbd006b500 0x7fcbd006d9c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbf41df4a0 0x7fcbf414b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbf41557f0 0x7fcbf414ad50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 --2- 172.21.15.67:0/2131819694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbd80039b0 0x7fcbf40721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 -- 172.21.15.67:0/2131819694 >> 172.21.15.67:0/2131819694 conn(0x7fcbf40aa890 msgr2=0x7fcbf41e12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:01.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 -- 172.21.15.67:0/2131819694 shutdown_connections 2024-03-09T09:11:01.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.189+0000 7fcbfb600640 1 -- 172.21.15.67:0/2131819694 wait complete. 2024-03-09T09:11:01.213 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:11:01.213 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:11:01.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:11:01.318 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 6% (29/481) Updating files: 7% (34/481) Updating files: 8% (39/481) Updating files: 9% (44/481) Updating files: 10% (49/481) Updating files: 11% (53/481) Updating files: 12% (58/481) Updating files: 13% (63/481) Updating files: 14% (68/481) Updating files: 15% (73/481) Updating files: 16% (77/481) Updating files: 17% (82/481) Updating files: 18% (87/481) Updating files: 19% (92/481) Updating files: 20% (97/481) Updating files: 20% (100/481) Updating files: 21% (102/481) Updating files: 22% (106/481) Updating files: 23% (111/481) Updating files: 24% (116/481) Updating files: 25% (121/481) Updating files: 26% (126/481) Updating files: 27% (130/481) Updating files: 28% (135/481) Updating files: 29% (140/481) Updating files: 30% (145/481) Updating files: 31% (150/481) Updating files: 32% (154/481) Updating files: 33% (159/481) Updating files: 34% (164/481) Updating files: 35% (169/481) Updating files: 36% (174/481) Updating files: 37% (178/481) Updating files: 38% (183/481) Updating files: 39% (188/481) Updating files: 40% (193/481) Updating files: 41% (198/481) Updating files: 42% (203/481) Updating files: 42% (204/481) Updating files: 43% (207/481) Updating files: 44% (212/481) Updating files: 45% (217/481) Updating files: 46% (222/481) Updating files: 47% (227/481) Updating files: 48% (231/481) Updating files: 49% (236/481) Updating files: 50% (241/481) Updating files: 51% (246/481) Updating files: 52% (251/481) Updating files: 53% (255/481) Updating files: 54% (260/481) Updating files: 54% (264/481) Updating files: 55% (265/481) Updating files: 56% (270/481) Updating files: 57% (275/481) Updating files: 58% (279/481) Updating files: 59% (284/481) Updating files: 60% (289/481) Updating files: 61% (294/481) Updating files: 61% (298/481) Updating files: 62% (299/481) Updating files: 63% (304/481) Updating files: 64% (308/481) Updating files: 65% (313/481) Updating files: 66% (318/481) Updating files: 67% (323/481) Updating files: 68% (328/481) Updating files: 69% (332/481) Updating files: 70% (337/481) Updating files: 71% (342/481) Updating files: 71% (344/481) Updating files: 72% (347/481) Updating files: 73% (352/481) Updating files: 74% (356/481) Updating files: 75% (361/481) Updating files: 76% (366/481) Updating files: 77% (371/481) Updating files: 78% (376/481) Updating files: 79% (380/481) Updating files: 80% (385/481) Updating files: 81% (390/481) Updating files: 82% (395/481) Updating files: 83% (400/481) Updating files: 84% (405/481) Updating files: 85% (409/481) Updating files: 86% (414/481) Updating files: 87% (419/481) Updating files: 88% (424/481) Updating files: 89% (429/481) Updating files: 90% (433/481) Updating files: 91% (438/481) Updating files: 92% (443/481) Updating files: 93% (448/481) Updating files: 94% (453/481) Updating files: 94% (455/481) Updating files: 95% (457/481) Updating files: 96% (462/481) Updating files: 97% (467/481) Updating files: 98% (472/481) Updating files: 99% (477/481) Updating files: 100% (481/481) Updating files: 100% (481/481), done. 2024-03-09T09:11:01.365 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at ca6265d0f4d v0.49 2024-03-09T09:11:01.378 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.49 2024-03-09T09:11:01.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.456+0000 7fd9c7000640 1 Processor -- start 2024-03-09T09:11:01.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.457+0000 7fd9c7000640 1 -- start start 2024-03-09T09:11:01.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.458+0000 7fd9c7000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c00c8c60 0x7fd9c00c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.458+0000 7fd9c7000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd9c00c9d50 0x7fd9c00caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.458+0000 7fd9c7000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c006b040 con 0x7fd9c0153460 2024-03-09T09:11:01.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.458+0000 7fd9c7000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c00ca590 con 0x7fd9c00c9d50 2024-03-09T09:11:01.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.458+0000 7fd9c7000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0156710 con 0x7fd9c00c8c60 2024-03-09T09:11:01.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.459+0000 7fd9bfe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd9c0153460 0x7fd9c0155ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38296/0 (socket says 172.21.15.67:38296) 2024-03-09T09:11:01.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.459+0000 7fd9bfe00640 1 -- 172.21.15.67:0/1909832877 learned_addr learned my addr 172.21.15.67:0/1909832877 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:11:01.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.459+0000 7fd9c5000640 1 --2- 172.21.15.67:0/1909832877 >> v2:172.21.15.134:3300/0 conn(0x7fd9c00c9d50 0x7fd9c00caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.459+0000 7fd9bf400640 1 --2- 172.21.15.67:0/1909832877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c00c8c60 0x7fd9c00c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.459+0000 7fd9bea00640 1 -- 172.21.15.67:0/1909832877 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875929195 0 0) 0x7fd9b0001110 con 0x7fd9c0153460 2024-03-09T09:11:01.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.459+0000 7fd9bea00640 1 -- 172.21.15.67:0/1909832877 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9a4003520 con 0x7fd9c0153460 2024-03-09T09:11:01.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.459+0000 7fd9bf400640 1 -- 172.21.15.67:0/1909832877 >> v2:172.21.15.134:3300/0 conn(0x7fd9c00c9d50 msgr2=0x7fd9c00caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:01.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.460+0000 7fd9bf400640 1 --2- 172.21.15.67:0/1909832877 >> v2:172.21.15.134:3300/0 conn(0x7fd9c00c9d50 0x7fd9c00caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.460+0000 7fd9bf400640 1 -- 172.21.15.67:0/1909832877 >> v1:172.21.15.67:6789/0 conn(0x7fd9c0153460 legacy=0x7fd9c0155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.460+0000 7fd9bf400640 1 -- 172.21.15.67:0/1909832877 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c01569f0 con 0x7fd9c00c8c60 2024-03-09T09:11:01.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.460+0000 7fd9c5000640 1 --2- 172.21.15.67:0/1909832877 >> v2:172.21.15.134:3300/0 conn(0x7fd9c00c9d50 0x7fd9c00caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:11:01.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.461+0000 7fd9bf400640 1 --2- 172.21.15.67:0/1909832877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c00c8c60 0x7fd9c00c9810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0097a0 tx=0x7fd9ac02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c2b4949eb04bdff server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:01.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.461+0000 7fd9bea00640 1 -- 172.21.15.67:0/1909832877 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac031780 con 0x7fd9c00c8c60 2024-03-09T09:11:01.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.461+0000 7fd9bea00640 1 -- 172.21.15.67:0/1909832877 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac037070 con 0x7fd9c00c8c60 2024-03-09T09:11:01.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.461+0000 7fd9bea00640 1 -- 172.21.15.67:0/1909832877 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac031b20 con 0x7fd9c00c8c60 2024-03-09T09:11:01.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.462+0000 7fd9c7000640 1 -- 172.21.15.67:0/1909832877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c00c8c60 msgr2=0x7fd9c00c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.462+0000 7fd9c7000640 1 --2- 172.21.15.67:0/1909832877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c00c8c60 0x7fd9c00c9810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0097a0 tx=0x7fd9ac02b820 comp rx=0 tx=0).stop 2024-03-09T09:11:01.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.462+0000 7fd9c7000640 1 -- 172.21.15.67:0/1909832877 shutdown_connections 2024-03-09T09:11:01.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.462+0000 7fd9c7000640 1 --2- 172.21.15.67:0/1909832877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c00c8c60 0x7fd9c00c9810 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.462+0000 7fd9c7000640 1 --2- 172.21.15.67:0/1909832877 >> v2:172.21.15.134:3300/0 conn(0x7fd9c00c9d50 0x7fd9c00caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.462+0000 7fd9c7000640 1 -- 172.21.15.67:0/1909832877 >> 172.21.15.67:0/1909832877 conn(0x7fd9c00c8170 msgr2=0x7fd9c00c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:01.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.462+0000 7fd9c7000640 1 -- 172.21.15.67:0/1909832877 shutdown_connections 2024-03-09T09:11:01.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.462+0000 7fd9c7000640 1 -- 172.21.15.67:0/1909832877 wait complete. 2024-03-09T09:11:01.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.464+0000 7fd9c7000640 1 Processor -- start 2024-03-09T09:11:01.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.464+0000 7fd9c7000640 1 -- start start 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.465+0000 7fd9c7000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c8c60 0x7fd9c0154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.465+0000 7fd9c7000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9c00c9d50 0x7fd9c0148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.465+0000 7fd9c7000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0153460 0x7fd9c0148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.465+0000 7fd9c7000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00caf60 con 0x7fd9c00c8c60 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.465+0000 7fd9c7000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0153840 con 0x7fd9c00c9d50 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.465+0000 7fd9c7000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00cb0e0 con 0x7fd9c0153460 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.465+0000 7fd9bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c8c60 0x7fd9c0154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.465+0000 7fd9bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c8c60 0x7fd9c0154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36958/0 (socket says 172.21.15.67:36958) 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.465+0000 7fd9bfe00640 1 -- 172.21.15.67:0/3883077880 learned_addr learned my addr 172.21.15.67:0/3883077880 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.466+0000 7fd9bf400640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9c00c9d50 0x7fd9c0148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.466+0000 7fd9c5000640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0153460 0x7fd9c0148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.466+0000 7fd9bfe00640 1 -- 172.21.15.67:0/3883077880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0153460 msgr2=0x7fd9c0148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.466+0000 7fd9bfe00640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd9c0153460 0x7fd9c0148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.466+0000 7fd9bfe00640 1 -- 172.21.15.67:0/3883077880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9c00c9d50 msgr2=0x7fd9c0148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.466+0000 7fd9bfe00640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9c00c9d50 0x7fd9c0148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.466+0000 7fd9bfe00640 1 -- 172.21.15.67:0/3883077880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c01efa70 con 0x7fd9c00c8c60 2024-03-09T09:11:01.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.467+0000 7fd9bfe00640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c8c60 0x7fd9c0154f60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c3a0 tx=0x7fd9b000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:01.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.467+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0017070 con 0x7fd9c00c8c60 2024-03-09T09:11:01.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.467+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0005ce0 con 0x7fd9c00c8c60 2024-03-09T09:11:01.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.467+0000 7fd9c7000640 1 -- 172.21.15.67:0/3883077880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c01599a0 con 0x7fd9c00c8c60 2024-03-09T09:11:01.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.467+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0007310 con 0x7fd9c00c8c60 2024-03-09T09:11:01.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.467+0000 7fd9c7000640 1 -- 172.21.15.67:0/3883077880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c0159f60 con 0x7fd9c00c8c60 2024-03-09T09:11:01.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.468+0000 7fd9c7000640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c0148e20 0x7fd9c00c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.468+0000 7fd9c7000640 1 -- 172.21.15.67:0/3883077880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00752b0 con 0x7fd9c0148e20 2024-03-09T09:11:01.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.468+0000 7fd9bf400640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c0148e20 0x7fd9c00c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.469+0000 7fd9bf400640 1 -- 172.21.15.67:0/3883077880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9c0075a00 con 0x7fd9c0148e20 2024-03-09T09:11:01.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.469+0000 7fd9bf400640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c0148e20 0x7fd9c00c3f60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00c80a0 tx=0x7fd9ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:01.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.470+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac04a070 con 0x7fd9c0148e20 2024-03-09T09:11:01.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.470+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd9b001a070 con 0x7fd9c00c8c60 2024-03-09T09:11:01.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.471+0000 7fd9b7e00640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd9a006b3d0 0x7fd9a006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.471+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd9b00898e0 con 0x7fd9c00c8c60 2024-03-09T09:11:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.471+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd9ac037040 con 0x7fd9c0148e20 2024-03-09T09:11:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.471+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c0148e20 msgr2=0x7fd9c00c3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.471+0000 7fd9b7e00640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c0148e20 0x7fd9c00c3f60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00c80a0 tx=0x7fd9ac0046c0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.471+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c0148e20 msgr2=0x7fd9c00c3f60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:01.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.471+0000 7fd9b7e00640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c0148e20 0x7fd9c00c3f60 secure :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00c80a0 tx=0x7fd9ac0046c0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.471+0000 7fd9c5000640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd9a006b3d0 0x7fd9a006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.472+0000 7fd9c5000640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd9a006b3d0 0x7fd9a006d890 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8007520 tx=0x7fd9b800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:01.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.485+0000 7fd9c7000640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c44a0 0x7fd9c00c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.485+0000 7fd9c7000640 1 -- 172.21.15.67:0/3883077880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00c7520 con 0x7fd9c00c44a0 2024-03-09T09:11:01.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.486+0000 7fd9bf400640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c44a0 0x7fd9c00c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.486+0000 7fd9bf400640 1 -- 172.21.15.67:0/3883077880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd9c00c95f0 con 0x7fd9c00c44a0 2024-03-09T09:11:01.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.486+0000 7fd9bf400640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c44a0 0x7fd9c00c4880 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac02b860 tx=0x7fd9ac031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:01.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.487+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac053070 con 0x7fd9c00c44a0 2024-03-09T09:11:01.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.487+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7fd9ac003e50 con 0x7fd9c00c44a0 2024-03-09T09:11:01.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.487+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c44a0 msgr2=0x7fd9c00c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.487+0000 7fd9b7e00640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c44a0 0x7fd9c00c4880 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac02b860 tx=0x7fd9ac031000 comp rx=0 tx=0).stop 2024-03-09T09:11:01.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.487+0000 7fd9b7e00640 1 -- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c44a0 msgr2=0x7fd9c00c4880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:01.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.487+0000 7fd9b7e00640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c44a0 0x7fd9c00c4880 secure :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac02b860 tx=0x7fd9ac031000 comp rx=0 tx=0).stop 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 136, 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:01.495 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:01.496 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.496 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:11:01.496 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:11:01.496 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:11:01.496 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:11:01.496 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:11:01.496 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:11:01.496 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:01.497 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:01.498 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.499 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:01.501 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.502 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.503 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.504 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:11:01.506 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:11:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.488+0000 7fd9c7000640 1 -- 172.21.15.67:0/3883077880 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd9a006b3d0 msgr2=0x7fd9a006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.488+0000 7fd9c7000640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd9a006b3d0 0x7fd9a006d890 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8007520 tx=0x7fd9b800b040 comp rx=0 tx=0).stop 2024-03-09T09:11:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.488+0000 7fd9c7000640 1 -- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c8c60 msgr2=0x7fd9c0154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.488+0000 7fd9c7000640 1 --2- 172.21.15.67:0/3883077880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9c00c8c60 0x7fd9c0154f60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c3a0 tx=0x7fd9b000c860 comp rx=0 tx=0).stop 2024-03-09T09:11:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.488+0000 7fd9c5000640 1 -- 172.21.15.67:0/3883077880 reap_dead start 2024-03-09T09:11:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.488+0000 7fd9c5000640 1 -- 172.21.15.67:0/3883077880 reap_dead start 2024-03-09T09:11:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.488+0000 7fd9c7000640 1 -- 172.21.15.67:0/3883077880 shutdown_connections 2024-03-09T09:11:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.488+0000 7fd9c7000640 1 -- 172.21.15.67:0/3883077880 >> 172.21.15.67:0/3883077880 conn(0x7fd9c00c8170 msgr2=0x7fd9c00c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.488+0000 7fd9c7000640 1 -- 172.21.15.67:0/3883077880 shutdown_connections 2024-03-09T09:11:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.488+0000 7fd9c7000640 1 -- 172.21.15.67:0/3883077880 wait complete. 2024-03-09T09:11:01.510 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:11:01.510 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:11:01.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:11:01.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.767+0000 7f76b7800640 1 Processor -- start 2024-03-09T09:11:01.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.767+0000 7f76b7800640 1 -- start start 2024-03-09T09:11:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.768+0000 7f76b7800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76b00c1d70 0x7f76b00c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.768+0000 7f76b7800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b00ac4e0 con 0x7f76b00c0ce0 2024-03-09T09:11:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.768+0000 7f76b7800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b00aa550 con 0x7f76b014c9a0 2024-03-09T09:11:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.768+0000 7f76b7800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b006d590 con 0x7f76b00c1d70 2024-03-09T09:11:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.769+0000 7f76b5800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f76b00c0ce0 0x7f76b00ac9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38312/0 (socket says 172.21.15.67:38312) 2024-03-09T09:11:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.769+0000 7f76b5800640 1 -- 172.21.15.67:0/807608747 learned_addr learned my addr 172.21.15.67:0/807608747 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:11:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.769+0000 7f76afe00640 1 --2- 172.21.15.67:0/807608747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76b00c1d70 0x7f76b00c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.769+0000 7f76af400640 1 -- 172.21.15.67:0/807608747 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224821597 0 0) 0x7f76b00ac4e0 con 0x7f76b00c0ce0 2024-03-09T09:11:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.769+0000 7f76af400640 1 -- 172.21.15.67:0/807608747 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7698003520 con 0x7f76b00c0ce0 2024-03-09T09:11:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.769+0000 7f76af400640 1 -- 172.21.15.67:0/807608747 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342334670 0 0) 0x7f76b00aa550 con 0x7f76b014c9a0 2024-03-09T09:11:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.769+0000 7f76afe00640 1 -- 172.21.15.67:0/807608747 >> v1:172.21.15.134:6789/0 conn(0x7f76b014c9a0 legacy=0x7f76b01567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.769+0000 7f76afe00640 1 -- 172.21.15.67:0/807608747 >> v1:172.21.15.67:6789/0 conn(0x7f76b00c0ce0 legacy=0x7f76b00ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.769+0000 7f76afe00640 1 -- 172.21.15.67:0/807608747 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b00aa830 con 0x7f76b00c1d70 2024-03-09T09:11:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.771+0000 7f76afe00640 1 --2- 172.21.15.67:0/807608747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76b00c1d70 0x7f76b00c07a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f769c0097d0 tx=0x7f769c02b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=748da5e72cd8e992 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.771+0000 7f76af400640 1 -- 172.21.15.67:0/807608747 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c037070 con 0x7f76b00c1d70 2024-03-09T09:11:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.771+0000 7f76af400640 1 -- 172.21.15.67:0/807608747 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c030710 con 0x7f76b00c1d70 2024-03-09T09:11:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.771+0000 7f76af400640 1 -- 172.21.15.67:0/807608747 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c030ac0 con 0x7f76b00c1d70 2024-03-09T09:11:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.772+0000 7f76b7800640 1 -- 172.21.15.67:0/807608747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76b00c1d70 msgr2=0x7f76b00c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.772+0000 7f76b7800640 1 --2- 172.21.15.67:0/807608747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76b00c1d70 0x7f76b00c07a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f769c0097d0 tx=0x7f769c02b770 comp rx=0 tx=0).stop 2024-03-09T09:11:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.772+0000 7f76b7800640 1 -- 172.21.15.67:0/807608747 shutdown_connections 2024-03-09T09:11:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.772+0000 7f76b7800640 1 --2- 172.21.15.67:0/807608747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76b00c1d70 0x7f76b00c07a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.772+0000 7f76b7800640 1 -- 172.21.15.67:0/807608747 >> 172.21.15.67:0/807608747 conn(0x7f76b00c7370 msgr2=0x7f76b00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.772+0000 7f76b7800640 1 -- 172.21.15.67:0/807608747 shutdown_connections 2024-03-09T09:11:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.772+0000 7f76b7800640 1 -- 172.21.15.67:0/807608747 wait complete. 2024-03-09T09:11:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.775+0000 7f76b7800640 1 Processor -- start 2024-03-09T09:11:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.775+0000 7f76b7800640 1 -- start start 2024-03-09T09:11:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.776+0000 7f76b7800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76b00c0ce0 0x7f76b014b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.776+0000 7f76b7800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b00c1d70 0x7f76b01eb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.776+0000 7f76b7800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b014c9a0 0x7f76b01eb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.776+0000 7f76b7800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00ad0d0 con 0x7f76b00c1d70 2024-03-09T09:11:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.776+0000 7f76b7800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00aadf0 con 0x7f76b014c9a0 2024-03-09T09:11:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.776+0000 7f76b7800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00ad250 con 0x7f76b00c0ce0 2024-03-09T09:11:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.777+0000 7f76afe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b00c1d70 0x7f76b01eb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.777+0000 7f76afe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b00c1d70 0x7f76b01eb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36996/0 (socket says 172.21.15.67:36996) 2024-03-09T09:11:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.777+0000 7f76afe00640 1 -- 172.21.15.67:0/4093077140 learned_addr learned my addr 172.21.15.67:0/4093077140 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.777+0000 7f76b4e00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76b00c0ce0 0x7f76b014b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.777+0000 7f76b5800640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b014c9a0 0x7f76b01eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.777+0000 7f76afe00640 1 -- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76b00c0ce0 msgr2=0x7f76b014b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.777+0000 7f76afe00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76b00c0ce0 0x7f76b014b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.777+0000 7f76afe00640 1 -- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b014c9a0 msgr2=0x7f76b01eb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.777+0000 7f76afe00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b014c9a0 0x7f76b01eb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.777+0000 7f76afe00640 1 -- 172.21.15.67:0/4093077140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b01ebbe0 con 0x7f76b00c1d70 2024-03-09T09:11:01.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.778+0000 7f76afe00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b00c1d70 0x7f76b01eb030 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f769c02b7b0 tx=0x7f769c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:01.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.778+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c042020 con 0x7f76b00c1d70 2024-03-09T09:11:01.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.778+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c030cf0 con 0x7f76b00c1d70 2024-03-09T09:11:01.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.778+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c037040 con 0x7f76b00c1d70 2024-03-09T09:11:01.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.778+0000 7f76b7800640 1 -- 172.21.15.67:0/4093077140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b01ebe80 con 0x7f76b00c1d70 2024-03-09T09:11:01.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.778+0000 7f76b7800640 1 -- 172.21.15.67:0/4093077140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b0152a50 con 0x7f76b00c1d70 2024-03-09T09:11:01.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.779+0000 7f76b7800640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b014b650 0x7f76b00720e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.779+0000 7f76b7800640 1 -- 172.21.15.67:0/4093077140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b0075a00 con 0x7f76b014b650 2024-03-09T09:11:01.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.780+0000 7f76b4e00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b014b650 0x7f76b00720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.780+0000 7f76b4e00640 1 -- 172.21.15.67:0/4093077140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76b00c15e0 con 0x7f76b014b650 2024-03-09T09:11:01.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.780+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f769c045020 con 0x7f76b00c1d70 2024-03-09T09:11:01.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.781+0000 7f76b4e00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b014b650 0x7f76b00720e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f76a400f540 tx=0x7f76a400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:01.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.781+0000 7f76acc00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f769006b4d0 0x7f769006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.781+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f769c0ac180 con 0x7f76b00c1d70 2024-03-09T09:11:01.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.782+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a40052e0 con 0x7f76b014b650 2024-03-09T09:11:01.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.782+0000 7f76b5800640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f769006b4d0 0x7f769006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.782+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f76a40052e0 con 0x7f76b014b650 2024-03-09T09:11:01.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.782+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b014b650 msgr2=0x7f76b00720e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.782+0000 7f76acc00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b014b650 0x7f76b00720e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f76a400f540 tx=0x7f76a400fa00 comp rx=0 tx=0).stop 2024-03-09T09:11:01.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.782+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b014b650 msgr2=0x7f76b00720e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:01.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.782+0000 7f76acc00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b014b650 0x7f76b00720e0 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f76a400f540 tx=0x7f76a400fa00 comp rx=0 tx=0).stop 2024-03-09T09:11:01.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.783+0000 7f76b5800640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f769006b4d0 0x7f769006d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f76a8001a60 tx=0x7f76a80019d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:01.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.789+0000 7f76b7800640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b00c37d0 0x7f76b0072620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:01.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.789+0000 7f76b7800640 1 -- 172.21.15.67:0/4093077140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b0067ef0 con 0x7f76b00c37d0 2024-03-09T09:11:01.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.790+0000 7f76b4e00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b00c37d0 0x7f76b0072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:01.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.790+0000 7f76b4e00640 1 -- 172.21.15.67:0/4093077140 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f76b00c49f0 con 0x7f76b00c37d0 2024-03-09T09:11:01.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.791+0000 7f76b4e00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b00c37d0 0x7f76b0072620 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f76a400f540 tx=0x7f76a4000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:01.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.791+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a4005490 con 0x7f76b00c37d0 2024-03-09T09:11:01.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f76a4003ad0 con 0x7f76b00c37d0 2024-03-09T09:11:01.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b00c37d0 msgr2=0x7f76b0072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76acc00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b00c37d0 0x7f76b0072620 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f76a400f540 tx=0x7f76a4000ed0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76acc00640 1 -- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b00c37d0 msgr2=0x7f76b0072620 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:01.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76acc00640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76b00c37d0 0x7f76b0072620 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.797 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:11:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:11:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 136, 2024-03-09T09:11:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:11:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:11:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:11:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:11:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:11:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:11:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:11:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:11:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:11:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:11:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:11:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:11:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:01.806 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:11:01.807 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.808 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:01.809 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:11:01.809 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:01.809 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:01.809 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:01.809 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:01.809 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:01.809 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:01.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:01.809 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:11:01.809 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:11:01.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76b7800640 1 -- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f769006b4d0 msgr2=0x7f769006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76b7800640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f769006b4d0 0x7f769006d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f76a8001a60 tx=0x7f76a80019d0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76b7800640 1 -- 172.21.15.67:0/4093077140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b00c1d70 msgr2=0x7f76b01eb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:01.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76b7800640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b00c1d70 0x7f76b01eb030 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f769c02b7b0 tx=0x7f769c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76b5800640 1 -- 172.21.15.67:0/4093077140 reap_dead start 2024-03-09T09:11:01.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76b7800640 1 -- 172.21.15.67:0/4093077140 shutdown_connections 2024-03-09T09:11:01.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76b7800640 1 --2- 172.21.15.67:0/4093077140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76b00c1d70 0x7f76b01eb030 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:01.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76b7800640 1 -- 172.21.15.67:0/4093077140 >> 172.21.15.67:0/4093077140 conn(0x7f76b00c7370 msgr2=0x7f76b00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:01.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.792+0000 7f76b7800640 1 -- 172.21.15.67:0/4093077140 shutdown_connections 2024-03-09T09:11:01.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:01.793+0000 7f76b7800640 1 -- 172.21.15.67:0/4093077140 wait complete. 2024-03-09T09:11:01.814 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:11:01.814 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:11:01.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:11:01.938 INFO:tasks.workunit.client.0.smithi067.stdout:v0.50 2024-03-09T09:11:01.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:11:01.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 50 -eq 48 ']' 2024-03-09T09:11:01.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.50 2024-03-09T09:11:01.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.50 2024-03-09T09:11:01.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.50 2024-03-09T09:11:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.026+0000 7f009b200640 1 Processor -- start 2024-03-09T09:11:02.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.026+0000 7f009b200640 1 -- start start 2024-03-09T09:11:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.027+0000 7f009b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f009c0ac930 0x7f009c0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.027+0000 7f009b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c151360 0x7f009c1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.028+0000 7f009b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009c0c2780 con 0x7f009c0c30a0 2024-03-09T09:11:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.028+0000 7f009b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009c070670 con 0x7f009c0ac930 2024-03-09T09:11:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.028+0000 7f009b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0708c0 con 0x7f009c151360 2024-03-09T09:11:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.028+0000 7f0099400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f009c0c30a0 0x7f009c150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38314/0 (socket says 172.21.15.67:38314) 2024-03-09T09:11:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.028+0000 7f0099e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f009c0ac930 0x7f009c0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.028+0000 7f0099400640 1 -- 172.21.15.67:0/3941592413 learned_addr learned my addr 172.21.15.67:0/3941592413 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:11:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.028+0000 7f009a800640 1 --2- 172.21.15.67:0/3941592413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c151360 0x7f009c1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.028+0000 7f0098a00640 1 -- 172.21.15.67:0/3941592413 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812573545 0 0) 0x7f009c0c2780 con 0x7f009c0c30a0 2024-03-09T09:11:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.028+0000 7f0098a00640 1 -- 172.21.15.67:0/3941592413 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f007c003520 con 0x7f009c0c30a0 2024-03-09T09:11:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.028+0000 7f0099e00640 1 -- 172.21.15.67:0/3941592413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c151360 msgr2=0x7f009c1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.028+0000 7f0099e00640 1 --2- 172.21.15.67:0/3941592413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c151360 0x7f009c1539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.029+0000 7f0099e00640 1 -- 172.21.15.67:0/3941592413 >> v1:172.21.15.67:6789/0 conn(0x7f009c0c30a0 legacy=0x7f009c150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.029+0000 7f0099e00640 1 -- 172.21.15.67:0/3941592413 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c070ba0 con 0x7f009c0ac930 2024-03-09T09:11:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.030+0000 7f0099e00640 1 --2- 172.21.15.67:0/3941592413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f009c0ac930 0x7f009c0c3660 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f008c0097a0 tx=0x7f008c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6161acdf258696 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.031+0000 7f0098a00640 1 -- 172.21.15.67:0/3941592413 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c035020 con 0x7f009c0ac930 2024-03-09T09:11:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.031+0000 7f0098a00640 1 -- 172.21.15.67:0/3941592413 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008c031810 con 0x7f009c0ac930 2024-03-09T09:11:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.031+0000 7f0098a00640 1 -- 172.21.15.67:0/3941592413 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c031bc0 con 0x7f009c0ac930 2024-03-09T09:11:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.031+0000 7f009b200640 1 -- 172.21.15.67:0/3941592413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f009c0ac930 msgr2=0x7f009c0c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.031+0000 7f009b200640 1 --2- 172.21.15.67:0/3941592413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f009c0ac930 0x7f009c0c3660 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f008c0097a0 tx=0x7f008c02b8b0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.031+0000 7f009b200640 1 -- 172.21.15.67:0/3941592413 shutdown_connections 2024-03-09T09:11:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.031+0000 7f009b200640 1 --2- 172.21.15.67:0/3941592413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c151360 0x7f009c1539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.031+0000 7f009b200640 1 --2- 172.21.15.67:0/3941592413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f009c0ac930 0x7f009c0c3660 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.031+0000 7f009b200640 1 -- 172.21.15.67:0/3941592413 >> 172.21.15.67:0/3941592413 conn(0x7f009c0c7220 msgr2=0x7f009c0c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.031+0000 7f009b200640 1 -- 172.21.15.67:0/3941592413 shutdown_connections 2024-03-09T09:11:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.031+0000 7f009b200640 1 -- 172.21.15.67:0/3941592413 wait complete. 2024-03-09T09:11:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.033+0000 7f009b200640 1 Processor -- start 2024-03-09T09:11:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.033+0000 7f009b200640 1 -- start start 2024-03-09T09:11:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.034+0000 7f009b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f009c0ac930 0x7f009c156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.034+0000 7f009b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f009c0c30a0 0x7f009c1e05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.034+0000 7f009b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c151360 0x7f009c152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.034+0000 7f009b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0ace90 con 0x7f009c0c30a0 2024-03-09T09:11:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.034+0000 7f009b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0acd10 con 0x7f009c0ac930 2024-03-09T09:11:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.034+0000 7f009b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c153f20 con 0x7f009c151360 2024-03-09T09:11:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.034+0000 7f0099400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f009c0c30a0 0x7f009c1e05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.034+0000 7f0099400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f009c0c30a0 0x7f009c1e05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37016/0 (socket says 172.21.15.67:37016) 2024-03-09T09:11:02.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.034+0000 7f0099400640 1 -- 172.21.15.67:0/3190235373 learned_addr learned my addr 172.21.15.67:0/3190235373 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:02.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.035+0000 7f009a800640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c151360 0x7f009c152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.035+0000 7f0099400640 1 -- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c151360 msgr2=0x7f009c152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.035+0000 7f0099400640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c151360 0x7f009c152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.035+0000 7f0099400640 1 -- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f009c0ac930 msgr2=0x7f009c156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.035+0000 7f0099400640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f009c0ac930 0x7f009c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.035+0000 7f0099400640 1 -- 172.21.15.67:0/3190235373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c14b2d0 con 0x7f009c0c30a0 2024-03-09T09:11:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.035+0000 7f0099400640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f009c0c30a0 0x7f009c1e05d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0084001460 tx=0x7f008400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.035+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084017070 con 0x7f009c0c30a0 2024-03-09T09:11:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.035+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008400ad00 con 0x7f009c0c30a0 2024-03-09T09:11:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.035+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084012640 con 0x7f009c0c30a0 2024-03-09T09:11:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.035+0000 7f009b200640 1 -- 172.21.15.67:0/3190235373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009c1e0b10 con 0x7f009c0c30a0 2024-03-09T09:11:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.036+0000 7f009b200640 1 -- 172.21.15.67:0/3190235373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009c1e9160 con 0x7f009c0c30a0 2024-03-09T09:11:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.036+0000 7f009b200640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c067b30 0x7f009c14bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.036+0000 7f009b200640 1 -- 172.21.15.67:0/3190235373 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c14c2c0 con 0x7f009c067b30 2024-03-09T09:11:02.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.037+0000 7f0099e00640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c067b30 0x7f009c14bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.038+0000 7f0099e00640 1 -- 172.21.15.67:0/3190235373 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f009c04f3c0 con 0x7f009c067b30 2024-03-09T09:11:02.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.038+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f008401a070 con 0x7f009c0c30a0 2024-03-09T09:11:02.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.038+0000 7f0099e00640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c067b30 0x7f009c14bd80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f009c0c8040 tx=0x7f008c032b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:02.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.039+0000 7f0092000640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f007806b500 0x7f007806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.039+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f008408a3f0 con 0x7f009c0c30a0 2024-03-09T09:11:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.039+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c035040 con 0x7f009c067b30 2024-03-09T09:11:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.039+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f008c003810 con 0x7f009c067b30 2024-03-09T09:11:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.039+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c067b30 msgr2=0x7f009c14bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.039+0000 7f0092000640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c067b30 0x7f009c14bd80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f009c0c8040 tx=0x7f008c032b50 comp rx=0 tx=0).stop 2024-03-09T09:11:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.039+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c067b30 msgr2=0x7f009c14bd80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.039+0000 7f0092000640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c067b30 0x7f009c14bd80 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f009c0c8040 tx=0x7f008c032b50 comp rx=0 tx=0).stop 2024-03-09T09:11:02.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.039+0000 7f009a800640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f007806b500 0x7f007806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.040+0000 7f009a800640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f007806b500 0x7f007806d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0094009a30 tx=0x7f0094008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:02.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.054+0000 7f009b200640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c1e9940 0x7f009c1e9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.054+0000 7f009b200640 1 -- 172.21.15.67:0/3190235373 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c1544c0 con 0x7f009c1e9940 2024-03-09T09:11:02.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.054+0000 7f0099e00640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c1e9940 0x7f009c1e9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.055+0000 7f0099e00640 1 -- 172.21.15.67:0/3190235373 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f009c153050 con 0x7f009c1e9940 2024-03-09T09:11:02.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.055+0000 7f0099e00640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c1e9940 0x7f009c1e9d20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f008c0097a0 tx=0x7f008c031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:02.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.056+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c035040 con 0x7f009c1e9940 2024-03-09T09:11:02.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.056+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4704 (secure 0 0 0) 0x7f008c0093c0 con 0x7f009c1e9940 2024-03-09T09:11:02.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.056+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c1e9940 msgr2=0x7f009c1e9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.056+0000 7f0092000640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c1e9940 0x7f009c1e9d20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f008c0097a0 tx=0x7f008c031000 comp rx=0 tx=0).stop 2024-03-09T09:11:02.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.056+0000 7f0092000640 1 -- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c1e9940 msgr2=0x7f009c1e9d20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:02.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.056+0000 7f0092000640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f009c1e9940 0x7f009c1e9d20 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f008c0097a0 tx=0x7f008c031000 comp rx=0 tx=0).stop 2024-03-09T09:11:02.062 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:11:02.062 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:02.062 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:02.062 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:11:02.062 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 136, 2024-03-09T09:11:02.062 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:11:02.062 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:11:02.062 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:02.063 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:11:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:11:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:11:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:11:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:11:02.075 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:11:02.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.057+0000 7f009b200640 1 -- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f007806b500 msgr2=0x7f007806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.057+0000 7f009b200640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f007806b500 0x7f007806d9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0094009a30 tx=0x7f0094008040 comp rx=0 tx=0).stop 2024-03-09T09:11:02.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.057+0000 7f009b200640 1 -- 172.21.15.67:0/3190235373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f009c0c30a0 msgr2=0x7f009c1e05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.057+0000 7f009b200640 1 --2- 172.21.15.67:0/3190235373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f009c0c30a0 0x7f009c1e05d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0084001460 tx=0x7f008400a430 comp rx=0 tx=0).stop 2024-03-09T09:11:02.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.057+0000 7f009a800640 1 -- 172.21.15.67:0/3190235373 reap_dead start 2024-03-09T09:11:02.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.057+0000 7f009a800640 1 -- 172.21.15.67:0/3190235373 reap_dead start 2024-03-09T09:11:02.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.057+0000 7f009b200640 1 -- 172.21.15.67:0/3190235373 shutdown_connections 2024-03-09T09:11:02.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.057+0000 7f009b200640 1 -- 172.21.15.67:0/3190235373 >> 172.21.15.67:0/3190235373 conn(0x7f009c0c7220 msgr2=0x7f009c073660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:02.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.057+0000 7f009b200640 1 -- 172.21.15.67:0/3190235373 shutdown_connections 2024-03-09T09:11:02.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.057+0000 7f009b200640 1 -- 172.21.15.67:0/3190235373 wait complete. 2024-03-09T09:11:02.078 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:11:02.078 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:11:02.078 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:11:02.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.283+0000 7f272aa00640 1 Processor -- start 2024-03-09T09:11:02.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.283+0000 7f272aa00640 1 -- start start 2024-03-09T09:11:02.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.284+0000 7f272aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f27240ac730 0x7f2724071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.285+0000 7f272aa00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27241581a0 con 0x7f2724157d60 2024-03-09T09:11:02.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.285+0000 7f272aa00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2724153d40 con 0x7f27240c22a0 2024-03-09T09:11:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.285+0000 7f272aa00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2724070ee0 con 0x7f27240ac730 2024-03-09T09:11:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.285+0000 7f2723e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f27240ac730 0x7f2724071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.285+0000 7f2723e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f27240ac730 0x7f2724071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37018/0 (socket says 172.21.15.67:37018) 2024-03-09T09:11:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.285+0000 7f2728a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2724157d60 0x7f272415e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38330/0 (socket says 172.21.15.67:38330) 2024-03-09T09:11:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.285+0000 7f2723e00640 1 -- 172.21.15.67:0/2104072244 learned_addr learned my addr 172.21.15.67:0/2104072244 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:02.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.285+0000 7f2723e00640 1 -- 172.21.15.67:0/2104072244 >> v1:172.21.15.134:6789/0 conn(0x7f27240c22a0 legacy=0x7f2724157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.285+0000 7f2723e00640 1 -- 172.21.15.67:0/2104072244 >> v1:172.21.15.67:6789/0 conn(0x7f2724157d60 legacy=0x7f272415e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.286+0000 7f2723e00640 1 -- 172.21.15.67:0/2104072244 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27240c2790 con 0x7f27240ac730 2024-03-09T09:11:02.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.286+0000 7f2722a00640 1 -- 172.21.15.67:0/2104072244 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131183302 0 0) 0x7f27241581a0 con 0x7f2724157d60 2024-03-09T09:11:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.287+0000 7f2723e00640 1 --2- 172.21.15.67:0/2104072244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27240ac730 0x7f2724071190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2714009870 tx=0x7f271402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837db98458487e93 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.288+0000 7f2722a00640 1 -- 172.21.15.67:0/2104072244 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714034070 con 0x7f27240ac730 2024-03-09T09:11:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.288+0000 7f2722a00640 1 -- 172.21.15.67:0/2104072244 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2714030860 con 0x7f27240ac730 2024-03-09T09:11:02.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.288+0000 7f2722a00640 1 -- 172.21.15.67:0/2104072244 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714030c30 con 0x7f27240ac730 2024-03-09T09:11:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.288+0000 7f272aa00640 1 -- 172.21.15.67:0/2104072244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27240ac730 msgr2=0x7f2724071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.288+0000 7f272aa00640 1 --2- 172.21.15.67:0/2104072244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27240ac730 0x7f2724071190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2714009870 tx=0x7f271402b9a0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.288+0000 7f272aa00640 1 -- 172.21.15.67:0/2104072244 shutdown_connections 2024-03-09T09:11:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.288+0000 7f272aa00640 1 --2- 172.21.15.67:0/2104072244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27240ac730 0x7f2724071190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.288+0000 7f272aa00640 1 -- 172.21.15.67:0/2104072244 >> 172.21.15.67:0/2104072244 conn(0x7f27240c7370 msgr2=0x7f27240c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.288+0000 7f272aa00640 1 -- 172.21.15.67:0/2104072244 shutdown_connections 2024-03-09T09:11:02.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.288+0000 7f272aa00640 1 -- 172.21.15.67:0/2104072244 wait complete. 2024-03-09T09:11:02.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.290+0000 7f272aa00640 1 Processor -- start 2024-03-09T09:11:02.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.290+0000 7f272aa00640 1 -- start start 2024-03-09T09:11:02.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.291+0000 7f272aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27240ac730 0x7f27240c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.291+0000 7f272aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27240c22a0 0x7f27240abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.291+0000 7f272aa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2724157d60 0x7f272414d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.291+0000 7f272aa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2724070c00 con 0x7f27240ac730 2024-03-09T09:11:02.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.291+0000 7f272aa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2724154d00 con 0x7f27240c22a0 2024-03-09T09:11:02.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.291+0000 7f272aa00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2724151610 con 0x7f2724157d60 2024-03-09T09:11:02.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.291+0000 7f2723e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27240ac730 0x7f27240c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.291+0000 7f2723e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27240ac730 0x7f27240c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37026/0 (socket says 172.21.15.67:37026) 2024-03-09T09:11:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.291+0000 7f2723e00640 1 -- 172.21.15.67:0/1154461505 learned_addr learned my addr 172.21.15.67:0/1154461505 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.292+0000 7f2728a00640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2724157d60 0x7f272414d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.292+0000 7f2723400640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27240c22a0 0x7f27240abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.292+0000 7f2723e00640 1 -- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2724157d60 msgr2=0x7f272414d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.292+0000 7f2723e00640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2724157d60 0x7f272414d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.292+0000 7f2723e00640 1 -- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27240c22a0 msgr2=0x7f27240abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.292+0000 7f2723e00640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27240c22a0 0x7f27240abd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.292+0000 7f2723e00640 1 -- 172.21.15.67:0/1154461505 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27241ef330 con 0x7f27240ac730 2024-03-09T09:11:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.292+0000 7f2723e00640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27240ac730 0x7f27240c0a10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2714009870 tx=0x7f27140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.292+0000 7f271be00640 1 -- 172.21.15.67:0/1154461505 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714034040 con 0x7f27240ac730 2024-03-09T09:11:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.293+0000 7f271be00640 1 -- 172.21.15.67:0/1154461505 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271403ad10 con 0x7f27240ac730 2024-03-09T09:11:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.293+0000 7f271be00640 1 -- 172.21.15.67:0/1154461505 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714004270 con 0x7f27240ac730 2024-03-09T09:11:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.293+0000 7f272aa00640 1 -- 172.21.15.67:0/1154461505 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27241ef610 con 0x7f27240ac730 2024-03-09T09:11:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.293+0000 7f272aa00640 1 -- 172.21.15.67:0/1154461505 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27241efbd0 con 0x7f27240ac730 2024-03-09T09:11:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.294+0000 7f272aa00640 1 -- 172.21.15.67:0/1154461505 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2724075a00 con 0x7f27240ac730 2024-03-09T09:11:02.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.296+0000 7f271be00640 1 -- 172.21.15.67:0/1154461505 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2714037070 con 0x7f27240ac730 2024-03-09T09:11:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.296+0000 7f271be00640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f270406b4d0 0x7f270406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.297+0000 7f271be00640 1 -- 172.21.15.67:0/1154461505 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f27140ac6a0 con 0x7f27240ac730 2024-03-09T09:11:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.297+0000 7f2723400640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f270406b4d0 0x7f270406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:02.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.298+0000 7f2723400640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f270406b4d0 0x7f270406d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f27240ab7a0 tx=0x7f2710009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.301+0000 7f271be00640 1 -- 172.21.15.67:0/1154461505 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f271403f070 con 0x7f27240ac730 2024-03-09T09:11:02.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.638+0000 7f272aa00640 1 -- 172.21.15.67:0/1154461505 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f27240c1d70 con 0x7f27240ac730 2024-03-09T09:11:02.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.640+0000 7f271be00640 1 -- 172.21.15.67:0/1154461505 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f271407b240 con 0x7f27240ac730 2024-03-09T09:11:02.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.643+0000 7f272aa00640 1 -- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f270406b4d0 msgr2=0x7f270406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.643+0000 7f272aa00640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f270406b4d0 0x7f270406d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f27240ab7a0 tx=0x7f2710009040 comp rx=0 tx=0).stop 2024-03-09T09:11:02.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.643+0000 7f272aa00640 1 -- 172.21.15.67:0/1154461505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27240ac730 msgr2=0x7f27240c0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:02.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.643+0000 7f272aa00640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27240ac730 0x7f27240c0a10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2714009870 tx=0x7f27140046c0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.643+0000 7f272aa00640 1 -- 172.21.15.67:0/1154461505 shutdown_connections 2024-03-09T09:11:02.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.643+0000 7f272aa00640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f270406b4d0 0x7f270406d990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.643+0000 7f272aa00640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2724157d60 0x7f272414d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.643+0000 7f272aa00640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27240c22a0 0x7f27240abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.643+0000 7f272aa00640 1 --2- 172.21.15.67:0/1154461505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27240ac730 0x7f27240c0a10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:02.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.643+0000 7f272aa00640 1 -- 172.21.15.67:0/1154461505 >> 172.21.15.67:0/1154461505 conn(0x7f27240c7370 msgr2=0x7f27240713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:02.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.644+0000 7f272aa00640 1 -- 172.21.15.67:0/1154461505 shutdown_connections 2024-03-09T09:11:02.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:02.644+0000 7f272aa00640 1 -- 172.21.15.67:0/1154461505 wait complete. 2024-03-09T09:11:02.667 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:11:03.578 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 45% (99/219) Updating files: 46% (101/219) Updating files: 47% (103/219) Updating files: 48% (106/219) Updating files: 49% (108/219) Updating files: 50% (110/219) Updating files: 51% (112/219) Updating files: 52% (114/219) Updating files: 53% (117/219) Updating files: 54% (119/219) Updating files: 55% (121/219) Updating files: 56% (123/219) Updating files: 57% (125/219) Updating files: 58% (128/219) Updating files: 59% (130/219) Updating files: 60% (132/219) Updating files: 61% (134/219) Updating files: 62% (136/219) Updating files: 63% (138/219) Updating files: 64% (141/219) Updating files: 65% (143/219) Updating files: 66% (145/219) Updating files: 67% (147/219) Updating files: 68% (149/219) Updating files: 69% (152/219) Updating files: 70% (154/219) Updating files: 71% (156/219) Updating files: 72% (158/219) Updating files: 73% (160/219) Updating files: 74% (163/219) Updating files: 75% (165/219) Updating files: 76% (167/219) Updating files: 77% (169/219) Updating files: 78% (171/219) Updating files: 79% (174/219) Updating files: 80% (176/219) Updating files: 81% (178/219) Updating files: 82% (180/219) Updating files: 83% (182/219) Updating files: 84% (184/219) Updating files: 85% (187/219) Updating files: 86% (189/219) Updating files: 87% (191/219) Updating files: 88% (193/219) Updating files: 89% (195/219) Updating files: 90% (198/219) Updating files: 91% (200/219) Updating files: 92% (202/219) Updating files: 93% (204/219) Updating files: 94% (206/219) Updating files: 95% (209/219) Updating files: 96% (211/219) Updating files: 97% (213/219) Updating files: 98% (215/219) Updating files: 99% (217/219) Updating files: 100% (219/219) Updating files: 100% (219/219), done. 2024-03-09T09:11:03.716 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at ad663d58c89 v0.47 2024-03-09T09:11:03.724 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.47 2024-03-09T09:11:03.832 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:11:03.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 48 -eq 48 ']' 2024-03-09T09:11:03.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ continue 2024-03-09T09:11:03.833 INFO:tasks.workunit.client.1.smithi134.stdout:v0.49 2024-03-09T09:11:03.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:11:03.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 49 -eq 48 ']' 2024-03-09T09:11:03.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.49 2024-03-09T09:11:03.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.49 2024-03-09T09:11:03.834 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.49 2024-03-09T09:11:12.668 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:11:12.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:11:12.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.918+0000 7f687d200640 1 Processor -- start 2024-03-09T09:11:12.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.918+0000 7f687d200640 1 -- start start 2024-03-09T09:11:12.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.919+0000 7f687d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6878153500 0x7f687806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:12.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.919+0000 7f687d200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f68780c6630 0x7f68780c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:12.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.919+0000 7f687d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f68780c5f60 0x7f68780c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:12.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.919+0000 7f687d200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6878159d70 con 0x7f68780c5f60 2024-03-09T09:11:12.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.919+0000 7f687d200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687815a2a0 con 0x7f68780c6630 2024-03-09T09:11:12.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.919+0000 7f687d200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687806bb30 con 0x7f6878153500 2024-03-09T09:11:12.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.920+0000 7f6876800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f68780c5f60 0x7f68780c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:12.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.920+0000 7f6875e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6878153500 0x7f687806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:12.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.920+0000 7f6875400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f68780c6630 0x7f68780c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:12.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.920+0000 7f6876800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f68780c5f60 0x7f68780c79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35298/0 (socket says 172.21.15.67:35298) 2024-03-09T09:11:12.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.920+0000 7f6876800640 1 -- 172.21.15.67:0/3546815443 learned_addr learned my addr 172.21.15.67:0/3546815443 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:12.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.920+0000 7f6876800640 1 -- 172.21.15.67:0/3546815443 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6878153500 msgr2=0x7f687806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:12.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.920+0000 7f6876800640 1 --2- 172.21.15.67:0/3546815443 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6878153500 0x7f687806b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:12.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.920+0000 7f6876800640 1 -- 172.21.15.67:0/3546815443 >> v2:172.21.15.134:3300/0 conn(0x7f68780c6630 msgr2=0x7f68780c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:12.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.920+0000 7f6876800640 1 --2- 172.21.15.67:0/3546815443 >> v2:172.21.15.134:3300/0 conn(0x7f68780c6630 0x7f68780c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:12.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.920+0000 7f6876800640 1 -- 172.21.15.67:0/3546815443 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6878159fc0 con 0x7f68780c5f60 2024-03-09T09:11:12.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.922+0000 7f6876800640 1 --2- 172.21.15.67:0/3546815443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780c5f60 0x7f68780c79f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f687808d330 tx=0x7f687002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a14925950d6d93f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:12.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.923+0000 7f6874a00640 1 -- 172.21.15.67:0/3546815443 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687002c950 con 0x7f68780c5f60 2024-03-09T09:11:12.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.923+0000 7f6874a00640 1 -- 172.21.15.67:0/3546815443 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687002caf0 con 0x7f68780c5f60 2024-03-09T09:11:12.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.923+0000 7f6874a00640 1 -- 172.21.15.67:0/3546815443 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68700343f0 con 0x7f68780c5f60 2024-03-09T09:11:12.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.923+0000 7f687d200640 1 -- 172.21.15.67:0/3546815443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780c5f60 msgr2=0x7f68780c79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:12.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.923+0000 7f687d200640 1 --2- 172.21.15.67:0/3546815443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780c5f60 0x7f68780c79f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f687808d330 tx=0x7f687002bdb0 comp rx=0 tx=0).stop 2024-03-09T09:11:12.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.924+0000 7f687d200640 1 -- 172.21.15.67:0/3546815443 shutdown_connections 2024-03-09T09:11:12.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.924+0000 7f687d200640 1 --2- 172.21.15.67:0/3546815443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780c5f60 0x7f68780c79f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:12.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.924+0000 7f687d200640 1 --2- 172.21.15.67:0/3546815443 >> v2:172.21.15.134:3300/0 conn(0x7f68780c6630 0x7f68780c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:12.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.924+0000 7f687d200640 1 --2- 172.21.15.67:0/3546815443 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6878153500 0x7f687806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:12.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.924+0000 7f687d200640 1 -- 172.21.15.67:0/3546815443 >> 172.21.15.67:0/3546815443 conn(0x7f68780c8060 msgr2=0x7f68780c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:12.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.924+0000 7f687d200640 1 -- 172.21.15.67:0/3546815443 shutdown_connections 2024-03-09T09:11:12.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.924+0000 7f687d200640 1 -- 172.21.15.67:0/3546815443 wait complete. 2024-03-09T09:11:12.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.925+0000 7f687d200640 1 Processor -- start 2024-03-09T09:11:12.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.926+0000 7f687d200640 1 -- start start 2024-03-09T09:11:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.927+0000 7f687d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780c5f60 0x7f68781586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.927+0000 7f687d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6878153500 0x7f68781e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.927+0000 7f687d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68781e2f50 0x7f68781eba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:12.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.927+0000 7f687d200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68780bed90 con 0x7f68780c5f60 2024-03-09T09:11:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.927+0000 7f687d200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68780c6f90 con 0x7f6878153500 2024-03-09T09:11:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.927+0000 7f687d200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68780c94e0 con 0x7f68781e2f50 2024-03-09T09:11:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.927+0000 7f6875e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780c5f60 0x7f68781586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.928+0000 7f6875e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780c5f60 0x7f68781586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35310/0 (socket says 172.21.15.67:35310) 2024-03-09T09:11:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.928+0000 7f6875e00640 1 -- 172.21.15.67:0/2138593065 learned_addr learned my addr 172.21.15.67:0/2138593065 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.928+0000 7f6876800640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68781e2f50 0x7f68781eba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:12.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.928+0000 7f6875400640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6878153500 0x7f68781e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:12.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.928+0000 7f6875e00640 1 -- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68781e2f50 msgr2=0x7f68781eba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:12.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.928+0000 7f6875e00640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68781e2f50 0x7f68781eba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:12.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.928+0000 7f6875e00640 1 -- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6878153500 msgr2=0x7f68781e2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:12.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.928+0000 7f6875e00640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6878153500 0x7f68781e2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:12.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.928+0000 7f6875e00640 1 -- 172.21.15.67:0/2138593065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6878158c20 con 0x7f68780c5f60 2024-03-09T09:11:12.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.928+0000 7f6875e00640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780c5f60 0x7f68781586e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f686800ea10 tx=0x7f686800eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:12.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.929+0000 7f686e000640 1 -- 172.21.15.67:0/2138593065 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868017070 con 0x7f68780c5f60 2024-03-09T09:11:12.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.929+0000 7f686e000640 1 -- 172.21.15.67:0/2138593065 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686800ca00 con 0x7f68780c5f60 2024-03-09T09:11:12.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.929+0000 7f686e000640 1 -- 172.21.15.67:0/2138593065 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800cdf0 con 0x7f68780c5f60 2024-03-09T09:11:12.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.929+0000 7f687d200640 1 -- 172.21.15.67:0/2138593065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6878158ec0 con 0x7f68780c5f60 2024-03-09T09:11:12.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.929+0000 7f687d200640 1 -- 172.21.15.67:0/2138593065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68781ef250 con 0x7f68780c5f60 2024-03-09T09:11:12.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.930+0000 7f687d200640 1 -- 172.21.15.67:0/2138593065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6878075a00 con 0x7f68780c5f60 2024-03-09T09:11:12.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.932+0000 7f686e000640 1 -- 172.21.15.67:0/2138593065 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f686801a070 con 0x7f68780c5f60 2024-03-09T09:11:12.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.933+0000 7f686e000640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f685806b500 0x7f685806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:12.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.933+0000 7f686e000640 1 -- 172.21.15.67:0/2138593065 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f686808acc0 con 0x7f68780c5f60 2024-03-09T09:11:12.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.934+0000 7f6875400640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f685806b500 0x7f685806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:12.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.935+0000 7f6875400640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f685806b500 0x7f685806d9c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6878150cb0 tx=0x7f6864009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:12.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:12.940+0000 7f686e000640 1 -- 172.21.15.67:0/2138593065 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6868059170 con 0x7f68780c5f60 2024-03-09T09:11:13.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.345+0000 7f687d200640 1 -- 172.21.15.67:0/2138593065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f68780c7bc0 con 0x7f68780c5f60 2024-03-09T09:11:13.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.345+0000 7f686e000640 1 -- 172.21.15.67:0/2138593065 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f686805d210 con 0x7f68780c5f60 2024-03-09T09:11:13.350 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":136,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:11:13.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 -- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f685806b500 msgr2=0x7f685806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f685806b500 0x7f685806d9c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6878150cb0 tx=0x7f6864009190 comp rx=0 tx=0).stop 2024-03-09T09:11:13.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 -- 172.21.15.67:0/2138593065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780c5f60 msgr2=0x7f68781586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780c5f60 0x7f68781586e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f686800ea10 tx=0x7f686800eed0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 -- 172.21.15.67:0/2138593065 shutdown_connections 2024-03-09T09:11:13.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f685806b500 0x7f685806d9c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68781e2f50 0x7f68781eba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6878153500 0x7f68781e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 --2- 172.21.15.67:0/2138593065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68780c5f60 0x7f68781586e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 -- 172.21.15.67:0/2138593065 >> 172.21.15.67:0/2138593065 conn(0x7f68780c8060 msgr2=0x7f68780c8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:13.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 -- 172.21.15.67:0/2138593065 shutdown_connections 2024-03-09T09:11:13.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.348+0000 7f687d200640 1 -- 172.21.15.67:0/2138593065 wait complete. 2024-03-09T09:11:13.369 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:11:13.369 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:11:13.369 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:11:13.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:11:13.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.564+0000 7f3a6b000640 1 Processor -- start 2024-03-09T09:11:13.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.564+0000 7f3a6b000640 1 -- start start 2024-03-09T09:11:13.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.566+0000 7f3a6b000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3a64153220 0x7f3a640c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.566+0000 7f3a6b000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3a640c8860 0x7f3a64159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.566+0000 7f3a6b000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a64159850 0x7f3a640c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a6b000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a640c9b10 con 0x7f3a640c8860 2024-03-09T09:11:13.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a6b000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a64159d60 con 0x7f3a64153220 2024-03-09T09:11:13.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a6b000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6415a040 con 0x7f3a64159850 2024-03-09T09:11:13.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a69200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3a640c8860 0x7f3a64159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a69200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3a640c8860 0x7f3a64159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35326/0 (socket says 172.21.15.67:35326) 2024-03-09T09:11:13.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a69200640 1 -- 172.21.15.67:0/2992599591 learned_addr learned my addr 172.21.15.67:0/2992599591 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:13.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a6a600640 1 --2- 172.21.15.67:0/2992599591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a64159850 0x7f3a640c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a69c00640 1 --2- 172.21.15.67:0/2992599591 >> v2:172.21.15.134:3300/0 conn(0x7f3a64153220 0x7f3a640c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a69200640 1 -- 172.21.15.67:0/2992599591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a64159850 msgr2=0x7f3a640c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:13.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a69200640 1 --2- 172.21.15.67:0/2992599591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a64159850 0x7f3a640c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a69200640 1 -- 172.21.15.67:0/2992599591 >> v2:172.21.15.134:3300/0 conn(0x7f3a64153220 msgr2=0x7f3a640c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:13.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a69200640 1 --2- 172.21.15.67:0/2992599591 >> v2:172.21.15.134:3300/0 conn(0x7f3a64153220 0x7f3a640c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a69200640 1 -- 172.21.15.67:0/2992599591 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a640c8060 con 0x7f3a640c8860 2024-03-09T09:11:13.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a6a600640 1 --2- 172.21.15.67:0/2992599591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a64159850 0x7f3a640c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:11:13.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.567+0000 7f3a69c00640 1 --2- 172.21.15.67:0/2992599591 >> v2:172.21.15.134:3300/0 conn(0x7f3a64153220 0x7f3a640c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:11:13.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.569+0000 7f3a69200640 1 --2- 172.21.15.67:0/2992599591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a640c8860 0x7f3a64159310 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3a6406df50 tx=0x7f3a5002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cb2f27fbdf7dbb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:13.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.569+0000 7f3a63e00640 1 -- 172.21.15.67:0/2992599591 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a500317d0 con 0x7f3a640c8860 2024-03-09T09:11:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.569+0000 7f3a63e00640 1 -- 172.21.15.67:0/2992599591 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50037070 con 0x7f3a640c8860 2024-03-09T09:11:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.569+0000 7f3a63e00640 1 -- 172.21.15.67:0/2992599591 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50031b90 con 0x7f3a640c8860 2024-03-09T09:11:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.570+0000 7f3a6b000640 1 -- 172.21.15.67:0/2992599591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a640c8860 msgr2=0x7f3a64159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.570+0000 7f3a6b000640 1 --2- 172.21.15.67:0/2992599591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a640c8860 0x7f3a64159310 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3a6406df50 tx=0x7f3a5002b870 comp rx=0 tx=0).stop 2024-03-09T09:11:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.570+0000 7f3a6b000640 1 -- 172.21.15.67:0/2992599591 shutdown_connections 2024-03-09T09:11:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.570+0000 7f3a6b000640 1 --2- 172.21.15.67:0/2992599591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a64159850 0x7f3a640c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.570+0000 7f3a6b000640 1 --2- 172.21.15.67:0/2992599591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a640c8860 0x7f3a64159310 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.570+0000 7f3a6b000640 1 --2- 172.21.15.67:0/2992599591 >> v2:172.21.15.134:3300/0 conn(0x7f3a64153220 0x7f3a640c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.570+0000 7f3a6b000640 1 -- 172.21.15.67:0/2992599591 >> 172.21.15.67:0/2992599591 conn(0x7f3a640ad060 msgr2=0x7f3a640ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.570+0000 7f3a6b000640 1 -- 172.21.15.67:0/2992599591 shutdown_connections 2024-03-09T09:11:13.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.570+0000 7f3a6b000640 1 -- 172.21.15.67:0/2992599591 wait complete. 2024-03-09T09:11:13.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.572+0000 7f3a6b000640 1 Processor -- start 2024-03-09T09:11:13.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.573+0000 7f3a6b000640 1 -- start start 2024-03-09T09:11:13.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.574+0000 7f3a6b000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a640c8860 0x7f3a64158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.574+0000 7f3a6b000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a64153220 0x7f3a641ec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.574+0000 7f3a6b000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64159850 0x7f3a641ef390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.574+0000 7f3a6b000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a641560a0 con 0x7f3a64159850 2024-03-09T09:11:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.574+0000 7f3a6b000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a64155f20 con 0x7f3a64153220 2024-03-09T09:11:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.574+0000 7f3a6b000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a64153600 con 0x7f3a640c8860 2024-03-09T09:11:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.574+0000 7f3a6a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64159850 0x7f3a641ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.574+0000 7f3a6a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64159850 0x7f3a641ef390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35340/0 (socket says 172.21.15.67:35340) 2024-03-09T09:11:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.574+0000 7f3a6a600640 1 -- 172.21.15.67:0/2391087936 learned_addr learned my addr 172.21.15.67:0/2391087936 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.574+0000 7f3a69200640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a64153220 0x7f3a641ec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.574+0000 7f3a69c00640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a640c8860 0x7f3a64158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.575+0000 7f3a6a600640 1 -- 172.21.15.67:0/2391087936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a640c8860 msgr2=0x7f3a64158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.575+0000 7f3a6a600640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a640c8860 0x7f3a64158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.575+0000 7f3a6a600640 1 -- 172.21.15.67:0/2391087936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a64153220 msgr2=0x7f3a641ec8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.575+0000 7f3a6a600640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a64153220 0x7f3a641ec8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.575+0000 7f3a6a600640 1 -- 172.21.15.67:0/2391087936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a641ef8d0 con 0x7f3a64159850 2024-03-09T09:11:13.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.575+0000 7f3a6a600640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64159850 0x7f3a641ef390 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00ce40 tx=0x7f3a5c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:13.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.576+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c01a070 con 0x7f3a64159850 2024-03-09T09:11:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.576+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c00d840 con 0x7f3a64159850 2024-03-09T09:11:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.576+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c00dc30 con 0x7f3a64159850 2024-03-09T09:11:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.576+0000 7f3a6b000640 1 -- 172.21.15.67:0/2391087936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a64158e30 con 0x7f3a64159850 2024-03-09T09:11:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.576+0000 7f3a6b000640 1 -- 172.21.15.67:0/2391087936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a64157180 con 0x7f3a64159850 2024-03-09T09:11:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.577+0000 7f3a6b000640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64151c10 0x7f3a64157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.577+0000 7f3a6b000640 1 -- 172.21.15.67:0/2391087936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a64156fe0 con 0x7f3a64151c10 2024-03-09T09:11:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.578+0000 7f3a69c00640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64151c10 0x7f3a64157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.578+0000 7f3a69c00640 1 -- 172.21.15.67:0/2391087936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a641ece00 con 0x7f3a64151c10 2024-03-09T09:11:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.578+0000 7f3a69c00640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64151c10 0x7f3a64157960 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800e770 tx=0x7f3a5800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.579+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58019070 con 0x7f3a64151c10 2024-03-09T09:11:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.579+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3a5c01d070 con 0x7f3a64159850 2024-03-09T09:11:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.580+0000 7f3a61600640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3a4406b400 0x7f3a4406d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.580+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f3a5c08ac80 con 0x7f3a64159850 2024-03-09T09:11:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.580+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f3a58004030 con 0x7f3a64151c10 2024-03-09T09:11:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.580+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64151c10 msgr2=0x7f3a64157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.580+0000 7f3a61600640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64151c10 0x7f3a64157960 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800e770 tx=0x7f3a5800ec30 comp rx=0 tx=0).stop 2024-03-09T09:11:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.580+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64151c10 msgr2=0x7f3a64157960 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:13.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.580+0000 7f3a61600640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64151c10 0x7f3a64157960 secure :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800e770 tx=0x7f3a5800ec30 comp rx=0 tx=0).stop 2024-03-09T09:11:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.580+0000 7f3a69200640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3a4406b400 0x7f3a4406d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.581+0000 7f3a69200640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3a4406b400 0x7f3a4406d8c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3a50036000 tx=0x7f3a5003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.588+0000 7f3a6b000640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6414c6d0 0x7f3a6414cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.588+0000 7f3a6b000640 1 -- 172.21.15.67:0/2391087936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a64159350 con 0x7f3a6414c6d0 2024-03-09T09:11:13.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.588+0000 7f3a69c00640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6414c6d0 0x7f3a6414cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.588+0000 7f3a69c00640 1 -- 172.21.15.67:0/2391087936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3a6404f3c0 con 0x7f3a6414c6d0 2024-03-09T09:11:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.588+0000 7f3a69c00640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6414c6d0 0x7f3a6414cab0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3a58018000 tx=0x7f3a580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.589+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58003610 con 0x7f3a6414c6d0 2024-03-09T09:11:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.589+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7f3a58003610 con 0x7f3a6414c6d0 2024-03-09T09:11:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.589+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6414c6d0 msgr2=0x7f3a6414cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.589+0000 7f3a61600640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6414c6d0 0x7f3a6414cab0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3a58018000 tx=0x7f3a580046c0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.589+0000 7f3a61600640 1 -- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6414c6d0 msgr2=0x7f3a6414cab0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.589+0000 7f3a61600640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a6414c6d0 0x7f3a6414cab0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.594 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 136, 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:11:13.595 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:11:13.596 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.597 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:11:13.598 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:13.599 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.601 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.602 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:13.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:11:13.604 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.605 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:13.606 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:11:13.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.606 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:13.606 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:13.606 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:13.606 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.606 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:13.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:11:13.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.607 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.609 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:11:13.609 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:11:13.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.590+0000 7f3a6b000640 1 -- 172.21.15.67:0/2391087936 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3a4406b400 msgr2=0x7f3a4406d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.590+0000 7f3a6b000640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3a4406b400 0x7f3a4406d8c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3a50036000 tx=0x7f3a5003d040 comp rx=0 tx=0).stop 2024-03-09T09:11:13.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.590+0000 7f3a6b000640 1 -- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64159850 msgr2=0x7f3a641ef390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.590+0000 7f3a6b000640 1 --2- 172.21.15.67:0/2391087936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a64159850 0x7f3a641ef390 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00ce40 tx=0x7f3a5c00a710 comp rx=0 tx=0).stop 2024-03-09T09:11:13.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.590+0000 7f3a6a600640 1 -- 172.21.15.67:0/2391087936 reap_dead start 2024-03-09T09:11:13.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.590+0000 7f3a6a600640 1 -- 172.21.15.67:0/2391087936 reap_dead start 2024-03-09T09:11:13.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.590+0000 7f3a6b000640 1 -- 172.21.15.67:0/2391087936 shutdown_connections 2024-03-09T09:11:13.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.590+0000 7f3a6b000640 1 -- 172.21.15.67:0/2391087936 >> 172.21.15.67:0/2391087936 conn(0x7f3a640ad060 msgr2=0x7f3a640c6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:13.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.590+0000 7f3a6b000640 1 -- 172.21.15.67:0/2391087936 shutdown_connections 2024-03-09T09:11:13.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.590+0000 7f3a6b000640 1 -- 172.21.15.67:0/2391087936 wait complete. 2024-03-09T09:11:13.616 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:11:13.616 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:11:13.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:11:13.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.859+0000 7fcadc000640 1 Processor -- start 2024-03-09T09:11:13.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.860+0000 7fcadc000640 1 -- start start 2024-03-09T09:11:13.864 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 3% (5/137) Updating files: 4% (6/137) Updating files: 5% (7/137) Updating files: 6% (9/137) Updating files: 7% (10/137) Updating files: 8% (11/137) Updating files: 9% (13/137) Updating files: 10% (14/137) Updating files: 11% (16/137) Updating files: 12% (17/137) Updating files: 13% (18/137) Updating files: 13% (19/137) Updating files: 14% (20/137) Updating files: 15% (21/137) Updating files: 16% (22/137) Updating files: 17% (24/137) Updating files: 18% (25/137) Updating files: 19% (27/137) Updating files: 20% (28/137) Updating files: 21% (29/137) Updating files: 22% (31/137) Updating files: 23% (32/137) Updating files: 24% (33/137) Updating files: 25% (35/137) Updating files: 26% (36/137) Updating files: 27% (37/137) Updating files: 28% (39/137) Updating files: 29% (40/137) Updating files: 30% (42/137) Updating files: 31% (43/137) Updating files: 32% (44/137) Updating files: 33% (46/137) Updating files: 34% (47/137) Updating files: 35% (48/137) Updating files: 36% (50/137) Updating files: 37% (51/137) Updating files: 38% (53/137) Updating files: 39% (54/137) Updating files: 40% (55/137) Updating files: 41% (57/137) Updating files: 42% (58/137) Updating files: 43% (59/137) Updating files: 44% (61/137) Updating files: 45% (62/137) Updating files: 46% (64/137) Updating files: 47% (65/137) Updating files: 48% (66/137) Updating files: 49% (68/137) Updating files: 50% (69/137) Updating files: 51% (70/137) Updating files: 52% (72/137) Updating files: 53% (73/137) Updating files: 54% (74/137) Updating files: 55% (76/137) Updating files: 56% (77/137) Updating files: 57% (79/137) Updating files: 58% (80/137) Updating files: 59% (81/137) Updating files: 60% (83/137) Updating files: 61% (84/137) Updating files: 62% (85/137) Updating files: 63% (87/137) Updating files: 64% (88/137) Updating files: 65% (90/137) Updating files: 66% (91/137) Updating files: 67% (92/137) Updating files: 68% (94/137) Updating files: 69% (95/137) Updating files: 70% (96/137) Updating files: 71% (98/137) Updating files: 72% (99/137) Updating files: 73% (101/137) Updating files: 74% (102/137) Updating files: 75% (103/137) Updating files: 76% (105/137) Updating files: 77% (106/137) Updating files: 78% (107/137) Updating files: 79% (109/137) Updating files: 80% (110/137) Updating files: 81% (111/137) Updating files: 82% (113/137) Updating files: 83% (114/137) Updating files: 84% (116/137) Updating files: 85% (117/137) Updating files: 86% (118/137) Updating files: 87% (120/137) Updating files: 88% (121/137) Updating files: 89% (122/137) Updating files: 90% (124/137) Updating files: 91% (125/137) Updating files: 92% (127/137) Updating files: 93% (128/137) Updating files: 94% (129/137) Updating files: 95% (131/137) Updating files: 96% (132/137) Updating files: 97% (133/137) Updating files: 98% (135/137) Updating files: 99% (136/137) Updating files: 100% (137/137) Updating files: 100% (137/137), done. 2024-03-09T09:11:13.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.861+0000 7fcadc000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcad40c2780 0x7fcad4154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.861+0000 7fcadc000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcad4071190 0x7fcad41546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.861+0000 7fcadc000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad40c8200 con 0x7fcad40ac9a0 2024-03-09T09:11:13.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.861+0000 7fcadc000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad406d590 con 0x7fcad40c2780 2024-03-09T09:11:13.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.861+0000 7fcadc000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad40c3740 con 0x7fcad4071190 2024-03-09T09:11:13.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.861+0000 7fcada000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcad4071190 0x7fcad41546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.861+0000 7fcada000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcad4071190 0x7fcad41546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:41354/0 (socket says 172.21.15.67:41354) 2024-03-09T09:11:13.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.861+0000 7fcada000640 1 -- 172.21.15.67:0/1455429831 learned_addr learned my addr 172.21.15.67:0/1455429831 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:13.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.861+0000 7fcad8c00640 1 --2- 172.21.15.67:0/1455429831 >> v2:172.21.15.134:3300/0 conn(0x7fcad40c2780 0x7fcad4154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.862+0000 7fcada000640 1 -- 172.21.15.67:0/1455429831 >> v2:172.21.15.134:3300/0 conn(0x7fcad40c2780 msgr2=0x7fcad4154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:13.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.862+0000 7fcada000640 1 --2- 172.21.15.67:0/1455429831 >> v2:172.21.15.134:3300/0 conn(0x7fcad40c2780 0x7fcad4154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.862+0000 7fcada000640 1 -- 172.21.15.67:0/1455429831 >> v1:172.21.15.134:6789/0 conn(0x7fcad40ac9a0 legacy=0x7fcad41510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.862+0000 7fcada000640 1 -- 172.21.15.67:0/1455429831 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad40c3990 con 0x7fcad4071190 2024-03-09T09:11:13.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.862+0000 7fcad8c00640 1 --2- 172.21.15.67:0/1455429831 >> v2:172.21.15.134:3300/0 conn(0x7fcad40c2780 0x7fcad4154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:11:13.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.863+0000 7fcada000640 1 --2- 172.21.15.67:0/1455429831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcad4071190 0x7fcad41546d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009e50 tx=0x7fcacc02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e26df1a7b81c72 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:13.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.863+0000 7fcad3e00640 1 -- 172.21.15.67:0/1455429831 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc02ca70 con 0x7fcad4071190 2024-03-09T09:11:13.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.863+0000 7fcad3e00640 1 -- 172.21.15.67:0/1455429831 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc02cc10 con 0x7fcad4071190 2024-03-09T09:11:13.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.863+0000 7fcad3e00640 1 -- 172.21.15.67:0/1455429831 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0343f0 con 0x7fcad4071190 2024-03-09T09:11:13.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.864+0000 7fcadc000640 1 -- 172.21.15.67:0/1455429831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcad4071190 msgr2=0x7fcad41546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.864+0000 7fcadc000640 1 --2- 172.21.15.67:0/1455429831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcad4071190 0x7fcad41546d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009e50 tx=0x7fcacc02bed0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.864+0000 7fcadc000640 1 -- 172.21.15.67:0/1455429831 shutdown_connections 2024-03-09T09:11:13.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.864+0000 7fcadc000640 1 --2- 172.21.15.67:0/1455429831 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcad4071190 0x7fcad41546d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.864+0000 7fcadc000640 1 --2- 172.21.15.67:0/1455429831 >> v2:172.21.15.134:3300/0 conn(0x7fcad40c2780 0x7fcad4154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.864+0000 7fcadc000640 1 -- 172.21.15.67:0/1455429831 >> 172.21.15.67:0/1455429831 conn(0x7fcad40c7370 msgr2=0x7fcad40c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:13.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.864+0000 7fcadc000640 1 -- 172.21.15.67:0/1455429831 shutdown_connections 2024-03-09T09:11:13.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.864+0000 7fcadc000640 1 -- 172.21.15.67:0/1455429831 wait complete. 2024-03-09T09:11:13.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.866+0000 7fcadc000640 1 Processor -- start 2024-03-09T09:11:13.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.866+0000 7fcadc000640 1 -- start start 2024-03-09T09:11:13.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcadc000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcad4071190 0x7fcad414f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcadc000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40ac9a0 0x7fcad4073640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcadc000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcad40c2780 0x7fcad41eb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcadc000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad4154c10 con 0x7fcad4071190 2024-03-09T09:11:13.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcadc000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad4070c00 con 0x7fcad40ac9a0 2024-03-09T09:11:13.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcadc000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad40acd80 con 0x7fcad40c2780 2024-03-09T09:11:13.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcad9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcad4071190 0x7fcad414f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcad9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcad4071190 0x7fcad414f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35374/0 (socket says 172.21.15.67:35374) 2024-03-09T09:11:13.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcad8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40ac9a0 0x7fcad4073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcad9600640 1 -- 172.21.15.67:0/2305021367 learned_addr learned my addr 172.21.15.67:0/2305021367 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:13.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcada000640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcad40c2780 0x7fcad41eb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.867+0000 7fcad8c00640 1 -- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcad40c2780 msgr2=0x7fcad41eb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.868+0000 7fcad8c00640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcad40c2780 0x7fcad41eb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.868+0000 7fcad8c00640 1 -- 172.21.15.67:0/2305021367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcad4071190 msgr2=0x7fcad414f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.868+0000 7fcad8c00640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcad4071190 0x7fcad414f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.868+0000 7fcad8c00640 1 -- 172.21.15.67:0/2305021367 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad40c7120 con 0x7fcad40ac9a0 2024-03-09T09:11:13.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.868+0000 7fcad9600640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcad4071190 0x7fcad414f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:11:13.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.868+0000 7fcad8c00640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40ac9a0 0x7fcad4073640 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcac000e840 tx=0x7fcac000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:13.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.868+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0017070 con 0x7fcad40ac9a0 2024-03-09T09:11:13.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.868+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac000c940 con 0x7fcad40ac9a0 2024-03-09T09:11:13.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.868+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac000cd10 con 0x7fcad40ac9a0 2024-03-09T09:11:13.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.868+0000 7fcadc000640 1 -- 172.21.15.67:0/2305021367 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad41efa10 con 0x7fcad40ac9a0 2024-03-09T09:11:13.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.868+0000 7fcadc000640 1 -- 172.21.15.67:0/2305021367 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad40c6e40 con 0x7fcad40ac9a0 2024-03-09T09:11:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.869+0000 7fcadc000640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40c6370 0x7fcad4147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.869+0000 7fcadc000640 1 -- 172.21.15.67:0/2305021367 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad414f830 con 0x7fcad40c6370 2024-03-09T09:11:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.870+0000 7fcad9600640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40c6370 0x7fcad4147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.870+0000 7fcad9600640 1 -- 172.21.15.67:0/2305021367 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcad40c24e0 con 0x7fcad40c6370 2024-03-09T09:11:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.870+0000 7fcad9600640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40c6370 0x7fcad4147d10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcac800fb10 tx=0x7fcac8001db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.871+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac80052e0 con 0x7fcad40c6370 2024-03-09T09:11:13.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.871+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcac001a070 con 0x7fcad40ac9a0 2024-03-09T09:11:13.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.871+0000 7fcad1600640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcab406b3d0 0x7fcab406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.871+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcac008a300 con 0x7fcad40ac9a0 2024-03-09T09:11:13.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.872+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fcac8005480 con 0x7fcad40c6370 2024-03-09T09:11:13.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.872+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40c6370 msgr2=0x7fcad4147d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.872+0000 7fcad1600640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40c6370 0x7fcad4147d10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcac800fb10 tx=0x7fcac8001db0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.872+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40c6370 msgr2=0x7fcad4147d10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:13.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.872+0000 7fcad1600640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40c6370 0x7fcad4147d10 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcac800fb10 tx=0x7fcac8001db0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.872+0000 7fcada000640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcab406b3d0 0x7fcab406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.873+0000 7fcada000640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcab406b3d0 0x7fcab406d890 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcacc035000 tx=0x7fcacc030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.879+0000 7fcadc000640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad41ebee0 0x7fcad4148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.879+0000 7fcadc000640 1 -- 172.21.15.67:0/2305021367 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad414f010 con 0x7fcad41ebee0 2024-03-09T09:11:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.879+0000 7fcad9600640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad41ebee0 0x7fcad4148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:13.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.880+0000 7fcad9600640 1 -- 172.21.15.67:0/2305021367 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcad4075a00 con 0x7fcad41ebee0 2024-03-09T09:11:13.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.881+0000 7fcad9600640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad41ebee0 0x7fcad4148250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcac800fb10 tx=0x7fcac8000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:13.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.881+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac800de40 con 0x7fcad41ebee0 2024-03-09T09:11:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.881+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7fcac800de40 con 0x7fcad41ebee0 2024-03-09T09:11:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.881+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad41ebee0 msgr2=0x7fcad4148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.881+0000 7fcad1600640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad41ebee0 0x7fcad4148250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcac800fb10 tx=0x7fcac8000ed0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.881+0000 7fcad1600640 1 -- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad41ebee0 msgr2=0x7fcad4148250 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:13.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.881+0000 7fcad1600640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad41ebee0 0x7fcad4148250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 136, 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:11:13.887 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:11:13.888 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:11:13.889 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:13.890 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:11:13.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:11:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:11:13.894 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.896 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:13.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:13.898 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:11:13.898 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:11:13.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.882+0000 7fcadc000640 1 -- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcab406b3d0 msgr2=0x7fcab406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.882+0000 7fcadc000640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcab406b3d0 0x7fcab406d890 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcacc035000 tx=0x7fcacc030000 comp rx=0 tx=0).stop 2024-03-09T09:11:13.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.882+0000 7fcadc000640 1 -- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40ac9a0 msgr2=0x7fcad4073640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:13.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.882+0000 7fcadc000640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40ac9a0 0x7fcad4073640 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcac000e840 tx=0x7fcac000ed00 comp rx=0 tx=0).stop 2024-03-09T09:11:13.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.882+0000 7fcada000640 1 -- 172.21.15.67:0/2305021367 reap_dead start 2024-03-09T09:11:13.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.882+0000 7fcadc000640 1 -- 172.21.15.67:0/2305021367 shutdown_connections 2024-03-09T09:11:13.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.882+0000 7fcadc000640 1 --2- 172.21.15.67:0/2305021367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcad40ac9a0 0x7fcad4073640 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:13.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.882+0000 7fcadc000640 1 -- 172.21.15.67:0/2305021367 >> 172.21.15.67:0/2305021367 conn(0x7fcad40c7370 msgr2=0x7fcad4071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:13.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.882+0000 7fcadc000640 1 -- 172.21.15.67:0/2305021367 shutdown_connections 2024-03-09T09:11:13.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:13.882+0000 7fcadc000640 1 -- 172.21.15.67:0/2305021367 wait complete. 2024-03-09T09:11:13.905 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:11:13.905 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:11:13.906 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:11:13.940 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at a4428bd0e19 v0.50 2024-03-09T09:11:13.947 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.50 2024-03-09T09:11:14.006 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:11:14.007 INFO:tasks.workunit.client.0.smithi067.stdout:v0.51 2024-03-09T09:11:14.007 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 51 -eq 48 ']' 2024-03-09T09:11:14.007 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.51 2024-03-09T09:11:14.007 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.51 2024-03-09T09:11:14.007 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.51 2024-03-09T09:11:14.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.131+0000 7fa7b1e00640 1 Processor -- start 2024-03-09T09:11:14.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.131+0000 7fa7b1e00640 1 -- start start 2024-03-09T09:11:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b1e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7ac153220 0x7fa7ac0c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b1e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa7ac0c8860 0x7fa7ac159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b1e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac159850 0x7fa7ac0c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b1e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac0c9b10 con 0x7fa7ac153220 2024-03-09T09:11:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b1e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac159d60 con 0x7fa7ac0c8860 2024-03-09T09:11:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b1e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac15a040 con 0x7fa7ac159850 2024-03-09T09:11:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7ac153220 0x7fa7ac0c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b0a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7ac153220 0x7fa7ac0c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35378/0 (socket says 172.21.15.67:35378) 2024-03-09T09:11:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b0a00640 1 -- 172.21.15.67:0/3706247606 learned_addr learned my addr 172.21.15.67:0/3706247606 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b0a00640 1 -- 172.21.15.67:0/3706247606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac159850 msgr2=0x7fa7ac0c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b0a00640 1 --2- 172.21.15.67:0/3706247606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac159850 0x7fa7ac0c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b0a00640 1 -- 172.21.15.67:0/3706247606 >> v2:172.21.15.134:3300/0 conn(0x7fa7ac0c8860 msgr2=0x7fa7ac159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b0a00640 1 --2- 172.21.15.67:0/3706247606 >> v2:172.21.15.134:3300/0 conn(0x7fa7ac0c8860 0x7fa7ac159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.133+0000 7fa7b0a00640 1 -- 172.21.15.67:0/3706247606 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac0c8060 con 0x7fa7ac153220 2024-03-09T09:11:14.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.134+0000 7fa7b0a00640 1 --2- 172.21.15.67:0/3706247606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7ac153220 0x7fa7ac0c95d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac156e90 tx=0x7fa7a002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794f5789aabcb092 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:14.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.135+0000 7fa7ab400640 1 -- 172.21.15.67:0/3706247606 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a002e070 con 0x7fa7ac153220 2024-03-09T09:11:14.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.135+0000 7fa7ab400640 1 -- 172.21.15.67:0/3706247606 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a0037070 con 0x7fa7ac153220 2024-03-09T09:11:14.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.135+0000 7fa7ab400640 1 -- 172.21.15.67:0/3706247606 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0031a00 con 0x7fa7ac153220 2024-03-09T09:11:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.136+0000 7fa7b1e00640 1 -- 172.21.15.67:0/3706247606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7ac153220 msgr2=0x7fa7ac0c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.136+0000 7fa7b1e00640 1 --2- 172.21.15.67:0/3706247606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7ac153220 0x7fa7ac0c95d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac156e90 tx=0x7fa7a002b880 comp rx=0 tx=0).stop 2024-03-09T09:11:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.136+0000 7fa7b1e00640 1 -- 172.21.15.67:0/3706247606 shutdown_connections 2024-03-09T09:11:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.136+0000 7fa7b1e00640 1 --2- 172.21.15.67:0/3706247606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac159850 0x7fa7ac0c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.136+0000 7fa7b1e00640 1 --2- 172.21.15.67:0/3706247606 >> v2:172.21.15.134:3300/0 conn(0x7fa7ac0c8860 0x7fa7ac159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.136+0000 7fa7b1e00640 1 --2- 172.21.15.67:0/3706247606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7ac153220 0x7fa7ac0c95d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.136+0000 7fa7b1e00640 1 -- 172.21.15.67:0/3706247606 >> 172.21.15.67:0/3706247606 conn(0x7fa7ac0ad060 msgr2=0x7fa7ac0ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.136+0000 7fa7b1e00640 1 -- 172.21.15.67:0/3706247606 shutdown_connections 2024-03-09T09:11:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.136+0000 7fa7b1e00640 1 -- 172.21.15.67:0/3706247606 wait complete. 2024-03-09T09:11:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.137+0000 7fa7b1e00640 1 Processor -- start 2024-03-09T09:11:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.137+0000 7fa7b1e00640 1 -- start start 2024-03-09T09:11:14.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.138+0000 7fa7b1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7ac0c8860 0x7fa7ac14ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.138+0000 7fa7b1e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7ac153220 0x7fa7ac152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.138+0000 7fa7b1e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac159850 0x7fa7ac152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.138+0000 7fa7b1e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac1560a0 con 0x7fa7ac0c8860 2024-03-09T09:11:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.138+0000 7fa7b1e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac155f20 con 0x7fa7ac153220 2024-03-09T09:11:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.138+0000 7fa7b1e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac153600 con 0x7fa7ac159850 2024-03-09T09:11:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.138+0000 7fa7b0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7ac0c8860 0x7fa7ac14ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.139+0000 7fa7b0a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7ac0c8860 0x7fa7ac14ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35394/0 (socket says 172.21.15.67:35394) 2024-03-09T09:11:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.139+0000 7fa7b0a00640 1 -- 172.21.15.67:0/856710044 learned_addr learned my addr 172.21.15.67:0/856710044 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.139+0000 7fa7b1400640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac159850 0x7fa7ac152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.139+0000 7fa7abe00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7ac153220 0x7fa7ac152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.139+0000 7fa7b0a00640 1 -- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac159850 msgr2=0x7fa7ac152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.139+0000 7fa7b0a00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac159850 0x7fa7ac152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.139+0000 7fa7b0a00640 1 -- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7ac153220 msgr2=0x7fa7ac152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.139+0000 7fa7b0a00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7ac153220 0x7fa7ac152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.139+0000 7fa7b0a00640 1 -- 172.21.15.67:0/856710044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac1ef510 con 0x7fa7ac0c8860 2024-03-09T09:11:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.139+0000 7fa7b0a00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7ac0c8860 0x7fa7ac14ec10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0000c00 tx=0x7fa7a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.140+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0032880 con 0x7fa7ac0c8860 2024-03-09T09:11:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.140+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a0032a20 con 0x7fa7ac0c8860 2024-03-09T09:11:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.140+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0032e10 con 0x7fa7ac0c8860 2024-03-09T09:11:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.140+0000 7fa7b1e00640 1 -- 172.21.15.67:0/856710044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ac0c2180 con 0x7fa7ac0c8860 2024-03-09T09:11:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.140+0000 7fa7b1e00640 1 -- 172.21.15.67:0/856710044 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac14f150 con 0x7fa7ac0c8860 2024-03-09T09:11:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.141+0000 7fa7b1e00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac0c28d0 0x7fa7ac149480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.141+0000 7fa7b1e00640 1 -- 172.21.15.67:0/856710044 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac0c1f30 con 0x7fa7ac0c28d0 2024-03-09T09:11:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.141+0000 7fa7abe00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac0c28d0 0x7fa7ac149480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.142+0000 7fa7abe00640 1 -- 172.21.15.67:0/856710044 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7ac04f3c0 con 0x7fa7ac0c28d0 2024-03-09T09:11:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.142+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa7a00453f0 con 0x7fa7ac0c8860 2024-03-09T09:11:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.142+0000 7fa7abe00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac0c28d0 0x7fa7ac149480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac1508b0 tx=0x7fa79800d960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.143+0000 7fa7a8c00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa78c06b4d0 0x7fa78c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.143+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa7a00ac800 con 0x7fa7ac0c8860 2024-03-09T09:11:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.143+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798018020 con 0x7fa7ac0c28d0 2024-03-09T09:11:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.143+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa798014620 con 0x7fa7ac0c28d0 2024-03-09T09:11:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.143+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac0c28d0 msgr2=0x7fa7ac149480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.143+0000 7fa7a8c00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac0c28d0 0x7fa7ac149480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac1508b0 tx=0x7fa79800d960 comp rx=0 tx=0).stop 2024-03-09T09:11:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.143+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac0c28d0 msgr2=0x7fa7ac149480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.143+0000 7fa7a8c00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac0c28d0 0x7fa7ac149480 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac1508b0 tx=0x7fa79800d960 comp rx=0 tx=0).stop 2024-03-09T09:11:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.144+0000 7fa7b1400640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa78c06b4d0 0x7fa78c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.145+0000 7fa7b1400640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa78c06b4d0 0x7fa78c06d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac154150 tx=0x7fa7a4009510 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:14.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.155+0000 7fa7b1e00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac1499c0 0x7fa7ac149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.155+0000 7fa7b1e00640 1 -- 172.21.15.67:0/856710044 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac0c93b0 con 0x7fa7ac1499c0 2024-03-09T09:11:14.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.155+0000 7fa7abe00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac1499c0 0x7fa7ac149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:14.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.156+0000 7fa7abe00640 1 -- 172.21.15.67:0/856710044 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa7ac0ad440 con 0x7fa7ac1499c0 2024-03-09T09:11:14.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.157+0000 7fa7abe00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac1499c0 0x7fa7ac149da0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac153dd0 tx=0x7fa798004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:14.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.157+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798018040 con 0x7fa7ac1499c0 2024-03-09T09:11:14.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.157+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4705 (secure 0 0 0) 0x7fa79801f860 con 0x7fa7ac1499c0 2024-03-09T09:11:14.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.157+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac1499c0 msgr2=0x7fa7ac149da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.157+0000 7fa7a8c00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac1499c0 0x7fa7ac149da0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac153dd0 tx=0x7fa798004730 comp rx=0 tx=0).stop 2024-03-09T09:11:14.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7a8c00640 1 -- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac1499c0 msgr2=0x7fa7ac149da0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:14.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7a8c00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7ac1499c0 0x7fa7ac149da0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac153dd0 tx=0x7fa798004730 comp rx=0 tx=0).stop 2024-03-09T09:11:14.164 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:11:14.164 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:14.164 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:14.164 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:11:14.164 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 136, 2024-03-09T09:11:14.164 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:11:14.164 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:11:14.165 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:11:14.165 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:11:14.165 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:14.165 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:11:14.165 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:14.165 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:11:14.165 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:11:14.165 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:14.165 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:14.165 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:14.166 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:14.166 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:14.166 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:14.166 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:14.167 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:14.167 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:14.167 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:14.167 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:14.168 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:14.168 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:11:14.168 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:11:14.168 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:14.168 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:14.168 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:14.168 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:14.168 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:14.168 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:14.168 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:14.168 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:14.169 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:14.169 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:14.169 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:14.169 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:14.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.169 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:11:14.169 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:11:14.169 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:11:14.169 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:11:14.170 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:11:14.170 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:11:14.170 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:14.170 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:14.170 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:11:14.170 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:11:14.170 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:11:14.170 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:11:14.170 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:11:14.170 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:14.171 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:11:14.172 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:14.173 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.174 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:14.175 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.176 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:14.177 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:14.178 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:11:14.178 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.178 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:14.178 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:14.178 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:14.178 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:14.178 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:14.178 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:11:14.178 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:14.178 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:14.179 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:14.179 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:14.179 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:14.179 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:14.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:14.179 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:11:14.179 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:11:14.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7b1e00640 1 -- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa78c06b4d0 msgr2=0x7fa78c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7b1e00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa78c06b4d0 0x7fa78c06d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac154150 tx=0x7fa7a4009510 comp rx=0 tx=0).stop 2024-03-09T09:11:14.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7b1e00640 1 -- 172.21.15.67:0/856710044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7ac0c8860 msgr2=0x7fa7ac14ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7b1e00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7ac0c8860 0x7fa7ac14ec10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0000c00 tx=0x7fa7a00046c0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7b1400640 1 -- 172.21.15.67:0/856710044 reap_dead start 2024-03-09T09:11:14.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7b1e00640 1 -- 172.21.15.67:0/856710044 shutdown_connections 2024-03-09T09:11:14.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7b1e00640 1 --2- 172.21.15.67:0/856710044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7ac0c8860 0x7fa7ac14ec10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7b1e00640 1 -- 172.21.15.67:0/856710044 >> 172.21.15.67:0/856710044 conn(0x7fa7ac0ad060 msgr2=0x7fa7ac0ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:14.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7b1e00640 1 -- 172.21.15.67:0/856710044 shutdown_connections 2024-03-09T09:11:14.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.158+0000 7fa7b1e00640 1 -- 172.21.15.67:0/856710044 wait complete. 2024-03-09T09:11:14.183 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:11:14.183 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T09:11:14.183 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:11:14.184 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T09:11:14.184 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:11:14.184 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:11:14.184 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:11:14.216 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:11:14.216 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:11:14.217 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:11:14.250 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 14% (69/481) Updating files: 15% (73/481) Updating files: 16% (77/481) Updating files: 17% (82/481) Updating files: 18% (87/481) Updating files: 18% (89/481) Updating files: 19% (92/481) Updating files: 20% (97/481) Updating files: 21% (102/481) Updating files: 22% (106/481) Updating files: 23% (111/481) Updating files: 24% (116/481) Updating files: 25% (121/481) Updating files: 26% (126/481) Updating files: 27% (130/481) Updating files: 28% (135/481) Updating files: 28% (137/481) Updating files: 29% (140/481) Updating files: 30% (145/481) Updating files: 31% (150/481) Updating files: 32% (154/481) Updating files: 33% (159/481) Updating files: 34% (164/481) Updating files: 35% (169/481) Updating files: 36% (174/481) Updating files: 37% (178/481) Updating files: 38% (183/481) Updating files: 38% (187/481) Updating files: 39% (188/481) Updating files: 40% (193/481) Updating files: 41% (198/481) Updating files: 42% (203/481) Updating files: 43% (207/481) Updating files: 44% (212/481) Updating files: 45% (217/481) Updating files: 46% (222/481) Updating files: 47% (227/481) Updating files: 48% (231/481) Updating files: 49% (236/481) Updating files: 50% (241/481) Updating files: 51% (246/481) Updating files: 52% (251/481) Updating files: 53% (255/481) Updating files: 54% (260/481) Updating files: 55% (265/481) Updating files: 56% (270/481) Updating files: 56% (274/481) Updating files: 57% (275/481) Updating files: 58% (279/481) Updating files: 59% (284/481) Updating files: 60% (289/481) Updating files: 61% (294/481) Updating files: 62% (299/481) Updating files: 63% (304/481) Updating files: 64% (308/481) Updating files: 65% (313/481) Updating files: 66% (318/481) Updating files: 67% (323/481) Updating files: 68% (328/481) Updating files: 69% (332/481) Updating files: 70% (337/481) Updating files: 71% (342/481) Updating files: 72% (347/481) Updating files: 73% (352/481) Updating files: 74% (356/481) Updating files: 75% (361/481) Updating files: 75% (363/481) Updating files: 76% (366/481) Updating files: 77% (371/481) Updating files: 78% (376/481) Updating files: 79% (380/481) Updating files: 79% (381/481) Updating files: 80% (385/481) Updating files: 81% (390/481) Updating files: 81% (393/481) Updating files: 82% (395/481) Updating files: 83% (400/481) Updating files: 84% (405/481) Updating files: 85% (409/481) Updating files: 86% (414/481) Updating files: 87% (419/481) Updating files: 88% (424/481) Updating files: 89% (429/481) Updating files: 90% (433/481) Updating files: 91% (438/481) Updating files: 92% (443/481) Updating files: 93% (448/481) Updating files: 94% (453/481) Updating files: 95% (457/481) Updating files: 96% (462/481) Updating files: 96% (465/481) Updating files: 97% (467/481) Updating files: 98% (472/481) Updating files: 99% (477/481) Updating files: 100% (481/481) Updating files: 100% (481/481), done. 2024-03-09T09:11:14.331 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at ca6265d0f4d v0.49 2024-03-09T09:11:14.344 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.49 2024-03-09T09:11:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.435+0000 7fe564000640 1 Processor -- start 2024-03-09T09:11:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.435+0000 7fe564000640 1 -- start start 2024-03-09T09:11:14.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.436+0000 7fe564000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe55c071190 0x7fe55c157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.436+0000 7fe564000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55c153d40 con 0x7fe55c0ac730 2024-03-09T09:11:14.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.436+0000 7fe564000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55c0c2790 con 0x7fe55c0c22a0 2024-03-09T09:11:14.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.436+0000 7fe564000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c070ee0 con 0x7fe55c071190 2024-03-09T09:11:14.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.437+0000 7fe561600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe55c0ac730 0x7fe55c157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44132/0 (socket says 172.21.15.67:44132) 2024-03-09T09:11:14.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.437+0000 7fe561600640 1 -- 172.21.15.67:0/3933297816 learned_addr learned my addr 172.21.15.67:0/3933297816 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:11:14.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.437+0000 7fe562000640 1 --2- 172.21.15.67:0/3933297816 >> v2:172.21.15.67:3300/0 conn(0x7fe55c071190 0x7fe55c157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:14.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.437+0000 7fe55be00640 1 -- 172.21.15.67:0/3933297816 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666310308 0 0) 0x7fe55c153d40 con 0x7fe55c0ac730 2024-03-09T09:11:14.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.437+0000 7fe562000640 1 -- 172.21.15.67:0/3933297816 >> v1:172.21.15.134:6789/0 conn(0x7fe55c0c22a0 legacy=0x7fe55c15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.437+0000 7fe562000640 1 -- 172.21.15.67:0/3933297816 >> v1:172.21.15.67:6789/0 conn(0x7fe55c0ac730 legacy=0x7fe55c157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.437+0000 7fe562000640 1 -- 172.21.15.67:0/3933297816 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c070920 con 0x7fe55c071190 2024-03-09T09:11:14.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.439+0000 7fe562000640 1 --2- 172.21.15.67:0/3933297816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe55c071190 0x7fe55c157d60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe554009d90 tx=0x7fe55402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e439bb7983aee5b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:14.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.439+0000 7fe55be00640 1 -- 172.21.15.67:0/3933297816 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554039070 con 0x7fe55c071190 2024-03-09T09:11:14.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.439+0000 7fe55be00640 1 -- 172.21.15.67:0/3933297816 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55402cc60 con 0x7fe55c071190 2024-03-09T09:11:14.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.439+0000 7fe55be00640 1 -- 172.21.15.67:0/3933297816 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554034660 con 0x7fe55c071190 2024-03-09T09:11:14.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.440+0000 7fe564000640 1 -- 172.21.15.67:0/3933297816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe55c071190 msgr2=0x7fe55c157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.440+0000 7fe564000640 1 --2- 172.21.15.67:0/3933297816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe55c071190 0x7fe55c157d60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe554009d90 tx=0x7fe55402beb0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.440+0000 7fe564000640 1 -- 172.21.15.67:0/3933297816 shutdown_connections 2024-03-09T09:11:14.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.440+0000 7fe564000640 1 --2- 172.21.15.67:0/3933297816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe55c071190 0x7fe55c157d60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.440+0000 7fe564000640 1 -- 172.21.15.67:0/3933297816 >> 172.21.15.67:0/3933297816 conn(0x7fe55c0c7370 msgr2=0x7fe55c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:14.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.440+0000 7fe564000640 1 -- 172.21.15.67:0/3933297816 shutdown_connections 2024-03-09T09:11:14.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.440+0000 7fe564000640 1 -- 172.21.15.67:0/3933297816 wait complete. 2024-03-09T09:11:14.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.442+0000 7fe564000640 1 Processor -- start 2024-03-09T09:11:14.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.442+0000 7fe564000640 1 -- start start 2024-03-09T09:11:14.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.443+0000 7fe564000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe55c071190 0x7fe55c14d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.443+0000 7fe564000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe55c0ac730 0x7fe55c1531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.443+0000 7fe564000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe55c0c22a0 0x7fe55c1ef700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.443+0000 7fe564000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c151610 con 0x7fe55c071190 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.443+0000 7fe564000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c154d00 con 0x7fe55c0ac730 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.443+0000 7fe564000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c151790 con 0x7fe55c0c22a0 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.443+0000 7fe561600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe55c071190 0x7fe55c14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.443+0000 7fe562000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe55c0c22a0 msgr2=0x7fe55c1ef700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.443+0000 7fe562000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe55c0c22a0 0x7fe55c1ef700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.443+0000 7fe561600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe55c071190 0x7fe55c14d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35420/0 (socket says 172.21.15.67:35420) 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.443+0000 7fe561600640 1 -- 172.21.15.67:0/1031729868 learned_addr learned my addr 172.21.15.67:0/1031729868 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.444+0000 7fe561600640 1 -- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe55c0c22a0 msgr2=0x7fe55c1ef700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:11:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.444+0000 7fe561600640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe55c0c22a0 0x7fe55c1ef700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.444+0000 7fe561600640 1 -- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe55c0ac730 msgr2=0x7fe55c1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.444+0000 7fe561600640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe55c0ac730 0x7fe55c1531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.444+0000 7fe561600640 1 -- 172.21.15.67:0/1031729868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c1ebdb0 con 0x7fe55c071190 2024-03-09T09:11:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.444+0000 7fe561600640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe55c071190 0x7fe55c14d130 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe55000c4a0 tx=0x7fe55000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.444+0000 7fe559600640 1 -- 172.21.15.67:0/1031729868 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550018070 con 0x7fe55c071190 2024-03-09T09:11:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.445+0000 7fe559600640 1 -- 172.21.15.67:0/1031729868 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5500074e0 con 0x7fe55c071190 2024-03-09T09:11:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.445+0000 7fe564000640 1 -- 172.21.15.67:0/1031729868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c0c1260 con 0x7fe55c071190 2024-03-09T09:11:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.445+0000 7fe559600640 1 -- 172.21.15.67:0/1031729868 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550004a20 con 0x7fe55c071190 2024-03-09T09:11:14.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.445+0000 7fe564000640 1 -- 172.21.15.67:0/1031729868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c0aa010 con 0x7fe55c071190 2024-03-09T09:11:14.452 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:11:14.452 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 50 -eq 48 ']' 2024-03-09T09:11:14.452 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.50 2024-03-09T09:11:14.452 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.50 2024-03-09T09:11:14.453 INFO:tasks.workunit.client.1.smithi134.stdout:v0.50 2024-03-09T09:11:14.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.446+0000 7fe564000640 1 -- 172.21.15.67:0/1031729868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55c075a00 con 0x7fe55c071190 2024-03-09T09:11:14.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.447+0000 7fe559600640 1 -- 172.21.15.67:0/1031729868 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe55001b070 con 0x7fe55c071190 2024-03-09T09:11:14.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.448+0000 7fe559600640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe53c06b480 0x7fe53c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:14.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.448+0000 7fe559600640 1 -- 172.21.15.67:0/1031729868 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe5500898b0 con 0x7fe55c071190 2024-03-09T09:11:14.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.449+0000 7fe560c00640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe53c06b480 0x7fe53c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:14.454 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.50 2024-03-09T09:11:14.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.450+0000 7fe560c00640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe53c06b480 0x7fe53c06d940 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe548000f60 tx=0x7fe5480079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:14.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.452+0000 7fe559600640 1 -- 172.21.15.67:0/1031729868 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe550058330 con 0x7fe55c071190 2024-03-09T09:11:14.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.873+0000 7fe564000640 1 -- 172.21.15.67:0/1031729868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe55c0c1d70 con 0x7fe55c071190 2024-03-09T09:11:14.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.874+0000 7fe559600640 1 -- 172.21.15.67:0/1031729868 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fe550057c30 con 0x7fe55c071190 2024-03-09T09:11:14.879 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":136,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:11:14.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.877+0000 7fe564000640 1 -- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe53c06b480 msgr2=0x7fe53c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.877+0000 7fe564000640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe53c06b480 0x7fe53c06d940 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe548000f60 tx=0x7fe5480079d0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.877+0000 7fe564000640 1 -- 172.21.15.67:0/1031729868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe55c071190 msgr2=0x7fe55c14d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:14.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.877+0000 7fe564000640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe55c071190 0x7fe55c14d130 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe55000c4a0 tx=0x7fe55000c960 comp rx=0 tx=0).stop 2024-03-09T09:11:14.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.877+0000 7fe564000640 1 -- 172.21.15.67:0/1031729868 shutdown_connections 2024-03-09T09:11:14.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.877+0000 7fe564000640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe53c06b480 0x7fe53c06d940 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.877+0000 7fe564000640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe55c0c22a0 0x7fe55c1ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.877+0000 7fe564000640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe55c0ac730 0x7fe55c1531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.877+0000 7fe564000640 1 --2- 172.21.15.67:0/1031729868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe55c071190 0x7fe55c14d130 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:14.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.877+0000 7fe564000640 1 -- 172.21.15.67:0/1031729868 >> 172.21.15.67:0/1031729868 conn(0x7fe55c0c7370 msgr2=0x7fe55c157650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:14.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.878+0000 7fe564000640 1 -- 172.21.15.67:0/1031729868 shutdown_connections 2024-03-09T09:11:14.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:14.878+0000 7fe564000640 1 -- 172.21.15.67:0/1031729868 wait complete. 2024-03-09T09:11:15.444 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at c03ca95d235 v0.51 2024-03-09T09:11:15.447 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.51 2024-03-09T09:11:15.608 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:11:15.608 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 52 -eq 48 ']' 2024-03-09T09:11:15.608 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.52 2024-03-09T09:11:15.608 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.52 2024-03-09T09:11:15.608 INFO:tasks.workunit.client.0.smithi067.stdout:v0.52 2024-03-09T09:11:15.608 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.52 2024-03-09T09:11:15.996 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 97% (134/137) Updating files: 98% (135/137) Updating files: 99% (136/137) Updating files: 100% (137/137) Updating files: 100% (137/137), done. 2024-03-09T09:11:16.074 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at a4428bd0e19 v0.50 2024-03-09T09:11:16.082 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.50 2024-03-09T09:11:16.715 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:11:16.715 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 51 -eq 48 ']' 2024-03-09T09:11:16.716 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.51 2024-03-09T09:11:16.716 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.51 2024-03-09T09:11:16.716 INFO:tasks.workunit.client.1.smithi134.stdout:v0.51 2024-03-09T09:11:16.716 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.51 2024-03-09T09:11:17.380 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:11:17.389 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:11:17.646 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 63% (116/183) Updating files: 64% (118/183) Updating files: 65% (119/183) Updating files: 66% (121/183) Updating files: 67% (123/183) Updating files: 68% (125/183) Updating files: 69% (127/183) Updating files: 70% (129/183) Updating files: 71% (130/183) Updating files: 72% (132/183) Updating files: 73% (134/183) Updating files: 74% (136/183) Updating files: 75% (138/183) Updating files: 76% (140/183) Updating files: 77% (141/183) Updating files: 78% (143/183) Updating files: 79% (145/183) Updating files: 80% (147/183) Updating files: 81% (149/183) Updating files: 82% (151/183) Updating files: 83% (152/183) Updating files: 84% (154/183) Updating files: 85% (156/183) Updating files: 86% (158/183) Updating files: 87% (160/183) Updating files: 88% (162/183) Updating files: 89% (163/183) Updating files: 90% (165/183) Updating files: 91% (167/183) Updating files: 92% (169/183) Updating files: 93% (171/183) Updating files: 94% (173/183) Updating files: 95% (174/183) Updating files: 96% (176/183) Updating files: 97% (178/183) Updating files: 98% (180/183) Updating files: 99% (182/183) Updating files: 100% (183/183) Updating files: 100% (183/183), done. 2024-03-09T09:11:17.719 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at e48859474c4 v0.52 2024-03-09T09:11:17.735 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.52 2024-03-09T09:11:17.901 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:11:17.958 INFO:tasks.workunit.client.0.smithi067.stdout:v0.53 2024-03-09T09:11:17.958 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:11:17.958 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 53 -eq 48 ']' 2024-03-09T09:11:17.958 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.53 2024-03-09T09:11:17.958 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.53 2024-03-09T09:11:17.959 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.53 2024-03-09T09:11:18.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.129+0000 7faa88800640 1 Processor -- start 2024-03-09T09:11:18.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.129+0000 7faa88800640 1 -- start start 2024-03-09T09:11:18.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.131+0000 7faa88800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faa80153370 0x7faa800c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:18.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.131+0000 7faa88800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa80153750 0x7faa800c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:18.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.131+0000 7faa88800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faa8006dc00 0x7faa800729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:18.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.131+0000 7faa88800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa80156dc0 con 0x7faa8006dc00 2024-03-09T09:11:18.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.131+0000 7faa88800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa800c4010 con 0x7faa80153370 2024-03-09T09:11:18.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.131+0000 7faa88800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa800c42f0 con 0x7faa80153750 2024-03-09T09:11:18.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.131+0000 7faa85400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa80153750 msgr2=0x7faa800c2ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:11:18.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.131+0000 7faa85400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa80153750 0x7faa800c2ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:18.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.131+0000 7faa85e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faa80153370 0x7faa800c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:18.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.132+0000 7faa86800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faa8006dc00 0x7faa800729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:18.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.132+0000 7faa85e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faa80153370 0x7faa800c29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:41462/0 (socket says 172.21.15.67:41462) 2024-03-09T09:11:18.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.132+0000 7faa85e00640 1 -- 172.21.15.67:0/2309484777 learned_addr learned my addr 172.21.15.67:0/2309484777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:18.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.132+0000 7faa86800640 1 -- 172.21.15.67:0/2309484777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa80153750 msgr2=0x7faa800c2ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:11:18.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.132+0000 7faa86800640 1 --2- 172.21.15.67:0/2309484777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa80153750 0x7faa800c2ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.132+0000 7faa86800640 1 -- 172.21.15.67:0/2309484777 >> v2:172.21.15.134:3300/0 conn(0x7faa80153370 msgr2=0x7faa800c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:18.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.132+0000 7faa86800640 1 --2- 172.21.15.67:0/2309484777 >> v2:172.21.15.134:3300/0 conn(0x7faa80153370 0x7faa800c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.132+0000 7faa86800640 1 -- 172.21.15.67:0/2309484777 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa800c5a50 con 0x7faa8006dc00 2024-03-09T09:11:18.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.133+0000 7faa86800640 1 --2- 172.21.15.67:0/2309484777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa8006dc00 0x7faa800729c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7faa800c5fd0 tx=0x7faa7c02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b635e0123d609cf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:18.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.134+0000 7faa84a00640 1 -- 172.21.15.67:0/2309484777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c039070 con 0x7faa8006dc00 2024-03-09T09:11:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.134+0000 7faa84a00640 1 -- 172.21.15.67:0/2309484777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c02cb70 con 0x7faa8006dc00 2024-03-09T09:11:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.134+0000 7faa84a00640 1 -- 172.21.15.67:0/2309484777 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c034440 con 0x7faa8006dc00 2024-03-09T09:11:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.135+0000 7faa88800640 1 -- 172.21.15.67:0/2309484777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa8006dc00 msgr2=0x7faa800729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.135+0000 7faa88800640 1 --2- 172.21.15.67:0/2309484777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa8006dc00 0x7faa800729c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7faa800c5fd0 tx=0x7faa7c02bdc0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.135+0000 7faa88800640 1 -- 172.21.15.67:0/2309484777 shutdown_connections 2024-03-09T09:11:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.135+0000 7faa88800640 1 --2- 172.21.15.67:0/2309484777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa8006dc00 0x7faa800729c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.135+0000 7faa88800640 1 --2- 172.21.15.67:0/2309484777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa80153750 0x7faa800c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.135+0000 7faa88800640 1 --2- 172.21.15.67:0/2309484777 >> v2:172.21.15.134:3300/0 conn(0x7faa80153370 0x7faa800c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.135+0000 7faa88800640 1 -- 172.21.15.67:0/2309484777 >> 172.21.15.67:0/2309484777 conn(0x7faa800ab320 msgr2=0x7faa800ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.135+0000 7faa88800640 1 -- 172.21.15.67:0/2309484777 shutdown_connections 2024-03-09T09:11:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.135+0000 7faa88800640 1 -- 172.21.15.67:0/2309484777 wait complete. 2024-03-09T09:11:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.138+0000 7faa88800640 1 Processor -- start 2024-03-09T09:11:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.138+0000 7faa88800640 1 -- start start 2024-03-09T09:11:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.139+0000 7faa88800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa8006dc00 0x7faa80159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.139+0000 7faa88800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa80153370 0x7faa801e1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.139+0000 7faa88800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa80153750 0x7faa801e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.139+0000 7faa88800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa800c5d30 con 0x7faa80153370 2024-03-09T09:11:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.139+0000 7faa88800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa80073dc0 con 0x7faa80153750 2024-03-09T09:11:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.139+0000 7faa88800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa800c3420 con 0x7faa8006dc00 2024-03-09T09:11:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.140+0000 7faa85e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa8006dc00 msgr2=0x7faa80159510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:11:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.140+0000 7faa85e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa8006dc00 0x7faa80159510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.140+0000 7faa85400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa80153370 0x7faa801e1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.140+0000 7faa85400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa80153370 0x7faa801e1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35452/0 (socket says 172.21.15.67:35452) 2024-03-09T09:11:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.140+0000 7faa85400640 1 -- 172.21.15.67:0/3689271507 learned_addr learned my addr 172.21.15.67:0/3689271507 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:18.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.140+0000 7faa86800640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa80153750 0x7faa801e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.141+0000 7faa85400640 1 -- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa8006dc00 msgr2=0x7faa80159510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:11:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.141+0000 7faa85400640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa8006dc00 0x7faa80159510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.141+0000 7faa85400640 1 -- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa80153750 msgr2=0x7faa801e1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.141+0000 7faa85400640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa80153750 0x7faa801e1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.141+0000 7faa85400640 1 -- 172.21.15.67:0/3689271507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80070350 con 0x7faa80153370 2024-03-09T09:11:18.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.141+0000 7faa86800640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa80153750 0x7faa801e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:11:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.142+0000 7faa85400640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa80153370 0x7faa801e1850 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faa7000b550 tx=0x7faa7000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.142+0000 7faa7a000640 1 -- 172.21.15.67:0/3689271507 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70013020 con 0x7faa80153370 2024-03-09T09:11:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.142+0000 7faa7a000640 1 -- 172.21.15.67:0/3689271507 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa70003de0 con 0x7faa80153370 2024-03-09T09:11:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.142+0000 7faa7a000640 1 -- 172.21.15.67:0/3689271507 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa700052b0 con 0x7faa80153370 2024-03-09T09:11:18.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.142+0000 7faa88800640 1 -- 172.21.15.67:0/3689271507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa80070630 con 0x7faa80153370 2024-03-09T09:11:18.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.143+0000 7faa88800640 1 -- 172.21.15.67:0/3689271507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa80153ed0 con 0x7faa80153370 2024-03-09T09:11:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.145+0000 7faa88800640 1 -- 172.21.15.67:0/3689271507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa80075a00 con 0x7faa80153370 2024-03-09T09:11:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.146+0000 7faa7a000640 1 -- 172.21.15.67:0/3689271507 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faa70016020 con 0x7faa80153370 2024-03-09T09:11:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.147+0000 7faa7a000640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa6406b4d0 0x7faa6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.147+0000 7faa7a000640 1 -- 172.21.15.67:0/3689271507 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7faa7008a210 con 0x7faa80153370 2024-03-09T09:11:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.147+0000 7faa85e00640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa6406b4d0 0x7faa6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.148+0000 7faa85e00640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa6406b4d0 0x7faa6406d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faa74006c20 tx=0x7faa74009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.152+0000 7faa7a000640 1 -- 172.21.15.67:0/3689271507 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faa7001d020 con 0x7faa80153370 2024-03-09T09:11:18.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.500+0000 7faa88800640 1 -- 172.21.15.67:0/3689271507 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7faa8006d260 con 0x7faa80153370 2024-03-09T09:11:18.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.500+0000 7faa7a000640 1 -- 172.21.15.67:0/3689271507 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7faa70058d10 con 0x7faa80153370 2024-03-09T09:11:18.505 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":136,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 -- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa6406b4d0 msgr2=0x7faa6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa6406b4d0 0x7faa6406d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faa74006c20 tx=0x7faa74009240 comp rx=0 tx=0).stop 2024-03-09T09:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 -- 172.21.15.67:0/3689271507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa80153370 msgr2=0x7faa801e1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa80153370 0x7faa801e1850 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faa7000b550 tx=0x7faa7000ba10 comp rx=0 tx=0).stop 2024-03-09T09:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 -- 172.21.15.67:0/3689271507 shutdown_connections 2024-03-09T09:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa6406b4d0 0x7faa6406d990 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa80153750 0x7faa801e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa80153370 0x7faa801e1850 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 --2- 172.21.15.67:0/3689271507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa8006dc00 0x7faa80159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:18.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 -- 172.21.15.67:0/3689271507 >> 172.21.15.67:0/3689271507 conn(0x7faa800ab320 msgr2=0x7faa800aa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 -- 172.21.15.67:0/3689271507 shutdown_connections 2024-03-09T09:11:18.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:18.503+0000 7faa88800640 1 -- 172.21.15.67:0/3689271507 wait complete. 2024-03-09T09:11:21.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:11:21.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.752+0000 7f3d88a00640 1 Processor -- start 2024-03-09T09:11:21.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.752+0000 7f3d88a00640 1 -- start start 2024-03-09T09:11:21.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.753+0000 7f3d88a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3d841532c0 0x7f3d840c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:21.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.753+0000 7f3d88a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3d840c2e80 0x7f3d8406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:21.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.753+0000 7f3d88a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d840729e0 con 0x7f3d84070430 2024-03-09T09:11:21.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.753+0000 7f3d88a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d84073310 con 0x7f3d840c2e80 2024-03-09T09:11:21.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.753+0000 7f3d88a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d840c2ab0 con 0x7f3d841532c0 2024-03-09T09:11:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.754+0000 7f3d82a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3d840c2e80 0x7f3d8406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.754+0000 7f3d83400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3d841532c0 0x7f3d840c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.754+0000 7f3d82a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3d840c2e80 0x7f3d8406fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52308/0 (socket says 172.21.15.67:52308) 2024-03-09T09:11:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.754+0000 7f3d82a00640 1 -- 172.21.15.67:0/465940026 learned_addr learned my addr 172.21.15.67:0/465940026 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.754+0000 7f3d83e00640 1 --1- 172.21.15.67:0/465940026 >> v1:172.21.15.67:6789/0 conn(0x7f3d84070430 0x7f3d84155e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42964/0 (socket says 172.21.15.67:42964) 2024-03-09T09:11:21.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.754+0000 7f3d82a00640 1 -- 172.21.15.67:0/465940026 >> v2:172.21.15.134:3300/0 conn(0x7f3d841532c0 msgr2=0x7f3d840c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.754+0000 7f3d82a00640 1 --2- 172.21.15.67:0/465940026 >> v2:172.21.15.134:3300/0 conn(0x7f3d841532c0 0x7f3d840c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.754+0000 7f3d82a00640 1 -- 172.21.15.67:0/465940026 >> v1:172.21.15.67:6789/0 conn(0x7f3d84070430 legacy=0x7f3d84155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.754+0000 7f3d82a00640 1 -- 172.21.15.67:0/465940026 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d84072cc0 con 0x7f3d840c2e80 2024-03-09T09:11:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.755+0000 7f3d82a00640 1 --2- 172.21.15.67:0/465940026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d840c2e80 0x7f3d8406fef0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3d840ca210 tx=0x7f3d7002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f812e575633cd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:21.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.756+0000 7f3d82000640 1 -- 172.21.15.67:0/465940026 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70035020 con 0x7f3d840c2e80 2024-03-09T09:11:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.756+0000 7f3d82000640 1 -- 172.21.15.67:0/465940026 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d70031870 con 0x7f3d840c2e80 2024-03-09T09:11:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.756+0000 7f3d82000640 1 -- 172.21.15.67:0/465940026 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d70031c40 con 0x7f3d840c2e80 2024-03-09T09:11:21.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.756+0000 7f3d88a00640 1 -- 172.21.15.67:0/465940026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d840c2e80 msgr2=0x7f3d8406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:21.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.756+0000 7f3d88a00640 1 --2- 172.21.15.67:0/465940026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d840c2e80 0x7f3d8406fef0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3d840ca210 tx=0x7f3d7002b910 comp rx=0 tx=0).stop 2024-03-09T09:11:21.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.757+0000 7f3d88a00640 1 -- 172.21.15.67:0/465940026 shutdown_connections 2024-03-09T09:11:21.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.757+0000 7f3d88a00640 1 --2- 172.21.15.67:0/465940026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d840c2e80 0x7f3d8406fef0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:21.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.757+0000 7f3d88a00640 1 --2- 172.21.15.67:0/465940026 >> v2:172.21.15.134:3300/0 conn(0x7f3d841532c0 0x7f3d840c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:21.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.757+0000 7f3d88a00640 1 -- 172.21.15.67:0/465940026 >> 172.21.15.67:0/465940026 conn(0x7f3d84066d90 msgr2=0x7f3d84067170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:21.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.757+0000 7f3d88a00640 1 -- 172.21.15.67:0/465940026 shutdown_connections 2024-03-09T09:11:21.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.757+0000 7f3d88a00640 1 -- 172.21.15.67:0/465940026 wait complete. 2024-03-09T09:11:21.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.760+0000 7f3d88a00640 1 Processor -- start 2024-03-09T09:11:21.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.760+0000 7f3d88a00640 1 -- start start 2024-03-09T09:11:21.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.762+0000 7f3d88a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d84070430 0x7f3d841ed030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:21.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.762+0000 7f3d88a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d840c2e80 0x7f3d84154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:21.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.762+0000 7f3d88a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d841532c0 0x7f3d841589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:21.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.762+0000 7f3d88a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d841566d0 con 0x7f3d840c2e80 2024-03-09T09:11:21.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.762+0000 7f3d88a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d84156550 con 0x7f3d841532c0 2024-03-09T09:11:21.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.762+0000 7f3d88a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d84156850 con 0x7f3d84070430 2024-03-09T09:11:21.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.762+0000 7f3d83400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d84070430 msgr2=0x7f3d841ed030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:11:21.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.762+0000 7f3d83400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d84070430 0x7f3d841ed030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:21.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.763+0000 7f3d83e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d841532c0 0x7f3d841589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:21.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.763+0000 7f3d83e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d841532c0 0x7f3d841589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:58020/0 (socket says 172.21.15.67:58020) 2024-03-09T09:11:21.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.763+0000 7f3d83e00640 1 -- 172.21.15.67:0/3583815771 learned_addr learned my addr 172.21.15.67:0/3583815771 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:21.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.764+0000 7f3d82a00640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d840c2e80 0x7f3d84154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:21.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.764+0000 7f3d83e00640 1 -- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d84070430 msgr2=0x7f3d841ed030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:11:21.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.764+0000 7f3d83e00640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d84070430 0x7f3d841ed030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:21.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.764+0000 7f3d83e00640 1 -- 172.21.15.67:0/3583815771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d840c2e80 msgr2=0x7f3d84154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:21.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.764+0000 7f3d83e00640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d840c2e80 0x7f3d84154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:21.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.764+0000 7f3d83e00640 1 -- 172.21.15.67:0/3583815771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8414dcc0 con 0x7f3d841532c0 2024-03-09T09:11:21.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.764+0000 7f3d82a00640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d840c2e80 0x7f3d84154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:11:21.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.766+0000 7f3d83e00640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d841532c0 0x7f3d841589e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00c780 tx=0x7f3d7c00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:21.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.766+0000 7f3d7b400640 1 -- 172.21.15.67:0/3583815771 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c017070 con 0x7f3d841532c0 2024-03-09T09:11:21.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.766+0000 7f3d7b400640 1 -- 172.21.15.67:0/3583815771 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7c00a640 con 0x7f3d841532c0 2024-03-09T09:11:21.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.766+0000 7f3d7b400640 1 -- 172.21.15.67:0/3583815771 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c00a9f0 con 0x7f3d841532c0 2024-03-09T09:11:21.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.766+0000 7f3d88a00640 1 -- 172.21.15.67:0/3583815771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8414dfa0 con 0x7f3d841532c0 2024-03-09T09:11:21.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.767+0000 7f3d88a00640 1 -- 172.21.15.67:0/3583815771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d84154c40 con 0x7f3d841532c0 2024-03-09T09:11:21.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.768+0000 7f3d88a00640 1 -- 172.21.15.67:0/3583815771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8404f3c0 con 0x7f3d841532c0 2024-03-09T09:11:21.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.769+0000 7f3d7b400640 1 -- 172.21.15.67:0/3583815771 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3d7c01a070 con 0x7f3d841532c0 2024-03-09T09:11:21.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.770+0000 7f3d7b400640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d6406b4d0 0x7f3d6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:21.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.770+0000 7f3d7b400640 1 -- 172.21.15.67:0/3583815771 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f3d7c089ec0 con 0x7f3d841532c0 2024-03-09T09:11:21.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.771+0000 7f3d83400640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d6406b4d0 0x7f3d6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:21.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.771+0000 7f3d83400640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d6406b4d0 0x7f3d6406d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009400 tx=0x7f3d74009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:21.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:21.778+0000 7f3d7b400640 1 -- 172.21.15.67:0/3583815771 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3d7c0582e0 con 0x7f3d841532c0 2024-03-09T09:11:22.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.082+0000 7f3d88a00640 1 -- 172.21.15.67:0/3583815771 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3d84154060 con 0x7f3d841532c0 2024-03-09T09:11:22.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.084+0000 7f3d7b400640 1 -- 172.21.15.67:0/3583815771 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f3d7c05c380 con 0x7f3d841532c0 2024-03-09T09:11:22.088 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":136,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:11:22.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.086+0000 7f3d88a00640 1 -- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d6406b4d0 msgr2=0x7f3d6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.086+0000 7f3d88a00640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d6406b4d0 0x7f3d6406d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009400 tx=0x7f3d74009190 comp rx=0 tx=0).stop 2024-03-09T09:11:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.086+0000 7f3d88a00640 1 -- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d841532c0 msgr2=0x7f3d841589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.086+0000 7f3d88a00640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d841532c0 0x7f3d841589e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00c780 tx=0x7f3d7c00cc40 comp rx=0 tx=0).stop 2024-03-09T09:11:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.086+0000 7f3d88a00640 1 -- 172.21.15.67:0/3583815771 shutdown_connections 2024-03-09T09:11:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.086+0000 7f3d88a00640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3d6406b4d0 0x7f3d6406d990 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.086+0000 7f3d88a00640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d841532c0 0x7f3d841589e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.086+0000 7f3d88a00640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d840c2e80 0x7f3d84154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.086+0000 7f3d88a00640 1 --2- 172.21.15.67:0/3583815771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d84070430 0x7f3d841ed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.086+0000 7f3d88a00640 1 -- 172.21.15.67:0/3583815771 >> 172.21.15.67:0/3583815771 conn(0x7f3d84066d90 msgr2=0x7f3d840c3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:22.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.087+0000 7f3d88a00640 1 -- 172.21.15.67:0/3583815771 shutdown_connections 2024-03-09T09:11:22.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:22.087+0000 7f3d88a00640 1 -- 172.21.15.67:0/3583815771 wait complete. 2024-03-09T09:11:25.112 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:11:25.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.338+0000 7f48c3000640 1 Processor -- start 2024-03-09T09:11:25.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.338+0000 7f48c3000640 1 -- start start 2024-03-09T09:11:25.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.340+0000 7f48c3000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.340+0000 7f48c3000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48bc156480 con 0x7f48bc155f80 2024-03-09T09:11:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.340+0000 7f48c3000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48bc06d830 con 0x7f48bc073be0 2024-03-09T09:11:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.340+0000 7f48c3000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48bc06db10 con 0x7f48bc073800 2024-03-09T09:11:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.340+0000 7f48c1200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f48bc155f80 0x7f48bc0c2cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42980/0 (socket says 172.21.15.67:42980) 2024-03-09T09:11:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.340+0000 7f48c1200640 1 -- 172.21.15.67:0/2946838095 learned_addr learned my addr 172.21.15.67:0/2946838095 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:11:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.341+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:25.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.341+0000 7f48c1c00640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 msgr2=0x7f48bc0c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:11:25.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.341+0000 7f48c1c00640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 msgr2=0x7f48bc0c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:11:25.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.341+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:11:25.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.341+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.543+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.543+0000 7f48c1c00640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 msgr2=0x7f48bc0c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:11:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.543+0000 7f48c1c00640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 msgr2=0x7f48bc0c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:11:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.543+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:11:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.543+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:11:25.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.945+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:25.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.945+0000 7f48c1c00640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 msgr2=0x7f48bc0c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:11:25.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.945+0000 7f48c1c00640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 msgr2=0x7f48bc0c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:11:25.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.945+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:11:25.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:25.945+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:11:26.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:26.747+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:26.747+0000 7f48c1c00640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 msgr2=0x7f48bc0c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:11:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:26.747+0000 7f48c1c00640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 msgr2=0x7f48bc0c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:11:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:26.748+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:11:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:26.748+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:11:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.341+0000 7f48baa00640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 msgr2=0x7f48bc0c3f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:11:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.341+0000 7f48baa00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.134:3300/0 conn(0x7f48bc073800 0x7f48bc0c3f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.341+0000 7f48baa00640 1 -- 172.21.15.67:0/2946838095 >> v1:172.21.15.134:6789/0 conn(0x7f48bc073be0 legacy=0x7f48bc0c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.341+0000 7f48baa00640 1 -- 172.21.15.67:0/2946838095 >> v1:172.21.15.67:6789/0 conn(0x7f48bc155f80 legacy=0x7f48bc0c2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.341+0000 7f48baa00640 1 --2- 172.21.15.67:0/2946838095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c005970 0x7f489c007da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.341+0000 7f48baa00640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 0x7f489c00a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.341+0000 7f48baa00640 1 -- 172.21.15.67:0/2946838095 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48bc06db10 con 0x7f489c0084c0 2024-03-09T09:11:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.341+0000 7f48baa00640 1 -- 172.21.15.67:0/2946838095 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c00aff0 con 0x7f489c005970 2024-03-09T09:11:28.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.342+0000 7f48c1200640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 0x7f489c00a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.342+0000 7f48c1c00640 1 -- 172.21.15.67:0/2946838095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c005970 msgr2=0x7f489c007da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:11:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.342+0000 7f48c1200640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 msgr2=0x7f489c00a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:11:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.342+0000 7f48c1200640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 msgr2=0x7f489c00a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:11:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.342+0000 7f48c1200640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 0x7f489c00a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:11:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.342+0000 7f48c1200640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 0x7f489c00a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.342+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c005970 0x7f489c007da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:28.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.543+0000 7f48c1c00640 1 -- 172.21.15.67:0/2946838095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c005970 msgr2=0x7f489c007da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:11:28.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.543+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c005970 0x7f489c007da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:11:28.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.543+0000 7f48c1200640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 0x7f489c00a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:28.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.543+0000 7f48c1200640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 msgr2=0x7f489c00a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:11:28.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.543+0000 7f48c1200640 1 -- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 msgr2=0x7f489c00a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:11:28.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.543+0000 7f48c1200640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 0x7f489c00a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:11:28.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.544+0000 7f48c1200640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 0x7f489c00a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:11:28.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.944+0000 7f48c1c00640 1 -- 172.21.15.67:0/2946838095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c005970 msgr2=0x7f489c007da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:11:28.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.944+0000 7f48c1c00640 1 --2- 172.21.15.67:0/2946838095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c005970 0x7f489c007da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:11:28.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.945+0000 7f48c1200640 1 --2- 172.21.15.67:0/2946838095 >> v2:172.21.15.67:3300/0 conn(0x7f489c0084c0 0x7f489c00a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:28.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.945+0000 7f48c1200640 1 -- 172.21.15.67:0/2946838095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c005970 msgr2=0x7f489c007da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:11:28.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.945+0000 7f48c1200640 1 --2- 172.21.15.67:0/2946838095 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c005970 0x7f489c007da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:28.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.945+0000 7f48c1200640 1 -- 172.21.15.67:0/2946838095 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc0c25c0 con 0x7f489c0084c0 2024-03-09T09:11:28.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.947+0000 7f48c1200640 1 --2- 172.21.15.67:0/2946838095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f489c0084c0 0x7f489c00a940 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f48bc1563b0 tx=0x7f48a802bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd32200df6e7d104 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:28.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.947+0000 7f48bbe00640 1 -- 172.21.15.67:0/2946838095 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a802d6f0 con 0x7f489c0084c0 2024-03-09T09:11:28.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.947+0000 7f48bbe00640 1 -- 172.21.15.67:0/2946838095 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a802d890 con 0x7f489c0084c0 2024-03-09T09:11:28.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.947+0000 7f48bbe00640 1 -- 172.21.15.67:0/2946838095 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a802dc60 con 0x7f489c0084c0 2024-03-09T09:11:28.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.948+0000 7f48c3000640 1 -- 172.21.15.67:0/2946838095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f489c0084c0 msgr2=0x7f489c00a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:28.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.948+0000 7f48c3000640 1 --2- 172.21.15.67:0/2946838095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f489c0084c0 0x7f489c00a940 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f48bc1563b0 tx=0x7f48a802bb40 comp rx=0 tx=0).stop 2024-03-09T09:11:28.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.948+0000 7f48c2600640 1 -- 172.21.15.67:0/2946838095 reap_dead start 2024-03-09T09:11:28.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.948+0000 7f48c3000640 1 -- 172.21.15.67:0/2946838095 shutdown_connections 2024-03-09T09:11:28.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.948+0000 7f48c3000640 1 --2- 172.21.15.67:0/2946838095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f489c0084c0 0x7f489c00a940 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:28.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.948+0000 7f48c3000640 1 -- 172.21.15.67:0/2946838095 >> 172.21.15.67:0/2946838095 conn(0x7f48bc0a97d0 msgr2=0x7f48bc0a9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:28.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.948+0000 7f48c3000640 1 -- 172.21.15.67:0/2946838095 shutdown_connections 2024-03-09T09:11:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.948+0000 7f48c3000640 1 -- 172.21.15.67:0/2946838095 wait complete. 2024-03-09T09:11:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.950+0000 7f48c3000640 1 Processor -- start 2024-03-09T09:11:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.951+0000 7f48c3000640 1 -- start start 2024-03-09T09:11:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c3000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c0084c0 0x7f48bc14fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c3000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48bc1ec490 0x7f48bc1ef720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c3000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc0c10c0 0x7f48bc151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c3000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc0c3560 con 0x7f48bc1ec490 2024-03-09T09:11:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c3000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc0c33e0 con 0x7f48bc0c10c0 2024-03-09T09:11:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c3000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc0c7040 con 0x7f489c0084c0 2024-03-09T09:11:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c1c00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c0084c0 msgr2=0x7f48bc14fe70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:11:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c1c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c0084c0 0x7f48bc14fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48bc1ec490 0x7f48bc1ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48bc1ec490 0x7f48bc1ef720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33870/0 (socket says 172.21.15.67:33870) 2024-03-09T09:11:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c1200640 1 -- 172.21.15.67:0/4117616058 learned_addr learned my addr 172.21.15.67:0/4117616058 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c2600640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc0c10c0 0x7f48bc151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c1200640 1 -- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c0084c0 msgr2=0x7f48bc14fe70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:11:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c1200640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c0084c0 0x7f48bc14fe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c1200640 1 -- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc0c10c0 msgr2=0x7f48bc151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c1200640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc0c10c0 0x7f48bc151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.952+0000 7f48c1200640 1 -- 172.21.15.67:0/4117616058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc1507a0 con 0x7f48bc1ec490 2024-03-09T09:11:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.953+0000 7f48c2600640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc0c10c0 0x7f48bc151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:11:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.953+0000 7f48c1200640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48bc1ec490 0x7f48bc1ef720 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f48a8005ce0 tx=0x7f48a8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.953+0000 7f48b9600640 1 -- 172.21.15.67:0/4117616058 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a803c040 con 0x7f48bc1ec490 2024-03-09T09:11:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.953+0000 7f48b9600640 1 -- 172.21.15.67:0/4117616058 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a8047070 con 0x7f48bc1ec490 2024-03-09T09:11:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.953+0000 7f48b9600640 1 -- 172.21.15.67:0/4117616058 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8005280 con 0x7f48bc1ec490 2024-03-09T09:11:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.953+0000 7f48c3000640 1 -- 172.21.15.67:0/4117616058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48bc1504c0 con 0x7f48bc1ec490 2024-03-09T09:11:28.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.953+0000 7f48c3000640 1 -- 172.21.15.67:0/4117616058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48bc1ebc90 con 0x7f48bc1ec490 2024-03-09T09:11:28.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.955+0000 7f48c3000640 1 -- 172.21.15.67:0/4117616058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48bc1eb7a0 con 0x7f48bc1ec490 2024-03-09T09:11:28.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.956+0000 7f48b9600640 1 -- 172.21.15.67:0/4117616058 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f48a8005420 con 0x7f48bc1ec490 2024-03-09T09:11:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.957+0000 7f48b9600640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f489c06c390 0x7f489c06e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.957+0000 7f48b9600640 1 -- 172.21.15.67:0/4117616058 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f48a80acc10 con 0x7f48bc1ec490 2024-03-09T09:11:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.958+0000 7f48c1c00640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f489c06c390 0x7f489c06e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:28.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.959+0000 7f48c1c00640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f489c06c390 0x7f489c06e850 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f48b0006fd0 tx=0x7f48b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:28.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:28.962+0000 7f48b9600640 1 -- 172.21.15.67:0/4117616058 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f48bc1507a0 con 0x7f48bc1ec490 2024-03-09T09:11:29.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.255+0000 7f48c3000640 1 -- 172.21.15.67:0/4117616058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f48bc04f3c0 con 0x7f48bc1ec490 2024-03-09T09:11:29.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.255+0000 7f48b9600640 1 -- 172.21.15.67:0/4117616058 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f48a807b600 con 0x7f48bc1ec490 2024-03-09T09:11:29.259 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":140,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:11:29.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 -- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f489c06c390 msgr2=0x7f489c06e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f489c06c390 0x7f489c06e850 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f48b0006fd0 tx=0x7f48b0008040 comp rx=0 tx=0).stop 2024-03-09T09:11:29.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 -- 172.21.15.67:0/4117616058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48bc1ec490 msgr2=0x7f48bc1ef720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48bc1ec490 0x7f48bc1ef720 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f48a8005ce0 tx=0x7f48a8001120 comp rx=0 tx=0).stop 2024-03-09T09:11:29.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 -- 172.21.15.67:0/4117616058 shutdown_connections 2024-03-09T09:11:29.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f489c06c390 0x7f489c06e850 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48bc0c10c0 0x7f48bc151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48bc1ec490 0x7f48bc1ef720 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 --2- 172.21.15.67:0/4117616058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f489c0084c0 0x7f48bc14fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 -- 172.21.15.67:0/4117616058 >> 172.21.15.67:0/4117616058 conn(0x7f48bc0a97d0 msgr2=0x7f48bc0c2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:29.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 -- 172.21.15.67:0/4117616058 shutdown_connections 2024-03-09T09:11:29.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.258+0000 7f48c3000640 1 -- 172.21.15.67:0/4117616058 wait complete. 2024-03-09T09:11:29.282 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:11:29.282 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:11:29.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:11:29.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.501+0000 7f9b4f800640 1 Processor -- start 2024-03-09T09:11:29.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.501+0000 7f9b4f800640 1 -- start start 2024-03-09T09:11:29.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.504+0000 7f9b4f800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9b480caa20 0x7f9b480cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.504+0000 7f9b4f800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b4806b040 con 0x7f9b48153460 2024-03-09T09:11:29.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.504+0000 7f9b4f800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b48156710 con 0x7f9b480c8c60 2024-03-09T09:11:29.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.504+0000 7f9b4f800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b480ca590 con 0x7f9b480caa20 2024-03-09T09:11:29.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.504+0000 7f9b4ce00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9b48153460 0x7f9b48155ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45592/0 (socket says 172.21.15.67:45592) 2024-03-09T09:11:29.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.504+0000 7f9b4ce00640 1 -- 172.21.15.67:0/2672321872 learned_addr learned my addr 172.21.15.67:0/2672321872 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:11:29.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.504+0000 7f9b4d800640 1 --2- 172.21.15.67:0/2672321872 >> v2:172.21.15.67:3300/0 conn(0x7f9b480caa20 0x7f9b480cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.504+0000 7f9b47400640 1 -- 172.21.15.67:0/2672321872 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997658524 0 0) 0x7f9b3c001490 con 0x7f9b48153460 2024-03-09T09:11:29.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.505+0000 7f9b47400640 1 -- 172.21.15.67:0/2672321872 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b30003520 con 0x7f9b48153460 2024-03-09T09:11:29.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.505+0000 7f9b4d800640 1 -- 172.21.15.67:0/2672321872 >> v1:172.21.15.134:6789/0 conn(0x7f9b480c8c60 legacy=0x7f9b480c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.505+0000 7f9b4d800640 1 -- 172.21.15.67:0/2672321872 >> v1:172.21.15.67:6789/0 conn(0x7f9b48153460 legacy=0x7f9b48155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.505+0000 7f9b4d800640 1 -- 172.21.15.67:0/2672321872 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b481569f0 con 0x7f9b480caa20 2024-03-09T09:11:29.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.506+0000 7f9b4d800640 1 --2- 172.21.15.67:0/2672321872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480caa20 0x7f9b480cae50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9b400097a0 tx=0x7f9b4002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e4da3b59d5fe6f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:29.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.506+0000 7f9b47400640 1 -- 172.21.15.67:0/2672321872 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40035020 con 0x7f9b480caa20 2024-03-09T09:11:29.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.506+0000 7f9b47400640 1 -- 172.21.15.67:0/2672321872 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b40031830 con 0x7f9b480caa20 2024-03-09T09:11:29.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.506+0000 7f9b47400640 1 -- 172.21.15.67:0/2672321872 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40031c00 con 0x7f9b480caa20 2024-03-09T09:11:29.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.507+0000 7f9b4f800640 1 -- 172.21.15.67:0/2672321872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480caa20 msgr2=0x7f9b480cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.507+0000 7f9b4f800640 1 --2- 172.21.15.67:0/2672321872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480caa20 0x7f9b480cae50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9b400097a0 tx=0x7f9b4002b8d0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.507+0000 7f9b4f800640 1 -- 172.21.15.67:0/2672321872 shutdown_connections 2024-03-09T09:11:29.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.507+0000 7f9b4f800640 1 --2- 172.21.15.67:0/2672321872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480caa20 0x7f9b480cae50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.507+0000 7f9b4f800640 1 -- 172.21.15.67:0/2672321872 >> 172.21.15.67:0/2672321872 conn(0x7f9b480c8170 msgr2=0x7f9b480c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:29.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.507+0000 7f9b4f800640 1 -- 172.21.15.67:0/2672321872 shutdown_connections 2024-03-09T09:11:29.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.507+0000 7f9b4f800640 1 -- 172.21.15.67:0/2672321872 wait complete. 2024-03-09T09:11:29.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.509+0000 7f9b4f800640 1 Processor -- start 2024-03-09T09:11:29.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.509+0000 7f9b4f800640 1 -- start start 2024-03-09T09:11:29.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4f800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480c8c60 0x7f9b481ef400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4f800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480caa20 0x7f9b481ed090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4f800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b48153460 0x7f9b481ed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4f800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b480c9f20 con 0x7f9b480c8c60 2024-03-09T09:11:29.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4f800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b48153840 con 0x7f9b480caa20 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4f800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b480ca0a0 con 0x7f9b48153460 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480c8c60 0x7f9b481ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4d800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b48153460 msgr2=0x7f9b481ed5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4d800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b48153460 0x7f9b481ed5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480c8c60 0x7f9b481ef400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33894/0 (socket says 172.21.15.67:33894) 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4ce00640 1 -- 172.21.15.67:0/1744094066 learned_addr learned my addr 172.21.15.67:0/1744094066 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b47e00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480caa20 0x7f9b481ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4ce00640 1 -- 172.21.15.67:0/1744094066 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b48153460 msgr2=0x7f9b481ed5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4ce00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b48153460 0x7f9b481ed5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4ce00640 1 -- 172.21.15.67:0/1744094066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480caa20 msgr2=0x7f9b481ed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4ce00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480caa20 0x7f9b481ed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.510+0000 7f9b4ce00640 1 -- 172.21.15.67:0/1744094066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b480c0190 con 0x7f9b480c8c60 2024-03-09T09:11:29.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.511+0000 7f9b4ce00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480c8c60 0x7f9b481ef400 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00cbf0 tx=0x7f9b3c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.511+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c017070 con 0x7f9b480c8c60 2024-03-09T09:11:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.511+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c00ad40 con 0x7f9b480c8c60 2024-03-09T09:11:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.511+0000 7f9b4f800640 1 -- 172.21.15.67:0/1744094066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4814a7b0 con 0x7f9b480c8c60 2024-03-09T09:11:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.511+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c012620 con 0x7f9b480c8c60 2024-03-09T09:11:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.511+0000 7f9b4f800640 1 -- 172.21.15.67:0/1744094066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4814ad70 con 0x7f9b480c8c60 2024-03-09T09:11:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.512+0000 7f9b4f800640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b481ef940 0x7f9b48148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.512+0000 7f9b4f800640 1 -- 172.21.15.67:0/1744094066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4814b570 con 0x7f9b481ef940 2024-03-09T09:11:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.513+0000 7f9b47e00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b481ef940 0x7f9b48148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.513+0000 7f9b47e00640 1 -- 172.21.15.67:0/1744094066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b481efef0 con 0x7f9b481ef940 2024-03-09T09:11:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.513+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9b3c01a070 con 0x7f9b480c8c60 2024-03-09T09:11:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.514+0000 7f9b47e00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b481ef940 0x7f9b48148590 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9b481e9a60 tx=0x7f9b3400e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:29.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.515+0000 7f9b44c00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b2806b480 0x7f9b2806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.515+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9b3c08a390 con 0x7f9b480c8c60 2024-03-09T09:11:29.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.515+0000 7f9b4d800640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b2806b480 0x7f9b2806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.515+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34019070 con 0x7f9b481ef940 2024-03-09T09:11:29.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.515+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f9b340052e0 con 0x7f9b481ef940 2024-03-09T09:11:29.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.515+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b481ef940 msgr2=0x7f9b48148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.515+0000 7f9b44c00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b481ef940 0x7f9b48148590 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9b481e9a60 tx=0x7f9b3400e8b0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.515+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b481ef940 msgr2=0x7f9b48148590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:29.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.515+0000 7f9b44c00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b481ef940 0x7f9b48148590 secure :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9b481e9a60 tx=0x7f9b3400e8b0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.516+0000 7f9b4d800640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b2806b480 0x7f9b2806d940 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9b40036000 tx=0x7f9b40008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:29.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.523+0000 7f9b4f800640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4814afb0 0x7f9b48148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.523+0000 7f9b4f800640 1 -- 172.21.15.67:0/1744094066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b480752b0 con 0x7f9b4814afb0 2024-03-09T09:11:29.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.523+0000 7f9b47e00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4814afb0 0x7f9b48148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.523+0000 7f9b47e00640 1 -- 172.21.15.67:0/1744094066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9b48075a00 con 0x7f9b4814afb0 2024-03-09T09:11:29.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.524+0000 7f9b47e00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4814afb0 0x7f9b48148ad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9b34016000 tx=0x7f9b34000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.524+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34019040 con 0x7f9b4814afb0 2024-03-09T09:11:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.524+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7f9b340035f0 con 0x7f9b4814afb0 2024-03-09T09:11:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.524+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4814afb0 msgr2=0x7f9b48148ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.524+0000 7f9b44c00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4814afb0 0x7f9b48148ad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9b34016000 tx=0x7f9b34000ed0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.524+0000 7f9b44c00640 1 -- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4814afb0 msgr2=0x7f9b48148ad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:29.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.524+0000 7f9b44c00640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b4814afb0 0x7f9b48148ad0 secure :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9b34016000 tx=0x7f9b34000ed0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.529 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:11:29.529 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:29.529 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:29.529 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:11:29.529 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 140, 2024-03-09T09:11:29.529 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:11:29.529 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:11:29.529 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:11:29.529 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:29.529 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:11:29.529 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:29.530 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:29.531 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:11:29.532 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:29.533 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:29.534 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:29.535 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:29.536 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:29.537 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:29.539 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:29.540 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.541 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:29.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:11:29.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.542 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:29.542 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:29.542 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:29.542 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.542 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.542 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:11:29.542 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:11:29.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.525+0000 7f9b4f800640 1 -- 172.21.15.67:0/1744094066 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b2806b480 msgr2=0x7f9b2806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.525+0000 7f9b4f800640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b2806b480 0x7f9b2806d940 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9b40036000 tx=0x7f9b40008030 comp rx=0 tx=0).stop 2024-03-09T09:11:29.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.525+0000 7f9b4f800640 1 -- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480c8c60 msgr2=0x7f9b481ef400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.525+0000 7f9b4f800640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480c8c60 0x7f9b481ef400 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00cbf0 tx=0x7f9b3c00a430 comp rx=0 tx=0).stop 2024-03-09T09:11:29.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.525+0000 7f9b4d800640 1 -- 172.21.15.67:0/1744094066 reap_dead start 2024-03-09T09:11:29.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.525+0000 7f9b4f800640 1 -- 172.21.15.67:0/1744094066 shutdown_connections 2024-03-09T09:11:29.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.525+0000 7f9b4f800640 1 --2- 172.21.15.67:0/1744094066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480c8c60 0x7f9b481ef400 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.525+0000 7f9b4f800640 1 -- 172.21.15.67:0/1744094066 >> 172.21.15.67:0/1744094066 conn(0x7f9b480c8170 msgr2=0x7f9b48152150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:29.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.525+0000 7f9b4f800640 1 -- 172.21.15.67:0/1744094066 shutdown_connections 2024-03-09T09:11:29.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.525+0000 7f9b4f800640 1 -- 172.21.15.67:0/1744094066 wait complete. 2024-03-09T09:11:29.551 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:11:29.551 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:11:29.552 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:11:29.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.804+0000 7f1759200640 1 Processor -- start 2024-03-09T09:11:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.804+0000 7f1759200640 1 -- start start 2024-03-09T09:11:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.805+0000 7f1759200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f17540ac9a0 0x7f17541510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.805+0000 7f1759200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1754071190 0x7f1754151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.805+0000 7f1759200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17540c8200 con 0x7f17540c2780 2024-03-09T09:11:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.805+0000 7f1759200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175406d590 con 0x7f1754071190 2024-03-09T09:11:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.805+0000 7f1759200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17540c3740 con 0x7f17540ac9a0 2024-03-09T09:11:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.806+0000 7f1752800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1754071190 0x7f1754151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.806+0000 7f1752800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1754071190 0x7f1754151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33922/0 (socket says 172.21.15.67:33922) 2024-03-09T09:11:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.806+0000 7f1752800640 1 -- 172.21.15.67:0/735507613 learned_addr learned my addr 172.21.15.67:0/735507613 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.806+0000 7f1751400640 1 --1- 172.21.15.67:0/735507613 >> v1:172.21.15.67:6789/0 conn(0x7f17540c2780 0x7f1754154190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:45608/0 (socket says 172.21.15.67:45608) 2024-03-09T09:11:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.806+0000 7f1751e00640 1 --2- 172.21.15.67:0/735507613 >> v2:172.21.15.134:3300/0 conn(0x7f17540ac9a0 0x7f17541510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.806+0000 7f1752800640 1 -- 172.21.15.67:0/735507613 >> v2:172.21.15.134:3300/0 conn(0x7f17540ac9a0 msgr2=0x7f17541510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.806+0000 7f1752800640 1 --2- 172.21.15.67:0/735507613 >> v2:172.21.15.134:3300/0 conn(0x7f17540ac9a0 0x7f17541510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.806+0000 7f1752800640 1 -- 172.21.15.67:0/735507613 >> v1:172.21.15.67:6789/0 conn(0x7f17540c2780 legacy=0x7f1754154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.806+0000 7f1752800640 1 -- 172.21.15.67:0/735507613 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17540c3990 con 0x7f1754071190 2024-03-09T09:11:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.806+0000 7f1750a00640 1 -- 172.21.15.67:0/735507613 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153140732 0 0) 0x7f17540c8200 con 0x7f17540c2780 2024-03-09T09:11:29.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.807+0000 7f1752800640 1 --2- 172.21.15.67:0/735507613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1754071190 0x7f1754151620 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f175406ed40 tx=0x7f174c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d28836108f1f2c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:29.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.808+0000 7f1750a00640 1 -- 172.21.15.67:0/735507613 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c02ca80 con 0x7f1754071190 2024-03-09T09:11:29.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.808+0000 7f1750a00640 1 -- 172.21.15.67:0/735507613 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c02cc20 con 0x7f1754071190 2024-03-09T09:11:29.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.808+0000 7f1750a00640 1 -- 172.21.15.67:0/735507613 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c0343f0 con 0x7f1754071190 2024-03-09T09:11:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.809+0000 7f1759200640 1 -- 172.21.15.67:0/735507613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1754071190 msgr2=0x7f1754151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.809+0000 7f1759200640 1 --2- 172.21.15.67:0/735507613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1754071190 0x7f1754151620 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f175406ed40 tx=0x7f174c02bee0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.809+0000 7f1759200640 1 -- 172.21.15.67:0/735507613 shutdown_connections 2024-03-09T09:11:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.809+0000 7f1759200640 1 --2- 172.21.15.67:0/735507613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1754071190 0x7f1754151620 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.809+0000 7f1759200640 1 --2- 172.21.15.67:0/735507613 >> v2:172.21.15.134:3300/0 conn(0x7f17540ac9a0 0x7f17541510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.809+0000 7f1759200640 1 -- 172.21.15.67:0/735507613 >> 172.21.15.67:0/735507613 conn(0x7f17540c7370 msgr2=0x7f17540c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.809+0000 7f1759200640 1 -- 172.21.15.67:0/735507613 shutdown_connections 2024-03-09T09:11:29.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.809+0000 7f1759200640 1 -- 172.21.15.67:0/735507613 wait complete. 2024-03-09T09:11:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.810+0000 7f1759200640 1 Processor -- start 2024-03-09T09:11:29.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.811+0000 7f1759200640 1 -- start start 2024-03-09T09:11:29.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.811+0000 7f1759200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1754071190 0x7f1754145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.811+0000 7f1759200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17540ac9a0 0x7f17541439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1759200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17540c2780 0x7f1754143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1759200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17540acd80 con 0x7f17540ac9a0 2024-03-09T09:11:29.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1759200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1754070c00 con 0x7f1754071190 2024-03-09T09:11:29.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1759200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17540acf00 con 0x7f17540c2780 2024-03-09T09:11:29.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1751400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17540ac9a0 0x7f17541439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1752800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17540c2780 msgr2=0x7f1754143f00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:11:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1751400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17540ac9a0 0x7f17541439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33932/0 (socket says 172.21.15.67:33932) 2024-03-09T09:11:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1751400640 1 -- 172.21.15.67:0/830118758 learned_addr learned my addr 172.21.15.67:0/830118758 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1752800640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17540c2780 0x7f1754143f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1751e00640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1754071190 0x7f1754145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1751400640 1 -- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17540c2780 msgr2=0x7f1754143f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:11:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1751400640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17540c2780 0x7f1754143f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1751400640 1 -- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1754071190 msgr2=0x7f1754145a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1751400640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1754071190 0x7f1754145a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.812+0000 7f1751400640 1 -- 172.21.15.67:0/830118758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17540aaeb0 con 0x7f17540ac9a0 2024-03-09T09:11:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.813+0000 7f1751400640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17540ac9a0 0x7f17541439c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f174000c410 tx=0x7f174000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.813+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17400074e0 con 0x7f17540ac9a0 2024-03-09T09:11:29.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.813+0000 7f1759200640 1 -- 172.21.15.67:0/830118758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17540ab170 con 0x7f17540ac9a0 2024-03-09T09:11:29.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.813+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17400047e0 con 0x7f17540ac9a0 2024-03-09T09:11:29.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.813+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740004bd0 con 0x7f17540ac9a0 2024-03-09T09:11:29.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.813+0000 7f1759200640 1 -- 172.21.15.67:0/830118758 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1754145fb0 con 0x7f17540ac9a0 2024-03-09T09:11:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.814+0000 7f1759200640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17540abdb0 0x7f1754152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.814+0000 7f1759200640 1 -- 172.21.15.67:0/830118758 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1754151f20 con 0x7f17540abdb0 2024-03-09T09:11:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.815+0000 7f1751e00640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17540abdb0 0x7f1754152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.815+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1740011070 con 0x7f17540ac9a0 2024-03-09T09:11:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.816+0000 7f1751e00640 1 -- 172.21.15.67:0/830118758 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1754075a00 con 0x7f17540abdb0 2024-03-09T09:11:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.816+0000 7f1751e00640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17540abdb0 0x7f1754152200 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f174400e710 tx=0x7f174400ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.817+0000 7f174a000640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f173406b480 0x7f173406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.817+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f174008a2a0 con 0x7f17540ac9a0 2024-03-09T09:11:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.817+0000 7f1752800640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f173406b480 0x7f173406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.819+0000 7f1752800640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f173406b480 0x7f173406d940 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f174c035000 tx=0x7f174c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:29.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.826+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744017070 con 0x7f17540abdb0 2024-03-09T09:11:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.827+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f174400c740 con 0x7f17540abdb0 2024-03-09T09:11:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.827+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17540abdb0 msgr2=0x7f1754152200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.827+0000 7f174a000640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17540abdb0 0x7f1754152200 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f174400e710 tx=0x7f174400ebd0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.827+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17540abdb0 msgr2=0x7f1754152200 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.827+0000 7f174a000640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17540abdb0 0x7f1754152200 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f174400e710 tx=0x7f174400ebd0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.839+0000 7f1759200640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17541ef210 0x7f1754152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:29.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.840+0000 7f1759200640 1 -- 172.21.15.67:0/830118758 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175406e050 con 0x7f17541ef210 2024-03-09T09:11:29.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.840+0000 7f1751e00640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17541ef210 0x7f1754152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:29.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.841+0000 7f1751e00640 1 -- 172.21.15.67:0/830118758 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1754146370 con 0x7f17541ef210 2024-03-09T09:11:29.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.841+0000 7f1751e00640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17541ef210 0x7f1754152740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f174400e710 tx=0x7f1744004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:29.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.841+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174400f040 con 0x7f17541ef210 2024-03-09T09:11:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.842+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4523 (secure 0 0 0) 0x7f1744013410 con 0x7f17541ef210 2024-03-09T09:11:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.842+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17541ef210 msgr2=0x7f1754152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.842+0000 7f174a000640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17541ef210 0x7f1754152740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f174400e710 tx=0x7f1744004730 comp rx=0 tx=0).stop 2024-03-09T09:11:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.842+0000 7f174a000640 1 -- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17541ef210 msgr2=0x7f1754152740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.842+0000 7f174a000640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17541ef210 0x7f1754152740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.846 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:11:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 140, 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:11:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:11:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:11:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:11:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:11:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:11:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:11:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:11:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:11:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:11:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:11:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:11:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:29.855 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:29.857 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:11:29.858 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:11:29.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.843+0000 7f1759200640 1 -- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f173406b480 msgr2=0x7f173406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.843+0000 7f1759200640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f173406b480 0x7f173406d940 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f174c035000 tx=0x7f174c030000 comp rx=0 tx=0).stop 2024-03-09T09:11:29.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.843+0000 7f1759200640 1 -- 172.21.15.67:0/830118758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17540ac9a0 msgr2=0x7f17541439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:29.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.843+0000 7f1759200640 1 --2- 172.21.15.67:0/830118758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17540ac9a0 0x7f17541439c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f174000c410 tx=0x7f174000c8d0 comp rx=0 tx=0).stop 2024-03-09T09:11:29.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.843+0000 7f1752800640 1 -- 172.21.15.67:0/830118758 reap_dead start 2024-03-09T09:11:29.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.843+0000 7f1752800640 1 -- 172.21.15.67:0/830118758 reap_dead start 2024-03-09T09:11:29.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.843+0000 7f1759200640 1 -- 172.21.15.67:0/830118758 shutdown_connections 2024-03-09T09:11:29.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.843+0000 7f1759200640 1 -- 172.21.15.67:0/830118758 >> 172.21.15.67:0/830118758 conn(0x7f17540c7370 msgr2=0x7f17540c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:29.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.843+0000 7f1759200640 1 -- 172.21.15.67:0/830118758 shutdown_connections 2024-03-09T09:11:29.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:29.843+0000 7f1759200640 1 -- 172.21.15.67:0/830118758 wait complete. 2024-03-09T09:11:29.867 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:11:29.867 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:11:29.975 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 14% (55/386) Updating files: 15% (58/386) Updating files: 16% (62/386) Updating files: 17% (66/386) Updating files: 18% (70/386) Updating files: 18% (73/386) Updating files: 19% (74/386) Updating files: 20% (78/386) Updating files: 21% (82/386) Updating files: 22% (85/386) Updating files: 23% (89/386) Updating files: 24% (93/386) Updating files: 25% (97/386) Updating files: 26% (101/386) Updating files: 27% (105/386) Updating files: 28% (109/386) Updating files: 29% (112/386) Updating files: 29% (115/386) Updating files: 30% (116/386) Updating files: 31% (120/386) Updating files: 32% (124/386) Updating files: 33% (128/386) Updating files: 34% (132/386) Updating files: 35% (136/386) Updating files: 36% (139/386) Updating files: 37% (143/386) Updating files: 38% (147/386) Updating files: 39% (151/386) Updating files: 40% (155/386) Updating files: 41% (159/386) Updating files: 42% (163/386) Updating files: 43% (166/386) Updating files: 44% (170/386) Updating files: 45% (174/386) Updating files: 46% (178/386) Updating files: 47% (182/386) Updating files: 48% (186/386) Updating files: 49% (190/386) Updating files: 50% (193/386) Updating files: 50% (195/386) Updating files: 51% (197/386) Updating files: 52% (201/386) Updating files: 53% (205/386) Updating files: 54% (209/386) Updating files: 55% (213/386) Updating files: 56% (217/386) Updating files: 57% (221/386) Updating files: 58% (224/386) Updating files: 59% (228/386) Updating files: 60% (232/386) Updating files: 61% (236/386) Updating files: 62% (240/386) Updating files: 63% (244/386) Updating files: 64% (248/386) Updating files: 65% (251/386) Updating files: 66% (255/386) Updating files: 67% (259/386) Updating files: 68% (263/386) Updating files: 69% (267/386) Updating files: 70% (271/386) Updating files: 71% (275/386) Updating files: 72% (278/386) Updating files: 73% (282/386) Updating files: 74% (286/386) Updating files: 75% (290/386) Updating files: 76% (294/386) Updating files: 77% (298/386) Updating files: 78% (302/386) Updating files: 79% (305/386) Updating files: 80% (309/386) Updating files: 81% (313/386) Updating files: 82% (317/386) Updating files: 83% (321/386) Updating files: 84% (325/386) Updating files: 85% (329/386) Updating files: 86% (332/386) Updating files: 87% (336/386) Updating files: 88% (340/386) Updating files: 89% (344/386) Updating files: 90% (348/386) Updating files: 91% (352/386) Updating files: 92% (356/386) Updating files: 93% (359/386) Updating files: 94% (363/386) Updating files: 95% (367/386) Updating files: 96% (371/386) Updating files: 97% (375/386) Updating files: 98% (379/386) Updating files: 99% (383/386) Updating files: 100% (386/386) Updating files: 100% (386/386), done. 2024-03-09T09:11:30.045 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 2528b5ee105 v0.43 2024-03-09T09:11:30.062 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.53 2024-03-09T09:11:30.935 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:11:30.935 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 54 -eq 48 ']' 2024-03-09T09:11:30.935 INFO:tasks.workunit.client.0.smithi067.stdout:v0.54 2024-03-09T09:11:30.936 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.54 2024-03-09T09:11:30.936 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.54 2024-03-09T09:11:30.936 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.54 2024-03-09T09:11:33.790 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 4% (5/105) Updating files: 5% (6/105) Updating files: 6% (7/105) Updating files: 7% (8/105) Updating files: 8% (9/105) Updating files: 9% (10/105) Updating files: 10% (11/105) Updating files: 11% (12/105) Updating files: 12% (13/105) Updating files: 13% (14/105) Updating files: 14% (15/105) Updating files: 15% (16/105) Updating files: 16% (17/105) Updating files: 17% (18/105) Updating files: 18% (19/105) Updating files: 19% (20/105) Updating files: 20% (21/105) Updating files: 21% (23/105) Updating files: 22% (24/105) Updating files: 23% (25/105) Updating files: 24% (26/105) Updating files: 25% (27/105) Updating files: 26% (28/105) Updating files: 27% (29/105) Updating files: 28% (30/105) Updating files: 29% (31/105) Updating files: 30% (32/105) Updating files: 31% (33/105) Updating files: 32% (34/105) Updating files: 33% (35/105) Updating files: 34% (36/105) Updating files: 35% (37/105) Updating files: 36% (38/105) Updating files: 37% (39/105) Updating files: 38% (40/105) Updating files: 39% (41/105) Updating files: 40% (42/105) Updating files: 41% (44/105) Updating files: 42% (45/105) Updating files: 43% (46/105) Updating files: 44% (47/105) Updating files: 45% (48/105) Updating files: 46% (49/105) Updating files: 47% (50/105) Updating files: 48% (51/105) Updating files: 49% (52/105) Updating files: 50% (53/105) Updating files: 51% (54/105) Updating files: 52% (55/105) Updating files: 53% (56/105) Updating files: 54% (57/105) Updating files: 55% (58/105) Updating files: 56% (59/105) Updating files: 57% (60/105) Updating files: 58% (61/105) Updating files: 59% (62/105) Updating files: 60% (63/105) Updating files: 61% (65/105) Updating files: 62% (66/105) Updating files: 63% (67/105) Updating files: 64% (68/105) Updating files: 65% (69/105) Updating files: 66% (70/105) Updating files: 67% (71/105) Updating files: 68% (72/105) Updating files: 69% (73/105) Updating files: 70% (74/105) Updating files: 71% (75/105) Updating files: 72% (76/105) Updating files: 73% (77/105) Updating files: 74% (78/105) Updating files: 75% (79/105) Updating files: 76% (80/105) Updating files: 77% (81/105) Updating files: 78% (82/105) Updating files: 79% (83/105) Updating files: 80% (84/105) Updating files: 81% (86/105) Updating files: 82% (87/105) Updating files: 83% (88/105) Updating files: 84% (89/105) Updating files: 85% (90/105) Updating files: 86% (91/105) Updating files: 87% (92/105) Updating files: 88% (93/105) Updating files: 89% (94/105) Updating files: 90% (95/105) Updating files: 91% (96/105) Updating files: 92% (97/105) Updating files: 93% (98/105) Updating files: 94% (99/105) Updating files: 95% (100/105) Updating files: 96% (101/105) Updating files: 97% (102/105) Updating files: 98% (103/105) Updating files: 99% (104/105) Updating files: 100% (105/105) Updating files: 100% (105/105), done. 2024-03-09T09:11:33.878 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at c03ca95d235 v0.51 2024-03-09T09:11:33.881 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.51 2024-03-09T09:11:34.017 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:11:34.017 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 52 -eq 48 ']' 2024-03-09T09:11:34.018 INFO:tasks.workunit.client.1.smithi134.stdout:v0.52 2024-03-09T09:11:34.018 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.52 2024-03-09T09:11:34.018 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.52 2024-03-09T09:11:34.018 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.52 2024-03-09T09:11:34.453 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 25% (55/212) Updating files: 26% (56/212) Updating files: 27% (58/212) Updating files: 28% (60/212) Updating files: 29% (62/212) Updating files: 30% (64/212) Updating files: 31% (66/212) Updating files: 32% (68/212) Updating files: 33% (70/212) Updating files: 34% (73/212) Updating files: 35% (75/212) Updating files: 36% (77/212) Updating files: 37% (79/212) Updating files: 38% (81/212) Updating files: 39% (83/212) Updating files: 40% (85/212) Updating files: 41% (87/212) Updating files: 42% (90/212) Updating files: 43% (92/212) Updating files: 44% (94/212) Updating files: 45% (96/212) Updating files: 46% (98/212) Updating files: 47% (100/212) Updating files: 48% (102/212) Updating files: 49% (104/212) Updating files: 50% (106/212) Updating files: 51% (109/212) Updating files: 52% (111/212) Updating files: 53% (113/212) Updating files: 54% (115/212) Updating files: 55% (117/212) Updating files: 56% (119/212) Updating files: 57% (121/212) Updating files: 58% (123/212) Updating files: 58% (124/212) Updating files: 59% (126/212) Updating files: 60% (128/212) Updating files: 61% (130/212) Updating files: 62% (132/212) Updating files: 63% (134/212) Updating files: 64% (136/212) Updating files: 65% (138/212) Updating files: 66% (140/212) Updating files: 67% (143/212) Updating files: 68% (145/212) Updating files: 69% (147/212) Updating files: 70% (149/212) Updating files: 71% (151/212) Updating files: 72% (153/212) Updating files: 73% (155/212) Updating files: 74% (157/212) Updating files: 75% (159/212) Updating files: 76% (162/212) Updating files: 77% (164/212) Updating files: 78% (166/212) Updating files: 79% (168/212) Updating files: 80% (170/212) Updating files: 81% (172/212) Updating files: 82% (174/212) Updating files: 83% (176/212) Updating files: 84% (179/212) Updating files: 85% (181/212) Updating files: 86% (183/212) Updating files: 86% (184/212) Updating files: 87% (185/212) Updating files: 88% (187/212) Updating files: 89% (189/212) Updating files: 90% (191/212) Updating files: 91% (193/212) Updating files: 92% (196/212) Updating files: 93% (198/212) Updating files: 94% (200/212) Updating files: 95% (202/212) Updating files: 96% (204/212) Updating files: 97% (206/212) Updating files: 98% (208/212) Updating files: 99% (210/212) Updating files: 100% (212/212) Updating files: 100% (212/212), done. 2024-03-09T09:11:34.536 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 60b84b095b1 v0.54 2024-03-09T09:11:34.553 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.54 2024-03-09T09:11:34.867 INFO:tasks.workunit.client.0.smithi067.stdout:v0.55 2024-03-09T09:11:34.868 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:11:34.868 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 55 -eq 48 ']' 2024-03-09T09:11:34.868 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.55 2024-03-09T09:11:34.868 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.55 2024-03-09T09:11:34.868 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.55 2024-03-09T09:11:38.114 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 32% (59/183) Updating files: 33% (61/183) Updating files: 34% (63/183) Updating files: 35% (65/183) Updating files: 36% (66/183) Updating files: 37% (68/183) Updating files: 38% (70/183) Updating files: 39% (72/183) Updating files: 40% (74/183) Updating files: 40% (75/183) Updating files: 41% (76/183) Updating files: 42% (77/183) Updating files: 43% (79/183) Updating files: 44% (81/183) Updating files: 45% (83/183) Updating files: 46% (85/183) Updating files: 47% (87/183) Updating files: 48% (88/183) Updating files: 49% (90/183) Updating files: 50% (92/183) Updating files: 51% (94/183) Updating files: 52% (96/183) Updating files: 53% (97/183) Updating files: 54% (99/183) Updating files: 55% (101/183) Updating files: 56% (103/183) Updating files: 57% (105/183) Updating files: 58% (107/183) Updating files: 59% (108/183) Updating files: 60% (110/183) Updating files: 61% (112/183) Updating files: 62% (114/183) Updating files: 63% (116/183) Updating files: 64% (118/183) Updating files: 65% (119/183) Updating files: 66% (121/183) Updating files: 67% (123/183) Updating files: 68% (125/183) Updating files: 69% (127/183) Updating files: 70% (129/183) Updating files: 71% (130/183) Updating files: 72% (132/183) Updating files: 73% (134/183) Updating files: 74% (136/183) Updating files: 75% (138/183) Updating files: 76% (140/183) Updating files: 77% (141/183) Updating files: 78% (143/183) Updating files: 79% (145/183) Updating files: 80% (147/183) Updating files: 81% (149/183) Updating files: 82% (151/183) Updating files: 83% (152/183) Updating files: 84% (154/183) Updating files: 85% (156/183) Updating files: 86% (158/183) Updating files: 87% (160/183) Updating files: 88% (162/183) Updating files: 89% (163/183) Updating files: 90% (165/183) Updating files: 91% (167/183) Updating files: 92% (169/183) Updating files: 93% (171/183) Updating files: 94% (173/183) Updating files: 95% (174/183) Updating files: 96% (176/183) Updating files: 97% (178/183) Updating files: 98% (180/183) Updating files: 99% (182/183) Updating files: 100% (183/183) Updating files: 100% (183/183), done. 2024-03-09T09:11:38.330 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at e48859474c4 v0.52 2024-03-09T09:11:38.338 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.52 2024-03-09T09:11:38.948 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:11:38.948 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 53 -eq 48 ']' 2024-03-09T09:11:38.949 INFO:tasks.workunit.client.1.smithi134.stdout:v0.53 2024-03-09T09:11:38.949 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.53 2024-03-09T09:11:38.949 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.53 2024-03-09T09:11:38.949 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.53 2024-03-09T09:11:45.102 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 35% (137/386) Updating files: 36% (139/386) Updating files: 37% (143/386) Updating files: 38% (147/386) Updating files: 39% (151/386) Updating files: 40% (155/386) Updating files: 41% (159/386) Updating files: 42% (163/386) Updating files: 43% (166/386) Updating files: 44% (170/386) Updating files: 45% (174/386) Updating files: 46% (178/386) Updating files: 47% (182/386) Updating files: 47% (184/386) Updating files: 48% (186/386) Updating files: 49% (190/386) Updating files: 50% (193/386) Updating files: 51% (197/386) Updating files: 52% (201/386) Updating files: 53% (205/386) Updating files: 54% (209/386) Updating files: 55% (213/386) Updating files: 56% (217/386) Updating files: 57% (221/386) Updating files: 58% (224/386) Updating files: 59% (228/386) Updating files: 60% (232/386) Updating files: 61% (236/386) Updating files: 62% (240/386) Updating files: 63% (244/386) Updating files: 64% (248/386) Updating files: 65% (251/386) Updating files: 66% (255/386) Updating files: 67% (259/386) Updating files: 68% (263/386) Updating files: 69% (267/386) Updating files: 70% (271/386) Updating files: 71% (275/386) Updating files: 72% (278/386) Updating files: 73% (282/386) Updating files: 74% (286/386) Updating files: 75% (290/386) Updating files: 76% (294/386) Updating files: 77% (298/386) Updating files: 78% (302/386) Updating files: 79% (305/386) Updating files: 80% (309/386) Updating files: 81% (313/386) Updating files: 82% (317/386) Updating files: 83% (321/386) Updating files: 84% (325/386) Updating files: 85% (329/386) Updating files: 85% (331/386) Updating files: 86% (332/386) Updating files: 87% (336/386) Updating files: 88% (340/386) Updating files: 89% (344/386) Updating files: 90% (348/386) Updating files: 91% (352/386) Updating files: 92% (356/386) Updating files: 93% (359/386) Updating files: 94% (363/386) Updating files: 95% (367/386) Updating files: 96% (371/386) Updating files: 97% (375/386) Updating files: 98% (379/386) Updating files: 99% (383/386) Updating files: 100% (386/386) Updating files: 100% (386/386), done. 2024-03-09T09:11:45.133 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 2528b5ee105 v0.43 2024-03-09T09:11:45.139 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.53 2024-03-09T09:11:45.930 INFO:tasks.workunit.client.1.smithi134.stdout:v0.54 2024-03-09T09:11:45.930 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:11:45.930 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 54 -eq 48 ']' 2024-03-09T09:11:45.930 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.54 2024-03-09T09:11:45.930 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.54 2024-03-09T09:11:45.930 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.54 2024-03-09T09:11:47.408 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 4% (24/540) Updating files: 5% (27/540) Updating files: 6% (33/540) Updating files: 7% (38/540) Updating files: 8% (44/540) Updating files: 9% (49/540) Updating files: 10% (54/540) Updating files: 11% (60/540) Updating files: 12% (65/540) Updating files: 12% (66/540) Updating files: 13% (71/540) Updating files: 14% (76/540) Updating files: 15% (81/540) Updating files: 16% (87/540) Updating files: 17% (92/540) Updating files: 18% (98/540) Updating files: 19% (103/540) Updating files: 20% (108/540) Updating files: 21% (114/540) Updating files: 22% (119/540) Updating files: 23% (125/540) Updating files: 24% (130/540) Updating files: 25% (135/540) Updating files: 26% (141/540) Updating files: 27% (146/540) Updating files: 28% (152/540) Updating files: 29% (157/540) Updating files: 30% (162/540) Updating files: 30% (166/540) Updating files: 31% (168/540) Updating files: 32% (173/540) Updating files: 33% (179/540) Updating files: 34% (184/540) Updating files: 35% (189/540) Updating files: 36% (195/540) Updating files: 37% (200/540) Updating files: 38% (206/540) Updating files: 39% (211/540) Updating files: 39% (213/540) Updating files: 40% (216/540) Updating files: 41% (222/540) Updating files: 42% (227/540) Updating files: 43% (233/540) Updating files: 44% (238/540) Updating files: 45% (243/540) Updating files: 46% (249/540) Updating files: 46% (250/540) Updating files: 47% (254/540) Updating files: 48% (260/540) Updating files: 49% (265/540) Updating files: 50% (270/540) Updating files: 51% (276/540) Updating files: 52% (281/540) Updating files: 53% (287/540) Updating files: 54% (292/540) Updating files: 55% (297/540) Updating files: 56% (303/540) Updating files: 57% (308/540) Updating files: 58% (314/540) Updating files: 59% (319/540) Updating files: 60% (324/540) Updating files: 61% (330/540) Updating files: 62% (335/540) Updating files: 63% (341/540) Updating files: 64% (346/540) Updating files: 65% (351/540) Updating files: 66% (357/540) Updating files: 67% (362/540) Updating files: 68% (368/540) Updating files: 69% (373/540) Updating files: 70% (378/540) Updating files: 71% (384/540) Updating files: 72% (389/540) Updating files: 73% (395/540) Updating files: 74% (400/540) Updating files: 75% (405/540) Updating files: 76% (411/540) Updating files: 77% (416/540) Updating files: 77% (420/540) Updating files: 78% (422/540) Updating files: 79% (427/540) Updating files: 80% (432/540) Updating files: 81% (438/540) Updating files: 82% (443/540) Updating files: 83% (449/540) Updating files: 84% (454/540) Updating files: 85% (459/540) Updating files: 86% (465/540) Updating files: 87% (470/540) Updating files: 88% (476/540) Updating files: 89% (481/540) Updating files: 89% (483/540) Updating files: 90% (486/540) Updating files: 91% (492/540) Updating files: 92% (497/540) Updating files: 93% (503/540) Updating files: 94% (508/540) Updating files: 95% (513/540) Updating files: 96% (519/540) Updating files: 97% (524/540) Updating files: 98% (530/540) Updating files: 99% (535/540) Updating files: 100% (540/540) Updating files: 100% (540/540), done. 2024-03-09T09:11:47.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:11:47.526 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:11:48.156 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 690f8175606 v0.55 2024-03-09T09:11:48.172 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.55 2024-03-09T09:11:48.274 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:11:48.274 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 56 -eq 48 ']' 2024-03-09T09:11:48.274 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.56 2024-03-09T09:11:48.274 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.56 2024-03-09T09:11:48.274 INFO:tasks.workunit.client.0.smithi067.stdout:v0.56 2024-03-09T09:11:48.275 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.56 2024-03-09T09:11:49.253 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 46% (99/212) Updating files: 47% (100/212) Updating files: 48% (102/212) Updating files: 49% (104/212) Updating files: 50% (106/212) Updating files: 51% (109/212) Updating files: 52% (111/212) Updating files: 53% (113/212) Updating files: 54% (115/212) Updating files: 55% (117/212) Updating files: 56% (119/212) Updating files: 57% (121/212) Updating files: 58% (123/212) Updating files: 59% (126/212) Updating files: 60% (128/212) Updating files: 61% (130/212) Updating files: 62% (132/212) Updating files: 63% (134/212) Updating files: 64% (136/212) Updating files: 65% (138/212) Updating files: 66% (140/212) Updating files: 66% (141/212) Updating files: 67% (143/212) Updating files: 68% (145/212) Updating files: 69% (147/212) Updating files: 70% (149/212) Updating files: 71% (151/212) Updating files: 72% (153/212) Updating files: 73% (155/212) Updating files: 74% (157/212) Updating files: 75% (159/212) Updating files: 76% (162/212) Updating files: 77% (164/212) Updating files: 78% (166/212) Updating files: 79% (168/212) Updating files: 80% (170/212) Updating files: 81% (172/212) Updating files: 82% (174/212) Updating files: 83% (176/212) Updating files: 84% (179/212) Updating files: 85% (181/212) Updating files: 86% (183/212) Updating files: 87% (185/212) Updating files: 88% (187/212) Updating files: 89% (189/212) Updating files: 90% (191/212) Updating files: 91% (193/212) Updating files: 92% (196/212) Updating files: 93% (198/212) Updating files: 94% (200/212) Updating files: 95% (202/212) Updating files: 96% (204/212) Updating files: 97% (206/212) Updating files: 98% (208/212) Updating files: 99% (210/212) Updating files: 100% (212/212) Updating files: 100% (212/212), done. 2024-03-09T09:11:49.315 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 60b84b095b1 v0.54 2024-03-09T09:11:49.334 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.54 2024-03-09T09:11:49.838 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:11:49.838 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 55 -eq 48 ']' 2024-03-09T09:11:49.838 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.55 2024-03-09T09:11:49.838 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.55 2024-03-09T09:11:49.839 INFO:tasks.workunit.client.1.smithi134.stdout:v0.55 2024-03-09T09:11:49.839 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.55 2024-03-09T09:11:49.867 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:11:49.867 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:11:49.867 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:11:49.868 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:11:49.872 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:11:49.872 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:11:49.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:11:50.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.096+0000 7fafedc00640 1 Processor -- start 2024-03-09T09:11:50.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.096+0000 7fafedc00640 1 -- start start 2024-03-09T09:11:50.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.098+0000 7fafedc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c6630 0x7fafe80c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:50.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafedc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fafe80c5f60 0x7fafe80c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:50.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafedc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe815a2a0 con 0x7fafe8153500 2024-03-09T09:11:50.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafedc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe8159d70 con 0x7fafe80c5f60 2024-03-09T09:11:50.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafedc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe806bb30 con 0x7fafe80c6630 2024-03-09T09:11:50.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafe6a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fafe8153500 0x7fafe806b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37324/0 (socket says 172.21.15.67:37324) 2024-03-09T09:11:50.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafe6a00640 1 -- 172.21.15.67:0/1098064923 learned_addr learned my addr 172.21.15.67:0/1098064923 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:11:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafe6000640 1 --2- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c6630 0x7fafe80c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafe7400640 1 --2- 172.21.15.67:0/1098064923 >> v2:172.21.15.134:3300/0 conn(0x7fafe80c5f60 0x7fafe80c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafe6000640 1 -- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c6630 msgr2=0x7fafe80c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:11:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafe6000640 1 -- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c6630 msgr2=0x7fafe80c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:11:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafe6000640 1 --2- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c6630 0x7fafe80c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:11:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.099+0000 7fafe6000640 1 --2- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c6630 0x7fafe80c74b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.100+0000 7fafe7400640 1 -- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c6630 msgr2=0x7fafe80c74b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:11:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.100+0000 7fafe7400640 1 --2- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c6630 0x7fafe80c74b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.100+0000 7fafe7400640 1 -- 172.21.15.67:0/1098064923 >> v1:172.21.15.67:6789/0 conn(0x7fafe8153500 legacy=0x7fafe806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.100+0000 7fafe7400640 1 -- 172.21.15.67:0/1098064923 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe8159fc0 con 0x7fafe80c5f60 2024-03-09T09:11:50.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.100+0000 7fafe5600640 1 -- 172.21.15.67:0/1098064923 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548040747 0 0) 0x7fafe815a2a0 con 0x7fafe8153500 2024-03-09T09:11:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.101+0000 7fafe7400640 1 --2- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe80c5f60 0x7fafe80c79f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009d90 tx=0x7fafe002bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b17e3c00e7372f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.101+0000 7fafe5600640 1 -- 172.21.15.67:0/1098064923 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0039070 con 0x7fafe80c5f60 2024-03-09T09:11:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.101+0000 7fafe5600640 1 -- 172.21.15.67:0/1098064923 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe002cb30 con 0x7fafe80c5f60 2024-03-09T09:11:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.101+0000 7fafe5600640 1 -- 172.21.15.67:0/1098064923 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0034440 con 0x7fafe80c5f60 2024-03-09T09:11:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.102+0000 7fafedc00640 1 -- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe80c5f60 msgr2=0x7fafe80c79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.102+0000 7fafedc00640 1 --2- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe80c5f60 0x7fafe80c79f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009d90 tx=0x7fafe002bd80 comp rx=0 tx=0).stop 2024-03-09T09:11:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.102+0000 7fafedc00640 1 -- 172.21.15.67:0/1098064923 shutdown_connections 2024-03-09T09:11:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.102+0000 7fafedc00640 1 --2- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c6630 0x7fafe80c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.102+0000 7fafedc00640 1 --2- 172.21.15.67:0/1098064923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe80c5f60 0x7fafe80c79f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.102+0000 7fafedc00640 1 -- 172.21.15.67:0/1098064923 >> 172.21.15.67:0/1098064923 conn(0x7fafe80c8060 msgr2=0x7fafe80c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.102+0000 7fafedc00640 1 -- 172.21.15.67:0/1098064923 shutdown_connections 2024-03-09T09:11:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.102+0000 7fafedc00640 1 -- 172.21.15.67:0/1098064923 wait complete. 2024-03-09T09:11:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.105+0000 7fafedc00640 1 Processor -- start 2024-03-09T09:11:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.105+0000 7fafedc00640 1 -- start start 2024-03-09T09:11:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.106+0000 7fafedc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c5f60 0x7fafe80c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.106+0000 7fafedc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe8153500 0x7fafe81ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.106+0000 7fafedc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafe81593b0 0x7fafe80c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.106+0000 7fafedc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe80bed90 con 0x7fafe81593b0 2024-03-09T09:11:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.106+0000 7fafedc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe80c6f90 con 0x7fafe8153500 2024-03-09T09:11:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.106+0000 7fafedc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe806b750 con 0x7fafe80c5f60 2024-03-09T09:11:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.106+0000 7fafe6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c5f60 0x7fafe80c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.106+0000 7fafe6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c5f60 0x7fafe80c5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46536/0 (socket says 172.21.15.67:46536) 2024-03-09T09:11:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.106+0000 7fafe6a00640 1 -- 172.21.15.67:0/1147631922 learned_addr learned my addr 172.21.15.67:0/1147631922 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.106+0000 7fafe6000640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe8153500 0x7fafe81ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.107+0000 7fafe7400640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafe81593b0 0x7fafe80c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.107+0000 7fafe6a00640 1 -- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c5f60 msgr2=0x7fafe80c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:11:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.107+0000 7fafe6a00640 1 -- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c5f60 msgr2=0x7fafe80c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:11:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.107+0000 7fafe6a00640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c5f60 0x7fafe80c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:11:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.107+0000 7fafe6a00640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c5f60 0x7fafe80c5210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.107+0000 7fafe6000640 1 -- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c5f60 msgr2=0x7fafe80c5210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:11:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.107+0000 7fafe6000640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c5f60 0x7fafe80c5210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.107+0000 7fafe6000640 1 -- 172.21.15.67:0/1147631922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafe81593b0 msgr2=0x7fafe80c33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.107+0000 7fafe6000640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafe81593b0 0x7fafe80c33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.107+0000 7fafe6000640 1 -- 172.21.15.67:0/1147631922 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe80b1150 con 0x7fafe8153500 2024-03-09T09:11:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.107+0000 7fafe6000640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe8153500 0x7fafe81ec5c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fafd400d5a0 tx=0x7fafd400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.108+0000 7fafdea00640 1 -- 172.21.15.67:0/1147631922 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40125f0 con 0x7fafe8153500 2024-03-09T09:11:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.108+0000 7fafdea00640 1 -- 172.21.15.67:0/1147631922 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd4019070 con 0x7fafe8153500 2024-03-09T09:11:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.108+0000 7fafdea00640 1 -- 172.21.15.67:0/1147631922 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40129d0 con 0x7fafe8153500 2024-03-09T09:11:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.108+0000 7fafedc00640 1 -- 172.21.15.67:0/1147631922 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe80b0e70 con 0x7fafe8153500 2024-03-09T09:11:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.108+0000 7fafedc00640 1 -- 172.21.15.67:0/1147631922 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe80c3900 con 0x7fafe8153500 2024-03-09T09:11:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.109+0000 7fafedc00640 1 -- 172.21.15.67:0/1147631922 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe80c2ee0 con 0x7fafe8153500 2024-03-09T09:11:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.110+0000 7fafdea00640 1 -- 172.21.15.67:0/1147631922 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fafd4012b70 con 0x7fafe8153500 2024-03-09T09:11:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.111+0000 7fafdea00640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafc806b4d0 0x7fafc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.111+0000 7fafdea00640 1 -- 172.21.15.67:0/1147631922 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fafd408a9c0 con 0x7fafe8153500 2024-03-09T09:11:50.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.112+0000 7fafe6a00640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafc806b4d0 0x7fafc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:50.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.113+0000 7fafe6a00640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafc806b4d0 0x7fafc806d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fafd8000f60 tx=0x7fafd8007cc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:50.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.118+0000 7fafdea00640 1 -- 172.21.15.67:0/1147631922 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fafe80c2ee0 con 0x7fafe8153500 2024-03-09T09:11:50.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.519+0000 7fafedc00640 1 -- 172.21.15.67:0/1147631922 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fafe8075a00 con 0x7fafe8153500 2024-03-09T09:11:50.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.519+0000 7fafdea00640 1 -- 172.21.15.67:0/1147631922 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fafd4059320 con 0x7fafe8153500 2024-03-09T09:11:50.524 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":140,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:11:50.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 -- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafc806b4d0 msgr2=0x7fafc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:50.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafc806b4d0 0x7fafc806d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fafd8000f60 tx=0x7fafd8007cc0 comp rx=0 tx=0).stop 2024-03-09T09:11:50.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 -- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe8153500 msgr2=0x7fafe81ec5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:50.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe8153500 0x7fafe81ec5c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fafd400d5a0 tx=0x7fafd400da60 comp rx=0 tx=0).stop 2024-03-09T09:11:50.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 -- 172.21.15.67:0/1147631922 shutdown_connections 2024-03-09T09:11:50.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafc806b4d0 0x7fafc806d990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafe81593b0 0x7fafe80c33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafe8153500 0x7fafe81ec5c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 --2- 172.21.15.67:0/1147631922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafe80c5f60 0x7fafe80c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 -- 172.21.15.67:0/1147631922 >> 172.21.15.67:0/1147631922 conn(0x7fafe80c8060 msgr2=0x7fafe806b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 -- 172.21.15.67:0/1147631922 shutdown_connections 2024-03-09T09:11:50.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:50.522+0000 7fafedc00640 1 -- 172.21.15.67:0/1147631922 wait complete. 2024-03-09T09:11:51.271 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 57% (142/246) Updating files: 58% (143/246) Updating files: 59% (146/246) Updating files: 60% (148/246) Updating files: 61% (151/246) Updating files: 62% (153/246) Updating files: 63% (155/246) Updating files: 64% (158/246) Updating files: 65% (160/246) Updating files: 66% (163/246) Updating files: 67% (165/246) Updating files: 68% (168/246) Updating files: 69% (170/246) Updating files: 70% (173/246) Updating files: 71% (175/246) Updating files: 72% (178/246) Updating files: 73% (180/246) Updating files: 74% (183/246) Updating files: 75% (185/246) Updating files: 76% (187/246) Updating files: 77% (190/246) Updating files: 78% (192/246) Updating files: 79% (195/246) Updating files: 80% (197/246) Updating files: 81% (200/246) Updating files: 82% (202/246) Updating files: 83% (205/246) Updating files: 84% (207/246) Updating files: 85% (210/246) Updating files: 86% (212/246) Updating files: 87% (215/246) Updating files: 88% (217/246) Updating files: 89% (219/246) Updating files: 90% (222/246) Updating files: 91% (224/246) Updating files: 92% (227/246) Updating files: 93% (229/246) Updating files: 94% (232/246) Updating files: 95% (234/246) Updating files: 96% (237/246) Updating files: 97% (239/246) Updating files: 98% (242/246) Updating files: 99% (244/246) Updating files: 100% (246/246) Updating files: 100% (246/246), done. 2024-03-09T09:11:51.349 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 1a32f0a0b42 v0.56 2024-03-09T09:11:51.361 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.56 2024-03-09T09:11:51.994 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:11:51.994 INFO:tasks.workunit.client.0.smithi067.stdout:v0.57 2024-03-09T09:11:51.995 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 57 -eq 48 ']' 2024-03-09T09:11:51.995 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.57 2024-03-09T09:11:51.995 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.57 2024-03-09T09:11:51.995 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.57 2024-03-09T09:11:53.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:11:53.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.760+0000 7f1ff4200640 1 Processor -- start 2024-03-09T09:11:53.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.760+0000 7f1ff4200640 1 -- start start 2024-03-09T09:11:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.761+0000 7f1ff4200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1fec0c5f50 0x7f1fec155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.761+0000 7f1ff4200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1fec06c650 0x7f1fec0c6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.761+0000 7f1ff4200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fec06b5b0 con 0x7f1fec06b040 2024-03-09T09:11:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.761+0000 7f1ff4200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fec0c9e00 con 0x7f1fec0c5f50 2024-03-09T09:11:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.761+0000 7f1ff4200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fec06cac0 con 0x7f1fec06c650 2024-03-09T09:11:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.762+0000 7f1ff1800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1fec0c5f50 0x7f1fec155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.762+0000 7f1ff1800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1fec0c5f50 0x7f1fec155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49286/0 (socket says 172.21.15.67:49286) 2024-03-09T09:11:53.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.762+0000 7f1ff1800640 1 -- 172.21.15.67:0/583782686 learned_addr learned my addr 172.21.15.67:0/583782686 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:53.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.762+0000 7f1ff2200640 1 --2- 172.21.15.67:0/583782686 >> v2:172.21.15.134:3300/0 conn(0x7f1fec06c650 0x7f1fec0c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:53.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.763+0000 7f1ff1800640 1 -- 172.21.15.67:0/583782686 >> v2:172.21.15.134:3300/0 conn(0x7f1fec06c650 msgr2=0x7f1fec0c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:53.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.763+0000 7f1ff1800640 1 --2- 172.21.15.67:0/583782686 >> v2:172.21.15.134:3300/0 conn(0x7f1fec06c650 0x7f1fec0c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:53.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.763+0000 7f1ff1800640 1 -- 172.21.15.67:0/583782686 >> v1:172.21.15.67:6789/0 conn(0x7f1fec06b040 legacy=0x7f1fec0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:53.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.763+0000 7f1ff1800640 1 -- 172.21.15.67:0/583782686 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec06b890 con 0x7f1fec0c5f50 2024-03-09T09:11:53.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.764+0000 7f1ff1800640 1 --2- 172.21.15.67:0/583782686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fec0c5f50 0x7f1fec155e20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0cb560 tx=0x7f1fe002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61504982b6d4b349 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:53.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.764+0000 7f1febe00640 1 -- 172.21.15.67:0/583782686 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0034070 con 0x7f1fec0c5f50 2024-03-09T09:11:53.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.764+0000 7f1febe00640 1 -- 172.21.15.67:0/583782686 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe0030860 con 0x7f1fec0c5f50 2024-03-09T09:11:53.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.764+0000 7f1febe00640 1 -- 172.21.15.67:0/583782686 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0030c30 con 0x7f1fec0c5f50 2024-03-09T09:11:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.765+0000 7f1ff4200640 1 -- 172.21.15.67:0/583782686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fec0c5f50 msgr2=0x7f1fec155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.765+0000 7f1ff4200640 1 --2- 172.21.15.67:0/583782686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fec0c5f50 0x7f1fec155e20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0cb560 tx=0x7f1fe002b9a0 comp rx=0 tx=0).stop 2024-03-09T09:11:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.765+0000 7f1ff4200640 1 -- 172.21.15.67:0/583782686 shutdown_connections 2024-03-09T09:11:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.765+0000 7f1ff4200640 1 --2- 172.21.15.67:0/583782686 >> v2:172.21.15.134:3300/0 conn(0x7f1fec06c650 0x7f1fec0c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.765+0000 7f1ff4200640 1 --2- 172.21.15.67:0/583782686 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fec0c5f50 0x7f1fec155e20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.765+0000 7f1ff4200640 1 -- 172.21.15.67:0/583782686 >> 172.21.15.67:0/583782686 conn(0x7f1fec0c8050 msgr2=0x7f1fec0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.765+0000 7f1ff4200640 1 -- 172.21.15.67:0/583782686 shutdown_connections 2024-03-09T09:11:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.765+0000 7f1ff4200640 1 -- 172.21.15.67:0/583782686 wait complete. 2024-03-09T09:11:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.767+0000 7f1ff4200640 1 Processor -- start 2024-03-09T09:11:53.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.767+0000 7f1ff4200640 1 -- start start 2024-03-09T09:11:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.768+0000 7f1ff4200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fec06b040 0x7f1fec1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:53.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.768+0000 7f1ff4200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fec06c650 0x7f1fec14b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.768+0000 7f1ff4200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fec0c5f50 0x7f1fec1ef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.768+0000 7f1ff4200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec06c130 con 0x7f1fec0c5f50 2024-03-09T09:11:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.768+0000 7f1ff4200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec06be50 con 0x7f1fec06c650 2024-03-09T09:11:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.768+0000 7f1ff4200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec14cc00 con 0x7f1fec06b040 2024-03-09T09:11:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.769+0000 7f1ff2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fec0c5f50 0x7f1fec1ef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.769+0000 7f1ff2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fec0c5f50 0x7f1fec1ef0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49298/0 (socket says 172.21.15.67:49298) 2024-03-09T09:11:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.769+0000 7f1ff2200640 1 -- 172.21.15.67:0/1726021442 learned_addr learned my addr 172.21.15.67:0/1726021442 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:53.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.769+0000 7f1ff1800640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fec06b040 0x7f1fec1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:53.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.769+0000 7f1ff2200640 1 -- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fec06b040 msgr2=0x7f1fec1571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:53.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.769+0000 7f1ff2200640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fec06b040 0x7f1fec1571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:53.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.769+0000 7f1ff2200640 1 -- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fec06c650 msgr2=0x7f1fec14b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:53.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.769+0000 7f1ff2200640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fec06c650 0x7f1fec14b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:53.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.769+0000 7f1ff2200640 1 -- 172.21.15.67:0/1726021442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec1ef930 con 0x7f1fec0c5f50 2024-03-09T09:11:53.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.770+0000 7f1ff2200640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fec0c5f50 0x7f1fec1ef0f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4007d90 tx=0x7f1fe400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:53.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.770+0000 7f1fe9600640 1 -- 172.21.15.67:0/1726021442 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4014420 con 0x7f1fec0c5f50 2024-03-09T09:11:53.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.770+0000 7f1fe9600640 1 -- 172.21.15.67:0/1726021442 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe4019070 con 0x7f1fec0c5f50 2024-03-09T09:11:53.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.770+0000 7f1ff4200640 1 -- 172.21.15.67:0/1726021442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fec1efc10 con 0x7f1fec0c5f50 2024-03-09T09:11:53.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.770+0000 7f1fe9600640 1 -- 172.21.15.67:0/1726021442 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4014820 con 0x7f1fec0c5f50 2024-03-09T09:11:53.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.770+0000 7f1ff4200640 1 -- 172.21.15.67:0/1726021442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fec1ef630 con 0x7f1fec0c5f50 2024-03-09T09:11:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.772+0000 7f1ff4200640 1 -- 172.21.15.67:0/1726021442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fec075a00 con 0x7f1fec0c5f50 2024-03-09T09:11:53.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.773+0000 7f1fe9600640 1 -- 172.21.15.67:0/1726021442 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1fe40149c0 con 0x7f1fec0c5f50 2024-03-09T09:11:53.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.774+0000 7f1fe9600640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1fcc06b4d0 0x7f1fcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:53.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.774+0000 7f1fe9600640 1 -- 172.21.15.67:0/1726021442 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1fe408ae60 con 0x7f1fec0c5f50 2024-03-09T09:11:53.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.775+0000 7f1ff1800640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1fcc06b4d0 0x7f1fcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:53.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.776+0000 7f1ff1800640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1fcc06b4d0 0x7f1fcc06d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0036000 tx=0x7f1fe003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:53.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:53.778+0000 7f1fe9600640 1 -- 172.21.15.67:0/1726021442 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1fe40598e0 con 0x7f1fec0c5f50 2024-03-09T09:11:54.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.189+0000 7f1ff4200640 1 -- 172.21.15.67:0/1726021442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1fec06cca0 con 0x7f1fec0c5f50 2024-03-09T09:11:54.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.191+0000 7f1fe9600640 1 -- 172.21.15.67:0/1726021442 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f1fe40591e0 con 0x7f1fec0c5f50 2024-03-09T09:11:54.196 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":142,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:11:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 -- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1fcc06b4d0 msgr2=0x7f1fcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1fcc06b4d0 0x7f1fcc06d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0036000 tx=0x7f1fe003b040 comp rx=0 tx=0).stop 2024-03-09T09:11:54.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 -- 172.21.15.67:0/1726021442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fec0c5f50 msgr2=0x7f1fec1ef0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fec0c5f50 0x7f1fec1ef0f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4007d90 tx=0x7f1fe400e4b0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 -- 172.21.15.67:0/1726021442 shutdown_connections 2024-03-09T09:11:54.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1fcc06b4d0 0x7f1fcc06d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fec0c5f50 0x7f1fec1ef0f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fec06c650 0x7f1fec14b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 --2- 172.21.15.67:0/1726021442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fec06b040 0x7f1fec1571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 -- 172.21.15.67:0/1726021442 >> 172.21.15.67:0/1726021442 conn(0x7f1fec0c8050 msgr2=0x7f1fec0c4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:54.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 -- 172.21.15.67:0/1726021442 shutdown_connections 2024-03-09T09:11:54.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.193+0000 7f1ff4200640 1 -- 172.21.15.67:0/1726021442 wait complete. 2024-03-09T09:11:54.215 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:11:54.215 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:11:54.215 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:11:54.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.436+0000 7f0c63e00640 1 Processor -- start 2024-03-09T09:11:54.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.436+0000 7f0c63e00640 1 -- start start 2024-03-09T09:11:54.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.437+0000 7f0c63e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0c6406e830 0x7f0c640c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.437+0000 7f0c63e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c641536c0 0x7f0c640c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.437+0000 7f0c63e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c640c19e0 con 0x7f0c640c1030 2024-03-09T09:11:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.437+0000 7f0c63e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6406ef60 con 0x7f0c6406e830 2024-03-09T09:11:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.437+0000 7f0c63e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c64156790 con 0x7f0c641536c0 2024-03-09T09:11:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.438+0000 7f0c62000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0c6406e830 0x7f0c640c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.438+0000 7f0c62000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0c6406e830 0x7f0c640c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49314/0 (socket says 172.21.15.67:49314) 2024-03-09T09:11:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.438+0000 7f0c62000640 1 -- 172.21.15.67:0/164741104 learned_addr learned my addr 172.21.15.67:0/164741104 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.438+0000 7f0c63400640 1 --2- 172.21.15.67:0/164741104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c641536c0 0x7f0c640c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.438+0000 7f0c62000640 1 -- 172.21.15.67:0/164741104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c641536c0 msgr2=0x7f0c640c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.438+0000 7f0c62000640 1 --2- 172.21.15.67:0/164741104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c641536c0 0x7f0c640c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.438+0000 7f0c62000640 1 -- 172.21.15.67:0/164741104 >> v1:172.21.15.134:6789/0 conn(0x7f0c640c1030 legacy=0x7f0c64152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.438+0000 7f0c62000640 1 -- 172.21.15.67:0/164741104 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c641569e0 con 0x7f0c6406e830 2024-03-09T09:11:54.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.439+0000 7f0c62000640 1 --2- 172.21.15.67:0/164741104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c6406e830 0x7f0c640c2910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009870 tx=0x7f0c5002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bf82466b07175b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:54.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.440+0000 7f0c61600640 1 -- 172.21.15.67:0/164741104 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c500317c0 con 0x7f0c6406e830 2024-03-09T09:11:54.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.440+0000 7f0c61600640 1 -- 172.21.15.67:0/164741104 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c50037070 con 0x7f0c6406e830 2024-03-09T09:11:54.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.440+0000 7f0c61600640 1 -- 172.21.15.67:0/164741104 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50031b80 con 0x7f0c6406e830 2024-03-09T09:11:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.440+0000 7f0c63e00640 1 -- 172.21.15.67:0/164741104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c6406e830 msgr2=0x7f0c640c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.440+0000 7f0c63e00640 1 --2- 172.21.15.67:0/164741104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c6406e830 0x7f0c640c2910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009870 tx=0x7f0c5002b860 comp rx=0 tx=0).stop 2024-03-09T09:11:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.440+0000 7f0c63e00640 1 -- 172.21.15.67:0/164741104 shutdown_connections 2024-03-09T09:11:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.440+0000 7f0c63e00640 1 --2- 172.21.15.67:0/164741104 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c641536c0 0x7f0c640c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.440+0000 7f0c63e00640 1 --2- 172.21.15.67:0/164741104 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c6406e830 0x7f0c640c2910 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.440+0000 7f0c63e00640 1 -- 172.21.15.67:0/164741104 >> 172.21.15.67:0/164741104 conn(0x7f0c640a9740 msgr2=0x7f0c640a9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:54.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.441+0000 7f0c63e00640 1 -- 172.21.15.67:0/164741104 shutdown_connections 2024-03-09T09:11:54.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.441+0000 7f0c63e00640 1 -- 172.21.15.67:0/164741104 wait complete. 2024-03-09T09:11:54.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.442+0000 7f0c63e00640 1 Processor -- start 2024-03-09T09:11:54.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.443+0000 7f0c63e00640 1 -- start start 2024-03-09T09:11:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.444+0000 7f0c63e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c640c1030 0x7f0c640aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.444+0000 7f0c63e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641536c0 0x7f0c64151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.444+0000 7f0c63e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c64151ac0 0x7f0c641eec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.444+0000 7f0c63e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c640c1e20 con 0x7f0c641536c0 2024-03-09T09:11:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.444+0000 7f0c63e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c64156cc0 con 0x7f0c64151ac0 2024-03-09T09:11:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.444+0000 7f0c63e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c640c1fa0 con 0x7f0c640c1030 2024-03-09T09:11:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.445+0000 7f0c62000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641536c0 0x7f0c64151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.445+0000 7f0c62000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641536c0 0x7f0c64151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49326/0 (socket says 172.21.15.67:49326) 2024-03-09T09:11:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.445+0000 7f0c62000640 1 -- 172.21.15.67:0/2456326777 learned_addr learned my addr 172.21.15.67:0/2456326777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.445+0000 7f0c63400640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c64151ac0 0x7f0c641eec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.445+0000 7f0c62a00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c640c1030 0x7f0c640aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.445+0000 7f0c62000640 1 -- 172.21.15.67:0/2456326777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c640c1030 msgr2=0x7f0c640aaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.445+0000 7f0c62000640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c640c1030 0x7f0c640aaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.445+0000 7f0c62000640 1 -- 172.21.15.67:0/2456326777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c64151ac0 msgr2=0x7f0c641eec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.445+0000 7f0c62000640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c64151ac0 0x7f0c641eec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.445+0000 7f0c62000640 1 -- 172.21.15.67:0/2456326777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c64150ae0 con 0x7f0c641536c0 2024-03-09T09:11:54.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.446+0000 7f0c62000640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641536c0 0x7f0c64151580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0c50000c00 tx=0x7f0c500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:54.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.446+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50032660 con 0x7f0c641536c0 2024-03-09T09:11:54.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.446+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c50032800 con 0x7f0c641536c0 2024-03-09T09:11:54.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.446+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50032bf0 con 0x7f0c641536c0 2024-03-09T09:11:54.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.446+0000 7f0c63e00640 1 -- 172.21.15.67:0/2456326777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c641ec2f0 con 0x7f0c641536c0 2024-03-09T09:11:54.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.446+0000 7f0c63e00640 1 -- 172.21.15.67:0/2456326777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c641ec5d0 con 0x7f0c641536c0 2024-03-09T09:11:54.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.447+0000 7f0c63e00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c6406e830 0x7f0c64151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.447+0000 7f0c63e00640 1 -- 172.21.15.67:0/2456326777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6404f3c0 con 0x7f0c6406e830 2024-03-09T09:11:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.448+0000 7f0c62a00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c6406e830 0x7f0c64151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.448+0000 7f0c62a00640 1 -- 172.21.15.67:0/2456326777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c640c5d60 con 0x7f0c6406e830 2024-03-09T09:11:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.449+0000 7f0c62a00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c6406e830 0x7f0c64151ed0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0c640aa1a0 tx=0x7f0c5400e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.449+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54019070 con 0x7f0c6406e830 2024-03-09T09:11:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.449+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f0c540052e0 con 0x7f0c6406e830 2024-03-09T09:11:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.449+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c6406e830 msgr2=0x7f0c64151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.449+0000 7f0c5aa00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c6406e830 0x7f0c64151ed0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0c640aa1a0 tx=0x7f0c5400e8b0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.449+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c6406e830 msgr2=0x7f0c64151ed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.449+0000 7f0c5aa00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c6406e830 0x7f0c64151ed0 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0c640aa1a0 tx=0x7f0c5400e8b0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.449+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0c5004c060 con 0x7f0c641536c0 2024-03-09T09:11:54.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.451+0000 7f0c5aa00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c4406aef0 0x7f0c4406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.451+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0c500ac180 con 0x7f0c641536c0 2024-03-09T09:11:54.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.451+0000 7f0c62a00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c4406aef0 0x7f0c4406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.453+0000 7f0c62a00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c4406aef0 0x7f0c4406d3b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400ee00 tx=0x7f0c54005000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:54.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.463+0000 7f0c63e00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641ef180 0x7f0c641ef560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.463+0000 7f0c63e00640 1 -- 172.21.15.67:0/2456326777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c641efaa0 con 0x7f0c641ef180 2024-03-09T09:11:54.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.464+0000 7f0c63400640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641ef180 0x7f0c641ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.464+0000 7f0c63400640 1 -- 172.21.15.67:0/2456326777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0c640ab4c0 con 0x7f0c641ef180 2024-03-09T09:11:54.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.465+0000 7f0c63400640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641ef180 0x7f0c641ef560 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c004610 tx=0x7f0c5c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:54.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.465+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c015420 con 0x7f0c641ef180 2024-03-09T09:11:54.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.465+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f0c5c0155c0 con 0x7f0c641ef180 2024-03-09T09:11:54.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.465+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641ef180 msgr2=0x7f0c641ef560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.465+0000 7f0c5aa00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641ef180 0x7f0c641ef560 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c004610 tx=0x7f0c5c007700 comp rx=0 tx=0).stop 2024-03-09T09:11:54.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.465+0000 7f0c5aa00640 1 -- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641ef180 msgr2=0x7f0c641ef560 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:54.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.465+0000 7f0c5aa00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641ef180 0x7f0c641ef560 secure :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c004610 tx=0x7f0c5c007700 comp rx=0 tx=0).stop 2024-03-09T09:11:54.471 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:11:54.471 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:54.471 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:54.471 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:11:54.471 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 142, 2024-03-09T09:11:54.471 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:11:54.471 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:11:54.471 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:11:54.471 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:11:54.471 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:54.472 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:54.473 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:54.474 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:54.474 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:54.474 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.474 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:11:54.474 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:11:54.474 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:11:54.474 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:11:54.474 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:11:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.476 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:11:54.477 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:54.478 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:54.479 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:54.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:54.479 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:11:54.479 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:54.479 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:54.480 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:54.482 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:54.483 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.484 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:54.485 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:11:54.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.485 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:54.485 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:54.485 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:11:54.485 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.485 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.486 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:11:54.486 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:11:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.466+0000 7f0c63e00640 1 -- 172.21.15.67:0/2456326777 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c4406aef0 msgr2=0x7f0c4406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.466+0000 7f0c63e00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c4406aef0 0x7f0c4406d3b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400ee00 tx=0x7f0c54005000 comp rx=0 tx=0).stop 2024-03-09T09:11:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.466+0000 7f0c63e00640 1 -- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641536c0 msgr2=0x7f0c64151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.466+0000 7f0c63e00640 1 --2- 172.21.15.67:0/2456326777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c641536c0 0x7f0c64151580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0c50000c00 tx=0x7f0c500046c0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.466+0000 7f0c63400640 1 -- 172.21.15.67:0/2456326777 reap_dead start 2024-03-09T09:11:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.466+0000 7f0c63e00640 1 -- 172.21.15.67:0/2456326777 shutdown_connections 2024-03-09T09:11:54.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.466+0000 7f0c63e00640 1 -- 172.21.15.67:0/2456326777 >> 172.21.15.67:0/2456326777 conn(0x7f0c640a9740 msgr2=0x7f0c6414eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:54.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.466+0000 7f0c63e00640 1 -- 172.21.15.67:0/2456326777 shutdown_connections 2024-03-09T09:11:54.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.466+0000 7f0c63e00640 1 -- 172.21.15.67:0/2456326777 wait complete. 2024-03-09T09:11:54.489 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:11:54.489 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:11:54.489 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:11:54.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.684+0000 7fafbe400640 1 Processor -- start 2024-03-09T09:11:54.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.684+0000 7fafbe400640 1 -- start start 2024-03-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.686+0000 7fafbe400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fafb81532c0 0x7fafb806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.686+0000 7fafbe400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafb80703f0 0x7fafb80c4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.686+0000 7fafbe400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fafb80c2ef0 0x7fafb80c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.686+0000 7fafbe400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb80c3b40 con 0x7fafb80c2ef0 2024-03-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.686+0000 7fafbe400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb8073230 con 0x7fafb81532c0 2024-03-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.686+0000 7fafbe400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb8072900 con 0x7fafb80703f0 2024-03-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.686+0000 7fafbda00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fafb80c2ef0 0x7fafb80c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.686+0000 7fafbda00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fafb80c2ef0 0x7fafb80c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49350/0 (socket says 172.21.15.67:49350) 2024-03-09T09:11:54.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.686+0000 7fafbda00640 1 -- 172.21.15.67:0/2194400080 learned_addr learned my addr 172.21.15.67:0/2194400080 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:54.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.686+0000 7fafb7e00640 1 --2- 172.21.15.67:0/2194400080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafb80703f0 0x7fafb80c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.686+0000 7fafbd000640 1 --2- 172.21.15.67:0/2194400080 >> v2:172.21.15.134:3300/0 conn(0x7fafb81532c0 0x7fafb806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.687+0000 7fafbda00640 1 -- 172.21.15.67:0/2194400080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafb80703f0 msgr2=0x7fafb80c4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.687+0000 7fafbda00640 1 --2- 172.21.15.67:0/2194400080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafb80703f0 0x7fafb80c4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.687+0000 7fafbda00640 1 -- 172.21.15.67:0/2194400080 >> v2:172.21.15.134:3300/0 conn(0x7fafb81532c0 msgr2=0x7fafb806feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.687+0000 7fafbda00640 1 --2- 172.21.15.67:0/2194400080 >> v2:172.21.15.134:3300/0 conn(0x7fafb81532c0 0x7fafb806feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.687+0000 7fafbda00640 1 -- 172.21.15.67:0/2194400080 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb8072be0 con 0x7fafb80c2ef0 2024-03-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.688+0000 7fafbda00640 1 --2- 172.21.15.67:0/2194400080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb80c2ef0 0x7fafb80c4e50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fafb80ac250 tx=0x7fafb002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c51c7a848d0b7f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.688+0000 7fafb7400640 1 -- 172.21.15.67:0/2194400080 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb002c9e0 con 0x7fafb80c2ef0 2024-03-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.688+0000 7fafb7400640 1 -- 172.21.15.67:0/2194400080 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb002cb80 con 0x7fafb80c2ef0 2024-03-09T09:11:54.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.688+0000 7fafb7400640 1 -- 172.21.15.67:0/2194400080 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb00343f0 con 0x7fafb80c2ef0 2024-03-09T09:11:54.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.689+0000 7fafbe400640 1 -- 172.21.15.67:0/2194400080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb80c2ef0 msgr2=0x7fafb80c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.689+0000 7fafbe400640 1 --2- 172.21.15.67:0/2194400080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb80c2ef0 0x7fafb80c4e50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fafb80ac250 tx=0x7fafb002be40 comp rx=0 tx=0).stop 2024-03-09T09:11:54.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.689+0000 7fafbe400640 1 -- 172.21.15.67:0/2194400080 shutdown_connections 2024-03-09T09:11:54.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.689+0000 7fafbe400640 1 --2- 172.21.15.67:0/2194400080 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb80c2ef0 0x7fafb80c4e50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.689+0000 7fafbe400640 1 --2- 172.21.15.67:0/2194400080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafb80703f0 0x7fafb80c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.689+0000 7fafbe400640 1 --2- 172.21.15.67:0/2194400080 >> v2:172.21.15.134:3300/0 conn(0x7fafb81532c0 0x7fafb806feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.689+0000 7fafbe400640 1 -- 172.21.15.67:0/2194400080 >> 172.21.15.67:0/2194400080 conn(0x7fafb8066d90 msgr2=0x7fafb8067170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:54.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.689+0000 7fafbe400640 1 -- 172.21.15.67:0/2194400080 shutdown_connections 2024-03-09T09:11:54.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.689+0000 7fafbe400640 1 -- 172.21.15.67:0/2194400080 wait complete. 2024-03-09T09:11:54.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.691+0000 7fafbe400640 1 Processor -- start 2024-03-09T09:11:54.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.691+0000 7fafbe400640 1 -- start start 2024-03-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.692+0000 7fafbe400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafb80703f0 0x7fafb81558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.692+0000 7fafbe400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb80c2ef0 0x7fafb806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.692+0000 7fafbe400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb81532c0 0x7fafb81562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.692+0000 7fafbe400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80c5390 con 0x7fafb80c2ef0 2024-03-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.692+0000 7fafbe400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80c2a30 con 0x7fafb81532c0 2024-03-09T09:11:54.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.692+0000 7fafbe400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb806d810 con 0x7fafb80703f0 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.692+0000 7fafb7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb80c2ef0 0x7fafb806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.692+0000 7fafb7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb80c2ef0 0x7fafb806d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49356/0 (socket says 172.21.15.67:49356) 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.692+0000 7fafb7e00640 1 -- 172.21.15.67:0/4068906376 learned_addr learned my addr 172.21.15.67:0/4068906376 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.692+0000 7fafbd000640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafb80703f0 0x7fafb81558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.693+0000 7fafbda00640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb81532c0 0x7fafb81562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.693+0000 7fafb7e00640 1 -- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafb80703f0 msgr2=0x7fafb81558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.693+0000 7fafb7e00640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafb80703f0 0x7fafb81558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.693+0000 7fafb7e00640 1 -- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb81532c0 msgr2=0x7fafb81562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.693+0000 7fafb7e00640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb81532c0 0x7fafb81562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.693+0000 7fafb7e00640 1 -- 172.21.15.67:0/4068906376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb8154e50 con 0x7fafb80c2ef0 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.693+0000 7fafb7e00640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb80c2ef0 0x7fafb806d2d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fafa400d5f0 tx=0x7fafa400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.693+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40125f0 con 0x7fafb80c2ef0 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.693+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa4019070 con 0x7fafb80c2ef0 2024-03-09T09:11:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.693+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40129d0 con 0x7fafb80c2ef0 2024-03-09T09:11:54.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.693+0000 7fafbe400640 1 -- 172.21.15.67:0/4068906376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb806da40 con 0x7fafb80c2ef0 2024-03-09T09:11:54.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.694+0000 7fafbe400640 1 -- 172.21.15.67:0/4068906376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb81550d0 con 0x7fafb80c2ef0 2024-03-09T09:11:54.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.694+0000 7fafbe400640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb806bb00 0x7fafb81eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.694+0000 7fafbe400640 1 -- 172.21.15.67:0/4068906376 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80c5510 con 0x7fafb806bb00 2024-03-09T09:11:54.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.695+0000 7fafbd000640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb806bb00 0x7fafb81eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.696+0000 7fafbd000640 1 -- 172.21.15.67:0/4068906376 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafb8075a00 con 0x7fafb806bb00 2024-03-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.696+0000 7fafbd000640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb806bb00 0x7fafb81eed30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fafb8152060 tx=0x7fafac00f910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.697+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac019020 con 0x7fafb806bb00 2024-03-09T09:11:54.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.697+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fafa4012b70 con 0x7fafb80c2ef0 2024-03-09T09:11:54.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.698+0000 7fafb4c00640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf9806b3d0 0x7faf9806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.698+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fafa408a950 con 0x7fafb80c2ef0 2024-03-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.698+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fafac014870 con 0x7fafb806bb00 2024-03-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.698+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb806bb00 msgr2=0x7fafb81eed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.698+0000 7fafb4c00640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb806bb00 0x7fafb81eed30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fafb8152060 tx=0x7fafac00f910 comp rx=0 tx=0).stop 2024-03-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.698+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb806bb00 msgr2=0x7fafb81eed30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.698+0000 7fafb4c00640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb806bb00 0x7fafb81eed30 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fafb8152060 tx=0x7fafac00f910 comp rx=0 tx=0).stop 2024-03-09T09:11:54.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.698+0000 7fafbda00640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf9806b3d0 0x7faf9806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.699+0000 7fafbda00640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf9806b3d0 0x7faf9806d890 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fafb0035000 tx=0x7fafb0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:54.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.713+0000 7fafbe400640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb81ef270 0x7fafb80c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:54.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.713+0000 7fafbe400640 1 -- 172.21.15.67:0/4068906376 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb804f3c0 con 0x7fafb81ef270 2024-03-09T09:11:54.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.713+0000 7fafbd000640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb81ef270 0x7fafb80c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:54.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.714+0000 7fafbd000640 1 -- 172.21.15.67:0/4068906376 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fafb806fc90 con 0x7fafb81ef270 2024-03-09T09:11:54.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.715+0000 7fafbd000640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb81ef270 0x7fafb80c8090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fafb8152060 tx=0x7fafac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:54.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.715+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac020240 con 0x7fafb81ef270 2024-03-09T09:11:54.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.716+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fafac020220 con 0x7fafb81ef270 2024-03-09T09:11:54.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.716+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb81ef270 msgr2=0x7fafb80c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.716+0000 7fafb4c00640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb81ef270 0x7fafb80c8090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fafb8152060 tx=0x7fafac0046c0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.716+0000 7fafb4c00640 1 -- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb81ef270 msgr2=0x7fafb80c8090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:54.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.716+0000 7fafb4c00640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafb81ef270 0x7fafb80c8090 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fafb8152060 tx=0x7fafac0046c0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.721 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:11:54.721 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:54.721 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 142, 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:54.722 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:54.723 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:54.724 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:54.724 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:54.724 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:54.724 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:54.724 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:54.724 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:54.724 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:54.724 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:11:54.725 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:11:54.726 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:11:54.726 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:11:54.726 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:11:54.726 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:11:54.726 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:54.726 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:11:54.726 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:11:54.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.728 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:11:54.728 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.728 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:54.728 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:54.728 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:54.728 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:54.728 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.728 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:54.728 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:54.729 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:54.731 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:11:54.732 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:54.733 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:11:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:54.734 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:11:54.734 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:11:54.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.717+0000 7fafbe400640 1 -- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf9806b3d0 msgr2=0x7faf9806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.717+0000 7fafbe400640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf9806b3d0 0x7faf9806d890 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fafb0035000 tx=0x7fafb0030000 comp rx=0 tx=0).stop 2024-03-09T09:11:54.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.717+0000 7fafbe400640 1 -- 172.21.15.67:0/4068906376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb80c2ef0 msgr2=0x7fafb806d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:54.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.717+0000 7fafbe400640 1 --2- 172.21.15.67:0/4068906376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb80c2ef0 0x7fafb806d2d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fafa400d5f0 tx=0x7fafa400dab0 comp rx=0 tx=0).stop 2024-03-09T09:11:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.717+0000 7fafbda00640 1 -- 172.21.15.67:0/4068906376 reap_dead start 2024-03-09T09:11:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.717+0000 7fafbda00640 1 -- 172.21.15.67:0/4068906376 reap_dead start 2024-03-09T09:11:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.717+0000 7fafbe400640 1 -- 172.21.15.67:0/4068906376 shutdown_connections 2024-03-09T09:11:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.717+0000 7fafbe400640 1 -- 172.21.15.67:0/4068906376 >> 172.21.15.67:0/4068906376 conn(0x7fafb8066d90 msgr2=0x7fafb814dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.717+0000 7fafbe400640 1 -- 172.21.15.67:0/4068906376 shutdown_connections 2024-03-09T09:11:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:54.717+0000 7fafbe400640 1 -- 172.21.15.67:0/4068906376 wait complete. 2024-03-09T09:11:54.745 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:11:54.745 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:11:54.746 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:11:55.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.036+0000 7f8777e00640 1 Processor -- start 2024-03-09T09:11:55.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.036+0000 7f8777e00640 1 -- start start 2024-03-09T09:11:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.037+0000 7f8777e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f87780c68a0 0x7f8778072810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.037+0000 7f8777e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8778072d50 con 0x7f87780c64c0 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.037+0000 7f8777e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87780aa640 con 0x7f87780c2e80 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.037+0000 7f8777e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877806f600 con 0x7f87780c68a0 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.038+0000 7f8776a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f87780c64c0 0x7f8778070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37332/0 (socket says 172.21.15.67:37332) 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.038+0000 7f8776a00640 1 -- 172.21.15.67:0/2648176597 learned_addr learned my addr 172.21.15.67:0/2648176597 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.038+0000 7f8776000640 1 --2- 172.21.15.67:0/2648176597 >> v2:172.21.15.134:3300/0 conn(0x7f87780c68a0 0x7f8778072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.038+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794165981 0 0) 0x7f8778072d50 con 0x7f87780c64c0 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.038+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f875c003520 con 0x7f87780c64c0 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 490889726 0 0) 0x7f875c003520 con 0x7f87780c64c0 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 >> v2:172.21.15.134:3300/0 conn(0x7f87780c68a0 msgr2=0x7f8778072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 --2- 172.21.15.67:0/2648176597 >> v2:172.21.15.134:3300/0 conn(0x7f87780c68a0 0x7f8778072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 >> v1:172.21.15.134:6789/0 conn(0x7f87780c2e80 legacy=0x7f8778155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87780aa920 con 0x7f87780c64c0 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f8768001670 con 0x7f87780c64c0 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 >> v1:172.21.15.67:6789/0 conn(0x7f87780c64c0 legacy=0x7f8778070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 --2- 172.21.15.67:0/2648176597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f875c003a20 0x7f875c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 --2- 172.21.15.67:0/2648176597 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f875c004800 0x7f875c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 --2- 172.21.15.67:0/2648176597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f875c005230 0x7f875c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877806f600 con 0x7f875c003a20 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875c00c190 con 0x7f875c005230 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c00c390 con 0x7f875c004800 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8776a00640 1 --2- 172.21.15.67:0/2648176597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f875c003a20 0x7f875c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.039+0000 7f8776000640 1 --2- 172.21.15.67:0/2648176597 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f875c004800 0x7f875c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.040+0000 7f8776a00640 1 -- 172.21.15.67:0/2648176597 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f875c004800 msgr2=0x7f875c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:55.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.040+0000 7f8776a00640 1 --2- 172.21.15.67:0/2648176597 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f875c004800 0x7f875c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.040+0000 7f8776a00640 1 -- 172.21.15.67:0/2648176597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f875c005230 msgr2=0x7f875c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:55.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.040+0000 7f8776a00640 1 --2- 172.21.15.67:0/2648176597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f875c005230 0x7f875c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.040+0000 7f8776a00640 1 -- 172.21.15.67:0/2648176597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c003520 con 0x7f875c003a20 2024-03-09T09:11:55.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.040+0000 7f8777400640 1 -- 172.21.15.67:0/2648176597 reap_dead start 2024-03-09T09:11:55.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.041+0000 7f8776a00640 1 --2- 172.21.15.67:0/2648176597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f875c003a20 0x7f875c003e40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f876800c240 tx=0x7f875c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f7382f0f3a0aa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:55.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.041+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768007d10 con 0x7f875c003a20 2024-03-09T09:11:55.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.041+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8768030070 con 0x7f875c003a20 2024-03-09T09:11:55.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.041+0000 7f8775600640 1 -- 172.21.15.67:0/2648176597 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768005d70 con 0x7f875c003a20 2024-03-09T09:11:55.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.041+0000 7f8777e00640 1 -- 172.21.15.67:0/2648176597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f875c003a20 msgr2=0x7f875c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.041+0000 7f8777e00640 1 --2- 172.21.15.67:0/2648176597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f875c003a20 0x7f875c003e40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f876800c240 tx=0x7f875c005830 comp rx=0 tx=0).stop 2024-03-09T09:11:55.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.042+0000 7f8777e00640 1 -- 172.21.15.67:0/2648176597 shutdown_connections 2024-03-09T09:11:55.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.042+0000 7f8777e00640 1 --2- 172.21.15.67:0/2648176597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f875c003a20 0x7f875c003e40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.042+0000 7f8777e00640 1 -- 172.21.15.67:0/2648176597 >> 172.21.15.67:0/2648176597 conn(0x7f877806f060 msgr2=0x7f8778066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:55.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.042+0000 7f8777e00640 1 -- 172.21.15.67:0/2648176597 shutdown_connections 2024-03-09T09:11:55.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.042+0000 7f8777e00640 1 -- 172.21.15.67:0/2648176597 wait complete. 2024-03-09T09:11:55.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.043+0000 7f8777e00640 1 Processor -- start 2024-03-09T09:11:55.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.044+0000 7f8777e00640 1 -- start start 2024-03-09T09:11:55.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.044+0000 7f8777e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f875c003a20 0x7f87780c2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.044+0000 7f8777e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87780c49b0 0x7f87780c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.044+0000 7f8777e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8778133a70 0x7f8778133ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.044+0000 7f8777e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8778072810 con 0x7f87780c49b0 2024-03-09T09:11:55.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.044+0000 7f8777e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877806fb30 con 0x7f8778133a70 2024-03-09T09:11:55.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.044+0000 7f8777e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8778072990 con 0x7f875c003a20 2024-03-09T09:11:55.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.045+0000 7f8776000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87780c49b0 0x7f87780c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.045+0000 7f8776000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87780c49b0 0x7f87780c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49376/0 (socket says 172.21.15.67:49376) 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.045+0000 7f8776000640 1 -- 172.21.15.67:0/2779533226 learned_addr learned my addr 172.21.15.67:0/2779533226 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.045+0000 7f8776a00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f875c003a20 0x7f87780c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.045+0000 7f8777400640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8778133a70 0x7f8778133ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.045+0000 7f8776000640 1 -- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f875c003a20 msgr2=0x7f87780c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.045+0000 7f8776000640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f875c003a20 0x7f87780c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.045+0000 7f8776000640 1 -- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8778133a70 msgr2=0x7f8778133ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.045+0000 7f8776000640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8778133a70 0x7f8778133ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.045+0000 7f8776000640 1 -- 172.21.15.67:0/2779533226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8778134430 con 0x7f87780c49b0 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.046+0000 7f8776000640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87780c49b0 0x7f87780c4e10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f87640107c0 tx=0x7f8764010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.046+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876400f040 con 0x7f87780c49b0 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.046+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876400eb00 con 0x7f87780c49b0 2024-03-09T09:11:55.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.046+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764010e70 con 0x7f87780c49b0 2024-03-09T09:11:55.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.046+0000 7f8777e00640 1 -- 172.21.15.67:0/2779533226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87780be4d0 con 0x7f87780c49b0 2024-03-09T09:11:55.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.046+0000 7f8777e00640 1 -- 172.21.15.67:0/2779533226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87781f3300 con 0x7f87780c49b0 2024-03-09T09:11:55.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.047+0000 7f8777e00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 0x7f87781f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.047+0000 7f8777e00640 1 -- 172.21.15.67:0/2779533226 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877804f3c0 con 0x7f87781f34d0 2024-03-09T09:11:55.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.048+0000 7f8776a00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 0x7f87781f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.048+0000 7f8776a00640 1 -- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 msgr2=0x7f87781f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-03-09T09:11:55.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.048+0000 7f8776a00640 1 -- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 msgr2=0x7f87781f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:11:55.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.048+0000 7f8776a00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 0x7f87781f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:11:55.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.048+0000 7f8776a00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 0x7f87781f38b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:11:55.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.049+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f876401c400 con 0x7f87780c49b0 2024-03-09T09:11:55.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.050+0000 7f876ea00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f875806b500 0x7f875806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.050+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f876408a9a0 con 0x7f87780c49b0 2024-03-09T09:11:55.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.051+0000 7f8777400640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f875806b500 0x7f875806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.052+0000 7f8777400640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f875806b500 0x7f875806d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f877806c040 tx=0x7f8770009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:55.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.250+0000 7f8776a00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 0x7f87781f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.251+0000 7f8776a00640 1 -- 172.21.15.67:0/2779533226 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87780c2c60 con 0x7f87781f34d0 2024-03-09T09:11:55.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.251+0000 7f8776a00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 0x7f87781f38b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f876800c210 tx=0x7f87780aa810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:55.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.252+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87680049e0 con 0x7f87781f34d0 2024-03-09T09:11:55.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.253+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f8768047070 con 0x7f87781f34d0 2024-03-09T09:11:55.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.254+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 msgr2=0x7f87781f38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.254+0000 7f876ea00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 0x7f87781f38b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f876800c210 tx=0x7f87780aa810 comp rx=0 tx=0).stop 2024-03-09T09:11:55.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.254+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 msgr2=0x7f87781f38b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:55.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.254+0000 7f876ea00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781f34d0 0x7f87781f38b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.262+0000 7f8777e00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781efb10 0x7f87781eff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.262+0000 7f8777e00640 1 -- 172.21.15.67:0/2779533226 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87780c70e0 con 0x7f87781efb10 2024-03-09T09:11:55.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.263+0000 7f8776a00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781efb10 0x7f87781eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.264+0000 7f8776a00640 1 -- 172.21.15.67:0/2779533226 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f87780c2650 con 0x7f87781efb10 2024-03-09T09:11:55.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.264+0000 7f8776a00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781efb10 0x7f87781eff20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8768043000 tx=0x7f8768039040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:55.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.264+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876804c9c0 con 0x7f87781efb10 2024-03-09T09:11:55.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.265+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f8768030040 con 0x7f87781efb10 2024-03-09T09:11:55.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.265+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781efb10 msgr2=0x7f87781eff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.265+0000 7f876ea00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781efb10 0x7f87781eff20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8768043000 tx=0x7f8768039040 comp rx=0 tx=0).stop 2024-03-09T09:11:55.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.266+0000 7f876ea00640 1 -- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781efb10 msgr2=0x7f87781eff20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:11:55.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.266+0000 7f876ea00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87781efb10 0x7f87781eff20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.271 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:11:55.271 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:55.271 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:55.271 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 142, 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:55.272 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:55.273 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:11:55.274 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:11:55.275 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:11:55.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:55.277 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:11:55.278 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:55.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.280 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:11:55.280 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:11:55.280 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:55.280 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:55.280 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:55.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:55.280 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:11:55.280 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:11:55.280 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.281 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:55.282 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:55.283 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:11:55.283 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:11:55.283 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:11:55.283 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:11:55.283 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:11:55.283 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:11:55.283 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:11:55.283 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:11:55.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:11:55.283 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:11:55.283 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:11:55.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.267+0000 7f8777e00640 1 -- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f875806b500 msgr2=0x7f875806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.267+0000 7f8777e00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f875806b500 0x7f875806d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f877806c040 tx=0x7f8770009040 comp rx=0 tx=0).stop 2024-03-09T09:11:55.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.267+0000 7f8777e00640 1 -- 172.21.15.67:0/2779533226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87780c49b0 msgr2=0x7f87780c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.267+0000 7f8777e00640 1 --2- 172.21.15.67:0/2779533226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87780c49b0 0x7f87780c4e10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f87640107c0 tx=0x7f8764010c80 comp rx=0 tx=0).stop 2024-03-09T09:11:55.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.267+0000 7f8777400640 1 -- 172.21.15.67:0/2779533226 reap_dead start 2024-03-09T09:11:55.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.267+0000 7f8777400640 1 -- 172.21.15.67:0/2779533226 reap_dead start 2024-03-09T09:11:55.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.267+0000 7f8777e00640 1 -- 172.21.15.67:0/2779533226 shutdown_connections 2024-03-09T09:11:55.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.267+0000 7f8777e00640 1 -- 172.21.15.67:0/2779533226 >> 172.21.15.67:0/2779533226 conn(0x7f877806f060 msgr2=0x7f877806c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:55.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.267+0000 7f8777e00640 1 -- 172.21.15.67:0/2779533226 shutdown_connections 2024-03-09T09:11:55.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.267+0000 7f8777e00640 1 -- 172.21.15.67:0/2779533226 wait complete. 2024-03-09T09:11:55.297 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:11:55.297 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:11:55.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:11:55.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.507+0000 7f3682600640 1 Processor -- start 2024-03-09T09:11:55.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.508+0000 7f3682600640 1 -- start start 2024-03-09T09:11:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.509+0000 7f3682600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f367c0710a0 0x7f367c153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.509+0000 7f3682600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f367c151360 0x7f367c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.509+0000 7f3682600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f367c0c81e0 con 0x7f367c0c1f70 2024-03-09T09:11:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.509+0000 7f3682600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f367c0c32b0 con 0x7f367c151360 2024-03-09T09:11:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.509+0000 7f3682600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367c0c2fd0 con 0x7f367c0710a0 2024-03-09T09:11:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.509+0000 7f3681c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f367c151360 0x7f367c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.509+0000 7f3681c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f367c151360 0x7f367c153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49386/0 (socket says 172.21.15.67:49386) 2024-03-09T09:11:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.509+0000 7f3681c00640 1 -- 172.21.15.67:0/3387623319 learned_addr learned my addr 172.21.15.67:0/3387623319 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.510+0000 7f367be00640 1 --2- 172.21.15.67:0/3387623319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f367c0710a0 0x7f367c153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.510+0000 7f3681c00640 1 -- 172.21.15.67:0/3387623319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f367c0710a0 msgr2=0x7f367c153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.510+0000 7f3681c00640 1 --2- 172.21.15.67:0/3387623319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f367c0710a0 0x7f367c153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.510+0000 7f3681c00640 1 -- 172.21.15.67:0/3387623319 >> v1:172.21.15.134:6789/0 conn(0x7f367c0c1f70 legacy=0x7f367c150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:11:55.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.510+0000 7f3681c00640 1 -- 172.21.15.67:0/3387623319 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c0c3700 con 0x7f367c151360 2024-03-09T09:11:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.511+0000 7f3681c00640 1 --2- 172.21.15.67:0/3387623319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f367c151360 0x7f367c153eb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3674009e50 tx=0x7f367402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191e6431500ea63a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.511+0000 7f367b400640 1 -- 172.21.15.67:0/3387623319 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367402c9e0 con 0x7f367c151360 2024-03-09T09:11:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.511+0000 7f367b400640 1 -- 172.21.15.67:0/3387623319 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367402cb80 con 0x7f367c151360 2024-03-09T09:11:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.511+0000 7f367b400640 1 -- 172.21.15.67:0/3387623319 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36740343f0 con 0x7f367c151360 2024-03-09T09:11:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.512+0000 7f3682600640 1 -- 172.21.15.67:0/3387623319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f367c151360 msgr2=0x7f367c153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.512+0000 7f3682600640 1 --2- 172.21.15.67:0/3387623319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f367c151360 0x7f367c153eb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3674009e50 tx=0x7f367402be40 comp rx=0 tx=0).stop 2024-03-09T09:11:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.512+0000 7f3682600640 1 -- 172.21.15.67:0/3387623319 shutdown_connections 2024-03-09T09:11:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.512+0000 7f3682600640 1 --2- 172.21.15.67:0/3387623319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f367c151360 0x7f367c153eb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.512+0000 7f3682600640 1 --2- 172.21.15.67:0/3387623319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f367c0710a0 0x7f367c153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.512+0000 7f3682600640 1 -- 172.21.15.67:0/3387623319 >> 172.21.15.67:0/3387623319 conn(0x7f367c0c7300 msgr2=0x7f367c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.512+0000 7f3682600640 1 -- 172.21.15.67:0/3387623319 shutdown_connections 2024-03-09T09:11:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.512+0000 7f3682600640 1 -- 172.21.15.67:0/3387623319 wait complete. 2024-03-09T09:11:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.514+0000 7f3682600640 1 Processor -- start 2024-03-09T09:11:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.514+0000 7f3682600640 1 -- start start 2024-03-09T09:11:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.515+0000 7f3682600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f367c0710a0 0x7f367c1eb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.515+0000 7f3682600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f367c0c1f70 0x7f367c1ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.515+0000 7f3682600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f367c151360 0x7f367c1dd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.515+0000 7f3682600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c070d50 con 0x7f367c151360 2024-03-09T09:11:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.515+0000 7f3682600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c070830 con 0x7f367c0710a0 2024-03-09T09:11:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.515+0000 7f3682600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367c070ed0 con 0x7f367c0c1f70 2024-03-09T09:11:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.515+0000 7f3681c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f367c151360 0x7f367c1dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.515+0000 7f3681c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f367c151360 0x7f367c1dd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49396/0 (socket says 172.21.15.67:49396) 2024-03-09T09:11:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.515+0000 7f3681c00640 1 -- 172.21.15.67:0/250706442 learned_addr learned my addr 172.21.15.67:0/250706442 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:11:55.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f367be00640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f367c0c1f70 0x7f367c1ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3681c00640 1 -- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f367c0c1f70 msgr2=0x7f367c1ebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3681c00640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f367c0c1f70 0x7f367c1ebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3681c00640 1 -- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f367c0710a0 msgr2=0x7f367c1eb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3681200640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f367c0710a0 0x7f367c1eb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3681c00640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f367c0710a0 0x7f367c1eb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3681c00640 1 -- 172.21.15.67:0/250706442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c1debb0 con 0x7f367c151360 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f367be00640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f367c0c1f70 0x7f367c1ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3681200640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f367c0710a0 0x7f367c1eb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3681c00640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f367c151360 0x7f367c1dd790 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3674009e50 tx=0x7f3674004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3678c00640 1 -- 172.21.15.67:0/250706442 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674043020 con 0x7f367c151360 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3678c00640 1 -- 172.21.15.67:0/250706442 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367403e450 con 0x7f367c151360 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3682600640 1 -- 172.21.15.67:0/250706442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367c1df9d0 con 0x7f367c151360 2024-03-09T09:11:55.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.516+0000 7f3678c00640 1 -- 172.21.15.67:0/250706442 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674009510 con 0x7f367c151360 2024-03-09T09:11:55.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.517+0000 7f3682600640 1 -- 172.21.15.67:0/250706442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367c1ec4b0 con 0x7f367c151360 2024-03-09T09:11:55.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.518+0000 7f3682600640 1 -- 172.21.15.67:0/250706442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367c04f3c0 con 0x7f367c151360 2024-03-09T09:11:55.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.519+0000 7f3678c00640 1 -- 172.21.15.67:0/250706442 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3674046020 con 0x7f367c151360 2024-03-09T09:11:55.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.520+0000 7f3678c00640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f365c06b4d0 0x7f365c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:11:55.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.520+0000 7f3678c00640 1 -- 172.21.15.67:0/250706442 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f36740ace00 con 0x7f367c151360 2024-03-09T09:11:55.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.520+0000 7f3681200640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f365c06b4d0 0x7f365c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:11:55.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.521+0000 7f3681200640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f365c06b4d0 0x7f365c06d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f367000a830 tx=0x7f3670008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:11:55.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.525+0000 7f3678c00640 1 -- 172.21.15.67:0/250706442 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f367404d030 con 0x7f367c151360 2024-03-09T09:11:55.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.986+0000 7f3682600640 1 -- 172.21.15.67:0/250706442 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f367c14d120 con 0x7f367c151360 2024-03-09T09:11:55.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.990+0000 7f3678c00640 1 -- 172.21.15.67:0/250706442 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f367407b880 con 0x7f367c151360 2024-03-09T09:11:55.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.993+0000 7f3682600640 1 -- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f365c06b4d0 msgr2=0x7f365c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.993+0000 7f3682600640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f365c06b4d0 0x7f365c06d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f367000a830 tx=0x7f3670008040 comp rx=0 tx=0).stop 2024-03-09T09:11:55.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.993+0000 7f3682600640 1 -- 172.21.15.67:0/250706442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f367c151360 msgr2=0x7f367c1dd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:11:55.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.993+0000 7f3682600640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f367c151360 0x7f367c1dd790 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3674009e50 tx=0x7f3674004a40 comp rx=0 tx=0).stop 2024-03-09T09:11:55.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.994+0000 7f3682600640 1 -- 172.21.15.67:0/250706442 shutdown_connections 2024-03-09T09:11:55.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.994+0000 7f3682600640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f365c06b4d0 0x7f365c06d990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.994+0000 7f3682600640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f367c151360 0x7f367c1dd790 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:55.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.994+0000 7f3682600640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f367c0c1f70 0x7f367c1ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:56.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.994+0000 7f3682600640 1 --2- 172.21.15.67:0/250706442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f367c0710a0 0x7f367c1eb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:11:56.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.994+0000 7f3682600640 1 -- 172.21.15.67:0/250706442 >> 172.21.15.67:0/250706442 conn(0x7f367c0c7300 msgr2=0x7f367c14f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:11:56.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.995+0000 7f3682600640 1 -- 172.21.15.67:0/250706442 shutdown_connections 2024-03-09T09:11:56.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:11:55.995+0000 7f3682600640 1 -- 172.21.15.67:0/250706442 wait complete. 2024-03-09T09:11:56.031 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:11:58.342 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 17% (53/304) Updating files: 18% (55/304) Updating files: 19% (58/304) Updating files: 20% (61/304) Updating files: 21% (64/304) Updating files: 22% (67/304) Updating files: 23% (70/304) Updating files: 24% (73/304) Updating files: 25% (76/304) Updating files: 26% (80/304) Updating files: 27% (83/304) Updating files: 28% (86/304) Updating files: 29% (89/304) Updating files: 30% (92/304) Updating files: 31% (95/304) Updating files: 32% (98/304) Updating files: 33% (101/304) Updating files: 34% (104/304) Updating files: 35% (107/304) Updating files: 36% (110/304) Updating files: 37% (113/304) Updating files: 38% (116/304) Updating files: 39% (119/304) Updating files: 40% (122/304) Updating files: 41% (125/304) Updating files: 42% (128/304) Updating files: 43% (131/304) Updating files: 43% (133/304) Updating files: 44% (134/304) Updating files: 45% (137/304) Updating files: 46% (140/304) Updating files: 47% (143/304) Updating files: 48% (146/304) Updating files: 49% (149/304) Updating files: 50% (152/304) Updating files: 51% (156/304) Updating files: 52% (159/304) Updating files: 53% (162/304) Updating files: 54% (165/304) Updating files: 55% (168/304) Updating files: 56% (171/304) Updating files: 57% (174/304) Updating files: 58% (177/304) Updating files: 59% (180/304) Updating files: 60% (183/304) Updating files: 61% (186/304) Updating files: 62% (189/304) Updating files: 63% (192/304) Updating files: 64% (195/304) Updating files: 65% (198/304) Updating files: 66% (201/304) Updating files: 67% (204/304) Updating files: 68% (207/304) Updating files: 69% (210/304) Updating files: 70% (213/304) Updating files: 71% (216/304) Updating files: 72% (219/304) Updating files: 73% (222/304) Updating files: 74% (225/304) Updating files: 75% (228/304) Updating files: 76% (232/304) Updating files: 77% (235/304) Updating files: 78% (238/304) Updating files: 79% (241/304) Updating files: 80% (244/304) Updating files: 81% (247/304) Updating files: 82% (250/304) Updating files: 83% (253/304) Updating files: 83% (254/304) Updating files: 84% (256/304) Updating files: 85% (259/304) Updating files: 86% (262/304) Updating files: 87% (265/304) Updating files: 88% (268/304) Updating files: 89% (271/304) Updating files: 90% (274/304) Updating files: 91% (277/304) Updating files: 92% (280/304) Updating files: 93% (283/304) Updating files: 94% (286/304) Updating files: 95% (289/304) Updating files: 96% (292/304) Updating files: 97% (295/304) Updating files: 98% (298/304) Updating files: 99% (301/304) Updating files: 100% (304/304) Updating files: 100% (304/304), done. 2024-03-09T09:11:58.381 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 9a7a9d06c06 v0.57 2024-03-09T09:11:58.388 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.57 2024-03-09T09:11:58.542 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:11:58.542 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 58 -eq 48 ']' 2024-03-09T09:11:58.542 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.58 2024-03-09T09:11:58.543 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.58 2024-03-09T09:11:58.543 INFO:tasks.workunit.client.0.smithi067.stdout:v0.58 2024-03-09T09:11:58.543 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.58 2024-03-09T09:12:05.823 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 23% (128/540) Updating files: 24% (130/540) Updating files: 25% (135/540) Updating files: 26% (141/540) Updating files: 27% (146/540) Updating files: 28% (152/540) Updating files: 29% (157/540) Updating files: 30% (162/540) Updating files: 31% (168/540) Updating files: 32% (173/540) Updating files: 33% (179/540) Updating files: 34% (184/540) Updating files: 35% (189/540) Updating files: 36% (195/540) Updating files: 37% (200/540) Updating files: 38% (206/540) Updating files: 39% (211/540) Updating files: 40% (216/540) Updating files: 41% (222/540) Updating files: 41% (224/540) Updating files: 42% (227/540) Updating files: 43% (233/540) Updating files: 44% (238/540) Updating files: 45% (243/540) Updating files: 46% (249/540) Updating files: 47% (254/540) Updating files: 48% (260/540) Updating files: 49% (265/540) Updating files: 50% (270/540) Updating files: 51% (276/540) Updating files: 52% (281/540) Updating files: 53% (287/540) Updating files: 54% (292/540) Updating files: 54% (295/540) Updating files: 55% (297/540) Updating files: 56% (303/540) Updating files: 57% (308/540) Updating files: 58% (314/540) Updating files: 59% (319/540) Updating files: 60% (324/540) Updating files: 61% (330/540) Updating files: 62% (335/540) Updating files: 63% (341/540) Updating files: 63% (343/540) Updating files: 64% (346/540) Updating files: 65% (351/540) Updating files: 66% (357/540) Updating files: 67% (362/540) Updating files: 68% (368/540) Updating files: 69% (373/540) Updating files: 69% (377/540) Updating files: 70% (378/540) Updating files: 71% (384/540) Updating files: 72% (389/540) Updating files: 73% (395/540) Updating files: 73% (399/540) Updating files: 74% (400/540) Updating files: 75% (405/540) Updating files: 75% (407/540) Updating files: 76% (411/540) Updating files: 77% (416/540) Updating files: 78% (422/540) Updating files: 79% (427/540) Updating files: 80% (432/540) Updating files: 81% (438/540) Updating files: 82% (443/540) Updating files: 83% (449/540) Updating files: 84% (454/540) Updating files: 85% (459/540) Updating files: 85% (463/540) Updating files: 86% (465/540) Updating files: 87% (470/540) Updating files: 88% (476/540) Updating files: 89% (481/540) Updating files: 90% (486/540) Updating files: 91% (492/540) Updating files: 92% (497/540) Updating files: 93% (503/540) Updating files: 94% (508/540) Updating files: 95% (513/540) Updating files: 96% (519/540) Updating files: 97% (524/540) Updating files: 98% (530/540) Updating files: 99% (535/540) Updating files: 100% (540/540) Updating files: 100% (540/540), done. 2024-03-09T09:12:05.891 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 690f8175606 v0.55 2024-03-09T09:12:05.911 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.55 2024-03-09T09:12:06.032 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:12:06.033 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:12:06.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.236+0000 7f6b34e00640 1 Processor -- start 2024-03-09T09:12:06.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.236+0000 7f6b34e00640 1 -- start start 2024-03-09T09:12:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.237+0000 7f6b34e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b300c4330 0x7f6b30157210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.237+0000 7f6b34e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6b3006ffd0 0x7f6b30157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.237+0000 7f6b34e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b30153ff0 con 0x7f6b30151510 2024-03-09T09:12:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.237+0000 7f6b34e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b301542d0 con 0x7f6b3006ffd0 2024-03-09T09:12:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.237+0000 7f6b34e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b301545b0 con 0x7f6b300c4330 2024-03-09T09:12:06.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.238+0000 7f6b2e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6b3006ffd0 0x7f6b30157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.238+0000 7f6b2e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6b3006ffd0 0x7f6b30157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33354/0 (socket says 172.21.15.67:33354) 2024-03-09T09:12:06.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.238+0000 7f6b2e800640 1 -- 172.21.15.67:0/3161049710 learned_addr learned my addr 172.21.15.67:0/3161049710 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:06.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.238+0000 7f6b2d400640 1 --2- 172.21.15.67:0/3161049710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b300c4330 0x7f6b30157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.238+0000 7f6b2e800640 1 -- 172.21.15.67:0/3161049710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b300c4330 msgr2=0x7f6b30157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:06.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.238+0000 7f6b2e800640 1 --2- 172.21.15.67:0/3161049710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b300c4330 0x7f6b30157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.238+0000 7f6b2e800640 1 -- 172.21.15.67:0/3161049710 >> v1:172.21.15.134:6789/0 conn(0x7f6b30151510 legacy=0x7f6b300729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.238+0000 7f6b2e800640 1 -- 172.21.15.67:0/3161049710 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b30154890 con 0x7f6b3006ffd0 2024-03-09T09:12:06.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.240+0000 7f6b2e800640 1 --2- 172.21.15.67:0/3161049710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b3006ffd0 0x7f6b30157750 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009e50 tx=0x7f6b2802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8159466512d6586b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:06.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.240+0000 7f6b2ca00640 1 -- 172.21.15.67:0/3161049710 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2802cb00 con 0x7f6b3006ffd0 2024-03-09T09:12:06.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.240+0000 7f6b2ca00640 1 -- 172.21.15.67:0/3161049710 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2802cca0 con 0x7f6b3006ffd0 2024-03-09T09:12:06.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.240+0000 7f6b2ca00640 1 -- 172.21.15.67:0/3161049710 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28034610 con 0x7f6b3006ffd0 2024-03-09T09:12:06.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.241+0000 7f6b34e00640 1 -- 172.21.15.67:0/3161049710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b3006ffd0 msgr2=0x7f6b30157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.241+0000 7f6b34e00640 1 --2- 172.21.15.67:0/3161049710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b3006ffd0 0x7f6b30157750 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009e50 tx=0x7f6b2802bf60 comp rx=0 tx=0).stop 2024-03-09T09:12:06.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.241+0000 7f6b34e00640 1 -- 172.21.15.67:0/3161049710 shutdown_connections 2024-03-09T09:12:06.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.241+0000 7f6b34e00640 1 --2- 172.21.15.67:0/3161049710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b3006ffd0 0x7f6b30157750 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.241+0000 7f6b34e00640 1 --2- 172.21.15.67:0/3161049710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b300c4330 0x7f6b30157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.241+0000 7f6b34e00640 1 -- 172.21.15.67:0/3161049710 >> 172.21.15.67:0/3161049710 conn(0x7f6b300ab320 msgr2=0x7f6b300ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:06.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.241+0000 7f6b34e00640 1 -- 172.21.15.67:0/3161049710 shutdown_connections 2024-03-09T09:12:06.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.241+0000 7f6b34e00640 1 -- 172.21.15.67:0/3161049710 wait complete. 2024-03-09T09:12:06.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.244+0000 7f6b34e00640 1 Processor -- start 2024-03-09T09:12:06.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.244+0000 7f6b34e00640 1 -- start start 2024-03-09T09:12:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.245+0000 7f6b34e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b3006ffd0 0x7f6b300c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.245+0000 7f6b34e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b300c4330 0x7f6b3014fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.245+0000 7f6b34e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b30150080 0x7f6b301ebca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.245+0000 7f6b34e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b30073250 con 0x7f6b30150080 2024-03-09T09:12:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.245+0000 7f6b34e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b300730d0 con 0x7f6b300c4330 2024-03-09T09:12:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.245+0000 7f6b34e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b300c3f70 con 0x7f6b3006ffd0 2024-03-09T09:12:06.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.246+0000 7f6b2e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b30150080 0x7f6b301ebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.246+0000 7f6b2e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b30150080 0x7f6b301ebca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33370/0 (socket says 172.21.15.67:33370) 2024-03-09T09:12:06.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.246+0000 7f6b2e800640 1 -- 172.21.15.67:0/432942935 learned_addr learned my addr 172.21.15.67:0/432942935 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:06.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.246+0000 7f6b2d400640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b300c4330 0x7f6b3014fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.246+0000 7f6b2de00640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b3006ffd0 0x7f6b300c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.246+0000 7f6b2e800640 1 -- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b3006ffd0 msgr2=0x7f6b300c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.246+0000 7f6b2e800640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b3006ffd0 0x7f6b300c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.246+0000 7f6b2e800640 1 -- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b300c4330 msgr2=0x7f6b3014fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.246+0000 7f6b2e800640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b300c4330 0x7f6b3014fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.246+0000 7f6b2e800640 1 -- 172.21.15.67:0/432942935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b301dfee0 con 0x7f6b30150080 2024-03-09T09:12:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.246+0000 7f6b2e800640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b30150080 0x7f6b301ebca0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009e50 tx=0x7f6b28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.247+0000 7f6b26000640 1 -- 172.21.15.67:0/432942935 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28043020 con 0x7f6b30150080 2024-03-09T09:12:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.247+0000 7f6b26000640 1 -- 172.21.15.67:0/432942935 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2802ce50 con 0x7f6b30150080 2024-03-09T09:12:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.247+0000 7f6b34e00640 1 -- 172.21.15.67:0/432942935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b30150530 con 0x7f6b30150080 2024-03-09T09:12:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.247+0000 7f6b26000640 1 -- 172.21.15.67:0/432942935 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28009510 con 0x7f6b30150080 2024-03-09T09:12:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.247+0000 7f6b34e00640 1 -- 172.21.15.67:0/432942935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b301dd670 con 0x7f6b30150080 2024-03-09T09:12:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.248+0000 7f6b34e00640 1 -- 172.21.15.67:0/432942935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b30075a00 con 0x7f6b30150080 2024-03-09T09:12:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.249+0000 7f6b26000640 1 -- 172.21.15.67:0/432942935 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6b28046020 con 0x7f6b30150080 2024-03-09T09:12:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.250+0000 7f6b26000640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b1006b500 0x7f6b1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.250+0000 7f6b26000640 1 -- 172.21.15.67:0/432942935 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6b280ac9b0 con 0x7f6b30150080 2024-03-09T09:12:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.251+0000 7f6b2de00640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b1006b500 0x7f6b1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.252+0000 7f6b2de00640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b1006b500 0x7f6b1006d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c001500 tx=0x7f6b1c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.254+0000 7f6b26000640 1 -- 172.21.15.67:0/432942935 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6b2804d070 con 0x7f6b30150080 2024-03-09T09:12:06.503 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:12:06.503 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 56 -eq 48 ']' 2024-03-09T09:12:06.503 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.56 2024-03-09T09:12:06.503 INFO:tasks.workunit.client.1.smithi134.stdout:v0.56 2024-03-09T09:12:06.504 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.56 2024-03-09T09:12:06.504 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.56 2024-03-09T09:12:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.546+0000 7f6b34e00640 1 -- 172.21.15.67:0/432942935 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6b30070b10 con 0x7f6b30150080 2024-03-09T09:12:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.547+0000 7f6b26000640 1 -- 172.21.15.67:0/432942935 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f6b301dfee0 con 0x7f6b30150080 2024-03-09T09:12:06.552 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":142,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:12:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 -- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b1006b500 msgr2=0x7f6b1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b1006b500 0x7f6b1006d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c001500 tx=0x7f6b1c009040 comp rx=0 tx=0).stop 2024-03-09T09:12:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 -- 172.21.15.67:0/432942935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b30150080 msgr2=0x7f6b301ebca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b30150080 0x7f6b301ebca0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009e50 tx=0x7f6b28004a40 comp rx=0 tx=0).stop 2024-03-09T09:12:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 -- 172.21.15.67:0/432942935 shutdown_connections 2024-03-09T09:12:06.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b1006b500 0x7f6b1006d9c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b30150080 0x7f6b301ebca0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b300c4330 0x7f6b3014fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 --2- 172.21.15.67:0/432942935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b3006ffd0 0x7f6b300c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 -- 172.21.15.67:0/432942935 >> 172.21.15.67:0/432942935 conn(0x7f6b300ab320 msgr2=0x7f6b301534d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 -- 172.21.15.67:0/432942935 shutdown_connections 2024-03-09T09:12:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.550+0000 7f6b34e00640 1 -- 172.21.15.67:0/432942935 wait complete. 2024-03-09T09:12:06.581 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:12:06.581 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:12:06.581 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:12:06.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:12:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.789+0000 7fc769800640 1 Processor -- start 2024-03-09T09:12:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.789+0000 7fc769800640 1 -- start start 2024-03-09T09:12:06.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.791+0000 7fc769800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc764151360 0x7fc7640c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.791+0000 7fc769800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc7640c30a0 0x7fc7641539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.791+0000 7fc769800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7640c2780 con 0x7fc7640ac930 2024-03-09T09:12:06.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.791+0000 7fc769800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc764070670 con 0x7fc7640c30a0 2024-03-09T09:12:06.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.791+0000 7fc769800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640708c0 con 0x7fc764151360 2024-03-09T09:12:06.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.791+0000 7fc763e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc7640ac930 0x7fc764150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:46274/0 (socket says 172.21.15.67:46274) 2024-03-09T09:12:06.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.791+0000 7fc763e00640 1 -- 172.21.15.67:0/1734908382 learned_addr learned my addr 172.21.15.67:0/1734908382 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:12:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.791+0000 7fc763400640 1 --2- 172.21.15.67:0/1734908382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc764151360 0x7fc7640c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.792+0000 7fc768e00640 1 --2- 172.21.15.67:0/1734908382 >> v2:172.21.15.134:3300/0 conn(0x7fc7640c30a0 0x7fc7641539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.792+0000 7fc763400640 1 -- 172.21.15.67:0/1734908382 >> v2:172.21.15.134:3300/0 conn(0x7fc7640c30a0 msgr2=0x7fc7641539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.792+0000 7fc763400640 1 --2- 172.21.15.67:0/1734908382 >> v2:172.21.15.134:3300/0 conn(0x7fc7640c30a0 0x7fc7641539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.792+0000 7fc763400640 1 -- 172.21.15.67:0/1734908382 >> v1:172.21.15.134:6789/0 conn(0x7fc7640ac930 legacy=0x7fc764150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.792+0000 7fc763400640 1 -- 172.21.15.67:0/1734908382 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc764070ba0 con 0x7fc764151360 2024-03-09T09:12:06.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.794+0000 7fc763400640 1 --2- 172.21.15.67:0/1734908382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc764151360 0x7fc7640c3660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc7500097a0 tx=0x7fc75002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7796f464d0e5955 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:06.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.794+0000 7fc762a00640 1 -- 172.21.15.67:0/1734908382 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750035020 con 0x7fc764151360 2024-03-09T09:12:06.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.794+0000 7fc762a00640 1 -- 172.21.15.67:0/1734908382 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc750031810 con 0x7fc764151360 2024-03-09T09:12:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.794+0000 7fc762a00640 1 -- 172.21.15.67:0/1734908382 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750031be0 con 0x7fc764151360 2024-03-09T09:12:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.795+0000 7fc769800640 1 -- 172.21.15.67:0/1734908382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc764151360 msgr2=0x7fc7640c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.795+0000 7fc769800640 1 --2- 172.21.15.67:0/1734908382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc764151360 0x7fc7640c3660 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc7500097a0 tx=0x7fc75002b8b0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.795+0000 7fc769800640 1 -- 172.21.15.67:0/1734908382 shutdown_connections 2024-03-09T09:12:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.795+0000 7fc769800640 1 --2- 172.21.15.67:0/1734908382 >> v2:172.21.15.134:3300/0 conn(0x7fc7640c30a0 0x7fc7641539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.795+0000 7fc769800640 1 --2- 172.21.15.67:0/1734908382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc764151360 0x7fc7640c3660 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.795+0000 7fc769800640 1 -- 172.21.15.67:0/1734908382 >> 172.21.15.67:0/1734908382 conn(0x7fc7640c7220 msgr2=0x7fc7640c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.795+0000 7fc769800640 1 -- 172.21.15.67:0/1734908382 shutdown_connections 2024-03-09T09:12:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.795+0000 7fc769800640 1 -- 172.21.15.67:0/1734908382 wait complete. 2024-03-09T09:12:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.797+0000 7fc769800640 1 Processor -- start 2024-03-09T09:12:06.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.798+0000 7fc769800640 1 -- start start 2024-03-09T09:12:06.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.799+0000 7fc769800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7640ac930 0x7fc764146670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.799+0000 7fc769800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7640c30a0 0x7fc76414dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.799+0000 7fc769800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc764151360 0x7fc76414dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.799+0000 7fc769800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640ace90 con 0x7fc7640c30a0 2024-03-09T09:12:06.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.799+0000 7fc769800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7640acd10 con 0x7fc7640ac930 2024-03-09T09:12:06.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.799+0000 7fc769800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc764153f20 con 0x7fc764151360 2024-03-09T09:12:06.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.799+0000 7fc763400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7640c30a0 0x7fc76414dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.799+0000 7fc763400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7640c30a0 0x7fc76414dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33386/0 (socket says 172.21.15.67:33386) 2024-03-09T09:12:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.799+0000 7fc763400640 1 -- 172.21.15.67:0/3587879378 learned_addr learned my addr 172.21.15.67:0/3587879378 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.800+0000 7fc768e00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc764151360 0x7fc76414dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.800+0000 7fc763e00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7640ac930 0x7fc764146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.800+0000 7fc763400640 1 -- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc764151360 msgr2=0x7fc76414dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.800+0000 7fc763400640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc764151360 0x7fc76414dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.800+0000 7fc763400640 1 -- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7640ac930 msgr2=0x7fc764146670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.800+0000 7fc763400640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7640ac930 0x7fc764146670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.800+0000 7fc763400640 1 -- 172.21.15.67:0/3587879378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7640c5930 con 0x7fc7640c30a0 2024-03-09T09:12:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.800+0000 7fc768e00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc764151360 0x7fc76414dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:12:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.800+0000 7fc763e00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7640ac930 0x7fc764146670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:12:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.800+0000 7fc763400640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7640c30a0 0x7fc76414dab0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc750034040 tx=0x7fc7500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.801+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750040070 con 0x7fc7640c30a0 2024-03-09T09:12:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.801+0000 7fc769800640 1 -- 172.21.15.67:0/3587879378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7640c5c10 con 0x7fc7640c30a0 2024-03-09T09:12:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.801+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc750039cf0 con 0x7fc7640c30a0 2024-03-09T09:12:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.801+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750035040 con 0x7fc7640c30a0 2024-03-09T09:12:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.801+0000 7fc769800640 1 -- 172.21.15.67:0/3587879378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7640c1130 con 0x7fc7640c30a0 2024-03-09T09:12:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.802+0000 7fc769800640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc76414fa50 0x7fc76414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.802+0000 7fc769800640 1 -- 172.21.15.67:0/3587879378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc764146c50 con 0x7fc76414fa50 2024-03-09T09:12:06.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.802+0000 7fc763e00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc76414fa50 0x7fc76414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.803+0000 7fc763e00640 1 -- 172.21.15.67:0/3587879378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc76404f3c0 con 0x7fc76414fa50 2024-03-09T09:12:06.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.803+0000 7fc763e00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc76414fa50 0x7fc76414fe30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc75400d450 tx=0x7fc75400d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.804+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc754005ae0 con 0x7fc76414fa50 2024-03-09T09:12:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.804+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc754005c80 con 0x7fc76414fa50 2024-03-09T09:12:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.804+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc76414fa50 msgr2=0x7fc76414fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.804+0000 7fc75be00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc76414fa50 0x7fc76414fe30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc75400d450 tx=0x7fc75400d910 comp rx=0 tx=0).stop 2024-03-09T09:12:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.804+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc76414fa50 msgr2=0x7fc76414fe30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.804+0000 7fc75be00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc76414fa50 0x7fc76414fe30 secure :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc75400d450 tx=0x7fc75400d910 comp rx=0 tx=0).stop 2024-03-09T09:12:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.804+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc750038020 con 0x7fc7640c30a0 2024-03-09T09:12:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.806+0000 7fc75be00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc74406afb0 0x7fc74406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.806+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc7500ac200 con 0x7fc7640c30a0 2024-03-09T09:12:06.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.806+0000 7fc763e00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc74406afb0 0x7fc74406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.808+0000 7fc763e00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc74406afb0 0x7fc74406d470 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc754000fd0 tx=0x7fc754011000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.813+0000 7fc769800640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764150370 0x7fc764149660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.813+0000 7fc769800640 1 -- 172.21.15.67:0/3587879378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76414f770 con 0x7fc764150370 2024-03-09T09:12:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.813+0000 7fc768e00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764150370 0x7fc764149660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:06.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.814+0000 7fc768e00640 1 -- 172.21.15.67:0/3587879378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc764150750 con 0x7fc764150370 2024-03-09T09:12:06.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.814+0000 7fc768e00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764150370 0x7fc764149660 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00f380 tx=0x7fc75c00f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:06.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.814+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c016420 con 0x7fc764150370 2024-03-09T09:12:06.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7fc75c016420 con 0x7fc764150370 2024-03-09T09:12:06.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764150370 msgr2=0x7fc764149660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc75be00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764150370 0x7fc764149660 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00f380 tx=0x7fc75c00f840 comp rx=0 tx=0).stop 2024-03-09T09:12:06.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc75be00640 1 -- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764150370 msgr2=0x7fc764149660 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:06.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc75be00640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764150370 0x7fc764149660 secure :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00f380 tx=0x7fc75c00f840 comp rx=0 tx=0).stop 2024-03-09T09:12:06.821 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:12:06.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:12:06.821 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:12:06.821 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:12:06.821 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 142, 2024-03-09T09:12:06.821 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:12:06.821 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:12:06.821 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:12:06.821 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:12:06.821 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:06.822 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:06.823 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:12:06.824 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:06.825 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:06.826 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:06.827 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:06.828 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:12:06.829 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.830 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:06.831 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:06.832 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:12:06.833 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:06.834 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:06.834 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:06.834 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:12:06.834 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:06.834 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:06.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:06.834 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:12:06.834 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:12:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc769800640 1 -- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc74406afb0 msgr2=0x7fc74406d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc769800640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc74406afb0 0x7fc74406d470 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc754000fd0 tx=0x7fc754011000 comp rx=0 tx=0).stop 2024-03-09T09:12:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc769800640 1 -- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7640c30a0 msgr2=0x7fc76414dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc769800640 1 --2- 172.21.15.67:0/3587879378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7640c30a0 0x7fc76414dab0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc750034040 tx=0x7fc7500046c0 comp rx=0 tx=0).stop 2024-03-09T09:12:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc768e00640 1 -- 172.21.15.67:0/3587879378 reap_dead start 2024-03-09T09:12:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc769800640 1 -- 172.21.15.67:0/3587879378 shutdown_connections 2024-03-09T09:12:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc769800640 1 -- 172.21.15.67:0/3587879378 >> 172.21.15.67:0/3587879378 conn(0x7fc7640c7220 msgr2=0x7fc764152b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc769800640 1 -- 172.21.15.67:0/3587879378 shutdown_connections 2024-03-09T09:12:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:06.815+0000 7fc769800640 1 -- 172.21.15.67:0/3587879378 wait complete. 2024-03-09T09:12:06.844 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:12:06.844 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:12:06.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:12:07.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.065+0000 7f2201600640 1 Processor -- start 2024-03-09T09:12:07.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.065+0000 7f2201600640 1 -- start start 2024-03-09T09:12:07.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.067+0000 7f2201600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21fc0c2780 0x7f21fc154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.067+0000 7f2201600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21fc0c8200 con 0x7f21fc0ac9a0 2024-03-09T09:12:07.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.067+0000 7f2201600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21fc0c3990 con 0x7f21fc071190 2024-03-09T09:12:07.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.067+0000 7f2201600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc06d590 con 0x7f21fc0c2780 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.067+0000 7f21faa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f21fc0ac9a0 0x7f21fc1510e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40000/0 (socket says 172.21.15.67:40000) 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.067+0000 7f21faa00640 1 -- 172.21.15.67:0/3874451726 learned_addr learned my addr 172.21.15.67:0/3874451726 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.067+0000 7f21fa000640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21fc0c2780 0x7f21fc154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.068+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843263104 0 0) 0x7f21fc0c8200 con 0x7f21fc0ac9a0 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.068+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e0003520 con 0x7f21fc0ac9a0 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.068+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408670352 0 0) 0x7f21fc0c3990 con 0x7f21fc071190 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.068+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21fc0c8200 con 0x7f21fc071190 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.068+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 967874946 0 0) 0x7f21e0003520 con 0x7f21fc0ac9a0 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.068+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21fc0c2780 msgr2=0x7f21fc154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.068+0000 7f21f9600640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21fc0c2780 0x7f21fc154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.068+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 >> v1:172.21.15.134:6789/0 conn(0x7f21fc071190 legacy=0x7f21fc1546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.068+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc070920 con 0x7f21fc0ac9a0 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.068+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f21ec001670 con 0x7f21fc0ac9a0 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 >> v1:172.21.15.67:6789/0 conn(0x7f21fc0ac9a0 legacy=0x7f21fc1510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21f9600640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21e0003b70 0x7f21e0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21f9600640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e0004800 0x7f21e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21f9600640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21e00051c0 0x7f21e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e000c090 con 0x7f21e0003b70 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e000c210 con 0x7f21e0004800 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e000c410 con 0x7f21e00051c0 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21fa000640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21fc0c2780 0x7f21fc154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21faa00640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21e0003b70 0x7f21e0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21fb400640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21e00051c0 0x7f21e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21faa00640 1 -- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21e00051c0 msgr2=0x7f21e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21faa00640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21e00051c0 0x7f21e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21faa00640 1 -- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e0004800 msgr2=0x7f21e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:07.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21faa00640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e0004800 0x7f21e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.069+0000 7f21faa00640 1 -- 172.21.15.67:0/3874451726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e0003520 con 0x7f21e0003b70 2024-03-09T09:12:07.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.070+0000 7f21fb400640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21e00051c0 0x7f21e000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:12:07.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.071+0000 7f21faa00640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21e0003b70 0x7f21e0003f70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c0e0 tx=0x7f21e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f779a610c934916 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.072+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec007c40 con 0x7f21e0003b70 2024-03-09T09:12:07.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.072+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ec030070 con 0x7f21e0003b70 2024-03-09T09:12:07.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.072+0000 7f21f9600640 1 -- 172.21.15.67:0/3874451726 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec005d50 con 0x7f21e0003b70 2024-03-09T09:12:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.072+0000 7f2201600640 1 -- 172.21.15.67:0/3874451726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21e0003b70 msgr2=0x7f21e0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.072+0000 7f2201600640 1 --2- 172.21.15.67:0/3874451726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21e0003b70 0x7f21e0003f70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c0e0 tx=0x7f21e0005830 comp rx=0 tx=0).stop 2024-03-09T09:12:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.072+0000 7f21fb400640 1 -- 172.21.15.67:0/3874451726 reap_dead start 2024-03-09T09:12:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.073+0000 7f2201600640 1 -- 172.21.15.67:0/3874451726 shutdown_connections 2024-03-09T09:12:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.073+0000 7f2201600640 1 -- 172.21.15.67:0/3874451726 >> 172.21.15.67:0/3874451726 conn(0x7f21fc0c7370 msgr2=0x7f21fc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:07.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.073+0000 7f2201600640 1 -- 172.21.15.67:0/3874451726 shutdown_connections 2024-03-09T09:12:07.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.073+0000 7f2201600640 1 -- 172.21.15.67:0/3874451726 wait complete. 2024-03-09T09:12:07.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.075+0000 7f2201600640 1 Processor -- start 2024-03-09T09:12:07.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.075+0000 7f2201600640 1 -- start start 2024-03-09T09:12:07.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.076+0000 7f2201600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e0003b70 0x7f21fc144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.076+0000 7f2201600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21fc153ac0 0x7f21fc14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.076+0000 7f2201600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21fc1eec90 0x7f21fc146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.076+0000 7f2201600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0acd80 con 0x7f21fc153ac0 2024-03-09T09:12:07.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.076+0000 7f2201600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0c3740 con 0x7f21e0003b70 2024-03-09T09:12:07.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.076+0000 7f2201600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0acf00 con 0x7f21fc1eec90 2024-03-09T09:12:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.077+0000 7f21fa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21fc153ac0 0x7f21fc14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.077+0000 7f21fa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21fc153ac0 0x7f21fc14cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33420/0 (socket says 172.21.15.67:33420) 2024-03-09T09:12:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.077+0000 7f21fa000640 1 -- 172.21.15.67:0/1521470731 learned_addr learned my addr 172.21.15.67:0/1521470731 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.077+0000 7f21fb400640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21fc1eec90 0x7f21fc146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.077+0000 7f21fa000640 1 -- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21fc1eec90 msgr2=0x7f21fc146240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.077+0000 7f21fa000640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21fc1eec90 0x7f21fc146240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.077+0000 7f21fa000640 1 -- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e0003b70 msgr2=0x7f21fc144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.077+0000 7f21fa000640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e0003b70 0x7f21fc144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.077+0000 7f21fa000640 1 -- 172.21.15.67:0/1521470731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc14c2e0 con 0x7f21fc153ac0 2024-03-09T09:12:07.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.079+0000 7f21fa000640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21fc153ac0 0x7f21fc14cd80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009fd0 tx=0x7f21e800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.079+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8017070 con 0x7f21fc153ac0 2024-03-09T09:12:07.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.079+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e800c8c0 con 0x7f21fc153ac0 2024-03-09T09:12:07.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.079+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e800ccb0 con 0x7f21fc153ac0 2024-03-09T09:12:07.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.079+0000 7f2201600640 1 -- 172.21.15.67:0/1521470731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21fc0ac0c0 con 0x7f21fc153ac0 2024-03-09T09:12:07.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.080+0000 7f2201600640 1 -- 172.21.15.67:0/1521470731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc14c580 con 0x7f21fc153ac0 2024-03-09T09:12:07.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.083+0000 7f2201600640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc1ef070 0x7f21fc0c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.084+0000 7f2201600640 1 -- 172.21.15.67:0/1521470731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc075a00 con 0x7f21fc1ef070 2024-03-09T09:12:07.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.085+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f21e801a070 con 0x7f21fc153ac0 2024-03-09T09:12:07.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.085+0000 7f21faa00640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc1ef070 0x7f21fc0c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.086+0000 7f21f2a00640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f21dc06b550 0x7f21dc06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.086+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f21e808a550 con 0x7f21fc153ac0 2024-03-09T09:12:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.087+0000 7f21fb400640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f21dc06b550 0x7f21dc06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.088+0000 7f21faa00640 1 -- 172.21.15.67:0/1521470731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21fc0c31e0 con 0x7f21fc1ef070 2024-03-09T09:12:07.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.089+0000 7f21fb400640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f21dc06b550 0x7f21dc06da10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f21fc071570 tx=0x7f21f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.090+0000 7f21faa00640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc1ef070 0x7f21fc0c07a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f21ec034000 tx=0x7f21ec001970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.091+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec001670 con 0x7f21fc1ef070 2024-03-09T09:12:07.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.092+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f21ec030040 con 0x7f21fc1ef070 2024-03-09T09:12:07.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.092+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc1ef070 msgr2=0x7f21fc0c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.092+0000 7f21f2a00640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc1ef070 0x7f21fc0c07a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f21ec034000 tx=0x7f21ec001970 comp rx=0 tx=0).stop 2024-03-09T09:12:07.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.092+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc1ef070 msgr2=0x7f21fc0c07a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:07.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.092+0000 7f21f2a00640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc1ef070 0x7f21fc0c07a0 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f21ec034000 tx=0x7f21ec001970 comp rx=0 tx=0).stop 2024-03-09T09:12:07.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.104+0000 7f2201600640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc146780 0x7f21fc14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.104+0000 7f2201600640 1 -- 172.21.15.67:0/1521470731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc067dd0 con 0x7f21fc146780 2024-03-09T09:12:07.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.105+0000 7f21faa00640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc146780 0x7f21fc14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.105+0000 7f21faa00640 1 -- 172.21.15.67:0/1521470731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f21fc068510 con 0x7f21fc146780 2024-03-09T09:12:07.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.106+0000 7f21faa00640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc146780 0x7f21fc14f9e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0680b0 tx=0x7f21ec02f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.106+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0075b0 con 0x7f21fc146780 2024-03-09T09:12:07.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f21ec0075b0 con 0x7f21fc146780 2024-03-09T09:12:07.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc146780 msgr2=0x7f21fc14f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f21f2a00640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc146780 0x7f21fc14f9e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0680b0 tx=0x7f21ec02f000 comp rx=0 tx=0).stop 2024-03-09T09:12:07.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f21f2a00640 1 -- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc146780 msgr2=0x7f21fc14f9e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:07.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f21f2a00640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21fc146780 0x7f21fc14f9e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.112 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:12:07.112 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:12:07.112 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:12:07.112 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:12:07.112 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 142, 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:07.113 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:07.114 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:12:07.115 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:07.116 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:12:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:12:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:12:07.123 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:12:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f2201600640 1 -- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f21dc06b550 msgr2=0x7f21dc06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f2201600640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f21dc06b550 0x7f21dc06da10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f21fc071570 tx=0x7f21f4009040 comp rx=0 tx=0).stop 2024-03-09T09:12:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f2201600640 1 -- 172.21.15.67:0/1521470731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21fc153ac0 msgr2=0x7f21fc14cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f2201600640 1 --2- 172.21.15.67:0/1521470731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f21fc153ac0 0x7f21fc14cd80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009fd0 tx=0x7f21e800ed40 comp rx=0 tx=0).stop 2024-03-09T09:12:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f21fb400640 1 -- 172.21.15.67:0/1521470731 reap_dead start 2024-03-09T09:12:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f21fb400640 1 -- 172.21.15.67:0/1521470731 reap_dead start 2024-03-09T09:12:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f2201600640 1 -- 172.21.15.67:0/1521470731 shutdown_connections 2024-03-09T09:12:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f2201600640 1 -- 172.21.15.67:0/1521470731 >> 172.21.15.67:0/1521470731 conn(0x7f21fc0c7370 msgr2=0x7f21fc154090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.107+0000 7f2201600640 1 -- 172.21.15.67:0/1521470731 shutdown_connections 2024-03-09T09:12:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.108+0000 7f2201600640 1 -- 172.21.15.67:0/1521470731 wait complete. 2024-03-09T09:12:07.134 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:12:07.135 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:12:07.135 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:12:07.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.343+0000 7f2709800640 1 Processor -- start 2024-03-09T09:12:07.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.343+0000 7f2709800640 1 -- start start 2024-03-09T09:12:07.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.344+0000 7f2709800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040c1d20 0x7f27040c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.344+0000 7f2709800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2704067410 con 0x7f2704159580 2024-03-09T09:12:07.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.344+0000 7f2709800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27040c3740 con 0x7f27040c5890 2024-03-09T09:12:07.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.344+0000 7f2709800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270415a070 con 0x7f27040c1d20 2024-03-09T09:12:07.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.345+0000 7f2703e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2704159580 0x7f2704159960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40014/0 (socket says 172.21.15.67:40014) 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.345+0000 7f2703e00640 1 -- 172.21.15.67:0/1815626254 learned_addr learned my addr 172.21.15.67:0/1815626254 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.345+0000 7f2703400640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040c1d20 0x7f27040c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.345+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612370540 0 0) 0x7f26f4001110 con 0x7f2704159580 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.345+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e8003520 con 0x7f2704159580 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1212559612 0 0) 0x7f26e8003520 con 0x7f2704159580 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040c1d20 msgr2=0x7f27040c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040c1d20 0x7f27040c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 >> v1:172.21.15.134:6789/0 conn(0x7f27040c5890 legacy=0x7f27040c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27040c0f60 con 0x7f2704159580 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f26f4001840 con 0x7f2704159580 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 >> v1:172.21.15.67:6789/0 conn(0x7f2704159580 legacy=0x7f2704159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26e8003a50 0x7f26e8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040c1d20 0x7f26e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26e8004d40 0x7f26e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e800ba90 con 0x7f26e8004d40 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e800bc40 con 0x7f26e8003a50 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e800be40 con 0x7f27040c1d20 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2708e00640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26e8004d40 0x7f26e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2703e00640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26e8003a50 0x7f26e8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.346+0000 7f2703400640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040c1d20 0x7f26e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.347+0000 7f2708e00640 1 -- 172.21.15.67:0/1815626254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040c1d20 msgr2=0x7f26e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.347+0000 7f2708e00640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27040c1d20 0x7f26e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.347+0000 7f2708e00640 1 -- 172.21.15.67:0/1815626254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26e8003a50 msgr2=0x7f26e8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.347+0000 7f2708e00640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26e8003a50 0x7f26e8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.347+0000 7f2708e00640 1 -- 172.21.15.67:0/1815626254 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8003520 con 0x7f26e8004d40 2024-03-09T09:12:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.348+0000 7f2708e00640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26e8004d40 0x7f26e80051c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f26fc002130 tx=0x7f26e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c8328fbbc1cce4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.348+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc039070 con 0x7f26e8004d40 2024-03-09T09:12:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.348+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26fc02c8c0 con 0x7f26e8004d40 2024-03-09T09:12:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.348+0000 7f2702a00640 1 -- 172.21.15.67:0/1815626254 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc001d00 con 0x7f26e8004d40 2024-03-09T09:12:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.348+0000 7f2709800640 1 -- 172.21.15.67:0/1815626254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26e8004d40 msgr2=0x7f26e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.349+0000 7f2709800640 1 --2- 172.21.15.67:0/1815626254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26e8004d40 0x7f26e80051c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f26fc002130 tx=0x7f26e8005830 comp rx=0 tx=0).stop 2024-03-09T09:12:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.349+0000 7f2708e00640 1 -- 172.21.15.67:0/1815626254 reap_dead start 2024-03-09T09:12:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.349+0000 7f2709800640 1 -- 172.21.15.67:0/1815626254 shutdown_connections 2024-03-09T09:12:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.349+0000 7f2709800640 1 -- 172.21.15.67:0/1815626254 >> 172.21.15.67:0/1815626254 conn(0x7f27040a9740 msgr2=0x7f27040a9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.349+0000 7f2709800640 1 -- 172.21.15.67:0/1815626254 shutdown_connections 2024-03-09T09:12:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.349+0000 7f2709800640 1 -- 172.21.15.67:0/1815626254 wait complete. 2024-03-09T09:12:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.350+0000 7f2709800640 1 Processor -- start 2024-03-09T09:12:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.351+0000 7f2709800640 1 -- start start 2024-03-09T09:12:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.351+0000 7f2709800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26e8004d40 0x7f27041ef100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2709800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2704154210 0x7f2704147920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2709800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2704147e60 0x7f2704072690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2709800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27040c1520 con 0x7f2704154210 2024-03-09T09:12:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2709800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27040c1890 con 0x7f2704147e60 2024-03-09T09:12:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2709800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27040c6aa0 con 0x7f26e8004d40 2024-03-09T09:12:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2703400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2704154210 0x7f2704147920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2703400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2704154210 0x7f2704147920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33426/0 (socket says 172.21.15.67:33426) 2024-03-09T09:12:07.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2703400640 1 -- 172.21.15.67:0/1409062477 learned_addr learned my addr 172.21.15.67:0/1409062477 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2708e00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2704147e60 0x7f2704072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2703e00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26e8004d40 0x7f27041ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2703400640 1 -- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26e8004d40 msgr2=0x7f27041ef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2703400640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f26e8004d40 0x7f27041ef100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2703400640 1 -- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2704147e60 msgr2=0x7f2704072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2703400640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2704147e60 0x7f2704072690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.352+0000 7f2703400640 1 -- 172.21.15.67:0/1409062477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2704154b00 con 0x7f2704154210 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.353+0000 7f2703400640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2704154210 0x7f2704147920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f26f0011630 tx=0x7f26f0011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.353+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000f920 con 0x7f2704154210 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.353+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f000fac0 con 0x7f2704154210 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.353+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000fe70 con 0x7f2704154210 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.353+0000 7f2709800640 1 -- 172.21.15.67:0/1409062477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2704154820 con 0x7f2704154210 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.353+0000 7f2709800640 1 -- 172.21.15.67:0/1409062477 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27041efa20 con 0x7f2704154210 2024-03-09T09:12:07.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.354+0000 7f2709800640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27041ef640 0x7f2704072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.354+0000 7f2709800640 1 -- 172.21.15.67:0/1409062477 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27041569a0 con 0x7f27041ef640 2024-03-09T09:12:07.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.355+0000 7f2703e00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27041ef640 0x7f2704072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.356+0000 7f2703e00640 1 -- 172.21.15.67:0/1409062477 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f270404f3c0 con 0x7f27041ef640 2024-03-09T09:12:07.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.356+0000 7f2703e00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27041ef640 0x7f2704072bd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f26f4004d80 tx=0x7f26f4007430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.357+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f26f0015460 con 0x7f2704154210 2024-03-09T09:12:07.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.358+0000 7f26fbe00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26e406b4d0 0x7f26e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.358+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f26f008af50 con 0x7f2704154210 2024-03-09T09:12:07.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.359+0000 7f2708e00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26e406b4d0 0x7f26e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.361+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4010960 con 0x7f27041ef640 2024-03-09T09:12:07.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.361+0000 7f2708e00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26e406b4d0 0x7f26e406d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f26fc036000 tx=0x7f26fc009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.361+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f26f4019070 con 0x7f27041ef640 2024-03-09T09:12:07.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.361+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27041ef640 msgr2=0x7f2704072bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.361+0000 7f26fbe00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27041ef640 0x7f2704072bd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f26f4004d80 tx=0x7f26f4007430 comp rx=0 tx=0).stop 2024-03-09T09:12:07.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.361+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27041ef640 msgr2=0x7f2704072bd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:07.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.361+0000 7f26fbe00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27041ef640 0x7f2704072bd0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f26f4004d80 tx=0x7f26f4007430 comp rx=0 tx=0).stop 2024-03-09T09:12:07.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.375+0000 7f2709800640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2704073110 0x7f27041eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.375+0000 7f2709800640 1 -- 172.21.15.67:0/1409062477 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27040a8290 con 0x7f2704073110 2024-03-09T09:12:07.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.377+0000 7f2703e00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2704073110 0x7f27041eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.377+0000 7f2703e00640 1 -- 172.21.15.67:0/1409062477 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2704073d80 con 0x7f2704073110 2024-03-09T09:12:07.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.378+0000 7f2703e00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2704073110 0x7f27041eb9b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f26f4004d80 tx=0x7f26f4007aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.378+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f401d820 con 0x7f2704073110 2024-03-09T09:12:07.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.379+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f26f401d820 con 0x7f2704073110 2024-03-09T09:12:07.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.379+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2704073110 msgr2=0x7f27041eb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.379+0000 7f26fbe00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2704073110 0x7f27041eb9b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f26f4004d80 tx=0x7f26f4007aa0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.379+0000 7f26fbe00640 1 -- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2704073110 msgr2=0x7f27041eb9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:07.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.379+0000 7f26fbe00640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2704073110 0x7f27041eb9b0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f26f4004d80 tx=0x7f26f4007aa0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.388 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:12:07.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:12:07.388 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:12:07.388 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 142, 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:07.389 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:07.390 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:07.390 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:07.390 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:07.390 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:07.390 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:07.390 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:07.390 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:07.390 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:07.390 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:12:07.390 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:12:07.390 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:12:07.391 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:07.392 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:12:07.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:07.394 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:12:07.395 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.396 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:07.398 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:07.399 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:07.400 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:07.400 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:07.400 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:07.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:07.400 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:12:07.400 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:12:07.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.380+0000 7f2709800640 1 -- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26e406b4d0 msgr2=0x7f26e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.380+0000 7f2709800640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f26e406b4d0 0x7f26e406d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f26fc036000 tx=0x7f26fc009030 comp rx=0 tx=0).stop 2024-03-09T09:12:07.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.380+0000 7f2709800640 1 -- 172.21.15.67:0/1409062477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2704154210 msgr2=0x7f2704147920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.380+0000 7f2709800640 1 --2- 172.21.15.67:0/1409062477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2704154210 0x7f2704147920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f26f0011630 tx=0x7f26f0011af0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.380+0000 7f2708e00640 1 -- 172.21.15.67:0/1409062477 reap_dead start 2024-03-09T09:12:07.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.380+0000 7f2708e00640 1 -- 172.21.15.67:0/1409062477 reap_dead start 2024-03-09T09:12:07.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.380+0000 7f2709800640 1 -- 172.21.15.67:0/1409062477 shutdown_connections 2024-03-09T09:12:07.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.380+0000 7f2709800640 1 -- 172.21.15.67:0/1409062477 >> 172.21.15.67:0/1409062477 conn(0x7f27040a9740 msgr2=0x7f27040c4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:07.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.380+0000 7f2709800640 1 -- 172.21.15.67:0/1409062477 shutdown_connections 2024-03-09T09:12:07.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.380+0000 7f2709800640 1 -- 172.21.15.67:0/1409062477 wait complete. 2024-03-09T09:12:07.407 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:12:07.407 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T09:12:07.407 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:12:07.407 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T09:12:07.408 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:12:07.408 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:12:07.408 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:12:07.427 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:12:07.428 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:12:07.428 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:12:07.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.640+0000 7fc3bae00640 1 Processor -- start 2024-03-09T09:12:07.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.640+0000 7fc3bae00640 1 -- start start 2024-03-09T09:12:07.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.643+0000 7fc3bae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3b4153300 0x7fc3b40b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.643+0000 7fc3bae00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b4157000 con 0x7fc3b40cadb0 2024-03-09T09:12:07.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.643+0000 7fc3bae00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b40ae430 con 0x7fc3b40c5e40 2024-03-09T09:12:07.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.643+0000 7fc3bae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40ae6f0 con 0x7fc3b4153300 2024-03-09T09:12:07.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.643+0000 7fc3b9a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3b4153300 msgr2=0x7fc3b40b0000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:07.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.643+0000 7fc3b9a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3b4153300 0x7fc3b40b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:07.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.643+0000 7fc3ba400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc3b40cadb0 0x7fc3b4160490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:40030/0 (socket says 172.21.15.67:40030) 2024-03-09T09:12:07.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.643+0000 7fc3ba400640 1 -- 172.21.15.67:0/119883894 learned_addr learned my addr 172.21.15.67:0/119883894 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:12:07.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138678684 0 0) 0x7fc3b4157000 con 0x7fc3b40cadb0 2024-03-09T09:12:07.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc39c003520 con 0x7fc3b40cadb0 2024-03-09T09:12:07.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2950916969 0 0) 0x7fc39c003520 con 0x7fc3b40cadb0 2024-03-09T09:12:07.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3b4153300 msgr2=0x7fc3b40b0000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:12:07.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 --2- 172.21.15.67:0/119883894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3b4153300 0x7fc3b40b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 >> v1:172.21.15.134:6789/0 conn(0x7fc3b40c5e40 legacy=0x7fc3b4156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b4155b10 con 0x7fc3b40cadb0 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fc3ac0019f0 con 0x7fc3b40cadb0 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 >> v1:172.21.15.67:6789/0 conn(0x7fc3b40cadb0 legacy=0x7fc3b4160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 --2- 172.21.15.67:0/119883894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc39c003a50 0x7fc39c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 --2- 172.21.15.67:0/119883894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3b4153300 0x7fc39c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 --2- 172.21.15.67:0/119883894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c004e60 0x7fc39c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40ae6f0 con 0x7fc39c004e60 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c005560 con 0x7fc39c003a50 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.644+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c00bf00 con 0x7fc3b4153300 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.645+0000 7fc3b9000640 1 -- 172.21.15.67:0/119883894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3b4153300 msgr2=0x7fc39c004800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.645+0000 7fc3b9000640 1 --2- 172.21.15.67:0/119883894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3b4153300 0x7fc39c004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.645+0000 7fc3ba400640 1 --2- 172.21.15.67:0/119883894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c004e60 0x7fc39c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.645+0000 7fc3b9a00640 1 --2- 172.21.15.67:0/119883894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc39c003a50 0x7fc39c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.645+0000 7fc3ba400640 1 -- 172.21.15.67:0/119883894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3b4153300 msgr2=0x7fc39c004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.645+0000 7fc3ba400640 1 --2- 172.21.15.67:0/119883894 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3b4153300 0x7fc39c004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.645+0000 7fc3ba400640 1 -- 172.21.15.67:0/119883894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc39c003a50 msgr2=0x7fc39c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:07.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.645+0000 7fc3ba400640 1 --2- 172.21.15.67:0/119883894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc39c003a50 0x7fc39c003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.645+0000 7fc3ba400640 1 -- 172.21.15.67:0/119883894 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c003520 con 0x7fc39c004e60 2024-03-09T09:12:07.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.646+0000 7fc3ba400640 1 --2- 172.21.15.67:0/119883894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c004e60 0x7fc39c00b9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00a830 tx=0x7fc39c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d236f05ee3ad61 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.646+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac00a450 con 0x7fc39c004e60 2024-03-09T09:12:07.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.647+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac0072d0 con 0x7fc39c004e60 2024-03-09T09:12:07.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.647+0000 7fc3b3e00640 1 -- 172.21.15.67:0/119883894 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0076c0 con 0x7fc39c004e60 2024-03-09T09:12:07.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.647+0000 7fc3bae00640 1 -- 172.21.15.67:0/119883894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c004e60 msgr2=0x7fc39c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.647+0000 7fc3bae00640 1 --2- 172.21.15.67:0/119883894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c004e60 0x7fc39c00b9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00a830 tx=0x7fc39c005830 comp rx=0 tx=0).stop 2024-03-09T09:12:07.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.647+0000 7fc3ba400640 1 -- 172.21.15.67:0/119883894 reap_dead start 2024-03-09T09:12:07.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.648+0000 7fc3bae00640 1 -- 172.21.15.67:0/119883894 shutdown_connections 2024-03-09T09:12:07.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.648+0000 7fc3bae00640 1 -- 172.21.15.67:0/119883894 >> 172.21.15.67:0/119883894 conn(0x7fc3b40732d0 msgr2=0x7fc3b40ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:07.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.648+0000 7fc3bae00640 1 -- 172.21.15.67:0/119883894 shutdown_connections 2024-03-09T09:12:07.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.648+0000 7fc3bae00640 1 -- 172.21.15.67:0/119883894 wait complete. 2024-03-09T09:12:07.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.649+0000 7fc3bae00640 1 Processor -- start 2024-03-09T09:12:07.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.650+0000 7fc3bae00640 1 -- start start 2024-03-09T09:12:07.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.650+0000 7fc3bae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc39c004e60 0x7fc3b414b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.651+0000 7fc3bae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3b414c440 0x7fc3b414be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.651+0000 7fc3bae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b41ebd00 0x7fc3b41ec180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.651+0000 7fc3bae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40c59e0 con 0x7fc3b41ebd00 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.651+0000 7fc3bae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b4159e90 con 0x7fc3b414c440 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.651+0000 7fc3bae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b4156720 con 0x7fc39c004e60 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.651+0000 7fc3b9a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc39c004e60 msgr2=0x7fc3b414b910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.651+0000 7fc3b9a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc39c004e60 0x7fc3b414b910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.651+0000 7fc3ba400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b41ebd00 0x7fc3b41ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.651+0000 7fc3ba400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b41ebd00 0x7fc3b41ec180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33436/0 (socket says 172.21.15.67:33436) 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.651+0000 7fc3ba400640 1 -- 172.21.15.67:0/1136010317 learned_addr learned my addr 172.21.15.67:0/1136010317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.651+0000 7fc3b9000640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3b414c440 0x7fc3b414be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.652+0000 7fc3ba400640 1 -- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc39c004e60 msgr2=0x7fc3b414b910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.652+0000 7fc3ba400640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc39c004e60 0x7fc3b414b910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.652+0000 7fc3ba400640 1 -- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3b414c440 msgr2=0x7fc3b414be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.652+0000 7fc3ba400640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3b414c440 0x7fc3b414be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.652+0000 7fc3ba400640 1 -- 172.21.15.67:0/1136010317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b41eca60 con 0x7fc3b41ebd00 2024-03-09T09:12:07.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.652+0000 7fc3ba400640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b41ebd00 0x7fc3b41ec180 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00a920 tx=0x7fc3b41536e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.652+0000 7fc3b1600640 1 -- 172.21.15.67:0/1136010317 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac039040 con 0x7fc3b41ebd00 2024-03-09T09:12:07.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.652+0000 7fc3b1600640 1 -- 172.21.15.67:0/1136010317 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac004b60 con 0x7fc3b41ebd00 2024-03-09T09:12:07.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.652+0000 7fc3bae00640 1 -- 172.21.15.67:0/1136010317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b406c710 con 0x7fc3b41ebd00 2024-03-09T09:12:07.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.652+0000 7fc3b1600640 1 -- 172.21.15.67:0/1136010317 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac00a450 con 0x7fc3b41ebd00 2024-03-09T09:12:07.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.653+0000 7fc3bae00640 1 -- 172.21.15.67:0/1136010317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b406ccd0 con 0x7fc3b41ebd00 2024-03-09T09:12:07.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.654+0000 7fc3bae00640 1 -- 172.21.15.67:0/1136010317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b4153750 con 0x7fc3b41ebd00 2024-03-09T09:12:07.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.655+0000 7fc3b1600640 1 -- 172.21.15.67:0/1136010317 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc3ac0072d0 con 0x7fc3b41ebd00 2024-03-09T09:12:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.656+0000 7fc3b1600640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc39406b4b0 0x7fc39406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.657+0000 7fc3b1600640 1 -- 172.21.15.67:0/1136010317 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc3ac0ab9a0 con 0x7fc3b41ebd00 2024-03-09T09:12:07.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.657+0000 7fc3b9a00640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc39406b4b0 0x7fc39406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:07.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.658+0000 7fc3b9a00640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc39406b4b0 0x7fc39406d970 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8006fd0 tx=0x7fc3a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:07.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.661+0000 7fc3b1600640 1 -- 172.21.15.67:0/1136010317 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc3b4153750 con 0x7fc3b41ebd00 2024-03-09T09:12:07.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.984+0000 7fc3bae00640 1 -- 172.21.15.67:0/1136010317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc3b404f3c0 con 0x7fc3b41ebd00 2024-03-09T09:12:07.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.985+0000 7fc3b1600640 1 -- 172.21.15.67:0/1136010317 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fc3ac079dd0 con 0x7fc3b41ebd00 2024-03-09T09:12:07.990 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":142,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 -- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc39406b4b0 msgr2=0x7fc39406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc39406b4b0 0x7fc39406d970 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8006fd0 tx=0x7fc3a8008040 comp rx=0 tx=0).stop 2024-03-09T09:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 -- 172.21.15.67:0/1136010317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b41ebd00 msgr2=0x7fc3b41ec180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b41ebd00 0x7fc3b41ec180 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00a920 tx=0x7fc3b41536e0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 -- 172.21.15.67:0/1136010317 shutdown_connections 2024-03-09T09:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc39406b4b0 0x7fc39406d970 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3b41ebd00 0x7fc3b41ec180 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3b414c440 0x7fc3b414be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 --2- 172.21.15.67:0/1136010317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc39c004e60 0x7fc3b414b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 -- 172.21.15.67:0/1136010317 >> 172.21.15.67:0/1136010317 conn(0x7fc3b40732d0 msgr2=0x7fc3b4151740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 -- 172.21.15.67:0/1136010317 shutdown_connections 2024-03-09T09:12:07.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:07.989+0000 7fc3bae00640 1 -- 172.21.15.67:0/1136010317 wait complete. 2024-03-09T09:12:08.808 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 53% (132/246) Updating files: 54% (133/246) Updating files: 55% (136/246) Updating files: 56% (138/246) Updating files: 57% (141/246) Updating files: 58% (143/246) Updating files: 59% (146/246) Updating files: 60% (148/246) Updating files: 61% (151/246) Updating files: 62% (153/246) Updating files: 63% (155/246) Updating files: 64% (158/246) Updating files: 65% (160/246) Updating files: 66% (163/246) Updating files: 67% (165/246) Updating files: 68% (168/246) Updating files: 69% (170/246) Updating files: 70% (173/246) Updating files: 71% (175/246) Updating files: 72% (178/246) Updating files: 73% (180/246) Updating files: 74% (183/246) Updating files: 75% (185/246) Updating files: 76% (187/246) Updating files: 77% (190/246) Updating files: 78% (192/246) Updating files: 79% (195/246) Updating files: 80% (197/246) Updating files: 81% (200/246) Updating files: 82% (202/246) Updating files: 83% (205/246) Updating files: 84% (207/246) Updating files: 85% (210/246) Updating files: 86% (212/246) Updating files: 87% (215/246) Updating files: 88% (217/246) Updating files: 89% (219/246) Updating files: 90% (222/246) Updating files: 91% (224/246) Updating files: 92% (227/246) Updating files: 93% (229/246) Updating files: 94% (232/246) Updating files: 95% (234/246) Updating files: 96% (237/246) Updating files: 97% (239/246) Updating files: 98% (242/246) Updating files: 99% (244/246) Updating files: 100% (246/246) Updating files: 100% (246/246), done. 2024-03-09T09:12:09.556 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 1a32f0a0b42 v0.56 2024-03-09T09:12:09.570 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.56 2024-03-09T09:12:09.727 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:12:09.727 INFO:tasks.workunit.client.1.smithi134.stdout:v0.57 2024-03-09T09:12:09.727 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 57 -eq 48 ']' 2024-03-09T09:12:09.727 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.57 2024-03-09T09:12:09.727 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.57 2024-03-09T09:12:09.728 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.57 2024-03-09T09:12:11.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:12:11.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.235+0000 7fb94a400640 1 Processor -- start 2024-03-09T09:12:11.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.235+0000 7fb94a400640 1 -- start start 2024-03-09T09:12:11.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb94a400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb944156160 0x7fb9440c4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:11.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb94a400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb94406ef00 0x7fb9440c3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:11.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb94a400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb94414fd30 0x7fb9441501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb94a400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9440c31d0 con 0x7fb94414fd30 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb94a400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9440c34b0 con 0x7fb94406ef00 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb94a400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb944072080 con 0x7fb944156160 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb949000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb944156160 msgr2=0x7fb9440c4610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb949000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb944156160 0x7fb9440c4610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb949a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb94414fd30 0x7fb9441501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb949a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb94414fd30 0x7fb9441501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51982/0 (socket says 172.21.15.67:51982) 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb949a00640 1 -- 172.21.15.67:0/1884017544 learned_addr learned my addr 172.21.15.67:0/1884017544 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.237+0000 7fb943e00640 1 --2- 172.21.15.67:0/1884017544 >> v2:172.21.15.134:3300/0 conn(0x7fb94406ef00 0x7fb9440c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.238+0000 7fb949a00640 1 -- 172.21.15.67:0/1884017544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb944156160 msgr2=0x7fb9440c4610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.238+0000 7fb949a00640 1 --2- 172.21.15.67:0/1884017544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb944156160 0x7fb9440c4610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.238+0000 7fb949a00640 1 -- 172.21.15.67:0/1884017544 >> v2:172.21.15.134:3300/0 conn(0x7fb94406ef00 msgr2=0x7fb9440c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.238+0000 7fb949a00640 1 --2- 172.21.15.67:0/1884017544 >> v2:172.21.15.134:3300/0 conn(0x7fb94406ef00 0x7fb9440c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.238+0000 7fb949a00640 1 -- 172.21.15.67:0/1884017544 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb944072360 con 0x7fb94414fd30 2024-03-09T09:12:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.239+0000 7fb949a00640 1 --2- 172.21.15.67:0/1884017544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94414fd30 0x7fb9441501c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb9440c5f40 tx=0x7fb93c02e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3c5bebc27fc797 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.239+0000 7fb943400640 1 -- 172.21.15.67:0/1884017544 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c02ec20 con 0x7fb94414fd30 2024-03-09T09:12:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.239+0000 7fb943400640 1 -- 172.21.15.67:0/1884017544 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c02edc0 con 0x7fb94414fd30 2024-03-09T09:12:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.239+0000 7fb943400640 1 -- 172.21.15.67:0/1884017544 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c036610 con 0x7fb94414fd30 2024-03-09T09:12:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.240+0000 7fb94a400640 1 -- 172.21.15.67:0/1884017544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94414fd30 msgr2=0x7fb9441501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.240+0000 7fb94a400640 1 --2- 172.21.15.67:0/1884017544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94414fd30 0x7fb9441501c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb9440c5f40 tx=0x7fb93c02e080 comp rx=0 tx=0).stop 2024-03-09T09:12:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.240+0000 7fb94a400640 1 -- 172.21.15.67:0/1884017544 shutdown_connections 2024-03-09T09:12:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.240+0000 7fb94a400640 1 --2- 172.21.15.67:0/1884017544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94414fd30 0x7fb9441501c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.240+0000 7fb94a400640 1 --2- 172.21.15.67:0/1884017544 >> v2:172.21.15.134:3300/0 conn(0x7fb94406ef00 0x7fb9440c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.240+0000 7fb94a400640 1 --2- 172.21.15.67:0/1884017544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb944156160 0x7fb9440c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:11.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.240+0000 7fb94a400640 1 -- 172.21.15.67:0/1884017544 >> 172.21.15.67:0/1884017544 conn(0x7fb9440c8380 msgr2=0x7fb9440c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:11.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.240+0000 7fb94a400640 1 -- 172.21.15.67:0/1884017544 shutdown_connections 2024-03-09T09:12:11.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.240+0000 7fb94a400640 1 -- 172.21.15.67:0/1884017544 wait complete. 2024-03-09T09:12:11.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.241+0000 7fb94a400640 1 Processor -- start 2024-03-09T09:12:11.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.242+0000 7fb94a400640 1 -- start start 2024-03-09T09:12:11.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb94a400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb94406ef00 0x7fb94414aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:11.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb94a400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb94414fd30 0x7fb9440ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb94a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944156160 0x7fb94413aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb94a400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440c3e60 con 0x7fb944156160 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb94a400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb944156b00 con 0x7fb94414fd30 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb94a400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94406f360 con 0x7fb94406ef00 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb949000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb94406ef00 msgr2=0x7fb94414aa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb949000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb94406ef00 0x7fb94414aa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb949a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944156160 0x7fb94413aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb949a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944156160 0x7fb94413aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51988/0 (socket says 172.21.15.67:51988) 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb949a00640 1 -- 172.21.15.67:0/3959969617 learned_addr learned my addr 172.21.15.67:0/3959969617 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb943e00640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb94414fd30 0x7fb9440ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb949a00640 1 -- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb94406ef00 msgr2=0x7fb94414aa50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb949a00640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb94406ef00 0x7fb94414aa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb949a00640 1 -- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb94414fd30 msgr2=0x7fb9440ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:11.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb949a00640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb94414fd30 0x7fb9440ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:11.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.243+0000 7fb949a00640 1 -- 172.21.15.67:0/3959969617 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94414b1f0 con 0x7fb944156160 2024-03-09T09:12:11.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.244+0000 7fb949a00640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944156160 0x7fb94413aa50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb93c038040 tx=0x7fb93c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:11.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.244+0000 7fb940c00640 1 -- 172.21.15.67:0/3959969617 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c009510 con 0x7fb944156160 2024-03-09T09:12:11.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.244+0000 7fb940c00640 1 -- 172.21.15.67:0/3959969617 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c044070 con 0x7fb944156160 2024-03-09T09:12:11.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.244+0000 7fb940c00640 1 -- 172.21.15.67:0/3959969617 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c02ec20 con 0x7fb944156160 2024-03-09T09:12:11.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.244+0000 7fb94a400640 1 -- 172.21.15.67:0/3959969617 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb944151b60 con 0x7fb944156160 2024-03-09T09:12:11.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.244+0000 7fb94a400640 1 -- 172.21.15.67:0/3959969617 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9440abbe0 con 0x7fb944156160 2024-03-09T09:12:11.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.246+0000 7fb94a400640 1 -- 172.21.15.67:0/3959969617 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94404f3c0 con 0x7fb944156160 2024-03-09T09:12:11.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.246+0000 7fb940c00640 1 -- 172.21.15.67:0/3959969617 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb93c03f6b0 con 0x7fb944156160 2024-03-09T09:12:11.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.247+0000 7fb940c00640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb92406b480 0x7fb92406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:11.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.247+0000 7fb940c00640 1 -- 172.21.15.67:0/3959969617 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb93c0ac880 con 0x7fb944156160 2024-03-09T09:12:11.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.248+0000 7fb949000640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb92406b480 0x7fb92406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:11.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.249+0000 7fb949000640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb92406b480 0x7fb92406d940 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb930006fd0 tx=0x7fb930006d80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:11.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.257+0000 7fb940c00640 1 -- 172.21.15.67:0/3959969617 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb93c07b4c0 con 0x7fb944156160 2024-03-09T09:12:11.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.536+0000 7fb94a400640 1 -- 172.21.15.67:0/3959969617 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb94414ea30 con 0x7fb944156160 2024-03-09T09:12:11.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.537+0000 7fb940c00640 1 -- 172.21.15.67:0/3959969617 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fb93c07adc0 con 0x7fb944156160 2024-03-09T09:12:11.542 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":142,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.539+0000 7fb94a400640 1 -- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb92406b480 msgr2=0x7fb92406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.539+0000 7fb94a400640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb92406b480 0x7fb92406d940 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb930006fd0 tx=0x7fb930006d80 comp rx=0 tx=0).stop 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.539+0000 7fb94a400640 1 -- 172.21.15.67:0/3959969617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944156160 msgr2=0x7fb94413aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.539+0000 7fb94a400640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944156160 0x7fb94413aa50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb93c038040 tx=0x7fb93c004a40 comp rx=0 tx=0).stop 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.540+0000 7fb94a400640 1 -- 172.21.15.67:0/3959969617 shutdown_connections 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.540+0000 7fb94a400640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb92406b480 0x7fb92406d940 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.540+0000 7fb94a400640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb944156160 0x7fb94413aa50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.540+0000 7fb94a400640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb94414fd30 0x7fb9440ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.540+0000 7fb94a400640 1 --2- 172.21.15.67:0/3959969617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb94406ef00 0x7fb94414aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.540+0000 7fb94a400640 1 -- 172.21.15.67:0/3959969617 >> 172.21.15.67:0/3959969617 conn(0x7fb9440c8380 msgr2=0x7fb94414e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.540+0000 7fb94a400640 1 -- 172.21.15.67:0/3959969617 shutdown_connections 2024-03-09T09:12:11.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:11.540+0000 7fb94a400640 1 -- 172.21.15.67:0/3959969617 wait complete. 2024-03-09T09:12:13.821 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 35% (94/268) Updating files: 36% (97/268) Updating files: 37% (100/268) Updating files: 38% (102/268) Updating files: 39% (105/268) Updating files: 40% (108/268) Updating files: 41% (110/268) Updating files: 42% (113/268) Updating files: 43% (116/268) Updating files: 44% (118/268) Updating files: 45% (121/268) Updating files: 46% (124/268) Updating files: 47% (126/268) Updating files: 48% (129/268) Updating files: 49% (132/268) Updating files: 50% (134/268) Updating files: 51% (137/268) Updating files: 52% (140/268) Updating files: 53% (143/268) Updating files: 54% (145/268) Updating files: 55% (148/268) Updating files: 56% (151/268) Updating files: 57% (153/268) Updating files: 58% (156/268) Updating files: 59% (159/268) Updating files: 60% (161/268) Updating files: 61% (164/268) Updating files: 62% (167/268) Updating files: 63% (169/268) Updating files: 64% (172/268) Updating files: 65% (175/268) Updating files: 66% (177/268) Updating files: 67% (180/268) Updating files: 68% (183/268) Updating files: 69% (185/268) Updating files: 70% (188/268) Updating files: 71% (191/268) Updating files: 72% (193/268) Updating files: 73% (196/268) Updating files: 74% (199/268) Updating files: 75% (201/268) Updating files: 76% (204/268) Updating files: 77% (207/268) Updating files: 78% (210/268) Updating files: 79% (212/268) Updating files: 80% (215/268) Updating files: 81% (218/268) Updating files: 82% (220/268) Updating files: 83% (223/268) Updating files: 84% (226/268) Updating files: 85% (228/268) Updating files: 86% (231/268) Updating files: 87% (234/268) Updating files: 88% (236/268) Updating files: 89% (239/268) Updating files: 90% (242/268) Updating files: 91% (244/268) Updating files: 92% (247/268) Updating files: 93% (250/268) Updating files: 94% (252/268) Updating files: 95% (255/268) Updating files: 96% (258/268) Updating files: 97% (260/268) Updating files: 97% (262/268) Updating files: 98% (263/268) Updating files: 99% (266/268) Updating files: 100% (268/268) Updating files: 100% (268/268), done. 2024-03-09T09:12:13.899 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at ba3f91e7504 v0.58 2024-03-09T09:12:13.921 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.58 2024-03-09T09:12:14.062 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:12:14.063 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 59 -eq 48 ']' 2024-03-09T09:12:14.063 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.59 2024-03-09T09:12:14.063 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.59 2024-03-09T09:12:14.063 INFO:tasks.workunit.client.0.smithi067.stdout:v0.59 2024-03-09T09:12:14.063 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.59 2024-03-09T09:12:14.563 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:12:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.801+0000 7fdf6a600640 1 Processor -- start 2024-03-09T09:12:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.801+0000 7fdf6a600640 1 -- start start 2024-03-09T09:12:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.803+0000 7fdf6a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdf640710a0 0x7fdf64153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.803+0000 7fdf6a600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdf64151360 0x7fdf64153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.803+0000 7fdf6a600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf640c81e0 con 0x7fdf640c1f70 2024-03-09T09:12:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.803+0000 7fdf6a600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640c32b0 con 0x7fdf640710a0 2024-03-09T09:12:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.803+0000 7fdf6a600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640c2fd0 con 0x7fdf64151360 2024-03-09T09:12:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.803+0000 7fdf63e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdf640710a0 0x7fdf64153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.803+0000 7fdf63e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdf640710a0 0x7fdf64153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51998/0 (socket says 172.21.15.67:51998) 2024-03-09T09:12:14.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.803+0000 7fdf63e00640 1 -- 172.21.15.67:0/2985596151 learned_addr learned my addr 172.21.15.67:0/2985596151 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.804+0000 7fdf69c00640 1 --2- 172.21.15.67:0/2985596151 >> v2:172.21.15.134:3300/0 conn(0x7fdf64151360 0x7fdf64153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.804+0000 7fdf63e00640 1 -- 172.21.15.67:0/2985596151 >> v2:172.21.15.134:3300/0 conn(0x7fdf64151360 msgr2=0x7fdf64153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.804+0000 7fdf63e00640 1 --2- 172.21.15.67:0/2985596151 >> v2:172.21.15.134:3300/0 conn(0x7fdf64151360 0x7fdf64153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.804+0000 7fdf63e00640 1 -- 172.21.15.67:0/2985596151 >> v1:172.21.15.67:6789/0 conn(0x7fdf640c1f70 legacy=0x7fdf64150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.804+0000 7fdf63e00640 1 -- 172.21.15.67:0/2985596151 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640c3700 con 0x7fdf640710a0 2024-03-09T09:12:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.804+0000 7fdf69c00640 1 --2- 172.21.15.67:0/2985596151 >> v2:172.21.15.134:3300/0 conn(0x7fdf64151360 0x7fdf64153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:12:14.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.805+0000 7fdf63e00640 1 --2- 172.21.15.67:0/2985596151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf640710a0 0x7fdf64153970 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdf64073ef0 tx=0x7fdf5002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dfbd1b847475d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:14.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.806+0000 7fdf63400640 1 -- 172.21.15.67:0/2985596151 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50035020 con 0x7fdf640710a0 2024-03-09T09:12:14.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.806+0000 7fdf63400640 1 -- 172.21.15.67:0/2985596151 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf50031870 con 0x7fdf640710a0 2024-03-09T09:12:14.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.806+0000 7fdf63400640 1 -- 172.21.15.67:0/2985596151 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50031c40 con 0x7fdf640710a0 2024-03-09T09:12:14.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.806+0000 7fdf6a600640 1 -- 172.21.15.67:0/2985596151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf640710a0 msgr2=0x7fdf64153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:14.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.806+0000 7fdf6a600640 1 --2- 172.21.15.67:0/2985596151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf640710a0 0x7fdf64153970 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdf64073ef0 tx=0x7fdf5002b910 comp rx=0 tx=0).stop 2024-03-09T09:12:14.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.807+0000 7fdf6a600640 1 -- 172.21.15.67:0/2985596151 shutdown_connections 2024-03-09T09:12:14.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.807+0000 7fdf6a600640 1 --2- 172.21.15.67:0/2985596151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf640710a0 0x7fdf64153970 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:14.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.807+0000 7fdf6a600640 1 --2- 172.21.15.67:0/2985596151 >> v2:172.21.15.134:3300/0 conn(0x7fdf64151360 0x7fdf64153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:14.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.807+0000 7fdf6a600640 1 -- 172.21.15.67:0/2985596151 >> 172.21.15.67:0/2985596151 conn(0x7fdf640c7300 msgr2=0x7fdf640c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:14.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.807+0000 7fdf6a600640 1 -- 172.21.15.67:0/2985596151 shutdown_connections 2024-03-09T09:12:14.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.807+0000 7fdf6a600640 1 -- 172.21.15.67:0/2985596151 wait complete. 2024-03-09T09:12:14.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.808+0000 7fdf6a600640 1 Processor -- start 2024-03-09T09:12:14.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.809+0000 7fdf6a600640 1 -- start start 2024-03-09T09:12:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.809+0000 7fdf6a600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf640710a0 0x7fdf640c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.809+0000 7fdf6a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf640c1f70 0x7fdf64157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.809+0000 7fdf6a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf64151360 0x7fdf64157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf6a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf64070d50 con 0x7fdf64151360 2024-03-09T09:12:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf6a600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf64070830 con 0x7fdf640c1f70 2024-03-09T09:12:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf6a600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf64070ed0 con 0x7fdf640710a0 2024-03-09T09:12:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf69200640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf640710a0 msgr2=0x7fdf640c6ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf69200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf640710a0 0x7fdf640c6ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf69c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf64151360 0x7fdf64157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf69c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf64151360 0x7fdf64157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52002/0 (socket says 172.21.15.67:52002) 2024-03-09T09:12:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf69c00640 1 -- 172.21.15.67:0/663459774 learned_addr learned my addr 172.21.15.67:0/663459774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf63e00640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf640c1f70 0x7fdf64157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf69c00640 1 -- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf640710a0 msgr2=0x7fdf640c6ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:12:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf69c00640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf640710a0 0x7fdf640c6ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf69c00640 1 -- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf640c1f70 msgr2=0x7fdf64157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf69c00640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf640c1f70 0x7fdf64157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.810+0000 7fdf69c00640 1 -- 172.21.15.67:0/663459774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64156a50 con 0x7fdf64151360 2024-03-09T09:12:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.811+0000 7fdf69c00640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf64151360 0x7fdf64157770 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00ced0 tx=0x7fdf5c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.811+0000 7fdf60c00640 1 -- 172.21.15.67:0/663459774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c003b90 con 0x7fdf64151360 2024-03-09T09:12:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.811+0000 7fdf60c00640 1 -- 172.21.15.67:0/663459774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c003d50 con 0x7fdf64151360 2024-03-09T09:12:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.811+0000 7fdf60c00640 1 -- 172.21.15.67:0/663459774 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c004260 con 0x7fdf64151360 2024-03-09T09:12:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.811+0000 7fdf6a600640 1 -- 172.21.15.67:0/663459774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf64155ba0 con 0x7fdf64151360 2024-03-09T09:12:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.811+0000 7fdf6a600640 1 -- 172.21.15.67:0/663459774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf640c7010 con 0x7fdf64151360 2024-03-09T09:12:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.812+0000 7fdf6a600640 1 -- 172.21.15.67:0/663459774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf64156860 con 0x7fdf64151360 2024-03-09T09:12:14.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.813+0000 7fdf60c00640 1 -- 172.21.15.67:0/663459774 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdf5c004400 con 0x7fdf64151360 2024-03-09T09:12:14.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.815+0000 7fdf60c00640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf4406b4b0 0x7fdf4406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:14.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.815+0000 7fdf60c00640 1 -- 172.21.15.67:0/663459774 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdf5c08aed0 con 0x7fdf64151360 2024-03-09T09:12:14.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.816+0000 7fdf69200640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf4406b4b0 0x7fdf4406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:14.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.817+0000 7fdf69200640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf4406b4b0 0x7fdf4406d970 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdf58000c00 tx=0x7fdf58007a10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:14.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:14.820+0000 7fdf60c00640 1 -- 172.21.15.67:0/663459774 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdf5c059ab0 con 0x7fdf64151360 2024-03-09T09:12:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.138+0000 7fdf6a600640 1 -- 172.21.15.67:0/663459774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdf6404f3c0 con 0x7fdf64151360 2024-03-09T09:12:15.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.138+0000 7fdf60c00640 1 -- 172.21.15.67:0/663459774 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fdf5c0593b0 con 0x7fdf64151360 2024-03-09T09:12:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":142,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:12:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 -- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf4406b4b0 msgr2=0x7fdf4406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf4406b4b0 0x7fdf4406d970 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdf58000c00 tx=0x7fdf58007a10 comp rx=0 tx=0).stop 2024-03-09T09:12:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 -- 172.21.15.67:0/663459774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf64151360 msgr2=0x7fdf64157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf64151360 0x7fdf64157770 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00ced0 tx=0x7fdf5c00a710 comp rx=0 tx=0).stop 2024-03-09T09:12:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 -- 172.21.15.67:0/663459774 shutdown_connections 2024-03-09T09:12:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdf4406b4b0 0x7fdf4406d970 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf64151360 0x7fdf64157770 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:15.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf640c1f70 0x7fdf64157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:15.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 --2- 172.21.15.67:0/663459774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf640710a0 0x7fdf640c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:15.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 -- 172.21.15.67:0/663459774 >> 172.21.15.67:0/663459774 conn(0x7fdf640c7300 msgr2=0x7fdf640c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:15.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 -- 172.21.15.67:0/663459774 shutdown_connections 2024-03-09T09:12:15.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:15.141+0000 7fdf6a600640 1 -- 172.21.15.67:0/663459774 wait complete. 2024-03-09T09:12:15.366 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 38% (118/304) Updating files: 39% (119/304) Updating files: 40% (122/304) Updating files: 41% (125/304) Updating files: 42% (128/304) Updating files: 43% (131/304) Updating files: 44% (134/304) Updating files: 45% (137/304) Updating files: 46% (140/304) Updating files: 47% (143/304) Updating files: 48% (146/304) Updating files: 49% (149/304) Updating files: 49% (150/304) Updating files: 50% (152/304) Updating files: 51% (156/304) Updating files: 52% (159/304) Updating files: 53% (162/304) Updating files: 54% (165/304) Updating files: 55% (168/304) Updating files: 56% (171/304) Updating files: 56% (173/304) Updating files: 57% (174/304) Updating files: 58% (177/304) Updating files: 59% (180/304) Updating files: 60% (183/304) Updating files: 61% (186/304) Updating files: 62% (189/304) Updating files: 63% (192/304) Updating files: 64% (195/304) Updating files: 65% (198/304) Updating files: 66% (201/304) Updating files: 67% (204/304) Updating files: 68% (207/304) Updating files: 69% (210/304) Updating files: 70% (213/304) Updating files: 71% (216/304) Updating files: 72% (219/304) Updating files: 73% (222/304) Updating files: 74% (225/304) Updating files: 75% (228/304) Updating files: 75% (230/304) Updating files: 76% (232/304) Updating files: 77% (235/304) Updating files: 78% (238/304) Updating files: 79% (241/304) Updating files: 80% (244/304) Updating files: 81% (247/304) Updating files: 82% (250/304) Updating files: 83% (253/304) Updating files: 84% (256/304) Updating files: 85% (259/304) Updating files: 86% (262/304) Updating files: 87% (265/304) Updating files: 87% (266/304) Updating files: 88% (268/304) Updating files: 89% (271/304) Updating files: 90% (274/304) Updating files: 91% (277/304) Updating files: 92% (280/304) Updating files: 93% (283/304) Updating files: 94% (286/304) Updating files: 95% (289/304) Updating files: 96% (292/304) Updating files: 97% (295/304) Updating files: 98% (298/304) Updating files: 99% (301/304) Updating files: 100% (304/304) Updating files: 100% (304/304), done. 2024-03-09T09:12:15.455 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 9a7a9d06c06 v0.57 2024-03-09T09:12:15.469 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.57 2024-03-09T09:12:15.744 INFO:tasks.workunit.client.1.smithi134.stdout:v0.58 2024-03-09T09:12:15.745 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:12:15.745 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 58 -eq 48 ']' 2024-03-09T09:12:15.745 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.58 2024-03-09T09:12:15.745 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.58 2024-03-09T09:12:15.745 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.58 2024-03-09T09:12:17.700 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:12:17.708 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:12:17.926 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-03-09T09:12:17.920+0000 7efd53a00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 47460) UID: 0 2024-03-09T09:12:17.929 INFO:tasks.ceph.mds.d.smithi134.stderr:2024-03-09T09:12:17.922+0000 7f9245e00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 47460) UID: 0 2024-03-09T09:12:18.166 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:12:18.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.403+0000 7f9975c00640 1 Processor -- start 2024-03-09T09:12:18.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.403+0000 7f9975c00640 1 -- start start 2024-03-09T09:12:18.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.404+0000 7f9975c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.405+0000 7f9975c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.405+0000 7f9975c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99701581a0 con 0x7f9970157650 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.405+0000 7f9975c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9970070ee0 con 0x7f99700ac730 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.405+0000 7f9975c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9970154d00 con 0x7f99700c22a0 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.405+0000 7f996ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.405+0000 7f996ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52016/0 (socket says 172.21.15.67:52016) 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.405+0000 7f996ea00640 1 -- 172.21.15.67:0/439240808 learned_addr learned my addr 172.21.15.67:0/439240808 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.405+0000 7f996ea00640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 msgr2=0x7f9970071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.405+0000 7f996ea00640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 msgr2=0x7f9970071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.405+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.405+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:18.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.406+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:18.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.406+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 msgr2=0x7f99700acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:12:18.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.406+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 msgr2=0x7f99700acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:18.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.406+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:18.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.406+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.606+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.606+0000 7f996ea00640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 msgr2=0x7f9970071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:12:18.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.606+0000 7f996ea00640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 msgr2=0x7f9970071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.606+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.607+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:12:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.607+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.607+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 msgr2=0x7f99700acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:12:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.607+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 msgr2=0x7f99700acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:18.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.607+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:18.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:18.607+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:12:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.008+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.008+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.009+0000 7f996ea00640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 msgr2=0x7f9970071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:12:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.009+0000 7f996ea00640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 msgr2=0x7f9970071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.009+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.009+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 msgr2=0x7f99700acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:12:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.009+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 msgr2=0x7f99700acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.009+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.009+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:12:19.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.009+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:12:19.585 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 18% (38/211) Updating files: 19% (41/211) Updating files: 20% (43/211) Updating files: 21% (45/211) Updating files: 22% (47/211) Updating files: 23% (49/211) Updating files: 24% (51/211) Updating files: 25% (53/211) Updating files: 26% (55/211) Updating files: 27% (57/211) Updating files: 28% (60/211) Updating files: 29% (62/211) Updating files: 30% (64/211) Updating files: 31% (66/211) Updating files: 32% (68/211) Updating files: 33% (70/211) Updating files: 34% (72/211) Updating files: 35% (74/211) Updating files: 36% (76/211) Updating files: 37% (79/211) Updating files: 38% (81/211) Updating files: 39% (83/211) Updating files: 40% (85/211) Updating files: 40% (86/211) Updating files: 41% (87/211) Updating files: 42% (89/211) Updating files: 43% (91/211) Updating files: 44% (93/211) Updating files: 45% (95/211) Updating files: 46% (98/211) Updating files: 47% (100/211) Updating files: 48% (102/211) Updating files: 49% (104/211) Updating files: 50% (106/211) Updating files: 51% (108/211) Updating files: 52% (110/211) Updating files: 53% (112/211) Updating files: 54% (114/211) Updating files: 55% (117/211) Updating files: 56% (119/211) Updating files: 57% (121/211) Updating files: 58% (123/211) Updating files: 59% (125/211) Updating files: 60% (127/211) Updating files: 61% (129/211) Updating files: 62% (131/211) Updating files: 63% (133/211) Updating files: 64% (136/211) Updating files: 65% (138/211) Updating files: 66% (140/211) Updating files: 67% (142/211) Updating files: 68% (144/211) Updating files: 69% (146/211) Updating files: 70% (148/211) Updating files: 71% (150/211) Updating files: 72% (152/211) Updating files: 73% (155/211) Updating files: 74% (157/211) Updating files: 75% (159/211) Updating files: 76% (161/211) Updating files: 77% (163/211) Updating files: 78% (165/211) Updating files: 79% (167/211) Updating files: 80% (169/211) Updating files: 81% (171/211) Updating files: 82% (174/211) Updating files: 83% (176/211) Updating files: 84% (178/211) Updating files: 85% (180/211) Updating files: 86% (182/211) Updating files: 87% (184/211) Updating files: 88% (186/211) Updating files: 89% (188/211) Updating files: 90% (190/211) Updating files: 91% (193/211) Updating files: 92% (195/211) Updating files: 93% (197/211) Updating files: 94% (199/211) Updating files: 94% (200/211) Updating files: 95% (201/211) Updating files: 96% (203/211) Updating files: 97% (205/211) Updating files: 98% (207/211) Updating files: 99% (209/211) Updating files: 100% (211/211) Updating files: 100% (211/211), done. 2024-03-09T09:12:19.640 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at cbae6a435c6 v0.59 2024-03-09T09:12:19.652 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.59 2024-03-09T09:12:19.780 INFO:tasks.workunit.client.0.smithi067.stdout:v0.60 2024-03-09T09:12:19.780 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:12:19.780 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 60 -eq 48 ']' 2024-03-09T09:12:19.780 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.60 2024-03-09T09:12:19.781 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.60 2024-03-09T09:12:19.781 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.60 2024-03-09T09:12:19.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.810+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:19.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.810+0000 7f996ea00640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 msgr2=0x7f9970071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:12:19.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.810+0000 7f996ea00640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 msgr2=0x7f9970071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:19.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.810+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:19.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.810+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:19.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.810+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:12:19.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.810+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 msgr2=0x7f99700acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:12:19.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.810+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 msgr2=0x7f99700acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:19.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.810+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:19.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:19.811+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:12:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.405+0000 7f9967e00640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 msgr2=0x7f99700acb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:12:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.405+0000 7f9967e00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.134:3300/0 conn(0x7f99700c22a0 0x7f99700acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.405+0000 7f9967e00640 1 -- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 msgr2=0x7f9970071190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:12:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.405+0000 7f9967e00640 1 --2- 172.21.15.67:0/439240808 >> v2:172.21.15.67:3300/0 conn(0x7f99700ac730 0x7f9970071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.405+0000 7f9967e00640 1 -- 172.21.15.67:0/439240808 >> v1:172.21.15.67:6789/0 conn(0x7f9970157650 legacy=0x7f9970157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.405+0000 7f9967e00640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99500072f0 0x7f99500097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.405+0000 7f9967e00640 1 -- 172.21.15.67:0/439240808 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9970153d40 con 0x7f99500036a0 2024-03-09T09:12:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.405+0000 7f9967e00640 1 -- 172.21.15.67:0/439240808 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9970070ee0 con 0x7f99500072f0 2024-03-09T09:12:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.406+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99500072f0 msgr2=0x7f99500097b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.406+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99500072f0 0x7f99500097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:21.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.607+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99500072f0 msgr2=0x7f99500097b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:21.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:21.607+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99500072f0 0x7f99500097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:12:22.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.008+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99500072f0 msgr2=0x7f99500097b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:22.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.008+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99500072f0 0x7f99500097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:12:22.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.393+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606016028 0 0) 0x7f9970153d40 con 0x7f99500036a0 2024-03-09T09:12:22.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.393+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9954003520 con 0x7f99500036a0 2024-03-09T09:12:22.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.394+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1179647056 0 0) 0x7f9954003520 con 0x7f99500036a0 2024-03-09T09:12:22.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.394+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99500072f0 msgr2=0x7f99500097b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:12:22.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.394+0000 7f996d600640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99500072f0 0x7f99500097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.394+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995000af10 con 0x7f99500036a0 2024-03-09T09:12:22.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.394+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f996000f040 con 0x7f99500036a0 2024-03-09T09:12:22.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.394+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 >> v1:172.21.15.134:6789/0 conn(0x7f99500036a0 legacy=0x7f9950005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:22.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.394+0000 7f996f400640 1 -- 172.21.15.67:0/439240808 reap_dead start 2024-03-09T09:12:22.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.394+0000 7f996d600640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99540039b0 0x7f9954003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:22.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.395+0000 7f996d600640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9954004800 0x7f9954004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:22.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.395+0000 7f996d600640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99540052e0 0x7f995400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.395+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9970070ee0 con 0x7f9954004800 2024-03-09T09:12:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.395+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995400c230 con 0x7f99540052e0 2024-03-09T09:12:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.395+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995400c430 con 0x7f99540039b0 2024-03-09T09:12:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.395+0000 7f996ea00640 1 -- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99540039b0 msgr2=0x7f9954003db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.395+0000 7f996ea00640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99540039b0 0x7f9954003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.395+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9954004800 0x7f9954004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.396+0000 7f996f400640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99540052e0 0x7f995400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:22.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.397+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99540039b0 msgr2=0x7f9954003db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:12:22.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.397+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99540039b0 0x7f9954003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.397+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99540052e0 msgr2=0x7f995400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:22.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.397+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99540052e0 0x7f995400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.397+0000 7f996e000640 1 -- 172.21.15.67:0/439240808 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9954003520 con 0x7f9954004800 2024-03-09T09:12:22.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.397+0000 7f996f400640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99540052e0 0x7f995400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:12:22.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.401+0000 7f996e000640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9954004800 0x7f9954004c80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f99700c26a0 tx=0x7f9954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ab5e7a2ae31ab6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:22.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.424+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c03a070 con 0x7f9954004800 2024-03-09T09:12:22.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.424+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c035410 con 0x7f9954004800 2024-03-09T09:12:22.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.424+0000 7f996d600640 1 -- 172.21.15.67:0/439240808 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c003610 con 0x7f9954004800 2024-03-09T09:12:22.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.426+0000 7f9975c00640 1 -- 172.21.15.67:0/439240808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9954004800 msgr2=0x7f9954004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:22.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.426+0000 7f9975c00640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9954004800 0x7f9954004c80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f99700c26a0 tx=0x7f9954005830 comp rx=0 tx=0).stop 2024-03-09T09:12:22.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.426+0000 7f9975c00640 1 -- 172.21.15.67:0/439240808 shutdown_connections 2024-03-09T09:12:22.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.426+0000 7f9975c00640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99540052e0 0x7f995400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.426+0000 7f9975c00640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9954004800 0x7f9954004c80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.426+0000 7f9975c00640 1 --2- 172.21.15.67:0/439240808 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99540039b0 0x7f9954003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.426+0000 7f9975c00640 1 -- 172.21.15.67:0/439240808 >> 172.21.15.67:0/439240808 conn(0x7f99700c7370 msgr2=0x7f99700c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.426+0000 7f9975c00640 1 -- 172.21.15.67:0/439240808 shutdown_connections 2024-03-09T09:12:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.426+0000 7f9975c00640 1 -- 172.21.15.67:0/439240808 wait complete. 2024-03-09T09:12:22.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.431+0000 7f9975c00640 1 Processor -- start 2024-03-09T09:12:22.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.431+0000 7f9975c00640 1 -- start start 2024-03-09T09:12:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.432+0000 7f9975c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99540039b0 0x7f99700c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.432+0000 7f9975c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9954004800 0x7f99700a8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.432+0000 7f9975c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540052e0 0x7f99700a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.432+0000 7f9975c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9970070c00 con 0x7f99540052e0 2024-03-09T09:12:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.432+0000 7f9975c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9970070920 con 0x7f99540039b0 2024-03-09T09:12:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.432+0000 7f9975c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9970151610 con 0x7f9954004800 2024-03-09T09:12:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.432+0000 7f996e000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9954004800 msgr2=0x7f99700a8820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.432+0000 7f996e000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9954004800 0x7f99700a8820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.432+0000 7f996ea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99540039b0 0x7f99700c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.432+0000 7f996f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540052e0 0x7f99700a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.433+0000 7f996f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540052e0 0x7f99700a8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43288/0 (socket says 172.21.15.67:43288) 2024-03-09T09:12:22.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.433+0000 7f996f400640 1 -- 172.21.15.67:0/1490339707 learned_addr learned my addr 172.21.15.67:0/1490339707 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.433+0000 7f996f400640 1 -- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9954004800 msgr2=0x7f99700a8820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:12:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.433+0000 7f996f400640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9954004800 0x7f99700a8820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.433+0000 7f996f400640 1 -- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99540039b0 msgr2=0x7f99700c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.433+0000 7f996f400640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99540039b0 0x7f99700c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.433+0000 7f996f400640 1 -- 172.21.15.67:0/1490339707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970151f50 con 0x7f99540052e0 2024-03-09T09:12:22.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.434+0000 7f996f400640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540052e0 0x7f99700a8d60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f997006ef00 tx=0x7f996800fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:22.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.437+0000 7f9966a00640 1 -- 172.21.15.67:0/1490339707 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968005d50 con 0x7f99540052e0 2024-03-09T09:12:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.437+0000 7f9975c00640 1 -- 172.21.15.67:0/1490339707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9970152230 con 0x7f99540052e0 2024-03-09T09:12:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.437+0000 7f9966a00640 1 -- 172.21.15.67:0/1490339707 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9968018070 con 0x7f99540052e0 2024-03-09T09:12:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.437+0000 7f9966a00640 1 -- 172.21.15.67:0/1490339707 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996800d8c0 con 0x7f99540052e0 2024-03-09T09:12:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.437+0000 7f9975c00640 1 -- 172.21.15.67:0/1490339707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99701527f0 con 0x7f99540052e0 2024-03-09T09:12:22.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.438+0000 7f9975c00640 1 -- 172.21.15.67:0/1490339707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9970075a00 con 0x7f99540052e0 2024-03-09T09:12:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.439+0000 7f9966a00640 1 -- 172.21.15.67:0/1490339707 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f996800da60 con 0x7f99540052e0 2024-03-09T09:12:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.440+0000 7f9966a00640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f995006bd50 0x7f995006e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.441+0000 7f9966a00640 1 -- 172.21.15.67:0/1490339707 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f996808a310 con 0x7f99540052e0 2024-03-09T09:12:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.442+0000 7f996ea00640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f995006bd50 0x7f995006e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:22.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.444+0000 7f996ea00640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f995006bd50 0x7f995006e210 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9960000c00 tx=0x7f9960008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:22.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.456+0000 7f9966a00640 1 -- 172.21.15.67:0/1490339707 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9968058d90 con 0x7f99540052e0 2024-03-09T09:12:22.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.737+0000 7f9975c00640 1 -- 172.21.15.67:0/1490339707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f99700c1d70 con 0x7f99540052e0 2024-03-09T09:12:22.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.738+0000 7f9966a00640 1 -- 172.21.15.67:0/1490339707 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f9968058690 con 0x7f99540052e0 2024-03-09T09:12:22.743 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":146,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:12:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 -- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f995006bd50 msgr2=0x7f995006e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f995006bd50 0x7f995006e210 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9960000c00 tx=0x7f9960008040 comp rx=0 tx=0).stop 2024-03-09T09:12:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 -- 172.21.15.67:0/1490339707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540052e0 msgr2=0x7f99700a8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540052e0 0x7f99700a8d60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f997006ef00 tx=0x7f996800fd10 comp rx=0 tx=0).stop 2024-03-09T09:12:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 -- 172.21.15.67:0/1490339707 shutdown_connections 2024-03-09T09:12:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f995006bd50 0x7f995006e210 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99540052e0 0x7f99700a8d60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9954004800 0x7f99700a8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 --2- 172.21.15.67:0/1490339707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99540039b0 0x7f99700c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:22.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 -- 172.21.15.67:0/1490339707 >> 172.21.15.67:0/1490339707 conn(0x7f99700c7370 msgr2=0x7f9970157a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:22.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 -- 172.21.15.67:0/1490339707 shutdown_connections 2024-03-09T09:12:22.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:22.741+0000 7f9975c00640 1 -- 172.21.15.67:0/1490339707 wait complete. 2024-03-09T09:12:22.764 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:12:22.764 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:12:22.765 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:12:23.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.058+0000 7fef9be00640 1 Processor -- start 2024-03-09T09:12:23.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.058+0000 7fef9be00640 1 -- start start 2024-03-09T09:12:23.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.061+0000 7fef9be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fef940aab20 0x7fef94153540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.061+0000 7fef9be00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef94153a80 con 0x7fef9406e050 2024-03-09T09:12:23.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.061+0000 7fef9be00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9406d590 con 0x7fef94149590 2024-03-09T09:12:23.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.061+0000 7fef9be00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef940c82b0 con 0x7fef940aab20 2024-03-09T09:12:23.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.062+0000 7fef98a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fef940aab20 0x7fef94153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.063+0000 7fef99400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fef94149590 0x7fef940ab620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:42102/0 (socket says 172.21.15.67:42102) 2024-03-09T09:12:23.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.063+0000 7fef99400640 1 -- 172.21.15.67:0/3751226549 learned_addr learned my addr 172.21.15.67:0/3751226549 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:12:23.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.066+0000 7fef93e00640 1 -- 172.21.15.67:0/3751226549 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712195554 0 0) 0x7fef88001110 con 0x7fef9406e050 2024-03-09T09:12:23.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.066+0000 7fef93e00640 1 -- 172.21.15.67:0/3751226549 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef7c005730 con 0x7fef9406e050 2024-03-09T09:12:23.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.066+0000 7fef93e00640 1 -- 172.21.15.67:0/3751226549 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924306684 0 0) 0x7fef9406d590 con 0x7fef94149590 2024-03-09T09:12:23.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.066+0000 7fef93e00640 1 -- 172.21.15.67:0/3751226549 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef88001110 con 0x7fef94149590 2024-03-09T09:12:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.067+0000 7fef98a00640 1 -- 172.21.15.67:0/3751226549 >> v1:172.21.15.134:6789/0 conn(0x7fef94149590 legacy=0x7fef940ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.067+0000 7fef98a00640 1 -- 172.21.15.67:0/3751226549 >> v1:172.21.15.67:6789/0 conn(0x7fef9406e050 legacy=0x7fef940c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.067+0000 7fef98a00640 1 -- 172.21.15.67:0/3751226549 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940aafc0 con 0x7fef940aab20 2024-03-09T09:12:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.074+0000 7fef98a00640 1 --2- 172.21.15.67:0/3751226549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef940aab20 0x7fef94153540 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef800097a0 tx=0x7fef8002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f44c011b82b2a1e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:23.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.077+0000 7fef93e00640 1 -- 172.21.15.67:0/3751226549 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80035020 con 0x7fef940aab20 2024-03-09T09:12:23.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.077+0000 7fef93e00640 1 -- 172.21.15.67:0/3751226549 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef80031830 con 0x7fef940aab20 2024-03-09T09:12:23.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.077+0000 7fef93e00640 1 -- 172.21.15.67:0/3751226549 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80031c00 con 0x7fef940aab20 2024-03-09T09:12:23.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.078+0000 7fef9be00640 1 -- 172.21.15.67:0/3751226549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef940aab20 msgr2=0x7fef94153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.078+0000 7fef9be00640 1 --2- 172.21.15.67:0/3751226549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef940aab20 0x7fef94153540 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef800097a0 tx=0x7fef8002b8d0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.079+0000 7fef9be00640 1 -- 172.21.15.67:0/3751226549 shutdown_connections 2024-03-09T09:12:23.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.079+0000 7fef9be00640 1 --2- 172.21.15.67:0/3751226549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef940aab20 0x7fef94153540 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.079+0000 7fef9be00640 1 -- 172.21.15.67:0/3751226549 >> 172.21.15.67:0/3751226549 conn(0x7fef940c7370 msgr2=0x7fef940c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:23.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.080+0000 7fef9be00640 1 -- 172.21.15.67:0/3751226549 shutdown_connections 2024-03-09T09:12:23.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.080+0000 7fef9be00640 1 -- 172.21.15.67:0/3751226549 wait complete. 2024-03-09T09:12:23.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.083+0000 7fef9be00640 1 Processor -- start 2024-03-09T09:12:23.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.087+0000 7fef9be00640 1 -- start start 2024-03-09T09:12:23.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.091+0000 7fef9be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef9406e050 0x7fef9414ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.092+0000 7fef99400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef9406e050 0x7fef9414ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.093+0000 7fef99400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef9406e050 0x7fef9414ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43300/0 (socket says 172.21.15.67:43300) 2024-03-09T09:12:23.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.093+0000 7fef99400640 1 -- 172.21.15.67:0/2445694017 learned_addr learned my addr 172.21.15.67:0/2445694017 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:23.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.093+0000 7fef9be00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940aab20 0x7fef941f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.094+0000 7fef9be00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef94149590 0x7fef94156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.094+0000 7fef98a00640 1 -- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940aab20 msgr2=0x7fef941f2940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:23.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.094+0000 7fef98a00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940aab20 0x7fef941f2940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:23.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.094+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9406d850 con 0x7fef9406e050 2024-03-09T09:12:23.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.095+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9406dbc0 con 0x7fef94149590 2024-03-09T09:12:23.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.095+0000 7fef99e00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef94149590 0x7fef94156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.095+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef94149970 con 0x7fef940aab20 2024-03-09T09:12:23.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.097+0000 7fef99e00640 1 -- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940aab20 msgr2=0x7fef941f2940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:12:23.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.097+0000 7fef99e00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef940aab20 0x7fef941f2940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.097+0000 7fef99e00640 1 -- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef9406e050 msgr2=0x7fef9414ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.097+0000 7fef99e00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef9406e050 0x7fef9414ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.097+0000 7fef99e00640 1 -- 172.21.15.67:0/2445694017 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef94154a90 con 0x7fef94149590 2024-03-09T09:12:23.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.099+0000 7fef99e00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef94149590 0x7fef94156760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c440 tx=0x7fef8800c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:23.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.100+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef880052d0 con 0x7fef94149590 2024-03-09T09:12:23.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.101+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef941544d0 con 0x7fef94149590 2024-03-09T09:12:23.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.101+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef94156240 con 0x7fef94149590 2024-03-09T09:12:23.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.103+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef88005470 con 0x7fef94149590 2024-03-09T09:12:23.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.104+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef880052d0 con 0x7fef94149590 2024-03-09T09:12:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.106+0000 7fef9be00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70002a70 0x7fef70004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.107+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef70007ec0 con 0x7fef70002a70 2024-03-09T09:12:23.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.109+0000 7fef99400640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70002a70 0x7fef70004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.111+0000 7fef99400640 1 -- 172.21.15.67:0/2445694017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef700080c0 con 0x7fef70002a70 2024-03-09T09:12:23.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.113+0000 7fef99400640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70002a70 0x7fef70004f30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fef94156170 tx=0x7fef8c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:23.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.114+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fef88012630 con 0x7fef94149590 2024-03-09T09:12:23.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.117+0000 7fef91600640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fef7406b680 0x7fef7406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.117+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fef88089c30 con 0x7fef94149590 2024-03-09T09:12:23.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.118+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00de30 con 0x7fef70002a70 2024-03-09T09:12:23.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.119+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fef8c018070 con 0x7fef70002a70 2024-03-09T09:12:23.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.119+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70002a70 msgr2=0x7fef70004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.119+0000 7fef91600640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70002a70 0x7fef70004f30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fef94156170 tx=0x7fef8c00d520 comp rx=0 tx=0).stop 2024-03-09T09:12:23.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.119+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70002a70 msgr2=0x7fef70004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:23.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.119+0000 7fef91600640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70002a70 0x7fef70004f30 secure :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fef94156170 tx=0x7fef8c00d520 comp rx=0 tx=0).stop 2024-03-09T09:12:23.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.123+0000 7fef98a00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fef7406b680 0x7fef7406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.128+0000 7fef98a00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fef7406b680 0x7fef7406db40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fef80036000 tx=0x7fef80008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.140+0000 7fef9be00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70006eb0 0x7fef70007340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.140+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7000a4a0 con 0x7fef70006eb0 2024-03-09T09:12:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.140+0000 7fef99400640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70006eb0 0x7fef70007340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.142+0000 7fef99400640 1 -- 172.21.15.67:0/2445694017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fef70007880 con 0x7fef70006eb0 2024-03-09T09:12:23.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.142+0000 7fef99400640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70006eb0 0x7fef70007340 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fef94156170 tx=0x7fef8c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:23.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.143+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c007630 con 0x7fef70006eb0 2024-03-09T09:12:23.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.144+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7fef8c007630 con 0x7fef70006eb0 2024-03-09T09:12:23.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.144+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70006eb0 msgr2=0x7fef70007340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.144+0000 7fef91600640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70006eb0 0x7fef70007340 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fef94156170 tx=0x7fef8c001250 comp rx=0 tx=0).stop 2024-03-09T09:12:23.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.144+0000 7fef91600640 1 -- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70006eb0 msgr2=0x7fef70007340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:23.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.144+0000 7fef91600640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef70006eb0 0x7fef70007340 secure :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fef94156170 tx=0x7fef8c001250 comp rx=0 tx=0).stop 2024-03-09T09:12:23.150 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:12:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:12:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:12:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:12:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 146, 2024-03-09T09:12:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:12:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:12:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:12:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:12:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:12:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:12:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:12:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:12:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:12:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:12:23.155 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:23.155 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:23.157 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:12:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:12:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:12:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:12:23.167 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:12:23.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.146+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fef7406b680 msgr2=0x7fef7406db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.146+0000 7fef9be00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fef7406b680 0x7fef7406db40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fef80036000 tx=0x7fef80008030 comp rx=0 tx=0).stop 2024-03-09T09:12:23.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.147+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef94149590 msgr2=0x7fef94156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.147+0000 7fef9be00640 1 --2- 172.21.15.67:0/2445694017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef94149590 0x7fef94156760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c440 tx=0x7fef8800c900 comp rx=0 tx=0).stop 2024-03-09T09:12:23.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.147+0000 7fef99e00640 1 -- 172.21.15.67:0/2445694017 reap_dead start 2024-03-09T09:12:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.147+0000 7fef99e00640 1 -- 172.21.15.67:0/2445694017 reap_dead start 2024-03-09T09:12:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.147+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 shutdown_connections 2024-03-09T09:12:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.147+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 >> 172.21.15.67:0/2445694017 conn(0x7fef940c7370 msgr2=0x7fef940ac4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.147+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 shutdown_connections 2024-03-09T09:12:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.148+0000 7fef9be00640 1 -- 172.21.15.67:0/2445694017 wait complete. 2024-03-09T09:12:23.194 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:12:23.194 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:12:23.195 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:12:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.447+0000 7f383d200640 1 Processor -- start 2024-03-09T09:12:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.447+0000 7f383d200640 1 -- start start 2024-03-09T09:12:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.448+0000 7f383d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f38380ac930 0x7f38380c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.448+0000 7f383d200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f38380c30a0 0x7f3838150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.448+0000 7f383d200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38380c2780 con 0x7f3838151190 2024-03-09T09:12:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.448+0000 7f383d200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3838070670 con 0x7f38380ac930 2024-03-09T09:12:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.448+0000 7f383d200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38380708c0 con 0x7f38380c30a0 2024-03-09T09:12:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.449+0000 7f3836a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f38380c30a0 0x7f3838150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.449+0000 7f3836a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f38380c30a0 0x7f3838150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:40720/0 (socket says 172.21.15.67:40720) 2024-03-09T09:12:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.449+0000 7f3836a00640 1 -- 172.21.15.67:0/1454426198 learned_addr learned my addr 172.21.15.67:0/1454426198 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.449+0000 7f3837400640 1 --2- 172.21.15.67:0/1454426198 >> v2:172.21.15.67:3300/0 conn(0x7f38380ac930 0x7f38380c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.449+0000 7f3836a00640 1 -- 172.21.15.67:0/1454426198 >> v2:172.21.15.67:3300/0 conn(0x7f38380ac930 msgr2=0x7f38380c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.449+0000 7f3836a00640 1 --2- 172.21.15.67:0/1454426198 >> v2:172.21.15.67:3300/0 conn(0x7f38380ac930 0x7f38380c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.449+0000 7f3836a00640 1 -- 172.21.15.67:0/1454426198 >> v1:172.21.15.67:6789/0 conn(0x7f3838151190 legacy=0x7f38381539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.449+0000 7f3836a00640 1 -- 172.21.15.67:0/1454426198 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3838070ba0 con 0x7f38380c30a0 2024-03-09T09:12:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.451+0000 7f3836a00640 1 --2- 172.21.15.67:0/1454426198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38380c30a0 0x7f3838150c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38380c8040 tx=0x7f382402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5445a3677ec08f75 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:23.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.451+0000 7f3836000640 1 -- 172.21.15.67:0/1454426198 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824034070 con 0x7f38380c30a0 2024-03-09T09:12:23.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.451+0000 7f3836000640 1 -- 172.21.15.67:0/1454426198 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3824030860 con 0x7f38380c30a0 2024-03-09T09:12:23.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.451+0000 7f3836000640 1 -- 172.21.15.67:0/1454426198 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824030c30 con 0x7f38380c30a0 2024-03-09T09:12:23.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.452+0000 7f383d200640 1 -- 172.21.15.67:0/1454426198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38380c30a0 msgr2=0x7f3838150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.452+0000 7f383d200640 1 --2- 172.21.15.67:0/1454426198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38380c30a0 0x7f3838150c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38380c8040 tx=0x7f382402b9a0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.452+0000 7f383d200640 1 -- 172.21.15.67:0/1454426198 shutdown_connections 2024-03-09T09:12:23.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.452+0000 7f383d200640 1 --2- 172.21.15.67:0/1454426198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38380c30a0 0x7f3838150c50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.452+0000 7f383d200640 1 --2- 172.21.15.67:0/1454426198 >> v2:172.21.15.67:3300/0 conn(0x7f38380ac930 0x7f38380c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.452+0000 7f383d200640 1 -- 172.21.15.67:0/1454426198 >> 172.21.15.67:0/1454426198 conn(0x7f38380c7220 msgr2=0x7f38380c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:23.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.452+0000 7f383d200640 1 -- 172.21.15.67:0/1454426198 shutdown_connections 2024-03-09T09:12:23.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.452+0000 7f383d200640 1 -- 172.21.15.67:0/1454426198 wait complete. 2024-03-09T09:12:23.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.454+0000 7f383d200640 1 Processor -- start 2024-03-09T09:12:23.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.455+0000 7f383d200640 1 -- start start 2024-03-09T09:12:23.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f383d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38380ac930 0x7f38381554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f383d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38380c30a0 0x7f38381ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f383d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3838151190 0x7f383814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f383d200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38380acd10 con 0x7f3838151190 2024-03-09T09:12:23.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f383d200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838151590 con 0x7f38380ac930 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f383d200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38380ace90 con 0x7f38380c30a0 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f3836a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38380c30a0 msgr2=0x7f38381ebaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f3836a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38380c30a0 0x7f38381ebaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f3837400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38380ac930 0x7f38381554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f3837400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38380ac930 0x7f38381554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:40724/0 (socket says 172.21.15.67:40724) 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f3837400640 1 -- 172.21.15.67:0/891324394 learned_addr learned my addr 172.21.15.67:0/891324394 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.456+0000 7f3837e00640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3838151190 0x7f383814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.457+0000 7f3837400640 1 -- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38380c30a0 msgr2=0x7f38381ebaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.457+0000 7f3837400640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38380c30a0 0x7f38381ebaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.457+0000 7f3837400640 1 -- 172.21.15.67:0/891324394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3838151190 msgr2=0x7f383814b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.457+0000 7f3837400640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3838151190 0x7f383814b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.457+0000 7f3837400640 1 -- 172.21.15.67:0/891324394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38381eafe0 con 0x7f38380ac930 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.457+0000 7f3837400640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38380ac930 0x7f38381554d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f382800e8a0 tx=0x7f382800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.457+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828017070 con 0x7f38380ac930 2024-03-09T09:12:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.458+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382800c940 con 0x7f38380ac930 2024-03-09T09:12:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.458+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800cd30 con 0x7f38380ac930 2024-03-09T09:12:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.458+0000 7f383d200640 1 -- 172.21.15.67:0/891324394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3838154f10 con 0x7f38380ac930 2024-03-09T09:12:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.458+0000 7f383d200640 1 -- 172.21.15.67:0/891324394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38381eb2a0 con 0x7f38380ac930 2024-03-09T09:12:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.459+0000 7f383d200640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38381eb4e0 0x7f383814c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.459+0000 7f383d200640 1 -- 172.21.15.67:0/891324394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38381485c0 con 0x7f38381eb4e0 2024-03-09T09:12:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.460+0000 7f3836a00640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38381eb4e0 0x7f383814c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.460+0000 7f3836a00640 1 -- 172.21.15.67:0/891324394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f383804f3c0 con 0x7f38381eb4e0 2024-03-09T09:12:23.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.460+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f382801a070 con 0x7f38380ac930 2024-03-09T09:12:23.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.460+0000 7f3836a00640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38381eb4e0 0x7f383814c0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38380c7fc0 tx=0x7f38240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:23.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.461+0000 7f382f400640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f381806b480 0x7f381806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.461+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f382808a3c0 con 0x7f38380ac930 2024-03-09T09:12:23.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.461+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824034040 con 0x7f38381eb4e0 2024-03-09T09:12:23.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.461+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f3824030e50 con 0x7f38381eb4e0 2024-03-09T09:12:23.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.461+0000 7f3837e00640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f381806b480 0x7f381806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.461+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38381eb4e0 msgr2=0x7f383814c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.461+0000 7f382f400640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38381eb4e0 0x7f383814c0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38380c7fc0 tx=0x7f38240046c0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.461+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38381eb4e0 msgr2=0x7f383814c0d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.461+0000 7f382f400640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38381eb4e0 0x7f383814c0d0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38380c7fc0 tx=0x7f38240046c0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.462+0000 7f3837e00640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f381806b480 0x7f381806d940 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f38300012e0 tx=0x7f3830009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:23.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.473+0000 7f383d200640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3838157b40 0x7f38380c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:23.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.473+0000 7f383d200640 1 -- 172.21.15.67:0/891324394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3838074670 con 0x7f3838157b40 2024-03-09T09:12:23.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.473+0000 7f3836a00640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3838157b40 0x7f38380c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:23.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.474+0000 7f3836a00640 1 -- 172.21.15.67:0/891324394 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3838067b30 con 0x7f3838157b40 2024-03-09T09:12:23.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.474+0000 7f3836a00640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3838157b40 0x7f38380c6370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3824000c00 tx=0x7f3824009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:23.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.475+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3824034040 con 0x7f3838157b40 2024-03-09T09:12:23.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.475+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4523 (secure 0 0 0) 0x7f3824009490 con 0x7f3838157b40 2024-03-09T09:12:23.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.475+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3838157b40 msgr2=0x7f38380c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.475+0000 7f382f400640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3838157b40 0x7f38380c6370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3824000c00 tx=0x7f3824009240 comp rx=0 tx=0).stop 2024-03-09T09:12:23.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.475+0000 7f382f400640 1 -- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3838157b40 msgr2=0x7f38380c6370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:23.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.475+0000 7f382f400640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3838157b40 0x7f38380c6370 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3824000c00 tx=0x7f3824009240 comp rx=0 tx=0).stop 2024-03-09T09:12:23.481 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:12:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:12:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:12:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:12:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 146, 2024-03-09T09:12:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:12:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:12:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:12:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:12:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:12:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:12:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:12:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:12:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:12:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:12:23.493 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:12:23.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.476+0000 7f383d200640 1 -- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f381806b480 msgr2=0x7f381806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.476+0000 7f383d200640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f381806b480 0x7f381806d940 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f38300012e0 tx=0x7f3830009040 comp rx=0 tx=0).stop 2024-03-09T09:12:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.476+0000 7f383d200640 1 -- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38380ac930 msgr2=0x7f38381554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.476+0000 7f383d200640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38380ac930 0x7f38381554d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f382800e8a0 tx=0x7f382800ed60 comp rx=0 tx=0).stop 2024-03-09T09:12:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.476+0000 7f3837e00640 1 -- 172.21.15.67:0/891324394 reap_dead start 2024-03-09T09:12:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.476+0000 7f383d200640 1 -- 172.21.15.67:0/891324394 shutdown_connections 2024-03-09T09:12:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.476+0000 7f383d200640 1 --2- 172.21.15.67:0/891324394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38380ac930 0x7f38381554d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.476+0000 7f383d200640 1 -- 172.21.15.67:0/891324394 >> 172.21.15.67:0/891324394 conn(0x7f38380c7220 msgr2=0x7f38380c0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.476+0000 7f383d200640 1 -- 172.21.15.67:0/891324394 shutdown_connections 2024-03-09T09:12:23.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:23.476+0000 7f383d200640 1 -- 172.21.15.67:0/891324394 wait complete. 2024-03-09T09:12:23.499 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:12:23.499 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:12:25.579 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 11% (30/268) Updating files: 12% (33/268) Updating files: 13% (35/268) Updating files: 14% (38/268) Updating files: 15% (41/268) Updating files: 16% (43/268) Updating files: 17% (46/268) Updating files: 18% (49/268) Updating files: 19% (51/268) Updating files: 20% (54/268) Updating files: 21% (57/268) Updating files: 22% (59/268) Updating files: 23% (62/268) Updating files: 24% (65/268) Updating files: 25% (67/268) Updating files: 26% (70/268) Updating files: 27% (73/268) Updating files: 28% (76/268) Updating files: 29% (78/268) Updating files: 30% (81/268) Updating files: 31% (84/268) Updating files: 32% (86/268) Updating files: 33% (89/268) Updating files: 33% (90/268) Updating files: 34% (92/268) Updating files: 35% (94/268) Updating files: 36% (97/268) Updating files: 37% (100/268) Updating files: 38% (102/268) Updating files: 39% (105/268) Updating files: 40% (108/268) Updating files: 41% (110/268) Updating files: 42% (113/268) Updating files: 43% (116/268) Updating files: 44% (118/268) Updating files: 45% (121/268) Updating files: 46% (124/268) Updating files: 47% (126/268) Updating files: 48% (129/268) Updating files: 49% (132/268) Updating files: 50% (134/268) Updating files: 51% (137/268) Updating files: 52% (140/268) Updating files: 53% (143/268) Updating files: 54% (145/268) Updating files: 55% (148/268) Updating files: 56% (151/268) Updating files: 57% (153/268) Updating files: 58% (156/268) Updating files: 59% (159/268) Updating files: 60% (161/268) Updating files: 61% (164/268) Updating files: 62% (167/268) Updating files: 63% (169/268) Updating files: 64% (172/268) Updating files: 65% (175/268) Updating files: 66% (177/268) Updating files: 67% (180/268) Updating files: 68% (183/268) Updating files: 69% (185/268) Updating files: 70% (188/268) Updating files: 71% (191/268) Updating files: 72% (193/268) Updating files: 73% (196/268) Updating files: 74% (199/268) Updating files: 75% (201/268) Updating files: 76% (204/268) Updating files: 77% (207/268) Updating files: 78% (210/268) Updating files: 79% (212/268) Updating files: 80% (215/268) Updating files: 81% (218/268) Updating files: 82% (220/268) Updating files: 83% (223/268) Updating files: 84% (226/268) Updating files: 85% (228/268) Updating files: 86% (231/268) Updating files: 87% (234/268) Updating files: 88% (236/268) Updating files: 89% (239/268) Updating files: 90% (242/268) Updating files: 91% (244/268) Updating files: 92% (247/268) Updating files: 93% (250/268) Updating files: 94% (252/268) Updating files: 94% (254/268) Updating files: 95% (255/268) Updating files: 96% (258/268) Updating files: 97% (260/268) Updating files: 98% (263/268) Updating files: 99% (266/268) Updating files: 100% (268/268) Updating files: 100% (268/268), done. 2024-03-09T09:12:26.231 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at ba3f91e7504 v0.58 2024-03-09T09:12:26.239 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.58 2024-03-09T09:12:26.851 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:12:26.851 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 59 -eq 48 ']' 2024-03-09T09:12:26.851 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.59 2024-03-09T09:12:26.851 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.59 2024-03-09T09:12:26.851 INFO:tasks.workunit.client.1.smithi134.stdout:v0.59 2024-03-09T09:12:26.852 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.59 2024-03-09T09:12:29.271 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 2% (7/256) Updating files: 3% (8/256) Updating files: 4% (11/256) Updating files: 5% (13/256) Updating files: 6% (16/256) Updating files: 7% (18/256) Updating files: 8% (21/256) Updating files: 9% (24/256) Updating files: 10% (26/256) Updating files: 11% (29/256) Updating files: 12% (31/256) Updating files: 13% (34/256) Updating files: 14% (36/256) Updating files: 15% (39/256) Updating files: 16% (41/256) Updating files: 17% (44/256) Updating files: 18% (47/256) Updating files: 19% (49/256) Updating files: 20% (52/256) Updating files: 21% (54/256) Updating files: 22% (57/256) Updating files: 23% (59/256) Updating files: 24% (62/256) Updating files: 25% (64/256) Updating files: 26% (67/256) Updating files: 26% (68/256) Updating files: 26% (69/256) Updating files: 27% (70/256) Updating files: 28% (72/256) Updating files: 29% (75/256) Updating files: 30% (77/256) Updating files: 31% (80/256) Updating files: 32% (82/256) Updating files: 33% (85/256) Updating files: 34% (88/256) Updating files: 35% (90/256) Updating files: 36% (93/256) Updating files: 37% (95/256) Updating files: 38% (98/256) Updating files: 39% (100/256) Updating files: 40% (103/256) Updating files: 41% (105/256) Updating files: 42% (108/256) Updating files: 43% (111/256) Updating files: 44% (113/256) Updating files: 45% (116/256) Updating files: 46% (118/256) Updating files: 47% (121/256) Updating files: 48% (123/256) Updating files: 49% (126/256) Updating files: 50% (128/256) Updating files: 51% (131/256) Updating files: 52% (134/256) Updating files: 53% (136/256) Updating files: 54% (139/256) Updating files: 55% (141/256) Updating files: 56% (144/256) Updating files: 57% (146/256) Updating files: 58% (149/256) Updating files: 59% (152/256) Updating files: 60% (154/256) Updating files: 61% (157/256) Updating files: 62% (159/256) Updating files: 63% (162/256) Updating files: 64% (164/256) Updating files: 65% (167/256) Updating files: 66% (169/256) Updating files: 67% (172/256) Updating files: 68% (175/256) Updating files: 69% (177/256) Updating files: 70% (180/256) Updating files: 71% (182/256) Updating files: 72% (185/256) Updating files: 73% (187/256) Updating files: 74% (190/256) Updating files: 75% (192/256) Updating files: 76% (195/256) Updating files: 77% (198/256) Updating files: 78% (200/256) Updating files: 79% (203/256) Updating files: 80% (205/256) Updating files: 81% (208/256) Updating files: 82% (210/256) Updating files: 83% (213/256) Updating files: 84% (216/256) Updating files: 85% (218/256) Updating files: 86% (221/256) Updating files: 87% (223/256) Updating files: 88% (226/256) Updating files: 89% (228/256) Updating files: 90% (231/256) Updating files: 91% (233/256) Updating files: 92% (236/256) Updating files: 93% (239/256) Updating files: 94% (241/256) Updating files: 95% (244/256) Updating files: 96% (246/256) Updating files: 97% (249/256) Updating files: 98% (251/256) Updating files: 99% (254/256) Updating files: 100% (256/256) Updating files: 100% (256/256), done. 2024-03-09T09:12:29.336 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at f26f7a39021 v0.60 2024-03-09T09:12:29.364 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.60 2024-03-09T09:12:29.482 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:12:29.482 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 61 -eq 48 ']' 2024-03-09T09:12:29.482 INFO:tasks.workunit.client.0.smithi067.stdout:v0.61 2024-03-09T09:12:29.483 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.61 2024-03-09T09:12:29.483 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.61 2024-03-09T09:12:29.483 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.61 2024-03-09T09:12:32.380 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 31% (67/211) Updating files: 32% (68/211) Updating files: 33% (70/211) Updating files: 34% (72/211) Updating files: 35% (74/211) Updating files: 36% (76/211) Updating files: 37% (79/211) Updating files: 38% (81/211) Updating files: 39% (83/211) Updating files: 40% (85/211) Updating files: 41% (87/211) Updating files: 42% (89/211) Updating files: 43% (91/211) Updating files: 44% (93/211) Updating files: 45% (95/211) Updating files: 46% (98/211) Updating files: 47% (100/211) Updating files: 48% (102/211) Updating files: 49% (104/211) Updating files: 50% (106/211) Updating files: 51% (108/211) Updating files: 52% (110/211) Updating files: 53% (112/211) Updating files: 54% (114/211) Updating files: 55% (117/211) Updating files: 56% (119/211) Updating files: 57% (121/211) Updating files: 58% (123/211) Updating files: 59% (125/211) Updating files: 60% (127/211) Updating files: 61% (129/211) Updating files: 62% (131/211) Updating files: 63% (133/211) Updating files: 64% (136/211) Updating files: 65% (138/211) Updating files: 66% (140/211) Updating files: 67% (142/211) Updating files: 68% (144/211) Updating files: 69% (146/211) Updating files: 70% (148/211) Updating files: 71% (150/211) Updating files: 72% (152/211) Updating files: 73% (155/211) Updating files: 74% (157/211) Updating files: 75% (159/211) Updating files: 76% (161/211) Updating files: 77% (163/211) Updating files: 78% (165/211) Updating files: 79% (167/211) Updating files: 80% (169/211) Updating files: 81% (171/211) Updating files: 82% (174/211) Updating files: 83% (176/211) Updating files: 84% (178/211) Updating files: 85% (180/211) Updating files: 86% (182/211) Updating files: 87% (184/211) Updating files: 88% (186/211) Updating files: 89% (188/211) Updating files: 90% (190/211) Updating files: 91% (193/211) Updating files: 92% (195/211) Updating files: 93% (197/211) Updating files: 94% (199/211) Updating files: 95% (201/211) Updating files: 96% (203/211) Updating files: 97% (205/211) Updating files: 98% (207/211) Updating files: 99% (209/211) Updating files: 100% (211/211) Updating files: 100% (211/211), done. 2024-03-09T09:12:32.449 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at cbae6a435c6 v0.59 2024-03-09T09:12:32.470 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.59 2024-03-09T09:12:32.688 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:12:32.688 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 60 -eq 48 ']' 2024-03-09T09:12:32.688 INFO:tasks.workunit.client.1.smithi134.stdout:v0.60 2024-03-09T09:12:32.688 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.60 2024-03-09T09:12:32.688 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.60 2024-03-09T09:12:32.689 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.60 2024-03-09T09:12:36.513 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 16% (43/256) Updating files: 17% (44/256) Updating files: 18% (47/256) Updating files: 19% (49/256) Updating files: 20% (52/256) Updating files: 21% (54/256) Updating files: 22% (57/256) Updating files: 23% (59/256) Updating files: 24% (62/256) Updating files: 25% (64/256) Updating files: 26% (67/256) Updating files: 27% (70/256) Updating files: 28% (72/256) Updating files: 29% (75/256) Updating files: 30% (77/256) Updating files: 31% (80/256) Updating files: 32% (82/256) Updating files: 33% (85/256) Updating files: 34% (88/256) Updating files: 35% (90/256) Updating files: 36% (93/256) Updating files: 37% (95/256) Updating files: 38% (98/256) Updating files: 39% (100/256) Updating files: 40% (103/256) Updating files: 41% (105/256) Updating files: 42% (108/256) Updating files: 43% (111/256) Updating files: 44% (113/256) Updating files: 45% (116/256) Updating files: 46% (118/256) Updating files: 47% (121/256) Updating files: 48% (123/256) Updating files: 49% (126/256) Updating files: 50% (128/256) Updating files: 51% (131/256) Updating files: 52% (134/256) Updating files: 53% (136/256) Updating files: 54% (139/256) Updating files: 55% (141/256) Updating files: 56% (144/256) Updating files: 57% (146/256) Updating files: 58% (149/256) Updating files: 59% (152/256) Updating files: 60% (154/256) Updating files: 61% (157/256) Updating files: 62% (159/256) Updating files: 63% (162/256) Updating files: 64% (164/256) Updating files: 65% (167/256) Updating files: 66% (169/256) Updating files: 67% (172/256) Updating files: 68% (175/256) Updating files: 69% (177/256) Updating files: 70% (180/256) Updating files: 71% (182/256) Updating files: 72% (185/256) Updating files: 73% (187/256) Updating files: 74% (190/256) Updating files: 75% (192/256) Updating files: 76% (195/256) Updating files: 77% (198/256) Updating files: 78% (200/256) Updating files: 78% (201/256) Updating files: 79% (203/256) Updating files: 80% (205/256) Updating files: 81% (208/256) Updating files: 82% (210/256) Updating files: 83% (213/256) Updating files: 84% (216/256) Updating files: 85% (218/256) Updating files: 86% (221/256) Updating files: 87% (223/256) Updating files: 88% (226/256) Updating files: 89% (228/256) Updating files: 90% (231/256) Updating files: 91% (233/256) Updating files: 92% (236/256) Updating files: 93% (239/256) Updating files: 94% (241/256) Updating files: 95% (244/256) Updating files: 96% (246/256) Updating files: 97% (249/256) Updating files: 98% (251/256) Updating files: 99% (254/256) Updating files: 100% (256/256) Updating files: 100% (256/256), done. 2024-03-09T09:12:36.570 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at f26f7a39021 v0.60 2024-03-09T09:12:36.582 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.60 2024-03-09T09:12:36.801 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:12:36.801 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 61 -eq 48 ']' 2024-03-09T09:12:36.801 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.61 2024-03-09T09:12:36.801 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.61 2024-03-09T09:12:36.801 INFO:tasks.workunit.client.1.smithi134.stdout:v0.61 2024-03-09T09:12:36.802 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.61 2024-03-09T09:12:39.895 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 1% (4/376) Updating files: 2% (8/376) Updating files: 3% (12/376) Updating files: 4% (16/376) Updating files: 5% (19/376) Updating files: 6% (23/376) Updating files: 7% (27/376) Updating files: 8% (31/376) Updating files: 9% (34/376) Updating files: 10% (38/376) Updating files: 11% (42/376) Updating files: 12% (46/376) Updating files: 13% (49/376) Updating files: 14% (53/376) Updating files: 15% (57/376) Updating files: 16% (61/376) Updating files: 17% (64/376) Updating files: 18% (68/376) Updating files: 19% (72/376) Updating files: 20% (76/376) Updating files: 21% (79/376) Updating files: 22% (83/376) Updating files: 23% (87/376) Updating files: 24% (91/376) Updating files: 25% (94/376) Updating files: 26% (98/376) Updating files: 27% (102/376) Updating files: 28% (106/376) Updating files: 29% (110/376) Updating files: 29% (112/376) Updating files: 30% (113/376) Updating files: 31% (117/376) Updating files: 32% (121/376) Updating files: 33% (125/376) Updating files: 34% (128/376) Updating files: 35% (132/376) Updating files: 36% (136/376) Updating files: 37% (140/376) Updating files: 38% (143/376) Updating files: 39% (147/376) Updating files: 40% (151/376) Updating files: 41% (155/376) Updating files: 42% (158/376) Updating files: 43% (162/376) Updating files: 44% (166/376) Updating files: 44% (168/376) Updating files: 45% (170/376) Updating files: 46% (173/376) Updating files: 47% (177/376) Updating files: 48% (181/376) Updating files: 49% (185/376) Updating files: 50% (188/376) Updating files: 51% (192/376) Updating files: 52% (196/376) Updating files: 53% (200/376) Updating files: 54% (204/376) Updating files: 55% (207/376) Updating files: 56% (211/376) Updating files: 57% (215/376) Updating files: 58% (219/376) Updating files: 59% (222/376) Updating files: 60% (226/376) Updating files: 61% (230/376) Updating files: 62% (234/376) Updating files: 63% (237/376) Updating files: 64% (241/376) Updating files: 65% (245/376) Updating files: 66% (249/376) Updating files: 67% (252/376) Updating files: 68% (256/376) Updating files: 69% (260/376) Updating files: 70% (264/376) Updating files: 71% (267/376) Updating files: 72% (271/376) Updating files: 73% (275/376) Updating files: 74% (279/376) Updating files: 75% (282/376) Updating files: 76% (286/376) Updating files: 77% (290/376) Updating files: 78% (294/376) Updating files: 79% (298/376) Updating files: 79% (299/376) Updating files: 80% (301/376) Updating files: 81% (305/376) Updating files: 82% (309/376) Updating files: 83% (313/376) Updating files: 84% (316/376) Updating files: 85% (320/376) Updating files: 86% (324/376) Updating files: 87% (328/376) Updating files: 87% (329/376) Updating files: 88% (331/376) Updating files: 89% (335/376) Updating files: 90% (339/376) Updating files: 91% (343/376) Updating files: 92% (346/376) Updating files: 93% (350/376) Updating files: 94% (354/376) Updating files: 95% (358/376) Updating files: 96% (361/376) Updating files: 97% (365/376) Updating files: 98% (369/376) Updating files: 99% (373/376) Updating files: 100% (376/376) Updating files: 100% (376/376), done. 2024-03-09T09:12:39.973 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 237f3f1e8d8 v0.61 2024-03-09T09:12:39.999 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.61 2024-03-09T09:12:40.173 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:12:40.174 INFO:tasks.workunit.client.0.smithi067.stdout:v0.62 2024-03-09T09:12:40.174 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 62 -eq 48 ']' 2024-03-09T09:12:40.174 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.62 2024-03-09T09:12:40.174 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.62 2024-03-09T09:12:40.174 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.62 2024-03-09T09:12:43.065 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 29% (110/376) Updating files: 30% (113/376) Updating files: 31% (117/376) Updating files: 32% (121/376) Updating files: 33% (125/376) Updating files: 34% (128/376) Updating files: 35% (132/376) Updating files: 36% (136/376) Updating files: 37% (140/376) Updating files: 38% (143/376) Updating files: 39% (147/376) Updating files: 40% (151/376) Updating files: 41% (155/376) Updating files: 42% (158/376) Updating files: 43% (162/376) Updating files: 44% (166/376) Updating files: 45% (170/376) Updating files: 46% (173/376) Updating files: 47% (177/376) Updating files: 47% (179/376) Updating files: 48% (181/376) Updating files: 49% (185/376) Updating files: 50% (188/376) Updating files: 51% (192/376) Updating files: 52% (196/376) Updating files: 53% (200/376) Updating files: 54% (204/376) Updating files: 55% (207/376) Updating files: 56% (211/376) Updating files: 57% (215/376) Updating files: 58% (219/376) Updating files: 59% (222/376) Updating files: 60% (226/376) Updating files: 61% (230/376) Updating files: 62% (234/376) Updating files: 63% (237/376) Updating files: 64% (241/376) Updating files: 65% (245/376) Updating files: 66% (249/376) Updating files: 67% (252/376) Updating files: 68% (256/376) Updating files: 69% (260/376) Updating files: 70% (264/376) Updating files: 71% (267/376) Updating files: 71% (269/376) Updating files: 72% (271/376) Updating files: 73% (275/376) Updating files: 74% (279/376) Updating files: 75% (282/376) Updating files: 76% (286/376) Updating files: 76% (288/376) Updating files: 77% (290/376) Updating files: 78% (294/376) Updating files: 79% (298/376) Updating files: 80% (301/376) Updating files: 81% (305/376) Updating files: 82% (309/376) Updating files: 83% (313/376) Updating files: 84% (316/376) Updating files: 85% (320/376) Updating files: 86% (324/376) Updating files: 87% (328/376) Updating files: 88% (331/376) Updating files: 88% (334/376) Updating files: 89% (335/376) Updating files: 90% (339/376) Updating files: 91% (343/376) Updating files: 92% (346/376) Updating files: 93% (350/376) Updating files: 94% (354/376) Updating files: 95% (358/376) Updating files: 96% (361/376) Updating files: 97% (365/376) Updating files: 98% (369/376) Updating files: 99% (373/376) Updating files: 100% (376/376) Updating files: 100% (376/376), done. 2024-03-09T09:12:43.499 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:12:43.500 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:12:43.500 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:12:43.500 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:12:43.507 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:12:43.507 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:12:43.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:12:43.732 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 237f3f1e8d8 v0.61 2024-03-09T09:12:43.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.732+0000 7f91af200640 1 Processor -- start 2024-03-09T09:12:43.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.732+0000 7f91af200640 1 -- start start 2024-03-09T09:12:43.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.733+0000 7f91af200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b0073180 0x7f91b00c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:43.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.733+0000 7f91af200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f91b006ed50 0x7f91b01592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:43.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.733+0000 7f91af200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b0155b30 con 0x7f91b00c5320 2024-03-09T09:12:43.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.733+0000 7f91af200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b0155df0 con 0x7f91b006ed50 2024-03-09T09:12:43.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.733+0000 7f91af200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b01560d0 con 0x7f91b0073180 2024-03-09T09:12:43.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.733+0000 7f91ad400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f91b006ed50 0x7f91b01592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:43.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.733+0000 7f91ad400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f91b006ed50 0x7f91b01592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47770/0 (socket says 172.21.15.67:47770) 2024-03-09T09:12:43.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.733+0000 7f91ad400640 1 -- 172.21.15.67:0/4167906336 learned_addr learned my addr 172.21.15.67:0/4167906336 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:43.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.734+0000 7f91ade00640 1 --2- 172.21.15.67:0/4167906336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b0073180 0x7f91b00c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:43.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.734+0000 7f91ade00640 1 -- 172.21.15.67:0/4167906336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b0073180 msgr2=0x7f91b00c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:12:43.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.734+0000 7f91ade00640 1 -- 172.21.15.67:0/4167906336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b0073180 msgr2=0x7f91b00c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:43.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.734+0000 7f91ade00640 1 --2- 172.21.15.67:0/4167906336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b0073180 0x7f91b00c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:43.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.734+0000 7f91ad400640 1 -- 172.21.15.67:0/4167906336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b0073180 msgr2=0x7f91b00c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:43.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.734+0000 7f91ade00640 1 --2- 172.21.15.67:0/4167906336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b0073180 0x7f91b00c4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:43.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.734+0000 7f91ad400640 1 --2- 172.21.15.67:0/4167906336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b0073180 0x7f91b00c4de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:43.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.734+0000 7f91ad400640 1 -- 172.21.15.67:0/4167906336 >> v1:172.21.15.67:6789/0 conn(0x7f91b00c5320 legacy=0x7f91b0159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:43.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.734+0000 7f91ad400640 1 -- 172.21.15.67:0/4167906336 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b01563b0 con 0x7f91b006ed50 2024-03-09T09:12:43.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.735+0000 7f91ad400640 1 --2- 172.21.15.67:0/4167906336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91b006ed50 0x7f91b01592d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f919c0098a0 tx=0x7f919c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea577d1a8279764 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:43.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.736+0000 7f91aca00640 1 -- 172.21.15.67:0/4167906336 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c034070 con 0x7f91b006ed50 2024-03-09T09:12:43.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.736+0000 7f91aca00640 1 -- 172.21.15.67:0/4167906336 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c030840 con 0x7f91b006ed50 2024-03-09T09:12:43.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.736+0000 7f91aca00640 1 -- 172.21.15.67:0/4167906336 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c030c10 con 0x7f91b006ed50 2024-03-09T09:12:43.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.736+0000 7f91af200640 1 -- 172.21.15.67:0/4167906336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91b006ed50 msgr2=0x7f91b01592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:43.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.736+0000 7f91af200640 1 --2- 172.21.15.67:0/4167906336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91b006ed50 0x7f91b01592d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f919c0098a0 tx=0x7f919c02b980 comp rx=0 tx=0).stop 2024-03-09T09:12:43.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.736+0000 7f91af200640 1 -- 172.21.15.67:0/4167906336 shutdown_connections 2024-03-09T09:12:43.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.736+0000 7f91af200640 1 --2- 172.21.15.67:0/4167906336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91b006ed50 0x7f91b01592d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:43.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.736+0000 7f91af200640 1 --2- 172.21.15.67:0/4167906336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b0073180 0x7f91b00c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:43.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.736+0000 7f91af200640 1 -- 172.21.15.67:0/4167906336 >> 172.21.15.67:0/4167906336 conn(0x7f91b006c150 msgr2=0x7f91b006c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:43.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.737+0000 7f91af200640 1 -- 172.21.15.67:0/4167906336 shutdown_connections 2024-03-09T09:12:43.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.737+0000 7f91af200640 1 -- 172.21.15.67:0/4167906336 wait complete. 2024-03-09T09:12:43.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.738+0000 7f91af200640 1 Processor -- start 2024-03-09T09:12:43.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.738+0000 7f91af200640 1 -- start start 2024-03-09T09:12:43.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.739+0000 7f91af200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91b006ed50 0x7f91b00c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:43.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.739+0000 7f91af200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91b0073180 0x7f91b00ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:43.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.739+0000 7f91af200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b00c5320 0x7f91b01eec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:43.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.739+0000 7f91af200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b0159f20 con 0x7f91b0073180 2024-03-09T09:12:43.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.739+0000 7f91af200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00c3920 con 0x7f91b006ed50 2024-03-09T09:12:43.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.739+0000 7f91af200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b0073560 con 0x7f91b00c5320 2024-03-09T09:12:43.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.739+0000 7f91ad400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91b0073180 0x7f91b00ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:43.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ad400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91b0073180 0x7f91b00ad440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47774/0 (socket says 172.21.15.67:47774) 2024-03-09T09:12:43.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ad400640 1 -- 172.21.15.67:0/1342573387 learned_addr learned my addr 172.21.15.67:0/1342573387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:43.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ade00640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91b006ed50 0x7f91b00c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:43.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ae800640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b00c5320 0x7f91b01eec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:43.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ae800640 1 -- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b00c5320 msgr2=0x7f91b01eec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ae800640 1 -- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b00c5320 msgr2=0x7f91b01eec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ae800640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b00c5320 0x7f91b01eec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ae800640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b00c5320 0x7f91b01eec40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ad400640 1 -- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b00c5320 msgr2=0x7f91b01eec40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ad400640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b00c5320 0x7f91b01eec40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ad400640 1 -- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91b006ed50 msgr2=0x7f91b00c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ad400640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91b006ed50 0x7f91b00c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ad400640 1 -- 172.21.15.67:0/1342573387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b00ad980 con 0x7f91b0073180 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.740+0000 7f91ad400640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91b0073180 0x7f91b00ad440 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f919c000c00 tx=0x7f919c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.741+0000 7f91a6000640 1 -- 172.21.15.67:0/1342573387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c034040 con 0x7f91b0073180 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.741+0000 7f91a6000640 1 -- 172.21.15.67:0/1342573387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c039d90 con 0x7f91b0073180 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.741+0000 7f91a6000640 1 -- 172.21.15.67:0/1342573387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c043610 con 0x7f91b0073180 2024-03-09T09:12:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.741+0000 7f91af200640 1 -- 172.21.15.67:0/1342573387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b014d970 con 0x7f91b0073180 2024-03-09T09:12:43.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.741+0000 7f91af200640 1 -- 172.21.15.67:0/1342573387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b01ef760 con 0x7f91b0073180 2024-03-09T09:12:43.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.742+0000 7f91af200640 1 -- 172.21.15.67:0/1342573387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b004f3c0 con 0x7f91b0073180 2024-03-09T09:12:43.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.743+0000 7f91a6000640 1 -- 172.21.15.67:0/1342573387 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f919c04a060 con 0x7f91b0073180 2024-03-09T09:12:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.745+0000 7f91a6000640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f919006b4d0 0x7f919006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.745+0000 7f91a6000640 1 -- 172.21.15.67:0/1342573387 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f919c0ac5c0 con 0x7f91b0073180 2024-03-09T09:12:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.745+0000 7f91ade00640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f919006b4d0 0x7f919006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.746+0000 7f91ade00640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f919006b4d0 0x7f919006d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:43.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:43.748+0000 7f91a6000640 1 -- 172.21.15.67:0/1342573387 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f919c03e070 con 0x7f91b0073180 2024-03-09T09:12:43.754 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.61 2024-03-09T09:12:43.869 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:12:43.871 INFO:tasks.workunit.client.1.smithi134.stdout:v0.62 2024-03-09T09:12:43.871 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 62 -eq 48 ']' 2024-03-09T09:12:43.871 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.62 2024-03-09T09:12:43.871 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.62 2024-03-09T09:12:43.871 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.62 2024-03-09T09:12:44.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.163+0000 7f91af200640 1 -- 172.21.15.67:0/1342573387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f91b0152cd0 con 0x7f91b0073180 2024-03-09T09:12:44.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.164+0000 7f91a6000640 1 -- 172.21.15.67:0/1342573387 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f919c07b040 con 0x7f91b0073180 2024-03-09T09:12:44.169 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":146,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:12:44.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.166+0000 7f91af200640 1 -- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f919006b4d0 msgr2=0x7f919006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:44.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.167+0000 7f91af200640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f919006b4d0 0x7f919006d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0008040 comp rx=0 tx=0).stop 2024-03-09T09:12:44.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.167+0000 7f91af200640 1 -- 172.21.15.67:0/1342573387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91b0073180 msgr2=0x7f91b00ad440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:44.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.167+0000 7f91af200640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91b0073180 0x7f91b00ad440 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f919c000c00 tx=0x7f919c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:12:44.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.167+0000 7f91af200640 1 -- 172.21.15.67:0/1342573387 shutdown_connections 2024-03-09T09:12:44.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.167+0000 7f91af200640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f919006b4d0 0x7f919006d990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:44.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.167+0000 7f91af200640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91b00c5320 0x7f91b01eec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:44.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.167+0000 7f91af200640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91b0073180 0x7f91b00ad440 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:44.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.167+0000 7f91af200640 1 --2- 172.21.15.67:0/1342573387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91b006ed50 0x7f91b00c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:44.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.167+0000 7f91af200640 1 -- 172.21.15.67:0/1342573387 >> 172.21.15.67:0/1342573387 conn(0x7f91b006c150 msgr2=0x7f91b006f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:44.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.167+0000 7f91af200640 1 -- 172.21.15.67:0/1342573387 shutdown_connections 2024-03-09T09:12:44.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:44.167+0000 7f91af200640 1 -- 172.21.15.67:0/1342573387 wait complete. 2024-03-09T09:12:47.191 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:12:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.391+0000 7fa680a00640 1 Processor -- start 2024-03-09T09:12:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.392+0000 7fa680a00640 1 -- start start 2024-03-09T09:12:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.393+0000 7fa680a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa67c159580 0x7fa67c0c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.393+0000 7fa680a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa67c159960 0x7fa67c1531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.393+0000 7fa680a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67c0c35d0 con 0x7fa67c0c3080 2024-03-09T09:12:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.393+0000 7fa680a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa67c0c2150 con 0x7fa67c159960 2024-03-09T09:12:47.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.393+0000 7fa680a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c0c0e50 con 0x7fa67c159580 2024-03-09T09:12:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.393+0000 7fa67be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa67c159960 0x7fa67c1531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.393+0000 7fa67be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa67c159960 0x7fa67c1531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47788/0 (socket says 172.21.15.67:47788) 2024-03-09T09:12:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.393+0000 7fa67be00640 1 -- 172.21.15.67:0/1099042953 learned_addr learned my addr 172.21.15.67:0/1099042953 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.394+0000 7fa67b400640 1 --2- 172.21.15.67:0/1099042953 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa67c159580 0x7fa67c0c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.394+0000 7fa67be00640 1 -- 172.21.15.67:0/1099042953 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa67c159580 msgr2=0x7fa67c0c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.394+0000 7fa67be00640 1 --2- 172.21.15.67:0/1099042953 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa67c159580 0x7fa67c0c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.394+0000 7fa67be00640 1 -- 172.21.15.67:0/1099042953 >> v1:172.21.15.134:6789/0 conn(0x7fa67c0c3080 legacy=0x7fa67c0c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:47.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.394+0000 7fa67be00640 1 -- 172.21.15.67:0/1099042953 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c0c1130 con 0x7fa67c159960 2024-03-09T09:12:47.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.395+0000 7fa67be00640 1 --2- 172.21.15.67:0/1099042953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa67c159960 0x7fa67c1531d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa674009870 tx=0x7fa67402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1626be21ba7eb434 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:47.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.395+0000 7fa67aa00640 1 -- 172.21.15.67:0/1099042953 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674035020 con 0x7fa67c159960 2024-03-09T09:12:47.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.395+0000 7fa67aa00640 1 -- 172.21.15.67:0/1099042953 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa674031640 con 0x7fa67c159960 2024-03-09T09:12:47.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.395+0000 7fa67aa00640 1 -- 172.21.15.67:0/1099042953 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67402d040 con 0x7fa67c159960 2024-03-09T09:12:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.396+0000 7fa680a00640 1 -- 172.21.15.67:0/1099042953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa67c159960 msgr2=0x7fa67c1531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.396+0000 7fa680a00640 1 --2- 172.21.15.67:0/1099042953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa67c159960 0x7fa67c1531d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa674009870 tx=0x7fa67402b8f0 comp rx=0 tx=0).stop 2024-03-09T09:12:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.396+0000 7fa680a00640 1 -- 172.21.15.67:0/1099042953 shutdown_connections 2024-03-09T09:12:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.396+0000 7fa680a00640 1 --2- 172.21.15.67:0/1099042953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa67c159960 0x7fa67c1531d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.396+0000 7fa680a00640 1 --2- 172.21.15.67:0/1099042953 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa67c159580 0x7fa67c0c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:47.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.396+0000 7fa680a00640 1 -- 172.21.15.67:0/1099042953 >> 172.21.15.67:0/1099042953 conn(0x7fa67c0a97d0 msgr2=0x7fa67c0a9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:47.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.396+0000 7fa680a00640 1 -- 172.21.15.67:0/1099042953 shutdown_connections 2024-03-09T09:12:47.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.396+0000 7fa680a00640 1 -- 172.21.15.67:0/1099042953 wait complete. 2024-03-09T09:12:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.398+0000 7fa680a00640 1 Processor -- start 2024-03-09T09:12:47.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.398+0000 7fa680a00640 1 -- start start 2024-03-09T09:12:47.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.399+0000 7fa680a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa67c0c3080 0x7fa67c1ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.399+0000 7fa680a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa67c159580 0x7fa67c071850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.399+0000 7fa680a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa67c159960 0x7fa67c071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.399+0000 7fa680a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c067280 con 0x7fa67c159580 2024-03-09T09:12:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.399+0000 7fa680a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c0c16f0 con 0x7fa67c0c3080 2024-03-09T09:12:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.399+0000 7fa680a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c067400 con 0x7fa67c159960 2024-03-09T09:12:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.399+0000 7fa672a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa67c159580 0x7fa67c071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.399+0000 7fa672a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa67c159580 0x7fa67c071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47802/0 (socket says 172.21.15.67:47802) 2024-03-09T09:12:47.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.399+0000 7fa672a00640 1 -- 172.21.15.67:0/2354791231 learned_addr learned my addr 172.21.15.67:0/2354791231 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.399+0000 7fa67b400640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa67c0c3080 0x7fa67c1ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa67be00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa67c159960 0x7fa67c071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa672a00640 1 -- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa67c159960 msgr2=0x7fa67c071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa672a00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa67c159960 0x7fa67c071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa672a00640 1 -- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa67c0c3080 msgr2=0x7fa67c1ee9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa672a00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa67c0c3080 0x7fa67c1ee9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa672a00640 1 -- 172.21.15.67:0/2354791231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c14b270 con 0x7fa67c159580 2024-03-09T09:12:47.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa67be00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa67c159960 0x7fa67c071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:12:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa672a00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa67c159580 0x7fa67c071850 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa66400c3c0 tx=0x7fa66400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa673e00640 1 -- 172.21.15.67:0/2354791231 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6640052d0 con 0x7fa67c159580 2024-03-09T09:12:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa673e00640 1 -- 172.21.15.67:0/2354791231 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa664005490 con 0x7fa67c159580 2024-03-09T09:12:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa673e00640 1 -- 172.21.15.67:0/2354791231 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa664007260 con 0x7fa67c159580 2024-03-09T09:12:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.400+0000 7fa680a00640 1 -- 172.21.15.67:0/2354791231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67c072aa0 con 0x7fa67c159580 2024-03-09T09:12:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.401+0000 7fa680a00640 1 -- 172.21.15.67:0/2354791231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c1ef4c0 con 0x7fa67c159580 2024-03-09T09:12:47.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.402+0000 7fa680a00640 1 -- 172.21.15.67:0/2354791231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67c04f3c0 con 0x7fa67c159580 2024-03-09T09:12:47.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.403+0000 7fa673e00640 1 -- 172.21.15.67:0/2354791231 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa664007400 con 0x7fa67c159580 2024-03-09T09:12:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.404+0000 7fa673e00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa65806b4d0 0x7fa65806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.404+0000 7fa673e00640 1 -- 172.21.15.67:0/2354791231 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa664089960 con 0x7fa67c159580 2024-03-09T09:12:47.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.405+0000 7fa67b400640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa65806b4d0 0x7fa65806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:47.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.406+0000 7fa67b400640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa65806b4d0 0x7fa65806d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa67c072ee0 tx=0x7fa66c009830 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:47.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.409+0000 7fa673e00640 1 -- 172.21.15.67:0/2354791231 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa664057e30 con 0x7fa67c159580 2024-03-09T09:12:47.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.696+0000 7fa680a00640 1 -- 172.21.15.67:0/2354791231 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa67c1eeee0 con 0x7fa67c159580 2024-03-09T09:12:47.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.697+0000 7fa673e00640 1 -- 172.21.15.67:0/2354791231 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fa66405bed0 con 0x7fa67c159580 2024-03-09T09:12:47.702 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":146,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:12:47.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 -- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa65806b4d0 msgr2=0x7fa65806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:47.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa65806b4d0 0x7fa65806d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa67c072ee0 tx=0x7fa66c009830 comp rx=0 tx=0).stop 2024-03-09T09:12:47.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 -- 172.21.15.67:0/2354791231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa67c159580 msgr2=0x7fa67c071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:47.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa67c159580 0x7fa67c071850 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa66400c3c0 tx=0x7fa66400c880 comp rx=0 tx=0).stop 2024-03-09T09:12:47.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 -- 172.21.15.67:0/2354791231 shutdown_connections 2024-03-09T09:12:47.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa65806b4d0 0x7fa65806d990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:47.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa67c159960 0x7fa67c071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:47.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa67c159580 0x7fa67c071850 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:47.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 --2- 172.21.15.67:0/2354791231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa67c0c3080 0x7fa67c1ee9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:47.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 -- 172.21.15.67:0/2354791231 >> 172.21.15.67:0/2354791231 conn(0x7fa67c0a97d0 msgr2=0x7fa67c152fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:47.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 -- 172.21.15.67:0/2354791231 shutdown_connections 2024-03-09T09:12:47.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:47.700+0000 7fa680a00640 1 -- 172.21.15.67:0/2354791231 wait complete. 2024-03-09T09:12:49.321 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 6% (4/59) Updating files: 8% (5/59) Updating files: 10% (6/59) Updating files: 11% (7/59) Updating files: 13% (8/59) Updating files: 15% (9/59) Updating files: 16% (10/59) Updating files: 18% (11/59) Updating files: 20% (12/59) Updating files: 22% (13/59) Updating files: 23% (14/59) Updating files: 25% (15/59) Updating files: 27% (16/59) Updating files: 28% (17/59) Updating files: 30% (18/59) Updating files: 32% (19/59) Updating files: 33% (20/59) Updating files: 35% (21/59) Updating files: 37% (22/59) Updating files: 38% (23/59) Updating files: 40% (24/59) Updating files: 42% (25/59) Updating files: 44% (26/59) Updating files: 45% (27/59) Updating files: 47% (28/59) Updating files: 49% (29/59) Updating files: 50% (30/59) Updating files: 52% (31/59) Updating files: 54% (32/59) Updating files: 55% (33/59) Updating files: 57% (34/59) Updating files: 59% (35/59) Updating files: 61% (36/59) Updating files: 62% (37/59) Updating files: 64% (38/59) Updating files: 66% (39/59) Updating files: 67% (40/59) Updating files: 69% (41/59) Updating files: 71% (42/59) Updating files: 72% (43/59) Updating files: 74% (44/59) Updating files: 76% (45/59) Updating files: 77% (46/59) Updating files: 79% (47/59) Updating files: 81% (48/59) Updating files: 83% (49/59) Updating files: 84% (50/59) Updating files: 86% (51/59) Updating files: 88% (52/59) Updating files: 89% (53/59) Updating files: 91% (54/59) Updating files: 93% (55/59) Updating files: 94% (56/59) Updating files: 96% (57/59) Updating files: 98% (58/59) Updating files: 100% (59/59) Updating files: 100% (59/59), done. 2024-03-09T09:12:49.639 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 9382379cc6b v0.62 2024-03-09T09:12:49.645 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.62 2024-03-09T09:12:49.674 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 6% (4/59) Updating files: 8% (5/59) Updating files: 10% (6/59) Updating files: 11% (7/59) Updating files: 13% (8/59) Updating files: 15% (9/59) Updating files: 16% (10/59) Updating files: 18% (11/59) Updating files: 20% (12/59) Updating files: 22% (13/59) Updating files: 23% (14/59) Updating files: 25% (15/59) Updating files: 27% (16/59) Updating files: 28% (17/59) Updating files: 30% (18/59) Updating files: 32% (19/59) Updating files: 33% (20/59) Updating files: 35% (21/59) Updating files: 37% (22/59) Updating files: 38% (23/59) Updating files: 40% (24/59) Updating files: 42% (25/59) Updating files: 44% (26/59) Updating files: 45% (27/59) Updating files: 47% (28/59) Updating files: 49% (29/59) Updating files: 50% (30/59) Updating files: 52% (31/59) Updating files: 54% (32/59) Updating files: 55% (33/59) Updating files: 57% (34/59) Updating files: 59% (35/59) Updating files: 61% (36/59) Updating files: 62% (37/59) Updating files: 64% (38/59) Updating files: 66% (39/59) Updating files: 67% (40/59) Updating files: 69% (41/59) Updating files: 71% (42/59) Updating files: 72% (43/59) Updating files: 74% (44/59) Updating files: 76% (45/59) Updating files: 77% (46/59) Updating files: 79% (47/59) Updating files: 81% (48/59) Updating files: 83% (49/59) Updating files: 84% (50/59) Updating files: 86% (51/59) Updating files: 88% (52/59) Updating files: 89% (53/59) Updating files: 91% (54/59) Updating files: 93% (55/59) Updating files: 94% (56/59) Updating files: 96% (57/59) Updating files: 98% (58/59) Updating files: 100% (59/59) Updating files: 100% (59/59), done. 2024-03-09T09:12:49.725 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 9382379cc6b v0.62 2024-03-09T09:12:49.730 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.62 2024-03-09T09:12:49.900 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:12:49.901 INFO:tasks.workunit.client.0.smithi067.stdout:v0.63 2024-03-09T09:12:49.901 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 63 -eq 48 ']' 2024-03-09T09:12:49.901 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.63 2024-03-09T09:12:49.901 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.63 2024-03-09T09:12:49.901 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.63 2024-03-09T09:12:49.966 INFO:tasks.workunit.client.1.smithi134.stdout:v0.63 2024-03-09T09:12:49.967 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:12:49.967 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 63 -eq 48 ']' 2024-03-09T09:12:49.967 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.63 2024-03-09T09:12:49.967 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.63 2024-03-09T09:12:49.967 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.63 2024-03-09T09:12:50.724 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:12:50.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.925+0000 7f0d32e00640 1 Processor -- start 2024-03-09T09:12:50.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.926+0000 7f0d32e00640 1 -- start start 2024-03-09T09:12:50.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.927+0000 7f0d32e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:50.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.928+0000 7f0d32e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2c157000 con 0x7f0d2c153300 2024-03-09T09:12:50.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.928+0000 7f0d32e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2c0ae430 con 0x7f0d2c0c5e40 2024-03-09T09:12:50.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.928+0000 7f0d32e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0ae6f0 con 0x7f0d2c0cadb0 2024-03-09T09:12:50.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.928+0000 7f0d31a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0d2c153300 0x7f0d2c0b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47304/0 (socket says 172.21.15.67:47304) 2024-03-09T09:12:50.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.928+0000 7f0d31a00640 1 -- 172.21.15.67:0/1471432868 learned_addr learned my addr 172.21.15.67:0/1471432868 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:12:50.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.928+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:50.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.929+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 msgr2=0x7f0d2c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:12:50.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.929+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 msgr2=0x7f0d2c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:50.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.929+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:50.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:50.929+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:51.130+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:51.131+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 msgr2=0x7f0d2c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:12:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:51.131+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 msgr2=0x7f0d2c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:51.131+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:51.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:51.131+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:12:51.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:51.532+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:51.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:51.533+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 msgr2=0x7f0d2c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:12:51.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:51.533+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 msgr2=0x7f0d2c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:51.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:51.533+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:51.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:51.533+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:12:52.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.335+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:52.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.335+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 msgr2=0x7f0d2c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:12:52.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.335+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 msgr2=0x7f0d2c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:52.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.335+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:52.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.335+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:12:52.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.949+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745131523 0 0) 0x7f0d2c0ae430 con 0x7f0d2c0c5e40 2024-03-09T09:12:52.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.949+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d14003520 con 0x7f0d2c0c5e40 2024-03-09T09:12:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1845859147 0 0) 0x7f0d14003520 con 0x7f0d2c0c5e40 2024-03-09T09:12:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 msgr2=0x7f0d2c156720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:12:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d2c156720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 >> v1:172.21.15.67:6789/0 conn(0x7f0d2c153300 legacy=0x7f0d2c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c155b10 con 0x7f0d2c0c5e40 2024-03-09T09:12:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f0d18001670 con 0x7f0d2c0c5e40 2024-03-09T09:12:52.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 >> v1:172.21.15.134:6789/0 conn(0x7f0d2c0c5e40 legacy=0x7f0d2c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d14003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d140040a0 0x7f0d14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14004d90 0x7f0d140051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0ae6f0 con 0x7f0d140040a0 2024-03-09T09:12:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1400bc00 con 0x7f0d14004d90 2024-03-09T09:12:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.950+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1400be00 con 0x7f0d2c0cadb0 2024-03-09T09:12:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.951+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14004d90 0x7f0d140051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.951+0000 7f0d31a00640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d14003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:52.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.951+0000 7f0d31000640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d140040a0 0x7f0d14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.952+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 msgr2=0x7f0d14003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.952+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c0cadb0 0x7f0d14003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.952+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d140040a0 msgr2=0x7f0d14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.952+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d140040a0 0x7f0d14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:52.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.952+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d14003520 con 0x7f0d14004d90 2024-03-09T09:12:52.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.954+0000 7f0d32400640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14004d90 0x7f0d140051f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0d24011b90 tx=0x7f0d14005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f0fa58e7c3e2834 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:52.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.954+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2403a070 con 0x7f0d14004d90 2024-03-09T09:12:52.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.955+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d24035410 con 0x7f0d14004d90 2024-03-09T09:12:52.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.955+0000 7f0d2be00640 1 -- 172.21.15.67:0/1471432868 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24011790 con 0x7f0d14004d90 2024-03-09T09:12:52.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.956+0000 7f0d32e00640 1 -- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14004d90 msgr2=0x7f0d140051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:52.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.956+0000 7f0d32e00640 1 --2- 172.21.15.67:0/1471432868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14004d90 0x7f0d140051f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0d24011b90 tx=0x7f0d14005830 comp rx=0 tx=0).stop 2024-03-09T09:12:52.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.956+0000 7f0d32400640 1 -- 172.21.15.67:0/1471432868 reap_dead start 2024-03-09T09:12:52.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.956+0000 7f0d32e00640 1 -- 172.21.15.67:0/1471432868 shutdown_connections 2024-03-09T09:12:52.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.956+0000 7f0d32e00640 1 -- 172.21.15.67:0/1471432868 >> 172.21.15.67:0/1471432868 conn(0x7f0d2c0732d0 msgr2=0x7f0d2c0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:52.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.956+0000 7f0d32e00640 1 -- 172.21.15.67:0/1471432868 shutdown_connections 2024-03-09T09:12:52.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.956+0000 7f0d32e00640 1 -- 172.21.15.67:0/1471432868 wait complete. 2024-03-09T09:12:52.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.959+0000 7f0d32e00640 1 Processor -- start 2024-03-09T09:12:52.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.959+0000 7f0d32e00640 1 -- start start 2024-03-09T09:12:52.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.960+0000 7f0d32e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14004d90 0x7f0d2c14cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:52.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.960+0000 7f0d32e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c06b040 0x7f0d2c0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:52.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.960+0000 7f0d32e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c1ec090 0x7f0d2c1eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:52.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.960+0000 7f0d32e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0c59e0 con 0x7f0d2c06b040 2024-03-09T09:12:52.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.960+0000 7f0d32e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c159e90 con 0x7f0d14004d90 2024-03-09T09:12:52.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.960+0000 7f0d32e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b0710 con 0x7f0d2c1ec090 2024-03-09T09:12:52.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.961+0000 7f0d31000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c06b040 0x7f0d2c0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.961+0000 7f0d31000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c06b040 0x7f0d2c0c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53972/0 (socket says 172.21.15.67:53972) 2024-03-09T09:12:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.961+0000 7f0d31000640 1 -- 172.21.15.67:0/2247740707 learned_addr learned my addr 172.21.15.67:0/2247740707 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.961+0000 7f0d31a00640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14004d90 0x7f0d2c14cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.961+0000 7f0d32400640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c1ec090 0x7f0d2c1eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.961+0000 7f0d31000640 1 -- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c1ec090 msgr2=0x7f0d2c1eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.961+0000 7f0d31000640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c1ec090 0x7f0d2c1eed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.961+0000 7f0d31000640 1 -- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14004d90 msgr2=0x7f0d2c14cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.961+0000 7f0d31000640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14004d90 0x7f0d2c14cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.961+0000 7f0d31000640 1 -- 172.21.15.67:0/2247740707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c0c8430 con 0x7f0d2c06b040 2024-03-09T09:12:52.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.962+0000 7f0d31000640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c06b040 0x7f0d2c0c6960 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800f910 tx=0x7f0d1800fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:52.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.966+0000 7f0d29600640 1 -- 172.21.15.67:0/2247740707 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d180052e0 con 0x7f0d2c06b040 2024-03-09T09:12:52.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.966+0000 7f0d29600640 1 -- 172.21.15.67:0/2247740707 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d18018070 con 0x7f0d2c06b040 2024-03-09T09:12:52.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.966+0000 7f0d29600640 1 -- 172.21.15.67:0/2247740707 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800d970 con 0x7f0d2c06b040 2024-03-09T09:12:52.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.966+0000 7f0d32e00640 1 -- 172.21.15.67:0/2247740707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2c1eb8d0 con 0x7f0d2c06b040 2024-03-09T09:12:52.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.966+0000 7f0d32e00640 1 -- 172.21.15.67:0/2247740707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2c06bcc0 con 0x7f0d2c06b040 2024-03-09T09:12:52.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.968+0000 7f0d32e00640 1 -- 172.21.15.67:0/2247740707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c04f3c0 con 0x7f0d2c06b040 2024-03-09T09:12:52.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.969+0000 7f0d29600640 1 -- 172.21.15.67:0/2247740707 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0d1800db10 con 0x7f0d2c06b040 2024-03-09T09:12:52.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.970+0000 7f0d29600640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0d0c06b4d0 0x7f0d0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:52.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.971+0000 7f0d29600640 1 -- 172.21.15.67:0/2247740707 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0d1808a620 con 0x7f0d2c06b040 2024-03-09T09:12:52.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.971+0000 7f0d31a00640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0d0c06b4d0 0x7f0d0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:52.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.973+0000 7f0d31a00640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0d0c06b4d0 0x7f0d0c06d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0d20001f20 tx=0x7f0d20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:52.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:52.981+0000 7f0d29600640 1 -- 172.21.15.67:0/2247740707 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0d18059140 con 0x7f0d2c06b040 2024-03-09T09:12:53.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.365+0000 7f0d32e00640 1 -- 172.21.15.67:0/2247740707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0d2c0cab90 con 0x7f0d2c06b040 2024-03-09T09:12:53.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.366+0000 7f0d29600640 1 -- 172.21.15.67:0/2247740707 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f0d18058a40 con 0x7f0d2c06b040 2024-03-09T09:12:53.371 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":150,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:12:53.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 -- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0d0c06b4d0 msgr2=0x7f0d0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:53.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0d0c06b4d0 0x7f0d0c06d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0d20001f20 tx=0x7f0d20009040 comp rx=0 tx=0).stop 2024-03-09T09:12:53.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 -- 172.21.15.67:0/2247740707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c06b040 msgr2=0x7f0d2c0c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c06b040 0x7f0d2c0c6960 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800f910 tx=0x7f0d1800fdd0 comp rx=0 tx=0).stop 2024-03-09T09:12:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 -- 172.21.15.67:0/2247740707 shutdown_connections 2024-03-09T09:12:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0d0c06b4d0 0x7f0d0c06d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d2c1ec090 0x7f0d2c1eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2c06b040 0x7f0d2c0c6960 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 --2- 172.21.15.67:0/2247740707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d14004d90 0x7f0d2c14cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 -- 172.21.15.67:0/2247740707 >> 172.21.15.67:0/2247740707 conn(0x7f0d2c0732d0 msgr2=0x7f0d2c150ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 -- 172.21.15.67:0/2247740707 shutdown_connections 2024-03-09T09:12:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:53.369+0000 7f0d32e00640 1 -- 172.21.15.67:0/2247740707 wait complete. 2024-03-09T09:12:53.911 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 2% (4/146) Updating files: 3% (5/146) Updating files: 4% (6/146) Updating files: 5% (8/146) Updating files: 6% (9/146) Updating files: 7% (11/146) Updating files: 8% (12/146) Updating files: 9% (14/146) Updating files: 10% (15/146) Updating files: 11% (17/146) Updating files: 12% (18/146) Updating files: 13% (19/146) Updating files: 14% (21/146) Updating files: 15% (22/146) Updating files: 16% (24/146) Updating files: 17% (25/146) Updating files: 18% (27/146) Updating files: 19% (28/146) Updating files: 20% (30/146) Updating files: 21% (31/146) Updating files: 22% (33/146) Updating files: 23% (34/146) Updating files: 24% (36/146) Updating files: 25% (37/146) Updating files: 26% (38/146) Updating files: 27% (40/146) Updating files: 28% (41/146) Updating files: 29% (43/146) Updating files: 30% (44/146) Updating files: 31% (46/146) Updating files: 32% (47/146) Updating files: 33% (49/146) Updating files: 34% (50/146) Updating files: 35% (52/146) Updating files: 36% (53/146) Updating files: 37% (55/146) Updating files: 38% (56/146) Updating files: 39% (57/146) Updating files: 40% (59/146) Updating files: 41% (60/146) Updating files: 42% (62/146) Updating files: 43% (63/146) Updating files: 44% (65/146) Updating files: 45% (66/146) Updating files: 46% (68/146) Updating files: 47% (69/146) Updating files: 48% (71/146) Updating files: 49% (72/146) Updating files: 50% (73/146) Updating files: 51% (75/146) Updating files: 52% (76/146) Updating files: 53% (78/146) Updating files: 54% (79/146) Updating files: 55% (81/146) Updating files: 56% (82/146) Updating files: 57% (84/146) Updating files: 58% (85/146) Updating files: 59% (87/146) Updating files: 60% (88/146) Updating files: 61% (90/146) Updating files: 62% (91/146) Updating files: 63% (92/146) Updating files: 64% (94/146) Updating files: 65% (95/146) Updating files: 66% (97/146) Updating files: 67% (98/146) Updating files: 68% (100/146) Updating files: 69% (101/146) Updating files: 70% (103/146) Updating files: 71% (104/146) Updating files: 72% (106/146) Updating files: 73% (107/146) Updating files: 74% (109/146) Updating files: 75% (110/146) Updating files: 76% (111/146) Updating files: 77% (113/146) Updating files: 78% (114/146) Updating files: 79% (116/146) Updating files: 80% (117/146) Updating files: 81% (119/146) Updating files: 82% (120/146) Updating files: 83% (122/146) Updating files: 84% (123/146) Updating files: 85% (125/146) Updating files: 86% (126/146) Updating files: 87% (128/146) Updating files: 88% (129/146) Updating files: 89% (130/146) Updating files: 90% (132/146) Updating files: 91% (133/146) Updating files: 92% (135/146) Updating files: 93% (136/146) Updating files: 94% (138/146) Updating files: 95% (139/146) Updating files: 96% (141/146) Updating files: 97% (142/146) Updating files: 98% (144/146) Updating files: 99% (145/146) Updating files: 100% (146/146) Updating files: 100% (146/146), done. 2024-03-09T09:12:53.921 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 35% (52/146) Updating files: 36% (53/146) Updating files: 37% (55/146) Updating files: 38% (56/146) Updating files: 39% (57/146) Updating files: 40% (59/146) Updating files: 41% (60/146) Updating files: 42% (62/146) Updating files: 43% (63/146) Updating files: 44% (65/146) Updating files: 45% (66/146) Updating files: 46% (68/146) Updating files: 47% (69/146) Updating files: 48% (71/146) Updating files: 49% (72/146) Updating files: 50% (73/146) Updating files: 51% (75/146) Updating files: 52% (76/146) Updating files: 53% (78/146) Updating files: 54% (79/146) Updating files: 55% (81/146) Updating files: 56% (82/146) Updating files: 57% (84/146) Updating files: 58% (85/146) Updating files: 59% (87/146) Updating files: 60% (88/146) Updating files: 61% (90/146) Updating files: 62% (91/146) Updating files: 63% (92/146) Updating files: 64% (94/146) Updating files: 65% (95/146) Updating files: 66% (97/146) Updating files: 67% (98/146) Updating files: 68% (100/146) Updating files: 69% (101/146) Updating files: 70% (103/146) Updating files: 71% (104/146) Updating files: 72% (106/146) Updating files: 73% (107/146) Updating files: 74% (109/146) Updating files: 75% (110/146) Updating files: 76% (111/146) Updating files: 77% (113/146) Updating files: 78% (114/146) Updating files: 79% (116/146) Updating files: 80% (117/146) Updating files: 81% (119/146) Updating files: 82% (120/146) Updating files: 83% (122/146) Updating files: 84% (123/146) Updating files: 85% (125/146) Updating files: 86% (126/146) Updating files: 87% (128/146) Updating files: 88% (129/146) Updating files: 89% (130/146) Updating files: 90% (132/146) Updating files: 91% (133/146) Updating files: 92% (135/146) Updating files: 93% (136/146) Updating files: 94% (138/146) Updating files: 95% (139/146) Updating files: 96% (141/146) Updating files: 97% (142/146) Updating files: 98% (144/146) Updating files: 99% (145/146) Updating files: 100% (146/146) Updating files: 100% (146/146), done. 2024-03-09T09:12:53.984 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 054e96cf79e v0.63 2024-03-09T09:12:54.003 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.63 2024-03-09T09:12:54.187 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 054e96cf79e v0.63 2024-03-09T09:12:54.199 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.63 2024-03-09T09:12:54.280 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:12:54.280 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 64 -eq 48 ']' 2024-03-09T09:12:54.280 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.64 2024-03-09T09:12:54.280 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.64 2024-03-09T09:12:54.280 INFO:tasks.workunit.client.0.smithi067.stdout:v0.64 2024-03-09T09:12:54.281 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.64 2024-03-09T09:12:54.353 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:12:54.353 INFO:tasks.workunit.client.1.smithi134.stdout:v0.64 2024-03-09T09:12:54.354 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 64 -eq 48 ']' 2024-03-09T09:12:54.354 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.64 2024-03-09T09:12:54.354 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.64 2024-03-09T09:12:54.354 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.64 2024-03-09T09:12:56.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:12:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.615+0000 7fcc04e00640 1 Processor -- start 2024-03-09T09:12:56.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.615+0000 7fcc04e00640 1 -- start start 2024-03-09T09:12:56.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.616+0000 7fcc04e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc000c4330 0x7fcc00157210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.616+0000 7fcc04e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcc0006ffd0 0x7fcc00157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.616+0000 7fcc04e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc00153ff0 con 0x7fcc00151510 2024-03-09T09:12:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.616+0000 7fcc04e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc001542d0 con 0x7fcc0006ffd0 2024-03-09T09:12:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.616+0000 7fcc04e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc001545b0 con 0x7fcc000c4330 2024-03-09T09:12:56.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbfe800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcc0006ffd0 0x7fcc00157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbfe800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcc0006ffd0 0x7fcc00157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:49112/0 (socket says 172.21.15.67:49112) 2024-03-09T09:12:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbfe800640 1 -- 172.21.15.67:0/1510144229 learned_addr learned my addr 172.21.15.67:0/1510144229 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbf7e00640 1 --2- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc000c4330 0x7fcc00157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbf7e00640 1 -- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc000c4330 msgr2=0x7fcc00157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:12:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbf7e00640 1 -- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc000c4330 msgr2=0x7fcc00157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbf7e00640 1 --2- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc000c4330 0x7fcc00157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbf7e00640 1 --2- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc000c4330 0x7fcc00157210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbfe800640 1 -- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc000c4330 msgr2=0x7fcc00157210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:12:56.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbfe800640 1 --2- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc000c4330 0x7fcc00157210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbfe800640 1 -- 172.21.15.67:0/1510144229 >> v1:172.21.15.134:6789/0 conn(0x7fcc00151510 legacy=0x7fcc000729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:56.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.617+0000 7fcbfe800640 1 -- 172.21.15.67:0/1510144229 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc00154890 con 0x7fcc0006ffd0 2024-03-09T09:12:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.619+0000 7fcbfe800640 1 --2- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc0006ffd0 0x7fcc00157750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009e50 tx=0x7fcbf802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71eeecff3258bbd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.619+0000 7fcbfd400640 1 -- 172.21.15.67:0/1510144229 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf802cb00 con 0x7fcc0006ffd0 2024-03-09T09:12:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.619+0000 7fcbfd400640 1 -- 172.21.15.67:0/1510144229 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf802cca0 con 0x7fcc0006ffd0 2024-03-09T09:12:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.619+0000 7fcbfd400640 1 -- 172.21.15.67:0/1510144229 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8034610 con 0x7fcc0006ffd0 2024-03-09T09:12:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.620+0000 7fcc04e00640 1 -- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc0006ffd0 msgr2=0x7fcc00157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.620+0000 7fcc04e00640 1 --2- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc0006ffd0 0x7fcc00157750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009e50 tx=0x7fcbf802bf60 comp rx=0 tx=0).stop 2024-03-09T09:12:56.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.620+0000 7fcc04e00640 1 -- 172.21.15.67:0/1510144229 shutdown_connections 2024-03-09T09:12:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.620+0000 7fcc04e00640 1 --2- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc0006ffd0 0x7fcc00157750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.620+0000 7fcc04e00640 1 --2- 172.21.15.67:0/1510144229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc000c4330 0x7fcc00157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.620+0000 7fcc04e00640 1 -- 172.21.15.67:0/1510144229 >> 172.21.15.67:0/1510144229 conn(0x7fcc000ab320 msgr2=0x7fcc000ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.620+0000 7fcc04e00640 1 -- 172.21.15.67:0/1510144229 shutdown_connections 2024-03-09T09:12:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.620+0000 7fcc04e00640 1 -- 172.21.15.67:0/1510144229 wait complete. 2024-03-09T09:12:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.622+0000 7fcc04e00640 1 Processor -- start 2024-03-09T09:12:56.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.622+0000 7fcc04e00640 1 -- start start 2024-03-09T09:12:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.623+0000 7fcc04e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc0006ffd0 0x7fcc000c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.623+0000 7fcc04e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc000c4330 0x7fcc001e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.623+0000 7fcc04e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc001ea040 0x7fcc001462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.623+0000 7fcc04e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc00073250 con 0x7fcc001ea040 2024-03-09T09:12:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.623+0000 7fcc04e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc000730d0 con 0x7fcc000c4330 2024-03-09T09:12:56.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.623+0000 7fcc04e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc000c3f70 con 0x7fcc0006ffd0 2024-03-09T09:12:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfe800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc001ea040 0x7fcc001462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfde00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc0006ffd0 0x7fcc000c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfe800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc001ea040 0x7fcc001462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53974/0 (socket says 172.21.15.67:53974) 2024-03-09T09:12:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfe800640 1 -- 172.21.15.67:0/436592881 learned_addr learned my addr 172.21.15.67:0/436592881 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbf7e00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc000c4330 0x7fcc001e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfde00640 1 -- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc0006ffd0 msgr2=0x7fcc000c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:12:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfde00640 1 -- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc0006ffd0 msgr2=0x7fcc000c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:12:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfde00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc0006ffd0 0x7fcc000c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfde00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc0006ffd0 0x7fcc000c4ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:56.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfe800640 1 -- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc0006ffd0 msgr2=0x7fcc000c4ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:12:56.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfe800640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc0006ffd0 0x7fcc000c4ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:56.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfe800640 1 -- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc000c4330 msgr2=0x7fcc001e9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:56.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfe800640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc000c4330 0x7fcc001e9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:56.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbfe800640 1 -- 172.21.15.67:0/436592881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc000c55d0 con 0x7fcc001ea040 2024-03-09T09:12:56.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.624+0000 7fcbf7e00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc000c4330 0x7fcc001e9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:12:56.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.625+0000 7fcbfe800640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc001ea040 0x7fcc001462a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009e50 tx=0x7fcbf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:56.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.625+0000 7fcbf6000640 1 -- 172.21.15.67:0/436592881 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8043020 con 0x7fcc001ea040 2024-03-09T09:12:56.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.625+0000 7fcbf6000640 1 -- 172.21.15.67:0/436592881 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf802ce50 con 0x7fcc001ea040 2024-03-09T09:12:56.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.625+0000 7fcbf6000640 1 -- 172.21.15.67:0/436592881 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8009510 con 0x7fcc001ea040 2024-03-09T09:12:56.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.625+0000 7fcc04e00640 1 -- 172.21.15.67:0/436592881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc000aaec0 con 0x7fcc001ea040 2024-03-09T09:12:56.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.626+0000 7fcc04e00640 1 -- 172.21.15.67:0/436592881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc001ed5a0 con 0x7fcc001ea040 2024-03-09T09:12:56.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.627+0000 7fcc04e00640 1 -- 172.21.15.67:0/436592881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc00075a00 con 0x7fcc001ea040 2024-03-09T09:12:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.628+0000 7fcbf6000640 1 -- 172.21.15.67:0/436592881 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcbf8046020 con 0x7fcc001ea040 2024-03-09T09:12:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.629+0000 7fcbf6000640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:56.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.630+0000 7fcbf6000640 1 -- 172.21.15.67:0/436592881 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcbf80acc20 con 0x7fcc001ea040 2024-03-09T09:12:56.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.630+0000 7fcbfde00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:56.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.631+0000 7fcbfde00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0007fd0 tx=0x7fcbf0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:56.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.633+0000 7fcbf6000640 1 -- 172.21.15.67:0/436592881 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcbf804d030 con 0x7fcc001ea040 2024-03-09T09:12:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.915+0000 7fcc04e00640 1 -- 172.21.15.67:0/436592881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcc00070b10 con 0x7fcc001ea040 2024-03-09T09:12:56.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.916+0000 7fcbf6000640 1 -- 172.21.15.67:0/436592881 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fcbf807b6a0 con 0x7fcc001ea040 2024-03-09T09:12:56.921 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:12:56.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.918+0000 7fcc04e00640 1 -- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbdc06b4d0 msgr2=0x7fcbdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:56.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.918+0000 7fcc04e00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0007fd0 tx=0x7fcbf0009040 comp rx=0 tx=0).stop 2024-03-09T09:12:56.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.918+0000 7fcc04e00640 1 -- 172.21.15.67:0/436592881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc001ea040 msgr2=0x7fcc001462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:56.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.918+0000 7fcc04e00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc001ea040 0x7fcc001462a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009e50 tx=0x7fcbf8004a40 comp rx=0 tx=0).stop 2024-03-09T09:12:56.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.919+0000 7fcc04e00640 1 -- 172.21.15.67:0/436592881 shutdown_connections 2024-03-09T09:12:56.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.919+0000 7fcc04e00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:56.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.919+0000 7fcc04e00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc001ea040 0x7fcc001462a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:56.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.919+0000 7fcc04e00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc000c4330 0x7fcc001e9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:56.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.919+0000 7fcc04e00640 1 --2- 172.21.15.67:0/436592881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc0006ffd0 0x7fcc000c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:56.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.919+0000 7fcc04e00640 1 -- 172.21.15.67:0/436592881 >> 172.21.15.67:0/436592881 conn(0x7fcc000ab320 msgr2=0x7fcc001534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:56.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.919+0000 7fcc04e00640 1 -- 172.21.15.67:0/436592881 shutdown_connections 2024-03-09T09:12:56.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:56.919+0000 7fcc04e00640 1 -- 172.21.15.67:0/436592881 wait complete. 2024-03-09T09:12:56.941 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:12:56.941 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:12:56.941 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:12:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.127+0000 7f29cd800640 1 Processor -- start 2024-03-09T09:12:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.128+0000 7f29cd800640 1 -- start start 2024-03-09T09:12:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.129+0000 7f29cd800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f29c80acf00 0x7f29c80c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.129+0000 7f29cd800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c80c97d0 con 0x7f29c80c5e00 2024-03-09T09:12:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.129+0000 7f29cd800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c80ad7c0 con 0x7f29c80c9380 2024-03-09T09:12:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.129+0000 7f29cd800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c806d740 con 0x7f29c80acf00 2024-03-09T09:12:57.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.129+0000 7f29c7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f29c80acf00 0x7f29c80c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.129+0000 7f29c7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f29c80c5e00 0x7f29c806c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47316/0 (socket says 172.21.15.67:47316) 2024-03-09T09:12:57.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.129+0000 7f29c7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f29c80acf00 0x7f29c80c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53978/0 (socket says 172.21.15.67:53978) 2024-03-09T09:12:57.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.129+0000 7f29c7e00640 1 -- 172.21.15.67:0/2968106411 learned_addr learned my addr 172.21.15.67:0/2968106411 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:12:57.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.130+0000 7f29c7400640 1 -- 172.21.15.67:0/2968106411 >> v1:172.21.15.134:6789/0 conn(0x7f29c80c9380 legacy=0x7f29c8160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.130+0000 7f29c7400640 1 -- 172.21.15.67:0/2968106411 >> v1:172.21.15.67:6789/0 conn(0x7f29c80c5e00 legacy=0x7f29c806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.130+0000 7f29c7400640 1 -- 172.21.15.67:0/2968106411 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c80ad4e0 con 0x7f29c80acf00 2024-03-09T09:12:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.130+0000 7f29c6a00640 1 -- 172.21.15.67:0/2968106411 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409078303 0 0) 0x7f29c80c97d0 con 0x7f29c80c5e00 2024-03-09T09:12:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.131+0000 7f29c7400640 1 --2- 172.21.15.67:0/2968106411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80acf00 0x7f29c80c61e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f29b40097d0 tx=0x7f29b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9902fd3a42748ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.131+0000 7f29c6a00640 1 -- 172.21.15.67:0/2968106411 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4035020 con 0x7f29c80acf00 2024-03-09T09:12:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.132+0000 7f29c6a00640 1 -- 172.21.15.67:0/2968106411 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4031830 con 0x7f29c80acf00 2024-03-09T09:12:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.132+0000 7f29c6a00640 1 -- 172.21.15.67:0/2968106411 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4031c00 con 0x7f29c80acf00 2024-03-09T09:12:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.132+0000 7f29cd800640 1 -- 172.21.15.67:0/2968106411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80acf00 msgr2=0x7f29c80c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.132+0000 7f29cd800640 1 --2- 172.21.15.67:0/2968106411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80acf00 0x7f29c80c61e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f29b40097d0 tx=0x7f29b402b8d0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.132+0000 7f29cd800640 1 -- 172.21.15.67:0/2968106411 shutdown_connections 2024-03-09T09:12:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.132+0000 7f29cd800640 1 --2- 172.21.15.67:0/2968106411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80acf00 0x7f29c80c61e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.132+0000 7f29cd800640 1 -- 172.21.15.67:0/2968106411 >> 172.21.15.67:0/2968106411 conn(0x7f29c80cb480 msgr2=0x7f29c80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.132+0000 7f29cd800640 1 -- 172.21.15.67:0/2968106411 shutdown_connections 2024-03-09T09:12:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.132+0000 7f29cd800640 1 -- 172.21.15.67:0/2968106411 wait complete. 2024-03-09T09:12:57.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.134+0000 7f29cd800640 1 Processor -- start 2024-03-09T09:12:57.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.134+0000 7f29cd800640 1 -- start start 2024-03-09T09:12:57.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.135+0000 7f29cd800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c80acf00 0x7f29c814c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.135+0000 7f29cd800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c5e00 0x7f29c80c13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.135+0000 7f29cd800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29c80c9380 0x7f29c81ede40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.135+0000 7f29cd800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c806bde0 con 0x7f29c80c5e00 2024-03-09T09:12:57.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.135+0000 7f29cd800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c8153680 con 0x7f29c80c9380 2024-03-09T09:12:57.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.135+0000 7f29cd800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c806c0c0 con 0x7f29c80acf00 2024-03-09T09:12:57.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.135+0000 7f29c7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c5e00 0x7f29c80c13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.135+0000 7f29c7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c5e00 0x7f29c80c13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53992/0 (socket says 172.21.15.67:53992) 2024-03-09T09:12:57.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.135+0000 7f29c7400640 1 -- 172.21.15.67:0/4231353303 learned_addr learned my addr 172.21.15.67:0/4231353303 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:57.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29c7e00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c80acf00 0x7f29c814c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29cce00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29c80c9380 0x7f29c81ede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29c7e00640 1 -- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c80acf00 msgr2=0x7f29c814c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:12:57.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29c7e00640 1 -- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c80acf00 msgr2=0x7f29c814c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:12:57.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29c7e00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c80acf00 0x7f29c814c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:57.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29c7400640 1 -- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c80acf00 msgr2=0x7f29c814c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29c7e00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c80acf00 0x7f29c814c880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:57.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29c7400640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29c80acf00 0x7f29c814c880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29c7400640 1 -- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29c80c9380 msgr2=0x7f29c81ede40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29c7400640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29c80c9380 0x7f29c81ede40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29c7400640 1 -- 172.21.15.67:0/4231353303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c80bf930 con 0x7f29c80c5e00 2024-03-09T09:12:57.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29cce00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29c80c9380 0x7f29c81ede40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:12:57.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.136+0000 7f29c7400640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c5e00 0x7f29c80c13a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f29b40097d0 tx=0x7f29b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.137+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4040070 con 0x7f29c80c5e00 2024-03-09T09:12:57.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.137+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4039a00 con 0x7f29c80c5e00 2024-03-09T09:12:57.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.137+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4035040 con 0x7f29c80c5e00 2024-03-09T09:12:57.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.137+0000 7f29cd800640 1 -- 172.21.15.67:0/4231353303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c80bfc10 con 0x7f29c80c5e00 2024-03-09T09:12:57.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.137+0000 7f29cd800640 1 -- 172.21.15.67:0/4231353303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c80c01d0 con 0x7f29c80c5e00 2024-03-09T09:12:57.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.138+0000 7f29cd800640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c03a0 0x7f29c81ee380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.138+0000 7f29cd800640 1 -- 172.21.15.67:0/4231353303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c81ec980 con 0x7f29c80c03a0 2024-03-09T09:12:57.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.138+0000 7f29c7e00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c03a0 0x7f29c81ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.139+0000 7f29c7e00640 1 -- 172.21.15.67:0/4231353303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29c804f3c0 con 0x7f29c80c03a0 2024-03-09T09:12:57.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.139+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f29b404b060 con 0x7f29c80c5e00 2024-03-09T09:12:57.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.139+0000 7f29c7e00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c03a0 0x7f29c81ee380 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f29c80acc10 tx=0x7f29b80016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.140+0000 7f29bfe00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a806b500 0x7f29a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.140+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f29b40abfa0 con 0x7f29c80c5e00 2024-03-09T09:12:57.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.140+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8019070 con 0x7f29c80c03a0 2024-03-09T09:12:57.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.140+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f29b8010510 con 0x7f29c80c03a0 2024-03-09T09:12:57.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.140+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c03a0 msgr2=0x7f29c81ee380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.140+0000 7f29bfe00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c03a0 0x7f29c81ee380 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f29c80acc10 tx=0x7f29b80016e0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.140+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c03a0 msgr2=0x7f29c81ee380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:57.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.140+0000 7f29bfe00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c03a0 0x7f29c81ee380 secure :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f29c80acc10 tx=0x7f29b80016e0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.141+0000 7f29cce00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a806b500 0x7f29a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.142+0000 7f29cce00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a806b500 0x7f29a806d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f29c80c0b50 tx=0x7f29c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.148+0000 7f29cd800640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c8159c40 0x7f29c81ecc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.148+0000 7f29cd800640 1 -- 172.21.15.67:0/4231353303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c806d2f0 con 0x7f29c8159c40 2024-03-09T09:12:57.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.148+0000 7f29c7e00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c8159c40 0x7f29c81ecc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.149+0000 7f29c7e00640 1 -- 172.21.15.67:0/4231353303 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f29c80affe0 con 0x7f29c8159c40 2024-03-09T09:12:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.149+0000 7f29c7e00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c8159c40 0x7f29c81ecc60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f29b8016000 tx=0x7f29b8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.149+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8019040 con 0x7f29c8159c40 2024-03-09T09:12:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f29b801c6c0 con 0x7f29c8159c40 2024-03-09T09:12:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c8159c40 msgr2=0x7f29c81ecc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29bfe00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c8159c40 0x7f29c81ecc60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f29b8016000 tx=0x7f29b8000ed0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29bfe00640 1 -- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c8159c40 msgr2=0x7f29c81ecc60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:57.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29bfe00640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c8159c40 0x7f29c81ecc60 secure :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f29b8016000 tx=0x7f29b8000ed0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.156 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:12:57.156 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:12:57.156 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:12:57.156 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 152, 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:57.157 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:12:57.158 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:12:57.159 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:12:57.160 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:12:57.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:57.162 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:12:57.163 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:12:57.164 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:12:57.165 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:57.166 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:57.167 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:57.168 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.169 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.170 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:12:57.170 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:12:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29cd800640 1 -- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a806b500 msgr2=0x7f29a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29cd800640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f29a806b500 0x7f29a806d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f29c80c0b50 tx=0x7f29c0009040 comp rx=0 tx=0).stop 2024-03-09T09:12:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29cd800640 1 -- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c5e00 msgr2=0x7f29c80c13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29cd800640 1 --2- 172.21.15.67:0/4231353303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29c80c5e00 0x7f29c80c13a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f29b40097d0 tx=0x7f29b40046c0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29cce00640 1 -- 172.21.15.67:0/4231353303 reap_dead start 2024-03-09T09:12:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29cce00640 1 -- 172.21.15.67:0/4231353303 reap_dead start 2024-03-09T09:12:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29cd800640 1 -- 172.21.15.67:0/4231353303 shutdown_connections 2024-03-09T09:12:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29cd800640 1 -- 172.21.15.67:0/4231353303 >> 172.21.15.67:0/4231353303 conn(0x7f29c80cb480 msgr2=0x7f29c80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29cd800640 1 -- 172.21.15.67:0/4231353303 shutdown_connections 2024-03-09T09:12:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.150+0000 7f29cd800640 1 -- 172.21.15.67:0/4231353303 wait complete. 2024-03-09T09:12:57.172 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:12:57.172 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:12:57.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:12:57.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.369+0000 7f7d2fe00640 1 Processor -- start 2024-03-09T09:12:57.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.369+0000 7f7d2fe00640 1 -- start start 2024-03-09T09:12:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d30153c00 0x7f7d30153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2fe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d300aaa40 con 0x7f7d300c7130 2024-03-09T09:12:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2fe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d300aad20 con 0x7f7d30066e90 2024-03-09T09:12:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300ab000 con 0x7f7d30153c00 2024-03-09T09:12:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7d300c7130 0x7f7d300c5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47320/0 (socket says 172.21.15.67:47320) 2024-03-09T09:12:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2f400640 1 -- 172.21.15.67:0/3507978712 learned_addr learned my addr 172.21.15.67:0/3507978712 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:12:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2ea00640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d30153c00 0x7f7d30153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2ea00640 1 -- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d30153c00 msgr2=0x7f7d30153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:12:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2ea00640 1 -- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d30153c00 msgr2=0x7f7d30153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2ea00640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d30153c00 0x7f7d30153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2ea00640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d30153c00 0x7f7d30153fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927059149 0 0) 0x7f7d300aaa40 con 0x7f7d300c7130 2024-03-09T09:12:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d14003520 con 0x7f7d300c7130 2024-03-09T09:12:57.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1357505004 0 0) 0x7f7d300aad20 con 0x7f7d30066e90 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d300aaa40 con 0x7f7d30066e90 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3544122521 0 0) 0x7f7d14003520 con 0x7f7d300c7130 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d30153c00 msgr2=0x7f7d30153fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d30153c00 0x7f7d30153fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 >> v1:172.21.15.134:6789/0 conn(0x7f7d30066e90 legacy=0x7f7d300c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d300ad100 con 0x7f7d300c7130 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f7d280019f0 con 0x7f7d300c7130 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 >> v1:172.21.15.67:6789/0 conn(0x7f7d300c7130 legacy=0x7f7d300c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d14003be0 0x7f7d14004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d14004800 0x7f7d14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d30153c00 0x7f7d1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300ab000 con 0x7f7d14003be0 2024-03-09T09:12:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d14005570 con 0x7f7d14004800 2024-03-09T09:12:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.371+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1400bf00 con 0x7f7d30153c00 2024-03-09T09:12:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.372+0000 7f7d2ea00640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d14003be0 0x7f7d14004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.372+0000 7f7d27e00640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d14004800 0x7f7d14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.372+0000 7f7d2ea00640 1 -- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d30153c00 msgr2=0x7f7d1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.372+0000 7f7d2ea00640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d30153c00 0x7f7d1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.372+0000 7f7d2ea00640 1 -- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d14004800 msgr2=0x7f7d14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.372+0000 7f7d2ea00640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d14004800 0x7f7d14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.372+0000 7f7d2ea00640 1 -- 172.21.15.67:0/3507978712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d14003520 con 0x7f7d14003be0 2024-03-09T09:12:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.373+0000 7f7d2ea00640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d14003be0 0x7f7d14004000 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000b9e0 tx=0x7f7d14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e973bddcb5a078d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.373+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20036020 con 0x7f7d14003be0 2024-03-09T09:12:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.373+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20032620 con 0x7f7d14003be0 2024-03-09T09:12:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.373+0000 7f7d2e000640 1 -- 172.21.15.67:0/3507978712 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000b5e0 con 0x7f7d14003be0 2024-03-09T09:12:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.374+0000 7f7d2fe00640 1 -- 172.21.15.67:0/3507978712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d14003be0 msgr2=0x7f7d14004000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.374+0000 7f7d2fe00640 1 --2- 172.21.15.67:0/3507978712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d14003be0 0x7f7d14004000 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000b9e0 tx=0x7f7d14005830 comp rx=0 tx=0).stop 2024-03-09T09:12:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.374+0000 7f7d2f400640 1 -- 172.21.15.67:0/3507978712 reap_dead start 2024-03-09T09:12:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.374+0000 7f7d2fe00640 1 -- 172.21.15.67:0/3507978712 shutdown_connections 2024-03-09T09:12:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.374+0000 7f7d2fe00640 1 -- 172.21.15.67:0/3507978712 >> 172.21.15.67:0/3507978712 conn(0x7f7d300a9740 msgr2=0x7f7d300a9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.374+0000 7f7d2fe00640 1 -- 172.21.15.67:0/3507978712 shutdown_connections 2024-03-09T09:12:57.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.374+0000 7f7d2fe00640 1 -- 172.21.15.67:0/3507978712 wait complete. 2024-03-09T09:12:57.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.377+0000 7f7d2fe00640 1 Processor -- start 2024-03-09T09:12:57.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.377+0000 7f7d2fe00640 1 -- start start 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.378+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d14003be0 0x7f7d300c1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.378+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d3006ea90 0x7f7d301ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.378+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d301ec050 0x7f7d301eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.378+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300c5fa0 con 0x7f7d14003be0 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.378+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300ad6c0 con 0x7f7d301ec050 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.378+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300c6120 con 0x7f7d3006ea90 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.378+0000 7f7d2ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d14003be0 0x7f7d300c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.378+0000 7f7d2ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d14003be0 0x7f7d300c1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54024/0 (socket says 172.21.15.67:54024) 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.378+0000 7f7d2ea00640 1 -- 172.21.15.67:0/2911135637 learned_addr learned my addr 172.21.15.67:0/2911135637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.379+0000 7f7d27e00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d3006ea90 0x7f7d301ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.379+0000 7f7d2f400640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d301ec050 0x7f7d301eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.379+0000 7f7d2ea00640 1 -- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d3006ea90 msgr2=0x7f7d301ebb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.379+0000 7f7d2ea00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7d3006ea90 0x7f7d301ebb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.379+0000 7f7d2ea00640 1 -- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d301ec050 msgr2=0x7f7d301eeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.379+0000 7f7d2ea00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d301ec050 0x7f7d301eeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.379+0000 7f7d2ea00640 1 -- 172.21.15.67:0/2911135637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d300711f0 con 0x7f7d14003be0 2024-03-09T09:12:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.379+0000 7f7d2f400640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d301ec050 0x7f7d301eeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:12:57.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.379+0000 7f7d2ea00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d14003be0 0x7f7d300c1030 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000bb00 tx=0x7f7d20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.380+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2003b620 con 0x7f7d14003be0 2024-03-09T09:12:57.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.380+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2003b7c0 con 0x7f7d14003be0 2024-03-09T09:12:57.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.380+0000 7f7d2fe00640 1 -- 172.21.15.67:0/2911135637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d301ec4d0 con 0x7f7d14003be0 2024-03-09T09:12:57.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.380+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2003bb90 con 0x7f7d14003be0 2024-03-09T09:12:57.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.380+0000 7f7d2fe00640 1 -- 172.21.15.67:0/2911135637 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d30070c30 con 0x7f7d14003be0 2024-03-09T09:12:57.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.381+0000 7f7d2fe00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d300715d0 0x7f7d300c3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.381+0000 7f7d2fe00640 1 -- 172.21.15.67:0/2911135637 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d30150a20 con 0x7f7d300715d0 2024-03-09T09:12:57.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.381+0000 7f7d27e00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d300715d0 0x7f7d300c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.382+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7d20039020 con 0x7f7d14003be0 2024-03-09T09:12:57.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.382+0000 7f7d27e00640 1 -- 172.21.15.67:0/2911135637 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d301eeb20 con 0x7f7d300715d0 2024-03-09T09:12:57.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.382+0000 7f7d27e00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d300715d0 0x7f7d300c3580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d300c3310 tx=0x7f7d180017a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.383+0000 7f7d27400640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d0c06b500 0x7f7d0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.383+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f7d200ac330 con 0x7f7d14003be0 2024-03-09T09:12:57.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.383+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18013570 con 0x7f7d300715d0 2024-03-09T09:12:57.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.383+0000 7f7d2f400640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d0c06b500 0x7f7d0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.384+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f7d18019070 con 0x7f7d300715d0 2024-03-09T09:12:57.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.384+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d300715d0 msgr2=0x7f7d300c3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.384+0000 7f7d27400640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d300715d0 0x7f7d300c3580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d300c3310 tx=0x7f7d180017a0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.384+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d300715d0 msgr2=0x7f7d300c3580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:57.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.384+0000 7f7d27400640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d300715d0 0x7f7d300c3580 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d300c3310 tx=0x7f7d180017a0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.384+0000 7f7d2f400640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d0c06b500 0x7f7d0c06d9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7d3006ca80 tx=0x7f7d280018d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.391+0000 7f7d2fe00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d3002beb0 0x7f7d3002c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.391+0000 7f7d2fe00640 1 -- 172.21.15.67:0/2911135637 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3004f3c0 con 0x7f7d3002beb0 2024-03-09T09:12:57.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.392+0000 7f7d27e00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d3002beb0 0x7f7d3002c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.392+0000 7f7d27e00640 1 -- 172.21.15.67:0/2911135637 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7d30070f10 con 0x7f7d3002beb0 2024-03-09T09:12:57.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.393+0000 7f7d27e00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d3002beb0 0x7f7d3002c2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d18015000 tx=0x7f7d18007a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.393+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18013b90 con 0x7f7d3002beb0 2024-03-09T09:12:57.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.394+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f7d18013b90 con 0x7f7d3002beb0 2024-03-09T09:12:57.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.394+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d3002beb0 msgr2=0x7f7d3002c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.394+0000 7f7d27400640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d3002beb0 0x7f7d3002c2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d18015000 tx=0x7f7d18007a70 comp rx=0 tx=0).stop 2024-03-09T09:12:57.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.394+0000 7f7d27400640 1 -- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d3002beb0 msgr2=0x7f7d3002c2c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:57.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.394+0000 7f7d27400640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d3002beb0 0x7f7d3002c2c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:12:57.400 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:12:57.400 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:12:57.401 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:12:57.401 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 152, 2024-03-09T09:12:57.401 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:12:57.401 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:12:57.401 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:12:57.401 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:12:57.401 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:57.401 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:12:57.401 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:57.402 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:12:57.402 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:12:57.402 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:57.402 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:57.402 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:57.402 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:57.402 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:57.402 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:57.402 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:57.403 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:57.403 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:57.403 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:57.403 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:57.403 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:57.403 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:12:57.403 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:12:57.403 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:57.404 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:57.404 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:57.404 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:57.404 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:57.404 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:57.404 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:57.404 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:57.404 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:57.404 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:57.405 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:57.405 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.405 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:12:57.405 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:12:57.405 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:12:57.405 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:12:57.405 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:12:57.405 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:12:57.405 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:57.405 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:57.406 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:57.407 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:12:57.408 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.410 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:57.412 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:12:57.413 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:12:57.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.395+0000 7f7d2fe00640 1 -- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d0c06b500 msgr2=0x7f7d0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.395+0000 7f7d2fe00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7d0c06b500 0x7f7d0c06d9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7d3006ca80 tx=0x7f7d280018d0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.395+0000 7f7d2fe00640 1 -- 172.21.15.67:0/2911135637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d14003be0 msgr2=0x7f7d300c1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.395+0000 7f7d2fe00640 1 --2- 172.21.15.67:0/2911135637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d14003be0 0x7f7d300c1030 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000bb00 tx=0x7f7d20004730 comp rx=0 tx=0).stop 2024-03-09T09:12:57.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.395+0000 7f7d2f400640 1 -- 172.21.15.67:0/2911135637 reap_dead start 2024-03-09T09:12:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.395+0000 7f7d2f400640 1 -- 172.21.15.67:0/2911135637 reap_dead start 2024-03-09T09:12:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.395+0000 7f7d2fe00640 1 -- 172.21.15.67:0/2911135637 shutdown_connections 2024-03-09T09:12:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.395+0000 7f7d2fe00640 1 -- 172.21.15.67:0/2911135637 >> 172.21.15.67:0/2911135637 conn(0x7f7d300a9740 msgr2=0x7f7d300ac650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.395+0000 7f7d2fe00640 1 -- 172.21.15.67:0/2911135637 shutdown_connections 2024-03-09T09:12:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.395+0000 7f7d2fe00640 1 -- 172.21.15.67:0/2911135637 wait complete. 2024-03-09T09:12:57.416 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:12:57.416 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:12:57.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:12:57.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.609+0000 7effa6000640 1 Processor -- start 2024-03-09T09:12:57.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.610+0000 7effa6000640 1 -- start start 2024-03-09T09:12:57.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.611+0000 7effa6000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa014c9a0 0x7effa01567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.611+0000 7effa6000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7effa00c1d70 0x7effa00c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.611+0000 7effa6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7effa00c0ce0 0x7effa00ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.611+0000 7effa6000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa006d590 con 0x7effa00c0ce0 2024-03-09T09:12:57.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.611+0000 7effa6000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa00ac4e0 con 0x7effa00c1d70 2024-03-09T09:12:57.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.611+0000 7effa6000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa0156d00 con 0x7effa014c9a0 2024-03-09T09:12:57.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7effa00c0ce0 0x7effa00ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7effa00c0ce0 0x7effa00ac9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54038/0 (socket says 172.21.15.67:54038) 2024-03-09T09:12:57.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9fe00640 1 -- 172.21.15.67:0/2206150438 learned_addr learned my addr 172.21.15.67:0/2206150438 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:57.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff97e00640 1 --2- 172.21.15.67:0/2206150438 >> v2:172.21.15.134:3300/0 conn(0x7effa00c1d70 0x7effa00c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9f400640 1 --2- 172.21.15.67:0/2206150438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa014c9a0 0x7effa01567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9f400640 1 -- 172.21.15.67:0/2206150438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa014c9a0 msgr2=0x7effa01567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9f400640 1 -- 172.21.15.67:0/2206150438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa014c9a0 msgr2=0x7effa01567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9f400640 1 --2- 172.21.15.67:0/2206150438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa014c9a0 0x7effa01567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9f400640 1 --2- 172.21.15.67:0/2206150438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa014c9a0 0x7effa01567c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9fe00640 1 -- 172.21.15.67:0/2206150438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa014c9a0 msgr2=0x7effa01567c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9fe00640 1 --2- 172.21.15.67:0/2206150438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa014c9a0 0x7effa01567c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9fe00640 1 -- 172.21.15.67:0/2206150438 >> v2:172.21.15.134:3300/0 conn(0x7effa00c1d70 msgr2=0x7effa00c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9fe00640 1 --2- 172.21.15.67:0/2206150438 >> v2:172.21.15.134:3300/0 conn(0x7effa00c1d70 0x7effa00c07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.612+0000 7eff9fe00640 1 -- 172.21.15.67:0/2206150438 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa00aa550 con 0x7effa00c0ce0 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.613+0000 7eff9fe00640 1 --2- 172.21.15.67:0/2206150438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effa00c0ce0 0x7effa00ac9c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7effa006ea30 tx=0x7eff9802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b073b158b5344f44 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.614+0000 7eff9ea00640 1 -- 172.21.15.67:0/2206150438 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9802c9e0 con 0x7effa00c0ce0 2024-03-09T09:12:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.614+0000 7eff9ea00640 1 -- 172.21.15.67:0/2206150438 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9802cb80 con 0x7effa00c0ce0 2024-03-09T09:12:57.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.614+0000 7eff9ea00640 1 -- 172.21.15.67:0/2206150438 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff980343f0 con 0x7effa00c0ce0 2024-03-09T09:12:57.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.614+0000 7effa6000640 1 -- 172.21.15.67:0/2206150438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effa00c0ce0 msgr2=0x7effa00ac9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.614+0000 7effa6000640 1 --2- 172.21.15.67:0/2206150438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effa00c0ce0 0x7effa00ac9c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7effa006ea30 tx=0x7eff9802be40 comp rx=0 tx=0).stop 2024-03-09T09:12:57.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.615+0000 7effa6000640 1 -- 172.21.15.67:0/2206150438 shutdown_connections 2024-03-09T09:12:57.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.615+0000 7effa6000640 1 --2- 172.21.15.67:0/2206150438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effa00c0ce0 0x7effa00ac9c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.615+0000 7effa6000640 1 --2- 172.21.15.67:0/2206150438 >> v2:172.21.15.134:3300/0 conn(0x7effa00c1d70 0x7effa00c07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.615+0000 7effa6000640 1 --2- 172.21.15.67:0/2206150438 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa014c9a0 0x7effa01567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.615+0000 7effa6000640 1 -- 172.21.15.67:0/2206150438 >> 172.21.15.67:0/2206150438 conn(0x7effa00c7370 msgr2=0x7effa00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:57.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.615+0000 7effa6000640 1 -- 172.21.15.67:0/2206150438 shutdown_connections 2024-03-09T09:12:57.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.615+0000 7effa6000640 1 -- 172.21.15.67:0/2206150438 wait complete. 2024-03-09T09:12:57.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.616+0000 7effa6000640 1 Processor -- start 2024-03-09T09:12:57.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.617+0000 7effa6000640 1 -- start start 2024-03-09T09:12:57.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.617+0000 7effa6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effa00c0ce0 0x7effa01ec870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.617+0000 7effa6000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa00c1d70 0x7effa0071000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.617+0000 7effa6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effa014c9a0 0x7effa0148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.617+0000 7effa6000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00aadf0 con 0x7effa00c0ce0 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.617+0000 7effa6000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00aab10 con 0x7effa014c9a0 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.617+0000 7effa6000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa00acf00 con 0x7effa00c1d70 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.618+0000 7eff9f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effa00c0ce0 0x7effa01ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.618+0000 7eff9f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effa00c0ce0 0x7effa01ec870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54040/0 (socket says 172.21.15.67:54040) 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.618+0000 7eff9f400640 1 -- 172.21.15.67:0/4256466665 learned_addr learned my addr 172.21.15.67:0/4256466665 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.618+0000 7eff9fe00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effa014c9a0 0x7effa0148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.618+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa00c1d70 0x7effa0071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.618+0000 7eff9f400640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa00c1d70 msgr2=0x7effa0071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.618+0000 7eff9f400640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa00c1d70 0x7effa0071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.618+0000 7eff9f400640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effa014c9a0 msgr2=0x7effa0148980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.618+0000 7eff9f400640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effa014c9a0 0x7effa0148980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:57.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.618+0000 7eff9f400640 1 -- 172.21.15.67:0/4256466665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa01df620 con 0x7effa00c0ce0 2024-03-09T09:12:57.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.619+0000 7eff9f400640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effa00c0ce0 0x7effa01ec870 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7eff9000d7d0 tx=0x7eff9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.619+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9000b6c0 con 0x7effa00c0ce0 2024-03-09T09:12:57.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.619+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff90019070 con 0x7effa00c0ce0 2024-03-09T09:12:57.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.619+0000 7effa6000640 1 -- 172.21.15.67:0/4256466665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa0158210 con 0x7effa00c0ce0 2024-03-09T09:12:57.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.619+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9000bac0 con 0x7effa00c0ce0 2024-03-09T09:12:57.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.619+0000 7effa6000640 1 -- 172.21.15.67:0/4256466665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa01587d0 con 0x7effa00c0ce0 2024-03-09T09:12:57.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.620+0000 7effa6000640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.620+0000 7effa6000640 1 -- 172.21.15.67:0/4256466665 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa01ecdb0 con 0x7effa0159ef0 2024-03-09T09:12:57.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.621+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.621+0000 7eff97e00640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 msgr2=0x7effa0148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-03-09T09:12:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.621+0000 7eff97e00640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 msgr2=0x7effa0148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:12:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.621+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.621+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:12:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.621+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7eff9000bc60 con 0x7effa00c0ce0 2024-03-09T09:12:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.622+0000 7eff97400640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7eff7c06b500 0x7eff7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.622+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7eff9008a920 con 0x7effa00c0ce0 2024-03-09T09:12:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.623+0000 7eff9fe00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7eff7c06b500 0x7eff7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.624+0000 7eff9fe00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7eff7c06b500 0x7eff7c06d9c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7eff98035000 tx=0x7eff98030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:57.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.822+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:57.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.824+0000 7eff97e00640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 msgr2=0x7effa0148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:12:57.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.824+0000 7eff97e00640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 msgr2=0x7effa0148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:12:57.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.824+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:57.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:57.824+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:12:58.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:58.225+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:58.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:58.225+0000 7eff97e00640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 msgr2=0x7effa0148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:12:58.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:58.225+0000 7eff97e00640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 msgr2=0x7effa0148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:12:58.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:58.225+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:12:58.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:58.225+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:12:59.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.027+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:59.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.028+0000 7eff97e00640 1 -- 172.21.15.67:0/4256466665 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7effa01df3d0 con 0x7effa0159ef0 2024-03-09T09:12:59.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.029+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7effa00736b0 tx=0x7eff88017b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:59.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.030+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88015b80 con 0x7effa0159ef0 2024-03-09T09:12:59.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.031+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7eff88015b80 con 0x7effa0159ef0 2024-03-09T09:12:59.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.031+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 msgr2=0x7effa0148ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:59.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.031+0000 7eff97400640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7effa00736b0 tx=0x7eff88017b10 comp rx=0 tx=0).stop 2024-03-09T09:12:59.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.031+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 msgr2=0x7effa0148ec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:59.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.031+0000 7eff97400640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0159ef0 0x7effa0148ec0 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7effa00736b0 tx=0x7eff88017b10 comp rx=0 tx=0).stop 2024-03-09T09:12:59.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.040+0000 7effa6000640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0158cf0 0x7effa0159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:59.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.040+0000 7effa6000640 1 -- 172.21.15.67:0/4256466665 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa0158a10 con 0x7effa0158cf0 2024-03-09T09:12:59.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.041+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0158cf0 0x7effa0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:59.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.042+0000 7eff97e00640 1 -- 172.21.15.67:0/4256466665 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7effa0149400 con 0x7effa0158cf0 2024-03-09T09:12:59.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.043+0000 7eff97e00640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0158cf0 0x7effa0159100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7effa00736b0 tx=0x7eff88004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:59.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.044+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88028040 con 0x7effa0158cf0 2024-03-09T09:12:59.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.044+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7eff88028570 con 0x7effa0158cf0 2024-03-09T09:12:59.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.044+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0158cf0 msgr2=0x7effa0159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:59.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.044+0000 7eff97400640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0158cf0 0x7effa0159100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7effa00736b0 tx=0x7eff88004730 comp rx=0 tx=0).stop 2024-03-09T09:12:59.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.044+0000 7eff97400640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0158cf0 msgr2=0x7effa0159100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:12:59.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.044+0000 7eff97400640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effa0158cf0 0x7effa0159100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:59.049 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 152, 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 5, 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:12:59.050 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:12:59.051 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:12:59.052 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:12:59.053 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:59.054 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:59.055 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:12:59.056 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:12:59.057 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:59.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:59.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:12:59.060 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:12:59.061 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:12:59.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:12:59.061 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:12:59.061 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:12:59.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.046+0000 7effa6000640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7eff7c06b500 msgr2=0x7eff7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.046+0000 7effa6000640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7eff7c06b500 0x7eff7c06d9c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7eff98035000 tx=0x7eff98030000 comp rx=0 tx=0).stop 2024-03-09T09:12:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.046+0000 7effa6000640 1 -- 172.21.15.67:0/4256466665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effa00c0ce0 msgr2=0x7effa01ec870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.046+0000 7effa6000640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effa00c0ce0 0x7effa01ec870 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7eff9000d7d0 tx=0x7eff9000dc90 comp rx=0 tx=0).stop 2024-03-09T09:12:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.046+0000 7eff9fe00640 1 -- 172.21.15.67:0/4256466665 reap_dead start 2024-03-09T09:12:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.046+0000 7effa6000640 1 -- 172.21.15.67:0/4256466665 shutdown_connections 2024-03-09T09:12:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.046+0000 7effa6000640 1 --2- 172.21.15.67:0/4256466665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effa00c0ce0 0x7effa01ec870 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.046+0000 7effa6000640 1 -- 172.21.15.67:0/4256466665 >> 172.21.15.67:0/4256466665 conn(0x7effa00c7370 msgr2=0x7effa00c6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.046+0000 7effa6000640 1 -- 172.21.15.67:0/4256466665 shutdown_connections 2024-03-09T09:12:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.046+0000 7effa6000640 1 -- 172.21.15.67:0/4256466665 wait complete. 2024-03-09T09:12:59.075 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:12:59.075 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:12:59.075 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:12:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.299+0000 7f4938400640 1 Processor -- start 2024-03-09T09:12:59.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.299+0000 7f4938400640 1 -- start start 2024-03-09T09:12:59.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.300+0000 7f4938400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4930153370 0x7f49300c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:59.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.300+0000 7f4938400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4930153750 0x7f49300c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:59.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.300+0000 7f4938400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49300c4010 con 0x7f493006dc00 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.300+0000 7f4938400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4930156dc0 con 0x7f4930153370 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.300+0000 7f4938400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49300c42f0 con 0x7f4930153750 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.301+0000 7f4935a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4930153370 0x7f49300c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.301+0000 7f4936400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f493006dc00 0x7f49300729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56400/0 (socket says 172.21.15.67:56400) 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.301+0000 7f4936400640 1 -- 172.21.15.67:0/3402305998 learned_addr learned my addr 172.21.15.67:0/3402305998 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.301+0000 7f4935000640 1 --2- 172.21.15.67:0/3402305998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4930153750 0x7f49300c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.301+0000 7f4935a00640 1 -- 172.21.15.67:0/3402305998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4930153750 msgr2=0x7f49300c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.301+0000 7f4935a00640 1 --2- 172.21.15.67:0/3402305998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4930153750 0x7f49300c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.301+0000 7f4935a00640 1 -- 172.21.15.67:0/3402305998 >> v1:172.21.15.67:6789/0 conn(0x7f493006dc00 legacy=0x7f49300729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.301+0000 7f4935a00640 1 -- 172.21.15.67:0/3402305998 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49300c5a50 con 0x7f4930153370 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.301+0000 7f4935000640 1 --2- 172.21.15.67:0/3402305998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4930153750 0x7f49300c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:12:59.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.301+0000 7f492fe00640 1 -- 172.21.15.67:0/3402305998 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814293485 0 0) 0x7f49300c4010 con 0x7f493006dc00 2024-03-09T09:12:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.303+0000 7f4935a00640 1 --2- 172.21.15.67:0/3402305998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930153370 0x7f49300c29a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f49240097a0 tx=0x7f492402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f192aa31e19c861 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.303+0000 7f492fe00640 1 -- 172.21.15.67:0/3402305998 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924031780 con 0x7f4930153370 2024-03-09T09:12:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.303+0000 7f492fe00640 1 -- 172.21.15.67:0/3402305998 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4924037070 con 0x7f4930153370 2024-03-09T09:12:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.303+0000 7f492fe00640 1 -- 172.21.15.67:0/3402305998 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924031b40 con 0x7f4930153370 2024-03-09T09:12:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.304+0000 7f4938400640 1 -- 172.21.15.67:0/3402305998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930153370 msgr2=0x7f49300c29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.304+0000 7f4938400640 1 --2- 172.21.15.67:0/3402305998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930153370 0x7f49300c29a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f49240097a0 tx=0x7f492402b820 comp rx=0 tx=0).stop 2024-03-09T09:12:59.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.304+0000 7f4938400640 1 -- 172.21.15.67:0/3402305998 shutdown_connections 2024-03-09T09:12:59.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.304+0000 7f4938400640 1 --2- 172.21.15.67:0/3402305998 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4930153750 0x7f49300c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:59.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.304+0000 7f4938400640 1 --2- 172.21.15.67:0/3402305998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930153370 0x7f49300c29a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:59.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.304+0000 7f4938400640 1 -- 172.21.15.67:0/3402305998 >> 172.21.15.67:0/3402305998 conn(0x7f49300ab320 msgr2=0x7f49300ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:59.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.304+0000 7f4938400640 1 -- 172.21.15.67:0/3402305998 shutdown_connections 2024-03-09T09:12:59.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.304+0000 7f4938400640 1 -- 172.21.15.67:0/3402305998 wait complete. 2024-03-09T09:12:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.305+0000 7f4938400640 1 Processor -- start 2024-03-09T09:12:59.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.306+0000 7f4938400640 1 -- start start 2024-03-09T09:12:59.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.306+0000 7f4938400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f493006dc00 0x7f49301482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:59.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.306+0000 7f4938400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930153370 0x7f49300aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:59.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4938400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4930153750 0x7f49301ef090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:59.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4938400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300c5d30 con 0x7f4930153370 2024-03-09T09:12:59.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4938400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4930073dc0 con 0x7f493006dc00 2024-03-09T09:12:59.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4938400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49300c3420 con 0x7f4930153750 2024-03-09T09:12:59.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4935000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930153370 0x7f49300aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:59.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4935000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930153370 0x7f49300aa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38062/0 (socket says 172.21.15.67:38062) 2024-03-09T09:12:59.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4935000640 1 -- 172.21.15.67:0/3201477741 learned_addr learned my addr 172.21.15.67:0/3201477741 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:12:59.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4935a00640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f493006dc00 0x7f49301482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:59.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4936400640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4930153750 0x7f49301ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:59.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4935000640 1 -- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4930153750 msgr2=0x7f49301ef090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:59.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4935000640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4930153750 0x7f49301ef090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:59.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4935000640 1 -- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f493006dc00 msgr2=0x7f49301482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:59.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4935000640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f493006dc00 0x7f49301482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:59.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.307+0000 7f4935000640 1 -- 172.21.15.67:0/3201477741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49301eb7b0 con 0x7f4930153370 2024-03-09T09:12:59.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.308+0000 7f4935000640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930153370 0x7f49300aa1b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f491c00d7d0 tx=0x7f491c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:59.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.308+0000 7f492d600640 1 -- 172.21.15.67:0/3201477741 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c004030 con 0x7f4930153370 2024-03-09T09:12:59.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.308+0000 7f492d600640 1 -- 172.21.15.67:0/3201477741 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c0041d0 con 0x7f4930153370 2024-03-09T09:12:59.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.308+0000 7f492d600640 1 -- 172.21.15.67:0/3201477741 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c0096a0 con 0x7f4930153370 2024-03-09T09:12:59.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.308+0000 7f4938400640 1 -- 172.21.15.67:0/3201477741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49300ad6b0 con 0x7f4930153370 2024-03-09T09:12:59.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.308+0000 7f4938400640 1 -- 172.21.15.67:0/3201477741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49301557f0 con 0x7f4930153370 2024-03-09T09:12:59.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.309+0000 7f4938400640 1 -- 172.21.15.67:0/3201477741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4930075a00 con 0x7f4930153370 2024-03-09T09:12:59.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.310+0000 7f492d600640 1 -- 172.21.15.67:0/3201477741 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f491c009840 con 0x7f4930153370 2024-03-09T09:12:59.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.311+0000 7f492d600640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f491006b4d0 0x7f491006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:12:59.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.311+0000 7f492d600640 1 -- 172.21.15.67:0/3201477741 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f491c08a7e0 con 0x7f4930153370 2024-03-09T09:12:59.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.312+0000 7f4935a00640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f491006b4d0 0x7f491006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:12:59.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.313+0000 7f4935a00640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f491006b4d0 0x7f491006d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4924036000 tx=0x7f4924008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:12:59.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.315+0000 7f492d600640 1 -- 172.21.15.67:0/3201477741 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f491c059260 con 0x7f4930153370 2024-03-09T09:12:59.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.591+0000 7f4938400640 1 -- 172.21.15.67:0/3201477741 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f4930074c00 con 0x7f4930153370 2024-03-09T09:12:59.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.592+0000 7f492d600640 1 -- 172.21.15.67:0/3201477741 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f491c058b60 con 0x7f4930153370 2024-03-09T09:12:59.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.595+0000 7f4938400640 1 -- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f491006b4d0 msgr2=0x7f491006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:59.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.595+0000 7f4938400640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f491006b4d0 0x7f491006d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4924036000 tx=0x7f4924008030 comp rx=0 tx=0).stop 2024-03-09T09:12:59.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.596+0000 7f4938400640 1 -- 172.21.15.67:0/3201477741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930153370 msgr2=0x7f49300aa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:12:59.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.596+0000 7f4938400640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930153370 0x7f49300aa1b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f491c00d7d0 tx=0x7f491c00dc90 comp rx=0 tx=0).stop 2024-03-09T09:12:59.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.596+0000 7f4938400640 1 -- 172.21.15.67:0/3201477741 shutdown_connections 2024-03-09T09:12:59.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.596+0000 7f4938400640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f491006b4d0 0x7f491006d990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:59.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.596+0000 7f4938400640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4930153750 0x7f49301ef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:59.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.596+0000 7f4938400640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4930153370 0x7f49300aa1b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:59.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.596+0000 7f4938400640 1 --2- 172.21.15.67:0/3201477741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f493006dc00 0x7f49301482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:12:59.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.596+0000 7f4938400640 1 -- 172.21.15.67:0/3201477741 >> 172.21.15.67:0/3201477741 conn(0x7f49300ab320 msgr2=0x7f49300c8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:12:59.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.596+0000 7f4938400640 1 -- 172.21.15.67:0/3201477741 shutdown_connections 2024-03-09T09:12:59.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:12:59.596+0000 7f4938400640 1 -- 172.21.15.67:0/3201477741 wait complete. 2024-03-09T09:12:59.620 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:13:05.046 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 23% (33/143) Updating files: 24% (35/143) Updating files: 25% (36/143) Updating files: 26% (38/143) Updating files: 27% (39/143) Updating files: 28% (41/143) Updating files: 29% (42/143) Updating files: 30% (43/143) Updating files: 31% (45/143) Updating files: 32% (46/143) Updating files: 33% (48/143) Updating files: 34% (49/143) Updating files: 35% (51/143) Updating files: 36% (52/143) Updating files: 37% (53/143) Updating files: 38% (55/143) Updating files: 39% (56/143) Updating files: 40% (58/143) Updating files: 41% (59/143) Updating files: 42% (61/143) Updating files: 43% (62/143) Updating files: 44% (63/143) Updating files: 45% (65/143) Updating files: 46% (66/143) Updating files: 47% (68/143) Updating files: 48% (69/143) Updating files: 49% (71/143) Updating files: 50% (72/143) Updating files: 51% (73/143) Updating files: 52% (75/143) Updating files: 53% (76/143) Updating files: 54% (78/143) Updating files: 55% (79/143) Updating files: 56% (81/143) Updating files: 57% (82/143) Updating files: 58% (83/143) Updating files: 59% (85/143) Updating files: 60% (86/143) Updating files: 61% (88/143) Updating files: 62% (89/143) Updating files: 63% (91/143) Updating files: 64% (92/143) Updating files: 65% (93/143) Updating files: 66% (95/143) Updating files: 67% (96/143) Updating files: 68% (98/143) Updating files: 69% (99/143) Updating files: 70% (101/143) Updating files: 71% (102/143) Updating files: 72% (103/143) Updating files: 73% (105/143) Updating files: 74% (106/143) Updating files: 75% (108/143) Updating files: 76% (109/143) Updating files: 77% (111/143) Updating files: 78% (112/143) Updating files: 79% (113/143) Updating files: 80% (115/143) Updating files: 81% (116/143) Updating files: 82% (118/143) Updating files: 83% (119/143) Updating files: 84% (121/143) Updating files: 85% (122/143) Updating files: 86% (123/143) Updating files: 87% (125/143) Updating files: 88% (126/143) Updating files: 89% (128/143) Updating files: 90% (129/143) Updating files: 91% (131/143) Updating files: 92% (132/143) Updating files: 93% (133/143) Updating files: 94% (135/143) Updating files: 95% (136/143) Updating files: 96% (138/143) Updating files: 97% (139/143) Updating files: 98% (141/143) Updating files: 99% (142/143) Updating files: 100% (143/143) Updating files: 100% (143/143), done. 2024-03-09T09:13:05.095 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 42e06c12db6 v0.64 2024-03-09T09:13:05.113 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.64 2024-03-09T09:13:05.386 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:13:05.386 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 65 -eq 48 ']' 2024-03-09T09:13:05.386 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.65 2024-03-09T09:13:05.386 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.65 2024-03-09T09:13:05.386 INFO:tasks.workunit.client.0.smithi067.stdout:v0.65 2024-03-09T09:13:05.387 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.65 2024-03-09T09:13:08.459 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 15% (32/203) Updating files: 16% (33/203) Updating files: 17% (35/203) Updating files: 18% (37/203) Updating files: 19% (39/203) Updating files: 20% (41/203) Updating files: 21% (43/203) Updating files: 22% (45/203) Updating files: 23% (47/203) Updating files: 24% (49/203) Updating files: 25% (51/203) Updating files: 26% (53/203) Updating files: 27% (55/203) Updating files: 28% (57/203) Updating files: 29% (59/203) Updating files: 30% (61/203) Updating files: 31% (63/203) Updating files: 32% (65/203) Updating files: 33% (67/203) Updating files: 34% (70/203) Updating files: 35% (72/203) Updating files: 36% (74/203) Updating files: 37% (76/203) Updating files: 38% (78/203) Updating files: 39% (80/203) Updating files: 40% (82/203) Updating files: 41% (84/203) Updating files: 42% (86/203) Updating files: 43% (88/203) Updating files: 44% (90/203) Updating files: 45% (92/203) Updating files: 46% (94/203) Updating files: 47% (96/203) Updating files: 48% (98/203) Updating files: 49% (100/203) Updating files: 50% (102/203) Updating files: 51% (104/203) Updating files: 52% (106/203) Updating files: 53% (108/203) Updating files: 54% (110/203) Updating files: 55% (112/203) Updating files: 56% (114/203) Updating files: 57% (116/203) Updating files: 58% (118/203) Updating files: 59% (120/203) Updating files: 60% (122/203) Updating files: 61% (124/203) Updating files: 62% (126/203) Updating files: 63% (128/203) Updating files: 64% (130/203) Updating files: 65% (132/203) Updating files: 66% (134/203) Updating files: 67% (137/203) Updating files: 68% (139/203) Updating files: 69% (141/203) Updating files: 70% (143/203) Updating files: 71% (145/203) Updating files: 72% (147/203) Updating files: 73% (149/203) Updating files: 74% (151/203) Updating files: 75% (153/203) Updating files: 76% (155/203) Updating files: 77% (157/203) Updating files: 78% (159/203) Updating files: 79% (161/203) Updating files: 79% (162/203) Updating files: 80% (163/203) Updating files: 81% (165/203) Updating files: 82% (167/203) Updating files: 83% (169/203) Updating files: 84% (171/203) Updating files: 85% (173/203) Updating files: 86% (175/203) Updating files: 87% (177/203) Updating files: 88% (179/203) Updating files: 89% (181/203) Updating files: 90% (183/203) Updating files: 91% (185/203) Updating files: 92% (187/203) Updating files: 93% (189/203) Updating files: 94% (191/203) Updating files: 95% (193/203) Updating files: 96% (195/203) Updating files: 97% (197/203) Updating files: 98% (199/203) Updating files: 99% (201/203) Updating files: 100% (203/203) Updating files: 100% (203/203), done. 2024-03-09T09:13:08.494 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at c2d517ef966 v0.65 2024-03-09T09:13:08.499 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.65 2024-03-09T09:13:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:v0.66 2024-03-09T09:13:08.932 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:13:08.933 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 66 -eq 48 ']' 2024-03-09T09:13:08.933 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.66 2024-03-09T09:13:08.933 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.66 2024-03-09T09:13:08.933 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.66 2024-03-09T09:13:09.621 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:13:09.621 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:13:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.828+0000 7fc293e00640 1 Processor -- start 2024-03-09T09:13:09.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.828+0000 7fc293e00640 1 -- start start 2024-03-09T09:13:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.830+0000 7fc293e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc2940c3bb0 0x7fc2940c7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.830+0000 7fc293e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc29406e820 0x7fc294159230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.830+0000 7fc293e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc29406dee0 0x7fc294159770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.830+0000 7fc293e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2940c7700 con 0x7fc29406dee0 2024-03-09T09:13:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.830+0000 7fc293e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc294156be0 con 0x7fc2940c3bb0 2024-03-09T09:13:09.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.830+0000 7fc293e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc294159cb0 con 0x7fc29406e820 2024-03-09T09:13:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.830+0000 7fc293400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc29406dee0 0x7fc294159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.831+0000 7fc292a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc2940c3bb0 0x7fc2940c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.830+0000 7fc28a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc29406e820 0x7fc294159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.831+0000 7fc293400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc29406dee0 0x7fc294159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36000/0 (socket says 172.21.15.67:36000) 2024-03-09T09:13:09.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.831+0000 7fc292a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc2940c3bb0 0x7fc2940c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45504/0 (socket says 172.21.15.67:45504) 2024-03-09T09:13:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.831+0000 7fc293400640 1 -- 172.21.15.67:0/1232374915 learned_addr learned my addr 172.21.15.67:0/1232374915 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.831+0000 7fc28a000640 1 -- 172.21.15.67:0/1232374915 >> v2:172.21.15.134:3300/0 conn(0x7fc2940c3bb0 msgr2=0x7fc2940c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.831+0000 7fc28a000640 1 --2- 172.21.15.67:0/1232374915 >> v2:172.21.15.134:3300/0 conn(0x7fc2940c3bb0 0x7fc2940c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.831+0000 7fc28a000640 1 -- 172.21.15.67:0/1232374915 >> v2:172.21.15.67:3300/0 conn(0x7fc29406dee0 msgr2=0x7fc294159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.831+0000 7fc28a000640 1 --2- 172.21.15.67:0/1232374915 >> v2:172.21.15.67:3300/0 conn(0x7fc29406dee0 0x7fc294159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.831+0000 7fc28a000640 1 -- 172.21.15.67:0/1232374915 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2940c7420 con 0x7fc29406e820 2024-03-09T09:13:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.831+0000 7fc293400640 1 --2- 172.21.15.67:0/1232374915 >> v2:172.21.15.67:3300/0 conn(0x7fc29406dee0 0x7fc294159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:13:09.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.833+0000 7fc28a000640 1 --2- 172.21.15.67:0/1232374915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc29406e820 0x7fc294159230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc2940708b0 tx=0x7fc27c02b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e68c61867a93405 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.833+0000 7fc292000640 1 -- 172.21.15.67:0/1232374915 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c038070 con 0x7fc29406e820 2024-03-09T09:13:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.833+0000 7fc292000640 1 -- 172.21.15.67:0/1232374915 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27c031700 con 0x7fc29406e820 2024-03-09T09:13:09.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.833+0000 7fc292000640 1 -- 172.21.15.67:0/1232374915 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c031ad0 con 0x7fc29406e820 2024-03-09T09:13:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.834+0000 7fc293e00640 1 -- 172.21.15.67:0/1232374915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc29406e820 msgr2=0x7fc294159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.834+0000 7fc293e00640 1 --2- 172.21.15.67:0/1232374915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc29406e820 0x7fc294159230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc2940708b0 tx=0x7fc27c02b7a0 comp rx=0 tx=0).stop 2024-03-09T09:13:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.834+0000 7fc293e00640 1 -- 172.21.15.67:0/1232374915 shutdown_connections 2024-03-09T09:13:09.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.834+0000 7fc293e00640 1 --2- 172.21.15.67:0/1232374915 >> v2:172.21.15.67:3300/0 conn(0x7fc29406dee0 0x7fc294159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:09.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.834+0000 7fc293e00640 1 --2- 172.21.15.67:0/1232374915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc29406e820 0x7fc294159230 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:09.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.834+0000 7fc293e00640 1 --2- 172.21.15.67:0/1232374915 >> v2:172.21.15.134:3300/0 conn(0x7fc2940c3bb0 0x7fc2940c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:09.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.834+0000 7fc293e00640 1 -- 172.21.15.67:0/1232374915 >> 172.21.15.67:0/1232374915 conn(0x7fc29406b040 msgr2=0x7fc29406b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:09.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.834+0000 7fc293e00640 1 -- 172.21.15.67:0/1232374915 shutdown_connections 2024-03-09T09:13:09.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.834+0000 7fc293e00640 1 -- 172.21.15.67:0/1232374915 wait complete. 2024-03-09T09:13:09.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.837+0000 7fc293e00640 1 Processor -- start 2024-03-09T09:13:09.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.837+0000 7fc293e00640 1 -- start start 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.838+0000 7fc293e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc29406dee0 0x7fc2941e2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.838+0000 7fc293e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29406e820 0x7fc2941527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.838+0000 7fc293e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2940c3bb0 0x7fc294152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.838+0000 7fc293e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2941566c0 con 0x7fc29406dee0 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.838+0000 7fc293e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2940c5c30 con 0x7fc29406e820 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.838+0000 7fc293e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29406ec40 con 0x7fc2940c3bb0 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.838+0000 7fc292a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc29406dee0 0x7fc2941e2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.838+0000 7fc292a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc29406dee0 0x7fc2941e2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36014/0 (socket says 172.21.15.67:36014) 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.838+0000 7fc292a00640 1 -- 172.21.15.67:0/1912940859 learned_addr learned my addr 172.21.15.67:0/1912940859 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.839+0000 7fc293400640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2940c3bb0 0x7fc294152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.839+0000 7fc292a00640 1 -- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2940c3bb0 msgr2=0x7fc294152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:09.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.839+0000 7fc292a00640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2940c3bb0 0x7fc294152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.839+0000 7fc292a00640 1 -- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29406e820 msgr2=0x7fc2941527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.839+0000 7fc292a00640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29406e820 0x7fc2941527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.839+0000 7fc292a00640 1 -- 172.21.15.67:0/1912940859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2941ec330 con 0x7fc29406dee0 2024-03-09T09:13:09.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.839+0000 7fc292a00640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc29406dee0 0x7fc2941e2230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc284009d10 tx=0x7fc28400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:09.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.839+0000 7fc28b400640 1 -- 172.21.15.67:0/1912940859 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284018020 con 0x7fc29406dee0 2024-03-09T09:13:09.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.840+0000 7fc28b400640 1 -- 172.21.15.67:0/1912940859 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc284014480 con 0x7fc29406dee0 2024-03-09T09:13:09.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.840+0000 7fc293e00640 1 -- 172.21.15.67:0/1912940859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2941ec610 con 0x7fc29406dee0 2024-03-09T09:13:09.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.840+0000 7fc28b400640 1 -- 172.21.15.67:0/1912940859 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284014830 con 0x7fc29406dee0 2024-03-09T09:13:09.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.840+0000 7fc293e00640 1 -- 172.21.15.67:0/1912940859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2941df880 con 0x7fc29406dee0 2024-03-09T09:13:09.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.841+0000 7fc293e00640 1 -- 172.21.15.67:0/1912940859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29404f3c0 con 0x7fc29406dee0 2024-03-09T09:13:09.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.842+0000 7fc28b400640 1 -- 172.21.15.67:0/1912940859 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc284005d10 con 0x7fc29406dee0 2024-03-09T09:13:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.843+0000 7fc28b400640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc27006b500 0x7fc27006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:09.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.843+0000 7fc28b400640 1 -- 172.21.15.67:0/1912940859 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc28408b010 con 0x7fc29406dee0 2024-03-09T09:13:09.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.844+0000 7fc28a000640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc27006b500 0x7fc27006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:09.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.845+0000 7fc28a000640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc27006b500 0x7fc27006d9c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc27c035000 tx=0x7fc27c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:09.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:09.849+0000 7fc28b400640 1 -- 172.21.15.67:0/1912940859 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc284022070 con 0x7fc29406dee0 2024-03-09T09:13:09.952 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 32% (46/143) Updating files: 33% (48/143) Updating files: 34% (49/143) Updating files: 35% (51/143) Updating files: 36% (52/143) Updating files: 37% (53/143) Updating files: 38% (55/143) Updating files: 39% (56/143) Updating files: 40% (58/143) Updating files: 41% (59/143) Updating files: 42% (61/143) Updating files: 43% (62/143) Updating files: 44% (63/143) Updating files: 45% (65/143) Updating files: 46% (66/143) Updating files: 47% (68/143) Updating files: 48% (69/143) Updating files: 49% (71/143) Updating files: 50% (72/143) Updating files: 51% (73/143) Updating files: 52% (75/143) Updating files: 53% (76/143) Updating files: 54% (78/143) Updating files: 55% (79/143) Updating files: 55% (80/143) Updating files: 56% (81/143) Updating files: 57% (82/143) Updating files: 58% (83/143) Updating files: 59% (85/143) Updating files: 60% (86/143) Updating files: 61% (88/143) Updating files: 62% (89/143) Updating files: 63% (91/143) Updating files: 64% (92/143) Updating files: 65% (93/143) Updating files: 66% (95/143) Updating files: 67% (96/143) Updating files: 68% (98/143) Updating files: 69% (99/143) Updating files: 70% (101/143) Updating files: 71% (102/143) Updating files: 72% (103/143) Updating files: 73% (105/143) Updating files: 74% (106/143) Updating files: 75% (108/143) Updating files: 76% (109/143) Updating files: 77% (111/143) Updating files: 78% (112/143) Updating files: 79% (113/143) Updating files: 80% (115/143) Updating files: 81% (116/143) Updating files: 82% (118/143) Updating files: 83% (119/143) Updating files: 84% (121/143) Updating files: 85% (122/143) Updating files: 86% (123/143) Updating files: 87% (125/143) Updating files: 88% (126/143) Updating files: 89% (128/143) Updating files: 90% (129/143) Updating files: 91% (131/143) Updating files: 92% (132/143) Updating files: 93% (133/143) Updating files: 94% (135/143) Updating files: 95% (136/143) Updating files: 96% (138/143) Updating files: 97% (139/143) Updating files: 98% (141/143) Updating files: 99% (142/143) Updating files: 100% (143/143) Updating files: 100% (143/143), done. 2024-03-09T09:13:10.022 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 42e06c12db6 v0.64 2024-03-09T09:13:10.041 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.64 2024-03-09T09:13:10.190 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:13:10.190 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 65 -eq 48 ']' 2024-03-09T09:13:10.190 INFO:tasks.workunit.client.1.smithi134.stdout:v0.65 2024-03-09T09:13:10.191 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.65 2024-03-09T09:13:10.191 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.65 2024-03-09T09:13:10.191 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.65 2024-03-09T09:13:10.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.225+0000 7fc293e00640 1 -- 172.21.15.67:0/1912940859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc294067110 con 0x7fc29406dee0 2024-03-09T09:13:10.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.226+0000 7fc28b400640 1 -- 172.21.15.67:0/1912940859 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fc284010040 con 0x7fc29406dee0 2024-03-09T09:13:10.231 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:13:10.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.230+0000 7fc293e00640 1 -- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc27006b500 msgr2=0x7fc27006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.230+0000 7fc293e00640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc27006b500 0x7fc27006d9c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc27c035000 tx=0x7fc27c030000 comp rx=0 tx=0).stop 2024-03-09T09:13:10.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.230+0000 7fc293e00640 1 -- 172.21.15.67:0/1912940859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc29406dee0 msgr2=0x7fc2941e2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.230+0000 7fc293e00640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc29406dee0 0x7fc2941e2230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc284009d10 tx=0x7fc28400c490 comp rx=0 tx=0).stop 2024-03-09T09:13:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.230+0000 7fc293e00640 1 -- 172.21.15.67:0/1912940859 shutdown_connections 2024-03-09T09:13:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.230+0000 7fc293e00640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc27006b500 0x7fc27006d9c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.230+0000 7fc293e00640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2940c3bb0 0x7fc294152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.230+0000 7fc293e00640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc29406e820 0x7fc2941527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.230+0000 7fc293e00640 1 --2- 172.21.15.67:0/1912940859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc29406dee0 0x7fc2941e2230 secure :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc284009d10 tx=0x7fc28400c490 comp rx=0 tx=0).stop 2024-03-09T09:13:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.230+0000 7fc293e00640 1 -- 172.21.15.67:0/1912940859 >> 172.21.15.67:0/1912940859 conn(0x7fc29406b040 msgr2=0x7fc2941559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.230+0000 7fc293e00640 1 -- 172.21.15.67:0/1912940859 shutdown_connections 2024-03-09T09:13:10.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.231+0000 7fc293e00640 1 -- 172.21.15.67:0/1912940859 wait complete. 2024-03-09T09:13:10.259 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:13:10.259 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:13:10.259 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:13:10.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:13:10.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.465+0000 7f23c2a00640 1 Processor -- start 2024-03-09T09:13:10.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.466+0000 7f23c2a00640 1 -- start start 2024-03-09T09:13:10.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.467+0000 7f23c2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23bc0ac930 0x7f23bc0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.467+0000 7f23c2a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f23bc0c30a0 0x7f23bc150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.467+0000 7f23c2a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f23bc151190 0x7f23bc1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.467+0000 7f23c2a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23bc070670 con 0x7f23bc0c30a0 2024-03-09T09:13:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.467+0000 7f23c2a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23bc0c2780 con 0x7f23bc151190 2024-03-09T09:13:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.467+0000 7f23c2a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0708c0 con 0x7f23bc0ac930 2024-03-09T09:13:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.467+0000 7f23c1600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23bc0ac930 0x7f23bc0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.468+0000 7f23c1600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23bc0ac930 0x7f23bc0c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:33198/0 (socket says 172.21.15.67:33198) 2024-03-09T09:13:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.468+0000 7f23c1600640 1 -- 172.21.15.67:0/1634611635 learned_addr learned my addr 172.21.15.67:0/1634611635 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.468+0000 7f23c2000640 1 --2- 172.21.15.67:0/1634611635 >> v2:172.21.15.134:3300/0 conn(0x7f23bc151190 0x7f23bc1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.468+0000 7f23c0c00640 1 --2- 172.21.15.67:0/1634611635 >> v2:172.21.15.67:3300/0 conn(0x7f23bc0c30a0 0x7f23bc150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.468+0000 7f23c1600640 1 -- 172.21.15.67:0/1634611635 >> v2:172.21.15.134:3300/0 conn(0x7f23bc151190 msgr2=0x7f23bc1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:10.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.468+0000 7f23c1600640 1 --2- 172.21.15.67:0/1634611635 >> v2:172.21.15.134:3300/0 conn(0x7f23bc151190 0x7f23bc1539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.468+0000 7f23c1600640 1 -- 172.21.15.67:0/1634611635 >> v2:172.21.15.67:3300/0 conn(0x7f23bc0c30a0 msgr2=0x7f23bc150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:10.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.468+0000 7f23c1600640 1 --2- 172.21.15.67:0/1634611635 >> v2:172.21.15.67:3300/0 conn(0x7f23bc0c30a0 0x7f23bc150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.468+0000 7f23c1600640 1 -- 172.21.15.67:0/1634611635 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc070ba0 con 0x7f23bc0ac930 2024-03-09T09:13:10.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.469+0000 7f23c1600640 1 --2- 172.21.15.67:0/1634611635 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23bc0ac930 0x7f23bc0c3660 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0c1590 tx=0x7f23b002b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed774928aeefab9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:10.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.470+0000 7f23bbe00640 1 -- 172.21.15.67:0/1634611635 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0035020 con 0x7f23bc0ac930 2024-03-09T09:13:10.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.470+0000 7f23bbe00640 1 -- 172.21.15.67:0/1634611635 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b0031880 con 0x7f23bc0ac930 2024-03-09T09:13:10.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.470+0000 7f23bbe00640 1 -- 172.21.15.67:0/1634611635 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0031c50 con 0x7f23bc0ac930 2024-03-09T09:13:10.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.471+0000 7f23c2a00640 1 -- 172.21.15.67:0/1634611635 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23bc0ac930 msgr2=0x7f23bc0c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.471+0000 7f23c2a00640 1 --2- 172.21.15.67:0/1634611635 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23bc0ac930 0x7f23bc0c3660 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0c1590 tx=0x7f23b002b920 comp rx=0 tx=0).stop 2024-03-09T09:13:10.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.471+0000 7f23c2a00640 1 -- 172.21.15.67:0/1634611635 shutdown_connections 2024-03-09T09:13:10.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.471+0000 7f23c2a00640 1 --2- 172.21.15.67:0/1634611635 >> v2:172.21.15.134:3300/0 conn(0x7f23bc151190 0x7f23bc1539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.471+0000 7f23c2a00640 1 --2- 172.21.15.67:0/1634611635 >> v2:172.21.15.67:3300/0 conn(0x7f23bc0c30a0 0x7f23bc150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.471+0000 7f23c2a00640 1 --2- 172.21.15.67:0/1634611635 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23bc0ac930 0x7f23bc0c3660 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.471+0000 7f23c2a00640 1 -- 172.21.15.67:0/1634611635 >> 172.21.15.67:0/1634611635 conn(0x7f23bc0c7220 msgr2=0x7f23bc0c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:10.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.471+0000 7f23c2a00640 1 -- 172.21.15.67:0/1634611635 shutdown_connections 2024-03-09T09:13:10.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.471+0000 7f23c2a00640 1 -- 172.21.15.67:0/1634611635 wait complete. 2024-03-09T09:13:10.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.474+0000 7f23c2a00640 1 Processor -- start 2024-03-09T09:13:10.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.474+0000 7f23c2a00640 1 -- start start 2024-03-09T09:13:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.475+0000 7f23c2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc0ac930 0x7f23bc152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.475+0000 7f23c2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23bc0c30a0 0x7f23bc0ab670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.475+0000 7f23c2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23bc151190 0x7f23bc13f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.475+0000 7f23c2a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0acd10 con 0x7f23bc0ac930 2024-03-09T09:13:10.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.475+0000 7f23c2a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1515a0 con 0x7f23bc0c30a0 2024-03-09T09:13:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.475+0000 7f23c2a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0ace90 con 0x7f23bc151190 2024-03-09T09:13:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.475+0000 7f23c1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc0ac930 0x7f23bc152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.475+0000 7f23c0c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23bc0c30a0 0x7f23bc0ab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.475+0000 7f23c1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc0ac930 0x7f23bc152dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36038/0 (socket says 172.21.15.67:36038) 2024-03-09T09:13:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.475+0000 7f23c1600640 1 -- 172.21.15.67:0/3471867755 learned_addr learned my addr 172.21.15.67:0/3471867755 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:10.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.475+0000 7f23c2000640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23bc151190 0x7f23bc13f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.476+0000 7f23c0c00640 1 -- 172.21.15.67:0/3471867755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23bc151190 msgr2=0x7f23bc13f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.476+0000 7f23c0c00640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23bc151190 0x7f23bc13f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.476+0000 7f23c0c00640 1 -- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc0ac930 msgr2=0x7f23bc152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.476+0000 7f23c0c00640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc0ac930 0x7f23bc152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.476+0000 7f23c0c00640 1 -- 172.21.15.67:0/3471867755 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc153300 con 0x7f23bc0c30a0 2024-03-09T09:13:10.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.476+0000 7f23c0c00640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23bc0c30a0 0x7f23bc0ab670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23a800d5f0 tx=0x7f23a800dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:10.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.476+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8018020 con 0x7f23bc0c30a0 2024-03-09T09:13:10.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.477+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a8014620 con 0x7f23bc0c30a0 2024-03-09T09:13:10.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.477+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8014a10 con 0x7f23bc0c30a0 2024-03-09T09:13:10.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.477+0000 7f23c2a00640 1 -- 172.21.15.67:0/3471867755 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc153730 con 0x7f23bc0c30a0 2024-03-09T09:13:10.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.477+0000 7f23c2a00640 1 -- 172.21.15.67:0/3471867755 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc153cf0 con 0x7f23bc0c30a0 2024-03-09T09:13:10.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.478+0000 7f23c2a00640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc154090 0x7f23bc13feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.478+0000 7f23c2a00640 1 -- 172.21.15.67:0/3471867755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc04f3c0 con 0x7f23bc154090 2024-03-09T09:13:10.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.478+0000 7f23c1600640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc154090 0x7f23bc13feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.479+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f23a801b020 con 0x7f23bc0c30a0 2024-03-09T09:13:10.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.479+0000 7f23c1600640 1 -- 172.21.15.67:0/3471867755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23bc14f060 con 0x7f23bc154090 2024-03-09T09:13:10.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.480+0000 7f23c1600640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc154090 0x7f23bc13feb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f23b0000c00 tx=0x7f23b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:10.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.480+0000 7f23b9600640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f239c06b550 0x7f239c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.480+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f23a808a910 con 0x7f23bc0c30a0 2024-03-09T09:13:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.481+0000 7f23c2000640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f239c06b550 0x7f239c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.481+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0035040 con 0x7f23bc154090 2024-03-09T09:13:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.481+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f23b0049070 con 0x7f23bc154090 2024-03-09T09:13:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.481+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc154090 msgr2=0x7f23bc13feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.481+0000 7f23b9600640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc154090 0x7f23bc13feb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f23b0000c00 tx=0x7f23b0004730 comp rx=0 tx=0).stop 2024-03-09T09:13:10.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.481+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc154090 msgr2=0x7f23bc13feb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:13:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.481+0000 7f23b9600640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc154090 0x7f23bc13feb0 secure :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f23b0000c00 tx=0x7f23b0004730 comp rx=0 tx=0).stop 2024-03-09T09:13:10.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.482+0000 7f23c2000640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f239c06b550 0x7f239c06da10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f23b4011340 tx=0x7f23b400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.494+0000 7f23c2a00640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc1403f0 0x7f23bc1ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.494+0000 7f23c2a00640 1 -- 172.21.15.67:0/3471867755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0abbb0 con 0x7f23bc1403f0 2024-03-09T09:13:10.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.495+0000 7f23c1600640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc1403f0 0x7f23bc1ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.495+0000 7f23c1600640 1 -- 172.21.15.67:0/3471867755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f23bc067b30 con 0x7f23bc1403f0 2024-03-09T09:13:10.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.496+0000 7f23c1600640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc1403f0 0x7f23bc1ee8f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f23b0045000 tx=0x7f23b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.496+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0055030 con 0x7f23bc1403f0 2024-03-09T09:13:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.496+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7f23b0009490 con 0x7f23bc1403f0 2024-03-09T09:13:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.496+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc1403f0 msgr2=0x7f23bc1ee8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.496+0000 7f23b9600640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc1403f0 0x7f23bc1ee8f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f23b0045000 tx=0x7f23b00046c0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.496+0000 7f23b9600640 1 -- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc1403f0 msgr2=0x7f23bc1ee8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:13:10.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.496+0000 7f23b9600640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23bc1403f0 0x7f23bc1ee8f0 secure :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f23b0045000 tx=0x7f23b00046c0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.502 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:13:10.502 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:13:10.502 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:13:10.502 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:13:10.502 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 152, 2024-03-09T09:13:10.502 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:13:10.502 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:10.503 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:10.504 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:13:10.505 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:13:10.509 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:10.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.511 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:13:10.511 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:13:10.511 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:10.511 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:10.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.511 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:10.511 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:13:10.511 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.512 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:13:10.513 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:13:10.514 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:13:10.515 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.516 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.516 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:13:10.516 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:13:10.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.497+0000 7f23c2a00640 1 -- 172.21.15.67:0/3471867755 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f239c06b550 msgr2=0x7f239c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.497+0000 7f23c2a00640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f239c06b550 0x7f239c06da10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f23b4011340 tx=0x7f23b400b040 comp rx=0 tx=0).stop 2024-03-09T09:13:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.497+0000 7f23c2a00640 1 -- 172.21.15.67:0/3471867755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23bc0c30a0 msgr2=0x7f23bc0ab670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.497+0000 7f23c2a00640 1 --2- 172.21.15.67:0/3471867755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23bc0c30a0 0x7f23bc0ab670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23a800d5f0 tx=0x7f23a800dab0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.497+0000 7f23c2000640 1 -- 172.21.15.67:0/3471867755 reap_dead start 2024-03-09T09:13:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.497+0000 7f23c2000640 1 -- 172.21.15.67:0/3471867755 reap_dead start 2024-03-09T09:13:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.497+0000 7f23c2a00640 1 -- 172.21.15.67:0/3471867755 shutdown_connections 2024-03-09T09:13:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.497+0000 7f23c2a00640 1 -- 172.21.15.67:0/3471867755 >> 172.21.15.67:0/3471867755 conn(0x7f23bc0c7220 msgr2=0x7f23bc072940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.497+0000 7f23c2a00640 1 -- 172.21.15.67:0/3471867755 shutdown_connections 2024-03-09T09:13:10.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.497+0000 7f23c2a00640 1 -- 172.21.15.67:0/3471867755 wait complete. 2024-03-09T09:13:10.520 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:13:10.520 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:13:10.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:13:10.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.730+0000 7f62e4c00640 1 Processor -- start 2024-03-09T09:13:10.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.730+0000 7f62e4c00640 1 -- start start 2024-03-09T09:13:10.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.731+0000 7f62e4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f62e00c9380 0x7f62e00acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.732+0000 7f62e4c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e00c9890 con 0x7f62e014fd90 2024-03-09T09:13:10.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.732+0000 7f62e4c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e00c5f80 con 0x7f62e00ad440 2024-03-09T09:13:10.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.732+0000 7f62e4c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e006ba60 con 0x7f62e00c9380 2024-03-09T09:13:10.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.732+0000 7f62dea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f62e00c9380 0x7f62e00acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.732+0000 7f62dea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f62e00c9380 0x7f62e00acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36070/0 (socket says 172.21.15.67:36070) 2024-03-09T09:13:10.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.732+0000 7f62dea00640 1 -- 172.21.15.67:0/2749047473 learned_addr learned my addr 172.21.15.67:0/2749047473 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:10.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.732+0000 7f62dea00640 1 -- 172.21.15.67:0/2749047473 >> v1:172.21.15.134:6789/0 conn(0x7f62e00ad440 legacy=0x7f62e006c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.732+0000 7f62dea00640 1 -- 172.21.15.67:0/2749047473 >> v1:172.21.15.67:6789/0 conn(0x7f62e014fd90 legacy=0x7f62e0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.732+0000 7f62dea00640 1 -- 172.21.15.67:0/2749047473 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e00c6240 con 0x7f62e00c9380 2024-03-09T09:13:10.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.732+0000 7f62de000640 1 -- 172.21.15.67:0/2749047473 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048236280 0 0) 0x7f62e00c9890 con 0x7f62e014fd90 2024-03-09T09:13:10.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.734+0000 7f62dea00640 1 --2- 172.21.15.67:0/2749047473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62e00c9380 0x7f62e00acf00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f62e006bbe0 tx=0x7f62cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a64c180305b6ae server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:10.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.734+0000 7f62de000640 1 -- 172.21.15.67:0/2749047473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc035020 con 0x7f62e00c9380 2024-03-09T09:13:10.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.734+0000 7f62de000640 1 -- 172.21.15.67:0/2749047473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc031640 con 0x7f62e00c9380 2024-03-09T09:13:10.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.734+0000 7f62de000640 1 -- 172.21.15.67:0/2749047473 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc02d040 con 0x7f62e00c9380 2024-03-09T09:13:10.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.734+0000 7f62e4c00640 1 -- 172.21.15.67:0/2749047473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62e00c9380 msgr2=0x7f62e00acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.734+0000 7f62e4c00640 1 --2- 172.21.15.67:0/2749047473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62e00c9380 0x7f62e00acf00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f62e006bbe0 tx=0x7f62cc02b8f0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.735+0000 7f62e4c00640 1 -- 172.21.15.67:0/2749047473 shutdown_connections 2024-03-09T09:13:10.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.735+0000 7f62e4c00640 1 --2- 172.21.15.67:0/2749047473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62e00c9380 0x7f62e00acf00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.735+0000 7f62e4c00640 1 -- 172.21.15.67:0/2749047473 >> 172.21.15.67:0/2749047473 conn(0x7f62e00cb480 msgr2=0x7f62e00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:10.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.735+0000 7f62e4c00640 1 -- 172.21.15.67:0/2749047473 shutdown_connections 2024-03-09T09:13:10.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.735+0000 7f62e4c00640 1 -- 172.21.15.67:0/2749047473 wait complete. 2024-03-09T09:13:10.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.736+0000 7f62e4c00640 1 Processor -- start 2024-03-09T09:13:10.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.736+0000 7f62e4c00640 1 -- start start 2024-03-09T09:13:10.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.737+0000 7f62e4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62e00ad440 0x7f62e00af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.737+0000 7f62e4c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00c9380 0x7f62e014d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.737+0000 7f62e4c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62e014fd90 0x7f62e01ef620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.737+0000 7f62e4c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e006c8e0 con 0x7f62e00ad440 2024-03-09T09:13:10.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.737+0000 7f62e4c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e006bcb0 con 0x7f62e00c9380 2024-03-09T09:13:10.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.737+0000 7f62e4c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e006ca60 con 0x7f62e014fd90 2024-03-09T09:13:10.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62df400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62e00ad440 0x7f62e00af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62df400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62e00ad440 0x7f62e00af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36078/0 (socket says 172.21.15.67:36078) 2024-03-09T09:13:10.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62df400640 1 -- 172.21.15.67:0/593499317 learned_addr learned my addr 172.21.15.67:0/593499317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:10.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62dea00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00c9380 0x7f62e014d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62dfe00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62e014fd90 0x7f62e01ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62df400640 1 -- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62e014fd90 msgr2=0x7f62e01ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62df400640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62e014fd90 0x7f62e01ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62df400640 1 -- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00c9380 msgr2=0x7f62e014d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62df400640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00c9380 0x7f62e014d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62df400640 1 -- 172.21.15.67:0/593499317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e00c2f50 con 0x7f62e00ad440 2024-03-09T09:13:10.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62dfe00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62e014fd90 0x7f62e01ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:13:10.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.738+0000 7f62df400640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62e00ad440 0x7f62e00af160 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c3c0 tx=0x7f62d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:10.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.739+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0017070 con 0x7f62e00ad440 2024-03-09T09:13:10.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.739+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d00035f0 con 0x7f62e00ad440 2024-03-09T09:13:10.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.739+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00039e0 con 0x7f62e00ad440 2024-03-09T09:13:10.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.739+0000 7f62e4c00640 1 -- 172.21.15.67:0/593499317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e00c3230 con 0x7f62e00ad440 2024-03-09T09:13:10.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.739+0000 7f62e4c00640 1 -- 172.21.15.67:0/593499317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e014dd60 con 0x7f62e00ad440 2024-03-09T09:13:10.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.740+0000 7f62e4c00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00c5370 0x7f62e014bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.740+0000 7f62e4c00640 1 -- 172.21.15.67:0/593499317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e014e2f0 con 0x7f62e00c5370 2024-03-09T09:13:10.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.740+0000 7f62dea00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00c5370 0x7f62e014bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.741+0000 7f62dea00640 1 -- 172.21.15.67:0/593499317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62e004f3c0 con 0x7f62e00c5370 2024-03-09T09:13:10.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.741+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f62d001a070 con 0x7f62e00ad440 2024-03-09T09:13:10.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.741+0000 7f62dea00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00c5370 0x7f62e014bae0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f62e00ada40 tx=0x7f62cc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:10.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.742+0000 7f62d7400640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f62c006b4d0 0x7f62c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.742+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f62d0089850 con 0x7f62e00ad440 2024-03-09T09:13:10.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.742+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc035040 con 0x7f62e00c5370 2024-03-09T09:13:10.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.743+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f62cc044400 con 0x7f62e00c5370 2024-03-09T09:13:10.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.743+0000 7f62dfe00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f62c006b4d0 0x7f62c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.743+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00c5370 msgr2=0x7f62e014bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.743+0000 7f62d7400640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00c5370 0x7f62e014bae0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f62e00ada40 tx=0x7f62cc004730 comp rx=0 tx=0).stop 2024-03-09T09:13:10.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.743+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00c5370 msgr2=0x7f62e014bae0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:13:10.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.743+0000 7f62d7400640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e00c5370 0x7f62e014bae0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f62e00ada40 tx=0x7f62cc004730 comp rx=0 tx=0).stop 2024-03-09T09:13:10.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.744+0000 7f62dfe00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f62c006b4d0 0x7f62c006d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f62e014e6c0 tx=0x7f62d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:10.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.750+0000 7f62e4c00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e006d030 0x7f62e014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:10.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.750+0000 7f62e4c00640 1 -- 172.21.15.67:0/593499317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00c5820 con 0x7f62e006d030 2024-03-09T09:13:10.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.751+0000 7f62dea00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e006d030 0x7f62e014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:10.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.751+0000 7f62dea00640 1 -- 172.21.15.67:0/593499317 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f62e014c560 con 0x7f62e006d030 2024-03-09T09:13:10.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.752+0000 7f62dea00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e006d030 0x7f62e014c020 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f62cc000c00 tx=0x7f62cc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:10.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.752+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc051070 con 0x7f62e006d030 2024-03-09T09:13:10.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f62cc04c920 con 0x7f62e006d030 2024-03-09T09:13:10.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e006d030 msgr2=0x7f62e014c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62d7400640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e006d030 0x7f62e014c020 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f62cc000c00 tx=0x7f62cc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62d7400640 1 -- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e006d030 msgr2=0x7f62e014c020 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:13:10.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62d7400640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e006d030 0x7f62e014c020 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f62cc000c00 tx=0x7f62cc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:13:10.758 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:13:10.758 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:13:10.758 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 152, 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:10.759 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:10.760 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:13:10.761 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:13:10.762 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.763 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.764 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.765 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:10.766 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.767 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:13:10.768 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:10.769 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:13:10.770 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:13:10.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62e4c00640 1 -- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f62c006b4d0 msgr2=0x7f62c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62e4c00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f62c006b4d0 0x7f62c006d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f62e014e6c0 tx=0x7f62d8009040 comp rx=0 tx=0).stop 2024-03-09T09:13:10.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62e4c00640 1 -- 172.21.15.67:0/593499317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62e00ad440 msgr2=0x7f62e00af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:10.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62e4c00640 1 --2- 172.21.15.67:0/593499317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f62e00ad440 0x7f62e00af160 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c3c0 tx=0x7f62d000c880 comp rx=0 tx=0).stop 2024-03-09T09:13:10.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62dfe00640 1 -- 172.21.15.67:0/593499317 reap_dead start 2024-03-09T09:13:10.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62dfe00640 1 -- 172.21.15.67:0/593499317 reap_dead start 2024-03-09T09:13:10.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62e4c00640 1 -- 172.21.15.67:0/593499317 shutdown_connections 2024-03-09T09:13:10.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62e4c00640 1 -- 172.21.15.67:0/593499317 >> 172.21.15.67:0/593499317 conn(0x7f62e00cb480 msgr2=0x7f62e0153a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:10.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.753+0000 7f62e4c00640 1 -- 172.21.15.67:0/593499317 shutdown_connections 2024-03-09T09:13:10.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:10.754+0000 7f62e4c00640 1 -- 172.21.15.67:0/593499317 wait complete. 2024-03-09T09:13:10.782 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:13:10.782 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:13:10.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:13:11.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.010+0000 7f6b4d000640 1 Processor -- start 2024-03-09T09:13:11.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.010+0000 7f6b4d000640 1 -- start start 2024-03-09T09:13:11.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.011+0000 7f6b4d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b480c9380 0x7f6b480acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.012+0000 7f6b4d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6b480ad440 0x7f6b4806c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.012+0000 7f6b4d000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b480c9890 con 0x7f6b4814fd90 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.012+0000 7f6b4d000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4806ba60 con 0x7f6b480ad440 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.012+0000 7f6b4d000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b480c5f80 con 0x7f6b480c9380 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.012+0000 7f6b47400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6b4814fd90 0x7f6b48156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59712/0 (socket says 172.21.15.67:59712) 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.012+0000 7f6b47400640 1 -- 172.21.15.67:0/239465770 learned_addr learned my addr 172.21.15.67:0/239465770 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.012+0000 7f6b47e00640 1 --2- 172.21.15.67:0/239465770 >> v2:172.21.15.67:3300/0 conn(0x7f6b480ad440 0x7f6b4806c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.012+0000 7f6b46a00640 1 --2- 172.21.15.67:0/239465770 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b480c9380 0x7f6b480acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.012+0000 7f6b46000640 1 -- 172.21.15.67:0/239465770 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435930087 0 0) 0x7f6b480c9890 con 0x7f6b4814fd90 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.012+0000 7f6b46000640 1 -- 172.21.15.67:0/239465770 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b2c003520 con 0x7f6b4814fd90 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.013+0000 7f6b47e00640 1 -- 172.21.15.67:0/239465770 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b480c9380 msgr2=0x7f6b480acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.013+0000 7f6b47e00640 1 --2- 172.21.15.67:0/239465770 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b480c9380 0x7f6b480acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.013+0000 7f6b47e00640 1 -- 172.21.15.67:0/239465770 >> v1:172.21.15.67:6789/0 conn(0x7f6b4814fd90 legacy=0x7f6b48156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.013+0000 7f6b47e00640 1 -- 172.21.15.67:0/239465770 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b480c6240 con 0x7f6b480ad440 2024-03-09T09:13:11.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.014+0000 7f6b47e00640 1 --2- 172.21.15.67:0/239465770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b480ad440 0x7f6b4806c1d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6b4806bbe0 tx=0x7f6b4002bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e522ac45fa315b68 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:11.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.014+0000 7f6b46000640 1 -- 172.21.15.67:0/239465770 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40039070 con 0x7f6b480ad440 2024-03-09T09:13:11.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.014+0000 7f6b46000640 1 -- 172.21.15.67:0/239465770 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4002cb20 con 0x7f6b480ad440 2024-03-09T09:13:11.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.014+0000 7f6b46000640 1 -- 172.21.15.67:0/239465770 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40034440 con 0x7f6b480ad440 2024-03-09T09:13:11.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.015+0000 7f6b4d000640 1 -- 172.21.15.67:0/239465770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b480ad440 msgr2=0x7f6b4806c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.015+0000 7f6b4d000640 1 --2- 172.21.15.67:0/239465770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b480ad440 0x7f6b4806c1d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6b4806bbe0 tx=0x7f6b4002bd70 comp rx=0 tx=0).stop 2024-03-09T09:13:11.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.015+0000 7f6b4d000640 1 -- 172.21.15.67:0/239465770 shutdown_connections 2024-03-09T09:13:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.015+0000 7f6b4d000640 1 --2- 172.21.15.67:0/239465770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b480ad440 0x7f6b4806c1d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.015+0000 7f6b4d000640 1 --2- 172.21.15.67:0/239465770 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b480c9380 0x7f6b480acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.015+0000 7f6b4d000640 1 -- 172.21.15.67:0/239465770 >> 172.21.15.67:0/239465770 conn(0x7f6b480cb480 msgr2=0x7f6b480cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.015+0000 7f6b4d000640 1 -- 172.21.15.67:0/239465770 shutdown_connections 2024-03-09T09:13:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.015+0000 7f6b4d000640 1 -- 172.21.15.67:0/239465770 wait complete. 2024-03-09T09:13:11.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.017+0000 7f6b4d000640 1 Processor -- start 2024-03-09T09:13:11.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.018+0000 7f6b4d000640 1 -- start start 2024-03-09T09:13:11.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.019+0000 7f6b4d000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b480ad440 0x7f6b48157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.019+0000 7f6b4d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b480c9380 0x7f6b481eea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.019+0000 7f6b4d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4814fd90 0x7f6b480af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.019+0000 7f6b4d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4806c710 con 0x7f6b4814fd90 2024-03-09T09:13:11.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.019+0000 7f6b4d000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4806bcb0 con 0x7f6b480ad440 2024-03-09T09:13:11.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.019+0000 7f6b4d000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4806c890 con 0x7f6b480c9380 2024-03-09T09:13:11.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.020+0000 7f6b47400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b480ad440 0x7f6b48157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.020+0000 7f6b47e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4814fd90 0x7f6b480af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.020+0000 7f6b47400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b480ad440 0x7f6b48157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45574/0 (socket says 172.21.15.67:45574) 2024-03-09T09:13:11.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.020+0000 7f6b47400640 1 -- 172.21.15.67:0/1684316698 learned_addr learned my addr 172.21.15.67:0/1684316698 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:11.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.020+0000 7f6b46a00640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b480c9380 0x7f6b481eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.020+0000 7f6b47400640 1 -- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b480c9380 msgr2=0x7f6b481eea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.020+0000 7f6b47400640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b480c9380 0x7f6b481eea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.020+0000 7f6b47400640 1 -- 172.21.15.67:0/1684316698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4814fd90 msgr2=0x7f6b480af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.020+0000 7f6b47400640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b4814fd90 0x7f6b480af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.020+0000 7f6b47400640 1 -- 172.21.15.67:0/1684316698 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b480b0f40 con 0x7f6b480ad440 2024-03-09T09:13:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.021+0000 7f6b47400640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b480ad440 0x7f6b48157c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800c390 tx=0x7f6b3800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.021+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b38017070 con 0x7f6b480ad440 2024-03-09T09:13:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.021+0000 7f6b4d000640 1 -- 172.21.15.67:0/1684316698 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b481ef2c0 con 0x7f6b480ad440 2024-03-09T09:13:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.021+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b38005ce0 con 0x7f6b480ad440 2024-03-09T09:13:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.021+0000 7f6b4d000640 1 -- 172.21.15.67:0/1684316698 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b480aeca0 con 0x7f6b480ad440 2024-03-09T09:13:11.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.021+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b380072d0 con 0x7f6b480ad440 2024-03-09T09:13:11.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.023+0000 7f6b4d000640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20002a70 0x7f6b20004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.023+0000 7f6b4d000640 1 -- 172.21.15.67:0/1684316698 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b20007ec0 con 0x7f6b20002a70 2024-03-09T09:13:11.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.024+0000 7f6b46a00640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20002a70 0x7f6b20004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.024+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6b3801a070 con 0x7f6b480ad440 2024-03-09T09:13:11.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.025+0000 7f6b46a00640 1 -- 172.21.15.67:0/1684316698 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b200080c0 con 0x7f6b20002a70 2024-03-09T09:13:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.025+0000 7f6b46a00640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20002a70 0x7f6b20004f30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6b4814f1d0 tx=0x7f6b34011d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.026+0000 7f6b3f400640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b2806b4d0 0x7f6b2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.026+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6b380898a0 con 0x7f6b480ad440 2024-03-09T09:13:11.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.026+0000 7f6b47e00640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b2806b4d0 0x7f6b2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.026+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34019070 con 0x7f6b20002a70 2024-03-09T09:13:11.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.026+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f6b34004300 con 0x7f6b20002a70 2024-03-09T09:13:11.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.026+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20002a70 msgr2=0x7f6b20004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.026+0000 7f6b3f400640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20002a70 0x7f6b20004f30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6b4814f1d0 tx=0x7f6b34011d40 comp rx=0 tx=0).stop 2024-03-09T09:13:11.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.026+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20002a70 msgr2=0x7f6b20004f30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:13:11.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.026+0000 7f6b3f400640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20002a70 0x7f6b20004f30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.027+0000 7f6b47e00640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b2806b4d0 0x7f6b2806d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6b40036000 tx=0x7f6b40030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:11.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.042+0000 7f6b4d000640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20006f30 0x7f6b200073c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.042+0000 7f6b4d000640 1 -- 172.21.15.67:0/1684316698 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2000a4a0 con 0x7f6b20006f30 2024-03-09T09:13:11.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.042+0000 7f6b46a00640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20006f30 0x7f6b200073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.043+0000 7f6b46a00640 1 -- 172.21.15.67:0/1684316698 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6b20007900 con 0x7f6b20006f30 2024-03-09T09:13:11.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.044+0000 7f6b46a00640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20006f30 0x7f6b200073c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6b3401b000 tx=0x7f6b340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:11.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.044+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34019040 con 0x7f6b20006f30 2024-03-09T09:13:11.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.044+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f6b3401e450 con 0x7f6b20006f30 2024-03-09T09:13:11.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.045+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20006f30 msgr2=0x7f6b200073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.045+0000 7f6b3f400640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20006f30 0x7f6b200073c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6b3401b000 tx=0x7f6b340046c0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.045+0000 7f6b3f400640 1 -- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20006f30 msgr2=0x7f6b200073c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:13:11.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.045+0000 7f6b3f400640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b20006f30 0x7f6b200073c0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6b3401b000 tx=0x7f6b340046c0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.050 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:13:11.050 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:13:11.050 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:13:11.050 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:13:11.050 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 152, 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 17, 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:11.051 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:11.052 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:13:11.053 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:13:11.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:13:11.056 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:13:11.057 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.058 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:11.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:11.060 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:13:11.061 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:11.062 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:11.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:11.062 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:13:11.062 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:13:11.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.045+0000 7f6b4d000640 1 -- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b2806b4d0 msgr2=0x7f6b2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.045+0000 7f6b4d000640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6b2806b4d0 0x7f6b2806d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6b40036000 tx=0x7f6b40030000 comp rx=0 tx=0).stop 2024-03-09T09:13:11.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.045+0000 7f6b4d000640 1 -- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b480ad440 msgr2=0x7f6b48157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.045+0000 7f6b4d000640 1 --2- 172.21.15.67:0/1684316698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b480ad440 0x7f6b48157c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800c390 tx=0x7f6b3800c850 comp rx=0 tx=0).stop 2024-03-09T09:13:11.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.045+0000 7f6b47e00640 1 -- 172.21.15.67:0/1684316698 reap_dead start 2024-03-09T09:13:11.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.045+0000 7f6b47e00640 1 -- 172.21.15.67:0/1684316698 reap_dead start 2024-03-09T09:13:11.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.046+0000 7f6b4d000640 1 -- 172.21.15.67:0/1684316698 shutdown_connections 2024-03-09T09:13:11.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.046+0000 7f6b4d000640 1 -- 172.21.15.67:0/1684316698 >> 172.21.15.67:0/1684316698 conn(0x7f6b480cb480 msgr2=0x7f6b480cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:11.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.046+0000 7f6b4d000640 1 -- 172.21.15.67:0/1684316698 shutdown_connections 2024-03-09T09:13:11.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.046+0000 7f6b4d000640 1 -- 172.21.15.67:0/1684316698 wait complete. 2024-03-09T09:13:11.074 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:13:11.075 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T09:13:11.075 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:13:11.075 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T09:13:11.075 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:13:11.075 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:13:11.075 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:13:11.098 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:13:11.098 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:13:11.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:13:11.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.290+0000 7f78af000640 1 Processor -- start 2024-03-09T09:13:11.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.291+0000 7f78af000640 1 -- start start 2024-03-09T09:13:11.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.292+0000 7f78af000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f78a80710a0 0x7f78a8153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.292+0000 7f78af000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f78a8151360 0x7f78a8153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.292+0000 7f78af000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a80c81e0 con 0x7f78a80c1f70 2024-03-09T09:13:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.292+0000 7f78af000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80c32b0 con 0x7f78a80710a0 2024-03-09T09:13:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.292+0000 7f78af000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80c2fd0 con 0x7f78a8151360 2024-03-09T09:13:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.292+0000 7f78ad200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f78a80710a0 0x7f78a8153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.292+0000 7f78ad200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f78a80710a0 0x7f78a8153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36112/0 (socket says 172.21.15.67:36112) 2024-03-09T09:13:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.293+0000 7f78ad200640 1 -- 172.21.15.67:0/891615859 learned_addr learned my addr 172.21.15.67:0/891615859 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.293+0000 7f78ae600640 1 --2- 172.21.15.67:0/891615859 >> v2:172.21.15.134:3300/0 conn(0x7f78a8151360 0x7f78a8153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.293+0000 7f78ad200640 1 -- 172.21.15.67:0/891615859 >> v2:172.21.15.134:3300/0 conn(0x7f78a8151360 msgr2=0x7f78a8153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.293+0000 7f78ad200640 1 --2- 172.21.15.67:0/891615859 >> v2:172.21.15.134:3300/0 conn(0x7f78a8151360 0x7f78a8153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.293+0000 7f78ad200640 1 -- 172.21.15.67:0/891615859 >> v1:172.21.15.134:6789/0 conn(0x7f78a80c1f70 legacy=0x7f78a8150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.293+0000 7f78ad200640 1 -- 172.21.15.67:0/891615859 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a80c3700 con 0x7f78a80710a0 2024-03-09T09:13:11.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.293+0000 7f78ae600640 1 --2- 172.21.15.67:0/891615859 >> v2:172.21.15.134:3300/0 conn(0x7f78a8151360 0x7f78a8153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:13:11.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.294+0000 7f78ad200640 1 --2- 172.21.15.67:0/891615859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a80710a0 0x7f78a8153970 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f78a8073ef0 tx=0x7f789402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cea159d24e2bbd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:11.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.294+0000 7f78a7e00640 1 -- 172.21.15.67:0/891615859 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894035020 con 0x7f78a80710a0 2024-03-09T09:13:11.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.295+0000 7f78a7e00640 1 -- 172.21.15.67:0/891615859 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7894031800 con 0x7f78a80710a0 2024-03-09T09:13:11.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.295+0000 7f78a7e00640 1 -- 172.21.15.67:0/891615859 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894031bd0 con 0x7f78a80710a0 2024-03-09T09:13:11.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.295+0000 7f78af000640 1 -- 172.21.15.67:0/891615859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a80710a0 msgr2=0x7f78a8153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.295+0000 7f78af000640 1 --2- 172.21.15.67:0/891615859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a80710a0 0x7f78a8153970 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f78a8073ef0 tx=0x7f789402b8a0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.295+0000 7f78af000640 1 -- 172.21.15.67:0/891615859 shutdown_connections 2024-03-09T09:13:11.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.295+0000 7f78af000640 1 --2- 172.21.15.67:0/891615859 >> v2:172.21.15.134:3300/0 conn(0x7f78a8151360 0x7f78a8153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.295+0000 7f78af000640 1 --2- 172.21.15.67:0/891615859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a80710a0 0x7f78a8153970 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.295+0000 7f78af000640 1 -- 172.21.15.67:0/891615859 >> 172.21.15.67:0/891615859 conn(0x7f78a80c7300 msgr2=0x7f78a80c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:11.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.296+0000 7f78af000640 1 -- 172.21.15.67:0/891615859 shutdown_connections 2024-03-09T09:13:11.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.296+0000 7f78af000640 1 -- 172.21.15.67:0/891615859 wait complete. 2024-03-09T09:13:11.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.297+0000 7f78af000640 1 Processor -- start 2024-03-09T09:13:11.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.297+0000 7f78af000640 1 -- start start 2024-03-09T09:13:11.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.298+0000 7f78af000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a80710a0 0x7f78a8147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.298+0000 7f78af000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a80c1f70 0x7f78a81eb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.298+0000 7f78af000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a8151360 0x7f78a81eb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.298+0000 7f78af000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a8070d50 con 0x7f78a8151360 2024-03-09T09:13:11.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.298+0000 7f78af000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a8070830 con 0x7f78a80710a0 2024-03-09T09:13:11.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.298+0000 7f78af000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a8070ed0 con 0x7f78a80c1f70 2024-03-09T09:13:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.298+0000 7f78ad200640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a80c1f70 msgr2=0x7f78a81eb300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.298+0000 7f78ae600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a8151360 0x7f78a81eb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.298+0000 7f78ad200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a80c1f70 0x7f78a81eb300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.299+0000 7f78ae600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a8151360 0x7f78a81eb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36124/0 (socket says 172.21.15.67:36124) 2024-03-09T09:13:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.299+0000 7f78ae600640 1 -- 172.21.15.67:0/1413669151 learned_addr learned my addr 172.21.15.67:0/1413669151 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.299+0000 7f78adc00640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a80710a0 0x7f78a8147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.299+0000 7f78ae600640 1 -- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a80c1f70 msgr2=0x7f78a81eb300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:13:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.299+0000 7f78ae600640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a80c1f70 0x7f78a81eb300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.299+0000 7f78ae600640 1 -- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a80710a0 msgr2=0x7f78a8147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.299+0000 7f78ae600640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a80710a0 0x7f78a8147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.299+0000 7f78ae600640 1 -- 172.21.15.67:0/1413669151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8154810 con 0x7f78a8151360 2024-03-09T09:13:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.299+0000 7f78ae600640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a8151360 0x7f78a81eb840 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f78a000ced0 tx=0x7f78a000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.299+0000 7f78a5600640 1 -- 172.21.15.67:0/1413669151 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0003b90 con 0x7f78a8151360 2024-03-09T09:13:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.300+0000 7f78a5600640 1 -- 172.21.15.67:0/1413669151 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a0003d50 con 0x7f78a8151360 2024-03-09T09:13:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.300+0000 7f78a5600640 1 -- 172.21.15.67:0/1413669151 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0004260 con 0x7f78a8151360 2024-03-09T09:13:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.300+0000 7f78af000640 1 -- 172.21.15.67:0/1413669151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a80c5b10 con 0x7f78a8151360 2024-03-09T09:13:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.300+0000 7f78af000640 1 -- 172.21.15.67:0/1413669151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a81eab00 con 0x7f78a8151360 2024-03-09T09:13:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.301+0000 7f78af000640 1 -- 172.21.15.67:0/1413669151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a804f3c0 con 0x7f78a8151360 2024-03-09T09:13:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.302+0000 7f78a5600640 1 -- 172.21.15.67:0/1413669151 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f78a0004400 con 0x7f78a8151360 2024-03-09T09:13:11.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.303+0000 7f78a5600640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788806b480 0x7f788806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:11.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.304+0000 7f78a5600640 1 -- 172.21.15.67:0/1413669151 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f78a008afa0 con 0x7f78a8151360 2024-03-09T09:13:11.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.304+0000 7f78adc00640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788806b480 0x7f788806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.305+0000 7f78adc00640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788806b480 0x7f788806d940 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f789c004ea0 tx=0x7f789c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:11.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.308+0000 7f78a5600640 1 -- 172.21.15.67:0/1413669151 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f78a0059af0 con 0x7f78a8151360 2024-03-09T09:13:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.580+0000 7f78af000640 1 -- 172.21.15.67:0/1413669151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f78a80728f0 con 0x7f78a8151360 2024-03-09T09:13:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.580+0000 7f78a5600640 1 -- 172.21.15.67:0/1413669151 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f78a00593f0 con 0x7f78a8151360 2024-03-09T09:13:11.585 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:13:11.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 -- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788806b480 msgr2=0x7f788806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788806b480 0x7f788806d940 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f789c004ea0 tx=0x7f789c009040 comp rx=0 tx=0).stop 2024-03-09T09:13:11.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 -- 172.21.15.67:0/1413669151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a8151360 msgr2=0x7f78a81eb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:11.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a8151360 0x7f78a81eb840 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f78a000ced0 tx=0x7f78a000a710 comp rx=0 tx=0).stop 2024-03-09T09:13:11.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 -- 172.21.15.67:0/1413669151 shutdown_connections 2024-03-09T09:13:11.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788806b480 0x7f788806d940 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a8151360 0x7f78a81eb840 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a80c1f70 0x7f78a81eb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 --2- 172.21.15.67:0/1413669151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a80710a0 0x7f78a8147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:11.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 -- 172.21.15.67:0/1413669151 >> 172.21.15.67:0/1413669151 conn(0x7f78a80c7300 msgr2=0x7f78a8150c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:11.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 -- 172.21.15.67:0/1413669151 shutdown_connections 2024-03-09T09:13:11.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:11.584+0000 7f78af000640 1 -- 172.21.15.67:0/1413669151 wait complete. 2024-03-09T09:13:14.152 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 31% (63/203) Updating files: 32% (65/203) Updating files: 33% (67/203) Updating files: 34% (70/203) Updating files: 35% (72/203) Updating files: 36% (74/203) Updating files: 37% (76/203) Updating files: 38% (78/203) Updating files: 39% (80/203) Updating files: 40% (82/203) Updating files: 41% (84/203) Updating files: 42% (86/203) Updating files: 43% (88/203) Updating files: 44% (90/203) Updating files: 45% (92/203) Updating files: 46% (94/203) Updating files: 47% (96/203) Updating files: 48% (98/203) Updating files: 49% (100/203) Updating files: 50% (102/203) Updating files: 51% (104/203) Updating files: 52% (106/203) Updating files: 53% (108/203) Updating files: 54% (110/203) Updating files: 55% (112/203) Updating files: 56% (114/203) Updating files: 57% (116/203) Updating files: 58% (118/203) Updating files: 59% (120/203) Updating files: 60% (122/203) Updating files: 61% (124/203) Updating files: 62% (126/203) Updating files: 63% (128/203) Updating files: 64% (130/203) Updating files: 65% (132/203) Updating files: 66% (134/203) Updating files: 67% (137/203) Updating files: 68% (139/203) Updating files: 69% (141/203) Updating files: 69% (142/203) Updating files: 70% (143/203) Updating files: 71% (145/203) Updating files: 72% (147/203) Updating files: 73% (149/203) Updating files: 74% (151/203) Updating files: 75% (153/203) Updating files: 76% (155/203) Updating files: 77% (157/203) Updating files: 78% (159/203) Updating files: 79% (161/203) Updating files: 80% (163/203) Updating files: 81% (165/203) Updating files: 82% (167/203) Updating files: 83% (169/203) Updating files: 84% (171/203) Updating files: 85% (173/203) Updating files: 86% (175/203) Updating files: 87% (177/203) Updating files: 88% (179/203) Updating files: 89% (181/203) Updating files: 90% (183/203) Updating files: 91% (185/203) Updating files: 92% (187/203) Updating files: 93% (189/203) Updating files: 94% (191/203) Updating files: 95% (193/203) Updating files: 96% (195/203) Updating files: 97% (197/203) Updating files: 98% (199/203) Updating files: 99% (201/203) Updating files: 100% (203/203) Updating files: 100% (203/203), done. 2024-03-09T09:13:14.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:13:14.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.817+0000 7fa7cc200640 1 Processor -- start 2024-03-09T09:13:14.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.817+0000 7fa7cc200640 1 -- start start 2024-03-09T09:13:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.819+0000 7fa7cc200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7c40c5d10 0x7fa7c40c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.819+0000 7fa7cc200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c7550 0x7fa7c40c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.819+0000 7fa7cc200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa7c40c6630 0x7fa7c4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.819+0000 7fa7cc200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c40a6d10 con 0x7fa7c40c5d10 2024-03-09T09:13:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.819+0000 7fa7cc200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c40c2920 con 0x7fa7c40c6630 2024-03-09T09:13:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.819+0000 7fa7cc200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40c2640 con 0x7fa7c40c7550 2024-03-09T09:13:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.820+0000 7fa7c8e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c7550 msgr2=0x7fa7c40c60f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.820+0000 7fa7c8e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c7550 0x7fa7c40c60f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.820+0000 7fa7c9800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7c40c5d10 0x7fa7c40c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.820+0000 7fa7c9800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa7c40c5d10 0x7fa7c40c3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36128/0 (socket says 172.21.15.67:36128) 2024-03-09T09:13:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.820+0000 7fa7c9800640 1 -- 172.21.15.67:0/4152525111 learned_addr learned my addr 172.21.15.67:0/4152525111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.820+0000 7fa7ca200640 1 --2- 172.21.15.67:0/4152525111 >> v2:172.21.15.134:3300/0 conn(0x7fa7c40c6630 0x7fa7c4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.820+0000 7fa7c9800640 1 -- 172.21.15.67:0/4152525111 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c7550 msgr2=0x7fa7c40c60f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:13:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.820+0000 7fa7c9800640 1 --2- 172.21.15.67:0/4152525111 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c7550 0x7fa7c40c60f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.820+0000 7fa7c9800640 1 -- 172.21.15.67:0/4152525111 >> v2:172.21.15.134:3300/0 conn(0x7fa7c40c6630 msgr2=0x7fa7c4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.820+0000 7fa7c9800640 1 --2- 172.21.15.67:0/4152525111 >> v2:172.21.15.134:3300/0 conn(0x7fa7c40c6630 0x7fa7c4159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.820+0000 7fa7c9800640 1 -- 172.21.15.67:0/4152525111 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c40a7300 con 0x7fa7c40c5d10 2024-03-09T09:13:14.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.821+0000 7fa7c9800640 1 --2- 172.21.15.67:0/4152525111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7c40c5d10 0x7fa7c40c3110 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4073400 tx=0x7fa7b802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3b3565e17411d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:14.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.822+0000 7fa7c3e00640 1 -- 172.21.15.67:0/4152525111 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8035020 con 0x7fa7c40c5d10 2024-03-09T09:13:14.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.822+0000 7fa7c3e00640 1 -- 172.21.15.67:0/4152525111 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b8031800 con 0x7fa7c40c5d10 2024-03-09T09:13:14.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.822+0000 7fa7c3e00640 1 -- 172.21.15.67:0/4152525111 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8031bd0 con 0x7fa7c40c5d10 2024-03-09T09:13:14.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.822+0000 7fa7cc200640 1 -- 172.21.15.67:0/4152525111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7c40c5d10 msgr2=0x7fa7c40c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:14.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.822+0000 7fa7cc200640 1 --2- 172.21.15.67:0/4152525111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7c40c5d10 0x7fa7c40c3110 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4073400 tx=0x7fa7b802b8a0 comp rx=0 tx=0).stop 2024-03-09T09:13:14.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.823+0000 7fa7cc200640 1 -- 172.21.15.67:0/4152525111 shutdown_connections 2024-03-09T09:13:14.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.823+0000 7fa7cc200640 1 --2- 172.21.15.67:0/4152525111 >> v2:172.21.15.134:3300/0 conn(0x7fa7c40c6630 0x7fa7c4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.823+0000 7fa7cc200640 1 --2- 172.21.15.67:0/4152525111 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c7550 0x7fa7c40c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.823+0000 7fa7cc200640 1 --2- 172.21.15.67:0/4152525111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7c40c5d10 0x7fa7c40c3110 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.823+0000 7fa7cc200640 1 -- 172.21.15.67:0/4152525111 >> 172.21.15.67:0/4152525111 conn(0x7fa7c40a9950 msgr2=0x7fa7c40a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.823+0000 7fa7cc200640 1 -- 172.21.15.67:0/4152525111 shutdown_connections 2024-03-09T09:13:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.823+0000 7fa7cc200640 1 -- 172.21.15.67:0/4152525111 wait complete. 2024-03-09T09:13:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.824+0000 7fa7cc200640 1 Processor -- start 2024-03-09T09:13:14.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.824+0000 7fa7cc200640 1 -- start start 2024-03-09T09:13:14.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.825+0000 7fa7cc200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7c40c5d10 0x7fa7c4151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:14.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.825+0000 7fa7cc200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c6630 0x7fa7c4154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:14.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.825+0000 7fa7cc200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7c40c7550 0x7fa7c4156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:14.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.825+0000 7fa7cc200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40c3650 con 0x7fa7c40c7550 2024-03-09T09:13:14.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.825+0000 7fa7cc200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4066e20 con 0x7fa7c40c5d10 2024-03-09T09:13:14.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.825+0000 7fa7cc200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c40a6f60 con 0x7fa7c40c6630 2024-03-09T09:13:14.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.825+0000 7fa7c8e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c6630 msgr2=0x7fa7c4154a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:14.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.825+0000 7fa7c8e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c6630 0x7fa7c4154a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:14.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.826+0000 7fa7ca200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7c40c7550 0x7fa7c4156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:14.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.826+0000 7fa7ca200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7c40c7550 0x7fa7c4156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36138/0 (socket says 172.21.15.67:36138) 2024-03-09T09:13:14.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.826+0000 7fa7ca200640 1 -- 172.21.15.67:0/232921599 learned_addr learned my addr 172.21.15.67:0/232921599 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:14.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.826+0000 7fa7c9800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7c40c5d10 0x7fa7c4151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:14.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.826+0000 7fa7ca200640 1 -- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c6630 msgr2=0x7fa7c4154a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:13:14.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.826+0000 7fa7ca200640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c6630 0x7fa7c4154a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:14.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.826+0000 7fa7ca200640 1 -- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7c40c5d10 msgr2=0x7fa7c4151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:14.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.826+0000 7fa7ca200640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7c40c5d10 0x7fa7c4151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:14.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.826+0000 7fa7ca200640 1 -- 172.21.15.67:0/232921599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c41eb850 con 0x7fa7c40c7550 2024-03-09T09:13:14.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.826+0000 7fa7c9800640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7c40c5d10 0x7fa7c4151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:13:14.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.826+0000 7fa7ca200640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7c40c7550 0x7fa7c4156bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc007e70 tx=0x7fa7bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.827+0000 7fa7c1600640 1 -- 172.21.15.67:0/232921599 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc014620 con 0x7fa7c40c7550 2024-03-09T09:13:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.827+0000 7fa7c1600640 1 -- 172.21.15.67:0/232921599 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc019070 con 0x7fa7c40c7550 2024-03-09T09:13:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.827+0000 7fa7c1600640 1 -- 172.21.15.67:0/232921599 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc014a20 con 0x7fa7c40c7550 2024-03-09T09:13:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.827+0000 7fa7cc200640 1 -- 172.21.15.67:0/232921599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c406d000 con 0x7fa7c40c7550 2024-03-09T09:13:14.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.827+0000 7fa7cc200640 1 -- 172.21.15.67:0/232921599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c406d5c0 con 0x7fa7c40c7550 2024-03-09T09:13:14.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.828+0000 7fa7cc200640 1 -- 172.21.15.67:0/232921599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c406d7c0 con 0x7fa7c40c7550 2024-03-09T09:13:14.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.829+0000 7fa7c1600640 1 -- 172.21.15.67:0/232921599 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa7bc014bc0 con 0x7fa7c40c7550 2024-03-09T09:13:14.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.830+0000 7fa7c1600640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7a406b480 0x7fa7a406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:14.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.830+0000 7fa7c1600640 1 -- 172.21.15.67:0/232921599 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa7bc08ae90 con 0x7fa7c40c7550 2024-03-09T09:13:14.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.831+0000 7fa7c9800640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7a406b480 0x7fa7a406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:14.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.832+0000 7fa7c9800640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7a406b480 0x7fa7a406d940 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8036000 tx=0x7fa7b803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:14.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:14.835+0000 7fa7c1600640 1 -- 172.21.15.67:0/232921599 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa7c406d7c0 con 0x7fa7c40c7550 2024-03-09T09:13:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.110+0000 7fa7cc200640 1 -- 172.21.15.67:0/232921599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa7c4075a00 con 0x7fa7c40c7550 2024-03-09T09:13:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.111+0000 7fa7c1600640 1 -- 172.21.15.67:0/232921599 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fa7bc08c080 con 0x7fa7c40c7550 2024-03-09T09:13:15.116 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:13:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 -- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7a406b480 msgr2=0x7fa7a406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7a406b480 0x7fa7a406d940 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8036000 tx=0x7fa7b803c040 comp rx=0 tx=0).stop 2024-03-09T09:13:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 -- 172.21.15.67:0/232921599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7c40c7550 msgr2=0x7fa7c4156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7c40c7550 0x7fa7c4156bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc007e70 tx=0x7fa7bc00e4b0 comp rx=0 tx=0).stop 2024-03-09T09:13:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 -- 172.21.15.67:0/232921599 shutdown_connections 2024-03-09T09:13:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7a406b480 0x7fa7a406d940 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7c40c7550 0x7fa7c4156bf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:15.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7c40c6630 0x7fa7c4154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:15.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 --2- 172.21.15.67:0/232921599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7c40c5d10 0x7fa7c4151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:15.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 -- 172.21.15.67:0/232921599 >> 172.21.15.67:0/232921599 conn(0x7fa7c40a9950 msgr2=0x7fa7c40c1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:15.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 -- 172.21.15.67:0/232921599 shutdown_connections 2024-03-09T09:13:15.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:15.114+0000 7fa7cc200640 1 -- 172.21.15.67:0/232921599 wait complete. 2024-03-09T09:13:15.341 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at c2d517ef966 v0.65 2024-03-09T09:13:15.357 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.65 2024-03-09T09:13:15.660 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:13:15.660 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 66 -eq 48 ']' 2024-03-09T09:13:15.660 INFO:tasks.workunit.client.1.smithi134.stdout:v0.66 2024-03-09T09:13:15.661 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.66 2024-03-09T09:13:15.661 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.66 2024-03-09T09:13:15.661 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.66 2024-03-09T09:13:18.137 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:13:18.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.377+0000 7f9097e00640 1 Processor -- start 2024-03-09T09:13:18.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.377+0000 7f9097e00640 1 -- start start 2024-03-09T09:13:18.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9097e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f90900c30f0 0x7f9090152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:18.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9097e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f90900aa420 0x7f90901490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:18.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9097e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9090149630 0x7f909006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:18.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9097e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90900c82d0 con 0x7f90900aa420 2024-03-09T09:13:18.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9097e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90900ab3c0 con 0x7f90900c30f0 2024-03-09T09:13:18.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9097e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90900aa8a0 con 0x7f9090149630 2024-03-09T09:13:18.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9095e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9090149630 msgr2=0x7f909006dbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:18.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9095e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9090149630 0x7f909006dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:18.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9094a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f90900aa420 0x7f90901490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:18.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9094a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f90900aa420 0x7f90901490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36148/0 (socket says 172.21.15.67:36148) 2024-03-09T09:13:18.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9094a00640 1 -- 172.21.15.67:0/4284133215 learned_addr learned my addr 172.21.15.67:0/4284133215 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:18.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.379+0000 7f9095400640 1 --2- 172.21.15.67:0/4284133215 >> v2:172.21.15.134:3300/0 conn(0x7f90900c30f0 0x7f9090152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:18.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.380+0000 7f9094a00640 1 -- 172.21.15.67:0/4284133215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9090149630 msgr2=0x7f909006dbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:13:18.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.380+0000 7f9094a00640 1 --2- 172.21.15.67:0/4284133215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9090149630 0x7f909006dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.380+0000 7f9094a00640 1 -- 172.21.15.67:0/4284133215 >> v2:172.21.15.134:3300/0 conn(0x7f90900c30f0 msgr2=0x7f9090152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:18.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.380+0000 7f9094a00640 1 --2- 172.21.15.67:0/4284133215 >> v2:172.21.15.134:3300/0 conn(0x7f90900c30f0 0x7f9090152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.380+0000 7f9094a00640 1 -- 172.21.15.67:0/4284133215 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90900aab20 con 0x7f90900aa420 2024-03-09T09:13:18.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.381+0000 7f9094a00640 1 --2- 172.21.15.67:0/4284133215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90900aa420 0x7f90901490f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f90900702f0 tx=0x7f907c02dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2d5b14f8bbeaf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:18.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.381+0000 7f908fe00640 1 -- 172.21.15.67:0/4284133215 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c03b070 con 0x7f90900aa420 2024-03-09T09:13:18.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.382+0000 7f908fe00640 1 -- 172.21.15.67:0/4284133215 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c02ed80 con 0x7f90900aa420 2024-03-09T09:13:18.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.382+0000 7f908fe00640 1 -- 172.21.15.67:0/4284133215 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c036660 con 0x7f90900aa420 2024-03-09T09:13:18.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.382+0000 7f9097e00640 1 -- 172.21.15.67:0/4284133215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90900aa420 msgr2=0x7f90901490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:18.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.382+0000 7f9097e00640 1 --2- 172.21.15.67:0/4284133215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90900aa420 0x7f90901490f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f90900702f0 tx=0x7f907c02dfd0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.382+0000 7f9097e00640 1 -- 172.21.15.67:0/4284133215 shutdown_connections 2024-03-09T09:13:18.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.382+0000 7f9097e00640 1 --2- 172.21.15.67:0/4284133215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9090149630 0x7f909006dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.382+0000 7f9097e00640 1 --2- 172.21.15.67:0/4284133215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90900aa420 0x7f90901490f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.382+0000 7f9097e00640 1 --2- 172.21.15.67:0/4284133215 >> v2:172.21.15.134:3300/0 conn(0x7f90900c30f0 0x7f9090152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.382+0000 7f9097e00640 1 -- 172.21.15.67:0/4284133215 >> 172.21.15.67:0/4284133215 conn(0x7f90900c7370 msgr2=0x7f90900c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:18.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.383+0000 7f9097e00640 1 -- 172.21.15.67:0/4284133215 shutdown_connections 2024-03-09T09:13:18.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.383+0000 7f9097e00640 1 -- 172.21.15.67:0/4284133215 wait complete. 2024-03-09T09:13:18.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.384+0000 7f9097e00640 1 Processor -- start 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.386+0000 7f9097e00640 1 -- start start 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.387+0000 7f9097e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90900aa420 0x7f909014b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.387+0000 7f9097e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90900c30f0 0x7f909014d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.387+0000 7f9097e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9090149630 0x7f909014da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.387+0000 7f9097e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909006d6c0 con 0x7f90900c30f0 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.387+0000 7f9097e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90900ab0e0 con 0x7f9090149630 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.387+0000 7f9097e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90900ab620 con 0x7f90900aa420 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.387+0000 7f9095400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90900aa420 msgr2=0x7f909014b8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.387+0000 7f9095400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90900aa420 0x7f909014b8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.389+0000 7f9094a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90900c30f0 0x7f909014d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.389+0000 7f9094a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90900c30f0 0x7f909014d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36164/0 (socket says 172.21.15.67:36164) 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.389+0000 7f9094a00640 1 -- 172.21.15.67:0/1959447248 learned_addr learned my addr 172.21.15.67:0/1959447248 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:18.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.390+0000 7f9094a00640 1 -- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90900aa420 msgr2=0x7f909014b8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:13:18.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.390+0000 7f9094a00640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90900aa420 0x7f909014b8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.390+0000 7f9094a00640 1 -- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9090149630 msgr2=0x7f909014da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:18.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.390+0000 7f9094a00640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9090149630 0x7f909014da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.390+0000 7f9094a00640 1 -- 172.21.15.67:0/1959447248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90901e7aa0 con 0x7f90900c30f0 2024-03-09T09:13:18.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.390+0000 7f9094a00640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90900c30f0 0x7f909014d4c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f907c00b2c0 tx=0x7f907c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:18.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.391+0000 7f908d600640 1 -- 172.21.15.67:0/1959447248 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c03b040 con 0x7f90900c30f0 2024-03-09T09:13:18.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.391+0000 7f908d600640 1 -- 172.21.15.67:0/1959447248 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c046070 con 0x7f90900c30f0 2024-03-09T09:13:18.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.391+0000 7f908d600640 1 -- 172.21.15.67:0/1959447248 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c041660 con 0x7f90900c30f0 2024-03-09T09:13:18.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.391+0000 7f9097e00640 1 -- 172.21.15.67:0/1959447248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90901e7d80 con 0x7f90900c30f0 2024-03-09T09:13:18.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.391+0000 7f9097e00640 1 -- 172.21.15.67:0/1959447248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90901e8340 con 0x7f90900c30f0 2024-03-09T09:13:18.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.392+0000 7f9097e00640 1 -- 172.21.15.67:0/1959447248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9090075a00 con 0x7f90900c30f0 2024-03-09T09:13:18.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.393+0000 7f908d600640 1 -- 172.21.15.67:0/1959447248 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f907c041800 con 0x7f90900c30f0 2024-03-09T09:13:18.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.394+0000 7f908d600640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f907006b4b0 0x7f907006d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:18.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.394+0000 7f908d600640 1 -- 172.21.15.67:0/1959447248 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f907c0aede0 con 0x7f90900c30f0 2024-03-09T09:13:18.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.395+0000 7f9095400640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f907006b4b0 0x7f907006d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:18.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.397+0000 7f9095400640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f907006b4b0 0x7f907006d970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9084009400 tx=0x7f9084009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:18.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.399+0000 7f908d600640 1 -- 172.21.15.67:0/1959447248 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f907c07d7d0 con 0x7f90900c30f0 2024-03-09T09:13:18.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.674+0000 7f9097e00640 1 -- 172.21.15.67:0/1959447248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f909014cd20 con 0x7f90900c30f0 2024-03-09T09:13:18.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.675+0000 7f908d600640 1 -- 172.21.15.67:0/1959447248 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f907c07d0d0 con 0x7f90900c30f0 2024-03-09T09:13:18.680 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:13:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.677+0000 7f9097e00640 1 -- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f907006b4b0 msgr2=0x7f907006d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.677+0000 7f9097e00640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f907006b4b0 0x7f907006d970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9084009400 tx=0x7f9084009190 comp rx=0 tx=0).stop 2024-03-09T09:13:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.677+0000 7f9097e00640 1 -- 172.21.15.67:0/1959447248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90900c30f0 msgr2=0x7f909014d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.677+0000 7f9097e00640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90900c30f0 0x7f909014d4c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f907c00b2c0 tx=0x7f907c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.678+0000 7f9097e00640 1 -- 172.21.15.67:0/1959447248 shutdown_connections 2024-03-09T09:13:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.678+0000 7f9097e00640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f907006b4b0 0x7f907006d970 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.678+0000 7f9097e00640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9090149630 0x7f909014da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.678+0000 7f9097e00640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90900c30f0 0x7f909014d4c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.678+0000 7f9097e00640 1 --2- 172.21.15.67:0/1959447248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90900aa420 0x7f909014b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:18.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.678+0000 7f9097e00640 1 -- 172.21.15.67:0/1959447248 >> 172.21.15.67:0/1959447248 conn(0x7f90900c7370 msgr2=0x7f9090073820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:18.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.678+0000 7f9097e00640 1 -- 172.21.15.67:0/1959447248 shutdown_connections 2024-03-09T09:13:18.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:18.678+0000 7f9097e00640 1 -- 172.21.15.67:0/1959447248 wait complete. 2024-03-09T09:13:21.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:13:21.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.923+0000 7fcd38200640 1 Processor -- start 2024-03-09T09:13:21.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.924+0000 7fcd38200640 1 -- start start 2024-03-09T09:13:21.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.926+0000 7fcd38200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30151290 0x7fcd300c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:21.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.926+0000 7fcd38200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:21.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.926+0000 7fcd38200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd301574e0 con 0x7fcd300a9480 2024-03-09T09:13:21.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.926+0000 7fcd38200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3006fa20 con 0x7fcd30073930 2024-03-09T09:13:21.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.926+0000 7fcd38200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd301577c0 con 0x7fcd30151290 2024-03-09T09:13:21.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.926+0000 7fcd35800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30151290 msgr2=0x7fcd300c4350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:21.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.926+0000 7fcd35800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30151290 0x7fcd300c4350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:21.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.927+0000 7fcd34e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:21.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.927+0000 7fcd34e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43678/0 (socket says 172.21.15.67:43678) 2024-03-09T09:13:21.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.927+0000 7fcd34e00640 1 -- 172.21.15.67:0/3842639948 learned_addr learned my addr 172.21.15.67:0/3842639948 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:21.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.927+0000 7fcd34e00640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 msgr2=0x7fcd300a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:13:21.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.927+0000 7fcd34e00640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 msgr2=0x7fcd300a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:21.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.927+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:21.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:21.927+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:22.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.127+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30151290 msgr2=0x7fcd300c4350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:22.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.127+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30151290 0x7fcd300c4350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:13:22.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.128+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:22.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.129+0000 7fcd34e00640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 msgr2=0x7fcd300a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:13:22.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.129+0000 7fcd34e00640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 msgr2=0x7fcd300a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:22.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.129+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:22.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.129+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:13:22.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.528+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30151290 msgr2=0x7fcd300c4350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:22.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.528+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30151290 0x7fcd300c4350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:13:22.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.530+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:22.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.530+0000 7fcd34e00640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 msgr2=0x7fcd300a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:13:22.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.530+0000 7fcd34e00640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 msgr2=0x7fcd300a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:22.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.530+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:22.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:22.531+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:13:23.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:23.329+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30151290 msgr2=0x7fcd300c4350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:23.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:23.329+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30151290 0x7fcd300c4350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:13:23.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:23.332+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:23.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:23.333+0000 7fcd34e00640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 msgr2=0x7fcd300a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:13:23.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:23.333+0000 7fcd34e00640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 msgr2=0x7fcd300a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:23.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:23.333+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:23.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:23.333+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:13:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.926+0000 7fcd2ea00640 1 -- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30151290 msgr2=0x7fcd300c4350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:13:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.926+0000 7fcd2ea00640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30151290 0x7fcd300c4350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.926+0000 7fcd2ea00640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 msgr2=0x7fcd300a8f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:13:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.926+0000 7fcd2ea00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.67:3300/0 conn(0x7fcd30073930 0x7fcd300a8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.926+0000 7fcd2ea00640 1 -- 172.21.15.67:0/3842639948 >> v1:172.21.15.134:6789/0 conn(0x7fcd300a9480 legacy=0x7fcd30153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.926+0000 7fcd2ea00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 0x7fcd10005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.927+0000 7fcd2ea00640 1 -- 172.21.15.67:0/3842639948 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd30157aa0 con 0x7fcd10006230 2024-03-09T09:13:24.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.927+0000 7fcd2ea00640 1 -- 172.21.15.67:0/3842639948 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd301577c0 con 0x7fcd100036e0 2024-03-09T09:13:24.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.927+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 0x7fcd10005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:24.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.928+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 msgr2=0x7fcd10005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:13:24.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.928+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 msgr2=0x7fcd10005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:24.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.928+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 0x7fcd10005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:24.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:24.928+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 0x7fcd10005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:25.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.129+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 0x7fcd10005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:25.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.130+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 msgr2=0x7fcd10005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:13:25.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.130+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 msgr2=0x7fcd10005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:25.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.130+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 0x7fcd10005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:25.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.130+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 0x7fcd10005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:13:25.183 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 33% (51/152) Updating files: 34% (52/152) Updating files: 35% (54/152) Updating files: 36% (55/152) Updating files: 37% (57/152) Updating files: 38% (58/152) Updating files: 39% (60/152) Updating files: 40% (61/152) Updating files: 41% (63/152) Updating files: 42% (64/152) Updating files: 43% (66/152) Updating files: 44% (67/152) Updating files: 45% (69/152) Updating files: 46% (70/152) Updating files: 47% (72/152) Updating files: 48% (73/152) Updating files: 49% (75/152) Updating files: 50% (76/152) Updating files: 51% (78/152) Updating files: 52% (80/152) Updating files: 53% (81/152) Updating files: 54% (83/152) Updating files: 55% (84/152) Updating files: 56% (86/152) Updating files: 57% (87/152) Updating files: 58% (89/152) Updating files: 59% (90/152) Updating files: 60% (92/152) Updating files: 61% (93/152) Updating files: 62% (95/152) Updating files: 63% (96/152) Updating files: 64% (98/152) Updating files: 65% (99/152) Updating files: 66% (101/152) Updating files: 67% (102/152) Updating files: 68% (104/152) Updating files: 69% (105/152) Updating files: 70% (107/152) Updating files: 71% (108/152) Updating files: 72% (110/152) Updating files: 73% (111/152) Updating files: 74% (113/152) Updating files: 75% (114/152) Updating files: 76% (116/152) Updating files: 77% (118/152) Updating files: 78% (119/152) Updating files: 79% (121/152) Updating files: 80% (122/152) Updating files: 81% (124/152) Updating files: 82% (125/152) Updating files: 83% (127/152) Updating files: 84% (128/152) Updating files: 85% (130/152) Updating files: 86% (131/152) Updating files: 87% (133/152) Updating files: 88% (134/152) Updating files: 89% (136/152) Updating files: 90% (137/152) Updating files: 91% (139/152) Updating files: 92% (140/152) Updating files: 93% (142/152) Updating files: 94% (143/152) Updating files: 95% (145/152) Updating files: 96% (146/152) Updating files: 97% (148/152) Updating files: 98% (149/152) Updating files: 99% (151/152) Updating files: 100% (152/152) Updating files: 100% (152/152), done. 2024-03-09T09:13:25.255 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at b6b48dbefad v0.66 2024-03-09T09:13:25.273 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.66 2024-03-09T09:13:25.427 INFO:tasks.workunit.client.0.smithi067.stdout:v0.67 2024-03-09T09:13:25.427 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:13:25.428 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 67 -eq 48 ']' 2024-03-09T09:13:25.428 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.67 2024-03-09T09:13:25.428 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.67 2024-03-09T09:13:25.428 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.67 2024-03-09T09:13:25.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.531+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 0x7fcd10005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:25.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.532+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 msgr2=0x7fcd10005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:13:25.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.532+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 msgr2=0x7fcd10005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:25.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.532+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 0x7fcd10005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:25.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.532+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 0x7fcd10005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:13:25.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.753+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091050608 0 0) 0x7fcd30157aa0 con 0x7fcd10006230 2024-03-09T09:13:25.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.754+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd18003520 con 0x7fcd10006230 2024-03-09T09:13:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.754+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2635024564 0 0) 0x7fcd18003520 con 0x7fcd10006230 2024-03-09T09:13:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.754+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 msgr2=0x7fcd10005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:13:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.754+0000 7fcd2fe00640 1 --2- 172.21.15.67:0/3842639948 >> v2:172.21.15.134:3300/0 conn(0x7fcd100036e0 0x7fcd10005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.754+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1000afc0 con 0x7fcd10006230 2024-03-09T09:13:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.754+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fcd1c00f040 con 0x7fcd10006230 2024-03-09T09:13:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.754+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 >> v1:172.21.15.67:6789/0 conn(0x7fcd10006230 legacy=0x7fcd100086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.755+0000 7fcd36200640 1 -- 172.21.15.67:0/3842639948 reap_dead start 2024-03-09T09:13:25.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.755+0000 7fcd2fe00640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd180039b0 0x7fcd18003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.755+0000 7fcd2fe00640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd18004800 0x7fcd18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.755+0000 7fcd2fe00640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180051c0 0x7fcd1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.755+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd301577c0 con 0x7fcd18004800 2024-03-09T09:13:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.755+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1800c0f0 con 0x7fcd180039b0 2024-03-09T09:13:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.755+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18010700 con 0x7fcd180051c0 2024-03-09T09:13:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.755+0000 7fcd36200640 1 -- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180051c0 msgr2=0x7fcd1800b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.755+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd180039b0 0x7fcd18003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:25.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.755+0000 7fcd36200640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180051c0 0x7fcd1800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:25.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.756+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180051c0 msgr2=0x7fcd1800b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:13:25.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.756+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180051c0 0x7fcd1800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:25.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.756+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd18004800 msgr2=0x7fcd18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:25.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.756+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd18004800 0x7fcd18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:25.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.756+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd18004800 0x7fcd18004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:25.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.756+0000 7fcd35800640 1 -- 172.21.15.67:0/3842639948 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1000afc0 con 0x7fcd180039b0 2024-03-09T09:13:25.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.758+0000 7fcd35800640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd180039b0 0x7fcd18003db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcd24003930 tx=0x7fcd18005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a821f4f1fa2b5d9b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:25.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.758+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24009b00 con 0x7fcd180039b0 2024-03-09T09:13:25.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.758+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd24039070 con 0x7fcd180039b0 2024-03-09T09:13:25.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.758+0000 7fcd2fe00640 1 -- 172.21.15.67:0/3842639948 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24034860 con 0x7fcd180039b0 2024-03-09T09:13:25.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.759+0000 7fcd38200640 1 -- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd180039b0 msgr2=0x7fcd18003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:25.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.759+0000 7fcd38200640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd180039b0 0x7fcd18003db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcd24003930 tx=0x7fcd18005830 comp rx=0 tx=0).stop 2024-03-09T09:13:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.759+0000 7fcd38200640 1 -- 172.21.15.67:0/3842639948 shutdown_connections 2024-03-09T09:13:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.759+0000 7fcd38200640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180051c0 0x7fcd1800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.759+0000 7fcd38200640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd18004800 0x7fcd18004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.759+0000 7fcd38200640 1 --2- 172.21.15.67:0/3842639948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd180039b0 0x7fcd18003db0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.759+0000 7fcd38200640 1 -- 172.21.15.67:0/3842639948 >> 172.21.15.67:0/3842639948 conn(0x7fcd300c9710 msgr2=0x7fcd300c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.759+0000 7fcd38200640 1 -- 172.21.15.67:0/3842639948 shutdown_connections 2024-03-09T09:13:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.759+0000 7fcd38200640 1 -- 172.21.15.67:0/3842639948 wait complete. 2024-03-09T09:13:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.761+0000 7fcd38200640 1 Processor -- start 2024-03-09T09:13:25.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.762+0000 7fcd38200640 1 -- start start 2024-03-09T09:13:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.762+0000 7fcd38200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180039b0 0x7fcd300c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd38200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd18004800 0x7fcd30150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd38200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd180051c0 0x7fcd30144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd38200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd30157240 con 0x7fcd180051c0 2024-03-09T09:13:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd38200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd301570c0 con 0x7fcd18004800 2024-03-09T09:13:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd38200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300a8f40 con 0x7fcd180039b0 2024-03-09T09:13:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd35800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180039b0 msgr2=0x7fcd300c57f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd35800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180039b0 0x7fcd300c57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd36200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd180051c0 0x7fcd30144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd34e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd18004800 0x7fcd30150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd34e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd18004800 0x7fcd30150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52700/0 (socket says 172.21.15.67:52700) 2024-03-09T09:13:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd34e00640 1 -- 172.21.15.67:0/3696312776 learned_addr learned my addr 172.21.15.67:0/3696312776 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.763+0000 7fcd36200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd180051c0 0x7fcd30144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43722/0 (socket says 172.21.15.67:43722) 2024-03-09T09:13:25.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.764+0000 7fcd36200640 1 -- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180039b0 msgr2=0x7fcd300c57f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:13:25.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.764+0000 7fcd36200640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180039b0 0x7fcd300c57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:25.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.764+0000 7fcd36200640 1 -- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd18004800 msgr2=0x7fcd30150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.764+0000 7fcd36200640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd18004800 0x7fcd30150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.764+0000 7fcd36200640 1 -- 172.21.15.67:0/3696312776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3014c5d0 con 0x7fcd180051c0 2024-03-09T09:13:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.764+0000 7fcd34e00640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd18004800 0x7fcd30150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:13:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.764+0000 7fcd36200640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd180051c0 0x7fcd30144900 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcd3006bb00 tx=0x7fcd2800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.764+0000 7fcd2d600640 1 -- 172.21.15.67:0/3696312776 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28017070 con 0x7fcd180051c0 2024-03-09T09:13:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.764+0000 7fcd2d600640 1 -- 172.21.15.67:0/3696312776 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd280070d0 con 0x7fcd180051c0 2024-03-09T09:13:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.765+0000 7fcd38200640 1 -- 172.21.15.67:0/3696312776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd300c3eb0 con 0x7fcd180051c0 2024-03-09T09:13:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.765+0000 7fcd2d600640 1 -- 172.21.15.67:0/3696312776 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280074c0 con 0x7fcd180051c0 2024-03-09T09:13:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.765+0000 7fcd38200640 1 -- 172.21.15.67:0/3696312776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd30154f60 con 0x7fcd180051c0 2024-03-09T09:13:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.766+0000 7fcd38200640 1 -- 172.21.15.67:0/3696312776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd300750f0 con 0x7fcd180051c0 2024-03-09T09:13:25.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.769+0000 7fcd2d600640 1 -- 172.21.15.67:0/3696312776 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcd2801a070 con 0x7fcd180051c0 2024-03-09T09:13:25.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.769+0000 7fcd2d600640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd1006c0f0 0x7fcd1006e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:25.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.770+0000 7fcd2d600640 1 -- 172.21.15.67:0/3696312776 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcd28089950 con 0x7fcd180051c0 2024-03-09T09:13:25.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.770+0000 7fcd35800640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd1006c0f0 0x7fcd1006e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:25.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.771+0000 7fcd35800640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd1006c0f0 0x7fcd1006e5b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcd24038000 tx=0x7fcd2400f040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:25.777+0000 7fcd2d600640 1 -- 172.21.15.67:0/3696312776 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcd3014c5d0 con 0x7fcd180051c0 2024-03-09T09:13:25.850 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 19% (30/152) Updating files: 20% (31/152) Updating files: 21% (32/152) Updating files: 22% (34/152) Updating files: 23% (35/152) Updating files: 24% (37/152) Updating files: 25% (38/152) Updating files: 26% (40/152) Updating files: 27% (42/152) Updating files: 28% (43/152) Updating files: 29% (45/152) Updating files: 30% (46/152) Updating files: 31% (48/152) Updating files: 32% (49/152) Updating files: 33% (51/152) Updating files: 34% (52/152) Updating files: 35% (54/152) Updating files: 36% (55/152) Updating files: 37% (57/152) Updating files: 38% (58/152) Updating files: 39% (60/152) Updating files: 40% (61/152) Updating files: 41% (63/152) Updating files: 42% (64/152) Updating files: 43% (66/152) Updating files: 44% (67/152) Updating files: 45% (69/152) Updating files: 46% (70/152) Updating files: 47% (72/152) Updating files: 48% (73/152) Updating files: 49% (75/152) Updating files: 50% (76/152) Updating files: 50% (77/152) Updating files: 51% (78/152) Updating files: 52% (80/152) Updating files: 53% (81/152) Updating files: 54% (83/152) Updating files: 55% (84/152) Updating files: 56% (86/152) Updating files: 57% (87/152) Updating files: 58% (89/152) Updating files: 59% (90/152) Updating files: 60% (92/152) Updating files: 61% (93/152) Updating files: 62% (95/152) Updating files: 63% (96/152) Updating files: 64% (98/152) Updating files: 65% (99/152) Updating files: 66% (101/152) Updating files: 67% (102/152) Updating files: 68% (104/152) Updating files: 69% (105/152) Updating files: 70% (107/152) Updating files: 71% (108/152) Updating files: 72% (110/152) Updating files: 73% (111/152) Updating files: 74% (113/152) Updating files: 75% (114/152) Updating files: 76% (116/152) Updating files: 77% (118/152) Updating files: 78% (119/152) Updating files: 79% (121/152) Updating files: 80% (122/152) Updating files: 81% (124/152) Updating files: 82% (125/152) Updating files: 83% (127/152) Updating files: 84% (128/152) Updating files: 85% (130/152) Updating files: 86% (131/152) Updating files: 87% (133/152) Updating files: 88% (134/152) Updating files: 89% (136/152) Updating files: 90% (137/152) Updating files: 91% (139/152) Updating files: 92% (140/152) Updating files: 93% (142/152) Updating files: 94% (143/152) Updating files: 95% (145/152) Updating files: 96% (146/152) Updating files: 97% (148/152) Updating files: 98% (149/152) Updating files: 99% (151/152) Updating files: 100% (152/152) Updating files: 100% (152/152), done. 2024-03-09T09:13:25.906 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at b6b48dbefad v0.66 2024-03-09T09:13:25.915 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.66 2024-03-09T09:13:26.059 INFO:tasks.workunit.client.1.smithi134.stdout:v0.67 2024-03-09T09:13:26.060 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:13:26.060 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 67 -eq 48 ']' 2024-03-09T09:13:26.060 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.67 2024-03-09T09:13:26.060 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.67 2024-03-09T09:13:26.060 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.67 2024-03-09T09:13:26.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.172+0000 7fcd38200640 1 -- 172.21.15.67:0/3696312776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcd30075a00 con 0x7fcd180051c0 2024-03-09T09:13:26.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.173+0000 7fcd2d600640 1 -- 172.21.15.67:0/3696312776 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fcd280583d0 con 0x7fcd180051c0 2024-03-09T09:13:26.179 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":156,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:13:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 -- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd1006c0f0 msgr2=0x7fcd1006e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd1006c0f0 0x7fcd1006e5b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcd24038000 tx=0x7fcd2400f040 comp rx=0 tx=0).stop 2024-03-09T09:13:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 -- 172.21.15.67:0/3696312776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd180051c0 msgr2=0x7fcd30144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd180051c0 0x7fcd30144900 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcd3006bb00 tx=0x7fcd2800c8c0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 -- 172.21.15.67:0/3696312776 shutdown_connections 2024-03-09T09:13:26.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd1006c0f0 0x7fcd1006e5b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd180051c0 0x7fcd30144900 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd18004800 0x7fcd30150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 --2- 172.21.15.67:0/3696312776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd180039b0 0x7fcd300c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 -- 172.21.15.67:0/3696312776 >> 172.21.15.67:0/3696312776 conn(0x7fcd300c9710 msgr2=0x7fcd300706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 -- 172.21.15.67:0/3696312776 shutdown_connections 2024-03-09T09:13:26.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.176+0000 7fcd38200640 1 -- 172.21.15.67:0/3696312776 wait complete. 2024-03-09T09:13:26.200 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:13:26.200 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:13:26.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:13:26.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.415+0000 7f0d27c00640 1 Processor -- start 2024-03-09T09:13:26.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.416+0000 7f0d27c00640 1 -- start start 2024-03-09T09:13:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.417+0000 7f0d27c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0d200c1240 0x7f0d20159e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.417+0000 7f0d27c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d200c6ab0 0x7f0d200aba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.417+0000 7f0d27c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d200670b0 con 0x7f0d2015a340 2024-03-09T09:13:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.417+0000 7f0d27c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d200c74b0 con 0x7f0d200c1240 2024-03-09T09:13:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.417+0000 7f0d27c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200c26d0 con 0x7f0d200c6ab0 2024-03-09T09:13:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.417+0000 7f0d25c00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d200c6ab0 msgr2=0x7f0d200aba20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.417+0000 7f0d25c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d200c6ab0 0x7f0d200aba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.417+0000 7f0d25200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0d200c1240 0x7f0d20159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.417+0000 7f0d25200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0d200c1240 0x7f0d20159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52702/0 (socket says 172.21.15.67:52702) 2024-03-09T09:13:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.417+0000 7f0d25200640 1 -- 172.21.15.67:0/3133987791 learned_addr learned my addr 172.21.15.67:0/3133987791 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.418+0000 7f0d25200640 1 -- 172.21.15.67:0/3133987791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d200c6ab0 msgr2=0x7f0d200aba20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:13:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.418+0000 7f0d25200640 1 --2- 172.21.15.67:0/3133987791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d200c6ab0 0x7f0d200aba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.418+0000 7f0d25200640 1 -- 172.21.15.67:0/3133987791 >> v1:172.21.15.134:6789/0 conn(0x7f0d2015a340 legacy=0x7f0d200ab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.418+0000 7f0d25200640 1 -- 172.21.15.67:0/3133987791 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d200c7700 con 0x7f0d200c1240 2024-03-09T09:13:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.420+0000 7f0d25200640 1 --2- 172.21.15.67:0/3133987791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d200c1240 0x7f0d20159e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009870 tx=0x7f0d1402b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=613e365bf1ce34f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.420+0000 7f0d1f400640 1 -- 172.21.15.67:0/3133987791 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1402e070 con 0x7f0d200c1240 2024-03-09T09:13:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.420+0000 7f0d1f400640 1 -- 172.21.15.67:0/3133987791 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d14037070 con 0x7f0d200c1240 2024-03-09T09:13:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.420+0000 7f0d1f400640 1 -- 172.21.15.67:0/3133987791 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d14031a00 con 0x7f0d200c1240 2024-03-09T09:13:26.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.421+0000 7f0d27c00640 1 -- 172.21.15.67:0/3133987791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d200c1240 msgr2=0x7f0d20159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.421+0000 7f0d27c00640 1 --2- 172.21.15.67:0/3133987791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d200c1240 0x7f0d20159e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009870 tx=0x7f0d1402b880 comp rx=0 tx=0).stop 2024-03-09T09:13:26.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.421+0000 7f0d27c00640 1 -- 172.21.15.67:0/3133987791 shutdown_connections 2024-03-09T09:13:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.421+0000 7f0d27c00640 1 --2- 172.21.15.67:0/3133987791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d200c6ab0 0x7f0d200aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.421+0000 7f0d27c00640 1 --2- 172.21.15.67:0/3133987791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d200c1240 0x7f0d20159e00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.421+0000 7f0d27c00640 1 -- 172.21.15.67:0/3133987791 >> 172.21.15.67:0/3133987791 conn(0x7f0d200a9950 msgr2=0x7f0d200a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.421+0000 7f0d27c00640 1 -- 172.21.15.67:0/3133987791 shutdown_connections 2024-03-09T09:13:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.421+0000 7f0d27c00640 1 -- 172.21.15.67:0/3133987791 wait complete. 2024-03-09T09:13:26.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.423+0000 7f0d27c00640 1 Processor -- start 2024-03-09T09:13:26.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.423+0000 7f0d27c00640 1 -- start start 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d27c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d200c1240 0x7f0d2009c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d27c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d200c6ab0 0x7f0d20156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d27c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d2015a340 0x7f0d2006cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d27c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200abf60 con 0x7f0d200c6ab0 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d27c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2015a7f0 con 0x7f0d2015a340 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d27c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200aade0 con 0x7f0d200c1240 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d25200640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d200c1240 msgr2=0x7f0d2009c610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d25200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d200c1240 0x7f0d2009c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d1fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d200c6ab0 0x7f0d20156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d25c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d2015a340 0x7f0d2006cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d1fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d200c6ab0 0x7f0d20156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43730/0 (socket says 172.21.15.67:43730) 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.424+0000 7f0d1fe00640 1 -- 172.21.15.67:0/3262388106 learned_addr learned my addr 172.21.15.67:0/3262388106 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.425+0000 7f0d25c00640 1 -- 172.21.15.67:0/3262388106 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d200c1240 msgr2=0x7f0d2009c610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:13:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.425+0000 7f0d25c00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d200c1240 0x7f0d2009c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.425+0000 7f0d25c00640 1 -- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d200c6ab0 msgr2=0x7f0d20156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.425+0000 7f0d25c00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d200c6ab0 0x7f0d20156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.425+0000 7f0d25c00640 1 -- 172.21.15.67:0/3262388106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d200a9470 con 0x7f0d2015a340 2024-03-09T09:13:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.425+0000 7f0d1fe00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d200c6ab0 0x7f0d20156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:13:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.425+0000 7f0d25c00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d2015a340 0x7f0d2006cdb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800bd80 tx=0x7f0d18009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.425+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800ce10 con 0x7f0d2015a340 2024-03-09T09:13:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.425+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d18010070 con 0x7f0d2015a340 2024-03-09T09:13:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.425+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d180156b0 con 0x7f0d2015a340 2024-03-09T09:13:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.425+0000 7f0d27c00640 1 -- 172.21.15.67:0/3262388106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d201ef730 con 0x7f0d2015a340 2024-03-09T09:13:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.426+0000 7f0d27c00640 1 -- 172.21.15.67:0/3262388106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d20156640 con 0x7f0d2015a340 2024-03-09T09:13:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.427+0000 7f0d27c00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2009d770 0x7f0d20153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.427+0000 7f0d27c00640 1 -- 172.21.15.67:0/3262388106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2004f3c0 con 0x7f0d2009d770 2024-03-09T09:13:26.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.427+0000 7f0d25200640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2009d770 0x7f0d20153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.428+0000 7f0d25200640 1 -- 172.21.15.67:0/3262388106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d201efa80 con 0x7f0d2009d770 2024-03-09T09:13:26.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.428+0000 7f0d25200640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2009d770 0x7f0d20153a70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0d200c5710 tx=0x7f0d140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:26.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.428+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0d18015910 con 0x7f0d2015a340 2024-03-09T09:13:26.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.429+0000 7f0d1cc00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0d0006b480 0x7f0d0006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.429+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0d1808a950 con 0x7f0d2015a340 2024-03-09T09:13:26.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.430+0000 7f0d1fe00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0d0006b480 0x7f0d0006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.430+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d14041070 con 0x7f0d2009d770 2024-03-09T09:13:26.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.430+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f0d14037040 con 0x7f0d2009d770 2024-03-09T09:13:26.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.430+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2009d770 msgr2=0x7f0d20153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.430+0000 7f0d1cc00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2009d770 0x7f0d20153a70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0d200c5710 tx=0x7f0d140046c0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.430+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2009d770 msgr2=0x7f0d20153a70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:13:26.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.430+0000 7f0d1cc00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2009d770 0x7f0d20153a70 secure :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0d200c5710 tx=0x7f0d140046c0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.431+0000 7f0d1fe00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0d0006b480 0x7f0d0006d940 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0d201e9410 tx=0x7f0d0c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:26.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.438+0000 7f0d27c00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2006ec20 0x7f0d20153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.438+0000 7f0d27c00640 1 -- 172.21.15.67:0/3262388106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2009c330 con 0x7f0d2006ec20 2024-03-09T09:13:26.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.438+0000 7f0d25200640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2006ec20 0x7f0d20153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.439+0000 7f0d25200640 1 -- 172.21.15.67:0/3262388106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0d2006bfd0 con 0x7f0d2006ec20 2024-03-09T09:13:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.439+0000 7f0d25200640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2006ec20 0x7f0d20153fb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0d14000c00 tx=0x7f0d14009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.439+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1404a2d0 con 0x7f0d2006ec20 2024-03-09T09:13:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7f0d14003610 con 0x7f0d2006ec20 2024-03-09T09:13:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2006ec20 msgr2=0x7f0d20153fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d1cc00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2006ec20 0x7f0d20153fb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0d14000c00 tx=0x7f0d14009240 comp rx=0 tx=0).stop 2024-03-09T09:13:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d1cc00640 1 -- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2006ec20 msgr2=0x7f0d20153fb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:13:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d1cc00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0d2006ec20 0x7f0d20153fb0 secure :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0d14000c00 tx=0x7f0d14009240 comp rx=0 tx=0).stop 2024-03-09T09:13:26.446 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:13:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:13:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:13:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:13:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 156, 2024-03-09T09:13:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:13:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:13:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:13:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:26.446 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:26.447 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:26.448 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:13:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:13:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:13:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:13:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:13:26.449 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:26.450 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.451 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:26.453 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:13:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:13:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:13:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:13:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:13:26.455 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:13:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:13:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:26.456 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.457 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:13:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:13:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:13:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:13:26.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:13:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:13:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:26.459 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:13:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:13:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:13:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:26.460 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:13:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:13:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:26.461 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:13:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:13:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.462 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:13:26.462 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:13:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d27c00640 1 -- 172.21.15.67:0/3262388106 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0d0006b480 msgr2=0x7f0d0006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d27c00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0d0006b480 0x7f0d0006d940 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0d201e9410 tx=0x7f0d0c009040 comp rx=0 tx=0).stop 2024-03-09T09:13:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d27c00640 1 -- 172.21.15.67:0/3262388106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d2015a340 msgr2=0x7f0d2006cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d27c00640 1 --2- 172.21.15.67:0/3262388106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d2015a340 0x7f0d2006cdb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800bd80 tx=0x7f0d18009ef0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d25c00640 1 -- 172.21.15.67:0/3262388106 reap_dead start 2024-03-09T09:13:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d25c00640 1 -- 172.21.15.67:0/3262388106 reap_dead start 2024-03-09T09:13:26.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d27c00640 1 -- 172.21.15.67:0/3262388106 shutdown_connections 2024-03-09T09:13:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d27c00640 1 -- 172.21.15.67:0/3262388106 >> 172.21.15.67:0/3262388106 conn(0x7f0d200a9950 msgr2=0x7f0d200c1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d27c00640 1 -- 172.21.15.67:0/3262388106 shutdown_connections 2024-03-09T09:13:26.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.440+0000 7f0d27c00640 1 -- 172.21.15.67:0/3262388106 wait complete. 2024-03-09T09:13:26.465 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:13:26.465 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:13:26.465 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:13:26.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.656+0000 7f0f35000640 1 Processor -- start 2024-03-09T09:13:26.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.656+0000 7f0f35000640 1 -- start start 2024-03-09T09:13:26.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.657+0000 7f0f35000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f300bec10 0x7f0f30072770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.657+0000 7f0f35000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0f300c9410 0x7f0f3006b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.657+0000 7f0f35000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f300c6010 con 0x7f0f300b0db0 2024-03-09T09:13:26.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.657+0000 7f0f35000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f300c62f0 con 0x7f0f300bec10 2024-03-09T09:13:26.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.657+0000 7f0f35000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f300c65d0 con 0x7f0f300c9410 2024-03-09T09:13:26.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.657+0000 7f0f2ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f300bec10 0x7f0f30072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.658+0000 7f0f2ea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f300bec10 0x7f0f30072770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43754/0 (socket says 172.21.15.67:43754) 2024-03-09T09:13:26.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.658+0000 7f0f2ea00640 1 -- 172.21.15.67:0/3309287697 learned_addr learned my addr 172.21.15.67:0/3309287697 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:26.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.658+0000 7f0f2fe00640 1 --2- 172.21.15.67:0/3309287697 >> v2:172.21.15.134:3300/0 conn(0x7f0f300c9410 0x7f0f3006b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.658+0000 7f0f2ea00640 1 -- 172.21.15.67:0/3309287697 >> v2:172.21.15.134:3300/0 conn(0x7f0f300c9410 msgr2=0x7f0f3006b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:26.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.658+0000 7f0f2ea00640 1 --2- 172.21.15.67:0/3309287697 >> v2:172.21.15.134:3300/0 conn(0x7f0f300c9410 0x7f0f3006b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.658+0000 7f0f2ea00640 1 -- 172.21.15.67:0/3309287697 >> v1:172.21.15.134:6789/0 conn(0x7f0f300b0db0 legacy=0x7f0f301565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.658+0000 7f0f2ea00640 1 -- 172.21.15.67:0/3309287697 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3006b040 con 0x7f0f300bec10 2024-03-09T09:13:26.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.659+0000 7f0f2ea00640 1 --2- 172.21.15.67:0/3309287697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f300bec10 0x7f0f30072770 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0f3006c670 tx=0x7f0f1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e491ab011ca0ade1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:26.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.660+0000 7f0f2e000640 1 -- 172.21.15.67:0/3309287697 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c035020 con 0x7f0f300bec10 2024-03-09T09:13:26.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.660+0000 7f0f2e000640 1 -- 172.21.15.67:0/3309287697 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c031870 con 0x7f0f300bec10 2024-03-09T09:13:26.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.660+0000 7f0f2e000640 1 -- 172.21.15.67:0/3309287697 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c031c40 con 0x7f0f300bec10 2024-03-09T09:13:26.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.660+0000 7f0f35000640 1 -- 172.21.15.67:0/3309287697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f300bec10 msgr2=0x7f0f30072770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.660+0000 7f0f35000640 1 --2- 172.21.15.67:0/3309287697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f300bec10 0x7f0f30072770 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0f3006c670 tx=0x7f0f1c02b910 comp rx=0 tx=0).stop 2024-03-09T09:13:26.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.661+0000 7f0f35000640 1 -- 172.21.15.67:0/3309287697 shutdown_connections 2024-03-09T09:13:26.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.661+0000 7f0f35000640 1 --2- 172.21.15.67:0/3309287697 >> v2:172.21.15.134:3300/0 conn(0x7f0f300c9410 0x7f0f3006b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.661+0000 7f0f35000640 1 --2- 172.21.15.67:0/3309287697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f300bec10 0x7f0f30072770 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.661+0000 7f0f35000640 1 -- 172.21.15.67:0/3309287697 >> 172.21.15.67:0/3309287697 conn(0x7f0f300c7f90 msgr2=0x7f0f300c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:26.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.661+0000 7f0f35000640 1 -- 172.21.15.67:0/3309287697 shutdown_connections 2024-03-09T09:13:26.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.661+0000 7f0f35000640 1 -- 172.21.15.67:0/3309287697 wait complete. 2024-03-09T09:13:26.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.663+0000 7f0f35000640 1 Processor -- start 2024-03-09T09:13:26.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.663+0000 7f0f35000640 1 -- start start 2024-03-09T09:13:26.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f35000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f300b0db0 0x7f0f30148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f35000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f300bec10 0x7f0f3014dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f35000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f300c9410 0x7f0f300c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f35000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30072e30 con 0x7f0f300c9410 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f35000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30072cb0 con 0x7f0f300b0db0 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f35000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30072fb0 con 0x7f0f300bec10 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f2ea00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f300bec10 msgr2=0x7f0f3014dd40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f2ea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f300bec10 0x7f0f3014dd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f300c9410 0x7f0f300c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f2fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f300c9410 0x7f0f300c35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43760/0 (socket says 172.21.15.67:43760) 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f2fe00640 1 -- 172.21.15.67:0/3136358279 learned_addr learned my addr 172.21.15.67:0/3136358279 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f2f400640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f300b0db0 0x7f0f30148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f2fe00640 1 -- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f300bec10 msgr2=0x7f0f3014dd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:13:26.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f2fe00640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f300bec10 0x7f0f3014dd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f2fe00640 1 -- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f300b0db0 msgr2=0x7f0f30148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f2fe00640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f300b0db0 0x7f0f30148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.664+0000 7f0f2fe00640 1 -- 172.21.15.67:0/3136358279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f30156970 con 0x7f0f300c9410 2024-03-09T09:13:26.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.665+0000 7f0f2fe00640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f300c9410 0x7f0f300c35f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009dd0 tx=0x7f0f2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:26.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.665+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28018020 con 0x7f0f300c9410 2024-03-09T09:13:26.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.665+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f28014480 con 0x7f0f300c9410 2024-03-09T09:13:26.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.665+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28014870 con 0x7f0f300c9410 2024-03-09T09:13:26.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.665+0000 7f0f35000640 1 -- 172.21.15.67:0/3136358279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f301ec2a0 con 0x7f0f300c9410 2024-03-09T09:13:26.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.665+0000 7f0f35000640 1 -- 172.21.15.67:0/3136358279 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f30148980 con 0x7f0f300c9410 2024-03-09T09:13:26.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.666+0000 7f0f35000640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f300c5570 0x7f0f300c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.666+0000 7f0f35000640 1 -- 172.21.15.67:0/3136358279 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f300c72f0 con 0x7f0f300c5570 2024-03-09T09:13:26.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.667+0000 7f0f2f400640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f300c5570 0x7f0f300c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.667+0000 7f0f2f400640 1 -- 172.21.15.67:0/3136358279 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f30156720 con 0x7f0f300c5570 2024-03-09T09:13:26.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.668+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0f28005d10 con 0x7f0f300c9410 2024-03-09T09:13:26.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.668+0000 7f0f2f400640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f300c5570 0x7f0f300c3b30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000f540 tx=0x7f0f2000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:26.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.669+0000 7f0f27400640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f1006b480 0x7f0f1006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.669+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0f2808aff0 con 0x7f0f300c9410 2024-03-09T09:13:26.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.669+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f200052e0 con 0x7f0f300c5570 2024-03-09T09:13:26.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.669+0000 7f0f2ea00640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f1006b480 0x7f0f1006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.669+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f0f20018070 con 0x7f0f300c5570 2024-03-09T09:13:26.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.669+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f300c5570 msgr2=0x7f0f300c3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.669+0000 7f0f27400640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f300c5570 0x7f0f300c3b30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000f540 tx=0x7f0f2000fa00 comp rx=0 tx=0).stop 2024-03-09T09:13:26.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.669+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f300c5570 msgr2=0x7f0f300c3b30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:13:26.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.669+0000 7f0f27400640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f300c5570 0x7f0f300c3b30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:26.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.670+0000 7f0f2ea00640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f1006b480 0x7f0f1006d940 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c036000 tx=0x7f0f1c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:26.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.679+0000 7f0f35000640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3014e280 0x7f0f301570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:26.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.679+0000 7f0f35000640 1 -- 172.21.15.67:0/3136358279 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f301ec7c0 con 0x7f0f3014e280 2024-03-09T09:13:26.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.679+0000 7f0f2f400640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3014e280 0x7f0f301570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:26.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.681+0000 7f0f2f400640 1 -- 172.21.15.67:0/3136358279 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0f3004f3c0 con 0x7f0f3014e280 2024-03-09T09:13:26.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.681+0000 7f0f2f400640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3014e280 0x7f0f301570d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000f540 tx=0x7f0f20001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:26.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.682+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20003b40 con 0x7f0f3014e280 2024-03-09T09:13:26.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.682+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4523 (secure 0 0 0) 0x7f0f20003b40 con 0x7f0f3014e280 2024-03-09T09:13:26.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.682+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3014e280 msgr2=0x7f0f301570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.682+0000 7f0f27400640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3014e280 0x7f0f301570d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000f540 tx=0x7f0f20001080 comp rx=0 tx=0).stop 2024-03-09T09:13:26.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.682+0000 7f0f27400640 1 -- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3014e280 msgr2=0x7f0f301570d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:13:26.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.682+0000 7f0f27400640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f3014e280 0x7f0f301570d0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000f540 tx=0x7f0f20001080 comp rx=0 tx=0).stop 2024-03-09T09:13:26.688 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:13:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:13:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:13:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:13:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 156, 2024-03-09T09:13:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:13:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:13:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:13:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:26.688 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:26.689 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:26.690 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:13:26.691 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:13:26.692 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:13:26.693 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.694 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:26.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:13:26.696 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.697 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.698 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:13:26.699 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:13:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:13:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:13:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:13:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:13:26.700 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:13:26.700 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.683+0000 7f0f35000640 1 -- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f1006b480 msgr2=0x7f0f1006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.683+0000 7f0f35000640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f1006b480 0x7f0f1006d940 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c036000 tx=0x7f0f1c008030 comp rx=0 tx=0).stop 2024-03-09T09:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.683+0000 7f0f35000640 1 -- 172.21.15.67:0/3136358279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f300c9410 msgr2=0x7f0f300c35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.683+0000 7f0f35000640 1 --2- 172.21.15.67:0/3136358279 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f300c9410 0x7f0f300c35f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009dd0 tx=0x7f0f2800c490 comp rx=0 tx=0).stop 2024-03-09T09:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.683+0000 7f0f2fe00640 1 -- 172.21.15.67:0/3136358279 reap_dead start 2024-03-09T09:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.683+0000 7f0f2fe00640 1 -- 172.21.15.67:0/3136358279 reap_dead start 2024-03-09T09:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.683+0000 7f0f35000640 1 -- 172.21.15.67:0/3136358279 shutdown_connections 2024-03-09T09:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.683+0000 7f0f35000640 1 -- 172.21.15.67:0/3136358279 >> 172.21.15.67:0/3136358279 conn(0x7f0f300c7f90 msgr2=0x7f0f300c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.683+0000 7f0f35000640 1 -- 172.21.15.67:0/3136358279 shutdown_connections 2024-03-09T09:13:26.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:26.683+0000 7f0f35000640 1 -- 172.21.15.67:0/3136358279 wait complete. 2024-03-09T09:13:26.710 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:13:26.710 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:13:39.847 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 46% (164/355) Updating files: 47% (167/355) Updating files: 48% (171/355) Updating files: 49% (174/355) Updating files: 50% (178/355) Updating files: 51% (182/355) Updating files: 52% (185/355) Updating files: 53% (189/355) Updating files: 54% (192/355) Updating files: 55% (196/355) Updating files: 56% (199/355) Updating files: 56% (202/355) Updating files: 57% (203/355) Updating files: 58% (206/355) Updating files: 59% (210/355) Updating files: 59% (211/355) Updating files: 60% (213/355) Updating files: 61% (217/355) Updating files: 62% (221/355) Updating files: 62% (222/355) Updating files: 63% (224/355) Updating files: 64% (228/355) Updating files: 65% (231/355) Updating files: 66% (235/355) Updating files: 67% (238/355) Updating files: 68% (242/355) Updating files: 69% (245/355) Updating files: 70% (249/355) Updating files: 71% (253/355) Updating files: 72% (256/355) Updating files: 73% (260/355) Updating files: 74% (263/355) Updating files: 75% (267/355) Updating files: 76% (270/355) Updating files: 77% (274/355) Updating files: 78% (277/355) Updating files: 79% (281/355) Updating files: 80% (284/355) Updating files: 81% (288/355) Updating files: 82% (292/355) Updating files: 83% (295/355) Updating files: 84% (299/355) Updating files: 85% (302/355) Updating files: 86% (306/355) Updating files: 87% (309/355) Updating files: 88% (313/355) Updating files: 89% (316/355) Updating files: 90% (320/355) Updating files: 91% (324/355) Updating files: 92% (327/355) Updating files: 93% (331/355) Updating files: 94% (334/355) Updating files: 95% (338/355) Updating files: 96% (341/355) Updating files: 97% (345/355) Updating files: 98% (348/355) Updating files: 99% (352/355) Updating files: 100% (355/355) Updating files: 100% (355/355), done. 2024-03-09T09:13:39.885 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at e3b7bc5bce8 v0.67 2024-03-09T09:13:39.893 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.67 2024-03-09T09:13:40.235 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 5% (19/355) Updating files: 6% (22/355) Updating files: 7% (25/355) Updating files: 8% (29/355) Updating files: 9% (32/355) Updating files: 10% (36/355) Updating files: 11% (40/355) Updating files: 12% (43/355) Updating files: 13% (47/355) Updating files: 14% (50/355) Updating files: 15% (54/355) Updating files: 16% (57/355) Updating files: 17% (61/355) Updating files: 18% (64/355) Updating files: 19% (68/355) Updating files: 20% (71/355) Updating files: 21% (75/355) Updating files: 22% (79/355) Updating files: 23% (82/355) Updating files: 24% (86/355) Updating files: 24% (88/355) Updating files: 25% (89/355) Updating files: 26% (93/355) Updating files: 27% (96/355) Updating files: 28% (100/355) Updating files: 29% (103/355) Updating files: 30% (107/355) Updating files: 31% (111/355) Updating files: 32% (114/355) Updating files: 33% (118/355) Updating files: 34% (121/355) Updating files: 35% (125/355) Updating files: 36% (128/355) Updating files: 37% (132/355) Updating files: 38% (135/355) Updating files: 38% (136/355) Updating files: 39% (139/355) Updating files: 40% (142/355) Updating files: 41% (146/355) Updating files: 42% (150/355) Updating files: 43% (153/355) Updating files: 44% (157/355) Updating files: 45% (160/355) Updating files: 46% (164/355) Updating files: 47% (167/355) Updating files: 48% (171/355) Updating files: 49% (174/355) Updating files: 50% (178/355) Updating files: 50% (180/355) Updating files: 51% (182/355) Updating files: 52% (185/355) Updating files: 53% (189/355) Updating files: 54% (192/355) Updating files: 55% (196/355) Updating files: 56% (199/355) Updating files: 57% (203/355) Updating files: 58% (206/355) Updating files: 59% (210/355) Updating files: 60% (213/355) Updating files: 61% (217/355) Updating files: 62% (221/355) Updating files: 63% (224/355) Updating files: 64% (228/355) Updating files: 65% (231/355) Updating files: 66% (235/355) Updating files: 67% (238/355) Updating files: 68% (242/355) Updating files: 69% (245/355) Updating files: 70% (249/355) Updating files: 71% (253/355) Updating files: 72% (256/355) Updating files: 73% (260/355) Updating files: 74% (263/355) Updating files: 75% (267/355) Updating files: 76% (270/355) Updating files: 77% (274/355) Updating files: 78% (277/355) Updating files: 79% (281/355) Updating files: 80% (284/355) Updating files: 80% (287/355) Updating files: 81% (288/355) Updating files: 82% (292/355) Updating files: 83% (295/355) Updating files: 84% (299/355) Updating files: 85% (302/355) Updating files: 86% (306/355) Updating files: 87% (309/355) Updating files: 88% (313/355) Updating files: 89% (316/355) Updating files: 90% (320/355) Updating files: 91% (324/355) Updating files: 92% (327/355) Updating files: 93% (331/355) Updating files: 94% (334/355) Updating files: 95% (338/355) Updating files: 96% (341/355) Updating files: 97% (345/355) Updating files: 98% (348/355) Updating files: 99% (352/355) Updating files: 100% (355/355) Updating files: 100% (355/355), done. 2024-03-09T09:13:40.282 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at e3b7bc5bce8 v0.67 2024-03-09T09:13:40.290 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.67 2024-03-09T09:13:40.430 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:13:40.430 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 68 -eq 48 ']' 2024-03-09T09:13:40.430 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.68 2024-03-09T09:13:40.430 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.68 2024-03-09T09:13:40.430 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.68 2024-03-09T09:13:40.431 INFO:tasks.workunit.client.1.smithi134.stdout:v0.68 2024-03-09T09:13:40.457 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:13:40.457 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 68 -eq 48 ']' 2024-03-09T09:13:40.457 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.68 2024-03-09T09:13:40.457 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.68 2024-03-09T09:13:40.457 INFO:tasks.workunit.client.0.smithi067.stdout:v0.68 2024-03-09T09:13:40.458 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.68 2024-03-09T09:13:43.994 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 21% (51/241) Updating files: 22% (54/241) Updating files: 23% (56/241) Updating files: 24% (58/241) Updating files: 25% (61/241) Updating files: 26% (63/241) Updating files: 27% (66/241) Updating files: 28% (68/241) Updating files: 29% (70/241) Updating files: 30% (73/241) Updating files: 31% (75/241) Updating files: 32% (78/241) Updating files: 33% (80/241) Updating files: 34% (82/241) Updating files: 35% (85/241) Updating files: 36% (87/241) Updating files: 37% (90/241) Updating files: 38% (92/241) Updating files: 39% (94/241) Updating files: 40% (97/241) Updating files: 41% (99/241) Updating files: 42% (102/241) Updating files: 43% (104/241) Updating files: 44% (107/241) Updating files: 45% (109/241) Updating files: 46% (111/241) Updating files: 47% (114/241) Updating files: 48% (116/241) Updating files: 49% (119/241) Updating files: 50% (121/241) Updating files: 51% (123/241) Updating files: 52% (126/241) Updating files: 53% (128/241) Updating files: 54% (131/241) Updating files: 55% (133/241) Updating files: 56% (135/241) Updating files: 57% (138/241) Updating files: 58% (140/241) Updating files: 59% (143/241) Updating files: 60% (145/241) Updating files: 61% (148/241) Updating files: 62% (150/241) Updating files: 63% (152/241) Updating files: 64% (155/241) Updating files: 65% (157/241) Updating files: 66% (160/241) Updating files: 67% (162/241) Updating files: 68% (164/241) Updating files: 69% (167/241) Updating files: 70% (169/241) Updating files: 71% (172/241) Updating files: 72% (174/241) Updating files: 73% (176/241) Updating files: 74% (179/241) Updating files: 75% (181/241) Updating files: 76% (184/241) Updating files: 77% (186/241) Updating files: 78% (188/241) Updating files: 78% (190/241) Updating files: 79% (191/241) Updating files: 80% (193/241) Updating files: 81% (196/241) Updating files: 82% (198/241) Updating files: 83% (201/241) Updating files: 84% (203/241) Updating files: 85% (205/241) Updating files: 86% (208/241) Updating files: 87% (210/241) Updating files: 88% (213/241) Updating files: 89% (215/241) Updating files: 90% (217/241) Updating files: 91% (220/241) Updating files: 92% (222/241) Updating files: 93% (225/241) Updating files: 94% (227/241) Updating files: 95% (229/241) Updating files: 96% (232/241) Updating files: 97% (234/241) Updating files: 98% (237/241) Updating files: 99% (239/241) Updating files: 100% (241/241) Updating files: 100% (241/241), done. 2024-03-09T09:13:44.038 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at b4cf0f2574b v0.68 2024-03-09T09:13:44.050 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.68 2024-03-09T09:13:44.254 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:13:44.255 INFO:tasks.workunit.client.0.smithi067.stdout:v0.69 2024-03-09T09:13:44.255 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 69 -eq 48 ']' 2024-03-09T09:13:44.255 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.69 2024-03-09T09:13:44.255 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.69 2024-03-09T09:13:44.255 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.69 2024-03-09T09:13:44.557 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 32% (78/241) Updating files: 33% (80/241) Updating files: 34% (82/241) Updating files: 35% (85/241) Updating files: 36% (87/241) Updating files: 37% (90/241) Updating files: 38% (92/241) Updating files: 39% (94/241) Updating files: 40% (97/241) Updating files: 41% (99/241) Updating files: 42% (102/241) Updating files: 43% (104/241) Updating files: 44% (107/241) Updating files: 45% (109/241) Updating files: 46% (111/241) Updating files: 47% (114/241) Updating files: 48% (116/241) Updating files: 49% (119/241) Updating files: 50% (121/241) Updating files: 51% (123/241) Updating files: 52% (126/241) Updating files: 53% (128/241) Updating files: 54% (131/241) Updating files: 55% (133/241) Updating files: 56% (135/241) Updating files: 57% (138/241) Updating files: 58% (140/241) Updating files: 59% (143/241) Updating files: 60% (145/241) Updating files: 61% (148/241) Updating files: 62% (150/241) Updating files: 63% (152/241) Updating files: 64% (155/241) Updating files: 65% (157/241) Updating files: 65% (158/241) Updating files: 66% (160/241) Updating files: 67% (162/241) Updating files: 68% (164/241) Updating files: 69% (167/241) Updating files: 70% (169/241) Updating files: 71% (172/241) Updating files: 72% (174/241) Updating files: 73% (176/241) Updating files: 74% (179/241) Updating files: 75% (181/241) Updating files: 76% (184/241) Updating files: 77% (186/241) Updating files: 78% (188/241) Updating files: 79% (191/241) Updating files: 80% (193/241) Updating files: 81% (196/241) Updating files: 82% (198/241) Updating files: 83% (201/241) Updating files: 84% (203/241) Updating files: 85% (205/241) Updating files: 86% (208/241) Updating files: 87% (210/241) Updating files: 88% (213/241) Updating files: 89% (215/241) Updating files: 90% (217/241) Updating files: 91% (220/241) Updating files: 92% (222/241) Updating files: 93% (225/241) Updating files: 94% (227/241) Updating files: 95% (229/241) Updating files: 96% (232/241) Updating files: 97% (234/241) Updating files: 98% (237/241) Updating files: 99% (239/241) Updating files: 100% (241/241) Updating files: 100% (241/241), done. 2024-03-09T09:13:44.611 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at b4cf0f2574b v0.68 2024-03-09T09:13:44.621 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.68 2024-03-09T09:13:44.764 INFO:tasks.workunit.client.1.smithi134.stdout:v0.69 2024-03-09T09:13:44.764 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:13:44.765 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 69 -eq 48 ']' 2024-03-09T09:13:44.765 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.69 2024-03-09T09:13:44.765 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.69 2024-03-09T09:13:44.765 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.69 2024-03-09T09:13:46.710 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:13:46.710 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:13:46.711 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:13:46.711 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:13:46.720 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:13:46.720 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:13:46.720 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:13:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.960+0000 7f471cc00640 1 Processor -- start 2024-03-09T09:13:46.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.960+0000 7f471cc00640 1 -- start start 2024-03-09T09:13:46.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.962+0000 7f471cc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f47180c7550 0x7f47180c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.962+0000 7f471cc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c1fe0 0x7f47180c6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.962+0000 7f471cc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47180c2640 con 0x7f47180c5d10 2024-03-09T09:13:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.962+0000 7f471cc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47180a6d10 con 0x7f47180c7550 2024-03-09T09:13:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.962+0000 7f471cc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47180c2920 con 0x7f47180c1fe0 2024-03-09T09:13:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.962+0000 7f4715e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f47180c5d10 0x7f47180c3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46804/0 (socket says 172.21.15.67:46804) 2024-03-09T09:13:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.962+0000 7f4715e00640 1 -- 172.21.15.67:0/3904516985 learned_addr learned my addr 172.21.15.67:0/3904516985 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:13:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.962+0000 7f4715400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f47180c7550 0x7f47180c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.963+0000 7f4716800640 1 --2- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c1fe0 0x7f47180c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.963+0000 7f4716800640 1 -- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c1fe0 msgr2=0x7f47180c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:13:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.963+0000 7f4716800640 1 -- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c1fe0 msgr2=0x7f47180c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.963+0000 7f4716800640 1 --2- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c1fe0 0x7f47180c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.963+0000 7f4716800640 1 --2- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c1fe0 0x7f47180c6630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.963+0000 7f4714a00640 1 -- 172.21.15.67:0/3904516985 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965549811 0 0) 0x7f47180c2640 con 0x7f47180c5d10 2024-03-09T09:13:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.963+0000 7f4714a00640 1 -- 172.21.15.67:0/3904516985 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46fc003520 con 0x7f47180c5d10 2024-03-09T09:13:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.963+0000 7f4715400640 1 -- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c1fe0 msgr2=0x7f47180c6630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:13:46.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.963+0000 7f4715400640 1 --2- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c1fe0 0x7f47180c6630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:46.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.963+0000 7f4715400640 1 -- 172.21.15.67:0/3904516985 >> v1:172.21.15.67:6789/0 conn(0x7f47180c5d10 legacy=0x7f47180c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:46.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.963+0000 7f4715400640 1 -- 172.21.15.67:0/3904516985 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47180a7300 con 0x7f47180c7550 2024-03-09T09:13:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.965+0000 7f4715400640 1 --2- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47180c7550 0x7f47180c60f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f47040097a0 tx=0x7f470402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff23aeb982bd45f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.965+0000 7f4714a00640 1 -- 172.21.15.67:0/3904516985 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704031780 con 0x7f47180c7550 2024-03-09T09:13:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.965+0000 7f4714a00640 1 -- 172.21.15.67:0/3904516985 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4704037070 con 0x7f47180c7550 2024-03-09T09:13:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.965+0000 7f4714a00640 1 -- 172.21.15.67:0/3904516985 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704031b40 con 0x7f47180c7550 2024-03-09T09:13:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.966+0000 7f471cc00640 1 -- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47180c7550 msgr2=0x7f47180c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.966+0000 7f471cc00640 1 --2- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47180c7550 0x7f47180c60f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f47040097a0 tx=0x7f470402b820 comp rx=0 tx=0).stop 2024-03-09T09:13:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.966+0000 7f471cc00640 1 -- 172.21.15.67:0/3904516985 shutdown_connections 2024-03-09T09:13:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.966+0000 7f471cc00640 1 --2- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c1fe0 0x7f47180c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.966+0000 7f471cc00640 1 --2- 172.21.15.67:0/3904516985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47180c7550 0x7f47180c60f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.966+0000 7f471cc00640 1 -- 172.21.15.67:0/3904516985 >> 172.21.15.67:0/3904516985 conn(0x7f47180a9950 msgr2=0x7f47180a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.966+0000 7f471cc00640 1 -- 172.21.15.67:0/3904516985 shutdown_connections 2024-03-09T09:13:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.967+0000 7f471cc00640 1 -- 172.21.15.67:0/3904516985 wait complete. 2024-03-09T09:13:46.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.969+0000 7f471cc00640 1 Processor -- start 2024-03-09T09:13:46.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.969+0000 7f471cc00640 1 -- start start 2024-03-09T09:13:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f471cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47180c1fe0 0x7f4718071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f471cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47180c5d10 0x7f47181ec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:46.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f471cc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c7550 0x7f471814e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f471cc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180a6f60 con 0x7f47180c5d10 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f471cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4718066e20 con 0x7f47180c1fe0 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f471cc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4718159d90 con 0x7f47180c7550 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f4715400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47180c5d10 0x7f47181ec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f4715400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47180c5d10 0x7f47181ec4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40600/0 (socket says 172.21.15.67:40600) 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f4715400640 1 -- 172.21.15.67:0/2480872278 learned_addr learned my addr 172.21.15.67:0/2480872278 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f4715e00640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47180c1fe0 0x7f4718071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f4716800640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c7550 0x7f471814e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f4716800640 1 -- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c7550 msgr2=0x7f471814e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f4716800640 1 -- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c7550 msgr2=0x7f471814e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f4716800640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c7550 0x7f471814e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.971+0000 7f4716800640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c7550 0x7f471814e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.972+0000 7f4715400640 1 -- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c7550 msgr2=0x7f471814e400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:13:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.972+0000 7f4715400640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c7550 0x7f471814e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.972+0000 7f4715400640 1 -- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47180c1fe0 msgr2=0x7f4718071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.972+0000 7f4715400640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47180c1fe0 0x7f4718071d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.972+0000 7f4715400640 1 -- 172.21.15.67:0/2480872278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471806e860 con 0x7f47180c5d10 2024-03-09T09:13:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.972+0000 7f4715400640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47180c5d10 0x7f47181ec4c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f470402be30 tx=0x7f47040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.973+0000 7f470e000640 1 -- 172.21.15.67:0/2480872278 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704032610 con 0x7f47180c5d10 2024-03-09T09:13:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.973+0000 7f470e000640 1 -- 172.21.15.67:0/2480872278 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47040327d0 con 0x7f47180c5d10 2024-03-09T09:13:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.973+0000 7f470e000640 1 -- 172.21.15.67:0/2480872278 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704032bc0 con 0x7f47180c5d10 2024-03-09T09:13:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.973+0000 7f471cc00640 1 -- 172.21.15.67:0/2480872278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47181eca00 con 0x7f47180c5d10 2024-03-09T09:13:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.973+0000 7f471cc00640 1 -- 172.21.15.67:0/2480872278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4718071250 con 0x7f47180c5d10 2024-03-09T09:13:46.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.975+0000 7f471cc00640 1 -- 172.21.15.67:0/2480872278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4718073be0 con 0x7f47180c5d10 2024-03-09T09:13:46.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.976+0000 7f470e000640 1 -- 172.21.15.67:0/2480872278 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f470404c060 con 0x7f47180c5d10 2024-03-09T09:13:46.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.977+0000 7f470e000640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46f806b4d0 0x7f46f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:46.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.977+0000 7f470e000640 1 -- 172.21.15.67:0/2480872278 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f47040abc00 con 0x7f47180c5d10 2024-03-09T09:13:46.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.978+0000 7f4715e00640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46f806b4d0 0x7f46f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:46.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.979+0000 7f4715e00640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46f806b4d0 0x7f46f806d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4708005880 tx=0x7f4708009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:46.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:46.986+0000 7f470e000640 1 -- 172.21.15.67:0/2480872278 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4718073be0 con 0x7f47180c5d10 2024-03-09T09:13:47.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.269+0000 7f471cc00640 1 -- 172.21.15.67:0/2480872278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4718075a00 con 0x7f47180c5d10 2024-03-09T09:13:47.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.270+0000 7f470e000640 1 -- 172.21.15.67:0/2480872278 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f47181eca00 con 0x7f47180c5d10 2024-03-09T09:13:47.275 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":156,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 -- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46f806b4d0 msgr2=0x7f46f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46f806b4d0 0x7f46f806d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4708005880 tx=0x7f4708009040 comp rx=0 tx=0).stop 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 -- 172.21.15.67:0/2480872278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47180c5d10 msgr2=0x7f47181ec4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47180c5d10 0x7f47181ec4c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f470402be30 tx=0x7f47040046c0 comp rx=0 tx=0).stop 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 -- 172.21.15.67:0/2480872278 shutdown_connections 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46f806b4d0 0x7f46f806d990 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47180c7550 0x7f471814e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47180c5d10 0x7f47181ec4c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 --2- 172.21.15.67:0/2480872278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47180c1fe0 0x7f4718071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 -- 172.21.15.67:0/2480872278 >> 172.21.15.67:0/2480872278 conn(0x7f47180a9950 msgr2=0x7f47180ac790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 -- 172.21.15.67:0/2480872278 shutdown_connections 2024-03-09T09:13:47.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:47.273+0000 7f471cc00640 1 -- 172.21.15.67:0/2480872278 wait complete. 2024-03-09T09:13:50.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:13:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.537+0000 7f0f2fa00640 1 Processor -- start 2024-03-09T09:13:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.538+0000 7f0f2fa00640 1 -- start start 2024-03-09T09:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.539+0000 7f0f2fa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0f280ac730 0x7f0f28071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.539+0000 7f0f2fa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f280c22a0 0x7f0f280acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.539+0000 7f0f2fa00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f281581a0 con 0x7f0f28157650 2024-03-09T09:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.539+0000 7f0f2fa00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f28070ee0 con 0x7f0f280c22a0 2024-03-09T09:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.539+0000 7f0f2fa00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f28154d00 con 0x7f0f280ac730 2024-03-09T09:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.539+0000 7f0f27e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f280c22a0 0x7f0f280acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.539+0000 7f0f27e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f280c22a0 0x7f0f280acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54148/0 (socket says 172.21.15.67:54148) 2024-03-09T09:13:50.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.539+0000 7f0f2da00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0f28157650 0x7f0f28157a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44626/0 (socket says 172.21.15.67:44626) 2024-03-09T09:13:50.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.539+0000 7f0f27e00640 1 -- 172.21.15.67:0/2181783431 learned_addr learned my addr 172.21.15.67:0/2181783431 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:50.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.539+0000 7f0f2d000640 1 --2- 172.21.15.67:0/2181783431 >> v2:172.21.15.134:3300/0 conn(0x7f0f280ac730 0x7f0f28071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:50.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.540+0000 7f0f27e00640 1 -- 172.21.15.67:0/2181783431 >> v2:172.21.15.134:3300/0 conn(0x7f0f280ac730 msgr2=0x7f0f28071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:50.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.540+0000 7f0f27e00640 1 --2- 172.21.15.67:0/2181783431 >> v2:172.21.15.134:3300/0 conn(0x7f0f280ac730 0x7f0f28071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:50.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.540+0000 7f0f27e00640 1 -- 172.21.15.67:0/2181783431 >> v1:172.21.15.67:6789/0 conn(0x7f0f28157650 legacy=0x7f0f28157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:50.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.540+0000 7f0f27e00640 1 -- 172.21.15.67:0/2181783431 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f28153d40 con 0x7f0f280c22a0 2024-03-09T09:13:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.541+0000 7f0f27e00640 1 --2- 172.21.15.67:0/2181783431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f280c22a0 0x7f0f280acb10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0f280ad380 tx=0x7f0f1402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfccebfd4e01c1b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.541+0000 7f0f27400640 1 -- 172.21.15.67:0/2181783431 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14034070 con 0x7f0f280c22a0 2024-03-09T09:13:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.541+0000 7f0f27400640 1 -- 172.21.15.67:0/2181783431 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f14030850 con 0x7f0f280c22a0 2024-03-09T09:13:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.541+0000 7f0f27400640 1 -- 172.21.15.67:0/2181783431 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14030c20 con 0x7f0f280c22a0 2024-03-09T09:13:50.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.542+0000 7f0f2fa00640 1 -- 172.21.15.67:0/2181783431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f280c22a0 msgr2=0x7f0f280acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:50.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.542+0000 7f0f2fa00640 1 --2- 172.21.15.67:0/2181783431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f280c22a0 0x7f0f280acb10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0f280ad380 tx=0x7f0f1402b990 comp rx=0 tx=0).stop 2024-03-09T09:13:50.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.542+0000 7f0f2fa00640 1 -- 172.21.15.67:0/2181783431 shutdown_connections 2024-03-09T09:13:50.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.542+0000 7f0f2fa00640 1 --2- 172.21.15.67:0/2181783431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f280c22a0 0x7f0f280acb10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:50.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.542+0000 7f0f2fa00640 1 --2- 172.21.15.67:0/2181783431 >> v2:172.21.15.134:3300/0 conn(0x7f0f280ac730 0x7f0f28071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:50.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.542+0000 7f0f2fa00640 1 -- 172.21.15.67:0/2181783431 >> 172.21.15.67:0/2181783431 conn(0x7f0f280c7370 msgr2=0x7f0f280c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:50.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.542+0000 7f0f2fa00640 1 -- 172.21.15.67:0/2181783431 shutdown_connections 2024-03-09T09:13:50.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.542+0000 7f0f2fa00640 1 -- 172.21.15.67:0/2181783431 wait complete. 2024-03-09T09:13:50.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.543+0000 7f0f2fa00640 1 Processor -- start 2024-03-09T09:13:50.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.544+0000 7f0f2fa00640 1 -- start start 2024-03-09T09:13:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.544+0000 7f0f2fa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f280ac730 0x7f0f28156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.544+0000 7f0f2fa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f280c22a0 0x7f0f280ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2fa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f28157650 0x7f0f28150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2fa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28070c00 con 0x7f0f28157650 2024-03-09T09:13:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2fa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28070920 con 0x7f0f280c22a0 2024-03-09T09:13:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2fa00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28151610 con 0x7f0f280ac730 2024-03-09T09:13:50.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f28157650 0x7f0f28150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f28157650 0x7f0f28150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54158/0 (socket says 172.21.15.67:54158) 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2da00640 1 -- 172.21.15.67:0/1799251464 learned_addr learned my addr 172.21.15.67:0/1799251464 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2d000640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f280ac730 0x7f0f28156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f27e00640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f280c22a0 0x7f0f280ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2d000640 1 -- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f280ac730 msgr2=0x7f0f28156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2d000640 1 -- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f280ac730 msgr2=0x7f0f28156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2d000640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f280ac730 0x7f0f28156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2d000640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f280ac730 0x7f0f28156aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2da00640 1 -- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f280ac730 msgr2=0x7f0f28156aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2da00640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f280ac730 0x7f0f28156aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2da00640 1 -- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f280c22a0 msgr2=0x7f0f280ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2da00640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f280c22a0 0x7f0f280ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.545+0000 7f0f2da00640 1 -- 172.21.15.67:0/1799251464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f281e12d0 con 0x7f0f28157650 2024-03-09T09:13:50.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.546+0000 7f0f2da00640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f28157650 0x7f0f28150590 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000c760 tx=0x7f0f2000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.546+0000 7f0f24c00640 1 -- 172.21.15.67:0/1799251464 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20017070 con 0x7f0f28157650 2024-03-09T09:13:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.546+0000 7f0f24c00640 1 -- 172.21.15.67:0/1799251464 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2000a640 con 0x7f0f28157650 2024-03-09T09:13:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.546+0000 7f0f24c00640 1 -- 172.21.15.67:0/1799251464 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2000a9f0 con 0x7f0f28157650 2024-03-09T09:13:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.546+0000 7f0f2fa00640 1 -- 172.21.15.67:0/1799251464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f28156fe0 con 0x7f0f28157650 2024-03-09T09:13:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.546+0000 7f0f2fa00640 1 -- 172.21.15.67:0/1799251464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f281eef60 con 0x7f0f28157650 2024-03-09T09:13:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.547+0000 7f0f2fa00640 1 -- 172.21.15.67:0/1799251464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f28075a00 con 0x7f0f28157650 2024-03-09T09:13:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.548+0000 7f0f24c00640 1 -- 172.21.15.67:0/1799251464 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0f2001a070 con 0x7f0f28157650 2024-03-09T09:13:50.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.549+0000 7f0f24c00640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f0806b4d0 0x7f0f0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:50.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.549+0000 7f0f24c00640 1 -- 172.21.15.67:0/1799251464 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0f20089a30 con 0x7f0f28157650 2024-03-09T09:13:50.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.550+0000 7f0f2d000640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f0806b4d0 0x7f0f0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:50.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.551+0000 7f0f2d000640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f0806b4d0 0x7f0f0806d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009500 tx=0x7f0f1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:50.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.554+0000 7f0f24c00640 1 -- 172.21.15.67:0/1799251464 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0f281e12d0 con 0x7f0f28157650 2024-03-09T09:13:50.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.830+0000 7f0f2fa00640 1 -- 172.21.15.67:0/1799251464 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0f280abf10 con 0x7f0f28157650 2024-03-09T09:13:50.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.830+0000 7f0f24c00640 1 -- 172.21.15.67:0/1799251464 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f0f2005ce20 con 0x7f0f28157650 2024-03-09T09:13:50.835 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":156,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:13:50.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 -- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f0806b4d0 msgr2=0x7f0f0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:50.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f0806b4d0 0x7f0f0806d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009500 tx=0x7f0f1c008040 comp rx=0 tx=0).stop 2024-03-09T09:13:50.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 -- 172.21.15.67:0/1799251464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f28157650 msgr2=0x7f0f28150590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:50.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f28157650 0x7f0f28150590 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000c760 tx=0x7f0f2000cc20 comp rx=0 tx=0).stop 2024-03-09T09:13:50.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 -- 172.21.15.67:0/1799251464 shutdown_connections 2024-03-09T09:13:50.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f0806b4d0 0x7f0f0806d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:50.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f28157650 0x7f0f28150590 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:50.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f280c22a0 0x7f0f280ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:50.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 --2- 172.21.15.67:0/1799251464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f280ac730 0x7f0f28156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:50.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 -- 172.21.15.67:0/1799251464 >> 172.21.15.67:0/1799251464 conn(0x7f0f280c7370 msgr2=0x7f0f28157a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:50.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 -- 172.21.15.67:0/1799251464 shutdown_connections 2024-03-09T09:13:50.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:50.833+0000 7f0f2fa00640 1 -- 172.21.15.67:0/1799251464 wait complete. 2024-03-09T09:13:53.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:13:54.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.072+0000 7f5009a00640 1 Processor -- start 2024-03-09T09:13:54.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.072+0000 7f5009a00640 1 -- start start 2024-03-09T09:13:54.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.074+0000 7f5009a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:54.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.074+0000 7f5009a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5004153630 con 0x7f50040c28a0 2024-03-09T09:13:54.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.074+0000 7f5009a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50040abd00 con 0x7f5004153250 2024-03-09T09:13:54.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.074+0000 7f5009a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50040aa5f0 con 0x7f5004073010 2024-03-09T09:13:54.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.074+0000 7f5009000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f50040c28a0 0x7f50040c2cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44632/0 (socket says 172.21.15.67:44632) 2024-03-09T09:13:54.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.074+0000 7f5009000640 1 -- 172.21.15.67:0/2716457014 learned_addr learned my addr 172.21.15.67:0/2716457014 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:13:54.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.074+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:54.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.075+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 msgr2=0x7f5004070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:13:54.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.075+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 msgr2=0x7f5004070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:54.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.075+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:54.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.075+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:54.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.276+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:54.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.277+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 msgr2=0x7f5004070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:13:54.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.277+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 msgr2=0x7f5004070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:54.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.277+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:54.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.277+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:13:54.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.679+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:54.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.679+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 msgr2=0x7f5004070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:13:54.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.679+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 msgr2=0x7f5004070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:54.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.679+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:54.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:54.679+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:13:55.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:55.480+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:55.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:55.481+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 msgr2=0x7f5004070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:13:55.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:55.481+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 msgr2=0x7f5004070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:55.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:55.481+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:55.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:55.481+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:13:57.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.074+0000 7f5001600640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 msgr2=0x7f5004070520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:13:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.074+0000 7f5001600640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.134:3300/0 conn(0x7f5004073010 0x7f5004070520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.074+0000 7f5001600640 1 -- 172.21.15.67:0/2716457014 >> v1:172.21.15.134:6789/0 conn(0x7f5004153250 legacy=0x7f500406fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.074+0000 7f5001600640 1 -- 172.21.15.67:0/2716457014 >> v1:172.21.15.67:6789/0 conn(0x7f50040c28a0 legacy=0x7f50040c2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.074+0000 7f5001600640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 0x7f4fe4005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.074+0000 7f5001600640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 0x7f4fe4008620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.074+0000 7f5001600640 1 -- 172.21.15.67:0/2716457014 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe4008d10 con 0x7f4fe40036c0 2024-03-09T09:13:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.074+0000 7f5001600640 1 -- 172.21.15.67:0/2716457014 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4008ec0 con 0x7f4fe4006180 2024-03-09T09:13:57.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.075+0000 7f5003e00640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 0x7f4fe4005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:57.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.075+0000 7f5003e00640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 msgr2=0x7f4fe4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:13:57.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.075+0000 7f5003e00640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 msgr2=0x7f4fe4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:57.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.075+0000 7f5003e00640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 0x7f4fe4005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:57.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.075+0000 7f5003e00640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 0x7f4fe4005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:57.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.075+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 0x7f4fe4008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:57.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.076+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 msgr2=0x7f4fe4008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:13:57.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.076+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 msgr2=0x7f4fe4008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:57.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.076+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 0x7f4fe4008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:57.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.076+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 0x7f4fe4008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:57.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.276+0000 7f5003e00640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 0x7f4fe4005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:57.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.277+0000 7f5003e00640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 msgr2=0x7f4fe4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:13:57.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.277+0000 7f5003e00640 1 -- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 msgr2=0x7f4fe4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:57.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.277+0000 7f5003e00640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 0x7f4fe4005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:57.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.277+0000 7f5003e00640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 0x7f4fe4005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:13:57.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.277+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 0x7f4fe4008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:57.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.278+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 msgr2=0x7f4fe4008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:13:57.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.278+0000 7f5003400640 1 -- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 msgr2=0x7f4fe4008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:13:57.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.278+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 0x7f4fe4008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:57.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.278+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 0x7f4fe4008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:13:57.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.678+0000 7f5003e00640 1 --2- 172.21.15.67:0/2716457014 >> v2:172.21.15.67:3300/0 conn(0x7f4fe40036c0 0x7f4fe4005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:57.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.679+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 0x7f4fe4008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:57.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.680+0000 7f5003e00640 1 -- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 msgr2=0x7f4fe4008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:13:57.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.680+0000 7f5003e00640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 0x7f4fe4008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:57.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.680+0000 7f5003e00640 1 -- 172.21.15.67:0/2716457014 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50041567c0 con 0x7f4fe40036c0 2024-03-09T09:13:57.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.680+0000 7f5003400640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fe4006180 0x7f4fe4008620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:13:57.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.684+0000 7f5003e00640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe40036c0 0x7f4fe4005b40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4007c30 tx=0x7f4ff402baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797761da3396dd80 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:57.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.685+0000 7f5002a00640 1 -- 172.21.15.67:0/2716457014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4034070 con 0x7f4fe40036c0 2024-03-09T09:13:57.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.685+0000 7f5002a00640 1 -- 172.21.15.67:0/2716457014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff402d710 con 0x7f4fe40036c0 2024-03-09T09:13:57.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.685+0000 7f5002a00640 1 -- 172.21.15.67:0/2716457014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff402db00 con 0x7f4fe40036c0 2024-03-09T09:13:57.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.687+0000 7f5009a00640 1 -- 172.21.15.67:0/2716457014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe40036c0 msgr2=0x7f4fe4005b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:57.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.687+0000 7f5009a00640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe40036c0 0x7f4fe4005b40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4007c30 tx=0x7f4ff402baa0 comp rx=0 tx=0).stop 2024-03-09T09:13:57.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.687+0000 7f5009000640 1 -- 172.21.15.67:0/2716457014 reap_dead start 2024-03-09T09:13:57.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.687+0000 7f5009a00640 1 -- 172.21.15.67:0/2716457014 shutdown_connections 2024-03-09T09:13:57.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.687+0000 7f5009a00640 1 --2- 172.21.15.67:0/2716457014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe40036c0 0x7f4fe4005b40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:57.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.687+0000 7f5009a00640 1 -- 172.21.15.67:0/2716457014 >> 172.21.15.67:0/2716457014 conn(0x7f5004066d90 msgr2=0x7f5004067170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:57.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.688+0000 7f5009a00640 1 -- 172.21.15.67:0/2716457014 shutdown_connections 2024-03-09T09:13:57.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.688+0000 7f5009a00640 1 -- 172.21.15.67:0/2716457014 wait complete. 2024-03-09T09:13:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.691+0000 7f5009a00640 1 Processor -- start 2024-03-09T09:13:57.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.691+0000 7f5009a00640 1 -- start start 2024-03-09T09:13:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.692+0000 7f5009a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe40036c0 0x7f500414e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.692+0000 7f5009a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f500414a240 0x7f5004149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.692+0000 7f5009a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50041496c0 0x7f50041ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:57.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.692+0000 7f5009a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500406f8f0 con 0x7f4fe40036c0 2024-03-09T09:13:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.692+0000 7f5009a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040abfe0 con 0x7f500414a240 2024-03-09T09:13:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.692+0000 7f5009a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5004070a60 con 0x7f50041496c0 2024-03-09T09:13:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.693+0000 7f5003e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe40036c0 0x7f500414e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.693+0000 7f5003e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe40036c0 0x7f500414e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54188/0 (socket says 172.21.15.67:54188) 2024-03-09T09:13:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.693+0000 7f5009000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50041496c0 0x7f50041ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.693+0000 7f5003e00640 1 -- 172.21.15.67:0/55001188 learned_addr learned my addr 172.21.15.67:0/55001188 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:13:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.693+0000 7f5003400640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f500414a240 0x7f5004149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.693+0000 7f5009000640 1 -- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50041496c0 msgr2=0x7f50041ebaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:13:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.693+0000 7f5009000640 1 -- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50041496c0 msgr2=0x7f50041ebaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:13:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.693+0000 7f5009000640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50041496c0 0x7f50041ebaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:13:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.693+0000 7f5009000640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50041496c0 0x7f50041ebaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:13:57.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.694+0000 7f5003e00640 1 -- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50041496c0 msgr2=0x7f50041ebaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:13:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.694+0000 7f5003e00640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50041496c0 0x7f50041ebaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.694+0000 7f5003e00640 1 -- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f500414a240 msgr2=0x7f5004149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.694+0000 7f5003e00640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f500414a240 0x7f5004149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.694+0000 7f5003e00640 1 -- 172.21.15.67:0/55001188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50040af210 con 0x7f4fe40036c0 2024-03-09T09:13:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.694+0000 7f5003e00640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe40036c0 0x7f500414e260 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4ff402c0b0 tx=0x7f4ff4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.695+0000 7f4ffbe00640 1 -- 172.21.15.67:0/55001188 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4047070 con 0x7f4fe40036c0 2024-03-09T09:13:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.695+0000 7f4ffbe00640 1 -- 172.21.15.67:0/55001188 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff4007bb0 con 0x7f4fe40036c0 2024-03-09T09:13:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.695+0000 7f4ffbe00640 1 -- 172.21.15.67:0/55001188 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4034070 con 0x7f4fe40036c0 2024-03-09T09:13:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.695+0000 7f5009a00640 1 -- 172.21.15.67:0/55001188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50041e11e0 con 0x7f4fe40036c0 2024-03-09T09:13:57.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.695+0000 7f5009a00640 1 -- 172.21.15.67:0/55001188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500414e7a0 con 0x7f4fe40036c0 2024-03-09T09:13:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.697+0000 7f5009a00640 1 -- 172.21.15.67:0/55001188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500404f3c0 con 0x7f4fe40036c0 2024-03-09T09:13:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.699+0000 7f4ffbe00640 1 -- 172.21.15.67:0/55001188 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4ff4037070 con 0x7f4fe40036c0 2024-03-09T09:13:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.701+0000 7f4ffbe00640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4fe406c4a0 0x7f4fe406e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:13:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.701+0000 7f4ffbe00640 1 -- 172.21.15.67:0/55001188 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4ff40ab8f0 con 0x7f4fe40036c0 2024-03-09T09:13:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.701+0000 7f5003400640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4fe406c4a0 0x7f4fe406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:13:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.702+0000 7f5003400640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4fe406c4a0 0x7f4fe406e960 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f500414cb70 tx=0x7f4ff0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:13:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.706+0000 7f4ffbe00640 1 -- 172.21.15.67:0/55001188 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4ff403f070 con 0x7f4fe40036c0 2024-03-09T09:13:57.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.981+0000 7f5009a00640 1 -- 172.21.15.67:0/55001188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5004149ac0 con 0x7f4fe40036c0 2024-03-09T09:13:57.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.982+0000 7f4ffbe00640 1 -- 172.21.15.67:0/55001188 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f4ff4079de0 con 0x7f4fe40036c0 2024-03-09T09:13:57.987 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":160,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:13:57.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 -- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4fe406c4a0 msgr2=0x7f4fe406e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:57.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4fe406c4a0 0x7f4fe406e960 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f500414cb70 tx=0x7f4ff0008040 comp rx=0 tx=0).stop 2024-03-09T09:13:57.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 -- 172.21.15.67:0/55001188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe40036c0 msgr2=0x7f500414e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:13:57.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe40036c0 0x7f500414e260 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4ff402c0b0 tx=0x7f4ff4001120 comp rx=0 tx=0).stop 2024-03-09T09:13:57.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 -- 172.21.15.67:0/55001188 shutdown_connections 2024-03-09T09:13:57.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4fe406c4a0 0x7f4fe406e960 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:57.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f50041496c0 0x7f50041ebaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:57.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f500414a240 0x7f5004149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:57.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 --2- 172.21.15.67:0/55001188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fe40036c0 0x7f500414e260 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:13:57.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 -- 172.21.15.67:0/55001188 >> 172.21.15.67:0/55001188 conn(0x7f5004066d90 msgr2=0x7f50040c2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:13:57.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 -- 172.21.15.67:0/55001188 shutdown_connections 2024-03-09T09:13:57.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:13:57.985+0000 7f5009a00640 1 -- 172.21.15.67:0/55001188 wait complete. 2024-03-09T09:13:58.995 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 14% (22/155) Updating files: 15% (24/155) Updating files: 16% (25/155) Updating files: 17% (27/155) Updating files: 18% (28/155) Updating files: 19% (30/155) Updating files: 20% (31/155) Updating files: 21% (33/155) Updating files: 22% (35/155) Updating files: 23% (36/155) Updating files: 24% (38/155) Updating files: 25% (39/155) Updating files: 26% (41/155) Updating files: 27% (42/155) Updating files: 28% (44/155) Updating files: 29% (45/155) Updating files: 30% (47/155) Updating files: 31% (49/155) Updating files: 32% (50/155) Updating files: 33% (52/155) Updating files: 34% (53/155) Updating files: 35% (55/155) Updating files: 36% (56/155) Updating files: 37% (58/155) Updating files: 38% (59/155) Updating files: 39% (61/155) Updating files: 40% (62/155) Updating files: 41% (64/155) Updating files: 42% (66/155) Updating files: 43% (67/155) Updating files: 44% (69/155) Updating files: 45% (70/155) Updating files: 46% (72/155) Updating files: 47% (73/155) Updating files: 48% (75/155) Updating files: 49% (76/155) Updating files: 50% (78/155) Updating files: 51% (80/155) Updating files: 52% (81/155) Updating files: 53% (83/155) Updating files: 54% (84/155) Updating files: 55% (86/155) Updating files: 56% (87/155) Updating files: 57% (89/155) Updating files: 58% (90/155) Updating files: 59% (92/155) Updating files: 60% (93/155) Updating files: 61% (95/155) Updating files: 62% (97/155) Updating files: 63% (98/155) Updating files: 64% (100/155) Updating files: 65% (101/155) Updating files: 66% (103/155) Updating files: 67% (104/155) Updating files: 68% (106/155) Updating files: 69% (107/155) Updating files: 70% (109/155) Updating files: 71% (111/155) Updating files: 72% (112/155) Updating files: 73% (114/155) Updating files: 74% (115/155) Updating files: 75% (117/155) Updating files: 76% (118/155) Updating files: 77% (120/155) Updating files: 78% (121/155) Updating files: 79% (123/155) Updating files: 80% (124/155) Updating files: 81% (126/155) Updating files: 82% (128/155) Updating files: 83% (129/155) Updating files: 84% (131/155) Updating files: 85% (132/155) Updating files: 85% (133/155) Updating files: 86% (134/155) Updating files: 87% (135/155) Updating files: 88% (137/155) Updating files: 89% (138/155) Updating files: 90% (140/155) Updating files: 91% (142/155) Updating files: 92% (143/155) Updating files: 93% (145/155) Updating files: 94% (146/155) Updating files: 95% (148/155) Updating files: 96% (149/155) Updating files: 97% (151/155) Updating files: 98% (152/155) Updating files: 99% (154/155) Updating files: 100% (155/155) Updating files: 100% (155/155), done. 2024-03-09T09:13:59.030 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 6ca6f2f9f75 v0.69 2024-03-09T09:13:59.038 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.69 2024-03-09T09:13:59.442 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:13:59.443 INFO:tasks.workunit.client.1.smithi134.stdout:v0.70 2024-03-09T09:13:59.444 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 70 -eq 48 ']' 2024-03-09T09:13:59.444 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.70 2024-03-09T09:13:59.444 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.70 2024-03-09T09:13:59.444 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.70 2024-03-09T09:14:00.392 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 46% (72/155) Updating files: 47% (73/155) Updating files: 48% (75/155) Updating files: 49% (76/155) Updating files: 50% (78/155) Updating files: 51% (80/155) Updating files: 52% (81/155) Updating files: 53% (83/155) Updating files: 54% (84/155) Updating files: 55% (86/155) Updating files: 56% (87/155) Updating files: 57% (89/155) Updating files: 58% (90/155) Updating files: 59% (92/155) Updating files: 60% (93/155) Updating files: 61% (95/155) Updating files: 62% (97/155) Updating files: 63% (98/155) Updating files: 64% (100/155) Updating files: 65% (101/155) Updating files: 66% (103/155) Updating files: 67% (104/155) Updating files: 68% (106/155) Updating files: 69% (107/155) Updating files: 70% (109/155) Updating files: 71% (111/155) Updating files: 72% (112/155) Updating files: 73% (114/155) Updating files: 74% (115/155) Updating files: 75% (117/155) Updating files: 76% (118/155) Updating files: 76% (119/155) Updating files: 77% (120/155) Updating files: 78% (121/155) Updating files: 79% (123/155) Updating files: 80% (124/155) Updating files: 81% (126/155) Updating files: 82% (128/155) Updating files: 83% (129/155) Updating files: 84% (131/155) Updating files: 85% (132/155) Updating files: 86% (134/155) Updating files: 87% (135/155) Updating files: 88% (137/155) Updating files: 89% (138/155) Updating files: 90% (140/155) Updating files: 91% (142/155) Updating files: 92% (143/155) Updating files: 93% (145/155) Updating files: 94% (146/155) Updating files: 95% (148/155) Updating files: 96% (149/155) Updating files: 97% (151/155) Updating files: 98% (152/155) Updating files: 99% (154/155) Updating files: 100% (155/155) Updating files: 100% (155/155), done. 2024-03-09T09:14:00.447 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 6ca6f2f9f75 v0.69 2024-03-09T09:14:00.460 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.69 2024-03-09T09:14:00.850 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at e3bb0656d92 v0.70 2024-03-09T09:14:00.851 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.70 2024-03-09T09:14:00.933 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:14:00.933 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 71 -eq 48 ']' 2024-03-09T09:14:00.933 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.71 2024-03-09T09:14:00.934 INFO:tasks.workunit.client.1.smithi134.stdout:v0.71 2024-03-09T09:14:00.934 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.71 2024-03-09T09:14:00.934 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.71 2024-03-09T09:14:01.010 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:14:01.165 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:14:01.166 INFO:tasks.workunit.client.0.smithi067.stdout:v0.70 2024-03-09T09:14:01.166 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 70 -eq 48 ']' 2024-03-09T09:14:01.166 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.70 2024-03-09T09:14:01.166 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.70 2024-03-09T09:14:01.166 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.70 2024-03-09T09:14:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.207+0000 7f15cf200640 1 Processor -- start 2024-03-09T09:14:01.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.208+0000 7f15cf200640 1 -- start start 2024-03-09T09:14:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cf200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d0153300 0x7f15d00b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cf200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f15d00c5e40 0x7f15d00b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cf200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f15d00cadb0 0x7f15d0156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cf200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d0157000 con 0x7f15d00cadb0 2024-03-09T09:14:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cf200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d00ae430 con 0x7f15d00c5e40 2024-03-09T09:14:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cf200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00ae6f0 con 0x7f15d0153300 2024-03-09T09:14:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15ce800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f15d00cadb0 0x7f15d0156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cde00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d0153300 0x7f15d00b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cde00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d0153300 0x7f15d00b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:45794/0 (socket says 172.21.15.67:45794) 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cde00640 1 -- 172.21.15.67:0/3602295159 learned_addr learned my addr 172.21.15.67:0/3602295159 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cd400640 1 --2- 172.21.15.67:0/3602295159 >> v2:172.21.15.134:3300/0 conn(0x7f15d00c5e40 0x7f15d00b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cde00640 1 -- 172.21.15.67:0/3602295159 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d0153300 msgr2=0x7f15d00b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cde00640 1 -- 172.21.15.67:0/3602295159 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d0153300 msgr2=0x7f15d00b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cde00640 1 --2- 172.21.15.67:0/3602295159 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d0153300 0x7f15d00b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.210+0000 7f15cde00640 1 --2- 172.21.15.67:0/3602295159 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d0153300 0x7f15d00b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.211+0000 7f15ce800640 1 -- 172.21.15.67:0/3602295159 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d0153300 msgr2=0x7f15d00b0000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.211+0000 7f15ce800640 1 --2- 172.21.15.67:0/3602295159 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d0153300 0x7f15d00b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.211+0000 7f15ce800640 1 -- 172.21.15.67:0/3602295159 >> v2:172.21.15.134:3300/0 conn(0x7f15d00c5e40 msgr2=0x7f15d00b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.211+0000 7f15ce800640 1 --2- 172.21.15.67:0/3602295159 >> v2:172.21.15.134:3300/0 conn(0x7f15d00c5e40 0x7f15d00b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.211+0000 7f15ce800640 1 -- 172.21.15.67:0/3602295159 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0155b10 con 0x7f15d00cadb0 2024-03-09T09:14:01.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.212+0000 7f15ce800640 1 --2- 172.21.15.67:0/3602295159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d00cadb0 0x7f15d0156010 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f15d00adf30 tx=0x7f15c802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8ad3e6b490e624 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.212+0000 7f15cca00640 1 -- 172.21.15.67:0/3602295159 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8039070 con 0x7f15d00cadb0 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.212+0000 7f15cca00640 1 -- 172.21.15.67:0/3602295159 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c802cbd0 con 0x7f15d00cadb0 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.212+0000 7f15cca00640 1 -- 172.21.15.67:0/3602295159 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8034440 con 0x7f15d00cadb0 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.213+0000 7f15cf200640 1 -- 172.21.15.67:0/3602295159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d00cadb0 msgr2=0x7f15d0156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.213+0000 7f15cf200640 1 --2- 172.21.15.67:0/3602295159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d00cadb0 0x7f15d0156010 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f15d00adf30 tx=0x7f15c802be20 comp rx=0 tx=0).stop 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.213+0000 7f15cf200640 1 -- 172.21.15.67:0/3602295159 shutdown_connections 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.213+0000 7f15cf200640 1 --2- 172.21.15.67:0/3602295159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d00cadb0 0x7f15d0156010 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.213+0000 7f15cf200640 1 --2- 172.21.15.67:0/3602295159 >> v2:172.21.15.134:3300/0 conn(0x7f15d00c5e40 0x7f15d00b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.213+0000 7f15cf200640 1 --2- 172.21.15.67:0/3602295159 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d0153300 0x7f15d00b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.213+0000 7f15cf200640 1 -- 172.21.15.67:0/3602295159 >> 172.21.15.67:0/3602295159 conn(0x7f15d00732d0 msgr2=0x7f15d00ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.213+0000 7f15cf200640 1 -- 172.21.15.67:0/3602295159 shutdown_connections 2024-03-09T09:14:01.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.213+0000 7f15cf200640 1 -- 172.21.15.67:0/3602295159 wait complete. 2024-03-09T09:14:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.216+0000 7f15cf200640 1 Processor -- start 2024-03-09T09:14:01.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.216+0000 7f15cf200640 1 -- start start 2024-03-09T09:14:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.217+0000 7f15cf200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d00c5e40 0x7f15d00ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.217+0000 7f15cf200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d00cadb0 0x7f15d00c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.217+0000 7f15cf200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d0153300 0x7f15d0147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.217+0000 7f15cf200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00c59e0 con 0x7f15d00cadb0 2024-03-09T09:14:01.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.217+0000 7f15cf200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d0159e90 con 0x7f15d0153300 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.217+0000 7f15cf200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d0156550 con 0x7f15d00c5e40 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d00cadb0 0x7f15d00c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d00cadb0 0x7f15d00c6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46114/0 (socket says 172.21.15.67:46114) 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cd400640 1 -- 172.21.15.67:0/2604813644 learned_addr learned my addr 172.21.15.67:0/2604813644 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cde00640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d00c5e40 0x7f15d00ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15ce800640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d0153300 0x7f15d0147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cde00640 1 -- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d00c5e40 msgr2=0x7f15d00ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cde00640 1 -- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d00c5e40 msgr2=0x7f15d00ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cd400640 1 -- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d00c5e40 msgr2=0x7f15d00ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cde00640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d00c5e40 0x7f15d00ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cde00640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d00c5e40 0x7f15d00ad2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cd400640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d00c5e40 0x7f15d00ad2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cd400640 1 -- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d0153300 msgr2=0x7f15d0147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cd400640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d0153300 0x7f15d0147cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.218+0000 7f15cd400640 1 -- 172.21.15.67:0/2604813644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d00c7350 con 0x7f15d00cadb0 2024-03-09T09:14:01.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.219+0000 7f15cd400640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d00cadb0 0x7f15d00c6740 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f15c000e9b0 tx=0x7f15c000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:01.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.219+0000 7f15c6000640 1 -- 172.21.15.67:0/2604813644 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c0017070 con 0x7f15d00cadb0 2024-03-09T09:14:01.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.219+0000 7f15c6000640 1 -- 172.21.15.67:0/2604813644 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c000c9a0 con 0x7f15d00cadb0 2024-03-09T09:14:01.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.219+0000 7f15c6000640 1 -- 172.21.15.67:0/2604813644 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c000cd90 con 0x7f15d00cadb0 2024-03-09T09:14:01.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.219+0000 7f15cf200640 1 -- 172.21.15.67:0/2604813644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d00aba00 con 0x7f15d00cadb0 2024-03-09T09:14:01.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.219+0000 7f15cf200640 1 -- 172.21.15.67:0/2604813644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d01eca20 con 0x7f15d00cadb0 2024-03-09T09:14:01.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.220+0000 7f15cf200640 1 -- 172.21.15.67:0/2604813644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d004f3c0 con 0x7f15d00cadb0 2024-03-09T09:14:01.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.221+0000 7f15c6000640 1 -- 172.21.15.67:0/2604813644 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f15c001a070 con 0x7f15d00cadb0 2024-03-09T09:14:01.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.222+0000 7f15c6000640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15ac06b500 0x7f15ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.222+0000 7f15c6000640 1 -- 172.21.15.67:0/2604813644 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f15c008a570 con 0x7f15d00cadb0 2024-03-09T09:14:01.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.223+0000 7f15cde00640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15ac06b500 0x7f15ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.223+0000 7f15cde00640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15ac06b500 0x7f15ac06d9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009100 tx=0x7f15b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:01.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.227+0000 7f15c6000640 1 -- 172.21.15.67:0/2604813644 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f15c0058910 con 0x7f15d00cadb0 2024-03-09T09:14:01.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.535+0000 7f15cf200640 1 -- 172.21.15.67:0/2604813644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f15d00cab90 con 0x7f15d00cadb0 2024-03-09T09:14:01.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.536+0000 7f15c6000640 1 -- 172.21.15.67:0/2604813644 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f15c005c9b0 con 0x7f15d00cadb0 2024-03-09T09:14:01.541 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":162,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:14:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 -- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15ac06b500 msgr2=0x7f15ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15ac06b500 0x7f15ac06d9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009100 tx=0x7f15b8008040 comp rx=0 tx=0).stop 2024-03-09T09:14:01.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 -- 172.21.15.67:0/2604813644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d00cadb0 msgr2=0x7f15d00c6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d00cadb0 0x7f15d00c6740 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f15c000e9b0 tx=0x7f15c000ee70 comp rx=0 tx=0).stop 2024-03-09T09:14:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 -- 172.21.15.67:0/2604813644 shutdown_connections 2024-03-09T09:14:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15ac06b500 0x7f15ac06d9c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15d0153300 0x7f15d0147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15d00cadb0 0x7f15d00c6740 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 --2- 172.21.15.67:0/2604813644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d00c5e40 0x7f15d00ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 -- 172.21.15.67:0/2604813644 >> 172.21.15.67:0/2604813644 conn(0x7f15d00732d0 msgr2=0x7f15d01568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 -- 172.21.15.67:0/2604813644 shutdown_connections 2024-03-09T09:14:01.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.538+0000 7f15cf200640 1 -- 172.21.15.67:0/2604813644 wait complete. 2024-03-09T09:14:01.562 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:14:01.562 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:14:01.562 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:14:01.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.761+0000 7f174b200640 1 Processor -- start 2024-03-09T09:14:01.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.762+0000 7f174b200640 1 -- start start 2024-03-09T09:14:01.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.764+0000 7f174b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f174c153350 0x7f174c070030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.764+0000 7f174b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174c0c5470 con 0x7f174c0c3590 2024-03-09T09:14:01.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.764+0000 7f174b200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174c070710 con 0x7f174c0c9400 2024-03-09T09:14:01.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.764+0000 7f174b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174c0709f0 con 0x7f174c153350 2024-03-09T09:14:01.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.764+0000 7f1749e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f174c153350 0x7f174c070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.764+0000 7f1749e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f174c153350 0x7f174c070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46122/0 (socket says 172.21.15.67:46122) 2024-03-09T09:14:01.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.764+0000 7f174a800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f174c0c3590 0x7f174c160480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35650/0 (socket says 172.21.15.67:35650) 2024-03-09T09:14:01.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.764+0000 7f1749e00640 1 -- 172.21.15.67:0/2013450514 learned_addr learned my addr 172.21.15.67:0/2013450514 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:01.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.765+0000 7f1749400640 1 -- 172.21.15.67:0/2013450514 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404052213 0 0) 0x7f174c0c5470 con 0x7f174c0c3590 2024-03-09T09:14:01.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.765+0000 7f1749400640 1 -- 172.21.15.67:0/2013450514 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1730003520 con 0x7f174c0c3590 2024-03-09T09:14:01.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.765+0000 7f1749400640 1 -- 172.21.15.67:0/2013450514 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156725790 0 0) 0x7f174c070710 con 0x7f174c0c9400 2024-03-09T09:14:01.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.765+0000 7f1749400640 1 -- 172.21.15.67:0/2013450514 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f174c0c5470 con 0x7f174c0c9400 2024-03-09T09:14:01.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.765+0000 7f1749e00640 1 -- 172.21.15.67:0/2013450514 >> v1:172.21.15.134:6789/0 conn(0x7f174c0c9400 legacy=0x7f174c0c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.765+0000 7f1749e00640 1 -- 172.21.15.67:0/2013450514 >> v1:172.21.15.67:6789/0 conn(0x7f174c0c3590 legacy=0x7f174c160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.765+0000 7f1749e00640 1 -- 172.21.15.67:0/2013450514 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c0c4990 con 0x7f174c153350 2024-03-09T09:14:01.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.766+0000 7f1749e00640 1 --2- 172.21.15.67:0/2013450514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c153350 0x7f174c070030 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f173c009870 tx=0x7f173c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ac1a2df327e5ea server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:01.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.767+0000 7f1749400640 1 -- 172.21.15.67:0/2013450514 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c02f840 con 0x7f174c153350 2024-03-09T09:14:01.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.767+0000 7f1749400640 1 -- 172.21.15.67:0/2013450514 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c02f9e0 con 0x7f174c153350 2024-03-09T09:14:01.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.767+0000 7f1749400640 1 -- 172.21.15.67:0/2013450514 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c02fdb0 con 0x7f174c153350 2024-03-09T09:14:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.767+0000 7f174b200640 1 -- 172.21.15.67:0/2013450514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c153350 msgr2=0x7f174c070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.767+0000 7f174b200640 1 --2- 172.21.15.67:0/2013450514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c153350 0x7f174c070030 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f173c009870 tx=0x7f173c02ba20 comp rx=0 tx=0).stop 2024-03-09T09:14:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.768+0000 7f174b200640 1 -- 172.21.15.67:0/2013450514 shutdown_connections 2024-03-09T09:14:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.768+0000 7f174b200640 1 --2- 172.21.15.67:0/2013450514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c153350 0x7f174c070030 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.768+0000 7f174b200640 1 -- 172.21.15.67:0/2013450514 >> 172.21.15.67:0/2013450514 conn(0x7f174c0aa890 msgr2=0x7f174c066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.768+0000 7f174b200640 1 -- 172.21.15.67:0/2013450514 shutdown_connections 2024-03-09T09:14:01.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.768+0000 7f174b200640 1 -- 172.21.15.67:0/2013450514 wait complete. 2024-03-09T09:14:01.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.769+0000 7f174b200640 1 Processor -- start 2024-03-09T09:14:01.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.769+0000 7f174b200640 1 -- start start 2024-03-09T09:14:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.770+0000 7f174b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f174c0c3590 0x7f174c14ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.770+0000 7f174b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0c9400 0x7f174c1ecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.770+0000 7f174b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c153350 0x7f174c1eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.770+0000 7f174b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c073270 con 0x7f174c153350 2024-03-09T09:14:01.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.770+0000 7f174b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c0c4fe0 con 0x7f174c0c3590 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.770+0000 7f174b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174c0c28a0 con 0x7f174c0c9400 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f174a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c153350 0x7f174c1eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f1749e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f174c0c3590 0x7f174c14ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f1749e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f174c0c3590 0x7f174c14ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:38902/0 (socket says 172.21.15.67:38902) 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f1749e00640 1 -- 172.21.15.67:0/2288118068 learned_addr learned my addr 172.21.15.67:0/2288118068 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f174a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c153350 0x7f174c1eecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46130/0 (socket says 172.21.15.67:46130) 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f1743e00640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0c9400 0x7f174c1ecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f1743e00640 1 -- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0c9400 msgr2=0x7f174c1ecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f1743e00640 1 -- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0c9400 msgr2=0x7f174c1ecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f1743e00640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0c9400 0x7f174c1ecbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f1743e00640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0c9400 0x7f174c1ecbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f174a800640 1 -- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0c9400 msgr2=0x7f174c1ecbd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f174a800640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f174c0c9400 0x7f174c1ecbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f174a800640 1 -- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f174c0c3590 msgr2=0x7f174c14ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f174a800640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f174c0c3590 0x7f174c14ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.771+0000 7f174a800640 1 -- 172.21.15.67:0/2288118068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c14b420 con 0x7f174c153350 2024-03-09T09:14:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.772+0000 7f174a800640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c153350 0x7f174c1eecd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f174400c730 tx=0x7f174400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.772+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744018070 con 0x7f174c153350 2024-03-09T09:14:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.772+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1744004d60 con 0x7f174c153350 2024-03-09T09:14:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.772+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174400a880 con 0x7f174c153350 2024-03-09T09:14:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.772+0000 7f174b200640 1 -- 172.21.15.67:0/2288118068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174c14a070 con 0x7f174c153350 2024-03-09T09:14:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.772+0000 7f174b200640 1 -- 172.21.15.67:0/2288118068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c1efa60 con 0x7f174c153350 2024-03-09T09:14:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.773+0000 7f174b200640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c1ef210 0x7f174c14a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.773+0000 7f174b200640 1 -- 172.21.15.67:0/2288118068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c04f3c0 con 0x7f174c1ef210 2024-03-09T09:14:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.774+0000 7f1749e00640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c1ef210 0x7f174c14a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.774+0000 7f1749e00640 1 -- 172.21.15.67:0/2288118068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f174c14a7b0 con 0x7f174c1ef210 2024-03-09T09:14:01.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.775+0000 7f1749e00640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c1ef210 0x7f174c14a270 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f173c0090b0 tx=0x7f173c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:01.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.775+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f174400aa20 con 0x7f174c153350 2024-03-09T09:14:01.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.776+0000 7f1742000640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f172806b4d0 0x7f172806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.776+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1744089960 con 0x7f174c153350 2024-03-09T09:14:01.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.776+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c037990 con 0x7f174c1ef210 2024-03-09T09:14:01.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.776+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f173c037b30 con 0x7f174c1ef210 2024-03-09T09:14:01.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.776+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c1ef210 msgr2=0x7f174c14a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.776+0000 7f1742000640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c1ef210 0x7f174c14a270 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f173c0090b0 tx=0x7f173c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.776+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c1ef210 msgr2=0x7f174c14a270 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.776+0000 7f1742000640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c1ef210 0x7f174c14a270 secure :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f173c0090b0 tx=0x7f173c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.776+0000 7f1743e00640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f172806b4d0 0x7f172806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.777+0000 7f1743e00640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f172806b4d0 0x7f172806d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f174c1eec30 tx=0x7f1734009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:01.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.784+0000 7f174b200640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c14c1c0 0x7f174c141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:01.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.784+0000 7f174b200640 1 -- 172.21.15.67:0/2288118068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c0c3010 con 0x7f174c14c1c0 2024-03-09T09:14:01.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.784+0000 7f1749e00640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c14c1c0 0x7f174c141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:01.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.785+0000 7f1749e00640 1 -- 172.21.15.67:0/2288118068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f174c06fcb0 con 0x7f174c14c1c0 2024-03-09T09:14:01.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.785+0000 7f1749e00640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c14c1c0 0x7f174c141a10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f173c038000 tx=0x7f173c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:01.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.785+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c0044b0 con 0x7f174c14c1c0 2024-03-09T09:14:01.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f173c0044b0 con 0x7f174c14c1c0 2024-03-09T09:14:01.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c14c1c0 msgr2=0x7f174c141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f1742000640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c14c1c0 0x7f174c141a10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f173c038000 tx=0x7f173c009240 comp rx=0 tx=0).stop 2024-03-09T09:14:01.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f1742000640 1 -- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c14c1c0 msgr2=0x7f174c141a10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:01.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f1742000640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c14c1c0 0x7f174c141a10 secure :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f173c038000 tx=0x7f173c009240 comp rx=0 tx=0).stop 2024-03-09T09:14:01.791 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 162, 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:14:01.792 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:14:01.793 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.794 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:14:01.795 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:01.796 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.798 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:01.799 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:01.801 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.802 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.803 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:01.804 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:14:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:01.805 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:14:01.805 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:14:01.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f174b200640 1 -- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f172806b4d0 msgr2=0x7f172806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f174b200640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f172806b4d0 0x7f172806d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f174c1eec30 tx=0x7f1734009040 comp rx=0 tx=0).stop 2024-03-09T09:14:01.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f174b200640 1 -- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c153350 msgr2=0x7f174c1eecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:01.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f174b200640 1 --2- 172.21.15.67:0/2288118068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f174c153350 0x7f174c1eecd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f174400c730 tx=0x7f174400cbf0 comp rx=0 tx=0).stop 2024-03-09T09:14:01.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f174a800640 1 -- 172.21.15.67:0/2288118068 reap_dead start 2024-03-09T09:14:01.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f174a800640 1 -- 172.21.15.67:0/2288118068 reap_dead start 2024-03-09T09:14:01.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f174b200640 1 -- 172.21.15.67:0/2288118068 shutdown_connections 2024-03-09T09:14:01.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f174b200640 1 -- 172.21.15.67:0/2288118068 >> 172.21.15.67:0/2288118068 conn(0x7f174c0aa890 msgr2=0x7f174c072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:01.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f174b200640 1 -- 172.21.15.67:0/2288118068 shutdown_connections 2024-03-09T09:14:01.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:01.786+0000 7f174b200640 1 -- 172.21.15.67:0/2288118068 wait complete. 2024-03-09T09:14:01.808 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:14:01.808 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:14:01.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:14:02.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.003+0000 7f9a7fe00640 1 Processor -- start 2024-03-09T09:14:02.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.003+0000 7f9a7fe00640 1 -- start start 2024-03-09T09:14:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.004+0000 7f9a7fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a7814fef0 0x7f9a7806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a780c5ff0 0x7f9a780ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7fe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a78155e70 con 0x7f9a780bed00 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7fe00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a780c9d60 con 0x7f9a7814fef0 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780c9710 con 0x7f9a780c5ff0 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a7814fef0 0x7f9a7806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9a7814fef0 0x7f9a7806b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46152/0 (socket says 172.21.15.67:46152) 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7d400640 1 -- 172.21.15.67:0/472890496 learned_addr learned my addr 172.21.15.67:0/472890496 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7ca00640 1 --2- 172.21.15.67:0/472890496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a780c5ff0 0x7f9a780ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7d400640 1 -- 172.21.15.67:0/472890496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a780c5ff0 msgr2=0x7f9a780ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7d400640 1 --2- 172.21.15.67:0/472890496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a780c5ff0 0x7f9a780ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7d400640 1 -- 172.21.15.67:0/472890496 >> v1:172.21.15.134:6789/0 conn(0x7f9a780bed00 legacy=0x7f9a780ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.005+0000 7f9a7d400640 1 -- 172.21.15.67:0/472890496 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780c99f0 con 0x7f9a7814fef0 2024-03-09T09:14:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.007+0000 7f9a7d400640 1 --2- 172.21.15.67:0/472890496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a7814fef0 0x7f9a7806b040 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009e50 tx=0x7f9a6c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92015d0166ff083e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.007+0000 7f9a77e00640 1 -- 172.21.15.67:0/472890496 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c02ca70 con 0x7f9a7814fef0 2024-03-09T09:14:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.007+0000 7f9a77e00640 1 -- 172.21.15.67:0/472890496 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c02cc10 con 0x7f9a7814fef0 2024-03-09T09:14:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.007+0000 7f9a77e00640 1 -- 172.21.15.67:0/472890496 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0343f0 con 0x7f9a7814fef0 2024-03-09T09:14:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.008+0000 7f9a7fe00640 1 -- 172.21.15.67:0/472890496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a7814fef0 msgr2=0x7f9a7806b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.008+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/472890496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a7814fef0 0x7f9a7806b040 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009e50 tx=0x7f9a6c02bed0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.008+0000 7f9a7fe00640 1 -- 172.21.15.67:0/472890496 shutdown_connections 2024-03-09T09:14:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.008+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/472890496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a780c5ff0 0x7f9a780ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.008+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/472890496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a7814fef0 0x7f9a7806b040 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.008+0000 7f9a7fe00640 1 -- 172.21.15.67:0/472890496 >> 172.21.15.67:0/472890496 conn(0x7f9a780cb670 msgr2=0x7f9a780cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.008+0000 7f9a7fe00640 1 -- 172.21.15.67:0/472890496 shutdown_connections 2024-03-09T09:14:02.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.008+0000 7f9a7fe00640 1 -- 172.21.15.67:0/472890496 wait complete. 2024-03-09T09:14:02.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.010+0000 7f9a7fe00640 1 Processor -- start 2024-03-09T09:14:02.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.010+0000 7f9a7fe00640 1 -- start start 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.011+0000 7f9a7fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a780bed00 0x7f9a78158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.011+0000 7f9a7fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a780c5ff0 0x7f9a780abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.011+0000 7f9a7fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7814fef0 0x7f9a78153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.011+0000 7f9a7fe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780cb470 con 0x7f9a780bed00 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.011+0000 7f9a7fe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780cb2f0 con 0x7f9a7814fef0 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.011+0000 7f9a7fe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7806b580 con 0x7f9a780c5ff0 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.011+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a780bed00 0x7f9a78158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.011+0000 7f9a7d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a780bed00 0x7f9a78158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46168/0 (socket says 172.21.15.67:46168) 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.011+0000 7f9a7d400640 1 -- 172.21.15.67:0/700580263 learned_addr learned my addr 172.21.15.67:0/700580263 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.011+0000 7f9a7de00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7814fef0 0x7f9a78153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.011+0000 7f9a7ca00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a780c5ff0 0x7f9a780abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.012+0000 7f9a7d400640 1 -- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a780c5ff0 msgr2=0x7f9a780abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.012+0000 7f9a7d400640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a780c5ff0 0x7f9a780abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.012+0000 7f9a7d400640 1 -- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7814fef0 msgr2=0x7f9a78153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.012+0000 7f9a7d400640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7814fef0 0x7f9a78153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.012+0000 7f9a7d400640 1 -- 172.21.15.67:0/700580263 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a781590b0 con 0x7f9a780bed00 2024-03-09T09:14:02.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.012+0000 7f9a7de00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a7814fef0 0x7f9a78153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:14:02.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.012+0000 7f9a7d400640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a780bed00 0x7f9a78158ad0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c036040 tx=0x7f9a6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.012+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c043020 con 0x7f9a780bed00 2024-03-09T09:14:02.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.012+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c03e450 con 0x7f9a780bed00 2024-03-09T09:14:02.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.012+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c009510 con 0x7f9a780bed00 2024-03-09T09:14:02.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.013+0000 7f9a7fe00640 1 -- 172.21.15.67:0/700580263 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a781ef240 con 0x7f9a780bed00 2024-03-09T09:14:02.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.013+0000 7f9a7fe00640 1 -- 172.21.15.67:0/700580263 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a781efd20 con 0x7f9a780bed00 2024-03-09T09:14:02.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.014+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a781489e0 0x7f9a781539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.014+0000 7f9a7fe00640 1 -- 172.21.15.67:0/700580263 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a781efef0 con 0x7f9a781489e0 2024-03-09T09:14:02.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.014+0000 7f9a7ca00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a781489e0 0x7f9a781539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.015+0000 7f9a7ca00640 1 -- 172.21.15.67:0/700580263 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a78075a00 con 0x7f9a781489e0 2024-03-09T09:14:02.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.015+0000 7f9a7ca00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a781489e0 0x7f9a781539d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9a64006fd0 tx=0x7f9a6400f930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.015+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9a6c046020 con 0x7f9a780bed00 2024-03-09T09:14:02.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.016+0000 7f9a75600640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9a5806b4d0 0x7f9a5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.016+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9a6c0acdb0 con 0x7f9a780bed00 2024-03-09T09:14:02.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.016+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64019030 con 0x7f9a781489e0 2024-03-09T09:14:02.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.016+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f9a64004020 con 0x7f9a781489e0 2024-03-09T09:14:02.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.016+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a781489e0 msgr2=0x7f9a781539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.016+0000 7f9a75600640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a781489e0 0x7f9a781539d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9a64006fd0 tx=0x7f9a6400f930 comp rx=0 tx=0).stop 2024-03-09T09:14:02.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.017+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a781489e0 msgr2=0x7f9a781539d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:02.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.017+0000 7f9a75600640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a781489e0 0x7f9a781539d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.017+0000 7f9a7de00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9a5806b4d0 0x7f9a5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.018+0000 7f9a7de00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9a5806b4d0 0x7f9a5806d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9a70007fd0 tx=0x7f9a70009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.031+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a780ac350 0x7f9a781eba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.031+0000 7f9a7fe00640 1 -- 172.21.15.67:0/700580263 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780cb0d0 con 0x7f9a780ac350 2024-03-09T09:14:02.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.031+0000 7f9a7ca00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a780ac350 0x7f9a781eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.032+0000 7f9a7ca00640 1 -- 172.21.15.67:0/700580263 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9a780c6c60 con 0x7f9a780ac350 2024-03-09T09:14:02.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.032+0000 7f9a7ca00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a780ac350 0x7f9a781eba80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9a64006fd0 tx=0x7f9a640046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.033+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64014dc0 con 0x7f9a780ac350 2024-03-09T09:14:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f9a64014dc0 con 0x7f9a780ac350 2024-03-09T09:14:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a780ac350 msgr2=0x7f9a781eba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a75600640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a780ac350 0x7f9a781eba80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9a64006fd0 tx=0x7f9a640046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a75600640 1 -- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a780ac350 msgr2=0x7f9a781eba80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:02.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a75600640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a780ac350 0x7f9a781eba80 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9a64006fd0 tx=0x7f9a640046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.040 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:14:02.040 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:14:02.040 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:14:02.040 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:14:02.040 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 162, 2024-03-09T09:14:02.040 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:14:02.040 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:14:02.040 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:14:02.040 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:14:02.041 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:02.041 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:14:02.041 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:02.041 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:14:02.041 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:14:02.041 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:02.041 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:02.041 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:02.041 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:02.041 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:02.042 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:02.042 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:02.042 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:02.042 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:02.042 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:02.042 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:02.042 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:02.042 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:14:02.042 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:14:02.042 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:02.042 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:02.043 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:02.043 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:02.043 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:02.043 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:02.043 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:02.043 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:02.043 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:02.043 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:02.043 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:02.043 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.044 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:14:02.044 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:14:02.044 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:14:02.044 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:14:02.044 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:14:02.044 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:14:02.044 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:02.045 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:02.045 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:14:02.045 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:14:02.045 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:14:02.045 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:14:02.045 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:14:02.045 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:14:02.045 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:14:02.045 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:02.045 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:02.046 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.047 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:14:02.048 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.049 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:02.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.051 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.052 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:02.053 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:14:02.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.053 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:02.053 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:02.053 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:02.053 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.053 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.053 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:14:02.053 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:14:02.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a7fe00640 1 -- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9a5806b4d0 msgr2=0x7f9a5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9a5806b4d0 0x7f9a5806d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9a70007fd0 tx=0x7f9a70009040 comp rx=0 tx=0).stop 2024-03-09T09:14:02.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a7fe00640 1 -- 172.21.15.67:0/700580263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a780bed00 msgr2=0x7f9a78158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a780bed00 0x7f9a78158ad0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c036040 tx=0x7f9a6c004a40 comp rx=0 tx=0).stop 2024-03-09T09:14:02.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a7de00640 1 -- 172.21.15.67:0/700580263 reap_dead start 2024-03-09T09:14:02.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a7fe00640 1 -- 172.21.15.67:0/700580263 shutdown_connections 2024-03-09T09:14:02.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/700580263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a780bed00 0x7f9a78158ad0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a7fe00640 1 -- 172.21.15.67:0/700580263 >> 172.21.15.67:0/700580263 conn(0x7f9a780cb670 msgr2=0x7f9a780ca2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:02.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a7fe00640 1 -- 172.21.15.67:0/700580263 shutdown_connections 2024-03-09T09:14:02.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.034+0000 7f9a7fe00640 1 -- 172.21.15.67:0/700580263 wait complete. 2024-03-09T09:14:02.058 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:14:02.058 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:14:02.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:14:02.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.289+0000 7f9ae4e00640 1 Processor -- start 2024-03-09T09:14:02.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.290+0000 7f9ae4e00640 1 -- start start 2024-03-09T09:14:02.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.291+0000 7f9ae4e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9ae0157890 0x7f9ae006d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.291+0000 7f9ae4e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ae006dad0 0x7f9ae015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.291+0000 7f9ae4e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae015a370 con 0x7f9ae006e630 2024-03-09T09:14:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.291+0000 7f9ae4e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae015b130 con 0x7f9ae006dad0 2024-03-09T09:14:02.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.291+0000 7f9ae4e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae015a650 con 0x7f9ae0157890 2024-03-09T09:14:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.292+0000 7f9ade800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ae006dad0 0x7f9ae015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.292+0000 7f9adde00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9ae0157890 0x7f9ae006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.292+0000 7f9ade800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9ae006dad0 0x7f9ae015bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46184/0 (socket says 172.21.15.67:46184) 2024-03-09T09:14:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.292+0000 7f9ade800640 1 -- 172.21.15.67:0/2517020719 learned_addr learned my addr 172.21.15.67:0/2517020719 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.292+0000 7f9adde00640 1 -- 172.21.15.67:0/2517020719 >> v2:172.21.15.67:3300/0 conn(0x7f9ae006dad0 msgr2=0x7f9ae015bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:14:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.292+0000 7f9adde00640 1 --2- 172.21.15.67:0/2517020719 >> v2:172.21.15.67:3300/0 conn(0x7f9ae006dad0 0x7f9ae015bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.293+0000 7f9adde00640 1 -- 172.21.15.67:0/2517020719 >> v1:172.21.15.134:6789/0 conn(0x7f9ae006e630 legacy=0x7f9ae015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.293+0000 7f9adde00640 1 -- 172.21.15.67:0/2517020719 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae015a930 con 0x7f9ae0157890 2024-03-09T09:14:02.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.293+0000 7f9ade800640 1 --2- 172.21.15.67:0/2517020719 >> v2:172.21.15.67:3300/0 conn(0x7f9ae006dad0 0x7f9ae015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:14:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.294+0000 7f9adde00640 1 --2- 172.21.15.67:0/2517020719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae0157890 0x7f9ae006d590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0070260 tx=0x7f9ad002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70db00093054599b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.294+0000 7f9adca00640 1 -- 172.21.15.67:0/2517020719 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0034070 con 0x7f9ae0157890 2024-03-09T09:14:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.294+0000 7f9adca00640 1 -- 172.21.15.67:0/2517020719 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0030610 con 0x7f9ae0157890 2024-03-09T09:14:02.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.294+0000 7f9adca00640 1 -- 172.21.15.67:0/2517020719 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad00309e0 con 0x7f9ae0157890 2024-03-09T09:14:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.295+0000 7f9ae4e00640 1 -- 172.21.15.67:0/2517020719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae0157890 msgr2=0x7f9ae006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.295+0000 7f9ae4e00640 1 --2- 172.21.15.67:0/2517020719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae0157890 0x7f9ae006d590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0070260 tx=0x7f9ad002b960 comp rx=0 tx=0).stop 2024-03-09T09:14:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.295+0000 7f9ae4e00640 1 -- 172.21.15.67:0/2517020719 shutdown_connections 2024-03-09T09:14:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.295+0000 7f9ae4e00640 1 --2- 172.21.15.67:0/2517020719 >> v2:172.21.15.67:3300/0 conn(0x7f9ae006dad0 0x7f9ae015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.295+0000 7f9ae4e00640 1 --2- 172.21.15.67:0/2517020719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae0157890 0x7f9ae006d590 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.295+0000 7f9ae4e00640 1 -- 172.21.15.67:0/2517020719 >> 172.21.15.67:0/2517020719 conn(0x7f9ae00c7370 msgr2=0x7f9ae00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.295+0000 7f9ae4e00640 1 -- 172.21.15.67:0/2517020719 shutdown_connections 2024-03-09T09:14:02.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.296+0000 7f9ae4e00640 1 -- 172.21.15.67:0/2517020719 wait complete. 2024-03-09T09:14:02.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.298+0000 7f9ae4e00640 1 Processor -- start 2024-03-09T09:14:02.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.298+0000 7f9ae4e00640 1 -- start start 2024-03-09T09:14:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.299+0000 7f9ae4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ae006dad0 0x7f9ae00a8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.299+0000 7f9ae4e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae006e630 0x7f9ae00a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.299+0000 7f9ae4e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae0157890 0x7f9ae00a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.299+0000 7f9ae4e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae00c1ef0 con 0x7f9ae006dad0 2024-03-09T09:14:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.299+0000 7f9ae4e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae00c1d70 con 0x7f9ae0157890 2024-03-09T09:14:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.299+0000 7f9ae4e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0157c70 con 0x7f9ae006e630 2024-03-09T09:14:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.300+0000 7f9adde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ae006dad0 0x7f9ae00a8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.300+0000 7f9adde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ae006dad0 0x7f9ae00a8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46194/0 (socket says 172.21.15.67:46194) 2024-03-09T09:14:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.300+0000 7f9ade800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae0157890 0x7f9ae00a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.300+0000 7f9adde00640 1 -- 172.21.15.67:0/85707468 learned_addr learned my addr 172.21.15.67:0/85707468 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:02.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.300+0000 7f9add400640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae006e630 0x7f9ae00a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.300+0000 7f9adde00640 1 -- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae006e630 msgr2=0x7f9ae00a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.300+0000 7f9adde00640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae006e630 0x7f9ae00a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.300+0000 7f9adde00640 1 -- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae0157890 msgr2=0x7f9ae00a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.300+0000 7f9adde00640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ae0157890 0x7f9ae00a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.300+0000 7f9adde00640 1 -- 172.21.15.67:0/85707468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae01530f0 con 0x7f9ae006dad0 2024-03-09T09:14:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.301+0000 7f9adde00640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ae006dad0 0x7f9ae00a8c70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009890 tx=0x7f9ad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.301+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0034040 con 0x7f9ae006dad0 2024-03-09T09:14:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.301+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad003aaa0 con 0x7f9ae006dad0 2024-03-09T09:14:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.301+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad00443f0 con 0x7f9ae006dad0 2024-03-09T09:14:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.301+0000 7f9ae4e00640 1 -- 172.21.15.67:0/85707468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae01f0880 con 0x7f9ae006dad0 2024-03-09T09:14:02.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.301+0000 7f9ae4e00640 1 -- 172.21.15.67:0/85707468 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae00715b0 con 0x7f9ae006dad0 2024-03-09T09:14:02.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.303+0000 7f9ae4e00640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a4ea0 0x7f9ae00717b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.303+0000 7f9ae4e00640 1 -- 172.21.15.67:0/85707468 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae006e290 con 0x7f9ae00a4ea0 2024-03-09T09:14:02.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.303+0000 7f9add400640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a4ea0 0x7f9ae00717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.304+0000 7f9add400640 1 -- 172.21.15.67:0/85707468 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ae01f0b60 con 0x7f9ae00a4ea0 2024-03-09T09:14:02.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.304+0000 7f9add400640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a4ea0 0x7f9ae00717b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0072d00 tx=0x7f9acc00e960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.304+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9ad0037070 con 0x7f9ae006dad0 2024-03-09T09:14:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.306+0000 7f9ad6000640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9ac006b4d0 0x7f9ac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.306+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9ad00ac490 con 0x7f9ae006dad0 2024-03-09T09:14:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.306+0000 7f9ade800640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9ac006b4d0 0x7f9ac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.306+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc0052e0 con 0x7f9ae00a4ea0 2024-03-09T09:14:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.306+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f9acc005480 con 0x7f9ae00a4ea0 2024-03-09T09:14:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.306+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a4ea0 msgr2=0x7f9ae00717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.306+0000 7f9ad6000640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a4ea0 0x7f9ae00717b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0072d00 tx=0x7f9acc00e960 comp rx=0 tx=0).stop 2024-03-09T09:14:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.306+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a4ea0 msgr2=0x7f9ae00717b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.306+0000 7f9ad6000640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a4ea0 0x7f9ae00717b0 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0072d00 tx=0x7f9acc00e960 comp rx=0 tx=0).stop 2024-03-09T09:14:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.307+0000 7f9ade800640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9ac006b4d0 0x7f9ac006d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009c90 tx=0x7f9ad800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.320+0000 7f9ae4e00640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a7220 0x7f9ae0071db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.320+0000 7f9ae4e00640 1 -- 172.21.15.67:0/85707468 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0157090 con 0x7f9ae00a7220 2024-03-09T09:14:02.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.320+0000 7f9add400640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a7220 0x7f9ae0071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.321+0000 7f9add400640 1 -- 172.21.15.67:0/85707468 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9ae00c3480 con 0x7f9ae00a7220 2024-03-09T09:14:02.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.321+0000 7f9add400640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a7220 0x7f9ae0071db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0072d00 tx=0x7f9acc001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.322+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc007660 con 0x7f9ae00a7220 2024-03-09T09:14:02.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.322+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f9acc007660 con 0x7f9ae00a7220 2024-03-09T09:14:02.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.322+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a7220 msgr2=0x7f9ae0071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.322+0000 7f9ad6000640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a7220 0x7f9ae0071db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0072d00 tx=0x7f9acc001120 comp rx=0 tx=0).stop 2024-03-09T09:14:02.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.322+0000 7f9ad6000640 1 -- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a7220 msgr2=0x7f9ae0071db0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:02.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.322+0000 7f9ad6000640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae00a7220 0x7f9ae0071db0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0072d00 tx=0x7f9acc001120 comp rx=0 tx=0).stop 2024-03-09T09:14:02.328 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:14:02.328 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:14:02.328 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:14:02.328 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:14:02.328 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 162, 2024-03-09T09:14:02.328 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:14:02.328 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:14:02.328 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:14:02.328 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:14:02.328 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:02.329 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:02.330 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:14:02.331 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:02.332 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:02.333 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:02.334 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.335 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:14:02.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:02.338 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:14:02.339 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:14:02.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.323+0000 7f9ae4e00640 1 -- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9ac006b4d0 msgr2=0x7f9ac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.323+0000 7f9ae4e00640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9ac006b4d0 0x7f9ac006d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009c90 tx=0x7f9ad800b040 comp rx=0 tx=0).stop 2024-03-09T09:14:02.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.323+0000 7f9ae4e00640 1 -- 172.21.15.67:0/85707468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ae006dad0 msgr2=0x7f9ae00a8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.323+0000 7f9ae4e00640 1 --2- 172.21.15.67:0/85707468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ae006dad0 0x7f9ae00a8c70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009890 tx=0x7f9ad00046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.323+0000 7f9ade800640 1 -- 172.21.15.67:0/85707468 reap_dead start 2024-03-09T09:14:02.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.323+0000 7f9ade800640 1 -- 172.21.15.67:0/85707468 reap_dead start 2024-03-09T09:14:02.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.323+0000 7f9ae4e00640 1 -- 172.21.15.67:0/85707468 shutdown_connections 2024-03-09T09:14:02.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.323+0000 7f9ae4e00640 1 -- 172.21.15.67:0/85707468 >> 172.21.15.67:0/85707468 conn(0x7f9ae00c7370 msgr2=0x7f9ae01598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:02.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.323+0000 7f9ae4e00640 1 -- 172.21.15.67:0/85707468 shutdown_connections 2024-03-09T09:14:02.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.323+0000 7f9ae4e00640 1 -- 172.21.15.67:0/85707468 wait complete. 2024-03-09T09:14:02.347 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:14:02.347 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:14:02.348 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:14:02.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.540+0000 7fc9d6e00640 1 Processor -- start 2024-03-09T09:14:02.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.540+0000 7fc9d6e00640 1 -- start start 2024-03-09T09:14:02.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.541+0000 7fc9d6e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9d00c30f0 0x7fc9d0152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.541+0000 7fc9d6e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d00ab3c0 con 0x7fc9d00aa420 2024-03-09T09:14:02.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.541+0000 7fc9d6e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d00aa8a0 con 0x7fc9d006dbe0 2024-03-09T09:14:02.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.541+0000 7fc9d6e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d00c82d0 con 0x7fc9d00c30f0 2024-03-09T09:14:02.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.541+0000 7fc9cf400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc9d00aa420 0x7fc9d01490f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:35652/0 (socket says 172.21.15.67:35652) 2024-03-09T09:14:02.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.541+0000 7fc9cf400640 1 -- 172.21.15.67:0/644223453 learned_addr learned my addr 172.21.15.67:0/644223453 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:14:02.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.542+0000 7fc9cfe00640 1 --2- 172.21.15.67:0/644223453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9d00c30f0 0x7fc9d0152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.542+0000 7fc9cea00640 1 -- 172.21.15.67:0/644223453 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548788001 0 0) 0x7fc9bc001110 con 0x7fc9d00aa420 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.542+0000 7fc9cea00640 1 -- 172.21.15.67:0/644223453 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9b4003520 con 0x7fc9d00aa420 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.542+0000 7fc9cfe00640 1 -- 172.21.15.67:0/644223453 >> v1:172.21.15.134:6789/0 conn(0x7fc9d006dbe0 legacy=0x7fc9d0072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.542+0000 7fc9cea00640 1 -- 172.21.15.67:0/644223453 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278401196 0 0) 0x7fc9c8001490 con 0x7fc9d006dbe0 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.542+0000 7fc9cfe00640 1 -- 172.21.15.67:0/644223453 >> v1:172.21.15.67:6789/0 conn(0x7fc9d00aa420 legacy=0x7fc9d01490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.542+0000 7fc9cfe00640 1 -- 172.21.15.67:0/644223453 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d00aab20 con 0x7fc9d00c30f0 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.543+0000 7fc9cfe00640 1 --2- 172.21.15.67:0/644223453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9d00c30f0 0x7fc9d0152a20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00097d0 tx=0x7fc9c002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474a7001b2d674a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.544+0000 7fc9cea00640 1 -- 172.21.15.67:0/644223453 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0035020 con 0x7fc9d00c30f0 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.544+0000 7fc9cea00640 1 -- 172.21.15.67:0/644223453 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c0031800 con 0x7fc9d00c30f0 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.544+0000 7fc9cea00640 1 -- 172.21.15.67:0/644223453 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0031bd0 con 0x7fc9d00c30f0 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.545+0000 7fc9d6e00640 1 -- 172.21.15.67:0/644223453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9d00c30f0 msgr2=0x7fc9d0152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.545+0000 7fc9d6e00640 1 --2- 172.21.15.67:0/644223453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9d00c30f0 0x7fc9d0152a20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00097d0 tx=0x7fc9c002b8a0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.545+0000 7fc9d6e00640 1 -- 172.21.15.67:0/644223453 shutdown_connections 2024-03-09T09:14:02.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.545+0000 7fc9d6e00640 1 --2- 172.21.15.67:0/644223453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9d00c30f0 0x7fc9d0152a20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.545+0000 7fc9d6e00640 1 -- 172.21.15.67:0/644223453 >> 172.21.15.67:0/644223453 conn(0x7fc9d00c7370 msgr2=0x7fc9d00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:02.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.545+0000 7fc9d6e00640 1 -- 172.21.15.67:0/644223453 shutdown_connections 2024-03-09T09:14:02.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.545+0000 7fc9d6e00640 1 -- 172.21.15.67:0/644223453 wait complete. 2024-03-09T09:14:02.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.547+0000 7fc9d6e00640 1 Processor -- start 2024-03-09T09:14:02.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.547+0000 7fc9d6e00640 1 -- start start 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.548+0000 7fc9d6e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d006dbe0 0x7fc9d0072c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.548+0000 7fc9d6e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9d00aa420 0x7fc9d013f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.548+0000 7fc9d6e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9d00c30f0 0x7fc9d013f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.548+0000 7fc9d6e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d006d6c0 con 0x7fc9d00c30f0 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.548+0000 7fc9d6e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d00ab0e0 con 0x7fc9d006dbe0 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.548+0000 7fc9d6e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d0149800 con 0x7fc9d00aa420 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.549+0000 7fc9cfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d006dbe0 0x7fc9d0072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.549+0000 7fc9cfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d006dbe0 0x7fc9d0072c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:38944/0 (socket says 172.21.15.67:38944) 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.549+0000 7fc9cfe00640 1 -- 172.21.15.67:0/458721031 learned_addr learned my addr 172.21.15.67:0/458721031 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.549+0000 7fc9d4e00640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9d00c30f0 0x7fc9d013f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.549+0000 7fc9cf400640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9d00aa420 0x7fc9d013f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.549+0000 7fc9cfe00640 1 -- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9d00aa420 msgr2=0x7fc9d013f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.549+0000 7fc9cfe00640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9d00aa420 0x7fc9d013f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.549+0000 7fc9cfe00640 1 -- 172.21.15.67:0/458721031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9d00c30f0 msgr2=0x7fc9d013f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.549+0000 7fc9cfe00640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9d00c30f0 0x7fc9d013f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.549+0000 7fc9cfe00640 1 -- 172.21.15.67:0/458721031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d0145bc0 con 0x7fc9d006dbe0 2024-03-09T09:14:02.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.550+0000 7fc9cfe00640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d006dbe0 0x7fc9d0072c90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0000c00 tx=0x7fc9c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.550+0000 7fc9c7e00640 1 -- 172.21.15.67:0/458721031 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0035050 con 0x7fc9d006dbe0 2024-03-09T09:14:02.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.550+0000 7fc9c7e00640 1 -- 172.21.15.67:0/458721031 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c003bd70 con 0x7fc9d006dbe0 2024-03-09T09:14:02.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.550+0000 7fc9c7e00640 1 -- 172.21.15.67:0/458721031 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0006e10 con 0x7fc9d006dbe0 2024-03-09T09:14:02.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.550+0000 7fc9d6e00640 1 -- 172.21.15.67:0/458721031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d0072720 con 0x7fc9d006dbe0 2024-03-09T09:14:02.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.550+0000 7fc9d6e00640 1 -- 172.21.15.67:0/458721031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d0146160 con 0x7fc9d006dbe0 2024-03-09T09:14:02.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.551+0000 7fc9d6e00640 1 -- 172.21.15.67:0/458721031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d0075a00 con 0x7fc9d006dbe0 2024-03-09T09:14:02.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.552+0000 7fc9c7e00640 1 -- 172.21.15.67:0/458721031 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc9c004c060 con 0x7fc9d006dbe0 2024-03-09T09:14:02.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.553+0000 7fc9c7e00640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc9b006b4d0 0x7fc9b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:02.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.553+0000 7fc9c7e00640 1 -- 172.21.15.67:0/458721031 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc9c00ac310 con 0x7fc9d006dbe0 2024-03-09T09:14:02.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.554+0000 7fc9cf400640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc9b006b4d0 0x7fc9b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:02.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.554+0000 7fc9cf400640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc9b006b4d0 0x7fc9b006d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00c5870 tx=0x7fc9bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:02.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.564+0000 7fc9c7e00640 1 -- 172.21.15.67:0/458721031 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc9c0040070 con 0x7fc9d006dbe0 2024-03-09T09:14:02.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.859+0000 7fc9d6e00640 1 -- 172.21.15.67:0/458721031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fc9d00731d0 con 0x7fc9d006dbe0 2024-03-09T09:14:02.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.862+0000 7fc9c7e00640 1 -- 172.21.15.67:0/458721031 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fc9c007a740 con 0x7fc9d006dbe0 2024-03-09T09:14:02.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.864+0000 7fc9d6e00640 1 -- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc9b006b4d0 msgr2=0x7fc9b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.864+0000 7fc9d6e00640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc9b006b4d0 0x7fc9b006d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00c5870 tx=0x7fc9bc009040 comp rx=0 tx=0).stop 2024-03-09T09:14:02.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.864+0000 7fc9d6e00640 1 -- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d006dbe0 msgr2=0x7fc9d0072c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:02.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.864+0000 7fc9d6e00640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d006dbe0 0x7fc9d0072c90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0000c00 tx=0x7fc9c00046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.864+0000 7fc9d6e00640 1 -- 172.21.15.67:0/458721031 shutdown_connections 2024-03-09T09:14:02.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.864+0000 7fc9d6e00640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc9b006b4d0 0x7fc9b006d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.864+0000 7fc9d6e00640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9d00c30f0 0x7fc9d013f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.864+0000 7fc9d6e00640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9d00aa420 0x7fc9d013f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.864+0000 7fc9d6e00640 1 --2- 172.21.15.67:0/458721031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d006dbe0 0x7fc9d0072c90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:02.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.865+0000 7fc9d6e00640 1 -- 172.21.15.67:0/458721031 >> 172.21.15.67:0/458721031 conn(0x7fc9d00c7370 msgr2=0x7fc9d0074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:02.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.865+0000 7fc9d6e00640 1 -- 172.21.15.67:0/458721031 shutdown_connections 2024-03-09T09:14:02.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:02.865+0000 7fc9d6e00640 1 -- 172.21.15.67:0/458721031 wait complete. 2024-03-09T09:14:02.888 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:14:04.133 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 50% (22/44) Updating files: 52% (23/44) Updating files: 54% (24/44) Updating files: 56% (25/44) Updating files: 59% (26/44) Updating files: 61% (27/44) Updating files: 63% (28/44) Updating files: 65% (29/44) Updating files: 68% (30/44) Updating files: 70% (31/44) Updating files: 72% (32/44) Updating files: 75% (33/44) Updating files: 77% (34/44) Updating files: 79% (35/44) Updating files: 81% (36/44) Updating files: 84% (37/44) Updating files: 86% (38/44) Updating files: 88% (39/44) Updating files: 90% (40/44) Updating files: 93% (41/44) Updating files: 95% (42/44) Updating files: 97% (43/44) Updating files: 100% (44/44) Updating files: 100% (44/44), done. 2024-03-09T09:14:04.217 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at e3bb0656d92 v0.70 2024-03-09T09:14:04.224 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.70 2024-03-09T09:14:04.343 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:14:04.345 INFO:tasks.workunit.client.0.smithi067.stdout:v0.71 2024-03-09T09:14:04.345 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 71 -eq 48 ']' 2024-03-09T09:14:04.345 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.71 2024-03-09T09:14:04.345 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.71 2024-03-09T09:14:04.345 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.71 2024-03-09T09:14:08.570 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 8% (34/405) Updating files: 9% (37/405) Updating files: 10% (41/405) Updating files: 11% (45/405) Updating files: 12% (49/405) Updating files: 13% (53/405) Updating files: 14% (57/405) Updating files: 15% (61/405) Updating files: 16% (65/405) Updating files: 17% (69/405) Updating files: 18% (73/405) Updating files: 19% (77/405) Updating files: 20% (81/405) Updating files: 21% (86/405) Updating files: 22% (90/405) Updating files: 23% (94/405) Updating files: 24% (98/405) Updating files: 25% (102/405) Updating files: 26% (106/405) Updating files: 27% (110/405) Updating files: 28% (114/405) Updating files: 29% (118/405) Updating files: 30% (122/405) Updating files: 31% (126/405) Updating files: 32% (130/405) Updating files: 33% (134/405) Updating files: 34% (138/405) Updating files: 35% (142/405) Updating files: 36% (146/405) Updating files: 37% (150/405) Updating files: 38% (154/405) Updating files: 39% (158/405) Updating files: 40% (162/405) Updating files: 41% (167/405) Updating files: 42% (171/405) Updating files: 43% (175/405) Updating files: 44% (179/405) Updating files: 45% (183/405) Updating files: 46% (187/405) Updating files: 46% (190/405) Updating files: 47% (191/405) Updating files: 48% (195/405) Updating files: 49% (199/405) Updating files: 50% (203/405) Updating files: 51% (207/405) Updating files: 52% (211/405) Updating files: 53% (215/405) Updating files: 54% (219/405) Updating files: 55% (223/405) Updating files: 56% (227/405) Updating files: 57% (231/405) Updating files: 58% (235/405) Updating files: 58% (236/405) Updating files: 59% (239/405) Updating files: 60% (243/405) Updating files: 61% (248/405) Updating files: 62% (252/405) Updating files: 63% (256/405) Updating files: 64% (260/405) Updating files: 65% (264/405) Updating files: 66% (268/405) Updating files: 67% (272/405) Updating files: 68% (276/405) Updating files: 69% (280/405) Updating files: 70% (284/405) Updating files: 71% (288/405) Updating files: 72% (292/405) Updating files: 73% (296/405) Updating files: 74% (300/405) Updating files: 75% (304/405) Updating files: 76% (308/405) Updating files: 77% (312/405) Updating files: 78% (316/405) Updating files: 79% (320/405) Updating files: 80% (324/405) Updating files: 81% (329/405) Updating files: 81% (330/405) Updating files: 82% (333/405) Updating files: 82% (334/405) Updating files: 83% (337/405) Updating files: 84% (341/405) Updating files: 85% (345/405) Updating files: 86% (349/405) Updating files: 87% (353/405) Updating files: 88% (357/405) Updating files: 89% (361/405) Updating files: 90% (365/405) Updating files: 91% (369/405) Updating files: 92% (373/405) Updating files: 93% (377/405) Updating files: 94% (381/405) Updating files: 95% (385/405) Updating files: 96% (389/405) Updating files: 97% (393/405) Updating files: 98% (397/405) Updating files: 99% (401/405) Updating files: 100% (405/405) Updating files: 100% (405/405), done. 2024-03-09T09:14:08.656 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at e509cb1e69c v0.71 2024-03-09T09:14:08.669 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.71 2024-03-09T09:14:08.803 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:14:08.804 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 72 -eq 48 ']' 2024-03-09T09:14:08.804 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.72 2024-03-09T09:14:08.804 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.72 2024-03-09T09:14:08.804 INFO:tasks.workunit.client.1.smithi134.stdout:v0.72 2024-03-09T09:14:08.804 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.72 2024-03-09T09:14:10.466 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 12% (52/405) Updating files: 13% (53/405) Updating files: 14% (57/405) Updating files: 15% (61/405) Updating files: 16% (65/405) Updating files: 17% (69/405) Updating files: 18% (73/405) Updating files: 19% (77/405) Updating files: 20% (81/405) Updating files: 21% (86/405) Updating files: 22% (90/405) Updating files: 23% (94/405) Updating files: 24% (98/405) Updating files: 25% (102/405) Updating files: 26% (106/405) Updating files: 27% (110/405) Updating files: 28% (114/405) Updating files: 29% (118/405) Updating files: 30% (122/405) Updating files: 31% (126/405) Updating files: 32% (130/405) Updating files: 33% (134/405) Updating files: 34% (138/405) Updating files: 35% (142/405) Updating files: 36% (146/405) Updating files: 36% (149/405) Updating files: 37% (150/405) Updating files: 38% (154/405) Updating files: 39% (158/405) Updating files: 40% (162/405) Updating files: 41% (167/405) Updating files: 42% (171/405) Updating files: 43% (175/405) Updating files: 44% (179/405) Updating files: 45% (183/405) Updating files: 45% (186/405) Updating files: 46% (187/405) Updating files: 47% (191/405) Updating files: 48% (195/405) Updating files: 49% (199/405) Updating files: 50% (203/405) Updating files: 51% (207/405) Updating files: 52% (211/405) Updating files: 53% (215/405) Updating files: 54% (219/405) Updating files: 55% (223/405) Updating files: 56% (227/405) Updating files: 57% (231/405) Updating files: 58% (235/405) Updating files: 59% (239/405) Updating files: 60% (243/405) Updating files: 60% (245/405) Updating files: 61% (248/405) Updating files: 62% (252/405) Updating files: 63% (256/405) Updating files: 64% (260/405) Updating files: 65% (264/405) Updating files: 66% (268/405) Updating files: 67% (272/405) Updating files: 68% (276/405) Updating files: 69% (280/405) Updating files: 70% (284/405) Updating files: 71% (288/405) Updating files: 72% (292/405) Updating files: 73% (296/405) Updating files: 74% (300/405) Updating files: 75% (304/405) Updating files: 76% (308/405) Updating files: 77% (312/405) Updating files: 78% (316/405) Updating files: 79% (320/405) Updating files: 80% (324/405) Updating files: 80% (325/405) Updating files: 81% (329/405) Updating files: 82% (333/405) Updating files: 83% (337/405) Updating files: 84% (341/405) Updating files: 85% (345/405) Updating files: 86% (349/405) Updating files: 87% (353/405) Updating files: 88% (357/405) Updating files: 89% (361/405) Updating files: 90% (365/405) Updating files: 91% (369/405) Updating files: 92% (373/405) Updating files: 93% (377/405) Updating files: 94% (381/405) Updating files: 95% (385/405) Updating files: 96% (389/405) Updating files: 97% (393/405) Updating files: 98% (397/405) Updating files: 99% (401/405) Updating files: 100% (405/405) Updating files: 100% (405/405), done. 2024-03-09T09:14:10.876 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at e509cb1e69c v0.71 2024-03-09T09:14:10.890 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.71 2024-03-09T09:14:11.635 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:14:11.635 INFO:tasks.workunit.client.0.smithi067.stdout:v0.72 2024-03-09T09:14:11.635 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 72 -eq 48 ']' 2024-03-09T09:14:11.635 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.72 2024-03-09T09:14:11.636 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.72 2024-03-09T09:14:11.636 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.72 2024-03-09T09:14:12.390 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 24% (47/191) Updating files: 25% (48/191) Updating files: 26% (50/191) Updating files: 27% (52/191) Updating files: 28% (54/191) Updating files: 29% (56/191) Updating files: 30% (58/191) Updating files: 31% (60/191) Updating files: 32% (62/191) Updating files: 33% (64/191) Updating files: 34% (65/191) Updating files: 35% (67/191) Updating files: 36% (69/191) Updating files: 37% (71/191) Updating files: 38% (73/191) Updating files: 39% (75/191) Updating files: 40% (77/191) Updating files: 41% (79/191) Updating files: 42% (81/191) Updating files: 43% (83/191) Updating files: 44% (85/191) Updating files: 45% (86/191) Updating files: 46% (88/191) Updating files: 47% (90/191) Updating files: 48% (92/191) Updating files: 49% (94/191) Updating files: 50% (96/191) Updating files: 51% (98/191) Updating files: 52% (100/191) Updating files: 53% (102/191) Updating files: 54% (104/191) Updating files: 55% (106/191) Updating files: 56% (107/191) Updating files: 57% (109/191) Updating files: 58% (111/191) Updating files: 59% (113/191) Updating files: 60% (115/191) Updating files: 61% (117/191) Updating files: 62% (119/191) Updating files: 63% (121/191) Updating files: 64% (123/191) Updating files: 65% (125/191) Updating files: 66% (127/191) Updating files: 67% (128/191) Updating files: 68% (130/191) Updating files: 69% (132/191) Updating files: 70% (134/191) Updating files: 71% (136/191) Updating files: 72% (138/191) Updating files: 73% (140/191) Updating files: 74% (142/191) Updating files: 75% (144/191) Updating files: 76% (146/191) Updating files: 77% (148/191) Updating files: 78% (149/191) Updating files: 79% (151/191) Updating files: 80% (153/191) Updating files: 81% (155/191) Updating files: 82% (157/191) Updating files: 83% (159/191) Updating files: 84% (161/191) Updating files: 85% (163/191) Updating files: 86% (165/191) Updating files: 87% (167/191) Updating files: 88% (169/191) Updating files: 89% (170/191) Updating files: 90% (172/191) Updating files: 91% (174/191) Updating files: 92% (176/191) Updating files: 93% (178/191) Updating files: 94% (180/191) Updating files: 95% (182/191) Updating files: 96% (184/191) Updating files: 97% (186/191) Updating files: 98% (188/191) Updating files: 99% (190/191) Updating files: 100% (191/191) Updating files: 100% (191/191), done. 2024-03-09T09:14:12.494 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 5832e2603c7 v0.72 2024-03-09T09:14:12.507 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.72 2024-03-09T09:14:12.665 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:14:12.666 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 73 -eq 48 ']' 2024-03-09T09:14:12.666 INFO:tasks.workunit.client.1.smithi134.stdout:v0.73 2024-03-09T09:14:12.666 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.73 2024-03-09T09:14:12.666 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.73 2024-03-09T09:14:12.666 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.73 2024-03-09T09:14:12.889 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:14:12.889 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:14:13.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.102+0000 7f8c08000640 1 Processor -- start 2024-03-09T09:14:13.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.102+0000 7f8c08000640 1 -- start start 2024-03-09T09:14:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.103+0000 7f8c08000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8c00153500 0x7f8c0006b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.103+0000 7f8c08000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c000c7bc0 0x7f8c001606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.103+0000 7f8c08000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0015a2a0 con 0x7f8c000c6630 2024-03-09T09:14:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.103+0000 7f8c08000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c00159d70 con 0x7f8c00153500 2024-03-09T09:14:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.103+0000 7f8c08000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0006bb30 con 0x7f8c000c7bc0 2024-03-09T09:14:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.104+0000 7f8c05600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8c00153500 0x7f8c0006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.104+0000 7f8c05600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8c00153500 0x7f8c0006b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48200/0 (socket says 172.21.15.67:48200) 2024-03-09T09:14:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.104+0000 7f8c05600640 1 -- 172.21.15.67:0/2180387996 learned_addr learned my addr 172.21.15.67:0/2180387996 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.104+0000 7f8c06000640 1 --2- 172.21.15.67:0/2180387996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c000c7bc0 0x7f8c001606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.104+0000 7f8c05600640 1 -- 172.21.15.67:0/2180387996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c000c7bc0 msgr2=0x7f8c001606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:14:13.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.104+0000 7f8c05600640 1 --2- 172.21.15.67:0/2180387996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c000c7bc0 0x7f8c001606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.104+0000 7f8c05600640 1 -- 172.21.15.67:0/2180387996 >> v1:172.21.15.67:6789/0 conn(0x7f8c000c6630 legacy=0x7f8c000c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.104+0000 7f8c05600640 1 -- 172.21.15.67:0/2180387996 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c00159fc0 con 0x7f8c00153500 2024-03-09T09:14:13.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.105+0000 7f8c05600640 1 --2- 172.21.15.67:0/2180387996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c00153500 0x7f8c0006b040 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009e50 tx=0x7f8bf402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3709219d961ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.105+0000 7f8bffe00640 1 -- 172.21.15.67:0/2180387996 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf402c9f0 con 0x7f8c00153500 2024-03-09T09:14:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.106+0000 7f8bffe00640 1 -- 172.21.15.67:0/2180387996 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf402cb90 con 0x7f8c00153500 2024-03-09T09:14:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.106+0000 7f8bffe00640 1 -- 172.21.15.67:0/2180387996 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40343f0 con 0x7f8c00153500 2024-03-09T09:14:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.106+0000 7f8c08000640 1 -- 172.21.15.67:0/2180387996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c00153500 msgr2=0x7f8c0006b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.106+0000 7f8c08000640 1 --2- 172.21.15.67:0/2180387996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c00153500 0x7f8c0006b040 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009e50 tx=0x7f8bf402be50 comp rx=0 tx=0).stop 2024-03-09T09:14:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.106+0000 7f8c08000640 1 -- 172.21.15.67:0/2180387996 shutdown_connections 2024-03-09T09:14:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.106+0000 7f8c08000640 1 --2- 172.21.15.67:0/2180387996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c000c7bc0 0x7f8c001606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.106+0000 7f8c08000640 1 --2- 172.21.15.67:0/2180387996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c00153500 0x7f8c0006b040 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.106+0000 7f8c08000640 1 -- 172.21.15.67:0/2180387996 >> 172.21.15.67:0/2180387996 conn(0x7f8c000c8060 msgr2=0x7f8c000c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:13.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.106+0000 7f8c08000640 1 -- 172.21.15.67:0/2180387996 shutdown_connections 2024-03-09T09:14:13.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.106+0000 7f8c08000640 1 -- 172.21.15.67:0/2180387996 wait complete. 2024-03-09T09:14:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.108+0000 7f8c08000640 1 Processor -- start 2024-03-09T09:14:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.108+0000 7f8c08000640 1 -- start start 2024-03-09T09:14:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.109+0000 7f8c08000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c000c7bc0 0x7f8c0014c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.109+0000 7f8c08000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c00153500 0x7f8c001ef530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.109+0000 7f8c08000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c00153ff0 0x7f8c001efa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.109+0000 7f8c08000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c000bed90 con 0x7f8c00153500 2024-03-09T09:14:13.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.109+0000 7f8c08000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c000c6f90 con 0x7f8c000c7bc0 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.109+0000 7f8c08000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c000c94e0 con 0x7f8c00153ff0 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.109+0000 7f8c04c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c00153500 0x7f8c001ef530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.109+0000 7f8c04c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c00153500 0x7f8c001ef530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48206/0 (socket says 172.21.15.67:48206) 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.109+0000 7f8c04c00640 1 -- 172.21.15.67:0/107463085 learned_addr learned my addr 172.21.15.67:0/107463085 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.109+0000 7f8c05600640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c000c7bc0 0x7f8c0014c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.109+0000 7f8c06000640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c00153ff0 0x7f8c001efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.110+0000 7f8c04c00640 1 -- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c00153ff0 msgr2=0x7f8c001efa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.110+0000 7f8c04c00640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c00153ff0 0x7f8c001efa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.110+0000 7f8c04c00640 1 -- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c000c7bc0 msgr2=0x7f8c0014c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.110+0000 7f8c04c00640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c000c7bc0 0x7f8c0014c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.110+0000 7f8c04c00640 1 -- 172.21.15.67:0/107463085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c001effb0 con 0x7f8c00153500 2024-03-09T09:14:13.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.110+0000 7f8c04c00640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c00153500 0x7f8c001ef530 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00c3c0 tx=0x7f8bec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.110+0000 7f8bfd600640 1 -- 172.21.15.67:0/107463085 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0052d0 con 0x7f8c00153500 2024-03-09T09:14:13.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.111+0000 7f8bfd600640 1 -- 172.21.15.67:0/107463085 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec005490 con 0x7f8c00153500 2024-03-09T09:14:13.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.111+0000 7f8c08000640 1 -- 172.21.15.67:0/107463085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c000c35a0 con 0x7f8c00153500 2024-03-09T09:14:13.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.111+0000 7f8bfd600640 1 -- 172.21.15.67:0/107463085 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec007260 con 0x7f8c00153500 2024-03-09T09:14:13.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.111+0000 7f8c08000640 1 -- 172.21.15.67:0/107463085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c000c3b60 con 0x7f8c00153500 2024-03-09T09:14:13.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.112+0000 7f8c08000640 1 -- 172.21.15.67:0/107463085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c001516f0 con 0x7f8c00153500 2024-03-09T09:14:13.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.113+0000 7f8bfd600640 1 -- 172.21.15.67:0/107463085 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8bec007400 con 0x7f8c00153500 2024-03-09T09:14:13.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.114+0000 7f8bfd600640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8be006b4d0 0x7f8be006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.114+0000 7f8bfd600640 1 -- 172.21.15.67:0/107463085 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8bec089990 con 0x7f8c00153500 2024-03-09T09:14:13.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.115+0000 7f8c05600640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8be006b4d0 0x7f8be006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.116+0000 7f8c05600640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8be006b4d0 0x7f8be006d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4035000 tx=0x7f8bf4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.122+0000 7f8bfd600640 1 -- 172.21.15.67:0/107463085 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8c001516f0 con 0x7f8c00153500 2024-03-09T09:14:13.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.472+0000 7f8c08000640 1 -- 172.21.15.67:0/107463085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8c000cafb0 con 0x7f8c00153500 2024-03-09T09:14:13.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.472+0000 7f8bfd600640 1 -- 172.21.15.67:0/107463085 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f8bec05ce20 con 0x7f8c00153500 2024-03-09T09:14:13.478 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":162,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:14:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 -- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8be006b4d0 msgr2=0x7f8be006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8be006b4d0 0x7f8be006d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4035000 tx=0x7f8bf4030000 comp rx=0 tx=0).stop 2024-03-09T09:14:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 -- 172.21.15.67:0/107463085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c00153500 msgr2=0x7f8c001ef530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c00153500 0x7f8c001ef530 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00c3c0 tx=0x7f8bec00c880 comp rx=0 tx=0).stop 2024-03-09T09:14:13.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 -- 172.21.15.67:0/107463085 shutdown_connections 2024-03-09T09:14:13.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8be006b4d0 0x7f8be006d990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8c00153ff0 0x7f8c001efa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8c00153500 0x7f8c001ef530 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 --2- 172.21.15.67:0/107463085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8c000c7bc0 0x7f8c0014c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 -- 172.21.15.67:0/107463085 >> 172.21.15.67:0/107463085 conn(0x7f8c000c8060 msgr2=0x7f8c0006b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:13.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 -- 172.21.15.67:0/107463085 shutdown_connections 2024-03-09T09:14:13.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.476+0000 7f8c08000640 1 -- 172.21.15.67:0/107463085 wait complete. 2024-03-09T09:14:13.499 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:14:13.499 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:14:13.500 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:14:13.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:14:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.725+0000 7f230fc00640 1 Processor -- start 2024-03-09T09:14:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.725+0000 7f230fc00640 1 -- start start 2024-03-09T09:14:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.726+0000 7f230fc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f230806b040 0x7f2308155f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.726+0000 7f230fc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23081594b0 con 0x7f230806b420 2024-03-09T09:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.726+0000 7f230fc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230815a380 con 0x7f23080c2010 2024-03-09T09:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.726+0000 7f230fc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23081597e0 con 0x7f230806b040 2024-03-09T09:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.727+0000 7f230dc00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f230806b420 0x7f2308153300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42254/0 (socket says 172.21.15.67:42254) 2024-03-09T09:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.727+0000 7f230dc00640 1 -- 172.21.15.67:0/2151822599 learned_addr learned my addr 172.21.15.67:0/2151822599 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:14:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.727+0000 7f230d200640 1 --2- 172.21.15.67:0/2151822599 >> v2:172.21.15.134:3300/0 conn(0x7f230806b040 0x7f2308155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.727+0000 7f2307400640 1 -- 172.21.15.67:0/2151822599 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271439668 0 0) 0x7f23081594b0 con 0x7f230806b420 2024-03-09T09:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.727+0000 7f2307400640 1 -- 172.21.15.67:0/2151822599 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22f0003520 con 0x7f230806b420 2024-03-09T09:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.727+0000 7f230d200640 1 -- 172.21.15.67:0/2151822599 >> v1:172.21.15.134:6789/0 conn(0x7f23080c2010 legacy=0x7f2308159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.727+0000 7f230d200640 1 -- 172.21.15.67:0/2151822599 >> v1:172.21.15.67:6789/0 conn(0x7f230806b420 legacy=0x7f2308153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.727+0000 7f230d200640 1 -- 172.21.15.67:0/2151822599 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23080c2510 con 0x7f230806b040 2024-03-09T09:14:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.729+0000 7f230d200640 1 --2- 172.21.15.67:0/2151822599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f230806b040 0x7f2308155f40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0097a0 tx=0x7f22fc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ac895efa2db57b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.729+0000 7f2307400640 1 -- 172.21.15.67:0/2151822599 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc034070 con 0x7f230806b040 2024-03-09T09:14:13.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.729+0000 7f2307400640 1 -- 172.21.15.67:0/2151822599 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22fc030610 con 0x7f230806b040 2024-03-09T09:14:13.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.729+0000 7f2307400640 1 -- 172.21.15.67:0/2151822599 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc0309c0 con 0x7f230806b040 2024-03-09T09:14:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.730+0000 7f230fc00640 1 -- 172.21.15.67:0/2151822599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f230806b040 msgr2=0x7f2308155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.730+0000 7f230fc00640 1 --2- 172.21.15.67:0/2151822599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f230806b040 0x7f2308155f40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0097a0 tx=0x7f22fc02b960 comp rx=0 tx=0).stop 2024-03-09T09:14:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.730+0000 7f230fc00640 1 -- 172.21.15.67:0/2151822599 shutdown_connections 2024-03-09T09:14:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.730+0000 7f230fc00640 1 --2- 172.21.15.67:0/2151822599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f230806b040 0x7f2308155f40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.730+0000 7f230fc00640 1 -- 172.21.15.67:0/2151822599 >> 172.21.15.67:0/2151822599 conn(0x7f23080c8b20 msgr2=0x7f23080c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:13.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.733+0000 7f230fc00640 1 -- 172.21.15.67:0/2151822599 shutdown_connections 2024-03-09T09:14:13.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.733+0000 7f230fc00640 1 -- 172.21.15.67:0/2151822599 wait complete. 2024-03-09T09:14:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.735+0000 7f230fc00640 1 Processor -- start 2024-03-09T09:14:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.735+0000 7f230fc00640 1 -- start start 2024-03-09T09:14:13.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f230fc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f230806b040 0x7f23080ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f230fc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230806b420 0x7f230814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f230fc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23080c2010 0x7f23081eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f230fc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23080c2ad0 con 0x7f230806b420 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f230fc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230806b8a0 con 0x7f230806b040 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f230fc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23080c1b50 con 0x7f23080c2010 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f2307e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230806b420 0x7f230814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f2307e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230806b420 0x7f230814d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48208/0 (socket says 172.21.15.67:48208) 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f2307e00640 1 -- 172.21.15.67:0/2007277737 learned_addr learned my addr 172.21.15.67:0/2007277737 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f230d200640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f230806b040 0x7f23080ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f230dc00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23080c2010 0x7f23081eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f2307e00640 1 -- 172.21.15.67:0/2007277737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23080c2010 msgr2=0x7f23081eb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f2307e00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23080c2010 0x7f23081eb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f2307e00640 1 -- 172.21.15.67:0/2007277737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f230806b040 msgr2=0x7f23080ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f2307e00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f230806b040 0x7f23080ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.736+0000 7f2307e00640 1 -- 172.21.15.67:0/2007277737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23080abdf0 con 0x7f230806b420 2024-03-09T09:14:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.737+0000 7f2307e00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230806b420 0x7f230814d970 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c440 tx=0x7f22f4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.737+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f40052d0 con 0x7f230806b420 2024-03-09T09:14:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.737+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f4005470 con 0x7f230806b420 2024-03-09T09:14:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.737+0000 7f230fc00640 1 -- 172.21.15.67:0/2007277737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23080ac0d0 con 0x7f230806b420 2024-03-09T09:14:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.737+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f4007260 con 0x7f230806b420 2024-03-09T09:14:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.737+0000 7f230fc00640 1 -- 172.21.15.67:0/2007277737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2308157fa0 con 0x7f230806b420 2024-03-09T09:14:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.738+0000 7f230fc00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230814deb0 0x7f23081eb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.738+0000 7f230fc00640 1 -- 172.21.15.67:0/2007277737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23080c5530 con 0x7f230814deb0 2024-03-09T09:14:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.739+0000 7f230d200640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230814deb0 0x7f23081eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.739+0000 7f230d200640 1 -- 172.21.15.67:0/2007277737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2308075a00 con 0x7f230814deb0 2024-03-09T09:14:13.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.739+0000 7f230d200640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230814deb0 0x7f23081eb810 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f230806c070 tx=0x7f22fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.740+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc034040 con 0x7f230814deb0 2024-03-09T09:14:13.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.740+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f22f4007400 con 0x7f230806b420 2024-03-09T09:14:13.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.741+0000 7f2304c00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22e806b3d0 0x7f22e806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.741+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f22f4089a90 con 0x7f230806b420 2024-03-09T09:14:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.741+0000 7f230dc00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22e806b3d0 0x7f22e806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.741+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f22fc048070 con 0x7f230814deb0 2024-03-09T09:14:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.741+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230814deb0 msgr2=0x7f23081eb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.741+0000 7f2304c00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230814deb0 0x7f23081eb810 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f230806c070 tx=0x7f22fc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.741+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230814deb0 msgr2=0x7f23081eb810 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.741+0000 7f2304c00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230814deb0 0x7f23081eb810 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.742+0000 7f230dc00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22e806b3d0 0x7f22e806d890 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2300005c00 tx=0x7f2300009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.749+0000 7f230fc00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23081ebd50 0x7f23081ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.749+0000 7f230fc00640 1 -- 172.21.15.67:0/2007277737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23080ae7f0 con 0x7f23081ebd50 2024-03-09T09:14:13.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.749+0000 7f230d200640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23081ebd50 0x7f23081ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.749+0000 7f230d200640 1 -- 172.21.15.67:0/2007277737 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2308153620 con 0x7f23081ebd50 2024-03-09T09:14:13.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.750+0000 7f230d200640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23081ebd50 0x7f23081ee880 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2308067980 tx=0x7f22fc031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.750+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc039e60 con 0x7f23081ebd50 2024-03-09T09:14:13.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7f22fc0093c0 con 0x7f23081ebd50 2024-03-09T09:14:13.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23081ebd50 msgr2=0x7f23081ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f2304c00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23081ebd50 0x7f23081ee880 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2308067980 tx=0x7f22fc031000 comp rx=0 tx=0).stop 2024-03-09T09:14:13.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f2304c00640 1 -- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23081ebd50 msgr2=0x7f23081ee880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:13.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f2304c00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23081ebd50 0x7f23081ee880 secure :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2308067980 tx=0x7f22fc031000 comp rx=0 tx=0).stop 2024-03-09T09:14:13.756 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 162, 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:14:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:14:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:14:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:14:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:13.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:13.765 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:13.766 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.767 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.768 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:13.769 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:13.770 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:14:13.770 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.770 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:13.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.770 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:14:13.770 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:14:13.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f230fc00640 1 -- 172.21.15.67:0/2007277737 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22e806b3d0 msgr2=0x7f22e806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f230fc00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22e806b3d0 0x7f22e806d890 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2300005c00 tx=0x7f2300009040 comp rx=0 tx=0).stop 2024-03-09T09:14:13.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f230fc00640 1 -- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230806b420 msgr2=0x7f230814d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f230fc00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230806b420 0x7f230814d970 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c440 tx=0x7f22f4001e70 comp rx=0 tx=0).stop 2024-03-09T09:14:13.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f230dc00640 1 -- 172.21.15.67:0/2007277737 reap_dead start 2024-03-09T09:14:13.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f230fc00640 1 -- 172.21.15.67:0/2007277737 shutdown_connections 2024-03-09T09:14:13.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f230fc00640 1 --2- 172.21.15.67:0/2007277737 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f230806b420 0x7f230814d970 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f230fc00640 1 -- 172.21.15.67:0/2007277737 >> 172.21.15.67:0/2007277737 conn(0x7f23080c8b20 msgr2=0x7f2308153db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:13.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f230fc00640 1 -- 172.21.15.67:0/2007277737 shutdown_connections 2024-03-09T09:14:13.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.751+0000 7f230fc00640 1 -- 172.21.15.67:0/2007277737 wait complete. 2024-03-09T09:14:13.772 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:14:13.773 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:14:13.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:14:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.960+0000 7f2f17c00640 1 Processor -- start 2024-03-09T09:14:13.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.961+0000 7f2f17c00640 1 -- start start 2024-03-09T09:14:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.962+0000 7f2f17c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f1014c9a0 0x7f2f101567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.962+0000 7f2f17c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2f100c1d70 0x7f2f100c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.962+0000 7f2f17c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f100ac4e0 con 0x7f2f100c0ce0 2024-03-09T09:14:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.962+0000 7f2f17c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1006d590 con 0x7f2f100c1d70 2024-03-09T09:14:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.962+0000 7f2f17c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10156d00 con 0x7f2f1014c9a0 2024-03-09T09:14:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.962+0000 7f2f0fe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2f100c1d70 0x7f2f100c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.962+0000 7f2f15200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f1014c9a0 0x7f2f101567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.962+0000 7f2f0fe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2f100c1d70 0x7f2f100c07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45456/0 (socket says 172.21.15.67:45456) 2024-03-09T09:14:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.962+0000 7f2f0fe00640 1 -- 172.21.15.67:0/204602941 learned_addr learned my addr 172.21.15.67:0/204602941 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.963+0000 7f2f0fe00640 1 -- 172.21.15.67:0/204602941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f1014c9a0 msgr2=0x7f2f101567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:14:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.963+0000 7f2f0fe00640 1 --2- 172.21.15.67:0/204602941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f1014c9a0 0x7f2f101567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.963+0000 7f2f0fe00640 1 -- 172.21.15.67:0/204602941 >> v1:172.21.15.134:6789/0 conn(0x7f2f100c0ce0 legacy=0x7f2f100ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.963+0000 7f2f0fe00640 1 -- 172.21.15.67:0/204602941 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f100aa550 con 0x7f2f100c1d70 2024-03-09T09:14:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.963+0000 7f2f15200640 1 --2- 172.21.15.67:0/204602941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f1014c9a0 0x7f2f101567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:14:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.964+0000 7f2f0fe00640 1 --2- 172.21.15.67:0/204602941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f100c1d70 0x7f2f100c07a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009870 tx=0x7f2efc02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80784184ac826aa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.964+0000 7f2f0f400640 1 -- 172.21.15.67:0/204602941 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc035020 con 0x7f2f100c1d70 2024-03-09T09:14:13.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.964+0000 7f2f0f400640 1 -- 172.21.15.67:0/204602941 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc031640 con 0x7f2f100c1d70 2024-03-09T09:14:13.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.964+0000 7f2f0f400640 1 -- 172.21.15.67:0/204602941 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc02d040 con 0x7f2f100c1d70 2024-03-09T09:14:13.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.965+0000 7f2f17c00640 1 -- 172.21.15.67:0/204602941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f100c1d70 msgr2=0x7f2f100c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.965+0000 7f2f17c00640 1 --2- 172.21.15.67:0/204602941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f100c1d70 0x7f2f100c07a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009870 tx=0x7f2efc02b8f0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.965+0000 7f2f17c00640 1 -- 172.21.15.67:0/204602941 shutdown_connections 2024-03-09T09:14:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.965+0000 7f2f17c00640 1 --2- 172.21.15.67:0/204602941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f100c1d70 0x7f2f100c07a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.965+0000 7f2f17c00640 1 --2- 172.21.15.67:0/204602941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f1014c9a0 0x7f2f101567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.965+0000 7f2f17c00640 1 -- 172.21.15.67:0/204602941 >> 172.21.15.67:0/204602941 conn(0x7f2f100c7370 msgr2=0x7f2f100c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.965+0000 7f2f17c00640 1 -- 172.21.15.67:0/204602941 shutdown_connections 2024-03-09T09:14:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.965+0000 7f2f17c00640 1 -- 172.21.15.67:0/204602941 wait complete. 2024-03-09T09:14:13.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.967+0000 7f2f17c00640 1 Processor -- start 2024-03-09T09:14:13.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.967+0000 7f2f17c00640 1 -- start start 2024-03-09T09:14:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.969+0000 7f2f17c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f100c0ce0 0x7f2f10148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.969+0000 7f2f17c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f100c1d70 0x7f2f101eef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.969+0000 7f2f17c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f1014c9a0 0x7f2f100724a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.969+0000 7f2f17c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100aadf0 con 0x7f2f1014c9a0 2024-03-09T09:14:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.969+0000 7f2f17c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100aab10 con 0x7f2f100c1d70 2024-03-09T09:14:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.969+0000 7f2f17c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100ad0d0 con 0x7f2f100c0ce0 2024-03-09T09:14:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.969+0000 7f2f15c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f1014c9a0 0x7f2f100724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.969+0000 7f2f15c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f1014c9a0 0x7f2f100724a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48238/0 (socket says 172.21.15.67:48238) 2024-03-09T09:14:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.969+0000 7f2f15c00640 1 -- 172.21.15.67:0/3008089979 learned_addr learned my addr 172.21.15.67:0/3008089979 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.969+0000 7f2f0fe00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f100c1d70 0x7f2f101eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.969+0000 7f2f15200640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f100c0ce0 0x7f2f10148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.970+0000 7f2f15c00640 1 -- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f100c0ce0 msgr2=0x7f2f10148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.970+0000 7f2f15c00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f100c0ce0 0x7f2f10148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.970+0000 7f2f15c00640 1 -- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f100c1d70 msgr2=0x7f2f101eef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.970+0000 7f2f15c00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f100c1d70 0x7f2f101eef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.970+0000 7f2f15c00640 1 -- 172.21.15.67:0/3008089979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f101df470 con 0x7f2f1014c9a0 2024-03-09T09:14:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.970+0000 7f2f15200640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f100c0ce0 0x7f2f10148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:14:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.970+0000 7f2f15c00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f1014c9a0 0x7f2f100724a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800c7a0 tx=0x7f2f0800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.970+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f08003b70 con 0x7f2f1014c9a0 2024-03-09T09:14:13.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.970+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f08003d30 con 0x7f2f1014c9a0 2024-03-09T09:14:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.970+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800aa30 con 0x7f2f1014c9a0 2024-03-09T09:14:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.970+0000 7f2f17c00640 1 -- 172.21.15.67:0/3008089979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f101efa40 con 0x7f2f1014c9a0 2024-03-09T09:14:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.971+0000 7f2f17c00640 1 -- 172.21.15.67:0/3008089979 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f100735b0 con 0x7f2f1014c9a0 2024-03-09T09:14:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.972+0000 7f2f17c00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10149160 0x7f2f101ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.972+0000 7f2f17c00640 1 -- 172.21.15.67:0/3008089979 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f101e2b10 con 0x7f2f10149160 2024-03-09T09:14:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.972+0000 7f2f15200640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10149160 0x7f2f101ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.973+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2f0800abd0 con 0x7f2f1014c9a0 2024-03-09T09:14:13.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.973+0000 7f2f15200640 1 -- 172.21.15.67:0/3008089979 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f10075a00 con 0x7f2f10149160 2024-03-09T09:14:13.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.974+0000 7f2f15200640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10149160 0x7f2f101ef4d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f04011600 tx=0x7f2f04011ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.974+0000 7f2f0cc00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ef006b500 0x7f2ef006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.974+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f2f0808a3a0 con 0x7f2f1014c9a0 2024-03-09T09:14:13.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.974+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04004220 con 0x7f2f10149160 2024-03-09T09:14:13.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.974+0000 7f2f0fe00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ef006b500 0x7f2ef006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.975+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f2f04004220 con 0x7f2f10149160 2024-03-09T09:14:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.975+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10149160 msgr2=0x7f2f101ef4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.975+0000 7f2f0cc00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10149160 0x7f2f101ef4d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f04011600 tx=0x7f2f04011ac0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.975+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10149160 msgr2=0x7f2f101ef4d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.975+0000 7f2f0cc00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10149160 0x7f2f101ef4d0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f04011600 tx=0x7f2f04011ac0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.975+0000 7f2f0fe00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ef006b500 0x7f2ef006d9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2efc036000 tx=0x7f2efc03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.982+0000 7f2f17c00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10148c10 0x7f2f100729e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.982+0000 7f2f17c00640 1 -- 172.21.15.67:0/3008089979 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10072f20 con 0x7f2f10148c10 2024-03-09T09:14:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.983+0000 7f2f15200640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10148c10 0x7f2f100729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.983+0000 7f2f15200640 1 -- 172.21.15.67:0/3008089979 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2f100c15e0 con 0x7f2f10148c10 2024-03-09T09:14:13.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.983+0000 7f2f15200640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10148c10 0x7f2f100729e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2f04011600 tx=0x7f2f040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f040044b0 con 0x7f2f10148c10 2024-03-09T09:14:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f2f040044b0 con 0x7f2f10148c10 2024-03-09T09:14:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10148c10 msgr2=0x7f2f100729e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f0cc00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10148c10 0x7f2f100729e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2f04011600 tx=0x7f2f040046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f0cc00640 1 -- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10148c10 msgr2=0x7f2f100729e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:13.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f0cc00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f10148c10 0x7f2f100729e0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2f04011600 tx=0x7f2f040046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:13.990 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:14:13.990 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:14:13.990 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:14:13.990 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:14:13.990 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 162, 2024-03-09T09:14:13.990 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:14:13.990 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:14:13.990 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:13.991 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:13.992 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:13.993 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:13.994 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:13.995 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:14:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:14:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:14:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:14:14.001 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:14:14.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f17c00640 1 -- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ef006b500 msgr2=0x7f2ef006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f17c00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2ef006b500 0x7f2ef006d9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2efc036000 tx=0x7f2efc03c040 comp rx=0 tx=0).stop 2024-03-09T09:14:14.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f17c00640 1 -- 172.21.15.67:0/3008089979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f1014c9a0 msgr2=0x7f2f100724a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f17c00640 1 --2- 172.21.15.67:0/3008089979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f1014c9a0 0x7f2f100724a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800c7a0 tx=0x7f2f0800cc60 comp rx=0 tx=0).stop 2024-03-09T09:14:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f15c00640 1 -- 172.21.15.67:0/3008089979 reap_dead start 2024-03-09T09:14:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.984+0000 7f2f15c00640 1 -- 172.21.15.67:0/3008089979 reap_dead start 2024-03-09T09:14:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.985+0000 7f2f17c00640 1 -- 172.21.15.67:0/3008089979 shutdown_connections 2024-03-09T09:14:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.985+0000 7f2f17c00640 1 -- 172.21.15.67:0/3008089979 >> 172.21.15.67:0/3008089979 conn(0x7f2f100c7370 msgr2=0x7f2f100c6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.985+0000 7f2f17c00640 1 -- 172.21.15.67:0/3008089979 shutdown_connections 2024-03-09T09:14:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:13.985+0000 7f2f17c00640 1 -- 172.21.15.67:0/3008089979 wait complete. 2024-03-09T09:14:14.007 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:14:14.007 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:14:14.007 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:14:14.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.214+0000 7f59eae00640 1 Processor -- start 2024-03-09T09:14:14.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.214+0000 7f59eae00640 1 -- start start 2024-03-09T09:14:14.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.215+0000 7f59eae00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f59e414fd90 0x7f59e4156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.215+0000 7f59eae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e40c9380 0x7f59e40acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.215+0000 7f59eae00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e4156bc0 con 0x7f59e40ad440 2024-03-09T09:14:14.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.215+0000 7f59eae00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e406ba60 con 0x7f59e414fd90 2024-03-09T09:14:14.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.215+0000 7f59eae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40c9890 con 0x7f59e40c9380 2024-03-09T09:14:14.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.216+0000 7f59e9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f59e414fd90 0x7f59e4156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.216+0000 7f59e9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f59e414fd90 0x7f59e4156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48254/0 (socket says 172.21.15.67:48254) 2024-03-09T09:14:14.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.216+0000 7f59e9a00640 1 -- 172.21.15.67:0/3569734991 learned_addr learned my addr 172.21.15.67:0/3569734991 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:14.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.216+0000 7f59e9000640 1 --2- 172.21.15.67:0/3569734991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e40c9380 0x7f59e40acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.216+0000 7f59e9a00640 1 -- 172.21.15.67:0/3569734991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e40c9380 msgr2=0x7f59e40acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:14:14.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.216+0000 7f59e9a00640 1 --2- 172.21.15.67:0/3569734991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e40c9380 0x7f59e40acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.216+0000 7f59e9a00640 1 -- 172.21.15.67:0/3569734991 >> v1:172.21.15.67:6789/0 conn(0x7f59e40ad440 legacy=0x7f59e406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.216+0000 7f59e9a00640 1 -- 172.21.15.67:0/3569734991 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40c5f80 con 0x7f59e414fd90 2024-03-09T09:14:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.217+0000 7f59e9a00640 1 --2- 172.21.15.67:0/3569734991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e414fd90 0x7f59e4156680 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009870 tx=0x7f59d802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8326d294c7338c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.218+0000 7f59e3e00640 1 -- 172.21.15.67:0/3569734991 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d80317c0 con 0x7f59e414fd90 2024-03-09T09:14:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.218+0000 7f59e3e00640 1 -- 172.21.15.67:0/3569734991 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d8037070 con 0x7f59e414fd90 2024-03-09T09:14:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.218+0000 7f59e3e00640 1 -- 172.21.15.67:0/3569734991 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8031b80 con 0x7f59e414fd90 2024-03-09T09:14:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.218+0000 7f59eae00640 1 -- 172.21.15.67:0/3569734991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e414fd90 msgr2=0x7f59e4156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.218+0000 7f59eae00640 1 --2- 172.21.15.67:0/3569734991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e414fd90 0x7f59e4156680 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009870 tx=0x7f59d802b860 comp rx=0 tx=0).stop 2024-03-09T09:14:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.218+0000 7f59eae00640 1 -- 172.21.15.67:0/3569734991 shutdown_connections 2024-03-09T09:14:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.218+0000 7f59eae00640 1 --2- 172.21.15.67:0/3569734991 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e40c9380 0x7f59e40acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.218+0000 7f59eae00640 1 --2- 172.21.15.67:0/3569734991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e414fd90 0x7f59e4156680 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.218+0000 7f59eae00640 1 -- 172.21.15.67:0/3569734991 >> 172.21.15.67:0/3569734991 conn(0x7f59e40cb480 msgr2=0x7f59e40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.218+0000 7f59eae00640 1 -- 172.21.15.67:0/3569734991 shutdown_connections 2024-03-09T09:14:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.218+0000 7f59eae00640 1 -- 172.21.15.67:0/3569734991 wait complete. 2024-03-09T09:14:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.220+0000 7f59eae00640 1 Processor -- start 2024-03-09T09:14:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.220+0000 7f59eae00640 1 -- start start 2024-03-09T09:14:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.221+0000 7f59eae00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e40ad440 0x7f59e4158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.221+0000 7f59eae00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e40c9380 0x7f59e40b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.221+0000 7f59eae00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e414fd90 0x7f59e40b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.221+0000 7f59eae00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e406bcb0 con 0x7f59e414fd90 2024-03-09T09:14:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.221+0000 7f59eae00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40c6520 con 0x7f59e40ad440 2024-03-09T09:14:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.221+0000 7f59eae00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e406c8e0 con 0x7f59e40c9380 2024-03-09T09:14:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.221+0000 7f59ea400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e414fd90 0x7f59e40b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.222+0000 7f59ea400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e414fd90 0x7f59e40b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48256/0 (socket says 172.21.15.67:48256) 2024-03-09T09:14:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.222+0000 7f59ea400640 1 -- 172.21.15.67:0/3728464950 learned_addr learned my addr 172.21.15.67:0/3728464950 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.222+0000 7f59e9000640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e40c9380 0x7f59e40b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.222+0000 7f59e9a00640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e40ad440 0x7f59e4158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.222+0000 7f59ea400640 1 -- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e40c9380 msgr2=0x7f59e40b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.222+0000 7f59ea400640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e40c9380 0x7f59e40b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.222+0000 7f59ea400640 1 -- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e40ad440 msgr2=0x7f59e4158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.222+0000 7f59ea400640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e40ad440 0x7f59e4158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.222+0000 7f59ea400640 1 -- 172.21.15.67:0/3728464950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e4156ff0 con 0x7f59e414fd90 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.222+0000 7f59e9a00640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59e40ad440 0x7f59e4158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.223+0000 7f59ea400640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e414fd90 0x7f59e40b0860 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c7d0 tx=0x7f59dc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.223+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc003b70 con 0x7f59e414fd90 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.223+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59dc003d30 con 0x7f59e414fd90 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.223+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc00aa90 con 0x7f59e414fd90 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.223+0000 7f59eae00640 1 -- 172.21.15.67:0/3728464950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e414f2c0 con 0x7f59e414fd90 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.223+0000 7f59eae00640 1 -- 172.21.15.67:0/3728464950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e4157590 con 0x7f59e414fd90 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.224+0000 7f59eae00640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414deb0 0x7f59e40aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.224+0000 7f59eae00640 1 -- 172.21.15.67:0/3728464950 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e404f3c0 con 0x7f59e414deb0 2024-03-09T09:14:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.225+0000 7f59e9a00640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414deb0 0x7f59e40aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.226+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f59dc00ac30 con 0x7f59e414fd90 2024-03-09T09:14:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.226+0000 7f59e9a00640 1 -- 172.21.15.67:0/3728464950 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59e406d2d0 con 0x7f59e414deb0 2024-03-09T09:14:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.226+0000 7f59e9a00640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414deb0 0x7f59e40aae20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f59e41537d0 tx=0x7f59d8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.227+0000 7f59e1600640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59c406b500 0x7f59c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.227+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f59dc08a540 con 0x7f59e414fd90 2024-03-09T09:14:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.227+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d804a070 con 0x7f59e414deb0 2024-03-09T09:14:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.227+0000 7f59e9000640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59c406b500 0x7f59c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.227+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f59d8037040 con 0x7f59e414deb0 2024-03-09T09:14:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.227+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414deb0 msgr2=0x7f59e40aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.227+0000 7f59e1600640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414deb0 0x7f59e40aae20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f59e41537d0 tx=0x7f59d8004730 comp rx=0 tx=0).stop 2024-03-09T09:14:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.227+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414deb0 msgr2=0x7f59e40aae20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.227+0000 7f59e1600640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414deb0 0x7f59e40aae20 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f59e41537d0 tx=0x7f59d8004730 comp rx=0 tx=0).stop 2024-03-09T09:14:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.228+0000 7f59e9000640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59c406b500 0x7f59c406d9c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f59e41ec7b0 tx=0x7f59d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:14.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.235+0000 7f59eae00640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414ed20 0x7f59e40ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.235+0000 7f59eae00640 1 -- 172.21.15.67:0/3728464950 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e4158690 con 0x7f59e414ed20 2024-03-09T09:14:14.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.235+0000 7f59e9a00640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414ed20 0x7f59e40ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.236+0000 7f59e9a00640 1 -- 172.21.15.67:0/3728464950 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f59e4159fd0 con 0x7f59e414ed20 2024-03-09T09:14:14.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.236+0000 7f59e9a00640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414ed20 0x7f59e40ab360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009870 tx=0x7f59d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:14.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.237+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d804a040 con 0x7f59e414ed20 2024-03-09T09:14:14.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.237+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f59d804c2c0 con 0x7f59e414ed20 2024-03-09T09:14:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.237+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414ed20 msgr2=0x7f59e40ab360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.237+0000 7f59e1600640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414ed20 0x7f59e40ab360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009870 tx=0x7f59d80046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.237+0000 7f59e1600640 1 -- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414ed20 msgr2=0x7f59e40ab360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.237+0000 7f59e1600640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59e414ed20 0x7f59e40ab360 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.243 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:14:14.243 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:14:14.243 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:14:14.243 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:14:14.243 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 162, 2024-03-09T09:14:14.243 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:14:14.243 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:14.244 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:14.245 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:14.245 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:14.245 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:14.245 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:14.245 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:14.245 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:14.245 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:14:14.245 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:14:14.245 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:14.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.247 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:14:14.247 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:14:14.247 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:14:14.247 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:14:14.247 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:14:14.247 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:14:14.247 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:14.247 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:14.247 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:14:14.247 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:14:14.247 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:14.248 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:14.249 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:14.249 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:14.249 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:14.249 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:14.249 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:14.249 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:14:14.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.249 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:14:14.249 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.249 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:14:14.249 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:14:14.250 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:14.250 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:14.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.250 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:14.250 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:14:14.250 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:14.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.250 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:14.250 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:14:14.250 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:14:14.251 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:14.252 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:14.253 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:14.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.253 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:14:14.253 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:14:14.253 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:14.253 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:14.253 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:14.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.254 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:14:14.254 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:14:14.254 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:14.254 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:14.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.254 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:14.255 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:14:14.255 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:14.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:14.255 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:14:14.255 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:14.255 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:14.255 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:14.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.256 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:14:14.256 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:14:14.256 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:14.256 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:14.256 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:14.256 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:14.256 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:14.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.256 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:14:14.256 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:14:14.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.257 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:14.257 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:14.257 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:14.257 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:14.257 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:14.257 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:14:14.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:14.257 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:14.257 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:14.258 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:14.258 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:14.258 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:14.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:14.258 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:14:14.258 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:14:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.238+0000 7f59eae00640 1 -- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59c406b500 msgr2=0x7f59c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.238+0000 7f59eae00640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59c406b500 0x7f59c406d9c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f59e41ec7b0 tx=0x7f59d0008040 comp rx=0 tx=0).stop 2024-03-09T09:14:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.238+0000 7f59eae00640 1 -- 172.21.15.67:0/3728464950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e414fd90 msgr2=0x7f59e40b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.238+0000 7f59eae00640 1 --2- 172.21.15.67:0/3728464950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59e414fd90 0x7f59e40b0860 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c7d0 tx=0x7f59dc00cc90 comp rx=0 tx=0).stop 2024-03-09T09:14:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.238+0000 7f59ea400640 1 -- 172.21.15.67:0/3728464950 reap_dead start 2024-03-09T09:14:14.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.238+0000 7f59ea400640 1 -- 172.21.15.67:0/3728464950 reap_dead start 2024-03-09T09:14:14.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.238+0000 7f59eae00640 1 -- 172.21.15.67:0/3728464950 shutdown_connections 2024-03-09T09:14:14.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.238+0000 7f59eae00640 1 -- 172.21.15.67:0/3728464950 >> 172.21.15.67:0/3728464950 conn(0x7f59e40cb480 msgr2=0x7f59e40c6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:14.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.238+0000 7f59eae00640 1 -- 172.21.15.67:0/3728464950 shutdown_connections 2024-03-09T09:14:14.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.238+0000 7f59eae00640 1 -- 172.21.15.67:0/3728464950 wait complete. 2024-03-09T09:14:14.268 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:14:14.268 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T09:14:14.268 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:14:14.268 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T09:14:14.268 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:14:14.269 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:14:14.269 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:14:14.302 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:14:14.302 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:14:14.302 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:14:14.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.535+0000 7efcef400640 1 Processor -- start 2024-03-09T09:14:14.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.535+0000 7efcef400640 1 -- start start 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efcef400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efce814df00 0x7efce80ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efcef400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efce806ca70 0x7efce80c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efcef400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80acc00 0x7efce80c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efcef400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce806e710 con 0x7efce806ca70 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efcef400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce80c3720 con 0x7efce814df00 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efcef400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce806d1e0 con 0x7efce80acc00 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efced400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80acc00 msgr2=0x7efce80c31e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efce7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efce806ca70 0x7efce80c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efced400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80acc00 0x7efce80c31e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efce7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efce806ca70 0x7efce80c2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48258/0 (socket says 172.21.15.67:48258) 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efce7e00640 1 -- 172.21.15.67:0/3800658851 learned_addr learned my addr 172.21.15.67:0/3800658851 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efceca00640 1 --2- 172.21.15.67:0/3800658851 >> v2:172.21.15.134:3300/0 conn(0x7efce814df00 0x7efce80ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efce7e00640 1 -- 172.21.15.67:0/3800658851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80acc00 msgr2=0x7efce80c31e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:14:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.537+0000 7efce7e00640 1 --2- 172.21.15.67:0/3800658851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80acc00 0x7efce80c31e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.538+0000 7efce7e00640 1 -- 172.21.15.67:0/3800658851 >> v2:172.21.15.134:3300/0 conn(0x7efce814df00 msgr2=0x7efce80ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:14:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.538+0000 7efce7e00640 1 --2- 172.21.15.67:0/3800658851 >> v2:172.21.15.134:3300/0 conn(0x7efce814df00 0x7efce80ac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.538+0000 7efce7e00640 1 -- 172.21.15.67:0/3800658851 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce80c4bc0 con 0x7efce806ca70 2024-03-09T09:14:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.538+0000 7efceca00640 1 --2- 172.21.15.67:0/3800658851 >> v2:172.21.15.134:3300/0 conn(0x7efce814df00 0x7efce80ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:14:14.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.539+0000 7efce7e00640 1 --2- 172.21.15.67:0/3800658851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce806ca70 0x7efce80c2ca0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efce806ff20 tx=0x7efcd402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5146c7b436a82fff server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:14.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.539+0000 7efce7400640 1 -- 172.21.15.67:0/3800658851 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4035020 con 0x7efce806ca70 2024-03-09T09:14:14.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.539+0000 7efce7400640 1 -- 172.21.15.67:0/3800658851 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd4031800 con 0x7efce806ca70 2024-03-09T09:14:14.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.539+0000 7efce7400640 1 -- 172.21.15.67:0/3800658851 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4031bd0 con 0x7efce806ca70 2024-03-09T09:14:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.540+0000 7efcef400640 1 -- 172.21.15.67:0/3800658851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce806ca70 msgr2=0x7efce80c2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.540+0000 7efcef400640 1 --2- 172.21.15.67:0/3800658851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce806ca70 0x7efce80c2ca0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efce806ff20 tx=0x7efcd402b8a0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.540+0000 7efcef400640 1 -- 172.21.15.67:0/3800658851 shutdown_connections 2024-03-09T09:14:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.540+0000 7efcef400640 1 --2- 172.21.15.67:0/3800658851 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80acc00 0x7efce80c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.540+0000 7efcef400640 1 --2- 172.21.15.67:0/3800658851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce806ca70 0x7efce80c2ca0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.540+0000 7efcef400640 1 --2- 172.21.15.67:0/3800658851 >> v2:172.21.15.134:3300/0 conn(0x7efce814df00 0x7efce80ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.540+0000 7efcef400640 1 -- 172.21.15.67:0/3800658851 >> 172.21.15.67:0/3800658851 conn(0x7efce80c3d90 msgr2=0x7efce80c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.540+0000 7efcef400640 1 -- 172.21.15.67:0/3800658851 shutdown_connections 2024-03-09T09:14:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.540+0000 7efcef400640 1 -- 172.21.15.67:0/3800658851 wait complete. 2024-03-09T09:14:14.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.542+0000 7efcef400640 1 Processor -- start 2024-03-09T09:14:14.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.542+0000 7efcef400640 1 -- start start 2024-03-09T09:14:14.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efcef400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efce80acc00 0x7efce80c0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efcef400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce814df00 0x7efce8143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efcef400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80c5030 0x7efce8143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efcef400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce806df10 con 0x7efce814df00 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efcef400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce806dc30 con 0x7efce80acc00 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efcef400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce806e1f0 con 0x7efce80c5030 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efced400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80c5030 msgr2=0x7efce8143940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efced400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80c5030 0x7efce8143940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efce7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce814df00 0x7efce8143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efceca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efce80acc00 0x7efce80c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efce7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce814df00 0x7efce8143400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48270/0 (socket says 172.21.15.67:48270) 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efce7e00640 1 -- 172.21.15.67:0/1376190201 learned_addr learned my addr 172.21.15.67:0/1376190201 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efceca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efce80acc00 0x7efce80c0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45540/0 (socket says 172.21.15.67:45540) 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efce7e00640 1 -- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80c5030 msgr2=0x7efce8143940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:14:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.543+0000 7efce7e00640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80c5030 0x7efce8143940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.544+0000 7efce7e00640 1 -- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efce80acc00 msgr2=0x7efce80c0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.544+0000 7efce7e00640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efce80acc00 0x7efce80c0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.544+0000 7efce7e00640 1 -- 172.21.15.67:0/1376190201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce80c2310 con 0x7efce814df00 2024-03-09T09:14:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.544+0000 7efce7e00640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce814df00 0x7efce8143400 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efcd40090b0 tx=0x7efcd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.544+0000 7efce4c00640 1 -- 172.21.15.67:0/1376190201 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4040070 con 0x7efce814df00 2024-03-09T09:14:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.544+0000 7efce4c00640 1 -- 172.21.15.67:0/1376190201 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd4039cf0 con 0x7efce814df00 2024-03-09T09:14:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.544+0000 7efcef400640 1 -- 172.21.15.67:0/1376190201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce81eef20 con 0x7efce814df00 2024-03-09T09:14:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.544+0000 7efce4c00640 1 -- 172.21.15.67:0/1376190201 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4035040 con 0x7efce814df00 2024-03-09T09:14:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.545+0000 7efcef400640 1 -- 172.21.15.67:0/1376190201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce81ef4e0 con 0x7efce814df00 2024-03-09T09:14:14.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.546+0000 7efcef400640 1 -- 172.21.15.67:0/1376190201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce804f430 con 0x7efce814df00 2024-03-09T09:14:14.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.546+0000 7efce4c00640 1 -- 172.21.15.67:0/1376190201 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7efcd404b060 con 0x7efce814df00 2024-03-09T09:14:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.547+0000 7efce4c00640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efcc806b480 0x7efcc806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.548+0000 7efce4c00640 1 -- 172.21.15.67:0/1376190201 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7efcd40abf20 con 0x7efce814df00 2024-03-09T09:14:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.548+0000 7efceca00640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efcc806b480 0x7efcc806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.549+0000 7efceca00640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efcc806b480 0x7efcc806d940 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efcdc006fd0 tx=0x7efcdc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:14.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.552+0000 7efce4c00640 1 -- 172.21.15.67:0/1376190201 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efcd407aa40 con 0x7efce814df00 2024-03-09T09:14:14.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.896+0000 7efcef400640 1 -- 172.21.15.67:0/1376190201 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7efce8147380 con 0x7efce814df00 2024-03-09T09:14:14.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.897+0000 7efce4c00640 1 -- 172.21.15.67:0/1376190201 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7efcd407a340 con 0x7efce814df00 2024-03-09T09:14:14.902 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":162,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:14:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.899+0000 7efcef400640 1 -- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efcc806b480 msgr2=0x7efcc806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.899+0000 7efcef400640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efcc806b480 0x7efcc806d940 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efcdc006fd0 tx=0x7efcdc008040 comp rx=0 tx=0).stop 2024-03-09T09:14:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.900+0000 7efcef400640 1 -- 172.21.15.67:0/1376190201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce814df00 msgr2=0x7efce8143400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.900+0000 7efcef400640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce814df00 0x7efce8143400 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efcd40090b0 tx=0x7efcd40046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.900+0000 7efcef400640 1 -- 172.21.15.67:0/1376190201 shutdown_connections 2024-03-09T09:14:14.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.900+0000 7efcef400640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efcc806b480 0x7efcc806d940 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.900+0000 7efcef400640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efce80c5030 0x7efce8143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.900+0000 7efcef400640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efce814df00 0x7efce8143400 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.900+0000 7efcef400640 1 --2- 172.21.15.67:0/1376190201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efce80acc00 0x7efce80c0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.900+0000 7efcef400640 1 -- 172.21.15.67:0/1376190201 >> 172.21.15.67:0/1376190201 conn(0x7efce80c3d90 msgr2=0x7efce8073450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.900+0000 7efcef400640 1 -- 172.21.15.67:0/1376190201 shutdown_connections 2024-03-09T09:14:14.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:14.900+0000 7efcef400640 1 -- 172.21.15.67:0/1376190201 wait complete. 2024-03-09T09:14:17.933 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:14:18.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.181+0000 7f47c5200640 1 Processor -- start 2024-03-09T09:14:18.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.183+0000 7f47c5200640 1 -- start start 2024-03-09T09:14:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.185+0000 7f47c5200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c0157600 0x7f47c015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.185+0000 7f47c5200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47c00c3600 0x7f47c00ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:18.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.185+0000 7f47c5200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c015af40 con 0x7f47c00abca0 2024-03-09T09:14:18.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.185+0000 7f47c5200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c01579e0 con 0x7f47c00c3600 2024-03-09T09:14:18.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.185+0000 7f47c5200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c006d4c0 con 0x7f47c0157600 2024-03-09T09:14:18.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.186+0000 7f47bea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47c00c3600 0x7f47c00ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:18.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.186+0000 7f47bea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47c00c3600 0x7f47c00ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48286/0 (socket says 172.21.15.67:48286) 2024-03-09T09:14:18.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.186+0000 7f47bea00640 1 -- 172.21.15.67:0/1366588234 learned_addr learned my addr 172.21.15.67:0/1366588234 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:18.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.186+0000 7f47bea00640 1 -- 172.21.15.67:0/1366588234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c0157600 msgr2=0x7f47c015aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T09:14:18.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.186+0000 7f47bea00640 1 --2- 172.21.15.67:0/1366588234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c0157600 0x7f47c015aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:18.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.186+0000 7f47bea00640 1 -- 172.21.15.67:0/1366588234 >> v1:172.21.15.134:6789/0 conn(0x7f47c00abca0 legacy=0x7f47c00aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:14:18.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.186+0000 7f47bea00640 1 -- 172.21.15.67:0/1366588234 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c006d7a0 con 0x7f47c00c3600 2024-03-09T09:14:18.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.190+0000 7f47bea00640 1 --2- 172.21.15.67:0/1366588234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c00c3600 0x7f47c00ab760 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009870 tx=0x7f47ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5376c42fe74209 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:18.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.191+0000 7f47be000640 1 -- 172.21.15.67:0/1366588234 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac035020 con 0x7f47c00c3600 2024-03-09T09:14:18.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.192+0000 7f47be000640 1 -- 172.21.15.67:0/1366588234 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac031870 con 0x7f47c00c3600 2024-03-09T09:14:18.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.192+0000 7f47be000640 1 -- 172.21.15.67:0/1366588234 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac031c40 con 0x7f47c00c3600 2024-03-09T09:14:18.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.193+0000 7f47c5200640 1 -- 172.21.15.67:0/1366588234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c00c3600 msgr2=0x7f47c00ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:18.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.193+0000 7f47c5200640 1 --2- 172.21.15.67:0/1366588234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c00c3600 0x7f47c00ab760 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009870 tx=0x7f47ac02b910 comp rx=0 tx=0).stop 2024-03-09T09:14:18.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.193+0000 7f47c5200640 1 -- 172.21.15.67:0/1366588234 shutdown_connections 2024-03-09T09:14:18.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.193+0000 7f47c5200640 1 --2- 172.21.15.67:0/1366588234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c00c3600 0x7f47c00ab760 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:18.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.193+0000 7f47c5200640 1 --2- 172.21.15.67:0/1366588234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c0157600 0x7f47c015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:18.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.193+0000 7f47c5200640 1 -- 172.21.15.67:0/1366588234 >> 172.21.15.67:0/1366588234 conn(0x7f47c00c7300 msgr2=0x7f47c00c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:18.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.194+0000 7f47c5200640 1 -- 172.21.15.67:0/1366588234 shutdown_connections 2024-03-09T09:14:18.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.194+0000 7f47c5200640 1 -- 172.21.15.67:0/1366588234 wait complete. 2024-03-09T09:14:18.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.197+0000 7f47c5200640 1 Processor -- start 2024-03-09T09:14:18.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.197+0000 7f47c5200640 1 -- start start 2024-03-09T09:14:18.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.199+0000 7f47c5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c00abca0 0x7f47c015b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:18.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.199+0000 7f47c5200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c00c3600 0x7f47c01505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:18.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.199+0000 7f47c5200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c0157600 0x7f47c0150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:18.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.199+0000 7f47c5200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c00aaf10 con 0x7f47c00abca0 2024-03-09T09:14:18.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.199+0000 7f47c5200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c006dd60 con 0x7f47c0157600 2024-03-09T09:14:18.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.199+0000 7f47c5200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c00ab090 con 0x7f47c00c3600 2024-03-09T09:14:18.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.199+0000 7f47bea00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c00c3600 msgr2=0x7f47c01505d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:18.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.199+0000 7f47bea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c00c3600 0x7f47c01505d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:18.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.199+0000 7f47bf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c00abca0 0x7f47c015b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:18.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.199+0000 7f47bf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c00abca0 0x7f47c015b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48290/0 (socket says 172.21.15.67:48290) 2024-03-09T09:14:18.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.199+0000 7f47bf400640 1 -- 172.21.15.67:0/2751005905 learned_addr learned my addr 172.21.15.67:0/2751005905 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.200+0000 7f47bf400640 1 -- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c00c3600 msgr2=0x7f47c01505d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:14:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.200+0000 7f47bf400640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c00c3600 0x7f47c01505d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.200+0000 7f47bf400640 1 -- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c0157600 msgr2=0x7f47c0150b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T09:14:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.200+0000 7f47bfe00640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c0157600 0x7f47c0150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.200+0000 7f47bf400640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c0157600 0x7f47c0150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:18.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.200+0000 7f47bf400640 1 -- 172.21.15.67:0/2751005905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c015cd60 con 0x7f47c00abca0 2024-03-09T09:14:18.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.201+0000 7f47bf400640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c00abca0 0x7f47c015b390 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f47b800ba50 tx=0x7f47b800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:18.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.201+0000 7f47b7400640 1 -- 172.21.15.67:0/2751005905 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8019070 con 0x7f47c00abca0 2024-03-09T09:14:18.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.202+0000 7f47b7400640 1 -- 172.21.15.67:0/2751005905 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b800cc10 con 0x7f47c00abca0 2024-03-09T09:14:18.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.202+0000 7f47b7400640 1 -- 172.21.15.67:0/2751005905 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8014440 con 0x7f47c00abca0 2024-03-09T09:14:18.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.202+0000 7f47c5200640 1 -- 172.21.15.67:0/2751005905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c0073580 con 0x7f47c00abca0 2024-03-09T09:14:18.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.202+0000 7f47c5200640 1 -- 172.21.15.67:0/2751005905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c01e4790 con 0x7f47c00abca0 2024-03-09T09:14:18.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.205+0000 7f47b7400640 1 -- 172.21.15.67:0/2751005905 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f47b801c070 con 0x7f47c00abca0 2024-03-09T09:14:18.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.205+0000 7f47c5200640 1 -- 172.21.15.67:0/2751005905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c004f3c0 con 0x7f47c00abca0 2024-03-09T09:14:18.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.206+0000 7f47b7400640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f47a006b4d0 0x7f47a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.206+0000 7f47b7400640 1 -- 172.21.15.67:0/2751005905 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f47b808a300 con 0x7f47c00abca0 2024-03-09T09:14:18.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.208+0000 7f47bea00640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f47a006b4d0 0x7f47a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.210+0000 7f47bea00640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f47a006b4d0 0x7f47a006d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f47ac036000 tx=0x7f47ac008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:18.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.215+0000 7f47b7400640 1 -- 172.21.15.67:0/2751005905 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f47b8058e00 con 0x7f47c00abca0 2024-03-09T09:14:18.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.547+0000 7f47c5200640 1 -- 172.21.15.67:0/2751005905 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f47c00aa1a0 con 0x7f47c00abca0 2024-03-09T09:14:18.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.547+0000 7f47b7400640 1 -- 172.21.15.67:0/2751005905 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f47b8058700 con 0x7f47c00abca0 2024-03-09T09:14:18.553 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":162,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:14:18.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 -- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f47a006b4d0 msgr2=0x7f47a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:18.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f47a006b4d0 0x7f47a006d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f47ac036000 tx=0x7f47ac008030 comp rx=0 tx=0).stop 2024-03-09T09:14:18.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 -- 172.21.15.67:0/2751005905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c00abca0 msgr2=0x7f47c015b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:18.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c00abca0 0x7f47c015b390 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f47b800ba50 tx=0x7f47b800bf10 comp rx=0 tx=0).stop 2024-03-09T09:14:18.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 -- 172.21.15.67:0/2751005905 shutdown_connections 2024-03-09T09:14:18.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f47a006b4d0 0x7f47a006d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:18.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47c0157600 0x7f47c0150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:18.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47c00c3600 0x7f47c01505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:18.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 --2- 172.21.15.67:0/2751005905 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47c00abca0 0x7f47c015b390 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:18.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 -- 172.21.15.67:0/2751005905 >> 172.21.15.67:0/2751005905 conn(0x7f47c00c7300 msgr2=0x7f47c00c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:18.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 -- 172.21.15.67:0/2751005905 shutdown_connections 2024-03-09T09:14:18.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:18.550+0000 7f47c5200640 1 -- 172.21.15.67:0/2751005905 wait complete. 2024-03-09T09:14:21.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:14:21.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.777+0000 7fc927800640 1 Processor -- start 2024-03-09T09:14:21.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.778+0000 7fc927800640 1 -- start start 2024-03-09T09:14:21.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.779+0000 7fc927800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 0x7fc920152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:21.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.779+0000 7fc927800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc9200aa420 0x7fc9201490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:21.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.779+0000 7fc927800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9200ab3c0 con 0x7fc920149630 2024-03-09T09:14:21.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.779+0000 7fc927800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9200c82d0 con 0x7fc9200aa420 2024-03-09T09:14:21.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.779+0000 7fc927800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200aa8a0 con 0x7fc9200c30f0 2024-03-09T09:14:21.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.779+0000 7fc924e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 msgr2=0x7fc920152a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:21.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.779+0000 7fc924e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 0x7fc920152a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:21.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.779+0000 7fc91fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc9200aa420 0x7fc9201490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:21.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.779+0000 7fc91fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc9200aa420 0x7fc9201490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33512/0 (socket says 172.21.15.67:33512) 2024-03-09T09:14:21.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.779+0000 7fc91fe00640 1 -- 172.21.15.67:0/976907088 learned_addr learned my addr 172.21.15.67:0/976907088 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:21.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.780+0000 7fc91fe00640 1 -- 172.21.15.67:0/976907088 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 msgr2=0x7fc920152a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:14:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.780+0000 7fc91fe00640 1 --2- 172.21.15.67:0/976907088 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 0x7fc920152a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.780+0000 7fc91fe00640 1 -- 172.21.15.67:0/976907088 >> v1:172.21.15.134:6789/0 conn(0x7fc920149630 legacy=0x7fc92006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.780+0000 7fc91fe00640 1 -- 172.21.15.67:0/976907088 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9200aab20 con 0x7fc9200aa420 2024-03-09T09:14:21.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.781+0000 7fc91fe00640 1 --2- 172.21.15.67:0/976907088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200aa420 0x7fc9201490f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009870 tx=0x7fc90c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9add4f27837c8250 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:21.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.781+0000 7fc91f400640 1 -- 172.21.15.67:0/976907088 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c02e070 con 0x7fc9200aa420 2024-03-09T09:14:21.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.781+0000 7fc91f400640 1 -- 172.21.15.67:0/976907088 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c037070 con 0x7fc9200aa420 2024-03-09T09:14:21.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.781+0000 7fc91f400640 1 -- 172.21.15.67:0/976907088 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c031a00 con 0x7fc9200aa420 2024-03-09T09:14:21.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.782+0000 7fc927800640 1 -- 172.21.15.67:0/976907088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200aa420 msgr2=0x7fc9201490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:21.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.782+0000 7fc927800640 1 --2- 172.21.15.67:0/976907088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200aa420 0x7fc9201490f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009870 tx=0x7fc90c02b880 comp rx=0 tx=0).stop 2024-03-09T09:14:21.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.782+0000 7fc927800640 1 -- 172.21.15.67:0/976907088 shutdown_connections 2024-03-09T09:14:21.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.782+0000 7fc927800640 1 --2- 172.21.15.67:0/976907088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200aa420 0x7fc9201490f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:21.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.782+0000 7fc927800640 1 --2- 172.21.15.67:0/976907088 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 0x7fc920152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:21.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.782+0000 7fc927800640 1 -- 172.21.15.67:0/976907088 >> 172.21.15.67:0/976907088 conn(0x7fc9200c7370 msgr2=0x7fc9200c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:21.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.782+0000 7fc927800640 1 -- 172.21.15.67:0/976907088 shutdown_connections 2024-03-09T09:14:21.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.782+0000 7fc927800640 1 -- 172.21.15.67:0/976907088 wait complete. 2024-03-09T09:14:21.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.783+0000 7fc927800640 1 Processor -- start 2024-03-09T09:14:21.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.784+0000 7fc927800640 1 -- start start 2024-03-09T09:14:21.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.785+0000 7fc927800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200aa420 0x7fc9201ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:21.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.785+0000 7fc927800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 0x7fc920144580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:21.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.785+0000 7fc927800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920149630 0x7fc920144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:21.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.785+0000 7fc927800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92006d6c0 con 0x7fc9200aa420 2024-03-09T09:14:21.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.785+0000 7fc927800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200ab0e0 con 0x7fc920149630 2024-03-09T09:14:21.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.785+0000 7fc927800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200ab620 con 0x7fc9200c30f0 2024-03-09T09:14:21.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc91fe00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 msgr2=0x7fc920144580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:21.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc91fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 0x7fc920144580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:21.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc924e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200aa420 0x7fc9201ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc924e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200aa420 0x7fc9201ecfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33514/0 (socket says 172.21.15.67:33514) 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc924e00640 1 -- 172.21.15.67:0/868393360 learned_addr learned my addr 172.21.15.67:0/868393360 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc925800640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920149630 0x7fc920144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc924e00640 1 -- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 msgr2=0x7fc920144580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc924e00640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 0x7fc920144580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc924e00640 1 -- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920149630 msgr2=0x7fc920144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc924e00640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920149630 0x7fc920144ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc924e00640 1 -- 172.21.15.67:0/868393360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9201effd0 con 0x7fc9200aa420 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc925800640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920149630 0x7fc920144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.786+0000 7fc924e00640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200aa420 0x7fc9201ecfe0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc91400b550 tx=0x7fc91400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.787+0000 7fc91cc00640 1 -- 172.21.15.67:0/868393360 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914013020 con 0x7fc9200aa420 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.787+0000 7fc91cc00640 1 -- 172.21.15.67:0/868393360 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc914003de0 con 0x7fc9200aa420 2024-03-09T09:14:21.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.787+0000 7fc91cc00640 1 -- 172.21.15.67:0/868393360 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140052b0 con 0x7fc9200aa420 2024-03-09T09:14:21.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.787+0000 7fc927800640 1 -- 172.21.15.67:0/868393360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc920151340 con 0x7fc9200aa420 2024-03-09T09:14:21.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.787+0000 7fc927800640 1 -- 172.21.15.67:0/868393360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc920151900 con 0x7fc9200aa420 2024-03-09T09:14:21.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.788+0000 7fc927800640 1 -- 172.21.15.67:0/868393360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc920151d30 con 0x7fc9200aa420 2024-03-09T09:14:21.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.789+0000 7fc91cc00640 1 -- 172.21.15.67:0/868393360 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc914016020 con 0x7fc9200aa420 2024-03-09T09:14:21.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.790+0000 7fc91cc00640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc90006b480 0x7fc90006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:21.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.790+0000 7fc91cc00640 1 -- 172.21.15.67:0/868393360 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc91408a310 con 0x7fc9200aa420 2024-03-09T09:14:21.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.791+0000 7fc91fe00640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc90006b480 0x7fc90006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:21.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.792+0000 7fc91fe00640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc90006b480 0x7fc90006d940 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc90c036000 tx=0x7fc90c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:21.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:21.795+0000 7fc91cc00640 1 -- 172.21.15.67:0/868393360 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc91401d020 con 0x7fc9200aa420 2024-03-09T09:14:22.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.097+0000 7fc927800640 1 -- 172.21.15.67:0/868393360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc920075a00 con 0x7fc9200aa420 2024-03-09T09:14:22.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.098+0000 7fc91cc00640 1 -- 172.21.15.67:0/868393360 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fc914058e10 con 0x7fc9200aa420 2024-03-09T09:14:22.103 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":162,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:14:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.100+0000 7fc927800640 1 -- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc90006b480 msgr2=0x7fc90006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.100+0000 7fc927800640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc90006b480 0x7fc90006d940 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc90c036000 tx=0x7fc90c008030 comp rx=0 tx=0).stop 2024-03-09T09:14:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.100+0000 7fc927800640 1 -- 172.21.15.67:0/868393360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200aa420 msgr2=0x7fc9201ecfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.100+0000 7fc927800640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200aa420 0x7fc9201ecfe0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc91400b550 tx=0x7fc91400ba10 comp rx=0 tx=0).stop 2024-03-09T09:14:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.101+0000 7fc927800640 1 -- 172.21.15.67:0/868393360 shutdown_connections 2024-03-09T09:14:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.101+0000 7fc927800640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc90006b480 0x7fc90006d940 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:22.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.101+0000 7fc927800640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc920149630 0x7fc920144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.101+0000 7fc927800640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9200c30f0 0x7fc920144580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.101+0000 7fc927800640 1 --2- 172.21.15.67:0/868393360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9200aa420 0x7fc9201ecfe0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.101+0000 7fc927800640 1 -- 172.21.15.67:0/868393360 >> 172.21.15.67:0/868393360 conn(0x7fc9200c7370 msgr2=0x7fc92006dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.101+0000 7fc927800640 1 -- 172.21.15.67:0/868393360 shutdown_connections 2024-03-09T09:14:22.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:22.101+0000 7fc927800640 1 -- 172.21.15.67:0/868393360 wait complete. 2024-03-09T09:14:24.363 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 23% (45/191) Updating files: 24% (46/191) Updating files: 25% (48/191) Updating files: 26% (50/191) Updating files: 27% (52/191) Updating files: 28% (54/191) Updating files: 29% (56/191) Updating files: 30% (58/191) Updating files: 31% (60/191) Updating files: 32% (62/191) Updating files: 33% (64/191) Updating files: 34% (65/191) Updating files: 35% (67/191) Updating files: 36% (69/191) Updating files: 37% (71/191) Updating files: 38% (73/191) Updating files: 39% (75/191) Updating files: 40% (77/191) Updating files: 41% (79/191) Updating files: 42% (81/191) Updating files: 43% (83/191) Updating files: 44% (85/191) Updating files: 45% (86/191) Updating files: 46% (88/191) Updating files: 47% (90/191) Updating files: 48% (92/191) Updating files: 49% (94/191) Updating files: 49% (95/191) Updating files: 50% (96/191) Updating files: 51% (98/191) Updating files: 52% (100/191) Updating files: 53% (102/191) Updating files: 54% (104/191) Updating files: 55% (106/191) Updating files: 56% (107/191) Updating files: 57% (109/191) Updating files: 58% (111/191) Updating files: 59% (113/191) Updating files: 60% (115/191) Updating files: 61% (117/191) Updating files: 62% (119/191) Updating files: 63% (121/191) Updating files: 64% (123/191) Updating files: 65% (125/191) Updating files: 66% (127/191) Updating files: 67% (128/191) Updating files: 68% (130/191) Updating files: 69% (132/191) Updating files: 70% (134/191) Updating files: 71% (136/191) Updating files: 72% (138/191) Updating files: 73% (140/191) Updating files: 74% (142/191) Updating files: 75% (144/191) Updating files: 76% (146/191) Updating files: 77% (148/191) Updating files: 78% (149/191) Updating files: 79% (151/191) Updating files: 80% (153/191) Updating files: 81% (155/191) Updating files: 82% (157/191) Updating files: 83% (159/191) Updating files: 84% (161/191) Updating files: 85% (163/191) Updating files: 86% (165/191) Updating files: 87% (167/191) Updating files: 88% (169/191) Updating files: 89% (170/191) Updating files: 89% (171/191) Updating files: 90% (172/191) Updating files: 91% (174/191) Updating files: 92% (176/191) Updating files: 93% (178/191) Updating files: 94% (180/191) Updating files: 95% (182/191) Updating files: 96% (184/191) Updating files: 96% (185/191) Updating files: 97% (186/191) Updating files: 98% (188/191) Updating files: 99% (190/191) Updating files: 100% (191/191) Updating files: 100% (191/191), done. 2024-03-09T09:14:24.399 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 17% (26/148) Updating files: 18% (27/148) Updating files: 19% (29/148) Updating files: 20% (30/148) Updating files: 21% (32/148) Updating files: 22% (33/148) Updating files: 23% (35/148) Updating files: 24% (36/148) Updating files: 25% (37/148) Updating files: 26% (39/148) Updating files: 27% (40/148) Updating files: 28% (42/148) Updating files: 29% (43/148) Updating files: 30% (45/148) Updating files: 31% (46/148) Updating files: 32% (48/148) Updating files: 33% (49/148) Updating files: 33% (50/148) Updating files: 34% (51/148) Updating files: 35% (52/148) Updating files: 36% (54/148) Updating files: 37% (55/148) Updating files: 38% (57/148) Updating files: 39% (58/148) Updating files: 40% (60/148) Updating files: 41% (61/148) Updating files: 42% (63/148) Updating files: 43% (64/148) Updating files: 44% (66/148) Updating files: 45% (67/148) Updating files: 46% (69/148) Updating files: 47% (70/148) Updating files: 48% (72/148) Updating files: 49% (73/148) Updating files: 50% (74/148) Updating files: 51% (76/148) Updating files: 52% (77/148) Updating files: 53% (79/148) Updating files: 54% (80/148) Updating files: 55% (82/148) Updating files: 56% (83/148) Updating files: 57% (85/148) Updating files: 58% (86/148) Updating files: 59% (88/148) Updating files: 60% (89/148) Updating files: 61% (91/148) Updating files: 62% (92/148) Updating files: 63% (94/148) Updating files: 64% (95/148) Updating files: 65% (97/148) Updating files: 66% (98/148) Updating files: 67% (100/148) Updating files: 68% (101/148) Updating files: 69% (103/148) Updating files: 70% (104/148) Updating files: 71% (106/148) Updating files: 72% (107/148) Updating files: 73% (109/148) Updating files: 74% (110/148) Updating files: 75% (111/148) Updating files: 76% (113/148) Updating files: 77% (114/148) Updating files: 78% (116/148) Updating files: 79% (117/148) Updating files: 80% (119/148) Updating files: 81% (120/148) Updating files: 82% (122/148) Updating files: 83% (123/148) Updating files: 84% (125/148) Updating files: 85% (126/148) Updating files: 86% (128/148) Updating files: 87% (129/148) Updating files: 88% (131/148) Updating files: 89% (132/148) Updating files: 90% (134/148) Updating files: 91% (135/148) Updating files: 92% (137/148) Updating files: 93% (138/148) Updating files: 94% (140/148) Updating files: 95% (141/148) Updating files: 96% (143/148) Updating files: 97% (144/148) Updating files: 98% (146/148) Updating files: 99% (147/148) Updating files: 100% (148/148) Updating files: 100% (148/148), done. 2024-03-09T09:14:24.472 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at d8ad51ee8ab v0.73 2024-03-09T09:14:24.483 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.73 2024-03-09T09:14:24.487 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 5832e2603c7 v0.72 2024-03-09T09:14:24.512 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.72 2024-03-09T09:14:24.732 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:14:24.732 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 74 -eq 48 ']' 2024-03-09T09:14:24.732 INFO:tasks.workunit.client.1.smithi134.stdout:v0.74 2024-03-09T09:14:24.733 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.74 2024-03-09T09:14:24.733 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.74 2024-03-09T09:14:24.733 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.74 2024-03-09T09:14:24.888 INFO:tasks.workunit.client.0.smithi067.stdout:v0.73 2024-03-09T09:14:24.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:14:24.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 73 -eq 48 ']' 2024-03-09T09:14:24.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.73 2024-03-09T09:14:24.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.73 2024-03-09T09:14:24.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.73 2024-03-09T09:14:25.134 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:14:25.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.343+0000 7f8d77200640 1 Processor -- start 2024-03-09T09:14:25.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.344+0000 7f8d77200640 1 -- start start 2024-03-09T09:14:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.345+0000 7f8d77200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.345+0000 7f8d77200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d78156700 con 0x7f8d781531c0 2024-03-09T09:14:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.345+0000 7f8d77200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d78157530 con 0x7f8d781570d0 2024-03-09T09:14:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.345+0000 7f8d77200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d780afba0 con 0x7f8d780cae50 2024-03-09T09:14:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.345+0000 7f8d75e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8d781531c0 0x7f8d780ae590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54814/0 (socket says 172.21.15.67:54814) 2024-03-09T09:14:25.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.345+0000 7f8d75e00640 1 -- 172.21.15.67:0/1563870611 learned_addr learned my addr 172.21.15.67:0/1563870611 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:14:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.346+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.346+0000 7f8d76800640 1 -- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 msgr2=0x7f8d780c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:14:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.346+0000 7f8d76800640 1 -- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 msgr2=0x7f8d780c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:14:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.346+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:25.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.346+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:25.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.547+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:25.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.548+0000 7f8d76800640 1 -- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 msgr2=0x7f8d780c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:14:25.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.548+0000 7f8d76800640 1 -- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 msgr2=0x7f8d780c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:14:25.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.548+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:25.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.548+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:14:25.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.949+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:25.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.950+0000 7f8d76800640 1 -- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 msgr2=0x7f8d780c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:14:25.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.950+0000 7f8d76800640 1 -- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 msgr2=0x7f8d780c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:14:25.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.950+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:25.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:25.950+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:14:26.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:26.751+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:26.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:26.752+0000 7f8d76800640 1 -- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 msgr2=0x7f8d780c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:14:26.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:26.752+0000 7f8d76800640 1 -- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 msgr2=0x7f8d780c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:14:26.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:26.752+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:26.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:26.752+0000 7f8d76800640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:14:27.368 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 47% (70/148) Updating files: 48% (72/148) Updating files: 49% (73/148) Updating files: 50% (74/148) Updating files: 51% (76/148) Updating files: 52% (77/148) Updating files: 53% (79/148) Updating files: 54% (80/148) Updating files: 55% (82/148) Updating files: 56% (83/148) Updating files: 57% (85/148) Updating files: 58% (86/148) Updating files: 59% (88/148) Updating files: 60% (89/148) Updating files: 61% (91/148) Updating files: 62% (92/148) Updating files: 63% (94/148) Updating files: 64% (95/148) Updating files: 65% (97/148) Updating files: 66% (98/148) Updating files: 67% (100/148) Updating files: 68% (101/148) Updating files: 69% (103/148) Updating files: 70% (104/148) Updating files: 71% (106/148) Updating files: 72% (107/148) Updating files: 73% (109/148) Updating files: 74% (110/148) Updating files: 75% (111/148) Updating files: 76% (113/148) Updating files: 77% (114/148) Updating files: 78% (116/148) Updating files: 79% (117/148) Updating files: 80% (119/148) Updating files: 81% (120/148) Updating files: 82% (122/148) Updating files: 83% (123/148) Updating files: 84% (125/148) Updating files: 85% (126/148) Updating files: 86% (128/148) Updating files: 87% (129/148) Updating files: 88% (131/148) Updating files: 89% (132/148) Updating files: 90% (134/148) Updating files: 91% (135/148) Updating files: 92% (137/148) Updating files: 93% (138/148) Updating files: 94% (140/148) Updating files: 95% (141/148) Updating files: 96% (143/148) Updating files: 97% (144/148) Updating files: 98% (146/148) Updating files: 99% (147/148) Updating files: 100% (148/148) Updating files: 100% (148/148), done. 2024-03-09T09:14:27.420 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at d8ad51ee8ab v0.73 2024-03-09T09:14:27.432 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.73 2024-03-09T09:14:27.510 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 49% (95/192) Updating files: 50% (96/192) Updating files: 51% (98/192) Updating files: 52% (100/192) Updating files: 53% (102/192) Updating files: 54% (104/192) Updating files: 55% (106/192) Updating files: 56% (108/192) Updating files: 57% (110/192) Updating files: 58% (112/192) Updating files: 59% (114/192) Updating files: 60% (116/192) Updating files: 61% (118/192) Updating files: 62% (120/192) Updating files: 63% (121/192) Updating files: 64% (123/192) Updating files: 65% (125/192) Updating files: 66% (127/192) Updating files: 67% (129/192) Updating files: 68% (131/192) Updating files: 69% (133/192) Updating files: 70% (135/192) Updating files: 71% (137/192) Updating files: 72% (139/192) Updating files: 73% (141/192) Updating files: 74% (143/192) Updating files: 75% (144/192) Updating files: 76% (146/192) Updating files: 77% (148/192) Updating files: 78% (150/192) Updating files: 79% (152/192) Updating files: 80% (154/192) Updating files: 81% (156/192) Updating files: 81% (157/192) Updating files: 82% (158/192) Updating files: 83% (160/192) Updating files: 84% (162/192) Updating files: 85% (164/192) Updating files: 86% (166/192) Updating files: 87% (168/192) Updating files: 88% (169/192) Updating files: 89% (171/192) Updating files: 90% (173/192) Updating files: 91% (175/192) Updating files: 92% (177/192) Updating files: 93% (179/192) Updating files: 94% (181/192) Updating files: 95% (183/192) Updating files: 96% (185/192) Updating files: 97% (187/192) Updating files: 98% (189/192) Updating files: 99% (191/192) Updating files: 100% (192/192) Updating files: 100% (192/192), done. 2024-03-09T09:14:27.557 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at c165483bc72 v0.74 2024-03-09T09:14:27.568 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.74 2024-03-09T09:14:27.755 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:14:27.755 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 75 -eq 48 ']' 2024-03-09T09:14:27.755 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.75 2024-03-09T09:14:27.755 INFO:tasks.workunit.client.1.smithi134.stdout:v0.75 2024-03-09T09:14:27.756 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.75 2024-03-09T09:14:27.756 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.75 2024-03-09T09:14:27.902 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:14:27.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 74 -eq 48 ']' 2024-03-09T09:14:27.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.74 2024-03-09T09:14:27.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.74 2024-03-09T09:14:27.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.74 2024-03-09T09:14:27.903 INFO:tasks.workunit.client.0.smithi067.stdout:v0.74 2024-03-09T09:14:28.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.345+0000 7f8d6f400640 1 -- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 msgr2=0x7f8d780c9b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:14:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.345+0000 7f8d6f400640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.134:3300/0 conn(0x7f8d780cae50 0x7f8d780c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.345+0000 7f8d6f400640 1 -- 172.21.15.67:0/1563870611 >> v1:172.21.15.134:6789/0 conn(0x7f8d781570d0 legacy=0x7f8d780c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.345+0000 7f8d6f400640 1 -- 172.21.15.67:0/1563870611 >> v1:172.21.15.67:6789/0 conn(0x7f8d781531c0 legacy=0x7f8d780ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.345+0000 7f8d6f400640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.67:3300/0 conn(0x7f8d580036c0 0x7f8d58005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.346+0000 7f8d6f400640 1 --2- 172.21.15.67:0/1563870611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d58006160 0x7f8d58008600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.346+0000 7f8d6f400640 1 -- 172.21.15.67:0/1563870611 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d58008d00 con 0x7f8d580036c0 2024-03-09T09:14:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.346+0000 7f8d6f400640 1 -- 172.21.15.67:0/1563870611 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d58008eb0 con 0x7f8d58006160 2024-03-09T09:14:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.346+0000 7f8d75e00640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.67:3300/0 conn(0x7f8d580036c0 0x7f8d58005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.346+0000 7f8d75400640 1 -- 172.21.15.67:0/1563870611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d58006160 msgr2=0x7f8d58008600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.346+0000 7f8d75400640 1 --2- 172.21.15.67:0/1563870611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d58006160 0x7f8d58008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.346+0000 7f8d75e00640 1 -- 172.21.15.67:0/1563870611 >> v2:172.21.15.67:3300/0 conn(0x7f8d580036c0 msgr2=0x7f8d58005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:14:28.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.346+0000 7f8d75e00640 1 -- 172.21.15.67:0/1563870611 >> v2:172.21.15.67:3300/0 conn(0x7f8d580036c0 msgr2=0x7f8d58005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:14:28.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.346+0000 7f8d75e00640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.67:3300/0 conn(0x7f8d580036c0 0x7f8d58005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:28.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.347+0000 7f8d75e00640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.67:3300/0 conn(0x7f8d580036c0 0x7f8d58005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:28.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.547+0000 7f8d75400640 1 -- 172.21.15.67:0/1563870611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d58006160 msgr2=0x7f8d58008600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:28.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.547+0000 7f8d75400640 1 --2- 172.21.15.67:0/1563870611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d58006160 0x7f8d58008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:14:28.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.557+0000 7f8d75e00640 1 --2- 172.21.15.67:0/1563870611 >> v2:172.21.15.67:3300/0 conn(0x7f8d580036c0 0x7f8d58005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:28.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.558+0000 7f8d75e00640 1 -- 172.21.15.67:0/1563870611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d58006160 msgr2=0x7f8d58008600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:14:28.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.558+0000 7f8d75e00640 1 --2- 172.21.15.67:0/1563870611 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d58006160 0x7f8d58008600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:28.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.559+0000 7f8d75e00640 1 -- 172.21.15.67:0/1563870611 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d781569a0 con 0x7f8d580036c0 2024-03-09T09:14:28.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.561+0000 7f8d75e00640 1 --2- 172.21.15.67:0/1563870611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d580036c0 0x7f8d58005b20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007d20 tx=0x7f8d6802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180651d12db2355b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:28.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.563+0000 7f8d74a00640 1 -- 172.21.15.67:0/1563870611 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68035020 con 0x7f8d580036c0 2024-03-09T09:14:28.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.563+0000 7f8d74a00640 1 -- 172.21.15.67:0/1563870611 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d680318b0 con 0x7f8d580036c0 2024-03-09T09:14:28.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.563+0000 7f8d74a00640 1 -- 172.21.15.67:0/1563870611 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68031ca0 con 0x7f8d580036c0 2024-03-09T09:14:28.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.564+0000 7f8d77200640 1 -- 172.21.15.67:0/1563870611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d580036c0 msgr2=0x7f8d58005b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:28.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.564+0000 7f8d77200640 1 --2- 172.21.15.67:0/1563870611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d580036c0 0x7f8d58005b20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007d20 tx=0x7f8d6802b930 comp rx=0 tx=0).stop 2024-03-09T09:14:28.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.564+0000 7f8d76800640 1 -- 172.21.15.67:0/1563870611 reap_dead start 2024-03-09T09:14:28.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.564+0000 7f8d77200640 1 -- 172.21.15.67:0/1563870611 shutdown_connections 2024-03-09T09:14:28.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.564+0000 7f8d77200640 1 --2- 172.21.15.67:0/1563870611 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d580036c0 0x7f8d58005b20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:28.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.564+0000 7f8d77200640 1 -- 172.21.15.67:0/1563870611 >> 172.21.15.67:0/1563870611 conn(0x7f8d780ad190 msgr2=0x7f8d780ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:28.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.564+0000 7f8d77200640 1 -- 172.21.15.67:0/1563870611 shutdown_connections 2024-03-09T09:14:28.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.564+0000 7f8d77200640 1 -- 172.21.15.67:0/1563870611 wait complete. 2024-03-09T09:14:28.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.567+0000 7f8d77200640 1 Processor -- start 2024-03-09T09:14:28.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.567+0000 7f8d77200640 1 -- start start 2024-03-09T09:14:28.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.568+0000 7f8d77200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d580036c0 0x7f8d7814b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:28.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.568+0000 7f8d77200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d781ef520 0x7f8d780ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:28.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.568+0000 7f8d77200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78149ec0 0x7f8d7814a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:28.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.568+0000 7f8d77200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780b06f0 con 0x7f8d580036c0 2024-03-09T09:14:28.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.568+0000 7f8d77200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780b01c0 con 0x7f8d781ef520 2024-03-09T09:14:28.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.568+0000 7f8d77200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d781535a0 con 0x7f8d78149ec0 2024-03-09T09:14:28.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.568+0000 7f8d76800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78149ec0 msgr2=0x7f8d7814a350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:28.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.568+0000 7f8d76800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78149ec0 0x7f8d7814a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:28.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.568+0000 7f8d75e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d580036c0 0x7f8d7814b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:28.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.569+0000 7f8d75400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d781ef520 0x7f8d780ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:28.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.569+0000 7f8d75e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d580036c0 0x7f8d7814b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33540/0 (socket says 172.21.15.67:33540) 2024-03-09T09:14:28.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.569+0000 7f8d75e00640 1 -- 172.21.15.67:0/778176235 learned_addr learned my addr 172.21.15.67:0/778176235 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.569+0000 7f8d75400640 1 -- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78149ec0 msgr2=0x7f8d7814a350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:14:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.569+0000 7f8d75400640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78149ec0 0x7f8d7814a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.569+0000 7f8d75400640 1 -- 172.21.15.67:0/778176235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d580036c0 msgr2=0x7f8d7814b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.569+0000 7f8d75400640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d580036c0 0x7f8d7814b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.569+0000 7f8d75400640 1 -- 172.21.15.67:0/778176235 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d78153910 con 0x7f8d781ef520 2024-03-09T09:14:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.569+0000 7f8d75400640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d781ef520 0x7f8d780ac230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400c3e0 tx=0x7f8d6400c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.570+0000 7f8d6e000640 1 -- 172.21.15.67:0/778176235 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64017070 con 0x7f8d781ef520 2024-03-09T09:14:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.570+0000 7f8d6e000640 1 -- 172.21.15.67:0/778176235 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d640074e0 con 0x7f8d781ef520 2024-03-09T09:14:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.570+0000 7f8d6e000640 1 -- 172.21.15.67:0/778176235 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640059e0 con 0x7f8d781ef520 2024-03-09T09:14:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.570+0000 7f8d77200640 1 -- 172.21.15.67:0/778176235 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7814b850 con 0x7f8d781ef520 2024-03-09T09:14:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.570+0000 7f8d77200640 1 -- 172.21.15.67:0/778176235 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d781eed20 con 0x7f8d781ef520 2024-03-09T09:14:28.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.571+0000 7f8d77200640 1 -- 172.21.15.67:0/778176235 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d780b1780 con 0x7f8d781ef520 2024-03-09T09:14:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.574+0000 7f8d6e000640 1 -- 172.21.15.67:0/778176235 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8d6401a070 con 0x7f8d781ef520 2024-03-09T09:14:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.574+0000 7f8d6e000640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d5806c4b0 0x7f8d5806e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.575+0000 7f8d6e000640 1 -- 172.21.15.67:0/778176235 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8d64089940 con 0x7f8d781ef520 2024-03-09T09:14:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.575+0000 7f8d75e00640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d5806c4b0 0x7f8d5806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:28.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.576+0000 7f8d75e00640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d5806c4b0 0x7f8d5806e970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8d68001030 tx=0x7f8d68009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:28.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.584+0000 7f8d6e000640 1 -- 172.21.15.67:0/778176235 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8d780b1780 con 0x7f8d781ef520 2024-03-09T09:14:28.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.956+0000 7f8d77200640 1 -- 172.21.15.67:0/778176235 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8d7804f3c0 con 0x7f8d781ef520 2024-03-09T09:14:28.962 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":166,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:14:29.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.957+0000 7f8d6e000640 1 -- 172.21.15.67:0/778176235 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f8d64057ce0 con 0x7f8d781ef520 2024-03-09T09:14:29.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.959+0000 7f8d77200640 1 -- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d5806c4b0 msgr2=0x7f8d5806e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.959+0000 7f8d77200640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d5806c4b0 0x7f8d5806e970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8d68001030 tx=0x7f8d68009030 comp rx=0 tx=0).stop 2024-03-09T09:14:29.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.959+0000 7f8d77200640 1 -- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d781ef520 msgr2=0x7f8d780ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.959+0000 7f8d77200640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d781ef520 0x7f8d780ac230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400c3e0 tx=0x7f8d6400c8a0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.959+0000 7f8d77200640 1 -- 172.21.15.67:0/778176235 shutdown_connections 2024-03-09T09:14:29.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.959+0000 7f8d77200640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8d5806c4b0 0x7f8d5806e970 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.959+0000 7f8d77200640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78149ec0 0x7f8d7814a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.959+0000 7f8d77200640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d781ef520 0x7f8d780ac230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.959+0000 7f8d77200640 1 --2- 172.21.15.67:0/778176235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d580036c0 0x7f8d7814b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.960+0000 7f8d77200640 1 -- 172.21.15.67:0/778176235 >> 172.21.15.67:0/778176235 conn(0x7f8d780ad190 msgr2=0x7f8d781503c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:29.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.960+0000 7f8d77200640 1 -- 172.21.15.67:0/778176235 shutdown_connections 2024-03-09T09:14:29.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:28.960+0000 7f8d77200640 1 -- 172.21.15.67:0/778176235 wait complete. 2024-03-09T09:14:29.110 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:14:29.110 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:14:29.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:14:29.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.340+0000 7fb52b200640 1 Processor -- start 2024-03-09T09:14:29.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.341+0000 7fb52b200640 1 -- start start 2024-03-09T09:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.342+0000 7fb52b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb52c0c9380 0x7fb52c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.342+0000 7fb52b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb52c0ad440 0x7fb52c06c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.342+0000 7fb52b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c0c9890 con 0x7fb52c14fd90 2024-03-09T09:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.342+0000 7fb52b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c06ba60 con 0x7fb52c0ad440 2024-03-09T09:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.342+0000 7fb52b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0c5f80 con 0x7fb52c0c9380 2024-03-09T09:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.343+0000 7fb529400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb52c0c9380 msgr2=0x7fb52c0acf00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.343+0000 7fb529400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb52c0c9380 0x7fb52c0acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.343+0000 7fb52a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb52c0ad440 0x7fb52c06c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.343+0000 7fb52a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb52c0ad440 0x7fb52c06c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41440/0 (socket says 172.21.15.67:41440) 2024-03-09T09:14:29.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.343+0000 7fb52a800640 1 -- 172.21.15.67:0/1830770827 learned_addr learned my addr 172.21.15.67:0/1830770827 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.343+0000 7fb52a800640 1 -- 172.21.15.67:0/1830770827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb52c0c9380 msgr2=0x7fb52c0acf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:14:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.343+0000 7fb52a800640 1 --2- 172.21.15.67:0/1830770827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb52c0c9380 0x7fb52c0acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.343+0000 7fb52a800640 1 -- 172.21.15.67:0/1830770827 >> v1:172.21.15.67:6789/0 conn(0x7fb52c14fd90 legacy=0x7fb52c156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.343+0000 7fb52a800640 1 -- 172.21.15.67:0/1830770827 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c0c6240 con 0x7fb52c0ad440 2024-03-09T09:14:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.345+0000 7fb52a800640 1 --2- 172.21.15.67:0/1830770827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c0ad440 0x7fb52c06c1d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb52c06bbe0 tx=0x7fb52402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d529d9938872ca47 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.345+0000 7fb528a00640 1 -- 172.21.15.67:0/1830770827 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52402c9e0 con 0x7fb52c0ad440 2024-03-09T09:14:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.345+0000 7fb528a00640 1 -- 172.21.15.67:0/1830770827 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52402cb80 con 0x7fb52c0ad440 2024-03-09T09:14:29.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.345+0000 7fb528a00640 1 -- 172.21.15.67:0/1830770827 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5240343f0 con 0x7fb52c0ad440 2024-03-09T09:14:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.346+0000 7fb52b200640 1 -- 172.21.15.67:0/1830770827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c0ad440 msgr2=0x7fb52c06c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.346+0000 7fb52b200640 1 --2- 172.21.15.67:0/1830770827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c0ad440 0x7fb52c06c1d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb52c06bbe0 tx=0x7fb52402be40 comp rx=0 tx=0).stop 2024-03-09T09:14:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.346+0000 7fb52b200640 1 -- 172.21.15.67:0/1830770827 shutdown_connections 2024-03-09T09:14:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.346+0000 7fb52b200640 1 --2- 172.21.15.67:0/1830770827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c0ad440 0x7fb52c06c1d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.346+0000 7fb52b200640 1 --2- 172.21.15.67:0/1830770827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb52c0c9380 0x7fb52c0acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.346+0000 7fb52b200640 1 -- 172.21.15.67:0/1830770827 >> 172.21.15.67:0/1830770827 conn(0x7fb52c0cb480 msgr2=0x7fb52c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:29.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.346+0000 7fb52b200640 1 -- 172.21.15.67:0/1830770827 shutdown_connections 2024-03-09T09:14:29.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.346+0000 7fb52b200640 1 -- 172.21.15.67:0/1830770827 wait complete. 2024-03-09T09:14:29.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.348+0000 7fb52b200640 1 Processor -- start 2024-03-09T09:14:29.370 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:14:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:14:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:14:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:14:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 166, 2024-03-09T09:14:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:14:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:14:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:14:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:29.370 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:14:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:14:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:14:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:29.371 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:29.372 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:14:29.373 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:14:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:14:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:14:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:14:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:14:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:14:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:14:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:14:29.374 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:29.375 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:14:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:14:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:14:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:29.379 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:14:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:14:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:29.380 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:14:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:14:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:14:29.381 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:29.382 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:14:29.383 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:14:29.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:14:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.385 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:14:29.385 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:14:29.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.348+0000 7fb52b200640 1 -- start start 2024-03-09T09:14:29.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.349+0000 7fb52b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb52c0ad440 0x7fb52c14ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.349+0000 7fb52b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c0c9380 0x7fb52c1ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.349+0000 7fb52b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb52c14fd90 0x7fb52c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.349+0000 7fb52b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c06c710 con 0x7fb52c0c9380 2024-03-09T09:14:29.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.349+0000 7fb52b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c06bcb0 con 0x7fb52c0ad440 2024-03-09T09:14:29.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.349+0000 7fb52b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c06c890 con 0x7fb52c14fd90 2024-03-09T09:14:29.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.350+0000 7fb52a800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb52c14fd90 msgr2=0x7fb52c1596f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.349+0000 7fb529400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c0c9380 0x7fb52c1ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.350+0000 7fb52a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb52c14fd90 0x7fb52c1596f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.350+0000 7fb529400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c0c9380 0x7fb52c1ebb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41454/0 (socket says 172.21.15.67:41454) 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.350+0000 7fb529400640 1 -- 172.21.15.67:0/2243152371 learned_addr learned my addr 172.21.15.67:0/2243152371 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.350+0000 7fb529e00640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb52c0ad440 0x7fb52c14ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.350+0000 7fb529400640 1 -- 172.21.15.67:0/2243152371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb52c14fd90 msgr2=0x7fb52c1596f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.350+0000 7fb529400640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb52c14fd90 0x7fb52c1596f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.350+0000 7fb529400640 1 -- 172.21.15.67:0/2243152371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb52c0ad440 msgr2=0x7fb52c14ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.350+0000 7fb529400640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb52c0ad440 0x7fb52c14ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.350+0000 7fb529400640 1 -- 172.21.15.67:0/2243152371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c159240 con 0x7fb52c0c9380 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.350+0000 7fb529400640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c0c9380 0x7fb52c1ebb00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb51800b500 tx=0x7fb51800b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.351+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518003dc0 con 0x7fb52c0c9380 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.351+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb518037070 con 0x7fb52c0c9380 2024-03-09T09:14:29.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.351+0000 7fb52b200640 1 -- 172.21.15.67:0/2243152371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c158d10 con 0x7fb52c0c9380 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.351+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518005290 con 0x7fb52c0c9380 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.351+0000 7fb52b200640 1 -- 172.21.15.67:0/2243152371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c1ec040 con 0x7fb52c0c9380 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.352+0000 7fb52b200640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14c7a0 0x7fb52c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.352+0000 7fb52b200640 1 -- 172.21.15.67:0/2243152371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c153440 con 0x7fb52c14c7a0 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.353+0000 7fb529e00640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14c7a0 0x7fb52c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.353+0000 7fb529e00640 1 -- 172.21.15.67:0/2243152371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb52c158ac0 con 0x7fb52c14c7a0 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.353+0000 7fb529e00640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14c7a0 0x7fb52c0af4a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e370 tx=0x7fb51c00e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.354+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c019070 con 0x7fb52c14c7a0 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.354+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb518005430 con 0x7fb52c0c9380 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.355+0000 7fb522000640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb50c06b3d0 0x7fb50c06d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.355+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb5180ac360 con 0x7fb52c0c9380 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.356+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb51c0052e0 con 0x7fb52c14c7a0 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.356+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14c7a0 msgr2=0x7fb52c0af4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.356+0000 7fb522000640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14c7a0 0x7fb52c0af4a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e370 tx=0x7fb51c00e830 comp rx=0 tx=0).stop 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.356+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14c7a0 msgr2=0x7fb52c0af4a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.356+0000 7fb522000640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14c7a0 0x7fb52c0af4a0 secure :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e370 tx=0x7fb51c00e830 comp rx=0 tx=0).stop 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.356+0000 7fb52a800640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb50c06b3d0 0x7fb50c06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.357+0000 7fb52a800640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb50c06b3d0 0x7fb50c06d890 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb524035000 tx=0x7fb524030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.363+0000 7fb52b200640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14ba50 0x7fb52c14be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.363+0000 7fb52b200640 1 -- 172.21.15.67:0/2243152371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c04f3c0 con 0x7fb52c14ba50 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.363+0000 7fb529e00640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14ba50 0x7fb52c14be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.364+0000 7fb529e00640 1 -- 172.21.15.67:0/2243152371 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb52c06d2d0 con 0x7fb52c14ba50 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.364+0000 7fb529e00640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14ba50 0x7fb52c14be30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e370 tx=0x7fb51c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.364+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c060e60 con 0x7fb52c14ba50 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.364+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7fb51c007020 con 0x7fb52c14ba50 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.364+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14ba50 msgr2=0x7fb52c14be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.364+0000 7fb522000640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14ba50 0x7fb52c14be30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e370 tx=0x7fb51c000ed0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.364+0000 7fb522000640 1 -- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14ba50 msgr2=0x7fb52c14be30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.364+0000 7fb522000640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c14ba50 0x7fb52c14be30 secure :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e370 tx=0x7fb51c000ed0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.365+0000 7fb52b200640 1 -- 172.21.15.67:0/2243152371 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb50c06b3d0 msgr2=0x7fb50c06d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.365+0000 7fb52b200640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb50c06b3d0 0x7fb50c06d890 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb524035000 tx=0x7fb524030000 comp rx=0 tx=0).stop 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.365+0000 7fb52b200640 1 -- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c0c9380 msgr2=0x7fb52c1ebb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.365+0000 7fb52b200640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c0c9380 0x7fb52c1ebb00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb51800b500 tx=0x7fb51800b9c0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.365+0000 7fb52a800640 1 -- 172.21.15.67:0/2243152371 reap_dead start 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.365+0000 7fb52b200640 1 -- 172.21.15.67:0/2243152371 shutdown_connections 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.365+0000 7fb52b200640 1 --2- 172.21.15.67:0/2243152371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb52c0c9380 0x7fb52c1ebb00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.365+0000 7fb52b200640 1 -- 172.21.15.67:0/2243152371 >> 172.21.15.67:0/2243152371 conn(0x7fb52c0cb480 msgr2=0x7fb52c155020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.365+0000 7fb52b200640 1 -- 172.21.15.67:0/2243152371 shutdown_connections 2024-03-09T09:14:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.365+0000 7fb52b200640 1 -- 172.21.15.67:0/2243152371 wait complete. 2024-03-09T09:14:29.398 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:14:29.398 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:14:29.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:14:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.645+0000 7f2512e00640 1 Processor -- start 2024-03-09T09:14:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.646+0000 7f2512e00640 1 -- start start 2024-03-09T09:14:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.647+0000 7f2512e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f250c0ac930 0x7f250c0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.648+0000 7f2512e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c151360 0x7f250c1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.648+0000 7f2512e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f250c0c2780 con 0x7f250c0c30a0 2024-03-09T09:14:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.648+0000 7f2512e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250c070670 con 0x7f250c0ac930 2024-03-09T09:14:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.648+0000 7f2512e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0708c0 con 0x7f250c151360 2024-03-09T09:14:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.648+0000 7f2512400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c151360 msgr2=0x7f250c1539e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.648+0000 7f2512400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c151360 0x7f250c1539e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.648+0000 7f2511a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f250c0ac930 0x7f250c0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.648+0000 7f2511a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f250c0ac930 0x7f250c0c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:56856/0 (socket says 172.21.15.67:56856) 2024-03-09T09:14:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.649+0000 7f2511a00640 1 -- 172.21.15.67:0/1848785230 learned_addr learned my addr 172.21.15.67:0/1848785230 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.649+0000 7f2511a00640 1 -- 172.21.15.67:0/1848785230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c151360 msgr2=0x7f250c1539e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:14:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.649+0000 7f2511a00640 1 --2- 172.21.15.67:0/1848785230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c151360 0x7f250c1539e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.649+0000 7f2511a00640 1 -- 172.21.15.67:0/1848785230 >> v1:172.21.15.134:6789/0 conn(0x7f250c0c30a0 legacy=0x7f250c150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.649+0000 7f2511a00640 1 -- 172.21.15.67:0/1848785230 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c070ba0 con 0x7f250c0ac930 2024-03-09T09:14:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.651+0000 7f2511a00640 1 --2- 172.21.15.67:0/1848785230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c0ac930 0x7f250c0c3660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2500009870 tx=0x7f250002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1580fe403964d7db server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:29.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.752+0000 7f250be00640 1 -- 172.21.15.67:0/1848785230 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500034070 con 0x7f250c0ac930 2024-03-09T09:14:29.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.752+0000 7f250be00640 1 -- 172.21.15.67:0/1848785230 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2500030860 con 0x7f250c0ac930 2024-03-09T09:14:29.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.752+0000 7f250be00640 1 -- 172.21.15.67:0/1848785230 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500030c30 con 0x7f250c0ac930 2024-03-09T09:14:29.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.753+0000 7f2512e00640 1 -- 172.21.15.67:0/1848785230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c0ac930 msgr2=0x7f250c0c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.753+0000 7f2512e00640 1 --2- 172.21.15.67:0/1848785230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c0ac930 0x7f250c0c3660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2500009870 tx=0x7f250002b9a0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.754+0000 7f2512e00640 1 -- 172.21.15.67:0/1848785230 shutdown_connections 2024-03-09T09:14:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.754+0000 7f2512e00640 1 --2- 172.21.15.67:0/1848785230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c151360 0x7f250c1539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.754+0000 7f2512e00640 1 --2- 172.21.15.67:0/1848785230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c0ac930 0x7f250c0c3660 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.754+0000 7f2512e00640 1 -- 172.21.15.67:0/1848785230 >> 172.21.15.67:0/1848785230 conn(0x7f250c0c7220 msgr2=0x7f250c0c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.755+0000 7f2512e00640 1 -- 172.21.15.67:0/1848785230 shutdown_connections 2024-03-09T09:14:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.755+0000 7f2512e00640 1 -- 172.21.15.67:0/1848785230 wait complete. 2024-03-09T09:14:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.757+0000 7f2512e00640 1 Processor -- start 2024-03-09T09:14:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.759+0000 7f2512e00640 1 -- start start 2024-03-09T09:14:29.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.760+0000 7f2512e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f250c0ac930 0x7f250c0a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.760+0000 7f2512e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c0c30a0 0x7f250c155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.760+0000 7f2512e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c151360 0x7f250c1464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.760+0000 7f2512e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0ace90 con 0x7f250c0ac930 2024-03-09T09:14:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.760+0000 7f2512e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0acd10 con 0x7f250c0c30a0 2024-03-09T09:14:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.760+0000 7f2512e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250c153f20 con 0x7f250c151360 2024-03-09T09:14:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.761+0000 7f2511000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c0c30a0 0x7f250c155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.761+0000 7f2511000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c0c30a0 0x7f250c155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:56864/0 (socket says 172.21.15.67:56864) 2024-03-09T09:14:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.761+0000 7f2511000640 1 -- 172.21.15.67:0/2196348867 learned_addr learned my addr 172.21.15.67:0/2196348867 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.761+0000 7f2512400640 1 -- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c151360 msgr2=0x7f250c1464c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:14:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.761+0000 7f2512400640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c151360 0x7f250c1464c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.763+0000 7f2511a00640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f250c0ac930 0x7f250c0a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.763+0000 7f2511a00640 1 -- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c151360 msgr2=0x7f250c1464c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:14:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.763+0000 7f2511a00640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c151360 0x7f250c1464c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.763+0000 7f2511a00640 1 -- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c0c30a0 msgr2=0x7f250c155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.763+0000 7f2511a00640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c0c30a0 0x7f250c155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.763+0000 7f2511a00640 1 -- 172.21.15.67:0/2196348867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c1f3950 con 0x7f250c0ac930 2024-03-09T09:14:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.764+0000 7f2511a00640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f250c0ac930 0x7f250c0a9760 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f250c06e520 tx=0x7f2500004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.765+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500048070 con 0x7f250c0ac930 2024-03-09T09:14:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.765+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2500039d10 con 0x7f250c0ac930 2024-03-09T09:14:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.765+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500034040 con 0x7f250c0ac930 2024-03-09T09:14:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.765+0000 7f2512e00640 1 -- 172.21.15.67:0/2196348867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250c153740 con 0x7f250c0ac930 2024-03-09T09:14:29.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.766+0000 7f2512e00640 1 -- 172.21.15.67:0/2196348867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250c0a9d20 con 0x7f250c0ac930 2024-03-09T09:14:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.768+0000 7f2512e00640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c151970 0x7f250c146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.768+0000 7f2512e00640 1 -- 172.21.15.67:0/2196348867 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c155c40 con 0x7f250c151970 2024-03-09T09:14:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.770+0000 7f2511000640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c151970 0x7f250c146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.771+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f250004b070 con 0x7f250c0ac930 2024-03-09T09:14:29.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.773+0000 7f2511000640 1 -- 172.21.15.67:0/2196348867 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f250c153c60 con 0x7f250c151970 2024-03-09T09:14:29.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.774+0000 7f2509600640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f24ec06b500 0x7f24ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.774+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f25000ac650 con 0x7f250c0ac930 2024-03-09T09:14:29.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.774+0000 7f2511000640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c151970 0x7f250c146a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f250c1505c0 tx=0x7f24f800e980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:29.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.777+0000 7f2512400640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f24ec06b500 0x7f24ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.779+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80052e0 con 0x7f250c151970 2024-03-09T09:14:29.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.779+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f24f8018070 con 0x7f250c151970 2024-03-09T09:14:29.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.779+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c151970 msgr2=0x7f250c146a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.779+0000 7f2509600640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c151970 0x7f250c146a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f250c1505c0 tx=0x7f24f800e980 comp rx=0 tx=0).stop 2024-03-09T09:14:29.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.779+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c151970 msgr2=0x7f250c146a00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:29.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.779+0000 7f2509600640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c151970 0x7f250c146a00 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f250c1505c0 tx=0x7f24f800e980 comp rx=0 tx=0).stop 2024-03-09T09:14:29.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.780+0000 7f2512400640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f24ec06b500 0x7f24ec06d9c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2504003cb0 tx=0x7f2504009270 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:29.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.796+0000 7f2512e00640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c155550 0x7f250c1f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:29.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.796+0000 7f2512e00640 1 -- 172.21.15.67:0/2196348867 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c1577b0 con 0x7f250c155550 2024-03-09T09:14:29.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.798+0000 7f2511000640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c155550 0x7f250c1f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:29.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.799+0000 7f2511000640 1 -- 172.21.15.67:0/2196348867 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f250c04f3c0 con 0x7f250c155550 2024-03-09T09:14:29.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.800+0000 7f2511000640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c155550 0x7f250c1f2d20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24f8017000 tx=0x7f24f8001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:29.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.803+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80035f0 con 0x7f250c155550 2024-03-09T09:14:29.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.805+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4523 (secure 0 0 0) 0x7f24f8003790 con 0x7f250c155550 2024-03-09T09:14:29.810 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:14:29.810 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:14:29.810 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 166, 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:29.811 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:29.812 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:14:29.813 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:14:29.814 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.815 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.816 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.817 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:29.818 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:14:29.819 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:14:29.820 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:14:29.821 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:14:29.822 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:14:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.805+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c155550 msgr2=0x7f250c1f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.805+0000 7f2509600640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c155550 0x7f250c1f2d20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24f8017000 tx=0x7f24f8001120 comp rx=0 tx=0).stop 2024-03-09T09:14:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.805+0000 7f2509600640 1 -- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c155550 msgr2=0x7f250c1f2d20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:14:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.805+0000 7f2509600640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f250c155550 0x7f250c1f2d20 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24f8017000 tx=0x7f24f8001120 comp rx=0 tx=0).stop 2024-03-09T09:14:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.810+0000 7f2512e00640 1 -- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f24ec06b500 msgr2=0x7f24ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.810+0000 7f2512e00640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f24ec06b500 0x7f24ec06d9c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2504003cb0 tx=0x7f2504009270 comp rx=0 tx=0).stop 2024-03-09T09:14:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.810+0000 7f2512e00640 1 -- 172.21.15.67:0/2196348867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f250c0ac930 msgr2=0x7f250c0a9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.810+0000 7f2512e00640 1 --2- 172.21.15.67:0/2196348867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f250c0ac930 0x7f250c0a9760 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f250c06e520 tx=0x7f2500004730 comp rx=0 tx=0).stop 2024-03-09T09:14:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.810+0000 7f2512400640 1 -- 172.21.15.67:0/2196348867 reap_dead start 2024-03-09T09:14:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.810+0000 7f2512400640 1 -- 172.21.15.67:0/2196348867 reap_dead start 2024-03-09T09:14:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.811+0000 7f2512e00640 1 -- 172.21.15.67:0/2196348867 shutdown_connections 2024-03-09T09:14:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.811+0000 7f2512e00640 1 -- 172.21.15.67:0/2196348867 >> 172.21.15.67:0/2196348867 conn(0x7f250c0c7220 msgr2=0x7f250c154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.812+0000 7f2512e00640 1 -- 172.21.15.67:0/2196348867 shutdown_connections 2024-03-09T09:14:29.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:29.812+0000 7f2512e00640 1 -- 172.21.15.67:0/2196348867 wait complete. 2024-03-09T09:14:29.843 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:14:29.843 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:14:29.979 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 65% (126/192) Updating files: 66% (127/192) Updating files: 67% (129/192) Updating files: 68% (131/192) Updating files: 69% (133/192) Updating files: 70% (135/192) Updating files: 71% (137/192) Updating files: 72% (139/192) Updating files: 73% (141/192) Updating files: 74% (143/192) Updating files: 75% (144/192) Updating files: 76% (146/192) Updating files: 77% (148/192) Updating files: 78% (150/192) Updating files: 79% (152/192) Updating files: 80% (154/192) Updating files: 81% (156/192) Updating files: 82% (158/192) Updating files: 83% (160/192) Updating files: 84% (162/192) Updating files: 85% (164/192) Updating files: 86% (166/192) Updating files: 87% (168/192) Updating files: 88% (169/192) Updating files: 89% (171/192) Updating files: 90% (173/192) Updating files: 91% (175/192) Updating files: 92% (177/192) Updating files: 93% (179/192) Updating files: 94% (181/192) Updating files: 95% (183/192) Updating files: 96% (185/192) Updating files: 97% (187/192) Updating files: 98% (189/192) Updating files: 99% (191/192) Updating files: 100% (192/192) Updating files: 100% (192/192), done. 2024-03-09T09:14:30.012 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at c165483bc72 v0.74 2024-03-09T09:14:30.019 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.74 2024-03-09T09:14:30.196 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:14:30.197 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 75 -eq 48 ']' 2024-03-09T09:14:30.197 INFO:tasks.workunit.client.0.smithi067.stdout:v0.75 2024-03-09T09:14:30.197 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.75 2024-03-09T09:14:30.197 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.75 2024-03-09T09:14:30.197 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.75 2024-03-09T09:14:33.504 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 16% (49/291) Updating files: 17% (50/291) Updating files: 18% (53/291) Updating files: 19% (56/291) Updating files: 20% (59/291) Updating files: 21% (62/291) Updating files: 22% (65/291) Updating files: 23% (67/291) Updating files: 24% (70/291) Updating files: 25% (73/291) Updating files: 26% (76/291) Updating files: 27% (79/291) Updating files: 28% (82/291) Updating files: 29% (85/291) Updating files: 30% (88/291) Updating files: 31% (91/291) Updating files: 32% (94/291) Updating files: 33% (97/291) Updating files: 34% (99/291) Updating files: 35% (102/291) Updating files: 36% (105/291) Updating files: 37% (108/291) Updating files: 38% (111/291) Updating files: 39% (114/291) Updating files: 40% (117/291) Updating files: 41% (120/291) Updating files: 42% (123/291) Updating files: 42% (125/291) Updating files: 43% (126/291) Updating files: 44% (129/291) Updating files: 45% (131/291) Updating files: 46% (134/291) Updating files: 47% (137/291) Updating files: 48% (140/291) Updating files: 49% (143/291) Updating files: 50% (146/291) Updating files: 51% (149/291) Updating files: 52% (152/291) Updating files: 53% (155/291) Updating files: 54% (158/291) Updating files: 55% (161/291) Updating files: 56% (163/291) Updating files: 57% (166/291) Updating files: 58% (169/291) Updating files: 59% (172/291) Updating files: 59% (173/291) Updating files: 60% (175/291) Updating files: 61% (178/291) Updating files: 62% (181/291) Updating files: 63% (184/291) Updating files: 64% (187/291) Updating files: 65% (190/291) Updating files: 66% (193/291) Updating files: 67% (195/291) Updating files: 68% (198/291) Updating files: 69% (201/291) Updating files: 70% (204/291) Updating files: 71% (207/291) Updating files: 72% (210/291) Updating files: 73% (213/291) Updating files: 74% (216/291) Updating files: 75% (219/291) Updating files: 76% (222/291) Updating files: 77% (225/291) Updating files: 78% (227/291) Updating files: 79% (230/291) Updating files: 80% (233/291) Updating files: 81% (236/291) Updating files: 82% (239/291) Updating files: 83% (242/291) Updating files: 84% (245/291) Updating files: 84% (246/291) Updating files: 85% (248/291) Updating files: 86% (251/291) Updating files: 87% (254/291) Updating files: 88% (257/291) Updating files: 89% (259/291) Updating files: 90% (262/291) Updating files: 91% (265/291) Updating files: 92% (268/291) Updating files: 93% (271/291) Updating files: 94% (274/291) Updating files: 95% (277/291) Updating files: 96% (280/291) Updating files: 97% (283/291) Updating files: 98% (286/291) Updating files: 99% (289/291) Updating files: 100% (291/291) Updating files: 100% (291/291), done. 2024-03-09T09:14:33.570 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 946d6036958 v0.75 2024-03-09T09:14:33.579 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.75 2024-03-09T09:14:33.857 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:14:33.857 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 76 -eq 48 ']' 2024-03-09T09:14:33.857 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.76 2024-03-09T09:14:33.857 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.76 2024-03-09T09:14:33.857 INFO:tasks.workunit.client.1.smithi134.stdout:v0.76 2024-03-09T09:14:33.857 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.76 2024-03-09T09:14:39.074 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 45% (76/168) Updating files: 46% (78/168) Updating files: 47% (79/168) Updating files: 48% (81/168) Updating files: 49% (83/168) Updating files: 50% (84/168) Updating files: 51% (86/168) Updating files: 52% (88/168) Updating files: 53% (90/168) Updating files: 54% (91/168) Updating files: 55% (93/168) Updating files: 56% (95/168) Updating files: 57% (96/168) Updating files: 57% (97/168) Updating files: 58% (98/168) Updating files: 59% (100/168) Updating files: 60% (101/168) Updating files: 61% (103/168) Updating files: 62% (105/168) Updating files: 63% (106/168) Updating files: 64% (108/168) Updating files: 65% (110/168) Updating files: 66% (111/168) Updating files: 67% (113/168) Updating files: 68% (115/168) Updating files: 69% (116/168) Updating files: 70% (118/168) Updating files: 71% (120/168) Updating files: 72% (121/168) Updating files: 73% (123/168) Updating files: 74% (125/168) Updating files: 75% (126/168) Updating files: 76% (128/168) Updating files: 77% (130/168) Updating files: 78% (132/168) Updating files: 79% (133/168) Updating files: 80% (135/168) Updating files: 81% (137/168) Updating files: 82% (138/168) Updating files: 83% (140/168) Updating files: 84% (142/168) Updating files: 85% (143/168) Updating files: 86% (145/168) Updating files: 87% (147/168) Updating files: 88% (148/168) Updating files: 89% (150/168) Updating files: 90% (152/168) Updating files: 91% (153/168) Updating files: 92% (155/168) Updating files: 93% (157/168) Updating files: 94% (158/168) Updating files: 95% (160/168) Updating files: 96% (162/168) Updating files: 97% (163/168) Updating files: 98% (165/168) Updating files: 99% (167/168) Updating files: 100% (168/168) Updating files: 100% (168/168), done. 2024-03-09T09:14:39.174 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 3b990136bfa v0.76 2024-03-09T09:14:39.188 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.76 2024-03-09T09:14:39.346 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:14:39.347 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 77 -eq 48 ']' 2024-03-09T09:14:39.347 INFO:tasks.workunit.client.1.smithi134.stdout:v0.77 2024-03-09T09:14:39.347 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.77 2024-03-09T09:14:39.347 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.77 2024-03-09T09:14:39.347 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.77 2024-03-09T09:14:42.385 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 20% (60/291) Updating files: 21% (62/291) Updating files: 22% (65/291) Updating files: 23% (67/291) Updating files: 24% (70/291) Updating files: 25% (73/291) Updating files: 26% (76/291) Updating files: 27% (79/291) Updating files: 28% (82/291) Updating files: 29% (85/291) Updating files: 30% (88/291) Updating files: 31% (91/291) Updating files: 32% (94/291) Updating files: 33% (97/291) Updating files: 34% (99/291) Updating files: 35% (102/291) Updating files: 36% (105/291) Updating files: 37% (108/291) Updating files: 38% (111/291) Updating files: 39% (114/291) Updating files: 40% (117/291) Updating files: 41% (120/291) Updating files: 42% (123/291) Updating files: 43% (126/291) Updating files: 44% (129/291) Updating files: 45% (131/291) Updating files: 46% (134/291) Updating files: 47% (137/291) Updating files: 48% (140/291) Updating files: 49% (143/291) Updating files: 50% (146/291) Updating files: 51% (149/291) Updating files: 52% (152/291) Updating files: 53% (155/291) Updating files: 54% (158/291) Updating files: 55% (161/291) Updating files: 56% (163/291) Updating files: 57% (166/291) Updating files: 58% (169/291) Updating files: 59% (172/291) Updating files: 60% (175/291) Updating files: 61% (178/291) Updating files: 62% (181/291) Updating files: 63% (184/291) Updating files: 64% (187/291) Updating files: 65% (190/291) Updating files: 66% (193/291) Updating files: 67% (195/291) Updating files: 68% (198/291) Updating files: 69% (201/291) Updating files: 70% (204/291) Updating files: 71% (207/291) Updating files: 72% (210/291) Updating files: 73% (213/291) Updating files: 74% (216/291) Updating files: 75% (219/291) Updating files: 76% (222/291) Updating files: 77% (225/291) Updating files: 78% (227/291) Updating files: 78% (228/291) Updating files: 79% (230/291) Updating files: 80% (233/291) Updating files: 81% (236/291) Updating files: 82% (239/291) Updating files: 83% (242/291) Updating files: 84% (245/291) Updating files: 85% (248/291) Updating files: 86% (251/291) Updating files: 87% (254/291) Updating files: 88% (257/291) Updating files: 89% (259/291) Updating files: 90% (262/291) Updating files: 91% (265/291) Updating files: 92% (268/291) Updating files: 93% (271/291) Updating files: 94% (274/291) Updating files: 95% (277/291) Updating files: 96% (280/291) Updating files: 97% (283/291) Updating files: 98% (286/291) Updating files: 99% (289/291) Updating files: 100% (291/291) Updating files: 100% (291/291), done. 2024-03-09T09:14:42.495 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 946d6036958 v0.75 2024-03-09T09:14:42.509 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.75 2024-03-09T09:14:42.738 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:14:42.738 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 76 -eq 48 ']' 2024-03-09T09:14:42.738 INFO:tasks.workunit.client.0.smithi067.stdout:v0.76 2024-03-09T09:14:42.738 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.76 2024-03-09T09:14:42.738 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.76 2024-03-09T09:14:42.739 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.76 2024-03-09T09:14:46.498 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 10% (30/298) Updating files: 11% (33/298) Updating files: 12% (36/298) Updating files: 13% (39/298) Updating files: 14% (42/298) Updating files: 15% (45/298) Updating files: 16% (48/298) Updating files: 17% (51/298) Updating files: 18% (54/298) Updating files: 19% (57/298) Updating files: 20% (60/298) Updating files: 21% (63/298) Updating files: 22% (66/298) Updating files: 23% (69/298) Updating files: 24% (72/298) Updating files: 25% (75/298) Updating files: 26% (78/298) Updating files: 27% (81/298) Updating files: 28% (84/298) Updating files: 29% (87/298) Updating files: 30% (90/298) Updating files: 31% (93/298) Updating files: 32% (96/298) Updating files: 33% (99/298) Updating files: 34% (102/298) Updating files: 34% (103/298) Updating files: 35% (105/298) Updating files: 36% (108/298) Updating files: 37% (111/298) Updating files: 38% (114/298) Updating files: 38% (116/298) Updating files: 39% (117/298) Updating files: 40% (120/298) Updating files: 41% (123/298) Updating files: 42% (126/298) Updating files: 43% (129/298) Updating files: 44% (132/298) Updating files: 45% (135/298) Updating files: 46% (138/298) Updating files: 47% (141/298) Updating files: 48% (144/298) Updating files: 49% (147/298) Updating files: 50% (149/298) Updating files: 51% (152/298) Updating files: 52% (155/298) Updating files: 53% (158/298) Updating files: 54% (161/298) Updating files: 55% (164/298) Updating files: 56% (167/298) Updating files: 57% (170/298) Updating files: 58% (173/298) Updating files: 59% (176/298) Updating files: 60% (179/298) Updating files: 61% (182/298) Updating files: 62% (185/298) Updating files: 63% (188/298) Updating files: 64% (191/298) Updating files: 65% (194/298) Updating files: 66% (197/298) Updating files: 67% (200/298) Updating files: 68% (203/298) Updating files: 69% (206/298) Updating files: 70% (209/298) Updating files: 70% (210/298) Updating files: 71% (212/298) Updating files: 72% (215/298) Updating files: 73% (218/298) Updating files: 74% (221/298) Updating files: 75% (224/298) Updating files: 76% (227/298) Updating files: 77% (230/298) Updating files: 78% (233/298) Updating files: 79% (236/298) Updating files: 80% (239/298) Updating files: 81% (242/298) Updating files: 82% (245/298) Updating files: 83% (248/298) Updating files: 84% (251/298) Updating files: 85% (254/298) Updating files: 86% (257/298) Updating files: 87% (260/298) Updating files: 88% (263/298) Updating files: 89% (266/298) Updating files: 90% (269/298) Updating files: 91% (272/298) Updating files: 92% (275/298) Updating files: 93% (278/298) Updating files: 94% (281/298) Updating files: 95% (284/298) Updating files: 96% (287/298) Updating files: 97% (290/298) Updating files: 98% (293/298) Updating files: 99% (296/298) Updating files: 100% (298/298) Updating files: 100% (298/298), done. 2024-03-09T09:14:46.572 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 1bca9c5c412 v0.77 2024-03-09T09:14:46.596 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.77 2024-03-09T09:14:46.919 INFO:tasks.workunit.client.1.smithi134.stdout:v0.78 2024-03-09T09:14:46.919 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:14:46.919 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 78 -eq 48 ']' 2024-03-09T09:14:46.919 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.78 2024-03-09T09:14:46.919 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.78 2024-03-09T09:14:46.919 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.78 2024-03-09T09:14:49.118 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 1% (3/168) Updating files: 2% (4/168) Updating files: 3% (6/168) Updating files: 4% (7/168) Updating files: 5% (9/168) Updating files: 6% (11/168) Updating files: 7% (12/168) Updating files: 8% (14/168) Updating files: 9% (16/168) Updating files: 10% (17/168) Updating files: 11% (19/168) Updating files: 12% (21/168) Updating files: 13% (22/168) Updating files: 14% (24/168) Updating files: 15% (26/168) Updating files: 16% (27/168) Updating files: 17% (29/168) Updating files: 18% (31/168) Updating files: 19% (32/168) Updating files: 20% (34/168) Updating files: 21% (36/168) Updating files: 22% (37/168) Updating files: 23% (39/168) Updating files: 24% (41/168) Updating files: 25% (42/168) Updating files: 26% (44/168) Updating files: 27% (46/168) Updating files: 28% (48/168) Updating files: 29% (49/168) Updating files: 30% (51/168) Updating files: 31% (53/168) Updating files: 32% (54/168) Updating files: 33% (56/168) Updating files: 34% (58/168) Updating files: 35% (59/168) Updating files: 36% (61/168) Updating files: 37% (63/168) Updating files: 38% (64/168) Updating files: 39% (66/168) Updating files: 40% (68/168) Updating files: 41% (69/168) Updating files: 42% (71/168) Updating files: 43% (73/168) Updating files: 44% (74/168) Updating files: 45% (76/168) Updating files: 46% (78/168) Updating files: 47% (79/168) Updating files: 48% (81/168) Updating files: 49% (83/168) Updating files: 50% (84/168) Updating files: 51% (86/168) Updating files: 52% (88/168) Updating files: 53% (90/168) Updating files: 54% (91/168) Updating files: 55% (93/168) Updating files: 56% (95/168) Updating files: 57% (96/168) Updating files: 58% (98/168) Updating files: 59% (100/168) Updating files: 60% (101/168) Updating files: 61% (103/168) Updating files: 62% (105/168) Updating files: 63% (106/168) Updating files: 64% (108/168) Updating files: 65% (110/168) Updating files: 66% (111/168) Updating files: 67% (113/168) Updating files: 68% (115/168) Updating files: 69% (116/168) Updating files: 70% (118/168) Updating files: 71% (120/168) Updating files: 72% (121/168) Updating files: 72% (122/168) Updating files: 73% (123/168) Updating files: 74% (125/168) Updating files: 75% (126/168) Updating files: 76% (128/168) Updating files: 77% (130/168) Updating files: 78% (132/168) Updating files: 79% (133/168) Updating files: 80% (135/168) Updating files: 81% (137/168) Updating files: 82% (138/168) Updating files: 83% (140/168) Updating files: 84% (142/168) Updating files: 85% (143/168) Updating files: 86% (145/168) Updating files: 87% (147/168) Updating files: 88% (148/168) Updating files: 89% (150/168) Updating files: 90% (152/168) Updating files: 91% (153/168) Updating files: 92% (155/168) Updating files: 93% (157/168) Updating files: 94% (158/168) Updating files: 95% (160/168) Updating files: 96% (162/168) Updating files: 97% (163/168) Updating files: 98% (165/168) Updating files: 99% (167/168) Updating files: 100% (168/168) Updating files: 100% (168/168), done. 2024-03-09T09:14:49.187 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 3b990136bfa v0.76 2024-03-09T09:14:49.199 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.76 2024-03-09T09:14:49.540 INFO:tasks.workunit.client.0.smithi067.stdout:v0.77 2024-03-09T09:14:49.541 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:14:49.541 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 77 -eq 48 ']' 2024-03-09T09:14:49.541 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.77 2024-03-09T09:14:49.541 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.77 2024-03-09T09:14:49.542 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.77 2024-03-09T09:14:49.844 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:14:49.844 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:14:49.844 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:14:49.845 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:14:49.853 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:14:49.853 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:14:49.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:14:50.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.068+0000 7f56b7e00640 1 Processor -- start 2024-03-09T09:14:50.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.068+0000 7f56b7e00640 1 -- start start 2024-03-09T09:14:50.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.069+0000 7f56b7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f56b00c22a0 0x7f56b0071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:50.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.069+0000 7f56b7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b01581a0 con 0x7f56b0157d60 2024-03-09T09:14:50.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.069+0000 7f56b7e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b0153d40 con 0x7f56b00ac730 2024-03-09T09:14:50.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.069+0000 7f56b7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b0070ee0 con 0x7f56b00c22a0 2024-03-09T09:14:50.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.070+0000 7f56b5e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f56b0157d60 0x7f56b015e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48360/0 (socket says 172.21.15.67:48360) 2024-03-09T09:14:50.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.070+0000 7f56b5e00640 1 -- 172.21.15.67:0/1655918179 learned_addr learned my addr 172.21.15.67:0/1655918179 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:14:50.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.070+0000 7f56b4a00640 1 --2- 172.21.15.67:0/1655918179 >> v2:172.21.15.67:3300/0 conn(0x7f56b00c22a0 0x7f56b0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.070+0000 7f56afe00640 1 -- 172.21.15.67:0/1655918179 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269064393 0 0) 0x7f56b0153d40 con 0x7f56b00ac730 2024-03-09T09:14:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.070+0000 7f56afe00640 1 -- 172.21.15.67:0/1655918179 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5694003520 con 0x7f56b00ac730 2024-03-09T09:14:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.070+0000 7f56b4a00640 1 -- 172.21.15.67:0/1655918179 >> v1:172.21.15.134:6789/0 conn(0x7f56b00ac730 legacy=0x7f56b0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:50.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.070+0000 7f56b4a00640 1 -- 172.21.15.67:0/1655918179 >> v1:172.21.15.67:6789/0 conn(0x7f56b0157d60 legacy=0x7f56b015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.070+0000 7f56b4a00640 1 -- 172.21.15.67:0/1655918179 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b00c2790 con 0x7f56b00c22a0 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.072+0000 7f56b4a00640 1 --2- 172.21.15.67:0/1655918179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b00c22a0 0x7f56b0071190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f569c0097a0 tx=0x7f569c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6511d8b5d90f34e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.072+0000 7f56afe00640 1 -- 172.21.15.67:0/1655918179 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c035020 con 0x7f56b00c22a0 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.072+0000 7f56afe00640 1 -- 172.21.15.67:0/1655918179 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c031830 con 0x7f56b00c22a0 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.072+0000 7f56afe00640 1 -- 172.21.15.67:0/1655918179 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c031be0 con 0x7f56b00c22a0 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.072+0000 7f56b7e00640 1 -- 172.21.15.67:0/1655918179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b00c22a0 msgr2=0x7f56b0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.072+0000 7f56b7e00640 1 --2- 172.21.15.67:0/1655918179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b00c22a0 0x7f56b0071190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f569c0097a0 tx=0x7f569c02b8d0 comp rx=0 tx=0).stop 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.072+0000 7f56b7e00640 1 -- 172.21.15.67:0/1655918179 shutdown_connections 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.072+0000 7f56b7e00640 1 --2- 172.21.15.67:0/1655918179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b00c22a0 0x7f56b0071190 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.072+0000 7f56b7e00640 1 -- 172.21.15.67:0/1655918179 >> 172.21.15.67:0/1655918179 conn(0x7f56b00c7370 msgr2=0x7f56b00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.073+0000 7f56b7e00640 1 -- 172.21.15.67:0/1655918179 shutdown_connections 2024-03-09T09:14:50.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.073+0000 7f56b7e00640 1 -- 172.21.15.67:0/1655918179 wait complete. 2024-03-09T09:14:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.075+0000 7f56b7e00640 1 Processor -- start 2024-03-09T09:14:50.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.075+0000 7f56b7e00640 1 -- start start 2024-03-09T09:14:50.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.077+0000 7f56b7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 0x7f56b01dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:50.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.077+0000 7f56b7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b00c22a0 0x7f56b01ef560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:50.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.077+0000 7f56b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b0157d60 0x7f56b014e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:50.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.077+0000 7f56b7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0070c00 con 0x7f56b0157d60 2024-03-09T09:14:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.077+0000 7f56b7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0154d00 con 0x7f56b00c22a0 2024-03-09T09:14:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.077+0000 7f56b7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0151610 con 0x7f56b00ac730 2024-03-09T09:14:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.077+0000 7f56b5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b0157d60 0x7f56b014e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.077+0000 7f56b5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b0157d60 0x7f56b014e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55514/0 (socket says 172.21.15.67:55514) 2024-03-09T09:14:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.077+0000 7f56b4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b00c22a0 0x7f56b01ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.077+0000 7f56b5e00640 1 -- 172.21.15.67:0/1418836142 learned_addr learned my addr 172.21.15.67:0/1418836142 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.077+0000 7f56b5400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 0x7f56b01dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.078+0000 7f56b5400640 1 -- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 msgr2=0x7f56b01dfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:14:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.078+0000 7f56b5400640 1 -- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 msgr2=0x7f56b01dfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:14:50.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.078+0000 7f56b5400640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 0x7f56b01dfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.078+0000 7f56b5400640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 0x7f56b01dfdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.078+0000 7f56b5e00640 1 -- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 msgr2=0x7f56b01dfdd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:14:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.078+0000 7f56b5e00640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 0x7f56b01dfdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.078+0000 7f56b5e00640 1 -- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b00c22a0 msgr2=0x7f56b01ef560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.078+0000 7f56b5e00640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b00c22a0 0x7f56b01ef560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.078+0000 7f56b5e00640 1 -- 172.21.15.67:0/1418836142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b01e9200 con 0x7f56b0157d60 2024-03-09T09:14:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.078+0000 7f56b5e00640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b0157d60 0x7f56b014e5e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f56a800c420 tx=0x7f56a800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:50.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.079+0000 7f56ad600640 1 -- 172.21.15.67:0/1418836142 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8017070 con 0x7f56b0157d60 2024-03-09T09:14:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.079+0000 7f56ad600640 1 -- 172.21.15.67:0/1418836142 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a8005ce0 con 0x7f56b0157d60 2024-03-09T09:14:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.079+0000 7f56b7e00640 1 -- 172.21.15.67:0/1418836142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b01def90 con 0x7f56b0157d60 2024-03-09T09:14:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.079+0000 7f56ad600640 1 -- 172.21.15.67:0/1418836142 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80072d0 con 0x7f56b0157d60 2024-03-09T09:14:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.079+0000 7f56b7e00640 1 -- 172.21.15.67:0/1418836142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b01efaa0 con 0x7f56b0157d60 2024-03-09T09:14:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.080+0000 7f56b7e00640 1 -- 172.21.15.67:0/1418836142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b0075a00 con 0x7f56b0157d60 2024-03-09T09:14:50.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.082+0000 7f56ad600640 1 -- 172.21.15.67:0/1418836142 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f56a801a070 con 0x7f56b0157d60 2024-03-09T09:14:50.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.083+0000 7f56ad600640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 0x7f569006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:50.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.083+0000 7f56ad600640 1 -- 172.21.15.67:0/1418836142 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f56a8089830 con 0x7f56b0157d60 2024-03-09T09:14:50.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.084+0000 7f56b5400640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 0x7f569006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:50.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.084+0000 7f56b5400640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 0x7f569006d990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f56a0001560 tx=0x7f56a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:50.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.088+0000 7f56ad600640 1 -- 172.21.15.67:0/1418836142 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f56a8057c60 con 0x7f56b0157d60 2024-03-09T09:14:50.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.412+0000 7f56b7e00640 1 -- 172.21.15.67:0/1418836142 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f56b00c1d70 con 0x7f56b0157d60 2024-03-09T09:14:50.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.413+0000 7f56ad600640 1 -- 172.21.15.67:0/1418836142 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f56a805bd00 con 0x7f56b0157d60 2024-03-09T09:14:50.418 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":166,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:14:50.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 -- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 msgr2=0x7f569006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 0x7f569006d990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f56a0001560 tx=0x7f56a0009040 comp rx=0 tx=0).stop 2024-03-09T09:14:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 -- 172.21.15.67:0/1418836142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b0157d60 msgr2=0x7f56b014e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b0157d60 0x7f56b014e5e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f56a800c420 tx=0x7f56a800c8e0 comp rx=0 tx=0).stop 2024-03-09T09:14:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 -- 172.21.15.67:0/1418836142 shutdown_connections 2024-03-09T09:14:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 0x7f569006d990 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b0157d60 0x7f56b014e5e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b00c22a0 0x7f56b01ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 --2- 172.21.15.67:0/1418836142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 0x7f56b01dfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 -- 172.21.15.67:0/1418836142 >> 172.21.15.67:0/1418836142 conn(0x7f56b00c7370 msgr2=0x7f56b00c7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 -- 172.21.15.67:0/1418836142 shutdown_connections 2024-03-09T09:14:50.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:50.416+0000 7f56b7e00640 1 -- 172.21.15.67:0/1418836142 wait complete. 2024-03-09T09:14:53.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:14:53.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.662+0000 7effc5000640 1 Processor -- start 2024-03-09T09:14:53.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.663+0000 7effc5000640 1 -- start start 2024-03-09T09:14:53.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.664+0000 7effc5000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7effc006b040 0x7effc0155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.664+0000 7effc5000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc00c6a40 con 0x7effc006c650 2024-03-09T09:14:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.664+0000 7effc5000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc006b5b0 con 0x7effc00c5f50 2024-03-09T09:14:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.664+0000 7effc5000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc00c9e00 con 0x7effc006b040 2024-03-09T09:14:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.665+0000 7effbde00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7effc00c5f50 0x7effc00c96f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:34940/0 (socket says 172.21.15.67:34940) 2024-03-09T09:14:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.665+0000 7effbde00640 1 -- 172.21.15.67:0/628648231 learned_addr learned my addr 172.21.15.67:0/628648231 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:14:53.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.665+0000 7effbd400640 1 --2- 172.21.15.67:0/628648231 >> v2:172.21.15.67:3300/0 conn(0x7effc006b040 0x7effc0155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.666+0000 7effbd400640 1 -- 172.21.15.67:0/628648231 >> v1:172.21.15.134:6789/0 conn(0x7effc00c5f50 legacy=0x7effc00c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.666+0000 7effbd400640 1 -- 172.21.15.67:0/628648231 >> v1:172.21.15.67:6789/0 conn(0x7effc006c650 legacy=0x7effc00c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.666+0000 7effbd400640 1 -- 172.21.15.67:0/628648231 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc006b890 con 0x7effc006b040 2024-03-09T09:14:53.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.666+0000 7effbca00640 1 -- 172.21.15.67:0/628648231 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842036521 0 0) 0x7effc006b5b0 con 0x7effc00c5f50 2024-03-09T09:14:53.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.667+0000 7effbd400640 1 --2- 172.21.15.67:0/628648231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effc006b040 0x7effc0155e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7effac0097a0 tx=0x7effac02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354057b2d17aacaa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:53.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.667+0000 7effbca00640 1 -- 172.21.15.67:0/628648231 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac034070 con 0x7effc006b040 2024-03-09T09:14:53.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.667+0000 7effbca00640 1 -- 172.21.15.67:0/628648231 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac030610 con 0x7effc006b040 2024-03-09T09:14:53.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.668+0000 7effbca00640 1 -- 172.21.15.67:0/628648231 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac0309e0 con 0x7effc006b040 2024-03-09T09:14:53.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.668+0000 7effc5000640 1 -- 172.21.15.67:0/628648231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effc006b040 msgr2=0x7effc0155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:53.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.668+0000 7effc5000640 1 --2- 172.21.15.67:0/628648231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effc006b040 0x7effc0155e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7effac0097a0 tx=0x7effac02b960 comp rx=0 tx=0).stop 2024-03-09T09:14:53.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.668+0000 7effc5000640 1 -- 172.21.15.67:0/628648231 shutdown_connections 2024-03-09T09:14:53.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.669+0000 7effc5000640 1 --2- 172.21.15.67:0/628648231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effc006b040 0x7effc0155e20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:53.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.669+0000 7effc5000640 1 -- 172.21.15.67:0/628648231 >> 172.21.15.67:0/628648231 conn(0x7effc00c8050 msgr2=0x7effc00c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:53.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.669+0000 7effc5000640 1 -- 172.21.15.67:0/628648231 shutdown_connections 2024-03-09T09:14:53.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.669+0000 7effc5000640 1 -- 172.21.15.67:0/628648231 wait complete. 2024-03-09T09:14:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.672+0000 7effc5000640 1 Processor -- start 2024-03-09T09:14:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.672+0000 7effc5000640 1 -- start start 2024-03-09T09:14:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.673+0000 7effc5000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effc006b040 0x7effc014bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effc5000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effc006c650 0x7effc01ef260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effc5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effc00c5f50 0x7effc014a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effc5000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc006be50 con 0x7effc006c650 2024-03-09T09:14:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effc5000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc006cac0 con 0x7effc00c5f50 2024-03-09T09:14:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effc5000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc006c130 con 0x7effc006b040 2024-03-09T09:14:53.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effbd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effc006c650 0x7effc01ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:53.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effbd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effc006c650 0x7effc01ef260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55524/0 (socket says 172.21.15.67:55524) 2024-03-09T09:14:53.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effbd400640 1 -- 172.21.15.67:0/3480817659 learned_addr learned my addr 172.21.15.67:0/3480817659 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:53.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effbde00640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effc006b040 0x7effc014bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effbe800640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effc00c5f50 0x7effc014a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effbde00640 1 -- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effc006b040 msgr2=0x7effc014bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:14:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effbde00640 1 -- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effc006b040 msgr2=0x7effc014bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:14:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effbde00640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effc006b040 0x7effc014bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.674+0000 7effbde00640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effc006b040 0x7effc014bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:53.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.675+0000 7effbd400640 1 -- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effc006b040 msgr2=0x7effc014bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:14:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.675+0000 7effbd400640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effc006b040 0x7effc014bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.675+0000 7effbd400640 1 -- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effc00c5f50 msgr2=0x7effc014a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.675+0000 7effbd400640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effc00c5f50 0x7effc014a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.675+0000 7effbd400640 1 -- 172.21.15.67:0/3480817659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc01e2410 con 0x7effc006c650 2024-03-09T09:14:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.675+0000 7effbd400640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effc006c650 0x7effc01ef260 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7effac000c00 tx=0x7effac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.676+0000 7effb6000640 1 -- 172.21.15.67:0/3480817659 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac034040 con 0x7effc006c650 2024-03-09T09:14:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.676+0000 7effb6000640 1 -- 172.21.15.67:0/3480817659 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac004030 con 0x7effc006c650 2024-03-09T09:14:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.676+0000 7effb6000640 1 -- 172.21.15.67:0/3480817659 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac004420 con 0x7effc006c650 2024-03-09T09:14:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.676+0000 7effc5000640 1 -- 172.21.15.67:0/3480817659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc01efce0 con 0x7effc006c650 2024-03-09T09:14:53.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.676+0000 7effc5000640 1 -- 172.21.15.67:0/3480817659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc00bf830 con 0x7effc006c650 2024-03-09T09:14:53.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.678+0000 7effc5000640 1 -- 172.21.15.67:0/3480817659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc0075a00 con 0x7effc006c650 2024-03-09T09:14:53.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.679+0000 7effb6000640 1 -- 172.21.15.67:0/3480817659 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7effac04c060 con 0x7effc006c650 2024-03-09T09:14:53.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.681+0000 7effb6000640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7effa006b520 0x7effa006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:53.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.681+0000 7effb6000640 1 -- 172.21.15.67:0/3480817659 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7effac0ac0e0 con 0x7effc006c650 2024-03-09T09:14:53.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.681+0000 7effbde00640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7effa006b520 0x7effa006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:53.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.682+0000 7effbde00640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7effa006b520 0x7effa006d9e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7effb0000f60 tx=0x7effb0007d00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:53.687+0000 7effb6000640 1 -- 172.21.15.67:0/3480817659 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7effac07ace0 con 0x7effc006c650 2024-03-09T09:14:54.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.120+0000 7effc5000640 1 -- 172.21.15.67:0/3480817659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7effc006cca0 con 0x7effc006c650 2024-03-09T09:14:54.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.120+0000 7effb6000640 1 -- 172.21.15.67:0/3480817659 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7effac07a5e0 con 0x7effc006c650 2024-03-09T09:14:54.126 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":166,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 -- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7effa006b520 msgr2=0x7effa006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7effa006b520 0x7effa006d9e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7effb0000f60 tx=0x7effb0007d00 comp rx=0 tx=0).stop 2024-03-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 -- 172.21.15.67:0/3480817659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effc006c650 msgr2=0x7effc01ef260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effc006c650 0x7effc01ef260 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7effac000c00 tx=0x7effac0046c0 comp rx=0 tx=0).stop 2024-03-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 -- 172.21.15.67:0/3480817659 shutdown_connections 2024-03-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7effa006b520 0x7effa006d9e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effc00c5f50 0x7effc014a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7effc006c650 0x7effc01ef260 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 --2- 172.21.15.67:0/3480817659 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7effc006b040 0x7effc014bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 -- 172.21.15.67:0/3480817659 >> 172.21.15.67:0/3480817659 conn(0x7effc00c8050 msgr2=0x7effc00ab030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 -- 172.21.15.67:0/3480817659 shutdown_connections 2024-03-09T09:14:54.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:54.123+0000 7effc5000640 1 -- 172.21.15.67:0/3480817659 wait complete. 2024-03-09T09:14:54.335 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 9% (29/298) Updating files: 10% (30/298) Updating files: 11% (33/298) Updating files: 12% (36/298) Updating files: 13% (39/298) Updating files: 14% (42/298) Updating files: 15% (45/298) Updating files: 16% (48/298) Updating files: 17% (51/298) Updating files: 18% (54/298) Updating files: 19% (57/298) Updating files: 19% (58/298) Updating files: 20% (60/298) Updating files: 21% (63/298) Updating files: 22% (66/298) Updating files: 23% (69/298) Updating files: 24% (72/298) Updating files: 25% (75/298) Updating files: 26% (78/298) Updating files: 27% (81/298) Updating files: 28% (84/298) Updating files: 29% (87/298) Updating files: 30% (90/298) Updating files: 31% (93/298) Updating files: 32% (96/298) Updating files: 33% (99/298) Updating files: 34% (102/298) Updating files: 35% (105/298) Updating files: 36% (108/298) Updating files: 37% (111/298) Updating files: 38% (114/298) Updating files: 39% (117/298) Updating files: 40% (120/298) Updating files: 41% (123/298) Updating files: 42% (126/298) Updating files: 43% (129/298) Updating files: 44% (132/298) Updating files: 45% (135/298) Updating files: 46% (138/298) Updating files: 47% (141/298) Updating files: 48% (144/298) Updating files: 49% (147/298) Updating files: 49% (148/298) Updating files: 50% (149/298) Updating files: 51% (152/298) Updating files: 52% (155/298) Updating files: 53% (158/298) Updating files: 54% (161/298) Updating files: 55% (164/298) Updating files: 56% (167/298) Updating files: 57% (170/298) Updating files: 58% (173/298) Updating files: 59% (176/298) Updating files: 60% (179/298) Updating files: 61% (182/298) Updating files: 62% (185/298) Updating files: 63% (188/298) Updating files: 64% (191/298) Updating files: 65% (194/298) Updating files: 66% (197/298) Updating files: 67% (200/298) Updating files: 68% (203/298) Updating files: 69% (206/298) Updating files: 70% (209/298) Updating files: 71% (212/298) Updating files: 72% (215/298) Updating files: 73% (218/298) Updating files: 74% (221/298) Updating files: 75% (224/298) Updating files: 76% (227/298) Updating files: 77% (230/298) Updating files: 78% (233/298) Updating files: 79% (236/298) Updating files: 80% (239/298) Updating files: 81% (242/298) Updating files: 82% (245/298) Updating files: 83% (248/298) Updating files: 84% (251/298) Updating files: 85% (254/298) Updating files: 85% (255/298) Updating files: 86% (257/298) Updating files: 87% (260/298) Updating files: 88% (263/298) Updating files: 89% (266/298) Updating files: 90% (269/298) Updating files: 91% (272/298) Updating files: 92% (275/298) Updating files: 93% (278/298) Updating files: 94% (281/298) Updating files: 95% (284/298) Updating files: 96% (287/298) Updating files: 97% (290/298) Updating files: 98% (293/298) Updating files: 99% (296/298) Updating files: 100% (298/298) Updating files: 100% (298/298), done. 2024-03-09T09:14:54.406 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 1bca9c5c412 v0.77 2024-03-09T09:14:54.418 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.77 2024-03-09T09:14:54.617 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:14:54.618 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 78 -eq 48 ']' 2024-03-09T09:14:54.618 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.78 2024-03-09T09:14:54.618 INFO:tasks.workunit.client.0.smithi067.stdout:v0.78 2024-03-09T09:14:54.618 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.78 2024-03-09T09:14:54.618 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.78 2024-03-09T09:14:57.146 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:14:57.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.353+0000 7f6b17200640 1 Processor -- start 2024-03-09T09:14:57.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.353+0000 7f6b17200640 1 -- start start 2024-03-09T09:14:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.355+0000 7f6b17200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.355+0000 7f6b17200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b18155d60 con 0x7f6b1806e480 2024-03-09T09:14:57.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.355+0000 7f6b17200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b18156c50 con 0x7f6b181592d0 2024-03-09T09:14:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.355+0000 7f6b17200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b18159a40 con 0x7f6b180c1bd0 2024-03-09T09:14:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.355+0000 7f6b15400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6b1806e480 0x7f6b18156540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48376/0 (socket says 172.21.15.67:48376) 2024-03-09T09:14:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.355+0000 7f6b15400640 1 -- 172.21.15.67:0/3812745271 learned_addr learned my addr 172.21.15.67:0/3812745271 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:14:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.355+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.356+0000 7f6b15e00640 1 -- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 msgr2=0x7f6b180c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:14:57.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.356+0000 7f6b15e00640 1 -- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 msgr2=0x7f6b180c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:14:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.356+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:57.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.356+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:57.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.557+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:57.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.558+0000 7f6b15e00640 1 -- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 msgr2=0x7f6b180c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:14:57.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.558+0000 7f6b15e00640 1 -- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 msgr2=0x7f6b180c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:14:57.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.558+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:57.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.558+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:14:57.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.959+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:57.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.960+0000 7f6b15e00640 1 -- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 msgr2=0x7f6b180c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:14:57.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.960+0000 7f6b15e00640 1 -- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 msgr2=0x7f6b180c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:14:57.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.960+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:57.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:57.960+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:14:58.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:58.762+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:58.762+0000 7f6b15e00640 1 -- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 msgr2=0x7f6b180c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:14:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:58.762+0000 7f6b15e00640 1 -- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 msgr2=0x7f6b180c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:14:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:58.762+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:58.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:58.762+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:14:59.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.858+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692017641 0 0) 0x7f6b18156c50 con 0x7f6b181592d0 2024-03-09T09:14:59.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.859+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6afc003520 con 0x7f6b181592d0 2024-03-09T09:14:59.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.859+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286862303 0 0) 0x7f6afc003520 con 0x7f6b181592d0 2024-03-09T09:14:59.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.859+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 msgr2=0x7f6b180c4360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:14:59.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.859+0000 7f6b14a00640 1 --2- 172.21.15.67:0/3812745271 >> v2:172.21.15.134:3300/0 conn(0x7f6b180c1bd0 0x7f6b180c4360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:59.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.860+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 >> v1:172.21.15.67:6789/0 conn(0x7f6b1806e480 legacy=0x7f6b18156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:59.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.860+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b18159d20 con 0x7f6b181592d0 2024-03-09T09:14:59.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.860+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f6b100019f0 con 0x7f6b181592d0 2024-03-09T09:14:59.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.860+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 >> v1:172.21.15.134:6789/0 conn(0x7f6b181592d0 legacy=0x7f6b1813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:59.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.860+0000 7f6b14a00640 1 --2- 172.21.15.67:0/3812745271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 0x7f6afc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:59.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.860+0000 7f6b14a00640 1 --2- 172.21.15.67:0/3812745271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6afc004800 0x7f6afc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:59.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.860+0000 7f6b14a00640 1 --2- 172.21.15.67:0/3812745271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6afc0052e0 0x7f6afc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:59.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.860+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18159a40 con 0x7f6afc004800 2024-03-09T09:14:59.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.860+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc00c230 con 0x7f6afc003a20 2024-03-09T09:14:59.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.860+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc00c430 con 0x7f6afc0052e0 2024-03-09T09:14:59.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.863+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 0x7f6afc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:59.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.863+0000 7f6b16800640 1 --2- 172.21.15.67:0/3812745271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6afc0052e0 0x7f6afc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:59.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.864+0000 7f6b15e00640 1 -- 172.21.15.67:0/3812745271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6afc0052e0 msgr2=0x7f6afc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:14:59.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.864+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6afc0052e0 0x7f6afc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:59.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.864+0000 7f6b15e00640 1 -- 172.21.15.67:0/3812745271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6afc004800 msgr2=0x7f6afc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:14:59.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.864+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6afc004800 0x7f6afc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:59.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.864+0000 7f6b15e00640 1 -- 172.21.15.67:0/3812745271 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b18159d20 con 0x7f6afc003a20 2024-03-09T09:14:59.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.866+0000 7f6b16800640 1 -- 172.21.15.67:0/3812745271 reap_dead start 2024-03-09T09:14:59.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.867+0000 7f6b15e00640 1 --2- 172.21.15.67:0/3812745271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 0x7f6afc003e40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b08006fd0 tx=0x7f6afc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=816a2681ba65b639 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:59.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.867+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0803a070 con 0x7f6afc003a20 2024-03-09T09:14:59.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.868+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08035410 con 0x7f6afc003a20 2024-03-09T09:14:59.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.868+0000 7f6b14a00640 1 -- 172.21.15.67:0/3812745271 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08011790 con 0x7f6afc003a20 2024-03-09T09:14:59.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.869+0000 7f6b17200640 1 -- 172.21.15.67:0/3812745271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 msgr2=0x7f6afc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:59.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.869+0000 7f6b17200640 1 --2- 172.21.15.67:0/3812745271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 0x7f6afc003e40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b08006fd0 tx=0x7f6afc005830 comp rx=0 tx=0).stop 2024-03-09T09:14:59.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.869+0000 7f6b17200640 1 -- 172.21.15.67:0/3812745271 shutdown_connections 2024-03-09T09:14:59.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.869+0000 7f6b17200640 1 --2- 172.21.15.67:0/3812745271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 0x7f6afc003e40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:59.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.869+0000 7f6b17200640 1 -- 172.21.15.67:0/3812745271 >> 172.21.15.67:0/3812745271 conn(0x7f6b180c8310 msgr2=0x7f6b180c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:14:59.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.870+0000 7f6b17200640 1 -- 172.21.15.67:0/3812745271 shutdown_connections 2024-03-09T09:14:59.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.870+0000 7f6b17200640 1 -- 172.21.15.67:0/3812745271 wait complete. 2024-03-09T09:14:59.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.873+0000 7f6b17200640 1 Processor -- start 2024-03-09T09:14:59.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.873+0000 7f6b17200640 1 -- start start 2024-03-09T09:14:59.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.874+0000 7f6b17200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 0x7f6b180ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:59.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.874+0000 7f6b17200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b181eafe0 0x7f6b180c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:59.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.874+0000 7f6b17200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b180728a0 0x7f6b18072d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:59.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.874+0000 7f6b17200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18159700 con 0x7f6b181eafe0 2024-03-09T09:14:59.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.874+0000 7f6b17200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18156020 con 0x7f6afc003a20 2024-03-09T09:14:59.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.874+0000 7f6b17200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b180c1fb0 con 0x7f6b180728a0 2024-03-09T09:14:59.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b15400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b181eafe0 0x7f6b180c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:59.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b15400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b181eafe0 0x7f6b180c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52364/0 (socket says 172.21.15.67:52364) 2024-03-09T09:14:59.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b15400640 1 -- 172.21.15.67:0/1877502891 learned_addr learned my addr 172.21.15.67:0/1877502891 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:14:59.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b15e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 0x7f6b180ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:59.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b16800640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b180728a0 0x7f6b18072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:59.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b16800640 1 -- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b180728a0 msgr2=0x7f6b18072d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:14:59.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b16800640 1 -- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b180728a0 msgr2=0x7f6b18072d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:14:59.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b16800640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b180728a0 0x7f6b18072d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:14:59.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b16800640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b180728a0 0x7f6b18072d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:14:59.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b15400640 1 -- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b180728a0 msgr2=0x7f6b18072d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:14:59.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b15400640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b180728a0 0x7f6b18072d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:59.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b15400640 1 -- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 msgr2=0x7f6b180ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:14:59.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b15400640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 0x7f6b180ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:14:59.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b15400640 1 -- 172.21.15.67:0/1877502891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b180c5070 con 0x7f6b181eafe0 2024-03-09T09:14:59.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.875+0000 7f6b15e00640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 0x7f6b180ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:14:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.876+0000 7f6b15400640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b181eafe0 0x7f6b180c71c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400c3f0 tx=0x7f6b0400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.876+0000 7f6b0e000640 1 -- 172.21.15.67:0/1877502891 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04017070 con 0x7f6b181eafe0 2024-03-09T09:14:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.876+0000 7f6b0e000640 1 -- 172.21.15.67:0/1877502891 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b04005cc0 con 0x7f6b181eafe0 2024-03-09T09:14:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.876+0000 7f6b0e000640 1 -- 172.21.15.67:0/1877502891 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04007310 con 0x7f6b181eafe0 2024-03-09T09:14:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.876+0000 7f6b17200640 1 -- 172.21.15.67:0/1877502891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b18071800 con 0x7f6b181eafe0 2024-03-09T09:14:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.876+0000 7f6b17200640 1 -- 172.21.15.67:0/1877502891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b18071dc0 con 0x7f6b181eafe0 2024-03-09T09:14:59.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.878+0000 7f6b17200640 1 -- 172.21.15.67:0/1877502891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1804f3c0 con 0x7f6b181eafe0 2024-03-09T09:14:59.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.879+0000 7f6b0e000640 1 -- 172.21.15.67:0/1877502891 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6b0401a070 con 0x7f6b181eafe0 2024-03-09T09:14:59.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.881+0000 7f6b0e000640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6af806b4d0 0x7f6af806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:14:59.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.881+0000 7f6b0e000640 1 -- 172.21.15.67:0/1877502891 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6b040897d0 con 0x7f6b181eafe0 2024-03-09T09:14:59.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.881+0000 7f6b15e00640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6af806b4d0 0x7f6af806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:14:59.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.882+0000 7f6b15e00640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6af806b4d0 0x7f6af806d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6b08037000 tx=0x7f6b08030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:14:59.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:14:59.886+0000 7f6b0e000640 1 -- 172.21.15.67:0/1877502891 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6b04057c80 con 0x7f6b181eafe0 2024-03-09T09:15:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.249+0000 7f6b17200640 1 -- 172.21.15.67:0/1877502891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6b1806fe20 con 0x7f6b181eafe0 2024-03-09T09:15:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.250+0000 7f6b0e000640 1 -- 172.21.15.67:0/1877502891 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f6b0405bd20 con 0x7f6b181eafe0 2024-03-09T09:15:00.256 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":170,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:15:00.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.253+0000 7f6b17200640 1 -- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6af806b4d0 msgr2=0x7f6af806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:00.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.253+0000 7f6b17200640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6af806b4d0 0x7f6af806d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6b08037000 tx=0x7f6b08030000 comp rx=0 tx=0).stop 2024-03-09T09:15:00.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.254+0000 7f6b17200640 1 -- 172.21.15.67:0/1877502891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b181eafe0 msgr2=0x7f6b180c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:00.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.254+0000 7f6b17200640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b181eafe0 0x7f6b180c71c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400c3f0 tx=0x7f6b0400c8b0 comp rx=0 tx=0).stop 2024-03-09T09:15:00.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.254+0000 7f6b17200640 1 -- 172.21.15.67:0/1877502891 shutdown_connections 2024-03-09T09:15:00.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.254+0000 7f6b17200640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6af806b4d0 0x7f6af806d990 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:00.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.254+0000 7f6b17200640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6b180728a0 0x7f6b18072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:00.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.254+0000 7f6b17200640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b181eafe0 0x7f6b180c71c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:00.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.254+0000 7f6b17200640 1 --2- 172.21.15.67:0/1877502891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6afc003a20 0x7f6b180ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:00.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.254+0000 7f6b17200640 1 -- 172.21.15.67:0/1877502891 >> 172.21.15.67:0/1877502891 conn(0x7f6b180c8310 msgr2=0x7f6b181564b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:00.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.254+0000 7f6b17200640 1 -- 172.21.15.67:0/1877502891 shutdown_connections 2024-03-09T09:15:00.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:00.254+0000 7f6b17200640 1 -- 172.21.15.67:0/1877502891 wait complete. 2024-03-09T09:15:00.493 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 9% (48/496) Updating files: 10% (50/496) Updating files: 11% (55/496) Updating files: 12% (60/496) Updating files: 13% (65/496) Updating files: 14% (70/496) Updating files: 15% (75/496) Updating files: 16% (80/496) Updating files: 17% (85/496) Updating files: 18% (90/496) Updating files: 19% (95/496) Updating files: 20% (100/496) Updating files: 21% (105/496) Updating files: 22% (110/496) Updating files: 23% (115/496) Updating files: 24% (120/496) Updating files: 25% (124/496) Updating files: 26% (129/496) Updating files: 27% (134/496) Updating files: 28% (139/496) Updating files: 29% (144/496) Updating files: 30% (149/496) Updating files: 31% (154/496) Updating files: 32% (159/496) Updating files: 33% (164/496) Updating files: 34% (169/496) Updating files: 35% (174/496) Updating files: 36% (179/496) Updating files: 37% (184/496) Updating files: 38% (189/496) Updating files: 39% (194/496) Updating files: 40% (199/496) Updating files: 40% (202/496) Updating files: 41% (204/496) Updating files: 42% (209/496) Updating files: 43% (214/496) Updating files: 44% (219/496) Updating files: 45% (224/496) Updating files: 46% (229/496) Updating files: 47% (234/496) Updating files: 48% (239/496) Updating files: 49% (244/496) Updating files: 50% (248/496) Updating files: 51% (253/496) Updating files: 52% (258/496) Updating files: 53% (263/496) Updating files: 53% (264/496) Updating files: 54% (268/496) Updating files: 55% (273/496) Updating files: 56% (278/496) Updating files: 57% (283/496) Updating files: 58% (288/496) Updating files: 59% (293/496) Updating files: 59% (294/496) Updating files: 60% (298/496) Updating files: 61% (303/496) Updating files: 62% (308/496) Updating files: 63% (313/496) Updating files: 64% (318/496) Updating files: 65% (323/496) Updating files: 66% (328/496) Updating files: 67% (333/496) Updating files: 68% (338/496) Updating files: 69% (343/496) Updating files: 70% (348/496) Updating files: 71% (353/496) Updating files: 72% (358/496) Updating files: 73% (363/496) Updating files: 73% (366/496) Updating files: 74% (368/496) Updating files: 75% (372/496) Updating files: 76% (377/496) Updating files: 76% (379/496) Updating files: 77% (382/496) Updating files: 78% (387/496) Updating files: 79% (392/496) Updating files: 80% (397/496) Updating files: 81% (402/496) Updating files: 82% (407/496) Updating files: 83% (412/496) Updating files: 84% (417/496) Updating files: 85% (422/496) Updating files: 86% (427/496) Updating files: 87% (432/496) Updating files: 88% (437/496) Updating files: 89% (442/496) Updating files: 90% (447/496) Updating files: 91% (452/496) Updating files: 92% (457/496) Updating files: 93% (462/496) Updating files: 94% (467/496) Updating files: 94% (469/496) Updating files: 95% (472/496) Updating files: 96% (477/496) Updating files: 97% (482/496) Updating files: 98% (487/496) Updating files: 99% (492/496) Updating files: 100% (496/496) Updating files: 100% (496/496), done. 2024-03-09T09:15:00.584 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at f6c746c314d 0.78 2024-03-09T09:15:00.605 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.78 2024-03-09T09:15:00.846 INFO:tasks.workunit.client.1.smithi134.stdout:v0.79 2024-03-09T09:15:00.846 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:15:00.846 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 79 -eq 48 ']' 2024-03-09T09:15:00.846 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.79 2024-03-09T09:15:00.847 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.79 2024-03-09T09:15:00.847 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.79 2024-03-09T09:15:03.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:15:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.459+0000 7ff681800640 1 Processor -- start 2024-03-09T09:15:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.459+0000 7ff681800640 1 -- start start 2024-03-09T09:15:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.460+0000 7ff681800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff67c0c3080 0x7ff67c1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.460+0000 7ff681800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff67c14c9f0 0x7ff67c1540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.460+0000 7ff681800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c06d920 con 0x7ff67c06d420 2024-03-09T09:15:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.460+0000 7ff681800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c155ba0 con 0x7ff67c14c9f0 2024-03-09T09:15:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.460+0000 7ff681800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c06dc00 con 0x7ff67c0c3080 2024-03-09T09:15:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.461+0000 7ff680e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff67c14c9f0 0x7ff67c1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.461+0000 7ff680e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff67c14c9f0 0x7ff67c1540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52372/0 (socket says 172.21.15.67:52372) 2024-03-09T09:15:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.461+0000 7ff680e00640 1 -- 172.21.15.67:0/4225467863 learned_addr learned my addr 172.21.15.67:0/4225467863 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.461+0000 7ff67be00640 1 --2- 172.21.15.67:0/4225467863 >> v2:172.21.15.134:3300/0 conn(0x7ff67c0c3080 0x7ff67c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.461+0000 7ff680e00640 1 -- 172.21.15.67:0/4225467863 >> v2:172.21.15.134:3300/0 conn(0x7ff67c0c3080 msgr2=0x7ff67c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.461+0000 7ff680e00640 1 --2- 172.21.15.67:0/4225467863 >> v2:172.21.15.134:3300/0 conn(0x7ff67c0c3080 0x7ff67c1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.461+0000 7ff680e00640 1 -- 172.21.15.67:0/4225467863 >> v1:172.21.15.134:6789/0 conn(0x7ff67c06d420 legacy=0x7ff67c14c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.461+0000 7ff680e00640 1 -- 172.21.15.67:0/4225467863 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c06dee0 con 0x7ff67c14c9f0 2024-03-09T09:15:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.462+0000 7ff680e00640 1 --2- 172.21.15.67:0/4225467863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c14c9f0 0x7ff67c1540a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff67c06ee50 tx=0x7ff67402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba04a263a472e8e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.462+0000 7ff67aa00640 1 -- 172.21.15.67:0/4225467863 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674035020 con 0x7ff67c14c9f0 2024-03-09T09:15:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.463+0000 7ff67aa00640 1 -- 172.21.15.67:0/4225467863 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff674031870 con 0x7ff67c14c9f0 2024-03-09T09:15:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.463+0000 7ff67aa00640 1 -- 172.21.15.67:0/4225467863 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674031c40 con 0x7ff67c14c9f0 2024-03-09T09:15:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.463+0000 7ff681800640 1 -- 172.21.15.67:0/4225467863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c14c9f0 msgr2=0x7ff67c1540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.463+0000 7ff681800640 1 --2- 172.21.15.67:0/4225467863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c14c9f0 0x7ff67c1540a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff67c06ee50 tx=0x7ff67402b910 comp rx=0 tx=0).stop 2024-03-09T09:15:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.463+0000 7ff681800640 1 -- 172.21.15.67:0/4225467863 shutdown_connections 2024-03-09T09:15:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.463+0000 7ff681800640 1 --2- 172.21.15.67:0/4225467863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c14c9f0 0x7ff67c1540a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.463+0000 7ff681800640 1 --2- 172.21.15.67:0/4225467863 >> v2:172.21.15.134:3300/0 conn(0x7ff67c0c3080 0x7ff67c1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.463+0000 7ff681800640 1 -- 172.21.15.67:0/4225467863 >> 172.21.15.67:0/4225467863 conn(0x7ff67c0c7300 msgr2=0x7ff67c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.463+0000 7ff681800640 1 -- 172.21.15.67:0/4225467863 shutdown_connections 2024-03-09T09:15:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.463+0000 7ff681800640 1 -- 172.21.15.67:0/4225467863 wait complete. 2024-03-09T09:15:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.466+0000 7ff681800640 1 Processor -- start 2024-03-09T09:15:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.466+0000 7ff681800640 1 -- start start 2024-03-09T09:15:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.468+0000 7ff681800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c06d420 0x7ff67c1ec750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.468+0000 7ff681800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff67c0c3080 0x7ff67c0ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.468+0000 7ff681800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff67c14c9f0 0x7ff67c0abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.468+0000 7ff681800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0c8030 con 0x7ff67c06d420 2024-03-09T09:15:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.468+0000 7ff681800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0c3460 con 0x7ff67c14c9f0 2024-03-09T09:15:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.468+0000 7ff681800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c073530 con 0x7ff67c0c3080 2024-03-09T09:15:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.468+0000 7ff67be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c06d420 0x7ff67c1ec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.468+0000 7ff67be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c06d420 0x7ff67c1ec750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52376/0 (socket says 172.21.15.67:52376) 2024-03-09T09:15:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.468+0000 7ff67be00640 1 -- 172.21.15.67:0/1578862148 learned_addr learned my addr 172.21.15.67:0/1578862148 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.468+0000 7ff680e00640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff67c14c9f0 0x7ff67c0abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.468+0000 7ff67b400640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff67c0c3080 0x7ff67c0ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.469+0000 7ff67be00640 1 -- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff67c0c3080 msgr2=0x7ff67c0ab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.469+0000 7ff67be00640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff67c0c3080 0x7ff67c0ab5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.469+0000 7ff67be00640 1 -- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff67c14c9f0 msgr2=0x7ff67c0abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.469+0000 7ff67be00640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff67c14c9f0 0x7ff67c0abb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.469+0000 7ff67be00640 1 -- 172.21.15.67:0/1578862148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c1402a0 con 0x7ff67c06d420 2024-03-09T09:15:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.469+0000 7ff67be00640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c06d420 0x7ff67c1ec750 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009db0 tx=0x7ff66c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.469+0000 7ff673e00640 1 -- 172.21.15.67:0/1578862148 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c018020 con 0x7ff67c06d420 2024-03-09T09:15:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.469+0000 7ff673e00640 1 -- 172.21.15.67:0/1578862148 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c014480 con 0x7ff67c06d420 2024-03-09T09:15:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.469+0000 7ff681800640 1 -- 172.21.15.67:0/1578862148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c13ffc0 con 0x7ff67c06d420 2024-03-09T09:15:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.469+0000 7ff673e00640 1 -- 172.21.15.67:0/1578862148 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c014850 con 0x7ff67c06d420 2024-03-09T09:15:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.470+0000 7ff681800640 1 -- 172.21.15.67:0/1578862148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c1ed270 con 0x7ff67c06d420 2024-03-09T09:15:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.471+0000 7ff681800640 1 -- 172.21.15.67:0/1578862148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c04f3c0 con 0x7ff67c06d420 2024-03-09T09:15:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.472+0000 7ff673e00640 1 -- 172.21.15.67:0/1578862148 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff66c005d10 con 0x7ff67c06d420 2024-03-09T09:15:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.472+0000 7ff673e00640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff65806b4d0 0x7ff65806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.473+0000 7ff673e00640 1 -- 172.21.15.67:0/1578862148 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff66c08b070 con 0x7ff67c06d420 2024-03-09T09:15:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.473+0000 7ff67b400640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff65806b4d0 0x7ff65806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:03.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.474+0000 7ff67b400640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff65806b4d0 0x7ff65806d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff664007fd0 tx=0x7ff664009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:03.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.477+0000 7ff673e00640 1 -- 172.21.15.67:0/1578862148 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff66c022070 con 0x7ff67c06d420 2024-03-09T09:15:03.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.856+0000 7ff681800640 1 -- 172.21.15.67:0/1578862148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff67c13fd60 con 0x7ff67c06d420 2024-03-09T09:15:03.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.862+0000 7ff673e00640 1 -- 172.21.15.67:0/1578862148 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7ff66c010040 con 0x7ff67c06d420 2024-03-09T09:15:03.868 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":172,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:15:03.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.868+0000 7ff681800640 1 -- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff65806b4d0 msgr2=0x7ff65806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:03.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.869+0000 7ff681800640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff65806b4d0 0x7ff65806d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff664007fd0 tx=0x7ff664009040 comp rx=0 tx=0).stop 2024-03-09T09:15:03.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.869+0000 7ff681800640 1 -- 172.21.15.67:0/1578862148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c06d420 msgr2=0x7ff67c1ec750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:03.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.869+0000 7ff681800640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c06d420 0x7ff67c1ec750 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009db0 tx=0x7ff66c00c490 comp rx=0 tx=0).stop 2024-03-09T09:15:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.869+0000 7ff681800640 1 -- 172.21.15.67:0/1578862148 shutdown_connections 2024-03-09T09:15:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.869+0000 7ff681800640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff65806b4d0 0x7ff65806d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.869+0000 7ff681800640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff67c14c9f0 0x7ff67c0abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.869+0000 7ff681800640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff67c0c3080 0x7ff67c0ab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.869+0000 7ff681800640 1 --2- 172.21.15.67:0/1578862148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff67c06d420 0x7ff67c1ec750 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.869+0000 7ff681800640 1 -- 172.21.15.67:0/1578862148 >> 172.21.15.67:0/1578862148 conn(0x7ff67c0c7300 msgr2=0x7ff67c151cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.869+0000 7ff681800640 1 -- 172.21.15.67:0/1578862148 shutdown_connections 2024-03-09T09:15:03.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:03.870+0000 7ff681800640 1 -- 172.21.15.67:0/1578862148 wait complete. 2024-03-09T09:15:03.908 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:15:03.908 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:15:03.908 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:15:04.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.163+0000 7f5912600640 1 Processor -- start 2024-03-09T09:15:04.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.164+0000 7f5912600640 1 -- start start 2024-03-09T09:15:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.165+0000 7f5912600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f590c0730c0 0x7f590c0aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.165+0000 7f5912600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0734a0 0x7f590c0c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f5912600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c152e00 con 0x7f590c0c33c0 2024-03-09T09:15:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f5912600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590c0aa960 con 0x7f590c0730c0 2024-03-09T09:15:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f5912600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c1530e0 con 0x7f590c0734a0 2024-03-09T09:15:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f590c0730c0 0x7f590c0aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f590c0c33c0 0x7f590c06d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36704/0 (socket says 172.21.15.67:36704) 2024-03-09T09:15:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590be00640 1 -- 172.21.15.67:0/2715423761 learned_addr learned my addr 172.21.15.67:0/2715423761 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:15:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590aa00640 1 --2- 172.21.15.67:0/2715423761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0734a0 0x7f590c0c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590aa00640 1 -- 172.21.15.67:0/2715423761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0734a0 msgr2=0x7f590c0c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:15:04.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590aa00640 1 -- 172.21.15.67:0/2715423761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0734a0 msgr2=0x7f590c0c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:04.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590aa00640 1 --2- 172.21.15.67:0/2715423761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0734a0 0x7f590c0c2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:04.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590aa00640 1 --2- 172.21.15.67:0/2715423761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0734a0 0x7f590c0c2e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:04.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590a000640 1 -- 172.21.15.67:0/2715423761 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051303596 0 0) 0x7f590c152e00 con 0x7f590c0c33c0 2024-03-09T09:15:04.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590a000640 1 -- 172.21.15.67:0/2715423761 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58f0003520 con 0x7f590c0c33c0 2024-03-09T09:15:04.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590b400640 1 -- 172.21.15.67:0/2715423761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0734a0 msgr2=0x7f590c0c2e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:04.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590b400640 1 --2- 172.21.15.67:0/2715423761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0734a0 0x7f590c0c2e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590b400640 1 -- 172.21.15.67:0/2715423761 >> v1:172.21.15.67:6789/0 conn(0x7f590c0c33c0 legacy=0x7f590c06d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.166+0000 7f590b400640 1 -- 172.21.15.67:0/2715423761 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c1533c0 con 0x7f590c0730c0 2024-03-09T09:15:04.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.168+0000 7f590b400640 1 --2- 172.21.15.67:0/2715423761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0730c0 0x7f590c0aa420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0097a0 tx=0x7f58fc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10207cc5495b34 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.168+0000 7f590a000640 1 -- 172.21.15.67:0/2715423761 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0315a0 con 0x7f590c0730c0 2024-03-09T09:15:04.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.168+0000 7f590a000640 1 -- 172.21.15.67:0/2715423761 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc037070 con 0x7f590c0730c0 2024-03-09T09:15:04.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.168+0000 7f590a000640 1 -- 172.21.15.67:0/2715423761 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc031940 con 0x7f590c0730c0 2024-03-09T09:15:04.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.168+0000 7f5912600640 1 -- 172.21.15.67:0/2715423761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0730c0 msgr2=0x7f590c0aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.168+0000 7f5912600640 1 --2- 172.21.15.67:0/2715423761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0730c0 0x7f590c0aa420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0097a0 tx=0x7f58fc02b840 comp rx=0 tx=0).stop 2024-03-09T09:15:04.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.168+0000 7f5912600640 1 -- 172.21.15.67:0/2715423761 shutdown_connections 2024-03-09T09:15:04.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.168+0000 7f5912600640 1 --2- 172.21.15.67:0/2715423761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0734a0 0x7f590c0c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.168+0000 7f5912600640 1 --2- 172.21.15.67:0/2715423761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0730c0 0x7f590c0aa420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.168+0000 7f5912600640 1 -- 172.21.15.67:0/2715423761 >> 172.21.15.67:0/2715423761 conn(0x7f590c0c7370 msgr2=0x7f590c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:04.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.169+0000 7f5912600640 1 -- 172.21.15.67:0/2715423761 shutdown_connections 2024-03-09T09:15:04.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.169+0000 7f5912600640 1 -- 172.21.15.67:0/2715423761 wait complete. 2024-03-09T09:15:04.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.170+0000 7f5912600640 1 Processor -- start 2024-03-09T09:15:04.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.170+0000 7f5912600640 1 -- start start 2024-03-09T09:15:04.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.171+0000 7f5912600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0730c0 0x7f590c14a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.171+0000 7f5912600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0734a0 0x7f590c154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.171+0000 7f5912600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0c33c0 0x7f590c1eb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.171+0000 7f5912600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c0aaef0 con 0x7f590c0734a0 2024-03-09T09:15:04.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.171+0000 7f5912600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c153980 con 0x7f590c0730c0 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.171+0000 7f5912600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c0ab070 con 0x7f590c0c33c0 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.171+0000 7f590aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0734a0 0x7f590c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.171+0000 7f590b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0730c0 0x7f590c14a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.171+0000 7f590aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0734a0 0x7f590c154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52404/0 (socket says 172.21.15.67:52404) 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590aa00640 1 -- 172.21.15.67:0/628830401 learned_addr learned my addr 172.21.15.67:0/628830401 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0c33c0 0x7f590c1eb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590be00640 1 -- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0c33c0 msgr2=0x7f590c1eb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590be00640 1 -- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0c33c0 msgr2=0x7f590c1eb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590be00640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0c33c0 0x7f590c1eb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590be00640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0c33c0 0x7f590c1eb720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590b400640 1 -- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0c33c0 msgr2=0x7f590c1eb720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590b400640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0c33c0 0x7f590c1eb720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590b400640 1 -- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0734a0 msgr2=0x7f590c154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590b400640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0734a0 0x7f590c154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590b400640 1 -- 172.21.15.67:0/628830401 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c1ec420 con 0x7f590c0730c0 2024-03-09T09:15:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.172+0000 7f590b400640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0730c0 0x7f590c14a7f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f58fc033000 tx=0x7f58fc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.173+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc032630 con 0x7f590c0730c0 2024-03-09T09:15:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.173+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc0327d0 con 0x7f590c0730c0 2024-03-09T09:15:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.173+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc032b80 con 0x7f590c0730c0 2024-03-09T09:15:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.173+0000 7f5912600640 1 -- 172.21.15.67:0/628830401 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c1eec90 con 0x7f590c0730c0 2024-03-09T09:15:04.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.173+0000 7f5912600640 1 -- 172.21.15.67:0/628830401 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c1ef250 con 0x7f590c0730c0 2024-03-09T09:15:04.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.174+0000 7f5912600640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c154c90 0x7f590c1ebc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.174+0000 7f5912600640 1 -- 172.21.15.67:0/628830401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c14ad30 con 0x7f590c154c90 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.174+0000 7f590aa00640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c154c90 0x7f590c1ebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.175+0000 7f590aa00640 1 -- 172.21.15.67:0/628830401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f590c1ec1a0 con 0x7f590c154c90 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.175+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f58fc04e430 con 0x7f590c0730c0 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.175+0000 7f590aa00640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c154c90 0x7f590c1ebc60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f58f8010890 tx=0x7f58f8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.176+0000 7f5903400640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58ec06b4d0 0x7f58ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.176+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f58fc02f030 con 0x7f590c0730c0 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.176+0000 7f590be00640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58ec06b4d0 0x7f58ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.176+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8018070 con 0x7f590c154c90 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.176+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f58f800eaa0 con 0x7f590c154c90 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.176+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c154c90 msgr2=0x7f590c1ebc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.176+0000 7f5903400640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c154c90 0x7f590c1ebc60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f58f8010890 tx=0x7f58f8010d50 comp rx=0 tx=0).stop 2024-03-09T09:15:04.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.176+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c154c90 msgr2=0x7f590c1ebc60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:04.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.176+0000 7f5903400640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c154c90 0x7f590c1ebc60 secure :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f58f8010890 tx=0x7f58f8010d50 comp rx=0 tx=0).stop 2024-03-09T09:15:04.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.177+0000 7f590be00640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58ec06b4d0 0x7f58ec06d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5904001560 tx=0x7f5904009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:15:04.188 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:15:04.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.184+0000 7f5912600640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c146880 0x7f590c148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.184+0000 7f5912600640 1 -- 172.21.15.67:0/628830401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c075a00 con 0x7f590c146880 2024-03-09T09:15:04.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.184+0000 7f590aa00640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c146880 0x7f590c148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.184+0000 7f590aa00640 1 -- 172.21.15.67:0/628830401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f590c0ac360 con 0x7f590c146880 2024-03-09T09:15:04.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.185+0000 7f590aa00640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c146880 0x7f590c148940 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f58f8010890 tx=0x7f58f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.185+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8014630 con 0x7f590c146880 2024-03-09T09:15:04.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.185+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f58f8014630 con 0x7f590c146880 2024-03-09T09:15:04.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.185+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c146880 msgr2=0x7f590c148940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.185+0000 7f5903400640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c146880 0x7f590c148940 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f58f8010890 tx=0x7f58f80046c0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.185+0000 7f5903400640 1 -- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c146880 msgr2=0x7f590c148940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:04.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.185+0000 7f5903400640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c146880 0x7f590c148940 secure :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f58f8010890 tx=0x7f58f80046c0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.191 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 172, 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:15:04.192 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:04.193 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:15:04.195 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:04.196 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.197 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.198 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.199 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:04.200 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:04.201 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:04.202 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:04.203 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.205 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:15:04.205 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:15:04.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.186+0000 7f5912600640 1 -- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58ec06b4d0 msgr2=0x7f58ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.186+0000 7f5912600640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58ec06b4d0 0x7f58ec06d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5904001560 tx=0x7f5904009040 comp rx=0 tx=0).stop 2024-03-09T09:15:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.186+0000 7f5912600640 1 -- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0730c0 msgr2=0x7f590c14a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.186+0000 7f5912600640 1 --2- 172.21.15.67:0/628830401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0730c0 0x7f590c14a7f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f58fc033000 tx=0x7f58fc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.186+0000 7f590be00640 1 -- 172.21.15.67:0/628830401 reap_dead start 2024-03-09T09:15:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.186+0000 7f590be00640 1 -- 172.21.15.67:0/628830401 reap_dead start 2024-03-09T09:15:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.186+0000 7f5912600640 1 -- 172.21.15.67:0/628830401 shutdown_connections 2024-03-09T09:15:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.186+0000 7f5912600640 1 -- 172.21.15.67:0/628830401 >> 172.21.15.67:0/628830401 conn(0x7f590c0c7370 msgr2=0x7f590c0c3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.186+0000 7f5912600640 1 -- 172.21.15.67:0/628830401 shutdown_connections 2024-03-09T09:15:04.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.186+0000 7f5912600640 1 -- 172.21.15.67:0/628830401 wait complete. 2024-03-09T09:15:04.208 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:15:04.208 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:15:04.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:15:04.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.458+0000 7fef78c00640 1 Processor -- start 2024-03-09T09:15:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.458+0000 7fef78c00640 1 -- start start 2024-03-09T09:15:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.459+0000 7fef78c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fef740c2b70 0x7fef740c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.459+0000 7fef78c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 0x7fef741632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.459+0000 7fef78c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef740c31f0 con 0x7fef740c4140 2024-03-09T09:15:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.459+0000 7fef78c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef7406e380 con 0x7fef740c2b70 2024-03-09T09:15:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.459+0000 7fef78c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef740c37b0 con 0x7fef740aa3b0 2024-03-09T09:15:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.460+0000 7fef73e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 0x7fef741632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.460+0000 7fef6aa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fef740c2b70 0x7fef740c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.460+0000 7fef73e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 0x7fef741632d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:58490/0 (socket says 172.21.15.67:58490) 2024-03-09T09:15:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.460+0000 7fef73e00640 1 -- 172.21.15.67:0/2201276625 learned_addr learned my addr 172.21.15.67:0/2201276625 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:04.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.461+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453240079 0 0) 0x7fef740c31f0 con 0x7fef740c4140 2024-03-09T09:15:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.656+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef58003500 con 0x7fef740c4140 2024-03-09T09:15:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.656+0000 7fef73e00640 1 -- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 msgr2=0x7fef741632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:15:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.656+0000 7fef73e00640 1 -- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 msgr2=0x7fef741632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.656+0000 7fef73e00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 0x7fef741632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.656+0000 7fef73e00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 0x7fef741632d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1818897895 0 0) 0x7fef58003500 con 0x7fef740c4140 2024-03-09T09:15:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 msgr2=0x7fef741632d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 0x7fef741632d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 >> v2:172.21.15.134:3300/0 conn(0x7fef740c2b70 msgr2=0x7fef740c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 --2- 172.21.15.67:0/2201276625 >> v2:172.21.15.134:3300/0 conn(0x7fef740c2b70 0x7fef740c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef740c34d0 con 0x7fef740c4140 2024-03-09T09:15:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fef640019f0 con 0x7fef740c4140 2024-03-09T09:15:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 >> v1:172.21.15.134:6789/0 conn(0x7fef740c4140 legacy=0x7fef7406dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef58003b80 0x7fef58003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 0x7fef58004710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef58004d90 0x7fef58005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef740c37b0 con 0x7fef58003b80 2024-03-09T09:15:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5800bbb0 con 0x7fef58004d90 2024-03-09T09:15:04.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.657+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5800bdb0 con 0x7fef740aa3b0 2024-03-09T09:15:04.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.658+0000 7fef6aa00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 0x7fef58004710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.658+0000 7fef73400640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef58003b80 0x7fef58003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.659+0000 7fef73e00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef58004d90 0x7fef58005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.659+0000 7fef6aa00640 1 -- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef58004d90 msgr2=0x7fef58005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:04.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.659+0000 7fef6aa00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef58004d90 0x7fef58005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.659+0000 7fef6aa00640 1 -- 172.21.15.67:0/2201276625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef58003b80 msgr2=0x7fef58003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:04.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.659+0000 7fef6aa00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef58003b80 0x7fef58003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.659+0000 7fef6aa00640 1 -- 172.21.15.67:0/2201276625 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58003500 con 0x7fef740aa3b0 2024-03-09T09:15:04.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.661+0000 7fef6aa00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 0x7fef58004710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009650 tx=0x7fef58003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5834fb749c30afc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.662+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c03b070 con 0x7fef740aa3b0 2024-03-09T09:15:04.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.662+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c02e920 con 0x7fef740aa3b0 2024-03-09T09:15:04.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.662+0000 7fef72a00640 1 -- 172.21.15.67:0/2201276625 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c009690 con 0x7fef740aa3b0 2024-03-09T09:15:04.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.662+0000 7fef78c00640 1 -- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 msgr2=0x7fef58004710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.662+0000 7fef78c00640 1 --2- 172.21.15.67:0/2201276625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef740aa3b0 0x7fef58004710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009650 tx=0x7fef58003890 comp rx=0 tx=0).stop 2024-03-09T09:15:04.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.663+0000 7fef73e00640 1 -- 172.21.15.67:0/2201276625 reap_dead start 2024-03-09T09:15:04.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.663+0000 7fef78c00640 1 -- 172.21.15.67:0/2201276625 shutdown_connections 2024-03-09T09:15:04.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.663+0000 7fef78c00640 1 -- 172.21.15.67:0/2201276625 >> 172.21.15.67:0/2201276625 conn(0x7fef7406b490 msgr2=0x7fef7406b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:04.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.663+0000 7fef78c00640 1 -- 172.21.15.67:0/2201276625 shutdown_connections 2024-03-09T09:15:04.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.663+0000 7fef78c00640 1 -- 172.21.15.67:0/2201276625 wait complete. 2024-03-09T09:15:04.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.666+0000 7fef78c00640 1 Processor -- start 2024-03-09T09:15:04.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.666+0000 7fef78c00640 1 -- start start 2024-03-09T09:15:04.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.667+0000 7fef78c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef740aa3b0 0x7fef741642b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.667+0000 7fef78c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef740c08c0 0x7fef740713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.667+0000 7fef78c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef74071930 0x7fef74071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.667+0000 7fef78c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7406d750 con 0x7fef740c08c0 2024-03-09T09:15:04.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.667+0000 7fef78c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef740aa840 con 0x7fef740aa3b0 2024-03-09T09:15:04.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.667+0000 7fef78c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7408cc60 con 0x7fef74071930 2024-03-09T09:15:04.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef6aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef740c08c0 0x7fef740713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef6aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef740c08c0 0x7fef740713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52432/0 (socket says 172.21.15.67:52432) 2024-03-09T09:15:04.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef6aa00640 1 -- 172.21.15.67:0/1766288710 learned_addr learned my addr 172.21.15.67:0/1766288710 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:04.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef73400640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef740aa3b0 0x7fef741642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef73e00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef74071930 0x7fef74071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef6aa00640 1 -- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef74071930 msgr2=0x7fef74071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef6aa00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef74071930 0x7fef74071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef6aa00640 1 -- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef740aa3b0 msgr2=0x7fef741642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef6aa00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef740aa3b0 0x7fef741642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef6aa00640 1 -- 172.21.15.67:0/1766288710 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef740c6b80 con 0x7fef740c08c0 2024-03-09T09:15:04.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef73e00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fef74071930 0x7fef74071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:15:04.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.668+0000 7fef73400640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef740aa3b0 0x7fef741642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:15:04.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.669+0000 7fef6aa00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef740c08c0 0x7fef740713f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fef5c039040 tx=0x7fef5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.669+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c03b040 con 0x7fef740c08c0 2024-03-09T09:15:04.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.669+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c044070 con 0x7fef740c08c0 2024-03-09T09:15:04.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.669+0000 7fef78c00640 1 -- 172.21.15.67:0/1766288710 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef74070910 con 0x7fef740c08c0 2024-03-09T09:15:04.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.669+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c003610 con 0x7fef740c08c0 2024-03-09T09:15:04.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.669+0000 7fef78c00640 1 -- 172.21.15.67:0/1766288710 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef74073b70 con 0x7fef740c08c0 2024-03-09T09:15:04.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.671+0000 7fef78c00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef74067b30 0x7fef740724b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.671+0000 7fef78c00640 1 -- 172.21.15.67:0/1766288710 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef740c8130 con 0x7fef74067b30 2024-03-09T09:15:04.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.671+0000 7fef73400640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef74067b30 0x7fef740724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.672+0000 7fef73400640 1 -- 172.21.15.67:0/1766288710 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef74071020 con 0x7fef74067b30 2024-03-09T09:15:04.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.672+0000 7fef73400640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef74067b30 0x7fef740724b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef64003f80 tx=0x7fef640054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.673+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64010990 con 0x7fef74067b30 2024-03-09T09:15:04.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.673+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fef5c0037b0 con 0x7fef740c08c0 2024-03-09T09:15:04.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.673+0000 7fef6be00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fef5006b3d0 0x7fef5006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.674+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fef5c0ab160 con 0x7fef740c08c0 2024-03-09T09:15:04.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.674+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fef64010b30 con 0x7fef74067b30 2024-03-09T09:15:04.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.674+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef74067b30 msgr2=0x7fef740724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.674+0000 7fef6be00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef74067b30 0x7fef740724b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef64003f80 tx=0x7fef640054e0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.674+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef74067b30 msgr2=0x7fef740724b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:04.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.674+0000 7fef6be00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef74067b30 0x7fef740724b0 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef64003f80 tx=0x7fef640054e0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.674+0000 7fef73e00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fef5006b3d0 0x7fef5006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.675+0000 7fef73e00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fef5006b3d0 0x7fef5006d890 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fef7406c2b0 tx=0x7fef6c00a040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.682+0000 7fef78c00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef741ff910 0x7fef741ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.682+0000 7fef78c00640 1 -- 172.21.15.67:0/1766288710 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7404f3c0 con 0x7fef741ff910 2024-03-09T09:15:04.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.683+0000 7fef73400640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef741ff910 0x7fef741ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.683+0000 7fef73400640 1 -- 172.21.15.67:0/1766288710 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fef740c2950 con 0x7fef741ff910 2024-03-09T09:15:04.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.684+0000 7fef73400640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef741ff910 0x7fef741ffd20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef64003f80 tx=0x7fef74162850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.685+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64014ae0 con 0x7fef741ff910 2024-03-09T09:15:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.685+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fef64014ae0 con 0x7fef741ff910 2024-03-09T09:15:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.685+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef741ff910 msgr2=0x7fef741ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.685+0000 7fef6be00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef741ff910 0x7fef741ffd20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef64003f80 tx=0x7fef74162850 comp rx=0 tx=0).stop 2024-03-09T09:15:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.685+0000 7fef6be00640 1 -- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef741ff910 msgr2=0x7fef741ffd20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:04.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.685+0000 7fef6be00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef741ff910 0x7fef741ffd20 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef64003f80 tx=0x7fef74162850 comp rx=0 tx=0).stop 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 172, 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:15:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:15:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:15:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:15:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:15:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:15:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:04.702 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:15:04.704 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:15:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.686+0000 7fef78c00640 1 -- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fef5006b3d0 msgr2=0x7fef5006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.686+0000 7fef78c00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fef5006b3d0 0x7fef5006d890 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fef7406c2b0 tx=0x7fef6c00a040 comp rx=0 tx=0).stop 2024-03-09T09:15:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.686+0000 7fef78c00640 1 -- 172.21.15.67:0/1766288710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef740c08c0 msgr2=0x7fef740713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.686+0000 7fef78c00640 1 --2- 172.21.15.67:0/1766288710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef740c08c0 0x7fef740713f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fef5c039040 tx=0x7fef5c004730 comp rx=0 tx=0).stop 2024-03-09T09:15:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.686+0000 7fef73e00640 1 -- 172.21.15.67:0/1766288710 reap_dead start 2024-03-09T09:15:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.686+0000 7fef73e00640 1 -- 172.21.15.67:0/1766288710 reap_dead start 2024-03-09T09:15:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.686+0000 7fef78c00640 1 -- 172.21.15.67:0/1766288710 shutdown_connections 2024-03-09T09:15:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.686+0000 7fef78c00640 1 -- 172.21.15.67:0/1766288710 >> 172.21.15.67:0/1766288710 conn(0x7fef7406b490 msgr2=0x7fef7408a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.686+0000 7fef78c00640 1 -- 172.21.15.67:0/1766288710 shutdown_connections 2024-03-09T09:15:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.686+0000 7fef78c00640 1 -- 172.21.15.67:0/1766288710 wait complete. 2024-03-09T09:15:04.708 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:15:04.708 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:15:04.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:15:04.751 INFO:tasks.ceph.mds.c.smithi067.stderr:2024-03-09T09:15:04.746+0000 7f88ba800640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 65851) UID: 0 2024-03-09T09:15:04.752 INFO:tasks.ceph.mds.a.smithi067.stderr:2024-03-09T09:15:04.746+0000 7f9e6c000640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 65851) UID: 0 2024-03-09T09:15:04.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.900+0000 7fd06f400640 1 Processor -- start 2024-03-09T09:15:04.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.901+0000 7fd06f400640 1 -- start start 2024-03-09T09:15:04.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.902+0000 7fd06f400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd068153460 0x7fd068155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.902+0000 7fd06f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0680c8c60 0x7fd0680c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.902+0000 7fd06f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 0x7fd0680caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.902+0000 7fd06f400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0680ca590 con 0x7fd0680c8c60 2024-03-09T09:15:04.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.902+0000 7fd06f400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06806b040 con 0x7fd068153460 2024-03-09T09:15:04.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.902+0000 7fd06f400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd068156710 con 0x7fd0680c9d50 2024-03-09T09:15:04.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.902+0000 7fd067e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0680c8c60 0x7fd0680c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.902+0000 7fd067e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0680c8c60 0x7fd0680c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52442/0 (socket says 172.21.15.67:52442) 2024-03-09T09:15:04.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.902+0000 7fd067e00640 1 -- 172.21.15.67:0/954498769 learned_addr learned my addr 172.21.15.67:0/954498769 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:04.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.902+0000 7fd06ca00640 1 --2- 172.21.15.67:0/954498769 >> v2:172.21.15.134:3300/0 conn(0x7fd068153460 0x7fd068155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.902+0000 7fd06d400640 1 --2- 172.21.15.67:0/954498769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 0x7fd0680caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.903+0000 7fd067e00640 1 -- 172.21.15.67:0/954498769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 msgr2=0x7fd0680caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:04.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.903+0000 7fd067e00640 1 --2- 172.21.15.67:0/954498769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 0x7fd0680caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.903+0000 7fd067e00640 1 -- 172.21.15.67:0/954498769 >> v2:172.21.15.134:3300/0 conn(0x7fd068153460 msgr2=0x7fd068155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:04.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.903+0000 7fd067e00640 1 --2- 172.21.15.67:0/954498769 >> v2:172.21.15.134:3300/0 conn(0x7fd068153460 0x7fd068155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.903+0000 7fd067e00640 1 -- 172.21.15.67:0/954498769 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0681569f0 con 0x7fd0680c8c60 2024-03-09T09:15:04.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.904+0000 7fd067e00640 1 --2- 172.21.15.67:0/954498769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0680c8c60 0x7fd0680c9810 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd0680c80a0 tx=0x7fd05402b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40250ffd81e6e1e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.905+0000 7fd067400640 1 -- 172.21.15.67:0/954498769 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0540317f0 con 0x7fd0680c8c60 2024-03-09T09:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.905+0000 7fd067400640 1 -- 172.21.15.67:0/954498769 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd054037070 con 0x7fd0680c8c60 2024-03-09T09:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.905+0000 7fd067400640 1 -- 172.21.15.67:0/954498769 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054031bb0 con 0x7fd0680c8c60 2024-03-09T09:15:04.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.906+0000 7fd06f400640 1 -- 172.21.15.67:0/954498769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0680c8c60 msgr2=0x7fd0680c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.906+0000 7fd06f400640 1 --2- 172.21.15.67:0/954498769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0680c8c60 0x7fd0680c9810 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd0680c80a0 tx=0x7fd05402b890 comp rx=0 tx=0).stop 2024-03-09T09:15:04.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.906+0000 7fd06f400640 1 -- 172.21.15.67:0/954498769 shutdown_connections 2024-03-09T09:15:04.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.906+0000 7fd06f400640 1 --2- 172.21.15.67:0/954498769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 0x7fd0680caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.906+0000 7fd06f400640 1 --2- 172.21.15.67:0/954498769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0680c8c60 0x7fd0680c9810 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.906+0000 7fd06f400640 1 --2- 172.21.15.67:0/954498769 >> v2:172.21.15.134:3300/0 conn(0x7fd068153460 0x7fd068155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.906+0000 7fd06f400640 1 -- 172.21.15.67:0/954498769 >> 172.21.15.67:0/954498769 conn(0x7fd0680c8170 msgr2=0x7fd0680c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:04.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.906+0000 7fd06f400640 1 -- 172.21.15.67:0/954498769 shutdown_connections 2024-03-09T09:15:04.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.906+0000 7fd06f400640 1 -- 172.21.15.67:0/954498769 wait complete. 2024-03-09T09:15:04.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.908+0000 7fd06f400640 1 Processor -- start 2024-03-09T09:15:04.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.908+0000 7fd06f400640 1 -- start start 2024-03-09T09:15:04.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.910+0000 7fd06f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0680c8c60 0x7fd068154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.910+0000 7fd06f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 0x7fd068148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.910+0000 7fd06f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd068153460 0x7fd0681ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.910+0000 7fd06f400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd068153840 con 0x7fd068153460 2024-03-09T09:15:04.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.910+0000 7fd06f400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd068156400 con 0x7fd0680c8c60 2024-03-09T09:15:04.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.910+0000 7fd06f400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680caf60 con 0x7fd0680c9d50 2024-03-09T09:15:04.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.910+0000 7fd06ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0680c8c60 0x7fd068154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.910+0000 7fd06ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0680c8c60 0x7fd068154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:44796/0 (socket says 172.21.15.67:44796) 2024-03-09T09:15:04.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.910+0000 7fd06ca00640 1 -- 172.21.15.67:0/2031033974 learned_addr learned my addr 172.21.15.67:0/2031033974 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:04.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.910+0000 7fd067e00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 0x7fd068148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.910+0000 7fd06d400640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd068153460 0x7fd0681ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.911+0000 7fd067e00640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 msgr2=0x7fd068148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:15:04.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.911+0000 7fd067e00640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 msgr2=0x7fd068148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:15:04.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.911+0000 7fd067e00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 0x7fd068148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:04.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.911+0000 7fd067e00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 0x7fd068148d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:04.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.911+0000 7fd06ca00640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 msgr2=0x7fd068148d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:15:04.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.911+0000 7fd06ca00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0680c9d50 0x7fd068148d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.911+0000 7fd06ca00640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd068153460 msgr2=0x7fd0681ecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:04.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.911+0000 7fd06ca00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd068153460 0x7fd0681ecb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:04.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.911+0000 7fd06ca00640 1 -- 172.21.15.67:0/2031033974 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0680accb0 con 0x7fd0680c8c60 2024-03-09T09:15:04.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.911+0000 7fd06d400640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd068153460 0x7fd0681ecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:15:04.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.912+0000 7fd06ca00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0680c8c60 0x7fd068154600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00d5f0 tx=0x7fd05c00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.912+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0125f0 con 0x7fd0680c8c60 2024-03-09T09:15:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.912+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c019070 con 0x7fd0680c8c60 2024-03-09T09:15:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.912+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0129d0 con 0x7fd0680c8c60 2024-03-09T09:15:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.912+0000 7fd06f400640 1 -- 172.21.15.67:0/2031033974 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd068153b60 con 0x7fd0680c8c60 2024-03-09T09:15:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.912+0000 7fd06f400640 1 -- 172.21.15.67:0/2031033974 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0681ef940 con 0x7fd0680c8c60 2024-03-09T09:15:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.914+0000 7fd06f400640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 0x7fd068153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.914+0000 7fd06f400640 1 -- 172.21.15.67:0/2031033974 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0681efc20 con 0x7fd0681555f0 2024-03-09T09:15:04.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.914+0000 7fd067e00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 0x7fd068153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.914+0000 7fd067e00640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 msgr2=0x7fd068153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-03-09T09:15:04.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.914+0000 7fd067e00640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 msgr2=0x7fd068153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:15:04.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.914+0000 7fd067e00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 0x7fd068153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:04.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.914+0000 7fd067e00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 0x7fd068153d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:04.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.915+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd05c012b70 con 0x7fd0680c8c60 2024-03-09T09:15:04.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.915+0000 7fd064c00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd04806b4d0 0x7fd04806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:04.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.916+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd05c08a910 con 0x7fd0680c8c60 2024-03-09T09:15:04.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.916+0000 7fd06d400640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd04806b4d0 0x7fd04806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:04.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:04.916+0000 7fd06d400640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd04806b4d0 0x7fd04806d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd060009fd0 tx=0x7fd060009790 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:05.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.115+0000 7fd067e00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 0x7fd068153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:05.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.116+0000 7fd067e00640 1 -- 172.21.15.67:0/2031033974 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd068154270 con 0x7fd0681555f0 2024-03-09T09:15:05.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.117+0000 7fd067e00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 0x7fd068153d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd06806dac0 tx=0x7fd054004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:05.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.117+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05404a070 con 0x7fd0681555f0 2024-03-09T09:15:05.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.118+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd054037040 con 0x7fd0681555f0 2024-03-09T09:15:05.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.118+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 msgr2=0x7fd068153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:05.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.118+0000 7fd064c00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 0x7fd068153d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd06806dac0 tx=0x7fd054004730 comp rx=0 tx=0).stop 2024-03-09T09:15:05.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.118+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 msgr2=0x7fd068153d30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:05.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.118+0000 7fd064c00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0681555f0 0x7fd068153d30 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd06806dac0 tx=0x7fd054004730 comp rx=0 tx=0).stop 2024-03-09T09:15:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.132+0000 7fd06f400640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd068157390 0x7fd06814bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:05.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.132+0000 7fd06f400640 1 -- 172.21.15.67:0/2031033974 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680752b0 con 0x7fd068157390 2024-03-09T09:15:05.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.133+0000 7fd067e00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd068157390 0x7fd06814bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:05.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.133+0000 7fd067e00640 1 -- 172.21.15.67:0/2031033974 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd068075a00 con 0x7fd068157390 2024-03-09T09:15:05.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.134+0000 7fd067e00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd068157390 0x7fd06814bfa0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd068050170 tx=0x7fd0540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:05.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.134+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05404a040 con 0x7fd068157390 2024-03-09T09:15:05.140 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:15:05.140 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:15:05.140 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:15:05.140 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:15:05.140 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 172, 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 4, 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:05.141 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:05.142 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:15:05.143 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:05.144 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:05.145 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:05.146 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:05.147 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:05.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:05.149 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:15:05.149 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:15:05.149 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:05.149 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:05.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:05.149 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:05.149 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:15:05.149 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:05.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:05.151 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:05.152 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:05.152 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:05.152 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:05.152 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:05.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:05.152 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:15:05.152 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:15:05.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.135+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7fd05402e070 con 0x7fd068157390 2024-03-09T09:15:05.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.135+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd068157390 msgr2=0x7fd06814bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:05.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.135+0000 7fd064c00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd068157390 0x7fd06814bfa0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd068050170 tx=0x7fd0540046c0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.135+0000 7fd064c00640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd068157390 msgr2=0x7fd06814bfa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:05.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.135+0000 7fd064c00640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd068157390 0x7fd06814bfa0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd068050170 tx=0x7fd0540046c0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.135+0000 7fd06f400640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd04806b4d0 msgr2=0x7fd04806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.136+0000 7fd06f400640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd04806b4d0 0x7fd04806d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd060009fd0 tx=0x7fd060009790 comp rx=0 tx=0).stop 2024-03-09T09:15:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.136+0000 7fd06f400640 1 -- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0680c8c60 msgr2=0x7fd068154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.136+0000 7fd06f400640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0680c8c60 0x7fd068154600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00d5f0 tx=0x7fd05c00dab0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.136+0000 7fd06d400640 1 -- 172.21.15.67:0/2031033974 reap_dead start 2024-03-09T09:15:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.136+0000 7fd06f400640 1 -- 172.21.15.67:0/2031033974 shutdown_connections 2024-03-09T09:15:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.136+0000 7fd06f400640 1 --2- 172.21.15.67:0/2031033974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0680c8c60 0x7fd068154600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.136+0000 7fd06f400640 1 -- 172.21.15.67:0/2031033974 >> 172.21.15.67:0/2031033974 conn(0x7fd0680c8170 msgr2=0x7fd0680ab190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.136+0000 7fd06f400640 1 -- 172.21.15.67:0/2031033974 shutdown_connections 2024-03-09T09:15:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.136+0000 7fd06f400640 1 -- 172.21.15.67:0/2031033974 wait complete. 2024-03-09T09:15:05.158 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:15:05.158 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:15:05.158 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:15:05.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.406+0000 7f7408600640 1 Processor -- start 2024-03-09T09:15:05.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.406+0000 7f7408600640 1 -- start start 2024-03-09T09:15:05.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.408+0000 7f7408600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f74000ac730 0x7f7400071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:05.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.408+0000 7f7408600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74000c22a0 0x7f74000acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:05.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.408+0000 7f7408600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7400153d40 con 0x7f7400157650 2024-03-09T09:15:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.408+0000 7f7408600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7400070ee0 con 0x7f74000ac730 2024-03-09T09:15:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.408+0000 7f7408600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7400154d00 con 0x7f74000c22a0 2024-03-09T09:15:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.408+0000 7f7405c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f74000ac730 0x7f7400071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.408+0000 7f7405c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f74000ac730 0x7f7400071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52456/0 (socket says 172.21.15.67:52456) 2024-03-09T09:15:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.408+0000 7f7406600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f7400157650 0x7f7400157b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36712/0 (socket says 172.21.15.67:36712) 2024-03-09T09:15:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.408+0000 7f7405c00640 1 -- 172.21.15.67:0/1009234303 learned_addr learned my addr 172.21.15.67:0/1009234303 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:05.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.408+0000 7f7405200640 1 --2- 172.21.15.67:0/1009234303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74000c22a0 0x7f74000acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:05.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.409+0000 7f7405c00640 1 -- 172.21.15.67:0/1009234303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74000c22a0 msgr2=0x7f74000acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:05.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.409+0000 7f7405c00640 1 --2- 172.21.15.67:0/1009234303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74000c22a0 0x7f74000acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.409+0000 7f7405c00640 1 -- 172.21.15.67:0/1009234303 >> v1:172.21.15.67:6789/0 conn(0x7f7400157650 legacy=0x7f7400157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:05.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.409+0000 7f7405c00640 1 -- 172.21.15.67:0/1009234303 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74000c2790 con 0x7f74000ac730 2024-03-09T09:15:05.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.409+0000 7f7405200640 1 --2- 172.21.15.67:0/1009234303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74000c22a0 0x7f74000acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:15:05.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.410+0000 7f7405c00640 1 --2- 172.21.15.67:0/1009234303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74000ac730 0x7f7400071190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009870 tx=0x7f73f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b6d077a9861200 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:05.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.410+0000 7f73ffe00640 1 -- 172.21.15.67:0/1009234303 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4035020 con 0x7f74000ac730 2024-03-09T09:15:05.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.410+0000 7f73ffe00640 1 -- 172.21.15.67:0/1009234303 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f4031870 con 0x7f74000ac730 2024-03-09T09:15:05.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.410+0000 7f73ffe00640 1 -- 172.21.15.67:0/1009234303 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4031c40 con 0x7f74000ac730 2024-03-09T09:15:05.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.411+0000 7f7408600640 1 -- 172.21.15.67:0/1009234303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74000ac730 msgr2=0x7f7400071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:05.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.411+0000 7f7408600640 1 --2- 172.21.15.67:0/1009234303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74000ac730 0x7f7400071190 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009870 tx=0x7f73f402b910 comp rx=0 tx=0).stop 2024-03-09T09:15:05.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.411+0000 7f7408600640 1 -- 172.21.15.67:0/1009234303 shutdown_connections 2024-03-09T09:15:05.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.411+0000 7f7408600640 1 --2- 172.21.15.67:0/1009234303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74000c22a0 0x7f74000acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.411+0000 7f7408600640 1 --2- 172.21.15.67:0/1009234303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74000ac730 0x7f7400071190 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.411+0000 7f7408600640 1 -- 172.21.15.67:0/1009234303 >> 172.21.15.67:0/1009234303 conn(0x7f74000c7370 msgr2=0x7f74000c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:05.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.411+0000 7f7408600640 1 -- 172.21.15.67:0/1009234303 shutdown_connections 2024-03-09T09:15:05.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.411+0000 7f7408600640 1 -- 172.21.15.67:0/1009234303 wait complete. 2024-03-09T09:15:05.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.413+0000 7f7408600640 1 Processor -- start 2024-03-09T09:15:05.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.413+0000 7f7408600640 1 -- start start 2024-03-09T09:15:05.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.414+0000 7f7408600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74000ac730 0x7f7400150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:05.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.414+0000 7f7408600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74000c22a0 0x7f74000ab880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:05.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.414+0000 7f7408600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400157650 0x7f74001dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:05.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.414+0000 7f7408600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7400151610 con 0x7f7400157650 2024-03-09T09:15:05.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.414+0000 7f7408600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7400070c00 con 0x7f74000c22a0 2024-03-09T09:15:05.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.414+0000 7f7408600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7400151790 con 0x7f74000ac730 2024-03-09T09:15:05.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.414+0000 7f7406600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400157650 0x7f74001dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:05.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.414+0000 7f7406600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400157650 0x7f74001dd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52458/0 (socket says 172.21.15.67:52458) 2024-03-09T09:15:05.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.414+0000 7f7406600640 1 -- 172.21.15.67:0/1859131585 learned_addr learned my addr 172.21.15.67:0/1859131585 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:05.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.414+0000 7f7405200640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74000c22a0 0x7f74000ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:05.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.415+0000 7f7405c00640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74000ac730 0x7f7400150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:05.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.415+0000 7f7406600640 1 -- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74000ac730 msgr2=0x7f7400150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:05.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.415+0000 7f7406600640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74000ac730 0x7f7400150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.415+0000 7f7406600640 1 -- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74000c22a0 msgr2=0x7f74000ab880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:05.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.415+0000 7f7406600640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74000c22a0 0x7f74000ab880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.415+0000 7f7406600640 1 -- 172.21.15.67:0/1859131585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7400150a60 con 0x7f7400157650 2024-03-09T09:15:05.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.415+0000 7f7406600640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400157650 0x7f74001dd2c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f73f800c730 tx=0x7f73f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:05.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.415+0000 7f73fd600640 1 -- 172.21.15.67:0/1859131585 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8017070 con 0x7f7400157650 2024-03-09T09:15:05.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.416+0000 7f73fd600640 1 -- 172.21.15.67:0/1859131585 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f800ce40 con 0x7f7400157650 2024-03-09T09:15:05.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.416+0000 7f73fd600640 1 -- 172.21.15.67:0/1859131585 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f800a840 con 0x7f7400157650 2024-03-09T09:15:05.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.416+0000 7f7408600640 1 -- 172.21.15.67:0/1859131585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7400150d40 con 0x7f7400157650 2024-03-09T09:15:05.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.416+0000 7f7408600640 1 -- 172.21.15.67:0/1859131585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74001ebc10 con 0x7f7400157650 2024-03-09T09:15:05.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.418+0000 7f7408600640 1 -- 172.21.15.67:0/1859131585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7400075a00 con 0x7f7400157650 2024-03-09T09:15:05.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.418+0000 7f73fd600640 1 -- 172.21.15.67:0/1859131585 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f73f801a070 con 0x7f7400157650 2024-03-09T09:15:05.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.419+0000 7f73fd600640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73e006b500 0x7f73e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:05.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.419+0000 7f73fd600640 1 -- 172.21.15.67:0/1859131585 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f73f80899a0 con 0x7f7400157650 2024-03-09T09:15:05.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.420+0000 7f7405c00640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73e006b500 0x7f73e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:05.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.421+0000 7f7405c00640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73e006b500 0x7f73e006d9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f73f4036000 tx=0x7f73f403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:05.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.424+0000 7f73fd600640 1 -- 172.21.15.67:0/1859131585 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f73f8057e70 con 0x7f7400157650 2024-03-09T09:15:05.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.786+0000 7f7408600640 1 -- 172.21.15.67:0/1859131585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f74000c6e30 con 0x7f7400157650 2024-03-09T09:15:05.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.787+0000 7f73fd600640 1 -- 172.21.15.67:0/1859131585 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f73f805bf10 con 0x7f7400157650 2024-03-09T09:15:05.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.790+0000 7f7408600640 1 -- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73e006b500 msgr2=0x7f73e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:05.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.790+0000 7f7408600640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73e006b500 0x7f73e006d9c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f73f4036000 tx=0x7f73f403c040 comp rx=0 tx=0).stop 2024-03-09T09:15:05.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.790+0000 7f7408600640 1 -- 172.21.15.67:0/1859131585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400157650 msgr2=0x7f74001dd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:05.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.790+0000 7f7408600640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400157650 0x7f74001dd2c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f73f800c730 tx=0x7f73f800cbf0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.790+0000 7f7408600640 1 -- 172.21.15.67:0/1859131585 shutdown_connections 2024-03-09T09:15:05.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.790+0000 7f7408600640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73e006b500 0x7f73e006d9c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.790+0000 7f7408600640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7400157650 0x7f74001dd2c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.790+0000 7f7408600640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74000c22a0 0x7f74000ab880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.790+0000 7f7408600640 1 --2- 172.21.15.67:0/1859131585 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f74000ac730 0x7f7400150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:05.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.790+0000 7f7408600640 1 -- 172.21.15.67:0/1859131585 >> 172.21.15.67:0/1859131585 conn(0x7f74000c7370 msgr2=0x7f74000c2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:05.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.791+0000 7f7408600640 1 -- 172.21.15.67:0/1859131585 shutdown_connections 2024-03-09T09:15:05.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:05.791+0000 7f7408600640 1 -- 172.21.15.67:0/1859131585 wait complete. 2024-03-09T09:15:05.821 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:15:09.521 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 19% (98/496) Updating files: 20% (100/496) Updating files: 21% (105/496) Updating files: 22% (110/496) Updating files: 23% (115/496) Updating files: 24% (120/496) Updating files: 25% (124/496) Updating files: 26% (129/496) Updating files: 27% (134/496) Updating files: 28% (139/496) Updating files: 29% (144/496) Updating files: 30% (149/496) Updating files: 31% (154/496) Updating files: 32% (159/496) Updating files: 33% (164/496) Updating files: 34% (169/496) Updating files: 35% (174/496) Updating files: 36% (179/496) Updating files: 37% (184/496) Updating files: 38% (189/496) Updating files: 39% (194/496) Updating files: 40% (199/496) Updating files: 41% (204/496) Updating files: 42% (209/496) Updating files: 43% (214/496) Updating files: 44% (219/496) Updating files: 45% (224/496) Updating files: 46% (229/496) Updating files: 46% (232/496) Updating files: 46% (233/496) Updating files: 47% (234/496) Updating files: 48% (239/496) Updating files: 49% (244/496) Updating files: 50% (248/496) Updating files: 51% (253/496) Updating files: 52% (258/496) Updating files: 53% (263/496) Updating files: 54% (268/496) Updating files: 55% (273/496) Updating files: 56% (278/496) Updating files: 57% (283/496) Updating files: 58% (288/496) Updating files: 59% (293/496) Updating files: 60% (298/496) Updating files: 61% (303/496) Updating files: 62% (308/496) Updating files: 63% (313/496) Updating files: 64% (318/496) Updating files: 64% (320/496) Updating files: 65% (323/496) Updating files: 66% (328/496) Updating files: 67% (333/496) Updating files: 68% (338/496) Updating files: 69% (343/496) Updating files: 70% (348/496) Updating files: 71% (353/496) Updating files: 72% (358/496) Updating files: 73% (363/496) Updating files: 74% (368/496) Updating files: 75% (372/496) Updating files: 76% (377/496) Updating files: 76% (378/496) Updating files: 76% (379/496) Updating files: 77% (382/496) Updating files: 78% (387/496) Updating files: 79% (392/496) Updating files: 80% (397/496) Updating files: 81% (402/496) Updating files: 82% (407/496) Updating files: 83% (412/496) Updating files: 84% (417/496) Updating files: 85% (422/496) Updating files: 86% (427/496) Updating files: 87% (432/496) Updating files: 88% (437/496) Updating files: 88% (439/496) Updating files: 89% (442/496) Updating files: 90% (447/496) Updating files: 91% (452/496) Updating files: 92% (457/496) Updating files: 93% (462/496) Updating files: 94% (467/496) Updating files: 95% (472/496) Updating files: 96% (477/496) Updating files: 97% (482/496) Updating files: 98% (487/496) Updating files: 99% (492/496) Updating files: 100% (496/496) Updating files: 100% (496/496), done. 2024-03-09T09:15:09.581 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at f6c746c314d 0.78 2024-03-09T09:15:09.601 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.78 2024-03-09T09:15:09.943 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:15:09.943 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 79 -eq 48 ']' 2024-03-09T09:15:09.943 INFO:tasks.workunit.client.0.smithi067.stdout:v0.79 2024-03-09T09:15:09.943 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.79 2024-03-09T09:15:09.944 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.79 2024-03-09T09:15:09.944 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.79 2024-03-09T09:15:11.828 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 3% (12/329) Updating files: 4% (14/329) Updating files: 5% (17/329) Updating files: 6% (20/329) Updating files: 7% (24/329) Updating files: 8% (27/329) Updating files: 9% (30/329) Updating files: 10% (33/329) Updating files: 11% (37/329) Updating files: 12% (40/329) Updating files: 13% (43/329) Updating files: 14% (47/329) Updating files: 15% (50/329) Updating files: 16% (53/329) Updating files: 16% (54/329) Updating files: 17% (56/329) Updating files: 18% (60/329) Updating files: 19% (63/329) Updating files: 20% (66/329) Updating files: 21% (70/329) Updating files: 22% (73/329) Updating files: 23% (76/329) Updating files: 24% (79/329) Updating files: 25% (83/329) Updating files: 26% (86/329) Updating files: 27% (89/329) Updating files: 27% (91/329) Updating files: 28% (93/329) Updating files: 29% (96/329) Updating files: 30% (99/329) Updating files: 31% (102/329) Updating files: 32% (106/329) Updating files: 33% (109/329) Updating files: 34% (112/329) Updating files: 35% (116/329) Updating files: 36% (119/329) Updating files: 37% (122/329) Updating files: 38% (126/329) Updating files: 39% (129/329) Updating files: 40% (132/329) Updating files: 41% (135/329) Updating files: 42% (139/329) Updating files: 42% (141/329) Updating files: 43% (142/329) Updating files: 44% (145/329) Updating files: 45% (149/329) Updating files: 46% (152/329) Updating files: 47% (155/329) Updating files: 48% (158/329) Updating files: 49% (162/329) Updating files: 50% (165/329) Updating files: 51% (168/329) Updating files: 52% (172/329) Updating files: 53% (175/329) Updating files: 54% (178/329) Updating files: 55% (181/329) Updating files: 56% (185/329) Updating files: 57% (188/329) Updating files: 58% (191/329) Updating files: 59% (195/329) Updating files: 60% (198/329) Updating files: 61% (201/329) Updating files: 62% (204/329) Updating files: 63% (208/329) Updating files: 63% (209/329) Updating files: 64% (211/329) Updating files: 65% (214/329) Updating files: 66% (218/329) Updating files: 67% (221/329) Updating files: 68% (224/329) Updating files: 69% (228/329) Updating files: 70% (231/329) Updating files: 71% (234/329) Updating files: 71% (236/329) Updating files: 72% (237/329) Updating files: 73% (241/329) Updating files: 74% (244/329) Updating files: 75% (247/329) Updating files: 76% (251/329) Updating files: 77% (254/329) Updating files: 78% (257/329) Updating files: 79% (260/329) Updating files: 80% (264/329) Updating files: 81% (267/329) Updating files: 81% (268/329) Updating files: 82% (270/329) Updating files: 83% (274/329) Updating files: 84% (277/329) Updating files: 85% (280/329) Updating files: 86% (283/329) Updating files: 87% (287/329) Updating files: 88% (290/329) Updating files: 88% (292/329) Updating files: 89% (293/329) Updating files: 90% (297/329) Updating files: 91% (300/329) Updating files: 92% (303/329) Updating files: 93% (306/329) Updating files: 94% (310/329) Updating files: 95% (313/329) Updating files: 96% (316/329) Updating files: 97% (320/329) Updating files: 98% (323/329) Updating files: 99% (326/329) Updating files: 100% (329/329) Updating files: 100% (329/329), done. 2024-03-09T09:15:12.000 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 4c2d73a5095 0.79 2024-03-09T09:15:12.014 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.79 2024-03-09T09:15:12.104 INFO:tasks.workunit.client.1.smithi134.stdout:v0.80 2024-03-09T09:15:12.105 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:15:12.105 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 80 -eq 48 ']' 2024-03-09T09:15:12.105 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.80 2024-03-09T09:15:12.105 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.80 2024-03-09T09:15:12.105 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.80 2024-03-09T09:15:15.821 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:15:15.822 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:15:16.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.018+0000 7fc627e00640 1 Processor -- start 2024-03-09T09:15:16.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.018+0000 7fc627e00640 1 -- start start 2024-03-09T09:15:16.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.019+0000 7fc627e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6280acd90 0x7fc6281565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc627e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc628159770 0x7fc628159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc627e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc628156b30 con 0x7fc6280c1d00 2024-03-09T09:15:16.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc627e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6281592e0 con 0x7fc6280acd90 2024-03-09T09:15:16.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc627e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62815a100 con 0x7fc628159770 2024-03-09T09:15:16.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc626a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6280acd90 0x7fc6281565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc626a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6280acd90 0x7fc6281565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54524/0 (socket says 172.21.15.67:54524) 2024-03-09T09:15:16.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc626a00640 1 -- 172.21.15.67:0/1266477320 learned_addr learned my addr 172.21.15.67:0/1266477320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:16.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc626000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc6280c1d00 0x7fc6280aa3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:32802/0 (socket says 172.21.15.67:32802) 2024-03-09T09:15:16.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc627400640 1 --2- 172.21.15.67:0/1266477320 >> v2:172.21.15.134:3300/0 conn(0x7fc628159770 0x7fc628159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc626a00640 1 -- 172.21.15.67:0/1266477320 >> v2:172.21.15.134:3300/0 conn(0x7fc628159770 msgr2=0x7fc628159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:16.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc626a00640 1 --2- 172.21.15.67:0/1266477320 >> v2:172.21.15.134:3300/0 conn(0x7fc628159770 0x7fc628159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc626a00640 1 -- 172.21.15.67:0/1266477320 >> v1:172.21.15.67:6789/0 conn(0x7fc6280c1d00 legacy=0x7fc6280aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc626a00640 1 -- 172.21.15.67:0/1266477320 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62806de60 con 0x7fc6280acd90 2024-03-09T09:15:16.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.020+0000 7fc627400640 1 --2- 172.21.15.67:0/1266477320 >> v2:172.21.15.134:3300/0 conn(0x7fc628159770 0x7fc628159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:15:16.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.022+0000 7fc626a00640 1 --2- 172.21.15.67:0/1266477320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6280acd90 0x7fc6281565f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc628153790 tx=0x7fc61802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fc06369c0b1c43 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.022+0000 7fc625600640 1 -- 172.21.15.67:0/1266477320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618034070 con 0x7fc6280acd90 2024-03-09T09:15:16.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.022+0000 7fc625600640 1 -- 172.21.15.67:0/1266477320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc618030870 con 0x7fc6280acd90 2024-03-09T09:15:16.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.022+0000 7fc625600640 1 -- 172.21.15.67:0/1266477320 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618030c40 con 0x7fc6280acd90 2024-03-09T09:15:16.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.023+0000 7fc627e00640 1 -- 172.21.15.67:0/1266477320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6280acd90 msgr2=0x7fc6281565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.023+0000 7fc627e00640 1 --2- 172.21.15.67:0/1266477320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6280acd90 0x7fc6281565f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc628153790 tx=0x7fc61802b9b0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.023+0000 7fc627e00640 1 -- 172.21.15.67:0/1266477320 shutdown_connections 2024-03-09T09:15:16.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.023+0000 7fc627e00640 1 --2- 172.21.15.67:0/1266477320 >> v2:172.21.15.134:3300/0 conn(0x7fc628159770 0x7fc628159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.023+0000 7fc627e00640 1 --2- 172.21.15.67:0/1266477320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6280acd90 0x7fc6281565f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.023+0000 7fc627e00640 1 -- 172.21.15.67:0/1266477320 >> 172.21.15.67:0/1266477320 conn(0x7fc6280c7300 msgr2=0x7fc6280c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.023+0000 7fc627e00640 1 -- 172.21.15.67:0/1266477320 shutdown_connections 2024-03-09T09:15:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.023+0000 7fc627e00640 1 -- 172.21.15.67:0/1266477320 wait complete. 2024-03-09T09:15:16.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.025+0000 7fc627e00640 1 Processor -- start 2024-03-09T09:15:16.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.026+0000 7fc627e00640 1 -- start start 2024-03-09T09:15:16.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.027+0000 7fc627e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6280acd90 0x7fc62814ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.027+0000 7fc627e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6280c1d00 0x7fc6280c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.027+0000 7fc627e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc628159770 0x7fc628072460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.027+0000 7fc627e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62806db80 con 0x7fc6280acd90 2024-03-09T09:15:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.027+0000 7fc627e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62806d8a0 con 0x7fc628159770 2024-03-09T09:15:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.027+0000 7fc627e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6280aaac0 con 0x7fc6280c1d00 2024-03-09T09:15:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.027+0000 7fc626a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6280acd90 0x7fc62814ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.027+0000 7fc626a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6280acd90 0x7fc62814ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54536/0 (socket says 172.21.15.67:54536) 2024-03-09T09:15:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.027+0000 7fc626a00640 1 -- 172.21.15.67:0/3550382291 learned_addr learned my addr 172.21.15.67:0/3550382291 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.028+0000 7fc626a00640 1 -- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6280c1d00 msgr2=0x7fc6280c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.028+0000 7fc626a00640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6280c1d00 0x7fc6280c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.028+0000 7fc626a00640 1 -- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc628159770 msgr2=0x7fc628072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.028+0000 7fc626a00640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc628159770 0x7fc628072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.028+0000 7fc626a00640 1 -- 172.21.15.67:0/3550382291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc628071ca0 con 0x7fc6280acd90 2024-03-09T09:15:16.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.028+0000 7fc626a00640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6280acd90 0x7fc62814ad80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc618000c00 tx=0x7fc618004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.028+0000 7fc61ea00640 1 -- 172.21.15.67:0/3550382291 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618048070 con 0x7fc6280acd90 2024-03-09T09:15:16.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.028+0000 7fc61ea00640 1 -- 172.21.15.67:0/3550382291 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc618039d90 con 0x7fc6280acd90 2024-03-09T09:15:16.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.028+0000 7fc61ea00640 1 -- 172.21.15.67:0/3550382291 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618034040 con 0x7fc6280acd90 2024-03-09T09:15:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.028+0000 7fc627e00640 1 -- 172.21.15.67:0/3550382291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6280c67a0 con 0x7fc6280acd90 2024-03-09T09:15:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.028+0000 7fc627e00640 1 -- 172.21.15.67:0/3550382291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6280c6d60 con 0x7fc6280acd90 2024-03-09T09:15:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.030+0000 7fc627e00640 1 -- 172.21.15.67:0/3550382291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62804f3c0 con 0x7fc6280acd90 2024-03-09T09:15:16.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.031+0000 7fc61ea00640 1 -- 172.21.15.67:0/3550382291 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc61804b070 con 0x7fc6280acd90 2024-03-09T09:15:16.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.032+0000 7fc61ea00640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60806b500 0x7fc60806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.032+0000 7fc61ea00640 1 -- 172.21.15.67:0/3550382291 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc6180ac960 con 0x7fc6280acd90 2024-03-09T09:15:16.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.032+0000 7fc626000640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60806b500 0x7fc60806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.033+0000 7fc626000640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60806b500 0x7fc60806d9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc614007fd0 tx=0x7fc6140079a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.037+0000 7fc61ea00640 1 -- 172.21.15.67:0/3550382291 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc61807b470 con 0x7fc6280acd90 2024-03-09T09:15:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.327+0000 7fc627e00640 1 -- 172.21.15.67:0/3550382291 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc62806e320 con 0x7fc6280acd90 2024-03-09T09:15:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.328+0000 7fc61ea00640 1 -- 172.21.15.67:0/3550382291 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fc6180aae20 con 0x7fc6280acd90 2024-03-09T09:15:16.333 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":172,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:15:16.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.330+0000 7fc627e00640 1 -- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60806b500 msgr2=0x7fc60806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.330+0000 7fc627e00640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60806b500 0x7fc60806d9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc614007fd0 tx=0x7fc6140079a0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.330+0000 7fc627e00640 1 -- 172.21.15.67:0/3550382291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6280acd90 msgr2=0x7fc62814ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.330+0000 7fc627e00640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6280acd90 0x7fc62814ad80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc618000c00 tx=0x7fc618004730 comp rx=0 tx=0).stop 2024-03-09T09:15:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.330+0000 7fc627e00640 1 -- 172.21.15.67:0/3550382291 shutdown_connections 2024-03-09T09:15:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.330+0000 7fc627e00640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc60806b500 0x7fc60806d9c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.330+0000 7fc627e00640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc628159770 0x7fc628072460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.330+0000 7fc627e00640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6280c1d00 0x7fc6280c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.330+0000 7fc627e00640 1 --2- 172.21.15.67:0/3550382291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6280acd90 0x7fc62814ad80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.331+0000 7fc627e00640 1 -- 172.21.15.67:0/3550382291 >> 172.21.15.67:0/3550382291 conn(0x7fc6280c7300 msgr2=0x7fc6280c2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.331+0000 7fc627e00640 1 -- 172.21.15.67:0/3550382291 shutdown_connections 2024-03-09T09:15:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.331+0000 7fc627e00640 1 -- 172.21.15.67:0/3550382291 wait complete. 2024-03-09T09:15:16.359 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:15:16.359 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:15:16.359 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:15:16.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:15:16.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.546+0000 7fe2ca000640 1 Processor -- start 2024-03-09T09:15:16.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.547+0000 7fe2ca000640 1 -- start start 2024-03-09T09:15:16.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.550+0000 7fe2ca000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe2c40c1cf0 0x7fe2c41561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.550+0000 7fe2ca000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c40c26c0 0x7fe2c4160440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.550+0000 7fe2ca000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c40c4ac0 con 0x7fe2c4153340 2024-03-09T09:15:16.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.550+0000 7fe2ca000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c4155d10 con 0x7fe2c40c1cf0 2024-03-09T09:15:16.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.550+0000 7fe2ca000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c40ab460 con 0x7fe2c40c26c0 2024-03-09T09:15:16.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.550+0000 7fe2c3e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe2c40c1cf0 0x7fe2c41561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.550+0000 7fe2c9600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c40c26c0 0x7fe2c4160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.550+0000 7fe2c3e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe2c40c1cf0 0x7fe2c41561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46080/0 (socket says 172.21.15.67:46080) 2024-03-09T09:15:16.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.550+0000 7fe2c3e00640 1 -- 172.21.15.67:0/3979054845 learned_addr learned my addr 172.21.15.67:0/3979054845 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.551+0000 7fe2c3e00640 1 -- 172.21.15.67:0/3979054845 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c40c26c0 msgr2=0x7fe2c4160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.551+0000 7fe2c3e00640 1 --2- 172.21.15.67:0/3979054845 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c40c26c0 0x7fe2c4160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.551+0000 7fe2c3e00640 1 -- 172.21.15.67:0/3979054845 >> v1:172.21.15.134:6789/0 conn(0x7fe2c4153340 legacy=0x7fe2c40c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.551+0000 7fe2c3e00640 1 -- 172.21.15.67:0/3979054845 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c40c47e0 con 0x7fe2c40c1cf0 2024-03-09T09:15:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.552+0000 7fe2c3e00640 1 --2- 172.21.15.67:0/3979054845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c40c1cf0 0x7fe2c41561a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009870 tx=0x7fe2b002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=192327d5c18c614c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.553+0000 7fe2c3400640 1 -- 172.21.15.67:0/3979054845 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0034070 con 0x7fe2c40c1cf0 2024-03-09T09:15:16.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.553+0000 7fe2c3400640 1 -- 172.21.15.67:0/3979054845 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b0030840 con 0x7fe2c40c1cf0 2024-03-09T09:15:16.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.553+0000 7fe2c3400640 1 -- 172.21.15.67:0/3979054845 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0030c10 con 0x7fe2c40c1cf0 2024-03-09T09:15:16.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.554+0000 7fe2ca000640 1 -- 172.21.15.67:0/3979054845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c40c1cf0 msgr2=0x7fe2c41561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.554+0000 7fe2ca000640 1 --2- 172.21.15.67:0/3979054845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c40c1cf0 0x7fe2c41561a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009870 tx=0x7fe2b002b980 comp rx=0 tx=0).stop 2024-03-09T09:15:16.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.554+0000 7fe2ca000640 1 -- 172.21.15.67:0/3979054845 shutdown_connections 2024-03-09T09:15:16.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.554+0000 7fe2ca000640 1 --2- 172.21.15.67:0/3979054845 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c40c26c0 0x7fe2c4160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.554+0000 7fe2ca000640 1 --2- 172.21.15.67:0/3979054845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c40c1cf0 0x7fe2c41561a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.554+0000 7fe2ca000640 1 -- 172.21.15.67:0/3979054845 >> 172.21.15.67:0/3979054845 conn(0x7fe2c406c150 msgr2=0x7fe2c406c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:16.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.554+0000 7fe2ca000640 1 -- 172.21.15.67:0/3979054845 shutdown_connections 2024-03-09T09:15:16.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.554+0000 7fe2ca000640 1 -- 172.21.15.67:0/3979054845 wait complete. 2024-03-09T09:15:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.556+0000 7fe2ca000640 1 Processor -- start 2024-03-09T09:15:16.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.557+0000 7fe2ca000640 1 -- start start 2024-03-09T09:15:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.558+0000 7fe2ca000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c40c1cf0 0x7fe2c414ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.558+0000 7fe2ca000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c40c26c0 0x7fe2c41eee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.558+0000 7fe2ca000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c4153340 0x7fe2c41ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.558+0000 7fe2ca000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c40c5580 con 0x7fe2c40c26c0 2024-03-09T09:15:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.558+0000 7fe2ca000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c40c5400 con 0x7fe2c4153340 2024-03-09T09:15:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.558+0000 7fe2ca000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c41590b0 con 0x7fe2c40c1cf0 2024-03-09T09:15:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.558+0000 7fe2c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c40c26c0 0x7fe2c41eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.558+0000 7fe2c3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c40c26c0 0x7fe2c41eee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54546/0 (socket says 172.21.15.67:54546) 2024-03-09T09:15:16.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.558+0000 7fe2c3e00640 1 -- 172.21.15.67:0/905631625 learned_addr learned my addr 172.21.15.67:0/905631625 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.559+0000 7fe2c8c00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c40c1cf0 0x7fe2c414ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.559+0000 7fe2c9600640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c4153340 0x7fe2c41ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.559+0000 7fe2c3e00640 1 -- 172.21.15.67:0/905631625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c40c1cf0 msgr2=0x7fe2c414ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.559+0000 7fe2c3e00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c40c1cf0 0x7fe2c414ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.559+0000 7fe2c3e00640 1 -- 172.21.15.67:0/905631625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c4153340 msgr2=0x7fe2c41ef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.559+0000 7fe2c3e00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c4153340 0x7fe2c41ef3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.559+0000 7fe2c3e00640 1 -- 172.21.15.67:0/905631625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c40c7510 con 0x7fe2c40c26c0 2024-03-09T09:15:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.559+0000 7fe2c3e00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c40c26c0 0x7fe2c41eee70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0000c00 tx=0x7fe2b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.560+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0048070 con 0x7fe2c40c26c0 2024-03-09T09:15:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.560+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b0030640 con 0x7fe2c40c26c0 2024-03-09T09:15:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.560+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0034040 con 0x7fe2c40c26c0 2024-03-09T09:15:16.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.560+0000 7fe2ca000640 1 -- 172.21.15.67:0/905631625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c40c77b0 con 0x7fe2c40c26c0 2024-03-09T09:15:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.560+0000 7fe2ca000640 1 -- 172.21.15.67:0/905631625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c40c7d70 con 0x7fe2c40c26c0 2024-03-09T09:15:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.561+0000 7fe2ca000640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4153950 0x7fe2c4153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.561+0000 7fe2ca000640 1 -- 172.21.15.67:0/905631625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c40c72e0 con 0x7fe2c4153950 2024-03-09T09:15:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.561+0000 7fe2c8c00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4153950 0x7fe2c4153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.562+0000 7fe2c8c00640 1 -- 172.21.15.67:0/905631625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2c404f3c0 con 0x7fe2c4153950 2024-03-09T09:15:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.562+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe2b004b070 con 0x7fe2c40c26c0 2024-03-09T09:15:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.562+0000 7fe2c8c00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4153950 0x7fe2c4153d30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000f60 tx=0x7fe2b800e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.563+0000 7fe2c0c00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe2a406b4d0 0x7fe2a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.563+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe2b00ac5d0 con 0x7fe2c40c26c0 2024-03-09T09:15:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.564+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8019070 con 0x7fe2c4153950 2024-03-09T09:15:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.564+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fe2b80052e0 con 0x7fe2c4153950 2024-03-09T09:15:16.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.564+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4153950 msgr2=0x7fe2c4153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.564+0000 7fe2c0c00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4153950 0x7fe2c4153d30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000f60 tx=0x7fe2b800e890 comp rx=0 tx=0).stop 2024-03-09T09:15:16.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.564+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4153950 msgr2=0x7fe2c4153d30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:16.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.564+0000 7fe2c0c00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4153950 0x7fe2c4153d30 secure :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000f60 tx=0x7fe2b800e890 comp rx=0 tx=0).stop 2024-03-09T09:15:16.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.564+0000 7fe2c9600640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe2a406b4d0 0x7fe2a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.565+0000 7fe2c9600640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe2a406b4d0 0x7fe2a406d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe2c414d890 tx=0x7fe2bc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.571+0000 7fe2ca000640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4154270 0x7fe2c414a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.571+0000 7fe2ca000640 1 -- 172.21.15.67:0/905631625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c41eeb90 con 0x7fe2c4154270 2024-03-09T09:15:16.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.571+0000 7fe2c8c00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4154270 0x7fe2c414a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.571+0000 7fe2c8c00640 1 -- 172.21.15.67:0/905631625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe2c40c2190 con 0x7fe2c4154270 2024-03-09T09:15:16.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.572+0000 7fe2c8c00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4154270 0x7fe2c414a370 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000f60 tx=0x7fe2b8001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.572+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8019040 con 0x7fe2c4154270 2024-03-09T09:15:16.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.572+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7fe2b801dad0 con 0x7fe2c4154270 2024-03-09T09:15:16.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.572+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4154270 msgr2=0x7fe2c414a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.572+0000 7fe2c0c00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4154270 0x7fe2c414a370 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000f60 tx=0x7fe2b8001080 comp rx=0 tx=0).stop 2024-03-09T09:15:16.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.572+0000 7fe2c0c00640 1 -- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4154270 msgr2=0x7fe2c414a370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:16.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.572+0000 7fe2c0c00640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c4154270 0x7fe2c414a370 secure :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8000f60 tx=0x7fe2b8001080 comp rx=0 tx=0).stop 2024-03-09T09:15:16.581 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:15:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:15:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:15:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:15:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 172, 2024-03-09T09:15:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:15:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:15:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:15:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:15:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:15:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:15:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:15:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:15:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:15:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:15:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:15:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:15:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:15:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:15:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:15:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:15:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:15:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:15:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:15:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:15:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:15:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:15:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 8 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:15:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:15:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:15:16.597 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:15:16.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.573+0000 7fe2ca000640 1 -- 172.21.15.67:0/905631625 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe2a406b4d0 msgr2=0x7fe2a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.573+0000 7fe2ca000640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe2a406b4d0 0x7fe2a406d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe2c414d890 tx=0x7fe2bc00b040 comp rx=0 tx=0).stop 2024-03-09T09:15:16.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.573+0000 7fe2ca000640 1 -- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c40c26c0 msgr2=0x7fe2c41eee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.573+0000 7fe2ca000640 1 --2- 172.21.15.67:0/905631625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2c40c26c0 0x7fe2c41eee70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0000c00 tx=0x7fe2b0004730 comp rx=0 tx=0).stop 2024-03-09T09:15:16.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.573+0000 7fe2c9600640 1 -- 172.21.15.67:0/905631625 reap_dead start 2024-03-09T09:15:16.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.573+0000 7fe2c9600640 1 -- 172.21.15.67:0/905631625 reap_dead start 2024-03-09T09:15:16.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.573+0000 7fe2ca000640 1 -- 172.21.15.67:0/905631625 shutdown_connections 2024-03-09T09:15:16.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.573+0000 7fe2ca000640 1 -- 172.21.15.67:0/905631625 >> 172.21.15.67:0/905631625 conn(0x7fe2c406c150 msgr2=0x7fe2c40c4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:16.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.573+0000 7fe2ca000640 1 -- 172.21.15.67:0/905631625 shutdown_connections 2024-03-09T09:15:16.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.573+0000 7fe2ca000640 1 -- 172.21.15.67:0/905631625 wait complete. 2024-03-09T09:15:16.599 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:15:16.600 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:15:16.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:15:16.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.780+0000 7fac4d200640 1 Processor -- start 2024-03-09T09:15:16.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.780+0000 7fac4d200640 1 -- start start 2024-03-09T09:15:16.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.782+0000 7fac4d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac481591f0 0x7fac48159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.782+0000 7fac4d200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fac481595d0 0x7fac48072d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.782+0000 7fac4d200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac480703f0 con 0x7fac48073270 2024-03-09T09:15:16.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.782+0000 7fac4d200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac48159a50 con 0x7fac481595d0 2024-03-09T09:15:16.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.782+0000 7fac4d200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac480aa6b0 con 0x7fac481591f0 2024-03-09T09:15:16.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.782+0000 7fac47400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac481591f0 0x7fac48159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.783+0000 7fac47400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac481591f0 0x7fac48159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:60594/0 (socket says 172.21.15.67:60594) 2024-03-09T09:15:16.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.783+0000 7fac47400640 1 -- 172.21.15.67:0/2130745913 learned_addr learned my addr 172.21.15.67:0/2130745913 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:16.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.783+0000 7fac3ea00640 1 --2- 172.21.15.67:0/2130745913 >> v2:172.21.15.134:3300/0 conn(0x7fac481595d0 0x7fac48072d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.783+0000 7fac47400640 1 -- 172.21.15.67:0/2130745913 >> v2:172.21.15.134:3300/0 conn(0x7fac481595d0 msgr2=0x7fac48072d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:16.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.783+0000 7fac47400640 1 --2- 172.21.15.67:0/2130745913 >> v2:172.21.15.134:3300/0 conn(0x7fac481595d0 0x7fac48072d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.784+0000 7fac47400640 1 -- 172.21.15.67:0/2130745913 >> v1:172.21.15.134:6789/0 conn(0x7fac48073270 legacy=0x7fac48155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.784+0000 7fac47400640 1 -- 172.21.15.67:0/2130745913 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac480aa990 con 0x7fac481591f0 2024-03-09T09:15:16.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.784+0000 7fac3ea00640 1 --2- 172.21.15.67:0/2130745913 >> v2:172.21.15.134:3300/0 conn(0x7fac481595d0 0x7fac48072d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:15:16.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.785+0000 7fac47400640 1 --2- 172.21.15.67:0/2130745913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac481591f0 0x7fac48159ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac38009e50 tx=0x7fac3802be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c3961f265b6bc62 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.785+0000 7fac46a00640 1 -- 172.21.15.67:0/2130745913 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3802c9f0 con 0x7fac481591f0 2024-03-09T09:15:16.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.785+0000 7fac46a00640 1 -- 172.21.15.67:0/2130745913 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3802cb90 con 0x7fac481591f0 2024-03-09T09:15:16.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.785+0000 7fac46a00640 1 -- 172.21.15.67:0/2130745913 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac380343f0 con 0x7fac481591f0 2024-03-09T09:15:16.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.786+0000 7fac4d200640 1 -- 172.21.15.67:0/2130745913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac481591f0 msgr2=0x7fac48159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.786+0000 7fac4d200640 1 --2- 172.21.15.67:0/2130745913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac481591f0 0x7fac48159ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac38009e50 tx=0x7fac3802be50 comp rx=0 tx=0).stop 2024-03-09T09:15:16.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.786+0000 7fac4d200640 1 -- 172.21.15.67:0/2130745913 shutdown_connections 2024-03-09T09:15:16.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.786+0000 7fac4d200640 1 --2- 172.21.15.67:0/2130745913 >> v2:172.21.15.134:3300/0 conn(0x7fac481595d0 0x7fac48072d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.786+0000 7fac4d200640 1 --2- 172.21.15.67:0/2130745913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac481591f0 0x7fac48159ce0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.786+0000 7fac4d200640 1 -- 172.21.15.67:0/2130745913 >> 172.21.15.67:0/2130745913 conn(0x7fac4806f000 msgr2=0x7fac48066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:16.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.786+0000 7fac4d200640 1 -- 172.21.15.67:0/2130745913 shutdown_connections 2024-03-09T09:15:16.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.786+0000 7fac4d200640 1 -- 172.21.15.67:0/2130745913 wait complete. 2024-03-09T09:15:16.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.788+0000 7fac4d200640 1 Processor -- start 2024-03-09T09:15:16.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.788+0000 7fac4d200640 1 -- start start 2024-03-09T09:15:16.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.789+0000 7fac4d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac48073270 0x7fac480ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.789+0000 7fac4d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac481591f0 0x7fac48146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.789+0000 7fac4d200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac481595d0 0x7fac48146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.789+0000 7fac4d200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac481536a0 con 0x7fac481591f0 2024-03-09T09:15:16.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.789+0000 7fac4d200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac48153520 con 0x7fac481595d0 2024-03-09T09:15:16.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.789+0000 7fac4d200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac480c3010 con 0x7fac48073270 2024-03-09T09:15:16.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.789+0000 7fac3ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac481591f0 0x7fac48146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.789+0000 7fac3ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac481591f0 0x7fac48146980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54572/0 (socket says 172.21.15.67:54572) 2024-03-09T09:15:16.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.789+0000 7fac3ea00640 1 -- 172.21.15.67:0/3333027813 learned_addr learned my addr 172.21.15.67:0/3333027813 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:16.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.789+0000 7fac47400640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac48073270 0x7fac480ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac47e00640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac481595d0 0x7fac48146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac3ea00640 1 -- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac48073270 msgr2=0x7fac480ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac3ea00640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac48073270 0x7fac480ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac3ea00640 1 -- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac481595d0 msgr2=0x7fac48146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac3ea00640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac481595d0 0x7fac48146ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac3ea00640 1 -- 172.21.15.67:0/3333027813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac481ec580 con 0x7fac481591f0 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac47400640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fac48073270 0x7fac480ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac3ea00640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac481591f0 0x7fac48146980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fac3000e990 tx=0x7fac3000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30017070 con 0x7fac481591f0 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3000c9e0 con 0x7fac481591f0 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3000cdb0 con 0x7fac481591f0 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.790+0000 7fac4d200640 1 -- 172.21.15.67:0/3333027813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac480c3dc0 con 0x7fac481591f0 2024-03-09T09:15:16.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.791+0000 7fac4d200640 1 -- 172.21.15.67:0/3333027813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac480c4380 con 0x7fac481591f0 2024-03-09T09:15:16.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.792+0000 7fac4d200640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac481e1b30 0x7fac480c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.792+0000 7fac4d200640 1 -- 172.21.15.67:0/3333027813 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac480c4550 con 0x7fac481e1b30 2024-03-09T09:15:16.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.792+0000 7fac47400640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac481e1b30 0x7fac480c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.793+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fac3001a070 con 0x7fac481591f0 2024-03-09T09:15:16.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.793+0000 7fac47400640 1 -- 172.21.15.67:0/3333027813 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac481ec330 con 0x7fac481e1b30 2024-03-09T09:15:16.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.794+0000 7fac3fe00640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fac2406b4d0 0x7fac2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.794+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fac3008a860 con 0x7fac481591f0 2024-03-09T09:15:16.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.794+0000 7fac47400640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac481e1b30 0x7fac480c48a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fac480ac1e0 tx=0x7fac38004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.794+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38009510 con 0x7fac481e1b30 2024-03-09T09:15:16.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.794+0000 7fac47e00640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fac2406b4d0 0x7fac2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.795+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fac3803d390 con 0x7fac481e1b30 2024-03-09T09:15:16.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.795+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac481e1b30 msgr2=0x7fac480c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.795+0000 7fac3fe00640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac481e1b30 0x7fac480c48a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fac480ac1e0 tx=0x7fac38004a40 comp rx=0 tx=0).stop 2024-03-09T09:15:16.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.795+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac481e1b30 msgr2=0x7fac480c48a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:16.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.795+0000 7fac3fe00640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac481e1b30 0x7fac480c48a0 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fac480ac1e0 tx=0x7fac38004a40 comp rx=0 tx=0).stop 2024-03-09T09:15:16.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.796+0000 7fac47e00640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fac2406b4d0 0x7fac2406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fac40000f60 tx=0x7fac40009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.802+0000 7fac4d200640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac480c4de0 0x7fac480c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:16.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.802+0000 7fac4d200640 1 -- 172.21.15.67:0/3333027813 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac480c6270 con 0x7fac480c4de0 2024-03-09T09:15:16.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.803+0000 7fac47400640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac480c4de0 0x7fac480c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:16.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.804+0000 7fac47400640 1 -- 172.21.15.67:0/3333027813 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fac481ec860 con 0x7fac480c4de0 2024-03-09T09:15:16.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.804+0000 7fac47400640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac480c4de0 0x7fac480c51c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fac38009e50 tx=0x7fac38009000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:16.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.805+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3803de60 con 0x7fac480c4de0 2024-03-09T09:15:16.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.805+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7fac38009af0 con 0x7fac480c4de0 2024-03-09T09:15:16.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.805+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac480c4de0 msgr2=0x7fac480c51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.805+0000 7fac3fe00640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac480c4de0 0x7fac480c51c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fac38009e50 tx=0x7fac38009000 comp rx=0 tx=0).stop 2024-03-09T09:15:16.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.805+0000 7fac3fe00640 1 -- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac480c4de0 msgr2=0x7fac480c51c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:16.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.805+0000 7fac3fe00640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac480c4de0 0x7fac480c51c0 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fac38009e50 tx=0x7fac38009000 comp rx=0 tx=0).stop 2024-03-09T09:15:16.812 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:15:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:15:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:15:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:15:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 172, 2024-03-09T09:15:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:15:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:15:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:15:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:15:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:15:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:15:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:15:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:15:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:15:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:15:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:15:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:15:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:15:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:15:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:15:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:15:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:15:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:15:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:15:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:15:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:15:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:15:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:15:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:15:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:15:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:15:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:15:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:15:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:15:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:15:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:15:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:15:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:15:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:15:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:15:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:15:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:15:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:15:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:15:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:15:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:16.828 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:15:16.828 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:15:16.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.806+0000 7fac4d200640 1 -- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fac2406b4d0 msgr2=0x7fac2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.806+0000 7fac4d200640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fac2406b4d0 0x7fac2406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fac40000f60 tx=0x7fac40009040 comp rx=0 tx=0).stop 2024-03-09T09:15:16.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.806+0000 7fac4d200640 1 -- 172.21.15.67:0/3333027813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac481591f0 msgr2=0x7fac48146980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:16.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.806+0000 7fac4d200640 1 --2- 172.21.15.67:0/3333027813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac481591f0 0x7fac48146980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fac3000e990 tx=0x7fac3000ee50 comp rx=0 tx=0).stop 2024-03-09T09:15:16.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.806+0000 7fac47e00640 1 -- 172.21.15.67:0/3333027813 reap_dead start 2024-03-09T09:15:16.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.806+0000 7fac47e00640 1 -- 172.21.15.67:0/3333027813 reap_dead start 2024-03-09T09:15:16.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.806+0000 7fac4d200640 1 -- 172.21.15.67:0/3333027813 shutdown_connections 2024-03-09T09:15:16.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.806+0000 7fac4d200640 1 -- 172.21.15.67:0/3333027813 >> 172.21.15.67:0/3333027813 conn(0x7fac4806f000 msgr2=0x7fac48159c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:16.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.806+0000 7fac4d200640 1 -- 172.21.15.67:0/3333027813 shutdown_connections 2024-03-09T09:15:16.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:16.806+0000 7fac4d200640 1 -- 172.21.15.67:0/3333027813 wait complete. 2024-03-09T09:15:16.831 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:15:16.831 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:15:16.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:15:17.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.050+0000 7fdb7a400640 1 Processor -- start 2024-03-09T09:15:17.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.050+0000 7fdb7a400640 1 -- start start 2024-03-09T09:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.051+0000 7fdb7a400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb740c9380 0x7fdb740acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.051+0000 7fdb7a400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb740c9890 con 0x7fdb7414fd90 2024-03-09T09:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.051+0000 7fdb7a400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb740c5f80 con 0x7fdb740ad440 2024-03-09T09:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.051+0000 7fdb7a400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7406ba60 con 0x7fdb740c9380 2024-03-09T09:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.052+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb740c9380 0x7fdb740acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.052+0000 7fdb79000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fdb7414fd90 0x7fdb74156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:32818/0 (socket says 172.21.15.67:32818) 2024-03-09T09:15:17.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.052+0000 7fdb79000640 1 -- 172.21.15.67:0/1256062442 learned_addr learned my addr 172.21.15.67:0/1256062442 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:15:17.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.052+0000 7fdb73e00640 1 -- 172.21.15.67:0/1256062442 >> v1:172.21.15.134:6789/0 conn(0x7fdb740ad440 legacy=0x7fdb7406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:17.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.052+0000 7fdb73e00640 1 -- 172.21.15.67:0/1256062442 >> v1:172.21.15.67:6789/0 conn(0x7fdb7414fd90 legacy=0x7fdb74156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.052+0000 7fdb73e00640 1 -- 172.21.15.67:0/1256062442 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb740c6240 con 0x7fdb740c9380 2024-03-09T09:15:17.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.053+0000 7fdb73e00640 1 --2- 172.21.15.67:0/1256062442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb740c9380 0x7fdb740acf00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb7406bbe0 tx=0x7fdb6002b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae112c429a9f9276 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:17.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.054+0000 7fdb73400640 1 -- 172.21.15.67:0/1256062442 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60037070 con 0x7fdb740c9380 2024-03-09T09:15:17.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.054+0000 7fdb73400640 1 -- 172.21.15.67:0/1256062442 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb60030710 con 0x7fdb740c9380 2024-03-09T09:15:17.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.054+0000 7fdb73400640 1 -- 172.21.15.67:0/1256062442 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60030ae0 con 0x7fdb740c9380 2024-03-09T09:15:17.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.054+0000 7fdb7a400640 1 -- 172.21.15.67:0/1256062442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb740c9380 msgr2=0x7fdb740acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.054+0000 7fdb7a400640 1 --2- 172.21.15.67:0/1256062442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb740c9380 0x7fdb740acf00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb7406bbe0 tx=0x7fdb6002b780 comp rx=0 tx=0).stop 2024-03-09T09:15:17.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.055+0000 7fdb7a400640 1 -- 172.21.15.67:0/1256062442 shutdown_connections 2024-03-09T09:15:17.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.055+0000 7fdb7a400640 1 --2- 172.21.15.67:0/1256062442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb740c9380 0x7fdb740acf00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.055+0000 7fdb7a400640 1 -- 172.21.15.67:0/1256062442 >> 172.21.15.67:0/1256062442 conn(0x7fdb740cb480 msgr2=0x7fdb740cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:17.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.055+0000 7fdb7a400640 1 -- 172.21.15.67:0/1256062442 shutdown_connections 2024-03-09T09:15:17.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.055+0000 7fdb7a400640 1 -- 172.21.15.67:0/1256062442 wait complete. 2024-03-09T09:15:17.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.056+0000 7fdb7a400640 1 Processor -- start 2024-03-09T09:15:17.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.057+0000 7fdb7a400640 1 -- start start 2024-03-09T09:15:17.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.057+0000 7fdb7a400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb740ad440 0x7fdb7414bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.057+0000 7fdb7a400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb740c9380 0x7fdb741e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.057+0000 7fdb7a400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7414fd90 0x7fdb741e0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.057+0000 7fdb7a400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7406c8e0 con 0x7fdb740c9380 2024-03-09T09:15:17.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.057+0000 7fdb7a400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7406bcb0 con 0x7fdb740ad440 2024-03-09T09:15:17.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.057+0000 7fdb7a400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7406ca60 con 0x7fdb7414fd90 2024-03-09T09:15:17.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.058+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb740c9380 0x7fdb741e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.058+0000 7fdb79a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7414fd90 0x7fdb741e0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.058+0000 7fdb79000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb740ad440 0x7fdb7414bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.058+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb740c9380 0x7fdb741e08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54588/0 (socket says 172.21.15.67:54588) 2024-03-09T09:15:17.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.058+0000 7fdb73e00640 1 -- 172.21.15.67:0/3448892509 learned_addr learned my addr 172.21.15.67:0/3448892509 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:17.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.058+0000 7fdb79a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7414fd90 0x7fdb741e0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:60620/0 (socket says 172.21.15.67:60620) 2024-03-09T09:15:17.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.059+0000 7fdb79a00640 1 -- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb740ad440 msgr2=0x7fdb7414bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.059+0000 7fdb79a00640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb740ad440 0x7fdb7414bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.059+0000 7fdb79a00640 1 -- 172.21.15.67:0/3448892509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb740c9380 msgr2=0x7fdb741e08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.059+0000 7fdb79a00640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb740c9380 0x7fdb741e08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.059+0000 7fdb79a00640 1 -- 172.21.15.67:0/3448892509 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb741eba10 con 0x7fdb7414fd90 2024-03-09T09:15:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.059+0000 7fdb73e00640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdb740c9380 0x7fdb741e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:15:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.059+0000 7fdb79a00640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7414fd90 0x7fdb741e0df0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00c7b0 tx=0x7fdb6c00cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.060+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c017070 con 0x7fdb7414fd90 2024-03-09T09:15:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.060+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6c003b90 con 0x7fdb7414fd90 2024-03-09T09:15:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.060+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c005070 con 0x7fdb7414fd90 2024-03-09T09:15:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.060+0000 7fdb7a400640 1 -- 172.21.15.67:0/3448892509 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb741ebcf0 con 0x7fdb7414fd90 2024-03-09T09:15:17.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.060+0000 7fdb7a400640 1 -- 172.21.15.67:0/3448892509 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb741ec2b0 con 0x7fdb7414fd90 2024-03-09T09:15:17.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.061+0000 7fdb7a400640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741558c0 0x7fdb74153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.061+0000 7fdb7a400640 1 -- 172.21.15.67:0/3448892509 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb741e1330 con 0x7fdb741558c0 2024-03-09T09:15:17.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.062+0000 7fdb79000640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741558c0 0x7fdb74153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.062+0000 7fdb79000640 1 -- 172.21.15.67:0/3448892509 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb7404f3c0 con 0x7fdb741558c0 2024-03-09T09:15:17.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.063+0000 7fdb79000640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741558c0 0x7fdb74153950 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdb68007fd0 tx=0x7fdb68001880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:17.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.063+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68011580 con 0x7fdb741558c0 2024-03-09T09:15:17.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.063+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fdb68011720 con 0x7fdb741558c0 2024-03-09T09:15:17.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.063+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741558c0 msgr2=0x7fdb74153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.063+0000 7fdb70c00640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741558c0 0x7fdb74153950 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdb68007fd0 tx=0x7fdb68001880 comp rx=0 tx=0).stop 2024-03-09T09:15:17.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.063+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741558c0 msgr2=0x7fdb74153950 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:17.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.063+0000 7fdb70c00640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741558c0 0x7fdb74153950 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdb68007fd0 tx=0x7fdb68001880 comp rx=0 tx=0).stop 2024-03-09T09:15:17.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.064+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdb6c005210 con 0x7fdb7414fd90 2024-03-09T09:15:17.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.065+0000 7fdb70c00640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb5406afb0 0x7fdb5406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.066+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdb6c089b80 con 0x7fdb7414fd90 2024-03-09T09:15:17.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.066+0000 7fdb79000640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb5406afb0 0x7fdb5406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.067+0000 7fdb79000640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb5406afb0 0x7fdb5406d470 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdb68001da0 tx=0x7fdb68009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:17.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.078+0000 7fdb7a400640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741ec4f0 0x7fdb74153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.078+0000 7fdb7a400640 1 -- 172.21.15.67:0/3448892509 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb741e20c0 con 0x7fdb741ec4f0 2024-03-09T09:15:17.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.079+0000 7fdb73e00640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741ec4f0 0x7fdb74153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.079+0000 7fdb73e00640 1 -- 172.21.15.67:0/3448892509 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdb7406d2d0 con 0x7fdb741ec4f0 2024-03-09T09:15:17.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.081+0000 7fdb73e00640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741ec4f0 0x7fdb74153e90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdb60000c00 tx=0x7fdb600046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:17.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.081+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60042070 con 0x7fdb741ec4f0 2024-03-09T09:15:17.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.081+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7fdb60032b40 con 0x7fdb741ec4f0 2024-03-09T09:15:17.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.081+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741ec4f0 msgr2=0x7fdb74153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.081+0000 7fdb70c00640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741ec4f0 0x7fdb74153e90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdb60000c00 tx=0x7fdb600046c0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.081+0000 7fdb70c00640 1 -- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741ec4f0 msgr2=0x7fdb74153e90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:17.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.081+0000 7fdb70c00640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb741ec4f0 0x7fdb74153e90 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdb60000c00 tx=0x7fdb600046c0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.086 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:15:17.086 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 172, 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:15:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:15:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:15:17.098 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.082+0000 7fdb7a400640 1 -- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb5406afb0 msgr2=0x7fdb5406d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.082+0000 7fdb7a400640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdb5406afb0 0x7fdb5406d470 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdb68001da0 tx=0x7fdb68009030 comp rx=0 tx=0).stop 2024-03-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.082+0000 7fdb7a400640 1 -- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7414fd90 msgr2=0x7fdb741e0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.082+0000 7fdb7a400640 1 --2- 172.21.15.67:0/3448892509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb7414fd90 0x7fdb741e0df0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00c7b0 tx=0x7fdb6c00cc70 comp rx=0 tx=0).stop 2024-03-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.082+0000 7fdb79a00640 1 -- 172.21.15.67:0/3448892509 reap_dead start 2024-03-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.082+0000 7fdb79a00640 1 -- 172.21.15.67:0/3448892509 reap_dead start 2024-03-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.082+0000 7fdb7a400640 1 -- 172.21.15.67:0/3448892509 shutdown_connections 2024-03-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.082+0000 7fdb7a400640 1 -- 172.21.15.67:0/3448892509 >> 172.21.15.67:0/3448892509 conn(0x7fdb740cb480 msgr2=0x7fdb7406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.083+0000 7fdb7a400640 1 -- 172.21.15.67:0/3448892509 shutdown_connections 2024-03-09T09:15:17.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.083+0000 7fdb7a400640 1 -- 172.21.15.67:0/3448892509 wait complete. 2024-03-09T09:15:17.111 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:15:17.111 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-03-09T09:15:17.111 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:15:17.111 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-03-09T09:15:17.111 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:15:17.111 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-09T09:15:17.111 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:15:17.146 INFO:tasks.ceph.mon.a:Stopped 2024-03-09T09:15:17.146 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:15:17.146 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:15:17.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.360+0000 7fb00de00640 1 Processor -- start 2024-03-09T09:15:17.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.360+0000 7fb00de00640 1 -- start start 2024-03-09T09:15:17.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.362+0000 7fb00de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0080c5f50 0x7fb008155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.362+0000 7fb00de00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0080c6a40 con 0x7fb00806c650 2024-03-09T09:15:17.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.362+0000 7fb00de00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00806b5b0 con 0x7fb00806b040 2024-03-09T09:15:17.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.362+0000 7fb00de00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0080c9e00 con 0x7fb0080c5f50 2024-03-09T09:15:17.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.362+0000 7fb006a00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fb0080c5f50 msgr2=0x7fb008155e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.362+0000 7fb007400640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7fb00806c650 legacy=0x7fb0080c6330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.362+0000 7fb006a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0080c5f50 0x7fb008155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.362+0000 7fb006000640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb00806b040 0x7fb0080c96f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:35448/0 (socket says 172.21.15.67:35448) 2024-03-09T09:15:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.362+0000 7fb006000640 1 -- 172.21.15.67:0/2158506668 learned_addr learned my addr 172.21.15.67:0/2158506668 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:15:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.363+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208838573 0 0) 0x7fb00806b5b0 con 0x7fb00806b040 2024-03-09T09:15:17.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.363+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafec003520 con 0x7fb00806b040 2024-03-09T09:15:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 493001869 0 0) 0x7fafec003520 con 0x7fb00806b040 2024-03-09T09:15:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 >> v2:172.21.15.67:3300/0 conn(0x7fb0080c5f50 msgr2=0x7fb008155e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 --2- 172.21.15.67:0/2158506668 >> v2:172.21.15.67:3300/0 conn(0x7fb0080c5f50 0x7fb008155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 >> v1:172.21.15.67:6789/0 conn(0x7fb00806c650 legacy=0x7fb0080c6330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:17.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00806b890 con 0x7fb00806b040 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7faff4001670 con 0x7fb00806b040 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 >> v1:172.21.15.134:6789/0 conn(0x7fb00806b040 legacy=0x7fb0080c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 --2- 172.21.15.67:0/2158506668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafec0039b0 0x7fafec003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 --2- 172.21.15.67:0/2158506668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafec004800 0x7fafec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 --2- 172.21.15.67:0/2158506668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafec005230 0x7fafec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080c9e00 con 0x7fafec005230 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec00c190 con 0x7fafec0039b0 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec00c390 con 0x7fafec004800 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb007400640 1 -- 172.21.15.67:0/2158506668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafec005230 msgr2=0x7fafec00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb007400640 1 --2- 172.21.15.67:0/2158506668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafec005230 0x7fafec00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb006000640 1 --2- 172.21.15.67:0/2158506668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafec004800 0x7fafec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.364+0000 7fb006a00640 1 --2- 172.21.15.67:0/2158506668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafec0039b0 0x7fafec003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.365+0000 7fb006a00640 1 -- 172.21.15.67:0/2158506668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafec004800 msgr2=0x7fafec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.365+0000 7fb006a00640 1 --2- 172.21.15.67:0/2158506668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafec004800 0x7fafec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.365+0000 7fb006a00640 1 -- 172.21.15.67:0/2158506668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafec005230 msgr2=0x7fafec00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.365+0000 7fb006a00640 1 --2- 172.21.15.67:0/2158506668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafec005230 0x7fafec00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.365+0000 7fb006a00640 1 -- 172.21.15.67:0/2158506668 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec003520 con 0x7fafec0039b0 2024-03-09T09:15:17.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.365+0000 7fb007400640 1 -- 172.21.15.67:0/2158506668 reap_dead start 2024-03-09T09:15:17.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.366+0000 7fb006a00640 1 --2- 172.21.15.67:0/2158506668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafec0039b0 0x7fafec003db0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faff8009900 tx=0x7fafec005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f27d324e28f594e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:17.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.367+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8009500 con 0x7fafec0039b0 2024-03-09T09:15:17.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.367+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff802f070 con 0x7fafec0039b0 2024-03-09T09:15:17.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.367+0000 7fb005600640 1 -- 172.21.15.67:0/2158506668 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8003df0 con 0x7fafec0039b0 2024-03-09T09:15:17.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.367+0000 7fb00de00640 1 -- 172.21.15.67:0/2158506668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafec0039b0 msgr2=0x7fafec003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.367+0000 7fb00de00640 1 --2- 172.21.15.67:0/2158506668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafec0039b0 0x7fafec003db0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faff8009900 tx=0x7fafec005830 comp rx=0 tx=0).stop 2024-03-09T09:15:17.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.367+0000 7fb00de00640 1 -- 172.21.15.67:0/2158506668 shutdown_connections 2024-03-09T09:15:17.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.367+0000 7fb00de00640 1 --2- 172.21.15.67:0/2158506668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafec0039b0 0x7fafec003db0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.367+0000 7fb00de00640 1 -- 172.21.15.67:0/2158506668 >> 172.21.15.67:0/2158506668 conn(0x7fb0080c8050 msgr2=0x7fb0080c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:17.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.367+0000 7fb00de00640 1 -- 172.21.15.67:0/2158506668 shutdown_connections 2024-03-09T09:15:17.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.367+0000 7fb00de00640 1 -- 172.21.15.67:0/2158506668 wait complete. 2024-03-09T09:15:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.369+0000 7fb00de00640 1 Processor -- start 2024-03-09T09:15:17.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.369+0000 7fb00de00640 1 -- start start 2024-03-09T09:15:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.370+0000 7fb00de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafec0039b0 0x7fb0081ef760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.370+0000 7fb00de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0080c1a00 0x7fb0080c2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.370+0000 7fb00de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0081e18c0 0x7fb0080bf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.370+0000 7fb00de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00806be50 con 0x7fb0081e18c0 2024-03-09T09:15:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.370+0000 7fb00de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00806cac0 con 0x7fb0080c1a00 2024-03-09T09:15:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.370+0000 7fb00de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00806c130 con 0x7fafec0039b0 2024-03-09T09:15:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.370+0000 7fb007400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0081e18c0 msgr2=0x7fb0080bf070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.370+0000 7fb007400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0081e18c0 0x7fb0080bf070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.371+0000 7fb006000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0080c1a00 0x7fb0080c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.371+0000 7fb006a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafec0039b0 0x7fb0081ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.371+0000 7fb006000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0080c1a00 0x7fb0080c2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46186/0 (socket says 172.21.15.67:46186) 2024-03-09T09:15:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.371+0000 7fb006000640 1 -- 172.21.15.67:0/1623053351 learned_addr learned my addr 172.21.15.67:0/1623053351 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.371+0000 7fb006000640 1 -- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafec0039b0 msgr2=0x7fb0081ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.371+0000 7fb006000640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafec0039b0 0x7fb0081ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.371+0000 7fb006000640 1 -- 172.21.15.67:0/1623053351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0081e18c0 msgr2=0x7fb0080bf070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:15:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.371+0000 7fb006000640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0081e18c0 0x7fb0080bf070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.371+0000 7fb006000640 1 -- 172.21.15.67:0/1623053351 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0081efca0 con 0x7fb0080c1a00 2024-03-09T09:15:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.372+0000 7fb006000640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0080c1a00 0x7fb0080c2610 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faff4007fd0 tx=0x7faff4007430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.372+0000 7fafff400640 1 -- 172.21.15.67:0/1623053351 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff400f730 con 0x7fb0080c1a00 2024-03-09T09:15:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.372+0000 7fafff400640 1 -- 172.21.15.67:0/1623053351 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff4018070 con 0x7fb0080c1a00 2024-03-09T09:15:17.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.372+0000 7fafff400640 1 -- 172.21.15.67:0/1623053351 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff400fb50 con 0x7fb0080c1a00 2024-03-09T09:15:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.372+0000 7fb00de00640 1 -- 172.21.15.67:0/1623053351 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb008145b30 con 0x7fb0080c1a00 2024-03-09T09:15:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.372+0000 7fb00de00640 1 -- 172.21.15.67:0/1623053351 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0080c0f20 con 0x7fb0080c1a00 2024-03-09T09:15:17.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.373+0000 7fb00de00640 1 -- 172.21.15.67:0/1623053351 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb008075a00 con 0x7fb0080c1a00 2024-03-09T09:15:17.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.375+0000 7fafff400640 1 -- 172.21.15.67:0/1623053351 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faff400fcf0 con 0x7fb0080c1a00 2024-03-09T09:15:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.376+0000 7fafff400640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafe806b4b0 0x7fafe806d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.376+0000 7fafff400640 1 -- 172.21.15.67:0/1623053351 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7faff408ade0 con 0x7fb0080c1a00 2024-03-09T09:15:17.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.376+0000 7fb006a00640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafe806b4b0 0x7fafe806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:17.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.377+0000 7fb006a00640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafe806b4b0 0x7fafe806d970 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7faff8033000 tx=0x7faff8005000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:17.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.391+0000 7fafff400640 1 -- 172.21.15.67:0/1623053351 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faff4059aa0 con 0x7fb0080c1a00 2024-03-09T09:15:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.853+0000 7fb00de00640 1 -- 172.21.15.67:0/1623053351 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb00806cca0 con 0x7fb0080c1a00 2024-03-09T09:15:17.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.854+0000 7fafff400640 1 -- 172.21.15.67:0/1623053351 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7faff40593a0 con 0x7fb0080c1a00 2024-03-09T09:15:17.859 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":172,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:15:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 -- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafe806b4b0 msgr2=0x7fafe806d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafe806b4b0 0x7fafe806d970 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7faff8033000 tx=0x7faff8005000 comp rx=0 tx=0).stop 2024-03-09T09:15:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 -- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0080c1a00 msgr2=0x7fb0080c2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0080c1a00 0x7fb0080c2610 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faff4007fd0 tx=0x7faff4007430 comp rx=0 tx=0).stop 2024-03-09T09:15:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 -- 172.21.15.67:0/1623053351 shutdown_connections 2024-03-09T09:15:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafe806b4b0 0x7fafe806d970 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0081e18c0 0x7fb0080bf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0080c1a00 0x7fb0080c2610 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 --2- 172.21.15.67:0/1623053351 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fafec0039b0 0x7fb0081ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:17.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 -- 172.21.15.67:0/1623053351 >> 172.21.15.67:0/1623053351 conn(0x7fb0080c8050 msgr2=0x7fb0080ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 -- 172.21.15.67:0/1623053351 shutdown_connections 2024-03-09T09:15:17.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:17.857+0000 7fb00de00640 1 -- 172.21.15.67:0/1623053351 wait complete. 2024-03-09T09:15:20.869 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 24% (80/329) Updating files: 25% (83/329) Updating files: 26% (86/329) Updating files: 27% (89/329) Updating files: 28% (93/329) Updating files: 29% (96/329) Updating files: 30% (99/329) Updating files: 31% (102/329) Updating files: 32% (106/329) Updating files: 33% (109/329) Updating files: 34% (112/329) Updating files: 35% (116/329) Updating files: 36% (119/329) Updating files: 37% (122/329) Updating files: 38% (126/329) Updating files: 39% (129/329) Updating files: 40% (132/329) Updating files: 41% (135/329) Updating files: 42% (139/329) Updating files: 43% (142/329) Updating files: 44% (145/329) Updating files: 45% (149/329) Updating files: 46% (152/329) Updating files: 47% (155/329) Updating files: 48% (158/329) Updating files: 49% (162/329) Updating files: 50% (165/329) Updating files: 51% (168/329) Updating files: 52% (172/329) Updating files: 53% (175/329) Updating files: 54% (178/329) Updating files: 55% (181/329) Updating files: 56% (185/329) Updating files: 57% (188/329) Updating files: 58% (191/329) Updating files: 59% (195/329) Updating files: 60% (198/329) Updating files: 60% (200/329) Updating files: 61% (201/329) Updating files: 62% (204/329) Updating files: 63% (208/329) Updating files: 64% (211/329) Updating files: 65% (214/329) Updating files: 66% (218/329) Updating files: 67% (221/329) Updating files: 68% (224/329) Updating files: 69% (228/329) Updating files: 70% (231/329) Updating files: 71% (234/329) Updating files: 72% (237/329) Updating files: 73% (241/329) Updating files: 74% (244/329) Updating files: 75% (247/329) Updating files: 76% (251/329) Updating files: 77% (254/329) Updating files: 78% (257/329) Updating files: 78% (259/329) Updating files: 79% (260/329) Updating files: 80% (264/329) Updating files: 81% (267/329) Updating files: 82% (270/329) Updating files: 83% (274/329) Updating files: 84% (277/329) Updating files: 85% (280/329) Updating files: 86% (283/329) Updating files: 87% (287/329) Updating files: 88% (290/329) Updating files: 89% (293/329) Updating files: 90% (297/329) Updating files: 91% (300/329) Updating files: 92% (303/329) Updating files: 93% (306/329) Updating files: 93% (308/329) Updating files: 94% (310/329) Updating files: 95% (313/329) Updating files: 96% (316/329) Updating files: 97% (320/329) Updating files: 98% (323/329) Updating files: 99% (326/329) Updating files: 100% (329/329) Updating files: 100% (329/329), done. 2024-03-09T09:15:20.880 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:15:21.008 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 4c2d73a5095 0.79 2024-03-09T09:15:21.017 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.79 2024-03-09T09:15:21.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.085+0000 7f389e200640 1 Processor -- start 2024-03-09T09:15:21.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.086+0000 7f389e200640 1 -- start start 2024-03-09T09:15:21.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.088+0000 7f389e200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f38980c30f0 0x7f3898152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:21.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.088+0000 7f389e200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f38980aa420 0x7f38981490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:21.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.088+0000 7f389e200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38980ab3c0 con 0x7f3898149630 2024-03-09T09:15:21.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.088+0000 7f389e200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38980c82d0 con 0x7f38980aa420 2024-03-09T09:15:21.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.088+0000 7f389e200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38980aa8a0 con 0x7f38980c30f0 2024-03-09T09:15:21.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.088+0000 7f3896a00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f38980aa420 msgr2=0x7f38981490f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:21.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.088+0000 7f3896a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f38980aa420 0x7f38981490f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:21.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.088+0000 7f3897400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f38980c30f0 0x7f3898152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:21.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.088+0000 7f3897400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f38980c30f0 0x7f3898152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:42174/0 (socket says 172.21.15.67:42174) 2024-03-09T09:15:21.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.088+0000 7f3897400640 1 -- 172.21.15.67:0/3283961423 learned_addr learned my addr 172.21.15.67:0/3283961423 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:21.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.089+0000 7f3897400640 1 -- 172.21.15.67:0/3283961423 >> v2:172.21.15.67:3300/0 conn(0x7f38980aa420 msgr2=0x7f38981490f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:21.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.089+0000 7f3897400640 1 --2- 172.21.15.67:0/3283961423 >> v2:172.21.15.67:3300/0 conn(0x7f38980aa420 0x7f38981490f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:21.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.089+0000 7f3897400640 1 -- 172.21.15.67:0/3283961423 >> v1:172.21.15.134:6789/0 conn(0x7f3898149630 legacy=0x7f389806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:21.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.089+0000 7f3897400640 1 -- 172.21.15.67:0/3283961423 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38980aab20 con 0x7f38980c30f0 2024-03-09T09:15:21.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.090+0000 7f3897400640 1 --2- 172.21.15.67:0/3283961423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38980c30f0 0x7f3898152a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f38980702f0 tx=0x7f388802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=533efb66cadbbcc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:21.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.091+0000 7f3896000640 1 -- 172.21.15.67:0/3283961423 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888034070 con 0x7f38980c30f0 2024-03-09T09:15:21.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.091+0000 7f3896000640 1 -- 172.21.15.67:0/3283961423 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3888030800 con 0x7f38980c30f0 2024-03-09T09:15:21.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.091+0000 7f3896000640 1 -- 172.21.15.67:0/3283961423 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888030bd0 con 0x7f38980c30f0 2024-03-09T09:15:21.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.091+0000 7f389e200640 1 -- 172.21.15.67:0/3283961423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38980c30f0 msgr2=0x7f3898152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:21.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.091+0000 7f389e200640 1 --2- 172.21.15.67:0/3283961423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38980c30f0 0x7f3898152a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f38980702f0 tx=0x7f388802b940 comp rx=0 tx=0).stop 2024-03-09T09:15:21.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.091+0000 7f389e200640 1 -- 172.21.15.67:0/3283961423 shutdown_connections 2024-03-09T09:15:21.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.091+0000 7f389e200640 1 --2- 172.21.15.67:0/3283961423 >> v2:172.21.15.67:3300/0 conn(0x7f38980aa420 0x7f38981490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:21.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.091+0000 7f389e200640 1 --2- 172.21.15.67:0/3283961423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38980c30f0 0x7f3898152a20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:21.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.091+0000 7f389e200640 1 -- 172.21.15.67:0/3283961423 >> 172.21.15.67:0/3283961423 conn(0x7f38980c7370 msgr2=0x7f38980c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:21.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.091+0000 7f389e200640 1 -- 172.21.15.67:0/3283961423 shutdown_connections 2024-03-09T09:15:21.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.091+0000 7f389e200640 1 -- 172.21.15.67:0/3283961423 wait complete. 2024-03-09T09:15:21.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.093+0000 7f389e200640 1 Processor -- start 2024-03-09T09:15:21.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.093+0000 7f389e200640 1 -- start start 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.094+0000 7f389e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38980aa420 0x7f3898152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.094+0000 7f389e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38980c30f0 0x7f3898147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.094+0000 7f389e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3898149630 0x7f38981480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.094+0000 7f389e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389806d6c0 con 0x7f3898149630 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.094+0000 7f389e200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38980ab0e0 con 0x7f38980aa420 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.094+0000 7f389e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38980ab620 con 0x7f38980c30f0 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.094+0000 7f3897e00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3898149630 msgr2=0x7f38981480c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.094+0000 7f3897e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3898149630 0x7f38981480c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.094+0000 7f3897400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38980aa420 0x7f3898152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.095+0000 7f3897400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38980aa420 0x7f3898152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:42190/0 (socket says 172.21.15.67:42190) 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.095+0000 7f3897400640 1 -- 172.21.15.67:0/3319841315 learned_addr learned my addr 172.21.15.67:0/3319841315 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:21.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.095+0000 7f3896a00640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38980c30f0 0x7f3898147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:21.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.095+0000 7f3897400640 1 -- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38980c30f0 msgr2=0x7f3898147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:21.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.095+0000 7f3897400640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38980c30f0 0x7f3898147b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:21.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.095+0000 7f3897400640 1 -- 172.21.15.67:0/3319841315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3898149630 msgr2=0x7f38981480c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:15:21.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.095+0000 7f3897400640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3898149630 0x7f38981480c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:21.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.095+0000 7f3897400640 1 -- 172.21.15.67:0/3319841315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38981ebe60 con 0x7f38980aa420 2024-03-09T09:15:21.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.096+0000 7f3897400640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38980aa420 0x7f3898152630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3888000c00 tx=0x7f38880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:21.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.096+0000 7f388f400640 1 -- 172.21.15.67:0/3319841315 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888034040 con 0x7f38980aa420 2024-03-09T09:15:21.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.096+0000 7f388f400640 1 -- 172.21.15.67:0/3319841315 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3888039d90 con 0x7f38980aa420 2024-03-09T09:15:21.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.096+0000 7f388f400640 1 -- 172.21.15.67:0/3319841315 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888043610 con 0x7f38980aa420 2024-03-09T09:15:21.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.096+0000 7f389e200640 1 -- 172.21.15.67:0/3319841315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38981ec140 con 0x7f38980aa420 2024-03-09T09:15:21.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.096+0000 7f389e200640 1 -- 172.21.15.67:0/3319841315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38981ec700 con 0x7f38980aa420 2024-03-09T09:15:21.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.098+0000 7f389e200640 1 -- 172.21.15.67:0/3319841315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3898075a00 con 0x7f38980aa420 2024-03-09T09:15:21.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.100+0000 7f388f400640 1 -- 172.21.15.67:0/3319841315 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f388804a060 con 0x7f38980aa420 2024-03-09T09:15:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.101+0000 7f388f400640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f387806b480 0x7f387806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.101+0000 7f388f400640 1 -- 172.21.15.67:0/3319841315 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f38880ac780 con 0x7f38980aa420 2024-03-09T09:15:21.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.101+0000 7f3896a00640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f387806b480 0x7f387806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:21.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.102+0000 7f3896a00640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f387806b480 0x7f387806d940 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3884009390 tx=0x7f3884009320 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:21.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.118+0000 7f388f400640 1 -- 172.21.15.67:0/3319841315 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f388803e070 con 0x7f38980aa420 2024-03-09T09:15:21.386 INFO:tasks.workunit.client.0.smithi067.stdout:v0.80 2024-03-09T09:15:21.386 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:15:21.386 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 80 -eq 48 ']' 2024-03-09T09:15:21.386 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.80 2024-03-09T09:15:21.386 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.80 2024-03-09T09:15:21.386 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.80 2024-03-09T09:15:21.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.466+0000 7f389e200640 1 -- 172.21.15.67:0/3319841315 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f389814d030 con 0x7f38980aa420 2024-03-09T09:15:21.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.467+0000 7f388f400640 1 -- 172.21.15.67:0/3319841315 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f388807b4a0 con 0x7f38980aa420 2024-03-09T09:15:21.473 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":172,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:15:21.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.469+0000 7f389e200640 1 -- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f387806b480 msgr2=0x7f387806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:21.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.469+0000 7f389e200640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f387806b480 0x7f387806d940 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3884009390 tx=0x7f3884009320 comp rx=0 tx=0).stop 2024-03-09T09:15:21.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.469+0000 7f389e200640 1 -- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38980aa420 msgr2=0x7f3898152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:21.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.469+0000 7f389e200640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38980aa420 0x7f3898152630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3888000c00 tx=0x7f38880046c0 comp rx=0 tx=0).stop 2024-03-09T09:15:21.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.470+0000 7f389e200640 1 -- 172.21.15.67:0/3319841315 shutdown_connections 2024-03-09T09:15:21.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.470+0000 7f389e200640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f387806b480 0x7f387806d940 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:21.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.470+0000 7f389e200640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3898149630 0x7f38981480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:21.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.470+0000 7f389e200640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38980c30f0 0x7f3898147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:21.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.470+0000 7f389e200640 1 --2- 172.21.15.67:0/3319841315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38980aa420 0x7f3898152630 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:21.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.470+0000 7f389e200640 1 -- 172.21.15.67:0/3319841315 >> 172.21.15.67:0/3319841315 conn(0x7f38980c7370 msgr2=0x7f3898071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:21.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.470+0000 7f389e200640 1 -- 172.21.15.67:0/3319841315 shutdown_connections 2024-03-09T09:15:21.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:21.470+0000 7f389e200640 1 -- 172.21.15.67:0/3319841315 wait complete. 2024-03-09T09:15:24.338 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 27% (49/176) Updating files: 28% (50/176) Updating files: 29% (52/176) Updating files: 30% (53/176) Updating files: 31% (55/176) Updating files: 32% (57/176) Updating files: 33% (59/176) Updating files: 34% (60/176) Updating files: 35% (62/176) Updating files: 36% (64/176) Updating files: 37% (66/176) Updating files: 38% (67/176) Updating files: 39% (69/176) Updating files: 40% (71/176) Updating files: 41% (73/176) Updating files: 42% (74/176) Updating files: 43% (76/176) Updating files: 44% (78/176) Updating files: 45% (80/176) Updating files: 46% (81/176) Updating files: 47% (83/176) Updating files: 48% (85/176) Updating files: 49% (87/176) Updating files: 50% (88/176) Updating files: 51% (90/176) Updating files: 52% (92/176) Updating files: 53% (94/176) Updating files: 54% (96/176) Updating files: 55% (97/176) Updating files: 56% (99/176) Updating files: 57% (101/176) Updating files: 58% (103/176) Updating files: 59% (104/176) Updating files: 60% (106/176) Updating files: 61% (108/176) Updating files: 62% (110/176) Updating files: 63% (111/176) Updating files: 64% (113/176) Updating files: 65% (115/176) Updating files: 66% (117/176) Updating files: 67% (118/176) Updating files: 68% (120/176) Updating files: 69% (122/176) Updating files: 70% (124/176) Updating files: 71% (125/176) Updating files: 72% (127/176) Updating files: 73% (129/176) Updating files: 74% (131/176) Updating files: 75% (132/176) Updating files: 76% (134/176) Updating files: 77% (136/176) Updating files: 78% (138/176) Updating files: 79% (140/176) Updating files: 80% (141/176) Updating files: 81% (143/176) Updating files: 82% (145/176) Updating files: 83% (147/176) Updating files: 84% (148/176) Updating files: 85% (150/176) Updating files: 86% (152/176) Updating files: 87% (154/176) Updating files: 88% (155/176) Updating files: 89% (157/176) Updating files: 90% (159/176) Updating files: 91% (161/176) Updating files: 92% (162/176) Updating files: 93% (164/176) Updating files: 94% (166/176) Updating files: 95% (168/176) Updating files: 96% (169/176) Updating files: 97% (171/176) Updating files: 98% (173/176) Updating files: 99% (175/176) Updating files: 100% (176/176) Updating files: 100% (176/176), done. 2024-03-09T09:15:24.435 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at b78644e7dee 0.80 2024-03-09T09:15:24.458 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.80 2024-03-09T09:15:24.501 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:15:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.688+0000 7f0316e00640 1 Processor -- start 2024-03-09T09:15:24.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.688+0000 7f0316e00640 1 -- start start 2024-03-09T09:15:24.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.690+0000 7f0316e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f03100c2e80 0x7f03100c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:24.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.690+0000 7f0315a00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f03101532c0 legacy=0x7f031006fef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:24.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.690+0000 7f0316e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03100729e0 con 0x7f03101532c0 2024-03-09T09:15:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.690+0000 7f0316e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0310072cc0 con 0x7f0310155e40 2024-03-09T09:15:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.690+0000 7f0316e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0310073310 con 0x7f03100c2e80 2024-03-09T09:15:24.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.690+0000 7f0315000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f03100c2e80 0x7f03100c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:24.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.691+0000 7f0315000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f03100c2e80 0x7f03100c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:42198/0 (socket says 172.21.15.67:42198) 2024-03-09T09:15:24.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.691+0000 7f0315000640 1 -- 172.21.15.67:0/2508370574 learned_addr learned my addr 172.21.15.67:0/2508370574 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.691+0000 7f0315000640 1 -- 172.21.15.67:0/2508370574 >> v1:172.21.15.134:6789/0 conn(0x7f0310155e40 legacy=0x7f0310156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.691+0000 7f0315000640 1 -- 172.21.15.67:0/2508370574 >> v1:172.21.15.67:6789/0 conn(0x7f03101532c0 legacy=0x7f031006fef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.691+0000 7f0315000640 1 -- 172.21.15.67:0/2508370574 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0310072fa0 con 0x7f03100c2e80 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.692+0000 7f0315000640 1 --2- 172.21.15.67:0/2508370574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03100c2e80 0x7f03100c3510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009870 tx=0x7f02fc02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38fb55d615940f9b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.693+0000 7f030fe00640 1 -- 172.21.15.67:0/2508370574 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc035020 con 0x7f03100c2e80 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.693+0000 7f030fe00640 1 -- 172.21.15.67:0/2508370574 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc031870 con 0x7f03100c2e80 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.693+0000 7f030fe00640 1 -- 172.21.15.67:0/2508370574 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc031c40 con 0x7f03100c2e80 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.694+0000 7f0316e00640 1 -- 172.21.15.67:0/2508370574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03100c2e80 msgr2=0x7f03100c3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.694+0000 7f0316e00640 1 --2- 172.21.15.67:0/2508370574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03100c2e80 0x7f03100c3510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009870 tx=0x7f02fc02b910 comp rx=0 tx=0).stop 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.694+0000 7f0316e00640 1 -- 172.21.15.67:0/2508370574 shutdown_connections 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.694+0000 7f0316e00640 1 --2- 172.21.15.67:0/2508370574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03100c2e80 0x7f03100c3510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.694+0000 7f0316e00640 1 -- 172.21.15.67:0/2508370574 >> 172.21.15.67:0/2508370574 conn(0x7f0310066d90 msgr2=0x7f0310067170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.694+0000 7f0316e00640 1 -- 172.21.15.67:0/2508370574 shutdown_connections 2024-03-09T09:15:24.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.694+0000 7f0316e00640 1 -- 172.21.15.67:0/2508370574 wait complete. 2024-03-09T09:15:24.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.697+0000 7f0316e00640 1 Processor -- start 2024-03-09T09:15:24.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.697+0000 7f0316e00640 1 -- start start 2024-03-09T09:15:24.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.698+0000 7f0316e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03100c2e80 0x7f0310159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.698+0000 7f0316e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03101532c0 0x7f031014ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.698+0000 7f0316e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0310155e40 0x7f03101ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.698+0000 7f0316e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03101569a0 con 0x7f0310155e40 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.698+0000 7f0316e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03100c2ab0 con 0x7f03100c2e80 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.698+0000 7f0316e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0310156b20 con 0x7f03101532c0 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.698+0000 7f0316400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0310155e40 msgr2=0x7f03101ef4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.698+0000 7f0316400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0310155e40 0x7f03101ef4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.699+0000 7f0315000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03101532c0 0x7f031014ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.699+0000 7f0315a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03100c2e80 0x7f0310159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.699+0000 7f0315000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03101532c0 0x7f031014ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:59444/0 (socket says 172.21.15.67:59444) 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.699+0000 7f0315000640 1 -- 172.21.15.67:0/2567869065 learned_addr learned my addr 172.21.15.67:0/2567869065 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:24.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.699+0000 7f0315a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03100c2e80 0x7f0310159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:42206/0 (socket says 172.21.15.67:42206) 2024-03-09T09:15:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.700+0000 7f0315000640 1 -- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03100c2e80 msgr2=0x7f0310159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:24.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.700+0000 7f0315000640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03100c2e80 0x7f0310159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.700+0000 7f0315000640 1 -- 172.21.15.67:0/2567869065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0310155e40 msgr2=0x7f03101ef4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:15:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.700+0000 7f0315000640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0310155e40 0x7f03101ef4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.700+0000 7f0315000640 1 -- 172.21.15.67:0/2567869065 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03101e07a0 con 0x7f03101532c0 2024-03-09T09:15:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.700+0000 7f0315000640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03101532c0 0x7f031014ea50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009870 tx=0x7f02fc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.701+0000 7f030d600640 1 -- 172.21.15.67:0/2567869065 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0443f0 con 0x7f03101532c0 2024-03-09T09:15:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.701+0000 7f0316e00640 1 -- 172.21.15.67:0/2567869065 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03101e0a80 con 0x7f03101532c0 2024-03-09T09:15:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.701+0000 7f030d600640 1 -- 172.21.15.67:0/2567869065 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc0445b0 con 0x7f03101532c0 2024-03-09T09:15:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.701+0000 7f030d600640 1 -- 172.21.15.67:0/2567869065 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0449a0 con 0x7f03101532c0 2024-03-09T09:15:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.701+0000 7f0316e00640 1 -- 172.21.15.67:0/2567869065 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03101eef90 con 0x7f03101532c0 2024-03-09T09:15:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.702+0000 7f0316e00640 1 -- 172.21.15.67:0/2567869065 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03101eedb0 con 0x7f03101532c0 2024-03-09T09:15:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.705+0000 7f030d600640 1 -- 172.21.15.67:0/2567869065 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f02fc04b060 con 0x7f03101532c0 2024-03-09T09:15:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.706+0000 7f030d600640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f02f006b4b0 0x7f02f006d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.706+0000 7f030d600640 1 -- 172.21.15.67:0/2567869065 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f02fc0acc10 con 0x7f03101532c0 2024-03-09T09:15:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.706+0000 7f0315a00640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f02f006b4b0 0x7f02f006d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.707+0000 7f0315a00640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f02f006b4b0 0x7f02f006d970 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0304009420 tx=0x7f030400d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:24.721+0000 7f030d600640 1 -- 172.21.15.67:0/2567869065 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f03101eedb0 con 0x7f03101532c0 2024-03-09T09:15:24.842 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:15:24.843 INFO:tasks.workunit.client.1.smithi134.stdout:v0.81 2024-03-09T09:15:24.844 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 81 -eq 48 ']' 2024-03-09T09:15:24.844 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.81 2024-03-09T09:15:24.844 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.81 2024-03-09T09:15:24.844 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.81 2024-03-09T09:15:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.070+0000 7f0316e00640 1 -- 172.21.15.67:0/2567869065 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f031004f3c0 con 0x7f03101532c0 2024-03-09T09:15:25.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.071+0000 7f030d600640 1 -- 172.21.15.67:0/2567869065 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f02fc03f070 con 0x7f03101532c0 2024-03-09T09:15:25.076 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":172,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 -- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f02f006b4b0 msgr2=0x7f02f006d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f02f006b4b0 0x7f02f006d970 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0304009420 tx=0x7f030400d040 comp rx=0 tx=0).stop 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 -- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03101532c0 msgr2=0x7f031014ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03101532c0 0x7f031014ea50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009870 tx=0x7f02fc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 -- 172.21.15.67:0/2567869065 shutdown_connections 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f02f006b4b0 0x7f02f006d970 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0310155e40 0x7f03101ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03101532c0 0x7f031014ea50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 --2- 172.21.15.67:0/2567869065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03100c2e80 0x7f0310159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 -- 172.21.15.67:0/2567869065 >> 172.21.15.67:0/2567869065 conn(0x7f0310066d90 msgr2=0x7f03100c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 -- 172.21.15.67:0/2567869065 shutdown_connections 2024-03-09T09:15:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:25.074+0000 7f0316e00640 1 -- 172.21.15.67:0/2567869065 wait complete. 2024-03-09T09:15:28.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:15:28.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.292+0000 7f0c37200640 1 Processor -- start 2024-03-09T09:15:28.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.292+0000 7f0c37200640 1 -- start start 2024-03-09T09:15:28.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.293+0000 7f0c37200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:28.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.293+0000 7f0c37200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:28.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.293+0000 7f0c37200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c380c81e0 con 0x7f0c380c1f70 2024-03-09T09:15:28.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.293+0000 7f0c37200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c380c32b0 con 0x7f0c380710a0 2024-03-09T09:15:28.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.293+0000 7f0c37200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c380c2fd0 con 0x7f0c38151360 2024-03-09T09:15:28.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c36800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:28.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c35400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:28.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c35e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f0c380c1f70 0x7f0c38150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:48640/0 (socket says 172.21.15.67:48640) 2024-03-09T09:15:28.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c35e00640 1 -- 172.21.15.67:0/1157752481 learned_addr learned my addr 172.21.15.67:0/1157752481 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:15:28.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 msgr2=0x7f0c38153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:15:28.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 msgr2=0x7f0c38153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:15:28.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 msgr2=0x7f0c38153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:28.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 msgr2=0x7f0c38153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:28.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:28.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:28.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:28.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.294+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:28.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.495+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:28.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.495+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:28.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.496+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 msgr2=0x7f0c38153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:15:28.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.496+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 msgr2=0x7f0c38153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:28.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.496+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:28.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.496+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:15:28.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.496+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 msgr2=0x7f0c38153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:15:28.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.496+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 msgr2=0x7f0c38153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:28.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.496+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:28.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.496+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:15:28.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.897+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:28.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.897+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:28.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.898+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 msgr2=0x7f0c38153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:15:28.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.898+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 msgr2=0x7f0c38153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:28.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.898+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:28.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.898+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:15:28.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.898+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 msgr2=0x7f0c38153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:15:28.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.898+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 msgr2=0x7f0c38153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:28.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.898+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:28.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:28.898+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:15:29.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:29.699+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:29.700+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:29.700+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 msgr2=0x7f0c38153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:15:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:29.700+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 msgr2=0x7f0c38153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:29.700+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:29.700+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:15:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:29.700+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 msgr2=0x7f0c38153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:15:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:29.700+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 msgr2=0x7f0c38153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:29.700+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:29.700+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:15:31.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.293+0000 7f0c2f400640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 msgr2=0x7f0c38153eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:31.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.293+0000 7f0c2f400640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c38151360 0x7f0c38153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:31.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.293+0000 7f0c2f400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 msgr2=0x7f0c38153970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:31.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.293+0000 7f0c2f400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c380710a0 0x7f0c38153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:31.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.293+0000 7f0c2f400640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.134:6789/0 conn(0x7f0c380c1f70 legacy=0x7f0c38150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:31.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.294+0000 7f0c2f400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.67:3300/0 conn(0x7f0c18007330 0x7f0c180097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:31.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.294+0000 7f0c35e00640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.67:6789/0 conn(0x7f0c180036a0 legacy=0x7f0c18005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:31.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.294+0000 7f0c2f400640 1 -- 172.21.15.67:0/1157752481 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c380c3700 con 0x7f0c180036a0 2024-03-09T09:15:31.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.294+0000 7f0c2f400640 1 -- 172.21.15.67:0/1157752481 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c380c32b0 con 0x7f0c18007330 2024-03-09T09:15:31.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.294+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.67:3300/0 conn(0x7f0c18007330 msgr2=0x7f0c180097f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:31.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.294+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.67:3300/0 conn(0x7f0c18007330 0x7f0c180097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:31.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.495+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.67:3300/0 conn(0x7f0c18007330 msgr2=0x7f0c180097f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:31.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.495+0000 7f0c35e00640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.67:6789/0 conn(0x7f0c180036a0 legacy=0x7f0c18005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:31.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.495+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.67:3300/0 conn(0x7f0c18007330 0x7f0c180097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:15:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.896+0000 7f0c35e00640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.67:6789/0 conn(0x7f0c180036a0 legacy=0x7f0c18005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:31.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.896+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.67:3300/0 conn(0x7f0c18007330 msgr2=0x7f0c180097f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:31.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:31.896+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.67:3300/0 conn(0x7f0c18007330 0x7f0c180097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:15:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:32.697+0000 7f0c35e00640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.67:6789/0 conn(0x7f0c180036a0 legacy=0x7f0c18005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:32.697+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.67:3300/0 conn(0x7f0c18007330 msgr2=0x7f0c180097f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:32.697+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.67:3300/0 conn(0x7f0c18007330 0x7f0c180097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:15:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.294+0000 7f0c2f400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.67:3300/0 conn(0x7f0c18007330 msgr2=0x7f0c180097f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.294+0000 7f0c2f400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.67:3300/0 conn(0x7f0c18007330 0x7f0c180097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.294+0000 7f0c2f400640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.67:6789/0 conn(0x7f0c180036a0 legacy=0x7f0c18005b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.294+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 reap_dead start 2024-03-09T09:15:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.295+0000 7f0c2f400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.295+0000 7f0c2f400640 1 -- 172.21.15.67:0/1157752481 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1800afc0 con 0x7f0c18005b90 2024-03-09T09:15:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.295+0000 7f0c2f400640 1 -- 172.21.15.67:0/1157752481 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c380c3700 con 0x7f0c18006230 2024-03-09T09:15:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.295+0000 7f0c2f400640 1 -- 172.21.15.67:0/1157752481 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c380c32b0 con 0x7f0c18009890 2024-03-09T09:15:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.295+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.67:6789/0 conn(0x7f0c18005b90 legacy=0x7f0c1800d400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.295+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.296+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 msgr2=0x7f0c18006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:15:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.296+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 msgr2=0x7f0c18006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.296+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.296+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.496+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.67:6789/0 conn(0x7f0c18005b90 legacy=0x7f0c1800d400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:34.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.497+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.498+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 msgr2=0x7f0c18006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:15:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.498+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 msgr2=0x7f0c18006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.498+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:34.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.498+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:15:34.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.897+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.67:6789/0 conn(0x7f0c18005b90 legacy=0x7f0c1800d400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:34.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.899+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:34.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.900+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 msgr2=0x7f0c18006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:15:34.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.900+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 msgr2=0x7f0c18006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:34.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.900+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:34.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:34.900+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:15:35.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:35.698+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.67:6789/0 conn(0x7f0c18005b90 legacy=0x7f0c1800d400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:35.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:35.702+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:35.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:35.702+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 msgr2=0x7f0c18006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:15:35.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:35.702+0000 7f0c35400640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 msgr2=0x7f0c18006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:35.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:35.702+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:35.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:35.702+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:15:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.423+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161287450 0 0) 0x7f0c380c3700 con 0x7f0c18006230 2024-03-09T09:15:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.423+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c1c003520 con 0x7f0c18006230 2024-03-09T09:15:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.434+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3118268740 0 0) 0x7f0c1c003520 con 0x7f0c18006230 2024-03-09T09:15:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.434+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 msgr2=0x7f0c18006d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.434+0000 7f0c34a00640 1 --2- 172.21.15.67:0/1157752481 >> v2:172.21.15.134:3300/0 conn(0x7f0c18009890 0x7f0c18006d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.434+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.67:6789/0 conn(0x7f0c18005b90 legacy=0x7f0c1800d400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.434+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c18010c00 con 0x7f0c18006230 2024-03-09T09:15:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.434+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f0c28001880 con 0x7f0c18006230 2024-03-09T09:15:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.434+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 >> v1:172.21.15.134:6789/0 conn(0x7f0c18006230 legacy=0x7f0c18006610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.434+0000 7f0c34a00640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 0x7f0c1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.434+0000 7f0c34a00640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c004800 0x7f0c1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:36.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.434+0000 7f0c34a00640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c1c0051c0 0x7f0c1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:36.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.435+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c380c32b0 con 0x7f0c1c0051c0 2024-03-09T09:15:36.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.435+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c00c110 con 0x7f0c1c004800 2024-03-09T09:15:36.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.435+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c00c310 con 0x7f0c1c003a20 2024-03-09T09:15:36.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.435+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c1c0051c0 msgr2=0x7f0c1c00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:36.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.435+0000 7f0c35e00640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 0x7f0c1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:36.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.435+0000 7f0c36800640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c1c0051c0 0x7f0c1c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:36.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.435+0000 7f0c35400640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c004800 0x7f0c1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:36.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.436+0000 7f0c35e00640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c004800 msgr2=0x7f0c1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:36.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.436+0000 7f0c35e00640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c004800 0x7f0c1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:36.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.436+0000 7f0c35e00640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c1c0051c0 msgr2=0x7f0c1c00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:36.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.436+0000 7f0c35e00640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c1c0051c0 0x7f0c1c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:36.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.436+0000 7f0c35e00640 1 -- 172.21.15.67:0/1157752481 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c003520 con 0x7f0c1c003a20 2024-03-09T09:15:36.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.436+0000 7f0c36800640 1 -- 172.21.15.67:0/1157752481 reap_dead start 2024-03-09T09:15:36.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.437+0000 7f0c35e00640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 0x7f0c1c003e20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800c210 tx=0x7f0c1c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f734bd1ef20957ab server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:36.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.442+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c28007d70 con 0x7f0c1c003a20 2024-03-09T09:15:36.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.442+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c28030070 con 0x7f0c1c003a20 2024-03-09T09:15:36.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.442+0000 7f0c34a00640 1 -- 172.21.15.67:0/1157752481 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2802cd20 con 0x7f0c1c003a20 2024-03-09T09:15:36.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.443+0000 7f0c37200640 1 -- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 msgr2=0x7f0c1c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:36.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.443+0000 7f0c37200640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 0x7f0c1c003e20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800c210 tx=0x7f0c1c005830 comp rx=0 tx=0).stop 2024-03-09T09:15:36.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.443+0000 7f0c37200640 1 -- 172.21.15.67:0/1157752481 shutdown_connections 2024-03-09T09:15:36.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.443+0000 7f0c37200640 1 --2- 172.21.15.67:0/1157752481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 0x7f0c1c003e20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:36.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.443+0000 7f0c37200640 1 -- 172.21.15.67:0/1157752481 >> 172.21.15.67:0/1157752481 conn(0x7f0c380c7300 msgr2=0x7f0c380c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:36.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.444+0000 7f0c37200640 1 -- 172.21.15.67:0/1157752481 shutdown_connections 2024-03-09T09:15:36.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.444+0000 7f0c37200640 1 -- 172.21.15.67:0/1157752481 wait complete. 2024-03-09T09:15:36.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.447+0000 7f0c37200640 1 Processor -- start 2024-03-09T09:15:36.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.447+0000 7f0c37200640 1 -- start start 2024-03-09T09:15:36.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.449+0000 7f0c37200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 0x7f0c380c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:36.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.449+0000 7f0c37200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c381ec0b0 0x7f0c381494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:36.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.450+0000 7f0c37200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c380ab950 0x7f0c38145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:36.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.450+0000 7f0c37200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c38070d50 con 0x7f0c381ec0b0 2024-03-09T09:15:36.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.450+0000 7f0c37200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c38070830 con 0x7f0c380ab950 2024-03-09T09:15:36.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.450+0000 7f0c37200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c38070ed0 con 0x7f0c1c003a20 2024-03-09T09:15:36.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.450+0000 7f0c35400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c381ec0b0 msgr2=0x7f0c381494b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:36.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.450+0000 7f0c35400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c381ec0b0 0x7f0c381494b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:36.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.450+0000 7f0c35e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 0x7f0c380c09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.450+0000 7f0c36800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c380ab950 0x7f0c38145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.450+0000 7f0c35e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 0x7f0c380c09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:57592/0 (socket says 172.21.15.67:57592) 2024-03-09T09:15:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.450+0000 7f0c35e00640 1 -- 172.21.15.67:0/4116497319 learned_addr learned my addr 172.21.15.67:0/4116497319 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.451+0000 7f0c35e00640 1 -- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c380ab950 msgr2=0x7f0c38145650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.451+0000 7f0c35e00640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c380ab950 0x7f0c38145650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.451+0000 7f0c35e00640 1 -- 172.21.15.67:0/4116497319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c381ec0b0 msgr2=0x7f0c381494b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:15:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.451+0000 7f0c35e00640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c381ec0b0 0x7f0c381494b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.451+0000 7f0c35e00640 1 -- 172.21.15.67:0/4116497319 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c381ef380 con 0x7f0c1c003a20 2024-03-09T09:15:36.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.451+0000 7f0c36800640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c380ab950 0x7f0c38145650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:15:36.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.452+0000 7f0c35e00640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 0x7f0c380c09a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800c330 tx=0x7f0c2802e070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:36.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.452+0000 7f0c2e000640 1 -- 172.21.15.67:0/4116497319 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c28007d70 con 0x7f0c1c003a20 2024-03-09T09:15:36.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.452+0000 7f0c2e000640 1 -- 172.21.15.67:0/4116497319 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c28005ce0 con 0x7f0c1c003a20 2024-03-09T09:15:36.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.452+0000 7f0c37200640 1 -- 172.21.15.67:0/4116497319 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c381ef600 con 0x7f0c1c003a20 2024-03-09T09:15:36.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.452+0000 7f0c2e000640 1 -- 172.21.15.67:0/4116497319 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c280049e0 con 0x7f0c1c003a20 2024-03-09T09:15:36.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.452+0000 7f0c37200640 1 -- 172.21.15.67:0/4116497319 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c381eb8b0 con 0x7f0c1c003a20 2024-03-09T09:15:36.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.454+0000 7f0c37200640 1 -- 172.21.15.67:0/4116497319 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3804f3c0 con 0x7f0c1c003a20 2024-03-09T09:15:36.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.457+0000 7f0c2e000640 1 -- 172.21.15.67:0/4116497319 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0c28004b80 con 0x7f0c1c003a20 2024-03-09T09:15:36.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.457+0000 7f0c2e000640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c1806ca90 0x7f0c1806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:36.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.458+0000 7f0c2e000640 1 -- 172.21.15.67:0/4116497319 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0c280abec0 con 0x7f0c1c003a20 2024-03-09T09:15:36.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.458+0000 7f0c35400640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c1806ca90 0x7f0c1806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:36.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.459+0000 7f0c35400640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c1806ca90 0x7f0c1806ef50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400f220 tx=0x7f0c24008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:36.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.472+0000 7f0c2e000640 1 -- 172.21.15.67:0/4116497319 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0c381ef380 con 0x7f0c1c003a20 2024-03-09T09:15:36.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.753+0000 7f0c37200640 1 -- 172.21.15.67:0/4116497319 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0c3814c1b0 con 0x7f0c1c003a20 2024-03-09T09:15:36.759 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":176,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:15:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.754+0000 7f0c2e000640 1 -- 172.21.15.67:0/4116497319 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f0c2807aac0 con 0x7f0c1c003a20 2024-03-09T09:15:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 -- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c1806ca90 msgr2=0x7f0c1806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c1806ca90 0x7f0c1806ef50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400f220 tx=0x7f0c24008040 comp rx=0 tx=0).stop 2024-03-09T09:15:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 -- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 msgr2=0x7f0c380c09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 0x7f0c380c09a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800c330 tx=0x7f0c2802e070 comp rx=0 tx=0).stop 2024-03-09T09:15:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 -- 172.21.15.67:0/4116497319 shutdown_connections 2024-03-09T09:15:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c1806ca90 0x7f0c1806ef50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c380ab950 0x7f0c38145650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c381ec0b0 0x7f0c381494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 --2- 172.21.15.67:0/4116497319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c003a20 0x7f0c380c09a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 -- 172.21.15.67:0/4116497319 >> 172.21.15.67:0/4116497319 conn(0x7f0c380c7300 msgr2=0x7f0c380728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 -- 172.21.15.67:0/4116497319 shutdown_connections 2024-03-09T09:15:36.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:36.757+0000 7f0c37200640 1 -- 172.21.15.67:0/4116497319 wait complete. 2024-03-09T09:15:36.930 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:15:36.930 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:15:36.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:15:37.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.173+0000 7ff9ad600640 1 Processor -- start 2024-03-09T09:15:37.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.173+0000 7ff9ad600640 1 -- start start 2024-03-09T09:15:37.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.175+0000 7ff9ad600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff9a80c5ff0 0x7ff9a80c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.175+0000 7ff9ad600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a81539e0 con 0x7ff9a8159ef0 2024-03-09T09:15:37.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.175+0000 7ff9ad600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a80c7a00 con 0x7ff9a8153600 2024-03-09T09:15:37.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.175+0000 7ff9ad600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a815a490 con 0x7ff9a80c5ff0 2024-03-09T09:15:37.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.175+0000 7ff99fe00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7ff9a80c5ff0 msgr2=0x7ff9a80c9570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:37.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.175+0000 7ff9a7400640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7ff9a8159ef0 legacy=0x7ff9a8160850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:37.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.175+0000 7ff99fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff9a80c5ff0 0x7ff9a80c9570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:37.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.176+0000 7ff9a6a00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff9a8153600 0x7ff9a806b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:42998/0 (socket says 172.21.15.67:42998) 2024-03-09T09:15:37.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.176+0000 7ff9a6a00640 1 -- 172.21.15.67:0/2137154620 learned_addr learned my addr 172.21.15.67:0/2137154620 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:15:37.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.176+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590814247 0 0) 0x7ff9a80c7a00 con 0x7ff9a8153600 2024-03-09T09:15:37.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.176+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff98c003520 con 0x7ff9a8153600 2024-03-09T09:15:37.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1254241210 0 0) 0x7ff98c003520 con 0x7ff9a8153600 2024-03-09T09:15:37.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 >> v2:172.21.15.67:3300/0 conn(0x7ff9a80c5ff0 msgr2=0x7ff9a80c9570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:37.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 --2- 172.21.15.67:0/2137154620 >> v2:172.21.15.67:3300/0 conn(0x7ff9a80c5ff0 0x7ff9a80c9570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 >> v1:172.21.15.67:6789/0 conn(0x7ff9a8159ef0 legacy=0x7ff9a8160850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:37.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a80c7ca0 con 0x7ff9a8153600 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7ff998001670 con 0x7ff9a8153600 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 >> v1:172.21.15.134:6789/0 conn(0x7ff9a8153600 legacy=0x7ff9a806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 --2- 172.21.15.67:0/2137154620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff98c0039b0 0x7ff98c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 --2- 172.21.15.67:0/2137154620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff98c004800 0x7ff98c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 --2- 172.21.15.67:0/2137154620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff98c0052e0 0x7ff98c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a815a490 con 0x7ff98c0052e0 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c00c230 con 0x7ff98c004800 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c00c430 con 0x7ff98c0039b0 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a7400640 1 -- 172.21.15.67:0/2137154620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff98c0052e0 msgr2=0x7ff98c00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.177+0000 7ff9a7400640 1 --2- 172.21.15.67:0/2137154620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff98c0052e0 0x7ff98c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.178+0000 7ff9a6a00640 1 --2- 172.21.15.67:0/2137154620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff98c0039b0 0x7ff98c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.178+0000 7ff99fe00640 1 --2- 172.21.15.67:0/2137154620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff98c004800 0x7ff98c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.178+0000 7ff9a6a00640 1 -- 172.21.15.67:0/2137154620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff98c004800 msgr2=0x7ff98c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:37.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.178+0000 7ff9a6a00640 1 --2- 172.21.15.67:0/2137154620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff98c004800 0x7ff98c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.178+0000 7ff9a6a00640 1 -- 172.21.15.67:0/2137154620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff98c0052e0 msgr2=0x7ff98c00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.178+0000 7ff9a6a00640 1 --2- 172.21.15.67:0/2137154620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff98c0052e0 0x7ff98c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.178+0000 7ff9a6a00640 1 -- 172.21.15.67:0/2137154620 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c003520 con 0x7ff98c0039b0 2024-03-09T09:15:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.179+0000 7ff9a7400640 1 -- 172.21.15.67:0/2137154620 reap_dead start 2024-03-09T09:15:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.180+0000 7ff9a6a00640 1 --2- 172.21.15.67:0/2137154620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff98c0039b0 0x7ff98c003db0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c160 tx=0x7ff98c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e5503a7a5731bdd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.180+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998007c60 con 0x7ff98c0039b0 2024-03-09T09:15:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.180+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff998030070 con 0x7ff98c0039b0 2024-03-09T09:15:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.180+0000 7ff9a6000640 1 -- 172.21.15.67:0/2137154620 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998005d50 con 0x7ff98c0039b0 2024-03-09T09:15:37.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.181+0000 7ff9ad600640 1 -- 172.21.15.67:0/2137154620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff98c0039b0 msgr2=0x7ff98c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.181+0000 7ff9ad600640 1 --2- 172.21.15.67:0/2137154620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff98c0039b0 0x7ff98c003db0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c160 tx=0x7ff98c005830 comp rx=0 tx=0).stop 2024-03-09T09:15:37.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.181+0000 7ff9ad600640 1 -- 172.21.15.67:0/2137154620 shutdown_connections 2024-03-09T09:15:37.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.181+0000 7ff9ad600640 1 --2- 172.21.15.67:0/2137154620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff98c0039b0 0x7ff98c003db0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.181+0000 7ff9ad600640 1 -- 172.21.15.67:0/2137154620 >> 172.21.15.67:0/2137154620 conn(0x7ff9a80ad160 msgr2=0x7ff9a80ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:37.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.181+0000 7ff9ad600640 1 -- 172.21.15.67:0/2137154620 shutdown_connections 2024-03-09T09:15:37.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.181+0000 7ff9ad600640 1 -- 172.21.15.67:0/2137154620 wait complete. 2024-03-09T09:15:37.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.475+0000 7ff9ad600640 1 Processor -- start 2024-03-09T09:15:37.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.477+0000 7ff9ad600640 1 -- start start 2024-03-09T09:15:37.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.479+0000 7ff9ad600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff98c0039b0 0x7ff9a8158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.479+0000 7ff9ad600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a81ec820 0x7ff9a8154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.479+0000 7ff9ad600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9a8154210 0x7ff9a8155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.479+0000 7ff9ad600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80c6e20 con 0x7ff9a81ec820 2024-03-09T09:15:37.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.479+0000 7ff9ad600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80c6990 con 0x7ff98c0039b0 2024-03-09T09:15:37.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.479+0000 7ff9ad600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a80c6fa0 con 0x7ff9a8154210 2024-03-09T09:15:37.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.479+0000 7ff99fe00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a81ec820 msgr2=0x7ff9a8154b10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:37.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.479+0000 7ff99fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a81ec820 0x7ff9a8154b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:37.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.484+0000 7ff9a7400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9a8154210 0x7ff9a8155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.485+0000 7ff9a7400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9a8154210 0x7ff9a8155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:57618/0 (socket says 172.21.15.67:57618) 2024-03-09T09:15:37.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.485+0000 7ff9a7400640 1 -- 172.21.15.67:0/2011046602 learned_addr learned my addr 172.21.15.67:0/2011046602 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:37.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.486+0000 7ff9a7400640 1 -- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff98c0039b0 msgr2=0x7ff9a8158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.487+0000 7ff9a7400640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff98c0039b0 0x7ff9a8158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.487+0000 7ff9a7400640 1 -- 172.21.15.67:0/2011046602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a81ec820 msgr2=0x7ff9a8154b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:15:37.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.487+0000 7ff9a7400640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9a81ec820 0x7ff9a8154b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.487+0000 7ff9a7400640 1 -- 172.21.15.67:0/2011046602 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a81ef140 con 0x7ff9a8154210 2024-03-09T09:15:37.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.489+0000 7ff9a7400640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9a8154210 0x7ff9a8155050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80bf170 tx=0x7ff9a0009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:37.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.494+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000ce10 con 0x7ff9a8154210 2024-03-09T09:15:37.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.495+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a0010070 con 0x7ff9a8154210 2024-03-09T09:15:37.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.495+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0015470 con 0x7ff9a8154210 2024-03-09T09:15:37.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.495+0000 7ff9ad600640 1 -- 172.21.15.67:0/2011046602 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a81ebd40 con 0x7ff9a8154210 2024-03-09T09:15:37.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.496+0000 7ff9ad600640 1 -- 172.21.15.67:0/2011046602 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a8158a40 con 0x7ff9a8154210 2024-03-09T09:15:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.498+0000 7ff9ad600640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814e9f0 0x7ff9a814edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.498+0000 7ff9ad600640 1 -- 172.21.15.67:0/2011046602 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a81ebb30 con 0x7ff9a814e9f0 2024-03-09T09:15:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.499+0000 7ff9a6a00640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814e9f0 0x7ff9a814edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.503+0000 7ff9a6a00640 1 -- 172.21.15.67:0/2011046602 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9a8075a00 con 0x7ff9a814e9f0 2024-03-09T09:15:37.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.505+0000 7ff9a6a00640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814e9f0 0x7ff9a814edd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff998031000 tx=0x7ff998001820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:37.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.508+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff9a0015610 con 0x7ff9a8154210 2024-03-09T09:15:37.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.509+0000 7ff99ea00640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff98406b500 0x7ff98406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.509+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff9a008a560 con 0x7ff9a8154210 2024-03-09T09:15:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.514+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99802dd70 con 0x7ff9a814e9f0 2024-03-09T09:15:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.515+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff998030070 con 0x7ff9a814e9f0 2024-03-09T09:15:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.515+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814e9f0 msgr2=0x7ff9a814edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.517+0000 7ff99ea00640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814e9f0 0x7ff9a814edd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff998031000 tx=0x7ff998001820 comp rx=0 tx=0).stop 2024-03-09T09:15:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.517+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814e9f0 msgr2=0x7ff9a814edd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.518+0000 7ff99ea00640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814e9f0 0x7ff9a814edd0 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff998031000 tx=0x7ff998001820 comp rx=0 tx=0).stop 2024-03-09T09:15:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.521+0000 7ff99fe00640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff98406b500 0x7ff98406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.527+0000 7ff99fe00640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff98406b500 0x7ff98406d9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff9a814d950 tx=0x7ff9900090e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.541+0000 7ff9ad600640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814f310 0x7ff9a81496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.541+0000 7ff9ad600640 1 -- 172.21.15.67:0/2011046602 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a8159250 con 0x7ff9a814f310 2024-03-09T09:15:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.542+0000 7ff9a6a00640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814f310 0x7ff9a81496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.544+0000 7ff9a6a00640 1 -- 172.21.15.67:0/2011046602 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff9a81597d0 con 0x7ff9a814f310 2024-03-09T09:15:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.545+0000 7ff9a6a00640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814f310 0x7ff9a81496a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff998001c80 tx=0x7ff99802f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.545+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980062e0 con 0x7ff9a814f310 2024-03-09T09:15:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.546+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7ff99804a070 con 0x7ff9a814f310 2024-03-09T09:15:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.546+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814f310 msgr2=0x7ff9a81496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.546+0000 7ff99ea00640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814f310 0x7ff9a81496a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff998001c80 tx=0x7ff99802f000 comp rx=0 tx=0).stop 2024-03-09T09:15:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.546+0000 7ff99ea00640 1 -- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814f310 msgr2=0x7ff9a81496a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.546+0000 7ff99ea00640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9a814f310 0x7ff9a81496a0 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff998001c80 tx=0x7ff99802f000 comp rx=0 tx=0).stop 2024-03-09T09:15:37.552 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:15:37.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:15:37.552 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:15:37.552 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:15:37.552 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 176, 2024-03-09T09:15:37.552 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:15:37.552 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:15:37.552 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:15:37.552 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:37.553 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:37.554 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:15:37.555 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:37.556 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:37.557 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:15:37.558 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.560 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.561 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:37.562 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.563 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.564 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:15:37.565 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.565 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:15:37.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.566 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:37.566 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.566 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:37.566 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.566 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.566 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:15:37.566 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:15:37.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.547+0000 7ff9ad600640 1 -- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff98406b500 msgr2=0x7ff98406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.547+0000 7ff9ad600640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff98406b500 0x7ff98406d9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff9a814d950 tx=0x7ff9900090e0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.547+0000 7ff9ad600640 1 -- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9a8154210 msgr2=0x7ff9a8155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.547+0000 7ff9ad600640 1 --2- 172.21.15.67:0/2011046602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9a8154210 0x7ff9a8155050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80bf170 tx=0x7ff9a0009e30 comp rx=0 tx=0).stop 2024-03-09T09:15:37.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.548+0000 7ff9a7400640 1 -- 172.21.15.67:0/2011046602 reap_dead start 2024-03-09T09:15:37.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.548+0000 7ff9a7400640 1 -- 172.21.15.67:0/2011046602 reap_dead start 2024-03-09T09:15:37.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.548+0000 7ff9ad600640 1 -- 172.21.15.67:0/2011046602 shutdown_connections 2024-03-09T09:15:37.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.548+0000 7ff9ad600640 1 -- 172.21.15.67:0/2011046602 >> 172.21.15.67:0/2011046602 conn(0x7ff9a80ad160 msgr2=0x7ff9a80ab240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:37.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.548+0000 7ff9ad600640 1 -- 172.21.15.67:0/2011046602 shutdown_connections 2024-03-09T09:15:37.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.549+0000 7ff9ad600640 1 -- 172.21.15.67:0/2011046602 wait complete. 2024-03-09T09:15:37.576 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:15:37.576 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:15:37.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:15:37.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.806+0000 7ffadb200640 1 Processor -- start 2024-03-09T09:15:37.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.807+0000 7ffadb200640 1 -- start start 2024-03-09T09:15:37.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.809+0000 7ffadb200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffadc0aab20 0x7ffadc0aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.809+0000 7ffadb200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffadc0c1c20 con 0x7ffadc1575b0 2024-03-09T09:15:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.809+0000 7ffadb200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffadc15a7f0 con 0x7ffadc0c3700 2024-03-09T09:15:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.809+0000 7ffadb200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffadc0aa3b0 con 0x7ffadc0aab20 2024-03-09T09:15:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.809+0000 7ffad9e00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7ffadc1575b0 legacy=0x7ffadc15a010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:15:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.809+0000 7ffada800640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7ffadc0aab20 msgr2=0x7ffadc0aaf70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.809+0000 7ffada800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ffadc0aab20 0x7ffadc0aaf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.810+0000 7ffad9400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ffadc0c3700 0x7ffadc0ab7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:43006/0 (socket says 172.21.15.67:43006) 2024-03-09T09:15:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.810+0000 7ffad9400640 1 -- 172.21.15.67:0/3310500665 learned_addr learned my addr 172.21.15.67:0/3310500665 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:15:37.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.813+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826233962 0 0) 0x7ffadc15a7f0 con 0x7ffadc0c3700 2024-03-09T09:15:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.813+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffac0003520 con 0x7ffadc0c3700 2024-03-09T09:15:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1683698113 0 0) 0x7ffac0003520 con 0x7ffadc0c3700 2024-03-09T09:15:37.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 >> v2:172.21.15.67:3300/0 conn(0x7ffadc0aab20 msgr2=0x7ffadc0aaf70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 --2- 172.21.15.67:0/3310500665 >> v2:172.21.15.67:3300/0 conn(0x7ffadc0aab20 0x7ffadc0aaf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 >> v1:172.21.15.67:6789/0 conn(0x7ffadc1575b0 legacy=0x7ffadc15a010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc15aad0 con 0x7ffadc0c3700 2024-03-09T09:15:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7ffacc001670 con 0x7ffadc0c3700 2024-03-09T09:15:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 >> v1:172.21.15.134:6789/0 conn(0x7ffadc0c3700 legacy=0x7ffadc0ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 --2- 172.21.15.67:0/3310500665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffac0003a20 0x7ffac0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 --2- 172.21.15.67:0/3310500665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac0004800 0x7ffac0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 --2- 172.21.15.67:0/3310500665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffac00053c0 0x7ffac000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0aa3b0 con 0x7ffac0004800 2024-03-09T09:15:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac000c280 con 0x7ffac0003a20 2024-03-09T09:15:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac000c480 con 0x7ffac00053c0 2024-03-09T09:15:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad9400640 1 -- 172.21.15.67:0/3310500665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac0004800 msgr2=0x7ffac0004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.814+0000 7ffad9400640 1 --2- 172.21.15.67:0/3310500665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac0004800 0x7ffac0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.815+0000 7ffad9e00640 1 --2- 172.21.15.67:0/3310500665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffac0003a20 0x7ffac0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.815+0000 7ffada800640 1 --2- 172.21.15.67:0/3310500665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffac00053c0 0x7ffac000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.815+0000 7ffad9e00640 1 -- 172.21.15.67:0/3310500665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffac00053c0 msgr2=0x7ffac000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:15:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.815+0000 7ffad9e00640 1 --2- 172.21.15.67:0/3310500665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffac00053c0 0x7ffac000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.815+0000 7ffad9e00640 1 -- 172.21.15.67:0/3310500665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac0004800 msgr2=0x7ffac0004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.815+0000 7ffad9e00640 1 --2- 172.21.15.67:0/3310500665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffac0004800 0x7ffac0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.815+0000 7ffad9e00640 1 -- 172.21.15.67:0/3310500665 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc15aad0 con 0x7ffac0003a20 2024-03-09T09:15:37.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.815+0000 7ffada800640 1 -- 172.21.15.67:0/3310500665 reap_dead start 2024-03-09T09:15:37.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.821+0000 7ffad9e00640 1 --2- 172.21.15.67:0/3310500665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffac0003a20 0x7ffac0003e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009900 tx=0x7ffac0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11e595a527f104b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.824+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4009500 con 0x7ffac0003a20 2024-03-09T09:15:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.825+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac402f070 con 0x7ffac0003a20 2024-03-09T09:15:37.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.825+0000 7ffad8a00640 1 -- 172.21.15.67:0/3310500665 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4003e30 con 0x7ffac0003a20 2024-03-09T09:15:37.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.826+0000 7ffadb200640 1 -- 172.21.15.67:0/3310500665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffac0003a20 msgr2=0x7ffac0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.826+0000 7ffadb200640 1 --2- 172.21.15.67:0/3310500665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffac0003a20 0x7ffac0003e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009900 tx=0x7ffac0005830 comp rx=0 tx=0).stop 2024-03-09T09:15:37.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.826+0000 7ffadb200640 1 -- 172.21.15.67:0/3310500665 shutdown_connections 2024-03-09T09:15:37.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.826+0000 7ffadb200640 1 --2- 172.21.15.67:0/3310500665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffac0003a20 0x7ffac0003e20 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009900 tx=0x7ffac0005830 comp rx=0 tx=0).stop 2024-03-09T09:15:37.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.826+0000 7ffadb200640 1 -- 172.21.15.67:0/3310500665 >> 172.21.15.67:0/3310500665 conn(0x7ffadc0c7220 msgr2=0x7ffadc0c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:37.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.828+0000 7ffadb200640 1 -- 172.21.15.67:0/3310500665 shutdown_connections 2024-03-09T09:15:37.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.829+0000 7ffadb200640 1 -- 172.21.15.67:0/3310500665 wait complete. 2024-03-09T09:15:37.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.831+0000 7ffadb200640 1 Processor -- start 2024-03-09T09:15:37.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.832+0000 7ffadb200640 1 -- start start 2024-03-09T09:15:37.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.833+0000 7ffadb200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffadc1f21b0 0x7ffadc152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.833+0000 7ffadb200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffadc1e39f0 0x7ffadc1f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.833+0000 7ffadb200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc070930 0x7ffadc06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.833+0000 7ffadb200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc157990 con 0x7ffadc1f21b0 2024-03-09T09:15:37.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.833+0000 7ffadb200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0aa600 con 0x7ffadc1e39f0 2024-03-09T09:15:37.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.833+0000 7ffadb200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc15a010 con 0x7ffadc070930 2024-03-09T09:15:37.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.833+0000 7ffad9e00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffadc1f21b0 msgr2=0x7ffadc152660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:15:37.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.833+0000 7ffad9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffadc1f21b0 0x7ffadc152660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:37.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.834+0000 7ffad9400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffadc1e39f0 0x7ffadc1f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.834+0000 7ffada800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc070930 0x7ffadc06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.834+0000 7ffada800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc070930 0x7ffadc06d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:57634/0 (socket says 172.21.15.67:57634) 2024-03-09T09:15:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.834+0000 7ffada800640 1 -- 172.21.15.67:0/311164750 learned_addr learned my addr 172.21.15.67:0/311164750 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.834+0000 7ffad9400640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffadc1e39f0 0x7ffadc1f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52976/0 (socket says 172.21.15.67:52976) 2024-03-09T09:15:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.834+0000 7ffada800640 1 -- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffadc1e39f0 msgr2=0x7ffadc1f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.834+0000 7ffada800640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffadc1e39f0 0x7ffadc1f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.834+0000 7ffada800640 1 -- 172.21.15.67:0/311164750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffadc1f21b0 msgr2=0x7ffadc152660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:15:37.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.834+0000 7ffada800640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffadc1f21b0 0x7ffadc152660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.834+0000 7ffada800640 1 -- 172.21.15.67:0/311164750 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc15dde0 con 0x7ffadc070930 2024-03-09T09:15:37.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.835+0000 7ffada800640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc070930 0x7ffadc06d8e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0708c0 tx=0x7ffad400ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:37.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.835+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad400cde0 con 0x7ffadc070930 2024-03-09T09:15:37.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.836+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad40050d0 con 0x7ffadc070930 2024-03-09T09:15:37.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.836+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad4005480 con 0x7ffadc070930 2024-03-09T09:15:37.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.836+0000 7ffadb200640 1 -- 172.21.15.67:0/311164750 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffadc15e0c0 con 0x7ffadc070930 2024-03-09T09:15:37.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.836+0000 7ffadb200640 1 -- 172.21.15.67:0/311164750 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffadc15d260 con 0x7ffadc070930 2024-03-09T09:15:37.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.837+0000 7ffadb200640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc06de20 0x7ffadc1f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.837+0000 7ffadb200640 1 -- 172.21.15.67:0/311164750 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc15cde0 con 0x7ffadc06de20 2024-03-09T09:15:37.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.838+0000 7ffad9e00640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc06de20 0x7ffadc1f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.839+0000 7ffad9e00640 1 -- 172.21.15.67:0/311164750 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffadc04f3c0 con 0x7ffadc06de20 2024-03-09T09:15:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.839+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ffad4005620 con 0x7ffadc070930 2024-03-09T09:15:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.839+0000 7ffad9e00640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc06de20 0x7ffadc1f0d20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffac4003980 tx=0x7ffac4005a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.841+0000 7ffad2000640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffab806b480 0x7ffab806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.841+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ffad408b280 con 0x7ffadc070930 2024-03-09T09:15:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.841+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4009590 con 0x7ffadc06de20 2024-03-09T09:15:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.841+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ffac402f040 con 0x7ffadc06de20 2024-03-09T09:15:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.841+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc06de20 msgr2=0x7ffadc1f0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.841+0000 7ffad2000640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc06de20 0x7ffadc1f0d20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffac4003980 tx=0x7ffac4005a10 comp rx=0 tx=0).stop 2024-03-09T09:15:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.841+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc06de20 msgr2=0x7ffadc1f0d20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.841+0000 7ffad2000640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc06de20 0x7ffadc1f0d20 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffac4003980 tx=0x7ffac4005a10 comp rx=0 tx=0).stop 2024-03-09T09:15:37.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.842+0000 7ffad9400640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffab806b480 0x7ffab806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.843+0000 7ffad9400640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffab806b480 0x7ffab806d940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffadc152ba0 tx=0x7ffacc001d50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:37.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.849+0000 7ffadb200640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc1f1260 0x7ffadc1f1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:37.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.849+0000 7ffadb200640 1 -- 172.21.15.67:0/311164750 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0c1c20 con 0x7ffadc1f1260 2024-03-09T09:15:37.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.850+0000 7ffad9e00640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc1f1260 0x7ffadc1f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:37.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.851+0000 7ffad9e00640 1 -- 172.21.15.67:0/311164750 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffadc0c1900 con 0x7ffadc1f1260 2024-03-09T09:15:37.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.851+0000 7ffad9e00640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc1f1260 0x7ffadc1f1640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffac4000c00 tx=0x7ffac40090e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:37.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.852+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4034880 con 0x7ffadc1f1260 2024-03-09T09:15:37.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.852+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5176 (secure 0 0 0) 0x7ffac4009590 con 0x7ffadc1f1260 2024-03-09T09:15:37.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.852+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc1f1260 msgr2=0x7ffadc1f1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.852+0000 7ffad2000640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc1f1260 0x7ffadc1f1640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffac4000c00 tx=0x7ffac40090e0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.852+0000 7ffad2000640 1 -- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc1f1260 msgr2=0x7ffadc1f1640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:15:37.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.852+0000 7ffad2000640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc1f1260 0x7ffadc1f1640 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffac4000c00 tx=0x7ffac40090e0 comp rx=0 tx=0).stop 2024-03-09T09:15:37.858 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:15:37.858 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:15:37.858 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 176, 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:37.859 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:37.860 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:15:37.861 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:15:37.862 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:15:37.863 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:15:37.865 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:37.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:15:37.867 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.868 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.869 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.870 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:15:37.871 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:15:37.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.853+0000 7ffadb200640 1 -- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffab806b480 msgr2=0x7ffab806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.853+0000 7ffadb200640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffab806b480 0x7ffab806d940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffadc152ba0 tx=0x7ffacc001d50 comp rx=0 tx=0).stop 2024-03-09T09:15:37.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.853+0000 7ffadb200640 1 -- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc070930 msgr2=0x7ffadc06d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:37.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.853+0000 7ffadb200640 1 --2- 172.21.15.67:0/311164750 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffadc070930 0x7ffadc06d8e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0708c0 tx=0x7ffad400ef70 comp rx=0 tx=0).stop 2024-03-09T09:15:37.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.853+0000 7ffada800640 1 -- 172.21.15.67:0/311164750 reap_dead start 2024-03-09T09:15:37.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.853+0000 7ffada800640 1 -- 172.21.15.67:0/311164750 reap_dead start 2024-03-09T09:15:37.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.853+0000 7ffadb200640 1 -- 172.21.15.67:0/311164750 shutdown_connections 2024-03-09T09:15:37.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.853+0000 7ffadb200640 1 -- 172.21.15.67:0/311164750 >> 172.21.15.67:0/311164750 conn(0x7ffadc0c7220 msgr2=0x7ffadc158810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:37.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.853+0000 7ffadb200640 1 -- 172.21.15.67:0/311164750 shutdown_connections 2024-03-09T09:15:37.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:37.854+0000 7ffadb200640 1 -- 172.21.15.67:0/311164750 wait complete. 2024-03-09T09:15:37.880 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:15:37.880 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:15:40.323 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 21% (37/176) Updating files: 22% (39/176) Updating files: 23% (41/176) Updating files: 24% (43/176) Updating files: 25% (44/176) Updating files: 26% (46/176) Updating files: 27% (48/176) Updating files: 28% (50/176) Updating files: 29% (52/176) Updating files: 30% (53/176) Updating files: 31% (55/176) Updating files: 32% (57/176) Updating files: 33% (59/176) Updating files: 34% (60/176) Updating files: 35% (62/176) Updating files: 36% (64/176) Updating files: 37% (66/176) Updating files: 38% (67/176) Updating files: 39% (69/176) Updating files: 40% (71/176) Updating files: 41% (73/176) Updating files: 42% (74/176) Updating files: 43% (76/176) Updating files: 44% (78/176) Updating files: 45% (80/176) Updating files: 46% (81/176) Updating files: 47% (83/176) Updating files: 48% (85/176) Updating files: 49% (87/176) Updating files: 50% (88/176) Updating files: 51% (90/176) Updating files: 52% (92/176) Updating files: 53% (94/176) Updating files: 54% (96/176) Updating files: 55% (97/176) Updating files: 56% (99/176) Updating files: 57% (101/176) Updating files: 58% (103/176) Updating files: 59% (104/176) Updating files: 60% (106/176) Updating files: 61% (108/176) Updating files: 62% (110/176) Updating files: 63% (111/176) Updating files: 63% (112/176) Updating files: 64% (113/176) Updating files: 65% (115/176) Updating files: 66% (117/176) Updating files: 67% (118/176) Updating files: 68% (120/176) Updating files: 69% (122/176) Updating files: 70% (124/176) Updating files: 71% (125/176) Updating files: 72% (127/176) Updating files: 73% (129/176) Updating files: 74% (131/176) Updating files: 75% (132/176) Updating files: 76% (134/176) Updating files: 77% (136/176) Updating files: 78% (138/176) Updating files: 79% (140/176) Updating files: 80% (141/176) Updating files: 81% (143/176) Updating files: 82% (145/176) Updating files: 83% (147/176) Updating files: 84% (148/176) Updating files: 85% (150/176) Updating files: 86% (152/176) Updating files: 87% (154/176) Updating files: 88% (155/176) Updating files: 89% (157/176) Updating files: 90% (159/176) Updating files: 91% (161/176) Updating files: 92% (162/176) Updating files: 93% (164/176) Updating files: 94% (166/176) Updating files: 95% (168/176) Updating files: 96% (169/176) Updating files: 97% (171/176) Updating files: 98% (173/176) Updating files: 99% (175/176) Updating files: 100% (176/176) Updating files: 100% (176/176), done. 2024-03-09T09:15:40.996 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 42% (130/309) Updating files: 43% (133/309) Updating files: 44% (136/309) Updating files: 45% (140/309) Updating files: 46% (143/309) Updating files: 47% (146/309) Updating files: 48% (149/309) Updating files: 49% (152/309) Updating files: 50% (155/309) Updating files: 51% (158/309) Updating files: 52% (161/309) Updating files: 53% (164/309) Updating files: 54% (167/309) Updating files: 55% (170/309) Updating files: 56% (174/309) Updating files: 57% (177/309) Updating files: 58% (180/309) Updating files: 59% (183/309) Updating files: 60% (186/309) Updating files: 61% (189/309) Updating files: 62% (192/309) Updating files: 63% (195/309) Updating files: 64% (198/309) Updating files: 65% (201/309) Updating files: 66% (204/309) Updating files: 67% (208/309) Updating files: 68% (211/309) Updating files: 69% (214/309) Updating files: 70% (217/309) Updating files: 71% (220/309) Updating files: 72% (223/309) Updating files: 73% (226/309) Updating files: 74% (229/309) Updating files: 75% (232/309) Updating files: 76% (235/309) Updating files: 77% (238/309) Updating files: 78% (242/309) Updating files: 79% (245/309) Updating files: 80% (248/309) Updating files: 81% (251/309) Updating files: 82% (254/309) Updating files: 83% (257/309) Updating files: 83% (258/309) Updating files: 84% (260/309) Updating files: 85% (263/309) Updating files: 86% (266/309) Updating files: 86% (267/309) Updating files: 87% (269/309) Updating files: 88% (272/309) Updating files: 89% (276/309) Updating files: 90% (279/309) Updating files: 91% (282/309) Updating files: 91% (283/309) Updating files: 92% (285/309) Updating files: 93% (288/309) Updating files: 94% (291/309) Updating files: 95% (294/309) Updating files: 96% (297/309) Updating files: 97% (300/309) Updating files: 98% (303/309) Updating files: 99% (306/309) Updating files: 100% (309/309) Updating files: 100% (309/309), done. 2024-03-09T09:15:41.073 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 8de9501df27 0.81 2024-03-09T09:15:41.099 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.81 2024-03-09T09:15:41.354 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at b78644e7dee 0.80 2024-03-09T09:15:41.370 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.80 2024-03-09T09:15:41.490 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:15:41.490 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 81 -eq 48 ']' 2024-03-09T09:15:41.491 INFO:tasks.workunit.client.0.smithi067.stdout:v0.81 2024-03-09T09:15:41.491 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.81 2024-03-09T09:15:41.491 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.81 2024-03-09T09:15:41.491 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.81 2024-03-09T09:15:41.719 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:15:41.719 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 82 -eq 48 ']' 2024-03-09T09:15:41.720 INFO:tasks.workunit.client.1.smithi134.stdout:v0.82 2024-03-09T09:15:41.720 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.82 2024-03-09T09:15:41.720 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.82 2024-03-09T09:15:41.720 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.82 2024-03-09T09:15:44.698 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 32% (66/204) Updating files: 33% (68/204) Updating files: 34% (70/204) Updating files: 35% (72/204) Updating files: 36% (74/204) Updating files: 37% (76/204) Updating files: 38% (78/204) Updating files: 39% (80/204) Updating files: 40% (82/204) Updating files: 41% (84/204) Updating files: 42% (86/204) Updating files: 43% (88/204) Updating files: 44% (90/204) Updating files: 45% (92/204) Updating files: 46% (94/204) Updating files: 47% (96/204) Updating files: 48% (98/204) Updating files: 49% (100/204) Updating files: 50% (102/204) Updating files: 51% (105/204) Updating files: 52% (107/204) Updating files: 53% (109/204) Updating files: 54% (111/204) Updating files: 55% (113/204) Updating files: 56% (115/204) Updating files: 57% (117/204) Updating files: 58% (119/204) Updating files: 59% (121/204) Updating files: 60% (123/204) Updating files: 61% (125/204) Updating files: 62% (127/204) Updating files: 63% (129/204) Updating files: 64% (131/204) Updating files: 65% (133/204) Updating files: 66% (135/204) Updating files: 67% (137/204) Updating files: 68% (139/204) Updating files: 69% (141/204) Updating files: 70% (143/204) Updating files: 71% (145/204) Updating files: 72% (147/204) Updating files: 73% (149/204) Updating files: 74% (151/204) Updating files: 75% (153/204) Updating files: 76% (156/204) Updating files: 77% (158/204) Updating files: 78% (160/204) Updating files: 79% (162/204) Updating files: 80% (164/204) Updating files: 81% (166/204) Updating files: 82% (168/204) Updating files: 83% (170/204) Updating files: 84% (172/204) Updating files: 85% (174/204) Updating files: 86% (176/204) Updating files: 87% (178/204) Updating files: 88% (180/204) Updating files: 89% (182/204) Updating files: 90% (184/204) Updating files: 91% (186/204) Updating files: 92% (188/204) Updating files: 93% (190/204) Updating files: 94% (192/204) Updating files: 95% (194/204) Updating files: 96% (196/204) Updating files: 97% (198/204) Updating files: 98% (200/204) Updating files: 99% (202/204) Updating files: 100% (204/204) Updating files: 100% (204/204), done. 2024-03-09T09:15:44.886 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 14085f42ddd 0.82 2024-03-09T09:15:44.904 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.82 2024-03-09T09:15:45.187 INFO:tasks.workunit.client.1.smithi134.stdout:v0.83 2024-03-09T09:15:45.187 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:15:45.187 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 83 -eq 48 ']' 2024-03-09T09:15:45.187 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.83 2024-03-09T09:15:45.187 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.83 2024-03-09T09:15:45.187 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.83 2024-03-09T09:15:48.457 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 8% (26/309) Updating files: 9% (28/309) Updating files: 10% (31/309) Updating files: 11% (34/309) Updating files: 12% (38/309) Updating files: 13% (41/309) Updating files: 14% (44/309) Updating files: 15% (47/309) Updating files: 16% (50/309) Updating files: 17% (53/309) Updating files: 18% (56/309) Updating files: 19% (59/309) Updating files: 20% (62/309) Updating files: 21% (65/309) Updating files: 22% (68/309) Updating files: 23% (72/309) Updating files: 24% (75/309) Updating files: 25% (78/309) Updating files: 26% (81/309) Updating files: 27% (84/309) Updating files: 28% (87/309) Updating files: 28% (88/309) Updating files: 29% (90/309) Updating files: 30% (93/309) Updating files: 31% (96/309) Updating files: 32% (99/309) Updating files: 33% (102/309) Updating files: 34% (106/309) Updating files: 35% (109/309) Updating files: 36% (112/309) Updating files: 37% (115/309) Updating files: 38% (118/309) Updating files: 39% (121/309) Updating files: 40% (124/309) Updating files: 41% (127/309) Updating files: 41% (129/309) Updating files: 42% (130/309) Updating files: 43% (133/309) Updating files: 44% (136/309) Updating files: 45% (140/309) Updating files: 46% (143/309) Updating files: 47% (146/309) Updating files: 48% (149/309) Updating files: 49% (152/309) Updating files: 50% (155/309) Updating files: 51% (158/309) Updating files: 52% (161/309) Updating files: 53% (164/309) Updating files: 54% (167/309) Updating files: 55% (170/309) Updating files: 56% (174/309) Updating files: 57% (177/309) Updating files: 58% (180/309) Updating files: 59% (183/309) Updating files: 60% (186/309) Updating files: 60% (187/309) Updating files: 61% (189/309) Updating files: 62% (192/309) Updating files: 63% (195/309) Updating files: 64% (198/309) Updating files: 65% (201/309) Updating files: 66% (204/309) Updating files: 67% (208/309) Updating files: 68% (211/309) Updating files: 69% (214/309) Updating files: 70% (217/309) Updating files: 71% (220/309) Updating files: 72% (223/309) Updating files: 73% (226/309) Updating files: 74% (229/309) Updating files: 75% (232/309) Updating files: 76% (235/309) Updating files: 77% (238/309) Updating files: 77% (240/309) Updating files: 78% (242/309) Updating files: 79% (245/309) Updating files: 80% (248/309) Updating files: 81% (251/309) Updating files: 82% (254/309) Updating files: 83% (257/309) Updating files: 84% (260/309) Updating files: 85% (263/309) Updating files: 86% (266/309) Updating files: 86% (267/309) Updating files: 87% (269/309) Updating files: 88% (272/309) Updating files: 89% (276/309) Updating files: 90% (279/309) Updating files: 91% (282/309) Updating files: 92% (285/309) Updating files: 93% (288/309) Updating files: 94% (291/309) Updating files: 95% (294/309) Updating files: 96% (297/309) Updating files: 97% (300/309) Updating files: 98% (303/309) Updating files: 99% (306/309) Updating files: 100% (309/309) Updating files: 100% (309/309), done. 2024-03-09T09:15:48.521 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 8de9501df27 0.81 2024-03-09T09:15:48.534 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.81 2024-03-09T09:15:48.676 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:15:48.676 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 82 -eq 48 ']' 2024-03-09T09:15:48.676 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.82 2024-03-09T09:15:48.676 INFO:tasks.workunit.client.0.smithi067.stdout:v0.82 2024-03-09T09:15:48.676 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.82 2024-03-09T09:15:48.676 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.82 2024-03-09T09:15:49.453 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 44% (97/218) Updating files: 45% (99/218) Updating files: 46% (101/218) Updating files: 47% (103/218) Updating files: 48% (105/218) Updating files: 49% (107/218) Updating files: 50% (109/218) Updating files: 51% (112/218) Updating files: 52% (114/218) Updating files: 53% (116/218) Updating files: 54% (118/218) Updating files: 55% (120/218) Updating files: 56% (123/218) Updating files: 57% (125/218) Updating files: 58% (127/218) Updating files: 59% (129/218) Updating files: 60% (131/218) Updating files: 61% (133/218) Updating files: 62% (136/218) Updating files: 63% (138/218) Updating files: 64% (140/218) Updating files: 65% (142/218) Updating files: 66% (144/218) Updating files: 67% (147/218) Updating files: 68% (149/218) Updating files: 69% (151/218) Updating files: 70% (153/218) Updating files: 71% (155/218) Updating files: 72% (157/218) Updating files: 72% (159/218) Updating files: 73% (160/218) Updating files: 74% (162/218) Updating files: 75% (164/218) Updating files: 76% (166/218) Updating files: 77% (168/218) Updating files: 78% (171/218) Updating files: 79% (173/218) Updating files: 80% (175/218) Updating files: 81% (177/218) Updating files: 82% (179/218) Updating files: 83% (181/218) Updating files: 84% (184/218) Updating files: 85% (186/218) Updating files: 86% (188/218) Updating files: 87% (190/218) Updating files: 88% (192/218) Updating files: 89% (195/218) Updating files: 89% (196/218) Updating files: 90% (197/218) Updating files: 91% (199/218) Updating files: 92% (201/218) Updating files: 93% (203/218) Updating files: 94% (205/218) Updating files: 95% (208/218) Updating files: 96% (210/218) Updating files: 97% (212/218) Updating files: 98% (214/218) Updating files: 99% (216/218) Updating files: 100% (218/218) Updating files: 100% (218/218), done. 2024-03-09T09:15:49.592 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 78ff1f0a5df 0.83 2024-03-09T09:15:49.606 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.83 2024-03-09T09:15:49.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:15:49.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 84 -eq 48 ']' 2024-03-09T09:15:49.746 INFO:tasks.workunit.client.1.smithi134.stdout:v0.84 2024-03-09T09:15:49.747 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.84 2024-03-09T09:15:49.747 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.84 2024-03-09T09:15:49.747 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.84 2024-03-09T09:15:51.692 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 17% (36/204) Updating files: 18% (37/204) Updating files: 19% (39/204) Updating files: 20% (41/204) Updating files: 21% (43/204) Updating files: 22% (45/204) Updating files: 23% (47/204) Updating files: 24% (49/204) Updating files: 25% (51/204) Updating files: 26% (54/204) Updating files: 27% (56/204) Updating files: 28% (58/204) Updating files: 29% (60/204) Updating files: 30% (62/204) Updating files: 31% (64/204) Updating files: 32% (66/204) Updating files: 33% (68/204) Updating files: 34% (70/204) Updating files: 35% (72/204) Updating files: 36% (74/204) Updating files: 37% (76/204) Updating files: 38% (78/204) Updating files: 39% (80/204) Updating files: 40% (82/204) Updating files: 41% (84/204) Updating files: 42% (86/204) Updating files: 43% (88/204) Updating files: 44% (90/204) Updating files: 45% (92/204) Updating files: 46% (94/204) Updating files: 47% (96/204) Updating files: 48% (98/204) Updating files: 49% (100/204) Updating files: 50% (102/204) Updating files: 51% (105/204) Updating files: 52% (107/204) Updating files: 53% (109/204) Updating files: 54% (111/204) Updating files: 55% (113/204) Updating files: 56% (115/204) Updating files: 57% (117/204) Updating files: 58% (119/204) Updating files: 59% (121/204) Updating files: 60% (123/204) Updating files: 61% (125/204) Updating files: 62% (127/204) Updating files: 63% (129/204) Updating files: 63% (130/204) Updating files: 64% (131/204) Updating files: 65% (133/204) Updating files: 66% (135/204) Updating files: 67% (137/204) Updating files: 68% (139/204) Updating files: 69% (141/204) Updating files: 70% (143/204) Updating files: 71% (145/204) Updating files: 72% (147/204) Updating files: 73% (149/204) Updating files: 74% (151/204) Updating files: 75% (153/204) Updating files: 76% (156/204) Updating files: 77% (158/204) Updating files: 78% (160/204) Updating files: 79% (162/204) Updating files: 80% (164/204) Updating files: 81% (166/204) Updating files: 82% (168/204) Updating files: 83% (170/204) Updating files: 84% (172/204) Updating files: 85% (174/204) Updating files: 86% (176/204) Updating files: 87% (178/204) Updating files: 88% (180/204) Updating files: 89% (182/204) Updating files: 90% (184/204) Updating files: 91% (186/204) Updating files: 92% (188/204) Updating files: 93% (190/204) Updating files: 94% (192/204) Updating files: 95% (194/204) Updating files: 96% (196/204) Updating files: 97% (198/204) Updating files: 98% (200/204) Updating files: 99% (202/204) Updating files: 100% (204/204) Updating files: 100% (204/204), done. 2024-03-09T09:15:51.773 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 14085f42ddd 0.82 2024-03-09T09:15:51.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.82 2024-03-09T09:15:52.055 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:15:52.055 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 83 -eq 48 ']' 2024-03-09T09:15:52.055 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.83 2024-03-09T09:15:52.056 INFO:tasks.workunit.client.0.smithi067.stdout:v0.83 2024-03-09T09:15:52.056 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.83 2024-03-09T09:15:52.056 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.83 2024-03-09T09:15:57.881 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:15:57.881 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-03-09T09:15:57.881 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-09T09:15:57.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-09T09:15:57.887 INFO:tasks.ceph.mon.a:Started 2024-03-09T09:15:57.887 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:15:57.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:15:58.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.143+0000 7f3bfca00640 1 Processor -- start 2024-03-09T09:15:58.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.143+0000 7f3bfca00640 1 -- start start 2024-03-09T09:15:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.146+0000 7f3bfca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf8153300 0x7f3bf80b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.147+0000 7f3bfca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3bf80cadb0 0x7f3bf80b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.147+0000 7f3bfca00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf8157000 con 0x7f3bf80c5e40 2024-03-09T09:15:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.147+0000 7f3bfca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf80ae430 con 0x7f3bf80cadb0 2024-03-09T09:15:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.147+0000 7f3bfca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80ae6f0 con 0x7f3bf8153300 2024-03-09T09:15:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.147+0000 7f3bf5e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f3bf80c5e40 0x7f3bf8156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:36678/0 (socket says 172.21.15.67:36678) 2024-03-09T09:15:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.147+0000 7f3bf5e00640 1 -- 172.21.15.67:0/660121028 learned_addr learned my addr 172.21.15.67:0/660121028 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:15:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.147+0000 7f3bf6800640 1 --2- 172.21.15.67:0/660121028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf8153300 0x7f3bf80b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.147+0000 7f3bf7200640 1 --2- 172.21.15.67:0/660121028 >> v2:172.21.15.67:3300/0 conn(0x7f3bf80cadb0 0x7f3bf80b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.148+0000 7f3bf7200640 1 -- 172.21.15.67:0/660121028 >> v2:172.21.15.67:3300/0 conn(0x7f3bf80cadb0 msgr2=0x7f3bf80b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:15:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.148+0000 7f3bf7200640 1 -- 172.21.15.67:0/660121028 >> v2:172.21.15.67:3300/0 conn(0x7f3bf80cadb0 msgr2=0x7f3bf80b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:15:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.148+0000 7f3bf7200640 1 --2- 172.21.15.67:0/660121028 >> v2:172.21.15.67:3300/0 conn(0x7f3bf80cadb0 0x7f3bf80b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:58.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.148+0000 7f3bf7200640 1 --2- 172.21.15.67:0/660121028 >> v2:172.21.15.67:3300/0 conn(0x7f3bf80cadb0 0x7f3bf80b0540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:58.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.148+0000 7f3bf6800640 1 -- 172.21.15.67:0/660121028 >> v2:172.21.15.67:3300/0 conn(0x7f3bf80cadb0 msgr2=0x7f3bf80b0540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:15:58.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.148+0000 7f3bf6800640 1 --2- 172.21.15.67:0/660121028 >> v2:172.21.15.67:3300/0 conn(0x7f3bf80cadb0 0x7f3bf80b0540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:58.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.148+0000 7f3bf6800640 1 -- 172.21.15.67:0/660121028 >> v1:172.21.15.134:6789/0 conn(0x7f3bf80c5e40 legacy=0x7f3bf8156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:58.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.148+0000 7f3bf5400640 1 -- 172.21.15.67:0/660121028 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801314466 0 0) 0x7f3bf8157000 con 0x7f3bf80c5e40 2024-03-09T09:15:58.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.148+0000 7f3bf6800640 1 -- 172.21.15.67:0/660121028 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf8155b10 con 0x7f3bf8153300 2024-03-09T09:15:58.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.150+0000 7f3bf6800640 1 --2- 172.21.15.67:0/660121028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf8153300 0x7f3bf80b0000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3be80097a0 tx=0x7f3be802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b01fbecd2bbc77 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:58.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.150+0000 7f3bf5400640 1 -- 172.21.15.67:0/660121028 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8035020 con 0x7f3bf8153300 2024-03-09T09:15:58.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.150+0000 7f3bf5400640 1 -- 172.21.15.67:0/660121028 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be8031830 con 0x7f3bf8153300 2024-03-09T09:15:58.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.151+0000 7f3bfca00640 1 -- 172.21.15.67:0/660121028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf8153300 msgr2=0x7f3bf80b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:58.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.152+0000 7f3bf5400640 1 -- 172.21.15.67:0/660121028 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be8035020 con 0x7f3bf8153300 2024-03-09T09:15:58.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.152+0000 7f3bfca00640 1 --2- 172.21.15.67:0/660121028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf8153300 0x7f3bf80b0000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3be80097a0 tx=0x7f3be802b8d0 comp rx=0 tx=0).stop 2024-03-09T09:15:58.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.152+0000 7f3bfca00640 1 -- 172.21.15.67:0/660121028 shutdown_connections 2024-03-09T09:15:58.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.152+0000 7f3bfca00640 1 --2- 172.21.15.67:0/660121028 >> v2:172.21.15.67:3300/0 conn(0x7f3bf80cadb0 0x7f3bf80b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:58.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.152+0000 7f3bfca00640 1 --2- 172.21.15.67:0/660121028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf8153300 0x7f3bf80b0000 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:58.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.152+0000 7f3bfca00640 1 -- 172.21.15.67:0/660121028 >> 172.21.15.67:0/660121028 conn(0x7f3bf80732d0 msgr2=0x7f3bf80ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:58.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.156+0000 7f3bfca00640 1 -- 172.21.15.67:0/660121028 shutdown_connections 2024-03-09T09:15:58.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.156+0000 7f3bfca00640 1 -- 172.21.15.67:0/660121028 wait complete. 2024-03-09T09:15:58.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.159+0000 7f3bfca00640 1 Processor -- start 2024-03-09T09:15:58.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.160+0000 7f3bfca00640 1 -- start start 2024-03-09T09:15:58.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.162+0000 7f3bfca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf80c5e40 0x7f3bf8154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:58.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.162+0000 7f3bfca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf80cadb0 0x7f3bf80ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:58.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.163+0000 7f3bfca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf8153300 0x7f3bf80abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:58.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.164+0000 7f3bf5e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf80cadb0 0x7f3bf80ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.164+0000 7f3bf5e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf80cadb0 0x7f3bf80ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:41316/0 (socket says 172.21.15.67:41316) 2024-03-09T09:15:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.164+0000 7f3bf7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf8153300 0x7f3bf80abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.164+0000 7f3bf7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf8153300 0x7f3bf80abb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:41462/0 (socket says 172.21.15.67:41462) 2024-03-09T09:15:58.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.164+0000 7f3bf6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf80c5e40 0x7f3bf8154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:58.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.165+0000 7f3bf6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf80c5e40 0x7f3bf8154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57514/0 (socket says 172.21.15.67:57514) 2024-03-09T09:15:58.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.165+0000 7f3bf5e00640 1 -- 172.21.15.67:0/795272119 learned_addr learned my addr 172.21.15.67:0/795272119 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:15:58.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.165+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80c59e0 con 0x7f3bf80c5e40 2024-03-09T09:15:58.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.165+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8159e90 con 0x7f3bf8153300 2024-03-09T09:15:58.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.165+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8156720 con 0x7f3bf80cadb0 2024-03-09T09:15:58.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.170+0000 7f3bf6800640 1 -- 172.21.15.67:0/795272119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf80c5e40 msgr2=0x7f3bf8154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:15:58.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.170+0000 7f3bf6800640 1 -- 172.21.15.67:0/795272119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf80c5e40 msgr2=0x7f3bf8154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:15:58.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.170+0000 7f3bf6800640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf80c5e40 0x7f3bf8154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:15:58.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.171+0000 7f3bf6800640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf80c5e40 0x7f3bf8154d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:15:58.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.171+0000 7f3bf5e00640 1 -- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf8153300 msgr2=0x7f3bf80abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:58.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.171+0000 7f3bf5e00640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf8153300 0x7f3bf80abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:58.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.171+0000 7f3bf5e00640 1 -- 172.21.15.67:0/795272119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf80c5e40 msgr2=0x7f3bf8154d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:15:58.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.171+0000 7f3bf5e00640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf80c5e40 0x7f3bf8154d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:58.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.171+0000 7f3bf5e00640 1 -- 172.21.15.67:0/795272119 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf80c0bb0 con 0x7f3bf80cadb0 2024-03-09T09:15:58.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.174+0000 7f3bf5e00640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf80cadb0 0x7f3bf80ab650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3be400c4a0 tx=0x7f3be400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:58.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.177+0000 7f3beea00640 1 -- 172.21.15.67:0/795272119 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4018070 con 0x7f3bf80cadb0 2024-03-09T09:15:58.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.177+0000 7f3beea00640 1 -- 172.21.15.67:0/795272119 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be40074e0 con 0x7f3bf80cadb0 2024-03-09T09:15:58.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.177+0000 7f3beea00640 1 -- 172.21.15.67:0/795272119 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4004a20 con 0x7f3bf80cadb0 2024-03-09T09:15:58.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.177+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf80c0440 con 0x7f3bf80cadb0 2024-03-09T09:15:58.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.177+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf80c0e30 con 0x7f3bf80cadb0 2024-03-09T09:15:58.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.183+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf81efb70 con 0x7f3bf80cadb0 2024-03-09T09:15:58.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.191+0000 7f3beea00640 1 -- 172.21.15.67:0/795272119 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3be401b070 con 0x7f3bf80cadb0 2024-03-09T09:15:58.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.198+0000 7f3beea00640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3bd806b5a0 0x7f3bd806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:15:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.199+0000 7f3beea00640 1 -- 172.21.15.67:0/795272119 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f3be4089c20 con 0x7f3bf80cadb0 2024-03-09T09:15:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.199+0000 7f3beea00640 1 -- 172.21.15.67:0/795272119 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3bf80c0bb0 con 0x7f3bf80cadb0 2024-03-09T09:15:58.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.202+0000 7f3bf6800640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3bd806b5a0 0x7f3bd806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:15:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.206+0000 7f3bf6800640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3bd806b5a0 0x7f3bd806da60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3be8036000 tx=0x7f3be8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:15:58.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.690+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3bf804f3c0 con 0x7f3bf80cadb0 2024-03-09T09:15:58.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.691+0000 7f3beea00640 1 -- 172.21.15.67:0/795272119 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f3be4058040 con 0x7f3bf80cadb0 2024-03-09T09:15:58.697 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":176,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:15:58.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.693+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3bd806b5a0 msgr2=0x7f3bd806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:58.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.693+0000 7f3bfca00640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3bd806b5a0 0x7f3bd806da60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3be8036000 tx=0x7f3be8008030 comp rx=0 tx=0).stop 2024-03-09T09:15:58.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.693+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf80cadb0 msgr2=0x7f3bf80ab650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:15:58.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.693+0000 7f3bfca00640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf80cadb0 0x7f3bf80ab650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3be400c4a0 tx=0x7f3be400c960 comp rx=0 tx=0).stop 2024-03-09T09:15:58.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.694+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 shutdown_connections 2024-03-09T09:15:58.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.694+0000 7f3bfca00640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3bd806b5a0 0x7f3bd806da60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:58.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.694+0000 7f3bfca00640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bf8153300 0x7f3bf80abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:58.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.694+0000 7f3bfca00640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3bf80cadb0 0x7f3bf80ab650 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:58.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.694+0000 7f3bfca00640 1 --2- 172.21.15.67:0/795272119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bf80c5e40 0x7f3bf8154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:15:58.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.694+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 >> 172.21.15.67:0/795272119 conn(0x7f3bf80732d0 msgr2=0x7f3bf8150b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:15:58.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.694+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 shutdown_connections 2024-03-09T09:15:58.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:15:58.694+0000 7f3bfca00640 1 -- 172.21.15.67:0/795272119 wait complete. 2024-03-09T09:16:01.719 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.918+0000 7f86b7400640 1 Processor -- start 2024-03-09T09:16:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.918+0000 7f86b7400640 1 -- start start 2024-03-09T09:16:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.920+0000 7f86b7400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f86b00c22a0 0x7f86b0071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.920+0000 7f86b7400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f86b0157d60 0x7f86b00acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.920+0000 7f86b7400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b01581a0 con 0x7f86b00ac730 2024-03-09T09:16:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.920+0000 7f86b7400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b0070ee0 con 0x7f86b0157d60 2024-03-09T09:16:01.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.920+0000 7f86b7400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b0154d00 con 0x7f86b00c22a0 2024-03-09T09:16:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.920+0000 7f86b5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f86b0157d60 0x7f86b00acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.920+0000 7f86b5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f86b0157d60 0x7f86b00acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41898/0 (socket says 172.21.15.67:41898) 2024-03-09T09:16:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.920+0000 7f86b5400640 1 -- 172.21.15.67:0/645694990 learned_addr learned my addr 172.21.15.67:0/645694990 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.920+0000 7f86a7e00640 1 --2- 172.21.15.67:0/645694990 >> v2:172.21.15.134:3300/0 conn(0x7f86b00c22a0 0x7f86b0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.921+0000 7f86b5400640 1 -- 172.21.15.67:0/645694990 >> v2:172.21.15.67:3300/0 conn(0x7f86b0157d60 msgr2=0x7f86b00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:16:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.921+0000 7f86b5400640 1 -- 172.21.15.67:0/645694990 >> v2:172.21.15.67:3300/0 conn(0x7f86b0157d60 msgr2=0x7f86b00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:16:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.921+0000 7f86b5400640 1 --2- 172.21.15.67:0/645694990 >> v2:172.21.15.67:3300/0 conn(0x7f86b0157d60 0x7f86b00acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.921+0000 7f86b5400640 1 --2- 172.21.15.67:0/645694990 >> v2:172.21.15.67:3300/0 conn(0x7f86b0157d60 0x7f86b00acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.921+0000 7f86a7e00640 1 -- 172.21.15.67:0/645694990 >> v2:172.21.15.67:3300/0 conn(0x7f86b0157d60 msgr2=0x7f86b00acb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.921+0000 7f86a7e00640 1 --2- 172.21.15.67:0/645694990 >> v2:172.21.15.67:3300/0 conn(0x7f86b0157d60 0x7f86b00acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:01.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.921+0000 7f86a7e00640 1 -- 172.21.15.67:0/645694990 >> v1:172.21.15.134:6789/0 conn(0x7f86b00ac730 legacy=0x7f86b0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:01.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.921+0000 7f86a7e00640 1 -- 172.21.15.67:0/645694990 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b0153d40 con 0x7f86b00c22a0 2024-03-09T09:16:01.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.922+0000 7f86a7e00640 1 --2- 172.21.15.67:0/645694990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b00c22a0 0x7f86b0071190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f86b00c1a30 tx=0x7f869802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a86951a2933a7a69 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.923+0000 7f86afe00640 1 -- 172.21.15.67:0/645694990 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980315a0 con 0x7f86b00c22a0 2024-03-09T09:16:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.923+0000 7f86afe00640 1 -- 172.21.15.67:0/645694990 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8698037070 con 0x7f86b00c22a0 2024-03-09T09:16:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.923+0000 7f86afe00640 1 -- 172.21.15.67:0/645694990 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698031960 con 0x7f86b00c22a0 2024-03-09T09:16:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.924+0000 7f86b7400640 1 -- 172.21.15.67:0/645694990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b00c22a0 msgr2=0x7f86b0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.924+0000 7f86b7400640 1 --2- 172.21.15.67:0/645694990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b00c22a0 0x7f86b0071190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f86b00c1a30 tx=0x7f869802b840 comp rx=0 tx=0).stop 2024-03-09T09:16:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.924+0000 7f86b7400640 1 -- 172.21.15.67:0/645694990 shutdown_connections 2024-03-09T09:16:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.924+0000 7f86b7400640 1 --2- 172.21.15.67:0/645694990 >> v2:172.21.15.67:3300/0 conn(0x7f86b0157d60 0x7f86b00acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.924+0000 7f86b7400640 1 --2- 172.21.15.67:0/645694990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b00c22a0 0x7f86b0071190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.924+0000 7f86b7400640 1 -- 172.21.15.67:0/645694990 >> 172.21.15.67:0/645694990 conn(0x7f86b00c7370 msgr2=0x7f86b00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.924+0000 7f86b7400640 1 -- 172.21.15.67:0/645694990 shutdown_connections 2024-03-09T09:16:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.924+0000 7f86b7400640 1 -- 172.21.15.67:0/645694990 wait complete. 2024-03-09T09:16:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.926+0000 7f86b7400640 1 Processor -- start 2024-03-09T09:16:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.927+0000 7f86b7400640 1 -- start start 2024-03-09T09:16:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.928+0000 7f86b7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b00ac730 0x7f86b0072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.928+0000 7f86b7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b00c22a0 0x7f86b01501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.928+0000 7f86b7400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86b0157d60 0x7f86b014b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.928+0000 7f86b7400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b0070c00 con 0x7f86b00c22a0 2024-03-09T09:16:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.928+0000 7f86b7400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b0070920 con 0x7f86b00ac730 2024-03-09T09:16:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.928+0000 7f86b7400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b0151610 con 0x7f86b0157d60 2024-03-09T09:16:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.928+0000 7f86a7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b00c22a0 0x7f86b01501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.928+0000 7f86a7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b00c22a0 0x7f86b01501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41904/0 (socket says 172.21.15.67:41904) 2024-03-09T09:16:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.928+0000 7f86a7e00640 1 -- 172.21.15.67:0/2077999148 learned_addr learned my addr 172.21.15.67:0/2077999148 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.928+0000 7f86b4a00640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b00ac730 0x7f86b0072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.929+0000 7f86a7e00640 1 -- 172.21.15.67:0/2077999148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b00c22a0 msgr2=0x7f86b01501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:16:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.929+0000 7f86a7e00640 1 -- 172.21.15.67:0/2077999148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b00c22a0 msgr2=0x7f86b01501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:16:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.929+0000 7f86a7e00640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b00c22a0 0x7f86b01501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.929+0000 7f86b5400640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86b0157d60 0x7f86b014b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.929+0000 7f86a7e00640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b00c22a0 0x7f86b01501c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.930+0000 7f86b4a00640 1 -- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86b0157d60 msgr2=0x7f86b014b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.930+0000 7f86b4a00640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86b0157d60 0x7f86b014b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.930+0000 7f86b4a00640 1 -- 172.21.15.67:0/2077999148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b00c22a0 msgr2=0x7f86b01501c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:16:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.930+0000 7f86b4a00640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b00c22a0 0x7f86b01501c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.930+0000 7f86b4a00640 1 -- 172.21.15.67:0/2077999148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b01ef1b0 con 0x7f86b00ac730 2024-03-09T09:16:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.930+0000 7f86b4a00640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b00ac730 0x7f86b0072be0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f86a000c370 tx=0x7f86a000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.931+0000 7f86ad600640 1 -- 172.21.15.67:0/2077999148 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a00052d0 con 0x7f86b00ac730 2024-03-09T09:16:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.931+0000 7f86ad600640 1 -- 172.21.15.67:0/2077999148 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a0005490 con 0x7f86b00ac730 2024-03-09T09:16:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.931+0000 7f86ad600640 1 -- 172.21.15.67:0/2077999148 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0007260 con 0x7f86b00ac730 2024-03-09T09:16:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.931+0000 7f86b7400640 1 -- 172.21.15.67:0/2077999148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b01ef450 con 0x7f86b00ac730 2024-03-09T09:16:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.931+0000 7f86b7400640 1 -- 172.21.15.67:0/2077999148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b01efa10 con 0x7f86b00ac730 2024-03-09T09:16:01.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.933+0000 7f86b7400640 1 -- 172.21.15.67:0/2077999148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b0075a00 con 0x7f86b00ac730 2024-03-09T09:16:01.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.936+0000 7f86ad600640 1 -- 172.21.15.67:0/2077999148 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f86a0007400 con 0x7f86b00ac730 2024-03-09T09:16:01.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.938+0000 7f86ad600640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f868c06b4d0 0x7f868c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:01.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.938+0000 7f86ad600640 1 -- 172.21.15.67:0/2077999148 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f86a0089dd0 con 0x7f86b00ac730 2024-03-09T09:16:01.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.939+0000 7f86a7e00640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f868c06b4d0 0x7f868c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:01.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.941+0000 7f86a7e00640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f868c06b4d0 0x7f868c06d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f869802b880 tx=0x7f869803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:01.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:01.949+0000 7f86ad600640 1 -- 172.21.15.67:0/2077999148 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f86a0058380 con 0x7f86b00ac730 2024-03-09T09:16:02.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.358+0000 7f86b7400640 1 -- 172.21.15.67:0/2077999148 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f86b00c1d70 con 0x7f86b00ac730 2024-03-09T09:16:02.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.363+0000 7f86ad600640 1 -- 172.21.15.67:0/2077999148 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f86a005c420 con 0x7f86b00ac730 2024-03-09T09:16:02.369 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":176,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:02.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 -- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f868c06b4d0 msgr2=0x7f868c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:02.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f868c06b4d0 0x7f868c06d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f869802b880 tx=0x7f869803d040 comp rx=0 tx=0).stop 2024-03-09T09:16:02.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 -- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b00ac730 msgr2=0x7f86b0072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:02.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b00ac730 0x7f86b0072be0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f86a000c370 tx=0x7f86a000c830 comp rx=0 tx=0).stop 2024-03-09T09:16:02.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 -- 172.21.15.67:0/2077999148 shutdown_connections 2024-03-09T09:16:02.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f868c06b4d0 0x7f868c06d990 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:02.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86b0157d60 0x7f86b014b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:02.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86b00c22a0 0x7f86b01501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:02.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 --2- 172.21.15.67:0/2077999148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86b00ac730 0x7f86b0072be0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:02.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 -- 172.21.15.67:0/2077999148 >> 172.21.15.67:0/2077999148 conn(0x7f86b00c7370 msgr2=0x7f86b0074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:02.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 -- 172.21.15.67:0/2077999148 shutdown_connections 2024-03-09T09:16:02.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:02.367+0000 7f86b7400640 1 -- 172.21.15.67:0/2077999148 wait complete. 2024-03-09T09:16:03.046 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 28% (102/356) Updating files: 29% (104/356) Updating files: 30% (107/356) Updating files: 31% (111/356) Updating files: 32% (114/356) Updating files: 33% (118/356) Updating files: 34% (122/356) Updating files: 35% (125/356) Updating files: 36% (129/356) Updating files: 37% (132/356) Updating files: 38% (136/356) Updating files: 39% (139/356) Updating files: 39% (142/356) Updating files: 40% (143/356) Updating files: 41% (146/356) Updating files: 42% (150/356) Updating files: 43% (154/356) Updating files: 44% (157/356) Updating files: 45% (161/356) Updating files: 46% (164/356) Updating files: 47% (168/356) Updating files: 48% (171/356) Updating files: 48% (172/356) Updating files: 49% (175/356) Updating files: 50% (178/356) Updating files: 51% (182/356) Updating files: 52% (186/356) Updating files: 53% (189/356) Updating files: 54% (193/356) Updating files: 55% (196/356) Updating files: 56% (200/356) Updating files: 57% (203/356) Updating files: 58% (207/356) Updating files: 59% (211/356) Updating files: 60% (214/356) Updating files: 61% (218/356) Updating files: 62% (221/356) Updating files: 62% (222/356) Updating files: 63% (225/356) Updating files: 64% (228/356) Updating files: 65% (232/356) Updating files: 66% (235/356) Updating files: 67% (239/356) Updating files: 68% (243/356) Updating files: 69% (246/356) Updating files: 70% (250/356) Updating files: 71% (253/356) Updating files: 72% (257/356) Updating files: 73% (260/356) Updating files: 74% (264/356) Updating files: 75% (267/356) Updating files: 76% (271/356) Updating files: 77% (275/356) Updating files: 78% (278/356) Updating files: 79% (282/356) Updating files: 80% (285/356) Updating files: 81% (289/356) Updating files: 82% (292/356) Updating files: 83% (296/356) Updating files: 84% (300/356) Updating files: 85% (303/356) Updating files: 86% (307/356) Updating files: 87% (310/356) Updating files: 88% (314/356) Updating files: 89% (317/356) Updating files: 90% (321/356) Updating files: 91% (324/356) Updating files: 92% (328/356) Updating files: 93% (332/356) Updating files: 94% (335/356) Updating files: 95% (339/356) Updating files: 95% (341/356) Updating files: 96% (342/356) Updating files: 97% (346/356) Updating files: 98% (349/356) Updating files: 99% (353/356) Updating files: 100% (356/356) Updating files: 100% (356/356), done. 2024-03-09T09:16:03.593 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 8336f81c5c7 0.84 2024-03-09T09:16:03.610 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.84 2024-03-09T09:16:03.985 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:16:03.985 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 85 -eq 48 ']' 2024-03-09T09:16:03.985 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.85 2024-03-09T09:16:03.985 INFO:tasks.workunit.client.1.smithi134.stdout:v0.85 2024-03-09T09:16:03.986 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.85 2024-03-09T09:16:03.986 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.85 2024-03-09T09:16:04.324 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 30% (66/218) Updating files: 31% (68/218) Updating files: 32% (70/218) Updating files: 33% (72/218) Updating files: 34% (75/218) Updating files: 35% (77/218) Updating files: 36% (79/218) Updating files: 37% (81/218) Updating files: 38% (83/218) Updating files: 39% (86/218) Updating files: 40% (88/218) Updating files: 41% (90/218) Updating files: 42% (92/218) Updating files: 43% (94/218) Updating files: 44% (96/218) Updating files: 45% (99/218) Updating files: 46% (101/218) Updating files: 47% (103/218) Updating files: 48% (105/218) Updating files: 49% (107/218) Updating files: 50% (109/218) Updating files: 51% (112/218) Updating files: 52% (114/218) Updating files: 53% (116/218) Updating files: 54% (118/218) Updating files: 55% (120/218) Updating files: 56% (123/218) Updating files: 57% (125/218) Updating files: 58% (127/218) Updating files: 59% (129/218) Updating files: 60% (131/218) Updating files: 61% (133/218) Updating files: 61% (134/218) Updating files: 62% (136/218) Updating files: 63% (138/218) Updating files: 64% (140/218) Updating files: 65% (142/218) Updating files: 66% (144/218) Updating files: 67% (147/218) Updating files: 68% (149/218) Updating files: 69% (151/218) Updating files: 70% (153/218) Updating files: 71% (155/218) Updating files: 72% (157/218) Updating files: 72% (159/218) Updating files: 73% (160/218) Updating files: 74% (162/218) Updating files: 75% (164/218) Updating files: 76% (166/218) Updating files: 77% (168/218) Updating files: 78% (171/218) Updating files: 79% (173/218) Updating files: 80% (175/218) Updating files: 81% (177/218) Updating files: 82% (179/218) Updating files: 83% (181/218) Updating files: 83% (183/218) Updating files: 84% (184/218) Updating files: 85% (186/218) Updating files: 86% (188/218) Updating files: 87% (190/218) Updating files: 88% (192/218) Updating files: 89% (195/218) Updating files: 90% (197/218) Updating files: 91% (199/218) Updating files: 92% (201/218) Updating files: 93% (203/218) Updating files: 94% (205/218) Updating files: 95% (208/218) Updating files: 96% (210/218) Updating files: 97% (212/218) Updating files: 98% (214/218) Updating files: 99% (216/218) Updating files: 100% (218/218) Updating files: 100% (218/218), done. 2024-03-09T09:16:04.384 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 78ff1f0a5df 0.83 2024-03-09T09:16:04.389 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.83 2024-03-09T09:16:04.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:16:04.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 84 -eq 48 ']' 2024-03-09T09:16:04.913 INFO:tasks.workunit.client.0.smithi067.stdout:v0.84 2024-03-09T09:16:04.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.84 2024-03-09T09:16:04.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.84 2024-03-09T09:16:04.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.84 2024-03-09T09:16:05.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.582+0000 7f46a7a00640 1 Processor -- start 2024-03-09T09:16:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.582+0000 7f46a7a00640 1 -- start start 2024-03-09T09:16:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.583+0000 7f46a7a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46a0152a20 0x7f46a00aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.584+0000 7f46a7a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00ab3c0 con 0x7f46a00c30f0 2024-03-09T09:16:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.584+0000 7f46a7a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00aab20 con 0x7f46a006dbe0 2024-03-09T09:16:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.584+0000 7f46a7a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a00c82d0 con 0x7f46a0152a20 2024-03-09T09:16:05.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.584+0000 7f469fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46a0152a20 0x7f46a00aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.584+0000 7f46a5000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f46a00c30f0 0x7f46a01490f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36506/0 (socket says 172.21.15.67:36506) 2024-03-09T09:16:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.584+0000 7f46a5000640 1 -- 172.21.15.67:0/1665067918 learned_addr learned my addr 172.21.15.67:0/1665067918 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:16:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.584+0000 7f469fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46a0152a20 0x7f46a00aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41908/0 (socket says 172.21.15.67:41908) 2024-03-09T09:16:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.584+0000 7f469fe00640 1 -- 172.21.15.67:0/1665067918 >> v2:172.21.15.67:3300/0 conn(0x7f46a0152a20 msgr2=0x7f46a00aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:16:05.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.584+0000 7f469fe00640 1 -- 172.21.15.67:0/1665067918 >> v2:172.21.15.67:3300/0 conn(0x7f46a0152a20 msgr2=0x7f46a00aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:16:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.584+0000 7f469fe00640 1 --2- 172.21.15.67:0/1665067918 >> v2:172.21.15.67:3300/0 conn(0x7f46a0152a20 0x7f46a00aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.584+0000 7f469fe00640 1 --2- 172.21.15.67:0/1665067918 >> v2:172.21.15.67:3300/0 conn(0x7f46a0152a20 0x7f46a00aa420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.585+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432712612 0 0) 0x7f46a00aab20 con 0x7f46a006dbe0 2024-03-09T09:16:05.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.585+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4688005730 con 0x7f46a006dbe0 2024-03-09T09:16:05.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1153216889 0 0) 0x7f4688005730 con 0x7f46a006dbe0 2024-03-09T09:16:05.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 >> v2:172.21.15.67:3300/0 conn(0x7f46a0152a20 msgr2=0x7f46a00aa420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:05.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 --2- 172.21.15.67:0/1665067918 >> v2:172.21.15.67:3300/0 conn(0x7f46a0152a20 0x7f46a00aa420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:05.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 >> v1:172.21.15.67:6789/0 conn(0x7f46a00c30f0 legacy=0x7f46a01490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00aae00 con 0x7f46a006dbe0 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f46980019f0 con 0x7f46a006dbe0 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 >> v1:172.21.15.134:6789/0 conn(0x7f46a006dbe0 legacy=0x7f46a0072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 --2- 172.21.15.67:0/1665067918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4688005bc0 0x7f4688005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 --2- 172.21.15.67:0/1665067918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4688006a10 0x7f4688006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 --2- 172.21.15.67:0/1665067918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46880073d0 0x7f468800ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880105c0 con 0x7f4688006a10 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4688010740 con 0x7f46880073d0 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4688010940 con 0x7f4688005bc0 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.586+0000 7f469fe00640 1 --2- 172.21.15.67:0/1665067918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4688006a10 0x7f4688006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.587+0000 7f46a5000640 1 --2- 172.21.15.67:0/1665067918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4688005bc0 0x7f4688005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.587+0000 7f469fe00640 1 -- 172.21.15.67:0/1665067918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4688005bc0 msgr2=0x7f4688005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.587+0000 7f469fe00640 1 --2- 172.21.15.67:0/1665067918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4688005bc0 0x7f4688005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:05.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.587+0000 7f469fe00640 1 -- 172.21.15.67:0/1665067918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46880073d0 msgr2=0x7f468800ffd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T09:16:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.587+0000 7f469fe00640 1 --2- 172.21.15.67:0/1665067918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46880073d0 0x7f468800ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.587+0000 7f469fe00640 1 -- 172.21.15.67:0/1665067918 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4688005730 con 0x7f4688006a10 2024-03-09T09:16:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.587+0000 7f46a5000640 1 --2- 172.21.15.67:0/1665067918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4688005bc0 0x7f4688005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:16:05.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.587+0000 7f46a5a00640 1 -- 172.21.15.67:0/1665067918 reap_dead start 2024-03-09T09:16:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.588+0000 7f469fe00640 1 --2- 172.21.15.67:0/1665067918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4688006a10 0x7f4688006e90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f468c00b9e0 tx=0x7f4688007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fbf48edefaea0c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.588+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c036020 con 0x7f4688006a10 2024-03-09T09:16:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.588+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c032630 con 0x7f4688006a10 2024-03-09T09:16:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.588+0000 7f469f400640 1 -- 172.21.15.67:0/1665067918 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c00b5e0 con 0x7f4688006a10 2024-03-09T09:16:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.589+0000 7f46a7a00640 1 -- 172.21.15.67:0/1665067918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4688006a10 msgr2=0x7f4688006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.589+0000 7f46a7a00640 1 --2- 172.21.15.67:0/1665067918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4688006a10 0x7f4688006e90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f468c00b9e0 tx=0x7f4688007a40 comp rx=0 tx=0).stop 2024-03-09T09:16:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.589+0000 7f46a7a00640 1 -- 172.21.15.67:0/1665067918 shutdown_connections 2024-03-09T09:16:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.589+0000 7f46a7a00640 1 --2- 172.21.15.67:0/1665067918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4688006a10 0x7f4688006e90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.589+0000 7f46a7a00640 1 -- 172.21.15.67:0/1665067918 >> 172.21.15.67:0/1665067918 conn(0x7f46a00c7370 msgr2=0x7f46a00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.589+0000 7f46a7a00640 1 -- 172.21.15.67:0/1665067918 shutdown_connections 2024-03-09T09:16:05.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.589+0000 7f46a7a00640 1 -- 172.21.15.67:0/1665067918 wait complete. 2024-03-09T09:16:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.591+0000 7f46a7a00640 1 Processor -- start 2024-03-09T09:16:05.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.591+0000 7f46a7a00640 1 -- start start 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a7a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4688006a10 0x7f46a014ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a7a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a014ee40 0x7f46a01de250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a7a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a01de790 0x7f46a01e7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a7a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00ab620 con 0x7f46a01de790 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a7a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a006d6c0 con 0x7f4688006a10 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a7a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00ab7a0 con 0x7f46a014ee40 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a01de790 0x7f46a01e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a01de790 0x7f46a01e7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41922/0 (socket says 172.21.15.67:41922) 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a5a00640 1 -- 172.21.15.67:0/2937254935 learned_addr learned my addr 172.21.15.67:0/2937254935 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a5a00640 1 -- 172.21.15.67:0/2937254935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a01de790 msgr2=0x7f46a01e7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a5a00640 1 -- 172.21.15.67:0/2937254935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a01de790 msgr2=0x7f46a01e7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a5a00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a01de790 0x7f46a01e7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:05.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a5000640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4688006a10 0x7f46a014ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:05.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f46a5a00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a01de790 0x7f46a01e7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:05.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.592+0000 7f469fe00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a014ee40 0x7f46a01de250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:05.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.593+0000 7f46a5000640 1 -- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a014ee40 msgr2=0x7f46a01de250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:05.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.593+0000 7f46a5000640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a014ee40 0x7f46a01de250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:05.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.593+0000 7f46a5000640 1 -- 172.21.15.67:0/2937254935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a01de790 msgr2=0x7f46a01e7010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:16:05.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.593+0000 7f46a5000640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a01de790 0x7f46a01e7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:05.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.593+0000 7f46a5000640 1 -- 172.21.15.67:0/2937254935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a01501a0 con 0x7f4688006a10 2024-03-09T09:16:05.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.593+0000 7f46a5000640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4688006a10 0x7f46a014ba70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f46a006c500 tx=0x7f469400d890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:05.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.594+0000 7f469cc00640 1 -- 172.21.15.67:0/2937254935 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694018070 con 0x7f4688006a10 2024-03-09T09:16:05.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.594+0000 7f469cc00640 1 -- 172.21.15.67:0/2937254935 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4694005ca0 con 0x7f4688006a10 2024-03-09T09:16:05.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.594+0000 7f46a7a00640 1 -- 172.21.15.67:0/2937254935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a01eb480 con 0x7f4688006a10 2024-03-09T09:16:05.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.594+0000 7f469cc00640 1 -- 172.21.15.67:0/2937254935 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46940072d0 con 0x7f4688006a10 2024-03-09T09:16:05.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.594+0000 7f46a7a00640 1 -- 172.21.15.67:0/2937254935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a01eb1a0 con 0x7f4688006a10 2024-03-09T09:16:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.595+0000 7f46a7a00640 1 -- 172.21.15.67:0/2937254935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a0075a00 con 0x7f4688006a10 2024-03-09T09:16:05.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.596+0000 7f469cc00640 1 -- 172.21.15.67:0/2937254935 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f469401b070 con 0x7f4688006a10 2024-03-09T09:16:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.597+0000 7f469cc00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f468006b4d0 0x7f468006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:05.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.597+0000 7f469cc00640 1 -- 172.21.15.67:0/2937254935 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4694089ee0 con 0x7f4688006a10 2024-03-09T09:16:05.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.598+0000 7f469fe00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f468006b4d0 0x7f468006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:05.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.599+0000 7f469fe00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f468006b4d0 0x7f468006d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f468c037000 tx=0x7f468c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:05.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.609+0000 7f469cc00640 1 -- 172.21.15.67:0/2937254935 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f46a00ab3c0 con 0x7f4688006a10 2024-03-09T09:16:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.921+0000 7f46a7a00640 1 -- 172.21.15.67:0/2937254935 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f46a0073dc0 con 0x7f4688006a10 2024-03-09T09:16:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.923+0000 7f469cc00640 1 -- 172.21.15.67:0/2937254935 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f46940583b0 con 0x7f4688006a10 2024-03-09T09:16:05.928 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":176,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":29,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:05.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.925+0000 7f46a7a00640 1 -- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f468006b4d0 msgr2=0x7f468006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:05.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.925+0000 7f46a7a00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f468006b4d0 0x7f468006d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f468c037000 tx=0x7f468c030000 comp rx=0 tx=0).stop 2024-03-09T09:16:05.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.925+0000 7f46a7a00640 1 -- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4688006a10 msgr2=0x7f46a014ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:05.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.925+0000 7f46a7a00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4688006a10 0x7f46a014ba70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f46a006c500 tx=0x7f469400d890 comp rx=0 tx=0).stop 2024-03-09T09:16:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.925+0000 7f46a7a00640 1 -- 172.21.15.67:0/2937254935 shutdown_connections 2024-03-09T09:16:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.925+0000 7f46a7a00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f468006b4d0 0x7f468006d990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.925+0000 7f46a7a00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a01de790 0x7f46a01e7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.925+0000 7f46a7a00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a014ee40 0x7f46a01de250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.925+0000 7f46a7a00640 1 --2- 172.21.15.67:0/2937254935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4688006a10 0x7f46a014ba70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.925+0000 7f46a7a00640 1 -- 172.21.15.67:0/2937254935 >> 172.21.15.67:0/2937254935 conn(0x7f46a00c7370 msgr2=0x7f46a0150ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.926+0000 7f46a7a00640 1 -- 172.21.15.67:0/2937254935 shutdown_connections 2024-03-09T09:16:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:05.926+0000 7f46a7a00640 1 -- 172.21.15.67:0/2937254935 wait complete. 2024-03-09T09:16:08.957 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:09.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.155+0000 7f5a24c00640 1 Processor -- start 2024-03-09T09:16:09.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.156+0000 7f5a24c00640 1 -- start start 2024-03-09T09:16:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.157+0000 7f5a24c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5a200c3780 0x7f5a20152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.157+0000 7f5a24c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a200c1d70 con 0x7f5a2014c5c0 2024-03-09T09:16:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.157+0000 7f5a24c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a200c2010 con 0x7f5a200aad00 2024-03-09T09:16:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.157+0000 7f5a24c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a200aaae0 con 0x7f5a200c3780 2024-03-09T09:16:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.158+0000 7f5a1de00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5a2014c5c0 0x7f5a2006df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39308/0 (socket says 172.21.15.67:39308) 2024-03-09T09:16:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.158+0000 7f5a1de00640 1 -- 172.21.15.67:0/1128259652 learned_addr learned my addr 172.21.15.67:0/1128259652 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:16:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.158+0000 7f5a17e00640 1 --2- 172.21.15.67:0/1128259652 >> v2:172.21.15.134:3300/0 conn(0x7f5a200c3780 0x7f5a20152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.159+0000 7f5a17e00640 1 -- 172.21.15.67:0/1128259652 >> v1:172.21.15.134:6789/0 conn(0x7f5a200aad00 legacy=0x7f5a200ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.159+0000 7f5a17e00640 1 -- 172.21.15.67:0/1128259652 >> v1:172.21.15.67:6789/0 conn(0x7f5a2014c5c0 legacy=0x7f5a2006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.159+0000 7f5a17e00640 1 -- 172.21.15.67:0/1128259652 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a200c22f0 con 0x7f5a200c3780 2024-03-09T09:16:09.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.160+0000 7f5a17e00640 1 --2- 172.21.15.67:0/1128259652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a200c3780 0x7f5a20152ce0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5a080097a0 tx=0x7f5a0802dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e35604fa3b6d1997 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:09.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.161+0000 7f5a1d400640 1 -- 172.21.15.67:0/1128259652 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0802f8a0 con 0x7f5a200c3780 2024-03-09T09:16:09.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.161+0000 7f5a1d400640 1 -- 172.21.15.67:0/1128259652 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0802fa40 con 0x7f5a200c3780 2024-03-09T09:16:09.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.161+0000 7f5a1d400640 1 -- 172.21.15.67:0/1128259652 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0802fe10 con 0x7f5a200c3780 2024-03-09T09:16:09.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.162+0000 7f5a24c00640 1 -- 172.21.15.67:0/1128259652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a200c3780 msgr2=0x7f5a20152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:09.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.162+0000 7f5a24c00640 1 --2- 172.21.15.67:0/1128259652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a200c3780 0x7f5a20152ce0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5a080097a0 tx=0x7f5a0802dae0 comp rx=0 tx=0).stop 2024-03-09T09:16:09.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.162+0000 7f5a24c00640 1 -- 172.21.15.67:0/1128259652 shutdown_connections 2024-03-09T09:16:09.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.162+0000 7f5a24c00640 1 --2- 172.21.15.67:0/1128259652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a200c3780 0x7f5a20152ce0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:09.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.162+0000 7f5a24c00640 1 -- 172.21.15.67:0/1128259652 >> 172.21.15.67:0/1128259652 conn(0x7f5a200c7370 msgr2=0x7f5a200c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:09.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.162+0000 7f5a24c00640 1 -- 172.21.15.67:0/1128259652 shutdown_connections 2024-03-09T09:16:09.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.162+0000 7f5a24c00640 1 -- 172.21.15.67:0/1128259652 wait complete. 2024-03-09T09:16:09.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.164+0000 7f5a24c00640 1 Processor -- start 2024-03-09T09:16:09.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.164+0000 7f5a24c00640 1 -- start start 2024-03-09T09:16:09.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a24c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a200aad00 0x7f5a201e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:09.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a24c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a200c3780 0x7f5a20148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:09.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a24c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a2014c5c0 0x7f5a201ebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:09.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a24c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a200c28b0 con 0x7f5a200c3780 2024-03-09T09:16:09.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a24c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2014c9a0 con 0x7f5a200aad00 2024-03-09T09:16:09.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a24c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a200ab820 con 0x7f5a2014c5c0 2024-03-09T09:16:09.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a17e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a200c3780 0x7f5a20148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:09.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a17e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a200c3780 0x7f5a20148bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43488/0 (socket says 172.21.15.67:43488) 2024-03-09T09:16:09.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a17e00640 1 -- 172.21.15.67:0/2294947416 learned_addr learned my addr 172.21.15.67:0/2294947416 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:09.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a17e00640 1 -- 172.21.15.67:0/2294947416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a200c3780 msgr2=0x7f5a20148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:16:09.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a17e00640 1 -- 172.21.15.67:0/2294947416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a200c3780 msgr2=0x7f5a20148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:16:09.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a17e00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a200c3780 0x7f5a20148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:09.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a1de00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a200aad00 0x7f5a201e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:09.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a17e00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a200c3780 0x7f5a20148bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:09.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.166+0000 7f5a1e800640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a2014c5c0 0x7f5a201ebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:09.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.167+0000 7f5a1de00640 1 -- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a2014c5c0 msgr2=0x7f5a201ebcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.167+0000 7f5a1de00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a2014c5c0 0x7f5a201ebcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.167+0000 7f5a1de00640 1 -- 172.21.15.67:0/2294947416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a200c3780 msgr2=0x7f5a20148bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:16:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.167+0000 7f5a1de00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a200c3780 0x7f5a20148bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.167+0000 7f5a1de00640 1 -- 172.21.15.67:0/2294947416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a201537f0 con 0x7f5a200aad00 2024-03-09T09:16:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.168+0000 7f5a1de00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a200aad00 0x7f5a201e7ec0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000c440 tx=0x7f5a10001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:09.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.168+0000 7f5a16000640 1 -- 172.21.15.67:0/2294947416 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10017070 con 0x7f5a200aad00 2024-03-09T09:16:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.168+0000 7f5a24c00640 1 -- 172.21.15.67:0/2294947416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a20153ad0 con 0x7f5a200aad00 2024-03-09T09:16:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.168+0000 7f5a16000640 1 -- 172.21.15.67:0/2294947416 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a10005ce0 con 0x7f5a200aad00 2024-03-09T09:16:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.169+0000 7f5a16000640 1 -- 172.21.15.67:0/2294947416 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a100072d0 con 0x7f5a200aad00 2024-03-09T09:16:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.169+0000 7f5a24c00640 1 -- 172.21.15.67:0/2294947416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a201eb7b0 con 0x7f5a200aad00 2024-03-09T09:16:09.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.170+0000 7f5a24c00640 1 -- 172.21.15.67:0/2294947416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a20075a00 con 0x7f5a200aad00 2024-03-09T09:16:09.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.172+0000 7f5a16000640 1 -- 172.21.15.67:0/2294947416 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5a1001a070 con 0x7f5a200aad00 2024-03-09T09:16:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.173+0000 7f5a16000640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59fc06b4d0 0x7f59fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.174+0000 7f5a16000640 1 -- 172.21.15.67:0/2294947416 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f5a1008a010 con 0x7f5a200aad00 2024-03-09T09:16:09.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.174+0000 7f5a17e00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59fc06b4d0 0x7f59fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:09.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.175+0000 7f5a17e00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59fc06b4d0 0x7f59fc06d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5a0802db20 tx=0x7f5a08008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:09.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.185+0000 7f5a16000640 1 -- 172.21.15.67:0/2294947416 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5a100584e0 con 0x7f5a200aad00 2024-03-09T09:16:09.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.551+0000 7f5a24c00640 1 -- 172.21.15.67:0/2294947416 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5a200c33d0 con 0x7f5a200aad00 2024-03-09T09:16:09.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.551+0000 7f5a16000640 1 -- 172.21.15.67:0/2294947416 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f5a1005c580 con 0x7f5a200aad00 2024-03-09T09:16:09.557 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":176,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":33,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 -- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59fc06b4d0 msgr2=0x7f59fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59fc06b4d0 0x7f59fc06d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5a0802db20 tx=0x7f5a08008030 comp rx=0 tx=0).stop 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 -- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a200aad00 msgr2=0x7f5a201e7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a200aad00 0x7f5a201e7ec0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000c440 tx=0x7f5a10001e70 comp rx=0 tx=0).stop 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 -- 172.21.15.67:0/2294947416 shutdown_connections 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59fc06b4d0 0x7f59fc06d990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a2014c5c0 0x7f5a201ebcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a200c3780 0x7f5a20148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 --2- 172.21.15.67:0/2294947416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a200aad00 0x7f5a201e7ec0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 -- 172.21.15.67:0/2294947416 >> 172.21.15.67:0/2294947416 conn(0x7f5a200c7370 msgr2=0x7f5a2014bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 -- 172.21.15.67:0/2294947416 shutdown_connections 2024-03-09T09:16:09.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:09.554+0000 7f5a24c00640 1 -- 172.21.15.67:0/2294947416 wait complete. 2024-03-09T09:16:12.578 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:12.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.770+0000 7f1559c00640 1 Processor -- start 2024-03-09T09:16:12.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.770+0000 7f1559c00640 1 -- start start 2024-03-09T09:16:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.772+0000 7f1559c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1554073900 0x7f15540c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.772+0000 7f1559c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15541540d0 con 0x7f15540c3e00 2024-03-09T09:16:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.772+0000 7f1559c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15541543b0 con 0x7f15540aa9d0 2024-03-09T09:16:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.772+0000 7f1559c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155406f6d0 con 0x7f1554073900 2024-03-09T09:16:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.772+0000 7f1553e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f15540c3e00 0x7f15541538c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39324/0 (socket says 172.21.15.67:39324) 2024-03-09T09:16:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.772+0000 7f1553e00640 1 -- 172.21.15.67:0/155196342 learned_addr learned my addr 172.21.15.67:0/155196342 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:16:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.773+0000 7f1559200640 1 --2- 172.21.15.67:0/155196342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1554073900 0x7f15540c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.774+0000 7f1552a00640 1 -- 172.21.15.67:0/155196342 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676230181 0 0) 0x7f15541543b0 con 0x7f15540aa9d0 2024-03-09T09:16:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.774+0000 7f1552a00640 1 -- 172.21.15.67:0/155196342 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1538003520 con 0x7f15540aa9d0 2024-03-09T09:16:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.774+0000 7f1559200640 1 -- 172.21.15.67:0/155196342 >> v1:172.21.15.134:6789/0 conn(0x7f15540aa9d0 legacy=0x7f155406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.774+0000 7f1559200640 1 -- 172.21.15.67:0/155196342 >> v1:172.21.15.67:6789/0 conn(0x7f15540c3e00 legacy=0x7f15541538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:12.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.774+0000 7f1559200640 1 -- 172.21.15.67:0/155196342 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1554154690 con 0x7f1554073900 2024-03-09T09:16:12.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.775+0000 7f1559200640 1 --2- 172.21.15.67:0/155196342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1554073900 0x7f15540c41e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f154c009dc0 tx=0x7f154c02be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fcb4bce588f4984 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:12.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.775+0000 7f1552a00640 1 -- 172.21.15.67:0/155196342 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c039070 con 0x7f1554073900 2024-03-09T09:16:12.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.776+0000 7f1552a00640 1 -- 172.21.15.67:0/155196342 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c02cbb0 con 0x7f1554073900 2024-03-09T09:16:12.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.776+0000 7f1552a00640 1 -- 172.21.15.67:0/155196342 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c034440 con 0x7f1554073900 2024-03-09T09:16:12.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.776+0000 7f1559c00640 1 -- 172.21.15.67:0/155196342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1554073900 msgr2=0x7f15540c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:12.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.776+0000 7f1559c00640 1 --2- 172.21.15.67:0/155196342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1554073900 0x7f15540c41e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f154c009dc0 tx=0x7f154c02be00 comp rx=0 tx=0).stop 2024-03-09T09:16:12.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.776+0000 7f1559c00640 1 -- 172.21.15.67:0/155196342 shutdown_connections 2024-03-09T09:16:12.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.776+0000 7f1559c00640 1 --2- 172.21.15.67:0/155196342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1554073900 0x7f15540c41e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:12.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.776+0000 7f1559c00640 1 -- 172.21.15.67:0/155196342 >> 172.21.15.67:0/155196342 conn(0x7f1554067180 msgr2=0x7f1554067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:12.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.776+0000 7f1559c00640 1 -- 172.21.15.67:0/155196342 shutdown_connections 2024-03-09T09:16:12.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.776+0000 7f1559c00640 1 -- 172.21.15.67:0/155196342 wait complete. 2024-03-09T09:16:12.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.778+0000 7f1559c00640 1 Processor -- start 2024-03-09T09:16:12.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.778+0000 7f1559c00640 1 -- start start 2024-03-09T09:16:12.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1559c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1554073900 0x7f15541eb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:12.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1559c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15540aa9d0 0x7f1554149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:12.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1559c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15540c3e00 0x7f15541ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:12.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1559c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1554070550 con 0x7f1554073900 2024-03-09T09:16:12.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1559c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155406f9b0 con 0x7f15540aa9d0 2024-03-09T09:16:12.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1559c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15540706d0 con 0x7f15540c3e00 2024-03-09T09:16:12.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1553e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1554073900 0x7f15541eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:12.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1553e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1554073900 0x7f15541eb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43496/0 (socket says 172.21.15.67:43496) 2024-03-09T09:16:12.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1553e00640 1 -- 172.21.15.67:0/2528729384 learned_addr learned my addr 172.21.15.67:0/2528729384 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:12.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1553e00640 1 -- 172.21.15.67:0/2528729384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1554073900 msgr2=0x7f15541eb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:16:12.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1553e00640 1 -- 172.21.15.67:0/2528729384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1554073900 msgr2=0x7f15541eb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:16:12.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1553e00640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1554073900 0x7f15541eb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:12.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.779+0000 7f1553e00640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1554073900 0x7f15541eb770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:12.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.780+0000 7f1553400640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15540aa9d0 0x7f1554149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:12.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.780+0000 7f1559200640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15540c3e00 0x7f15541ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:12.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.780+0000 7f1559200640 1 -- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15540aa9d0 msgr2=0x7f1554149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:12.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.780+0000 7f1559200640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15540aa9d0 0x7f1554149370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:12.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.780+0000 7f1559200640 1 -- 172.21.15.67:0/2528729384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1554073900 msgr2=0x7f15541eb770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:16:12.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.780+0000 7f1559200640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1554073900 0x7f15541eb770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:12.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.780+0000 7f1559200640 1 -- 172.21.15.67:0/2528729384 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15540c8ac0 con 0x7f15540c3e00 2024-03-09T09:16:12.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.780+0000 7f1553400640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15540aa9d0 0x7f1554149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:16:12.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.781+0000 7f1559200640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15540c3e00 0x7f15541ef5b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f154c033000 tx=0x7f154c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:12.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.781+0000 7f154be00640 1 -- 172.21.15.67:0/2528729384 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c043020 con 0x7f15540c3e00 2024-03-09T09:16:12.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.781+0000 7f154be00640 1 -- 172.21.15.67:0/2528729384 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c003e10 con 0x7f15540c3e00 2024-03-09T09:16:12.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.781+0000 7f1559c00640 1 -- 172.21.15.67:0/2528729384 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15540c8da0 con 0x7f15540c3e00 2024-03-09T09:16:12.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.781+0000 7f154be00640 1 -- 172.21.15.67:0/2528729384 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c039040 con 0x7f15540c3e00 2024-03-09T09:16:12.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.782+0000 7f1559c00640 1 -- 172.21.15.67:0/2528729384 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15541498b0 con 0x7f15540c3e00 2024-03-09T09:16:12.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.783+0000 7f1559c00640 1 -- 172.21.15.67:0/2528729384 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155404f3c0 con 0x7f15540c3e00 2024-03-09T09:16:12.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.785+0000 7f154be00640 1 -- 172.21.15.67:0/2528729384 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f154c046020 con 0x7f15540c3e00 2024-03-09T09:16:12.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.786+0000 7f154be00640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f153406b4d0 0x7f153406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:12.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.786+0000 7f154be00640 1 -- 172.21.15.67:0/2528729384 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f154c0ace40 con 0x7f15540c3e00 2024-03-09T09:16:12.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.787+0000 7f1553e00640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f153406b4d0 0x7f153406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.788+0000 7f1553e00640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f153406b4d0 0x7f153406d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1544005880 tx=0x7f1544009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:12.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:12.799+0000 7f154be00640 1 -- 172.21.15.67:0/2528729384 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f154c04d030 con 0x7f15540c3e00 2024-03-09T09:16:13.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.125+0000 7f1559c00640 1 -- 172.21.15.67:0/2528729384 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f15540736e0 con 0x7f15540c3e00 2024-03-09T09:16:13.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.126+0000 7f154be00640 1 -- 172.21.15.67:0/2528729384 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f154c07b8c0 con 0x7f15540c3e00 2024-03-09T09:16:13.132 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":176,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":36,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:13.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.130+0000 7f1559c00640 1 -- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f153406b4d0 msgr2=0x7f153406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:13.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.130+0000 7f1559c00640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f153406b4d0 0x7f153406d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1544005880 tx=0x7f1544009040 comp rx=0 tx=0).stop 2024-03-09T09:16:13.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.130+0000 7f1559c00640 1 -- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15540c3e00 msgr2=0x7f15541ef5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:13.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.130+0000 7f1559c00640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15540c3e00 0x7f15541ef5b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f154c033000 tx=0x7f154c004a40 comp rx=0 tx=0).stop 2024-03-09T09:16:13.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.131+0000 7f1559c00640 1 -- 172.21.15.67:0/2528729384 shutdown_connections 2024-03-09T09:16:13.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.131+0000 7f1559c00640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f153406b4d0 0x7f153406d990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:13.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.131+0000 7f1559c00640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15540c3e00 0x7f15541ef5b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:13.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.131+0000 7f1559c00640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15540aa9d0 0x7f1554149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:13.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.131+0000 7f1559c00640 1 --2- 172.21.15.67:0/2528729384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1554073900 0x7f15541eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:13.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.131+0000 7f1559c00640 1 -- 172.21.15.67:0/2528729384 >> 172.21.15.67:0/2528729384 conn(0x7f1554067180 msgr2=0x7f15540aba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:13.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.131+0000 7f1559c00640 1 -- 172.21.15.67:0/2528729384 shutdown_connections 2024-03-09T09:16:13.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:13.131+0000 7f1559c00640 1 -- 172.21.15.67:0/2528729384 wait complete. 2024-03-09T09:16:13.468 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 8% (31/356) Updating files: 8% (32/356) Updating files: 9% (33/356) Updating files: 10% (36/356) Updating files: 11% (40/356) Updating files: 12% (43/356) Updating files: 13% (47/356) Updating files: 14% (50/356) Updating files: 15% (54/356) Updating files: 16% (57/356) Updating files: 17% (61/356) Updating files: 18% (65/356) Updating files: 19% (68/356) Updating files: 20% (72/356) Updating files: 21% (75/356) Updating files: 22% (79/356) Updating files: 23% (82/356) Updating files: 24% (86/356) Updating files: 25% (89/356) Updating files: 26% (93/356) Updating files: 27% (97/356) Updating files: 28% (100/356) Updating files: 29% (104/356) Updating files: 30% (107/356) Updating files: 30% (108/356) Updating files: 31% (111/356) Updating files: 32% (114/356) Updating files: 33% (118/356) Updating files: 34% (122/356) Updating files: 35% (125/356) Updating files: 36% (129/356) Updating files: 37% (132/356) Updating files: 38% (136/356) Updating files: 39% (139/356) Updating files: 40% (143/356) Updating files: 41% (146/356) Updating files: 42% (150/356) Updating files: 43% (154/356) Updating files: 44% (157/356) Updating files: 45% (161/356) Updating files: 46% (164/356) Updating files: 47% (168/356) Updating files: 48% (171/356) Updating files: 49% (175/356) Updating files: 50% (178/356) Updating files: 51% (182/356) Updating files: 52% (186/356) Updating files: 53% (189/356) Updating files: 54% (193/356) Updating files: 55% (196/356) Updating files: 56% (200/356) Updating files: 57% (203/356) Updating files: 58% (207/356) Updating files: 59% (211/356) Updating files: 60% (214/356) Updating files: 61% (218/356) Updating files: 62% (221/356) Updating files: 63% (225/356) Updating files: 64% (228/356) Updating files: 65% (232/356) Updating files: 66% (235/356) Updating files: 67% (239/356) Updating files: 68% (243/356) Updating files: 69% (246/356) Updating files: 70% (250/356) Updating files: 71% (253/356) Updating files: 72% (257/356) Updating files: 73% (260/356) Updating files: 74% (264/356) Updating files: 74% (266/356) Updating files: 75% (267/356) Updating files: 76% (271/356) Updating files: 77% (275/356) Updating files: 78% (278/356) Updating files: 79% (282/356) Updating files: 80% (285/356) Updating files: 81% (289/356) Updating files: 82% (292/356) Updating files: 83% (296/356) Updating files: 84% (300/356) Updating files: 85% (303/356) Updating files: 86% (307/356) Updating files: 87% (310/356) Updating files: 87% (312/356) Updating files: 88% (314/356) Updating files: 89% (317/356) Updating files: 90% (321/356) Updating files: 91% (324/356) Updating files: 92% (328/356) Updating files: 93% (332/356) Updating files: 94% (335/356) Updating files: 95% (339/356) Updating files: 96% (342/356) Updating files: 97% (346/356) Updating files: 98% (349/356) Updating files: 99% (353/356) Updating files: 100% (356/356) Updating files: 100% (356/356), done. 2024-03-09T09:16:13.510 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 8336f81c5c7 0.84 2024-03-09T09:16:13.525 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.84 2024-03-09T09:16:13.664 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:16:13.664 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 85 -eq 48 ']' 2024-03-09T09:16:13.664 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.85 2024-03-09T09:16:13.664 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.85 2024-03-09T09:16:13.664 INFO:tasks.workunit.client.0.smithi067.stdout:v0.85 2024-03-09T09:16:13.664 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.85 2024-03-09T09:16:14.227 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 11% (28/243) Updating files: 12% (30/243) Updating files: 13% (32/243) Updating files: 14% (35/243) Updating files: 15% (37/243) Updating files: 16% (39/243) Updating files: 17% (42/243) Updating files: 18% (44/243) Updating files: 19% (47/243) Updating files: 20% (49/243) Updating files: 21% (52/243) Updating files: 22% (54/243) Updating files: 23% (56/243) Updating files: 24% (59/243) Updating files: 25% (61/243) Updating files: 26% (64/243) Updating files: 27% (66/243) Updating files: 28% (69/243) Updating files: 29% (71/243) Updating files: 30% (73/243) Updating files: 31% (76/243) Updating files: 32% (78/243) Updating files: 33% (81/243) Updating files: 34% (83/243) Updating files: 35% (86/243) Updating files: 36% (88/243) Updating files: 37% (90/243) Updating files: 38% (93/243) Updating files: 39% (95/243) Updating files: 40% (98/243) Updating files: 41% (100/243) Updating files: 42% (103/243) Updating files: 43% (105/243) Updating files: 44% (107/243) Updating files: 45% (110/243) Updating files: 46% (112/243) Updating files: 47% (115/243) Updating files: 48% (117/243) Updating files: 49% (120/243) Updating files: 50% (122/243) Updating files: 51% (124/243) Updating files: 52% (127/243) Updating files: 53% (129/243) Updating files: 54% (132/243) Updating files: 55% (134/243) Updating files: 56% (137/243) Updating files: 57% (139/243) Updating files: 58% (141/243) Updating files: 59% (144/243) Updating files: 60% (146/243) Updating files: 61% (149/243) Updating files: 62% (151/243) Updating files: 63% (154/243) Updating files: 64% (156/243) Updating files: 65% (158/243) Updating files: 66% (161/243) Updating files: 67% (163/243) Updating files: 68% (166/243) Updating files: 69% (168/243) Updating files: 70% (171/243) Updating files: 71% (173/243) Updating files: 72% (175/243) Updating files: 73% (178/243) Updating files: 74% (180/243) Updating files: 75% (183/243) Updating files: 76% (185/243) Updating files: 77% (188/243) Updating files: 78% (190/243) Updating files: 79% (192/243) Updating files: 80% (195/243) Updating files: 81% (197/243) Updating files: 82% (200/243) Updating files: 83% (202/243) Updating files: 83% (204/243) Updating files: 84% (205/243) Updating files: 85% (207/243) Updating files: 86% (209/243) Updating files: 87% (212/243) Updating files: 88% (214/243) Updating files: 89% (217/243) Updating files: 90% (219/243) Updating files: 91% (222/243) Updating files: 92% (224/243) Updating files: 93% (226/243) Updating files: 94% (229/243) Updating files: 95% (231/243) Updating files: 96% (234/243) Updating files: 97% (236/243) Updating files: 98% (239/243) Updating files: 99% (241/243) Updating files: 100% (243/243) Updating files: 100% (243/243), done. 2024-03-09T09:16:14.314 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at a0c22842db9 0.85 2024-03-09T09:16:14.321 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.85 2024-03-09T09:16:14.439 INFO:tasks.workunit.client.1.smithi134.stdout:v0.86 2024-03-09T09:16:14.439 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:16:14.439 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 86 -eq 48 ']' 2024-03-09T09:16:14.439 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.86 2024-03-09T09:16:14.439 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.86 2024-03-09T09:16:14.439 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.86 2024-03-09T09:16:16.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:16.360 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 50% (123/243) Updating files: 51% (124/243) Updating files: 52% (127/243) Updating files: 53% (129/243) Updating files: 54% (132/243) Updating files: 55% (134/243) Updating files: 56% (137/243) Updating files: 57% (139/243) Updating files: 58% (141/243) Updating files: 59% (144/243) Updating files: 60% (146/243) Updating files: 61% (149/243) Updating files: 62% (151/243) Updating files: 63% (154/243) Updating files: 64% (156/243) Updating files: 65% (158/243) Updating files: 66% (161/243) Updating files: 67% (163/243) Updating files: 68% (166/243) Updating files: 69% (168/243) Updating files: 70% (171/243) Updating files: 71% (173/243) Updating files: 72% (175/243) Updating files: 72% (177/243) Updating files: 73% (178/243) Updating files: 74% (180/243) Updating files: 75% (183/243) Updating files: 76% (185/243) Updating files: 77% (188/243) Updating files: 78% (190/243) Updating files: 79% (192/243) Updating files: 80% (195/243) Updating files: 81% (197/243) Updating files: 82% (200/243) Updating files: 83% (202/243) Updating files: 84% (205/243) Updating files: 85% (207/243) Updating files: 86% (209/243) Updating files: 87% (212/243) Updating files: 88% (214/243) Updating files: 89% (217/243) Updating files: 90% (219/243) Updating files: 91% (222/243) Updating files: 92% (224/243) Updating files: 93% (226/243) Updating files: 94% (229/243) Updating files: 95% (231/243) Updating files: 96% (234/243) Updating files: 97% (236/243) Updating files: 98% (239/243) Updating files: 99% (241/243) Updating files: 100% (243/243) Updating files: 100% (243/243), done. 2024-03-09T09:16:16.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.383+0000 7f8432800640 1 Processor -- start 2024-03-09T09:16:16.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.383+0000 7f8432800640 1 -- start start 2024-03-09T09:16:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.385+0000 7f8432800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 0x7f842c152810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.385+0000 7f8432800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f842c06d600 0x7f842c0c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.385+0000 7f8432800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c152d50 con 0x7f842c153280 2024-03-09T09:16:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.385+0000 7f8432800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c0c1ef0 con 0x7f842c06d600 2024-03-09T09:16:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.385+0000 7f8432800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0c21d0 con 0x7f842c0c7e90 2024-03-09T09:16:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.385+0000 7f842aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f842c06d600 0x7f842c0c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.385+0000 7f842be00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f842c153280 0x7f842c153710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:46788/0 (socket says 172.21.15.67:46788) 2024-03-09T09:16:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.385+0000 7f842be00640 1 -- 172.21.15.67:0/3235244103 learned_addr learned my addr 172.21.15.67:0/3235244103 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:16:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.385+0000 7f842b400640 1 --2- 172.21.15.67:0/3235244103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 0x7f842c152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.386+0000 7f842aa00640 1 -- 172.21.15.67:0/3235244103 >> v2:172.21.15.67:3300/0 conn(0x7f842c06d600 msgr2=0x7f842c0c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:16:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.386+0000 7f842aa00640 1 -- 172.21.15.67:0/3235244103 >> v2:172.21.15.67:3300/0 conn(0x7f842c06d600 msgr2=0x7f842c0c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:16:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.386+0000 7f842aa00640 1 --2- 172.21.15.67:0/3235244103 >> v2:172.21.15.67:3300/0 conn(0x7f842c06d600 0x7f842c0c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.386+0000 7f842aa00640 1 --2- 172.21.15.67:0/3235244103 >> v2:172.21.15.67:3300/0 conn(0x7f842c06d600 0x7f842c0c3590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.386+0000 7f842b400640 1 -- 172.21.15.67:0/3235244103 >> v2:172.21.15.67:3300/0 conn(0x7f842c06d600 msgr2=0x7f842c0c3590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:16.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.386+0000 7f842b400640 1 --2- 172.21.15.67:0/3235244103 >> v2:172.21.15.67:3300/0 conn(0x7f842c06d600 0x7f842c0c3590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:16.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.386+0000 7f842b400640 1 -- 172.21.15.67:0/3235244103 >> v1:172.21.15.134:6789/0 conn(0x7f842c153280 legacy=0x7f842c153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:16.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.386+0000 7f842b400640 1 -- 172.21.15.67:0/3235244103 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0c24b0 con 0x7f842c0c7e90 2024-03-09T09:16:16.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.388+0000 7f842b400640 1 --2- 172.21.15.67:0/3235244103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 0x7f842c152810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f841c0097a0 tx=0x7f841c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b966a94520519f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:16.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.388+0000 7f842a000640 1 -- 172.21.15.67:0/3235244103 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c031780 con 0x7f842c0c7e90 2024-03-09T09:16:16.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.388+0000 7f842a000640 1 -- 172.21.15.67:0/3235244103 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841c037070 con 0x7f842c0c7e90 2024-03-09T09:16:16.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.389+0000 7f842a000640 1 -- 172.21.15.67:0/3235244103 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c031b40 con 0x7f842c0c7e90 2024-03-09T09:16:16.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.389+0000 7f8432800640 1 -- 172.21.15.67:0/3235244103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 msgr2=0x7f842c152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.389+0000 7f8432800640 1 --2- 172.21.15.67:0/3235244103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 0x7f842c152810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f841c0097a0 tx=0x7f841c02b820 comp rx=0 tx=0).stop 2024-03-09T09:16:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.390+0000 7f8432800640 1 -- 172.21.15.67:0/3235244103 shutdown_connections 2024-03-09T09:16:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.390+0000 7f8432800640 1 --2- 172.21.15.67:0/3235244103 >> v2:172.21.15.67:3300/0 conn(0x7f842c06d600 0x7f842c0c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.390+0000 7f8432800640 1 --2- 172.21.15.67:0/3235244103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 0x7f842c152810 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.390+0000 7f8432800640 1 -- 172.21.15.67:0/3235244103 >> 172.21.15.67:0/3235244103 conn(0x7f842c0c7370 msgr2=0x7f842c0c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.390+0000 7f8432800640 1 -- 172.21.15.67:0/3235244103 shutdown_connections 2024-03-09T09:16:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.390+0000 7f8432800640 1 -- 172.21.15.67:0/3235244103 wait complete. 2024-03-09T09:16:16.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.392+0000 7f8432800640 1 Processor -- start 2024-03-09T09:16:16.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.392+0000 7f8432800640 1 -- start start 2024-03-09T09:16:16.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.393+0000 7f8432800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f842c06d600 0x7f842c141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:16.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.393+0000 7f8432800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 0x7f842c0c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:16.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.393+0000 7f8432800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c153280 0x7f842c153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:16.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.393+0000 7f8432800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0c3ad0 con 0x7f842c06d600 2024-03-09T09:16:16.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.393+0000 7f8432800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0c2a70 con 0x7f842c153280 2024-03-09T09:16:16.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.393+0000 7f8432800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0c2f90 con 0x7f842c0c7e90 2024-03-09T09:16:16.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.394+0000 7f842be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c153280 0x7f842c153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:16.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.394+0000 7f842be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c153280 0x7f842c153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:51900/0 (socket says 172.21.15.67:51900) 2024-03-09T09:16:16.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.394+0000 7f842be00640 1 -- 172.21.15.67:0/1202763281 learned_addr learned my addr 172.21.15.67:0/1202763281 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:16.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.394+0000 7f842aa00640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 0x7f842c0c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:16.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.394+0000 7f842b400640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f842c06d600 0x7f842c141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:16.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.395+0000 7f842aa00640 1 -- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c153280 msgr2=0x7f842c153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:16.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.395+0000 7f842aa00640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c153280 0x7f842c153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:16.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.395+0000 7f842aa00640 1 -- 172.21.15.67:0/1202763281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f842c06d600 msgr2=0x7f842c141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:16.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.395+0000 7f842aa00640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f842c06d600 0x7f842c141990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:16.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.395+0000 7f842aa00640 1 -- 172.21.15.67:0/1202763281 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c1424e0 con 0x7f842c0c7e90 2024-03-09T09:16:16.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.395+0000 7f842aa00640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 0x7f842c0c64a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f841400d5a0 tx=0x7f841400da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:16.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.396+0000 7f8423400640 1 -- 172.21.15.67:0/1202763281 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84140125f0 con 0x7f842c0c7e90 2024-03-09T09:16:16.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.396+0000 7f8423400640 1 -- 172.21.15.67:0/1202763281 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8414019070 con 0x7f842c0c7e90 2024-03-09T09:16:16.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.396+0000 7f8423400640 1 -- 172.21.15.67:0/1202763281 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84140129d0 con 0x7f842c0c7e90 2024-03-09T09:16:16.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.396+0000 7f8432800640 1 -- 172.21.15.67:0/1202763281 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c1427a0 con 0x7f842c0c7e90 2024-03-09T09:16:16.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.396+0000 7f8432800640 1 -- 172.21.15.67:0/1202763281 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c140700 con 0x7f842c0c7e90 2024-03-09T09:16:16.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.398+0000 7f8432800640 1 -- 172.21.15.67:0/1202763281 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8404005290 con 0x7f842c0c7e90 2024-03-09T09:16:16.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.399+0000 7f8423400640 1 -- 172.21.15.67:0/1202763281 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8414012b70 con 0x7f842c0c7e90 2024-03-09T09:16:16.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.401+0000 7f8423400640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f840806b520 0x7f840806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:16.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.401+0000 7f8423400640 1 -- 172.21.15.67:0/1202763281 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f841408a7b0 con 0x7f842c0c7e90 2024-03-09T09:16:16.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.401+0000 7f842b400640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f840806b520 0x7f840806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:16.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.402+0000 7f842b400640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f840806b520 0x7f840806d9e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f841c036000 tx=0x7f841c03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:16.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.408+0000 7f8423400640 1 -- 172.21.15.67:0/1202763281 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8414059320 con 0x7f842c0c7e90 2024-03-09T09:16:16.416 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at a0c22842db9 0.85 2024-03-09T09:16:16.431 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.85 2024-03-09T09:16:16.545 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:16:16.545 INFO:tasks.workunit.client.0.smithi067.stdout:v0.86 2024-03-09T09:16:16.545 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 86 -eq 48 ']' 2024-03-09T09:16:16.546 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.86 2024-03-09T09:16:16.546 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.86 2024-03-09T09:16:16.546 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.86 2024-03-09T09:16:16.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.755+0000 7f8432800640 1 -- 172.21.15.67:0/1202763281 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8404005850 con 0x7f842c0c7e90 2024-03-09T09:16:16.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.756+0000 7f8423400640 1 -- 172.21.15.67:0/1202763281 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f8414058c20 con 0x7f842c0c7e90 2024-03-09T09:16:16.762 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":176,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":40,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:16.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.758+0000 7f8432800640 1 -- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f840806b520 msgr2=0x7f840806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:16.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.758+0000 7f8432800640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f840806b520 0x7f840806d9e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f841c036000 tx=0x7f841c03d040 comp rx=0 tx=0).stop 2024-03-09T09:16:16.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.758+0000 7f8432800640 1 -- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 msgr2=0x7f842c0c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:16.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.758+0000 7f8432800640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 0x7f842c0c64a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f841400d5a0 tx=0x7f841400da60 comp rx=0 tx=0).stop 2024-03-09T09:16:16.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.759+0000 7f8432800640 1 -- 172.21.15.67:0/1202763281 shutdown_connections 2024-03-09T09:16:16.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.759+0000 7f8432800640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f840806b520 0x7f840806d9e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:16.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.759+0000 7f8432800640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c153280 0x7f842c153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:16.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.759+0000 7f8432800640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f842c0c7e90 0x7f842c0c64a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:16.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.759+0000 7f8432800640 1 --2- 172.21.15.67:0/1202763281 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f842c06d600 0x7f842c141990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:16.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.759+0000 7f8432800640 1 -- 172.21.15.67:0/1202763281 >> 172.21.15.67:0/1202763281 conn(0x7f842c0c7370 msgr2=0x7f842c14ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:16.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.759+0000 7f8432800640 1 -- 172.21.15.67:0/1202763281 shutdown_connections 2024-03-09T09:16:16.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:16.759+0000 7f8432800640 1 -- 172.21.15.67:0/1202763281 wait complete. 2024-03-09T09:16:19.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.990+0000 7fe14e600640 1 Processor -- start 2024-03-09T09:16:19.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.990+0000 7fe14e600640 1 -- start start 2024-03-09T09:16:19.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.991+0000 7fe14e600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe1480ac730 0x7fe148071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:19.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.992+0000 7fe14e600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1481581a0 con 0x7fe1480c22a0 2024-03-09T09:16:19.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.992+0000 7fe14e600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe148153d40 con 0x7fe148157d60 2024-03-09T09:16:19.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.992+0000 7fe14e600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe148070ee0 con 0x7fe1480ac730 2024-03-09T09:16:19.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.992+0000 7fe13fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe1480c22a0 0x7fe148157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43826/0 (socket says 172.21.15.67:43826) 2024-03-09T09:16:19.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.992+0000 7fe13fe00640 1 -- 172.21.15.67:0/1865176478 learned_addr learned my addr 172.21.15.67:0/1865176478 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:16:19.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.992+0000 7fe147400640 1 --2- 172.21.15.67:0/1865176478 >> v2:172.21.15.134:3300/0 conn(0x7fe1480ac730 0x7fe148071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:19.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.993+0000 7fe147400640 1 -- 172.21.15.67:0/1865176478 >> v1:172.21.15.134:6789/0 conn(0x7fe148157d60 legacy=0x7fe14815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:19.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.993+0000 7fe147400640 1 -- 172.21.15.67:0/1865176478 >> v1:172.21.15.67:6789/0 conn(0x7fe1480c22a0 legacy=0x7fe148157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:19.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.993+0000 7fe147400640 1 -- 172.21.15.67:0/1865176478 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480c2790 con 0x7fe1480ac730 2024-03-09T09:16:19.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.994+0000 7fe147400640 1 --2- 172.21.15.67:0/1865176478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1480ac730 0x7fe148071190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1380097a0 tx=0x7fe13802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893bcc76c4ec1610 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.994+0000 7fe146a00640 1 -- 172.21.15.67:0/1865176478 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138035020 con 0x7fe1480ac730 2024-03-09T09:16:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.995+0000 7fe146a00640 1 -- 172.21.15.67:0/1865176478 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138031830 con 0x7fe1480ac730 2024-03-09T09:16:20.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.995+0000 7fe146a00640 1 -- 172.21.15.67:0/1865176478 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138031c00 con 0x7fe1480ac730 2024-03-09T09:16:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.995+0000 7fe14e600640 1 -- 172.21.15.67:0/1865176478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1480ac730 msgr2=0x7fe148071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.995+0000 7fe14e600640 1 --2- 172.21.15.67:0/1865176478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1480ac730 0x7fe148071190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1380097a0 tx=0x7fe13802b8d0 comp rx=0 tx=0).stop 2024-03-09T09:16:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.995+0000 7fe14e600640 1 -- 172.21.15.67:0/1865176478 shutdown_connections 2024-03-09T09:16:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.995+0000 7fe14e600640 1 --2- 172.21.15.67:0/1865176478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1480ac730 0x7fe148071190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.995+0000 7fe14e600640 1 -- 172.21.15.67:0/1865176478 >> 172.21.15.67:0/1865176478 conn(0x7fe1480c7370 msgr2=0x7fe1480c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.995+0000 7fe14e600640 1 -- 172.21.15.67:0/1865176478 shutdown_connections 2024-03-09T09:16:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.995+0000 7fe14e600640 1 -- 172.21.15.67:0/1865176478 wait complete. 2024-03-09T09:16:20.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.997+0000 7fe14e600640 1 Processor -- start 2024-03-09T09:16:20.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.997+0000 7fe14e600640 1 -- start start 2024-03-09T09:16:20.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe14e600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1480ac730 0x7fe1480c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:20.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe14e600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1480c22a0 0x7fe1480ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:20.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe14e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe148157d60 0x7fe1481ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:20.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe14e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe148070c00 con 0x7fe148157d60 2024-03-09T09:16:20.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe14e600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe148154d00 con 0x7fe1480c22a0 2024-03-09T09:16:20.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe14e600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe148151610 con 0x7fe1480ac730 2024-03-09T09:16:20.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe147e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe148157d60 0x7fe1481ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:20.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe147e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe148157d60 0x7fe1481ebc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56748/0 (socket says 172.21.15.67:56748) 2024-03-09T09:16:20.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe147e00640 1 -- 172.21.15.67:0/1872429453 learned_addr learned my addr 172.21.15.67:0/1872429453 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:20.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe147e00640 1 -- 172.21.15.67:0/1872429453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe148157d60 msgr2=0x7fe1481ebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:16:20.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe147e00640 1 -- 172.21.15.67:0/1872429453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe148157d60 msgr2=0x7fe1481ebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:16:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe147e00640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe148157d60 0x7fe1481ebc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe13fe00640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1480c22a0 0x7fe1480ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.998+0000 7fe147e00640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe148157d60 0x7fe1481ebc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.999+0000 7fe147400640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1480ac730 0x7fe1480c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.999+0000 7fe13fe00640 1 -- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1480ac730 msgr2=0x7fe1480c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.999+0000 7fe13fe00640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1480ac730 0x7fe1480c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.999+0000 7fe13fe00640 1 -- 172.21.15.67:0/1872429453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe148157d60 msgr2=0x7fe1481ebc90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:16:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.999+0000 7fe13fe00640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe148157d60 0x7fe1481ebc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:20.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:19.999+0000 7fe13fe00640 1 -- 172.21.15.67:0/1872429453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480ac370 con 0x7fe1480c22a0 2024-03-09T09:16:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.000+0000 7fe13fe00640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1480c22a0 0x7fe1480ab190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe13000c410 tx=0x7fe130001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.000+0000 7fe13f400640 1 -- 172.21.15.67:0/1872429453 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130017070 con 0x7fe1480c22a0 2024-03-09T09:16:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.000+0000 7fe13f400640 1 -- 172.21.15.67:0/1872429453 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe130005ce0 con 0x7fe1480c22a0 2024-03-09T09:16:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.000+0000 7fe14e600640 1 -- 172.21.15.67:0/1872429453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1481ef450 con 0x7fe1480c22a0 2024-03-09T09:16:20.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.000+0000 7fe13f400640 1 -- 172.21.15.67:0/1872429453 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300072d0 con 0x7fe1480c22a0 2024-03-09T09:16:20.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.000+0000 7fe14e600640 1 -- 172.21.15.67:0/1872429453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1481efa10 con 0x7fe1480c22a0 2024-03-09T09:16:20.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.001+0000 7fe14e600640 1 -- 172.21.15.67:0/1872429453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe148068d50 con 0x7fe1480c22a0 2024-03-09T09:16:20.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.003+0000 7fe13f400640 1 -- 172.21.15.67:0/1872429453 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe13001a070 con 0x7fe1480c22a0 2024-03-09T09:16:20.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.004+0000 7fe13f400640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12406b500 0x7fe12406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:20.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.004+0000 7fe13f400640 1 -- 172.21.15.67:0/1872429453 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe130089c70 con 0x7fe1480c22a0 2024-03-09T09:16:20.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.004+0000 7fe147400640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12406b500 0x7fe12406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:20.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.005+0000 7fe147400640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12406b500 0x7fe12406d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe138036000 tx=0x7fe13803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:20.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.016+0000 7fe13f400640 1 -- 172.21.15.67:0/1872429453 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe148068d50 con 0x7fe1480c22a0 2024-03-09T09:16:20.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.424+0000 7fe14e600640 1 -- 172.21.15.67:0/1872429453 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe148075a00 con 0x7fe1480c22a0 2024-03-09T09:16:20.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.424+0000 7fe13f400640 1 -- 172.21.15.67:0/1872429453 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fe130058120 con 0x7fe1480c22a0 2024-03-09T09:16:20.433 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":176,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":44,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:20.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.428+0000 7fe14e600640 1 -- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12406b500 msgr2=0x7fe12406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:20.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.428+0000 7fe14e600640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12406b500 0x7fe12406d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe138036000 tx=0x7fe13803c040 comp rx=0 tx=0).stop 2024-03-09T09:16:20.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.428+0000 7fe14e600640 1 -- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1480c22a0 msgr2=0x7fe1480ab190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:20.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.428+0000 7fe14e600640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1480c22a0 0x7fe1480ab190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe13000c410 tx=0x7fe130001e70 comp rx=0 tx=0).stop 2024-03-09T09:16:20.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.429+0000 7fe14e600640 1 -- 172.21.15.67:0/1872429453 shutdown_connections 2024-03-09T09:16:20.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.429+0000 7fe14e600640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe12406b500 0x7fe12406d9c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:20.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.429+0000 7fe14e600640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe148157d60 0x7fe1481ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:20.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.429+0000 7fe14e600640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1480c22a0 0x7fe1480ab190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:20.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.429+0000 7fe14e600640 1 --2- 172.21.15.67:0/1872429453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1480ac730 0x7fe1480c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:20.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.429+0000 7fe14e600640 1 -- 172.21.15.67:0/1872429453 >> 172.21.15.67:0/1872429453 conn(0x7fe1480c7370 msgr2=0x7fe1480c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:20.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.429+0000 7fe14e600640 1 -- 172.21.15.67:0/1872429453 shutdown_connections 2024-03-09T09:16:20.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:20.429+0000 7fe14e600640 1 -- 172.21.15.67:0/1872429453 wait complete. 2024-03-09T09:16:23.460 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:23.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.667+0000 7feca7e00640 1 Processor -- start 2024-03-09T09:16:23.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.668+0000 7feca7e00640 1 -- start start 2024-03-09T09:16:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feca00ac730 0x7feca0071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca7e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7feca00c22a0 0x7feca00acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca7e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca01581a0 con 0x7feca0157650 2024-03-09T09:16:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca0070ee0 con 0x7feca00ac730 2024-03-09T09:16:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca7e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca0154d00 con 0x7feca00c22a0 2024-03-09T09:16:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feca00ac730 0x7feca0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feca00ac730 0x7feca0071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56762/0 (socket says 172.21.15.67:56762) 2024-03-09T09:16:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca5400640 1 -- 172.21.15.67:0/2015691570 learned_addr learned my addr 172.21.15.67:0/2015691570 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca4a00640 1 --2- 172.21.15.67:0/2015691570 >> v2:172.21.15.134:3300/0 conn(0x7feca00c22a0 0x7feca00acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca5400640 1 -- 172.21.15.67:0/2015691570 >> v2:172.21.15.134:3300/0 conn(0x7feca00c22a0 msgr2=0x7feca00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca5400640 1 --2- 172.21.15.67:0/2015691570 >> v2:172.21.15.134:3300/0 conn(0x7feca00c22a0 0x7feca00acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca5400640 1 -- 172.21.15.67:0/2015691570 >> v1:172.21.15.134:6789/0 conn(0x7feca0157650 legacy=0x7feca0157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.669+0000 7feca5400640 1 -- 172.21.15.67:0/2015691570 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca0153d40 con 0x7feca00ac730 2024-03-09T09:16:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.670+0000 7feca5400640 1 --2- 172.21.15.67:0/2015691570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feca00ac730 0x7feca0071190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feca00c1a30 tx=0x7fec9402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3ae5490c5e76f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.671+0000 7fec9fe00640 1 -- 172.21.15.67:0/2015691570 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94035020 con 0x7feca00ac730 2024-03-09T09:16:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.671+0000 7fec9fe00640 1 -- 172.21.15.67:0/2015691570 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec94031870 con 0x7feca00ac730 2024-03-09T09:16:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.671+0000 7fec9fe00640 1 -- 172.21.15.67:0/2015691570 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94031c40 con 0x7feca00ac730 2024-03-09T09:16:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.671+0000 7feca7e00640 1 -- 172.21.15.67:0/2015691570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feca00ac730 msgr2=0x7feca0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.671+0000 7feca7e00640 1 --2- 172.21.15.67:0/2015691570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feca00ac730 0x7feca0071190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feca00c1a30 tx=0x7fec9402b910 comp rx=0 tx=0).stop 2024-03-09T09:16:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.671+0000 7feca7e00640 1 -- 172.21.15.67:0/2015691570 shutdown_connections 2024-03-09T09:16:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.671+0000 7feca7e00640 1 --2- 172.21.15.67:0/2015691570 >> v2:172.21.15.134:3300/0 conn(0x7feca00c22a0 0x7feca00acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.671+0000 7feca7e00640 1 --2- 172.21.15.67:0/2015691570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feca00ac730 0x7feca0071190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.671+0000 7feca7e00640 1 -- 172.21.15.67:0/2015691570 >> 172.21.15.67:0/2015691570 conn(0x7feca00c7370 msgr2=0x7feca00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.672+0000 7feca7e00640 1 -- 172.21.15.67:0/2015691570 shutdown_connections 2024-03-09T09:16:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.672+0000 7feca7e00640 1 -- 172.21.15.67:0/2015691570 wait complete. 2024-03-09T09:16:23.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.673+0000 7feca7e00640 1 Processor -- start 2024-03-09T09:16:23.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.673+0000 7feca7e00640 1 -- start start 2024-03-09T09:16:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.674+0000 7feca7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feca00ac730 0x7feca01efff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.674+0000 7feca7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feca00c22a0 0x7feca0156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.674+0000 7feca7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca0157650 0x7feca014c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.674+0000 7feca7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca0070c00 con 0x7feca00c22a0 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.674+0000 7feca7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca0070920 con 0x7feca0157650 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.674+0000 7feca7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca0151610 con 0x7feca00ac730 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feca00c22a0 0x7feca0156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feca00c22a0 0x7feca0156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56772/0 (socket says 172.21.15.67:56772) 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca4a00640 1 -- 172.21.15.67:0/75499725 learned_addr learned my addr 172.21.15.67:0/75499725 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca5400640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feca00ac730 0x7feca01efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca4a00640 1 -- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feca00ac730 msgr2=0x7feca01efff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca4a00640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feca00ac730 0x7feca01efff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca4a00640 1 -- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca0157650 msgr2=0x7feca014c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca5e00640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca0157650 0x7feca014c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca4a00640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca0157650 0x7feca014c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca4a00640 1 -- 172.21.15.67:0/75499725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca01f0530 con 0x7feca00c22a0 2024-03-09T09:16:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca5e00640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca0157650 0x7feca014c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:16:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.675+0000 7feca4a00640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feca00c22a0 0x7feca0156450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00d7d0 tx=0x7fec8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.676+0000 7fec9d600640 1 -- 172.21.15.67:0/75499725 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c00b6c0 con 0x7feca00c22a0 2024-03-09T09:16:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.676+0000 7fec9d600640 1 -- 172.21.15.67:0/75499725 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c019070 con 0x7feca00c22a0 2024-03-09T09:16:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.676+0000 7fec9d600640 1 -- 172.21.15.67:0/75499725 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c00ba80 con 0x7feca00c22a0 2024-03-09T09:16:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.676+0000 7feca7e00640 1 -- 172.21.15.67:0/75499725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca01e2160 con 0x7feca00c22a0 2024-03-09T09:16:23.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.676+0000 7feca7e00640 1 -- 172.21.15.67:0/75499725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca01dfb00 con 0x7feca00c22a0 2024-03-09T09:16:23.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.677+0000 7feca7e00640 1 -- 172.21.15.67:0/75499725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca00714f0 con 0x7feca00c22a0 2024-03-09T09:16:23.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.678+0000 7fec9d600640 1 -- 172.21.15.67:0/75499725 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fec8c00bc20 con 0x7feca00c22a0 2024-03-09T09:16:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.679+0000 7fec9d600640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fec8006b4d0 0x7fec8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.679+0000 7fec9d600640 1 -- 172.21.15.67:0/75499725 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fec8c08aa30 con 0x7feca00c22a0 2024-03-09T09:16:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.679+0000 7feca5400640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fec8006b4d0 0x7fec8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:23.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.680+0000 7feca5400640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fec8006b4d0 0x7fec8006d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fec94036000 tx=0x7fec94008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:23.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.684+0000 7fec9d600640 1 -- 172.21.15.67:0/75499725 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feca00714f0 con 0x7feca00c22a0 2024-03-09T09:16:23.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.967+0000 7feca7e00640 1 -- 172.21.15.67:0/75499725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7feca00c2c80 con 0x7feca00c22a0 2024-03-09T09:16:23.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.968+0000 7fec9d600640 1 -- 172.21.15.67:0/75499725 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7feca01f0530 con 0x7feca00c22a0 2024-03-09T09:16:23.973 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":180,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:23.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 -- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fec8006b4d0 msgr2=0x7fec8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:23.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fec8006b4d0 0x7fec8006d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fec94036000 tx=0x7fec94008030 comp rx=0 tx=0).stop 2024-03-09T09:16:23.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 -- 172.21.15.67:0/75499725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feca00c22a0 msgr2=0x7feca0156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:23.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feca00c22a0 0x7feca0156450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00d7d0 tx=0x7fec8c00dc90 comp rx=0 tx=0).stop 2024-03-09T09:16:23.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 -- 172.21.15.67:0/75499725 shutdown_connections 2024-03-09T09:16:23.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fec8006b4d0 0x7fec8006d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:23.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feca0157650 0x7feca014c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:23.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feca00c22a0 0x7feca0156450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:23.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 --2- 172.21.15.67:0/75499725 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feca00ac730 0x7feca01efff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:23.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 -- 172.21.15.67:0/75499725 >> 172.21.15.67:0/75499725 conn(0x7feca00c7370 msgr2=0x7feca0152100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:23.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 -- 172.21.15.67:0/75499725 shutdown_connections 2024-03-09T09:16:23.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:23.970+0000 7feca7e00640 1 -- 172.21.15.67:0/75499725 wait complete. 2024-03-09T09:16:23.992 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:16:23.992 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:16:23.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:16:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.196+0000 7f81e0600640 1 Processor -- start 2024-03-09T09:16:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.196+0000 7f81e0600640 1 -- start start 2024-03-09T09:16:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.197+0000 7f81e0600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81d8157890 0x7f81d806d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.197+0000 7f81e0600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f81d806e630 0x7f81d815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.197+0000 7f81e0600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d815a370 con 0x7f81d806dad0 2024-03-09T09:16:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.197+0000 7f81e0600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d815b130 con 0x7f81d806e630 2024-03-09T09:16:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.197+0000 7f81e0600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d815a650 con 0x7f81d8157890 2024-03-09T09:16:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.197+0000 7f81dd200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f81d806e630 0x7f81d815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.197+0000 7f81dd200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f81d806e630 0x7f81d815b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56778/0 (socket says 172.21.15.67:56778) 2024-03-09T09:16:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.197+0000 7f81dd200640 1 -- 172.21.15.67:0/1942651821 learned_addr learned my addr 172.21.15.67:0/1942651821 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.197+0000 7f81ddc00640 1 --2- 172.21.15.67:0/1942651821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81d8157890 0x7f81d806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.198+0000 7f81dd200640 1 -- 172.21.15.67:0/1942651821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81d8157890 msgr2=0x7f81d806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.198+0000 7f81dd200640 1 --2- 172.21.15.67:0/1942651821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81d8157890 0x7f81d806d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.198+0000 7f81dd200640 1 -- 172.21.15.67:0/1942651821 >> v1:172.21.15.134:6789/0 conn(0x7f81d806dad0 legacy=0x7f81d815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.198+0000 7f81dd200640 1 -- 172.21.15.67:0/1942651821 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d815a930 con 0x7f81d806e630 2024-03-09T09:16:24.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.198+0000 7f81ddc00640 1 --2- 172.21.15.67:0/1942651821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81d8157890 0x7f81d806d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:16:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.199+0000 7f81dd200640 1 --2- 172.21.15.67:0/1942651821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d806e630 0x7f81d815b5d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009870 tx=0x7f81c802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc140c5c2022bcf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.199+0000 7f81d7e00640 1 -- 172.21.15.67:0/1942651821 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c802f840 con 0x7f81d806e630 2024-03-09T09:16:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.199+0000 7f81d7e00640 1 -- 172.21.15.67:0/1942651821 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c802f9e0 con 0x7f81d806e630 2024-03-09T09:16:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.199+0000 7f81d7e00640 1 -- 172.21.15.67:0/1942651821 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c802fdb0 con 0x7f81d806e630 2024-03-09T09:16:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.199+0000 7f81e0600640 1 -- 172.21.15.67:0/1942651821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d806e630 msgr2=0x7f81d815b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.199+0000 7f81e0600640 1 --2- 172.21.15.67:0/1942651821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d806e630 0x7f81d815b5d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009870 tx=0x7f81c802ba20 comp rx=0 tx=0).stop 2024-03-09T09:16:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.200+0000 7f81e0600640 1 -- 172.21.15.67:0/1942651821 shutdown_connections 2024-03-09T09:16:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.200+0000 7f81e0600640 1 --2- 172.21.15.67:0/1942651821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d806e630 0x7f81d815b5d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.200+0000 7f81e0600640 1 --2- 172.21.15.67:0/1942651821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81d8157890 0x7f81d806d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.200+0000 7f81e0600640 1 -- 172.21.15.67:0/1942651821 >> 172.21.15.67:0/1942651821 conn(0x7f81d80c7370 msgr2=0x7f81d80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.200+0000 7f81e0600640 1 -- 172.21.15.67:0/1942651821 shutdown_connections 2024-03-09T09:16:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.200+0000 7f81e0600640 1 -- 172.21.15.67:0/1942651821 wait complete. 2024-03-09T09:16:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.201+0000 7f81e0600640 1 Processor -- start 2024-03-09T09:16:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.201+0000 7f81e0600640 1 -- start start 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.202+0000 7f81e0600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d806dad0 0x7f81d8154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.202+0000 7f81e0600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81d806e630 0x7f81d81f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.202+0000 7f81e0600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81d8157890 0x7f81d81f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.202+0000 7f81e0600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80c1ef0 con 0x7f81d806dad0 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.202+0000 7f81e0600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80c1d70 con 0x7f81d806e630 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.202+0000 7f81e0600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d8157c70 con 0x7f81d8157890 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.203+0000 7f81ddc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d806dad0 0x7f81d8154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.203+0000 7f81ddc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d806dad0 0x7f81d8154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56786/0 (socket says 172.21.15.67:56786) 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.203+0000 7f81ddc00640 1 -- 172.21.15.67:0/3930401373 learned_addr learned my addr 172.21.15.67:0/3930401373 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.203+0000 7f81ddc00640 1 -- 172.21.15.67:0/3930401373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81d8157890 msgr2=0x7f81d81f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.203+0000 7f81ddc00640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81d8157890 0x7f81d81f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.203+0000 7f81ddc00640 1 -- 172.21.15.67:0/3930401373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81d806e630 msgr2=0x7f81d81f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.203+0000 7f81ddc00640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81d806e630 0x7f81d81f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.203+0000 7f81ddc00640 1 -- 172.21.15.67:0/3930401373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d81f3eb0 con 0x7f81d806dad0 2024-03-09T09:16:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.203+0000 7f81ddc00640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d806dad0 0x7f81d8154400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00e7e0 tx=0x7f81cc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.203+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00f040 con 0x7f81d806dad0 2024-03-09T09:16:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.204+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc004030 con 0x7f81d806dad0 2024-03-09T09:16:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.204+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc004420 con 0x7f81d806dad0 2024-03-09T09:16:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.204+0000 7f81e0600640 1 -- 172.21.15.67:0/3930401373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d8154940 con 0x7f81d806dad0 2024-03-09T09:16:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.204+0000 7f81e0600640 1 -- 172.21.15.67:0/3930401373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d8153920 con 0x7f81d806dad0 2024-03-09T09:16:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.205+0000 7f81e0600640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d81f3720 0x7f81d8153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.205+0000 7f81e0600640 1 -- 172.21.15.67:0/3930401373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d81540a0 con 0x7f81d81f3720 2024-03-09T09:16:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.205+0000 7f81dd200640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d81f3720 0x7f81d8153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.205+0000 7f81dd200640 1 -- 172.21.15.67:0/3930401373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81d8075a00 con 0x7f81d81f3720 2024-03-09T09:16:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.206+0000 7f81dd200640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d81f3720 0x7f81d8153b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f81d8070710 tx=0x7f81c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.206+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8045020 con 0x7f81d81f3720 2024-03-09T09:16:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.206+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f81c8009490 con 0x7f81d81f3720 2024-03-09T09:16:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.206+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d81f3720 msgr2=0x7f81d8153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.206+0000 7f81d5600640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d81f3720 0x7f81d8153b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f81d8070710 tx=0x7f81c80046c0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.207+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d81f3720 msgr2=0x7f81d8153b60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.207+0000 7f81d5600640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d81f3720 0x7f81d8153b60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.207+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f81cc00c6d0 con 0x7f81d806dad0 2024-03-09T09:16:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.208+0000 7f81d5600640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f81bc06af40 0x7f81bc06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.208+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f81cc08a390 con 0x7f81d806dad0 2024-03-09T09:16:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.209+0000 7f81dd200640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f81bc06af40 0x7f81bc06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.210+0000 7f81dd200640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f81bc06af40 0x7f81bc06d400 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f81d80c5c80 tx=0x7f81c803e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.214+0000 7f81e0600640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d8152210 0x7f81d814f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.214+0000 7f81e0600640 1 -- 172.21.15.67:0/3930401373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80c2a40 con 0x7f81d8152210 2024-03-09T09:16:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.215+0000 7f81de600640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d8152210 0x7f81d814f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.215+0000 7f81de600640 1 -- 172.21.15.67:0/3930401373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f81d8073ba0 con 0x7f81d8152210 2024-03-09T09:16:24.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.215+0000 7f81de600640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d8152210 0x7f81d814f530 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c680 tx=0x7f81d000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0006d50 con 0x7f81d8152210 2024-03-09T09:16:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4536 (secure 0 0 0) 0x7f81d000ce80 con 0x7f81d8152210 2024-03-09T09:16:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d8152210 msgr2=0x7f81d814f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81d5600640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d8152210 0x7f81d814f530 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c680 tx=0x7f81d000cb40 comp rx=0 tx=0).stop 2024-03-09T09:16:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81d5600640 1 -- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d8152210 msgr2=0x7f81d814f530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81d5600640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d8152210 0x7f81d814f530 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c680 tx=0x7f81d000cb40 comp rx=0 tx=0).stop 2024-03-09T09:16:24.223 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:16:24.223 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:16:24.223 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:16:24.223 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:16:24.223 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 180, 2024-03-09T09:16:24.223 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:16:24.223 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:16:24.223 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:16:24.223 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:16:24.223 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.223 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:16:24.224 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:24.224 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:16:24.224 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:16:24.224 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:24.224 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:24.224 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:24.224 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:24.224 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:24.224 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:24.224 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:24.224 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:24.225 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:24.225 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:24.225 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:24.226 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:24.227 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:24.227 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:24.227 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.227 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:16:24.227 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:16:24.227 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:16:24.227 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:16:24.227 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:16:24.227 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:16:24.227 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:24.228 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:24.229 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:24.229 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:24.229 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:24.229 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:24.229 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:24.229 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:24.229 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:24.229 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:24.229 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:24.229 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.229 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:16:24.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.230 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:16:24.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.230 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:16:24.230 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:16:24.230 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:24.230 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:24.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.230 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:16:24.231 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:16:24.232 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:24.232 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:24.232 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:24.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.232 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:16:24.232 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:16:24.232 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:24.232 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:24.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.232 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:24.233 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:16:24.233 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.233 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:24.233 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:16:24.233 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.233 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.233 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:24.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.235 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:24.235 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:16:24.235 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.235 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:24.235 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:16:24.235 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.235 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:16:24.236 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:16:24.237 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.238 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.238 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:16:24.238 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:16:24.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81e0600640 1 -- 172.21.15.67:0/3930401373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f81bc06af40 msgr2=0x7f81bc06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81e0600640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f81bc06af40 0x7f81bc06d400 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f81d80c5c80 tx=0x7f81c803e000 comp rx=0 tx=0).stop 2024-03-09T09:16:24.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81e0600640 1 -- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d806dad0 msgr2=0x7f81d8154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81e0600640 1 --2- 172.21.15.67:0/3930401373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f81d806dad0 0x7f81d8154400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00e7e0 tx=0x7f81cc00eca0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81de600640 1 -- 172.21.15.67:0/3930401373 reap_dead start 2024-03-09T09:16:24.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81de600640 1 -- 172.21.15.67:0/3930401373 reap_dead start 2024-03-09T09:16:24.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81e0600640 1 -- 172.21.15.67:0/3930401373 shutdown_connections 2024-03-09T09:16:24.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.216+0000 7f81e0600640 1 -- 172.21.15.67:0/3930401373 >> 172.21.15.67:0/3930401373 conn(0x7f81d80c7370 msgr2=0x7f81d80c6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:24.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.217+0000 7f81e0600640 1 -- 172.21.15.67:0/3930401373 shutdown_connections 2024-03-09T09:16:24.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.217+0000 7f81e0600640 1 -- 172.21.15.67:0/3930401373 wait complete. 2024-03-09T09:16:24.241 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:16:24.241 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:16:24.241 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:16:24.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.448+0000 7fc4ab600640 1 Processor -- start 2024-03-09T09:16:24.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.449+0000 7fc4ab600640 1 -- start start 2024-03-09T09:16:24.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4ab600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4a40c2780 0x7fc4a4154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4ab600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc4a4071190 0x7fc4a41546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4ab600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a40c8200 con 0x7fc4a40ac9a0 2024-03-09T09:16:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4ab600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a406d590 con 0x7fc4a40c2780 2024-03-09T09:16:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4ab600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a40c3740 con 0x7fc4a4071190 2024-03-09T09:16:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4a3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4a40c2780 0x7fc4a4154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4a3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc4a40c2780 0x7fc4a4154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56814/0 (socket says 172.21.15.67:56814) 2024-03-09T09:16:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4a3e00640 1 -- 172.21.15.67:0/383710129 learned_addr learned my addr 172.21.15.67:0/383710129 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4a9600640 1 --2- 172.21.15.67:0/383710129 >> v2:172.21.15.134:3300/0 conn(0x7fc4a4071190 0x7fc4a41546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4a3e00640 1 -- 172.21.15.67:0/383710129 >> v2:172.21.15.134:3300/0 conn(0x7fc4a4071190 msgr2=0x7fc4a41546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4a3e00640 1 --2- 172.21.15.67:0/383710129 >> v2:172.21.15.134:3300/0 conn(0x7fc4a4071190 0x7fc4a41546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4a3e00640 1 -- 172.21.15.67:0/383710129 >> v1:172.21.15.134:6789/0 conn(0x7fc4a40ac9a0 legacy=0x7fc4a41510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.450+0000 7fc4a3e00640 1 -- 172.21.15.67:0/383710129 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a40c3990 con 0x7fc4a40c2780 2024-03-09T09:16:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.451+0000 7fc4a3e00640 1 --2- 172.21.15.67:0/383710129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4a40c2780 0x7fc4a4154190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40c3190 tx=0x7fc49002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8359dc6af3f3aab server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.452+0000 7fc4a3400640 1 -- 172.21.15.67:0/383710129 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490035020 con 0x7fc4a40c2780 2024-03-09T09:16:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.452+0000 7fc4a3400640 1 -- 172.21.15.67:0/383710129 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc490031870 con 0x7fc4a40c2780 2024-03-09T09:16:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.452+0000 7fc4a3400640 1 -- 172.21.15.67:0/383710129 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490031c40 con 0x7fc4a40c2780 2024-03-09T09:16:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.452+0000 7fc4ab600640 1 -- 172.21.15.67:0/383710129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4a40c2780 msgr2=0x7fc4a4154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.452+0000 7fc4ab600640 1 --2- 172.21.15.67:0/383710129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4a40c2780 0x7fc4a4154190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40c3190 tx=0x7fc49002b910 comp rx=0 tx=0).stop 2024-03-09T09:16:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.452+0000 7fc4ab600640 1 -- 172.21.15.67:0/383710129 shutdown_connections 2024-03-09T09:16:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.452+0000 7fc4ab600640 1 --2- 172.21.15.67:0/383710129 >> v2:172.21.15.134:3300/0 conn(0x7fc4a4071190 0x7fc4a41546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.452+0000 7fc4ab600640 1 --2- 172.21.15.67:0/383710129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4a40c2780 0x7fc4a4154190 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.452+0000 7fc4ab600640 1 -- 172.21.15.67:0/383710129 >> 172.21.15.67:0/383710129 conn(0x7fc4a40c7370 msgr2=0x7fc4a40c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.453+0000 7fc4ab600640 1 -- 172.21.15.67:0/383710129 shutdown_connections 2024-03-09T09:16:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.453+0000 7fc4ab600640 1 -- 172.21.15.67:0/383710129 wait complete. 2024-03-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.454+0000 7fc4ab600640 1 Processor -- start 2024-03-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.454+0000 7fc4ab600640 1 -- start start 2024-03-09T09:16:24.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.455+0000 7fc4ab600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4a4071190 0x7fc4a4153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.455+0000 7fc4ab600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4a40ac9a0 0x7fc4a40c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.455+0000 7fc4ab600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a40c2780 0x7fc4a41504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.455+0000 7fc4ab600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a4154c10 con 0x7fc4a4071190 2024-03-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.455+0000 7fc4ab600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a4070c00 con 0x7fc4a40c2780 2024-03-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.455+0000 7fc4ab600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40acd80 con 0x7fc4a40ac9a0 2024-03-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.455+0000 7fc4a8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4a4071190 0x7fc4a4153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.455+0000 7fc4a8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4a4071190 0x7fc4a4153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56822/0 (socket says 172.21.15.67:56822) 2024-03-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.455+0000 7fc4a8c00640 1 -- 172.21.15.67:0/2354693364 learned_addr learned my addr 172.21.15.67:0/2354693364 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a9600640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a40c2780 0x7fc4a41504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a3e00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4a40ac9a0 0x7fc4a40c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a8c00640 1 -- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4a40ac9a0 msgr2=0x7fc4a40c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a8c00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4a40ac9a0 0x7fc4a40c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a8c00640 1 -- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a40c2780 msgr2=0x7fc4a41504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a8c00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a40c2780 0x7fc4a41504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a8c00640 1 -- 172.21.15.67:0/2354693364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a414eb20 con 0x7fc4a4071190 2024-03-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a9600640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a40c2780 0x7fc4a41504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a3e00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4a40ac9a0 0x7fc4a40c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a8c00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4a4071190 0x7fc4a4153a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc49800c400 tx=0x7fc49800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4980052d0 con 0x7fc4a4071190 2024-03-09T09:16:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc498005490 con 0x7fc4a4071190 2024-03-09T09:16:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.456+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498007260 con 0x7fc4a4071190 2024-03-09T09:16:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.457+0000 7fc4ab600640 1 -- 172.21.15.67:0/2354693364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a41534d0 con 0x7fc4a4071190 2024-03-09T09:16:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.457+0000 7fc4ab600640 1 -- 172.21.15.67:0/2354693364 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a41ece60 con 0x7fc4a4071190 2024-03-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.458+0000 7fc4ab600640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a41ec9d0 0x7fc4a41484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.458+0000 7fc4ab600640 1 -- 172.21.15.67:0/2354693364 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a414efd0 con 0x7fc4a41ec9d0 2024-03-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.458+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc498007400 con 0x7fc4a4071190 2024-03-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.459+0000 7fc4a3e00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a41ec9d0 0x7fc4a41484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.459+0000 7fc4a3e00640 1 -- 172.21.15.67:0/2354693364 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4a4075a00 con 0x7fc4a41ec9d0 2024-03-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.459+0000 7fc4a3e00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a41ec9d0 0x7fc4a41484c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc4a406ed40 tx=0x7fc490004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.460+0000 7fc4a0c00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc48406b4d0 0x7fc48406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.460+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc498089b50 con 0x7fc4a4071190 2024-03-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.461+0000 7fc4a9600640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc48406b4d0 0x7fc48406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.461+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490049070 con 0x7fc4a41ec9d0 2024-03-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.461+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc49003aa00 con 0x7fc4a41ec9d0 2024-03-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.461+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a41ec9d0 msgr2=0x7fc4a41484c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.461+0000 7fc4a0c00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a41ec9d0 0x7fc4a41484c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc4a406ed40 tx=0x7fc490004730 comp rx=0 tx=0).stop 2024-03-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.461+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a41ec9d0 msgr2=0x7fc4a41484c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.461+0000 7fc4a0c00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a41ec9d0 0x7fc4a41484c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.461+0000 7fc4a9600640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc48406b4d0 0x7fc48406d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc49c007380 tx=0x7fc49c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.475+0000 7fc4ab600640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a414ced0 0x7fc4a4148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.475+0000 7fc4ab600640 1 -- 172.21.15.67:0/2354693364 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a4148f40 con 0x7fc4a414ced0 2024-03-09T09:16:24.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.475+0000 7fc4a3e00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a414ced0 0x7fc4a4148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.476+0000 7fc4a3e00640 1 -- 172.21.15.67:0/2354693364 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc4a4154e00 con 0x7fc4a414ced0 2024-03-09T09:16:24.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.476+0000 7fc4a3e00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a414ced0 0x7fc4a4148a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc490000c00 tx=0x7fc4900046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.476+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490049040 con 0x7fc4a414ced0 2024-03-09T09:16:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7fc49004cad0 con 0x7fc4a414ced0 2024-03-09T09:16:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a414ced0 msgr2=0x7fc4a4148a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4a0c00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a414ced0 0x7fc4a4148a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc490000c00 tx=0x7fc4900046c0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4a0c00640 1 -- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a414ced0 msgr2=0x7fc4a4148a00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4a0c00640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc4a414ced0 0x7fc4a4148a00 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc490000c00 tx=0x7fc4900046c0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.483 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:16:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:16:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:16:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:16:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 180, 2024-03-09T09:16:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:16:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:16:24.483 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:24.484 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:24.485 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:16:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:16:24.486 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:24.487 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:16:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:24.489 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:16:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:16:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:16:24.490 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:24.491 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:16:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:16:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:16:24.492 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:16:24.493 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:16:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.494 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:16:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:16:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:16:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:16:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.495 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:16:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:16:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.496 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:16:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:16:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:16:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.497 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.498 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.499 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:16:24.499 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:16:24.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4ab600640 1 -- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc48406b4d0 msgr2=0x7fc48406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4ab600640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc48406b4d0 0x7fc48406d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc49c007380 tx=0x7fc49c00b040 comp rx=0 tx=0).stop 2024-03-09T09:16:24.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4ab600640 1 -- 172.21.15.67:0/2354693364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4a4071190 msgr2=0x7fc4a4153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4ab600640 1 --2- 172.21.15.67:0/2354693364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc4a4071190 0x7fc4a4153a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc49800c400 tx=0x7fc49800c8c0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4a9600640 1 -- 172.21.15.67:0/2354693364 reap_dead start 2024-03-09T09:16:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4a9600640 1 -- 172.21.15.67:0/2354693364 reap_dead start 2024-03-09T09:16:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4ab600640 1 -- 172.21.15.67:0/2354693364 shutdown_connections 2024-03-09T09:16:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.477+0000 7fc4ab600640 1 -- 172.21.15.67:0/2354693364 >> 172.21.15.67:0/2354693364 conn(0x7fc4a40c7370 msgr2=0x7fc4a41498e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.478+0000 7fc4ab600640 1 -- 172.21.15.67:0/2354693364 shutdown_connections 2024-03-09T09:16:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.478+0000 7fc4ab600640 1 -- 172.21.15.67:0/2354693364 wait complete. 2024-03-09T09:16:24.501 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:16:24.501 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:16:24.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:16:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.709+0000 7f4e5e200640 1 Processor -- start 2024-03-09T09:16:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.709+0000 7f4e5e200640 1 -- start start 2024-03-09T09:16:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.711+0000 7f4e5e200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e58151360 0x7f4e580c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.711+0000 7f4e5e200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e580c2780 con 0x7f4e580ac930 2024-03-09T09:16:24.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.711+0000 7f4e5e200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e580708c0 con 0x7f4e580c30a0 2024-03-09T09:16:24.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.711+0000 7f4e5e200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e58070670 con 0x7f4e58151360 2024-03-09T09:16:24.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.711+0000 7f4e4fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e58151360 0x7f4e580c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.711+0000 7f4e4fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e58151360 0x7f4e580c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56828/0 (socket says 172.21.15.67:56828) 2024-03-09T09:16:24.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.711+0000 7f4e4fe00640 1 -- 172.21.15.67:0/109051209 learned_addr learned my addr 172.21.15.67:0/109051209 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:24.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.712+0000 7f4e4fe00640 1 -- 172.21.15.67:0/109051209 >> v1:172.21.15.134:6789/0 conn(0x7f4e580c30a0 legacy=0x7f4e581539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:24.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.712+0000 7f4e4fe00640 1 -- 172.21.15.67:0/109051209 >> v1:172.21.15.67:6789/0 conn(0x7f4e580ac930 legacy=0x7f4e58150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.712+0000 7f4e4fe00640 1 -- 172.21.15.67:0/109051209 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58070ba0 con 0x7f4e58151360 2024-03-09T09:16:24.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.713+0000 7f4e4fe00640 1 --2- 172.21.15.67:0/109051209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e58151360 0x7f4e580c3660 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009870 tx=0x7f4e4002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d241370e261d7cd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.713+0000 7f4e57e00640 1 -- 172.21.15.67:0/109051209 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40034070 con 0x7f4e58151360 2024-03-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.713+0000 7f4e57e00640 1 -- 172.21.15.67:0/109051209 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e40030850 con 0x7f4e58151360 2024-03-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.713+0000 7f4e57e00640 1 -- 172.21.15.67:0/109051209 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40030c20 con 0x7f4e58151360 2024-03-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.714+0000 7f4e5e200640 1 -- 172.21.15.67:0/109051209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e58151360 msgr2=0x7f4e580c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.714+0000 7f4e5e200640 1 --2- 172.21.15.67:0/109051209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e58151360 0x7f4e580c3660 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009870 tx=0x7f4e4002b990 comp rx=0 tx=0).stop 2024-03-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.714+0000 7f4e5e200640 1 -- 172.21.15.67:0/109051209 shutdown_connections 2024-03-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.714+0000 7f4e5e200640 1 --2- 172.21.15.67:0/109051209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e58151360 0x7f4e580c3660 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.714+0000 7f4e5e200640 1 -- 172.21.15.67:0/109051209 >> 172.21.15.67:0/109051209 conn(0x7f4e580c7220 msgr2=0x7f4e580c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.714+0000 7f4e5e200640 1 -- 172.21.15.67:0/109051209 shutdown_connections 2024-03-09T09:16:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.714+0000 7f4e5e200640 1 -- 172.21.15.67:0/109051209 wait complete. 2024-03-09T09:16:24.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.715+0000 7f4e5e200640 1 Processor -- start 2024-03-09T09:16:24.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.716+0000 7f4e5e200640 1 -- start start 2024-03-09T09:16:24.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.716+0000 7f4e5e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e580ac930 0x7f4e580ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.716+0000 7f4e5e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580c30a0 0x7f4e580c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.716+0000 7f4e5e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e58151360 0x7f4e58156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.716+0000 7f4e5e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580ace90 con 0x7f4e580c30a0 2024-03-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.716+0000 7f4e5e200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580acd10 con 0x7f4e58151360 2024-03-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.716+0000 7f4e5e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e581540f0 con 0x7f4e580ac930 2024-03-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.717+0000 7f4e4fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580c30a0 0x7f4e580c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.717+0000 7f4e4fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580c30a0 0x7f4e580c08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56836/0 (socket says 172.21.15.67:56836) 2024-03-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.717+0000 7f4e4fe00640 1 -- 172.21.15.67:0/288141411 learned_addr learned my addr 172.21.15.67:0/288141411 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.717+0000 7f4e5ce00640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e580ac930 0x7f4e580ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.717+0000 7f4e5d800640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e58151360 0x7f4e58156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.717+0000 7f4e4fe00640 1 -- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e580ac930 msgr2=0x7f4e580ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.717+0000 7f4e4fe00640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e580ac930 0x7f4e580ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.717+0000 7f4e4fe00640 1 -- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e58151360 msgr2=0x7f4e58156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.717+0000 7f4e4fe00640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e58151360 0x7f4e58156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.717+0000 7f4e4fe00640 1 -- 172.21.15.67:0/288141411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58155f90 con 0x7f4e580c30a0 2024-03-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.718+0000 7f4e4fe00640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580c30a0 0x7f4e580c08c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.718+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40034040 con 0x7f4e580c30a0 2024-03-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.718+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e400433f0 con 0x7f4e580c30a0 2024-03-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.718+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e400437e0 con 0x7f4e580c30a0 2024-03-09T09:16:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.718+0000 7f4e5e200640 1 -- 172.21.15.67:0/288141411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e580ab330 con 0x7f4e580c30a0 2024-03-09T09:16:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.718+0000 7f4e5e200640 1 -- 172.21.15.67:0/288141411 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e581564e0 con 0x7f4e580c30a0 2024-03-09T09:16:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.720+0000 7f4e5e200640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58156850 0x7f4e58157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.720+0000 7f4e5e200640 1 -- 172.21.15.67:0/288141411 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5804f3c0 con 0x7f4e58156850 2024-03-09T09:16:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.720+0000 7f4e5ce00640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58156850 0x7f4e58157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.721+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4e4004a060 con 0x7f4e580c30a0 2024-03-09T09:16:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.721+0000 7f4e5ce00640 1 -- 172.21.15.67:0/288141411 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e580722a0 con 0x7f4e58156850 2024-03-09T09:16:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.721+0000 7f4e55600640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e3406b4d0 0x7f4e3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.722+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4e400ac760 con 0x7f4e580c30a0 2024-03-09T09:16:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.722+0000 7f4e5ce00640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58156850 0x7f4e58157220 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800f8b0 tx=0x7f4e4800fd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.722+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48019070 con 0x7f4e58156850 2024-03-09T09:16:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.723+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4e48006d60 con 0x7f4e58156850 2024-03-09T09:16:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.723+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58156850 msgr2=0x7f4e58157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.723+0000 7f4e55600640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58156850 0x7f4e58157220 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800f8b0 tx=0x7f4e4800fd70 comp rx=0 tx=0).stop 2024-03-09T09:16:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.723+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58156850 msgr2=0x7f4e58157220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.723+0000 7f4e55600640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58156850 0x7f4e58157220 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800f8b0 tx=0x7f4e4800fd70 comp rx=0 tx=0).stop 2024-03-09T09:16:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.723+0000 7f4e5d800640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e3406b4d0 0x7f4e3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.725+0000 7f4e5d800640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e3406b4d0 0x7f4e3406d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4e50004e20 tx=0x7f4e50009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.733+0000 7f4e5e200640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58157760 0x7f4e581eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.733+0000 7f4e5e200640 1 -- 172.21.15.67:0/288141411 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58067b30 con 0x7f4e58157760 2024-03-09T09:16:24.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.734+0000 7f4e5ce00640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58157760 0x7f4e581eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.735+0000 7f4e5ce00640 1 -- 172.21.15.67:0/288141411 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4e580c0e00 con 0x7f4e58157760 2024-03-09T09:16:24.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.735+0000 7f4e5ce00640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58157760 0x7f4e581eeac0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800f8b0 tx=0x7f4e48001250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.735+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48019070 con 0x7f4e58157760 2024-03-09T09:16:24.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.736+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7f4e480056e0 con 0x7f4e58157760 2024-03-09T09:16:24.742 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:16:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:16:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:16:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:16:24.742 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 180, 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:24.743 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:24.744 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:16:24.745 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:24.746 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.747 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.748 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:24.749 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:24.750 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:24.751 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:16:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:16:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:16:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:24.752 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.753 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:16:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:16:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.754 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:16:24.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:16:24.756 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:16:24.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.736+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58157760 msgr2=0x7f4e581eeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.736+0000 7f4e55600640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58157760 0x7f4e581eeac0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800f8b0 tx=0x7f4e48001250 comp rx=0 tx=0).stop 2024-03-09T09:16:24.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.736+0000 7f4e55600640 1 -- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58157760 msgr2=0x7f4e581eeac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:24.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.736+0000 7f4e55600640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e58157760 0x7f4e581eeac0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800f8b0 tx=0x7f4e48001250 comp rx=0 tx=0).stop 2024-03-09T09:16:24.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.737+0000 7f4e5e200640 1 -- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e3406b4d0 msgr2=0x7f4e3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.737+0000 7f4e5e200640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e3406b4d0 0x7f4e3406d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4e50004e20 tx=0x7f4e50009040 comp rx=0 tx=0).stop 2024-03-09T09:16:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.737+0000 7f4e5e200640 1 -- 172.21.15.67:0/288141411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580c30a0 msgr2=0x7f4e580c08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.737+0000 7f4e5e200640 1 --2- 172.21.15.67:0/288141411 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e580c30a0 0x7f4e580c08c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e400046c0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.737+0000 7f4e5d800640 1 -- 172.21.15.67:0/288141411 reap_dead start 2024-03-09T09:16:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.737+0000 7f4e5d800640 1 -- 172.21.15.67:0/288141411 reap_dead start 2024-03-09T09:16:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.737+0000 7f4e5e200640 1 -- 172.21.15.67:0/288141411 shutdown_connections 2024-03-09T09:16:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.737+0000 7f4e5e200640 1 -- 172.21.15.67:0/288141411 >> 172.21.15.67:0/288141411 conn(0x7f4e580c7220 msgr2=0x7f4e5814fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.737+0000 7f4e5e200640 1 -- 172.21.15.67:0/288141411 shutdown_connections 2024-03-09T09:16:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.737+0000 7f4e5e200640 1 -- 172.21.15.67:0/288141411 wait complete. 2024-03-09T09:16:24.766 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:16:24.766 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:16:24.766 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:16:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.955+0000 7f4711600640 1 Processor -- start 2024-03-09T09:16:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.955+0000 7f4711600640 1 -- start start 2024-03-09T09:16:24.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.957+0000 7f4711600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f470c0c5e40 0x7f470c156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.957+0000 7f4711600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0cadb0 0x7f470c156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.957+0000 7f4711600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470c156a90 con 0x7f470c153300 2024-03-09T09:16:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.957+0000 7f4711600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470c157000 con 0x7f470c0c5e40 2024-03-09T09:16:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.957+0000 7f4711600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0ae430 con 0x7f470c0cadb0 2024-03-09T09:16:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.957+0000 7f4703e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f470c0c5e40 0x7f470c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.957+0000 7f4703e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f470c0c5e40 0x7f470c156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56846/0 (socket says 172.21.15.67:56846) 2024-03-09T09:16:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.957+0000 7f4703e00640 1 -- 172.21.15.67:0/3867861460 learned_addr learned my addr 172.21.15.67:0/3867861460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.957+0000 7f470be00640 1 --1- 172.21.15.67:0/3867861460 >> v1:172.21.15.67:6789/0 conn(0x7f470c153300 0x7f470c0b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43856/0 (socket says 172.21.15.67:43856) 2024-03-09T09:16:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.957+0000 7f4710c00640 1 --2- 172.21.15.67:0/3867861460 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0cadb0 0x7f470c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.958+0000 7f4703e00640 1 -- 172.21.15.67:0/3867861460 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0cadb0 msgr2=0x7f470c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.958+0000 7f4703e00640 1 --2- 172.21.15.67:0/3867861460 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0cadb0 0x7f470c156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.958+0000 7f4703e00640 1 -- 172.21.15.67:0/3867861460 >> v1:172.21.15.67:6789/0 conn(0x7f470c153300 legacy=0x7f470c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.958+0000 7f4703e00640 1 -- 172.21.15.67:0/3867861460 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c0ae6f0 con 0x7f470c0c5e40 2024-03-09T09:16:24.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.958+0000 7f470b400640 1 -- 172.21.15.67:0/3867861460 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028656070 0 0) 0x7f470c156a90 con 0x7f470c153300 2024-03-09T09:16:24.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.959+0000 7f4703e00640 1 --2- 172.21.15.67:0/3867861460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470c0c5e40 0x7f470c156010 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009870 tx=0x7f46f402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249694192150430e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.959+0000 7f470b400640 1 -- 172.21.15.67:0/3867861460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4034070 con 0x7f470c0c5e40 2024-03-09T09:16:24.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.959+0000 7f470b400640 1 -- 172.21.15.67:0/3867861460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f4030840 con 0x7f470c0c5e40 2024-03-09T09:16:24.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.959+0000 7f470b400640 1 -- 172.21.15.67:0/3867861460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4030c10 con 0x7f470c0c5e40 2024-03-09T09:16:24.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.960+0000 7f4711600640 1 -- 172.21.15.67:0/3867861460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470c0c5e40 msgr2=0x7f470c156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.960+0000 7f4711600640 1 --2- 172.21.15.67:0/3867861460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470c0c5e40 0x7f470c156010 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009870 tx=0x7f46f402b980 comp rx=0 tx=0).stop 2024-03-09T09:16:24.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.960+0000 7f4711600640 1 -- 172.21.15.67:0/3867861460 shutdown_connections 2024-03-09T09:16:24.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.960+0000 7f4711600640 1 --2- 172.21.15.67:0/3867861460 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c0cadb0 0x7f470c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.960+0000 7f4711600640 1 --2- 172.21.15.67:0/3867861460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470c0c5e40 0x7f470c156010 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.960+0000 7f4711600640 1 -- 172.21.15.67:0/3867861460 >> 172.21.15.67:0/3867861460 conn(0x7f470c0732d0 msgr2=0x7f470c0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:24.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.960+0000 7f4711600640 1 -- 172.21.15.67:0/3867861460 shutdown_connections 2024-03-09T09:16:24.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.960+0000 7f4711600640 1 -- 172.21.15.67:0/3867861460 wait complete. 2024-03-09T09:16:24.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.963+0000 7f4711600640 1 Processor -- start 2024-03-09T09:16:24.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.963+0000 7f4711600640 1 -- start start 2024-03-09T09:16:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.964+0000 7f4711600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470c0c5e40 0x7f470c14f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.964+0000 7f4711600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470c0cadb0 0x7f470c1ee970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.964+0000 7f4711600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c153300 0x7f470c147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.964+0000 7f4711600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c159e90 con 0x7f470c0cadb0 2024-03-09T09:16:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.964+0000 7f4711600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c159bb0 con 0x7f470c0c5e40 2024-03-09T09:16:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.964+0000 7f4711600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0c59e0 con 0x7f470c153300 2024-03-09T09:16:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.965+0000 7f4703e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470c0cadb0 0x7f470c1ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.965+0000 7f4703e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470c0cadb0 0x7f470c1ee970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56860/0 (socket says 172.21.15.67:56860) 2024-03-09T09:16:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.965+0000 7f4703e00640 1 -- 172.21.15.67:0/2164295873 learned_addr learned my addr 172.21.15.67:0/2164295873 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.965+0000 7f470be00640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470c0c5e40 0x7f470c14f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.965+0000 7f4710c00640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c153300 0x7f470c147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.965+0000 7f4703e00640 1 -- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c153300 msgr2=0x7f470c147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.965+0000 7f4703e00640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c153300 0x7f470c147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.965+0000 7f4703e00640 1 -- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470c0c5e40 msgr2=0x7f470c14f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.965+0000 7f4703e00640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470c0c5e40 0x7f470c14f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.965+0000 7f4703e00640 1 -- 172.21.15.67:0/2164295873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c153840 con 0x7f470c0cadb0 2024-03-09T09:16:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.966+0000 7f4703e00640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470c0cadb0 0x7f470c1ee970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46f4000c00 tx=0x7f46f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.966+0000 7f4708c00640 1 -- 172.21.15.67:0/2164295873 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4034040 con 0x7f470c0cadb0 2024-03-09T09:16:24.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.966+0000 7f4708c00640 1 -- 172.21.15.67:0/2164295873 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f403ab90 con 0x7f470c0cadb0 2024-03-09T09:16:24.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.966+0000 7f4711600640 1 -- 172.21.15.67:0/2164295873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c1ef1b0 con 0x7f470c0cadb0 2024-03-09T09:16:24.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.966+0000 7f4708c00640 1 -- 172.21.15.67:0/2164295873 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4004030 con 0x7f470c0cadb0 2024-03-09T09:16:24.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.966+0000 7f4711600640 1 -- 172.21.15.67:0/2164295873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c0c05f0 con 0x7f470c0cadb0 2024-03-09T09:16:24.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.967+0000 7f4711600640 1 -- 172.21.15.67:0/2164295873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c04f3c0 con 0x7f470c0cadb0 2024-03-09T09:16:24.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.968+0000 7f4708c00640 1 -- 172.21.15.67:0/2164295873 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f46f404b060 con 0x7f470c0cadb0 2024-03-09T09:16:24.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.969+0000 7f4708c00640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46e806b500 0x7f46e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:24.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.970+0000 7f4708c00640 1 -- 172.21.15.67:0/2164295873 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f46f40ac2f0 con 0x7f470c0cadb0 2024-03-09T09:16:24.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.970+0000 7f470be00640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46e806b500 0x7f46e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:24.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.971+0000 7f470be00640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46e806b500 0x7f46e806d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f46fc000f60 tx=0x7f46fc007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:24.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:24.973+0000 7f4708c00640 1 -- 172.21.15.67:0/2164295873 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f46f403f070 con 0x7f470c0cadb0 2024-03-09T09:16:25.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.298+0000 7f4711600640 1 -- 172.21.15.67:0/2164295873 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f470c0cab90 con 0x7f470c0cadb0 2024-03-09T09:16:25.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.299+0000 7f4708c00640 1 -- 172.21.15.67:0/2164295873 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f46f407ae10 con 0x7f470c0cadb0 2024-03-09T09:16:25.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.302+0000 7f4711600640 1 -- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46e806b500 msgr2=0x7f46e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:25.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.302+0000 7f4711600640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46e806b500 0x7f46e806d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f46fc000f60 tx=0x7f46fc007b50 comp rx=0 tx=0).stop 2024-03-09T09:16:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.302+0000 7f4711600640 1 -- 172.21.15.67:0/2164295873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470c0cadb0 msgr2=0x7f470c1ee970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.302+0000 7f4711600640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470c0cadb0 0x7f470c1ee970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46f4000c00 tx=0x7f46f40046c0 comp rx=0 tx=0).stop 2024-03-09T09:16:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.303+0000 7f4711600640 1 -- 172.21.15.67:0/2164295873 shutdown_connections 2024-03-09T09:16:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.303+0000 7f4711600640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f46e806b500 0x7f46e806d9c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.303+0000 7f4711600640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f470c153300 0x7f470c147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.303+0000 7f4711600640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f470c0cadb0 0x7f470c1ee970 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.303+0000 7f4711600640 1 --2- 172.21.15.67:0/2164295873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470c0c5e40 0x7f470c14f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.303+0000 7f4711600640 1 -- 172.21.15.67:0/2164295873 >> 172.21.15.67:0/2164295873 conn(0x7f470c0732d0 msgr2=0x7f470c157d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.303+0000 7f4711600640 1 -- 172.21.15.67:0/2164295873 shutdown_connections 2024-03-09T09:16:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:25.303+0000 7f4711600640 1 -- 172.21.15.67:0/2164295873 wait complete. 2024-03-09T09:16:25.331 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:16:30.083 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 14% (66/447) Updating files: 15% (68/447) Updating files: 16% (72/447) Updating files: 17% (76/447) Updating files: 18% (81/447) Updating files: 19% (85/447) Updating files: 20% (90/447) Updating files: 21% (94/447) Updating files: 22% (99/447) Updating files: 23% (103/447) Updating files: 24% (108/447) Updating files: 25% (112/447) Updating files: 26% (117/447) Updating files: 27% (121/447) Updating files: 28% (126/447) Updating files: 29% (130/447) Updating files: 30% (135/447) Updating files: 31% (139/447) Updating files: 32% (144/447) Updating files: 33% (148/447) Updating files: 34% (152/447) Updating files: 35% (157/447) Updating files: 36% (161/447) Updating files: 37% (166/447) Updating files: 38% (170/447) Updating files: 38% (172/447) Updating files: 39% (175/447) Updating files: 40% (179/447) Updating files: 41% (184/447) Updating files: 41% (187/447) Updating files: 42% (188/447) Updating files: 43% (193/447) Updating files: 43% (194/447) Updating files: 44% (197/447) Updating files: 45% (202/447) Updating files: 46% (206/447) Updating files: 47% (211/447) Updating files: 48% (215/447) Updating files: 49% (220/447) Updating files: 50% (224/447) Updating files: 51% (228/447) Updating files: 52% (233/447) Updating files: 53% (237/447) Updating files: 54% (242/447) Updating files: 55% (246/447) Updating files: 56% (251/447) Updating files: 57% (255/447) Updating files: 58% (260/447) Updating files: 59% (264/447) Updating files: 60% (269/447) Updating files: 61% (273/447) Updating files: 62% (278/447) Updating files: 63% (282/447) Updating files: 64% (287/447) Updating files: 65% (291/447) Updating files: 66% (296/447) Updating files: 67% (300/447) Updating files: 68% (304/447) Updating files: 68% (307/447) Updating files: 69% (309/447) Updating files: 70% (313/447) Updating files: 71% (318/447) Updating files: 72% (322/447) Updating files: 73% (327/447) Updating files: 74% (331/447) Updating files: 75% (336/447) Updating files: 76% (340/447) Updating files: 77% (345/447) Updating files: 78% (349/447) Updating files: 79% (354/447) Updating files: 80% (358/447) Updating files: 81% (363/447) Updating files: 82% (367/447) Updating files: 83% (372/447) Updating files: 84% (376/447) Updating files: 85% (380/447) Updating files: 85% (384/447) Updating files: 86% (385/447) Updating files: 87% (389/447) Updating files: 87% (390/447) Updating files: 88% (394/447) Updating files: 89% (398/447) Updating files: 90% (403/447) Updating files: 91% (407/447) Updating files: 92% (412/447) Updating files: 93% (416/447) Updating files: 93% (418/447) Updating files: 94% (421/447) Updating files: 95% (425/447) Updating files: 96% (430/447) Updating files: 97% (434/447) Updating files: 98% (439/447) Updating files: 99% (443/447) Updating files: 100% (447/447) Updating files: 100% (447/447), done. 2024-03-09T09:16:30.171 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 97dcc0539df 0.86 2024-03-09T09:16:30.196 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.86 2024-03-09T09:16:30.453 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:16:30.453 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 87 -eq 48 ']' 2024-03-09T09:16:30.453 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.87 2024-03-09T09:16:30.453 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.87 2024-03-09T09:16:30.453 INFO:tasks.workunit.client.0.smithi067.stdout:v0.87 2024-03-09T09:16:30.453 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.87 2024-03-09T09:16:31.225 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 9% (41/447) Updating files: 10% (45/447) Updating files: 11% (50/447) Updating files: 12% (54/447) Updating files: 13% (59/447) Updating files: 14% (63/447) Updating files: 15% (68/447) Updating files: 16% (72/447) Updating files: 17% (76/447) Updating files: 18% (81/447) Updating files: 19% (85/447) Updating files: 20% (90/447) Updating files: 21% (94/447) Updating files: 22% (99/447) Updating files: 23% (103/447) Updating files: 24% (108/447) Updating files: 25% (112/447) Updating files: 26% (117/447) Updating files: 27% (121/447) Updating files: 28% (126/447) Updating files: 29% (130/447) Updating files: 30% (135/447) Updating files: 31% (139/447) Updating files: 32% (144/447) Updating files: 33% (148/447) Updating files: 34% (152/447) Updating files: 35% (157/447) Updating files: 36% (161/447) Updating files: 37% (166/447) Updating files: 38% (170/447) Updating files: 39% (175/447) Updating files: 40% (179/447) Updating files: 41% (184/447) Updating files: 42% (188/447) Updating files: 43% (193/447) Updating files: 44% (197/447) Updating files: 45% (202/447) Updating files: 46% (206/447) Updating files: 47% (211/447) Updating files: 48% (215/447) Updating files: 49% (220/447) Updating files: 49% (223/447) Updating files: 50% (224/447) Updating files: 51% (228/447) Updating files: 52% (233/447) Updating files: 53% (237/447) Updating files: 54% (242/447) Updating files: 55% (246/447) Updating files: 56% (251/447) Updating files: 57% (255/447) Updating files: 58% (260/447) Updating files: 59% (264/447) Updating files: 59% (265/447) Updating files: 60% (269/447) Updating files: 61% (273/447) Updating files: 62% (278/447) Updating files: 63% (282/447) Updating files: 64% (287/447) Updating files: 65% (291/447) Updating files: 66% (296/447) Updating files: 67% (300/447) Updating files: 68% (304/447) Updating files: 69% (309/447) Updating files: 70% (313/447) Updating files: 71% (318/447) Updating files: 72% (322/447) Updating files: 73% (327/447) Updating files: 74% (331/447) Updating files: 75% (336/447) Updating files: 76% (340/447) Updating files: 77% (345/447) Updating files: 77% (348/447) Updating files: 78% (349/447) Updating files: 79% (354/447) Updating files: 80% (358/447) Updating files: 81% (363/447) Updating files: 82% (367/447) Updating files: 83% (372/447) Updating files: 84% (376/447) Updating files: 85% (380/447) Updating files: 86% (385/447) Updating files: 87% (389/447) Updating files: 88% (394/447) Updating files: 89% (398/447) Updating files: 89% (401/447) Updating files: 90% (403/447) Updating files: 91% (407/447) Updating files: 92% (412/447) Updating files: 93% (416/447) Updating files: 94% (421/447) Updating files: 95% (425/447) Updating files: 95% (428/447) Updating files: 96% (430/447) Updating files: 97% (434/447) Updating files: 98% (439/447) Updating files: 99% (443/447) Updating files: 100% (447/447) Updating files: 100% (447/447), done. 2024-03-09T09:16:31.303 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 97dcc0539df 0.86 2024-03-09T09:16:31.328 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.86 2024-03-09T09:16:31.535 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at c51c8f9d80f 0.87 2024-03-09T09:16:31.540 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.87 2024-03-09T09:16:31.693 INFO:tasks.workunit.client.1.smithi134.stdout:v0.87 2024-03-09T09:16:31.693 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:16:31.693 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 87 -eq 48 ']' 2024-03-09T09:16:31.694 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.87 2024-03-09T09:16:31.694 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.87 2024-03-09T09:16:31.694 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.87 2024-03-09T09:16:31.714 INFO:tasks.workunit.client.0.smithi067.stdout:v0.88 2024-03-09T09:16:31.714 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:16:31.714 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 88 -eq 48 ']' 2024-03-09T09:16:31.714 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.88 2024-03-09T09:16:31.714 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.88 2024-03-09T09:16:31.714 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.88 2024-03-09T09:16:32.957 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 77% (46/59) Updating files: 79% (47/59) Updating files: 81% (48/59) Updating files: 83% (49/59) Updating files: 84% (50/59) Updating files: 86% (51/59) Updating files: 88% (52/59) Updating files: 89% (53/59) Updating files: 91% (54/59) Updating files: 93% (55/59) Updating files: 94% (56/59) Updating files: 96% (57/59) Updating files: 98% (58/59) Updating files: 100% (59/59) Updating files: 100% (59/59), done. 2024-03-09T09:16:33.073 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at c51c8f9d80f 0.87 2024-03-09T09:16:33.079 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.87 2024-03-09T09:16:33.506 INFO:tasks.workunit.client.1.smithi134.stdout:v0.88 2024-03-09T09:16:33.506 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:16:33.506 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 88 -eq 48 ']' 2024-03-09T09:16:33.506 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.88 2024-03-09T09:16:33.506 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.88 2024-03-09T09:16:33.506 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.88 2024-03-09T09:16:35.333 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:16:35.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.571+0000 7f35c4a00640 1 Processor -- start 2024-03-09T09:16:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.572+0000 7f35c4a00640 1 -- start start 2024-03-09T09:16:35.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.573+0000 7f35c4a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f35c006d420 0x7f35c01527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.573+0000 7f35c4a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f35c014c9f0 0x7f35c01540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.573+0000 7f35c4a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c006d920 con 0x7f35c00c3080 2024-03-09T09:16:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.573+0000 7f35c4a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c0155ba0 con 0x7f35c014c9f0 2024-03-09T09:16:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.573+0000 7f35c4a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c006dc00 con 0x7f35c006d420 2024-03-09T09:16:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.573+0000 7f35bf400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f35c00c3080 0x7f35c014c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52472/0 (socket says 172.21.15.67:52472) 2024-03-09T09:16:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.573+0000 7f35bf400640 1 -- 172.21.15.67:0/4194343438 learned_addr learned my addr 172.21.15.67:0/4194343438 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:16:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.573+0000 7f35bfe00640 1 --2- 172.21.15.67:0/4194343438 >> v2:172.21.15.67:3300/0 conn(0x7f35c014c9f0 0x7f35c01540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:35.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.573+0000 7f35b7e00640 1 --2- 172.21.15.67:0/4194343438 >> v2:172.21.15.134:3300/0 conn(0x7f35c006d420 0x7f35c01527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:35.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.574+0000 7f35b7e00640 1 -- 172.21.15.67:0/4194343438 >> v2:172.21.15.67:3300/0 conn(0x7f35c014c9f0 msgr2=0x7f35c01540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:35.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.574+0000 7f35b7e00640 1 --2- 172.21.15.67:0/4194343438 >> v2:172.21.15.67:3300/0 conn(0x7f35c014c9f0 0x7f35c01540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:35.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.574+0000 7f35bea00640 1 -- 172.21.15.67:0/4194343438 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820923323 0 0) 0x7f35c006d920 con 0x7f35c00c3080 2024-03-09T09:16:35.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.574+0000 7f35b7e00640 1 -- 172.21.15.67:0/4194343438 >> v1:172.21.15.67:6789/0 conn(0x7f35c00c3080 legacy=0x7f35c014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:35.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.574+0000 7f35b7e00640 1 -- 172.21.15.67:0/4194343438 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c006dee0 con 0x7f35c006d420 2024-03-09T09:16:35.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.574+0000 7f35bfe00640 1 --2- 172.21.15.67:0/4194343438 >> v2:172.21.15.67:3300/0 conn(0x7f35c014c9f0 0x7f35c01540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:16:35.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.575+0000 7f35b7e00640 1 --2- 172.21.15.67:0/4194343438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35c006d420 0x7f35c01527d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f35c006e660 tx=0x7f35b002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=915d9c59ec13a0ab server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:35.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.575+0000 7f35bea00640 1 -- 172.21.15.67:0/4194343438 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00315a0 con 0x7f35c006d420 2024-03-09T09:16:35.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.576+0000 7f35bea00640 1 -- 172.21.15.67:0/4194343438 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b0037070 con 0x7f35c006d420 2024-03-09T09:16:35.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.576+0000 7f35bea00640 1 -- 172.21.15.67:0/4194343438 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0031960 con 0x7f35c006d420 2024-03-09T09:16:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.576+0000 7f35c4a00640 1 -- 172.21.15.67:0/4194343438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35c006d420 msgr2=0x7f35c01527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.576+0000 7f35c4a00640 1 --2- 172.21.15.67:0/4194343438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35c006d420 0x7f35c01527d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f35c006e660 tx=0x7f35b002b840 comp rx=0 tx=0).stop 2024-03-09T09:16:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.576+0000 7f35c4a00640 1 -- 172.21.15.67:0/4194343438 shutdown_connections 2024-03-09T09:16:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.576+0000 7f35c4a00640 1 --2- 172.21.15.67:0/4194343438 >> v2:172.21.15.67:3300/0 conn(0x7f35c014c9f0 0x7f35c01540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.576+0000 7f35c4a00640 1 --2- 172.21.15.67:0/4194343438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35c006d420 0x7f35c01527d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.576+0000 7f35c4a00640 1 -- 172.21.15.67:0/4194343438 >> 172.21.15.67:0/4194343438 conn(0x7f35c00c7300 msgr2=0x7f35c00c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.577+0000 7f35c4a00640 1 -- 172.21.15.67:0/4194343438 shutdown_connections 2024-03-09T09:16:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.577+0000 7f35c4a00640 1 -- 172.21.15.67:0/4194343438 wait complete. 2024-03-09T09:16:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.578+0000 7f35c4a00640 1 Processor -- start 2024-03-09T09:16:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.579+0000 7f35c4a00640 1 -- start start 2024-03-09T09:16:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.580+0000 7f35c4a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35c006d420 0x7f35c014ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.580+0000 7f35c4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35c00c3080 0x7f35c014b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.580+0000 7f35c4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35c014c9f0 0x7f35c00c1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.580+0000 7f35c4a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c00c8030 con 0x7f35c00c3080 2024-03-09T09:16:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.580+0000 7f35c4a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c00c3460 con 0x7f35c014c9f0 2024-03-09T09:16:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.580+0000 7f35c4a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c01ec150 con 0x7f35c006d420 2024-03-09T09:16:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.580+0000 7f35b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35c00c3080 0x7f35c014b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:35.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.580+0000 7f35b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35c00c3080 0x7f35c014b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50478/0 (socket says 172.21.15.67:50478) 2024-03-09T09:16:35.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.580+0000 7f35bfe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35c014c9f0 0x7f35c00c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:35.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.580+0000 7f35b7e00640 1 -- 172.21.15.67:0/2059398539 learned_addr learned my addr 172.21.15.67:0/2059398539 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:35.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.580+0000 7f35bf400640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35c006d420 0x7f35c014ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.581+0000 7f35b7e00640 1 -- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35c006d420 msgr2=0x7f35c014ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.581+0000 7f35b7e00640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35c006d420 0x7f35c014ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.581+0000 7f35b7e00640 1 -- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35c014c9f0 msgr2=0x7f35c00c1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.581+0000 7f35b7e00640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35c014c9f0 0x7f35c00c1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.581+0000 7f35b7e00640 1 -- 172.21.15.67:0/2059398539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c015fb20 con 0x7f35c00c3080 2024-03-09T09:16:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.581+0000 7f35b7e00640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35c00c3080 0x7f35c014b8f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009890 tx=0x7f35b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.582+0000 7f35b7400640 1 -- 172.21.15.67:0/2059398539 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0032630 con 0x7f35c00c3080 2024-03-09T09:16:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.582+0000 7f35b7400640 1 -- 172.21.15.67:0/2059398539 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b00327f0 con 0x7f35c00c3080 2024-03-09T09:16:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.582+0000 7f35c4a00640 1 -- 172.21.15.67:0/2059398539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c015fe00 con 0x7f35c00c3080 2024-03-09T09:16:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.582+0000 7f35b7400640 1 -- 172.21.15.67:0/2059398539 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0032be0 con 0x7f35c00c3080 2024-03-09T09:16:35.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.582+0000 7f35c4a00640 1 -- 172.21.15.67:0/2059398539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c014be30 con 0x7f35c00c3080 2024-03-09T09:16:35.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.583+0000 7f35c4a00640 1 -- 172.21.15.67:0/2059398539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c014da00 con 0x7f35c00c3080 2024-03-09T09:16:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.584+0000 7f35b7400640 1 -- 172.21.15.67:0/2059398539 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f35b004c020 con 0x7f35c00c3080 2024-03-09T09:16:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.585+0000 7f35b7400640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f359c06b4d0 0x7f359c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.585+0000 7f35b7400640 1 -- 172.21.15.67:0/2059398539 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f35b002f020 con 0x7f35c00c3080 2024-03-09T09:16:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.586+0000 7f35bf400640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f359c06b4d0 0x7f359c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.587+0000 7f35bf400640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f359c06b4d0 0x7f359c06d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f35a8000f60 tx=0x7f35a8007b70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:35.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.597+0000 7f35b7400640 1 -- 172.21.15.67:0/2059398539 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35b0040070 con 0x7f35c00c3080 2024-03-09T09:16:35.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.873+0000 7f35c4a00640 1 -- 172.21.15.67:0/2059398539 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f35c01ebc60 con 0x7f35c00c3080 2024-03-09T09:16:35.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.874+0000 7f35b7400640 1 -- 172.21.15.67:0/2059398539 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f35b004c310 con 0x7f35c00c3080 2024-03-09T09:16:35.880 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":180,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:35.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 -- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f359c06b4d0 msgr2=0x7f359c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:35.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f359c06b4d0 0x7f359c06d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f35a8000f60 tx=0x7f35a8007b70 comp rx=0 tx=0).stop 2024-03-09T09:16:35.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 -- 172.21.15.67:0/2059398539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35c00c3080 msgr2=0x7f35c014b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:35.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35c00c3080 0x7f35c014b8f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009890 tx=0x7f35b00046c0 comp rx=0 tx=0).stop 2024-03-09T09:16:35.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 -- 172.21.15.67:0/2059398539 shutdown_connections 2024-03-09T09:16:35.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f359c06b4d0 0x7f359c06d990 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:35.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35c014c9f0 0x7f35c00c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:35.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35c00c3080 0x7f35c014b8f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:35.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 --2- 172.21.15.67:0/2059398539 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35c006d420 0x7f35c014ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:35.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 -- 172.21.15.67:0/2059398539 >> 172.21.15.67:0/2059398539 conn(0x7f35c00c7300 msgr2=0x7f35c00c63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:35.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 -- 172.21.15.67:0/2059398539 shutdown_connections 2024-03-09T09:16:35.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:35.877+0000 7f35c4a00640 1 -- 172.21.15.67:0/2059398539 wait complete. 2024-03-09T09:16:35.900 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:16:35.900 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:16:35.900 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:16:35.901 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:16:35.956 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 31% (105/335) Updating files: 32% (108/335) Updating files: 33% (111/335) Updating files: 34% (114/335) Updating files: 35% (118/335) Updating files: 36% (121/335) Updating files: 37% (124/335) Updating files: 38% (128/335) Updating files: 39% (131/335) Updating files: 40% (134/335) Updating files: 41% (138/335) Updating files: 42% (141/335) Updating files: 43% (145/335) Updating files: 44% (148/335) Updating files: 45% (151/335) Updating files: 46% (155/335) Updating files: 47% (158/335) Updating files: 48% (161/335) Updating files: 49% (165/335) Updating files: 50% (168/335) Updating files: 51% (171/335) Updating files: 52% (175/335) Updating files: 53% (178/335) Updating files: 54% (181/335) Updating files: 55% (185/335) Updating files: 56% (188/335) Updating files: 57% (191/335) Updating files: 58% (195/335) Updating files: 59% (198/335) Updating files: 60% (201/335) Updating files: 61% (205/335) Updating files: 62% (208/335) Updating files: 63% (212/335) Updating files: 64% (215/335) Updating files: 65% (218/335) Updating files: 66% (222/335) Updating files: 67% (225/335) Updating files: 68% (228/335) Updating files: 69% (232/335) Updating files: 70% (235/335) Updating files: 71% (238/335) Updating files: 72% (242/335) Updating files: 73% (245/335) Updating files: 74% (248/335) Updating files: 75% (252/335) Updating files: 76% (255/335) Updating files: 77% (258/335) Updating files: 78% (262/335) Updating files: 79% (265/335) Updating files: 80% (268/335) Updating files: 81% (272/335) Updating files: 82% (275/335) Updating files: 83% (279/335) Updating files: 84% (282/335) Updating files: 85% (285/335) Updating files: 86% (289/335) Updating files: 87% (292/335) Updating files: 88% (295/335) Updating files: 89% (299/335) Updating files: 90% (302/335) Updating files: 91% (305/335) Updating files: 92% (309/335) Updating files: 93% (312/335) Updating files: 94% (315/335) Updating files: 95% (319/335) Updating files: 96% (322/335) Updating files: 97% (325/335) Updating files: 97% (327/335) Updating files: 98% (329/335) Updating files: 99% (332/335) Updating files: 100% (335/335) Updating files: 100% (335/335), done. 2024-03-09T09:16:36.063 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 4be687bf448 0.88 2024-03-09T09:16:36.083 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.88 2024-03-09T09:16:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.082+0000 7fe6b7200640 1 Processor -- start 2024-03-09T09:16:36.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.082+0000 7fe6b7200640 1 -- start start 2024-03-09T09:16:36.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.085+0000 7fe6b7200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe6b814fd90 0x7fe6b8156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.085+0000 7fe6b7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6b806c1d0 0x7fe6b806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.085+0000 7fe6b7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b8156bc0 con 0x7fe6b80c9380 2024-03-09T09:16:36.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.085+0000 7fe6b7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b806ba60 con 0x7fe6b806c1d0 2024-03-09T09:16:36.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.085+0000 7fe6b7200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b80c9890 con 0x7fe6b814fd90 2024-03-09T09:16:36.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.086+0000 7fe6b6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6b806c1d0 0x7fe6b806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.086+0000 7fe6b6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6b806c1d0 0x7fe6b806c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50480/0 (socket says 172.21.15.67:50480) 2024-03-09T09:16:36.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.086+0000 7fe6b6800640 1 -- 172.21.15.67:0/4262009759 learned_addr learned my addr 172.21.15.67:0/4262009759 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:36.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.086+0000 7fe6b5e00640 1 --2- 172.21.15.67:0/4262009759 >> v2:172.21.15.134:3300/0 conn(0x7fe6b814fd90 0x7fe6b8156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.086+0000 7fe6b6800640 1 -- 172.21.15.67:0/4262009759 >> v2:172.21.15.134:3300/0 conn(0x7fe6b814fd90 msgr2=0x7fe6b8156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:36.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.086+0000 7fe6b6800640 1 --2- 172.21.15.67:0/4262009759 >> v2:172.21.15.134:3300/0 conn(0x7fe6b814fd90 0x7fe6b8156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.086+0000 7fe6b6800640 1 -- 172.21.15.67:0/4262009759 >> v1:172.21.15.67:6789/0 conn(0x7fe6b80c9380 legacy=0x7fe6b80acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.086+0000 7fe6b4a00640 1 -- 172.21.15.67:0/4262009759 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608487751 0 0) 0x7fe6a4001110 con 0x7fe6b80c9380 2024-03-09T09:16:36.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.086+0000 7fe6b6800640 1 -- 172.21.15.67:0/4262009759 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b80c5f80 con 0x7fe6b806c1d0 2024-03-09T09:16:36.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.088+0000 7fe6b6800640 1 --2- 172.21.15.67:0/4262009759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6b806c1d0 0x7fe6b806c5f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80ada40 tx=0x7fe6b002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c29bbed9f3b8bc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.088+0000 7fe6b4a00640 1 -- 172.21.15.67:0/4262009759 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b002c9f0 con 0x7fe6b806c1d0 2024-03-09T09:16:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.088+0000 7fe6b4a00640 1 -- 172.21.15.67:0/4262009759 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b002cb90 con 0x7fe6b806c1d0 2024-03-09T09:16:36.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.088+0000 7fe6b4a00640 1 -- 172.21.15.67:0/4262009759 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00343f0 con 0x7fe6b806c1d0 2024-03-09T09:16:36.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.089+0000 7fe6b7200640 1 -- 172.21.15.67:0/4262009759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6b806c1d0 msgr2=0x7fe6b806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.089+0000 7fe6b7200640 1 --2- 172.21.15.67:0/4262009759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6b806c1d0 0x7fe6b806c5f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80ada40 tx=0x7fe6b002be50 comp rx=0 tx=0).stop 2024-03-09T09:16:36.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.089+0000 7fe6b7200640 1 -- 172.21.15.67:0/4262009759 shutdown_connections 2024-03-09T09:16:36.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.089+0000 7fe6b7200640 1 --2- 172.21.15.67:0/4262009759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6b806c1d0 0x7fe6b806c5f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.089+0000 7fe6b7200640 1 --2- 172.21.15.67:0/4262009759 >> v2:172.21.15.134:3300/0 conn(0x7fe6b814fd90 0x7fe6b8156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.089+0000 7fe6b7200640 1 -- 172.21.15.67:0/4262009759 >> 172.21.15.67:0/4262009759 conn(0x7fe6b80cb480 msgr2=0x7fe6b80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:36.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.090+0000 7fe6b7200640 1 -- 172.21.15.67:0/4262009759 shutdown_connections 2024-03-09T09:16:36.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.090+0000 7fe6b7200640 1 -- 172.21.15.67:0/4262009759 wait complete. 2024-03-09T09:16:36.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.091+0000 7fe6b7200640 1 Processor -- start 2024-03-09T09:16:36.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.092+0000 7fe6b7200640 1 -- start start 2024-03-09T09:16:36.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.093+0000 7fe6b7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6b806c1d0 0x7fe6b80af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.093+0000 7fe6b7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6b80c9380 0x7fe6b8148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.093+0000 7fe6b7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6b814fd90 0x7fe6b81486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.093+0000 7fe6b7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b806bcb0 con 0x7fe6b814fd90 2024-03-09T09:16:36.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.093+0000 7fe6b7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80c6520 con 0x7fe6b806c1d0 2024-03-09T09:16:36.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.093+0000 7fe6b7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80ad610 con 0x7fe6b80c9380 2024-03-09T09:16:36.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.093+0000 7fe6b6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6b814fd90 0x7fe6b81486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.093+0000 7fe6b6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6b814fd90 0x7fe6b81486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50494/0 (socket says 172.21.15.67:50494) 2024-03-09T09:16:36.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.093+0000 7fe6b6800640 1 -- 172.21.15.67:0/3521942755 learned_addr learned my addr 172.21.15.67:0/3521942755 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:36.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.093+0000 7fe6b5e00640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6b806c1d0 0x7fe6b80af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.093+0000 7fe6b5400640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6b80c9380 0x7fe6b8148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.094+0000 7fe6b6800640 1 -- 172.21.15.67:0/3521942755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6b80c9380 msgr2=0x7fe6b8148180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.094+0000 7fe6b6800640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6b80c9380 0x7fe6b8148180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.094+0000 7fe6b6800640 1 -- 172.21.15.67:0/3521942755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6b806c1d0 msgr2=0x7fe6b80af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.094+0000 7fe6b6800640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6b806c1d0 0x7fe6b80af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.094+0000 7fe6b6800640 1 -- 172.21.15.67:0/3521942755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b80ae4e0 con 0x7fe6b814fd90 2024-03-09T09:16:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.094+0000 7fe6b6800640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6b814fd90 0x7fe6b81486c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009f40 tx=0x7fe6b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.095+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0044020 con 0x7fe6b814fd90 2024-03-09T09:16:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.095+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0040470 con 0x7fe6b814fd90 2024-03-09T09:16:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.095+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0009510 con 0x7fe6b814fd90 2024-03-09T09:16:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.095+0000 7fe6b7200640 1 -- 172.21.15.67:0/3521942755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b80ac860 con 0x7fe6b814fd90 2024-03-09T09:16:36.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.095+0000 7fe6b7200640 1 -- 172.21.15.67:0/3521942755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b80ae0a0 con 0x7fe6b814fd90 2024-03-09T09:16:36.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.096+0000 7fe6a3400640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c002a70 0x7fe68c004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.096+0000 7fe6a3400640 1 -- 172.21.15.67:0/3521942755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c007ec0 con 0x7fe68c002a70 2024-03-09T09:16:36.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.097+0000 7fe6b5e00640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c002a70 0x7fe68c004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.097+0000 7fe6b5e00640 1 -- 172.21.15.67:0/3521942755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe68c0080c0 con 0x7fe68c002a70 2024-03-09T09:16:36.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.098+0000 7fe6b5e00640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c002a70 0x7fe68c004f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8011890 tx=0x7fe6a8011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.098+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe6b0047020 con 0x7fe6b814fd90 2024-03-09T09:16:36.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.100+0000 7fe6ae000640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe69806b4d0 0x7fe69806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.100+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe6b00ace30 con 0x7fe6b814fd90 2024-03-09T09:16:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.100+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8019070 con 0x7fe68c002a70 2024-03-09T09:16:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.100+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fe6a8004240 con 0x7fe68c002a70 2024-03-09T09:16:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.100+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c002a70 msgr2=0x7fe68c004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.100+0000 7fe6ae000640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c002a70 0x7fe68c004f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8011890 tx=0x7fe6a8011d50 comp rx=0 tx=0).stop 2024-03-09T09:16:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.100+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c002a70 msgr2=0x7fe68c004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.100+0000 7fe6ae000640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c002a70 0x7fe68c004f30 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8011890 tx=0x7fe6a8011d50 comp rx=0 tx=0).stop 2024-03-09T09:16:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.101+0000 7fe6b5400640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe69806b4d0 0x7fe69806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.102+0000 7fe6b5400640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe69806b4d0 0x7fe69806d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4007fd0 tx=0x7fe6a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.113+0000 7fe6a3400640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c006f00 0x7fe68c007390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.113+0000 7fe6a3400640 1 -- 172.21.15.67:0/3521942755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c00a4a0 con 0x7fe68c006f00 2024-03-09T09:16:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.113+0000 7fe6b5e00640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c006f00 0x7fe68c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.113+0000 7fe6b5e00640 1 -- 172.21.15.67:0/3521942755 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe68c0078d0 con 0x7fe68c006f00 2024-03-09T09:16:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.114+0000 7fe6b5e00640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c006f00 0x7fe68c007390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8011890 tx=0x7fe6a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.114+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8019040 con 0x7fe68c006f00 2024-03-09T09:16:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4537 (secure 0 0 0) 0x7fe6a800fe70 con 0x7fe68c006f00 2024-03-09T09:16:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c006f00 msgr2=0x7fe68c007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6ae000640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c006f00 0x7fe68c007390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8011890 tx=0x7fe6a80046c0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6ae000640 1 -- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c006f00 msgr2=0x7fe68c007390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6ae000640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe68c006f00 0x7fe68c007390 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8011890 tx=0x7fe6a80046c0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.121 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:16:36.121 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:16:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:16:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:16:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 180, 2024-03-09T09:16:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:16:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:16:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:16:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:16:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:16:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:36.122 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.123 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:36.124 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:16:36.125 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:36.126 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:36.127 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:16:36.128 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.131 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.132 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:16:36.133 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:16:36.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6a3400640 1 -- 172.21.15.67:0/3521942755 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe69806b4d0 msgr2=0x7fe69806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6a3400640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe69806b4d0 0x7fe69806d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4007fd0 tx=0x7fe6a4009040 comp rx=0 tx=0).stop 2024-03-09T09:16:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6a3400640 1 -- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6b814fd90 msgr2=0x7fe6b81486c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6a3400640 1 --2- 172.21.15.67:0/3521942755 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6b814fd90 0x7fe6b81486c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009f40 tx=0x7fe6b0004a40 comp rx=0 tx=0).stop 2024-03-09T09:16:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6b6800640 1 -- 172.21.15.67:0/3521942755 reap_dead start 2024-03-09T09:16:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6b6800640 1 -- 172.21.15.67:0/3521942755 reap_dead start 2024-03-09T09:16:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6a3400640 1 -- 172.21.15.67:0/3521942755 shutdown_connections 2024-03-09T09:16:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.115+0000 7fe6a3400640 1 -- 172.21.15.67:0/3521942755 >> 172.21.15.67:0/3521942755 conn(0x7fe6b80cb480 msgr2=0x7fe6b80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.116+0000 7fe6a3400640 1 -- 172.21.15.67:0/3521942755 shutdown_connections 2024-03-09T09:16:36.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.116+0000 7fe6a3400640 1 -- 172.21.15.67:0/3521942755 wait complete. 2024-03-09T09:16:36.138 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:16:36.138 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:16:36.138 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:16:36.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.419+0000 7fb482800640 1 Processor -- start 2024-03-09T09:16:36.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.419+0000 7fb482800640 1 -- start start 2024-03-09T09:16:36.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.421+0000 7fb482800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb47c0c96f0 0x7fb47c06b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.421+0000 7fb482800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c5f50 0x7fb47c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.421+0000 7fb482800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb47c06cf30 con 0x7fb47c153560 2024-03-09T09:16:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.421+0000 7fb482800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb47c159d80 con 0x7fb47c0c96f0 2024-03-09T09:16:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.421+0000 7fb482800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c15a060 con 0x7fb47c0c5f50 2024-03-09T09:16:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.421+0000 7fb47b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb47c153560 0x7fb47c0cac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52494/0 (socket says 172.21.15.67:52494) 2024-03-09T09:16:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.421+0000 7fb47b400640 1 -- 172.21.15.67:0/1577884769 learned_addr learned my addr 172.21.15.67:0/1577884769 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:16:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.421+0000 7fb47aa00640 1 --2- 172.21.15.67:0/1577884769 >> v2:172.21.15.67:3300/0 conn(0x7fb47c0c96f0 0x7fb47c06b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.422+0000 7fb47be00640 1 --2- 172.21.15.67:0/1577884769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c5f50 0x7fb47c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.422+0000 7fb47a000640 1 -- 172.21.15.67:0/1577884769 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057439789 0 0) 0x7fb47c06cf30 con 0x7fb47c153560 2024-03-09T09:16:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.422+0000 7fb47a000640 1 -- 172.21.15.67:0/1577884769 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb460003520 con 0x7fb47c153560 2024-03-09T09:16:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.422+0000 7fb47aa00640 1 -- 172.21.15.67:0/1577884769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c5f50 msgr2=0x7fb47c0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.422+0000 7fb47aa00640 1 --2- 172.21.15.67:0/1577884769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c5f50 0x7fb47c0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.422+0000 7fb47aa00640 1 -- 172.21.15.67:0/1577884769 >> v1:172.21.15.67:6789/0 conn(0x7fb47c153560 legacy=0x7fb47c0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.422+0000 7fb47aa00640 1 -- 172.21.15.67:0/1577884769 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c0ca440 con 0x7fb47c0c96f0 2024-03-09T09:16:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.423+0000 7fb47aa00640 1 --2- 172.21.15.67:0/1577884769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb47c0c96f0 0x7fb47c06b040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb4680097a0 tx=0x7fb46802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1511a47c21a7da server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.423+0000 7fb47a000640 1 -- 172.21.15.67:0/1577884769 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468031780 con 0x7fb47c0c96f0 2024-03-09T09:16:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.424+0000 7fb47a000640 1 -- 172.21.15.67:0/1577884769 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468037070 con 0x7fb47c0c96f0 2024-03-09T09:16:36.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.424+0000 7fb47a000640 1 -- 172.21.15.67:0/1577884769 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468031b20 con 0x7fb47c0c96f0 2024-03-09T09:16:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.424+0000 7fb482800640 1 -- 172.21.15.67:0/1577884769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb47c0c96f0 msgr2=0x7fb47c06b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.424+0000 7fb482800640 1 --2- 172.21.15.67:0/1577884769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb47c0c96f0 0x7fb47c06b040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb4680097a0 tx=0x7fb46802b820 comp rx=0 tx=0).stop 2024-03-09T09:16:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.424+0000 7fb482800640 1 -- 172.21.15.67:0/1577884769 shutdown_connections 2024-03-09T09:16:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.424+0000 7fb482800640 1 --2- 172.21.15.67:0/1577884769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c5f50 0x7fb47c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.424+0000 7fb482800640 1 --2- 172.21.15.67:0/1577884769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb47c0c96f0 0x7fb47c06b040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.424+0000 7fb482800640 1 -- 172.21.15.67:0/1577884769 >> 172.21.15.67:0/1577884769 conn(0x7fb47c0c8050 msgr2=0x7fb47c0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.424+0000 7fb482800640 1 -- 172.21.15.67:0/1577884769 shutdown_connections 2024-03-09T09:16:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.424+0000 7fb482800640 1 -- 172.21.15.67:0/1577884769 wait complete. 2024-03-09T09:16:36.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.427+0000 7fb482800640 1 Processor -- start 2024-03-09T09:16:36.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.427+0000 7fb482800640 1 -- start start 2024-03-09T09:16:36.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.428+0000 7fb482800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb47c0c5f50 0x7fb47c154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.428+0000 7fb482800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c96f0 0x7fb47c1ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.428+0000 7fb482800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c153560 0x7fb47c1ec290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.428+0000 7fb482800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c0c68e0 con 0x7fb47c0c5f50 2024-03-09T09:16:36.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.428+0000 7fb482800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c0c9ff0 con 0x7fb47c153560 2024-03-09T09:16:36.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.428+0000 7fb482800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c0c6a60 con 0x7fb47c0c96f0 2024-03-09T09:16:36.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.428+0000 7fb47b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb47c0c5f50 0x7fb47c154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.428+0000 7fb47b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb47c0c5f50 0x7fb47c154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50526/0 (socket says 172.21.15.67:50526) 2024-03-09T09:16:36.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.428+0000 7fb47b400640 1 -- 172.21.15.67:0/578391039 learned_addr learned my addr 172.21.15.67:0/578391039 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:36.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.429+0000 7fb47b400640 1 -- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c96f0 msgr2=0x7fb47c1ebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.429+0000 7fb47b400640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb47c0c96f0 0x7fb47c1ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.429+0000 7fb47b400640 1 -- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c153560 msgr2=0x7fb47c1ec290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.429+0000 7fb47b400640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c153560 0x7fb47c1ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.429+0000 7fb47b400640 1 -- 172.21.15.67:0/578391039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c154520 con 0x7fb47c0c5f50 2024-03-09T09:16:36.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.430+0000 7fb47b400640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb47c0c5f50 0x7fb47c154ce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00c3c0 tx=0x7fb46c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.430+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c017070 con 0x7fb47c0c5f50 2024-03-09T09:16:36.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.430+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c005ce0 con 0x7fb47c0c5f50 2024-03-09T09:16:36.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.430+0000 7fb482800640 1 -- 172.21.15.67:0/578391039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47c156070 con 0x7fb47c0c5f50 2024-03-09T09:16:36.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.430+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c0072f0 con 0x7fb47c0c5f50 2024-03-09T09:16:36.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.430+0000 7fb482800640 1 -- 172.21.15.67:0/578391039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47c14b760 con 0x7fb47c0c5f50 2024-03-09T09:16:36.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.432+0000 7fb482800640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c155220 0x7fb47c148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.432+0000 7fb482800640 1 -- 172.21.15.67:0/578391039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c155e20 con 0x7fb47c155220 2024-03-09T09:16:36.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.432+0000 7fb47aa00640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c155220 0x7fb47c148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.433+0000 7fb47aa00640 1 -- 172.21.15.67:0/578391039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb47c156590 con 0x7fb47c155220 2024-03-09T09:16:36.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.433+0000 7fb47aa00640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c155220 0x7fb47c148590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb47c073c90 tx=0x7fb468031ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.433+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468041070 con 0x7fb47c155220 2024-03-09T09:16:36.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.434+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb468037040 con 0x7fb47c155220 2024-03-09T09:16:36.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.434+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c155220 msgr2=0x7fb47c148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.434+0000 7fb473400640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c155220 0x7fb47c148590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb47c073c90 tx=0x7fb468031ab0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.434+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c155220 msgr2=0x7fb47c148590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.434+0000 7fb473400640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c155220 0x7fb47c148590 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb47c073c90 tx=0x7fb468031ab0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.434+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb46c01a070 con 0x7fb47c0c5f50 2024-03-09T09:16:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.435+0000 7fb473400640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb45c06afe0 0x7fb45c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.435+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb46c0899f0 con 0x7fb47c0c5f50 2024-03-09T09:16:36.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.435+0000 7fb47aa00640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb45c06afe0 0x7fb45c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.436+0000 7fb47aa00640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb45c06afe0 0x7fb45c06d4a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb47c155fa0 tx=0x7fb468041000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.443+0000 7fb482800640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0c2a40 0x7fb47c148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.443+0000 7fb482800640 1 -- 172.21.15.67:0/578391039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47c0c94d0 con 0x7fb47c0c2a40 2024-03-09T09:16:36.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.444+0000 7fb47be00640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0c2a40 0x7fb47c148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.444+0000 7fb47be00640 1 -- 172.21.15.67:0/578391039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb47c1520f0 con 0x7fb47c0c2a40 2024-03-09T09:16:36.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.446+0000 7fb47be00640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0c2a40 0x7fb47c148ad0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb474007cb0 tx=0x7fb47400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.447+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47400ee40 con 0x7fb47c0c2a40 2024-03-09T09:16:36.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.447+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7fb47400ee40 con 0x7fb47c0c2a40 2024-03-09T09:16:36.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.447+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0c2a40 msgr2=0x7fb47c148ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.447+0000 7fb473400640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0c2a40 0x7fb47c148ad0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb474007cb0 tx=0x7fb47400e4b0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.447+0000 7fb473400640 1 -- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0c2a40 msgr2=0x7fb47c148ad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:36.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.447+0000 7fb473400640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb47c0c2a40 0x7fb47c148ad0 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb474007cb0 tx=0x7fb47400e4b0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.453 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 180, 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:16:36.454 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:16:36.455 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:16:36.456 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:36.457 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:16:36.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:36.459 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.460 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.461 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.462 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:36.463 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.464 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:16:36.465 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:36.466 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.467 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.467 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:16:36.467 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:16:36.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.448+0000 7fb482800640 1 -- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb45c06afe0 msgr2=0x7fb45c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.448+0000 7fb482800640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb45c06afe0 0x7fb45c06d4a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb47c155fa0 tx=0x7fb468041000 comp rx=0 tx=0).stop 2024-03-09T09:16:36.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.448+0000 7fb482800640 1 -- 172.21.15.67:0/578391039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb47c0c5f50 msgr2=0x7fb47c154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.448+0000 7fb482800640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb47c0c5f50 0x7fb47c154ce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00c3c0 tx=0x7fb46c00c880 comp rx=0 tx=0).stop 2024-03-09T09:16:36.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.448+0000 7fb47be00640 1 -- 172.21.15.67:0/578391039 reap_dead start 2024-03-09T09:16:36.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.448+0000 7fb482800640 1 -- 172.21.15.67:0/578391039 shutdown_connections 2024-03-09T09:16:36.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.448+0000 7fb482800640 1 --2- 172.21.15.67:0/578391039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb47c0c5f50 0x7fb47c154ce0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.448+0000 7fb482800640 1 -- 172.21.15.67:0/578391039 >> 172.21.15.67:0/578391039 conn(0x7fb47c0c8050 msgr2=0x7fb47c14df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:36.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.448+0000 7fb482800640 1 -- 172.21.15.67:0/578391039 shutdown_connections 2024-03-09T09:16:36.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.448+0000 7fb482800640 1 -- 172.21.15.67:0/578391039 wait complete. 2024-03-09T09:16:36.470 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:16:36.471 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:16:36.471 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:16:36.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.683+0000 7fa6c6400640 1 Processor -- start 2024-03-09T09:16:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.683+0000 7fa6c6400640 1 -- start start 2024-03-09T09:16:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.685+0000 7fa6c6400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6c00c9380 0x7fa6c00acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.685+0000 7fa6c6400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c00c9890 con 0x7fa6c00ad440 2024-03-09T09:16:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.685+0000 7fa6c6400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c00c5f80 con 0x7fa6c014fd90 2024-03-09T09:16:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.685+0000 7fa6c6400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c006ba60 con 0x7fa6c00c9380 2024-03-09T09:16:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.685+0000 7fa6bfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa6c00c9380 0x7fa6c00acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.685+0000 7fa6c5a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa6c00ad440 0x7fa6c006c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52508/0 (socket says 172.21.15.67:52508) 2024-03-09T09:16:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.685+0000 7fa6c5a00640 1 -- 172.21.15.67:0/731956823 learned_addr learned my addr 172.21.15.67:0/731956823 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:16:36.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.686+0000 7fa6bfe00640 1 -- 172.21.15.67:0/731956823 >> v1:172.21.15.134:6789/0 conn(0x7fa6c014fd90 legacy=0x7fa6c0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:36.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.686+0000 7fa6bfe00640 1 -- 172.21.15.67:0/731956823 >> v1:172.21.15.67:6789/0 conn(0x7fa6c00ad440 legacy=0x7fa6c006c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.686+0000 7fa6bfe00640 1 -- 172.21.15.67:0/731956823 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c00c6240 con 0x7fa6c00c9380 2024-03-09T09:16:36.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.686+0000 7fa6bf400640 1 -- 172.21.15.67:0/731956823 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917659968 0 0) 0x7fa6c00c9890 con 0x7fa6c00ad440 2024-03-09T09:16:36.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.687+0000 7fa6bfe00640 1 --2- 172.21.15.67:0/731956823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c00c9380 0x7fa6c00acf00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa6c006bbe0 tx=0x7fa6ac02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69312932907f3ab server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.687+0000 7fa6bf400640 1 -- 172.21.15.67:0/731956823 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac038070 con 0x7fa6c00c9380 2024-03-09T09:16:36.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.687+0000 7fa6bf400640 1 -- 172.21.15.67:0/731956823 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac031700 con 0x7fa6c00c9380 2024-03-09T09:16:36.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.687+0000 7fa6bf400640 1 -- 172.21.15.67:0/731956823 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac031ad0 con 0x7fa6c00c9380 2024-03-09T09:16:36.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.688+0000 7fa6c6400640 1 -- 172.21.15.67:0/731956823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c00c9380 msgr2=0x7fa6c00acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.688+0000 7fa6c6400640 1 --2- 172.21.15.67:0/731956823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c00c9380 0x7fa6c00acf00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa6c006bbe0 tx=0x7fa6ac02b7a0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.688+0000 7fa6c6400640 1 -- 172.21.15.67:0/731956823 shutdown_connections 2024-03-09T09:16:36.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.688+0000 7fa6c6400640 1 --2- 172.21.15.67:0/731956823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c00c9380 0x7fa6c00acf00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.688+0000 7fa6c6400640 1 -- 172.21.15.67:0/731956823 >> 172.21.15.67:0/731956823 conn(0x7fa6c00cb480 msgr2=0x7fa6c00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:36.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.688+0000 7fa6c6400640 1 -- 172.21.15.67:0/731956823 shutdown_connections 2024-03-09T09:16:36.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.688+0000 7fa6c6400640 1 -- 172.21.15.67:0/731956823 wait complete. 2024-03-09T09:16:36.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.690+0000 7fa6c6400640 1 Processor -- start 2024-03-09T09:16:36.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.690+0000 7fa6c6400640 1 -- start start 2024-03-09T09:16:36.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.691+0000 7fa6c6400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6c00ad440 0x7fa6c00af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.691+0000 7fa6c6400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c00c9380 0x7fa6c0157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.691+0000 7fa6c6400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c014fd90 0x7fa6c0159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.691+0000 7fa6c6400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c006c8e0 con 0x7fa6c014fd90 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.691+0000 7fa6c6400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c006bcb0 con 0x7fa6c00ad440 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.691+0000 7fa6c6400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c006ca60 con 0x7fa6c00c9380 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.691+0000 7fa6c5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c014fd90 0x7fa6c0159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.691+0000 7fa6c5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c014fd90 0x7fa6c0159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50532/0 (socket says 172.21.15.67:50532) 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.691+0000 7fa6c5a00640 1 -- 172.21.15.67:0/1116078795 learned_addr learned my addr 172.21.15.67:0/1116078795 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.691+0000 7fa6c5000640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6c00ad440 0x7fa6c00af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.691+0000 7fa6bfe00640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c00c9380 0x7fa6c0157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.692+0000 7fa6c5a00640 1 -- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c00c9380 msgr2=0x7fa6c0157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.692+0000 7fa6c5a00640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c00c9380 0x7fa6c0157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.692+0000 7fa6c5a00640 1 -- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6c00ad440 msgr2=0x7fa6c00af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.692+0000 7fa6c5a00640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6c00ad440 0x7fa6c00af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.692+0000 7fa6c5a00640 1 -- 172.21.15.67:0/1116078795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c00ac300 con 0x7fa6c014fd90 2024-03-09T09:16:36.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.692+0000 7fa6c5a00640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c014fd90 0x7fa6c0159a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800c7a0 tx=0x7fa6b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.692+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8018070 con 0x7fa6c014fd90 2024-03-09T09:16:36.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.692+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b800a640 con 0x7fa6c014fd90 2024-03-09T09:16:36.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.692+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b800aab0 con 0x7fa6c014fd90 2024-03-09T09:16:36.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.692+0000 7fa6c6400640 1 -- 172.21.15.67:0/1116078795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c00ab3d0 con 0x7fa6c014fd90 2024-03-09T09:16:36.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.692+0000 7fa6c6400640 1 -- 172.21.15.67:0/1116078795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c00ab990 con 0x7fa6c014fd90 2024-03-09T09:16:36.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.693+0000 7fa6c6400640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c0156ff0 0x7fa6c01eec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.693+0000 7fa6c6400640 1 -- 172.21.15.67:0/1116078795 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c004f3c0 con 0x7fa6c0156ff0 2024-03-09T09:16:36.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.694+0000 7fa6c5000640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c0156ff0 0x7fa6c01eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.694+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa6b801b070 con 0x7fa6c014fd90 2024-03-09T09:16:36.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.695+0000 7fa6c5000640 1 -- 172.21.15.67:0/1116078795 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6c00b0f20 con 0x7fa6c0156ff0 2024-03-09T09:16:36.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.695+0000 7fa6c5000640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c0156ff0 0x7fa6c01eec10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400d500 tx=0x7fa6b400d9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.695+0000 7fa6bcc00640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa6a006b500 0x7fa6a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.696+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa6b8089b30 con 0x7fa6c014fd90 2024-03-09T09:16:36.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.696+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4018020 con 0x7fa6c0156ff0 2024-03-09T09:16:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.696+0000 7fa6bfe00640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa6a006b500 0x7fa6a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.697+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa6b4014620 con 0x7fa6c0156ff0 2024-03-09T09:16:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.697+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c0156ff0 msgr2=0x7fa6c01eec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.697+0000 7fa6bcc00640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c0156ff0 0x7fa6c01eec10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400d500 tx=0x7fa6b400d9c0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.697+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c0156ff0 msgr2=0x7fa6c01eec10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:36.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.697+0000 7fa6bcc00640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c0156ff0 0x7fa6c01eec10 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400d500 tx=0x7fa6b400d9c0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.697+0000 7fa6bfe00640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa6a006b500 0x7fa6a006d9c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac035000 tx=0x7fa6ac008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.704+0000 7fa6c6400640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c01ef380 0x7fa6c01ef760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:36.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.704+0000 7fa6c6400640 1 -- 172.21.15.67:0/1116078795 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c006d2d0 con 0x7fa6c01ef380 2024-03-09T09:16:36.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.704+0000 7fa6c5000640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c01ef380 0x7fa6c01ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:36.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.705+0000 7fa6c5000640 1 -- 172.21.15.67:0/1116078795 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa6c00c69e0 con 0x7fa6c01ef380 2024-03-09T09:16:36.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.705+0000 7fa6c5000640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c01ef380 0x7fa6c01ef760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400d500 tx=0x7fa6b40046f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:36.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.706+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4026070 con 0x7fa6c01ef380 2024-03-09T09:16:36.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7fa6b401fa60 con 0x7fa6c01ef380 2024-03-09T09:16:36.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c01ef380 msgr2=0x7fa6c01ef760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6bcc00640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c01ef380 0x7fa6c01ef760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400d500 tx=0x7fa6b40046f0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6bcc00640 1 -- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c01ef380 msgr2=0x7fa6c01ef760 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:36.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6bcc00640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6c01ef380 0x7fa6c01ef760 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400d500 tx=0x7fa6b40046f0 comp rx=0 tx=0).stop 2024-03-09T09:16:36.713 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:16:36.713 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:16:36.713 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:16:36.713 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:16:36.713 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 180, 2024-03-09T09:16:36.713 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:16:36.713 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:16:36.713 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:16:36.713 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:36.714 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:36.715 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:36.716 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:36.717 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:36.718 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:16:36.719 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.720 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:36.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:16:36.722 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:16:36.723 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.724 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.725 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:16:36.726 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:16:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6c6400640 1 -- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa6a006b500 msgr2=0x7fa6a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6c6400640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa6a006b500 0x7fa6a006d9c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac035000 tx=0x7fa6ac008030 comp rx=0 tx=0).stop 2024-03-09T09:16:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6c6400640 1 -- 172.21.15.67:0/1116078795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c014fd90 msgr2=0x7fa6c0159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6c6400640 1 --2- 172.21.15.67:0/1116078795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c014fd90 0x7fa6c0159a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800c7a0 tx=0x7fa6b800cc60 comp rx=0 tx=0).stop 2024-03-09T09:16:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6c5a00640 1 -- 172.21.15.67:0/1116078795 reap_dead start 2024-03-09T09:16:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6c5a00640 1 -- 172.21.15.67:0/1116078795 reap_dead start 2024-03-09T09:16:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6c6400640 1 -- 172.21.15.67:0/1116078795 shutdown_connections 2024-03-09T09:16:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6c6400640 1 -- 172.21.15.67:0/1116078795 >> 172.21.15.67:0/1116078795 conn(0x7fa6c00cb480 msgr2=0x7fa6c014e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:36.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6c6400640 1 -- 172.21.15.67:0/1116078795 shutdown_connections 2024-03-09T09:16:36.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:36.707+0000 7fa6c6400640 1 -- 172.21.15.67:0/1116078795 wait complete. 2024-03-09T09:16:36.729 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:16:36.729 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-03-09T09:16:36.729 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:16:36.729 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-03-09T09:16:36.729 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:16:36.730 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-09T09:16:36.730 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:16:36.769 INFO:tasks.ceph.mon.b:Stopped 2024-03-09T09:16:36.769 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:16:36.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:36.913 INFO:tasks.workunit.client.0.smithi067.stdout:v0.89 2024-03-09T09:16:36.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:16:36.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 89 -eq 48 ']' 2024-03-09T09:16:36.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.89 2024-03-09T09:16:36.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.89 2024-03-09T09:16:36.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.89 2024-03-09T09:16:37.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.001+0000 7f1041c00640 1 Processor -- start 2024-03-09T09:16:37.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.002+0000 7f1041c00640 1 -- start start 2024-03-09T09:16:37.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.004+0000 7f103be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f103c153300 0x7f103c0b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:52510/0 (socket says 172.21.15.67:52510) 2024-03-09T09:16:37.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.004+0000 7f1041c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f103c0cadb0 0x7f103c156720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:37.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.004+0000 7f1041c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f103c157000 con 0x7f103c153300 2024-03-09T09:16:37.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.004+0000 7f1041c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f103c0ae430 con 0x7f103c0c5e40 2024-03-09T09:16:37.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.004+0000 7f1041c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103c0ae6f0 con 0x7f103c0cadb0 2024-03-09T09:16:37.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.004+0000 7f103be00640 1 -- 172.21.15.67:0/3832949787 learned_addr learned my addr 172.21.15.67:0/3832949787 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:16:37.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.004+0000 7f1041200640 1 -- 172.21.15.67:0/3832949787 >> v2:172.21.15.134:3300/0 conn(0x7f103c0cadb0 msgr2=0x7f103c156720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:37.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.004+0000 7f1041200640 1 --2- 172.21.15.67:0/3832949787 >> v2:172.21.15.134:3300/0 conn(0x7f103c0cadb0 0x7f103c156720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:37.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.005+0000 7f103b400640 1 -- 172.21.15.67:0/3832949787 >> v1:172.21.15.134:6789/0 conn(0x7f103c0c5e40 legacy=0x7f103c156010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:16:37.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.005+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094243651 0 0) 0x7f103c157000 con 0x7f103c153300 2024-03-09T09:16:37.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.005+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1020003520 con 0x7f103c153300 2024-03-09T09:16:37.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.005+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2607065446 0 0) 0x7f1020003520 con 0x7f103c153300 2024-03-09T09:16:37.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.005+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 >> v2:172.21.15.134:3300/0 conn(0x7f103c0cadb0 msgr2=0x7f103c156720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:37.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.005+0000 7f103aa00640 1 --2- 172.21.15.67:0/3832949787 >> v2:172.21.15.134:3300/0 conn(0x7f103c0cadb0 0x7f103c156720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:37.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.005+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 >> v1:172.21.15.134:6789/0 conn(0x7f103c0c5e40 legacy=0x7f103c156010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:37.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.005+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c155b10 con 0x7f103c153300 2024-03-09T09:16:37.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.006+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f102c0015a0 con 0x7f103c153300 2024-03-09T09:16:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.006+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 >> v1:172.21.15.67:6789/0 conn(0x7f103c153300 legacy=0x7f103c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.006+0000 7f103aa00640 1 --2- 172.21.15.67:0/3832949787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1020003a20 0x7f1020003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.006+0000 7f103aa00640 1 --2- 172.21.15.67:0/3832949787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1020004800 0x7f1020004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.006+0000 7f103aa00640 1 --2- 172.21.15.67:0/3832949787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10200052e0 0x7f102000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.006+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c0ae6f0 con 0x7f1020004800 2024-03-09T09:16:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.006+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102000c1b0 con 0x7f1020003a20 2024-03-09T09:16:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.006+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102000c3b0 con 0x7f10200052e0 2024-03-09T09:16:37.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.006+0000 7f103be00640 1 -- 172.21.15.67:0/3832949787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1020003a20 msgr2=0x7f1020003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.006+0000 7f103be00640 1 --2- 172.21.15.67:0/3832949787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1020003a20 0x7f1020003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.006+0000 7f103b400640 1 --2- 172.21.15.67:0/3832949787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1020004800 0x7f1020004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.007+0000 7f1041200640 1 --2- 172.21.15.67:0/3832949787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10200052e0 0x7f102000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.007+0000 7f103b400640 1 -- 172.21.15.67:0/3832949787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10200052e0 msgr2=0x7f102000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.007+0000 7f103b400640 1 --2- 172.21.15.67:0/3832949787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10200052e0 0x7f102000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.007+0000 7f103b400640 1 -- 172.21.15.67:0/3832949787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1020003a20 msgr2=0x7f1020003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.007+0000 7f103b400640 1 --2- 172.21.15.67:0/3832949787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1020003a20 0x7f1020003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.007+0000 7f103b400640 1 -- 172.21.15.67:0/3832949787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c155b10 con 0x7f1020004800 2024-03-09T09:16:37.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.007+0000 7f1041200640 1 -- 172.21.15.67:0/3832949787 reap_dead start 2024-03-09T09:16:37.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.009+0000 7f103b400640 1 --2- 172.21.15.67:0/3832949787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1020004800 0x7f1020004c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1028003930 tx=0x7f1020005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cba02713d09a55 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:37.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.010+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028009500 con 0x7f1020004800 2024-03-09T09:16:37.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.010+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1028003bb0 con 0x7f1020004800 2024-03-09T09:16:37.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.010+0000 7f103aa00640 1 -- 172.21.15.67:0/3832949787 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028005070 con 0x7f1020004800 2024-03-09T09:16:37.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.010+0000 7f1041c00640 1 -- 172.21.15.67:0/3832949787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1020004800 msgr2=0x7f1020004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:37.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.011+0000 7f1041c00640 1 --2- 172.21.15.67:0/3832949787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1020004800 0x7f1020004c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1028003930 tx=0x7f1020005830 comp rx=0 tx=0).stop 2024-03-09T09:16:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.011+0000 7f1041c00640 1 -- 172.21.15.67:0/3832949787 shutdown_connections 2024-03-09T09:16:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.011+0000 7f1041c00640 1 --2- 172.21.15.67:0/3832949787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1020004800 0x7f1020004c80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.011+0000 7f1041c00640 1 -- 172.21.15.67:0/3832949787 >> 172.21.15.67:0/3832949787 conn(0x7f103c0732d0 msgr2=0x7f103c0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.011+0000 7f1041c00640 1 -- 172.21.15.67:0/3832949787 shutdown_connections 2024-03-09T09:16:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.011+0000 7f1041c00640 1 -- 172.21.15.67:0/3832949787 wait complete. 2024-03-09T09:16:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.013+0000 7f1041c00640 1 Processor -- start 2024-03-09T09:16:37.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.013+0000 7f1041c00640 1 -- start start 2024-03-09T09:16:37.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.014+0000 7f1041c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f103c1eec90 0x7f103c0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:37.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.014+0000 7f1041c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f103c1ef070 0x7f103c06bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:37.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.014+0000 7f1041c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c06c0c0 0x7f103c14b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:37.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.014+0000 7f1041c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c0c59e0 con 0x7f103c06c0c0 2024-03-09T09:16:37.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.014+0000 7f1041c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c159e90 con 0x7f103c1ef070 2024-03-09T09:16:37.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.014+0000 7f1041c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103c0b0000 con 0x7f103c1eec90 2024-03-09T09:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.015+0000 7f103b400640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f103c1ef070 msgr2=0x7f103c06bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.015+0000 7f1041200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c06c0c0 0x7f103c14b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.015+0000 7f103b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f103c1ef070 0x7f103c06bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.015+0000 7f1041200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c06c0c0 0x7f103c14b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50610/0 (socket says 172.21.15.67:50610) 2024-03-09T09:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.015+0000 7f1041200640 1 -- 172.21.15.67:0/3594530783 learned_addr learned my addr 172.21.15.67:0/3594530783 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.015+0000 7f103be00640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f103c1eec90 0x7f103c0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.016+0000 7f1041200640 1 -- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f103c1eec90 msgr2=0x7f103c0aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.016+0000 7f1041200640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f103c1eec90 0x7f103c0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.016+0000 7f1041200640 1 -- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f103c1ef070 msgr2=0x7f103c06bb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:16:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.016+0000 7f1041200640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f103c1ef070 0x7f103c06bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.016+0000 7f1041200640 1 -- 172.21.15.67:0/3594530783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c150fd0 con 0x7f103c06c0c0 2024-03-09T09:16:37.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.016+0000 7f1041200640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c06c0c0 0x7f103c14b7f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1034003cb0 tx=0x7f103400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:37.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.016+0000 7f1033e00640 1 -- 172.21.15.67:0/3594530783 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103400e400 con 0x7f103c06c0c0 2024-03-09T09:16:37.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.017+0000 7f1033e00640 1 -- 172.21.15.67:0/3594530783 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103400e5c0 con 0x7f103c06c0c0 2024-03-09T09:16:37.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.017+0000 7f1033e00640 1 -- 172.21.15.67:0/3594530783 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103400e9b0 con 0x7f103c06c0c0 2024-03-09T09:16:37.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.017+0000 7f1041c00640 1 -- 172.21.15.67:0/3594530783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103c0c7810 con 0x7f103c06c0c0 2024-03-09T09:16:37.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.017+0000 7f1041c00640 1 -- 172.21.15.67:0/3594530783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103c06b660 con 0x7f103c06c0c0 2024-03-09T09:16:37.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.020+0000 7f1041c00640 1 -- 172.21.15.67:0/3594530783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103c06b860 con 0x7f103c06c0c0 2024-03-09T09:16:37.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.020+0000 7f1033e00640 1 -- 172.21.15.67:0/3594530783 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f103400eb50 con 0x7f103c06c0c0 2024-03-09T09:16:37.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.022+0000 7f1033e00640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f101c06b4d0 0x7f101c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:37.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.022+0000 7f1033e00640 1 -- 172.21.15.67:0/3594530783 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f103408aa20 con 0x7f103c06c0c0 2024-03-09T09:16:37.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.023+0000 7f103be00640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f101c06b4d0 0x7f101c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:37.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.024+0000 7f103be00640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f101c06b4d0 0x7f101c06d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f102c001710 tx=0x7f102c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:37.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.029+0000 7f1033e00640 1 -- 172.21.15.67:0/3594530783 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f103c06b860 con 0x7f103c06c0c0 2024-03-09T09:16:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.437+0000 7f1041c00640 1 -- 172.21.15.67:0/3594530783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f103c04f3c0 con 0x7f103c06c0c0 2024-03-09T09:16:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.437+0000 7f1033e00640 1 -- 172.21.15.67:0/3594530783 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f1034059520 con 0x7f103c06c0c0 2024-03-09T09:16:37.443 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":180,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 -- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f101c06b4d0 msgr2=0x7f101c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f101c06b4d0 0x7f101c06d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f102c001710 tx=0x7f102c009040 comp rx=0 tx=0).stop 2024-03-09T09:16:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 -- 172.21.15.67:0/3594530783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c06c0c0 msgr2=0x7f103c14b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c06c0c0 0x7f103c14b7f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1034003cb0 tx=0x7f103400b6e0 comp rx=0 tx=0).stop 2024-03-09T09:16:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 -- 172.21.15.67:0/3594530783 shutdown_connections 2024-03-09T09:16:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f101c06b4d0 0x7f101c06d990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c06c0c0 0x7f103c14b7f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f103c1ef070 0x7f103c06bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 --2- 172.21.15.67:0/3594530783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f103c1eec90 0x7f103c0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 -- 172.21.15.67:0/3594530783 >> 172.21.15.67:0/3594530783 conn(0x7f103c0732d0 msgr2=0x7f103c0c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:37.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 -- 172.21.15.67:0/3594530783 shutdown_connections 2024-03-09T09:16:37.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:37.441+0000 7f1041c00640 1 -- 172.21.15.67:0/3594530783 wait complete. 2024-03-09T09:16:38.879 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 27% (91/335) Updating files: 28% (94/335) Updating files: 29% (98/335) Updating files: 30% (101/335) Updating files: 31% (104/335) Updating files: 32% (108/335) Updating files: 33% (111/335) Updating files: 34% (114/335) Updating files: 35% (118/335) Updating files: 36% (121/335) Updating files: 37% (124/335) Updating files: 38% (128/335) Updating files: 39% (131/335) Updating files: 40% (134/335) Updating files: 41% (138/335) Updating files: 42% (141/335) Updating files: 43% (145/335) Updating files: 44% (148/335) Updating files: 44% (150/335) Updating files: 45% (151/335) Updating files: 46% (155/335) Updating files: 47% (158/335) Updating files: 48% (161/335) Updating files: 49% (165/335) Updating files: 50% (168/335) Updating files: 51% (171/335) Updating files: 52% (175/335) Updating files: 53% (178/335) Updating files: 54% (181/335) Updating files: 55% (185/335) Updating files: 56% (188/335) Updating files: 57% (191/335) Updating files: 58% (195/335) Updating files: 59% (198/335) Updating files: 60% (201/335) Updating files: 61% (205/335) Updating files: 62% (208/335) Updating files: 63% (212/335) Updating files: 64% (215/335) Updating files: 65% (218/335) Updating files: 65% (221/335) Updating files: 66% (222/335) Updating files: 67% (225/335) Updating files: 68% (228/335) Updating files: 69% (232/335) Updating files: 70% (235/335) Updating files: 71% (238/335) Updating files: 72% (242/335) Updating files: 73% (245/335) Updating files: 74% (248/335) Updating files: 75% (252/335) Updating files: 76% (255/335) Updating files: 77% (258/335) Updating files: 78% (262/335) Updating files: 79% (265/335) Updating files: 80% (268/335) Updating files: 81% (272/335) Updating files: 82% (275/335) Updating files: 83% (279/335) Updating files: 84% (282/335) Updating files: 85% (285/335) Updating files: 86% (289/335) Updating files: 87% (292/335) Updating files: 88% (295/335) Updating files: 89% (299/335) Updating files: 90% (302/335) Updating files: 91% (305/335) Updating files: 91% (307/335) Updating files: 92% (309/335) Updating files: 93% (312/335) Updating files: 94% (315/335) Updating files: 95% (319/335) Updating files: 96% (322/335) Updating files: 97% (325/335) Updating files: 98% (329/335) Updating files: 99% (332/335) Updating files: 100% (335/335) Updating files: 100% (335/335), done. 2024-03-09T09:16:39.056 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 4be687bf448 0.88 2024-03-09T09:16:39.076 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.88 2024-03-09T09:16:39.717 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:16:39.717 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 89 -eq 48 ']' 2024-03-09T09:16:39.717 INFO:tasks.workunit.client.1.smithi134.stdout:v0.89 2024-03-09T09:16:39.717 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.89 2024-03-09T09:16:39.718 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.89 2024-03-09T09:16:39.718 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.89 2024-03-09T09:16:40.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.656+0000 7f3c45400640 1 Processor -- start 2024-03-09T09:16:40.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.656+0000 7f3c45400640 1 -- start start 2024-03-09T09:16:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.657+0000 7f3c45400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c400c1f70 0x7f3c40150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.657+0000 7f3c45400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3c400710a0 0x7f3c40151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.657+0000 7f3c45400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c400c27a0 0x7f3c40153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.657+0000 7f3c45400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c400c81e0 con 0x7f3c400c27a0 2024-03-09T09:16:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.657+0000 7f3c45400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c400c32b0 con 0x7f3c400710a0 2024-03-09T09:16:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.657+0000 7f3c45400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400c2fd0 con 0x7f3c400c1f70 2024-03-09T09:16:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.658+0000 7f3c3f400640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f3c400710a0 msgr2=0x7f3c40151190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.658+0000 7f3c3f400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3c400710a0 0x7f3c40151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.658+0000 7f3c44a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c400c27a0 0x7f3c40153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.658+0000 7f3c44a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f3c400c27a0 0x7f3c40153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50878/0 (socket says 172.21.15.67:50878) 2024-03-09T09:16:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.658+0000 7f3c44a00640 1 -- 172.21.15.67:0/822926671 learned_addr learned my addr 172.21.15.67:0/822926671 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.658+0000 7f3c3fe00640 1 --2- 172.21.15.67:0/822926671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c400c1f70 0x7f3c40150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.658+0000 7f3c44a00640 1 -- 172.21.15.67:0/822926671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c400c1f70 msgr2=0x7f3c40150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.658+0000 7f3c44a00640 1 --2- 172.21.15.67:0/822926671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c400c1f70 0x7f3c40150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.658+0000 7f3c44a00640 1 -- 172.21.15.67:0/822926671 >> v2:172.21.15.134:3300/0 conn(0x7f3c400710a0 msgr2=0x7f3c40151190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.658+0000 7f3c44a00640 1 --2- 172.21.15.67:0/822926671 >> v2:172.21.15.134:3300/0 conn(0x7f3c400710a0 0x7f3c40151190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.658+0000 7f3c44a00640 1 -- 172.21.15.67:0/822926671 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400c3700 con 0x7f3c400c27a0 2024-03-09T09:16:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.659+0000 7f3c44a00640 1 --2- 172.21.15.67:0/822926671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c400c27a0 0x7f3c40153970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c4006bfd0 tx=0x7f3c3802be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf1069a8a0eaf20 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c3ea00640 1 -- 172.21.15.67:0/822926671 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3802ca10 con 0x7f3c400c27a0 2024-03-09T09:16:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c3ea00640 1 -- 172.21.15.67:0/822926671 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3802cbb0 con 0x7f3c400c27a0 2024-03-09T09:16:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c3ea00640 1 -- 172.21.15.67:0/822926671 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c380343f0 con 0x7f3c400c27a0 2024-03-09T09:16:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c45400640 1 -- 172.21.15.67:0/822926671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c400c27a0 msgr2=0x7f3c40153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c45400640 1 --2- 172.21.15.67:0/822926671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c400c27a0 0x7f3c40153970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c4006bfd0 tx=0x7f3c3802be70 comp rx=0 tx=0).stop 2024-03-09T09:16:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c45400640 1 -- 172.21.15.67:0/822926671 shutdown_connections 2024-03-09T09:16:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c45400640 1 --2- 172.21.15.67:0/822926671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c400c27a0 0x7f3c40153970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c45400640 1 --2- 172.21.15.67:0/822926671 >> v2:172.21.15.134:3300/0 conn(0x7f3c400710a0 0x7f3c40151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c45400640 1 --2- 172.21.15.67:0/822926671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c400c1f70 0x7f3c40150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c45400640 1 -- 172.21.15.67:0/822926671 >> 172.21.15.67:0/822926671 conn(0x7f3c400c7300 msgr2=0x7f3c400c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c45400640 1 -- 172.21.15.67:0/822926671 shutdown_connections 2024-03-09T09:16:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.660+0000 7f3c45400640 1 -- 172.21.15.67:0/822926671 wait complete. 2024-03-09T09:16:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.662+0000 7f3c45400640 1 Processor -- start 2024-03-09T09:16:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.662+0000 7f3c45400640 1 -- start start 2024-03-09T09:16:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.663+0000 7f3c45400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c400710a0 0x7f3c400c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.663+0000 7f3c45400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c400c1f70 0x7f3c40148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.663+0000 7f3c45400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c400c27a0 0x7f3c401485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.663+0000 7f3c45400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c40070d50 con 0x7f3c400c1f70 2024-03-09T09:16:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.663+0000 7f3c45400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c40070830 con 0x7f3c400710a0 2024-03-09T09:16:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.663+0000 7f3c45400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c40070ed0 con 0x7f3c400c27a0 2024-03-09T09:16:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.663+0000 7f3c3f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c400c1f70 0x7f3c40148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.663+0000 7f3c3f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c400c1f70 0x7f3c40148070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50888/0 (socket says 172.21.15.67:50888) 2024-03-09T09:16:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.663+0000 7f3c3f400640 1 -- 172.21.15.67:0/3973293947 learned_addr learned my addr 172.21.15.67:0/3973293947 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.663+0000 7f3c3fe00640 1 -- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c400710a0 msgr2=0x7f3c400c6390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.664+0000 7f3c3fe00640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c400710a0 0x7f3c400c6390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.664+0000 7f3c3f400640 1 -- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c400c27a0 msgr2=0x7f3c401485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.664+0000 7f3c3f400640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c400c27a0 0x7f3c401485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.664+0000 7f3c3f400640 1 -- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c400710a0 msgr2=0x7f3c400c6390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:16:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.664+0000 7f3c3f400640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c400710a0 0x7f3c400c6390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.664+0000 7f3c3f400640 1 -- 172.21.15.67:0/3973293947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400c70b0 con 0x7f3c400c1f70 2024-03-09T09:16:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.664+0000 7f3c3f400640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c400c1f70 0x7f3c40148070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00b580 tx=0x7f3c2c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.664+0000 7f3c37e00640 1 -- 172.21.15.67:0/3973293947 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c013020 con 0x7f3c400c1f70 2024-03-09T09:16:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.664+0000 7f3c37e00640 1 -- 172.21.15.67:0/3973293947 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c003de0 con 0x7f3c400c1f70 2024-03-09T09:16:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.664+0000 7f3c37e00640 1 -- 172.21.15.67:0/3973293947 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0052b0 con 0x7f3c400c1f70 2024-03-09T09:16:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.664+0000 7f3c45400640 1 -- 172.21.15.67:0/3973293947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4014a960 con 0x7f3c400c1f70 2024-03-09T09:16:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.665+0000 7f3c45400640 1 -- 172.21.15.67:0/3973293947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4014af20 con 0x7f3c400c1f70 2024-03-09T09:16:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.666+0000 7f3c45400640 1 -- 172.21.15.67:0/3973293947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c400679a0 con 0x7f3c400c1f70 2024-03-09T09:16:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.667+0000 7f3c37e00640 1 -- 172.21.15.67:0/3973293947 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3c2c016020 con 0x7f3c400c1f70 2024-03-09T09:16:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.668+0000 7f3c37e00640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c2006b480 0x7f3c2006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.668+0000 7f3c37e00640 1 -- 172.21.15.67:0/3973293947 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f3c2c08a3e0 con 0x7f3c400c1f70 2024-03-09T09:16:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.669+0000 7f3c3fe00640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c2006b480 0x7f3c2006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.670+0000 7f3c3fe00640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c2006b480 0x7f3c2006d940 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3c30006fd0 tx=0x7f3c30008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.673+0000 7f3c37e00640 1 -- 172.21.15.67:0/3973293947 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3c400679a0 con 0x7f3c400c1f70 2024-03-09T09:16:40.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.976+0000 7f3c45400640 1 -- 172.21.15.67:0/3973293947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3c4004f3c0 con 0x7f3c400c1f70 2024-03-09T09:16:40.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.977+0000 7f3c37e00640 1 -- 172.21.15.67:0/3973293947 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f3c2c01d020 con 0x7f3c400c1f70 2024-03-09T09:16:40.983 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":180,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:40.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.980+0000 7f3c45400640 1 -- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c2006b480 msgr2=0x7f3c2006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:40.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.980+0000 7f3c45400640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c2006b480 0x7f3c2006d940 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3c30006fd0 tx=0x7f3c30008040 comp rx=0 tx=0).stop 2024-03-09T09:16:40.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.980+0000 7f3c45400640 1 -- 172.21.15.67:0/3973293947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c400c1f70 msgr2=0x7f3c40148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:40.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.980+0000 7f3c45400640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c400c1f70 0x7f3c40148070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00b580 tx=0x7f3c2c00ba40 comp rx=0 tx=0).stop 2024-03-09T09:16:40.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.980+0000 7f3c45400640 1 -- 172.21.15.67:0/3973293947 shutdown_connections 2024-03-09T09:16:40.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.980+0000 7f3c45400640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c2006b480 0x7f3c2006d940 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:40.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.980+0000 7f3c45400640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c400c27a0 0x7f3c401485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:40.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.980+0000 7f3c45400640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c400c1f70 0x7f3c40148070 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:40.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.980+0000 7f3c45400640 1 --2- 172.21.15.67:0/3973293947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c400710a0 0x7f3c400c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:40.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.980+0000 7f3c45400640 1 -- 172.21.15.67:0/3973293947 >> 172.21.15.67:0/3973293947 conn(0x7f3c400c7300 msgr2=0x7f3c400728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:40.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.980+0000 7f3c45400640 1 -- 172.21.15.67:0/3973293947 shutdown_connections 2024-03-09T09:16:40.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:40.981+0000 7f3c45400640 1 -- 172.21.15.67:0/3973293947 wait complete. 2024-03-09T09:16:44.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:44.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.196+0000 7f89be000640 1 Processor -- start 2024-03-09T09:16:44.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.197+0000 7f89be000640 1 -- start start 2024-03-09T09:16:44.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.198+0000 7f89be000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f89b81510f0 0x7f89b80a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.198+0000 7f89be000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f89b80c20f0 0x7f89b80a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.198+0000 7f89be000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b80a9bd0 0x7f89b806db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.198+0000 7f89be000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b8153b40 con 0x7f89b80c20f0 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.198+0000 7f89be000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b8153e20 con 0x7f89b81510f0 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.198+0000 7f89be000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b8154100 con 0x7f89b80a9bd0 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.198+0000 7f89bcc00640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f89b81510f0 msgr2=0x7f89b80a6d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.198+0000 7f89b7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f89b80c20f0 0x7f89b80a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.198+0000 7f89bcc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f89b81510f0 0x7f89b80a6d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.198+0000 7f89b7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f89b80c20f0 0x7f89b80a9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50898/0 (socket says 172.21.15.67:50898) 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.198+0000 7f89b7e00640 1 -- 172.21.15.67:0/2898879402 learned_addr learned my addr 172.21.15.67:0/2898879402 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.199+0000 7f89bd600640 1 --2- 172.21.15.67:0/2898879402 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b80a9bd0 0x7f89b806db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.199+0000 7f89b7e00640 1 -- 172.21.15.67:0/2898879402 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b80a9bd0 msgr2=0x7f89b806db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:44.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.199+0000 7f89b7e00640 1 --2- 172.21.15.67:0/2898879402 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b80a9bd0 0x7f89b806db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.199+0000 7f89b7e00640 1 -- 172.21.15.67:0/2898879402 >> v2:172.21.15.134:3300/0 conn(0x7f89b81510f0 msgr2=0x7f89b80a6d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:44.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.199+0000 7f89b7e00640 1 --2- 172.21.15.67:0/2898879402 >> v2:172.21.15.134:3300/0 conn(0x7f89b81510f0 0x7f89b80a6d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.199+0000 7f89b7e00640 1 -- 172.21.15.67:0/2898879402 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b81543e0 con 0x7f89b80c20f0 2024-03-09T09:16:44.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.199+0000 7f89bd600640 1 --2- 172.21.15.67:0/2898879402 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b80a9bd0 0x7f89b806db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:16:44.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.201+0000 7f89b7e00640 1 --2- 172.21.15.67:0/2898879402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89b80c20f0 0x7f89b80a9690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89b806f900 tx=0x7f89a402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d17e19c415bc33d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:44.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.201+0000 7f89b7400640 1 -- 172.21.15.67:0/2898879402 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4035020 con 0x7f89b80c20f0 2024-03-09T09:16:44.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.201+0000 7f89b7400640 1 -- 172.21.15.67:0/2898879402 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4031800 con 0x7f89b80c20f0 2024-03-09T09:16:44.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.201+0000 7f89b7400640 1 -- 172.21.15.67:0/2898879402 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4031bd0 con 0x7f89b80c20f0 2024-03-09T09:16:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.202+0000 7f89be000640 1 -- 172.21.15.67:0/2898879402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89b80c20f0 msgr2=0x7f89b80a9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.202+0000 7f89be000640 1 --2- 172.21.15.67:0/2898879402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89b80c20f0 0x7f89b80a9690 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89b806f900 tx=0x7f89a402b8a0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.202+0000 7f89be000640 1 -- 172.21.15.67:0/2898879402 shutdown_connections 2024-03-09T09:16:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.202+0000 7f89be000640 1 --2- 172.21.15.67:0/2898879402 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b80a9bd0 0x7f89b806db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.202+0000 7f89be000640 1 --2- 172.21.15.67:0/2898879402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89b80c20f0 0x7f89b80a9690 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.202+0000 7f89be000640 1 --2- 172.21.15.67:0/2898879402 >> v2:172.21.15.134:3300/0 conn(0x7f89b81510f0 0x7f89b80a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.202+0000 7f89be000640 1 -- 172.21.15.67:0/2898879402 >> 172.21.15.67:0/2898879402 conn(0x7f89b80c76f0 msgr2=0x7f89b80c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.202+0000 7f89be000640 1 -- 172.21.15.67:0/2898879402 shutdown_connections 2024-03-09T09:16:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.202+0000 7f89be000640 1 -- 172.21.15.67:0/2898879402 wait complete. 2024-03-09T09:16:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.203+0000 7f89be000640 1 Processor -- start 2024-03-09T09:16:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.204+0000 7f89be000640 1 -- start start 2024-03-09T09:16:44.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.204+0000 7f89be000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b80a9bd0 0x7f89b81ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:44.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.204+0000 7f89be000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89b80c20f0 0x7f89b8152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:44.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.204+0000 7f89be000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b81510f0 0x7f89b814c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:44.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.204+0000 7f89be000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b806e240 con 0x7f89b80c20f0 2024-03-09T09:16:44.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.204+0000 7f89be000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b806e0c0 con 0x7f89b80a9bd0 2024-03-09T09:16:44.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.204+0000 7f89be000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b806e3c0 con 0x7f89b81510f0 2024-03-09T09:16:44.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.205+0000 7f89bcc00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b80a9bd0 msgr2=0x7f89b81ef6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:44.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.205+0000 7f89bcc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b80a9bd0 0x7f89b81ef6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:44.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.205+0000 7f89b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89b80c20f0 0x7f89b8152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.205+0000 7f89b7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89b80c20f0 0x7f89b8152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50912/0 (socket says 172.21.15.67:50912) 2024-03-09T09:16:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.205+0000 7f89b7e00640 1 -- 172.21.15.67:0/2236700017 learned_addr learned my addr 172.21.15.67:0/2236700017 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.205+0000 7f89bd600640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b81510f0 0x7f89b814c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.205+0000 7f89b7e00640 1 -- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b81510f0 msgr2=0x7f89b814c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.205+0000 7f89b7e00640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b81510f0 0x7f89b814c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.205+0000 7f89b7e00640 1 -- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b80a9bd0 msgr2=0x7f89b81ef6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:16:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.205+0000 7f89b7e00640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b80a9bd0 0x7f89b81ef6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.205+0000 7f89b7e00640 1 -- 172.21.15.67:0/2236700017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b80c6770 con 0x7f89b80c20f0 2024-03-09T09:16:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.206+0000 7f89b7e00640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89b80c20f0 0x7f89b8152880 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89a4000c00 tx=0x7f89a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:44.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.206+0000 7f89b4c00640 1 -- 172.21.15.67:0/2236700017 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4040070 con 0x7f89b80c20f0 2024-03-09T09:16:44.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.206+0000 7f89b4c00640 1 -- 172.21.15.67:0/2236700017 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4039cf0 con 0x7f89b80c20f0 2024-03-09T09:16:44.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.206+0000 7f89b4c00640 1 -- 172.21.15.67:0/2236700017 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4035040 con 0x7f89b80c20f0 2024-03-09T09:16:44.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.206+0000 7f89be000640 1 -- 172.21.15.67:0/2236700017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b80c6a50 con 0x7f89b80c20f0 2024-03-09T09:16:44.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.206+0000 7f89be000640 1 -- 172.21.15.67:0/2236700017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b8070e30 con 0x7f89b80c20f0 2024-03-09T09:16:44.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.207+0000 7f89be000640 1 -- 172.21.15.67:0/2236700017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b804f3c0 con 0x7f89b80c20f0 2024-03-09T09:16:44.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.208+0000 7f89b4c00640 1 -- 172.21.15.67:0/2236700017 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f89a4038020 con 0x7f89b80c20f0 2024-03-09T09:16:44.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.209+0000 7f89b4c00640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f899806b480 0x7f899806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:44.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.209+0000 7f89b4c00640 1 -- 172.21.15.67:0/2236700017 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f89a40ac310 con 0x7f89b80c20f0 2024-03-09T09:16:44.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.209+0000 7f89bcc00640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f899806b480 0x7f899806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:44.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.211+0000 7f89bcc00640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f899806b480 0x7f899806d940 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f89ac006fd0 tx=0x7f89ac006d80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:44.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.217+0000 7f89b4c00640 1 -- 172.21.15.67:0/2236700017 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f89a407ad90 con 0x7f89b80c20f0 2024-03-09T09:16:44.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.622+0000 7f89be000640 1 -- 172.21.15.67:0/2236700017 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f89b8157e50 con 0x7f89b80c20f0 2024-03-09T09:16:44.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.623+0000 7f89b4c00640 1 -- 172.21.15.67:0/2236700017 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f89a40ac0d0 con 0x7f89b80c20f0 2024-03-09T09:16:44.628 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":180,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:44.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 -- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f899806b480 msgr2=0x7f899806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:44.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f899806b480 0x7f899806d940 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f89ac006fd0 tx=0x7f89ac006d80 comp rx=0 tx=0).stop 2024-03-09T09:16:44.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 -- 172.21.15.67:0/2236700017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89b80c20f0 msgr2=0x7f89b8152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:44.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89b80c20f0 0x7f89b8152880 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89a4000c00 tx=0x7f89a40046c0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 -- 172.21.15.67:0/2236700017 shutdown_connections 2024-03-09T09:16:44.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f899806b480 0x7f899806d940 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b81510f0 0x7f89b814c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89b80c20f0 0x7f89b8152880 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 --2- 172.21.15.67:0/2236700017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b80a9bd0 0x7f89b81ef6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:44.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 -- 172.21.15.67:0/2236700017 >> 172.21.15.67:0/2236700017 conn(0x7f89b80c76f0 msgr2=0x7f89b80c0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:44.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 -- 172.21.15.67:0/2236700017 shutdown_connections 2024-03-09T09:16:44.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:44.625+0000 7f89be000640 1 -- 172.21.15.67:0/2236700017 wait complete. 2024-03-09T09:16:47.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:16:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.848+0000 7f12c8a00640 1 Processor -- start 2024-03-09T09:16:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.848+0000 7f12c8a00640 1 -- start start 2024-03-09T09:16:47.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.850+0000 7f12c8a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:47.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.850+0000 7f12c8a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c40c81e0 con 0x7f12c40c1f70 2024-03-09T09:16:47.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.850+0000 7f12c8a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c40c32b0 con 0x7f12c40710a0 2024-03-09T09:16:47.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.850+0000 7f12c8a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c40c2fd0 con 0x7f12c4151360 2024-03-09T09:16:47.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.850+0000 7f12c2a00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f12c40710a0 legacy=0x7f12c4153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:16:47.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.850+0000 7f12c3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:47.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.850+0000 7f12c3400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f12c40c1f70 0x7f12c4150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:46752/0 (socket says 172.21.15.67:46752) 2024-03-09T09:16:47.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.850+0000 7f12c3400640 1 -- 172.21.15.67:0/3518684399 learned_addr learned my addr 172.21.15.67:0/3518684399 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:16:47.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.850+0000 7f12c3e00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 msgr2=0x7f12c415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:16:47.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.850+0000 7f12c3e00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 msgr2=0x7f12c415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:16:47.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.850+0000 7f12c3e00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:47.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:47.851+0000 7f12c3e00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:48.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.051+0000 7f12c2a00640 1 -- 172.21.15.67:0/3518684399 >> v1:172.21.15.134:6789/0 conn(0x7f12c40710a0 legacy=0x7f12c4153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:16:48.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.052+0000 7f12c3e00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:48.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.053+0000 7f12c3e00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 msgr2=0x7f12c415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:16:48.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.053+0000 7f12c3e00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 msgr2=0x7f12c415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:16:48.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.053+0000 7f12c3e00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:48.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.053+0000 7f12c3e00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:16:48.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.452+0000 7f12c2a00640 1 -- 172.21.15.67:0/3518684399 >> v1:172.21.15.134:6789/0 conn(0x7f12c40710a0 legacy=0x7f12c4153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:16:48.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.454+0000 7f12c3e00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:48.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.454+0000 7f12c3e00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 msgr2=0x7f12c415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:16:48.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.454+0000 7f12c3e00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 msgr2=0x7f12c415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:16:48.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.454+0000 7f12c3e00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:48.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:48.454+0000 7f12c3e00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:16:49.107 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 52% (103/197) Updating files: 53% (105/197) Updating files: 54% (107/197) Updating files: 55% (109/197) Updating files: 56% (111/197) Updating files: 57% (113/197) Updating files: 58% (115/197) Updating files: 59% (117/197) Updating files: 60% (119/197) Updating files: 61% (121/197) Updating files: 61% (122/197) Updating files: 62% (123/197) Updating files: 63% (125/197) Updating files: 64% (127/197) Updating files: 65% (129/197) Updating files: 66% (131/197) Updating files: 67% (132/197) Updating files: 68% (134/197) Updating files: 69% (136/197) Updating files: 70% (138/197) Updating files: 71% (140/197) Updating files: 72% (142/197) Updating files: 73% (144/197) Updating files: 74% (146/197) Updating files: 75% (148/197) Updating files: 76% (150/197) Updating files: 77% (152/197) Updating files: 78% (154/197) Updating files: 79% (156/197) Updating files: 80% (158/197) Updating files: 81% (160/197) Updating files: 82% (162/197) Updating files: 83% (164/197) Updating files: 84% (166/197) Updating files: 85% (168/197) Updating files: 86% (170/197) Updating files: 87% (172/197) Updating files: 88% (174/197) Updating files: 89% (176/197) Updating files: 90% (178/197) Updating files: 91% (180/197) Updating files: 92% (182/197) Updating files: 93% (184/197) Updating files: 94% (186/197) Updating files: 94% (187/197) Updating files: 95% (188/197) Updating files: 96% (190/197) Updating files: 97% (192/197) Updating files: 98% (194/197) Updating files: 99% (196/197) Updating files: 100% (197/197) Updating files: 100% (197/197), done. 2024-03-09T09:16:49.169 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 68fdc0f68e6 0.89 2024-03-09T09:16:49.190 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.89 2024-03-09T09:16:49.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:49.253+0000 7f12c2a00640 1 -- 172.21.15.67:0/3518684399 >> v1:172.21.15.134:6789/0 conn(0x7f12c40710a0 legacy=0x7f12c4153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:16:49.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:49.255+0000 7f12c3e00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:49.255+0000 7f12c3e00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 msgr2=0x7f12c415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:49.255+0000 7f12c3e00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 msgr2=0x7f12c415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:49.255+0000 7f12c3e00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:49.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:49.255+0000 7f12c3e00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:16:49.303 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:16:49.303 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 90 -eq 48 ']' 2024-03-09T09:16:49.303 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.90 2024-03-09T09:16:49.303 INFO:tasks.workunit.client.0.smithi067.stdout:v0.90 2024-03-09T09:16:49.304 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo v0.90 2024-03-09T09:16:49.304 INFO:tasks.workunit.client.0.smithi067.stderr:+ git reset --hard v0.90 2024-03-09T09:16:50.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.849+0000 7f12c0c00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 msgr2=0x7f12c415f2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:50.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.849+0000 7f12c0c00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.67:3300/0 conn(0x7f12c4151360 0x7f12c415f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:50.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.849+0000 7f12c0c00640 1 -- 172.21.15.67:0/3518684399 >> v1:172.21.15.134:6789/0 conn(0x7f12c40710a0 legacy=0x7f12c4153970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:50.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.849+0000 7f12c0c00640 1 -- 172.21.15.67:0/3518684399 >> v1:172.21.15.67:6789/0 conn(0x7f12c40c1f70 legacy=0x7f12c4150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:50.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.849+0000 7f12c0c00640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:50.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.849+0000 7f12c0c00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.134:3300/0 conn(0x7f12a4006160 0x7f12a4008620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:50.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.849+0000 7f12c0c00640 1 -- 172.21.15.67:0/3518684399 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c40c2fd0 con 0x7f12a4006160 2024-03-09T09:16:50.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.849+0000 7f12c0c00640 1 -- 172.21.15.67:0/3518684399 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a4008ce0 con 0x7f12a40036a0 2024-03-09T09:16:50.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.849+0000 7f12c2a00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.134:3300/0 conn(0x7f12a4006160 msgr2=0x7f12a4008620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:50.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.850+0000 7f12c2a00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.134:3300/0 conn(0x7f12a4006160 0x7f12a4008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:50.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.850+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:50.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.850+0000 7f12c3400640 1 -- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 msgr2=0x7f12a4005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:16:50.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.850+0000 7f12c3400640 1 -- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 msgr2=0x7f12a4005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:16:50.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.850+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:50.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:50.850+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.051+0000 7f12c2a00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.134:3300/0 conn(0x7f12a4006160 msgr2=0x7f12a4008620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.051+0000 7f12c2a00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.134:3300/0 conn(0x7f12a4006160 0x7f12a4008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:16:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.051+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.052+0000 7f12c3400640 1 -- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 msgr2=0x7f12a4005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:16:51.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.052+0000 7f12c3400640 1 -- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 msgr2=0x7f12a4005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:16:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.052+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.052+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:16:51.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.452+0000 7f12c2a00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.134:3300/0 conn(0x7f12a4006160 msgr2=0x7f12a4008620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:51.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.452+0000 7f12c2a00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.134:3300/0 conn(0x7f12a4006160 0x7f12a4008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:16:51.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.454+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:51.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.454+0000 7f12c3400640 1 -- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 msgr2=0x7f12a4005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:16:51.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.454+0000 7f12c3400640 1 -- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 msgr2=0x7f12a4005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:16:51.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.454+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:16:51.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:51.454+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:16:52.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.253+0000 7f12c2a00640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.134:3300/0 conn(0x7f12a4006160 msgr2=0x7f12a4008620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:52.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.253+0000 7f12c2a00640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.134:3300/0 conn(0x7f12a4006160 0x7f12a4008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:16:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.256+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:52.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.257+0000 7f12c3400640 1 -- 172.21.15.67:0/3518684399 >> v2:172.21.15.134:3300/0 conn(0x7f12a4006160 msgr2=0x7f12a4008620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:52.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.257+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> v2:172.21.15.134:3300/0 conn(0x7f12a4006160 0x7f12a4008620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.257+0000 7f12c3400640 1 -- 172.21.15.67:0/3518684399 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c40c3700 con 0x7f12a40036a0 2024-03-09T09:16:52.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.260+0000 7f12c3400640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f12b4007d20 tx=0x7f12b402bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d48e7d62db5d5223 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:52.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.261+0000 7f12c2000640 1 -- 172.21.15.67:0/3518684399 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4032070 con 0x7f12a40036a0 2024-03-09T09:16:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.261+0000 7f12c2000640 1 -- 172.21.15.67:0/3518684399 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b402ca90 con 0x7f12a40036a0 2024-03-09T09:16:52.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.261+0000 7f12c2000640 1 -- 172.21.15.67:0/3518684399 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b402ce40 con 0x7f12a40036a0 2024-03-09T09:16:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.262+0000 7f12c8a00640 1 -- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 msgr2=0x7f12a4005af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.262+0000 7f12c8a00640 1 --2- 172.21.15.67:0/3518684399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a40036a0 0x7f12a4005af0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f12b4007d20 tx=0x7f12b402bef0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.262+0000 7f12c3e00640 1 -- 172.21.15.67:0/3518684399 reap_dead start 2024-03-09T09:16:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.262+0000 7f12c8a00640 1 -- 172.21.15.67:0/3518684399 shutdown_connections 2024-03-09T09:16:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.262+0000 7f12c8a00640 1 -- 172.21.15.67:0/3518684399 >> 172.21.15.67:0/3518684399 conn(0x7f12c40c7300 msgr2=0x7f12c40c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.262+0000 7f12c8a00640 1 -- 172.21.15.67:0/3518684399 shutdown_connections 2024-03-09T09:16:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.262+0000 7f12c8a00640 1 -- 172.21.15.67:0/3518684399 wait complete. 2024-03-09T09:16:52.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.265+0000 7f12c8a00640 1 Processor -- start 2024-03-09T09:16:52.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.265+0000 7f12c8a00640 1 -- start start 2024-03-09T09:16:52.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.266+0000 7f12c8a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12c4148030 0x7f12c4144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4142010 0x7f12c4144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c8a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c41ee8d0 0x7f12c40ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c8a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4070d50 con 0x7f12c4142010 2024-03-09T09:16:52.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c8a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4070830 con 0x7f12c41ee8d0 2024-03-09T09:16:52.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c8a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4070ed0 con 0x7f12c4148030 2024-03-09T09:16:52.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c3e00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c41ee8d0 msgr2=0x7f12c40ab870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:52.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c41ee8d0 0x7f12c40ab870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:52.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4142010 0x7f12c4144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:52.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4142010 0x7f12c4144d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35722/0 (socket says 172.21.15.67:35722) 2024-03-09T09:16:52.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c2a00640 1 -- 172.21.15.67:0/3112691701 learned_addr learned my addr 172.21.15.67:0/3112691701 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:52.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c3400640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12c4148030 0x7f12c4144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:52.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c2a00640 1 -- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12c4148030 msgr2=0x7f12c4144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:52.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c2a00640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12c4148030 0x7f12c4144840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c2a00640 1 -- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c41ee8d0 msgr2=0x7f12c40ab870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:16:52.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c2a00640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c41ee8d0 0x7f12c40ab870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.267+0000 7f12c2a00640 1 -- 172.21.15.67:0/3112691701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4148cf0 con 0x7f12c4142010 2024-03-09T09:16:52.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.268+0000 7f12c2a00640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4142010 0x7f12c4144d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f12b000b920 tx=0x7f12b000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:52.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.268+0000 7f12bb400640 1 -- 172.21.15.67:0/3112691701 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b000cbd0 con 0x7f12c4142010 2024-03-09T09:16:52.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.268+0000 7f12bb400640 1 -- 172.21.15.67:0/3112691701 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b0010030 con 0x7f12c4142010 2024-03-09T09:16:52.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.268+0000 7f12bb400640 1 -- 172.21.15.67:0/3112691701 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0015420 con 0x7f12c4142010 2024-03-09T09:16:52.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.268+0000 7f12c8a00640 1 -- 172.21.15.67:0/3112691701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c40aa4b0 con 0x7f12c4142010 2024-03-09T09:16:52.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.268+0000 7f12c8a00640 1 -- 172.21.15.67:0/3112691701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c40aaa70 con 0x7f12c4142010 2024-03-09T09:16:52.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.270+0000 7f12c8a00640 1 -- 172.21.15.67:0/3112691701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c404f3c0 con 0x7f12c4142010 2024-03-09T09:16:52.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.270+0000 7f12bb400640 1 -- 172.21.15.67:0/3112691701 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f12b00143d0 con 0x7f12c4142010 2024-03-09T09:16:52.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.272+0000 7f12bb400640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a406c240 0x7f12a406e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.272+0000 7f12bb400640 1 -- 172.21.15.67:0/3112691701 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f12b0089f00 con 0x7f12c4142010 2024-03-09T09:16:52.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.272+0000 7f12c3400640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a406c240 0x7f12a406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:52.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.273+0000 7f12c3400640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a406c240 0x7f12a406e700 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f12b4037000 tx=0x7f12b4010040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:52.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.279+0000 7f12bb400640 1 -- 172.21.15.67:0/3112691701 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f12b0058a20 con 0x7f12c4142010 2024-03-09T09:16:52.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.592+0000 7f12c8a00640 1 -- 172.21.15.67:0/3112691701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f12c40acc90 con 0x7f12c4142010 2024-03-09T09:16:52.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.593+0000 7f12bb400640 1 -- 172.21.15.67:0/3112691701 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f12b0058320 con 0x7f12c4142010 2024-03-09T09:16:52.599 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":184,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:16:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 -- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a406c240 msgr2=0x7f12a406e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a406c240 0x7f12a406e700 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f12b4037000 tx=0x7f12b4010040 comp rx=0 tx=0).stop 2024-03-09T09:16:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 -- 172.21.15.67:0/3112691701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4142010 msgr2=0x7f12c4144d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4142010 0x7f12c4144d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f12b000b920 tx=0x7f12b000bde0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 -- 172.21.15.67:0/3112691701 shutdown_connections 2024-03-09T09:16:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f12a406c240 0x7f12a406e700 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12c41ee8d0 0x7f12c40ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12c4142010 0x7f12c4144d80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 --2- 172.21.15.67:0/3112691701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12c4148030 0x7f12c4144840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 -- 172.21.15.67:0/3112691701 >> 172.21.15.67:0/3112691701 conn(0x7f12c40c7300 msgr2=0x7f12c40c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:52.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 -- 172.21.15.67:0/3112691701 shutdown_connections 2024-03-09T09:16:52.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.596+0000 7f12c8a00640 1 -- 172.21.15.67:0/3112691701 wait complete. 2024-03-09T09:16:52.619 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:16:52.619 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:16:52.619 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:16:52.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.805+0000 7fcd53200640 1 Processor -- start 2024-03-09T09:16:52.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.805+0000 7fcd53200640 1 -- start start 2024-03-09T09:16:52.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.807+0000 7fcd53200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcd4c0730d0 0x7fcd4c06dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.807+0000 7fcd53200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4c156a40 con 0x7fcd4c0c5900 2024-03-09T09:16:52.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.807+0000 7fcd53200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4c073850 con 0x7fcd4c1535e0 2024-03-09T09:16:52.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.807+0000 7fcd53200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4c073570 con 0x7fcd4c0730d0 2024-03-09T09:16:52.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.807+0000 7fcd4be00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7fcd4c1535e0 legacy=0x7fcd4c0729c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:16:52.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.807+0000 7fcd43400640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7fcd4c0730d0 msgr2=0x7fcd4c06dce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:52.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.807+0000 7fcd43400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcd4c0730d0 0x7fcd4c06dce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:52.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.807+0000 7fcd51200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcd4c0c5900 0x7fcd4c0c2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:54586/0 (socket says 172.21.15.67:54586) 2024-03-09T09:16:52.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.807+0000 7fcd51200640 1 -- 172.21.15.67:0/1088455168 learned_addr learned my addr 172.21.15.67:0/1088455168 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:16:52.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480570599 0 0) 0x7fcd4c156a40 con 0x7fcd4c0c5900 2024-03-09T09:16:52.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd30003520 con 0x7fcd4c0c5900 2024-03-09T09:16:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2461218588 0 0) 0x7fcd30003520 con 0x7fcd4c0c5900 2024-03-09T09:16:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 >> v2:172.21.15.134:3300/0 conn(0x7fcd4c0730d0 msgr2=0x7fcd4c06dce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 --2- 172.21.15.67:0/1088455168 >> v2:172.21.15.134:3300/0 conn(0x7fcd4c0730d0 0x7fcd4c06dce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 >> v1:172.21.15.134:6789/0 conn(0x7fcd4c1535e0 legacy=0x7fcd4c0729c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c156ce0 con 0x7fcd4c0c5900 2024-03-09T09:16:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fcd44001670 con 0x7fcd4c0c5900 2024-03-09T09:16:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 >> v1:172.21.15.67:6789/0 conn(0x7fcd4c0c5900 legacy=0x7fcd4c0c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 --2- 172.21.15.67:0/1088455168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd300039b0 0x7fcd30003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 --2- 172.21.15.67:0/1088455168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30004800 0x7fcd30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 --2- 172.21.15.67:0/1088455168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd30005210 0x7fcd3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c073570 con 0x7fcd300039b0 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3000c140 con 0x7fcd30005210 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.808+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3000c340 con 0x7fcd30004800 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.809+0000 7fcd4be00640 1 --2- 172.21.15.67:0/1088455168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd300039b0 0x7fcd30003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.809+0000 7fcd51200640 1 -- 172.21.15.67:0/1088455168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd30005210 msgr2=0x7fcd3000b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.809+0000 7fcd51200640 1 --2- 172.21.15.67:0/1088455168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd30005210 0x7fcd3000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.809+0000 7fcd4be00640 1 -- 172.21.15.67:0/1088455168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30004800 msgr2=0x7fcd30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.809+0000 7fcd4be00640 1 --2- 172.21.15.67:0/1088455168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd30004800 0x7fcd30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.809+0000 7fcd4be00640 1 -- 172.21.15.67:0/1088455168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd30005210 msgr2=0x7fcd3000b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.809+0000 7fcd4be00640 1 --2- 172.21.15.67:0/1088455168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd30005210 0x7fcd3000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.809+0000 7fcd4be00640 1 -- 172.21.15.67:0/1088455168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd30003520 con 0x7fcd300039b0 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.809+0000 7fcd51200640 1 -- 172.21.15.67:0/1088455168 reap_dead start 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.810+0000 7fcd4be00640 1 --2- 172.21.15.67:0/1088455168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd300039b0 0x7fcd30003db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009e30 tx=0x7fcd30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ee7d67a91deb52 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:52.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.810+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c009a30 con 0x7fcd300039b0 2024-03-09T09:16:52.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.810+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c030070 con 0x7fcd300039b0 2024-03-09T09:16:52.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.810+0000 7fcd4b400640 1 -- 172.21.15.67:0/1088455168 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c035470 con 0x7fcd300039b0 2024-03-09T09:16:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.811+0000 7fcd53200640 1 -- 172.21.15.67:0/1088455168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd300039b0 msgr2=0x7fcd30003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.811+0000 7fcd53200640 1 --2- 172.21.15.67:0/1088455168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd300039b0 0x7fcd30003db0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009e30 tx=0x7fcd30005830 comp rx=0 tx=0).stop 2024-03-09T09:16:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.811+0000 7fcd53200640 1 -- 172.21.15.67:0/1088455168 shutdown_connections 2024-03-09T09:16:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.811+0000 7fcd53200640 1 --2- 172.21.15.67:0/1088455168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd300039b0 0x7fcd30003db0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.811+0000 7fcd53200640 1 -- 172.21.15.67:0/1088455168 >> 172.21.15.67:0/1088455168 conn(0x7fcd4c0ab320 msgr2=0x7fcd4c0ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.811+0000 7fcd53200640 1 -- 172.21.15.67:0/1088455168 shutdown_connections 2024-03-09T09:16:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.811+0000 7fcd53200640 1 -- 172.21.15.67:0/1088455168 wait complete. 2024-03-09T09:16:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.813+0000 7fcd53200640 1 Processor -- start 2024-03-09T09:16:52.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.813+0000 7fcd53200640 1 -- start start 2024-03-09T09:16:52.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd53200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd300039b0 0x7fcd4c1ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd53200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0af370 0x7fcd4c14fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd53200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd4c157980 0x7fcd4c157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd53200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c153b40 con 0x7fcd4c0af370 2024-03-09T09:16:52.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd53200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c1539c0 con 0x7fcd4c157980 2024-03-09T09:16:52.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd53200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c06dce0 con 0x7fcd300039b0 2024-03-09T09:16:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd51200640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd4c157980 msgr2=0x7fcd4c157e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd51200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd4c157980 0x7fcd4c157e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd43400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0af370 0x7fcd4c14fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd43400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0af370 0x7fcd4c14fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35740/0 (socket says 172.21.15.67:35740) 2024-03-09T09:16:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd43400640 1 -- 172.21.15.67:0/859591277 learned_addr learned my addr 172.21.15.67:0/859591277 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.814+0000 7fcd4be00640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd300039b0 0x7fcd4c1ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.815+0000 7fcd43400640 1 -- 172.21.15.67:0/859591277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd300039b0 msgr2=0x7fcd4c1ef0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.815+0000 7fcd43400640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd300039b0 0x7fcd4c1ef0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.815+0000 7fcd43400640 1 -- 172.21.15.67:0/859591277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd4c157980 msgr2=0x7fcd4c157e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:16:52.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.815+0000 7fcd43400640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd4c157980 0x7fcd4c157e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.815+0000 7fcd43400640 1 -- 172.21.15.67:0/859591277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c158370 con 0x7fcd4c0af370 2024-03-09T09:16:52.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.815+0000 7fcd43400640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0af370 0x7fcd4c14fef0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400b460 tx=0x7fcd3400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:52.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.815+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34016070 con 0x7fcd4c0af370 2024-03-09T09:16:52.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.815+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd34003dc0 con 0x7fcd4c0af370 2024-03-09T09:16:52.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.815+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd340052b0 con 0x7fcd4c0af370 2024-03-09T09:16:52.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.815+0000 7fcd53200640 1 -- 172.21.15.67:0/859591277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4c159a90 con 0x7fcd4c0af370 2024-03-09T09:16:52.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.816+0000 7fcd53200640 1 -- 172.21.15.67:0/859591277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4c15a050 con 0x7fcd4c0af370 2024-03-09T09:16:52.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.817+0000 7fcd53200640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0ac620 0x7fcd4c0aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.817+0000 7fcd53200640 1 -- 172.21.15.67:0/859591277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c159230 con 0x7fcd4c0ac620 2024-03-09T09:16:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.817+0000 7fcd4be00640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0ac620 0x7fcd4c0aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.817+0000 7fcd4be00640 1 -- 172.21.15.67:0/859591277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd4c075a00 con 0x7fcd4c0ac620 2024-03-09T09:16:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.818+0000 7fcd4be00640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0ac620 0x7fcd4c0aca00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c003cb0 tx=0x7fcd3c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.818+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c035930 con 0x7fcd4c0ac620 2024-03-09T09:16:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.818+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fcd3c030050 con 0x7fcd4c0ac620 2024-03-09T09:16:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.818+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0ac620 msgr2=0x7fcd4c0aca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.818+0000 7fcd48c00640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0ac620 0x7fcd4c0aca00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c003cb0 tx=0x7fcd3c005d90 comp rx=0 tx=0).stop 2024-03-09T09:16:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.818+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0ac620 msgr2=0x7fcd4c0aca00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.818+0000 7fcd48c00640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0ac620 0x7fcd4c0aca00 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c003cb0 tx=0x7fcd3c005d90 comp rx=0 tx=0).stop 2024-03-09T09:16:52.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.819+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcd34019070 con 0x7fcd4c0af370 2024-03-09T09:16:52.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.819+0000 7fcd48c00640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd2806b0d0 0x7fcd2806d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.819+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcd34089680 con 0x7fcd4c0af370 2024-03-09T09:16:52.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.820+0000 7fcd4be00640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd2806b0d0 0x7fcd2806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:52.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.821+0000 7fcd4be00640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd2806b0d0 0x7fcd2806d590 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c068fc0 tx=0x7fcd3c040000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:52.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.828+0000 7fcd53200640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0acf40 0x7fcd4c14daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:52.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.828+0000 7fcd53200640 1 -- 172.21.15.67:0/859591277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c1ef5f0 con 0x7fcd4c0acf40 2024-03-09T09:16:52.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.828+0000 7fcd51200640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0acf40 0x7fcd4c14daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:52.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.829+0000 7fcd51200640 1 -- 172.21.15.67:0/859591277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcd4c15a290 con 0x7fcd4c0acf40 2024-03-09T09:16:52.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.829+0000 7fcd51200640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0acf40 0x7fcd4c14daf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c071fc0 tx=0x7fcd4400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.829+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd440074e0 con 0x7fcd4c0acf40 2024-03-09T09:16:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.829+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5008 (secure 0 0 0) 0x7fcd440074e0 con 0x7fcd4c0acf40 2024-03-09T09:16:52.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.829+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0acf40 msgr2=0x7fcd4c14daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.829+0000 7fcd48c00640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0acf40 0x7fcd4c14daf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c071fc0 tx=0x7fcd4400c810 comp rx=0 tx=0).stop 2024-03-09T09:16:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.829+0000 7fcd48c00640 1 -- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0acf40 msgr2=0x7fcd4c14daf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:52.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.829+0000 7fcd48c00640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0acf40 0x7fcd4c14daf0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c071fc0 tx=0x7fcd4400c810 comp rx=0 tx=0).stop 2024-03-09T09:16:52.836 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:16:52.836 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:16:52.836 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:16:52.836 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:16:52.836 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 184, 2024-03-09T09:16:52.836 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:16:52.836 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:16:52.836 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:52.837 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:52.838 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:16:52.839 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:52.840 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:52.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:16:52.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:16:52.840 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:16:52.840 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:16:52.840 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:16:52.840 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:16:52.840 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:16:52.840 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:52.840 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:16:52.840 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.841 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:52.842 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:52.843 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:16:52.844 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.845 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:52.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:52.847 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:16:52.848 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:52.849 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:52.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.849 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:52.849 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:52.849 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:52.849 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:16:52.849 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:52.849 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:52.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:52.849 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:16:52.849 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:16:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.830+0000 7fcd53200640 1 -- 172.21.15.67:0/859591277 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd2806b0d0 msgr2=0x7fcd2806d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.830+0000 7fcd53200640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd2806b0d0 0x7fcd2806d590 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c068fc0 tx=0x7fcd3c040000 comp rx=0 tx=0).stop 2024-03-09T09:16:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.830+0000 7fcd53200640 1 -- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0af370 msgr2=0x7fcd4c14fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.830+0000 7fcd53200640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0af370 0x7fcd4c14fef0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400b460 tx=0x7fcd3400b920 comp rx=0 tx=0).stop 2024-03-09T09:16:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.830+0000 7fcd51200640 1 -- 172.21.15.67:0/859591277 reap_dead start 2024-03-09T09:16:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.830+0000 7fcd53200640 1 -- 172.21.15.67:0/859591277 shutdown_connections 2024-03-09T09:16:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.830+0000 7fcd53200640 1 --2- 172.21.15.67:0/859591277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd4c0af370 0x7fcd4c14fef0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.830+0000 7fcd53200640 1 -- 172.21.15.67:0/859591277 >> 172.21.15.67:0/859591277 conn(0x7fcd4c0ab320 msgr2=0x7fcd4c1549f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:52.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.830+0000 7fcd53200640 1 -- 172.21.15.67:0/859591277 shutdown_connections 2024-03-09T09:16:52.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:52.830+0000 7fcd53200640 1 -- 172.21.15.67:0/859591277 wait complete. 2024-03-09T09:16:52.852 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:16:52.852 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:16:52.852 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:16:53.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.077+0000 7fc2cf200640 1 Processor -- start 2024-03-09T09:16:53.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.078+0000 7fc2cf200640 1 -- start start 2024-03-09T09:16:53.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.079+0000 7fc2cf200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc2d00c9380 0x7fc2d00acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:53.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.079+0000 7fc2cf200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc2d00ad440 0x7fc2d006c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.079+0000 7fc2cf200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d00c9890 con 0x7fc2d014fd90 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.079+0000 7fc2cf200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d006ba60 con 0x7fc2d00ad440 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.079+0000 7fc2cf200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d00c5f80 con 0x7fc2d00c9380 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.079+0000 7fc2cde00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7fc2d014fd90 legacy=0x7fc2d0156680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.079+0000 7fc2cd400640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7fc2d00c9380 msgr2=0x7fc2d00acf00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.079+0000 7fc2cd400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc2d00c9380 0x7fc2d00acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.079+0000 7fc2ce800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc2d00ad440 0x7fc2d006c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.080+0000 7fc2ce800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc2d00ad440 0x7fc2d006c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35756/0 (socket says 172.21.15.67:35756) 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.080+0000 7fc2ce800640 1 -- 172.21.15.67:0/1520709759 learned_addr learned my addr 172.21.15.67:0/1520709759 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.080+0000 7fc2ce800640 1 -- 172.21.15.67:0/1520709759 >> v2:172.21.15.134:3300/0 conn(0x7fc2d00c9380 msgr2=0x7fc2d00acf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.080+0000 7fc2ce800640 1 --2- 172.21.15.67:0/1520709759 >> v2:172.21.15.134:3300/0 conn(0x7fc2d00c9380 0x7fc2d00acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.080+0000 7fc2ce800640 1 -- 172.21.15.67:0/1520709759 >> v1:172.21.15.134:6789/0 conn(0x7fc2d014fd90 legacy=0x7fc2d0156680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:16:53.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.080+0000 7fc2ce800640 1 -- 172.21.15.67:0/1520709759 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d00c6240 con 0x7fc2d00ad440 2024-03-09T09:16:53.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.081+0000 7fc2ce800640 1 --2- 172.21.15.67:0/1520709759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d00ad440 0x7fc2d006c1d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00ae040 tx=0x7fc2c802bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed9a7a96bb8f40a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:53.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.081+0000 7fc2cca00640 1 -- 172.21.15.67:0/1520709759 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c802c980 con 0x7fc2d00ad440 2024-03-09T09:16:53.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.081+0000 7fc2cca00640 1 -- 172.21.15.67:0/1520709759 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c802cb20 con 0x7fc2d00ad440 2024-03-09T09:16:53.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.081+0000 7fc2cca00640 1 -- 172.21.15.67:0/1520709759 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80343f0 con 0x7fc2d00ad440 2024-03-09T09:16:53.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.082+0000 7fc2cf200640 1 -- 172.21.15.67:0/1520709759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d00ad440 msgr2=0x7fc2d006c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:53.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.082+0000 7fc2cf200640 1 --2- 172.21.15.67:0/1520709759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d00ad440 0x7fc2d006c1d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00ae040 tx=0x7fc2c802bde0 comp rx=0 tx=0).stop 2024-03-09T09:16:53.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.082+0000 7fc2cf200640 1 -- 172.21.15.67:0/1520709759 shutdown_connections 2024-03-09T09:16:53.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.082+0000 7fc2cf200640 1 --2- 172.21.15.67:0/1520709759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d00ad440 0x7fc2d006c1d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:53.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.082+0000 7fc2cf200640 1 --2- 172.21.15.67:0/1520709759 >> v2:172.21.15.134:3300/0 conn(0x7fc2d00c9380 0x7fc2d00acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:53.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.082+0000 7fc2cf200640 1 -- 172.21.15.67:0/1520709759 >> 172.21.15.67:0/1520709759 conn(0x7fc2d00cb480 msgr2=0x7fc2d00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:53.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.082+0000 7fc2cf200640 1 -- 172.21.15.67:0/1520709759 shutdown_connections 2024-03-09T09:16:53.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.082+0000 7fc2cf200640 1 -- 172.21.15.67:0/1520709759 wait complete. 2024-03-09T09:16:53.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.085+0000 7fc2cf200640 1 Processor -- start 2024-03-09T09:16:53.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.085+0000 7fc2cf200640 1 -- start start 2024-03-09T09:16:53.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.086+0000 7fc2cf200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d00ad440 0x7fc2d01461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.086+0000 7fc2cf200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00c9380 0x7fc2d0147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.086+0000 7fc2cf200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d014fd90 0x7fc2d01ec060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.086+0000 7fc2cf200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d006c710 con 0x7fc2d00ad440 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.086+0000 7fc2cf200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d006bcb0 con 0x7fc2d014fd90 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.086+0000 7fc2cf200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d006c890 con 0x7fc2d00c9380 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2ce800640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d014fd90 msgr2=0x7fc2d01ec060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2cde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d00ad440 0x7fc2d01461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2ce800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d014fd90 0x7fc2d01ec060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2cde00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d00ad440 0x7fc2d01461f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35762/0 (socket says 172.21.15.67:35762) 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2cde00640 1 -- 172.21.15.67:0/2464259743 learned_addr learned my addr 172.21.15.67:0/2464259743 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2cd400640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00c9380 0x7fc2d0147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:53.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2cde00640 1 -- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00c9380 msgr2=0x7fc2d0147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2cde00640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00c9380 0x7fc2d0147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2cde00640 1 -- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d014fd90 msgr2=0x7fc2d01ec060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:16:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2cde00640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2d014fd90 0x7fc2d01ec060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:16:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2cde00640 1 -- 172.21.15.67:0/2464259743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d0146de0 con 0x7fc2d00ad440 2024-03-09T09:16:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2cd400640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00c9380 0x7fc2d0147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:16:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.087+0000 7fc2cde00640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d00ad440 0x7fc2d01461f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000b550 tx=0x7fc2c000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.088+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0013020 con 0x7fc2d00ad440 2024-03-09T09:16:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.088+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c0003de0 con 0x7fc2d00ad440 2024-03-09T09:16:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.088+0000 7fc2cf200640 1 -- 172.21.15.67:0/2464259743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d01efbb0 con 0x7fc2d00ad440 2024-03-09T09:16:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.088+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00052b0 con 0x7fc2d00ad440 2024-03-09T09:16:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.088+0000 7fc2cf200640 1 -- 172.21.15.67:0/2464259743 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d01ef880 con 0x7fc2d00ad440 2024-03-09T09:16:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.089+0000 7fc2cf200640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d0146730 0x7fc2d00aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.089+0000 7fc2cf200640 1 -- 172.21.15.67:0/2464259743 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d0146b90 con 0x7fc2d0146730 2024-03-09T09:16:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.090+0000 7fc2cd400640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d0146730 0x7fc2d00aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.090+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc2c0016020 con 0x7fc2d00ad440 2024-03-09T09:16:53.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.090+0000 7fc2cd400640 1 -- 172.21.15.67:0/2464259743 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc2d01eb950 con 0x7fc2d0146730 2024-03-09T09:16:53.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.091+0000 7fc2cd400640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d0146730 0x7fc2d00aade0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc003930 tx=0x7fc2bc00cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.091+0000 7fc2c6000640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc2b006b480 0x7fc2b006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.091+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc2c008a350 con 0x7fc2d00ad440 2024-03-09T09:16:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.092+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc003bb0 con 0x7fc2d0146730 2024-03-09T09:16:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.092+0000 7fc2ce800640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc2b006b480 0x7fc2b006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.092+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc2bc016070 con 0x7fc2d0146730 2024-03-09T09:16:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.092+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d0146730 msgr2=0x7fc2d00aade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.092+0000 7fc2c6000640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d0146730 0x7fc2d00aade0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc003930 tx=0x7fc2bc00cb80 comp rx=0 tx=0).stop 2024-03-09T09:16:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.092+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d0146730 msgr2=0x7fc2d00aade0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:53.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.092+0000 7fc2c6000640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d0146730 0x7fc2d00aade0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc003930 tx=0x7fc2bc00cb80 comp rx=0 tx=0).stop 2024-03-09T09:16:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.093+0000 7fc2ce800640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc2b006b480 0x7fc2b006d940 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8035000 tx=0x7fc2c8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:53.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.099+0000 7fc2cf200640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00ac090 0x7fc2d00ab320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:16:53.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.099+0000 7fc2cf200640 1 -- 172.21.15.67:0/2464259743 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d01efd80 con 0x7fc2d00ac090 2024-03-09T09:16:53.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.100+0000 7fc2cd400640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00ac090 0x7fc2d00ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:16:53.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.101+0000 7fc2cd400640 1 -- 172.21.15.67:0/2464259743 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc2d00c19e0 con 0x7fc2d00ac090 2024-03-09T09:16:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.101+0000 7fc2cd400640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00ac090 0x7fc2d00ab320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc000c00 tx=0x7fc2bc0059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:16:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.102+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0052e0 con 0x7fc2d00ac090 2024-03-09T09:16:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.102+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5176 (secure 0 0 0) 0x7fc2bc0052e0 con 0x7fc2d00ac090 2024-03-09T09:16:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.102+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00ac090 msgr2=0x7fc2d00ab320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.102+0000 7fc2c6000640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00ac090 0x7fc2d00ab320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc000c00 tx=0x7fc2bc0059d0 comp rx=0 tx=0).stop 2024-03-09T09:16:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.102+0000 7fc2c6000640 1 -- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00ac090 msgr2=0x7fc2d00ab320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:16:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.102+0000 7fc2c6000640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2d00ac090 0x7fc2d00ab320 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc000c00 tx=0x7fc2bc0059d0 comp rx=0 tx=0).stop 2024-03-09T09:16:53.109 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:16:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:16:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:16:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:16:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 184, 2024-03-09T09:16:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:16:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:16:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:16:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:16:53.109 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:53.110 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:53.111 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:16:53.112 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:16:53.113 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:16:53.114 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.115 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:53.116 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:16:53.117 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:16:53.118 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:53.119 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:53.120 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:16:53.121 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:16:53.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:16:53.122 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:16:53.122 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:16:53.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.103+0000 7fc2cf200640 1 -- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc2b006b480 msgr2=0x7fc2b006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:53.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.103+0000 7fc2cf200640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc2b006b480 0x7fc2b006d940 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8035000 tx=0x7fc2c8030000 comp rx=0 tx=0).stop 2024-03-09T09:16:53.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.103+0000 7fc2cf200640 1 -- 172.21.15.67:0/2464259743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d00ad440 msgr2=0x7fc2d01461f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:16:53.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.103+0000 7fc2cf200640 1 --2- 172.21.15.67:0/2464259743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2d00ad440 0x7fc2d01461f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000b550 tx=0x7fc2c000ba10 comp rx=0 tx=0).stop 2024-03-09T09:16:53.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.103+0000 7fc2ce800640 1 -- 172.21.15.67:0/2464259743 reap_dead start 2024-03-09T09:16:53.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.103+0000 7fc2ce800640 1 -- 172.21.15.67:0/2464259743 reap_dead start 2024-03-09T09:16:53.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.103+0000 7fc2cf200640 1 -- 172.21.15.67:0/2464259743 shutdown_connections 2024-03-09T09:16:53.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.103+0000 7fc2cf200640 1 -- 172.21.15.67:0/2464259743 >> 172.21.15.67:0/2464259743 conn(0x7fc2d00cb480 msgr2=0x7fc2d00ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:16:53.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.103+0000 7fc2cf200640 1 -- 172.21.15.67:0/2464259743 shutdown_connections 2024-03-09T09:16:53.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:16:53.103+0000 7fc2cf200640 1 -- 172.21.15.67:0/2464259743 wait complete. 2024-03-09T09:16:53.128 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:16:53.128 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:16:53.600 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 23% (62/265) Updating files: 24% (64/265) Updating files: 25% (67/265) Updating files: 26% (69/265) Updating files: 27% (72/265) Updating files: 28% (75/265) Updating files: 29% (77/265) Updating files: 30% (80/265) Updating files: 31% (83/265) Updating files: 32% (85/265) Updating files: 33% (88/265) Updating files: 34% (91/265) Updating files: 35% (93/265) Updating files: 36% (96/265) Updating files: 37% (99/265) Updating files: 38% (101/265) Updating files: 39% (104/265) Updating files: 40% (106/265) Updating files: 41% (109/265) Updating files: 42% (112/265) Updating files: 43% (114/265) Updating files: 44% (117/265) Updating files: 45% (120/265) Updating files: 46% (122/265) Updating files: 47% (125/265) Updating files: 48% (128/265) Updating files: 49% (130/265) Updating files: 50% (133/265) Updating files: 51% (136/265) Updating files: 52% (138/265) Updating files: 53% (141/265) Updating files: 54% (144/265) Updating files: 55% (146/265) Updating files: 56% (149/265) Updating files: 57% (152/265) Updating files: 58% (154/265) Updating files: 59% (157/265) Updating files: 60% (159/265) Updating files: 61% (162/265) Updating files: 62% (165/265) Updating files: 63% (167/265) Updating files: 64% (170/265) Updating files: 65% (173/265) Updating files: 66% (175/265) Updating files: 67% (178/265) Updating files: 68% (181/265) Updating files: 69% (183/265) Updating files: 70% (186/265) Updating files: 71% (189/265) Updating files: 72% (191/265) Updating files: 73% (194/265) Updating files: 74% (197/265) Updating files: 75% (199/265) Updating files: 76% (202/265) Updating files: 77% (205/265) Updating files: 78% (207/265) Updating files: 79% (210/265) Updating files: 80% (212/265) Updating files: 81% (215/265) Updating files: 82% (218/265) Updating files: 83% (220/265) Updating files: 84% (223/265) Updating files: 85% (226/265) Updating files: 86% (228/265) Updating files: 87% (231/265) Updating files: 88% (234/265) Updating files: 89% (236/265) Updating files: 89% (238/265) Updating files: 90% (239/265) Updating files: 91% (242/265) Updating files: 92% (244/265) Updating files: 93% (247/265) Updating files: 94% (250/265) Updating files: 95% (252/265) Updating files: 96% (255/265) Updating files: 97% (258/265) Updating files: 98% (260/265) Updating files: 99% (263/265) Updating files: 100% (265/265) Updating files: 100% (265/265), done. 2024-03-09T09:16:53.634 INFO:tasks.workunit.client.0.smithi067.stdout:HEAD is now at 08bd1e1eee3 0.90 2024-03-09T09:16:53.642 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/v0.90 2024-03-09T09:16:53.969 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:16:53.970 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 48 ']' 2024-03-09T09:16:53.970 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.1 2024-03-09T09:16:53.970 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.1 2024-03-09T09:16:53.970 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.1 2024-03-09T09:16:53.970 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.1 2024-03-09T09:16:54.553 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:16:54.852 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 11% (22/197) Updating files: 12% (24/197) Updating files: 13% (26/197) Updating files: 14% (28/197) Updating files: 15% (30/197) Updating files: 16% (32/197) Updating files: 17% (34/197) Updating files: 18% (36/197) Updating files: 19% (38/197) Updating files: 20% (40/197) Updating files: 21% (42/197) Updating files: 22% (44/197) Updating files: 23% (46/197) Updating files: 24% (48/197) Updating files: 25% (50/197) Updating files: 26% (52/197) Updating files: 27% (54/197) Updating files: 27% (55/197) Updating files: 28% (56/197) Updating files: 29% (58/197) Updating files: 30% (60/197) Updating files: 31% (62/197) Updating files: 32% (64/197) Updating files: 33% (66/197) Updating files: 34% (67/197) Updating files: 35% (69/197) Updating files: 36% (71/197) Updating files: 37% (73/197) Updating files: 38% (75/197) Updating files: 39% (77/197) Updating files: 40% (79/197) Updating files: 41% (81/197) Updating files: 42% (83/197) Updating files: 43% (85/197) Updating files: 44% (87/197) Updating files: 45% (89/197) Updating files: 46% (91/197) Updating files: 47% (93/197) Updating files: 48% (95/197) Updating files: 49% (97/197) Updating files: 50% (99/197) Updating files: 51% (101/197) Updating files: 52% (103/197) Updating files: 53% (105/197) Updating files: 54% (107/197) Updating files: 54% (108/197) Updating files: 55% (109/197) Updating files: 56% (111/197) Updating files: 57% (113/197) Updating files: 58% (115/197) Updating files: 59% (117/197) Updating files: 60% (119/197) Updating files: 61% (121/197) Updating files: 62% (123/197) Updating files: 63% (125/197) Updating files: 64% (127/197) Updating files: 65% (129/197) Updating files: 66% (131/197) Updating files: 67% (132/197) Updating files: 68% (134/197) Updating files: 69% (136/197) Updating files: 70% (138/197) Updating files: 71% (140/197) Updating files: 72% (142/197) Updating files: 73% (144/197) Updating files: 74% (146/197) Updating files: 75% (148/197) Updating files: 76% (150/197) Updating files: 77% (152/197) Updating files: 78% (154/197) Updating files: 79% (156/197) Updating files: 80% (158/197) Updating files: 81% (160/197) Updating files: 82% (162/197) Updating files: 83% (164/197) Updating files: 84% (166/197) Updating files: 85% (168/197) Updating files: 86% (170/197) Updating files: 87% (172/197) Updating files: 88% (174/197) Updating files: 89% (176/197) Updating files: 90% (178/197) Updating files: 91% (180/197) Updating files: 92% (182/197) Updating files: 93% (184/197) Updating files: 94% (186/197) Updating files: 95% (188/197) Updating files: 96% (190/197) Updating files: 97% (192/197) Updating files: 98% (194/197) Updating files: 99% (196/197) Updating files: 100% (197/197) Updating files: 100% (197/197), done. 2024-03-09T09:16:54.951 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 68fdc0f68e6 0.89 2024-03-09T09:16:54.970 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.89 2024-03-09T09:16:55.102 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:16:55.102 INFO:tasks.workunit.client.1.smithi134.stdout:v0.90 2024-03-09T09:16:55.103 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 90 -eq 48 ']' 2024-03-09T09:16:55.103 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.90 2024-03-09T09:16:55.103 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo v0.90 2024-03-09T09:16:55.103 INFO:tasks.workunit.client.1.smithi134.stderr:+ git reset --hard v0.90 2024-03-09T09:16:58.987 INFO:tasks.workunit.client.1.smithi134.stderr:Updating files: 21% (58/265) Updating files: 22% (59/265) Updating files: 23% (61/265) Updating files: 24% (64/265) Updating files: 25% (67/265) Updating files: 26% (69/265) Updating files: 27% (72/265) Updating files: 28% (75/265) Updating files: 29% (77/265) Updating files: 30% (80/265) Updating files: 31% (83/265) Updating files: 32% (85/265) Updating files: 33% (88/265) Updating files: 34% (91/265) Updating files: 35% (93/265) Updating files: 36% (96/265) Updating files: 37% (99/265) Updating files: 38% (101/265) Updating files: 39% (104/265) Updating files: 40% (106/265) Updating files: 41% (109/265) Updating files: 42% (112/265) Updating files: 43% (114/265) Updating files: 44% (117/265) Updating files: 45% (120/265) Updating files: 46% (122/265) Updating files: 47% (125/265) Updating files: 48% (128/265) Updating files: 49% (130/265) Updating files: 50% (133/265) Updating files: 51% (136/265) Updating files: 52% (138/265) Updating files: 53% (141/265) Updating files: 54% (144/265) Updating files: 55% (146/265) Updating files: 56% (149/265) Updating files: 57% (152/265) Updating files: 58% (154/265) Updating files: 59% (157/265) Updating files: 60% (159/265) Updating files: 61% (162/265) Updating files: 62% (165/265) Updating files: 63% (167/265) Updating files: 64% (170/265) Updating files: 65% (173/265) Updating files: 66% (175/265) Updating files: 67% (178/265) Updating files: 68% (181/265) Updating files: 69% (183/265) Updating files: 70% (186/265) Updating files: 71% (189/265) Updating files: 72% (191/265) Updating files: 73% (194/265) Updating files: 74% (197/265) Updating files: 75% (199/265) Updating files: 76% (202/265) Updating files: 77% (205/265) Updating files: 78% (207/265) Updating files: 79% (210/265) Updating files: 80% (212/265) Updating files: 81% (215/265) Updating files: 82% (218/265) Updating files: 83% (220/265) Updating files: 84% (223/265) Updating files: 85% (226/265) Updating files: 86% (228/265) Updating files: 87% (231/265) Updating files: 88% (234/265) Updating files: 89% (236/265) Updating files: 90% (239/265) Updating files: 90% (241/265) Updating files: 91% (242/265) Updating files: 92% (244/265) Updating files: 93% (247/265) Updating files: 94% (250/265) Updating files: 95% (252/265) Updating files: 96% (255/265) Updating files: 97% (258/265) Updating files: 98% (260/265) Updating files: 99% (263/265) Updating files: 100% (265/265) Updating files: 100% (265/265), done. 2024-03-09T09:16:59.071 INFO:tasks.workunit.client.1.smithi134.stdout:HEAD is now at 08bd1e1eee3 0.90 2024-03-09T09:16:59.084 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/v0.90 2024-03-09T09:16:59.135 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:16:59.135 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:16:59.135 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 2 -eq 48 ']' 2024-03-09T09:16:59.135 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.2 2024-03-09T09:16:59.135 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.2 2024-03-09T09:16:59.136 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.2 2024-03-09T09:16:59.136 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.2 2024-03-09T09:16:59.137 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:16:59.409 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:16:59.409 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.1 2024-03-09T09:16:59.410 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 1 -eq 48 ']' 2024-03-09T09:16:59.410 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.1 2024-03-09T09:16:59.410 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.1 2024-03-09T09:16:59.410 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.1 2024-03-09T09:16:59.617 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:17:02.567 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:02.567 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:02.567 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 3 -eq 48 ']' 2024-03-09T09:17:02.568 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.3 2024-03-09T09:17:02.568 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.3 2024-03-09T09:17:02.568 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.3 2024-03-09T09:17:02.568 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.3 2024-03-09T09:17:02.571 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:06.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:06.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:06.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 4 -eq 48 ']' 2024-03-09T09:17:06.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.4 2024-03-09T09:17:06.765 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.4 2024-03-09T09:17:06.766 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.4 2024-03-09T09:17:06.766 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.4 2024-03-09T09:17:06.768 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:11.025 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:11.025 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:11.025 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 5 -eq 48 ']' 2024-03-09T09:17:11.025 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.5 2024-03-09T09:17:11.025 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.5 2024-03-09T09:17:11.025 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.5 2024-03-09T09:17:11.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.5 2024-03-09T09:17:11.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:13.129 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:17:13.129 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-03-09T09:17:13.129 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-09T09:17:13.129 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-09T09:17:13.137 INFO:tasks.ceph.mon.b:Started 2024-03-09T09:17:13.137 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:17:13.138 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:17:13.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.368+0000 7f1448600640 1 Processor -- start 2024-03-09T09:17:13.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.368+0000 7f1448600640 1 -- start start 2024-03-09T09:17:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1448600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f14400c30f0 0x7f1440152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1448600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f14400aa420 0x7f14401490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1448600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14400ab3c0 con 0x7f1440149630 2024-03-09T09:17:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1448600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14400c82d0 con 0x7f14400aa420 2024-03-09T09:17:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1448600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14400aa8a0 con 0x7f14400c30f0 2024-03-09T09:17:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1445200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f14400aa420 0x7f14401490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1446600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1440149630 0x7f144006dbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37472/0 (socket says 172.21.15.67:37472) 2024-03-09T09:17:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1446600640 1 -- 172.21.15.67:0/1502674367 learned_addr learned my addr 172.21.15.67:0/1502674367 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:17:13.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1445c00640 1 --2- 172.21.15.67:0/1502674367 >> v2:172.21.15.134:3300/0 conn(0x7f14400c30f0 0x7f1440152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:13.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1445c00640 1 -- 172.21.15.67:0/1502674367 >> v2:172.21.15.134:3300/0 conn(0x7f14400c30f0 msgr2=0x7f1440152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:17:13.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1445c00640 1 -- 172.21.15.67:0/1502674367 >> v2:172.21.15.134:3300/0 conn(0x7f14400c30f0 msgr2=0x7f1440152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:17:13.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.370+0000 7f1445c00640 1 --2- 172.21.15.67:0/1502674367 >> v2:172.21.15.134:3300/0 conn(0x7f14400c30f0 0x7f1440152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:13.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.371+0000 7f1445c00640 1 --2- 172.21.15.67:0/1502674367 >> v2:172.21.15.134:3300/0 conn(0x7f14400c30f0 0x7f1440152a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:17:13.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.371+0000 7f1445200640 1 -- 172.21.15.67:0/1502674367 >> v2:172.21.15.134:3300/0 conn(0x7f14400c30f0 msgr2=0x7f1440152a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:17:13.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.371+0000 7f1445200640 1 --2- 172.21.15.67:0/1502674367 >> v2:172.21.15.134:3300/0 conn(0x7f14400c30f0 0x7f1440152a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:13.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.371+0000 7f1445200640 1 -- 172.21.15.67:0/1502674367 >> v1:172.21.15.67:6789/0 conn(0x7f1440149630 legacy=0x7f144006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:13.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.371+0000 7f1445200640 1 -- 172.21.15.67:0/1502674367 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14400aab20 con 0x7f14400aa420 2024-03-09T09:17:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.372+0000 7f1445200640 1 --2- 172.21.15.67:0/1502674367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14400aa420 0x7f14401490f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f14400702f0 tx=0x7f142c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41af253d9cb08640 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.372+0000 7f143fe00640 1 -- 172.21.15.67:0/1502674367 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0315a0 con 0x7f14400aa420 2024-03-09T09:17:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.372+0000 7f143fe00640 1 -- 172.21.15.67:0/1502674367 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c037070 con 0x7f14400aa420 2024-03-09T09:17:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.372+0000 7f143fe00640 1 -- 172.21.15.67:0/1502674367 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c031960 con 0x7f14400aa420 2024-03-09T09:17:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.373+0000 7f1448600640 1 -- 172.21.15.67:0/1502674367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14400aa420 msgr2=0x7f14401490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.373+0000 7f1448600640 1 --2- 172.21.15.67:0/1502674367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14400aa420 0x7f14401490f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f14400702f0 tx=0x7f142c02b840 comp rx=0 tx=0).stop 2024-03-09T09:17:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.373+0000 7f1448600640 1 -- 172.21.15.67:0/1502674367 shutdown_connections 2024-03-09T09:17:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.373+0000 7f1448600640 1 --2- 172.21.15.67:0/1502674367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14400aa420 0x7f14401490f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.373+0000 7f1448600640 1 --2- 172.21.15.67:0/1502674367 >> v2:172.21.15.134:3300/0 conn(0x7f14400c30f0 0x7f1440152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.373+0000 7f1448600640 1 -- 172.21.15.67:0/1502674367 >> 172.21.15.67:0/1502674367 conn(0x7f14400c7370 msgr2=0x7f14400c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.373+0000 7f1448600640 1 -- 172.21.15.67:0/1502674367 shutdown_connections 2024-03-09T09:17:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.373+0000 7f1448600640 1 -- 172.21.15.67:0/1502674367 wait complete. 2024-03-09T09:17:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.374+0000 7f1448600640 1 Processor -- start 2024-03-09T09:17:13.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.375+0000 7f1448600640 1 -- start start 2024-03-09T09:17:13.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.375+0000 7f1448600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14400aa420 0x7f1440146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:13.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.375+0000 7f1448600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14400c30f0 0x7f1440145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:13.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.375+0000 7f1448600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1440149630 0x7f1440144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:13.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1448600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144006d6c0 con 0x7f14400aa420 2024-03-09T09:17:13.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1448600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14400ab0e0 con 0x7f1440149630 2024-03-09T09:17:13.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1448600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14400ab620 con 0x7f14400c30f0 2024-03-09T09:17:13.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1445c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14400aa420 0x7f1440146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:13.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1445c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14400aa420 0x7f1440146800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43148/0 (socket says 172.21.15.67:43148) 2024-03-09T09:17:13.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1445c00640 1 -- 172.21.15.67:0/1230151466 learned_addr learned my addr 172.21.15.67:0/1230151466 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1445200640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14400c30f0 0x7f1440145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1445c00640 1 -- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14400c30f0 msgr2=0x7f1440145350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1446600640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1440149630 0x7f1440144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1445c00640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14400c30f0 0x7f1440145350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1445c00640 1 -- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1440149630 msgr2=0x7f1440144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1445c00640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1440149630 0x7f1440144990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1445c00640 1 -- 172.21.15.67:0/1230151466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440145ad0 con 0x7f14400aa420 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.376+0000 7f1445200640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14400c30f0 0x7f1440145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.377+0000 7f1445c00640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14400aa420 0x7f1440146800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1434008710 tx=0x7f1434008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.377+0000 7f143d600640 1 -- 172.21.15.67:0/1230151466 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434019070 con 0x7f14400aa420 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.377+0000 7f143d600640 1 -- 172.21.15.67:0/1230151466 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143400d630 con 0x7f14400aa420 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.377+0000 7f143d600640 1 -- 172.21.15.67:0/1230151466 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143400da20 con 0x7f14400aa420 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.377+0000 7f1448600640 1 -- 172.21.15.67:0/1230151466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14401ebc30 con 0x7f14400aa420 2024-03-09T09:17:13.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.377+0000 7f1448600640 1 -- 172.21.15.67:0/1230151466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1440144ed0 con 0x7f14400aa420 2024-03-09T09:17:13.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.378+0000 7f1448600640 1 -- 172.21.15.67:0/1230151466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1440075a00 con 0x7f14400aa420 2024-03-09T09:17:13.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.379+0000 7f143d600640 1 -- 172.21.15.67:0/1230151466 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f143401c070 con 0x7f14400aa420 2024-03-09T09:17:13.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.380+0000 7f143d600640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f142006b500 0x7f142006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:13.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.380+0000 7f143d600640 1 -- 172.21.15.67:0/1230151466 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1434089c60 con 0x7f14400aa420 2024-03-09T09:17:13.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.380+0000 7f1445200640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f142006b500 0x7f142006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:13.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.381+0000 7f1445200640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f142006b500 0x7f142006d9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f142c036000 tx=0x7f142c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:13.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.389+0000 7f143d600640 1 -- 172.21.15.67:0/1230151466 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f14340586e0 con 0x7f14400aa420 2024-03-09T09:17:13.602 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:13.602 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:13.602 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 6 -eq 48 ']' 2024-03-09T09:17:13.602 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.6 2024-03-09T09:17:13.602 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.6 2024-03-09T09:17:13.603 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.6 2024-03-09T09:17:13.603 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.6 2024-03-09T09:17:13.605 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:13.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.687+0000 7f1448600640 1 -- 172.21.15.67:0/1230151466 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f14400c54f0 con 0x7f14400aa420 2024-03-09T09:17:13.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.687+0000 7f143d600640 1 -- 172.21.15.67:0/1230151466 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f1434057fe0 con 0x7f14400aa420 2024-03-09T09:17:13.693 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":184,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:17:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 -- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f142006b500 msgr2=0x7f142006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f142006b500 0x7f142006d9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f142c036000 tx=0x7f142c008030 comp rx=0 tx=0).stop 2024-03-09T09:17:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 -- 172.21.15.67:0/1230151466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14400aa420 msgr2=0x7f1440146800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14400aa420 0x7f1440146800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1434008710 tx=0x7f1434008800 comp rx=0 tx=0).stop 2024-03-09T09:17:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 -- 172.21.15.67:0/1230151466 shutdown_connections 2024-03-09T09:17:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f142006b500 0x7f142006d9c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1440149630 0x7f1440144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:13.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f14400c30f0 0x7f1440145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 --2- 172.21.15.67:0/1230151466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14400aa420 0x7f1440146800 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 -- 172.21.15.67:0/1230151466 >> 172.21.15.67:0/1230151466 conn(0x7f14400c7370 msgr2=0x7f14401528a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 -- 172.21.15.67:0/1230151466 shutdown_connections 2024-03-09T09:17:13.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:13.690+0000 7f1448600640 1 -- 172.21.15.67:0/1230151466 wait complete. 2024-03-09T09:17:15.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:15.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:15.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 7 -eq 48 ']' 2024-03-09T09:17:15.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.7 2024-03-09T09:17:15.723 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.7 2024-03-09T09:17:15.723 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.7 2024-03-09T09:17:15.723 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.7 2024-03-09T09:17:15.725 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:15.892 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:17:15.892 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:17:15.892 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 2 -eq 48 ']' 2024-03-09T09:17:15.892 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.2 2024-03-09T09:17:15.892 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.2 2024-03-09T09:17:15.893 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.2 2024-03-09T09:17:15.893 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.2 2024-03-09T09:17:15.896 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:17:16.712 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:17:16.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.903+0000 7fd096a00640 1 Processor -- start 2024-03-09T09:17:16.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.904+0000 7fd096a00640 1 -- start start 2024-03-09T09:17:16.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.905+0000 7fd096a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0900c5e40 0x7fd090156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:16.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.905+0000 7fd096a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd0900cadb0 0x7fd090156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:16.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.905+0000 7fd096a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd090156a90 con 0x7fd090153300 2024-03-09T09:17:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.905+0000 7fd096a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd090157000 con 0x7fd0900c5e40 2024-03-09T09:17:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.905+0000 7fd096a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0900ae430 con 0x7fd0900cadb0 2024-03-09T09:17:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.905+0000 7fd095600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd090153300 0x7fd0900b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37476/0 (socket says 172.21.15.67:37476) 2024-03-09T09:17:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.905+0000 7fd095600640 1 -- 172.21.15.67:0/2214315479 learned_addr learned my addr 172.21.15.67:0/2214315479 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:17:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.906+0000 7fd094c00640 1 --2- 172.21.15.67:0/2214315479 >> v2:172.21.15.67:3300/0 conn(0x7fd0900c5e40 0x7fd090156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.906+0000 7fd08fe00640 1 -- 172.21.15.67:0/2214315479 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905507417 0 0) 0x7fd090156a90 con 0x7fd090153300 2024-03-09T09:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.906+0000 7fd094c00640 1 -- 172.21.15.67:0/2214315479 >> v2:172.21.15.134:3300/0 conn(0x7fd0900cadb0 msgr2=0x7fd090156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.906+0000 7fd096000640 1 --2- 172.21.15.67:0/2214315479 >> v2:172.21.15.134:3300/0 conn(0x7fd0900cadb0 0x7fd090156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.906+0000 7fd094c00640 1 --2- 172.21.15.67:0/2214315479 >> v2:172.21.15.134:3300/0 conn(0x7fd0900cadb0 0x7fd090156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.906+0000 7fd094c00640 1 -- 172.21.15.67:0/2214315479 >> v1:172.21.15.67:6789/0 conn(0x7fd090153300 legacy=0x7fd0900b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.906+0000 7fd094c00640 1 -- 172.21.15.67:0/2214315479 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0900ae6f0 con 0x7fd0900c5e40 2024-03-09T09:17:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.906+0000 7fd096000640 1 --2- 172.21.15.67:0/2214315479 >> v2:172.21.15.134:3300/0 conn(0x7fd0900cadb0 0x7fd090156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:17:16.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.907+0000 7fd094c00640 1 --2- 172.21.15.67:0/2214315479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0900c5e40 0x7fd090156010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd0900adc60 tx=0x7fd07c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f379335566516318 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:16.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.908+0000 7fd08fe00640 1 -- 172.21.15.67:0/2214315479 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c035020 con 0x7fd0900c5e40 2024-03-09T09:17:16.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.908+0000 7fd08fe00640 1 -- 172.21.15.67:0/2214315479 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c031830 con 0x7fd0900c5e40 2024-03-09T09:17:16.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.908+0000 7fd08fe00640 1 -- 172.21.15.67:0/2214315479 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c031c00 con 0x7fd0900c5e40 2024-03-09T09:17:16.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.908+0000 7fd096a00640 1 -- 172.21.15.67:0/2214315479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0900c5e40 msgr2=0x7fd090156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:16.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.908+0000 7fd096a00640 1 --2- 172.21.15.67:0/2214315479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0900c5e40 0x7fd090156010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd0900adc60 tx=0x7fd07c02b8d0 comp rx=0 tx=0).stop 2024-03-09T09:17:16.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.908+0000 7fd096a00640 1 -- 172.21.15.67:0/2214315479 shutdown_connections 2024-03-09T09:17:16.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.908+0000 7fd096a00640 1 --2- 172.21.15.67:0/2214315479 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0900c5e40 0x7fd090156010 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:16.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.908+0000 7fd096a00640 1 --2- 172.21.15.67:0/2214315479 >> v2:172.21.15.134:3300/0 conn(0x7fd0900cadb0 0x7fd090156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:16.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.908+0000 7fd096a00640 1 -- 172.21.15.67:0/2214315479 >> 172.21.15.67:0/2214315479 conn(0x7fd0900732d0 msgr2=0x7fd0900ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:16.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.908+0000 7fd096a00640 1 -- 172.21.15.67:0/2214315479 shutdown_connections 2024-03-09T09:17:16.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.908+0000 7fd096a00640 1 -- 172.21.15.67:0/2214315479 wait complete. 2024-03-09T09:17:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.910+0000 7fd096a00640 1 Processor -- start 2024-03-09T09:17:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.910+0000 7fd096a00640 1 -- start start 2024-03-09T09:17:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.911+0000 7fd096a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0900c5e40 0x7fd09006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.911+0000 7fd096a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0900cadb0 0x7fd09014e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.911+0000 7fd096a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd090153300 0x7fd0901ee740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:16.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.911+0000 7fd096a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd090159e90 con 0x7fd0900c5e40 2024-03-09T09:17:16.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.911+0000 7fd096a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd090159bb0 con 0x7fd090153300 2024-03-09T09:17:16.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.911+0000 7fd096a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900c59e0 con 0x7fd0900cadb0 2024-03-09T09:17:16.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.911+0000 7fd095600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0900c5e40 0x7fd09006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:16.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.911+0000 7fd095600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0900c5e40 0x7fd09006bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43170/0 (socket says 172.21.15.67:43170) 2024-03-09T09:17:16.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.911+0000 7fd095600640 1 -- 172.21.15.67:0/3044299455 learned_addr learned my addr 172.21.15.67:0/3044299455 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:16.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.912+0000 7fd095600640 1 -- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0900cadb0 msgr2=0x7fd09014e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:16.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.912+0000 7fd095600640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0900cadb0 0x7fd09014e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:16.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.912+0000 7fd095600640 1 -- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd090153300 msgr2=0x7fd0901ee740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:16.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.912+0000 7fd095600640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd090153300 0x7fd0901ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:16.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.912+0000 7fd095600640 1 -- 172.21.15.67:0/3044299455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09006b1e0 con 0x7fd0900c5e40 2024-03-09T09:17:16.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.912+0000 7fd095600640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0900c5e40 0x7fd09006bf70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c460 tx=0x7fd08400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:16.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.912+0000 7fd08d600640 1 -- 172.21.15.67:0/3044299455 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084018070 con 0x7fd0900c5e40 2024-03-09T09:17:16.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.912+0000 7fd08d600640 1 -- 172.21.15.67:0/3044299455 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd084007500 con 0x7fd0900c5e40 2024-03-09T09:17:16.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.912+0000 7fd096a00640 1 -- 172.21.15.67:0/3044299455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0901ef0f0 con 0x7fd0900c5e40 2024-03-09T09:17:16.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.912+0000 7fd08d600640 1 -- 172.21.15.67:0/3044299455 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084004a20 con 0x7fd0900c5e40 2024-03-09T09:17:16.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.913+0000 7fd096a00640 1 -- 172.21.15.67:0/3044299455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09014e760 con 0x7fd0900c5e40 2024-03-09T09:17:16.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.914+0000 7fd096a00640 1 -- 172.21.15.67:0/3044299455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09004f3c0 con 0x7fd0900c5e40 2024-03-09T09:17:16.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.916+0000 7fd08d600640 1 -- 172.21.15.67:0/3044299455 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd08401b070 con 0x7fd0900c5e40 2024-03-09T09:17:16.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.917+0000 7fd08d600640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd07006b500 0x7fd07006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:16.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.917+0000 7fd08d600640 1 -- 172.21.15.67:0/3044299455 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd0840897b0 con 0x7fd0900c5e40 2024-03-09T09:17:16.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.918+0000 7fd094c00640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd07006b500 0x7fd07006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:16.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.919+0000 7fd094c00640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd07006b500 0x7fd07006d9c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd07c036000 tx=0x7fd07c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:16.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:16.922+0000 7fd08d600640 1 -- 172.21.15.67:0/3044299455 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0840582d0 con 0x7fd0900c5e40 2024-03-09T09:17:17.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.195+0000 7fd096a00640 1 -- 172.21.15.67:0/3044299455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd0900cab90 con 0x7fd0900c5e40 2024-03-09T09:17:17.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.199+0000 7fd08d600640 1 -- 172.21.15.67:0/3044299455 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fd084057bd0 con 0x7fd0900c5e40 2024-03-09T09:17:17.205 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":186,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:17:17.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 -- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd07006b500 msgr2=0x7fd07006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd07006b500 0x7fd07006d9c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd07c036000 tx=0x7fd07c030000 comp rx=0 tx=0).stop 2024-03-09T09:17:17.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 -- 172.21.15.67:0/3044299455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0900c5e40 msgr2=0x7fd09006bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0900c5e40 0x7fd09006bf70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c460 tx=0x7fd08400c920 comp rx=0 tx=0).stop 2024-03-09T09:17:17.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 -- 172.21.15.67:0/3044299455 shutdown_connections 2024-03-09T09:17:17.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd07006b500 0x7fd07006d9c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd090153300 0x7fd0901ee740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0900cadb0 0x7fd09014e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 --2- 172.21.15.67:0/3044299455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0900c5e40 0x7fd09006bf70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 -- 172.21.15.67:0/3044299455 >> 172.21.15.67:0/3044299455 conn(0x7fd0900732d0 msgr2=0x7fd090150a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:17.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 -- 172.21.15.67:0/3044299455 shutdown_connections 2024-03-09T09:17:17.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.202+0000 7fd096a00640 1 -- 172.21.15.67:0/3044299455 wait complete. 2024-03-09T09:17:17.225 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:17:17.225 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:17:17.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:17:17.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.464+0000 7fd5fe600640 1 Processor -- start 2024-03-09T09:17:17.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.464+0000 7fd5fe600640 1 -- start start 2024-03-09T09:17:17.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.465+0000 7fd5fe600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5f80c22a0 0x7fd5f8159410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.465+0000 7fd5fe600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd5f806e8d0 0x7fd5f80c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.465+0000 7fd5fe600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f81538e0 con 0x7fd5f8153500 2024-03-09T09:17:17.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.465+0000 7fd5fe600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f806ed60 con 0x7fd5f806e8d0 2024-03-09T09:17:17.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.465+0000 7fd5fe600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f806f8d0 con 0x7fd5f80c22a0 2024-03-09T09:17:17.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.465+0000 7fd5f7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd5f8153500 0x7fd5f806f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37480/0 (socket says 172.21.15.67:37480) 2024-03-09T09:17:17.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.465+0000 7fd5f7400640 1 -- 172.21.15.67:0/679579992 learned_addr learned my addr 172.21.15.67:0/679579992 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:17:17.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.466+0000 7fd5f7e00640 1 --2- 172.21.15.67:0/679579992 >> v2:172.21.15.67:3300/0 conn(0x7fd5f806e8d0 0x7fd5f80c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.466+0000 7fd5eea00640 1 --2- 172.21.15.67:0/679579992 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5f80c22a0 0x7fd5f8159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.466+0000 7fd5f6a00640 1 -- 172.21.15.67:0/679579992 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714002279 0 0) 0x7fd5e8001490 con 0x7fd5f8153500 2024-03-09T09:17:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.466+0000 7fd5f6a00640 1 -- 172.21.15.67:0/679579992 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5dc003520 con 0x7fd5f8153500 2024-03-09T09:17:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.466+0000 7fd5f7e00640 1 -- 172.21.15.67:0/679579992 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5f80c22a0 msgr2=0x7fd5f8159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.466+0000 7fd5f7e00640 1 --2- 172.21.15.67:0/679579992 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5f80c22a0 0x7fd5f8159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.466+0000 7fd5f7e00640 1 -- 172.21.15.67:0/679579992 >> v1:172.21.15.67:6789/0 conn(0x7fd5f8153500 legacy=0x7fd5f806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.466+0000 7fd5f7e00640 1 -- 172.21.15.67:0/679579992 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f806b160 con 0x7fd5f806e8d0 2024-03-09T09:17:17.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.467+0000 7fd5f7e00640 1 --2- 172.21.15.67:0/679579992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f806e8d0 0x7fd5f80c8370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00097a0 tx=0x7fd5f002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914dd05ed221390d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:17.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.468+0000 7fd5f6a00640 1 -- 172.21.15.67:0/679579992 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0035020 con 0x7fd5f806e8d0 2024-03-09T09:17:17.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.468+0000 7fd5f6a00640 1 -- 172.21.15.67:0/679579992 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f0031810 con 0x7fd5f806e8d0 2024-03-09T09:17:17.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.468+0000 7fd5f6a00640 1 -- 172.21.15.67:0/679579992 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0031be0 con 0x7fd5f806e8d0 2024-03-09T09:17:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.468+0000 7fd5fe600640 1 -- 172.21.15.67:0/679579992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f806e8d0 msgr2=0x7fd5f80c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.468+0000 7fd5fe600640 1 --2- 172.21.15.67:0/679579992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f806e8d0 0x7fd5f80c8370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00097a0 tx=0x7fd5f002b8b0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.469+0000 7fd5fe600640 1 -- 172.21.15.67:0/679579992 shutdown_connections 2024-03-09T09:17:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.469+0000 7fd5fe600640 1 --2- 172.21.15.67:0/679579992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f806e8d0 0x7fd5f80c8370 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.469+0000 7fd5fe600640 1 --2- 172.21.15.67:0/679579992 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5f80c22a0 0x7fd5f8159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.469+0000 7fd5fe600640 1 -- 172.21.15.67:0/679579992 >> 172.21.15.67:0/679579992 conn(0x7fd5f80aa890 msgr2=0x7fd5f80ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.469+0000 7fd5fe600640 1 -- 172.21.15.67:0/679579992 shutdown_connections 2024-03-09T09:17:17.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.469+0000 7fd5fe600640 1 -- 172.21.15.67:0/679579992 wait complete. 2024-03-09T09:17:17.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.470+0000 7fd5fe600640 1 Processor -- start 2024-03-09T09:17:17.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.471+0000 7fd5fe600640 1 -- start start 2024-03-09T09:17:17.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.471+0000 7fd5fe600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f806e8d0 0x7fd5f814a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.471+0000 7fd5fe600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5f80c22a0 0x7fd5f8072680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.471+0000 7fd5fe600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5f8153500 0x7fd5f8072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.471+0000 7fd5fe600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80c47e0 con 0x7fd5f806e8d0 2024-03-09T09:17:17.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.471+0000 7fd5fe600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80c4500 con 0x7fd5f8153500 2024-03-09T09:17:17.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.471+0000 7fd5fe600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80c4ac0 con 0x7fd5f80c22a0 2024-03-09T09:17:17.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.472+0000 7fd5f7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f806e8d0 0x7fd5f814a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.472+0000 7fd5f7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f806e8d0 0x7fd5f814a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43188/0 (socket says 172.21.15.67:43188) 2024-03-09T09:17:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.472+0000 7fd5f7400640 1 -- 172.21.15.67:0/122090423 learned_addr learned my addr 172.21.15.67:0/122090423 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.472+0000 7fd5f7e00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5f8153500 0x7fd5f8072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.472+0000 7fd5eea00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5f80c22a0 0x7fd5f8072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.472+0000 7fd5f7400640 1 -- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5f80c22a0 msgr2=0x7fd5f8072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.472+0000 7fd5f7400640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5f80c22a0 0x7fd5f8072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.472+0000 7fd5f7400640 1 -- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5f8153500 msgr2=0x7fd5f8072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.472+0000 7fd5f7400640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5f8153500 0x7fd5f8072bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.472+0000 7fd5f7400640 1 -- 172.21.15.67:0/122090423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f8151e50 con 0x7fd5f806e8d0 2024-03-09T09:17:17.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.472+0000 7fd5f7e00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5f8153500 0x7fd5f8072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:17:17.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.473+0000 7fd5f7400640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f806e8d0 0x7fd5f814a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800cb10 tx=0x7fd5e800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:17.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.473+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8017070 con 0x7fd5f806e8d0 2024-03-09T09:17:17.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.473+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e800ac10 con 0x7fd5f806e8d0 2024-03-09T09:17:17.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.473+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8012400 con 0x7fd5f806e8d0 2024-03-09T09:17:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.473+0000 7fd5fe600640 1 -- 172.21.15.67:0/122090423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f81ec6e0 con 0x7fd5f806e8d0 2024-03-09T09:17:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.473+0000 7fd5fe600640 1 -- 172.21.15.67:0/122090423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f814a350 con 0x7fd5f806e8d0 2024-03-09T09:17:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.474+0000 7fd5fe600640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f814bc40 0x7fd5f8144c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.474+0000 7fd5fe600640 1 -- 172.21.15.67:0/122090423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f81ec490 con 0x7fd5f814bc40 2024-03-09T09:17:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.475+0000 7fd5eea00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f814bc40 0x7fd5f8144c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.475+0000 7fd5eea00640 1 -- 172.21.15.67:0/122090423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5f8075a00 con 0x7fd5f814bc40 2024-03-09T09:17:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.475+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd5e801a070 con 0x7fd5f806e8d0 2024-03-09T09:17:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.475+0000 7fd5eea00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f814bc40 0x7fd5f8144c60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0011560 tx=0x7fd5e0011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:17.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.477+0000 7fd5efe00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd5d406b4d0 0x7fd5d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.477+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd5e808a350 con 0x7fd5f806e8d0 2024-03-09T09:17:17.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.478+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0004230 con 0x7fd5f814bc40 2024-03-09T09:17:17.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.478+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd5e00043d0 con 0x7fd5f814bc40 2024-03-09T09:17:17.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.478+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f814bc40 msgr2=0x7fd5f8144c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.478+0000 7fd5efe00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f814bc40 0x7fd5f8144c60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0011560 tx=0x7fd5e0011a20 comp rx=0 tx=0).stop 2024-03-09T09:17:17.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.478+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f814bc40 msgr2=0x7fd5f8144c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:17.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.478+0000 7fd5efe00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f814bc40 0x7fd5f8144c60 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0011560 tx=0x7fd5e0011a20 comp rx=0 tx=0).stop 2024-03-09T09:17:17.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.478+0000 7fd5f7e00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd5d406b4d0 0x7fd5d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.479+0000 7fd5f7e00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd5d406b4d0 0x7fd5d406d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0036000 tx=0x7fd5f0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:17.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.487+0000 7fd5fe600640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f80735f0 0x7fd5f81451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.487+0000 7fd5fe600640 1 -- 172.21.15.67:0/122090423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f8073e20 con 0x7fd5f80735f0 2024-03-09T09:17:17.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.487+0000 7fd5eea00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f80735f0 0x7fd5f81451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.488+0000 7fd5eea00640 1 -- 172.21.15.67:0/122090423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd5f806b3c0 con 0x7fd5f80735f0 2024-03-09T09:17:17.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.488+0000 7fd5eea00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f80735f0 0x7fd5f81451a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0011560 tx=0x7fd5e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:17.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.489+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0003610 con 0x7fd5f80735f0 2024-03-09T09:17:17.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.489+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5019 (secure 0 0 0) 0x7fd5e0003610 con 0x7fd5f80735f0 2024-03-09T09:17:17.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.489+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f80735f0 msgr2=0x7fd5f81451a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.489+0000 7fd5efe00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f80735f0 0x7fd5f81451a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0011560 tx=0x7fd5e00046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.489+0000 7fd5efe00640 1 -- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f80735f0 msgr2=0x7fd5f81451a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:17.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.489+0000 7fd5efe00640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f80735f0 0x7fd5f81451a0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0011560 tx=0x7fd5e00046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.495 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:17:17.495 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:17:17.495 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:17:17.495 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 186, 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:17.496 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:17.497 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:17:17.498 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:17:17.499 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.500 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:17:17.501 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:17:17.502 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:17.503 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.504 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:17.505 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:17:17.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:17.507 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:17:17.508 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:17:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.489+0000 7fd5fe600640 1 -- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd5d406b4d0 msgr2=0x7fd5d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.489+0000 7fd5fe600640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd5d406b4d0 0x7fd5d406d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0036000 tx=0x7fd5f0008030 comp rx=0 tx=0).stop 2024-03-09T09:17:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.489+0000 7fd5fe600640 1 -- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f806e8d0 msgr2=0x7fd5f814a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.489+0000 7fd5fe600640 1 --2- 172.21.15.67:0/122090423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5f806e8d0 0x7fd5f814a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800cb10 tx=0x7fd5e800a060 comp rx=0 tx=0).stop 2024-03-09T09:17:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.489+0000 7fd5f7e00640 1 -- 172.21.15.67:0/122090423 reap_dead start 2024-03-09T09:17:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.490+0000 7fd5f7e00640 1 -- 172.21.15.67:0/122090423 reap_dead start 2024-03-09T09:17:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.490+0000 7fd5fe600640 1 -- 172.21.15.67:0/122090423 shutdown_connections 2024-03-09T09:17:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.490+0000 7fd5fe600640 1 -- 172.21.15.67:0/122090423 >> 172.21.15.67:0/122090423 conn(0x7fd5f80aa890 msgr2=0x7fd5f806c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.490+0000 7fd5fe600640 1 -- 172.21.15.67:0/122090423 shutdown_connections 2024-03-09T09:17:17.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.490+0000 7fd5fe600640 1 -- 172.21.15.67:0/122090423 wait complete. 2024-03-09T09:17:17.511 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:17:17.511 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:17:17.511 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:17:17.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.699+0000 7f0c5e600640 1 Processor -- start 2024-03-09T09:17:17.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.699+0000 7f0c5e600640 1 -- start start 2024-03-09T09:17:17.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.701+0000 7f0c5e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0c5814fef0 0x7f0c5806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.702+0000 7f0c5e600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c580bed00 0x7f0c580ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.702+0000 7f0c5e600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c58155e70 con 0x7f0c580c5ff0 2024-03-09T09:17:17.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.702+0000 7f0c5e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c580c9d60 con 0x7f0c5814fef0 2024-03-09T09:17:17.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.702+0000 7f0c5e600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580c9710 con 0x7f0c580bed00 2024-03-09T09:17:17.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.702+0000 7f0c57400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0c5814fef0 0x7f0c5806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.702+0000 7f0c56a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0c580c5ff0 0x7f0c580ca1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37496/0 (socket says 172.21.15.67:37496) 2024-03-09T09:17:17.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.702+0000 7f0c56a00640 1 -- 172.21.15.67:0/2188807140 learned_addr learned my addr 172.21.15.67:0/2188807140 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:17:17.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.702+0000 7f0c57e00640 1 --2- 172.21.15.67:0/2188807140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c580bed00 0x7f0c580ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.702+0000 7f0c57400640 1 -- 172.21.15.67:0/2188807140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c580bed00 msgr2=0x7f0c580ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:17.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.702+0000 7f0c57400640 1 --2- 172.21.15.67:0/2188807140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c580bed00 0x7f0c580ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.703+0000 7f0c57400640 1 -- 172.21.15.67:0/2188807140 >> v1:172.21.15.67:6789/0 conn(0x7f0c580c5ff0 legacy=0x7f0c580ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.703+0000 7f0c57400640 1 -- 172.21.15.67:0/2188807140 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c580c99f0 con 0x7f0c5814fef0 2024-03-09T09:17:17.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.703+0000 7f0c56000640 1 -- 172.21.15.67:0/2188807140 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257898469 0 0) 0x7f0c58155e70 con 0x7f0c580c5ff0 2024-03-09T09:17:17.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.704+0000 7f0c57400640 1 --2- 172.21.15.67:0/2188807140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5814fef0 0x7f0c5806b040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0c480097a0 tx=0x7f0c4802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d93f994657dd47a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:17.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.704+0000 7f0c56000640 1 -- 172.21.15.67:0/2188807140 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c480315a0 con 0x7f0c5814fef0 2024-03-09T09:17:17.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.704+0000 7f0c56000640 1 -- 172.21.15.67:0/2188807140 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c48037070 con 0x7f0c5814fef0 2024-03-09T09:17:17.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.704+0000 7f0c56000640 1 -- 172.21.15.67:0/2188807140 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48031960 con 0x7f0c5814fef0 2024-03-09T09:17:17.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.705+0000 7f0c5e600640 1 -- 172.21.15.67:0/2188807140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5814fef0 msgr2=0x7f0c5806b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.705+0000 7f0c5e600640 1 --2- 172.21.15.67:0/2188807140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5814fef0 0x7f0c5806b040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0c480097a0 tx=0x7f0c4802b840 comp rx=0 tx=0).stop 2024-03-09T09:17:17.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.705+0000 7f0c5e600640 1 -- 172.21.15.67:0/2188807140 shutdown_connections 2024-03-09T09:17:17.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.705+0000 7f0c5e600640 1 --2- 172.21.15.67:0/2188807140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c580bed00 0x7f0c580ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.705+0000 7f0c5e600640 1 --2- 172.21.15.67:0/2188807140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c5814fef0 0x7f0c5806b040 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.705+0000 7f0c5e600640 1 -- 172.21.15.67:0/2188807140 >> 172.21.15.67:0/2188807140 conn(0x7f0c580cb670 msgr2=0x7f0c580cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:17.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.705+0000 7f0c5e600640 1 -- 172.21.15.67:0/2188807140 shutdown_connections 2024-03-09T09:17:17.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.705+0000 7f0c5e600640 1 -- 172.21.15.67:0/2188807140 wait complete. 2024-03-09T09:17:17.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.707+0000 7f0c5e600640 1 Processor -- start 2024-03-09T09:17:17.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.708+0000 7f0c5e600640 1 -- start start 2024-03-09T09:17:17.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.708+0000 7f0c5e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c580bed00 0x7f0c580bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.708+0000 7f0c5e600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c580c5ff0 0x7f0c580c02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.708+0000 7f0c5e600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c5814fef0 0x7f0c580c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.708+0000 7f0c5e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580cb470 con 0x7f0c580bed00 2024-03-09T09:17:17.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.708+0000 7f0c5e600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580cb2f0 con 0x7f0c5814fef0 2024-03-09T09:17:17.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.708+0000 7f0c5e600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5806b580 con 0x7f0c580c5ff0 2024-03-09T09:17:17.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.709+0000 7f0c57400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c580bed00 0x7f0c580bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.709+0000 7f0c57400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c580bed00 0x7f0c580bf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43228/0 (socket says 172.21.15.67:43228) 2024-03-09T09:17:17.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.709+0000 7f0c57400640 1 -- 172.21.15.67:0/3621593696 learned_addr learned my addr 172.21.15.67:0/3621593696 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:17.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.709+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c580c5ff0 0x7f0c580c02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.709+0000 7f0c57e00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c5814fef0 0x7f0c580c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.709+0000 7f0c57400640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c580c5ff0 msgr2=0x7f0c580c02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.709+0000 7f0c57400640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c580c5ff0 0x7f0c580c02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.709+0000 7f0c57400640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c5814fef0 msgr2=0x7f0c580c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:17.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.709+0000 7f0c57400640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c5814fef0 0x7f0c580c30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:17.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.709+0000 7f0c57400640 1 -- 172.21.15.67:0/3621593696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c580c2050 con 0x7f0c580bed00 2024-03-09T09:17:17.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.709+0000 7f0c57400640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c580bed00 0x7f0c580bf1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0c48000c00 tx=0x7f0c480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:17.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.710+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48004230 con 0x7f0c580bed00 2024-03-09T09:17:17.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.710+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c480043f0 con 0x7f0c580bed00 2024-03-09T09:17:17.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.710+0000 7f0c5e600640 1 -- 172.21.15.67:0/3621593696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c580c28e0 con 0x7f0c580bed00 2024-03-09T09:17:17.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.710+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48003850 con 0x7f0c580bed00 2024-03-09T09:17:17.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.710+0000 7f0c5e600640 1 -- 172.21.15.67:0/3621593696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c580c8810 con 0x7f0c580bed00 2024-03-09T09:17:17.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.711+0000 7f0c5e600640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.711+0000 7f0c5e600640 1 -- 172.21.15.67:0/3621593696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580c1e20 con 0x7f0c580c3620 2024-03-09T09:17:17.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.712+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.712+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0c480039f0 con 0x7f0c580bed00 2024-03-09T09:17:17.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.712+0000 7f0c56a00640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 msgr2=0x7f0c581487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-03-09T09:17:17.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.712+0000 7f0c56a00640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 msgr2=0x7f0c581487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:17:17.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.712+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:17.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.712+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:17:17.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.714+0000 7f0c4f400640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c3806b4d0 0x7f0c3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:17.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.714+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0c4802f030 con 0x7f0c580bed00 2024-03-09T09:17:17.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.714+0000 7f0c57e00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c3806b4d0 0x7f0c3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.715+0000 7f0c57e00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c3806b4d0 0x7f0c3806d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0c580bfe10 tx=0x7f0c5000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:17.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.913+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:17.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.914+0000 7f0c56a00640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 msgr2=0x7f0c581487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:17:17.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.914+0000 7f0c56a00640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 msgr2=0x7f0c581487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:17:17.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.914+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:17.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:17.914+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:17:18.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:18.315+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:18.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:18.316+0000 7f0c56a00640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 msgr2=0x7f0c581487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:17:18.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:18.316+0000 7f0c56a00640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 msgr2=0x7f0c581487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:17:18.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:18.316+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:18.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:18.316+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:17:19.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.117+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.119+0000 7f0c56a00640 1 -- 172.21.15.67:0/3621593696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c580c3a00 con 0x7f0c580c3620 2024-03-09T09:17:19.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.119+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0c580c2450 tx=0x7f0c44001970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:19.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.120+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44015740 con 0x7f0c580c3620 2024-03-09T09:17:19.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.121+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f0c4401b070 con 0x7f0c580c3620 2024-03-09T09:17:19.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.122+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 msgr2=0x7f0c581487a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.122+0000 7f0c4f400640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0c580c2450 tx=0x7f0c44001970 comp rx=0 tx=0).stop 2024-03-09T09:17:19.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.122+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 msgr2=0x7f0c581487a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:19.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.122+0000 7f0c4f400640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c580c3620 0x7f0c581487a0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0c580c2450 tx=0x7f0c44001970 comp rx=0 tx=0).stop 2024-03-09T09:17:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.129+0000 7f0c5e600640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c58148ce0 0x7f0c581ee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.129+0000 7f0c5e600640 1 -- 172.21.15.67:0/3621593696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580c2720 con 0x7f0c58148ce0 2024-03-09T09:17:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.130+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c58148ce0 0x7f0c581ee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.131+0000 7f0c56a00640 1 -- 172.21.15.67:0/3621593696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0c580684d0 con 0x7f0c58148ce0 2024-03-09T09:17:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.131+0000 7f0c56a00640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c58148ce0 0x7f0c581ee9b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c580738a0 tx=0x7f0c44000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:19.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.132+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44015d90 con 0x7f0c58148ce0 2024-03-09T09:17:19.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.133+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f0c44015d90 con 0x7f0c58148ce0 2024-03-09T09:17:19.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.133+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c58148ce0 msgr2=0x7f0c581ee9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.133+0000 7f0c4f400640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c58148ce0 0x7f0c581ee9b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c580738a0 tx=0x7f0c44000ed0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.133+0000 7f0c4f400640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c58148ce0 msgr2=0x7f0c581ee9b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:19.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.133+0000 7f0c4f400640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c58148ce0 0x7f0c581ee9b0 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c580738a0 tx=0x7f0c44000ed0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.139 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 186, 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:17:19.140 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:17:19.141 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.142 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:17:19.143 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:19.144 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:17:19.146 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:17:19.148 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:19.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.150 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:19.151 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:19.151 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:19.151 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.151 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.151 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:17:19.151 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:17:19.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.134+0000 7f0c5e600640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c3806b4d0 msgr2=0x7f0c3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.134+0000 7f0c5e600640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c3806b4d0 0x7f0c3806d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0c580bfe10 tx=0x7f0c5000b040 comp rx=0 tx=0).stop 2024-03-09T09:17:19.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.134+0000 7f0c5e600640 1 -- 172.21.15.67:0/3621593696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c580bed00 msgr2=0x7f0c580bf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.134+0000 7f0c5e600640 1 --2- 172.21.15.67:0/3621593696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c580bed00 0x7f0c580bf1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0c48000c00 tx=0x7f0c480046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.134+0000 7f0c57e00640 1 -- 172.21.15.67:0/3621593696 reap_dead start 2024-03-09T09:17:19.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.134+0000 7f0c57e00640 1 -- 172.21.15.67:0/3621593696 reap_dead start 2024-03-09T09:17:19.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.134+0000 7f0c5e600640 1 -- 172.21.15.67:0/3621593696 shutdown_connections 2024-03-09T09:17:19.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.134+0000 7f0c5e600640 1 -- 172.21.15.67:0/3621593696 >> 172.21.15.67:0/3621593696 conn(0x7f0c580cb670 msgr2=0x7f0c580ca1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:19.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.134+0000 7f0c5e600640 1 -- 172.21.15.67:0/3621593696 shutdown_connections 2024-03-09T09:17:19.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.134+0000 7f0c5e600640 1 -- 172.21.15.67:0/3621593696 wait complete. 2024-03-09T09:17:19.162 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:17:19.162 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:17:19.162 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:17:19.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.367+0000 7fc695800640 1 Processor -- start 2024-03-09T09:17:19.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.367+0000 7fc695800640 1 -- start start 2024-03-09T09:17:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.368+0000 7fc695800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690153590 0x7fc690159410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.368+0000 7fc695800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc6900c22a0 0x7fc6900c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.368+0000 7fc695800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69006f8d0 con 0x7fc69006e8d0 2024-03-09T09:17:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.368+0000 7fc695800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69006ed50 con 0x7fc6900c22a0 2024-03-09T09:17:19.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.368+0000 7fc695800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900c4220 con 0x7fc690153590 2024-03-09T09:17:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.369+0000 7fc68ea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690153590 0x7fc690159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.369+0000 7fc68ea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690153590 0x7fc690159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:52428/0 (socket says 172.21.15.67:52428) 2024-03-09T09:17:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.369+0000 7fc68ea00640 1 -- 172.21.15.67:0/4291245356 learned_addr learned my addr 172.21.15.67:0/4291245356 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:19.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.369+0000 7fc68f400640 1 --2- 172.21.15.67:0/4291245356 >> v2:172.21.15.134:3300/0 conn(0x7fc6900c22a0 0x7fc6900c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.370+0000 7fc68ea00640 1 -- 172.21.15.67:0/4291245356 >> v2:172.21.15.134:3300/0 conn(0x7fc6900c22a0 msgr2=0x7fc6900c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:19.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.370+0000 7fc68ea00640 1 --2- 172.21.15.67:0/4291245356 >> v2:172.21.15.134:3300/0 conn(0x7fc6900c22a0 0x7fc6900c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.370+0000 7fc68ea00640 1 -- 172.21.15.67:0/4291245356 >> v1:172.21.15.134:6789/0 conn(0x7fc69006e8d0 legacy=0x7fc69006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:19.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.370+0000 7fc68ea00640 1 -- 172.21.15.67:0/4291245356 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6900c4500 con 0x7fc690153590 2024-03-09T09:17:19.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.371+0000 7fc68ea00640 1 --2- 172.21.15.67:0/4291245356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690153590 0x7fc690159410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc680009870 tx=0x7fc68002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25945113c25d13fb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:19.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.371+0000 7fc68d600640 1 -- 172.21.15.67:0/4291245356 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680035020 con 0x7fc690153590 2024-03-09T09:17:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.372+0000 7fc68d600640 1 -- 172.21.15.67:0/4291245356 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc680031640 con 0x7fc690153590 2024-03-09T09:17:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.372+0000 7fc68d600640 1 -- 172.21.15.67:0/4291245356 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68002d040 con 0x7fc690153590 2024-03-09T09:17:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.372+0000 7fc695800640 1 -- 172.21.15.67:0/4291245356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690153590 msgr2=0x7fc690159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.372+0000 7fc695800640 1 --2- 172.21.15.67:0/4291245356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690153590 0x7fc690159410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc680009870 tx=0x7fc68002b8f0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.372+0000 7fc695800640 1 -- 172.21.15.67:0/4291245356 shutdown_connections 2024-03-09T09:17:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.372+0000 7fc695800640 1 --2- 172.21.15.67:0/4291245356 >> v2:172.21.15.134:3300/0 conn(0x7fc6900c22a0 0x7fc6900c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.372+0000 7fc695800640 1 --2- 172.21.15.67:0/4291245356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690153590 0x7fc690159410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.372+0000 7fc695800640 1 -- 172.21.15.67:0/4291245356 >> 172.21.15.67:0/4291245356 conn(0x7fc6900aa890 msgr2=0x7fc6900ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.372+0000 7fc695800640 1 -- 172.21.15.67:0/4291245356 shutdown_connections 2024-03-09T09:17:19.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.372+0000 7fc695800640 1 -- 172.21.15.67:0/4291245356 wait complete. 2024-03-09T09:17:19.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.374+0000 7fc695800640 1 Processor -- start 2024-03-09T09:17:19.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.374+0000 7fc695800640 1 -- start start 2024-03-09T09:17:19.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.375+0000 7fc695800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc69006e8d0 0x7fc69006b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.375+0000 7fc695800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6900c22a0 0x7fc6901eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.375+0000 7fc695800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc690153590 0x7fc6901ef200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.375+0000 7fc695800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69006b160 con 0x7fc690153590 2024-03-09T09:17:19.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.375+0000 7fc695800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900c4ac0 con 0x7fc6900c22a0 2024-03-09T09:17:19.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.375+0000 7fc695800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900c88b0 con 0x7fc69006e8d0 2024-03-09T09:17:19.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.375+0000 7fc68f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc690153590 0x7fc6901ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.376+0000 7fc68f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc690153590 0x7fc6901ef200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42062/0 (socket says 172.21.15.67:42062) 2024-03-09T09:17:19.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.376+0000 7fc68f400640 1 -- 172.21.15.67:0/2669003664 learned_addr learned my addr 172.21.15.67:0/2669003664 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:19.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.376+0000 7fc68ea00640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc69006e8d0 0x7fc69006b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.376+0000 7fc68e000640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6900c22a0 0x7fc6901eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.376+0000 7fc68f400640 1 -- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc69006e8d0 msgr2=0x7fc69006b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.376+0000 7fc68f400640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc69006e8d0 0x7fc69006b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.376+0000 7fc68f400640 1 -- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6900c22a0 msgr2=0x7fc6901eecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.376+0000 7fc68f400640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6900c22a0 0x7fc6901eecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.376+0000 7fc68f400640 1 -- 172.21.15.67:0/2669003664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc690149060 con 0x7fc690153590 2024-03-09T09:17:19.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.376+0000 7fc68f400640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc690153590 0x7fc6901ef200 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc688007e40 tx=0x7fc68800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:19.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.377+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688014620 con 0x7fc690153590 2024-03-09T09:17:19.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.377+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc688019070 con 0x7fc690153590 2024-03-09T09:17:19.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.377+0000 7fc695800640 1 -- 172.21.15.67:0/2669003664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc690148b80 con 0x7fc690153590 2024-03-09T09:17:19.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.377+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688014a20 con 0x7fc690153590 2024-03-09T09:17:19.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.377+0000 7fc695800640 1 -- 172.21.15.67:0/2669003664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6900c7730 con 0x7fc690153590 2024-03-09T09:17:19.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.378+0000 7fc695800640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6900ae580 0x7fc690147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.378+0000 7fc695800640 1 -- 172.21.15.67:0/2669003664 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6901489e0 con 0x7fc6900ae580 2024-03-09T09:17:19.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.378+0000 7fc68ea00640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6900ae580 0x7fc690147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.379+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc688014bc0 con 0x7fc690153590 2024-03-09T09:17:19.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.379+0000 7fc68ea00640 1 -- 172.21.15.67:0/2669003664 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc690073480 con 0x7fc6900ae580 2024-03-09T09:17:19.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.380+0000 7fc68ea00640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6900ae580 0x7fc690147940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc6900c8f60 tx=0x7fc680004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:19.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.380+0000 7fc686a00640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc67006b4d0 0x7fc67006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.380+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc68808ae80 con 0x7fc690153590 2024-03-09T09:17:19.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.380+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680035040 con 0x7fc6900ae580 2024-03-09T09:17:19.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.380+0000 7fc68e000640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc67006b4d0 0x7fc67006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.381+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc680044400 con 0x7fc6900ae580 2024-03-09T09:17:19.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.381+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6900ae580 msgr2=0x7fc690147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.381+0000 7fc686a00640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6900ae580 0x7fc690147940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc6900c8f60 tx=0x7fc680004730 comp rx=0 tx=0).stop 2024-03-09T09:17:19.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.381+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6900ae580 msgr2=0x7fc690147940 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.381+0000 7fc686a00640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6900ae580 0x7fc690147940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.382+0000 7fc68e000640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc67006b4d0 0x7fc67006d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc6900ae9e0 tx=0x7fc67c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:19.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.388+0000 7fc695800640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690147e80 0x7fc690148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.388+0000 7fc695800640 1 -- 172.21.15.67:0/2669003664 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6901487d0 con 0x7fc690147e80 2024-03-09T09:17:19.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.388+0000 7fc68ea00640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690147e80 0x7fc690148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.389+0000 7fc68ea00640 1 -- 172.21.15.67:0/2669003664 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc69006b3c0 con 0x7fc690147e80 2024-03-09T09:17:19.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.389+0000 7fc68ea00640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690147e80 0x7fc690148290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc680006b10 tx=0x7fc6800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:19.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.390+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680035040 con 0x7fc690147e80 2024-03-09T09:17:19.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.390+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7fc68004cb30 con 0x7fc690147e80 2024-03-09T09:17:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.390+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690147e80 msgr2=0x7fc690148290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.390+0000 7fc686a00640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690147e80 0x7fc690148290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc680006b10 tx=0x7fc6800046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.390+0000 7fc686a00640 1 -- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690147e80 msgr2=0x7fc690148290 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.390+0000 7fc686a00640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc690147e80 0x7fc690148290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.402 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:17:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:17:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:17:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:17:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 186, 2024-03-09T09:17:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:17:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:17:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:17:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:17:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:17:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:17:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:17:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:17:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:17:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:17:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:17:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:17:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:17:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:17:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:17:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:19.410 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:19.410 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:17:19.410 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:17:19.410 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:17:19.410 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:17:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:17:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:17:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:17:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:17:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:17:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:17:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:17:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:17:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:17:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:17:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:17:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:17:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:17:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:17:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:17:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:17:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:17:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:17:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:17:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:17:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:17:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:17:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:17:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:17:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:17:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:17:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:17:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:17:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:17:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:17:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:17:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:17:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:17:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:17:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:17:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:17:19.432 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:17:19.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.391+0000 7fc695800640 1 -- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc67006b4d0 msgr2=0x7fc67006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.391+0000 7fc695800640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc67006b4d0 0x7fc67006d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc6900ae9e0 tx=0x7fc67c009040 comp rx=0 tx=0).stop 2024-03-09T09:17:19.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.391+0000 7fc695800640 1 -- 172.21.15.67:0/2669003664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc690153590 msgr2=0x7fc6901ef200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.391+0000 7fc695800640 1 --2- 172.21.15.67:0/2669003664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc690153590 0x7fc6901ef200 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc688007e40 tx=0x7fc68800e4b0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.391+0000 7fc68f400640 1 -- 172.21.15.67:0/2669003664 reap_dead start 2024-03-09T09:17:19.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.391+0000 7fc68f400640 1 -- 172.21.15.67:0/2669003664 reap_dead start 2024-03-09T09:17:19.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.391+0000 7fc695800640 1 -- 172.21.15.67:0/2669003664 shutdown_connections 2024-03-09T09:17:19.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.391+0000 7fc695800640 1 -- 172.21.15.67:0/2669003664 >> 172.21.15.67:0/2669003664 conn(0x7fc6900aa890 msgr2=0x7fc69014c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:19.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.391+0000 7fc695800640 1 -- 172.21.15.67:0/2669003664 shutdown_connections 2024-03-09T09:17:19.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.391+0000 7fc695800640 1 -- 172.21.15.67:0/2669003664 wait complete. 2024-03-09T09:17:19.437 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:17:19.437 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:17:19.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:17:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.635+0000 7f9cf8c00640 1 Processor -- start 2024-03-09T09:17:19.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.635+0000 7f9cf8c00640 1 -- start start 2024-03-09T09:17:19.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.637+0000 7f9cf8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9cf4153300 0x7f9cf40b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.637+0000 7f9cf8c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf40cadb0 0x7f9cf40b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.637+0000 7f9cf8c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf4157000 con 0x7f9cf40c5e40 2024-03-09T09:17:19.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.637+0000 7f9cf8c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf40ae430 con 0x7f9cf4153300 2024-03-09T09:17:19.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.637+0000 7f9cf8c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40ae6f0 con 0x7f9cf40cadb0 2024-03-09T09:17:19.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.637+0000 7f9cf2a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9cf40c5e40 0x7f9cf4156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43210/0 (socket says 172.21.15.67:43210) 2024-03-09T09:17:19.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.637+0000 7f9cf3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9cf4153300 0x7f9cf40b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.637+0000 7f9cf2a00640 1 -- 172.21.15.67:0/2122978263 learned_addr learned my addr 172.21.15.67:0/2122978263 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:17:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.638+0000 7f9cf3e00640 1 --2- 172.21.15.67:0/2122978263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf40cadb0 0x7f9cf40b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.638+0000 7f9cf3400640 1 -- 172.21.15.67:0/2122978263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf40cadb0 msgr2=0x7f9cf40b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.638+0000 7f9cf3400640 1 --2- 172.21.15.67:0/2122978263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf40cadb0 0x7f9cf40b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.638+0000 7f9cf3400640 1 -- 172.21.15.67:0/2122978263 >> v1:172.21.15.67:6789/0 conn(0x7f9cf40c5e40 legacy=0x7f9cf4156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.638+0000 7f9cf2000640 1 -- 172.21.15.67:0/2122978263 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2680659929 0 0) 0x7f9cf4157000 con 0x7f9cf40c5e40 2024-03-09T09:17:19.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.638+0000 7f9cf3400640 1 -- 172.21.15.67:0/2122978263 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf4155b10 con 0x7f9cf4153300 2024-03-09T09:17:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.639+0000 7f9cf3400640 1 --2- 172.21.15.67:0/2122978263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf4153300 0x7f9cf40b0000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40097a0 tx=0x7f9ce402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e369f0ea72eb5eb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.639+0000 7f9cf2000640 1 -- 172.21.15.67:0/2122978263 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4035020 con 0x7f9cf4153300 2024-03-09T09:17:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.639+0000 7f9cf2000640 1 -- 172.21.15.67:0/2122978263 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce4031810 con 0x7f9cf4153300 2024-03-09T09:17:19.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.639+0000 7f9cf2000640 1 -- 172.21.15.67:0/2122978263 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4031be0 con 0x7f9cf4153300 2024-03-09T09:17:19.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.640+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2122978263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf4153300 msgr2=0x7f9cf40b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.640+0000 7f9cf8c00640 1 --2- 172.21.15.67:0/2122978263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf4153300 0x7f9cf40b0000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40097a0 tx=0x7f9ce402b8b0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.640+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2122978263 shutdown_connections 2024-03-09T09:17:19.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.640+0000 7f9cf8c00640 1 --2- 172.21.15.67:0/2122978263 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf40cadb0 0x7f9cf40b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.640+0000 7f9cf8c00640 1 --2- 172.21.15.67:0/2122978263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf4153300 0x7f9cf40b0000 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.640+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2122978263 >> 172.21.15.67:0/2122978263 conn(0x7f9cf40732d0 msgr2=0x7f9cf40ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:19.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.640+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2122978263 shutdown_connections 2024-03-09T09:17:19.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.640+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2122978263 wait complete. 2024-03-09T09:17:19.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.642+0000 7f9cf8c00640 1 Processor -- start 2024-03-09T09:17:19.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.642+0000 7f9cf8c00640 1 -- start start 2024-03-09T09:17:19.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf40c5e40 0x7f9cf40c0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf40cadb0 0x7f9cf40acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf8c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf4153300 0x7f9cf41ee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf8c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40c59e0 con 0x7f9cf40c5e40 2024-03-09T09:17:19.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf8c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf4159e90 con 0x7f9cf40cadb0 2024-03-09T09:17:19.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf8c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf4156720 con 0x7f9cf4153300 2024-03-09T09:17:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf40c5e40 0x7f9cf40c0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf40c5e40 0x7f9cf40c0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42084/0 (socket says 172.21.15.67:42084) 2024-03-09T09:17:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf3400640 1 -- 172.21.15.67:0/2395536157 learned_addr learned my addr 172.21.15.67:0/2395536157 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf3400640 1 -- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf4153300 msgr2=0x7f9cf41ee7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf3400640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf4153300 0x7f9cf41ee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf3400640 1 -- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf40cadb0 msgr2=0x7f9cf40acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.643+0000 7f9cf3400640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf40cadb0 0x7f9cf40acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.644+0000 7f9cf3400640 1 -- 172.21.15.67:0/2395536157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf41ebc30 con 0x7f9cf40c5e40 2024-03-09T09:17:19.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.644+0000 7f9cf3400640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf40c5e40 0x7f9cf40c0610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4034040 tx=0x7f9ce40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:19.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.644+0000 7f9ceb400640 1 -- 172.21.15.67:0/2395536157 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4041070 con 0x7f9cf40c5e40 2024-03-09T09:17:19.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.644+0000 7f9ceb400640 1 -- 172.21.15.67:0/2395536157 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce403ac80 con 0x7f9cf40c5e40 2024-03-09T09:17:19.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.644+0000 7f9ceb400640 1 -- 172.21.15.67:0/2395536157 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4035040 con 0x7f9cf40c5e40 2024-03-09T09:17:19.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.644+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2395536157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf41ebf10 con 0x7f9cf40c5e40 2024-03-09T09:17:19.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.644+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2395536157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf40ac340 con 0x7f9cf40c5e40 2024-03-09T09:17:19.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.645+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2395536157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf404f3c0 con 0x7f9cf40c5e40 2024-03-09T09:17:19.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.646+0000 7f9ceb400640 1 -- 172.21.15.67:0/2395536157 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9ce4038020 con 0x7f9cf40c5e40 2024-03-09T09:17:19.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.647+0000 7f9ceb400640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cd406b4d0 0x7f9cd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:19.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.647+0000 7f9ceb400640 1 -- 172.21.15.67:0/2395536157 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9ce40ac240 con 0x7f9cf40c5e40 2024-03-09T09:17:19.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.648+0000 7f9cf2a00640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cd406b4d0 0x7f9cd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:19.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.649+0000 7f9cf2a00640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cd406b4d0 0x7f9cd406d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0007fd0 tx=0x7f9ce0007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:19.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.652+0000 7f9ceb400640 1 -- 172.21.15.67:0/2395536157 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9ce407acc0 con 0x7f9cf40c5e40 2024-03-09T09:17:19.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.971+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2395536157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f9cf40cab90 con 0x7f9cf40c5e40 2024-03-09T09:17:19.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.972+0000 7f9ceb400640 1 -- 172.21.15.67:0/2395536157 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f9ce404de80 con 0x7f9cf40c5e40 2024-03-09T09:17:19.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cd406b4d0 msgr2=0x7f9cd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cd406b4d0 0x7f9cd406d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0007fd0 tx=0x7f9ce0007920 comp rx=0 tx=0).stop 2024-03-09T09:17:19.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2395536157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf40c5e40 msgr2=0x7f9cf40c0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:19.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf40c5e40 0x7f9cf40c0610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4034040 tx=0x7f9ce40046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2395536157 shutdown_connections 2024-03-09T09:17:19.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9cd406b4d0 0x7f9cd406d990 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cf4153300 0x7f9cf41ee7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cf40cadb0 0x7f9cf40acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 --2- 172.21.15.67:0/2395536157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cf40c5e40 0x7f9cf40c0610 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:19.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2395536157 >> 172.21.15.67:0/2395536157 conn(0x7f9cf40732d0 msgr2=0x7f9cf4150500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:19.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2395536157 shutdown_connections 2024-03-09T09:17:19.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:19.976+0000 7f9cf8c00640 1 -- 172.21.15.67:0/2395536157 wait complete. 2024-03-09T09:17:20.001 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:17:20.607 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:20.607 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:20.607 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 8 -eq 48 ']' 2024-03-09T09:17:20.608 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.8 2024-03-09T09:17:20.608 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.8 2024-03-09T09:17:20.608 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.8 2024-03-09T09:17:20.608 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.8 2024-03-09T09:17:20.610 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:20.784 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:17:20.884 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:17:20.884 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 3 -eq 48 ']' 2024-03-09T09:17:20.884 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.3 2024-03-09T09:17:20.884 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.3 2024-03-09T09:17:20.885 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.3 2024-03-09T09:17:20.885 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.3 2024-03-09T09:17:20.885 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:17:22.843 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:22.844 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:22.844 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 9 -eq 48 ']' 2024-03-09T09:17:22.844 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.9 2024-03-09T09:17:22.844 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.9 2024-03-09T09:17:22.844 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.9 2024-03-09T09:17:22.845 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.9 2024-03-09T09:17:22.851 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:25.104 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:17:25.104 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:17:25.104 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 4 -eq 48 ']' 2024-03-09T09:17:25.104 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.4 2024-03-09T09:17:25.104 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.4 2024-03-09T09:17:25.104 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.4 2024-03-09T09:17:25.104 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.4 2024-03-09T09:17:25.106 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:17:26.305 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:26.306 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:26.306 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 10 -eq 48 ']' 2024-03-09T09:17:26.306 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.10 2024-03-09T09:17:26.306 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.10 2024-03-09T09:17:26.306 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.10 2024-03-09T09:17:26.306 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.10 2024-03-09T09:17:26.309 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:27.293 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:17:27.293 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:17:27.293 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 5 -eq 48 ']' 2024-03-09T09:17:27.293 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.5 2024-03-09T09:17:27.294 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.5 2024-03-09T09:17:27.294 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.5 2024-03-09T09:17:27.294 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.5 2024-03-09T09:17:27.298 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:17:30.002 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:17:30.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:17:30.103 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:30.103 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:30.104 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 11 -eq 48 ']' 2024-03-09T09:17:30.104 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.11 2024-03-09T09:17:30.105 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.11 2024-03-09T09:17:30.105 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.11 2024-03-09T09:17:30.105 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.11 2024-03-09T09:17:30.105 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.261+0000 7fc0c3200640 1 Processor -- start 2024-03-09T09:17:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.262+0000 7fc0c3200640 1 -- start start 2024-03-09T09:17:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.264+0000 7fc0c3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc0bc06ffd0 0x7fc0bc157920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.264+0000 7fc0c3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0bc153ff0 con 0x7fc0bc0c4330 2024-03-09T09:17:30.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.264+0000 7fc0bb400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc0bc0c4330 0x7fc0bc157210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43932/0 (socket says 172.21.15.67:43932) 2024-03-09T09:17:30.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.264+0000 7fc0bb400640 1 -- 172.21.15.67:0/574264899 learned_addr learned my addr 172.21.15.67:0/574264899 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:17:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.264+0000 7fc0bbe00640 1 --1- 172.21.15.67:0/574264899 >> v1:172.21.15.134:6789/0 conn(0x7fc0bc151510 0x7fc0bc0729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:51310/0 (socket says 172.21.15.67:51310) 2024-03-09T09:17:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.264+0000 7fc0c3200640 1 -- 172.21.15.67:0/574264899 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0bc1542d0 con 0x7fc0bc151510 2024-03-09T09:17:30.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.264+0000 7fc0c3200640 1 -- 172.21.15.67:0/574264899 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0bc1545b0 con 0x7fc0bc06ffd0 2024-03-09T09:17:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.265+0000 7fc0c1200640 1 --2- 172.21.15.67:0/574264899 >> v2:172.21.15.67:3300/0 conn(0x7fc0bc06ffd0 0x7fc0bc157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.265+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652532545 0 0) 0x7fc0bc153ff0 con 0x7fc0bc0c4330 2024-03-09T09:17:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.265+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a0003520 con 0x7fc0bc0c4330 2024-03-09T09:17:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.265+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839580953 0 0) 0x7fc0bc1542d0 con 0x7fc0bc151510 2024-03-09T09:17:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.265+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0bc153ff0 con 0x7fc0bc151510 2024-03-09T09:17:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.265+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3113062340 0 0) 0x7fc0a0003520 con 0x7fc0bc0c4330 2024-03-09T09:17:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.265+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 >> v2:172.21.15.67:3300/0 conn(0x7fc0bc06ffd0 msgr2=0x7fc0bc157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.265+0000 7fc0baa00640 1 --2- 172.21.15.67:0/574264899 >> v2:172.21.15.67:3300/0 conn(0x7fc0bc06ffd0 0x7fc0bc157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.266+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 >> v1:172.21.15.134:6789/0 conn(0x7fc0bc151510 legacy=0x7fc0bc0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.266+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc154890 con 0x7fc0bc0c4330 2024-03-09T09:17:30.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.266+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fc0a80015a0 con 0x7fc0bc0c4330 2024-03-09T09:17:30.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.266+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 >> v1:172.21.15.67:6789/0 conn(0x7fc0bc0c4330 legacy=0x7fc0bc157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.266+0000 7fc0baa00640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0a0004bf0 0x7fc0a0004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.266+0000 7fc0baa00640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0a00037a0 0x7fc0a0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.266+0000 7fc0baa00640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0a0009a30 0x7fc0a00041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.266+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc1545b0 con 0x7fc0a0004bf0 2024-03-09T09:17:30.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.266+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0014940 con 0x7fc0a0009a30 2024-03-09T09:17:30.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.266+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0014b40 con 0x7fc0a00037a0 2024-03-09T09:17:30.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.266+0000 7fc0bb400640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0a00037a0 0x7fc0a0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.267+0000 7fc0c1200640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0a0009a30 0x7fc0a00041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.267+0000 7fc0bb400640 1 -- 172.21.15.67:0/574264899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0a0009a30 msgr2=0x7fc0a00041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:30.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.267+0000 7fc0bb400640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0a0009a30 0x7fc0a00041c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.267+0000 7fc0bb400640 1 -- 172.21.15.67:0/574264899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0a0004bf0 msgr2=0x7fc0a0004ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:17:30.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.267+0000 7fc0bbe00640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0a0004bf0 0x7fc0a0004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.267+0000 7fc0bb400640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0a0004bf0 0x7fc0a0004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.267+0000 7fc0c1200640 1 -- 172.21.15.67:0/574264899 reap_dead start 2024-03-09T09:17:30.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.267+0000 7fc0bb400640 1 -- 172.21.15.67:0/574264899 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a0003520 con 0x7fc0a00037a0 2024-03-09T09:17:30.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.267+0000 7fc0bbe00640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0a0004bf0 0x7fc0a0004ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:17:30.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.271+0000 7fc0bb400640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0a00037a0 0x7fc0a0003c20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8007fb0 tx=0x7fc0a0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fec3b7afcba7b22 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:30.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.271+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8007c00 con 0x7fc0a00037a0 2024-03-09T09:17:30.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.271+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a8030070 con 0x7fc0a00037a0 2024-03-09T09:17:30.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.271+0000 7fc0baa00640 1 -- 172.21.15.67:0/574264899 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8005d70 con 0x7fc0a00037a0 2024-03-09T09:17:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.272+0000 7fc0c3200640 1 -- 172.21.15.67:0/574264899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0a00037a0 msgr2=0x7fc0a0003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.272+0000 7fc0c3200640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0a00037a0 0x7fc0a0003c20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8007fb0 tx=0x7fc0a0005830 comp rx=0 tx=0).stop 2024-03-09T09:17:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.272+0000 7fc0c3200640 1 -- 172.21.15.67:0/574264899 shutdown_connections 2024-03-09T09:17:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.272+0000 7fc0c3200640 1 --2- 172.21.15.67:0/574264899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0a00037a0 0x7fc0a0003c20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.272+0000 7fc0c3200640 1 -- 172.21.15.67:0/574264899 >> 172.21.15.67:0/574264899 conn(0x7fc0bc0ab320 msgr2=0x7fc0bc0ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.272+0000 7fc0c3200640 1 -- 172.21.15.67:0/574264899 shutdown_connections 2024-03-09T09:17:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.273+0000 7fc0c3200640 1 -- 172.21.15.67:0/574264899 wait complete. 2024-03-09T09:17:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.275+0000 7fc0c3200640 1 Processor -- start 2024-03-09T09:17:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.275+0000 7fc0c3200640 1 -- start start 2024-03-09T09:17:30.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.276+0000 7fc0c3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0a00037a0 0x7fc0bc1eb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.276+0000 7fc0c3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0bc0ace70 0x7fc0bc0aa160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.276+0000 7fc0c3200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0bc0ad640 0x7fc0bc14a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.276+0000 7fc0c3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0c40f0 con 0x7fc0bc0ace70 2024-03-09T09:17:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.276+0000 7fc0c3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0c3f70 con 0x7fc0a00037a0 2024-03-09T09:17:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.276+0000 7fc0c3200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0729c0 con 0x7fc0bc0ad640 2024-03-09T09:17:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.277+0000 7fc0bbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0a00037a0 0x7fc0bc1eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.277+0000 7fc0c1200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0bc0ad640 0x7fc0bc14a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.277+0000 7fc0c1200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0bc0ad640 0x7fc0bc14a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:55948/0 (socket says 172.21.15.67:55948) 2024-03-09T09:17:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.277+0000 7fc0c1200640 1 -- 172.21.15.67:0/3699115052 learned_addr learned my addr 172.21.15.67:0/3699115052 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.277+0000 7fc0bb400640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0bc0ace70 0x7fc0bc0aa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.277+0000 7fc0c1200640 1 -- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0a00037a0 msgr2=0x7fc0bc1eb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.277+0000 7fc0c1200640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0a00037a0 0x7fc0bc1eb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.277+0000 7fc0c1200640 1 -- 172.21.15.67:0/3699115052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0bc0ace70 msgr2=0x7fc0bc0aa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.277+0000 7fc0c1200640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0bc0ace70 0x7fc0bc0aa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.277+0000 7fc0c1200640 1 -- 172.21.15.67:0/3699115052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc0ac410 con 0x7fc0bc0ad640 2024-03-09T09:17:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.278+0000 7fc0c1200640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0bc0ad640 0x7fc0bc14a2e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4007cf0 tx=0x7fc0b40115a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.278+0000 7fc0b3e00640 1 -- 172.21.15.67:0/3699115052 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4011de0 con 0x7fc0bc0ad640 2024-03-09T09:17:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.278+0000 7fc0b3e00640 1 -- 172.21.15.67:0/3699115052 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b401b070 con 0x7fc0bc0ad640 2024-03-09T09:17:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.278+0000 7fc0b3e00640 1 -- 172.21.15.67:0/3699115052 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4003990 con 0x7fc0bc0ad640 2024-03-09T09:17:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.278+0000 7fc0c3200640 1 -- 172.21.15.67:0/3699115052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0bc0ac6f0 con 0x7fc0bc0ad640 2024-03-09T09:17:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.278+0000 7fc0c3200640 1 -- 172.21.15.67:0/3699115052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc1ea860 con 0x7fc0bc0ad640 2024-03-09T09:17:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.280+0000 7fc0c3200640 1 -- 172.21.15.67:0/3699115052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0bc075a00 con 0x7fc0bc0ad640 2024-03-09T09:17:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.282+0000 7fc0b3e00640 1 -- 172.21.15.67:0/3699115052 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc0b4003b30 con 0x7fc0bc0ad640 2024-03-09T09:17:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.283+0000 7fc0b3e00640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc09c06b500 0x7fc09c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.283+0000 7fc0b3e00640 1 -- 172.21.15.67:0/3699115052 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc0b408acd0 con 0x7fc0bc0ad640 2024-03-09T09:17:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.283+0000 7fc0bbe00640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc09c06b500 0x7fc09c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.285+0000 7fc0bbe00640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc09c06b500 0x7fc09c06d9c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000f60 tx=0x7fc0ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:30.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.289+0000 7fc0b3e00640 1 -- 172.21.15.67:0/3699115052 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc0b4003e20 con 0x7fc0bc0ad640 2024-03-09T09:17:30.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.719+0000 7fc0c3200640 1 -- 172.21.15.67:0/3699115052 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc0bc070b10 con 0x7fc0bc0ad640 2024-03-09T09:17:30.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.720+0000 7fc0b3e00640 1 -- 172.21.15.67:0/3699115052 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fc0b4059750 con 0x7fc0bc0ad640 2024-03-09T09:17:30.725 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":186,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:17:30.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 -- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc09c06b500 msgr2=0x7fc09c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc09c06b500 0x7fc09c06d9c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac000f60 tx=0x7fc0ac009040 comp rx=0 tx=0).stop 2024-03-09T09:17:30.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 -- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0bc0ad640 msgr2=0x7fc0bc14a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0bc0ad640 0x7fc0bc14a2e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4007cf0 tx=0x7fc0b40115a0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 -- 172.21.15.67:0/3699115052 shutdown_connections 2024-03-09T09:17:30.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc09c06b500 0x7fc09c06d9c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0bc0ad640 0x7fc0bc14a2e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc0bc0ace70 0x7fc0bc0aa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 --2- 172.21.15.67:0/3699115052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0a00037a0 0x7fc0bc1eb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 -- 172.21.15.67:0/3699115052 >> 172.21.15.67:0/3699115052 conn(0x7fc0bc0ab320 msgr2=0x7fc0bc0c5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:30.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 -- 172.21.15.67:0/3699115052 shutdown_connections 2024-03-09T09:17:30.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.722+0000 7fc0c3200640 1 -- 172.21.15.67:0/3699115052 wait complete. 2024-03-09T09:17:30.746 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:17:30.746 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:17:30.746 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:17:30.746 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:17:30.910 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:17:30.910 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:17:30.910 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 6 -eq 48 ']' 2024-03-09T09:17:30.910 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.6 2024-03-09T09:17:30.911 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.6 2024-03-09T09:17:30.911 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.6 2024-03-09T09:17:30.911 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.6 2024-03-09T09:17:30.915 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:17:30.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.947+0000 7fb0a5600640 1 Processor -- start 2024-03-09T09:17:30.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.948+0000 7fb0a5600640 1 -- start start 2024-03-09T09:17:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.949+0000 7fb0a5600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0a00c9380 0x7fb0a00acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.949+0000 7fb0a5600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb0a00ad440 0x7fb0a006c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.949+0000 7fb0a5600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a00c9890 con 0x7fb0a014fd90 2024-03-09T09:17:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.949+0000 7fb0a5600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a006ba60 con 0x7fb0a00ad440 2024-03-09T09:17:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.949+0000 7fb0a5600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00c5f80 con 0x7fb0a00c9380 2024-03-09T09:17:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.949+0000 7fb09fe00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb0a014fd90 0x7fb0a0156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:51324/0 (socket says 172.21.15.67:51324) 2024-03-09T09:17:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.949+0000 7fb09fe00640 1 -- 172.21.15.67:0/3410413287 learned_addr learned my addr 172.21.15.67:0/3410413287 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:17:30.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.950+0000 7fb09f400640 1 --2- 172.21.15.67:0/3410413287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0a00c9380 0x7fb0a00acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.950+0000 7fb0a4c00640 1 --2- 172.21.15.67:0/3410413287 >> v2:172.21.15.134:3300/0 conn(0x7fb0a00ad440 0x7fb0a006c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.950+0000 7fb09f400640 1 -- 172.21.15.67:0/3410413287 >> v2:172.21.15.134:3300/0 conn(0x7fb0a00ad440 msgr2=0x7fb0a006c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:30.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.950+0000 7fb09f400640 1 --2- 172.21.15.67:0/3410413287 >> v2:172.21.15.134:3300/0 conn(0x7fb0a00ad440 0x7fb0a006c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.950+0000 7fb09f400640 1 -- 172.21.15.67:0/3410413287 >> v1:172.21.15.134:6789/0 conn(0x7fb0a014fd90 legacy=0x7fb0a0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.950+0000 7fb09f400640 1 -- 172.21.15.67:0/3410413287 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a00c6240 con 0x7fb0a00c9380 2024-03-09T09:17:30.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.952+0000 7fb09f400640 1 --2- 172.21.15.67:0/3410413287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0a00c9380 0x7fb0a00acf00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb0a006bbe0 tx=0x7fb08c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c365aa6f81c150d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:30.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.953+0000 7fb09ea00640 1 -- 172.21.15.67:0/3410413287 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c039070 con 0x7fb0a00c9380 2024-03-09T09:17:30.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.953+0000 7fb09ea00640 1 -- 172.21.15.67:0/3410413287 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c0316f0 con 0x7fb0a00c9380 2024-03-09T09:17:30.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.953+0000 7fb09ea00640 1 -- 172.21.15.67:0/3410413287 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c031ac0 con 0x7fb0a00c9380 2024-03-09T09:17:30.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.953+0000 7fb0a5600640 1 -- 172.21.15.67:0/3410413287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0a00c9380 msgr2=0x7fb0a00acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.953+0000 7fb0a5600640 1 --2- 172.21.15.67:0/3410413287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0a00c9380 0x7fb0a00acf00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb0a006bbe0 tx=0x7fb08c02b790 comp rx=0 tx=0).stop 2024-03-09T09:17:30.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.954+0000 7fb0a5600640 1 -- 172.21.15.67:0/3410413287 shutdown_connections 2024-03-09T09:17:30.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.954+0000 7fb0a5600640 1 --2- 172.21.15.67:0/3410413287 >> v2:172.21.15.134:3300/0 conn(0x7fb0a00ad440 0x7fb0a006c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.954+0000 7fb0a5600640 1 --2- 172.21.15.67:0/3410413287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0a00c9380 0x7fb0a00acf00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.954+0000 7fb0a5600640 1 -- 172.21.15.67:0/3410413287 >> 172.21.15.67:0/3410413287 conn(0x7fb0a00cb480 msgr2=0x7fb0a00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:30.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.954+0000 7fb0a5600640 1 -- 172.21.15.67:0/3410413287 shutdown_connections 2024-03-09T09:17:30.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.954+0000 7fb0a5600640 1 -- 172.21.15.67:0/3410413287 wait complete. 2024-03-09T09:17:30.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.956+0000 7fb0a5600640 1 Processor -- start 2024-03-09T09:17:30.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.956+0000 7fb0a5600640 1 -- start start 2024-03-09T09:17:30.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.958+0000 7fb0a5600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a00ad440 0x7fb0a0157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.958+0000 7fb0a5600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0a00c9380 0x7fb0a014d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.958+0000 7fb0a5600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014fd90 0x7fb0a01585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.958+0000 7fb0a5600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a006c710 con 0x7fb0a014fd90 2024-03-09T09:17:30.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.958+0000 7fb0a5600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a006bcb0 con 0x7fb0a00ad440 2024-03-09T09:17:30.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.958+0000 7fb0a5600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a006c890 con 0x7fb0a00c9380 2024-03-09T09:17:30.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.958+0000 7fb0a4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014fd90 0x7fb0a01585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.958+0000 7fb09fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a00ad440 0x7fb0a0157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.958+0000 7fb0a4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014fd90 0x7fb0a01585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54242/0 (socket says 172.21.15.67:54242) 2024-03-09T09:17:30.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.958+0000 7fb0a4c00640 1 -- 172.21.15.67:0/3262964051 learned_addr learned my addr 172.21.15.67:0/3262964051 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:30.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.958+0000 7fb09f400640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0a00c9380 0x7fb0a014d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.959+0000 7fb0a4c00640 1 -- 172.21.15.67:0/3262964051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0a00c9380 msgr2=0x7fb0a014d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.959+0000 7fb0a4c00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0a00c9380 0x7fb0a014d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.959+0000 7fb0a4c00640 1 -- 172.21.15.67:0/3262964051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a00ad440 msgr2=0x7fb0a0157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.959+0000 7fb0a4c00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0a00ad440 0x7fb0a0157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.959+0000 7fb0a4c00640 1 -- 172.21.15.67:0/3262964051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a015a060 con 0x7fb0a014fd90 2024-03-09T09:17:30.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.960+0000 7fb0a4c00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014fd90 0x7fb0a01585f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb098009dd0 tx=0x7fb09800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:30.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.960+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098018020 con 0x7fb0a014fd90 2024-03-09T09:17:30.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.960+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb098014480 con 0x7fb0a014fd90 2024-03-09T09:17:30.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.960+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098014870 con 0x7fb0a014fd90 2024-03-09T09:17:30.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.960+0000 7fb0a5600640 1 -- 172.21.15.67:0/3262964051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a0152e00 con 0x7fb0a014fd90 2024-03-09T09:17:30.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.960+0000 7fb0a5600640 1 -- 172.21.15.67:0/3262964051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a0157240 con 0x7fb0a014fd90 2024-03-09T09:17:30.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.961+0000 7fb0a5600640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a0150530 0x7fb0a014c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.961+0000 7fb0a5600640 1 -- 172.21.15.67:0/3262964051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a0157f80 con 0x7fb0a0150530 2024-03-09T09:17:30.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.962+0000 7fb09fe00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a0150530 0x7fb0a014c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.962+0000 7fb09fe00640 1 -- 172.21.15.67:0/3262964051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0a01532c0 con 0x7fb0a0150530 2024-03-09T09:17:30.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.963+0000 7fb09fe00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a0150530 0x7fb0a014c680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb090000f60 tx=0x7fb09000f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:30.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.963+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090005ac0 con 0x7fb0a0150530 2024-03-09T09:17:30.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.964+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb090018070 con 0x7fb0a0150530 2024-03-09T09:17:30.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.964+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a0150530 msgr2=0x7fb0a014c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.964+0000 7fb097e00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a0150530 0x7fb0a014c680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb090000f60 tx=0x7fb09000f9d0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.964+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a0150530 msgr2=0x7fb0a014c680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:30.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.964+0000 7fb097e00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a0150530 0x7fb0a014c680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.964+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb098005d10 con 0x7fb0a014fd90 2024-03-09T09:17:30.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.965+0000 7fb097e00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb08006af10 0x7fb08006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.965+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb09808b270 con 0x7fb0a014fd90 2024-03-09T09:17:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.970+0000 7fb09fe00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb08006af10 0x7fb08006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.971+0000 7fb09fe00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb08006af10 0x7fb08006d3d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb09000ff20 tx=0x7fb090007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.979+0000 7fb0a5600640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014cbc0 0x7fb0a01eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.979+0000 7fb0a5600640 1 -- 172.21.15.67:0/3262964051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a004f3c0 con 0x7fb0a014cbc0 2024-03-09T09:17:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.979+0000 7fb09f400640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014cbc0 0x7fb0a01eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.980+0000 7fb09f400640 1 -- 172.21.15.67:0/3262964051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb0a006d2d0 con 0x7fb0a014cbc0 2024-03-09T09:17:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.980+0000 7fb09f400640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014cbc0 0x7fb0a01eecf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00ada40 tx=0x7fb08c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.980+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c039040 con 0x7fb0a014cbc0 2024-03-09T09:17:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5020 (secure 0 0 0) 0x7fb08c031d90 con 0x7fb0a014cbc0 2024-03-09T09:17:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014cbc0 msgr2=0x7fb0a01eecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb097e00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014cbc0 0x7fb0a01eecf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00ada40 tx=0x7fb08c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb097e00640 1 -- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014cbc0 msgr2=0x7fb0a01eecf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb097e00640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014cbc0 0x7fb0a01eecf0 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00ada40 tx=0x7fb08c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:30.987 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:17:30.987 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:17:30.987 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:17:30.987 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 186, 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:30.988 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:30.989 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:17:30.990 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:17:30.991 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:17:30.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:30.993 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:17:30.994 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:30.995 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:30.996 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:30.997 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:17:30.998 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:17:30.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:17:31.000 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:17:31.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb0a5600640 1 -- 172.21.15.67:0/3262964051 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb08006af10 msgr2=0x7fb08006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb0a5600640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb08006af10 0x7fb08006d3d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb09000ff20 tx=0x7fb090007000 comp rx=0 tx=0).stop 2024-03-09T09:17:31.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb0a5600640 1 -- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014fd90 msgr2=0x7fb0a01585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb0a5600640 1 --2- 172.21.15.67:0/3262964051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0a014fd90 0x7fb0a01585f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb098009dd0 tx=0x7fb09800c490 comp rx=0 tx=0).stop 2024-03-09T09:17:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb0a4c00640 1 -- 172.21.15.67:0/3262964051 reap_dead start 2024-03-09T09:17:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb0a4c00640 1 -- 172.21.15.67:0/3262964051 reap_dead start 2024-03-09T09:17:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb0a5600640 1 -- 172.21.15.67:0/3262964051 shutdown_connections 2024-03-09T09:17:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.981+0000 7fb0a5600640 1 -- 172.21.15.67:0/3262964051 >> 172.21.15.67:0/3262964051 conn(0x7fb0a00cb480 msgr2=0x7fb0a00ca570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.982+0000 7fb0a5600640 1 -- 172.21.15.67:0/3262964051 shutdown_connections 2024-03-09T09:17:31.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:30.982+0000 7fb0a5600640 1 -- 172.21.15.67:0/3262964051 wait complete. 2024-03-09T09:17:31.003 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:17:31.003 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:17:31.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:17:31.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.186+0000 7f095b000640 1 Processor -- start 2024-03-09T09:17:31.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.186+0000 7f095b000640 1 -- start start 2024-03-09T09:17:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.187+0000 7f095b000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f09540c9380 0x7f09540acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.187+0000 7f095b000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f09540ad440 0x7f095406c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.187+0000 7f095b000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09540c9890 con 0x7f095414fd90 2024-03-09T09:17:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.187+0000 7f095b000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095406ba60 con 0x7f09540ad440 2024-03-09T09:17:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.187+0000 7f095b000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09540c5f80 con 0x7f09540c9380 2024-03-09T09:17:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.188+0000 7f095a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f09540ad440 0x7f095406c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.188+0000 7f095a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f09540ad440 0x7f095406c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54258/0 (socket says 172.21.15.67:54258) 2024-03-09T09:17:31.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.188+0000 7f095a600640 1 -- 172.21.15.67:0/2627808591 learned_addr learned my addr 172.21.15.67:0/2627808591 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.188+0000 7f0959200640 1 --2- 172.21.15.67:0/2627808591 >> v2:172.21.15.134:3300/0 conn(0x7f09540c9380 0x7f09540acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.188+0000 7f095a600640 1 -- 172.21.15.67:0/2627808591 >> v2:172.21.15.134:3300/0 conn(0x7f09540c9380 msgr2=0x7f09540acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.188+0000 7f095a600640 1 --2- 172.21.15.67:0/2627808591 >> v2:172.21.15.134:3300/0 conn(0x7f09540c9380 0x7f09540acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.188+0000 7f095a600640 1 -- 172.21.15.67:0/2627808591 >> v1:172.21.15.134:6789/0 conn(0x7f095414fd90 legacy=0x7f0954156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.188+0000 7f095a600640 1 -- 172.21.15.67:0/2627808591 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09540c6240 con 0x7f09540ad440 2024-03-09T09:17:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.188+0000 7f0959200640 1 --2- 172.21.15.67:0/2627808591 >> v2:172.21.15.134:3300/0 conn(0x7f09540c9380 0x7f09540acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:17:31.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.190+0000 7f095a600640 1 --2- 172.21.15.67:0/2627808591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09540ad440 0x7f095406c1d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f09540ae040 tx=0x7f094c02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f99c2b17116a22 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:31.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.190+0000 7f0953e00640 1 -- 172.21.15.67:0/2627808591 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c02c960 con 0x7f09540ad440 2024-03-09T09:17:31.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.190+0000 7f0953e00640 1 -- 172.21.15.67:0/2627808591 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c02cb00 con 0x7f09540ad440 2024-03-09T09:17:31.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.190+0000 7f0953e00640 1 -- 172.21.15.67:0/2627808591 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0343f0 con 0x7f09540ad440 2024-03-09T09:17:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.190+0000 7f095b000640 1 -- 172.21.15.67:0/2627808591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09540ad440 msgr2=0x7f095406c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.190+0000 7f095b000640 1 --2- 172.21.15.67:0/2627808591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09540ad440 0x7f095406c1d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f09540ae040 tx=0x7f094c02bdc0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.191+0000 7f095b000640 1 -- 172.21.15.67:0/2627808591 shutdown_connections 2024-03-09T09:17:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.191+0000 7f095b000640 1 --2- 172.21.15.67:0/2627808591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09540ad440 0x7f095406c1d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.191+0000 7f095b000640 1 --2- 172.21.15.67:0/2627808591 >> v2:172.21.15.134:3300/0 conn(0x7f09540c9380 0x7f09540acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.191+0000 7f095b000640 1 -- 172.21.15.67:0/2627808591 >> 172.21.15.67:0/2627808591 conn(0x7f09540cb480 msgr2=0x7f09540cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.191+0000 7f095b000640 1 -- 172.21.15.67:0/2627808591 shutdown_connections 2024-03-09T09:17:31.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.191+0000 7f095b000640 1 -- 172.21.15.67:0/2627808591 wait complete. 2024-03-09T09:17:31.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.192+0000 7f095b000640 1 Processor -- start 2024-03-09T09:17:31.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.192+0000 7f095b000640 1 -- start start 2024-03-09T09:17:31.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.193+0000 7f095b000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09540ad440 0x7f09541eed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.193+0000 7f095b000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09540c9380 0x7f0954151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.193+0000 7f095b000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f095414fd90 0x7f0954151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.193+0000 7f095b000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095406c710 con 0x7f09540ad440 2024-03-09T09:17:31.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.193+0000 7f095b000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095406bcb0 con 0x7f09540c9380 2024-03-09T09:17:31.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.193+0000 7f095b000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095406c890 con 0x7f095414fd90 2024-03-09T09:17:31.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.194+0000 7f0959c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09540ad440 0x7f09541eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.194+0000 7f0959c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09540ad440 0x7f09541eed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54262/0 (socket says 172.21.15.67:54262) 2024-03-09T09:17:31.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.194+0000 7f0959c00640 1 -- 172.21.15.67:0/1723603289 learned_addr learned my addr 172.21.15.67:0/1723603289 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.194+0000 7f0959200640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09540c9380 0x7f0954151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.194+0000 7f0959c00640 1 -- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f095414fd90 msgr2=0x7f0954151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.194+0000 7f0959c00640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f095414fd90 0x7f0954151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.194+0000 7f0959c00640 1 -- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09540c9380 msgr2=0x7f0954151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.194+0000 7f0959c00640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09540c9380 0x7f0954151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.194+0000 7f0959c00640 1 -- 172.21.15.67:0/1723603289 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09540b0f00 con 0x7f09540ad440 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.194+0000 7f0959c00640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09540ad440 0x7f09541eed80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f094800b3e0 tx=0x7f094800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.195+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948015020 con 0x7f09540ad440 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.195+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0948011850 con 0x7f09540ad440 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.195+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948011c40 con 0x7f09540ad440 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.195+0000 7f095b000640 1 -- 172.21.15.67:0/1723603289 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0954152c30 con 0x7f09540ad440 2024-03-09T09:17:31.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.195+0000 7f095b000640 1 -- 172.21.15.67:0/1723603289 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09541ef5c0 con 0x7f09540ad440 2024-03-09T09:17:31.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.196+0000 7f095b000640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09541530f0 0x7f095414c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.196+0000 7f095b000640 1 -- 172.21.15.67:0/1723603289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0954150b80 con 0x7f09541530f0 2024-03-09T09:17:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.197+0000 7f0959200640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09541530f0 0x7f095414c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.197+0000 7f0959200640 1 -- 172.21.15.67:0/1723603289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f095404f3c0 con 0x7f09541530f0 2024-03-09T09:17:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.198+0000 7f0959200640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09541530f0 0x7f095414c480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0954152800 tx=0x7f094000dbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.198+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0948018020 con 0x7f09540ad440 2024-03-09T09:17:31.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.199+0000 7f0951600640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f093406b4d0 0x7f093406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.200+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f094808a4d0 con 0x7f09540ad440 2024-03-09T09:17:31.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.200+0000 7f095a600640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f093406b4d0 0x7f093406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.200+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940016070 con 0x7f09541530f0 2024-03-09T09:17:31.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.200+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f09400088d0 con 0x7f09541530f0 2024-03-09T09:17:31.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.200+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09541530f0 msgr2=0x7f095414c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.200+0000 7f0951600640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09541530f0 0x7f095414c480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0954152800 tx=0x7f094000dbf0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.200+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09541530f0 msgr2=0x7f095414c480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:31.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.200+0000 7f0951600640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09541530f0 0x7f095414c480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.201+0000 7f095a600640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f093406b4d0 0x7f093406d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f094c035000 tx=0x7f094c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:31.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.214+0000 7f095b000640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f095414c9c0 0x7f09541eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.214+0000 7f095b000640 1 -- 172.21.15.67:0/1723603289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095414e240 con 0x7f095414c9c0 2024-03-09T09:17:31.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.215+0000 7f0959200640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f095414c9c0 0x7f09541eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.216+0000 7f0959200640 1 -- 172.21.15.67:0/1723603289 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f095406c4c0 con 0x7f095414c9c0 2024-03-09T09:17:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.217+0000 7f0959200640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f095414c9c0 0x7f09541eb5a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f09541534d0 tx=0x7f09400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:31.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.217+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094001cda0 con 0x7f095414c9c0 2024-03-09T09:17:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.218+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f094001cda0 con 0x7f095414c9c0 2024-03-09T09:17:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.218+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f095414c9c0 msgr2=0x7f09541eb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.218+0000 7f0951600640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f095414c9c0 0x7f09541eb5a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f09541534d0 tx=0x7f09400046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.218+0000 7f0951600640 1 -- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f095414c9c0 msgr2=0x7f09541eb5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:31.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.218+0000 7f0951600640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f095414c9c0 0x7f09541eb5a0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f09541534d0 tx=0x7f09400046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.224 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 186, 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:17:31.225 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:17:31.226 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:17:31.227 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:17:31.228 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:31.229 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:17:31.230 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:31.231 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.232 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.233 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.234 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:31.235 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:17:31.236 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:17:31.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.218+0000 7f095b000640 1 -- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f093406b4d0 msgr2=0x7f093406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.218+0000 7f095b000640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f093406b4d0 0x7f093406d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f094c035000 tx=0x7f094c02e000 comp rx=0 tx=0).stop 2024-03-09T09:17:31.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.218+0000 7f095b000640 1 -- 172.21.15.67:0/1723603289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09540ad440 msgr2=0x7f09541eed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.218+0000 7f095b000640 1 --2- 172.21.15.67:0/1723603289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09540ad440 0x7f09541eed80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f094800b3e0 tx=0x7f094800b8a0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.219+0000 7f095a600640 1 -- 172.21.15.67:0/1723603289 reap_dead start 2024-03-09T09:17:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.219+0000 7f095a600640 1 -- 172.21.15.67:0/1723603289 reap_dead start 2024-03-09T09:17:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.219+0000 7f095b000640 1 -- 172.21.15.67:0/1723603289 shutdown_connections 2024-03-09T09:17:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.219+0000 7f095b000640 1 -- 172.21.15.67:0/1723603289 >> 172.21.15.67:0/1723603289 conn(0x7f09540cb480 msgr2=0x7f09540c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.219+0000 7f095b000640 1 -- 172.21.15.67:0/1723603289 shutdown_connections 2024-03-09T09:17:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.219+0000 7f095b000640 1 -- 172.21.15.67:0/1723603289 wait complete. 2024-03-09T09:17:31.244 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:17:31.244 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:17:31.245 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:17:31.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.457+0000 7f4b8a600640 1 Processor -- start 2024-03-09T09:17:31.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.457+0000 7f4b8a600640 1 -- start start 2024-03-09T09:17:31.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.458+0000 7f4b8a600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4b840c9530 0x7f4b8406b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.458+0000 7f4b8a600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840bec10 0x7f4b8406b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.458+0000 7f4b8a600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b84156b30 con 0x7f4b840b0b40 2024-03-09T09:17:31.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.458+0000 7f4b8a600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b840c80c0 con 0x7f4b840c9530 2024-03-09T09:17:31.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.458+0000 7f4b8a600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840c83a0 con 0x7f4b840bec10 2024-03-09T09:17:31.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.459+0000 7f4b89c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840bec10 0x7f4b8406b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.459+0000 7f4b83e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4b840c9530 0x7f4b8406b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.459+0000 7f4b89c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840bec10 0x7f4b8406b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:55992/0 (socket says 172.21.15.67:55992) 2024-03-09T09:17:31.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.459+0000 7f4b89c00640 1 -- 172.21.15.67:0/3330224035 learned_addr learned my addr 172.21.15.67:0/3330224035 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:31.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.460+0000 7f4b83e00640 1 -- 172.21.15.67:0/3330224035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840bec10 msgr2=0x7f4b8406b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:31.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.460+0000 7f4b83e00640 1 --2- 172.21.15.67:0/3330224035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840bec10 0x7f4b8406b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.460+0000 7f4b83e00640 1 -- 172.21.15.67:0/3330224035 >> v1:172.21.15.134:6789/0 conn(0x7f4b840b0b40 legacy=0x7f4b840c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.460+0000 7f4b83e00640 1 -- 172.21.15.67:0/3330224035 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840c8680 con 0x7f4b840c9530 2024-03-09T09:17:31.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.461+0000 7f4b83e00640 1 --2- 172.21.15.67:0/3330224035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b840c9530 0x7f4b8406b400 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4b700097a0 tx=0x7f4b7002b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c137bda161c1cb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:31.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.462+0000 7f4b83400640 1 -- 172.21.15.67:0/3330224035 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70038070 con 0x7f4b840c9530 2024-03-09T09:17:31.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.462+0000 7f4b83400640 1 -- 172.21.15.67:0/3330224035 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b70031700 con 0x7f4b840c9530 2024-03-09T09:17:31.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.462+0000 7f4b83400640 1 -- 172.21.15.67:0/3330224035 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70031ad0 con 0x7f4b840c9530 2024-03-09T09:17:31.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.462+0000 7f4b8a600640 1 -- 172.21.15.67:0/3330224035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b840c9530 msgr2=0x7f4b8406b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.462+0000 7f4b8a600640 1 --2- 172.21.15.67:0/3330224035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b840c9530 0x7f4b8406b400 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4b700097a0 tx=0x7f4b7002b7a0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.462+0000 7f4b8a600640 1 -- 172.21.15.67:0/3330224035 shutdown_connections 2024-03-09T09:17:31.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.462+0000 7f4b8a600640 1 --2- 172.21.15.67:0/3330224035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840bec10 0x7f4b8406b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.462+0000 7f4b8a600640 1 --2- 172.21.15.67:0/3330224035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b840c9530 0x7f4b8406b400 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.462+0000 7f4b8a600640 1 -- 172.21.15.67:0/3330224035 >> 172.21.15.67:0/3330224035 conn(0x7f4b840ad0d0 msgr2=0x7f4b840ad4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:31.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.463+0000 7f4b8a600640 1 -- 172.21.15.67:0/3330224035 shutdown_connections 2024-03-09T09:17:31.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.463+0000 7f4b8a600640 1 -- 172.21.15.67:0/3330224035 wait complete. 2024-03-09T09:17:31.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.464+0000 7f4b8a600640 1 Processor -- start 2024-03-09T09:17:31.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.464+0000 7f4b8a600640 1 -- start start 2024-03-09T09:17:31.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.465+0000 7f4b8a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b840b0b40 0x7f4b8414d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.465+0000 7f4b8a600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840bec10 0x7f4b840c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.465+0000 7f4b8a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840c9530 0x7f4b840c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.465+0000 7f4b8a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840c6790 con 0x7f4b840c9530 2024-03-09T09:17:31.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.465+0000 7f4b8a600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840c6610 con 0x7f4b840b0b40 2024-03-09T09:17:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.465+0000 7f4b8a600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8406be80 con 0x7f4b840bec10 2024-03-09T09:17:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.466+0000 7f4b89c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840c9530 0x7f4b840c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.466+0000 7f4b89c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840c9530 0x7f4b840c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54274/0 (socket says 172.21.15.67:54274) 2024-03-09T09:17:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.466+0000 7f4b89c00640 1 -- 172.21.15.67:0/3561776345 learned_addr learned my addr 172.21.15.67:0/3561776345 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.466+0000 7f4b89200640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b840b0b40 0x7f4b8414d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.466+0000 7f4b83e00640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840bec10 0x7f4b840c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.466+0000 7f4b89c00640 1 -- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840bec10 msgr2=0x7f4b840c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.466+0000 7f4b89c00640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840bec10 0x7f4b840c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.466+0000 7f4b89c00640 1 -- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b840b0b40 msgr2=0x7f4b8414d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.466+0000 7f4b89c00640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b840b0b40 0x7f4b8414d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.466+0000 7f4b89c00640 1 -- 172.21.15.67:0/3561776345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840c3b40 con 0x7f4b840c9530 2024-03-09T09:17:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.467+0000 7f4b89c00640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840c9530 0x7f4b840c46e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009d90 tx=0x7f4b7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.467+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00cd20 con 0x7f4b840c9530 2024-03-09T09:17:31.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.467+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c003990 con 0x7f4b840c9530 2024-03-09T09:17:31.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.467+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c003d60 con 0x7f4b840c9530 2024-03-09T09:17:31.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.467+0000 7f4b8a600640 1 -- 172.21.15.67:0/3561776345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8414eeb0 con 0x7f4b840c9530 2024-03-09T09:17:31.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.467+0000 7f4b8a600640 1 -- 172.21.15.67:0/3561776345 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8414d740 con 0x7f4b840c9530 2024-03-09T09:17:31.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.468+0000 7f4b8a600640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840c51d0 0x7f4b841ea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.468+0000 7f4b8a600640 1 -- 172.21.15.67:0/3561776345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8404f3c0 con 0x7f4b840c51d0 2024-03-09T09:17:31.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.469+0000 7f4b89200640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840c51d0 0x7f4b841ea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.469+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4b7c012050 con 0x7f4b840c9530 2024-03-09T09:17:31.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.470+0000 7f4b89200640 1 -- 172.21.15.67:0/3561776345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b8406c8f0 con 0x7f4b840c51d0 2024-03-09T09:17:31.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.470+0000 7f4b89200640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840c51d0 0x7f4b841ea5a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b840c7ee0 tx=0x7f4b7800f9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:31.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.471+0000 7f4b80c00640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6406b4d0 0x7f4b6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.471+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4b7c08aa00 con 0x7f4b840c9530 2024-03-09T09:17:31.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.471+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780052e0 con 0x7f4b840c51d0 2024-03-09T09:17:31.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.471+0000 7f4b83e00640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6406b4d0 0x7f4b6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.471+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4b78018070 con 0x7f4b840c51d0 2024-03-09T09:17:31.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.472+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840c51d0 msgr2=0x7f4b841ea5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.472+0000 7f4b80c00640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840c51d0 0x7f4b841ea5a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b840c7ee0 tx=0x7f4b7800f9d0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.472+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840c51d0 msgr2=0x7f4b841ea5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:31.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.472+0000 7f4b80c00640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b840c51d0 0x7f4b841ea5a0 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b840c7ee0 tx=0x7f4b7800f9d0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.473+0000 7f4b83e00640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6406b4d0 0x7f4b6406d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4b70035000 tx=0x7f4b70008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:31.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.479+0000 7f4b8a600640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b841eaae0 0x7f4b841eaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:31.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.479+0000 7f4b8a600640 1 -- 172.21.15.67:0/3561776345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b841eb430 con 0x7f4b841eaae0 2024-03-09T09:17:31.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.480+0000 7f4b89200640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b841eaae0 0x7f4b841eaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:31.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.481+0000 7f4b89200640 1 -- 172.21.15.67:0/3561776345 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4b84073aa0 con 0x7f4b841eaae0 2024-03-09T09:17:31.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.481+0000 7f4b89200640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b841eaae0 0x7f4b841eaef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ff20 tx=0x7f4b78000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:31.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.482+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78007240 con 0x7f4b841eaae0 2024-03-09T09:17:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.483+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f4b78007240 con 0x7f4b841eaae0 2024-03-09T09:17:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.483+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b841eaae0 msgr2=0x7f4b841eaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.483+0000 7f4b80c00640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b841eaae0 0x7f4b841eaef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ff20 tx=0x7f4b78000ed0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.483+0000 7f4b80c00640 1 -- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b841eaae0 msgr2=0x7f4b841eaef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.483+0000 7f4b80c00640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b841eaae0 0x7f4b841eaef0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ff20 tx=0x7f4b78000ed0 comp rx=0 tx=0).stop 2024-03-09T09:17:31.489 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:17:31.489 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:17:31.489 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:17:31.489 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 186, 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:31.490 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:31.491 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:17:31.492 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:31.493 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:17:31.494 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.495 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:17:31.496 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:31.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.498 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:17:31.498 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:17:31.498 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:31.498 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:31.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.498 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:31.498 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:17:31.498 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.499 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:17:31.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:31.502 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:17:31.503 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:17:31.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.483+0000 7f4b8a600640 1 -- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6406b4d0 msgr2=0x7f4b6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.483+0000 7f4b8a600640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6406b4d0 0x7f4b6406d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4b70035000 tx=0x7f4b70008030 comp rx=0 tx=0).stop 2024-03-09T09:17:31.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.483+0000 7f4b8a600640 1 -- 172.21.15.67:0/3561776345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840c9530 msgr2=0x7f4b840c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:31.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.483+0000 7f4b8a600640 1 --2- 172.21.15.67:0/3561776345 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b840c9530 0x7f4b840c46e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009d90 tx=0x7f4b7c00c490 comp rx=0 tx=0).stop 2024-03-09T09:17:31.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.483+0000 7f4b89c00640 1 -- 172.21.15.67:0/3561776345 reap_dead start 2024-03-09T09:17:31.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.483+0000 7f4b89c00640 1 -- 172.21.15.67:0/3561776345 reap_dead start 2024-03-09T09:17:31.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.484+0000 7f4b8a600640 1 -- 172.21.15.67:0/3561776345 shutdown_connections 2024-03-09T09:17:31.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.484+0000 7f4b8a600640 1 -- 172.21.15.67:0/3561776345 >> 172.21.15.67:0/3561776345 conn(0x7f4b840ad0d0 msgr2=0x7f4b840c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:31.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.484+0000 7f4b8a600640 1 -- 172.21.15.67:0/3561776345 shutdown_connections 2024-03-09T09:17:31.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:31.484+0000 7f4b8a600640 1 -- 172.21.15.67:0/3561776345 wait complete. 2024-03-09T09:17:31.511 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:17:31.512 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T09:17:31.512 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a'] 2024-03-09T09:17:31.512 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T09:17:31.512 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:17:31.512 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:17:31.512 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:17:31.553 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:17:31.553 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-03-09T09:17:31.553 INFO:tasks.ceph.mon.a:Sent signal 19 2024-03-09T09:17:31.553 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-03-09T09:17:32.452 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:32.453 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:32.453 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 12 -eq 48 ']' 2024-03-09T09:17:32.453 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.12 2024-03-09T09:17:32.453 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.12 2024-03-09T09:17:32.453 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.12 2024-03-09T09:17:32.453 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.12 2024-03-09T09:17:32.456 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:33.682 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:17:33.683 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:17:33.683 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 7 -eq 48 ']' 2024-03-09T09:17:33.683 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.7 2024-03-09T09:17:33.683 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.7 2024-03-09T09:17:33.683 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.7 2024-03-09T09:17:33.683 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.7 2024-03-09T09:17:33.686 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:17:35.820 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:35.821 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:35.821 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 13 -eq 48 ']' 2024-03-09T09:17:35.821 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.13 2024-03-09T09:17:35.821 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.13 2024-03-09T09:17:35.822 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.13 2024-03-09T09:17:35.822 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.13 2024-03-09T09:17:35.824 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:37.050 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:17:37.050 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:17:37.051 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 8 -eq 48 ']' 2024-03-09T09:17:37.051 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.8 2024-03-09T09:17:37.051 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.8 2024-03-09T09:17:37.051 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.8 2024-03-09T09:17:37.051 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.8 2024-03-09T09:17:37.055 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:17:40.712 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:40.712 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:40.712 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 14 -eq 48 ']' 2024-03-09T09:17:40.712 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.14 2024-03-09T09:17:40.712 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.14 2024-03-09T09:17:40.713 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.14 2024-03-09T09:17:40.713 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.14 2024-03-09T09:17:40.716 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:42.264 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:17:42.265 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:17:42.265 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 9 -eq 48 ']' 2024-03-09T09:17:42.265 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.9 2024-03-09T09:17:42.266 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.9 2024-03-09T09:17:42.266 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.9 2024-03-09T09:17:42.266 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.9 2024-03-09T09:17:42.269 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:17:44.414 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:44.414 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:44.414 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 15 -eq 48 ']' 2024-03-09T09:17:44.415 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.15 2024-03-09T09:17:44.415 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.15 2024-03-09T09:17:44.415 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.15 2024-03-09T09:17:44.415 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.15 2024-03-09T09:17:44.418 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:46.554 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-03-09T09:17:46.555 INFO:tasks.ceph.mon.a:Sent signal 18 2024-03-09T09:17:46.555 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:17:46.555 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:17:46.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.756+0000 7f1b87e00640 1 Processor -- start 2024-03-09T09:17:46.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.757+0000 7f1b87e00640 1 -- start start 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.758+0000 7f1b87e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.758+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b800aa420 0x7f1b801490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.758+0000 7f1b87e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.758+0000 7f1b87e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b800c82d0 con 0x7f1b80149630 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.758+0000 7f1b87e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b800ab3c0 con 0x7f1b800c30f0 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.758+0000 7f1b87e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800aa8a0 con 0x7f1b800aa420 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.758+0000 7f1b84a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b800aa420 msgr2=0x7f1b801490f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.758+0000 7f1b84a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b800aa420 0x7f1b801490f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.758+0000 7f1b85e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.758+0000 7f1b85e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38860/0 (socket says 172.21.15.67:38860) 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.758+0000 7f1b85e00640 1 -- 172.21.15.67:0/438228268 learned_addr learned my addr 172.21.15.67:0/438228268 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.759+0000 7f1b85e00640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 msgr2=0x7f1b8006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:17:46.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.759+0000 7f1b85e00640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 msgr2=0x7f1b8006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:17:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.759+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.759+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:17:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.759+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.759+0000 7f1b85400640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 msgr2=0x7f1b80152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:17:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.759+0000 7f1b85400640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 msgr2=0x7f1b80152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:17:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.759+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:46.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.759+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:17:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.959+0000 7f1b84a00640 1 -- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b800aa420 msgr2=0x7f1b801490f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:17:46.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.959+0000 7f1b84a00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b800aa420 0x7f1b801490f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:17:46.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.960+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:46.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.960+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:46.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.960+0000 7f1b85e00640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 msgr2=0x7f1b8006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:17:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.960+0000 7f1b85e00640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 msgr2=0x7f1b8006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:17:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.960+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.960+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:17:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.961+0000 7f1b85400640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 msgr2=0x7f1b80152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:17:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.961+0000 7f1b85400640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 msgr2=0x7f1b80152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:17:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.961+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:46.961+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:17:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.360+0000 7f1b84a00640 1 -- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b800aa420 msgr2=0x7f1b801490f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:17:47.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.360+0000 7f1b84a00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b800aa420 0x7f1b801490f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:17:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.361+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.361+0000 7f1b85e00640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 msgr2=0x7f1b8006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:17:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.361+0000 7f1b85e00640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 msgr2=0x7f1b8006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:17:47.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.361+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:47.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.361+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:17:47.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.362+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:47.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.363+0000 7f1b85400640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 msgr2=0x7f1b80152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:17:47.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.363+0000 7f1b85400640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 msgr2=0x7f1b80152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:17:47.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.363+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:47.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:47.363+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:17:48.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.161+0000 7f1b84a00640 1 -- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b800aa420 msgr2=0x7f1b801490f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:17:48.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.161+0000 7f1b84a00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b800aa420 0x7f1b801490f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:17:48.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.163+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:48.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.163+0000 7f1b85e00640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 msgr2=0x7f1b8006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:17:48.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.163+0000 7f1b85e00640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 msgr2=0x7f1b8006dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:17:48.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.163+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:48.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.163+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:17:48.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.164+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:48.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.165+0000 7f1b85400640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 msgr2=0x7f1b80152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:17:48.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.165+0000 7f1b85400640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 msgr2=0x7f1b80152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:17:48.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.165+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:17:48.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:48.165+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:17:49.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:49.758+0000 7f1b7ea00640 1 -- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b800aa420 msgr2=0x7f1b801490f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:17:49.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:49.758+0000 7f1b7ea00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b800aa420 0x7f1b801490f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:49.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:49.758+0000 7f1b7ea00640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 msgr2=0x7f1b80152a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:17:49.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:49.758+0000 7f1b7ea00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.134:3300/0 conn(0x7f1b800c30f0 0x7f1b80152a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:49.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:49.758+0000 7f1b7ea00640 1 -- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 msgr2=0x7f1b8006dbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:17:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:49.758+0000 7f1b7ea00640 1 --2- 172.21.15.67:0/438228268 >> v2:172.21.15.67:3300/0 conn(0x7f1b80149630 0x7f1b8006dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:49.759+0000 7f1b7ea00640 1 -- 172.21.15.67:0/438228268 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b800aab20 con 0x7f1b600095d0 2024-03-09T09:17:49.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:49.759+0000 7f1b7ea00640 1 -- 172.21.15.67:0/438228268 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6000e350 con 0x7f1b60005880 2024-03-09T09:17:51.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.579+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461641204 0 0) 0x7f1b800aab20 con 0x7f1b600095d0 2024-03-09T09:17:51.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.579+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b68003520 con 0x7f1b600095d0 2024-03-09T09:17:51.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.581+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737957414 0 0) 0x7f1b6000e350 con 0x7f1b60005880 2024-03-09T09:17:51.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.581+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b800aab20 con 0x7f1b60005880 2024-03-09T09:17:51.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2456421697 0 0) 0x7f1b68003520 con 0x7f1b600095d0 2024-03-09T09:17:51.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 >> v1:172.21.15.134:6789/0 conn(0x7f1b60005880 legacy=0x7f1b60007d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:51.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6000f590 con 0x7f1b600095d0 2024-03-09T09:17:51.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f1b6c004330 con 0x7f1b600095d0 2024-03-09T09:17:51.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 >> v1:172.21.15.67:6789/0 conn(0x7f1b600095d0 legacy=0x7f1b6000ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:51.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b85e00640 1 -- 172.21.15.67:0/438228268 reap_dead start 2024-03-09T09:17:51.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b7fe00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b68003b70 0x7f1b68003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:51.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b7fe00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b68004800 0x7f1b68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:51.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b7fe00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b680052e0 0x7f1b6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:51.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6800c1d0 con 0x7f1b68004800 2024-03-09T09:17:51.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6800c380 con 0x7f1b68003b70 2024-03-09T09:17:51.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6800c580 con 0x7f1b680052e0 2024-03-09T09:17:51.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.585+0000 7f1b85e00640 1 -- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b680052e0 msgr2=0x7f1b6800b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:17:51.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.586+0000 7f1b85e00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b680052e0 0x7f1b6800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:17:51.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.586+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b68003b70 0x7f1b68003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:51.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.587+0000 7f1b85400640 1 -- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b680052e0 msgr2=0x7f1b6800b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:17:51.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.587+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b680052e0 0x7f1b6800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.587+0000 7f1b85400640 1 -- 172.21.15.67:0/438228268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b68004800 msgr2=0x7f1b68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:51.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.587+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b68004800 0x7f1b68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.587+0000 7f1b85400640 1 -- 172.21.15.67:0/438228268 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6000f590 con 0x7f1b68003b70 2024-03-09T09:17:51.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.588+0000 7f1b85400640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b68003b70 0x7f1b68003f50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b74006fd0 tx=0x7f1b68005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e666247fde3789d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:51.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.589+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b74011830 con 0x7f1b68003b70 2024-03-09T09:17:51.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.589+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b74035400 con 0x7f1b68003b70 2024-03-09T09:17:51.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.589+0000 7f1b7fe00640 1 -- 172.21.15.67:0/438228268 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b740355f0 con 0x7f1b68003b70 2024-03-09T09:17:51.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.590+0000 7f1b87e00640 1 -- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b68003b70 msgr2=0x7f1b68003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:51.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.590+0000 7f1b87e00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b68003b70 0x7f1b68003f50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b74006fd0 tx=0x7f1b68005830 comp rx=0 tx=0).stop 2024-03-09T09:17:51.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.590+0000 7f1b87e00640 1 -- 172.21.15.67:0/438228268 shutdown_connections 2024-03-09T09:17:51.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.590+0000 7f1b87e00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b680052e0 0x7f1b6800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.590+0000 7f1b87e00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b68004800 0x7f1b68004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.590+0000 7f1b87e00640 1 --2- 172.21.15.67:0/438228268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b68003b70 0x7f1b68003f50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.590+0000 7f1b87e00640 1 -- 172.21.15.67:0/438228268 >> 172.21.15.67:0/438228268 conn(0x7f1b800c7370 msgr2=0x7f1b800c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:51.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.590+0000 7f1b87e00640 1 -- 172.21.15.67:0/438228268 shutdown_connections 2024-03-09T09:17:51.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.590+0000 7f1b87e00640 1 -- 172.21.15.67:0/438228268 wait complete. 2024-03-09T09:17:51.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.594+0000 7f1b87e00640 1 Processor -- start 2024-03-09T09:17:51.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.594+0000 7f1b87e00640 1 -- start start 2024-03-09T09:17:51.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b68004800 0x7f1b8006dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:51.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b680052e0 0x7f1b8014b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:51.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b801462f0 0x7f1b801467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b87e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8006d6c0 con 0x7f1b68004800 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b87e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800ab0e0 con 0x7f1b680052e0 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b87e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800ab620 con 0x7f1b801462f0 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b85400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b68004800 0x7f1b8006dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b85400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b68004800 0x7f1b8006dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52384/0 (socket says 172.21.15.67:52384) 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b85e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b801462f0 msgr2=0x7f1b801467a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b85400640 1 -- 172.21.15.67:0/2417853399 learned_addr learned my addr 172.21.15.67:0/2417853399 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b85e00640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b801462f0 0x7f1b801467a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.596+0000 7f1b84a00640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b680052e0 0x7f1b8014b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.597+0000 7f1b85400640 1 -- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b801462f0 msgr2=0x7f1b801467a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.597+0000 7f1b85400640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b801462f0 0x7f1b801467a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.597+0000 7f1b85400640 1 -- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b680052e0 msgr2=0x7f1b8014b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.597+0000 7f1b85400640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b680052e0 0x7f1b8014b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.597+0000 7f1b85400640 1 -- 172.21.15.67:0/2417853399 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80146ef0 con 0x7f1b68004800 2024-03-09T09:17:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.597+0000 7f1b85400640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b68004800 0x7f1b8006dca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1b74037040 tx=0x7f1b740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.597+0000 7f1b7d600640 1 -- 172.21.15.67:0/2417853399 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b74045020 con 0x7f1b68004800 2024-03-09T09:17:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.598+0000 7f1b7d600640 1 -- 172.21.15.67:0/2417853399 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b74035e40 con 0x7f1b68004800 2024-03-09T09:17:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.598+0000 7f1b87e00640 1 -- 172.21.15.67:0/2417853399 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8014c1c0 con 0x7f1b68004800 2024-03-09T09:17:51.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.598+0000 7f1b7d600640 1 -- 172.21.15.67:0/2417853399 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b74011830 con 0x7f1b68004800 2024-03-09T09:17:51.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.598+0000 7f1b87e00640 1 -- 172.21.15.67:0/2417853399 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8014c780 con 0x7f1b68004800 2024-03-09T09:17:51.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.600+0000 7f1b87e00640 1 -- 172.21.15.67:0/2417853399 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b800ad500 con 0x7f1b68004800 2024-03-09T09:17:51.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.600+0000 7f1b7d600640 1 -- 172.21.15.67:0/2417853399 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1b74048020 con 0x7f1b68004800 2024-03-09T09:17:51.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.601+0000 7f1b7d600640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b6006c1b0 0x7f1b6006e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:51.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.601+0000 7f1b7d600640 1 -- 172.21.15.67:0/2417853399 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1b740acc00 con 0x7f1b68004800 2024-03-09T09:17:51.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.601+0000 7f1b84a00640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b6006c1b0 0x7f1b6006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:51.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.603+0000 7f1b84a00640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b6006c1b0 0x7f1b6006e670 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1b8006e550 tx=0x7f1b6c004170 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:51.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.606+0000 7f1b7d600640 1 -- 172.21.15.67:0/2417853399 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1b800ad500 con 0x7f1b68004800 2024-03-09T09:17:51.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.886+0000 7f1b87e00640 1 -- 172.21.15.67:0/2417853399 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1b80075a00 con 0x7f1b68004800 2024-03-09T09:17:51.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.886+0000 7f1b7d600640 1 -- 172.21.15.67:0/2417853399 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f1b7404f030 con 0x7f1b68004800 2024-03-09T09:17:51.893 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":190,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:17:51.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 -- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b6006c1b0 msgr2=0x7f1b6006e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:51.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b6006c1b0 0x7f1b6006e670 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1b8006e550 tx=0x7f1b6c004170 comp rx=0 tx=0).stop 2024-03-09T09:17:51.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 -- 172.21.15.67:0/2417853399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b68004800 msgr2=0x7f1b8006dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:51.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b68004800 0x7f1b8006dca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1b74037040 tx=0x7f1b740046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 -- 172.21.15.67:0/2417853399 shutdown_connections 2024-03-09T09:17:51.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1b6006c1b0 0x7f1b6006e670 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b801462f0 0x7f1b801467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b680052e0 0x7f1b8014b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 --2- 172.21.15.67:0/2417853399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1b68004800 0x7f1b8006dca0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:51.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 -- 172.21.15.67:0/2417853399 >> 172.21.15.67:0/2417853399 conn(0x7f1b800c7370 msgr2=0x7f1b80071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:51.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 -- 172.21.15.67:0/2417853399 shutdown_connections 2024-03-09T09:17:51.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:51.889+0000 7f1b87e00640 1 -- 172.21.15.67:0/2417853399 wait complete. 2024-03-09T09:17:51.913 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:17:51.913 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:17:51.913 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:17:52.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.107+0000 7fb008400640 1 Processor -- start 2024-03-09T09:17:52.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.107+0000 7fb008400640 1 -- start start 2024-03-09T09:17:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.108+0000 7fb008400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0000c1d70 0x7fb0000c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.108+0000 7fb008400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0000ac4e0 con 0x7fb00014c9a0 2024-03-09T09:17:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.108+0000 7fb008400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0000aa550 con 0x7fb0000c0ce0 2024-03-09T09:17:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.108+0000 7fb008400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00006d590 con 0x7fb0000c1d70 2024-03-09T09:17:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.108+0000 7fb005000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0000c1d70 0x7fb0000c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.109+0000 7fb005000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0000c1d70 0x7fb0000c07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52394/0 (socket says 172.21.15.67:52394) 2024-03-09T09:17:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.109+0000 7fb005000640 1 -- 172.21.15.67:0/2201045711 learned_addr learned my addr 172.21.15.67:0/2201045711 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:52.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.109+0000 7fb005a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb00014c9a0 0x7fb0001567c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47222/0 (socket says 172.21.15.67:47222) 2024-03-09T09:17:52.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.109+0000 7fb005000640 1 -- 172.21.15.67:0/2201045711 >> v1:172.21.15.134:6789/0 conn(0x7fb0000c0ce0 legacy=0x7fb0000ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:17:52.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.109+0000 7fb005000640 1 -- 172.21.15.67:0/2201045711 >> v1:172.21.15.67:6789/0 conn(0x7fb00014c9a0 legacy=0x7fb0001567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.109+0000 7fb005000640 1 -- 172.21.15.67:0/2201045711 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0000aa830 con 0x7fb0000c1d70 2024-03-09T09:17:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.111+0000 7fb005000640 1 --2- 172.21.15.67:0/2201045711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0000c1d70 0x7fb0000c07a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fafec009870 tx=0x7fafec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d51b34c88758e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.111+0000 7fafffe00640 1 -- 172.21.15.67:0/2201045711 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec035020 con 0x7fb0000c1d70 2024-03-09T09:17:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.111+0000 7fafffe00640 1 -- 172.21.15.67:0/2201045711 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec031870 con 0x7fb0000c1d70 2024-03-09T09:17:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.111+0000 7fafffe00640 1 -- 172.21.15.67:0/2201045711 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec031c40 con 0x7fb0000c1d70 2024-03-09T09:17:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.112+0000 7fb008400640 1 -- 172.21.15.67:0/2201045711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0000c1d70 msgr2=0x7fb0000c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.112+0000 7fb008400640 1 --2- 172.21.15.67:0/2201045711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0000c1d70 0x7fb0000c07a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fafec009870 tx=0x7fafec02b910 comp rx=0 tx=0).stop 2024-03-09T09:17:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.112+0000 7fb008400640 1 -- 172.21.15.67:0/2201045711 shutdown_connections 2024-03-09T09:17:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.112+0000 7fb008400640 1 --2- 172.21.15.67:0/2201045711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0000c1d70 0x7fb0000c07a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.112+0000 7fb008400640 1 -- 172.21.15.67:0/2201045711 >> 172.21.15.67:0/2201045711 conn(0x7fb0000c7370 msgr2=0x7fb0000c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.112+0000 7fb008400640 1 -- 172.21.15.67:0/2201045711 shutdown_connections 2024-03-09T09:17:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.112+0000 7fb008400640 1 -- 172.21.15.67:0/2201045711 wait complete. 2024-03-09T09:17:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.113+0000 7fb008400640 1 Processor -- start 2024-03-09T09:17:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.114+0000 7fb008400640 1 -- start start 2024-03-09T09:17:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.114+0000 7fb008400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0000c0ce0 0x7fb0001ef060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.114+0000 7fb008400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0000c1d70 0x7fb000071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.114+0000 7fb008400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb00014c9a0 0x7fb000071f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb008400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0000ad0d0 con 0x7fb0000c0ce0 2024-03-09T09:17:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb008400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0000aadf0 con 0x7fb00014c9a0 2024-03-09T09:17:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb008400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0000ad250 con 0x7fb0000c1d70 2024-03-09T09:17:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb005000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0000c1d70 msgr2=0x7fb000071a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:17:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb005000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0000c1d70 0x7fb000071a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:17:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb005a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0000c0ce0 0x7fb0001ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb005a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0000c0ce0 0x7fb0001ef060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52408/0 (socket says 172.21.15.67:52408) 2024-03-09T09:17:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb005a00640 1 -- 172.21.15.67:0/2394316163 learned_addr learned my addr 172.21.15.67:0/2394316163 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb006400640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb00014c9a0 0x7fb000071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb005a00640 1 -- 172.21.15.67:0/2394316163 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0000c1d70 msgr2=0x7fb000071a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:17:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb005a00640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0000c1d70 0x7fb000071a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb005a00640 1 -- 172.21.15.67:0/2394316163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb00014c9a0 msgr2=0x7fb000071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb005a00640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb00014c9a0 0x7fb000071f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.115+0000 7fb005a00640 1 -- 172.21.15.67:0/2394316163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0001efcf0 con 0x7fb0000c0ce0 2024-03-09T09:17:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.116+0000 7fb005a00640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0000c0ce0 0x7fb0001ef060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faff400c730 tx=0x7faff400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.116+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4017070 con 0x7fb0000c0ce0 2024-03-09T09:17:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.116+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff400ce40 con 0x7fb0000c0ce0 2024-03-09T09:17:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.116+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff400a860 con 0x7fb0000c0ce0 2024-03-09T09:17:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.116+0000 7fb008400640 1 -- 172.21.15.67:0/2394316163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb000145630 con 0x7fb0000c0ce0 2024-03-09T09:17:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.116+0000 7fb008400640 1 -- 172.21.15.67:0/2394316163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb000146250 con 0x7fb0000c0ce0 2024-03-09T09:17:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.117+0000 7fb008400640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb000157160 0x7fb000159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.117+0000 7fb008400640 1 -- 172.21.15.67:0/2394316163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb000075a00 con 0x7fb000157160 2024-03-09T09:17:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.117+0000 7fb005000640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb000157160 0x7fb000159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.118+0000 7fb005000640 1 -- 172.21.15.67:0/2394316163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0000c15e0 con 0x7fb000157160 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.118+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faff401a070 con 0x7fb0000c0ce0 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.118+0000 7fb005000640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb000157160 0x7fb000159600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0000c83f0 tx=0x7fafec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.119+0000 7faffd600640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafe006b480 0x7fafe006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.119+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7faff4089c20 con 0x7fb0000c0ce0 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.120+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec041070 con 0x7fb000157160 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.120+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fafec03ac70 con 0x7fb000157160 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.120+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb000157160 msgr2=0x7fb000159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.120+0000 7faffd600640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb000157160 0x7fb000159600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0000c83f0 tx=0x7fafec0046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.120+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb000157160 msgr2=0x7fb000159600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.120+0000 7faffd600640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb000157160 0x7fb000159600 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0000c83f0 tx=0x7fafec0046c0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.120+0000 7fb006400640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafe006b480 0x7fafe006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.121+0000 7fb006400640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafe006b480 0x7fafe006d940 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb0000738c0 tx=0x7faff8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:52.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.127+0000 7fb008400640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0001ef5a0 0x7fb000159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.127+0000 7fb008400640 1 -- 172.21.15.67:0/2394316163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0000c5380 con 0x7fb0001ef5a0 2024-03-09T09:17:52.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.127+0000 7fb005000640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0001ef5a0 0x7fb000159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.127+0000 7fb005000640 1 -- 172.21.15.67:0/2394316163 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb000157b90 con 0x7fb0001ef5a0 2024-03-09T09:17:52.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.128+0000 7fb005000640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0001ef5a0 0x7fb000159b40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fafec009870 tx=0x7fafec009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:52.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.128+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec035040 con 0x7fb0001ef5a0 2024-03-09T09:17:52.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.128+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7fafec04a2d0 con 0x7fb0001ef5a0 2024-03-09T09:17:52.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.128+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0001ef5a0 msgr2=0x7fb000159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.128+0000 7faffd600640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0001ef5a0 0x7fb000159b40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fafec009870 tx=0x7fafec009240 comp rx=0 tx=0).stop 2024-03-09T09:17:52.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.128+0000 7faffd600640 1 -- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0001ef5a0 msgr2=0x7fb000159b40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:52.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.128+0000 7faffd600640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0001ef5a0 0x7fb000159b40 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fafec009870 tx=0x7fafec009240 comp rx=0 tx=0).stop 2024-03-09T09:17:52.135 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:17:52.135 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:17:52.135 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:17:52.135 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 190, 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:52.136 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:52.137 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:17:52.138 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:17:52.139 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:17:52.139 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:52.139 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:52.139 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:17:52.139 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:17:52.139 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:17:52.139 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:17:52.139 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:17:52.139 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:17:52.139 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:17:52.139 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.140 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.142 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:52.142 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:17:52.142 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.142 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.142 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.143 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:17:52.143 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:17:52.143 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:52.143 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:52.143 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:52.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.143 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:17:52.143 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:17:52.143 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:52.143 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:17:52.144 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:17:52.145 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:52.145 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:52.145 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:52.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.145 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:17:52.145 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:17:52.145 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:52.146 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:52.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:52.146 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:17:52.146 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:52.146 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:17:52.146 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.147 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.147 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.147 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:17:52.147 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:17:52.147 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:52.147 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:52.147 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:52.147 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.147 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:17:52.148 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.149 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.149 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:17:52.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.149 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:52.149 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:52.149 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:17:52.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.149 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.149 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:17:52.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.151 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:52.151 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:52.151 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:52.151 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.151 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.151 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:17:52.151 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:17:52.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.129+0000 7fb008400640 1 -- 172.21.15.67:0/2394316163 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafe006b480 msgr2=0x7fafe006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.129+0000 7fb008400640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fafe006b480 0x7fafe006d940 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb0000738c0 tx=0x7faff8009040 comp rx=0 tx=0).stop 2024-03-09T09:17:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.129+0000 7fb008400640 1 -- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0000c0ce0 msgr2=0x7fb0001ef060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.129+0000 7fb008400640 1 --2- 172.21.15.67:0/2394316163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0000c0ce0 0x7fb0001ef060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faff400c730 tx=0x7faff400cbf0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.129+0000 7fb006400640 1 -- 172.21.15.67:0/2394316163 reap_dead start 2024-03-09T09:17:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.129+0000 7fb006400640 1 -- 172.21.15.67:0/2394316163 reap_dead start 2024-03-09T09:17:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.129+0000 7fb008400640 1 -- 172.21.15.67:0/2394316163 shutdown_connections 2024-03-09T09:17:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.129+0000 7fb008400640 1 -- 172.21.15.67:0/2394316163 >> 172.21.15.67:0/2394316163 conn(0x7fb0000c7370 msgr2=0x7fb0000ac9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.129+0000 7fb008400640 1 -- 172.21.15.67:0/2394316163 shutdown_connections 2024-03-09T09:17:52.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.129+0000 7fb008400640 1 -- 172.21.15.67:0/2394316163 wait complete. 2024-03-09T09:17:52.155 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:17:52.155 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:17:52.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:17:52.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.330+0000 7fd047e00640 1 Processor -- start 2024-03-09T09:17:52.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.330+0000 7fd047e00640 1 -- start start 2024-03-09T09:17:52.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.331+0000 7fd047e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0480ac930 0x7fd0480c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.331+0000 7fd047e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd048151360 0x7fd0481539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.331+0000 7fd047e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0480c2780 con 0x7fd0480c30a0 2024-03-09T09:17:52.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.331+0000 7fd047e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd048070670 con 0x7fd048151360 2024-03-09T09:17:52.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.331+0000 7fd047e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480708c0 con 0x7fd0480ac930 2024-03-09T09:17:52.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.332+0000 7fd046a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0480ac930 msgr2=0x7fd0480c3660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:17:52.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.332+0000 7fd046a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0480ac930 0x7fd0480c3660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:17:52.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.332+0000 7fd047400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd048151360 0x7fd0481539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.332+0000 7fd047400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd048151360 0x7fd0481539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52432/0 (socket says 172.21.15.67:52432) 2024-03-09T09:17:52.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.332+0000 7fd047400640 1 -- 172.21.15.67:0/1349805565 learned_addr learned my addr 172.21.15.67:0/1349805565 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:52.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.332+0000 7fd047400640 1 -- 172.21.15.67:0/1349805565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0480ac930 msgr2=0x7fd0480c3660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:17:52.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.332+0000 7fd047400640 1 --2- 172.21.15.67:0/1349805565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0480ac930 0x7fd0480c3660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.332+0000 7fd047400640 1 -- 172.21.15.67:0/1349805565 >> v1:172.21.15.134:6789/0 conn(0x7fd0480c30a0 legacy=0x7fd048150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.332+0000 7fd047400640 1 -- 172.21.15.67:0/1349805565 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd048070ba0 con 0x7fd048151360 2024-03-09T09:17:52.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.334+0000 7fd047400640 1 --2- 172.21.15.67:0/1349805565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd048151360 0x7fd0481539e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd040009e50 tx=0x7fd04002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda71c8575e9f287 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:52.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.334+0000 7fd045600640 1 -- 172.21.15.67:0/1349805565 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04002cb20 con 0x7fd048151360 2024-03-09T09:17:52.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.334+0000 7fd045600640 1 -- 172.21.15.67:0/1349805565 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04002ccc0 con 0x7fd048151360 2024-03-09T09:17:52.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.334+0000 7fd045600640 1 -- 172.21.15.67:0/1349805565 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040034610 con 0x7fd048151360 2024-03-09T09:17:52.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.335+0000 7fd047e00640 1 -- 172.21.15.67:0/1349805565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd048151360 msgr2=0x7fd0481539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.335+0000 7fd047e00640 1 --2- 172.21.15.67:0/1349805565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd048151360 0x7fd0481539e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd040009e50 tx=0x7fd04002bf80 comp rx=0 tx=0).stop 2024-03-09T09:17:52.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.335+0000 7fd047e00640 1 -- 172.21.15.67:0/1349805565 shutdown_connections 2024-03-09T09:17:52.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.335+0000 7fd047e00640 1 --2- 172.21.15.67:0/1349805565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd048151360 0x7fd0481539e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.335+0000 7fd047e00640 1 --2- 172.21.15.67:0/1349805565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0480ac930 0x7fd0480c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.335+0000 7fd047e00640 1 -- 172.21.15.67:0/1349805565 >> 172.21.15.67:0/1349805565 conn(0x7fd0480c7220 msgr2=0x7fd0480c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:52.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.335+0000 7fd047e00640 1 -- 172.21.15.67:0/1349805565 shutdown_connections 2024-03-09T09:17:52.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.335+0000 7fd047e00640 1 -- 172.21.15.67:0/1349805565 wait complete. 2024-03-09T09:17:52.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.337+0000 7fd047e00640 1 Processor -- start 2024-03-09T09:17:52.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.337+0000 7fd047e00640 1 -- start start 2024-03-09T09:17:52.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd047e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0480ac930 0x7fd04814f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd047e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0480c30a0 0x7fd048148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd047e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd048151360 0x7fd0480c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd047e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480ace90 con 0x7fd0480ac930 2024-03-09T09:17:52.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd047e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480acd10 con 0x7fd0480c30a0 2024-03-09T09:17:52.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd047e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd048153f20 con 0x7fd048151360 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd046a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0480ac930 0x7fd04814f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd047400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd048151360 msgr2=0x7fd0480c6670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd046a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0480ac930 0x7fd04814f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52434/0 (socket says 172.21.15.67:52434) 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd046a00640 1 -- 172.21.15.67:0/2858866004 learned_addr learned my addr 172.21.15.67:0/2858866004 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd047400640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd048151360 0x7fd0480c6670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd046000640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0480c30a0 0x7fd048148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd046a00640 1 -- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd048151360 msgr2=0x7fd0480c6670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd046a00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd048151360 0x7fd0480c6670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd046a00640 1 -- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0480c30a0 msgr2=0x7fd048148510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd046a00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0480c30a0 0x7fd048148510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd046a00640 1 -- 172.21.15.67:0/2858866004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd048150570 con 0x7fd0480ac930 2024-03-09T09:17:52.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.338+0000 7fd046000640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0480c30a0 0x7fd048148510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:17:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.339+0000 7fd046a00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0480ac930 0x7fd04814f760 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd03800b550 tx=0x7fd03800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.339+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038013020 con 0x7fd0480ac930 2024-03-09T09:17:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.339+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd038003de0 con 0x7fd0480ac930 2024-03-09T09:17:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.339+0000 7fd047e00640 1 -- 172.21.15.67:0/2858866004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd048148230 con 0x7fd0480ac930 2024-03-09T09:17:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.339+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0380052b0 con 0x7fd0480ac930 2024-03-09T09:17:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.339+0000 7fd047e00640 1 -- 172.21.15.67:0/2858866004 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04806dcc0 con 0x7fd0480ac930 2024-03-09T09:17:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.340+0000 7fd047e00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048154d70 0x7fd0481ee580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.340+0000 7fd047e00640 1 -- 172.21.15.67:0/2858866004 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04814fca0 con 0x7fd048154d70 2024-03-09T09:17:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.341+0000 7fd046000640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048154d70 0x7fd0481ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.341+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd038016020 con 0x7fd0480ac930 2024-03-09T09:17:52.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.342+0000 7fd046000640 1 -- 172.21.15.67:0/2858866004 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd048148a50 con 0x7fd048154d70 2024-03-09T09:17:52.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.342+0000 7fd03ea00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd02806b480 0x7fd02806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.342+0000 7fd046000640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048154d70 0x7fd0481ee580 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd0480c71f0 tx=0x7fd03400d8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:52.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.342+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd03808a3e0 con 0x7fd0480ac930 2024-03-09T09:17:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.343+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034018070 con 0x7fd048154d70 2024-03-09T09:17:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.343+0000 7fd047400640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd02806b480 0x7fd02806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.343+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd034003610 con 0x7fd048154d70 2024-03-09T09:17:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.343+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048154d70 msgr2=0x7fd0481ee580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.343+0000 7fd03ea00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048154d70 0x7fd0481ee580 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd0480c71f0 tx=0x7fd03400d8a0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.343+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048154d70 msgr2=0x7fd0481ee580 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:52.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.343+0000 7fd03ea00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048154d70 0x7fd0481ee580 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.344+0000 7fd047400640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd02806b480 0x7fd02806d940 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd040035000 tx=0x7fd040030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.350+0000 7fd047e00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048155150 0x7fd0481eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:17:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.351+0000 7fd047e00640 1 -- 172.21.15.67:0/2858866004 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480729e0 con 0x7fd048155150 2024-03-09T09:17:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.351+0000 7fd046000640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048155150 0x7fd0481eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:17:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.352+0000 7fd046000640 1 -- 172.21.15.67:0/2858866004 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd048067b30 con 0x7fd048155150 2024-03-09T09:17:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.352+0000 7fd046000640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048155150 0x7fd0481eeac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd034015000 tx=0x7fd034000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:17:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.353+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034018040 con 0x7fd048155150 2024-03-09T09:17:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.353+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4863 (secure 0 0 0) 0x7fd0340067d0 con 0x7fd048155150 2024-03-09T09:17:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.353+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048155150 msgr2=0x7fd0481eeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.353+0000 7fd03ea00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048155150 0x7fd0481eeac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd034015000 tx=0x7fd034000ed0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.353+0000 7fd03ea00640 1 -- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048155150 msgr2=0x7fd0481eeac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:17:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.353+0000 7fd03ea00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd048155150 0x7fd0481eeac0 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd034015000 tx=0x7fd034000ed0 comp rx=0 tx=0).stop 2024-03-09T09:17:52.360 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:17:52.360 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:17:52.360 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:17:52.360 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:17:52.360 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 190, 2024-03-09T09:17:52.360 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:17:52.360 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:17:52.360 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:17:52.361 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.361 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:17:52.361 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:52.361 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:17:52.361 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:17:52.361 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:52.361 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:52.361 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:52.361 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:52.361 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:52.361 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:52.362 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:52.362 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:52.362 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:52.362 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:52.362 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:52.362 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.362 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:17:52.362 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:17:52.362 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:52.363 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:52.363 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:52.363 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:52.363 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:52.363 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:52.363 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:52.363 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:52.363 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:52.363 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:52.363 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:52.363 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:17:52.364 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:17:52.365 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:17:52.366 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:17:52.366 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:17:52.366 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:17:52.366 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.366 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:17:52.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.366 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:17:52.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.366 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:17:52.366 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:17:52.366 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.367 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:52.368 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:52.369 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.369 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:52.369 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:17:52.369 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.369 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.369 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:52.369 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:17:52.369 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:17:52.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.371 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.372 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.373 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:17:52.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:17:52.375 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:17:52.375 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:17:52.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.354+0000 7fd047e00640 1 -- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd02806b480 msgr2=0x7fd02806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.354+0000 7fd047e00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd02806b480 0x7fd02806d940 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd040035000 tx=0x7fd040030000 comp rx=0 tx=0).stop 2024-03-09T09:17:52.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.354+0000 7fd047e00640 1 -- 172.21.15.67:0/2858866004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0480ac930 msgr2=0x7fd04814f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:17:52.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.354+0000 7fd047e00640 1 --2- 172.21.15.67:0/2858866004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0480ac930 0x7fd04814f760 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd03800b550 tx=0x7fd03800ba10 comp rx=0 tx=0).stop 2024-03-09T09:17:52.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.354+0000 7fd047400640 1 -- 172.21.15.67:0/2858866004 reap_dead start 2024-03-09T09:17:52.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.354+0000 7fd047400640 1 -- 172.21.15.67:0/2858866004 reap_dead start 2024-03-09T09:17:52.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.354+0000 7fd047e00640 1 -- 172.21.15.67:0/2858866004 shutdown_connections 2024-03-09T09:17:52.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.354+0000 7fd047e00640 1 -- 172.21.15.67:0/2858866004 >> 172.21.15.67:0/2858866004 conn(0x7fd0480c7220 msgr2=0x7fd04806d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:17:52.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.354+0000 7fd047e00640 1 -- 172.21.15.67:0/2858866004 shutdown_connections 2024-03-09T09:17:52.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:17:52.354+0000 7fd047e00640 1 -- 172.21.15.67:0/2858866004 wait complete. 2024-03-09T09:17:52.381 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:17:52.382 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:17:55.392 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:17:55.392 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:17:55.392 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 10 -eq 48 ']' 2024-03-09T09:17:55.392 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.10 2024-03-09T09:17:55.392 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.10 2024-03-09T09:17:55.393 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.10 2024-03-09T09:17:55.393 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.10 2024-03-09T09:17:55.396 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:17:55.922 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:55.923 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:55.923 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 16 -eq 48 ']' 2024-03-09T09:17:55.923 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.16 2024-03-09T09:17:55.923 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.16 2024-03-09T09:17:55.923 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.16 2024-03-09T09:17:55.923 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.16 2024-03-09T09:17:55.926 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:58.041 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:17:58.041 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:17:58.041 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 17 -eq 48 ']' 2024-03-09T09:17:58.041 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.17 2024-03-09T09:17:58.041 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.17 2024-03-09T09:17:58.042 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.17 2024-03-09T09:17:58.042 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.17 2024-03-09T09:17:58.045 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:17:58.955 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:17:58.955 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:17:58.955 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 11 -eq 48 ']' 2024-03-09T09:17:58.956 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.11 2024-03-09T09:17:58.956 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.11 2024-03-09T09:17:58.956 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.11 2024-03-09T09:17:58.956 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.11 2024-03-09T09:17:58.960 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:01.194 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:01.194 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:01.195 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 18 -eq 48 ']' 2024-03-09T09:18:01.195 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.18 2024-03-09T09:18:01.195 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.18 2024-03-09T09:18:01.195 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.18 2024-03-09T09:18:01.195 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.18 2024-03-09T09:18:01.199 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:02.157 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:02.157 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:02.157 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 12 -eq 48 ']' 2024-03-09T09:18:02.158 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.12 2024-03-09T09:18:02.158 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.12 2024-03-09T09:18:02.158 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.12 2024-03-09T09:18:02.158 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.12 2024-03-09T09:18:02.162 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:05.492 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:05.492 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:05.492 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 13 -eq 48 ']' 2024-03-09T09:18:05.492 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.13 2024-03-09T09:18:05.493 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.13 2024-03-09T09:18:05.493 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.13 2024-03-09T09:18:05.493 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.13 2024-03-09T09:18:05.496 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:06.360 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:06.360 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:06.360 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 19 -eq 48 ']' 2024-03-09T09:18:06.360 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.19 2024-03-09T09:18:06.361 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.19 2024-03-09T09:18:06.361 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.19 2024-03-09T09:18:06.361 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.19 2024-03-09T09:18:06.365 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:09.636 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:09.636 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:09.637 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 14 -eq 48 ']' 2024-03-09T09:18:09.637 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.14 2024-03-09T09:18:09.637 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.14 2024-03-09T09:18:09.637 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.14 2024-03-09T09:18:09.637 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.14 2024-03-09T09:18:09.642 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:10.335 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:10.335 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:10.335 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 20 -eq 48 ']' 2024-03-09T09:18:10.335 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.20 2024-03-09T09:18:10.336 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.20 2024-03-09T09:18:10.336 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.20 2024-03-09T09:18:10.336 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.20 2024-03-09T09:18:10.339 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:12.056 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:12.056 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:12.056 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 15 -eq 48 ']' 2024-03-09T09:18:12.056 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.15 2024-03-09T09:18:12.057 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.15 2024-03-09T09:18:12.057 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.15 2024-03-09T09:18:12.057 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.15 2024-03-09T09:18:12.060 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:12.382 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:18:12.382 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:18:12.383 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:18:12.383 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:18:12.391 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:18:12.391 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-03-09T09:18:12.391 INFO:tasks.ceph.mon.a:Sent signal 19 2024-03-09T09:18:12.391 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-03-09T09:18:13.616 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:13.617 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:13.617 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 21 -eq 48 ']' 2024-03-09T09:18:13.617 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.21 2024-03-09T09:18:13.617 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.21 2024-03-09T09:18:13.617 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.21 2024-03-09T09:18:13.617 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.21 2024-03-09T09:18:13.621 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:14.851 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:14.852 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:14.852 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 16 -eq 48 ']' 2024-03-09T09:18:14.852 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.16 2024-03-09T09:18:14.853 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.16 2024-03-09T09:18:14.853 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.16 2024-03-09T09:18:14.853 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.16 2024-03-09T09:18:14.857 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:17.087 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:17.087 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:17.087 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 22 -eq 48 ']' 2024-03-09T09:18:17.087 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.22 2024-03-09T09:18:17.087 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.22 2024-03-09T09:18:17.088 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.22 2024-03-09T09:18:17.088 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.22 2024-03-09T09:18:17.092 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:18.869 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:18.869 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:18.869 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 17 -eq 48 ']' 2024-03-09T09:18:18.869 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.17 2024-03-09T09:18:18.869 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.17 2024-03-09T09:18:18.870 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.17 2024-03-09T09:18:18.870 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.17 2024-03-09T09:18:18.873 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:21.992 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:21.992 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:21.992 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 23 -eq 48 ']' 2024-03-09T09:18:21.993 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.23 2024-03-09T09:18:21.993 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.23 2024-03-09T09:18:21.993 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.23 2024-03-09T09:18:21.993 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.23 2024-03-09T09:18:21.996 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:22.187 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:22.187 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:22.187 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 18 -eq 48 ']' 2024-03-09T09:18:22.187 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.18 2024-03-09T09:18:22.188 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.18 2024-03-09T09:18:22.188 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.18 2024-03-09T09:18:22.188 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.18 2024-03-09T09:18:22.192 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:24.154 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:24.154 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:24.155 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 24 -eq 48 ']' 2024-03-09T09:18:24.155 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.24 2024-03-09T09:18:24.155 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.24 2024-03-09T09:18:24.155 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.24 2024-03-09T09:18:24.155 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.24 2024-03-09T09:18:24.158 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:27.392 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-03-09T09:18:27.393 INFO:tasks.ceph.mon.a:Sent signal 18 2024-03-09T09:18:27.393 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:18:27.393 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:18:27.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.611+0000 7f1ecd800640 1 Processor -- start 2024-03-09T09:18:27.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.611+0000 7f1ecd800640 1 -- start start 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.614+0000 7f1ecd800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1ec8072770 0x7f1ec8072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.614+0000 7f1ecd800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80bebc0 0x7f1ec8156650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.614+0000 7f1ecd800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec806cd30 con 0x7f1ec806b690 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.614+0000 7f1ecd800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec806b200 con 0x7f1ec8072770 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.614+0000 7f1ecd800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c5eb0 con 0x7f1ec80bebc0 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.614+0000 7f1ec7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1ec8072770 0x7f1ec8072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.614+0000 7f1ecce00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1ec806b690 0x7f1ec806bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42874/0 (socket says 172.21.15.67:42874) 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.614+0000 7f1ecce00640 1 -- 172.21.15.67:0/1969862020 learned_addr learned my addr 172.21.15.67:0/1969862020 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.614+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/1969862020 >> v2:172.21.15.67:3300/0 conn(0x7f1ec8072770 0x7f1ec8072b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55704/0 (socket says 172.21.15.67:55704) 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.614+0000 7f1ec7400640 1 --2- 172.21.15.67:0/1969862020 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80bebc0 0x7f1ec8156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.615+0000 7f1ec7e00640 1 -- 172.21.15.67:0/1969862020 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80bebc0 msgr2=0x7f1ec8156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:27.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.615+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/1969862020 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80bebc0 0x7f1ec8156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:27.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.615+0000 7f1ec7e00640 1 -- 172.21.15.67:0/1969862020 >> v1:172.21.15.67:6789/0 conn(0x7f1ec806b690 legacy=0x7f1ec806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:27.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.615+0000 7f1ec7e00640 1 -- 172.21.15.67:0/1969862020 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80c6150 con 0x7f1ec8072770 2024-03-09T09:18:27.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.615+0000 7f1ec6a00640 1 -- 172.21.15.67:0/1969862020 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982692481 0 0) 0x7f1ec806cd30 con 0x7f1ec806b690 2024-03-09T09:18:27.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.616+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/1969862020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec8072770 0x7f1ec8072b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80097a0 tx=0x7f1eb802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f011720ad26a6091 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:27.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.616+0000 7f1ec6a00640 1 -- 172.21.15.67:0/1969862020 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8038070 con 0x7f1ec8072770 2024-03-09T09:18:27.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.616+0000 7f1ec6a00640 1 -- 172.21.15.67:0/1969862020 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb8031700 con 0x7f1ec8072770 2024-03-09T09:18:27.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.616+0000 7f1ec6a00640 1 -- 172.21.15.67:0/1969862020 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8031ad0 con 0x7f1ec8072770 2024-03-09T09:18:27.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.617+0000 7f1ecd800640 1 -- 172.21.15.67:0/1969862020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec8072770 msgr2=0x7f1ec8072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:27.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.617+0000 7f1ecd800640 1 --2- 172.21.15.67:0/1969862020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec8072770 0x7f1ec8072b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1eb80097a0 tx=0x7f1eb802b7a0 comp rx=0 tx=0).stop 2024-03-09T09:18:27.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.617+0000 7f1ecd800640 1 -- 172.21.15.67:0/1969862020 shutdown_connections 2024-03-09T09:18:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.617+0000 7f1ecd800640 1 --2- 172.21.15.67:0/1969862020 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80bebc0 0x7f1ec8156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.617+0000 7f1ecd800640 1 --2- 172.21.15.67:0/1969862020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec8072770 0x7f1ec8072b50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.617+0000 7f1ecd800640 1 -- 172.21.15.67:0/1969862020 >> 172.21.15.67:0/1969862020 conn(0x7f1ec80c7fb0 msgr2=0x7f1ec80c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.617+0000 7f1ecd800640 1 -- 172.21.15.67:0/1969862020 shutdown_connections 2024-03-09T09:18:27.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.617+0000 7f1ecd800640 1 -- 172.21.15.67:0/1969862020 wait complete. 2024-03-09T09:18:27.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.619+0000 7f1ecd800640 1 Processor -- start 2024-03-09T09:18:27.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.619+0000 7f1ecd800640 1 -- start start 2024-03-09T09:18:27.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.620+0000 7f1ecd800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec806b690 0x7f1ec8159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:27.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.620+0000 7f1ecd800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec8072770 0x7f1ec8159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.620+0000 7f1ecd800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80bebc0 0x7f1ec80c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.620+0000 7f1ecd800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c9430 con 0x7f1ec8072770 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.620+0000 7f1ecd800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c6710 con 0x7f1ec806b690 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.620+0000 7f1ecd800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c3210 con 0x7f1ec80bebc0 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.620+0000 7f1ec7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec8072770 0x7f1ec8159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.620+0000 7f1ec7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec8072770 0x7f1ec8159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55710/0 (socket says 172.21.15.67:55710) 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.620+0000 7f1ec7400640 1 -- 172.21.15.67:0/2174588616 learned_addr learned my addr 172.21.15.67:0/2174588616 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.620+0000 7f1ecce00640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80bebc0 0x7f1ec80c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.620+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec806b690 0x7f1ec8159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.621+0000 7f1ec7400640 1 -- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80bebc0 msgr2=0x7f1ec80c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.621+0000 7f1ec7400640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80bebc0 0x7f1ec80c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:27.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.621+0000 7f1ec7400640 1 -- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec806b690 msgr2=0x7f1ec8159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:27.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.621+0000 7f1ec7400640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec806b690 0x7f1ec8159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:27.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.621+0000 7f1ec7400640 1 -- 172.21.15.67:0/2174588616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80c2600 con 0x7f1ec8072770 2024-03-09T09:18:27.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.621+0000 7f1ecce00640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80bebc0 0x7f1ec80c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:18:27.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.621+0000 7f1ec7400640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec8072770 0x7f1ec8159c70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d800 tx=0x7f1eb400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:27.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.621+0000 7f1ebfe00640 1 -- 172.21.15.67:0/2174588616 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400b6c0 con 0x7f1ec8072770 2024-03-09T09:18:27.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.621+0000 7f1ebfe00640 1 -- 172.21.15.67:0/2174588616 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb4019070 con 0x7f1ec8072770 2024-03-09T09:18:27.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.621+0000 7f1ecd800640 1 -- 172.21.15.67:0/2174588616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec80c28e0 con 0x7f1ec8072770 2024-03-09T09:18:27.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.621+0000 7f1ebfe00640 1 -- 172.21.15.67:0/2174588616 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400ba80 con 0x7f1ec8072770 2024-03-09T09:18:27.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.622+0000 7f1ecd800640 1 -- 172.21.15.67:0/2174588616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec80c2ea0 con 0x7f1ec8072770 2024-03-09T09:18:27.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.623+0000 7f1ecd800640 1 -- 172.21.15.67:0/2174588616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec804f3c0 con 0x7f1ec8072770 2024-03-09T09:18:27.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.624+0000 7f1ebfe00640 1 -- 172.21.15.67:0/2174588616 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1eb400bc20 con 0x7f1ec8072770 2024-03-09T09:18:27.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.625+0000 7f1ebfe00640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ea806b4d0 0x7f1ea806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:27.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.625+0000 7f1ebfe00640 1 -- 172.21.15.67:0/2174588616 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1eb408ab40 con 0x7f1ec8072770 2024-03-09T09:18:27.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.626+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ea806b4d0 0x7f1ea806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:27.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.627+0000 7f1ec7e00640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ea806b4d0 0x7f1ea806d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8035000 tx=0x7f1eb8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:27.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.629+0000 7f1ebfe00640 1 -- 172.21.15.67:0/2174588616 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1eb4059530 con 0x7f1ec8072770 2024-03-09T09:18:27.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.927+0000 7f1ecd800640 1 -- 172.21.15.67:0/2174588616 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1ec806c980 con 0x7f1ec8072770 2024-03-09T09:18:27.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.928+0000 7f1ebfe00640 1 -- 172.21.15.67:0/2174588616 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f1eb4058e30 con 0x7f1ec8072770 2024-03-09T09:18:27.934 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":194,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:18:27.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 -- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ea806b4d0 msgr2=0x7f1ea806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:27.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ea806b4d0 0x7f1ea806d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8035000 tx=0x7f1eb8008030 comp rx=0 tx=0).stop 2024-03-09T09:18:27.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 -- 172.21.15.67:0/2174588616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec8072770 msgr2=0x7f1ec8159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:27.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec8072770 0x7f1ec8159c70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d800 tx=0x7f1eb400dcc0 comp rx=0 tx=0).stop 2024-03-09T09:18:27.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 -- 172.21.15.67:0/2174588616 shutdown_connections 2024-03-09T09:18:27.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ea806b4d0 0x7f1ea806d990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:27.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1ec80bebc0 0x7f1ec80c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:27.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ec8072770 0x7f1ec8159c70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:27.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 --2- 172.21.15.67:0/2174588616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ec806b690 0x7f1ec8159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:27.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 -- 172.21.15.67:0/2174588616 >> 172.21.15.67:0/2174588616 conn(0x7f1ec80c7fb0 msgr2=0x7f1ec8155080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:27.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 -- 172.21.15.67:0/2174588616 shutdown_connections 2024-03-09T09:18:27.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:27.931+0000 7f1ecd800640 1 -- 172.21.15.67:0/2174588616 wait complete. 2024-03-09T09:18:27.955 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:18:27.955 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:18:27.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:18:28.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.214+0000 7f2321c00640 1 Processor -- start 2024-03-09T09:18:28.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.214+0000 7f2321c00640 1 -- start start 2024-03-09T09:18:28.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.215+0000 7f2321c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f231c0c9380 0x7f231c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.215+0000 7f2321c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c0ad440 0x7f231c06c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.215+0000 7f2321c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231c0c9890 con 0x7f231c14fd90 2024-03-09T09:18:28.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.215+0000 7f2321c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231c06ba60 con 0x7f231c0c9380 2024-03-09T09:18:28.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.215+0000 7f2321c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c0c5f80 con 0x7f231c0ad440 2024-03-09T09:18:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.216+0000 7f2321200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c0ad440 0x7f231c06c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.216+0000 7f2313400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f231c0c9380 0x7f231c0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.216+0000 7f2313400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f231c0c9380 0x7f231c0acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52598/0 (socket says 172.21.15.67:52598) 2024-03-09T09:18:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.216+0000 7f2313400640 1 -- 172.21.15.67:0/2336718018 learned_addr learned my addr 172.21.15.67:0/2336718018 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:28.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.217+0000 7f2313400640 1 -- 172.21.15.67:0/2336718018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c0ad440 msgr2=0x7f231c06c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:28.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.217+0000 7f2313400640 1 --2- 172.21.15.67:0/2336718018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c0ad440 0x7f231c06c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.217+0000 7f2313400640 1 -- 172.21.15.67:0/2336718018 >> v1:172.21.15.134:6789/0 conn(0x7f231c14fd90 legacy=0x7f231c156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.217+0000 7f2313400640 1 -- 172.21.15.67:0/2336718018 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c0c6240 con 0x7f231c0c9380 2024-03-09T09:18:28.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.217+0000 7f2321200640 1 --2- 172.21.15.67:0/2336718018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c0ad440 0x7f231c06c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:18:28.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.219+0000 7f2313400640 1 --2- 172.21.15.67:0/2336718018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c0c9380 0x7f231c0acf00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2304009870 tx=0x7f230402b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d062337bfcbebc15 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.219+0000 7f231b400640 1 -- 172.21.15.67:0/2336718018 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23040317c0 con 0x7f231c0c9380 2024-03-09T09:18:28.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.219+0000 7f231b400640 1 -- 172.21.15.67:0/2336718018 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2304037070 con 0x7f231c0c9380 2024-03-09T09:18:28.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.219+0000 7f231b400640 1 -- 172.21.15.67:0/2336718018 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304031b80 con 0x7f231c0c9380 2024-03-09T09:18:28.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.220+0000 7f2321c00640 1 -- 172.21.15.67:0/2336718018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c0c9380 msgr2=0x7f231c0acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.220+0000 7f2321c00640 1 --2- 172.21.15.67:0/2336718018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c0c9380 0x7f231c0acf00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2304009870 tx=0x7f230402b860 comp rx=0 tx=0).stop 2024-03-09T09:18:28.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.220+0000 7f2321c00640 1 -- 172.21.15.67:0/2336718018 shutdown_connections 2024-03-09T09:18:28.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.220+0000 7f2321c00640 1 --2- 172.21.15.67:0/2336718018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c0ad440 0x7f231c06c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.220+0000 7f2321c00640 1 --2- 172.21.15.67:0/2336718018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c0c9380 0x7f231c0acf00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.220+0000 7f2321c00640 1 -- 172.21.15.67:0/2336718018 >> 172.21.15.67:0/2336718018 conn(0x7f231c0cb480 msgr2=0x7f231c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:28.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.220+0000 7f2321c00640 1 -- 172.21.15.67:0/2336718018 shutdown_connections 2024-03-09T09:18:28.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.220+0000 7f2321c00640 1 -- 172.21.15.67:0/2336718018 wait complete. 2024-03-09T09:18:28.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.222+0000 7f2321c00640 1 Processor -- start 2024-03-09T09:18:28.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.222+0000 7f2321c00640 1 -- start start 2024-03-09T09:18:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f2321c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0ad440 0x7f231c0c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f2321c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c0c9380 0x7f231c14c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f2321c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c14fd90 0x7f231c14cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f2321c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c06c710 con 0x7f231c0ad440 2024-03-09T09:18:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f2321c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c06bcb0 con 0x7f231c14fd90 2024-03-09T09:18:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f2321c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c06c890 con 0x7f231c0c9380 2024-03-09T09:18:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f231be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0ad440 0x7f231c0c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f231be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0ad440 0x7f231c0c3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55722/0 (socket says 172.21.15.67:55722) 2024-03-09T09:18:28.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f231be00640 1 -- 172.21.15.67:0/3919129473 learned_addr learned my addr 172.21.15.67:0/3919129473 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:28.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f2313400640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c0c9380 0x7f231c14c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f2321200640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c14fd90 0x7f231c14cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f231be00640 1 -- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c0c9380 msgr2=0x7f231c14c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f231be00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f231c0c9380 0x7f231c14c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f231be00640 1 -- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c14fd90 msgr2=0x7f231c14cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f231be00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c14fd90 0x7f231c14cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f231be00640 1 -- 172.21.15.67:0/3919129473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c0c54e0 con 0x7f231c0ad440 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.224+0000 7f2321200640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f231c14fd90 0x7f231c14cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.225+0000 7f231be00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0ad440 0x7f231c0c3540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f230c00c730 tx=0x7f230c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.225+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c016020 con 0x7f231c0ad440 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.225+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230c006d50 con 0x7f231c0ad440 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.225+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c003bb0 con 0x7f231c0ad440 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.225+0000 7f2321c00640 1 -- 172.21.15.67:0/3919129473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231c0c5200 con 0x7f231c0ad440 2024-03-09T09:18:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.226+0000 7f2321c00640 1 -- 172.21.15.67:0/3919129473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231c1589e0 con 0x7f231c0ad440 2024-03-09T09:18:28.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.227+0000 7f2321c00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2aa0 0x7f231c1ef200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.227+0000 7f2321c00640 1 -- 172.21.15.67:0/3919129473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c04f3c0 con 0x7f231c0c2aa0 2024-03-09T09:18:28.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.227+0000 7f2313400640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2aa0 0x7f231c1ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.228+0000 7f2313400640 1 -- 172.21.15.67:0/3919129473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f231c06d2d0 con 0x7f231c0c2aa0 2024-03-09T09:18:28.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.228+0000 7f2313400640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2aa0 0x7f231c1ef200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f231c0ada40 tx=0x7f2304004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.228+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f230c00a850 con 0x7f231c0ad440 2024-03-09T09:18:28.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.229+0000 7f2318c00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22f806b4d0 0x7f22f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.229+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f230c08a1d0 con 0x7f231c0ad440 2024-03-09T09:18:28.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.230+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230404a070 con 0x7f231c0c2aa0 2024-03-09T09:18:28.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.230+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f2304037040 con 0x7f231c0c2aa0 2024-03-09T09:18:28.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.230+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2aa0 msgr2=0x7f231c1ef200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.230+0000 7f2318c00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2aa0 0x7f231c1ef200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f231c0ada40 tx=0x7f2304004730 comp rx=0 tx=0).stop 2024-03-09T09:18:28.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.230+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2aa0 msgr2=0x7f231c1ef200 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:28.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.230+0000 7f2318c00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2aa0 0x7f231c1ef200 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f231c0ada40 tx=0x7f2304004730 comp rx=0 tx=0).stop 2024-03-09T09:18:28.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.230+0000 7f2321200640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22f806b4d0 0x7f22f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.231+0000 7f2321200640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22f806b4d0 0x7f22f806d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2314009aa0 tx=0x7f2314008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.240+0000 7f2321c00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2550 0x7f231c1ef740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.240+0000 7f2321c00640 1 -- 172.21.15.67:0/3919129473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c14f050 con 0x7f231c0c2550 2024-03-09T09:18:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.241+0000 7f2313400640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2550 0x7f231c1ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.241+0000 7f2313400640 1 -- 172.21.15.67:0/3919129473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f231c0cba00 con 0x7f231c0c2550 2024-03-09T09:18:28.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.242+0000 7f2313400640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2550 0x7f231c1ef740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2304009870 tx=0x7f23040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.242+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304053070 con 0x7f231c0c2550 2024-03-09T09:18:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.242+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f2304031ea0 con 0x7f231c0c2550 2024-03-09T09:18:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.242+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2550 msgr2=0x7f231c1ef740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.242+0000 7f2318c00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2550 0x7f231c1ef740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2304009870 tx=0x7f23040046c0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.242+0000 7f2318c00640 1 -- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2550 msgr2=0x7f231c1ef740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.242+0000 7f2318c00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0c2550 0x7f231c1ef740 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2304009870 tx=0x7f23040046c0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.249 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:18:28.249 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:18:28.249 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:18:28.249 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:18:28.249 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 194, 2024-03-09T09:18:28.249 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:18:28.249 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:18:28.249 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:18:28.249 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:18:28.249 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:28.250 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:28.251 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:18:28.252 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:28.253 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.254 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:28.255 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:28.256 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:18:28.257 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:28.259 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.260 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.261 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:18:28.262 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.262 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:28.262 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.262 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:28.262 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.262 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.262 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:18:28.262 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:18:28.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.243+0000 7f2321c00640 1 -- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22f806b4d0 msgr2=0x7f22f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.243+0000 7f2321c00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f22f806b4d0 0x7f22f806d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2314009aa0 tx=0x7f2314008040 comp rx=0 tx=0).stop 2024-03-09T09:18:28.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.243+0000 7f2321c00640 1 -- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0ad440 msgr2=0x7f231c0c3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.243+0000 7f2321c00640 1 --2- 172.21.15.67:0/3919129473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f231c0ad440 0x7f231c0c3540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f230c00c730 tx=0x7f230c00cbf0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.243+0000 7f2321200640 1 -- 172.21.15.67:0/3919129473 reap_dead start 2024-03-09T09:18:28.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.243+0000 7f2321200640 1 -- 172.21.15.67:0/3919129473 reap_dead start 2024-03-09T09:18:28.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.243+0000 7f2321c00640 1 -- 172.21.15.67:0/3919129473 shutdown_connections 2024-03-09T09:18:28.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.243+0000 7f2321c00640 1 -- 172.21.15.67:0/3919129473 >> 172.21.15.67:0/3919129473 conn(0x7f231c0cb480 msgr2=0x7f231c1540f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:28.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.243+0000 7f2321c00640 1 -- 172.21.15.67:0/3919129473 shutdown_connections 2024-03-09T09:18:28.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.243+0000 7f2321c00640 1 -- 172.21.15.67:0/3919129473 wait complete. 2024-03-09T09:18:28.267 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:18:28.267 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:18:28.267 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:18:28.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.481+0000 7fcc5d600640 1 Processor -- start 2024-03-09T09:18:28.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.481+0000 7fcc5d600640 1 -- start start 2024-03-09T09:18:28.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.482+0000 7fcc5d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcc5806e630 0x7fcc5815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.482+0000 7fcc5d600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5815a370 con 0x7fcc580c38b0 2024-03-09T09:18:28.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.482+0000 7fcc5d600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5815a650 con 0x7fcc58157890 2024-03-09T09:18:28.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.482+0000 7fcc5d600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5815b130 con 0x7fcc5806e630 2024-03-09T09:18:28.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.482+0000 7fcc4e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcc5806e630 0x7fcc5815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.482+0000 7fcc4e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcc5806e630 0x7fcc5815b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55756/0 (socket says 172.21.15.67:55756) 2024-03-09T09:18:28.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.482+0000 7fcc57400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcc580c38b0 0x7fcc5815bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:42878/0 (socket says 172.21.15.67:42878) 2024-03-09T09:18:28.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.482+0000 7fcc4e000640 1 -- 172.21.15.67:0/4288163926 learned_addr learned my addr 172.21.15.67:0/4288163926 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:28.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.483+0000 7fcc4e000640 1 -- 172.21.15.67:0/4288163926 >> v1:172.21.15.134:6789/0 conn(0x7fcc58157890 legacy=0x7fcc5806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:28.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.483+0000 7fcc4e000640 1 -- 172.21.15.67:0/4288163926 >> v1:172.21.15.67:6789/0 conn(0x7fcc580c38b0 legacy=0x7fcc5815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.483+0000 7fcc4e000640 1 -- 172.21.15.67:0/4288163926 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5815a930 con 0x7fcc5806e630 2024-03-09T09:18:28.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.485+0000 7fcc4e000640 1 --2- 172.21.15.67:0/4288163926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5806e630 0x7fcc5815b5d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009870 tx=0x7fcc4002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0d2b038cd4ffd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.485+0000 7fcc56000640 1 -- 172.21.15.67:0/4288163926 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40034070 con 0x7fcc5806e630 2024-03-09T09:18:28.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.485+0000 7fcc56000640 1 -- 172.21.15.67:0/4288163926 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc40030860 con 0x7fcc5806e630 2024-03-09T09:18:28.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.485+0000 7fcc56000640 1 -- 172.21.15.67:0/4288163926 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40030c30 con 0x7fcc5806e630 2024-03-09T09:18:28.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.486+0000 7fcc5d600640 1 -- 172.21.15.67:0/4288163926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5806e630 msgr2=0x7fcc5815b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.486+0000 7fcc5d600640 1 --2- 172.21.15.67:0/4288163926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5806e630 0x7fcc5815b5d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009870 tx=0x7fcc4002b9a0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.486+0000 7fcc5d600640 1 -- 172.21.15.67:0/4288163926 shutdown_connections 2024-03-09T09:18:28.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.486+0000 7fcc5d600640 1 --2- 172.21.15.67:0/4288163926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5806e630 0x7fcc5815b5d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.486+0000 7fcc5d600640 1 -- 172.21.15.67:0/4288163926 >> 172.21.15.67:0/4288163926 conn(0x7fcc580c7370 msgr2=0x7fcc580c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:28.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.486+0000 7fcc5d600640 1 -- 172.21.15.67:0/4288163926 shutdown_connections 2024-03-09T09:18:28.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.486+0000 7fcc5d600640 1 -- 172.21.15.67:0/4288163926 wait complete. 2024-03-09T09:18:28.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.489+0000 7fcc5d600640 1 Processor -- start 2024-03-09T09:18:28.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.489+0000 7fcc5d600640 1 -- start start 2024-03-09T09:18:28.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.490+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5806e630 0x7fcc581f0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.490+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc580c38b0 0x7fcc58159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.490+0000 7fcc5d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58157890 0x7fcc581587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.490+0000 7fcc5d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5806de20 con 0x7fcc5806e630 2024-03-09T09:18:28.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.490+0000 7fcc5d600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5806dca0 con 0x7fcc58157890 2024-03-09T09:18:28.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.490+0000 7fcc5d600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc580c1d70 con 0x7fcc580c38b0 2024-03-09T09:18:28.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.491+0000 7fcc56a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5806e630 0x7fcc581f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.491+0000 7fcc56a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5806e630 0x7fcc581f0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55772/0 (socket says 172.21.15.67:55772) 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.491+0000 7fcc56a00640 1 -- 172.21.15.67:0/3361031039 learned_addr learned my addr 172.21.15.67:0/3361031039 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.491+0000 7fcc4e000640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc580c38b0 0x7fcc58159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.491+0000 7fcc57400640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58157890 0x7fcc581587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.491+0000 7fcc56a00640 1 -- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc580c38b0 msgr2=0x7fcc58159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.491+0000 7fcc56a00640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc580c38b0 0x7fcc58159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.491+0000 7fcc56a00640 1 -- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58157890 msgr2=0x7fcc581587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.491+0000 7fcc56a00640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58157890 0x7fcc581587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.491+0000 7fcc56a00640 1 -- 172.21.15.67:0/3361031039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc581e5de0 con 0x7fcc5806e630 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.491+0000 7fcc57400640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58157890 0x7fcc581587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.492+0000 7fcc56a00640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5806e630 0x7fcc581f0270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c7c0 tx=0x7fcc4800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.492+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48016020 con 0x7fcc5806e630 2024-03-09T09:18:28.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.492+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4800aa50 con 0x7fcc5806e630 2024-03-09T09:18:28.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.492+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48012420 con 0x7fcc5806e630 2024-03-09T09:18:28.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.492+0000 7fcc5d600640 1 -- 172.21.15.67:0/3361031039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc58152b60 con 0x7fcc5806e630 2024-03-09T09:18:28.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.492+0000 7fcc5d600640 1 -- 172.21.15.67:0/3361031039 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc58153120 con 0x7fcc5806e630 2024-03-09T09:18:28.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.493+0000 7fcc5d600640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58152660 0x7fcc58157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.493+0000 7fcc5d600640 1 -- 172.21.15.67:0/3361031039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc58158510 con 0x7fcc58152660 2024-03-09T09:18:28.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.494+0000 7fcc4e000640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58152660 0x7fcc58157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.494+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcc48019020 con 0x7fcc5806e630 2024-03-09T09:18:28.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.495+0000 7fcc4e000640 1 -- 172.21.15.67:0/3361031039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc58075a00 con 0x7fcc58152660 2024-03-09T09:18:28.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.495+0000 7fcc4f400640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc3406b500 0x7fcc3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.495+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcc4808a3a0 con 0x7fcc5806e630 2024-03-09T09:18:28.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.495+0000 7fcc4e000640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58152660 0x7fcc58157fd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc580c4f10 tx=0x7fcc40004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.496+0000 7fcc57400640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc3406b500 0x7fcc3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.496+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40034040 con 0x7fcc58152660 2024-03-09T09:18:28.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.497+0000 7fcc57400640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc3406b500 0x7fcc3406d9c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcc50000f60 tx=0x7fcc50009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.497+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fcc40039c30 con 0x7fcc58152660 2024-03-09T09:18:28.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.497+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58152660 msgr2=0x7fcc58157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.497+0000 7fcc4f400640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58152660 0x7fcc58157fd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc580c4f10 tx=0x7fcc40004730 comp rx=0 tx=0).stop 2024-03-09T09:18:28.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.497+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58152660 msgr2=0x7fcc58157fd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:28.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.497+0000 7fcc4f400640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58152660 0x7fcc58157fd0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc580c4f10 tx=0x7fcc40004730 comp rx=0 tx=0).stop 2024-03-09T09:18:28.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.504+0000 7fcc5d600640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58158d10 0x7fcc581e4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.504+0000 7fcc5d600640 1 -- 172.21.15.67:0/3361031039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc581f07b0 con 0x7fcc58158d10 2024-03-09T09:18:28.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.505+0000 7fcc4e000640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58158d10 0x7fcc581e4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.505+0000 7fcc4e000640 1 -- 172.21.15.67:0/3361031039 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcc580c7cb0 con 0x7fcc58158d10 2024-03-09T09:18:28.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.506+0000 7fcc4e000640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58158d10 0x7fcc581e4ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009870 tx=0x7fcc400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.506+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40050070 con 0x7fcc58158d10 2024-03-09T09:18:28.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.507+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4874 (secure 0 0 0) 0x7fcc4004aa80 con 0x7fcc58158d10 2024-03-09T09:18:28.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.507+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58158d10 msgr2=0x7fcc581e4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.507+0000 7fcc4f400640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58158d10 0x7fcc581e4ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009870 tx=0x7fcc400046c0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.507+0000 7fcc4f400640 1 -- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58158d10 msgr2=0x7fcc581e4ed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:28.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.507+0000 7fcc4f400640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc58158d10 0x7fcc581e4ed0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009870 tx=0x7fcc400046c0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.514 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:18:28.514 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:18:28.514 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:18:28.514 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:18:28.514 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 194, 2024-03-09T09:18:28.514 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:18:28.515 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:18:28.515 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:18:28.515 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:18:28.515 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.515 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:18:28.515 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:28.515 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:18:28.515 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:18:28.515 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:28.515 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:28.515 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:28.516 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:28.516 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:28.516 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:28.516 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:28.516 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:28.516 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:28.516 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:28.516 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:28.516 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.516 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:18:28.516 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.518 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:18:28.518 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:18:28.518 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:18:28.518 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:18:28.518 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:18:28.518 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:18:28.518 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:28.518 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:28.518 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:18:28.518 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:18:28.518 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:28.519 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:18:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:18:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:18:28.521 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:18:28.521 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:28.521 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:28.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.521 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:28.521 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:18:28.521 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.521 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:28.521 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:18:28.522 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.522 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.522 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.522 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:18:28.522 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:18:28.522 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:28.522 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:28.522 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:28.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:28.523 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:18:28.524 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.524 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.524 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.524 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:18:28.524 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:18:28.524 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:28.524 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:28.524 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:28.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.525 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:18:28.525 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:18:28.525 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:28.525 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:28.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.525 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:28.525 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:18:28.525 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:28.526 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:18:28.526 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.526 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.526 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:18:28.526 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:18:28.526 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:28.527 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:28.528 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:18:28.529 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:18:28.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.508+0000 7fcc5d600640 1 -- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc3406b500 msgr2=0x7fcc3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.508+0000 7fcc5d600640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc3406b500 0x7fcc3406d9c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcc50000f60 tx=0x7fcc50009040 comp rx=0 tx=0).stop 2024-03-09T09:18:28.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.508+0000 7fcc5d600640 1 -- 172.21.15.67:0/3361031039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5806e630 msgr2=0x7fcc581f0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.508+0000 7fcc5d600640 1 --2- 172.21.15.67:0/3361031039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc5806e630 0x7fcc581f0270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c7c0 tx=0x7fcc4800cc80 comp rx=0 tx=0).stop 2024-03-09T09:18:28.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.508+0000 7fcc57400640 1 -- 172.21.15.67:0/3361031039 reap_dead start 2024-03-09T09:18:28.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.508+0000 7fcc57400640 1 -- 172.21.15.67:0/3361031039 reap_dead start 2024-03-09T09:18:28.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.508+0000 7fcc5d600640 1 -- 172.21.15.67:0/3361031039 shutdown_connections 2024-03-09T09:18:28.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.508+0000 7fcc5d600640 1 -- 172.21.15.67:0/3361031039 >> 172.21.15.67:0/3361031039 conn(0x7fcc580c7370 msgr2=0x7fcc58154f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:28.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.508+0000 7fcc5d600640 1 -- 172.21.15.67:0/3361031039 shutdown_connections 2024-03-09T09:18:28.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.508+0000 7fcc5d600640 1 -- 172.21.15.67:0/3361031039 wait complete. 2024-03-09T09:18:28.535 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:18:28.535 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:18:28.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:18:28.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:18:28.746 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:18:28.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.790+0000 7f7442e00640 1 Processor -- start 2024-03-09T09:18:28.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.790+0000 7f7442e00640 1 -- start start 2024-03-09T09:18:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.792+0000 7f7442e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f743c0c9380 0x7f743c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.792+0000 7f7442e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f743c0ad440 0x7f743c06c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.792+0000 7f7442e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c0c9890 con 0x7f743c14fd90 2024-03-09T09:18:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.792+0000 7f7442e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743c06ba60 con 0x7f743c0ad440 2024-03-09T09:18:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.792+0000 7f7442e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743c0c5f80 con 0x7f743c0c9380 2024-03-09T09:18:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.793+0000 7f7442400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f743c0ad440 0x7f743c06c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.793+0000 7f7442400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f743c0ad440 0x7f743c06c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55786/0 (socket says 172.21.15.67:55786) 2024-03-09T09:18:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.793+0000 7f7442400640 1 -- 172.21.15.67:0/4081134497 learned_addr learned my addr 172.21.15.67:0/4081134497 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:28.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.793+0000 7f7441000640 1 --2- 172.21.15.67:0/4081134497 >> v2:172.21.15.134:3300/0 conn(0x7f743c0c9380 0x7f743c0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.793+0000 7f7442400640 1 -- 172.21.15.67:0/4081134497 >> v2:172.21.15.134:3300/0 conn(0x7f743c0c9380 msgr2=0x7f743c0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:28.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.793+0000 7f7442400640 1 --2- 172.21.15.67:0/4081134497 >> v2:172.21.15.134:3300/0 conn(0x7f743c0c9380 0x7f743c0acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.793+0000 7f7442400640 1 -- 172.21.15.67:0/4081134497 >> v1:172.21.15.134:6789/0 conn(0x7f743c14fd90 legacy=0x7f743c156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.793+0000 7f7442400640 1 -- 172.21.15.67:0/4081134497 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c0c6240 con 0x7f743c0ad440 2024-03-09T09:18:28.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.795+0000 7f7442400640 1 --2- 172.21.15.67:0/4081134497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f743c0ad440 0x7f743c06c1d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f743c0ae040 tx=0x7f743402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18716c801b4f37b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.796+0000 7f743be00640 1 -- 172.21.15.67:0/4081134497 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743402c9f0 con 0x7f743c0ad440 2024-03-09T09:18:28.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.796+0000 7f743be00640 1 -- 172.21.15.67:0/4081134497 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743402cb90 con 0x7f743c0ad440 2024-03-09T09:18:28.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.796+0000 7f743be00640 1 -- 172.21.15.67:0/4081134497 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340343f0 con 0x7f743c0ad440 2024-03-09T09:18:28.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.796+0000 7f7442e00640 1 -- 172.21.15.67:0/4081134497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f743c0ad440 msgr2=0x7f743c06c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.796+0000 7f7442e00640 1 --2- 172.21.15.67:0/4081134497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f743c0ad440 0x7f743c06c1d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f743c0ae040 tx=0x7f743402be50 comp rx=0 tx=0).stop 2024-03-09T09:18:28.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.797+0000 7f7442e00640 1 -- 172.21.15.67:0/4081134497 shutdown_connections 2024-03-09T09:18:28.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.797+0000 7f7442e00640 1 --2- 172.21.15.67:0/4081134497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f743c0ad440 0x7f743c06c1d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.797+0000 7f7442e00640 1 --2- 172.21.15.67:0/4081134497 >> v2:172.21.15.134:3300/0 conn(0x7f743c0c9380 0x7f743c0acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.797+0000 7f7442e00640 1 -- 172.21.15.67:0/4081134497 >> 172.21.15.67:0/4081134497 conn(0x7f743c0cb480 msgr2=0x7f743c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:28.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.797+0000 7f7442e00640 1 -- 172.21.15.67:0/4081134497 shutdown_connections 2024-03-09T09:18:28.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.797+0000 7f7442e00640 1 -- 172.21.15.67:0/4081134497 wait complete. 2024-03-09T09:18:28.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.798+0000 7f7442e00640 1 Processor -- start 2024-03-09T09:18:28.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.799+0000 7f7442e00640 1 -- start start 2024-03-09T09:18:28.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.799+0000 7f7442e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f743c0ad440 0x7f743c149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.799+0000 7f7442e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c0c9380 0x7f743c1eea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.799+0000 7f7442e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f743c14fd90 0x7f743c1eef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.799+0000 7f7442e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c06c710 con 0x7f743c14fd90 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.799+0000 7f7442e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c06bcb0 con 0x7f743c0ad440 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.799+0000 7f7442e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c06c890 con 0x7f743c0c9380 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.800+0000 7f7442400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f743c14fd90 0x7f743c1eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.800+0000 7f7442400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f743c14fd90 0x7f743c1eef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55802/0 (socket says 172.21.15.67:55802) 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.800+0000 7f7442400640 1 -- 172.21.15.67:0/1035317531 learned_addr learned my addr 172.21.15.67:0/1035317531 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.800+0000 7f7441a00640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f743c0ad440 0x7f743c149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.800+0000 7f7441000640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c0c9380 0x7f743c1eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.800+0000 7f7442400640 1 -- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c0c9380 msgr2=0x7f743c1eea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.800+0000 7f7442400640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c0c9380 0x7f743c1eea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.800+0000 7f7442400640 1 -- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f743c0ad440 msgr2=0x7f743c149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.800+0000 7f7442400640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f743c0ad440 0x7f743c149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.800+0000 7f7442400640 1 -- 172.21.15.67:0/1035317531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c149b30 con 0x7f743c14fd90 2024-03-09T09:18:28.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.801+0000 7f7442400640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f743c14fd90 0x7f743c1eef70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7434036040 tx=0x7f7434004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.801+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434044020 con 0x7f743c14fd90 2024-03-09T09:18:28.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.801+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7434040470 con 0x7f743c14fd90 2024-03-09T09:18:28.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.801+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434040860 con 0x7f743c14fd90 2024-03-09T09:18:28.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.801+0000 7f7442e00640 1 -- 172.21.15.67:0/1035317531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c159c00 con 0x7f743c14fd90 2024-03-09T09:18:28.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.801+0000 7f7442e00640 1 -- 172.21.15.67:0/1035317531 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c149e10 con 0x7f743c14fd90 2024-03-09T09:18:28.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.803+0000 7f7442e00640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c14a080 0x7f743c150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.803+0000 7f7442e00640 1 -- 172.21.15.67:0/1035317531 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c159020 con 0x7f743c14a080 2024-03-09T09:18:28.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.803+0000 7f7441a00640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c14a080 0x7f743c150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.804+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7434047020 con 0x7f743c14fd90 2024-03-09T09:18:28.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.804+0000 7f7441a00640 1 -- 172.21.15.67:0/1035317531 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f743c1ef4b0 con 0x7f743c14a080 2024-03-09T09:18:28.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.804+0000 7f7441a00640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c14a080 0x7f743c150f10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f743000f470 tx=0x7f743000f930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.805+0000 7f7439600640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f741c06b4d0 0x7f741c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.805+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f74340acee0 con 0x7f743c14fd90 2024-03-09T09:18:28.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.805+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430019070 con 0x7f743c14a080 2024-03-09T09:18:28.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.805+0000 7f7441000640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f741c06b4d0 0x7f741c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.806+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f7430005320 con 0x7f743c14a080 2024-03-09T09:18:28.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.806+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c14a080 msgr2=0x7f743c150f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.806+0000 7f7439600640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c14a080 0x7f743c150f10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f743000f470 tx=0x7f743000f930 comp rx=0 tx=0).stop 2024-03-09T09:18:28.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.806+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c14a080 msgr2=0x7f743c150f10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:28.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.806+0000 7f7439600640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c14a080 0x7f743c150f10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.807+0000 7f7441000640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f741c06b4d0 0x7f741c06d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f743c0c5780 tx=0x7f7428009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.814+0000 7f7442e00640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c151450 0x7f743c151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:28.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.814+0000 7f7442e00640 1 -- 172.21.15.67:0/1035317531 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c04f3c0 con 0x7f743c151450 2024-03-09T09:18:28.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.814+0000 7f7441a00640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c151450 0x7f743c151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:28.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.815+0000 7f7441a00640 1 -- 172.21.15.67:0/1035317531 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f743c06d2d0 con 0x7f743c151450 2024-03-09T09:18:28.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.816+0000 7f7441a00640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c151450 0x7f743c151860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7430000fb0 tx=0x7f7430000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:28.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.816+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430019070 con 0x7f743c151450 2024-03-09T09:18:28.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f7430006da0 con 0x7f743c151450 2024-03-09T09:18:28.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c151450 msgr2=0x7f743c151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7439600640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c151450 0x7f743c151860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7430000fb0 tx=0x7f7430000ed0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7439600640 1 -- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c151450 msgr2=0x7f743c151860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:28.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7439600640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f743c151450 0x7f743c151860 secure :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7430000fb0 tx=0x7f7430000ed0 comp rx=0 tx=0).stop 2024-03-09T09:18:28.823 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:18:28.823 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:18:28.823 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 194, 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:28.824 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:28.825 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.826 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:18:28.827 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:28.828 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:28.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.831 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:28.832 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:18:28.833 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.834 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:28.835 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:18:28.835 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:28.835 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:28.835 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:28.835 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:28.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:28.835 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:28.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:28.835 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:18:28.835 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:18:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7442e00640 1 -- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f741c06b4d0 msgr2=0x7f741c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7442e00640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f741c06b4d0 0x7f741c06d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f743c0c5780 tx=0x7f7428009190 comp rx=0 tx=0).stop 2024-03-09T09:18:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7442e00640 1 -- 172.21.15.67:0/1035317531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f743c14fd90 msgr2=0x7f743c1eef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7442e00640 1 --2- 172.21.15.67:0/1035317531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f743c14fd90 0x7f743c1eef70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7434036040 tx=0x7f7434004a40 comp rx=0 tx=0).stop 2024-03-09T09:18:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7442400640 1 -- 172.21.15.67:0/1035317531 reap_dead start 2024-03-09T09:18:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7442400640 1 -- 172.21.15.67:0/1035317531 reap_dead start 2024-03-09T09:18:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7442e00640 1 -- 172.21.15.67:0/1035317531 shutdown_connections 2024-03-09T09:18:28.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7442e00640 1 -- 172.21.15.67:0/1035317531 >> 172.21.15.67:0/1035317531 conn(0x7f743c0cb480 msgr2=0x7f743c154cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.817+0000 7f7442e00640 1 -- 172.21.15.67:0/1035317531 shutdown_connections 2024-03-09T09:18:28.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:28.818+0000 7f7442e00640 1 -- 172.21.15.67:0/1035317531 wait complete. 2024-03-09T09:18:28.839 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:18:28.839 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:18:28.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:18:29.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.020+0000 7ff0ff200640 1 Processor -- start 2024-03-09T09:18:29.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.020+0000 7ff0ff200640 1 -- start start 2024-03-09T09:18:29.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.022+0000 7ff0ff200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff100153300 0x7ff1000b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:29.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.022+0000 7ff0ff200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff1000c5e40 0x7ff1000b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:29.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.022+0000 7ff0ff200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff100157000 con 0x7ff1000cadb0 2024-03-09T09:18:29.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.022+0000 7ff0ff200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1000ae430 con 0x7ff1000c5e40 2024-03-09T09:18:29.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.022+0000 7ff0ff200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000ae6f0 con 0x7ff100153300 2024-03-09T09:18:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.022+0000 7ff0fe800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff1000cadb0 0x7ff100156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56010/0 (socket says 172.21.15.67:56010) 2024-03-09T09:18:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.022+0000 7ff0fe800640 1 -- 172.21.15.67:0/4271030576 learned_addr learned my addr 172.21.15.67:0/4271030576 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:18:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fd400640 1 --2- 172.21.15.67:0/4271030576 >> v2:172.21.15.134:3300/0 conn(0x7ff1000c5e40 0x7ff1000b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fde00640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff100153300 0x7ff1000b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096515956 0 0) 0x7ff100157000 con 0x7ff1000cadb0 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e4003520 con 0x7ff1000cadb0 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1192033720 0 0) 0x7ff0e4003520 con 0x7ff1000cadb0 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff100153300 msgr2=0x7ff1000b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff100153300 0x7ff1000b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 >> v2:172.21.15.134:3300/0 conn(0x7ff1000c5e40 msgr2=0x7ff1000b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 --2- 172.21.15.67:0/4271030576 >> v2:172.21.15.134:3300/0 conn(0x7ff1000c5e40 0x7ff1000b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff100155b10 con 0x7ff1000cadb0 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7ff0f80019f0 con 0x7ff1000cadb0 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 >> v1:172.21.15.67:6789/0 conn(0x7ff1000cadb0 legacy=0x7ff100156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e4003a20 0x7ff0e4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e4004800 0x7ff0e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:29.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff100153300 0x7ff0e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:29.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000ae430 con 0x7ff0e4004800 2024-03-09T09:18:29.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000ae6f0 con 0x7ff0e4003a20 2024-03-09T09:18:29.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.023+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e400b9f0 con 0x7ff100153300 2024-03-09T09:18:29.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.024+0000 7ff0fd400640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e4004800 0x7ff0e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:29.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.024+0000 7ff0fde00640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e4003a20 0x7ff0e4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:29.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.024+0000 7ff0fe800640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff100153300 0x7ff0e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.024+0000 7ff0fd400640 1 -- 172.21.15.67:0/4271030576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff100153300 msgr2=0x7ff0e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.024+0000 7ff0fd400640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff100153300 0x7ff0e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.024+0000 7ff0fd400640 1 -- 172.21.15.67:0/4271030576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e4003a20 msgr2=0x7ff0e4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.024+0000 7ff0fd400640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e4003a20 0x7ff0e4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.024+0000 7ff0fd400640 1 -- 172.21.15.67:0/4271030576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e4003520 con 0x7ff0e4004800 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.025+0000 7ff0fd400640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e4004800 0x7ff0e4004c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009ac0 tx=0x7ff0e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf97c4ad2b86c66f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.026+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec039070 con 0x7ff0e4004800 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.026+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec02c710 con 0x7ff0e4004800 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.026+0000 7ff0fca00640 1 -- 172.21.15.67:0/4271030576 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec009690 con 0x7ff0e4004800 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.026+0000 7ff0ff200640 1 -- 172.21.15.67:0/4271030576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e4004800 msgr2=0x7ff0e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.026+0000 7ff0ff200640 1 --2- 172.21.15.67:0/4271030576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e4004800 0x7ff0e4004c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009ac0 tx=0x7ff0e4005830 comp rx=0 tx=0).stop 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.026+0000 7ff0fe800640 1 -- 172.21.15.67:0/4271030576 reap_dead start 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.026+0000 7ff0ff200640 1 -- 172.21.15.67:0/4271030576 shutdown_connections 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.026+0000 7ff0ff200640 1 -- 172.21.15.67:0/4271030576 >> 172.21.15.67:0/4271030576 conn(0x7ff1000732d0 msgr2=0x7ff1000ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.027+0000 7ff0ff200640 1 -- 172.21.15.67:0/4271030576 shutdown_connections 2024-03-09T09:18:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.027+0000 7ff0ff200640 1 -- 172.21.15.67:0/4271030576 wait complete. 2024-03-09T09:18:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.028+0000 7ff0ff200640 1 Processor -- start 2024-03-09T09:18:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.028+0000 7ff0ff200640 1 -- start start 2024-03-09T09:18:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.029+0000 7ff0ff200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4004800 0x7ff1000cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.029+0000 7ff0ff200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff10014e8d0 0x7ff1000c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.029+0000 7ff0ff200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1000c7b30 0x7ff1000c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.029+0000 7ff0ff200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000c59e0 con 0x7ff10014e8d0 2024-03-09T09:18:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.029+0000 7ff0ff200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff100159e90 con 0x7ff1000c7b30 2024-03-09T09:18:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.029+0000 7ff0ff200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000b0000 con 0x7ff0e4004800 2024-03-09T09:18:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff10014e8d0 0x7ff1000c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff10014e8d0 0x7ff1000c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56426/0 (socket says 172.21.15.67:56426) 2024-03-09T09:18:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fd400640 1 -- 172.21.15.67:0/464935338 learned_addr learned my addr 172.21.15.67:0/464935338 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fe800640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1000c7b30 0x7ff1000c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fde00640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4004800 0x7ff1000cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fd400640 1 -- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4004800 msgr2=0x7ff1000cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fd400640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4004800 0x7ff1000cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fd400640 1 -- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1000c7b30 msgr2=0x7ff1000c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fd400640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1000c7b30 0x7ff1000c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fd400640 1 -- 172.21.15.67:0/464935338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10014e4f0 con 0x7ff10014e8d0 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fde00640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4004800 0x7ff1000cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.030+0000 7ff0fd400640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff10014e8d0 0x7ff1000c6f10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009be0 tx=0x7ff0ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.031+0000 7ff0f6000640 1 -- 172.21.15.67:0/464935338 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec042020 con 0x7ff10014e8d0 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.031+0000 7ff0f6000640 1 -- 172.21.15.67:0/464935338 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec03a040 con 0x7ff10014e8d0 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.031+0000 7ff0f6000640 1 -- 172.21.15.67:0/464935338 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec039070 con 0x7ff10014e8d0 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.031+0000 7ff0ff200640 1 -- 172.21.15.67:0/464935338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10014e210 con 0x7ff10014e8d0 2024-03-09T09:18:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.031+0000 7ff0ff200640 1 -- 172.21.15.67:0/464935338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1001ef820 con 0x7ff10014e8d0 2024-03-09T09:18:29.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.032+0000 7ff0ff200640 1 -- 172.21.15.67:0/464935338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10004f3c0 con 0x7ff10014e8d0 2024-03-09T09:18:29.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.033+0000 7ff0f6000640 1 -- 172.21.15.67:0/464935338 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff0ec045020 con 0x7ff10014e8d0 2024-03-09T09:18:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.034+0000 7ff0f6000640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0e006b500 0x7ff0e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.034+0000 7ff0f6000640 1 -- 172.21.15.67:0/464935338 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff0ec0aba80 con 0x7ff10014e8d0 2024-03-09T09:18:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.034+0000 7ff0fde00640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0e006b500 0x7ff0e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:29.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.036+0000 7ff0fde00640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0e006b500 0x7ff0e006d9c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff1000c9d40 tx=0x7ff0f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:29.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.043+0000 7ff0f6000640 1 -- 172.21.15.67:0/464935338 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff0ec0ade20 con 0x7ff10014e8d0 2024-03-09T09:18:29.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.359+0000 7ff0ff200640 1 -- 172.21.15.67:0/464935338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7ff1000cab90 con 0x7ff10014e8d0 2024-03-09T09:18:29.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.360+0000 7ff0f6000640 1 -- 172.21.15.67:0/464935338 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7ff0ec04c070 con 0x7ff10014e8d0 2024-03-09T09:18:29.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.363+0000 7ff0ff200640 1 -- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0e006b500 msgr2=0x7ff0e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:29.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.363+0000 7ff0ff200640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0e006b500 0x7ff0e006d9c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff1000c9d40 tx=0x7ff0f0008040 comp rx=0 tx=0).stop 2024-03-09T09:18:29.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.363+0000 7ff0ff200640 1 -- 172.21.15.67:0/464935338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff10014e8d0 msgr2=0x7ff1000c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:29.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.363+0000 7ff0ff200640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff10014e8d0 0x7ff1000c6f10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009be0 tx=0x7ff0ec004730 comp rx=0 tx=0).stop 2024-03-09T09:18:29.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.363+0000 7ff0ff200640 1 -- 172.21.15.67:0/464935338 shutdown_connections 2024-03-09T09:18:29.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.363+0000 7ff0ff200640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0e006b500 0x7ff0e006d9c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:29.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.363+0000 7ff0ff200640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1000c7b30 0x7ff1000c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:29.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.363+0000 7ff0ff200640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff10014e8d0 0x7ff1000c6f10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:29.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.363+0000 7ff0ff200640 1 --2- 172.21.15.67:0/464935338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e4004800 0x7ff1000cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:29.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.363+0000 7ff0ff200640 1 -- 172.21.15.67:0/464935338 >> 172.21.15.67:0/464935338 conn(0x7ff1000732d0 msgr2=0x7ff10014f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:29.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.363+0000 7ff0ff200640 1 -- 172.21.15.67:0/464935338 shutdown_connections 2024-03-09T09:18:29.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:29.364+0000 7ff0ff200640 1 -- 172.21.15.67:0/464935338 wait complete. 2024-03-09T09:18:29.390 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:18:31.331 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:31.331 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:31.331 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 19 -eq 48 ']' 2024-03-09T09:18:31.331 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.19 2024-03-09T09:18:31.331 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.19 2024-03-09T09:18:31.332 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.19 2024-03-09T09:18:31.332 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.19 2024-03-09T09:18:31.336 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:33.341 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:33.341 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:33.342 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 20 -eq 48 ']' 2024-03-09T09:18:33.342 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.20 2024-03-09T09:18:33.342 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.20 2024-03-09T09:18:33.342 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.20 2024-03-09T09:18:33.342 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.20 2024-03-09T09:18:33.346 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:35.503 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:35.503 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:35.503 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 25 -eq 48 ']' 2024-03-09T09:18:35.504 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.25 2024-03-09T09:18:35.504 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.25 2024-03-09T09:18:35.504 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.25 2024-03-09T09:18:35.504 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.25 2024-03-09T09:18:35.508 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:37.722 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:37.722 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:37.722 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 21 -eq 48 ']' 2024-03-09T09:18:37.723 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.21 2024-03-09T09:18:37.723 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.21 2024-03-09T09:18:37.723 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.21 2024-03-09T09:18:37.723 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.21 2024-03-09T09:18:37.725 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:38.577 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:38.577 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:38.578 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 26 -eq 48 ']' 2024-03-09T09:18:38.578 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.26 2024-03-09T09:18:38.578 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.26 2024-03-09T09:18:38.578 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.26 2024-03-09T09:18:38.578 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.26 2024-03-09T09:18:38.582 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:39.391 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:18:39.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:18:39.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.613+0000 7f5282000640 1 Processor -- start 2024-03-09T09:18:39.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.613+0000 7f5282000640 1 -- start start 2024-03-09T09:18:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.615+0000 7f5282000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f527c0cae50 0x7f527c0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.615+0000 7f5282000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527c156700 con 0x7f527c1570d0 2024-03-09T09:18:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.615+0000 7f5282000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527c157530 con 0x7f527c1531c0 2024-03-09T09:18:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.615+0000 7f5282000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527c0afba0 con 0x7f527c0cae50 2024-03-09T09:18:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.615+0000 7f527be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f527c1570d0 0x7f527c0c9440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44904/0 (socket says 172.21.15.67:44904) 2024-03-09T09:18:39.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.615+0000 7f527be00640 1 -- 172.21.15.67:0/1121348282 learned_addr learned my addr 172.21.15.67:0/1121348282 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.615+0000 7f5281600640 1 --2- 172.21.15.67:0/1121348282 >> v2:172.21.15.134:3300/0 conn(0x7f527c0cae50 0x7f527c0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267988692 0 0) 0x7f527c156700 con 0x7f527c1570d0 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5264003520 con 0x7f527c1570d0 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3114297920 0 0) 0x7f5264003520 con 0x7f527c1570d0 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 >> v2:172.21.15.134:3300/0 conn(0x7f527c0cae50 msgr2=0x7f527c0c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 --2- 172.21.15.67:0/1121348282 >> v2:172.21.15.134:3300/0 conn(0x7f527c0cae50 0x7f527c0c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 >> v1:172.21.15.134:6789/0 conn(0x7f527c1531c0 legacy=0x7f527c0ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c1569a0 con 0x7f527c1570d0 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f526c001670 con 0x7f527c1570d0 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 >> v1:172.21.15.67:6789/0 conn(0x7f527c1570d0 legacy=0x7f527c0c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 --2- 172.21.15.67:0/1121348282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5264003a20 0x7f5264003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:39.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 --2- 172.21.15.67:0/1121348282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5264004800 0x7f5264004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 --2- 172.21.15.67:0/1121348282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5264005230 0x7f526400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526400c160 con 0x7f5264005230 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526400c310 con 0x7f5264003a20 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.616+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526400c510 con 0x7f5264004800 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.617+0000 7f5281600640 1 --2- 172.21.15.67:0/1121348282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5264005230 0x7f526400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.617+0000 7f5280c00640 1 --2- 172.21.15.67:0/1121348282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5264003a20 0x7f5264003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.617+0000 7f527be00640 1 --2- 172.21.15.67:0/1121348282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5264004800 0x7f5264004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.617+0000 7f5281600640 1 -- 172.21.15.67:0/1121348282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5264004800 msgr2=0x7f5264004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.617+0000 7f5281600640 1 --2- 172.21.15.67:0/1121348282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5264004800 0x7f5264004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.617+0000 7f5281600640 1 -- 172.21.15.67:0/1121348282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5264003a20 msgr2=0x7f5264003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.617+0000 7f5281600640 1 --2- 172.21.15.67:0/1121348282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5264003a20 0x7f5264003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:39.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.617+0000 7f5281600640 1 -- 172.21.15.67:0/1121348282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5264003520 con 0x7f5264005230 2024-03-09T09:18:39.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.619+0000 7f5281600640 1 -- 172.21.15.67:0/1121348282 reap_dead start 2024-03-09T09:18:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.619+0000 7f5281600640 1 --2- 172.21.15.67:0/1121348282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5264005230 0x7f526400b9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5274009fc0 tx=0x7f5264005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f6e348bbae81d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.620+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274009c10 con 0x7f5264005230 2024-03-09T09:18:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.620+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5274003bd0 con 0x7f5264005230 2024-03-09T09:18:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.620+0000 7f527b400640 1 -- 172.21.15.67:0/1121348282 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527402c990 con 0x7f5264005230 2024-03-09T09:18:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.620+0000 7f5282000640 1 -- 172.21.15.67:0/1121348282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5264005230 msgr2=0x7f526400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.620+0000 7f5282000640 1 --2- 172.21.15.67:0/1121348282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5264005230 0x7f526400b9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5274009fc0 tx=0x7f5264005830 comp rx=0 tx=0).stop 2024-03-09T09:18:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.620+0000 7f5282000640 1 -- 172.21.15.67:0/1121348282 shutdown_connections 2024-03-09T09:18:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.620+0000 7f5282000640 1 --2- 172.21.15.67:0/1121348282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5264005230 0x7f526400b9c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.620+0000 7f5282000640 1 -- 172.21.15.67:0/1121348282 >> 172.21.15.67:0/1121348282 conn(0x7f527c0ad190 msgr2=0x7f527c0ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.620+0000 7f5282000640 1 -- 172.21.15.67:0/1121348282 shutdown_connections 2024-03-09T09:18:39.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.620+0000 7f5282000640 1 -- 172.21.15.67:0/1121348282 wait complete. 2024-03-09T09:18:39.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.623+0000 7f5282000640 1 Processor -- start 2024-03-09T09:18:39.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.623+0000 7f5282000640 1 -- start start 2024-03-09T09:18:39.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.624+0000 7f5282000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5264005230 0x7f527c0c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:39.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.624+0000 7f5282000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c06bba0 0x7f527c06b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:39.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.624+0000 7f5282000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f527c06b580 0x7f527c1eb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:39.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.624+0000 7f5282000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0b06f0 con 0x7f527c06bba0 2024-03-09T09:18:39.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.624+0000 7f5282000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0b01c0 con 0x7f527c06b580 2024-03-09T09:18:39.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.624+0000 7f5282000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c1535a0 con 0x7f5264005230 2024-03-09T09:18:39.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.624+0000 7f527be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c06bba0 0x7f527c06b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:39.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.624+0000 7f527be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c06bba0 0x7f527c06b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55456/0 (socket says 172.21.15.67:55456) 2024-03-09T09:18:39.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.624+0000 7f527be00640 1 -- 172.21.15.67:0/194535803 learned_addr learned my addr 172.21.15.67:0/194535803 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:39.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.624+0000 7f5280c00640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5264005230 0x7f527c0c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:39.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f5281600640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f527c06b580 0x7f527c1eb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:39.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f527be00640 1 -- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5264005230 msgr2=0x7f527c0c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:39.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f527be00640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5264005230 0x7f527c0c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:39.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f527be00640 1 -- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f527c06b580 msgr2=0x7f527c1eb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:39.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f527be00640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f527c06b580 0x7f527c1eb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:39.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f527be00640 1 -- 172.21.15.67:0/194535803 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c0c50e0 con 0x7f527c06bba0 2024-03-09T09:18:39.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f5281600640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f527c06b580 0x7f527c1eb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:18:39.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f527be00640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c06bba0 0x7f527c06b040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f526c00e7b0 tx=0x7f526c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:39.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f5278c00640 1 -- 172.21.15.67:0/194535803 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c00ee60 con 0x7f527c06bba0 2024-03-09T09:18:39.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f5278c00640 1 -- 172.21.15.67:0/194535803 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c018070 con 0x7f527c06bba0 2024-03-09T09:18:39.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f5278c00640 1 -- 172.21.15.67:0/194535803 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c00cb10 con 0x7f527c06bba0 2024-03-09T09:18:39.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.625+0000 7f5282000640 1 -- 172.21.15.67:0/194535803 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c1ee520 con 0x7f527c06bba0 2024-03-09T09:18:39.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.626+0000 7f5282000640 1 -- 172.21.15.67:0/194535803 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c1eeae0 con 0x7f527c06bba0 2024-03-09T09:18:39.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.627+0000 7f5282000640 1 -- 172.21.15.67:0/194535803 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527c04f3c0 con 0x7f527c06bba0 2024-03-09T09:18:39.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.628+0000 7f5278c00640 1 -- 172.21.15.67:0/194535803 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f526c00ccb0 con 0x7f527c06bba0 2024-03-09T09:18:39.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.629+0000 7f5278c00640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f526006b4d0 0x7f526006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:39.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.629+0000 7f5278c00640 1 -- 172.21.15.67:0/194535803 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f526c08a390 con 0x7f527c06bba0 2024-03-09T09:18:39.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.630+0000 7f5280c00640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f526006b4d0 0x7f526006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:39.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.631+0000 7f5280c00640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f526006b4d0 0x7f526006d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5270000c00 tx=0x7f5270009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:39.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.635+0000 7f5278c00640 1 -- 172.21.15.67:0/194535803 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f526c058eb0 con 0x7f527c06bba0 2024-03-09T09:18:39.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.921+0000 7f5282000640 1 -- 172.21.15.67:0/194535803 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f527c159f20 con 0x7f527c06bba0 2024-03-09T09:18:39.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.922+0000 7f5278c00640 1 -- 172.21.15.67:0/194535803 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f526c0587b0 con 0x7f527c06bba0 2024-03-09T09:18:39.929 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":194,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:18:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 -- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f526006b4d0 msgr2=0x7f526006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f526006b4d0 0x7f526006d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5270000c00 tx=0x7f5270009040 comp rx=0 tx=0).stop 2024-03-09T09:18:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 -- 172.21.15.67:0/194535803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c06bba0 msgr2=0x7f527c06b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c06bba0 0x7f527c06b040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f526c00e7b0 tx=0x7f526c00ec70 comp rx=0 tx=0).stop 2024-03-09T09:18:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 -- 172.21.15.67:0/194535803 shutdown_connections 2024-03-09T09:18:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f526006b4d0 0x7f526006d990 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f527c06b580 0x7f527c1eb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c06bba0 0x7f527c06b040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 --2- 172.21.15.67:0/194535803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5264005230 0x7f527c0c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 -- 172.21.15.67:0/194535803 >> 172.21.15.67:0/194535803 conn(0x7f527c0ad190 msgr2=0x7f527c153d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 -- 172.21.15.67:0/194535803 shutdown_connections 2024-03-09T09:18:39.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:39.926+0000 7f5282000640 1 -- 172.21.15.67:0/194535803 wait complete. 2024-03-09T09:18:39.952 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:18:39.952 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:18:39.952 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:18:39.953 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:18:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.145+0000 7f331b200640 1 Processor -- start 2024-03-09T09:18:40.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.145+0000 7f331b200640 1 -- start start 2024-03-09T09:18:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.146+0000 7f331b200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f331406e830 0x7f33140c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f331b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f33141536c0 0x7f33140c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f331b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33140c19e0 con 0x7f33140c1030 2024-03-09T09:18:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f331b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331406ef60 con 0x7f33141536c0 2024-03-09T09:18:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f331b200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3314156790 con 0x7f331406e830 2024-03-09T09:18:40.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f331a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f33141536c0 0x7f33140c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f3319e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f33140c1030 0x7f3314152fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44908/0 (socket says 172.21.15.67:44908) 2024-03-09T09:18:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f3319e00640 1 -- 172.21.15.67:0/739179924 learned_addr learned my addr 172.21.15.67:0/739179924 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:18:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f3319400640 1 --2- 172.21.15.67:0/739179924 >> v2:172.21.15.134:3300/0 conn(0x7f331406e830 0x7f33140c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f331a800640 1 -- 172.21.15.67:0/739179924 >> v2:172.21.15.134:3300/0 conn(0x7f331406e830 msgr2=0x7f33140c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f331a800640 1 --2- 172.21.15.67:0/739179924 >> v2:172.21.15.134:3300/0 conn(0x7f331406e830 0x7f33140c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f331a800640 1 -- 172.21.15.67:0/739179924 >> v1:172.21.15.67:6789/0 conn(0x7f33140c1030 legacy=0x7f3314152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f331a800640 1 -- 172.21.15.67:0/739179924 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33141569e0 con 0x7f33141536c0 2024-03-09T09:18:40.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.147+0000 7f3319400640 1 --2- 172.21.15.67:0/739179924 >> v2:172.21.15.134:3300/0 conn(0x7f331406e830 0x7f33140c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:18:40.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.148+0000 7f331a800640 1 --2- 172.21.15.67:0/739179924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33141536c0 0x7f33140c2e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f331406cad0 tx=0x7f331002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a767e0bd0c0a668a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.149+0000 7f3318a00640 1 -- 172.21.15.67:0/739179924 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310039070 con 0x7f33141536c0 2024-03-09T09:18:40.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.149+0000 7f3318a00640 1 -- 172.21.15.67:0/739179924 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331002cb60 con 0x7f33141536c0 2024-03-09T09:18:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.149+0000 7f3318a00640 1 -- 172.21.15.67:0/739179924 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310034440 con 0x7f33141536c0 2024-03-09T09:18:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.149+0000 7f331b200640 1 -- 172.21.15.67:0/739179924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33141536c0 msgr2=0x7f33140c2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.149+0000 7f331b200640 1 --2- 172.21.15.67:0/739179924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33141536c0 0x7f33140c2e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f331406cad0 tx=0x7f331002bdb0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.150+0000 7f331b200640 1 -- 172.21.15.67:0/739179924 shutdown_connections 2024-03-09T09:18:40.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.150+0000 7f331b200640 1 --2- 172.21.15.67:0/739179924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33141536c0 0x7f33140c2e50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.150+0000 7f331b200640 1 --2- 172.21.15.67:0/739179924 >> v2:172.21.15.134:3300/0 conn(0x7f331406e830 0x7f33140c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.150+0000 7f331b200640 1 -- 172.21.15.67:0/739179924 >> 172.21.15.67:0/739179924 conn(0x7f33140a9740 msgr2=0x7f33140a9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.150+0000 7f331b200640 1 -- 172.21.15.67:0/739179924 shutdown_connections 2024-03-09T09:18:40.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.150+0000 7f331b200640 1 -- 172.21.15.67:0/739179924 wait complete. 2024-03-09T09:18:40.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.151+0000 7f331b200640 1 Processor -- start 2024-03-09T09:18:40.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.151+0000 7f331b200640 1 -- start start 2024-03-09T09:18:40.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.152+0000 7f331b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33140c1030 0x7f33140aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.152+0000 7f331b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33141536c0 0x7f33140abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.152+0000 7f331b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33140a6e90 0x7f3314158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.152+0000 7f331b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33140c1e20 con 0x7f33140a6e90 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.152+0000 7f331b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3314156cc0 con 0x7f33140c1030 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.152+0000 7f331b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33140c1fa0 con 0x7f33141536c0 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.153+0000 7f331a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33140a6e90 0x7f3314158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.153+0000 7f331a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33140a6e90 0x7f3314158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55476/0 (socket says 172.21.15.67:55476) 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.153+0000 7f331a800640 1 -- 172.21.15.67:0/2839964726 learned_addr learned my addr 172.21.15.67:0/2839964726 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.153+0000 7f3319e00640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33140c1030 0x7f33140aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.153+0000 7f331a800640 1 -- 172.21.15.67:0/2839964726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33141536c0 msgr2=0x7f33140abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.153+0000 7f331a800640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33141536c0 0x7f33140abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.153+0000 7f331a800640 1 -- 172.21.15.67:0/2839964726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33140c1030 msgr2=0x7f33140aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.153+0000 7f331a800640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33140c1030 0x7f33140aba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.153+0000 7f331a800640 1 -- 172.21.15.67:0/2839964726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33141ec410 con 0x7f33140a6e90 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.153+0000 7f331a800640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33140a6e90 0x7f3314158be0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3310009e80 tx=0x7f3310004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.154+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310039050 con 0x7f33140a6e90 2024-03-09T09:18:40.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.154+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3310045070 con 0x7f33140a6e90 2024-03-09T09:18:40.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.154+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331003e830 con 0x7f33140a6e90 2024-03-09T09:18:40.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.154+0000 7f331b200640 1 -- 172.21.15.67:0/2839964726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33141576d0 con 0x7f33140a6e90 2024-03-09T09:18:40.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.154+0000 7f331b200640 1 -- 172.21.15.67:0/2839964726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33141ec690 con 0x7f33140a6e90 2024-03-09T09:18:40.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.155+0000 7f331b200640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f331406e830 0x7f33140c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.155+0000 7f331b200640 1 -- 172.21.15.67:0/2839964726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331404f3c0 con 0x7f331406e830 2024-03-09T09:18:40.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.155+0000 7f3319e00640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f331406e830 0x7f33140c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.156+0000 7f3319e00640 1 -- 172.21.15.67:0/2839964726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33140c6f30 con 0x7f331406e830 2024-03-09T09:18:40.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.156+0000 7f3319e00640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f331406e830 0x7f33140c4410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3308000f60 tx=0x7f330800e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.156+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308019070 con 0x7f331406e830 2024-03-09T09:18:40.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.156+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f331002cd10 con 0x7f33140a6e90 2024-03-09T09:18:40.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.157+0000 7f330e000640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f32f806b3d0 0x7f32f806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.157+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f33100ad120 con 0x7f33140a6e90 2024-03-09T09:18:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.158+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f3308005ac0 con 0x7f331406e830 2024-03-09T09:18:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.158+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f331406e830 msgr2=0x7f33140c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.158+0000 7f330e000640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f331406e830 0x7f33140c4410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3308000f60 tx=0x7f330800e820 comp rx=0 tx=0).stop 2024-03-09T09:18:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.158+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f331406e830 msgr2=0x7f33140c4410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.158+0000 7f330e000640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f331406e830 0x7f33140c4410 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3308000f60 tx=0x7f330800e820 comp rx=0 tx=0).stop 2024-03-09T09:18:40.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.158+0000 7f3319400640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f32f806b3d0 0x7f32f806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.159+0000 7f3319400640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f32f806b3d0 0x7f32f806d890 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3304006fd0 tx=0x7f3304008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.165+0000 7f331b200640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314157110 0x7f33141eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.165+0000 7f331b200640 1 -- 172.21.15.67:0/2839964726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33140aa260 con 0x7f3314157110 2024-03-09T09:18:40.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.165+0000 7f3319e00640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314157110 0x7f33141eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.166+0000 7f3319e00640 1 -- 172.21.15.67:0/2839964726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f33140c5da0 con 0x7f3314157110 2024-03-09T09:18:40.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.166+0000 7f3319e00640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314157110 0x7f33141eef30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f330800ed70 tx=0x7f3308000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.166+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308007620 con 0x7f3314157110 2024-03-09T09:18:40.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.166+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7f3308007620 con 0x7f3314157110 2024-03-09T09:18:40.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314157110 msgr2=0x7f33141eef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f330e000640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314157110 0x7f33141eef30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f330800ed70 tx=0x7f3308000ed0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f330e000640 1 -- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314157110 msgr2=0x7f33141eef30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:40.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f330e000640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3314157110 0x7f33141eef30 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f330800ed70 tx=0x7f3308000ed0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.173 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 194, 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:18:40.174 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.175 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:18:40.177 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:40.178 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:40.179 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:40.180 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:40.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.181 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.181 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:18:40.181 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:18:40.181 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:40.181 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:40.181 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.181 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:40.181 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:18:40.182 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.183 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.184 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:40.184 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:18:40.184 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.184 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.184 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.184 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:18:40.184 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:18:40.184 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:40.184 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:40.185 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:40.185 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.185 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.185 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:18:40.185 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:18:40.185 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.185 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:40.185 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.186 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:40.186 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.186 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.186 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:18:40.186 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.186 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:40.186 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.186 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:18:40.186 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:18:40.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.188 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.189 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:18:40.189 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:18:40.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f331b200640 1 -- 172.21.15.67:0/2839964726 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f32f806b3d0 msgr2=0x7f32f806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f331b200640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f32f806b3d0 0x7f32f806d890 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3304006fd0 tx=0x7f3304008040 comp rx=0 tx=0).stop 2024-03-09T09:18:40.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f331b200640 1 -- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33140a6e90 msgr2=0x7f3314158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f331b200640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33140a6e90 0x7f3314158be0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3310009e80 tx=0x7f3310004a40 comp rx=0 tx=0).stop 2024-03-09T09:18:40.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f331a800640 1 -- 172.21.15.67:0/2839964726 reap_dead start 2024-03-09T09:18:40.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f331b200640 1 -- 172.21.15.67:0/2839964726 shutdown_connections 2024-03-09T09:18:40.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f331b200640 1 --2- 172.21.15.67:0/2839964726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33140a6e90 0x7f3314158be0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f331b200640 1 -- 172.21.15.67:0/2839964726 >> 172.21.15.67:0/2839964726 conn(0x7f33140a9740 msgr2=0x7f3314154dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:40.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f331b200640 1 -- 172.21.15.67:0/2839964726 shutdown_connections 2024-03-09T09:18:40.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.167+0000 7f331b200640 1 -- 172.21.15.67:0/2839964726 wait complete. 2024-03-09T09:18:40.191 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:18:40.191 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:18:40.191 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:18:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.387+0000 7fdd7a000640 1 Processor -- start 2024-03-09T09:18:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.388+0000 7fdd7a000640 1 -- start start 2024-03-09T09:18:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.389+0000 7fdd7a000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fdd740730c0 0x7fdd740aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.389+0000 7fdd7a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd740734a0 0x7fdd740c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.389+0000 7fdd7a000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd74152e00 con 0x7fdd740c33c0 2024-03-09T09:18:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.389+0000 7fdd7a000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd740aa960 con 0x7fdd740734a0 2024-03-09T09:18:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.389+0000 7fdd7a000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd741530e0 con 0x7fdd740730c0 2024-03-09T09:18:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.390+0000 7fdd6be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd740734a0 0x7fdd740c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.390+0000 7fdd6be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd740734a0 0x7fdd740c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55508/0 (socket says 172.21.15.67:55508) 2024-03-09T09:18:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.390+0000 7fdd6be00640 1 -- 172.21.15.67:0/482283292 learned_addr learned my addr 172.21.15.67:0/482283292 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.390+0000 7fdd73400640 1 --2- 172.21.15.67:0/482283292 >> v2:172.21.15.134:3300/0 conn(0x7fdd740730c0 0x7fdd740aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.390+0000 7fdd6be00640 1 -- 172.21.15.67:0/482283292 >> v2:172.21.15.134:3300/0 conn(0x7fdd740730c0 msgr2=0x7fdd740aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.390+0000 7fdd6be00640 1 --2- 172.21.15.67:0/482283292 >> v2:172.21.15.134:3300/0 conn(0x7fdd740730c0 0x7fdd740aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.390+0000 7fdd6be00640 1 -- 172.21.15.67:0/482283292 >> v1:172.21.15.134:6789/0 conn(0x7fdd740c33c0 legacy=0x7fdd7406d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.390+0000 7fdd6be00640 1 -- 172.21.15.67:0/482283292 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd741533c0 con 0x7fdd740734a0 2024-03-09T09:18:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.392+0000 7fdd6be00640 1 --2- 172.21.15.67:0/482283292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd740734a0 0x7fdd740c2e80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd7406c090 tx=0x7fdd5c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d60379dba7ceca server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.392+0000 7fdd72a00640 1 -- 172.21.15.67:0/482283292 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c02e070 con 0x7fdd740734a0 2024-03-09T09:18:40.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.392+0000 7fdd72a00640 1 -- 172.21.15.67:0/482283292 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c037070 con 0x7fdd740734a0 2024-03-09T09:18:40.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.392+0000 7fdd72a00640 1 -- 172.21.15.67:0/482283292 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c031a00 con 0x7fdd740734a0 2024-03-09T09:18:40.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.393+0000 7fdd7a000640 1 -- 172.21.15.67:0/482283292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd740734a0 msgr2=0x7fdd740c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.393+0000 7fdd7a000640 1 --2- 172.21.15.67:0/482283292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd740734a0 0x7fdd740c2e80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd7406c090 tx=0x7fdd5c02b880 comp rx=0 tx=0).stop 2024-03-09T09:18:40.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.393+0000 7fdd7a000640 1 -- 172.21.15.67:0/482283292 shutdown_connections 2024-03-09T09:18:40.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.393+0000 7fdd7a000640 1 --2- 172.21.15.67:0/482283292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd740734a0 0x7fdd740c2e80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.393+0000 7fdd7a000640 1 --2- 172.21.15.67:0/482283292 >> v2:172.21.15.134:3300/0 conn(0x7fdd740730c0 0x7fdd740aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.393+0000 7fdd7a000640 1 -- 172.21.15.67:0/482283292 >> 172.21.15.67:0/482283292 conn(0x7fdd740c7370 msgr2=0x7fdd740c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:40.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.393+0000 7fdd7a000640 1 -- 172.21.15.67:0/482283292 shutdown_connections 2024-03-09T09:18:40.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.393+0000 7fdd7a000640 1 -- 172.21.15.67:0/482283292 wait complete. 2024-03-09T09:18:40.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.395+0000 7fdd7a000640 1 Processor -- start 2024-03-09T09:18:40.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.396+0000 7fdd7a000640 1 -- start start 2024-03-09T09:18:40.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.397+0000 7fdd7a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd740730c0 0x7fdd74155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.397+0000 7fdd7a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd740734a0 0x7fdd741ebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.397+0000 7fdd7a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd740c33c0 0x7fdd741efa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.397+0000 7fdd7a000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740aaef0 con 0x7fdd740734a0 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.397+0000 7fdd7a000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd74153980 con 0x7fdd740c33c0 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.397+0000 7fdd7a000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740ab070 con 0x7fdd740730c0 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.397+0000 7fdd6be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd740734a0 0x7fdd741ebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.397+0000 7fdd6be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd740734a0 0x7fdd741ebcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55510/0 (socket says 172.21.15.67:55510) 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.397+0000 7fdd6be00640 1 -- 172.21.15.67:0/3618269590 learned_addr learned my addr 172.21.15.67:0/3618269590 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.398+0000 7fdd6be00640 1 -- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd740730c0 msgr2=0x7fdd74155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.398+0000 7fdd6be00640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd740730c0 0x7fdd74155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.398+0000 7fdd6be00640 1 -- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd740c33c0 msgr2=0x7fdd741efa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.398+0000 7fdd6be00640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd740c33c0 0x7fdd741efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.398+0000 7fdd6be00640 1 -- 172.21.15.67:0/3618269590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd741560c0 con 0x7fdd740734a0 2024-03-09T09:18:40.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.398+0000 7fdd6be00640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd740734a0 0x7fdd741ebcd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c02b8c0 tx=0x7fdd5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.398+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c032880 con 0x7fdd740734a0 2024-03-09T09:18:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.398+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c032a20 con 0x7fdd740734a0 2024-03-09T09:18:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.398+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c032e10 con 0x7fdd740734a0 2024-03-09T09:18:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.398+0000 7fdd7a000640 1 -- 172.21.15.67:0/3618269590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd741563a0 con 0x7fdd740734a0 2024-03-09T09:18:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.399+0000 7fdd7a000640 1 -- 172.21.15.67:0/3618269590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd741ef240 con 0x7fdd740734a0 2024-03-09T09:18:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.399+0000 7fdd7a000640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd741ef480 0x7fdd74153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.399+0000 7fdd7a000640 1 -- 172.21.15.67:0/3618269590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd74075a00 con 0x7fdd741ef480 2024-03-09T09:18:40.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.400+0000 7fdd73400640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd741ef480 0x7fdd74153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.400+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdd5c0453f0 con 0x7fdd740734a0 2024-03-09T09:18:40.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.401+0000 7fdd73400640 1 -- 172.21.15.67:0/3618269590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd74156ee0 con 0x7fdd741ef480 2024-03-09T09:18:40.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.401+0000 7fdd6b400640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd5006b4d0 0x7fdd5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.401+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdd5c0ac6d0 con 0x7fdd740734a0 2024-03-09T09:18:40.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.401+0000 7fdd73400640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd741ef480 0x7fdd74153e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d4e0 tx=0x7fdd6400d9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.402+0000 7fdd73e00640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd5006b4d0 0x7fdd5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.402+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd640125f0 con 0x7fdd741ef480 2024-03-09T09:18:40.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.403+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fdd64019070 con 0x7fdd741ef480 2024-03-09T09:18:40.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.403+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd741ef480 msgr2=0x7fdd74153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.403+0000 7fdd6b400640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd741ef480 0x7fdd74153e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d4e0 tx=0x7fdd6400d9a0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.403+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd741ef480 msgr2=0x7fdd74153e30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:40.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.403+0000 7fdd6b400640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd741ef480 0x7fdd74153e30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.403+0000 7fdd73e00640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd5006b4d0 0x7fdd5006d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00ab10 tx=0x7fdd6c009230 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.410+0000 7fdd7a000640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd74154370 0x7fdd7414ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.410+0000 7fdd7a000640 1 -- 172.21.15.67:0/3618269590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740abe10 con 0x7fdd74154370 2024-03-09T09:18:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.410+0000 7fdd73400640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd74154370 0x7fdd7414ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.411+0000 7fdd73400640 1 -- 172.21.15.67:0/3618269590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdd740c1a60 con 0x7fdd74154370 2024-03-09T09:18:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.412+0000 7fdd73400640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd74154370 0x7fdd7414ef00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d4b0 tx=0x7fdd640046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.412+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64012bf0 con 0x7fdd74154370 2024-03-09T09:18:40.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.412+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4875 (secure 0 0 0) 0x7fdd64012bf0 con 0x7fdd74154370 2024-03-09T09:18:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd74154370 msgr2=0x7fdd7414ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd6b400640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd74154370 0x7fdd7414ef00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d4b0 tx=0x7fdd640046c0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.419 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:18:40.419 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:18:40.419 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:18:40.419 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:18:40.419 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 194, 2024-03-09T09:18:40.419 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:40.420 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:40.421 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:18:40.422 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:40.423 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.424 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:18:40.425 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:40.426 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:18:40.427 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.429 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.430 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.431 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:40.432 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.432 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.432 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:18:40.432 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:18:40.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd6b400640 1 -- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd74154370 msgr2=0x7fdd7414ef00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:40.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd6b400640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd74154370 0x7fdd7414ef00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd7a000640 1 -- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd5006b4d0 msgr2=0x7fdd5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd7a000640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd5006b4d0 0x7fdd5006d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00ab10 tx=0x7fdd6c009230 comp rx=0 tx=0).stop 2024-03-09T09:18:40.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd7a000640 1 -- 172.21.15.67:0/3618269590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd740734a0 msgr2=0x7fdd741ebcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd7a000640 1 --2- 172.21.15.67:0/3618269590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd740734a0 0x7fdd741ebcd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c02b8c0 tx=0x7fdd5c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd73e00640 1 -- 172.21.15.67:0/3618269590 reap_dead start 2024-03-09T09:18:40.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd73e00640 1 -- 172.21.15.67:0/3618269590 reap_dead start 2024-03-09T09:18:40.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd7a000640 1 -- 172.21.15.67:0/3618269590 shutdown_connections 2024-03-09T09:18:40.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd7a000640 1 -- 172.21.15.67:0/3618269590 >> 172.21.15.67:0/3618269590 conn(0x7fdd740c7370 msgr2=0x7fdd7414a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:40.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd7a000640 1 -- 172.21.15.67:0/3618269590 shutdown_connections 2024-03-09T09:18:40.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.413+0000 7fdd7a000640 1 -- 172.21.15.67:0/3618269590 wait complete. 2024-03-09T09:18:40.435 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:18:40.435 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:18:40.436 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:18:40.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.632+0000 7f0795e00640 1 Processor -- start 2024-03-09T09:18:40.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.632+0000 7f0795e00640 1 -- start start 2024-03-09T09:18:40.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.634+0000 7f0795e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790153240 0x7f0790159d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.634+0000 7f0795e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0790071ae0 0x7f0790155d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.634+0000 7f0795e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0790156800 con 0x7f0790071660 2024-03-09T09:18:40.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.634+0000 7f0795e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0790156ae0 con 0x7f0790071ae0 2024-03-09T09:18:40.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.634+0000 7f0795e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07901598b0 con 0x7f0790153240 2024-03-09T09:18:40.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.634+0000 7f078fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0790071660 0x7f079006b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44922/0 (socket says 172.21.15.67:44922) 2024-03-09T09:18:40.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.634+0000 7f078fe00640 1 -- 172.21.15.67:0/3203132379 learned_addr learned my addr 172.21.15.67:0/3203132379 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:18:40.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f0794a00640 1 --2- 172.21.15.67:0/3203132379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790153240 0x7f0790159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828969965 0 0) 0x7f0790156800 con 0x7f0790071660 2024-03-09T09:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0778003520 con 0x7f0790071660 2024-03-09T09:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f0795400640 1 --2- 172.21.15.67:0/3203132379 >> v2:172.21.15.134:3300/0 conn(0x7f0790071ae0 0x7f0790155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2759720218 0 0) 0x7f0778003520 con 0x7f0790071660 2024-03-09T09:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790153240 msgr2=0x7f0790159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f078f400640 1 --2- 172.21.15.67:0/3203132379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790153240 0x7f0790159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 >> v2:172.21.15.134:3300/0 conn(0x7f0790071ae0 msgr2=0x7f0790155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f078f400640 1 --2- 172.21.15.67:0/3203132379 >> v2:172.21.15.134:3300/0 conn(0x7f0790071ae0 0x7f0790155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0790159260 con 0x7f0790071660 2024-03-09T09:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f077c001670 con 0x7f0790071660 2024-03-09T09:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.635+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 >> v1:172.21.15.67:6789/0 conn(0x7f0790071660 legacy=0x7f079006b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078f400640 1 --2- 172.21.15.67:0/3203132379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790153240 0x7f0778003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078f400640 1 --2- 172.21.15.67:0/3203132379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0778003ff0 0x7f0778004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078f400640 1 --2- 172.21.15.67:0/3203132379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0778004d40 0x7f07780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790156ae0 con 0x7f0778003ff0 2024-03-09T09:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901598b0 con 0x7f0778004d40 2024-03-09T09:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077800bc30 con 0x7f0790153240 2024-03-09T09:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078fe00640 1 --2- 172.21.15.67:0/3203132379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0778003ff0 0x7f0778004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f0795400640 1 --2- 172.21.15.67:0/3203132379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0778004d40 0x7f07780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078fe00640 1 -- 172.21.15.67:0/3203132379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790153240 msgr2=0x7f0778003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078fe00640 1 --2- 172.21.15.67:0/3203132379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790153240 0x7f0778003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078fe00640 1 -- 172.21.15.67:0/3203132379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0778004d40 msgr2=0x7f07780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078fe00640 1 --2- 172.21.15.67:0/3203132379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0778004d40 0x7f07780051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.636+0000 7f078fe00640 1 -- 172.21.15.67:0/3203132379 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778003520 con 0x7f0778003ff0 2024-03-09T09:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.638+0000 7f078fe00640 1 --2- 172.21.15.67:0/3203132379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0778003ff0 0x7f0778004800 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f077c001050 tx=0x7f0778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa1dd1466ecb894 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.638+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c007ca0 con 0x7f0778003ff0 2024-03-09T09:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.638+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c030070 con 0x7f0778003ff0 2024-03-09T09:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.638+0000 7f078f400640 1 -- 172.21.15.67:0/3203132379 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c005d10 con 0x7f0778003ff0 2024-03-09T09:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.639+0000 7f0795e00640 1 -- 172.21.15.67:0/3203132379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0778003ff0 msgr2=0x7f0778004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.639+0000 7f0795e00640 1 --2- 172.21.15.67:0/3203132379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0778003ff0 0x7f0778004800 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f077c001050 tx=0x7f0778005830 comp rx=0 tx=0).stop 2024-03-09T09:18:40.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.639+0000 7f0795400640 1 -- 172.21.15.67:0/3203132379 reap_dead start 2024-03-09T09:18:40.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.639+0000 7f0795e00640 1 -- 172.21.15.67:0/3203132379 shutdown_connections 2024-03-09T09:18:40.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.639+0000 7f0795e00640 1 -- 172.21.15.67:0/3203132379 >> 172.21.15.67:0/3203132379 conn(0x7f07900c82a0 msgr2=0x7f07900c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:40.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.639+0000 7f0795e00640 1 -- 172.21.15.67:0/3203132379 shutdown_connections 2024-03-09T09:18:40.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.639+0000 7f0795e00640 1 -- 172.21.15.67:0/3203132379 wait complete. 2024-03-09T09:18:40.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.641+0000 7f0795e00640 1 Processor -- start 2024-03-09T09:18:40.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.641+0000 7f0795e00640 1 -- start start 2024-03-09T09:18:40.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.642+0000 7f0795e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f079006ede0 0x7f07900ac440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.642+0000 7f0795e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f079014cf20 0x7f07900c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.642+0000 7f0795e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07900ace20 0x7f079014c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.642+0000 7f0795e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07900c39b0 con 0x7f079014cf20 2024-03-09T09:18:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.642+0000 7f0795e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07900c3830 con 0x7f07900ace20 2024-03-09T09:18:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.642+0000 7f0795e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079006b750 con 0x7f079006ede0 2024-03-09T09:18:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.642+0000 7f078fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f079014cf20 0x7f07900c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.642+0000 7f078fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f079014cf20 0x7f07900c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55514/0 (socket says 172.21.15.67:55514) 2024-03-09T09:18:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.642+0000 7f078fe00640 1 -- 172.21.15.67:0/3627451594 learned_addr learned my addr 172.21.15.67:0/3627451594 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.642+0000 7f0794a00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f079006ede0 0x7f07900ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f0795400640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07900ace20 0x7f079014c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f078fe00640 1 -- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f079006ede0 msgr2=0x7f07900ac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f078fe00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f079006ede0 0x7f07900ac440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f078fe00640 1 -- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07900ace20 msgr2=0x7f079014c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f078fe00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f07900ace20 0x7f079014c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f078fe00640 1 -- 172.21.15.67:0/3627451594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07901ef3d0 con 0x7f079014cf20 2024-03-09T09:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f0794a00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f079006ede0 0x7f07900ac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f078fe00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f079014cf20 0x7f07900c5c10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f077c02c4c0 tx=0x7f077c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c001bf0 con 0x7f079014cf20 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c02cc60 con 0x7f079014cf20 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f0795e00640 1 -- 172.21.15.67:0/3627451594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07900ab250 con 0x7f079014cf20 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.643+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c0072b0 con 0x7f079014cf20 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.644+0000 7f0795e00640 1 -- 172.21.15.67:0/3627451594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07900ab8a0 con 0x7f079014cf20 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.645+0000 7f0795e00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07900ad200 0x7f079014c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.645+0000 7f0795e00640 1 -- 172.21.15.67:0/3627451594 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079004f3c0 con 0x7f07900ad200 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.646+0000 7f0794a00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07900ad200 0x7f079014c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.646+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f077c048020 con 0x7f079014cf20 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.646+0000 7f0794a00640 1 -- 172.21.15.67:0/3627451594 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07900c3460 con 0x7f07900ad200 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.647+0000 7f078cc00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f077006b4d0 0x7f077006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.647+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f077c0ab9e0 con 0x7f079014cf20 2024-03-09T09:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.647+0000 7f0794a00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07900ad200 0x7f079014c8f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0784010710 tx=0x7f0784010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.648+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400f040 con 0x7f07900ad200 2024-03-09T09:18:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.648+0000 7f0795400640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f077006b4d0 0x7f077006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.649+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f078401c400 con 0x7f07900ad200 2024-03-09T09:18:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.649+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07900ad200 msgr2=0x7f079014c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.649+0000 7f078cc00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07900ad200 0x7f079014c8f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0784010710 tx=0x7f0784010bd0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.649+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07900ad200 msgr2=0x7f079014c8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.649+0000 7f078cc00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f07900ad200 0x7f079014c8f0 secure :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0784010710 tx=0x7f0784010bd0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.650+0000 7f0795400640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f077006b4d0 0x7f077006d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f079006d580 tx=0x7f078800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.656+0000 7f0795e00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790154970 0x7f0790154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.656+0000 7f0795e00640 1 -- 172.21.15.67:0/3627451594 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0790156280 con 0x7f0790154970 2024-03-09T09:18:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.657+0000 7f0794a00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790154970 0x7f0790154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.657+0000 7f0794a00640 1 -- 172.21.15.67:0/3627451594 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f07900c58f0 con 0x7f0790154970 2024-03-09T09:18:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.658+0000 7f0794a00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790154970 0x7f0790154d50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0784010710 tx=0x7f0784009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.658+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078401c7b0 con 0x7f0790154970 2024-03-09T09:18:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.659+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f0784006000 con 0x7f0790154970 2024-03-09T09:18:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.659+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790154970 msgr2=0x7f0790154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.659+0000 7f078cc00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790154970 0x7f0790154d50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0784010710 tx=0x7f0784009190 comp rx=0 tx=0).stop 2024-03-09T09:18:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.659+0000 7f078cc00640 1 -- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790154970 msgr2=0x7f0790154d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.659+0000 7f078cc00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0790154970 0x7f0790154d50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.669 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:18:40.669 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:18:40.669 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:18:40.669 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:18:40.669 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 194, 2024-03-09T09:18:40.669 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:18:40.669 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:18:40.669 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:18:40.669 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:40.670 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:40.671 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:18:40.672 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:40.673 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:18:40.674 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.675 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:40.677 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:40.678 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:40.679 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:40.680 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:40.680 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:40.680 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:40.680 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:40.680 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:18:40.680 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:18:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.660+0000 7f0795e00640 1 -- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f077006b4d0 msgr2=0x7f077006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.660+0000 7f0795e00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f077006b4d0 0x7f077006d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f079006d580 tx=0x7f078800b040 comp rx=0 tx=0).stop 2024-03-09T09:18:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.660+0000 7f0795e00640 1 -- 172.21.15.67:0/3627451594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f079014cf20 msgr2=0x7f07900c5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.660+0000 7f0795e00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f079014cf20 0x7f07900c5c10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f077c02c4c0 tx=0x7f077c001970 comp rx=0 tx=0).stop 2024-03-09T09:18:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.660+0000 7f0795400640 1 -- 172.21.15.67:0/3627451594 reap_dead start 2024-03-09T09:18:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.660+0000 7f0795e00640 1 -- 172.21.15.67:0/3627451594 shutdown_connections 2024-03-09T09:18:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.660+0000 7f0795e00640 1 --2- 172.21.15.67:0/3627451594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f079014cf20 0x7f07900c5c10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.660+0000 7f0795e00640 1 -- 172.21.15.67:0/3627451594 >> 172.21.15.67:0/3627451594 conn(0x7f07900c82a0 msgr2=0x7f0790157ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.660+0000 7f0795e00640 1 -- 172.21.15.67:0/3627451594 shutdown_connections 2024-03-09T09:18:40.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.660+0000 7f0795e00640 1 -- 172.21.15.67:0/3627451594 wait complete. 2024-03-09T09:18:40.703 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:18:40.703 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T09:18:40.703 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:18:40.703 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T09:18:40.703 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:18:40.704 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:18:40.704 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:18:40.739 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:18:40.739 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:18:40.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:18:40.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.935+0000 7f466f000640 1 Processor -- start 2024-03-09T09:18:40.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.935+0000 7f466f000640 1 -- start start 2024-03-09T09:18:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.937+0000 7f466f000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f466806e8d0 0x7f466806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.937+0000 7f466f000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4668159570 0x7f4668159a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.937+0000 7f466f000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4668155dc0 con 0x7f4668150080 2024-03-09T09:18:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.937+0000 7f466f000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4668156c90 con 0x7f466806e8d0 2024-03-09T09:18:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.937+0000 7f466f000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4668156080 con 0x7f4668159570 2024-03-09T09:18:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.938+0000 7f4667e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4668150080 0x7f46680c83f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44934/0 (socket says 172.21.15.67:44934) 2024-03-09T09:18:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.938+0000 7f4667e00640 1 -- 172.21.15.67:0/2046978534 learned_addr learned my addr 172.21.15.67:0/2046978534 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:18:40.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.938+0000 7f4667400640 1 --2- 172.21.15.67:0/2046978534 >> v2:172.21.15.67:3300/0 conn(0x7f466806e8d0 0x7f466806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.938+0000 7f466d000640 1 --2- 172.21.15.67:0/2046978534 >> v2:172.21.15.134:3300/0 conn(0x7f4668159570 0x7f4668159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.938+0000 7f4666a00640 1 -- 172.21.15.67:0/2046978534 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137960134 0 0) 0x7f4668155dc0 con 0x7f4668150080 2024-03-09T09:18:40.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.938+0000 7f4666a00640 1 -- 172.21.15.67:0/2046978534 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f464c003520 con 0x7f4668150080 2024-03-09T09:18:40.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.938+0000 7f4667400640 1 -- 172.21.15.67:0/2046978534 >> v2:172.21.15.134:3300/0 conn(0x7f4668159570 msgr2=0x7f4668159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:40.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.938+0000 7f4667400640 1 --2- 172.21.15.67:0/2046978534 >> v2:172.21.15.134:3300/0 conn(0x7f4668159570 0x7f4668159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.939+0000 7f4667400640 1 -- 172.21.15.67:0/2046978534 >> v1:172.21.15.67:6789/0 conn(0x7f4668150080 legacy=0x7f46680c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.939+0000 7f4667400640 1 -- 172.21.15.67:0/2046978534 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4668156360 con 0x7f466806e8d0 2024-03-09T09:18:40.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.939+0000 7f466d000640 1 --2- 172.21.15.67:0/2046978534 >> v2:172.21.15.134:3300/0 conn(0x7f4668159570 0x7f4668159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:18:40.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.940+0000 7f4667400640 1 --2- 172.21.15.67:0/2046978534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466806e8d0 0x7f466806bab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f46680716e0 tx=0x7f465402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe1fdc1fc2f5b16 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.940+0000 7f4666a00640 1 -- 172.21.15.67:0/2046978534 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46540315a0 con 0x7f466806e8d0 2024-03-09T09:18:40.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.941+0000 7f4666a00640 1 -- 172.21.15.67:0/2046978534 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4654037070 con 0x7f466806e8d0 2024-03-09T09:18:40.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.941+0000 7f4666a00640 1 -- 172.21.15.67:0/2046978534 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654031960 con 0x7f466806e8d0 2024-03-09T09:18:40.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.941+0000 7f466f000640 1 -- 172.21.15.67:0/2046978534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466806e8d0 msgr2=0x7f466806bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.941+0000 7f466f000640 1 --2- 172.21.15.67:0/2046978534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466806e8d0 0x7f466806bab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f46680716e0 tx=0x7f465402b840 comp rx=0 tx=0).stop 2024-03-09T09:18:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.941+0000 7f466f000640 1 -- 172.21.15.67:0/2046978534 shutdown_connections 2024-03-09T09:18:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.941+0000 7f466f000640 1 --2- 172.21.15.67:0/2046978534 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466806e8d0 0x7f466806bab0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.941+0000 7f466f000640 1 --2- 172.21.15.67:0/2046978534 >> v2:172.21.15.134:3300/0 conn(0x7f4668159570 0x7f4668159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.941+0000 7f466f000640 1 -- 172.21.15.67:0/2046978534 >> 172.21.15.67:0/2046978534 conn(0x7f46680aa890 msgr2=0x7f46680ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.941+0000 7f466f000640 1 -- 172.21.15.67:0/2046978534 shutdown_connections 2024-03-09T09:18:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.941+0000 7f466f000640 1 -- 172.21.15.67:0/2046978534 wait complete. 2024-03-09T09:18:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.943+0000 7f466f000640 1 Processor -- start 2024-03-09T09:18:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.943+0000 7f466f000640 1 -- start start 2024-03-09T09:18:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.944+0000 7f466f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466806e8d0 0x7f4668143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.944+0000 7f466f000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4668150080 0x7f466814a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.944+0000 7f466f000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4668159570 0x7f466814adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.944+0000 7f466f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4668150460 con 0x7f466806e8d0 2024-03-09T09:18:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.944+0000 7f466f000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4668156920 con 0x7f4668150080 2024-03-09T09:18:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.944+0000 7f466f000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46680c8b00 con 0x7f4668159570 2024-03-09T09:18:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.944+0000 7f4667e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466806e8d0 0x7f4668143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.944+0000 7f466d000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4668159570 msgr2=0x7f466814adb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.944+0000 7f466d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4668159570 0x7f466814adb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.944+0000 7f4667e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466806e8d0 0x7f4668143c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55526/0 (socket says 172.21.15.67:55526) 2024-03-09T09:18:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.944+0000 7f4667e00640 1 -- 172.21.15.67:0/329281313 learned_addr learned my addr 172.21.15.67:0/329281313 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:40.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.945+0000 7f4667400640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4668150080 0x7f466814a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.945+0000 7f4667e00640 1 -- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4668159570 msgr2=0x7f466814adb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:18:40.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.945+0000 7f4667e00640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4668159570 0x7f466814adb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.945+0000 7f4667e00640 1 -- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4668150080 msgr2=0x7f466814a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:40.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.945+0000 7f4667e00640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4668150080 0x7f466814a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:40.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.945+0000 7f4667e00640 1 -- 172.21.15.67:0/329281313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46681ef520 con 0x7f466806e8d0 2024-03-09T09:18:40.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.945+0000 7f4667e00640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466806e8d0 0x7f4668143c50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f465800c430 tx=0x7f465800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.945+0000 7f465fe00640 1 -- 172.21.15.67:0/329281313 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658017070 con 0x7f466806e8d0 2024-03-09T09:18:40.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.945+0000 7f465fe00640 1 -- 172.21.15.67:0/329281313 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4658005ce0 con 0x7f466806e8d0 2024-03-09T09:18:40.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.945+0000 7f465fe00640 1 -- 172.21.15.67:0/329281313 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46580072f0 con 0x7f466806e8d0 2024-03-09T09:18:40.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.945+0000 7f466f000640 1 -- 172.21.15.67:0/329281313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4668154df0 con 0x7f466806e8d0 2024-03-09T09:18:40.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.946+0000 7f466f000640 1 -- 172.21.15.67:0/329281313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4668153f80 con 0x7f466806e8d0 2024-03-09T09:18:40.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.947+0000 7f466f000640 1 -- 172.21.15.67:0/329281313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4668075a00 con 0x7f466806e8d0 2024-03-09T09:18:40.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.948+0000 7f465fe00640 1 -- 172.21.15.67:0/329281313 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f465801a070 con 0x7f466806e8d0 2024-03-09T09:18:40.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.949+0000 7f465fe00640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f464806b480 0x7f464806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:40.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.949+0000 7f465fe00640 1 -- 172.21.15.67:0/329281313 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4658089a70 con 0x7f466806e8d0 2024-03-09T09:18:40.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.950+0000 7f4667400640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f464806b480 0x7f464806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:40.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.951+0000 7f4667400640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f464806b480 0x7f464806d940 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4654036000 tx=0x7f465403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:40.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:40.954+0000 7f465fe00640 1 -- 172.21.15.67:0/329281313 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f46580584f0 con 0x7f466806e8d0 2024-03-09T09:18:41.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.224+0000 7f466f000640 1 -- 172.21.15.67:0/329281313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f466814f760 con 0x7f466806e8d0 2024-03-09T09:18:41.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.224+0000 7f465fe00640 1 -- 172.21.15.67:0/329281313 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f4658057df0 con 0x7f466806e8d0 2024-03-09T09:18:41.244 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":194,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:18:41.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.227+0000 7f466f000640 1 -- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f464806b480 msgr2=0x7f464806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:41.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.227+0000 7f466f000640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f464806b480 0x7f464806d940 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4654036000 tx=0x7f465403d040 comp rx=0 tx=0).stop 2024-03-09T09:18:41.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.227+0000 7f466f000640 1 -- 172.21.15.67:0/329281313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466806e8d0 msgr2=0x7f4668143c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:41.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.227+0000 7f466f000640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466806e8d0 0x7f4668143c50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f465800c430 tx=0x7f465800c8f0 comp rx=0 tx=0).stop 2024-03-09T09:18:41.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.227+0000 7f466f000640 1 -- 172.21.15.67:0/329281313 shutdown_connections 2024-03-09T09:18:41.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.227+0000 7f466f000640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f464806b480 0x7f464806d940 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:41.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.227+0000 7f466f000640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4668159570 0x7f466814adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:41.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.227+0000 7f466f000640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4668150080 0x7f466814a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:41.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.227+0000 7f466f000640 1 --2- 172.21.15.67:0/329281313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f466806e8d0 0x7f4668143c50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.227+0000 7f466f000640 1 -- 172.21.15.67:0/329281313 >> 172.21.15.67:0/329281313 conn(0x7f46680aa890 msgr2=0x7f466814ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.227+0000 7f466f000640 1 -- 172.21.15.67:0/329281313 shutdown_connections 2024-03-09T09:18:41.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:41.228+0000 7f466f000640 1 -- 172.21.15.67:0/329281313 wait complete. 2024-03-09T09:18:41.342 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-03-09T09:18:42.206 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:42.206 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:42.206 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 22 -eq 48 ']' 2024-03-09T09:18:42.207 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.22 2024-03-09T09:18:42.207 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.22 2024-03-09T09:18:42.208 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.22 2024-03-09T09:18:42.208 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.22 2024-03-09T09:18:42.210 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:42.258 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:42.258 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:42.258 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 27 -eq 48 ']' 2024-03-09T09:18:42.258 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.27 2024-03-09T09:18:42.259 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.27 2024-03-09T09:18:42.259 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.27 2024-03-09T09:18:42.259 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.27 2024-03-09T09:18:42.260 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:44.255 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:18:44.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.470+0000 7faf2a800640 1 Processor -- start 2024-03-09T09:18:44.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.470+0000 7faf2a800640 1 -- start start 2024-03-09T09:18:44.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.472+0000 7faf2a800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faf240c5e40 0x7faf24156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:44.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.472+0000 7faf2a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7faf240cadb0 0x7faf24156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:44.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.472+0000 7faf2a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf24156a90 con 0x7faf24153300 2024-03-09T09:18:44.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.472+0000 7faf2a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf24157000 con 0x7faf240cadb0 2024-03-09T09:18:44.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.472+0000 7faf2a800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf240ae430 con 0x7faf240c5e40 2024-03-09T09:18:44.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.472+0000 7faf29400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faf24153300 0x7faf240b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44936/0 (socket says 172.21.15.67:44936) 2024-03-09T09:18:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.472+0000 7faf29400640 1 -- 172.21.15.67:0/638696371 learned_addr learned my addr 172.21.15.67:0/638696371 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:18:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.472+0000 7faf29e00640 1 --2- 172.21.15.67:0/638696371 >> v2:172.21.15.67:3300/0 conn(0x7faf240cadb0 0x7faf24156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.472+0000 7faf28a00640 1 --2- 172.21.15.67:0/638696371 >> v2:172.21.15.134:3300/0 conn(0x7faf240c5e40 0x7faf24156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.473+0000 7faf23e00640 1 -- 172.21.15.67:0/638696371 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580726289 0 0) 0x7faf24156a90 con 0x7faf24153300 2024-03-09T09:18:44.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.473+0000 7faf23e00640 1 -- 172.21.15.67:0/638696371 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf0c003520 con 0x7faf24153300 2024-03-09T09:18:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.473+0000 7faf29e00640 1 -- 172.21.15.67:0/638696371 >> v2:172.21.15.134:3300/0 conn(0x7faf240c5e40 msgr2=0x7faf24156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.473+0000 7faf29e00640 1 --2- 172.21.15.67:0/638696371 >> v2:172.21.15.134:3300/0 conn(0x7faf240c5e40 0x7faf24156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.473+0000 7faf29e00640 1 -- 172.21.15.67:0/638696371 >> v1:172.21.15.67:6789/0 conn(0x7faf24153300 legacy=0x7faf240b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:44.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.473+0000 7faf29e00640 1 -- 172.21.15.67:0/638696371 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240ae6f0 con 0x7faf240cadb0 2024-03-09T09:18:44.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.474+0000 7faf29e00640 1 --2- 172.21.15.67:0/638696371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf240cadb0 0x7faf24156550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faf240c8c10 tx=0x7faf1c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713bc336d02d4f4b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:44.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.474+0000 7faf23e00640 1 -- 172.21.15.67:0/638696371 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c035020 con 0x7faf240cadb0 2024-03-09T09:18:44.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.474+0000 7faf23e00640 1 -- 172.21.15.67:0/638696371 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c031830 con 0x7faf240cadb0 2024-03-09T09:18:44.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.474+0000 7faf23e00640 1 -- 172.21.15.67:0/638696371 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c031c00 con 0x7faf240cadb0 2024-03-09T09:18:44.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.475+0000 7faf2a800640 1 -- 172.21.15.67:0/638696371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf240cadb0 msgr2=0x7faf24156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:44.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.475+0000 7faf2a800640 1 --2- 172.21.15.67:0/638696371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf240cadb0 0x7faf24156550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faf240c8c10 tx=0x7faf1c02b8d0 comp rx=0 tx=0).stop 2024-03-09T09:18:44.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.475+0000 7faf2a800640 1 -- 172.21.15.67:0/638696371 shutdown_connections 2024-03-09T09:18:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.475+0000 7faf2a800640 1 --2- 172.21.15.67:0/638696371 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf240cadb0 0x7faf24156550 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.475+0000 7faf2a800640 1 --2- 172.21.15.67:0/638696371 >> v2:172.21.15.134:3300/0 conn(0x7faf240c5e40 0x7faf24156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.475+0000 7faf2a800640 1 -- 172.21.15.67:0/638696371 >> 172.21.15.67:0/638696371 conn(0x7faf240732d0 msgr2=0x7faf240ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.475+0000 7faf2a800640 1 -- 172.21.15.67:0/638696371 shutdown_connections 2024-03-09T09:18:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.475+0000 7faf2a800640 1 -- 172.21.15.67:0/638696371 wait complete. 2024-03-09T09:18:44.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.477+0000 7faf2a800640 1 Processor -- start 2024-03-09T09:18:44.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.477+0000 7faf2a800640 1 -- start start 2024-03-09T09:18:44.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.478+0000 7faf2a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf240c5e40 0x7faf241ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:44.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.478+0000 7faf2a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf240cadb0 0x7faf2414b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:44.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.478+0000 7faf2a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf24153300 0x7faf24155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:44.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.478+0000 7faf2a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf24159e90 con 0x7faf240cadb0 2024-03-09T09:18:44.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.478+0000 7faf2a800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf24159bb0 con 0x7faf240c5e40 2024-03-09T09:18:44.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.478+0000 7faf2a800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf240c59e0 con 0x7faf24153300 2024-03-09T09:18:44.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf29e00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf24153300 msgr2=0x7faf24155240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf28a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf240cadb0 0x7faf2414b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf29e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf24153300 0x7faf24155240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf28a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf240cadb0 0x7faf2414b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55536/0 (socket says 172.21.15.67:55536) 2024-03-09T09:18:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf28a00640 1 -- 172.21.15.67:0/2993174690 learned_addr learned my addr 172.21.15.67:0/2993174690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf29400640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf240c5e40 0x7faf241ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf28a00640 1 -- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf24153300 msgr2=0x7faf24155240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:18:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf28a00640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf24153300 0x7faf24155240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf28a00640 1 -- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf240c5e40 msgr2=0x7faf241ee6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf28a00640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf240c5e40 0x7faf241ee6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:44.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf28a00640 1 -- 172.21.15.67:0/2993174690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf24153780 con 0x7faf240cadb0 2024-03-09T09:18:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.479+0000 7faf28a00640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf240cadb0 0x7faf2414b070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faf1000e9b0 tx=0x7faf1000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.480+0000 7faf21600640 1 -- 172.21.15.67:0/2993174690 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10017070 con 0x7faf240cadb0 2024-03-09T09:18:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.480+0000 7faf21600640 1 -- 172.21.15.67:0/2993174690 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1000c9a0 con 0x7faf240cadb0 2024-03-09T09:18:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.480+0000 7faf21600640 1 -- 172.21.15.67:0/2993174690 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1000cd90 con 0x7faf240cadb0 2024-03-09T09:18:44.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.480+0000 7faf2a800640 1 -- 172.21.15.67:0/2993174690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf24153a60 con 0x7faf240cadb0 2024-03-09T09:18:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.480+0000 7faf2a800640 1 -- 172.21.15.67:0/2993174690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2414bc20 con 0x7faf240cadb0 2024-03-09T09:18:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.481+0000 7faf2a800640 1 -- 172.21.15.67:0/2993174690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf24075a00 con 0x7faf240cadb0 2024-03-09T09:18:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.482+0000 7faf21600640 1 -- 172.21.15.67:0/2993174690 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faf1001a070 con 0x7faf240cadb0 2024-03-09T09:18:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.483+0000 7faf21600640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf0406b480 0x7faf0406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.483+0000 7faf21600640 1 -- 172.21.15.67:0/2993174690 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7faf1008a990 con 0x7faf240cadb0 2024-03-09T09:18:44.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.483+0000 7faf29400640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf0406b480 0x7faf0406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:44.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.485+0000 7faf29400640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf0406b480 0x7faf0406d940 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7faf18005c00 tx=0x7faf18009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:44.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.487+0000 7faf21600640 1 -- 172.21.15.67:0/2993174690 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf24075a00 con 0x7faf240cadb0 2024-03-09T09:18:44.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.769+0000 7faf2a800640 1 -- 172.21.15.67:0/2993174690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7faf2404f3c0 con 0x7faf240cadb0 2024-03-09T09:18:44.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.770+0000 7faf21600640 1 -- 172.21.15.67:0/2993174690 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7faf10059410 con 0x7faf240cadb0 2024-03-09T09:18:44.776 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":194,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:18:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 -- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf0406b480 msgr2=0x7faf0406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf0406b480 0x7faf0406d940 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7faf18005c00 tx=0x7faf18009040 comp rx=0 tx=0).stop 2024-03-09T09:18:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 -- 172.21.15.67:0/2993174690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf240cadb0 msgr2=0x7faf2414b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf240cadb0 0x7faf2414b070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faf1000e9b0 tx=0x7faf1000ee70 comp rx=0 tx=0).stop 2024-03-09T09:18:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 -- 172.21.15.67:0/2993174690 shutdown_connections 2024-03-09T09:18:44.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf0406b480 0x7faf0406d940 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf24153300 0x7faf24155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf240cadb0 0x7faf2414b070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 --2- 172.21.15.67:0/2993174690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf240c5e40 0x7faf241ee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 -- 172.21.15.67:0/2993174690 >> 172.21.15.67:0/2993174690 conn(0x7faf240732d0 msgr2=0x7faf24156010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 -- 172.21.15.67:0/2993174690 shutdown_connections 2024-03-09T09:18:44.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:44.773+0000 7faf2a800640 1 -- 172.21.15.67:0/2993174690 wait complete. 2024-03-09T09:18:46.109 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:46.109 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:46.109 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 23 -eq 48 ']' 2024-03-09T09:18:46.109 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.23 2024-03-09T09:18:46.109 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.23 2024-03-09T09:18:46.109 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.23 2024-03-09T09:18:46.110 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.23 2024-03-09T09:18:46.114 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:47.455 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:47.456 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:47.456 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 28 -eq 48 ']' 2024-03-09T09:18:47.456 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.28 2024-03-09T09:18:47.456 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.28 2024-03-09T09:18:47.457 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.28 2024-03-09T09:18:47.457 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.28 2024-03-09T09:18:47.462 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:47.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:18:47.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-03-09T09:18:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.009+0000 7f8bc7e00640 1 Processor -- start 2024-03-09T09:18:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.009+0000 7f8bc7e00640 1 -- start start 2024-03-09T09:18:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.011+0000 7f8bc7e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8bc8153300 0x7f8bc80b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.011+0000 7f8bc7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc8157000 con 0x7f8bc80cadb0 2024-03-09T09:18:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.011+0000 7f8bc7e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc80ae430 con 0x7f8bc80c5e40 2024-03-09T09:18:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.011+0000 7f8bc7e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc80ae6f0 con 0x7f8bc8153300 2024-03-09T09:18:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.011+0000 7f8bc7400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f8bc80cadb0 0x7f8bc8160490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44944/0 (socket says 172.21.15.67:44944) 2024-03-09T09:18:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.011+0000 7f8bc7400640 1 -- 172.21.15.67:0/40061209 learned_addr learned my addr 172.21.15.67:0/40061209 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:18:48.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.011+0000 7f8bc6a00640 1 --2- 172.21.15.67:0/40061209 >> v2:172.21.15.134:3300/0 conn(0x7f8bc8153300 0x7f8bc80b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:48.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.012+0000 7f8bc6a00640 1 -- 172.21.15.67:0/40061209 >> v1:172.21.15.134:6789/0 conn(0x7f8bc80c5e40 legacy=0x7f8bc8156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:48.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.012+0000 7f8bc6a00640 1 -- 172.21.15.67:0/40061209 >> v1:172.21.15.67:6789/0 conn(0x7f8bc80cadb0 legacy=0x7f8bc8160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:48.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.012+0000 7f8bc6a00640 1 -- 172.21.15.67:0/40061209 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc8155b10 con 0x7f8bc8153300 2024-03-09T09:18:48.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.014+0000 7f8bc6a00640 1 --2- 172.21.15.67:0/40061209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc8153300 0x7f8bc80b0000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009d90 tx=0x7f8bb802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1247568c49e93e84 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:48.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.014+0000 7f8bc6000640 1 -- 172.21.15.67:0/40061209 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8039070 con 0x7f8bc8153300 2024-03-09T09:18:48.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.014+0000 7f8bc6000640 1 -- 172.21.15.67:0/40061209 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb802cd00 con 0x7f8bc8153300 2024-03-09T09:18:48.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.014+0000 7f8bc6000640 1 -- 172.21.15.67:0/40061209 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8034660 con 0x7f8bc8153300 2024-03-09T09:18:48.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.015+0000 7f8bc7e00640 1 -- 172.21.15.67:0/40061209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc8153300 msgr2=0x7f8bc80b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:48.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.015+0000 7f8bc7e00640 1 --2- 172.21.15.67:0/40061209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc8153300 0x7f8bc80b0000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009d90 tx=0x7f8bb802bf50 comp rx=0 tx=0).stop 2024-03-09T09:18:48.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.015+0000 7f8bc7e00640 1 -- 172.21.15.67:0/40061209 shutdown_connections 2024-03-09T09:18:48.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.015+0000 7f8bc7e00640 1 --2- 172.21.15.67:0/40061209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc8153300 0x7f8bc80b0000 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:48.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.015+0000 7f8bc7e00640 1 -- 172.21.15.67:0/40061209 >> 172.21.15.67:0/40061209 conn(0x7f8bc80732d0 msgr2=0x7f8bc80ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:48.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.015+0000 7f8bc7e00640 1 -- 172.21.15.67:0/40061209 shutdown_connections 2024-03-09T09:18:48.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.015+0000 7f8bc7e00640 1 -- 172.21.15.67:0/40061209 wait complete. 2024-03-09T09:18:48.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.017+0000 7f8bc7e00640 1 Processor -- start 2024-03-09T09:18:48.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.017+0000 7f8bc7e00640 1 -- start start 2024-03-09T09:18:48.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bc7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bc80c5e40 0x7f8bc81ef740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:48.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bc7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bc80cadb0 0x7f8bc80ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:48.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bc7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc8153300 0x7f8bc806b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bc7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc80c59e0 con 0x7f8bc80cadb0 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bc7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc8159e90 con 0x7f8bc8153300 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bc7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc80b0540 con 0x7f8bc80c5e40 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bc6a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bc80c5e40 msgr2=0x7f8bc81ef740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bc6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bc80c5e40 0x7f8bc81ef740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bbe000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bc80cadb0 0x7f8bc80ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bbe000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bc80cadb0 0x7f8bc80ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55552/0 (socket says 172.21.15.67:55552) 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bbe000640 1 -- 172.21.15.67:0/430217494 learned_addr learned my addr 172.21.15.67:0/430217494 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bc7400640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc8153300 0x7f8bc806b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bbe000640 1 -- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bc80c5e40 msgr2=0x7f8bc81ef740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bbe000640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bc80c5e40 0x7f8bc81ef740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bbe000640 1 -- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc8153300 msgr2=0x7f8bc806b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bbe000640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc8153300 0x7f8bc806b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:48.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.018+0000 7f8bbe000640 1 -- 172.21.15.67:0/430217494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc81eb8d0 con 0x7f8bc80cadb0 2024-03-09T09:18:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.019+0000 7f8bbe000640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bc80cadb0 0x7f8bc80ac390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000c400 tx=0x7f8bb000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.019+0000 7f8bbfe00640 1 -- 172.21.15.67:0/430217494 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb00052d0 con 0x7f8bc80cadb0 2024-03-09T09:18:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.019+0000 7f8bbfe00640 1 -- 172.21.15.67:0/430217494 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb0005490 con 0x7f8bc80cadb0 2024-03-09T09:18:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.019+0000 7f8bbfe00640 1 -- 172.21.15.67:0/430217494 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0007260 con 0x7f8bc80cadb0 2024-03-09T09:18:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.019+0000 7f8bc7e00640 1 -- 172.21.15.67:0/430217494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc81ebbb0 con 0x7f8bc80cadb0 2024-03-09T09:18:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.019+0000 7f8bc7e00640 1 -- 172.21.15.67:0/430217494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc81ec170 con 0x7f8bc80cadb0 2024-03-09T09:18:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.021+0000 7f8bc7e00640 1 -- 172.21.15.67:0/430217494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc804f3c0 con 0x7f8bc80cadb0 2024-03-09T09:18:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.021+0000 7f8bbfe00640 1 -- 172.21.15.67:0/430217494 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8bb0007400 con 0x7f8bc80cadb0 2024-03-09T09:18:48.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.022+0000 7f8bbfe00640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ba406b480 0x7f8ba406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:48.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.022+0000 7f8bbfe00640 1 -- 172.21.15.67:0/430217494 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8bb0089ae0 con 0x7f8bc80cadb0 2024-03-09T09:18:48.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.023+0000 7f8bc6a00640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ba406b480 0x7f8ba406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:48.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.024+0000 7f8bc6a00640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ba406b480 0x7f8ba406d940 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8036000 tx=0x7f8bb8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:48.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.027+0000 7f8bbfe00640 1 -- 172.21.15.67:0/430217494 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8bb0058560 con 0x7f8bc80cadb0 2024-03-09T09:18:48.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.348+0000 7f8bc7e00640 1 -- 172.21.15.67:0/430217494 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8bc80c53f0 con 0x7f8bc80cadb0 2024-03-09T09:18:48.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.349+0000 7f8bbfe00640 1 -- 172.21.15.67:0/430217494 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f8bb0057e60 con 0x7f8bc80cadb0 2024-03-09T09:18:48.355 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":194,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:18:48.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.351+0000 7f8bc7e00640 1 -- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ba406b480 msgr2=0x7f8ba406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:48.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.351+0000 7f8bc7e00640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ba406b480 0x7f8ba406d940 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8036000 tx=0x7f8bb8030000 comp rx=0 tx=0).stop 2024-03-09T09:18:48.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.351+0000 7f8bc7e00640 1 -- 172.21.15.67:0/430217494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bc80cadb0 msgr2=0x7f8bc80ac390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:48.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.351+0000 7f8bc7e00640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bc80cadb0 0x7f8bc80ac390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000c400 tx=0x7f8bb000c8c0 comp rx=0 tx=0).stop 2024-03-09T09:18:48.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.352+0000 7f8bc7e00640 1 -- 172.21.15.67:0/430217494 shutdown_connections 2024-03-09T09:18:48.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.352+0000 7f8bc7e00640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ba406b480 0x7f8ba406d940 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:48.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.352+0000 7f8bc7e00640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bc8153300 0x7f8bc806b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:48.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.352+0000 7f8bc7e00640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8bc80cadb0 0x7f8bc80ac390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:48.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.352+0000 7f8bc7e00640 1 --2- 172.21.15.67:0/430217494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8bc80c5e40 0x7f8bc81ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:48.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.352+0000 7f8bc7e00640 1 -- 172.21.15.67:0/430217494 >> 172.21.15.67:0/430217494 conn(0x7f8bc80732d0 msgr2=0x7f8bc80cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.352+0000 7f8bc7e00640 1 -- 172.21.15.67:0/430217494 shutdown_connections 2024-03-09T09:18:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:48.352+0000 7f8bc7e00640 1 -- 172.21.15.67:0/430217494 wait complete. 2024-03-09T09:18:51.379 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:18:51.560 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:51.560 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:51.560 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 24 -eq 48 ']' 2024-03-09T09:18:51.560 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.24 2024-03-09T09:18:51.561 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.24 2024-03-09T09:18:51.562 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.24 2024-03-09T09:18:51.562 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.24 2024-03-09T09:18:51.564 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:18:51.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.591+0000 7f33d7200640 1 Processor -- start 2024-03-09T09:18:51.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.591+0000 7f33d7200640 1 -- start start 2024-03-09T09:18:51.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.593+0000 7f33d7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33d80c4360 0x7f33d806e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:51.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.593+0000 7f33d7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:51.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.593+0000 7f33d7200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d8155d60 con 0x7f33d80c1bd0 2024-03-09T09:18:51.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.593+0000 7f33d7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d8156c50 con 0x7f33d81592d0 2024-03-09T09:18:51.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.593+0000 7f33d7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8159a40 con 0x7f33d80c4360 2024-03-09T09:18:51.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.593+0000 7f33d5400640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33d80c4360 msgr2=0x7f33d806e480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.593+0000 7f33d5400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33d80c4360 0x7f33d806e480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.594+0000 7f33d6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.594+0000 7f33d6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57826/0 (socket says 172.21.15.67:57826) 2024-03-09T09:18:51.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.594+0000 7f33d6800640 1 -- 172.21.15.67:0/625048952 learned_addr learned my addr 172.21.15.67:0/625048952 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.594+0000 7f33d6800640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 msgr2=0x7f33d813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:18:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.594+0000 7f33d6800640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 msgr2=0x7f33d813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:18:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.594+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:18:51.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.594+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:51.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.794+0000 7f33d5400640 1 -- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33d80c4360 msgr2=0x7f33d806e480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:51.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.794+0000 7f33d5400640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33d80c4360 0x7f33d806e480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:18:51.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.795+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:51.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.795+0000 7f33d6800640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 msgr2=0x7f33d813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:18:51.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.795+0000 7f33d6800640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 msgr2=0x7f33d813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:18:51.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.795+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:18:51.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:51.795+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:18:52.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.195+0000 7f33d5400640 1 -- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33d80c4360 msgr2=0x7f33d806e480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:52.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.195+0000 7f33d5400640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33d80c4360 0x7f33d806e480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:18:52.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.196+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:52.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.197+0000 7f33d6800640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 msgr2=0x7f33d813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:18:52.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.197+0000 7f33d6800640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 msgr2=0x7f33d813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:18:52.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.197+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:18:52.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.197+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:18:53.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.996+0000 7f33d5400640 1 -- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33d80c4360 msgr2=0x7f33d806e480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:53.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.996+0000 7f33d5400640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33d80c4360 0x7f33d806e480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:18:53.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.999+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:53.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.999+0000 7f33d6800640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 msgr2=0x7f33d813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:18:53.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.999+0000 7f33d6800640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 msgr2=0x7f33d813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:18:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:52.999+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:18:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:53.000+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:18:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.593+0000 7f33cf400640 1 -- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33d80c4360 msgr2=0x7f33d806e480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:18:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.593+0000 7f33cf400640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33d80c4360 0x7f33d806e480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.593+0000 7f33cf400640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 msgr2=0x7f33d813e9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:18:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.593+0000 7f33cf400640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.67:3300/0 conn(0x7f33d81592d0 0x7f33d813e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.593+0000 7f33cf400640 1 -- 172.21.15.67:0/625048952 >> v1:172.21.15.134:6789/0 conn(0x7f33d80c1bd0 legacy=0x7f33d8156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.593+0000 7f33cf400640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 0x7f33b8005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.593+0000 7f33cf400640 1 -- 172.21.15.67:0/625048952 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d8159d20 con 0x7f33b8006190 2024-03-09T09:18:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.593+0000 7f33cf400640 1 -- 172.21.15.67:0/625048952 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d8156c50 con 0x7f33b80036e0 2024-03-09T09:18:54.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.594+0000 7f33d5e00640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 0x7f33b8005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:54.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.595+0000 7f33d5e00640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 msgr2=0x7f33b8005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:18:54.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.595+0000 7f33d5e00640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 msgr2=0x7f33b8005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:18:54.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.595+0000 7f33d5e00640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 0x7f33b8005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:18:54.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.595+0000 7f33d5e00640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 0x7f33b8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:54.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.796+0000 7f33d5e00640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 0x7f33b8005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:54.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.797+0000 7f33d5e00640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 msgr2=0x7f33b8005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:18:54.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.797+0000 7f33d5e00640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 msgr2=0x7f33b8005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:18:54.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.797+0000 7f33d5e00640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 0x7f33b8005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:18:54.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:54.797+0000 7f33d5e00640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 0x7f33b8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:18:54.910 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:54.910 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:54.910 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 29 -eq 48 ']' 2024-03-09T09:18:54.910 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.29 2024-03-09T09:18:54.910 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.29 2024-03-09T09:18:54.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.29 2024-03-09T09:18:54.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.29 2024-03-09T09:18:54.916 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:55.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.198+0000 7f33d5e00640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 0x7f33b8005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:55.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.199+0000 7f33d5e00640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 msgr2=0x7f33b8005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:18:55.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.199+0000 7f33d5e00640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 msgr2=0x7f33b8005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:18:55.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.199+0000 7f33d5e00640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 0x7f33b8005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:18:55.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.199+0000 7f33d5e00640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 0x7f33b8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:18:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.496+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085927056 0 0) 0x7f33d8159d20 con 0x7f33b8006190 2024-03-09T09:18:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.496+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33bc003520 con 0x7f33b8006190 2024-03-09T09:18:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.497+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 234877642 0 0) 0x7f33bc003520 con 0x7f33b8006190 2024-03-09T09:18:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.497+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 msgr2=0x7f33b8005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:18:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.497+0000 7f33d4a00640 1 --2- 172.21.15.67:0/625048952 >> v2:172.21.15.134:3300/0 conn(0x7f33b80036e0 0x7f33b8005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.497+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b800af40 con 0x7f33b8006190 2024-03-09T09:18:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.497+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f33c4004340 con 0x7f33b8006190 2024-03-09T09:18:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.497+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 >> v1:172.21.15.67:6789/0 conn(0x7f33b8006190 legacy=0x7f33b8008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.497+0000 7f33d6800640 1 -- 172.21.15.67:0/625048952 reap_dead start 2024-03-09T09:18:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.498+0000 7f33d4a00640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33bc003a20 0x7f33bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.498+0000 7f33d4a00640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc004800 0x7f33bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.498+0000 7f33d4a00640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 0x7f33bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.498+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8156c50 con 0x7f33bc003a20 2024-03-09T09:18:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.498+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc00c1b0 con 0x7f33bc004800 2024-03-09T09:18:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.498+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc00c3b0 con 0x7f33bc0052e0 2024-03-09T09:18:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.498+0000 7f33d6800640 1 -- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 msgr2=0x7f33bc00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.499+0000 7f33d6800640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 0x7f33bc00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.499+0000 7f33d5400640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc004800 0x7f33bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.500+0000 7f33d5400640 1 -- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 msgr2=0x7f33bc00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:18:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.500+0000 7f33d5400640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 0x7f33bc00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.500+0000 7f33d5400640 1 -- 172.21.15.67:0/625048952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33bc003a20 msgr2=0x7f33bc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:55.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.500+0000 7f33d5400640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33bc003a20 0x7f33bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.500+0000 7f33d5400640 1 -- 172.21.15.67:0/625048952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc003520 con 0x7f33bc004800 2024-03-09T09:18:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.504+0000 7f33d5400640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc004800 0x7f33bc004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f33c4003930 tx=0x7f33bc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8432d715bac47fdc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.505+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c400c0e0 con 0x7f33bc004800 2024-03-09T09:18:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.505+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c4030070 con 0x7f33bc004800 2024-03-09T09:18:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.505+0000 7f33d4a00640 1 -- 172.21.15.67:0/625048952 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4035470 con 0x7f33bc004800 2024-03-09T09:18:55.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.506+0000 7f33d7200640 1 -- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc004800 msgr2=0x7f33bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:55.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.506+0000 7f33d7200640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc004800 0x7f33bc004c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f33c4003930 tx=0x7f33bc005830 comp rx=0 tx=0).stop 2024-03-09T09:18:55.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.507+0000 7f33d7200640 1 -- 172.21.15.67:0/625048952 shutdown_connections 2024-03-09T09:18:55.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.507+0000 7f33d7200640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 0x7f33bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.507+0000 7f33d7200640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc004800 0x7f33bc004c80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.507+0000 7f33d7200640 1 --2- 172.21.15.67:0/625048952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33bc003a20 0x7f33bc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.507+0000 7f33d7200640 1 -- 172.21.15.67:0/625048952 >> 172.21.15.67:0/625048952 conn(0x7f33d80c8310 msgr2=0x7f33d80c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:55.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.507+0000 7f33d7200640 1 -- 172.21.15.67:0/625048952 shutdown_connections 2024-03-09T09:18:55.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.507+0000 7f33d7200640 1 -- 172.21.15.67:0/625048952 wait complete. 2024-03-09T09:18:55.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.511+0000 7f33d7200640 1 Processor -- start 2024-03-09T09:18:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.511+0000 7f33d7200640 1 -- start start 2024-03-09T09:18:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.512+0000 7f33d7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc003a20 0x7f33d813a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.512+0000 7f33d7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33bc004800 0x7f33d8158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.512+0000 7f33d7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 0x7f33d80ab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.512+0000 7f33d7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8159740 con 0x7f33bc004800 2024-03-09T09:18:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.512+0000 7f33d7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8156020 con 0x7f33bc003a20 2024-03-09T09:18:55.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.512+0000 7f33d7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80c1fb0 con 0x7f33bc0052e0 2024-03-09T09:18:55.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.513+0000 7f33d5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33bc004800 0x7f33d8158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.513+0000 7f33d6800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 msgr2=0x7f33d80ab600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.513+0000 7f33d6800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 0x7f33d80ab600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.513+0000 7f33d5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33bc004800 0x7f33d8158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57876/0 (socket says 172.21.15.67:57876) 2024-03-09T09:18:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.513+0000 7f33d5400640 1 -- 172.21.15.67:0/1977774270 learned_addr learned my addr 172.21.15.67:0/1977774270 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.513+0000 7f33d5e00640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc003a20 0x7f33d813a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.513+0000 7f33d5400640 1 -- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 msgr2=0x7f33d80ab600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:18:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.513+0000 7f33d5400640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 0x7f33d80ab600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.513+0000 7f33d5400640 1 -- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc003a20 msgr2=0x7f33d813a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.513+0000 7f33d5400640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc003a20 0x7f33d813a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.513+0000 7f33d5400640 1 -- 172.21.15.67:0/1977774270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d80c6e30 con 0x7f33bc004800 2024-03-09T09:18:55.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.514+0000 7f33d5400640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33bc004800 0x7f33d8158250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f33c402c6f0 tx=0x7f33c40055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:55.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.514+0000 7f33ce000640 1 -- 172.21.15.67:0/1977774270 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4003910 con 0x7f33bc004800 2024-03-09T09:18:55.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.514+0000 7f33ce000640 1 -- 172.21.15.67:0/1977774270 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c4037840 con 0x7f33bc004800 2024-03-09T09:18:55.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.514+0000 7f33ce000640 1 -- 172.21.15.67:0/1977774270 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4037bf0 con 0x7f33bc004800 2024-03-09T09:18:55.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.514+0000 7f33d7200640 1 -- 172.21.15.67:0/1977774270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d80c7b60 con 0x7f33bc004800 2024-03-09T09:18:55.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.515+0000 7f33d7200640 1 -- 172.21.15.67:0/1977774270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d81eed00 con 0x7f33bc004800 2024-03-09T09:18:55.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.516+0000 7f33d7200640 1 -- 172.21.15.67:0/1977774270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d804f3c0 con 0x7f33bc004800 2024-03-09T09:18:55.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.517+0000 7f33ce000640 1 -- 172.21.15.67:0/1977774270 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f33c4035470 con 0x7f33bc004800 2024-03-09T09:18:55.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.518+0000 7f33ce000640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f33b806bcd0 0x7f33b806e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:55.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.518+0000 7f33ce000640 1 -- 172.21.15.67:0/1977774270 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f33c40ab6d0 con 0x7f33bc004800 2024-03-09T09:18:55.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.519+0000 7f33d5e00640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f33b806bcd0 0x7f33b806e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:55.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.520+0000 7f33d5e00640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f33b806bcd0 0x7f33b806e190 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f33c8001050 tx=0x7f33c8007c60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:55.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.524+0000 7f33ce000640 1 -- 172.21.15.67:0/1977774270 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f33c407a1f0 con 0x7f33bc004800 2024-03-09T09:18:55.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.950+0000 7f33d7200640 1 -- 172.21.15.67:0/1977774270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f33d8067ac0 con 0x7f33bc004800 2024-03-09T09:18:55.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.951+0000 7f33ce000640 1 -- 172.21.15.67:0/1977774270 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f33c4079af0 con 0x7f33bc004800 2024-03-09T09:18:55.957 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":198,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:18:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 -- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f33b806bcd0 msgr2=0x7f33b806e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f33b806bcd0 0x7f33b806e190 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f33c8001050 tx=0x7f33c8007c60 comp rx=0 tx=0).stop 2024-03-09T09:18:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 -- 172.21.15.67:0/1977774270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33bc004800 msgr2=0x7f33d8158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33bc004800 0x7f33d8158250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f33c402c6f0 tx=0x7f33c40055b0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 -- 172.21.15.67:0/1977774270 shutdown_connections 2024-03-09T09:18:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f33b806bcd0 0x7f33b806e190 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f33bc0052e0 0x7f33d80ab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f33bc004800 0x7f33d8158250 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 --2- 172.21.15.67:0/1977774270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc003a20 0x7f33d813a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 -- 172.21.15.67:0/1977774270 >> 172.21.15.67:0/1977774270 conn(0x7f33d80c8310 msgr2=0x7f33d8154770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 -- 172.21.15.67:0/1977774270 shutdown_connections 2024-03-09T09:18:55.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:55.954+0000 7f33d7200640 1 -- 172.21.15.67:0/1977774270 wait complete. 2024-03-09T09:18:55.979 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:18:55.980 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:18:55.980 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:18:56.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.186+0000 7f9b50600640 1 Processor -- start 2024-03-09T09:18:56.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.186+0000 7f9b50600640 1 -- start start 2024-03-09T09:18:56.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b50600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9b480ac9a0 0x7f9b481510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b50600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b480c2780 0x7f9b48151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b50600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9b48071190 0x7f9b48154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b50600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4806d590 con 0x7f9b48071190 2024-03-09T09:18:56.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b50600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b480c3740 con 0x7f9b480ac9a0 2024-03-09T09:18:56.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b50600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b480c8200 con 0x7f9b480c2780 2024-03-09T09:18:56.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b4d200640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b480c2780 msgr2=0x7f9b48151620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:56.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b4d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b480c2780 0x7f9b48151620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:56.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b4e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9b48071190 0x7f9b48154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b4e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9b48071190 0x7f9b48154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57890/0 (socket says 172.21.15.67:57890) 2024-03-09T09:18:56.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b4e600640 1 -- 172.21.15.67:0/3695919470 learned_addr learned my addr 172.21.15.67:0/3695919470 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:56.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.188+0000 7f9b4dc00640 1 --2- 172.21.15.67:0/3695919470 >> v2:172.21.15.134:3300/0 conn(0x7f9b480ac9a0 0x7f9b481510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.189+0000 7f9b4e600640 1 -- 172.21.15.67:0/3695919470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b480c2780 msgr2=0x7f9b48151620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:18:56.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.189+0000 7f9b4e600640 1 --2- 172.21.15.67:0/3695919470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b480c2780 0x7f9b48151620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.189+0000 7f9b4e600640 1 -- 172.21.15.67:0/3695919470 >> v2:172.21.15.134:3300/0 conn(0x7f9b480ac9a0 msgr2=0x7f9b481510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:56.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.189+0000 7f9b4e600640 1 --2- 172.21.15.67:0/3695919470 >> v2:172.21.15.134:3300/0 conn(0x7f9b480ac9a0 0x7f9b481510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.189+0000 7f9b4e600640 1 -- 172.21.15.67:0/3695919470 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b480c3990 con 0x7f9b48071190 2024-03-09T09:18:56.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.190+0000 7f9b4e600640 1 --2- 172.21.15.67:0/3695919470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48071190 0x7f9b48154190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b4806ed40 tx=0x7f9b4002be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b847365fb58546d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:56.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.190+0000 7f9b47e00640 1 -- 172.21.15.67:0/3695919470 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4002ca10 con 0x7f9b48071190 2024-03-09T09:18:56.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.190+0000 7f9b47e00640 1 -- 172.21.15.67:0/3695919470 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4002cbb0 con 0x7f9b48071190 2024-03-09T09:18:56.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.190+0000 7f9b47e00640 1 -- 172.21.15.67:0/3695919470 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b400343f0 con 0x7f9b48071190 2024-03-09T09:18:56.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.191+0000 7f9b50600640 1 -- 172.21.15.67:0/3695919470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48071190 msgr2=0x7f9b48154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.191+0000 7f9b50600640 1 --2- 172.21.15.67:0/3695919470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48071190 0x7f9b48154190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b4806ed40 tx=0x7f9b4002be70 comp rx=0 tx=0).stop 2024-03-09T09:18:56.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.191+0000 7f9b50600640 1 -- 172.21.15.67:0/3695919470 shutdown_connections 2024-03-09T09:18:56.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.191+0000 7f9b50600640 1 --2- 172.21.15.67:0/3695919470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48071190 0x7f9b48154190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.191+0000 7f9b50600640 1 --2- 172.21.15.67:0/3695919470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b480c2780 0x7f9b48151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.191+0000 7f9b50600640 1 --2- 172.21.15.67:0/3695919470 >> v2:172.21.15.134:3300/0 conn(0x7f9b480ac9a0 0x7f9b481510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.191+0000 7f9b50600640 1 -- 172.21.15.67:0/3695919470 >> 172.21.15.67:0/3695919470 conn(0x7f9b480c7370 msgr2=0x7f9b480c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:56.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.191+0000 7f9b50600640 1 -- 172.21.15.67:0/3695919470 shutdown_connections 2024-03-09T09:18:56.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.191+0000 7f9b50600640 1 -- 172.21.15.67:0/3695919470 wait complete. 2024-03-09T09:18:56.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.193+0000 7f9b50600640 1 Processor -- start 2024-03-09T09:18:56.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.193+0000 7f9b50600640 1 -- start start 2024-03-09T09:18:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.194+0000 7f9b50600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b48071190 0x7f9b48150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.194+0000 7f9b50600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480ac9a0 0x7f9b481ebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.194+0000 7f9b50600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480c2780 0x7f9b481ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.194+0000 7f9b50600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b480acd80 con 0x7f9b480ac9a0 2024-03-09T09:18:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.194+0000 7f9b50600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b48070c00 con 0x7f9b480c2780 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.194+0000 7f9b50600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b480acf00 con 0x7f9b48071190 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.194+0000 7f9b4dc00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b48071190 msgr2=0x7f9b48150500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.194+0000 7f9b4dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b48071190 0x7f9b48150500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.195+0000 7f9b4d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480ac9a0 0x7f9b481ebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.195+0000 7f9b4d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480ac9a0 0x7f9b481ebcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57894/0 (socket says 172.21.15.67:57894) 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.195+0000 7f9b4d200640 1 -- 172.21.15.67:0/4109908234 learned_addr learned my addr 172.21.15.67:0/4109908234 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.195+0000 7f9b4e600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480c2780 0x7f9b481ec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.195+0000 7f9b4d200640 1 -- 172.21.15.67:0/4109908234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b48071190 msgr2=0x7f9b48150500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.195+0000 7f9b4d200640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b48071190 0x7f9b48150500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.195+0000 7f9b4d200640 1 -- 172.21.15.67:0/4109908234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480c2780 msgr2=0x7f9b481ec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.195+0000 7f9b4d200640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480c2780 0x7f9b481ec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.195+0000 7f9b4d200640 1 -- 172.21.15.67:0/4109908234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b481521a0 con 0x7f9b480ac9a0 2024-03-09T09:18:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.195+0000 7f9b4d200640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480ac9a0 0x7f9b481ebcb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400b4e0 tx=0x7f9b3400b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.196+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34003db0 con 0x7f9b480ac9a0 2024-03-09T09:18:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.196+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b34015070 con 0x7f9b480ac9a0 2024-03-09T09:18:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.196+0000 7f9b50600640 1 -- 172.21.15.67:0/4109908234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b481ef3a0 con 0x7f9b480ac9a0 2024-03-09T09:18:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.196+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b340052b0 con 0x7f9b480ac9a0 2024-03-09T09:18:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.196+0000 7f9b50600640 1 -- 172.21.15.67:0/4109908234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b481ef960 con 0x7f9b480ac9a0 2024-03-09T09:18:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.197+0000 7f9b50600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48153c10 0x7f9b48145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.197+0000 7f9b50600640 1 -- 172.21.15.67:0/4109908234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4814deb0 con 0x7f9b48153c10 2024-03-09T09:18:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.197+0000 7f9b4dc00640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48153c10 0x7f9b48145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.198+0000 7f9b4dc00640 1 -- 172.21.15.67:0/4109908234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b481efb60 con 0x7f9b48153c10 2024-03-09T09:18:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.198+0000 7f9b4dc00640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48153c10 0x7f9b48145570 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e7e0 tx=0x7f9b3c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.198+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c017070 con 0x7f9b48153c10 2024-03-09T09:18:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.198+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9b34005450 con 0x7f9b480ac9a0 2024-03-09T09:18:56.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.200+0000 7f9b45600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b2806b3d0 0x7f9b2806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.200+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f9b3c004020 con 0x7f9b48153c10 2024-03-09T09:18:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.200+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48153c10 msgr2=0x7f9b48145570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.200+0000 7f9b45600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48153c10 0x7f9b48145570 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e7e0 tx=0x7f9b3c00eca0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.200+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48153c10 msgr2=0x7f9b48145570 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.200+0000 7f9b45600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48153c10 0x7f9b48145570 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e7e0 tx=0x7f9b3c00eca0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.200+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9b3408a4a0 con 0x7f9b480ac9a0 2024-03-09T09:18:56.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.201+0000 7f9b4e600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b2806b3d0 0x7f9b2806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.202+0000 7f9b4e600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b2806b3d0 0x7f9b2806d890 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9b40035000 tx=0x7f9b40030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.208+0000 7f9b50600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48075a00 0x7f9b48145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.208+0000 7f9b50600640 1 -- 172.21.15.67:0/4109908234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b480731e0 con 0x7f9b48075a00 2024-03-09T09:18:56.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.208+0000 7f9b4dc00640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48075a00 0x7f9b48145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.208+0000 7f9b4dc00640 1 -- 172.21.15.67:0/4109908234 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9b48072f00 con 0x7f9b48075a00 2024-03-09T09:18:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.209+0000 7f9b4dc00640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48075a00 0x7f9b48145ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e7e0 tx=0x7f9b3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:56.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.209+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c003ae0 con 0x7f9b48075a00 2024-03-09T09:18:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.209+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7f9b3c003ae0 con 0x7f9b48075a00 2024-03-09T09:18:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.209+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48075a00 msgr2=0x7f9b48145ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.209+0000 7f9b45600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48075a00 0x7f9b48145ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e7e0 tx=0x7f9b3c004730 comp rx=0 tx=0).stop 2024-03-09T09:18:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.209+0000 7f9b45600640 1 -- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48075a00 msgr2=0x7f9b48145ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:56.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.209+0000 7f9b45600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b48075a00 0x7f9b48145ab0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e7e0 tx=0x7f9b3c004730 comp rx=0 tx=0).stop 2024-03-09T09:18:56.216 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:18:56.216 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:18:56.216 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:18:56.217 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:18:56.217 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 198, 2024-03-09T09:18:56.217 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:18:56.217 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:18:56.217 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:18:56.217 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.217 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:18:56.217 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:56.217 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:18:56.218 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:18:56.218 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:56.218 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:56.218 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:56.218 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:56.218 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:56.218 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:56.218 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:56.218 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:56.219 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:56.220 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:56.221 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:18:56.222 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.224 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:18:56.224 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:18:56.224 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:56.224 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:56.224 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:56.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.224 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.226 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:18:56.226 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:18:56.226 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:56.226 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:56.226 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:56.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.226 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:18:56.226 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:18:56.226 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:56.226 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:18:56.227 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:56.228 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:56.229 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:18:56.230 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.231 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.231 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:18:56.231 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.231 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:56.231 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:56.231 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:56.231 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.231 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.231 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:18:56.231 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:18:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.210+0000 7f9b50600640 1 -- 172.21.15.67:0/4109908234 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b2806b3d0 msgr2=0x7f9b2806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.210+0000 7f9b50600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9b2806b3d0 0x7f9b2806d890 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9b40035000 tx=0x7f9b40030000 comp rx=0 tx=0).stop 2024-03-09T09:18:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.210+0000 7f9b50600640 1 -- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480ac9a0 msgr2=0x7f9b481ebcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.210+0000 7f9b50600640 1 --2- 172.21.15.67:0/4109908234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480ac9a0 0x7f9b481ebcb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400b4e0 tx=0x7f9b3400b9a0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.210+0000 7f9b4e600640 1 -- 172.21.15.67:0/4109908234 reap_dead start 2024-03-09T09:18:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.210+0000 7f9b4e600640 1 -- 172.21.15.67:0/4109908234 reap_dead start 2024-03-09T09:18:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.210+0000 7f9b50600640 1 -- 172.21.15.67:0/4109908234 shutdown_connections 2024-03-09T09:18:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.210+0000 7f9b50600640 1 -- 172.21.15.67:0/4109908234 >> 172.21.15.67:0/4109908234 conn(0x7f9b480c7370 msgr2=0x7f9b480c1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.210+0000 7f9b50600640 1 -- 172.21.15.67:0/4109908234 shutdown_connections 2024-03-09T09:18:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.210+0000 7f9b50600640 1 -- 172.21.15.67:0/4109908234 wait complete. 2024-03-09T09:18:56.235 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:18:56.235 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:18:56.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:18:56.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.433+0000 7f8e3a800640 1 Processor -- start 2024-03-09T09:18:56.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.433+0000 7f8e3a800640 1 -- start start 2024-03-09T09:18:56.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.434+0000 7f8e3a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e340c9380 0x7f8e340acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.434+0000 7f8e3a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e340ad440 0x7f8e3406c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.434+0000 7f8e3a800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e340c9890 con 0x7f8e3414fd90 2024-03-09T09:18:56.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.434+0000 7f8e3a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3406ba60 con 0x7f8e340ad440 2024-03-09T09:18:56.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.434+0000 7f8e3a800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e340c5f80 con 0x7f8e340c9380 2024-03-09T09:18:56.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.435+0000 7f8e38a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e340c9380 msgr2=0x7f8e340acf00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:56.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.435+0000 7f8e38a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e340c9380 0x7f8e340acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:56.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.435+0000 7f8e39e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e340ad440 0x7f8e3406c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.435+0000 7f8e39e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e340ad440 0x7f8e3406c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57916/0 (socket says 172.21.15.67:57916) 2024-03-09T09:18:56.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.435+0000 7f8e39e00640 1 -- 172.21.15.67:0/1143948081 learned_addr learned my addr 172.21.15.67:0/1143948081 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:56.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.435+0000 7f8e39e00640 1 -- 172.21.15.67:0/1143948081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e340c9380 msgr2=0x7f8e340acf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:18:56.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.435+0000 7f8e39e00640 1 --2- 172.21.15.67:0/1143948081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e340c9380 0x7f8e340acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.435+0000 7f8e39e00640 1 -- 172.21.15.67:0/1143948081 >> v1:172.21.15.134:6789/0 conn(0x7f8e3414fd90 legacy=0x7f8e34156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:18:56.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.435+0000 7f8e39e00640 1 -- 172.21.15.67:0/1143948081 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e340c6240 con 0x7f8e340ad440 2024-03-09T09:18:56.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.436+0000 7f8e39e00640 1 --2- 172.21.15.67:0/1143948081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e340ad440 0x7f8e3406c1d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8e3406bbe0 tx=0x7f8e2c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4d5d601b28e13f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:56.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.437+0000 7f8e33e00640 1 -- 172.21.15.67:0/1143948081 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c02c950 con 0x7f8e340ad440 2024-03-09T09:18:56.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.437+0000 7f8e33e00640 1 -- 172.21.15.67:0/1143948081 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c02caf0 con 0x7f8e340ad440 2024-03-09T09:18:56.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.437+0000 7f8e33e00640 1 -- 172.21.15.67:0/1143948081 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c0343f0 con 0x7f8e340ad440 2024-03-09T09:18:56.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.438+0000 7f8e3a800640 1 -- 172.21.15.67:0/1143948081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e340ad440 msgr2=0x7f8e3406c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.438+0000 7f8e3a800640 1 --2- 172.21.15.67:0/1143948081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e340ad440 0x7f8e3406c1d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8e3406bbe0 tx=0x7f8e2c02bdb0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.438+0000 7f8e3a800640 1 -- 172.21.15.67:0/1143948081 shutdown_connections 2024-03-09T09:18:56.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.438+0000 7f8e3a800640 1 --2- 172.21.15.67:0/1143948081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e340ad440 0x7f8e3406c1d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.438+0000 7f8e3a800640 1 --2- 172.21.15.67:0/1143948081 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e340c9380 0x7f8e340acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.438+0000 7f8e3a800640 1 -- 172.21.15.67:0/1143948081 >> 172.21.15.67:0/1143948081 conn(0x7f8e340cb480 msgr2=0x7f8e340cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:56.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.438+0000 7f8e3a800640 1 -- 172.21.15.67:0/1143948081 shutdown_connections 2024-03-09T09:18:56.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.438+0000 7f8e3a800640 1 -- 172.21.15.67:0/1143948081 wait complete. 2024-03-09T09:18:56.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.440+0000 7f8e3a800640 1 Processor -- start 2024-03-09T09:18:56.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.440+0000 7f8e3a800640 1 -- start start 2024-03-09T09:18:56.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e3a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e340ad440 0x7f8e34148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e3a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e340c9380 0x7f8e341e0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e3a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3414fd90 0x7f8e341e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e3a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3406c710 con 0x7f8e3414fd90 2024-03-09T09:18:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e3a800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3406bcb0 con 0x7f8e340ad440 2024-03-09T09:18:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e3a800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3406c890 con 0x7f8e340c9380 2024-03-09T09:18:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e38a00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e340c9380 msgr2=0x7f8e341e0630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:18:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e39e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3414fd90 0x7f8e341e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e38a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e340c9380 0x7f8e341e0630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:18:56.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e39e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3414fd90 0x7f8e341e0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57926/0 (socket says 172.21.15.67:57926) 2024-03-09T09:18:56.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e39e00640 1 -- 172.21.15.67:0/3784600088 learned_addr learned my addr 172.21.15.67:0/3784600088 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:18:56.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e39400640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e340ad440 0x7f8e34148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e39e00640 1 -- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e340c9380 msgr2=0x7f8e341e0630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:18:56.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e39e00640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e340c9380 0x7f8e341e0630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e39e00640 1 -- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e340ad440 msgr2=0x7f8e34148380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e39e00640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e340ad440 0x7f8e34148380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.441+0000 7f8e39e00640 1 -- 172.21.15.67:0/3784600088 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e340ac730 con 0x7f8e3414fd90 2024-03-09T09:18:56.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.442+0000 7f8e39e00640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3414fd90 0x7f8e341e0b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009f40 tx=0x7f8e2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:56.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.442+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c043020 con 0x7f8e3414fd90 2024-03-09T09:18:56.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.442+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c03e450 con 0x7f8e3414fd90 2024-03-09T09:18:56.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.442+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c03e820 con 0x7f8e3414fd90 2024-03-09T09:18:56.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.442+0000 7f8e3a800640 1 -- 172.21.15.67:0/3784600088 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e341efba0 con 0x7f8e3414fd90 2024-03-09T09:18:56.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.442+0000 7f8e3a800640 1 -- 172.21.15.67:0/3784600088 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e340accd0 con 0x7f8e3414fd90 2024-03-09T09:18:56.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.443+0000 7f8e3a800640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341eb960 0x7f8e341ebd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.444+0000 7f8e3a800640 1 -- 172.21.15.67:0/3784600088 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3404f3c0 con 0x7f8e341eb960 2024-03-09T09:18:56.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.444+0000 7f8e39400640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341eb960 0x7f8e341ebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.444+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8e2c046020 con 0x7f8e3414fd90 2024-03-09T09:18:56.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.445+0000 7f8e39400640 1 -- 172.21.15.67:0/3784600088 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e3406d2d0 con 0x7f8e341eb960 2024-03-09T09:18:56.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.445+0000 7f8e39400640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341eb960 0x7f8e341ebd40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800d500 tx=0x7f8e2800d9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:56.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.445+0000 7f8e31600640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e1406b4b0 0x7f8e1406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.445+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8e2c0acc50 con 0x7f8e3414fd90 2024-03-09T09:18:56.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.446+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28018020 con 0x7f8e341eb960 2024-03-09T09:18:56.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.446+0000 7f8e38a00640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e1406b4b0 0x7f8e1406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.446+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f8e28014620 con 0x7f8e341eb960 2024-03-09T09:18:56.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.446+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341eb960 msgr2=0x7f8e341ebd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.446+0000 7f8e31600640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341eb960 0x7f8e341ebd40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800d500 tx=0x7f8e2800d9c0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.447+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341eb960 msgr2=0x7f8e341ebd40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:56.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.447+0000 7f8e31600640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341eb960 0x7f8e341ebd40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.447+0000 7f8e38a00640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e1406b4b0 0x7f8e1406d970 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8e20006fd0 tx=0x7f8e20006d80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:56.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.454+0000 7f8e3a800640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341ec280 0x7f8e340aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:18:56.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.454+0000 7f8e3a800640 1 -- 172.21.15.67:0/3784600088 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e341e10b0 con 0x7f8e341ec280 2024-03-09T09:18:56.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.455+0000 7f8e39400640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341ec280 0x7f8e340aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:18:56.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.456+0000 7f8e39400640 1 -- 172.21.15.67:0/3784600088 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8e340ca9d0 con 0x7f8e341ec280 2024-03-09T09:18:56.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.456+0000 7f8e39400640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341ec280 0x7f8e340aad80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800d500 tx=0x7f8e280046f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:18:56.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.456+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28026070 con 0x7f8e341ec280 2024-03-09T09:18:56.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.457+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4863 (secure 0 0 0) 0x7f8e28003bc0 con 0x7f8e341ec280 2024-03-09T09:18:56.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.457+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341ec280 msgr2=0x7f8e340aad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.457+0000 7f8e31600640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341ec280 0x7f8e340aad80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800d500 tx=0x7f8e280046f0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.457+0000 7f8e31600640 1 -- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341ec280 msgr2=0x7f8e340aad80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:18:56.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.457+0000 7f8e31600640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e341ec280 0x7f8e340aad80 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800d500 tx=0x7f8e280046f0 comp rx=0 tx=0).stop 2024-03-09T09:18:56.464 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:18:56.464 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:18:56.464 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 198, 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:56.465 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:56.466 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:56.466 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:56.466 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:56.466 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:56.466 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:56.466 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:56.466 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.466 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:18:56.466 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:18:56.466 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:56.466 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:56.467 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:56.467 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:56.467 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:56.467 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:56.467 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:56.467 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:56.467 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:56.467 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:56.467 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:56.467 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:18:56.468 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:18:56.469 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:18:56.469 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:18:56.469 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:18:56.469 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:18:56.469 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:18:56.469 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:18:56.469 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:18:56.469 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:18:56.469 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:18:56.469 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:18:56.469 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:56.470 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:56.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.471 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:56.471 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:18:56.471 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.471 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:56.471 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:18:56.471 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.471 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.472 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.472 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:18:56.472 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:18:56.472 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:56.472 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:56.472 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:56.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.472 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:18:56.473 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:18:56.473 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:56.473 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:56.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.473 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:56.473 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:18:56.473 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.474 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:56.474 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:18:56.474 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.474 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.474 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.474 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:18:56.474 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:18:56.474 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:56.475 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:56.475 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:56.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.475 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:18:56.475 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:18:56.475 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:18:56.475 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:18:56.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:18:56.476 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:18:56.476 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.476 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:18:56.476 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:18:56.476 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:18:56.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.476 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:56.477 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:56.478 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:56.478 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.478 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.478 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:18:56.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.478 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:56.478 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:56.478 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:56.478 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.479 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.479 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:18:56.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.479 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:56.479 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:56.479 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:56.479 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.479 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:18:56.479 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:18:56.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:18:56.480 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:18:56.480 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:18:56.480 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:18:56.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:18:56.480 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:18:56.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:18:56.480 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:18:56.480 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:18:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.458+0000 7f8e3a800640 1 -- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e1406b4b0 msgr2=0x7f8e1406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.458+0000 7f8e3a800640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e1406b4b0 0x7f8e1406d970 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8e20006fd0 tx=0x7f8e20006d80 comp rx=0 tx=0).stop 2024-03-09T09:18:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.458+0000 7f8e3a800640 1 -- 172.21.15.67:0/3784600088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3414fd90 msgr2=0x7f8e341e0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:18:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.458+0000 7f8e3a800640 1 --2- 172.21.15.67:0/3784600088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e3414fd90 0x7f8e341e0b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009f40 tx=0x7f8e2c004a40 comp rx=0 tx=0).stop 2024-03-09T09:18:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.458+0000 7f8e39e00640 1 -- 172.21.15.67:0/3784600088 reap_dead start 2024-03-09T09:18:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.458+0000 7f8e39e00640 1 -- 172.21.15.67:0/3784600088 reap_dead start 2024-03-09T09:18:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.458+0000 7f8e3a800640 1 -- 172.21.15.67:0/3784600088 shutdown_connections 2024-03-09T09:18:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.458+0000 7f8e3a800640 1 -- 172.21.15.67:0/3784600088 >> 172.21.15.67:0/3784600088 conn(0x7f8e340cb480 msgr2=0x7f8e340c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:18:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.458+0000 7f8e3a800640 1 -- 172.21.15.67:0/3784600088 shutdown_connections 2024-03-09T09:18:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:18:56.458+0000 7f8e3a800640 1 -- 172.21.15.67:0/3784600088 wait complete. 2024-03-09T09:18:56.488 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:18:56.488 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:18:57.757 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:18:57.757 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:18:57.757 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 30 -eq 48 ']' 2024-03-09T09:18:57.758 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.30 2024-03-09T09:18:57.758 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.30 2024-03-09T09:18:57.758 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.30 2024-03-09T09:18:57.758 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.30 2024-03-09T09:18:57.762 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:18:58.871 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:18:58.879 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:18:59.471 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:18:59.471 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:18:59.471 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 25 -eq 48 ']' 2024-03-09T09:18:59.471 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.25 2024-03-09T09:18:59.471 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.25 2024-03-09T09:18:59.472 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.25 2024-03-09T09:18:59.472 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.25 2024-03-09T09:18:59.474 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:01.229 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:01.229 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:01.229 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 31 -eq 48 ']' 2024-03-09T09:19:01.229 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.31 2024-03-09T09:19:01.229 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.31 2024-03-09T09:19:01.230 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.31 2024-03-09T09:19:01.230 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.31 2024-03-09T09:19:01.233 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:02.338 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:02.338 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:02.338 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 26 -eq 48 ']' 2024-03-09T09:19:02.338 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.26 2024-03-09T09:19:02.339 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.26 2024-03-09T09:19:02.339 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.26 2024-03-09T09:19:02.339 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.26 2024-03-09T09:19:02.342 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:03.809 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:03.810 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:03.810 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 32 -eq 48 ']' 2024-03-09T09:19:03.811 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.32 2024-03-09T09:19:03.811 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.32 2024-03-09T09:19:03.811 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.32 2024-03-09T09:19:03.811 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.32 2024-03-09T09:19:03.815 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:06.657 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:06.657 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:06.657 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 27 -eq 48 ']' 2024-03-09T09:19:06.657 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.27 2024-03-09T09:19:06.657 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.27 2024-03-09T09:19:06.658 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.27 2024-03-09T09:19:06.658 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.27 2024-03-09T09:19:06.662 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:07.349 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:07.350 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:07.350 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 33 -eq 48 ']' 2024-03-09T09:19:07.350 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.33 2024-03-09T09:19:07.350 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.33 2024-03-09T09:19:07.351 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.33 2024-03-09T09:19:07.351 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.33 2024-03-09T09:19:07.354 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:11.895 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:11.895 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:11.895 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 34 -eq 48 ']' 2024-03-09T09:19:11.895 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.34 2024-03-09T09:19:11.895 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.34 2024-03-09T09:19:11.896 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.34 2024-03-09T09:19:11.896 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.34 2024-03-09T09:19:11.897 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:11.897 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:11.897 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 28 -eq 48 ']' 2024-03-09T09:19:11.897 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.28 2024-03-09T09:19:11.897 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.28 2024-03-09T09:19:11.897 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.28 2024-03-09T09:19:11.897 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.28 2024-03-09T09:19:11.899 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:11.899 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:15.335 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:15.335 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:15.335 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 29 -eq 48 ']' 2024-03-09T09:19:15.335 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.29 2024-03-09T09:19:15.335 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.29 2024-03-09T09:19:15.336 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.29 2024-03-09T09:19:15.336 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.29 2024-03-09T09:19:15.341 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:15.778 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:15.778 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:15.778 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 35 -eq 48 ']' 2024-03-09T09:19:15.779 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.35 2024-03-09T09:19:15.779 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.35 2024-03-09T09:19:15.779 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.35 2024-03-09T09:19:15.779 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.35 2024-03-09T09:19:15.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:16.490 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:19:16.490 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:19:16.490 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:19:16.490 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:19:16.498 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:19:16.498 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:19:16.499 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:19:16.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.719+0000 7ff3fce00640 1 Processor -- start 2024-03-09T09:19:16.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.720+0000 7ff3fce00640 1 -- start start 2024-03-09T09:19:16.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.721+0000 7ff3fce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 0x7ff3f806e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:16.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.721+0000 7ff3fce00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f8151b50 con 0x7ff3f80a73e0 2024-03-09T09:19:16.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.721+0000 7ff3fce00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f8154dc0 con 0x7ff3f80c2a80 2024-03-09T09:19:16.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.721+0000 7ff3fce00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f8154930 con 0x7ff3f80c2630 2024-03-09T09:19:16.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.721+0000 7ff3f5e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff3f80a73e0 0x7ff3f8066d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50072/0 (socket says 172.21.15.67:50072) 2024-03-09T09:19:16.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.721+0000 7ff3f5e00640 1 -- 172.21.15.67:0/2982386497 learned_addr learned my addr 172.21.15.67:0/2982386497 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:19:16.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.721+0000 7ff3efe00640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 0x7ff3f806e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:16.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.721+0000 7ff3efe00640 1 -- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 msgr2=0x7ff3f806e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.721+0000 7ff3efe00640 1 -- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 msgr2=0x7ff3f806e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.721+0000 7ff3efe00640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 0x7ff3f806e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3efe00640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 0x7ff3f806e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780906271 0 0) 0x7ff3f8154dc0 con 0x7ff3f80c2a80 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3dc003520 con 0x7ff3f80c2a80 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356644346 0 0) 0x7ff3f8151b50 con 0x7ff3f80a73e0 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f8154dc0 con 0x7ff3f80a73e0 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 981342826 0 0) 0x7ff3f8154dc0 con 0x7ff3f80a73e0 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 msgr2=0x7ff3f806e630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 0x7ff3f806e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 >> v1:172.21.15.134:6789/0 conn(0x7ff3f80c2a80 legacy=0x7ff3f815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:16.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f80734b0 con 0x7ff3f80a73e0 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7ff3e80015f0 con 0x7ff3f80a73e0 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 >> v1:172.21.15.67:6789/0 conn(0x7ff3f80a73e0 legacy=0x7ff3f8066d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3dc004c00 0x7ff3dc004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 0x7ff3dc0037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3dc003ce0 0x7ff3dc004160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc00fe80 con 0x7ff3dc004c00 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc010030 con 0x7ff3dc003ce0 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc010230 con 0x7ff3f80c2630 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.722+0000 7ff3f5e00640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3dc004c00 0x7ff3dc004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.723+0000 7ff3efe00640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 0x7ff3dc0037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.723+0000 7ff3f6800640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3dc003ce0 0x7ff3dc004160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.723+0000 7ff3f5e00640 1 -- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 msgr2=0x7ff3dc0037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.723+0000 7ff3f5e00640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f80c2630 0x7ff3dc0037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:16.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.723+0000 7ff3f5e00640 1 -- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3dc003ce0 msgr2=0x7ff3dc004160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:16.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.723+0000 7ff3f5e00640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3dc003ce0 0x7ff3dc004160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:16.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.723+0000 7ff3f5e00640 1 -- 172.21.15.67:0/2982386497 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f8154dc0 con 0x7ff3dc004c00 2024-03-09T09:19:16.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.724+0000 7ff3f5e00640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3dc004c00 0x7ff3dc004fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c160 tx=0x7ff3dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c8932dcf4c21d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:16.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.724+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8007c40 con 0x7ff3dc004c00 2024-03-09T09:19:16.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.724+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e8030070 con 0x7ff3dc004c00 2024-03-09T09:19:16.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.724+0000 7ff3f5400640 1 -- 172.21.15.67:0/2982386497 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8005d30 con 0x7ff3dc004c00 2024-03-09T09:19:16.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.725+0000 7ff3fce00640 1 -- 172.21.15.67:0/2982386497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3dc004c00 msgr2=0x7ff3dc004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:16.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.725+0000 7ff3fce00640 1 --2- 172.21.15.67:0/2982386497 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3dc004c00 0x7ff3dc004fe0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800c160 tx=0x7ff3dc005830 comp rx=0 tx=0).stop 2024-03-09T09:19:16.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.725+0000 7ff3f6800640 1 -- 172.21.15.67:0/2982386497 reap_dead start 2024-03-09T09:19:16.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.725+0000 7ff3fce00640 1 -- 172.21.15.67:0/2982386497 shutdown_connections 2024-03-09T09:19:16.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.725+0000 7ff3fce00640 1 -- 172.21.15.67:0/2982386497 >> 172.21.15.67:0/2982386497 conn(0x7ff3f80a9950 msgr2=0x7ff3f80a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:16.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.725+0000 7ff3fce00640 1 -- 172.21.15.67:0/2982386497 shutdown_connections 2024-03-09T09:19:16.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.725+0000 7ff3fce00640 1 -- 172.21.15.67:0/2982386497 wait complete. 2024-03-09T09:19:16.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.727+0000 7ff3fce00640 1 Processor -- start 2024-03-09T09:19:16.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.727+0000 7ff3fce00640 1 -- start start 2024-03-09T09:19:16.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.728+0000 7ff3fce00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3dc004c00 0x7ff3f80a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:16.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.728+0000 7ff3fce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3f81508c0 0x7ff3f814aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:16.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.728+0000 7ff3fce00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f8148e80 0x7ff3f8149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:16.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.728+0000 7ff3fce00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f806f450 con 0x7ff3f81508c0 2024-03-09T09:19:16.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.728+0000 7ff3fce00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f806f170 con 0x7ff3dc004c00 2024-03-09T09:19:16.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.728+0000 7ff3fce00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f8073720 con 0x7ff3f8148e80 2024-03-09T09:19:16.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.728+0000 7ff3efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3f81508c0 0x7ff3f814aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:16.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3efe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3f81508c0 0x7ff3f814aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59140/0 (socket says 172.21.15.67:59140) 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3efe00640 1 -- 172.21.15.67:0/1522534847 learned_addr learned my addr 172.21.15.67:0/1522534847 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3f6800640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f8148e80 0x7ff3f8149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3f5e00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3dc004c00 0x7ff3f80a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3f6800640 1 -- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f8148e80 msgr2=0x7ff3f8149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3f6800640 1 -- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f8148e80 msgr2=0x7ff3f8149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3f6800640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f8148e80 0x7ff3f8149330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3f6800640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f8148e80 0x7ff3f8149330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3efe00640 1 -- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f8148e80 msgr2=0x7ff3f8149330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3efe00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f8148e80 0x7ff3f8149330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3efe00640 1 -- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3dc004c00 msgr2=0x7ff3f80a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3efe00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3dc004c00 0x7ff3f80a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.729+0000 7ff3efe00640 1 -- 172.21.15.67:0/1522534847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f814a610 con 0x7ff3f81508c0 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.730+0000 7ff3efe00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3f81508c0 0x7ff3f814aef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000bfd0 tx=0x7ff3e0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:16.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.730+0000 7ff3ee000640 1 -- 172.21.15.67:0/1522534847 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e001a020 con 0x7ff3f81508c0 2024-03-09T09:19:16.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.730+0000 7ff3ee000640 1 -- 172.21.15.67:0/1522534847 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e000e9c0 con 0x7ff3f81508c0 2024-03-09T09:19:16.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.730+0000 7ff3fce00640 1 -- 172.21.15.67:0/1522534847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f81504f0 con 0x7ff3f81508c0 2024-03-09T09:19:16.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.730+0000 7ff3ee000640 1 -- 172.21.15.67:0/1522534847 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e000edb0 con 0x7ff3f81508c0 2024-03-09T09:19:16.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.730+0000 7ff3fce00640 1 -- 172.21.15.67:0/1522534847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f814a0f0 con 0x7ff3f81508c0 2024-03-09T09:19:16.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.731+0000 7ff3fce00640 1 -- 172.21.15.67:0/1522534847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f81e7900 con 0x7ff3f81508c0 2024-03-09T09:19:16.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.733+0000 7ff3ee000640 1 -- 172.21.15.67:0/1522534847 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff3e001d020 con 0x7ff3f81508c0 2024-03-09T09:19:16.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.735+0000 7ff3ee000640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff3d406b4d0 0x7ff3d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:16.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.735+0000 7ff3ee000640 1 -- 172.21.15.67:0/1522534847 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff3e008aea0 con 0x7ff3f81508c0 2024-03-09T09:19:16.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.735+0000 7ff3f5e00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff3d406b4d0 0x7ff3d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:16.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.736+0000 7ff3f5e00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff3d406b4d0 0x7ff3d406d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8034000 tx=0x7ff3e8009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:16.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:16.740+0000 7ff3ee000640 1 -- 172.21.15.67:0/1522534847 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff3f81e7900 con 0x7ff3f81508c0 2024-03-09T09:19:17.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.017+0000 7ff3fce00640 1 -- 172.21.15.67:0/1522534847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff3f8075a00 con 0x7ff3f81508c0 2024-03-09T09:19:17.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.018+0000 7ff3ee000640 1 -- 172.21.15.67:0/1522534847 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7ff3e0024020 con 0x7ff3f81508c0 2024-03-09T09:19:17.024 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":198,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:19:17.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.020+0000 7ff3fce00640 1 -- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff3d406b4d0 msgr2=0x7ff3d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:17.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.020+0000 7ff3fce00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff3d406b4d0 0x7ff3d406d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8034000 tx=0x7ff3e8009030 comp rx=0 tx=0).stop 2024-03-09T09:19:17.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.020+0000 7ff3fce00640 1 -- 172.21.15.67:0/1522534847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3f81508c0 msgr2=0x7ff3f814aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:17.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.020+0000 7ff3fce00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3f81508c0 0x7ff3f814aef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000bfd0 tx=0x7ff3e0010d50 comp rx=0 tx=0).stop 2024-03-09T09:19:17.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.021+0000 7ff3fce00640 1 -- 172.21.15.67:0/1522534847 shutdown_connections 2024-03-09T09:19:17.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.021+0000 7ff3fce00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff3d406b4d0 0x7ff3d406d990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:17.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.021+0000 7ff3fce00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3f8148e80 0x7ff3f8149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:17.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.021+0000 7ff3fce00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3f81508c0 0x7ff3f814aef0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:17.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.021+0000 7ff3fce00640 1 --2- 172.21.15.67:0/1522534847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3dc004c00 0x7ff3f80a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:17.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.021+0000 7ff3fce00640 1 -- 172.21.15.67:0/1522534847 >> 172.21.15.67:0/1522534847 conn(0x7ff3f80a9950 msgr2=0x7ff3f80a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:17.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.021+0000 7ff3fce00640 1 -- 172.21.15.67:0/1522534847 shutdown_connections 2024-03-09T09:19:17.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:17.021+0000 7ff3fce00640 1 -- 172.21.15.67:0/1522534847 wait complete. 2024-03-09T09:19:17.669 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:17.669 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:17.669 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 30 -eq 48 ']' 2024-03-09T09:19:17.670 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.30 2024-03-09T09:19:17.670 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.30 2024-03-09T09:19:17.670 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.30 2024-03-09T09:19:17.670 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.30 2024-03-09T09:19:17.673 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:19.450 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:19.451 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:19.451 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 36 -eq 48 ']' 2024-03-09T09:19:19.451 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.36 2024-03-09T09:19:19.451 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.36 2024-03-09T09:19:19.451 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.36 2024-03-09T09:19:19.451 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.36 2024-03-09T09:19:19.456 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:20.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:19:20.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.267+0000 7f1e0e800640 1 Processor -- start 2024-03-09T09:19:20.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.268+0000 7f1e0e800640 1 -- start start 2024-03-09T09:19:20.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.270+0000 7f1e0e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 0x7f1e080b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.271+0000 7f1e0e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1e080cadb0 0x7f1e080b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.271+0000 7f1e0e800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e08157000 con 0x7f1e080c5e40 2024-03-09T09:19:20.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.271+0000 7f1e0e800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e080ae430 con 0x7f1e080cadb0 2024-03-09T09:19:20.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.271+0000 7f1e0e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080ae6f0 con 0x7f1e08153300 2024-03-09T09:19:20.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.271+0000 7f1e0de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1e080cadb0 0x7f1e080b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.271+0000 7f1e0de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1e080cadb0 0x7f1e080b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34582/0 (socket says 172.21.15.67:34582) 2024-03-09T09:19:20.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.271+0000 7f1e0de00640 1 -- 172.21.15.67:0/2735777830 learned_addr learned my addr 172.21.15.67:0/2735777830 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:20.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.271+0000 7f1e0d400640 1 --2- 172.21.15.67:0/2735777830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 0x7f1e080b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.272+0000 7f1e0d400640 1 -- 172.21.15.67:0/2735777830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 msgr2=0x7f1e080b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:19:20.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.272+0000 7f1e0d400640 1 -- 172.21.15.67:0/2735777830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 msgr2=0x7f1e080b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:20.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.272+0000 7f1e0d400640 1 --2- 172.21.15.67:0/2735777830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 0x7f1e080b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:20.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.272+0000 7f1e0d400640 1 --2- 172.21.15.67:0/2735777830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 0x7f1e080b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:20.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.272+0000 7f1e0de00640 1 -- 172.21.15.67:0/2735777830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 msgr2=0x7f1e080b0000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:20.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.272+0000 7f1e0de00640 1 --2- 172.21.15.67:0/2735777830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 0x7f1e080b0000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.272+0000 7f1e0de00640 1 -- 172.21.15.67:0/2735777830 >> v1:172.21.15.134:6789/0 conn(0x7f1e080c5e40 legacy=0x7f1e08156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.272+0000 7f1e0de00640 1 -- 172.21.15.67:0/2735777830 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e08155b10 con 0x7f1e080cadb0 2024-03-09T09:19:20.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.273+0000 7f1e0de00640 1 --2- 172.21.15.67:0/2735777830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e080cadb0 0x7f1e080b0540 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009870 tx=0x7f1e0002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f413776983bf4d7e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:20.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.273+0000 7f1e07e00640 1 -- 172.21.15.67:0/2735777830 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00034070 con 0x7f1e080cadb0 2024-03-09T09:19:20.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.273+0000 7f1e07e00640 1 -- 172.21.15.67:0/2735777830 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e00030840 con 0x7f1e080cadb0 2024-03-09T09:19:20.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.273+0000 7f1e07e00640 1 -- 172.21.15.67:0/2735777830 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00030c10 con 0x7f1e080cadb0 2024-03-09T09:19:20.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.274+0000 7f1e0e800640 1 -- 172.21.15.67:0/2735777830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e080cadb0 msgr2=0x7f1e080b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.274+0000 7f1e0e800640 1 --2- 172.21.15.67:0/2735777830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e080cadb0 0x7f1e080b0540 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1e00009870 tx=0x7f1e0002b980 comp rx=0 tx=0).stop 2024-03-09T09:19:20.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.274+0000 7f1e0e800640 1 -- 172.21.15.67:0/2735777830 shutdown_connections 2024-03-09T09:19:20.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.274+0000 7f1e0e800640 1 --2- 172.21.15.67:0/2735777830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e080cadb0 0x7f1e080b0540 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.274+0000 7f1e0e800640 1 --2- 172.21.15.67:0/2735777830 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 0x7f1e080b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.274+0000 7f1e0e800640 1 -- 172.21.15.67:0/2735777830 >> 172.21.15.67:0/2735777830 conn(0x7f1e080732d0 msgr2=0x7f1e080ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:20.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.274+0000 7f1e0e800640 1 -- 172.21.15.67:0/2735777830 shutdown_connections 2024-03-09T09:19:20.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.274+0000 7f1e0e800640 1 -- 172.21.15.67:0/2735777830 wait complete. 2024-03-09T09:19:20.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.276+0000 7f1e0e800640 1 Processor -- start 2024-03-09T09:19:20.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.276+0000 7f1e0e800640 1 -- start start 2024-03-09T09:19:20.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.277+0000 7f1e0e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e080c5e40 0x7f1e081ee860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.277+0000 7f1e0e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e080cadb0 0x7f1e081584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.277+0000 7f1e0e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 0x7f1e080ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.277+0000 7f1e0e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080c59e0 con 0x7f1e080cadb0 2024-03-09T09:19:20.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.277+0000 7f1e0e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e08159e90 con 0x7f1e080c5e40 2024-03-09T09:19:20.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.277+0000 7f1e0e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e08156720 con 0x7f1e08153300 2024-03-09T09:19:20.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.277+0000 7f1e0ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e080cadb0 0x7f1e081584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.277+0000 7f1e0ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e080cadb0 0x7f1e081584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34596/0 (socket says 172.21.15.67:34596) 2024-03-09T09:19:20.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.277+0000 7f1e0ca00640 1 -- 172.21.15.67:0/3010646636 learned_addr learned my addr 172.21.15.67:0/3010646636 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.277+0000 7f1e0d400640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e080c5e40 0x7f1e081ee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.277+0000 7f1e0de00640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 0x7f1e080ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.278+0000 7f1e0ca00640 1 -- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 msgr2=0x7f1e080ab560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.278+0000 7f1e0ca00640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 0x7f1e080ab560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.278+0000 7f1e0ca00640 1 -- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e080c5e40 msgr2=0x7f1e081ee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.278+0000 7f1e0ca00640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e080c5e40 0x7f1e081ee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.278+0000 7f1e0ca00640 1 -- 172.21.15.67:0/3010646636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e080acb50 con 0x7f1e080cadb0 2024-03-09T09:19:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.278+0000 7f1e0ca00640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e080cadb0 0x7f1e081584e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c450 tx=0x7f1df400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.278+0000 7f1e05600640 1 -- 172.21.15.67:0/3010646636 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40052d0 con 0x7f1e080cadb0 2024-03-09T09:19:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.278+0000 7f1e05600640 1 -- 172.21.15.67:0/3010646636 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df4005470 con 0x7f1e080cadb0 2024-03-09T09:19:20.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.278+0000 7f1e0e800640 1 -- 172.21.15.67:0/3010646636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e080c7f50 con 0x7f1e080cadb0 2024-03-09T09:19:20.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.278+0000 7f1e05600640 1 -- 172.21.15.67:0/3010646636 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4012870 con 0x7f1e080cadb0 2024-03-09T09:19:20.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.278+0000 7f1e0e800640 1 -- 172.21.15.67:0/3010646636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e080c7280 con 0x7f1e080cadb0 2024-03-09T09:19:20.288 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:20.288 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:20.288 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 31 -eq 48 ']' 2024-03-09T09:19:20.288 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.31 2024-03-09T09:19:20.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.280+0000 7f1e0e800640 1 -- 172.21.15.67:0/3010646636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0804f3c0 con 0x7f1e080cadb0 2024-03-09T09:19:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.281+0000 7f1e05600640 1 -- 172.21.15.67:0/3010646636 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1df4012a10 con 0x7f1e080cadb0 2024-03-09T09:19:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.281+0000 7f1e05600640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1de806b4d0 0x7f1de806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.282+0000 7f1e05600640 1 -- 172.21.15.67:0/3010646636 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f1df408a1f0 con 0x7f1e080cadb0 2024-03-09T09:19:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.282+0000 7f1e0d400640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1de806b4d0 0x7f1de806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.289 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.31 2024-03-09T09:19:20.289 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.31 2024-03-09T09:19:20.289 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.31 2024-03-09T09:19:20.289 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:20.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.283+0000 7f1e0d400640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1de806b4d0 0x7f1de806d990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009610 tx=0x7f1dfc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:20.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.287+0000 7f1e05600640 1 -- 172.21.15.67:0/3010646636 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1e080acb50 con 0x7f1e080cadb0 2024-03-09T09:19:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.604+0000 7f1e0e800640 1 -- 172.21.15.67:0/3010646636 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1e080cab90 con 0x7f1e080cadb0 2024-03-09T09:19:20.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.605+0000 7f1e05600640 1 -- 172.21.15.67:0/3010646636 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f1df4058d10 con 0x7f1e080cadb0 2024-03-09T09:19:20.612 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":200,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:19:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.608+0000 7f1e0e800640 1 -- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1de806b4d0 msgr2=0x7f1de806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.608+0000 7f1e0e800640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1de806b4d0 0x7f1de806d990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009610 tx=0x7f1dfc00b040 comp rx=0 tx=0).stop 2024-03-09T09:19:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.608+0000 7f1e0e800640 1 -- 172.21.15.67:0/3010646636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e080cadb0 msgr2=0x7f1e081584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.608+0000 7f1e0e800640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e080cadb0 0x7f1e081584e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c450 tx=0x7f1df400c910 comp rx=0 tx=0).stop 2024-03-09T09:19:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.608+0000 7f1e0e800640 1 -- 172.21.15.67:0/3010646636 shutdown_connections 2024-03-09T09:19:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.608+0000 7f1e0e800640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1de806b4d0 0x7f1de806d990 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.609+0000 7f1e0e800640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e08153300 0x7f1e080ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.609+0000 7f1e0e800640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e080cadb0 0x7f1e081584e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.609+0000 7f1e0e800640 1 --2- 172.21.15.67:0/3010646636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e080c5e40 0x7f1e081ee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.609+0000 7f1e0e800640 1 -- 172.21.15.67:0/3010646636 >> 172.21.15.67:0/3010646636 conn(0x7f1e080732d0 msgr2=0x7f1e08151a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:20.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.609+0000 7f1e0e800640 1 -- 172.21.15.67:0/3010646636 shutdown_connections 2024-03-09T09:19:20.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.609+0000 7f1e0e800640 1 -- 172.21.15.67:0/3010646636 wait complete. 2024-03-09T09:19:20.636 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:19:20.636 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:19:20.636 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:19:20.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.869+0000 7fd6cbe00640 1 Processor -- start 2024-03-09T09:19:20.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.870+0000 7fd6cbe00640 1 -- start start 2024-03-09T09:19:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.871+0000 7fd6cbe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd6cc0c61e0 0x7fd6cc0c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.871+0000 7fd6cbe00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc0c6630 con 0x7fd6cc0c5e00 2024-03-09T09:19:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.871+0000 7fd6cbe00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc0ad7c0 con 0x7fd6cc0acf00 2024-03-09T09:19:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.871+0000 7fd6cbe00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6cc06d740 con 0x7fd6cc0c61e0 2024-03-09T09:19:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.872+0000 7fd6caa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd6cc0c5e00 0x7fd6cc06c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49142/0 (socket says 172.21.15.67:49142) 2024-03-09T09:19:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.872+0000 7fd6caa00640 1 -- 172.21.15.67:0/3451609378 learned_addr learned my addr 172.21.15.67:0/3451609378 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:19:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.872+0000 7fd6cb400640 1 --2- 172.21.15.67:0/3451609378 >> v2:172.21.15.134:3300/0 conn(0x7fd6cc0c61e0 0x7fd6cc0c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.872+0000 7fd6c9600640 1 -- 172.21.15.67:0/3451609378 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069676525 0 0) 0x7fd6bc001110 con 0x7fd6cc0c5e00 2024-03-09T09:19:20.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.872+0000 7fd6c9600640 1 -- 172.21.15.67:0/3451609378 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b0003520 con 0x7fd6cc0c5e00 2024-03-09T09:19:20.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.873+0000 7fd6cb400640 1 -- 172.21.15.67:0/3451609378 >> v1:172.21.15.134:6789/0 conn(0x7fd6cc0acf00 legacy=0x7fd6cc15e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.873+0000 7fd6cb400640 1 -- 172.21.15.67:0/3451609378 >> v1:172.21.15.67:6789/0 conn(0x7fd6cc0c5e00 legacy=0x7fd6cc06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.873+0000 7fd6cb400640 1 -- 172.21.15.67:0/3451609378 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc0ad4e0 con 0x7fd6cc0c61e0 2024-03-09T09:19:20.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.874+0000 7fd6cb400640 1 --2- 172.21.15.67:0/3451609378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6cc0c61e0 0x7fd6cc0c9380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009dc0 tx=0x7fd6c402be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eaacc46c3e7c2a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:20.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.875+0000 7fd6c9600640 1 -- 172.21.15.67:0/3451609378 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4039070 con 0x7fd6cc0c61e0 2024-03-09T09:19:20.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.875+0000 7fd6c9600640 1 -- 172.21.15.67:0/3451609378 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c402cc40 con 0x7fd6cc0c61e0 2024-03-09T09:19:20.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.875+0000 7fd6c9600640 1 -- 172.21.15.67:0/3451609378 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4034440 con 0x7fd6cc0c61e0 2024-03-09T09:19:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.876+0000 7fd6cbe00640 1 -- 172.21.15.67:0/3451609378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6cc0c61e0 msgr2=0x7fd6cc0c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.876+0000 7fd6cbe00640 1 --2- 172.21.15.67:0/3451609378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6cc0c61e0 0x7fd6cc0c9380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009dc0 tx=0x7fd6c402be90 comp rx=0 tx=0).stop 2024-03-09T09:19:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.876+0000 7fd6cbe00640 1 -- 172.21.15.67:0/3451609378 shutdown_connections 2024-03-09T09:19:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.876+0000 7fd6cbe00640 1 --2- 172.21.15.67:0/3451609378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6cc0c61e0 0x7fd6cc0c9380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.876+0000 7fd6cbe00640 1 -- 172.21.15.67:0/3451609378 >> 172.21.15.67:0/3451609378 conn(0x7fd6cc0cb480 msgr2=0x7fd6cc0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:20.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.876+0000 7fd6cbe00640 1 -- 172.21.15.67:0/3451609378 shutdown_connections 2024-03-09T09:19:20.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.876+0000 7fd6cbe00640 1 -- 172.21.15.67:0/3451609378 wait complete. 2024-03-09T09:19:20.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.879+0000 7fd6cbe00640 1 Processor -- start 2024-03-09T09:19:20.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.879+0000 7fd6cbe00640 1 -- start start 2024-03-09T09:19:20.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.880+0000 7fd6cbe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc0acf00 0x7fd6cc1ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.880+0000 7fd6cbe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6cc0c5e00 0x7fd6cc1505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.881+0000 7fd6cbe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6cc0c61e0 0x7fd6cc150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.881+0000 7fd6cbe00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc06bde0 con 0x7fd6cc0acf00 2024-03-09T09:19:20.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.881+0000 7fd6cbe00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc153680 con 0x7fd6cc0c61e0 2024-03-09T09:19:20.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.881+0000 7fd6cbe00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc06c0c0 con 0x7fd6cc0c5e00 2024-03-09T09:19:20.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.881+0000 7fd6caa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc0acf00 0x7fd6cc1ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.881+0000 7fd6caa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc0acf00 0x7fd6cc1ef2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34608/0 (socket says 172.21.15.67:34608) 2024-03-09T09:19:20.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.881+0000 7fd6caa00640 1 -- 172.21.15.67:0/1907550167 learned_addr learned my addr 172.21.15.67:0/1907550167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:20.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.881+0000 7fd6ca000640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6cc0c5e00 0x7fd6cc1505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.881+0000 7fd6cb400640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6cc0c61e0 0x7fd6cc150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.881+0000 7fd6caa00640 1 -- 172.21.15.67:0/1907550167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6cc0c5e00 msgr2=0x7fd6cc1505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.881+0000 7fd6caa00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6cc0c5e00 0x7fd6cc1505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.882+0000 7fd6caa00640 1 -- 172.21.15.67:0/1907550167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6cc0c61e0 msgr2=0x7fd6cc150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.882+0000 7fd6caa00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6cc0c61e0 0x7fd6cc150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.882+0000 7fd6caa00640 1 -- 172.21.15.67:0/1907550167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc1efae0 con 0x7fd6cc0acf00 2024-03-09T09:19:20.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.882+0000 7fd6caa00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc0acf00 0x7fd6cc1ef2a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c760 tx=0x7fd6bc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:20.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.882+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc017070 con 0x7fd6cc0acf00 2024-03-09T09:19:20.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.882+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc005ab0 con 0x7fd6cc0acf00 2024-03-09T09:19:20.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.882+0000 7fd6cbe00640 1 -- 172.21.15.67:0/1907550167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc1efdc0 con 0x7fd6cc0acf00 2024-03-09T09:19:20.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.882+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc007070 con 0x7fd6cc0acf00 2024-03-09T09:19:20.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.883+0000 7fd6cbe00640 1 -- 172.21.15.67:0/1907550167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc1ef7e0 con 0x7fd6cc0acf00 2024-03-09T09:19:20.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.884+0000 7fd6cbe00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156f80 0x7fd6cc0c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.884+0000 7fd6cbe00640 1 -- 172.21.15.67:0/1907550167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc04f3c0 con 0x7fd6cc156f80 2024-03-09T09:19:20.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.884+0000 7fd6ca000640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156f80 0x7fd6cc0c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.884+0000 7fd6ca000640 1 -- 172.21.15.67:0/1907550167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6cc06d2f0 con 0x7fd6cc156f80 2024-03-09T09:19:20.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.885+0000 7fd6ca000640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156f80 0x7fd6cc0c08c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0c1e00 tx=0x7fd6b800e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:20.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.885+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8019070 con 0x7fd6cc156f80 2024-03-09T09:19:20.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.885+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd6bc01a070 con 0x7fd6cc0acf00 2024-03-09T09:19:20.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.886+0000 7fd6c2a00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd6ac06b3d0 0x7fd6ac06d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.886+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd6bc0899f0 con 0x7fd6cc0acf00 2024-03-09T09:19:20.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.887+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd6b80052e0 con 0x7fd6cc156f80 2024-03-09T09:19:20.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.887+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156f80 msgr2=0x7fd6cc0c08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.887+0000 7fd6c2a00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156f80 0x7fd6cc0c08c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0c1e00 tx=0x7fd6b800e890 comp rx=0 tx=0).stop 2024-03-09T09:19:20.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.887+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156f80 msgr2=0x7fd6cc0c08c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:20.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.887+0000 7fd6c2a00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156f80 0x7fd6cc0c08c0 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0c1e00 tx=0x7fd6b800e890 comp rx=0 tx=0).stop 2024-03-09T09:19:20.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.887+0000 7fd6cb400640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd6ac06b3d0 0x7fd6ac06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.888+0000 7fd6cb400640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd6ac06b3d0 0x7fd6ac06d890 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4036000 tx=0x7fd6c4030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:20.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.897+0000 7fd6cbe00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156a50 0x7fd6cc0c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:20.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.897+0000 7fd6cbe00640 1 -- 172.21.15.67:0/1907550167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0af770 con 0x7fd6cc156a50 2024-03-09T09:19:20.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.897+0000 7fd6ca000640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156a50 0x7fd6cc0c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:20.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.897+0000 7fd6ca000640 1 -- 172.21.15.67:0/1907550167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd6cc0bff70 con 0x7fd6cc156a50 2024-03-09T09:19:20.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.898+0000 7fd6ca000640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156a50 0x7fd6cc0c0e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8013000 tx=0x7fd6b8001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:20.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.898+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b80037f0 con 0x7fd6cc156a50 2024-03-09T09:19:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.898+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7fd6b80037f0 con 0x7fd6cc156a50 2024-03-09T09:19:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.898+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156a50 msgr2=0x7fd6cc0c0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.898+0000 7fd6c2a00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156a50 0x7fd6cc0c0e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8013000 tx=0x7fd6b8001080 comp rx=0 tx=0).stop 2024-03-09T09:19:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.898+0000 7fd6c2a00640 1 -- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156a50 msgr2=0x7fd6cc0c0e00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.898+0000 7fd6c2a00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc156a50 0x7fd6cc0c0e00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.905 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:19:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:19:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 200, 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:19:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:19:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:19:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:19:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:19:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:19:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:19:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:19:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:19:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:19:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:19:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:19:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:19:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:19:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:19:20.919 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:19:20.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.899+0000 7fd6cbe00640 1 -- 172.21.15.67:0/1907550167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd6ac06b3d0 msgr2=0x7fd6ac06d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.899+0000 7fd6cbe00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd6ac06b3d0 0x7fd6ac06d890 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4036000 tx=0x7fd6c4030000 comp rx=0 tx=0).stop 2024-03-09T09:19:20.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.899+0000 7fd6cbe00640 1 -- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc0acf00 msgr2=0x7fd6cc1ef2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:20.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.899+0000 7fd6cbe00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc0acf00 0x7fd6cc1ef2a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c760 tx=0x7fd6bc00cc20 comp rx=0 tx=0).stop 2024-03-09T09:19:20.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.899+0000 7fd6cb400640 1 -- 172.21.15.67:0/1907550167 reap_dead start 2024-03-09T09:19:20.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.899+0000 7fd6cbe00640 1 -- 172.21.15.67:0/1907550167 shutdown_connections 2024-03-09T09:19:20.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.899+0000 7fd6cbe00640 1 --2- 172.21.15.67:0/1907550167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6cc0acf00 0x7fd6cc1ef2a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:20.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.899+0000 7fd6cbe00640 1 -- 172.21.15.67:0/1907550167 >> 172.21.15.67:0/1907550167 conn(0x7fd6cc0cb480 msgr2=0x7fd6cc151e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:20.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.899+0000 7fd6cbe00640 1 -- 172.21.15.67:0/1907550167 shutdown_connections 2024-03-09T09:19:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:20.899+0000 7fd6cbe00640 1 -- 172.21.15.67:0/1907550167 wait complete. 2024-03-09T09:19:20.926 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:19:20.926 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:19:20.926 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:19:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.124+0000 7fc5bea00640 1 Processor -- start 2024-03-09T09:19:21.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.124+0000 7fc5bea00640 1 -- start start 2024-03-09T09:19:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.126+0000 7fc5bea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 0x7fc5b8155f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.126+0000 7fc5bea00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b806f6d0 con 0x7fc5b8153300 2024-03-09T09:19:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.126+0000 7fc5bea00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b80c23f0 con 0x7fc5b8159ec0 2024-03-09T09:19:21.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.126+0000 7fc5bea00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b806e850 con 0x7fc5b806b2b0 2024-03-09T09:19:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5bca00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc5b8153300 0x7fc5b81607e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49150/0 (socket says 172.21.15.67:49150) 2024-03-09T09:19:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5bca00640 1 -- 172.21.15.67:0/3269550384 learned_addr learned my addr 172.21.15.67:0/3269550384 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:19:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b7e00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 0x7fc5b8155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886019529 0 0) 0x7fc5b806f6d0 con 0x7fc5b8153300 2024-03-09T09:19:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc59c003520 con 0x7fc5b8153300 2024-03-09T09:19:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b7e00640 1 -- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 msgr2=0x7fc5b8155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:19:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b7e00640 1 -- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 msgr2=0x7fc5b8155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b7e00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 0x7fc5b8155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b7e00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 0x7fc5b8155f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:21.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3095713519 0 0) 0x7fc59c003520 con 0x7fc5b8153300 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 msgr2=0x7fc5b8155f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b6a00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 0x7fc5b8155f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 >> v1:172.21.15.134:6789/0 conn(0x7fc5b8159ec0 legacy=0x7fc5b806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b80c26d0 con 0x7fc5b8153300 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.127+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fc5b00019f0 con 0x7fc5b8153300 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 >> v1:172.21.15.67:6789/0 conn(0x7fc5b8153300 legacy=0x7fc5b81607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5b6a00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 0x7fc59c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5b6a00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc59c003ff0 0x7fc59c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5b6a00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc59c004d90 0x7fc59c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b806e850 con 0x7fc59c004d90 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c00bc70 con 0x7fc59c003ff0 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c00be70 con 0x7fc5b806b2b0 2024-03-09T09:19:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5bca00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc59c004d90 0x7fc59c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5b7400640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc59c003ff0 0x7fc59c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5b7e00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 0x7fc59c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5bca00640 1 -- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 msgr2=0x7fc59c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5bca00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b806b2b0 0x7fc59c0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5bca00640 1 -- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc59c003ff0 msgr2=0x7fc59c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5bca00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc59c003ff0 0x7fc59c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5bca00640 1 -- 172.21.15.67:0/3269550384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c003520 con 0x7fc59c004d90 2024-03-09T09:19:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.128+0000 7fc5b7400640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc59c003ff0 0x7fc59c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:19:21.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.129+0000 7fc5bca00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc59c004d90 0x7fc59c005210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000a920 tx=0x7fc59c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8832c8375b4f2f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.130+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b000a4c0 con 0x7fc59c004d90 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.130+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b0034650 con 0x7fc59c004d90 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.130+0000 7fc5b6a00640 1 -- 172.21.15.67:0/3269550384 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0034a80 con 0x7fc59c004d90 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.130+0000 7fc5bea00640 1 -- 172.21.15.67:0/3269550384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc59c004d90 msgr2=0x7fc59c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.130+0000 7fc5bea00640 1 --2- 172.21.15.67:0/3269550384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc59c004d90 0x7fc59c005210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000a920 tx=0x7fc59c005830 comp rx=0 tx=0).stop 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.130+0000 7fc5bca00640 1 -- 172.21.15.67:0/3269550384 reap_dead start 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.130+0000 7fc5bea00640 1 -- 172.21.15.67:0/3269550384 shutdown_connections 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.131+0000 7fc5bea00640 1 -- 172.21.15.67:0/3269550384 >> 172.21.15.67:0/3269550384 conn(0x7fc5b80c8b20 msgr2=0x7fc5b80c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.131+0000 7fc5bea00640 1 -- 172.21.15.67:0/3269550384 shutdown_connections 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.131+0000 7fc5bea00640 1 -- 172.21.15.67:0/3269550384 wait complete. 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.132+0000 7fc5bea00640 1 Processor -- start 2024-03-09T09:19:21.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.132+0000 7fc5bea00640 1 -- start start 2024-03-09T09:19:21.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.133+0000 7fc5bea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc59c004d90 0x7fc5b80a9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.133+0000 7fc5bea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b81eafe0 0x7fc5b80a85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.133+0000 7fc5bea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b80a8b30 0x7fc5b8155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.133+0000 7fc5bea00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b80c57e0 con 0x7fc5b80a8b30 2024-03-09T09:19:21.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.133+0000 7fc5bea00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b806eb30 con 0x7fc59c004d90 2024-03-09T09:19:21.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.133+0000 7fc5bea00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b815a320 con 0x7fc5b81eafe0 2024-03-09T09:19:21.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5bca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b80a8b30 0x7fc5b8155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5bca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b80a8b30 0x7fc5b8155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34626/0 (socket says 172.21.15.67:34626) 2024-03-09T09:19:21.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5bca00640 1 -- 172.21.15.67:0/527270504 learned_addr learned my addr 172.21.15.67:0/527270504 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:21.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5b7400640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b81eafe0 0x7fc5b80a85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5b7e00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc59c004d90 0x7fc5b80a9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5b7400640 1 -- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b81eafe0 msgr2=0x7fc5b80a85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5b7400640 1 -- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b81eafe0 msgr2=0x7fc5b80a85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5b7400640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b81eafe0 0x7fc5b80a85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5b7400640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b81eafe0 0x7fc5b80a85f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5bca00640 1 -- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b81eafe0 msgr2=0x7fc5b80a85f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5bca00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5b81eafe0 0x7fc5b80a85f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5bca00640 1 -- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc59c004d90 msgr2=0x7fc5b80a9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5bca00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc59c004d90 0x7fc5b80a9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.134+0000 7fc5bca00640 1 -- 172.21.15.67:0/527270504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b814fa60 con 0x7fc5b80a8b30 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.135+0000 7fc5bca00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b80a8b30 0x7fc5b8155d00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc5b002cc80 tx=0x7fc5b0001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.135+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b000a4c0 con 0x7fc5b80a8b30 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.135+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b003a040 con 0x7fc5b80a8b30 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.135+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00019c0 con 0x7fc5b80a8b30 2024-03-09T09:19:21.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.135+0000 7fc5bea00640 1 -- 172.21.15.67:0/527270504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b814fd00 con 0x7fc5b80a8b30 2024-03-09T09:19:21.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.135+0000 7fc5bea00640 1 -- 172.21.15.67:0/527270504 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b81502c0 con 0x7fc5b80a8b30 2024-03-09T09:19:21.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.136+0000 7fc5bea00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b8156240 0x7fc5b80ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.136+0000 7fc5bea00640 1 -- 172.21.15.67:0/527270504 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b814e390 con 0x7fc5b8156240 2024-03-09T09:19:21.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.137+0000 7fc5b7e00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b8156240 0x7fc5b80ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.138+0000 7fc5b7e00640 1 -- 172.21.15.67:0/527270504 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5b8156620 con 0x7fc5b8156240 2024-03-09T09:19:21.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.138+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc5b0034650 con 0x7fc5b80a8b30 2024-03-09T09:19:21.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.138+0000 7fc5b7e00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b8156240 0x7fc5b80ad7f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8010790 tx=0x7fc5a8010c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:21.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.139+0000 7fc5afe00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc59806b500 0x7fc59806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.139+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc5b00ac720 con 0x7fc5b80a8b30 2024-03-09T09:19:21.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.139+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a801a020 con 0x7fc5b8156240 2024-03-09T09:19:21.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.139+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc5a800c9a0 con 0x7fc5b8156240 2024-03-09T09:19:21.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.139+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b8156240 msgr2=0x7fc5b80ad7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:21.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.139+0000 7fc5afe00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b8156240 0x7fc5b80ad7f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8010790 tx=0x7fc5a8010c50 comp rx=0 tx=0).stop 2024-03-09T09:19:21.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.139+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b8156240 msgr2=0x7fc5b80ad7f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:21.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.139+0000 7fc5afe00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b8156240 0x7fc5b80ad7f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.139+0000 7fc5b7400640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc59806b500 0x7fc59806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.140+0000 7fc5b7400640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc59806b500 0x7fc59806d9c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc5b81ef610 tx=0x7fc5a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:21.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.147+0000 7fc5bea00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b80add30 0x7fc5b81ee550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.147+0000 7fc5bea00640 1 -- 172.21.15.67:0/527270504 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b80ae140 con 0x7fc5b80add30 2024-03-09T09:19:21.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.148+0000 7fc5b7e00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b80add30 0x7fc5b81ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.148+0000 7fc5b7e00640 1 -- 172.21.15.67:0/527270504 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc5b8075a00 con 0x7fc5b80add30 2024-03-09T09:19:21.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.149+0000 7fc5b7e00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b80add30 0x7fc5b81ee550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8010790 tx=0x7fc5a80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:21.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.149+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a801a030 con 0x7fc5b80add30 2024-03-09T09:19:21.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.150+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4874 (secure 0 0 0) 0x7fc5a8015e00 con 0x7fc5b80add30 2024-03-09T09:19:21.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.150+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b80add30 msgr2=0x7fc5b81ee550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:21.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.150+0000 7fc5afe00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b80add30 0x7fc5b81ee550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8010790 tx=0x7fc5a80046c0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.150+0000 7fc5afe00640 1 -- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b80add30 msgr2=0x7fc5b81ee550 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:21.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.150+0000 7fc5afe00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b80add30 0x7fc5b81ee550 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8010790 tx=0x7fc5a80046c0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.157 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:19:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:19:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:19:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:19:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 200, 2024-03-09T09:19:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:19:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:19:21.157 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:21.158 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:21.159 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:21.160 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:21.161 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:19:21.162 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:21.163 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.164 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:21.165 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:21.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:21.168 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:21.169 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:21.170 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:19:21.170 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:19:21.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.151+0000 7fc5bea00640 1 -- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc59806b500 msgr2=0x7fc59806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:21.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.151+0000 7fc5bea00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc59806b500 0x7fc59806d9c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc5b81ef610 tx=0x7fc5a4009040 comp rx=0 tx=0).stop 2024-03-09T09:19:21.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.151+0000 7fc5bea00640 1 -- 172.21.15.67:0/527270504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b80a8b30 msgr2=0x7fc5b8155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:21.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.151+0000 7fc5bea00640 1 --2- 172.21.15.67:0/527270504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5b80a8b30 0x7fc5b8155d00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc5b002cc80 tx=0x7fc5b0001dc0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.151+0000 7fc5bca00640 1 -- 172.21.15.67:0/527270504 reap_dead start 2024-03-09T09:19:21.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.151+0000 7fc5bca00640 1 -- 172.21.15.67:0/527270504 reap_dead start 2024-03-09T09:19:21.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.151+0000 7fc5bea00640 1 -- 172.21.15.67:0/527270504 shutdown_connections 2024-03-09T09:19:21.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.151+0000 7fc5bea00640 1 -- 172.21.15.67:0/527270504 >> 172.21.15.67:0/527270504 conn(0x7fc5b80c8b20 msgr2=0x7fc5b80c60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:21.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.151+0000 7fc5bea00640 1 -- 172.21.15.67:0/527270504 shutdown_connections 2024-03-09T09:19:21.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.151+0000 7fc5bea00640 1 -- 172.21.15.67:0/527270504 wait complete. 2024-03-09T09:19:21.181 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:19:21.181 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:19:21.181 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:19:21.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.395+0000 7f063e000640 1 Processor -- start 2024-03-09T09:19:21.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.396+0000 7f063e000640 1 -- start start 2024-03-09T09:19:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.397+0000 7f063e000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06380c1c20 0x7f0638155d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.397+0000 7f063e000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f06380738b0 0x7f0638156290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.397+0000 7f063e000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063806de50 con 0x7f0638153200 2024-03-09T09:19:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.397+0000 7f063e000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06381535e0 con 0x7f06380738b0 2024-03-09T09:19:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.397+0000 7f063e000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0638159120 con 0x7f06380c1c20 2024-03-09T09:19:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.397+0000 7f063d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f06380738b0 0x7f0638156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.397+0000 7f063d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f06380738b0 0x7f0638156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34636/0 (socket says 172.21.15.67:34636) 2024-03-09T09:19:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.397+0000 7f063d600640 1 -- 172.21.15.67:0/1792630483 learned_addr learned my addr 172.21.15.67:0/1792630483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.397+0000 7f0637e00640 1 --2- 172.21.15.67:0/1792630483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06380c1c20 0x7f0638155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.398+0000 7f063d600640 1 -- 172.21.15.67:0/1792630483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06380c1c20 msgr2=0x7f0638155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.398+0000 7f063d600640 1 --2- 172.21.15.67:0/1792630483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06380c1c20 0x7f0638155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.398+0000 7f063d600640 1 -- 172.21.15.67:0/1792630483 >> v1:172.21.15.134:6789/0 conn(0x7f0638153200 legacy=0x7f06380aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:21.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.398+0000 7f063d600640 1 -- 172.21.15.67:0/1792630483 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638159370 con 0x7f06380738b0 2024-03-09T09:19:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.399+0000 7f063d600640 1 --2- 172.21.15.67:0/1792630483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06380738b0 0x7f0638156290 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0630009e50 tx=0x7f063002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0c4cfb44f6687b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.399+0000 7f0637400640 1 -- 172.21.15.67:0/1792630483 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063002c9e0 con 0x7f06380738b0 2024-03-09T09:19:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.399+0000 7f0637400640 1 -- 172.21.15.67:0/1792630483 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063002cb80 con 0x7f06380738b0 2024-03-09T09:19:21.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.399+0000 7f0637400640 1 -- 172.21.15.67:0/1792630483 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06300343f0 con 0x7f06380738b0 2024-03-09T09:19:21.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.400+0000 7f063e000640 1 -- 172.21.15.67:0/1792630483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06380738b0 msgr2=0x7f0638156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:21.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.400+0000 7f063e000640 1 --2- 172.21.15.67:0/1792630483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06380738b0 0x7f0638156290 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0630009e50 tx=0x7f063002be40 comp rx=0 tx=0).stop 2024-03-09T09:19:21.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.400+0000 7f063e000640 1 -- 172.21.15.67:0/1792630483 shutdown_connections 2024-03-09T09:19:21.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.400+0000 7f063e000640 1 --2- 172.21.15.67:0/1792630483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06380738b0 0x7f0638156290 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.400+0000 7f063e000640 1 --2- 172.21.15.67:0/1792630483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06380c1c20 0x7f0638155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.400+0000 7f063e000640 1 -- 172.21.15.67:0/1792630483 >> 172.21.15.67:0/1792630483 conn(0x7f06380c7220 msgr2=0x7f06380c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.400+0000 7f063e000640 1 -- 172.21.15.67:0/1792630483 shutdown_connections 2024-03-09T09:19:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.400+0000 7f063e000640 1 -- 172.21.15.67:0/1792630483 wait complete. 2024-03-09T09:19:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.402+0000 7f063e000640 1 Processor -- start 2024-03-09T09:19:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.402+0000 7f063e000640 1 -- start start 2024-03-09T09:19:21.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.403+0000 7f063e000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06380738b0 0x7f06380c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.403+0000 7f063e000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06380c1c20 0x7f0638152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.403+0000 7f063e000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0638153200 0x7f06381453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.403+0000 7f063e000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638159930 con 0x7f06380c1c20 2024-03-09T09:19:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.403+0000 7f063e000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638159f80 con 0x7f0638153200 2024-03-09T09:19:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.403+0000 7f063e000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0638159c10 con 0x7f06380738b0 2024-03-09T09:19:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.403+0000 7f0637e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06380c1c20 0x7f0638152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.403+0000 7f0637e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06380c1c20 0x7f0638152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34638/0 (socket says 172.21.15.67:34638) 2024-03-09T09:19:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.403+0000 7f0637e00640 1 -- 172.21.15.67:0/2898615884 learned_addr learned my addr 172.21.15.67:0/2898615884 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.404+0000 7f063d600640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0638153200 0x7f06381453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.404+0000 7f0637e00640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06380738b0 msgr2=0x7f06380c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.404+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06380738b0 0x7f06380c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.404+0000 7f0637e00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06380738b0 0x7f06380c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.404+0000 7f0637e00640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0638153200 msgr2=0x7f06381453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.404+0000 7f0637e00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0638153200 0x7f06381453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.404+0000 7f0637e00640 1 -- 172.21.15.67:0/2898615884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06381eb9f0 con 0x7f06380c1c20 2024-03-09T09:19:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.404+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06380738b0 0x7f06380c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:19:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.404+0000 7f0637e00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06380c1c20 0x7f0638152140 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f062800d5f0 tx=0x7f062800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.404+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06280125f0 con 0x7f06380c1c20 2024-03-09T09:19:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.405+0000 7f063e000640 1 -- 172.21.15.67:0/2898615884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06381ebcd0 con 0x7f06380c1c20 2024-03-09T09:19:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.405+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0628019070 con 0x7f06380c1c20 2024-03-09T09:19:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.405+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06280129d0 con 0x7f06380c1c20 2024-03-09T09:19:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.405+0000 7f063e000640 1 -- 172.21.15.67:0/2898615884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06381ec290 con 0x7f06380c1c20 2024-03-09T09:19:21.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.406+0000 7f063e000640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 0x7f06381458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.406+0000 7f063e000640 1 -- 172.21.15.67:0/2898615884 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0638145e30 con 0x7f0638152680 2024-03-09T09:19:21.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.406+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 0x7f06381458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.407+0000 7f063cc00640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 msgr2=0x7f06381458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-03-09T09:19:21.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.407+0000 7f063cc00640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 msgr2=0x7f06381458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:19:21.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.407+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 0x7f06381458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:21.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.407+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 0x7f06381458f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:21.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.407+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0628012b70 con 0x7f06380c1c20 2024-03-09T09:19:21.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.408+0000 7f0634c00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f061406b4d0 0x7f061406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:21.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.408+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f062808a9d0 con 0x7f06380c1c20 2024-03-09T09:19:21.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.409+0000 7f063d600640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f061406b4d0 0x7f061406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.410+0000 7f063d600640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f061406b4d0 0x7f061406d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0630035000 tx=0x7f0630030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:21.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.608+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 0x7f06381458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:21.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.609+0000 7f063cc00640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 msgr2=0x7f06381458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:19:21.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.609+0000 7f063cc00640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 msgr2=0x7f06381458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:19:21.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.609+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 0x7f06381458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:21.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:21.609+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 0x7f06381458f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:19:22.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.011+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 0x7f06381458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:22.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.012+0000 7f063cc00640 1 -- 172.21.15.67:0/2898615884 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f063804f3c0 con 0x7f0638152680 2024-03-09T09:19:22.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.013+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 0x7f06381458f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f06200107f0 tx=0x7f0620010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:22.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.014+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062001a030 con 0x7f0638152680 2024-03-09T09:19:22.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.015+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f062000a9f0 con 0x7f0638152680 2024-03-09T09:19:22.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.015+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 msgr2=0x7f06381458f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:22.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.015+0000 7f0634c00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 0x7f06381458f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f06200107f0 tx=0x7f0620010cb0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.015+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 msgr2=0x7f06381458f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:22.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.015+0000 7f0634c00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0638152680 0x7f06381458f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.034+0000 7f063e000640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06381509a0 0x7f0638150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:22.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.034+0000 7f063e000640 1 -- 172.21.15.67:0/2898615884 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06380c3b00 con 0x7f06381509a0 2024-03-09T09:19:22.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.035+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06381509a0 0x7f0638150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:22.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.036+0000 7f063cc00640 1 -- 172.21.15.67:0/2898615884 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f063806e330 con 0x7f06381509a0 2024-03-09T09:19:22.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.037+0000 7f063cc00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06381509a0 0x7f0638150db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f06200107f0 tx=0x7f06200046f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:22.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.037+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620020be0 con 0x7f06381509a0 2024-03-09T09:19:22.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.038+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f0620020be0 con 0x7f06381509a0 2024-03-09T09:19:22.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.039+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06381509a0 msgr2=0x7f0638150db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:22.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.039+0000 7f0634c00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06381509a0 0x7f0638150db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f06200107f0 tx=0x7f06200046f0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.039+0000 7f0634c00640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06381509a0 msgr2=0x7f0638150db0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:22.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.039+0000 7f0634c00640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06381509a0 0x7f0638150db0 secure :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f06200107f0 tx=0x7f06200046f0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.045 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:19:22.045 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 200, 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 4, 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:19:22.046 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:19:22.047 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.048 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:19:22.049 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:22.050 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:22.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:22.052 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:19:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:19:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:19:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:22.053 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:22.054 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:19:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:19:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:19:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:22.055 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:22.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:22.057 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:22.058 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:19:22.058 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:19:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.040+0000 7f063e000640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f061406b4d0 msgr2=0x7f061406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.040+0000 7f063e000640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f061406b4d0 0x7f061406d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0630035000 tx=0x7f0630030000 comp rx=0 tx=0).stop 2024-03-09T09:19:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.040+0000 7f063e000640 1 -- 172.21.15.67:0/2898615884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06380c1c20 msgr2=0x7f0638152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.040+0000 7f063e000640 1 --2- 172.21.15.67:0/2898615884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06380c1c20 0x7f0638152140 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f062800d5f0 tx=0x7f062800dab0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.040+0000 7f063d600640 1 -- 172.21.15.67:0/2898615884 reap_dead start 2024-03-09T09:19:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.040+0000 7f063d600640 1 -- 172.21.15.67:0/2898615884 reap_dead start 2024-03-09T09:19:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.040+0000 7f063e000640 1 -- 172.21.15.67:0/2898615884 shutdown_connections 2024-03-09T09:19:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.040+0000 7f063e000640 1 -- 172.21.15.67:0/2898615884 >> 172.21.15.67:0/2898615884 conn(0x7f06380c7220 msgr2=0x7f06380c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.040+0000 7f063e000640 1 -- 172.21.15.67:0/2898615884 shutdown_connections 2024-03-09T09:19:22.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.040+0000 7f063e000640 1 -- 172.21.15.67:0/2898615884 wait complete. 2024-03-09T09:19:22.067 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:19:22.067 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:19:22.068 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:19:22.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.295+0000 7fcde7200640 1 Processor -- start 2024-03-09T09:19:22.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.296+0000 7fcde7200640 1 -- start start 2024-03-09T09:19:22.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.297+0000 7fcde7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcde80c3bb0 0x7fcde80c7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:22.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.297+0000 7fcde7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcde806dee0 0x7fcde8159940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:22.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.297+0000 7fcde7200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde8156be0 con 0x7fcde806e820 2024-03-09T09:19:22.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.298+0000 7fcde7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde80c7700 con 0x7fcde80c3bb0 2024-03-09T09:19:22.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.298+0000 7fcde7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80c7420 con 0x7fcde806dee0 2024-03-09T09:19:22.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.298+0000 7fcde5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcde80c3bb0 0x7fcde80c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:22.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.298+0000 7fcde5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcde80c3bb0 0x7fcde80c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34650/0 (socket says 172.21.15.67:34650) 2024-03-09T09:19:22.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.298+0000 7fcde5e00640 1 -- 172.21.15.67:0/3456496456 learned_addr learned my addr 172.21.15.67:0/3456496456 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:22.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.298+0000 7fcde6800640 1 --2- 172.21.15.67:0/3456496456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcde806dee0 0x7fcde8159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:22.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.298+0000 7fcde5e00640 1 -- 172.21.15.67:0/3456496456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcde806dee0 msgr2=0x7fcde8159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:22.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.298+0000 7fcde5e00640 1 --2- 172.21.15.67:0/3456496456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcde806dee0 0x7fcde8159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.299+0000 7fcde5e00640 1 -- 172.21.15.67:0/3456496456 >> v1:172.21.15.134:6789/0 conn(0x7fcde806e820 legacy=0x7fcde8159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:22.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.299+0000 7fcde5e00640 1 -- 172.21.15.67:0/3456496456 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80c5950 con 0x7fcde80c3bb0 2024-03-09T09:19:22.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.300+0000 7fcde5e00640 1 --2- 172.21.15.67:0/3456496456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcde80c3bb0 0x7fcde80c7940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009870 tx=0x7fcdd802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e094b28102fa2c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:22.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.301+0000 7fcde4a00640 1 -- 172.21.15.67:0/3456496456 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80317c0 con 0x7fcde80c3bb0 2024-03-09T09:19:22.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.301+0000 7fcde4a00640 1 -- 172.21.15.67:0/3456496456 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd8037070 con 0x7fcde80c3bb0 2024-03-09T09:19:22.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.301+0000 7fcde4a00640 1 -- 172.21.15.67:0/3456496456 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8031b80 con 0x7fcde80c3bb0 2024-03-09T09:19:22.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.302+0000 7fcde7200640 1 -- 172.21.15.67:0/3456496456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcde80c3bb0 msgr2=0x7fcde80c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:22.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.302+0000 7fcde7200640 1 --2- 172.21.15.67:0/3456496456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcde80c3bb0 0x7fcde80c7940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009870 tx=0x7fcdd802b860 comp rx=0 tx=0).stop 2024-03-09T09:19:22.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.302+0000 7fcde7200640 1 -- 172.21.15.67:0/3456496456 shutdown_connections 2024-03-09T09:19:22.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.302+0000 7fcde7200640 1 --2- 172.21.15.67:0/3456496456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcde806dee0 0x7fcde8159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.302+0000 7fcde7200640 1 --2- 172.21.15.67:0/3456496456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcde80c3bb0 0x7fcde80c7940 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.302+0000 7fcde7200640 1 -- 172.21.15.67:0/3456496456 >> 172.21.15.67:0/3456496456 conn(0x7fcde806b040 msgr2=0x7fcde806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:22.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.303+0000 7fcde7200640 1 -- 172.21.15.67:0/3456496456 shutdown_connections 2024-03-09T09:19:22.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.303+0000 7fcde7200640 1 -- 172.21.15.67:0/3456496456 wait complete. 2024-03-09T09:19:22.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.305+0000 7fcde7200640 1 Processor -- start 2024-03-09T09:19:22.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.306+0000 7fcde7200640 1 -- start start 2024-03-09T09:19:22.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.307+0000 7fcde7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcde806dee0 0x7fcde80bd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:22.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.307+0000 7fcde7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcde806e820 0x7fcde80ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:22.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.307+0000 7fcde7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcde80c3bb0 0x7fcde80ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:22.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.307+0000 7fcde7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde806ec40 con 0x7fcde806dee0 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.307+0000 7fcde7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde81566c0 con 0x7fcde80c3bb0 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.307+0000 7fcde7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde806edc0 con 0x7fcde806e820 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.307+0000 7fcde5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcde806dee0 0x7fcde80bd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.307+0000 7fcde5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcde806dee0 0x7fcde80bd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34660/0 (socket says 172.21.15.67:34660) 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.307+0000 7fcde5e00640 1 -- 172.21.15.67:0/4225798274 learned_addr learned my addr 172.21.15.67:0/4225798274 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.308+0000 7fcde5400640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcde806e820 0x7fcde80ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.308+0000 7fcde6800640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcde80c3bb0 0x7fcde80ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.308+0000 7fcde5e00640 1 -- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcde806e820 msgr2=0x7fcde80ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.308+0000 7fcde5e00640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcde806e820 0x7fcde80ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.308+0000 7fcde5e00640 1 -- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcde80c3bb0 msgr2=0x7fcde80ac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.308+0000 7fcde5e00640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcde80c3bb0 0x7fcde80ac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.308+0000 7fcde5e00640 1 -- 172.21.15.67:0/4225798274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde81f0410 con 0x7fcde806dee0 2024-03-09T09:19:22.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.308+0000 7fcde5400640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcde806e820 0x7fcde80ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:19:22.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.309+0000 7fcde5e00640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcde806dee0 0x7fcde80bd150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000c00 tx=0x7fcdd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:22.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.309+0000 7fcdde000640 1 -- 172.21.15.67:0/4225798274 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8032650 con 0x7fcde806dee0 2024-03-09T09:19:22.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.309+0000 7fcde7200640 1 -- 172.21.15.67:0/4225798274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde81f06b0 con 0x7fcde806dee0 2024-03-09T09:19:22.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.309+0000 7fcdde000640 1 -- 172.21.15.67:0/4225798274 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd80327f0 con 0x7fcde806dee0 2024-03-09T09:19:22.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.309+0000 7fcdde000640 1 -- 172.21.15.67:0/4225798274 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8032be0 con 0x7fcde806dee0 2024-03-09T09:19:22.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.309+0000 7fcde7200640 1 -- 172.21.15.67:0/4225798274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde8072300 con 0x7fcde806dee0 2024-03-09T09:19:22.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.311+0000 7fcde7200640 1 -- 172.21.15.67:0/4225798274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde80c8660 con 0x7fcde806dee0 2024-03-09T09:19:22.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.312+0000 7fcdde000640 1 -- 172.21.15.67:0/4225798274 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcdd804c060 con 0x7fcde806dee0 2024-03-09T09:19:22.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.313+0000 7fcdde000640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcdc806b4d0 0x7fcdc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:22.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.313+0000 7fcdde000640 1 -- 172.21.15.67:0/4225798274 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fcdd80ac170 con 0x7fcde806dee0 2024-03-09T09:19:22.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.314+0000 7fcde5400640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcdc806b4d0 0x7fcdc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:22.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.315+0000 7fcde5400640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcdc806b4d0 0x7fcdc806d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcde8155ff0 tx=0x7fcdd4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:22.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.318+0000 7fcdde000640 1 -- 172.21.15.67:0/4225798274 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcde80c8660 con 0x7fcde806dee0 2024-03-09T09:19:22.649 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:22.649 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:22.649 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 37 -eq 48 ']' 2024-03-09T09:19:22.649 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.37 2024-03-09T09:19:22.650 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.37 2024-03-09T09:19:22.650 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.37 2024-03-09T09:19:22.650 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.37 2024-03-09T09:19:22.653 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:22.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.666+0000 7fcde7200640 1 -- 172.21.15.67:0/4225798274 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fcde804f3c0 con 0x7fcde806dee0 2024-03-09T09:19:22.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.667+0000 7fcdde000640 1 -- 172.21.15.67:0/4225798274 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fcdd8040070 con 0x7fcde806dee0 2024-03-09T09:19:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.670+0000 7fcde7200640 1 -- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcdc806b4d0 msgr2=0x7fcdc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.670+0000 7fcde7200640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcdc806b4d0 0x7fcdc806d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcde8155ff0 tx=0x7fcdd4009040 comp rx=0 tx=0).stop 2024-03-09T09:19:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.670+0000 7fcde7200640 1 -- 172.21.15.67:0/4225798274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcde806dee0 msgr2=0x7fcde80bd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.670+0000 7fcde7200640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcde806dee0 0x7fcde80bd150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000c00 tx=0x7fcdd80046c0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.671+0000 7fcde7200640 1 -- 172.21.15.67:0/4225798274 shutdown_connections 2024-03-09T09:19:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.671+0000 7fcde7200640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcdc806b4d0 0x7fcdc806d990 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.671+0000 7fcde7200640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcde80c3bb0 0x7fcde80ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.671+0000 7fcde7200640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcde806e820 0x7fcde80ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.671+0000 7fcde7200640 1 --2- 172.21.15.67:0/4225798274 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcde806dee0 0x7fcde80bd150 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.671+0000 7fcde7200640 1 -- 172.21.15.67:0/4225798274 >> 172.21.15.67:0/4225798274 conn(0x7fcde806b040 msgr2=0x7fcde80c2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.671+0000 7fcde7200640 1 -- 172.21.15.67:0/4225798274 shutdown_connections 2024-03-09T09:19:22.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:22.671+0000 7fcde7200640 1 -- 172.21.15.67:0/4225798274 wait complete. 2024-03-09T09:19:22.698 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:19:24.051 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:24.051 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:24.051 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 32 -eq 48 ']' 2024-03-09T09:19:24.052 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.32 2024-03-09T09:19:24.052 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.32 2024-03-09T09:19:24.052 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.32 2024-03-09T09:19:24.052 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.32 2024-03-09T09:19:24.055 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:27.675 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:27.676 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:27.676 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 33 -eq 48 ']' 2024-03-09T09:19:27.676 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.33 2024-03-09T09:19:27.676 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.33 2024-03-09T09:19:27.676 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.33 2024-03-09T09:19:27.676 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.33 2024-03-09T09:19:27.680 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:28.563 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:28.564 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:28.564 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 38 -eq 48 ']' 2024-03-09T09:19:28.564 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.38 2024-03-09T09:19:28.564 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.38 2024-03-09T09:19:28.564 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.38 2024-03-09T09:19:28.565 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.38 2024-03-09T09:19:28.567 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:29.012 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:19:29.124 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:19:32.604 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:32.604 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:32.604 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 34 -eq 48 ']' 2024-03-09T09:19:32.604 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.34 2024-03-09T09:19:32.604 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.34 2024-03-09T09:19:32.604 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.34 2024-03-09T09:19:32.604 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.34 2024-03-09T09:19:32.608 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:32.699 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:19:32.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:19:32.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.904+0000 7f77ca000640 1 Processor -- start 2024-03-09T09:19:32.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.904+0000 7f77ca000640 1 -- start start 2024-03-09T09:19:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.905+0000 7f77ca000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f77c41531c0 0x7f77c41570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.905+0000 7f77ca000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77c40cae50 0x7f77c40ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.905+0000 7f77ca000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f77c40c9440 0x7f77c40c9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77ca000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c40afba0 con 0x7f77c40c9440 2024-03-09T09:19:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77ca000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c40b01c0 con 0x7f77c41531c0 2024-03-09T09:19:32.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77ca000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c4156700 con 0x7f77c40cae50 2024-03-09T09:19:32.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77c9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f77c40c9440 0x7f77c40c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:32.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77c9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f77c40c9440 0x7f77c40c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48442/0 (socket says 172.21.15.67:48442) 2024-03-09T09:19:32.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77c9600640 1 -- 172.21.15.67:0/4169628251 learned_addr learned my addr 172.21.15.67:0/4169628251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:32.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77c8c00640 1 --2- 172.21.15.67:0/4169628251 >> v2:172.21.15.134:3300/0 conn(0x7f77c41531c0 0x7f77c41570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:32.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77c3e00640 1 --2- 172.21.15.67:0/4169628251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77c40cae50 0x7f77c40ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77c9600640 1 -- 172.21.15.67:0/4169628251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77c40cae50 msgr2=0x7f77c40ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77c9600640 1 --2- 172.21.15.67:0/4169628251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77c40cae50 0x7f77c40ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77c9600640 1 -- 172.21.15.67:0/4169628251 >> v2:172.21.15.134:3300/0 conn(0x7f77c41531c0 msgr2=0x7f77c41570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.906+0000 7f77c9600640 1 --2- 172.21.15.67:0/4169628251 >> v2:172.21.15.134:3300/0 conn(0x7f77c41531c0 0x7f77c41570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.907+0000 7f77c9600640 1 -- 172.21.15.67:0/4169628251 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c41569a0 con 0x7f77c40c9440 2024-03-09T09:19:32.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.908+0000 7f77c9600640 1 --2- 172.21.15.67:0/4169628251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c40c9440 0x7f77c40c9890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f77c40ca960 tx=0x7f77bc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229f03bde0e1b416 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:32.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.908+0000 7f77c3400640 1 -- 172.21.15.67:0/4169628251 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc02ca70 con 0x7f77c40c9440 2024-03-09T09:19:32.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.908+0000 7f77c3400640 1 -- 172.21.15.67:0/4169628251 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc02cc10 con 0x7f77c40c9440 2024-03-09T09:19:32.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.908+0000 7f77c3400640 1 -- 172.21.15.67:0/4169628251 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0343f0 con 0x7f77c40c9440 2024-03-09T09:19:32.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.909+0000 7f77ca000640 1 -- 172.21.15.67:0/4169628251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c40c9440 msgr2=0x7f77c40c9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:32.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.909+0000 7f77ca000640 1 --2- 172.21.15.67:0/4169628251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c40c9440 0x7f77c40c9890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f77c40ca960 tx=0x7f77bc02bed0 comp rx=0 tx=0).stop 2024-03-09T09:19:32.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.909+0000 7f77ca000640 1 -- 172.21.15.67:0/4169628251 shutdown_connections 2024-03-09T09:19:32.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.909+0000 7f77ca000640 1 --2- 172.21.15.67:0/4169628251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c40c9440 0x7f77c40c9890 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:32.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.909+0000 7f77ca000640 1 --2- 172.21.15.67:0/4169628251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77c40cae50 0x7f77c40ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:32.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.909+0000 7f77ca000640 1 --2- 172.21.15.67:0/4169628251 >> v2:172.21.15.134:3300/0 conn(0x7f77c41531c0 0x7f77c41570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:32.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.909+0000 7f77ca000640 1 -- 172.21.15.67:0/4169628251 >> 172.21.15.67:0/4169628251 conn(0x7f77c40ad190 msgr2=0x7f77c40ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:32.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.909+0000 7f77ca000640 1 -- 172.21.15.67:0/4169628251 shutdown_connections 2024-03-09T09:19:32.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.909+0000 7f77ca000640 1 -- 172.21.15.67:0/4169628251 wait complete. 2024-03-09T09:19:32.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.911+0000 7f77ca000640 1 Processor -- start 2024-03-09T09:19:32.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.911+0000 7f77ca000640 1 -- start start 2024-03-09T09:19:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.912+0000 7f77ca000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77c40c9440 0x7f77c41ec740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.912+0000 7f77ca000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77c40cae50 0x7f77c4146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.912+0000 7f77ca000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c41531c0 0x7f77c40ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.912+0000 7f77ca000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c41535a0 con 0x7f77c41531c0 2024-03-09T09:19:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.912+0000 7f77ca000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40b06f0 con 0x7f77c40c9440 2024-03-09T09:19:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.912+0000 7f77ca000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40cb2b0 con 0x7f77c40cae50 2024-03-09T09:19:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.912+0000 7f77c9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c41531c0 0x7f77c40ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.912+0000 7f77c9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c41531c0 0x7f77c40ac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48452/0 (socket says 172.21.15.67:48452) 2024-03-09T09:19:32.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.912+0000 7f77c9600640 1 -- 172.21.15.67:0/126415960 learned_addr learned my addr 172.21.15.67:0/126415960 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.913+0000 7f77c8c00640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77c40c9440 0x7f77c41ec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.913+0000 7f77c3e00640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77c40cae50 0x7f77c4146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.913+0000 7f77c9600640 1 -- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77c40cae50 msgr2=0x7f77c4146940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.913+0000 7f77c9600640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77c40cae50 0x7f77c4146940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.913+0000 7f77c9600640 1 -- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77c40c9440 msgr2=0x7f77c41ec740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.913+0000 7f77c9600640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77c40c9440 0x7f77c41ec740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:32.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.913+0000 7f77c9600640 1 -- 172.21.15.67:0/126415960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c41ecc80 con 0x7f77c41531c0 2024-03-09T09:19:32.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.913+0000 7f77c9600640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c41531c0 0x7f77c40ac180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009f40 tx=0x7f77bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:32.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.914+0000 7f77c0c00640 1 -- 172.21.15.67:0/126415960 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc043020 con 0x7f77c41531c0 2024-03-09T09:19:32.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.914+0000 7f77c0c00640 1 -- 172.21.15.67:0/126415960 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77bc03e450 con 0x7f77c41531c0 2024-03-09T09:19:32.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.914+0000 7f77c0c00640 1 -- 172.21.15.67:0/126415960 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77bc009510 con 0x7f77c41531c0 2024-03-09T09:19:32.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.914+0000 7f77ca000640 1 -- 172.21.15.67:0/126415960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c40c0fd0 con 0x7f77c41531c0 2024-03-09T09:19:32.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.914+0000 7f77ca000640 1 -- 172.21.15.67:0/126415960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c40c5010 con 0x7f77c41531c0 2024-03-09T09:19:32.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.915+0000 7f77ca000640 1 -- 172.21.15.67:0/126415960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c40c4e00 con 0x7f77c41531c0 2024-03-09T09:19:32.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.916+0000 7f77c0c00640 1 -- 172.21.15.67:0/126415960 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f77bc046020 con 0x7f77c41531c0 2024-03-09T09:19:32.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.918+0000 7f77c0c00640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f77a406b4d0 0x7f77a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:32.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.918+0000 7f77c0c00640 1 -- 172.21.15.67:0/126415960 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f77bc0ace20 con 0x7f77c41531c0 2024-03-09T09:19:32.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.918+0000 7f77c8c00640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f77a406b4d0 0x7f77a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:32.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.920+0000 7f77c8c00640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f77a406b4d0 0x7f77a406d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f77b8006fd0 tx=0x7f77b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:32.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:32.923+0000 7f77c0c00640 1 -- 172.21.15.67:0/126415960 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f77bc04d030 con 0x7f77c41531c0 2024-03-09T09:19:33.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.268+0000 7f77ca000640 1 -- 172.21.15.67:0/126415960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f77c404f3c0 con 0x7f77c41531c0 2024-03-09T09:19:33.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.269+0000 7f77c0c00640 1 -- 172.21.15.67:0/126415960 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f77bc07b8b0 con 0x7f77c41531c0 2024-03-09T09:19:33.275 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":200,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:19:33.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 -- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f77a406b4d0 msgr2=0x7f77a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f77a406b4d0 0x7f77a406d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f77b8006fd0 tx=0x7f77b8008040 comp rx=0 tx=0).stop 2024-03-09T09:19:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 -- 172.21.15.67:0/126415960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c41531c0 msgr2=0x7f77c40ac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c41531c0 0x7f77c40ac180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009f40 tx=0x7f77bc004a40 comp rx=0 tx=0).stop 2024-03-09T09:19:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 -- 172.21.15.67:0/126415960 shutdown_connections 2024-03-09T09:19:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f77a406b4d0 0x7f77a406d990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77c41531c0 0x7f77c40ac180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77c40cae50 0x7f77c4146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 --2- 172.21.15.67:0/126415960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77c40c9440 0x7f77c41ec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 -- 172.21.15.67:0/126415960 >> 172.21.15.67:0/126415960 conn(0x7f77c40ad190 msgr2=0x7f77c4158f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 -- 172.21.15.67:0/126415960 shutdown_connections 2024-03-09T09:19:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.271+0000 7f77ca000640 1 -- 172.21.15.67:0/126415960 wait complete. 2024-03-09T09:19:33.297 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:19:33.297 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:19:33.297 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:19:33.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:19:33.440 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:33.440 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:33.440 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 39 -eq 48 ']' 2024-03-09T09:19:33.440 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.39 2024-03-09T09:19:33.441 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.39 2024-03-09T09:19:33.441 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.39 2024-03-09T09:19:33.441 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.39 2024-03-09T09:19:33.445 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:33.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.515+0000 7f3061200640 1 Processor -- start 2024-03-09T09:19:33.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.515+0000 7f3061200640 1 -- start start 2024-03-09T09:19:33.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.516+0000 7f3061200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f305c153610 0x7f305c159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:33.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.517+0000 7f3061200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c06f9e0 0x7f305c072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:33.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.517+0000 7f3061200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c155d20 con 0x7f305c0c2b20 2024-03-09T09:19:33.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.517+0000 7f3061200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305c1539f0 con 0x7f305c153610 2024-03-09T09:19:33.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.517+0000 7f3061200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c156070 con 0x7f305c06f9e0 2024-03-09T09:19:33.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.517+0000 7f3059400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f305c0c2b20 0x7f305c156500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51582/0 (socket says 172.21.15.67:51582) 2024-03-09T09:19:33.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.517+0000 7f3059400640 1 -- 172.21.15.67:0/2261403042 learned_addr learned my addr 172.21.15.67:0/2261403042 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:19:33.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.517+0000 7f3059e00640 1 --2- 172.21.15.67:0/2261403042 >> v2:172.21.15.134:3300/0 conn(0x7f305c153610 0x7f305c159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:33.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.517+0000 7f305a800640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c06f9e0 0x7f305c072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:33.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.517+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090329583 0 0) 0x7f305c155d20 con 0x7f305c0c2b20 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3040003520 con 0x7f305c0c2b20 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2911112245 0 0) 0x7f3040003520 con 0x7f305c0c2b20 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c06f9e0 msgr2=0x7f305c072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c06f9e0 0x7f305c072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 >> v2:172.21.15.134:3300/0 conn(0x7f305c153610 msgr2=0x7f305c159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 --2- 172.21.15.67:0/2261403042 >> v2:172.21.15.134:3300/0 conn(0x7f305c153610 0x7f305c159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c156c10 con 0x7f305c0c2b20 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f3048001670 con 0x7f305c0c2b20 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 >> v1:172.21.15.67:6789/0 conn(0x7f305c0c2b20 legacy=0x7f305c156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30400039b0 0x7f3040003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 0x7f3040004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c06f9e0 0x7f30400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:33.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c1539f0 con 0x7f3040004800 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c156070 con 0x7f30400039b0 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304000baf0 con 0x7f305c06f9e0 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.518+0000 7f3059400640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 0x7f3040004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.519+0000 7f305a800640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c06f9e0 0x7f30400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.519+0000 7f3059e00640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30400039b0 0x7f3040003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.519+0000 7f3059400640 1 -- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c06f9e0 msgr2=0x7f30400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.519+0000 7f3059400640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c06f9e0 0x7f30400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.519+0000 7f3059400640 1 -- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30400039b0 msgr2=0x7f3040003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.519+0000 7f3059400640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30400039b0 0x7f3040003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.519+0000 7f3059400640 1 -- 172.21.15.67:0/2261403042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3040003520 con 0x7f3040004800 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.519+0000 7f305a800640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c06f9e0 0x7f30400051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:19:33.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.519+0000 7f3059e00640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30400039b0 0x7f3040003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:19:33.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.520+0000 7f3059400640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 0x7f3040004c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3048001eb0 tx=0x7f3040005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440f82151c0da57b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:33.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.520+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048007ca0 con 0x7f3040004800 2024-03-09T09:19:33.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.520+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3048030070 con 0x7f3040004800 2024-03-09T09:19:33.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.520+0000 7f3058a00640 1 -- 172.21.15.67:0/2261403042 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048005d10 con 0x7f3040004800 2024-03-09T09:19:33.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.521+0000 7f3061200640 1 -- 172.21.15.67:0/2261403042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 msgr2=0x7f3040004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:33.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.521+0000 7f3061200640 1 --2- 172.21.15.67:0/2261403042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 0x7f3040004c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3048001eb0 tx=0x7f3040005830 comp rx=0 tx=0).stop 2024-03-09T09:19:33.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.521+0000 7f305a800640 1 -- 172.21.15.67:0/2261403042 reap_dead start 2024-03-09T09:19:33.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.521+0000 7f3061200640 1 -- 172.21.15.67:0/2261403042 shutdown_connections 2024-03-09T09:19:33.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.521+0000 7f3061200640 1 -- 172.21.15.67:0/2261403042 >> 172.21.15.67:0/2261403042 conn(0x7f305c0aa680 msgr2=0x7f305c0aaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:33.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.521+0000 7f3061200640 1 -- 172.21.15.67:0/2261403042 shutdown_connections 2024-03-09T09:19:33.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.521+0000 7f3061200640 1 -- 172.21.15.67:0/2261403042 wait complete. 2024-03-09T09:19:33.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.523+0000 7f3061200640 1 Processor -- start 2024-03-09T09:19:33.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.523+0000 7f3061200640 1 -- start start 2024-03-09T09:19:33.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.525+0000 7f3061200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 0x7f305c0c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.525+0000 7f3061200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c145ea0 0x7f305c1eee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.525+0000 7f3061200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c1463f0 0x7f305c1ef390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.525+0000 7f3061200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c15a100 con 0x7f3040004800 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.525+0000 7f3061200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c159e20 con 0x7f305c145ea0 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.525+0000 7f3061200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c159850 con 0x7f305c1463f0 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.525+0000 7f3059e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 0x7f305c0c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.525+0000 7f3059e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 0x7f305c0c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48474/0 (socket says 172.21.15.67:48474) 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.525+0000 7f3059e00640 1 -- 172.21.15.67:0/3547702298 learned_addr learned my addr 172.21.15.67:0/3547702298 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.525+0000 7f3059400640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c145ea0 0x7f305c1eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.525+0000 7f305a800640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c1463f0 0x7f305c1ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.526+0000 7f3059e00640 1 -- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c1463f0 msgr2=0x7f305c1ef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:33.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.526+0000 7f3059e00640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f305c1463f0 0x7f305c1ef390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.526+0000 7f3059e00640 1 -- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c145ea0 msgr2=0x7f305c1eee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:33.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.526+0000 7f3059e00640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c145ea0 0x7f305c1eee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.526+0000 7f3059e00640 1 -- 172.21.15.67:0/3547702298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c06c480 con 0x7f3040004800 2024-03-09T09:19:33.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.526+0000 7f3059400640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c145ea0 0x7f305c1eee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:19:33.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.526+0000 7f3059e00640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 0x7f305c0c89a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f304c010a00 tx=0x7f304c010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:33.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.526+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c00f040 con 0x7f3040004800 2024-03-09T09:19:33.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.526+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304c00ec20 con 0x7f3040004800 2024-03-09T09:19:33.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.526+0000 7f3061200640 1 -- 172.21.15.67:0/3547702298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305c158970 con 0x7f3040004800 2024-03-09T09:19:33.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.526+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c01c400 con 0x7f3040004800 2024-03-09T09:19:33.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.527+0000 7f3061200640 1 -- 172.21.15.67:0/3547702298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c158f30 con 0x7f3040004800 2024-03-09T09:19:33.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.527+0000 7f3061200640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7480 0x7f305c0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:33.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.527+0000 7f3061200640 1 -- 172.21.15.67:0/3547702298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c157fe0 con 0x7f305c0c7480 2024-03-09T09:19:33.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.528+0000 7f3059400640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7480 0x7f305c0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:33.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.528+0000 7f3059400640 1 -- 172.21.15.67:0/3547702298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f305c075a00 con 0x7f305c0c7480 2024-03-09T09:19:33.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.529+0000 7f3059400640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7480 0x7f305c0c7880 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30400057e0 tx=0x7f3048000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:33.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.529+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048046070 con 0x7f305c0c7480 2024-03-09T09:19:33.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.529+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f3048030040 con 0x7f305c0c7480 2024-03-09T09:19:33.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.529+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7480 msgr2=0x7f305c0c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:33.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.529+0000 7f3052000640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7480 0x7f305c0c7880 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30400057e0 tx=0x7f3048000fe0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.529+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7480 msgr2=0x7f305c0c7880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:33.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.529+0000 7f3052000640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7480 0x7f305c0c7880 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30400057e0 tx=0x7f3048000fe0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.530+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f304c01c5a0 con 0x7f3040004800 2024-03-09T09:19:33.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.531+0000 7f3052000640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f303c06afb0 0x7f303c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:33.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.531+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f304c08ad50 con 0x7f3040004800 2024-03-09T09:19:33.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.532+0000 7f3059400640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f303c06afb0 0x7f303c06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:33.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.533+0000 7f3059400640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f303c06afb0 0x7f303c06d470 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f305c068bb0 tx=0x7f3048039000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:33.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.537+0000 7f3061200640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7dc0 0x7f305c144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:33.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.537+0000 7f3061200640 1 -- 172.21.15.67:0/3547702298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c1ef8d0 con 0x7f305c0c7dc0 2024-03-09T09:19:33.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.537+0000 7f305a800640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7dc0 0x7f305c144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:33.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.538+0000 7f305a800640 1 -- 172.21.15.67:0/3547702298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f305c0c5ef0 con 0x7f305c0c7dc0 2024-03-09T09:19:33.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.538+0000 7f305a800640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7dc0 0x7f305c144e50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f305c0ca480 tx=0x7f3054010510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:33.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.538+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305401b020 con 0x7f305c0c7dc0 2024-03-09T09:19:33.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7f3054010e80 con 0x7f305c0c7dc0 2024-03-09T09:19:33.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7dc0 msgr2=0x7f305c144e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:33.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3052000640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7dc0 0x7f305c144e50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f305c0ca480 tx=0x7f3054010510 comp rx=0 tx=0).stop 2024-03-09T09:19:33.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3052000640 1 -- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7dc0 msgr2=0x7f305c144e50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:33.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3052000640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f305c0c7dc0 0x7f305c144e50 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f305c0ca480 tx=0x7f3054010510 comp rx=0 tx=0).stop 2024-03-09T09:19:33.547 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:19:33.547 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:19:33.547 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:19:33.547 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:19:33.547 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 200, 2024-03-09T09:19:33.547 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:19:33.547 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:19:33.548 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:19:33.548 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:19:33.548 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:33.548 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T09:19:33.548 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:33.548 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:19:33.548 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:19:33.548 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:33.548 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:33.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:33.548 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:33.549 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:33.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:33.550 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:33.550 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:33.550 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:33.550 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:33.550 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:33.550 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:33.550 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:33.550 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:33.551 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:33.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.551 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:19:33.551 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:19:33.551 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:19:33.551 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:19:33.551 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:19:33.551 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:19:33.551 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:33.552 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:33.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:19:33.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:19:33.552 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:19:33.552 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:19:33.552 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:19:33.552 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:19:33.552 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:19:33.552 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:33.553 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:19:33.553 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:33.553 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:33.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:33.553 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:33.553 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:33.553 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:33.553 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:33.553 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:33.554 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:33.554 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:33.554 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:33.554 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:33.554 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:19:33.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:19:33.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.555 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:19:33.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:19:33.555 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:33.555 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:33.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.555 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:33.555 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:19:33.555 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:33.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.556 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:33.556 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:19:33.556 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:33.556 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:33.556 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:33.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.557 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:19:33.557 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:19:33.557 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:33.557 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:33.558 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:33.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.559 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:19:33.559 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:19:33.559 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:33.559 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:33.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:33.559 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:19:33.559 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:33.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.560 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:33.947 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:19:33.947 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:33.947 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:33.948 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:33.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.948 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:19:33.948 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:19:33.948 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:33.948 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:33.948 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:33.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.948 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.948 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:19:33.948 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:33.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.950 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.951 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:33.952 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:19:33.953 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:19:33.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3061200640 1 -- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f303c06afb0 msgr2=0x7f303c06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:33.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3061200640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f303c06afb0 0x7f303c06d470 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f305c068bb0 tx=0x7f3048039000 comp rx=0 tx=0).stop 2024-03-09T09:19:33.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3061200640 1 -- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 msgr2=0x7f305c0c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:33.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3061200640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 0x7f305c0c89a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f304c010a00 tx=0x7f304c010ec0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f305a800640 1 -- 172.21.15.67:0/3547702298 reap_dead start 2024-03-09T09:19:33.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3061200640 1 -- 172.21.15.67:0/3547702298 shutdown_connections 2024-03-09T09:19:33.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3061200640 1 --2- 172.21.15.67:0/3547702298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3040004800 0x7f305c0c89a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:33.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3061200640 1 -- 172.21.15.67:0/3547702298 >> 172.21.15.67:0/3547702298 conn(0x7f305c0aa680 msgr2=0x7f305c06fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:33.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.539+0000 7f3061200640 1 -- 172.21.15.67:0/3547702298 shutdown_connections 2024-03-09T09:19:33.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:33.540+0000 7f3061200640 1 -- 172.21.15.67:0/3547702298 wait complete. 2024-03-09T09:19:33.956 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:19:33.956 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:19:33.956 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:19:34.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.192+0000 7f5079a00640 1 Processor -- start 2024-03-09T09:19:34.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.192+0000 7f5079a00640 1 -- start start 2024-03-09T09:19:34.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.193+0000 7f5079a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f507406c1d0 0x7f507406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.193+0000 7f5079a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50740c9890 con 0x7f507414fd90 2024-03-09T09:19:34.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.193+0000 7f5079a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50740c5f80 con 0x7f50740c9380 2024-03-09T09:19:34.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.193+0000 7f5079a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507406ba60 con 0x7f507406c1d0 2024-03-09T09:19:34.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.193+0000 7f5073e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f507414fd90 0x7f5074156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51596/0 (socket says 172.21.15.67:51596) 2024-03-09T09:19:34.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.194+0000 7f5073e00640 1 -- 172.21.15.67:0/3156739999 learned_addr learned my addr 172.21.15.67:0/3156739999 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:19:34.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.194+0000 7f5079000640 1 --2- 172.21.15.67:0/3156739999 >> v2:172.21.15.67:3300/0 conn(0x7f507406c1d0 0x7f507406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.194+0000 7f5079000640 1 -- 172.21.15.67:0/3156739999 >> v1:172.21.15.134:6789/0 conn(0x7f50740c9380 legacy=0x7f50740acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.194+0000 7f5073400640 1 -- 172.21.15.67:0/3156739999 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589064201 0 0) 0x7f50740c9890 con 0x7f507414fd90 2024-03-09T09:19:34.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.194+0000 7f5079000640 1 -- 172.21.15.67:0/3156739999 >> v1:172.21.15.67:6789/0 conn(0x7f507414fd90 legacy=0x7f5074156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.194+0000 7f5079000640 1 -- 172.21.15.67:0/3156739999 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50740c6240 con 0x7f507406c1d0 2024-03-09T09:19:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.196+0000 7f5079000640 1 --2- 172.21.15.67:0/3156739999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406c1d0 0x7f507406c5f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f507406bbe0 tx=0x7f506c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fa79b606afa8ce server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.196+0000 7f5073400640 1 -- 172.21.15.67:0/3156739999 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c039070 con 0x7f507406c1d0 2024-03-09T09:19:34.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.196+0000 7f5073400640 1 -- 172.21.15.67:0/3156739999 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c02cbc0 con 0x7f507406c1d0 2024-03-09T09:19:34.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.197+0000 7f5073400640 1 -- 172.21.15.67:0/3156739999 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c034440 con 0x7f507406c1d0 2024-03-09T09:19:34.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.197+0000 7f5079a00640 1 -- 172.21.15.67:0/3156739999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406c1d0 msgr2=0x7f507406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.197+0000 7f5079a00640 1 --2- 172.21.15.67:0/3156739999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406c1d0 0x7f507406c5f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f507406bbe0 tx=0x7f506c02be10 comp rx=0 tx=0).stop 2024-03-09T09:19:34.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.197+0000 7f5079a00640 1 -- 172.21.15.67:0/3156739999 shutdown_connections 2024-03-09T09:19:34.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.197+0000 7f5079a00640 1 --2- 172.21.15.67:0/3156739999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507406c1d0 0x7f507406c5f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.197+0000 7f5079a00640 1 -- 172.21.15.67:0/3156739999 >> 172.21.15.67:0/3156739999 conn(0x7f50740cb480 msgr2=0x7f50740cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:34.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.197+0000 7f5079a00640 1 -- 172.21.15.67:0/3156739999 shutdown_connections 2024-03-09T09:19:34.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.197+0000 7f5079a00640 1 -- 172.21.15.67:0/3156739999 wait complete. 2024-03-09T09:19:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.199+0000 7f5079a00640 1 Processor -- start 2024-03-09T09:19:34.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.200+0000 7f5079a00640 1 -- start start 2024-03-09T09:19:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.201+0000 7f5079a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f507406c1d0 0x7f50740ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.201+0000 7f5079a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50740c9380 0x7f50741ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.201+0000 7f5079a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507414fd90 0x7f50741ec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.201+0000 7f5079a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50740ad610 con 0x7f507414fd90 2024-03-09T09:19:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.201+0000 7f5079a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507406bcb0 con 0x7f50740c9380 2024-03-09T09:19:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.201+0000 7f5079a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50740ad790 con 0x7f507406c1d0 2024-03-09T09:19:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.201+0000 7f5079000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507414fd90 0x7f50741ec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.201+0000 7f5079000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507414fd90 0x7f50741ec0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48502/0 (socket says 172.21.15.67:48502) 2024-03-09T09:19:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.201+0000 7f5079000640 1 -- 172.21.15.67:0/2930574038 learned_addr learned my addr 172.21.15.67:0/2930574038 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.201+0000 7f506b400640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50740c9380 0x7f50741ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.201+0000 7f5073e00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f507406c1d0 0x7f50740ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.202+0000 7f5079000640 1 -- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f507406c1d0 msgr2=0x7f50740ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.202+0000 7f5079000640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f507406c1d0 0x7f50740ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.202+0000 7f5079000640 1 -- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50740c9380 msgr2=0x7f50741ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.202+0000 7f5079000640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50740c9380 0x7f50741ebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.202+0000 7f5079000640 1 -- 172.21.15.67:0/2930574038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50740c1e30 con 0x7f507414fd90 2024-03-09T09:19:34.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.202+0000 7f5079000640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507414fd90 0x7f50741ec0b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f506c037040 tx=0x7f506c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.202+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c039050 con 0x7f507414fd90 2024-03-09T09:19:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.203+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c045070 con 0x7f507414fd90 2024-03-09T09:19:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.203+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c03e810 con 0x7f507414fd90 2024-03-09T09:19:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.203+0000 7f5079a00640 1 -- 172.21.15.67:0/2930574038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50741ef330 con 0x7f507414fd90 2024-03-09T09:19:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.203+0000 7f5079a00640 1 -- 172.21.15.67:0/2930574038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50741ef8f0 con 0x7f507414fd90 2024-03-09T09:19:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.204+0000 7f5079a00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50740c1930 0x7f5074148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.204+0000 7f5079a00640 1 -- 172.21.15.67:0/2930574038 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50741efaf0 con 0x7f50740c1930 2024-03-09T09:19:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.205+0000 7f5073e00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50740c1930 0x7f5074148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.205+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f506c03e9b0 con 0x7f507414fd90 2024-03-09T09:19:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.205+0000 7f5073e00640 1 -- 172.21.15.67:0/2930574038 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f507404f3c0 con 0x7f50740c1930 2024-03-09T09:19:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.206+0000 7f5073e00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50740c1930 0x7f5074148210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f50741ecca0 tx=0x7f5064001970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.206+0000 7f5070c00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f505006b520 0x7f505006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.206+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f506c0ad430 con 0x7f507414fd90 2024-03-09T09:19:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.207+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064012590 con 0x7f50740c1930 2024-03-09T09:19:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.207+0000 7f506b400640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f505006b520 0x7f505006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.207+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f5064018070 con 0x7f50740c1930 2024-03-09T09:19:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.207+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50740c1930 msgr2=0x7f5074148210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.207+0000 7f5070c00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50740c1930 0x7f5074148210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f50741ecca0 tx=0x7f5064001970 comp rx=0 tx=0).stop 2024-03-09T09:19:34.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.207+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50740c1930 msgr2=0x7f5074148210 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.207+0000 7f5070c00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f50740c1930 0x7f5074148210 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.208+0000 7f506b400640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f505006b520 0x7f505006d9e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f50741e99b0 tx=0x7f505c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.218+0000 7f5079a00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5074158e10 0x7f5074148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.218+0000 7f5079a00640 1 -- 172.21.15.67:0/2930574038 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507414c460 con 0x7f5074158e10 2024-03-09T09:19:34.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.219+0000 7f5073e00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5074158e10 0x7f5074148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.220+0000 7f5073e00640 1 -- 172.21.15.67:0/2930574038 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5074155f10 con 0x7f5074158e10 2024-03-09T09:19:34.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.220+0000 7f5073e00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5074158e10 0x7f5074148750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5064017000 tx=0x7f5064000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.221+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064012d70 con 0x7f5074158e10 2024-03-09T09:19:34.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.221+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4875 (secure 0 0 0) 0x7f5064012d70 con 0x7f5074158e10 2024-03-09T09:19:34.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.221+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5074158e10 msgr2=0x7f5074148750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.221+0000 7f5070c00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5074158e10 0x7f5074148750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5064017000 tx=0x7f5064000ed0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.221+0000 7f5070c00640 1 -- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5074158e10 msgr2=0x7f5074148750 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:34.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.221+0000 7f5070c00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5074158e10 0x7f5074148750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.228 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 200, 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:19:34.229 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:19:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:34.230 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:34.231 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:34.232 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:34.233 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:19:34.234 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.235 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:34.236 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:19:34.237 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:19:34.238 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:34.238 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:34.238 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:34.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.238 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:19:34.238 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:19:34.238 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:34.238 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:34.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.239 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:34.239 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:19:34.239 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.239 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:34.239 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:19:34.239 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.239 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.241 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.241 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:19:34.241 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:19:34.241 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:34.241 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:34.241 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:34.241 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.241 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:34.242 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:34.243 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.244 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.244 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:19:34.244 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:19:34.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.222+0000 7f5079a00640 1 -- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f505006b520 msgr2=0x7f505006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.222+0000 7f5079a00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f505006b520 0x7f505006d9e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f50741e99b0 tx=0x7f505c009040 comp rx=0 tx=0).stop 2024-03-09T09:19:34.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.222+0000 7f5079a00640 1 -- 172.21.15.67:0/2930574038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507414fd90 msgr2=0x7f50741ec0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.222+0000 7f5079a00640 1 --2- 172.21.15.67:0/2930574038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f507414fd90 0x7f50741ec0b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f506c037040 tx=0x7f506c004a40 comp rx=0 tx=0).stop 2024-03-09T09:19:34.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.222+0000 7f5079000640 1 -- 172.21.15.67:0/2930574038 reap_dead start 2024-03-09T09:19:34.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.222+0000 7f5079000640 1 -- 172.21.15.67:0/2930574038 reap_dead start 2024-03-09T09:19:34.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.222+0000 7f5079a00640 1 -- 172.21.15.67:0/2930574038 shutdown_connections 2024-03-09T09:19:34.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.222+0000 7f5079a00640 1 -- 172.21.15.67:0/2930574038 >> 172.21.15.67:0/2930574038 conn(0x7f50740cb480 msgr2=0x7f507414f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:34.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.222+0000 7f5079a00640 1 -- 172.21.15.67:0/2930574038 shutdown_connections 2024-03-09T09:19:34.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.222+0000 7f5079a00640 1 -- 172.21.15.67:0/2930574038 wait complete. 2024-03-09T09:19:34.246 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:19:34.246 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:19:34.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:19:34.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.455+0000 7f5eab000640 1 Processor -- start 2024-03-09T09:19:34.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.455+0000 7f5eab000640 1 -- start start 2024-03-09T09:19:34.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.456+0000 7f5eab000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea406c1d0 0x7f5ea406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.456+0000 7f5eab000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea40c9890 con 0x7f5ea40c9380 2024-03-09T09:19:34.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.456+0000 7f5eab000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea40c5f80 con 0x7f5ea414fd90 2024-03-09T09:19:34.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.456+0000 7f5eab000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea406ba60 con 0x7f5ea406c1d0 2024-03-09T09:19:34.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.457+0000 7f5ea9200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5ea40c9380 0x7f5ea40acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51602/0 (socket says 172.21.15.67:51602) 2024-03-09T09:19:34.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.457+0000 7f5ea9200640 1 -- 172.21.15.67:0/2206408074 learned_addr learned my addr 172.21.15.67:0/2206408074 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:19:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.457+0000 7f5eaa600640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea406c1d0 0x7f5ea406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.457+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230590130 0 0) 0x7f5ea40c9890 con 0x7f5ea40c9380 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e8c003520 con 0x7f5ea40c9380 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4294381303 0 0) 0x7f5e8c003520 con 0x7f5ea40c9380 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea406c1d0 msgr2=0x7f5ea406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea406c1d0 0x7f5ea406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 >> v1:172.21.15.134:6789/0 conn(0x7f5ea414fd90 legacy=0x7f5ea4156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea40c6240 con 0x7f5ea40c9380 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f5e90001670 con 0x7f5ea40c9380 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 >> v1:172.21.15.67:6789/0 conn(0x7f5ea40c9380 legacy=0x7f5ea40acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e8c003a50 0x7f5e8c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 0x7f5e8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea406c1d0 0x7f5e8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea406ba60 con 0x7f5e8c004800 2024-03-09T09:19:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c005580 con 0x7f5e8c003a50 2024-03-09T09:19:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c00bf00 con 0x7f5ea406c1d0 2024-03-09T09:19:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.458+0000 7f5ea9200640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 0x7f5e8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.459+0000 7f5ea9c00640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e8c003a50 0x7f5e8c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.459+0000 7f5eaa600640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea406c1d0 0x7f5e8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.459+0000 7f5ea9200640 1 -- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea406c1d0 msgr2=0x7f5e8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.459+0000 7f5ea9200640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea406c1d0 0x7f5e8c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.459+0000 7f5ea9200640 1 -- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e8c003a50 msgr2=0x7f5e8c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.459+0000 7f5ea9200640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e8c003a50 0x7f5e8c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.459+0000 7f5ea9200640 1 -- 172.21.15.67:0/2206408074 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c003520 con 0x7f5e8c004800 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.459+0000 7f5eaa600640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea406c1d0 0x7f5e8c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.460+0000 7f5ea9200640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 0x7f5e8c004c80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000c190 tx=0x7f5e8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3d419e0e49804c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.460+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90007c40 con 0x7f5e8c004800 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.460+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e90030070 con 0x7f5e8c004800 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.460+0000 7f5ea3e00640 1 -- 172.21.15.67:0/2206408074 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90005cd0 con 0x7f5e8c004800 2024-03-09T09:19:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.461+0000 7f5eab000640 1 -- 172.21.15.67:0/2206408074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 msgr2=0x7f5e8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.461+0000 7f5eab000640 1 --2- 172.21.15.67:0/2206408074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 0x7f5e8c004c80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000c190 tx=0x7f5e8c005830 comp rx=0 tx=0).stop 2024-03-09T09:19:34.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.461+0000 7f5eaa600640 1 -- 172.21.15.67:0/2206408074 reap_dead start 2024-03-09T09:19:34.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.461+0000 7f5eab000640 1 -- 172.21.15.67:0/2206408074 shutdown_connections 2024-03-09T09:19:34.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.461+0000 7f5eab000640 1 -- 172.21.15.67:0/2206408074 >> 172.21.15.67:0/2206408074 conn(0x7f5ea40cb480 msgr2=0x7f5ea40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:34.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.461+0000 7f5eab000640 1 -- 172.21.15.67:0/2206408074 shutdown_connections 2024-03-09T09:19:34.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.461+0000 7f5eab000640 1 -- 172.21.15.67:0/2206408074 wait complete. 2024-03-09T09:19:34.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.463+0000 7f5eab000640 1 Processor -- start 2024-03-09T09:19:34.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.463+0000 7f5eab000640 1 -- start start 2024-03-09T09:19:34.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.464+0000 7f5eab000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 0x7f5ea41e1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.464+0000 7f5eab000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea414c250 0x7f5ea414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.464+0000 7f5eab000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40c2700 0x7f5ea40c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.464+0000 7f5eab000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea40ad610 con 0x7f5e8c004800 2024-03-09T09:19:34.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.464+0000 7f5eab000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea406bcb0 con 0x7f5ea414c250 2024-03-09T09:19:34.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.464+0000 7f5eab000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea40ad790 con 0x7f5ea40c2700 2024-03-09T09:19:34.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.464+0000 7f5ea9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 0x7f5ea41e1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.465+0000 7f5ea9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 0x7f5ea41e1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48518/0 (socket says 172.21.15.67:48518) 2024-03-09T09:19:34.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.465+0000 7f5ea9c00640 1 -- 172.21.15.67:0/2822680177 learned_addr learned my addr 172.21.15.67:0/2822680177 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:34.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.465+0000 7f5ea9c00640 1 -- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40c2700 msgr2=0x7f5ea40c2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.465+0000 7f5ea9c00640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40c2700 0x7f5ea40c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.465+0000 7f5ea9c00640 1 -- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea414c250 msgr2=0x7f5ea414c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.465+0000 7f5ea9c00640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ea414c250 0x7f5ea414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.465+0000 7f5ea9c00640 1 -- 172.21.15.67:0/2822680177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea41ef700 con 0x7f5e8c004800 2024-03-09T09:19:34.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.465+0000 7f5ea9c00640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 0x7f5ea41e1920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800e4d0 tx=0x7f5e9800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.466+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e980052e0 con 0x7f5e8c004800 2024-03-09T09:19:34.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.466+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e98018070 con 0x7f5e8c004800 2024-03-09T09:19:34.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.466+0000 7f5eab000640 1 -- 172.21.15.67:0/2822680177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea40afb90 con 0x7f5e8c004800 2024-03-09T09:19:34.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.466+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98003810 con 0x7f5e8c004800 2024-03-09T09:19:34.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.466+0000 7f5eab000640 1 -- 172.21.15.67:0/2822680177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea40c57a0 con 0x7f5e8c004800 2024-03-09T09:19:34.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.467+0000 7f5eab000640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40c30f0 0x7f5ea40adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.467+0000 7f5eab000640 1 -- 172.21.15.67:0/2822680177 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea41ef4b0 con 0x7f5ea40c30f0 2024-03-09T09:19:34.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.468+0000 7f5ea9200640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40c30f0 0x7f5ea40adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.468+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5e980039b0 con 0x7f5e8c004800 2024-03-09T09:19:34.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.469+0000 7f5ea9200640 1 -- 172.21.15.67:0/2822680177 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ea41ef9e0 con 0x7f5ea40c30f0 2024-03-09T09:19:34.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.469+0000 7f5ea9200640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40c30f0 0x7f5ea40adb80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000c160 tx=0x7f5ea4155c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.470+0000 7f5ea1600640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5e8406b500 0x7f5e8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.470+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f5e9808a220 con 0x7f5e8c004800 2024-03-09T09:19:34.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.470+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e900047e0 con 0x7f5ea40c30f0 2024-03-09T09:19:34.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.470+0000 7f5eaa600640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5e8406b500 0x7f5e8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.471+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f5e90030040 con 0x7f5ea40c30f0 2024-03-09T09:19:34.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.471+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40c30f0 msgr2=0x7f5ea40adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.471+0000 7f5ea1600640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40c30f0 0x7f5ea40adb80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000c160 tx=0x7f5ea4155c00 comp rx=0 tx=0).stop 2024-03-09T09:19:34.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.471+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40c30f0 msgr2=0x7f5ea40adb80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:34.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.471+0000 7f5ea1600640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40c30f0 0x7f5ea40adb80 secure :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000c160 tx=0x7f5ea4155c00 comp rx=0 tx=0).stop 2024-03-09T09:19:34.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.471+0000 7f5eaa600640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5e8406b500 0x7f5e8406d9c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40c9760 tx=0x7f5e9c009750 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.478+0000 7f5eab000640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40ae0c0 0x7f5ea40ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.478+0000 7f5eab000640 1 -- 172.21.15.67:0/2822680177 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea40aea10 con 0x7f5ea40ae0c0 2024-03-09T09:19:34.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.479+0000 7f5ea9200640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40ae0c0 0x7f5ea40ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.480+0000 7f5ea9200640 1 -- 172.21.15.67:0/2822680177 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5ea404f3c0 con 0x7f5ea40ae0c0 2024-03-09T09:19:34.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.481+0000 7f5ea9200640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40ae0c0 0x7f5ea40ae4d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5ea41ef630 tx=0x7f5e9002f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.481+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90007550 con 0x7f5ea40ae0c0 2024-03-09T09:19:34.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f5e90007550 con 0x7f5ea40ae0c0 2024-03-09T09:19:34.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40ae0c0 msgr2=0x7f5ea40ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5ea1600640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40ae0c0 0x7f5ea40ae4d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5ea41ef630 tx=0x7f5e9002f000 comp rx=0 tx=0).stop 2024-03-09T09:19:34.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5ea1600640 1 -- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40ae0c0 msgr2=0x7f5ea40ae4d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:34.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5ea1600640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ea40ae0c0 0x7f5ea40ae4d0 secure :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5ea41ef630 tx=0x7f5e9002f000 comp rx=0 tx=0).stop 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 200, 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 17, 2024-03-09T09:19:34.489 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:34.490 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:34.491 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:19:34.492 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:34.493 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:34.494 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:34.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:19:34.496 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:19:34.497 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:34.498 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:34.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.498 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:34.498 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:19:34.498 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.498 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:34.498 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:19:34.498 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:34.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:19:34.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:34.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:34.501 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:19:34.501 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:19:34.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5eab000640 1 -- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5e8406b500 msgr2=0x7f5e8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5eab000640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5e8406b500 0x7f5e8406d9c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40c9760 tx=0x7f5e9c009750 comp rx=0 tx=0).stop 2024-03-09T09:19:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5eab000640 1 -- 172.21.15.67:0/2822680177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 msgr2=0x7f5ea41e1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5eab000640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 0x7f5ea41e1920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800e4d0 tx=0x7f5e9800e990 comp rx=0 tx=0).stop 2024-03-09T09:19:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5eaa600640 1 -- 172.21.15.67:0/2822680177 reap_dead start 2024-03-09T09:19:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5eab000640 1 -- 172.21.15.67:0/2822680177 shutdown_connections 2024-03-09T09:19:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5eab000640 1 --2- 172.21.15.67:0/2822680177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e8c004800 0x7f5ea41e1920 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5eab000640 1 -- 172.21.15.67:0/2822680177 >> 172.21.15.67:0/2822680177 conn(0x7f5ea40cb480 msgr2=0x7f5ea40cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.482+0000 7f5eab000640 1 -- 172.21.15.67:0/2822680177 shutdown_connections 2024-03-09T09:19:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.483+0000 7f5eab000640 1 -- 172.21.15.67:0/2822680177 wait complete. 2024-03-09T09:19:34.505 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:19:34.505 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-03-09T09:19:34.506 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:19:34.506 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-03-09T09:19:34.506 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:19:34.506 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-09T09:19:34.506 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:19:34.536 INFO:tasks.ceph.mon.a:Stopped 2024-03-09T09:19:34.536 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:19:34.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:19:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.797+0000 7fb0ef200640 1 Processor -- start 2024-03-09T09:19:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.798+0000 7fb0ef200640 1 -- start start 2024-03-09T09:19:34.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.800+0000 7fb0ef200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0f00cae50 0x7fb0f00c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.800+0000 7fb0ef200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f0156700 con 0x7fb0f01570d0 2024-03-09T09:19:34.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.800+0000 7fb0ef200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f0157530 con 0x7fb0f01531c0 2024-03-09T09:19:34.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.800+0000 7fb0ef200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f00afba0 con 0x7fb0f00cae50 2024-03-09T09:19:34.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.800+0000 7fb0ed400640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7fb0f01570d0 legacy=0x7fb0f00c9440 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:19:34.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.801+0000 7fb0ee800640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fb0f00cae50 msgr2=0x7fb0f00c9b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:34.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.801+0000 7fb0ee800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb0f00cae50 0x7fb0f00c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:34.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.801+0000 7fb0ede00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fb0f01531c0 0x7fb0f00ae590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:48624/0 (socket says 172.21.15.67:48624) 2024-03-09T09:19:34.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.801+0000 7fb0ede00640 1 -- 172.21.15.67:0/3580232167 learned_addr learned my addr 172.21.15.67:0/3580232167 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:19:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.802+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266395514 0 0) 0x7fb0f0157530 con 0x7fb0f01531c0 2024-03-09T09:19:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.802+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0d4003520 con 0x7fb0f01531c0 2024-03-09T09:19:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4239575245 0 0) 0x7fb0d4003520 con 0x7fb0f01531c0 2024-03-09T09:19:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 >> v2:172.21.15.67:3300/0 conn(0x7fb0f00cae50 msgr2=0x7fb0f00c9b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 --2- 172.21.15.67:0/3580232167 >> v2:172.21.15.67:3300/0 conn(0x7fb0f00cae50 0x7fb0f00c9b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 >> v1:172.21.15.67:6789/0 conn(0x7fb0f01570d0 legacy=0x7fb0f00c9440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f01569a0 con 0x7fb0f01531c0 2024-03-09T09:19:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb0e0001690 con 0x7fb0f01531c0 2024-03-09T09:19:34.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 >> v1:172.21.15.134:6789/0 conn(0x7fb0f01531c0 legacy=0x7fb0f00ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 --2- 172.21.15.67:0/3580232167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0d4003a20 0x7fb0d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 --2- 172.21.15.67:0/3580232167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 0x7fb0d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 --2- 172.21.15.67:0/3580232167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d40053c0 0x7fb0d400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d400c2b0 con 0x7fb0d40053c0 2024-03-09T09:19:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d400c460 con 0x7fb0d4004800 2024-03-09T09:19:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d400c660 con 0x7fb0d4003a20 2024-03-09T09:19:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0ee800640 1 -- 172.21.15.67:0/3580232167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d40053c0 msgr2=0x7fb0d400ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:34.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0ee800640 1 --2- 172.21.15.67:0/3580232167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d40053c0 0x7fb0d400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:34.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.803+0000 7fb0ede00640 1 --2- 172.21.15.67:0/3580232167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0d4003a20 0x7fb0d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.804+0000 7fb0ed400640 1 --2- 172.21.15.67:0/3580232167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 0x7fb0d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.804+0000 7fb0ed400640 1 -- 172.21.15.67:0/3580232167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0d4003a20 msgr2=0x7fb0d4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:34.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.804+0000 7fb0ed400640 1 --2- 172.21.15.67:0/3580232167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0d4003a20 0x7fb0d4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.804+0000 7fb0ed400640 1 -- 172.21.15.67:0/3580232167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d40053c0 msgr2=0x7fb0d400ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:34.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.804+0000 7fb0ed400640 1 --2- 172.21.15.67:0/3580232167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0d40053c0 0x7fb0d400ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.804+0000 7fb0ed400640 1 -- 172.21.15.67:0/3580232167 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d4003520 con 0x7fb0d4004800 2024-03-09T09:19:34.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.804+0000 7fb0ee800640 1 -- 172.21.15.67:0/3580232167 reap_dead start 2024-03-09T09:19:34.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.806+0000 7fb0ed400640 1 --2- 172.21.15.67:0/3580232167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 0x7fb0d4004c80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009900 tx=0x7fb0d4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2de58b0bbb2cd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.807+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc009500 con 0x7fb0d4004800 2024-03-09T09:19:34.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.807+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc02f070 con 0x7fb0d4004800 2024-03-09T09:19:34.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.807+0000 7fb0eca00640 1 -- 172.21.15.67:0/3580232167 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc003df0 con 0x7fb0d4004800 2024-03-09T09:19:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.807+0000 7fb0ef200640 1 -- 172.21.15.67:0/3580232167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 msgr2=0x7fb0d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.807+0000 7fb0ef200640 1 --2- 172.21.15.67:0/3580232167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 0x7fb0d4004c80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009900 tx=0x7fb0d4005830 comp rx=0 tx=0).stop 2024-03-09T09:19:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.807+0000 7fb0ef200640 1 -- 172.21.15.67:0/3580232167 shutdown_connections 2024-03-09T09:19:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.807+0000 7fb0ef200640 1 --2- 172.21.15.67:0/3580232167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 0x7fb0d4004c80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.807+0000 7fb0ef200640 1 -- 172.21.15.67:0/3580232167 >> 172.21.15.67:0/3580232167 conn(0x7fb0f00ad190 msgr2=0x7fb0f00ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.807+0000 7fb0ef200640 1 -- 172.21.15.67:0/3580232167 shutdown_connections 2024-03-09T09:19:34.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.807+0000 7fb0ef200640 1 -- 172.21.15.67:0/3580232167 wait complete. 2024-03-09T09:19:34.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.809+0000 7fb0ef200640 1 Processor -- start 2024-03-09T09:19:34.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.809+0000 7fb0ef200640 1 -- start start 2024-03-09T09:19:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.810+0000 7fb0ef200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 0x7fb0f00acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.810+0000 7fb0ef200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f006cdb0 0x7fb0f014e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.810+0000 7fb0ef200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0f01e8670 0x7fb0f01e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.810+0000 7fb0ef200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00b06f0 con 0x7fb0f006cdb0 2024-03-09T09:19:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.810+0000 7fb0ef200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00b01c0 con 0x7fb0d4004800 2024-03-09T09:19:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.810+0000 7fb0ef200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f01535a0 con 0x7fb0f01e8670 2024-03-09T09:19:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.810+0000 7fb0ed400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f006cdb0 msgr2=0x7fb0f014e6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.810+0000 7fb0ed400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f006cdb0 0x7fb0f014e6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.811+0000 7fb0ede00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 0x7fb0f00acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.811+0000 7fb0ee800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0f01e8670 0x7fb0f01e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.811+0000 7fb0ede00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 0x7fb0f00acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:34546/0 (socket says 172.21.15.67:34546) 2024-03-09T09:19:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.811+0000 7fb0ede00640 1 -- 172.21.15.67:0/2291568073 learned_addr learned my addr 172.21.15.67:0/2291568073 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.812+0000 7fb0ee800640 1 -- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 msgr2=0x7fb0f00acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.812+0000 7fb0ee800640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 0x7fb0f00acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.812+0000 7fb0ee800640 1 -- 172.21.15.67:0/2291568073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f006cdb0 msgr2=0x7fb0f014e6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:19:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.812+0000 7fb0ee800640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f006cdb0 0x7fb0f014e6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:34.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.812+0000 7fb0ee800640 1 -- 172.21.15.67:0/2291568073 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f01ef710 con 0x7fb0f01e8670 2024-03-09T09:19:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.812+0000 7fb0ee800640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0f01e8670 0x7fb0f01e8b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0159130 tx=0x7fb0e8009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.813+0000 7fb0e6000640 1 -- 172.21.15.67:0/2291568073 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8015670 con 0x7fb0f01e8670 2024-03-09T09:19:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.813+0000 7fb0e6000640 1 -- 172.21.15.67:0/2291568073 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e8010070 con 0x7fb0f01e8670 2024-03-09T09:19:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.813+0000 7fb0e6000640 1 -- 172.21.15.67:0/2291568073 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8015a70 con 0x7fb0f01e8670 2024-03-09T09:19:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.813+0000 7fb0ef200640 1 -- 172.21.15.67:0/2291568073 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f006b170 con 0x7fb0f01e8670 2024-03-09T09:19:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.813+0000 7fb0ef200640 1 -- 172.21.15.67:0/2291568073 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f014ec00 con 0x7fb0f01e8670 2024-03-09T09:19:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.814+0000 7fb0ef200640 1 -- 172.21.15.67:0/2291568073 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f004f3c0 con 0x7fb0f01e8670 2024-03-09T09:19:34.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.817+0000 7fb0e6000640 1 -- 172.21.15.67:0/2291568073 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb0e8005070 con 0x7fb0f01e8670 2024-03-09T09:19:34.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.818+0000 7fb0e6000640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb0d006b4b0 0x7fb0d006d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:34.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.818+0000 7fb0e6000640 1 -- 172.21.15.67:0/2291568073 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb0e808b460 con 0x7fb0f01e8670 2024-03-09T09:19:34.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.819+0000 7fb0ede00640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb0d006b4b0 0x7fb0d006d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:34.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.820+0000 7fb0ede00640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb0d006b4b0 0x7fb0d006d970 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0001050 tx=0x7fb0e0000fc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:34.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:34.832+0000 7fb0e6000640 1 -- 172.21.15.67:0/2291568073 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb0e8023080 con 0x7fb0f01e8670 2024-03-09T09:19:35.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.106+0000 7fb0ef200640 1 -- 172.21.15.67:0/2291568073 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb0f00cac30 con 0x7fb0f01e8670 2024-03-09T09:19:35.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.107+0000 7fb0e6000640 1 -- 172.21.15.67:0/2291568073 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fb0e8059f60 con 0x7fb0f01e8670 2024-03-09T09:19:35.113 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":200,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:19:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.110+0000 7fb0ef200640 1 -- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb0d006b4b0 msgr2=0x7fb0d006d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.110+0000 7fb0ef200640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb0d006b4b0 0x7fb0d006d970 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0001050 tx=0x7fb0e0000fc0 comp rx=0 tx=0).stop 2024-03-09T09:19:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.110+0000 7fb0ef200640 1 -- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0f01e8670 msgr2=0x7fb0f01e8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.110+0000 7fb0ef200640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0f01e8670 0x7fb0f01e8b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0159130 tx=0x7fb0e8009eb0 comp rx=0 tx=0).stop 2024-03-09T09:19:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.111+0000 7fb0ef200640 1 -- 172.21.15.67:0/2291568073 shutdown_connections 2024-03-09T09:19:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.111+0000 7fb0ef200640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb0d006b4b0 0x7fb0d006d970 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.111+0000 7fb0ef200640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0f01e8670 0x7fb0f01e8b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.111+0000 7fb0ef200640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb0f006cdb0 0x7fb0f014e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.111+0000 7fb0ef200640 1 --2- 172.21.15.67:0/2291568073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0d4004800 0x7fb0f00acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:35.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.111+0000 7fb0ef200640 1 -- 172.21.15.67:0/2291568073 >> 172.21.15.67:0/2291568073 conn(0x7fb0f00ad190 msgr2=0x7fb0f0158270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.111+0000 7fb0ef200640 1 -- 172.21.15.67:0/2291568073 shutdown_connections 2024-03-09T09:19:35.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:35.111+0000 7fb0ef200640 1 -- 172.21.15.67:0/2291568073 wait complete. 2024-03-09T09:19:38.111 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:38.111 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:38.111 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 35 -eq 48 ']' 2024-03-09T09:19:38.111 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.35 2024-03-09T09:19:38.111 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.35 2024-03-09T09:19:38.112 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.35 2024-03-09T09:19:38.112 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.35 2024-03-09T09:19:38.115 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:38.141 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:19:38.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.377+0000 7efc23200640 1 Processor -- start 2024-03-09T09:19:38.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.377+0000 7efc23200640 1 -- start start 2024-03-09T09:19:38.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.379+0000 7efc23200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efc1c0c2db0 0x7efc1c06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:38.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.379+0000 7efc23200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc1c0c4660 0x7efc1c159360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:38.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.379+0000 7efc23200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc1c0c4c50 con 0x7efc1c1598a0 2024-03-09T09:19:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.379+0000 7efc23200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc1c071460 con 0x7efc1c0c4660 2024-03-09T09:19:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.379+0000 7efc23200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc1c0c41d0 con 0x7efc1c0c2db0 2024-03-09T09:19:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.379+0000 7efc21200640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7efc1c1598a0 legacy=0x7efc1c14fce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:19:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.379+0000 7efc1b400640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7efc1c0c4660 msgr2=0x7efc1c159360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:38.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.379+0000 7efc1b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc1c0c4660 0x7efc1c159360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:38.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.379+0000 7efc1be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efc1c0c2db0 0x7efc1c06d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:38.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.379+0000 7efc1be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efc1c0c2db0 0x7efc1c06d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:34562/0 (socket says 172.21.15.67:34562) 2024-03-09T09:19:38.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.379+0000 7efc1be00640 1 -- 172.21.15.67:0/3609379838 learned_addr learned my addr 172.21.15.67:0/3609379838 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:38.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.380+0000 7efc1be00640 1 -- 172.21.15.67:0/3609379838 >> v2:172.21.15.67:3300/0 conn(0x7efc1c0c4660 msgr2=0x7efc1c159360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:38.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.380+0000 7efc1be00640 1 --2- 172.21.15.67:0/3609379838 >> v2:172.21.15.67:3300/0 conn(0x7efc1c0c4660 0x7efc1c159360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:38.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.380+0000 7efc1be00640 1 -- 172.21.15.67:0/3609379838 >> v1:172.21.15.67:6789/0 conn(0x7efc1c1598a0 legacy=0x7efc1c14fce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:38.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.380+0000 7efc1be00640 1 -- 172.21.15.67:0/3609379838 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0c4f30 con 0x7efc1c0c2db0 2024-03-09T09:19:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.381+0000 7efc1be00640 1 --2- 172.21.15.67:0/3609379838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc1c0c2db0 0x7efc1c06d2d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc1c073a90 tx=0x7efc0c02b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af82eec03c0804b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.382+0000 7efc1aa00640 1 -- 172.21.15.67:0/3609379838 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c031770 con 0x7efc1c0c2db0 2024-03-09T09:19:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.382+0000 7efc1aa00640 1 -- 172.21.15.67:0/3609379838 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c037070 con 0x7efc1c0c2db0 2024-03-09T09:19:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.382+0000 7efc1aa00640 1 -- 172.21.15.67:0/3609379838 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c031b30 con 0x7efc1c0c2db0 2024-03-09T09:19:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.382+0000 7efc23200640 1 -- 172.21.15.67:0/3609379838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc1c0c2db0 msgr2=0x7efc1c06d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.382+0000 7efc23200640 1 --2- 172.21.15.67:0/3609379838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc1c0c2db0 0x7efc1c06d2d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efc1c073a90 tx=0x7efc0c02b810 comp rx=0 tx=0).stop 2024-03-09T09:19:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.383+0000 7efc23200640 1 -- 172.21.15.67:0/3609379838 shutdown_connections 2024-03-09T09:19:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.383+0000 7efc23200640 1 --2- 172.21.15.67:0/3609379838 >> v2:172.21.15.67:3300/0 conn(0x7efc1c0c4660 0x7efc1c159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.383+0000 7efc23200640 1 --2- 172.21.15.67:0/3609379838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc1c0c2db0 0x7efc1c06d2d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.383+0000 7efc23200640 1 -- 172.21.15.67:0/3609379838 >> 172.21.15.67:0/3609379838 conn(0x7efc1c0c9710 msgr2=0x7efc1c0c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.383+0000 7efc23200640 1 -- 172.21.15.67:0/3609379838 shutdown_connections 2024-03-09T09:19:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.383+0000 7efc23200640 1 -- 172.21.15.67:0/3609379838 wait complete. 2024-03-09T09:19:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.384+0000 7efc23200640 1 Processor -- start 2024-03-09T09:19:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.385+0000 7efc23200640 1 -- start start 2024-03-09T09:19:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.385+0000 7efc23200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc1c0c2db0 0x7efc1c06fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.386+0000 7efc23200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1c0c4660 0x7efc1c1ebec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.386+0000 7efc23200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc1c1598a0 0x7efc1c1ec400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:38.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.386+0000 7efc23200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c0c5390 con 0x7efc1c0c4660 2024-03-09T09:19:38.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.386+0000 7efc23200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c0c5210 con 0x7efc1c1598a0 2024-03-09T09:19:38.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.386+0000 7efc23200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c1503f0 con 0x7efc1c0c2db0 2024-03-09T09:19:38.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.386+0000 7efc1b400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1c0c4660 msgr2=0x7efc1c1ebec0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:38.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.386+0000 7efc1b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1c0c4660 0x7efc1c1ebec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:38.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.386+0000 7efc1be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc1c0c2db0 0x7efc1c06fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:38.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.386+0000 7efc1be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc1c0c2db0 0x7efc1c06fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:60442/0 (socket says 172.21.15.67:60442) 2024-03-09T09:19:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.386+0000 7efc21200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc1c1598a0 0x7efc1c1ec400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.386+0000 7efc1be00640 1 -- 172.21.15.67:0/845733924 learned_addr learned my addr 172.21.15.67:0/845733924 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.387+0000 7efc1be00640 1 -- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc1c1598a0 msgr2=0x7efc1c1ec400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.387+0000 7efc1be00640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc1c1598a0 0x7efc1c1ec400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.387+0000 7efc1be00640 1 -- 172.21.15.67:0/845733924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1c0c4660 msgr2=0x7efc1c1ebec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:19:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.387+0000 7efc1be00640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1c0c4660 0x7efc1c1ebec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:38.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.387+0000 7efc1be00640 1 -- 172.21.15.67:0/845733924 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c1efca0 con 0x7efc1c0c2db0 2024-03-09T09:19:38.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.387+0000 7efc21200640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc1c1598a0 0x7efc1c1ec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:19:38.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.387+0000 7efc1be00640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc1c0c2db0 0x7efc1c06fab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efc0c000c00 tx=0x7efc0c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:38.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.388+0000 7efc13e00640 1 -- 172.21.15.67:0/845733924 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c032810 con 0x7efc1c0c2db0 2024-03-09T09:19:38.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.388+0000 7efc13e00640 1 -- 172.21.15.67:0/845733924 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c0329b0 con 0x7efc1c0c2db0 2024-03-09T09:19:38.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.388+0000 7efc23200640 1 -- 172.21.15.67:0/845733924 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1c1586e0 con 0x7efc1c0c2db0 2024-03-09T09:19:38.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.388+0000 7efc13e00640 1 -- 172.21.15.67:0/845733924 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c032d80 con 0x7efc1c0c2db0 2024-03-09T09:19:38.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.388+0000 7efc23200640 1 -- 172.21.15.67:0/845733924 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1c158ca0 con 0x7efc1c0c2db0 2024-03-09T09:19:38.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.389+0000 7efc23200640 1 -- 172.21.15.67:0/845733924 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1c0750f0 con 0x7efc1c0c2db0 2024-03-09T09:19:38.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.391+0000 7efc13e00640 1 -- 172.21.15.67:0/845733924 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7efc0c04c060 con 0x7efc1c0c2db0 2024-03-09T09:19:38.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.392+0000 7efc13e00640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efbfc06b480 0x7efbfc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:38.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.392+0000 7efc13e00640 1 -- 172.21.15.67:0/845733924 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7efc0c0abfd0 con 0x7efc1c0c2db0 2024-03-09T09:19:38.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.393+0000 7efc1b400640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efbfc06b480 0x7efbfc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:38.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.394+0000 7efc1b400640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efbfc06b480 0x7efbfc06d940 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efc1c06d0f0 tx=0x7efc08006d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:38.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.403+0000 7efc13e00640 1 -- 172.21.15.67:0/845733924 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efc0c040070 con 0x7efc1c0c2db0 2024-03-09T09:19:38.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.698+0000 7efc23200640 1 -- 172.21.15.67:0/845733924 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7efc1c075a00 con 0x7efc1c0c2db0 2024-03-09T09:19:38.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.700+0000 7efc13e00640 1 -- 172.21.15.67:0/845733924 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7efc0c07abd0 con 0x7efc1c0c2db0 2024-03-09T09:19:38.706 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":200,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:19:38.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.702+0000 7efc23200640 1 -- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efbfc06b480 msgr2=0x7efbfc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:38.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.702+0000 7efc23200640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efbfc06b480 0x7efbfc06d940 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efc1c06d0f0 tx=0x7efc08006d30 comp rx=0 tx=0).stop 2024-03-09T09:19:38.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.702+0000 7efc23200640 1 -- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc1c0c2db0 msgr2=0x7efc1c06fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:38.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.702+0000 7efc23200640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc1c0c2db0 0x7efc1c06fab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efc0c000c00 tx=0x7efc0c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:19:38.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.702+0000 7efc23200640 1 -- 172.21.15.67:0/845733924 shutdown_connections 2024-03-09T09:19:38.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.702+0000 7efc23200640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efbfc06b480 0x7efbfc06d940 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:38.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.702+0000 7efc23200640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc1c1598a0 0x7efc1c1ec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:38.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.702+0000 7efc23200640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc1c0c4660 0x7efc1c1ebec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:38.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.702+0000 7efc23200640 1 --2- 172.21.15.67:0/845733924 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc1c0c2db0 0x7efc1c06fab0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:38.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.702+0000 7efc23200640 1 -- 172.21.15.67:0/845733924 >> 172.21.15.67:0/845733924 conn(0x7efc1c0c9710 msgr2=0x7efc1c14d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:38.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.703+0000 7efc23200640 1 -- 172.21.15.67:0/845733924 shutdown_connections 2024-03-09T09:19:38.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:38.703+0000 7efc23200640 1 -- 172.21.15.67:0/845733924 wait complete. 2024-03-09T09:19:39.205 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:39.205 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:39.205 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 40 -eq 48 ']' 2024-03-09T09:19:39.205 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.40 2024-03-09T09:19:39.205 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.40 2024-03-09T09:19:39.205 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.40 2024-03-09T09:19:39.205 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.40 2024-03-09T09:19:39.209 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:41.727 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:19:41.828 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:41.828 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:41.828 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 36 -eq 48 ']' 2024-03-09T09:19:41.828 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.36 2024-03-09T09:19:41.829 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.36 2024-03-09T09:19:41.829 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.36 2024-03-09T09:19:41.829 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.36 2024-03-09T09:19:41.832 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:41.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:41.937+0000 7ff46de00640 1 Processor -- start 2024-03-09T09:19:41.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:41.938+0000 7ff46de00640 1 -- start start 2024-03-09T09:19:41.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:41.939+0000 7ff46de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff468151360 0x7ff46815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:41.939+0000 7ff46de00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680c81e0 con 0x7ff4680c1f70 2024-03-09T09:19:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:41.939+0000 7ff46de00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680c32b0 con 0x7ff4680710a0 2024-03-09T09:19:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:41.939+0000 7ff46de00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4680c2fd0 con 0x7ff468151360 2024-03-09T09:19:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:41.939+0000 7ff46ca00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7ff4680c1f70 legacy=0x7ff468150c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:19:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:41.939+0000 7ff46d400640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7ff468151360 msgr2=0x7ff46815f2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:41.939+0000 7ff46d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff468151360 0x7ff46815f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:41.939+0000 7ff467e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7ff4680710a0 0x7ff468153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:42408/0 (socket says 172.21.15.67:42408) 2024-03-09T09:19:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:41.940+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 learned_addr learned my addr 172.21.15.67:0/1041767767 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:19:42.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:42.140+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.67:3300/0 conn(0x7ff468151360 msgr2=0x7ff46815f2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:42.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:42.140+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> v1:172.21.15.67:6789/0 conn(0x7ff4680c1f70 legacy=0x7ff468150c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:19:42.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:42.140+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.67:3300/0 conn(0x7ff468151360 0x7ff46815f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:19:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:42.541+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> v1:172.21.15.67:6789/0 conn(0x7ff4680c1f70 legacy=0x7ff468150c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:19:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:42.541+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.67:3300/0 conn(0x7ff468151360 msgr2=0x7ff46815f2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:42.541+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.67:3300/0 conn(0x7ff468151360 0x7ff46815f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:19:43.206 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:43.206 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:43.206 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 41 -eq 48 ']' 2024-03-09T09:19:43.206 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.41 2024-03-09T09:19:43.207 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.41 2024-03-09T09:19:43.207 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.41 2024-03-09T09:19:43.207 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.41 2024-03-09T09:19:43.210 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:43.342+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> v1:172.21.15.67:6789/0 conn(0x7ff4680c1f70 legacy=0x7ff468150c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:19:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:43.342+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.67:3300/0 conn(0x7ff468151360 msgr2=0x7ff46815f2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:43.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:43.342+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.67:3300/0 conn(0x7ff468151360 0x7ff46815f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:19:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.939+0000 7ff466000640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.67:3300/0 conn(0x7ff468151360 msgr2=0x7ff46815f2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.939+0000 7ff466000640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.67:3300/0 conn(0x7ff468151360 0x7ff46815f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.939+0000 7ff466000640 1 -- 172.21.15.67:0/1041767767 >> v1:172.21.15.134:6789/0 conn(0x7ff4680710a0 legacy=0x7ff468153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.939+0000 7ff466000640 1 -- 172.21.15.67:0/1041767767 >> v1:172.21.15.67:6789/0 conn(0x7ff4680c1f70 legacy=0x7ff468150c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.939+0000 7ff466000640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.939+0000 7ff466000640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.939+0000 7ff466000640 1 -- 172.21.15.67:0/1041767767 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4680c2fd0 con 0x7ff4480061a0 2024-03-09T09:19:44.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.939+0000 7ff466000640 1 -- 172.21.15.67:0/1041767767 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff448008d20 con 0x7ff4480036c0 2024-03-09T09:19:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.940+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.940+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.941+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 msgr2=0x7ff448005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:19:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.941+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 msgr2=0x7ff448005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.941+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:44.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.941+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.941+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 msgr2=0x7ff448008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:19:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.941+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 msgr2=0x7ff448008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.941+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:44.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:44.941+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.142+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:45.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.143+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.143+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 msgr2=0x7ff448005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:19:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.143+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 msgr2=0x7ff448005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.143+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.143+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:19:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.143+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 msgr2=0x7ff448008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:19:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.143+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 msgr2=0x7ff448008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.143+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:45.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.143+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:19:45.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.544+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:45.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.545+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:45.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.545+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 msgr2=0x7ff448005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:19:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.545+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 msgr2=0x7ff448005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.545+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.545+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:19:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.545+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 msgr2=0x7ff448008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:19:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.545+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 msgr2=0x7ff448008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.545+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:45.545+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:19:46.185 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:46.185 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:46.185 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 37 -eq 48 ']' 2024-03-09T09:19:46.185 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.37 2024-03-09T09:19:46.186 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.37 2024-03-09T09:19:46.186 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.37 2024-03-09T09:19:46.186 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.37 2024-03-09T09:19:46.190 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:46.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:46.346+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:46.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:46.346+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:46.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:46.347+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 msgr2=0x7ff448005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:19:46.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:46.347+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 msgr2=0x7ff448005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:46.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:46.347+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:46.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:46.347+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 msgr2=0x7ff448008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:19:46.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:46.347+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 msgr2=0x7ff448008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:46.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:46.347+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:46.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:46.347+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:19:46.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:46.347+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:19:47.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.939+0000 7ff466000640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 msgr2=0x7ff448005b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.939+0000 7ff466000640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4480036c0 0x7ff448005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.940+0000 7ff466000640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 msgr2=0x7ff448008660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.940+0000 7ff466000640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff4480061a0 0x7ff448008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.940+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 reap_dead start 2024-03-09T09:19:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.940+0000 7ff466000640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 0x7ff44800e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.940+0000 7ff466000640 1 -- 172.21.15.67:0/1041767767 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680c3700 con 0x7ff448006650 2024-03-09T09:19:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.940+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> v1:172.21.15.67:6789/0 conn(0x7ff448006650 legacy=0x7ff448003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:19:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.940+0000 7ff466000640 1 -- 172.21.15.67:0/1041767767 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680c81e0 con 0x7ff448005200 2024-03-09T09:19:47.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.940+0000 7ff466000640 1 -- 172.21.15.67:0/1041767767 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4680c2fd0 con 0x7ff44800c200 2024-03-09T09:19:47.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.941+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 0x7ff44800e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:47.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.941+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 msgr2=0x7ff44800e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:19:47.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.941+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 msgr2=0x7ff44800e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:47.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.941+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 0x7ff44800e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:47.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:47.941+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 0x7ff44800e5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:48.092 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:48.092 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:48.092 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 42 -eq 48 ']' 2024-03-09T09:19:48.092 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.42 2024-03-09T09:19:48.093 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.42 2024-03-09T09:19:48.093 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.42 2024-03-09T09:19:48.093 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.42 2024-03-09T09:19:48.096 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:48.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.141+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> v1:172.21.15.67:6789/0 conn(0x7ff448006650 legacy=0x7ff448003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:19:48.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.142+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 0x7ff44800e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:48.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.143+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 msgr2=0x7ff44800e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:19:48.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.143+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 msgr2=0x7ff44800e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:48.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.143+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 0x7ff44800e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:48.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.143+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 0x7ff44800e5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:19:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.542+0000 7ff46ca00640 1 -- 172.21.15.67:0/1041767767 >> v1:172.21.15.67:6789/0 conn(0x7ff448006650 legacy=0x7ff448003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:19:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.544+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 0x7ff44800e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.545+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 msgr2=0x7ff44800e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:19:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.545+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 msgr2=0x7ff44800e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:19:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.545+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 0x7ff44800e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:19:48.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.545+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 0x7ff44800e5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:19:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.788+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854724811 0 0) 0x7ff4680c81e0 con 0x7ff448005200 2024-03-09T09:19:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.789+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff450003520 con 0x7ff448005200 2024-03-09T09:19:48.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.794+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1731890706 0 0) 0x7ff450003520 con 0x7ff448005200 2024-03-09T09:19:48.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.794+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 msgr2=0x7ff44800e5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:48.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.794+0000 7ff467400640 1 --2- 172.21.15.67:0/1041767767 >> v2:172.21.15.134:3300/0 conn(0x7ff44800c200 0x7ff44800e5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:48.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.794+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 >> v1:172.21.15.67:6789/0 conn(0x7ff448006650 legacy=0x7ff448003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:48.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.795+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff448010cd0 con 0x7ff448005200 2024-03-09T09:19:48.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.795+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7ff4540015a0 con 0x7ff448005200 2024-03-09T09:19:48.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.795+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 >> v1:172.21.15.134:6789/0 conn(0x7ff448005200 legacy=0x7ff448005690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:48.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.795+0000 7ff467400640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff450003a20 0x7ff450003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:48.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.795+0000 7ff467400640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff450004800 0x7ff450004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:48.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.795+0000 7ff467400640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500052e0 0x7ff45000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:48.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.795+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680c2fd0 con 0x7ff4500052e0 2024-03-09T09:19:48.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.795+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45000c1b0 con 0x7ff450003a20 2024-03-09T09:19:48.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.795+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45000c3b0 con 0x7ff450004800 2024-03-09T09:19:48.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.795+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500052e0 msgr2=0x7ff45000b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:48.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.795+0000 7ff46d400640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500052e0 0x7ff45000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.796+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff450004800 0x7ff450004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.796+0000 7ff46ca00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff450003a20 0x7ff450003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.797+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff450003a20 msgr2=0x7ff450003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:19:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.797+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff450003a20 0x7ff450003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.797+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500052e0 msgr2=0x7ff45000b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.797+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4500052e0 0x7ff45000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.797+0000 7ff467e00640 1 -- 172.21.15.67:0/1041767767 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff450003520 con 0x7ff450004800 2024-03-09T09:19:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.797+0000 7ff46d400640 1 -- 172.21.15.67:0/1041767767 reap_dead start 2024-03-09T09:19:48.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.802+0000 7ff467e00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff450004800 0x7ff450004c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff454009010 tx=0x7ff450005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aa362a2ceb65648 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:48.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.803+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff454001980 con 0x7ff450004800 2024-03-09T09:19:48.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.803+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff454035410 con 0x7ff450004800 2024-03-09T09:19:48.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.803+0000 7ff467400640 1 -- 172.21.15.67:0/1041767767 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4540357e0 con 0x7ff450004800 2024-03-09T09:19:48.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.804+0000 7ff46de00640 1 -- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff450004800 msgr2=0x7ff450004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.804+0000 7ff46de00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff450004800 0x7ff450004c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff454009010 tx=0x7ff450005830 comp rx=0 tx=0).stop 2024-03-09T09:19:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.805+0000 7ff46de00640 1 -- 172.21.15.67:0/1041767767 shutdown_connections 2024-03-09T09:19:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.805+0000 7ff46de00640 1 --2- 172.21.15.67:0/1041767767 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff450004800 0x7ff450004c80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.805+0000 7ff46de00640 1 -- 172.21.15.67:0/1041767767 >> 172.21.15.67:0/1041767767 conn(0x7ff4680c7300 msgr2=0x7ff4680c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.805+0000 7ff46de00640 1 -- 172.21.15.67:0/1041767767 shutdown_connections 2024-03-09T09:19:48.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.805+0000 7ff46de00640 1 -- 172.21.15.67:0/1041767767 wait complete. 2024-03-09T09:19:48.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.810+0000 7ff46de00640 1 Processor -- start 2024-03-09T09:19:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.811+0000 7ff46de00640 1 -- start start 2024-03-09T09:19:48.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.813+0000 7ff46de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff450004800 0x7ff4680c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:48.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.813+0000 7ff46de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4681542d0 0x7ff4680c03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:48.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.813+0000 7ff46de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4680c0910 0x7ff46814c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:48.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.813+0000 7ff46de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff468070d50 con 0x7ff4681542d0 2024-03-09T09:19:48.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.813+0000 7ff46de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff468070830 con 0x7ff4680c0910 2024-03-09T09:19:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.813+0000 7ff46de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff468070ed0 con 0x7ff450004800 2024-03-09T09:19:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.814+0000 7ff467e00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4681542d0 msgr2=0x7ff4680c03d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.814+0000 7ff467e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4681542d0 0x7ff4680c03d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.814+0000 7ff46d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4680c0910 0x7ff46814c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.814+0000 7ff46d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4680c0910 0x7ff46814c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:44622/0 (socket says 172.21.15.67:44622) 2024-03-09T09:19:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.814+0000 7ff46d400640 1 -- 172.21.15.67:0/3035610131 learned_addr learned my addr 172.21.15.67:0/3035610131 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:48.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.814+0000 7ff46ca00640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff450004800 0x7ff4680c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:48.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.815+0000 7ff46d400640 1 -- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff450004800 msgr2=0x7ff4680c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:48.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.815+0000 7ff46d400640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff450004800 0x7ff4680c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:48.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.815+0000 7ff46d400640 1 -- 172.21.15.67:0/3035610131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4681542d0 msgr2=0x7ff4680c03d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:19:48.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.815+0000 7ff46d400640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4681542d0 0x7ff4680c03d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:48.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.815+0000 7ff46d400640 1 -- 172.21.15.67:0/3035610131 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4680bda60 con 0x7ff4680c0910 2024-03-09T09:19:48.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.815+0000 7ff46d400640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4680c0910 0x7ff46814c0b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff46000c700 tx=0x7ff46000cb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:48.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.816+0000 7ff464c00640 1 -- 172.21.15.67:0/3035610131 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46000f070 con 0x7ff4680c0910 2024-03-09T09:19:48.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.816+0000 7ff464c00640 1 -- 172.21.15.67:0/3035610131 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46000f210 con 0x7ff4680c0910 2024-03-09T09:19:48.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.816+0000 7ff464c00640 1 -- 172.21.15.67:0/3035610131 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46000b8d0 con 0x7ff4680c0910 2024-03-09T09:19:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.816+0000 7ff46de00640 1 -- 172.21.15.67:0/3035610131 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4681e77a0 con 0x7ff4680c0910 2024-03-09T09:19:48.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.816+0000 7ff46de00640 1 -- 172.21.15.67:0/3035610131 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4680c5c20 con 0x7ff4680c0910 2024-03-09T09:19:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.818+0000 7ff46de00640 1 -- 172.21.15.67:0/3035610131 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46804f3c0 con 0x7ff4680c0910 2024-03-09T09:19:48.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.819+0000 7ff464c00640 1 -- 172.21.15.67:0/3035610131 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff460017030 con 0x7ff4680c0910 2024-03-09T09:19:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.821+0000 7ff464c00640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff44806d030 0x7ff44806f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.821+0000 7ff464c00640 1 -- 172.21.15.67:0/3035610131 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff46008a4a0 con 0x7ff4680c0910 2024-03-09T09:19:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.821+0000 7ff46ca00640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff44806d030 0x7ff44806f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:48.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.822+0000 7ff46ca00640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff44806d030 0x7ff44806f4f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff45c000ea0 tx=0x7ff45c005040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:48.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:48.836+0000 7ff464c00640 1 -- 172.21.15.67:0/3035610131 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff460058f10 con 0x7ff4680c0910 2024-03-09T09:19:49.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.140+0000 7ff46de00640 1 -- 172.21.15.67:0/3035610131 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff46814cf10 con 0x7ff4680c0910 2024-03-09T09:19:49.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.141+0000 7ff464c00640 1 -- 172.21.15.67:0/3035610131 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7ff460058810 con 0x7ff4680c0910 2024-03-09T09:19:49.147 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":202,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:19:49.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 -- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff44806d030 msgr2=0x7ff44806f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff44806d030 0x7ff44806f4f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff45c000ea0 tx=0x7ff45c005040 comp rx=0 tx=0).stop 2024-03-09T09:19:49.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 -- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4680c0910 msgr2=0x7ff46814c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4680c0910 0x7ff46814c0b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff46000c700 tx=0x7ff46000cb00 comp rx=0 tx=0).stop 2024-03-09T09:19:49.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 -- 172.21.15.67:0/3035610131 shutdown_connections 2024-03-09T09:19:49.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff44806d030 0x7ff44806f4f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4680c0910 0x7ff46814c0b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff4681542d0 0x7ff4680c03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 --2- 172.21.15.67:0/3035610131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff450004800 0x7ff4680c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 -- 172.21.15.67:0/3035610131 >> 172.21.15.67:0/3035610131 conn(0x7ff4680c7300 msgr2=0x7ff46814fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:49.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 -- 172.21.15.67:0/3035610131 shutdown_connections 2024-03-09T09:19:49.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.144+0000 7ff46de00640 1 -- 172.21.15.67:0/3035610131 wait complete. 2024-03-09T09:19:49.171 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:19:49.171 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:19:49.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:19:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.378+0000 7fdf03200640 1 Processor -- start 2024-03-09T09:19:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.379+0000 7fdf03200640 1 -- start start 2024-03-09T09:19:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.380+0000 7fdf03200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0b0db0 0x7fdefc1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.380+0000 7fdf03200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc0c6010 con 0x7fdefc0c9410 2024-03-09T09:19:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.380+0000 7fdf03200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc0c62f0 con 0x7fdefc0bec10 2024-03-09T09:19:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.380+0000 7fdf03200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0c65d0 con 0x7fdefc0b0db0 2024-03-09T09:19:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.380+0000 7fdf02800640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7fdefc0c9410 legacy=0x7fdefc06b840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:19:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.381+0000 7fdf01e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0b0db0 0x7fdefc1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.381+0000 7fdf01e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0b0db0 0x7fdefc1565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:51862/0 (socket says 172.21.15.67:51862) 2024-03-09T09:19:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.381+0000 7fdf01e00640 1 -- 172.21.15.67:0/3169789579 learned_addr learned my addr 172.21.15.67:0/3169789579 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.382+0000 7fdf01e00640 1 -- 172.21.15.67:0/3169789579 >> v1:172.21.15.134:6789/0 conn(0x7fdefc0bec10 legacy=0x7fdefc072770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.382+0000 7fdf01e00640 1 -- 172.21.15.67:0/3169789579 >> v1:172.21.15.67:6789/0 conn(0x7fdefc0c9410 legacy=0x7fdefc06b840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.382+0000 7fdf01e00640 1 -- 172.21.15.67:0/3169789579 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc06b040 con 0x7fdefc0b0db0 2024-03-09T09:19:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.383+0000 7fdf01e00640 1 --2- 172.21.15.67:0/3169789579 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0b0db0 0x7fdefc1565a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdef00098a0 tx=0x7fdef002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca7e2f9f1af5260 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.383+0000 7fdf00a00640 1 -- 172.21.15.67:0/3169789579 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0035020 con 0x7fdefc0b0db0 2024-03-09T09:19:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.383+0000 7fdf00a00640 1 -- 172.21.15.67:0/3169789579 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef0031640 con 0x7fdefc0b0db0 2024-03-09T09:19:49.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.383+0000 7fdf00a00640 1 -- 172.21.15.67:0/3169789579 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef002d040 con 0x7fdefc0b0db0 2024-03-09T09:19:49.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.384+0000 7fdf03200640 1 -- 172.21.15.67:0/3169789579 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0b0db0 msgr2=0x7fdefc1565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.384+0000 7fdf03200640 1 --2- 172.21.15.67:0/3169789579 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0b0db0 0x7fdefc1565a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdef00098a0 tx=0x7fdef002b8f0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.384+0000 7fdf03200640 1 -- 172.21.15.67:0/3169789579 shutdown_connections 2024-03-09T09:19:49.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.384+0000 7fdf03200640 1 --2- 172.21.15.67:0/3169789579 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0b0db0 0x7fdefc1565a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.384+0000 7fdf03200640 1 -- 172.21.15.67:0/3169789579 >> 172.21.15.67:0/3169789579 conn(0x7fdefc0c7f90 msgr2=0x7fdefc0c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:49.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.384+0000 7fdf03200640 1 -- 172.21.15.67:0/3169789579 shutdown_connections 2024-03-09T09:19:49.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.384+0000 7fdf03200640 1 -- 172.21.15.67:0/3169789579 wait complete. 2024-03-09T09:19:49.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.386+0000 7fdf03200640 1 Processor -- start 2024-03-09T09:19:49.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.386+0000 7fdf03200640 1 -- start start 2024-03-09T09:19:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf03200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0b0db0 0x7fdefc0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf03200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0bec10 0x7fdefc156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf03200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc0c9410 0x7fdefc14fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf03200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc072e80 con 0x7fdefc0c9410 2024-03-09T09:19:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf03200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc156ae0 con 0x7fdefc0b0db0 2024-03-09T09:19:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf03200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc073000 con 0x7fdefc0bec10 2024-03-09T09:19:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf02800640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc0c9410 msgr2=0x7fdefc14fee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf02800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc0c9410 0x7fdefc14fee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf01400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0bec10 0x7fdefc156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf01e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0b0db0 0x7fdefc0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf01e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0b0db0 0x7fdefc0c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:34688/0 (socket says 172.21.15.67:34688) 2024-03-09T09:19:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf01e00640 1 -- 172.21.15.67:0/2283827099 learned_addr learned my addr 172.21.15.67:0/2283827099 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.387+0000 7fdf01400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0bec10 0x7fdefc156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:51876/0 (socket says 172.21.15.67:51876) 2024-03-09T09:19:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.388+0000 7fdf01e00640 1 -- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0bec10 msgr2=0x7fdefc156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.388+0000 7fdf01e00640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdefc0bec10 0x7fdefc156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.388+0000 7fdf01e00640 1 -- 172.21.15.67:0/2283827099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc0c9410 msgr2=0x7fdefc14fee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:19:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.388+0000 7fdf01e00640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdefc0c9410 0x7fdefc14fee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.388+0000 7fdf01e00640 1 -- 172.21.15.67:0/2283827099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0c1150 con 0x7fdefc0b0db0 2024-03-09T09:19:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.388+0000 7fdf01e00640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0b0db0 0x7fdefc0c5370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdef0000c00 tx=0x7fdef00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.389+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef003ac60 con 0x7fdefc0b0db0 2024-03-09T09:19:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.389+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef003ae00 con 0x7fdefc0b0db0 2024-03-09T09:19:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.389+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0044610 con 0x7fdefc0b0db0 2024-03-09T09:19:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.389+0000 7fdf03200640 1 -- 172.21.15.67:0/2283827099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc0c1430 con 0x7fdefc0b0db0 2024-03-09T09:19:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.389+0000 7fdf03200640 1 -- 172.21.15.67:0/2283827099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc159d90 con 0x7fdefc0b0db0 2024-03-09T09:19:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.390+0000 7fdf03200640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc159930 0x7fdefc150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.390+0000 7fdf03200640 1 -- 172.21.15.67:0/2283827099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0c72f0 con 0x7fdefc159930 2024-03-09T09:19:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.391+0000 7fdf01400640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc159930 0x7fdefc150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.391+0000 7fdf01400640 1 -- 172.21.15.67:0/2283827099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdefc0c1950 con 0x7fdefc159930 2024-03-09T09:19:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.392+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdef004b020 con 0x7fdefc0b0db0 2024-03-09T09:19:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.392+0000 7fdf01400640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc159930 0x7fdefc150420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdefc158b80 tx=0x7fdeec00f9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.393+0000 7fdef6000640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdee006b4b0 0x7fdee006d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.393+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fdef00ac440 con 0x7fdefc0b0db0 2024-03-09T09:19:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.393+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec019070 con 0x7fdefc159930 2024-03-09T09:19:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.393+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fdeec005320 con 0x7fdefc159930 2024-03-09T09:19:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.393+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc159930 msgr2=0x7fdefc150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.393+0000 7fdef6000640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc159930 0x7fdefc150420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdefc158b80 tx=0x7fdeec00f9d0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.393+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc159930 msgr2=0x7fdefc150420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.393+0000 7fdef6000640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc159930 0x7fdefc150420 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdefc158b80 tx=0x7fdeec00f9d0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.393+0000 7fdf02800640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdee006b4b0 0x7fdee006d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.394+0000 7fdf02800640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdee006b4b0 0x7fdee006d970 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0cc560 tx=0x7fdef8009270 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:49.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.400+0000 7fdf03200640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0c0be0 0x7fdefc1eeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.400+0000 7fdf03200640 1 -- 172.21.15.67:0/2283827099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc156d40 con 0x7fdefc0c0be0 2024-03-09T09:19:49.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.401+0000 7fdf01400640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0c0be0 0x7fdefc1eeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.401+0000 7fdf01400640 1 -- 172.21.15.67:0/2283827099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdefc04f3c0 con 0x7fdefc0c0be0 2024-03-09T09:19:49.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.402+0000 7fdf01400640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0c0be0 0x7fdefc1eeaf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdefc158b80 tx=0x7fdeec000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:49.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.402+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec019030 con 0x7fdefc0c0be0 2024-03-09T09:19:49.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.403+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7fdeec006aa0 con 0x7fdefc0c0be0 2024-03-09T09:19:49.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.403+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0c0be0 msgr2=0x7fdefc1eeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.403+0000 7fdef6000640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0c0be0 0x7fdefc1eeaf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdefc158b80 tx=0x7fdeec000ed0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.403+0000 7fdef6000640 1 -- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0c0be0 msgr2=0x7fdefc1eeaf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:49.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.403+0000 7fdef6000640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0c0be0 0x7fdefc1eeaf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.410 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:19:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:19:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:19:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:19:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 202, 2024-03-09T09:19:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:19:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:19:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:19:49.410 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:49.411 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:49.412 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:19:49.413 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:49.414 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:49.415 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:49.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.417 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:19:49.418 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:49.420 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:19:49.421 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:19:49.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.423 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:49.423 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:49.423 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:19:49.423 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.423 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.423 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:19:49.423 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:19:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.404+0000 7fdf03200640 1 -- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdee006b4b0 msgr2=0x7fdee006d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.404+0000 7fdf03200640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdee006b4b0 0x7fdee006d970 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0cc560 tx=0x7fdef8009270 comp rx=0 tx=0).stop 2024-03-09T09:19:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.404+0000 7fdf03200640 1 -- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0b0db0 msgr2=0x7fdefc0c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.404+0000 7fdf03200640 1 --2- 172.21.15.67:0/2283827099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdefc0b0db0 0x7fdefc0c5370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdef0000c00 tx=0x7fdef00046c0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.404+0000 7fdf02800640 1 -- 172.21.15.67:0/2283827099 reap_dead start 2024-03-09T09:19:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.404+0000 7fdf02800640 1 -- 172.21.15.67:0/2283827099 reap_dead start 2024-03-09T09:19:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.404+0000 7fdf03200640 1 -- 172.21.15.67:0/2283827099 shutdown_connections 2024-03-09T09:19:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.404+0000 7fdf03200640 1 -- 172.21.15.67:0/2283827099 >> 172.21.15.67:0/2283827099 conn(0x7fdefc0c7f90 msgr2=0x7fdefc0ae6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.404+0000 7fdf03200640 1 -- 172.21.15.67:0/2283827099 shutdown_connections 2024-03-09T09:19:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.404+0000 7fdf03200640 1 -- 172.21.15.67:0/2283827099 wait complete. 2024-03-09T09:19:49.427 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:19:49.427 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:19:49.427 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:19:49.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.644+0000 7ff2a5200640 1 Processor -- start 2024-03-09T09:19:49.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.644+0000 7ff2a5200640 1 -- start start 2024-03-09T09:19:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.645+0000 7ff2a5200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff2a014fd90 0x7ff2a0156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.646+0000 7ff2a5200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a0156bc0 con 0x7ff2a00c9380 2024-03-09T09:19:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.646+0000 7ff2a5200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a00c9890 con 0x7ff2a006c1d0 2024-03-09T09:19:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.646+0000 7ff2a5200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a006cd00 con 0x7ff2a014fd90 2024-03-09T09:19:49.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.646+0000 7ff29ea00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7ff2a00c9380 legacy=0x7ff2a00acf00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:19:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.646+0000 7ff29f400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff2a014fd90 0x7ff2a0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.646+0000 7ff29f400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff2a014fd90 0x7ff2a0156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:34718/0 (socket says 172.21.15.67:34718) 2024-03-09T09:19:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.646+0000 7ff29f400640 1 -- 172.21.15.67:0/718138674 learned_addr learned my addr 172.21.15.67:0/718138674 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.647+0000 7ff29f400640 1 -- 172.21.15.67:0/718138674 >> v1:172.21.15.134:6789/0 conn(0x7ff2a006c1d0 legacy=0x7ff2a006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.647+0000 7ff29f400640 1 -- 172.21.15.67:0/718138674 >> v1:172.21.15.67:6789/0 conn(0x7ff2a00c9380 legacy=0x7ff2a00acf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:19:49.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.647+0000 7ff29f400640 1 -- 172.21.15.67:0/718138674 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a00c5f80 con 0x7ff2a014fd90 2024-03-09T09:19:49.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.649+0000 7ff29f400640 1 --2- 172.21.15.67:0/718138674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a014fd90 0x7ff2a0156680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff290009870 tx=0x7ff29002b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d893e7484a1e631a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:49.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.649+0000 7ff29e000640 1 -- 172.21.15.67:0/718138674 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29002e070 con 0x7ff2a014fd90 2024-03-09T09:19:49.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.649+0000 7ff29e000640 1 -- 172.21.15.67:0/718138674 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff290037070 con 0x7ff2a014fd90 2024-03-09T09:19:49.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.649+0000 7ff29e000640 1 -- 172.21.15.67:0/718138674 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff290031a00 con 0x7ff2a014fd90 2024-03-09T09:19:49.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.650+0000 7ff2a5200640 1 -- 172.21.15.67:0/718138674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a014fd90 msgr2=0x7ff2a0156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.650+0000 7ff2a5200640 1 --2- 172.21.15.67:0/718138674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a014fd90 0x7ff2a0156680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff290009870 tx=0x7ff29002b880 comp rx=0 tx=0).stop 2024-03-09T09:19:49.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.650+0000 7ff2a5200640 1 -- 172.21.15.67:0/718138674 shutdown_connections 2024-03-09T09:19:49.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.650+0000 7ff2a5200640 1 --2- 172.21.15.67:0/718138674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a014fd90 0x7ff2a0156680 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.650+0000 7ff2a5200640 1 -- 172.21.15.67:0/718138674 >> 172.21.15.67:0/718138674 conn(0x7ff2a00cb480 msgr2=0x7ff2a00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:49.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.650+0000 7ff2a5200640 1 -- 172.21.15.67:0/718138674 shutdown_connections 2024-03-09T09:19:49.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.650+0000 7ff2a5200640 1 -- 172.21.15.67:0/718138674 wait complete. 2024-03-09T09:19:49.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.652+0000 7ff2a5200640 1 Processor -- start 2024-03-09T09:19:49.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.652+0000 7ff2a5200640 1 -- start start 2024-03-09T09:19:49.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.653+0000 7ff2a5200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a006c1d0 0x7ff2a00c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.653+0000 7ff2a5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2a00c9380 0x7ff2a00ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.653+0000 7ff2a5200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a014fd90 0x7ff2a01eba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.653+0000 7ff2a5200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00c6520 con 0x7ff2a00c9380 2024-03-09T09:19:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.653+0000 7ff2a5200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a006ba60 con 0x7ff2a014fd90 2024-03-09T09:19:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.653+0000 7ff2a5200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a006bcb0 con 0x7ff2a006c1d0 2024-03-09T09:19:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.653+0000 7ff29ea00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2a00c9380 msgr2=0x7ff2a00ac1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:19:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.653+0000 7ff29ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2a00c9380 0x7ff2a00ac1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:19:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.653+0000 7ff29fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a014fd90 0x7ff2a01eba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.653+0000 7ff29f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a006c1d0 0x7ff2a00c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.653+0000 7ff29fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a014fd90 0x7ff2a01eba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:34732/0 (socket says 172.21.15.67:34732) 2024-03-09T09:19:49.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.654+0000 7ff29fe00640 1 -- 172.21.15.67:0/3502359559 learned_addr learned my addr 172.21.15.67:0/3502359559 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:19:49.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.654+0000 7ff29fe00640 1 -- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a006c1d0 msgr2=0x7ff2a00c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.654+0000 7ff29fe00640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a006c1d0 0x7ff2a00c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.654+0000 7ff29fe00640 1 -- 172.21.15.67:0/3502359559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2a00c9380 msgr2=0x7ff2a00ac1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:19:49.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.654+0000 7ff29fe00640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2a00c9380 0x7ff2a00ac1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.654+0000 7ff29fe00640 1 -- 172.21.15.67:0/3502359559 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a00c3a50 con 0x7ff2a014fd90 2024-03-09T09:19:49.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.655+0000 7ff29fe00640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a014fd90 0x7ff2a01eba00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff29800c6b0 tx=0x7ff29800cb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:49.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.655+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298003b70 con 0x7ff2a014fd90 2024-03-09T09:19:49.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.655+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff298003d30 con 0x7ff2a014fd90 2024-03-09T09:19:49.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.655+0000 7ff2a5200640 1 -- 172.21.15.67:0/3502359559 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a00c5250 con 0x7ff2a014fd90 2024-03-09T09:19:49.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.655+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298005310 con 0x7ff2a014fd90 2024-03-09T09:19:49.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.655+0000 7ff2a5200640 1 -- 172.21.15.67:0/3502359559 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a00ca6f0 con 0x7ff2a014fd90 2024-03-09T09:19:49.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.656+0000 7ff2a5200640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00c2fe0 0x7ff2a0148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.656+0000 7ff2a5200640 1 -- 172.21.15.67:0/3502359559 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00c5020 con 0x7ff2a00c2fe0 2024-03-09T09:19:49.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.657+0000 7ff29f400640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00c2fe0 0x7ff2a0148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.658+0000 7ff29f400640 1 -- 172.21.15.67:0/3502359559 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2a004f3c0 con 0x7ff2a00c2fe0 2024-03-09T09:19:49.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.658+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff2980054b0 con 0x7ff2a014fd90 2024-03-09T09:19:49.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.659+0000 7ff29f400640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00c2fe0 0x7ff2a0148760 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00ae040 tx=0x7ff290004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:49.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.659+0000 7ff297400640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff28006b480 0x7ff28006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.660+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff29808a3f0 con 0x7ff2a014fd90 2024-03-09T09:19:49.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.660+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff290032680 con 0x7ff2a00c2fe0 2024-03-09T09:19:49.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.660+0000 7ff29ea00640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff28006b480 0x7ff28006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.660+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff290037040 con 0x7ff2a00c2fe0 2024-03-09T09:19:49.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.660+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00c2fe0 msgr2=0x7ff2a0148760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.660+0000 7ff297400640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00c2fe0 0x7ff2a0148760 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00ae040 tx=0x7ff290004730 comp rx=0 tx=0).stop 2024-03-09T09:19:49.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.660+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00c2fe0 msgr2=0x7ff2a0148760 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:49.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.660+0000 7ff297400640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00c2fe0 0x7ff2a0148760 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.661+0000 7ff29ea00640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff28006b480 0x7ff28006d940 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff28c006fd0 tx=0x7ff28c006d80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:49.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.674+0000 7ff2a5200640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00aca00 0x7ff2a0148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:19:49.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.674+0000 7ff2a5200640 1 -- 172.21.15.67:0/3502359559 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a006d2d0 con 0x7ff2a00aca00 2024-03-09T09:19:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.675+0000 7ff29f400640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00aca00 0x7ff2a0148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:19:49.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.676+0000 7ff29f400640 1 -- 172.21.15.67:0/3502359559 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff2a00cba00 con 0x7ff2a00aca00 2024-03-09T09:19:49.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.676+0000 7ff29f400640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00aca00 0x7ff2a0148ca0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff290000c00 tx=0x7ff2900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:19:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.677+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29004db30 con 0x7ff2a00aca00 2024-03-09T09:19:49.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.677+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5006 (secure 0 0 0) 0x7ff29004db30 con 0x7ff2a00aca00 2024-03-09T09:19:49.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.677+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00aca00 msgr2=0x7ff2a0148ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.677+0000 7ff297400640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00aca00 0x7ff2a0148ca0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff290000c00 tx=0x7ff2900046c0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.677+0000 7ff297400640 1 -- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00aca00 msgr2=0x7ff2a0148ca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:19:49.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.677+0000 7ff297400640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2a00aca00 0x7ff2a0148ca0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff290000c00 tx=0x7ff2900046c0 comp rx=0 tx=0).stop 2024-03-09T09:19:49.683 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:19:49.683 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:19:49.683 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:19:49.683 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 202, 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:49.684 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:49.685 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:19:49.686 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:19:49.687 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.688 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:19:49.689 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:19:49.690 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.691 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:19:49.692 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.693 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.694 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.695 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:19:49.696 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:19:49.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.678+0000 7ff2a5200640 1 -- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff28006b480 msgr2=0x7ff28006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.678+0000 7ff2a5200640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff28006b480 0x7ff28006d940 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff28c006fd0 tx=0x7ff28c006d80 comp rx=0 tx=0).stop 2024-03-09T09:19:49.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.678+0000 7ff2a5200640 1 -- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a014fd90 msgr2=0x7ff2a01eba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:19:49.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.678+0000 7ff2a5200640 1 --2- 172.21.15.67:0/3502359559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2a014fd90 0x7ff2a01eba00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff29800c6b0 tx=0x7ff29800cb70 comp rx=0 tx=0).stop 2024-03-09T09:19:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.678+0000 7ff29fe00640 1 -- 172.21.15.67:0/3502359559 reap_dead start 2024-03-09T09:19:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.678+0000 7ff29fe00640 1 -- 172.21.15.67:0/3502359559 reap_dead start 2024-03-09T09:19:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.678+0000 7ff2a5200640 1 -- 172.21.15.67:0/3502359559 shutdown_connections 2024-03-09T09:19:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.678+0000 7ff2a5200640 1 -- 172.21.15.67:0/3502359559 >> 172.21.15.67:0/3502359559 conn(0x7ff2a00cb480 msgr2=0x7ff2a014c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:19:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.678+0000 7ff2a5200640 1 -- 172.21.15.67:0/3502359559 shutdown_connections 2024-03-09T09:19:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:19:49.678+0000 7ff2a5200640 1 -- 172.21.15.67:0/3502359559 wait complete. 2024-03-09T09:19:49.701 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:19:49.701 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:19:55.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:19:55.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:19:55.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 38 -eq 48 ']' 2024-03-09T09:19:55.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.38 2024-03-09T09:19:55.450 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.38 2024-03-09T09:19:55.450 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.38 2024-03-09T09:19:55.450 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.38 2024-03-09T09:19:55.454 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:19:57.604 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:19:57.604 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:19:57.605 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 43 -eq 48 ']' 2024-03-09T09:19:57.605 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.43 2024-03-09T09:19:57.605 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.43 2024-03-09T09:19:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.43 2024-03-09T09:19:57.605 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.43 2024-03-09T09:19:57.609 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:19:59.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:19:59.327 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:20:01.749 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:20:01.749 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:20:01.749 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 39 -eq 48 ']' 2024-03-09T09:20:01.749 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.39 2024-03-09T09:20:01.750 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.39 2024-03-09T09:20:01.750 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.39 2024-03-09T09:20:01.750 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.39 2024-03-09T09:20:01.753 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:20:02.897 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:20:02.897 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:02.897 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 44 -eq 48 ']' 2024-03-09T09:20:02.898 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.44 2024-03-09T09:20:02.898 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.44 2024-03-09T09:20:02.898 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.44 2024-03-09T09:20:02.898 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.44 2024-03-09T09:20:02.899 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:20:05.926 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:20:05.926 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:20:05.926 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 40 -eq 48 ']' 2024-03-09T09:20:05.927 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.40 2024-03-09T09:20:05.927 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.40 2024-03-09T09:20:05.927 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.40 2024-03-09T09:20:05.927 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.40 2024-03-09T09:20:05.931 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:20:09.562 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:20:09.563 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:09.563 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 45 -eq 48 ']' 2024-03-09T09:20:09.563 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.45 2024-03-09T09:20:09.563 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.45 2024-03-09T09:20:09.563 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.45 2024-03-09T09:20:09.563 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.45 2024-03-09T09:20:09.566 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:20:09.702 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:20:09.702 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-03-09T09:20:09.702 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-09T09:20:09.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-09T09:20:09.712 INFO:tasks.ceph.mon.a:Started 2024-03-09T09:20:09.712 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:20:09.712 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:20:09.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.926+0000 7f726ec00640 1 Processor -- start 2024-03-09T09:20:09.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.926+0000 7f726ec00640 1 -- start start 2024-03-09T09:20:09.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.927+0000 7f726ec00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 0x7f72680c0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:09.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.927+0000 7f726ec00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72680c74f0 con 0x7f726806f450 2024-03-09T09:20:09.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.927+0000 7f726ec00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72680c1dc0 con 0x7f72680c1520 2024-03-09T09:20:09.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.927+0000 7f726ec00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72680c1b70 con 0x7f72680c0c00 2024-03-09T09:20:09.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.928+0000 7f725fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f726806f450 0x7f72680c2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53904/0 (socket says 172.21.15.67:53904) 2024-03-09T09:20:09.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.928+0000 7f725fe00640 1 -- 172.21.15.67:0/2720028010 learned_addr learned my addr 172.21.15.67:0/2720028010 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:20:09.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.928+0000 7f7267e00640 1 --2- 172.21.15.67:0/2720028010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 0x7f72680c0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:09.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.929+0000 7f7267e00640 1 -- 172.21.15.67:0/2720028010 >> v1:172.21.15.134:6789/0 conn(0x7f72680c1520 legacy=0x7f7268160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:09.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.929+0000 7f7267e00640 1 -- 172.21.15.67:0/2720028010 >> v1:172.21.15.67:6789/0 conn(0x7f726806f450 legacy=0x7f72680c2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:09.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.929+0000 7f7267e00640 1 -- 172.21.15.67:0/2720028010 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72680c2590 con 0x7f72680c0c00 2024-03-09T09:20:09.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.930+0000 7f7267e00640 1 --2- 172.21.15.67:0/2720028010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 0x7f72680c0fe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f72580097d0 tx=0x7f725802b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9804dbe86e26428 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:09.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.933+0000 7f7267400640 1 -- 172.21.15.67:0/2720028010 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258031770 con 0x7f72680c0c00 2024-03-09T09:20:09.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.933+0000 7f7267400640 1 -- 172.21.15.67:0/2720028010 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7258037070 con 0x7f72680c0c00 2024-03-09T09:20:09.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.933+0000 7f7267400640 1 -- 172.21.15.67:0/2720028010 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258031b30 con 0x7f72680c0c00 2024-03-09T09:20:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.933+0000 7f726ec00640 1 -- 172.21.15.67:0/2720028010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 msgr2=0x7f72680c0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.933+0000 7f726ec00640 1 --2- 172.21.15.67:0/2720028010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 0x7f72680c0fe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f72580097d0 tx=0x7f725802b810 comp rx=0 tx=0).stop 2024-03-09T09:20:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.933+0000 7f726ec00640 1 -- 172.21.15.67:0/2720028010 shutdown_connections 2024-03-09T09:20:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.933+0000 7f726ec00640 1 --2- 172.21.15.67:0/2720028010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 0x7f72680c0fe0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.933+0000 7f726ec00640 1 -- 172.21.15.67:0/2720028010 >> 172.21.15.67:0/2720028010 conn(0x7f72680a9580 msgr2=0x7f72680a9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.934+0000 7f726ec00640 1 -- 172.21.15.67:0/2720028010 shutdown_connections 2024-03-09T09:20:09.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.934+0000 7f726ec00640 1 -- 172.21.15.67:0/2720028010 wait complete. 2024-03-09T09:20:09.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.935+0000 7f726ec00640 1 Processor -- start 2024-03-09T09:20:09.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.935+0000 7f726ec00640 1 -- start start 2024-03-09T09:20:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.936+0000 7f726ec00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 0x7f72681ed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.936+0000 7f726ec00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72680c1520 0x7f7268150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.936+0000 7f726ec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72681e2e70 0x7f72681503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.936+0000 7f726ec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7268155ed0 con 0x7f72681e2e70 2024-03-09T09:20:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.936+0000 7f726ec00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72680c2870 con 0x7f72680c1520 2024-03-09T09:20:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.936+0000 7f726ec00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726806ea50 con 0x7f72680c0c00 2024-03-09T09:20:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.936+0000 7f726cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72681e2e70 0x7f72681503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.936+0000 7f726cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72681e2e70 0x7f72681503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58604/0 (socket says 172.21.15.67:58604) 2024-03-09T09:20:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.936+0000 7f726cc00640 1 -- 172.21.15.67:0/3293254981 learned_addr learned my addr 172.21.15.67:0/3293254981 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.937+0000 7f726cc00640 1 -- 172.21.15.67:0/3293254981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72681e2e70 msgr2=0x7f72681503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:20:09.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.937+0000 7f726cc00640 1 -- 172.21.15.67:0/3293254981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72681e2e70 msgr2=0x7f72681503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:20:09.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.937+0000 7f726cc00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72681e2e70 0x7f72681503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:09.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.937+0000 7f726cc00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72681e2e70 0x7f72681503a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:09.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.937+0000 7f725fe00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72680c1520 0x7f7268150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:09.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.937+0000 7f7267e00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 0x7f72681ed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.938+0000 7f725fe00640 1 -- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 msgr2=0x7f72681ed1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.938+0000 7f725fe00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 0x7f72681ed1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.938+0000 7f725fe00640 1 -- 172.21.15.67:0/3293254981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72681e2e70 msgr2=0x7f72681503a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:20:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.938+0000 7f725fe00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72681e2e70 0x7f72681503a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.938+0000 7f725fe00640 1 -- 172.21.15.67:0/3293254981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7268151460 con 0x7f72680c1520 2024-03-09T09:20:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.938+0000 7f7267e00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 0x7f72681ed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:20:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.938+0000 7f725fe00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72680c1520 0x7f7268150d30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f725000c470 tx=0x7f7250001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.938+0000 7f7264c00640 1 -- 172.21.15.67:0/3293254981 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250017070 con 0x7f72680c1520 2024-03-09T09:20:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.938+0000 7f7264c00640 1 -- 172.21.15.67:0/3293254981 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7250005ce0 con 0x7f72680c1520 2024-03-09T09:20:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.938+0000 7f726ec00640 1 -- 172.21.15.67:0/3293254981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72681ec7e0 con 0x7f72680c1520 2024-03-09T09:20:09.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.939+0000 7f7264c00640 1 -- 172.21.15.67:0/3293254981 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72500072d0 con 0x7f72680c1520 2024-03-09T09:20:09.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.939+0000 7f726ec00640 1 -- 172.21.15.67:0/3293254981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726814fba0 con 0x7f72680c1520 2024-03-09T09:20:09.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.940+0000 7f726ec00640 1 -- 172.21.15.67:0/3293254981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72680c20d0 con 0x7f72680c1520 2024-03-09T09:20:09.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.941+0000 7f7264c00640 1 -- 172.21.15.67:0/3293254981 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f725001a070 con 0x7f72680c1520 2024-03-09T09:20:09.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.942+0000 7f7264c00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f724406b500 0x7f724406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:09.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.942+0000 7f7264c00640 1 -- 172.21.15.67:0/3293254981 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f7250089940 con 0x7f72680c1520 2024-03-09T09:20:09.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.943+0000 7f7267e00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f724406b500 0x7f724406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:09.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.944+0000 7f7267e00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f724406b500 0x7f724406d9c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7258036000 tx=0x7f725803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:09.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:09.956+0000 7f7264c00640 1 -- 172.21.15.67:0/3293254981 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f72680c20d0 con 0x7f72680c1520 2024-03-09T09:20:10.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.287+0000 7f726ec00640 1 -- 172.21.15.67:0/3293254981 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7268075a00 con 0x7f72680c1520 2024-03-09T09:20:10.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.288+0000 7f7264c00640 1 -- 172.21.15.67:0/3293254981 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f7250057cf0 con 0x7f72680c1520 2024-03-09T09:20:10.294 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":202,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:20:10.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.290+0000 7f726ec00640 1 -- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f724406b500 msgr2=0x7f724406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:10.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.290+0000 7f726ec00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f724406b500 0x7f724406d9c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7258036000 tx=0x7f725803d040 comp rx=0 tx=0).stop 2024-03-09T09:20:10.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.290+0000 7f726ec00640 1 -- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72680c1520 msgr2=0x7f7268150d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:10.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.290+0000 7f726ec00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72680c1520 0x7f7268150d30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f725000c470 tx=0x7f7250001e70 comp rx=0 tx=0).stop 2024-03-09T09:20:10.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.290+0000 7f726ec00640 1 -- 172.21.15.67:0/3293254981 shutdown_connections 2024-03-09T09:20:10.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.290+0000 7f726ec00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f724406b500 0x7f724406d9c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:10.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.291+0000 7f726ec00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f72681e2e70 0x7f72681503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:10.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.291+0000 7f726ec00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72680c1520 0x7f7268150d30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:10.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.291+0000 7f726ec00640 1 --2- 172.21.15.67:0/3293254981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72680c0c00 0x7f72681ed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:10.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.291+0000 7f726ec00640 1 -- 172.21.15.67:0/3293254981 >> 172.21.15.67:0/3293254981 conn(0x7f72680a9580 msgr2=0x7f7268156050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:10.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.291+0000 7f726ec00640 1 -- 172.21.15.67:0/3293254981 shutdown_connections 2024-03-09T09:20:10.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:10.291+0000 7f726ec00640 1 -- 172.21.15.67:0/3293254981 wait complete. 2024-03-09T09:20:13.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:20:13.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.536+0000 7fa71b200640 1 Processor -- start 2024-03-09T09:20:13.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.536+0000 7fa71b200640 1 -- start start 2024-03-09T09:20:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.538+0000 7fa71b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa71c072770 0x7fa71c072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.538+0000 7fa71b200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa71c06bda0 0x7fa71c156650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.538+0000 7fa71b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71c06cd30 con 0x7fa71c0bebc0 2024-03-09T09:20:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.538+0000 7fa71b200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71c06b200 con 0x7fa71c06bda0 2024-03-09T09:20:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.538+0000 7fa71b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c0c5eb0 con 0x7fa71c072770 2024-03-09T09:20:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.538+0000 7fa711400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa71c0bebc0 0x7fa71c06b690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:53910/0 (socket says 172.21.15.67:53910) 2024-03-09T09:20:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.538+0000 7fa711400640 1 -- 172.21.15.67:0/2646213634 learned_addr learned my addr 172.21.15.67:0/2646213634 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:20:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.538+0000 7fa719e00640 1 --2- 172.21.15.67:0/2646213634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa71c072770 0x7fa71c072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.539+0000 7fa71a800640 1 --2- 172.21.15.67:0/2646213634 >> v2:172.21.15.134:3300/0 conn(0x7fa71c06bda0 0x7fa71c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.539+0000 7fa719e00640 1 -- 172.21.15.67:0/2646213634 >> v2:172.21.15.134:3300/0 conn(0x7fa71c06bda0 msgr2=0x7fa71c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.539+0000 7fa719e00640 1 --2- 172.21.15.67:0/2646213634 >> v2:172.21.15.134:3300/0 conn(0x7fa71c06bda0 0x7fa71c156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.539+0000 7fa719e00640 1 -- 172.21.15.67:0/2646213634 >> v1:172.21.15.67:6789/0 conn(0x7fa71c0bebc0 legacy=0x7fa71c06b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.539+0000 7fa719e00640 1 -- 172.21.15.67:0/2646213634 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c0c6150 con 0x7fa71c072770 2024-03-09T09:20:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.541+0000 7fa719e00640 1 --2- 172.21.15.67:0/2646213634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa71c072770 0x7fa71c072b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0097a0 tx=0x7fa70c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb00e8922565864 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:13.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.541+0000 7fa719400640 1 -- 172.21.15.67:0/2646213634 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c039070 con 0x7fa71c072770 2024-03-09T09:20:13.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.541+0000 7fa719400640 1 -- 172.21.15.67:0/2646213634 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c0316f0 con 0x7fa71c072770 2024-03-09T09:20:13.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.541+0000 7fa719400640 1 -- 172.21.15.67:0/2646213634 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c031ac0 con 0x7fa71c072770 2024-03-09T09:20:13.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.542+0000 7fa71b200640 1 -- 172.21.15.67:0/2646213634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa71c072770 msgr2=0x7fa71c072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:13.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.542+0000 7fa71b200640 1 --2- 172.21.15.67:0/2646213634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa71c072770 0x7fa71c072b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0097a0 tx=0x7fa70c02b790 comp rx=0 tx=0).stop 2024-03-09T09:20:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.542+0000 7fa71b200640 1 -- 172.21.15.67:0/2646213634 shutdown_connections 2024-03-09T09:20:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.542+0000 7fa71b200640 1 --2- 172.21.15.67:0/2646213634 >> v2:172.21.15.134:3300/0 conn(0x7fa71c06bda0 0x7fa71c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.542+0000 7fa71b200640 1 --2- 172.21.15.67:0/2646213634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa71c072770 0x7fa71c072b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.542+0000 7fa71b200640 1 -- 172.21.15.67:0/2646213634 >> 172.21.15.67:0/2646213634 conn(0x7fa71c0c7fb0 msgr2=0x7fa71c0c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.542+0000 7fa71b200640 1 -- 172.21.15.67:0/2646213634 shutdown_connections 2024-03-09T09:20:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.542+0000 7fa71b200640 1 -- 172.21.15.67:0/2646213634 wait complete. 2024-03-09T09:20:13.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.545+0000 7fa71b200640 1 Processor -- start 2024-03-09T09:20:13.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.545+0000 7fa71b200640 1 -- start start 2024-03-09T09:20:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa71b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c06bda0 0x7fa71c0c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa71b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa71c072770 0x7fa71c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa71b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa71c0bebc0 0x7fa71c14c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa71b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c0c9430 con 0x7fa71c072770 2024-03-09T09:20:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa71b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c0c6710 con 0x7fa71c06bda0 2024-03-09T09:20:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa71b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c159020 con 0x7fa71c0bebc0 2024-03-09T09:20:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa711400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa71c072770 0x7fa71c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa711400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa71c072770 0x7fa71c1579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58620/0 (socket says 172.21.15.67:58620) 2024-03-09T09:20:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa711400640 1 -- 172.21.15.67:0/2536994022 learned_addr learned my addr 172.21.15.67:0/2536994022 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa711400640 1 -- 172.21.15.67:0/2536994022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa71c072770 msgr2=0x7fa71c1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:20:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa711400640 1 -- 172.21.15.67:0/2536994022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa71c072770 msgr2=0x7fa71c1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:20:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa711400640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa71c072770 0x7fa71c1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa719e00640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c06bda0 0x7fa71c0c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa711400640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa71c072770 0x7fa71c1579a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.547+0000 7fa71a800640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa71c0bebc0 0x7fa71c14c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.548+0000 7fa719e00640 1 -- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa71c0bebc0 msgr2=0x7fa71c14c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.548+0000 7fa719e00640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa71c0bebc0 0x7fa71c14c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.548+0000 7fa719e00640 1 -- 172.21.15.67:0/2536994022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa71c072770 msgr2=0x7fa71c1579a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:20:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.548+0000 7fa719e00640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa71c072770 0x7fa71c1579a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.548+0000 7fa719e00640 1 -- 172.21.15.67:0/2536994022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c1efca0 con 0x7fa71c06bda0 2024-03-09T09:20:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.549+0000 7fa719e00640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c06bda0 0x7fa71c0c1fc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa70c033000 tx=0x7fa70c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.549+0000 7fa712a00640 1 -- 172.21.15.67:0/2536994022 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c044070 con 0x7fa71c06bda0 2024-03-09T09:20:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.549+0000 7fa712a00640 1 -- 172.21.15.67:0/2536994022 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c0346c0 con 0x7fa71c06bda0 2024-03-09T09:20:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.549+0000 7fa71b200640 1 -- 172.21.15.67:0/2536994022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71c0c2780 con 0x7fa71c06bda0 2024-03-09T09:20:13.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.549+0000 7fa712a00640 1 -- 172.21.15.67:0/2536994022 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c039040 con 0x7fa71c06bda0 2024-03-09T09:20:13.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.550+0000 7fa71b200640 1 -- 172.21.15.67:0/2536994022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71c150700 con 0x7fa71c06bda0 2024-03-09T09:20:13.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.551+0000 7fa71b200640 1 -- 172.21.15.67:0/2536994022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa71c1efec0 con 0x7fa71c06bda0 2024-03-09T09:20:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.553+0000 7fa712a00640 1 -- 172.21.15.67:0/2536994022 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa70c047070 con 0x7fa71c06bda0 2024-03-09T09:20:13.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.554+0000 7fa712a00640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa6f806b500 0x7fa6f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:13.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.555+0000 7fa712a00640 1 -- 172.21.15.67:0/2536994022 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa70c0acb40 con 0x7fa71c06bda0 2024-03-09T09:20:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.555+0000 7fa711400640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa6f806b500 0x7fa6f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:13.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.556+0000 7fa711400640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa6f806b500 0x7fa6f806d9c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa71c158b90 tx=0x7fa704009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:13.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.575+0000 7fa712a00640 1 -- 172.21.15.67:0/2536994022 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa71c1efec0 con 0x7fa71c06bda0 2024-03-09T09:20:13.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.892+0000 7fa71b200640 1 -- 172.21.15.67:0/2536994022 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa71c0adf30 con 0x7fa71c06bda0 2024-03-09T09:20:13.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.893+0000 7fa712a00640 1 -- 172.21.15.67:0/2536994022 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fa71c0adf30 con 0x7fa71c06bda0 2024-03-09T09:20:13.899 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":202,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:20:13.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 -- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa6f806b500 msgr2=0x7fa6f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa6f806b500 0x7fa6f806d9c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa71c158b90 tx=0x7fa704009040 comp rx=0 tx=0).stop 2024-03-09T09:20:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 -- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c06bda0 msgr2=0x7fa71c0c1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c06bda0 0x7fa71c0c1fc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa70c033000 tx=0x7fa70c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:20:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 -- 172.21.15.67:0/2536994022 shutdown_connections 2024-03-09T09:20:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa6f806b500 0x7fa6f806d9c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa71c0bebc0 0x7fa71c14c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa71c072770 0x7fa71c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 --2- 172.21.15.67:0/2536994022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa71c06bda0 0x7fa71c0c1fc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 -- 172.21.15.67:0/2536994022 >> 172.21.15.67:0/2536994022 conn(0x7fa71c0c7fb0 msgr2=0x7fa71c155430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 -- 172.21.15.67:0/2536994022 shutdown_connections 2024-03-09T09:20:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:13.896+0000 7fa71b200640 1 -- 172.21.15.67:0/2536994022 wait complete. 2024-03-09T09:20:14.193 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:20:14.193 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:14.193 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 46 -eq 48 ']' 2024-03-09T09:20:14.194 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.46 2024-03-09T09:20:14.194 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.46 2024-03-09T09:20:14.194 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.46 2024-03-09T09:20:14.194 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.46 2024-03-09T09:20:14.197 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:20:14.626 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:20:14.626 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:20:14.626 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 41 -eq 48 ']' 2024-03-09T09:20:14.626 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.41 2024-03-09T09:20:14.626 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.41 2024-03-09T09:20:14.626 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.41 2024-03-09T09:20:14.627 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.41 2024-03-09T09:20:14.630 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:20:16.925 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:20:17.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.164+0000 7f275e800640 1 Processor -- start 2024-03-09T09:20:17.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.165+0000 7f275e800640 1 -- start start 2024-03-09T09:20:17.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.166+0000 7f275e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:17.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.166+0000 7f275e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:17.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.166+0000 7f275e800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27580c52c0 con 0x7f2758073b90 2024-03-09T09:20:17.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.166+0000 7f275e800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580c3520 con 0x7f27580c7a30 2024-03-09T09:20:17.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.166+0000 7f275e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27580c2f10 con 0x7f27580c7e10 2024-03-09T09:20:17.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.166+0000 7f2757400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.166+0000 7f2757400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58626/0 (socket says 172.21.15.67:58626) 2024-03-09T09:20:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.166+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 learned_addr learned my addr 172.21.15.67:0/3404026857 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.166+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f275806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.166+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f275806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.166+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.167+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.167+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.167+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 msgr2=0x7f275806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.167+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 msgr2=0x7f275806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.167+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:17.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.167+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.368+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.368+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:17.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.369+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f275806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.369+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f275806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.369+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.369+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 msgr2=0x7f275806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.369+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 msgr2=0x7f275806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.369+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.369+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:20:17.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.369+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:20:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.770+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.770+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.771+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 msgr2=0x7f275806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:17.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.771+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 msgr2=0x7f275806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:17.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.771+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:17.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.771+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f275806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:17.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.771+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f275806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:17.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.771+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:20:17.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.771+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:17.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:17.771+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:20:18.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:18.572+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:18.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:18.573+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:18.573+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f275806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:18.573+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f275806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:18.573+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:18.573+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 msgr2=0x7f275806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:18.573+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 msgr2=0x7f275806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:18.573+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:18.573+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:20:18.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:18.573+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:20:19.802 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:20:19.802 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:19.802 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 47 -eq 48 ']' 2024-03-09T09:20:19.802 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.47 2024-03-09T09:20:19.803 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.47 2024-03-09T09:20:19.803 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.47 2024-03-09T09:20:19.803 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.47 2024-03-09T09:20:19.807 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:20:20.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.166+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 msgr2=0x7f275806e610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:20.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.166+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27580c7e10 0x7f275806e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:20.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.166+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f275806e0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:20.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.166+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f275806e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:20.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.166+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> v1:172.21.15.134:6789/0 conn(0x7f2758073b90 legacy=0x7f27580c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:20.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.166+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:20.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.166+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275806eb50 con 0x7f27380036a0 2024-03-09T09:20:20.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.166+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580c3520 con 0x7f2738007330 2024-03-09T09:20:20.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.167+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:20.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.168+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:20.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.168+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:20.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.168+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:20.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.168+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:20.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.369+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:20.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.370+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:20.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.370+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:20.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.370+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:20.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.370+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:20:20.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.772+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:20.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.772+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:20.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.772+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:20.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.772+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:20.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:20.772+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:20:20.933 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:20:20.933 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:20:20.933 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 42 -eq 48 ']' 2024-03-09T09:20:20.933 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.42 2024-03-09T09:20:20.933 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.42 2024-03-09T09:20:20.934 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.42 2024-03-09T09:20:20.934 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.42 2024-03-09T09:20:20.937 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:20:21.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:21.574+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:21.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:21.575+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:21.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:21.575+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:21.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:21.575+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:21.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:21.575+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:20:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.167+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.167+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.167+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> v1:172.21.15.67:6789/0 conn(0x7f27380036a0 legacy=0x7f2738005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.167+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.167+0000 7f2757e00640 1 -- 172.21.15.67:0/3404026857 reap_dead start 2024-03-09T09:20:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.167+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.167+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273800afa0 con 0x7f273800a540 2024-03-09T09:20:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.167+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580c3520 con 0x7f27580c7a30 2024-03-09T09:20:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.167+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580c2f10 con 0x7f2738007330 2024-03-09T09:20:23.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.168+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:23.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.168+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:23.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.168+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:23.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.168+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:23.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.168+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:23.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.168+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:23.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.168+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:23.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.169+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:23.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.169+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:23.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.169+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.369+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.370+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.370+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.370+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.370+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:20:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.370+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.371+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.371+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:23.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.371+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:23.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.371+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:20:23.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.771+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:23.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.772+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:23.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.772+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:23.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.772+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:23.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.772+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:20:23.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.772+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:23.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.773+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:23.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.773+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:23.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.773+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:23.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:23.773+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:20:23.839 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:20:23.840 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:23.840 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 48 -eq 48 ']' 2024-03-09T09:20:23.840 INFO:tasks.workunit.client.0.smithi067.stderr:+ continue 2024-03-09T09:20:23.840 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:23.840 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.49 2024-03-09T09:20:23.841 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 49 -eq 48 ']' 2024-03-09T09:20:23.841 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.49 2024-03-09T09:20:23.841 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.49 2024-03-09T09:20:23.841 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.49 2024-03-09T09:20:23.844 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:20:24.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:24.573+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:24.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:24.574+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:24.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:24.574+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:24.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:24.574+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:24.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:24.574+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:20:24.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:24.575+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:24.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:24.575+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:24.575+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:24.575+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:24.575+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:20:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.167+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:26.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.167+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738007330 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:26.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.167+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:26.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.167+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f27580c7a30 0x7f2738009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:26.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.167+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> v1:172.21.15.67:6789/0 conn(0x7f273800a540 legacy=0x7f2738005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:26.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.168+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:26.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.168+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273800ab10 con 0x7f273800d2e0 2024-03-09T09:20:26.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.168+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27580c3520 con 0x7f2738010e90 2024-03-09T09:20:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.169+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.169+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 msgr2=0x7f27380132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.169+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 msgr2=0x7f27380132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.169+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.169+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:26.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.370+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:26.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.371+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 msgr2=0x7f27380132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:26.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.371+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 msgr2=0x7f27380132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:26.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.371+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:26.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.371+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:20:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.772+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.773+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 msgr2=0x7f27380132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.773+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 msgr2=0x7f27380132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.773+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:26.773+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:20:26.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:20:26.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:26.912 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 50 -eq 48 ']' 2024-03-09T09:20:26.912 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.50 2024-03-09T09:20:26.912 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.50 2024-03-09T09:20:26.912 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.50 2024-03-09T09:20:26.912 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.50 2024-03-09T09:20:26.916 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:20:27.469 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:20:27.469 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:20:27.470 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 43 -eq 48 ']' 2024-03-09T09:20:27.470 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.43 2024-03-09T09:20:27.470 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.43 2024-03-09T09:20:27.470 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.43 2024-03-09T09:20:27.470 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.43 2024-03-09T09:20:27.476 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:20:27.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:27.575+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:27.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:27.575+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 msgr2=0x7f27380132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:27.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:27.575+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 msgr2=0x7f27380132b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:27.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:27.575+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:27.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:27.575+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:20:28.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:28.349+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v1:172.21.15.134:6789/0 conn(0x7f273800d2e0 legacy=0x7f273800f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-09T09:20:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:28.349+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v1:172.21.15.134:6789/0 conn(0x7f273800d2e0 legacy=0x7f273800f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:20:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:28.349+0000 7f2757400640 1 --1- 172.21.15.67:0/3404026857 >> v1:172.21.15.134:6789/0 conn(0x7f273800d2e0 0x7f273800f6d0 :-1 s=OPENED pgs=119 cs=1 l=1).handle_message read tag failed 2024-03-09T09:20:28.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:28.349+0000 7f2757400640 1 --1- 172.21.15.67:0/3404026857 >> v1:172.21.15.134:6789/0 conn(0x7f273800d2e0 0x7f273800f6d0 :-1 s=OPENED pgs=119 cs=1 l=1).fault on lossy channel, failing 2024-03-09T09:20:29.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.168+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 msgr2=0x7f27380132b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:29.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.168+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2738010e90 0x7f27380132b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.168+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> v1:172.21.15.134:6789/0 conn(0x7f273800d2e0 legacy=0x7f273800f6d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.168+0000 7f2757e00640 1 -- 172.21.15.67:0/3404026857 reap_dead start 2024-03-09T09:20:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.168+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.168+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.168+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2738014a60 con 0x7f2738013f50 2024-03-09T09:20:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.168+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580c3520 con 0x7f2738007330 2024-03-09T09:20:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.168+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580c2f10 con 0x7f2738013a10 2024-03-09T09:20:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.169+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:29.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.169+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:29.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.169+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:29.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.169+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:29.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.169+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:29.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.170+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.170+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.170+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.170+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:29.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.170+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.371+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.371+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.372+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.372+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.372+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:29.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.372+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:20:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.372+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.372+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.372+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:29.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.372+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:20:29.448 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:20:29.457 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:20:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.773+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.774+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.774+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.774+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.774+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.774+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.774+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.774+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:20:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.774+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:29.774+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:20:30.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:30.576+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:30.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:30.576+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:30.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:30.576+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:20:30.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:30.576+0000 7f2757400640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:30.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:30.576+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:30.577+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:20:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:30.577+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:30.577+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:30.577+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:30.577+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:20:32.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.168+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 msgr2=0x7f27380097f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:32.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.168+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.134:3300/0 conn(0x7f2738013a10 0x7f27380097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.168+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 msgr2=0x7f2738009fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:32.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.168+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> v2:172.21.15.67:3300/0 conn(0x7f2738007330 0x7f2738009fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.168+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 >> v1:172.21.15.67:6789/0 conn(0x7f2738013f50 legacy=0x7f273800fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:32.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.169+0000 7f2754c00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273800f6e0 0x7f2738005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:32.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.169+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2738013390 con 0x7f27380143e0 2024-03-09T09:20:32.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.169+0000 7f2754c00640 1 -- 172.21.15.67:0/3404026857 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27580c3520 con 0x7f273800f6e0 2024-03-09T09:20:32.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.170+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273800f6e0 0x7f2738005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:32.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.170+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273800f6e0 msgr2=0x7f2738005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:20:32.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.170+0000 7f2756a00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273800f6e0 msgr2=0x7f2738005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:20:32.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.170+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273800f6e0 0x7f2738005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:32.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.170+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273800f6e0 0x7f2738005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:32.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.192+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672518164 0 0) 0x7f2738013390 con 0x7f27380143e0 2024-03-09T09:20:32.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.192+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f273c003520 con 0x7f27380143e0 2024-03-09T09:20:32.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2637334446 0 0) 0x7f273c003520 con 0x7f27380143e0 2024-03-09T09:20:32.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273800f6e0 msgr2=0x7f2738005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:32.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273800f6e0 0x7f2738005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2738007070 con 0x7f27380143e0 2024-03-09T09:20:32.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f274800f040 con 0x7f27380143e0 2024-03-09T09:20:32.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 >> v1:172.21.15.134:6789/0 conn(0x7f27380143e0 legacy=0x7f2738010590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:32.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2757e00640 1 -- 172.21.15.67:0/3404026857 reap_dead start 2024-03-09T09:20:32.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c0039b0 0x7f273c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:32.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273c004800 0x7f273c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:32.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c005210 0x7f273c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:32.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580c3520 con 0x7f273c005210 2024-03-09T09:20:32.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c00c140 con 0x7f273c0039b0 2024-03-09T09:20:32.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.194+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c00c340 con 0x7f273c004800 2024-03-09T09:20:32.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.195+0000 7f2757e00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c005210 0x7f273c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:32.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.195+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c0039b0 0x7f273c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:32.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.195+0000 7f2756a00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273c004800 0x7f273c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:32.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.196+0000 7f2757e00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273c004800 msgr2=0x7f273c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:32.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.196+0000 7f2757e00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273c004800 0x7f273c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.196+0000 7f2757e00640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c0039b0 msgr2=0x7f273c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:32.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.196+0000 7f2757e00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c0039b0 0x7f273c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.196+0000 7f2757e00640 1 -- 172.21.15.67:0/3404026857 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c003520 con 0x7f273c005210 2024-03-09T09:20:32.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.196+0000 7f2757400640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c0039b0 0x7f273c003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:20:32.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.199+0000 7f2757e00640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c005210 0x7f273c00b9c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f27580ad210 tx=0x7f273c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d936abcbc48e4e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:32.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.200+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750001810 con 0x7f273c005210 2024-03-09T09:20:32.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.200+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2750005e00 con 0x7f273c005210 2024-03-09T09:20:32.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.200+0000 7f2756000640 1 -- 172.21.15.67:0/3404026857 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27500072b0 con 0x7f273c005210 2024-03-09T09:20:32.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.201+0000 7f275e800640 1 -- 172.21.15.67:0/3404026857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c005210 msgr2=0x7f273c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:32.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.201+0000 7f275e800640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c005210 0x7f273c00b9c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f27580ad210 tx=0x7f273c005830 comp rx=0 tx=0).stop 2024-03-09T09:20:32.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.202+0000 7f275e800640 1 -- 172.21.15.67:0/3404026857 shutdown_connections 2024-03-09T09:20:32.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.202+0000 7f275e800640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c005210 0x7f273c00b9c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.202+0000 7f275e800640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273c004800 0x7f273c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.202+0000 7f275e800640 1 --2- 172.21.15.67:0/3404026857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c0039b0 0x7f273c003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.202+0000 7f275e800640 1 -- 172.21.15.67:0/3404026857 >> 172.21.15.67:0/3404026857 conn(0x7f275806b040 msgr2=0x7f275806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:32.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.202+0000 7f275e800640 1 -- 172.21.15.67:0/3404026857 shutdown_connections 2024-03-09T09:20:32.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.202+0000 7f275e800640 1 -- 172.21.15.67:0/3404026857 wait complete. 2024-03-09T09:20:32.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.207+0000 7f275e800640 1 Processor -- start 2024-03-09T09:20:32.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.207+0000 7f275e800640 1 -- start start 2024-03-09T09:20:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.209+0000 7f275e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c0039b0 0x7f27581dc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.209+0000 7f275e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273c004800 0x7f27580ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:32.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.209+0000 7f275e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c005210 0x7f27581d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:32.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.209+0000 7f275e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580c5b80 con 0x7f273c0039b0 2024-03-09T09:20:32.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.209+0000 7f275e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580c39b0 con 0x7f273c005210 2024-03-09T09:20:32.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.209+0000 7f275e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27580c5d00 con 0x7f273c004800 2024-03-09T09:20:32.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.209+0000 7f2757400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c0039b0 0x7f27581dc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:32.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.209+0000 7f2757400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c0039b0 0x7f27581dc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59908/0 (socket says 172.21.15.67:59908) 2024-03-09T09:20:32.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.209+0000 7f2757400640 1 -- 172.21.15.67:0/3720242173 learned_addr learned my addr 172.21.15.67:0/3720242173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:32.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.210+0000 7f2756a00640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273c004800 0x7f27580ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:32.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.210+0000 7f2757e00640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c005210 0x7f27581d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:32.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.210+0000 7f2757400640 1 -- 172.21.15.67:0/3720242173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c0039b0 msgr2=0x7f27581dc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:20:32.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.210+0000 7f2757400640 1 -- 172.21.15.67:0/3720242173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c0039b0 msgr2=0x7f27581dc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:20:32.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.210+0000 7f2757400640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c0039b0 0x7f27581dc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:20:32.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.210+0000 7f2757400640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c0039b0 0x7f27581dc660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:32.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.210+0000 7f2757e00640 1 -- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273c004800 msgr2=0x7f27580ab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:32.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.210+0000 7f2757e00640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273c004800 0x7f27580ab3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.211+0000 7f2757e00640 1 -- 172.21.15.67:0/3720242173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c0039b0 msgr2=0x7f27581dc660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:20:32.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.211+0000 7f2757e00640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c0039b0 0x7f27581dc660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.211+0000 7f2757e00640 1 -- 172.21.15.67:0/3720242173 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27581ee790 con 0x7f273c005210 2024-03-09T09:20:32.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.211+0000 7f2757e00640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c005210 0x7f27581d93c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2750007fd0 tx=0x7f275002d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:32.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.212+0000 7f274f400640 1 -- 172.21.15.67:0/3720242173 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750007610 con 0x7f273c005210 2024-03-09T09:20:32.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.212+0000 7f274f400640 1 -- 172.21.15.67:0/3720242173 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27500077b0 con 0x7f273c005210 2024-03-09T09:20:32.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.212+0000 7f274f400640 1 -- 172.21.15.67:0/3720242173 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750003970 con 0x7f273c005210 2024-03-09T09:20:32.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.212+0000 7f275e800640 1 -- 172.21.15.67:0/3720242173 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27581eea70 con 0x7f273c005210 2024-03-09T09:20:32.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.212+0000 7f275e800640 1 -- 172.21.15.67:0/3720242173 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27581ef030 con 0x7f273c005210 2024-03-09T09:20:32.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.213+0000 7f275e800640 1 -- 172.21.15.67:0/3720242173 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2758075a00 con 0x7f273c005210 2024-03-09T09:20:32.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.215+0000 7f274f400640 1 -- 172.21.15.67:0/3720242173 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2750046060 con 0x7f273c005210 2024-03-09T09:20:32.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.216+0000 7f274f400640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f273806d3f0 0x7f273806f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:32.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.216+0000 7f274f400640 1 -- 172.21.15.67:0/3720242173 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f27500aaae0 con 0x7f273c005210 2024-03-09T09:20:32.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.217+0000 7f2757400640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f273806d3f0 0x7f273806f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:32.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.218+0000 7f2757400640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f273806d3f0 0x7f273806f8b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2748006a60 tx=0x7f2748008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:32.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.231+0000 7f274f400640 1 -- 172.21.15.67:0/3720242173 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f27500794c0 con 0x7f273c005210 2024-03-09T09:20:32.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.516+0000 7f275e800640 1 -- 172.21.15.67:0/3720242173 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2758075a00 con 0x7f273c005210 2024-03-09T09:20:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.517+0000 7f274f400640 1 -- 172.21.15.67:0/3720242173 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f2750078dc0 con 0x7f273c005210 2024-03-09T09:20:32.524 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":214,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:20:32.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 -- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f273806d3f0 msgr2=0x7f273806f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:32.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f273806d3f0 0x7f273806f8b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2748006a60 tx=0x7f2748008040 comp rx=0 tx=0).stop 2024-03-09T09:20:32.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 -- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c005210 msgr2=0x7f27581d93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:32.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c005210 0x7f27581d93c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2750007fd0 tx=0x7f275002d000 comp rx=0 tx=0).stop 2024-03-09T09:20:32.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 -- 172.21.15.67:0/3720242173 shutdown_connections 2024-03-09T09:20:32.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f273806d3f0 0x7f273806f8b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f273c005210 0x7f27581d93c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f273c004800 0x7f27580ab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 --2- 172.21.15.67:0/3720242173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f273c0039b0 0x7f27581dc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:32.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 -- 172.21.15.67:0/3720242173 >> 172.21.15.67:0/3720242173 conn(0x7f275806b040 msgr2=0x7f27580728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:32.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 -- 172.21.15.67:0/3720242173 shutdown_connections 2024-03-09T09:20:32.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:32.520+0000 7f275e800640 1 -- 172.21.15.67:0/3720242173 wait complete. 2024-03-09T09:20:35.057 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:20:35.058 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:35.058 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 51 -eq 48 ']' 2024-03-09T09:20:35.058 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.51 2024-03-09T09:20:35.058 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.51 2024-03-09T09:20:35.058 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.51 2024-03-09T09:20:35.058 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.51 2024-03-09T09:20:35.061 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:20:35.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:20:35.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.759+0000 7f2d80a00640 1 Processor -- start 2024-03-09T09:20:35.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.759+0000 7f2d80a00640 1 -- start start 2024-03-09T09:20:35.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.761+0000 7f2d80a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d7c0c1d70 0x7f2d7c0ac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:35.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.761+0000 7f2d80a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2d7c0c2400 0x7f2d7c0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:35.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.761+0000 7f2d80a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7c156d50 con 0x7f2d7c153370 2024-03-09T09:20:35.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.761+0000 7f2d80a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c0aadf0 con 0x7f2d7c0c2400 2024-03-09T09:20:35.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.761+0000 7f2d80a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0aa4c0 con 0x7f2d7c0c1d70 2024-03-09T09:20:35.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.761+0000 7f2d79e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2d7c153370 0x7f2d7c0c07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:57674/0 (socket says 172.21.15.67:57674) 2024-03-09T09:20:35.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.761+0000 7f2d79e00640 1 -- 172.21.15.67:0/3419055521 learned_addr learned my addr 172.21.15.67:0/3419055521 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d73e00640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d7c0c1d70 0x7f2d7c0ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d7a800640 1 --2- 172.21.15.67:0/3419055521 >> v2:172.21.15.134:3300/0 conn(0x7f2d7c0c2400 0x7f2d7c0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147373526 0 0) 0x7f2d7c156d50 con 0x7f2d7c153370 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d60003520 con 0x7f2d7c153370 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1619605720 0 0) 0x7f2d60003520 con 0x7f2d7c153370 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d7c0c1d70 msgr2=0x7f2d7c0ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d79400640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d7c0c1d70 0x7f2d7c0ac610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 >> v2:172.21.15.134:3300/0 conn(0x7f2d7c0c2400 msgr2=0x7f2d7c0acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d79400640 1 --2- 172.21.15.67:0/3419055521 >> v2:172.21.15.134:3300/0 conn(0x7f2d7c0c2400 0x7f2d7c0acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c0aa7a0 con 0x7f2d7c153370 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f2d6c001670 con 0x7f2d7c153370 2024-03-09T09:20:35.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 >> v1:172.21.15.67:6789/0 conn(0x7f2d7c153370 legacy=0x7f2d7c0c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.762+0000 7f2d79400640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d600039b0 0x7f2d60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d79400640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d60004800 0x7f2d60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d79400640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d7c0c1d70 0x7f2d600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0aadf0 con 0x7f2d600039b0 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0aa4c0 con 0x7f2d60004800 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6000b9c0 con 0x7f2d7c0c1d70 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d79e00640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d600039b0 0x7f2d60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d73e00640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d60004800 0x7f2d60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d7a800640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d7c0c1d70 0x7f2d600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d79e00640 1 -- 172.21.15.67:0/3419055521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d7c0c1d70 msgr2=0x7f2d600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d79e00640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d7c0c1d70 0x7f2d600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d79e00640 1 -- 172.21.15.67:0/3419055521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d60004800 msgr2=0x7f2d60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:35.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d79e00640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d60004800 0x7f2d60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:35.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.763+0000 7f2d79e00640 1 -- 172.21.15.67:0/3419055521 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d60003520 con 0x7f2d600039b0 2024-03-09T09:20:35.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.765+0000 7f2d79e00640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d600039b0 0x7f2d60003dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c2c0 tx=0x7f2d60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5274a22d84ed2a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:35.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.765+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c007d50 con 0x7f2d600039b0 2024-03-09T09:20:35.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.765+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c030070 con 0x7f2d600039b0 2024-03-09T09:20:35.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.765+0000 7f2d79400640 1 -- 172.21.15.67:0/3419055521 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c02cd80 con 0x7f2d600039b0 2024-03-09T09:20:35.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.766+0000 7f2d80a00640 1 -- 172.21.15.67:0/3419055521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d600039b0 msgr2=0x7f2d60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:35.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.766+0000 7f2d80a00640 1 --2- 172.21.15.67:0/3419055521 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d600039b0 0x7f2d60003dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c2c0 tx=0x7f2d60005830 comp rx=0 tx=0).stop 2024-03-09T09:20:35.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.766+0000 7f2d7a800640 1 -- 172.21.15.67:0/3419055521 reap_dead start 2024-03-09T09:20:35.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.766+0000 7f2d80a00640 1 -- 172.21.15.67:0/3419055521 shutdown_connections 2024-03-09T09:20:35.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.766+0000 7f2d80a00640 1 -- 172.21.15.67:0/3419055521 >> 172.21.15.67:0/3419055521 conn(0x7f2d7c0c7370 msgr2=0x7f2d7c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:35.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.766+0000 7f2d80a00640 1 -- 172.21.15.67:0/3419055521 shutdown_connections 2024-03-09T09:20:35.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.766+0000 7f2d80a00640 1 -- 172.21.15.67:0/3419055521 wait complete. 2024-03-09T09:20:35.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.768+0000 7f2d80a00640 1 Processor -- start 2024-03-09T09:20:35.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.768+0000 7f2d80a00640 1 -- start start 2024-03-09T09:20:35.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.769+0000 7f2d80a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d600039b0 0x7f2d7c14a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:35.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.769+0000 7f2d80a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7c148930 0x7f2d7c1477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:35.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.769+0000 7f2d80a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c147d00 0x7f2d7c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:35.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.769+0000 7f2d80a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0c1030 con 0x7f2d7c147d00 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.769+0000 7f2d80a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0c0eb0 con 0x7f2d7c148930 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.769+0000 7f2d80a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c153750 con 0x7f2d600039b0 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.769+0000 7f2d7a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c147d00 0x7f2d7c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.769+0000 7f2d7a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c147d00 0x7f2d7c150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59914/0 (socket says 172.21.15.67:59914) 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.769+0000 7f2d7a800640 1 -- 172.21.15.67:0/603649554 learned_addr learned my addr 172.21.15.67:0/603649554 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.770+0000 7f2d79e00640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d600039b0 0x7f2d7c14a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.770+0000 7f2d73e00640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7c148930 0x7f2d7c1477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.770+0000 7f2d7a800640 1 -- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d600039b0 msgr2=0x7f2d7c14a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.770+0000 7f2d7a800640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d600039b0 0x7f2d7c14a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.770+0000 7f2d7a800640 1 -- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7c148930 msgr2=0x7f2d7c1477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.770+0000 7f2d7a800640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7c148930 0x7f2d7c1477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.770+0000 7f2d7a800640 1 -- 172.21.15.67:0/603649554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c14b210 con 0x7f2d7c147d00 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.770+0000 7f2d79e00640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d600039b0 0x7f2d7c14a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:20:35.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.770+0000 7f2d7a800640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c147d00 0x7f2d7c150a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d74007d10 tx=0x7f2d740115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:35.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.770+0000 7f2d72000640 1 -- 172.21.15.67:0/603649554 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74011de0 con 0x7f2d7c147d00 2024-03-09T09:20:35.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.771+0000 7f2d72000640 1 -- 172.21.15.67:0/603649554 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7401b070 con 0x7f2d7c147d00 2024-03-09T09:20:35.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.771+0000 7f2d72000640 1 -- 172.21.15.67:0/603649554 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74003990 con 0x7f2d7c147d00 2024-03-09T09:20:35.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.771+0000 7f2d80a00640 1 -- 172.21.15.67:0/603649554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c1ebb80 con 0x7f2d7c147d00 2024-03-09T09:20:35.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.771+0000 7f2d80a00640 1 -- 172.21.15.67:0/603649554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c1ec140 con 0x7f2d7c147d00 2024-03-09T09:20:35.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.772+0000 7f2d80a00640 1 -- 172.21.15.67:0/603649554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c075a00 con 0x7f2d7c147d00 2024-03-09T09:20:35.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.773+0000 7f2d72000640 1 -- 172.21.15.67:0/603649554 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2d74005d10 con 0x7f2d7c147d00 2024-03-09T09:20:35.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.774+0000 7f2d72000640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d5806b4d0 0x7f2d5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:35.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.774+0000 7f2d72000640 1 -- 172.21.15.67:0/603649554 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f2d7408a980 con 0x7f2d7c147d00 2024-03-09T09:20:35.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.774+0000 7f2d79e00640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d5806b4d0 0x7f2d5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:35.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.775+0000 7f2d79e00640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d5806b4d0 0x7f2d5806d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c034000 tx=0x7f2d6c007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:35.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:35.779+0000 7f2d72000640 1 -- 172.21.15.67:0/603649554 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2d7c14b210 con 0x7f2d7c147d00 2024-03-09T09:20:36.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.071+0000 7f2d80a00640 1 -- 172.21.15.67:0/603649554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2d7c0c4d40 con 0x7f2d7c147d00 2024-03-09T09:20:36.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.072+0000 7f2d72000640 1 -- 172.21.15.67:0/603649554 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f2d74014350 con 0x7f2d7c147d00 2024-03-09T09:20:36.077 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":216,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:20:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.074+0000 7f2d80a00640 1 -- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d5806b4d0 msgr2=0x7f2d5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.074+0000 7f2d80a00640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d5806b4d0 0x7f2d5806d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c034000 tx=0x7f2d6c007000 comp rx=0 tx=0).stop 2024-03-09T09:20:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.074+0000 7f2d80a00640 1 -- 172.21.15.67:0/603649554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c147d00 msgr2=0x7f2d7c150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.074+0000 7f2d80a00640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c147d00 0x7f2d7c150a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d74007d10 tx=0x7f2d740115a0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.074+0000 7f2d80a00640 1 -- 172.21.15.67:0/603649554 shutdown_connections 2024-03-09T09:20:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.074+0000 7f2d80a00640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d5806b4d0 0x7f2d5806d990 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.074+0000 7f2d80a00640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d7c147d00 0x7f2d7c150a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.074+0000 7f2d80a00640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d7c148930 0x7f2d7c1477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.074+0000 7f2d80a00640 1 --2- 172.21.15.67:0/603649554 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d600039b0 0x7f2d7c14a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.074+0000 7f2d80a00640 1 -- 172.21.15.67:0/603649554 >> 172.21.15.67:0/603649554 conn(0x7f2d7c0c7370 msgr2=0x7f2d7c155ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:36.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.075+0000 7f2d80a00640 1 -- 172.21.15.67:0/603649554 shutdown_connections 2024-03-09T09:20:36.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.075+0000 7f2d80a00640 1 -- 172.21.15.67:0/603649554 wait complete. 2024-03-09T09:20:36.099 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:20:36.099 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:20:36.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:20:36.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.294+0000 7f03eda00640 1 Processor -- start 2024-03-09T09:20:36.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.294+0000 7f03eda00640 1 -- start start 2024-03-09T09:20:36.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.295+0000 7f03eda00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f03e80c3740 0x7f03e8152c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.295+0000 7f03eda00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f03e80aaca0 0x7f03e80ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.295+0000 7f03eda00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e80c2d40 con 0x7f03e814f7a0 2024-03-09T09:20:36.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.295+0000 7f03eda00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e80aaa70 con 0x7f03e80c3740 2024-03-09T09:20:36.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.295+0000 7f03eda00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e80c1f80 con 0x7f03e80aaca0 2024-03-09T09:20:36.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.296+0000 7f03de000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f03e80c3740 0x7f03e8152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.296+0000 7f03de000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f03e80c3740 0x7f03e8152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59924/0 (socket says 172.21.15.67:59924) 2024-03-09T09:20:36.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.296+0000 7f03de000640 1 -- 172.21.15.67:0/2299456568 learned_addr learned my addr 172.21.15.67:0/2299456568 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.296+0000 7f03e7400640 1 --2- 172.21.15.67:0/2299456568 >> v2:172.21.15.134:3300/0 conn(0x7f03e80aaca0 0x7f03e80ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.296+0000 7f03de000640 1 -- 172.21.15.67:0/2299456568 >> v2:172.21.15.134:3300/0 conn(0x7f03e80aaca0 msgr2=0x7f03e80ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.296+0000 7f03de000640 1 --2- 172.21.15.67:0/2299456568 >> v2:172.21.15.134:3300/0 conn(0x7f03e80aaca0 0x7f03e80ab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.296+0000 7f03de000640 1 -- 172.21.15.67:0/2299456568 >> v1:172.21.15.134:6789/0 conn(0x7f03e814f7a0 legacy=0x7f03e806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:36.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.296+0000 7f03de000640 1 -- 172.21.15.67:0/2299456568 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80c2260 con 0x7f03e80c3740 2024-03-09T09:20:36.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.297+0000 7f03de000640 1 --2- 172.21.15.67:0/2299456568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e80c3740 0x7f03e8152c30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f03e80c3040 tx=0x7f03d002cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71ff0500737d447 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.298+0000 7f03e6000640 1 -- 172.21.15.67:0/2299456568 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d003a070 con 0x7f03e80c3740 2024-03-09T09:20:36.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.298+0000 7f03e6000640 1 -- 172.21.15.67:0/2299456568 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d002db00 con 0x7f03e80c3740 2024-03-09T09:20:36.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.298+0000 7f03e6000640 1 -- 172.21.15.67:0/2299456568 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d000b040 con 0x7f03e80c3740 2024-03-09T09:20:36.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.298+0000 7f03eda00640 1 -- 172.21.15.67:0/2299456568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e80c3740 msgr2=0x7f03e8152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.298+0000 7f03eda00640 1 --2- 172.21.15.67:0/2299456568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e80c3740 0x7f03e8152c30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f03e80c3040 tx=0x7f03d002cd50 comp rx=0 tx=0).stop 2024-03-09T09:20:36.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.298+0000 7f03eda00640 1 -- 172.21.15.67:0/2299456568 shutdown_connections 2024-03-09T09:20:36.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.298+0000 7f03eda00640 1 --2- 172.21.15.67:0/2299456568 >> v2:172.21.15.134:3300/0 conn(0x7f03e80aaca0 0x7f03e80ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.298+0000 7f03eda00640 1 --2- 172.21.15.67:0/2299456568 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e80c3740 0x7f03e8152c30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.298+0000 7f03eda00640 1 -- 172.21.15.67:0/2299456568 >> 172.21.15.67:0/2299456568 conn(0x7f03e80c7370 msgr2=0x7f03e80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.299+0000 7f03eda00640 1 -- 172.21.15.67:0/2299456568 shutdown_connections 2024-03-09T09:20:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.299+0000 7f03eda00640 1 -- 172.21.15.67:0/2299456568 wait complete. 2024-03-09T09:20:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.300+0000 7f03eda00640 1 Processor -- start 2024-03-09T09:20:36.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.300+0000 7f03eda00640 1 -- start start 2024-03-09T09:20:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.301+0000 7f03eda00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03e80aaca0 0x7f03e81e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.301+0000 7f03eda00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03e80c3740 0x7f03e8153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.301+0000 7f03eda00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e814f7a0 0x7f03e814bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.301+0000 7f03eda00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80ab630 con 0x7f03e814f7a0 2024-03-09T09:20:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.301+0000 7f03eda00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e814fb80 con 0x7f03e80aaca0 2024-03-09T09:20:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.301+0000 7f03eda00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e806e6a0 con 0x7f03e80c3740 2024-03-09T09:20:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.302+0000 7f03e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e814f7a0 0x7f03e814bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.302+0000 7f03e7400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e814f7a0 0x7f03e814bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59932/0 (socket says 172.21.15.67:59932) 2024-03-09T09:20:36.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.302+0000 7f03e7400640 1 -- 172.21.15.67:0/1391329062 learned_addr learned my addr 172.21.15.67:0/1391329062 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:36.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.302+0000 7f03e6a00640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03e80aaca0 0x7f03e81e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.302+0000 7f03de000640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03e80c3740 0x7f03e8153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.302+0000 7f03e7400640 1 -- 172.21.15.67:0/1391329062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03e80c3740 msgr2=0x7f03e8153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.302+0000 7f03e7400640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03e80c3740 0x7f03e8153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.302+0000 7f03e7400640 1 -- 172.21.15.67:0/1391329062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03e80aaca0 msgr2=0x7f03e81e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.302+0000 7f03e7400640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03e80aaca0 0x7f03e81e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.302+0000 7f03e7400640 1 -- 172.21.15.67:0/1391329062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e81e7120 con 0x7f03e814f7a0 2024-03-09T09:20:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.303+0000 7f03e7400640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e814f7a0 0x7f03e814bcd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03e000d5d0 tx=0x7f03e000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.303+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00125f0 con 0x7f03e814f7a0 2024-03-09T09:20:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.303+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e0019070 con 0x7f03e814f7a0 2024-03-09T09:20:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.303+0000 7f03eda00640 1 -- 172.21.15.67:0/1391329062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e8152c60 con 0x7f03e814f7a0 2024-03-09T09:20:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.303+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00129f0 con 0x7f03e814f7a0 2024-03-09T09:20:36.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.303+0000 7f03eda00640 1 -- 172.21.15.67:0/1391329062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e814c210 con 0x7f03e814f7a0 2024-03-09T09:20:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.304+0000 7f03eda00640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e8152600 0x7f03e81ea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.304+0000 7f03eda00640 1 -- 172.21.15.67:0/1391329062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e81e78e0 con 0x7f03e8152600 2024-03-09T09:20:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.304+0000 7f03e6a00640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e8152600 0x7f03e81ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.305+0000 7f03e6a00640 1 -- 172.21.15.67:0/1391329062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f03e8075a00 con 0x7f03e8152600 2024-03-09T09:20:36.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.305+0000 7f03e6a00640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e8152600 0x7f03e81ea5c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f03d800e7c0 tx=0x7f03d800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.305+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f03e0012b90 con 0x7f03e814f7a0 2024-03-09T09:20:36.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.306+0000 7f03df400640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f03c406b500 0x7f03c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.306+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f03e008a9f0 con 0x7f03e814f7a0 2024-03-09T09:20:36.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.307+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8006b50 con 0x7f03e8152600 2024-03-09T09:20:36.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.307+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f03d8006cf0 con 0x7f03e8152600 2024-03-09T09:20:36.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.307+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e8152600 msgr2=0x7f03e81ea5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.307+0000 7f03df400640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e8152600 0x7f03e81ea5c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f03d800e7c0 tx=0x7f03d800ec80 comp rx=0 tx=0).stop 2024-03-09T09:20:36.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.307+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e8152600 msgr2=0x7f03e81ea5c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:36.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.307+0000 7f03df400640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e8152600 0x7f03e81ea5c0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f03d800e7c0 tx=0x7f03d800ec80 comp rx=0 tx=0).stop 2024-03-09T09:20:36.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.307+0000 7f03de000640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f03c406b500 0x7f03c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.308+0000 7f03de000640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f03c406b500 0x7f03c406d9c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f03d0037000 tx=0x7f03d0031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.314+0000 7f03eda00640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e81466d0 0x7f03e8146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.314+0000 7f03eda00640 1 -- 172.21.15.67:0/1391329062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80c1b50 con 0x7f03e81466d0 2024-03-09T09:20:36.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.315+0000 7f03e6a00640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e81466d0 0x7f03e8146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.315+0000 7f03e6a00640 1 -- 172.21.15.67:0/1391329062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f03e80c33e0 con 0x7f03e81466d0 2024-03-09T09:20:36.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.316+0000 7f03e6a00640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e81466d0 0x7f03e8146ab0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03d8018000 tx=0x7f03d8003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.316+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8006b50 con 0x7f03e81466d0 2024-03-09T09:20:36.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.316+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5019 (secure 0 0 0) 0x7f03d8015430 con 0x7f03e81466d0 2024-03-09T09:20:36.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.316+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e81466d0 msgr2=0x7f03e8146ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.316+0000 7f03df400640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e81466d0 0x7f03e8146ab0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03d8018000 tx=0x7f03d8003ae0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.316+0000 7f03df400640 1 -- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e81466d0 msgr2=0x7f03e8146ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:36.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.316+0000 7f03df400640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e81466d0 0x7f03e8146ab0 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03d8018000 tx=0x7f03d8003ae0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.322 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:20:36.322 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:20:36.322 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 216, 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:36.323 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:36.324 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:20:36.325 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:20:36.326 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.328 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:36.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:20:36.330 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.333 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.335 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.335 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.335 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:20:36.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.335 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.335 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:20:36.335 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:20:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.317+0000 7f03eda00640 1 -- 172.21.15.67:0/1391329062 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f03c406b500 msgr2=0x7f03c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.317+0000 7f03eda00640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f03c406b500 0x7f03c406d9c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f03d0037000 tx=0x7f03d0031000 comp rx=0 tx=0).stop 2024-03-09T09:20:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.317+0000 7f03eda00640 1 -- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e814f7a0 msgr2=0x7f03e814bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.317+0000 7f03eda00640 1 --2- 172.21.15.67:0/1391329062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03e814f7a0 0x7f03e814bcd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f03e000d5d0 tx=0x7f03e000da90 comp rx=0 tx=0).stop 2024-03-09T09:20:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.317+0000 7f03e7400640 1 -- 172.21.15.67:0/1391329062 reap_dead start 2024-03-09T09:20:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.317+0000 7f03e7400640 1 -- 172.21.15.67:0/1391329062 reap_dead start 2024-03-09T09:20:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.317+0000 7f03eda00640 1 -- 172.21.15.67:0/1391329062 shutdown_connections 2024-03-09T09:20:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.317+0000 7f03eda00640 1 -- 172.21.15.67:0/1391329062 >> 172.21.15.67:0/1391329062 conn(0x7f03e80c7370 msgr2=0x7f03e80c45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.317+0000 7f03eda00640 1 -- 172.21.15.67:0/1391329062 shutdown_connections 2024-03-09T09:20:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.317+0000 7f03eda00640 1 -- 172.21.15.67:0/1391329062 wait complete. 2024-03-09T09:20:36.339 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:20:36.339 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:20:36.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:20:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.577+0000 7f2b7ec00640 1 Processor -- start 2024-03-09T09:20:36.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.577+0000 7f2b7ec00640 1 -- start start 2024-03-09T09:20:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.579+0000 7f2b7ec00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2b78153100 0x7f2b780c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.579+0000 7f2b7ec00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b780c55f0 0x7f2b78159240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.579+0000 7f2b7ec00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b78156a90 con 0x7f2b7806e480 2024-03-09T09:20:36.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.579+0000 7f2b7ec00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b780c42f0 con 0x7f2b78153100 2024-03-09T09:20:36.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.579+0000 7f2b7ec00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780c45d0 con 0x7f2b780c55f0 2024-03-09T09:20:36.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.579+0000 7f2b7e200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f2b7806e480 0x7f2b78159780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:57684/0 (socket says 172.21.15.67:57684) 2024-03-09T09:20:36.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.579+0000 7f2b7e200640 1 -- 172.21.15.67:0/416779874 learned_addr learned my addr 172.21.15.67:0/416779874 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:20:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.580+0000 7f2b7d800640 1 --2- 172.21.15.67:0/416779874 >> v2:172.21.15.134:3300/0 conn(0x7f2b78153100 0x7f2b780c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.580+0000 7f2b7ce00640 1 --2- 172.21.15.67:0/416779874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b780c55f0 0x7f2b78159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.580+0000 7f2b77e00640 1 -- 172.21.15.67:0/416779874 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566074719 0 0) 0x7f2b78156a90 con 0x7f2b7806e480 2024-03-09T09:20:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.580+0000 7f2b77e00640 1 -- 172.21.15.67:0/416779874 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b60003520 con 0x7f2b7806e480 2024-03-09T09:20:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.580+0000 7f2b7d800640 1 -- 172.21.15.67:0/416779874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b780c55f0 msgr2=0x7f2b78159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.580+0000 7f2b77e00640 1 -- 172.21.15.67:0/416779874 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1559962480 0 0) 0x7f2b60003520 con 0x7f2b7806e480 2024-03-09T09:20:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.580+0000 7f2b7d800640 1 --2- 172.21.15.67:0/416779874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b780c55f0 0x7f2b78159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.580+0000 7f2b7d800640 1 -- 172.21.15.67:0/416779874 >> v1:172.21.15.67:6789/0 conn(0x7f2b7806e480 legacy=0x7f2b78159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.580+0000 7f2b7d800640 1 -- 172.21.15.67:0/416779874 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b780c48b0 con 0x7f2b78153100 2024-03-09T09:20:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.582+0000 7f2b7d800640 1 --2- 172.21.15.67:0/416779874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b78153100 0x7f2b780c50b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097a0 tx=0x7f2b6c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=738e07c195080135 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.582+0000 7f2b77e00640 1 -- 172.21.15.67:0/416779874 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c037070 con 0x7f2b78153100 2024-03-09T09:20:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.582+0000 7f2b77e00640 1 -- 172.21.15.67:0/416779874 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c030510 con 0x7f2b78153100 2024-03-09T09:20:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.582+0000 7f2b77e00640 1 -- 172.21.15.67:0/416779874 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b700019f0 con 0x7f2b78153100 2024-03-09T09:20:36.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.583+0000 7f2b7ec00640 1 -- 172.21.15.67:0/416779874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b78153100 msgr2=0x7f2b780c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.583+0000 7f2b7ec00640 1 --2- 172.21.15.67:0/416779874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b78153100 0x7f2b780c50b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097a0 tx=0x7f2b6c02b790 comp rx=0 tx=0).stop 2024-03-09T09:20:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.583+0000 7f2b7ec00640 1 -- 172.21.15.67:0/416779874 shutdown_connections 2024-03-09T09:20:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.583+0000 7f2b7ec00640 1 --2- 172.21.15.67:0/416779874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b780c55f0 0x7f2b78159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.583+0000 7f2b7ec00640 1 --2- 172.21.15.67:0/416779874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b78153100 0x7f2b780c50b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.583+0000 7f2b7ec00640 1 -- 172.21.15.67:0/416779874 >> 172.21.15.67:0/416779874 conn(0x7f2b780c8310 msgr2=0x7f2b780c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.583+0000 7f2b7ec00640 1 -- 172.21.15.67:0/416779874 shutdown_connections 2024-03-09T09:20:36.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.583+0000 7f2b7ec00640 1 -- 172.21.15.67:0/416779874 wait complete. 2024-03-09T09:20:36.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.586+0000 7f2b7ec00640 1 Processor -- start 2024-03-09T09:20:36.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.586+0000 7f2b7ec00640 1 -- start start 2024-03-09T09:20:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.587+0000 7f2b7ec00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b7806e480 0x7f2b781ebd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.587+0000 7f2b7ec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b780c55f0 0x7f2b7814f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.587+0000 7f2b7ec00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b78153100 0x7f2b7814cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.587+0000 7f2b7ec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780c2060 con 0x7f2b780c55f0 2024-03-09T09:20:36.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.587+0000 7f2b7ec00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780c1ee0 con 0x7f2b78153100 2024-03-09T09:20:36.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.587+0000 7f2b7ec00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b781566d0 con 0x7f2b7806e480 2024-03-09T09:20:36.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.587+0000 7f2b7ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b780c55f0 0x7f2b7814f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.587+0000 7f2b7ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b780c55f0 0x7f2b7814f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59966/0 (socket says 172.21.15.67:59966) 2024-03-09T09:20:36.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.587+0000 7f2b7ce00640 1 -- 172.21.15.67:0/3481566321 learned_addr learned my addr 172.21.15.67:0/3481566321 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:36.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.588+0000 7f2b7d800640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b7806e480 0x7f2b781ebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.588+0000 7f2b7e200640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b78153100 0x7f2b7814cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.588+0000 7f2b7ce00640 1 -- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b7806e480 msgr2=0x7f2b781ebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.588+0000 7f2b7ce00640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b7806e480 0x7f2b781ebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.588+0000 7f2b7ce00640 1 -- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b78153100 msgr2=0x7f2b7814cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.588+0000 7f2b7ce00640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b78153100 0x7f2b7814cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.588+0000 7f2b7ce00640 1 -- 172.21.15.67:0/3481566321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b781ec2d0 con 0x7f2b780c55f0 2024-03-09T09:20:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.588+0000 7f2b7ce00640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b780c55f0 0x7f2b7814f020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400e9b0 tx=0x7f2b6400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.589+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64017070 con 0x7f2b780c55f0 2024-03-09T09:20:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.589+0000 7f2b7ec00640 1 -- 172.21.15.67:0/3481566321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b780716d0 con 0x7f2b780c55f0 2024-03-09T09:20:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.589+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6400c9a0 con 0x7f2b780c55f0 2024-03-09T09:20:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.589+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6400cd90 con 0x7f2b780c55f0 2024-03-09T09:20:36.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.589+0000 7f2b7ec00640 1 -- 172.21.15.67:0/3481566321 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b78156fb0 con 0x7f2b780c55f0 2024-03-09T09:20:36.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.590+0000 7f2b7ec00640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b781571b0 0x7f2b7814d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.590+0000 7f2b7ec00640 1 -- 172.21.15.67:0/3481566321 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b78158540 con 0x7f2b781571b0 2024-03-09T09:20:36.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.591+0000 7f2b7d800640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b781571b0 0x7f2b7814d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.591+0000 7f2b7d800640 1 -- 172.21.15.67:0/3481566321 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b7804f3c0 con 0x7f2b781571b0 2024-03-09T09:20:36.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.592+0000 7f2b7d800640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b781571b0 0x7f2b7814d140 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2b78155c50 tx=0x7f2b6c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.592+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c043070 con 0x7f2b781571b0 2024-03-09T09:20:36.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.592+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2b6401a070 con 0x7f2b780c55f0 2024-03-09T09:20:36.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.593+0000 7f2b75600640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2b5806b400 0x7f2b5806d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.593+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f2b6408a550 con 0x7f2b780c55f0 2024-03-09T09:20:36.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.593+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f2b6c03c640 con 0x7f2b781571b0 2024-03-09T09:20:36.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.593+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b781571b0 msgr2=0x7f2b7814d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.593+0000 7f2b75600640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b781571b0 0x7f2b7814d140 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2b78155c50 tx=0x7f2b6c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.593+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b781571b0 msgr2=0x7f2b7814d140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:36.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.593+0000 7f2b75600640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b781571b0 0x7f2b7814d140 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2b78155c50 tx=0x7f2b6c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.594+0000 7f2b7e200640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2b5806b400 0x7f2b5806d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.595+0000 7f2b7e200640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2b5806b400 0x7f2b5806d8c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2b781eb350 tx=0x7f2b70009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.602+0000 7f2b7ec00640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b780ac4a0 0x7f2b780ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.602+0000 7f2b7ec00640 1 -- 172.21.15.67:0/3481566321 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780c19a0 con 0x7f2b780ac4a0 2024-03-09T09:20:36.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.603+0000 7f2b7d800640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b780ac4a0 0x7f2b780ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.603+0000 7f2b7d800640 1 -- 172.21.15.67:0/3481566321 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2b78153f80 con 0x7f2b780ac4a0 2024-03-09T09:20:36.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.604+0000 7f2b7d800640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b780ac4a0 0x7f2b780ac880 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097a0 tx=0x7f2b6c008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.604+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c037040 con 0x7f2b780ac4a0 2024-03-09T09:20:36.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.605+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5044 (secure 0 0 0) 0x7f2b6c003610 con 0x7f2b780ac4a0 2024-03-09T09:20:36.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.605+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b780ac4a0 msgr2=0x7f2b780ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.605+0000 7f2b75600640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b780ac4a0 0x7f2b780ac880 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097a0 tx=0x7f2b6c008030 comp rx=0 tx=0).stop 2024-03-09T09:20:36.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.605+0000 7f2b75600640 1 -- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b780ac4a0 msgr2=0x7f2b780ac880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:36.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.605+0000 7f2b75600640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b780ac4a0 0x7f2b780ac880 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0097a0 tx=0x7f2b6c008030 comp rx=0 tx=0).stop 2024-03-09T09:20:36.611 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:20:36.611 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:20:36.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:20:36.611 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:20:36.611 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 216, 2024-03-09T09:20:36.611 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:20:36.611 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:20:36.611 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:36.612 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:36.613 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:36.614 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:36.614 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:36.614 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.614 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:20:36.614 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:20:36.614 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:20:36.615 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.616 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:20:36.617 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:36.618 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:20:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.621 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.622 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.623 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:20:36.624 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:20:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.606+0000 7f2b7ec00640 1 -- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2b5806b400 msgr2=0x7f2b5806d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.606+0000 7f2b7ec00640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2b5806b400 0x7f2b5806d8c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2b781eb350 tx=0x7f2b70009040 comp rx=0 tx=0).stop 2024-03-09T09:20:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.606+0000 7f2b7ec00640 1 -- 172.21.15.67:0/3481566321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b780c55f0 msgr2=0x7f2b7814f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.606+0000 7f2b7ec00640 1 --2- 172.21.15.67:0/3481566321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2b780c55f0 0x7f2b7814f020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400e9b0 tx=0x7f2b6400ee70 comp rx=0 tx=0).stop 2024-03-09T09:20:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.606+0000 7f2b7e200640 1 -- 172.21.15.67:0/3481566321 reap_dead start 2024-03-09T09:20:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.606+0000 7f2b7e200640 1 -- 172.21.15.67:0/3481566321 reap_dead start 2024-03-09T09:20:36.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.606+0000 7f2b7ec00640 1 -- 172.21.15.67:0/3481566321 shutdown_connections 2024-03-09T09:20:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.606+0000 7f2b7ec00640 1 -- 172.21.15.67:0/3481566321 >> 172.21.15.67:0/3481566321 conn(0x7f2b780c8310 msgr2=0x7f2b78154530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.606+0000 7f2b7ec00640 1 -- 172.21.15.67:0/3481566321 shutdown_connections 2024-03-09T09:20:36.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.606+0000 7f2b7ec00640 1 -- 172.21.15.67:0/3481566321 wait complete. 2024-03-09T09:20:36.630 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:20:36.630 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:20:36.630 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:20:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.844+0000 7fb120400640 1 Processor -- start 2024-03-09T09:20:36.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.845+0000 7fb120400640 1 -- start start 2024-03-09T09:20:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.846+0000 7fb120400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb11814c9a0 0x7fb1181567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.846+0000 7fb120400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1180c1d70 0x7fb1180c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.846+0000 7fb120400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1180ac4e0 con 0x7fb1180c0ce0 2024-03-09T09:20:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.846+0000 7fb120400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11806d590 con 0x7fb11814c9a0 2024-03-09T09:20:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.846+0000 7fb120400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb118156d00 con 0x7fb1180c1d70 2024-03-09T09:20:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.846+0000 7fb11da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb11814c9a0 0x7fb1181567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.846+0000 7fb11e400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb1180c0ce0 0x7fb1180ac9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:57698/0 (socket says 172.21.15.67:57698) 2024-03-09T09:20:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.846+0000 7fb11e400640 1 -- 172.21.15.67:0/868215142 learned_addr learned my addr 172.21.15.67:0/868215142 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:20:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.847+0000 7fb11d000640 1 --2- 172.21.15.67:0/868215142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1180c1d70 0x7fb1180c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.847+0000 7fb11da00640 1 -- 172.21.15.67:0/868215142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1180c1d70 msgr2=0x7fb1180c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:36.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.847+0000 7fb11da00640 1 --2- 172.21.15.67:0/868215142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1180c1d70 0x7fb1180c07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.847+0000 7fb11da00640 1 -- 172.21.15.67:0/868215142 >> v1:172.21.15.67:6789/0 conn(0x7fb1180c0ce0 legacy=0x7fb1180ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.847+0000 7fb11da00640 1 -- 172.21.15.67:0/868215142 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1180aa550 con 0x7fb11814c9a0 2024-03-09T09:20:36.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.848+0000 7fb11da00640 1 --2- 172.21.15.67:0/868215142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb11814c9a0 0x7fb1181567c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0097a0 tx=0x7fb10c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cda7756e238c5de server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.849+0000 7fb117e00640 1 -- 172.21.15.67:0/868215142 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c031780 con 0x7fb11814c9a0 2024-03-09T09:20:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.849+0000 7fb117e00640 1 -- 172.21.15.67:0/868215142 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c037070 con 0x7fb11814c9a0 2024-03-09T09:20:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.849+0000 7fb117e00640 1 -- 172.21.15.67:0/868215142 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c031b40 con 0x7fb11814c9a0 2024-03-09T09:20:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.849+0000 7fb120400640 1 -- 172.21.15.67:0/868215142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb11814c9a0 msgr2=0x7fb1181567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.849+0000 7fb120400640 1 --2- 172.21.15.67:0/868215142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb11814c9a0 0x7fb1181567c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0097a0 tx=0x7fb10c02b820 comp rx=0 tx=0).stop 2024-03-09T09:20:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.850+0000 7fb120400640 1 -- 172.21.15.67:0/868215142 shutdown_connections 2024-03-09T09:20:36.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.850+0000 7fb120400640 1 --2- 172.21.15.67:0/868215142 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1180c1d70 0x7fb1180c07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.850+0000 7fb120400640 1 --2- 172.21.15.67:0/868215142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb11814c9a0 0x7fb1181567c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.850+0000 7fb120400640 1 -- 172.21.15.67:0/868215142 >> 172.21.15.67:0/868215142 conn(0x7fb1180c7370 msgr2=0x7fb1180c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.850+0000 7fb120400640 1 -- 172.21.15.67:0/868215142 shutdown_connections 2024-03-09T09:20:36.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.850+0000 7fb120400640 1 -- 172.21.15.67:0/868215142 wait complete. 2024-03-09T09:20:36.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.852+0000 7fb120400640 1 Processor -- start 2024-03-09T09:20:36.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.852+0000 7fb120400640 1 -- start start 2024-03-09T09:20:36.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.853+0000 7fb120400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1180c0ce0 0x7fb1181ef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.853+0000 7fb120400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1180c1d70 0x7fb118151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.853+0000 7fb120400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb11814c9a0 0x7fb118072250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.853+0000 7fb120400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180aadf0 con 0x7fb1180c0ce0 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.853+0000 7fb120400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180aab10 con 0x7fb11814c9a0 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.853+0000 7fb120400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180ad0d0 con 0x7fb1180c1d70 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.854+0000 7fb11da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1180c0ce0 0x7fb1181ef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.854+0000 7fb11da00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1180c0ce0 0x7fb1181ef0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59982/0 (socket says 172.21.15.67:59982) 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.854+0000 7fb11da00640 1 -- 172.21.15.67:0/703279238 learned_addr learned my addr 172.21.15.67:0/703279238 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.854+0000 7fb11d000640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1180c1d70 0x7fb118151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.854+0000 7fb11da00640 1 -- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1180c1d70 msgr2=0x7fb118151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.854+0000 7fb11da00640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1180c1d70 0x7fb118151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.854+0000 7fb11da00640 1 -- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb11814c9a0 msgr2=0x7fb118072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.854+0000 7fb11da00640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb11814c9a0 0x7fb118072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.854+0000 7fb11da00640 1 -- 172.21.15.67:0/703279238 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1181efb50 con 0x7fb1180c0ce0 2024-03-09T09:20:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.855+0000 7fb11da00640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1180c0ce0 0x7fb1181ef0d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb10c034040 tx=0x7fb10c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.855+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c032610 con 0x7fb1180c0ce0 2024-03-09T09:20:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.855+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c0327b0 con 0x7fb1180c0ce0 2024-03-09T09:20:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.855+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c032ba0 con 0x7fb1180c0ce0 2024-03-09T09:20:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.855+0000 7fb120400640 1 -- 172.21.15.67:0/703279238 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb118071770 con 0x7fb1180c0ce0 2024-03-09T09:20:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.855+0000 7fb120400640 1 -- 172.21.15.67:0/703279238 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb118071d30 con 0x7fb1180c0ce0 2024-03-09T09:20:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.856+0000 7fb120400640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb118148fb0 0x7fb1181eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.856+0000 7fb120400640 1 -- 172.21.15.67:0/703279238 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb118071f70 con 0x7fb118148fb0 2024-03-09T09:20:36.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.857+0000 7fb11d000640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb118148fb0 0x7fb1181eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.857+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb10c04e430 con 0x7fb1180c0ce0 2024-03-09T09:20:36.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.858+0000 7fb11d000640 1 -- 172.21.15.67:0/703279238 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb118072f50 con 0x7fb118148fb0 2024-03-09T09:20:36.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.858+0000 7fb11d000640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb118148fb0 0x7fb1181eb8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb1180733b0 tx=0x7fb1080119f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.859+0000 7fb115600640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb0fc06b4d0 0x7fb0fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.859+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb10c0ac600 con 0x7fb1180c0ce0 2024-03-09T09:20:36.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.860+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108004230 con 0x7fb118148fb0 2024-03-09T09:20:36.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.860+0000 7fb11e400640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb0fc06b4d0 0x7fb0fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.860+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb108004230 con 0x7fb118148fb0 2024-03-09T09:20:36.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.860+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb118148fb0 msgr2=0x7fb1181eb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.860+0000 7fb115600640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb118148fb0 0x7fb1181eb8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb1180733b0 tx=0x7fb1080119f0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.860+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb118148fb0 msgr2=0x7fb1181eb8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:36.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.860+0000 7fb115600640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb118148fb0 0x7fb1181eb8f0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb1180733b0 tx=0x7fb1080119f0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.861+0000 7fb11e400640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb0fc06b4d0 0x7fb0fc06d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb110001ff0 tx=0x7fb110001660 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.868+0000 7fb120400640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1181ef610 0x7fb1181ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:36.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.868+0000 7fb120400640 1 -- 172.21.15.67:0/703279238 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11814bf70 con 0x7fb1181ef610 2024-03-09T09:20:36.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.869+0000 7fb11d000640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1181ef610 0x7fb1181ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:36.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.869+0000 7fb11d000640 1 -- 172.21.15.67:0/703279238 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb1180c15e0 con 0x7fb1181ef610 2024-03-09T09:20:36.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.870+0000 7fb11d000640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1181ef610 0x7fb1181ebe30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb1180733b0 tx=0x7fb1080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:36.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.870+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10802b020 con 0x7fb1181ef610 2024-03-09T09:20:36.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.871+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5044 (secure 0 0 0) 0x7fb108003ca0 con 0x7fb1181ef610 2024-03-09T09:20:36.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.871+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1181ef610 msgr2=0x7fb1181ebe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.871+0000 7fb115600640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1181ef610 0x7fb1181ebe30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb1180733b0 tx=0x7fb1080046c0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.871+0000 7fb115600640 1 -- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1181ef610 msgr2=0x7fb1181ebe30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:36.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.871+0000 7fb115600640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1181ef610 0x7fb1181ebe30 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb1180733b0 tx=0x7fb1080046c0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.878 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:20:36.878 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:20:36.878 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:20:36.878 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:20:36.878 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 216, 2024-03-09T09:20:36.878 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:20:36.878 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:20:36.878 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:20:36.878 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:20:36.878 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.878 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:36.879 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:36.880 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:20:36.881 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:36.882 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:20:36.883 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:36.884 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:36.885 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:20:36.886 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.888 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:20:36.889 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:36.890 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:36.891 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:36.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:36.891 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:20:36.891 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:20:36.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.872+0000 7fb120400640 1 -- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb0fc06b4d0 msgr2=0x7fb0fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.872+0000 7fb120400640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb0fc06b4d0 0x7fb0fc06d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb110001ff0 tx=0x7fb110001660 comp rx=0 tx=0).stop 2024-03-09T09:20:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.872+0000 7fb120400640 1 -- 172.21.15.67:0/703279238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1180c0ce0 msgr2=0x7fb1181ef0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.872+0000 7fb120400640 1 --2- 172.21.15.67:0/703279238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1180c0ce0 0x7fb1181ef0d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb10c034040 tx=0x7fb10c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:20:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.872+0000 7fb11e400640 1 -- 172.21.15.67:0/703279238 reap_dead start 2024-03-09T09:20:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.872+0000 7fb11e400640 1 -- 172.21.15.67:0/703279238 reap_dead start 2024-03-09T09:20:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.872+0000 7fb120400640 1 -- 172.21.15.67:0/703279238 shutdown_connections 2024-03-09T09:20:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.872+0000 7fb120400640 1 -- 172.21.15.67:0/703279238 >> 172.21.15.67:0/703279238 conn(0x7fb1180c7370 msgr2=0x7fb1180c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.872+0000 7fb120400640 1 -- 172.21.15.67:0/703279238 shutdown_connections 2024-03-09T09:20:36.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:36.872+0000 7fb120400640 1 -- 172.21.15.67:0/703279238 wait complete. 2024-03-09T09:20:36.895 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:20:36.896 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:20:36.896 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:20:37.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.088+0000 7f5b43000640 1 Processor -- start 2024-03-09T09:20:37.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.088+0000 7f5b43000640 1 -- start start 2024-03-09T09:20:37.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.090+0000 7f5b43000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5b3c0c1f70 0x7f5b3c150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:37.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.090+0000 7f5b43000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0710a0 0x7f5b3c151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:37.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.090+0000 7f5b43000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b3c0c27a0 0x7f5b3c153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:37.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.090+0000 7f5b43000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3c0c81e0 con 0x7f5b3c0c27a0 2024-03-09T09:20:37.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.090+0000 7f5b43000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3c0c32b0 con 0x7f5b3c0c1f70 2024-03-09T09:20:37.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.090+0000 7f5b43000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c0c2fd0 con 0x7f5b3c0710a0 2024-03-09T09:20:37.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.090+0000 7f5b42600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b3c0c27a0 0x7f5b3c153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:37.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.090+0000 7f5b42600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5b3c0c27a0 0x7f5b3c153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59984/0 (socket says 172.21.15.67:59984) 2024-03-09T09:20:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.090+0000 7f5b42600640 1 -- 172.21.15.67:0/1755647338 learned_addr learned my addr 172.21.15.67:0/1755647338 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.090+0000 7f5b41c00640 1 --2- 172.21.15.67:0/1755647338 >> v2:172.21.15.134:3300/0 conn(0x7f5b3c0c1f70 0x7f5b3c150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.090+0000 7f5b41200640 1 --2- 172.21.15.67:0/1755647338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0710a0 0x7f5b3c151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.091+0000 7f5b42600640 1 -- 172.21.15.67:0/1755647338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0710a0 msgr2=0x7f5b3c151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.091+0000 7f5b42600640 1 --2- 172.21.15.67:0/1755647338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0710a0 0x7f5b3c151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.091+0000 7f5b42600640 1 -- 172.21.15.67:0/1755647338 >> v2:172.21.15.134:3300/0 conn(0x7f5b3c0c1f70 msgr2=0x7f5b3c150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.091+0000 7f5b42600640 1 --2- 172.21.15.67:0/1755647338 >> v2:172.21.15.134:3300/0 conn(0x7f5b3c0c1f70 0x7f5b3c150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.091+0000 7f5b42600640 1 -- 172.21.15.67:0/1755647338 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c0c3700 con 0x7f5b3c0c27a0 2024-03-09T09:20:37.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.093+0000 7f5b42600640 1 --2- 172.21.15.67:0/1755647338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b3c0c27a0 0x7f5b3c153970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c06e890 tx=0x7f5b3402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8011da5ea7e9c9ba server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:37.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.093+0000 7f5b3be00640 1 -- 172.21.15.67:0/1755647338 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3402c9e0 con 0x7f5b3c0c27a0 2024-03-09T09:20:37.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.093+0000 7f5b3be00640 1 -- 172.21.15.67:0/1755647338 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3402cb80 con 0x7f5b3c0c27a0 2024-03-09T09:20:37.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.093+0000 7f5b3be00640 1 -- 172.21.15.67:0/1755647338 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b340343f0 con 0x7f5b3c0c27a0 2024-03-09T09:20:37.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.094+0000 7f5b43000640 1 -- 172.21.15.67:0/1755647338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b3c0c27a0 msgr2=0x7f5b3c153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:37.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.094+0000 7f5b43000640 1 --2- 172.21.15.67:0/1755647338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b3c0c27a0 0x7f5b3c153970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c06e890 tx=0x7f5b3402be40 comp rx=0 tx=0).stop 2024-03-09T09:20:37.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.094+0000 7f5b43000640 1 -- 172.21.15.67:0/1755647338 shutdown_connections 2024-03-09T09:20:37.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.094+0000 7f5b43000640 1 --2- 172.21.15.67:0/1755647338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b3c0c27a0 0x7f5b3c153970 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.094+0000 7f5b43000640 1 --2- 172.21.15.67:0/1755647338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0710a0 0x7f5b3c151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.094+0000 7f5b43000640 1 --2- 172.21.15.67:0/1755647338 >> v2:172.21.15.134:3300/0 conn(0x7f5b3c0c1f70 0x7f5b3c150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.094+0000 7f5b43000640 1 -- 172.21.15.67:0/1755647338 >> 172.21.15.67:0/1755647338 conn(0x7f5b3c0c7300 msgr2=0x7f5b3c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:37.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.094+0000 7f5b43000640 1 -- 172.21.15.67:0/1755647338 shutdown_connections 2024-03-09T09:20:37.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.094+0000 7f5b43000640 1 -- 172.21.15.67:0/1755647338 wait complete. 2024-03-09T09:20:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.096+0000 7f5b43000640 1 Processor -- start 2024-03-09T09:20:37.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.096+0000 7f5b43000640 1 -- start start 2024-03-09T09:20:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.097+0000 7f5b43000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b3c0710a0 0x7f5b3c1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.097+0000 7f5b43000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0c1f70 0x7f5b3c0abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.097+0000 7f5b43000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c0c27a0 0x7f5b3c154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.097+0000 7f5b43000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c070d50 con 0x7f5b3c0710a0 2024-03-09T09:20:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.097+0000 7f5b43000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c070830 con 0x7f5b3c0c27a0 2024-03-09T09:20:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.097+0000 7f5b43000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c070ed0 con 0x7f5b3c0c1f70 2024-03-09T09:20:37.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.097+0000 7f5b41c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b3c0710a0 0x7f5b3c1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.097+0000 7f5b41c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b3c0710a0 0x7f5b3c1573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59992/0 (socket says 172.21.15.67:59992) 2024-03-09T09:20:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.097+0000 7f5b41c00640 1 -- 172.21.15.67:0/2102669444 learned_addr learned my addr 172.21.15.67:0/2102669444 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.098+0000 7f5b41200640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0c1f70 0x7f5b3c0abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.098+0000 7f5b42600640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c0c27a0 0x7f5b3c154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.098+0000 7f5b41c00640 1 -- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0c1f70 msgr2=0x7f5b3c0abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.098+0000 7f5b41c00640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0c1f70 0x7f5b3c0abde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.098+0000 7f5b41c00640 1 -- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c0c27a0 msgr2=0x7f5b3c154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.098+0000 7f5b41c00640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c0c27a0 0x7f5b3c154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.098+0000 7f5b41c00640 1 -- 172.21.15.67:0/2102669444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c154790 con 0x7f5b3c0710a0 2024-03-09T09:20:37.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.098+0000 7f5b41c00640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b3c0710a0 0x7f5b3c1573a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000d620 tx=0x7f5b3000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:37.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.098+0000 7f5b39600640 1 -- 172.21.15.67:0/2102669444 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3000b620 con 0x7f5b3c0710a0 2024-03-09T09:20:37.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.099+0000 7f5b39600640 1 -- 172.21.15.67:0/2102669444 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b30019070 con 0x7f5b3c0710a0 2024-03-09T09:20:37.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.099+0000 7f5b43000640 1 -- 172.21.15.67:0/2102669444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c1eb220 con 0x7f5b3c0710a0 2024-03-09T09:20:37.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.099+0000 7f5b39600640 1 -- 172.21.15.67:0/2102669444 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3000ba00 con 0x7f5b3c0710a0 2024-03-09T09:20:37.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.099+0000 7f5b43000640 1 -- 172.21.15.67:0/2102669444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c152960 con 0x7f5b3c0710a0 2024-03-09T09:20:37.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.100+0000 7f5b43000640 1 -- 172.21.15.67:0/2102669444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3c153560 con 0x7f5b3c0710a0 2024-03-09T09:20:37.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.102+0000 7f5b39600640 1 -- 172.21.15.67:0/2102669444 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5b3000bba0 con 0x7f5b3c0710a0 2024-03-09T09:20:37.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.103+0000 7f5b39600640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b1c06b4d0 0x7f5b1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:37.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.103+0000 7f5b39600640 1 -- 172.21.15.67:0/2102669444 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f5b3008aa40 con 0x7f5b3c0710a0 2024-03-09T09:20:37.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.104+0000 7f5b41200640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b1c06b4d0 0x7f5b1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:37.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.105+0000 7f5b41200640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b1c06b4d0 0x7f5b1c06d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5b28006fd0 tx=0x7f5b28008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:37.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.113+0000 7f5b39600640 1 -- 172.21.15.67:0/2102669444 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5b300594c0 con 0x7f5b3c0710a0 2024-03-09T09:20:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.404+0000 7f5b43000640 1 -- 172.21.15.67:0/2102669444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f5b3c04f3c0 con 0x7f5b3c0710a0 2024-03-09T09:20:37.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.406+0000 7f5b39600640 1 -- 172.21.15.67:0/2102669444 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f5b30058dc0 con 0x7f5b3c0710a0 2024-03-09T09:20:37.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 -- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b1c06b4d0 msgr2=0x7f5b1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b1c06b4d0 0x7f5b1c06d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5b28006fd0 tx=0x7f5b28008040 comp rx=0 tx=0).stop 2024-03-09T09:20:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 -- 172.21.15.67:0/2102669444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b3c0710a0 msgr2=0x7f5b3c1573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b3c0710a0 0x7f5b3c1573a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000d620 tx=0x7f5b3000dae0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 -- 172.21.15.67:0/2102669444 shutdown_connections 2024-03-09T09:20:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5b1c06b4d0 0x7f5b1c06d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b3c0c27a0 0x7f5b3c154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5b3c0c1f70 0x7f5b3c0abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 --2- 172.21.15.67:0/2102669444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b3c0710a0 0x7f5b3c1573a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 -- 172.21.15.67:0/2102669444 >> 172.21.15.67:0/2102669444 conn(0x7f5b3c0c7300 msgr2=0x7f5b3c14fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 -- 172.21.15.67:0/2102669444 shutdown_connections 2024-03-09T09:20:37.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:37.409+0000 7f5b43000640 1 -- 172.21.15.67:0/2102669444 wait complete. 2024-03-09T09:20:37.438 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:20:37.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:20:37.747 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:20:37.747 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 44 -eq 48 ']' 2024-03-09T09:20:37.747 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.44 2024-03-09T09:20:37.747 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.44 2024-03-09T09:20:37.747 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.44 2024-03-09T09:20:37.747 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.44 2024-03-09T09:20:37.750 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:20:40.728 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:20:40.728 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:40.728 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 52 -eq 48 ']' 2024-03-09T09:20:40.729 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.52 2024-03-09T09:20:40.729 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.52 2024-03-09T09:20:40.729 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.52 2024-03-09T09:20:40.729 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.52 2024-03-09T09:20:40.733 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:20:42.991 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:20:42.991 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:20:42.991 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 45 -eq 48 ']' 2024-03-09T09:20:42.992 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.45 2024-03-09T09:20:42.992 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.45 2024-03-09T09:20:42.992 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.45 2024-03-09T09:20:42.992 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.45 2024-03-09T09:20:42.996 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:20:45.038 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:20:45.038 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:45.038 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 53 -eq 48 ']' 2024-03-09T09:20:45.038 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.53 2024-03-09T09:20:45.039 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.53 2024-03-09T09:20:45.039 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.53 2024-03-09T09:20:45.039 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.53 2024-03-09T09:20:45.043 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:20:47.439 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:20:47.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:20:47.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.645+0000 7f2ac6c00640 1 Processor -- start 2024-03-09T09:20:47.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.645+0000 7f2ac6c00640 1 -- start start 2024-03-09T09:20:47.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.646+0000 7f2ac6c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2ac00c3600 0x7f2ac00ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:47.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.647+0000 7f2ac6c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ac00abca0 0x7f2ac00aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:47.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.647+0000 7f2ac6c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac01579e0 con 0x7f2ac0157600 2024-03-09T09:20:47.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.647+0000 7f2ac6c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac006d4c0 con 0x7f2ac00c3600 2024-03-09T09:20:47.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.647+0000 7f2ac6c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac006d7a0 con 0x7f2ac00abca0 2024-03-09T09:20:47.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.647+0000 7f2ac4e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2ac00c3600 0x7f2ac00ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:47.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.647+0000 7f2ac4e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2ac00c3600 0x7f2ac00ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46998/0 (socket says 172.21.15.67:46998) 2024-03-09T09:20:47.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.647+0000 7f2ac4e00640 1 -- 172.21.15.67:0/85962311 learned_addr learned my addr 172.21.15.67:0/85962311 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:47.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.647+0000 7f2ac6200640 1 --2- 172.21.15.67:0/85962311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ac00abca0 0x7f2ac00aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:47.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.648+0000 7f2ac4e00640 1 -- 172.21.15.67:0/85962311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ac00abca0 msgr2=0x7f2ac00aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:47.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.648+0000 7f2ac4e00640 1 --2- 172.21.15.67:0/85962311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ac00abca0 0x7f2ac00aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:47.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.648+0000 7f2ac4e00640 1 -- 172.21.15.67:0/85962311 >> v1:172.21.15.134:6789/0 conn(0x7f2ac0157600 legacy=0x7f2ac015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:47.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.648+0000 7f2ac4e00640 1 -- 172.21.15.67:0/85962311 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac006da80 con 0x7f2ac00c3600 2024-03-09T09:20:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.650+0000 7f2ac4e00640 1 --2- 172.21.15.67:0/85962311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac00c3600 0x7f2ac00ab760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009870 tx=0x7f2aac02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=989cba83d0c1de3e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:47.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.650+0000 7f2abfe00640 1 -- 172.21.15.67:0/85962311 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac034070 con 0x7f2ac00c3600 2024-03-09T09:20:47.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.650+0000 7f2abfe00640 1 -- 172.21.15.67:0/85962311 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac030840 con 0x7f2ac00c3600 2024-03-09T09:20:47.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.650+0000 7f2abfe00640 1 -- 172.21.15.67:0/85962311 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac030c10 con 0x7f2ac00c3600 2024-03-09T09:20:47.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.651+0000 7f2ac6c00640 1 -- 172.21.15.67:0/85962311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac00c3600 msgr2=0x7f2ac00ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.651+0000 7f2ac6c00640 1 --2- 172.21.15.67:0/85962311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac00c3600 0x7f2ac00ab760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009870 tx=0x7f2aac02b980 comp rx=0 tx=0).stop 2024-03-09T09:20:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.651+0000 7f2ac6c00640 1 -- 172.21.15.67:0/85962311 shutdown_connections 2024-03-09T09:20:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.651+0000 7f2ac6c00640 1 --2- 172.21.15.67:0/85962311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ac00abca0 0x7f2ac00aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.651+0000 7f2ac6c00640 1 --2- 172.21.15.67:0/85962311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac00c3600 0x7f2ac00ab760 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.651+0000 7f2ac6c00640 1 -- 172.21.15.67:0/85962311 >> 172.21.15.67:0/85962311 conn(0x7f2ac00c7300 msgr2=0x7f2ac00c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.651+0000 7f2ac6c00640 1 -- 172.21.15.67:0/85962311 shutdown_connections 2024-03-09T09:20:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.651+0000 7f2ac6c00640 1 -- 172.21.15.67:0/85962311 wait complete. 2024-03-09T09:20:47.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.653+0000 7f2ac6c00640 1 Processor -- start 2024-03-09T09:20:47.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.654+0000 7f2ac6c00640 1 -- start start 2024-03-09T09:20:47.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.655+0000 7f2ac6c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ac00abca0 0x7f2ac015dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:47.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.655+0000 7f2ac6c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac00c3600 0x7f2ac01e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:47.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.655+0000 7f2ac6c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac0157600 0x7f2ac01efc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:47.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.655+0000 7f2ac6c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00aaec0 con 0x7f2ac0157600 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.655+0000 7f2ac6c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00aad40 con 0x7f2ac00c3600 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.655+0000 7f2ac6c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00ab040 con 0x7f2ac00abca0 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.655+0000 7f2ac6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac0157600 0x7f2ac01efc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.655+0000 7f2ac6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac0157600 0x7f2ac01efc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59038/0 (socket says 172.21.15.67:59038) 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.655+0000 7f2ac6200640 1 -- 172.21.15.67:0/4175093322 learned_addr learned my addr 172.21.15.67:0/4175093322 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.655+0000 7f2ac4e00640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac00c3600 0x7f2ac01e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.655+0000 7f2ac5800640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ac00abca0 0x7f2ac015dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.656+0000 7f2ac6200640 1 -- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ac00abca0 msgr2=0x7f2ac015dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.656+0000 7f2ac6200640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ac00abca0 0x7f2ac015dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.656+0000 7f2ac6200640 1 -- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac00c3600 msgr2=0x7f2ac01e5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.656+0000 7f2ac6200640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac00c3600 0x7f2ac01e5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.656+0000 7f2ac6200640 1 -- 172.21.15.67:0/4175093322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac01f31f0 con 0x7f2ac0157600 2024-03-09T09:20:47.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.656+0000 7f2ac6200640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac0157600 0x7f2ac01efc40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009dd0 tx=0x7f2ab800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:47.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.656+0000 7f2abd600640 1 -- 172.21.15.67:0/4175093322 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8018020 con 0x7f2ac0157600 2024-03-09T09:20:47.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.657+0000 7f2abd600640 1 -- 172.21.15.67:0/4175093322 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab8014480 con 0x7f2ac0157600 2024-03-09T09:20:47.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.657+0000 7f2abd600640 1 -- 172.21.15.67:0/4175093322 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8014870 con 0x7f2ac0157600 2024-03-09T09:20:47.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.657+0000 7f2ac6c00640 1 -- 172.21.15.67:0/4175093322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac01f34d0 con 0x7f2ac0157600 2024-03-09T09:20:47.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.657+0000 7f2ac6c00640 1 -- 172.21.15.67:0/4175093322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac01f3a90 con 0x7f2ac0157600 2024-03-09T09:20:47.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.658+0000 7f2ac6c00640 1 -- 172.21.15.67:0/4175093322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac00aa000 con 0x7f2ac0157600 2024-03-09T09:20:47.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.659+0000 7f2abd600640 1 -- 172.21.15.67:0/4175093322 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2ab8005d10 con 0x7f2ac0157600 2024-03-09T09:20:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.660+0000 7f2abd600640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2aa006b4d0 0x7f2aa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.660+0000 7f2abd600640 1 -- 172.21.15.67:0/4175093322 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f2ab808b1b0 con 0x7f2ac0157600 2024-03-09T09:20:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.661+0000 7f2ac5800640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2aa006b4d0 0x7f2aa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:47.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.662+0000 7f2ac5800640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2aa006b4d0 0x7f2aa006d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4007fd0 tx=0x7f2ab4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:47.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:47.665+0000 7f2abd600640 1 -- 172.21.15.67:0/4175093322 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2ac00aa000 con 0x7f2ac0157600 2024-03-09T09:20:48.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.054+0000 7f2ac6c00640 1 -- 172.21.15.67:0/4175093322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2ac004f3c0 con 0x7f2ac0157600 2024-03-09T09:20:48.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.054+0000 7f2abd600640 1 -- 172.21.15.67:0/4175093322 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f2ab8022070 con 0x7f2ac0157600 2024-03-09T09:20:48.061 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":216,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:20:48.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 -- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2aa006b4d0 msgr2=0x7f2aa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2aa006b4d0 0x7f2aa006d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4007fd0 tx=0x7f2ab4009040 comp rx=0 tx=0).stop 2024-03-09T09:20:48.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 -- 172.21.15.67:0/4175093322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac0157600 msgr2=0x7f2ac01efc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac0157600 0x7f2ac01efc40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009dd0 tx=0x7f2ab800c490 comp rx=0 tx=0).stop 2024-03-09T09:20:48.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 -- 172.21.15.67:0/4175093322 shutdown_connections 2024-03-09T09:20:48.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2aa006b4d0 0x7f2aa006d990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ac0157600 0x7f2ac01efc40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac00c3600 0x7f2ac01e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 --2- 172.21.15.67:0/4175093322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ac00abca0 0x7f2ac015dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 -- 172.21.15.67:0/4175093322 >> 172.21.15.67:0/4175093322 conn(0x7f2ac00c7300 msgr2=0x7f2ac0158e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:48.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 -- 172.21.15.67:0/4175093322 shutdown_connections 2024-03-09T09:20:48.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.057+0000 7f2ac6c00640 1 -- 172.21.15.67:0/4175093322 wait complete. 2024-03-09T09:20:48.083 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:20:48.083 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:20:48.083 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:20:48.083 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:20:48.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.307+0000 7f94f3e00640 1 Processor -- start 2024-03-09T09:20:48.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.307+0000 7f94f3e00640 1 -- start start 2024-03-09T09:20:48.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.309+0000 7f94f3e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f94f406d5f0 0x7f94f41575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.309+0000 7f94f3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94f4157b20 0x7f94f40c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.309+0000 7f94f3e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f4154110 con 0x7f94f4150db0 2024-03-09T09:20:48.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.309+0000 7f94f3e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f41543f0 con 0x7f94f4157b20 2024-03-09T09:20:48.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.309+0000 7f94f3e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f41546d0 con 0x7f94f406d5f0 2024-03-09T09:20:48.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.309+0000 7f94f3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94f4157b20 0x7f94f40c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.310+0000 7f94f3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f94f4157b20 0x7f94f40c4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59054/0 (socket says 172.21.15.67:59054) 2024-03-09T09:20:48.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.310+0000 7f94f3400640 1 -- 172.21.15.67:0/3184208615 learned_addr learned my addr 172.21.15.67:0/3184208615 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:48.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.310+0000 7f94f2000640 1 --2- 172.21.15.67:0/3184208615 >> v2:172.21.15.134:3300/0 conn(0x7f94f406d5f0 0x7f94f41575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.310+0000 7f94f3400640 1 -- 172.21.15.67:0/3184208615 >> v2:172.21.15.134:3300/0 conn(0x7f94f406d5f0 msgr2=0x7f94f41575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:48.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.310+0000 7f94f3400640 1 --2- 172.21.15.67:0/3184208615 >> v2:172.21.15.134:3300/0 conn(0x7f94f406d5f0 0x7f94f41575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.310+0000 7f94f3400640 1 -- 172.21.15.67:0/3184208615 >> v1:172.21.15.134:6789/0 conn(0x7f94f4150db0 legacy=0x7f94f41538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:48.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.310+0000 7f94f3400640 1 -- 172.21.15.67:0/3184208615 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f41570c0 con 0x7f94f4157b20 2024-03-09T09:20:48.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.312+0000 7f94f3400640 1 --2- 172.21.15.67:0/3184208615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f4157b20 0x7f94f40c4ce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f94f4073e30 tx=0x7f94ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e72846346dbcea server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.312+0000 7f94f1600640 1 -- 172.21.15.67:0/3184208615 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec035020 con 0x7f94f4157b20 2024-03-09T09:20:48.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.312+0000 7f94f1600640 1 -- 172.21.15.67:0/3184208615 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ec031870 con 0x7f94f4157b20 2024-03-09T09:20:48.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.312+0000 7f94f1600640 1 -- 172.21.15.67:0/3184208615 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec031c40 con 0x7f94f4157b20 2024-03-09T09:20:48.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.313+0000 7f94f3e00640 1 -- 172.21.15.67:0/3184208615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f4157b20 msgr2=0x7f94f40c4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.313+0000 7f94f3e00640 1 --2- 172.21.15.67:0/3184208615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f4157b20 0x7f94f40c4ce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f94f4073e30 tx=0x7f94ec02b910 comp rx=0 tx=0).stop 2024-03-09T09:20:48.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.313+0000 7f94f3e00640 1 -- 172.21.15.67:0/3184208615 shutdown_connections 2024-03-09T09:20:48.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.313+0000 7f94f3e00640 1 --2- 172.21.15.67:0/3184208615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f4157b20 0x7f94f40c4ce0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.313+0000 7f94f3e00640 1 --2- 172.21.15.67:0/3184208615 >> v2:172.21.15.134:3300/0 conn(0x7f94f406d5f0 0x7f94f41575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.313+0000 7f94f3e00640 1 -- 172.21.15.67:0/3184208615 >> 172.21.15.67:0/3184208615 conn(0x7f94f406c150 msgr2=0x7f94f406c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:48.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.313+0000 7f94f3e00640 1 -- 172.21.15.67:0/3184208615 shutdown_connections 2024-03-09T09:20:48.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.313+0000 7f94f3e00640 1 -- 172.21.15.67:0/3184208615 wait complete. 2024-03-09T09:20:48.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.316+0000 7f94f3e00640 1 Processor -- start 2024-03-09T09:20:48.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.316+0000 7f94f3e00640 1 -- start start 2024-03-09T09:20:48.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.317+0000 7f94f3e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94f406d5f0 0x7f94f41e2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.317+0000 7f94f3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f4150db0 0x7f94f41eb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.317+0000 7f94f3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94f4157b20 0x7f94f41ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f3e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40c3560 con 0x7f94f4150db0 2024-03-09T09:20:48.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f3e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40c33e0 con 0x7f94f4157b20 2024-03-09T09:20:48.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f3e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40c36e0 con 0x7f94f406d5f0 2024-03-09T09:20:48.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f4150db0 0x7f94f41eb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f4150db0 0x7f94f41eb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59068/0 (socket says 172.21.15.67:59068) 2024-03-09T09:20:48.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f2000640 1 -- 172.21.15.67:0/871903383 learned_addr learned my addr 172.21.15.67:0/871903383 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:48.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f3400640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94f4157b20 0x7f94f41ebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f2a00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94f406d5f0 0x7f94f41e2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f2000640 1 -- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94f406d5f0 msgr2=0x7f94f41e2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f2000640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94f406d5f0 0x7f94f41e2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f2000640 1 -- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94f4157b20 msgr2=0x7f94f41ebe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f2000640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94f4157b20 0x7f94f41ebe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f2000640 1 -- 172.21.15.67:0/871903383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f41eeed0 con 0x7f94f4150db0 2024-03-09T09:20:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.318+0000 7f94f2a00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94f406d5f0 0x7f94f41e2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:20:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.319+0000 7f94f2000640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f4150db0 0x7f94f41eb910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94e000d600 tx=0x7f94e000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.319+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0004240 con 0x7f94f4150db0 2024-03-09T09:20:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.319+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e00043e0 con 0x7f94f4150db0 2024-03-09T09:20:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.319+0000 7f94f3e00640 1 -- 172.21.15.67:0/871903383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f41ef170 con 0x7f94f4150db0 2024-03-09T09:20:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.319+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0003830 con 0x7f94f4150db0 2024-03-09T09:20:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.319+0000 7f94f3e00640 1 -- 172.21.15.67:0/871903383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f41ef730 con 0x7f94f4150db0 2024-03-09T09:20:48.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.321+0000 7f94f3e00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ef970 0x7f94f4152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.321+0000 7f94f3e00640 1 -- 172.21.15.67:0/871903383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f404f3c0 con 0x7f94f41ef970 2024-03-09T09:20:48.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.321+0000 7f94f2a00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ef970 0x7f94f4152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.321+0000 7f94f2a00640 1 -- 172.21.15.67:0/871903383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94f4150560 con 0x7f94f41ef970 2024-03-09T09:20:48.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.321+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f94e00039d0 con 0x7f94f4150db0 2024-03-09T09:20:48.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.322+0000 7f94f2a00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ef970 0x7f94f4152970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f94e400ccd0 tx=0x7f94e400a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.322+0000 7f94eaa00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f94d406b4d0 0x7f94d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.323+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f94e008a960 con 0x7f94f4150db0 2024-03-09T09:20:48.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.323+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400f620 con 0x7f94f41ef970 2024-03-09T09:20:48.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.323+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f94e400f7c0 con 0x7f94f41ef970 2024-03-09T09:20:48.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.323+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ef970 msgr2=0x7f94f4152970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.323+0000 7f94eaa00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ef970 0x7f94f4152970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f94e400ccd0 tx=0x7f94e400a5f0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.323+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ef970 msgr2=0x7f94f4152970 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:48.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.323+0000 7f94eaa00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ef970 0x7f94f4152970 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f94e400ccd0 tx=0x7f94e400a5f0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.323+0000 7f94f3400640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f94d406b4d0 0x7f94d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.324+0000 7f94f3400640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f94d406b4d0 0x7f94d406d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f94ec036000 tx=0x7f94ec03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.330+0000 7f94f3e00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ec390 0x7f94f4152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.330+0000 7f94f3e00640 1 -- 172.21.15.67:0/871903383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40c5380 con 0x7f94f41ec390 2024-03-09T09:20:48.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.330+0000 7f94f2a00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ec390 0x7f94f4152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.331+0000 7f94f2a00640 1 -- 172.21.15.67:0/871903383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f94f41e2b80 con 0x7f94f41ec390 2024-03-09T09:20:48.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.331+0000 7f94f2a00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ec390 0x7f94f4152eb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f94e400ccd0 tx=0x7f94e4004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e401d770 con 0x7f94f41ec390 2024-03-09T09:20:48.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5020 (secure 0 0 0) 0x7f94e401d770 con 0x7f94f41ec390 2024-03-09T09:20:48.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ec390 msgr2=0x7f94f4152eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94eaa00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ec390 0x7f94f4152eb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f94e400ccd0 tx=0x7f94e4004a70 comp rx=0 tx=0).stop 2024-03-09T09:20:48.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94eaa00640 1 -- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ec390 msgr2=0x7f94f4152eb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:48.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94eaa00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f41ec390 0x7f94f4152eb0 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f94e400ccd0 tx=0x7f94e4004a70 comp rx=0 tx=0).stop 2024-03-09T09:20:48.340 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:20:48.340 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:20:48.340 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:20:48.340 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:20:48.340 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 216, 2024-03-09T09:20:48.340 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:20:48.341 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:20:48.341 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:20:48.341 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:20:48.341 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.341 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:20:48.341 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:48.341 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:20:48.342 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:20:48.342 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:48.342 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:48.342 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:48.342 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:48.342 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:48.342 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:48.343 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:48.343 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:48.343 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:48.343 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:48.343 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:48.344 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:48.345 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:48.345 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:48.345 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.345 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:20:48.345 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:20:48.346 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:48.347 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:48.348 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:48.348 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.348 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:20:48.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.348 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:20:48.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.348 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:20:48.348 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:20:48.348 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:48.348 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.350 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:20:48.350 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:20:48.350 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:48.350 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:48.350 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:48.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.350 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:20:48.350 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:20:48.350 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:48.350 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.352 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:20:48.353 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:20:48.354 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:48.354 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:48.354 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:48.354 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.354 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.354 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:20:48.354 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:20:48.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.354 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.355 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.356 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.357 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:20:48.357 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.357 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.357 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:20:48.357 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:20:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94f3e00640 1 -- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f94d406b4d0 msgr2=0x7f94d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94f3e00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f94d406b4d0 0x7f94d406d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f94ec036000 tx=0x7f94ec03c040 comp rx=0 tx=0).stop 2024-03-09T09:20:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94f3e00640 1 -- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f4150db0 msgr2=0x7f94f41eb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94f3e00640 1 --2- 172.21.15.67:0/871903383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94f4150db0 0x7f94f41eb910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94e000d600 tx=0x7f94e000dac0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94f3400640 1 -- 172.21.15.67:0/871903383 reap_dead start 2024-03-09T09:20:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94f3400640 1 -- 172.21.15.67:0/871903383 reap_dead start 2024-03-09T09:20:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94f3e00640 1 -- 172.21.15.67:0/871903383 shutdown_connections 2024-03-09T09:20:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.332+0000 7f94f3e00640 1 -- 172.21.15.67:0/871903383 >> 172.21.15.67:0/871903383 conn(0x7f94f406c150 msgr2=0x7f94f40c73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.333+0000 7f94f3e00640 1 -- 172.21.15.67:0/871903383 shutdown_connections 2024-03-09T09:20:48.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.333+0000 7f94f3e00640 1 -- 172.21.15.67:0/871903383 wait complete. 2024-03-09T09:20:48.360 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:20:48.360 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:20:48.360 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:20:48.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.593+0000 7f84ff200640 1 Processor -- start 2024-03-09T09:20:48.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.593+0000 7f84ff200640 1 -- start start 2024-03-09T09:20:48.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.594+0000 7f84ff200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f84f8150140 0x7f84f806d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.594+0000 7f84ff200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f84f8152790 0x7f84f8071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.594+0000 7f84ff200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f806d940 con 0x7f84f80c2fa0 2024-03-09T09:20:48.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.594+0000 7f84ff200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f8071b20 con 0x7f84f8152790 2024-03-09T09:20:48.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.594+0000 7f84ff200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f806dc20 con 0x7f84f8150140 2024-03-09T09:20:48.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.595+0000 7f84fe800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f84f8152790 0x7f84f8071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.595+0000 7f84fd400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f84f8150140 0x7f84f806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.595+0000 7f84fe800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f84f8152790 0x7f84f8071fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59092/0 (socket says 172.21.15.67:59092) 2024-03-09T09:20:48.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.595+0000 7f84fe800640 1 -- 172.21.15.67:0/3075541446 learned_addr learned my addr 172.21.15.67:0/3075541446 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:48.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.595+0000 7f84fe800640 1 -- 172.21.15.67:0/3075541446 >> v2:172.21.15.134:3300/0 conn(0x7f84f8150140 msgr2=0x7f84f806d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:48.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.595+0000 7f84fe800640 1 --2- 172.21.15.67:0/3075541446 >> v2:172.21.15.134:3300/0 conn(0x7f84f8150140 0x7f84f806d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.596+0000 7f84fe800640 1 -- 172.21.15.67:0/3075541446 >> v1:172.21.15.134:6789/0 conn(0x7f84f80c2fa0 legacy=0x7f84f814fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.596+0000 7f84fe800640 1 -- 172.21.15.67:0/3075541446 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f806df00 con 0x7f84f8152790 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.598+0000 7f84fe800640 1 --2- 172.21.15.67:0/3075541446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f8152790 0x7f84f8071fb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84f80aa170 tx=0x7f84f402e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3d8f51be265b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.598+0000 7f84fca00640 1 -- 172.21.15.67:0/3075541446 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f402ea00 con 0x7f84f8152790 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.599+0000 7f84fca00640 1 -- 172.21.15.67:0/3075541446 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f402eba0 con 0x7f84f8152790 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.599+0000 7f84fca00640 1 -- 172.21.15.67:0/3075541446 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4009600 con 0x7f84f8152790 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.599+0000 7f84ff200640 1 -- 172.21.15.67:0/3075541446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f8152790 msgr2=0x7f84f8071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.599+0000 7f84ff200640 1 --2- 172.21.15.67:0/3075541446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f8152790 0x7f84f8071fb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84f80aa170 tx=0x7f84f402e060 comp rx=0 tx=0).stop 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.600+0000 7f84ff200640 1 -- 172.21.15.67:0/3075541446 shutdown_connections 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.600+0000 7f84ff200640 1 --2- 172.21.15.67:0/3075541446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f8152790 0x7f84f8071fb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.600+0000 7f84ff200640 1 --2- 172.21.15.67:0/3075541446 >> v2:172.21.15.134:3300/0 conn(0x7f84f8150140 0x7f84f806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.600+0000 7f84ff200640 1 -- 172.21.15.67:0/3075541446 >> 172.21.15.67:0/3075541446 conn(0x7f84f80c7220 msgr2=0x7f84f80c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:48.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.600+0000 7f84ff200640 1 -- 172.21.15.67:0/3075541446 shutdown_connections 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.600+0000 7f84ff200640 1 -- 172.21.15.67:0/3075541446 wait complete. 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.602+0000 7f84ff200640 1 Processor -- start 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.603+0000 7f84ff200640 1 -- start start 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.604+0000 7f84ff200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f80c2fa0 0x7f84f80c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.604+0000 7f84ff200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f8150140 0x7f84f8144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.604+0000 7f84ff200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f8152790 0x7f84f8144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.604+0000 7f84ff200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f8071340 con 0x7f84f8152790 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.604+0000 7f84ff200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f80711c0 con 0x7f84f8150140 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.604+0000 7f84ff200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f80c3380 con 0x7f84f80c2fa0 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.605+0000 7f84fe800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f8152790 0x7f84f8144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.605+0000 7f84fe800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f8152790 0x7f84f8144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59104/0 (socket says 172.21.15.67:59104) 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.605+0000 7f84fe800640 1 -- 172.21.15.67:0/1921844447 learned_addr learned my addr 172.21.15.67:0/1921844447 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.605+0000 7f84fd400640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f8150140 0x7f84f8144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.605+0000 7f84fde00640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f80c2fa0 0x7f84f80c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.605+0000 7f84fe800640 1 -- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f80c2fa0 msgr2=0x7f84f80c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.605+0000 7f84fe800640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84f80c2fa0 0x7f84f80c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.605+0000 7f84fe800640 1 -- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f8150140 msgr2=0x7f84f8144310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.605+0000 7f84fe800640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f8150140 0x7f84f8144310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.605+0000 7f84fe800640 1 -- 172.21.15.67:0/1921844447 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f8144d90 con 0x7f84f8152790 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.605+0000 7f84fd400640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f8150140 0x7f84f8144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.606+0000 7f84fe800640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f8152790 0x7f84f8144850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84f400d240 tx=0x7f84f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.606+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f402ea00 con 0x7f84f8152790 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.606+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f4045070 con 0x7f84f8152790 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.606+0000 7f84ff200640 1 -- 172.21.15.67:0/1921844447 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f814f4e0 con 0x7f84f8152790 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.606+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f400b720 con 0x7f84f8152790 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.606+0000 7f84ff200640 1 -- 172.21.15.67:0/1921844447 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f81eb950 con 0x7f84f8152790 2024-03-09T09:20:48.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.607+0000 7f84ff200640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f814eb60 0x7f84f81ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.607+0000 7f84ff200640 1 -- 172.21.15.67:0/1921844447 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f804f3c0 con 0x7f84f814eb60 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.608+0000 7f84fde00640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f814eb60 0x7f84f81ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.608+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f84f4036a40 con 0x7f84f8152790 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.609+0000 7f84fde00640 1 -- 172.21.15.67:0/1921844447 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84f80ad500 con 0x7f84f814eb60 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.609+0000 7f84fde00640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f814eb60 0x7f84f81ebc90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e4d0 tx=0x7f84ec00e990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.609+0000 7f84f2000640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f84dc06b4d0 0x7f84dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.610+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f84f40ad4f0 con 0x7f84f8152790 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.610+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec007240 con 0x7f84f814eb60 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.610+0000 7f84fd400640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f84dc06b4d0 0x7f84dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.611+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f84ec007240 con 0x7f84f814eb60 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.611+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f814eb60 msgr2=0x7f84f81ebc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.611+0000 7f84f2000640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f814eb60 0x7f84f81ebc90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e4d0 tx=0x7f84ec00e990 comp rx=0 tx=0).stop 2024-03-09T09:20:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.611+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f814eb60 msgr2=0x7f84f81ebc90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.611+0000 7f84f2000640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f814eb60 0x7f84f81ebc90 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e4d0 tx=0x7f84ec00e990 comp rx=0 tx=0).stop 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.612+0000 7f84fd400640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f84dc06b4d0 0x7f84dc06d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f84f80c4960 tx=0x7f84e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.619+0000 7f84ff200640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f81ee890 0x7f84f80c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.619+0000 7f84ff200640 1 -- 172.21.15.67:0/1921844447 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f8149090 con 0x7f84f81ee890 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.620+0000 7f84fde00640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f81ee890 0x7f84f80c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.621+0000 7f84fde00640 1 -- 172.21.15.67:0/1921844447 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f84f80c0e00 con 0x7f84f81ee890 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.622+0000 7f84fde00640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f81ee890 0x7f84f80c5020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e4d0 tx=0x7f84ec003760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.622+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec01e760 con 0x7f84f81ee890 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5045 (secure 0 0 0) 0x7f84ec028460 con 0x7f84f81ee890 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f81ee890 msgr2=0x7f84f80c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84f2000640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f81ee890 0x7f84f80c5020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e4d0 tx=0x7f84ec003760 comp rx=0 tx=0).stop 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84f2000640 1 -- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f81ee890 msgr2=0x7f84f80c5020 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:48.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84f2000640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f81ee890 0x7f84f80c5020 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e4d0 tx=0x7f84ec003760 comp rx=0 tx=0).stop 2024-03-09T09:20:48.683 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:20:48.683 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:20:48.683 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:20:48.683 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:20:48.683 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 216, 2024-03-09T09:20:48.683 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:20:48.683 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:20:48.683 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:20:48.683 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:20:48.683 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:48.684 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:48.685 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:48.686 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:48.687 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:48.688 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:20:48.689 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:20:48.690 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:20:48.692 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.693 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.694 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.695 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.696 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.696 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:48.696 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.696 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.696 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:20:48.696 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:20:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84ff200640 1 -- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f84dc06b4d0 msgr2=0x7f84dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84ff200640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f84dc06b4d0 0x7f84dc06d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f84f80c4960 tx=0x7f84e8008040 comp rx=0 tx=0).stop 2024-03-09T09:20:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84ff200640 1 -- 172.21.15.67:0/1921844447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f8152790 msgr2=0x7f84f8144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84ff200640 1 --2- 172.21.15.67:0/1921844447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84f8152790 0x7f84f8144850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84f400d240 tx=0x7f84f4004a40 comp rx=0 tx=0).stop 2024-03-09T09:20:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84fe800640 1 -- 172.21.15.67:0/1921844447 reap_dead start 2024-03-09T09:20:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84fe800640 1 -- 172.21.15.67:0/1921844447 reap_dead start 2024-03-09T09:20:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84ff200640 1 -- 172.21.15.67:0/1921844447 shutdown_connections 2024-03-09T09:20:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.623+0000 7f84ff200640 1 -- 172.21.15.67:0/1921844447 >> 172.21.15.67:0/1921844447 conn(0x7f84f80c7220 msgr2=0x7f84f814b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.624+0000 7f84ff200640 1 -- 172.21.15.67:0/1921844447 shutdown_connections 2024-03-09T09:20:48.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.624+0000 7f84ff200640 1 -- 172.21.15.67:0/1921844447 wait complete. 2024-03-09T09:20:48.698 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:20:48.698 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:20:48.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:20:48.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.938+0000 7f77a7200640 1 Processor -- start 2024-03-09T09:20:48.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.939+0000 7f77a7200640 1 -- start start 2024-03-09T09:20:48.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.940+0000 7f77a7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f77a00c5e00 0x7f77a00acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.940+0000 7f77a7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c61e0 0x7f77a00c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.940+0000 7f77a7200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a00ad7c0 con 0x7f77a0156660 2024-03-09T09:20:48.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.940+0000 7f77a7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a006d740 con 0x7f77a00c5e00 2024-03-09T09:20:48.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.940+0000 7f77a7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a00c6660 con 0x7f77a00c61e0 2024-03-09T09:20:48.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.940+0000 7f77a6800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f77a0156660 0x7f77a006c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:37664/0 (socket says 172.21.15.67:37664) 2024-03-09T09:20:48.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.940+0000 7f77a6800640 1 -- 172.21.15.67:0/1331004288 learned_addr learned my addr 172.21.15.67:0/1331004288 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:20:48.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.940+0000 7f77a5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f77a00c5e00 0x7f77a00acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.941+0000 7f77a5400640 1 --2- 172.21.15.67:0/1331004288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c61e0 0x7f77a00c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.941+0000 7f77a5e00640 1 -- 172.21.15.67:0/1331004288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c61e0 msgr2=0x7f77a00c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.941+0000 7f77a5e00640 1 --2- 172.21.15.67:0/1331004288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c61e0 0x7f77a00c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.941+0000 7f77a5e00640 1 -- 172.21.15.67:0/1331004288 >> v1:172.21.15.67:6789/0 conn(0x7f77a0156660 legacy=0x7f77a006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.941+0000 7f77a5e00640 1 -- 172.21.15.67:0/1331004288 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0156af0 con 0x7f77a00c5e00 2024-03-09T09:20:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.941+0000 7f77a4a00640 1 -- 172.21.15.67:0/1331004288 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843427483 0 0) 0x7f77a00ad7c0 con 0x7f77a0156660 2024-03-09T09:20:48.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.942+0000 7f77a5e00640 1 --2- 172.21.15.67:0/1331004288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a00c5e00 0x7f77a00acf00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f77940097a0 tx=0x7f779402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549a96c70fd8c7e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.944+0000 7f77a4a00640 1 -- 172.21.15.67:0/1331004288 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794035020 con 0x7f77a00c5e00 2024-03-09T09:20:48.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.944+0000 7f77a4a00640 1 -- 172.21.15.67:0/1331004288 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794031810 con 0x7f77a00c5e00 2024-03-09T09:20:48.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.944+0000 7f77a4a00640 1 -- 172.21.15.67:0/1331004288 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794031be0 con 0x7f77a00c5e00 2024-03-09T09:20:48.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.945+0000 7f77a7200640 1 -- 172.21.15.67:0/1331004288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a00c5e00 msgr2=0x7f77a00acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.945+0000 7f77a7200640 1 --2- 172.21.15.67:0/1331004288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a00c5e00 0x7f77a00acf00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f77940097a0 tx=0x7f779402b8b0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.945+0000 7f77a7200640 1 -- 172.21.15.67:0/1331004288 shutdown_connections 2024-03-09T09:20:48.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.945+0000 7f77a7200640 1 --2- 172.21.15.67:0/1331004288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c61e0 0x7f77a00c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.945+0000 7f77a7200640 1 --2- 172.21.15.67:0/1331004288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a00c5e00 0x7f77a00acf00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.945+0000 7f77a7200640 1 -- 172.21.15.67:0/1331004288 >> 172.21.15.67:0/1331004288 conn(0x7f77a00cb480 msgr2=0x7f77a00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:48.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.945+0000 7f77a7200640 1 -- 172.21.15.67:0/1331004288 shutdown_connections 2024-03-09T09:20:48.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.945+0000 7f77a7200640 1 -- 172.21.15.67:0/1331004288 wait complete. 2024-03-09T09:20:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.947+0000 7f77a7200640 1 Processor -- start 2024-03-09T09:20:48.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.947+0000 7f77a7200640 1 -- start start 2024-03-09T09:20:48.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.948+0000 7f77a7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c5e00 0x7f77a0159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.948+0000 7f77a7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a00c61e0 0x7f77a00c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.948+0000 7f77a7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77a0156660 0x7f77a00c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.948+0000 7f77a7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a006bb00 con 0x7f77a00c61e0 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.948+0000 7f77a7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a0153680 con 0x7f77a0156660 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.948+0000 7f77a7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a006bde0 con 0x7f77a00c5e00 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.948+0000 7f77a5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a00c61e0 0x7f77a00c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.948+0000 7f77a5400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a00c61e0 0x7f77a00c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46190/0 (socket says 172.21.15.67:46190) 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.948+0000 7f77a5400640 1 -- 172.21.15.67:0/212996425 learned_addr learned my addr 172.21.15.67:0/212996425 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.949+0000 7f77a5e00640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c5e00 0x7f77a0159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.949+0000 7f77a6800640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77a0156660 0x7f77a00c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.949+0000 7f77a5400640 1 -- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c5e00 msgr2=0x7f77a0159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.949+0000 7f77a5400640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c5e00 0x7f77a0159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.949+0000 7f77a5400640 1 -- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77a0156660 msgr2=0x7f77a00c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.949+0000 7f77a5400640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77a0156660 0x7f77a00c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.949+0000 7f77a5400640 1 -- 172.21.15.67:0/212996425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a00c4ec0 con 0x7f77a00c61e0 2024-03-09T09:20:48.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.949+0000 7f77a5e00640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c5e00 0x7f77a0159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:20:48.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.949+0000 7f77a6800640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77a0156660 0x7f77a00c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:20:48.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.949+0000 7f77a5400640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a00c61e0 0x7f77a00c4440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f779000d800 tx=0x7f779000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.949+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779000b6c0 con 0x7f77a00c61e0 2024-03-09T09:20:48.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.950+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7790019070 con 0x7f77a00c61e0 2024-03-09T09:20:48.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.950+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779000ba80 con 0x7f77a00c61e0 2024-03-09T09:20:48.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.950+0000 7f77a7200640 1 -- 172.21.15.67:0/212996425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a0151920 con 0x7f77a00c61e0 2024-03-09T09:20:48.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.950+0000 7f77a7200640 1 -- 172.21.15.67:0/212996425 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a0159ed0 con 0x7f77a00c61e0 2024-03-09T09:20:48.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.951+0000 7f77a7200640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c1b00 0x7f77a01ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.951+0000 7f77a7200640 1 -- 172.21.15.67:0/212996425 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a0151330 con 0x7f77a00c1b00 2024-03-09T09:20:48.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.952+0000 7f77a5e00640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c1b00 0x7f77a01ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.952+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f779000bc20 con 0x7f77a00c61e0 2024-03-09T09:20:48.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.953+0000 7f77a5e00640 1 -- 172.21.15.67:0/212996425 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77a00c98c0 con 0x7f77a00c1b00 2024-03-09T09:20:48.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.953+0000 7f779a000640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f778406b4d0 0x7f778406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.953+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f779008ab40 con 0x7f77a00c61e0 2024-03-09T09:20:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.953+0000 7f77a5e00640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c1b00 0x7f77a01ee9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f77a006b1f0 tx=0x7f77940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.954+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794006e10 con 0x7f77a00c1b00 2024-03-09T09:20:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.954+0000 7f77a6800640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f778406b4d0 0x7f778406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.955+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f77940093c0 con 0x7f77a00c1b00 2024-03-09T09:20:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.955+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c1b00 msgr2=0x7f77a01ee9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.955+0000 7f779a000640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c1b00 0x7f77a01ee9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f77a006b1f0 tx=0x7f77940046c0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.955+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c1b00 msgr2=0x7f77a01ee9a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.955+0000 7f779a000640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a00c1b00 0x7f77a01ee9a0 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f77a006b1f0 tx=0x7f77940046c0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.955+0000 7f77a6800640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f778406b4d0 0x7f778406d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f77a00c41c0 tx=0x7f779c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.969+0000 7f77a7200640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a01eeee0 0x7f77a01ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:48.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.969+0000 7f77a7200640 1 -- 172.21.15.67:0/212996425 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a01ef800 con 0x7f77a01eeee0 2024-03-09T09:20:48.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.970+0000 7f77a5e00640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a01eeee0 0x7f77a01ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:48.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.971+0000 7f77a5e00640 1 -- 172.21.15.67:0/212996425 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f77a00c24c0 con 0x7f77a01eeee0 2024-03-09T09:20:48.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.971+0000 7f77a5e00640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a01eeee0 0x7f77a01ef2c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f77940097a0 tx=0x7f7794031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:48.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.972+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779404a880 con 0x7f77a01eeee0 2024-03-09T09:20:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5045 (secure 0 0 0) 0x7f779404a880 con 0x7f77a01eeee0 2024-03-09T09:20:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a01eeee0 msgr2=0x7f77a01ef2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f779a000640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a01eeee0 0x7f77a01ef2c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f77940097a0 tx=0x7f7794031000 comp rx=0 tx=0).stop 2024-03-09T09:20:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f779a000640 1 -- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a01eeee0 msgr2=0x7f77a01ef2c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:20:48.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f779a000640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77a01eeee0 0x7f77a01ef2c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.980 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:20:48.980 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:20:48.980 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:20:48.980 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:20:48.980 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 216, 2024-03-09T09:20:48.980 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:20:48.980 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:20:48.980 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:20:48.980 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:20:48.980 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.980 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:48.981 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:48.982 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:20:48.983 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:20:48.984 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:48.985 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:48.986 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.987 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:20:48.988 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.990 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.991 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:20:48.992 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:20:48.993 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:20:48.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:20:48.993 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:20:48.993 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:20:48.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f77a7200640 1 -- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f778406b4d0 msgr2=0x7f778406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f77a7200640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f778406b4d0 0x7f778406d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f77a00c41c0 tx=0x7f779c009040 comp rx=0 tx=0).stop 2024-03-09T09:20:48.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f77a7200640 1 -- 172.21.15.67:0/212996425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a00c61e0 msgr2=0x7f77a00c4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:48.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f77a7200640 1 --2- 172.21.15.67:0/212996425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a00c61e0 0x7f77a00c4440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f779000d800 tx=0x7f779000dcc0 comp rx=0 tx=0).stop 2024-03-09T09:20:48.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f77a6800640 1 -- 172.21.15.67:0/212996425 reap_dead start 2024-03-09T09:20:48.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f77a6800640 1 -- 172.21.15.67:0/212996425 reap_dead start 2024-03-09T09:20:48.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f77a7200640 1 -- 172.21.15.67:0/212996425 shutdown_connections 2024-03-09T09:20:48.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f77a7200640 1 -- 172.21.15.67:0/212996425 >> 172.21.15.67:0/212996425 conn(0x7f77a00cb480 msgr2=0x7f77a0152b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:48.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.973+0000 7f77a7200640 1 -- 172.21.15.67:0/212996425 shutdown_connections 2024-03-09T09:20:48.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:48.974+0000 7f77a7200640 1 -- 172.21.15.67:0/212996425 wait complete. 2024-03-09T09:20:48.998 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:20:48.999 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-03-09T09:20:48.999 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:20:48.999 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-03-09T09:20:48.999 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:20:48.999 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-09T09:20:48.999 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:20:49.027 INFO:tasks.ceph.mon.a:Stopped 2024-03-09T09:20:49.027 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:20:49.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:20:49.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.285+0000 7f5d62000640 1 Processor -- start 2024-03-09T09:20:49.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.285+0000 7f5d62000640 1 -- start start 2024-03-09T09:20:49.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.286+0000 7f5d62000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5d5c06e8d0 0x7f5d5c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:49.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.286+0000 7f5d62000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5d5c159570 0x7f5d5c159a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:49.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.286+0000 7f5d62000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5c155dc0 con 0x7f5d5c150080 2024-03-09T09:20:49.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.286+0000 7f5d62000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5c156c90 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.286+0000 7f5d62000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5c156080 con 0x7f5d5c159570 2024-03-09T09:20:49.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.287+0000 7f5d5b400640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f5d5c150080 legacy=0x7f5d5c0c83f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:20:49.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.287+0000 7f5d52a00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f5d5c06e8d0 msgr2=0x7f5d5c06bab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:20:49.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.287+0000 7f5d52a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5d5c06e8d0 0x7f5d5c06bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:49.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.287+0000 7f5d5be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5d5c159570 0x7f5d5c159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:49.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.287+0000 7f5d5be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5d5c159570 0x7f5d5c159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:35144/0 (socket says 172.21.15.67:35144) 2024-03-09T09:20:49.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.287+0000 7f5d5be00640 1 -- 172.21.15.67:0/932353336 learned_addr learned my addr 172.21.15.67:0/932353336 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:49.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.288+0000 7f5d5be00640 1 -- 172.21.15.67:0/932353336 >> v2:172.21.15.67:3300/0 conn(0x7f5d5c06e8d0 msgr2=0x7f5d5c06bab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:49.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.288+0000 7f5d5be00640 1 --2- 172.21.15.67:0/932353336 >> v2:172.21.15.67:3300/0 conn(0x7f5d5c06e8d0 0x7f5d5c06bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:49.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.288+0000 7f5d5be00640 1 -- 172.21.15.67:0/932353336 >> v1:172.21.15.67:6789/0 conn(0x7f5d5c150080 legacy=0x7f5d5c0c83f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:49.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.288+0000 7f5d5be00640 1 -- 172.21.15.67:0/932353336 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c156360 con 0x7f5d5c159570 2024-03-09T09:20:49.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.289+0000 7f5d5be00640 1 --2- 172.21.15.67:0/932353336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c159570 0x7f5d5c159a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0716e0 tx=0x7f5d5402b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68ef3e1341ed5e5f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:49.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.290+0000 7f5d5aa00640 1 -- 172.21.15.67:0/932353336 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54035020 con 0x7f5d5c159570 2024-03-09T09:20:49.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.290+0000 7f5d5aa00640 1 -- 172.21.15.67:0/932353336 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d54031800 con 0x7f5d5c159570 2024-03-09T09:20:49.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.290+0000 7f5d5aa00640 1 -- 172.21.15.67:0/932353336 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54031bd0 con 0x7f5d5c159570 2024-03-09T09:20:49.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.290+0000 7f5d62000640 1 -- 172.21.15.67:0/932353336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c159570 msgr2=0x7f5d5c159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:49.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.290+0000 7f5d62000640 1 --2- 172.21.15.67:0/932353336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c159570 0x7f5d5c159a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0716e0 tx=0x7f5d5402b8a0 comp rx=0 tx=0).stop 2024-03-09T09:20:49.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.290+0000 7f5d62000640 1 -- 172.21.15.67:0/932353336 shutdown_connections 2024-03-09T09:20:49.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.290+0000 7f5d62000640 1 --2- 172.21.15.67:0/932353336 >> v2:172.21.15.67:3300/0 conn(0x7f5d5c06e8d0 0x7f5d5c06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:49.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.290+0000 7f5d62000640 1 --2- 172.21.15.67:0/932353336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c159570 0x7f5d5c159a00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:49.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.290+0000 7f5d62000640 1 -- 172.21.15.67:0/932353336 >> 172.21.15.67:0/932353336 conn(0x7f5d5c0aa890 msgr2=0x7f5d5c0ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:49.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.291+0000 7f5d62000640 1 -- 172.21.15.67:0/932353336 shutdown_connections 2024-03-09T09:20:49.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.291+0000 7f5d62000640 1 -- 172.21.15.67:0/932353336 wait complete. 2024-03-09T09:20:49.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.292+0000 7f5d62000640 1 Processor -- start 2024-03-09T09:20:49.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.292+0000 7f5d62000640 1 -- start start 2024-03-09T09:20:49.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.293+0000 7f5d62000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c06e8d0 0x7f5d5c14d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:49.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.293+0000 7f5d62000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d5c150080 0x7f5d5c0c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:49.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.293+0000 7f5d62000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d5c159570 0x7f5d5c149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:49.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.293+0000 7f5d62000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c150460 con 0x7f5d5c159570 2024-03-09T09:20:49.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.293+0000 7f5d62000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c156920 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.293+0000 7f5d62000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0c8b00 con 0x7f5d5c150080 2024-03-09T09:20:49.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.293+0000 7f5d5be00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d5c159570 msgr2=0x7f5d5c149d20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:20:49.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.293+0000 7f5d5be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d5c159570 0x7f5d5c149d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:49.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.294+0000 7f5d5b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c06e8d0 0x7f5d5c14d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:49.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.294+0000 7f5d5b400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c06e8d0 0x7f5d5c14d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:35146/0 (socket says 172.21.15.67:35146) 2024-03-09T09:20:49.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.294+0000 7f5d5b400640 1 -- 172.21.15.67:0/3395339391 learned_addr learned my addr 172.21.15.67:0/3395339391 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:49.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.294+0000 7f5d52a00640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d5c150080 0x7f5d5c0c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:49.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.295+0000 7f5d5b400640 1 -- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d5c150080 msgr2=0x7f5d5c0c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:49.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.295+0000 7f5d5b400640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d5c150080 0x7f5d5c0c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:49.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.295+0000 7f5d5b400640 1 -- 172.21.15.67:0/3395339391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d5c159570 msgr2=0x7f5d5c149d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:20:49.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.295+0000 7f5d5b400640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d5c159570 0x7f5d5c149d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:49.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.295+0000 7f5d5b400640 1 -- 172.21.15.67:0/3395339391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c1eb700 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.295+0000 7f5d5b400640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c06e8d0 0x7f5d5c14d410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00ba10 tx=0x7f5d4c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:49.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.296+0000 7f5d53e00640 1 -- 172.21.15.67:0/3395339391 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00cb20 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.296+0000 7f5d53e00640 1 -- 172.21.15.67:0/3395339391 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c010070 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.296+0000 7f5d62000640 1 -- 172.21.15.67:0/3395339391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c1eb9a0 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.296+0000 7f5d53e00640 1 -- 172.21.15.67:0/3395339391 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c015470 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.296+0000 7f5d62000640 1 -- 172.21.15.67:0/3395339391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c1ebf60 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.297+0000 7f5d62000640 1 -- 172.21.15.67:0/3395339391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c1ec160 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.301+0000 7f5d53e00640 1 -- 172.21.15.67:0/3395339391 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5d4c01e3f0 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.301+0000 7f5d53e00640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d3806b4b0 0x7f5d3806d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:49.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.301+0000 7f5d53e00640 1 -- 172.21.15.67:0/3395339391 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f5d4c08aa50 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.302+0000 7f5d52a00640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d3806b4b0 0x7f5d3806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:49.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.303+0000 7f5d52a00640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d3806b4b0 0x7f5d3806d970 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c14b290 tx=0x7f5d44009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:49.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.312+0000 7f5d53e00640 1 -- 172.21.15.67:0/3395339391 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5d5c1ec160 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.659+0000 7f5d62000640 1 -- 172.21.15.67:0/3395339391 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5d5c154680 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.661+0000 7f5d53e00640 1 -- 172.21.15.67:0/3395339391 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f5d4c059650 con 0x7f5d5c06e8d0 2024-03-09T09:20:49.667 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":216,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:20:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.663+0000 7f5d62000640 1 -- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d3806b4b0 msgr2=0x7f5d3806d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.663+0000 7f5d62000640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d3806b4b0 0x7f5d3806d970 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c14b290 tx=0x7f5d44009110 comp rx=0 tx=0).stop 2024-03-09T09:20:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.663+0000 7f5d62000640 1 -- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c06e8d0 msgr2=0x7f5d5c14d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:49.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.663+0000 7f5d62000640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c06e8d0 0x7f5d5c14d410 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00ba10 tx=0x7f5d4c00be10 comp rx=0 tx=0).stop 2024-03-09T09:20:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.664+0000 7f5d62000640 1 -- 172.21.15.67:0/3395339391 shutdown_connections 2024-03-09T09:20:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.664+0000 7f5d62000640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5d3806b4b0 0x7f5d3806d970 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.664+0000 7f5d62000640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5d5c159570 0x7f5d5c149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.664+0000 7f5d62000640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5d5c150080 0x7f5d5c0c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.664+0000 7f5d62000640 1 --2- 172.21.15.67:0/3395339391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c06e8d0 0x7f5d5c14d410 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.664+0000 7f5d62000640 1 -- 172.21.15.67:0/3395339391 >> 172.21.15.67:0/3395339391 conn(0x7f5d5c0aa890 msgr2=0x7f5d5c0c4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.664+0000 7f5d62000640 1 -- 172.21.15.67:0/3395339391 shutdown_connections 2024-03-09T09:20:49.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:49.664+0000 7f5d62000640 1 -- 172.21.15.67:0/3395339391 wait complete. 2024-03-09T09:20:49.912 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:20:49.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:49.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 54 -eq 48 ']' 2024-03-09T09:20:49.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.54 2024-03-09T09:20:49.913 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.54 2024-03-09T09:20:49.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.54 2024-03-09T09:20:49.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.54 2024-03-09T09:20:49.918 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:20:51.198 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:20:51.199 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:20:51.199 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 46 -eq 48 ']' 2024-03-09T09:20:51.199 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.46 2024-03-09T09:20:51.199 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.46 2024-03-09T09:20:51.199 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.46 2024-03-09T09:20:51.199 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.46 2024-03-09T09:20:51.202 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:20:52.691 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:20:52.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.909+0000 7fc735a00640 1 Processor -- start 2024-03-09T09:20:52.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.909+0000 7fc735a00640 1 -- start start 2024-03-09T09:20:52.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.910+0000 7fc735a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 0x7fc7300b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:52.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.911+0000 7fc735a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc7300cadb0 0x7fc7300b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:52.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.911+0000 7fc735a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc730157000 con 0x7fc7300c5e40 2024-03-09T09:20:52.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.911+0000 7fc735a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7300ae430 con 0x7fc7300cadb0 2024-03-09T09:20:52.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.911+0000 7fc735a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300ae6f0 con 0x7fc730153300 2024-03-09T09:20:52.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.911+0000 7fc72fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 0x7fc7300b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:52.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.911+0000 7fc72fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 0x7fc7300b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:45594/0 (socket says 172.21.15.67:45594) 2024-03-09T09:20:52.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.911+0000 7fc72fe00640 1 -- 172.21.15.67:0/1360069494 learned_addr learned my addr 172.21.15.67:0/1360069494 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:52.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.911+0000 7fc735000640 1 --2- 172.21.15.67:0/1360069494 >> v2:172.21.15.134:3300/0 conn(0x7fc7300cadb0 0x7fc7300b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:52.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.912+0000 7fc72fe00640 1 -- 172.21.15.67:0/1360069494 >> v2:172.21.15.134:3300/0 conn(0x7fc7300cadb0 msgr2=0x7fc7300b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:52.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.912+0000 7fc72fe00640 1 --2- 172.21.15.67:0/1360069494 >> v2:172.21.15.134:3300/0 conn(0x7fc7300cadb0 0x7fc7300b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:52.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.912+0000 7fc72fe00640 1 -- 172.21.15.67:0/1360069494 >> v1:172.21.15.134:6789/0 conn(0x7fc7300c5e40 legacy=0x7fc730156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:52.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.912+0000 7fc72fe00640 1 -- 172.21.15.67:0/1360069494 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730155b10 con 0x7fc730153300 2024-03-09T09:20:52.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.912+0000 7fc735000640 1 --2- 172.21.15.67:0/1360069494 >> v2:172.21.15.134:3300/0 conn(0x7fc7300cadb0 0x7fc7300b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:20:52.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.914+0000 7fc72fe00640 1 --2- 172.21.15.67:0/1360069494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 0x7fc7300b0000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc720009e50 tx=0x7fc72002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a7cc69481d7c27 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:52.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.915+0000 7fc72ea00640 1 -- 172.21.15.67:0/1360069494 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72002cb00 con 0x7fc730153300 2024-03-09T09:20:52.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.915+0000 7fc72ea00640 1 -- 172.21.15.67:0/1360069494 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72002cca0 con 0x7fc730153300 2024-03-09T09:20:52.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.915+0000 7fc72ea00640 1 -- 172.21.15.67:0/1360069494 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720034610 con 0x7fc730153300 2024-03-09T09:20:52.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.915+0000 7fc735a00640 1 -- 172.21.15.67:0/1360069494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 msgr2=0x7fc7300b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:52.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.915+0000 7fc735a00640 1 --2- 172.21.15.67:0/1360069494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 0x7fc7300b0000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc720009e50 tx=0x7fc72002bf60 comp rx=0 tx=0).stop 2024-03-09T09:20:52.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.916+0000 7fc735a00640 1 -- 172.21.15.67:0/1360069494 shutdown_connections 2024-03-09T09:20:52.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.916+0000 7fc735a00640 1 --2- 172.21.15.67:0/1360069494 >> v2:172.21.15.134:3300/0 conn(0x7fc7300cadb0 0x7fc7300b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:52.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.916+0000 7fc735a00640 1 --2- 172.21.15.67:0/1360069494 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 0x7fc7300b0000 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:52.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.916+0000 7fc735a00640 1 -- 172.21.15.67:0/1360069494 >> 172.21.15.67:0/1360069494 conn(0x7fc7300732d0 msgr2=0x7fc7300ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:52.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.916+0000 7fc735a00640 1 -- 172.21.15.67:0/1360069494 shutdown_connections 2024-03-09T09:20:52.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.916+0000 7fc735a00640 1 -- 172.21.15.67:0/1360069494 wait complete. 2024-03-09T09:20:52.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.917+0000 7fc735a00640 1 Processor -- start 2024-03-09T09:20:52.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.918+0000 7fc735a00640 1 -- start start 2024-03-09T09:20:52.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.918+0000 7fc735a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7300c5e40 0x7fc7301ec7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:52.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.918+0000 7fc735a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7300cadb0 0x7fc7300c03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:52.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.918+0000 7fc735a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 0x7fc7300aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:52.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.918+0000 7fc735a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300c59e0 con 0x7fc7300cadb0 2024-03-09T09:20:52.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.918+0000 7fc735a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc730159e90 con 0x7fc7300c5e40 2024-03-09T09:20:52.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.918+0000 7fc735a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc730156720 con 0x7fc730153300 2024-03-09T09:20:52.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.919+0000 7fc72f400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7300cadb0 msgr2=0x7fc7300c03a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:20:52.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.919+0000 7fc72f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7300cadb0 0x7fc7300c03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:52.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.919+0000 7fc72fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7300c5e40 0x7fc7301ec7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:52.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.919+0000 7fc735000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 0x7fc7300aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:52.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.919+0000 7fc72fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7300c5e40 0x7fc7301ec7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:35160/0 (socket says 172.21.15.67:35160) 2024-03-09T09:20:52.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.919+0000 7fc72fe00640 1 -- 172.21.15.67:0/4101365777 learned_addr learned my addr 172.21.15.67:0/4101365777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:52.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.920+0000 7fc72fe00640 1 -- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 msgr2=0x7fc7300aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:52.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.920+0000 7fc72fe00640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 0x7fc7300aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:52.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.920+0000 7fc72fe00640 1 -- 172.21.15.67:0/4101365777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7300cadb0 msgr2=0x7fc7300c03a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:20:52.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.920+0000 7fc72fe00640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7300cadb0 0x7fc7300c03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:52.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.920+0000 7fc72fe00640 1 -- 172.21.15.67:0/4101365777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7301ebd50 con 0x7fc7300c5e40 2024-03-09T09:20:52.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.920+0000 7fc735000640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 0x7fc7300aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:20:52.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.920+0000 7fc72fe00640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7300c5e40 0x7fc7301ec7c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc720036040 tx=0x7fc720004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:52.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.921+0000 7fc727e00640 1 -- 172.21.15.67:0/4101365777 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720043020 con 0x7fc7300c5e40 2024-03-09T09:20:52.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.921+0000 7fc727e00640 1 -- 172.21.15.67:0/4101365777 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72002ce50 con 0x7fc7300c5e40 2024-03-09T09:20:52.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.921+0000 7fc727e00640 1 -- 172.21.15.67:0/4101365777 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720009510 con 0x7fc7300c5e40 2024-03-09T09:20:52.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.921+0000 7fc735a00640 1 -- 172.21.15.67:0/4101365777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73006d580 con 0x7fc7300c5e40 2024-03-09T09:20:52.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.921+0000 7fc735a00640 1 -- 172.21.15.67:0/4101365777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7301ebfc0 con 0x7fc7300c5e40 2024-03-09T09:20:52.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.922+0000 7fc735a00640 1 -- 172.21.15.67:0/4101365777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73004f3c0 con 0x7fc7300c5e40 2024-03-09T09:20:52.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.924+0000 7fc727e00640 1 -- 172.21.15.67:0/4101365777 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc720046020 con 0x7fc7300c5e40 2024-03-09T09:20:52.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.925+0000 7fc727e00640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc70c06b480 0x7fc70c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:52.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.925+0000 7fc727e00640 1 -- 172.21.15.67:0/4101365777 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc7200acd50 con 0x7fc7300c5e40 2024-03-09T09:20:52.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.925+0000 7fc72f400640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc70c06b480 0x7fc70c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:52.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.927+0000 7fc72f400640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc70c06b480 0x7fc70c06d940 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc7301e9a80 tx=0x7fc7180079f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:52.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:52.940+0000 7fc727e00640 1 -- 172.21.15.67:0/4101365777 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc72004d030 con 0x7fc7300c5e40 2024-03-09T09:20:53.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.299+0000 7fc735a00640 1 -- 172.21.15.67:0/4101365777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc7300cab90 con 0x7fc7300c5e40 2024-03-09T09:20:53.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.300+0000 7fc727e00640 1 -- 172.21.15.67:0/4101365777 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fc72007b950 con 0x7fc7300c5e40 2024-03-09T09:20:53.306 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":216,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:20:53.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 -- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc70c06b480 msgr2=0x7fc70c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:53.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc70c06b480 0x7fc70c06d940 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc7301e9a80 tx=0x7fc7180079f0 comp rx=0 tx=0).stop 2024-03-09T09:20:53.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 -- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7300c5e40 msgr2=0x7fc7301ec7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:53.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7300c5e40 0x7fc7301ec7c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc720036040 tx=0x7fc720004a40 comp rx=0 tx=0).stop 2024-03-09T09:20:53.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 -- 172.21.15.67:0/4101365777 shutdown_connections 2024-03-09T09:20:53.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc70c06b480 0x7fc70c06d940 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:53.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc730153300 0x7fc7300aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:53.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7300cadb0 0x7fc7300c03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:53.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 --2- 172.21.15.67:0/4101365777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7300c5e40 0x7fc7301ec7c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:53.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 -- 172.21.15.67:0/4101365777 >> 172.21.15.67:0/4101365777 conn(0x7fc7300732d0 msgr2=0x7fc730151940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:53.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 -- 172.21.15.67:0/4101365777 shutdown_connections 2024-03-09T09:20:53.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:53.303+0000 7fc735a00640 1 -- 172.21.15.67:0/4101365777 wait complete. 2024-03-09T09:20:55.259 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:20:55.259 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:20:55.260 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 47 -eq 48 ']' 2024-03-09T09:20:55.260 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.47 2024-03-09T09:20:55.260 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.47 2024-03-09T09:20:55.260 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.47 2024-03-09T09:20:55.260 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.47 2024-03-09T09:20:55.263 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:20:56.266 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:20:56.267 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:20:56.267 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 55 -eq 48 ']' 2024-03-09T09:20:56.267 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.55 2024-03-09T09:20:56.267 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.55 2024-03-09T09:20:56.267 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.55 2024-03-09T09:20:56.267 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.55 2024-03-09T09:20:56.272 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:20:56.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:20:56.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.530+0000 7f6d1f000640 1 Processor -- start 2024-03-09T09:20:56.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.530+0000 7f6d1f000640 1 -- start start 2024-03-09T09:20:56.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.532+0000 7f6d1f000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6d180c84e0 0x7f6d1806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:56.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.532+0000 7f6d1f000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6d1806f4f0 0x7f6d180c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:56.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.532+0000 7f6d1f000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 0x7f6d180c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:56.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.532+0000 7f6d1f000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d18155c60 con 0x7f6d180c84e0 2024-03-09T09:20:56.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.532+0000 7f6d1f000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d18155f40 con 0x7f6d1806f4f0 2024-03-09T09:20:56.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.532+0000 7f6d1f000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d18156220 con 0x7f6d1806bd30 2024-03-09T09:20:56.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.532+0000 7f6d1dc00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f6d180c84e0 msgr2=0x7f6d1806b7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:20:56.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.532+0000 7f6d1dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6d180c84e0 0x7f6d1806b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.533+0000 7f6d1d200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6d1806f4f0 0x7f6d180c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.533+0000 7f6d1e600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 0x7f6d180c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.533+0000 7f6d1d200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6d1806f4f0 0x7f6d180c53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:35174/0 (socket says 172.21.15.67:35174) 2024-03-09T09:20:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.533+0000 7f6d1d200640 1 -- 172.21.15.67:0/2096120519 learned_addr learned my addr 172.21.15.67:0/2096120519 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.533+0000 7f6d1d200640 1 -- 172.21.15.67:0/2096120519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 msgr2=0x7f6d180c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:20:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.533+0000 7f6d1d200640 1 --2- 172.21.15.67:0/2096120519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 0x7f6d180c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.533+0000 7f6d1d200640 1 -- 172.21.15.67:0/2096120519 >> v2:172.21.15.67:3300/0 conn(0x7f6d180c84e0 msgr2=0x7f6d1806b7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:20:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.533+0000 7f6d1d200640 1 --2- 172.21.15.67:0/2096120519 >> v2:172.21.15.67:3300/0 conn(0x7f6d180c84e0 0x7f6d1806b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:56.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.533+0000 7f6d1d200640 1 -- 172.21.15.67:0/2096120519 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d18156500 con 0x7f6d1806f4f0 2024-03-09T09:20:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.535+0000 7f6d1d200640 1 --2- 172.21.15.67:0/2096120519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d1806f4f0 0x7f6d180c53b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009870 tx=0x7f6d0402b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af50dcc98e5dafc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:56.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.535+0000 7f6d17e00640 1 -- 172.21.15.67:0/2096120519 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0402e070 con 0x7f6d1806f4f0 2024-03-09T09:20:56.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.535+0000 7f6d17e00640 1 -- 172.21.15.67:0/2096120519 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d04037070 con 0x7f6d1806f4f0 2024-03-09T09:20:56.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.535+0000 7f6d17e00640 1 -- 172.21.15.67:0/2096120519 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04031a00 con 0x7f6d1806f4f0 2024-03-09T09:20:56.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.536+0000 7f6d1f000640 1 -- 172.21.15.67:0/2096120519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d1806f4f0 msgr2=0x7f6d180c53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:56.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.536+0000 7f6d1f000640 1 --2- 172.21.15.67:0/2096120519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d1806f4f0 0x7f6d180c53b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009870 tx=0x7f6d0402b880 comp rx=0 tx=0).stop 2024-03-09T09:20:56.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.536+0000 7f6d1f000640 1 -- 172.21.15.67:0/2096120519 shutdown_connections 2024-03-09T09:20:56.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.536+0000 7f6d1f000640 1 --2- 172.21.15.67:0/2096120519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 0x7f6d180c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:56.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.536+0000 7f6d1f000640 1 --2- 172.21.15.67:0/2096120519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d1806f4f0 0x7f6d180c53b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:56.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.536+0000 7f6d1f000640 1 --2- 172.21.15.67:0/2096120519 >> v2:172.21.15.67:3300/0 conn(0x7f6d180c84e0 0x7f6d1806b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:56.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.536+0000 7f6d1f000640 1 -- 172.21.15.67:0/2096120519 >> 172.21.15.67:0/2096120519 conn(0x7f6d1806d650 msgr2=0x7f6d1806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:56.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.536+0000 7f6d1f000640 1 -- 172.21.15.67:0/2096120519 shutdown_connections 2024-03-09T09:20:56.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.536+0000 7f6d1f000640 1 -- 172.21.15.67:0/2096120519 wait complete. 2024-03-09T09:20:56.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.539+0000 7f6d1f000640 1 Processor -- start 2024-03-09T09:20:56.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.539+0000 7f6d1f000640 1 -- start start 2024-03-09T09:20:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.540+0000 7f6d1f000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 0x7f6d18157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.540+0000 7f6d1f000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d1806f4f0 0x7f6d181f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.540+0000 7f6d1f000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d180c84e0 0x7f6d181503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.540+0000 7f6d1f000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d180c58f0 con 0x7f6d1806f4f0 2024-03-09T09:20:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.540+0000 7f6d1f000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d180c8ad0 con 0x7f6d180c84e0 2024-03-09T09:20:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.540+0000 7f6d1f000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d180c5a70 con 0x7f6d1806bd30 2024-03-09T09:20:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.540+0000 7f6d1d200640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d1806f4f0 msgr2=0x7f6d181f37a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:20:56.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.540+0000 7f6d1d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d1806f4f0 0x7f6d181f37a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:20:56.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.541+0000 7f6d1dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 0x7f6d18157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:56.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.541+0000 7f6d1e600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d180c84e0 0x7f6d181503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:56.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.541+0000 7f6d1dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 0x7f6d18157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:45620/0 (socket says 172.21.15.67:45620) 2024-03-09T09:20:56.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.541+0000 7f6d1dc00640 1 -- 172.21.15.67:0/1684777031 learned_addr learned my addr 172.21.15.67:0/1684777031 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:20:56.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.542+0000 7f6d1e600640 1 -- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 msgr2=0x7f6d18157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:56.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.542+0000 7f6d1e600640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 0x7f6d18157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:56.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.542+0000 7f6d1e600640 1 -- 172.21.15.67:0/1684777031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d1806f4f0 msgr2=0x7f6d181f37a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:20:56.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.542+0000 7f6d1e600640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d1806f4f0 0x7f6d181f37a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:56.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.542+0000 7f6d1e600640 1 -- 172.21.15.67:0/1684777031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d181f3ce0 con 0x7f6d180c84e0 2024-03-09T09:20:56.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.542+0000 7f6d1dc00640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 0x7f6d18157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:20:56.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.542+0000 7f6d1e600640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d180c84e0 0x7f6d181503f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000eb70 tx=0x7f6d1000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:56.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.543+0000 7f6d15600640 1 -- 172.21.15.67:0/1684777031 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10018020 con 0x7f6d180c84e0 2024-03-09T09:20:56.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.543+0000 7f6d15600640 1 -- 172.21.15.67:0/1684777031 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1000cd20 con 0x7f6d180c84e0 2024-03-09T09:20:56.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.543+0000 7f6d1f000640 1 -- 172.21.15.67:0/1684777031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d18158240 con 0x7f6d180c84e0 2024-03-09T09:20:56.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.543+0000 7f6d15600640 1 -- 172.21.15.67:0/1684777031 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d100146c0 con 0x7f6d180c84e0 2024-03-09T09:20:56.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.543+0000 7f6d1f000640 1 -- 172.21.15.67:0/1684777031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d18158aa0 con 0x7f6d180c84e0 2024-03-09T09:20:56.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.544+0000 7f6d1f000640 1 -- 172.21.15.67:0/1684777031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1804f3c0 con 0x7f6d180c84e0 2024-03-09T09:20:56.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.547+0000 7f6d15600640 1 -- 172.21.15.67:0/1684777031 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6d10005d10 con 0x7f6d180c84e0 2024-03-09T09:20:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.548+0000 7f6d15600640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cf806b480 0x7f6cf806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:20:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.548+0000 7f6d15600640 1 -- 172.21.15.67:0/1684777031 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6d1008b770 con 0x7f6d180c84e0 2024-03-09T09:20:56.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.549+0000 7f6d1dc00640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cf806b480 0x7f6cf806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:20:56.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.550+0000 7f6d1dc00640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cf806b480 0x7f6cf806d940 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009810 tx=0x7f6d0c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:20:56.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.558+0000 7f6d15600640 1 -- 172.21.15.67:0/1684777031 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6d10022030 con 0x7f6d180c84e0 2024-03-09T09:20:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.838+0000 7f6d1f000640 1 -- 172.21.15.67:0/1684777031 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6d18155510 con 0x7f6d180c84e0 2024-03-09T09:20:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.839+0000 7f6d15600640 1 -- 172.21.15.67:0/1684777031 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f6d10010040 con 0x7f6d180c84e0 2024-03-09T09:20:56.845 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":216,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:20:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.842+0000 7f6d1f000640 1 -- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cf806b480 msgr2=0x7f6cf806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.842+0000 7f6d1f000640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cf806b480 0x7f6cf806d940 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009810 tx=0x7f6d0c009190 comp rx=0 tx=0).stop 2024-03-09T09:20:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.843+0000 7f6d1f000640 1 -- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d180c84e0 msgr2=0x7f6d181503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:20:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.843+0000 7f6d1f000640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d180c84e0 0x7f6d181503f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000eb70 tx=0x7f6d1000c490 comp rx=0 tx=0).stop 2024-03-09T09:20:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.843+0000 7f6d1f000640 1 -- 172.21.15.67:0/1684777031 shutdown_connections 2024-03-09T09:20:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.843+0000 7f6d1f000640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6cf806b480 0x7f6cf806d940 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.843+0000 7f6d1f000640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d180c84e0 0x7f6d181503f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.843+0000 7f6d1f000640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d1806f4f0 0x7f6d181f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.843+0000 7f6d1f000640 1 --2- 172.21.15.67:0/1684777031 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d1806bd30 0x7f6d18157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:20:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.843+0000 7f6d1f000640 1 -- 172.21.15.67:0/1684777031 >> 172.21.15.67:0/1684777031 conn(0x7f6d1806d650 msgr2=0x7f6d1806b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:20:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.843+0000 7f6d1f000640 1 -- 172.21.15.67:0/1684777031 shutdown_connections 2024-03-09T09:20:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:20:56.843+0000 7f6d1f000640 1 -- 172.21.15.67:0/1684777031 wait complete. 2024-03-09T09:20:59.598 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:20:59.608 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:20:59.877 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:21:00.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.091+0000 7f15c6400640 1 Processor -- start 2024-03-09T09:21:00.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.091+0000 7f15c6400640 1 -- start start 2024-03-09T09:21:00.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.092+0000 7f15c6400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f15c00ac730 0x7f15c0071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:00.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.092+0000 7f15c6400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:00.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.092+0000 7f15c6400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c01581a0 con 0x7f15c00c22a0 2024-03-09T09:21:00.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.092+0000 7f15c6400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c0070ee0 con 0x7f15c00ac730 2024-03-09T09:21:00.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.092+0000 7f15c6400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c0154d00 con 0x7f15c0157d60 2024-03-09T09:21:00.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.092+0000 7f15bea00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f15c00c22a0 legacy=0x7f15c0157650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:21:00.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.092+0000 7f15bf400640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f15c00ac730 msgr2=0x7f15c0071190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:00.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.092+0000 7f15bf400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f15c00ac730 0x7f15c0071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:00.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.092+0000 7f15bfe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:00.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.093+0000 7f15bfe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:42112/0 (socket says 172.21.15.67:42112) 2024-03-09T09:21:00.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.093+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 learned_addr learned my addr 172.21.15.67:0/2742166672 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:00.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.093+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 msgr2=0x7f15c00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:21:00.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.093+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 msgr2=0x7f15c00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:00.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.093+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:00.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.093+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.293+0000 7f15bea00640 1 -- 172.21.15.67:0/2742166672 >> v1:172.21.15.67:6789/0 conn(0x7f15c00c22a0 legacy=0x7f15c0157650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:21:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.293+0000 7f15bf400640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.67:3300/0 conn(0x7f15c00ac730 msgr2=0x7f15c0071190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.293+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.67:3300/0 conn(0x7f15c00ac730 0x7f15c0071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:21:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.294+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:00.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.295+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 msgr2=0x7f15c00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:00.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.295+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 msgr2=0x7f15c00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:00.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.295+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:00.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.295+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:21:00.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.694+0000 7f15bf400640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.67:3300/0 conn(0x7f15c00ac730 msgr2=0x7f15c0071190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:00.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.694+0000 7f15bea00640 1 -- 172.21.15.67:0/2742166672 >> v1:172.21.15.67:6789/0 conn(0x7f15c00c22a0 legacy=0x7f15c0157650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:21:00.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.694+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.67:3300/0 conn(0x7f15c00ac730 0x7f15c0071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:21:00.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.696+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:00.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.697+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 msgr2=0x7f15c00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:00.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.697+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 msgr2=0x7f15c00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:00.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.697+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:00.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:00.697+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:21:01.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:01.495+0000 7f15bf400640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.67:3300/0 conn(0x7f15c00ac730 msgr2=0x7f15c0071190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:01.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:01.495+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.67:3300/0 conn(0x7f15c00ac730 0x7f15c0071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:21:01.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:01.495+0000 7f15bea00640 1 -- 172.21.15.67:0/2742166672 >> v1:172.21.15.67:6789/0 conn(0x7f15c00c22a0 legacy=0x7f15c0157650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:21:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:01.499+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:01.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:01.499+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 msgr2=0x7f15c00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:01.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:01.499+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 msgr2=0x7f15c00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:01.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:01.499+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:01.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:01.499+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:21:03.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.092+0000 7f15bcc00640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 msgr2=0x7f15c00acb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:03.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.092+0000 7f15bcc00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.134:3300/0 conn(0x7f15c0157d60 0x7f15c00acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:03.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.092+0000 7f15bcc00640 1 -- 172.21.15.67:0/2742166672 >> v2:172.21.15.67:3300/0 conn(0x7f15c00ac730 msgr2=0x7f15c0071190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:03.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.092+0000 7f15bcc00640 1 --2- 172.21.15.67:0/2742166672 >> v2:172.21.15.67:3300/0 conn(0x7f15c00ac730 0x7f15c0071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:03.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.092+0000 7f15bcc00640 1 -- 172.21.15.67:0/2742166672 >> v1:172.21.15.67:6789/0 conn(0x7f15c00c22a0 legacy=0x7f15c0157650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:03.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.092+0000 7f15bcc00640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 0x7f15a0005ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:03.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.092+0000 7f15bcc00640 1 -- 172.21.15.67:0/2742166672 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c0153d40 con 0x7f15a0006100 2024-03-09T09:21:03.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.092+0000 7f15bcc00640 1 -- 172.21.15.67:0/2742166672 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0070ee0 con 0x7f15a0003670 2024-03-09T09:21:03.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.093+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 0x7f15a0005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:03.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.094+0000 7f15bf400640 1 -- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 msgr2=0x7f15a0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:03.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.094+0000 7f15bf400640 1 -- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 msgr2=0x7f15a0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:03.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.094+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 0x7f15a0005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:03.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.094+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 0x7f15a0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:03.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.295+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 0x7f15a0005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:03.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.296+0000 7f15bf400640 1 -- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 msgr2=0x7f15a0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:03.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.296+0000 7f15bf400640 1 -- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 msgr2=0x7f15a0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:03.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.296+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 0x7f15a0005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:03.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.296+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 0x7f15a0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:21:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.697+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 0x7f15a0005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.698+0000 7f15bf400640 1 -- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 msgr2=0x7f15a0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.698+0000 7f15bf400640 1 -- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 msgr2=0x7f15a0005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.698+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 0x7f15a0005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.698+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 0x7f15a0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:21:03.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.973+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836352050 0 0) 0x7f15c0153d40 con 0x7f15a0006100 2024-03-09T09:21:03.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.973+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15a4003520 con 0x7f15a0006100 2024-03-09T09:21:03.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.982+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 445067479 0 0) 0x7f15a4003520 con 0x7f15a0006100 2024-03-09T09:21:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.982+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 msgr2=0x7f15a0005ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.982+0000 7f15be000640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a0003670 0x7f15a0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.982+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c01581a0 con 0x7f15a0006100 2024-03-09T09:21:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.982+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f15ac0041c0 con 0x7f15a0006100 2024-03-09T09:21:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.983+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 >> v1:172.21.15.134:6789/0 conn(0x7f15a0006100 legacy=0x7f15a00085c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.983+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 reap_dead start 2024-03-09T09:21:03.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.983+0000 7f15be000640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a40039b0 0x7f15a4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:03.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.983+0000 7f15be000640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a4004800 0x7f15a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:03.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.983+0000 7f15be000640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4005210 0x7f15a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:03.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.983+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0070ee0 con 0x7f15a40039b0 2024-03-09T09:21:03.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.983+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a400c140 con 0x7f15a4005210 2024-03-09T09:21:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.983+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a400c340 con 0x7f15a4004800 2024-03-09T09:21:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.983+0000 7f15bf400640 1 -- 172.21.15.67:0/2742166672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a40039b0 msgr2=0x7f15a4003db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.983+0000 7f15bf400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a40039b0 0x7f15a4003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.984+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4005210 0x7f15a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:03.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.984+0000 7f15bea00640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a4004800 0x7f15a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:03.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.985+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a4004800 msgr2=0x7f15a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:03.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.985+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a4004800 0x7f15a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:03.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.985+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a40039b0 msgr2=0x7f15a4003db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:03.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.985+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a40039b0 0x7f15a4003db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.985+0000 7f15bfe00640 1 -- 172.21.15.67:0/2742166672 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a4003520 con 0x7f15a4005210 2024-03-09T09:21:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.987+0000 7f15bfe00640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4005210 0x7f15a400b9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f15c0071ef0 tx=0x7f15a4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af49d826f4f73144 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.988+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b803a070 con 0x7f15a4005210 2024-03-09T09:21:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.988+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b8035410 con 0x7f15a4005210 2024-03-09T09:21:03.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.988+0000 7f15be000640 1 -- 172.21.15.67:0/2742166672 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b8009700 con 0x7f15a4005210 2024-03-09T09:21:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.989+0000 7f15c6400640 1 -- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4005210 msgr2=0x7f15a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.989+0000 7f15c6400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4005210 0x7f15a400b9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f15c0071ef0 tx=0x7f15a4005830 comp rx=0 tx=0).stop 2024-03-09T09:21:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.989+0000 7f15c6400640 1 -- 172.21.15.67:0/2742166672 shutdown_connections 2024-03-09T09:21:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.989+0000 7f15c6400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4005210 0x7f15a400b9c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:03.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.989+0000 7f15c6400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a4004800 0x7f15a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.989+0000 7f15c6400640 1 --2- 172.21.15.67:0/2742166672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a40039b0 0x7f15a4003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.989+0000 7f15c6400640 1 -- 172.21.15.67:0/2742166672 >> 172.21.15.67:0/2742166672 conn(0x7f15c00c7370 msgr2=0x7f15c00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:04.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.989+0000 7f15c6400640 1 -- 172.21.15.67:0/2742166672 shutdown_connections 2024-03-09T09:21:04.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.990+0000 7f15c6400640 1 -- 172.21.15.67:0/2742166672 wait complete. 2024-03-09T09:21:04.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.995+0000 7f15c6400640 1 Processor -- start 2024-03-09T09:21:04.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.995+0000 7f15c6400640 1 -- start start 2024-03-09T09:21:04.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.997+0000 7f15c6400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a40039b0 0x7f15c01ebd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.997+0000 7f15c6400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4004800 0x7f15c0072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.997+0000 7f15c6400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a4005210 0x7f15c014e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.997+0000 7f15c6400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0070c00 con 0x7f15a4005210 2024-03-09T09:21:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.997+0000 7f15c6400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0070920 con 0x7f15a4004800 2024-03-09T09:21:04.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.997+0000 7f15c6400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c0151610 con 0x7f15a40039b0 2024-03-09T09:21:04.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.997+0000 7f15bfe00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a4005210 msgr2=0x7f15c014e690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:04.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.998+0000 7f15bfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a4005210 0x7f15c014e690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:04.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.998+0000 7f15bea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4004800 0x7f15c0072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.998+0000 7f15bf400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a40039b0 0x7f15c01ebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.998+0000 7f15bea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4004800 0x7f15c0072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:42176/0 (socket says 172.21.15.67:42176) 2024-03-09T09:21:04.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.998+0000 7f15bea00640 1 -- 172.21.15.67:0/2476024552 learned_addr learned my addr 172.21.15.67:0/2476024552 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:04.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.999+0000 7f15bea00640 1 -- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a40039b0 msgr2=0x7f15c01ebd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.999+0000 7f15bea00640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a40039b0 0x7f15c01ebd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.999+0000 7f15bea00640 1 -- 172.21.15.67:0/2476024552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a4005210 msgr2=0x7f15c014e690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:21:04.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.999+0000 7f15bea00640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a4005210 0x7f15c014e690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.999+0000 7f15bea00640 1 -- 172.21.15.67:0/2476024552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c0151c50 con 0x7f15a4004800 2024-03-09T09:21:04.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:03.999+0000 7f15bea00640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4004800 0x7f15c0072ba0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f15ac003d00 tx=0x7f15ac00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.000+0000 7f15b7400640 1 -- 172.21.15.67:0/2476024552 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00c8a0 con 0x7f15a4004800 2024-03-09T09:21:04.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.000+0000 7f15b7400640 1 -- 172.21.15.67:0/2476024552 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac01a070 con 0x7f15a4004800 2024-03-09T09:21:04.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.000+0000 7f15b7400640 1 -- 172.21.15.67:0/2476024552 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00cc60 con 0x7f15a4004800 2024-03-09T09:21:04.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.000+0000 7f15c6400640 1 -- 172.21.15.67:0/2476024552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c01ec2a0 con 0x7f15a4004800 2024-03-09T09:21:04.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.000+0000 7f15c6400640 1 -- 172.21.15.67:0/2476024552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c00aba90 con 0x7f15a4004800 2024-03-09T09:21:04.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.002+0000 7f15c6400640 1 -- 172.21.15.67:0/2476024552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c0071410 con 0x7f15a4004800 2024-03-09T09:21:04.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.004+0000 7f15b7400640 1 -- 172.21.15.67:0/2476024552 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f15ac015430 con 0x7f15a4004800 2024-03-09T09:21:04.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.005+0000 7f15b7400640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15a006bb10 0x7f15a006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.005+0000 7f15b7400640 1 -- 172.21.15.67:0/2476024552 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f15ac08a6a0 con 0x7f15a4004800 2024-03-09T09:21:04.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.006+0000 7f15bf400640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15a006bb10 0x7f15a006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.007+0000 7f15bf400640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15a006bb10 0x7f15a006dfd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f15b0003890 tx=0x7f15b0003b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.018+0000 7f15b7400640 1 -- 172.21.15.67:0/2476024552 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f15c0071410 con 0x7f15a4004800 2024-03-09T09:21:04.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.330+0000 7f15c6400640 1 -- 172.21.15.67:0/2476024552 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f15c0075a00 con 0x7f15a4004800 2024-03-09T09:21:04.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.331+0000 7f15b7400640 1 -- 172.21.15.67:0/2476024552 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f15c01ec2a0 con 0x7f15a4004800 2024-03-09T09:21:04.337 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":218,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:21:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 -- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15a006bb10 msgr2=0x7f15a006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15a006bb10 0x7f15a006dfd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f15b0003890 tx=0x7f15b0003b50 comp rx=0 tx=0).stop 2024-03-09T09:21:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 -- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4004800 msgr2=0x7f15c0072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4004800 0x7f15c0072ba0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f15ac003d00 tx=0x7f15ac00ecd0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 -- 172.21.15.67:0/2476024552 shutdown_connections 2024-03-09T09:21:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f15a006bb10 0x7f15a006dfd0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a4005210 0x7f15c014e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15a4004800 0x7f15c0072ba0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 --2- 172.21.15.67:0/2476024552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15a40039b0 0x7f15c01ebd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 -- 172.21.15.67:0/2476024552 >> 172.21.15.67:0/2476024552 conn(0x7f15c00c7370 msgr2=0x7f15c00c0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 -- 172.21.15.67:0/2476024552 shutdown_connections 2024-03-09T09:21:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.334+0000 7f15c6400640 1 -- 172.21.15.67:0/2476024552 wait complete. 2024-03-09T09:21:04.361 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:21:04.362 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:21:04.362 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:21:04.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.562+0000 7fbb49c00640 1 Processor -- start 2024-03-09T09:21:04.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.562+0000 7fbb49c00640 1 -- start start 2024-03-09T09:21:04.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.564+0000 7fbb49c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbb440c3e20 0x7fbb440c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.564+0000 7fbb49c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb440c35f0 0x7fbb4406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.564+0000 7fbb49c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb44152c10 con 0x7fbb440c6720 2024-03-09T09:21:04.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.564+0000 7fbb49c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb440c3160 con 0x7fbb440c3e20 2024-03-09T09:21:04.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.564+0000 7fbb49c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb440c2b10 con 0x7fbb440c35f0 2024-03-09T09:21:04.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.564+0000 7fbb3be00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fbb440c3e20 msgr2=0x7fbb440c5890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:04.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.564+0000 7fbb3be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbb440c3e20 0x7fbb440c5890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:04.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.564+0000 7fbb43e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fbb440c6720 0x7fbb4406cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:34250/0 (socket says 172.21.15.67:34250) 2024-03-09T09:21:04.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.564+0000 7fbb43e00640 1 -- 172.21.15.67:0/3672331693 learned_addr learned my addr 172.21.15.67:0/3672331693 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:21:04.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.564+0000 7fbb49200640 1 --2- 172.21.15.67:0/3672331693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb440c35f0 0x7fbb4406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.565+0000 7fbb49200640 1 -- 172.21.15.67:0/3672331693 >> v2:172.21.15.67:3300/0 conn(0x7fbb440c3e20 msgr2=0x7fbb440c5890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:04.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.565+0000 7fbb49200640 1 --2- 172.21.15.67:0/3672331693 >> v2:172.21.15.67:3300/0 conn(0x7fbb440c3e20 0x7fbb440c5890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.565+0000 7fbb49200640 1 -- 172.21.15.67:0/3672331693 >> v1:172.21.15.134:6789/0 conn(0x7fbb440c6720 legacy=0x7fbb4406cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.565+0000 7fbb43400640 1 -- 172.21.15.67:0/3672331693 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283976348 0 0) 0x7fbb44152c10 con 0x7fbb440c6720 2024-03-09T09:21:04.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.565+0000 7fbb49200640 1 -- 172.21.15.67:0/3672331693 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb440c2df0 con 0x7fbb440c35f0 2024-03-09T09:21:04.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.566+0000 7fbb49200640 1 --2- 172.21.15.67:0/3672331693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb440c35f0 0x7fbb4406e6c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009d90 tx=0x7fbb3c02d060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fa6b6a355a7ce10 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.567+0000 7fbb43400640 1 -- 172.21.15.67:0/3672331693 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c03a070 con 0x7fbb440c35f0 2024-03-09T09:21:04.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.567+0000 7fbb43400640 1 -- 172.21.15.67:0/3672331693 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c02de10 con 0x7fbb440c35f0 2024-03-09T09:21:04.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.567+0000 7fbb43400640 1 -- 172.21.15.67:0/3672331693 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c035660 con 0x7fbb440c35f0 2024-03-09T09:21:04.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.568+0000 7fbb49c00640 1 -- 172.21.15.67:0/3672331693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb440c35f0 msgr2=0x7fbb4406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.568+0000 7fbb49c00640 1 --2- 172.21.15.67:0/3672331693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb440c35f0 0x7fbb4406e6c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009d90 tx=0x7fbb3c02d060 comp rx=0 tx=0).stop 2024-03-09T09:21:04.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.568+0000 7fbb49c00640 1 -- 172.21.15.67:0/3672331693 shutdown_connections 2024-03-09T09:21:04.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.568+0000 7fbb49c00640 1 --2- 172.21.15.67:0/3672331693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb440c35f0 0x7fbb4406e6c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.568+0000 7fbb49c00640 1 --2- 172.21.15.67:0/3672331693 >> v2:172.21.15.67:3300/0 conn(0x7fbb440c3e20 0x7fbb440c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.568+0000 7fbb49c00640 1 -- 172.21.15.67:0/3672331693 >> 172.21.15.67:0/3672331693 conn(0x7fbb4406ba00 msgr2=0x7fbb440a9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:04.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.568+0000 7fbb49c00640 1 -- 172.21.15.67:0/3672331693 shutdown_connections 2024-03-09T09:21:04.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.568+0000 7fbb49c00640 1 -- 172.21.15.67:0/3672331693 wait complete. 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.571+0000 7fbb49c00640 1 Processor -- start 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.571+0000 7fbb49c00640 1 -- start start 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.572+0000 7fbb49c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440c35f0 0x7fbb440c1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.572+0000 7fbb49c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb440c3e20 0x7fbb4414aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.572+0000 7fbb49c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb440c6720 0x7fbb4414b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.572+0000 7fbb49c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb440c3c00 con 0x7fbb440c6720 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.572+0000 7fbb49c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb440c3a80 con 0x7fbb440c35f0 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.572+0000 7fbb49c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4406ec00 con 0x7fbb440c3e20 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.573+0000 7fbb49200640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb440c6720 msgr2=0x7fbb4414b030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.573+0000 7fbb49200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb440c6720 0x7fbb4414b030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.573+0000 7fbb43e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440c35f0 0x7fbb440c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.573+0000 7fbb43e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440c35f0 0x7fbb440c1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:42186/0 (socket says 172.21.15.67:42186) 2024-03-09T09:21:04.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.573+0000 7fbb43e00640 1 -- 172.21.15.67:0/2209831505 learned_addr learned my addr 172.21.15.67:0/2209831505 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:04.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.573+0000 7fbb3be00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb440c3e20 0x7fbb4414aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.574+0000 7fbb43e00640 1 -- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb440c3e20 msgr2=0x7fbb4414aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.574+0000 7fbb43e00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb440c3e20 0x7fbb4414aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.574+0000 7fbb43e00640 1 -- 172.21.15.67:0/2209831505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb440c6720 msgr2=0x7fbb4414b030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:21:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.574+0000 7fbb43e00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb440c6720 0x7fbb4414b030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.574+0000 7fbb43e00640 1 -- 172.21.15.67:0/2209831505 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb441eeb70 con 0x7fbb440c35f0 2024-03-09T09:21:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.574+0000 7fbb43e00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440c35f0 0x7fbb440c1030 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400c430 tx=0x7fbb3400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.575+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb340074e0 con 0x7fbb440c35f0 2024-03-09T09:21:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.575+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb340047e0 con 0x7fbb440c35f0 2024-03-09T09:21:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.575+0000 7fbb49c00640 1 -- 172.21.15.67:0/2209831505 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb441eee50 con 0x7fbb440c35f0 2024-03-09T09:21:04.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.575+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34004bd0 con 0x7fbb440c35f0 2024-03-09T09:21:04.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.575+0000 7fbb49c00640 1 -- 172.21.15.67:0/2209831505 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb441ef410 con 0x7fbb440c35f0 2024-03-09T09:21:04.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.576+0000 7fbb49c00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb4414b570 0x7fbb44145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.576+0000 7fbb49c00640 1 -- 172.21.15.67:0/2209831505 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb441ef740 con 0x7fbb4414b570 2024-03-09T09:21:04.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.577+0000 7fbb3be00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb4414b570 0x7fbb44145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.578+0000 7fbb3be00640 1 -- 172.21.15.67:0/2209831505 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb44075a00 con 0x7fbb4414b570 2024-03-09T09:21:04.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.578+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbb34011070 con 0x7fbb440c35f0 2024-03-09T09:21:04.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.579+0000 7fbb3be00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb4414b570 0x7fbb44145bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c003980 tx=0x7fbb2c009aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.579+0000 7fbb40c00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb2006b480 0x7fbb2006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.579+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbb3408a1e0 con 0x7fbb440c35f0 2024-03-09T09:21:04.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.580+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c016070 con 0x7fbb4414b570 2024-03-09T09:21:04.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.580+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fbb2c005a40 con 0x7fbb4414b570 2024-03-09T09:21:04.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.580+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb4414b570 msgr2=0x7fbb44145bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.580+0000 7fbb40c00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb4414b570 0x7fbb44145bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c003980 tx=0x7fbb2c009aa0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.580+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb4414b570 msgr2=0x7fbb44145bd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:04.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.580+0000 7fbb40c00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb4414b570 0x7fbb44145bd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.580+0000 7fbb49200640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb2006b480 0x7fbb2006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.581+0000 7fbb49200640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb2006b480 0x7fbb2006d940 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c037000 tx=0x7fbb3c00d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.588+0000 7fbb49c00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440a73c0 0x7fbb44146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.588+0000 7fbb49c00640 1 -- 172.21.15.67:0/2209831505 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4404f3c0 con 0x7fbb440a73c0 2024-03-09T09:21:04.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.589+0000 7fbb3be00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440a73c0 0x7fbb44146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.590+0000 7fbb3be00640 1 -- 172.21.15.67:0/2209831505 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbb44067400 con 0x7fbb440a73c0 2024-03-09T09:21:04.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.591+0000 7fbb3be00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440a73c0 0x7fbb44146110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009fd0 tx=0x7fbb2c0059d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.591+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c016050 con 0x7fbb440a73c0 2024-03-09T09:21:04.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7fbb2c006780 con 0x7fbb440a73c0 2024-03-09T09:21:04.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440a73c0 msgr2=0x7fbb44146110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb40c00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440a73c0 0x7fbb44146110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009fd0 tx=0x7fbb2c0059d0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb40c00640 1 -- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440a73c0 msgr2=0x7fbb44146110 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:04.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb40c00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440a73c0 0x7fbb44146110 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009fd0 tx=0x7fbb2c0059d0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.599 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:21:04.599 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:21:04.599 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:21:04.599 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:21:04.599 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 218, 2024-03-09T09:21:04.599 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:21:04.599 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:21:04.599 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:21:04.599 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.600 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:21:04.600 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:04.600 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:21:04.600 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:21:04.600 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:04.600 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:04.600 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:04.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:04.600 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:04.600 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:04.601 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:04.601 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:04.601 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:04.601 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:04.601 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:04.601 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.601 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:21:04.601 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:21:04.601 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:04.601 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:04.601 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:04.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:04.602 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:04.602 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:04.602 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:04.602 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:04.602 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:04.602 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:04.602 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:04.602 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.603 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:21:04.603 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:21:04.603 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:21:04.603 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:21:04.603 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:21:04.603 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:21:04.604 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:04.604 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:04.604 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:21:04.604 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:21:04.604 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:21:04.604 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:21:04.604 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:21:04.604 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:21:04.604 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:21:04.604 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:21:04.608 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:21:04.609 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:04.610 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:04.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:04.610 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:21:04.610 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:04.610 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:21:04.610 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.610 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.611 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.611 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:21:04.611 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:21:04.611 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:04.611 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:04.611 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:04.611 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.611 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.612 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:21:04.613 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.613 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.613 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:21:04.613 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.613 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:04.613 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.613 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:21:04.613 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.613 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.613 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:21:04.614 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.615 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:04.615 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.615 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:04.615 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.615 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.615 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:21:04.615 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:21:04.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb49c00640 1 -- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb2006b480 msgr2=0x7fbb2006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb49c00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbb2006b480 0x7fbb2006d940 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c037000 tx=0x7fbb3c00d040 comp rx=0 tx=0).stop 2024-03-09T09:21:04.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb49c00640 1 -- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440c35f0 msgr2=0x7fbb440c1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb49c00640 1 --2- 172.21.15.67:0/2209831505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb440c35f0 0x7fbb440c1030 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400c430 tx=0x7fbb3400c830 comp rx=0 tx=0).stop 2024-03-09T09:21:04.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb49200640 1 -- 172.21.15.67:0/2209831505 reap_dead start 2024-03-09T09:21:04.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb49200640 1 -- 172.21.15.67:0/2209831505 reap_dead start 2024-03-09T09:21:04.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb49c00640 1 -- 172.21.15.67:0/2209831505 shutdown_connections 2024-03-09T09:21:04.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb49c00640 1 -- 172.21.15.67:0/2209831505 >> 172.21.15.67:0/2209831505 conn(0x7fbb4406ba00 msgr2=0x7fbb4414eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:04.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb49c00640 1 -- 172.21.15.67:0/2209831505 shutdown_connections 2024-03-09T09:21:04.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.592+0000 7fbb49c00640 1 -- 172.21.15.67:0/2209831505 wait complete. 2024-03-09T09:21:04.619 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:21:04.619 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:21:04.619 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:21:04.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.809+0000 7fb4d1800640 1 Processor -- start 2024-03-09T09:21:04.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.809+0000 7fb4d1800640 1 -- start start 2024-03-09T09:21:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.811+0000 7fb4d1800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb4cc14fd90 0x7fb4cc156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.811+0000 7fb4d1800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4cc0c9380 0x7fb4cc0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.811+0000 7fb4d1800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc156bc0 con 0x7fb4cc0ad440 2024-03-09T09:21:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.811+0000 7fb4d1800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc06ba60 con 0x7fb4cc0c9380 2024-03-09T09:21:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.811+0000 7fb4d1800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc0c9890 con 0x7fb4cc14fd90 2024-03-09T09:21:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.811+0000 7fb4cb400640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fb4cc0c9380 msgr2=0x7fb4cc0acf00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:04.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.811+0000 7fb4cb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4cc0c9380 0x7fb4cc0acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.812+0000 7fb4cbe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb4cc14fd90 0x7fb4cc156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.812+0000 7fb4cbe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb4cc14fd90 0x7fb4cc156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:42192/0 (socket says 172.21.15.67:42192) 2024-03-09T09:21:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.812+0000 7fb4cbe00640 1 -- 172.21.15.67:0/1437248678 learned_addr learned my addr 172.21.15.67:0/1437248678 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.812+0000 7fb4cbe00640 1 -- 172.21.15.67:0/1437248678 >> v2:172.21.15.67:3300/0 conn(0x7fb4cc0c9380 msgr2=0x7fb4cc0acf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.812+0000 7fb4cbe00640 1 --2- 172.21.15.67:0/1437248678 >> v2:172.21.15.67:3300/0 conn(0x7fb4cc0c9380 0x7fb4cc0acf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.812+0000 7fb4cbe00640 1 -- 172.21.15.67:0/1437248678 >> v1:172.21.15.134:6789/0 conn(0x7fb4cc0ad440 legacy=0x7fb4cc06c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.812+0000 7fb4cbe00640 1 -- 172.21.15.67:0/1437248678 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0c5f80 con 0x7fb4cc14fd90 2024-03-09T09:21:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.814+0000 7fb4cbe00640 1 --2- 172.21.15.67:0/1437248678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14fd90 0x7fb4cc156680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0ada40 tx=0x7fb4bc02b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12d2cc0dcf5a7b02 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.814+0000 7fb4caa00640 1 -- 172.21.15.67:0/1437248678 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc035020 con 0x7fb4cc14fd90 2024-03-09T09:21:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.814+0000 7fb4caa00640 1 -- 172.21.15.67:0/1437248678 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc031800 con 0x7fb4cc14fd90 2024-03-09T09:21:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.814+0000 7fb4caa00640 1 -- 172.21.15.67:0/1437248678 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc031bd0 con 0x7fb4cc14fd90 2024-03-09T09:21:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.815+0000 7fb4d1800640 1 -- 172.21.15.67:0/1437248678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14fd90 msgr2=0x7fb4cc156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.815+0000 7fb4d1800640 1 --2- 172.21.15.67:0/1437248678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14fd90 0x7fb4cc156680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0ada40 tx=0x7fb4bc02b8a0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.815+0000 7fb4d1800640 1 -- 172.21.15.67:0/1437248678 shutdown_connections 2024-03-09T09:21:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.815+0000 7fb4d1800640 1 --2- 172.21.15.67:0/1437248678 >> v2:172.21.15.67:3300/0 conn(0x7fb4cc0c9380 0x7fb4cc0acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.815+0000 7fb4d1800640 1 --2- 172.21.15.67:0/1437248678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14fd90 0x7fb4cc156680 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.815+0000 7fb4d1800640 1 -- 172.21.15.67:0/1437248678 >> 172.21.15.67:0/1437248678 conn(0x7fb4cc0cb480 msgr2=0x7fb4cc0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.815+0000 7fb4d1800640 1 -- 172.21.15.67:0/1437248678 shutdown_connections 2024-03-09T09:21:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.815+0000 7fb4d1800640 1 -- 172.21.15.67:0/1437248678 wait complete. 2024-03-09T09:21:04.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.816+0000 7fb4d1800640 1 Processor -- start 2024-03-09T09:21:04.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.817+0000 7fb4d1800640 1 -- start start 2024-03-09T09:21:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.817+0000 7fb4d1800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0ad440 0x7fb4cc1eb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.817+0000 7fb4d1800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc0c9380 0x7fb4cc1ef340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.818+0000 7fb4d1800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14fd90 0x7fb4cc14e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.818+0000 7fb4d1800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc06bcb0 con 0x7fb4cc0c9380 2024-03-09T09:21:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.818+0000 7fb4d1800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0c6520 con 0x7fb4cc14fd90 2024-03-09T09:21:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.818+0000 7fb4d1800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc06c8e0 con 0x7fb4cc0ad440 2024-03-09T09:21:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.818+0000 7fb4cb400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc0c9380 msgr2=0x7fb4cc1ef340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.818+0000 7fb4cb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc0c9380 0x7fb4cc1ef340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.818+0000 7fb4cbe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0ad440 0x7fb4cc1eb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.818+0000 7fb4cbe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0ad440 0x7fb4cc1eb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:42668/0 (socket says 172.21.15.67:42668) 2024-03-09T09:21:04.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.818+0000 7fb4cbe00640 1 -- 172.21.15.67:0/1939961395 learned_addr learned my addr 172.21.15.67:0/1939961395 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:04.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.818+0000 7fb4d0e00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14fd90 0x7fb4cc14e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.819+0000 7fb4d0e00640 1 -- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0ad440 msgr2=0x7fb4cc1eb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.819+0000 7fb4d0e00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0ad440 0x7fb4cc1eb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.819+0000 7fb4d0e00640 1 -- 172.21.15.67:0/1939961395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc0c9380 msgr2=0x7fb4cc1ef340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:21:04.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.819+0000 7fb4d0e00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc0c9380 0x7fb4cc1ef340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.819+0000 7fb4d0e00640 1 -- 172.21.15.67:0/1939961395 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc1598e0 con 0x7fb4cc14fd90 2024-03-09T09:21:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.819+0000 7fb4d0e00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14fd90 0x7fb4cc14e180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400c690 tx=0x7fb4c400cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.820+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4003b70 con 0x7fb4cc14fd90 2024-03-09T09:21:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.820+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c4003d10 con 0x7fb4cc14fd90 2024-03-09T09:21:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.820+0000 7fb4d1800640 1 -- 172.21.15.67:0/1939961395 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc159bc0 con 0x7fb4cc14fd90 2024-03-09T09:21:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.820+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4005310 con 0x7fb4cc14fd90 2024-03-09T09:21:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.820+0000 7fb4d1800640 1 -- 172.21.15.67:0/1939961395 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc159140 con 0x7fb4cc14fd90 2024-03-09T09:21:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.821+0000 7fb4d1800640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0b0f20 0x7fb4cc0b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.821+0000 7fb4d1800640 1 -- 172.21.15.67:0/1939961395 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc1eec80 con 0x7fb4cc0b0f20 2024-03-09T09:21:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.822+0000 7fb4cbe00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0b0f20 0x7fb4cc0b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.822+0000 7fb4cbe00640 1 -- 172.21.15.67:0/1939961395 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4cc159690 con 0x7fb4cc0b0f20 2024-03-09T09:21:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.823+0000 7fb4cbe00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0b0f20 0x7fb4cc0b0290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc034040 tx=0x7fb4bc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.823+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc03ac00 con 0x7fb4cc0b0f20 2024-03-09T09:21:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.823+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb4c40054b0 con 0x7fb4cc14fd90 2024-03-09T09:21:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.824+0000 7fb4c3e00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb4ac06b3d0 0x7fb4ac06d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.824+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb4c408a310 con 0x7fb4cc14fd90 2024-03-09T09:21:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.824+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb4bc04a070 con 0x7fb4cc0b0f20 2024-03-09T09:21:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.824+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0b0f20 msgr2=0x7fb4cc0b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.824+0000 7fb4c3e00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0b0f20 0x7fb4cc0b0290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc034040 tx=0x7fb4bc004730 comp rx=0 tx=0).stop 2024-03-09T09:21:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.824+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0b0f20 msgr2=0x7fb4cc0b0290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.824+0000 7fb4c3e00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc0b0f20 0x7fb4cc0b0290 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc034040 tx=0x7fb4bc004730 comp rx=0 tx=0).stop 2024-03-09T09:21:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.825+0000 7fb4cb400640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb4ac06b3d0 0x7fb4ac06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.826+0000 7fb4cb400640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb4ac06b3d0 0x7fb4ac06d890 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc1590f0 tx=0x7fb4b8006d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.831+0000 7fb4d1800640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc1ef880 0x7fb4cc0b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:04.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.831+0000 7fb4d1800640 1 -- 172.21.15.67:0/1939961395 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc1ef060 con 0x7fb4cc1ef880 2024-03-09T09:21:04.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.832+0000 7fb4cbe00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc1ef880 0x7fb4cc0b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.833+0000 7fb4cbe00640 1 -- 172.21.15.67:0/1939961395 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb4cc04f3c0 con 0x7fb4cc1ef880 2024-03-09T09:21:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.833+0000 7fb4cbe00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc1ef880 0x7fb4cc0b07d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0c9e20 tx=0x7fb4bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.833+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc003bb0 con 0x7fb4cc1ef880 2024-03-09T09:21:04.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.834+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5176 (secure 0 0 0) 0x7fb4bc003bb0 con 0x7fb4cc1ef880 2024-03-09T09:21:04.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.834+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc1ef880 msgr2=0x7fb4cc0b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.834+0000 7fb4c3e00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc1ef880 0x7fb4cc0b07d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0c9e20 tx=0x7fb4bc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.834+0000 7fb4c3e00640 1 -- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc1ef880 msgr2=0x7fb4cc0b07d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:04.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.834+0000 7fb4c3e00640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc1ef880 0x7fb4cc0b07d0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0c9e20 tx=0x7fb4bc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:21:04.841 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:21:04.841 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:21:04.841 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:21:04.841 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:21:04.841 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 218, 2024-03-09T09:21:04.841 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:21:04.842 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:21:04.842 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:21:04.842 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.842 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:21:04.842 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:04.842 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:21:04.842 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:21:04.842 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:04.842 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:04.843 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:04.843 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:04.843 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:04.843 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:04.843 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:04.843 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:04.843 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:04.843 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:04.843 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:04.843 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.844 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:21:04.844 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:21:04.844 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:04.844 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:04.844 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:04.844 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:04.844 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:04.844 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:04.844 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:04.844 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:21:04.845 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:04.846 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:04.847 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:21:04.848 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.849 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.850 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.852 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.853 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.854 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:21:04.855 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:21:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.835+0000 7fb4d1800640 1 -- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb4ac06b3d0 msgr2=0x7fb4ac06d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.835+0000 7fb4d1800640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb4ac06b3d0 0x7fb4ac06d890 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc1590f0 tx=0x7fb4b8006d30 comp rx=0 tx=0).stop 2024-03-09T09:21:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.835+0000 7fb4d1800640 1 -- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14fd90 msgr2=0x7fb4cc14e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.835+0000 7fb4d1800640 1 --2- 172.21.15.67:0/1939961395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14fd90 0x7fb4cc14e180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400c690 tx=0x7fb4c400cb50 comp rx=0 tx=0).stop 2024-03-09T09:21:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.835+0000 7fb4d0e00640 1 -- 172.21.15.67:0/1939961395 reap_dead start 2024-03-09T09:21:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.835+0000 7fb4d0e00640 1 -- 172.21.15.67:0/1939961395 reap_dead start 2024-03-09T09:21:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.835+0000 7fb4d1800640 1 -- 172.21.15.67:0/1939961395 shutdown_connections 2024-03-09T09:21:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.835+0000 7fb4d1800640 1 -- 172.21.15.67:0/1939961395 >> 172.21.15.67:0/1939961395 conn(0x7fb4cc0cb480 msgr2=0x7fb4cc14b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.835+0000 7fb4d1800640 1 -- 172.21.15.67:0/1939961395 shutdown_connections 2024-03-09T09:21:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:04.835+0000 7fb4d1800640 1 -- 172.21.15.67:0/1939961395 wait complete. 2024-03-09T09:21:04.862 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:21:04.863 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:21:08.701 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:21:08.701 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:21:08.701 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 56 -eq 48 ']' 2024-03-09T09:21:08.702 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.56 2024-03-09T09:21:08.702 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.56 2024-03-09T09:21:08.702 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.56 2024-03-09T09:21:08.702 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.56 2024-03-09T09:21:08.707 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:21:08.817 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:21:08.817 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:08.817 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 48 -eq 48 ']' 2024-03-09T09:21:08.818 INFO:tasks.workunit.client.1.smithi134.stderr:+ continue 2024-03-09T09:21:08.818 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:08.818 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.49 2024-03-09T09:21:08.818 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 49 -eq 48 ']' 2024-03-09T09:21:08.818 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.49 2024-03-09T09:21:08.818 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.49 2024-03-09T09:21:08.818 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.49 2024-03-09T09:21:08.821 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:21:13.221 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:21:13.222 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:13.222 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 50 -eq 48 ']' 2024-03-09T09:21:13.222 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.50 2024-03-09T09:21:13.222 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.50 2024-03-09T09:21:13.222 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.50 2024-03-09T09:21:13.222 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.50 2024-03-09T09:21:13.226 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:21:13.892 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:21:13.892 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:21:13.892 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 57 -eq 48 ']' 2024-03-09T09:21:13.892 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.57 2024-03-09T09:21:13.892 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.57 2024-03-09T09:21:13.893 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.57 2024-03-09T09:21:13.893 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.57 2024-03-09T09:21:13.896 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:21:16.117 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:21:16.117 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:16.118 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 51 -eq 48 ']' 2024-03-09T09:21:16.118 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.51 2024-03-09T09:21:16.118 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.51 2024-03-09T09:21:16.118 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.51 2024-03-09T09:21:16.118 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.51 2024-03-09T09:21:16.122 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:21:20.425 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:21:20.426 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:21:20.426 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 58 -eq 48 ']' 2024-03-09T09:21:20.426 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.58 2024-03-09T09:21:20.426 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.58 2024-03-09T09:21:20.427 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.58 2024-03-09T09:21:20.427 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.58 2024-03-09T09:21:20.431 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:21:20.473 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:21:20.473 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:20.473 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 52 -eq 48 ']' 2024-03-09T09:21:20.473 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.52 2024-03-09T09:21:20.474 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.52 2024-03-09T09:21:20.474 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.52 2024-03-09T09:21:20.474 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.52 2024-03-09T09:21:20.478 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:21:24.863 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:21:24.863 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-03-09T09:21:24.863 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-09T09:21:24.864 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-09T09:21:24.872 INFO:tasks.ceph.mon.a:Started 2024-03-09T09:21:24.872 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:21:24.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:21:25.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.057+0000 7fee2a600640 1 Processor -- start 2024-03-09T09:21:25.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.058+0000 7fee2a600640 1 -- start start 2024-03-09T09:21:25.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.059+0000 7fee2a600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fee240ac730 0x7fee24071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:25.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.059+0000 7fee2a600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee241581a0 con 0x7fee240c22a0 2024-03-09T09:21:25.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.059+0000 7fee2a600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee24153d40 con 0x7fee24157d60 2024-03-09T09:21:25.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.059+0000 7fee2a600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee24070ee0 con 0x7fee240ac730 2024-03-09T09:21:25.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.059+0000 7fee22a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fee240c22a0 0x7fee24157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:43030/0 (socket says 172.21.15.67:43030) 2024-03-09T09:21:25.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.059+0000 7fee22a00640 1 -- 172.21.15.67:0/3608461248 learned_addr learned my addr 172.21.15.67:0/3608461248 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:21:25.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.059+0000 7fee23400640 1 --2- 172.21.15.67:0/3608461248 >> v2:172.21.15.134:3300/0 conn(0x7fee240ac730 0x7fee24071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.060+0000 7fee23400640 1 -- 172.21.15.67:0/3608461248 >> v1:172.21.15.134:6789/0 conn(0x7fee24157d60 legacy=0x7fee2415e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.060+0000 7fee23400640 1 -- 172.21.15.67:0/3608461248 >> v1:172.21.15.67:6789/0 conn(0x7fee240c22a0 legacy=0x7fee24157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.060+0000 7fee23400640 1 -- 172.21.15.67:0/3608461248 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee240c2790 con 0x7fee240ac730 2024-03-09T09:21:25.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.061+0000 7fee23400640 1 --2- 172.21.15.67:0/3608461248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee240ac730 0x7fee24071190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fee140097a0 tx=0x7fee1402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be78821cb978093 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:25.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.062+0000 7fee22000640 1 -- 172.21.15.67:0/3608461248 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14035020 con 0x7fee240ac730 2024-03-09T09:21:25.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.062+0000 7fee22000640 1 -- 172.21.15.67:0/3608461248 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee14031830 con 0x7fee240ac730 2024-03-09T09:21:25.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.062+0000 7fee22000640 1 -- 172.21.15.67:0/3608461248 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14031c00 con 0x7fee240ac730 2024-03-09T09:21:25.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.062+0000 7fee2a600640 1 -- 172.21.15.67:0/3608461248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee240ac730 msgr2=0x7fee24071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:25.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.062+0000 7fee2a600640 1 --2- 172.21.15.67:0/3608461248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee240ac730 0x7fee24071190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fee140097a0 tx=0x7fee1402b8d0 comp rx=0 tx=0).stop 2024-03-09T09:21:25.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.062+0000 7fee2a600640 1 -- 172.21.15.67:0/3608461248 shutdown_connections 2024-03-09T09:21:25.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.062+0000 7fee2a600640 1 --2- 172.21.15.67:0/3608461248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee240ac730 0x7fee24071190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:25.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.062+0000 7fee2a600640 1 -- 172.21.15.67:0/3608461248 >> 172.21.15.67:0/3608461248 conn(0x7fee240c7370 msgr2=0x7fee240c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:25.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.063+0000 7fee2a600640 1 -- 172.21.15.67:0/3608461248 shutdown_connections 2024-03-09T09:21:25.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.063+0000 7fee2a600640 1 -- 172.21.15.67:0/3608461248 wait complete. 2024-03-09T09:21:25.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.064+0000 7fee2a600640 1 Processor -- start 2024-03-09T09:21:25.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.064+0000 7fee2a600640 1 -- start start 2024-03-09T09:21:25.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.065+0000 7fee2a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee240ac730 0x7fee241ec570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:25.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.065+0000 7fee2a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee240c22a0 0x7fee240c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.065+0000 7fee2a600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee24157d60 0x7fee2414d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.065+0000 7fee2a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee24070c00 con 0x7fee240ac730 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.065+0000 7fee2a600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee24154d00 con 0x7fee240c22a0 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.065+0000 7fee2a600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee24151610 con 0x7fee24157d60 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.066+0000 7fee23400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee240ac730 0x7fee241ec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.066+0000 7fee23400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee240ac730 0x7fee241ec570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38402/0 (socket says 172.21.15.67:38402) 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.066+0000 7fee23400640 1 -- 172.21.15.67:0/1409856811 learned_addr learned my addr 172.21.15.67:0/1409856811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.066+0000 7fee23400640 1 -- 172.21.15.67:0/1409856811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee240ac730 msgr2=0x7fee241ec570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.066+0000 7fee23400640 1 -- 172.21.15.67:0/1409856811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee240ac730 msgr2=0x7fee241ec570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.066+0000 7fee23400640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee240ac730 0x7fee241ec570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.066+0000 7fee23400640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee240ac730 0x7fee241ec570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.066+0000 7fee22a00640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee240c22a0 0x7fee240c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.066+0000 7fee23e00640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee24157d60 0x7fee2414d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.067+0000 7fee23e00640 1 -- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee240c22a0 msgr2=0x7fee240c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.067+0000 7fee23e00640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee240c22a0 0x7fee240c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.067+0000 7fee23e00640 1 -- 172.21.15.67:0/1409856811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee240ac730 msgr2=0x7fee241ec570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.067+0000 7fee23e00640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee240ac730 0x7fee241ec570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.067+0000 7fee23e00640 1 -- 172.21.15.67:0/1409856811 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee240abf70 con 0x7fee24157d60 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.067+0000 7fee22a00640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee240c22a0 0x7fee240c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.067+0000 7fee23e00640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee24157d60 0x7fee2414d9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00c6c0 tx=0x7fee1c00cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.068+0000 7fee1b400640 1 -- 172.21.15.67:0/1409856811 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c003b70 con 0x7fee24157d60 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.068+0000 7fee1b400640 1 -- 172.21.15.67:0/1409856811 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c003d10 con 0x7fee24157d60 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.068+0000 7fee2a600640 1 -- 172.21.15.67:0/1409856811 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee240ac250 con 0x7fee24157d60 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.068+0000 7fee1b400640 1 -- 172.21.15.67:0/1409856811 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0052f0 con 0x7fee24157d60 2024-03-09T09:21:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.068+0000 7fee2a600640 1 -- 172.21.15.67:0/1409856811 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee240ab330 con 0x7fee24157d60 2024-03-09T09:21:25.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.069+0000 7fee2a600640 1 -- 172.21.15.67:0/1409856811 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee240ab530 con 0x7fee24157d60 2024-03-09T09:21:25.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.070+0000 7fee1b400640 1 -- 172.21.15.67:0/1409856811 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fee1c005490 con 0x7fee24157d60 2024-03-09T09:21:25.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.071+0000 7fee1b400640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fee0406b4d0 0x7fee0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:25.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.072+0000 7fee1b400640 1 -- 172.21.15.67:0/1409856811 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fee1c08ad10 con 0x7fee24157d60 2024-03-09T09:21:25.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.072+0000 7fee23400640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fee0406b4d0 0x7fee0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:25.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.073+0000 7fee23400640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fee0406b4d0 0x7fee0406d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fee1402b910 tx=0x7fee14008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:25.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.085+0000 7fee1b400640 1 -- 172.21.15.67:0/1409856811 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fee240ab530 con 0x7fee24157d60 2024-03-09T09:21:25.165 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:21:25.166 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:21:25.166 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 59 -eq 48 ']' 2024-03-09T09:21:25.166 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.59 2024-03-09T09:21:25.166 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.59 2024-03-09T09:21:25.167 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.59 2024-03-09T09:21:25.167 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.59 2024-03-09T09:21:25.172 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:21:25.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.426+0000 7fee2a600640 1 -- 172.21.15.67:0/1409856811 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fee24075a00 con 0x7fee24157d60 2024-03-09T09:21:25.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.427+0000 7fee1b400640 1 -- 172.21.15.67:0/1409856811 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fee1c059910 con 0x7fee24157d60 2024-03-09T09:21:25.433 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":218,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:21:25.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.429+0000 7fee2a600640 1 -- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fee0406b4d0 msgr2=0x7fee0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:25.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.429+0000 7fee2a600640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fee0406b4d0 0x7fee0406d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fee1402b910 tx=0x7fee14008030 comp rx=0 tx=0).stop 2024-03-09T09:21:25.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.429+0000 7fee2a600640 1 -- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee24157d60 msgr2=0x7fee2414d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.429+0000 7fee2a600640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee24157d60 0x7fee2414d9d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00c6c0 tx=0x7fee1c00cb80 comp rx=0 tx=0).stop 2024-03-09T09:21:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.430+0000 7fee2a600640 1 -- 172.21.15.67:0/1409856811 shutdown_connections 2024-03-09T09:21:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.430+0000 7fee2a600640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fee0406b4d0 0x7fee0406d990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.430+0000 7fee2a600640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee24157d60 0x7fee2414d9d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.430+0000 7fee2a600640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee240c22a0 0x7fee240c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.430+0000 7fee2a600640 1 --2- 172.21.15.67:0/1409856811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee240ac730 0x7fee241ec570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.430+0000 7fee2a600640 1 -- 172.21.15.67:0/1409856811 >> 172.21.15.67:0/1409856811 conn(0x7fee240c7370 msgr2=0x7fee240c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.430+0000 7fee2a600640 1 -- 172.21.15.67:0/1409856811 shutdown_connections 2024-03-09T09:21:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:25.430+0000 7fee2a600640 1 -- 172.21.15.67:0/1409856811 wait complete. 2024-03-09T09:21:26.078 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:21:26.079 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:26.079 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 53 -eq 48 ']' 2024-03-09T09:21:26.079 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.53 2024-03-09T09:21:26.079 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.53 2024-03-09T09:21:26.079 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.53 2024-03-09T09:21:26.080 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.53 2024-03-09T09:21:27.230 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:21:28.456 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:21:28.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.666+0000 7ff0f2200640 1 Processor -- start 2024-03-09T09:21:28.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.666+0000 7ff0f2200640 1 -- start start 2024-03-09T09:21:28.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0f2200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff0ec070420 0x7ff0ec1543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:28.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0f2200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 0x7ff0ec0ab680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:28.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0f2200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec06f800 con 0x7ff0ec150f00 2024-03-09T09:21:28.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0f2200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ec06fae0 con 0x7ff0ec070420 2024-03-09T09:21:28.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0f2200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec06fdc0 con 0x7ff0ec0ab210 2024-03-09T09:21:28.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0ebe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff0ec070420 0x7ff0ec1543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:28.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0ebe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff0ec070420 0x7ff0ec1543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38404/0 (socket says 172.21.15.67:38404) 2024-03-09T09:21:28.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0ebe00640 1 -- 172.21.15.67:0/1204723240 learned_addr learned my addr 172.21.15.67:0/1204723240 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:28.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0ebe00640 1 -- 172.21.15.67:0/1204723240 >> v2:172.21.15.67:3300/0 conn(0x7ff0ec070420 msgr2=0x7ff0ec1543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:21:28.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0ebe00640 1 -- 172.21.15.67:0/1204723240 >> v2:172.21.15.67:3300/0 conn(0x7ff0ec070420 msgr2=0x7ff0ec1543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:28.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0ebe00640 1 --2- 172.21.15.67:0/1204723240 >> v2:172.21.15.67:3300/0 conn(0x7ff0ec070420 0x7ff0ec1543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:28.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.668+0000 7ff0ebe00640 1 --2- 172.21.15.67:0/1204723240 >> v2:172.21.15.67:3300/0 conn(0x7ff0ec070420 0x7ff0ec1543a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:28.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.674+0000 7ff0f1800640 1 --2- 172.21.15.67:0/1204723240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 0x7ff0ec0ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:28.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.675+0000 7ff0f1800640 1 -- 172.21.15.67:0/1204723240 >> v2:172.21.15.67:3300/0 conn(0x7ff0ec070420 msgr2=0x7ff0ec1543a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:28.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.675+0000 7ff0f1800640 1 --2- 172.21.15.67:0/1204723240 >> v2:172.21.15.67:3300/0 conn(0x7ff0ec070420 0x7ff0ec1543a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:28.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.675+0000 7ff0f1800640 1 -- 172.21.15.67:0/1204723240 >> v1:172.21.15.134:6789/0 conn(0x7ff0ec150f00 legacy=0x7ff0ec0c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:28.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.675+0000 7ff0f1800640 1 -- 172.21.15.67:0/1204723240 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec1512e0 con 0x7ff0ec0ab210 2024-03-09T09:21:28.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.675+0000 7ff0eb400640 1 -- 172.21.15.67:0/1204723240 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626469228 0 0) 0x7ff0ec06f800 con 0x7ff0ec150f00 2024-03-09T09:21:28.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.677+0000 7ff0f1800640 1 --2- 172.21.15.67:0/1204723240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 0x7ff0ec0ab680 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009d90 tx=0x7ff0e402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace56937df2ad0ba server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:28.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.678+0000 7ff0eb400640 1 -- 172.21.15.67:0/1204723240 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4039070 con 0x7ff0ec0ab210 2024-03-09T09:21:28.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.678+0000 7ff0eb400640 1 -- 172.21.15.67:0/1204723240 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e402cbc0 con 0x7ff0ec0ab210 2024-03-09T09:21:28.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.678+0000 7ff0eb400640 1 -- 172.21.15.67:0/1204723240 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4034440 con 0x7ff0ec0ab210 2024-03-09T09:21:28.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.678+0000 7ff0f2200640 1 -- 172.21.15.67:0/1204723240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 msgr2=0x7ff0ec0ab680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:28.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.678+0000 7ff0f2200640 1 --2- 172.21.15.67:0/1204723240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 0x7ff0ec0ab680 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009d90 tx=0x7ff0e402be10 comp rx=0 tx=0).stop 2024-03-09T09:21:28.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.678+0000 7ff0f2200640 1 -- 172.21.15.67:0/1204723240 shutdown_connections 2024-03-09T09:21:28.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.678+0000 7ff0f2200640 1 --2- 172.21.15.67:0/1204723240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 0x7ff0ec0ab680 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:28.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.678+0000 7ff0f2200640 1 --2- 172.21.15.67:0/1204723240 >> v2:172.21.15.67:3300/0 conn(0x7ff0ec070420 0x7ff0ec1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:28.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.678+0000 7ff0f2200640 1 -- 172.21.15.67:0/1204723240 >> 172.21.15.67:0/1204723240 conn(0x7ff0ec0c9400 msgr2=0x7ff0ec0c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:28.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.678+0000 7ff0f2200640 1 -- 172.21.15.67:0/1204723240 shutdown_connections 2024-03-09T09:21:28.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.678+0000 7ff0f2200640 1 -- 172.21.15.67:0/1204723240 wait complete. 2024-03-09T09:21:28.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.680+0000 7ff0f2200640 1 Processor -- start 2024-03-09T09:21:28.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.680+0000 7ff0f2200640 1 -- start start 2024-03-09T09:21:28.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.681+0000 7ff0f2200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0ec070420 0x7ff0ec0c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:28.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.681+0000 7ff0f2200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 0x7ff0ec157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.681+0000 7ff0f2200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec150f00 0x7ff0ec0c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.681+0000 7ff0f2200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec0c4690 con 0x7ff0ec070420 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.681+0000 7ff0f2200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec0c4510 con 0x7ff0ec150f00 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.681+0000 7ff0f2200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec0abbc0 con 0x7ff0ec0ab210 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.681+0000 7ff0f0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0ec070420 0x7ff0ec0c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.681+0000 7ff0f0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0ec070420 0x7ff0ec0c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38408/0 (socket says 172.21.15.67:38408) 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.681+0000 7ff0f0e00640 1 -- 172.21.15.67:0/4213999426 learned_addr learned my addr 172.21.15.67:0/4213999426 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.682+0000 7ff0f0e00640 1 -- 172.21.15.67:0/4213999426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0ec070420 msgr2=0x7ff0ec0c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.682+0000 7ff0f0e00640 1 -- 172.21.15.67:0/4213999426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0ec070420 msgr2=0x7ff0ec0c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.682+0000 7ff0f0e00640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0ec070420 0x7ff0ec0c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.682+0000 7ff0f0e00640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0ec070420 0x7ff0ec0c8d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.682+0000 7ff0f1800640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec150f00 0x7ff0ec0c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:28.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.682+0000 7ff0ebe00640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 0x7ff0ec157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:28.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.683+0000 7ff0ebe00640 1 -- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec150f00 msgr2=0x7ff0ec0c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.683+0000 7ff0ebe00640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec150f00 0x7ff0ec0c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.683+0000 7ff0ebe00640 1 -- 172.21.15.67:0/4213999426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0ec070420 msgr2=0x7ff0ec0c8d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.683+0000 7ff0ebe00640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0ec070420 0x7ff0ec0c8d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.683+0000 7ff0ebe00640 1 -- 172.21.15.67:0/4213999426 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec0c7560 con 0x7ff0ec0ab210 2024-03-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.683+0000 7ff0f1800640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec150f00 0x7ff0ec0c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.683+0000 7ff0ebe00640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 0x7ff0ec157660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009100 tx=0x7ff0d800ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.683+0000 7ff0e8c00640 1 -- 172.21.15.67:0/4213999426 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8017070 con 0x7ff0ec0ab210 2024-03-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.684+0000 7ff0e8c00640 1 -- 172.21.15.67:0/4213999426 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d8004020 con 0x7ff0ec0ab210 2024-03-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.684+0000 7ff0e8c00640 1 -- 172.21.15.67:0/4213999426 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800f040 con 0x7ff0ec0ab210 2024-03-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.684+0000 7ff0f2200640 1 -- 172.21.15.67:0/4213999426 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ec1ec050 con 0x7ff0ec0ab210 2024-03-09T09:21:28.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.684+0000 7ff0f2200640 1 -- 172.21.15.67:0/4213999426 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ec06c9e0 con 0x7ff0ec0ab210 2024-03-09T09:21:28.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.685+0000 7ff0f2200640 1 -- 172.21.15.67:0/4213999426 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ec073be0 con 0x7ff0ec0ab210 2024-03-09T09:21:28.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.686+0000 7ff0e8c00640 1 -- 172.21.15.67:0/4213999426 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff0d800c6d0 con 0x7ff0ec0ab210 2024-03-09T09:21:28.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.687+0000 7ff0e8c00640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0cc06b500 0x7ff0cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:28.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.687+0000 7ff0e8c00640 1 -- 172.21.15.67:0/4213999426 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff0d808a3e0 con 0x7ff0ec0ab210 2024-03-09T09:21:28.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.688+0000 7ff0f0e00640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0cc06b500 0x7ff0cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:28.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.689+0000 7ff0f0e00640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0cc06b500 0x7ff0cc06d9c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0000f60 tx=0x7ff0e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.700+0000 7ff0e8c00640 1 -- 172.21.15.67:0/4213999426 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff0ec073be0 con 0x7ff0ec0ab210 2024-03-09T09:21:28.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.986+0000 7ff0f2200640 1 -- 172.21.15.67:0/4213999426 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff0ec06c7f0 con 0x7ff0ec0ab210 2024-03-09T09:21:28.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.988+0000 7ff0e8c00640 1 -- 172.21.15.67:0/4213999426 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7ff0d8058890 con 0x7ff0ec0ab210 2024-03-09T09:21:28.994 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":218,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.990+0000 7ff0f2200640 1 -- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0cc06b500 msgr2=0x7ff0cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.990+0000 7ff0f2200640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0cc06b500 0x7ff0cc06d9c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0000f60 tx=0x7ff0e0009040 comp rx=0 tx=0).stop 2024-03-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.990+0000 7ff0f2200640 1 -- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 msgr2=0x7ff0ec157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.990+0000 7ff0f2200640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 0x7ff0ec157660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009100 tx=0x7ff0d800ec70 comp rx=0 tx=0).stop 2024-03-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.990+0000 7ff0f2200640 1 -- 172.21.15.67:0/4213999426 shutdown_connections 2024-03-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.990+0000 7ff0f2200640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0cc06b500 0x7ff0cc06d9c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.990+0000 7ff0f2200640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0ec150f00 0x7ff0ec0c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.990+0000 7ff0f2200640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0ec0ab210 0x7ff0ec157660 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.990+0000 7ff0f2200640 1 --2- 172.21.15.67:0/4213999426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0ec070420 0x7ff0ec0c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.990+0000 7ff0f2200640 1 -- 172.21.15.67:0/4213999426 >> 172.21.15.67:0/4213999426 conn(0x7ff0ec0c9400 msgr2=0x7ff0ec1540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.991+0000 7ff0f2200640 1 -- 172.21.15.67:0/4213999426 shutdown_connections 2024-03-09T09:21:28.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:28.991+0000 7ff0f2200640 1 -- 172.21.15.67:0/4213999426 wait complete. 2024-03-09T09:21:29.751 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:21:29.759 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:21:30.450 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:21:30.450 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:21:30.450 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 60 -eq 48 ']' 2024-03-09T09:21:30.451 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.60 2024-03-09T09:21:30.451 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.60 2024-03-09T09:21:30.451 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.60 2024-03-09T09:21:30.451 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.60 2024-03-09T09:21:30.455 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:21:31.564 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:21:31.564 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:31.564 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 54 -eq 48 ']' 2024-03-09T09:21:31.564 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.54 2024-03-09T09:21:31.564 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.54 2024-03-09T09:21:31.565 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.54 2024-03-09T09:21:31.565 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.54 2024-03-09T09:21:31.568 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:21:32.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:21:32.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.230+0000 7f496f200640 1 Processor -- start 2024-03-09T09:21:32.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.230+0000 7f496f200640 1 -- start start 2024-03-09T09:21:32.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.232+0000 7f496f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:32.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.232+0000 7f496f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:32.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.232+0000 7f496f200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497014ccb0 con 0x7f497014c820 2024-03-09T09:21:32.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.232+0000 7f496f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4970155ba0 con 0x7f49700c3080 2024-03-09T09:21:32.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.232+0000 7f496f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497006d920 con 0x7f497006d420 2024-03-09T09:21:32.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.233+0000 7f496de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:32.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.233+0000 7f496de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39938/0 (socket says 172.21.15.67:39938) 2024-03-09T09:21:32.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.233+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 learned_addr learned my addr 172.21.15.67:0/630344376 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:32.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.233+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 msgr2=0x7f49701527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:32.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.233+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 msgr2=0x7f49701527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:32.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.233+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:32.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.233+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:32.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.233+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:32.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.241+0000 7f496d400640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 msgr2=0x7f497014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:21:32.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.241+0000 7f496d400640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 msgr2=0x7f497014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:32.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.241+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:32.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.241+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.434+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.434+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 msgr2=0x7f49701527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.434+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 msgr2=0x7f49701527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.434+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.434+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:21:32.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.442+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:32.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.443+0000 7f496d400640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 msgr2=0x7f497014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:32.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.443+0000 7f496d400640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 msgr2=0x7f497014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:32.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.443+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:32.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.443+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:21:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.835+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:32.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.836+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 msgr2=0x7f49701527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:32.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.836+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 msgr2=0x7f49701527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:32.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.836+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:32.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.836+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:21:32.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.844+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.845+0000 7f496d400640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 msgr2=0x7f497014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.845+0000 7f496d400640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 msgr2=0x7f497014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:32.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.845+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:32.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:32.845+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:21:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:33.637+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:33.638+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 msgr2=0x7f49701527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:33.638+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 msgr2=0x7f49701527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:33.638+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:33.638+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:21:33.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:33.647+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:33.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:33.648+0000 7f496d400640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 msgr2=0x7f497014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:33.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:33.648+0000 7f496d400640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 msgr2=0x7f497014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:33.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:33.648+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:33.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:33.648+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:21:35.073 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:21:35.073 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:35.073 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 55 -eq 48 ']' 2024-03-09T09:21:35.073 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.55 2024-03-09T09:21:35.073 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.55 2024-03-09T09:21:35.073 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.55 2024-03-09T09:21:35.074 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.55 2024-03-09T09:21:35.077 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:21:35.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.232+0000 7f4967400640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 msgr2=0x7f497014c2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.232+0000 7f4967400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f497006d420 0x7f497014c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.232+0000 7f4967400640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 msgr2=0x7f49701527d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.232+0000 7f4967400640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.67:3300/0 conn(0x7f49700c3080 0x7f49701527d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.232+0000 7f4967400640 1 -- 172.21.15.67:0/630344376 >> v1:172.21.15.134:6789/0 conn(0x7f497014c820 legacy=0x7f49701540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.233+0000 7f4967400640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.233+0000 7f4967400640 1 -- 172.21.15.67:0/630344376 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497006dc00 con 0x7f4950006190 2024-03-09T09:21:35.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.233+0000 7f4967400640 1 -- 172.21.15.67:0/630344376 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4970155ba0 con 0x7f49500036e0 2024-03-09T09:21:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.233+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.234+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 msgr2=0x7f4950005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.234+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 msgr2=0x7f4950005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.234+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:35.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.234+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:35.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.435+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:35.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.436+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 msgr2=0x7f4950005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:35.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.436+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 msgr2=0x7f4950005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:35.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.436+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:35.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.436+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:21:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.837+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.838+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 msgr2=0x7f4950005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.838+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 msgr2=0x7f4950005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.838+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:35.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:35.838+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:21:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:36.640+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:36.640+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 msgr2=0x7f4950005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:21:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:36.641+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 msgr2=0x7f4950005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:21:36.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:36.641+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:21:36.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:36.641+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:21:37.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.707+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461526546 0 0) 0x7f497006dc00 con 0x7f4950006190 2024-03-09T09:21:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.707+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4954003520 con 0x7f4950006190 2024-03-09T09:21:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.708+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2058622988 0 0) 0x7f4954003520 con 0x7f4950006190 2024-03-09T09:21:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.708+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 msgr2=0x7f4950005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:37.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.708+0000 7f496ca00640 1 --2- 172.21.15.67:0/630344376 >> v2:172.21.15.134:3300/0 conn(0x7f49500036e0 0x7f4950005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:37.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.708+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495000af60 con 0x7f4950006190 2024-03-09T09:21:37.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.708+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f495c00d580 con 0x7f4950006190 2024-03-09T09:21:37.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.708+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 >> v1:172.21.15.67:6789/0 conn(0x7f4950006190 legacy=0x7f4950008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:37.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.708+0000 7f496e800640 1 -- 172.21.15.67:0/630344376 reap_dead start 2024-03-09T09:21:37.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.709+0000 7f496ca00640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4954003a70 0x7f4954003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:37.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.709+0000 7f496ca00640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4954004800 0x7f4954004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:37.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.709+0000 7f496ca00640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49540053a0 0x7f495400ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:37.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.709+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970155ba0 con 0x7f4954004800 2024-03-09T09:21:37.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.709+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495400c2e0 con 0x7f49540053a0 2024-03-09T09:21:37.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.709+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495400c4e0 con 0x7f4954003a70 2024-03-09T09:21:37.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.710+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4954003a70 0x7f4954003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:37.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.710+0000 7f496e800640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49540053a0 0x7f495400ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:37.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.710+0000 7f496d400640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4954004800 0x7f4954004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:37.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.711+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49540053a0 msgr2=0x7f495400ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:37.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.711+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49540053a0 0x7f495400ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:37.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.711+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4954004800 msgr2=0x7f4954004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:37.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.711+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4954004800 0x7f4954004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:37.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.711+0000 7f496de00640 1 -- 172.21.15.67:0/630344376 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4954003520 con 0x7f4954003a70 2024-03-09T09:21:37.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.713+0000 7f496de00640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4954003a70 0x7f4954003e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4960019ae0 tx=0x7f4954005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cfbdc57c2901f6a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:37.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.717+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496000f040 con 0x7f4954003a70 2024-03-09T09:21:37.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.717+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4960034600 con 0x7f4954003a70 2024-03-09T09:21:37.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.717+0000 7f496ca00640 1 -- 172.21.15.67:0/630344376 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960034a10 con 0x7f4954003a70 2024-03-09T09:21:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.719+0000 7f496f200640 1 -- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4954003a70 msgr2=0x7f4954003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.719+0000 7f496f200640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4954003a70 0x7f4954003e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4960019ae0 tx=0x7f4954005830 comp rx=0 tx=0).stop 2024-03-09T09:21:37.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.719+0000 7f496f200640 1 -- 172.21.15.67:0/630344376 shutdown_connections 2024-03-09T09:21:37.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.719+0000 7f496f200640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49540053a0 0x7f495400ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:37.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.719+0000 7f496f200640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4954004800 0x7f4954004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:37.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.719+0000 7f496f200640 1 --2- 172.21.15.67:0/630344376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4954003a70 0x7f4954003e70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:37.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.719+0000 7f496f200640 1 -- 172.21.15.67:0/630344376 >> 172.21.15.67:0/630344376 conn(0x7f49700c7300 msgr2=0x7f49700c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:37.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.719+0000 7f496f200640 1 -- 172.21.15.67:0/630344376 shutdown_connections 2024-03-09T09:21:37.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.719+0000 7f496f200640 1 -- 172.21.15.67:0/630344376 wait complete. 2024-03-09T09:21:37.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.722+0000 7f496f200640 1 Processor -- start 2024-03-09T09:21:37.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.723+0000 7f496f200640 1 -- start start 2024-03-09T09:21:37.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.724+0000 7f496f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4954003a70 0x7f49700ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:37.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.724+0000 7f496f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4954004800 0x7f49700ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:37.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.724+0000 7f496f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49540053a0 0x7f497013f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:37.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.724+0000 7f496f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700c8030 con 0x7f49540053a0 2024-03-09T09:21:37.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.724+0000 7f496f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497006e1c0 con 0x7f4954004800 2024-03-09T09:21:37.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.724+0000 7f496f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49701ec3e0 con 0x7f4954003a70 2024-03-09T09:21:37.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.725+0000 7f496e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49540053a0 0x7f497013f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:37.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.725+0000 7f496e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49540053a0 0x7f497013f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39998/0 (socket says 172.21.15.67:39998) 2024-03-09T09:21:37.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.725+0000 7f496e800640 1 -- 172.21.15.67:0/2216353271 learned_addr learned my addr 172.21.15.67:0/2216353271 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:37.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.725+0000 7f496d400640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4954004800 0x7f49700ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:37.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.725+0000 7f496de00640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4954003a70 0x7f49700ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:37.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.726+0000 7f496e800640 1 -- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4954003a70 msgr2=0x7f49700ac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.726+0000 7f496e800640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4954003a70 0x7f49700ac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.726+0000 7f496e800640 1 -- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4954004800 msgr2=0x7f49700ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.726+0000 7f496e800640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4954004800 0x7f49700ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.726+0000 7f496e800640 1 -- 172.21.15.67:0/2216353271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700acd30 con 0x7f49540053a0 2024-03-09T09:21:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.726+0000 7f496e800640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49540053a0 0x7f497013f430 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4950009e50 tx=0x7f496800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.726+0000 7f4966000640 1 -- 172.21.15.67:0/2216353271 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968005b40 con 0x7f49540053a0 2024-03-09T09:21:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.727+0000 7f496f200640 1 -- 172.21.15.67:0/2216353271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49700aaaa0 con 0x7f49540053a0 2024-03-09T09:21:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.727+0000 7f4966000640 1 -- 172.21.15.67:0/2216353271 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968005d00 con 0x7f49540053a0 2024-03-09T09:21:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.727+0000 7f4966000640 1 -- 172.21.15.67:0/2216353271 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968003970 con 0x7f49540053a0 2024-03-09T09:21:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.727+0000 7f496f200640 1 -- 172.21.15.67:0/2216353271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49700aa7c0 con 0x7f49540053a0 2024-03-09T09:21:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.728+0000 7f496f200640 1 -- 172.21.15.67:0/2216353271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497004f3c0 con 0x7f49540053a0 2024-03-09T09:21:37.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.730+0000 7f4966000640 1 -- 172.21.15.67:0/2216353271 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f496801a020 con 0x7f49540053a0 2024-03-09T09:21:37.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.731+0000 7f4966000640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f495006bd20 0x7f495006e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:37.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.731+0000 7f4966000640 1 -- 172.21.15.67:0/2216353271 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f496808a600 con 0x7f49540053a0 2024-03-09T09:21:37.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.731+0000 7f496de00640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f495006bd20 0x7f495006e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:37.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.733+0000 7f496de00640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f495006bd20 0x7f495006e1e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4960037000 tx=0x7f4960032000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:37.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:37.740+0000 7f4966000640 1 -- 172.21.15.67:0/2216353271 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f497014ccb0 con 0x7f49540053a0 2024-03-09T09:21:37.832 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:21:37.833 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:21:37.833 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 61 -eq 48 ']' 2024-03-09T09:21:37.833 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.61 2024-03-09T09:21:37.833 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.61 2024-03-09T09:21:37.833 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.61 2024-03-09T09:21:37.833 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.61 2024-03-09T09:21:37.837 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:21:38.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.055+0000 7f496f200640 1 -- 172.21.15.67:0/2216353271 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4970071860 con 0x7f49540053a0 2024-03-09T09:21:38.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.056+0000 7f4966000640 1 -- 172.21.15.67:0/2216353271 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f4968058ab0 con 0x7f49540053a0 2024-03-09T09:21:38.063 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":226,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:21:38.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 -- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f495006bd20 msgr2=0x7f495006e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f495006bd20 0x7f495006e1e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4960037000 tx=0x7f4960032000 comp rx=0 tx=0).stop 2024-03-09T09:21:38.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 -- 172.21.15.67:0/2216353271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49540053a0 msgr2=0x7f497013f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49540053a0 0x7f497013f430 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4950009e50 tx=0x7f496800fc00 comp rx=0 tx=0).stop 2024-03-09T09:21:38.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 -- 172.21.15.67:0/2216353271 shutdown_connections 2024-03-09T09:21:38.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f495006bd20 0x7f495006e1e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49540053a0 0x7f497013f430 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4954004800 0x7f49700ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 --2- 172.21.15.67:0/2216353271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4954003a70 0x7f49700ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 -- 172.21.15.67:0/2216353271 >> 172.21.15.67:0/2216353271 conn(0x7f49700c7300 msgr2=0x7f49700c6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:38.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 -- 172.21.15.67:0/2216353271 shutdown_connections 2024-03-09T09:21:38.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.059+0000 7f496f200640 1 -- 172.21.15.67:0/2216353271 wait complete. 2024-03-09T09:21:38.089 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:21:38.089 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:21:38.089 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:21:38.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.297+0000 7fb345000640 1 Processor -- start 2024-03-09T09:21:38.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.297+0000 7fb345000640 1 -- start start 2024-03-09T09:21:38.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.299+0000 7fb345000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb34014fd90 0x7fb340156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.299+0000 7fb345000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb340156bc0 con 0x7fb3400c9380 2024-03-09T09:21:38.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.299+0000 7fb345000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3400c9890 con 0x7fb34006c1d0 2024-03-09T09:21:38.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.299+0000 7fb345000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34006cd00 con 0x7fb34014fd90 2024-03-09T09:21:38.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.300+0000 7fb33ea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb3400c9380 0x7fb3400acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39026/0 (socket says 172.21.15.67:39026) 2024-03-09T09:21:38.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.300+0000 7fb33ea00640 1 -- 172.21.15.67:0/2913563774 learned_addr learned my addr 172.21.15.67:0/2913563774 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:21:38.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.300+0000 7fb33f400640 1 --2- 172.21.15.67:0/2913563774 >> v2:172.21.15.134:3300/0 conn(0x7fb34014fd90 0x7fb340156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.300+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901144660 0 0) 0x7fb328001110 con 0x7fb3400c9380 2024-03-09T09:21:38.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.300+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb324003520 con 0x7fb3400c9380 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 585640301 0 0) 0x7fb324003520 con 0x7fb3400c9380 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 >> v2:172.21.15.134:3300/0 conn(0x7fb34014fd90 msgr2=0x7fb340156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 --2- 172.21.15.67:0/2913563774 >> v2:172.21.15.134:3300/0 conn(0x7fb34014fd90 0x7fb340156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 >> v1:172.21.15.134:6789/0 conn(0x7fb34006c1d0 legacy=0x7fb34006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400c5f80 con 0x7fb3400c9380 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb328001840 con 0x7fb3400c9380 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 >> v1:172.21.15.67:6789/0 conn(0x7fb3400c9380 legacy=0x7fb3400acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb324003a20 0x7fb324003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb324004800 0x7fb324004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3240052c0 0x7fb32400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34006cd00 con 0x7fb324003a20 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32400c200 con 0x7fb324004800 2024-03-09T09:21:38.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32400c400 con 0x7fb3240052c0 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.301+0000 7fb33f400640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb324003a20 0x7fb324003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.302+0000 7fb33ea00640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb324004800 0x7fb324004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.302+0000 7fb33fe00640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3240052c0 0x7fb32400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.302+0000 7fb33f400640 1 -- 172.21.15.67:0/2913563774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3240052c0 msgr2=0x7fb32400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.302+0000 7fb33f400640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3240052c0 0x7fb32400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.302+0000 7fb33f400640 1 -- 172.21.15.67:0/2913563774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb324004800 msgr2=0x7fb324004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.302+0000 7fb33f400640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb324004800 0x7fb324004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.302+0000 7fb33f400640 1 -- 172.21.15.67:0/2913563774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb324003520 con 0x7fb324003a20 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.302+0000 7fb33fe00640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3240052c0 0x7fb32400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.302+0000 7fb33fe00640 1 -- 172.21.15.67:0/2913563774 reap_dead start 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.304+0000 7fb33f400640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb324003a20 0x7fb324003e40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb330009ac0 tx=0x7fb324005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff660d7b0c79d69 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.304+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330039070 con 0x7fb324003a20 2024-03-09T09:21:38.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.304+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33002c6b0 con 0x7fb324003a20 2024-03-09T09:21:38.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.304+0000 7fb33e000640 1 -- 172.21.15.67:0/2913563774 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330009690 con 0x7fb324003a20 2024-03-09T09:21:38.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.305+0000 7fb345000640 1 -- 172.21.15.67:0/2913563774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb324003a20 msgr2=0x7fb324003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.305+0000 7fb345000640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb324003a20 0x7fb324003e40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb330009ac0 tx=0x7fb324005830 comp rx=0 tx=0).stop 2024-03-09T09:21:38.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.305+0000 7fb345000640 1 -- 172.21.15.67:0/2913563774 shutdown_connections 2024-03-09T09:21:38.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.305+0000 7fb345000640 1 --2- 172.21.15.67:0/2913563774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb324003a20 0x7fb324003e40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.305+0000 7fb345000640 1 -- 172.21.15.67:0/2913563774 >> 172.21.15.67:0/2913563774 conn(0x7fb3400cb480 msgr2=0x7fb3400cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:38.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.306+0000 7fb345000640 1 -- 172.21.15.67:0/2913563774 shutdown_connections 2024-03-09T09:21:38.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.306+0000 7fb345000640 1 -- 172.21.15.67:0/2913563774 wait complete. 2024-03-09T09:21:38.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.308+0000 7fb345000640 1 Processor -- start 2024-03-09T09:21:38.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.309+0000 7fb345000640 1 -- start start 2024-03-09T09:21:38.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.310+0000 7fb345000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb324003a20 0x7fb3401f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.310+0000 7fb345000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3401f3ea0 0x7fb3401f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.310+0000 7fb345000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f24e0 0x7fb3401df260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.310+0000 7fb345000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400c6520 con 0x7fb3401f24e0 2024-03-09T09:21:38.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.310+0000 7fb345000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34006ba60 con 0x7fb324003a20 2024-03-09T09:21:38.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.310+0000 7fb345000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34006bcb0 con 0x7fb3401f3ea0 2024-03-09T09:21:38.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.310+0000 7fb33fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f24e0 0x7fb3401df260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.310+0000 7fb33f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb324003a20 0x7fb3401f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.310+0000 7fb33fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f24e0 0x7fb3401df260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40020/0 (socket says 172.21.15.67:40020) 2024-03-09T09:21:38.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.310+0000 7fb33fe00640 1 -- 172.21.15.67:0/2457160232 learned_addr learned my addr 172.21.15.67:0/2457160232 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:38.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.311+0000 7fb33fe00640 1 -- 172.21.15.67:0/2457160232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3401f3ea0 msgr2=0x7fb3401f1fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T09:21:38.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.311+0000 7fb33fe00640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3401f3ea0 0x7fb3401f1fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.311+0000 7fb33fe00640 1 -- 172.21.15.67:0/2457160232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb324003a20 msgr2=0x7fb3401f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.311+0000 7fb33fe00640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb324003a20 0x7fb3401f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.311+0000 7fb33fe00640 1 -- 172.21.15.67:0/2457160232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3401f4300 con 0x7fb3401f24e0 2024-03-09T09:21:38.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.311+0000 7fb33f400640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb324003a20 0x7fb3401f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:21:38.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.311+0000 7fb33fe00640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f24e0 0x7fb3401df260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3400aacd0 tx=0x7fb33800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.312+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338006d50 con 0x7fb3401f24e0 2024-03-09T09:21:38.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.312+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338003970 con 0x7fb3401f24e0 2024-03-09T09:21:38.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.312+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338003da0 con 0x7fb3401f24e0 2024-03-09T09:21:38.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.312+0000 7fb345000640 1 -- 172.21.15.67:0/2457160232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3401e1b00 con 0x7fb3401f24e0 2024-03-09T09:21:38.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.312+0000 7fb345000640 1 -- 172.21.15.67:0/2457160232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3401f17a0 con 0x7fb3401f24e0 2024-03-09T09:21:38.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.313+0000 7fb345000640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3400c5450 0x7fb340153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.313+0000 7fb345000640 1 -- 172.21.15.67:0/2457160232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3401f4590 con 0x7fb3400c5450 2024-03-09T09:21:38.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.314+0000 7fb33f400640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3400c5450 0x7fb340153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.315+0000 7fb33f400640 1 -- 172.21.15.67:0/2457160232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3400c5ad0 con 0x7fb3400c5450 2024-03-09T09:21:38.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.315+0000 7fb33f400640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3400c5450 0x7fb340153a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb330009a90 tx=0x7fb330004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.315+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb33801b020 con 0x7fb3401f24e0 2024-03-09T09:21:38.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.317+0000 7fb337400640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb31c06b4d0 0x7fb31c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.317+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb33808a010 con 0x7fb3401f24e0 2024-03-09T09:21:38.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.317+0000 7fb33ea00640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb31c06b4d0 0x7fb31c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.317+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330043030 con 0x7fb3400c5450 2024-03-09T09:21:38.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.317+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb33003a040 con 0x7fb3400c5450 2024-03-09T09:21:38.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.317+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3400c5450 msgr2=0x7fb340153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.317+0000 7fb337400640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3400c5450 0x7fb340153a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb330009a90 tx=0x7fb330004730 comp rx=0 tx=0).stop 2024-03-09T09:21:38.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.317+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3400c5450 msgr2=0x7fb340153a30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:38.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.317+0000 7fb337400640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3400c5450 0x7fb340153a30 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb330009a90 tx=0x7fb330004730 comp rx=0 tx=0).stop 2024-03-09T09:21:38.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.318+0000 7fb33ea00640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb31c06b4d0 0x7fb31c06d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb3280012b0 tx=0x7fb328001320 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.326+0000 7fb345000640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f1b10 0x7fb340153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.326+0000 7fb345000640 1 -- 172.21.15.67:0/2457160232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34004f3c0 con 0x7fb3401f1b10 2024-03-09T09:21:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.326+0000 7fb33f400640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f1b10 0x7fb340153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.327+0000 7fb33f400640 1 -- 172.21.15.67:0/2457160232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb34006d2d0 con 0x7fb3401f1b10 2024-03-09T09:21:38.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.327+0000 7fb33f400640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f1b10 0x7fb340153f70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb330006f80 tx=0x7fb330008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.327+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330043070 con 0x7fb3401f1b10 2024-03-09T09:21:38.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.328+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4536 (secure 0 0 0) 0x7fb330034440 con 0x7fb3401f1b10 2024-03-09T09:21:38.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.328+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f1b10 msgr2=0x7fb340153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.328+0000 7fb337400640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f1b10 0x7fb340153f70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb330006f80 tx=0x7fb330008030 comp rx=0 tx=0).stop 2024-03-09T09:21:38.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.328+0000 7fb337400640 1 -- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f1b10 msgr2=0x7fb340153f70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:38.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.328+0000 7fb337400640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f1b10 0x7fb340153f70 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb330006f80 tx=0x7fb330008030 comp rx=0 tx=0).stop 2024-03-09T09:21:38.335 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:21:38.335 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:21:38.335 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:21:38.335 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:21:38.335 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 226, 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:38.336 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:38.337 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:21:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:21:38.339 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.340 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:38.341 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:21:38.342 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.343 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:21:38.345 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.346 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:21:38.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.347 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.347 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.347 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:21:38.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.347 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.347 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:21:38.347 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:21:38.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.328+0000 7fb345000640 1 -- 172.21.15.67:0/2457160232 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb31c06b4d0 msgr2=0x7fb31c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.328+0000 7fb345000640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb31c06b4d0 0x7fb31c06d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb3280012b0 tx=0x7fb328001320 comp rx=0 tx=0).stop 2024-03-09T09:21:38.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.328+0000 7fb345000640 1 -- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f24e0 msgr2=0x7fb3401df260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.328+0000 7fb345000640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f24e0 0x7fb3401df260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3400aacd0 tx=0x7fb33800dc00 comp rx=0 tx=0).stop 2024-03-09T09:21:38.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.329+0000 7fb33fe00640 1 -- 172.21.15.67:0/2457160232 reap_dead start 2024-03-09T09:21:38.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.329+0000 7fb345000640 1 -- 172.21.15.67:0/2457160232 shutdown_connections 2024-03-09T09:21:38.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.329+0000 7fb345000640 1 --2- 172.21.15.67:0/2457160232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb3401f24e0 0x7fb3401df260 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.329+0000 7fb345000640 1 -- 172.21.15.67:0/2457160232 >> 172.21.15.67:0/2457160232 conn(0x7fb3400cb480 msgr2=0x7fb340074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:38.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.329+0000 7fb345000640 1 -- 172.21.15.67:0/2457160232 shutdown_connections 2024-03-09T09:21:38.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.329+0000 7fb345000640 1 -- 172.21.15.67:0/2457160232 wait complete. 2024-03-09T09:21:38.359 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:21:38.359 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:21:38.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:21:38.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.542+0000 7ff0e5200640 1 Processor -- start 2024-03-09T09:21:38.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.542+0000 7ff0e5200640 1 -- start start 2024-03-09T09:21:38.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.545+0000 7ff0e5200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e014fd90 0x7ff0e0156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.545+0000 7ff0e5200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff0e006c1d0 0x7ff0e006c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.545+0000 7ff0e5200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e0156bc0 con 0x7ff0e00c9380 2024-03-09T09:21:38.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.545+0000 7ff0e5200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e006ba60 con 0x7ff0e006c1d0 2024-03-09T09:21:38.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.545+0000 7ff0e5200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00c9890 con 0x7ff0e014fd90 2024-03-09T09:21:38.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.545+0000 7ff0dea00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff0e00c9380 0x7ff0e00acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39036/0 (socket says 172.21.15.67:39036) 2024-03-09T09:21:38.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.545+0000 7ff0dea00640 1 -- 172.21.15.67:0/3659538013 learned_addr learned my addr 172.21.15.67:0/3659538013 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:21:38.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0df400640 1 --2- 172.21.15.67:0/3659538013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e014fd90 0x7ff0e0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0dfe00640 1 --2- 172.21.15.67:0/3659538013 >> v2:172.21.15.134:3300/0 conn(0x7ff0e006c1d0 0x7ff0e006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668171298 0 0) 0x7ff0c8001110 con 0x7ff0e00c9380 2024-03-09T09:21:38.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0c4003520 con 0x7ff0e00c9380 2024-03-09T09:21:38.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3794873958 0 0) 0x7ff0c4003520 con 0x7ff0e00c9380 2024-03-09T09:21:38.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e014fd90 msgr2=0x7ff0e0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 --2- 172.21.15.67:0/3659538013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e014fd90 0x7ff0e0156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 >> v2:172.21.15.134:3300/0 conn(0x7ff0e006c1d0 msgr2=0x7ff0e006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 --2- 172.21.15.67:0/3659538013 >> v2:172.21.15.134:3300/0 conn(0x7ff0e006c1d0 0x7ff0e006c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00c5f80 con 0x7ff0e00c9380 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7ff0c8001840 con 0x7ff0e00c9380 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 >> v1:172.21.15.67:6789/0 conn(0x7ff0e00c9380 legacy=0x7ff0e00acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 --2- 172.21.15.67:0/3659538013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0c4003a20 0x7ff0c4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 --2- 172.21.15.67:0/3659538013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0c4004800 0x7ff0c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 --2- 172.21.15.67:0/3659538013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e014fd90 0x7ff0c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e006ba60 con 0x7ff0c4004800 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00c9890 con 0x7ff0c4003a20 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.546+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c400bac0 con 0x7ff0e014fd90 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.547+0000 7ff0dea00640 1 --2- 172.21.15.67:0/3659538013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0c4004800 0x7ff0c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.547+0000 7ff0dfe00640 1 --2- 172.21.15.67:0/3659538013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e014fd90 0x7ff0c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.547+0000 7ff0dea00640 1 -- 172.21.15.67:0/3659538013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e014fd90 msgr2=0x7ff0c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.547+0000 7ff0dea00640 1 --2- 172.21.15.67:0/3659538013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0e014fd90 0x7ff0c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.547+0000 7ff0dea00640 1 -- 172.21.15.67:0/3659538013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0c4003a20 msgr2=0x7ff0c4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.547+0000 7ff0dea00640 1 --2- 172.21.15.67:0/3659538013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0c4003a20 0x7ff0c4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.547+0000 7ff0dea00640 1 -- 172.21.15.67:0/3659538013 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4003520 con 0x7ff0c4004800 2024-03-09T09:21:38.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.549+0000 7ff0dea00640 1 --2- 172.21.15.67:0/3659538013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0c4004800 0x7ff0c4004c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00c97e0 tx=0x7ff0c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7405f48715db8347 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.549+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c800c0e0 con 0x7ff0c4004800 2024-03-09T09:21:38.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.549+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c8030070 con 0x7ff0c4004800 2024-03-09T09:21:38.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.549+0000 7ff0de000640 1 -- 172.21.15.67:0/3659538013 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c80058d0 con 0x7ff0c4004800 2024-03-09T09:21:38.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.550+0000 7ff0e5200640 1 -- 172.21.15.67:0/3659538013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0c4004800 msgr2=0x7ff0c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.550+0000 7ff0e5200640 1 --2- 172.21.15.67:0/3659538013 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0c4004800 0x7ff0c4004c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00c97e0 tx=0x7ff0c4005830 comp rx=0 tx=0).stop 2024-03-09T09:21:38.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.550+0000 7ff0dfe00640 1 -- 172.21.15.67:0/3659538013 reap_dead start 2024-03-09T09:21:38.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.550+0000 7ff0e5200640 1 -- 172.21.15.67:0/3659538013 shutdown_connections 2024-03-09T09:21:38.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.550+0000 7ff0e5200640 1 -- 172.21.15.67:0/3659538013 >> 172.21.15.67:0/3659538013 conn(0x7ff0e00cb480 msgr2=0x7ff0e00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:38.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.550+0000 7ff0e5200640 1 -- 172.21.15.67:0/3659538013 shutdown_connections 2024-03-09T09:21:38.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.550+0000 7ff0e5200640 1 -- 172.21.15.67:0/3659538013 wait complete. 2024-03-09T09:21:38.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.553+0000 7ff0e5200640 1 Processor -- start 2024-03-09T09:21:38.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.553+0000 7ff0e5200640 1 -- start start 2024-03-09T09:21:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.555+0000 7ff0e5200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0c4004800 0x7ff0e00acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.555+0000 7ff0e5200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01ef7e0 0x7ff0e01ec030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.555+0000 7ff0e5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e01ec570 0x7ff0e0148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.555+0000 7ff0e5200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e006bcb0 con 0x7ff0e01ec570 2024-03-09T09:21:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.555+0000 7ff0e5200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00c6520 con 0x7ff0e01ef7e0 2024-03-09T09:21:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.555+0000 7ff0e5200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00ad610 con 0x7ff0c4004800 2024-03-09T09:21:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.555+0000 7ff0dfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e01ec570 0x7ff0e0148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.555+0000 7ff0dfe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e01ec570 0x7ff0e0148440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40076/0 (socket says 172.21.15.67:40076) 2024-03-09T09:21:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.555+0000 7ff0dfe00640 1 -- 172.21.15.67:0/1420470999 learned_addr learned my addr 172.21.15.67:0/1420470999 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.555+0000 7ff0df400640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0c4004800 0x7ff0e00acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.556+0000 7ff0dea00640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01ef7e0 0x7ff0e01ec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.556+0000 7ff0dfe00640 1 -- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0c4004800 msgr2=0x7ff0e00acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.556+0000 7ff0dfe00640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0c4004800 0x7ff0e00acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.556+0000 7ff0dfe00640 1 -- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01ef7e0 msgr2=0x7ff0e01ec030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.556+0000 7ff0dfe00640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01ef7e0 0x7ff0e01ec030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.556+0000 7ff0dfe00640 1 -- 172.21.15.67:0/1420470999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e00b0230 con 0x7ff0e01ec570 2024-03-09T09:21:38.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.556+0000 7ff0dfe00640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e01ec570 0x7ff0e0148440 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff0e006d2a0 tx=0x7ff0d8011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.556+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8019020 con 0x7ff0e01ec570 2024-03-09T09:21:38.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.556+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d8003610 con 0x7ff0e01ec570 2024-03-09T09:21:38.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.557+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8003a40 con 0x7ff0e01ec570 2024-03-09T09:21:38.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.557+0000 7ff0e5200640 1 -- 172.21.15.67:0/1420470999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e00b0510 con 0x7ff0e01ec570 2024-03-09T09:21:38.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.557+0000 7ff0e5200640 1 -- 172.21.15.67:0/1420470999 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e00b0ad0 con 0x7ff0e01ec570 2024-03-09T09:21:38.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.558+0000 7ff0e5200640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01efbc0 0x7ff0e0148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.558+0000 7ff0e5200640 1 -- 172.21.15.67:0/1420470999 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00b1490 con 0x7ff0e01efbc0 2024-03-09T09:21:38.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.559+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff0d801c020 con 0x7ff0e01ec570 2024-03-09T09:21:38.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.559+0000 7ff0df400640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01efbc0 0x7ff0e0148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.560+0000 7ff0df400640 1 -- 172.21.15.67:0/1420470999 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0e0148ec0 con 0x7ff0e01efbc0 2024-03-09T09:21:38.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.560+0000 7ff0d7400640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0bc06b4d0 0x7ff0bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.560+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff0d808a7e0 con 0x7ff0e01ec570 2024-03-09T09:21:38.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.560+0000 7ff0dea00640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0bc06b4d0 0x7ff0bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.560+0000 7ff0df400640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01efbc0 0x7ff0e0148980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009380 tx=0x7ff0d000f500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.561+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d001a070 con 0x7ff0e01efbc0 2024-03-09T09:21:38.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.562+0000 7ff0dea00640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0bc06b4d0 0x7ff0bc06d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40057e0 tx=0x7ff0c8007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.562+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff0d001f450 con 0x7ff0e01efbc0 2024-03-09T09:21:38.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.562+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01efbc0 msgr2=0x7ff0e0148980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.562+0000 7ff0d7400640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01efbc0 0x7ff0e0148980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009380 tx=0x7ff0d000f500 comp rx=0 tx=0).stop 2024-03-09T09:21:38.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.562+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01efbc0 msgr2=0x7ff0e0148980 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:38.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.562+0000 7ff0d7400640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01efbc0 0x7ff0e0148980 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.569+0000 7ff0e5200640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01eba00 0x7ff0e0159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.569+0000 7ff0e5200640 1 -- 172.21.15.67:0/1420470999 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e004f3c0 con 0x7ff0e01eba00 2024-03-09T09:21:38.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.570+0000 7ff0df400640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01eba00 0x7ff0e0159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.571+0000 7ff0df400640 1 -- 172.21.15.67:0/1420470999 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff0e006d2d0 con 0x7ff0e01eba00 2024-03-09T09:21:38.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.572+0000 7ff0df400640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01eba00 0x7ff0e0159380 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0d001c000 tx=0x7ff0d0009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.572+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0016ae0 con 0x7ff0e01eba00 2024-03-09T09:21:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.573+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7ff0d0009e50 con 0x7ff0e01eba00 2024-03-09T09:21:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.573+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01eba00 msgr2=0x7ff0e0159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.573+0000 7ff0d7400640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01eba00 0x7ff0e0159380 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0d001c000 tx=0x7ff0d0009750 comp rx=0 tx=0).stop 2024-03-09T09:21:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.573+0000 7ff0d7400640 1 -- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01eba00 msgr2=0x7ff0e0159380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.573+0000 7ff0d7400640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0e01eba00 0x7ff0e0159380 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0d001c000 tx=0x7ff0d0009750 comp rx=0 tx=0).stop 2024-03-09T09:21:38.580 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:21:38.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:21:38.580 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:21:38.580 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 226, 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:38.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:38.582 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:21:38.583 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:21:38.584 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.585 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:21:38.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.586 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:21:38.586 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:21:38.586 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:38.586 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:38.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.586 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:38.586 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:21:38.586 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.587 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:38.587 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:21:38.587 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.587 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.587 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.587 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:21:38.587 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:21:38.590 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:21:38.591 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:21:38.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.593 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:21:38.594 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:21:38.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.574+0000 7ff0e5200640 1 -- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0bc06b4d0 msgr2=0x7ff0bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.574+0000 7ff0e5200640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0bc06b4d0 0x7ff0bc06d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40057e0 tx=0x7ff0c8007000 comp rx=0 tx=0).stop 2024-03-09T09:21:38.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.574+0000 7ff0e5200640 1 -- 172.21.15.67:0/1420470999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e01ec570 msgr2=0x7ff0e0148440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.574+0000 7ff0e5200640 1 --2- 172.21.15.67:0/1420470999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e01ec570 0x7ff0e0148440 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff0e006d2a0 tx=0x7ff0d8011860 comp rx=0 tx=0).stop 2024-03-09T09:21:38.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.574+0000 7ff0dfe00640 1 -- 172.21.15.67:0/1420470999 reap_dead start 2024-03-09T09:21:38.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.574+0000 7ff0dfe00640 1 -- 172.21.15.67:0/1420470999 reap_dead start 2024-03-09T09:21:38.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.574+0000 7ff0e5200640 1 -- 172.21.15.67:0/1420470999 shutdown_connections 2024-03-09T09:21:38.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.574+0000 7ff0e5200640 1 -- 172.21.15.67:0/1420470999 >> 172.21.15.67:0/1420470999 conn(0x7ff0e00cb480 msgr2=0x7ff0e014d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:38.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.574+0000 7ff0e5200640 1 -- 172.21.15.67:0/1420470999 shutdown_connections 2024-03-09T09:21:38.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.574+0000 7ff0e5200640 1 -- 172.21.15.67:0/1420470999 wait complete. 2024-03-09T09:21:38.599 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:21:38.600 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:21:38.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:21:38.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.805+0000 7f7538c00640 1 Processor -- start 2024-03-09T09:21:38.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.805+0000 7f7538c00640 1 -- start start 2024-03-09T09:21:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.806+0000 7f7538c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7534157890 0x7f753406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.806+0000 7f7538c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f753406e630 0x7f753415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.806+0000 7f7538c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753415a370 con 0x7f753406dad0 2024-03-09T09:21:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.806+0000 7f7538c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753415b130 con 0x7f7534157890 2024-03-09T09:21:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.806+0000 7f7538c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753415a650 con 0x7f753406e630 2024-03-09T09:21:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.807+0000 7f7531e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7534157890 0x7f753406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.807+0000 7f7532800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f753406dad0 0x7f753415bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39048/0 (socket says 172.21.15.67:39048) 2024-03-09T09:21:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.807+0000 7f7532800640 1 -- 172.21.15.67:0/1369860339 learned_addr learned my addr 172.21.15.67:0/1369860339 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:21:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.807+0000 7f7531400640 1 --2- 172.21.15.67:0/1369860339 >> v2:172.21.15.134:3300/0 conn(0x7f753406e630 0x7f753415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.807+0000 7f7531e00640 1 -- 172.21.15.67:0/1369860339 >> v2:172.21.15.134:3300/0 conn(0x7f753406e630 msgr2=0x7f753415b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.807+0000 7f7531e00640 1 --2- 172.21.15.67:0/1369860339 >> v2:172.21.15.134:3300/0 conn(0x7f753406e630 0x7f753415b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.807+0000 7f7531e00640 1 -- 172.21.15.67:0/1369860339 >> v1:172.21.15.67:6789/0 conn(0x7f753406dad0 legacy=0x7f753415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.807+0000 7f7531e00640 1 -- 172.21.15.67:0/1369860339 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753415a930 con 0x7f7534157890 2024-03-09T09:21:38.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.809+0000 7f7531e00640 1 --2- 172.21.15.67:0/1369860339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7534157890 0x7f753406d590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7534070320 tx=0x7f752402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be87ef26a2aee6e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.809+0000 7f7530a00640 1 -- 172.21.15.67:0/1369860339 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524034070 con 0x7f7534157890 2024-03-09T09:21:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.809+0000 7f7530a00640 1 -- 172.21.15.67:0/1369860339 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7524030610 con 0x7f7534157890 2024-03-09T09:21:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.809+0000 7f7530a00640 1 -- 172.21.15.67:0/1369860339 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75240309e0 con 0x7f7534157890 2024-03-09T09:21:38.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.810+0000 7f7538c00640 1 -- 172.21.15.67:0/1369860339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7534157890 msgr2=0x7f753406d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.810+0000 7f7538c00640 1 --2- 172.21.15.67:0/1369860339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7534157890 0x7f753406d590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7534070320 tx=0x7f752402b960 comp rx=0 tx=0).stop 2024-03-09T09:21:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.810+0000 7f7538c00640 1 -- 172.21.15.67:0/1369860339 shutdown_connections 2024-03-09T09:21:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.810+0000 7f7538c00640 1 --2- 172.21.15.67:0/1369860339 >> v2:172.21.15.134:3300/0 conn(0x7f753406e630 0x7f753415b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.810+0000 7f7538c00640 1 --2- 172.21.15.67:0/1369860339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7534157890 0x7f753406d590 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.810+0000 7f7538c00640 1 -- 172.21.15.67:0/1369860339 >> 172.21.15.67:0/1369860339 conn(0x7f75340c7370 msgr2=0x7f75340c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.810+0000 7f7538c00640 1 -- 172.21.15.67:0/1369860339 shutdown_connections 2024-03-09T09:21:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.810+0000 7f7538c00640 1 -- 172.21.15.67:0/1369860339 wait complete. 2024-03-09T09:21:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.812+0000 7f7538c00640 1 Processor -- start 2024-03-09T09:21:38.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.812+0000 7f7538c00640 1 -- start start 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.813+0000 7f7538c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f753406dad0 0x7f75340a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.813+0000 7f7538c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753406e630 0x7f75340a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.813+0000 7f7538c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7534157890 0x7f7534154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.813+0000 7f7538c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75340c1ef0 con 0x7f753406dad0 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.813+0000 7f7538c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75340c1d70 con 0x7f753406e630 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.813+0000 7f7538c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7534157c70 con 0x7f7534157890 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7531e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f753406dad0 0x7f75340a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7531e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f753406dad0 0x7f75340a6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40092/0 (socket says 172.21.15.67:40092) 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7531e00640 1 -- 172.21.15.67:0/3975477357 learned_addr learned my addr 172.21.15.67:0/3975477357 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7531400640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753406e630 0x7f75340a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7532800640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7534157890 0x7f7534154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7531e00640 1 -- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7534157890 msgr2=0x7f7534154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7531e00640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7534157890 0x7f7534154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7531e00640 1 -- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753406e630 msgr2=0x7f75340a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7531e00640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753406e630 0x7f75340a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7531e00640 1 -- 172.21.15.67:0/3975477357 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75340a8680 con 0x7f753406dad0 2024-03-09T09:21:38.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7531400640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753406e630 0x7f75340a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:21:38.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.814+0000 7f7531e00640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f753406dad0 0x7f75340a6e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7524009890 tx=0x7f75240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.815+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524034040 con 0x7f753406dad0 2024-03-09T09:21:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.815+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752403acc0 con 0x7f753406dad0 2024-03-09T09:21:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.815+0000 7f7538c00640 1 -- 172.21.15.67:0/3975477357 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75340bc530 con 0x7f753406dad0 2024-03-09T09:21:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.815+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524044640 con 0x7f753406dad0 2024-03-09T09:21:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.815+0000 7f7538c00640 1 -- 172.21.15.67:0/3975477357 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75340a8920 con 0x7f753406dad0 2024-03-09T09:21:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.817+0000 7f7538c00640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340a4790 0x7f75341553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.817+0000 7f7538c00640 1 -- 172.21.15.67:0/3975477357 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75340a8b60 con 0x7f75340a4790 2024-03-09T09:21:38.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.817+0000 7f7531400640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340a4790 0x7f75341553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.817+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7524037070 con 0x7f753406dad0 2024-03-09T09:21:38.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.818+0000 7f7531400640 1 -- 172.21.15.67:0/3975477357 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75340bca50 con 0x7f75340a4790 2024-03-09T09:21:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.818+0000 7f7531400640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340a4790 0x7f75341553b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f75340735e0 tx=0x7f751c007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.818+0000 7f752a000640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f751006b4d0 0x7f751006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.818+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f75240ac4a0 con 0x7f753406dad0 2024-03-09T09:21:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.819+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c014610 con 0x7f75340a4790 2024-03-09T09:21:38.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.819+0000 7f7532800640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f751006b4d0 0x7f751006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.819+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f751c019070 con 0x7f75340a4790 2024-03-09T09:21:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.819+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340a4790 msgr2=0x7f75341553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.819+0000 7f752a000640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340a4790 0x7f75341553b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f75340735e0 tx=0x7f751c007700 comp rx=0 tx=0).stop 2024-03-09T09:21:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.819+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340a4790 msgr2=0x7f75341553b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.819+0000 7f752a000640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340a4790 0x7f75341553b0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f75340735e0 tx=0x7f751c007700 comp rx=0 tx=0).stop 2024-03-09T09:21:38.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.820+0000 7f7532800640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f751006b4d0 0x7f751006d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f752c005880 tx=0x7f752c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.827+0000 7f7538c00640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340c1160 0x7f75341f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:38.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.827+0000 7f7538c00640 1 -- 172.21.15.67:0/3975477357 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75340c53a0 con 0x7f75340c1160 2024-03-09T09:21:38.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.827+0000 7f7531400640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340c1160 0x7f75341f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:38.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.828+0000 7f7531400640 1 -- 172.21.15.67:0/3975477357 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f75340a7370 con 0x7f75340c1160 2024-03-09T09:21:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.828+0000 7f7531400640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340c1160 0x7f75341f2bc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7534073870 tx=0x7f751c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:38.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.829+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c01e220 con 0x7f75340c1160 2024-03-09T09:21:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.829+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7f751c01e220 con 0x7f75340c1160 2024-03-09T09:21:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.829+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340c1160 msgr2=0x7f75341f2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.829+0000 7f752a000640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340c1160 0x7f75341f2bc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7534073870 tx=0x7f751c004a40 comp rx=0 tx=0).stop 2024-03-09T09:21:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.829+0000 7f752a000640 1 -- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340c1160 msgr2=0x7f75341f2bc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.829+0000 7f752a000640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75340c1160 0x7f75341f2bc0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7534073870 tx=0x7f751c004a40 comp rx=0 tx=0).stop 2024-03-09T09:21:38.836 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 226, 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:21:38.837 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.838 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:38.839 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:21:38.840 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:38.841 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:21:38.842 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:38.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:21:38.844 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:38.845 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:21:38.846 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.847 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:38.849 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:38.850 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:38.850 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:38.850 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:38.850 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:38.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:38.850 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:21:38.850 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:21:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.830+0000 7f7538c00640 1 -- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f751006b4d0 msgr2=0x7f751006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.830+0000 7f7538c00640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f751006b4d0 0x7f751006d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f752c005880 tx=0x7f752c009040 comp rx=0 tx=0).stop 2024-03-09T09:21:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.830+0000 7f7538c00640 1 -- 172.21.15.67:0/3975477357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f753406dad0 msgr2=0x7f75340a6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.830+0000 7f7538c00640 1 --2- 172.21.15.67:0/3975477357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f753406dad0 0x7f75340a6e30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7524009890 tx=0x7f75240046c0 comp rx=0 tx=0).stop 2024-03-09T09:21:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.830+0000 7f7532800640 1 -- 172.21.15.67:0/3975477357 reap_dead start 2024-03-09T09:21:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.830+0000 7f7532800640 1 -- 172.21.15.67:0/3975477357 reap_dead start 2024-03-09T09:21:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.830+0000 7f7538c00640 1 -- 172.21.15.67:0/3975477357 shutdown_connections 2024-03-09T09:21:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.830+0000 7f7538c00640 1 -- 172.21.15.67:0/3975477357 >> 172.21.15.67:0/3975477357 conn(0x7f75340c7370 msgr2=0x7f75340c2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:38.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.830+0000 7f7538c00640 1 -- 172.21.15.67:0/3975477357 shutdown_connections 2024-03-09T09:21:38.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:38.830+0000 7f7538c00640 1 -- 172.21.15.67:0/3975477357 wait complete. 2024-03-09T09:21:38.855 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:21:38.855 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:21:38.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:21:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.047+0000 7fea6c800640 1 Processor -- start 2024-03-09T09:21:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.047+0000 7fea6c800640 1 -- start start 2024-03-09T09:21:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.048+0000 7fea6c800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fea640ac730 0x7fea64071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.048+0000 7fea6c800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea640c22a0 0x7fea640acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.048+0000 7fea6c800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea64153d40 con 0x7fea64157650 2024-03-09T09:21:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.049+0000 7fea6c800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea64070ee0 con 0x7fea640ac730 2024-03-09T09:21:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.049+0000 7fea6c800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea64154d00 con 0x7fea640c22a0 2024-03-09T09:21:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.049+0000 7fea69e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fea640ac730 0x7fea64071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.049+0000 7fea6a800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fea64157650 0x7fea64157b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38970/0 (socket says 172.21.15.67:38970) 2024-03-09T09:21:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.049+0000 7fea6a800640 1 -- 172.21.15.67:0/3537945704 learned_addr learned my addr 172.21.15.67:0/3537945704 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:21:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.049+0000 7fea69e00640 1 --2- 172.21.15.67:0/3537945704 >> v2:172.21.15.67:3300/0 conn(0x7fea640ac730 0x7fea64071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36534/0 (socket says 172.21.15.67:36534) 2024-03-09T09:21:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.049+0000 7fea69400640 1 --2- 172.21.15.67:0/3537945704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea640c22a0 0x7fea640acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.049+0000 7fea69e00640 1 -- 172.21.15.67:0/3537945704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea640c22a0 msgr2=0x7fea640acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.049+0000 7fea69e00640 1 --2- 172.21.15.67:0/3537945704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea640c22a0 0x7fea640acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.049+0000 7fea69e00640 1 -- 172.21.15.67:0/3537945704 >> v1:172.21.15.67:6789/0 conn(0x7fea64157650 legacy=0x7fea64157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.050+0000 7fea69e00640 1 -- 172.21.15.67:0/3537945704 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea640c2790 con 0x7fea640ac730 2024-03-09T09:21:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.049+0000 7fea68a00640 1 -- 172.21.15.67:0/3537945704 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332804077 0 0) 0x7fea64153d40 con 0x7fea64157650 2024-03-09T09:21:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.051+0000 7fea69e00640 1 --2- 172.21.15.67:0/3537945704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640ac730 0x7fea64071190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea580097a0 tx=0x7fea5802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2922d7babc7041 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.051+0000 7fea68a00640 1 -- 172.21.15.67:0/3537945704 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea580315a0 con 0x7fea640ac730 2024-03-09T09:21:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.051+0000 7fea68a00640 1 -- 172.21.15.67:0/3537945704 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea58037070 con 0x7fea640ac730 2024-03-09T09:21:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.051+0000 7fea68a00640 1 -- 172.21.15.67:0/3537945704 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58031960 con 0x7fea640ac730 2024-03-09T09:21:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.052+0000 7fea6c800640 1 -- 172.21.15.67:0/3537945704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640ac730 msgr2=0x7fea64071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.052+0000 7fea6c800640 1 --2- 172.21.15.67:0/3537945704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640ac730 0x7fea64071190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea580097a0 tx=0x7fea5802b840 comp rx=0 tx=0).stop 2024-03-09T09:21:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.052+0000 7fea6c800640 1 -- 172.21.15.67:0/3537945704 shutdown_connections 2024-03-09T09:21:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.052+0000 7fea6c800640 1 --2- 172.21.15.67:0/3537945704 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea640c22a0 0x7fea640acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.052+0000 7fea6c800640 1 --2- 172.21.15.67:0/3537945704 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640ac730 0x7fea64071190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.052+0000 7fea6c800640 1 -- 172.21.15.67:0/3537945704 >> 172.21.15.67:0/3537945704 conn(0x7fea640c7370 msgr2=0x7fea640c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.052+0000 7fea6c800640 1 -- 172.21.15.67:0/3537945704 shutdown_connections 2024-03-09T09:21:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.052+0000 7fea6c800640 1 -- 172.21.15.67:0/3537945704 wait complete. 2024-03-09T09:21:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.053+0000 7fea6c800640 1 Processor -- start 2024-03-09T09:21:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.054+0000 7fea6c800640 1 -- start start 2024-03-09T09:21:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.054+0000 7fea6c800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea640ac730 0x7fea641465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.054+0000 7fea6c800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640c22a0 0x7fea64146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.054+0000 7fea6c800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea64157650 0x7fea6409be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.054+0000 7fea6c800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea64151610 con 0x7fea640c22a0 2024-03-09T09:21:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.054+0000 7fea6c800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea64070c00 con 0x7fea64157650 2024-03-09T09:21:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.054+0000 7fea6c800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea64151790 con 0x7fea640ac730 2024-03-09T09:21:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.055+0000 7fea69400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640c22a0 0x7fea64146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.055+0000 7fea69400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640c22a0 0x7fea64146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36536/0 (socket says 172.21.15.67:36536) 2024-03-09T09:21:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.055+0000 7fea69400640 1 -- 172.21.15.67:0/1051769856 learned_addr learned my addr 172.21.15.67:0/1051769856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.055+0000 7fea69e00640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea640ac730 0x7fea641465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.055+0000 7fea6a800640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea64157650 0x7fea6409be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.055+0000 7fea69400640 1 -- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea640ac730 msgr2=0x7fea641465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.055+0000 7fea69400640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea640ac730 0x7fea641465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.055+0000 7fea69400640 1 -- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea64157650 msgr2=0x7fea6409be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.055+0000 7fea69400640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea64157650 0x7fea6409be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.055+0000 7fea69400640 1 -- 172.21.15.67:0/1051769856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea640c0370 con 0x7fea640c22a0 2024-03-09T09:21:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.056+0000 7fea69400640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640c22a0 0x7fea64146ae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea5400d800 tx=0x7fea5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.056+0000 7fea5e000640 1 -- 172.21.15.67:0/1051769856 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5400b6c0 con 0x7fea640c22a0 2024-03-09T09:21:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.056+0000 7fea5e000640 1 -- 172.21.15.67:0/1051769856 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea54019070 con 0x7fea640c22a0 2024-03-09T09:21:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.056+0000 7fea5e000640 1 -- 172.21.15.67:0/1051769856 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5400ba80 con 0x7fea640c22a0 2024-03-09T09:21:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.056+0000 7fea6c800640 1 -- 172.21.15.67:0/1051769856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea640bf5b0 con 0x7fea640c22a0 2024-03-09T09:21:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.056+0000 7fea6c800640 1 -- 172.21.15.67:0/1051769856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea640bfb70 con 0x7fea640c22a0 2024-03-09T09:21:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.057+0000 7fea6c800640 1 -- 172.21.15.67:0/1051769856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea64075a00 con 0x7fea640c22a0 2024-03-09T09:21:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.058+0000 7fea5e000640 1 -- 172.21.15.67:0/1051769856 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fea5400bc20 con 0x7fea640c22a0 2024-03-09T09:21:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.059+0000 7fea5e000640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea4806b4d0 0x7fea4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.059+0000 7fea5e000640 1 -- 172.21.15.67:0/1051769856 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fea5408a9c0 con 0x7fea640c22a0 2024-03-09T09:21:39.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.060+0000 7fea69e00640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea4806b4d0 0x7fea4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.061+0000 7fea69e00640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea4806b4d0 0x7fea4806d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fea58036000 tx=0x7fea58008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.069+0000 7fea5e000640 1 -- 172.21.15.67:0/1051769856 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fea5408ce20 con 0x7fea640c22a0 2024-03-09T09:21:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.345+0000 7fea6c800640 1 -- 172.21.15.67:0/1051769856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fea64154620 con 0x7fea640c22a0 2024-03-09T09:21:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.346+0000 7fea5e000640 1 -- 172.21.15.67:0/1051769856 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fea5401c360 con 0x7fea640c22a0 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 -- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea4806b4d0 msgr2=0x7fea4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea4806b4d0 0x7fea4806d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fea58036000 tx=0x7fea58008030 comp rx=0 tx=0).stop 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 -- 172.21.15.67:0/1051769856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640c22a0 msgr2=0x7fea64146ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640c22a0 0x7fea64146ae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea5400d800 tx=0x7fea5400dcc0 comp rx=0 tx=0).stop 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 -- 172.21.15.67:0/1051769856 shutdown_connections 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fea4806b4d0 0x7fea4806d990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea64157650 0x7fea6409be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea640c22a0 0x7fea64146ae0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 --2- 172.21.15.67:0/1051769856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea640ac730 0x7fea641465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 -- 172.21.15.67:0/1051769856 >> 172.21.15.67:0/1051769856 conn(0x7fea640c7370 msgr2=0x7fea64157b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 -- 172.21.15.67:0/1051769856 shutdown_connections 2024-03-09T09:21:39.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:39.349+0000 7fea6c800640 1 -- 172.21.15.67:0/1051769856 wait complete. 2024-03-09T09:21:39.378 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:21:42.034 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:21:42.034 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:42.034 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 56 -eq 48 ']' 2024-03-09T09:21:42.034 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.56 2024-03-09T09:21:42.035 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.56 2024-03-09T09:21:42.035 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.56 2024-03-09T09:21:42.035 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.56 2024-03-09T09:21:42.038 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:21:43.473 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:21:43.473 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:21:43.473 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 62 -eq 48 ']' 2024-03-09T09:21:43.474 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.62 2024-03-09T09:21:43.474 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.62 2024-03-09T09:21:43.474 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.62 2024-03-09T09:21:43.474 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.62 2024-03-09T09:21:43.478 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:21:48.147 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:21:48.147 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:21:48.148 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 63 -eq 48 ']' 2024-03-09T09:21:48.148 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.63 2024-03-09T09:21:48.148 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.63 2024-03-09T09:21:48.148 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.63 2024-03-09T09:21:48.149 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.63 2024-03-09T09:21:48.151 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:21:48.404 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:21:48.404 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:48.405 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 57 -eq 48 ']' 2024-03-09T09:21:48.405 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.57 2024-03-09T09:21:48.405 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.57 2024-03-09T09:21:48.405 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.57 2024-03-09T09:21:48.406 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.57 2024-03-09T09:21:48.409 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:21:49.378 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:21:49.379 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:21:49.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.605+0000 7f9cb7200640 1 Processor -- start 2024-03-09T09:21:49.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.606+0000 7f9cb7200640 1 -- start start 2024-03-09T09:21:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.607+0000 7f9cb7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cb80c3080 0x7f9cb81527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.607+0000 7f9cb7200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9cb806d420 0x7f9cb814c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.607+0000 7f9cb7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9cb814c820 0x7f9cb81540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.607+0000 7f9cb7200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb8155ba0 con 0x7f9cb814c820 2024-03-09T09:21:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.607+0000 7f9cb7200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb814ccb0 con 0x7f9cb806d420 2024-03-09T09:21:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.607+0000 7f9cb7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb806d920 con 0x7f9cb80c3080 2024-03-09T09:21:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.608+0000 7f9cb6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9cb814c820 0x7f9cb81540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.608+0000 7f9cb6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9cb814c820 0x7f9cb81540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56826/0 (socket says 172.21.15.67:56826) 2024-03-09T09:21:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.608+0000 7f9cb6800640 1 -- 172.21.15.67:0/3396890747 learned_addr learned my addr 172.21.15.67:0/3396890747 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.608+0000 7f9cb5e00640 1 --2- 172.21.15.67:0/3396890747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cb80c3080 0x7f9cb81527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.608+0000 7f9cb5400640 1 --2- 172.21.15.67:0/3396890747 >> v2:172.21.15.134:3300/0 conn(0x7f9cb806d420 0x7f9cb814c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:49.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.608+0000 7f9cb6800640 1 -- 172.21.15.67:0/3396890747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cb80c3080 msgr2=0x7f9cb81527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.608+0000 7f9cb6800640 1 --2- 172.21.15.67:0/3396890747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cb80c3080 0x7f9cb81527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.608+0000 7f9cb6800640 1 -- 172.21.15.67:0/3396890747 >> v2:172.21.15.134:3300/0 conn(0x7f9cb806d420 msgr2=0x7f9cb814c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.608+0000 7f9cb6800640 1 --2- 172.21.15.67:0/3396890747 >> v2:172.21.15.134:3300/0 conn(0x7f9cb806d420 0x7f9cb814c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.608+0000 7f9cb6800640 1 -- 172.21.15.67:0/3396890747 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb806dc00 con 0x7f9cb814c820 2024-03-09T09:21:49.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.608+0000 7f9cb5400640 1 --2- 172.21.15.67:0/3396890747 >> v2:172.21.15.134:3300/0 conn(0x7f9cb806d420 0x7f9cb814c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:21:49.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.609+0000 7f9cb6800640 1 --2- 172.21.15.67:0/3396890747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb814c820 0x7f9cb81540a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9cb806bfd0 tx=0x7f9cb002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c112d62073079bc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:49.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.610+0000 7f9cb4a00640 1 -- 172.21.15.67:0/3396890747 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb002c9e0 con 0x7f9cb814c820 2024-03-09T09:21:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.610+0000 7f9cb4a00640 1 -- 172.21.15.67:0/3396890747 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb002cb80 con 0x7f9cb814c820 2024-03-09T09:21:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.610+0000 7f9cb4a00640 1 -- 172.21.15.67:0/3396890747 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00343f0 con 0x7f9cb814c820 2024-03-09T09:21:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.610+0000 7f9cb7200640 1 -- 172.21.15.67:0/3396890747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb814c820 msgr2=0x7f9cb81540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.610+0000 7f9cb7200640 1 --2- 172.21.15.67:0/3396890747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb814c820 0x7f9cb81540a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9cb806bfd0 tx=0x7f9cb002be40 comp rx=0 tx=0).stop 2024-03-09T09:21:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.611+0000 7f9cb7200640 1 -- 172.21.15.67:0/3396890747 shutdown_connections 2024-03-09T09:21:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.611+0000 7f9cb7200640 1 --2- 172.21.15.67:0/3396890747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb814c820 0x7f9cb81540a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.611+0000 7f9cb7200640 1 --2- 172.21.15.67:0/3396890747 >> v2:172.21.15.134:3300/0 conn(0x7f9cb806d420 0x7f9cb814c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.611+0000 7f9cb7200640 1 --2- 172.21.15.67:0/3396890747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cb80c3080 0x7f9cb81527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.611+0000 7f9cb7200640 1 -- 172.21.15.67:0/3396890747 >> 172.21.15.67:0/3396890747 conn(0x7f9cb80c7300 msgr2=0x7f9cb80c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:49.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.611+0000 7f9cb7200640 1 -- 172.21.15.67:0/3396890747 shutdown_connections 2024-03-09T09:21:49.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.611+0000 7f9cb7200640 1 -- 172.21.15.67:0/3396890747 wait complete. 2024-03-09T09:21:49.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.612+0000 7f9cb7200640 1 Processor -- start 2024-03-09T09:21:49.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.612+0000 7f9cb7200640 1 -- start start 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.613+0000 7f9cb7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb806d420 0x7f9cb8152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.613+0000 7f9cb7200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cb80c3080 0x7f9cb8147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.613+0000 7f9cb7200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cb814c820 0x7f9cb8147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.613+0000 7f9cb7200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb80c8030 con 0x7f9cb806d420 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.613+0000 7f9cb7200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb806e1c0 con 0x7f9cb814c820 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.613+0000 7f9cb7200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb80ac190 con 0x7f9cb80c3080 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.614+0000 7f9cb5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb806d420 0x7f9cb8152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.614+0000 7f9cb5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb806d420 0x7f9cb8152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56840/0 (socket says 172.21.15.67:56840) 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.614+0000 7f9cb5e00640 1 -- 172.21.15.67:0/699737817 learned_addr learned my addr 172.21.15.67:0/699737817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.614+0000 7f9cb6800640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cb814c820 0x7f9cb8147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.614+0000 7f9cb5400640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cb80c3080 0x7f9cb8147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.614+0000 7f9cb5e00640 1 -- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cb80c3080 msgr2=0x7f9cb8147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.614+0000 7f9cb5e00640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cb80c3080 0x7f9cb8147340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.614+0000 7f9cb5e00640 1 -- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cb814c820 msgr2=0x7f9cb8147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.614+0000 7f9cb5e00640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cb814c820 0x7f9cb8147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.614+0000 7f9cb5e00640 1 -- 172.21.15.67:0/699737817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb8147dc0 con 0x7f9cb806d420 2024-03-09T09:21:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.615+0000 7f9cb5e00640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb806d420 0x7f9cb8152370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800d800 tx=0x7f9ca800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.615+0000 7f9cae000640 1 -- 172.21.15.67:0/699737817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800b6c0 con 0x7f9cb806d420 2024-03-09T09:21:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.615+0000 7f9cae000640 1 -- 172.21.15.67:0/699737817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca8019070 con 0x7f9cb806d420 2024-03-09T09:21:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.615+0000 7f9cb7200640 1 -- 172.21.15.67:0/699737817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb80ac400 con 0x7f9cb806d420 2024-03-09T09:21:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.615+0000 7f9cae000640 1 -- 172.21.15.67:0/699737817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800ba80 con 0x7f9cb806d420 2024-03-09T09:21:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.615+0000 7f9cb7200640 1 -- 172.21.15.67:0/699737817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb81528b0 con 0x7f9cb806d420 2024-03-09T09:21:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.616+0000 7f9cb7200640 1 -- 172.21.15.67:0/699737817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb80ac920 con 0x7f9cb806d420 2024-03-09T09:21:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.617+0000 7f9cae000640 1 -- 172.21.15.67:0/699737817 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9ca800bc20 con 0x7f9cb806d420 2024-03-09T09:21:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.619+0000 7f9cae000640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c9806b500 0x7f9c9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.619+0000 7f9cae000640 1 -- 172.21.15.67:0/699737817 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9ca808a9c0 con 0x7f9cb806d420 2024-03-09T09:21:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.620+0000 7f9cb5400640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c9806b500 0x7f9c9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:49.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.621+0000 7f9cb5400640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c9806b500 0x7f9c9806d9c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80728f0 tx=0x7f9ca4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.624+0000 7f9cae000640 1 -- 172.21.15.67:0/699737817 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9ca8059440 con 0x7f9cb806d420 2024-03-09T09:21:49.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.921+0000 7f9cb7200640 1 -- 172.21.15.67:0/699737817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9cb804f3c0 con 0x7f9cb806d420 2024-03-09T09:21:49.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.922+0000 7f9cae000640 1 -- 172.21.15.67:0/699737817 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f9ca8058d40 con 0x7f9cb806d420 2024-03-09T09:21:49.928 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":226,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:21:49.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.924+0000 7f9cb7200640 1 -- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c9806b500 msgr2=0x7f9c9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:49.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.924+0000 7f9cb7200640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c9806b500 0x7f9c9806d9c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80728f0 tx=0x7f9ca4008040 comp rx=0 tx=0).stop 2024-03-09T09:21:49.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.924+0000 7f9cb7200640 1 -- 172.21.15.67:0/699737817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb806d420 msgr2=0x7f9cb8152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:49.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.924+0000 7f9cb7200640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb806d420 0x7f9cb8152370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800d800 tx=0x7f9ca800dcc0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.924+0000 7f9cb7200640 1 -- 172.21.15.67:0/699737817 shutdown_connections 2024-03-09T09:21:49.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.924+0000 7f9cb7200640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9c9806b500 0x7f9c9806d9c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.924+0000 7f9cb7200640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cb814c820 0x7f9cb8147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.924+0000 7f9cb7200640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9cb80c3080 0x7f9cb8147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.924+0000 7f9cb7200640 1 --2- 172.21.15.67:0/699737817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9cb806d420 0x7f9cb8152370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:49.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.924+0000 7f9cb7200640 1 -- 172.21.15.67:0/699737817 >> 172.21.15.67:0/699737817 conn(0x7f9cb80c7300 msgr2=0x7f9cb814aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:49.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.925+0000 7f9cb7200640 1 -- 172.21.15.67:0/699737817 shutdown_connections 2024-03-09T09:21:49.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:49.925+0000 7f9cb7200640 1 -- 172.21.15.67:0/699737817 wait complete. 2024-03-09T09:21:49.953 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:21:49.953 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:21:49.953 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:21:49.953 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:21:50.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.180+0000 7fb948a00640 1 Processor -- start 2024-03-09T09:21:50.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.181+0000 7fb948a00640 1 -- start start 2024-03-09T09:21:50.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.183+0000 7fb948a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb94406c1d0 0x7fb94406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.183+0000 7fb948a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9440c9890 con 0x7fb94414fd90 2024-03-09T09:21:50.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.183+0000 7fb948a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9440c5f80 con 0x7fb9440c9380 2024-03-09T09:21:50.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.183+0000 7fb948a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94406ba60 con 0x7fb94406c1d0 2024-03-09T09:21:50.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.183+0000 7fb942800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb94414fd90 0x7fb944156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36612/0 (socket says 172.21.15.67:36612) 2024-03-09T09:21:50.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.183+0000 7fb942800640 1 -- 172.21.15.67:0/3541985724 learned_addr learned my addr 172.21.15.67:0/3541985724 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:21:50.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.184+0000 7fb943200640 1 --2- 172.21.15.67:0/3541985724 >> v2:172.21.15.67:3300/0 conn(0x7fb94406c1d0 0x7fb94406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.184+0000 7fb941400640 1 -- 172.21.15.67:0/3541985724 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833468573 0 0) 0x7fb9440c9890 con 0x7fb94414fd90 2024-03-09T09:21:50.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.184+0000 7fb941400640 1 -- 172.21.15.67:0/3541985724 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb928003520 con 0x7fb94414fd90 2024-03-09T09:21:50.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.185+0000 7fb943200640 1 -- 172.21.15.67:0/3541985724 >> v1:172.21.15.134:6789/0 conn(0x7fb9440c9380 legacy=0x7fb9440acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.185+0000 7fb943200640 1 -- 172.21.15.67:0/3541985724 >> v1:172.21.15.67:6789/0 conn(0x7fb94414fd90 legacy=0x7fb944156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.185+0000 7fb943200640 1 -- 172.21.15.67:0/3541985724 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440c6240 con 0x7fb94406c1d0 2024-03-09T09:21:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.187+0000 7fb943200640 1 --2- 172.21.15.67:0/3541985724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94406c1d0 0x7fb94406c5f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb94406bbe0 tx=0x7fb93c02bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8706109fa244c2a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.187+0000 7fb941400640 1 -- 172.21.15.67:0/3541985724 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c039070 con 0x7fb94406c1d0 2024-03-09T09:21:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.187+0000 7fb941400640 1 -- 172.21.15.67:0/3541985724 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c02c9a0 con 0x7fb94406c1d0 2024-03-09T09:21:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.187+0000 7fb941400640 1 -- 172.21.15.67:0/3541985724 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c02cd70 con 0x7fb94406c1d0 2024-03-09T09:21:50.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.188+0000 7fb948a00640 1 -- 172.21.15.67:0/3541985724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94406c1d0 msgr2=0x7fb94406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.188+0000 7fb948a00640 1 --2- 172.21.15.67:0/3541985724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94406c1d0 0x7fb94406c5f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb94406bbe0 tx=0x7fb93c02bc70 comp rx=0 tx=0).stop 2024-03-09T09:21:50.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.188+0000 7fb948a00640 1 -- 172.21.15.67:0/3541985724 shutdown_connections 2024-03-09T09:21:50.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.188+0000 7fb948a00640 1 --2- 172.21.15.67:0/3541985724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94406c1d0 0x7fb94406c5f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.188+0000 7fb948a00640 1 -- 172.21.15.67:0/3541985724 >> 172.21.15.67:0/3541985724 conn(0x7fb9440cb480 msgr2=0x7fb9440cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:50.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.188+0000 7fb948a00640 1 -- 172.21.15.67:0/3541985724 shutdown_connections 2024-03-09T09:21:50.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.188+0000 7fb948a00640 1 -- 172.21.15.67:0/3541985724 wait complete. 2024-03-09T09:21:50.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.190+0000 7fb948a00640 1 Processor -- start 2024-03-09T09:21:50.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.190+0000 7fb948a00640 1 -- start start 2024-03-09T09:21:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.191+0000 7fb948a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb94406c1d0 0x7fb9440ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.191+0000 7fb948a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9440c9380 0x7fb9440abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.191+0000 7fb948a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94414fd90 0x7fb9440ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.191+0000 7fb948a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440ad610 con 0x7fb94414fd90 2024-03-09T09:21:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.191+0000 7fb948a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94406bcb0 con 0x7fb94406c1d0 2024-03-09T09:21:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.191+0000 7fb948a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440ad790 con 0x7fb9440c9380 2024-03-09T09:21:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.191+0000 7fb943200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94414fd90 0x7fb9440ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.191+0000 7fb943200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94414fd90 0x7fb9440ac2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56866/0 (socket says 172.21.15.67:56866) 2024-03-09T09:21:50.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.191+0000 7fb943200640 1 -- 172.21.15.67:0/4040639717 learned_addr learned my addr 172.21.15.67:0/4040639717 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:50.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.192+0000 7fb942800640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb94406c1d0 0x7fb9440ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.192+0000 7fb941e00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9440c9380 0x7fb9440abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.192+0000 7fb943200640 1 -- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9440c9380 msgr2=0x7fb9440abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.192+0000 7fb943200640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9440c9380 0x7fb9440abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.192+0000 7fb943200640 1 -- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb94406c1d0 msgr2=0x7fb9440ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.192+0000 7fb943200640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb94406c1d0 0x7fb9440ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.192+0000 7fb943200640 1 -- 172.21.15.67:0/4040639717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440ab300 con 0x7fb94414fd90 2024-03-09T09:21:50.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.192+0000 7fb942800640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb94406c1d0 0x7fb9440ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:21:50.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.192+0000 7fb943200640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94414fd90 0x7fb9440ac2e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb93c037040 tx=0x7fb93c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.193+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c039050 con 0x7fb94414fd90 2024-03-09T09:21:50.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.193+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c045070 con 0x7fb94414fd90 2024-03-09T09:21:50.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.193+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c009450 con 0x7fb94414fd90 2024-03-09T09:21:50.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.193+0000 7fb948a00640 1 -- 172.21.15.67:0/4040639717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9441ef650 con 0x7fb94414fd90 2024-03-09T09:21:50.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.193+0000 7fb948a00640 1 -- 172.21.15.67:0/4040639717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9441ef930 con 0x7fb94414fd90 2024-03-09T09:21:50.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.194+0000 7fb948a00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440ac820 0x7fb944147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.194+0000 7fb948a00640 1 -- 172.21.15.67:0/4040639717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94404f3c0 con 0x7fb9440ac820 2024-03-09T09:21:50.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.195+0000 7fb942800640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440ac820 0x7fb944147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.195+0000 7fb942800640 1 -- 172.21.15.67:0/4040639717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb94406d2d0 con 0x7fb9440ac820 2024-03-09T09:21:50.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.196+0000 7fb942800640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440ac820 0x7fb944147ec0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb9340011e0 tx=0x7fb93400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.196+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb93c040450 con 0x7fb94414fd90 2024-03-09T09:21:50.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.197+0000 7fb93aa00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb92406b500 0x7fb92406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.198+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9340058b0 con 0x7fb9440ac820 2024-03-09T09:21:50.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.198+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb93c0ac8c0 con 0x7fb94414fd90 2024-03-09T09:21:50.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.198+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb934018070 con 0x7fb9440ac820 2024-03-09T09:21:50.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.198+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440ac820 msgr2=0x7fb944147ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.198+0000 7fb93aa00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440ac820 0x7fb944147ec0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb9340011e0 tx=0x7fb93400eb90 comp rx=0 tx=0).stop 2024-03-09T09:21:50.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.198+0000 7fb941e00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb92406b500 0x7fb92406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.198+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440ac820 msgr2=0x7fb944147ec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:50.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.198+0000 7fb93aa00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440ac820 0x7fb944147ec0 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb9340011e0 tx=0x7fb93400eb90 comp rx=0 tx=0).stop 2024-03-09T09:21:50.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.199+0000 7fb941e00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb92406b500 0x7fb92406d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb94414c430 tx=0x7fb930009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.206+0000 7fb948a00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440c1a40 0x7fb944148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.206+0000 7fb948a00640 1 -- 172.21.15.67:0/4040639717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440ae7f0 con 0x7fb9440c1a40 2024-03-09T09:21:50.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.206+0000 7fb942800640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440c1a40 0x7fb944148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.206+0000 7fb942800640 1 -- 172.21.15.67:0/4040639717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb944073a30 con 0x7fb9440c1a40 2024-03-09T09:21:50.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.207+0000 7fb942800640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440c1a40 0x7fb944148400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb9340011e0 tx=0x7fb934001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.207+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9340074e0 con 0x7fb9440c1a40 2024-03-09T09:21:50.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.207+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4537 (secure 0 0 0) 0x7fb9340074e0 con 0x7fb9440c1a40 2024-03-09T09:21:50.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.207+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440c1a40 msgr2=0x7fb944148400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.207+0000 7fb93aa00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440c1a40 0x7fb944148400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb9340011e0 tx=0x7fb934001150 comp rx=0 tx=0).stop 2024-03-09T09:21:50.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.207+0000 7fb93aa00640 1 -- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440c1a40 msgr2=0x7fb944148400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:50.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.207+0000 7fb93aa00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb9440c1a40 0x7fb944148400 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb9340011e0 tx=0x7fb934001150 comp rx=0 tx=0).stop 2024-03-09T09:21:50.215 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:21:50.215 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:21:50.215 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:21:50.215 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:21:50.215 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 226, 2024-03-09T09:21:50.215 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:21:50.215 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:21:50.215 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:50.216 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:50.217 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:21:50.218 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:50.219 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:21:50.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:21:50.221 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:21:50.222 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.223 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.224 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:50.225 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:21:50.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.227 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.227 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.227 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:21:50.227 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.227 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.227 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:21:50.227 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:21:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.208+0000 7fb948a00640 1 -- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb92406b500 msgr2=0x7fb92406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.208+0000 7fb948a00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb92406b500 0x7fb92406d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb94414c430 tx=0x7fb930009040 comp rx=0 tx=0).stop 2024-03-09T09:21:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.208+0000 7fb948a00640 1 -- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94414fd90 msgr2=0x7fb9440ac2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.208+0000 7fb948a00640 1 --2- 172.21.15.67:0/4040639717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb94414fd90 0x7fb9440ac2e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb93c037040 tx=0x7fb93c004a40 comp rx=0 tx=0).stop 2024-03-09T09:21:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.208+0000 7fb943200640 1 -- 172.21.15.67:0/4040639717 reap_dead start 2024-03-09T09:21:50.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.208+0000 7fb943200640 1 -- 172.21.15.67:0/4040639717 reap_dead start 2024-03-09T09:21:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.208+0000 7fb948a00640 1 -- 172.21.15.67:0/4040639717 shutdown_connections 2024-03-09T09:21:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.208+0000 7fb948a00640 1 -- 172.21.15.67:0/4040639717 >> 172.21.15.67:0/4040639717 conn(0x7fb9440cb480 msgr2=0x7fb944155270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.208+0000 7fb948a00640 1 -- 172.21.15.67:0/4040639717 shutdown_connections 2024-03-09T09:21:50.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.208+0000 7fb948a00640 1 -- 172.21.15.67:0/4040639717 wait complete. 2024-03-09T09:21:50.239 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:21:50.239 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:21:50.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:21:50.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.435+0000 7f9a86200640 1 Processor -- start 2024-03-09T09:21:50.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.436+0000 7f9a86200640 1 -- start start 2024-03-09T09:21:50.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.437+0000 7f9a86200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9a800ac9c0 0x7f9a800ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.437+0000 7f9a86200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a800ac4e0 con 0x7f9a8014c9a0 2024-03-09T09:21:50.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.437+0000 7f9a86200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a800aa550 con 0x7f9a800c1d70 2024-03-09T09:21:50.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.437+0000 7f9a86200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8006d590 con 0x7f9a800ac9c0 2024-03-09T09:21:50.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.437+0000 7f9a7f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9a8014c9a0 0x7f9a801567c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36614/0 (socket says 172.21.15.67:36614) 2024-03-09T09:21:50.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.437+0000 7f9a7f400640 1 -- 172.21.15.67:0/3626689813 learned_addr learned my addr 172.21.15.67:0/3626689813 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:21:50.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.438+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/3626689813 >> v2:172.21.15.134:3300/0 conn(0x7f9a800ac9c0 0x7f9a800ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.438+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897535081 0 0) 0x7f9a800ac4e0 con 0x7f9a8014c9a0 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.438+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a64003520 con 0x7f9a8014c9a0 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.438+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434142904 0 0) 0x7f9a64003520 con 0x7f9a8014c9a0 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.438+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 >> v2:172.21.15.134:3300/0 conn(0x7f9a800ac9c0 msgr2=0x7f9a800ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.438+0000 7f9a7ea00640 1 --2- 172.21.15.67:0/3626689813 >> v2:172.21.15.134:3300/0 conn(0x7f9a800ac9c0 0x7f9a800ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.438+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 >> v1:172.21.15.134:6789/0 conn(0x7f9a800c1d70 legacy=0x7f9a800c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.438+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a800aa830 con 0x7f9a8014c9a0 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.438+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f9a70001670 con 0x7f9a8014c9a0 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.438+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 >> v1:172.21.15.67:6789/0 conn(0x7f9a8014c9a0 legacy=0x7f9a801567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.439+0000 7f9a7ea00640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 0x7f9a64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.439+0000 7f9a7ea00640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a64004800 0x7f9a64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.439+0000 7f9a7ea00640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a640051c0 0x7f9a6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.439+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6400c130 con 0x7f9a640039b0 2024-03-09T09:21:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.439+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6400c2e0 con 0x7f9a640051c0 2024-03-09T09:21:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.439+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6400c4e0 con 0x7f9a64004800 2024-03-09T09:21:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.439+0000 7f9a7f400640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 0x7f9a64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.439+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a640051c0 0x7f9a6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.439+0000 7f9a77e00640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a64004800 0x7f9a64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.440+0000 7f9a7f400640 1 -- 172.21.15.67:0/3626689813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a64004800 msgr2=0x7f9a64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.440+0000 7f9a7f400640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a64004800 0x7f9a64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.440+0000 7f9a7f400640 1 -- 172.21.15.67:0/3626689813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a640051c0 msgr2=0x7f9a6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.440+0000 7f9a7f400640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a640051c0 0x7f9a6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.440+0000 7f9a7f400640 1 -- 172.21.15.67:0/3626689813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a64003520 con 0x7f9a640039b0 2024-03-09T09:21:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.440+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a640051c0 0x7f9a6400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:21:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.440+0000 7f9a77e00640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a64004800 0x7f9a64004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:21:50.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.440+0000 7f9a7fe00640 1 -- 172.21.15.67:0/3626689813 reap_dead start 2024-03-09T09:21:50.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.441+0000 7f9a7f400640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 0x7f9a64003dd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c240 tx=0x7f9a64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba0cfd1c6d7fa7a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.442+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70007cf0 con 0x7f9a640039b0 2024-03-09T09:21:50.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.442+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a70030070 con 0x7f9a640039b0 2024-03-09T09:21:50.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.442+0000 7f9a7ea00640 1 -- 172.21.15.67:0/3626689813 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70005d30 con 0x7f9a640039b0 2024-03-09T09:21:50.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.442+0000 7f9a86200640 1 -- 172.21.15.67:0/3626689813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 msgr2=0x7f9a64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.442+0000 7f9a86200640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 0x7f9a64003dd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c240 tx=0x7f9a64005830 comp rx=0 tx=0).stop 2024-03-09T09:21:50.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.442+0000 7f9a86200640 1 -- 172.21.15.67:0/3626689813 shutdown_connections 2024-03-09T09:21:50.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.442+0000 7f9a86200640 1 --2- 172.21.15.67:0/3626689813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 0x7f9a64003dd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.442+0000 7f9a86200640 1 -- 172.21.15.67:0/3626689813 >> 172.21.15.67:0/3626689813 conn(0x7f9a800c7370 msgr2=0x7f9a800c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:50.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.443+0000 7f9a86200640 1 -- 172.21.15.67:0/3626689813 shutdown_connections 2024-03-09T09:21:50.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.443+0000 7f9a86200640 1 -- 172.21.15.67:0/3626689813 wait complete. 2024-03-09T09:21:50.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.444+0000 7f9a86200640 1 Processor -- start 2024-03-09T09:21:50.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.444+0000 7f9a86200640 1 -- start start 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.445+0000 7f9a86200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 0x7f9a801475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.445+0000 7f9a86200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80147af0 0x7f9a801eede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.445+0000 7f9a86200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a801ef320 0x7f9a80157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.445+0000 7f9a86200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a801567c0 con 0x7f9a640039b0 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.445+0000 7f9a86200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a800aadf0 con 0x7f9a801ef320 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.445+0000 7f9a86200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a80156940 con 0x7f9a80147af0 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.445+0000 7f9a7f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 0x7f9a801475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.445+0000 7f9a7f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 0x7f9a801475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56884/0 (socket says 172.21.15.67:56884) 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.445+0000 7f9a7f400640 1 -- 172.21.15.67:0/2833024798 learned_addr learned my addr 172.21.15.67:0/2833024798 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.446+0000 7f9a7f400640 1 -- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80147af0 msgr2=0x7f9a801eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.446+0000 7f9a77e00640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80147af0 0x7f9a801eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.446+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a801ef320 0x7f9a80157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.446+0000 7f9a7f400640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9a80147af0 0x7f9a801eede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.446+0000 7f9a7f400640 1 -- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a801ef320 msgr2=0x7f9a80157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.446+0000 7f9a7f400640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a801ef320 0x7f9a80157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.446+0000 7f9a7f400640 1 -- 172.21.15.67:0/2833024798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a80147f70 con 0x7f9a640039b0 2024-03-09T09:21:50.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.446+0000 7f9a7f400640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 0x7f9a801475b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c360 tx=0x7f9a801503c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.447+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70001c30 con 0x7f9a640039b0 2024-03-09T09:21:50.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.447+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a70004bf0 con 0x7f9a640039b0 2024-03-09T09:21:50.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.447+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70007070 con 0x7f9a640039b0 2024-03-09T09:21:50.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.447+0000 7f9a86200640 1 -- 172.21.15.67:0/2833024798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a80070c70 con 0x7f9a640039b0 2024-03-09T09:21:50.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.447+0000 7f9a86200640 1 -- 172.21.15.67:0/2833024798 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a80159e30 con 0x7f9a640039b0 2024-03-09T09:21:50.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.448+0000 7f9a86200640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158450 0x7f9a801eb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.448+0000 7f9a86200640 1 -- 172.21.15.67:0/2833024798 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a80150a80 con 0x7f9a80158450 2024-03-09T09:21:50.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.449+0000 7f9a77e00640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158450 0x7f9a801eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.449+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9a70048020 con 0x7f9a640039b0 2024-03-09T09:21:50.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.449+0000 7f9a77e00640 1 -- 172.21.15.67:0/2833024798 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a80075a00 con 0x7f9a80158450 2024-03-09T09:21:50.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.450+0000 7f9a77e00640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158450 0x7f9a801eb600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a8014b540 tx=0x7f9a6800f890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.450+0000 7f9a77400640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.450+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9a700ab900 con 0x7f9a640039b0 2024-03-09T09:21:50.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.450+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a680052e0 con 0x7f9a80158450 2024-03-09T09:21:50.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.451+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.451+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f9a68018070 con 0x7f9a80158450 2024-03-09T09:21:50.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.451+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158450 msgr2=0x7f9a801eb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.451+0000 7f9a77400640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158450 0x7f9a801eb600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a8014b540 tx=0x7f9a6800f890 comp rx=0 tx=0).stop 2024-03-09T09:21:50.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.451+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158450 msgr2=0x7f9a801eb600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:50.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.451+0000 7f9a77400640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158450 0x7f9a801eb600 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9a8014b540 tx=0x7f9a6800f890 comp rx=0 tx=0).stop 2024-03-09T09:21:50.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.452+0000 7f9a7fe00640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9a800702f0 tx=0x7f9a7800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.458+0000 7f9a86200640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158830 0x7f9a801ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.458+0000 7f9a86200640 1 -- 172.21.15.67:0/2833024798 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a80157b70 con 0x7f9a80158830 2024-03-09T09:21:50.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.459+0000 7f9a77e00640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158830 0x7f9a801ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.459+0000 7f9a77e00640 1 -- 172.21.15.67:0/2833024798 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9a800c15e0 con 0x7f9a80158830 2024-03-09T09:21:50.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.460+0000 7f9a77e00640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158830 0x7f9a801ebb40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9a68017000 tx=0x7f9a68001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.461+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68003940 con 0x7f9a80158830 2024-03-09T09:21:50.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.461+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f9a68003940 con 0x7f9a80158830 2024-03-09T09:21:50.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.461+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158830 msgr2=0x7f9a801ebb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.461+0000 7f9a77400640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158830 0x7f9a801ebb40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9a68017000 tx=0x7f9a68001120 comp rx=0 tx=0).stop 2024-03-09T09:21:50.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.461+0000 7f9a77400640 1 -- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158830 msgr2=0x7f9a801ebb40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:50.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.461+0000 7f9a77400640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a80158830 0x7f9a801ebb40 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9a68017000 tx=0x7f9a68001120 comp rx=0 tx=0).stop 2024-03-09T09:21:50.468 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 226, 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:21:50.469 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:21:50.470 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.471 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:21:50.472 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:50.473 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.475 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.476 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:50.478 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:21:50.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:21:50.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.481 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.482 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:50.482 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.482 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.482 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:21:50.482 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:21:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.462+0000 7f9a86200640 1 -- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9a5c06b4d0 msgr2=0x7f9a5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.462+0000 7f9a86200640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9a800702f0 tx=0x7f9a7800b040 comp rx=0 tx=0).stop 2024-03-09T09:21:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.462+0000 7f9a86200640 1 -- 172.21.15.67:0/2833024798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 msgr2=0x7f9a801475b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.462+0000 7f9a86200640 1 --2- 172.21.15.67:0/2833024798 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a640039b0 0x7f9a801475b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000c360 tx=0x7f9a801503c0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.462+0000 7f9a7fe00640 1 -- 172.21.15.67:0/2833024798 reap_dead start 2024-03-09T09:21:50.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.462+0000 7f9a7fe00640 1 -- 172.21.15.67:0/2833024798 reap_dead start 2024-03-09T09:21:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.462+0000 7f9a86200640 1 -- 172.21.15.67:0/2833024798 shutdown_connections 2024-03-09T09:21:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.462+0000 7f9a86200640 1 -- 172.21.15.67:0/2833024798 >> 172.21.15.67:0/2833024798 conn(0x7f9a800c7370 msgr2=0x7f9a800c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.462+0000 7f9a86200640 1 -- 172.21.15.67:0/2833024798 shutdown_connections 2024-03-09T09:21:50.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.462+0000 7f9a86200640 1 -- 172.21.15.67:0/2833024798 wait complete. 2024-03-09T09:21:50.485 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:21:50.485 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:21:50.485 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:21:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.723+0000 7f8447200640 1 Processor -- start 2024-03-09T09:21:50.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.723+0000 7f8447200640 1 -- start start 2024-03-09T09:21:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.725+0000 7f8447200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84480c3630 0x7f844806b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.725+0000 7f8447200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f844814ff60 con 0x7f844806b040 2024-03-09T09:21:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.725+0000 7f8447200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8448071f90 con 0x7f84480717f0 2024-03-09T09:21:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.725+0000 7f8447200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8448071d40 con 0x7f84480c3630 2024-03-09T09:21:50.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.726+0000 7f8445e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f844806b040 0x7f8448159240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36620/0 (socket says 172.21.15.67:36620) 2024-03-09T09:21:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.726+0000 7f8445e00640 1 -- 172.21.15.67:0/2709679200 learned_addr learned my addr 172.21.15.67:0/2709679200 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:21:50.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.726+0000 7f8446800640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84480c3630 0x7f844806b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.726+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918253337 0 0) 0x7f844814ff60 con 0x7f844806b040 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.726+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f842c003520 con 0x7f844806b040 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4138611090 0 0) 0x7f842c003520 con 0x7f844806b040 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84480c3630 msgr2=0x7f844806b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84480c3630 0x7f844806b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 >> v1:172.21.15.134:6789/0 conn(0x7f84480717f0 legacy=0x7f8448159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8448072270 con 0x7f844806b040 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f8438001670 con 0x7f844806b040 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 >> v1:172.21.15.67:6789/0 conn(0x7f844806b040 legacy=0x7f8448159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84480c3630 0x7f842c003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f842c003fb0 0x7f842c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c004d40 0x7f842c0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8448071d40 con 0x7f842c003fb0 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c00bc50 con 0x7f842c004d40 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c00be50 con 0x7f84480c3630 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.727+0000 7f8445400640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f842c003fb0 0x7f842c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.728+0000 7f8445e00640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84480c3630 0x7f842c003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.728+0000 7f8446800640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c004d40 0x7f842c0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.728+0000 7f8445400640 1 -- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84480c3630 msgr2=0x7f842c003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.728+0000 7f8445400640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84480c3630 0x7f842c003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.728+0000 7f8445400640 1 -- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c004d40 msgr2=0x7f842c0051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.728+0000 7f8445400640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c004d40 0x7f842c0051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.728+0000 7f8445400640 1 -- 172.21.15.67:0/2709679200 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c003520 con 0x7f842c003fb0 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.728+0000 7f8446800640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c004d40 0x7f842c0051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:21:50.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.728+0000 7f8445e00640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84480c3630 0x7f842c003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:21:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.729+0000 7f8445400640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f842c003fb0 0x7f842c004800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8434001860 tx=0x7f842c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331722c41b7b105c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.730+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434038070 con 0x7f842c003fb0 2024-03-09T09:21:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.730+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8434033410 con 0x7f842c003fb0 2024-03-09T09:21:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.730+0000 7f8444a00640 1 -- 172.21.15.67:0/2709679200 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434001430 con 0x7f842c003fb0 2024-03-09T09:21:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.730+0000 7f8447200640 1 -- 172.21.15.67:0/2709679200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f842c003fb0 msgr2=0x7f842c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.730+0000 7f8447200640 1 --2- 172.21.15.67:0/2709679200 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f842c003fb0 0x7f842c004800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8434001860 tx=0x7f842c005830 comp rx=0 tx=0).stop 2024-03-09T09:21:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.730+0000 7f8446800640 1 -- 172.21.15.67:0/2709679200 reap_dead start 2024-03-09T09:21:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.730+0000 7f8447200640 1 -- 172.21.15.67:0/2709679200 shutdown_connections 2024-03-09T09:21:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.730+0000 7f8447200640 1 -- 172.21.15.67:0/2709679200 >> 172.21.15.67:0/2709679200 conn(0x7f84480c8480 msgr2=0x7f84480c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:50.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.731+0000 7f8447200640 1 -- 172.21.15.67:0/2709679200 shutdown_connections 2024-03-09T09:21:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.731+0000 7f8447200640 1 -- 172.21.15.67:0/2709679200 wait complete. 2024-03-09T09:21:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.733+0000 7f8447200640 1 Processor -- start 2024-03-09T09:21:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.733+0000 7f8447200640 1 -- start start 2024-03-09T09:21:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.734+0000 7f8447200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c003fb0 0x7f844814c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.734+0000 7f8447200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f844814c650 0x7f844814dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.734+0000 7f8447200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844814e500 0x7f84481eb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.734+0000 7f8447200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8448072830 con 0x7f844814e500 2024-03-09T09:21:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.734+0000 7f8447200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8448072ba0 con 0x7f842c003fb0 2024-03-09T09:21:50.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.734+0000 7f8447200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844806b420 con 0x7f844814c650 2024-03-09T09:21:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.734+0000 7f8446800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844814e500 0x7f84481eb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.734+0000 7f8446800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844814e500 0x7f84481eb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56898/0 (socket says 172.21.15.67:56898) 2024-03-09T09:21:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.734+0000 7f8446800640 1 -- 172.21.15.67:0/2274002764 learned_addr learned my addr 172.21.15.67:0/2274002764 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.734+0000 7f8445e00640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c003fb0 0x7f844814c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.734+0000 7f8445400640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f844814c650 0x7f844814dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.735+0000 7f8446800640 1 -- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f844814c650 msgr2=0x7f844814dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.735+0000 7f8446800640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f844814c650 0x7f844814dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.735+0000 7f8446800640 1 -- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c003fb0 msgr2=0x7f844814c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.735+0000 7f8446800640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f842c003fb0 0x7f844814c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.735+0000 7f8446800640 1 -- 172.21.15.67:0/2274002764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8448145d00 con 0x7f844814e500 2024-03-09T09:21:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.735+0000 7f8446800640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844814e500 0x7f84481eb6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f84480c33e0 tx=0x7f844000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.735+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844001c070 con 0x7f844814e500 2024-03-09T09:21:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.736+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8440017400 con 0x7f844814e500 2024-03-09T09:21:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.736+0000 7f8447200640 1 -- 172.21.15.67:0/2274002764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8448145a20 con 0x7f844814e500 2024-03-09T09:21:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.736+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84400177d0 con 0x7f844814e500 2024-03-09T09:21:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.736+0000 7f8447200640 1 -- 172.21.15.67:0/2274002764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84481efa10 con 0x7f844814e500 2024-03-09T09:21:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.737+0000 7f8447200640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8448145f50 0x7f84481ebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.737+0000 7f8447200640 1 -- 172.21.15.67:0/2274002764 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844804f3c0 con 0x7f8448145f50 2024-03-09T09:21:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.737+0000 7f8445e00640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8448145f50 0x7f84481ebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.738+0000 7f8445e00640 1 -- 172.21.15.67:0/2274002764 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84481534c0 con 0x7f8448145f50 2024-03-09T09:21:50.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.738+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f844001f070 con 0x7f844814e500 2024-03-09T09:21:50.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.739+0000 7f8445e00640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8448145f50 0x7f84481ebbf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84480c5e50 tx=0x7f843800fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.739+0000 7f843e000640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f842806b500 0x7f842806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.740+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f844008a650 con 0x7f844814e500 2024-03-09T09:21:50.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.740+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438017020 con 0x7f8448145f50 2024-03-09T09:21:50.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.740+0000 7f8445400640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f842806b500 0x7f842806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.740+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f843800d750 con 0x7f8448145f50 2024-03-09T09:21:50.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.740+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8448145f50 msgr2=0x7f84481ebbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.740+0000 7f843e000640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8448145f50 0x7f84481ebbf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84480c5e50 tx=0x7f843800fc00 comp rx=0 tx=0).stop 2024-03-09T09:21:50.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.740+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8448145f50 msgr2=0x7f84481ebbf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:50.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.740+0000 7f843e000640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8448145f50 0x7f84481ebbf0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84480c5e50 tx=0x7f843800fc00 comp rx=0 tx=0).stop 2024-03-09T09:21:50.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.742+0000 7f8445400640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f842806b500 0x7f842806d9c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8434035000 tx=0x7f8434009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.748+0000 7f8447200640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84481ec130 0x7f8448157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.748+0000 7f8447200640 1 -- 172.21.15.67:0/2274002764 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84480c3410 con 0x7f84481ec130 2024-03-09T09:21:50.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.748+0000 7f8445e00640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84481ec130 0x7f8448157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:50.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.749+0000 7f8445e00640 1 -- 172.21.15.67:0/2274002764 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f844814f340 con 0x7f84481ec130 2024-03-09T09:21:50.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.750+0000 7f8445e00640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84481ec130 0x7f8448157ee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8438005fd0 tx=0x7f8448155aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:50.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.751+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438017040 con 0x7f84481ec130 2024-03-09T09:21:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.752+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f843801e220 con 0x7f84481ec130 2024-03-09T09:21:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.752+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84481ec130 msgr2=0x7f8448157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.752+0000 7f843e000640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84481ec130 0x7f8448157ee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8438005fd0 tx=0x7f8448155aa0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.752+0000 7f843e000640 1 -- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84481ec130 msgr2=0x7f8448157ee0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:21:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.752+0000 7f843e000640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84481ec130 0x7f8448157ee0 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8438005fd0 tx=0x7f8448155aa0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.760 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:21:50.760 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:21:50.760 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:21:50.760 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:21:50.760 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 226, 2024-03-09T09:21:50.760 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:21:50.760 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:21:50.760 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:21:50.760 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:21:50.760 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.760 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:50.761 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:50.762 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:21:50.763 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:21:50.764 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:50.765 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:50.766 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.767 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:21:50.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.770 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.771 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.773 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:21:50.773 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:21:50.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:21:50.773 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:21:50.773 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:21:50.773 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:21:50.773 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:21:50.773 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:21:50.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:21:50.773 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:21:50.773 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:21:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.753+0000 7f8447200640 1 -- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f842806b500 msgr2=0x7f842806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.753+0000 7f8447200640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f842806b500 0x7f842806d9c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8434035000 tx=0x7f8434009030 comp rx=0 tx=0).stop 2024-03-09T09:21:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.753+0000 7f8447200640 1 -- 172.21.15.67:0/2274002764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844814e500 msgr2=0x7f84481eb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.753+0000 7f8447200640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844814e500 0x7f84481eb6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f84480c33e0 tx=0x7f844000f4a0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.753+0000 7f8446800640 1 -- 172.21.15.67:0/2274002764 reap_dead start 2024-03-09T09:21:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.753+0000 7f8447200640 1 -- 172.21.15.67:0/2274002764 shutdown_connections 2024-03-09T09:21:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.753+0000 7f8447200640 1 --2- 172.21.15.67:0/2274002764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f844814e500 0x7f84481eb6b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.753+0000 7f8447200640 1 -- 172.21.15.67:0/2274002764 >> 172.21.15.67:0/2274002764 conn(0x7f84480c8480 msgr2=0x7f84481deed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.754+0000 7f8447200640 1 -- 172.21.15.67:0/2274002764 shutdown_connections 2024-03-09T09:21:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:50.754+0000 7f8447200640 1 -- 172.21.15.67:0/2274002764 wait complete. 2024-03-09T09:21:50.783 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:21:50.783 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-03-09T09:21:50.783 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:21:50.783 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-03-09T09:21:50.783 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:21:50.783 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-09T09:21:50.784 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:21:50.814 INFO:tasks.ceph.mon.a:Stopped 2024-03-09T09:21:50.814 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:21:50.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:21:51.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.072+0000 7f2405c00640 1 Processor -- start 2024-03-09T09:21:51.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.072+0000 7f2405c00640 1 -- start start 2024-03-09T09:21:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.073+0000 7f2405c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f24000c1f70 0x7f2400150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.073+0000 7f2405c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000710a0 0x7f2400151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.074+0000 7f2405c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24000c81e0 con 0x7f24000c27a0 2024-03-09T09:21:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.074+0000 7f2405c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24000c32b0 con 0x7f24000c1f70 2024-03-09T09:21:51.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.074+0000 7f2405c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24000c2fd0 con 0x7f24000710a0 2024-03-09T09:21:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.074+0000 7f23ff400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000710a0 0x7f2400151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.074+0000 7f23ffe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f24000c1f70 0x7f2400150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.074+0000 7f2405200640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f24000c27a0 0x7f2400153970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:41416/0 (socket says 172.21.15.67:41416) 2024-03-09T09:21:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.074+0000 7f23ffe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f24000c1f70 0x7f2400150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:57050/0 (socket says 172.21.15.67:57050) 2024-03-09T09:21:51.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.074+0000 7f2405200640 1 -- 172.21.15.67:0/229158952 learned_addr learned my addr 172.21.15.67:0/229158952 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:21:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.075+0000 7f23ffe00640 1 -- 172.21.15.67:0/229158952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000710a0 msgr2=0x7f2400151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.075+0000 7f23ffe00640 1 --2- 172.21.15.67:0/229158952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000710a0 0x7f2400151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.075+0000 7f23ffe00640 1 -- 172.21.15.67:0/229158952 >> v1:172.21.15.134:6789/0 conn(0x7f24000c27a0 legacy=0x7f2400153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.075+0000 7f23ffe00640 1 -- 172.21.15.67:0/229158952 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24000c3700 con 0x7f24000c1f70 2024-03-09T09:21:51.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.075+0000 7f23ff400640 1 --2- 172.21.15.67:0/229158952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000710a0 0x7f2400151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:21:51.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.077+0000 7f23ffe00640 1 --2- 172.21.15.67:0/229158952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24000c1f70 0x7f2400150c50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f23f00097a0 tx=0x7f23f002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55bffc5607a0516a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:51.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.077+0000 7f23fea00640 1 -- 172.21.15.67:0/229158952 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0031780 con 0x7f24000c1f70 2024-03-09T09:21:51.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.077+0000 7f23fea00640 1 -- 172.21.15.67:0/229158952 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f0037070 con 0x7f24000c1f70 2024-03-09T09:21:51.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.077+0000 7f23fea00640 1 -- 172.21.15.67:0/229158952 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0031b40 con 0x7f24000c1f70 2024-03-09T09:21:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.078+0000 7f2405c00640 1 -- 172.21.15.67:0/229158952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24000c1f70 msgr2=0x7f2400150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.078+0000 7f2405c00640 1 --2- 172.21.15.67:0/229158952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24000c1f70 0x7f2400150c50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f23f00097a0 tx=0x7f23f002b820 comp rx=0 tx=0).stop 2024-03-09T09:21:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.078+0000 7f2405c00640 1 -- 172.21.15.67:0/229158952 shutdown_connections 2024-03-09T09:21:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.078+0000 7f2405c00640 1 --2- 172.21.15.67:0/229158952 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000710a0 0x7f2400151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.078+0000 7f2405c00640 1 --2- 172.21.15.67:0/229158952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24000c1f70 0x7f2400150c50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:51.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.078+0000 7f2405c00640 1 -- 172.21.15.67:0/229158952 >> 172.21.15.67:0/229158952 conn(0x7f24000c7300 msgr2=0x7f24000c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.078+0000 7f2405c00640 1 -- 172.21.15.67:0/229158952 shutdown_connections 2024-03-09T09:21:51.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.078+0000 7f2405c00640 1 -- 172.21.15.67:0/229158952 wait complete. 2024-03-09T09:21:51.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.081+0000 7f2405c00640 1 Processor -- start 2024-03-09T09:21:51.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.081+0000 7f2405c00640 1 -- start start 2024-03-09T09:21:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.082+0000 7f2405c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24000710a0 0x7f2400156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.082+0000 7f2405c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000c1f70 0x7f2400156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.082+0000 7f2405c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000c27a0 0x7f24001eb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.082+0000 7f2405c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2400070d50 con 0x7f24000c1f70 2024-03-09T09:21:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.082+0000 7f2405c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2400070830 con 0x7f24000710a0 2024-03-09T09:21:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.082+0000 7f2405c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2400070ed0 con 0x7f24000c27a0 2024-03-09T09:21:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.083+0000 7f23ff400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000c1f70 msgr2=0x7f2400156f80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:51.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.083+0000 7f23ff400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000c1f70 0x7f2400156f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:51.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.083+0000 7f2405200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000c27a0 0x7f24001eb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:51.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.083+0000 7f23ffe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24000710a0 0x7f2400156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:51.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.083+0000 7f23ffe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24000710a0 0x7f2400156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:57066/0 (socket says 172.21.15.67:57066) 2024-03-09T09:21:51.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.083+0000 7f2405200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000c27a0 0x7f24001eb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:51462/0 (socket says 172.21.15.67:51462) 2024-03-09T09:21:51.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.083+0000 7f23ffe00640 1 -- 172.21.15.67:0/2585104172 learned_addr learned my addr 172.21.15.67:0/2585104172 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:51.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.084+0000 7f2405200640 1 -- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24000710a0 msgr2=0x7f2400156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:51.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.084+0000 7f2405200640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24000710a0 0x7f2400156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:51.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.084+0000 7f2405200640 1 -- 172.21.15.67:0/2585104172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000c1f70 msgr2=0x7f2400156f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:21:51.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.084+0000 7f2405200640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000c1f70 0x7f2400156f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:51.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.084+0000 7f2405200640 1 -- 172.21.15.67:0/2585104172 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24001ddb50 con 0x7f24000c27a0 2024-03-09T09:21:51.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.085+0000 7f2405200640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000c27a0 0x7f24001eb7b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23f800cad0 tx=0x7f23f800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:51.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.085+0000 7f23f7e00640 1 -- 172.21.15.67:0/2585104172 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8017070 con 0x7f24000c27a0 2024-03-09T09:21:51.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.085+0000 7f2405c00640 1 -- 172.21.15.67:0/2585104172 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24001dde30 con 0x7f24000c27a0 2024-03-09T09:21:51.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.085+0000 7f23f7e00640 1 -- 172.21.15.67:0/2585104172 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f800ac30 con 0x7f24000c27a0 2024-03-09T09:21:51.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.085+0000 7f23f7e00640 1 -- 172.21.15.67:0/2585104172 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8012400 con 0x7f24000c27a0 2024-03-09T09:21:51.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.085+0000 7f2405c00640 1 -- 172.21.15.67:0/2585104172 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24001de3f0 con 0x7f24000c27a0 2024-03-09T09:21:51.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.087+0000 7f2405c00640 1 -- 172.21.15.67:0/2585104172 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24001de630 con 0x7f24000c27a0 2024-03-09T09:21:51.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.090+0000 7f23f7e00640 1 -- 172.21.15.67:0/2585104172 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f23f801a070 con 0x7f24000c27a0 2024-03-09T09:21:51.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.090+0000 7f23f7e00640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23e006b480 0x7f23e006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:51.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.091+0000 7f23f7e00640 1 -- 172.21.15.67:0/2585104172 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f23f808a820 con 0x7f24000c27a0 2024-03-09T09:21:51.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.091+0000 7f23ffe00640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23e006b480 0x7f23e006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:51.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.092+0000 7f23ffe00640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23e006b480 0x7f23e006d940 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f23f002b860 tx=0x7f23f003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:51.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.108+0000 7f23f7e00640 1 -- 172.21.15.67:0/2585104172 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f24001de630 con 0x7f24000c27a0 2024-03-09T09:21:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.398+0000 7f2405c00640 1 -- 172.21.15.67:0/2585104172 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f240004f3c0 con 0x7f24000c27a0 2024-03-09T09:21:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.399+0000 7f23f7e00640 1 -- 172.21.15.67:0/2585104172 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f23f8058bc0 con 0x7f24000c27a0 2024-03-09T09:21:51.406 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":226,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 -- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23e006b480 msgr2=0x7f23e006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23e006b480 0x7f23e006d940 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f23f002b860 tx=0x7f23f003d040 comp rx=0 tx=0).stop 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 -- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000c27a0 msgr2=0x7f24001eb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000c27a0 0x7f24001eb7b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23f800cad0 tx=0x7f23f800cf90 comp rx=0 tx=0).stop 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 -- 172.21.15.67:0/2585104172 shutdown_connections 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23e006b480 0x7f23e006d940 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24000c27a0 0x7f24001eb7b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24000c1f70 0x7f2400156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 --2- 172.21.15.67:0/2585104172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24000710a0 0x7f2400156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 -- 172.21.15.67:0/2585104172 >> 172.21.15.67:0/2585104172 conn(0x7f24000c7300 msgr2=0x7f24000735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 -- 172.21.15.67:0/2585104172 shutdown_connections 2024-03-09T09:21:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:51.402+0000 7f2405c00640 1 -- 172.21.15.67:0/2585104172 wait complete. 2024-03-09T09:21:53.217 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:21:53.217 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:53.220 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 58 -eq 48 ']' 2024-03-09T09:21:53.220 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.58 2024-03-09T09:21:53.220 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.58 2024-03-09T09:21:53.220 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.58 2024-03-09T09:21:53.221 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.58 2024-03-09T09:21:53.222 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:21:54.300 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:21:54.300 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:21:54.300 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 64 -eq 48 ']' 2024-03-09T09:21:54.300 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.64 2024-03-09T09:21:54.301 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.64 2024-03-09T09:21:54.301 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.64 2024-03-09T09:21:54.301 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.64 2024-03-09T09:21:54.306 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:21:54.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:21:54.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.636+0000 7f9566800640 1 Processor -- start 2024-03-09T09:21:54.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.636+0000 7f9566800640 1 -- start start 2024-03-09T09:21:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.638+0000 7f9566800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 0x7f9560070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.638+0000 7f9566800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f95600c28a0 0x7f95600c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.638+0000 7f9566800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9560153630 con 0x7f9560153250 2024-03-09T09:21:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.638+0000 7f9566800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95600aa5f0 con 0x7f95600c28a0 2024-03-09T09:21:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.638+0000 7f9566800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95600abfe0 con 0x7f9560073010 2024-03-09T09:21:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.638+0000 7f9565e00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f95600c28a0 msgr2=0x7f95600c2d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.638+0000 7f9565400640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f9560153250 legacy=0x7f956006fe10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:21:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.638+0000 7f9565e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f95600c28a0 0x7f95600c2d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.638+0000 7f9564a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 0x7f9560070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:54.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.638+0000 7f9564a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 0x7f9560070520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:51474/0 (socket says 172.21.15.67:51474) 2024-03-09T09:21:54.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.638+0000 7f9564a00640 1 -- 172.21.15.67:0/951861337 learned_addr learned my addr 172.21.15.67:0/951861337 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:54.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.639+0000 7f9564a00640 1 -- 172.21.15.67:0/951861337 >> v2:172.21.15.67:3300/0 conn(0x7f95600c28a0 msgr2=0x7f95600c2d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:54.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.639+0000 7f9564a00640 1 --2- 172.21.15.67:0/951861337 >> v2:172.21.15.67:3300/0 conn(0x7f95600c28a0 0x7f95600c2d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:54.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.639+0000 7f9564a00640 1 -- 172.21.15.67:0/951861337 >> v1:172.21.15.67:6789/0 conn(0x7f9560153250 legacy=0x7f956006fe10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:21:54.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.639+0000 7f9564a00640 1 -- 172.21.15.67:0/951861337 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600abd00 con 0x7f9560073010 2024-03-09T09:21:54.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.640+0000 7f9564a00640 1 --2- 172.21.15.67:0/951861337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 0x7f9560070520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f954c009870 tx=0x7f954c02b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe32fac81114ee57 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:54.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.641+0000 7f955fe00640 1 -- 172.21.15.67:0/951861337 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c035020 con 0x7f9560073010 2024-03-09T09:21:54.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.641+0000 7f955fe00640 1 -- 172.21.15.67:0/951861337 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c031870 con 0x7f9560073010 2024-03-09T09:21:54.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.641+0000 7f955fe00640 1 -- 172.21.15.67:0/951861337 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c031c40 con 0x7f9560073010 2024-03-09T09:21:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.641+0000 7f9566800640 1 -- 172.21.15.67:0/951861337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 msgr2=0x7f9560070520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.641+0000 7f9566800640 1 --2- 172.21.15.67:0/951861337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 0x7f9560070520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f954c009870 tx=0x7f954c02b910 comp rx=0 tx=0).stop 2024-03-09T09:21:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.642+0000 7f9566800640 1 -- 172.21.15.67:0/951861337 shutdown_connections 2024-03-09T09:21:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.642+0000 7f9566800640 1 --2- 172.21.15.67:0/951861337 >> v2:172.21.15.67:3300/0 conn(0x7f95600c28a0 0x7f95600c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.642+0000 7f9566800640 1 --2- 172.21.15.67:0/951861337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 0x7f9560070520 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.642+0000 7f9566800640 1 -- 172.21.15.67:0/951861337 >> 172.21.15.67:0/951861337 conn(0x7f9560066d90 msgr2=0x7f9560067170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.642+0000 7f9566800640 1 -- 172.21.15.67:0/951861337 shutdown_connections 2024-03-09T09:21:54.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.642+0000 7f9566800640 1 -- 172.21.15.67:0/951861337 wait complete. 2024-03-09T09:21:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.643+0000 7f9566800640 1 Processor -- start 2024-03-09T09:21:54.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.644+0000 7f9566800640 1 -- start start 2024-03-09T09:21:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.644+0000 7f9566800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 0x7f9560156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.645+0000 7f9566800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95600c28a0 0x7f9560154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.645+0000 7f9566800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9560153250 0x7f95601ef530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.645+0000 7f9566800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956006f8f0 con 0x7f9560153250 2024-03-09T09:21:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.645+0000 7f9566800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9560156aa0 con 0x7f95600c28a0 2024-03-09T09:21:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.645+0000 7f9566800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9560070a60 con 0x7f9560073010 2024-03-09T09:21:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.645+0000 7f9565e00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9560153250 msgr2=0x7f95601ef530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.645+0000 7f9565e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9560153250 0x7f95601ef530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:54.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.645+0000 7f9564a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95600c28a0 0x7f9560154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:54.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.645+0000 7f9564a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95600c28a0 0x7f9560154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:57076/0 (socket says 172.21.15.67:57076) 2024-03-09T09:21:54.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.645+0000 7f9564a00640 1 -- 172.21.15.67:0/3592368384 learned_addr learned my addr 172.21.15.67:0/3592368384 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:54.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.645+0000 7f9565400640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 0x7f9560156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:54.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.646+0000 7f9564a00640 1 -- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 msgr2=0x7f9560156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:54.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.646+0000 7f9564a00640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 0x7f9560156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:54.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.646+0000 7f9564a00640 1 -- 172.21.15.67:0/3592368384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9560153250 msgr2=0x7f95601ef530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:21:54.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.646+0000 7f9564a00640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9560153250 0x7f95601ef530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:54.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.646+0000 7f9564a00640 1 -- 172.21.15.67:0/3592368384 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956014eaf0 con 0x7f95600c28a0 2024-03-09T09:21:54.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.647+0000 7f9564a00640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95600c28a0 0x7f9560154bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f954c000c00 tx=0x7f954c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:54.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.647+0000 7f955d600640 1 -- 172.21.15.67:0/3592368384 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c03ad60 con 0x7f95600c28a0 2024-03-09T09:21:54.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.647+0000 7f955d600640 1 -- 172.21.15.67:0/3592368384 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c0443f0 con 0x7f95600c28a0 2024-03-09T09:21:54.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.647+0000 7f955d600640 1 -- 172.21.15.67:0/3592368384 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c0447c0 con 0x7f95600c28a0 2024-03-09T09:21:54.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.647+0000 7f9566800640 1 -- 172.21.15.67:0/3592368384 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956014ed90 con 0x7f95600c28a0 2024-03-09T09:21:54.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.647+0000 7f9566800640 1 -- 172.21.15.67:0/3592368384 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9560155130 con 0x7f95600c28a0 2024-03-09T09:21:54.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.648+0000 7f9566800640 1 -- 172.21.15.67:0/3592368384 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956004f3c0 con 0x7f95600c28a0 2024-03-09T09:21:54.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.650+0000 7f955d600640 1 -- 172.21.15.67:0/3592368384 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f954c04b060 con 0x7f95600c28a0 2024-03-09T09:21:54.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.651+0000 7f955d600640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f954006b480 0x7f954006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:54.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.651+0000 7f955d600640 1 -- 172.21.15.67:0/3592368384 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f954c0aca80 con 0x7f95600c28a0 2024-03-09T09:21:54.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.651+0000 7f9565400640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f954006b480 0x7f954006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:54.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.652+0000 7f9565400640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f954006b480 0x7f954006d940 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9554003980 tx=0x7f9554009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:54.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.658+0000 7f955d600640 1 -- 172.21.15.67:0/3592368384 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f954c03f070 con 0x7f95600c28a0 2024-03-09T09:21:54.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.951+0000 7f9566800640 1 -- 172.21.15.67:0/3592368384 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9560073b20 con 0x7f95600c28a0 2024-03-09T09:21:54.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.952+0000 7f955d600640 1 -- 172.21.15.67:0/3592368384 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f954c02f320 con 0x7f95600c28a0 2024-03-09T09:21:54.958 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":226,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:21:54.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.954+0000 7f9566800640 1 -- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f954006b480 msgr2=0x7f954006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:54.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.954+0000 7f9566800640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f954006b480 0x7f954006d940 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9554003980 tx=0x7f9554009110 comp rx=0 tx=0).stop 2024-03-09T09:21:54.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.954+0000 7f9566800640 1 -- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95600c28a0 msgr2=0x7f9560154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:54.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.954+0000 7f9566800640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95600c28a0 0x7f9560154bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f954c000c00 tx=0x7f954c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:21:54.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.955+0000 7f9566800640 1 -- 172.21.15.67:0/3592368384 shutdown_connections 2024-03-09T09:21:54.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.955+0000 7f9566800640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f954006b480 0x7f954006d940 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:54.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.955+0000 7f9566800640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9560153250 0x7f95601ef530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:54.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.955+0000 7f9566800640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95600c28a0 0x7f9560154bf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:54.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.955+0000 7f9566800640 1 --2- 172.21.15.67:0/3592368384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9560073010 0x7f9560156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:54.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.955+0000 7f9566800640 1 -- 172.21.15.67:0/3592368384 >> 172.21.15.67:0/3592368384 conn(0x7f9560066d90 msgr2=0x7f956014fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:54.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.955+0000 7f9566800640 1 -- 172.21.15.67:0/3592368384 shutdown_connections 2024-03-09T09:21:54.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:54.955+0000 7f9566800640 1 -- 172.21.15.67:0/3592368384 wait complete. 2024-03-09T09:21:57.980 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:21:58.099 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:21:58.099 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:21:58.099 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 59 -eq 48 ']' 2024-03-09T09:21:58.099 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.59 2024-03-09T09:21:58.100 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.59 2024-03-09T09:21:58.100 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.59 2024-03-09T09:21:58.100 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.59 2024-03-09T09:21:58.104 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:21:58.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.228+0000 7fba9a000640 1 Processor -- start 2024-03-09T09:21:58.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.229+0000 7fba9a000640 1 -- start start 2024-03-09T09:21:58.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.231+0000 7fba9a000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fba940c5e40 0x7fba94156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:58.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.231+0000 7fba9a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba940cadb0 0x7fba94156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:58.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.231+0000 7fba9a000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba94156a90 con 0x7fba94153300 2024-03-09T09:21:58.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.231+0000 7fba9a000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba94157000 con 0x7fba940c5e40 2024-03-09T09:21:58.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.231+0000 7fba9a000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba940ae430 con 0x7fba940cadb0 2024-03-09T09:21:58.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.231+0000 7fba99600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba940cadb0 0x7fba94156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:58.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.231+0000 7fba99600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba940cadb0 0x7fba94156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:51494/0 (socket says 172.21.15.67:51494) 2024-03-09T09:21:58.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.231+0000 7fba99600640 1 -- 172.21.15.67:0/749966634 learned_addr learned my addr 172.21.15.67:0/749966634 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:58.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.232+0000 7fba93e00640 1 --2- 172.21.15.67:0/749966634 >> v2:172.21.15.134:3300/0 conn(0x7fba940c5e40 0x7fba94156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:58.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.232+0000 7fba99600640 1 -- 172.21.15.67:0/749966634 >> v2:172.21.15.134:3300/0 conn(0x7fba940c5e40 msgr2=0x7fba94156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:21:58.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.232+0000 7fba99600640 1 --2- 172.21.15.67:0/749966634 >> v2:172.21.15.134:3300/0 conn(0x7fba940c5e40 0x7fba94156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:58.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.232+0000 7fba99600640 1 -- 172.21.15.67:0/749966634 >> v1:172.21.15.134:6789/0 conn(0x7fba94153300 legacy=0x7fba940b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:58.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.232+0000 7fba99600640 1 -- 172.21.15.67:0/749966634 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba940ae6f0 con 0x7fba940cadb0 2024-03-09T09:21:58.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.233+0000 7fba99600640 1 --2- 172.21.15.67:0/749966634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba940cadb0 0x7fba94156550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fba84009870 tx=0x7fba8402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7638ab0eabebe0d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:58.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.234+0000 7fba93400640 1 -- 172.21.15.67:0/749966634 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84034070 con 0x7fba940cadb0 2024-03-09T09:21:58.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.234+0000 7fba93400640 1 -- 172.21.15.67:0/749966634 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba84030840 con 0x7fba940cadb0 2024-03-09T09:21:58.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.234+0000 7fba93400640 1 -- 172.21.15.67:0/749966634 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84030c10 con 0x7fba940cadb0 2024-03-09T09:21:58.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.235+0000 7fba9a000640 1 -- 172.21.15.67:0/749966634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba940cadb0 msgr2=0x7fba94156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:58.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.235+0000 7fba9a000640 1 --2- 172.21.15.67:0/749966634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba940cadb0 0x7fba94156550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fba84009870 tx=0x7fba8402b980 comp rx=0 tx=0).stop 2024-03-09T09:21:58.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.235+0000 7fba9a000640 1 -- 172.21.15.67:0/749966634 shutdown_connections 2024-03-09T09:21:58.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.235+0000 7fba9a000640 1 --2- 172.21.15.67:0/749966634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba940cadb0 0x7fba94156550 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:58.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.235+0000 7fba9a000640 1 --2- 172.21.15.67:0/749966634 >> v2:172.21.15.134:3300/0 conn(0x7fba940c5e40 0x7fba94156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:58.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.235+0000 7fba9a000640 1 -- 172.21.15.67:0/749966634 >> 172.21.15.67:0/749966634 conn(0x7fba940732d0 msgr2=0x7fba940ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:58.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.235+0000 7fba9a000640 1 -- 172.21.15.67:0/749966634 shutdown_connections 2024-03-09T09:21:58.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.235+0000 7fba9a000640 1 -- 172.21.15.67:0/749966634 wait complete. 2024-03-09T09:21:58.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.236+0000 7fba9a000640 1 Processor -- start 2024-03-09T09:21:58.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.237+0000 7fba9a000640 1 -- start start 2024-03-09T09:21:58.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.237+0000 7fba9a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba940c5e40 0x7fba9406c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.237+0000 7fba9a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba940cadb0 0x7fba9414b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.238+0000 7fba9a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba94153300 0x7fba9414ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.238+0000 7fba9a000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba94159e90 con 0x7fba940c5e40 2024-03-09T09:21:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.238+0000 7fba9a000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba94159bb0 con 0x7fba940cadb0 2024-03-09T09:21:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.238+0000 7fba9a000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba940c59e0 con 0x7fba94153300 2024-03-09T09:21:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.238+0000 7fba98c00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba940c5e40 msgr2=0x7fba9406c300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:21:58.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.238+0000 7fba98c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba940c5e40 0x7fba9406c300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:21:58.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.238+0000 7fba93e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba940cadb0 0x7fba9414b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:58.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.238+0000 7fba93e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba940cadb0 0x7fba9414b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:57102/0 (socket says 172.21.15.67:57102) 2024-03-09T09:21:58.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.238+0000 7fba93e00640 1 -- 172.21.15.67:0/1209122082 learned_addr learned my addr 172.21.15.67:0/1209122082 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:21:58.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.238+0000 7fba99600640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba94153300 0x7fba9414ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.239+0000 7fba93e00640 1 -- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba94153300 msgr2=0x7fba9414ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.239+0000 7fba93e00640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba94153300 0x7fba9414ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.239+0000 7fba93e00640 1 -- 172.21.15.67:0/1209122082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba940c5e40 msgr2=0x7fba9406c300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:21:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.239+0000 7fba93e00640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba940c5e40 0x7fba9406c300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.239+0000 7fba93e00640 1 -- 172.21.15.67:0/1209122082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba941ec2f0 con 0x7fba940cadb0 2024-03-09T09:21:58.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.240+0000 7fba93e00640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba940cadb0 0x7fba9414b4d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00e8a0 tx=0x7fba7c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.240+0000 7fba90c00640 1 -- 172.21.15.67:0/1209122082 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c017070 con 0x7fba940cadb0 2024-03-09T09:21:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.240+0000 7fba90c00640 1 -- 172.21.15.67:0/1209122082 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c00c940 con 0x7fba940cadb0 2024-03-09T09:21:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.240+0000 7fba90c00640 1 -- 172.21.15.67:0/1209122082 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00cd30 con 0x7fba940cadb0 2024-03-09T09:21:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.240+0000 7fba9a000640 1 -- 172.21.15.67:0/1209122082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba940ac3e0 con 0x7fba940cadb0 2024-03-09T09:21:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.240+0000 7fba9a000640 1 -- 172.21.15.67:0/1209122082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba940ac9a0 con 0x7fba940cadb0 2024-03-09T09:21:58.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.241+0000 7fba9a000640 1 -- 172.21.15.67:0/1209122082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9404f3c0 con 0x7fba940cadb0 2024-03-09T09:21:58.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.243+0000 7fba90c00640 1 -- 172.21.15.67:0/1209122082 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fba7c01a070 con 0x7fba940cadb0 2024-03-09T09:21:58.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.244+0000 7fba90c00640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba7006b480 0x7fba7006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:21:58.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.244+0000 7fba90c00640 1 -- 172.21.15.67:0/1209122082 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fba7c08ab70 con 0x7fba940cadb0 2024-03-09T09:21:58.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.245+0000 7fba98c00640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba7006b480 0x7fba7006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:21:58.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.246+0000 7fba98c00640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba7006b480 0x7fba7006d940 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0013b0 tx=0x7fba8c007db0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:21:58.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.261+0000 7fba90c00640 1 -- 172.21.15.67:0/1209122082 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fba7c059020 con 0x7fba940cadb0 2024-03-09T09:21:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.554+0000 7fba9a000640 1 -- 172.21.15.67:0/1209122082 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fba940cab90 con 0x7fba940cadb0 2024-03-09T09:21:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.555+0000 7fba90c00640 1 -- 172.21.15.67:0/1209122082 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fba7c05d0c0 con 0x7fba940cadb0 2024-03-09T09:21:58.562 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":226,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:21:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 -- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba7006b480 msgr2=0x7fba7006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba7006b480 0x7fba7006d940 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0013b0 tx=0x7fba8c007db0 comp rx=0 tx=0).stop 2024-03-09T09:21:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 -- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba940cadb0 msgr2=0x7fba9414b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:21:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba940cadb0 0x7fba9414b4d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00e8a0 tx=0x7fba7c00ed60 comp rx=0 tx=0).stop 2024-03-09T09:21:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 -- 172.21.15.67:0/1209122082 shutdown_connections 2024-03-09T09:21:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fba7006b480 0x7fba7006d940 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fba94153300 0x7fba9414ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba940cadb0 0x7fba9414b4d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 --2- 172.21.15.67:0/1209122082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba940c5e40 0x7fba9406c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:21:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 -- 172.21.15.67:0/1209122082 >> 172.21.15.67:0/1209122082 conn(0x7fba940732d0 msgr2=0x7fba94150e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:21:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 -- 172.21.15.67:0/1209122082 shutdown_connections 2024-03-09T09:21:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:21:58.558+0000 7fba9a000640 1 -- 172.21.15.67:0/1209122082 wait complete. 2024-03-09T09:21:59.731 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:21:59.731 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:21:59.731 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 65 -eq 48 ']' 2024-03-09T09:21:59.731 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.65 2024-03-09T09:21:59.731 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.65 2024-03-09T09:21:59.731 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.65 2024-03-09T09:21:59.732 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.65 2024-03-09T09:21:59.735 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:21:59.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:21:59.900 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:22:01.584 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:22:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.829+0000 7f2d86200640 1 Processor -- start 2024-03-09T09:22:01.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.830+0000 7f2d86200640 1 -- start start 2024-03-09T09:22:01.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.831+0000 7f2d86200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.831+0000 7f2d86200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.831+0000 7f2d86200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d800c6a40 con 0x7f2d800c96f0 2024-03-09T09:22:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.831+0000 7f2d86200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8006b5b0 con 0x7f2d8006b040 2024-03-09T09:22:01.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.831+0000 7f2d86200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8006b890 con 0x7f2d800c5f50 2024-03-09T09:22:01.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7ea00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7f400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:51496/0 (socket says 172.21.15.67:51496) 2024-03-09T09:22:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 learned_addr learned my addr 172.21.15.67:0/726196878 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7fe00640 1 --1- 172.21.15.67:0/726196878 >> v1:172.21.15.134:6789/0 conn(0x7f2d800c96f0 0x7f2d800c6330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:47518/0 (socket says 172.21.15.67:47518) 2024-03-09T09:22:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 msgr2=0x7f2d8006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 msgr2=0x7f2d8006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 msgr2=0x7f2d80155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:22:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 msgr2=0x7f2d80155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:01.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:01.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:01.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:01.832+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:02.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.033+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:02.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.034+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.034+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 msgr2=0x7f2d8006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:22:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.034+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 msgr2=0x7f2d8006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.034+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.034+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 msgr2=0x7f2d80155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:02.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.034+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 msgr2=0x7f2d80155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:02.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.034+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:02.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.034+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:22:02.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.034+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:22:02.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.436+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:02.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.436+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:02.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.436+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 msgr2=0x7f2d8006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:22:02.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.436+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 msgr2=0x7f2d8006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:02.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.436+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:02.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.436+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 msgr2=0x7f2d80155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:02.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.436+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 msgr2=0x7f2d80155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:02.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.436+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:02.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.436+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:22:02.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:02.436+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:22:03.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:03.237+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:03.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:03.238+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:03.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:03.238+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 msgr2=0x7f2d8006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:22:03.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:03.238+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 msgr2=0x7f2d8006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:03.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:03.238+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 msgr2=0x7f2d80155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:03.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:03.238+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 msgr2=0x7f2d80155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:03.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:03.238+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:03.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:03.238+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:03.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:03.238+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:22:03.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:03.238+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:22:04.368 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:22:04.368 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:22:04.368 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 60 -eq 48 ']' 2024-03-09T09:22:04.368 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.60 2024-03-09T09:22:04.369 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.60 2024-03-09T09:22:04.369 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.60 2024-03-09T09:22:04.369 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.60 2024-03-09T09:22:04.372 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:22:04.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:04.831+0000 7f2d7cc00640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 msgr2=0x7f2d80155e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:04.831+0000 7f2d7cc00640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d800c5f50 0x7f2d80155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:04.831+0000 7f2d7cc00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 msgr2=0x7f2d8006c650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:04.831+0000 7f2d7cc00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.134:3300/0 conn(0x7f2d8006b040 0x7f2d8006c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:04.831+0000 7f2d7cc00640 1 -- 172.21.15.67:0/726196878 >> v1:172.21.15.134:6789/0 conn(0x7f2d800c96f0 legacy=0x7f2d800c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:04.831+0000 7f2d7cc00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 0x7f2d60009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:04.831+0000 7f2d7cc00640 1 -- 172.21.15.67:0/726196878 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8006bb70 con 0x7f2d600036c0 2024-03-09T09:22:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:04.831+0000 7f2d7cc00640 1 -- 172.21.15.67:0/726196878 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8006b5b0 con 0x7f2d60007350 2024-03-09T09:22:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:04.832+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> v1:172.21.15.67:6789/0 conn(0x7f2d600036c0 legacy=0x7f2d60005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:22:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:04.832+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 msgr2=0x7f2d60009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:04.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:04.832+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 0x7f2d60009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:05.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:05.033+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 msgr2=0x7f2d60009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:05.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:05.033+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> v1:172.21.15.67:6789/0 conn(0x7f2d600036c0 legacy=0x7f2d60005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:22:05.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:05.033+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 0x7f2d60009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:22:05.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:05.434+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 msgr2=0x7f2d60009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:05.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:05.434+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> v1:172.21.15.67:6789/0 conn(0x7f2d600036c0 legacy=0x7f2d60005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:22:05.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:05.434+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 0x7f2d60009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:22:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:06.235+0000 7f2d7ea00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 msgr2=0x7f2d60009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:06.235+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 0x7f2d60009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:22:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:06.235+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> v1:172.21.15.67:6789/0 conn(0x7f2d600036c0 legacy=0x7f2d60005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:22:07.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.833+0000 7f2d7cc00640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 msgr2=0x7f2d60009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.833+0000 7f2d7cc00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 0x7f2d60009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.833+0000 7f2d7cc00640 1 -- 172.21.15.67:0/726196878 >> v1:172.21.15.67:6789/0 conn(0x7f2d600036c0 legacy=0x7f2d60005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.833+0000 7f2d7fe00640 1 -- 172.21.15.67:0/726196878 reap_dead start 2024-03-09T09:22:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.833+0000 7f2d7cc00640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 0x7f2d60009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.833+0000 7f2d7cc00640 1 -- 172.21.15.67:0/726196878 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6000af70 con 0x7f2d6000a5f0 2024-03-09T09:22:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.833+0000 7f2d7cc00640 1 -- 172.21.15.67:0/726196878 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8006bb70 con 0x7f2d800c96f0 2024-03-09T09:22:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.833+0000 7f2d7cc00640 1 -- 172.21.15.67:0/726196878 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8006b5b0 con 0x7f2d60007350 2024-03-09T09:22:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.833+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 msgr2=0x7f2d60009810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.833+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 0x7f2d60009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.833+0000 7f2d7fe00640 1 -- 172.21.15.67:0/726196878 >> v1:172.21.15.67:6789/0 conn(0x7f2d6000a5f0 legacy=0x7f2d60005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:22:07.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.835+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835316171 0 0) 0x7f2d8006bb70 con 0x7f2d800c96f0 2024-03-09T09:22:07.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.835+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d64003520 con 0x7f2d800c96f0 2024-03-09T09:22:07.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.836+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1854612533 0 0) 0x7f2d64003520 con 0x7f2d800c96f0 2024-03-09T09:22:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.836+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 msgr2=0x7f2d60009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.836+0000 7f2d7e000640 1 --2- 172.21.15.67:0/726196878 >> v2:172.21.15.67:3300/0 conn(0x7f2d60007350 0x7f2d60009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.836+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 >> v1:172.21.15.67:6789/0 conn(0x7f2d6000a5f0 legacy=0x7f2d60005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.836+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6000abe0 con 0x7f2d800c96f0 2024-03-09T09:22:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.836+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f2d6c00f730 con 0x7f2d800c96f0 2024-03-09T09:22:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.836+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 >> v1:172.21.15.134:6789/0 conn(0x7f2d800c96f0 legacy=0x7f2d60009eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.836+0000 7f2d7e000640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 0x7f2d64003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.837+0000 7f2d7e000640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d64004800 0x7f2d64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.837+0000 7f2d7e000640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d640052e0 0x7f2d6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:07.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.837+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8006b5b0 con 0x7f2d640052e0 2024-03-09T09:22:07.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.837+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6400c230 con 0x7f2d640039b0 2024-03-09T09:22:07.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.837+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6400c430 con 0x7f2d64004800 2024-03-09T09:22:07.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.837+0000 7f2d7fe00640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d640052e0 msgr2=0x7f2d6400b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:07.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.837+0000 7f2d7fe00640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d640052e0 0x7f2d6400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:07.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.837+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 0x7f2d64003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:07.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.837+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d64004800 0x7f2d64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:07.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.838+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d64004800 msgr2=0x7f2d64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:07.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.838+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d64004800 0x7f2d64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:07.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.838+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d640052e0 msgr2=0x7f2d6400b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:07.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.838+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d640052e0 0x7f2d6400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:07.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.838+0000 7f2d7fe00640 1 -- 172.21.15.67:0/726196878 reap_dead start 2024-03-09T09:22:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.838+0000 7f2d7f400640 1 -- 172.21.15.67:0/726196878 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64003520 con 0x7f2d640039b0 2024-03-09T09:22:07.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.841+0000 7f2d7f400640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 0x7f2d64003db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009100 tx=0x7f2d64005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aedf61a6dba069e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:07.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.842+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7003a070 con 0x7f2d640039b0 2024-03-09T09:22:07.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.842+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d70035410 con 0x7f2d640039b0 2024-03-09T09:22:08.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.842+0000 7f2d7e000640 1 -- 172.21.15.67:0/726196878 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000f040 con 0x7f2d640039b0 2024-03-09T09:22:08.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.843+0000 7f2d86200640 1 -- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 msgr2=0x7f2d64003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.843+0000 7f2d86200640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 0x7f2d64003db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009100 tx=0x7f2d64005830 comp rx=0 tx=0).stop 2024-03-09T09:22:08.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.844+0000 7f2d86200640 1 -- 172.21.15.67:0/726196878 shutdown_connections 2024-03-09T09:22:08.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.844+0000 7f2d86200640 1 --2- 172.21.15.67:0/726196878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 0x7f2d64003db0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.844+0000 7f2d86200640 1 -- 172.21.15.67:0/726196878 >> 172.21.15.67:0/726196878 conn(0x7f2d800c8050 msgr2=0x7f2d800c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:08.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.844+0000 7f2d86200640 1 -- 172.21.15.67:0/726196878 shutdown_connections 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.844+0000 7f2d86200640 1 -- 172.21.15.67:0/726196878 wait complete. 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.847+0000 7f2d86200640 1 Processor -- start 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.847+0000 7f2d86200640 1 -- start start 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.849+0000 7f2d86200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 0x7f2d800c0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.849+0000 7f2d86200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d800ac170 0x7f2d800c0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.849+0000 7f2d86200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d801ef180 0x7f2d801ef610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.849+0000 7f2d86200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8014cc00 con 0x7f2d800ac170 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.849+0000 7f2d86200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8006c130 con 0x7f2d640039b0 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.849+0000 7f2d86200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800c9b80 con 0x7f2d801ef180 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.849+0000 7f2d7ea00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d800ac170 msgr2=0x7f2d800c0ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.849+0000 7f2d7ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d800ac170 0x7f2d800c0ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.849+0000 7f2d7f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 0x7f2d800c0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.850+0000 7f2d7f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 0x7f2d800c0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:33198/0 (socket says 172.21.15.67:33198) 2024-03-09T09:22:08.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.850+0000 7f2d7f400640 1 -- 172.21.15.67:0/3005924617 learned_addr learned my addr 172.21.15.67:0/3005924617 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.850+0000 7f2d7fe00640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d801ef180 0x7f2d801ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.850+0000 7f2d7f400640 1 -- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d801ef180 msgr2=0x7f2d801ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.850+0000 7f2d7f400640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d801ef180 0x7f2d801ef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.850+0000 7f2d7f400640 1 -- 172.21.15.67:0/3005924617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d800ac170 msgr2=0x7f2d800c0ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.850+0000 7f2d7f400640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d800ac170 0x7f2d800c0ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.850+0000 7f2d7f400640 1 -- 172.21.15.67:0/3005924617 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d800c1af0 con 0x7f2d640039b0 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.851+0000 7f2d7f400640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 0x7f2d800c0980 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2d70010300 tx=0x7f2d70004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.855+0000 7f2d77400640 1 -- 172.21.15.67:0/3005924617 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70044070 con 0x7f2d640039b0 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.855+0000 7f2d86200640 1 -- 172.21.15.67:0/3005924617 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d800c2070 con 0x7f2d640039b0 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.855+0000 7f2d77400640 1 -- 172.21.15.67:0/3005924617 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d700100b0 con 0x7f2d640039b0 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.855+0000 7f2d77400640 1 -- 172.21.15.67:0/3005924617 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7003a040 con 0x7f2d640039b0 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.855+0000 7f2d86200640 1 -- 172.21.15.67:0/3005924617 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8014d470 con 0x7f2d640039b0 2024-03-09T09:22:08.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.856+0000 7f2d86200640 1 -- 172.21.15.67:0/3005924617 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d80075a00 con 0x7f2d640039b0 2024-03-09T09:22:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.863+0000 7f2d77400640 1 -- 172.21.15.67:0/3005924617 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2d7003d070 con 0x7f2d640039b0 2024-03-09T09:22:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.864+0000 7f2d77400640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d6006c5d0 0x7f2d6006ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.864+0000 7f2d77400640 1 -- 172.21.15.67:0/3005924617 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f2d700ac3b0 con 0x7f2d640039b0 2024-03-09T09:22:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.865+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d6006c5d0 0x7f2d6006ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.866+0000 7f2d7ea00640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d6006c5d0 0x7f2d6006ea90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2d800ac550 tx=0x7f2d6c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:07.880+0000 7f2d77400640 1 -- 172.21.15.67:0/3005924617 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2d7007a760 con 0x7f2d640039b0 2024-03-09T09:22:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.166+0000 7f2d86200640 1 -- 172.21.15.67:0/3005924617 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2d8006cca0 con 0x7f2d640039b0 2024-03-09T09:22:08.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.168+0000 7f2d77400640 1 -- 172.21.15.67:0/3005924617 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f2d7007e800 con 0x7f2d640039b0 2024-03-09T09:22:08.207 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":228,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.171+0000 7f2d86200640 1 -- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d6006c5d0 msgr2=0x7f2d6006ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.171+0000 7f2d86200640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d6006c5d0 0x7f2d6006ea90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2d800ac550 tx=0x7f2d6c008040 comp rx=0 tx=0).stop 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.171+0000 7f2d86200640 1 -- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 msgr2=0x7f2d800c0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.171+0000 7f2d86200640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 0x7f2d800c0980 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2d70010300 tx=0x7f2d70004730 comp rx=0 tx=0).stop 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.171+0000 7f2d86200640 1 -- 172.21.15.67:0/3005924617 shutdown_connections 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.171+0000 7f2d86200640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2d6006c5d0 0x7f2d6006ea90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.171+0000 7f2d86200640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2d801ef180 0x7f2d801ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.171+0000 7f2d86200640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2d800ac170 0x7f2d800c0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.171+0000 7f2d86200640 1 --2- 172.21.15.67:0/3005924617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d640039b0 0x7f2d800c0980 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.171+0000 7f2d86200640 1 -- 172.21.15.67:0/3005924617 >> 172.21.15.67:0/3005924617 conn(0x7f2d800c8050 msgr2=0x7f2d80157de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.172+0000 7f2d86200640 1 -- 172.21.15.67:0/3005924617 shutdown_connections 2024-03-09T09:22:08.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.172+0000 7f2d86200640 1 -- 172.21.15.67:0/3005924617 wait complete. 2024-03-09T09:22:08.210 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:22:08.210 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:22:08.210 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:22:08.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.410+0000 7f2c66000640 1 Processor -- start 2024-03-09T09:22:08.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.411+0000 7f2c66000640 1 -- start start 2024-03-09T09:22:08.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.412+0000 7f2c66000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c60150fb0 0x7f2c600c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.412+0000 7f2c66000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c60154410 0x7f2c6006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.412+0000 7f2c66000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2c600c6360 0x7f2c6006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.412+0000 7f2c66000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c600c5aa0 con 0x7f2c60154410 2024-03-09T09:22:08.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.412+0000 7f2c66000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c600c51b0 con 0x7f2c600c6360 2024-03-09T09:22:08.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.412+0000 7f2c66000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60151390 con 0x7f2c60150fb0 2024-03-09T09:22:08.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.412+0000 7f2c57e00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f2c60154410 msgr2=0x7f2c6006f790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:08.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.413+0000 7f2c57e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c60154410 0x7f2c6006f790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:08.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.413+0000 7f2c64c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c60150fb0 0x7f2c600c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.413+0000 7f2c64c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c60150fb0 0x7f2c600c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:51548/0 (socket says 172.21.15.67:51548) 2024-03-09T09:22:08.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.413+0000 7f2c65600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2c600c6360 0x7f2c6006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.413+0000 7f2c64c00640 1 -- 172.21.15.67:0/2013678974 learned_addr learned my addr 172.21.15.67:0/2013678974 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.414+0000 7f2c64c00640 1 -- 172.21.15.67:0/2013678974 >> v2:172.21.15.134:3300/0 conn(0x7f2c600c6360 msgr2=0x7f2c6006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.414+0000 7f2c64c00640 1 --2- 172.21.15.67:0/2013678974 >> v2:172.21.15.134:3300/0 conn(0x7f2c600c6360 0x7f2c6006fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.414+0000 7f2c64c00640 1 -- 172.21.15.67:0/2013678974 >> v2:172.21.15.67:3300/0 conn(0x7f2c60154410 msgr2=0x7f2c6006f790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.414+0000 7f2c64c00640 1 --2- 172.21.15.67:0/2013678974 >> v2:172.21.15.67:3300/0 conn(0x7f2c60154410 0x7f2c6006f790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.414+0000 7f2c64c00640 1 -- 172.21.15.67:0/2013678974 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c600c5450 con 0x7f2c60150fb0 2024-03-09T09:22:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.414+0000 7f2c65600640 1 --2- 172.21.15.67:0/2013678974 >> v2:172.21.15.134:3300/0 conn(0x7f2c600c6360 0x7f2c6006fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:22:08.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.415+0000 7f2c64c00640 1 --2- 172.21.15.67:0/2013678974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c60150fb0 0x7f2c600c5e20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2c6006c060 tx=0x7f2c5002bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6284cd096f508ce6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.415+0000 7f2c5fe00640 1 -- 172.21.15.67:0/2013678974 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5002c980 con 0x7f2c60150fb0 2024-03-09T09:22:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.416+0000 7f2c5fe00640 1 -- 172.21.15.67:0/2013678974 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5002cb20 con 0x7f2c60150fb0 2024-03-09T09:22:08.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.416+0000 7f2c5fe00640 1 -- 172.21.15.67:0/2013678974 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c500343f0 con 0x7f2c60150fb0 2024-03-09T09:22:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.416+0000 7f2c66000640 1 -- 172.21.15.67:0/2013678974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c60150fb0 msgr2=0x7f2c600c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.416+0000 7f2c66000640 1 --2- 172.21.15.67:0/2013678974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c60150fb0 0x7f2c600c5e20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2c6006c060 tx=0x7f2c5002bde0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.416+0000 7f2c66000640 1 -- 172.21.15.67:0/2013678974 shutdown_connections 2024-03-09T09:22:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.416+0000 7f2c66000640 1 --2- 172.21.15.67:0/2013678974 >> v2:172.21.15.134:3300/0 conn(0x7f2c600c6360 0x7f2c6006fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.416+0000 7f2c66000640 1 --2- 172.21.15.67:0/2013678974 >> v2:172.21.15.67:3300/0 conn(0x7f2c60154410 0x7f2c6006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.416+0000 7f2c66000640 1 --2- 172.21.15.67:0/2013678974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c60150fb0 0x7f2c600c5e20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.416+0000 7f2c66000640 1 -- 172.21.15.67:0/2013678974 >> 172.21.15.67:0/2013678974 conn(0x7f2c6006f060 msgr2=0x7f2c60066d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:08.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.416+0000 7f2c66000640 1 -- 172.21.15.67:0/2013678974 shutdown_connections 2024-03-09T09:22:08.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.416+0000 7f2c66000640 1 -- 172.21.15.67:0/2013678974 wait complete. 2024-03-09T09:22:08.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.418+0000 7f2c66000640 1 Processor -- start 2024-03-09T09:22:08.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.418+0000 7f2c66000640 1 -- start start 2024-03-09T09:22:08.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.419+0000 7f2c66000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c600c6360 0x7f2c60149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.419+0000 7f2c66000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c60150fb0 0x7f2c6014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.419+0000 7f2c66000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c60154410 0x7f2c600ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.419+0000 7f2c66000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c601548f0 con 0x7f2c60154410 2024-03-09T09:22:08.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.419+0000 7f2c66000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60072a50 con 0x7f2c60150fb0 2024-03-09T09:22:08.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.419+0000 7f2c66000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60073bc0 con 0x7f2c600c6360 2024-03-09T09:22:08.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.419+0000 7f2c65600640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c60154410 msgr2=0x7f2c600ac430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:08.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.419+0000 7f2c65600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c60154410 0x7f2c600ac430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:08.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.420+0000 7f2c57e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c60150fb0 0x7f2c6014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.420+0000 7f2c64c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c600c6360 0x7f2c60149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.420+0000 7f2c57e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c60150fb0 0x7f2c6014cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:33222/0 (socket says 172.21.15.67:33222) 2024-03-09T09:22:08.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.420+0000 7f2c57e00640 1 -- 172.21.15.67:0/410230840 learned_addr learned my addr 172.21.15.67:0/410230840 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:08.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.420+0000 7f2c57e00640 1 -- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c600c6360 msgr2=0x7f2c60149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.420+0000 7f2c57e00640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c600c6360 0x7f2c60149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.420+0000 7f2c57e00640 1 -- 172.21.15.67:0/410230840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c60154410 msgr2=0x7f2c600ac430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:22:08.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.420+0000 7f2c57e00640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c60154410 0x7f2c600ac430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.420+0000 7f2c57e00640 1 -- 172.21.15.67:0/410230840 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c60147910 con 0x7f2c60150fb0 2024-03-09T09:22:08.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.421+0000 7f2c57e00640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c60150fb0 0x7f2c6014cee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800b550 tx=0x7f2c4800ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.421+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c48003dc0 con 0x7f2c60150fb0 2024-03-09T09:22:08.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.421+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c48015070 con 0x7f2c60150fb0 2024-03-09T09:22:08.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.421+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c480052b0 con 0x7f2c60150fb0 2024-03-09T09:22:08.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.421+0000 7f2c66000640 1 -- 172.21.15.67:0/410230840 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c60149c40 con 0x7f2c60150fb0 2024-03-09T09:22:08.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.421+0000 7f2c66000640 1 -- 172.21.15.67:0/410230840 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c601ebf60 con 0x7f2c60150fb0 2024-03-09T09:22:08.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.422+0000 7f2c66000640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600ae160 0x7f2c600ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.422+0000 7f2c66000640 1 -- 172.21.15.67:0/410230840 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6004f3c0 con 0x7f2c600ae160 2024-03-09T09:22:08.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.423+0000 7f2c64c00640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600ae160 0x7f2c600ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.424+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2c48005450 con 0x7f2c60150fb0 2024-03-09T09:22:08.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.424+0000 7f2c64c00640 1 -- 172.21.15.67:0/410230840 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c60073790 con 0x7f2c600ae160 2024-03-09T09:22:08.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.425+0000 7f2c64c00640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600ae160 0x7f2c600ac970 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c6006f560 tx=0x7f2c50004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.425+0000 7f2c5d600640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c3c06b480 0x7f2c3c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.425+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f2c4808a220 con 0x7f2c60150fb0 2024-03-09T09:22:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.425+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50045070 con 0x7f2c600ae160 2024-03-09T09:22:08.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.426+0000 7f2c65600640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c3c06b480 0x7f2c3c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.426+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f2c500094d0 con 0x7f2c600ae160 2024-03-09T09:22:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.426+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600ae160 msgr2=0x7f2c600ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.426+0000 7f2c5d600640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600ae160 0x7f2c600ac970 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c6006f560 tx=0x7f2c50004a40 comp rx=0 tx=0).stop 2024-03-09T09:22:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.426+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600ae160 msgr2=0x7f2c600ac970 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.426+0000 7f2c5d600640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600ae160 0x7f2c600ac970 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c6006f560 tx=0x7f2c50004a40 comp rx=0 tx=0).stop 2024-03-09T09:22:08.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.427+0000 7f2c65600640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c3c06b480 0x7f2c3c06d940 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009560 tx=0x7f2c58009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.433+0000 7f2c66000640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600aceb0 0x7f2c601ec130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.433+0000 7f2c66000640 1 -- 172.21.15.67:0/410230840 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c601eb9c0 con 0x7f2c600aceb0 2024-03-09T09:22:08.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.434+0000 7f2c64c00640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600aceb0 0x7f2c601ec130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.435+0000 7f2c64c00640 1 -- 172.21.15.67:0/410230840 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2c600ac180 con 0x7f2c600aceb0 2024-03-09T09:22:08.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.435+0000 7f2c64c00640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600aceb0 0x7f2c601ec130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2c6004fcf0 tx=0x7f2c50009000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.436+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50004240 con 0x7f2c600aceb0 2024-03-09T09:22:08.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.436+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7f2c50004240 con 0x7f2c600aceb0 2024-03-09T09:22:08.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.436+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600aceb0 msgr2=0x7f2c601ec130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.436+0000 7f2c5d600640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600aceb0 0x7f2c601ec130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2c6004fcf0 tx=0x7f2c50009000 comp rx=0 tx=0).stop 2024-03-09T09:22:08.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.436+0000 7f2c5d600640 1 -- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600aceb0 msgr2=0x7f2c601ec130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:08.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.436+0000 7f2c5d600640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c600aceb0 0x7f2c601ec130 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2c6004fcf0 tx=0x7f2c50009000 comp rx=0 tx=0).stop 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 228, 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:22:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:22:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:22:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:22:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:22:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:22:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:22:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:22:08.457 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:22:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.437+0000 7f2c66000640 1 -- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c3c06b480 msgr2=0x7f2c3c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.437+0000 7f2c66000640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c3c06b480 0x7f2c3c06d940 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009560 tx=0x7f2c58009190 comp rx=0 tx=0).stop 2024-03-09T09:22:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.437+0000 7f2c66000640 1 -- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c60150fb0 msgr2=0x7f2c6014cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.437+0000 7f2c66000640 1 --2- 172.21.15.67:0/410230840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c60150fb0 0x7f2c6014cee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800b550 tx=0x7f2c4800ba10 comp rx=0 tx=0).stop 2024-03-09T09:22:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.437+0000 7f2c65600640 1 -- 172.21.15.67:0/410230840 reap_dead start 2024-03-09T09:22:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.437+0000 7f2c65600640 1 -- 172.21.15.67:0/410230840 reap_dead start 2024-03-09T09:22:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.437+0000 7f2c66000640 1 -- 172.21.15.67:0/410230840 shutdown_connections 2024-03-09T09:22:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.437+0000 7f2c66000640 1 -- 172.21.15.67:0/410230840 >> 172.21.15.67:0/410230840 conn(0x7f2c6006f060 msgr2=0x7f2c60153330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.437+0000 7f2c66000640 1 -- 172.21.15.67:0/410230840 shutdown_connections 2024-03-09T09:22:08.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.437+0000 7f2c66000640 1 -- 172.21.15.67:0/410230840 wait complete. 2024-03-09T09:22:08.460 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:22:08.461 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:22:08.461 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:22:08.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.671+0000 7f4f87200640 1 Processor -- start 2024-03-09T09:22:08.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.672+0000 7f4f87200640 1 -- start start 2024-03-09T09:22:08.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.673+0000 7f4f87200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 0x7f4f88156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.673+0000 7f4f87200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4f880c9380 0x7f4f880acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.673+0000 7f4f87200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f880ad440 0x7f4f8806c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.673+0000 7f4f87200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8806ba60 con 0x7f4f880ad440 2024-03-09T09:22:08.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.673+0000 7f4f87200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f88156bc0 con 0x7f4f880c9380 2024-03-09T09:22:08.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.673+0000 7f4f87200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f880c9890 con 0x7f4f8814fd90 2024-03-09T09:22:08.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.674+0000 7f4f86800640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f4f880ad440 msgr2=0x7f4f8806c1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:08.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.674+0000 7f4f86800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4f880ad440 0x7f4f8806c1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:08.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.674+0000 7f4f85e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 0x7f4f88156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.674+0000 7f4f85400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4f880c9380 0x7f4f880acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.674+0000 7f4f85e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 0x7f4f88156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:51564/0 (socket says 172.21.15.67:51564) 2024-03-09T09:22:08.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.674+0000 7f4f85e00640 1 -- 172.21.15.67:0/3155240220 learned_addr learned my addr 172.21.15.67:0/3155240220 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:08.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.675+0000 7f4f85e00640 1 -- 172.21.15.67:0/3155240220 >> v2:172.21.15.134:3300/0 conn(0x7f4f880c9380 msgr2=0x7f4f880acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:08.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.675+0000 7f4f85e00640 1 --2- 172.21.15.67:0/3155240220 >> v2:172.21.15.134:3300/0 conn(0x7f4f880c9380 0x7f4f880acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.675+0000 7f4f85e00640 1 -- 172.21.15.67:0/3155240220 >> v2:172.21.15.67:3300/0 conn(0x7f4f880ad440 msgr2=0x7f4f8806c1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:08.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.675+0000 7f4f85e00640 1 --2- 172.21.15.67:0/3155240220 >> v2:172.21.15.67:3300/0 conn(0x7f4f880ad440 0x7f4f8806c1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.675+0000 7f4f85e00640 1 -- 172.21.15.67:0/3155240220 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f880c5f80 con 0x7f4f8814fd90 2024-03-09T09:22:08.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.676+0000 7f4f85e00640 1 --2- 172.21.15.67:0/3155240220 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 0x7f4f88156680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4f880aad00 tx=0x7f4f7802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ad410b645e1405 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.676+0000 7f4f84a00640 1 -- 172.21.15.67:0/3155240220 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78031780 con 0x7f4f8814fd90 2024-03-09T09:22:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.676+0000 7f4f84a00640 1 -- 172.21.15.67:0/3155240220 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f78037070 con 0x7f4f8814fd90 2024-03-09T09:22:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.676+0000 7f4f84a00640 1 -- 172.21.15.67:0/3155240220 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78031b40 con 0x7f4f8814fd90 2024-03-09T09:22:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.677+0000 7f4f87200640 1 -- 172.21.15.67:0/3155240220 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 msgr2=0x7f4f88156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.677+0000 7f4f87200640 1 --2- 172.21.15.67:0/3155240220 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 0x7f4f88156680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4f880aad00 tx=0x7f4f7802b820 comp rx=0 tx=0).stop 2024-03-09T09:22:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.677+0000 7f4f87200640 1 -- 172.21.15.67:0/3155240220 shutdown_connections 2024-03-09T09:22:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.677+0000 7f4f87200640 1 --2- 172.21.15.67:0/3155240220 >> v2:172.21.15.67:3300/0 conn(0x7f4f880ad440 0x7f4f8806c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.677+0000 7f4f87200640 1 --2- 172.21.15.67:0/3155240220 >> v2:172.21.15.134:3300/0 conn(0x7f4f880c9380 0x7f4f880acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.677+0000 7f4f87200640 1 --2- 172.21.15.67:0/3155240220 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 0x7f4f88156680 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.677+0000 7f4f87200640 1 -- 172.21.15.67:0/3155240220 >> 172.21.15.67:0/3155240220 conn(0x7f4f880cb480 msgr2=0x7f4f880cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:08.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.677+0000 7f4f87200640 1 -- 172.21.15.67:0/3155240220 shutdown_connections 2024-03-09T09:22:08.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.677+0000 7f4f87200640 1 -- 172.21.15.67:0/3155240220 wait complete. 2024-03-09T09:22:08.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.679+0000 7f4f87200640 1 Processor -- start 2024-03-09T09:22:08.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.679+0000 7f4f87200640 1 -- start start 2024-03-09T09:22:08.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f87200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f880ad440 0x7f4f881eef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f87200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f880c9380 0x7f4f8814c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f87200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 0x7f4f880b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f87200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8806bcb0 con 0x7f4f880c9380 2024-03-09T09:22:08.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f87200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f880c6520 con 0x7f4f880ad440 2024-03-09T09:22:08.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f87200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8806c710 con 0x7f4f8814fd90 2024-03-09T09:22:08.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f85400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f880c9380 msgr2=0x7f4f8814c090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:08.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f85400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f880c9380 0x7f4f8814c090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:08.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f85e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f880ad440 0x7f4f881eef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f86800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 0x7f4f880b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f86800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 0x7f4f880b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:51578/0 (socket says 172.21.15.67:51578) 2024-03-09T09:22:08.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f85e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f880ad440 0x7f4f881eef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:33254/0 (socket says 172.21.15.67:33254) 2024-03-09T09:22:08.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.680+0000 7f4f86800640 1 -- 172.21.15.67:0/3798923396 learned_addr learned my addr 172.21.15.67:0/3798923396 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.681+0000 7f4f86800640 1 -- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f880ad440 msgr2=0x7f4f881eef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.681+0000 7f4f86800640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f880ad440 0x7f4f881eef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.681+0000 7f4f86800640 1 -- 172.21.15.67:0/3798923396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f880c9380 msgr2=0x7f4f8814c090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:22:08.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.681+0000 7f4f86800640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f880c9380 0x7f4f8814c090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.681+0000 7f4f86800640 1 -- 172.21.15.67:0/3798923396 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f881ef4e0 con 0x7f4f8814fd90 2024-03-09T09:22:08.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.682+0000 7f4f86800640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 0x7f4f880b0720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000bec0 tx=0x7f4f80009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.682+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80016430 con 0x7f4f8814fd90 2024-03-09T09:22:08.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.682+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f80010030 con 0x7f4f8814fd90 2024-03-09T09:22:08.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.682+0000 7f4f87200640 1 -- 172.21.15.67:0/3798923396 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f881ef7c0 con 0x7f4f8814fd90 2024-03-09T09:22:08.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.682+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80016830 con 0x7f4f8814fd90 2024-03-09T09:22:08.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.682+0000 7f4f87200640 1 -- 172.21.15.67:0/3798923396 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f881efd80 con 0x7f4f8814fd90 2024-03-09T09:22:08.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.684+0000 7f4f87200640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f880ac090 0x7f4f880b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.684+0000 7f4f87200640 1 -- 172.21.15.67:0/3798923396 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f880b11a0 con 0x7f4f880ac090 2024-03-09T09:22:08.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.684+0000 7f4f85e00640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f880ac090 0x7f4f880b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.685+0000 7f4f85e00640 1 -- 172.21.15.67:0/3798923396 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f8804f3c0 con 0x7f4f880ac090 2024-03-09T09:22:08.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.685+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4f800153e0 con 0x7f4f8814fd90 2024-03-09T09:22:08.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.685+0000 7f4f85e00640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f880ac090 0x7f4f880b0c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4f881537d0 tx=0x7f4f78004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.686+0000 7f4f7e000640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f6806b4b0 0x7f4f6806d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.686+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4f8008ac80 con 0x7f4f8814fd90 2024-03-09T09:22:08.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.687+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78031e60 con 0x7f4f880ac090 2024-03-09T09:22:08.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.687+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4f78037040 con 0x7f4f880ac090 2024-03-09T09:22:08.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.687+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f880ac090 msgr2=0x7f4f880b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.687+0000 7f4f7e000640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f880ac090 0x7f4f880b0c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4f881537d0 tx=0x7f4f78004730 comp rx=0 tx=0).stop 2024-03-09T09:22:08.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.687+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f880ac090 msgr2=0x7f4f880b0c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:08.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.687+0000 7f4f7e000640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f880ac090 0x7f4f880b0c60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.687+0000 7f4f85400640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f6806b4b0 0x7f4f6806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.688+0000 7f4f85400640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f6806b4b0 0x7f4f6806d970 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4f880ae0f0 tx=0x7f4f74009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.695+0000 7f4f87200640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f88157200 0x7f4f881577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:08.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.695+0000 7f4f87200640 1 -- 172.21.15.67:0/3798923396 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f880c6ae0 con 0x7f4f88157200 2024-03-09T09:22:08.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.695+0000 7f4f85e00640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f88157200 0x7f4f881577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:08.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.696+0000 7f4f85e00640 1 -- 172.21.15.67:0/3798923396 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4f88073a30 con 0x7f4f88157200 2024-03-09T09:22:08.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.696+0000 7f4f85e00640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f88157200 0x7f4f881577c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f7802b860 tx=0x7f4f780046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:08.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.697+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7804db30 con 0x7f4f88157200 2024-03-09T09:22:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.697+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5176 (secure 0 0 0) 0x7f4f7804db30 con 0x7f4f88157200 2024-03-09T09:22:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.697+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f88157200 msgr2=0x7f4f881577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.697+0000 7f4f7e000640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f88157200 0x7f4f881577c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f7802b860 tx=0x7f4f780046c0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.697+0000 7f4f7e000640 1 -- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f88157200 msgr2=0x7f4f881577c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.697+0000 7f4f7e000640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f88157200 0x7f4f881577c0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f7802b860 tx=0x7f4f780046c0 comp rx=0 tx=0).stop 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 228, 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:22:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:22:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:22:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:22:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:22:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:22:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:22:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:22:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:22:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:22:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:22:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:22:08.718 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:22:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.698+0000 7f4f87200640 1 -- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f6806b4b0 msgr2=0x7f4f6806d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.698+0000 7f4f87200640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4f6806b4b0 0x7f4f6806d970 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4f880ae0f0 tx=0x7f4f74009190 comp rx=0 tx=0).stop 2024-03-09T09:22:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.698+0000 7f4f87200640 1 -- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 msgr2=0x7f4f880b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.698+0000 7f4f87200640 1 --2- 172.21.15.67:0/3798923396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4f8814fd90 0x7f4f880b0720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000bec0 tx=0x7f4f80009f30 comp rx=0 tx=0).stop 2024-03-09T09:22:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.698+0000 7f4f86800640 1 -- 172.21.15.67:0/3798923396 reap_dead start 2024-03-09T09:22:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.698+0000 7f4f86800640 1 -- 172.21.15.67:0/3798923396 reap_dead start 2024-03-09T09:22:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.698+0000 7f4f87200640 1 -- 172.21.15.67:0/3798923396 shutdown_connections 2024-03-09T09:22:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.698+0000 7f4f87200640 1 -- 172.21.15.67:0/3798923396 >> 172.21.15.67:0/3798923396 conn(0x7f4f880cb480 msgr2=0x7f4f881540f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:08.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.698+0000 7f4f87200640 1 -- 172.21.15.67:0/3798923396 shutdown_connections 2024-03-09T09:22:08.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:08.698+0000 7f4f87200640 1 -- 172.21.15.67:0/3798923396 wait complete. 2024-03-09T09:22:08.726 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:22:08.727 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:22:09.646 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:22:09.646 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:22:09.646 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 66 -eq 48 ']' 2024-03-09T09:22:09.646 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.66 2024-03-09T09:22:09.646 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.66 2024-03-09T09:22:09.647 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.66 2024-03-09T09:22:09.647 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.66 2024-03-09T09:22:09.650 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:22:15.049 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:22:15.049 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:22:15.049 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 61 -eq 48 ']' 2024-03-09T09:22:15.050 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.61 2024-03-09T09:22:15.050 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.61 2024-03-09T09:22:15.050 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.61 2024-03-09T09:22:15.050 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.61 2024-03-09T09:22:15.054 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:22:15.475 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:22:15.476 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:22:15.476 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 67 -eq 48 ']' 2024-03-09T09:22:15.476 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.67 2024-03-09T09:22:15.476 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.67 2024-03-09T09:22:15.476 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.67 2024-03-09T09:22:15.476 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.67 2024-03-09T09:22:15.480 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:22:19.368 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:22:19.368 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:22:19.368 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 62 -eq 48 ']' 2024-03-09T09:22:19.368 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.62 2024-03-09T09:22:19.369 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.62 2024-03-09T09:22:19.369 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.62 2024-03-09T09:22:19.369 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.62 2024-03-09T09:22:19.372 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:22:21.593 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:22:21.593 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:22:21.593 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 68 -eq 48 ']' 2024-03-09T09:22:21.594 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.68 2024-03-09T09:22:21.594 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.68 2024-03-09T09:22:21.594 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.68 2024-03-09T09:22:21.594 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.68 2024-03-09T09:22:21.598 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:22:27.411 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:22:27.411 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:22:27.411 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 63 -eq 48 ']' 2024-03-09T09:22:27.411 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.63 2024-03-09T09:22:27.412 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.63 2024-03-09T09:22:27.412 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.63 2024-03-09T09:22:27.412 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.63 2024-03-09T09:22:27.415 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:22:28.122 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:22:28.122 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:22:28.122 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 69 -eq 48 ']' 2024-03-09T09:22:28.122 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.69 2024-03-09T09:22:28.122 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.69 2024-03-09T09:22:28.123 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.69 2024-03-09T09:22:28.123 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.69 2024-03-09T09:22:28.413 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:22:28.728 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:22:28.728 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-03-09T09:22:28.728 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-09T09:22:28.728 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-09T09:22:28.737 INFO:tasks.ceph.mon.a:Started 2024-03-09T09:22:28.737 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:22:28.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:22:28.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.934+0000 7f7b33200640 1 Processor -- start 2024-03-09T09:22:28.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.934+0000 7f7b33200640 1 -- start start 2024-03-09T09:22:28.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.935+0000 7f7b33200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 0x7f7b340b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:28.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.936+0000 7f7b33200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7b340c5e40 0x7f7b340b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:28.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.936+0000 7f7b33200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b34157000 con 0x7f7b340cadb0 2024-03-09T09:22:28.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.936+0000 7f7b33200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b340ae430 con 0x7f7b340c5e40 2024-03-09T09:22:28.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.936+0000 7f7b33200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340ae6f0 con 0x7f7b34153300 2024-03-09T09:22:28.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.936+0000 7f7b31e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 0x7f7b340b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:28.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.936+0000 7f7b2be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7b340c5e40 0x7f7b340b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:28.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.936+0000 7f7b31e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 0x7f7b340b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:53380/0 (socket says 172.21.15.67:53380) 2024-03-09T09:22:28.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.936+0000 7f7b31e00640 1 -- 172.21.15.67:0/406518780 learned_addr learned my addr 172.21.15.67:0/406518780 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:28.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.937+0000 7f7b31e00640 1 -- 172.21.15.67:0/406518780 >> v2:172.21.15.134:3300/0 conn(0x7f7b340c5e40 msgr2=0x7f7b340b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:28.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.937+0000 7f7b31e00640 1 --2- 172.21.15.67:0/406518780 >> v2:172.21.15.134:3300/0 conn(0x7f7b340c5e40 0x7f7b340b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:28.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.937+0000 7f7b31e00640 1 -- 172.21.15.67:0/406518780 >> v1:172.21.15.134:6789/0 conn(0x7f7b340cadb0 legacy=0x7f7b34156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:28.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.937+0000 7f7b31e00640 1 -- 172.21.15.67:0/406518780 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b34155b10 con 0x7f7b34153300 2024-03-09T09:22:28.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.939+0000 7f7b31e00640 1 --2- 172.21.15.67:0/406518780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 0x7f7b340b0000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009870 tx=0x7f7b2402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ed2db9bef55026c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.939+0000 7f7b31400640 1 -- 172.21.15.67:0/406518780 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24034070 con 0x7f7b34153300 2024-03-09T09:22:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.939+0000 7f7b31400640 1 -- 172.21.15.67:0/406518780 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b24030840 con 0x7f7b34153300 2024-03-09T09:22:28.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.939+0000 7f7b31400640 1 -- 172.21.15.67:0/406518780 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24030c10 con 0x7f7b34153300 2024-03-09T09:22:28.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.940+0000 7f7b33200640 1 -- 172.21.15.67:0/406518780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 msgr2=0x7f7b340b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:28.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.940+0000 7f7b33200640 1 --2- 172.21.15.67:0/406518780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 0x7f7b340b0000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009870 tx=0x7f7b2402b980 comp rx=0 tx=0).stop 2024-03-09T09:22:28.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.940+0000 7f7b33200640 1 -- 172.21.15.67:0/406518780 shutdown_connections 2024-03-09T09:22:28.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.940+0000 7f7b33200640 1 --2- 172.21.15.67:0/406518780 >> v2:172.21.15.134:3300/0 conn(0x7f7b340c5e40 0x7f7b340b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:28.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.940+0000 7f7b33200640 1 --2- 172.21.15.67:0/406518780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 0x7f7b340b0000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:28.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.940+0000 7f7b33200640 1 -- 172.21.15.67:0/406518780 >> 172.21.15.67:0/406518780 conn(0x7f7b340732d0 msgr2=0x7f7b340ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:28.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.940+0000 7f7b33200640 1 -- 172.21.15.67:0/406518780 shutdown_connections 2024-03-09T09:22:28.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.940+0000 7f7b33200640 1 -- 172.21.15.67:0/406518780 wait complete. 2024-03-09T09:22:28.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.942+0000 7f7b33200640 1 Processor -- start 2024-03-09T09:22:28.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.942+0000 7f7b33200640 1 -- start start 2024-03-09T09:22:28.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b33200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b340c5e40 0x7f7b340accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:28.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b33200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b340cadb0 0x7f7b3414b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:28.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b33200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 0x7f7b3406b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:28.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b33200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340c59e0 con 0x7f7b340cadb0 2024-03-09T09:22:28.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b33200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34159e90 con 0x7f7b340c5e40 2024-03-09T09:22:28.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b33200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34156720 con 0x7f7b34153300 2024-03-09T09:22:28.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b2be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b340cadb0 0x7f7b3414b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:28.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b2be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b340cadb0 0x7f7b3414b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42054/0 (socket says 172.21.15.67:42054) 2024-03-09T09:22:28.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b2be00640 1 -- 172.21.15.67:0/2543004197 learned_addr learned my addr 172.21.15.67:0/2543004197 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:28.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b2be00640 1 -- 172.21.15.67:0/2543004197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b340cadb0 msgr2=0x7f7b3414b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:22:28.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b2be00640 1 -- 172.21.15.67:0/2543004197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b340cadb0 msgr2=0x7f7b3414b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:22:28.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b2be00640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b340cadb0 0x7f7b3414b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:28.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b2be00640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b340cadb0 0x7f7b3414b910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:28.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b31e00640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b340c5e40 0x7f7b340accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:28.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.943+0000 7f7b32800640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 0x7f7b3406b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.944+0000 7f7b32800640 1 -- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b340c5e40 msgr2=0x7f7b340accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.944+0000 7f7b32800640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b340c5e40 0x7f7b340accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.944+0000 7f7b32800640 1 -- 172.21.15.67:0/2543004197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b340cadb0 msgr2=0x7f7b3414b910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:22:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.944+0000 7f7b32800640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b340cadb0 0x7f7b3414b910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.944+0000 7f7b32800640 1 -- 172.21.15.67:0/2543004197 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b340ab040 con 0x7f7b34153300 2024-03-09T09:22:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.945+0000 7f7b32800640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 0x7f7b3406b830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00c760 tx=0x7f7b2c00cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.945+0000 7f7b2a000640 1 -- 172.21.15.67:0/2543004197 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c016020 con 0x7f7b34153300 2024-03-09T09:22:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.945+0000 7f7b2a000640 1 -- 172.21.15.67:0/2543004197 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c006d50 con 0x7f7b34153300 2024-03-09T09:22:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.945+0000 7f7b2a000640 1 -- 172.21.15.67:0/2543004197 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c003bb0 con 0x7f7b34153300 2024-03-09T09:22:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.946+0000 7f7b33200640 1 -- 172.21.15.67:0/2543004197 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b341ec4b0 con 0x7f7b34153300 2024-03-09T09:22:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.946+0000 7f7b33200640 1 -- 172.21.15.67:0/2543004197 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3406ca10 con 0x7f7b34153300 2024-03-09T09:22:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.947+0000 7f7b33200640 1 -- 172.21.15.67:0/2543004197 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b340af7c0 con 0x7f7b34153300 2024-03-09T09:22:28.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.949+0000 7f7b2a000640 1 -- 172.21.15.67:0/2543004197 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7b2c00a850 con 0x7f7b34153300 2024-03-09T09:22:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.950+0000 7f7b2a000640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b1006b4d0 0x7f7b1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:28.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.950+0000 7f7b2a000640 1 -- 172.21.15.67:0/2543004197 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f7b2c08af50 con 0x7f7b34153300 2024-03-09T09:22:28.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.950+0000 7f7b31e00640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b1006b4d0 0x7f7b1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:28.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.951+0000 7f7b31e00640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b1006b4d0 0x7f7b1006d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7b24036000 tx=0x7f7b2403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:28.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:28.963+0000 7f7b2a000640 1 -- 172.21.15.67:0/2543004197 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7b340af7c0 con 0x7f7b34153300 2024-03-09T09:22:29.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.256+0000 7f7b33200640 1 -- 172.21.15.67:0/2543004197 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7b340ab210 con 0x7f7b34153300 2024-03-09T09:22:29.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.257+0000 7f7b2a000640 1 -- 172.21.15.67:0/2543004197 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f7b2c020020 con 0x7f7b34153300 2024-03-09T09:22:29.264 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":228,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:22:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 -- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b1006b4d0 msgr2=0x7f7b1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b1006b4d0 0x7f7b1006d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7b24036000 tx=0x7f7b2403b040 comp rx=0 tx=0).stop 2024-03-09T09:22:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 -- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 msgr2=0x7f7b3406b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 0x7f7b3406b830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00c760 tx=0x7f7b2c00cc20 comp rx=0 tx=0).stop 2024-03-09T09:22:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 -- 172.21.15.67:0/2543004197 shutdown_connections 2024-03-09T09:22:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b1006b4d0 0x7f7b1006d990 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b34153300 0x7f7b3406b830 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b340cadb0 0x7f7b3414b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 --2- 172.21.15.67:0/2543004197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b340c5e40 0x7f7b340accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:29.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 -- 172.21.15.67:0/2543004197 >> 172.21.15.67:0/2543004197 conn(0x7f7b340732d0 msgr2=0x7f7b341515f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 -- 172.21.15.67:0/2543004197 shutdown_connections 2024-03-09T09:22:29.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:29.260+0000 7f7b33200640 1 -- 172.21.15.67:0/2543004197 wait complete. 2024-03-09T09:22:30.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:22:30.056 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:22:32.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:22:32.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.536+0000 7f0e8a800640 1 Processor -- start 2024-03-09T09:22:32.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.536+0000 7f0e8a800640 1 -- start start 2024-03-09T09:22:32.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.537+0000 7f0e8a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 0x7f0e840c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:32.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.537+0000 7f0e8a800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0e8406dc00 0x7f0e840729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:32.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.537+0000 7f0e8a800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e840c4010 con 0x7f0e84153750 2024-03-09T09:22:32.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.537+0000 7f0e8a800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e84156dc0 con 0x7f0e8406dc00 2024-03-09T09:22:32.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.537+0000 7f0e8a800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840c42f0 con 0x7f0e84153370 2024-03-09T09:22:32.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.538+0000 7f0e83400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 0x7f0e840c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:32.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.538+0000 7f0e83400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 0x7f0e840c29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:53430/0 (socket says 172.21.15.67:53430) 2024-03-09T09:22:32.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.538+0000 7f0e83400640 1 -- 172.21.15.67:0/368991950 learned_addr learned my addr 172.21.15.67:0/368991950 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:32.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.538+0000 7f0e83e00640 1 --2- 172.21.15.67:0/368991950 >> v2:172.21.15.134:3300/0 conn(0x7f0e8406dc00 0x7f0e840729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:32.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.539+0000 7f0e83400640 1 -- 172.21.15.67:0/368991950 >> v2:172.21.15.134:3300/0 conn(0x7f0e8406dc00 msgr2=0x7f0e840729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:32.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.539+0000 7f0e83400640 1 --2- 172.21.15.67:0/368991950 >> v2:172.21.15.134:3300/0 conn(0x7f0e8406dc00 0x7f0e840729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:32.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.539+0000 7f0e83400640 1 -- 172.21.15.67:0/368991950 >> v1:172.21.15.134:6789/0 conn(0x7f0e84153750 legacy=0x7f0e840c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:32.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.539+0000 7f0e83400640 1 -- 172.21.15.67:0/368991950 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e840c5a50 con 0x7f0e84153370 2024-03-09T09:22:32.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.540+0000 7f0e83400640 1 --2- 172.21.15.67:0/368991950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 0x7f0e840c29a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009870 tx=0x7f0e7402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ef6605f86895d2a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:32.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.540+0000 7f0e82000640 1 -- 172.21.15.67:0/368991950 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74035020 con 0x7f0e84153370 2024-03-09T09:22:32.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.541+0000 7f0e82000640 1 -- 172.21.15.67:0/368991950 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e74031640 con 0x7f0e84153370 2024-03-09T09:22:32.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.541+0000 7f0e82000640 1 -- 172.21.15.67:0/368991950 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7402d040 con 0x7f0e84153370 2024-03-09T09:22:32.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.541+0000 7f0e8a800640 1 -- 172.21.15.67:0/368991950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 msgr2=0x7f0e840c29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:32.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.541+0000 7f0e8a800640 1 --2- 172.21.15.67:0/368991950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 0x7f0e840c29a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009870 tx=0x7f0e7402b8f0 comp rx=0 tx=0).stop 2024-03-09T09:22:32.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.541+0000 7f0e8a800640 1 -- 172.21.15.67:0/368991950 shutdown_connections 2024-03-09T09:22:32.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.541+0000 7f0e8a800640 1 --2- 172.21.15.67:0/368991950 >> v2:172.21.15.134:3300/0 conn(0x7f0e8406dc00 0x7f0e840729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:32.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.541+0000 7f0e8a800640 1 --2- 172.21.15.67:0/368991950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 0x7f0e840c29a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:32.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.541+0000 7f0e8a800640 1 -- 172.21.15.67:0/368991950 >> 172.21.15.67:0/368991950 conn(0x7f0e840ab320 msgr2=0x7f0e840ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:32.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.541+0000 7f0e8a800640 1 -- 172.21.15.67:0/368991950 shutdown_connections 2024-03-09T09:22:32.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.541+0000 7f0e8a800640 1 -- 172.21.15.67:0/368991950 wait complete. 2024-03-09T09:22:32.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.543+0000 7f0e8a800640 1 Processor -- start 2024-03-09T09:22:32.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.544+0000 7f0e8a800640 1 -- start start 2024-03-09T09:22:32.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.545+0000 7f0e8a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8406dc00 0x7f0e841efa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:32.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.545+0000 7f0e8a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 0x7f0e8406fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:32.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.545+0000 7f0e8a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e84153750 0x7f0e841ec160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:32.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.545+0000 7f0e8a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840c5d30 con 0x7f0e8406dc00 2024-03-09T09:22:32.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.545+0000 7f0e8a800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84073dc0 con 0x7f0e84153750 2024-03-09T09:22:32.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.545+0000 7f0e8a800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84072f00 con 0x7f0e84153370 2024-03-09T09:22:32.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.546+0000 7f0e83400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8406dc00 0x7f0e841efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:32.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.546+0000 7f0e83400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8406dc00 0x7f0e841efa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42056/0 (socket says 172.21.15.67:42056) 2024-03-09T09:22:32.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.546+0000 7f0e83400640 1 -- 172.21.15.67:0/841206201 learned_addr learned my addr 172.21.15.67:0/841206201 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:32.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.546+0000 7f0e83400640 1 -- 172.21.15.67:0/841206201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8406dc00 msgr2=0x7f0e841efa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:22:32.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.546+0000 7f0e83400640 1 -- 172.21.15.67:0/841206201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8406dc00 msgr2=0x7f0e841efa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:22:32.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.546+0000 7f0e83400640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8406dc00 0x7f0e841efa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:32.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.546+0000 7f0e82a00640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 0x7f0e8406fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:32.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.546+0000 7f0e83400640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8406dc00 0x7f0e841efa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:32.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.546+0000 7f0e83e00640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e84153750 0x7f0e841ec160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.547+0000 7f0e82a00640 1 -- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e84153750 msgr2=0x7f0e841ec160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.547+0000 7f0e82a00640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e84153750 0x7f0e841ec160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.547+0000 7f0e82a00640 1 -- 172.21.15.67:0/841206201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8406dc00 msgr2=0x7f0e841efa20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:22:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.547+0000 7f0e82a00640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8406dc00 0x7f0e841efa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.547+0000 7f0e82a00640 1 -- 172.21.15.67:0/841206201 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84155260 con 0x7f0e84153370 2024-03-09T09:22:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.548+0000 7f0e82a00640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 0x7f0e8406fc70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000c340 tx=0x7f0e7000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.548+0000 7f0e7b400640 1 -- 172.21.15.67:0/841206201 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e700052d0 con 0x7f0e84153370 2024-03-09T09:22:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.548+0000 7f0e7b400640 1 -- 172.21.15.67:0/841206201 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e70005490 con 0x7f0e84153370 2024-03-09T09:22:32.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.548+0000 7f0e7b400640 1 -- 172.21.15.67:0/841206201 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70007260 con 0x7f0e84153370 2024-03-09T09:22:32.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.548+0000 7f0e8a800640 1 -- 172.21.15.67:0/841206201 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e84155540 con 0x7f0e84153370 2024-03-09T09:22:32.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.548+0000 7f0e8a800640 1 -- 172.21.15.67:0/841206201 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e841ebc40 con 0x7f0e84153370 2024-03-09T09:22:32.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.549+0000 7f0e8a800640 1 -- 172.21.15.67:0/841206201 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e841ebe40 con 0x7f0e84153370 2024-03-09T09:22:32.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.551+0000 7f0e7b400640 1 -- 172.21.15.67:0/841206201 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0e70007400 con 0x7f0e84153370 2024-03-09T09:22:32.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.552+0000 7f0e7b400640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e6406b4d0 0x7f0e6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:32.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.552+0000 7f0e7b400640 1 -- 172.21.15.67:0/841206201 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f0e70089e90 con 0x7f0e84153370 2024-03-09T09:22:32.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.552+0000 7f0e83400640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e6406b4d0 0x7f0e6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:32.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.554+0000 7f0e83400640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e6406b4d0 0x7f0e6406d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009630 tx=0x7f0e74008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:32.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.564+0000 7f0e7b400640 1 -- 172.21.15.67:0/841206201 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0e841ebe40 con 0x7f0e84153370 2024-03-09T09:22:32.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.863+0000 7f0e8a800640 1 -- 172.21.15.67:0/841206201 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0e84075a00 con 0x7f0e84153370 2024-03-09T09:22:32.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.864+0000 7f0e7b400640 1 -- 172.21.15.67:0/841206201 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f0e70058340 con 0x7f0e84153370 2024-03-09T09:22:32.871 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":228,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":27,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:22:32.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 -- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e6406b4d0 msgr2=0x7f0e6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:32.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e6406b4d0 0x7f0e6406d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009630 tx=0x7f0e74008030 comp rx=0 tx=0).stop 2024-03-09T09:22:32.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 -- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 msgr2=0x7f0e8406fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:32.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 0x7f0e8406fc70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000c340 tx=0x7f0e7000c800 comp rx=0 tx=0).stop 2024-03-09T09:22:32.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 -- 172.21.15.67:0/841206201 shutdown_connections 2024-03-09T09:22:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0e6406b4d0 0x7f0e6406d990 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e84153750 0x7f0e841ec160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e84153370 0x7f0e8406fc70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 --2- 172.21.15.67:0/841206201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e8406dc00 0x7f0e841efa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 -- 172.21.15.67:0/841206201 >> 172.21.15.67:0/841206201 conn(0x7f0e840ab320 msgr2=0x7f0e8406cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 -- 172.21.15.67:0/841206201 shutdown_connections 2024-03-09T09:22:32.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:32.867+0000 7f0e8a800640 1 -- 172.21.15.67:0/841206201 wait complete. 2024-03-09T09:22:34.052 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:22:34.052 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:22:34.052 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 70 -eq 48 ']' 2024-03-09T09:22:34.052 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.70 2024-03-09T09:22:34.052 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.70 2024-03-09T09:22:34.053 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.70 2024-03-09T09:22:34.053 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.70 2024-03-09T09:22:34.832 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:22:35.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:22:35.964 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:22:35.964 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:22:35.966 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 64 -eq 48 ']' 2024-03-09T09:22:35.966 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.64 2024-03-09T09:22:35.967 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.64 2024-03-09T09:22:35.967 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.64 2024-03-09T09:22:35.967 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.64 2024-03-09T09:22:35.972 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:22:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.111+0000 7fd8bdc00640 1 Processor -- start 2024-03-09T09:22:36.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.111+0000 7fd8bdc00640 1 -- start start 2024-03-09T09:22:36.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.113+0000 7fd8bdc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:36.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.113+0000 7fd8bdc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:36.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.113+0000 7fd8bdc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b80c28e0 con 0x7fd8b80c7290 2024-03-09T09:22:36.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.113+0000 7fd8bdc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b806e7e0 con 0x7fd8b814ca20 2024-03-09T09:22:36.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.113+0000 7fd8bdc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b814ce00 con 0x7fd8b80c58e0 2024-03-09T09:22:36.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8b7400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:36.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8b6a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:36.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8ae000640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd8b80c7290 0x7fd8b80c21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:41742/0 (socket says 172.21.15.67:41742) 2024-03-09T09:22:36.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8ae000640 1 -- 172.21.15.67:0/581412238 learned_addr learned my addr 172.21.15.67:0/581412238 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:22:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8b6a00640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 msgr2=0x7fd8b806ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8b6a00640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 msgr2=0x7fd8b806ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:36.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 msgr2=0x7fd8b8156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:22:36.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 msgr2=0x7fd8b8156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:36.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:36.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.114+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:36.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.315+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:36.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.315+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:36.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.315+0000 7fd8b6a00640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 msgr2=0x7fd8b806ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:22:36.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.315+0000 7fd8b6a00640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 msgr2=0x7fd8b806ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:36.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.315+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:36.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.316+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:22:36.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.316+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 msgr2=0x7fd8b8156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:36.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.316+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 msgr2=0x7fd8b8156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:36.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.316+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:36.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.316+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:22:36.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.717+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.718+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.718+0000 7fd8b6a00640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 msgr2=0x7fd8b806ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.718+0000 7fd8b6a00640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 msgr2=0x7fd8b806ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:36.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.718+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:36.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.718+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:22:36.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.718+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 msgr2=0x7fd8b8156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:22:36.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.718+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 msgr2=0x7fd8b8156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:36.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.718+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:36.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:36.718+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:22:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:37.520+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:37.520+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:37.520+0000 7fd8b6a00640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 msgr2=0x7fd8b806ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:37.520+0000 7fd8b6a00640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 msgr2=0x7fd8b806ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:37.520+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:37.520+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 msgr2=0x7fd8b8156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:22:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:37.520+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 msgr2=0x7fd8b8156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:37.520+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:37.520+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:22:37.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:37.521+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:22:38.971 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:22:38.972 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:22:38.972 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 71 -eq 48 ']' 2024-03-09T09:22:38.972 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.71 2024-03-09T09:22:38.972 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.71 2024-03-09T09:22:38.972 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.71 2024-03-09T09:22:38.972 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.71 2024-03-09T09:22:38.976 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:22:39.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.113+0000 7fd8b4c00640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 msgr2=0x7fd8b8156000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.113+0000 7fd8b4c00640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8b80c58e0 0x7fd8b8156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.113+0000 7fd8b4c00640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 msgr2=0x7fd8b806ec70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.113+0000 7fd8b4c00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.67:3300/0 conn(0x7fd8b814ca20 0x7fd8b806ec70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.113+0000 7fd8b4c00640 1 -- 172.21.15.67:0/581412238 >> v1:172.21.15.134:6789/0 conn(0x7fd8b80c7290 legacy=0x7fd8b80c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.114+0000 7fd8b4c00640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.114+0000 7fd8b4c00640 1 -- 172.21.15.67:0/581412238 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b8156db0 con 0x7fd8940036a0 2024-03-09T09:22:39.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.114+0000 7fd8b4c00640 1 -- 172.21.15.67:0/581412238 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b814ce00 con 0x7fd894007350 2024-03-09T09:22:39.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.115+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:39.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.115+0000 7fd8ae000640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 msgr2=0x7fd8940097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:39.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.115+0000 7fd8ae000640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 msgr2=0x7fd8940097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:39.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.115+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:39.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.115+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:39.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.316+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:39.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.317+0000 7fd8ae000640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 msgr2=0x7fd8940097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:39.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.317+0000 7fd8ae000640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 msgr2=0x7fd8940097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:39.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.317+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:39.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.317+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:22:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.719+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.719+0000 7fd8ae000640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 msgr2=0x7fd8940097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.719+0000 7fd8ae000640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 msgr2=0x7fd8940097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.719+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:39.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:39.719+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:22:40.442 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:22:40.443 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:22:40.443 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 65 -eq 48 ']' 2024-03-09T09:22:40.443 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.65 2024-03-09T09:22:40.443 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.65 2024-03-09T09:22:40.444 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.65 2024-03-09T09:22:40.444 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.65 2024-03-09T09:22:40.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:22:40.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:40.521+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:40.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:40.521+0000 7fd8ae000640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 msgr2=0x7fd8940097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:22:40.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:40.521+0000 7fd8ae000640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 msgr2=0x7fd8940097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:22:40.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:40.521+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:22:40.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:40.522+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:22:41.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.598+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446363436 0 0) 0x7fd8b8156db0 con 0x7fd8940036a0 2024-03-09T09:22:41.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.598+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd89c003520 con 0x7fd8940036a0 2024-03-09T09:22:41.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.599+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1009018279 0 0) 0x7fd89c003520 con 0x7fd8940036a0 2024-03-09T09:22:41.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.599+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 msgr2=0x7fd8940097f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:41.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.599+0000 7fd8b6000640 1 --2- 172.21.15.67:0/581412238 >> v2:172.21.15.134:3300/0 conn(0x7fd894007350 0x7fd8940097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.599+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89400aef0 con 0x7fd8940036a0 2024-03-09T09:22:41.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.599+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fd8a800d580 con 0x7fd8940036a0 2024-03-09T09:22:41.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.599+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 >> v1:172.21.15.67:6789/0 conn(0x7fd8940036a0 legacy=0x7fd894005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:41.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.599+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 reap_dead start 2024-03-09T09:22:41.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.599+0000 7fd8b6000640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c003a00 0x7fd89c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:41.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.600+0000 7fd8b6000640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd89c004800 0x7fd89c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:41.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.600+0000 7fd8b6000640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c0053c0 0x7fd89c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:41.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.600+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b814ce00 con 0x7fd89c003a00 2024-03-09T09:22:41.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.600+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c00c280 con 0x7fd89c0053c0 2024-03-09T09:22:41.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.600+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c00c480 con 0x7fd89c004800 2024-03-09T09:22:41.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.600+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c0053c0 0x7fd89c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:41.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.600+0000 7fd8ae000640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd89c004800 0x7fd89c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:41.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.601+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c003a00 0x7fd89c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:41.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.601+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd89c004800 msgr2=0x7fd89c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:41.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.601+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd89c004800 0x7fd89c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.601+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c003a00 msgr2=0x7fd89c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.601+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c003a00 0x7fd89c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.602+0000 7fd8b7400640 1 -- 172.21.15.67:0/581412238 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c003520 con 0x7fd89c0053c0 2024-03-09T09:22:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.602+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c003a00 0x7fd89c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:22:41.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.605+0000 7fd8b7400640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c0053c0 0x7fd89c00b9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd8b806c010 tx=0x7fd89c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9fd3d0eb974cd3f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:41.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.606+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0032070 con 0x7fd89c0053c0 2024-03-09T09:22:41.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.607+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b002cc30 con 0x7fd89c0053c0 2024-03-09T09:22:41.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.607+0000 7fd8b6000640 1 -- 172.21.15.67:0/581412238 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00115d0 con 0x7fd89c0053c0 2024-03-09T09:22:41.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.608+0000 7fd8bdc00640 1 -- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c0053c0 msgr2=0x7fd89c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:41.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.608+0000 7fd8bdc00640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c0053c0 0x7fd89c00b9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd8b806c010 tx=0x7fd89c005830 comp rx=0 tx=0).stop 2024-03-09T09:22:41.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.608+0000 7fd8bdc00640 1 -- 172.21.15.67:0/581412238 shutdown_connections 2024-03-09T09:22:41.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.608+0000 7fd8bdc00640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c0053c0 0x7fd89c00b9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.608+0000 7fd8bdc00640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd89c004800 0x7fd89c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.608+0000 7fd8bdc00640 1 --2- 172.21.15.67:0/581412238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c003a00 0x7fd89c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.608+0000 7fd8bdc00640 1 -- 172.21.15.67:0/581412238 >> 172.21.15.67:0/581412238 conn(0x7fd8b80a9580 msgr2=0x7fd8b80a9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:41.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.608+0000 7fd8bdc00640 1 -- 172.21.15.67:0/581412238 shutdown_connections 2024-03-09T09:22:41.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.608+0000 7fd8bdc00640 1 -- 172.21.15.67:0/581412238 wait complete. 2024-03-09T09:22:41.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.611+0000 7fd8bdc00640 1 Processor -- start 2024-03-09T09:22:41.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.611+0000 7fd8bdc00640 1 -- start start 2024-03-09T09:22:41.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.613+0000 7fd8bdc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c003a00 0x7fd8b8154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:41.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.613+0000 7fd8bdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c004800 0x7fd8b813ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:41.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.613+0000 7fd8bdc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd89c0053c0 0x7fd8b813b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:41.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.613+0000 7fd8bdc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b806e470 con 0x7fd89c004800 2024-03-09T09:22:41.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.613+0000 7fd8bdc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b806e190 con 0x7fd89c003a00 2024-03-09T09:22:41.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.613+0000 7fd8bdc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b806ec70 con 0x7fd89c0053c0 2024-03-09T09:22:41.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.613+0000 7fd8ae000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c004800 0x7fd8b813ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:41.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.613+0000 7fd8ae000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c004800 0x7fd8b813ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57826/0 (socket says 172.21.15.67:57826) 2024-03-09T09:22:41.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.613+0000 7fd8ae000640 1 -- 172.21.15.67:0/4007359367 learned_addr learned my addr 172.21.15.67:0/4007359367 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:41.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.613+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c003a00 0x7fd8b8154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:41.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.613+0000 7fd8b7400640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd89c0053c0 0x7fd8b813b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:41.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.614+0000 7fd8ae000640 1 -- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd89c0053c0 msgr2=0x7fd8b813b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:41.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.614+0000 7fd8ae000640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd89c0053c0 0x7fd8b813b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.614+0000 7fd8ae000640 1 -- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c003a00 msgr2=0x7fd8b8154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:41.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.614+0000 7fd8ae000640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c003a00 0x7fd8b8154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.614+0000 7fd8ae000640 1 -- 172.21.15.67:0/4007359367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b8144590 con 0x7fd89c004800 2024-03-09T09:22:41.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.614+0000 7fd8ae000640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c004800 0x7fd8b813ab70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0017e00 tx=0x7fd8a0017ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:41.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.614+0000 7fd8af400640 1 -- 172.21.15.67:0/4007359367 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0004e30 con 0x7fd89c004800 2024-03-09T09:22:41.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.615+0000 7fd8af400640 1 -- 172.21.15.67:0/4007359367 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a001c030 con 0x7fd89c004800 2024-03-09T09:22:41.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.615+0000 7fd8bdc00640 1 -- 172.21.15.67:0/4007359367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b8144870 con 0x7fd89c004800 2024-03-09T09:22:41.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.615+0000 7fd8af400640 1 -- 172.21.15.67:0/4007359367 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0007990 con 0x7fd89c004800 2024-03-09T09:22:41.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.615+0000 7fd8bdc00640 1 -- 172.21.15.67:0/4007359367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b81439e0 con 0x7fd89c004800 2024-03-09T09:22:41.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.616+0000 7fd8bdc00640 1 -- 172.21.15.67:0/4007359367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b8075a00 con 0x7fd89c004800 2024-03-09T09:22:41.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.618+0000 7fd8af400640 1 -- 172.21.15.67:0/4007359367 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd8a002b480 con 0x7fd89c004800 2024-03-09T09:22:41.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.619+0000 7fd8af400640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd89406bce0 0x7fd89406e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:41.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.619+0000 7fd8af400640 1 -- 172.21.15.67:0/4007359367 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd8a008bd20 con 0x7fd89c004800 2024-03-09T09:22:41.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.620+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd89406bce0 0x7fd89406e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:41.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.621+0000 7fd8b6a00640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd89406bce0 0x7fd89406e1a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80a8d90 tx=0x7fd8a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:41.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.625+0000 7fd8af400640 1 -- 172.21.15.67:0/4007359367 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd8a005a1d0 con 0x7fd89c004800 2024-03-09T09:22:41.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.935+0000 7fd8bdc00640 1 -- 172.21.15.67:0/4007359367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd8b80c1840 con 0x7fd89c004800 2024-03-09T09:22:41.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.936+0000 7fd8af400640 1 -- 172.21.15.67:0/4007359367 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fd8a005e270 con 0x7fd89c004800 2024-03-09T09:22:41.943 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":236,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:22:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.939+0000 7fd8bdc00640 1 -- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd89406bce0 msgr2=0x7fd89406e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.939+0000 7fd8bdc00640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd89406bce0 0x7fd89406e1a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80a8d90 tx=0x7fd8a8008040 comp rx=0 tx=0).stop 2024-03-09T09:22:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.940+0000 7fd8bdc00640 1 -- 172.21.15.67:0/4007359367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c004800 msgr2=0x7fd8b813ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:41.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.940+0000 7fd8bdc00640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c004800 0x7fd8b813ab70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0017e00 tx=0x7fd8a0017ef0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.940+0000 7fd8bdc00640 1 -- 172.21.15.67:0/4007359367 shutdown_connections 2024-03-09T09:22:41.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.940+0000 7fd8bdc00640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd89406bce0 0x7fd89406e1a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.940+0000 7fd8bdc00640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd89c0053c0 0x7fd8b813b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.940+0000 7fd8bdc00640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd89c004800 0x7fd8b813ab70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.940+0000 7fd8bdc00640 1 --2- 172.21.15.67:0/4007359367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd89c003a00 0x7fd8b8154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:41.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.940+0000 7fd8bdc00640 1 -- 172.21.15.67:0/4007359367 >> 172.21.15.67:0/4007359367 conn(0x7fd8b80a9580 msgr2=0x7fd8b80c20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:41.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.940+0000 7fd8bdc00640 1 -- 172.21.15.67:0/4007359367 shutdown_connections 2024-03-09T09:22:41.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:41.940+0000 7fd8bdc00640 1 -- 172.21.15.67:0/4007359367 wait complete. 2024-03-09T09:22:41.968 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:22:41.968 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:22:41.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:22:42.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.200+0000 7f8efa200640 1 Processor -- start 2024-03-09T09:22:42.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.200+0000 7f8efa200640 1 -- start start 2024-03-09T09:22:42.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.202+0000 7f8efa200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8ef414fd90 0x7f8ef4156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.202+0000 7f8efa200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8ef40c9380 0x7f8ef40acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.202+0000 7f8efa200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef40ad440 0x7f8ef406c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.202+0000 7f8efa200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef406ba60 con 0x7f8ef40c9380 2024-03-09T09:22:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.202+0000 7f8efa200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef4156bc0 con 0x7f8ef414fd90 2024-03-09T09:22:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.202+0000 7f8efa200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40c9890 con 0x7f8ef40ad440 2024-03-09T09:22:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.202+0000 7f8ef3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8ef40c9380 0x7f8ef40acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.202+0000 7f8ef3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8ef40c9380 0x7f8ef40acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57838/0 (socket says 172.21.15.67:57838) 2024-03-09T09:22:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.202+0000 7f8ef3e00640 1 -- 172.21.15.67:0/3393052891 learned_addr learned my addr 172.21.15.67:0/3393052891 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.203+0000 7f8ef8e00640 1 --2- 172.21.15.67:0/3393052891 >> v2:172.21.15.134:3300/0 conn(0x7f8ef414fd90 0x7f8ef4156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.203+0000 7f8ef9800640 1 --2- 172.21.15.67:0/3393052891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef40ad440 0x7f8ef406c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.203+0000 7f8ef3e00640 1 -- 172.21.15.67:0/3393052891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef40ad440 msgr2=0x7f8ef406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:42.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.203+0000 7f8ef3e00640 1 --2- 172.21.15.67:0/3393052891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef40ad440 0x7f8ef406c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.203+0000 7f8ef3e00640 1 -- 172.21.15.67:0/3393052891 >> v2:172.21.15.134:3300/0 conn(0x7f8ef414fd90 msgr2=0x7f8ef4156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:42.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.203+0000 7f8ef3e00640 1 --2- 172.21.15.67:0/3393052891 >> v2:172.21.15.134:3300/0 conn(0x7f8ef414fd90 0x7f8ef4156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.203+0000 7f8ef3e00640 1 -- 172.21.15.67:0/3393052891 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40c5f80 con 0x7f8ef40c9380 2024-03-09T09:22:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.204+0000 7f8ef3e00640 1 --2- 172.21.15.67:0/3393052891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef40c9380 0x7f8ef40acf00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40ada40 tx=0x7f8ee002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97373b04f9769f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.204+0000 7f8ef3400640 1 -- 172.21.15.67:0/3393052891 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0038070 con 0x7f8ef40c9380 2024-03-09T09:22:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.205+0000 7f8ef3400640 1 -- 172.21.15.67:0/3393052891 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee002f050 con 0x7f8ef40c9380 2024-03-09T09:22:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.205+0000 7f8ef3400640 1 -- 172.21.15.67:0/3393052891 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0031970 con 0x7f8ef40c9380 2024-03-09T09:22:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.205+0000 7f8efa200640 1 -- 172.21.15.67:0/3393052891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef40c9380 msgr2=0x7f8ef40acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.205+0000 7f8efa200640 1 --2- 172.21.15.67:0/3393052891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef40c9380 0x7f8ef40acf00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40ada40 tx=0x7f8ee002b7f0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.205+0000 7f8efa200640 1 -- 172.21.15.67:0/3393052891 shutdown_connections 2024-03-09T09:22:42.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.205+0000 7f8efa200640 1 --2- 172.21.15.67:0/3393052891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef40ad440 0x7f8ef406c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.205+0000 7f8efa200640 1 --2- 172.21.15.67:0/3393052891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef40c9380 0x7f8ef40acf00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.205+0000 7f8efa200640 1 --2- 172.21.15.67:0/3393052891 >> v2:172.21.15.134:3300/0 conn(0x7f8ef414fd90 0x7f8ef4156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.205+0000 7f8efa200640 1 -- 172.21.15.67:0/3393052891 >> 172.21.15.67:0/3393052891 conn(0x7f8ef40cb480 msgr2=0x7f8ef40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:42.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.205+0000 7f8efa200640 1 -- 172.21.15.67:0/3393052891 shutdown_connections 2024-03-09T09:22:42.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.205+0000 7f8efa200640 1 -- 172.21.15.67:0/3393052891 wait complete. 2024-03-09T09:22:42.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.207+0000 7f8efa200640 1 Processor -- start 2024-03-09T09:22:42.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.207+0000 7f8efa200640 1 -- start start 2024-03-09T09:22:42.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.208+0000 7f8efa200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef40ad440 0x7f8ef40c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.208+0000 7f8efa200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef40c9380 0x7f8ef41eef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.208+0000 7f8efa200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414fd90 0x7f8ef40af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.208+0000 7f8efa200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef406bcb0 con 0x7f8ef414fd90 2024-03-09T09:22:42.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.208+0000 7f8efa200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40c6520 con 0x7f8ef40ad440 2024-03-09T09:22:42.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.208+0000 7f8efa200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef406c710 con 0x7f8ef40c9380 2024-03-09T09:22:42.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.209+0000 7f8ef9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414fd90 0x7f8ef40af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.209+0000 7f8ef9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414fd90 0x7f8ef40af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57850/0 (socket says 172.21.15.67:57850) 2024-03-09T09:22:42.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.209+0000 7f8ef9800640 1 -- 172.21.15.67:0/2899896320 learned_addr learned my addr 172.21.15.67:0/2899896320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:42.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.209+0000 7f8ef8e00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef40ad440 0x7f8ef40c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.209+0000 7f8ef3e00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef40c9380 0x7f8ef41eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.209+0000 7f8ef9800640 1 -- 172.21.15.67:0/2899896320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef40c9380 msgr2=0x7f8ef41eef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.209+0000 7f8ef9800640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef40c9380 0x7f8ef41eef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.209+0000 7f8ef9800640 1 -- 172.21.15.67:0/2899896320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef40ad440 msgr2=0x7f8ef40c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.209+0000 7f8ef9800640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef40ad440 0x7f8ef40c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.209+0000 7f8ef9800640 1 -- 172.21.15.67:0/2899896320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40aeb20 con 0x7f8ef414fd90 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.210+0000 7f8ef9800640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414fd90 0x7f8ef40af2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8eec007e90 tx=0x7f8eec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.210+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec014420 con 0x7f8ef414fd90 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.210+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec019070 con 0x7f8ef414fd90 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.210+0000 7f8efa200640 1 -- 172.21.15.67:0/2899896320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef4153590 con 0x7f8ef414fd90 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.210+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec014820 con 0x7f8ef414fd90 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.210+0000 7f8efa200640 1 -- 172.21.15.67:0/2899896320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef41efbb0 con 0x7f8ef414fd90 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.211+0000 7f8efa200640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef40af820 0x7f8ef414c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.211+0000 7f8efa200640 1 -- 172.21.15.67:0/2899896320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef404f3c0 con 0x7f8ef40af820 2024-03-09T09:22:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.212+0000 7f8ef8e00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef40af820 0x7f8ef414c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.212+0000 7f8ef8e00640 1 -- 172.21.15.67:0/2899896320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ef406d2d0 con 0x7f8ef40af820 2024-03-09T09:22:42.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.212+0000 7f8ef8e00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef40af820 0x7f8ef414c480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800f440 tx=0x7f8ee800f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.213+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8019030 con 0x7f8ef40af820 2024-03-09T09:22:42.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.213+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f8ee8004020 con 0x7f8ef40af820 2024-03-09T09:22:42.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.213+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef40af820 msgr2=0x7f8ef414c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.213+0000 7f8ef0c00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef40af820 0x7f8ef414c480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800f440 tx=0x7f8ee800f900 comp rx=0 tx=0).stop 2024-03-09T09:22:42.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.213+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef40af820 msgr2=0x7f8ef414c480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:42.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.213+0000 7f8ef0c00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef40af820 0x7f8ef414c480 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800f440 tx=0x7f8ee800f900 comp rx=0 tx=0).stop 2024-03-09T09:22:42.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.213+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8eec0149c0 con 0x7f8ef414fd90 2024-03-09T09:22:42.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.215+0000 7f8ef0c00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ed406afe0 0x7f8ed406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.215+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f8eec08ae30 con 0x7f8ef414fd90 2024-03-09T09:22:42.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.215+0000 7f8ef8e00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ed406afe0 0x7f8ed406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.217+0000 7f8ef8e00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ed406afe0 0x7f8ed406d4a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8ee801f450 tx=0x7f8ee8014000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.222+0000 7f8efa200640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414c9c0 0x7f8ef41eb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.222+0000 7f8efa200640 1 -- 172.21.15.67:0/2899896320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4156980 con 0x7f8ef414c9c0 2024-03-09T09:22:42.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.222+0000 7f8ef3e00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414c9c0 0x7f8ef41eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.222+0000 7f8ef3e00640 1 -- 172.21.15.67:0/2899896320 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8ef40c6f40 con 0x7f8ef414c9c0 2024-03-09T09:22:42.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.223+0000 7f8ef3e00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414c9c0 0x7f8ef41eb750 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40ae040 tx=0x7f8ee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.223+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0038040 con 0x7f8ef414c9c0 2024-03-09T09:22:42.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.223+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4536 (secure 0 0 0) 0x7f8ee0009490 con 0x7f8ef414c9c0 2024-03-09T09:22:42.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.223+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414c9c0 msgr2=0x7f8ef41eb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.223+0000 7f8ef0c00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414c9c0 0x7f8ef41eb750 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40ae040 tx=0x7f8ee00046c0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.223+0000 7f8ef0c00640 1 -- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414c9c0 msgr2=0x7f8ef41eb750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:42.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.223+0000 7f8ef0c00640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414c9c0 0x7f8ef41eb750 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40ae040 tx=0x7f8ee00046c0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.231 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:22:42.231 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:22:42.231 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:22:42.232 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:22:42.232 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 236, 2024-03-09T09:22:42.232 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:22:42.232 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:22:42.232 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:22:42.232 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:22:42.232 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.232 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:22:42.232 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:42.232 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:22:42.232 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:22:42.233 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:22:42.235 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:42.236 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:42.237 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:42.237 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.237 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:22:42.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.237 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:22:42.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.237 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:22:42.237 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:22:42.237 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:42.237 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:42.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.238 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:42.238 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:22:42.238 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.238 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:42.238 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:22:42.238 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.238 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.238 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.239 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:22:42.239 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:22:42.239 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:42.239 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:42.239 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:42.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.239 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:22:42.239 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:22:42.239 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:42.239 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:22:42.240 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.242 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.243 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:42.244 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.244 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.244 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:22:42.244 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:22:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.224+0000 7f8efa200640 1 -- 172.21.15.67:0/2899896320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ed406afe0 msgr2=0x7f8ed406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.224+0000 7f8efa200640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8ed406afe0 0x7f8ed406d4a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8ee801f450 tx=0x7f8ee8014000 comp rx=0 tx=0).stop 2024-03-09T09:22:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.224+0000 7f8efa200640 1 -- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414fd90 msgr2=0x7f8ef40af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.224+0000 7f8efa200640 1 --2- 172.21.15.67:0/2899896320 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef414fd90 0x7f8ef40af2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8eec007e90 tx=0x7f8eec00e4b0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.224+0000 7f8ef9800640 1 -- 172.21.15.67:0/2899896320 reap_dead start 2024-03-09T09:22:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.224+0000 7f8ef9800640 1 -- 172.21.15.67:0/2899896320 reap_dead start 2024-03-09T09:22:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.224+0000 7f8efa200640 1 -- 172.21.15.67:0/2899896320 shutdown_connections 2024-03-09T09:22:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.224+0000 7f8efa200640 1 -- 172.21.15.67:0/2899896320 >> 172.21.15.67:0/2899896320 conn(0x7f8ef40cb480 msgr2=0x7f8ef4153e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.224+0000 7f8efa200640 1 -- 172.21.15.67:0/2899896320 shutdown_connections 2024-03-09T09:22:42.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.224+0000 7f8efa200640 1 -- 172.21.15.67:0/2899896320 wait complete. 2024-03-09T09:22:42.247 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:22:42.247 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:22:42.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:22:42.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.443+0000 7fbf0f200640 1 Processor -- start 2024-03-09T09:22:42.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.443+0000 7fbf0f200640 1 -- start start 2024-03-09T09:22:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.444+0000 7fbf0f200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbf080acf00 0x7fbf080c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.444+0000 7fbf0f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbf080c9380 0x7fbf08156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.445+0000 7fbf0f200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf080c97d0 con 0x7fbf080c5e00 2024-03-09T09:22:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.445+0000 7fbf0f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0806d740 con 0x7fbf080c9380 2024-03-09T09:22:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.445+0000 7fbf0f200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf08153680 con 0x7fbf080acf00 2024-03-09T09:22:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.445+0000 7fbf0e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbf080c9380 0x7fbf08156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.445+0000 7fbf0e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbf080c9380 0x7fbf08156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57886/0 (socket says 172.21.15.67:57886) 2024-03-09T09:22:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.445+0000 7fbf0e800640 1 -- 172.21.15.67:0/1411829867 learned_addr learned my addr 172.21.15.67:0/1411829867 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.445+0000 7fbf0d400640 1 --2- 172.21.15.67:0/1411829867 >> v2:172.21.15.134:3300/0 conn(0x7fbf080acf00 0x7fbf080c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.445+0000 7fbf0e800640 1 -- 172.21.15.67:0/1411829867 >> v2:172.21.15.134:3300/0 conn(0x7fbf080acf00 msgr2=0x7fbf080c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.445+0000 7fbf0e800640 1 --2- 172.21.15.67:0/1411829867 >> v2:172.21.15.134:3300/0 conn(0x7fbf080acf00 0x7fbf080c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.445+0000 7fbf0e800640 1 -- 172.21.15.67:0/1411829867 >> v1:172.21.15.134:6789/0 conn(0x7fbf080c5e00 legacy=0x7fbf0806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.445+0000 7fbf0e800640 1 -- 172.21.15.67:0/1411829867 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf080ad7c0 con 0x7fbf080c9380 2024-03-09T09:22:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.447+0000 7fbf0e800640 1 --2- 172.21.15.67:0/1411829867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf080c9380 0x7fbf08156660 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009e80 tx=0x7fbf0402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23f8d078b8c5eb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.447+0000 7fbf0ca00640 1 -- 172.21.15.67:0/1411829867 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0402c9e0 con 0x7fbf080c9380 2024-03-09T09:22:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.447+0000 7fbf0ca00640 1 -- 172.21.15.67:0/1411829867 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0402cb80 con 0x7fbf080c9380 2024-03-09T09:22:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.447+0000 7fbf0ca00640 1 -- 172.21.15.67:0/1411829867 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf040343f0 con 0x7fbf080c9380 2024-03-09T09:22:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.448+0000 7fbf0f200640 1 -- 172.21.15.67:0/1411829867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf080c9380 msgr2=0x7fbf08156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.448+0000 7fbf0f200640 1 --2- 172.21.15.67:0/1411829867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf080c9380 0x7fbf08156660 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009e80 tx=0x7fbf0402be40 comp rx=0 tx=0).stop 2024-03-09T09:22:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.448+0000 7fbf0f200640 1 -- 172.21.15.67:0/1411829867 shutdown_connections 2024-03-09T09:22:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.448+0000 7fbf0f200640 1 --2- 172.21.15.67:0/1411829867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf080c9380 0x7fbf08156660 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.448+0000 7fbf0f200640 1 --2- 172.21.15.67:0/1411829867 >> v2:172.21.15.134:3300/0 conn(0x7fbf080acf00 0x7fbf080c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.448+0000 7fbf0f200640 1 -- 172.21.15.67:0/1411829867 >> 172.21.15.67:0/1411829867 conn(0x7fbf080cb480 msgr2=0x7fbf080cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.448+0000 7fbf0f200640 1 -- 172.21.15.67:0/1411829867 shutdown_connections 2024-03-09T09:22:42.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.448+0000 7fbf0f200640 1 -- 172.21.15.67:0/1411829867 wait complete. 2024-03-09T09:22:42.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.450+0000 7fbf0f200640 1 Processor -- start 2024-03-09T09:22:42.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.450+0000 7fbf0f200640 1 -- start start 2024-03-09T09:22:42.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.451+0000 7fbf0f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf080acf00 0x7fbf081eb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.451+0000 7fbf0f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf080c5e00 0x7fbf080c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.451+0000 7fbf0f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf080c9380 0x7fbf081503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.451+0000 7fbf0f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0806bde0 con 0x7fbf080c5e00 2024-03-09T09:22:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.451+0000 7fbf0f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0806bb00 con 0x7fbf080c9380 2024-03-09T09:22:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.451+0000 7fbf0f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0806c0c0 con 0x7fbf080acf00 2024-03-09T09:22:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.451+0000 7fbf0d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf080c5e00 0x7fbf080c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.451+0000 7fbf0d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf080c5e00 0x7fbf080c1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57902/0 (socket says 172.21.15.67:57902) 2024-03-09T09:22:42.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.451+0000 7fbf0d400640 1 -- 172.21.15.67:0/4258573232 learned_addr learned my addr 172.21.15.67:0/4258573232 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:42.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.452+0000 7fbf0de00640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf080acf00 0x7fbf081eb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.452+0000 7fbf0e800640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf080c9380 0x7fbf081503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.452+0000 7fbf0d400640 1 -- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf080acf00 msgr2=0x7fbf081eb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.452+0000 7fbf0d400640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf080acf00 0x7fbf081eb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.452+0000 7fbf0d400640 1 -- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf080c9380 msgr2=0x7fbf081503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.452+0000 7fbf0d400640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf080c9380 0x7fbf081503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.452+0000 7fbf0d400640 1 -- 172.21.15.67:0/4258573232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf081ef8f0 con 0x7fbf080c5e00 2024-03-09T09:22:42.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.452+0000 7fbf0d400640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf080c5e00 0x7fbf080c1d60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbef800d5f0 tx=0x7fbef800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.453+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80125f0 con 0x7fbf080c5e00 2024-03-09T09:22:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.453+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef8019070 con 0x7fbf080c5e00 2024-03-09T09:22:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.453+0000 7fbf0f200640 1 -- 172.21.15.67:0/4258573232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf081efbd0 con 0x7fbf080c5e00 2024-03-09T09:22:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.453+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80129d0 con 0x7fbf080c5e00 2024-03-09T09:22:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.453+0000 7fbf0f200640 1 -- 172.21.15.67:0/4258573232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf08153d90 con 0x7fbf080c5e00 2024-03-09T09:22:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.454+0000 7fbf0f200640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf08155800 0x7fbf080c0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.454+0000 7fbf0f200640 1 -- 172.21.15.67:0/4258573232 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf081508f0 con 0x7fbf08155800 2024-03-09T09:22:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.454+0000 7fbf0de00640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf08155800 0x7fbf080c0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.455+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbef8012b70 con 0x7fbf080c5e00 2024-03-09T09:22:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.455+0000 7fbf0de00640 1 -- 172.21.15.67:0/4258573232 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf0804f3c0 con 0x7fbf08155800 2024-03-09T09:22:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.456+0000 7fbf0de00640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf08155800 0x7fbf080c0060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00d460 tx=0x7fbefc00d920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.456+0000 7fbf02000640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbeec06b500 0x7fbeec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.456+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbef808a9b0 con 0x7fbf080c5e00 2024-03-09T09:22:42.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.456+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc018020 con 0x7fbf08155800 2024-03-09T09:22:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.456+0000 7fbf0e800640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbeec06b500 0x7fbeec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.457+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fbefc014620 con 0x7fbf08155800 2024-03-09T09:22:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.457+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf08155800 msgr2=0x7fbf080c0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.457+0000 7fbf02000640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf08155800 0x7fbf080c0060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00d460 tx=0x7fbefc00d920 comp rx=0 tx=0).stop 2024-03-09T09:22:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.457+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf08155800 msgr2=0x7fbf080c0060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.457+0000 7fbf02000640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf08155800 0x7fbf080c0060 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00d460 tx=0x7fbefc00d920 comp rx=0 tx=0).stop 2024-03-09T09:22:42.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.457+0000 7fbf0e800640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbeec06b500 0x7fbeec06d9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbf04035000 tx=0x7fbf04030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.465+0000 7fbf0f200640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf080c05a0 0x7fbf08159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.465+0000 7fbf0f200640 1 -- 172.21.15.67:0/4258573232 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0806d2f0 con 0x7fbf080c05a0 2024-03-09T09:22:42.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.465+0000 7fbf0de00640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf080c05a0 0x7fbf08159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.466+0000 7fbf0de00640 1 -- 172.21.15.67:0/4258573232 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbf080cb900 con 0x7fbf080c05a0 2024-03-09T09:22:42.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.467+0000 7fbf0de00640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf080c05a0 0x7fbf08159130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbefc019000 tx=0x7fbefc004680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.467+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc018030 con 0x7fbf080c05a0 2024-03-09T09:22:42.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.468+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7fbefc01fd80 con 0x7fbf080c05a0 2024-03-09T09:22:42.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.468+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf080c05a0 msgr2=0x7fbf08159130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.468+0000 7fbf02000640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf080c05a0 0x7fbf08159130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbefc019000 tx=0x7fbefc004680 comp rx=0 tx=0).stop 2024-03-09T09:22:42.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.468+0000 7fbf02000640 1 -- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf080c05a0 msgr2=0x7fbf08159130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:42.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.468+0000 7fbf02000640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf080c05a0 0x7fbf08159130 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbefc019000 tx=0x7fbefc004680 comp rx=0 tx=0).stop 2024-03-09T09:22:42.475 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:22:42.476 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:22:42.476 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:22:42.476 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:22:42.476 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 236, 2024-03-09T09:22:42.476 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:22:42.476 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:22:42.476 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:22:42.476 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:22:42.476 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:42.477 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:42.478 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:42.478 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:42.478 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:42.478 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.478 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:22:42.478 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:22:42.478 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:22:42.479 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:22:42.480 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:42.481 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:22:42.482 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.483 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:42.483 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:22:42.483 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.483 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.483 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.483 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:22:42.484 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:22:42.484 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:42.484 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:42.484 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:42.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.484 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:22:42.484 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:42.486 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:22:42.487 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.487 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:42.487 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:22:42.487 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.487 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.487 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.488 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:42.489 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.489 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.489 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:22:42.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.489 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.489 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.489 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:42.489 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.490 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.490 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:22:42.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.490 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.490 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.490 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:42.490 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.490 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.490 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:22:42.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.491 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:22:42.491 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.491 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:42.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.491 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.491 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.491 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:22:42.491 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.492 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.492 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:22:42.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.492 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.492 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.492 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:42.492 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.492 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.492 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:22:42.492 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:22:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.468+0000 7fbf0f200640 1 -- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbeec06b500 msgr2=0x7fbeec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.468+0000 7fbf0f200640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbeec06b500 0x7fbeec06d9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbf04035000 tx=0x7fbf04030000 comp rx=0 tx=0).stop 2024-03-09T09:22:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.468+0000 7fbf0f200640 1 -- 172.21.15.67:0/4258573232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf080c5e00 msgr2=0x7fbf080c1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.468+0000 7fbf0f200640 1 --2- 172.21.15.67:0/4258573232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf080c5e00 0x7fbf080c1d60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbef800d5f0 tx=0x7fbef800dab0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.468+0000 7fbf0e800640 1 -- 172.21.15.67:0/4258573232 reap_dead start 2024-03-09T09:22:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.469+0000 7fbf0e800640 1 -- 172.21.15.67:0/4258573232 reap_dead start 2024-03-09T09:22:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.469+0000 7fbf0f200640 1 -- 172.21.15.67:0/4258573232 shutdown_connections 2024-03-09T09:22:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.469+0000 7fbf0f200640 1 -- 172.21.15.67:0/4258573232 >> 172.21.15.67:0/4258573232 conn(0x7fbf080cb480 msgr2=0x7fbf081df240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.469+0000 7fbf0f200640 1 -- 172.21.15.67:0/4258573232 shutdown_connections 2024-03-09T09:22:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.469+0000 7fbf0f200640 1 -- 172.21.15.67:0/4258573232 wait complete. 2024-03-09T09:22:42.496 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:22:42.496 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:22:42.496 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:22:42.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.694+0000 7fa2f7e00640 1 Processor -- start 2024-03-09T09:22:42.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.695+0000 7fa2f7e00640 1 -- start start 2024-03-09T09:22:42.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.696+0000 7fa2f7e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa2f0153460 0x7fa2f0155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.696+0000 7fa2f7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f00c8c60 0x7fa2f00c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.696+0000 7fa2f7e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f006b040 con 0x7fa2f00c9d50 2024-03-09T09:22:42.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.696+0000 7fa2f7e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f00ca590 con 0x7fa2f0153460 2024-03-09T09:22:42.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.696+0000 7fa2f7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f0156710 con 0x7fa2f00c8c60 2024-03-09T09:22:42.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.697+0000 7fa2f4a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f00c8c60 0x7fa2f00c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.697+0000 7fa2f4a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f00c8c60 0x7fa2f00c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:50120/0 (socket says 172.21.15.67:50120) 2024-03-09T09:22:42.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.697+0000 7fa2f5400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa2f0153460 0x7fa2f0155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.697+0000 7fa2f4a00640 1 -- 172.21.15.67:0/2816667039 learned_addr learned my addr 172.21.15.67:0/2816667039 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:42.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.697+0000 7fa2f4a00640 1 -- 172.21.15.67:0/2816667039 >> v2:172.21.15.134:3300/0 conn(0x7fa2f0153460 msgr2=0x7fa2f0155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:42.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.697+0000 7fa2f4a00640 1 --2- 172.21.15.67:0/2816667039 >> v2:172.21.15.134:3300/0 conn(0x7fa2f0153460 0x7fa2f0155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.697+0000 7fa2f4a00640 1 -- 172.21.15.67:0/2816667039 >> v1:172.21.15.134:6789/0 conn(0x7fa2f00c9d50 legacy=0x7fa2f00caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.698+0000 7fa2f4a00640 1 -- 172.21.15.67:0/2816667039 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f01569f0 con 0x7fa2f00c8c60 2024-03-09T09:22:42.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.699+0000 7fa2f4a00640 1 --2- 172.21.15.67:0/2816667039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f00c8c60 0x7fa2f00c9810 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009870 tx=0x7fa2dc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef87244164a1c744 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.699+0000 7fa2efe00640 1 -- 172.21.15.67:0/2816667039 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc035020 con 0x7fa2f00c8c60 2024-03-09T09:22:42.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.699+0000 7fa2efe00640 1 -- 172.21.15.67:0/2816667039 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2dc031640 con 0x7fa2f00c8c60 2024-03-09T09:22:42.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.699+0000 7fa2efe00640 1 -- 172.21.15.67:0/2816667039 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc02d040 con 0x7fa2f00c8c60 2024-03-09T09:22:42.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.700+0000 7fa2f7e00640 1 -- 172.21.15.67:0/2816667039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f00c8c60 msgr2=0x7fa2f00c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.700+0000 7fa2f7e00640 1 --2- 172.21.15.67:0/2816667039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f00c8c60 0x7fa2f00c9810 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009870 tx=0x7fa2dc02b8f0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.700+0000 7fa2f7e00640 1 -- 172.21.15.67:0/2816667039 shutdown_connections 2024-03-09T09:22:42.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.700+0000 7fa2f7e00640 1 --2- 172.21.15.67:0/2816667039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f00c8c60 0x7fa2f00c9810 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.700+0000 7fa2f7e00640 1 --2- 172.21.15.67:0/2816667039 >> v2:172.21.15.134:3300/0 conn(0x7fa2f0153460 0x7fa2f0155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.700+0000 7fa2f7e00640 1 -- 172.21.15.67:0/2816667039 >> 172.21.15.67:0/2816667039 conn(0x7fa2f00c8170 msgr2=0x7fa2f00c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:42.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.700+0000 7fa2f7e00640 1 -- 172.21.15.67:0/2816667039 shutdown_connections 2024-03-09T09:22:42.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.700+0000 7fa2f7e00640 1 -- 172.21.15.67:0/2816667039 wait complete. 2024-03-09T09:22:42.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.703+0000 7fa2f7e00640 1 Processor -- start 2024-03-09T09:22:42.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.703+0000 7fa2f7e00640 1 -- start start 2024-03-09T09:22:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.705+0000 7fa2f7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00c8c60 0x7fa2f00c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.705+0000 7fa2f7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f00c9d50 0x7fa2f01ec900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.705+0000 7fa2f7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f0153460 0x7fa2f01ece40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.705+0000 7fa2f7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f0153840 con 0x7fa2f0153460 2024-03-09T09:22:42.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.705+0000 7fa2f7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f0156400 con 0x7fa2f00c8c60 2024-03-09T09:22:42.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.705+0000 7fa2f7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f00cb130 con 0x7fa2f00c9d50 2024-03-09T09:22:42.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.705+0000 7fa2f5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f0153460 0x7fa2f01ece40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.705+0000 7fa2f5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f0153460 0x7fa2f01ece40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57912/0 (socket says 172.21.15.67:57912) 2024-03-09T09:22:42.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.705+0000 7fa2f5e00640 1 -- 172.21.15.67:0/4265102867 learned_addr learned my addr 172.21.15.67:0/4265102867 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:42.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.705+0000 7fa2f5400640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00c8c60 0x7fa2f00c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.706+0000 7fa2f4a00640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f00c9d50 0x7fa2f01ec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.706+0000 7fa2f5e00640 1 -- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f00c9d50 msgr2=0x7fa2f01ec900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.706+0000 7fa2f5e00640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f00c9d50 0x7fa2f01ec900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.706+0000 7fa2f5e00640 1 -- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00c8c60 msgr2=0x7fa2f00c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.706+0000 7fa2f5e00640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00c8c60 0x7fa2f00c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.706+0000 7fa2f5e00640 1 -- 172.21.15.67:0/4265102867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f01576f0 con 0x7fa2f0153460 2024-03-09T09:22:42.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.706+0000 7fa2f5400640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2f00c8c60 0x7fa2f00c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:22:42.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.706+0000 7fa2f5e00640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f0153460 0x7fa2f01ece40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800c7a0 tx=0x7fa2e800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.707+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8003b70 con 0x7fa2f0153460 2024-03-09T09:22:42.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.707+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e8003d30 con 0x7fa2f0153460 2024-03-09T09:22:42.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.707+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e800aa90 con 0x7fa2f0153460 2024-03-09T09:22:42.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.707+0000 7fa2f7e00640 1 -- 172.21.15.67:0/4265102867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f00c32e0 con 0x7fa2f0153460 2024-03-09T09:22:42.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.707+0000 7fa2f7e00640 1 -- 172.21.15.67:0/4265102867 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f00c3860 con 0x7fa2f0153460 2024-03-09T09:22:42.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.708+0000 7fa2f7e00640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ed380 0x7fa2f00c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.708+0000 7fa2f7e00640 1 -- 172.21.15.67:0/4265102867 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f00752b0 con 0x7fa2f01ed380 2024-03-09T09:22:42.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.709+0000 7fa2f5400640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ed380 0x7fa2f00c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.709+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa2e800ac30 con 0x7fa2f0153460 2024-03-09T09:22:42.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.710+0000 7fa2f5400640 1 -- 172.21.15.67:0/4265102867 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2f0075a00 con 0x7fa2f01ed380 2024-03-09T09:22:42.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.710+0000 7fa2ed600640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2d006b4d0 0x7fa2d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.710+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa2e808a5d0 con 0x7fa2f0153460 2024-03-09T09:22:42.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.710+0000 7fa2f5400640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ed380 0x7fa2f00c3a30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40115d0 tx=0x7fa2e4011a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.711+0000 7fa2f4a00640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2d006b4d0 0x7fa2d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.711+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400f910 con 0x7fa2f01ed380 2024-03-09T09:22:42.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.712+0000 7fa2f4a00640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2d006b4d0 0x7fa2d006d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc036000 tx=0x7fa2dc03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.712+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa2e400f910 con 0x7fa2f01ed380 2024-03-09T09:22:42.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.712+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ed380 msgr2=0x7fa2f00c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.712+0000 7fa2ed600640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ed380 0x7fa2f00c3a30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40115d0 tx=0x7fa2e4011a90 comp rx=0 tx=0).stop 2024-03-09T09:22:42.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.712+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ed380 msgr2=0x7fa2f00c3a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:42.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.712+0000 7fa2ed600640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ed380 0x7fa2f00c3a30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.719+0000 7fa2f7e00640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ee9b0 0x7fa2f01eedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.719+0000 7fa2f7e00640 1 -- 172.21.15.67:0/4265102867 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f00c7520 con 0x7fa2f01ee9b0 2024-03-09T09:22:42.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.720+0000 7fa2f5400640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ee9b0 0x7fa2f01eedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.721+0000 7fa2f5400640 1 -- 172.21.15.67:0/4265102867 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa2f00bece0 con 0x7fa2f01ee9b0 2024-03-09T09:22:42.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.721+0000 7fa2f5400640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ee9b0 0x7fa2f01eedc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40115d0 tx=0x7fa2e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.722+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40067d0 con 0x7fa2f01ee9b0 2024-03-09T09:22:42.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.723+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7fa2e4026070 con 0x7fa2f01ee9b0 2024-03-09T09:22:42.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.723+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ee9b0 msgr2=0x7fa2f01eedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.723+0000 7fa2ed600640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ee9b0 0x7fa2f01eedc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40115d0 tx=0x7fa2e40046c0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.723+0000 7fa2ed600640 1 -- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ee9b0 msgr2=0x7fa2f01eedc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:42.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.723+0000 7fa2ed600640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa2f01ee9b0 0x7fa2f01eedc0 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40115d0 tx=0x7fa2e40046c0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.730 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:22:42.730 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:22:42.730 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:22:42.731 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:22:42.731 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 236, 2024-03-09T09:22:42.731 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:22:42.731 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:22:42.731 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:22:42.731 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:22:42.731 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.731 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:22:42.731 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:42.732 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:22:42.732 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:22:42.732 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:42.732 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:42.732 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:42.732 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:42.732 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:42.732 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:42.732 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:42.733 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:42.733 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:42.733 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:42.733 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:42.733 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.733 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:22:42.733 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:22:42.733 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:42.733 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:42.734 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:42.734 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:42.734 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:42.734 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:42.734 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:42.734 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:42.734 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:42.734 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:42.734 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:42.735 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.735 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:22:42.735 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:22:42.735 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:22:42.735 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:22:42.735 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:22:42.735 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:22:42.735 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:42.736 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:42.736 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:22:42.736 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:22:42.736 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:22:42.736 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:22:42.736 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:22:42.736 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:22:42.736 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:22:42.736 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:42.737 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:22:42.737 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:42.737 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:42.737 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:42.737 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:42.737 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:42.737 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:42.737 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:42.738 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:42.738 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:42.738 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:42.738 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:42.738 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.738 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:22:42.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.738 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:22:42.738 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.739 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:22:42.739 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:22:42.739 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:42.739 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:42.739 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:42.739 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:22:42.739 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.739 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.740 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:42.741 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.742 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:42.743 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:22:42.744 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.745 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:22:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:22:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:22:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.746 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:22:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:42.747 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:22:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:42.748 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:22:42.748 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:22:42.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.723+0000 7fa2f7e00640 1 -- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2d006b4d0 msgr2=0x7fa2d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.723+0000 7fa2f7e00640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2d006b4d0 0x7fa2d006d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc036000 tx=0x7fa2dc03c040 comp rx=0 tx=0).stop 2024-03-09T09:22:42.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.723+0000 7fa2f7e00640 1 -- 172.21.15.67:0/4265102867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f0153460 msgr2=0x7fa2f01ece40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.723+0000 7fa2f7e00640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f0153460 0x7fa2f01ece40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800c7a0 tx=0x7fa2e800cc60 comp rx=0 tx=0).stop 2024-03-09T09:22:42.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.723+0000 7fa2f5e00640 1 -- 172.21.15.67:0/4265102867 reap_dead start 2024-03-09T09:22:42.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.724+0000 7fa2f7e00640 1 -- 172.21.15.67:0/4265102867 shutdown_connections 2024-03-09T09:22:42.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.724+0000 7fa2f7e00640 1 --2- 172.21.15.67:0/4265102867 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2f0153460 0x7fa2f01ece40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.724+0000 7fa2f7e00640 1 -- 172.21.15.67:0/4265102867 >> 172.21.15.67:0/4265102867 conn(0x7fa2f00c8170 msgr2=0x7fa2f0151cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:42.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.724+0000 7fa2f7e00640 1 -- 172.21.15.67:0/4265102867 shutdown_connections 2024-03-09T09:22:42.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.724+0000 7fa2f7e00640 1 -- 172.21.15.67:0/4265102867 wait complete. 2024-03-09T09:22:42.751 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:22:42.751 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:22:42.752 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:22:42.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.966+0000 7f9687a00640 1 Processor -- start 2024-03-09T09:22:42.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.966+0000 7f9687a00640 1 -- start start 2024-03-09T09:22:42.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.969+0000 7f9687a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f968006b040 0x7f968006b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.969+0000 7f9687a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f968014a270 0x7f968014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.969+0000 7f9687a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9680154610 con 0x7f9680157b50 2024-03-09T09:22:42.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.969+0000 7f9687a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96801576c0 con 0x7f968014a270 2024-03-09T09:22:42.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.969+0000 7f9687a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96801548f0 con 0x7f968006b040 2024-03-09T09:22:42.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.969+0000 7f9685a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f968014a270 0x7f968014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.970+0000 7f9685a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f968014a270 0x7f968014a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57924/0 (socket says 172.21.15.67:57924) 2024-03-09T09:22:42.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.970+0000 7f9685a00640 1 -- 172.21.15.67:0/1267620495 learned_addr learned my addr 172.21.15.67:0/1267620495 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:42.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.970+0000 7f9685000640 1 --2- 172.21.15.67:0/1267620495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f968006b040 0x7f968006b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.970+0000 7f9685a00640 1 -- 172.21.15.67:0/1267620495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f968006b040 msgr2=0x7f968006b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:42.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.970+0000 7f9685a00640 1 --2- 172.21.15.67:0/1267620495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f968006b040 0x7f968006b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.970+0000 7f9685a00640 1 -- 172.21.15.67:0/1267620495 >> v1:172.21.15.134:6789/0 conn(0x7f9680157b50 legacy=0x7f9680153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.970+0000 7f9685a00640 1 -- 172.21.15.67:0/1267620495 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9680157350 con 0x7f968014a270 2024-03-09T09:22:42.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.971+0000 7f9685a00640 1 --2- 172.21.15.67:0/1267620495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f968014a270 0x7f968014a6d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9678009e50 tx=0x7f967802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94037bcd115e13a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.972+0000 7f967f400640 1 -- 172.21.15.67:0/1267620495 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967802cb00 con 0x7f968014a270 2024-03-09T09:22:42.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.972+0000 7f967f400640 1 -- 172.21.15.67:0/1267620495 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967802cca0 con 0x7f968014a270 2024-03-09T09:22:42.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.972+0000 7f967f400640 1 -- 172.21.15.67:0/1267620495 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678034610 con 0x7f968014a270 2024-03-09T09:22:42.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.972+0000 7f9687a00640 1 -- 172.21.15.67:0/1267620495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f968014a270 msgr2=0x7f968014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.972+0000 7f9687a00640 1 --2- 172.21.15.67:0/1267620495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f968014a270 0x7f968014a6d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9678009e50 tx=0x7f967802bf60 comp rx=0 tx=0).stop 2024-03-09T09:22:42.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.972+0000 7f9687a00640 1 -- 172.21.15.67:0/1267620495 shutdown_connections 2024-03-09T09:22:42.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.972+0000 7f9687a00640 1 --2- 172.21.15.67:0/1267620495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f968014a270 0x7f968014a6d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.972+0000 7f9687a00640 1 --2- 172.21.15.67:0/1267620495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f968006b040 0x7f968006b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.972+0000 7f9687a00640 1 -- 172.21.15.67:0/1267620495 >> 172.21.15.67:0/1267620495 conn(0x7f96800c8b20 msgr2=0x7f968006c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:42.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.972+0000 7f9687a00640 1 -- 172.21.15.67:0/1267620495 shutdown_connections 2024-03-09T09:22:42.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.973+0000 7f9687a00640 1 -- 172.21.15.67:0/1267620495 wait complete. 2024-03-09T09:22:42.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.974+0000 7f9687a00640 1 Processor -- start 2024-03-09T09:22:42.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.974+0000 7f9687a00640 1 -- start start 2024-03-09T09:22:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.975+0000 7f9687a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f968006b040 0x7f96801eb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.975+0000 7f9687a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f968014a270 0x7f968014e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.975+0000 7f9687a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9680157b50 0x7f96801ef2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.975+0000 7f9687a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96800c3010 con 0x7f968014a270 2024-03-09T09:22:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.975+0000 7f9687a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9680157fd0 con 0x7f9680157b50 2024-03-09T09:22:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.975+0000 7f9687a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96800c3190 con 0x7f968006b040 2024-03-09T09:22:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.976+0000 7f967fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f968014a270 0x7f968014e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.976+0000 7f967fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f968014a270 0x7f968014e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57940/0 (socket says 172.21.15.67:57940) 2024-03-09T09:22:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.976+0000 7f967fe00640 1 -- 172.21.15.67:0/1648141941 learned_addr learned my addr 172.21.15.67:0/1648141941 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:42.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.976+0000 7f9685000640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f968006b040 0x7f96801eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.976+0000 7f9685a00640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9680157b50 0x7f96801ef2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.976+0000 7f967fe00640 1 -- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f968006b040 msgr2=0x7f96801eb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.976+0000 7f967fe00640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f968006b040 0x7f96801eb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.976+0000 7f967fe00640 1 -- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9680157b50 msgr2=0x7f96801ef2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:42.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.976+0000 7f967fe00640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9680157b50 0x7f96801ef2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:42.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.976+0000 7f967fe00640 1 -- 172.21.15.67:0/1648141941 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9680071b30 con 0x7f968014a270 2024-03-09T09:22:42.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.977+0000 7f967fe00640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f968014a270 0x7f968014e900 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f966c00c3f0 tx=0x7f966c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.977+0000 7f967cc00640 1 -- 172.21.15.67:0/1648141941 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c0052d0 con 0x7f968014a270 2024-03-09T09:22:42.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.977+0000 7f967cc00640 1 -- 172.21.15.67:0/1648141941 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966c005490 con 0x7f968014a270 2024-03-09T09:22:42.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.977+0000 7f967cc00640 1 -- 172.21.15.67:0/1648141941 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c007260 con 0x7f968014a270 2024-03-09T09:22:42.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.977+0000 7f9687a00640 1 -- 172.21.15.67:0/1648141941 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9680071dd0 con 0x7f968014a270 2024-03-09T09:22:42.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.977+0000 7f9687a00640 1 -- 172.21.15.67:0/1648141941 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96801eed90 con 0x7f968014a270 2024-03-09T09:22:42.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.979+0000 7f9687a00640 1 -- 172.21.15.67:0/1648141941 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96801eeb80 con 0x7f968014a270 2024-03-09T09:22:42.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.980+0000 7f967cc00640 1 -- 172.21.15.67:0/1648141941 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f966c007400 con 0x7f968014a270 2024-03-09T09:22:42.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.981+0000 7f967cc00640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f966006b4d0 0x7f966006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:42.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.982+0000 7f967cc00640 1 -- 172.21.15.67:0/1648141941 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f966c089a30 con 0x7f968014a270 2024-03-09T09:22:42.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.982+0000 7f9685000640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f966006b4d0 0x7f966006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:42.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.983+0000 7f9685000640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f966006b4d0 0x7f966006d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9674006fd0 tx=0x7f9674008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:42.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:42.987+0000 7f967cc00640 1 -- 172.21.15.67:0/1648141941 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f966c057e60 con 0x7f968014a270 2024-03-09T09:22:43.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.270+0000 7f9687a00640 1 -- 172.21.15.67:0/1648141941 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f9680075a00 con 0x7f968014a270 2024-03-09T09:22:43.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.272+0000 7f967cc00640 1 -- 172.21.15.67:0/1648141941 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f966c05ce20 con 0x7f968014a270 2024-03-09T09:22:43.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.274+0000 7f9687a00640 1 -- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f966006b4d0 msgr2=0x7f966006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:43.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.274+0000 7f9687a00640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f966006b4d0 0x7f966006d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9674006fd0 tx=0x7f9674008040 comp rx=0 tx=0).stop 2024-03-09T09:22:43.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.275+0000 7f9687a00640 1 -- 172.21.15.67:0/1648141941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f968014a270 msgr2=0x7f968014e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:43.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.275+0000 7f9687a00640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f968014a270 0x7f968014e900 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f966c00c3f0 tx=0x7f966c00c8b0 comp rx=0 tx=0).stop 2024-03-09T09:22:43.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.275+0000 7f9687a00640 1 -- 172.21.15.67:0/1648141941 shutdown_connections 2024-03-09T09:22:43.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.275+0000 7f9687a00640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f966006b4d0 0x7f966006d990 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:43.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.275+0000 7f9687a00640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9680157b50 0x7f96801ef2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:43.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.275+0000 7f9687a00640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f968014a270 0x7f968014e900 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:43.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.275+0000 7f9687a00640 1 --2- 172.21.15.67:0/1648141941 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f968006b040 0x7f96801eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:43.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.275+0000 7f9687a00640 1 -- 172.21.15.67:0/1648141941 >> 172.21.15.67:0/1648141941 conn(0x7f96800c8b20 msgr2=0x7f9680153950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:43.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.275+0000 7f9687a00640 1 -- 172.21.15.67:0/1648141941 shutdown_connections 2024-03-09T09:22:43.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:43.275+0000 7f9687a00640 1 -- 172.21.15.67:0/1648141941 wait complete. 2024-03-09T09:22:43.310 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:22:44.107 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:22:44.107 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:22:44.107 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 72 -eq 48 ']' 2024-03-09T09:22:44.107 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.72 2024-03-09T09:22:44.107 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.72 2024-03-09T09:22:44.108 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.72 2024-03-09T09:22:44.108 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.72 2024-03-09T09:22:44.114 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:22:48.108 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:22:48.108 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:22:48.108 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 66 -eq 48 ']' 2024-03-09T09:22:48.108 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.66 2024-03-09T09:22:48.109 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.66 2024-03-09T09:22:48.109 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.66 2024-03-09T09:22:48.109 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.66 2024-03-09T09:22:48.113 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:22:49.930 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:22:49.930 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:22:49.930 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 73 -eq 48 ']' 2024-03-09T09:22:49.930 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.73 2024-03-09T09:22:49.931 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.73 2024-03-09T09:22:49.931 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.73 2024-03-09T09:22:49.931 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.73 2024-03-09T09:22:49.934 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:22:53.315 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:22:53.315 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:22:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.533+0000 7f4693000640 1 Processor -- start 2024-03-09T09:22:53.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.533+0000 7f4693000640 1 -- start start 2024-03-09T09:22:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.535+0000 7f4693000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f468c0c48b0 0x7f468c0c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.535+0000 7f4693000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f468c156870 0x7f468c159170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.535+0000 7f4693000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468c06eee0 con 0x7f468c0c88c0 2024-03-09T09:22:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.535+0000 7f4693000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468c06f610 con 0x7f468c0c48b0 2024-03-09T09:22:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.535+0000 7f4693000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468c155d90 con 0x7f468c156870 2024-03-09T09:22:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.535+0000 7f4691c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f468c0c48b0 0x7f468c0c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.535+0000 7f4691c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f468c0c48b0 0x7f468c0c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42476/0 (socket says 172.21.15.67:42476) 2024-03-09T09:22:53.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.535+0000 7f4691c00640 1 -- 172.21.15.67:0/1923397009 learned_addr learned my addr 172.21.15.67:0/1923397009 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.536+0000 7f4691200640 1 --2- 172.21.15.67:0/1923397009 >> v2:172.21.15.134:3300/0 conn(0x7f468c156870 0x7f468c159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.536+0000 7f4691c00640 1 -- 172.21.15.67:0/1923397009 >> v2:172.21.15.134:3300/0 conn(0x7f468c156870 msgr2=0x7f468c159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.536+0000 7f4691c00640 1 --2- 172.21.15.67:0/1923397009 >> v2:172.21.15.134:3300/0 conn(0x7f468c156870 0x7f468c159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.536+0000 7f4691c00640 1 -- 172.21.15.67:0/1923397009 >> v1:172.21.15.134:6789/0 conn(0x7f468c0c88c0 legacy=0x7f468c0c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:53.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.536+0000 7f4691c00640 1 -- 172.21.15.67:0/1923397009 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c156070 con 0x7f468c0c48b0 2024-03-09T09:22:53.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.537+0000 7f4691c00640 1 --2- 172.21.15.67:0/1923397009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f468c0c48b0 0x7f468c0c8380 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f468c06b580 tx=0x7f468002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a1322d4b493025 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.537+0000 7f468be00640 1 -- 172.21.15.67:0/1923397009 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680035020 con 0x7f468c0c48b0 2024-03-09T09:22:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.538+0000 7f468be00640 1 -- 172.21.15.67:0/1923397009 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4680031870 con 0x7f468c0c48b0 2024-03-09T09:22:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.538+0000 7f468be00640 1 -- 172.21.15.67:0/1923397009 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680031c40 con 0x7f468c0c48b0 2024-03-09T09:22:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.538+0000 7f4693000640 1 -- 172.21.15.67:0/1923397009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f468c0c48b0 msgr2=0x7f468c0c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.538+0000 7f4693000640 1 --2- 172.21.15.67:0/1923397009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f468c0c48b0 0x7f468c0c8380 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f468c06b580 tx=0x7f468002b910 comp rx=0 tx=0).stop 2024-03-09T09:22:53.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.538+0000 7f4693000640 1 -- 172.21.15.67:0/1923397009 shutdown_connections 2024-03-09T09:22:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.538+0000 7f4693000640 1 --2- 172.21.15.67:0/1923397009 >> v2:172.21.15.134:3300/0 conn(0x7f468c156870 0x7f468c159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.538+0000 7f4693000640 1 --2- 172.21.15.67:0/1923397009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f468c0c48b0 0x7f468c0c8380 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.538+0000 7f4693000640 1 -- 172.21.15.67:0/1923397009 >> 172.21.15.67:0/1923397009 conn(0x7f468c06d670 msgr2=0x7f468c06da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.539+0000 7f4693000640 1 -- 172.21.15.67:0/1923397009 shutdown_connections 2024-03-09T09:22:53.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.539+0000 7f4693000640 1 -- 172.21.15.67:0/1923397009 wait complete. 2024-03-09T09:22:53.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.541+0000 7f4693000640 1 Processor -- start 2024-03-09T09:22:53.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.541+0000 7f4693000640 1 -- start start 2024-03-09T09:22:53.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.542+0000 7f4693000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f468c0c48b0 0x7f468c14b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:53.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.542+0000 7f4693000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f468c0c88c0 0x7f468c1ec480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:53.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.542+0000 7f4693000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f468c156870 0x7f468c1ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:53.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.542+0000 7f4693000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c06e740 con 0x7f468c0c48b0 2024-03-09T09:22:53.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.542+0000 7f4693000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c06e5c0 con 0x7f468c156870 2024-03-09T09:22:53.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.542+0000 7f4693000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468c06ba70 con 0x7f468c0c88c0 2024-03-09T09:22:53.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.543+0000 7f4691c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f468c0c48b0 0x7f468c14b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:53.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.543+0000 7f4691c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f468c0c48b0 0x7f468c14b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42490/0 (socket says 172.21.15.67:42490) 2024-03-09T09:22:53.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.543+0000 7f4691c00640 1 -- 172.21.15.67:0/3065731029 learned_addr learned my addr 172.21.15.67:0/3065731029 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:53.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.543+0000 7f4691200640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f468c0c88c0 0x7f468c1ec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:53.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.543+0000 7f4692600640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f468c156870 0x7f468c1ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:53.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.543+0000 7f4691c00640 1 -- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f468c0c88c0 msgr2=0x7f468c1ec480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:53.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.543+0000 7f4691c00640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f468c0c88c0 0x7f468c1ec480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:53.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.543+0000 7f4691c00640 1 -- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f468c156870 msgr2=0x7f468c1ec9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:53.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.543+0000 7f4691c00640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f468c156870 0x7f468c1ec9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:53.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.543+0000 7f4691c00640 1 -- 172.21.15.67:0/3065731029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c0ab4e0 con 0x7f468c0c48b0 2024-03-09T09:22:53.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.543+0000 7f4691200640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f468c0c88c0 0x7f468c1ec480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:22:53.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.544+0000 7f4691c00640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f468c0c48b0 0x7f468c14b5f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f468002b950 tx=0x7f46800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:53.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.544+0000 7f4689600640 1 -- 172.21.15.67:0/3065731029 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468003ad60 con 0x7f468c0c48b0 2024-03-09T09:22:53.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.544+0000 7f4689600640 1 -- 172.21.15.67:0/3065731029 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46800443f0 con 0x7f468c0c48b0 2024-03-09T09:22:53.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.544+0000 7f4689600640 1 -- 172.21.15.67:0/3065731029 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46800447e0 con 0x7f468c0c48b0 2024-03-09T09:22:53.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.544+0000 7f4693000640 1 -- 172.21.15.67:0/3065731029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c0ab7c0 con 0x7f468c0c48b0 2024-03-09T09:22:53.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.544+0000 7f4693000640 1 -- 172.21.15.67:0/3065731029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c1ebf60 con 0x7f468c0c48b0 2024-03-09T09:22:53.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.545+0000 7f4693000640 1 -- 172.21.15.67:0/3065731029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468c068b10 con 0x7f468c0c48b0 2024-03-09T09:22:53.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.546+0000 7f4689600640 1 -- 172.21.15.67:0/3065731029 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4680038020 con 0x7f468c0c48b0 2024-03-09T09:22:53.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.547+0000 7f4689600640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f466c06b4d0 0x7f466c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:53.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.547+0000 7f4689600640 1 -- 172.21.15.67:0/3065731029 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f46800ac720 con 0x7f468c0c48b0 2024-03-09T09:22:53.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.548+0000 7f4691200640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f466c06b4d0 0x7f466c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:53.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.549+0000 7f4691200640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f466c06b4d0 0x7f466c06d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4678006fd0 tx=0x7f4678008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:53.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.553+0000 7f4689600640 1 -- 172.21.15.67:0/3065731029 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f468c068b10 con 0x7f468c0c48b0 2024-03-09T09:22:53.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.840+0000 7f4693000640 1 -- 172.21.15.67:0/3065731029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f468c067930 con 0x7f468c0c48b0 2024-03-09T09:22:53.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.841+0000 7f4689600640 1 -- 172.21.15.67:0/3065731029 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f468c067930 con 0x7f468c0c48b0 2024-03-09T09:22:53.847 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":236,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:22:53.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.844+0000 7f4693000640 1 -- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f466c06b4d0 msgr2=0x7f466c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:53.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.844+0000 7f4693000640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f466c06b4d0 0x7f466c06d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4678006fd0 tx=0x7f4678008040 comp rx=0 tx=0).stop 2024-03-09T09:22:53.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.844+0000 7f4693000640 1 -- 172.21.15.67:0/3065731029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f468c0c48b0 msgr2=0x7f468c14b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:53.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.844+0000 7f4693000640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f468c0c48b0 0x7f468c14b5f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f468002b950 tx=0x7f46800046c0 comp rx=0 tx=0).stop 2024-03-09T09:22:53.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.845+0000 7f4693000640 1 -- 172.21.15.67:0/3065731029 shutdown_connections 2024-03-09T09:22:53.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.845+0000 7f4693000640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f466c06b4d0 0x7f466c06d990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:53.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.845+0000 7f4693000640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f468c156870 0x7f468c1ec9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:53.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.845+0000 7f4693000640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f468c0c88c0 0x7f468c1ec480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:53.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.845+0000 7f4693000640 1 --2- 172.21.15.67:0/3065731029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f468c0c48b0 0x7f468c14b5f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:53.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.845+0000 7f4693000640 1 -- 172.21.15.67:0/3065731029 >> 172.21.15.67:0/3065731029 conn(0x7f468c06d670 msgr2=0x7f468c155110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:53.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.845+0000 7f4693000640 1 -- 172.21.15.67:0/3065731029 shutdown_connections 2024-03-09T09:22:53.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:53.845+0000 7f4693000640 1 -- 172.21.15.67:0/3065731029 wait complete. 2024-03-09T09:22:53.869 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:22:53.869 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:22:53.869 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:22:53.870 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:22:53.896 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:22:53.897 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:22:53.897 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 67 -eq 48 ']' 2024-03-09T09:22:53.897 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.67 2024-03-09T09:22:53.897 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.67 2024-03-09T09:22:53.897 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.67 2024-03-09T09:22:53.898 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.67 2024-03-09T09:22:53.901 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:22:54.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.079+0000 7fc585800640 1 Processor -- start 2024-03-09T09:22:54.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.079+0000 7fc585800640 1 -- start start 2024-03-09T09:22:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.082+0000 7fc585800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc580150140 0x7fc58006d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.082+0000 7fc585800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58006d940 con 0x7fc5800c2fa0 2024-03-09T09:22:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.082+0000 7fc585800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58006dc20 con 0x7fc580152790 2024-03-09T09:22:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.082+0000 7fc585800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc580071b20 con 0x7fc580150140 2024-03-09T09:22:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.082+0000 7fc577e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc580150140 0x7fc58006d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.083+0000 7fc57fe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc5800c2fa0 0x7fc58014fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58062/0 (socket says 172.21.15.67:58062) 2024-03-09T09:22:54.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.083+0000 7fc57fe00640 1 -- 172.21.15.67:0/2134737738 learned_addr learned my addr 172.21.15.67:0/2134737738 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:22:54.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.083+0000 7fc577e00640 1 -- 172.21.15.67:0/2134737738 >> v1:172.21.15.134:6789/0 conn(0x7fc580152790 legacy=0x7fc580071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.083+0000 7fc577e00640 1 -- 172.21.15.67:0/2134737738 >> v1:172.21.15.67:6789/0 conn(0x7fc5800c2fa0 legacy=0x7fc58014fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.083+0000 7fc577e00640 1 -- 172.21.15.67:0/2134737738 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58006df00 con 0x7fc580150140 2024-03-09T09:22:54.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.083+0000 7fc57f400640 1 -- 172.21.15.67:0/2134737738 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826011591 0 0) 0x7fc58006d940 con 0x7fc5800c2fa0 2024-03-09T09:22:54.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.084+0000 7fc577e00640 1 --2- 172.21.15.67:0/2134737738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580150140 0x7fc58006d340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc580071ca0 tx=0x7fc56802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfac6f6744e689a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.085+0000 7fc57f400640 1 -- 172.21.15.67:0/2134737738 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc568035020 con 0x7fc580150140 2024-03-09T09:22:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.085+0000 7fc57f400640 1 -- 172.21.15.67:0/2134737738 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc568031830 con 0x7fc580150140 2024-03-09T09:22:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.085+0000 7fc57f400640 1 -- 172.21.15.67:0/2134737738 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc568031c00 con 0x7fc580150140 2024-03-09T09:22:54.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.085+0000 7fc585800640 1 -- 172.21.15.67:0/2134737738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580150140 msgr2=0x7fc58006d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.085+0000 7fc585800640 1 --2- 172.21.15.67:0/2134737738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580150140 0x7fc58006d340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc580071ca0 tx=0x7fc56802b8d0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.086+0000 7fc585800640 1 -- 172.21.15.67:0/2134737738 shutdown_connections 2024-03-09T09:22:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.086+0000 7fc585800640 1 --2- 172.21.15.67:0/2134737738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580150140 0x7fc58006d340 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.086+0000 7fc585800640 1 -- 172.21.15.67:0/2134737738 >> 172.21.15.67:0/2134737738 conn(0x7fc5800c7220 msgr2=0x7fc5800c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.086+0000 7fc585800640 1 -- 172.21.15.67:0/2134737738 shutdown_connections 2024-03-09T09:22:54.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.086+0000 7fc585800640 1 -- 172.21.15.67:0/2134737738 wait complete. 2024-03-09T09:22:54.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.088+0000 7fc585800640 1 Processor -- start 2024-03-09T09:22:54.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.088+0000 7fc585800640 1 -- start start 2024-03-09T09:22:54.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.090+0000 7fc585800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5800c2fa0 0x7fc5800c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.090+0000 7fc585800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580150140 0x7fc580150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.090+0000 7fc585800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580152790 0x7fc58014d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.090+0000 7fc585800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc580071340 con 0x7fc580152790 2024-03-09T09:22:54.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.090+0000 7fc585800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5800711c0 con 0x7fc580150140 2024-03-09T09:22:54.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.090+0000 7fc585800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5800c3380 con 0x7fc5800c2fa0 2024-03-09T09:22:54.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.090+0000 7fc584e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580152790 0x7fc58014d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.090+0000 7fc584e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580152790 0x7fc58014d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42500/0 (socket says 172.21.15.67:42500) 2024-03-09T09:22:54.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.090+0000 7fc584e00640 1 -- 172.21.15.67:0/2124519738 learned_addr learned my addr 172.21.15.67:0/2124519738 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:54.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.090+0000 7fc57fe00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5800c2fa0 0x7fc5800c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.090+0000 7fc577e00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580150140 0x7fc580150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.091+0000 7fc584e00640 1 -- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5800c2fa0 msgr2=0x7fc5800c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.091+0000 7fc584e00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5800c2fa0 0x7fc5800c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.091+0000 7fc584e00640 1 -- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580150140 msgr2=0x7fc580150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.091+0000 7fc584e00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580150140 0x7fc580150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.091+0000 7fc584e00640 1 -- 172.21.15.67:0/2124519738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58014f160 con 0x7fc580152790 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.091+0000 7fc57fe00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5800c2fa0 0x7fc5800c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.091+0000 7fc577e00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc580150140 0x7fc580150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.091+0000 7fc584e00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580152790 0x7fc58014d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc57800c6e0 tx=0x7fc57800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.091+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578003b70 con 0x7fc580152790 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.091+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc578003d30 con 0x7fc580152790 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.091+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578005310 con 0x7fc580152790 2024-03-09T09:22:54.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.092+0000 7fc585800640 1 -- 172.21.15.67:0/2124519738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc580152020 con 0x7fc580152790 2024-03-09T09:22:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.092+0000 7fc585800640 1 -- 172.21.15.67:0/2124519738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5800c5220 con 0x7fc580152790 2024-03-09T09:22:54.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.093+0000 7fc585800640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580151440 0x7fc5800c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.093+0000 7fc585800640 1 -- 172.21.15.67:0/2124519738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58004f3c0 con 0x7fc580151440 2024-03-09T09:22:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.093+0000 7fc57fe00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580151440 0x7fc5800c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.094+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc5780054b0 con 0x7fc580152790 2024-03-09T09:22:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.094+0000 7fc57fe00640 1 -- 172.21.15.67:0/2124519738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5800c1100 con 0x7fc580151440 2024-03-09T09:22:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.094+0000 7fc57fe00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580151440 0x7fc5800c5460 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc570000c00 tx=0x7fc570001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.095+0000 7fc57cc00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc55c06b500 0x7fc55c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.095+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc57808a440 con 0x7fc580152790 2024-03-09T09:22:54.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.095+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57000e5f0 con 0x7fc580151440 2024-03-09T09:22:54.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.095+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc570018070 con 0x7fc580151440 2024-03-09T09:22:54.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.095+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580151440 msgr2=0x7fc5800c5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.095+0000 7fc57cc00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580151440 0x7fc5800c5460 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc570000c00 tx=0x7fc570001970 comp rx=0 tx=0).stop 2024-03-09T09:22:54.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.095+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580151440 msgr2=0x7fc5800c5460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:54.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.095+0000 7fc57cc00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580151440 0x7fc5800c5460 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc570000c00 tx=0x7fc570001970 comp rx=0 tx=0).stop 2024-03-09T09:22:54.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.096+0000 7fc577e00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc55c06b500 0x7fc55c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.097+0000 7fc577e00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc55c06b500 0x7fc55c06d9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc568036000 tx=0x7fc568008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.103+0000 7fc585800640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc58014d990 0x7fc580140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.103+0000 7fc585800640 1 -- 172.21.15.67:0/2124519738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc580067b30 con 0x7fc58014d990 2024-03-09T09:22:54.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.103+0000 7fc57fe00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc58014d990 0x7fc580140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.103+0000 7fc57fe00640 1 -- 172.21.15.67:0/2124519738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc5800c5b50 con 0x7fc58014d990 2024-03-09T09:22:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.104+0000 7fc57fe00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc58014d990 0x7fc580140f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc570017000 tx=0x7fc570000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.104+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57000eae0 con 0x7fc58014d990 2024-03-09T09:22:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.104+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4537 (secure 0 0 0) 0x7fc57000eae0 con 0x7fc58014d990 2024-03-09T09:22:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.104+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc58014d990 msgr2=0x7fc580140f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.104+0000 7fc57cc00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc58014d990 0x7fc580140f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc570017000 tx=0x7fc570000ed0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.104+0000 7fc57cc00640 1 -- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc58014d990 msgr2=0x7fc580140f20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:54.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.104+0000 7fc57cc00640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc58014d990 0x7fc580140f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 236, 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:54.111 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.112 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:22:54.114 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:54.115 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:22:54.116 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:54.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:22:54.118 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:54.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:22:54.120 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:22:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:22:54.122 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:22:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.105+0000 7fc585800640 1 -- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc55c06b500 msgr2=0x7fc55c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.105+0000 7fc585800640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc55c06b500 0x7fc55c06d9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc568036000 tx=0x7fc568008030 comp rx=0 tx=0).stop 2024-03-09T09:22:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.105+0000 7fc585800640 1 -- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580152790 msgr2=0x7fc58014d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.105+0000 7fc585800640 1 --2- 172.21.15.67:0/2124519738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc580152790 0x7fc58014d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc57800c6e0 tx=0x7fc57800cba0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.105+0000 7fc584e00640 1 -- 172.21.15.67:0/2124519738 reap_dead start 2024-03-09T09:22:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.105+0000 7fc584e00640 1 -- 172.21.15.67:0/2124519738 reap_dead start 2024-03-09T09:22:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.105+0000 7fc585800640 1 -- 172.21.15.67:0/2124519738 shutdown_connections 2024-03-09T09:22:54.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.105+0000 7fc585800640 1 -- 172.21.15.67:0/2124519738 >> 172.21.15.67:0/2124519738 conn(0x7fc5800c7220 msgr2=0x7fc580072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:54.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.105+0000 7fc585800640 1 -- 172.21.15.67:0/2124519738 shutdown_connections 2024-03-09T09:22:54.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.105+0000 7fc585800640 1 -- 172.21.15.67:0/2124519738 wait complete. 2024-03-09T09:22:54.126 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:22:54.130 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:22:54.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:22:54.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.332+0000 7f4ff5600640 1 Processor -- start 2024-03-09T09:22:54.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.333+0000 7f4ff5600640 1 -- start start 2024-03-09T09:22:54.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.335+0000 7f4ff5600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4ff014fd90 0x7f4ff0156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.335+0000 7f4ff5600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff0156bc0 con 0x7f4ff006c1d0 2024-03-09T09:22:54.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.335+0000 7f4ff5600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff00c9890 con 0x7f4ff00c9380 2024-03-09T09:22:54.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.335+0000 7f4ff5600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff006cd00 con 0x7f4ff014fd90 2024-03-09T09:22:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.335+0000 7f4ff4c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4ff006c1d0 0x7f4ff006c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58066/0 (socket says 172.21.15.67:58066) 2024-03-09T09:22:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.335+0000 7f4ff4c00640 1 -- 172.21.15.67:0/3335889814 learned_addr learned my addr 172.21.15.67:0/3335889814 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:22:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.335+0000 7f4fefe00640 1 --2- 172.21.15.67:0/3335889814 >> v2:172.21.15.134:3300/0 conn(0x7f4ff014fd90 0x7f4ff0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159706840 0 0) 0x7f4fe8001490 con 0x7f4ff006c1d0 2024-03-09T09:22:54.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fd4003520 con 0x7f4ff006c1d0 2024-03-09T09:22:54.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219267913 0 0) 0x7f4ff00c9890 con 0x7f4ff00c9380 2024-03-09T09:22:54.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe8001490 con 0x7f4ff00c9380 2024-03-09T09:22:54.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1323989874 0 0) 0x7f4fd4003520 con 0x7f4ff006c1d0 2024-03-09T09:22:54.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 >> v2:172.21.15.134:3300/0 conn(0x7f4ff014fd90 msgr2=0x7f4ff0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:54.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 --2- 172.21.15.67:0/3335889814 >> v2:172.21.15.134:3300/0 conn(0x7f4ff014fd90 0x7f4ff0156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 >> v1:172.21.15.134:6789/0 conn(0x7f4ff00c9380 legacy=0x7f4ff00acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff00c5f80 con 0x7f4ff006c1d0 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f4fe8001b40 con 0x7f4ff006c1d0 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 >> v1:172.21.15.67:6789/0 conn(0x7f4ff006c1d0 legacy=0x7f4ff006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 --2- 172.21.15.67:0/3335889814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4003c00 0x7f4fd4004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 --2- 172.21.15.67:0/3335889814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fd4004800 0x7f4fd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 --2- 172.21.15.67:0/3335889814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fd40052f0 0x7f4fd400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff006cd00 con 0x7f4fd4004800 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd400c230 con 0x7f4fd4003c00 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.336+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd400c430 con 0x7f4fd40052f0 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.337+0000 7f4fef400640 1 --2- 172.21.15.67:0/3335889814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fd4004800 0x7f4fd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.337+0000 7f4ff4c00640 1 --2- 172.21.15.67:0/3335889814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fd40052f0 0x7f4fd400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.337+0000 7f4fefe00640 1 --2- 172.21.15.67:0/3335889814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4003c00 0x7f4fd4004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.337+0000 7f4fef400640 1 -- 172.21.15.67:0/3335889814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fd40052f0 msgr2=0x7f4fd400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.337+0000 7f4fef400640 1 --2- 172.21.15.67:0/3335889814 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fd40052f0 0x7f4fd400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.337+0000 7f4fef400640 1 -- 172.21.15.67:0/3335889814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4003c00 msgr2=0x7f4fd4004000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.337+0000 7f4fef400640 1 --2- 172.21.15.67:0/3335889814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4003c00 0x7f4fd4004000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.337+0000 7f4fef400640 1 -- 172.21.15.67:0/3335889814 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4003520 con 0x7f4fd4004800 2024-03-09T09:22:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.337+0000 7f4ff4c00640 1 -- 172.21.15.67:0/3335889814 reap_dead start 2024-03-09T09:22:54.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.339+0000 7f4fef400640 1 --2- 172.21.15.67:0/3335889814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fd4004800 0x7f4fd4004c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0018d0 tx=0x7f4fd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5bea3d1172b030 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.339+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc037070 con 0x7f4fd4004800 2024-03-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.339+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc0052d0 con 0x7f4fd4004800 2024-03-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.339+0000 7f4feea00640 1 -- 172.21.15.67:0/3335889814 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0014f0 con 0x7f4fd4004800 2024-03-09T09:22:54.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.341+0000 7f4ff5600640 1 -- 172.21.15.67:0/3335889814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fd4004800 msgr2=0x7f4fd4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.341+0000 7f4ff5600640 1 --2- 172.21.15.67:0/3335889814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fd4004800 0x7f4fd4004c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0018d0 tx=0x7f4fd4005830 comp rx=0 tx=0).stop 2024-03-09T09:22:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.341+0000 7f4ff5600640 1 -- 172.21.15.67:0/3335889814 shutdown_connections 2024-03-09T09:22:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.341+0000 7f4ff5600640 1 --2- 172.21.15.67:0/3335889814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fd4004800 0x7f4fd4004c80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.341+0000 7f4ff5600640 1 -- 172.21.15.67:0/3335889814 >> 172.21.15.67:0/3335889814 conn(0x7f4ff00cb480 msgr2=0x7f4ff00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.341+0000 7f4ff5600640 1 -- 172.21.15.67:0/3335889814 shutdown_connections 2024-03-09T09:22:54.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.341+0000 7f4ff5600640 1 -- 172.21.15.67:0/3335889814 wait complete. 2024-03-09T09:22:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.343+0000 7f4ff5600640 1 Processor -- start 2024-03-09T09:22:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.343+0000 7f4ff5600640 1 -- start start 2024-03-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.344+0000 7f4ff5600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4004800 0x7f4ff0149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.344+0000 7f4ff5600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff0150170 0x7f4ff01579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.344+0000 7f4ff5600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff0157f10 0x7f4ff01eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.344+0000 7f4ff5600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00c6520 con 0x7f4ff0157f10 2024-03-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.344+0000 7f4ff5600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff006ba60 con 0x7f4fd4004800 2024-03-09T09:22:54.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.344+0000 7f4ff5600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff006bcb0 con 0x7f4ff0150170 2024-03-09T09:22:54.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.344+0000 7f4ff4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff0157f10 0x7f4ff01eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.344+0000 7f4ff4c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff0157f10 0x7f4ff01eec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42540/0 (socket says 172.21.15.67:42540) 2024-03-09T09:22:54.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.345+0000 7f4ff4c00640 1 -- 172.21.15.67:0/3274196304 learned_addr learned my addr 172.21.15.67:0/3274196304 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:54.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.345+0000 7f4fef400640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff0150170 0x7f4ff01579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.345+0000 7f4fefe00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4004800 0x7f4ff0149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.345+0000 7f4ff4c00640 1 -- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff0150170 msgr2=0x7f4ff01579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.345+0000 7f4ff4c00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff0150170 0x7f4ff01579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.345+0000 7f4ff4c00640 1 -- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4004800 msgr2=0x7f4ff0149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.345+0000 7f4ff4c00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fd4004800 0x7f4ff0149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.345+0000 7f4ff4c00640 1 -- 172.21.15.67:0/3274196304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0158bb0 con 0x7f4ff0157f10 2024-03-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.346+0000 7f4ff4c00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff0157f10 0x7f4ff01eec60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0073730 tx=0x7f4fe800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.346+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800ec10 con 0x7f4ff0157f10 2024-03-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.346+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe8018070 con 0x7f4ff0157f10 2024-03-09T09:22:54.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.346+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8013420 con 0x7f4ff0157f10 2024-03-09T09:22:54.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.346+0000 7f4ff5600640 1 -- 172.21.15.67:0/3274196304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff0149bd0 con 0x7f4ff0157f10 2024-03-09T09:22:54.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.346+0000 7f4ff5600640 1 -- 172.21.15.67:0/3274196304 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff014a2d0 con 0x7f4ff0157f10 2024-03-09T09:22:54.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.348+0000 7f4ff5600640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0149da0 0x7f4ff01ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.348+0000 7f4ff5600640 1 -- 172.21.15.67:0/3274196304 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff004f3c0 con 0x7f4ff0149da0 2024-03-09T09:22:54.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.348+0000 7f4fefe00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0149da0 0x7f4ff01ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.348+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4fe80135c0 con 0x7f4ff0157f10 2024-03-09T09:22:54.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.349+0000 7f4fefe00640 1 -- 172.21.15.67:0/3274196304 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ff006d2d0 con 0x7f4ff0149da0 2024-03-09T09:22:54.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.349+0000 7f4fe7e00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4fd006b500 0x7f4fd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.349+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4fe808a770 con 0x7f4ff0157f10 2024-03-09T09:22:54.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.349+0000 7f4fefe00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0149da0 0x7f4ff01ef1a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0068970 tx=0x7f4fe0014a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.350+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0004440 con 0x7f4ff0149da0 2024-03-09T09:22:54.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.350+0000 7f4fef400640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4fd006b500 0x7f4fd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.351+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4fe0004440 con 0x7f4ff0149da0 2024-03-09T09:22:54.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.351+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0149da0 msgr2=0x7f4ff01ef1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.351+0000 7f4fe7e00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0149da0 0x7f4ff01ef1a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0068970 tx=0x7f4fe0014a80 comp rx=0 tx=0).stop 2024-03-09T09:22:54.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.351+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0149da0 msgr2=0x7f4ff01ef1a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:54.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.351+0000 7f4fe7e00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0149da0 0x7f4ff01ef1a0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0068970 tx=0x7f4fe0014a80 comp rx=0 tx=0).stop 2024-03-09T09:22:54.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.351+0000 7f4fef400640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4fd006b500 0x7f4fd006d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc034000 tx=0x7f4fdc031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.364+0000 7f4ff5600640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0150840 0x7f4ff0159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.364+0000 7f4ff5600640 1 -- 172.21.15.67:0/3274196304 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00746e0 con 0x7f4ff0150840 2024-03-09T09:22:54.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.364+0000 7f4fefe00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0150840 0x7f4ff0159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.365+0000 7f4fefe00640 1 -- 172.21.15.67:0/3274196304 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4ff01efb40 con 0x7f4ff0150840 2024-03-09T09:22:54.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.365+0000 7f4fefe00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0150840 0x7f4ff0159810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0068970 tx=0x7f4fe0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.366+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0012d40 con 0x7f4ff0150840 2024-03-09T09:22:54.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f4fe0012d40 con 0x7f4ff0150840 2024-03-09T09:22:54.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0150840 msgr2=0x7f4ff0159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4fe7e00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0150840 0x7f4ff0159810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0068970 tx=0x7f4fe0004730 comp rx=0 tx=0).stop 2024-03-09T09:22:54.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4fe7e00640 1 -- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0150840 msgr2=0x7f4ff0159810 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:54.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4fe7e00640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0150840 0x7f4ff0159810 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0068970 tx=0x7f4fe0004730 comp rx=0 tx=0).stop 2024-03-09T09:22:54.373 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:22:54.373 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:22:54.373 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:22:54.373 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:22:54.373 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 236, 2024-03-09T09:22:54.373 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:22:54.373 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:22:54.373 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:22:54.373 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:54.374 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:54.375 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:22:54.376 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:54.377 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:54.379 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:54.380 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:54.381 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:22:54.382 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.383 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:22:54.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.385 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.386 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:54.386 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.386 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.386 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:22:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:22:54.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4ff5600640 1 -- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4fd006b500 msgr2=0x7f4fd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4ff5600640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4fd006b500 0x7f4fd006d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc034000 tx=0x7f4fdc031000 comp rx=0 tx=0).stop 2024-03-09T09:22:54.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4ff5600640 1 -- 172.21.15.67:0/3274196304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff0157f10 msgr2=0x7f4ff01eec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4ff5600640 1 --2- 172.21.15.67:0/3274196304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ff0157f10 0x7f4ff01eec60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0073730 tx=0x7f4fe800e530 comp rx=0 tx=0).stop 2024-03-09T09:22:54.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4ff4c00640 1 -- 172.21.15.67:0/3274196304 reap_dead start 2024-03-09T09:22:54.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4ff4c00640 1 -- 172.21.15.67:0/3274196304 reap_dead start 2024-03-09T09:22:54.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4ff5600640 1 -- 172.21.15.67:0/3274196304 shutdown_connections 2024-03-09T09:22:54.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.367+0000 7f4ff5600640 1 -- 172.21.15.67:0/3274196304 >> 172.21.15.67:0/3274196304 conn(0x7f4ff00cb480 msgr2=0x7f4ff014dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:54.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.368+0000 7f4ff5600640 1 -- 172.21.15.67:0/3274196304 shutdown_connections 2024-03-09T09:22:54.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.368+0000 7f4ff5600640 1 -- 172.21.15.67:0/3274196304 wait complete. 2024-03-09T09:22:54.395 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:22:54.396 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:22:54.396 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:22:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.598+0000 7f7722e00640 1 Processor -- start 2024-03-09T09:22:54.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.598+0000 7f7722e00640 1 -- start start 2024-03-09T09:22:54.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.600+0000 7f7722e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f771c1535e0 0x7f771c06dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.600+0000 7f7722e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c0730d0 0x7f771c0c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.600+0000 7f7722e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771c156a40 con 0x7f771c0c5900 2024-03-09T09:22:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.600+0000 7f7722e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771c073850 con 0x7f771c1535e0 2024-03-09T09:22:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.600+0000 7f7722e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c073570 con 0x7f771c0730d0 2024-03-09T09:22:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.600+0000 7f771be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f771c1535e0 0x7f771c06dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.600+0000 7f771be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f771c1535e0 0x7f771c06dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42546/0 (socket says 172.21.15.67:42546) 2024-03-09T09:22:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.600+0000 7f771be00640 1 -- 172.21.15.67:0/2090191179 learned_addr learned my addr 172.21.15.67:0/2090191179 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.600+0000 7f771b400640 1 --1- 172.21.15.67:0/2090191179 >> v1:172.21.15.67:6789/0 conn(0x7f771c0c5900 0x7f771c0729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:58068/0 (socket says 172.21.15.67:58068) 2024-03-09T09:22:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.600+0000 7f7720e00640 1 --2- 172.21.15.67:0/2090191179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c0730d0 0x7f771c0c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.601+0000 7f771be00640 1 -- 172.21.15.67:0/2090191179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c0730d0 msgr2=0x7f771c0c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:54.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.601+0000 7f771be00640 1 --2- 172.21.15.67:0/2090191179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c0730d0 0x7f771c0c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.601+0000 7f771be00640 1 -- 172.21.15.67:0/2090191179 >> v1:172.21.15.67:6789/0 conn(0x7f771c0c5900 legacy=0x7f771c0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.601+0000 7f771be00640 1 -- 172.21.15.67:0/2090191179 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c156ce0 con 0x7f771c1535e0 2024-03-09T09:22:54.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.602+0000 7f771be00640 1 --2- 172.21.15.67:0/2090191179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f771c1535e0 0x7f771c06dce0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f770c009870 tx=0x7f770c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9710a5b72fda2f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.602+0000 7f771aa00640 1 -- 172.21.15.67:0/2090191179 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c035020 con 0x7f771c1535e0 2024-03-09T09:22:54.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.602+0000 7f771aa00640 1 -- 172.21.15.67:0/2090191179 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c031640 con 0x7f771c1535e0 2024-03-09T09:22:54.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.602+0000 7f771aa00640 1 -- 172.21.15.67:0/2090191179 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c02d040 con 0x7f771c1535e0 2024-03-09T09:22:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.603+0000 7f7722e00640 1 -- 172.21.15.67:0/2090191179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f771c1535e0 msgr2=0x7f771c06dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.603+0000 7f7722e00640 1 --2- 172.21.15.67:0/2090191179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f771c1535e0 0x7f771c06dce0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f770c009870 tx=0x7f770c02b8f0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.603+0000 7f7722e00640 1 -- 172.21.15.67:0/2090191179 shutdown_connections 2024-03-09T09:22:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.603+0000 7f7722e00640 1 --2- 172.21.15.67:0/2090191179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c0730d0 0x7f771c0c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.603+0000 7f7722e00640 1 --2- 172.21.15.67:0/2090191179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f771c1535e0 0x7f771c06dce0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.603+0000 7f7722e00640 1 -- 172.21.15.67:0/2090191179 >> 172.21.15.67:0/2090191179 conn(0x7f771c0ab320 msgr2=0x7f771c0ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.603+0000 7f7722e00640 1 -- 172.21.15.67:0/2090191179 shutdown_connections 2024-03-09T09:22:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.603+0000 7f7722e00640 1 -- 172.21.15.67:0/2090191179 wait complete. 2024-03-09T09:22:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.605+0000 7f7722e00640 1 Processor -- start 2024-03-09T09:22:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.605+0000 7f7722e00640 1 -- start start 2024-03-09T09:22:54.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.606+0000 7f7722e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f771c0730d0 0x7f771c06ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.606+0000 7f7722e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c0c5900 0x7f771c14c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.606+0000 7f7722e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f771c1535e0 0x7f771c14c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.606+0000 7f7722e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c153b40 con 0x7f771c1535e0 2024-03-09T09:22:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.606+0000 7f7722e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c1539c0 con 0x7f771c0730d0 2024-03-09T09:22:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.606+0000 7f7722e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c0c3090 con 0x7f771c0c5900 2024-03-09T09:22:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.606+0000 7f7720e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f771c1535e0 0x7f771c14c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.606+0000 7f7720e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f771c1535e0 0x7f771c14c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42560/0 (socket says 172.21.15.67:42560) 2024-03-09T09:22:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.606+0000 7f7720e00640 1 -- 172.21.15.67:0/981536336 learned_addr learned my addr 172.21.15.67:0/981536336 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.606+0000 7f771be00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f771c0730d0 0x7f771c06ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.606+0000 7f771b400640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c0c5900 0x7f771c14c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.607+0000 7f7720e00640 1 -- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c0c5900 msgr2=0x7f771c14c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.607+0000 7f7720e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c0c5900 0x7f771c14c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.607+0000 7f7720e00640 1 -- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f771c0730d0 msgr2=0x7f771c06ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.607+0000 7f7720e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f771c0730d0 0x7f771c06ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.607+0000 7f7720e00640 1 -- 172.21.15.67:0/981536336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c14cb60 con 0x7f771c1535e0 2024-03-09T09:22:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.607+0000 7f7720e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f771c1535e0 0x7f771c14c620 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7714007e10 tx=0x7f771400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.607+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714014620 con 0x7f771c1535e0 2024-03-09T09:22:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.607+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7714019070 con 0x7f771c1535e0 2024-03-09T09:22:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.607+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714014a20 con 0x7f771c1535e0 2024-03-09T09:22:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.607+0000 7f7722e00640 1 -- 172.21.15.67:0/981536336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771c14e020 con 0x7f771c1535e0 2024-03-09T09:22:54.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.608+0000 7f7722e00640 1 -- 172.21.15.67:0/981536336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771c1585e0 con 0x7f771c1535e0 2024-03-09T09:22:54.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.609+0000 7f7722e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c075a00 0x7f771c1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.609+0000 7f7722e00640 1 -- 172.21.15.67:0/981536336 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c158440 con 0x7f771c075a00 2024-03-09T09:22:54.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.609+0000 7f771be00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c075a00 0x7f771c1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.610+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7714014bc0 con 0x7f771c1535e0 2024-03-09T09:22:54.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.610+0000 7f771be00640 1 -- 172.21.15.67:0/981536336 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f771c154180 con 0x7f771c075a00 2024-03-09T09:22:54.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.610+0000 7f771be00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c075a00 0x7f771c1587b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f771c071fc0 tx=0x7f770c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.611+0000 7f7713e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f76fc06b500 0x7f76fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.611+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f771408adb0 con 0x7f771c1535e0 2024-03-09T09:22:54.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.611+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c049070 con 0x7f771c075a00 2024-03-09T09:22:54.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.611+0000 7f771b400640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f76fc06b500 0x7f76fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.612+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f770c044400 con 0x7f771c075a00 2024-03-09T09:22:54.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.612+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c075a00 msgr2=0x7f771c1587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.612+0000 7f7713e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c075a00 0x7f771c1587b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f771c071fc0 tx=0x7f770c004730 comp rx=0 tx=0).stop 2024-03-09T09:22:54.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.612+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c075a00 msgr2=0x7f771c1587b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:54.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.612+0000 7f7713e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c075a00 0x7f771c1587b0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f771c071fc0 tx=0x7f770c004730 comp rx=0 tx=0).stop 2024-03-09T09:22:54.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.612+0000 7f771b400640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f76fc06b500 0x7f76fc06d9c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f771c06def0 tx=0x7f7708007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.626+0000 7f7722e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c157260 0x7f771c1590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.626+0000 7f7722e00640 1 -- 172.21.15.67:0/981536336 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c06dc40 con 0x7f771c157260 2024-03-09T09:22:54.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.627+0000 7f771be00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c157260 0x7f771c1590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.628+0000 7f771be00640 1 -- 172.21.15.67:0/981536336 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f771c070450 con 0x7f771c157260 2024-03-09T09:22:54.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.628+0000 7f771be00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c157260 0x7f771c1590c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f770c000c00 tx=0x7f770c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.629+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c04f070 con 0x7f771c157260 2024-03-09T09:22:54.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.630+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f770c04ba30 con 0x7f771c157260 2024-03-09T09:22:54.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.630+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c157260 msgr2=0x7f771c1590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.630+0000 7f7713e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c157260 0x7f771c1590c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f770c000c00 tx=0x7f770c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.630+0000 7f7713e00640 1 -- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c157260 msgr2=0x7f771c1590c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:22:54.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.630+0000 7f7713e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f771c157260 0x7f771c1590c0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f770c000c00 tx=0x7f770c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 236, 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:22:54.637 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:54.638 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:22:54.638 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:22:54.638 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:54.638 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:54.638 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:54.638 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:54.638 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:54.638 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:54.638 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:54.638 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:54.638 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:22:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:22:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:54.640 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:54.640 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:54.640 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:54.640 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:54.640 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:54.640 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.640 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:22:54.640 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:22:54.640 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:22:54.640 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:22:54.641 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:22:54.641 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:22:54.641 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:54.641 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:22:54.641 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:22:54.641 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:22:54.641 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:22:54.641 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:22:54.641 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:22:54.641 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:22:54.641 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:22:54.642 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:22:54.642 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:22:54.642 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:22:54.642 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:22:54.642 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:22:54.642 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:22:54.642 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:22:54.642 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:22:54.642 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:22:54.642 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:22:54.643 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:22:54.643 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:22:54.643 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:22:54.643 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.643 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:22:54.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.643 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:22:54.643 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:22:54.643 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:22:54.643 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:54.644 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:54.644 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.644 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:54.644 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:22:54.644 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.644 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.644 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:54.644 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:22:54.644 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.645 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.645 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.645 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:22:54.645 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:22:54.645 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:54.645 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:54.645 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:54.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.645 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:22:54.646 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:22:54.646 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:54.646 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:54.646 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.646 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:54.646 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:22:54.646 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.646 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.646 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:54.647 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:22:54.647 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.647 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.647 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.647 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:22:54.647 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:22:54.647 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:54.647 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:54.647 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:54.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.648 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:22:54.648 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:22:54.648 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:22:54.648 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:22:54.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.648 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:22:54.648 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:22:54.648 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:22:54.649 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:22:54.649 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:22:54.649 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.649 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.649 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:22:54.649 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:22:54.649 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:22:54.649 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:22:54.650 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:22:54.650 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.650 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.650 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:22:54.650 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:22:54.650 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.650 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.650 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.650 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:54.650 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.651 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.652 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:22:54.652 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.652 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.652 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:22:54.652 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.652 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:22:54.652 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.652 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:54.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.652 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.652 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.653 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.653 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:54.653 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.653 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:22:54.653 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:22:54.653 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:22:54.653 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:22:54.653 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:22:54.653 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:22:54.653 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:22:54.653 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:22:54.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:22:54.654 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:22:54.654 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:22:54.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.631+0000 7f7722e00640 1 -- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f76fc06b500 msgr2=0x7f76fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.631+0000 7f7722e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f76fc06b500 0x7f76fc06d9c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f771c06def0 tx=0x7f7708007e80 comp rx=0 tx=0).stop 2024-03-09T09:22:54.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.631+0000 7f7722e00640 1 -- 172.21.15.67:0/981536336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f771c1535e0 msgr2=0x7f771c14c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.631+0000 7f7722e00640 1 --2- 172.21.15.67:0/981536336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f771c1535e0 0x7f771c14c620 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7714007e10 tx=0x7f771400e4b0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.631+0000 7f7720e00640 1 -- 172.21.15.67:0/981536336 reap_dead start 2024-03-09T09:22:54.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.631+0000 7f7720e00640 1 -- 172.21.15.67:0/981536336 reap_dead start 2024-03-09T09:22:54.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.631+0000 7f7722e00640 1 -- 172.21.15.67:0/981536336 shutdown_connections 2024-03-09T09:22:54.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.631+0000 7f7722e00640 1 -- 172.21.15.67:0/981536336 >> 172.21.15.67:0/981536336 conn(0x7f771c0ab320 msgr2=0x7f771c0729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:54.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.631+0000 7f7722e00640 1 -- 172.21.15.67:0/981536336 shutdown_connections 2024-03-09T09:22:54.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.631+0000 7f7722e00640 1 -- 172.21.15.67:0/981536336 wait complete. 2024-03-09T09:22:54.656 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:22:54.657 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-03-09T09:22:54.657 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:22:54.657 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-03-09T09:22:54.657 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:22:54.657 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-09T09:22:54.657 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:22:54.681 INFO:tasks.ceph.mon.a:Stopped 2024-03-09T09:22:54.681 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:22:54.681 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:22:54.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.928+0000 7f9d35600640 1 Processor -- start 2024-03-09T09:22:54.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.928+0000 7f9d35600640 1 -- start start 2024-03-09T09:22:54.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.931+0000 7f9d35600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9d300cadb0 0x7f9d30156720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.931+0000 7f9d35600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d30157000 con 0x7f9d30153300 2024-03-09T09:22:54.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.931+0000 7f9d35600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d300ae430 con 0x7f9d300c5e40 2024-03-09T09:22:54.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.931+0000 7f9d35600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d300ae6f0 con 0x7f9d300cadb0 2024-03-09T09:22:54.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.931+0000 7f9d34c00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f9d300cadb0 msgr2=0x7f9d30156720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:54.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.931+0000 7f9d2fe00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f9d30153300 legacy=0x7f9d300b0000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:22:54.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.931+0000 7f9d34c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9d300cadb0 0x7f9d30156720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:54.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.931+0000 7f9d27e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f9d300c5e40 0x7f9d30156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:42622/0 (socket says 172.21.15.67:42622) 2024-03-09T09:22:54.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.931+0000 7f9d27e00640 1 -- 172.21.15.67:0/2813598157 learned_addr learned my addr 172.21.15.67:0/2813598157 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:22:54.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.932+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685359506 0 0) 0x7f9d300ae430 con 0x7f9d300c5e40 2024-03-09T09:22:54.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.932+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d14003520 con 0x7f9d300c5e40 2024-03-09T09:22:54.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3681962637 0 0) 0x7f9d14003520 con 0x7f9d300c5e40 2024-03-09T09:22:54.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 >> v2:172.21.15.67:3300/0 conn(0x7f9d300cadb0 msgr2=0x7f9d30156720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:54.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 --2- 172.21.15.67:0/2813598157 >> v2:172.21.15.67:3300/0 conn(0x7f9d300cadb0 0x7f9d30156720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 >> v1:172.21.15.67:6789/0 conn(0x7f9d30153300 legacy=0x7f9d300b0000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:54.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d30155b10 con 0x7f9d300c5e40 2024-03-09T09:22:54.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f9d18001670 con 0x7f9d300c5e40 2024-03-09T09:22:54.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 >> v1:172.21.15.134:6789/0 conn(0x7f9d300c5e40 legacy=0x7f9d30156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 --2- 172.21.15.67:0/2813598157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d14003a20 0x7f9d14003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 --2- 172.21.15.67:0/2813598157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d14004800 0x7f9d14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 --2- 172.21.15.67:0/2813598157 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d140052e0 0x7f9d1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300ae6f0 con 0x7f9d14003a20 2024-03-09T09:22:54.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1400c1e0 con 0x7f9d14004800 2024-03-09T09:22:54.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1400c3e0 con 0x7f9d140052e0 2024-03-09T09:22:54.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2fe00640 1 -- 172.21.15.67:0/2813598157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d14003a20 msgr2=0x7f9d14003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:54.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.933+0000 7f9d2fe00640 1 --2- 172.21.15.67:0/2813598157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d14003a20 0x7f9d14003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:54.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.934+0000 7f9d27e00640 1 --2- 172.21.15.67:0/2813598157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d14004800 0x7f9d14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.934+0000 7f9d34c00640 1 --2- 172.21.15.67:0/2813598157 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d140052e0 0x7f9d1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.934+0000 7f9d27e00640 1 -- 172.21.15.67:0/2813598157 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d140052e0 msgr2=0x7f9d1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:22:54.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.934+0000 7f9d27e00640 1 --2- 172.21.15.67:0/2813598157 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d140052e0 0x7f9d1400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.935+0000 7f9d27e00640 1 -- 172.21.15.67:0/2813598157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d14003a20 msgr2=0x7f9d14003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:22:54.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.935+0000 7f9d27e00640 1 --2- 172.21.15.67:0/2813598157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d14003a20 0x7f9d14003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.935+0000 7f9d27e00640 1 -- 172.21.15.67:0/2813598157 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d14003520 con 0x7f9d14004800 2024-03-09T09:22:54.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.935+0000 7f9d34c00640 1 -- 172.21.15.67:0/2813598157 reap_dead start 2024-03-09T09:22:54.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.936+0000 7f9d27e00640 1 --2- 172.21.15.67:0/2813598157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d14004800 0x7f9d14004c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800c110 tx=0x7f9d14005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eee43feddad9347 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.936+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18007c40 con 0x7f9d14004800 2024-03-09T09:22:54.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.936+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d18030070 con 0x7f9d14004800 2024-03-09T09:22:54.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.936+0000 7f9d2f400640 1 -- 172.21.15.67:0/2813598157 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18005d30 con 0x7f9d14004800 2024-03-09T09:22:54.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.937+0000 7f9d35600640 1 -- 172.21.15.67:0/2813598157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d14004800 msgr2=0x7f9d14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.937+0000 7f9d35600640 1 --2- 172.21.15.67:0/2813598157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d14004800 0x7f9d14004c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800c110 tx=0x7f9d14005830 comp rx=0 tx=0).stop 2024-03-09T09:22:54.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.937+0000 7f9d35600640 1 -- 172.21.15.67:0/2813598157 shutdown_connections 2024-03-09T09:22:54.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.937+0000 7f9d35600640 1 --2- 172.21.15.67:0/2813598157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d14004800 0x7f9d14004c80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.937+0000 7f9d35600640 1 -- 172.21.15.67:0/2813598157 >> 172.21.15.67:0/2813598157 conn(0x7f9d300732d0 msgr2=0x7f9d300ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:54.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.937+0000 7f9d35600640 1 -- 172.21.15.67:0/2813598157 shutdown_connections 2024-03-09T09:22:54.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.938+0000 7f9d35600640 1 -- 172.21.15.67:0/2813598157 wait complete. 2024-03-09T09:22:54.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.941+0000 7f9d35600640 1 Processor -- start 2024-03-09T09:22:54.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.941+0000 7f9d35600640 1 -- start start 2024-03-09T09:22:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.943+0000 7f9d35600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d14004800 0x7f9d3006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.943+0000 7f9d35600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d300aafa0 0x7f9d3006bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.943+0000 7f9d35600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d300c67a0 0x7f9d300c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.943+0000 7f9d35600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300c59e0 con 0x7f9d300c67a0 2024-03-09T09:22:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.943+0000 7f9d35600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30159e90 con 0x7f9d14004800 2024-03-09T09:22:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.943+0000 7f9d35600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300b0000 con 0x7f9d300aafa0 2024-03-09T09:22:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.943+0000 7f9d34c00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d300c67a0 msgr2=0x7f9d300c70d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:22:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.944+0000 7f9d34c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d300c67a0 0x7f9d300c70d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:22:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.944+0000 7f9d2fe00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d14004800 0x7f9d3006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.944+0000 7f9d27e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d300aafa0 0x7f9d3006bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.944+0000 7f9d27e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d300aafa0 0x7f9d3006bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:39832/0 (socket says 172.21.15.67:39832) 2024-03-09T09:22:54.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.944+0000 7f9d27e00640 1 -- 172.21.15.67:0/3084057185 learned_addr learned my addr 172.21.15.67:0/3084057185 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:22:54.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.945+0000 7f9d27e00640 1 -- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d14004800 msgr2=0x7f9d3006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:54.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.945+0000 7f9d27e00640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d14004800 0x7f9d3006b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.945+0000 7f9d27e00640 1 -- 172.21.15.67:0/3084057185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d300c67a0 msgr2=0x7f9d300c70d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:22:54.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.945+0000 7f9d27e00640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d300c67a0 0x7f9d300c70d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:54.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.945+0000 7f9d27e00640 1 -- 172.21.15.67:0/3084057185 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d301f03d0 con 0x7f9d300aafa0 2024-03-09T09:22:54.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.945+0000 7f9d27e00640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d300aafa0 0x7f9d3006bf20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800c230 tx=0x7f9d1802e070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.946+0000 7f9d2cc00640 1 -- 172.21.15.67:0/3084057185 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18007c40 con 0x7f9d300aafa0 2024-03-09T09:22:54.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.946+0000 7f9d2cc00640 1 -- 172.21.15.67:0/3084057185 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d18004bf0 con 0x7f9d300aafa0 2024-03-09T09:22:54.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.946+0000 7f9d35600640 1 -- 172.21.15.67:0/3084057185 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d301f0690 con 0x7f9d300aafa0 2024-03-09T09:22:54.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.946+0000 7f9d2cc00640 1 -- 172.21.15.67:0/3084057185 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18007070 con 0x7f9d300aafa0 2024-03-09T09:22:54.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.946+0000 7f9d35600640 1 -- 172.21.15.67:0/3084057185 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3006b4c0 con 0x7f9d300aafa0 2024-03-09T09:22:54.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.947+0000 7f9d35600640 1 -- 172.21.15.67:0/3084057185 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3004f3c0 con 0x7f9d300aafa0 2024-03-09T09:22:54.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.949+0000 7f9d2cc00640 1 -- 172.21.15.67:0/3084057185 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9d18007210 con 0x7f9d300aafa0 2024-03-09T09:22:54.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.950+0000 7f9d2cc00640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d0c06b500 0x7f9d0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:22:54.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.950+0000 7f9d2cc00640 1 -- 172.21.15.67:0/3084057185 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f9d180ac350 con 0x7f9d300aafa0 2024-03-09T09:22:54.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.951+0000 7f9d2fe00640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d0c06b500 0x7f9d0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:22:54.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.952+0000 7f9d2fe00640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d0c06b500 0x7f9d0c06d9c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9d200093b0 tx=0x7f9d20009340 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:22:54.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:54.964+0000 7f9d2cc00640 1 -- 172.21.15.67:0/3084057185 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9d1807af50 con 0x7f9d300aafa0 2024-03-09T09:22:55.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.251+0000 7f9d35600640 1 -- 172.21.15.67:0/3084057185 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9d3004f3c0 con 0x7f9d300aafa0 2024-03-09T09:22:55.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.252+0000 7f9d2cc00640 1 -- 172.21.15.67:0/3084057185 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f9d1807a850 con 0x7f9d300aafa0 2024-03-09T09:22:55.258 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":236,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:22:55.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 -- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d0c06b500 msgr2=0x7f9d0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:55.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d0c06b500 0x7f9d0c06d9c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9d200093b0 tx=0x7f9d20009340 comp rx=0 tx=0).stop 2024-03-09T09:22:55.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 -- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d300aafa0 msgr2=0x7f9d3006bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:22:55.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d300aafa0 0x7f9d3006bf20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800c230 tx=0x7f9d1802e070 comp rx=0 tx=0).stop 2024-03-09T09:22:55.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 -- 172.21.15.67:0/3084057185 shutdown_connections 2024-03-09T09:22:55.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f9d0c06b500 0x7f9d0c06d9c0 secure :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9d200093b0 tx=0x7f9d20009340 comp rx=0 tx=0).stop 2024-03-09T09:22:55.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9d300c67a0 0x7f9d300c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:55.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9d300aafa0 0x7f9d3006bf20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:55.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 --2- 172.21.15.67:0/3084057185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d14004800 0x7f9d3006b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:22:55.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 -- 172.21.15.67:0/3084057185 >> 172.21.15.67:0/3084057185 conn(0x7f9d300732d0 msgr2=0x7f9d301585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:22:55.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 -- 172.21.15.67:0/3084057185 shutdown_connections 2024-03-09T09:22:55.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:55.255+0000 7f9d35600640 1 -- 172.21.15.67:0/3084057185 wait complete. 2024-03-09T09:22:56.045 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:22:56.045 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:22:56.045 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 74 -eq 48 ']' 2024-03-09T09:22:56.045 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.74 2024-03-09T09:22:56.046 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.74 2024-03-09T09:22:56.046 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.74 2024-03-09T09:22:56.046 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.74 2024-03-09T09:22:56.319 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:22:58.281 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:22:58.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.689+0000 7f963dc00640 1 Processor -- start 2024-03-09T09:23:00.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.689+0000 7f963dc00640 1 -- start start 2024-03-09T09:23:00.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:23:00.630 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:23:00.630 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:23:00.630 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 68 -eq 48 ']' 2024-03-09T09:23:00.630 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.68 2024-03-09T09:23:00.631 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.68 2024-03-09T09:23:00.632 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.68 2024-03-09T09:23:00.632 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.68 2024-03-09T09:23:00.632 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:23:00.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.691+0000 7f963dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96380c2d10 0x7f9638155f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:00.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.691+0000 7f963dc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963806f1b0 con 0x7f96380ab3c0 2024-03-09T09:23:00.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.691+0000 7f963dc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963806f490 con 0x7f96380c8510 2024-03-09T09:23:00.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.691+0000 7f963dc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9638153930 con 0x7f96380c2d10 2024-03-09T09:23:00.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.691+0000 7f963d200640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f96380ab3c0 legacy=0x7f96380acee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.692+0000 7f9637e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f96380c8510 0x7f9638153220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:42630/0 (socket says 172.21.15.67:42630) 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.692+0000 7f9637400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96380c2d10 0x7f9638155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.692+0000 7f9637e00640 1 -- 172.21.15.67:0/1592869702 learned_addr learned my addr 172.21.15.67:0/1592869702 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.692+0000 7f9636a00640 1 -- 172.21.15.67:0/1592869702 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764301236 0 0) 0x7f963806f490 con 0x7f96380c8510 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.692+0000 7f9636a00640 1 -- 172.21.15.67:0/1592869702 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f961c005730 con 0x7f96380c8510 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.692+0000 7f9637400640 1 -- 172.21.15.67:0/1592869702 >> v1:172.21.15.134:6789/0 conn(0x7f96380c8510 legacy=0x7f9638153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.693+0000 7f9637400640 1 -- 172.21.15.67:0/1592869702 >> v1:172.21.15.67:6789/0 conn(0x7f96380ab3c0 legacy=0x7f96380acee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.693+0000 7f9637400640 1 -- 172.21.15.67:0/1592869702 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963806f770 con 0x7f96380c2d10 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.694+0000 7f9637400640 1 --2- 172.21.15.67:0/1592869702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96380c2d10 0x7f9638155f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f96240097d0 tx=0x7f962402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e427d3d454ec7372 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.694+0000 7f9636a00640 1 -- 172.21.15.67:0/1592869702 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624035020 con 0x7f96380c2d10 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.694+0000 7f9636a00640 1 -- 172.21.15.67:0/1592869702 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9624031800 con 0x7f96380c2d10 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.694+0000 7f9636a00640 1 -- 172.21.15.67:0/1592869702 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624031bb0 con 0x7f96380c2d10 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.695+0000 7f963dc00640 1 -- 172.21.15.67:0/1592869702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96380c2d10 msgr2=0x7f9638155f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:00.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.695+0000 7f963dc00640 1 --2- 172.21.15.67:0/1592869702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96380c2d10 0x7f9638155f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f96240097d0 tx=0x7f962402b8a0 comp rx=0 tx=0).stop 2024-03-09T09:23:00.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.695+0000 7f963dc00640 1 -- 172.21.15.67:0/1592869702 shutdown_connections 2024-03-09T09:23:00.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.695+0000 7f963dc00640 1 --2- 172.21.15.67:0/1592869702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96380c2d10 0x7f9638155f00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:00.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.695+0000 7f963dc00640 1 -- 172.21.15.67:0/1592869702 >> 172.21.15.67:0/1592869702 conn(0x7f963806d650 msgr2=0x7f963806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:00.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.695+0000 7f963dc00640 1 -- 172.21.15.67:0/1592869702 shutdown_connections 2024-03-09T09:23:00.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.695+0000 7f963dc00640 1 -- 172.21.15.67:0/1592869702 wait complete. 2024-03-09T09:23:00.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.697+0000 7f963dc00640 1 Processor -- start 2024-03-09T09:23:00.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.697+0000 7f963dc00640 1 -- start start 2024-03-09T09:23:00.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.698+0000 7f963dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96380ab3c0 0x7f9638146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:00.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.698+0000 7f963dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96380c2d10 0x7f9638147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:00.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.698+0000 7f963dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96380c8510 0x7f96381eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.698+0000 7f963dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963806edd0 con 0x7f96380c8510 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.698+0000 7f963dc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9638152d00 con 0x7f96380c2d10 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.698+0000 7f963dc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963806ef50 con 0x7f96380ab3c0 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.698+0000 7f963d200640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96380c8510 msgr2=0x7f96381eeb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.698+0000 7f963d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96380c8510 0x7f96381eeb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.698+0000 7f9637400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96380c2d10 0x7f9638147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.699+0000 7f9637e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96380ab3c0 0x7f9638146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.699+0000 7f9637400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96380c2d10 0x7f9638147410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:46174/0 (socket says 172.21.15.67:46174) 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.699+0000 7f9637400640 1 -- 172.21.15.67:0/225870856 learned_addr learned my addr 172.21.15.67:0/225870856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.699+0000 7f9637400640 1 -- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96380ab3c0 msgr2=0x7f9638146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.699+0000 7f9637400640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96380ab3c0 0x7f9638146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:00.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.699+0000 7f9637400640 1 -- 172.21.15.67:0/225870856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96380c8510 msgr2=0x7f96381eeb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.699+0000 7f9637400640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96380c8510 0x7f96381eeb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.699+0000 7f9637400640 1 -- 172.21.15.67:0/225870856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9638073470 con 0x7f96380c2d10 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.700+0000 7f9637400640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96380c2d10 0x7f9638147410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f96240097d0 tx=0x7f96240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.700+0000 7f962fe00640 1 -- 172.21.15.67:0/225870856 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624049070 con 0x7f96380c2d10 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.700+0000 7f962fe00640 1 -- 172.21.15.67:0/225870856 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9624003820 con 0x7f96380c2d10 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.700+0000 7f962fe00640 1 -- 172.21.15.67:0/225870856 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624003bf0 con 0x7f96380c2d10 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.700+0000 7f963dc00640 1 -- 172.21.15.67:0/225870856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9638151d50 con 0x7f96380c2d10 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.700+0000 7f963dc00640 1 -- 172.21.15.67:0/225870856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9638152010 con 0x7f96380c2d10 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.702+0000 7f963dc00640 1 -- 172.21.15.67:0/225870856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9638150ff0 con 0x7f96380c2d10 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.703+0000 7f962fe00640 1 -- 172.21.15.67:0/225870856 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9624038020 con 0x7f96380c2d10 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.704+0000 7f962fe00640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f961806b480 0x7f961806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.704+0000 7f962fe00640 1 -- 172.21.15.67:0/225870856 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f96240ac4b0 con 0x7f96380c2d10 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.705+0000 7f9637e00640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f961806b480 0x7f961806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:00.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.706+0000 7f9637e00640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f961806b480 0x7f961806d940 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f96280011e0 tx=0x7f9628007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:00.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:58.717+0000 7f962fe00640 1 -- 172.21.15.67:0/225870856 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9624052080 con 0x7f96380c2d10 2024-03-09T09:23:00.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.023+0000 7f963dc00640 1 -- 172.21.15.67:0/225870856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f96380c7d20 con 0x7f96380c2d10 2024-03-09T09:23:00.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.024+0000 7f962fe00640 1 -- 172.21.15.67:0/225870856 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f96380c7d20 con 0x7f96380c2d10 2024-03-09T09:23:00.641 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":236,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:23:00.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 -- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f961806b480 msgr2=0x7f961806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:00.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f961806b480 0x7f961806d940 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f96280011e0 tx=0x7f9628007d30 comp rx=0 tx=0).stop 2024-03-09T09:23:00.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 -- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96380c2d10 msgr2=0x7f9638147410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:00.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96380c2d10 0x7f9638147410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f96240097d0 tx=0x7f96240046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:00.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 -- 172.21.15.67:0/225870856 shutdown_connections 2024-03-09T09:23:00.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f961806b480 0x7f961806d940 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:00.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96380c8510 0x7f96381eeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:00.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96380c2d10 0x7f9638147410 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:00.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 --2- 172.21.15.67:0/225870856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96380ab3c0 0x7f9638146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:00.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 -- 172.21.15.67:0/225870856 >> 172.21.15.67:0/225870856 conn(0x7f963806d650 msgr2=0x7f96380c7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:00.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 -- 172.21.15.67:0/225870856 shutdown_connections 2024-03-09T09:23:00.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:22:59.027+0000 7f963dc00640 1 -- 172.21.15.67:0/225870856 wait complete. 2024-03-09T09:23:00.691 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:23:01.689 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:23:01.689 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:23:01.689 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 75 -eq 48 ']' 2024-03-09T09:23:01.690 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.75 2024-03-09T09:23:01.690 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.75 2024-03-09T09:23:01.690 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.75 2024-03-09T09:23:01.690 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.75 2024-03-09T09:23:01.695 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:23:03.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:23:03.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.890+0000 7f0454a00640 1 Processor -- start 2024-03-09T09:23:03.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.890+0000 7f0454a00640 1 -- start start 2024-03-09T09:23:03.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.891+0000 7f0454a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f045014dbf0 0x7f04500c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.892+0000 7f0454a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f04500c5aa0 0x7f0450153c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.892+0000 7f0454a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0450157b10 con 0x7f045006f9e0 2024-03-09T09:23:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.892+0000 7f0454a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04500c36a0 con 0x7f045014dbf0 2024-03-09T09:23:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.892+0000 7f0454a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04500c3980 con 0x7f04500c5aa0 2024-03-09T09:23:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.892+0000 7f044f400640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f045014dbf0 msgr2=0x7f04500c4da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.892+0000 7f044f400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f045014dbf0 0x7f04500c4da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.892+0000 7f044ea00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f04500c5aa0 0x7f0450153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.892+0000 7f044ea00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f04500c5aa0 0x7f0450153c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:52990/0 (socket says 172.21.15.67:52990) 2024-03-09T09:23:03.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.892+0000 7f044ea00640 1 -- 172.21.15.67:0/1916324706 learned_addr learned my addr 172.21.15.67:0/1916324706 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:03.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.893+0000 7f044ea00640 1 -- 172.21.15.67:0/1916324706 >> v2:172.21.15.67:3300/0 conn(0x7f045014dbf0 msgr2=0x7f04500c4da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:03.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.893+0000 7f044ea00640 1 --2- 172.21.15.67:0/1916324706 >> v2:172.21.15.67:3300/0 conn(0x7f045014dbf0 0x7f04500c4da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:03.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.893+0000 7f044ea00640 1 -- 172.21.15.67:0/1916324706 >> v1:172.21.15.134:6789/0 conn(0x7f045006f9e0 legacy=0x7f0450154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:03.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.893+0000 7f044ea00640 1 -- 172.21.15.67:0/1916324706 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0450154850 con 0x7f04500c5aa0 2024-03-09T09:23:03.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.894+0000 7f044ea00640 1 --2- 172.21.15.67:0/1916324706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04500c5aa0 0x7f0450153c00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f04500ada90 tx=0x7f043c02b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8edac8f840175dc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:03.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.895+0000 7f044e000640 1 -- 172.21.15.67:0/1916324706 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c035020 con 0x7f04500c5aa0 2024-03-09T09:23:03.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.895+0000 7f044e000640 1 -- 172.21.15.67:0/1916324706 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c031800 con 0x7f04500c5aa0 2024-03-09T09:23:03.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.895+0000 7f044e000640 1 -- 172.21.15.67:0/1916324706 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c031bd0 con 0x7f04500c5aa0 2024-03-09T09:23:03.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.896+0000 7f0454a00640 1 -- 172.21.15.67:0/1916324706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04500c5aa0 msgr2=0x7f0450153c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:03.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.896+0000 7f0454a00640 1 --2- 172.21.15.67:0/1916324706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04500c5aa0 0x7f0450153c00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f04500ada90 tx=0x7f043c02b8a0 comp rx=0 tx=0).stop 2024-03-09T09:23:03.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.896+0000 7f0454a00640 1 -- 172.21.15.67:0/1916324706 shutdown_connections 2024-03-09T09:23:03.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.896+0000 7f0454a00640 1 --2- 172.21.15.67:0/1916324706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04500c5aa0 0x7f0450153c00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:03.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.896+0000 7f0454a00640 1 --2- 172.21.15.67:0/1916324706 >> v2:172.21.15.67:3300/0 conn(0x7f045014dbf0 0x7f04500c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:03.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.896+0000 7f0454a00640 1 -- 172.21.15.67:0/1916324706 >> 172.21.15.67:0/1916324706 conn(0x7f04500c8310 msgr2=0x7f04500c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:03.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.896+0000 7f0454a00640 1 -- 172.21.15.67:0/1916324706 shutdown_connections 2024-03-09T09:23:03.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.896+0000 7f0454a00640 1 -- 172.21.15.67:0/1916324706 wait complete. 2024-03-09T09:23:03.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.898+0000 7f0454a00640 1 Processor -- start 2024-03-09T09:23:03.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.898+0000 7f0454a00640 1 -- start start 2024-03-09T09:23:03.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.898+0000 7f0454a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045006f9e0 0x7f0450155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.899+0000 7f0454a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f045014dbf0 0x7f045014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.899+0000 7f0454a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f045014dfd0 0x7f0450150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.899+0000 7f0454a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04500c2e90 con 0x7f045014dbf0 2024-03-09T09:23:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.899+0000 7f0454a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04500c2d10 con 0x7f045006f9e0 2024-03-09T09:23:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.899+0000 7f0454a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04500c52e0 con 0x7f045014dfd0 2024-03-09T09:23:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.899+0000 7f044ea00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f045014dbf0 msgr2=0x7f045014fff0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.899+0000 7f044ea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f045014dbf0 0x7f045014fff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.899+0000 7f044f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045006f9e0 0x7f0450155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.899+0000 7f044f400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045006f9e0 0x7f0450155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:53004/0 (socket says 172.21.15.67:53004) 2024-03-09T09:23:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.899+0000 7f044f400640 1 -- 172.21.15.67:0/754580884 learned_addr learned my addr 172.21.15.67:0/754580884 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:03.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.899+0000 7f044fe00640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f045014dfd0 0x7f0450150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:03.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.900+0000 7f044f400640 1 -- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f045014dfd0 msgr2=0x7f0450150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:03.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.900+0000 7f044f400640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f045014dfd0 0x7f0450150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:03.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.900+0000 7f044f400640 1 -- 172.21.15.67:0/754580884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f045014dbf0 msgr2=0x7f045014fff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:23:03.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.900+0000 7f044f400640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f045014dbf0 0x7f045014fff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:03.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.900+0000 7f044f400640 1 -- 172.21.15.67:0/754580884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0450153230 con 0x7f045006f9e0 2024-03-09T09:23:03.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.900+0000 7f044f400640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045006f9e0 0x7f0450155e60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f044000b550 tx=0x7f044000ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:03.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.901+0000 7f0447400640 1 -- 172.21.15.67:0/754580884 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440013020 con 0x7f045006f9e0 2024-03-09T09:23:03.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.901+0000 7f0447400640 1 -- 172.21.15.67:0/754580884 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0440003de0 con 0x7f045006f9e0 2024-03-09T09:23:03.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.901+0000 7f0447400640 1 -- 172.21.15.67:0/754580884 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04400052b0 con 0x7f045006f9e0 2024-03-09T09:23:03.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.901+0000 7f0454a00640 1 -- 172.21.15.67:0/754580884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04501ec0f0 con 0x7f045006f9e0 2024-03-09T09:23:03.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.901+0000 7f0454a00640 1 -- 172.21.15.67:0/754580884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04501563a0 con 0x7f045006f9e0 2024-03-09T09:23:03.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.903+0000 7f0454a00640 1 -- 172.21.15.67:0/754580884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045004f3c0 con 0x7f045006f9e0 2024-03-09T09:23:03.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.904+0000 7f0447400640 1 -- 172.21.15.67:0/754580884 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0440016020 con 0x7f045006f9e0 2024-03-09T09:23:03.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.905+0000 7f0447400640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f043006b480 0x7f043006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:03.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.905+0000 7f0447400640 1 -- 172.21.15.67:0/754580884 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f044008a3c0 con 0x7f045006f9e0 2024-03-09T09:23:03.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.905+0000 7f044ea00640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f043006b480 0x7f043006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:03.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.907+0000 7f044ea00640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f043006b480 0x7f043006d940 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f043c036000 tx=0x7f043c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:03.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:03.925+0000 7f0447400640 1 -- 172.21.15.67:0/754580884 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f044001d020 con 0x7f045006f9e0 2024-03-09T09:23:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.214+0000 7f0454a00640 1 -- 172.21.15.67:0/754580884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f04500008d0 con 0x7f045006f9e0 2024-03-09T09:23:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.215+0000 7f0447400640 1 -- 172.21.15.67:0/754580884 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f0440058e40 con 0x7f045006f9e0 2024-03-09T09:23:04.222 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":236,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 -- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f043006b480 msgr2=0x7f043006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f043006b480 0x7f043006d940 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f043c036000 tx=0x7f043c008030 comp rx=0 tx=0).stop 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 -- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045006f9e0 msgr2=0x7f0450155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045006f9e0 0x7f0450155e60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f044000b550 tx=0x7f044000ba10 comp rx=0 tx=0).stop 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 -- 172.21.15.67:0/754580884 shutdown_connections 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f043006b480 0x7f043006d940 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f045014dfd0 0x7f0450150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f045014dbf0 0x7f045014fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 --2- 172.21.15.67:0/754580884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f045006f9e0 0x7f0450155e60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 -- 172.21.15.67:0/754580884 >> 172.21.15.67:0/754580884 conn(0x7f04500c8310 msgr2=0x7f045006e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 -- 172.21.15.67:0/754580884 shutdown_connections 2024-03-09T09:23:04.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:04.219+0000 7f0454a00640 1 -- 172.21.15.67:0/754580884 wait complete. 2024-03-09T09:23:06.060 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:23:06.060 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:23:06.060 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 69 -eq 48 ']' 2024-03-09T09:23:06.060 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.69 2024-03-09T09:23:06.060 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.69 2024-03-09T09:23:06.060 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.69 2024-03-09T09:23:06.061 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.69 2024-03-09T09:23:06.065 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:23:07.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:23:07.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.492+0000 7fa40dc00640 1 Processor -- start 2024-03-09T09:23:07.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.493+0000 7fa40dc00640 1 -- start start 2024-03-09T09:23:07.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.495+0000 7fa40dc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:07.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.495+0000 7fa40dc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa40806bd30 0x7fa4080c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:07.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.495+0000 7fa40dc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa408155c60 con 0x7fa40806f4f0 2024-03-09T09:23:07.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.495+0000 7fa40dc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa408155f40 con 0x7fa40806bd30 2024-03-09T09:23:07.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.495+0000 7fa40dc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa408156220 con 0x7fa4080c84e0 2024-03-09T09:23:07.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.495+0000 7fa40d200640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fa40806bd30 msgr2=0x7fa4080c1840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:07.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.495+0000 7fa40d200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa40806bd30 0x7fa4080c1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:07.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.495+0000 7fa407e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:07.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.495+0000 7fa407e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:53010/0 (socket says 172.21.15.67:53010) 2024-03-09T09:23:07.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.496+0000 7fa407e00640 1 -- 172.21.15.67:0/3798097496 learned_addr learned my addr 172.21.15.67:0/3798097496 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:07.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.496+0000 7fa407e00640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 msgr2=0x7fa40806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:23:07.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.496+0000 7fa407e00640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 msgr2=0x7fa40806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:07.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.496+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:07.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.496+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:07.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.696+0000 7fa40d200640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.67:3300/0 conn(0x7fa40806bd30 msgr2=0x7fa4080c1840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:07.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.696+0000 7fa40d200640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.67:3300/0 conn(0x7fa40806bd30 0x7fa4080c1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:23:07.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.697+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:07.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.698+0000 7fa407e00640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 msgr2=0x7fa40806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:23:07.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.698+0000 7fa407e00640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 msgr2=0x7fa40806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:07.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.698+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:07.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:07.698+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:23:07.955 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:23:07.955 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:23:07.955 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 76 -eq 48 ']' 2024-03-09T09:23:07.955 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.76 2024-03-09T09:23:07.955 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.76 2024-03-09T09:23:07.956 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.76 2024-03-09T09:23:07.956 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.76 2024-03-09T09:23:07.960 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:23:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.097+0000 7fa40d200640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.67:3300/0 conn(0x7fa40806bd30 msgr2=0x7fa4080c1840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.097+0000 7fa40d200640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.67:3300/0 conn(0x7fa40806bd30 0x7fa4080c1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:23:08.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.099+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:08.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.100+0000 7fa407e00640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 msgr2=0x7fa40806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:23:08.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.100+0000 7fa407e00640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 msgr2=0x7fa40806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:08.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.100+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:08.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.100+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:23:08.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.898+0000 7fa40d200640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.67:3300/0 conn(0x7fa40806bd30 msgr2=0x7fa4080c1840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:08.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.898+0000 7fa40d200640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.67:3300/0 conn(0x7fa40806bd30 0x7fa4080c1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:23:08.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.902+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:08.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.902+0000 7fa407e00640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 msgr2=0x7fa40806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:23:08.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.902+0000 7fa407e00640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 msgr2=0x7fa40806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:08.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.902+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:08.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:08.902+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:23:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.659+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278271454 0 0) 0x7fa408155c60 con 0x7fa40806f4f0 2024-03-09T09:23:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.660+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3ec003520 con 0x7fa40806f4f0 2024-03-09T09:23:09.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.668+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 99503720 0 0) 0x7fa3ec003520 con 0x7fa40806f4f0 2024-03-09T09:23:09.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.668+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 msgr2=0x7fa40806b7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:09.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.668+0000 7fa407400640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.134:3300/0 conn(0x7fa4080c84e0 0x7fa40806b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:09.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.668+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 >> v2:172.21.15.67:3300/0 conn(0x7fa40806bd30 msgr2=0x7fa4080c1840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:09.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.668+0000 7fa407400640 1 --2- 172.21.15.67:0/3798097496 >> v2:172.21.15.67:3300/0 conn(0x7fa40806bd30 0x7fa4080c1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:09.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.668+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408156500 con 0x7fa40806f4f0 2024-03-09T09:23:09.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.668+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fa3f0001650 con 0x7fa40806f4f0 2024-03-09T09:23:09.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.668+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 >> v1:172.21.15.134:6789/0 conn(0x7fa40806f4f0 legacy=0x7fa4080c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:09.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.669+0000 7fa407400640 1 --2- 172.21.15.67:0/3798097496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3ec003a20 0x7fa3ec003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:09.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.669+0000 7fa407400640 1 --2- 172.21.15.67:0/3798097496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ec004800 0x7fa3ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:09.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.669+0000 7fa407400640 1 --2- 172.21.15.67:0/3798097496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3ec005210 0x7fa3ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:09.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.669+0000 7fa407e00640 1 -- 172.21.15.67:0/3798097496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3ec003a20 msgr2=0x7fa3ec003e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:09.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.669+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa408155f40 con 0x7fa3ec003a20 2024-03-09T09:23:09.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.669+0000 7fa407e00640 1 --2- 172.21.15.67:0/3798097496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3ec003a20 0x7fa3ec003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:09.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.669+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa408156220 con 0x7fa3ec004800 2024-03-09T09:23:09.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.669+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec00c1c0 con 0x7fa3ec005210 2024-03-09T09:23:09.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.670+0000 7fa40d200640 1 --2- 172.21.15.67:0/3798097496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3ec005210 0x7fa3ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:09.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.670+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/3798097496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ec004800 0x7fa3ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:09.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.671+0000 7fa40d200640 1 -- 172.21.15.67:0/3798097496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ec004800 msgr2=0x7fa3ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:09.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.671+0000 7fa40d200640 1 --2- 172.21.15.67:0/3798097496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ec004800 0x7fa3ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:09.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.671+0000 7fa40d200640 1 -- 172.21.15.67:0/3798097496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3ec003a20 msgr2=0x7fa3ec003e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:09.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.671+0000 7fa40d200640 1 --2- 172.21.15.67:0/3798097496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3ec003a20 0x7fa3ec003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:09.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.671+0000 7fa40d200640 1 -- 172.21.15.67:0/3798097496 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec003520 con 0x7fa3ec005210 2024-03-09T09:23:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.672+0000 7fa40d200640 1 -- 172.21.15.67:0/3798097496 reap_dead start 2024-03-09T09:23:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.673+0000 7fa40d200640 1 --2- 172.21.15.67:0/3798097496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3ec005210 0x7fa3ec00b9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa400003d00 tx=0x7fa3ec005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=354189ec6e869dca server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:09.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.675+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400038070 con 0x7fa3ec005210 2024-03-09T09:23:09.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.675+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa400033640 con 0x7fa3ec005210 2024-03-09T09:23:09.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.675+0000 7fa407400640 1 -- 172.21.15.67:0/3798097496 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400007c00 con 0x7fa3ec005210 2024-03-09T09:23:09.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.677+0000 7fa40dc00640 1 -- 172.21.15.67:0/3798097496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3ec005210 msgr2=0x7fa3ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:09.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.677+0000 7fa40dc00640 1 --2- 172.21.15.67:0/3798097496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3ec005210 0x7fa3ec00b9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa400003d00 tx=0x7fa3ec005830 comp rx=0 tx=0).stop 2024-03-09T09:23:09.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.677+0000 7fa40dc00640 1 -- 172.21.15.67:0/3798097496 shutdown_connections 2024-03-09T09:23:09.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.677+0000 7fa40dc00640 1 --2- 172.21.15.67:0/3798097496 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3ec005210 0x7fa3ec00b9c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:09.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.677+0000 7fa40dc00640 1 -- 172.21.15.67:0/3798097496 >> 172.21.15.67:0/3798097496 conn(0x7fa40806d650 msgr2=0x7fa40806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:09.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.678+0000 7fa40dc00640 1 -- 172.21.15.67:0/3798097496 shutdown_connections 2024-03-09T09:23:09.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.678+0000 7fa40dc00640 1 -- 172.21.15.67:0/3798097496 wait complete. 2024-03-09T09:23:09.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.682+0000 7fa40dc00640 1 Processor -- start 2024-03-09T09:23:09.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.682+0000 7fa40dc00640 1 -- start start 2024-03-09T09:23:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.685+0000 7fa40dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ec005210 0x7fa4081ee600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.685+0000 7fa40dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa408159d60 0x7fa4081589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.685+0000 7fa40dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa408156f40 0x7fa408158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.685+0000 7fa40dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40806b7f0 con 0x7fa408156f40 2024-03-09T09:23:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.685+0000 7fa40dc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4080c8ad0 con 0x7fa3ec005210 2024-03-09T09:23:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.685+0000 7fa40dc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40806b970 con 0x7fa408159d60 2024-03-09T09:23:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.686+0000 7fa40d200640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa408156f40 msgr2=0x7fa408158f30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:09.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.686+0000 7fa40d200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa408156f40 0x7fa408158f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:09.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.686+0000 7fa407e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ec005210 0x7fa4081ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:09.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.686+0000 7fa407e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ec005210 0x7fa4081ee600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:60698/0 (socket says 172.21.15.67:60698) 2024-03-09T09:23:09.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.686+0000 7fa407e00640 1 -- 172.21.15.67:0/1989456678 learned_addr learned my addr 172.21.15.67:0/1989456678 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:09.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.686+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa408159d60 0x7fa4081589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:09.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.687+0000 7fa407e00640 1 -- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa408159d60 msgr2=0x7fa4081589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:09.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.687+0000 7fa407e00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa408159d60 0x7fa4081589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.687+0000 7fa407e00640 1 -- 172.21.15.67:0/1989456678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa408156f40 msgr2=0x7fa408158f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:23:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.687+0000 7fa407e00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa408156f40 0x7fa408158f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.687+0000 7fa407e00640 1 -- 172.21.15.67:0/1989456678 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4081eef20 con 0x7fa3ec005210 2024-03-09T09:23:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.688+0000 7fa407e00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ec005210 0x7fa4081ee600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80146e0 tx=0x7fa3f8014ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.688+0000 7fa404c00640 1 -- 172.21.15.67:0/1989456678 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f801b030 con 0x7fa3ec005210 2024-03-09T09:23:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.688+0000 7fa404c00640 1 -- 172.21.15.67:0/1989456678 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f8010730 con 0x7fa3ec005210 2024-03-09T09:23:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.688+0000 7fa404c00640 1 -- 172.21.15.67:0/1989456678 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800f040 con 0x7fa3ec005210 2024-03-09T09:23:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.689+0000 7fa40dc00640 1 -- 172.21.15.67:0/1989456678 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4081ef200 con 0x7fa3ec005210 2024-03-09T09:23:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.689+0000 7fa40dc00640 1 -- 172.21.15.67:0/1989456678 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4081581f0 con 0x7fa3ec005210 2024-03-09T09:23:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.690+0000 7fa40dc00640 1 -- 172.21.15.67:0/1989456678 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa408158520 con 0x7fa3ec005210 2024-03-09T09:23:09.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.692+0000 7fa404c00640 1 -- 172.21.15.67:0/1989456678 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa3f801e030 con 0x7fa3ec005210 2024-03-09T09:23:09.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.693+0000 7fa404c00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3e406b4b0 0x7fa3e406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:09.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.693+0000 7fa404c00640 1 -- 172.21.15.67:0/1989456678 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa3f808a940 con 0x7fa3ec005210 2024-03-09T09:23:09.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.693+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3e406b4b0 0x7fa3e406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:09.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.694+0000 7fa3ffe00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3e406b4b0 0x7fa3e406d970 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000fd0 tx=0x7fa3f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:09.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:09.701+0000 7fa404c00640 1 -- 172.21.15.67:0/1989456678 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa3f8059330 con 0x7fa3ec005210 2024-03-09T09:23:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.067+0000 7fa40dc00640 1 -- 172.21.15.67:0/1989456678 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa40804f3c0 con 0x7fa3ec005210 2024-03-09T09:23:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.068+0000 7fa404c00640 1 -- 172.21.15.67:0/1989456678 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fa3f8058c30 con 0x7fa3ec005210 2024-03-09T09:23:10.075 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":238,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:23:10.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 -- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3e406b4b0 msgr2=0x7fa3e406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3e406b4b0 0x7fa3e406d970 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000fd0 tx=0x7fa3f0009040 comp rx=0 tx=0).stop 2024-03-09T09:23:10.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 -- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ec005210 msgr2=0x7fa4081ee600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ec005210 0x7fa4081ee600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80146e0 tx=0x7fa3f8014ae0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 -- 172.21.15.67:0/1989456678 shutdown_connections 2024-03-09T09:23:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3e406b4b0 0x7fa3e406d970 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa408156f40 0x7fa408158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa408159d60 0x7fa4081589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 --2- 172.21.15.67:0/1989456678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3ec005210 0x7fa4081ee600 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 -- 172.21.15.67:0/1989456678 >> 172.21.15.67:0/1989456678 conn(0x7fa40806d650 msgr2=0x7fa4080729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 -- 172.21.15.67:0/1989456678 shutdown_connections 2024-03-09T09:23:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.071+0000 7fa40dc00640 1 -- 172.21.15.67:0/1989456678 wait complete. 2024-03-09T09:23:10.102 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:23:10.102 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:23:10.102 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:23:10.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.369+0000 7f7b8de00640 1 Processor -- start 2024-03-09T09:23:10.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.369+0000 7f7b8de00640 1 -- start start 2024-03-09T09:23:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.371+0000 7f7b8de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7b8814fd90 0x7f7b88156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.371+0000 7f7b8de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880c9380 0x7f7b880acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.371+0000 7f7b8de00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b88156bc0 con 0x7f7b880ad440 2024-03-09T09:23:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.371+0000 7f7b8de00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8806ba60 con 0x7f7b8814fd90 2024-03-09T09:23:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.371+0000 7f7b8de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880c9890 con 0x7f7b880c9380 2024-03-09T09:23:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.371+0000 7f7b8ca00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f7b8814fd90 msgr2=0x7f7b88156680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.371+0000 7f7b8ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7b8814fd90 0x7f7b88156680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.371+0000 7f7b8d400640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f7b880ad440 legacy=0x7f7b8806c1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:23:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.372+0000 7f7b87e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880c9380 0x7f7b880acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.372+0000 7f7b87e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880c9380 0x7f7b880acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:53144/0 (socket says 172.21.15.67:53144) 2024-03-09T09:23:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.372+0000 7f7b87e00640 1 -- 172.21.15.67:0/2570643626 learned_addr learned my addr 172.21.15.67:0/2570643626 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:10.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.373+0000 7f7b87e00640 1 -- 172.21.15.67:0/2570643626 >> v2:172.21.15.67:3300/0 conn(0x7f7b8814fd90 msgr2=0x7f7b88156680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:10.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.373+0000 7f7b87e00640 1 --2- 172.21.15.67:0/2570643626 >> v2:172.21.15.67:3300/0 conn(0x7f7b8814fd90 0x7f7b88156680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.373+0000 7f7b87e00640 1 -- 172.21.15.67:0/2570643626 >> v1:172.21.15.67:6789/0 conn(0x7f7b880ad440 legacy=0x7f7b8806c1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:10.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.373+0000 7f7b87e00640 1 -- 172.21.15.67:0/2570643626 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b880c5f80 con 0x7f7b880c9380 2024-03-09T09:23:10.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.374+0000 7f7b87e00640 1 --2- 172.21.15.67:0/2570643626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880c9380 0x7f7b880acf00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009e50 tx=0x7f7b7802bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec46da358e283876 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:10.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.375+0000 7f7b87400640 1 -- 172.21.15.67:0/2570643626 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7802c980 con 0x7f7b880c9380 2024-03-09T09:23:10.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.375+0000 7f7b87400640 1 -- 172.21.15.67:0/2570643626 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7802cb20 con 0x7f7b880c9380 2024-03-09T09:23:10.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.375+0000 7f7b87400640 1 -- 172.21.15.67:0/2570643626 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b780343f0 con 0x7f7b880c9380 2024-03-09T09:23:10.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.375+0000 7f7b8de00640 1 -- 172.21.15.67:0/2570643626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880c9380 msgr2=0x7f7b880acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.375+0000 7f7b8de00640 1 --2- 172.21.15.67:0/2570643626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880c9380 0x7f7b880acf00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009e50 tx=0x7f7b7802bde0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.376+0000 7f7b8de00640 1 -- 172.21.15.67:0/2570643626 shutdown_connections 2024-03-09T09:23:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.376+0000 7f7b8de00640 1 --2- 172.21.15.67:0/2570643626 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880c9380 0x7f7b880acf00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.376+0000 7f7b8de00640 1 --2- 172.21.15.67:0/2570643626 >> v2:172.21.15.67:3300/0 conn(0x7f7b8814fd90 0x7f7b88156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.376+0000 7f7b8de00640 1 -- 172.21.15.67:0/2570643626 >> 172.21.15.67:0/2570643626 conn(0x7f7b880cb480 msgr2=0x7f7b880cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.376+0000 7f7b8de00640 1 -- 172.21.15.67:0/2570643626 shutdown_connections 2024-03-09T09:23:10.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.376+0000 7f7b8de00640 1 -- 172.21.15.67:0/2570643626 wait complete. 2024-03-09T09:23:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.378+0000 7f7b8de00640 1 Processor -- start 2024-03-09T09:23:10.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.378+0000 7f7b8de00640 1 -- start start 2024-03-09T09:23:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.380+0000 7f7b8de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880ad440 0x7f7b881475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.380+0000 7f7b8de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b880c9380 0x7f7b8814c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.380+0000 7f7b8de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b8814fd90 0x7f7b881e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.380+0000 7f7b8de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8806bcb0 con 0x7f7b880c9380 2024-03-09T09:23:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.380+0000 7f7b8de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880c6520 con 0x7f7b8814fd90 2024-03-09T09:23:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.380+0000 7f7b8de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8806c8e0 con 0x7f7b880ad440 2024-03-09T09:23:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.380+0000 7f7b87e00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b880c9380 msgr2=0x7f7b8814c9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.380+0000 7f7b87e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b880c9380 0x7f7b8814c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.381+0000 7f7b8d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b8814fd90 0x7f7b881e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.381+0000 7f7b8ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880ad440 0x7f7b881475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.381+0000 7f7b8ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880ad440 0x7f7b881475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:53158/0 (socket says 172.21.15.67:53158) 2024-03-09T09:23:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.381+0000 7f7b8d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b8814fd90 0x7f7b881e15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:60700/0 (socket says 172.21.15.67:60700) 2024-03-09T09:23:10.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.381+0000 7f7b8ca00640 1 -- 172.21.15.67:0/3637545219 learned_addr learned my addr 172.21.15.67:0/3637545219 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.382+0000 7f7b8ca00640 1 -- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b8814fd90 msgr2=0x7f7b881e15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.382+0000 7f7b8ca00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b8814fd90 0x7f7b881e15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.382+0000 7f7b8ca00640 1 -- 172.21.15.67:0/3637545219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b880c9380 msgr2=0x7f7b8814c9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:23:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.382+0000 7f7b8ca00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b880c9380 0x7f7b8814c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.382+0000 7f7b8ca00640 1 -- 172.21.15.67:0/3637545219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b88155ac0 con 0x7f7b880ad440 2024-03-09T09:23:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.382+0000 7f7b8d400640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b8814fd90 0x7f7b881e15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:23:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.382+0000 7f7b8ca00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880ad440 0x7f7b881475c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000b550 tx=0x7f7b7000ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.383+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70013020 con 0x7f7b880ad440 2024-03-09T09:23:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.383+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b70003de0 con 0x7f7b880ad440 2024-03-09T09:23:10.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.383+0000 7f7b8de00640 1 -- 172.21.15.67:0/3637545219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b881f3ba0 con 0x7f7b880ad440 2024-03-09T09:23:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.383+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b700052b0 con 0x7f7b880ad440 2024-03-09T09:23:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.383+0000 7f7b8de00640 1 -- 172.21.15.67:0/3637545219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b881588a0 con 0x7f7b880ad440 2024-03-09T09:23:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.384+0000 7f7b8de00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b88158be0 0x7f7b881e1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.384+0000 7f7b8de00640 1 -- 172.21.15.67:0/3637545219 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b881e2030 con 0x7f7b88158be0 2024-03-09T09:23:10.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.384+0000 7f7b87e00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b88158be0 0x7f7b881e1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.385+0000 7f7b87e00640 1 -- 172.21.15.67:0/3637545219 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b8804f3c0 con 0x7f7b88158be0 2024-03-09T09:23:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.385+0000 7f7b87e00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b88158be0 0x7f7b881e1af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b881537d0 tx=0x7f7b78004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.386+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b780343f0 con 0x7f7b88158be0 2024-03-09T09:23:10.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.386+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7b70016020 con 0x7f7b880ad440 2024-03-09T09:23:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.388+0000 7f7b84c00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b6406b3d0 0x7f7b6406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.388+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f7b7008a350 con 0x7f7b880ad440 2024-03-09T09:23:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.388+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f7b78009af0 con 0x7f7b88158be0 2024-03-09T09:23:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.388+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b88158be0 msgr2=0x7f7b881e1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.388+0000 7f7b84c00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b88158be0 0x7f7b881e1af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b881537d0 tx=0x7f7b78004a40 comp rx=0 tx=0).stop 2024-03-09T09:23:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.388+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b88158be0 msgr2=0x7f7b881e1af0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.388+0000 7f7b84c00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b88158be0 0x7f7b881e1af0 secure :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b881537d0 tx=0x7f7b78004a40 comp rx=0 tx=0).stop 2024-03-09T09:23:10.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.388+0000 7f7b8d400640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b6406b3d0 0x7f7b6406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.390+0000 7f7b8d400640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b6406b3d0 0x7f7b6406d890 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:10.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.403+0000 7f7b8de00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b881590a0 0x7f7b88159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.403+0000 7f7b8de00640 1 -- 172.21.15.67:0/3637545219 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8814cf10 con 0x7f7b881590a0 2024-03-09T09:23:10.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.404+0000 7f7b87e00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b881590a0 0x7f7b88159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.404+0000 7f7b87e00640 1 -- 172.21.15.67:0/3637545219 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7b88073a30 con 0x7f7b881590a0 2024-03-09T09:23:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.405+0000 7f7b87e00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b881590a0 0x7f7b88159480 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009e50 tx=0x7f7b78009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.405+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78048b60 con 0x7f7b881590a0 2024-03-09T09:23:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.406+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7f7b78040470 con 0x7f7b881590a0 2024-03-09T09:23:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.406+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b881590a0 msgr2=0x7f7b88159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.406+0000 7f7b84c00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b881590a0 0x7f7b88159480 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009e50 tx=0x7f7b78009810 comp rx=0 tx=0).stop 2024-03-09T09:23:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.406+0000 7f7b84c00640 1 -- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b881590a0 msgr2=0x7f7b88159480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.406+0000 7f7b84c00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b881590a0 0x7f7b88159480 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009e50 tx=0x7f7b78009810 comp rx=0 tx=0).stop 2024-03-09T09:23:10.414 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:23:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:23:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:23:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:23:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 238, 2024-03-09T09:23:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:23:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:23:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:23:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:23:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:23:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:23:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:23:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:23:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:23:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:23:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:23:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:23:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:23:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:23:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:23:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:23:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:23:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:23:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:23:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:23:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:23:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:23:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:23:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:23:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:23:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:23:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:23:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:23:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:23:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:23:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:23:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:23:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:23:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:23:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:23:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:23:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:23:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:23:10.431 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:23:10.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.407+0000 7f7b8de00640 1 -- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b6406b3d0 msgr2=0x7f7b6406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.407+0000 7f7b8de00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7b6406b3d0 0x7f7b6406d890 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80009060 comp rx=0 tx=0).stop 2024-03-09T09:23:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.407+0000 7f7b8de00640 1 -- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880ad440 msgr2=0x7f7b881475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.407+0000 7f7b8de00640 1 --2- 172.21.15.67:0/3637545219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b880ad440 0x7f7b881475c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000b550 tx=0x7f7b7000ba10 comp rx=0 tx=0).stop 2024-03-09T09:23:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.407+0000 7f7b8d400640 1 -- 172.21.15.67:0/3637545219 reap_dead start 2024-03-09T09:23:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.407+0000 7f7b8d400640 1 -- 172.21.15.67:0/3637545219 reap_dead start 2024-03-09T09:23:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.407+0000 7f7b8de00640 1 -- 172.21.15.67:0/3637545219 shutdown_connections 2024-03-09T09:23:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.407+0000 7f7b8de00640 1 -- 172.21.15.67:0/3637545219 >> 172.21.15.67:0/3637545219 conn(0x7f7b880cb480 msgr2=0x7f7b880cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.407+0000 7f7b8de00640 1 -- 172.21.15.67:0/3637545219 shutdown_connections 2024-03-09T09:23:10.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.407+0000 7f7b8de00640 1 -- 172.21.15.67:0/3637545219 wait complete. 2024-03-09T09:23:10.433 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:23:10.433 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:23:10.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:23:10.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.646+0000 7ff1b8c00640 1 Processor -- start 2024-03-09T09:23:10.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.647+0000 7ff1b8c00640 1 -- start start 2024-03-09T09:23:10.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.648+0000 7ff1b8c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff1b40c5e00 0x7ff1b40acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.648+0000 7ff1b8c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b40c61e0 0x7ff1b40c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.648+0000 7ff1b8c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b40ad7c0 con 0x7ff1b4156660 2024-03-09T09:23:10.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.648+0000 7ff1b8c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b406d740 con 0x7ff1b40c5e00 2024-03-09T09:23:10.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.648+0000 7ff1b8c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40c6660 con 0x7ff1b40c61e0 2024-03-09T09:23:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.648+0000 7ff1b3400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff1b40c5e00 0x7ff1b40acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.649+0000 7ff1b2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b40c61e0 0x7ff1b40c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.649+0000 7ff1b3400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff1b40c5e00 0x7ff1b40acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:60732/0 (socket says 172.21.15.67:60732) 2024-03-09T09:23:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.649+0000 7ff1b3400640 1 -- 172.21.15.67:0/4277546418 learned_addr learned my addr 172.21.15.67:0/4277546418 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.649+0000 7ff1b3400640 1 -- 172.21.15.67:0/4277546418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b40c61e0 msgr2=0x7ff1b40c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.649+0000 7ff1b3400640 1 --2- 172.21.15.67:0/4277546418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b40c61e0 0x7ff1b40c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.649+0000 7ff1b3400640 1 -- 172.21.15.67:0/4277546418 >> v1:172.21.15.134:6789/0 conn(0x7ff1b4156660 legacy=0x7ff1b406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.649+0000 7ff1b3400640 1 -- 172.21.15.67:0/4277546418 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b4156af0 con 0x7ff1b40c5e00 2024-03-09T09:23:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.651+0000 7ff1b3400640 1 --2- 172.21.15.67:0/4277546418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b40c5e00 0x7ff1b40acf00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009870 tx=0x7ff1a402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=492df7d9ad1f4308 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:10.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.651+0000 7ff1b2000640 1 -- 172.21.15.67:0/4277546418 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4034070 con 0x7ff1b40c5e00 2024-03-09T09:23:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.651+0000 7ff1b2000640 1 -- 172.21.15.67:0/4277546418 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a4030840 con 0x7ff1b40c5e00 2024-03-09T09:23:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.651+0000 7ff1b2000640 1 -- 172.21.15.67:0/4277546418 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4030c10 con 0x7ff1b40c5e00 2024-03-09T09:23:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.652+0000 7ff1b8c00640 1 -- 172.21.15.67:0/4277546418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b40c5e00 msgr2=0x7ff1b40acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.652+0000 7ff1b8c00640 1 --2- 172.21.15.67:0/4277546418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b40c5e00 0x7ff1b40acf00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009870 tx=0x7ff1a402b980 comp rx=0 tx=0).stop 2024-03-09T09:23:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.652+0000 7ff1b8c00640 1 -- 172.21.15.67:0/4277546418 shutdown_connections 2024-03-09T09:23:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.652+0000 7ff1b8c00640 1 --2- 172.21.15.67:0/4277546418 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b40c61e0 0x7ff1b40c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.652+0000 7ff1b8c00640 1 --2- 172.21.15.67:0/4277546418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b40c5e00 0x7ff1b40acf00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.652+0000 7ff1b8c00640 1 -- 172.21.15.67:0/4277546418 >> 172.21.15.67:0/4277546418 conn(0x7ff1b40cb480 msgr2=0x7ff1b40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.652+0000 7ff1b8c00640 1 -- 172.21.15.67:0/4277546418 shutdown_connections 2024-03-09T09:23:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.652+0000 7ff1b8c00640 1 -- 172.21.15.67:0/4277546418 wait complete. 2024-03-09T09:23:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.654+0000 7ff1b8c00640 1 Processor -- start 2024-03-09T09:23:10.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.654+0000 7ff1b8c00640 1 -- start start 2024-03-09T09:23:10.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b8c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b40c5e00 0x7ff1b40c15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b8c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b40c61e0 0x7ff1b40c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1b4156660 0x7ff1b40c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b8c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b406bb00 con 0x7ff1b4156660 2024-03-09T09:23:10.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b8c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b4153680 con 0x7ff1b40c5e00 2024-03-09T09:23:10.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b8c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b406bde0 con 0x7ff1b40c61e0 2024-03-09T09:23:10.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b3e00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1b4156660 msgr2=0x7ff1b40c5120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:10.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b3e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1b4156660 0x7ff1b40c5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b40c61e0 0x7ff1b40c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b3400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b40c5e00 0x7ff1b40c15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b2a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b40c61e0 0x7ff1b40c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:53176/0 (socket says 172.21.15.67:53176) 2024-03-09T09:23:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.655+0000 7ff1b2a00640 1 -- 172.21.15.67:0/2764702939 learned_addr learned my addr 172.21.15.67:0/2764702939 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.656+0000 7ff1b3400640 1 -- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b40c61e0 msgr2=0x7ff1b40c4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.656+0000 7ff1b3400640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b40c61e0 0x7ff1b40c4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.656+0000 7ff1b3400640 1 -- 172.21.15.67:0/2764702939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1b4156660 msgr2=0x7ff1b40c5120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:23:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.656+0000 7ff1b3400640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1b4156660 0x7ff1b40c5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.656+0000 7ff1b3400640 1 -- 172.21.15.67:0/2764702939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b40bfb80 con 0x7ff1b40c5e00 2024-03-09T09:23:10.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.657+0000 7ff1b3400640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b40c5e00 0x7ff1b40c15b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4033000 tx=0x7ff1a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:10.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.657+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4034040 con 0x7ff1b40c5e00 2024-03-09T09:23:10.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.657+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a4039d90 con 0x7ff1b40c5e00 2024-03-09T09:23:10.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.657+0000 7ff1b8c00640 1 -- 172.21.15.67:0/2764702939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b40bfe60 con 0x7ff1b40c5e00 2024-03-09T09:23:10.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.657+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4043610 con 0x7ff1b40c5e00 2024-03-09T09:23:10.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.657+0000 7ff1b8c00640 1 -- 172.21.15.67:0/2764702939 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b40c0420 con 0x7ff1b40c5e00 2024-03-09T09:23:10.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.658+0000 7ff1b8c00640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b414fc20 0x7ff1b4150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.658+0000 7ff1b8c00640 1 -- 172.21.15.67:0/2764702939 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40c5660 con 0x7ff1b414fc20 2024-03-09T09:23:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.659+0000 7ff1b2a00640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b414fc20 0x7ff1b4150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.659+0000 7ff1b2a00640 1 -- 172.21.15.67:0/2764702939 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1b40bf930 con 0x7ff1b414fc20 2024-03-09T09:23:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.660+0000 7ff1b2a00640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b414fc20 0x7ff1b4150000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff1b406d110 tx=0x7ff1a0011e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:10.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.660+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff1a404a060 con 0x7ff1b40c5e00 2024-03-09T09:23:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.661+0000 7ff1ab400640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff19406b480 0x7ff19406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.661+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff1a40ac3a0 con 0x7ff1b40c5e00 2024-03-09T09:23:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.661+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0019070 con 0x7ff1b414fc20 2024-03-09T09:23:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.661+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff1a0004300 con 0x7ff1b414fc20 2024-03-09T09:23:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.661+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b414fc20 msgr2=0x7ff1b4150000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.661+0000 7ff1ab400640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b414fc20 0x7ff1b4150000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff1b406d110 tx=0x7ff1a0011e30 comp rx=0 tx=0).stop 2024-03-09T09:23:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.661+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b414fc20 msgr2=0x7ff1b4150000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.661+0000 7ff1ab400640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b414fc20 0x7ff1b4150000 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff1b406d110 tx=0x7ff1a0011e30 comp rx=0 tx=0).stop 2024-03-09T09:23:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.662+0000 7ff1b3e00640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff19406b480 0x7ff19406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.663+0000 7ff1b3e00640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff19406b480 0x7ff19406d940 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0013b0 tx=0x7ff1ac007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:10.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.668+0000 7ff1b8c00640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b41575b0 0x7ff1b4157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:10.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.669+0000 7ff1b8c00640 1 -- 172.21.15.67:0/2764702939 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40c5ad0 con 0x7ff1b41575b0 2024-03-09T09:23:10.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.669+0000 7ff1b2a00640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b41575b0 0x7ff1b4157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:10.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.670+0000 7ff1b2a00640 1 -- 172.21.15.67:0/2764702939 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff1b40c6ad0 con 0x7ff1b41575b0 2024-03-09T09:23:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.670+0000 7ff1b2a00640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b41575b0 0x7ff1b4157990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff1a001b000 tx=0x7ff1a00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.671+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0019040 con 0x7ff1b41575b0 2024-03-09T09:23:10.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.671+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5176 (secure 0 0 0) 0x7ff1a0003df0 con 0x7ff1b41575b0 2024-03-09T09:23:10.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.671+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b41575b0 msgr2=0x7ff1b4157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.671+0000 7ff1ab400640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b41575b0 0x7ff1b4157990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff1a001b000 tx=0x7ff1a00046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1ab400640 1 -- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b41575b0 msgr2=0x7ff1b4157990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:10.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1ab400640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1b41575b0 0x7ff1b4157990 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff1a001b000 tx=0x7ff1a00046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.679 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:23:10.679 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:23:10.679 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:23:10.679 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:23:10.679 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 238, 2024-03-09T09:23:10.679 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:23:10.680 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:23:10.680 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:23:10.680 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.680 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:23:10.680 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:10.680 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:23:10.680 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:23:10.680 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:10.680 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:10.680 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:23:10.681 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:10.682 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:10.682 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:10.682 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:10.682 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:10.682 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:10.682 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:10.682 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:10.682 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:10.682 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:10.682 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:10.683 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.683 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:23:10.683 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:23:10.683 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:23:10.683 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:23:10.683 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:23:10.683 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:23:10.683 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:10.683 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:10.683 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:23:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:23:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:23:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:23:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:23:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:23:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:23:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:23:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:10.684 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.685 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:23:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:23:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:23:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:23:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:10.686 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:23:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:23:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.687 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:23:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:23:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:23:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:23:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:10.688 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:23:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:23:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:23:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:23:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:23:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:23:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:10.690 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:23:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:23:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.691 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.692 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.693 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.694 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:23:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:10.695 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:23:10.695 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:23:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1b8c00640 1 -- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff19406b480 msgr2=0x7ff19406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1b8c00640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff19406b480 0x7ff19406d940 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0013b0 tx=0x7ff1ac007d30 comp rx=0 tx=0).stop 2024-03-09T09:23:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1b8c00640 1 -- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b40c5e00 msgr2=0x7ff1b40c15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1b8c00640 1 --2- 172.21.15.67:0/2764702939 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1b40c5e00 0x7ff1b40c15b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4033000 tx=0x7ff1a40046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1b3e00640 1 -- 172.21.15.67:0/2764702939 reap_dead start 2024-03-09T09:23:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1b3e00640 1 -- 172.21.15.67:0/2764702939 reap_dead start 2024-03-09T09:23:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1b8c00640 1 -- 172.21.15.67:0/2764702939 shutdown_connections 2024-03-09T09:23:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1b8c00640 1 -- 172.21.15.67:0/2764702939 >> 172.21.15.67:0/2764702939 conn(0x7ff1b40cb480 msgr2=0x7ff1b40ae080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1b8c00640 1 -- 172.21.15.67:0/2764702939 shutdown_connections 2024-03-09T09:23:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:10.672+0000 7ff1b8c00640 1 -- 172.21.15.67:0/2764702939 wait complete. 2024-03-09T09:23:10.697 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:23:10.697 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:23:15.445 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:23:15.446 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:23:15.446 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 70 -eq 48 ']' 2024-03-09T09:23:15.446 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.70 2024-03-09T09:23:15.448 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.70 2024-03-09T09:23:15.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.70 2024-03-09T09:23:15.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.70 2024-03-09T09:23:15.450 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:23:18.999 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:23:18.999 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:23:19.000 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 77 -eq 48 ']' 2024-03-09T09:23:19.000 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.77 2024-03-09T09:23:19.000 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.77 2024-03-09T09:23:19.000 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.77 2024-03-09T09:23:19.000 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.77 2024-03-09T09:23:19.003 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:23:20.544 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:23:20.544 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:23:20.545 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 71 -eq 48 ']' 2024-03-09T09:23:20.545 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.71 2024-03-09T09:23:20.545 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.71 2024-03-09T09:23:20.545 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.71 2024-03-09T09:23:20.545 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.71 2024-03-09T09:23:20.551 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:23:26.204 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:23:26.204 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:23:26.204 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.78 2024-03-09T09:23:26.205 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 78 -eq 48 ']' 2024-03-09T09:23:26.205 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.78 2024-03-09T09:23:26.205 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.78 2024-03-09T09:23:26.205 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.78 2024-03-09T09:23:26.210 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:23:26.278 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:23:26.279 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:23:26.279 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 72 -eq 48 ']' 2024-03-09T09:23:26.279 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.72 2024-03-09T09:23:26.279 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.72 2024-03-09T09:23:26.279 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.72 2024-03-09T09:23:26.279 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.72 2024-03-09T09:23:26.282 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:23:30.698 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:23:30.698 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-03-09T09:23:30.698 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-09T09:23:30.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-09T09:23:30.707 INFO:tasks.ceph.mon.a:Started 2024-03-09T09:23:30.707 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:23:30.707 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:23:30.816 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:23:30.824 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:23:30.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.901+0000 7f83c9800640 1 Processor -- start 2024-03-09T09:23:30.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.901+0000 7f83c9800640 1 -- start start 2024-03-09T09:23:30.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.903+0000 7f83c9800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c5d10 0x7f83c40c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:30.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.903+0000 7f83c9800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f83c40c6800 0x7f83c4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:30.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.903+0000 7f83c9800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c40c2640 con 0x7f83c40c7550 2024-03-09T09:23:30.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.903+0000 7f83c9800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c40a6d10 con 0x7f83c40c6800 2024-03-09T09:23:30.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.903+0000 7f83c9800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40c2920 con 0x7f83c40c5d10 2024-03-09T09:23:30.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.903+0000 7f83c2000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f83c40c7550 0x7f83c40c60f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41946/0 (socket says 172.21.15.67:41946) 2024-03-09T09:23:30.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.903+0000 7f83c2000640 1 -- 172.21.15.67:0/4127677655 learned_addr learned my addr 172.21.15.67:0/4127677655 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:23:30.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.904+0000 7f83c2a00640 1 --2- 172.21.15.67:0/4127677655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c5d10 0x7f83c40c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:30.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.904+0000 7f83c3400640 1 --2- 172.21.15.67:0/4127677655 >> v2:172.21.15.134:3300/0 conn(0x7f83c40c6800 0x7f83c4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:30.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.904+0000 7f83c2a00640 1 -- 172.21.15.67:0/4127677655 >> v2:172.21.15.134:3300/0 conn(0x7f83c40c6800 msgr2=0x7f83c4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:30.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.904+0000 7f83c2a00640 1 --2- 172.21.15.67:0/4127677655 >> v2:172.21.15.134:3300/0 conn(0x7f83c40c6800 0x7f83c4159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:30.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.904+0000 7f83c2a00640 1 -- 172.21.15.67:0/4127677655 >> v1:172.21.15.67:6789/0 conn(0x7f83c40c7550 legacy=0x7f83c40c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:30.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.904+0000 7f83c2a00640 1 -- 172.21.15.67:0/4127677655 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c40a7300 con 0x7f83c40c5d10 2024-03-09T09:23:30.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.906+0000 7f83c2a00640 1 --2- 172.21.15.67:0/4127677655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c5d10 0x7f83c40c3110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83b40097a0 tx=0x7f83b402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bbd22214d2466e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:30.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.906+0000 7f83c1600640 1 -- 172.21.15.67:0/4127677655 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b40315a0 con 0x7f83c40c5d10 2024-03-09T09:23:30.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.906+0000 7f83c1600640 1 -- 172.21.15.67:0/4127677655 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b4037070 con 0x7f83c40c5d10 2024-03-09T09:23:30.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.906+0000 7f83c1600640 1 -- 172.21.15.67:0/4127677655 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4031960 con 0x7f83c40c5d10 2024-03-09T09:23:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.907+0000 7f83c9800640 1 -- 172.21.15.67:0/4127677655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c5d10 msgr2=0x7f83c40c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.907+0000 7f83c9800640 1 --2- 172.21.15.67:0/4127677655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c5d10 0x7f83c40c3110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83b40097a0 tx=0x7f83b402b840 comp rx=0 tx=0).stop 2024-03-09T09:23:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.907+0000 7f83c9800640 1 -- 172.21.15.67:0/4127677655 shutdown_connections 2024-03-09T09:23:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.907+0000 7f83c9800640 1 --2- 172.21.15.67:0/4127677655 >> v2:172.21.15.134:3300/0 conn(0x7f83c40c6800 0x7f83c4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.907+0000 7f83c9800640 1 --2- 172.21.15.67:0/4127677655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c5d10 0x7f83c40c3110 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.907+0000 7f83c9800640 1 -- 172.21.15.67:0/4127677655 >> 172.21.15.67:0/4127677655 conn(0x7f83c40a9950 msgr2=0x7f83c40a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:30.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.907+0000 7f83c9800640 1 -- 172.21.15.67:0/4127677655 shutdown_connections 2024-03-09T09:23:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.908+0000 7f83c9800640 1 -- 172.21.15.67:0/4127677655 wait complete. 2024-03-09T09:23:30.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.909+0000 7f83c9800640 1 Processor -- start 2024-03-09T09:23:30.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.909+0000 7f83c9800640 1 -- start start 2024-03-09T09:23:30.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.910+0000 7f83c9800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c40c5d10 0x7f83c409c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:30.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.910+0000 7f83c9800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c6800 0x7f83c41ef830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:30.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.910+0000 7f83c9800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c40c7550 0x7f83c4072820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:30.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.910+0000 7f83c9800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40c3650 con 0x7f83c40c5d10 2024-03-09T09:23:30.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.910+0000 7f83c9800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c4066e20 con 0x7f83c40c7550 2024-03-09T09:23:30.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.910+0000 7f83c9800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40a6f60 con 0x7f83c40c6800 2024-03-09T09:23:30.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.911+0000 7f83c2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c40c5d10 0x7f83c409c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:30.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.911+0000 7f83c2a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c40c5d10 0x7f83c409c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52562/0 (socket says 172.21.15.67:52562) 2024-03-09T09:23:30.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.911+0000 7f83c2a00640 1 -- 172.21.15.67:0/4106640638 learned_addr learned my addr 172.21.15.67:0/4106640638 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:30.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.911+0000 7f83c2a00640 1 -- 172.21.15.67:0/4106640638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c40c5d10 msgr2=0x7f83c409c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:23:30.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.911+0000 7f83c2a00640 1 -- 172.21.15.67:0/4106640638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c40c5d10 msgr2=0x7f83c409c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:23:30.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.911+0000 7f83c2a00640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c40c5d10 0x7f83c409c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:30.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.911+0000 7f83c2a00640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c40c5d10 0x7f83c409c450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:30.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.911+0000 7f83c2000640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c6800 0x7f83c41ef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:30.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.911+0000 7f83c3400640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c40c7550 0x7f83c4072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:30.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.912+0000 7f83c2000640 1 -- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c40c7550 msgr2=0x7f83c4072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:30.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.912+0000 7f83c2000640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c40c7550 0x7f83c4072820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:30.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.912+0000 7f83c2000640 1 -- 172.21.15.67:0/4106640638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c40c5d10 msgr2=0x7f83c409c450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:23:30.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.912+0000 7f83c2000640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c40c5d10 0x7f83c409c450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:30.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.912+0000 7f83c2000640 1 -- 172.21.15.67:0/4106640638 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c406ec40 con 0x7f83c40c6800 2024-03-09T09:23:30.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.912+0000 7f83c3400640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c40c7550 0x7f83c4072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:23:30.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.912+0000 7f83c2000640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c6800 0x7f83c41ef830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f83c40c7990 tx=0x7f83b000c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:30.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.913+0000 7f83baa00640 1 -- 172.21.15.67:0/4106640638 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0017070 con 0x7f83c40c6800 2024-03-09T09:23:30.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.913+0000 7f83baa00640 1 -- 172.21.15.67:0/4106640638 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b0005ce0 con 0x7f83c40c6800 2024-03-09T09:23:30.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.913+0000 7f83baa00640 1 -- 172.21.15.67:0/4106640638 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0007310 con 0x7f83c40c6800 2024-03-09T09:23:30.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.913+0000 7f83c9800640 1 -- 172.21.15.67:0/4106640638 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c41efd70 con 0x7f83c40c6800 2024-03-09T09:23:30.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.913+0000 7f83c9800640 1 -- 172.21.15.67:0/4106640638 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c406f190 con 0x7f83c40c6800 2024-03-09T09:23:30.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.914+0000 7f83c9800640 1 -- 172.21.15.67:0/4106640638 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c4075a00 con 0x7f83c40c6800 2024-03-09T09:23:30.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.916+0000 7f83baa00640 1 -- 172.21.15.67:0/4106640638 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f83b001a070 con 0x7f83c40c6800 2024-03-09T09:23:30.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.917+0000 7f83baa00640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83a406b4d0 0x7f83a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:30.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.917+0000 7f83baa00640 1 -- 172.21.15.67:0/4106640638 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f83b0089bb0 con 0x7f83c40c6800 2024-03-09T09:23:30.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.918+0000 7f83c2a00640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83a406b4d0 0x7f83a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:30.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.919+0000 7f83c2a00640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83a406b4d0 0x7f83a406d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f83b4034040 tx=0x7f83b403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:30.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:30.929+0000 7f83baa00640 1 -- 172.21.15.67:0/4106640638 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f83b0021020 con 0x7f83c40c6800 2024-03-09T09:23:31.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.211+0000 7f83c9800640 1 -- 172.21.15.67:0/4106640638 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f83c4067310 con 0x7f83c40c6800 2024-03-09T09:23:31.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.212+0000 7f83baa00640 1 -- 172.21.15.67:0/4106640638 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f83b0058060 con 0x7f83c40c6800 2024-03-09T09:23:31.219 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":238,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.214+0000 7f83c9800640 1 -- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83a406b4d0 msgr2=0x7f83a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.214+0000 7f83c9800640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83a406b4d0 0x7f83a406d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f83b4034040 tx=0x7f83b403d040 comp rx=0 tx=0).stop 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.214+0000 7f83c9800640 1 -- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c6800 msgr2=0x7f83c41ef830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.214+0000 7f83c9800640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c6800 0x7f83c41ef830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f83c40c7990 tx=0x7f83b000c760 comp rx=0 tx=0).stop 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.214+0000 7f83c9800640 1 -- 172.21.15.67:0/4106640638 shutdown_connections 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.214+0000 7f83c9800640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f83a406b4d0 0x7f83a406d990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.215+0000 7f83c9800640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83c40c7550 0x7f83c4072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.215+0000 7f83c9800640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83c40c6800 0x7f83c41ef830 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.215+0000 7f83c9800640 1 --2- 172.21.15.67:0/4106640638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83c40c5d10 0x7f83c409c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.215+0000 7f83c9800640 1 -- 172.21.15.67:0/4106640638 >> 172.21.15.67:0/4106640638 conn(0x7f83c40a9950 msgr2=0x7f83c40c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.215+0000 7f83c9800640 1 -- 172.21.15.67:0/4106640638 shutdown_connections 2024-03-09T09:23:31.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:31.215+0000 7f83c9800640 1 -- 172.21.15.67:0/4106640638 wait complete. 2024-03-09T09:23:31.711 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:23:31.711 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:23:31.711 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 73 -eq 48 ']' 2024-03-09T09:23:31.711 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.73 2024-03-09T09:23:31.711 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.73 2024-03-09T09:23:31.712 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.73 2024-03-09T09:23:31.712 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.73 2024-03-09T09:23:31.714 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:23:32.412 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:23:32.412 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:23:32.412 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 79 -eq 48 ']' 2024-03-09T09:23:32.412 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.79 2024-03-09T09:23:32.413 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.79 2024-03-09T09:23:32.413 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.79 2024-03-09T09:23:32.413 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.79 2024-03-09T09:23:32.418 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:23:34.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:23:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.447+0000 7f6df8800640 1 Processor -- start 2024-03-09T09:23:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.447+0000 7f6df8800640 1 -- start start 2024-03-09T09:23:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.448+0000 7f6df8800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6df00c5f50 0x7f6df0155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.448+0000 7f6df8800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6df006b040 0x7f6df006c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.448+0000 7f6df8800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6df00c96f0 0x7f6df00c6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df8800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df006b5b0 con 0x7f6df006b040 2024-03-09T09:23:34.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df8800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df006b890 con 0x7f6df00c96f0 2024-03-09T09:23:34.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df8800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df006bb70 con 0x7f6df00c5f50 2024-03-09T09:23:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6df006b040 0x7f6df006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df5400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6df006b040 0x7f6df006c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52576/0 (socket says 172.21.15.67:52576) 2024-03-09T09:23:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df5400640 1 -- 172.21.15.67:0/2187064219 learned_addr learned my addr 172.21.15.67:0/2187064219 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df5400640 1 -- 172.21.15.67:0/2187064219 >> v2:172.21.15.67:3300/0 conn(0x7f6df006b040 msgr2=0x7f6df006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:23:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df5400640 1 -- 172.21.15.67:0/2187064219 >> v2:172.21.15.67:3300/0 conn(0x7f6df006b040 msgr2=0x7f6df006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df5400640 1 --2- 172.21.15.67:0/2187064219 >> v2:172.21.15.67:3300/0 conn(0x7f6df006b040 0x7f6df006c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df5e00640 1 --2- 172.21.15.67:0/2187064219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6df00c5f50 0x7f6df0155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df5400640 1 --2- 172.21.15.67:0/2187064219 >> v2:172.21.15.67:3300/0 conn(0x7f6df006b040 0x7f6df006c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:34.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.449+0000 7f6df6800640 1 --2- 172.21.15.67:0/2187064219 >> v2:172.21.15.134:3300/0 conn(0x7f6df00c96f0 0x7f6df00c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.450+0000 7f6df5e00640 1 -- 172.21.15.67:0/2187064219 >> v2:172.21.15.134:3300/0 conn(0x7f6df00c96f0 msgr2=0x7f6df00c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.450+0000 7f6df5e00640 1 --2- 172.21.15.67:0/2187064219 >> v2:172.21.15.134:3300/0 conn(0x7f6df00c96f0 0x7f6df00c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.450+0000 7f6df5e00640 1 -- 172.21.15.67:0/2187064219 >> v2:172.21.15.67:3300/0 conn(0x7f6df006b040 msgr2=0x7f6df006c650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.450+0000 7f6df5e00640 1 --2- 172.21.15.67:0/2187064219 >> v2:172.21.15.67:3300/0 conn(0x7f6df006b040 0x7f6df006c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.450+0000 7f6df5e00640 1 -- 172.21.15.67:0/2187064219 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df006be50 con 0x7f6df00c5f50 2024-03-09T09:23:34.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.451+0000 7f6df5e00640 1 --2- 172.21.15.67:0/2187064219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6df00c5f50 0x7f6df0155e20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6df00c57d0 tx=0x7f6de402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c3cb4d31a3d10b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.452+0000 7f6df4a00640 1 -- 172.21.15.67:0/2187064219 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de40315a0 con 0x7f6df00c5f50 2024-03-09T09:23:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.452+0000 7f6df4a00640 1 -- 172.21.15.67:0/2187064219 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de4037070 con 0x7f6df00c5f50 2024-03-09T09:23:34.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.452+0000 7f6df4a00640 1 -- 172.21.15.67:0/2187064219 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4031960 con 0x7f6df00c5f50 2024-03-09T09:23:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.453+0000 7f6df8800640 1 -- 172.21.15.67:0/2187064219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6df00c5f50 msgr2=0x7f6df0155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.453+0000 7f6df8800640 1 --2- 172.21.15.67:0/2187064219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6df00c5f50 0x7f6df0155e20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6df00c57d0 tx=0x7f6de402b840 comp rx=0 tx=0).stop 2024-03-09T09:23:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.453+0000 7f6df8800640 1 -- 172.21.15.67:0/2187064219 shutdown_connections 2024-03-09T09:23:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.453+0000 7f6df8800640 1 --2- 172.21.15.67:0/2187064219 >> v2:172.21.15.134:3300/0 conn(0x7f6df00c96f0 0x7f6df00c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.453+0000 7f6df8800640 1 --2- 172.21.15.67:0/2187064219 >> v2:172.21.15.67:3300/0 conn(0x7f6df006b040 0x7f6df006c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.453+0000 7f6df8800640 1 --2- 172.21.15.67:0/2187064219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6df00c5f50 0x7f6df0155e20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.453+0000 7f6df8800640 1 -- 172.21.15.67:0/2187064219 >> 172.21.15.67:0/2187064219 conn(0x7f6df00c8050 msgr2=0x7f6df00c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.453+0000 7f6df8800640 1 -- 172.21.15.67:0/2187064219 shutdown_connections 2024-03-09T09:23:34.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.453+0000 7f6df8800640 1 -- 172.21.15.67:0/2187064219 wait complete. 2024-03-09T09:23:34.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.455+0000 7f6df8800640 1 Processor -- start 2024-03-09T09:23:34.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.455+0000 7f6df8800640 1 -- start start 2024-03-09T09:23:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df8800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6df006b040 0x7f6df01554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df8800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6df00c5f50 0x7f6df01e2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df8800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df00c96f0 0x7f6df01e27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df8800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00c6870 con 0x7f6df00c96f0 2024-03-09T09:23:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df8800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df014cc00 con 0x7f6df006b040 2024-03-09T09:23:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df8800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00c69f0 con 0x7f6df00c5f50 2024-03-09T09:23:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df00c96f0 0x7f6df01e27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df6800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df00c96f0 0x7f6df01e27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52590/0 (socket says 172.21.15.67:52590) 2024-03-09T09:23:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df6800640 1 -- 172.21.15.67:0/2164931732 learned_addr learned my addr 172.21.15.67:0/2164931732 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df5e00640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6df006b040 0x7f6df01554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df6800640 1 -- 172.21.15.67:0/2164931732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df00c96f0 msgr2=0x7f6df01e27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:23:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df6800640 1 -- 172.21.15.67:0/2164931732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df00c96f0 msgr2=0x7f6df01e27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:23:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df6800640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df00c96f0 0x7f6df01e27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df6800640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df00c96f0 0x7f6df01e27c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.457+0000 7f6df5400640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6df00c5f50 0x7f6df01e2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.458+0000 7f6df5e00640 1 -- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6df00c5f50 msgr2=0x7f6df01e2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.458+0000 7f6df5e00640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6df00c5f50 0x7f6df01e2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.458+0000 7f6df5e00640 1 -- 172.21.15.67:0/2164931732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df00c96f0 msgr2=0x7f6df01e27c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:23:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.458+0000 7f6df5e00640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df00c96f0 0x7f6df01e27c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.458+0000 7f6df5e00640 1 -- 172.21.15.67:0/2164931732 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df01ef6d0 con 0x7f6df006b040 2024-03-09T09:23:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.459+0000 7f6df5e00640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6df006b040 0x7f6df01554a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6df006bcf0 tx=0x7f6de40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.459+0000 7f6dea000640 1 -- 172.21.15.67:0/2164931732 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4032630 con 0x7f6df006b040 2024-03-09T09:23:34.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.459+0000 7f6dea000640 1 -- 172.21.15.67:0/2164931732 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de40327d0 con 0x7f6df006b040 2024-03-09T09:23:34.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.459+0000 7f6dea000640 1 -- 172.21.15.67:0/2164931732 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4032bc0 con 0x7f6df006b040 2024-03-09T09:23:34.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.459+0000 7f6df8800640 1 -- 172.21.15.67:0/2164931732 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df00c40d0 con 0x7f6df006b040 2024-03-09T09:23:34.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.460+0000 7f6df8800640 1 -- 172.21.15.67:0/2164931732 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df01562a0 con 0x7f6df006b040 2024-03-09T09:23:34.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.461+0000 7f6ddf400640 1 -- 172.21.15.67:0/2164931732 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df0075a00 con 0x7f6df006b040 2024-03-09T09:23:34.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.463+0000 7f6dea000640 1 -- 172.21.15.67:0/2164931732 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6de4005990 con 0x7f6df006b040 2024-03-09T09:23:34.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.464+0000 7f6dea000640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6dd406b4d0 0x7f6dd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:34.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.464+0000 7f6dea000640 1 -- 172.21.15.67:0/2164931732 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6de402f030 con 0x7f6df006b040 2024-03-09T09:23:34.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.465+0000 7f6df5400640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6dd406b4d0 0x7f6dd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:34.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.466+0000 7f6df5400640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6dd406b4d0 0x7f6dd406d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6df00cad00 tx=0x7f6de0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:34.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.481+0000 7f6dea000640 1 -- 172.21.15.67:0/2164931732 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6de4040070 con 0x7f6df006b040 2024-03-09T09:23:34.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.773+0000 7f6ddf400640 1 -- 172.21.15.67:0/2164931732 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6df006cca0 con 0x7f6df006b040 2024-03-09T09:23:34.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.775+0000 7f6dea000640 1 -- 172.21.15.67:0/2164931732 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f6de407b010 con 0x7f6df006b040 2024-03-09T09:23:34.781 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":238,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:23:34.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 -- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6dd406b4d0 msgr2=0x7f6dd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6dd406b4d0 0x7f6dd406d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6df00cad00 tx=0x7f6de0008040 comp rx=0 tx=0).stop 2024-03-09T09:23:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 -- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6df006b040 msgr2=0x7f6df01554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:34.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6df006b040 0x7f6df01554a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6df006bcf0 tx=0x7f6de40046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 -- 172.21.15.67:0/2164931732 shutdown_connections 2024-03-09T09:23:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6dd406b4d0 0x7f6dd406d990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6df00c96f0 0x7f6df01e27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6df00c5f50 0x7f6df01e2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 --2- 172.21.15.67:0/2164931732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6df006b040 0x7f6df01554a0 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6df006bcf0 tx=0x7f6de40046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 -- 172.21.15.67:0/2164931732 >> 172.21.15.67:0/2164931732 conn(0x7f6df00c8050 msgr2=0x7f6df00ac9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 -- 172.21.15.67:0/2164931732 shutdown_connections 2024-03-09T09:23:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:34.778+0000 7f6ddf400640 1 -- 172.21.15.67:0/2164931732 wait complete. 2024-03-09T09:23:37.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:23:38.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.019+0000 7ff7c8a00640 1 Processor -- start 2024-03-09T09:23:38.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.020+0000 7ff7c8a00640 1 -- start start 2024-03-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.021+0000 7ff7c8a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.021+0000 7ff7c8a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.021+0000 7ff7c8a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c415a2a0 con 0x7ff7c40c5f60 2024-03-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.021+0000 7ff7c8a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c4159d70 con 0x7ff7c4153500 2024-03-09T09:23:38.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.021+0000 7ff7c8a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c406bb30 con 0x7ff7c40c6630 2024-03-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:57850/0 (socket says 172.21.15.67:57850) 2024-03-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 learned_addr learned my addr 172.21.15.67:0/2533641949 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 msgr2=0x7ff7c40c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 msgr2=0x7ff7c40c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:38.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 msgr2=0x7ff7c406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:23:38.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 msgr2=0x7ff7c406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:38.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:38.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.022+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:38.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.223+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:38.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.223+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:38.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.224+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 msgr2=0x7ff7c40c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:23:38.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.224+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 msgr2=0x7ff7c40c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:38.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.224+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:38.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.224+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:23:38.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.224+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 msgr2=0x7ff7c406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:23:38.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.224+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 msgr2=0x7ff7c406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:38.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.224+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:38.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.224+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:23:38.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.625+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:38.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.626+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:38.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.626+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 msgr2=0x7ff7c406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:23:38.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.626+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 msgr2=0x7ff7c406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:38.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.626+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:38.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.626+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 msgr2=0x7ff7c40c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:23:38.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.626+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 msgr2=0x7ff7c40c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:38.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.626+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:38.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.626+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:23:38.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:38.626+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:23:39.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:39.428+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:39.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:39.428+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:39.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:39.428+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 msgr2=0x7ff7c40c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:23:39.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:39.428+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 msgr2=0x7ff7c40c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:39.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:39.428+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:39.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:39.428+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:23:39.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:39.428+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 msgr2=0x7ff7c406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:23:39.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:39.428+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 msgr2=0x7ff7c406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:39.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:39.428+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:39.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:39.430+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:23:40.718 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:23:40.785 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:23:40.785 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 80 -eq 48 ']' 2024-03-09T09:23:40.785 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.80 2024-03-09T09:23:40.786 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.80 2024-03-09T09:23:40.786 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.80 2024-03-09T09:23:40.786 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.80 2024-03-09T09:23:40.786 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:23:40.843 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:23:40.843 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:23:40.843 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 74 -eq 48 ']' 2024-03-09T09:23:40.844 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.74 2024-03-09T09:23:40.844 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.74 2024-03-09T09:23:40.844 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.74 2024-03-09T09:23:40.844 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.74 2024-03-09T09:23:41.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.022+0000 7ff7bb400640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 msgr2=0x7ff7c40c74b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:41.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.022+0000 7ff7bb400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7c40c6630 0x7ff7c40c74b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:41.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.022+0000 7ff7bb400640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 msgr2=0x7ff7c406b040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:41.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.022+0000 7ff7bb400640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.134:3300/0 conn(0x7ff7c4153500 0x7ff7c406b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:41.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.022+0000 7ff7bb400640 1 -- 172.21.15.67:0/2533641949 >> v1:172.21.15.134:6789/0 conn(0x7ff7c40c5f60 legacy=0x7ff7c4155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:41.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.022+0000 7ff7bb400640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:41.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.022+0000 7ff7bb400640 1 -- 172.21.15.67:0/2533641949 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c4159fc0 con 0x7ff7a4006210 2024-03-09T09:23:41.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.022+0000 7ff7bb400640 1 -- 172.21.15.67:0/2533641949 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c4159d70 con 0x7ff7a40036c0 2024-03-09T09:23:41.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.023+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:41.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.023+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 msgr2=0x7ff7a4005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:23:41.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.023+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 msgr2=0x7ff7a4005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:41.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.023+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:41.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.024+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.225+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.226+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 msgr2=0x7ff7a4005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:23:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.226+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 msgr2=0x7ff7a4005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.226+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.226+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:23:41.280 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:23:41.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.627+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:41.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.628+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 msgr2=0x7ff7a4005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:23:41.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.628+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 msgr2=0x7ff7a4005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:41.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.628+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:41.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:41.628+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:23:42.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:42.428+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:42.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:42.429+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 msgr2=0x7ff7a4005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:23:42.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:42.429+0000 7ff7c1e00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 msgr2=0x7ff7a4005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:23:42.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:42.429+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:23:42.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:42.429+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:23:43.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.568+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141630769 0 0) 0x7ff7c4159fc0 con 0x7ff7a4006210 2024-03-09T09:23:43.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.568+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a8003520 con 0x7ff7a4006210 2024-03-09T09:23:43.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3162651402 0 0) 0x7ff7a8003520 con 0x7ff7a4006210 2024-03-09T09:23:43.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 msgr2=0x7ff7a4005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:43.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 --2- 172.21.15.67:0/2533641949 >> v2:172.21.15.67:3300/0 conn(0x7ff7a40036c0 0x7ff7a4005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a400afc0 con 0x7ff7a4006210 2024-03-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7ff7b000f040 con 0x7ff7a4006210 2024-03-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 >> v1:172.21.15.67:6789/0 conn(0x7ff7a4006210 legacy=0x7ff7a40086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7a80039b0 0x7ff7a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a8004800 0x7ff7a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7a80051c0 0x7ff7a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4159d70 con 0x7ff7a8004800 2024-03-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a800c090 con 0x7ff7a80051c0 2024-03-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.571+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a800c290 con 0x7ff7a80039b0 2024-03-09T09:23:43.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.572+0000 7ff7c2800640 1 -- 172.21.15.67:0/2533641949 reap_dead start 2024-03-09T09:23:43.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.572+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a8004800 0x7ff7a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:43.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.572+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7a80039b0 0x7ff7a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:43.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.573+0000 7ff7c2800640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7a80051c0 0x7ff7a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:43.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.573+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7a80039b0 msgr2=0x7ff7a8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:43.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.573+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7a80039b0 0x7ff7a8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.573+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7a80051c0 msgr2=0x7ff7a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:43.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.573+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7a80051c0 0x7ff7a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.573+0000 7ff7c1400640 1 -- 172.21.15.67:0/2533641949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a400afc0 con 0x7ff7a8004800 2024-03-09T09:23:43.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.576+0000 7ff7c1400640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a8004800 0x7ff7a8004c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00143c0 tx=0x7ff7a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f1e1ed768a3866 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:43.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.577+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0033070 con 0x7ff7a8004800 2024-03-09T09:23:43.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.577+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b002d9c0 con 0x7ff7a8004800 2024-03-09T09:23:43.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.577+0000 7ff7c0a00640 1 -- 172.21.15.67:0/2533641949 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000cc90 con 0x7ff7a8004800 2024-03-09T09:23:43.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.578+0000 7ff7c8a00640 1 -- 172.21.15.67:0/2533641949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a8004800 msgr2=0x7ff7a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:43.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.578+0000 7ff7c8a00640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a8004800 0x7ff7a8004c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00143c0 tx=0x7ff7a8005830 comp rx=0 tx=0).stop 2024-03-09T09:23:43.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.578+0000 7ff7c8a00640 1 -- 172.21.15.67:0/2533641949 shutdown_connections 2024-03-09T09:23:43.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.578+0000 7ff7c8a00640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7a80051c0 0x7ff7a800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.578+0000 7ff7c8a00640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7a8004800 0x7ff7a8004c80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.578+0000 7ff7c8a00640 1 --2- 172.21.15.67:0/2533641949 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7a80039b0 0x7ff7a8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.578+0000 7ff7c8a00640 1 -- 172.21.15.67:0/2533641949 >> 172.21.15.67:0/2533641949 conn(0x7ff7c40c8060 msgr2=0x7ff7c40c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:43.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.579+0000 7ff7c8a00640 1 -- 172.21.15.67:0/2533641949 shutdown_connections 2024-03-09T09:23:43.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.579+0000 7ff7c8a00640 1 -- 172.21.15.67:0/2533641949 wait complete. 2024-03-09T09:23:43.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.581+0000 7ff7c8a00640 1 Processor -- start 2024-03-09T09:23:43.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.582+0000 7ff7c8a00640 1 -- start start 2024-03-09T09:23:43.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.583+0000 7ff7c8a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7a80039b0 0x7ff7c40c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:43.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.583+0000 7ff7c8a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7a80051c0 0x7ff7c41ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:43.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.583+0000 7ff7c8a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7c40bef10 0x7ff7c414cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:43.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.583+0000 7ff7c8a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40bed90 con 0x7ff7c40bef10 2024-03-09T09:23:43.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.583+0000 7ff7c8a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40c6f90 con 0x7ff7a80051c0 2024-03-09T09:23:43.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.583+0000 7ff7c8a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40c94e0 con 0x7ff7a80039b0 2024-03-09T09:23:43.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.584+0000 7ff7c2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7c40bef10 0x7ff7c414cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.584+0000 7ff7c2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7c40bef10 0x7ff7c414cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53244/0 (socket says 172.21.15.67:53244) 2024-03-09T09:23:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.584+0000 7ff7c2800640 1 -- 172.21.15.67:0/3628788449 learned_addr learned my addr 172.21.15.67:0/3628788449 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.584+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7a80039b0 0x7ff7c40c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.584+0000 7ff7c1400640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7a80051c0 0x7ff7c41ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.585+0000 7ff7c2800640 1 -- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7a80039b0 msgr2=0x7ff7c40c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.585+0000 7ff7c2800640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7a80039b0 0x7ff7c40c18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.585+0000 7ff7c2800640 1 -- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7a80051c0 msgr2=0x7ff7c41ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.585+0000 7ff7c2800640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7a80051c0 0x7ff7c41ef440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.585+0000 7ff7c2800640 1 -- 172.21.15.67:0/3628788449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c41ee9a0 con 0x7ff7c40bef10 2024-03-09T09:23:43.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.585+0000 7ff7c2800640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7c40bef10 0x7ff7c414cef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff7c408d8d0 tx=0x7ff7bc00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:43.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.586+0000 7ff7ba000640 1 -- 172.21.15.67:0/3628788449 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc005d50 con 0x7ff7c40bef10 2024-03-09T09:23:43.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.586+0000 7ff7ba000640 1 -- 172.21.15.67:0/3628788449 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc018070 con 0x7ff7c40bef10 2024-03-09T09:23:43.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.586+0000 7ff7ba000640 1 -- 172.21.15.67:0/3628788449 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc007310 con 0x7ff7c40bef10 2024-03-09T09:23:43.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.586+0000 7ff7c8a00640 1 -- 172.21.15.67:0/3628788449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c40bf3a0 con 0x7ff7c40bef10 2024-03-09T09:23:43.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.586+0000 7ff7c8a00640 1 -- 172.21.15.67:0/3628788449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c41eef20 con 0x7ff7c40bef10 2024-03-09T09:23:43.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.588+0000 7ff7c8a00640 1 -- 172.21.15.67:0/3628788449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c4075a00 con 0x7ff7c40bef10 2024-03-09T09:23:43.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.591+0000 7ff7ba000640 1 -- 172.21.15.67:0/3628788449 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff7bc003970 con 0x7ff7c40bef10 2024-03-09T09:23:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.593+0000 7ff7ba000640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff7a406c270 0x7ff7a406e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:43.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.593+0000 7ff7ba000640 1 -- 172.21.15.67:0/3628788449 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7ff7bc08a4d0 con 0x7ff7c40bef10 2024-03-09T09:23:43.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.593+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff7a406c270 0x7ff7a406e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:43.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.595+0000 7ff7c1e00640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff7a406c270 0x7ff7a406e730 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4006fd0 tx=0x7ff7b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:43.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.597+0000 7ff7ba000640 1 -- 172.21.15.67:0/3628788449 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff7bc021020 con 0x7ff7c40bef10 2024-03-09T09:23:43.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.891+0000 7ff7c8a00640 1 -- 172.21.15.67:0/3628788449 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff7c41ef0f0 con 0x7ff7c40bef10 2024-03-09T09:23:43.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.892+0000 7ff7ba000640 1 -- 172.21.15.67:0/3628788449 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7ff7bc0589f0 con 0x7ff7c40bef10 2024-03-09T09:23:43.899 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":246,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:23:43.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.894+0000 7ff7c8a00640 1 -- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff7a406c270 msgr2=0x7ff7a406e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.894+0000 7ff7c8a00640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff7a406c270 0x7ff7a406e730 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4006fd0 tx=0x7ff7b4008040 comp rx=0 tx=0).stop 2024-03-09T09:23:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.894+0000 7ff7c8a00640 1 -- 172.21.15.67:0/3628788449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7c40bef10 msgr2=0x7ff7c414cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.894+0000 7ff7c8a00640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7c40bef10 0x7ff7c414cef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff7c408d8d0 tx=0x7ff7bc00ea00 comp rx=0 tx=0).stop 2024-03-09T09:23:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.894+0000 7ff7c8a00640 1 -- 172.21.15.67:0/3628788449 shutdown_connections 2024-03-09T09:23:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.894+0000 7ff7c8a00640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff7a406c270 0x7ff7a406e730 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.894+0000 7ff7c8a00640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7c40bef10 0x7ff7c414cef0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.894+0000 7ff7c8a00640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7a80051c0 0x7ff7c41ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.894+0000 7ff7c8a00640 1 --2- 172.21.15.67:0/3628788449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff7a80039b0 0x7ff7c40c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:43.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.894+0000 7ff7c8a00640 1 -- 172.21.15.67:0/3628788449 >> 172.21.15.67:0/3628788449 conn(0x7ff7c40c8060 msgr2=0x7ff7c41e0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.895+0000 7ff7c8a00640 1 -- 172.21.15.67:0/3628788449 shutdown_connections 2024-03-09T09:23:43.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:43.895+0000 7ff7c8a00640 1 -- 172.21.15.67:0/3628788449 wait complete. 2024-03-09T09:23:43.929 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:23:43.929 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:23:43.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:23:44.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.139+0000 7f395e200640 1 Processor -- start 2024-03-09T09:23:44.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.139+0000 7f395e200640 1 -- start start 2024-03-09T09:23:44.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.140+0000 7f395e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39580ad8d0 0x7f3958159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.140+0000 7f395e200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f395806b880 0x7f3958153080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.140+0000 7f395e200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f395806c040 con 0x7f39580c6650 2024-03-09T09:23:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.140+0000 7f395e200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39580c61f0 con 0x7f395806b880 2024-03-09T09:23:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.140+0000 7f395e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395806bd60 con 0x7f39580ad8d0 2024-03-09T09:23:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.141+0000 7f3957e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f395806b880 0x7f3958153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.141+0000 7f3957e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f395806b880 0x7f3958153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53252/0 (socket says 172.21.15.67:53252) 2024-03-09T09:23:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.141+0000 7f3957e00640 1 -- 172.21.15.67:0/4145241421 learned_addr learned my addr 172.21.15.67:0/4145241421 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.141+0000 7f395ce00640 1 --2- 172.21.15.67:0/4145241421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39580ad8d0 0x7f3958159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.141+0000 7f3957e00640 1 -- 172.21.15.67:0/4145241421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39580ad8d0 msgr2=0x7f3958159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:44.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.141+0000 7f3957e00640 1 --2- 172.21.15.67:0/4145241421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39580ad8d0 0x7f3958159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.141+0000 7f3957e00640 1 -- 172.21.15.67:0/4145241421 >> v1:172.21.15.134:6789/0 conn(0x7f39580c6650 legacy=0x7f3958155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:44.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.141+0000 7f3957e00640 1 -- 172.21.15.67:0/4145241421 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39580c5e80 con 0x7f395806b880 2024-03-09T09:23:44.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.142+0000 7f3957e00640 1 --2- 172.21.15.67:0/4145241421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f395806b880 0x7f3958153080 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3944009870 tx=0x7f394402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f25ac6d8820e5df server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.143+0000 7f3957400640 1 -- 172.21.15.67:0/4145241421 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944035020 con 0x7f395806b880 2024-03-09T09:23:44.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.143+0000 7f3957400640 1 -- 172.21.15.67:0/4145241421 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3944031640 con 0x7f395806b880 2024-03-09T09:23:44.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.143+0000 7f3957400640 1 -- 172.21.15.67:0/4145241421 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394402d040 con 0x7f395806b880 2024-03-09T09:23:44.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.143+0000 7f395e200640 1 -- 172.21.15.67:0/4145241421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f395806b880 msgr2=0x7f3958153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.143+0000 7f395e200640 1 --2- 172.21.15.67:0/4145241421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f395806b880 0x7f3958153080 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3944009870 tx=0x7f394402b8f0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.143+0000 7f395e200640 1 -- 172.21.15.67:0/4145241421 shutdown_connections 2024-03-09T09:23:44.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.143+0000 7f395e200640 1 --2- 172.21.15.67:0/4145241421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f395806b880 0x7f3958153080 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.143+0000 7f395e200640 1 --2- 172.21.15.67:0/4145241421 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39580ad8d0 0x7f3958159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.143+0000 7f395e200640 1 -- 172.21.15.67:0/4145241421 >> 172.21.15.67:0/4145241421 conn(0x7f39580c8510 msgr2=0x7f39580acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:44.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.144+0000 7f395e200640 1 -- 172.21.15.67:0/4145241421 shutdown_connections 2024-03-09T09:23:44.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.144+0000 7f395e200640 1 -- 172.21.15.67:0/4145241421 wait complete. 2024-03-09T09:23:44.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.145+0000 7f395e200640 1 Processor -- start 2024-03-09T09:23:44.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.146+0000 7f395e200640 1 -- start start 2024-03-09T09:23:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.146+0000 7f395e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f395806b880 0x7f39580bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39580ad8d0 0x7f3958156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c6650 0x7f39581567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39581535c0 con 0x7f39580c6650 2024-03-09T09:23:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395e200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39580adec0 con 0x7f395806b880 2024-03-09T09:23:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3958153740 con 0x7f39580ad8d0 2024-03-09T09:23:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c6650 0x7f39581567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c6650 0x7f39581567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53256/0 (socket says 172.21.15.67:53256) 2024-03-09T09:23:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395d800640 1 -- 172.21.15.67:0/909756774 learned_addr learned my addr 172.21.15.67:0/909756774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:44.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f3957e00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39580ad8d0 0x7f3958156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395d800640 1 -- 172.21.15.67:0/909756774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39580ad8d0 msgr2=0x7f3958156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395d800640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39580ad8d0 0x7f3958156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395d800640 1 -- 172.21.15.67:0/909756774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f395806b880 msgr2=0x7f39580bf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395d800640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f395806b880 0x7f39580bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.147+0000 7f395d800640 1 -- 172.21.15.67:0/909756774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39580c09e0 con 0x7f39580c6650 2024-03-09T09:23:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.148+0000 7f395d800640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c6650 0x7f39581567b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f395000c6e0 tx=0x7f395000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.148+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950003b70 con 0x7f39580c6650 2024-03-09T09:23:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.148+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3950003d30 con 0x7f39580c6650 2024-03-09T09:23:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.148+0000 7f395e200640 1 -- 172.21.15.67:0/909756774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39580bf1f0 con 0x7f39580c6650 2024-03-09T09:23:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.148+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950005310 con 0x7f39580c6650 2024-03-09T09:23:44.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.148+0000 7f395e200640 1 -- 172.21.15.67:0/909756774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39580bf490 con 0x7f39580c6650 2024-03-09T09:23:44.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.149+0000 7f395e200640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3958157370 0x7f39581ee910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.149+0000 7f395e200640 1 -- 172.21.15.67:0/909756774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39580b1470 con 0x7f3958157370 2024-03-09T09:23:44.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.150+0000 7f395ce00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3958157370 0x7f39581ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.150+0000 7f395ce00640 1 -- 172.21.15.67:0/909756774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f395804f3c0 con 0x7f3958157370 2024-03-09T09:23:44.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.151+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f39500054b0 con 0x7f39580c6650 2024-03-09T09:23:44.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.151+0000 7f395ce00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3958157370 0x7f39581ee910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f394c00d4e0 tx=0x7f394c00d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.152+0000 7f3954c00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f393806b4d0 0x7f393806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.152+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f395008a4d0 con 0x7f39580c6650 2024-03-09T09:23:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.152+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c0125f0 con 0x7f3958157370 2024-03-09T09:23:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.152+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f394c019070 con 0x7f3958157370 2024-03-09T09:23:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.152+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3958157370 msgr2=0x7f39581ee910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.152+0000 7f3954c00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3958157370 0x7f39581ee910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f394c00d4e0 tx=0x7f394c00d9a0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.152+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3958157370 msgr2=0x7f39581ee910 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.152+0000 7f3954c00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3958157370 0x7f39581ee910 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f394c00d4e0 tx=0x7f394c00d9a0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.152+0000 7f3957e00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f393806b4d0 0x7f393806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.153+0000 7f3957e00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f393806b4d0 0x7f393806d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3944036000 tx=0x7f394403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.159+0000 7f395e200640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c13f0 0x7f39581eee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.159+0000 7f395e200640 1 -- 172.21.15.67:0/909756774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39580bf6d0 con 0x7f39580c13f0 2024-03-09T09:23:44.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.160+0000 7f395ce00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c13f0 0x7f39581eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.160+0000 7f395ce00640 1 -- 172.21.15.67:0/909756774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3958152500 con 0x7f39580c13f0 2024-03-09T09:23:44.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.160+0000 7f395ce00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c13f0 0x7f39581eee50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f394c00d4b0 tx=0x7f394c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.161+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c012960 con 0x7f39580c13f0 2024-03-09T09:23:44.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.161+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4536 (secure 0 0 0) 0x7f394c012960 con 0x7f39580c13f0 2024-03-09T09:23:44.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.161+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c13f0 msgr2=0x7f39581eee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.161+0000 7f3954c00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c13f0 0x7f39581eee50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f394c00d4b0 tx=0x7f394c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.161+0000 7f3954c00640 1 -- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c13f0 msgr2=0x7f39581eee50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:44.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.161+0000 7f3954c00640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c13f0 0x7f39581eee50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.169 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:23:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:23:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:23:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:23:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 246, 2024-03-09T09:23:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:23:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:23:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:23:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:23:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:23:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:23:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:23:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:23:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:23:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:23:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:23:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:23:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:23:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:23:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:23:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:23:44.181 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:23:44.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.161+0000 7f395e200640 1 -- 172.21.15.67:0/909756774 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f393806b4d0 msgr2=0x7f393806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.161+0000 7f395e200640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f393806b4d0 0x7f393806d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3944036000 tx=0x7f394403c040 comp rx=0 tx=0).stop 2024-03-09T09:23:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.162+0000 7f395e200640 1 -- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c6650 msgr2=0x7f39581567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.162+0000 7f395e200640 1 --2- 172.21.15.67:0/909756774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39580c6650 0x7f39581567b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f395000c6e0 tx=0x7f395000cba0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.162+0000 7f395d800640 1 -- 172.21.15.67:0/909756774 reap_dead start 2024-03-09T09:23:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.162+0000 7f395d800640 1 -- 172.21.15.67:0/909756774 reap_dead start 2024-03-09T09:23:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.162+0000 7f395e200640 1 -- 172.21.15.67:0/909756774 shutdown_connections 2024-03-09T09:23:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.162+0000 7f395e200640 1 -- 172.21.15.67:0/909756774 >> 172.21.15.67:0/909756774 conn(0x7f39580c8510 msgr2=0x7f3958151df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.162+0000 7f395e200640 1 -- 172.21.15.67:0/909756774 shutdown_connections 2024-03-09T09:23:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.162+0000 7f395e200640 1 -- 172.21.15.67:0/909756774 wait complete. 2024-03-09T09:23:44.185 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:23:44.185 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:23:44.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:23:44.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.408+0000 7f0993200640 1 Processor -- start 2024-03-09T09:23:44.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.408+0000 7f0993200640 1 -- start start 2024-03-09T09:23:44.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.410+0000 7f0993200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f09940ac930 0x7f09940c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.410+0000 7f0993200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f09940c30a0 0x7f0994150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.410+0000 7f0993200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0994151190 0x7f09941539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.410+0000 7f0993200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0994070670 con 0x7f09940ac930 2024-03-09T09:23:44.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.410+0000 7f0993200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09940c2780 con 0x7f09940c30a0 2024-03-09T09:23:44.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.410+0000 7f0993200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09940708c0 con 0x7f0994151190 2024-03-09T09:23:44.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.410+0000 7f0991e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f09940ac930 0x7f09940c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.410+0000 7f0991e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f09940ac930 0x7f09940c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53264/0 (socket says 172.21.15.67:53264) 2024-03-09T09:23:44.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.410+0000 7f0991e00640 1 -- 172.21.15.67:0/1980646930 learned_addr learned my addr 172.21.15.67:0/1980646930 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:44.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.411+0000 7f0991400640 1 --2- 172.21.15.67:0/1980646930 >> v2:172.21.15.134:3300/0 conn(0x7f09940c30a0 0x7f0994150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.411+0000 7f0992800640 1 --2- 172.21.15.67:0/1980646930 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0994151190 0x7f09941539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.411+0000 7f0991e00640 1 -- 172.21.15.67:0/1980646930 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0994151190 msgr2=0x7f09941539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:44.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.411+0000 7f0991e00640 1 --2- 172.21.15.67:0/1980646930 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0994151190 0x7f09941539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.411+0000 7f0991e00640 1 -- 172.21.15.67:0/1980646930 >> v2:172.21.15.134:3300/0 conn(0x7f09940c30a0 msgr2=0x7f0994150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:44.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.411+0000 7f0991e00640 1 --2- 172.21.15.67:0/1980646930 >> v2:172.21.15.134:3300/0 conn(0x7f09940c30a0 0x7f0994150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.411+0000 7f0991e00640 1 -- 172.21.15.67:0/1980646930 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994070ba0 con 0x7f09940ac930 2024-03-09T09:23:44.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.412+0000 7f0991e00640 1 --2- 172.21.15.67:0/1980646930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09940ac930 0x7f09940c3660 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f09940c8040 tx=0x7f098402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6482dff4067047e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.412+0000 7f0990a00640 1 -- 172.21.15.67:0/1980646930 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984035020 con 0x7f09940ac930 2024-03-09T09:23:44.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.412+0000 7f0990a00640 1 -- 172.21.15.67:0/1980646930 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0984031870 con 0x7f09940ac930 2024-03-09T09:23:44.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.412+0000 7f0990a00640 1 -- 172.21.15.67:0/1980646930 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984031c40 con 0x7f09940ac930 2024-03-09T09:23:44.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.413+0000 7f0993200640 1 -- 172.21.15.67:0/1980646930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09940ac930 msgr2=0x7f09940c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.413+0000 7f0993200640 1 --2- 172.21.15.67:0/1980646930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09940ac930 0x7f09940c3660 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f09940c8040 tx=0x7f098402b910 comp rx=0 tx=0).stop 2024-03-09T09:23:44.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.413+0000 7f0993200640 1 -- 172.21.15.67:0/1980646930 shutdown_connections 2024-03-09T09:23:44.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.413+0000 7f0993200640 1 --2- 172.21.15.67:0/1980646930 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0994151190 0x7f09941539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.413+0000 7f0993200640 1 --2- 172.21.15.67:0/1980646930 >> v2:172.21.15.134:3300/0 conn(0x7f09940c30a0 0x7f0994150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.413+0000 7f0993200640 1 --2- 172.21.15.67:0/1980646930 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09940ac930 0x7f09940c3660 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.413+0000 7f0993200640 1 -- 172.21.15.67:0/1980646930 >> 172.21.15.67:0/1980646930 conn(0x7f09940c7220 msgr2=0x7f09940c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:44.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.414+0000 7f0993200640 1 -- 172.21.15.67:0/1980646930 shutdown_connections 2024-03-09T09:23:44.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.414+0000 7f0993200640 1 -- 172.21.15.67:0/1980646930 wait complete. 2024-03-09T09:23:44.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.415+0000 7f0993200640 1 Processor -- start 2024-03-09T09:23:44.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.415+0000 7f0993200640 1 -- start start 2024-03-09T09:23:44.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.417+0000 7f0993200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09940ac930 0x7f09941eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.417+0000 7f0993200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09940c30a0 0x7f09941ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.417+0000 7f0993200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0994151190 0x7f0994143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.417+0000 7f0993200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09940acd10 con 0x7f0994151190 2024-03-09T09:23:44.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.417+0000 7f0993200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941515e0 con 0x7f09940ac930 2024-03-09T09:23:44.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.417+0000 7f0993200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09940ace90 con 0x7f09940c30a0 2024-03-09T09:23:44.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.417+0000 7f0992800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0994151190 0x7f0994143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.417+0000 7f0992800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0994151190 0x7f0994143f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53278/0 (socket says 172.21.15.67:53278) 2024-03-09T09:23:44.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.417+0000 7f0992800640 1 -- 172.21.15.67:0/255075773 learned_addr learned my addr 172.21.15.67:0/255075773 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:44.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.417+0000 7f0991e00640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09940ac930 0x7f09941eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f0991400640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09940c30a0 0x7f09941ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f0992800640 1 -- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09940c30a0 msgr2=0x7f09941ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f0992800640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09940c30a0 0x7f09941ef2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f0992800640 1 -- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09940ac930 msgr2=0x7f09941eb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f0992800640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09940ac930 0x7f09941eb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f0992800640 1 -- 172.21.15.67:0/255075773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994144b60 con 0x7f0994151190 2024-03-09T09:23:44.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f0991e00640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09940ac930 0x7f09941eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:23:44.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f0992800640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0994151190 0x7f0994143f50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f098c007e40 tx=0x7f098c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c014620 con 0x7f0994151190 2024-03-09T09:23:44.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c019070 con 0x7f0994151190 2024-03-09T09:23:44.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f0993200640 1 -- 172.21.15.67:0/255075773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0994144880 con 0x7f0994151190 2024-03-09T09:23:44.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.418+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c014a20 con 0x7f0994151190 2024-03-09T09:23:44.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.419+0000 7f0993200640 1 -- 172.21.15.67:0/255075773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09941eeaf0 con 0x7f0994151190 2024-03-09T09:23:44.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.420+0000 7f0993200640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994155050 0x7f0994149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.420+0000 7f0993200640 1 -- 172.21.15.67:0/255075773 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09940c6e20 con 0x7f0994155050 2024-03-09T09:23:44.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.420+0000 7f0991e00640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994155050 0x7f0994149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.421+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f098c014bc0 con 0x7f0994151190 2024-03-09T09:23:44.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.421+0000 7f0991e00640 1 -- 172.21.15.67:0/255075773 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f099404f3c0 con 0x7f0994155050 2024-03-09T09:23:44.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.422+0000 7f098a000640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f097406b4d0 0x7f097406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.422+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f098c08aea0 con 0x7f0994151190 2024-03-09T09:23:44.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.422+0000 7f0991e00640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994155050 0x7f0994149b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f099406edb0 tx=0x7f0984004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.422+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984049070 con 0x7f0994155050 2024-03-09T09:23:44.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.422+0000 7f0991400640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f097406b4d0 0x7f097406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.423+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f0984009490 con 0x7f0994155050 2024-03-09T09:23:44.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.423+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994155050 msgr2=0x7f0994149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.423+0000 7f098a000640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994155050 0x7f0994149b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f099406edb0 tx=0x7f0984004730 comp rx=0 tx=0).stop 2024-03-09T09:23:44.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.423+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994155050 msgr2=0x7f0994149b00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:44.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.423+0000 7f098a000640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994155050 0x7f0994149b00 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f099406edb0 tx=0x7f0984004730 comp rx=0 tx=0).stop 2024-03-09T09:23:44.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.424+0000 7f0991400640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f097406b4d0 0x7f097406d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f09940c24a0 tx=0x7f0980008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.431+0000 7f0993200640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09941571c0 0x7f09941f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.431+0000 7f0993200640 1 -- 172.21.15.67:0/255075773 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941eba20 con 0x7f09941571c0 2024-03-09T09:23:44.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.432+0000 7f0991e00640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09941571c0 0x7f09941f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.433+0000 7f0991e00640 1 -- 172.21.15.67:0/255075773 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f099414ff30 con 0x7f09941571c0 2024-03-09T09:23:44.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.433+0000 7f0991e00640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09941571c0 0x7f09941f7d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0984000c00 tx=0x7f09840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.434+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984049040 con 0x7f09941571c0 2024-03-09T09:23:44.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.434+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7f098404b2c0 con 0x7f09941571c0 2024-03-09T09:23:44.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.434+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09941571c0 msgr2=0x7f09941f7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.434+0000 7f098a000640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09941571c0 0x7f09941f7d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0984000c00 tx=0x7f09840046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.434+0000 7f098a000640 1 -- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09941571c0 msgr2=0x7f09941f7d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:44.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.434+0000 7f098a000640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09941571c0 0x7f09941f7d90 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0984000c00 tx=0x7f09840046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.448 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:23:44.449 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:23:44.449 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:23:44.449 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:23:44.449 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 246, 2024-03-09T09:23:44.449 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:23:44.449 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:23:44.449 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:23:44.449 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:23:44.449 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.449 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:23:44.450 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:44.450 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:23:44.450 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:23:44.450 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:44.450 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:44.450 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:44.450 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:44.450 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:44.451 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:44.452 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:44.452 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:44.452 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:44.452 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:44.452 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:44.452 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:44.452 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:44.452 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:44.452 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.453 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:23:44.453 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:23:44.453 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:23:44.453 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:23:44.453 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:23:44.453 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:23:44.453 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:44.453 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:44.454 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:23:44.454 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:23:44.454 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:23:44.454 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:23:44.454 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:23:44.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:23:44.454 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:23:44.454 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:44.454 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:23:44.454 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:44.455 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:44.455 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:44.455 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:44.455 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:44.455 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:44.455 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:44.455 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:44.455 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:44.455 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:44.455 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:44.455 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.456 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:23:44.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.456 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:23:44.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.456 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:23:44.456 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:23:44.456 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:44.456 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:44.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.457 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:44.457 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:23:44.457 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.457 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:44.457 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:23:44.457 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.457 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.458 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.458 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:23:44.458 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:23:44.458 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:44.458 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:44.458 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:44.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.458 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.459 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.460 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.460 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:23:44.460 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:23:44.460 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:44.460 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:44.460 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:44.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.460 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:23:44.460 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:23:44.461 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:44.461 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:44.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:44.461 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:23:44.461 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:44.462 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:23:44.462 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.462 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.462 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.462 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:23:44.462 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:23:44.462 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:44.462 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:44.462 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:44.462 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:23:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:23:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.463 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:23:44.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.464 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.464 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.464 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:23:44.464 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.464 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.464 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:23:44.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.464 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.464 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.464 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:23:44.465 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.465 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.465 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:23:44.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.465 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:23:44.465 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.465 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:44.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.466 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.466 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.466 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:23:44.466 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.466 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.466 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:23:44.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.466 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.466 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.466 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:44.466 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.467 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.467 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:23:44.467 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:23:44.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.435+0000 7f0993200640 1 -- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f097406b4d0 msgr2=0x7f097406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.435+0000 7f0993200640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f097406b4d0 0x7f097406d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f09940c24a0 tx=0x7f0980008040 comp rx=0 tx=0).stop 2024-03-09T09:23:44.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.435+0000 7f0993200640 1 -- 172.21.15.67:0/255075773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0994151190 msgr2=0x7f0994143f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.435+0000 7f0993200640 1 --2- 172.21.15.67:0/255075773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0994151190 0x7f0994143f50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f098c007e40 tx=0x7f098c00e4b0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.435+0000 7f0992800640 1 -- 172.21.15.67:0/255075773 reap_dead start 2024-03-09T09:23:44.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.435+0000 7f0992800640 1 -- 172.21.15.67:0/255075773 reap_dead start 2024-03-09T09:23:44.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.435+0000 7f0993200640 1 -- 172.21.15.67:0/255075773 shutdown_connections 2024-03-09T09:23:44.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.435+0000 7f0993200640 1 -- 172.21.15.67:0/255075773 >> 172.21.15.67:0/255075773 conn(0x7f09940c7220 msgr2=0x7f09940c1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:44.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.435+0000 7f0993200640 1 -- 172.21.15.67:0/255075773 shutdown_connections 2024-03-09T09:23:44.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.435+0000 7f0993200640 1 -- 172.21.15.67:0/255075773 wait complete. 2024-03-09T09:23:44.470 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:23:44.470 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:23:44.471 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:23:44.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.722+0000 7fae4ca00640 1 Processor -- start 2024-03-09T09:23:44.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.723+0000 7fae4ca00640 1 -- start start 2024-03-09T09:23:44.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.724+0000 7fae4ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fae4806ec60 0x7fae480c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.724+0000 7fae4ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae48157910 0x7fae48157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.724+0000 7fae4ca00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae480c2810 con 0x7fae480c34c0 2024-03-09T09:23:44.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.724+0000 7fae4ca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4806fa20 con 0x7fae4806ec60 2024-03-09T09:23:44.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.724+0000 7fae4ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae480c7270 con 0x7fae48157910 2024-03-09T09:23:44.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.724+0000 7fae3d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fae4806ec60 0x7fae480c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.724+0000 7fae3d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fae4806ec60 0x7fae480c2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53282/0 (socket says 172.21.15.67:53282) 2024-03-09T09:23:44.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.724+0000 7fae3d400640 1 -- 172.21.15.67:0/2411213434 learned_addr learned my addr 172.21.15.67:0/2411213434 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:44.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.725+0000 7fae46800640 1 --2- 172.21.15.67:0/2411213434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae48157910 0x7fae48157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.725+0000 7fae3d400640 1 -- 172.21.15.67:0/2411213434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae48157910 msgr2=0x7fae48157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:44.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.725+0000 7fae3d400640 1 --2- 172.21.15.67:0/2411213434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae48157910 0x7fae48157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.725+0000 7fae3d400640 1 -- 172.21.15.67:0/2411213434 >> v1:172.21.15.134:6789/0 conn(0x7fae480c34c0 legacy=0x7fae4806f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.725+0000 7fae3d400640 1 -- 172.21.15.67:0/2411213434 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae480c74c0 con 0x7fae4806ec60 2024-03-09T09:23:44.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.726+0000 7fae3d400640 1 --2- 172.21.15.67:0/2411213434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae4806ec60 0x7fae480c2af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae30009870 tx=0x7fae3002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a3dd4c6f9ee72c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.726+0000 7fae45400640 1 -- 172.21.15.67:0/2411213434 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30035020 con 0x7fae4806ec60 2024-03-09T09:23:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.726+0000 7fae45400640 1 -- 172.21.15.67:0/2411213434 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae30031640 con 0x7fae4806ec60 2024-03-09T09:23:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.726+0000 7fae45400640 1 -- 172.21.15.67:0/2411213434 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3002d040 con 0x7fae4806ec60 2024-03-09T09:23:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.727+0000 7fae4ca00640 1 -- 172.21.15.67:0/2411213434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae4806ec60 msgr2=0x7fae480c2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.727+0000 7fae4ca00640 1 --2- 172.21.15.67:0/2411213434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae4806ec60 0x7fae480c2af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae30009870 tx=0x7fae3002b8f0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.727+0000 7fae4ca00640 1 -- 172.21.15.67:0/2411213434 shutdown_connections 2024-03-09T09:23:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.727+0000 7fae4ca00640 1 --2- 172.21.15.67:0/2411213434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae48157910 0x7fae48157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.727+0000 7fae4ca00640 1 --2- 172.21.15.67:0/2411213434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae4806ec60 0x7fae480c2af0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.727+0000 7fae4ca00640 1 -- 172.21.15.67:0/2411213434 >> 172.21.15.67:0/2411213434 conn(0x7fae480c7a90 msgr2=0x7fae480c7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.727+0000 7fae4ca00640 1 -- 172.21.15.67:0/2411213434 shutdown_connections 2024-03-09T09:23:44.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.727+0000 7fae4ca00640 1 -- 172.21.15.67:0/2411213434 wait complete. 2024-03-09T09:23:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.729+0000 7fae4ca00640 1 Processor -- start 2024-03-09T09:23:44.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.729+0000 7fae4ca00640 1 -- start start 2024-03-09T09:23:44.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.730+0000 7fae4ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae4806ec60 0x7fae4806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.730+0000 7fae4ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae480c34c0 0x7fae481eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.730+0000 7fae4ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae48157910 0x7fae481ef420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.730+0000 7fae4ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae48073cf0 con 0x7fae4806ec60 2024-03-09T09:23:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.730+0000 7fae4ca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae48066d90 con 0x7fae480c34c0 2024-03-09T09:23:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.730+0000 7fae4ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae480672b0 con 0x7fae48157910 2024-03-09T09:23:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae45e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae4806ec60 0x7fae4806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae45e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae4806ec60 0x7fae4806e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53284/0 (socket says 172.21.15.67:53284) 2024-03-09T09:23:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae45e00640 1 -- 172.21.15.67:0/2410570298 learned_addr learned my addr 172.21.15.67:0/2410570298 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae3d400640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae480c34c0 0x7fae481eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae46800640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae48157910 0x7fae481ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae45e00640 1 -- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae48157910 msgr2=0x7fae481ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae45e00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae48157910 0x7fae481ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae45e00640 1 -- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae480c34c0 msgr2=0x7fae481eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae45e00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae480c34c0 0x7fae481eeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae45e00640 1 -- 172.21.15.67:0/2410570298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae481eb9e0 con 0x7fae4806ec60 2024-03-09T09:23:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae46800640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae48157910 0x7fae481ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:23:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.731+0000 7fae3d400640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae480c34c0 0x7fae481eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:23:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.732+0000 7fae45e00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae4806ec60 0x7fae4806e4c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c7f0 tx=0x7fae3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.732+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38003b90 con 0x7fae4806ec60 2024-03-09T09:23:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.732+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae38003d30 con 0x7fae4806ec60 2024-03-09T09:23:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.732+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3800aa90 con 0x7fae4806ec60 2024-03-09T09:23:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.732+0000 7fae4ca00640 1 -- 172.21.15.67:0/2410570298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae481ebcc0 con 0x7fae4806ec60 2024-03-09T09:23:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.732+0000 7fae4ca00640 1 -- 172.21.15.67:0/2410570298 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae481ec280 con 0x7fae4806ec60 2024-03-09T09:23:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.733+0000 7fae4ca00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae4809ba40 0x7fae481ddac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.733+0000 7fae4ca00640 1 -- 172.21.15.67:0/2410570298 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae480c1550 con 0x7fae4809ba40 2024-03-09T09:23:44.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.734+0000 7fae3d400640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae4809ba40 0x7fae481ddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.734+0000 7fae3d400640 1 -- 172.21.15.67:0/2410570298 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae48075a00 con 0x7fae4809ba40 2024-03-09T09:23:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.735+0000 7fae3d400640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae4809ba40 0x7fae481ddac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fae4806caf0 tx=0x7fae30004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.735+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30049070 con 0x7fae4809ba40 2024-03-09T09:23:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.735+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fae3800ac30 con 0x7fae4806ec60 2024-03-09T09:23:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.736+0000 7fae3ea00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fae2406b3d0 0x7fae2406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.736+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fae3808a550 con 0x7fae4806ec60 2024-03-09T09:23:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.736+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fae3003a6f0 con 0x7fae4809ba40 2024-03-09T09:23:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.736+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae4809ba40 msgr2=0x7fae481ddac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.736+0000 7fae3ea00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae4809ba40 0x7fae481ddac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fae4806caf0 tx=0x7fae30004730 comp rx=0 tx=0).stop 2024-03-09T09:23:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.736+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae4809ba40 msgr2=0x7fae481ddac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.736+0000 7fae3ea00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae4809ba40 0x7fae481ddac0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fae4806caf0 tx=0x7fae30004730 comp rx=0 tx=0).stop 2024-03-09T09:23:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.737+0000 7fae46800640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fae2406b3d0 0x7fae2406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.738+0000 7fae46800640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fae2406b3d0 0x7fae2406d890 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fae40006fd0 tx=0x7fae40008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.744+0000 7fae4ca00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae481de000 0x7fae481de420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.744+0000 7fae4ca00640 1 -- 172.21.15.67:0/2410570298 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae480c1bb0 con 0x7fae481de000 2024-03-09T09:23:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.745+0000 7fae3d400640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae481de000 0x7fae481de420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.745+0000 7fae3d400640 1 -- 172.21.15.67:0/2410570298 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fae4806ea00 con 0x7fae481de000 2024-03-09T09:23:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.746+0000 7fae3d400640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae481de000 0x7fae481de420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae30009870 tx=0x7fae300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:44.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.746+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30035040 con 0x7fae481de000 2024-03-09T09:23:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.747+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7fae3004cad0 con 0x7fae481de000 2024-03-09T09:23:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.747+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae481de000 msgr2=0x7fae481de420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.747+0000 7fae3ea00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae481de000 0x7fae481de420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae30009870 tx=0x7fae300046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.747+0000 7fae3ea00640 1 -- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae481de000 msgr2=0x7fae481de420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.747+0000 7fae3ea00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae481de000 0x7fae481de420 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae30009870 tx=0x7fae300046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.755 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:23:44.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:23:44.755 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:23:44.755 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:23:44.755 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 246, 2024-03-09T09:23:44.755 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:23:44.755 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:23:44.755 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:23:44.755 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:23:44.755 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.755 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:44.756 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:44.757 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:44.758 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:44.758 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:44.758 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.758 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:23:44.758 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:23:44.758 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:23:44.758 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:23:44.758 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:23:44.758 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:23:44.759 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:44.759 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:44.759 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:23:44.759 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:23:44.759 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:44.760 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:44.761 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:44.761 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:44.761 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:44.761 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:44.761 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:44.761 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.761 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:23:44.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.761 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:23:44.761 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.762 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:23:44.762 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:23:44.762 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:44.762 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:44.762 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.762 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:44.762 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:23:44.762 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.763 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.763 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:44.763 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:23:44.763 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.763 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.763 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.764 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:23:44.764 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:23:44.764 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:44.764 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:44.764 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:44.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.764 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.765 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:23:44.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:23:44.765 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:44.765 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:44.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.765 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:44.765 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:23:44.765 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.765 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:44.766 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:23:44.766 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.766 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.766 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.766 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:23:44.766 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:23:44.766 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:44.766 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:44.767 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:44.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.770 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.770 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:23:44.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:23:44.771 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:44.771 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:44.771 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:44.772 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:23:44.772 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:44.773 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:23:44.773 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:44.773 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.773 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.774 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:23:44.774 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:23:44.774 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:44.774 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:44.775 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:44.775 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.775 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.776 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:23:44.776 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:23:44.776 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.776 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.776 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.777 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:44.777 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.777 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.777 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:23:44.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.777 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.777 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.778 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:23:44.778 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.778 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.778 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:23:44.778 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.778 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.778 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.779 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:23:44.779 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.779 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.779 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:23:44.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.779 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:23:44.779 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:44.780 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:23:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:44.781 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:44.782 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:23:44.782 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:23:44.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.747+0000 7fae4ca00640 1 -- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fae2406b3d0 msgr2=0x7fae2406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.747+0000 7fae4ca00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fae2406b3d0 0x7fae2406d890 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fae40006fd0 tx=0x7fae40008040 comp rx=0 tx=0).stop 2024-03-09T09:23:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.747+0000 7fae4ca00640 1 -- 172.21.15.67:0/2410570298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae4806ec60 msgr2=0x7fae4806e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.747+0000 7fae4ca00640 1 --2- 172.21.15.67:0/2410570298 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae4806ec60 0x7fae4806e4c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c7f0 tx=0x7fae3800ccb0 comp rx=0 tx=0).stop 2024-03-09T09:23:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.747+0000 7fae46800640 1 -- 172.21.15.67:0/2410570298 reap_dead start 2024-03-09T09:23:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.748+0000 7fae46800640 1 -- 172.21.15.67:0/2410570298 reap_dead start 2024-03-09T09:23:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.748+0000 7fae4ca00640 1 -- 172.21.15.67:0/2410570298 shutdown_connections 2024-03-09T09:23:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.748+0000 7fae4ca00640 1 -- 172.21.15.67:0/2410570298 >> 172.21.15.67:0/2410570298 conn(0x7fae480c7a90 msgr2=0x7fae481570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.748+0000 7fae4ca00640 1 -- 172.21.15.67:0/2410570298 shutdown_connections 2024-03-09T09:23:44.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.748+0000 7fae4ca00640 1 -- 172.21.15.67:0/2410570298 wait complete. 2024-03-09T09:23:44.786 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:23:44.786 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:23:44.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:23:44.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.987+0000 7f616b200640 1 Processor -- start 2024-03-09T09:23:44.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.987+0000 7f616b200640 1 -- start start 2024-03-09T09:23:44.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.989+0000 7f616b200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f616c0a6d00 0x7f616c06e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.989+0000 7f616b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f616c0c4e10 0x7f616c155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:44.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.989+0000 7f616b200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616c0c5260 con 0x7f616c14f7e0 2024-03-09T09:23:44.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.990+0000 7f616b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616c153d70 con 0x7f616c0c4e10 2024-03-09T09:23:44.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.990+0000 7f616b200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616c14fbc0 con 0x7f616c0a6d00 2024-03-09T09:23:44.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.990+0000 7f616a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f616c0c4e10 0x7f616c155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:44.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.990+0000 7f6169e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f616c14f7e0 0x7f616c153660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:44742/0 (socket says 172.21.15.67:44742) 2024-03-09T09:23:44.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.990+0000 7f616a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f616c0c4e10 0x7f616c155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53296/0 (socket says 172.21.15.67:53296) 2024-03-09T09:23:44.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.990+0000 7f6169e00640 1 -- 172.21.15.67:0/167926513 learned_addr learned my addr 172.21.15.67:0/167926513 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:23:44.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.991+0000 7f6169400640 1 --2- 172.21.15.67:0/167926513 >> v2:172.21.15.134:3300/0 conn(0x7f616c0a6d00 0x7f616c06e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.991+0000 7f6168a00640 1 -- 172.21.15.67:0/167926513 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528341869 0 0) 0x7f616c0c5260 con 0x7f616c14f7e0 2024-03-09T09:23:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.991+0000 7f6168a00640 1 -- 172.21.15.67:0/167926513 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6150005730 con 0x7f616c14f7e0 2024-03-09T09:23:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.991+0000 7f616a800640 1 -- 172.21.15.67:0/167926513 >> v2:172.21.15.134:3300/0 conn(0x7f616c0a6d00 msgr2=0x7f616c06e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.991+0000 7f616a800640 1 --2- 172.21.15.67:0/167926513 >> v2:172.21.15.134:3300/0 conn(0x7f616c0a6d00 0x7f616c06e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.991+0000 7f616a800640 1 -- 172.21.15.67:0/167926513 >> v1:172.21.15.134:6789/0 conn(0x7f616c14f7e0 legacy=0x7f616c153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.991+0000 7f616a800640 1 -- 172.21.15.67:0/167926513 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c0a7490 con 0x7f616c0c4e10 2024-03-09T09:23:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.993+0000 7f616a800640 1 --2- 172.21.15.67:0/167926513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f616c0c4e10 0x7f616c155760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f616c0c5550 tx=0x7f616402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70e226605b2dc08 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.993+0000 7f6168a00640 1 -- 172.21.15.67:0/167926513 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164039070 con 0x7f616c0c4e10 2024-03-09T09:23:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.993+0000 7f6168a00640 1 -- 172.21.15.67:0/167926513 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616402cb60 con 0x7f616c0c4e10 2024-03-09T09:23:45.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.993+0000 7f6168a00640 1 -- 172.21.15.67:0/167926513 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164034440 con 0x7f616c0c4e10 2024-03-09T09:23:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.994+0000 7f616b200640 1 -- 172.21.15.67:0/167926513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f616c0c4e10 msgr2=0x7f616c155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.994+0000 7f616b200640 1 --2- 172.21.15.67:0/167926513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f616c0c4e10 0x7f616c155760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f616c0c5550 tx=0x7f616402bdb0 comp rx=0 tx=0).stop 2024-03-09T09:23:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.994+0000 7f616b200640 1 -- 172.21.15.67:0/167926513 shutdown_connections 2024-03-09T09:23:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.994+0000 7f616b200640 1 --2- 172.21.15.67:0/167926513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f616c0c4e10 0x7f616c155760 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.994+0000 7f616b200640 1 --2- 172.21.15.67:0/167926513 >> v2:172.21.15.134:3300/0 conn(0x7f616c0a6d00 0x7f616c06e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.994+0000 7f616b200640 1 -- 172.21.15.67:0/167926513 >> 172.21.15.67:0/167926513 conn(0x7f616c06b040 msgr2=0x7f616c06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.994+0000 7f616b200640 1 -- 172.21.15.67:0/167926513 shutdown_connections 2024-03-09T09:23:45.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.994+0000 7f616b200640 1 -- 172.21.15.67:0/167926513 wait complete. 2024-03-09T09:23:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.996+0000 7f616b200640 1 Processor -- start 2024-03-09T09:23:45.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.996+0000 7f616b200640 1 -- start start 2024-03-09T09:23:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.997+0000 7f616b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f616c0a6d00 0x7f616c1e0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.997+0000 7f616b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f616c0c4e10 0x7f616c1e0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:45.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.997+0000 7f616b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f616c14f7e0 0x7f616c1dda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.997+0000 7f616b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c0c4330 con 0x7f616c0c4e10 2024-03-09T09:23:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.997+0000 7f616b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c156580 con 0x7f616c0a6d00 2024-03-09T09:23:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.997+0000 7f616b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616c0c4610 con 0x7f616c14f7e0 2024-03-09T09:23:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.997+0000 7f6169400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f616c0c4e10 0x7f616c1e0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.997+0000 7f6169400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f616c0c4e10 0x7f616c1e0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53308/0 (socket says 172.21.15.67:53308) 2024-03-09T09:23:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.998+0000 7f6169400640 1 -- 172.21.15.67:0/343609273 learned_addr learned my addr 172.21.15.67:0/343609273 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.998+0000 7f6169e00640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f616c0a6d00 0x7f616c1e0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:45.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.998+0000 7f616a800640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f616c14f7e0 0x7f616c1dda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.998+0000 7f6169400640 1 -- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f616c14f7e0 msgr2=0x7f616c1dda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.998+0000 7f6169400640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f616c14f7e0 0x7f616c1dda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.998+0000 7f6169400640 1 -- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f616c0a6d00 msgr2=0x7f616c1e0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.998+0000 7f6169400640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f616c0a6d00 0x7f616c1e0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.998+0000 7f6169400640 1 -- 172.21.15.67:0/343609273 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c1e02f0 con 0x7f616c0c4e10 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.998+0000 7f6169e00640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f616c0a6d00 0x7f616c1e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.998+0000 7f6169400640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f616c0c4e10 0x7f616c1e0f50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f615800e9b0 tx=0x7f615800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.999+0000 7f6162000640 1 -- 172.21.15.67:0/343609273 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158017070 con 0x7f616c0c4e10 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.999+0000 7f6162000640 1 -- 172.21.15.67:0/343609273 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615800c9a0 con 0x7f616c0c4e10 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.999+0000 7f6162000640 1 -- 172.21.15.67:0/343609273 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800cd90 con 0x7f616c0c4e10 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.999+0000 7f616b200640 1 -- 172.21.15.67:0/343609273 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616c1ef870 con 0x7f616c0c4e10 2024-03-09T09:23:45.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:44.999+0000 7f616b200640 1 -- 172.21.15.67:0/343609273 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616c1dee90 con 0x7f616c0c4e10 2024-03-09T09:23:45.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.000+0000 7f616b200640 1 -- 172.21.15.67:0/343609273 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616c04f3c0 con 0x7f616c0c4e10 2024-03-09T09:23:45.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.004+0000 7f6162000640 1 -- 172.21.15.67:0/343609273 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f615801a070 con 0x7f616c0c4e10 2024-03-09T09:23:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.005+0000 7f6162000640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f614c06b4d0 0x7f614c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.005+0000 7f6162000640 1 -- 172.21.15.67:0/343609273 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f615808a640 con 0x7f616c0c4e10 2024-03-09T09:23:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.006+0000 7f6169e00640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f614c06b4d0 0x7f614c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:45.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.007+0000 7f6169e00640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f614c06b4d0 0x7f614c06d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f616c0c5460 tx=0x7f615c007c80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.011+0000 7f6162000640 1 -- 172.21.15.67:0/343609273 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f61580590c0 con 0x7f616c0c4e10 2024-03-09T09:23:45.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.339+0000 7f616b200640 1 -- 172.21.15.67:0/343609273 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f616c0acf60 con 0x7f616c0c4e10 2024-03-09T09:23:45.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.341+0000 7f6162000640 1 -- 172.21.15.67:0/343609273 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f61580589c0 con 0x7f616c0c4e10 2024-03-09T09:23:45.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.345+0000 7f616b200640 1 -- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f614c06b4d0 msgr2=0x7f614c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:45.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.345+0000 7f616b200640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f614c06b4d0 0x7f614c06d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f616c0c5460 tx=0x7f615c007c80 comp rx=0 tx=0).stop 2024-03-09T09:23:45.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.345+0000 7f616b200640 1 -- 172.21.15.67:0/343609273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f616c0c4e10 msgr2=0x7f616c1e0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:45.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.345+0000 7f616b200640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f616c0c4e10 0x7f616c1e0f50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f615800e9b0 tx=0x7f615800ee70 comp rx=0 tx=0).stop 2024-03-09T09:23:45.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.345+0000 7f616b200640 1 -- 172.21.15.67:0/343609273 shutdown_connections 2024-03-09T09:23:45.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.345+0000 7f616b200640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f614c06b4d0 0x7f614c06d990 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:45.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.345+0000 7f616b200640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f616c14f7e0 0x7f616c1dda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:45.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.345+0000 7f616b200640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f616c0c4e10 0x7f616c1e0f50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:45.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.345+0000 7f616b200640 1 --2- 172.21.15.67:0/343609273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f616c0a6d00 0x7f616c1e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:45.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.345+0000 7f616b200640 1 -- 172.21.15.67:0/343609273 >> 172.21.15.67:0/343609273 conn(0x7f616c06b040 msgr2=0x7f616c148ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:45.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.345+0000 7f616b200640 1 -- 172.21.15.67:0/343609273 shutdown_connections 2024-03-09T09:23:45.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:45.346+0000 7f616b200640 1 -- 172.21.15.67:0/343609273 wait complete. 2024-03-09T09:23:45.378 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:23:48.810 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:23:48.810 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:23:48.810 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 75 -eq 48 ']' 2024-03-09T09:23:48.811 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.75 2024-03-09T09:23:48.811 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.75 2024-03-09T09:23:48.811 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.75 2024-03-09T09:23:48.811 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.75 2024-03-09T09:23:48.815 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:23:49.821 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:23:49.821 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:23:49.821 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 81 -eq 48 ']' 2024-03-09T09:23:49.821 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.81 2024-03-09T09:23:49.821 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.81 2024-03-09T09:23:49.822 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.81 2024-03-09T09:23:49.822 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.81 2024-03-09T09:23:49.827 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:23:54.953 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:23:54.954 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:23:54.954 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 82 -eq 48 ']' 2024-03-09T09:23:54.954 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.82 2024-03-09T09:23:54.954 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.82 2024-03-09T09:23:54.954 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.82 2024-03-09T09:23:54.954 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.82 2024-03-09T09:23:54.958 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:23:55.379 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:23:55.379 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:23:55.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.602+0000 7f6c8bc00640 1 Processor -- start 2024-03-09T09:23:55.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.602+0000 7f6c8bc00640 1 -- start start 2024-03-09T09:23:55.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.603+0000 7f6c8bc00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6c840c5f50 0x7f6c84155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:55.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.603+0000 7f6c8bc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6c8406b040 0x7f6c8406c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:55.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.603+0000 7f6c8bc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c840c6a40 con 0x7f6c840c96f0 2024-03-09T09:23:55.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.603+0000 7f6c8bc00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8406b5b0 con 0x7f6c840c5f50 2024-03-09T09:23:55.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.603+0000 7f6c8bc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8406b890 con 0x7f6c8406b040 2024-03-09T09:23:55.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.604+0000 7f6c89200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6c840c5f50 0x7f6c84155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:55.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.604+0000 7f6c89200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f6c840c5f50 0x7f6c84155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50422/0 (socket says 172.21.15.67:50422) 2024-03-09T09:23:55.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.604+0000 7f6c89c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f6c840c96f0 0x7f6c840c6330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41024/0 (socket says 172.21.15.67:41024) 2024-03-09T09:23:55.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.604+0000 7f6c89200640 1 -- 172.21.15.67:0/2376103933 learned_addr learned my addr 172.21.15.67:0/2376103933 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:55.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.604+0000 7f6c83e00640 1 --2- 172.21.15.67:0/2376103933 >> v2:172.21.15.134:3300/0 conn(0x7f6c8406b040 0x7f6c8406c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:55.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.604+0000 7f6c89200640 1 -- 172.21.15.67:0/2376103933 >> v2:172.21.15.134:3300/0 conn(0x7f6c8406b040 msgr2=0x7f6c8406c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:55.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.604+0000 7f6c89200640 1 --2- 172.21.15.67:0/2376103933 >> v2:172.21.15.134:3300/0 conn(0x7f6c8406b040 0x7f6c8406c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:55.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.604+0000 7f6c89200640 1 -- 172.21.15.67:0/2376103933 >> v1:172.21.15.67:6789/0 conn(0x7f6c840c96f0 legacy=0x7f6c840c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:55.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.604+0000 7f6c89200640 1 -- 172.21.15.67:0/2376103933 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8406bb70 con 0x7f6c840c5f50 2024-03-09T09:23:55.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.605+0000 7f6c89200640 1 --2- 172.21.15.67:0/2376103933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c840c5f50 0x7f6c84155e20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6c8408d460 tx=0x7f6c7c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766f6d0a4103a644 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:55.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.606+0000 7f6c83400640 1 -- 172.21.15.67:0/2376103933 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c02cb10 con 0x7f6c840c5f50 2024-03-09T09:23:55.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.606+0000 7f6c83400640 1 -- 172.21.15.67:0/2376103933 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c02ccb0 con 0x7f6c840c5f50 2024-03-09T09:23:55.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.606+0000 7f6c83400640 1 -- 172.21.15.67:0/2376103933 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c02ce60 con 0x7f6c840c5f50 2024-03-09T09:23:55.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.606+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2376103933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c840c5f50 msgr2=0x7f6c84155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:55.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.606+0000 7f6c8bc00640 1 --2- 172.21.15.67:0/2376103933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c840c5f50 0x7f6c84155e20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6c8408d460 tx=0x7f6c7c02bf70 comp rx=0 tx=0).stop 2024-03-09T09:23:55.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.607+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2376103933 shutdown_connections 2024-03-09T09:23:55.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.607+0000 7f6c8bc00640 1 --2- 172.21.15.67:0/2376103933 >> v2:172.21.15.134:3300/0 conn(0x7f6c8406b040 0x7f6c8406c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:55.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.607+0000 7f6c8bc00640 1 --2- 172.21.15.67:0/2376103933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c840c5f50 0x7f6c84155e20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:55.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.607+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2376103933 >> 172.21.15.67:0/2376103933 conn(0x7f6c840c8050 msgr2=0x7f6c840c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:55.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.607+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2376103933 shutdown_connections 2024-03-09T09:23:55.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.607+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2376103933 wait complete. 2024-03-09T09:23:55.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.608+0000 7f6c8bc00640 1 Processor -- start 2024-03-09T09:23:55.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.609+0000 7f6c8bc00640 1 -- start start 2024-03-09T09:23:55.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c8bc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8406b040 0x7f6c8414b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:55.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c8bc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c840c5f50 0x7f6c841e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:55.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c8bc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c840c96f0 0x7f6c841df690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:55.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c8bc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8414cc00 con 0x7f6c8406b040 2024-03-09T09:23:55.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c8bc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8406c130 con 0x7f6c840c96f0 2024-03-09T09:23:55.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c8bc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c840c9b60 con 0x7f6c840c5f50 2024-03-09T09:23:55.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c89200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8406b040 0x7f6c8414b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:55.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c89200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8406b040 0x7f6c8414b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50434/0 (socket says 172.21.15.67:50434) 2024-03-09T09:23:55.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c89200640 1 -- 172.21.15.67:0/2807399846 learned_addr learned my addr 172.21.15.67:0/2807399846 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c83e00640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c840c5f50 0x7f6c841e2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c89200640 1 -- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c840c5f50 msgr2=0x7f6c841e2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c89200640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c840c5f50 0x7f6c841e2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c89200640 1 -- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c840c96f0 msgr2=0x7f6c841df690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:55.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c89c00640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c840c96f0 0x7f6c841df690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c89200640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c840c96f0 0x7f6c841df690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.610+0000 7f6c89200640 1 -- 172.21.15.67:0/2807399846 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c840bf230 con 0x7f6c8406b040 2024-03-09T09:23:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.611+0000 7f6c89200640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8406b040 0x7f6c8414b9c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009f40 tx=0x7f6c7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.611+0000 7f6c80c00640 1 -- 172.21.15.67:0/2807399846 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c043020 con 0x7f6c8406b040 2024-03-09T09:23:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.611+0000 7f6c80c00640 1 -- 172.21.15.67:0/2807399846 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c03e450 con 0x7f6c8406b040 2024-03-09T09:23:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.611+0000 7f6c80c00640 1 -- 172.21.15.67:0/2807399846 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c02ce60 con 0x7f6c8406b040 2024-03-09T09:23:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.611+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2807399846 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c841ebca0 con 0x7f6c8406b040 2024-03-09T09:23:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.611+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2807399846 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c841ec260 con 0x7f6c8406b040 2024-03-09T09:23:55.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.612+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2807399846 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c84075a00 con 0x7f6c8406b040 2024-03-09T09:23:55.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.613+0000 7f6c80c00640 1 -- 172.21.15.67:0/2807399846 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6c7c005d10 con 0x7f6c8406b040 2024-03-09T09:23:55.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.615+0000 7f6c80c00640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c6006b4d0 0x7f6c6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:55.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.615+0000 7f6c80c00640 1 -- 172.21.15.67:0/2807399846 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f6c7c0acfd0 con 0x7f6c8406b040 2024-03-09T09:23:55.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.615+0000 7f6c83e00640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c6006b4d0 0x7f6c6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:55.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.617+0000 7f6c83e00640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c6006b4d0 0x7f6c6006d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6c840c22d0 tx=0x7f6c6c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:55.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.620+0000 7f6c80c00640 1 -- 172.21.15.67:0/2807399846 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c7c07ba50 con 0x7f6c8406b040 2024-03-09T09:23:55.664 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:23:55.664 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:23:55.664 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 76 -eq 48 ']' 2024-03-09T09:23:55.665 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.76 2024-03-09T09:23:55.665 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.76 2024-03-09T09:23:55.665 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.76 2024-03-09T09:23:55.665 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.76 2024-03-09T09:23:55.669 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:23:55.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.985+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2807399846 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6c840ac1f0 con 0x7f6c8406b040 2024-03-09T09:23:55.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.986+0000 7f6c80c00640 1 -- 172.21.15.67:0/2807399846 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f6c7c07b350 con 0x7f6c8406b040 2024-03-09T09:23:55.993 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":246,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:23:55.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.988+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c6006b4d0 msgr2=0x7f6c6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:55.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.988+0000 7f6c8bc00640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c6006b4d0 0x7f6c6006d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6c840c22d0 tx=0x7f6c6c008040 comp rx=0 tx=0).stop 2024-03-09T09:23:55.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.988+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2807399846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8406b040 msgr2=0x7f6c8414b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:55.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.988+0000 7f6c8bc00640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8406b040 0x7f6c8414b9c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009f40 tx=0x7f6c7c004a40 comp rx=0 tx=0).stop 2024-03-09T09:23:55.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.988+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2807399846 shutdown_connections 2024-03-09T09:23:55.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.988+0000 7f6c8bc00640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6c6006b4d0 0x7f6c6006d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:55.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.988+0000 7f6c8bc00640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c840c96f0 0x7f6c841df690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:55.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.988+0000 7f6c8bc00640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c840c5f50 0x7f6c841e2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:55.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.988+0000 7f6c8bc00640 1 --2- 172.21.15.67:0/2807399846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c8406b040 0x7f6c8414b9c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:55.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.988+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2807399846 >> 172.21.15.67:0/2807399846 conn(0x7f6c840c8050 msgr2=0x7f6c84158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:55.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.989+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2807399846 shutdown_connections 2024-03-09T09:23:55.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:55.989+0000 7f6c8bc00640 1 -- 172.21.15.67:0/2807399846 wait complete. 2024-03-09T09:23:56.018 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:23:56.018 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:23:56.018 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:23:56.018 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:23:56.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.215+0000 7f3199000640 1 Processor -- start 2024-03-09T09:23:56.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.215+0000 7f3199000640 1 -- start start 2024-03-09T09:23:56.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.216+0000 7f3199000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f319406ec60 0x7f31940c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.216+0000 7f3199000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3194156370 con 0x7f3194067160 2024-03-09T09:23:56.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.216+0000 7f3199000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31940a71f0 con 0x7f31940c7b90 2024-03-09T09:23:56.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.216+0000 7f3199000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31940c5b90 con 0x7f319406ec60 2024-03-09T09:23:56.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.217+0000 7f3192a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f319406ec60 0x7f31940c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.217+0000 7f3193400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f31940c7b90 0x7f3194155c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:56538/0 (socket says 172.21.15.67:56538) 2024-03-09T09:23:56.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.217+0000 7f3193e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3194067160 0x7f319406dfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41030/0 (socket says 172.21.15.67:41030) 2024-03-09T09:23:56.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.217+0000 7f3193400640 1 -- 172.21.15.67:0/477826991 learned_addr learned my addr 172.21.15.67:0/477826991 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:23:56.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.217+0000 7f3192a00640 1 -- 172.21.15.67:0/477826991 >> v1:172.21.15.134:6789/0 conn(0x7f31940c7b90 legacy=0x7f3194155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.217+0000 7f3192a00640 1 -- 172.21.15.67:0/477826991 >> v1:172.21.15.67:6789/0 conn(0x7f3194067160 legacy=0x7f319406dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.217+0000 7f3192a00640 1 -- 172.21.15.67:0/477826991 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31940a6dd0 con 0x7f319406ec60 2024-03-09T09:23:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.218+0000 7f3192a00640 1 --2- 172.21.15.67:0/477826991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f319406ec60 0x7f31940c2650 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f31800097a0 tx=0x7f318002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9880ab17ecf5b6e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.219+0000 7f3192000640 1 -- 172.21.15.67:0/477826991 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31800315a0 con 0x7f319406ec60 2024-03-09T09:23:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.219+0000 7f3192000640 1 -- 172.21.15.67:0/477826991 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3180037070 con 0x7f319406ec60 2024-03-09T09:23:56.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.219+0000 7f3192000640 1 -- 172.21.15.67:0/477826991 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180031960 con 0x7f319406ec60 2024-03-09T09:23:56.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.219+0000 7f3199000640 1 -- 172.21.15.67:0/477826991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f319406ec60 msgr2=0x7f31940c2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.219+0000 7f3199000640 1 --2- 172.21.15.67:0/477826991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f319406ec60 0x7f31940c2650 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f31800097a0 tx=0x7f318002b840 comp rx=0 tx=0).stop 2024-03-09T09:23:56.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.219+0000 7f3199000640 1 -- 172.21.15.67:0/477826991 shutdown_connections 2024-03-09T09:23:56.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.219+0000 7f3199000640 1 --2- 172.21.15.67:0/477826991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f319406ec60 0x7f31940c2650 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.219+0000 7f3199000640 1 -- 172.21.15.67:0/477826991 >> 172.21.15.67:0/477826991 conn(0x7f319406ba00 msgr2=0x7f31940a9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:56.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.219+0000 7f3199000640 1 -- 172.21.15.67:0/477826991 shutdown_connections 2024-03-09T09:23:56.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.220+0000 7f3199000640 1 -- 172.21.15.67:0/477826991 wait complete. 2024-03-09T09:23:56.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.221+0000 7f3199000640 1 Processor -- start 2024-03-09T09:23:56.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.221+0000 7f3199000640 1 -- start start 2024-03-09T09:23:56.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.222+0000 7f3199000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3194067160 0x7f31941df2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.222+0000 7f3199000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f319406ec60 0x7f31940c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.222+0000 7f3199000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31940c7b90 0x7f31940720c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.222+0000 7f3199000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319406e6d0 con 0x7f319406ec60 2024-03-09T09:23:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.222+0000 7f3199000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940a74d0 con 0x7f31940c7b90 2024-03-09T09:23:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.222+0000 7f3199000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319406e850 con 0x7f3194067160 2024-03-09T09:23:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.222+0000 7f3192a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f319406ec60 0x7f31940c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.222+0000 7f3192a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f319406ec60 0x7f31940c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50448/0 (socket says 172.21.15.67:50448) 2024-03-09T09:23:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.222+0000 7f3192a00640 1 -- 172.21.15.67:0/2097721294 learned_addr learned my addr 172.21.15.67:0/2097721294 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.223+0000 7f3193400640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3194067160 0x7f31941df2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.223+0000 7f3193e00640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31940c7b90 0x7f31940720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.223+0000 7f3192a00640 1 -- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3194067160 msgr2=0x7f31941df2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.223+0000 7f3192a00640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3194067160 0x7f31941df2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.223+0000 7f3192a00640 1 -- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31940c7b90 msgr2=0x7f31940720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.223+0000 7f3192a00640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31940c7b90 0x7f31940720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.223+0000 7f3192a00640 1 -- 172.21.15.67:0/2097721294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31941ef710 con 0x7f319406ec60 2024-03-09T09:23:56.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.223+0000 7f3193e00640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31940c7b90 0x7f31940720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:23:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.223+0000 7f3192a00640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f319406ec60 0x7f31940c4690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3180034040 tx=0x7f31800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.224+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180032840 con 0x7f319406ec60 2024-03-09T09:23:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.224+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3180032a00 con 0x7f319406ec60 2024-03-09T09:23:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.224+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180032df0 con 0x7f319406ec60 2024-03-09T09:23:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.224+0000 7f3199000640 1 -- 172.21.15.67:0/2097721294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31940713d0 con 0x7f319406ec60 2024-03-09T09:23:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.224+0000 7f3199000640 1 -- 172.21.15.67:0/2097721294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31940c4bd0 con 0x7f319406ec60 2024-03-09T09:23:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.225+0000 7f3199000640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194152da0 0x7f3194072600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.225+0000 7f3199000640 1 -- 172.21.15.67:0/2097721294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940718f0 con 0x7f3194152da0 2024-03-09T09:23:56.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.225+0000 7f3193400640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194152da0 0x7f3194072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.226+0000 7f3193400640 1 -- 172.21.15.67:0/2097721294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f319404f3c0 con 0x7f3194152da0 2024-03-09T09:23:56.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.227+0000 7f3193400640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194152da0 0x7f3194072600 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3184000f60 tx=0x7f318400f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.227+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184005ac0 con 0x7f3194152da0 2024-03-09T09:23:56.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.227+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f318004e430 con 0x7f319406ec60 2024-03-09T09:23:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.228+0000 7f318b400640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f317406b3d0 0x7f317406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.228+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f318002f030 con 0x7f319406ec60 2024-03-09T09:23:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.228+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f3184018070 con 0x7f3194152da0 2024-03-09T09:23:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.228+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194152da0 msgr2=0x7f3194072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.228+0000 7f318b400640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194152da0 0x7f3194072600 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3184000f60 tx=0x7f318400f9d0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.228+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194152da0 msgr2=0x7f3194072600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.228+0000 7f318b400640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194152da0 0x7f3194072600 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3184000f60 tx=0x7f318400f9d0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.229+0000 7f3193e00640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f317406b3d0 0x7f317406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.230+0000 7f3193e00640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f317406b3d0 0x7f317406d890 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f318c0012e0 tx=0x7f318c005bb0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.236+0000 7f3199000640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194072b40 0x7f3194154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.236+0000 7f3199000640 1 -- 172.21.15.67:0/2097721294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3194066f40 con 0x7f3194072b40 2024-03-09T09:23:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.236+0000 7f3193400640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194072b40 0x7f3194154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.237+0000 7f3193400640 1 -- 172.21.15.67:0/2097721294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f31940c83f0 con 0x7f3194072b40 2024-03-09T09:23:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.237+0000 7f3193400640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194072b40 0x7f3194154ad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3184000f60 tx=0x7f31840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.237+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31840076f0 con 0x7f3194072b40 2024-03-09T09:23:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4537 (secure 0 0 0) 0x7f31840076f0 con 0x7f3194072b40 2024-03-09T09:23:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194072b40 msgr2=0x7f3194154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f318b400640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194072b40 0x7f3194154ad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3184000f60 tx=0x7f31840057f0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f318b400640 1 -- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194072b40 msgr2=0x7f3194154ad0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:56.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f318b400640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3194072b40 0x7f3194154ad0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.245 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:23:56.245 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:23:56.245 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:23:56.245 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:23:56.245 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 246, 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:56.246 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:56.247 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:23:56.248 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:23:56.249 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:23:56.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:56.251 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:23:56.252 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:56.253 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:56.253 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.253 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:56.253 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:23:56.253 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.253 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.253 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:56.253 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:23:56.253 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:23:56.254 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:56.256 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.257 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.257 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:23:56.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.257 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.257 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.257 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:23:56.257 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.257 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.257 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:23:56.257 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:23:56.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f3199000640 1 -- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f317406b3d0 msgr2=0x7f317406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f3199000640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f317406b3d0 0x7f317406d890 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f318c0012e0 tx=0x7f318c005bb0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f3199000640 1 -- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f319406ec60 msgr2=0x7f31940c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f3199000640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f319406ec60 0x7f31940c4690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3180034040 tx=0x7f31800046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f3193e00640 1 -- 172.21.15.67:0/2097721294 reap_dead start 2024-03-09T09:23:56.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f3199000640 1 -- 172.21.15.67:0/2097721294 shutdown_connections 2024-03-09T09:23:56.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f3199000640 1 --2- 172.21.15.67:0/2097721294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f319406ec60 0x7f31940c4690 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f3199000640 1 -- 172.21.15.67:0/2097721294 >> 172.21.15.67:0/2097721294 conn(0x7f319406ba00 msgr2=0x7f31940ac6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:56.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f3199000640 1 -- 172.21.15.67:0/2097721294 shutdown_connections 2024-03-09T09:23:56.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.238+0000 7f3199000640 1 -- 172.21.15.67:0/2097721294 wait complete. 2024-03-09T09:23:56.261 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:23:56.261 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:23:56.261 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:23:56.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.460+0000 7f5441a00640 1 Processor -- start 2024-03-09T09:23:56.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.460+0000 7f5441a00640 1 -- start start 2024-03-09T09:23:56.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.462+0000 7f5441a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f543c0c9380 0x7f543c0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.462+0000 7f5441a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f543c0ad440 0x7f543c06c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.462+0000 7f5441a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543c0c9890 con 0x7f543c14fd90 2024-03-09T09:23:56.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.462+0000 7f5441a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543c06ba60 con 0x7f543c0c9380 2024-03-09T09:23:56.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.462+0000 7f5441a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0c5f80 con 0x7f543c0ad440 2024-03-09T09:23:56.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.462+0000 7f543be00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f543c14fd90 0x7f543c156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41034/0 (socket says 172.21.15.67:41034) 2024-03-09T09:23:56.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.462+0000 7f543be00640 1 -- 172.21.15.67:0/2711276632 learned_addr learned my addr 172.21.15.67:0/2711276632 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:23:56.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.462+0000 7f543b400640 1 --2- 172.21.15.67:0/2711276632 >> v2:172.21.15.67:3300/0 conn(0x7f543c0c9380 0x7f543c0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.462+0000 7f5441000640 1 --2- 172.21.15.67:0/2711276632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f543c0ad440 0x7f543c06c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.463+0000 7f543b400640 1 -- 172.21.15.67:0/2711276632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f543c0ad440 msgr2=0x7f543c06c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:56.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.463+0000 7f543b400640 1 --2- 172.21.15.67:0/2711276632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f543c0ad440 0x7f543c06c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.463+0000 7f543b400640 1 -- 172.21.15.67:0/2711276632 >> v1:172.21.15.67:6789/0 conn(0x7f543c14fd90 legacy=0x7f543c156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.463+0000 7f543b400640 1 -- 172.21.15.67:0/2711276632 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c0c6240 con 0x7f543c0c9380 2024-03-09T09:23:56.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.463+0000 7f543aa00640 1 -- 172.21.15.67:0/2711276632 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582427630 0 0) 0x7f543c0c9890 con 0x7f543c14fd90 2024-03-09T09:23:56.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.465+0000 7f543b400640 1 --2- 172.21.15.67:0/2711276632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f543c0c9380 0x7f543c0acf00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f54280097a0 tx=0x7f542802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e53b7bf5b56912 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.465+0000 7f543aa00640 1 -- 172.21.15.67:0/2711276632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428039070 con 0x7f543c0c9380 2024-03-09T09:23:56.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.465+0000 7f543aa00640 1 -- 172.21.15.67:0/2711276632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54280316f0 con 0x7f543c0c9380 2024-03-09T09:23:56.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.465+0000 7f543aa00640 1 -- 172.21.15.67:0/2711276632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428031ac0 con 0x7f543c0c9380 2024-03-09T09:23:56.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.466+0000 7f5441a00640 1 -- 172.21.15.67:0/2711276632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f543c0c9380 msgr2=0x7f543c0acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.466+0000 7f5441a00640 1 --2- 172.21.15.67:0/2711276632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f543c0c9380 0x7f543c0acf00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f54280097a0 tx=0x7f542802b790 comp rx=0 tx=0).stop 2024-03-09T09:23:56.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.466+0000 7f5441a00640 1 -- 172.21.15.67:0/2711276632 shutdown_connections 2024-03-09T09:23:56.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.466+0000 7f5441a00640 1 --2- 172.21.15.67:0/2711276632 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f543c0ad440 0x7f543c06c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.466+0000 7f5441a00640 1 --2- 172.21.15.67:0/2711276632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f543c0c9380 0x7f543c0acf00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.466+0000 7f5441a00640 1 -- 172.21.15.67:0/2711276632 >> 172.21.15.67:0/2711276632 conn(0x7f543c0cb480 msgr2=0x7f543c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:56.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.466+0000 7f5441a00640 1 -- 172.21.15.67:0/2711276632 shutdown_connections 2024-03-09T09:23:56.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.466+0000 7f5441a00640 1 -- 172.21.15.67:0/2711276632 wait complete. 2024-03-09T09:23:56.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.469+0000 7f5441a00640 1 Processor -- start 2024-03-09T09:23:56.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.469+0000 7f5441a00640 1 -- start start 2024-03-09T09:23:56.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.470+0000 7f5441a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0ad440 0x7f543c154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.470+0000 7f5441a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f543c0c9380 0x7f543c0ab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.470+0000 7f5441a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f543c14fd90 0x7f543c0ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.470+0000 7f5441a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c06c710 con 0x7f543c14fd90 2024-03-09T09:23:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.470+0000 7f5441a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c06bcb0 con 0x7f543c0ad440 2024-03-09T09:23:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.470+0000 7f5441a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c06c890 con 0x7f543c0c9380 2024-03-09T09:23:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.471+0000 7f5441000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f543c14fd90 0x7f543c0ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.471+0000 7f5441000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f543c14fd90 0x7f543c0ab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50490/0 (socket says 172.21.15.67:50490) 2024-03-09T09:23:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.471+0000 7f5441000640 1 -- 172.21.15.67:0/4067778856 learned_addr learned my addr 172.21.15.67:0/4067778856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.471+0000 7f543be00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0ad440 0x7f543c154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.471+0000 7f5441000640 1 -- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f543c0c9380 msgr2=0x7f543c0ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.471+0000 7f5441000640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f543c0c9380 0x7f543c0ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.471+0000 7f5441000640 1 -- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0ad440 msgr2=0x7f543c154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.471+0000 7f5441000640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0ad440 0x7f543c154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.471+0000 7f5441000640 1 -- 172.21.15.67:0/4067778856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c1ebd80 con 0x7f543c14fd90 2024-03-09T09:23:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.471+0000 7f5441000640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f543c14fd90 0x7f543c0ab7b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5434007e40 tx=0x7f543400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.472+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434014620 con 0x7f543c14fd90 2024-03-09T09:23:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.472+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5434019070 con 0x7f543c14fd90 2024-03-09T09:23:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.472+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434014a20 con 0x7f543c14fd90 2024-03-09T09:23:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.472+0000 7f5441a00640 1 -- 172.21.15.67:0/4067778856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543c1ec060 con 0x7f543c14fd90 2024-03-09T09:23:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.472+0000 7f5441a00640 1 -- 172.21.15.67:0/4067778856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c1ec620 con 0x7f543c14fd90 2024-03-09T09:23:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.473+0000 7f5441a00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0abcf0 0x7f543c0ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.473+0000 7f5441a00640 1 -- 172.21.15.67:0/4067778856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c04f3c0 con 0x7f543c0abcf0 2024-03-09T09:23:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.474+0000 7f543be00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0abcf0 0x7f543c0ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.474+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5434014bc0 con 0x7f543c14fd90 2024-03-09T09:23:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.474+0000 7f543be00640 1 -- 172.21.15.67:0/4067778856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f543c0cb0f0 con 0x7f543c0abcf0 2024-03-09T09:23:56.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.475+0000 7f543be00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0abcf0 0x7f543c0ac2c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f542c001380 tx=0x7f542c001840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.475+0000 7f5433e00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f541c06b4d0 0x7f541c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.476+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f543408af40 con 0x7f543c14fd90 2024-03-09T09:23:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.476+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c019070 con 0x7f543c0abcf0 2024-03-09T09:23:56.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.476+0000 7f543b400640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f541c06b4d0 0x7f541c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.476+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f542c010510 con 0x7f543c0abcf0 2024-03-09T09:23:56.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.476+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0abcf0 msgr2=0x7f543c0ac2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.476+0000 7f5433e00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0abcf0 0x7f543c0ac2c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f542c001380 tx=0x7f542c001840 comp rx=0 tx=0).stop 2024-03-09T09:23:56.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.476+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0abcf0 msgr2=0x7f543c0ac2c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:56.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.476+0000 7f5433e00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0abcf0 0x7f543c0ac2c0 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f542c001380 tx=0x7f542c001840 comp rx=0 tx=0).stop 2024-03-09T09:23:56.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.477+0000 7f543b400640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f541c06b4d0 0x7f541c06d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5428036000 tx=0x7f5428030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.484+0000 7f5441a00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0aade0 0x7f543c0ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.484+0000 7f5441a00640 1 -- 172.21.15.67:0/4067778856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c155460 con 0x7f543c0aade0 2024-03-09T09:23:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.484+0000 7f543be00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0aade0 0x7f543c0ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.485+0000 7f543be00640 1 -- 172.21.15.67:0/4067778856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f543c073a30 con 0x7f543c0aade0 2024-03-09T09:23:56.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.486+0000 7f543be00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0aade0 0x7f543c0ac800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f542c001380 tx=0x7f542c001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.486+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c019040 con 0x7f543c0aade0 2024-03-09T09:23:56.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f542c014480 con 0x7f543c0aade0 2024-03-09T09:23:56.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0aade0 msgr2=0x7f543c0ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5433e00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0aade0 0x7f543c0ac800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f542c001380 tx=0x7f542c001080 comp rx=0 tx=0).stop 2024-03-09T09:23:56.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5433e00640 1 -- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0aade0 msgr2=0x7f543c0ac800 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:56.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5433e00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f543c0aade0 0x7f543c0ac800 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f542c001380 tx=0x7f542c001080 comp rx=0 tx=0).stop 2024-03-09T09:23:56.494 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:23:56.494 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:23:56.494 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:23:56.494 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:23:56.494 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 246, 2024-03-09T09:23:56.494 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:23:56.494 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:56.495 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:56.496 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:23:56.497 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:56.498 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:23:56.499 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.500 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:56.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:56.502 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:56.503 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:23:56.504 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.505 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.506 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:23:56.507 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:23:56.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5441a00640 1 -- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f541c06b4d0 msgr2=0x7f541c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5441a00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f541c06b4d0 0x7f541c06d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5428036000 tx=0x7f5428030000 comp rx=0 tx=0).stop 2024-03-09T09:23:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5441a00640 1 -- 172.21.15.67:0/4067778856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f543c14fd90 msgr2=0x7f543c0ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5441a00640 1 --2- 172.21.15.67:0/4067778856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f543c14fd90 0x7f543c0ab7b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5434007e40 tx=0x7f543400e4b0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5441000640 1 -- 172.21.15.67:0/4067778856 reap_dead start 2024-03-09T09:23:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5441000640 1 -- 172.21.15.67:0/4067778856 reap_dead start 2024-03-09T09:23:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5441a00640 1 -- 172.21.15.67:0/4067778856 shutdown_connections 2024-03-09T09:23:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5441a00640 1 -- 172.21.15.67:0/4067778856 >> 172.21.15.67:0/4067778856 conn(0x7f543c0cb480 msgr2=0x7f543c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.487+0000 7f5441a00640 1 -- 172.21.15.67:0/4067778856 shutdown_connections 2024-03-09T09:23:56.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.488+0000 7f5441a00640 1 -- 172.21.15.67:0/4067778856 wait complete. 2024-03-09T09:23:56.511 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:23:56.511 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:23:56.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:23:56.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.707+0000 7fb412600640 1 Processor -- start 2024-03-09T09:23:56.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.707+0000 7fb412600640 1 -- start start 2024-03-09T09:23:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.708+0000 7fb412600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c06d5f0 0x7fb40c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.708+0000 7fb412600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb40c157b20 0x7fb40c0c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.709+0000 7fb412600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40c154110 con 0x7fb40c150db0 2024-03-09T09:23:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.709+0000 7fb412600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c1543f0 con 0x7fb40c157b20 2024-03-09T09:23:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.709+0000 7fb412600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c1546d0 con 0x7fb40c06d5f0 2024-03-09T09:23:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.709+0000 7fb411c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb40c157b20 0x7fb40c0c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.709+0000 7fb411c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb40c157b20 0x7fb40c0c4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50496/0 (socket says 172.21.15.67:50496) 2024-03-09T09:23:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.709+0000 7fb411200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb40c150db0 0x7fb40c1538c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41040/0 (socket says 172.21.15.67:41040) 2024-03-09T09:23:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.709+0000 7fb411c00640 1 -- 172.21.15.67:0/3194984219 learned_addr learned my addr 172.21.15.67:0/3194984219 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:56.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.709+0000 7fb40be00640 1 --2- 172.21.15.67:0/3194984219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c06d5f0 0x7fb40c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.710+0000 7fb411c00640 1 -- 172.21.15.67:0/3194984219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c06d5f0 msgr2=0x7fb40c1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:23:56.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.710+0000 7fb411c00640 1 --2- 172.21.15.67:0/3194984219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c06d5f0 0x7fb40c1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.710+0000 7fb411c00640 1 -- 172.21.15.67:0/3194984219 >> v1:172.21.15.67:6789/0 conn(0x7fb40c150db0 legacy=0x7fb40c1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.710+0000 7fb411c00640 1 -- 172.21.15.67:0/3194984219 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c1570c0 con 0x7fb40c157b20 2024-03-09T09:23:56.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.710+0000 7fb40b400640 1 -- 172.21.15.67:0/3194984219 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296817765 0 0) 0x7fb40c154110 con 0x7fb40c150db0 2024-03-09T09:23:56.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.711+0000 7fb411c00640 1 --2- 172.21.15.67:0/3194984219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb40c157b20 0x7fb40c0c4ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb404009e50 tx=0x7fb40402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db154cd7ba7e2fa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.711+0000 7fb40b400640 1 -- 172.21.15.67:0/3194984219 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40402cb00 con 0x7fb40c157b20 2024-03-09T09:23:56.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.711+0000 7fb40b400640 1 -- 172.21.15.67:0/3194984219 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40402cca0 con 0x7fb40c157b20 2024-03-09T09:23:56.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.711+0000 7fb40b400640 1 -- 172.21.15.67:0/3194984219 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404034610 con 0x7fb40c157b20 2024-03-09T09:23:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.712+0000 7fb412600640 1 -- 172.21.15.67:0/3194984219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb40c157b20 msgr2=0x7fb40c0c4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.712+0000 7fb412600640 1 --2- 172.21.15.67:0/3194984219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb40c157b20 0x7fb40c0c4ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb404009e50 tx=0x7fb40402bf60 comp rx=0 tx=0).stop 2024-03-09T09:23:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.712+0000 7fb412600640 1 -- 172.21.15.67:0/3194984219 shutdown_connections 2024-03-09T09:23:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.712+0000 7fb412600640 1 --2- 172.21.15.67:0/3194984219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb40c157b20 0x7fb40c0c4ce0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.712+0000 7fb412600640 1 --2- 172.21.15.67:0/3194984219 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c06d5f0 0x7fb40c1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.712+0000 7fb412600640 1 -- 172.21.15.67:0/3194984219 >> 172.21.15.67:0/3194984219 conn(0x7fb40c06c150 msgr2=0x7fb40c06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.712+0000 7fb412600640 1 -- 172.21.15.67:0/3194984219 shutdown_connections 2024-03-09T09:23:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.712+0000 7fb412600640 1 -- 172.21.15.67:0/3194984219 wait complete. 2024-03-09T09:23:56.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.715+0000 7fb412600640 1 Processor -- start 2024-03-09T09:23:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.715+0000 7fb412600640 1 -- start start 2024-03-09T09:23:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb412600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c06d5f0 0x7fb40c1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb412600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb40c150db0 0x7fb40c1ee890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb412600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb40c157b20 0x7fb40c1eedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb412600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0c3560 con 0x7fb40c157b20 2024-03-09T09:23:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb412600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0c33e0 con 0x7fb40c150db0 2024-03-09T09:23:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb412600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0c36e0 con 0x7fb40c06d5f0 2024-03-09T09:23:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb411c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb40c157b20 0x7fb40c1eedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb411c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb40c157b20 0x7fb40c1eedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50510/0 (socket says 172.21.15.67:50510) 2024-03-09T09:23:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb411c00640 1 -- 172.21.15.67:0/3542249323 learned_addr learned my addr 172.21.15.67:0/3542249323 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb411200640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c06d5f0 0x7fb40c1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb40be00640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb40c150db0 0x7fb40c1ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb411c00640 1 -- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c06d5f0 msgr2=0x7fb40c1556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb411c00640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c06d5f0 0x7fb40c1556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb411c00640 1 -- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb40c150db0 msgr2=0x7fb40c1ee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb411c00640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb40c150db0 0x7fb40c1ee890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.717+0000 7fb411c00640 1 -- 172.21.15.67:0/3542249323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c1dd0c0 con 0x7fb40c157b20 2024-03-09T09:23:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.718+0000 7fb411c00640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb40c157b20 0x7fb40c1eedd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb404036040 tx=0x7fb404004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.718+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404043030 con 0x7fb40c157b20 2024-03-09T09:23:56.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.718+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40402ce50 con 0x7fb40c157b20 2024-03-09T09:23:56.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.718+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404009510 con 0x7fb40c157b20 2024-03-09T09:23:56.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.718+0000 7fb412600640 1 -- 172.21.15.67:0/3542249323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c1517d0 con 0x7fb40c157b20 2024-03-09T09:23:56.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.718+0000 7fb412600640 1 -- 172.21.15.67:0/3542249323 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40c155c00 con 0x7fb40c157b20 2024-03-09T09:23:56.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.719+0000 7fb412600640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1dd290 0x7fb40c14cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.719+0000 7fb412600640 1 -- 172.21.15.67:0/3542249323 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c151cf0 con 0x7fb40c1dd290 2024-03-09T09:23:56.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.720+0000 7fb411200640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1dd290 0x7fb40c14cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.721+0000 7fb411200640 1 -- 172.21.15.67:0/3542249323 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb40c04f3c0 con 0x7fb40c1dd290 2024-03-09T09:23:56.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.721+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb404046070 con 0x7fb40c157b20 2024-03-09T09:23:56.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.721+0000 7fb411200640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1dd290 0x7fb40c14cf70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb400001380 tx=0x7fb400001840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.722+0000 7fb408c00640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb3ec06b4d0 0x7fb3ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.722+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb4040ac7e0 con 0x7fb40c157b20 2024-03-09T09:23:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.723+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400019070 con 0x7fb40c1dd290 2024-03-09T09:23:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.723+0000 7fb40be00640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb3ec06b4d0 0x7fb3ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.723+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb400010510 con 0x7fb40c1dd290 2024-03-09T09:23:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.723+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1dd290 msgr2=0x7fb40c14cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.723+0000 7fb408c00640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1dd290 0x7fb40c14cf70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb400001380 tx=0x7fb400001840 comp rx=0 tx=0).stop 2024-03-09T09:23:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.723+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1dd290 msgr2=0x7fb40c14cf70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.723+0000 7fb408c00640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1dd290 0x7fb40c14cf70 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb400001380 tx=0x7fb400001840 comp rx=0 tx=0).stop 2024-03-09T09:23:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.724+0000 7fb40be00640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb3ec06b4d0 0x7fb3ec06d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb40c06c880 tx=0x7fb3f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.730+0000 7fb412600640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1ef310 0x7fb40c14d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.730+0000 7fb412600640 1 -- 172.21.15.67:0/3542249323 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0c31c0 con 0x7fb40c1ef310 2024-03-09T09:23:56.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.731+0000 7fb411200640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1ef310 0x7fb40c14d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:56.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.731+0000 7fb411200640 1 -- 172.21.15.67:0/3542249323 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb40c06c5d0 con 0x7fb40c1ef310 2024-03-09T09:23:56.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.732+0000 7fb411200640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1ef310 0x7fb40c14d4b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb400001380 tx=0x7fb400001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:56.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.732+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400019040 con 0x7fb40c1ef310 2024-03-09T09:23:56.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.733+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7fb40001c6c0 con 0x7fb40c1ef310 2024-03-09T09:23:56.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.733+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1ef310 msgr2=0x7fb40c14d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.733+0000 7fb408c00640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1ef310 0x7fb40c14d4b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb400001380 tx=0x7fb400001080 comp rx=0 tx=0).stop 2024-03-09T09:23:56.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.733+0000 7fb408c00640 1 -- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1ef310 msgr2=0x7fb40c14d4b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:23:56.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.733+0000 7fb408c00640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb40c1ef310 0x7fb40c14d4b0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb400001380 tx=0x7fb400001080 comp rx=0 tx=0).stop 2024-03-09T09:23:56.740 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:23:56.740 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:23:56.740 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:23:56.740 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 246, 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:56.741 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:56.742 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:23:56.743 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:23:56.744 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:56.746 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:56.747 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:23:56.748 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:23:56.749 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:56.749 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:56.749 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:56.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.749 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:23:56.749 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:23:56.749 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:23:56.749 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:23:56.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.749 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:23:56.750 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:23:56.750 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.750 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:23:56.750 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:23:56.750 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:23:56.750 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.750 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.750 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:56.751 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:23:56.752 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:23:56.753 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:23:56.754 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:23:56.754 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:23:56.754 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:23:56.754 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:23:56.754 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:23:56.754 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:23:56.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:23:56.754 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:23:56.754 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:23:56.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.734+0000 7fb412600640 1 -- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb3ec06b4d0 msgr2=0x7fb3ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.734+0000 7fb412600640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb3ec06b4d0 0x7fb3ec06d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb40c06c880 tx=0x7fb3f8008040 comp rx=0 tx=0).stop 2024-03-09T09:23:56.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.734+0000 7fb412600640 1 -- 172.21.15.67:0/3542249323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb40c157b20 msgr2=0x7fb40c1eedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:56.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.734+0000 7fb412600640 1 --2- 172.21.15.67:0/3542249323 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb40c157b20 0x7fb40c1eedd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb404036040 tx=0x7fb404004a40 comp rx=0 tx=0).stop 2024-03-09T09:23:56.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.734+0000 7fb411c00640 1 -- 172.21.15.67:0/3542249323 reap_dead start 2024-03-09T09:23:56.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.734+0000 7fb411c00640 1 -- 172.21.15.67:0/3542249323 reap_dead start 2024-03-09T09:23:56.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.734+0000 7fb412600640 1 -- 172.21.15.67:0/3542249323 shutdown_connections 2024-03-09T09:23:56.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.734+0000 7fb412600640 1 -- 172.21.15.67:0/3542249323 >> 172.21.15.67:0/3542249323 conn(0x7fb40c06c150 msgr2=0x7fb40c0acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:56.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.734+0000 7fb412600640 1 -- 172.21.15.67:0/3542249323 shutdown_connections 2024-03-09T09:23:56.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:56.734+0000 7fb412600640 1 -- 172.21.15.67:0/3542249323 wait complete. 2024-03-09T09:23:56.758 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:23:56.759 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-03-09T09:23:56.759 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:23:56.759 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-03-09T09:23:56.759 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:23:56.759 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-09T09:23:56.759 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:23:56.788 INFO:tasks.ceph.mon.a:Stopped 2024-03-09T09:23:56.788 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:23:56.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:23:57.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.028+0000 7fbcfd400640 1 Processor -- start 2024-03-09T09:23:57.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.029+0000 7fbcfd400640 1 -- start start 2024-03-09T09:23:57.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.030+0000 7fbcfd400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 0x7fbcf8071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:57.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.030+0000 7fbcfd400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbcf8157d60 0x7fbcf80acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:57.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.030+0000 7fbcfd400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf8153d40 con 0x7fbcf80c22a0 2024-03-09T09:23:57.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.030+0000 7fbcfd400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf8070ee0 con 0x7fbcf8157d60 2024-03-09T09:23:57.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.030+0000 7fbcfd400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8154d00 con 0x7fbcf80ac730 2024-03-09T09:23:57.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.030+0000 7fbcf7200640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fbcf8157d60 msgr2=0x7fbcf80acb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:57.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.030+0000 7fbcf7200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbcf8157d60 0x7fbcf80acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.031+0000 7fbcf6800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 0x7fbcf8071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.031+0000 7fbcf5e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fbcf80c22a0 0x7fbcf8157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:56550/0 (socket says 172.21.15.67:56550) 2024-03-09T09:23:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.031+0000 7fbcf5e00640 1 -- 172.21.15.67:0/3639623797 learned_addr learned my addr 172.21.15.67:0/3639623797 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:23:57.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.032+0000 7fbcf5400640 1 -- 172.21.15.67:0/3639623797 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708530635 0 0) 0x7fbcf8153d40 con 0x7fbcf80c22a0 2024-03-09T09:23:57.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.032+0000 7fbcf6800640 1 -- 172.21.15.67:0/3639623797 >> v2:172.21.15.67:3300/0 conn(0x7fbcf8157d60 msgr2=0x7fbcf80acb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:23:57.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.032+0000 7fbcf6800640 1 --2- 172.21.15.67:0/3639623797 >> v2:172.21.15.67:3300/0 conn(0x7fbcf8157d60 0x7fbcf80acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:57.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.032+0000 7fbcf6800640 1 -- 172.21.15.67:0/3639623797 >> v1:172.21.15.134:6789/0 conn(0x7fbcf80c22a0 legacy=0x7fbcf8157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:57.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.032+0000 7fbcf6800640 1 -- 172.21.15.67:0/3639623797 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf80c2790 con 0x7fbcf80ac730 2024-03-09T09:23:57.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.034+0000 7fbcf6800640 1 --2- 172.21.15.67:0/3639623797 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 0x7fbcf8071190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbce80097a0 tx=0x7fbce802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c793786427aea2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:57.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.035+0000 7fbcf5400640 1 -- 172.21.15.67:0/3639623797 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce80315a0 con 0x7fbcf80ac730 2024-03-09T09:23:57.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.035+0000 7fbcf5400640 1 -- 172.21.15.67:0/3639623797 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce8037070 con 0x7fbcf80ac730 2024-03-09T09:23:57.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.035+0000 7fbcf5400640 1 -- 172.21.15.67:0/3639623797 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8031960 con 0x7fbcf80ac730 2024-03-09T09:23:57.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.036+0000 7fbcfd400640 1 -- 172.21.15.67:0/3639623797 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 msgr2=0x7fbcf8071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:57.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.036+0000 7fbcfd400640 1 --2- 172.21.15.67:0/3639623797 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 0x7fbcf8071190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbce80097a0 tx=0x7fbce802b840 comp rx=0 tx=0).stop 2024-03-09T09:23:57.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.036+0000 7fbcfd400640 1 -- 172.21.15.67:0/3639623797 shutdown_connections 2024-03-09T09:23:57.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.036+0000 7fbcfd400640 1 --2- 172.21.15.67:0/3639623797 >> v2:172.21.15.67:3300/0 conn(0x7fbcf8157d60 0x7fbcf80acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:57.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.036+0000 7fbcfd400640 1 --2- 172.21.15.67:0/3639623797 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 0x7fbcf8071190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:57.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.036+0000 7fbcfd400640 1 -- 172.21.15.67:0/3639623797 >> 172.21.15.67:0/3639623797 conn(0x7fbcf80c7370 msgr2=0x7fbcf80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:57.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.036+0000 7fbcfd400640 1 -- 172.21.15.67:0/3639623797 shutdown_connections 2024-03-09T09:23:57.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.036+0000 7fbcfd400640 1 -- 172.21.15.67:0/3639623797 wait complete. 2024-03-09T09:23:57.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.038+0000 7fbcfd400640 1 Processor -- start 2024-03-09T09:23:57.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.038+0000 7fbcfd400640 1 -- start start 2024-03-09T09:23:57.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.039+0000 7fbcfd400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 0x7fbcf8150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:57.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.039+0000 7fbcfd400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcf80c22a0 0x7fbcf81eed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:57.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.039+0000 7fbcfd400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcf8157d60 0x7fbcf81ef250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:57.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.039+0000 7fbcfd400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8151610 con 0x7fbcf8157d60 2024-03-09T09:23:57.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.039+0000 7fbcfd400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8070c00 con 0x7fbcf80c22a0 2024-03-09T09:23:57.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.039+0000 7fbcfd400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8151790 con 0x7fbcf80ac730 2024-03-09T09:23:57.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.039+0000 7fbcf7200640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcf8157d60 msgr2=0x7fbcf81ef250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:23:57.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.039+0000 7fbcf7200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcf8157d60 0x7fbcf81ef250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:23:57.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.040+0000 7fbcf6800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 0x7fbcf8150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:57.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.040+0000 7fbcf5e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcf80c22a0 0x7fbcf81eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:57.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.040+0000 7fbcf6800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 0x7fbcf8150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46692/0 (socket says 172.21.15.67:46692) 2024-03-09T09:23:57.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.040+0000 7fbcf6800640 1 -- 172.21.15.67:0/3294410089 learned_addr learned my addr 172.21.15.67:0/3294410089 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:23:57.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.041+0000 7fbcf6800640 1 -- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcf80c22a0 msgr2=0x7fbcf81eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:57.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.041+0000 7fbcf6800640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcf80c22a0 0x7fbcf81eed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:57.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.041+0000 7fbcf6800640 1 -- 172.21.15.67:0/3294410089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcf8157d60 msgr2=0x7fbcf81ef250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:23:57.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.041+0000 7fbcf6800640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcf8157d60 0x7fbcf81ef250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:57.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.041+0000 7fbcf6800640 1 -- 172.21.15.67:0/3294410089 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf81ef8e0 con 0x7fbcf80ac730 2024-03-09T09:23:57.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.041+0000 7fbcf6800640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 0x7fbcf8150620 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbce80097a0 tx=0x7fbce80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:57.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.042+0000 7fbceea00640 1 -- 172.21.15.67:0/3294410089 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8004230 con 0x7fbcf80ac730 2024-03-09T09:23:57.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.042+0000 7fbceea00640 1 -- 172.21.15.67:0/3294410089 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce80043d0 con 0x7fbcf80ac730 2024-03-09T09:23:57.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.042+0000 7fbcfd400640 1 -- 172.21.15.67:0/3294410089 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf81dfa00 con 0x7fbcf80ac730 2024-03-09T09:23:57.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.042+0000 7fbceea00640 1 -- 172.21.15.67:0/3294410089 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8032880 con 0x7fbcf80ac730 2024-03-09T09:23:57.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.042+0000 7fbcfd400640 1 -- 172.21.15.67:0/3294410089 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf81dffc0 con 0x7fbcf80ac730 2024-03-09T09:23:57.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.043+0000 7fbcfd400640 1 -- 172.21.15.67:0/3294410089 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf8075a00 con 0x7fbcf80ac730 2024-03-09T09:23:57.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.045+0000 7fbceea00640 1 -- 172.21.15.67:0/3294410089 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbce8032a20 con 0x7fbcf80ac730 2024-03-09T09:23:57.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.046+0000 7fbceea00640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbcd406b480 0x7fbcd406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:23:57.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.046+0000 7fbceea00640 1 -- 172.21.15.67:0/3294410089 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbce802f030 con 0x7fbcf80ac730 2024-03-09T09:23:57.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.047+0000 7fbcf5e00640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbcd406b480 0x7fbcd406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:23:57.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.048+0000 7fbcf5e00640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbcd406b480 0x7fbcd406d940 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbce0001920 tx=0x7fbce00018b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:23:57.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.060+0000 7fbceea00640 1 -- 172.21.15.67:0/3294410089 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbce8041070 con 0x7fbcf80ac730 2024-03-09T09:23:57.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.392+0000 7fbcfd400640 1 -- 172.21.15.67:0/3294410089 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbcf81eca40 con 0x7fbcf80ac730 2024-03-09T09:23:57.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.393+0000 7fbceea00640 1 -- 172.21.15.67:0/3294410089 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fbce807b410 con 0x7fbcf80ac730 2024-03-09T09:23:57.400 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":246,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:23:57.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 -- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbcd406b480 msgr2=0x7fbcd406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:57.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbcd406b480 0x7fbcd406d940 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbce0001920 tx=0x7fbce00018b0 comp rx=0 tx=0).stop 2024-03-09T09:23:57.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 -- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 msgr2=0x7fbcf8150620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:23:57.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 0x7fbcf8150620 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbce80097a0 tx=0x7fbce80046c0 comp rx=0 tx=0).stop 2024-03-09T09:23:57.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 -- 172.21.15.67:0/3294410089 shutdown_connections 2024-03-09T09:23:57.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbcd406b480 0x7fbcd406d940 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:57.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcf8157d60 0x7fbcf81ef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:57.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbcf80c22a0 0x7fbcf81eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 --2- 172.21.15.67:0/3294410089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbcf80ac730 0x7fbcf8150620 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:23:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 -- 172.21.15.67:0/3294410089 >> 172.21.15.67:0/3294410089 conn(0x7fbcf80c7370 msgr2=0x7fbcf80c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:23:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 -- 172.21.15.67:0/3294410089 shutdown_connections 2024-03-09T09:23:57.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:23:57.396+0000 7fbcfd400640 1 -- 172.21.15.67:0/3294410089 wait complete. 2024-03-09T09:24:00.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:24:00.525 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:24:00.525 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:24:00.525 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 77 -eq 48 ']' 2024-03-09T09:24:00.526 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.77 2024-03-09T09:24:00.526 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.77 2024-03-09T09:24:00.526 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.77 2024-03-09T09:24:00.526 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.77 2024-03-09T09:24:00.528 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:24:00.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.678+0000 7f5af3800640 1 Processor -- start 2024-03-09T09:24:00.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.678+0000 7f5af3800640 1 -- start start 2024-03-09T09:24:00.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.679+0000 7f5af3800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5aec06b040 0x7f5aec155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:00.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.679+0000 7f5af3800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06c650 0x7f5aec0c6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:00.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.679+0000 7f5af3800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec06b5b0 con 0x7f5aec0c5f50 2024-03-09T09:24:00.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.679+0000 7f5af3800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec0c9e00 con 0x7f5aec06b040 2024-03-09T09:24:00.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.679+0000 7f5af3800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec06cae0 con 0x7f5aec06c650 2024-03-09T09:24:00.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.679+0000 7f5af0e00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f5aec0c5f50 legacy=0x7f5aec0c96f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:24:00.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.680+0000 7f5aebe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5aec06b040 0x7f5aec155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:00.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.680+0000 7f5af1800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06c650 0x7f5aec0c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:00.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.680+0000 7f5aebe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5aec06b040 0x7f5aec155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:56354/0 (socket says 172.21.15.67:56354) 2024-03-09T09:24:00.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.680+0000 7f5af1800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06c650 0x7f5aec0c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:60832/0 (socket says 172.21.15.67:60832) 2024-03-09T09:24:00.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.680+0000 7f5aebe00640 1 -- 172.21.15.67:0/1206202982 learned_addr learned my addr 172.21.15.67:0/1206202982 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:00.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.681+0000 7f5af1800640 1 -- 172.21.15.67:0/1206202982 >> v2:172.21.15.134:3300/0 conn(0x7f5aec06b040 msgr2=0x7f5aec155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:00.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.681+0000 7f5af1800640 1 --2- 172.21.15.67:0/1206202982 >> v2:172.21.15.134:3300/0 conn(0x7f5aec06b040 0x7f5aec155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:00.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.681+0000 7f5af1800640 1 -- 172.21.15.67:0/1206202982 >> v1:172.21.15.67:6789/0 conn(0x7f5aec0c5f50 legacy=0x7f5aec0c96f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:00.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.681+0000 7f5af1800640 1 -- 172.21.15.67:0/1206202982 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec06b890 con 0x7f5aec06c650 2024-03-09T09:24:00.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.682+0000 7f5af1800640 1 --2- 172.21.15.67:0/1206202982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06c650 0x7f5aec0c6330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009d90 tx=0x7f5ae402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=702592fa2514258b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:00.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.683+0000 7f5aeb400640 1 -- 172.21.15.67:0/1206202982 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4039070 con 0x7f5aec06c650 2024-03-09T09:24:00.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.683+0000 7f5aeb400640 1 -- 172.21.15.67:0/1206202982 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae402cbc0 con 0x7f5aec06c650 2024-03-09T09:24:00.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.683+0000 7f5aeb400640 1 -- 172.21.15.67:0/1206202982 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4034440 con 0x7f5aec06c650 2024-03-09T09:24:00.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.683+0000 7f5af3800640 1 -- 172.21.15.67:0/1206202982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06c650 msgr2=0x7f5aec0c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:00.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.683+0000 7f5af3800640 1 --2- 172.21.15.67:0/1206202982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06c650 0x7f5aec0c6330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009d90 tx=0x7f5ae402be10 comp rx=0 tx=0).stop 2024-03-09T09:24:00.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.683+0000 7f5af3800640 1 -- 172.21.15.67:0/1206202982 shutdown_connections 2024-03-09T09:24:00.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.683+0000 7f5af3800640 1 --2- 172.21.15.67:0/1206202982 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06c650 0x7f5aec0c6330 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:00.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.683+0000 7f5af3800640 1 --2- 172.21.15.67:0/1206202982 >> v2:172.21.15.134:3300/0 conn(0x7f5aec06b040 0x7f5aec155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:00.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.683+0000 7f5af3800640 1 -- 172.21.15.67:0/1206202982 >> 172.21.15.67:0/1206202982 conn(0x7f5aec0c8050 msgr2=0x7f5aec0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:00.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.683+0000 7f5af3800640 1 -- 172.21.15.67:0/1206202982 shutdown_connections 2024-03-09T09:24:00.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.684+0000 7f5af3800640 1 -- 172.21.15.67:0/1206202982 wait complete. 2024-03-09T09:24:00.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.685+0000 7f5af3800640 1 Processor -- start 2024-03-09T09:24:00.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.685+0000 7f5af3800640 1 -- start start 2024-03-09T09:24:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.686+0000 7f5af3800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06b040 0x7f5aec148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.686+0000 7f5af3800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aec06c650 0x7f5aec0c1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.686+0000 7f5af3800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aec0c5f50 0x7f5aec159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.686+0000 7f5af3800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec06c130 con 0x7f5aec06c650 2024-03-09T09:24:00.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.686+0000 7f5af3800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec06be50 con 0x7f5aec0c5f50 2024-03-09T09:24:00.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.686+0000 7f5af3800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec14cc00 con 0x7f5aec06b040 2024-03-09T09:24:00.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.686+0000 7f5aebe00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aec06c650 msgr2=0x7f5aec0c1c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:24:00.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.687+0000 7f5aebe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aec06c650 0x7f5aec0c1c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.687+0000 7f5af0e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06b040 0x7f5aec148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.687+0000 7f5af1800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aec0c5f50 0x7f5aec159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.687+0000 7f5af0e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06b040 0x7f5aec148ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:60836/0 (socket says 172.21.15.67:60836) 2024-03-09T09:24:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.687+0000 7f5af0e00640 1 -- 172.21.15.67:0/1149035441 learned_addr learned my addr 172.21.15.67:0/1149035441 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.688+0000 7f5af0e00640 1 -- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aec0c5f50 msgr2=0x7f5aec159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.688+0000 7f5af0e00640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aec0c5f50 0x7f5aec159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.688+0000 7f5af0e00640 1 -- 172.21.15.67:0/1149035441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aec06c650 msgr2=0x7f5aec0c1c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:24:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.688+0000 7f5af0e00640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aec06c650 0x7f5aec0c1c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.688+0000 7f5af0e00640 1 -- 172.21.15.67:0/1149035441 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec148c00 con 0x7f5aec06b040 2024-03-09T09:24:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.688+0000 7f5af0e00640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06b040 0x7f5aec148ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000b580 tx=0x7f5ae000ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:00.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.689+0000 7f5ae8c00640 1 -- 172.21.15.67:0/1149035441 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0003dc0 con 0x7f5aec06b040 2024-03-09T09:24:00.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.689+0000 7f5ae8c00640 1 -- 172.21.15.67:0/1149035441 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae0015070 con 0x7f5aec06b040 2024-03-09T09:24:00.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.689+0000 7f5af3800640 1 -- 172.21.15.67:0/1149035441 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec157720 con 0x7f5aec06b040 2024-03-09T09:24:00.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.689+0000 7f5ae8c00640 1 -- 172.21.15.67:0/1149035441 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0005330 con 0x7f5aec06b040 2024-03-09T09:24:00.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.689+0000 7f5af3800640 1 -- 172.21.15.67:0/1149035441 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec159780 con 0x7f5aec06b040 2024-03-09T09:24:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.690+0000 7f5af3800640 1 -- 172.21.15.67:0/1149035441 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec075a00 con 0x7f5aec06b040 2024-03-09T09:24:00.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.693+0000 7f5ae8c00640 1 -- 172.21.15.67:0/1149035441 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5ae00054d0 con 0x7f5aec06b040 2024-03-09T09:24:00.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.694+0000 7f5ae8c00640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5ad006b480 0x7f5ad006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:00.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.695+0000 7f5ae8c00640 1 -- 172.21.15.67:0/1149035441 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f5ae008ad80 con 0x7f5aec06b040 2024-03-09T09:24:00.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.695+0000 7f5aebe00640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5ad006b480 0x7f5ad006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:00.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.697+0000 7f5aebe00640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5ad006b480 0x7f5ad006d940 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0ac9d0 tx=0x7f5adc009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:00.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:00.705+0000 7f5ae8c00640 1 -- 172.21.15.67:0/1149035441 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5ae0059880 con 0x7f5aec06b040 2024-03-09T09:24:00.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:24:00.958 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:24:01.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.051+0000 7f5af3800640 1 -- 172.21.15.67:0/1149035441 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5aec06cca0 con 0x7f5aec06b040 2024-03-09T09:24:01.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.052+0000 7f5ae8c00640 1 -- 172.21.15.67:0/1149035441 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f5ae0059180 con 0x7f5aec06b040 2024-03-09T09:24:01.059 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":246,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:24:01.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 -- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5ad006b480 msgr2=0x7f5ad006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:01.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5ad006b480 0x7f5ad006d940 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0ac9d0 tx=0x7f5adc009240 comp rx=0 tx=0).stop 2024-03-09T09:24:01.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 -- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06b040 msgr2=0x7f5aec148ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:01.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06b040 0x7f5aec148ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000b580 tx=0x7f5ae000ba40 comp rx=0 tx=0).stop 2024-03-09T09:24:01.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 -- 172.21.15.67:0/1149035441 shutdown_connections 2024-03-09T09:24:01.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5ad006b480 0x7f5ad006d940 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:01.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aec0c5f50 0x7f5aec159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:01.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5aec06c650 0x7f5aec0c1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:01.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 --2- 172.21.15.67:0/1149035441 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5aec06b040 0x7f5aec148ee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:01.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 -- 172.21.15.67:0/1149035441 >> 172.21.15.67:0/1149035441 conn(0x7f5aec0c8050 msgr2=0x7f5aec0ad370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:01.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 -- 172.21.15.67:0/1149035441 shutdown_connections 2024-03-09T09:24:01.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:01.054+0000 7f5af3800640 1 -- 172.21.15.67:0/1149035441 wait complete. 2024-03-09T09:24:02.980 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:24:02.980 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:02.980 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 83 -eq 48 ']' 2024-03-09T09:24:02.980 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.83 2024-03-09T09:24:02.981 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.83 2024-03-09T09:24:02.981 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.83 2024-03-09T09:24:02.981 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.83 2024-03-09T09:24:02.985 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:24:04.082 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:24:04.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.302+0000 7fe0f9600640 1 Processor -- start 2024-03-09T09:24:04.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.303+0000 7fe0f9600640 1 -- start start 2024-03-09T09:24:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.304+0000 7fe0f9600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f4072830 0x7fe0f4072c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.304+0000 7fe0f9600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe0f4159070 0x7fe0f4159500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.304+0000 7fe0f9600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f40c2b50 con 0x7fe0f41530c0 2024-03-09T09:24:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.304+0000 7fe0f9600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f4156700 con 0x7fe0f4159070 2024-03-09T09:24:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.304+0000 7fe0f9600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f4155dd0 con 0x7fe0f4072830 2024-03-09T09:24:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.304+0000 7fe0f3e00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7fe0f41530c0 legacy=0x7fe0f40c6470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:24:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.305+0000 7fe0f8c00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fe0f4159070 msgr2=0x7fe0f4159500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:24:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.305+0000 7fe0f8c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe0f4159070 0x7fe0f4159500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.305+0000 7fe0f3400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f4072830 0x7fe0f4072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.305+0000 7fe0f3400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f4072830 0x7fe0f4072c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:60848/0 (socket says 172.21.15.67:60848) 2024-03-09T09:24:04.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.305+0000 7fe0f3400640 1 -- 172.21.15.67:0/3317126962 learned_addr learned my addr 172.21.15.67:0/3317126962 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:04.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.306+0000 7fe0f3400640 1 -- 172.21.15.67:0/3317126962 >> v2:172.21.15.67:3300/0 conn(0x7fe0f4159070 msgr2=0x7fe0f4159500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:04.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.306+0000 7fe0f3400640 1 --2- 172.21.15.67:0/3317126962 >> v2:172.21.15.67:3300/0 conn(0x7fe0f4159070 0x7fe0f4159500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:04.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.306+0000 7fe0f3400640 1 -- 172.21.15.67:0/3317126962 >> v1:172.21.15.67:6789/0 conn(0x7fe0f41530c0 legacy=0x7fe0f40c6470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:04.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.306+0000 7fe0f3400640 1 -- 172.21.15.67:0/3317126962 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f41560b0 con 0x7fe0f4072830 2024-03-09T09:24:04.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.307+0000 7fe0f3400640 1 --2- 172.21.15.67:0/3317126962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f4072830 0x7fe0f4072c90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009870 tx=0x7fe0e002b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=724ed22160519fae server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:04.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.307+0000 7fe0f2a00640 1 -- 172.21.15.67:0/3317126962 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0035020 con 0x7fe0f4072830 2024-03-09T09:24:04.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.308+0000 7fe0f2a00640 1 -- 172.21.15.67:0/3317126962 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e0031870 con 0x7fe0f4072830 2024-03-09T09:24:04.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.308+0000 7fe0f2a00640 1 -- 172.21.15.67:0/3317126962 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0031c40 con 0x7fe0f4072830 2024-03-09T09:24:04.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.308+0000 7fe0f9600640 1 -- 172.21.15.67:0/3317126962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f4072830 msgr2=0x7fe0f4072c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:04.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.308+0000 7fe0f9600640 1 --2- 172.21.15.67:0/3317126962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f4072830 0x7fe0f4072c90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009870 tx=0x7fe0e002b910 comp rx=0 tx=0).stop 2024-03-09T09:24:04.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.308+0000 7fe0f9600640 1 -- 172.21.15.67:0/3317126962 shutdown_connections 2024-03-09T09:24:04.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.308+0000 7fe0f9600640 1 --2- 172.21.15.67:0/3317126962 >> v2:172.21.15.67:3300/0 conn(0x7fe0f4159070 0x7fe0f4159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:04.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.308+0000 7fe0f9600640 1 --2- 172.21.15.67:0/3317126962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f4072830 0x7fe0f4072c90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:04.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.308+0000 7fe0f9600640 1 -- 172.21.15.67:0/3317126962 >> 172.21.15.67:0/3317126962 conn(0x7fe0f4067180 msgr2=0x7fe0f4067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:04.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.309+0000 7fe0f9600640 1 -- 172.21.15.67:0/3317126962 shutdown_connections 2024-03-09T09:24:04.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.309+0000 7fe0f9600640 1 -- 172.21.15.67:0/3317126962 wait complete. 2024-03-09T09:24:04.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.310+0000 7fe0f9600640 1 Processor -- start 2024-03-09T09:24:04.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.310+0000 7fe0f9600640 1 -- start start 2024-03-09T09:24:04.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.311+0000 7fe0f9600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0f4072830 0x7fe0f414e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.311+0000 7fe0f9600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f41530c0 0x7fe0f4151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.311+0000 7fe0f9600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0f4159070 0x7fe0f41523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.311+0000 7fe0f9600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f40c6b80 con 0x7fe0f4159070 2024-03-09T09:24:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.311+0000 7fe0f9600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f4155a80 con 0x7fe0f4072830 2024-03-09T09:24:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.311+0000 7fe0f9600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f40c6d00 con 0x7fe0f41530c0 2024-03-09T09:24:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.311+0000 7fe0f8c00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0f4159070 msgr2=0x7fe0f41523f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:24:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.311+0000 7fe0f8c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0f4159070 0x7fe0f41523f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.312+0000 7fe0f3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0f4072830 0x7fe0f414e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.312+0000 7fe0f3400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f41530c0 0x7fe0f4151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.312+0000 7fe0f3e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0f4072830 0x7fe0f414e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:56372/0 (socket says 172.21.15.67:56372) 2024-03-09T09:24:04.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.312+0000 7fe0f3e00640 1 -- 172.21.15.67:0/1325052761 learned_addr learned my addr 172.21.15.67:0/1325052761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:04.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.312+0000 7fe0f3e00640 1 -- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f41530c0 msgr2=0x7fe0f4151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:04.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.312+0000 7fe0f3e00640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f41530c0 0x7fe0f4151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:04.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.312+0000 7fe0f3e00640 1 -- 172.21.15.67:0/1325052761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0f4159070 msgr2=0x7fe0f41523f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:24:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.312+0000 7fe0f3e00640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0f4159070 0x7fe0f41523f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.312+0000 7fe0f3e00640 1 -- 172.21.15.67:0/1325052761 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f414f190 con 0x7fe0f4072830 2024-03-09T09:24:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.313+0000 7fe0f3e00640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0f4072830 0x7fe0f414e8c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400b550 tx=0x7fe0e400ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.313+0000 7fe0ebe00640 1 -- 172.21.15.67:0/1325052761 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4013020 con 0x7fe0f4072830 2024-03-09T09:24:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.313+0000 7fe0ebe00640 1 -- 172.21.15.67:0/1325052761 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e4003de0 con 0x7fe0f4072830 2024-03-09T09:24:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.313+0000 7fe0ebe00640 1 -- 172.21.15.67:0/1325052761 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40052b0 con 0x7fe0f4072830 2024-03-09T09:24:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.313+0000 7fe0f9600640 1 -- 172.21.15.67:0/1325052761 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f41ec3c0 con 0x7fe0f4072830 2024-03-09T09:24:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.313+0000 7fe0f9600640 1 -- 172.21.15.67:0/1325052761 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f41511a0 con 0x7fe0f4072830 2024-03-09T09:24:04.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.314+0000 7fe0f9600640 1 -- 172.21.15.67:0/1325052761 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f404f3c0 con 0x7fe0f4072830 2024-03-09T09:24:04.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.316+0000 7fe0ebe00640 1 -- 172.21.15.67:0/1325052761 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe0e4016020 con 0x7fe0f4072830 2024-03-09T09:24:04.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.317+0000 7fe0ebe00640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe0d406b480 0x7fe0d406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:04.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.317+0000 7fe0ebe00640 1 -- 172.21.15.67:0/1325052761 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fe0e408ad10 con 0x7fe0f4072830 2024-03-09T09:24:04.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.317+0000 7fe0f3400640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe0d406b480 0x7fe0d406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:04.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.318+0000 7fe0f3400640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe0d406b480 0x7fe0d406d940 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0036000 tx=0x7fe0e003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:04.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.330+0000 7fe0ebe00640 1 -- 172.21.15.67:0/1325052761 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe0e401d020 con 0x7fe0f4072830 2024-03-09T09:24:04.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.651+0000 7fe0f9600640 1 -- 172.21.15.67:0/1325052761 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe0f4072f50 con 0x7fe0f4072830 2024-03-09T09:24:04.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.652+0000 7fe0ebe00640 1 -- 172.21.15.67:0/1325052761 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fe0e4059810 con 0x7fe0f4072830 2024-03-09T09:24:04.658 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":246,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:24:04.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 -- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe0d406b480 msgr2=0x7fe0d406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:04.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe0d406b480 0x7fe0d406d940 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0036000 tx=0x7fe0e003c040 comp rx=0 tx=0).stop 2024-03-09T09:24:04.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 -- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0f4072830 msgr2=0x7fe0f414e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:04.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0f4072830 0x7fe0f414e8c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400b550 tx=0x7fe0e400ba10 comp rx=0 tx=0).stop 2024-03-09T09:24:04.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 -- 172.21.15.67:0/1325052761 shutdown_connections 2024-03-09T09:24:04.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe0d406b480 0x7fe0d406d940 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0f4159070 0x7fe0f41523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0f41530c0 0x7fe0f4151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 --2- 172.21.15.67:0/1325052761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0f4072830 0x7fe0f414e8c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 -- 172.21.15.67:0/1325052761 >> 172.21.15.67:0/1325052761 conn(0x7fe0f4067180 msgr2=0x7fe0f40aa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 -- 172.21.15.67:0/1325052761 shutdown_connections 2024-03-09T09:24:04.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:04.655+0000 7fe0f9600640 1 -- 172.21.15.67:0/1325052761 wait complete. 2024-03-09T09:24:06.062 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:24:06.063 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:24:06.063 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 78 -eq 48 ']' 2024-03-09T09:24:06.063 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.78 2024-03-09T09:24:06.063 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.78 2024-03-09T09:24:06.063 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.78 2024-03-09T09:24:06.063 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.78 2024-03-09T09:24:06.066 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:24:07.683 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:24:07.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.893+0000 7fbc3e800640 1 Processor -- start 2024-03-09T09:24:07.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.893+0000 7fbc3e800640 1 -- start start 2024-03-09T09:24:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.895+0000 7fbc3e800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.895+0000 7fbc3e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.895+0000 7fbc3e800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3806b5b0 con 0x7fbc3806b040 2024-03-09T09:24:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.895+0000 7fbc3e800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc380c9e00 con 0x7fbc380c5f50 2024-03-09T09:24:07.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.895+0000 7fbc3e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3806cae0 con 0x7fbc3806c650 2024-03-09T09:24:07.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.896+0000 7fbc37e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:07.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.896+0000 7fbc37e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:60866/0 (socket says 172.21.15.67:60866) 2024-03-09T09:24:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.896+0000 7fbc37e00640 1 -- 172.21.15.67:0/924042022 learned_addr learned my addr 172.21.15.67:0/924042022 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.896+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.896+0000 7fbc37e00640 1 -- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 msgr2=0x7fbc380c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:24:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.896+0000 7fbc37e00640 1 -- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 msgr2=0x7fbc380c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.896+0000 7fbc37e00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.896+0000 7fbc37e00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.897+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 msgr2=0x7fbc38155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:24:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.897+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 msgr2=0x7fbc38155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.897+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:07.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:07.897+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:08.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.097+0000 7fbc37e00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:08.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.098+0000 7fbc37e00640 1 -- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 msgr2=0x7fbc380c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:24:08.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.098+0000 7fbc37e00640 1 -- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 msgr2=0x7fbc380c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:08.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.098+0000 7fbc37e00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:08.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.098+0000 7fbc37e00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:24:08.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.098+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:08.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.099+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 msgr2=0x7fbc38155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:24:08.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.099+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 msgr2=0x7fbc38155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:08.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.099+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:08.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.099+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:24:08.206 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:24:08.206 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:08.206 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 84 -eq 48 ']' 2024-03-09T09:24:08.206 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.84 2024-03-09T09:24:08.207 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.84 2024-03-09T09:24:08.207 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.84 2024-03-09T09:24:08.207 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.84 2024-03-09T09:24:08.211 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:24:08.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.500+0000 7fbc37e00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:08.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.500+0000 7fbc37e00640 1 -- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 msgr2=0x7fbc380c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:24:08.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.500+0000 7fbc37e00640 1 -- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 msgr2=0x7fbc380c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:08.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.500+0000 7fbc37e00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:08.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.500+0000 7fbc37e00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:24:08.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.500+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:08.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.501+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 msgr2=0x7fbc38155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:24:08.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.501+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 msgr2=0x7fbc38155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:08.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.501+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:08.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:08.501+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:24:09.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:09.302+0000 7fbc37e00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:09.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:09.302+0000 7fbc37e00640 1 -- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 msgr2=0x7fbc380c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:24:09.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:09.302+0000 7fbc37e00640 1 -- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 msgr2=0x7fbc380c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:09.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:09.302+0000 7fbc37e00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:09.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:09.302+0000 7fbc37e00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:24:09.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:09.303+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:09.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:09.303+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 msgr2=0x7fbc38155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:24:09.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:09.303+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 msgr2=0x7fbc38155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:09.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:09.303+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:09.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:09.303+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:24:10.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:10.895+0000 7fbc34c00640 1 -- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 msgr2=0x7fbc380c6330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:10.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:10.895+0000 7fbc34c00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc380c6330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:10.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:10.895+0000 7fbc34c00640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 msgr2=0x7fbc38155e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:10.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:10.895+0000 7fbc34c00640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.134:3300/0 conn(0x7fbc380c5f50 0x7fbc38155e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:10.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:10.895+0000 7fbc34c00640 1 -- 172.21.15.67:0/924042022 >> v1:172.21.15.134:6789/0 conn(0x7fbc3806b040 legacy=0x7fbc380c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:10.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:10.896+0000 7fbc34c00640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.67:3300/0 conn(0x7fbc18007350 0x7fbc180097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:10.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:10.896+0000 7fbc34c00640 1 -- 172.21.15.67:0/924042022 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3806b890 con 0x7fbc180036a0 2024-03-09T09:24:10.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:10.896+0000 7fbc34c00640 1 -- 172.21.15.67:0/924042022 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc380c9e00 con 0x7fbc18007350 2024-03-09T09:24:10.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:10.896+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v1:172.21.15.67:6789/0 conn(0x7fbc180036a0 legacy=0x7fbc18005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:24:10.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:10.896+0000 7fbc36a00640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.67:3300/0 conn(0x7fbc18007350 msgr2=0x7fbc180097f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:24:10.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:10.896+0000 7fbc36a00640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.67:3300/0 conn(0x7fbc18007350 0x7fbc180097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:11.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:11.097+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v1:172.21.15.67:6789/0 conn(0x7fbc180036a0 legacy=0x7fbc18005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:24:11.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:11.097+0000 7fbc36a00640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.67:3300/0 conn(0x7fbc18007350 msgr2=0x7fbc180097f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:24:11.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:11.097+0000 7fbc36a00640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.67:3300/0 conn(0x7fbc18007350 0x7fbc180097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:24:11.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:11.498+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v1:172.21.15.67:6789/0 conn(0x7fbc180036a0 legacy=0x7fbc18005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:24:11.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:11.498+0000 7fbc36a00640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.67:3300/0 conn(0x7fbc18007350 msgr2=0x7fbc180097f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:24:11.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:11.498+0000 7fbc36a00640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.67:3300/0 conn(0x7fbc18007350 0x7fbc180097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:24:12.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:12.299+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v1:172.21.15.67:6789/0 conn(0x7fbc180036a0 legacy=0x7fbc18005b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:24:12.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:12.299+0000 7fbc36a00640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.67:3300/0 conn(0x7fbc18007350 msgr2=0x7fbc180097f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:24:12.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:12.299+0000 7fbc36a00640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.67:3300/0 conn(0x7fbc18007350 0x7fbc180097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:24:13.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.896+0000 7fbc34c00640 1 -- 172.21.15.67:0/924042022 >> v2:172.21.15.67:3300/0 conn(0x7fbc18007350 msgr2=0x7fbc180097f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:13.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.896+0000 7fbc34c00640 1 --2- 172.21.15.67:0/924042022 >> v2:172.21.15.67:3300/0 conn(0x7fbc18007350 0x7fbc180097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:13.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.896+0000 7fbc34c00640 1 -- 172.21.15.67:0/924042022 >> v1:172.21.15.67:6789/0 conn(0x7fbc180036a0 legacy=0x7fbc18005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:13.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.896+0000 7fbc37e00640 1 -- 172.21.15.67:0/924042022 reap_dead start 2024-03-09T09:24:13.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.896+0000 7fbc34c00640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc18009e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:13.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.897+0000 7fbc34c00640 1 -- 172.21.15.67:0/924042022 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1800af50 con 0x7fbc180098b0 2024-03-09T09:24:13.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.897+0000 7fbc34c00640 1 -- 172.21.15.67:0/924042022 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3806b890 con 0x7fbc1800a3b0 2024-03-09T09:24:13.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.897+0000 7fbc34c00640 1 -- 172.21.15.67:0/924042022 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc380c9e00 con 0x7fbc3806c650 2024-03-09T09:24:13.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.897+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc18009e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:13.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.897+0000 7fbc37e00640 1 -- 172.21.15.67:0/924042022 >> v1:172.21.15.67:6789/0 conn(0x7fbc180098b0 legacy=0x7fbc18006960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:24:13.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.898+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v1:172.21.15.134:6789/0 conn(0x7fbc1800a3b0 legacy=0x7fbc18006250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:13.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.898+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 >> v1:172.21.15.67:6789/0 conn(0x7fbc180098b0 legacy=0x7fbc18006960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:13.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.898+0000 7fbc37400640 1 -- 172.21.15.67:0/924042022 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1800a880 con 0x7fbc3806c650 2024-03-09T09:24:13.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.902+0000 7fbc37400640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc18009e70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc28003690 tx=0x7fbc2802c570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5e53c0dc202025a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:13.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.903+0000 7fbc36000640 1 -- 172.21.15.67:0/924042022 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2803a070 con 0x7fbc3806c650 2024-03-09T09:24:13.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.903+0000 7fbc36000640 1 -- 172.21.15.67:0/924042022 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc28035410 con 0x7fbc3806c650 2024-03-09T09:24:13.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.903+0000 7fbc36000640 1 -- 172.21.15.67:0/924042022 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc28035800 con 0x7fbc3806c650 2024-03-09T09:24:13.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.904+0000 7fbc3e800640 1 -- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 msgr2=0x7fbc18009e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:13.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.904+0000 7fbc3e800640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc18009e70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc28003690 tx=0x7fbc2802c570 comp rx=0 tx=0).stop 2024-03-09T09:24:13.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.904+0000 7fbc3e800640 1 -- 172.21.15.67:0/924042022 shutdown_connections 2024-03-09T09:24:13.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.904+0000 7fbc3e800640 1 --2- 172.21.15.67:0/924042022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc18009e70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:13.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.904+0000 7fbc3e800640 1 -- 172.21.15.67:0/924042022 >> 172.21.15.67:0/924042022 conn(0x7fbc380c8050 msgr2=0x7fbc380c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:13.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.905+0000 7fbc3e800640 1 -- 172.21.15.67:0/924042022 shutdown_connections 2024-03-09T09:24:13.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.905+0000 7fbc3e800640 1 -- 172.21.15.67:0/924042022 wait complete. 2024-03-09T09:24:13.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.907+0000 7fbc3e800640 1 Processor -- start 2024-03-09T09:24:13.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.908+0000 7fbc3e800640 1 -- start start 2024-03-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.909+0000 7fbc3e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc180098b0 0x7fbc381eb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.909+0000 7fbc3e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1800a3b0 0x7fbc38159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.909+0000 7fbc3e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc381ef640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.909+0000 7fbc3e800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3806c130 con 0x7fbc1800a3b0 2024-03-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.909+0000 7fbc3e800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3806be50 con 0x7fbc180098b0 2024-03-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.909+0000 7fbc3e800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3814cc00 con 0x7fbc3806c650 2024-03-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.909+0000 7fbc36a00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1800a3b0 msgr2=0x7fbc38159630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:24:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.909+0000 7fbc36a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1800a3b0 0x7fbc38159630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.910+0000 7fbc37400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc180098b0 0x7fbc381eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.910+0000 7fbc37e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc381ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.910+0000 7fbc37e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc381ef640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:34698/0 (socket says 172.21.15.67:34698) 2024-03-09T09:24:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.910+0000 7fbc37e00640 1 -- 172.21.15.67:0/2671146114 learned_addr learned my addr 172.21.15.67:0/2671146114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.911+0000 7fbc37e00640 1 -- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc180098b0 msgr2=0x7fbc381eb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.911+0000 7fbc37e00640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc180098b0 0x7fbc381eb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.911+0000 7fbc37e00640 1 -- 172.21.15.67:0/2671146114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1800a3b0 msgr2=0x7fbc38159630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:24:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.911+0000 7fbc37e00640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1800a3b0 0x7fbc38159630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.911+0000 7fbc37e00640 1 -- 172.21.15.67:0/2671146114 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc381563f0 con 0x7fbc3806c650 2024-03-09T09:24:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.911+0000 7fbc37e00640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc381ef640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbc300096d0 tx=0x7fbc30007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.912+0000 7fbc2f400640 1 -- 172.21.15.67:0/2671146114 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3001a020 con 0x7fbc3806c650 2024-03-09T09:24:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.912+0000 7fbc2f400640 1 -- 172.21.15.67:0/2671146114 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3000b540 con 0x7fbc3806c650 2024-03-09T09:24:13.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.912+0000 7fbc3e800640 1 -- 172.21.15.67:0/2671146114 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc380c19e0 con 0x7fbc3806c650 2024-03-09T09:24:13.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.912+0000 7fbc2f400640 1 -- 172.21.15.67:0/2671146114 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc30003bb0 con 0x7fbc3806c650 2024-03-09T09:24:13.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.912+0000 7fbc3e800640 1 -- 172.21.15.67:0/2671146114 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc381eee40 con 0x7fbc3806c650 2024-03-09T09:24:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.915+0000 7fbc3e800640 1 -- 172.21.15.67:0/2671146114 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc38075a00 con 0x7fbc3806c650 2024-03-09T09:24:13.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.915+0000 7fbc2f400640 1 -- 172.21.15.67:0/2671146114 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbc3001d020 con 0x7fbc3806c650 2024-03-09T09:24:13.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.917+0000 7fbc2f400640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbc1806cda0 0x7fbc1806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.917+0000 7fbc2f400640 1 -- 172.21.15.67:0/2671146114 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbc3008ae30 con 0x7fbc3806c650 2024-03-09T09:24:13.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.918+0000 7fbc37400640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbc1806cda0 0x7fbc1806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:13.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.919+0000 7fbc37400640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbc1806cda0 0x7fbc1806f260 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbc28037000 tx=0x7fbc28030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:13.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:13.934+0000 7fbc2f400640 1 -- 172.21.15.67:0/2671146114 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbc30024020 con 0x7fbc3806c650 2024-03-09T09:24:14.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.247+0000 7fbc3e800640 1 -- 172.21.15.67:0/2671146114 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbc3806cca0 con 0x7fbc3806c650 2024-03-09T09:24:14.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.249+0000 7fbc2f400640 1 -- 172.21.15.67:0/2671146114 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fbc30059950 con 0x7fbc3806c650 2024-03-09T09:24:14.255 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":248,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:24:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.251+0000 7fbc3e800640 1 -- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbc1806cda0 msgr2=0x7fbc1806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.251+0000 7fbc3e800640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbc1806cda0 0x7fbc1806f260 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbc28037000 tx=0x7fbc28030000 comp rx=0 tx=0).stop 2024-03-09T09:24:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.251+0000 7fbc3e800640 1 -- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 msgr2=0x7fbc381ef640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.251+0000 7fbc3e800640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc381ef640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbc300096d0 tx=0x7fbc30007d30 comp rx=0 tx=0).stop 2024-03-09T09:24:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.252+0000 7fbc3e800640 1 -- 172.21.15.67:0/2671146114 shutdown_connections 2024-03-09T09:24:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.252+0000 7fbc3e800640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbc1806cda0 0x7fbc1806f260 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.252+0000 7fbc3e800640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbc3806c650 0x7fbc381ef640 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.252+0000 7fbc3e800640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc1800a3b0 0x7fbc38159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.252+0000 7fbc3e800640 1 --2- 172.21.15.67:0/2671146114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc180098b0 0x7fbc381eb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.252+0000 7fbc3e800640 1 -- 172.21.15.67:0/2671146114 >> 172.21.15.67:0/2671146114 conn(0x7fbc380c8050 msgr2=0x7fbc380c6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:14.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.252+0000 7fbc3e800640 1 -- 172.21.15.67:0/2671146114 shutdown_connections 2024-03-09T09:24:14.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.252+0000 7fbc3e800640 1 -- 172.21.15.67:0/2671146114 wait complete. 2024-03-09T09:24:14.277 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:24:14.277 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:24:14.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:24:14.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.473+0000 7f7659e00640 1 Processor -- start 2024-03-09T09:24:14.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.473+0000 7f7659e00640 1 -- start start 2024-03-09T09:24:14.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.475+0000 7f7659e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7654159580 0x7f76540c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.475+0000 7f7659e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f76540c3010 0x7f7654159960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.475+0000 7f7659e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7654067410 con 0x7f76540c5890 2024-03-09T09:24:14.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.476+0000 7f7659e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76540c1890 con 0x7f7654159580 2024-03-09T09:24:14.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.476+0000 7f7659e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76540c3740 con 0x7f76540c3010 2024-03-09T09:24:14.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.476+0000 7f7659400640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f76540c5890 legacy=0x7f76540c6390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:24:14.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.476+0000 7f7658a00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f7654159580 msgr2=0x7f76540c1d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:24:14.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.476+0000 7f7658a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7654159580 0x7f76540c1d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.476+0000 7f764be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f76540c3010 0x7f7654159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.477+0000 7f764be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f76540c3010 0x7f7654159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:36960/0 (socket says 172.21.15.67:36960) 2024-03-09T09:24:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.477+0000 7f764be00640 1 -- 172.21.15.67:0/987080620 learned_addr learned my addr 172.21.15.67:0/987080620 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.477+0000 7f764be00640 1 -- 172.21.15.67:0/987080620 >> v2:172.21.15.67:3300/0 conn(0x7f7654159580 msgr2=0x7f76540c1d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.477+0000 7f764be00640 1 --2- 172.21.15.67:0/987080620 >> v2:172.21.15.67:3300/0 conn(0x7f7654159580 0x7f76540c1d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.477+0000 7f764be00640 1 -- 172.21.15.67:0/987080620 >> v1:172.21.15.67:6789/0 conn(0x7f76540c5890 legacy=0x7f76540c6390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.477+0000 7f764be00640 1 -- 172.21.15.67:0/987080620 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76540c3470 con 0x7f76540c3010 2024-03-09T09:24:14.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.479+0000 7f764be00640 1 --2- 172.21.15.67:0/987080620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76540c3010 0x7f7654159960 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f76540aa310 tx=0x7f764002b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e8cf0cd73f724e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:14.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.479+0000 7f7653e00640 1 -- 172.21.15.67:0/987080620 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640035020 con 0x7f76540c3010 2024-03-09T09:24:14.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.479+0000 7f7653e00640 1 -- 172.21.15.67:0/987080620 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7640031800 con 0x7f76540c3010 2024-03-09T09:24:14.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.479+0000 7f7653e00640 1 -- 172.21.15.67:0/987080620 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640031bd0 con 0x7f76540c3010 2024-03-09T09:24:14.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.480+0000 7f7659e00640 1 -- 172.21.15.67:0/987080620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76540c3010 msgr2=0x7f7654159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.480+0000 7f7659e00640 1 --2- 172.21.15.67:0/987080620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76540c3010 0x7f7654159960 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f76540aa310 tx=0x7f764002b8a0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.480+0000 7f7659e00640 1 -- 172.21.15.67:0/987080620 shutdown_connections 2024-03-09T09:24:14.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.480+0000 7f7659e00640 1 --2- 172.21.15.67:0/987080620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76540c3010 0x7f7654159960 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.480+0000 7f7659e00640 1 --2- 172.21.15.67:0/987080620 >> v2:172.21.15.67:3300/0 conn(0x7f7654159580 0x7f76540c1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.480+0000 7f7659e00640 1 -- 172.21.15.67:0/987080620 >> 172.21.15.67:0/987080620 conn(0x7f76540a9740 msgr2=0x7f76540a9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:14.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.480+0000 7f7659e00640 1 -- 172.21.15.67:0/987080620 shutdown_connections 2024-03-09T09:24:14.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.480+0000 7f7659e00640 1 -- 172.21.15.67:0/987080620 wait complete. 2024-03-09T09:24:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.483+0000 7f7659e00640 1 Processor -- start 2024-03-09T09:24:14.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.483+0000 7f7659e00640 1 -- start start 2024-03-09T09:24:14.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f7659e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76540c3010 0x7f765406dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f7659e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76540c5890 0x7f76540ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f7659e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7654159580 0x7f76540ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f7659e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76540c1520 con 0x7f76540c5890 2024-03-09T09:24:14.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f7659e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76540c1240 con 0x7f7654159580 2024-03-09T09:24:14.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f7659e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7654159ea0 con 0x7f76540c3010 2024-03-09T09:24:14.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f764be00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76540c5890 msgr2=0x7f76540ac4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:24:14.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f764be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76540c5890 0x7f76540ac4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:14.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f7658a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76540c3010 0x7f765406dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f7658a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76540c3010 0x7f765406dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:34714/0 (socket says 172.21.15.67:34714) 2024-03-09T09:24:14.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f7658a00640 1 -- 172.21.15.67:0/851182032 learned_addr learned my addr 172.21.15.67:0/851182032 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:14.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.485+0000 7f7659400640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7654159580 0x7f76540ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.486+0000 7f7658a00640 1 -- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7654159580 msgr2=0x7f76540ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.486+0000 7f7658a00640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7654159580 0x7f76540ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.486+0000 7f7658a00640 1 -- 172.21.15.67:0/851182032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76540c5890 msgr2=0x7f76540ac4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:24:14.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.486+0000 7f7658a00640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f76540c5890 0x7f76540ac4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.486+0000 7f7658a00640 1 -- 172.21.15.67:0/851182032 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76540aba30 con 0x7f76540c3010 2024-03-09T09:24:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.487+0000 7f7658a00640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76540c3010 0x7f765406dfa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f763800b550 tx=0x7f763800ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.487+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638003dc0 con 0x7f76540c3010 2024-03-09T09:24:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.488+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7638015070 con 0x7f76540c3010 2024-03-09T09:24:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.488+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380052b0 con 0x7f76540c3010 2024-03-09T09:24:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.488+0000 7f7659e00640 1 -- 172.21.15.67:0/851182032 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76541ef430 con 0x7f76540c3010 2024-03-09T09:24:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.488+0000 7f7659e00640 1 -- 172.21.15.67:0/851182032 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76540abf80 con 0x7f76540c3010 2024-03-09T09:24:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.489+0000 7f7659e00640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76540acf20 0x7f7654071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.489+0000 7f7659e00640 1 -- 172.21.15.67:0/851182032 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76541ef830 con 0x7f76540acf20 2024-03-09T09:24:14.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.490+0000 7f764be00640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76540acf20 0x7f7654071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.491+0000 7f764be00640 1 -- 172.21.15.67:0/851182032 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76540ac1c0 con 0x7f76540acf20 2024-03-09T09:24:14.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.491+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7638005450 con 0x7f76540c3010 2024-03-09T09:24:14.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.491+0000 7f764be00640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76540acf20 0x7f7654071c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f765415a3b0 tx=0x7f76400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:14.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.493+0000 7f7651600640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f762806b480 0x7f762806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.494+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f763808a310 con 0x7f76540c3010 2024-03-09T09:24:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.494+0000 7f7659400640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f762806b480 0x7f762806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.494+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640031b90 con 0x7f76540acf20 2024-03-09T09:24:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.494+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f7640004020 con 0x7f76540acf20 2024-03-09T09:24:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.494+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76540acf20 msgr2=0x7f7654071c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.494+0000 7f7651600640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76540acf20 0x7f7654071c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f765415a3b0 tx=0x7f76400046c0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.494+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76540acf20 msgr2=0x7f7654071c80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:24:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.494+0000 7f7651600640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76540acf20 0x7f7654071c80 secure :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f765415a3b0 tx=0x7f76400046c0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.495+0000 7f7659400640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f762806b480 0x7f762806d940 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f76540ade30 tx=0x7f764c0095f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:14.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.512+0000 7f7659e00640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7654153c10 0x7f76540721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.512+0000 7f7659e00640 1 -- 172.21.15.67:0/851182032 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7654159a00 con 0x7f7654153c10 2024-03-09T09:24:14.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.513+0000 7f764be00640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7654153c10 0x7f76540721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.514+0000 7f764be00640 1 -- 172.21.15.67:0/851182032 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f76540c2c60 con 0x7f7654153c10 2024-03-09T09:24:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.514+0000 7f764be00640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7654153c10 0x7f76540721c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7640000c00 tx=0x7f7640009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.515+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764004f070 con 0x7f7654153c10 2024-03-09T09:24:14.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.515+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7f7640003c30 con 0x7f7654153c10 2024-03-09T09:24:14.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.515+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7654153c10 msgr2=0x7f76540721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.515+0000 7f7651600640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7654153c10 0x7f76540721c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7640000c00 tx=0x7f7640009240 comp rx=0 tx=0).stop 2024-03-09T09:24:14.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.515+0000 7f7651600640 1 -- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7654153c10 msgr2=0x7f76540721c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:24:14.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.515+0000 7f7651600640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7654153c10 0x7f76540721c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.523 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:24:14.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:24:14.523 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:24:14.523 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:24:14.523 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 248, 2024-03-09T09:24:14.523 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:24:14.523 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:24:14.523 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:24:14.523 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.524 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:24:14.524 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:24:14.524 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:24:14.524 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:24:14.524 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:14.524 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:14.524 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:14.524 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:14.524 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:14.525 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:24:14.526 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:24:14.527 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:24:14.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:24:14.530 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.532 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:24:14.533 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.534 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.535 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:24:14.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.535 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:14.535 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.535 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:24:14.535 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.535 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.535 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:24:14.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.535 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:14.536 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.536 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:24:14.536 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.536 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.536 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:24:14.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.536 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:24:14.536 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.536 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:14.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:14.537 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.538 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.538 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:24:14.538 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:24:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.516+0000 7f7659e00640 1 -- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f762806b480 msgr2=0x7f762806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.516+0000 7f7659e00640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f762806b480 0x7f762806d940 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f76540ade30 tx=0x7f764c0095f0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.516+0000 7f7659e00640 1 -- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76540c3010 msgr2=0x7f765406dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.516+0000 7f7659e00640 1 --2- 172.21.15.67:0/851182032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76540c3010 0x7f765406dfa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f763800b550 tx=0x7f763800ba10 comp rx=0 tx=0).stop 2024-03-09T09:24:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.516+0000 7f7659400640 1 -- 172.21.15.67:0/851182032 reap_dead start 2024-03-09T09:24:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.516+0000 7f7659400640 1 -- 172.21.15.67:0/851182032 reap_dead start 2024-03-09T09:24:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.516+0000 7f7659e00640 1 -- 172.21.15.67:0/851182032 shutdown_connections 2024-03-09T09:24:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.516+0000 7f7659e00640 1 -- 172.21.15.67:0/851182032 >> 172.21.15.67:0/851182032 conn(0x7f76540a9740 msgr2=0x7f76540c2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.516+0000 7f7659e00640 1 -- 172.21.15.67:0/851182032 shutdown_connections 2024-03-09T09:24:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.516+0000 7f7659e00640 1 -- 172.21.15.67:0/851182032 wait complete. 2024-03-09T09:24:14.541 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:24:14.541 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:24:14.541 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:24:14.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.755+0000 7f175d400640 1 Processor -- start 2024-03-09T09:24:14.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.755+0000 7f175d400640 1 -- start start 2024-03-09T09:24:14.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.756+0000 7f175d400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f175814c9a0 0x7f17581567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.756+0000 7f175d400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17580ac9c0 0x7f17580ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.756+0000 7f175d400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17580ac4e0 con 0x7f17580c1d70 2024-03-09T09:24:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.756+0000 7f175d400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175806d590 con 0x7f175814c9a0 2024-03-09T09:24:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.756+0000 7f175d400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1758156d00 con 0x7f17580ac9c0 2024-03-09T09:24:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.756+0000 7f174fe00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f17580c1d70 legacy=0x7f17580c07a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:24:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.757+0000 7f1757200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17580ac9c0 0x7f17580ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.757+0000 7f1756800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f175814c9a0 0x7f17581567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.757+0000 7f1757200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17580ac9c0 0x7f17580ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:34722/0 (socket says 172.21.15.67:34722) 2024-03-09T09:24:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.757+0000 7f1757200640 1 -- 172.21.15.67:0/2289245165 learned_addr learned my addr 172.21.15.67:0/2289245165 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:14.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.757+0000 7f1756800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f175814c9a0 0x7f17581567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:36994/0 (socket says 172.21.15.67:36994) 2024-03-09T09:24:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.758+0000 7f1757200640 1 -- 172.21.15.67:0/2289245165 >> v2:172.21.15.134:3300/0 conn(0x7f175814c9a0 msgr2=0x7f17581567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.758+0000 7f1757200640 1 --2- 172.21.15.67:0/2289245165 >> v2:172.21.15.134:3300/0 conn(0x7f175814c9a0 0x7f17581567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.758+0000 7f1757200640 1 -- 172.21.15.67:0/2289245165 >> v1:172.21.15.67:6789/0 conn(0x7f17580c1d70 legacy=0x7f17580c07a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.758+0000 7f1757200640 1 -- 172.21.15.67:0/2289245165 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580aa550 con 0x7f17580ac9c0 2024-03-09T09:24:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.758+0000 7f1756800640 1 --2- 172.21.15.67:0/2289245165 >> v2:172.21.15.134:3300/0 conn(0x7f175814c9a0 0x7f17581567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:24:14.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.759+0000 7f1757200640 1 --2- 172.21.15.67:0/2289245165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17580ac9c0 0x7f17580ace50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1750009e50 tx=0x7f175002be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d68880935d4411af server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:14.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.759+0000 7f1755e00640 1 -- 172.21.15.67:0/2289245165 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175002c9f0 con 0x7f17580ac9c0 2024-03-09T09:24:14.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.759+0000 7f1755e00640 1 -- 172.21.15.67:0/2289245165 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175002cb90 con 0x7f17580ac9c0 2024-03-09T09:24:14.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.759+0000 7f1755e00640 1 -- 172.21.15.67:0/2289245165 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17500343f0 con 0x7f17580ac9c0 2024-03-09T09:24:14.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.760+0000 7f175d400640 1 -- 172.21.15.67:0/2289245165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17580ac9c0 msgr2=0x7f17580ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.760+0000 7f175d400640 1 --2- 172.21.15.67:0/2289245165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17580ac9c0 0x7f17580ace50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1750009e50 tx=0x7f175002be50 comp rx=0 tx=0).stop 2024-03-09T09:24:14.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.760+0000 7f175d400640 1 -- 172.21.15.67:0/2289245165 shutdown_connections 2024-03-09T09:24:14.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.760+0000 7f175d400640 1 --2- 172.21.15.67:0/2289245165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17580ac9c0 0x7f17580ace50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.760+0000 7f175d400640 1 --2- 172.21.15.67:0/2289245165 >> v2:172.21.15.134:3300/0 conn(0x7f175814c9a0 0x7f17581567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.760+0000 7f175d400640 1 -- 172.21.15.67:0/2289245165 >> 172.21.15.67:0/2289245165 conn(0x7f17580c7370 msgr2=0x7f17580c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.760+0000 7f175d400640 1 -- 172.21.15.67:0/2289245165 shutdown_connections 2024-03-09T09:24:14.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.760+0000 7f175d400640 1 -- 172.21.15.67:0/2289245165 wait complete. 2024-03-09T09:24:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.762+0000 7f175d400640 1 Processor -- start 2024-03-09T09:24:14.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.762+0000 7f175d400640 1 -- start start 2024-03-09T09:24:14.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.763+0000 7f175d400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17580ac9c0 0x7f17581486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.763+0000 7f175d400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17580c1d70 0x7f1758148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.763+0000 7f175d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f175814c9a0 0x7f17581ef190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.763+0000 7f175d400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580aadf0 con 0x7f175814c9a0 2024-03-09T09:24:14.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.763+0000 7f175d400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580aab10 con 0x7f17580ac9c0 2024-03-09T09:24:14.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.763+0000 7f175d400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17580c0eb0 con 0x7f17580c1d70 2024-03-09T09:24:14.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.763+0000 7f1757200640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f175814c9a0 msgr2=0x7f17581ef190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:24:14.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.763+0000 7f1757200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f175814c9a0 0x7f17581ef190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:14.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.763+0000 7f1756800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17580ac9c0 0x7f17581486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.764+0000 7f1756800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17580ac9c0 0x7f17581486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:37004/0 (socket says 172.21.15.67:37004) 2024-03-09T09:24:14.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.764+0000 7f1756800640 1 -- 172.21.15.67:0/1991133488 learned_addr learned my addr 172.21.15.67:0/1991133488 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:14.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.764+0000 7f174fe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17580c1d70 0x7f1758148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.764+0000 7f1756800640 1 -- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17580c1d70 msgr2=0x7f1758148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.764+0000 7f1756800640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17580c1d70 0x7f1758148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.764+0000 7f1756800640 1 -- 172.21.15.67:0/1991133488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f175814c9a0 msgr2=0x7f17581ef190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:24:14.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.764+0000 7f1756800640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f175814c9a0 0x7f17581ef190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.764+0000 7f1756800640 1 -- 172.21.15.67:0/1991133488 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17581efe30 con 0x7f17580ac9c0 2024-03-09T09:24:14.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.765+0000 7f1756800640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17580ac9c0 0x7f17581486f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f174800ebe0 tx=0x7f1748009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:14.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.765+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748017070 con 0x7f17580ac9c0 2024-03-09T09:24:14.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.765+0000 7f175d400640 1 -- 172.21.15.67:0/1991133488 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17581580c0 con 0x7f17580ac9c0 2024-03-09T09:24:14.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.765+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174800ca00 con 0x7f17580ac9c0 2024-03-09T09:24:14.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.765+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174800cdd0 con 0x7f17580ac9c0 2024-03-09T09:24:14.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.765+0000 7f175d400640 1 -- 172.21.15.67:0/1991133488 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17581df7e0 con 0x7f17580ac9c0 2024-03-09T09:24:14.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.766+0000 7f175d400640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758149260 0x7f17581ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.766+0000 7f175d400640 1 -- 172.21.15.67:0/1991133488 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17581efc10 con 0x7f1758149260 2024-03-09T09:24:14.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.767+0000 7f174fe00640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758149260 0x7f17581ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.767+0000 7f174fe00640 1 -- 172.21.15.67:0/1991133488 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17581dfa20 con 0x7f1758149260 2024-03-09T09:24:14.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.768+0000 7f174fe00640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758149260 0x7f17581ef6d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1758073050 tx=0x7f1740009aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:14.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.768+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740016070 con 0x7f1758149260 2024-03-09T09:24:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.768+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f174801a070 con 0x7f17580ac9c0 2024-03-09T09:24:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.769+0000 7f174ea00640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f173406b3d0 0x7f173406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.769+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f174808a720 con 0x7f17580ac9c0 2024-03-09T09:24:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.770+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f1740005a40 con 0x7f1758149260 2024-03-09T09:24:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.770+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758149260 msgr2=0x7f17581ef6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.770+0000 7f174ea00640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758149260 0x7f17581ef6d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1758073050 tx=0x7f1740009aa0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.770+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758149260 msgr2=0x7f17581ef6d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:24:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.770+0000 7f174ea00640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758149260 0x7f17581ef6d0 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1758073050 tx=0x7f1740009aa0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.770+0000 7f1757200640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f173406b3d0 0x7f173406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.771+0000 7f1757200640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f173406b3d0 0x7f173406d890 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1750035000 tx=0x7f1750030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:14.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.777+0000 7f175d400640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758071e10 0x7f1758072220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:14.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.777+0000 7f175d400640 1 -- 172.21.15.67:0/1991133488 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1758157f10 con 0x7f1758071e10 2024-03-09T09:24:14.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.777+0000 7f174fe00640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758071e10 0x7f1758072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:14.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.778+0000 7f174fe00640 1 -- 172.21.15.67:0/1991133488 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1758075a00 con 0x7f1758071e10 2024-03-09T09:24:14.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.779+0000 7f174fe00640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758071e10 0x7f1758072220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1758073050 tx=0x7f17400059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:14.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.779+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174001bc20 con 0x7f1758071e10 2024-03-09T09:24:14.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.780+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5176 (secure 0 0 0) 0x7f17400103c0 con 0x7f1758071e10 2024-03-09T09:24:14.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.780+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758071e10 msgr2=0x7f1758072220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.780+0000 7f174ea00640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758071e10 0x7f1758072220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1758073050 tx=0x7f17400059d0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.780+0000 7f174ea00640 1 -- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758071e10 msgr2=0x7f1758072220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:24:14.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.780+0000 7f174ea00640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1758071e10 0x7f1758072220 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1758073050 tx=0x7f17400059d0 comp rx=0 tx=0).stop 2024-03-09T09:24:14.787 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:24:14.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 248, 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:24:14.788 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:24:14.789 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.790 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:24:14.791 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:14.792 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.793 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:24:14.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.793 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:24:14.793 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.793 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:24:14.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:24:14.793 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:14.793 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:14.793 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:24:14.794 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:14.795 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:14.795 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:14.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.795 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:24:14.795 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:24:14.795 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:14.795 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.797 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:24:14.797 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:24:14.797 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:14.797 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:14.797 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:14.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.797 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:24:14.797 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:24:14.797 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:14.797 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.799 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:24:14.799 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:24:14.799 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:14.799 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:14.799 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:14.799 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.799 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.799 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:24:14.799 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:24:14.800 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.801 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.801 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:24:14.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.801 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:14.801 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.801 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:24:14.801 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.801 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.801 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:24:14.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.802 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:24:14.802 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.802 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:14.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.802 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:14.802 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.802 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:14.802 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.803 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:14.803 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:24:14.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:14.803 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:14.803 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:14.803 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:14.803 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:14.803 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:14.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:14.803 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:24:14.804 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:24:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.781+0000 7f175d400640 1 -- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f173406b3d0 msgr2=0x7f173406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.781+0000 7f175d400640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f173406b3d0 0x7f173406d890 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1750035000 tx=0x7f1750030000 comp rx=0 tx=0).stop 2024-03-09T09:24:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.781+0000 7f175d400640 1 -- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17580ac9c0 msgr2=0x7f17581486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.781+0000 7f175d400640 1 --2- 172.21.15.67:0/1991133488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17580ac9c0 0x7f17581486f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f174800ebe0 tx=0x7f1748009e20 comp rx=0 tx=0).stop 2024-03-09T09:24:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.781+0000 7f1757200640 1 -- 172.21.15.67:0/1991133488 reap_dead start 2024-03-09T09:24:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.781+0000 7f1757200640 1 -- 172.21.15.67:0/1991133488 reap_dead start 2024-03-09T09:24:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.781+0000 7f175d400640 1 -- 172.21.15.67:0/1991133488 shutdown_connections 2024-03-09T09:24:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.781+0000 7f175d400640 1 -- 172.21.15.67:0/1991133488 >> 172.21.15.67:0/1991133488 conn(0x7f17580c7370 msgr2=0x7f17580c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.781+0000 7f175d400640 1 -- 172.21.15.67:0/1991133488 shutdown_connections 2024-03-09T09:24:14.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:14.781+0000 7f175d400640 1 -- 172.21.15.67:0/1991133488 wait complete. 2024-03-09T09:24:14.817 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:24:14.817 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:24:19.141 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:24:19.141 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:24:19.141 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 79 -eq 48 ']' 2024-03-09T09:24:19.141 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.79 2024-03-09T09:24:19.142 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.79 2024-03-09T09:24:19.142 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.79 2024-03-09T09:24:19.142 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.79 2024-03-09T09:24:19.145 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:24:19.215 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:24:19.215 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:19.215 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 85 -eq 48 ']' 2024-03-09T09:24:19.215 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.85 2024-03-09T09:24:19.215 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.85 2024-03-09T09:24:19.216 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.85 2024-03-09T09:24:19.216 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.85 2024-03-09T09:24:19.220 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:24:26.247 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:24:26.248 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:26.248 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 86 -eq 48 ']' 2024-03-09T09:24:26.248 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.86 2024-03-09T09:24:26.248 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.86 2024-03-09T09:24:26.249 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.86 2024-03-09T09:24:26.249 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.86 2024-03-09T09:24:26.253 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:24:28.007 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:24:28.007 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:24:28.008 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 80 -eq 48 ']' 2024-03-09T09:24:28.008 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.80 2024-03-09T09:24:28.008 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.80 2024-03-09T09:24:28.008 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.80 2024-03-09T09:24:28.008 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.80 2024-03-09T09:24:28.012 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:24:31.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:24:31.088 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:24:33.702 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:24:33.702 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:24:33.702 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 81 -eq 48 ']' 2024-03-09T09:24:33.702 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.81 2024-03-09T09:24:33.703 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.81 2024-03-09T09:24:33.703 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.81 2024-03-09T09:24:33.703 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.81 2024-03-09T09:24:33.707 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:24:33.946 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:24:33.946 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:33.946 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 87 -eq 48 ']' 2024-03-09T09:24:33.946 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.87 2024-03-09T09:24:33.946 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.87 2024-03-09T09:24:33.947 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.87 2024-03-09T09:24:33.947 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.87 2024-03-09T09:24:33.951 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:24:34.817 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:24:34.818 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-03-09T09:24:34.818 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-09T09:24:34.818 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-09T09:24:34.826 INFO:tasks.ceph.mon.a:Started 2024-03-09T09:24:34.826 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:24:34.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:24:35.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.023+0000 7fbeb2600640 1 Processor -- start 2024-03-09T09:24:35.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.023+0000 7fbeb2600640 1 -- start start 2024-03-09T09:24:35.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.024+0000 7fbeb2600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbeac0a9ab0 0x7fbeac0ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:35.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.024+0000 7fbeb2600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbeac15a1b0 0x7fbeac0aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:35.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.024+0000 7fbeb2600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbeac0abcd0 0x7fbeac0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:35.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.024+0000 7fbeb2600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeac15a700 con 0x7fbeac0abcd0 2024-03-09T09:24:35.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.024+0000 7fbeb2600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeac15a9e0 con 0x7fbeac15a1b0 2024-03-09T09:24:35.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.024+0000 7fbeb2600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac15acc0 con 0x7fbeac0a9ab0 2024-03-09T09:24:35.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.025+0000 7fbeabe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbeac0abcd0 0x7fbeac0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:35.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.025+0000 7fbeabe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbeac0abcd0 0x7fbeac0aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60544/0 (socket says 172.21.15.67:60544) 2024-03-09T09:24:35.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.025+0000 7fbeabe00640 1 -- 172.21.15.67:0/718139922 learned_addr learned my addr 172.21.15.67:0/718139922 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.025+0000 7fbeab400640 1 --2- 172.21.15.67:0/718139922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbeac0a9ab0 0x7fbeac0ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.025+0000 7fbeabe00640 1 -- 172.21.15.67:0/718139922 >> v2:172.21.15.67:3300/0 conn(0x7fbeac0abcd0 msgr2=0x7fbeac0aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.025+0000 7fbeabe00640 1 -- 172.21.15.67:0/718139922 >> v2:172.21.15.67:3300/0 conn(0x7fbeac0abcd0 msgr2=0x7fbeac0aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.025+0000 7fbeabe00640 1 --2- 172.21.15.67:0/718139922 >> v2:172.21.15.67:3300/0 conn(0x7fbeac0abcd0 0x7fbeac0aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.025+0000 7fbeaaa00640 1 --2- 172.21.15.67:0/718139922 >> v2:172.21.15.134:3300/0 conn(0x7fbeac15a1b0 0x7fbeac0aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.025+0000 7fbeabe00640 1 --2- 172.21.15.67:0/718139922 >> v2:172.21.15.67:3300/0 conn(0x7fbeac0abcd0 0x7fbeac0aae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.026+0000 7fbeab400640 1 -- 172.21.15.67:0/718139922 >> v2:172.21.15.134:3300/0 conn(0x7fbeac15a1b0 msgr2=0x7fbeac0aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.026+0000 7fbeab400640 1 --2- 172.21.15.67:0/718139922 >> v2:172.21.15.134:3300/0 conn(0x7fbeac15a1b0 0x7fbeac0aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.026+0000 7fbeab400640 1 -- 172.21.15.67:0/718139922 >> v2:172.21.15.67:3300/0 conn(0x7fbeac0abcd0 msgr2=0x7fbeac0aae40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.026+0000 7fbeab400640 1 --2- 172.21.15.67:0/718139922 >> v2:172.21.15.67:3300/0 conn(0x7fbeac0abcd0 0x7fbeac0aae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.026+0000 7fbeab400640 1 -- 172.21.15.67:0/718139922 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac15afa0 con 0x7fbeac0a9ab0 2024-03-09T09:24:35.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.027+0000 7fbeab400640 1 --2- 172.21.15.67:0/718139922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbeac0a9ab0 0x7fbeac0ab790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0097a0 tx=0x7fbe9c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd3eceb757f37d72 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.027+0000 7fbeaa000640 1 -- 172.21.15.67:0/718139922 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c031780 con 0x7fbeac0a9ab0 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.027+0000 7fbeaa000640 1 -- 172.21.15.67:0/718139922 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c037070 con 0x7fbeac0a9ab0 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.027+0000 7fbeaa000640 1 -- 172.21.15.67:0/718139922 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c031b40 con 0x7fbeac0a9ab0 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.028+0000 7fbeb2600640 1 -- 172.21.15.67:0/718139922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbeac0a9ab0 msgr2=0x7fbeac0ab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.028+0000 7fbeb2600640 1 --2- 172.21.15.67:0/718139922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbeac0a9ab0 0x7fbeac0ab790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0097a0 tx=0x7fbe9c02b820 comp rx=0 tx=0).stop 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.028+0000 7fbeb2600640 1 -- 172.21.15.67:0/718139922 shutdown_connections 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.028+0000 7fbeb2600640 1 --2- 172.21.15.67:0/718139922 >> v2:172.21.15.67:3300/0 conn(0x7fbeac0abcd0 0x7fbeac0aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.028+0000 7fbeb2600640 1 --2- 172.21.15.67:0/718139922 >> v2:172.21.15.134:3300/0 conn(0x7fbeac15a1b0 0x7fbeac0aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.028+0000 7fbeb2600640 1 --2- 172.21.15.67:0/718139922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbeac0a9ab0 0x7fbeac0ab790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.028+0000 7fbeb2600640 1 -- 172.21.15.67:0/718139922 >> 172.21.15.67:0/718139922 conn(0x7fbeac0c7370 msgr2=0x7fbeac0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.028+0000 7fbeb2600640 1 -- 172.21.15.67:0/718139922 shutdown_connections 2024-03-09T09:24:35.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.028+0000 7fbeb2600640 1 -- 172.21.15.67:0/718139922 wait complete. 2024-03-09T09:24:35.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.030+0000 7fbeb2600640 1 Processor -- start 2024-03-09T09:24:35.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.030+0000 7fbeb2600640 1 -- start start 2024-03-09T09:24:35.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.031+0000 7fbeb2600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbeac0a9ab0 0x7fbeac14dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:35.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.031+0000 7fbeb2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeac0abcd0 0x7fbeac15d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:35.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.031+0000 7fbeb2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbeac15a1b0 0x7fbeac15dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.031+0000 7fbeb2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac06e120 con 0x7fbeac0abcd0 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.031+0000 7fbeb2600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac06dfa0 con 0x7fbeac15a1b0 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.031+0000 7fbeb2600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac0ab380 con 0x7fbeac0a9ab0 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.031+0000 7fbeaaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeac0abcd0 0x7fbeac15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.031+0000 7fbeaaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeac0abcd0 0x7fbeac15d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60550/0 (socket says 172.21.15.67:60550) 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.031+0000 7fbeaaa00640 1 -- 172.21.15.67:0/1262447246 learned_addr learned my addr 172.21.15.67:0/1262447246 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.032+0000 7fbeaaa00640 1 -- 172.21.15.67:0/1262447246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeac0abcd0 msgr2=0x7fbeac15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.032+0000 7fbeaaa00640 1 -- 172.21.15.67:0/1262447246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeac0abcd0 msgr2=0x7fbeac15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.032+0000 7fbeaaa00640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeac0abcd0 0x7fbeac15d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.032+0000 7fbeaaa00640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeac0abcd0 0x7fbeac15d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.032+0000 7fbeabe00640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbeac15a1b0 0x7fbeac15dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.032+0000 7fbeab400640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbeac0a9ab0 0x7fbeac14dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:35.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.032+0000 7fbeabe00640 1 -- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbeac0a9ab0 msgr2=0x7fbeac14dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:35.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.032+0000 7fbeabe00640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbeac0a9ab0 0x7fbeac14dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.032+0000 7fbeabe00640 1 -- 172.21.15.67:0/1262447246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeac0abcd0 msgr2=0x7fbeac15d720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:24:35.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.032+0000 7fbeabe00640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeac0abcd0 0x7fbeac15d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.032+0000 7fbeabe00640 1 -- 172.21.15.67:0/1262447246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac0735e0 con 0x7fbeac15a1b0 2024-03-09T09:24:35.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.033+0000 7fbeabe00640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbeac15a1b0 0x7fbeac15dc60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009fd0 tx=0x7fbea4007ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:35.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.033+0000 7fbea3400640 1 -- 172.21.15.67:0/1262447246 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4003b90 con 0x7fbeac15a1b0 2024-03-09T09:24:35.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.033+0000 7fbea3400640 1 -- 172.21.15.67:0/1262447246 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea4003d50 con 0x7fbeac15a1b0 2024-03-09T09:24:35.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.033+0000 7fbea3400640 1 -- 172.21.15.67:0/1262447246 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea400dbc0 con 0x7fbeac15a1b0 2024-03-09T09:24:35.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.033+0000 7fbeb2600640 1 -- 172.21.15.67:0/1262447246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeac157a20 con 0x7fbeac15a1b0 2024-03-09T09:24:35.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.033+0000 7fbeb2600640 1 -- 172.21.15.67:0/1262447246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac070850 con 0x7fbeac15a1b0 2024-03-09T09:24:35.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.035+0000 7fbeb2600640 1 -- 172.21.15.67:0/1262447246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeac072f40 con 0x7fbeac15a1b0 2024-03-09T09:24:35.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.036+0000 7fbea3400640 1 -- 172.21.15.67:0/1262447246 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbea4003f00 con 0x7fbeac15a1b0 2024-03-09T09:24:35.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.037+0000 7fbea3400640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbe8c06b4d0 0x7fbe8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:35.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.037+0000 7fbea3400640 1 -- 172.21.15.67:0/1262447246 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbea408acd0 con 0x7fbeac15a1b0 2024-03-09T09:24:35.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.038+0000 7fbeab400640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbe8c06b4d0 0x7fbe8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:35.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.038+0000 7fbeab400640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbe8c06b4d0 0x7fbe8c06d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c036000 tx=0x7fbe9c03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:35.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.050+0000 7fbea3400640 1 -- 172.21.15.67:0/1262447246 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbea4010070 con 0x7fbeac15a1b0 2024-03-09T09:24:35.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.360+0000 7fbeb2600640 1 -- 172.21.15.67:0/1262447246 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbeac075a00 con 0x7fbeac15a1b0 2024-03-09T09:24:35.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.361+0000 7fbea3400640 1 -- 172.21.15.67:0/1262447246 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fbea4059180 con 0x7fbeac15a1b0 2024-03-09T09:24:35.368 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":248,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:24:35.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.363+0000 7fbeb2600640 1 -- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbe8c06b4d0 msgr2=0x7fbe8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:35.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.363+0000 7fbeb2600640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbe8c06b4d0 0x7fbe8c06d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c036000 tx=0x7fbe9c03d040 comp rx=0 tx=0).stop 2024-03-09T09:24:35.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.363+0000 7fbeb2600640 1 -- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbeac15a1b0 msgr2=0x7fbeac15dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:35.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.363+0000 7fbeb2600640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbeac15a1b0 0x7fbeac15dc60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009fd0 tx=0x7fbea4007ef0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.363+0000 7fbeb2600640 1 -- 172.21.15.67:0/1262447246 shutdown_connections 2024-03-09T09:24:35.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.363+0000 7fbeb2600640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbe8c06b4d0 0x7fbe8c06d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.363+0000 7fbeb2600640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbeac15a1b0 0x7fbeac15dc60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.363+0000 7fbeb2600640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbeac0abcd0 0x7fbeac15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.363+0000 7fbeb2600640 1 --2- 172.21.15.67:0/1262447246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbeac0a9ab0 0x7fbeac14dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:35.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.363+0000 7fbeb2600640 1 -- 172.21.15.67:0/1262447246 >> 172.21.15.67:0/1262447246 conn(0x7fbeac0c7370 msgr2=0x7fbeac158ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:35.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.364+0000 7fbeb2600640 1 -- 172.21.15.67:0/1262447246 shutdown_connections 2024-03-09T09:24:35.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:35.364+0000 7fbeb2600640 1 -- 172.21.15.67:0/1262447246 wait complete. 2024-03-09T09:24:38.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:24:38.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.591+0000 7f23d8600640 1 Processor -- start 2024-03-09T09:24:38.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.591+0000 7f23d8600640 1 -- start start 2024-03-09T09:24:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.592+0000 7f23d8600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 0x7f23d0157d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.592+0000 7f23d8600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d0153d40 con 0x7f23d00ac730 2024-03-09T09:24:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.592+0000 7f23d8600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d00c2790 con 0x7f23d00c22a0 2024-03-09T09:24:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.592+0000 7f23d8600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0070ee0 con 0x7f23d0071190 2024-03-09T09:24:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.592+0000 7f23d5c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f23d00ac730 0x7f23d0157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:51262/0 (socket says 172.21.15.67:51262) 2024-03-09T09:24:38.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.592+0000 7f23d5c00640 1 -- 172.21.15.67:0/2592058911 learned_addr learned my addr 172.21.15.67:0/2592058911 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:24:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.593+0000 7f23d6600640 1 --2- 172.21.15.67:0/2592058911 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 0x7f23d0157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.594+0000 7f23cfe00640 1 -- 172.21.15.67:0/2592058911 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866239675 0 0) 0x7f23d00c2790 con 0x7f23d00c22a0 2024-03-09T09:24:38.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.594+0000 7f23cfe00640 1 -- 172.21.15.67:0/2592058911 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23b8003520 con 0x7f23d00c22a0 2024-03-09T09:24:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.594+0000 7f23d6600640 1 -- 172.21.15.67:0/2592058911 >> v1:172.21.15.134:6789/0 conn(0x7f23d00c22a0 legacy=0x7f23d015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.594+0000 7f23d6600640 1 -- 172.21.15.67:0/2592058911 >> v1:172.21.15.67:6789/0 conn(0x7f23d00ac730 legacy=0x7f23d0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.594+0000 7f23d6600640 1 -- 172.21.15.67:0/2592058911 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d0070920 con 0x7f23d0071190 2024-03-09T09:24:38.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.595+0000 7f23d6600640 1 --2- 172.21.15.67:0/2592058911 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 0x7f23d0157d60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009dc0 tx=0x7f23c802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=345b623e65b11a92 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:38.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.596+0000 7f23cfe00640 1 -- 172.21.15.67:0/2592058911 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8039070 con 0x7f23d0071190 2024-03-09T09:24:38.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.596+0000 7f23cfe00640 1 -- 172.21.15.67:0/2592058911 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c802cc40 con 0x7f23d0071190 2024-03-09T09:24:38.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.596+0000 7f23cfe00640 1 -- 172.21.15.67:0/2592058911 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8034440 con 0x7f23d0071190 2024-03-09T09:24:38.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.597+0000 7f23d8600640 1 -- 172.21.15.67:0/2592058911 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 msgr2=0x7f23d0157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:38.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.597+0000 7f23d8600640 1 --2- 172.21.15.67:0/2592058911 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 0x7f23d0157d60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009dc0 tx=0x7f23c802be90 comp rx=0 tx=0).stop 2024-03-09T09:24:38.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.597+0000 7f23d8600640 1 -- 172.21.15.67:0/2592058911 shutdown_connections 2024-03-09T09:24:38.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.597+0000 7f23d8600640 1 --2- 172.21.15.67:0/2592058911 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 0x7f23d0157d60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:38.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.597+0000 7f23d8600640 1 -- 172.21.15.67:0/2592058911 >> 172.21.15.67:0/2592058911 conn(0x7f23d00c7370 msgr2=0x7f23d00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:38.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.597+0000 7f23d8600640 1 -- 172.21.15.67:0/2592058911 shutdown_connections 2024-03-09T09:24:38.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.597+0000 7f23d8600640 1 -- 172.21.15.67:0/2592058911 wait complete. 2024-03-09T09:24:38.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.599+0000 7f23d8600640 1 Processor -- start 2024-03-09T09:24:38.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.599+0000 7f23d8600640 1 -- start start 2024-03-09T09:24:38.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d8600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 0x7f23d01ea730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:38.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d8600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d00ac730 0x7f23d00c1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:38.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d8600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d00c22a0 0x7f23d00bf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d8600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0151610 con 0x7f23d00ac730 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d8600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0154d00 con 0x7f23d00c22a0 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d8600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0151790 con 0x7f23d0071190 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d00ac730 0x7f23d00c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d5200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d00ac730 0x7f23d00c1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60556/0 (socket says 172.21.15.67:60556) 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d5200640 1 -- 172.21.15.67:0/559595862 learned_addr learned my addr 172.21.15.67:0/559595862 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d5200640 1 -- 172.21.15.67:0/559595862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d00ac730 msgr2=0x7f23d00c1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d5200640 1 -- 172.21.15.67:0/559595862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d00ac730 msgr2=0x7f23d00c1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d5200640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d00ac730 0x7f23d00c1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.600+0000 7f23d5200640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d00ac730 0x7f23d00c1130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.601+0000 7f23d5c00640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 0x7f23d01ea730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:38.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.601+0000 7f23d6600640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d00c22a0 0x7f23d00bf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:38.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.601+0000 7f23d6600640 1 -- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 msgr2=0x7f23d01ea730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:38.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.601+0000 7f23d6600640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 0x7f23d01ea730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:38.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.602+0000 7f23d6600640 1 -- 172.21.15.67:0/559595862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d00ac730 msgr2=0x7f23d00c1130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:24:38.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.602+0000 7f23d6600640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d00ac730 0x7f23d00c1130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:38.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.602+0000 7f23d6600640 1 -- 172.21.15.67:0/559595862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d00bd1a0 con 0x7f23d00c22a0 2024-03-09T09:24:38.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.602+0000 7f23d5c00640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 0x7f23d01ea730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:24:38.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.602+0000 7f23d6600640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d00c22a0 0x7f23d00bf540 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23c8033000 tx=0x7f23c8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:38.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.603+0000 7f23cd600640 1 -- 172.21.15.67:0/559595862 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8042030 con 0x7f23d00c22a0 2024-03-09T09:24:38.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.603+0000 7f23cd600640 1 -- 172.21.15.67:0/559595862 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c803a040 con 0x7f23d00c22a0 2024-03-09T09:24:38.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.603+0000 7f23d8600640 1 -- 172.21.15.67:0/559595862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d00bd480 con 0x7f23d00c22a0 2024-03-09T09:24:38.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.603+0000 7f23cd600640 1 -- 172.21.15.67:0/559595862 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8039070 con 0x7f23d00c22a0 2024-03-09T09:24:38.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.603+0000 7f23d8600640 1 -- 172.21.15.67:0/559595862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d00bfa80 con 0x7f23d00c22a0 2024-03-09T09:24:38.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.604+0000 7f23d8600640 1 -- 172.21.15.67:0/559595862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d0068d50 con 0x7f23d00c22a0 2024-03-09T09:24:38.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.606+0000 7f23cd600640 1 -- 172.21.15.67:0/559595862 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f23c8045070 con 0x7f23d00c22a0 2024-03-09T09:24:38.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.606+0000 7f23cd600640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b4d0 0x7f23b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:38.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.607+0000 7f23cd600640 1 -- 172.21.15.67:0/559595862 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f23c80ac9f0 con 0x7f23d00c22a0 2024-03-09T09:24:38.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.607+0000 7f23d5c00640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b4d0 0x7f23b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:38.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.608+0000 7f23d5c00640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b4d0 0x7f23b006d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f23c4000f60 tx=0x7f23c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:38.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.619+0000 7f23cd600640 1 -- 172.21.15.67:0/559595862 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f23d0068d50 con 0x7f23d00c22a0 2024-03-09T09:24:38.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.960+0000 7f23d8600640 1 -- 172.21.15.67:0/559595862 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f23d0075a00 con 0x7f23d00c22a0 2024-03-09T09:24:38.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.961+0000 7f23cd600640 1 -- 172.21.15.67:0/559595862 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f23c804c070 con 0x7f23d00c22a0 2024-03-09T09:24:38.968 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":248,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":27,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:24:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 -- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b4d0 msgr2=0x7f23b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b4d0 0x7f23b006d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f23c4000f60 tx=0x7f23c4009040 comp rx=0 tx=0).stop 2024-03-09T09:24:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 -- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d00c22a0 msgr2=0x7f23d00bf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d00c22a0 0x7f23d00bf540 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23c8033000 tx=0x7f23c8004a40 comp rx=0 tx=0).stop 2024-03-09T09:24:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 -- 172.21.15.67:0/559595862 shutdown_connections 2024-03-09T09:24:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f23b006b4d0 0x7f23b006d990 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f23d00c22a0 0x7f23d00bf540 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23d00ac730 0x7f23d00c1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 --2- 172.21.15.67:0/559595862 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f23d0071190 0x7f23d01ea730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 -- 172.21.15.67:0/559595862 >> 172.21.15.67:0/559595862 conn(0x7f23d00c7370 msgr2=0x7f23d01574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 -- 172.21.15.67:0/559595862 shutdown_connections 2024-03-09T09:24:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:38.964+0000 7f23d8600640 1 -- 172.21.15.67:0/559595862 wait complete. 2024-03-09T09:24:39.910 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:24:39.910 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:39.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 88 -eq 48 ']' 2024-03-09T09:24:39.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.88 2024-03-09T09:24:39.911 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.88 2024-03-09T09:24:39.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.88 2024-03-09T09:24:39.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.88 2024-03-09T09:24:39.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:24:40.379 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:24:40.379 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:24:40.379 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 82 -eq 48 ']' 2024-03-09T09:24:40.379 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.82 2024-03-09T09:24:40.379 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.82 2024-03-09T09:24:40.380 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.82 2024-03-09T09:24:40.380 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.82 2024-03-09T09:24:40.383 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:24:41.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:24:42.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.218+0000 7fa1c5a00640 1 Processor -- start 2024-03-09T09:24:42.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.218+0000 7fa1c5a00640 1 -- start start 2024-03-09T09:24:42.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.219+0000 7fa1c5a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:42.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.219+0000 7fa1c5a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:42.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.219+0000 7fa1c5a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:42.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.219+0000 7fa1c5a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c00c3520 con 0x7fa1c00c7e10 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.219+0000 7fa1c5a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c00c52c0 con 0x7fa1c00c7a30 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.219+0000 7fa1c5a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00c2f10 con 0x7fa1c0073b90 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1be000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1be000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49134/0 (socket says 172.21.15.67:49134) 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 learned_addr learned my addr 172.21.15.67:0/450492841 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 msgr2=0x7fa1c006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 msgr2=0x7fa1c006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1bf400640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 msgr2=0x7fa1c00c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:24:42.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1bf400640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 msgr2=0x7fa1c00c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:42.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:42.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1bea00640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 msgr2=0x7fa1c006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:24:42.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:42.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1bea00640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 msgr2=0x7fa1c006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:42.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:42.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.220+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:24:42.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.421+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:42.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.421+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:42.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.421+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:42.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1bea00640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 msgr2=0x7fa1c006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:24:42.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1bea00640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 msgr2=0x7fa1c006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:42.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:24:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1bf400640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 msgr2=0x7fa1c00c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:24:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1bf400640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 msgr2=0x7fa1c00c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 msgr2=0x7fa1c006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:24:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 msgr2=0x7fa1c006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:24:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.422+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:24:42.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.823+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.823+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.823+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.824+0000 7fa1bea00640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 msgr2=0x7fa1c006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:24:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.824+0000 7fa1bea00640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 msgr2=0x7fa1c006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.824+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.824+0000 7fa1bf400640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 msgr2=0x7fa1c00c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:24:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.824+0000 7fa1bf400640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 msgr2=0x7fa1c00c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.824+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.824+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:24:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.824+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 msgr2=0x7fa1c006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:24:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.824+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 msgr2=0x7fa1c006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.824+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.824+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:24:42.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:42.825+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:24:43.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.626+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.626+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.626+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1bf400640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 msgr2=0x7fa1c00c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1bf400640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 msgr2=0x7fa1c00c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1bea00640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 msgr2=0x7fa1c006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1bea00640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 msgr2=0x7fa1c006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1bf400640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 msgr2=0x7fa1c006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 msgr2=0x7fa1c006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:24:43.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:43.627+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:24:45.169 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:24:45.170 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:45.170 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 89 -eq 48 ']' 2024-03-09T09:24:45.170 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.89 2024-03-09T09:24:45.171 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.89 2024-03-09T09:24:45.171 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.89 2024-03-09T09:24:45.171 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.89 2024-03-09T09:24:45.176 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:24:45.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:45.219+0000 7fa1b7e00640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 msgr2=0x7fa1c00c5ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:45.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:45.219+0000 7fa1b7e00640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c5ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:45.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:45.219+0000 7fa1b7e00640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 msgr2=0x7fa1c006e0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:45.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:45.219+0000 7fa1b7e00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.134:3300/0 conn(0x7fa1c00c7a30 0x7fa1c006e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:45.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:45.219+0000 7fa1b7e00640 1 -- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 msgr2=0x7fa1c006e610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:24:45.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:45.219+0000 7fa1b7e00640 1 --2- 172.21.15.67:0/450492841 >> v2:172.21.15.67:3300/0 conn(0x7fa1c00c7e10 0x7fa1c006e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:45.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:45.220+0000 7fa1b7e00640 1 -- 172.21.15.67:0/450492841 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c006eb50 con 0x7fa1a4003650 2024-03-09T09:24:45.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:45.220+0000 7fa1b7e00640 1 -- 172.21.15.67:0/450492841 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a400c070 con 0x7fa1a4007300 2024-03-09T09:24:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:46.290+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777872492 0 0) 0x7fa1a400c070 con 0x7fa1a4007300 2024-03-09T09:24:46.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:46.290+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa19c003520 con 0x7fa1a4007300 2024-03-09T09:24:47.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.666+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2782254062 0 0) 0x7fa19c003520 con 0x7fa1a4007300 2024-03-09T09:24:47.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.666+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 >> v1:172.21.15.67:6789/0 conn(0x7fa1a4003650 legacy=0x7fa1a4005ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:47.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.666+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a400d370 con 0x7fa1a4007300 2024-03-09T09:24:47.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.666+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fa1a800f040 con 0x7fa1a4007300 2024-03-09T09:24:47.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.667+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 >> v1:172.21.15.134:6789/0 conn(0x7fa1a4007300 legacy=0x7fa1a40097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:47.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.667+0000 7fa1bd600640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa19c003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:47.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.667+0000 7fa1bd600640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa19c004190 0x7fa19c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:47.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.667+0000 7fa1bd600640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa19c004f40 0x7fa19c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:47.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.667+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c00bf00 con 0x7fa19c004f40 2024-03-09T09:24:47.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.667+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c00c080 con 0x7fa19c004190 2024-03-09T09:24:47.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.667+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c00c200 con 0x7fa1c0073b90 2024-03-09T09:24:47.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.667+0000 7fa1bf400640 1 -- 172.21.15.67:0/450492841 reap_dead start 2024-03-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.668+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa19c004190 0x7fa19c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.668+0000 7fa1bea00640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa19c003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.669+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 msgr2=0x7fa19c003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.669+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa19c003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.669+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa19c004f40 msgr2=0x7fa19c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.669+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa19c004f40 0x7fa19c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:47.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.669+0000 7fa1be000640 1 -- 172.21.15.67:0/450492841 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c003520 con 0x7fa19c004190 2024-03-09T09:24:47.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.672+0000 7fa1be000640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa19c004190 0x7fa19c004800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400af30 tx=0x7fa19c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa913038b9daf02 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:47.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.673+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8037070 con 0x7fa19c004190 2024-03-09T09:24:47.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.673+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a8031c00 con 0x7fa19c004190 2024-03-09T09:24:47.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.673+0000 7fa1bd600640 1 -- 172.21.15.67:0/450492841 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800cb60 con 0x7fa19c004190 2024-03-09T09:24:47.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.674+0000 7fa1c5a00640 1 -- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa19c004190 msgr2=0x7fa19c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:47.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.674+0000 7fa1c5a00640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa19c004190 0x7fa19c004800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400af30 tx=0x7fa19c005830 comp rx=0 tx=0).stop 2024-03-09T09:24:47.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.674+0000 7fa1c5a00640 1 -- 172.21.15.67:0/450492841 shutdown_connections 2024-03-09T09:24:47.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.674+0000 7fa1c5a00640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa19c004f40 0x7fa19c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:47.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.674+0000 7fa1c5a00640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa19c004190 0x7fa19c004800 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:47.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.674+0000 7fa1c5a00640 1 --2- 172.21.15.67:0/450492841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa19c003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:47.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.674+0000 7fa1c5a00640 1 -- 172.21.15.67:0/450492841 >> 172.21.15.67:0/450492841 conn(0x7fa1c006b040 msgr2=0x7fa1c006b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:47.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.674+0000 7fa1c5a00640 1 -- 172.21.15.67:0/450492841 shutdown_connections 2024-03-09T09:24:47.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.674+0000 7fa1c5a00640 1 -- 172.21.15.67:0/450492841 wait complete. 2024-03-09T09:24:47.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.677+0000 7fa1c5a00640 1 Processor -- start 2024-03-09T09:24:47.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.677+0000 7fa1c5a00640 1 -- start start 2024-03-09T09:24:47.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.678+0000 7fa1c5a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa19c004190 0x7fa1c01ea690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:47.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.678+0000 7fa1c5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa19c004f40 0x7fa1c00ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:47.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.678+0000 7fa1c5a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:47.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.678+0000 7fa1c5a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00c5b80 con 0x7fa19c004f40 2024-03-09T09:24:47.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.678+0000 7fa1c5a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00c39b0 con 0x7fa19c004190 2024-03-09T09:24:47.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.678+0000 7fa1c5a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00c5d00 con 0x7fa1c0073b90 2024-03-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.679+0000 7fa1be000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa19c004f40 0x7fa1c00ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.679+0000 7fa1be000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa19c004f40 0x7fa1c00ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49164/0 (socket says 172.21.15.67:49164) 2024-03-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.679+0000 7fa1be000640 1 -- 172.21.15.67:0/3777746366 learned_addr learned my addr 172.21.15.67:0/3777746366 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.679+0000 7fa1bea00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa19c004190 0x7fa1c01ea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.679+0000 7fa1bf400640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.680+0000 7fa1be000640 1 -- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 msgr2=0x7fa1c00c1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.680+0000 7fa1be000640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.680+0000 7fa1be000640 1 -- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa19c004190 msgr2=0x7fa1c01ea690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.680+0000 7fa1be000640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa19c004190 0x7fa1c01ea690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:47.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.680+0000 7fa1be000640 1 -- 172.21.15.67:0/3777746366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c01eabd0 con 0x7fa19c004f40 2024-03-09T09:24:47.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.680+0000 7fa1be000640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa19c004f40 0x7fa1c00ac390 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00c8240 tx=0x7fa1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.682+0000 7fa1b6a00640 1 -- 172.21.15.67:0/3777746366 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a804a020 con 0x7fa19c004f40 2024-03-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.682+0000 7fa1b6a00640 1 -- 172.21.15.67:0/3777746366 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a8005f70 con 0x7fa19c004f40 2024-03-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.682+0000 7fa1b6a00640 1 -- 172.21.15.67:0/3777746366 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8006120 con 0x7fa19c004f40 2024-03-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.682+0000 7fa1c5a00640 1 -- 172.21.15.67:0/3777746366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c00a85e0 con 0x7fa19c004f40 2024-03-09T09:24:47.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.683+0000 7fa1c5a00640 1 -- 172.21.15.67:0/3777746366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c00a8300 con 0x7fa19c004f40 2024-03-09T09:24:47.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.685+0000 7fa1c5a00640 1 -- 172.21.15.67:0/3777746366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c0075a00 con 0x7fa19c004f40 2024-03-09T09:24:47.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.685+0000 7fa1b6a00640 1 -- 172.21.15.67:0/3777746366 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa1a804d020 con 0x7fa19c004f40 2024-03-09T09:24:47.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.687+0000 7fa1b6a00640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa1a406c130 0x7fa1a406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:47.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.687+0000 7fa1b6a00640 1 -- 172.21.15.67:0/3777746366 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fa1a80b1980 con 0x7fa19c004f40 2024-03-09T09:24:47.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.687+0000 7fa1bea00640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa1a406c130 0x7fa1a406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:47.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.689+0000 7fa1bea00640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa1a406c130 0x7fa1a406e5f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0006e80 tx=0x7fa1b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:47.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:47.693+0000 7fa1b6a00640 1 -- 172.21.15.67:0/3777746366 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa1a8054020 con 0x7fa19c004f40 2024-03-09T09:24:48.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.019+0000 7fa1c5a00640 1 -- 172.21.15.67:0/3777746366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa1c006c680 con 0x7fa19c004f40 2024-03-09T09:24:48.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.020+0000 7fa1b6a00640 1 -- 172.21.15.67:0/3777746366 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7fa1a807fe40 con 0x7fa19c004f40 2024-03-09T09:24:48.027 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":256,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:24:48.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.022+0000 7fa1c5a00640 1 -- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa1a406c130 msgr2=0x7fa1a406e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.022+0000 7fa1c5a00640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa1a406c130 0x7fa1a406e5f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0006e80 tx=0x7fa1b0008040 comp rx=0 tx=0).stop 2024-03-09T09:24:48.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.022+0000 7fa1c5a00640 1 -- 172.21.15.67:0/3777746366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa19c004f40 msgr2=0x7fa1c00ac390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.022+0000 7fa1c5a00640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa19c004f40 0x7fa1c00ac390 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00c8240 tx=0x7fa1a80046c0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.023+0000 7fa1c5a00640 1 -- 172.21.15.67:0/3777746366 shutdown_connections 2024-03-09T09:24:48.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.023+0000 7fa1c5a00640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa1a406c130 0x7fa1a406e5f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.023+0000 7fa1c5a00640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1c0073b90 0x7fa1c00c1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.023+0000 7fa1c5a00640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa19c004f40 0x7fa1c00ac390 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.023+0000 7fa1c5a00640 1 --2- 172.21.15.67:0/3777746366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa19c004190 0x7fa1c01ea690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.023+0000 7fa1c5a00640 1 -- 172.21.15.67:0/3777746366 >> 172.21.15.67:0/3777746366 conn(0x7fa1c006b040 msgr2=0x7fa1c00728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:48.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.023+0000 7fa1c5a00640 1 -- 172.21.15.67:0/3777746366 shutdown_connections 2024-03-09T09:24:48.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.023+0000 7fa1c5a00640 1 -- 172.21.15.67:0/3777746366 wait complete. 2024-03-09T09:24:48.052 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:24:48.052 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:24:48.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:24:48.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.257+0000 7fd1f0e00640 1 Processor -- start 2024-03-09T09:24:48.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.258+0000 7fd1f0e00640 1 -- start start 2024-03-09T09:24:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.259+0000 7fd1f0e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd1ec072540 0x7fd1ec0c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.259+0000 7fd1f0e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1ec06d250 0x7fd1ec06dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.259+0000 7fd1f0e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec06dd80 con 0x7fd1ec14ff40 2024-03-09T09:24:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.259+0000 7fd1f0e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec153760 con 0x7fd1ec06d250 2024-03-09T09:24:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.259+0000 7fd1f0e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec06d7c0 con 0x7fd1ec072540 2024-03-09T09:24:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.260+0000 7fd1ebe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1ec06d250 0x7fd1ec06dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.260+0000 7fd1ebe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd1ec06d250 0x7fd1ec06dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49166/0 (socket says 172.21.15.67:49166) 2024-03-09T09:24:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.260+0000 7fd1ebe00640 1 -- 172.21.15.67:0/3056215677 learned_addr learned my addr 172.21.15.67:0/3056215677 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:48.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.260+0000 7fd1e3e00640 1 --2- 172.21.15.67:0/3056215677 >> v2:172.21.15.134:3300/0 conn(0x7fd1ec072540 0x7fd1ec0c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.260+0000 7fd1ebe00640 1 -- 172.21.15.67:0/3056215677 >> v2:172.21.15.134:3300/0 conn(0x7fd1ec072540 msgr2=0x7fd1ec0c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.260+0000 7fd1ebe00640 1 --2- 172.21.15.67:0/3056215677 >> v2:172.21.15.134:3300/0 conn(0x7fd1ec072540 0x7fd1ec0c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.260+0000 7fd1ebe00640 1 -- 172.21.15.67:0/3056215677 >> v1:172.21.15.134:6789/0 conn(0x7fd1ec14ff40 legacy=0x7fd1ec153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.261+0000 7fd1ebe00640 1 -- 172.21.15.67:0/3056215677 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec06daa0 con 0x7fd1ec06d250 2024-03-09T09:24:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.262+0000 7fd1ebe00640 1 --2- 172.21.15.67:0/3056215677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec06d250 0x7fd1ec06dfd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0c2ad0 tx=0x7fd1e402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a754db4c4ac200cc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.262+0000 7fd1eaa00640 1 -- 172.21.15.67:0/3056215677 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e402ca80 con 0x7fd1ec06d250 2024-03-09T09:24:48.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.262+0000 7fd1eaa00640 1 -- 172.21.15.67:0/3056215677 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e402cc20 con 0x7fd1ec06d250 2024-03-09T09:24:48.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.262+0000 7fd1eaa00640 1 -- 172.21.15.67:0/3056215677 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40343f0 con 0x7fd1ec06d250 2024-03-09T09:24:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.263+0000 7fd1f0e00640 1 -- 172.21.15.67:0/3056215677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec06d250 msgr2=0x7fd1ec06dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.263+0000 7fd1f0e00640 1 --2- 172.21.15.67:0/3056215677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec06d250 0x7fd1ec06dfd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0c2ad0 tx=0x7fd1e402bee0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.263+0000 7fd1f0e00640 1 -- 172.21.15.67:0/3056215677 shutdown_connections 2024-03-09T09:24:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.263+0000 7fd1f0e00640 1 --2- 172.21.15.67:0/3056215677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec06d250 0x7fd1ec06dfd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.263+0000 7fd1f0e00640 1 --2- 172.21.15.67:0/3056215677 >> v2:172.21.15.134:3300/0 conn(0x7fd1ec072540 0x7fd1ec0c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.263+0000 7fd1f0e00640 1 -- 172.21.15.67:0/3056215677 >> 172.21.15.67:0/3056215677 conn(0x7fd1ec0c8480 msgr2=0x7fd1ec0c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.263+0000 7fd1f0e00640 1 -- 172.21.15.67:0/3056215677 shutdown_connections 2024-03-09T09:24:48.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.263+0000 7fd1f0e00640 1 -- 172.21.15.67:0/3056215677 wait complete. 2024-03-09T09:24:48.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.266+0000 7fd1f0e00640 1 Processor -- start 2024-03-09T09:24:48.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.266+0000 7fd1f0e00640 1 -- start start 2024-03-09T09:24:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1f0e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec06d250 0x7fd1ec0c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1f0e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1ec072540 0x7fd1ec1ebf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1f0e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1ec14ff40 0x7fd1ec1ec480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1f0e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec073760 con 0x7fd1ec06d250 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1f0e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec152b30 con 0x7fd1ec072540 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1f0e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0738e0 con 0x7fd1ec14ff40 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1eb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec06d250 0x7fd1ec0c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1eb400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec06d250 0x7fd1ec0c1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49174/0 (socket says 172.21.15.67:49174) 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1eb400640 1 -- 172.21.15.67:0/584661071 learned_addr learned my addr 172.21.15.67:0/584661071 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1e3e00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1ec072540 0x7fd1ec1ebf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1eb400640 1 -- 172.21.15.67:0/584661071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1ec14ff40 msgr2=0x7fd1ec1ec480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1eb400640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd1ec14ff40 0x7fd1ec1ec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1eb400640 1 -- 172.21.15.67:0/584661071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1ec072540 msgr2=0x7fd1ec1ebf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1eb400640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1ec072540 0x7fd1ec1ebf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1eb400640 1 -- 172.21.15.67:0/584661071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec1ec9c0 con 0x7fd1ec06d250 2024-03-09T09:24:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.268+0000 7fd1e3e00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1ec072540 0x7fd1ec1ebf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:24:48.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.269+0000 7fd1eb400640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec06d250 0x7fd1ec0c1940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c430 tx=0x7fd1dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.269+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0052d0 con 0x7fd1ec06d250 2024-03-09T09:24:48.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.269+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc005490 con 0x7fd1ec06d250 2024-03-09T09:24:48.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.269+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc007260 con 0x7fd1ec06d250 2024-03-09T09:24:48.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.269+0000 7fd1f0e00640 1 -- 172.21.15.67:0/584661071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec13feb0 con 0x7fd1ec06d250 2024-03-09T09:24:48.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.269+0000 7fd1f0e00640 1 -- 172.21.15.67:0/584661071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec151f70 con 0x7fd1ec06d250 2024-03-09T09:24:48.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.270+0000 7fd1f0e00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec1eefb0 0x7fd1ec1ef790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.270+0000 7fd1f0e00640 1 -- 172.21.15.67:0/584661071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1ebc40 con 0x7fd1ec1eefb0 2024-03-09T09:24:48.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.271+0000 7fd1e3e00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec1eefb0 0x7fd1ec1ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.271+0000 7fd1e3e00640 1 -- 172.21.15.67:0/584661071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1ec04f3c0 con 0x7fd1ec1eefb0 2024-03-09T09:24:48.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.271+0000 7fd1e3e00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec1eefb0 0x7fd1ec1ef790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec071af0 tx=0x7fd1d4010c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.272+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd1dc007400 con 0x7fd1ec06d250 2024-03-09T09:24:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.273+0000 7fd1e3400640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd1c806b500 0x7fd1c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.273+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd1dc089ae0 con 0x7fd1ec06d250 2024-03-09T09:24:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.273+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400f040 con 0x7fd1ec1eefb0 2024-03-09T09:24:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.273+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd1d4004220 con 0x7fd1ec1eefb0 2024-03-09T09:24:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.273+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec1eefb0 msgr2=0x7fd1ec1ef790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.273+0000 7fd1e3400640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec1eefb0 0x7fd1ec1ef790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec071af0 tx=0x7fd1d4010c90 comp rx=0 tx=0).stop 2024-03-09T09:24:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.273+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec1eefb0 msgr2=0x7fd1ec1ef790 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:24:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.273+0000 7fd1e3400640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec1eefb0 0x7fd1ec1ef790 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec071af0 tx=0x7fd1d4010c90 comp rx=0 tx=0).stop 2024-03-09T09:24:48.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.273+0000 7fd1ebe00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd1c806b500 0x7fd1c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.275+0000 7fd1ebe00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd1c806b500 0x7fd1c806d9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4035000 tx=0x7fd1e402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.280+0000 7fd1f0e00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec0c7b50 0x7fd1ec13e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.280+0000 7fd1f0e00640 1 -- 172.21.15.67:0/584661071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1ef4b0 con 0x7fd1ec0c7b50 2024-03-09T09:24:48.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.281+0000 7fd1e3e00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec0c7b50 0x7fd1ec13e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.281+0000 7fd1e3e00640 1 -- 172.21.15.67:0/584661071 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd1ec071840 con 0x7fd1ec0c7b50 2024-03-09T09:24:48.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.281+0000 7fd1e3e00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec0c7b50 0x7fd1ec13e920 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009fd0 tx=0x7fd1d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4003ad0 con 0x7fd1ec0c7b50 2024-03-09T09:24:48.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4536 (secure 0 0 0) 0x7fd1d4003ad0 con 0x7fd1ec0c7b50 2024-03-09T09:24:48.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec0c7b50 msgr2=0x7fd1ec13e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1e3400640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec0c7b50 0x7fd1ec13e920 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009fd0 tx=0x7fd1d40046c0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1e3400640 1 -- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec0c7b50 msgr2=0x7fd1ec13e920 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:24:48.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1e3400640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec0c7b50 0x7fd1ec13e920 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009fd0 tx=0x7fd1d40046c0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.290 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:24:48.290 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:24:48.290 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:24:48.290 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:24:48.290 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 256, 2024-03-09T09:24:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:24:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:24:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:24:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:24:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:24:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:24:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:24:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:24:48.291 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:48.292 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:48.293 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:24:48.294 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:24:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:24:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:24:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:24:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:24:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:24:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:24:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:24:48.295 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.296 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:24:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:24:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:24:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:48.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:24:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:24:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.299 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:24:48.299 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:24:48.299 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:48.299 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:48.299 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:48.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.299 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.299 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:24:48.299 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:24:48.299 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:48.299 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:48.300 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.300 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:48.300 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:24:48.300 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.300 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.300 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:48.300 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:24:48.300 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.300 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.300 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:48.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.302 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:48.302 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:24:48.302 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.302 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.302 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:48.302 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:24:48.302 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.302 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:24:48.303 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:24:48.304 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.304 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.304 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.304 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:48.304 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.304 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.304 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:24:48.304 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.304 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.305 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.305 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:24:48.305 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.305 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.305 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:24:48.305 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:24:48.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1f0e00640 1 -- 172.21.15.67:0/584661071 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd1c806b500 msgr2=0x7fd1c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1f0e00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd1c806b500 0x7fd1c806d9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4035000 tx=0x7fd1e402e000 comp rx=0 tx=0).stop 2024-03-09T09:24:48.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1f0e00640 1 -- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec06d250 msgr2=0x7fd1ec0c1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1f0e00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec06d250 0x7fd1ec0c1940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c430 tx=0x7fd1dc00c8f0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1ebe00640 1 -- 172.21.15.67:0/584661071 reap_dead start 2024-03-09T09:24:48.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1f0e00640 1 -- 172.21.15.67:0/584661071 shutdown_connections 2024-03-09T09:24:48.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1f0e00640 1 --2- 172.21.15.67:0/584661071 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1ec06d250 0x7fd1ec0c1940 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.282+0000 7fd1f0e00640 1 -- 172.21.15.67:0/584661071 >> 172.21.15.67:0/584661071 conn(0x7fd1ec0c8480 msgr2=0x7fd1ec072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:48.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.283+0000 7fd1f0e00640 1 -- 172.21.15.67:0/584661071 shutdown_connections 2024-03-09T09:24:48.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.283+0000 7fd1f0e00640 1 -- 172.21.15.67:0/584661071 wait complete. 2024-03-09T09:24:48.308 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:24:48.308 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:24:48.308 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:24:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.498+0000 7f4125400640 1 Processor -- start 2024-03-09T09:24:48.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.498+0000 7f4125400640 1 -- start start 2024-03-09T09:24:48.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.499+0000 7f4125400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4120151360 0x7f41200c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.499+0000 7f4125400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f41200c30a0 0x7f41201539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.499+0000 7f4125400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41200c2780 con 0x7f41200ac930 2024-03-09T09:24:48.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.499+0000 7f4125400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4120070670 con 0x7f41200c30a0 2024-03-09T09:24:48.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.499+0000 7f4125400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41200708c0 con 0x7f4120151360 2024-03-09T09:24:48.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.500+0000 7f4117e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4120151360 0x7f41200c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.500+0000 7f4117e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4120151360 0x7f41200c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:42456/0 (socket says 172.21.15.67:42456) 2024-03-09T09:24:48.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.500+0000 7f4117e00640 1 -- 172.21.15.67:0/991919184 learned_addr learned my addr 172.21.15.67:0/991919184 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:48.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.500+0000 7f4124a00640 1 --2- 172.21.15.67:0/991919184 >> v2:172.21.15.134:3300/0 conn(0x7f41200c30a0 0x7f41201539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.501+0000 7f4117e00640 1 -- 172.21.15.67:0/991919184 >> v2:172.21.15.134:3300/0 conn(0x7f41200c30a0 msgr2=0x7f41201539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:48.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.501+0000 7f4117e00640 1 --2- 172.21.15.67:0/991919184 >> v2:172.21.15.134:3300/0 conn(0x7f41200c30a0 0x7f41201539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.501+0000 7f4117e00640 1 -- 172.21.15.67:0/991919184 >> v1:172.21.15.134:6789/0 conn(0x7f41200ac930 legacy=0x7f4120150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.501+0000 7f4117e00640 1 -- 172.21.15.67:0/991919184 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120070ba0 con 0x7f4120151360 2024-03-09T09:24:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.502+0000 7f4117e00640 1 --2- 172.21.15.67:0/991919184 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4120151360 0x7f41200c3660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4108009870 tx=0x7f410802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc7eaf9bf0b5da2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.502+0000 7f411f400640 1 -- 172.21.15.67:0/991919184 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108034070 con 0x7f4120151360 2024-03-09T09:24:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.503+0000 7f411f400640 1 -- 172.21.15.67:0/991919184 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4108030840 con 0x7f4120151360 2024-03-09T09:24:48.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.503+0000 7f411f400640 1 -- 172.21.15.67:0/991919184 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108030c10 con 0x7f4120151360 2024-03-09T09:24:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.503+0000 7f4125400640 1 -- 172.21.15.67:0/991919184 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4120151360 msgr2=0x7f41200c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.503+0000 7f4125400640 1 --2- 172.21.15.67:0/991919184 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4120151360 0x7f41200c3660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4108009870 tx=0x7f410802b980 comp rx=0 tx=0).stop 2024-03-09T09:24:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.503+0000 7f4125400640 1 -- 172.21.15.67:0/991919184 shutdown_connections 2024-03-09T09:24:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.503+0000 7f4125400640 1 --2- 172.21.15.67:0/991919184 >> v2:172.21.15.134:3300/0 conn(0x7f41200c30a0 0x7f41201539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.503+0000 7f4125400640 1 --2- 172.21.15.67:0/991919184 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4120151360 0x7f41200c3660 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.503+0000 7f4125400640 1 -- 172.21.15.67:0/991919184 >> 172.21.15.67:0/991919184 conn(0x7f41200c7220 msgr2=0x7f41200c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:48.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.503+0000 7f4125400640 1 -- 172.21.15.67:0/991919184 shutdown_connections 2024-03-09T09:24:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.503+0000 7f4125400640 1 -- 172.21.15.67:0/991919184 wait complete. 2024-03-09T09:24:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.505+0000 7f4125400640 1 Processor -- start 2024-03-09T09:24:48.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.505+0000 7f4125400640 1 -- start start 2024-03-09T09:24:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.506+0000 7f4125400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41200ac930 0x7f4120140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.506+0000 7f4125400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41200c30a0 0x7f41201ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.506+0000 7f4125400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4120151360 0x7f41201ec000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.506+0000 7f4125400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200ace90 con 0x7f41200ac930 2024-03-09T09:24:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.506+0000 7f4125400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200acd10 con 0x7f41200c30a0 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.506+0000 7f4125400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4120153f20 con 0x7f4120151360 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.506+0000 7f411fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41200ac930 0x7f4120140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.507+0000 7f411fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41200ac930 0x7f4120140f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49196/0 (socket says 172.21.15.67:49196) 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.507+0000 7f411fe00640 1 -- 172.21.15.67:0/2328135378 learned_addr learned my addr 172.21.15.67:0/2328135378 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.507+0000 7f4117e00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41200c30a0 0x7f41201ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.507+0000 7f4124a00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4120151360 0x7f41201ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.507+0000 7f411fe00640 1 -- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4120151360 msgr2=0x7f41201ec000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.507+0000 7f411fe00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4120151360 0x7f41201ec000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.507+0000 7f411fe00640 1 -- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41200c30a0 msgr2=0x7f41201ebac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.507+0000 7f411fe00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41200c30a0 0x7f41201ebac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.507+0000 7f411fe00640 1 -- 172.21.15.67:0/2328135378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41200c4f70 con 0x7f41200ac930 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.507+0000 7f4117e00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41200c30a0 0x7f41201ebac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:24:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.508+0000 7f411fe00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41200ac930 0x7f4120140f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f411000c430 tx=0x7f411000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.508+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41100052d0 con 0x7f41200ac930 2024-03-09T09:24:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.508+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4110005490 con 0x7f41200ac930 2024-03-09T09:24:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.508+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110007260 con 0x7f41200ac930 2024-03-09T09:24:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.508+0000 7f4125400640 1 -- 172.21.15.67:0/2328135378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41201ec540 con 0x7f41200ac930 2024-03-09T09:24:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.508+0000 7f4125400640 1 -- 172.21.15.67:0/2328135378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41201eefc0 con 0x7f41200ac930 2024-03-09T09:24:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.509+0000 7f4125400640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41201eeb30 0x7f41201ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.509+0000 7f4125400640 1 -- 172.21.15.67:0/2328135378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41201eb7a0 con 0x7f41201eeb30 2024-03-09T09:24:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.510+0000 7f4117e00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41201eeb30 0x7f41201ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.510+0000 7f4117e00640 1 -- 172.21.15.67:0/2328135378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4120153a40 con 0x7f41201eeb30 2024-03-09T09:24:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.511+0000 7f4117e00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41201eeb30 0x7f41201ef2a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f412006edb0 tx=0x7f4108004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.511+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4110007400 con 0x7f41200ac930 2024-03-09T09:24:48.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.512+0000 7f411cc00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40fc06b4d0 0x7f40fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.512+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108034040 con 0x7f41201eeb30 2024-03-09T09:24:48.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.512+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4110089a50 con 0x7f41200ac930 2024-03-09T09:24:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.512+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4108030e20 con 0x7f41201eeb30 2024-03-09T09:24:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.512+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41201eeb30 msgr2=0x7f41201ef2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.513+0000 7f411cc00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41201eeb30 0x7f41201ef2a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f412006edb0 tx=0x7f4108004730 comp rx=0 tx=0).stop 2024-03-09T09:24:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.513+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41201eeb30 msgr2=0x7f41201ef2a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:24:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.513+0000 7f411cc00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41201eeb30 0x7f41201ef2a0 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f412006edb0 tx=0x7f4108004730 comp rx=0 tx=0).stop 2024-03-09T09:24:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.513+0000 7f4124a00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40fc06b4d0 0x7f40fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.514+0000 7f4124a00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40fc06b4d0 0x7f40fc06d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f412014d280 tx=0x7f411800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.520+0000 7f4125400640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4120141460 0x7f412002beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.520+0000 7f4125400640 1 -- 172.21.15.67:0/2328135378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200c06f0 con 0x7f4120141460 2024-03-09T09:24:48.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.520+0000 7f4117e00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4120141460 0x7f412002beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.521+0000 7f4117e00640 1 -- 172.21.15.67:0/2328135378 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4120150ee0 con 0x7f4120141460 2024-03-09T09:24:48.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.521+0000 7f4117e00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4120141460 0x7f412002beb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4108009870 tx=0x7f41080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.522+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108051070 con 0x7f4120141460 2024-03-09T09:24:48.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.522+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7f410804a670 con 0x7f4120141460 2024-03-09T09:24:48.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.522+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4120141460 msgr2=0x7f412002beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.522+0000 7f411cc00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4120141460 0x7f412002beb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4108009870 tx=0x7f41080046c0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.522+0000 7f411cc00640 1 -- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4120141460 msgr2=0x7f412002beb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:24:48.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.522+0000 7f411cc00640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4120141460 0x7f412002beb0 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4108009870 tx=0x7f41080046c0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.530 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:24:48.530 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:24:48.530 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:24:48.530 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:24:48.530 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 256, 2024-03-09T09:24:48.530 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:24:48.531 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:24:48.531 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:24:48.531 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:24:48.531 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.531 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:24:48.531 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:24:48.531 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:24:48.531 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:24:48.531 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:48.531 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:48.531 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:48.532 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:24:48.533 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:24:48.534 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:48.535 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.537 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:24:48.538 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:24:48.538 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:48.538 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:48.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:48.538 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:24:48.538 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.539 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:48.539 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:24:48.539 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.539 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.539 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.539 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:24:48.539 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:48.540 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:24:48.541 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.541 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:48.541 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:24:48.541 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.541 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.541 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.542 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:24:48.542 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:24:48.542 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:48.542 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:48.542 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:48.542 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.542 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.542 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.543 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:24:48.544 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.544 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.544 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:24:48.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.544 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.544 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.544 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:24:48.544 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.544 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.545 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:24:48.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.546 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.546 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.546 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:48.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.546 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.546 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:24:48.547 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:24:48.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.523+0000 7f4125400640 1 -- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40fc06b4d0 msgr2=0x7f40fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.523+0000 7f4125400640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f40fc06b4d0 0x7f40fc06d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f412014d280 tx=0x7f411800b040 comp rx=0 tx=0).stop 2024-03-09T09:24:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.523+0000 7f4125400640 1 -- 172.21.15.67:0/2328135378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41200ac930 msgr2=0x7f4120140f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.523+0000 7f4125400640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41200ac930 0x7f4120140f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f411000c430 tx=0x7f411000c8f0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.523+0000 7f4124a00640 1 -- 172.21.15.67:0/2328135378 reap_dead start 2024-03-09T09:24:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.523+0000 7f4125400640 1 -- 172.21.15.67:0/2328135378 shutdown_connections 2024-03-09T09:24:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.523+0000 7f4125400640 1 --2- 172.21.15.67:0/2328135378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41200ac930 0x7f4120140f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.523+0000 7f4125400640 1 -- 172.21.15.67:0/2328135378 >> 172.21.15.67:0/2328135378 conn(0x7f41200c7220 msgr2=0x7f41200c1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:48.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.523+0000 7f4125400640 1 -- 172.21.15.67:0/2328135378 shutdown_connections 2024-03-09T09:24:48.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.523+0000 7f4125400640 1 -- 172.21.15.67:0/2328135378 wait complete. 2024-03-09T09:24:48.553 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:24:48.553 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:24:48.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:24:48.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.770+0000 7fc88b000640 1 Processor -- start 2024-03-09T09:24:48.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.770+0000 7fc88b000640 1 -- start start 2024-03-09T09:24:48.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.772+0000 7fc88b000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc8840caa20 0x7fc8840cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.772+0000 7fc88b000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88406b040 con 0x7fc8840c8c60 2024-03-09T09:24:48.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.772+0000 7fc88b000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc884156710 con 0x7fc884153460 2024-03-09T09:24:48.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.772+0000 7fc88b000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8840ca590 con 0x7fc8840caa20 2024-03-09T09:24:48.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.772+0000 7fc883400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc8840c8c60 0x7fc8840c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:38984/0 (socket says 172.21.15.67:38984) 2024-03-09T09:24:48.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.772+0000 7fc883400640 1 -- 172.21.15.67:0/1775168880 learned_addr learned my addr 172.21.15.67:0/1775168880 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:24:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc889000640 1 --2- 172.21.15.67:0/1775168880 >> v2:172.21.15.134:3300/0 conn(0x7fc8840caa20 0x7fc8840cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602045928 0 0) 0x7fc86c001490 con 0x7fc8840c8c60 2024-03-09T09:24:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc868003520 con 0x7fc8840c8c60 2024-03-09T09:24:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2747876475 0 0) 0x7fc868003520 con 0x7fc8840c8c60 2024-03-09T09:24:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 >> v2:172.21.15.134:3300/0 conn(0x7fc8840caa20 msgr2=0x7fc8840cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:48.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 --2- 172.21.15.67:0/1775168880 >> v2:172.21.15.134:3300/0 conn(0x7fc8840caa20 0x7fc8840cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 >> v1:172.21.15.134:6789/0 conn(0x7fc884153460 legacy=0x7fc884155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8841569f0 con 0x7fc8840c8c60 2024-03-09T09:24:48.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fc86c001bc0 con 0x7fc8840c8c60 2024-03-09T09:24:48.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 >> v1:172.21.15.67:6789/0 conn(0x7fc8840c8c60 legacy=0x7fc8840c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 --2- 172.21.15.67:0/1775168880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8680039b0 0x7fc868003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 --2- 172.21.15.67:0/1775168880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc868004800 0x7fc868004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 --2- 172.21.15.67:0/1775168880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 0x7fc86800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840ca590 con 0x7fc8680051c0 2024-03-09T09:24:48.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86800c160 con 0x7fc8680039b0 2024-03-09T09:24:48.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.773+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86800c360 con 0x7fc868004800 2024-03-09T09:24:48.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.774+0000 7fc889000640 1 --2- 172.21.15.67:0/1775168880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 0x7fc86800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.774+0000 7fc883e00640 1 --2- 172.21.15.67:0/1775168880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8680039b0 0x7fc868003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.774+0000 7fc883400640 1 --2- 172.21.15.67:0/1775168880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc868004800 0x7fc868004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.774+0000 7fc889000640 1 -- 172.21.15.67:0/1775168880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc868004800 msgr2=0x7fc868004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:48.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.774+0000 7fc889000640 1 --2- 172.21.15.67:0/1775168880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc868004800 0x7fc868004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.774+0000 7fc889000640 1 -- 172.21.15.67:0/1775168880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8680039b0 msgr2=0x7fc868003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:48.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.774+0000 7fc889000640 1 --2- 172.21.15.67:0/1775168880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8680039b0 0x7fc868003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.774+0000 7fc889000640 1 -- 172.21.15.67:0/1775168880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868003520 con 0x7fc8680051c0 2024-03-09T09:24:48.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.776+0000 7fc889000640 1 -- 172.21.15.67:0/1775168880 reap_dead start 2024-03-09T09:24:48.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.776+0000 7fc889000640 1 --2- 172.21.15.67:0/1775168880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 0x7fc86800b9c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009ac0 tx=0x7fc868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beee74fc36cdd787 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.776+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c009690 con 0x7fc8680051c0 2024-03-09T09:24:48.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.776+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c004020 con 0x7fc8680051c0 2024-03-09T09:24:48.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.776+0000 7fc882a00640 1 -- 172.21.15.67:0/1775168880 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c004410 con 0x7fc8680051c0 2024-03-09T09:24:48.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.777+0000 7fc88b000640 1 -- 172.21.15.67:0/1775168880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 msgr2=0x7fc86800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.777+0000 7fc88b000640 1 --2- 172.21.15.67:0/1775168880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 0x7fc86800b9c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009ac0 tx=0x7fc868005830 comp rx=0 tx=0).stop 2024-03-09T09:24:48.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.777+0000 7fc88b000640 1 -- 172.21.15.67:0/1775168880 shutdown_connections 2024-03-09T09:24:48.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.777+0000 7fc88b000640 1 --2- 172.21.15.67:0/1775168880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 0x7fc86800b9c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.777+0000 7fc88b000640 1 -- 172.21.15.67:0/1775168880 >> 172.21.15.67:0/1775168880 conn(0x7fc8840c8170 msgr2=0x7fc8840c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:48.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.777+0000 7fc88b000640 1 -- 172.21.15.67:0/1775168880 shutdown_connections 2024-03-09T09:24:48.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.777+0000 7fc88b000640 1 -- 172.21.15.67:0/1775168880 wait complete. 2024-03-09T09:24:48.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.779+0000 7fc88b000640 1 Processor -- start 2024-03-09T09:24:48.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.779+0000 7fc88b000640 1 -- start start 2024-03-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc88b000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 0x7fc8840ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc88b000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc88415a160 0x7fc8840c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc88b000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8840ac610 0x7fc8840c3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc88b000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc884155ec0 con 0x7fc8680051c0 2024-03-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc88b000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc884153840 con 0x7fc8840ac610 2024-03-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc88b000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc884156040 con 0x7fc88415a160 2024-03-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc883e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 0x7fc8840ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc883e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 0x7fc8840ac0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49224/0 (socket says 172.21.15.67:49224) 2024-03-09T09:24:48.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc883e00640 1 -- 172.21.15.67:0/2172793103 learned_addr learned my addr 172.21.15.67:0/2172793103 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc883400640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc88415a160 0x7fc8840c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc889000640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8840ac610 0x7fc8840c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc883e00640 1 -- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc88415a160 msgr2=0x7fc8840c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc883e00640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc88415a160 0x7fc8840c2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc883e00640 1 -- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8840ac610 msgr2=0x7fc8840c3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc883e00640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8840ac610 0x7fc8840c3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc883e00640 1 -- 172.21.15.67:0/2172793103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8841ec580 con 0x7fc8680051c0 2024-03-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.781+0000 7fc883400640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc88415a160 0x7fc8840c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.782+0000 7fc883e00640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 0x7fc8840ac0d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc87400f620 tx=0x7fc87400fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.782+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8740052e0 con 0x7fc8680051c0 2024-03-09T09:24:48.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.782+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc874018070 con 0x7fc8680051c0 2024-03-09T09:24:48.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.782+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874003810 con 0x7fc8680051c0 2024-03-09T09:24:48.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.782+0000 7fc88b000640 1 -- 172.21.15.67:0/2172793103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8841502c0 con 0x7fc8680051c0 2024-03-09T09:24:48.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.782+0000 7fc88b000640 1 -- 172.21.15.67:0/2172793103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8841ec840 con 0x7fc8680051c0 2024-03-09T09:24:48.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.783+0000 7fc88b000640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc884075a00 0x7fc884148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.783+0000 7fc88b000640 1 -- 172.21.15.67:0/2172793103 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840c41d0 con 0x7fc884075a00 2024-03-09T09:24:48.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.784+0000 7fc883400640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc884075a00 0x7fc884148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.784+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc8740039b0 con 0x7fc8680051c0 2024-03-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.785+0000 7fc883400640 1 -- 172.21.15.67:0/2172793103 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc884159d60 con 0x7fc884075a00 2024-03-09T09:24:48.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.785+0000 7fc87be00640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc86006b500 0x7fc86006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.785+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fc87408a330 con 0x7fc8680051c0 2024-03-09T09:24:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.785+0000 7fc883400640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc884075a00 0x7fc884148480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc8840acc20 tx=0x7fc86c00f540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.786+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c010040 con 0x7fc884075a00 2024-03-09T09:24:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.786+0000 7fc889000640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc86006b500 0x7fc86006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.786+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fc86c019070 con 0x7fc884075a00 2024-03-09T09:24:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.786+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc884075a00 msgr2=0x7fc884148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.786+0000 7fc87be00640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc884075a00 0x7fc884148480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc8840acc20 tx=0x7fc86c00f540 comp rx=0 tx=0).stop 2024-03-09T09:24:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.786+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc884075a00 msgr2=0x7fc884148480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:24:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.786+0000 7fc87be00640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc884075a00 0x7fc884148480 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc8840acc20 tx=0x7fc86c00f540 comp rx=0 tx=0).stop 2024-03-09T09:24:48.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.787+0000 7fc889000640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc86006b500 0x7fc86006d9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc86800c130 tx=0x7fc87c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.794+0000 7fc88b000640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8840c95f0 0x7fc8840c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:48.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.794+0000 7fc88b000640 1 -- 172.21.15.67:0/2172793103 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841eca10 con 0x7fc8840c95f0 2024-03-09T09:24:48.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.795+0000 7fc883400640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8840c95f0 0x7fc8840c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.795+0000 7fc883400640 1 -- 172.21.15.67:0/2172793103 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc884158cb0 con 0x7fc8840c95f0 2024-03-09T09:24:48.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.796+0000 7fc883400640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8840c95f0 0x7fc8840c99d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc8840acc20 tx=0x7fc8841568e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:48.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.796+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c014b90 con 0x7fc8840c95f0 2024-03-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.797+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7fc86c014b90 con 0x7fc8840c95f0 2024-03-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.797+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8840c95f0 msgr2=0x7fc8840c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.797+0000 7fc87be00640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8840c95f0 0x7fc8840c99d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc8840acc20 tx=0x7fc8841568e0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.797+0000 7fc87be00640 1 -- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8840c95f0 msgr2=0x7fc8840c99d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:24:48.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.797+0000 7fc87be00640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc8840c95f0 0x7fc8840c99d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.805 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:24:48.805 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:24:48.805 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:24:48.805 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:24:48.805 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 256, 2024-03-09T09:24:48.805 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:24:48.805 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:24:48.805 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:24:48.805 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:48.806 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:48.807 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:48.808 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:48.808 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:48.808 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:48.808 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:48.808 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:48.808 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:48.808 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.808 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:24:48.808 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:24:48.808 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:24:48.809 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:24:48.810 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:24:48.810 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:24:48.810 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:24:48.810 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:24:48.810 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:24:48.810 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:24:48.810 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:24:48.810 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:24:48.810 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:24:48.810 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:48.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.812 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:24:48.813 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:24:48.813 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:48.813 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:48.813 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:48.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.813 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:24:48.814 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.815 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.815 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.815 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:24:48.815 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:24:48.815 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:48.815 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:48.815 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.817 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:24:48.817 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:24:48.817 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:24:48.817 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.817 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.817 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:24:48.817 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:24:48.817 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:24:48.817 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:24:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:24:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:24:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:24:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.818 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:24:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:24:48.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:24:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:24:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:24:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.820 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:24:48.821 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.821 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:48.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.821 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.821 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.821 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:48.821 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.821 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:24:48.821 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:24:48.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:24:48.822 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:24:48.822 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:24:48.822 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:24:48.822 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:24:48.822 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:24:48.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:24:48.822 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:24:48.822 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:24:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.798+0000 7fc88b000640 1 -- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc86006b500 msgr2=0x7fc86006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.798+0000 7fc88b000640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc86006b500 0x7fc86006d9c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc86800c130 tx=0x7fc87c030000 comp rx=0 tx=0).stop 2024-03-09T09:24:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.798+0000 7fc88b000640 1 -- 172.21.15.67:0/2172793103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 msgr2=0x7fc8840ac0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.798+0000 7fc88b000640 1 --2- 172.21.15.67:0/2172793103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8680051c0 0x7fc8840ac0d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc87400f620 tx=0x7fc87400fae0 comp rx=0 tx=0).stop 2024-03-09T09:24:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.798+0000 7fc889000640 1 -- 172.21.15.67:0/2172793103 reap_dead start 2024-03-09T09:24:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.798+0000 7fc889000640 1 -- 172.21.15.67:0/2172793103 reap_dead start 2024-03-09T09:24:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.798+0000 7fc88b000640 1 -- 172.21.15.67:0/2172793103 shutdown_connections 2024-03-09T09:24:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.798+0000 7fc88b000640 1 -- 172.21.15.67:0/2172793103 >> 172.21.15.67:0/2172793103 conn(0x7fc8840c8170 msgr2=0x7fc884074540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.798+0000 7fc88b000640 1 -- 172.21.15.67:0/2172793103 shutdown_connections 2024-03-09T09:24:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:48.798+0000 7fc88b000640 1 -- 172.21.15.67:0/2172793103 wait complete. 2024-03-09T09:24:48.825 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:24:48.825 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:24:48.825 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:24:49.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.017+0000 7f7623800640 1 Processor -- start 2024-03-09T09:24:49.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.017+0000 7f7623800640 1 -- start start 2024-03-09T09:24:49.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.019+0000 7f7623800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c06e8d0 0x7f761c06f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.019+0000 7f7623800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f761c1593f0 0x7f761c159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.019+0000 7f7623800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f761c155e00 con 0x7f761c150080 2024-03-09T09:24:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.019+0000 7f7623800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f761c1560c0 con 0x7f761c1593f0 2024-03-09T09:24:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.019+0000 7f7623800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761c1563a0 con 0x7f761c06e8d0 2024-03-09T09:24:49.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.019+0000 7f7620e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f761c150080 0x7f761c0c83d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56174/0 (socket says 172.21.15.67:56174) 2024-03-09T09:24:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.019+0000 7f7620e00640 1 -- 172.21.15.67:0/2547933544 learned_addr learned my addr 172.21.15.67:0/2547933544 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:24:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761be00640 1 --2- 172.21.15.67:0/2547933544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c06e8d0 0x7f761c06f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195974631 0 0) 0x7f761c155e00 con 0x7f761c150080 2024-03-09T09:24:49.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7604003520 con 0x7f761c150080 2024-03-09T09:24:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2948662355 0 0) 0x7f7604003520 con 0x7f761c150080 2024-03-09T09:24:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c06e8d0 msgr2=0x7f761c06f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761b400640 1 --2- 172.21.15.67:0/2547933544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c06e8d0 0x7f761c06f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 >> v2:172.21.15.134:3300/0 conn(0x7f761c1593f0 msgr2=0x7f761c159880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T09:24:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761b400640 1 --2- 172.21.15.67:0/2547933544 >> v2:172.21.15.134:3300/0 conn(0x7f761c1593f0 0x7f761c159880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c156680 con 0x7f761c150080 2024-03-09T09:24:49.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f7610001670 con 0x7f761c150080 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 >> v1:172.21.15.67:6789/0 conn(0x7f761c150080 legacy=0x7f761c0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.020+0000 7f761b400640 1 --2- 172.21.15.67:0/2547933544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c06e8d0 0x7f76040039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.021+0000 7f761b400640 1 --2- 172.21.15.67:0/2547933544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7604003f80 0x7f7604004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.021+0000 7f761b400640 1 --2- 172.21.15.67:0/2547933544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7604004d40 0x7f76040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.021+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c1560c0 con 0x7f7604004d40 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.021+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c1563a0 con 0x7f7604003f80 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.021+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760400bc80 con 0x7f761c06e8d0 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.021+0000 7f7621800640 1 --2- 172.21.15.67:0/2547933544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7604004d40 0x7f76040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.021+0000 7f7621800640 1 -- 172.21.15.67:0/2547933544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c06e8d0 msgr2=0x7f76040039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.021+0000 7f7621800640 1 --2- 172.21.15.67:0/2547933544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f761c06e8d0 0x7f76040039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.021+0000 7f7621800640 1 -- 172.21.15.67:0/2547933544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7604003f80 msgr2=0x7f7604004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.021+0000 7f7621800640 1 --2- 172.21.15.67:0/2547933544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7604003f80 0x7f7604004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:49.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.021+0000 7f7621800640 1 -- 172.21.15.67:0/2547933544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7604003520 con 0x7f7604004d40 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.022+0000 7f7621800640 1 --2- 172.21.15.67:0/2547933544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7604004d40 0x7f76040051c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7614009e60 tx=0x7f7604005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a99bde1dc9be644 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.023+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614039070 con 0x7f7604004d40 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.023+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761402cda0 con 0x7f7604004d40 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.023+0000 7f761b400640 1 -- 172.21.15.67:0/2547933544 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614009a30 con 0x7f7604004d40 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.023+0000 7f7623800640 1 -- 172.21.15.67:0/2547933544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7604004d40 msgr2=0x7f76040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.023+0000 7f7623800640 1 --2- 172.21.15.67:0/2547933544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7604004d40 0x7f76040051c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7614009e60 tx=0x7f7604005830 comp rx=0 tx=0).stop 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.023+0000 7f7621800640 1 -- 172.21.15.67:0/2547933544 reap_dead start 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.024+0000 7f7623800640 1 -- 172.21.15.67:0/2547933544 shutdown_connections 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.024+0000 7f7623800640 1 -- 172.21.15.67:0/2547933544 >> 172.21.15.67:0/2547933544 conn(0x7f761c0aa890 msgr2=0x7f761c0ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.024+0000 7f7623800640 1 -- 172.21.15.67:0/2547933544 shutdown_connections 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.024+0000 7f7623800640 1 -- 172.21.15.67:0/2547933544 wait complete. 2024-03-09T09:24:49.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.025+0000 7f7623800640 1 Processor -- start 2024-03-09T09:24:49.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.026+0000 7f7623800640 1 -- start start 2024-03-09T09:24:49.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.026+0000 7f7623800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7604004d40 0x7f761c06c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:49.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.026+0000 7f7623800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f761c0acb30 0x7f761c147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:49.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.026+0000 7f7623800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f761c1480d0 0x7f761c1ebc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.026+0000 7f7623800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c0c8ae0 con 0x7f761c1480d0 2024-03-09T09:24:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.026+0000 7f7623800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c150460 con 0x7f761c0acb30 2024-03-09T09:24:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.026+0000 7f7623800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761c0c8c60 con 0x7f7604004d40 2024-03-09T09:24:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.027+0000 7f7621800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f761c1480d0 0x7f761c1ebc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.027+0000 7f7621800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f761c1480d0 0x7f761c1ebc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51084/0 (socket says 172.21.15.67:51084) 2024-03-09T09:24:49.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.027+0000 7f7621800640 1 -- 172.21.15.67:0/1958550049 learned_addr learned my addr 172.21.15.67:0/1958550049 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.027+0000 7f7620e00640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7604004d40 0x7f761c06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.027+0000 7f761be00640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f761c0acb30 0x7f761c147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.027+0000 7f7621800640 1 -- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7604004d40 msgr2=0x7f761c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.027+0000 7f7621800640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7604004d40 0x7f761c06c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.027+0000 7f7621800640 1 -- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f761c0acb30 msgr2=0x7f761c147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.027+0000 7f7621800640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f761c0acb30 0x7f761c147b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.027+0000 7f7621800640 1 -- 172.21.15.67:0/1958550049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c1ef7c0 con 0x7f761c1480d0 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.027+0000 7f7621800640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f761c1480d0 0x7f761c1ebc20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7614009f80 tx=0x7f7614004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.028+0000 7f7618c00640 1 -- 172.21.15.67:0/1958550049 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614039040 con 0x7f761c1480d0 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.028+0000 7f7618c00640 1 -- 172.21.15.67:0/1958550049 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761403f470 con 0x7f761c1480d0 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.028+0000 7f7618c00640 1 -- 172.21.15.67:0/1958550049 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76140092f0 con 0x7f761c1480d0 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.028+0000 7f7623800640 1 -- 172.21.15.67:0/1958550049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761c1efaa0 con 0x7f761c1480d0 2024-03-09T09:24:49.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.028+0000 7f7623800640 1 -- 172.21.15.67:0/1958550049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761c155a90 con 0x7f761c1480d0 2024-03-09T09:24:49.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.029+0000 7f7623800640 1 -- 172.21.15.67:0/1958550049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761c156960 con 0x7f761c1480d0 2024-03-09T09:24:49.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.030+0000 7f7618c00640 1 -- 172.21.15.67:0/1958550049 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f761403f860 con 0x7f761c1480d0 2024-03-09T09:24:49.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.031+0000 7f7618c00640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f75fc06b500 0x7f75fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:49.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.031+0000 7f7618c00640 1 -- 172.21.15.67:0/1958550049 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f76140ad370 con 0x7f761c1480d0 2024-03-09T09:24:49.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.032+0000 7f7620e00640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f75fc06b500 0x7f75fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:49.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.033+0000 7f7620e00640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f75fc06b500 0x7f75fc06d9c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f761c06cc30 tx=0x7f7610009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:49.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.041+0000 7f7618c00640 1 -- 172.21.15.67:0/1958550049 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f761c156960 con 0x7f761c1480d0 2024-03-09T09:24:49.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.409+0000 7f7623800640 1 -- 172.21.15.67:0/1958550049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f761c075a00 con 0x7f761c1480d0 2024-03-09T09:24:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.410+0000 7f7618c00640 1 -- 172.21.15.67:0/1958550049 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f7614008030 con 0x7f761c1480d0 2024-03-09T09:24:49.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.414+0000 7f7623800640 1 -- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f75fc06b500 msgr2=0x7f75fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:49.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.414+0000 7f7623800640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f75fc06b500 0x7f75fc06d9c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f761c06cc30 tx=0x7f7610009040 comp rx=0 tx=0).stop 2024-03-09T09:24:49.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.414+0000 7f7623800640 1 -- 172.21.15.67:0/1958550049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f761c1480d0 msgr2=0x7f761c1ebc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:49.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.414+0000 7f7623800640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f761c1480d0 0x7f761c1ebc20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7614009f80 tx=0x7f7614004a80 comp rx=0 tx=0).stop 2024-03-09T09:24:49.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.414+0000 7f7623800640 1 -- 172.21.15.67:0/1958550049 shutdown_connections 2024-03-09T09:24:49.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.414+0000 7f7623800640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f75fc06b500 0x7f75fc06d9c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:49.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.414+0000 7f7623800640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f761c1480d0 0x7f761c1ebc20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.414+0000 7f7623800640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f761c0acb30 0x7f761c147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.414+0000 7f7623800640 1 --2- 172.21.15.67:0/1958550049 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7604004d40 0x7f761c06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.414+0000 7f7623800640 1 -- 172.21.15.67:0/1958550049 >> 172.21.15.67:0/1958550049 conn(0x7f761c0aa890 msgr2=0x7f761c0c3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.415+0000 7f7623800640 1 -- 172.21.15.67:0/1958550049 shutdown_connections 2024-03-09T09:24:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:49.415+0000 7f7623800640 1 -- 172.21.15.67:0/1958550049 wait complete. 2024-03-09T09:24:49.445 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:24:50.014 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:24:50.014 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:24:50.014 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 83 -eq 48 ']' 2024-03-09T09:24:50.014 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.83 2024-03-09T09:24:50.014 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.83 2024-03-09T09:24:50.014 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.83 2024-03-09T09:24:50.015 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.83 2024-03-09T09:24:50.018 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:24:52.462 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:24:52.462 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:52.462 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 90 -eq 48 ']' 2024-03-09T09:24:52.462 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.90 2024-03-09T09:24:52.463 INFO:tasks.workunit.client.0.smithi067.stdout:checking v0.90 2024-03-09T09:24:52.463 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo checking v0.90 2024-03-09T09:24:52.463 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd .snap/v0.90 2024-03-09T09:24:52.467 INFO:tasks.workunit.client.0.smithi067.stderr:+ git diff --exit-code 2024-03-09T09:24:55.644 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:24:55.659 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:24:55.660 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 84 -eq 48 ']' 2024-03-09T09:24:55.660 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.84 2024-03-09T09:24:55.660 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.84 2024-03-09T09:24:55.660 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.84 2024-03-09T09:24:55.661 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.84 2024-03-09T09:24:55.661 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:24:58.524 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd ../.. 2024-03-09T09:24:58.525 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:58.525 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 48 ']' 2024-03-09T09:24:58.525 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.1 2024-03-09T09:24:58.526 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.1 2024-03-09T09:24:58.574 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:58.574 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 2 -eq 48 ']' 2024-03-09T09:24:58.574 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.2 2024-03-09T09:24:58.574 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.2 2024-03-09T09:24:58.610 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:58.610 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 3 -eq 48 ']' 2024-03-09T09:24:58.610 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.3 2024-03-09T09:24:58.610 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.3 2024-03-09T09:24:58.652 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:58.653 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 4 -eq 48 ']' 2024-03-09T09:24:58.653 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.4 2024-03-09T09:24:58.653 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.4 2024-03-09T09:24:58.688 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:58.688 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 5 -eq 48 ']' 2024-03-09T09:24:58.688 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.5 2024-03-09T09:24:58.688 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.5 2024-03-09T09:24:58.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:58.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 6 -eq 48 ']' 2024-03-09T09:24:58.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.6 2024-03-09T09:24:58.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.6 2024-03-09T09:24:58.750 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:58.750 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 7 -eq 48 ']' 2024-03-09T09:24:58.750 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.7 2024-03-09T09:24:58.750 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.7 2024-03-09T09:24:58.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:58.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 8 -eq 48 ']' 2024-03-09T09:24:58.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.8 2024-03-09T09:24:58.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.8 2024-03-09T09:24:59.446 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:24:59.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:24:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.674+0000 7faaa0000640 1 Processor -- start 2024-03-09T09:24:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.675+0000 7faaa0000640 1 -- start start 2024-03-09T09:24:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.676+0000 7faaa0000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa98151510 0x7faa980729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.676+0000 7faaa0000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faa9806ffd0 0x7faa98157920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.676+0000 7faaa0000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa98153ff0 con 0x7faa980c4330 2024-03-09T09:24:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.676+0000 7faaa0000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa981542d0 con 0x7faa9806ffd0 2024-03-09T09:24:59.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.676+0000 7faaa0000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa981545b0 con 0x7faa98151510 2024-03-09T09:24:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.677+0000 7faa9e000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faa9806ffd0 0x7faa98157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.677+0000 7faa9d600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa98151510 0x7faa980729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.677+0000 7faa9e000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faa9806ffd0 0x7faa98157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:45804/0 (socket says 172.21.15.67:45804) 2024-03-09T09:24:59.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.677+0000 7faa9e000640 1 -- 172.21.15.67:0/4223000122 learned_addr learned my addr 172.21.15.67:0/4223000122 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:59.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.677+0000 7faa9e000640 1 -- 172.21.15.67:0/4223000122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa98151510 msgr2=0x7faa980729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:24:59.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.677+0000 7faa9e000640 1 --2- 172.21.15.67:0/4223000122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa98151510 0x7faa980729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.677+0000 7faa9e000640 1 -- 172.21.15.67:0/4223000122 >> v1:172.21.15.134:6789/0 conn(0x7faa980c4330 legacy=0x7faa98157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.677+0000 7faa9e000640 1 -- 172.21.15.67:0/4223000122 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa98154890 con 0x7faa9806ffd0 2024-03-09T09:24:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.679+0000 7faa9e000640 1 --2- 172.21.15.67:0/4223000122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa9806ffd0 0x7faa98157920 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faa90009e50 tx=0x7faa9002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81047a6595efa1b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.680+0000 7faa97e00640 1 -- 172.21.15.67:0/4223000122 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9002cb00 con 0x7faa9806ffd0 2024-03-09T09:24:59.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.680+0000 7faa97e00640 1 -- 172.21.15.67:0/4223000122 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9002cca0 con 0x7faa9806ffd0 2024-03-09T09:24:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.680+0000 7faa97e00640 1 -- 172.21.15.67:0/4223000122 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa90034610 con 0x7faa9806ffd0 2024-03-09T09:24:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.681+0000 7faaa0000640 1 -- 172.21.15.67:0/4223000122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa9806ffd0 msgr2=0x7faa98157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.681+0000 7faaa0000640 1 --2- 172.21.15.67:0/4223000122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa9806ffd0 0x7faa98157920 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faa90009e50 tx=0x7faa9002bf60 comp rx=0 tx=0).stop 2024-03-09T09:24:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.681+0000 7faaa0000640 1 -- 172.21.15.67:0/4223000122 shutdown_connections 2024-03-09T09:24:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.681+0000 7faaa0000640 1 --2- 172.21.15.67:0/4223000122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa9806ffd0 0x7faa98157920 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.681+0000 7faaa0000640 1 --2- 172.21.15.67:0/4223000122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa98151510 0x7faa980729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.681+0000 7faaa0000640 1 -- 172.21.15.67:0/4223000122 >> 172.21.15.67:0/4223000122 conn(0x7faa980ab320 msgr2=0x7faa980ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:24:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.681+0000 7faaa0000640 1 -- 172.21.15.67:0/4223000122 shutdown_connections 2024-03-09T09:24:59.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.681+0000 7faaa0000640 1 -- 172.21.15.67:0/4223000122 wait complete. 2024-03-09T09:24:59.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.684+0000 7faaa0000640 1 Processor -- start 2024-03-09T09:24:59.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.684+0000 7faaa0000640 1 -- start start 2024-03-09T09:24:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.685+0000 7faaa0000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa9806ffd0 0x7faa980aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.685+0000 7faaa0000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa980c4330 0x7faa9815ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.685+0000 7faaa0000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa980aa8c0 0x7faa9815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.685+0000 7faaa0000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa980c3f70 con 0x7faa9806ffd0 2024-03-09T09:24:59.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.685+0000 7faaa0000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa98153ca0 con 0x7faa980c4330 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.685+0000 7faaa0000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa980c40f0 con 0x7faa980aa8c0 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.686+0000 7faa9d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa9806ffd0 0x7faa980aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.686+0000 7faa9d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa9806ffd0 0x7faa980aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37904/0 (socket says 172.21.15.67:37904) 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.686+0000 7faa9d600640 1 -- 172.21.15.67:0/1650624760 learned_addr learned my addr 172.21.15.67:0/1650624760 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.686+0000 7faa9cc00640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa980c4330 0x7faa9815ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.686+0000 7faa9e000640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa980aa8c0 0x7faa9815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.686+0000 7faa9d600640 1 -- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa980aa8c0 msgr2=0x7faa9815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.686+0000 7faa9d600640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa980aa8c0 0x7faa9815ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.686+0000 7faa9d600640 1 -- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa980c4330 msgr2=0x7faa9815ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.686+0000 7faa9d600640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa980c4330 0x7faa9815ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.686+0000 7faa9d600640 1 -- 172.21.15.67:0/1650624760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa981e9620 con 0x7faa9806ffd0 2024-03-09T09:24:59.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.686+0000 7faa9e000640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa980aa8c0 0x7faa9815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:24:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.687+0000 7faa9d600640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa9806ffd0 0x7faa980aa380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00e870 tx=0x7faa8c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.687+0000 7faa95600640 1 -- 172.21.15.67:0/1650624760 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c017070 con 0x7faa9806ffd0 2024-03-09T09:24:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.687+0000 7faa95600640 1 -- 172.21.15.67:0/1650624760 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8c00c8e0 con 0x7faa9806ffd0 2024-03-09T09:24:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.687+0000 7faa95600640 1 -- 172.21.15.67:0/1650624760 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00ccd0 con 0x7faa9806ffd0 2024-03-09T09:24:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.687+0000 7faaa0000640 1 -- 172.21.15.67:0/1650624760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa981ebd40 con 0x7faa9806ffd0 2024-03-09T09:24:59.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.687+0000 7faaa0000640 1 -- 172.21.15.67:0/1650624760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa981ec300 con 0x7faa9806ffd0 2024-03-09T09:24:59.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.689+0000 7faaa0000640 1 -- 172.21.15.67:0/1650624760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa98075a00 con 0x7faa9806ffd0 2024-03-09T09:24:59.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.690+0000 7faa95600640 1 -- 172.21.15.67:0/1650624760 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faa8c01a070 con 0x7faa9806ffd0 2024-03-09T09:24:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.691+0000 7faa95600640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa7806b4d0 0x7faa7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:24:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.692+0000 7faa95600640 1 -- 172.21.15.67:0/1650624760 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7faa8c08a840 con 0x7faa9806ffd0 2024-03-09T09:24:59.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.692+0000 7faa9cc00640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa7806b4d0 0x7faa7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:24:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.693+0000 7faa9cc00640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa7806b4d0 0x7faa7806d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7faa980c3eb0 tx=0x7faa84009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:24:59.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:24:59.698+0000 7faa95600640 1 -- 172.21.15.67:0/1650624760 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faa8c059240 con 0x7faa9806ffd0 2024-03-09T09:24:59.723 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:59.723 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 9 -eq 48 ']' 2024-03-09T09:24:59.723 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.9 2024-03-09T09:24:59.724 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.9 2024-03-09T09:24:59.763 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:59.763 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 10 -eq 48 ']' 2024-03-09T09:24:59.763 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.10 2024-03-09T09:24:59.764 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.10 2024-03-09T09:24:59.799 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:59.799 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 11 -eq 48 ']' 2024-03-09T09:24:59.799 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.11 2024-03-09T09:24:59.799 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.11 2024-03-09T09:24:59.828 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:59.828 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 12 -eq 48 ']' 2024-03-09T09:24:59.828 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.12 2024-03-09T09:24:59.828 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.12 2024-03-09T09:24:59.859 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:59.859 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 13 -eq 48 ']' 2024-03-09T09:24:59.859 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.13 2024-03-09T09:24:59.860 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.13 2024-03-09T09:24:59.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:59.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 14 -eq 48 ']' 2024-03-09T09:24:59.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.14 2024-03-09T09:24:59.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.14 2024-03-09T09:24:59.922 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:59.922 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 15 -eq 48 ']' 2024-03-09T09:24:59.922 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.15 2024-03-09T09:24:59.922 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.15 2024-03-09T09:24:59.953 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:59.953 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 16 -eq 48 ']' 2024-03-09T09:24:59.953 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.16 2024-03-09T09:24:59.953 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.16 2024-03-09T09:24:59.989 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:24:59.990 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 17 -eq 48 ']' 2024-03-09T09:24:59.990 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.17 2024-03-09T09:24:59.990 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.17 2024-03-09T09:25:00.020 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.020 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 18 -eq 48 ']' 2024-03-09T09:25:00.020 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.18 2024-03-09T09:25:00.020 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.18 2024-03-09T09:25:00.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.020+0000 7faaa0000640 1 -- 172.21.15.67:0/1650624760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7faa98070b10 con 0x7faa9806ffd0 2024-03-09T09:25:00.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.021+0000 7faa95600640 1 -- 172.21.15.67:0/1650624760 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7faa8c058b40 con 0x7faa9806ffd0 2024-03-09T09:25:00.028 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":256,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:25:00.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.023+0000 7faaa0000640 1 -- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa7806b4d0 msgr2=0x7faa7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.023+0000 7faaa0000640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa7806b4d0 0x7faa7806d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7faa980c3eb0 tx=0x7faa84009040 comp rx=0 tx=0).stop 2024-03-09T09:25:00.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.023+0000 7faaa0000640 1 -- 172.21.15.67:0/1650624760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa9806ffd0 msgr2=0x7faa980aa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.023+0000 7faaa0000640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa9806ffd0 0x7faa980aa380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00e870 tx=0x7faa8c00ed30 comp rx=0 tx=0).stop 2024-03-09T09:25:00.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.023+0000 7faaa0000640 1 -- 172.21.15.67:0/1650624760 shutdown_connections 2024-03-09T09:25:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.023+0000 7faaa0000640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faa7806b4d0 0x7faa7806d990 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.023+0000 7faaa0000640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa980aa8c0 0x7faa9815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.023+0000 7faaa0000640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa980c4330 0x7faa9815ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.023+0000 7faaa0000640 1 --2- 172.21.15.67:0/1650624760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa9806ffd0 0x7faa980aa380 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.023+0000 7faaa0000640 1 -- 172.21.15.67:0/1650624760 >> 172.21.15.67:0/1650624760 conn(0x7faa980ab320 msgr2=0x7faa98152300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:00.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.024+0000 7faaa0000640 1 -- 172.21.15.67:0/1650624760 shutdown_connections 2024-03-09T09:25:00.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.024+0000 7faaa0000640 1 -- 172.21.15.67:0/1650624760 wait complete. 2024-03-09T09:25:00.046 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.046 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 19 -eq 48 ']' 2024-03-09T09:25:00.046 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.19 2024-03-09T09:25:00.046 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.19 2024-03-09T09:25:00.050 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:25:00.050 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:25:00.050 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:25:00.050 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:25:00.072 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.073 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 20 -eq 48 ']' 2024-03-09T09:25:00.073 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.20 2024-03-09T09:25:00.073 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.20 2024-03-09T09:25:00.100 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.100 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 21 -eq 48 ']' 2024-03-09T09:25:00.100 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.21 2024-03-09T09:25:00.100 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.21 2024-03-09T09:25:00.127 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.127 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 22 -eq 48 ']' 2024-03-09T09:25:00.127 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.22 2024-03-09T09:25:00.127 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.22 2024-03-09T09:25:00.154 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.154 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 23 -eq 48 ']' 2024-03-09T09:25:00.155 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.23 2024-03-09T09:25:00.155 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.23 2024-03-09T09:25:00.188 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.188 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 24 -eq 48 ']' 2024-03-09T09:25:00.188 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.24 2024-03-09T09:25:00.189 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.24 2024-03-09T09:25:00.214 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.214 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 25 -eq 48 ']' 2024-03-09T09:25:00.214 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.25 2024-03-09T09:25:00.214 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.25 2024-03-09T09:25:00.239 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.240 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 26 -eq 48 ']' 2024-03-09T09:25:00.240 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.26 2024-03-09T09:25:00.240 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.26 2024-03-09T09:25:00.269 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.269 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 27 -eq 48 ']' 2024-03-09T09:25:00.269 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.27 2024-03-09T09:25:00.269 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.27 2024-03-09T09:25:00.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.286+0000 7f545b000640 1 Processor -- start 2024-03-09T09:25:00.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.287+0000 7f545b000640 1 -- start start 2024-03-09T09:25:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.288+0000 7f545b000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54540ad8d0 0x7f5454159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.288+0000 7f545b000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f545406b880 0x7f5454153080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.288+0000 7f545b000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f54540c6650 0x7f54541535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.288+0000 7f545b000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54540c61f0 con 0x7f54540c6650 2024-03-09T09:25:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.288+0000 7f545b000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545406c040 con 0x7f545406b880 2024-03-09T09:25:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.288+0000 7f545b000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545406bd60 con 0x7f54540ad8d0 2024-03-09T09:25:00.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.288+0000 7f545a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f54540c6650 0x7f54541535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.296 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.296 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 28 -eq 48 ']' 2024-03-09T09:25:00.296 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.28 2024-03-09T09:25:00.297 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.28 2024-03-09T09:25:00.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.288+0000 7f545a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f54540c6650 0x7f54541535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37918/0 (socket says 172.21.15.67:37918) 2024-03-09T09:25:00.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.288+0000 7f545a600640 1 -- 172.21.15.67:0/3399377525 learned_addr learned my addr 172.21.15.67:0/3399377525 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:00.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.288+0000 7f5459c00640 1 --2- 172.21.15.67:0/3399377525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54540ad8d0 0x7f5454159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.289+0000 7f5459200640 1 --2- 172.21.15.67:0/3399377525 >> v2:172.21.15.134:3300/0 conn(0x7f545406b880 0x7f5454153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.289+0000 7f545a600640 1 -- 172.21.15.67:0/3399377525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54540ad8d0 msgr2=0x7f5454159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.289+0000 7f545a600640 1 --2- 172.21.15.67:0/3399377525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54540ad8d0 0x7f5454159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.289+0000 7f545a600640 1 -- 172.21.15.67:0/3399377525 >> v2:172.21.15.134:3300/0 conn(0x7f545406b880 msgr2=0x7f5454153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.289+0000 7f545a600640 1 --2- 172.21.15.67:0/3399377525 >> v2:172.21.15.134:3300/0 conn(0x7f545406b880 0x7f5454153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.289+0000 7f545a600640 1 -- 172.21.15.67:0/3399377525 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54540c5e80 con 0x7f54540c6650 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.289+0000 7f5459200640 1 --2- 172.21.15.67:0/3399377525 >> v2:172.21.15.134:3300/0 conn(0x7f545406b880 0x7f5454153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.290+0000 7f545a600640 1 --2- 172.21.15.67:0/3399377525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540c6650 0x7f54541535c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f54540c8e70 tx=0x7f544c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bc0b11d0093246 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.290+0000 7f5453e00640 1 -- 172.21.15.67:0/3399377525 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c02c9e0 con 0x7f54540c6650 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.290+0000 7f5453e00640 1 -- 172.21.15.67:0/3399377525 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c02cb80 con 0x7f54540c6650 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.290+0000 7f5453e00640 1 -- 172.21.15.67:0/3399377525 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c0343f0 con 0x7f54540c6650 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.291+0000 7f545b000640 1 -- 172.21.15.67:0/3399377525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540c6650 msgr2=0x7f54541535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.291+0000 7f545b000640 1 --2- 172.21.15.67:0/3399377525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540c6650 0x7f54541535c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f54540c8e70 tx=0x7f544c02be40 comp rx=0 tx=0).stop 2024-03-09T09:25:00.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.291+0000 7f545b000640 1 -- 172.21.15.67:0/3399377525 shutdown_connections 2024-03-09T09:25:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.291+0000 7f545b000640 1 --2- 172.21.15.67:0/3399377525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540c6650 0x7f54541535c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.291+0000 7f545b000640 1 --2- 172.21.15.67:0/3399377525 >> v2:172.21.15.134:3300/0 conn(0x7f545406b880 0x7f5454153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.291+0000 7f545b000640 1 --2- 172.21.15.67:0/3399377525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54540ad8d0 0x7f5454159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.291+0000 7f545b000640 1 -- 172.21.15.67:0/3399377525 >> 172.21.15.67:0/3399377525 conn(0x7f54540c8510 msgr2=0x7f54540acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.291+0000 7f545b000640 1 -- 172.21.15.67:0/3399377525 shutdown_connections 2024-03-09T09:25:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.291+0000 7f545b000640 1 -- 172.21.15.67:0/3399377525 wait complete. 2024-03-09T09:25:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.293+0000 7f545b000640 1 Processor -- start 2024-03-09T09:25:00.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.293+0000 7f545b000640 1 -- start start 2024-03-09T09:25:00.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.294+0000 7f545b000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f545406b880 0x7f545414ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.294+0000 7f545b000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ad8d0 0x7f54540ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f545b000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54540c6650 0x7f54540af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f545b000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5454155e40 con 0x7f54540ad8d0 2024-03-09T09:25:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f545b000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54540adec0 con 0x7f54540c6650 2024-03-09T09:25:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f545b000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5454155fc0 con 0x7f545406b880 2024-03-09T09:25:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f5459200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ad8d0 0x7f54540ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f5459200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ad8d0 0x7f54540ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37922/0 (socket says 172.21.15.67:37922) 2024-03-09T09:25:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f5459200640 1 -- 172.21.15.67:0/3138987819 learned_addr learned my addr 172.21.15.67:0/3138987819 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:00.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f545a600640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54540c6650 0x7f54540af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f5459200640 1 -- 172.21.15.67:0/3138987819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f545406b880 msgr2=0x7f545414ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f5459200640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f545406b880 0x7f545414ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f5459200640 1 -- 172.21.15.67:0/3138987819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54540c6650 msgr2=0x7f54540af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f5459200640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54540c6650 0x7f54540af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.295+0000 7f5459200640 1 -- 172.21.15.67:0/3138987819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54540ac500 con 0x7f54540ad8d0 2024-03-09T09:25:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.296+0000 7f5459200640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ad8d0 0x7f54540ae290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f544000c7f0 tx=0x7f544000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.296+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440015070 con 0x7f54540ad8d0 2024-03-09T09:25:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.296+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5440007a30 con 0x7f54540ad8d0 2024-03-09T09:25:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.296+0000 7f545b000640 1 -- 172.21.15.67:0/3138987819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54540ac7e0 con 0x7f54540ad8d0 2024-03-09T09:25:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.296+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440007e00 con 0x7f54540ad8d0 2024-03-09T09:25:00.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.297+0000 7f545b000640 1 -- 172.21.15.67:0/3138987819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54541504c0 con 0x7f54540ad8d0 2024-03-09T09:25:00.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.298+0000 7f545b000640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54541ee430 0x7f54540af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.298+0000 7f545b000640 1 -- 172.21.15.67:0/3138987819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545414edb0 con 0x7f54541ee430 2024-03-09T09:25:00.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.298+0000 7f5459c00640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54541ee430 0x7f54540af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.298+0000 7f5459c00640 1 -- 172.21.15.67:0/3138987819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f545404f3c0 con 0x7f54541ee430 2024-03-09T09:25:00.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.299+0000 7f5459c00640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54541ee430 0x7f54540af670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f544800d4e0 tx=0x7f544800d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.299+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54480125f0 con 0x7f54541ee430 2024-03-09T09:25:00.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.299+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f5448019070 con 0x7f54541ee430 2024-03-09T09:25:00.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.299+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54541ee430 msgr2=0x7f54540af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.299+0000 7f5451600640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54541ee430 0x7f54540af670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f544800d4e0 tx=0x7f544800d9a0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.299+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54541ee430 msgr2=0x7f54540af670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:00.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.299+0000 7f5451600640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54541ee430 0x7f54540af670 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f544800d4e0 tx=0x7f544800d9a0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.300+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5440018070 con 0x7f54540ad8d0 2024-03-09T09:25:00.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.301+0000 7f5451600640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f543406afe0 0x7f543406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.301+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f544008a4c0 con 0x7f54540ad8d0 2024-03-09T09:25:00.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.302+0000 7f5459c00640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f543406afe0 0x7f543406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.303+0000 7f5459c00640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f543406afe0 0x7f543406d4a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5448018000 tx=0x7f5448012000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.307+0000 7f545b000640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ae7d0 0x7f54540c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.307+0000 7f545b000640 1 -- 172.21.15.67:0/3138987819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54540ace30 con 0x7f54540ae7d0 2024-03-09T09:25:00.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.307+0000 7f545a600640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ae7d0 0x7f54540c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.308+0000 7f545a600640 1 -- 172.21.15.67:0/3138987819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5454073a60 con 0x7f54540ae7d0 2024-03-09T09:25:00.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.308+0000 7f545a600640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ae7d0 0x7f54540c46f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f54540cab30 tx=0x7f544c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.308+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c044070 con 0x7f54540ae7d0 2024-03-09T09:25:00.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4537 (secure 0 0 0) 0x7f544c03e630 con 0x7f54540ae7d0 2024-03-09T09:25:00.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ae7d0 msgr2=0x7f54540c46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f5451600640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ae7d0 0x7f54540c46f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f54540cab30 tx=0x7f544c004a40 comp rx=0 tx=0).stop 2024-03-09T09:25:00.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f5451600640 1 -- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ae7d0 msgr2=0x7f54540c46f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:00.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f5451600640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ae7d0 0x7f54540c46f0 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f54540cab30 tx=0x7f544c004a40 comp rx=0 tx=0).stop 2024-03-09T09:25:00.316 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:25:00.316 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 256, 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:00.317 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:25:00.318 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:25:00.319 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:00.320 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.321 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:25:00.322 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:00.323 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:25:00.324 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.327 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.328 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:25:00.328 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:25:00.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f545b000640 1 -- 172.21.15.67:0/3138987819 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f543406afe0 msgr2=0x7f543406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f545b000640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f543406afe0 0x7f543406d4a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5448018000 tx=0x7f5448012000 comp rx=0 tx=0).stop 2024-03-09T09:25:00.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f545b000640 1 -- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ad8d0 msgr2=0x7f54540ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f545b000640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ad8d0 0x7f54540ae290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f544000c7f0 tx=0x7f544000ccb0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f545a600640 1 -- 172.21.15.67:0/3138987819 reap_dead start 2024-03-09T09:25:00.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f545b000640 1 -- 172.21.15.67:0/3138987819 shutdown_connections 2024-03-09T09:25:00.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f545b000640 1 --2- 172.21.15.67:0/3138987819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54540ad8d0 0x7f54540ae290 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.309+0000 7f545b000640 1 -- 172.21.15.67:0/3138987819 >> 172.21.15.67:0/3138987819 conn(0x7f54540c8510 msgr2=0x7f5454151220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:00.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.310+0000 7f545b000640 1 -- 172.21.15.67:0/3138987819 shutdown_connections 2024-03-09T09:25:00.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.310+0000 7f545b000640 1 -- 172.21.15.67:0/3138987819 wait complete. 2024-03-09T09:25:00.330 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.330 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 29 -eq 48 ']' 2024-03-09T09:25:00.330 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.29 2024-03-09T09:25:00.330 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.29 2024-03-09T09:25:00.336 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:25:00.337 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:25:00.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:25:00.353 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.353 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 30 -eq 48 ']' 2024-03-09T09:25:00.353 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.30 2024-03-09T09:25:00.353 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.30 2024-03-09T09:25:00.379 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.380 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 31 -eq 48 ']' 2024-03-09T09:25:00.380 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.31 2024-03-09T09:25:00.380 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.31 2024-03-09T09:25:00.412 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.412 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 32 -eq 48 ']' 2024-03-09T09:25:00.412 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.32 2024-03-09T09:25:00.413 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.32 2024-03-09T09:25:00.444 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.444 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 33 -eq 48 ']' 2024-03-09T09:25:00.444 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.33 2024-03-09T09:25:00.444 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.33 2024-03-09T09:25:00.473 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.473 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 34 -eq 48 ']' 2024-03-09T09:25:00.473 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.34 2024-03-09T09:25:00.473 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.34 2024-03-09T09:25:00.517 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.517 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 35 -eq 48 ']' 2024-03-09T09:25:00.517 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.35 2024-03-09T09:25:00.517 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.35 2024-03-09T09:25:00.554 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.554 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 36 -eq 48 ']' 2024-03-09T09:25:00.554 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.36 2024-03-09T09:25:00.554 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.36 2024-03-09T09:25:00.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.554+0000 7fd66e600640 1 Processor -- start 2024-03-09T09:25:00.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.554+0000 7fd66e600640 1 -- start start 2024-03-09T09:25:00.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.556+0000 7fd66e600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6681535e0 0x7fd66806dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.556+0000 7fd66e600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd6680730d0 0x7fd6680c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.556+0000 7fd66e600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd668156a40 con 0x7fd6680c5900 2024-03-09T09:25:00.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.556+0000 7fd66e600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd668073850 con 0x7fd6680730d0 2024-03-09T09:25:00.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.556+0000 7fd66e600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd668073570 con 0x7fd6681535e0 2024-03-09T09:25:00.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.557+0000 7fd667e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd6680730d0 0x7fd6680c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.557+0000 7fd666a00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd6680c5900 0x7fd6680729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:50404/0 (socket says 172.21.15.67:50404) 2024-03-09T09:25:00.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.557+0000 7fd666a00640 1 -- 172.21.15.67:0/3094510121 learned_addr learned my addr 172.21.15.67:0/3094510121 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:00.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.557+0000 7fd667400640 1 --2- 172.21.15.67:0/3094510121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6681535e0 0x7fd66806dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.558+0000 7fd666000640 1 -- 172.21.15.67:0/3094510121 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210885266 0 0) 0x7fd668156a40 con 0x7fd6680c5900 2024-03-09T09:25:00.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.558+0000 7fd666000640 1 -- 172.21.15.67:0/3094510121 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd64c003520 con 0x7fd6680c5900 2024-03-09T09:25:00.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.558+0000 7fd667400640 1 -- 172.21.15.67:0/3094510121 >> v2:172.21.15.67:3300/0 conn(0x7fd6680730d0 msgr2=0x7fd6680c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:00.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.558+0000 7fd667400640 1 --2- 172.21.15.67:0/3094510121 >> v2:172.21.15.67:3300/0 conn(0x7fd6680730d0 0x7fd6680c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.558+0000 7fd667400640 1 -- 172.21.15.67:0/3094510121 >> v1:172.21.15.134:6789/0 conn(0x7fd6680c5900 legacy=0x7fd6680729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.558+0000 7fd667400640 1 -- 172.21.15.67:0/3094510121 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd668156ce0 con 0x7fd6681535e0 2024-03-09T09:25:00.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.559+0000 7fd667400640 1 --2- 172.21.15.67:0/3094510121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6681535e0 0x7fd66806dce0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd6580097a0 tx=0x7fd65802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde243034bb58d30 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.560+0000 7fd666000640 1 -- 172.21.15.67:0/3094510121 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6580315a0 con 0x7fd6681535e0 2024-03-09T09:25:00.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.560+0000 7fd666000640 1 -- 172.21.15.67:0/3094510121 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658037070 con 0x7fd6681535e0 2024-03-09T09:25:00.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.560+0000 7fd666000640 1 -- 172.21.15.67:0/3094510121 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658031940 con 0x7fd6681535e0 2024-03-09T09:25:00.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.561+0000 7fd66e600640 1 -- 172.21.15.67:0/3094510121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6681535e0 msgr2=0x7fd66806dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.561+0000 7fd66e600640 1 --2- 172.21.15.67:0/3094510121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6681535e0 0x7fd66806dce0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd6580097a0 tx=0x7fd65802b840 comp rx=0 tx=0).stop 2024-03-09T09:25:00.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.561+0000 7fd66e600640 1 -- 172.21.15.67:0/3094510121 shutdown_connections 2024-03-09T09:25:00.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.561+0000 7fd66e600640 1 --2- 172.21.15.67:0/3094510121 >> v2:172.21.15.67:3300/0 conn(0x7fd6680730d0 0x7fd6680c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.561+0000 7fd66e600640 1 --2- 172.21.15.67:0/3094510121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6681535e0 0x7fd66806dce0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.561+0000 7fd66e600640 1 -- 172.21.15.67:0/3094510121 >> 172.21.15.67:0/3094510121 conn(0x7fd6680ab320 msgr2=0x7fd6680ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:00.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.561+0000 7fd66e600640 1 -- 172.21.15.67:0/3094510121 shutdown_connections 2024-03-09T09:25:00.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.561+0000 7fd66e600640 1 -- 172.21.15.67:0/3094510121 wait complete. 2024-03-09T09:25:00.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.564+0000 7fd66e600640 1 Processor -- start 2024-03-09T09:25:00.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.564+0000 7fd66e600640 1 -- start start 2024-03-09T09:25:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.565+0000 7fd66e600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6680730d0 0x7fd6680af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.565+0000 7fd66e600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6680c5900 0x7fd668154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.565+0000 7fd66e600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6681535e0 0x7fd6681eee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.565+0000 7fd66e600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd668153b40 con 0x7fd6680c5900 2024-03-09T09:25:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.565+0000 7fd66e600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681539c0 con 0x7fd6680730d0 2024-03-09T09:25:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.565+0000 7fd66e600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680c3090 con 0x7fd6681535e0 2024-03-09T09:25:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.566+0000 7fd666a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6680c5900 0x7fd668154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.566+0000 7fd666a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6680c5900 0x7fd668154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37948/0 (socket says 172.21.15.67:37948) 2024-03-09T09:25:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.566+0000 7fd666a00640 1 -- 172.21.15.67:0/3130908838 learned_addr learned my addr 172.21.15.67:0/3130908838 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.566+0000 7fd667e00640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6681535e0 0x7fd6681eee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.566+0000 7fd667400640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6680730d0 0x7fd6680af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.566+0000 7fd666a00640 1 -- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6681535e0 msgr2=0x7fd6681eee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.566+0000 7fd666a00640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6681535e0 0x7fd6681eee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.566+0000 7fd666a00640 1 -- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6680730d0 msgr2=0x7fd6680af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.566+0000 7fd666a00640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6680730d0 0x7fd6680af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.566+0000 7fd666a00640 1 -- 172.21.15.67:0/3130908838 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6681ee9b0 con 0x7fd6680c5900 2024-03-09T09:25:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.566+0000 7fd667e00640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6681535e0 0x7fd6681eee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:25:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.567+0000 7fd666a00640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6680c5900 0x7fd668154d60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd65400c430 tx=0x7fd65400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.567+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654017070 con 0x7fd6680c5900 2024-03-09T09:25:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.567+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd654005ce0 con 0x7fd6680c5900 2024-03-09T09:25:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.567+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6540072f0 con 0x7fd6680c5900 2024-03-09T09:25:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.567+0000 7fd66e600640 1 -- 172.21.15.67:0/3130908838 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6681ec1e0 con 0x7fd6680c5900 2024-03-09T09:25:00.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.567+0000 7fd66e600640 1 -- 172.21.15.67:0/3130908838 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66814e750 con 0x7fd6680c5900 2024-03-09T09:25:00.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.568+0000 7fd66e600640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6681552a0 0x7fd6681ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.568+0000 7fd66e600640 1 -- 172.21.15.67:0/3130908838 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66814c470 con 0x7fd6681552a0 2024-03-09T09:25:00.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.569+0000 7fd667400640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6681552a0 0x7fd6681ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.569+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd65401a070 con 0x7fd6680c5900 2024-03-09T09:25:00.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.570+0000 7fd667400640 1 -- 172.21.15.67:0/3130908838 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd668075a00 con 0x7fd6681552a0 2024-03-09T09:25:00.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.570+0000 7fd65f400640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd64806b4d0 0x7fd64806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.570+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fd654089a40 con 0x7fd6680c5900 2024-03-09T09:25:00.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.571+0000 7fd667400640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6681552a0 0x7fd6681ef3d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd6680723d0 tx=0x7fd6580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.571+0000 7fd667e00640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd64806b4d0 0x7fd64806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.571+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd658006e10 con 0x7fd6681552a0 2024-03-09T09:25:00.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.572+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fd658037040 con 0x7fd6681552a0 2024-03-09T09:25:00.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.572+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6681552a0 msgr2=0x7fd6681ef3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.572+0000 7fd65f400640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6681552a0 0x7fd6681ef3d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd6680723d0 tx=0x7fd6580046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.572+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6681552a0 msgr2=0x7fd6681ef3d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:00.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.572+0000 7fd65f400640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6681552a0 0x7fd6681ef3d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.572+0000 7fd667e00640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd64806b4d0 0x7fd64806d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd66806bb00 tx=0x7fd66000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.583 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.583 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 37 -eq 48 ']' 2024-03-09T09:25:00.583 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.37 2024-03-09T09:25:00.583 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.37 2024-03-09T09:25:00.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.579+0000 7fd66e600640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd66814db70 0x7fd66814c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.579+0000 7fd66e600640 1 -- 172.21.15.67:0/3130908838 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680c8ae0 con 0x7fd66814db70 2024-03-09T09:25:00.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.580+0000 7fd667400640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd66814db70 0x7fd66814c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.581+0000 7fd667400640 1 -- 172.21.15.67:0/3130908838 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd6680c8fe0 con 0x7fd66814db70 2024-03-09T09:25:00.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.581+0000 7fd667400640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd66814db70 0x7fd66814c950 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd658000c00 tx=0x7fd658031000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.582+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6580093c0 con 0x7fd66814db70 2024-03-09T09:25:00.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7fd6580093c0 con 0x7fd66814db70 2024-03-09T09:25:00.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd66814db70 msgr2=0x7fd66814c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd65f400640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd66814db70 0x7fd66814c950 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd658000c00 tx=0x7fd658031000 comp rx=0 tx=0).stop 2024-03-09T09:25:00.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd65f400640 1 -- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd66814db70 msgr2=0x7fd66814c950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:00.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd65f400640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd66814db70 0x7fd66814c950 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.590 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 256, 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 12, 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:25:00.591 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:25:00.592 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:25:00.593 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:25:00.594 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.595 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.597 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.598 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:00.600 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.601 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.602 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.603 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:00.604 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.604 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.604 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:25:00.604 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:25:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd66e600640 1 -- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd64806b4d0 msgr2=0x7fd64806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd66e600640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd64806b4d0 0x7fd64806d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd66806bb00 tx=0x7fd66000b040 comp rx=0 tx=0).stop 2024-03-09T09:25:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd66e600640 1 -- 172.21.15.67:0/3130908838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6680c5900 msgr2=0x7fd668154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd66e600640 1 --2- 172.21.15.67:0/3130908838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd6680c5900 0x7fd668154d60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd65400c430 tx=0x7fd65400c8f0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd667e00640 1 -- 172.21.15.67:0/3130908838 reap_dead start 2024-03-09T09:25:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd667e00640 1 -- 172.21.15.67:0/3130908838 reap_dead start 2024-03-09T09:25:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd66e600640 1 -- 172.21.15.67:0/3130908838 shutdown_connections 2024-03-09T09:25:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.583+0000 7fd66e600640 1 -- 172.21.15.67:0/3130908838 >> 172.21.15.67:0/3130908838 conn(0x7fd6680ab320 msgr2=0x7fd6680729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:00.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.584+0000 7fd66e600640 1 -- 172.21.15.67:0/3130908838 shutdown_connections 2024-03-09T09:25:00.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.584+0000 7fd66e600640 1 -- 172.21.15.67:0/3130908838 wait complete. 2024-03-09T09:25:00.607 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:25:00.607 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:25:00.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:25:00.611 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.611 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 38 -eq 48 ']' 2024-03-09T09:25:00.611 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.38 2024-03-09T09:25:00.611 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.38 2024-03-09T09:25:00.637 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.638 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 39 -eq 48 ']' 2024-03-09T09:25:00.638 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.39 2024-03-09T09:25:00.638 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.39 2024-03-09T09:25:00.671 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.671 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 40 -eq 48 ']' 2024-03-09T09:25:00.671 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.40 2024-03-09T09:25:00.671 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.40 2024-03-09T09:25:00.703 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.703 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 41 -eq 48 ']' 2024-03-09T09:25:00.703 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.41 2024-03-09T09:25:00.703 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.41 2024-03-09T09:25:00.736 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.736 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 42 -eq 48 ']' 2024-03-09T09:25:00.736 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.42 2024-03-09T09:25:00.736 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.42 2024-03-09T09:25:00.781 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.781 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 43 -eq 48 ']' 2024-03-09T09:25:00.781 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.43 2024-03-09T09:25:00.781 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.43 2024-03-09T09:25:00.812 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.812 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 44 -eq 48 ']' 2024-03-09T09:25:00.813 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.44 2024-03-09T09:25:00.813 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.44 2024-03-09T09:25:00.851 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.851 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 45 -eq 48 ']' 2024-03-09T09:25:00.851 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.45 2024-03-09T09:25:00.851 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.45 2024-03-09T09:25:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.844+0000 7f3f90800640 1 Processor -- start 2024-03-09T09:25:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.844+0000 7f3f90800640 1 -- start start 2024-03-09T09:25:00.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.846+0000 7f3f90800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3f8814c9a0 0x7f3f881567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.846+0000 7f3f90800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f880ac4e0 con 0x7f3f880ac9c0 2024-03-09T09:25:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.846+0000 7f3f90800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f88156d00 con 0x7f3f880c1d70 2024-03-09T09:25:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.846+0000 7f3f90800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f8806d590 con 0x7f3f8814c9a0 2024-03-09T09:25:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.846+0000 7f3f8e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f3f880ac9c0 0x7f3f880ace30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50312/0 (socket says 172.21.15.67:50312) 2024-03-09T09:25:00.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.846+0000 7f3f8e800640 1 -- 172.21.15.67:0/385070280 learned_addr learned my addr 172.21.15.67:0/385070280 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:00.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.846+0000 7f3f8de00640 1 --2- 172.21.15.67:0/385070280 >> v2:172.21.15.134:3300/0 conn(0x7f3f8814c9a0 0x7f3f881567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.847+0000 7f3f8ca00640 1 -- 172.21.15.67:0/385070280 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 209451057 0 0) 0x7f3f880ac4e0 con 0x7f3f880ac9c0 2024-03-09T09:25:00.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.847+0000 7f3f8ca00640 1 -- 172.21.15.67:0/385070280 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f70003520 con 0x7f3f880ac9c0 2024-03-09T09:25:00.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.847+0000 7f3f8de00640 1 -- 172.21.15.67:0/385070280 >> v1:172.21.15.134:6789/0 conn(0x7f3f880c1d70 legacy=0x7f3f880c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.847+0000 7f3f8de00640 1 -- 172.21.15.67:0/385070280 >> v1:172.21.15.67:6789/0 conn(0x7f3f880ac9c0 legacy=0x7f3f880ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.847+0000 7f3f8de00640 1 -- 172.21.15.67:0/385070280 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f880aa550 con 0x7f3f8814c9a0 2024-03-09T09:25:00.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.849+0000 7f3f8de00640 1 --2- 172.21.15.67:0/385070280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f8814c9a0 0x7f3f881567c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f880ac6b0 tx=0x7f3f7c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a69e7bf5434589a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.850+0000 7f3f8ca00640 1 -- 172.21.15.67:0/385070280 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c035020 con 0x7f3f8814c9a0 2024-03-09T09:25:00.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.855+0000 7f3f8ca00640 1 -- 172.21.15.67:0/385070280 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c031810 con 0x7f3f8814c9a0 2024-03-09T09:25:00.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.855+0000 7f3f8ca00640 1 -- 172.21.15.67:0/385070280 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c031bc0 con 0x7f3f8814c9a0 2024-03-09T09:25:00.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.855+0000 7f3f90800640 1 -- 172.21.15.67:0/385070280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f8814c9a0 msgr2=0x7f3f881567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.855+0000 7f3f90800640 1 --2- 172.21.15.67:0/385070280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f8814c9a0 0x7f3f881567c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f880ac6b0 tx=0x7f3f7c02b8b0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.855+0000 7f3f90800640 1 -- 172.21.15.67:0/385070280 shutdown_connections 2024-03-09T09:25:00.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.855+0000 7f3f90800640 1 --2- 172.21.15.67:0/385070280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f8814c9a0 0x7f3f881567c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.855+0000 7f3f90800640 1 -- 172.21.15.67:0/385070280 >> 172.21.15.67:0/385070280 conn(0x7f3f880c7370 msgr2=0x7f3f880c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.855+0000 7f3f90800640 1 -- 172.21.15.67:0/385070280 shutdown_connections 2024-03-09T09:25:00.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.855+0000 7f3f90800640 1 -- 172.21.15.67:0/385070280 wait complete. 2024-03-09T09:25:00.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.857+0000 7f3f90800640 1 Processor -- start 2024-03-09T09:25:00.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.857+0000 7f3f90800640 1 -- start start 2024-03-09T09:25:00.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.858+0000 7f3f90800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880ac9c0 0x7f3f881ec830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.858+0000 7f3f90800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f880c1d70 0x7f3f88072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.858+0000 7f3f90800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f8814c9a0 0x7f3f881e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.858+0000 7f3f90800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880aab10 con 0x7f3f8814c9a0 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.858+0000 7f3f90800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880ad540 con 0x7f3f880c1d70 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.858+0000 7f3f90800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880aadf0 con 0x7f3f880ac9c0 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.858+0000 7f3f8e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f8814c9a0 0x7f3f881e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.859+0000 7f3f8e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f8814c9a0 0x7f3f881e8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37964/0 (socket says 172.21.15.67:37964) 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.859+0000 7f3f8e800640 1 -- 172.21.15.67:0/190110407 learned_addr learned my addr 172.21.15.67:0/190110407 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.859+0000 7f3f8d400640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f880c1d70 0x7f3f88072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.859+0000 7f3f8de00640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880ac9c0 0x7f3f881ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.859+0000 7f3f8e800640 1 -- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880ac9c0 msgr2=0x7f3f881ec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.859+0000 7f3f8e800640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880ac9c0 0x7f3f881ec830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.859+0000 7f3f8e800640 1 -- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f880c1d70 msgr2=0x7f3f88072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.859+0000 7f3f8e800640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f880c1d70 0x7f3f88072980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.859+0000 7f3f8e800640 1 -- 172.21.15.67:0/190110407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8814d7f0 con 0x7f3f8814c9a0 2024-03-09T09:25:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.859+0000 7f3f8de00640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880ac9c0 0x7f3f881ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:25:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.859+0000 7f3f8e800640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f8814c9a0 0x7f3f881e8690 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400cb40 tx=0x7f3f8400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.860+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84017070 con 0x7f3f8814c9a0 2024-03-09T09:25:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.860+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8400aca0 con 0x7f3f8814c9a0 2024-03-09T09:25:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.860+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84012620 con 0x7f3f8814c9a0 2024-03-09T09:25:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.860+0000 7f3f90800640 1 -- 172.21.15.67:0/190110407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f88157040 con 0x7f3f8814c9a0 2024-03-09T09:25:00.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.860+0000 7f3f90800640 1 -- 172.21.15.67:0/190110407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f88157600 con 0x7f3f8814c9a0 2024-03-09T09:25:00.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.861+0000 7f3f90800640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f88148ea0 0x7f3f881e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.861+0000 7f3f90800640 1 -- 172.21.15.67:0/190110407 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881421f0 con 0x7f3f88148ea0 2024-03-09T09:25:00.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.862+0000 7f3f8de00640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f88148ea0 0x7f3f881e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.863+0000 7f3f8de00640 1 -- 172.21.15.67:0/190110407 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f88075a00 con 0x7f3f88148ea0 2024-03-09T09:25:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.863+0000 7f3f8de00640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f88148ea0 0x7f3f881e8bd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f8815a400 tx=0x7f3f7c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.863+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3f8401a070 con 0x7f3f8814c9a0 2024-03-09T09:25:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.864+0000 7f3f82000640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3f6c06b4d0 0x7f3f6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.864+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f3f8408a3c0 con 0x7f3f8814c9a0 2024-03-09T09:25:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.865+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c035040 con 0x7f3f88148ea0 2024-03-09T09:25:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.865+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f3f7c003810 con 0x7f3f88148ea0 2024-03-09T09:25:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.865+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f88148ea0 msgr2=0x7f3f881e8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.865+0000 7f3f82000640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f88148ea0 0x7f3f881e8bd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f8815a400 tx=0x7f3f7c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.865+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f88148ea0 msgr2=0x7f3f881e8bd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.865+0000 7f3f82000640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f88148ea0 0x7f3f881e8bd0 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f8815a400 tx=0x7f3f7c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.865+0000 7f3f8d400640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3f6c06b4d0 0x7f3f6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.866+0000 7f3f8d400640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3f6c06b4d0 0x7f3f6c06d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3f880c7190 tx=0x7f3f78009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.878 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.878 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 46 -eq 48 ']' 2024-03-09T09:25:00.878 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.46 2024-03-09T09:25:00.878 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.46 2024-03-09T09:25:00.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.872+0000 7f3f90800640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880720e0 0x7f3f881eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:00.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.872+0000 7f3f90800640 1 -- 172.21.15.67:0/190110407 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880c4920 con 0x7f3f880720e0 2024-03-09T09:25:00.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.873+0000 7f3f8de00640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880720e0 0x7f3f881eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:00.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.874+0000 7f3f8de00640 1 -- 172.21.15.67:0/190110407 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3f88067dd0 con 0x7f3f880720e0 2024-03-09T09:25:00.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.874+0000 7f3f8de00640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880720e0 0x7f3f881eeb40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006ce0 tx=0x7f3f7c031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:00.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.875+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c035040 con 0x7f3f880720e0 2024-03-09T09:25:00.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f3f7c04a980 con 0x7f3f880720e0 2024-03-09T09:25:00.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880720e0 msgr2=0x7f3f881eeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f82000640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880720e0 0x7f3f881eeb40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006ce0 tx=0x7f3f7c031000 comp rx=0 tx=0).stop 2024-03-09T09:25:00.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f82000640 1 -- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880720e0 msgr2=0x7f3f881eeb40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:00.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f82000640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f880720e0 0x7f3f881eeb40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:00.883 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 256, 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:25:00.884 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:25:00.885 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.886 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:25:00.887 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.888 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.889 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:00.890 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:25:00.892 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:00.893 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.894 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:25:00.895 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:00.896 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:00.897 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:00.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:00.897 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:25:00.897 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:25:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f90800640 1 -- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3f6c06b4d0 msgr2=0x7f3f6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f90800640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3f6c06b4d0 0x7f3f6c06d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3f880c7190 tx=0x7f3f78009040 comp rx=0 tx=0).stop 2024-03-09T09:25:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f90800640 1 -- 172.21.15.67:0/190110407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f8814c9a0 msgr2=0x7f3f881e8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f90800640 1 --2- 172.21.15.67:0/190110407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f8814c9a0 0x7f3f881e8690 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400cb40 tx=0x7f3f8400a060 comp rx=0 tx=0).stop 2024-03-09T09:25:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f8e800640 1 -- 172.21.15.67:0/190110407 reap_dead start 2024-03-09T09:25:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f8e800640 1 -- 172.21.15.67:0/190110407 reap_dead start 2024-03-09T09:25:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f90800640 1 -- 172.21.15.67:0/190110407 shutdown_connections 2024-03-09T09:25:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.876+0000 7f3f90800640 1 -- 172.21.15.67:0/190110407 >> 172.21.15.67:0/190110407 conn(0x7f3f880c7370 msgr2=0x7f3f88153460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.877+0000 7f3f90800640 1 -- 172.21.15.67:0/190110407 shutdown_connections 2024-03-09T09:25:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:00.877+0000 7f3f90800640 1 -- 172.21.15.67:0/190110407 wait complete. 2024-03-09T09:25:00.901 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:25:00.901 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-03-09T09:25:00.901 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:25:00.901 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-03-09T09:25:00.901 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:25:00.902 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-09T09:25:00.902 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:25:00.905 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.905 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 47 -eq 48 ']' 2024-03-09T09:25:00.905 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.47 2024-03-09T09:25:00.905 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.47 2024-03-09T09:25:00.935 INFO:tasks.ceph.mon.b:Stopped 2024-03-09T09:25:00.935 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:25:00.935 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:25:00.941 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.941 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 48 -eq 48 ']' 2024-03-09T09:25:00.941 INFO:tasks.workunit.client.0.smithi067.stderr:+ continue 2024-03-09T09:25:00.941 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.941 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 49 -eq 48 ']' 2024-03-09T09:25:00.941 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.49 2024-03-09T09:25:00.941 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.49 2024-03-09T09:25:00.965 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.965 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 50 -eq 48 ']' 2024-03-09T09:25:00.965 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.50 2024-03-09T09:25:00.965 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.50 2024-03-09T09:25:00.992 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:00.992 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 51 -eq 48 ']' 2024-03-09T09:25:00.992 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.51 2024-03-09T09:25:00.992 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.51 2024-03-09T09:25:01.019 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.019 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 52 -eq 48 ']' 2024-03-09T09:25:01.019 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.52 2024-03-09T09:25:01.019 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.52 2024-03-09T09:25:01.053 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.054 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 53 -eq 48 ']' 2024-03-09T09:25:01.054 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.53 2024-03-09T09:25:01.054 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.53 2024-03-09T09:25:01.088 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.088 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 54 -eq 48 ']' 2024-03-09T09:25:01.088 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.54 2024-03-09T09:25:01.089 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.54 2024-03-09T09:25:01.127 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.127 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 55 -eq 48 ']' 2024-03-09T09:25:01.127 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.55 2024-03-09T09:25:01.127 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.55 2024-03-09T09:25:01.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.136+0000 7fbace800640 1 Processor -- start 2024-03-09T09:25:01.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.136+0000 7fbace800640 1 -- start start 2024-03-09T09:25:01.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.139+0000 7fbace800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbac80c5da0 0x7fbac80c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:01.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.139+0000 7fbace800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbac81511f0 0x7fbac8070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:01.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.139+0000 7fbace800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac80c3ba0 con 0x7fbac80c6b70 2024-03-09T09:25:01.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.139+0000 7fbace800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac8154370 con 0x7fbac80c5da0 2024-03-09T09:25:01.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.139+0000 7fbace800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac81548b0 con 0x7fbac81511f0 2024-03-09T09:25:01.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.139+0000 7fbacde00640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7fbac81511f0 msgr2=0x7fbac8070170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:01.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.139+0000 7fbacde00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbac81511f0 0x7fbac8070170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:01.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.139+0000 7fbacca00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbac80c6b70 0x7fbac8150ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50322/0 (socket says 172.21.15.67:50322) 2024-03-09T09:25:01.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.139+0000 7fbacca00640 1 -- 172.21.15.67:0/1387030668 learned_addr learned my addr 172.21.15.67:0/1387030668 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:01.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.140+0000 7fbacd400640 1 --2- 172.21.15.67:0/1387030668 >> v2:172.21.15.67:3300/0 conn(0x7fbac80c5da0 0x7fbac80c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:01.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.140+0000 7fbacd400640 1 -- 172.21.15.67:0/1387030668 >> v2:172.21.15.134:3300/0 conn(0x7fbac81511f0 msgr2=0x7fbac8070170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:25:01.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.140+0000 7fbacd400640 1 --2- 172.21.15.67:0/1387030668 >> v2:172.21.15.134:3300/0 conn(0x7fbac81511f0 0x7fbac8070170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:01.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.140+0000 7fbac7e00640 1 -- 172.21.15.67:0/1387030668 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582424154 0 0) 0x7fbac80c3ba0 con 0x7fbac80c6b70 2024-03-09T09:25:01.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.140+0000 7fbacd400640 1 -- 172.21.15.67:0/1387030668 >> v1:172.21.15.67:6789/0 conn(0x7fbac80c6b70 legacy=0x7fbac8150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:01.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.140+0000 7fbacd400640 1 -- 172.21.15.67:0/1387030668 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac81545d0 con 0x7fbac80c5da0 2024-03-09T09:25:01.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.141+0000 7fbacd400640 1 --2- 172.21.15.67:0/1387030668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac80c5da0 0x7fbac80c6180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbac806e740 tx=0x7fbabc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2b7c9dcf5897e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:01.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.142+0000 7fbac7e00640 1 -- 172.21.15.67:0/1387030668 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc035020 con 0x7fbac80c5da0 2024-03-09T09:25:01.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.142+0000 7fbac7e00640 1 -- 172.21.15.67:0/1387030668 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc031640 con 0x7fbac80c5da0 2024-03-09T09:25:01.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.142+0000 7fbac7e00640 1 -- 172.21.15.67:0/1387030668 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc02d040 con 0x7fbac80c5da0 2024-03-09T09:25:01.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.143+0000 7fbace800640 1 -- 172.21.15.67:0/1387030668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac80c5da0 msgr2=0x7fbac80c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:01.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.143+0000 7fbace800640 1 --2- 172.21.15.67:0/1387030668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac80c5da0 0x7fbac80c6180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbac806e740 tx=0x7fbabc02b8f0 comp rx=0 tx=0).stop 2024-03-09T09:25:01.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.143+0000 7fbace800640 1 -- 172.21.15.67:0/1387030668 shutdown_connections 2024-03-09T09:25:01.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.143+0000 7fbace800640 1 --2- 172.21.15.67:0/1387030668 >> v2:172.21.15.134:3300/0 conn(0x7fbac81511f0 0x7fbac8070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:01.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.143+0000 7fbace800640 1 --2- 172.21.15.67:0/1387030668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac80c5da0 0x7fbac80c6180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:01.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.143+0000 7fbace800640 1 -- 172.21.15.67:0/1387030668 >> 172.21.15.67:0/1387030668 conn(0x7fbac80c9470 msgr2=0x7fbac80c9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:01.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.143+0000 7fbace800640 1 -- 172.21.15.67:0/1387030668 shutdown_connections 2024-03-09T09:25:01.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.143+0000 7fbace800640 1 -- 172.21.15.67:0/1387030668 wait complete. 2024-03-09T09:25:01.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.146+0000 7fbace800640 1 Processor -- start 2024-03-09T09:25:01.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.146+0000 7fbace800640 1 -- start start 2024-03-09T09:25:01.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbace800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbac80c5da0 0x7fbac8155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:01.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbace800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac80c6b70 0x7fbac8145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:01.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbace800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81511f0 0x7fbac8151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:01.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbace800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80c4560 con 0x7fbac80c6b70 2024-03-09T09:25:01.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbace800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80c41f0 con 0x7fbac81511f0 2024-03-09T09:25:01.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbace800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac80c66c0 con 0x7fbac80c5da0 2024-03-09T09:25:01.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbacde00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81511f0 msgr2=0x7fbac8151c10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:01.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbacde00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81511f0 0x7fbac8151c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:01.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbacca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac80c6b70 0x7fbac8145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:01.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbacca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac80c6b70 0x7fbac8145e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37998/0 (socket says 172.21.15.67:37998) 2024-03-09T09:25:01.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbacca00640 1 -- 172.21.15.67:0/499244730 learned_addr learned my addr 172.21.15.67:0/499244730 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:01.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.147+0000 7fbacd400640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbac80c5da0 0x7fbac8155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:01.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.148+0000 7fbacca00640 1 -- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbac80c5da0 msgr2=0x7fbac8155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:01.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.148+0000 7fbacca00640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbac80c5da0 0x7fbac8155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:01.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.148+0000 7fbacca00640 1 -- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81511f0 msgr2=0x7fbac8151c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:25:01.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.148+0000 7fbacca00640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81511f0 0x7fbac8151c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:01.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.148+0000 7fbacca00640 1 -- 172.21.15.67:0/499244730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac8153d70 con 0x7fbac80c6b70 2024-03-09T09:25:01.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.148+0000 7fbacca00640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac80c6b70 0x7fbac8145e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c4d0 tx=0x7fbab400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:01.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.149+0000 7fbac5600640 1 -- 172.21.15.67:0/499244730 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4018070 con 0x7fbac80c6b70 2024-03-09T09:25:01.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.149+0000 7fbac5600640 1 -- 172.21.15.67:0/499244730 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab4007500 con 0x7fbac80c6b70 2024-03-09T09:25:01.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.149+0000 7fbac5600640 1 -- 172.21.15.67:0/499244730 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4004a20 con 0x7fbac80c6b70 2024-03-09T09:25:01.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.149+0000 7fbace800640 1 -- 172.21.15.67:0/499244730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac8155990 con 0x7fbac80c6b70 2024-03-09T09:25:01.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.149+0000 7fbace800640 1 -- 172.21.15.67:0/499244730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac8155170 con 0x7fbac80c6b70 2024-03-09T09:25:01.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.151+0000 7fbace800640 1 -- 172.21.15.67:0/499244730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac804f3c0 con 0x7fbac80c6b70 2024-03-09T09:25:01.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.153+0000 7fbac5600640 1 -- 172.21.15.67:0/499244730 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbab4004bc0 con 0x7fbac80c6b70 2024-03-09T09:25:01.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.154+0000 7fbac5600640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbaa806b480 0x7fbaa806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:01.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.154+0000 7fbac5600640 1 -- 172.21.15.67:0/499244730 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fbab4089700 con 0x7fbac80c6b70 2024-03-09T09:25:01.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.155+0000 7fbacd400640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbaa806b480 0x7fbaa806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:01.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.156+0000 7fbacd400640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbaa806b480 0x7fbaa806d940 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbabc036000 tx=0x7fbabc03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:01.164 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.164 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 56 -eq 48 ']' 2024-03-09T09:25:01.164 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.56 2024-03-09T09:25:01.164 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.56 2024-03-09T09:25:01.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.164+0000 7fbac5600640 1 -- 172.21.15.67:0/499244730 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbab4057bd0 con 0x7fbac80c6b70 2024-03-09T09:25:01.198 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.198 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 57 -eq 48 ']' 2024-03-09T09:25:01.198 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.57 2024-03-09T09:25:01.198 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.57 2024-03-09T09:25:01.220 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:25:01.227 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:25:01.233 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.233 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 58 -eq 48 ']' 2024-03-09T09:25:01.233 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.58 2024-03-09T09:25:01.233 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.58 2024-03-09T09:25:01.264 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.264 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 59 -eq 48 ']' 2024-03-09T09:25:01.264 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.59 2024-03-09T09:25:01.264 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.59 2024-03-09T09:25:01.287 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.287 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 60 -eq 48 ']' 2024-03-09T09:25:01.287 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.60 2024-03-09T09:25:01.287 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.60 2024-03-09T09:25:01.318 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.318 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 61 -eq 48 ']' 2024-03-09T09:25:01.318 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.61 2024-03-09T09:25:01.318 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.61 2024-03-09T09:25:01.345 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.345 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 62 -eq 48 ']' 2024-03-09T09:25:01.345 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.62 2024-03-09T09:25:01.345 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.62 2024-03-09T09:25:01.380 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.380 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 63 -eq 48 ']' 2024-03-09T09:25:01.380 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.63 2024-03-09T09:25:01.380 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.63 2024-03-09T09:25:01.416 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.416 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 64 -eq 48 ']' 2024-03-09T09:25:01.416 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.64 2024-03-09T09:25:01.416 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.64 2024-03-09T09:25:01.457 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.458 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 65 -eq 48 ']' 2024-03-09T09:25:01.458 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.65 2024-03-09T09:25:01.458 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.65 2024-03-09T09:25:01.496 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.496 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 66 -eq 48 ']' 2024-03-09T09:25:01.496 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.66 2024-03-09T09:25:01.496 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.66 2024-03-09T09:25:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.488+0000 7fbace800640 1 -- 172.21.15.67:0/499244730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbac814e670 con 0x7fbac80c6b70 2024-03-09T09:25:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.489+0000 7fbac5600640 1 -- 172.21.15.67:0/499244730 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fbab405bc70 con 0x7fbac80c6b70 2024-03-09T09:25:01.497 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":256,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:25:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 -- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbaa806b480 msgr2=0x7fbaa806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbaa806b480 0x7fbaa806d940 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbabc036000 tx=0x7fbabc03d040 comp rx=0 tx=0).stop 2024-03-09T09:25:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 -- 172.21.15.67:0/499244730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac80c6b70 msgr2=0x7fbac8145e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac80c6b70 0x7fbac8145e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c4d0 tx=0x7fbab400c990 comp rx=0 tx=0).stop 2024-03-09T09:25:01.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 -- 172.21.15.67:0/499244730 shutdown_connections 2024-03-09T09:25:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbaa806b480 0x7fbaa806d940 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81511f0 0x7fbac8151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbac80c6b70 0x7fbac8145e30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 --2- 172.21.15.67:0/499244730 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbac80c5da0 0x7fbac8155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 -- 172.21.15.67:0/499244730 >> 172.21.15.67:0/499244730 conn(0x7fbac80c9470 msgr2=0x7fbac80c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 -- 172.21.15.67:0/499244730 shutdown_connections 2024-03-09T09:25:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:01.493+0000 7fbace800640 1 -- 172.21.15.67:0/499244730 wait complete. 2024-03-09T09:25:01.529 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.529 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 67 -eq 48 ']' 2024-03-09T09:25:01.529 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.67 2024-03-09T09:25:01.529 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.67 2024-03-09T09:25:01.566 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.566 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 68 -eq 48 ']' 2024-03-09T09:25:01.566 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.68 2024-03-09T09:25:01.566 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.68 2024-03-09T09:25:01.609 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.610 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 69 -eq 48 ']' 2024-03-09T09:25:01.610 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.69 2024-03-09T09:25:01.610 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.69 2024-03-09T09:25:01.645 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.645 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 70 -eq 48 ']' 2024-03-09T09:25:01.646 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.70 2024-03-09T09:25:01.646 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.70 2024-03-09T09:25:01.687 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.687 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 71 -eq 48 ']' 2024-03-09T09:25:01.687 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.71 2024-03-09T09:25:01.687 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.71 2024-03-09T09:25:01.720 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.720 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 72 -eq 48 ']' 2024-03-09T09:25:01.721 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.72 2024-03-09T09:25:01.721 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.72 2024-03-09T09:25:01.768 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:01.769 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 73 -eq 48 ']' 2024-03-09T09:25:01.769 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.73 2024-03-09T09:25:01.769 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.73 2024-03-09T09:25:01.836 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:25:01.837 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:01.837 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 85 -eq 48 ']' 2024-03-09T09:25:01.837 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.85 2024-03-09T09:25:01.837 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.85 2024-03-09T09:25:01.837 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.85 2024-03-09T09:25:01.838 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.85 2024-03-09T09:25:01.841 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:25:02.034 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.034 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 74 -eq 48 ']' 2024-03-09T09:25:02.035 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.74 2024-03-09T09:25:02.035 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.74 2024-03-09T09:25:02.076 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.077 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 75 -eq 48 ']' 2024-03-09T09:25:02.077 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.75 2024-03-09T09:25:02.077 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.75 2024-03-09T09:25:02.116 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.117 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 76 -eq 48 ']' 2024-03-09T09:25:02.117 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.76 2024-03-09T09:25:02.117 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.76 2024-03-09T09:25:02.157 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.157 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 77 -eq 48 ']' 2024-03-09T09:25:02.157 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.77 2024-03-09T09:25:02.157 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.77 2024-03-09T09:25:02.196 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.196 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 78 -eq 48 ']' 2024-03-09T09:25:02.196 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.78 2024-03-09T09:25:02.196 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.78 2024-03-09T09:25:02.251 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.251 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 79 -eq 48 ']' 2024-03-09T09:25:02.251 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.79 2024-03-09T09:25:02.251 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.79 2024-03-09T09:25:02.293 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.293 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 80 -eq 48 ']' 2024-03-09T09:25:02.294 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.80 2024-03-09T09:25:02.294 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.80 2024-03-09T09:25:02.337 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.338 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 81 -eq 48 ']' 2024-03-09T09:25:02.338 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.81 2024-03-09T09:25:02.338 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.81 2024-03-09T09:25:02.373 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.373 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 82 -eq 48 ']' 2024-03-09T09:25:02.373 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.82 2024-03-09T09:25:02.373 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.82 2024-03-09T09:25:02.418 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.418 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 83 -eq 48 ']' 2024-03-09T09:25:02.418 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.83 2024-03-09T09:25:02.418 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.83 2024-03-09T09:25:02.454 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.454 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 84 -eq 48 ']' 2024-03-09T09:25:02.454 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.84 2024-03-09T09:25:02.454 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.84 2024-03-09T09:25:02.494 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.494 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 85 -eq 48 ']' 2024-03-09T09:25:02.494 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.85 2024-03-09T09:25:02.494 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.85 2024-03-09T09:25:02.533 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.533 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 86 -eq 48 ']' 2024-03-09T09:25:02.533 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.86 2024-03-09T09:25:02.533 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.86 2024-03-09T09:25:02.572 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.572 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 87 -eq 48 ']' 2024-03-09T09:25:02.572 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.87 2024-03-09T09:25:02.572 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.87 2024-03-09T09:25:02.609 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.609 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 88 -eq 48 ']' 2024-03-09T09:25:02.609 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.88 2024-03-09T09:25:02.609 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.88 2024-03-09T09:25:02.652 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.653 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 89 -eq 48 ']' 2024-03-09T09:25:02.653 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.89 2024-03-09T09:25:02.653 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.89 2024-03-09T09:25:02.691 INFO:tasks.workunit.client.0.smithi067.stderr:+ for v in $versions 2024-03-09T09:25:02.691 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 90 -eq 48 ']' 2024-03-09T09:25:02.691 INFO:tasks.workunit.client.0.smithi067.stderr:+ ver=v0.90 2024-03-09T09:25:02.691 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/v0.90 2024-03-09T09:25:02.731 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T09:25:02.732 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T09:25:02.733 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:25:02.733 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T09:25:04.521 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:25:04.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.726+0000 7f18e3000640 1 Processor -- start 2024-03-09T09:25:04.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.726+0000 7f18e3000640 1 -- start start 2024-03-09T09:25:04.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.728+0000 7f18e3000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f18dc153300 0x7f18dc0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:04.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.728+0000 7f18e3000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc0c5e40 0x7f18dc0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:04.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.728+0000 7f18e3000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18dc157000 con 0x7f18dc0cadb0 2024-03-09T09:25:04.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.728+0000 7f18e3000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc0ae430 con 0x7f18dc153300 2024-03-09T09:25:04.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.728+0000 7f18e3000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0ae6f0 con 0x7f18dc0c5e40 2024-03-09T09:25:04.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.728+0000 7f18e2600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f18dc0cadb0 0x7f18dc156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50330/0 (socket says 172.21.15.67:50330) 2024-03-09T09:25:04.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.728+0000 7f18e2600640 1 -- 172.21.15.67:0/617363426 learned_addr learned my addr 172.21.15.67:0/617363426 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:04.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.729+0000 7f18e1c00640 1 --2- 172.21.15.67:0/617363426 >> v2:172.21.15.67:3300/0 conn(0x7f18dc153300 0x7f18dc0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:04.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.729+0000 7f18e1200640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc0c5e40 0x7f18dc0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:04.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.729+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972914165 0 0) 0x7f18dc157000 con 0x7f18dc0cadb0 2024-03-09T09:25:04.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.729+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c4003520 con 0x7f18dc0cadb0 2024-03-09T09:25:04.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.729+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3678490870 0 0) 0x7f18c4003520 con 0x7f18dc0cadb0 2024-03-09T09:25:04.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.729+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc0c5e40 msgr2=0x7f18dc0b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:04.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.729+0000 7f18dbe00640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc0c5e40 0x7f18dc0b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:04.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.729+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 >> v2:172.21.15.67:3300/0 conn(0x7f18dc153300 msgr2=0x7f18dc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:04.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.729+0000 7f18dbe00640 1 --2- 172.21.15.67:0/617363426 >> v2:172.21.15.67:3300/0 conn(0x7f18dc153300 0x7f18dc0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.729+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc155b10 con 0x7f18dc0cadb0 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.729+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f18d40019f0 con 0x7f18dc0cadb0 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 >> v1:172.21.15.67:6789/0 conn(0x7f18dc0cadb0 legacy=0x7f18dc156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18dbe00640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18c4003a20 0x7f18c4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18dbe00640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc0c5e40 0x7f18c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18dbe00640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18c4004d40 0x7f18c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0ae430 con 0x7f18c4004d40 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0ae6f0 con 0x7f18c4003a20 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c400b9c0 con 0x7f18dc0c5e40 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18e1c00640 1 -- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18c4003a20 msgr2=0x7f18c4003e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18e1c00640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18c4003a20 0x7f18c4003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:04.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18e2600640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18c4004d40 0x7f18c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18e2600640 1 -- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc0c5e40 msgr2=0x7f18c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18e1200640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc0c5e40 0x7f18c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18e2600640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc0c5e40 0x7f18c4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18e2600640 1 -- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18c4003a20 msgr2=0x7f18c4003e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18e2600640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18c4003a20 0x7f18c4003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.730+0000 7f18e2600640 1 -- 172.21.15.67:0/617363426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c4003520 con 0x7f18c4004d40 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.732+0000 7f18e2600640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18c4004d40 0x7f18c40051c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18d400a920 tx=0x7f18c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988f4b416cf0ad77 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.732+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d400a4c0 con 0x7f18c4004d40 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.732+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d4034630 con 0x7f18c4004d40 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.732+0000 7f18dbe00640 1 -- 172.21.15.67:0/617363426 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4034a20 con 0x7f18c4004d40 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.732+0000 7f18e3000640 1 -- 172.21.15.67:0/617363426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18c4004d40 msgr2=0x7f18c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:04.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.732+0000 7f18e3000640 1 --2- 172.21.15.67:0/617363426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18c4004d40 0x7f18c40051c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18d400a920 tx=0x7f18c4005830 comp rx=0 tx=0).stop 2024-03-09T09:25:04.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.733+0000 7f18e2600640 1 -- 172.21.15.67:0/617363426 reap_dead start 2024-03-09T09:25:04.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.733+0000 7f18e3000640 1 -- 172.21.15.67:0/617363426 shutdown_connections 2024-03-09T09:25:04.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.733+0000 7f18e3000640 1 -- 172.21.15.67:0/617363426 >> 172.21.15.67:0/617363426 conn(0x7f18dc0732d0 msgr2=0x7f18dc0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:04.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.733+0000 7f18e3000640 1 -- 172.21.15.67:0/617363426 shutdown_connections 2024-03-09T09:25:04.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.733+0000 7f18e3000640 1 -- 172.21.15.67:0/617363426 wait complete. 2024-03-09T09:25:04.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.734+0000 7f18e3000640 1 Processor -- start 2024-03-09T09:25:04.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.735+0000 7f18e3000640 1 -- start start 2024-03-09T09:25:04.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.735+0000 7f18e3000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18c4004d40 0x7f18dc06c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:04.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.735+0000 7f18e3000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc157490 0x7f18dc14bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:04.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.735+0000 7f18e3000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18dc14c230 0x7f18dc1eb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:04.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e3000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0c59e0 con 0x7f18dc14c230 2024-03-09T09:25:04.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e3000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc159e90 con 0x7f18c4004d40 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e3000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0b0540 con 0x7f18dc157490 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e1c00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18c4004d40 msgr2=0x7f18dc06c320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18c4004d40 0x7f18dc06c320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18dc14c230 0x7f18dc1eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18dc14c230 0x7f18dc1eb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38022/0 (socket says 172.21.15.67:38022) 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e2600640 1 -- 172.21.15.67:0/3465419403 learned_addr learned my addr 172.21.15.67:0/3465419403 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e1200640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc157490 0x7f18dc14bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e2600640 1 -- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc157490 msgr2=0x7f18dc14bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e2600640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc157490 0x7f18dc14bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e2600640 1 -- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18c4004d40 msgr2=0x7f18dc06c320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e2600640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18c4004d40 0x7f18dc06c320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.736+0000 7f18e2600640 1 -- 172.21.15.67:0/3465419403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0acf90 con 0x7f18dc14c230 2024-03-09T09:25:04.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.737+0000 7f18e2600640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18dc14c230 0x7f18dc1eb190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18d400aa10 tx=0x7f18dc1536e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:04.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.737+0000 7f18d9600640 1 -- 172.21.15.67:0/3465419403 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4045070 con 0x7f18dc14c230 2024-03-09T09:25:04.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.737+0000 7f18d9600640 1 -- 172.21.15.67:0/3465419403 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d403a040 con 0x7f18dc14c230 2024-03-09T09:25:04.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.737+0000 7f18d9600640 1 -- 172.21.15.67:0/3465419403 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d400a4c0 con 0x7f18dc14c230 2024-03-09T09:25:04.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.737+0000 7f18e3000640 1 -- 172.21.15.67:0/3465419403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc0ab2e0 con 0x7f18dc14c230 2024-03-09T09:25:04.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.737+0000 7f18e3000640 1 -- 172.21.15.67:0/3465419403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc0ab8a0 con 0x7f18dc14c230 2024-03-09T09:25:04.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.738+0000 7f18e3000640 1 -- 172.21.15.67:0/3465419403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc04f3c0 con 0x7f18dc14c230 2024-03-09T09:25:04.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.739+0000 7f18d9600640 1 -- 172.21.15.67:0/3465419403 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f18d4048070 con 0x7f18dc14c230 2024-03-09T09:25:04.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.741+0000 7f18d9600640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f18bc06b480 0x7f18bc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:04.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.741+0000 7f18d9600640 1 -- 172.21.15.67:0/3465419403 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f18d40ac0f0 con 0x7f18dc14c230 2024-03-09T09:25:04.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.741+0000 7f18e1c00640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f18bc06b480 0x7f18bc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:04.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.742+0000 7f18e1c00640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f18bc06b480 0x7f18bc06d940 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f18d00094a0 tx=0x7f18d0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:04.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:04.745+0000 7f18d9600640 1 -- 172.21.15.67:0/3465419403 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f18dc0acf90 con 0x7f18dc14c230 2024-03-09T09:25:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.141+0000 7f18e3000640 1 -- 172.21.15.67:0/3465419403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f18dc157000 con 0x7f18dc14c230 2024-03-09T09:25:05.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.142+0000 7f18d9600640 1 -- 172.21.15.67:0/3465419403 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f18d407a700 con 0x7f18dc14c230 2024-03-09T09:25:05.149 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":256,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:25:05.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 -- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f18bc06b480 msgr2=0x7f18bc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:05.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f18bc06b480 0x7f18bc06d940 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f18d00094a0 tx=0x7f18d0009190 comp rx=0 tx=0).stop 2024-03-09T09:25:05.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 -- 172.21.15.67:0/3465419403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18dc14c230 msgr2=0x7f18dc1eb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:05.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18dc14c230 0x7f18dc1eb190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18d400aa10 tx=0x7f18dc1536e0 comp rx=0 tx=0).stop 2024-03-09T09:25:05.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 -- 172.21.15.67:0/3465419403 shutdown_connections 2024-03-09T09:25:05.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f18bc06b480 0x7f18bc06d940 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:05.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f18dc14c230 0x7f18dc1eb190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:05.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18dc157490 0x7f18dc14bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:05.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 --2- 172.21.15.67:0/3465419403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18c4004d40 0x7f18dc06c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:05.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 -- 172.21.15.67:0/3465419403 >> 172.21.15.67:0/3465419403 conn(0x7f18dc0732d0 msgr2=0x7f18dc158220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:05.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 -- 172.21.15.67:0/3465419403 shutdown_connections 2024-03-09T09:25:05.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:05.145+0000 7f18e3000640 1 -- 172.21.15.67:0/3465419403 wait complete. 2024-03-09T09:25:08.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:25:08.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.375+0000 7f4e46200640 1 Processor -- start 2024-03-09T09:25:08.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.376+0000 7f4e46200640 1 -- start start 2024-03-09T09:25:08.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.378+0000 7f4e46200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40153300 0x7f4e400b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:08.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.379+0000 7f4e46200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e400c5e40 0x7f4e400b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:08.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.379+0000 7f4e46200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e40157000 con 0x7f4e400cadb0 2024-03-09T09:25:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.379+0000 7f4e46200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e400ae430 con 0x7f4e400c5e40 2024-03-09T09:25:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.379+0000 7f4e46200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400ae6f0 con 0x7f4e40153300 2024-03-09T09:25:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.379+0000 7f4e3fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e400c5e40 0x7f4e400b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.379+0000 7f4e45800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4e400cadb0 0x7f4e40156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50346/0 (socket says 172.21.15.67:50346) 2024-03-09T09:25:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.379+0000 7f4e45800640 1 -- 172.21.15.67:0/221062076 learned_addr learned my addr 172.21.15.67:0/221062076 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.379+0000 7f4e3fe00640 1 --2- 172.21.15.67:0/221062076 >> v2:172.21.15.67:3300/0 conn(0x7f4e400c5e40 0x7f4e400b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38030/0 (socket says 172.21.15.67:38030) 2024-03-09T09:25:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.379+0000 7f4e44e00640 1 --2- 172.21.15.67:0/221062076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40153300 0x7f4e400b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.379+0000 7f4e3fe00640 1 -- 172.21.15.67:0/221062076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40153300 msgr2=0x7f4e400b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.380+0000 7f4e3fe00640 1 --2- 172.21.15.67:0/221062076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40153300 0x7f4e400b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.380+0000 7f4e3fe00640 1 -- 172.21.15.67:0/221062076 >> v1:172.21.15.67:6789/0 conn(0x7f4e400cadb0 legacy=0x7f4e40156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.380+0000 7f4e3fe00640 1 -- 172.21.15.67:0/221062076 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40155b10 con 0x7f4e400c5e40 2024-03-09T09:25:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.381+0000 7f4e3fe00640 1 --2- 172.21.15.67:0/221062076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e400c5e40 0x7f4e400b0540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0097a0 tx=0x7f4e2c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe6fca82c139e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.381+0000 7f4e3f400640 1 -- 172.21.15.67:0/221062076 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c035020 con 0x7f4e400c5e40 2024-03-09T09:25:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.381+0000 7f4e3f400640 1 -- 172.21.15.67:0/221062076 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2c031810 con 0x7f4e400c5e40 2024-03-09T09:25:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.381+0000 7f4e3f400640 1 -- 172.21.15.67:0/221062076 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c031be0 con 0x7f4e400c5e40 2024-03-09T09:25:08.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.382+0000 7f4e46200640 1 -- 172.21.15.67:0/221062076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e400c5e40 msgr2=0x7f4e400b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:08.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.382+0000 7f4e46200640 1 --2- 172.21.15.67:0/221062076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e400c5e40 0x7f4e400b0540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0097a0 tx=0x7f4e2c02b8b0 comp rx=0 tx=0).stop 2024-03-09T09:25:08.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.382+0000 7f4e46200640 1 -- 172.21.15.67:0/221062076 shutdown_connections 2024-03-09T09:25:08.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.382+0000 7f4e46200640 1 --2- 172.21.15.67:0/221062076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e400c5e40 0x7f4e400b0540 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:08.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.382+0000 7f4e46200640 1 --2- 172.21.15.67:0/221062076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e40153300 0x7f4e400b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:08.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.382+0000 7f4e46200640 1 -- 172.21.15.67:0/221062076 >> 172.21.15.67:0/221062076 conn(0x7f4e400732d0 msgr2=0x7f4e400ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:08.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.382+0000 7f4e46200640 1 -- 172.21.15.67:0/221062076 shutdown_connections 2024-03-09T09:25:08.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.382+0000 7f4e46200640 1 -- 172.21.15.67:0/221062076 wait complete. 2024-03-09T09:25:08.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.384+0000 7f4e46200640 1 Processor -- start 2024-03-09T09:25:08.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.384+0000 7f4e46200640 1 -- start start 2024-03-09T09:25:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e46200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e400c5e40 0x7f4e401ef760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e46200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e400cadb0 0x7f4e4014b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e46200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e40153300 0x7f4e4014de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e46200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e400c59e0 con 0x7f4e400c5e40 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e46200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e40159e90 con 0x7f4e40153300 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e46200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e40156720 con 0x7f4e400cadb0 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e45800640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e40153300 msgr2=0x7f4e4014de60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e44e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e400c5e40 0x7f4e401ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e45800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e40153300 0x7f4e4014de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e44e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e400c5e40 0x7f4e401ef760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38040/0 (socket says 172.21.15.67:38040) 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e44e00640 1 -- 172.21.15.67:0/2921941475 learned_addr learned my addr 172.21.15.67:0/2921941475 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e3fe00640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e400cadb0 0x7f4e4014b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.385+0000 7f4e44e00640 1 -- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e400cadb0 msgr2=0x7f4e4014b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.386+0000 7f4e44e00640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e400cadb0 0x7f4e4014b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.386+0000 7f4e44e00640 1 -- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e40153300 msgr2=0x7f4e4014de60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.386+0000 7f4e44e00640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e40153300 0x7f4e4014de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.386+0000 7f4e44e00640 1 -- 172.21.15.67:0/2921941475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e400ab2e0 con 0x7f4e400c5e40 2024-03-09T09:25:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.386+0000 7f4e3fe00640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e400cadb0 0x7f4e4014b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:25:08.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.386+0000 7f4e44e00640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e400c5e40 0x7f4e401ef760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e30007e40 tx=0x7f4e3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:08.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.386+0000 7f4e3cc00640 1 -- 172.21.15.67:0/2921941475 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30014620 con 0x7f4e400c5e40 2024-03-09T09:25:08.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.386+0000 7f4e3cc00640 1 -- 172.21.15.67:0/2921941475 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e30019070 con 0x7f4e400c5e40 2024-03-09T09:25:08.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.386+0000 7f4e3cc00640 1 -- 172.21.15.67:0/2921941475 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30014a00 con 0x7f4e400c5e40 2024-03-09T09:25:08.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.386+0000 7f4e46200640 1 -- 172.21.15.67:0/2921941475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e400ab5c0 con 0x7f4e400c5e40 2024-03-09T09:25:08.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.386+0000 7f4e46200640 1 -- 172.21.15.67:0/2921941475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e400abb80 con 0x7f4e400c5e40 2024-03-09T09:25:08.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.388+0000 7f4e46200640 1 -- 172.21.15.67:0/2921941475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4004f3c0 con 0x7f4e400c5e40 2024-03-09T09:25:08.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.389+0000 7f4e3cc00640 1 -- 172.21.15.67:0/2921941475 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4e30014ba0 con 0x7f4e400c5e40 2024-03-09T09:25:08.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.391+0000 7f4e3cc00640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e2006b480 0x7f4e2006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:08.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.391+0000 7f4e3cc00640 1 -- 172.21.15.67:0/2921941475 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7f4e3008ae50 con 0x7f4e400c5e40 2024-03-09T09:25:08.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.391+0000 7f4e3fe00640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e2006b480 0x7f4e2006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:08.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.392+0000 7f4e3fe00640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e2006b480 0x7f4e2006d940 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c036000 tx=0x7f4e2c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:08.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.396+0000 7f4e3cc00640 1 -- 172.21.15.67:0/2921941475 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4e300598d0 con 0x7f4e400c5e40 2024-03-09T09:25:08.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.672+0000 7f4e46200640 1 -- 172.21.15.67:0/2921941475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4e400cab90 con 0x7f4e400c5e40 2024-03-09T09:25:08.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.673+0000 7f4e3cc00640 1 -- 172.21.15.67:0/2921941475 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f4e300591d0 con 0x7f4e400c5e40 2024-03-09T09:25:08.679 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":256,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:25:08.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.675+0000 7f4e46200640 1 -- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e2006b480 msgr2=0x7f4e2006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:08.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.675+0000 7f4e46200640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e2006b480 0x7f4e2006d940 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c036000 tx=0x7f4e2c008030 comp rx=0 tx=0).stop 2024-03-09T09:25:08.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.675+0000 7f4e46200640 1 -- 172.21.15.67:0/2921941475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e400c5e40 msgr2=0x7f4e401ef760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.675+0000 7f4e46200640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e400c5e40 0x7f4e401ef760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e30007e40 tx=0x7f4e3000e4b0 comp rx=0 tx=0).stop 2024-03-09T09:25:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.675+0000 7f4e46200640 1 -- 172.21.15.67:0/2921941475 shutdown_connections 2024-03-09T09:25:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.675+0000 7f4e46200640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e2006b480 0x7f4e2006d940 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.675+0000 7f4e46200640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e40153300 0x7f4e4014de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.675+0000 7f4e46200640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e400cadb0 0x7f4e4014b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.675+0000 7f4e46200640 1 --2- 172.21.15.67:0/2921941475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e400c5e40 0x7f4e401ef760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.675+0000 7f4e46200640 1 -- 172.21.15.67:0/2921941475 >> 172.21.15.67:0/2921941475 conn(0x7f4e400732d0 msgr2=0x7f4e40151c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.676+0000 7f4e46200640 1 -- 172.21.15.67:0/2921941475 shutdown_connections 2024-03-09T09:25:08.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:08.676+0000 7f4e46200640 1 -- 172.21.15.67:0/2921941475 wait complete. 2024-03-09T09:25:09.163 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:25:09.164 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:09.164 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 86 -eq 48 ']' 2024-03-09T09:25:09.164 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.86 2024-03-09T09:25:09.164 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.86 2024-03-09T09:25:09.165 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.86 2024-03-09T09:25:09.165 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.86 2024-03-09T09:25:09.168 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:25:11.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:25:11.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.913+0000 7fb613600640 1 Processor -- start 2024-03-09T09:25:11.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.913+0000 7fb613600640 1 -- start start 2024-03-09T09:25:11.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.914+0000 7fb613600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:11.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.914+0000 7fb613600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60c0c9380 con 0x7fb60c0c8fa0 2024-03-09T09:25:11.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.914+0000 7fb613600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60c153d00 con 0x7fb60c0705c0 2024-03-09T09:25:11.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.914+0000 7fb613600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c153fe0 con 0x7fb60c071090 2024-03-09T09:25:11.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.915+0000 7fb610c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb60c0c8fa0 0x7fb60c0c6480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33120/0 (socket says 172.21.15.67:33120) 2024-03-09T09:25:11.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.915+0000 7fb610c00640 1 -- 172.21.15.67:0/1399652309 learned_addr learned my addr 172.21.15.67:0/1399652309 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:11.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.915+0000 7fb611600640 1 -- 172.21.15.67:0/1399652309 >> v1:172.21.15.134:6789/0 conn(0x7fb60c0705c0 legacy=0x7fb60c157770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:25:11.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.915+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:11.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.915+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 msgr2=0x7fb60c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:25:11.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.915+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 msgr2=0x7fb60c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:25:11.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.915+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:25:11.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:11.915+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:12.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.116+0000 7fb611600640 1 -- 172.21.15.67:0/1399652309 >> v1:172.21.15.134:6789/0 conn(0x7fb60c0705c0 legacy=0x7fb60c157770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:25:12.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.116+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.117+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 msgr2=0x7fb60c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:25:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.117+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 msgr2=0x7fb60c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:25:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.117+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:25:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.117+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:25:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.517+0000 7fb611600640 1 -- 172.21.15.67:0/1399652309 >> v1:172.21.15.134:6789/0 conn(0x7fb60c0705c0 legacy=0x7fb60c157770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:25:12.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.518+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.519+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 msgr2=0x7fb60c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:25:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.519+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 msgr2=0x7fb60c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:25:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.519+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:25:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:12.519+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:25:13.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:13.318+0000 7fb611600640 1 -- 172.21.15.67:0/1399652309 >> v1:172.21.15.134:6789/0 conn(0x7fb60c0705c0 legacy=0x7fb60c157770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:25:13.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:13.321+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:13.321+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 msgr2=0x7fb60c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:25:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:13.321+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 msgr2=0x7fb60c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:25:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:13.321+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:25:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:13.321+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:25:14.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.913+0000 7fb60a000640 1 -- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 msgr2=0x7fb60c157230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:25:14.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.913+0000 7fb60a000640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c071090 0x7fb60c157230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:14.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.913+0000 7fb60a000640 1 -- 172.21.15.67:0/1399652309 >> v1:172.21.15.134:6789/0 conn(0x7fb60c0705c0 legacy=0x7fb60c157770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:25:14.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.913+0000 7fb60a000640 1 -- 172.21.15.67:0/1399652309 >> v1:172.21.15.67:6789/0 conn(0x7fb60c0c8fa0 legacy=0x7fb60c0c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:14.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.914+0000 7fb60a000640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.134:3300/0 conn(0x7fb5ec0036d0 0x7fb5ec005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:14.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.914+0000 7fb60a000640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:14.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.914+0000 7fb60a000640 1 -- 172.21.15.67:0/1399652309 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60c153fe0 con 0x7fb5ec0061a0 2024-03-09T09:25:14.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.914+0000 7fb60a000640 1 -- 172.21.15.67:0/1399652309 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec008d10 con 0x7fb5ec0036d0 2024-03-09T09:25:14.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.914+0000 7fb610c00640 1 -- 172.21.15.67:0/1399652309 >> v2:172.21.15.134:3300/0 conn(0x7fb5ec0036d0 msgr2=0x7fb5ec005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:14.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.914+0000 7fb610c00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.134:3300/0 conn(0x7fb5ec0036d0 0x7fb5ec005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:14.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.914+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:14.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.915+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 msgr2=0x7fb5ec008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:25:14.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.915+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 msgr2=0x7fb5ec008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:25:14.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.915+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:25:14.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:14.915+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:15.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.115+0000 7fb610c00640 1 -- 172.21.15.67:0/1399652309 >> v2:172.21.15.134:3300/0 conn(0x7fb5ec0036d0 msgr2=0x7fb5ec005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:15.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.115+0000 7fb610c00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.134:3300/0 conn(0x7fb5ec0036d0 0x7fb5ec005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:25:15.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.116+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:15.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.117+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 msgr2=0x7fb5ec008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:25:15.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.117+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 msgr2=0x7fb5ec008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:25:15.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.117+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:25:15.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.117+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:25:15.340 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:25:15.341 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:15.341 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 87 -eq 48 ']' 2024-03-09T09:25:15.341 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.87 2024-03-09T09:25:15.341 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.87 2024-03-09T09:25:15.341 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.87 2024-03-09T09:25:15.341 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.87 2024-03-09T09:25:15.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.516+0000 7fb610c00640 1 -- 172.21.15.67:0/1399652309 >> v2:172.21.15.134:3300/0 conn(0x7fb5ec0036d0 msgr2=0x7fb5ec005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:15.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.516+0000 7fb610c00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.134:3300/0 conn(0x7fb5ec0036d0 0x7fb5ec005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:25:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.518+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.519+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 msgr2=0x7fb5ec008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:25:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.519+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 msgr2=0x7fb5ec008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:25:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.519+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:25:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:15.519+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:25:16.212 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:25:16.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.317+0000 7fb610c00640 1 -- 172.21.15.67:0/1399652309 >> v2:172.21.15.134:3300/0 conn(0x7fb5ec0036d0 msgr2=0x7fb5ec005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:16.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.317+0000 7fb610c00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.134:3300/0 conn(0x7fb5ec0036d0 0x7fb5ec005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:25:16.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.320+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.67:3300/0 conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:16.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.322+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 >> v2:172.21.15.134:3300/0 conn(0x7fb5ec0036d0 msgr2=0x7fb5ec005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:25:16.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.322+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> v2:172.21.15.134:3300/0 conn(0x7fb5ec0036d0 0x7fb5ec005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:16.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.322+0000 7fb60be00640 1 -- 172.21.15.67:0/1399652309 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c1542c0 con 0x7fb5ec0061a0 2024-03-09T09:25:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.326+0000 7fb60be00640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5ec0061a0 0x7fb5ec008660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec003660 tx=0x7fb5f802d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3ceb8fca292512 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.327+0000 7fb60b400640 1 -- 172.21.15.67:0/1399652309 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8037420 con 0x7fb5ec0061a0 2024-03-09T09:25:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.327+0000 7fb60b400640 1 -- 172.21.15.67:0/1399652309 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f80375c0 con 0x7fb5ec0061a0 2024-03-09T09:25:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.327+0000 7fb60b400640 1 -- 172.21.15.67:0/1399652309 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8037770 con 0x7fb5ec0061a0 2024-03-09T09:25:16.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.328+0000 7fb613600640 1 -- 172.21.15.67:0/1399652309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5ec0061a0 msgr2=0x7fb5ec008660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:16.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.328+0000 7fb613600640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5ec0061a0 0x7fb5ec008660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec003660 tx=0x7fb5f802d6c0 comp rx=0 tx=0).stop 2024-03-09T09:25:16.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.329+0000 7fb611600640 1 -- 172.21.15.67:0/1399652309 reap_dead start 2024-03-09T09:25:16.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.329+0000 7fb613600640 1 -- 172.21.15.67:0/1399652309 shutdown_connections 2024-03-09T09:25:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.329+0000 7fb613600640 1 --2- 172.21.15.67:0/1399652309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb5ec0061a0 0x7fb5ec008660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.329+0000 7fb613600640 1 -- 172.21.15.67:0/1399652309 >> 172.21.15.67:0/1399652309 conn(0x7fb60c066f70 msgr2=0x7fb60c067350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.329+0000 7fb613600640 1 -- 172.21.15.67:0/1399652309 shutdown_connections 2024-03-09T09:25:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.329+0000 7fb613600640 1 -- 172.21.15.67:0/1399652309 wait complete. 2024-03-09T09:25:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.331+0000 7fb613600640 1 Processor -- start 2024-03-09T09:25:16.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.332+0000 7fb613600640 1 -- start start 2024-03-09T09:25:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.333+0000 7fb613600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb60c150c50 0x7fb60c1ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.333+0000 7fb613600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c14e710 0x7fb60c14eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.333+0000 7fb613600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb60c14f0d0 0x7fb60c1490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.333+0000 7fb613600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0c6b90 con 0x7fb60c14f0d0 2024-03-09T09:25:16.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.333+0000 7fb613600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c157e80 con 0x7fb60c150c50 2024-03-09T09:25:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.333+0000 7fb613600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0c6d10 con 0x7fb60c14e710 2024-03-09T09:25:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.333+0000 7fb610c00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb60c150c50 msgr2=0x7fb60c1ec1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.333+0000 7fb610c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb60c150c50 0x7fb60c1ec1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.333+0000 7fb611600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb60c14f0d0 0x7fb60c1490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.334+0000 7fb60be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c14e710 0x7fb60c14eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.334+0000 7fb60be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c14e710 0x7fb60c14eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:52794/0 (socket says 172.21.15.67:52794) 2024-03-09T09:25:16.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.334+0000 7fb60be00640 1 -- 172.21.15.67:0/2663262383 learned_addr learned my addr 172.21.15.67:0/2663262383 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.335+0000 7fb611600640 1 -- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c14e710 msgr2=0x7fb60c14eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.335+0000 7fb611600640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c14e710 0x7fb60c14eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.335+0000 7fb611600640 1 -- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb60c150c50 msgr2=0x7fb60c1ec1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:25:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.335+0000 7fb611600640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb60c150c50 0x7fb60c1ec1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.335+0000 7fb611600640 1 -- 172.21.15.67:0/2663262383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c1e1ad0 con 0x7fb60c14f0d0 2024-03-09T09:25:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.335+0000 7fb60be00640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c14e710 0x7fb60c14eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:25:16.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.335+0000 7fb611600640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb60c14f0d0 0x7fb60c1490e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0c3cc0 tx=0x7fb60400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.336+0000 7fb608c00640 1 -- 172.21.15.67:0/2663262383 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60400cbd0 con 0x7fb60c14f0d0 2024-03-09T09:25:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.336+0000 7fb608c00640 1 -- 172.21.15.67:0/2663262383 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb604010020 con 0x7fb60c14f0d0 2024-03-09T09:25:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.336+0000 7fb608c00640 1 -- 172.21.15.67:0/2663262383 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604013320 con 0x7fb60c14f0d0 2024-03-09T09:25:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.336+0000 7fb613600640 1 -- 172.21.15.67:0/2663262383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c1e1db0 con 0x7fb60c14f0d0 2024-03-09T09:25:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.336+0000 7fb613600640 1 -- 172.21.15.67:0/2663262383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c14fdc0 con 0x7fb60c14f0d0 2024-03-09T09:25:16.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.339+0000 7fb613600640 1 -- 172.21.15.67:0/2663262383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c075a00 con 0x7fb60c14f0d0 2024-03-09T09:25:16.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.342+0000 7fb608c00640 1 -- 172.21.15.67:0/2663262383 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb6040134c0 con 0x7fb60c14f0d0 2024-03-09T09:25:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.344+0000 7fb608c00640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb5ec06c1f0 0x7fb5ec06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:16.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.344+0000 7fb608c00640 1 -- 172.21.15.67:0/2663262383 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6709+0+0 (secure 0 0 0) 0x7fb604089d90 con 0x7fb60c14f0d0 2024-03-09T09:25:16.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.345+0000 7fb610c00640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb5ec06c1f0 0x7fb5ec06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:16.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.346+0000 7fb610c00640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb5ec06c1f0 0x7fb5ec06e6b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb600005880 tx=0x7fb600007a10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:16.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.350+0000 7fb608c00640 1 -- 172.21.15.67:0/2663262383 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb60401a030 con 0x7fb60c14f0d0 2024-03-09T09:25:16.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.646+0000 7fb613600640 1 -- 172.21.15.67:0/2663262383 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb60c151ef0 con 0x7fb60c14f0d0 2024-03-09T09:25:16.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.647+0000 7fb608c00640 1 -- 172.21.15.67:0/2663262383 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fb604058820 con 0x7fb60c14f0d0 2024-03-09T09:25:16.654 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":260,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:25:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 -- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb5ec06c1f0 msgr2=0x7fb5ec06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb5ec06c1f0 0x7fb5ec06e6b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb600005880 tx=0x7fb600007a10 comp rx=0 tx=0).stop 2024-03-09T09:25:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 -- 172.21.15.67:0/2663262383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb60c14f0d0 msgr2=0x7fb60c1490e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb60c14f0d0 0x7fb60c1490e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0c3cc0 tx=0x7fb60400be60 comp rx=0 tx=0).stop 2024-03-09T09:25:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 -- 172.21.15.67:0/2663262383 shutdown_connections 2024-03-09T09:25:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb5ec06c1f0 0x7fb5ec06e6b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb60c14f0d0 0x7fb60c1490e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb60c14e710 0x7fb60c14eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 --2- 172.21.15.67:0/2663262383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb60c150c50 0x7fb60c1ec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 -- 172.21.15.67:0/2663262383 >> 172.21.15.67:0/2663262383 conn(0x7fb60c066f70 msgr2=0x7fb60c0c7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 -- 172.21.15.67:0/2663262383 shutdown_connections 2024-03-09T09:25:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.650+0000 7fb613600640 1 -- 172.21.15.67:0/2663262383 wait complete. 2024-03-09T09:25:16.678 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:25:16.678 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:25:16.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:25:17.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.992+0000 7f65e4600640 1 Processor -- start 2024-03-09T09:25:17.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.993+0000 7f65e4600640 1 -- start start 2024-03-09T09:25:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.994+0000 7f65e4600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f65dc0729c0 0x7f65dc072da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.994+0000 7f65e4600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f65dc0c4450 0x7f65dc0c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.994+0000 7f65e4600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65dc073af0 con 0x7f65dc0c3f70 2024-03-09T09:25:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.994+0000 7f65e4600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65dc0c5780 con 0x7f65dc0729c0 2024-03-09T09:25:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.994+0000 7f65e4600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65dc0c9020 con 0x7f65dc0c4450 2024-03-09T09:25:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.995+0000 7f65e2600640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f65dc0c4450 msgr2=0x7f65dc0c5240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.995+0000 7f65e2600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f65dc0c4450 0x7f65dc0c5240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.995+0000 7f65e1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f65dc0729c0 0x7f65dc072da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.995+0000 7f65e1c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f65dc0729c0 0x7f65dc072da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60970/0 (socket says 172.21.15.67:60970) 2024-03-09T09:25:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.995+0000 7f65e1c00640 1 -- 172.21.15.67:0/3736040929 learned_addr learned my addr 172.21.15.67:0/3736040929 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.995+0000 7f65e1200640 1 -- 172.21.15.67:0/3736040929 >> v1:172.21.15.134:6789/0 conn(0x7f65dc0c3f70 legacy=0x7f65dc153d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:25:17.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.996+0000 7f65e1c00640 1 -- 172.21.15.67:0/3736040929 >> v2:172.21.15.134:3300/0 conn(0x7f65dc0c4450 msgr2=0x7f65dc0c5240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:25:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.996+0000 7f65e1c00640 1 --2- 172.21.15.67:0/3736040929 >> v2:172.21.15.134:3300/0 conn(0x7f65dc0c4450 0x7f65dc0c5240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.996+0000 7f65e1c00640 1 -- 172.21.15.67:0/3736040929 >> v1:172.21.15.134:6789/0 conn(0x7f65dc0c3f70 legacy=0x7f65dc153d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:25:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:16.996+0000 7f65e1c00640 1 -- 172.21.15.67:0/3736040929 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc0c4a40 con 0x7f65dc0729c0 2024-03-09T09:25:17.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.001+0000 7f65e1c00640 1 --2- 172.21.15.67:0/3736040929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc0729c0 0x7f65dc072da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f65dc06bae0 tx=0x7f65d002be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d99d924089ddf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:17.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.003+0000 7f65dbe00640 1 -- 172.21.15.67:0/3736040929 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d002ca10 con 0x7f65dc0729c0 2024-03-09T09:25:17.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.003+0000 7f65dbe00640 1 -- 172.21.15.67:0/3736040929 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d002cbb0 con 0x7f65dc0729c0 2024-03-09T09:25:17.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.003+0000 7f65dbe00640 1 -- 172.21.15.67:0/3736040929 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d00343f0 con 0x7f65dc0729c0 2024-03-09T09:25:17.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.004+0000 7f65e4600640 1 -- 172.21.15.67:0/3736040929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc0729c0 msgr2=0x7f65dc072da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.004+0000 7f65e4600640 1 --2- 172.21.15.67:0/3736040929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc0729c0 0x7f65dc072da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f65dc06bae0 tx=0x7f65d002be70 comp rx=0 tx=0).stop 2024-03-09T09:25:17.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.004+0000 7f65e4600640 1 -- 172.21.15.67:0/3736040929 shutdown_connections 2024-03-09T09:25:17.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.004+0000 7f65e4600640 1 --2- 172.21.15.67:0/3736040929 >> v2:172.21.15.134:3300/0 conn(0x7f65dc0c4450 0x7f65dc0c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.004+0000 7f65e4600640 1 --2- 172.21.15.67:0/3736040929 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc0729c0 0x7f65dc072da0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.004+0000 7f65e4600640 1 -- 172.21.15.67:0/3736040929 >> 172.21.15.67:0/3736040929 conn(0x7f65dc0ab320 msgr2=0x7f65dc0ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:17.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.004+0000 7f65e4600640 1 -- 172.21.15.67:0/3736040929 shutdown_connections 2024-03-09T09:25:17.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.004+0000 7f65e4600640 1 -- 172.21.15.67:0/3736040929 wait complete. 2024-03-09T09:25:17.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.008+0000 7f65e4600640 1 Processor -- start 2024-03-09T09:25:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.008+0000 7f65e4600640 1 -- start start 2024-03-09T09:25:17.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.010+0000 7f65e4600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc0729c0 0x7f65dc152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.010+0000 7f65e4600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65dc0c3f70 0x7f65dc161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.010+0000 7f65e4600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65dc0c4450 0x7f65dc161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.011+0000 7f65e4600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc0c9300 con 0x7f65dc0729c0 2024-03-09T09:25:17.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.011+0000 7f65e4600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc073dd0 con 0x7f65dc0c3f70 2024-03-09T09:25:17.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.011+0000 7f65e4600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc070250 con 0x7f65dc0c4450 2024-03-09T09:25:17.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.011+0000 7f65e1200640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65dc0c3f70 msgr2=0x7f65dc161ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:17.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.011+0000 7f65e1200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65dc0c3f70 0x7f65dc161ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:17.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.011+0000 7f65e2600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65dc0c4450 0x7f65dc161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.012+0000 7f65e2600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65dc0c4450 0x7f65dc161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:52804/0 (socket says 172.21.15.67:52804) 2024-03-09T09:25:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.012+0000 7f65e2600640 1 -- 172.21.15.67:0/3439493575 learned_addr learned my addr 172.21.15.67:0/3439493575 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:17.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.012+0000 7f65e1c00640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc0729c0 0x7f65dc152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.012+0000 7f65e2600640 1 -- 172.21.15.67:0/3439493575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65dc0c3f70 msgr2=0x7f65dc161ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:25:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.012+0000 7f65e2600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65dc0c3f70 0x7f65dc161ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.012+0000 7f65e2600640 1 -- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc0729c0 msgr2=0x7f65dc152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.012+0000 7f65e2600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc0729c0 0x7f65dc152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.012+0000 7f65e2600640 1 -- 172.21.15.67:0/3439493575 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc153f00 con 0x7f65dc0c4450 2024-03-09T09:25:17.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.014+0000 7f65e2600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65dc0c4450 0x7f65dc161ff0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f65d400b390 tx=0x7f65d400b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.014+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4017070 con 0x7f65dc0c4450 2024-03-09T09:25:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.014+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d4003d90 con 0x7f65dc0c4450 2024-03-09T09:25:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.014+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4006310 con 0x7f65dc0c4450 2024-03-09T09:25:17.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.015+0000 7f65e4600640 1 -- 172.21.15.67:0/3439493575 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc1541e0 con 0x7f65dc0c4450 2024-03-09T09:25:17.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.015+0000 7f65e4600640 1 -- 172.21.15.67:0/3439493575 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc151cb0 con 0x7f65dc0c4450 2024-03-09T09:25:17.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.017+0000 7f65e4600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc153080 0x7f65dc154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.017+0000 7f65e4600640 1 -- 172.21.15.67:0/3439493575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc075a00 con 0x7f65dc153080 2024-03-09T09:25:17.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.018+0000 7f65e1c00640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc153080 0x7f65dc154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.018+0000 7f65e1c00640 1 -- 172.21.15.67:0/3439493575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65dc0738d0 con 0x7f65dc153080 2024-03-09T09:25:17.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.019+0000 7f65e1c00640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc153080 0x7f65dc154ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0c8eb0 tx=0x7f65d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:17.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.020+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f65d401a070 con 0x7f65dc0c4450 2024-03-09T09:25:17.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.022+0000 7f65d9600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f65bc06b4d0 0x7f65bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.022+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f65d4089850 con 0x7f65dc0c4450 2024-03-09T09:25:17.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.023+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0034d80 con 0x7f65dc153080 2024-03-09T09:25:17.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.023+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f65d00333a0 con 0x7f65dc153080 2024-03-09T09:25:17.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.023+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc153080 msgr2=0x7f65dc154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.023+0000 7f65d9600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc153080 0x7f65dc154ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0c8eb0 tx=0x7f65d0004a40 comp rx=0 tx=0).stop 2024-03-09T09:25:17.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.023+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc153080 msgr2=0x7f65dc154ca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:17.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.023+0000 7f65d9600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc153080 0x7f65dc154ca0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0c8eb0 tx=0x7f65d0004a40 comp rx=0 tx=0).stop 2024-03-09T09:25:17.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.023+0000 7f65e1200640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f65bc06b4d0 0x7f65bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.025+0000 7f65e1200640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f65bc06b4d0 0x7f65bc06d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f65c8006fd0 tx=0x7f65c8006d80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:17.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.040+0000 7f65e4600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc1551e0 0x7f65dc1f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.040+0000 7f65e4600640 1 -- 172.21.15.67:0/3439493575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc0c5240 con 0x7f65dc1551e0 2024-03-09T09:25:17.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.040+0000 7f65e1c00640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc1551e0 0x7f65dc1f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.041+0000 7f65e1c00640 1 -- 172.21.15.67:0/3439493575 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f65dc06d5a0 con 0x7f65dc1551e0 2024-03-09T09:25:17.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.041+0000 7f65e1c00640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc1551e0 0x7f65dc1f33a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65d002beb0 tx=0x7f65d0009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:17.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.042+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d00487a0 con 0x7f65dc1551e0 2024-03-09T09:25:17.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.042+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4865 (secure 0 0 0) 0x7f65d004c070 con 0x7f65dc1551e0 2024-03-09T09:25:17.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.042+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc1551e0 msgr2=0x7f65dc1f33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.042+0000 7f65d9600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc1551e0 0x7f65dc1f33a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65d002beb0 tx=0x7f65d0009000 comp rx=0 tx=0).stop 2024-03-09T09:25:17.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.042+0000 7f65d9600640 1 -- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc1551e0 msgr2=0x7f65dc1f33a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:17.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.042+0000 7f65d9600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65dc1551e0 0x7f65dc1f33a0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f65d002beb0 tx=0x7f65d0009000 comp rx=0 tx=0).stop 2024-03-09T09:25:17.051 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:25:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:25:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 260, 2024-03-09T09:25:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:25:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:25:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:25:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:25:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:25:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:25:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:25:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:25:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:25:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:25:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:25:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:25:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:25:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:25:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:25:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:25:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:25:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:25:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:25:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:25:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:25:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:25:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:25:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:25:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:25:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:25:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:25:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:25:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:25:17.070 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:25:17.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.043+0000 7f65e4600640 1 -- 172.21.15.67:0/3439493575 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f65bc06b4d0 msgr2=0x7f65bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.043+0000 7f65e4600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f65bc06b4d0 0x7f65bc06d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f65c8006fd0 tx=0x7f65c8006d80 comp rx=0 tx=0).stop 2024-03-09T09:25:17.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.043+0000 7f65e4600640 1 -- 172.21.15.67:0/3439493575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65dc0c4450 msgr2=0x7f65dc161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.043+0000 7f65e4600640 1 --2- 172.21.15.67:0/3439493575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65dc0c4450 0x7f65dc161ff0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f65d400b390 tx=0x7f65d400b850 comp rx=0 tx=0).stop 2024-03-09T09:25:17.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.043+0000 7f65e2600640 1 -- 172.21.15.67:0/3439493575 reap_dead start 2024-03-09T09:25:17.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.043+0000 7f65e2600640 1 -- 172.21.15.67:0/3439493575 reap_dead start 2024-03-09T09:25:17.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.043+0000 7f65e4600640 1 -- 172.21.15.67:0/3439493575 shutdown_connections 2024-03-09T09:25:17.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.043+0000 7f65e4600640 1 -- 172.21.15.67:0/3439493575 >> 172.21.15.67:0/3439493575 conn(0x7f65dc0ab320 msgr2=0x7f65dc06db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:17.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.043+0000 7f65e4600640 1 -- 172.21.15.67:0/3439493575 shutdown_connections 2024-03-09T09:25:17.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.044+0000 7f65e4600640 1 -- 172.21.15.67:0/3439493575 wait complete. 2024-03-09T09:25:17.076 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:25:17.077 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:25:17.077 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:25:17.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.393+0000 7fe879600640 1 Processor -- start 2024-03-09T09:25:17.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.393+0000 7fe879600640 1 -- start start 2024-03-09T09:25:17.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.395+0000 7fe879600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe8740c1d70 0x7fe87415e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.396+0000 7fe879600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8740aad50 0x7fe8740ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.396+0000 7fe879600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87406d970 con 0x7fe874157820 2024-03-09T09:25:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.396+0000 7fe879600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87406df30 con 0x7fe8740c1d70 2024-03-09T09:25:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.396+0000 7fe879600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87406dc50 con 0x7fe8740aad50 2024-03-09T09:25:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.396+0000 7fe872000640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7fe8740c1d70 msgr2=0x7fe87415e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.396+0000 7fe872000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fe8740c1d70 0x7fe87415e250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.396+0000 7fe872a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe874157820 0x7fe8740ab840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:33128/0 (socket says 172.21.15.67:33128) 2024-03-09T09:25:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.396+0000 7fe872a00640 1 -- 172.21.15.67:0/3435184304 learned_addr learned my addr 172.21.15.67:0/3435184304 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.397+0000 7fe873400640 1 --2- 172.21.15.67:0/3435184304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8740aad50 0x7fe8740ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.397+0000 7fe871600640 1 -- 172.21.15.67:0/3435184304 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548230406 0 0) 0x7fe87406d970 con 0x7fe874157820 2024-03-09T09:25:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.397+0000 7fe871600640 1 -- 172.21.15.67:0/3435184304 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe858003520 con 0x7fe874157820 2024-03-09T09:25:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.397+0000 7fe873400640 1 -- 172.21.15.67:0/3435184304 >> v2:172.21.15.134:3300/0 conn(0x7fe8740c1d70 msgr2=0x7fe87415e250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:25:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.397+0000 7fe873400640 1 --2- 172.21.15.67:0/3435184304 >> v2:172.21.15.134:3300/0 conn(0x7fe8740c1d70 0x7fe87415e250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.397+0000 7fe873400640 1 -- 172.21.15.67:0/3435184304 >> v1:172.21.15.67:6789/0 conn(0x7fe874157820 legacy=0x7fe8740ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.397+0000 7fe873400640 1 -- 172.21.15.67:0/3435184304 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8740aa550 con 0x7fe8740aad50 2024-03-09T09:25:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.400+0000 7fe873400640 1 --2- 172.21.15.67:0/3435184304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8740aad50 0x7fe8740ab1e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8640097a0 tx=0x7fe86402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59fcf4167407335 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.400+0000 7fe871600640 1 -- 172.21.15.67:0/3435184304 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8640315a0 con 0x7fe8740aad50 2024-03-09T09:25:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.401+0000 7fe871600640 1 -- 172.21.15.67:0/3435184304 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe864037070 con 0x7fe8740aad50 2024-03-09T09:25:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.401+0000 7fe871600640 1 -- 172.21.15.67:0/3435184304 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864031940 con 0x7fe8740aad50 2024-03-09T09:25:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.402+0000 7fe879600640 1 -- 172.21.15.67:0/3435184304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8740aad50 msgr2=0x7fe8740ab1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.402+0000 7fe879600640 1 --2- 172.21.15.67:0/3435184304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8740aad50 0x7fe8740ab1e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8640097a0 tx=0x7fe86402b840 comp rx=0 tx=0).stop 2024-03-09T09:25:17.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.406+0000 7fe879600640 1 -- 172.21.15.67:0/3435184304 shutdown_connections 2024-03-09T09:25:17.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.406+0000 7fe879600640 1 --2- 172.21.15.67:0/3435184304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8740aad50 0x7fe8740ab1e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.406+0000 7fe879600640 1 --2- 172.21.15.67:0/3435184304 >> v2:172.21.15.134:3300/0 conn(0x7fe8740c1d70 0x7fe87415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.406+0000 7fe879600640 1 -- 172.21.15.67:0/3435184304 >> 172.21.15.67:0/3435184304 conn(0x7fe8740c7370 msgr2=0x7fe8740c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.407+0000 7fe879600640 1 -- 172.21.15.67:0/3435184304 shutdown_connections 2024-03-09T09:25:17.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.407+0000 7fe879600640 1 -- 172.21.15.67:0/3435184304 wait complete. 2024-03-09T09:25:17.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.410+0000 7fe879600640 1 Processor -- start 2024-03-09T09:25:17.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.410+0000 7fe879600640 1 -- start start 2024-03-09T09:25:17.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.414+0000 7fe879600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8740aad50 0x7fe87415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.415+0000 7fe879600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8740c1d70 0x7fe874155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.415+0000 7fe872a00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8740aad50 msgr2=0x7fe87415a080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:25:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.415+0000 7fe872a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8740aad50 0x7fe87415a080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.415+0000 7fe879600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874157820 0x7fe8741ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.416+0000 7fe879600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740c3b80 con 0x7fe874157820 2024-03-09T09:25:17.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.416+0000 7fe879600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe874157c00 con 0x7fe8740aad50 2024-03-09T09:25:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.416+0000 7fe879600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740abf50 con 0x7fe8740c1d70 2024-03-09T09:25:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.416+0000 7fe872000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8740c1d70 0x7fe874155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.417+0000 7fe873400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874157820 0x7fe8741ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.417+0000 7fe873400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874157820 0x7fe8741ef8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60996/0 (socket says 172.21.15.67:60996) 2024-03-09T09:25:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.417+0000 7fe873400640 1 -- 172.21.15.67:0/1682093182 learned_addr learned my addr 172.21.15.67:0/1682093182 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.419+0000 7fe873400640 1 -- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8740c1d70 msgr2=0x7fe874155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.420+0000 7fe873400640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8740c1d70 0x7fe874155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.420+0000 7fe873400640 1 -- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8740aad50 msgr2=0x7fe87415a080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:25:17.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.420+0000 7fe873400640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8740aad50 0x7fe87415a080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.420+0000 7fe873400640 1 -- 172.21.15.67:0/1682093182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87415a660 con 0x7fe874157820 2024-03-09T09:25:17.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.422+0000 7fe873400640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874157820 0x7fe8741ef8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe864034040 tx=0x7fe8640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:17.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.423+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8640317a0 con 0x7fe874157820 2024-03-09T09:25:17.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.424+0000 7fe879600640 1 -- 172.21.15.67:0/1682093182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8741f3030 con 0x7fe874157820 2024-03-09T09:25:17.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.424+0000 7fe879600640 1 -- 172.21.15.67:0/1682093182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8741f35f0 con 0x7fe874157820 2024-03-09T09:25:17.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.424+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe864031940 con 0x7fe874157820 2024-03-09T09:25:17.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.424+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864031d10 con 0x7fe874157820 2024-03-09T09:25:17.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.428+0000 7fe879600640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c002a70 0x7fe84c004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.428+0000 7fe879600640 1 -- 172.21.15.67:0/1682093182 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c007ec0 con 0x7fe84c002a70 2024-03-09T09:25:17.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.429+0000 7fe872a00640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c002a70 0x7fe84c004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.430+0000 7fe872a00640 1 -- 172.21.15.67:0/1682093182 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe84c0080c0 con 0x7fe84c002a70 2024-03-09T09:25:17.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.431+0000 7fe872a00640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c002a70 0x7fe84c004f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe86c004b90 tx=0x7fe86c0021e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:17.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.432+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe864047480 con 0x7fe874157820 2024-03-09T09:25:17.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.433+0000 7fe86aa00640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe85406b4b0 0x7fe85406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.434+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6693+0+0 (secure 0 0 0) 0x7fe86402f020 con 0x7fe874157820 2024-03-09T09:25:17.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.434+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c017070 con 0x7fe84c002a70 2024-03-09T09:25:17.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.434+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fe86c00ac10 con 0x7fe84c002a70 2024-03-09T09:25:17.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.434+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c002a70 msgr2=0x7fe84c004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.434+0000 7fe86aa00640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c002a70 0x7fe84c004f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe86c004b90 tx=0x7fe86c0021e0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.434+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c002a70 msgr2=0x7fe84c004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:17.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.434+0000 7fe86aa00640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c002a70 0x7fe84c004f30 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe86c004b90 tx=0x7fe86c0021e0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.434+0000 7fe872000640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe85406b4b0 0x7fe85406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.437+0000 7fe872000640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe85406b4b0 0x7fe85406d970 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe874073a50 tx=0x7fe860008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:17.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.453+0000 7fe879600640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c006f00 0x7fe84c007390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:17.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.453+0000 7fe879600640 1 -- 172.21.15.67:0/1682093182 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c00a4a0 con 0x7fe84c006f00 2024-03-09T09:25:17.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.455+0000 7fe872a00640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c006f00 0x7fe84c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:17.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.455+0000 7fe872a00640 1 -- 172.21.15.67:0/1682093182 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe84c0078d0 con 0x7fe84c006f00 2024-03-09T09:25:17.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.456+0000 7fe872a00640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c006f00 0x7fe84c007390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe84c008090 tx=0x7fe86c001250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:17.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.457+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c017040 con 0x7fe84c006f00 2024-03-09T09:25:17.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.458+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5176 (secure 0 0 0) 0x7fe86c023400 con 0x7fe84c006f00 2024-03-09T09:25:17.468 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:25:17.468 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:17.468 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:17.468 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:25:17.468 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 260, 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:17.469 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:17.471 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:17.471 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:17.471 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:17.471 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:17.471 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.471 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:25:17.471 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:25:17.471 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:25:17.471 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:25:17.471 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:25:17.472 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:25:17.472 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:17.472 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:17.472 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:25:17.472 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:17.473 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:17.474 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:17.474 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:17.474 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:17.474 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:17.474 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.474 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:25:17.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.474 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:25:17.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:25:17.475 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.476 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.476 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:25:17.476 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:25:17.477 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:17.477 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:17.477 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:17.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.477 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:17.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:17.480 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.481 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.482 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:25:17.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.482 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:17.482 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:17.482 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:17.482 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.482 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.482 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:25:17.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.482 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:17.483 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:17.484 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:25:17.485 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:25:17.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.459+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c006f00 msgr2=0x7fe84c007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.459+0000 7fe86aa00640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c006f00 0x7fe84c007390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe84c008090 tx=0x7fe86c001250 comp rx=0 tx=0).stop 2024-03-09T09:25:17.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.459+0000 7fe86aa00640 1 -- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c006f00 msgr2=0x7fe84c007390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:17.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.459+0000 7fe86aa00640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe84c006f00 0x7fe84c007390 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe84c008090 tx=0x7fe86c001250 comp rx=0 tx=0).stop 2024-03-09T09:25:17.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.461+0000 7fe879600640 1 -- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe85406b4b0 msgr2=0x7fe85406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.461+0000 7fe879600640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe85406b4b0 0x7fe85406d970 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe874073a50 tx=0x7fe860008040 comp rx=0 tx=0).stop 2024-03-09T09:25:17.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.461+0000 7fe879600640 1 -- 172.21.15.67:0/1682093182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874157820 msgr2=0x7fe8741ef8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:17.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.461+0000 7fe879600640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874157820 0x7fe8741ef8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe864034040 tx=0x7fe8640046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.461+0000 7fe873400640 1 -- 172.21.15.67:0/1682093182 reap_dead start 2024-03-09T09:25:17.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.462+0000 7fe879600640 1 -- 172.21.15.67:0/1682093182 shutdown_connections 2024-03-09T09:25:17.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.462+0000 7fe879600640 1 --2- 172.21.15.67:0/1682093182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe874157820 0x7fe8741ef8a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:17.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.462+0000 7fe879600640 1 -- 172.21.15.67:0/1682093182 >> 172.21.15.67:0/1682093182 conn(0x7fe8740c7370 msgr2=0x7fe8740ab840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:17.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.462+0000 7fe879600640 1 -- 172.21.15.67:0/1682093182 shutdown_connections 2024-03-09T09:25:17.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:17.463+0000 7fe879600640 1 -- 172.21.15.67:0/1682093182 wait complete. 2024-03-09T09:25:17.496 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:25:17.496 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:25:22.647 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:25:22.648 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.88 2024-03-09T09:25:22.648 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:22.648 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 88 -eq 48 ']' 2024-03-09T09:25:22.648 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.88 2024-03-09T09:25:22.648 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.88 2024-03-09T09:25:22.648 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.88 2024-03-09T09:25:22.657 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:25:29.092 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:25:29.093 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:29.093 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 89 -eq 48 ']' 2024-03-09T09:25:29.093 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.89 2024-03-09T09:25:29.093 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.89 2024-03-09T09:25:29.093 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.89 2024-03-09T09:25:29.094 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.89 2024-03-09T09:25:29.099 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:25:31.503 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:25:31.512 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:25:36.492 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:25:36.493 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:36.493 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 90 -eq 48 ']' 2024-03-09T09:25:36.493 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.90 2024-03-09T09:25:36.493 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo checking v0.90 2024-03-09T09:25:36.494 INFO:tasks.workunit.client.1.smithi134.stdout:checking v0.90 2024-03-09T09:25:36.494 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd .snap/v0.90 2024-03-09T09:25:36.500 INFO:tasks.workunit.client.1.smithi134.stderr:+ git diff --exit-code 2024-03-09T09:25:37.497 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:25:37.497 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-03-09T09:25:37.497 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-09T09:25:37.497 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-09T09:25:37.504 INFO:tasks.ceph.mon.b:Started 2024-03-09T09:25:37.504 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:25:37.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:25:37.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.739+0000 7fc6ab000640 1 Processor -- start 2024-03-09T09:25:37.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.739+0000 7fc6ab000640 1 -- start start 2024-03-09T09:25:37.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.740+0000 7fc6ab000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6a40cadb0 0x7fc6a4156720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:37.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.740+0000 7fc6ab000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a4157000 con 0x7fc6a40c5e40 2024-03-09T09:25:37.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.740+0000 7fc6ab000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a40ae430 con 0x7fc6a4153300 2024-03-09T09:25:37.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.741+0000 7fc6ab000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a40ae6f0 con 0x7fc6a40cadb0 2024-03-09T09:25:37.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.741+0000 7fc6aa600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc6a40cadb0 0x7fc6a4156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:37.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.741+0000 7fc6a9c00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fc6a4153300 0x7fc6a40b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:38984/0 (socket says 172.21.15.67:38984) 2024-03-09T09:25:37.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.741+0000 7fc6a9200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc6a40c5e40 0x7fc6a4156010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:56106/0 (socket says 172.21.15.67:56106) 2024-03-09T09:25:37.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.741+0000 7fc6a9c00640 1 -- 172.21.15.67:0/3638079299 learned_addr learned my addr 172.21.15.67:0/3638079299 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:37.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.741+0000 7fc6aa600640 1 -- 172.21.15.67:0/3638079299 >> v1:172.21.15.134:6789/0 conn(0x7fc6a4153300 legacy=0x7fc6a40b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:37.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.741+0000 7fc6aa600640 1 -- 172.21.15.67:0/3638079299 >> v1:172.21.15.67:6789/0 conn(0x7fc6a40c5e40 legacy=0x7fc6a4156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:37.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.741+0000 7fc6aa600640 1 -- 172.21.15.67:0/3638079299 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a4155b10 con 0x7fc6a40cadb0 2024-03-09T09:25:37.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.742+0000 7fc6aa600640 1 --2- 172.21.15.67:0/3638079299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a40cadb0 0x7fc6a4156720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009d90 tx=0x7fc69c02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb91f563e8c5e87f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:37.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.743+0000 7fc6a3e00640 1 -- 172.21.15.67:0/3638079299 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c039070 con 0x7fc6a40cadb0 2024-03-09T09:25:37.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.743+0000 7fc6a3e00640 1 -- 172.21.15.67:0/3638079299 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c02ccf0 con 0x7fc6a40cadb0 2024-03-09T09:25:37.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.743+0000 7fc6a3e00640 1 -- 172.21.15.67:0/3638079299 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c034660 con 0x7fc6a40cadb0 2024-03-09T09:25:37.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.743+0000 7fc6ab000640 1 -- 172.21.15.67:0/3638079299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a40cadb0 msgr2=0x7fc6a4156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:37.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.743+0000 7fc6ab000640 1 --2- 172.21.15.67:0/3638079299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a40cadb0 0x7fc6a4156720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009d90 tx=0x7fc69c02bf40 comp rx=0 tx=0).stop 2024-03-09T09:25:37.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.743+0000 7fc6ab000640 1 -- 172.21.15.67:0/3638079299 shutdown_connections 2024-03-09T09:25:37.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.743+0000 7fc6ab000640 1 --2- 172.21.15.67:0/3638079299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a40cadb0 0x7fc6a4156720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:37.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.743+0000 7fc6ab000640 1 -- 172.21.15.67:0/3638079299 >> 172.21.15.67:0/3638079299 conn(0x7fc6a40732d0 msgr2=0x7fc6a40ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:37.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.744+0000 7fc6ab000640 1 -- 172.21.15.67:0/3638079299 shutdown_connections 2024-03-09T09:25:37.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.744+0000 7fc6ab000640 1 -- 172.21.15.67:0/3638079299 wait complete. 2024-03-09T09:25:37.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.745+0000 7fc6ab000640 1 Processor -- start 2024-03-09T09:25:37.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.745+0000 7fc6ab000640 1 -- start start 2024-03-09T09:25:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.746+0000 7fc6ab000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a40c5e40 0x7fc6a40c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.746+0000 7fc6ab000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a40cadb0 0x7fc6a41eee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.746+0000 7fc6ab000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6a4153300 0x7fc6a406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.746+0000 7fc6ab000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a40c59e0 con 0x7fc6a40c5e40 2024-03-09T09:25:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.746+0000 7fc6ab000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a4159e90 con 0x7fc6a40cadb0 2024-03-09T09:25:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.746+0000 7fc6ab000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a40b0710 con 0x7fc6a4153300 2024-03-09T09:25:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.747+0000 7fc6a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a40c5e40 0x7fc6a40c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.747+0000 7fc6a9c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a40c5e40 0x7fc6a40c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34056/0 (socket says 172.21.15.67:34056) 2024-03-09T09:25:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.747+0000 7fc6a9c00640 1 -- 172.21.15.67:0/1901082476 learned_addr learned my addr 172.21.15.67:0/1901082476 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.747+0000 7fc6aa600640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6a4153300 0x7fc6a406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.747+0000 7fc6a9200640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a40cadb0 0x7fc6a41eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.747+0000 7fc6a9c00640 1 -- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6a4153300 msgr2=0x7fc6a406ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.747+0000 7fc6a9c00640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6a4153300 0x7fc6a406ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.747+0000 7fc6a9c00640 1 -- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a40cadb0 msgr2=0x7fc6a41eee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.747+0000 7fc6a9c00640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a40cadb0 0x7fc6a41eee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.747+0000 7fc6a9c00640 1 -- 172.21.15.67:0/1901082476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a414bd60 con 0x7fc6a40c5e40 2024-03-09T09:25:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.747+0000 7fc6a9c00640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a40c5e40 0x7fc6a40c7420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc69800c4b0 tx=0x7fc69800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.748+0000 7fc6a1600640 1 -- 172.21.15.67:0/1901082476 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698017070 con 0x7fc6a40c5e40 2024-03-09T09:25:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.748+0000 7fc6a1600640 1 -- 172.21.15.67:0/1901082476 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc698005ce0 con 0x7fc6a40c5e40 2024-03-09T09:25:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.748+0000 7fc6a1600640 1 -- 172.21.15.67:0/1901082476 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698007270 con 0x7fc6a40c5e40 2024-03-09T09:25:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.748+0000 7fc6ab000640 1 -- 172.21.15.67:0/1901082476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a414c000 con 0x7fc6a40c5e40 2024-03-09T09:25:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.748+0000 7fc6ab000640 1 -- 172.21.15.67:0/1901082476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a41ef9c0 con 0x7fc6a40c5e40 2024-03-09T09:25:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.749+0000 7fc6ab000640 1 -- 172.21.15.67:0/1901082476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a404f3c0 con 0x7fc6a40c5e40 2024-03-09T09:25:37.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.750+0000 7fc6a1600640 1 -- 172.21.15.67:0/1901082476 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc69801a070 con 0x7fc6a40c5e40 2024-03-09T09:25:37.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.751+0000 7fc6a1600640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc68406b4d0 0x7fc68406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:37.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.751+0000 7fc6a1600640 1 -- 172.21.15.67:0/1901082476 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6693+0+0 (secure 0 0 0) 0x7fc6980897d0 con 0x7fc6a40c5e40 2024-03-09T09:25:37.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.751+0000 7fc6a9200640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc68406b4d0 0x7fc68406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:37.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.752+0000 7fc6a9200640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc68406b4d0 0x7fc68406d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4152830 tx=0x7fc690007cc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:37.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:37.756+0000 7fc6a1600640 1 -- 172.21.15.67:0/1901082476 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc698057c10 con 0x7fc6a40c5e40 2024-03-09T09:25:38.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.040+0000 7fc6ab000640 1 -- 172.21.15.67:0/1901082476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc6a40cab90 con 0x7fc6a40c5e40 2024-03-09T09:25:38.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.041+0000 7fc6a1600640 1 -- 172.21.15.67:0/1901082476 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fc69805bcb0 con 0x7fc6a40c5e40 2024-03-09T09:25:38.052 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":260,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:25:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 -- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc68406b4d0 msgr2=0x7fc68406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc68406b4d0 0x7fc68406d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4152830 tx=0x7fc690007cc0 comp rx=0 tx=0).stop 2024-03-09T09:25:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 -- 172.21.15.67:0/1901082476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a40c5e40 msgr2=0x7fc6a40c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a40c5e40 0x7fc6a40c7420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc69800c4b0 tx=0x7fc69800c970 comp rx=0 tx=0).stop 2024-03-09T09:25:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 -- 172.21.15.67:0/1901082476 shutdown_connections 2024-03-09T09:25:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc68406b4d0 0x7fc68406d990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6a4153300 0x7fc6a406ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a40cadb0 0x7fc6a41eee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 --2- 172.21.15.67:0/1901082476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc6a40c5e40 0x7fc6a40c7420 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:38.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 -- 172.21.15.67:0/1901082476 >> 172.21.15.67:0/1901082476 conn(0x7fc6a40732d0 msgr2=0x7fc6a41513d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 -- 172.21.15.67:0/1901082476 shutdown_connections 2024-03-09T09:25:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:38.044+0000 7fc6ab000640 1 -- 172.21.15.67:0/1901082476 wait complete. 2024-03-09T09:25:41.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:25:41.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.291+0000 7f7c51a00640 1 Processor -- start 2024-03-09T09:25:41.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.291+0000 7f7c51a00640 1 -- start start 2024-03-09T09:25:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.293+0000 7f7c51a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f7c4c0c5f50 0x7f7c4c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.293+0000 7f7c51a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7c4c06b040 0x7f7c4c06c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.293+0000 7f7c51a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c4c0c6a40 con 0x7f7c4c0c96f0 2024-03-09T09:25:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.293+0000 7f7c51a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4c06b5b0 con 0x7f7c4c06b040 2024-03-09T09:25:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.293+0000 7f7c51a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4c06b890 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.293+0000 7f7c43e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7c4c06b040 0x7f7c4c06c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.293+0000 7f7c43e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f7c4c06b040 0x7f7c4c06c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49700/0 (socket says 172.21.15.67:49700) 2024-03-09T09:25:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.293+0000 7f7c43e00640 1 -- 172.21.15.67:0/2741481943 learned_addr learned my addr 172.21.15.67:0/2741481943 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.293+0000 7f7c43e00640 1 -- 172.21.15.67:0/2741481943 >> v2:172.21.15.134:3300/0 conn(0x7f7c4c0c5f50 msgr2=0x7f7c4c155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.293+0000 7f7c4aa00640 1 --2- 172.21.15.67:0/2741481943 >> v2:172.21.15.134:3300/0 conn(0x7f7c4c0c5f50 0x7f7c4c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.294+0000 7f7c43e00640 1 --2- 172.21.15.67:0/2741481943 >> v2:172.21.15.134:3300/0 conn(0x7f7c4c0c5f50 0x7f7c4c155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.294+0000 7f7c43e00640 1 -- 172.21.15.67:0/2741481943 >> v1:172.21.15.134:6789/0 conn(0x7f7c4c0c96f0 legacy=0x7f7c4c0c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.294+0000 7f7c43e00640 1 -- 172.21.15.67:0/2741481943 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c06bb70 con 0x7f7c4c06b040 2024-03-09T09:25:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.294+0000 7f7c4aa00640 1 --2- 172.21.15.67:0/2741481943 >> v2:172.21.15.134:3300/0 conn(0x7f7c4c0c5f50 0x7f7c4c155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:25:41.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.295+0000 7f7c43e00640 1 --2- 172.21.15.67:0/2741481943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c4c06b040 0x7f7c4c06c650 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c073c20 tx=0x7f7c3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd49fb2252e8b2f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.296+0000 7f7c4a000640 1 -- 172.21.15.67:0/2741481943 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34035020 con 0x7f7c4c06b040 2024-03-09T09:25:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.296+0000 7f7c4a000640 1 -- 172.21.15.67:0/2741481943 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c34031870 con 0x7f7c4c06b040 2024-03-09T09:25:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.296+0000 7f7c4a000640 1 -- 172.21.15.67:0/2741481943 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34031c40 con 0x7f7c4c06b040 2024-03-09T09:25:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.296+0000 7f7c51a00640 1 -- 172.21.15.67:0/2741481943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c4c06b040 msgr2=0x7f7c4c06c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.296+0000 7f7c51a00640 1 --2- 172.21.15.67:0/2741481943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c4c06b040 0x7f7c4c06c650 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c073c20 tx=0x7f7c3402b910 comp rx=0 tx=0).stop 2024-03-09T09:25:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.297+0000 7f7c51a00640 1 -- 172.21.15.67:0/2741481943 shutdown_connections 2024-03-09T09:25:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.297+0000 7f7c51a00640 1 --2- 172.21.15.67:0/2741481943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c4c06b040 0x7f7c4c06c650 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.297+0000 7f7c51a00640 1 --2- 172.21.15.67:0/2741481943 >> v2:172.21.15.134:3300/0 conn(0x7f7c4c0c5f50 0x7f7c4c155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.297+0000 7f7c51a00640 1 -- 172.21.15.67:0/2741481943 >> 172.21.15.67:0/2741481943 conn(0x7f7c4c0c8050 msgr2=0x7f7c4c0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.297+0000 7f7c51a00640 1 -- 172.21.15.67:0/2741481943 shutdown_connections 2024-03-09T09:25:41.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.297+0000 7f7c51a00640 1 -- 172.21.15.67:0/2741481943 wait complete. 2024-03-09T09:25:41.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.299+0000 7f7c51a00640 1 Processor -- start 2024-03-09T09:25:41.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.300+0000 7f7c51a00640 1 -- start start 2024-03-09T09:25:41.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.301+0000 7f7c51a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c4c06b040 0x7f7c4c0bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:41.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.301+0000 7f7c51a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c4c0c5f50 0x7f7c4c154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:41.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.301+0000 7f7c51a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c4c0c96f0 0x7f7c4c1e1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:41.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.301+0000 7f7c51a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c14cc00 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.301+0000 7f7c51a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c06c130 con 0x7f7c4c0c96f0 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.301+0000 7f7c51a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c0c9b60 con 0x7f7c4c06b040 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.302+0000 7f7c43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c4c0c5f50 0x7f7c4c154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.302+0000 7f7c43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c4c0c5f50 0x7f7c4c154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49710/0 (socket says 172.21.15.67:49710) 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.302+0000 7f7c43e00640 1 -- 172.21.15.67:0/3313875356 learned_addr learned my addr 172.21.15.67:0/3313875356 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.302+0000 7f7c4aa00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c4c06b040 0x7f7c4c0bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.302+0000 7f7c4b400640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c4c0c96f0 0x7f7c4c1e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.302+0000 7f7c43e00640 1 -- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c4c06b040 msgr2=0x7f7c4c0bfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.302+0000 7f7c43e00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c4c06b040 0x7f7c4c0bfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.302+0000 7f7c43e00640 1 -- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c4c0c96f0 msgr2=0x7f7c4c1e1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.302+0000 7f7c43e00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c4c0c96f0 0x7f7c4c1e1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.302+0000 7f7c43e00640 1 -- 172.21.15.67:0/3313875356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c0bf110 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.302+0000 7f7c4b400640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c4c0c96f0 0x7f7c4c1e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:25:41.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.303+0000 7f7c43e00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c4c0c5f50 0x7f7c4c154ec0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c34000c00 tx=0x7f7c340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:41.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.303+0000 7f7c42a00640 1 -- 172.21.15.67:0/3313875356 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34035040 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.303+0000 7f7c42a00640 1 -- 172.21.15.67:0/3313875356 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c34045480 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.303+0000 7f7c42a00640 1 -- 172.21.15.67:0/3313875356 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34045870 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.303+0000 7f7c51a00640 1 -- 172.21.15.67:0/3313875356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4c0c2020 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.303+0000 7f7c51a00640 1 -- 172.21.15.67:0/3313875356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4c155400 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.305+0000 7f7c51a00640 1 -- 172.21.15.67:0/3313875356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4c075a00 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.305+0000 7f7c42a00640 1 -- 172.21.15.67:0/3313875356 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7c34038020 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.306+0000 7f7c42a00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7c2806b500 0x7f7c2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:41.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.306+0000 7f7c42a00640 1 -- 172.21.15.67:0/3313875356 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f7c340aca00 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.307+0000 7f7c4aa00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7c2806b500 0x7f7c2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:41.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.308+0000 7f7c4aa00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7c2806b500 0x7f7c2806d9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00a970 tx=0x7f7c3c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:41.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.316+0000 7f7c42a00640 1 -- 172.21.15.67:0/3313875356 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7c3407b530 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.598+0000 7f7c51a00640 1 -- 172.21.15.67:0/3313875356 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7c4c075a00 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.599+0000 7f7c42a00640 1 -- 172.21.15.67:0/3313875356 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f7c3407ae30 con 0x7f7c4c0c5f50 2024-03-09T09:25:41.606 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":260,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:25:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.601+0000 7f7c51a00640 1 -- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7c2806b500 msgr2=0x7f7c2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.601+0000 7f7c51a00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7c2806b500 0x7f7c2806d9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00a970 tx=0x7f7c3c008040 comp rx=0 tx=0).stop 2024-03-09T09:25:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.601+0000 7f7c51a00640 1 -- 172.21.15.67:0/3313875356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c4c0c5f50 msgr2=0x7f7c4c154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.601+0000 7f7c51a00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c4c0c5f50 0x7f7c4c154ec0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c34000c00 tx=0x7f7c340046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.602+0000 7f7c51a00640 1 -- 172.21.15.67:0/3313875356 shutdown_connections 2024-03-09T09:25:41.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.602+0000 7f7c51a00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f7c2806b500 0x7f7c2806d9c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:41.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.602+0000 7f7c51a00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c4c0c96f0 0x7f7c4c1e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:41.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.602+0000 7f7c51a00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c4c0c5f50 0x7f7c4c154ec0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:41.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.602+0000 7f7c51a00640 1 --2- 172.21.15.67:0/3313875356 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c4c06b040 0x7f7c4c0bfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:41.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.602+0000 7f7c51a00640 1 -- 172.21.15.67:0/3313875356 >> 172.21.15.67:0/3313875356 conn(0x7f7c4c0c8050 msgr2=0x7f7c4c0ad780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:41.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.602+0000 7f7c51a00640 1 -- 172.21.15.67:0/3313875356 shutdown_connections 2024-03-09T09:25:41.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:41.602+0000 7f7c51a00640 1 -- 172.21.15.67:0/3313875356 wait complete. 2024-03-09T09:25:43.771 INFO:tasks.workunit.client.1.smithi134.stderr:+ cd ../.. 2024-03-09T09:25:43.771 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:43.771 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 1 -eq 48 ']' 2024-03-09T09:25:43.771 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.1 2024-03-09T09:25:43.771 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.1 2024-03-09T09:25:43.809 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:43.809 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 2 -eq 48 ']' 2024-03-09T09:25:43.810 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.2 2024-03-09T09:25:43.810 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.2 2024-03-09T09:25:43.843 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:43.843 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 3 -eq 48 ']' 2024-03-09T09:25:43.843 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.3 2024-03-09T09:25:43.843 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.3 2024-03-09T09:25:43.872 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:43.873 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 4 -eq 48 ']' 2024-03-09T09:25:43.873 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.4 2024-03-09T09:25:43.873 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.4 2024-03-09T09:25:43.902 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:43.902 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 5 -eq 48 ']' 2024-03-09T09:25:43.902 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.5 2024-03-09T09:25:43.902 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.5 2024-03-09T09:25:43.941 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:43.941 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 6 -eq 48 ']' 2024-03-09T09:25:43.942 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.6 2024-03-09T09:25:43.942 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.6 2024-03-09T09:25:43.972 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:43.972 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 7 -eq 48 ']' 2024-03-09T09:25:43.972 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.7 2024-03-09T09:25:43.972 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.7 2024-03-09T09:25:44.002 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.002 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 8 -eq 48 ']' 2024-03-09T09:25:44.002 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.8 2024-03-09T09:25:44.002 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.8 2024-03-09T09:25:44.381 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.381 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 9 -eq 48 ']' 2024-03-09T09:25:44.381 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.9 2024-03-09T09:25:44.381 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.9 2024-03-09T09:25:44.415 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.415 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 10 -eq 48 ']' 2024-03-09T09:25:44.415 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.10 2024-03-09T09:25:44.415 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.10 2024-03-09T09:25:44.448 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 11 -eq 48 ']' 2024-03-09T09:25:44.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.11 2024-03-09T09:25:44.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.11 2024-03-09T09:25:44.482 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.482 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 12 -eq 48 ']' 2024-03-09T09:25:44.482 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.12 2024-03-09T09:25:44.482 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.12 2024-03-09T09:25:44.514 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.514 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 13 -eq 48 ']' 2024-03-09T09:25:44.514 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.13 2024-03-09T09:25:44.515 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.13 2024-03-09T09:25:44.547 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.547 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 14 -eq 48 ']' 2024-03-09T09:25:44.547 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.14 2024-03-09T09:25:44.547 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.14 2024-03-09T09:25:44.591 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.591 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 15 -eq 48 ']' 2024-03-09T09:25:44.591 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.15 2024-03-09T09:25:44.591 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.15 2024-03-09T09:25:44.628 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.628 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 16 -eq 48 ']' 2024-03-09T09:25:44.629 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.16 2024-03-09T09:25:44.629 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.16 2024-03-09T09:25:44.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:25:44.659 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.659 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 17 -eq 48 ']' 2024-03-09T09:25:44.659 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.17 2024-03-09T09:25:44.660 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.17 2024-03-09T09:25:44.686 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.687 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 18 -eq 48 ']' 2024-03-09T09:25:44.687 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.18 2024-03-09T09:25:44.687 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.18 2024-03-09T09:25:44.719 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.719 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 19 -eq 48 ']' 2024-03-09T09:25:44.719 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.19 2024-03-09T09:25:44.719 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.19 2024-03-09T09:25:44.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 20 -eq 48 ']' 2024-03-09T09:25:44.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.20 2024-03-09T09:25:44.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.20 2024-03-09T09:25:44.775 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.776 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 21 -eq 48 ']' 2024-03-09T09:25:44.776 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.21 2024-03-09T09:25:44.776 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.21 2024-03-09T09:25:44.807 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.808 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 22 -eq 48 ']' 2024-03-09T09:25:44.808 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.22 2024-03-09T09:25:44.808 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.22 2024-03-09T09:25:44.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 23 -eq 48 ']' 2024-03-09T09:25:44.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.23 2024-03-09T09:25:44.833 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.23 2024-03-09T09:25:44.860 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.860 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 24 -eq 48 ']' 2024-03-09T09:25:44.860 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.24 2024-03-09T09:25:44.860 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.24 2024-03-09T09:25:44.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.853+0000 7f4e7e400640 1 Processor -- start 2024-03-09T09:25:44.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.854+0000 7f4e7e400640 1 -- start start 2024-03-09T09:25:44.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.855+0000 7f4e7e400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e78153500 0x7f4e7806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:44.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.855+0000 7f4e7e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e780c6630 0x7f4e780c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:44.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.855+0000 7f4e7e400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e7815a2a0 con 0x7f4e780c5f60 2024-03-09T09:25:44.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.855+0000 7f4e7e400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e78159d70 con 0x7f4e78153500 2024-03-09T09:25:44.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.855+0000 7f4e7e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7806bb30 con 0x7f4e780c6630 2024-03-09T09:25:44.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.855+0000 7f4e77400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e78153500 0x7f4e7806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:44.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.855+0000 7f4e77e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4e780c5f60 0x7f4e78155db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47336/0 (socket says 172.21.15.67:47336) 2024-03-09T09:25:44.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.855+0000 7f4e77e00640 1 -- 172.21.15.67:0/2427445884 learned_addr learned my addr 172.21.15.67:0/2427445884 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:44.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.856+0000 7f4e76a00640 1 --2- 172.21.15.67:0/2427445884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e780c6630 0x7f4e780c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:44.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.856+0000 7f4e77400640 1 -- 172.21.15.67:0/2427445884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e780c6630 msgr2=0x7f4e780c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:44.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.856+0000 7f4e77400640 1 --2- 172.21.15.67:0/2427445884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e780c6630 0x7f4e780c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:44.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.856+0000 7f4e77400640 1 -- 172.21.15.67:0/2427445884 >> v1:172.21.15.67:6789/0 conn(0x7f4e780c5f60 legacy=0x7f4e78155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:44.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.856+0000 7f4e76000640 1 -- 172.21.15.67:0/2427445884 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217177971 0 0) 0x7f4e7815a2a0 con 0x7f4e780c5f60 2024-03-09T09:25:44.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.856+0000 7f4e77400640 1 -- 172.21.15.67:0/2427445884 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e78159fc0 con 0x7f4e78153500 2024-03-09T09:25:44.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.857+0000 7f4e77400640 1 --2- 172.21.15.67:0/2427445884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e78153500 0x7f4e7806b040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e680097a0 tx=0x7f4e6802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d65d9d25de97738 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:44.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.857+0000 7f4e76000640 1 -- 172.21.15.67:0/2427445884 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68038070 con 0x7f4e78153500 2024-03-09T09:25:44.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.858+0000 7f4e76000640 1 -- 172.21.15.67:0/2427445884 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e68031700 con 0x7f4e78153500 2024-03-09T09:25:44.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.858+0000 7f4e76000640 1 -- 172.21.15.67:0/2427445884 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68031ad0 con 0x7f4e78153500 2024-03-09T09:25:44.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.858+0000 7f4e7e400640 1 -- 172.21.15.67:0/2427445884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e78153500 msgr2=0x7f4e7806b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:44.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.858+0000 7f4e7e400640 1 --2- 172.21.15.67:0/2427445884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e78153500 0x7f4e7806b040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e680097a0 tx=0x7f4e6802b7a0 comp rx=0 tx=0).stop 2024-03-09T09:25:44.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.858+0000 7f4e7e400640 1 -- 172.21.15.67:0/2427445884 shutdown_connections 2024-03-09T09:25:44.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.858+0000 7f4e7e400640 1 --2- 172.21.15.67:0/2427445884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e780c6630 0x7f4e780c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:44.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.858+0000 7f4e7e400640 1 --2- 172.21.15.67:0/2427445884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e78153500 0x7f4e7806b040 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e680097a0 tx=0x7f4e6802b7a0 comp rx=0 tx=0).stop 2024-03-09T09:25:44.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.858+0000 7f4e7e400640 1 -- 172.21.15.67:0/2427445884 >> 172.21.15.67:0/2427445884 conn(0x7f4e780c8060 msgr2=0x7f4e780c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:44.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.858+0000 7f4e7e400640 1 -- 172.21.15.67:0/2427445884 shutdown_connections 2024-03-09T09:25:44.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.859+0000 7f4e7e400640 1 -- 172.21.15.67:0/2427445884 wait complete. 2024-03-09T09:25:44.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.860+0000 7f4e7e400640 1 Processor -- start 2024-03-09T09:25:44.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.860+0000 7f4e7e400640 1 -- start start 2024-03-09T09:25:44.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.861+0000 7f4e7e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e780c5f60 0x7f4e7814c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:44.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.861+0000 7f4e7e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e7814ccc0 0x7f4e781ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:44.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.862+0000 7f4e7e400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e781557e0 0x7f4e78155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:44.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.862+0000 7f4e7e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780bed90 con 0x7f4e780c5f60 2024-03-09T09:25:44.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.862+0000 7f4e7e400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780c6f90 con 0x7f4e781557e0 2024-03-09T09:25:44.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.862+0000 7f4e7e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780c94e0 con 0x7f4e7814ccc0 2024-03-09T09:25:44.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.862+0000 7f4e77400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e780c5f60 0x7f4e7814c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:44.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.862+0000 7f4e77400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e780c5f60 0x7f4e7814c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49714/0 (socket says 172.21.15.67:49714) 2024-03-09T09:25:44.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.862+0000 7f4e77400640 1 -- 172.21.15.67:0/273760495 learned_addr learned my addr 172.21.15.67:0/273760495 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:44.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.862+0000 7f4e76a00640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e7814ccc0 0x7f4e781ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:44.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.862+0000 7f4e77e00640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e781557e0 0x7f4e78155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:44.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.862+0000 7f4e77400640 1 -- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e7814ccc0 msgr2=0x7f4e781ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:44.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.862+0000 7f4e77400640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e7814ccc0 0x7f4e781ef2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:44.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.863+0000 7f4e77400640 1 -- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e781557e0 msgr2=0x7f4e78155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:44.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.863+0000 7f4e77400640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e781557e0 0x7f4e78155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:44.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.863+0000 7f4e77400640 1 -- 172.21.15.67:0/273760495 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e781ef830 con 0x7f4e780c5f60 2024-03-09T09:25:44.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.863+0000 7f4e77400640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e780c5f60 0x7f4e7814c620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e6802bd80 tx=0x7f4e680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:44.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.863+0000 7f4e6f400640 1 -- 172.21.15.67:0/273760495 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68042070 con 0x7f4e780c5f60 2024-03-09T09:25:44.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.863+0000 7f4e6f400640 1 -- 172.21.15.67:0/273760495 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e68031eb0 con 0x7f4e780c5f60 2024-03-09T09:25:44.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.863+0000 7f4e7e400640 1 -- 172.21.15.67:0/273760495 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e781efad0 con 0x7f4e780c5f60 2024-03-09T09:25:44.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.863+0000 7f4e6f400640 1 -- 172.21.15.67:0/273760495 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68038040 con 0x7f4e780c5f60 2024-03-09T09:25:44.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.864+0000 7f4e7e400640 1 -- 172.21.15.67:0/273760495 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e781f0090 con 0x7f4e780c5f60 2024-03-09T09:25:44.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.865+0000 7f4e7e400640 1 -- 172.21.15.67:0/273760495 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e78075a00 con 0x7f4e780c5f60 2024-03-09T09:25:44.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.866+0000 7f4e6f400640 1 -- 172.21.15.67:0/273760495 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4e68045070 con 0x7f4e780c5f60 2024-03-09T09:25:44.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.868+0000 7f4e6f400640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e5806b4d0 0x7f4e5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:44.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.868+0000 7f4e6f400640 1 -- 172.21.15.67:0/273760495 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f4e680ac110 con 0x7f4e780c5f60 2024-03-09T09:25:44.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.868+0000 7f4e76a00640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e5806b4d0 0x7f4e5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:44.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.869+0000 7f4e76a00640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e5806b4d0 0x7f4e5806d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4e780c9cb0 tx=0x7f4e64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:44.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:44.872+0000 7f4e6f400640 1 -- 172.21.15.67:0/273760495 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4e68030070 con 0x7f4e780c5f60 2024-03-09T09:25:44.891 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.891 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 25 -eq 48 ']' 2024-03-09T09:25:44.891 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.25 2024-03-09T09:25:44.891 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.25 2024-03-09T09:25:44.919 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.919 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 26 -eq 48 ']' 2024-03-09T09:25:44.919 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.26 2024-03-09T09:25:44.919 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.26 2024-03-09T09:25:44.945 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:44.945 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 27 -eq 48 ']' 2024-03-09T09:25:44.945 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.27 2024-03-09T09:25:44.945 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.27 2024-03-09T09:25:45.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.161+0000 7f4e7e400640 1 -- 172.21.15.67:0/273760495 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4e780cafb0 con 0x7f4e780c5f60 2024-03-09T09:25:45.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.162+0000 7f4e6f400640 1 -- 172.21.15.67:0/273760495 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f4e6807a570 con 0x7f4e780c5f60 2024-03-09T09:25:45.169 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":262,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:25:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.164+0000 7f4e7e400640 1 -- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e5806b4d0 msgr2=0x7f4e5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.164+0000 7f4e7e400640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e5806b4d0 0x7f4e5806d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4e780c9cb0 tx=0x7f4e64008040 comp rx=0 tx=0).stop 2024-03-09T09:25:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.164+0000 7f4e7e400640 1 -- 172.21.15.67:0/273760495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e780c5f60 msgr2=0x7f4e7814c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.164+0000 7f4e7e400640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e780c5f60 0x7f4e7814c620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e6802bd80 tx=0x7f4e680046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.165+0000 7f4e7e400640 1 -- 172.21.15.67:0/273760495 shutdown_connections 2024-03-09T09:25:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.165+0000 7f4e7e400640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e5806b4d0 0x7f4e5806d990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.165+0000 7f4e7e400640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e781557e0 0x7f4e78155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.165+0000 7f4e7e400640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e7814ccc0 0x7f4e781ef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.165+0000 7f4e7e400640 1 --2- 172.21.15.67:0/273760495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e780c5f60 0x7f4e7814c620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.165+0000 7f4e7e400640 1 -- 172.21.15.67:0/273760495 >> 172.21.15.67:0/273760495 conn(0x7f4e780c8060 msgr2=0x7f4e780ca520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:45.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.165+0000 7f4e7e400640 1 -- 172.21.15.67:0/273760495 shutdown_connections 2024-03-09T09:25:45.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.165+0000 7f4e7e400640 1 -- 172.21.15.67:0/273760495 wait complete. 2024-03-09T09:25:45.189 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:25:45.189 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:25:45.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:25:45.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.402+0000 7fe004c00640 1 Processor -- start 2024-03-09T09:25:45.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.403+0000 7fe004c00640 1 -- start start 2024-03-09T09:25:45.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.404+0000 7fe004c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0000c9380 0x7fe0000acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.404+0000 7fe004c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe0000ad440 0x7fe00006c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.404+0000 7fe004c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0000c9890 con 0x7fe00014fd90 2024-03-09T09:25:45.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.404+0000 7fe004c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00006ba60 con 0x7fe0000ad440 2024-03-09T09:25:45.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.404+0000 7fe004c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000c5f80 con 0x7fe0000c9380 2024-03-09T09:25:45.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.404+0000 7fdfffe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe0000ad440 0x7fe00006c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.405+0000 7fdfffe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe0000ad440 0x7fe00006c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49730/0 (socket says 172.21.15.67:49730) 2024-03-09T09:25:45.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.405+0000 7fdfffe00640 1 -- 172.21.15.67:0/1440589514 learned_addr learned my addr 172.21.15.67:0/1440589514 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:45.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.405+0000 7fdffea00640 1 --2- 172.21.15.67:0/1440589514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0000c9380 0x7fe0000acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.405+0000 7fdfffe00640 1 -- 172.21.15.67:0/1440589514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0000c9380 msgr2=0x7fe0000acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:45.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.405+0000 7fdfffe00640 1 --2- 172.21.15.67:0/1440589514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0000c9380 0x7fe0000acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.405+0000 7fdfffe00640 1 -- 172.21.15.67:0/1440589514 >> v1:172.21.15.134:6789/0 conn(0x7fe00014fd90 legacy=0x7fe000156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.405+0000 7fdfffe00640 1 -- 172.21.15.67:0/1440589514 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0000c6240 con 0x7fe0000ad440 2024-03-09T09:25:45.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.406+0000 7fdfffe00640 1 --2- 172.21.15.67:0/1440589514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000ad440 0x7fe00006c1d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe00006bbe0 tx=0x7fdff802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbded72bd6898b36 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.406+0000 7fdffe000640 1 -- 172.21.15.67:0/1440589514 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff802c9c0 con 0x7fe0000ad440 2024-03-09T09:25:45.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.407+0000 7fdffe000640 1 -- 172.21.15.67:0/1440589514 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff802cb60 con 0x7fe0000ad440 2024-03-09T09:25:45.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.407+0000 7fdffe000640 1 -- 172.21.15.67:0/1440589514 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff80343f0 con 0x7fe0000ad440 2024-03-09T09:25:45.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.407+0000 7fe004c00640 1 -- 172.21.15.67:0/1440589514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000ad440 msgr2=0x7fe00006c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.407+0000 7fe004c00640 1 --2- 172.21.15.67:0/1440589514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000ad440 0x7fe00006c1d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe00006bbe0 tx=0x7fdff802be20 comp rx=0 tx=0).stop 2024-03-09T09:25:45.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.407+0000 7fe004c00640 1 -- 172.21.15.67:0/1440589514 shutdown_connections 2024-03-09T09:25:45.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.407+0000 7fe004c00640 1 --2- 172.21.15.67:0/1440589514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000ad440 0x7fe00006c1d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.407+0000 7fe004c00640 1 --2- 172.21.15.67:0/1440589514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0000c9380 0x7fe0000acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.407+0000 7fe004c00640 1 -- 172.21.15.67:0/1440589514 >> 172.21.15.67:0/1440589514 conn(0x7fe0000cb480 msgr2=0x7fe0000cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:45.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.407+0000 7fe004c00640 1 -- 172.21.15.67:0/1440589514 shutdown_connections 2024-03-09T09:25:45.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.408+0000 7fe004c00640 1 -- 172.21.15.67:0/1440589514 wait complete. 2024-03-09T09:25:45.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.409+0000 7fe004c00640 1 Processor -- start 2024-03-09T09:25:45.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.409+0000 7fe004c00640 1 -- start start 2024-03-09T09:25:45.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.410+0000 7fe004c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000ad440 0x7fe0001ef100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.410+0000 7fe004c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0000c9380 0x7fe0001579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.410+0000 7fe004c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe00014fd90 0x7fe000152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.410+0000 7fe004c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00006c710 con 0x7fe0000ad440 2024-03-09T09:25:45.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.410+0000 7fe004c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00006bcb0 con 0x7fe0000c9380 2024-03-09T09:25:45.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.410+0000 7fe004c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00006c890 con 0x7fe00014fd90 2024-03-09T09:25:45.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.410+0000 7fdfff400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000ad440 0x7fe0001ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.410+0000 7fdfff400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000ad440 0x7fe0001ef100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49734/0 (socket says 172.21.15.67:49734) 2024-03-09T09:25:45.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.410+0000 7fdfff400640 1 -- 172.21.15.67:0/2263205412 learned_addr learned my addr 172.21.15.67:0/2263205412 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:45.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.411+0000 7fdfffe00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe00014fd90 0x7fe000152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.411+0000 7fdfff400640 1 -- 172.21.15.67:0/2263205412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe00014fd90 msgr2=0x7fe000152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.411+0000 7fdfff400640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe00014fd90 0x7fe000152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.411+0000 7fdfff400640 1 -- 172.21.15.67:0/2263205412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0000c9380 msgr2=0x7fe0001579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.411+0000 7fdffea00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0000c9380 0x7fe0001579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.411+0000 7fdfff400640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0000c9380 0x7fe0001579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.411+0000 7fdfff400640 1 -- 172.21.15.67:0/2263205412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00014c920 con 0x7fe0000ad440 2024-03-09T09:25:45.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.411+0000 7fdfff400640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000ad440 0x7fe0001ef100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c400 tx=0x7fdff000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.411+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0017070 con 0x7fe0000ad440 2024-03-09T09:25:45.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.412+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff00052d0 con 0x7fe0000ad440 2024-03-09T09:25:45.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.412+0000 7fe004c00640 1 -- 172.21.15.67:0/2263205412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0001efab0 con 0x7fe0000ad440 2024-03-09T09:25:45.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.412+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0003830 con 0x7fe0000ad440 2024-03-09T09:25:45.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.412+0000 7fe004c00640 1 -- 172.21.15.67:0/2263205412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00014cde0 con 0x7fe0000ad440 2024-03-09T09:25:45.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.413+0000 7fe004c00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000c1a50 0x7fe000152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.413+0000 7fe004c00640 1 -- 172.21.15.67:0/2263205412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00004f3c0 con 0x7fe0000c1a50 2024-03-09T09:25:45.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.413+0000 7fdffea00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000c1a50 0x7fe000152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.414+0000 7fdffea00640 1 -- 172.21.15.67:0/2263205412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe00006d2d0 con 0x7fe0000c1a50 2024-03-09T09:25:45.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.414+0000 7fdffea00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000c1a50 0x7fe000152590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe0000c6ba0 tx=0x7fdfec00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.414+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec017070 con 0x7fe0000c1a50 2024-03-09T09:25:45.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.414+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdff001a070 con 0x7fe0000ad440 2024-03-09T09:25:45.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.415+0000 7fdff7400640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdfe006b3d0 0x7fdfe006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.416+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fdff0089850 con 0x7fe0000ad440 2024-03-09T09:25:45.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.416+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fdfec00c710 con 0x7fe0000c1a50 2024-03-09T09:25:45.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.416+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000c1a50 msgr2=0x7fe000152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.416+0000 7fdff7400640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000c1a50 0x7fe000152590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe0000c6ba0 tx=0x7fdfec00eba0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.416+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000c1a50 msgr2=0x7fe000152590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:45.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.416+0000 7fdff7400640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000c1a50 0x7fe000152590 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.416+0000 7fdfffe00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdfe006b3d0 0x7fdfe006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.417+0000 7fdfffe00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdfe006b3d0 0x7fdfe006d890 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdff8035000 tx=0x7fdff8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.424+0000 7fe004c00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe000157f00 0x7fe0001ec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.424+0000 7fe004c00640 1 -- 172.21.15.67:0/2263205412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00006c4d0 con 0x7fe000157f00 2024-03-09T09:25:45.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.424+0000 7fdffea00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe000157f00 0x7fe0001ec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.424+0000 7fdffea00640 1 -- 172.21.15.67:0/2263205412 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe00014f450 con 0x7fe000157f00 2024-03-09T09:25:45.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.425+0000 7fdffea00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe000157f00 0x7fe0001ec0e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe00004f540 tx=0x7fdfec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.425+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec00f040 con 0x7fe000157f00 2024-03-09T09:25:45.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.425+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4876 (secure 0 0 0) 0x7fdfec003af0 con 0x7fe000157f00 2024-03-09T09:25:45.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.425+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe000157f00 msgr2=0x7fe0001ec0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.425+0000 7fdff7400640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe000157f00 0x7fe0001ec0e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe00004f540 tx=0x7fdfec0046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.425+0000 7fdff7400640 1 -- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe000157f00 msgr2=0x7fe0001ec0e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:45.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.425+0000 7fdff7400640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe000157f00 0x7fe0001ec0e0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe00004f540 tx=0x7fdfec0046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.434 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:25:45.434 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:45.434 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:45.434 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:25:45.434 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 262, 2024-03-09T09:25:45.434 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:25:45.434 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:25:45.434 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:45.435 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:45.436 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:45.436 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:45.436 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:45.436 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.436 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:25:45.436 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:25:45.436 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:45.436 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:45.436 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:45.436 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:45.436 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:25:45.437 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:25:45.438 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:45.438 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:45.438 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:25:45.438 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:25:45.438 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:25:45.438 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:25:45.438 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:25:45.438 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:25:45.438 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:25:45.438 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:45.439 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:45.440 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.440 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:25:45.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.440 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:25:45.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.440 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:45.440 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:45.440 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:45.440 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:45.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:45.441 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:25:45.441 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.441 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:45.441 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:25:45.441 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.441 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.441 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.442 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:25:45.442 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:25:45.442 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:45.442 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:45.442 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:45.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.442 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:45.442 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:45.442 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:45.443 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:45.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.443 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:45.443 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:25:45.443 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.444 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:45.444 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:25:45.444 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.444 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.444 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.444 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:25:45.444 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:25:45.444 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:45.445 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:45.445 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:45.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.445 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:45.445 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:45.445 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:45.445 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:45.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:25:45.446 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.447 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.448 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.449 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:25:45.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.449 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:45.449 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.449 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:25:45.449 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.449 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.449 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:25:45.449 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:25:45.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.426+0000 7fe004c00640 1 -- 172.21.15.67:0/2263205412 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdfe006b3d0 msgr2=0x7fdfe006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.426+0000 7fe004c00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdfe006b3d0 0x7fdfe006d890 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdff8035000 tx=0x7fdff8030000 comp rx=0 tx=0).stop 2024-03-09T09:25:45.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.426+0000 7fe004c00640 1 -- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000ad440 msgr2=0x7fe0001ef100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.426+0000 7fe004c00640 1 --2- 172.21.15.67:0/2263205412 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0000ad440 0x7fe0001ef100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdff000c400 tx=0x7fdff000c8c0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.426+0000 7fdfffe00640 1 -- 172.21.15.67:0/2263205412 reap_dead start 2024-03-09T09:25:45.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.426+0000 7fdfffe00640 1 -- 172.21.15.67:0/2263205412 reap_dead start 2024-03-09T09:25:45.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.426+0000 7fe004c00640 1 -- 172.21.15.67:0/2263205412 shutdown_connections 2024-03-09T09:25:45.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.426+0000 7fe004c00640 1 -- 172.21.15.67:0/2263205412 >> 172.21.15.67:0/2263205412 conn(0x7fe0000cb480 msgr2=0x7fe000155020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:45.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.426+0000 7fe004c00640 1 -- 172.21.15.67:0/2263205412 shutdown_connections 2024-03-09T09:25:45.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.426+0000 7fe004c00640 1 -- 172.21.15.67:0/2263205412 wait complete. 2024-03-09T09:25:45.459 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:25:45.459 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:25:45.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:25:45.581 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:45.582 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 28 -eq 48 ']' 2024-03-09T09:25:45.582 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.28 2024-03-09T09:25:45.582 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.28 2024-03-09T09:25:45.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.663+0000 7f56fca00640 1 Processor -- start 2024-03-09T09:25:45.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.663+0000 7f56fca00640 1 -- start start 2024-03-09T09:25:45.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.664+0000 7f56fca00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f56f80ac9a0 0x7f56f81510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.664+0000 7f56fca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f56f8071190 0x7f56f8151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.664+0000 7f56fca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f80c8200 con 0x7f56f80c2780 2024-03-09T09:25:45.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.664+0000 7f56fca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f806d590 con 0x7f56f8071190 2024-03-09T09:25:45.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.664+0000 7f56fca00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f80c3740 con 0x7f56f80ac9a0 2024-03-09T09:25:45.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.665+0000 7f56f6800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f56f8071190 0x7f56f8151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.665+0000 7f56f5400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f56f80c2780 0x7f56f8154190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47340/0 (socket says 172.21.15.67:47340) 2024-03-09T09:25:45.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.665+0000 7f56f5400640 1 -- 172.21.15.67:0/1361689120 learned_addr learned my addr 172.21.15.67:0/1361689120 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:45.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.665+0000 7f56f5e00640 1 --2- 172.21.15.67:0/1361689120 >> v2:172.21.15.134:3300/0 conn(0x7f56f80ac9a0 0x7f56f81510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.665+0000 7f56f6800640 1 -- 172.21.15.67:0/1361689120 >> v2:172.21.15.134:3300/0 conn(0x7f56f80ac9a0 msgr2=0x7f56f81510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:45.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.665+0000 7f56f6800640 1 --2- 172.21.15.67:0/1361689120 >> v2:172.21.15.134:3300/0 conn(0x7f56f80ac9a0 0x7f56f81510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.665+0000 7f56f6800640 1 -- 172.21.15.67:0/1361689120 >> v1:172.21.15.67:6789/0 conn(0x7f56f80c2780 legacy=0x7f56f8154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.665+0000 7f56f6800640 1 -- 172.21.15.67:0/1361689120 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f80c3990 con 0x7f56f8071190 2024-03-09T09:25:45.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.666+0000 7f56f6800640 1 --2- 172.21.15.67:0/1361689120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56f8071190 0x7f56f8151620 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f56f806ed40 tx=0x7f56f002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca328f6852030bf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.667+0000 7f56f4a00640 1 -- 172.21.15.67:0/1361689120 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0039070 con 0x7f56f8071190 2024-03-09T09:25:45.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.667+0000 7f56f4a00640 1 -- 172.21.15.67:0/1361689120 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f002cbe0 con 0x7f56f8071190 2024-03-09T09:25:45.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.667+0000 7f56f4a00640 1 -- 172.21.15.67:0/1361689120 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0034440 con 0x7f56f8071190 2024-03-09T09:25:45.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.667+0000 7f56fca00640 1 -- 172.21.15.67:0/1361689120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56f8071190 msgr2=0x7f56f8151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.667+0000 7f56fca00640 1 --2- 172.21.15.67:0/1361689120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56f8071190 0x7f56f8151620 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f56f806ed40 tx=0x7f56f002be30 comp rx=0 tx=0).stop 2024-03-09T09:25:45.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.667+0000 7f56fca00640 1 -- 172.21.15.67:0/1361689120 shutdown_connections 2024-03-09T09:25:45.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.667+0000 7f56fca00640 1 --2- 172.21.15.67:0/1361689120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56f8071190 0x7f56f8151620 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.667+0000 7f56fca00640 1 --2- 172.21.15.67:0/1361689120 >> v2:172.21.15.134:3300/0 conn(0x7f56f80ac9a0 0x7f56f81510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.667+0000 7f56fca00640 1 -- 172.21.15.67:0/1361689120 >> 172.21.15.67:0/1361689120 conn(0x7f56f80c7370 msgr2=0x7f56f80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:45.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.668+0000 7f56fca00640 1 -- 172.21.15.67:0/1361689120 shutdown_connections 2024-03-09T09:25:45.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.668+0000 7f56fca00640 1 -- 172.21.15.67:0/1361689120 wait complete. 2024-03-09T09:25:45.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.669+0000 7f56fca00640 1 Processor -- start 2024-03-09T09:25:45.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.669+0000 7f56fca00640 1 -- start start 2024-03-09T09:25:45.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.670+0000 7f56fca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56f8071190 0x7f56f80abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.670+0000 7f56fca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56f80ac9a0 0x7f56f8148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.670+0000 7f56fca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f80c2780 0x7f56f813a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.670+0000 7f56fca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80acd80 con 0x7f56f8071190 2024-03-09T09:25:45.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.670+0000 7f56fca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8070c00 con 0x7f56f80c2780 2024-03-09T09:25:45.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.670+0000 7f56fca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80acf00 con 0x7f56f80ac9a0 2024-03-09T09:25:45.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.670+0000 7f56f5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56f8071190 0x7f56f80abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.671+0000 7f56f5e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56f8071190 0x7f56f80abc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49776/0 (socket says 172.21.15.67:49776) 2024-03-09T09:25:45.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.671+0000 7f56f5e00640 1 -- 172.21.15.67:0/4204416523 learned_addr learned my addr 172.21.15.67:0/4204416523 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:45.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.671+0000 7f56f6800640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f80c2780 0x7f56f813a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.671+0000 7f56f5400640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56f80ac9a0 0x7f56f8148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.671+0000 7f56f5e00640 1 -- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56f80ac9a0 msgr2=0x7f56f8148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.671+0000 7f56f5e00640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56f80ac9a0 0x7f56f8148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.671+0000 7f56f5e00640 1 -- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f80c2780 msgr2=0x7f56f813a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.671+0000 7f56f5e00640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f80c2780 0x7f56f813a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.671+0000 7f56f5e00640 1 -- 172.21.15.67:0/4204416523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f81ebed0 con 0x7f56f8071190 2024-03-09T09:25:45.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.671+0000 7f56f5400640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56f80ac9a0 0x7f56f8148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:25:45.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.671+0000 7f56f5e00640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56f8071190 0x7f56f80abc40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f56e800d830 tx=0x7f56e800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.672+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e800b6c0 con 0x7f56f8071190 2024-03-09T09:25:45.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.672+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e8019070 con 0x7f56f8071190 2024-03-09T09:25:45.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.672+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e800ba80 con 0x7f56f8071190 2024-03-09T09:25:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.672+0000 7f56fca00640 1 -- 172.21.15.67:0/4204416523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f81ec1b0 con 0x7f56f8071190 2024-03-09T09:25:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.672+0000 7f56fca00640 1 -- 172.21.15.67:0/4204416523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f81ec770 con 0x7f56f8071190 2024-03-09T09:25:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.673+0000 7f56fca00640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81491e0 0x7f56f813a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.673+0000 7f56fca00640 1 -- 172.21.15.67:0/4204416523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8075a00 con 0x7f56f81491e0 2024-03-09T09:25:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.673+0000 7f56f5400640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81491e0 0x7f56f813a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.674+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f56e801e460 con 0x7f56f8071190 2024-03-09T09:25:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.674+0000 7f56f5400640 1 -- 172.21.15.67:0/4204416523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56f80ad2e0 con 0x7f56f81491e0 2024-03-09T09:25:45.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.675+0000 7f56f5400640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81491e0 0x7f56f813a770 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f56f80ac700 tx=0x7f56e400d880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.675+0000 7f56ee000640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f56d806b520 0x7f56d806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.675+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f56e808ac20 con 0x7f56f8071190 2024-03-09T09:25:45.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.675+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4005ce0 con 0x7f56f81491e0 2024-03-09T09:25:45.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.676+0000 7f56f6800640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f56d806b520 0x7f56d806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.676+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f56e4005ce0 con 0x7f56f81491e0 2024-03-09T09:25:45.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.676+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81491e0 msgr2=0x7f56f813a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.676+0000 7f56ee000640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81491e0 0x7f56f813a770 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f56f80ac700 tx=0x7f56e400d880 comp rx=0 tx=0).stop 2024-03-09T09:25:45.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.676+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81491e0 msgr2=0x7f56f813a770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.676+0000 7f56ee000640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81491e0 0x7f56f813a770 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f56f80ac700 tx=0x7f56e400d880 comp rx=0 tx=0).stop 2024-03-09T09:25:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.677+0000 7f56f6800640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f56d806b520 0x7f56d806d9e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f56f0036000 tx=0x7f56f0030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.684+0000 7f56fca00640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81ef3d0 0x7f56f81ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.684+0000 7f56fca00640 1 -- 172.21.15.67:0/4204416523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80c48b0 con 0x7f56f81ef3d0 2024-03-09T09:25:45.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.684+0000 7f56f5400640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81ef3d0 0x7f56f81ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.685+0000 7f56f5400640 1 -- 172.21.15.67:0/4204416523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f56f80c7bd0 con 0x7f56f81ef3d0 2024-03-09T09:25:45.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.685+0000 7f56f5400640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81ef3d0 0x7f56f81ef7e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f56e4014000 tx=0x7f56e4000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.686+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4004cc0 con 0x7f56f81ef3d0 2024-03-09T09:25:45.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.686+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f56e4004cc0 con 0x7f56f81ef3d0 2024-03-09T09:25:45.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.686+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81ef3d0 msgr2=0x7f56f81ef7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.686+0000 7f56ee000640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81ef3d0 0x7f56f81ef7e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f56e4014000 tx=0x7f56e4000ed0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.686+0000 7f56ee000640 1 -- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81ef3d0 msgr2=0x7f56f81ef7e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:45.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.686+0000 7f56ee000640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56f81ef3d0 0x7f56f81ef7e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.694 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:25:45.694 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:45.695 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:45.695 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:25:45.695 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 262, 2024-03-09T09:25:45.695 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:25:45.695 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:25:45.695 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:25:45.695 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:25:45.695 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.695 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:45.696 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:45.697 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:45.697 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:45.697 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.697 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:25:45.697 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:25:45.697 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:45.697 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:45.697 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:45.698 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:45.698 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:45.698 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:45.698 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:45.698 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:45.698 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:45.698 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:45.698 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:45.698 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.698 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:25:45.699 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:25:45.699 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:25:45.699 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:25:45.699 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:25:45.699 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:25:45.699 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:45.699 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:45.699 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:25:45.699 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:25:45.699 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:25:45.700 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:25:45.700 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:25:45.700 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:25:45.700 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:25:45.700 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:45.700 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:25:45.700 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:45.700 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:45.701 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:45.701 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:45.701 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:45.701 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:45.701 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:45.701 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:45.701 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:45.701 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:45.702 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:45.702 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.702 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:25:45.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.702 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:25:45.702 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.702 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:45.703 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:45.703 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:45.703 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:45.703 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.703 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:45.703 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:25:45.703 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.703 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.704 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:45.704 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:25:45.704 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.704 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.704 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.704 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:25:45.704 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:25:45.704 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:45.704 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:45.704 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.705 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:45.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.707 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:45.708 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:25:45.708 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.708 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.708 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.708 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:25:45.708 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:25:45.708 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:45.708 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:45.708 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:45.708 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.709 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.709 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:25:45.709 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:25:45.709 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.709 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:45.709 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.710 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:45.710 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.710 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.710 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:25:45.710 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.710 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:45.710 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.710 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:45.710 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.710 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.711 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:25:45.711 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:25:45.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.687+0000 7f56fca00640 1 -- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f56d806b520 msgr2=0x7f56d806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.687+0000 7f56fca00640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f56d806b520 0x7f56d806d9e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f56f0036000 tx=0x7f56f0030000 comp rx=0 tx=0).stop 2024-03-09T09:25:45.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.687+0000 7f56fca00640 1 -- 172.21.15.67:0/4204416523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56f8071190 msgr2=0x7f56f80abc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.687+0000 7f56fca00640 1 --2- 172.21.15.67:0/4204416523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56f8071190 0x7f56f80abc40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f56e800d830 tx=0x7f56e800dcf0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.687+0000 7f56f6800640 1 -- 172.21.15.67:0/4204416523 reap_dead start 2024-03-09T09:25:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.687+0000 7f56f6800640 1 -- 172.21.15.67:0/4204416523 reap_dead start 2024-03-09T09:25:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.687+0000 7f56fca00640 1 -- 172.21.15.67:0/4204416523 shutdown_connections 2024-03-09T09:25:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.687+0000 7f56fca00640 1 -- 172.21.15.67:0/4204416523 >> 172.21.15.67:0/4204416523 conn(0x7f56f80c7370 msgr2=0x7f56f80c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.687+0000 7f56fca00640 1 -- 172.21.15.67:0/4204416523 shutdown_connections 2024-03-09T09:25:45.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.687+0000 7f56fca00640 1 -- 172.21.15.67:0/4204416523 wait complete. 2024-03-09T09:25:45.718 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:25:45.718 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:25:45.718 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:25:45.858 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:45.858 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 29 -eq 48 ']' 2024-03-09T09:25:45.858 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.29 2024-03-09T09:25:45.858 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.29 2024-03-09T09:25:45.901 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:45.901 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 30 -eq 48 ']' 2024-03-09T09:25:45.901 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.30 2024-03-09T09:25:45.901 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.30 2024-03-09T09:25:45.933 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:45.933 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 31 -eq 48 ']' 2024-03-09T09:25:45.934 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.31 2024-03-09T09:25:45.934 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.31 2024-03-09T09:25:45.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.944+0000 7fa321200640 1 Processor -- start 2024-03-09T09:25:45.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.944+0000 7fa321200640 1 -- start start 2024-03-09T09:25:45.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.945+0000 7fa321200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c0734a0 0x7fa31c0aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.945+0000 7fa321200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa31c06d830 0x7fa31c06dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.945+0000 7fa321200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31c152e00 con 0x7fa31c0730c0 2024-03-09T09:25:45.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.945+0000 7fa321200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c0aa960 con 0x7fa31c06d830 2024-03-09T09:25:45.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.945+0000 7fa321200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c1530e0 con 0x7fa31c0734a0 2024-03-09T09:25:45.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.946+0000 7fa319e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa31c0730c0 0x7fa31c0c2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47350/0 (socket says 172.21.15.67:47350) 2024-03-09T09:25:45.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.946+0000 7fa319e00640 1 -- 172.21.15.67:0/2104469150 learned_addr learned my addr 172.21.15.67:0/2104469150 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:45.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.946+0000 7fa319400640 1 --2- 172.21.15.67:0/2104469150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c0734a0 0x7fa31c0aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.946+0000 7fa31a800640 1 --2- 172.21.15.67:0/2104469150 >> v2:172.21.15.134:3300/0 conn(0x7fa31c06d830 0x7fa31c06dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.946+0000 7fa318a00640 1 -- 172.21.15.67:0/2104469150 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067996850 0 0) 0x7fa31c152e00 con 0x7fa31c0730c0 2024-03-09T09:25:45.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.946+0000 7fa318a00640 1 -- 172.21.15.67:0/2104469150 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa300003520 con 0x7fa31c0730c0 2024-03-09T09:25:45.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.946+0000 7fa31a800640 1 -- 172.21.15.67:0/2104469150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c0734a0 msgr2=0x7fa31c0aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:45.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.946+0000 7fa31a800640 1 --2- 172.21.15.67:0/2104469150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c0734a0 0x7fa31c0aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.946+0000 7fa31a800640 1 -- 172.21.15.67:0/2104469150 >> v1:172.21.15.67:6789/0 conn(0x7fa31c0730c0 legacy=0x7fa31c0c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.946+0000 7fa31a800640 1 -- 172.21.15.67:0/2104469150 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c1533c0 con 0x7fa31c06d830 2024-03-09T09:25:45.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.946+0000 7fa319400640 1 --2- 172.21.15.67:0/2104469150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c0734a0 0x7fa31c0aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:25:45.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.948+0000 7fa31a800640 1 --2- 172.21.15.67:0/2104469150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa31c06d830 0x7fa31c06dcc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa314009d90 tx=0x7fa31402be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad768221c8f35aa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.948+0000 7fa318a00640 1 -- 172.21.15.67:0/2104469150 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa314039070 con 0x7fa31c06d830 2024-03-09T09:25:45.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.948+0000 7fa318a00640 1 -- 172.21.15.67:0/2104469150 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31402cbd0 con 0x7fa31c06d830 2024-03-09T09:25:45.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.948+0000 7fa318a00640 1 -- 172.21.15.67:0/2104469150 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa314034440 con 0x7fa31c06d830 2024-03-09T09:25:45.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.948+0000 7fa321200640 1 -- 172.21.15.67:0/2104469150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa31c06d830 msgr2=0x7fa31c06dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.948+0000 7fa321200640 1 --2- 172.21.15.67:0/2104469150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa31c06d830 0x7fa31c06dcc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa314009d90 tx=0x7fa31402be20 comp rx=0 tx=0).stop 2024-03-09T09:25:45.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.949+0000 7fa321200640 1 -- 172.21.15.67:0/2104469150 shutdown_connections 2024-03-09T09:25:45.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.949+0000 7fa321200640 1 --2- 172.21.15.67:0/2104469150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c0734a0 0x7fa31c0aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.949+0000 7fa321200640 1 --2- 172.21.15.67:0/2104469150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa31c06d830 0x7fa31c06dcc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.949+0000 7fa321200640 1 -- 172.21.15.67:0/2104469150 >> 172.21.15.67:0/2104469150 conn(0x7fa31c0c7370 msgr2=0x7fa31c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:45.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.949+0000 7fa321200640 1 -- 172.21.15.67:0/2104469150 shutdown_connections 2024-03-09T09:25:45.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.949+0000 7fa321200640 1 -- 172.21.15.67:0/2104469150 wait complete. 2024-03-09T09:25:45.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.950+0000 7fa321200640 1 Processor -- start 2024-03-09T09:25:45.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.950+0000 7fa321200640 1 -- start start 2024-03-09T09:25:45.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.951+0000 7fa321200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa31c06d830 0x7fa31c0c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.951+0000 7fa321200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c0730c0 0x7fa31c145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.951+0000 7fa321200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa31c0734a0 0x7fa31c145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.951+0000 7fa321200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c0c3590 con 0x7fa31c0734a0 2024-03-09T09:25:45.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.951+0000 7fa321200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c153980 con 0x7fa31c06d830 2024-03-09T09:25:45.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.951+0000 7fa321200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c0c3710 con 0x7fa31c0730c0 2024-03-09T09:25:45.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.952+0000 7fa31a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa31c0734a0 0x7fa31c145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.952+0000 7fa31a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa31c0734a0 0x7fa31c145680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49790/0 (socket says 172.21.15.67:49790) 2024-03-09T09:25:45.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.952+0000 7fa31a800640 1 -- 172.21.15.67:0/1816815105 learned_addr learned my addr 172.21.15.67:0/1816815105 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:45.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.952+0000 7fa319400640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c0730c0 0x7fa31c145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.952+0000 7fa319e00640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa31c06d830 0x7fa31c0c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.952+0000 7fa319400640 1 -- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa31c06d830 msgr2=0x7fa31c0c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.952+0000 7fa319400640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa31c06d830 0x7fa31c0c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.952+0000 7fa319400640 1 -- 172.21.15.67:0/1816815105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa31c0734a0 msgr2=0x7fa31c145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.952+0000 7fa319400640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa31c0734a0 0x7fa31c145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.952+0000 7fa319400640 1 -- 172.21.15.67:0/1816815105 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c152530 con 0x7fa31c0730c0 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.952+0000 7fa31a800640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa31c0734a0 0x7fa31c145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.953+0000 7fa319400640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c0730c0 0x7fa31c145140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e870 tx=0x7fa30800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.953+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308017070 con 0x7fa31c0730c0 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.953+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30800c940 con 0x7fa31c0730c0 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.953+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30800cd10 con 0x7fa31c0730c0 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.953+0000 7fa321200640 1 -- 172.21.15.67:0/1816815105 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c152810 con 0x7fa31c0730c0 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.953+0000 7fa321200640 1 -- 172.21.15.67:0/1816815105 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c143bf0 con 0x7fa31c0730c0 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.954+0000 7fa321200640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c1ef2b0 0x7fa31c150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.954+0000 7fa321200640 1 -- 172.21.15.67:0/1816815105 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c143700 con 0x7fa31c1ef2b0 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.955+0000 7fa319e00640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c1ef2b0 0x7fa31c150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.955+0000 7fa319e00640 1 -- 172.21.15.67:0/1816815105 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa31c075a00 con 0x7fa31c1ef2b0 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.955+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa30801a070 con 0x7fa31c0730c0 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.956+0000 7fa319e00640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c1ef2b0 0x7fa31c150890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa30c005880 tx=0x7fa30c00e960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.956+0000 7fa312000640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2fc06b4d0 0x7fa2fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.956+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fa30808a5e0 con 0x7fa31c0730c0 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.957+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c005ac0 con 0x7fa31c1ef2b0 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.957+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa30c018070 con 0x7fa31c1ef2b0 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.957+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c1ef2b0 msgr2=0x7fa31c150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.957+0000 7fa312000640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c1ef2b0 0x7fa31c150890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa30c005880 tx=0x7fa30c00e960 comp rx=0 tx=0).stop 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.957+0000 7fa31a800640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2fc06b4d0 0x7fa2fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.957+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c1ef2b0 msgr2=0x7fa31c150890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.957+0000 7fa312000640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c1ef2b0 0x7fa31c150890 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa30c005880 tx=0x7fa30c00e960 comp rx=0 tx=0).stop 2024-03-09T09:25:45.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.958+0000 7fa31a800640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2fc06b4d0 0x7fa2fc06d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa314009bd0 tx=0x7fa314030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.969 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:45.969 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 32 -eq 48 ']' 2024-03-09T09:25:45.969 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.32 2024-03-09T09:25:45.969 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.32 2024-03-09T09:25:45.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.966+0000 7fa321200640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c145bc0 0x7fa31c150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:45.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.966+0000 7fa321200640 1 -- 172.21.15.67:0/1816815105 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c151930 con 0x7fa31c145bc0 2024-03-09T09:25:45.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.966+0000 7fa319e00640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c145bc0 0x7fa31c150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:45.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.966+0000 7fa319e00640 1 -- 172.21.15.67:0/1816815105 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa31c0c6f10 con 0x7fa31c145bc0 2024-03-09T09:25:45.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.967+0000 7fa319e00640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c145bc0 0x7fa31c150dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa30c017000 tx=0x7fa30c0010a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:45.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.967+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c01d7c0 con 0x7fa31c145bc0 2024-03-09T09:25:45.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7fa30c01d7c0 con 0x7fa31c145bc0 2024-03-09T09:25:45.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c145bc0 msgr2=0x7fa31c150dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa312000640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c145bc0 0x7fa31c150dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa30c017000 tx=0x7fa30c0010a0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa312000640 1 -- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c145bc0 msgr2=0x7fa31c150dd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:45.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa312000640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c145bc0 0x7fa31c150dd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:45.975 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:25:45.975 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:45.975 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:45.975 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:25:45.975 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 262, 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:45.976 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:45.977 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:25:45.978 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:45.979 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.980 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.981 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.982 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:45.983 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:45.984 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:25:45.985 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.986 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.987 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:25:45.988 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:25:45.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa321200640 1 -- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2fc06b4d0 msgr2=0x7fa2fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa321200640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa2fc06b4d0 0x7fa2fc06d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa314009bd0 tx=0x7fa314030000 comp rx=0 tx=0).stop 2024-03-09T09:25:45.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa321200640 1 -- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c0730c0 msgr2=0x7fa31c145140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:45.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa321200640 1 --2- 172.21.15.67:0/1816815105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa31c0730c0 0x7fa31c145140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e870 tx=0x7fa30800ed30 comp rx=0 tx=0).stop 2024-03-09T09:25:45.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa31a800640 1 -- 172.21.15.67:0/1816815105 reap_dead start 2024-03-09T09:25:45.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa31a800640 1 -- 172.21.15.67:0/1816815105 reap_dead start 2024-03-09T09:25:45.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa321200640 1 -- 172.21.15.67:0/1816815105 shutdown_connections 2024-03-09T09:25:45.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa321200640 1 -- 172.21.15.67:0/1816815105 >> 172.21.15.67:0/1816815105 conn(0x7fa31c0c7370 msgr2=0x7fa31c0aaed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:45.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa321200640 1 -- 172.21.15.67:0/1816815105 shutdown_connections 2024-03-09T09:25:45.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:45.968+0000 7fa321200640 1 -- 172.21.15.67:0/1816815105 wait complete. 2024-03-09T09:25:45.990 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:45.991 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 33 -eq 48 ']' 2024-03-09T09:25:45.991 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.33 2024-03-09T09:25:45.991 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.33 2024-03-09T09:25:45.993 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:25:45.993 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:25:45.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:25:46.012 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.013 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 34 -eq 48 ']' 2024-03-09T09:25:46.013 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.34 2024-03-09T09:25:46.013 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.34 2024-03-09T09:25:46.038 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.038 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 35 -eq 48 ']' 2024-03-09T09:25:46.038 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.35 2024-03-09T09:25:46.038 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.35 2024-03-09T09:25:46.067 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.067 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 36 -eq 48 ']' 2024-03-09T09:25:46.067 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.36 2024-03-09T09:25:46.067 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.36 2024-03-09T09:25:46.105 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.105 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 37 -eq 48 ']' 2024-03-09T09:25:46.105 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.37 2024-03-09T09:25:46.105 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.37 2024-03-09T09:25:46.133 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.134 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 38 -eq 48 ']' 2024-03-09T09:25:46.134 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.38 2024-03-09T09:25:46.134 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.38 2024-03-09T09:25:46.167 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.168 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 39 -eq 48 ']' 2024-03-09T09:25:46.168 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.39 2024-03-09T09:25:46.168 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.39 2024-03-09T09:25:46.196 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.196 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 40 -eq 48 ']' 2024-03-09T09:25:46.196 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.40 2024-03-09T09:25:46.196 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.40 2024-03-09T09:25:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.192+0000 7f523f200640 1 Processor -- start 2024-03-09T09:25:46.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.193+0000 7f523f200640 1 -- start start 2024-03-09T09:25:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.194+0000 7f523f200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5238153300 0x7f52380b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.194+0000 7f523f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f52380c5e40 0x7f52380b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.194+0000 7f523f200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5238157000 con 0x7f52380cadb0 2024-03-09T09:25:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.194+0000 7f523f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52380ae430 con 0x7f52380c5e40 2024-03-09T09:25:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.194+0000 7f523f200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52380ae6f0 con 0x7f5238153300 2024-03-09T09:25:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.194+0000 7f523d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f52380c5e40 0x7f52380b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.194+0000 7f523d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f52380c5e40 0x7f52380b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49802/0 (socket says 172.21.15.67:49802) 2024-03-09T09:25:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.194+0000 7f523d400640 1 -- 172.21.15.67:0/2962485863 learned_addr learned my addr 172.21.15.67:0/2962485863 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.195+0000 7f523de00640 1 --2- 172.21.15.67:0/2962485863 >> v2:172.21.15.134:3300/0 conn(0x7f5238153300 0x7f52380b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.195+0000 7f523d400640 1 -- 172.21.15.67:0/2962485863 >> v2:172.21.15.134:3300/0 conn(0x7f5238153300 msgr2=0x7f52380b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.195+0000 7f523d400640 1 --2- 172.21.15.67:0/2962485863 >> v2:172.21.15.134:3300/0 conn(0x7f5238153300 0x7f52380b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.195+0000 7f523d400640 1 -- 172.21.15.67:0/2962485863 >> v1:172.21.15.134:6789/0 conn(0x7f52380cadb0 legacy=0x7f5238156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.195+0000 7f523d400640 1 -- 172.21.15.67:0/2962485863 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5238155b10 con 0x7f52380c5e40 2024-03-09T09:25:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.197+0000 7f523d400640 1 --2- 172.21.15.67:0/2962485863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52380c5e40 0x7f52380b0540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f52380ade40 tx=0x7f522802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a31e0b0f89063f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:46.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.197+0000 7f523ca00640 1 -- 172.21.15.67:0/2962485863 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228035020 con 0x7f52380c5e40 2024-03-09T09:25:46.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.197+0000 7f523ca00640 1 -- 172.21.15.67:0/2962485863 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5228031870 con 0x7f52380c5e40 2024-03-09T09:25:46.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.197+0000 7f523ca00640 1 -- 172.21.15.67:0/2962485863 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228031c40 con 0x7f52380c5e40 2024-03-09T09:25:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.198+0000 7f523f200640 1 -- 172.21.15.67:0/2962485863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52380c5e40 msgr2=0x7f52380b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.198+0000 7f523f200640 1 --2- 172.21.15.67:0/2962485863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52380c5e40 0x7f52380b0540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f52380ade40 tx=0x7f522802b910 comp rx=0 tx=0).stop 2024-03-09T09:25:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.198+0000 7f523f200640 1 -- 172.21.15.67:0/2962485863 shutdown_connections 2024-03-09T09:25:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.198+0000 7f523f200640 1 --2- 172.21.15.67:0/2962485863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52380c5e40 0x7f52380b0540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.198+0000 7f523f200640 1 --2- 172.21.15.67:0/2962485863 >> v2:172.21.15.134:3300/0 conn(0x7f5238153300 0x7f52380b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.198+0000 7f523f200640 1 -- 172.21.15.67:0/2962485863 >> 172.21.15.67:0/2962485863 conn(0x7f52380732d0 msgr2=0x7f52380ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:46.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.198+0000 7f523f200640 1 -- 172.21.15.67:0/2962485863 shutdown_connections 2024-03-09T09:25:46.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.198+0000 7f523f200640 1 -- 172.21.15.67:0/2962485863 wait complete. 2024-03-09T09:25:46.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.200+0000 7f523f200640 1 Processor -- start 2024-03-09T09:25:46.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.201+0000 7f523f200640 1 -- start start 2024-03-09T09:25:46.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.201+0000 7f523f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52380c5e40 0x7f523806ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:46.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.201+0000 7f523f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52380cadb0 0x7f52380c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:46.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.201+0000 7f523f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5238153300 0x7f523814b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:46.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.201+0000 7f523f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52380c59e0 con 0x7f52380c5e40 2024-03-09T09:25:46.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.201+0000 7f523f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5238159e90 con 0x7f52380cadb0 2024-03-09T09:25:46.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.201+0000 7f523f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5238156720 con 0x7f5238153300 2024-03-09T09:25:46.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.202+0000 7f523de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52380c5e40 0x7f523806ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:46.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.202+0000 7f523de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52380c5e40 0x7f523806ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49814/0 (socket says 172.21.15.67:49814) 2024-03-09T09:25:46.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.202+0000 7f523de00640 1 -- 172.21.15.67:0/3599754322 learned_addr learned my addr 172.21.15.67:0/3599754322 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:46.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.202+0000 7f523e800640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5238153300 0x7f523814b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:46.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.202+0000 7f523d400640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52380cadb0 0x7f52380c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:46.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.202+0000 7f523de00640 1 -- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5238153300 msgr2=0x7f523814b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:46.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.202+0000 7f523de00640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5238153300 0x7f523814b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:46.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.202+0000 7f523de00640 1 -- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52380cadb0 msgr2=0x7f52380c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:46.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.202+0000 7f523de00640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52380cadb0 0x7f52380c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:46.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.202+0000 7f523de00640 1 -- 172.21.15.67:0/3599754322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380c65b0 con 0x7f52380c5e40 2024-03-09T09:25:46.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.203+0000 7f523de00640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52380c5e40 0x7f523806ba80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c820 tx=0x7f522c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:46.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.203+0000 7f5232000640 1 -- 172.21.15.67:0/3599754322 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c015070 con 0x7f52380c5e40 2024-03-09T09:25:46.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.203+0000 7f5232000640 1 -- 172.21.15.67:0/3599754322 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c007830 con 0x7f52380c5e40 2024-03-09T09:25:46.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.203+0000 7f523f200640 1 -- 172.21.15.67:0/3599754322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52380c6850 con 0x7f52380c5e40 2024-03-09T09:25:46.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.203+0000 7f5232000640 1 -- 172.21.15.67:0/3599754322 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c007be0 con 0x7f52380c5e40 2024-03-09T09:25:46.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.203+0000 7f523f200640 1 -- 172.21.15.67:0/3599754322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523814c4f0 con 0x7f52380c5e40 2024-03-09T09:25:46.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.204+0000 7f523f200640 1 -- 172.21.15.67:0/3599754322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523804f3c0 con 0x7f52380c5e40 2024-03-09T09:25:46.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.205+0000 7f5232000640 1 -- 172.21.15.67:0/3599754322 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f522c018070 con 0x7f52380c5e40 2024-03-09T09:25:46.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.206+0000 7f5232000640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f521c06b4d0 0x7f521c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:46.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.206+0000 7f5232000640 1 -- 172.21.15.67:0/3599754322 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f522c08a540 con 0x7f52380c5e40 2024-03-09T09:25:46.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.207+0000 7f523d400640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f521c06b4d0 0x7f521c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.208+0000 7f523d400640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f521c06b4d0 0x7f521c06d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5228036000 tx=0x7f522803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.211+0000 7f5232000640 1 -- 172.21.15.67:0/3599754322 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f522c058ff0 con 0x7f52380c5e40 2024-03-09T09:25:46.232 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.232 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 41 -eq 48 ']' 2024-03-09T09:25:46.232 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.41 2024-03-09T09:25:46.232 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.41 2024-03-09T09:25:46.271 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.271 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 42 -eq 48 ']' 2024-03-09T09:25:46.271 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.42 2024-03-09T09:25:46.271 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.42 2024-03-09T09:25:46.300 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.300 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 43 -eq 48 ']' 2024-03-09T09:25:46.300 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.43 2024-03-09T09:25:46.300 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.43 2024-03-09T09:25:46.331 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.331 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 44 -eq 48 ']' 2024-03-09T09:25:46.331 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.44 2024-03-09T09:25:46.331 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.44 2024-03-09T09:25:46.355 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.355 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 45 -eq 48 ']' 2024-03-09T09:25:46.355 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.45 2024-03-09T09:25:46.355 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.45 2024-03-09T09:25:46.382 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.382 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 46 -eq 48 ']' 2024-03-09T09:25:46.382 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.46 2024-03-09T09:25:46.382 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.46 2024-03-09T09:25:46.416 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.416 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 47 -eq 48 ']' 2024-03-09T09:25:46.416 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.47 2024-03-09T09:25:46.416 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.47 2024-03-09T09:25:46.451 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.451 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 48 -eq 48 ']' 2024-03-09T09:25:46.451 INFO:tasks.workunit.client.1.smithi134.stderr:+ continue 2024-03-09T09:25:46.451 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.451 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 49 -eq 48 ']' 2024-03-09T09:25:46.452 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.49 2024-03-09T09:25:46.452 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.49 2024-03-09T09:25:46.483 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.483 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 50 -eq 48 ']' 2024-03-09T09:25:46.483 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.50 2024-03-09T09:25:46.483 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.50 2024-03-09T09:25:46.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.505+0000 7f523f200640 1 -- 172.21.15.67:0/3599754322 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f5238155d30 con 0x7f52380c5e40 2024-03-09T09:25:46.512 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.513 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 51 -eq 48 ']' 2024-03-09T09:25:46.513 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.51 2024-03-09T09:25:46.513 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.51 2024-03-09T09:25:46.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.506+0000 7f5232000640 1 -- 172.21.15.67:0/3599754322 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f522c0588f0 con 0x7f52380c5e40 2024-03-09T09:25:46.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 -- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f521c06b4d0 msgr2=0x7f521c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:46.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f521c06b4d0 0x7f521c06d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5228036000 tx=0x7f522803c040 comp rx=0 tx=0).stop 2024-03-09T09:25:46.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 -- 172.21.15.67:0/3599754322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52380c5e40 msgr2=0x7f523806ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:46.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52380c5e40 0x7f523806ba80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c820 tx=0x7f522c00cce0 comp rx=0 tx=0).stop 2024-03-09T09:25:46.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 -- 172.21.15.67:0/3599754322 shutdown_connections 2024-03-09T09:25:46.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f521c06b4d0 0x7f521c06d990 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5238153300 0x7f523814b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52380cadb0 0x7f52380c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 --2- 172.21.15.67:0/3599754322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52380c5e40 0x7f523806ba80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 -- 172.21.15.67:0/3599754322 >> 172.21.15.67:0/3599754322 conn(0x7f52380732d0 msgr2=0x7f52380ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 -- 172.21.15.67:0/3599754322 shutdown_connections 2024-03-09T09:25:46.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:46.509+0000 7f523f200640 1 -- 172.21.15.67:0/3599754322 wait complete. 2024-03-09T09:25:46.540 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:25:46.549 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.549 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 52 -eq 48 ']' 2024-03-09T09:25:46.549 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.52 2024-03-09T09:25:46.549 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.52 2024-03-09T09:25:46.581 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.581 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 53 -eq 48 ']' 2024-03-09T09:25:46.581 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.53 2024-03-09T09:25:46.581 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.53 2024-03-09T09:25:46.623 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.623 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 54 -eq 48 ']' 2024-03-09T09:25:46.623 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.54 2024-03-09T09:25:46.623 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.54 2024-03-09T09:25:46.667 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.667 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 55 -eq 48 ']' 2024-03-09T09:25:46.667 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.55 2024-03-09T09:25:46.667 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.55 2024-03-09T09:25:46.703 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.703 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 56 -eq 48 ']' 2024-03-09T09:25:46.703 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.56 2024-03-09T09:25:46.703 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.56 2024-03-09T09:25:46.742 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.742 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 57 -eq 48 ']' 2024-03-09T09:25:46.742 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.57 2024-03-09T09:25:46.742 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.57 2024-03-09T09:25:46.783 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.783 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 58 -eq 48 ']' 2024-03-09T09:25:46.783 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.58 2024-03-09T09:25:46.783 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.58 2024-03-09T09:25:46.820 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.821 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 59 -eq 48 ']' 2024-03-09T09:25:46.821 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.59 2024-03-09T09:25:46.821 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.59 2024-03-09T09:25:46.861 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.861 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 60 -eq 48 ']' 2024-03-09T09:25:46.861 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.60 2024-03-09T09:25:46.861 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.60 2024-03-09T09:25:46.915 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.915 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 61 -eq 48 ']' 2024-03-09T09:25:46.915 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.61 2024-03-09T09:25:46.915 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.61 2024-03-09T09:25:46.959 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:46.960 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 62 -eq 48 ']' 2024-03-09T09:25:46.960 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.62 2024-03-09T09:25:46.960 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.62 2024-03-09T09:25:47.020 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.021 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 63 -eq 48 ']' 2024-03-09T09:25:47.021 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.63 2024-03-09T09:25:47.021 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.63 2024-03-09T09:25:47.048 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.048 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 64 -eq 48 ']' 2024-03-09T09:25:47.048 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.64 2024-03-09T09:25:47.048 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.64 2024-03-09T09:25:47.080 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.081 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 65 -eq 48 ']' 2024-03-09T09:25:47.081 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.65 2024-03-09T09:25:47.081 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.65 2024-03-09T09:25:47.113 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.113 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 66 -eq 48 ']' 2024-03-09T09:25:47.113 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.66 2024-03-09T09:25:47.113 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.66 2024-03-09T09:25:47.141 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.142 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 67 -eq 48 ']' 2024-03-09T09:25:47.142 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.67 2024-03-09T09:25:47.142 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.67 2024-03-09T09:25:47.173 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.173 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 68 -eq 48 ']' 2024-03-09T09:25:47.173 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.68 2024-03-09T09:25:47.173 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.68 2024-03-09T09:25:47.208 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.208 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 69 -eq 48 ']' 2024-03-09T09:25:47.208 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.69 2024-03-09T09:25:47.208 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.69 2024-03-09T09:25:47.240 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.240 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 70 -eq 48 ']' 2024-03-09T09:25:47.240 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.70 2024-03-09T09:25:47.240 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.70 2024-03-09T09:25:47.279 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.279 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 71 -eq 48 ']' 2024-03-09T09:25:47.279 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.71 2024-03-09T09:25:47.279 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.71 2024-03-09T09:25:47.317 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.317 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 72 -eq 48 ']' 2024-03-09T09:25:47.317 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.72 2024-03-09T09:25:47.317 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.72 2024-03-09T09:25:47.352 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.352 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 73 -eq 48 ']' 2024-03-09T09:25:47.353 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.73 2024-03-09T09:25:47.353 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.73 2024-03-09T09:25:47.391 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.391 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 74 -eq 48 ']' 2024-03-09T09:25:47.391 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.74 2024-03-09T09:25:47.391 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.74 2024-03-09T09:25:47.427 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.427 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 75 -eq 48 ']' 2024-03-09T09:25:47.427 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.75 2024-03-09T09:25:47.427 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.75 2024-03-09T09:25:47.463 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.463 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 76 -eq 48 ']' 2024-03-09T09:25:47.463 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.76 2024-03-09T09:25:47.463 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.76 2024-03-09T09:25:47.501 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.501 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 77 -eq 48 ']' 2024-03-09T09:25:47.501 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.77 2024-03-09T09:25:47.501 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.77 2024-03-09T09:25:47.537 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.537 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 78 -eq 48 ']' 2024-03-09T09:25:47.537 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.78 2024-03-09T09:25:47.537 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.78 2024-03-09T09:25:47.574 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.574 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 79 -eq 48 ']' 2024-03-09T09:25:47.574 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.79 2024-03-09T09:25:47.574 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.79 2024-03-09T09:25:47.613 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.613 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 80 -eq 48 ']' 2024-03-09T09:25:47.613 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.80 2024-03-09T09:25:47.613 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.80 2024-03-09T09:25:47.651 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.651 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 81 -eq 48 ']' 2024-03-09T09:25:47.651 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.81 2024-03-09T09:25:47.651 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.81 2024-03-09T09:25:47.694 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.694 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 82 -eq 48 ']' 2024-03-09T09:25:47.694 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.82 2024-03-09T09:25:47.695 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.82 2024-03-09T09:25:47.738 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.738 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 83 -eq 48 ']' 2024-03-09T09:25:47.738 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.83 2024-03-09T09:25:47.739 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.83 2024-03-09T09:25:47.781 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.781 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 84 -eq 48 ']' 2024-03-09T09:25:47.782 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.84 2024-03-09T09:25:47.782 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.84 2024-03-09T09:25:47.828 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.828 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 85 -eq 48 ']' 2024-03-09T09:25:47.828 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.85 2024-03-09T09:25:47.828 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.85 2024-03-09T09:25:47.875 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.876 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 86 -eq 48 ']' 2024-03-09T09:25:47.876 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.86 2024-03-09T09:25:47.876 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.86 2024-03-09T09:25:47.931 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.931 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 87 -eq 48 ']' 2024-03-09T09:25:47.931 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.87 2024-03-09T09:25:47.932 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.87 2024-03-09T09:25:47.976 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:47.976 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 88 -eq 48 ']' 2024-03-09T09:25:47.976 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.88 2024-03-09T09:25:47.976 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.88 2024-03-09T09:25:48.025 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:48.026 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 89 -eq 48 ']' 2024-03-09T09:25:48.026 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.89 2024-03-09T09:25:48.026 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.89 2024-03-09T09:25:48.618 INFO:tasks.workunit.client.1.smithi134.stderr:+ for v in $versions 2024-03-09T09:25:48.618 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' 90 -eq 48 ']' 2024-03-09T09:25:48.619 INFO:tasks.workunit.client.1.smithi134.stderr:+ ver=v0.90 2024-03-09T09:25:48.619 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/v0.90 2024-03-09T09:25:48.680 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T09:25:48.680 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T09:25:48.682 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:25:48.682 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T09:25:56.542 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:25:56.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:25:56.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.837+0000 7f31cec00640 1 Processor -- start 2024-03-09T09:25:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.837+0000 7f31cec00640 1 -- start start 2024-03-09T09:25:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.839+0000 7f31cec00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c815a1b0 0x7f31c80aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.839+0000 7f31cec00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c80ab550 con 0x7f31c80a9ab0 2024-03-09T09:25:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.839+0000 7f31cec00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c815a700 con 0x7f31c80c37c0 2024-03-09T09:25:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.839+0000 7f31cec00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c815a9e0 con 0x7f31c815a1b0 2024-03-09T09:25:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.840+0000 7f31c7e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f31c80a9ab0 0x7f31c80ab790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44754/0 (socket says 172.21.15.67:44754) 2024-03-09T09:25:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.840+0000 7f31c7e00640 1 -- 172.21.15.67:0/2680922406 learned_addr learned my addr 172.21.15.67:0/2680922406 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.840+0000 7f31ccc00640 1 --1- 172.21.15.67:0/2680922406 >> v1:172.21.15.134:6789/0 conn(0x7f31c80c37c0 0x7f31c80aae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:58230/0 (socket says 172.21.15.67:58230) 2024-03-09T09:25:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.840+0000 7f31c7400640 1 --2- 172.21.15.67:0/2680922406 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c815a1b0 0x7f31c80aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.841+0000 7f31c7400640 1 -- 172.21.15.67:0/2680922406 >> v1:172.21.15.134:6789/0 conn(0x7f31c80c37c0 legacy=0x7f31c80aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.841+0000 7f31c7400640 1 -- 172.21.15.67:0/2680922406 >> v1:172.21.15.67:6789/0 conn(0x7f31c80a9ab0 legacy=0x7f31c80ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.841+0000 7f31c6a00640 1 -- 172.21.15.67:0/2680922406 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593405882 0 0) 0x7f31c80ab550 con 0x7f31c80a9ab0 2024-03-09T09:25:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.841+0000 7f31c7400640 1 -- 172.21.15.67:0/2680922406 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c815acc0 con 0x7f31c815a1b0 2024-03-09T09:25:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.843+0000 7f31c7400640 1 --2- 172.21.15.67:0/2680922406 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c815a1b0 0x7f31c80aa900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009800 tx=0x7f31b402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb60f33c468218fa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:56.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.845+0000 7f31c6a00640 1 -- 172.21.15.67:0/2680922406 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4034070 con 0x7f31c815a1b0 2024-03-09T09:25:56.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.845+0000 7f31c6a00640 1 -- 172.21.15.67:0/2680922406 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b4030610 con 0x7f31c815a1b0 2024-03-09T09:25:56.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.845+0000 7f31c6a00640 1 -- 172.21.15.67:0/2680922406 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b40309e0 con 0x7f31c815a1b0 2024-03-09T09:25:56.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.848+0000 7f31cec00640 1 -- 172.21.15.67:0/2680922406 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c815a1b0 msgr2=0x7f31c80aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.848+0000 7f31cec00640 1 --2- 172.21.15.67:0/2680922406 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c815a1b0 0x7f31c80aa900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009800 tx=0x7f31b402b960 comp rx=0 tx=0).stop 2024-03-09T09:25:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.849+0000 7f31cec00640 1 -- 172.21.15.67:0/2680922406 shutdown_connections 2024-03-09T09:25:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.849+0000 7f31cec00640 1 --2- 172.21.15.67:0/2680922406 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c815a1b0 0x7f31c80aa900 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:56.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.849+0000 7f31cec00640 1 -- 172.21.15.67:0/2680922406 >> 172.21.15.67:0/2680922406 conn(0x7f31c80c7370 msgr2=0x7f31c80c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.850+0000 7f31cec00640 1 -- 172.21.15.67:0/2680922406 shutdown_connections 2024-03-09T09:25:56.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.850+0000 7f31cec00640 1 -- 172.21.15.67:0/2680922406 wait complete. 2024-03-09T09:25:56.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.852+0000 7f31cec00640 1 Processor -- start 2024-03-09T09:25:56.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.853+0000 7f31cec00640 1 -- start start 2024-03-09T09:25:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.855+0000 7f31cec00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31c80a9ab0 0x7f31c81e4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.855+0000 7f31cec00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c80c37c0 0x7f31c81f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.855+0000 7f31cec00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c815a1b0 0x7f31c815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.855+0000 7f31cec00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c80abea0 con 0x7f31c80a9ab0 2024-03-09T09:25:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.855+0000 7f31cec00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c80a9e90 con 0x7f31c80c37c0 2024-03-09T09:25:56.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.855+0000 7f31cec00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c80ac020 con 0x7f31c815a1b0 2024-03-09T09:25:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.856+0000 7f31c7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c80c37c0 0x7f31c81f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.856+0000 7f31c7400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c80c37c0 0x7f31c81f3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:43984/0 (socket says 172.21.15.67:43984) 2024-03-09T09:25:56.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.856+0000 7f31c7400640 1 -- 172.21.15.67:0/2359435376 learned_addr learned my addr 172.21.15.67:0/2359435376 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.856+0000 7f31ccc00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c815a1b0 0x7f31c815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.857+0000 7f31c7400640 1 -- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c815a1b0 msgr2=0x7f31c815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.857+0000 7f31c7400640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c815a1b0 0x7f31c815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.857+0000 7f31c7400640 1 -- 172.21.15.67:0/2359435376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31c80a9ab0 msgr2=0x7f31c81e4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.857+0000 7f31c7e00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31c80a9ab0 0x7f31c81e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.857+0000 7f31c7400640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31c80a9ab0 0x7f31c81e4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.857+0000 7f31c7400640 1 -- 172.21.15.67:0/2359435376 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c81eff00 con 0x7f31c80c37c0 2024-03-09T09:25:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.858+0000 7f31c7400640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c80c37c0 0x7f31c81f3db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009920 tx=0x7f31b40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:56.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.858+0000 7f31c7e00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31c80a9ab0 0x7f31c81e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:25:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.859+0000 7f31bfe00640 1 -- 172.21.15.67:0/2359435376 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4034040 con 0x7f31c80c37c0 2024-03-09T09:25:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.859+0000 7f31bfe00640 1 -- 172.21.15.67:0/2359435376 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b403aa20 con 0x7f31c80c37c0 2024-03-09T09:25:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.859+0000 7f31bfe00640 1 -- 172.21.15.67:0/2359435376 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b403adf0 con 0x7f31c80c37c0 2024-03-09T09:25:56.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.859+0000 7f31cec00640 1 -- 172.21.15.67:0/2359435376 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c81f01a0 con 0x7f31c80c37c0 2024-03-09T09:25:56.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.859+0000 7f31cec00640 1 -- 172.21.15.67:0/2359435376 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c815df20 con 0x7f31c80c37c0 2024-03-09T09:25:56.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.862+0000 7f31cec00640 1 -- 172.21.15.67:0/2359435376 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a0005290 con 0x7f31c80c37c0 2024-03-09T09:25:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.866+0000 7f31bfe00640 1 -- 172.21.15.67:0/2359435376 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f31b4037070 con 0x7f31c80c37c0 2024-03-09T09:25:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.868+0000 7f31bfe00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31a806b5a0 0x7f31a806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.868+0000 7f31bfe00640 1 -- 172.21.15.67:0/2359435376 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f31b40ad2b0 con 0x7f31c80c37c0 2024-03-09T09:25:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.869+0000 7f31c7e00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31a806b5a0 0x7f31a806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.871+0000 7f31c7e00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31a806b5a0 0x7f31a806da60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f31c00012e0 tx=0x7f31c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:56.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:56.874+0000 7f31bfe00640 1 -- 172.21.15.67:0/2359435376 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f31b406a030 con 0x7f31c80c37c0 2024-03-09T09:25:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.371+0000 7f31cec00640 1 -- 172.21.15.67:0/2359435376 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f31a0005580 con 0x7f31c80c37c0 2024-03-09T09:25:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.374+0000 7f31bfe00640 1 -- 172.21.15.67:0/2359435376 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f31b407be50 con 0x7f31c80c37c0 2024-03-09T09:25:57.381 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":262,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:25:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.378+0000 7f31cec00640 1 -- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31a806b5a0 msgr2=0x7f31a806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.378+0000 7f31cec00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31a806b5a0 0x7f31a806da60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f31c00012e0 tx=0x7f31c0009040 comp rx=0 tx=0).stop 2024-03-09T09:25:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.378+0000 7f31cec00640 1 -- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c80c37c0 msgr2=0x7f31c81f3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.378+0000 7f31cec00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c80c37c0 0x7f31c81f3db0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009920 tx=0x7f31b40046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.378+0000 7f31cec00640 1 -- 172.21.15.67:0/2359435376 shutdown_connections 2024-03-09T09:25:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.378+0000 7f31cec00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31a806b5a0 0x7f31a806da60 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.378+0000 7f31cec00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31c815a1b0 0x7f31c815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.378+0000 7f31cec00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f31c80c37c0 0x7f31c81f3db0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.378+0000 7f31cec00640 1 --2- 172.21.15.67:0/2359435376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31c80a9ab0 0x7f31c81e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.378+0000 7f31cec00640 1 -- 172.21.15.67:0/2359435376 >> 172.21.15.67:0/2359435376 conn(0x7f31c80c7370 msgr2=0x7f31c80aa900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:57.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.379+0000 7f31cec00640 1 -- 172.21.15.67:0/2359435376 shutdown_connections 2024-03-09T09:25:57.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.379+0000 7f31cec00640 1 -- 172.21.15.67:0/2359435376 wait complete. 2024-03-09T09:25:57.413 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:25:57.413 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:25:57.413 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:25:57.413 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:25:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.699+0000 7fdd68a00640 1 Processor -- start 2024-03-09T09:25:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.699+0000 7fdd68a00640 1 -- start start 2024-03-09T09:25:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.703+0000 7fdd68a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd640730c0 0x7fdd640aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.704+0000 7fdd68a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fdd6406d830 0x7fdd6406dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.704+0000 7fdd61400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fdd640734a0 0x7fdd640c2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:58234/0 (socket says 172.21.15.67:58234) 2024-03-09T09:25:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.704+0000 7fdd61400640 1 -- 172.21.15.67:0/1423406196 learned_addr learned my addr 172.21.15.67:0/1423406196 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:25:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.704+0000 7fdd68a00640 1 -- 172.21.15.67:0/1423406196 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd64152e00 con 0x7fdd640734a0 2024-03-09T09:25:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.704+0000 7fdd68a00640 1 -- 172.21.15.67:0/1423406196 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd640aa960 con 0x7fdd6406d830 2024-03-09T09:25:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.704+0000 7fdd68a00640 1 -- 172.21.15.67:0/1423406196 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd641530e0 con 0x7fdd640730c0 2024-03-09T09:25:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.705+0000 7fdd60a00640 1 -- 172.21.15.67:0/1423406196 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179853621 0 0) 0x7fdd64152e00 con 0x7fdd640734a0 2024-03-09T09:25:57.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.705+0000 7fdd60a00640 1 -- 172.21.15.67:0/1423406196 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd48003520 con 0x7fdd640734a0 2024-03-09T09:25:57.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.705+0000 7fdd62800640 1 --2- 172.21.15.67:0/1423406196 >> v2:172.21.15.67:3300/0 conn(0x7fdd6406d830 0x7fdd6406dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:57.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.706+0000 7fdd62800640 1 -- 172.21.15.67:0/1423406196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd640730c0 msgr2=0x7fdd640aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:57.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.706+0000 7fdd62800640 1 --2- 172.21.15.67:0/1423406196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd640730c0 0x7fdd640aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.706+0000 7fdd62800640 1 -- 172.21.15.67:0/1423406196 >> v1:172.21.15.134:6789/0 conn(0x7fdd640734a0 legacy=0x7fdd640c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:57.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.706+0000 7fdd62800640 1 -- 172.21.15.67:0/1423406196 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd641533c0 con 0x7fdd6406d830 2024-03-09T09:25:57.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.708+0000 7fdd62800640 1 --2- 172.21.15.67:0/1423406196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6406d830 0x7fdd6406dcc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd500097a0 tx=0x7fdd5002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cc7b22dc20e444 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:57.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.708+0000 7fdd60a00640 1 -- 172.21.15.67:0/1423406196 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd500315a0 con 0x7fdd6406d830 2024-03-09T09:25:57.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.708+0000 7fdd60a00640 1 -- 172.21.15.67:0/1423406196 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50037070 con 0x7fdd6406d830 2024-03-09T09:25:57.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.708+0000 7fdd60a00640 1 -- 172.21.15.67:0/1423406196 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd500319a0 con 0x7fdd6406d830 2024-03-09T09:25:57.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.710+0000 7fdd68a00640 1 -- 172.21.15.67:0/1423406196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6406d830 msgr2=0x7fdd6406dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:57.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.710+0000 7fdd68a00640 1 --2- 172.21.15.67:0/1423406196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6406d830 0x7fdd6406dcc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd500097a0 tx=0x7fdd5002b840 comp rx=0 tx=0).stop 2024-03-09T09:25:57.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.711+0000 7fdd68a00640 1 -- 172.21.15.67:0/1423406196 shutdown_connections 2024-03-09T09:25:57.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.711+0000 7fdd68a00640 1 --2- 172.21.15.67:0/1423406196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6406d830 0x7fdd6406dcc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.711+0000 7fdd68a00640 1 --2- 172.21.15.67:0/1423406196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd640730c0 0x7fdd640aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.711+0000 7fdd68a00640 1 -- 172.21.15.67:0/1423406196 >> 172.21.15.67:0/1423406196 conn(0x7fdd640c7370 msgr2=0x7fdd640c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:57.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.712+0000 7fdd68a00640 1 -- 172.21.15.67:0/1423406196 shutdown_connections 2024-03-09T09:25:57.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.713+0000 7fdd68a00640 1 -- 172.21.15.67:0/1423406196 wait complete. 2024-03-09T09:25:57.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.715+0000 7fdd68a00640 1 Processor -- start 2024-03-09T09:25:57.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.716+0000 7fdd68a00640 1 -- start start 2024-03-09T09:25:57.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.717+0000 7fdd68a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6406d830 0x7fdd641fb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:57.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.717+0000 7fdd68a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd640730c0 0x7fdd641fb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:57.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.717+0000 7fdd68a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd640734a0 0x7fdd641f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:57.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.717+0000 7fdd68a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd640c3590 con 0x7fdd6406d830 2024-03-09T09:25:57.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.717+0000 7fdd68a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd64153980 con 0x7fdd640734a0 2024-03-09T09:25:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.717+0000 7fdd68a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd640c3710 con 0x7fdd640730c0 2024-03-09T09:25:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.717+0000 7fdd61e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6406d830 0x7fdd641fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.718+0000 7fdd61e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6406d830 0x7fdd641fb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49028/0 (socket says 172.21.15.67:49028) 2024-03-09T09:25:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.718+0000 7fdd61e00640 1 -- 172.21.15.67:0/3885559605 learned_addr learned my addr 172.21.15.67:0/3885559605 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.718+0000 7fdd61e00640 1 -- 172.21.15.67:0/3885559605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd640730c0 msgr2=0x7fdd641fb790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:25:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.718+0000 7fdd61e00640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd640730c0 0x7fdd641fb790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.718+0000 7fdd61e00640 1 -- 172.21.15.67:0/3885559605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd640734a0 msgr2=0x7fdd641f4c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-09T09:25:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.718+0000 7fdd61e00640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd640734a0 0x7fdd641f4c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.718+0000 7fdd61e00640 1 -- 172.21.15.67:0/3885559605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd641fbe40 con 0x7fdd6406d830 2024-03-09T09:25:57.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.719+0000 7fdd61e00640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6406d830 0x7fdd641fb250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400b4a0 tx=0x7fdd5400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:57.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.720+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54014070 con 0x7fdd6406d830 2024-03-09T09:25:57.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.720+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd54010610 con 0x7fdd6406d830 2024-03-09T09:25:57.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.720+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54010a00 con 0x7fdd6406d830 2024-03-09T09:25:57.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.720+0000 7fdd68a00640 1 -- 172.21.15.67:0/3885559605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd64156a10 con 0x7fdd6406d830 2024-03-09T09:25:57.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.720+0000 7fdd68a00640 1 -- 172.21.15.67:0/3885559605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6414c230 con 0x7fdd6406d830 2024-03-09T09:25:57.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.725+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdd54017070 con 0x7fdd6406d830 2024-03-09T09:25:57.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.726+0000 7fdd5a000640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd4406b4d0 0x7fdd4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:57.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.727+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6693+0+0 (secure 0 0 0) 0x7fdd54056440 con 0x7fdd6406d830 2024-03-09T09:25:57.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.727+0000 7fdd68a00640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414a610 0x7fdd641f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:57.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.727+0000 7fdd68a00640 1 -- 172.21.15.67:0/3885559605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd641f56a0 con 0x7fdd6414a610 2024-03-09T09:25:57.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.728+0000 7fdd61400640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd4406b4d0 0x7fdd4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:57.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.728+0000 7fdd62800640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414a610 0x7fdd641f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:57.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.731+0000 7fdd61400640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd4406b4d0 0x7fdd4406d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdd640c6200 tx=0x7fdd5c001400 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:57.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.732+0000 7fdd62800640 1 -- 172.21.15.67:0/3885559605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd64075a00 con 0x7fdd6414a610 2024-03-09T09:25:57.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.733+0000 7fdd62800640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414a610 0x7fdd641f5160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdd6406ea30 tx=0x7fdd500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:57.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.734+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50041070 con 0x7fdd6414a610 2024-03-09T09:25:57.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.735+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fdd50037040 con 0x7fdd6414a610 2024-03-09T09:25:57.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.735+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414a610 msgr2=0x7fdd641f5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:57.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.735+0000 7fdd5a000640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414a610 0x7fdd641f5160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdd6406ea30 tx=0x7fdd500046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.735+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414a610 msgr2=0x7fdd641f5160 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:57.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.735+0000 7fdd5a000640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414a610 0x7fdd641f5160 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdd6406ea30 tx=0x7fdd500046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.754+0000 7fdd68a00640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414dd10 0x7fdd6414e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.754+0000 7fdd68a00640 1 -- 172.21.15.67:0/3885559605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6414bfe0 con 0x7fdd6414dd10 2024-03-09T09:25:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.755+0000 7fdd62800640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414dd10 0x7fdd6414e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:57.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.756+0000 7fdd62800640 1 -- 172.21.15.67:0/3885559605 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdd6414c430 con 0x7fdd6414dd10 2024-03-09T09:25:57.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.757+0000 7fdd62800640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414dd10 0x7fdd6414e0f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdd500097a0 tx=0x7fdd50009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:57.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.758+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd500093b0 con 0x7fdd6414dd10 2024-03-09T09:25:57.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.758+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4877 (secure 0 0 0) 0x7fdd50032430 con 0x7fdd6414dd10 2024-03-09T09:25:57.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.758+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414dd10 msgr2=0x7fdd6414e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:57.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.758+0000 7fdd5a000640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414dd10 0x7fdd6414e0f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdd500097a0 tx=0x7fdd50009190 comp rx=0 tx=0).stop 2024-03-09T09:25:57.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.758+0000 7fdd5a000640 1 -- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414dd10 msgr2=0x7fdd6414e0f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:57.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.758+0000 7fdd5a000640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6414dd10 0x7fdd6414e0f0 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdd500097a0 tx=0x7fdd50009190 comp rx=0 tx=0).stop 2024-03-09T09:25:57.764 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 262, 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 15, 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:25:57.765 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:57.766 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.767 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:25:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:25:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:25:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:25:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:25:57.777 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:25:57.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.759+0000 7fdd68a00640 1 -- 172.21.15.67:0/3885559605 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd4406b4d0 msgr2=0x7fdd4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:57.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.759+0000 7fdd68a00640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fdd4406b4d0 0x7fdd4406d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdd640c6200 tx=0x7fdd5c001400 comp rx=0 tx=0).stop 2024-03-09T09:25:57.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.759+0000 7fdd68a00640 1 -- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6406d830 msgr2=0x7fdd641fb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:57.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.759+0000 7fdd68a00640 1 --2- 172.21.15.67:0/3885559605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd6406d830 0x7fdd641fb250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400b4a0 tx=0x7fdd5400b960 comp rx=0 tx=0).stop 2024-03-09T09:25:57.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.761+0000 7fdd62800640 1 -- 172.21.15.67:0/3885559605 reap_dead start 2024-03-09T09:25:57.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.761+0000 7fdd68a00640 1 -- 172.21.15.67:0/3885559605 shutdown_connections 2024-03-09T09:25:57.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.761+0000 7fdd68a00640 1 -- 172.21.15.67:0/3885559605 >> 172.21.15.67:0/3885559605 conn(0x7fdd640c7370 msgr2=0x7fdd6406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:57.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.762+0000 7fdd68a00640 1 -- 172.21.15.67:0/3885559605 shutdown_connections 2024-03-09T09:25:57.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:57.762+0000 7fdd68a00640 1 -- 172.21.15.67:0/3885559605 wait complete. 2024-03-09T09:25:57.797 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:25:57.797 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:25:57.797 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:25:58.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.108+0000 7fe61f200640 1 Processor -- start 2024-03-09T09:25:58.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.109+0000 7fe61f200640 1 -- start start 2024-03-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.112+0000 7fe61f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6180c96f0 0x7fe61806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.112+0000 7fe61f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6180c5f50 0x7fe6180c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.114+0000 7fe61f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61806cf30 con 0x7fe618153560 2024-03-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.114+0000 7fe61f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe618159d80 con 0x7fe6180c96f0 2024-03-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.114+0000 7fe61f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61815a060 con 0x7fe6180c5f50 2024-03-09T09:25:58.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.114+0000 7fe617400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6180c96f0 0x7fe61806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.114+0000 7fe617400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6180c96f0 0x7fe61806b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49064/0 (socket says 172.21.15.67:49064) 2024-03-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.114+0000 7fe617400640 1 -- 172.21.15.67:0/1821567777 learned_addr learned my addr 172.21.15.67:0/1821567777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:58.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.115+0000 7fe61d200640 1 --2- 172.21.15.67:0/1821567777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6180c5f50 0x7fe6180c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.115+0000 7fe61d200640 1 -- 172.21.15.67:0/1821567777 >> v2:172.21.15.67:3300/0 conn(0x7fe6180c96f0 msgr2=0x7fe61806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.116+0000 7fe61d200640 1 --2- 172.21.15.67:0/1821567777 >> v2:172.21.15.67:3300/0 conn(0x7fe6180c96f0 0x7fe61806b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.116+0000 7fe61d200640 1 -- 172.21.15.67:0/1821567777 >> v1:172.21.15.67:6789/0 conn(0x7fe618153560 legacy=0x7fe6180cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.117+0000 7fe61d200640 1 -- 172.21.15.67:0/1821567777 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6180ca440 con 0x7fe6180c5f50 2024-03-09T09:25:58.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.117+0000 7fe617400640 1 --2- 172.21.15.67:0/1821567777 >> v2:172.21.15.67:3300/0 conn(0x7fe6180c96f0 0x7fe61806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.121+0000 7fe61d200640 1 --2- 172.21.15.67:0/1821567777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6180c5f50 0x7fe6180c63a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe610009d90 tx=0x7fe61002bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=565db9486ce907a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:58.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.121+0000 7fe616a00640 1 -- 172.21.15.67:0/1821567777 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610039070 con 0x7fe6180c5f50 2024-03-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.121+0000 7fe616a00640 1 -- 172.21.15.67:0/1821567777 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61002cb40 con 0x7fe6180c5f50 2024-03-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.121+0000 7fe616a00640 1 -- 172.21.15.67:0/1821567777 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610034440 con 0x7fe6180c5f50 2024-03-09T09:25:58.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.123+0000 7fe61f200640 1 -- 172.21.15.67:0/1821567777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6180c5f50 msgr2=0x7fe6180c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.123+0000 7fe61f200640 1 --2- 172.21.15.67:0/1821567777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6180c5f50 0x7fe6180c63a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe610009d90 tx=0x7fe61002bd90 comp rx=0 tx=0).stop 2024-03-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.123+0000 7fe61f200640 1 -- 172.21.15.67:0/1821567777 shutdown_connections 2024-03-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.123+0000 7fe61f200640 1 --2- 172.21.15.67:0/1821567777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6180c5f50 0x7fe6180c63a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.123+0000 7fe61f200640 1 --2- 172.21.15.67:0/1821567777 >> v2:172.21.15.67:3300/0 conn(0x7fe6180c96f0 0x7fe61806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.124+0000 7fe61f200640 1 -- 172.21.15.67:0/1821567777 >> 172.21.15.67:0/1821567777 conn(0x7fe6180c8050 msgr2=0x7fe6180c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:58.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.128+0000 7fe61f200640 1 -- 172.21.15.67:0/1821567777 shutdown_connections 2024-03-09T09:25:58.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.129+0000 7fe61f200640 1 -- 172.21.15.67:0/1821567777 wait complete. 2024-03-09T09:25:58.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.132+0000 7fe61f200640 1 Processor -- start 2024-03-09T09:25:58.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.137+0000 7fe61f200640 1 -- start start 2024-03-09T09:25:58.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.144+0000 7fe61f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6180c5f50 0x7fe6181f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.144+0000 7fe61f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180c96f0 0x7fe6180ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.145+0000 7fe61f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe618153560 0x7fe6180ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.145+0000 7fe61f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180c68e0 con 0x7fe6180c5f50 2024-03-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.146+0000 7fe61f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180c9ff0 con 0x7fe6180c96f0 2024-03-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.146+0000 7fe61f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180c6a60 con 0x7fe618153560 2024-03-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.147+0000 7fe61d200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe618153560 0x7fe6180ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.148+0000 7fe617e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6180c5f50 0x7fe6181f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.148+0000 7fe617e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6180c5f50 0x7fe6181f6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49072/0 (socket says 172.21.15.67:49072) 2024-03-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.148+0000 7fe617e00640 1 -- 172.21.15.67:0/3663812419 learned_addr learned my addr 172.21.15.67:0/3663812419 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:58.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.148+0000 7fe617400640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180c96f0 0x7fe6180ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.148+0000 7fe617e00640 1 -- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe618153560 msgr2=0x7fe6180ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.148+0000 7fe617e00640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe618153560 0x7fe6180ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.148+0000 7fe617e00640 1 -- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180c96f0 msgr2=0x7fe6180ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.149+0000 7fe617e00640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180c96f0 0x7fe6180ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.149+0000 7fe617e00640 1 -- 172.21.15.67:0/3663812419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6180adec0 con 0x7fe6180c5f50 2024-03-09T09:25:58.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.153+0000 7fe617e00640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6180c5f50 0x7fe6181f6310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe60800b3f0 tx=0x7fe60800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.153+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608015020 con 0x7fe6180c5f50 2024-03-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.153+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe608011830 con 0x7fe6180c5f50 2024-03-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.153+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608011c20 con 0x7fe6180c5f50 2024-03-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.156+0000 7fe61f200640 1 -- 172.21.15.67:0/3663812419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6180c1120 con 0x7fe6180c5f50 2024-03-09T09:25:58.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.156+0000 7fe61f200640 1 -- 172.21.15.67:0/3663812419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6180c16e0 con 0x7fe6180c5f50 2024-03-09T09:25:58.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.158+0000 7fe61f200640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180c18e0 0x7fe6181fc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.158+0000 7fe61f200640 1 -- 172.21.15.67:0/3663812419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180682c0 con 0x7fe6180c18e0 2024-03-09T09:25:58.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.159+0000 7fe617400640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180c18e0 0x7fe6181fc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.161+0000 7fe617400640 1 -- 172.21.15.67:0/3663812419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe618075a00 con 0x7fe6180c18e0 2024-03-09T09:25:58.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.164+0000 7fe617400640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180c18e0 0x7fe6181fc2a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe618152d90 tx=0x7fe604011d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.166+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60400fa70 con 0x7fe6180c18e0 2024-03-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.166+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fe60400fc10 con 0x7fe6180c18e0 2024-03-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.166+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180c18e0 msgr2=0x7fe6181fc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.166+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180c18e0 0x7fe6181fc2a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe618152d90 tx=0x7fe604011d80 comp rx=0 tx=0).stop 2024-03-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.166+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180c18e0 msgr2=0x7fe6181fc2a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:58.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.166+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180c18e0 0x7fe6181fc2a0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe618152d90 tx=0x7fe604011d80 comp rx=0 tx=0).stop 2024-03-09T09:25:58.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.167+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe608018020 con 0x7fe6180c5f50 2024-03-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.175+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe5f806af40 0x7fe5f806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.175+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6693+0+0 (secure 0 0 0) 0x7fe60808a460 con 0x7fe6180c5f50 2024-03-09T09:25:58.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.177+0000 7fe617400640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe5f806af40 0x7fe5f806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.178+0000 7fe617400640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe5f806af40 0x7fe5f806d400 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe60401c000 tx=0x7fe604008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.183+0000 7fe61f200640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180aebd0 0x7fe6181fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.183+0000 7fe61f200640 1 -- 172.21.15.67:0/3663812419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618151dc0 con 0x7fe6180aebd0 2024-03-09T09:25:58.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.191+0000 7fe61d200640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180aebd0 0x7fe6181fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.192+0000 7fe61d200640 1 -- 172.21.15.67:0/3663812419 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe618151ae0 con 0x7fe6180aebd0 2024-03-09T09:25:58.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.193+0000 7fe61d200640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180aebd0 0x7fe6181fc7e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe618073c90 tx=0x7fe610004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:58.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.195+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610044030 con 0x7fe6180aebd0 2024-03-09T09:25:58.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.197+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7fe610040460 con 0x7fe6180aebd0 2024-03-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.197+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180aebd0 msgr2=0x7fe6181fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.197+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180aebd0 0x7fe6181fc7e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe618073c90 tx=0x7fe610004a40 comp rx=0 tx=0).stop 2024-03-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.197+0000 7fe60fe00640 1 -- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180aebd0 msgr2=0x7fe6181fc7e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:58.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.197+0000 7fe60fe00640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6180aebd0 0x7fe6181fc7e0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe618073c90 tx=0x7fe610004a40 comp rx=0 tx=0).stop 2024-03-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:58.205 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 262, 2024-03-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:25:58.206 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:58.207 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:58.208 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:58.209 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:25:58.210 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:58.211 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:58.212 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:58.213 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:25:58.214 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:25:58.215 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:58.216 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:58.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:25:58.218 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.220 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:58.221 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.223 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:25:58.224 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.202+0000 7fe61f200640 1 -- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe5f806af40 msgr2=0x7fe5f806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.202+0000 7fe61f200640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe5f806af40 0x7fe5f806d400 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe60401c000 tx=0x7fe604008030 comp rx=0 tx=0).stop 2024-03-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.202+0000 7fe61d200640 1 -- 172.21.15.67:0/3663812419 reap_dead start 2024-03-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.202+0000 7fe61f200640 1 -- 172.21.15.67:0/3663812419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6180c5f50 msgr2=0x7fe6181f6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.202+0000 7fe61f200640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6180c5f50 0x7fe6181f6310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe60800b3f0 tx=0x7fe60800b8b0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.202+0000 7fe61f200640 1 -- 172.21.15.67:0/3663812419 shutdown_connections 2024-03-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.202+0000 7fe61f200640 1 --2- 172.21.15.67:0/3663812419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6180c5f50 0x7fe6181f6310 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe60800b3f0 tx=0x7fe60800b8b0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.202+0000 7fe61f200640 1 -- 172.21.15.67:0/3663812419 >> 172.21.15.67:0/3663812419 conn(0x7fe6180c8050 msgr2=0x7fe61806bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.205+0000 7fe61f200640 1 -- 172.21.15.67:0/3663812419 shutdown_connections 2024-03-09T09:25:58.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.205+0000 7fe61f200640 1 -- 172.21.15.67:0/3663812419 wait complete. 2024-03-09T09:25:58.237 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:25:58.237 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:25:58.237 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:25:58.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.540+0000 7ff6d4600640 1 Processor -- start 2024-03-09T09:25:58.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.540+0000 7ff6d4600640 1 -- start start 2024-03-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.542+0000 7ff6d4600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc14c9a0 0x7ff6cc1567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.542+0000 7ff6d4600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff6cc0ac9c0 0x7ff6cc0ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.542+0000 7ff6d4600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc0ac4e0 con 0x7ff6cc0c1d70 2024-03-09T09:25:58.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.542+0000 7ff6d4600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6cc06d590 con 0x7ff6cc0ac9c0 2024-03-09T09:25:58.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.542+0000 7ff6d4600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc156d00 con 0x7ff6cc14c9a0 2024-03-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.544+0000 7ff6d1c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc14c9a0 0x7ff6cc1567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.544+0000 7ff6d1c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc14c9a0 0x7ff6cc1567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:44514/0 (socket says 172.21.15.67:44514) 2024-03-09T09:25:58.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.544+0000 7ff6d1c00640 1 -- 172.21.15.67:0/338191885 learned_addr learned my addr 172.21.15.67:0/338191885 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:58.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.545+0000 7ff6d2600640 1 --2- 172.21.15.67:0/338191885 >> v2:172.21.15.134:3300/0 conn(0x7ff6cc0ac9c0 0x7ff6cc0ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.545+0000 7ff6d1c00640 1 -- 172.21.15.67:0/338191885 >> v2:172.21.15.134:3300/0 conn(0x7ff6cc0ac9c0 msgr2=0x7ff6cc0ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.545+0000 7ff6d1c00640 1 --2- 172.21.15.67:0/338191885 >> v2:172.21.15.134:3300/0 conn(0x7ff6cc0ac9c0 0x7ff6cc0ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.545+0000 7ff6d1c00640 1 -- 172.21.15.67:0/338191885 >> v1:172.21.15.134:6789/0 conn(0x7ff6cc0c1d70 legacy=0x7ff6cc0c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.545+0000 7ff6d1c00640 1 -- 172.21.15.67:0/338191885 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc0aa550 con 0x7ff6cc14c9a0 2024-03-09T09:25:58.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.551+0000 7ff6d1c00640 1 --2- 172.21.15.67:0/338191885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc14c9a0 0x7ff6cc1567c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009870 tx=0x7ff6c002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b63381a2863c688e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:58.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.553+0000 7ff6cbe00640 1 -- 172.21.15.67:0/338191885 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0035020 con 0x7ff6cc14c9a0 2024-03-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.553+0000 7ff6cbe00640 1 -- 172.21.15.67:0/338191885 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c0031640 con 0x7ff6cc14c9a0 2024-03-09T09:25:58.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.553+0000 7ff6cbe00640 1 -- 172.21.15.67:0/338191885 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c002d040 con 0x7ff6cc14c9a0 2024-03-09T09:25:58.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.554+0000 7ff6d4600640 1 -- 172.21.15.67:0/338191885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc14c9a0 msgr2=0x7ff6cc1567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.554+0000 7ff6d4600640 1 --2- 172.21.15.67:0/338191885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc14c9a0 0x7ff6cc1567c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009870 tx=0x7ff6c002b8f0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.555+0000 7ff6d4600640 1 -- 172.21.15.67:0/338191885 shutdown_connections 2024-03-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.555+0000 7ff6d4600640 1 --2- 172.21.15.67:0/338191885 >> v2:172.21.15.134:3300/0 conn(0x7ff6cc0ac9c0 0x7ff6cc0ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.555+0000 7ff6d4600640 1 --2- 172.21.15.67:0/338191885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc14c9a0 0x7ff6cc1567c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.555+0000 7ff6d4600640 1 -- 172.21.15.67:0/338191885 >> 172.21.15.67:0/338191885 conn(0x7ff6cc0c7370 msgr2=0x7ff6cc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:58.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.559+0000 7ff6d4600640 1 -- 172.21.15.67:0/338191885 shutdown_connections 2024-03-09T09:25:58.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.559+0000 7ff6d4600640 1 -- 172.21.15.67:0/338191885 wait complete. 2024-03-09T09:25:58.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.561+0000 7ff6d4600640 1 Processor -- start 2024-03-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.562+0000 7ff6d4600640 1 -- start start 2024-03-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.563+0000 7ff6d4600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc0ac9c0 0x7ff6cc1ebd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.563+0000 7ff6d4600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc0c1d70 0x7ff6cc146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.563+0000 7ff6d4600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6cc14c9a0 0x7ff6cc1470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.563+0000 7ff6d4600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0aadf0 con 0x7ff6cc14c9a0 2024-03-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.563+0000 7ff6d4600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0aab10 con 0x7ff6cc0ac9c0 2024-03-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.563+0000 7ff6d4600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0c0eb0 con 0x7ff6cc0c1d70 2024-03-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.563+0000 7ff6d2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6cc14c9a0 0x7ff6cc1470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.564+0000 7ff6d1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc0ac9c0 0x7ff6cc1ebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.564+0000 7ff6d1c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc0ac9c0 0x7ff6cc1ebd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:44048/0 (socket says 172.21.15.67:44048) 2024-03-09T09:25:58.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.564+0000 7ff6d1c00640 1 -- 172.21.15.67:0/3592992270 learned_addr learned my addr 172.21.15.67:0/3592992270 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.565+0000 7ff6d1c00640 1 -- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc0c1d70 msgr2=0x7ff6cc146b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.565+0000 7ff6d1200640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc0c1d70 0x7ff6cc146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.566+0000 7ff6d1c00640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc0c1d70 0x7ff6cc146b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.566+0000 7ff6d1c00640 1 -- 172.21.15.67:0/3592992270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6cc14c9a0 msgr2=0x7ff6cc1470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.566+0000 7ff6d1c00640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6cc14c9a0 0x7ff6cc1470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.566+0000 7ff6d1c00640 1 -- 172.21.15.67:0/3592992270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc1ec730 con 0x7ff6cc0ac9c0 2024-03-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.566+0000 7ff6d1200640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc0c1d70 0x7ff6cc146b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:25:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.566+0000 7ff6d2600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6cc14c9a0 0x7ff6cc1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:25:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.568+0000 7ff6d1c00640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc0ac9c0 0x7ff6cc1ebd40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0000c00 tx=0x7ff6c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:58.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.569+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0040070 con 0x7ff6cc0ac9c0 2024-03-09T09:25:58.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.569+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c00399f0 con 0x7ff6cc0ac9c0 2024-03-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.569+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0035040 con 0x7ff6cc0ac9c0 2024-03-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.573+0000 7ff6d4600640 1 -- 172.21.15.67:0/3592992270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc073010 con 0x7ff6cc0ac9c0 2024-03-09T09:25:58.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.573+0000 7ff6d4600640 1 -- 172.21.15.67:0/3592992270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc147dd0 con 0x7ff6cc0ac9c0 2024-03-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.575+0000 7ff6d4600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc1ec280 0x7ff6cc0718e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.575+0000 7ff6d4600640 1 -- 172.21.15.67:0/3592992270 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc072e70 con 0x7ff6cc1ec280 2024-03-09T09:25:58.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.576+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff6c004b060 con 0x7ff6cc0ac9c0 2024-03-09T09:25:58.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.577+0000 7ff6d1200640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc1ec280 0x7ff6cc0718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.577+0000 7ff6c9600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff6ac06b4d0 0x7ff6ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.577+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6693+0+0 (secure 0 0 0) 0x7ff6c00abea0 con 0x7ff6cc0ac9c0 2024-03-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.577+0000 7ff6d2600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff6ac06b4d0 0x7ff6ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.580+0000 7ff6d2600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff6ac06b4d0 0x7ff6ac06d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc159aa0 tx=0x7ff6c4009780 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.580+0000 7ff6d1200640 1 -- 172.21.15.67:0/3592992270 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6cc071e20 con 0x7ff6cc1ec280 2024-03-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.581+0000 7ff6d1200640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc1ec280 0x7ff6cc0718e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800d4c0 tx=0x7ff6b800d980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.581+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8016070 con 0x7ff6cc1ec280 2024-03-09T09:25:58.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.582+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff6b8007220 con 0x7ff6cc1ec280 2024-03-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.582+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc1ec280 msgr2=0x7ff6cc0718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.582+0000 7ff6c9600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc1ec280 0x7ff6cc0718e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800d4c0 tx=0x7ff6b800d980 comp rx=0 tx=0).stop 2024-03-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.582+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc1ec280 msgr2=0x7ff6cc0718e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:58.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.582+0000 7ff6c9600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc1ec280 0x7ff6cc0718e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.595+0000 7ff6d4600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc156f10 0x7ff6cc1ef030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.595+0000 7ff6d4600640 1 -- 172.21.15.67:0/3592992270 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc075a00 con 0x7ff6cc156f10 2024-03-09T09:25:58.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.596+0000 7ff6d1200640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc156f10 0x7ff6cc1ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.597+0000 7ff6d1200640 1 -- 172.21.15.67:0/3592992270 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff6cc14b660 con 0x7ff6cc156f10 2024-03-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.597+0000 7ff6d1200640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc156f10 0x7ff6cc1ef030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800d4c0 tx=0x7ff6b8000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:58.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.598+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8016070 con 0x7ff6cc156f10 2024-03-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.598+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7ff6b8012b60 con 0x7ff6cc156f10 2024-03-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.598+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc156f10 msgr2=0x7ff6cc1ef030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.598+0000 7ff6c9600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc156f10 0x7ff6cc1ef030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800d4c0 tx=0x7ff6b8000ed0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.598+0000 7ff6c9600640 1 -- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc156f10 msgr2=0x7ff6cc1ef030 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.598+0000 7ff6c9600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6cc156f10 0x7ff6cc1ef030 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800d4c0 tx=0x7ff6b8000ed0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 262, 2024-03-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:25:58.607 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:25:58.608 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 16, 2024-03-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:25:58.609 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:58.610 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:58.611 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:58.612 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:58.613 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:25:58.616 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:25:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:25:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:58.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:58.620 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:25:58.621 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.622 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:25:58.623 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:58.624 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:25:58.625 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:25:58.626 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.600+0000 7ff6d4600640 1 -- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff6ac06b4d0 msgr2=0x7ff6ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.600+0000 7ff6d4600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff6ac06b4d0 0x7ff6ac06d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc159aa0 tx=0x7ff6c4009780 comp rx=0 tx=0).stop 2024-03-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.600+0000 7ff6d4600640 1 -- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc0ac9c0 msgr2=0x7ff6cc1ebd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.600+0000 7ff6d4600640 1 --2- 172.21.15.67:0/3592992270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6cc0ac9c0 0x7ff6cc1ebd40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0000c00 tx=0x7ff6c00046c0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.600+0000 7ff6d2600640 1 -- 172.21.15.67:0/3592992270 reap_dead start 2024-03-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.600+0000 7ff6d2600640 1 -- 172.21.15.67:0/3592992270 reap_dead start 2024-03-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.600+0000 7ff6d4600640 1 -- 172.21.15.67:0/3592992270 shutdown_connections 2024-03-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.600+0000 7ff6d4600640 1 -- 172.21.15.67:0/3592992270 >> 172.21.15.67:0/3592992270 conn(0x7ff6cc0c7370 msgr2=0x7ff6cc0c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.601+0000 7ff6d4600640 1 -- 172.21.15.67:0/3592992270 shutdown_connections 2024-03-09T09:25:58.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.602+0000 7ff6d4600640 1 -- 172.21.15.67:0/3592992270 wait complete. 2024-03-09T09:25:58.635 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:25:58.635 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-03-09T09:25:58.635 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:25:58.635 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-03-09T09:25:58.635 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:25:58.636 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-09T09:25:58.636 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:25:58.667 INFO:tasks.ceph.mon.a:Stopped 2024-03-09T09:25:58.668 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:25:58.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:25:58.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.986+0000 7f5cd6400640 1 Processor -- start 2024-03-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.986+0000 7f5cd6400640 1 -- start start 2024-03-09T09:25:58.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.988+0000 7f5cd6400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5cd00a9ab0 0x7f5cd00ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.988+0000 7f5cd6400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5cd00abcd0 0x7f5cd00ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.988+0000 7f5cd6400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd00ab550 con 0x7f5cd015a1b0 2024-03-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.988+0000 7f5cd6400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd015a700 con 0x7f5cd00abcd0 2024-03-09T09:25:58.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.988+0000 7f5cd6400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd015a9e0 con 0x7f5cd00a9ab0 2024-03-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.988+0000 7f5cc7e00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f5cd015a1b0 legacy=0x7f5cd00aa900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.988+0000 7f5ccfe00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f5cd00abcd0 msgr2=0x7f5cd00ab010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.988+0000 7f5ccfe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5cd00abcd0 0x7f5cd00ab010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.988+0000 7f5ccf400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5cd00a9ab0 0x7f5cd00ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.988+0000 7f5ccf400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f5cd00a9ab0 0x7f5cd00ab790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:55252/0 (socket says 172.21.15.67:55252) 2024-03-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.988+0000 7f5ccf400640 1 -- 172.21.15.67:0/184059296 learned_addr learned my addr 172.21.15.67:0/184059296 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.989+0000 7f5ccf400640 1 -- 172.21.15.67:0/184059296 >> v2:172.21.15.67:3300/0 conn(0x7f5cd00abcd0 msgr2=0x7f5cd00ab010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.989+0000 7f5ccf400640 1 --2- 172.21.15.67:0/184059296 >> v2:172.21.15.67:3300/0 conn(0x7f5cd00abcd0 0x7f5cd00ab010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.989+0000 7f5ccf400640 1 -- 172.21.15.67:0/184059296 >> v1:172.21.15.67:6789/0 conn(0x7f5cd015a1b0 legacy=0x7f5cd00aa900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:25:58.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.989+0000 7f5ccf400640 1 -- 172.21.15.67:0/184059296 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd015acc0 con 0x7f5cd00a9ab0 2024-03-09T09:25:58.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.992+0000 7f5ccf400640 1 --2- 172.21.15.67:0/184059296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cd00a9ab0 0x7f5cd00ab790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0070370 tx=0x7f5cc002b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=428a6a0e90495e8b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.993+0000 7f5ccea00640 1 -- 172.21.15.67:0/184059296 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0035020 con 0x7f5cd00a9ab0 2024-03-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.993+0000 7f5ccea00640 1 -- 172.21.15.67:0/184059296 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc0031890 con 0x7f5cd00a9ab0 2024-03-09T09:25:58.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.993+0000 7f5ccea00640 1 -- 172.21.15.67:0/184059296 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0031c60 con 0x7f5cd00a9ab0 2024-03-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.994+0000 7f5cd6400640 1 -- 172.21.15.67:0/184059296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cd00a9ab0 msgr2=0x7f5cd00ab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.994+0000 7f5cd6400640 1 --2- 172.21.15.67:0/184059296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cd00a9ab0 0x7f5cd00ab790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0070370 tx=0x7f5cc002b930 comp rx=0 tx=0).stop 2024-03-09T09:25:59.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.995+0000 7f5cd6400640 1 -- 172.21.15.67:0/184059296 shutdown_connections 2024-03-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.995+0000 7f5cd6400640 1 --2- 172.21.15.67:0/184059296 >> v2:172.21.15.67:3300/0 conn(0x7f5cd00abcd0 0x7f5cd00ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.995+0000 7f5cd6400640 1 --2- 172.21.15.67:0/184059296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cd00a9ab0 0x7f5cd00ab790 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:59.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.995+0000 7f5cd6400640 1 -- 172.21.15.67:0/184059296 >> 172.21.15.67:0/184059296 conn(0x7f5cd00c7370 msgr2=0x7f5cd00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.998+0000 7f5cd6400640 1 -- 172.21.15.67:0/184059296 shutdown_connections 2024-03-09T09:25:59.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:58.998+0000 7f5cd6400640 1 -- 172.21.15.67:0/184059296 wait complete. 2024-03-09T09:25:59.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.000+0000 7f5cd6400640 1 Processor -- start 2024-03-09T09:25:59.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.000+0000 7f5cd6400640 1 -- start start 2024-03-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.002+0000 7f5cd6400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cd00a9ab0 0x7f5cd015b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.002+0000 7f5cd6400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cd00abcd0 0x7f5cd01f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.002+0000 7f5cd6400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cd015a1b0 0x7f5cd01f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.002+0000 7f5cd6400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd006dfa0 con 0x7f5cd00a9ab0 2024-03-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.002+0000 7f5cd6400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00a9e90 con 0x7f5cd00abcd0 2024-03-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.002+0000 7f5cd6400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd006e120 con 0x7f5cd015a1b0 2024-03-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.002+0000 7f5ccf400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cd00a9ab0 msgr2=0x7f5cd015b810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.002+0000 7f5ccf400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cd00a9ab0 0x7f5cd015b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:25:59.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.003+0000 7f5ccfe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cd015a1b0 0x7f5cd01f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.003+0000 7f5ccfe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cd015a1b0 0x7f5cd01f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46224/0 (socket says 172.21.15.67:46224) 2024-03-09T09:25:59.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.003+0000 7f5ccfe00640 1 -- 172.21.15.67:0/3938918383 learned_addr learned my addr 172.21.15.67:0/3938918383 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.003+0000 7f5cc7e00640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cd00abcd0 0x7f5cd01f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.004+0000 7f5ccfe00640 1 -- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cd00abcd0 msgr2=0x7f5cd01f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.004+0000 7f5ccfe00640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cd00abcd0 0x7f5cd01f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.004+0000 7f5ccfe00640 1 -- 172.21.15.67:0/3938918383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cd00a9ab0 msgr2=0x7f5cd015b810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.004+0000 7f5ccfe00640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cd00a9ab0 0x7f5cd015b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.004+0000 7f5ccfe00640 1 -- 172.21.15.67:0/3938918383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd015e540 con 0x7f5cd015a1b0 2024-03-09T09:25:59.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.005+0000 7f5cc7e00640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cd00abcd0 0x7f5cd01f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.006+0000 7f5ccfe00640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cd015a1b0 0x7f5cd01f3c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800be90 tx=0x7f5cc800bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.006+0000 7f5cc7400640 1 -- 172.21.15.67:0/3938918383 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8015470 con 0x7f5cd015a1b0 2024-03-09T09:25:59.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.006+0000 7f5cc7400640 1 -- 172.21.15.67:0/3938918383 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc8010070 con 0x7f5cd015a1b0 2024-03-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.007+0000 7f5cc7400640 1 -- 172.21.15.67:0/3938918383 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8015850 con 0x7f5cd015a1b0 2024-03-09T09:25:59.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.007+0000 7f5cd6400640 1 -- 172.21.15.67:0/3938918383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd00c1b00 con 0x7f5cd015a1b0 2024-03-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.007+0000 7f5cd6400640 1 -- 172.21.15.67:0/3938918383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd00c20c0 con 0x7f5cd015a1b0 2024-03-09T09:25:59.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.009+0000 7f5cd6400640 1 -- 172.21.15.67:0/3938918383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd006e7d0 con 0x7f5cd015a1b0 2024-03-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.011+0000 7f5cc7400640 1 -- 172.21.15.67:0/3938918383 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5cc8005070 con 0x7f5cd015a1b0 2024-03-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.013+0000 7f5cc7400640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5cac06b480 0x7f5cac06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.013+0000 7f5cc7400640 1 -- 172.21.15.67:0/3938918383 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f5cc808aea0 con 0x7f5cd015a1b0 2024-03-09T09:25:59.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.014+0000 7f5ccf400640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5cac06b480 0x7f5cac06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:25:59.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.016+0000 7f5ccf400640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5cac06b480 0x7f5cac06d940 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0036000 tx=0x7f5cc0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:25:59.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.018+0000 7f5cc7400640 1 -- 172.21.15.67:0/3938918383 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5cc8059890 con 0x7f5cd015a1b0 2024-03-09T09:25:59.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.492+0000 7f5cd6400640 1 -- 172.21.15.67:0/3938918383 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5cd0158710 con 0x7f5cd015a1b0 2024-03-09T09:25:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.495+0000 7f5cc7400640 1 -- 172.21.15.67:0/3938918383 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f5cc8059190 con 0x7f5cd015a1b0 2024-03-09T09:25:59.502 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":262,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:25:59.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.498+0000 7f5cd6400640 1 -- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5cac06b480 msgr2=0x7f5cac06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:59.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.498+0000 7f5cd6400640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5cac06b480 0x7f5cac06d940 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0036000 tx=0x7f5cc0008030 comp rx=0 tx=0).stop 2024-03-09T09:25:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.499+0000 7f5cd6400640 1 -- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cd015a1b0 msgr2=0x7f5cd01f3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:25:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.499+0000 7f5cd6400640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cd015a1b0 0x7f5cd01f3c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800be90 tx=0x7f5cc800bf80 comp rx=0 tx=0).stop 2024-03-09T09:25:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.499+0000 7f5cd6400640 1 -- 172.21.15.67:0/3938918383 shutdown_connections 2024-03-09T09:25:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.499+0000 7f5cd6400640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5cac06b480 0x7f5cac06d940 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:59.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.499+0000 7f5cd6400640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cd015a1b0 0x7f5cd01f3c80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.499+0000 7f5cd6400640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cd00abcd0 0x7f5cd01f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.499+0000 7f5cd6400640 1 --2- 172.21.15.67:0/3938918383 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cd00a9ab0 0x7f5cd015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:25:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.499+0000 7f5cd6400640 1 -- 172.21.15.67:0/3938918383 >> 172.21.15.67:0/3938918383 conn(0x7f5cd00c7370 msgr2=0x7f5cd00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:25:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.499+0000 7f5cd6400640 1 -- 172.21.15.67:0/3938918383 shutdown_connections 2024-03-09T09:25:59.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:25:59.499+0000 7f5cd6400640 1 -- 172.21.15.67:0/3938918383 wait complete. 2024-03-09T09:25:59.539 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-03-09T09:25:59.531+0000 7feee6c00640 -1 calc_pg_upmaps abort due to max <= 0 2024-03-09T09:26:01.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:26:01.669 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:26:02.532 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:26:02.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.810+0000 7f56b4e00640 1 Processor -- start 2024-03-09T09:26:02.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.810+0000 7f56b4e00640 1 -- start start 2024-03-09T09:26:02.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.812+0000 7f56b4e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 0x7f56b0071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:02.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.812+0000 7f56b4e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f56b0157d60 0x7f56b00acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:02.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.812+0000 7f56b4e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b0153d40 con 0x7f56b00c22a0 2024-03-09T09:26:02.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.812+0000 7f56b4e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b0070ee0 con 0x7f56b0157d60 2024-03-09T09:26:02.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.812+0000 7f56b4e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0154d00 con 0x7f56b00ac730 2024-03-09T09:26:02.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.814+0000 7f56ae800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f56b0157d60 0x7f56b00acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:02.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.814+0000 7f56ae800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f56b0157d60 0x7f56b00acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:55270/0 (socket says 172.21.15.67:55270) 2024-03-09T09:26:02.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.814+0000 7f56ae800640 1 -- 172.21.15.67:0/1488200179 learned_addr learned my addr 172.21.15.67:0/1488200179 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.814+0000 7f56ade00640 1 --2- 172.21.15.67:0/1488200179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 0x7f56b0071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.814+0000 7f56ade00640 1 -- 172.21.15.67:0/1488200179 >> v2:172.21.15.134:3300/0 conn(0x7f56b0157d60 msgr2=0x7f56b00acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:26:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.814+0000 7f56ade00640 1 --2- 172.21.15.67:0/1488200179 >> v2:172.21.15.134:3300/0 conn(0x7f56b0157d60 0x7f56b00acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.814+0000 7f56ade00640 1 -- 172.21.15.67:0/1488200179 >> v1:172.21.15.134:6789/0 conn(0x7f56b00c22a0 legacy=0x7f56b0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:26:02.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.814+0000 7f56ade00640 1 -- 172.21.15.67:0/1488200179 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b00c2790 con 0x7f56b00ac730 2024-03-09T09:26:02.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.818+0000 7f56ade00640 1 --2- 172.21.15.67:0/1488200179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 0x7f56b0071190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009d90 tx=0x7f56a802bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=169e256bf6dd6e12 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:02.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.820+0000 7f56aca00640 1 -- 172.21.15.67:0/1488200179 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8039070 con 0x7f56b00ac730 2024-03-09T09:26:02.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.820+0000 7f56aca00640 1 -- 172.21.15.67:0/1488200179 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a802cb40 con 0x7f56b00ac730 2024-03-09T09:26:02.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.820+0000 7f56aca00640 1 -- 172.21.15.67:0/1488200179 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8034440 con 0x7f56b00ac730 2024-03-09T09:26:02.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.823+0000 7f56b4e00640 1 -- 172.21.15.67:0/1488200179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 msgr2=0x7f56b0071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:02.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.823+0000 7f56b4e00640 1 --2- 172.21.15.67:0/1488200179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 0x7f56b0071190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009d90 tx=0x7f56a802bd90 comp rx=0 tx=0).stop 2024-03-09T09:26:02.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.824+0000 7f56b4e00640 1 -- 172.21.15.67:0/1488200179 shutdown_connections 2024-03-09T09:26:02.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.824+0000 7f56b4e00640 1 --2- 172.21.15.67:0/1488200179 >> v2:172.21.15.134:3300/0 conn(0x7f56b0157d60 0x7f56b00acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:02.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.824+0000 7f56b4e00640 1 --2- 172.21.15.67:0/1488200179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00ac730 0x7f56b0071190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:02.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.824+0000 7f56b4e00640 1 -- 172.21.15.67:0/1488200179 >> 172.21.15.67:0/1488200179 conn(0x7f56b00c7370 msgr2=0x7f56b00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:02.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.824+0000 7f56b4e00640 1 -- 172.21.15.67:0/1488200179 shutdown_connections 2024-03-09T09:26:02.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.824+0000 7f56b4e00640 1 -- 172.21.15.67:0/1488200179 wait complete. 2024-03-09T09:26:02.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.829+0000 7f56b4e00640 1 Processor -- start 2024-03-09T09:26:02.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.830+0000 7f56b4e00640 1 -- start start 2024-03-09T09:26:02.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.832+0000 7f56b4e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b00ac730 0x7f56b01550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:02.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.832+0000 7f56b4e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00c22a0 0x7f56b0156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:02.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.832+0000 7f56ade00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b00ac730 msgr2=0x7f56b01550b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.832+0000 7f56b4e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b0157d60 0x7f56b01effb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.832+0000 7f56b4e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0151610 con 0x7f56b00ac730 2024-03-09T09:26:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.833+0000 7f56ad400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00c22a0 0x7f56b0156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:02.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.833+0000 7f56ad400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00c22a0 0x7f56b0156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46234/0 (socket says 172.21.15.67:46234) 2024-03-09T09:26:02.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.833+0000 7f56ad400640 1 -- 172.21.15.67:0/1024747331 learned_addr learned my addr 172.21.15.67:0/1024747331 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:02.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.833+0000 7f56ade00640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b00ac730 0x7f56b01550b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:02.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.833+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0070c00 con 0x7f56b0157d60 2024-03-09T09:26:02.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.833+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0151790 con 0x7f56b00c22a0 2024-03-09T09:26:02.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.834+0000 7f56ae800640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b0157d60 0x7f56b01effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:02.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.834+0000 7f56ad400640 1 -- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b0157d60 msgr2=0x7f56b01effb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:02.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.834+0000 7f56ad400640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b0157d60 0x7f56b01effb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.834+0000 7f56ad400640 1 -- 172.21.15.67:0/1024747331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b00ac730 msgr2=0x7f56b01550b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:26:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.834+0000 7f56ad400640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b00ac730 0x7f56b01550b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.834+0000 7f56ad400640 1 -- 172.21.15.67:0/1024747331 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b0155780 con 0x7f56b00c22a0 2024-03-09T09:26:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.835+0000 7f56ad400640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00c22a0 0x7f56b0156c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f569c00c430 tx=0x7f569c00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.835+0000 7f56a6000640 1 -- 172.21.15.67:0/1024747331 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c011790 con 0x7f56b00c22a0 2024-03-09T09:26:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.835+0000 7f56a6000640 1 -- 172.21.15.67:0/1024747331 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c017070 con 0x7f56b00c22a0 2024-03-09T09:26:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.835+0000 7f56a6000640 1 -- 172.21.15.67:0/1024747331 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c011b70 con 0x7f56b00c22a0 2024-03-09T09:26:02.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.835+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b0155a60 con 0x7f56b00c22a0 2024-03-09T09:26:02.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.835+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b0156020 con 0x7f56b00c22a0 2024-03-09T09:26:02.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.840+0000 7f56a6000640 1 -- 172.21.15.67:0/1024747331 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f569c011d10 con 0x7f56b00c22a0 2024-03-09T09:26:02.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.840+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b0075a00 con 0x7f56b00c22a0 2024-03-09T09:26:02.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.841+0000 7f56a6000640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 0x7f569006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:02.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.842+0000 7f56a6000640 1 -- 172.21.15.67:0/1024747331 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f569c08a380 con 0x7f56b00c22a0 2024-03-09T09:26:02.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.843+0000 7f56ade00640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 0x7f569006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:02.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.844+0000 7f56ade00640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 0x7f569006d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f56a8036000 tx=0x7f56a8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:02.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:02.851+0000 7f56a6000640 1 -- 172.21.15.67:0/1024747331 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f569c058e90 con 0x7f56b00c22a0 2024-03-09T09:26:03.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.347+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f56b00c1d70 con 0x7f56b00c22a0 2024-03-09T09:26:03.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.348+0000 7f56a6000640 1 -- 172.21.15.67:0/1024747331 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f569c058790 con 0x7f56b00c22a0 2024-03-09T09:26:03.358 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":262,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:26:03.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.351+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 msgr2=0x7f569006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:03.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.353+0000 7f56b4e00640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 0x7f569006d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f56a8036000 tx=0x7f56a8030000 comp rx=0 tx=0).stop 2024-03-09T09:26:03.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.353+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00c22a0 msgr2=0x7f56b0156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:03.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.353+0000 7f56b4e00640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00c22a0 0x7f56b0156c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f569c00c430 tx=0x7f569c00c8f0 comp rx=0 tx=0).stop 2024-03-09T09:26:03.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.353+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 shutdown_connections 2024-03-09T09:26:03.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.353+0000 7f56b4e00640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f569006b4d0 0x7f569006d990 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:03.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.353+0000 7f56b4e00640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b0157d60 0x7f56b01effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:03.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.353+0000 7f56b4e00640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b00c22a0 0x7f56b0156c00 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f569c00c430 tx=0x7f569c00c8f0 comp rx=0 tx=0).stop 2024-03-09T09:26:03.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.353+0000 7f56b4e00640 1 --2- 172.21.15.67:0/1024747331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56b00ac730 0x7f56b01550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:03.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.353+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 >> 172.21.15.67:0/1024747331 conn(0x7f56b00c7370 msgr2=0x7f56b0151bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:03.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.353+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 shutdown_connections 2024-03-09T09:26:03.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:03.353+0000 7f56b4e00640 1 -- 172.21.15.67:0/1024747331 wait complete. 2024-03-09T09:26:05.076 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-hardlink.sh... 2024-03-09T09:26:05.076 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-hardlink.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-hardlink.sh 2024-03-09T09:26:05.163 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T09:26:05.164 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir 1 2 2024-03-09T09:26:05.172 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo asdf 2024-03-09T09:26:05.172 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo asdf 2024-03-09T09:26:05.173 INFO:tasks.workunit.client.0.smithi067.stderr:+ ln 1/file1 2/file1 2024-03-09T09:26:05.178 INFO:tasks.workunit.client.0.smithi067.stderr:+ ln 1/file2 2/file2 2024-03-09T09:26:05.188 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir 2/.snap/s1 2024-03-09T09:26:06.021 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo qwer 2024-03-09T09:26:06.038 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep asdf 2/.snap/s1/file1 2024-03-09T09:26:06.064 INFO:tasks.workunit.client.0.smithi067.stdout:asdf 2024-03-09T09:26:06.065 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -f 1/file2 2024-03-09T09:26:06.074 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep asdf 2/.snap/s1/file2 2024-03-09T09:26:06.129 INFO:tasks.workunit.client.0.smithi067.stdout:asdf 2024-03-09T09:26:06.129 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -f 2/file2 2024-03-09T09:26:06.140 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep asdf 2/.snap/s1/file2 2024-03-09T09:26:06.143 INFO:tasks.workunit.client.0.smithi067.stdout:asdf 2024-03-09T09:26:06.143 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir 2/.snap/s1 2024-03-09T09:26:06.179 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -rf 1 2 2024-03-09T09:26:06.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:26:06.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.667+0000 7efeb2000640 1 Processor -- start 2024-03-09T09:26:06.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.668+0000 7efeb2000640 1 -- start start 2024-03-09T09:26:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.669+0000 7efeb2000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efeac072770 0x7efeac072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.669+0000 7efeb2000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeac06cd30 con 0x7efeac0bebc0 2024-03-09T09:26:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.669+0000 7efeb2000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeac0c5eb0 con 0x7efeac06bda0 2024-03-09T09:26:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.669+0000 7efeb2000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeac06b200 con 0x7efeac072770 2024-03-09T09:26:06.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.669+0000 7efeabe00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7efeac0bebc0 legacy=0x7efeac06b690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:26:06.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.669+0000 7efeb0c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efeac072770 0x7efeac072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:06.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.669+0000 7efeb0c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7efeac072770 0x7efeac072b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:55290/0 (socket says 172.21.15.67:55290) 2024-03-09T09:26:06.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.669+0000 7efeb0c00640 1 -- 172.21.15.67:0/3500952171 learned_addr learned my addr 172.21.15.67:0/3500952171 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:06.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.670+0000 7efeb0c00640 1 -- 172.21.15.67:0/3500952171 >> v1:172.21.15.134:6789/0 conn(0x7efeac06bda0 legacy=0x7efeac160550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:26:06.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.670+0000 7efeb0c00640 1 -- 172.21.15.67:0/3500952171 >> v1:172.21.15.67:6789/0 conn(0x7efeac0bebc0 legacy=0x7efeac06b690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:06.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.670+0000 7efeb0c00640 1 -- 172.21.15.67:0/3500952171 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac0c6150 con 0x7efeac072770 2024-03-09T09:26:06.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.671+0000 7efeb0c00640 1 --2- 172.21.15.67:0/3500952171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac072770 0x7efeac072b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efeac0aae90 tx=0x7efea002b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a92897062b067470 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:06.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.672+0000 7efeab400640 1 -- 172.21.15.67:0/3500952171 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0038070 con 0x7efeac072770 2024-03-09T09:26:06.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.672+0000 7efeab400640 1 -- 172.21.15.67:0/3500952171 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0031710 con 0x7efeac072770 2024-03-09T09:26:06.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.672+0000 7efeab400640 1 -- 172.21.15.67:0/3500952171 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0031ae0 con 0x7efeac072770 2024-03-09T09:26:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.673+0000 7efeb2000640 1 -- 172.21.15.67:0/3500952171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac072770 msgr2=0x7efeac072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.673+0000 7efeb2000640 1 --2- 172.21.15.67:0/3500952171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac072770 0x7efeac072b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efeac0aae90 tx=0x7efea002b7b0 comp rx=0 tx=0).stop 2024-03-09T09:26:06.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.673+0000 7efeb2000640 1 -- 172.21.15.67:0/3500952171 shutdown_connections 2024-03-09T09:26:06.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.673+0000 7efeb2000640 1 --2- 172.21.15.67:0/3500952171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac072770 0x7efeac072b50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:06.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.673+0000 7efeb2000640 1 -- 172.21.15.67:0/3500952171 >> 172.21.15.67:0/3500952171 conn(0x7efeac0c7fb0 msgr2=0x7efeac0c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:06.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.673+0000 7efeb2000640 1 -- 172.21.15.67:0/3500952171 shutdown_connections 2024-03-09T09:26:06.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.674+0000 7efeb2000640 1 -- 172.21.15.67:0/3500952171 wait complete. 2024-03-09T09:26:06.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.675+0000 7efeb2000640 1 Processor -- start 2024-03-09T09:26:06.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.676+0000 7efeb2000640 1 -- start start 2024-03-09T09:26:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.677+0000 7efeb2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac06bda0 0x7efeac0c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:06.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.677+0000 7efeb2000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeac0bebc0 0x7efeac0bf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.677+0000 7efeb2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeac0aaf30 0x7efeac0ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.677+0000 7efeb2000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0c9430 con 0x7efeac0aaf30 2024-03-09T09:26:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.677+0000 7efeb2000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0c6710 con 0x7efeac06bda0 2024-03-09T09:26:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.677+0000 7efeb2000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0c1620 con 0x7efeac0bebc0 2024-03-09T09:26:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.678+0000 7efeb1600640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeac0aaf30 msgr2=0x7efeac0ab3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.678+0000 7efeb1600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeac0aaf30 0x7efeac0ab3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:06.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.678+0000 7efeabe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeac0bebc0 0x7efeac0bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:06.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.678+0000 7efeabe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeac0bebc0 0x7efeac0bf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:46248/0 (socket says 172.21.15.67:46248) 2024-03-09T09:26:06.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.678+0000 7efeabe00640 1 -- 172.21.15.67:0/1865344689 learned_addr learned my addr 172.21.15.67:0/1865344689 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:06.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.679+0000 7efeb0c00640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac06bda0 0x7efeac0c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:06.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.680+0000 7efeabe00640 1 -- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac06bda0 msgr2=0x7efeac0c10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:06.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.680+0000 7efeabe00640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac06bda0 0x7efeac0c10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:06.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.680+0000 7efeabe00640 1 -- 172.21.15.67:0/1865344689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeac0aaf30 msgr2=0x7efeac0ab3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:26:06.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.680+0000 7efeabe00640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeac0aaf30 0x7efeac0ab3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:06.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.680+0000 7efeabe00640 1 -- 172.21.15.67:0/1865344689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac1f3480 con 0x7efeac0bebc0 2024-03-09T09:26:06.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.680+0000 7efeabe00640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeac0bebc0 0x7efeac0bf060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00b5b0 tx=0x7efe9c00ba70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:06.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.681+0000 7efea8c00640 1 -- 172.21.15.67:0/1865344689 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c013020 con 0x7efeac0bebc0 2024-03-09T09:26:06.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.681+0000 7efea8c00640 1 -- 172.21.15.67:0/1865344689 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c003dc0 con 0x7efeac0bebc0 2024-03-09T09:26:06.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.681+0000 7efea8c00640 1 -- 172.21.15.67:0/1865344689 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c005270 con 0x7efeac0bebc0 2024-03-09T09:26:06.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.682+0000 7efeb2000640 1 -- 172.21.15.67:0/1865344689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac1f3760 con 0x7efeac0bebc0 2024-03-09T09:26:06.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.682+0000 7efeb2000640 1 -- 172.21.15.67:0/1865344689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac1f3d20 con 0x7efeac0bebc0 2024-03-09T09:26:06.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.684+0000 7efeb2000640 1 -- 172.21.15.67:0/1865344689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeac0c57f0 con 0x7efeac0bebc0 2024-03-09T09:26:06.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.690+0000 7efea8c00640 1 -- 172.21.15.67:0/1865344689 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7efe9c016020 con 0x7efeac0bebc0 2024-03-09T09:26:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.693+0000 7efea8c00640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efe9006b480 0x7efe9006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:06.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.694+0000 7efea8c00640 1 -- 172.21.15.67:0/1865344689 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6693+0+0 (secure 0 0 0) 0x7efe9c08a7d0 con 0x7efeac0bebc0 2024-03-09T09:26:06.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.695+0000 7efea8c00640 1 -- 172.21.15.67:0/1865344689 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efeac0c57f0 con 0x7efeac0bebc0 2024-03-09T09:26:06.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.696+0000 7efeb0c00640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efe9006b480 0x7efe9006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:06.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:06.697+0000 7efeb0c00640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efe9006b480 0x7efe9006d940 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efea0035000 tx=0x7efea0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:07.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.106+0000 7efeb2000640 1 -- 172.21.15.67:0/1865344689 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7efeac04f3c0 con 0x7efeac0bebc0 2024-03-09T09:26:07.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.107+0000 7efea8c00640 1 -- 172.21.15.67:0/1865344689 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7efe9c01d020 con 0x7efeac0bebc0 2024-03-09T09:26:07.114 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":262,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:26:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 -- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efe9006b480 msgr2=0x7efe9006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efe9006b480 0x7efe9006d940 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efea0035000 tx=0x7efea0008030 comp rx=0 tx=0).stop 2024-03-09T09:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 -- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeac0bebc0 msgr2=0x7efeac0bf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeac0bebc0 0x7efeac0bf060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00b5b0 tx=0x7efe9c00ba70 comp rx=0 tx=0).stop 2024-03-09T09:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 -- 172.21.15.67:0/1865344689 shutdown_connections 2024-03-09T09:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efe9006b480 0x7efe9006d940 secure :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efea0035000 tx=0x7efea0008030 comp rx=0 tx=0).stop 2024-03-09T09:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeac0aaf30 0x7efeac0ab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efeac0bebc0 0x7efeac0bf060 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 --2- 172.21.15.67:0/1865344689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac06bda0 0x7efeac0c10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 -- 172.21.15.67:0/1865344689 >> 172.21.15.67:0/1865344689 conn(0x7efeac0c7fb0 msgr2=0x7efeac06b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 -- 172.21.15.67:0/1865344689 shutdown_connections 2024-03-09T09:26:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:07.110+0000 7efeb2000640 1 -- 172.21.15.67:0/1865344689 wait complete. 2024-03-09T09:26:10.140 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:26:10.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.368+0000 7fd64f200640 1 Processor -- start 2024-03-09T09:26:10.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.368+0000 7fd64f200640 1 -- start start 2024-03-09T09:26:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.370+0000 7fd64f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd6500710a0 0x7fd650153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.370+0000 7fd64f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.370+0000 7fd64f200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6500c81e0 con 0x7fd6500c1f70 2024-03-09T09:26:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.370+0000 7fd64f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6500c32b0 con 0x7fd6500710a0 2024-03-09T09:26:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.370+0000 7fd64f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6500c2fd0 con 0x7fd650151360 2024-03-09T09:26:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.370+0000 7fd64d400640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fd6500710a0 msgr2=0x7fd650153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.370+0000 7fd64d400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd6500710a0 0x7fd650153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.370+0000 7fd64de00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fd6500c1f70 0x7fd650150c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:53322/0 (socket says 172.21.15.67:53322) 2024-03-09T09:26:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.370+0000 7fd64de00640 1 -- 172.21.15.67:0/4283184376 learned_addr learned my addr 172.21.15.67:0/4283184376 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:26:10.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.370+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.371+0000 7fd64e800640 1 -- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 msgr2=0x7fd650153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:26:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.371+0000 7fd64e800640 1 -- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 msgr2=0x7fd650153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.371+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:10.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.371+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:10.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.571+0000 7fd64d400640 1 -- 172.21.15.67:0/4283184376 >> v2:172.21.15.67:3300/0 conn(0x7fd6500710a0 msgr2=0x7fd650153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:10.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.571+0000 7fd64d400640 1 --2- 172.21.15.67:0/4283184376 >> v2:172.21.15.67:3300/0 conn(0x7fd6500710a0 0x7fd650153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:26:10.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.572+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:10.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.572+0000 7fd64e800640 1 -- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 msgr2=0x7fd650153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:10.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.572+0000 7fd64e800640 1 -- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 msgr2=0x7fd650153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:10.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.572+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:10.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.572+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:26:10.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.972+0000 7fd64d400640 1 -- 172.21.15.67:0/4283184376 >> v2:172.21.15.67:3300/0 conn(0x7fd6500710a0 msgr2=0x7fd650153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:10.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.972+0000 7fd64d400640 1 --2- 172.21.15.67:0/4283184376 >> v2:172.21.15.67:3300/0 conn(0x7fd6500710a0 0x7fd650153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:26:10.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.973+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:10.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.974+0000 7fd64e800640 1 -- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 msgr2=0x7fd650153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:10.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.974+0000 7fd64e800640 1 -- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 msgr2=0x7fd650153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:10.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.974+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:10.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:10.974+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:26:11.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:11.773+0000 7fd64d400640 1 -- 172.21.15.67:0/4283184376 >> v2:172.21.15.67:3300/0 conn(0x7fd6500710a0 msgr2=0x7fd650153970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:11.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:11.773+0000 7fd64d400640 1 --2- 172.21.15.67:0/4283184376 >> v2:172.21.15.67:3300/0 conn(0x7fd6500710a0 0x7fd650153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:26:11.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:11.777+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:11.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:11.777+0000 7fd64e800640 1 -- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 msgr2=0x7fd650153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:11.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:11.777+0000 7fd64e800640 1 -- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 msgr2=0x7fd650153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:11.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:11.777+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:11.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:11.777+0000 7fd64e800640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:26:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.370+0000 7fd647400640 1 -- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 msgr2=0x7fd650153eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.370+0000 7fd647400640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd650151360 0x7fd650153eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.370+0000 7fd647400640 1 -- 172.21.15.67:0/4283184376 >> v2:172.21.15.67:3300/0 conn(0x7fd6500710a0 msgr2=0x7fd650153970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.370+0000 7fd647400640 1 --2- 172.21.15.67:0/4283184376 >> v2:172.21.15.67:3300/0 conn(0x7fd6500710a0 0x7fd650153970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.370+0000 7fd647400640 1 -- 172.21.15.67:0/4283184376 >> v1:172.21.15.134:6789/0 conn(0x7fd6500c1f70 legacy=0x7fd650150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:13.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.370+0000 7fd647400640 1 --2- 172.21.15.67:0/4283184376 >> v2:172.21.15.134:3300/0 conn(0x7fd6300036c0 0x7fd630005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.370+0000 7fd647400640 1 -- 172.21.15.67:0/4283184376 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6500c3700 con 0x7fd630006190 2024-03-09T09:26:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.370+0000 7fd647400640 1 -- 172.21.15.67:0/4283184376 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6500c32b0 con 0x7fd6300036c0 2024-03-09T09:26:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.370+0000 7fd64d400640 1 -- 172.21.15.67:0/4283184376 >> v1:172.21.15.67:6789/0 conn(0x7fd630006190 legacy=0x7fd630008650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:26:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.370+0000 7fd64de00640 1 --2- 172.21.15.67:0/4283184376 >> v2:172.21.15.134:3300/0 conn(0x7fd6300036c0 0x7fd630005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.371+0000 7fd64de00640 1 -- 172.21.15.67:0/4283184376 >> v2:172.21.15.134:3300/0 conn(0x7fd6300036c0 msgr2=0x7fd630005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:26:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.371+0000 7fd64de00640 1 -- 172.21.15.67:0/4283184376 >> v2:172.21.15.134:3300/0 conn(0x7fd6300036c0 msgr2=0x7fd630005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.371+0000 7fd64de00640 1 --2- 172.21.15.67:0/4283184376 >> v2:172.21.15.134:3300/0 conn(0x7fd6300036c0 0x7fd630005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:13.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.371+0000 7fd64de00640 1 --2- 172.21.15.67:0/4283184376 >> v2:172.21.15.134:3300/0 conn(0x7fd6300036c0 0x7fd630005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:13.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.571+0000 7fd64d400640 1 -- 172.21.15.67:0/4283184376 >> v1:172.21.15.67:6789/0 conn(0x7fd630006190 legacy=0x7fd630008650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:26:13.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.572+0000 7fd64de00640 1 --2- 172.21.15.67:0/4283184376 >> v2:172.21.15.134:3300/0 conn(0x7fd6300036c0 0x7fd630005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.573+0000 7fd64de00640 1 -- 172.21.15.67:0/4283184376 >> v1:172.21.15.67:6789/0 conn(0x7fd630006190 legacy=0x7fd630008650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:13.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.573+0000 7fd64de00640 1 -- 172.21.15.67:0/4283184376 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63000afe0 con 0x7fd6300036c0 2024-03-09T09:26:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.575+0000 7fd64de00640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6300036c0 0x7fd630005b40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd640007d20 tx=0x7fd64002b950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=494f475ec2714e9c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.575+0000 7fd64ca00640 1 -- 172.21.15.67:0/4283184376 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640034070 con 0x7fd6300036c0 2024-03-09T09:26:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.575+0000 7fd64ca00640 1 -- 172.21.15.67:0/4283184376 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd640030610 con 0x7fd6300036c0 2024-03-09T09:26:13.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.575+0000 7fd64ca00640 1 -- 172.21.15.67:0/4283184376 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640034070 con 0x7fd6300036c0 2024-03-09T09:26:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.577+0000 7fd64f200640 1 -- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6300036c0 msgr2=0x7fd630005b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.577+0000 7fd64f200640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6300036c0 0x7fd630005b40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd640007d20 tx=0x7fd64002b950 comp rx=0 tx=0).stop 2024-03-09T09:26:13.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.577+0000 7fd64e800640 1 -- 172.21.15.67:0/4283184376 reap_dead start 2024-03-09T09:26:13.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.577+0000 7fd64f200640 1 -- 172.21.15.67:0/4283184376 shutdown_connections 2024-03-09T09:26:13.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.577+0000 7fd64f200640 1 --2- 172.21.15.67:0/4283184376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6300036c0 0x7fd630005b40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.577+0000 7fd64f200640 1 -- 172.21.15.67:0/4283184376 >> 172.21.15.67:0/4283184376 conn(0x7fd6500c7300 msgr2=0x7fd6500c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.577+0000 7fd64f200640 1 -- 172.21.15.67:0/4283184376 shutdown_connections 2024-03-09T09:26:13.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.577+0000 7fd64f200640 1 -- 172.21.15.67:0/4283184376 wait complete. 2024-03-09T09:26:13.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.583+0000 7fd64f200640 1 Processor -- start 2024-03-09T09:26:13.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.584+0000 7fd64f200640 1 -- start start 2024-03-09T09:26:13.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.586+0000 7fd64f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6300036c0 0x7fd65014a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.586+0000 7fd64f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd650149f60 0x7fd650147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.586+0000 7fd64f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd650147fb0 0x7fd650153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.586+0000 7fd64f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd650070d50 con 0x7fd650149f60 2024-03-09T09:26:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.586+0000 7fd64f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd650070830 con 0x7fd650147fb0 2024-03-09T09:26:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.586+0000 7fd64f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd650070ed0 con 0x7fd6300036c0 2024-03-09T09:26:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.586+0000 7fd64d400640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd650149f60 msgr2=0x7fd650147a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:13.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.586+0000 7fd64d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd650149f60 0x7fd650147a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.587+0000 7fd64e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd650147fb0 0x7fd650153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.587+0000 7fd64de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6300036c0 0x7fd65014a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.587+0000 7fd64e800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd650147fb0 0x7fd650153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:57626/0 (socket says 172.21.15.67:57626) 2024-03-09T09:26:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.587+0000 7fd64e800640 1 -- 172.21.15.67:0/1751253443 learned_addr learned my addr 172.21.15.67:0/1751253443 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.588+0000 7fd64e800640 1 -- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6300036c0 msgr2=0x7fd65014a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.588+0000 7fd64e800640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6300036c0 0x7fd65014a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.588+0000 7fd64e800640 1 -- 172.21.15.67:0/1751253443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd650149f60 msgr2=0x7fd650147a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:26:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.588+0000 7fd64e800640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd650149f60 0x7fd650147a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.588+0000 7fd64e800640 1 -- 172.21.15.67:0/1751253443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd650152e00 con 0x7fd650147fb0 2024-03-09T09:26:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.588+0000 7fd64de00640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6300036c0 0x7fd65014a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:26:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.590+0000 7fd64e800640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd650147fb0 0x7fd650153b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd6501517b0 tx=0x7fd648009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.590+0000 7fd646000640 1 -- 172.21.15.67:0/1751253443 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64801b070 con 0x7fd650147fb0 2024-03-09T09:26:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.590+0000 7fd646000640 1 -- 172.21.15.67:0/1751253443 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd648013a40 con 0x7fd650147fb0 2024-03-09T09:26:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.590+0000 7fd646000640 1 -- 172.21.15.67:0/1751253443 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648013e30 con 0x7fd650147fb0 2024-03-09T09:26:13.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.591+0000 7fd64f200640 1 -- 172.21.15.67:0/1751253443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65014b5a0 con 0x7fd650147fb0 2024-03-09T09:26:13.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.591+0000 7fd64f200640 1 -- 172.21.15.67:0/1751253443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6501eacb0 con 0x7fd650147fb0 2024-03-09T09:26:13.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.592+0000 7fd64f200640 1 -- 172.21.15.67:0/1751253443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65004f3c0 con 0x7fd650147fb0 2024-03-09T09:26:13.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.595+0000 7fd646000640 1 -- 172.21.15.67:0/1751253443 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd64801e070 con 0x7fd650147fb0 2024-03-09T09:26:13.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.597+0000 7fd646000640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd63006c280 0x7fd63006e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:13.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.597+0000 7fd646000640 1 -- 172.21.15.67:0/1751253443 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fd64808b930 con 0x7fd650147fb0 2024-03-09T09:26:13.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.602+0000 7fd64de00640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd63006c280 0x7fd63006e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:13.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.603+0000 7fd64de00640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd63006c280 0x7fd63006e740 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd640036000 tx=0x7fd640009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:13.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:13.606+0000 7fd646000640 1 -- 172.21.15.67:0/1751253443 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd648059dd0 con 0x7fd650147fb0 2024-03-09T09:26:14.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.058+0000 7fd64f200640 1 -- 172.21.15.67:0/1751253443 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd65014aee0 con 0x7fd650147fb0 2024-03-09T09:26:14.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.059+0000 7fd646000640 1 -- 172.21.15.67:0/1751253443 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7fd64805de70 con 0x7fd650147fb0 2024-03-09T09:26:14.072 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":264,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:26:14.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.066+0000 7fd64f200640 1 -- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd63006c280 msgr2=0x7fd63006e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.066+0000 7fd64f200640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd63006c280 0x7fd63006e740 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd640036000 tx=0x7fd640009030 comp rx=0 tx=0).stop 2024-03-09T09:26:14.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.066+0000 7fd64f200640 1 -- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd650147fb0 msgr2=0x7fd650153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.066+0000 7fd64f200640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd650147fb0 0x7fd650153b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd6501517b0 tx=0x7fd648009da0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.066+0000 7fd64f200640 1 -- 172.21.15.67:0/1751253443 shutdown_connections 2024-03-09T09:26:14.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.066+0000 7fd64f200640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd63006c280 0x7fd63006e740 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.066+0000 7fd64f200640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd650147fb0 0x7fd650153b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.066+0000 7fd64f200640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd650149f60 0x7fd650147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.066+0000 7fd64f200640 1 --2- 172.21.15.67:0/1751253443 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6300036c0 0x7fd65014a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.066+0000 7fd64f200640 1 -- 172.21.15.67:0/1751253443 >> 172.21.15.67:0/1751253443 conn(0x7fd6500c7300 msgr2=0x7fd6500c3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:14.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.066+0000 7fd64f200640 1 -- 172.21.15.67:0/1751253443 shutdown_connections 2024-03-09T09:26:14.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.067+0000 7fd64f200640 1 -- 172.21.15.67:0/1751253443 wait complete. 2024-03-09T09:26:14.120 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:26:14.121 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:26:14.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:26:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.524+0000 7f9302800640 1 Processor -- start 2024-03-09T09:26:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.525+0000 7f9302800640 1 -- start start 2024-03-09T09:26:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.526+0000 7f9302800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92fc157890 0x7f92fc06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.526+0000 7f9302800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f92fc06e630 0x7f92fc15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.526+0000 7f9302800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f92fc06dad0 0x7f92fc15bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.526+0000 7f9302800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92fc15b130 con 0x7f92fc06dad0 2024-03-09T09:26:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.526+0000 7f9302800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92fc15a370 con 0x7f92fc06e630 2024-03-09T09:26:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.526+0000 7f9302800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc15a650 con 0x7f92fc157890 2024-03-09T09:26:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.527+0000 7f92fbe00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7f92fc06dad0 msgr2=0x7f92fc15bb10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.527+0000 7f92fbe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f92fc06dad0 0x7f92fc15bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.527+0000 7f92faa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f92fc06e630 0x7f92fc15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.527+0000 7f92faa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f92fc06e630 0x7f92fc15b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:57640/0 (socket says 172.21.15.67:57640) 2024-03-09T09:26:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.527+0000 7f92faa00640 1 -- 172.21.15.67:0/3187742499 learned_addr learned my addr 172.21.15.67:0/3187742499 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.528+0000 7f92fb400640 1 --2- 172.21.15.67:0/3187742499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92fc157890 0x7f92fc06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.528+0000 7f92faa00640 1 -- 172.21.15.67:0/3187742499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92fc157890 msgr2=0x7f92fc06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:26:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.528+0000 7f92faa00640 1 --2- 172.21.15.67:0/3187742499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92fc157890 0x7f92fc06d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.528+0000 7f92faa00640 1 -- 172.21.15.67:0/3187742499 >> v2:172.21.15.67:3300/0 conn(0x7f92fc06dad0 msgr2=0x7f92fc15bb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.528+0000 7f92faa00640 1 --2- 172.21.15.67:0/3187742499 >> v2:172.21.15.67:3300/0 conn(0x7f92fc06dad0 0x7f92fc15bb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.528+0000 7f92faa00640 1 -- 172.21.15.67:0/3187742499 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc15a930 con 0x7f92fc06e630 2024-03-09T09:26:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.529+0000 7f92fb400640 1 --2- 172.21.15.67:0/3187742499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92fc157890 0x7f92fc06d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:26:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.529+0000 7f92faa00640 1 --2- 172.21.15.67:0/3187742499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc06e630 0x7f92fc15b5d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0c3dd0 tx=0x7f92e402b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa8da02ec5ead513 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.530+0000 7f92fa000640 1 -- 172.21.15.67:0/3187742499 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4034070 con 0x7f92fc06e630 2024-03-09T09:26:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.530+0000 7f92fa000640 1 -- 172.21.15.67:0/3187742499 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e4030890 con 0x7f92fc06e630 2024-03-09T09:26:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.530+0000 7f92fa000640 1 -- 172.21.15.67:0/3187742499 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4030ca0 con 0x7f92fc06e630 2024-03-09T09:26:14.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.531+0000 7f9302800640 1 -- 172.21.15.67:0/3187742499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc06e630 msgr2=0x7f92fc15b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.531+0000 7f9302800640 1 --2- 172.21.15.67:0/3187742499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc06e630 0x7f92fc15b5d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0c3dd0 tx=0x7f92e402b9d0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.531+0000 7f9302800640 1 -- 172.21.15.67:0/3187742499 shutdown_connections 2024-03-09T09:26:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.531+0000 7f9302800640 1 --2- 172.21.15.67:0/3187742499 >> v2:172.21.15.67:3300/0 conn(0x7f92fc06dad0 0x7f92fc15bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.531+0000 7f9302800640 1 --2- 172.21.15.67:0/3187742499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc06e630 0x7f92fc15b5d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.531+0000 7f9302800640 1 --2- 172.21.15.67:0/3187742499 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92fc157890 0x7f92fc06d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.531+0000 7f9302800640 1 -- 172.21.15.67:0/3187742499 >> 172.21.15.67:0/3187742499 conn(0x7f92fc0c7370 msgr2=0x7f92fc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.532+0000 7f9302800640 1 -- 172.21.15.67:0/3187742499 shutdown_connections 2024-03-09T09:26:14.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.532+0000 7f9302800640 1 -- 172.21.15.67:0/3187742499 wait complete. 2024-03-09T09:26:14.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.534+0000 7f9302800640 1 Processor -- start 2024-03-09T09:26:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.535+0000 7f9302800640 1 -- start start 2024-03-09T09:26:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.536+0000 7f9302800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc06dad0 0x7f92fc0c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.536+0000 7f9302800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92fc06e630 0x7f92fc1e5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.536+0000 7f9302800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92fc157890 0x7f92fc1e5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.536+0000 7f9302800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc0c1d70 con 0x7f92fc06e630 2024-03-09T09:26:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.536+0000 7f9302800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc15c050 con 0x7f92fc06dad0 2024-03-09T09:26:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.536+0000 7f9302800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc0c1ef0 con 0x7f92fc157890 2024-03-09T09:26:14.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.536+0000 7f92faa00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92fc06e630 msgr2=0x7f92fc1e5030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:14.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.536+0000 7f92faa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92fc06e630 0x7f92fc1e5030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.537+0000 7f92fb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc06dad0 0x7f92fc0c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.537+0000 7f92fb400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc06dad0 0x7f92fc0c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:57644/0 (socket says 172.21.15.67:57644) 2024-03-09T09:26:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.537+0000 7f92fb400640 1 -- 172.21.15.67:0/3217066781 learned_addr learned my addr 172.21.15.67:0/3217066781 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.537+0000 7f92fbe00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92fc157890 0x7f92fc1e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.538+0000 7f92fb400640 1 -- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92fc157890 msgr2=0x7f92fc1e5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.538+0000 7f92fb400640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92fc157890 0x7f92fc1e5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.538+0000 7f92fb400640 1 -- 172.21.15.67:0/3217066781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92fc06e630 msgr2=0x7f92fc1e5030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:26:14.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.538+0000 7f92fb400640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92fc06e630 0x7f92fc1e5030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.538+0000 7f92fb400640 1 -- 172.21.15.67:0/3217066781 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc1f12f0 con 0x7f92fc06dad0 2024-03-09T09:26:14.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.538+0000 7f92fbe00640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92fc157890 0x7f92fc1e5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:26:14.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.538+0000 7f92fb400640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc06dad0 0x7f92fc0c5e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00c7f0 tx=0x7f92ec00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:14.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.539+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec015070 con 0x7f92fc06dad0 2024-03-09T09:26:14.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.539+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ec007820 con 0x7f92fc06dad0 2024-03-09T09:26:14.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.539+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec007bd0 con 0x7f92fc06dad0 2024-03-09T09:26:14.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.539+0000 7f9302800640 1 -- 172.21.15.67:0/3217066781 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92fc1f1560 con 0x7f92fc06dad0 2024-03-09T09:26:14.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.539+0000 7f9302800640 1 -- 172.21.15.67:0/3217066781 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92fc1e5ab0 con 0x7f92fc06dad0 2024-03-09T09:26:14.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.541+0000 7f9302800640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc075a00 0x7f92fc1f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.541+0000 7f9302800640 1 -- 172.21.15.67:0/3217066781 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc157c70 con 0x7f92fc075a00 2024-03-09T09:26:14.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.541+0000 7f92faa00640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc075a00 0x7f92fc1f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.543+0000 7f92faa00640 1 -- 172.21.15.67:0/3217066781 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92fc068d20 con 0x7f92fc075a00 2024-03-09T09:26:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.543+0000 7f92faa00640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc075a00 0x7f92fc1f2040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f92fc070710 tx=0x7f92e40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.544+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f92ec018070 con 0x7f92fc06dad0 2024-03-09T09:26:14.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.545+0000 7f92f3400640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92d806b4d0 0x7f92d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.545+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f92ec08a400 con 0x7f92fc06dad0 2024-03-09T09:26:14.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.546+0000 7f92fbe00640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92d806b4d0 0x7f92d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.546+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4034040 con 0x7f92fc075a00 2024-03-09T09:26:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.546+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f92e4004020 con 0x7f92fc075a00 2024-03-09T09:26:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.546+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc075a00 msgr2=0x7f92fc1f2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.546+0000 7f92f3400640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc075a00 0x7f92fc1f2040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f92fc070710 tx=0x7f92e40046c0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.546+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc075a00 msgr2=0x7f92fc1f2040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:26:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.546+0000 7f92f3400640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc075a00 0x7f92fc1f2040 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f92fc070710 tx=0x7f92e40046c0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.547+0000 7f92fbe00640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92d806b4d0 0x7f92d806d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f92fc151c20 tx=0x7f92f400d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.559+0000 7f9302800640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc151eb0 0x7f92fc1f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.560+0000 7f9302800640 1 -- 172.21.15.67:0/3217066781 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc0c37e0 con 0x7f92fc151eb0 2024-03-09T09:26:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.561+0000 7f92faa00640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc151eb0 0x7f92fc1f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.561+0000 7f92faa00640 1 -- 172.21.15.67:0/3217066781 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f92fc1f1c20 con 0x7f92fc151eb0 2024-03-09T09:26:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.562+0000 7f92faa00640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc151eb0 0x7f92fc1f2580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f92e4000c00 tx=0x7f92e4009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:14.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.562+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4034040 con 0x7f92fc151eb0 2024-03-09T09:26:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.563+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4865 (secure 0 0 0) 0x7f92e4049880 con 0x7f92fc151eb0 2024-03-09T09:26:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.563+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc151eb0 msgr2=0x7f92fc1f2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.563+0000 7f92f3400640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc151eb0 0x7f92fc1f2580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f92e4000c00 tx=0x7f92e4009240 comp rx=0 tx=0).stop 2024-03-09T09:26:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.563+0000 7f92f3400640 1 -- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc151eb0 msgr2=0x7f92fc1f2580 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:26:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.563+0000 7f92f3400640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc151eb0 0x7f92fc1f2580 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f92e4000c00 tx=0x7f92e4009240 comp rx=0 tx=0).stop 2024-03-09T09:26:14.571 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:26:14.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:26:14.571 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:26:14.572 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:26:14.572 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 264, 2024-03-09T09:26:14.573 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:26:14.573 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:26:14.573 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:26:14.573 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.573 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:26:14.574 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:26:14.574 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:26:14.574 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:26:14.574 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:26:14.574 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:26:14.574 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:26:14.574 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:26:14.574 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:26:14.574 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:26:14.574 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:26:14.574 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:26:14.575 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:26:14.575 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:26:14.575 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:26:14.575 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.575 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:26:14.575 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:26:14.575 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:26:14.576 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:26:14.576 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:26:14.576 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:26:14.576 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:26:14.576 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:26:14.576 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:26:14.576 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:26:14.576 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:26:14.576 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:26:14.576 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:26:14.576 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.577 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:26:14.577 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:26:14.577 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:26:14.577 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:26:14.577 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:26:14.577 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:26:14.577 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:26:14.577 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:26:14.578 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:26:14.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.580 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:26:14.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:26:14.580 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:26:14.580 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:26:14.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:26:14.580 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:26:14.580 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:26:14.581 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:26:14.581 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.581 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.581 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.581 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:26:14.581 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:26:14.581 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:26:14.583 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:26:14.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.584 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:26:14.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:26:14.584 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:26:14.584 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:26:14.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.584 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:26:14.584 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:26:14.585 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:26:14.586 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:26:14.587 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.588 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.588 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:26:14.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.588 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:26:14.588 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:26:14.588 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:26:14.588 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.588 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.588 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:26:14.588 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:26:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.563+0000 7f9302800640 1 -- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92d806b4d0 msgr2=0x7f92d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.563+0000 7f9302800640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f92d806b4d0 0x7f92d806d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f92fc151c20 tx=0x7f92f400d040 comp rx=0 tx=0).stop 2024-03-09T09:26:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.563+0000 7f9302800640 1 -- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc06dad0 msgr2=0x7f92fc0c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.563+0000 7f9302800640 1 --2- 172.21.15.67:0/3217066781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92fc06dad0 0x7f92fc0c5e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00c7f0 tx=0x7f92ec00ccb0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.563+0000 7f92fbe00640 1 -- 172.21.15.67:0/3217066781 reap_dead start 2024-03-09T09:26:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.564+0000 7f92fbe00640 1 -- 172.21.15.67:0/3217066781 reap_dead start 2024-03-09T09:26:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.564+0000 7f9302800640 1 -- 172.21.15.67:0/3217066781 shutdown_connections 2024-03-09T09:26:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.564+0000 7f9302800640 1 -- 172.21.15.67:0/3217066781 >> 172.21.15.67:0/3217066781 conn(0x7f92fc0c7370 msgr2=0x7f92fc0c6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.564+0000 7f9302800640 1 -- 172.21.15.67:0/3217066781 shutdown_connections 2024-03-09T09:26:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.564+0000 7f9302800640 1 -- 172.21.15.67:0/3217066781 wait complete. 2024-03-09T09:26:14.591 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:26:14.591 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:26:14.591 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:26:14.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.805+0000 7f8b99000640 1 Processor -- start 2024-03-09T09:26:14.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.806+0000 7f8b99000640 1 -- start start 2024-03-09T09:26:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.807+0000 7f8b99000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8b940c5900 0x7f8b9406dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.807+0000 7f8b99000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b94156a40 con 0x7f8b940730d0 2024-03-09T09:26:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.807+0000 7f8b99000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b94073850 con 0x7f8b941535e0 2024-03-09T09:26:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.807+0000 7f8b99000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b94073570 con 0x7f8b940c5900 2024-03-09T09:26:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.807+0000 7f8b91400640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7f8b940730d0 legacy=0x7f8b940c2b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:26:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.808+0000 7f8b91e00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f8b941535e0 0x7f8b940729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:53336/0 (socket says 172.21.15.67:53336) 2024-03-09T09:26:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.808+0000 7f8b91e00640 1 -- 172.21.15.67:0/3555344133 learned_addr learned my addr 172.21.15.67:0/3555344133 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:26:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.808+0000 7f8b92800640 1 --2- 172.21.15.67:0/3555344133 >> v2:172.21.15.134:3300/0 conn(0x7f8b940c5900 0x7f8b9406dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.808+0000 7f8b90a00640 1 -- 172.21.15.67:0/3555344133 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074682147 0 0) 0x7f8b94073850 con 0x7f8b941535e0 2024-03-09T09:26:14.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.808+0000 7f8b90a00640 1 -- 172.21.15.67:0/3555344133 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b78003520 con 0x7f8b941535e0 2024-03-09T09:26:14.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.808+0000 7f8b92800640 1 -- 172.21.15.67:0/3555344133 >> v1:172.21.15.134:6789/0 conn(0x7f8b941535e0 legacy=0x7f8b940729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.808+0000 7f8b92800640 1 -- 172.21.15.67:0/3555344133 >> v1:172.21.15.67:6789/0 conn(0x7f8b940730d0 legacy=0x7f8b940c2b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:14.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.808+0000 7f8b92800640 1 -- 172.21.15.67:0/3555344133 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b94156ce0 con 0x7f8b940c5900 2024-03-09T09:26:14.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.809+0000 7f8b92800640 1 --2- 172.21.15.67:0/3555344133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b940c5900 0x7f8b9406dce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009d90 tx=0x7f8b8c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce94a726c884aaaa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:14.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.810+0000 7f8b90a00640 1 -- 172.21.15.67:0/3555344133 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c039070 con 0x7f8b940c5900 2024-03-09T09:26:14.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.810+0000 7f8b90a00640 1 -- 172.21.15.67:0/3555344133 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c02cc60 con 0x7f8b940c5900 2024-03-09T09:26:14.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.810+0000 7f8b90a00640 1 -- 172.21.15.67:0/3555344133 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c034660 con 0x7f8b940c5900 2024-03-09T09:26:14.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.810+0000 7f8b99000640 1 -- 172.21.15.67:0/3555344133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b940c5900 msgr2=0x7f8b9406dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.810+0000 7f8b99000640 1 --2- 172.21.15.67:0/3555344133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b940c5900 0x7f8b9406dce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009d90 tx=0x7f8b8c02beb0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.810+0000 7f8b99000640 1 -- 172.21.15.67:0/3555344133 shutdown_connections 2024-03-09T09:26:14.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.810+0000 7f8b99000640 1 --2- 172.21.15.67:0/3555344133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b940c5900 0x7f8b9406dce0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.810+0000 7f8b99000640 1 -- 172.21.15.67:0/3555344133 >> 172.21.15.67:0/3555344133 conn(0x7f8b940ab320 msgr2=0x7f8b940ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:14.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.811+0000 7f8b99000640 1 -- 172.21.15.67:0/3555344133 shutdown_connections 2024-03-09T09:26:14.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.811+0000 7f8b99000640 1 -- 172.21.15.67:0/3555344133 wait complete. 2024-03-09T09:26:14.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.813+0000 7f8b99000640 1 Processor -- start 2024-03-09T09:26:14.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.813+0000 7f8b99000640 1 -- start start 2024-03-09T09:26:14.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b99000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b940730d0 0x7f8b941ec180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b99000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940c5900 0x7f8b940aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b99000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b941535e0 0x7f8b940ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b99000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b94153b40 con 0x7f8b940730d0 2024-03-09T09:26:14.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b99000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b941539c0 con 0x7f8b941535e0 2024-03-09T09:26:14.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b99000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b940c3260 con 0x7f8b940c5900 2024-03-09T09:26:14.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b91e00640 1 -- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b940730d0 msgr2=0x7f8b941ec180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:14.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b91e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b940730d0 0x7f8b941ec180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b92800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b941535e0 0x7f8b940ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b92800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b941535e0 0x7f8b940ad6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:57658/0 (socket says 172.21.15.67:57658) 2024-03-09T09:26:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b92800640 1 -- 172.21.15.67:0/1183717806 learned_addr learned my addr 172.21.15.67:0/1183717806 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.815+0000 7f8b91400640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940c5900 0x7f8b940aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.816+0000 7f8b92800640 1 -- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940c5900 msgr2=0x7f8b940aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.816+0000 7f8b92800640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940c5900 0x7f8b940aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.816+0000 7f8b92800640 1 -- 172.21.15.67:0/1183717806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b940730d0 msgr2=0x7f8b941ec180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:26:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.816+0000 7f8b92800640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b940730d0 0x7f8b941ec180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.816+0000 7f8b92800640 1 -- 172.21.15.67:0/1183717806 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b94157750 con 0x7f8b941535e0 2024-03-09T09:26:14.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.816+0000 7f8b92800640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b941535e0 0x7f8b940ad6c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009d90 tx=0x7f8b8c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:14.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.817+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c044020 con 0x7f8b941535e0 2024-03-09T09:26:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.817+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c03f470 con 0x7f8b941535e0 2024-03-09T09:26:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.817+0000 7f8b99000640 1 -- 172.21.15.67:0/1183717806 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b940ac620 con 0x7f8b941535e0 2024-03-09T09:26:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.817+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c039040 con 0x7f8b941535e0 2024-03-09T09:26:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.817+0000 7f8b99000640 1 -- 172.21.15.67:0/1183717806 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b940acbe0 con 0x7f8b941535e0 2024-03-09T09:26:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.818+0000 7f8b99000640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940adc00 0x7f8b9414b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.818+0000 7f8b99000640 1 -- 172.21.15.67:0/1183717806 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b94075a00 con 0x7f8b940adc00 2024-03-09T09:26:14.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.819+0000 7f8b91e00640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940adc00 0x7f8b9414b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.819+0000 7f8b91e00640 1 -- 172.21.15.67:0/1183717806 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b940af650 con 0x7f8b940adc00 2024-03-09T09:26:14.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.819+0000 7f8b91e00640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940adc00 0x7f8b9414b520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b94154860 tx=0x7f8b8400c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:14.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.820+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8b8c047020 con 0x7f8b941535e0 2024-03-09T09:26:14.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.821+0000 7f8b8a000640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8b7406b4b0 0x7f8b7406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.821+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f8b8c0acfb0 con 0x7f8b941535e0 2024-03-09T09:26:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.822+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84017070 con 0x7f8b940adc00 2024-03-09T09:26:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.822+0000 7f8b91400640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8b7406b4b0 0x7f8b7406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.822+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f8b84005da0 con 0x7f8b940adc00 2024-03-09T09:26:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.822+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940adc00 msgr2=0x7f8b9414b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.822+0000 7f8b8a000640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940adc00 0x7f8b9414b520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b94154860 tx=0x7f8b8400c7a0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.822+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940adc00 msgr2=0x7f8b9414b520 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:26:14.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.822+0000 7f8b8a000640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940adc00 0x7f8b9414b520 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.823+0000 7f8b91400640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8b7406b4b0 0x7f8b7406d970 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8b800093d0 tx=0x7f8b80009360 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:14.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.831+0000 7f8b99000640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940ad2a0 0x7f8b9414ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:14.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.831+0000 7f8b99000640 1 -- 172.21.15.67:0/1183717806 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b940af3d0 con 0x7f8b940ad2a0 2024-03-09T09:26:14.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.831+0000 7f8b91e00640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940ad2a0 0x7f8b9414ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:14.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.832+0000 7f8b91e00640 1 -- 172.21.15.67:0/1183717806 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8b9406bc50 con 0x7f8b940ad2a0 2024-03-09T09:26:14.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.832+0000 7f8b91e00640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940ad2a0 0x7f8b9414ba60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b84000c00 tx=0x7f8b84000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:14.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.832+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84017040 con 0x7f8b940ad2a0 2024-03-09T09:26:14.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5176 (secure 0 0 0) 0x7f8b84012400 con 0x7f8b940ad2a0 2024-03-09T09:26:14.841 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:26:14.841 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:26:14.841 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:26:14.841 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:26:14.842 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 264, 2024-03-09T09:26:14.842 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:26:14.842 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:26:14.842 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:26:14.842 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.842 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:26:14.843 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:26:14.844 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:26:14.845 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:26:14.846 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:26:14.846 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:26:14.846 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:26:14.846 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:26:14.846 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:26:14.846 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:26:14.846 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:26:14.846 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:26:14.846 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:26:14.846 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:26:14.846 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:26:14.847 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:26:14.847 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:26:14.847 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:26:14.847 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:26:14.847 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:26:14.847 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:26:14.847 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:26:14.847 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.848 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:26:14.849 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:26:14.849 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.849 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:26:14.849 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:26:14.849 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.849 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.850 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.850 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:26:14.850 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:26:14.850 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:26:14.850 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:26:14.850 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:26:14.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:26:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:26:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:26:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:26:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:26:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:26:14.851 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.852 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:26:14.852 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:26:14.852 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.852 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.852 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.853 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:26:14.853 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:26:14.853 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:26:14.853 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:26:14.853 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:26:14.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.853 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:26:14.854 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:26:14.854 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:26:14.854 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:26:14.854 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:26:14.854 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:26:14.854 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.854 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.855 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.856 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:26:14.856 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:26:14.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.856 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:26:14.856 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:26:14.856 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:26:14.856 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.856 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.856 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:26:14.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.857 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.858 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:26:14.859 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:26:14.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:26:14.859 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:26:14.859 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:26:14.859 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:26:14.859 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:26:14.859 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:26:14.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:26:14.859 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:26:14.860 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:26:14.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940ad2a0 msgr2=0x7f8b9414ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b8a000640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940ad2a0 0x7f8b9414ba60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b84000c00 tx=0x7f8b84000ed0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b8a000640 1 -- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940ad2a0 msgr2=0x7f8b9414ba60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:26:14.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b8a000640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b940ad2a0 0x7f8b9414ba60 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8b84000c00 tx=0x7f8b84000ed0 comp rx=0 tx=0).stop 2024-03-09T09:26:14.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b99000640 1 -- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8b7406b4b0 msgr2=0x7f8b7406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b99000640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8b7406b4b0 0x7f8b7406d970 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8b800093d0 tx=0x7f8b80009360 comp rx=0 tx=0).stop 2024-03-09T09:26:14.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b99000640 1 -- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b941535e0 msgr2=0x7f8b940ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:14.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b99000640 1 --2- 172.21.15.67:0/1183717806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b941535e0 0x7f8b940ad6c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009d90 tx=0x7f8b8c004a40 comp rx=0 tx=0).stop 2024-03-09T09:26:14.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b92800640 1 -- 172.21.15.67:0/1183717806 reap_dead start 2024-03-09T09:26:14.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b92800640 1 -- 172.21.15.67:0/1183717806 reap_dead start 2024-03-09T09:26:14.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b99000640 1 -- 172.21.15.67:0/1183717806 shutdown_connections 2024-03-09T09:26:14.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b99000640 1 -- 172.21.15.67:0/1183717806 >> 172.21.15.67:0/1183717806 conn(0x7f8b940ab320 msgr2=0x7f8b9406db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:14.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b99000640 1 -- 172.21.15.67:0/1183717806 shutdown_connections 2024-03-09T09:26:14.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:14.834+0000 7f8b99000640 1 -- 172.21.15.67:0/1183717806 wait complete. 2024-03-09T09:26:14.862 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:26:14.862 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:26:18.182 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T09:26:18.183 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T09:26:18.184 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:18.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T09:26:18.566 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-intodir.sh... 2024-03-09T09:26:18.566 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-intodir.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-intodir.sh 2024-03-09T09:26:18.627 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir foo 2024-03-09T09:26:18.632 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir foo/.snap/one 2024-03-09T09:26:18.671 INFO:tasks.workunit.client.0.smithi067.stderr:+ touch bar 2024-03-09T09:26:18.675 INFO:tasks.workunit.client.0.smithi067.stderr:+ mv bar foo 2024-03-09T09:26:18.681 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T09:26:22.635 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir baz 2024-03-09T09:26:22.641 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir baz/.snap/two 2024-03-09T09:26:22.704 INFO:tasks.workunit.client.0.smithi067.stderr:+ mv baz foo 2024-03-09T09:26:22.714 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T09:26:22.985 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir foo/baz/.snap/two 2024-03-09T09:26:22.995 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir foo/.snap/one 2024-03-09T09:26:23.005 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -r foo 2024-03-09T09:26:23.025 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T09:26:23.025 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T09:26:23.026 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:23.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T09:26:23.131 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-multiple-capsnaps.sh... 2024-03-09T09:26:23.132 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-multiple-capsnaps.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-multiple-capsnaps.sh 2024-03-09T09:26:23.188 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T09:26:23.189 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo asdf 2024-03-09T09:26:23.190 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/1 2024-03-09T09:26:23.227 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 777 a 2024-03-09T09:26:23.229 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/2 2024-03-09T09:26:23.312 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo qwer 2024-03-09T09:26:23.325 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/3 2024-03-09T09:26:23.357 INFO:tasks.workunit.client.0.smithi067.stderr:+ chmod 666 a 2024-03-09T09:26:23.360 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/4 2024-03-09T09:26:23.394 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo zxcv 2024-03-09T09:26:23.404 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/5 2024-03-09T09:26:23.494 INFO:tasks.workunit.client.0.smithi067.stderr:+ ls -al .snap/1/a .snap/2/a .snap/3/a .snap/4/a .snap/5/a 2024-03-09T09:26:23.498 INFO:tasks.workunit.client.0.smithi067.stdout:-rw-r--r--. 1 ubuntu ubuntu 5 Mar 9 09:26 .snap/1/a 2024-03-09T09:26:23.498 INFO:tasks.workunit.client.0.smithi067.stdout:-rwxrwxrwx. 1 ubuntu ubuntu 5 Mar 9 09:26 .snap/2/a 2024-03-09T09:26:23.498 INFO:tasks.workunit.client.0.smithi067.stdout:-rwxrwxrwx. 1 ubuntu ubuntu 5 Mar 9 09:26 .snap/3/a 2024-03-09T09:26:23.498 INFO:tasks.workunit.client.0.smithi067.stdout:-rw-rw-rw-. 1 ubuntu ubuntu 5 Mar 9 09:26 .snap/4/a 2024-03-09T09:26:23.498 INFO:tasks.workunit.client.0.smithi067.stdout:-rw-rw-rw-. 1 ubuntu ubuntu 5 Mar 9 09:26 .snap/5/a 2024-03-09T09:26:23.499 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep asdf .snap/1/a 2024-03-09T09:26:23.502 INFO:tasks.workunit.client.0.smithi067.stdout:asdf 2024-03-09T09:26:23.503 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/1/a 2024-03-09T09:26:23.504 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'Size: 5' 2024-03-09T09:26:23.506 INFO:tasks.workunit.client.0.smithi067.stdout: Size: 5 Blocks: 1 IO Block: 4194304 regular file 2024-03-09T09:26:23.506 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep asdf .snap/2/a 2024-03-09T09:26:23.509 INFO:tasks.workunit.client.0.smithi067.stdout:asdf 2024-03-09T09:26:23.510 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/2/a 2024-03-09T09:26:23.510 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'Size: 5' 2024-03-09T09:26:23.513 INFO:tasks.workunit.client.0.smithi067.stdout: Size: 5 Blocks: 1 IO Block: 4194304 regular file 2024-03-09T09:26:23.514 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/2/a 2024-03-09T09:26:23.515 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep -- -rwxrwxrwx 2024-03-09T09:26:23.517 INFO:tasks.workunit.client.0.smithi067.stdout:Access: (0777/-rwxrwxrwx) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-09T09:26:23.517 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep qwer .snap/3/a 2024-03-09T09:26:23.520 INFO:tasks.workunit.client.0.smithi067.stdout:qwer 2024-03-09T09:26:23.521 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/3/a 2024-03-09T09:26:23.522 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'Size: 5' 2024-03-09T09:26:23.524 INFO:tasks.workunit.client.0.smithi067.stdout: Size: 5 Blocks: 1 IO Block: 4194304 regular file 2024-03-09T09:26:23.525 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/3/a 2024-03-09T09:26:23.525 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep -- -rwxrwxrwx 2024-03-09T09:26:23.529 INFO:tasks.workunit.client.0.smithi067.stdout:Access: (0777/-rwxrwxrwx) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-09T09:26:23.529 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep qwer .snap/4/a 2024-03-09T09:26:23.532 INFO:tasks.workunit.client.0.smithi067.stdout:qwer 2024-03-09T09:26:23.533 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/4/a 2024-03-09T09:26:23.534 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'Size: 5' 2024-03-09T09:26:23.536 INFO:tasks.workunit.client.0.smithi067.stdout: Size: 5 Blocks: 1 IO Block: 4194304 regular file 2024-03-09T09:26:23.537 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/4/a 2024-03-09T09:26:23.538 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep -- -rw-rw-rw- 2024-03-09T09:26:23.541 INFO:tasks.workunit.client.0.smithi067.stdout:Access: (0666/-rw-rw-rw-) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-09T09:26:23.541 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep zxcv .snap/5/a 2024-03-09T09:26:23.545 INFO:tasks.workunit.client.0.smithi067.stdout:zxcv 2024-03-09T09:26:23.546 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/5/a 2024-03-09T09:26:23.546 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep 'Size: 5' 2024-03-09T09:26:23.549 INFO:tasks.workunit.client.0.smithi067.stdout: Size: 5 Blocks: 1 IO Block: 4194304 regular file 2024-03-09T09:26:23.550 INFO:tasks.workunit.client.0.smithi067.stderr:+ stat .snap/5/a 2024-03-09T09:26:23.550 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep -- -rw-rw-rw- 2024-03-09T09:26:23.552 INFO:tasks.workunit.client.0.smithi067.stdout:Access: (0666/-rw-rw-rw-) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-09T09:26:23.553 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/1 .snap/2 .snap/3 .snap/4 .snap/5 2024-03-09T09:26:23.679 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T09:26:23.680 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T09:26:23.680 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:23.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T09:26:23.791 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-name-limits.sh... 2024-03-09T09:26:23.791 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-name-limits.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-name-limits.sh 2024-03-09T09:26:23.923 INFO:tasks.workunit.client.0.smithi067.stderr:rmdir: failed to remove 'd1/.snap/*': No such file or directory 2024-03-09T09:26:23.940 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T09:26:23.943 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:23.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T09:26:24.227 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-parents.sh... 2024-03-09T09:26:24.227 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-parents.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-parents.sh 2024-03-09T09:26:24.338 INFO:tasks.workunit.client.0.smithi067.stdout:making directory tree and files 2024-03-09T09:26:24.338 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'making directory tree and files' 2024-03-09T09:26:24.338 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir -p 1/a/b/c/ 2024-03-09T09:26:24.346 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'i'\''m file1' 2024-03-09T09:26:24.347 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'i'\''m file2' 2024-03-09T09:26:24.349 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'i'\''m file3' 2024-03-09T09:26:24.350 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo snapshotting 2024-03-09T09:26:24.350 INFO:tasks.workunit.client.0.smithi067.stdout:snapshotting 2024-03-09T09:26:24.351 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir 1/.snap/foosnap1 2024-03-09T09:26:24.401 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir 2 2024-03-09T09:26:24.408 INFO:tasks.workunit.client.0.smithi067.stdout:moving tree 2024-03-09T09:26:24.408 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'moving tree' 2024-03-09T09:26:24.409 INFO:tasks.workunit.client.0.smithi067.stderr:+ mv 1/a 2 2024-03-09T09:26:24.414 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'checking snapshot contains tree...' 2024-03-09T09:26:24.415 INFO:tasks.workunit.client.0.smithi067.stdout:checking snapshot contains tree... 2024-03-09T09:26:24.417 INFO:tasks.workunit.client.0.smithi067.stderr:++ find 1/.snap/foosnap1 2024-03-09T09:26:24.417 INFO:tasks.workunit.client.0.smithi067.stderr:++ wc -w 2024-03-09T09:26:24.455 INFO:tasks.workunit.client.0.smithi067.stderr:+ dir1=7 2024-03-09T09:26:24.460 INFO:tasks.workunit.client.0.smithi067.stderr:++ find 2/ 2024-03-09T09:26:24.460 INFO:tasks.workunit.client.0.smithi067.stderr:++ wc -w 2024-03-09T09:26:24.468 INFO:tasks.workunit.client.0.smithi067.stderr:+ dir2=7 2024-03-09T09:26:24.468 INFO:tasks.workunit.client.0.smithi067.stderr:+ test 7==7 2024-03-09T09:26:24.468 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'Success!' 2024-03-09T09:26:24.468 INFO:tasks.workunit.client.0.smithi067.stdout:Success! 2024-03-09T09:26:24.468 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'adding folder and file to tree...' 2024-03-09T09:26:24.469 INFO:tasks.workunit.client.0.smithi067.stdout:adding folder and file to tree... 2024-03-09T09:26:24.469 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir 2/a/b/c/d 2024-03-09T09:26:24.474 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'i'\''m file 4!' 2024-03-09T09:26:24.477 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'snapshotting tree 2' 2024-03-09T09:26:24.478 INFO:tasks.workunit.client.0.smithi067.stdout:snapshotting tree 2 2024-03-09T09:26:24.478 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir 2/.snap/barsnap2 2024-03-09T09:26:24.529 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'comparing snapshots' 2024-03-09T09:26:24.530 INFO:tasks.workunit.client.0.smithi067.stdout:comparing snapshots 2024-03-09T09:26:24.531 INFO:tasks.workunit.client.0.smithi067.stderr:++ find 1/.snap/foosnap1/ -maxdepth 2 2024-03-09T09:26:24.531 INFO:tasks.workunit.client.0.smithi067.stderr:++ wc -w 2024-03-09T09:26:24.538 INFO:tasks.workunit.client.0.smithi067.stderr:+ dir1=4 2024-03-09T09:26:24.540 INFO:tasks.workunit.client.0.smithi067.stderr:++ find 2/.snap/barsnap2/ -maxdepth 2 2024-03-09T09:26:24.540 INFO:tasks.workunit.client.0.smithi067.stderr:++ wc -w 2024-03-09T09:26:24.558 INFO:tasks.workunit.client.0.smithi067.stderr:+ dir2=4 2024-03-09T09:26:24.558 INFO:tasks.workunit.client.0.smithi067.stdout:Success! 2024-03-09T09:26:24.558 INFO:tasks.workunit.client.0.smithi067.stderr:+ test 4==4 2024-03-09T09:26:24.558 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'Success!' 2024-03-09T09:26:24.559 INFO:tasks.workunit.client.0.smithi067.stdout:moving subtree to first folder 2024-03-09T09:26:24.559 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'moving subtree to first folder' 2024-03-09T09:26:24.559 INFO:tasks.workunit.client.0.smithi067.stderr:+ mv 2/a/b/c 1 2024-03-09T09:26:24.565 INFO:tasks.workunit.client.0.smithi067.stdout:comparing snapshots and new tree 2024-03-09T09:26:24.565 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'comparing snapshots and new tree' 2024-03-09T09:26:24.566 INFO:tasks.workunit.client.0.smithi067.stderr:++ find 1/ 2024-03-09T09:26:24.567 INFO:tasks.workunit.client.0.smithi067.stderr:++ wc -w 2024-03-09T09:26:24.588 INFO:tasks.workunit.client.0.smithi067.stderr:+ dir1=5 2024-03-09T09:26:24.589 INFO:tasks.workunit.client.0.smithi067.stderr:++ find 2/.snap/barsnap2/a/b/c 2024-03-09T09:26:24.590 INFO:tasks.workunit.client.0.smithi067.stderr:++ wc -w 2024-03-09T09:26:24.604 INFO:tasks.workunit.client.0.smithi067.stderr:+ dir2=4 2024-03-09T09:26:24.604 INFO:tasks.workunit.client.0.smithi067.stderr:+ test 5==4 2024-03-09T09:26:24.605 INFO:tasks.workunit.client.0.smithi067.stdout:Success! 2024-03-09T09:26:24.605 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'Success!' 2024-03-09T09:26:24.605 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir 1/.snap/foosnap1 2024-03-09T09:26:24.615 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir 2/.snap/barsnap2 2024-03-09T09:26:24.626 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T09:26:24.627 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T09:26:24.627 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:24.627 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T09:26:25.197 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-realm-split.sh... 2024-03-09T09:26:25.197 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-realm-split.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-realm-split.sh 2024-03-09T09:26:25.350 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T09:26:25.350 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir -p 1/a 2024-03-09T09:26:25.360 INFO:tasks.workunit.client.0.smithi067.stderr:+ exec 2024-03-09T09:26:25.363 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo -n a 2024-03-09T09:26:25.363 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir 1/.snap/s1 2024-03-09T09:26:25.504 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo -n b 2024-03-09T09:26:25.519 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir 2 2024-03-09T09:26:25.531 INFO:tasks.workunit.client.0.smithi067.stderr:+ mv 1/a 2 2024-03-09T09:26:25.538 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir 2/.snap/s2 2024-03-09T09:26:25.602 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo -n c 2024-03-09T09:26:26.219 INFO:tasks.workunit.client.0.smithi067.stderr:+ exec 2024-03-09T09:26:26.219 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep '^a$' 1/.snap/s1/a/file1 2024-03-09T09:26:26.230 INFO:tasks.workunit.client.0.smithi067.stdout:a 2024-03-09T09:26:26.233 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep '^ab$' 2/.snap/s2/a/file1 2024-03-09T09:26:26.857 INFO:tasks.workunit.client.0.smithi067.stdout:ab 2024-03-09T09:26:26.857 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep '^abc$' 2/a/file1 2024-03-09T09:26:26.870 INFO:tasks.workunit.client.0.smithi067.stdout:abc 2024-03-09T09:26:26.874 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir 1/.snap/s1 2024-03-09T09:26:26.901 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir 2/.snap/s2 2024-03-09T09:26:26.923 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -rf 1 2 2024-03-09T09:26:27.181 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T09:26:27.182 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T09:26:27.182 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:27.183 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T09:26:27.374 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-snap-rename.sh... 2024-03-09T09:26:27.375 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-snap-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-snap-rename.sh 2024-03-09T09:26:27.469 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T09:26:27.469 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir -p d1/d2 2024-03-09T09:26:27.480 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir -p d1/d3 2024-03-09T09:26:27.485 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir d1/.snap/foo 2024-03-09T09:26:27.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir d1/d2/.snap/foo 2024-03-09T09:26:27.769 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir d1/d3/.snap/foo 2024-03-09T09:26:28.548 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir d1/d3/.snap/bar 2024-03-09T09:26:28.582 INFO:tasks.workunit.client.0.smithi067.stderr:+ mv d1/d2/.snap/foo d1/d2/.snap/bar 2024-03-09T09:26:28.686 INFO:tasks.workunit.client.0.smithi067.stderr:+ expect_failure mv d1/d2/.snap/bar d1/d2/.snap/_bar 2024-03-09T09:26:28.687 INFO:tasks.workunit.client.0.smithi067.stderr:+ mv d1/d2/.snap/bar d1/d2/.snap/_bar 2024-03-09T09:26:28.696 INFO:tasks.workunit.client.0.smithi067.stderr:mv: cannot move 'd1/d2/.snap/bar' to a subdirectory of itself, 'd1/d2/.snap/_bar' 2024-03-09T09:26:28.697 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 0 2024-03-09T09:26:28.699 INFO:tasks.workunit.client.0.smithi067.stderr:+ expect_failure mv d1/d2/.snap/_foo_1099511669189 d1/d2/.snap/foo 2024-03-09T09:26:28.699 INFO:tasks.workunit.client.0.smithi067.stderr:+ mv d1/d2/.snap/_foo_1099511669189 d1/d2/.snap/foo 2024-03-09T09:26:28.718 INFO:tasks.workunit.client.0.smithi067.stderr:mv: cannot move 'd1/d2/.snap/_foo_1099511669189' to a subdirectory of itself, 'd1/d2/.snap/foo' 2024-03-09T09:26:28.719 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 0 2024-03-09T09:26:28.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ expect_failure mv d1/d2/.snap/_foo_1099511669189 d1/d2/.snap/_foo_1 2024-03-09T09:26:28.722 INFO:tasks.workunit.client.0.smithi067.stderr:+ mv d1/d2/.snap/_foo_1099511669189 d1/d2/.snap/_foo_1 2024-03-09T09:26:28.734 INFO:tasks.workunit.client.0.smithi067.stderr:mv: cannot move 'd1/d2/.snap/_foo_1099511669189' to a subdirectory of itself, 'd1/d2/.snap/_foo_1' 2024-03-09T09:26:28.734 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 0 2024-03-09T09:26:28.734 INFO:tasks.workunit.client.0.smithi067.stderr:+ expect_failure mv d1/d2/.snap/bar d1/.snap/ 2024-03-09T09:26:28.735 INFO:tasks.workunit.client.0.smithi067.stderr:+ mv d1/d2/.snap/bar d1/.snap/ 2024-03-09T09:26:28.742 INFO:tasks.workunit.client.0.smithi067.stderr:mv: cannot move 'd1/d2/.snap/bar' to 'd1/.snap/bar': Read-only file system 2024-03-09T09:26:28.743 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 0 2024-03-09T09:26:28.743 INFO:tasks.workunit.client.0.smithi067.stderr:+ expect_failure python3 -c 'import os; os.rename('\''d1/d3/.snap/foo'\'', '\''d1/d3/.snap/bar'\'')' 2024-03-09T09:26:28.743 INFO:tasks.workunit.client.0.smithi067.stderr:+ python3 -c 'import os; os.rename('\''d1/d3/.snap/foo'\'', '\''d1/d3/.snap/bar'\'')' 2024-03-09T09:26:28.783 INFO:tasks.workunit.client.0.smithi067.stderr:Traceback (most recent call last): 2024-03-09T09:26:28.783 INFO:tasks.workunit.client.0.smithi067.stderr: File "", line 1, in 2024-03-09T09:26:28.783 INFO:tasks.workunit.client.0.smithi067.stderr:FileExistsError: [Errno 17] File exists: 'd1/d3/.snap/foo' -> 'd1/d3/.snap/bar' 2024-03-09T09:26:28.791 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 0 2024-03-09T09:26:28.791 INFO:tasks.workunit.client.0.smithi067.stderr:+ expect_failure python3 -c 'import os; os.rename('\''d1/.snap/foo'\'', '\''d1/foo'\'')' 2024-03-09T09:26:28.791 INFO:tasks.workunit.client.0.smithi067.stderr:+ python3 -c 'import os; os.rename('\''d1/.snap/foo'\'', '\''d1/foo'\'')' 2024-03-09T09:26:28.834 INFO:tasks.workunit.client.0.smithi067.stderr:Traceback (most recent call last): 2024-03-09T09:26:28.834 INFO:tasks.workunit.client.0.smithi067.stderr: File "", line 1, in 2024-03-09T09:26:28.834 INFO:tasks.workunit.client.0.smithi067.stderr:OSError: [Errno 18] Invalid cross-device link: 'd1/.snap/foo' -> 'd1/foo' 2024-03-09T09:26:28.837 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 0 2024-03-09T09:26:28.837 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir d1/.snap/foo 2024-03-09T09:26:28.983 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir d1/d2/.snap/bar 2024-03-09T09:26:29.008 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir d1/d3/.snap/foo 2024-03-09T09:26:29.025 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir d1/d3/.snap/bar 2024-03-09T09:26:29.048 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -rf d1 2024-03-09T09:26:29.171 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T09:26:29.171 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T09:26:29.176 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:29.176 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T09:26:29.383 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-snap-rm-cmp.sh... 2024-03-09T09:26:29.383 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-snap-rm-cmp.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-snap-rm-cmp.sh 2024-03-09T09:26:29.492 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T09:26:29.496 INFO:tasks.workunit.client.0.smithi067.stderr:+ file=linux-2.6.33.tar.bz2 2024-03-09T09:26:29.496 INFO:tasks.workunit.client.0.smithi067.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2024-03-09T09:26:32.564 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum linux-2.6.33.tar.bz2 2024-03-09T09:26:32.564 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:32.695 INFO:tasks.workunit.client.0.smithi067.stderr:+ real=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:32.695 INFO:tasks.workunit.client.0.smithi067.stderr:++ seq 1 20 2024-03-09T09:26:32.698 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:32.698 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 1 2024-03-09T09:26:32.698 INFO:tasks.workunit.client.0.smithi067.stdout:1 2024-03-09T09:26:32.699 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:32.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:26:32.707 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:26:32.835 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:32.875 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:32.882 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:34.708 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:34.709 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:34.827 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:34.828 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:34.828 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:34.863 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-03-09T09:26:34.863 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-03-09T09:26:34.864 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-09T09:26:34.864 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-09T09:26:34.876 INFO:tasks.ceph.mon.a:Started 2024-03-09T09:26:34.876 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:26:34.876 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:26:34.952 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:34.952 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 2 2024-03-09T09:26:34.952 INFO:tasks.workunit.client.0.smithi067.stdout:2 2024-03-09T09:26:34.953 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:35.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.146+0000 7fabf7e00640 1 Processor -- start 2024-03-09T09:26:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.147+0000 7fabf7e00640 1 -- start start 2024-03-09T09:26:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.148+0000 7fabf7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fabf00c37c0 0x7fabf00ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.149+0000 7fabf7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf00ab550 con 0x7fabf015a1b0 2024-03-09T09:26:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.149+0000 7fabf7e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf015a700 con 0x7fabf00a9ab0 2024-03-09T09:26:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.149+0000 7fabf7e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf015a9e0 con 0x7fabf00c37c0 2024-03-09T09:26:35.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.149+0000 7fabf5e00640 1 -- >> v2:172.21.15.67:3300/0 conn(0x7fabf00c37c0 msgr2=0x7fabf00ab010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:35.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.149+0000 7fabf5e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fabf00c37c0 0x7fabf00ab010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:35.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.149+0000 7fabf4a00640 1 -- >> v1:172.21.15.67:6789/0 conn(0x7fabf015a1b0 legacy=0x7fabf00aa900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.67:6789/0 2024-03-09T09:26:35.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.149+0000 7fabf5400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fabf00a9ab0 0x7fabf00ab790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:58622/0 (socket says 172.21.15.67:58622) 2024-03-09T09:26:35.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.149+0000 7fabf5400640 1 -- 172.21.15.67:0/1655678692 learned_addr learned my addr 172.21.15.67:0/1655678692 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:26:35.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.150+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457480900 0 0) 0x7fabf015a700 con 0x7fabf00a9ab0 2024-03-09T09:26:35.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.150+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd8003520 con 0x7fabf00a9ab0 2024-03-09T09:26:35.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4040846294 0 0) 0x7fabd8003520 con 0x7fabf00a9ab0 2024-03-09T09:26:35.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 >> v2:172.21.15.67:3300/0 conn(0x7fabf00c37c0 msgr2=0x7fabf00ab010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:35.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 --2- 172.21.15.67:0/1655678692 >> v2:172.21.15.67:3300/0 conn(0x7fabf00c37c0 0x7fabf00ab010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:35.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 >> v1:172.21.15.67:6789/0 conn(0x7fabf015a1b0 legacy=0x7fabf00aa900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:35.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf015acc0 con 0x7fabf00a9ab0 2024-03-09T09:26:35.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fabe40019f0 con 0x7fabf00a9ab0 2024-03-09T09:26:35.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 >> v1:172.21.15.134:6789/0 conn(0x7fabf00a9ab0 legacy=0x7fabf00ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 --2- 172.21.15.67:0/1655678692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd80039b0 0x7fabd8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 --2- 172.21.15.67:0/1655678692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabd8004800 0x7fabd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 --2- 172.21.15.67:0/1655678692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabd8005210 0x7fabd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf015a9e0 con 0x7fabd8004800 2024-03-09T09:26:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd800c140 con 0x7fabd80039b0 2024-03-09T09:26:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd800c340 con 0x7fabd8005210 2024-03-09T09:26:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabf4a00640 1 -- 172.21.15.67:0/1655678692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabd8004800 msgr2=0x7fabd8004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.67:3300/0 2024-03-09T09:26:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabf4a00640 1 --2- 172.21.15.67:0/1655678692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabd8004800 0x7fabd8004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.151+0000 7fabf5400640 1 --2- 172.21.15.67:0/1655678692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd80039b0 0x7fabd8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.152+0000 7fabf5e00640 1 --2- 172.21.15.67:0/1655678692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabd8005210 0x7fabd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:35.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.153+0000 7fabf5e00640 1 -- 172.21.15.67:0/1655678692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd80039b0 msgr2=0x7fabd8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:26:35.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.153+0000 7fabf5e00640 1 --2- 172.21.15.67:0/1655678692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabd80039b0 0x7fabd8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:35.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.153+0000 7fabf5e00640 1 -- 172.21.15.67:0/1655678692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabd8004800 msgr2=0x7fabd8004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:35.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.153+0000 7fabf5e00640 1 --2- 172.21.15.67:0/1655678692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabd8004800 0x7fabd8004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:35.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.153+0000 7fabf5e00640 1 -- 172.21.15.67:0/1655678692 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf015acc0 con 0x7fabd8005210 2024-03-09T09:26:35.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.154+0000 7fabf5e00640 1 -- 172.21.15.67:0/1655678692 reap_dead start 2024-03-09T09:26:35.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.154+0000 7fabf5e00640 1 --2- 172.21.15.67:0/1655678692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabd8005210 0x7fabd800b9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fabe80098f0 tx=0x7fabd8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f5ca65d19ea61c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:35.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.155+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80094f0 con 0x7fabd8005210 2024-03-09T09:26:35.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.155+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe8003b80 con 0x7fabd8005210 2024-03-09T09:26:35.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.155+0000 7fabefe00640 1 -- 172.21.15.67:0/1655678692 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8005110 con 0x7fabd8005210 2024-03-09T09:26:35.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.156+0000 7fabf7e00640 1 -- 172.21.15.67:0/1655678692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabd8005210 msgr2=0x7fabd800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:35.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.156+0000 7fabf7e00640 1 --2- 172.21.15.67:0/1655678692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabd8005210 0x7fabd800b9c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fabe80098f0 tx=0x7fabd8005830 comp rx=0 tx=0).stop 2024-03-09T09:26:35.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.156+0000 7fabf7e00640 1 -- 172.21.15.67:0/1655678692 shutdown_connections 2024-03-09T09:26:35.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.156+0000 7fabf7e00640 1 --2- 172.21.15.67:0/1655678692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabd8005210 0x7fabd800b9c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:35.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.156+0000 7fabf7e00640 1 -- 172.21.15.67:0/1655678692 >> 172.21.15.67:0/1655678692 conn(0x7fabf00c7370 msgr2=0x7fabf00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:35.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.157+0000 7fabf7e00640 1 -- 172.21.15.67:0/1655678692 shutdown_connections 2024-03-09T09:26:35.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.157+0000 7fabf7e00640 1 -- 172.21.15.67:0/1655678692 wait complete. 2024-03-09T09:26:35.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.159+0000 7fabf7e00640 1 Processor -- start 2024-03-09T09:26:35.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.160+0000 7fabf7e00640 1 -- start start 2024-03-09T09:26:35.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.161+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabd8005210 0x7fabf0154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:35.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.161+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf0155f10 0x7fabf014f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:35.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.161+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf015ceb0 0x7fabf014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:35.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.161+0000 7fabf7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf006dfa0 con 0x7fabd8005210 2024-03-09T09:26:35.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.161+0000 7fabf7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00a9e90 con 0x7fabf0155f10 2024-03-09T09:26:35.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.161+0000 7fabf7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf006e120 con 0x7fabf015ceb0 2024-03-09T09:26:35.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.162+0000 7fabf4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf0155f10 0x7fabf014f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:35.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.162+0000 7fabf4a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf0155f10 0x7fabf014f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:53950/0 (socket says 172.21.15.67:53950) 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.162+0000 7fabf4a00640 1 -- 172.21.15.67:0/1593614492 learned_addr learned my addr 172.21.15.67:0/1593614492 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.162+0000 7fabf5e00640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf015ceb0 0x7fabf014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.163+0000 7fabf4a00640 1 -- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf015ceb0 msgr2=0x7fabf014fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.163+0000 7fabf4a00640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf015ceb0 0x7fabf014fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.163+0000 7fabf4a00640 1 -- 172.21.15.67:0/1593614492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabd8005210 msgr2=0x7fabf0154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.163+0000 7fabf4a00640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabd8005210 0x7fabf0154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.163+0000 7fabf4a00640 1 -- 172.21.15.67:0/1593614492 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf015c000 con 0x7fabf0155f10 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.163+0000 7fabf4a00640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf0155f10 0x7fabf014f6a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00b550 tx=0x7fabdc00ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.163+0000 7fabf5e00640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf015ceb0 0x7fabf014fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.163+0000 7fabed600640 1 -- 172.21.15.67:0/1593614492 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc003b90 con 0x7fabf0155f10 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.164+0000 7fabed600640 1 -- 172.21.15.67:0/1593614492 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc015070 con 0x7fabf0155f10 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.164+0000 7fabed600640 1 -- 172.21.15.67:0/1593614492 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc005070 con 0x7fabf0155f10 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.164+0000 7fabf7e00640 1 -- 172.21.15.67:0/1593614492 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf01f08a0 con 0x7fabf0155f10 2024-03-09T09:26:35.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.164+0000 7fabf7e00640 1 -- 172.21.15.67:0/1593614492 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf015c5a0 con 0x7fabf0155f10 2024-03-09T09:26:35.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.166+0000 7fabf7e00640 1 -- 172.21.15.67:0/1593614492 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf015c7a0 con 0x7fabf0155f10 2024-03-09T09:26:35.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.167+0000 7fabed600640 1 -- 172.21.15.67:0/1593614492 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fabdc005210 con 0x7fabf0155f10 2024-03-09T09:26:35.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.168+0000 7fabed600640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fabd006b550 0x7fabd006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:35.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.168+0000 7fabed600640 1 -- 172.21.15.67:0/1593614492 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fabdc089f30 con 0x7fabf0155f10 2024-03-09T09:26:35.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.169+0000 7fabf5400640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fabd006b550 0x7fabd006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:35.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.170+0000 7fabf5400640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fabd006b550 0x7fabd006da10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fabe4001370 tx=0x7fabe40017b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.177+0000 7fabed600640 1 -- 172.21.15.67:0/1593614492 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fabf015c7a0 con 0x7fabf0155f10 2024-03-09T09:26:35.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.553+0000 7fabf7e00640 1 -- 172.21.15.67:0/1593614492 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fabf0075a00 con 0x7fabf0155f10 2024-03-09T09:26:35.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.554+0000 7fabed600640 1 -- 172.21.15.67:0/1593614492 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fabdc017310 con 0x7fabf0155f10 2024-03-09T09:26:35.561 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":264,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:26:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.556+0000 7fabf7e00640 1 -- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fabd006b550 msgr2=0x7fabd006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.556+0000 7fabf7e00640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fabd006b550 0x7fabd006da10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fabe4001370 tx=0x7fabe40017b0 comp rx=0 tx=0).stop 2024-03-09T09:26:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.556+0000 7fabf7e00640 1 -- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf0155f10 msgr2=0x7fabf014f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.556+0000 7fabf7e00640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf0155f10 0x7fabf014f6a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00b550 tx=0x7fabdc00ba10 comp rx=0 tx=0).stop 2024-03-09T09:26:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.556+0000 7fabf7e00640 1 -- 172.21.15.67:0/1593614492 shutdown_connections 2024-03-09T09:26:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.556+0000 7fabf7e00640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fabd006b550 0x7fabd006da10 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.556+0000 7fabf7e00640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabf015ceb0 0x7fabf014fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.556+0000 7fabf7e00640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabf0155f10 0x7fabf014f6a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:35.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.556+0000 7fabf7e00640 1 --2- 172.21.15.67:0/1593614492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fabd8005210 0x7fabf0154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:35.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.557+0000 7fabf7e00640 1 -- 172.21.15.67:0/1593614492 >> 172.21.15.67:0/1593614492 conn(0x7fabf00c7370 msgr2=0x7fabf01590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:35.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.557+0000 7fabf7e00640 1 -- 172.21.15.67:0/1593614492 shutdown_connections 2024-03-09T09:26:35.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:35.557+0000 7fabf7e00640 1 -- 172.21.15.67:0/1593614492 wait complete. 2024-03-09T09:26:35.593 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:35.637 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:35.650 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:36.307 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:36.307 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:36.439 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:36.440 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:36.440 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:36.476 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:36.476 INFO:tasks.workunit.client.0.smithi067.stdout:3 2024-03-09T09:26:36.476 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 3 2024-03-09T09:26:36.477 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:36.626 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:36.670 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:36.677 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:37.492 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:37.492 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:37.631 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:37.631 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:37.631 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:37.675 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:37.675 INFO:tasks.workunit.client.0.smithi067.stdout:4 2024-03-09T09:26:37.675 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 4 2024-03-09T09:26:37.676 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:37.803 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:38.583 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:26:38.712 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:38.719 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:38.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.847+0000 7f0c9c200640 1 Processor -- start 2024-03-09T09:26:38.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.847+0000 7f0c9c200640 1 -- start start 2024-03-09T09:26:38.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.849+0000 7f0c9c200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c941635b0 0x7f0c9408c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:38.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.849+0000 7f0c9c200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0c940aac00 0x7f0c940ab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:38.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.849+0000 7f0c9c200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c940aa420 con 0x7f0c94073200 2024-03-09T09:26:38.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.849+0000 7f0c9c200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9406de20 con 0x7f0c940aac00 2024-03-09T09:26:38.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.849+0000 7f0c9c200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c940c3510 con 0x7f0c941635b0 2024-03-09T09:26:38.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.850+0000 7f0c99800640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f0c94073200 0x7f0c9408c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:40130/0 (socket says 172.21.15.67:40130) 2024-03-09T09:26:38.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.850+0000 7f0c99800640 1 -- 172.21.15.67:0/1618531976 learned_addr learned my addr 172.21.15.67:0/1618531976 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:26:38.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.850+0000 7f0c9a200640 1 --2- 172.21.15.67:0/1618531976 >> v2:172.21.15.134:3300/0 conn(0x7f0c940aac00 0x7f0c940ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:38.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.850+0000 7f0c98e00640 1 --2- 172.21.15.67:0/1618531976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c941635b0 0x7f0c9408c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:38.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.851+0000 7f0c9a200640 1 -- 172.21.15.67:0/1618531976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c941635b0 msgr2=0x7f0c9408c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:26:38.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.851+0000 7f0c9a200640 1 --2- 172.21.15.67:0/1618531976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c941635b0 0x7f0c9408c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:38.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.851+0000 7f0c9a200640 1 -- 172.21.15.67:0/1618531976 >> v1:172.21.15.134:6789/0 conn(0x7f0c94073200 legacy=0x7f0c9408c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:26:38.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.851+0000 7f0c9a200640 1 -- 172.21.15.67:0/1618531976 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c940c37f0 con 0x7f0c940aac00 2024-03-09T09:26:38.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.856+0000 7f0c9a200640 1 --2- 172.21.15.67:0/1618531976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c940aac00 0x7f0c940ab080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009d90 tx=0x7f0c8c02bdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=785c7aa51d7fa666 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:38.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.858+0000 7f0c93e00640 1 -- 172.21.15.67:0/1618531976 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c039070 con 0x7f0c940aac00 2024-03-09T09:26:38.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.858+0000 7f0c93e00640 1 -- 172.21.15.67:0/1618531976 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8c02cb60 con 0x7f0c940aac00 2024-03-09T09:26:38.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.858+0000 7f0c93e00640 1 -- 172.21.15.67:0/1618531976 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c034440 con 0x7f0c940aac00 2024-03-09T09:26:38.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.859+0000 7f0c9c200640 1 -- 172.21.15.67:0/1618531976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c940aac00 msgr2=0x7f0c940ab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:38.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.859+0000 7f0c9c200640 1 --2- 172.21.15.67:0/1618531976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c940aac00 0x7f0c940ab080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009d90 tx=0x7f0c8c02bdb0 comp rx=0 tx=0).stop 2024-03-09T09:26:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.861+0000 7f0c9c200640 1 -- 172.21.15.67:0/1618531976 shutdown_connections 2024-03-09T09:26:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.861+0000 7f0c9c200640 1 --2- 172.21.15.67:0/1618531976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c940aac00 0x7f0c940ab080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.861+0000 7f0c9c200640 1 --2- 172.21.15.67:0/1618531976 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c941635b0 0x7f0c9408c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.861+0000 7f0c9c200640 1 -- 172.21.15.67:0/1618531976 >> 172.21.15.67:0/1618531976 conn(0x7f0c940c7370 msgr2=0x7f0c940c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.861+0000 7f0c9c200640 1 -- 172.21.15.67:0/1618531976 shutdown_connections 2024-03-09T09:26:38.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.861+0000 7f0c9c200640 1 -- 172.21.15.67:0/1618531976 wait complete. 2024-03-09T09:26:38.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.865+0000 7f0c9c200640 1 Processor -- start 2024-03-09T09:26:38.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.865+0000 7f0c9c200640 1 -- start start 2024-03-09T09:26:38.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.867+0000 7f0c9c200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c94073200 0x7f0c94204ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:38.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.867+0000 7f0c9c200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c940aac00 0x7f0c94071630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:38.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.868+0000 7f0c9c200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c941635b0 0x7f0c94071b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:38.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.868+0000 7f0c9c200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c940c1d70 con 0x7f0c94073200 2024-03-09T09:26:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.868+0000 7f0c9c200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c940aa6e0 con 0x7f0c941635b0 2024-03-09T09:26:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.870+0000 7f0c99800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c94073200 0x7f0c94204ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:38.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.870+0000 7f0c9a200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c941635b0 0x7f0c94071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:38.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.870+0000 7f0c9a200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c941635b0 0x7f0c94071b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:56404/0 (socket says 172.21.15.67:56404) 2024-03-09T09:26:38.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.871+0000 7f0c9a200640 1 -- 172.21.15.67:0/3222642856 learned_addr learned my addr 172.21.15.67:0/3222642856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:38.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.870+0000 7f0c99800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c94073200 0x7f0c94204ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39966/0 (socket says 172.21.15.67:39966) 2024-03-09T09:26:38.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.871+0000 7f0c9c200640 1 -- 172.21.15.67:0/3222642856 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c941639f0 con 0x7f0c940aac00 2024-03-09T09:26:38.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.871+0000 7f0c98e00640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c940aac00 0x7f0c94071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:38.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.874+0000 7f0c9a200640 1 -- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c940aac00 msgr2=0x7f0c94071630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:38.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.874+0000 7f0c9a200640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c940aac00 0x7f0c94071630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:38.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.874+0000 7f0c9a200640 1 -- 172.21.15.67:0/3222642856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c94073200 msgr2=0x7f0c94204ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:38.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.874+0000 7f0c9a200640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c94073200 0x7f0c94204ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:38.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.874+0000 7f0c9a200640 1 -- 172.21.15.67:0/3222642856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c940a9700 con 0x7f0c941635b0 2024-03-09T09:26:38.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.878+0000 7f0c9a200640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c941635b0 0x7f0c94071b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009eb0 tx=0x7f0c8c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:38.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.879+0000 7f0c91600640 1 -- 172.21.15.67:0/3222642856 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c042020 con 0x7f0c941635b0 2024-03-09T09:26:38.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.879+0000 7f0c9c200640 1 -- 172.21.15.67:0/3222642856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c940a99e0 con 0x7f0c941635b0 2024-03-09T09:26:38.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.879+0000 7f0c9c200640 1 -- 172.21.15.67:0/3222642856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c94162c50 con 0x7f0c941635b0 2024-03-09T09:26:38.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.880+0000 7f0c91600640 1 -- 172.21.15.67:0/3222642856 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8c003e10 con 0x7f0c941635b0 2024-03-09T09:26:38.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.880+0000 7f0c91600640 1 -- 172.21.15.67:0/3222642856 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c039040 con 0x7f0c941635b0 2024-03-09T09:26:38.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.884+0000 7f0c9c200640 1 -- 172.21.15.67:0/3222642856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c70005290 con 0x7f0c941635b0 2024-03-09T09:26:38.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.887+0000 7f0c91600640 1 -- 172.21.15.67:0/3222642856 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0c8c045020 con 0x7f0c941635b0 2024-03-09T09:26:38.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.889+0000 7f0c91600640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c7406b5a0 0x7f0c7406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:38.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.889+0000 7f0c91600640 1 -- 172.21.15.67:0/3222642856 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f0c8c0acaa0 con 0x7f0c941635b0 2024-03-09T09:26:38.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.895+0000 7f0c99800640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c7406b5a0 0x7f0c7406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:38.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.900+0000 7f0c99800640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c7406b5a0 0x7f0c7406da60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c7f0 tx=0x7f0c80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:38.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:38.902+0000 7f0c91600640 1 -- 172.21.15.67:0/3222642856 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0c8c04c070 con 0x7f0c941635b0 2024-03-09T09:26:39.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.509+0000 7f0c9c200640 1 -- 172.21.15.67:0/3222642856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0c70005580 con 0x7f0c941635b0 2024-03-09T09:26:39.526 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":264,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":26,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:26:39.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.512+0000 7f0c91600640 1 -- 172.21.15.67:0/3222642856 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f0c8c07b5b0 con 0x7f0c941635b0 2024-03-09T09:26:39.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.523+0000 7f0c9c200640 1 -- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c7406b5a0 msgr2=0x7f0c7406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:39.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.523+0000 7f0c9c200640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c7406b5a0 0x7f0c7406da60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000c7f0 tx=0x7f0c80009040 comp rx=0 tx=0).stop 2024-03-09T09:26:39.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.523+0000 7f0c9c200640 1 -- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c941635b0 msgr2=0x7f0c94071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:39.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.523+0000 7f0c9c200640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c941635b0 0x7f0c94071b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009eb0 tx=0x7f0c8c004a40 comp rx=0 tx=0).stop 2024-03-09T09:26:39.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.523+0000 7f0c9c200640 1 -- 172.21.15.67:0/3222642856 shutdown_connections 2024-03-09T09:26:39.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.523+0000 7f0c9c200640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0c7406b5a0 0x7f0c7406da60 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:39.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.523+0000 7f0c9c200640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c941635b0 0x7f0c94071b70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:39.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.523+0000 7f0c9c200640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c940aac00 0x7f0c94071630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:39.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.523+0000 7f0c9c200640 1 --2- 172.21.15.67:0/3222642856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c94073200 0x7f0c94204ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:39.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.523+0000 7f0c9c200640 1 -- 172.21.15.67:0/3222642856 >> 172.21.15.67:0/3222642856 conn(0x7f0c940c7370 msgr2=0x7f0c940aafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:39.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.525+0000 7f0c9c200640 1 -- 172.21.15.67:0/3222642856 shutdown_connections 2024-03-09T09:26:39.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:39.525+0000 7f0c9c200640 1 -- 172.21.15.67:0/3222642856 wait complete. 2024-03-09T09:26:39.788 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:39.790 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:39.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:39.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:39.911 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:40.137 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:40.137 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 5 2024-03-09T09:26:40.137 INFO:tasks.workunit.client.0.smithi067.stdout:5 2024-03-09T09:26:40.137 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:40.285 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:41.076 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:41.080 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:41.642 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:41.642 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:41.758 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:41.758 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:41.759 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:41.798 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:41.798 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 6 2024-03-09T09:26:41.799 INFO:tasks.workunit.client.0.smithi067.stdout:6 2024-03-09T09:26:41.799 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:41.890 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:42.210 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-hardlink.sh... 2024-03-09T09:26:42.210 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-hardlink.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-hardlink.sh 2024-03-09T09:26:42.277 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T09:26:42.277 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir 1 2 2024-03-09T09:26:42.284 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo asdf 2024-03-09T09:26:42.287 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo asdf 2024-03-09T09:26:42.287 INFO:tasks.workunit.client.1.smithi134.stderr:+ ln 1/file1 2/file1 2024-03-09T09:26:42.294 INFO:tasks.workunit.client.1.smithi134.stderr:+ ln 1/file2 2/file2 2024-03-09T09:26:42.316 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:42.318 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir 2/.snap/s1 2024-03-09T09:26:42.322 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:42.422 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo qwer 2024-03-09T09:26:42.432 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep asdf 2/.snap/s1/file1 2024-03-09T09:26:42.443 INFO:tasks.workunit.client.1.smithi134.stdout:asdf 2024-03-09T09:26:42.444 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm -f 1/file2 2024-03-09T09:26:42.449 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep asdf 2/.snap/s1/file2 2024-03-09T09:26:42.471 INFO:tasks.workunit.client.1.smithi134.stdout:asdf 2024-03-09T09:26:42.472 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm -f 2/file2 2024-03-09T09:26:42.480 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep asdf 2/.snap/s1/file2 2024-03-09T09:26:42.484 INFO:tasks.workunit.client.1.smithi134.stdout:asdf 2024-03-09T09:26:42.484 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir 2/.snap/s1 2024-03-09T09:26:42.509 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm -rf 1 2 2024-03-09T09:26:42.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:26:42.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.803+0000 7f890a200640 1 Processor -- start 2024-03-09T09:26:42.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.804+0000 7f890a200640 1 -- start start 2024-03-09T09:26:42.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.806+0000 7f890a200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f89040c7e90 0x7f8904152810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:42.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.806+0000 7f890a200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890414f130 0x7f890414f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:42.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.806+0000 7f890a200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8904152d50 con 0x7f890406d600 2024-03-09T09:26:42.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.806+0000 7f890a200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89040c1ef0 con 0x7f89040c7e90 2024-03-09T09:26:42.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.806+0000 7f890a200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89040c21d0 con 0x7f890414f130 2024-03-09T09:26:42.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.807+0000 7f8902a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f890406d600 0x7f89040c3590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59852/0 (socket says 172.21.15.67:59852) 2024-03-09T09:26:42.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.807+0000 7f8902a00640 1 -- 172.21.15.67:0/3084384951 learned_addr learned my addr 172.21.15.67:0/3084384951 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:26:42.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.807+0000 7f8903400640 1 --2- 172.21.15.67:0/3084384951 >> v2:172.21.15.134:3300/0 conn(0x7f89040c7e90 0x7f8904152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:42.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.807+0000 7f8903e00640 1 --2- 172.21.15.67:0/3084384951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890414f130 0x7f890414f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:42.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.807+0000 7f8903e00640 1 -- 172.21.15.67:0/3084384951 >> v2:172.21.15.134:3300/0 conn(0x7f89040c7e90 msgr2=0x7f8904152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:26:42.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.807+0000 7f8903e00640 1 --2- 172.21.15.67:0/3084384951 >> v2:172.21.15.134:3300/0 conn(0x7f89040c7e90 0x7f8904152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:42.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.807+0000 7f8903e00640 1 -- 172.21.15.67:0/3084384951 >> v1:172.21.15.67:6789/0 conn(0x7f890406d600 legacy=0x7f89040c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:42.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.807+0000 7f8903e00640 1 -- 172.21.15.67:0/3084384951 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89040c24b0 con 0x7f890414f130 2024-03-09T09:26:42.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.808+0000 7f8903e00640 1 --2- 172.21.15.67:0/3084384951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890414f130 0x7f890414f5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009d90 tx=0x7f88fc02bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552731deabdae568 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:42.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.809+0000 7f8902000640 1 -- 172.21.15.67:0/3084384951 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc039070 con 0x7f890414f130 2024-03-09T09:26:42.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.809+0000 7f8902000640 1 -- 172.21.15.67:0/3084384951 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88fc02cb40 con 0x7f890414f130 2024-03-09T09:26:42.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.809+0000 7f8902000640 1 -- 172.21.15.67:0/3084384951 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc034440 con 0x7f890414f130 2024-03-09T09:26:42.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.810+0000 7f890a200640 1 -- 172.21.15.67:0/3084384951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890414f130 msgr2=0x7f890414f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:42.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.810+0000 7f890a200640 1 --2- 172.21.15.67:0/3084384951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890414f130 0x7f890414f5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009d90 tx=0x7f88fc02bd90 comp rx=0 tx=0).stop 2024-03-09T09:26:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.810+0000 7f890a200640 1 -- 172.21.15.67:0/3084384951 shutdown_connections 2024-03-09T09:26:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.810+0000 7f890a200640 1 --2- 172.21.15.67:0/3084384951 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890414f130 0x7f890414f5a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.810+0000 7f890a200640 1 --2- 172.21.15.67:0/3084384951 >> v2:172.21.15.134:3300/0 conn(0x7f89040c7e90 0x7f8904152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.810+0000 7f890a200640 1 -- 172.21.15.67:0/3084384951 >> 172.21.15.67:0/3084384951 conn(0x7f89040c7370 msgr2=0x7f89040c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.811+0000 7f890a200640 1 -- 172.21.15.67:0/3084384951 shutdown_connections 2024-03-09T09:26:42.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.811+0000 7f890a200640 1 -- 172.21.15.67:0/3084384951 wait complete. 2024-03-09T09:26:42.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.813+0000 7f890a200640 1 Processor -- start 2024-03-09T09:26:42.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.814+0000 7f890a200640 1 -- start start 2024-03-09T09:26:42.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.815+0000 7f890a200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890406d600 0x7f890413f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:42.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.815+0000 7f890a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89040c7e90 0x7f89040c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:42.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.815+0000 7f890a200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f890414f130 0x7f8904140c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:42.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.815+0000 7f890a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89040c2f90 con 0x7f89040c7e90 2024-03-09T09:26:42.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.815+0000 7f890a200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89040c2a70 con 0x7f890414f130 2024-03-09T09:26:42.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.815+0000 7f890a200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89040c3110 con 0x7f890406d600 2024-03-09T09:26:42.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8903400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890406d600 0x7f890413f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:42.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8903400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890406d600 0x7f890413f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:55224/0 (socket says 172.21.15.67:55224) 2024-03-09T09:26:42.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8903400640 1 -- 172.21.15.67:0/2877170559 learned_addr learned my addr 172.21.15.67:0/2877170559 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:42.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8902a00640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89040c7e90 0x7f89040c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:42.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8903e00640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f890414f130 0x7f8904140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8902a00640 1 -- 172.21.15.67:0/2877170559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89040c7e90 msgr2=0x7f89040c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8902a00640 1 -- 172.21.15.67:0/2877170559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89040c7e90 msgr2=0x7f89040c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8902a00640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89040c7e90 0x7f89040c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8903400640 1 -- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f890414f130 msgr2=0x7f8904140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8902a00640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89040c7e90 0x7f89040c6b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8903400640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f890414f130 0x7f8904140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8903400640 1 -- 172.21.15.67:0/2877170559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89040c7e90 msgr2=0x7f89040c6b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8903400640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89040c7e90 0x7f89040c6b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.816+0000 7f8903400640 1 -- 172.21.15.67:0/2877170559 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8904150cc0 con 0x7f890406d600 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.817+0000 7f8903400640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890406d600 0x7f890413f030 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f88f400e8a0 tx=0x7f88f400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.817+0000 7f88fb400640 1 -- 172.21.15.67:0/2877170559 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4017070 con 0x7f890406d600 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.817+0000 7f88fb400640 1 -- 172.21.15.67:0/2877170559 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f400c940 con 0x7f890406d600 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.817+0000 7f88fb400640 1 -- 172.21.15.67:0/2877170559 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f400cd30 con 0x7f890406d600 2024-03-09T09:26:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.817+0000 7f890a200640 1 -- 172.21.15.67:0/2877170559 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8904150f60 con 0x7f890406d600 2024-03-09T09:26:42.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.817+0000 7f890a200640 1 -- 172.21.15.67:0/2877170559 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89040c7050 con 0x7f890406d600 2024-03-09T09:26:42.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.818+0000 7f890a200640 1 -- 172.21.15.67:0/2877170559 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8904075a00 con 0x7f890406d600 2024-03-09T09:26:42.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.821+0000 7f88fb400640 1 -- 172.21.15.67:0/2877170559 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f88f401a070 con 0x7f890406d600 2024-03-09T09:26:42.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.822+0000 7f88fb400640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f88e406b4d0 0x7f88e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:42.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.822+0000 7f88fb400640 1 -- 172.21.15.67:0/2877170559 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f88f408a790 con 0x7f890406d600 2024-03-09T09:26:42.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.822+0000 7f8902a00640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f88e406b4d0 0x7f88e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:42.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.823+0000 7f8902a00640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f88e406b4d0 0x7f88e406d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f89041563c0 tx=0x7f88f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:42.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:42.827+0000 7f88fb400640 1 -- 172.21.15.67:0/2877170559 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f88f4058bd0 con 0x7f890406d600 2024-03-09T09:26:42.951 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:42.952 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:43.023 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T09:26:43.023 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T09:26:43.024 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:43.025 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T09:26:43.072 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:43.073 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:43.073 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:43.110 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:43.110 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 7 2024-03-09T09:26:43.110 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:43.111 INFO:tasks.workunit.client.0.smithi067.stdout:7 2024-03-09T09:26:43.127 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-intodir.sh... 2024-03-09T09:26:43.127 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-intodir.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-intodir.sh 2024-03-09T09:26:43.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.121+0000 7f890a200640 1 -- 172.21.15.67:0/2877170559 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f890414f5e0 con 0x7f890406d600 2024-03-09T09:26:43.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.122+0000 7f88fb400640 1 -- 172.21.15.67:0/2877170559 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f88f405cc70 con 0x7f890406d600 2024-03-09T09:26:43.128 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":264,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":29,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:26:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 -- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f88e406b4d0 msgr2=0x7f88e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f88e406b4d0 0x7f88e406d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f89041563c0 tx=0x7f88f0009040 comp rx=0 tx=0).stop 2024-03-09T09:26:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 -- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890406d600 msgr2=0x7f890413f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890406d600 0x7f890413f030 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f88f400e8a0 tx=0x7f88f400ed60 comp rx=0 tx=0).stop 2024-03-09T09:26:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 -- 172.21.15.67:0/2877170559 shutdown_connections 2024-03-09T09:26:43.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f88e406b4d0 0x7f88e406d990 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f890414f130 0x7f8904140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89040c7e90 0x7f89040c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 --2- 172.21.15.67:0/2877170559 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f890406d600 0x7f890413f030 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 -- 172.21.15.67:0/2877170559 >> 172.21.15.67:0/2877170559 conn(0x7f89040c7370 msgr2=0x7f8904148ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 -- 172.21.15.67:0/2877170559 shutdown_connections 2024-03-09T09:26:43.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:43.126+0000 7f890a200640 1 -- 172.21.15.67:0/2877170559 wait complete. 2024-03-09T09:26:43.193 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir foo 2024-03-09T09:26:43.214 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir foo/.snap/one 2024-03-09T09:26:43.223 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:43.249 INFO:tasks.workunit.client.1.smithi134.stderr:+ touch bar 2024-03-09T09:26:43.254 INFO:tasks.workunit.client.1.smithi134.stderr:+ mv bar foo 2024-03-09T09:26:43.262 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:43.262 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T09:26:43.267 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:43.746 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir baz 2024-03-09T09:26:43.755 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir baz/.snap/two 2024-03-09T09:26:43.850 INFO:tasks.workunit.client.1.smithi134.stderr:+ mv baz foo 2024-03-09T09:26:43.862 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T09:26:43.968 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:43.969 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:44.038 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir foo/baz/.snap/two 2024-03-09T09:26:44.047 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir foo/.snap/one 2024-03-09T09:26:44.054 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm -r foo 2024-03-09T09:26:44.071 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T09:26:44.072 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T09:26:44.072 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:44.072 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T09:26:44.103 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:44.104 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:44.104 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:44.128 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:44.129 INFO:tasks.workunit.client.0.smithi067.stdout:8 2024-03-09T09:26:44.130 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 8 2024-03-09T09:26:44.130 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:44.149 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-multiple-capsnaps.sh... 2024-03-09T09:26:44.149 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-multiple-capsnaps.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-multiple-capsnaps.sh 2024-03-09T09:26:44.205 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T09:26:44.205 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo asdf 2024-03-09T09:26:44.207 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/1 2024-03-09T09:26:44.227 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:44.250 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 777 a 2024-03-09T09:26:44.252 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/2 2024-03-09T09:26:44.281 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:44.289 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo qwer 2024-03-09T09:26:44.300 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:44.303 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/3 2024-03-09T09:26:44.340 INFO:tasks.workunit.client.1.smithi134.stderr:+ chmod 666 a 2024-03-09T09:26:44.344 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/4 2024-03-09T09:26:44.635 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo zxcv 2024-03-09T09:26:44.656 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/5 2024-03-09T09:26:44.720 INFO:tasks.workunit.client.1.smithi134.stderr:+ ls -al .snap/1/a .snap/2/a .snap/3/a .snap/4/a .snap/5/a 2024-03-09T09:26:44.728 INFO:tasks.workunit.client.1.smithi134.stdout:-rw-r--r--. 1 ubuntu ubuntu 5 Mar 9 09:26 .snap/1/a 2024-03-09T09:26:44.728 INFO:tasks.workunit.client.1.smithi134.stdout:-rwxrwxrwx. 1 ubuntu ubuntu 5 Mar 9 09:26 .snap/2/a 2024-03-09T09:26:44.728 INFO:tasks.workunit.client.1.smithi134.stdout:-rwxrwxrwx. 1 ubuntu ubuntu 5 Mar 9 09:26 .snap/3/a 2024-03-09T09:26:44.728 INFO:tasks.workunit.client.1.smithi134.stdout:-rw-rw-rw-. 1 ubuntu ubuntu 5 Mar 9 09:26 .snap/4/a 2024-03-09T09:26:44.728 INFO:tasks.workunit.client.1.smithi134.stdout:-rw-rw-rw-. 1 ubuntu ubuntu 5 Mar 9 09:26 .snap/5/a 2024-03-09T09:26:44.728 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep asdf .snap/1/a 2024-03-09T09:26:44.736 INFO:tasks.workunit.client.1.smithi134.stdout:asdf 2024-03-09T09:26:44.736 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/1/a 2024-03-09T09:26:44.737 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'Size: 5' 2024-03-09T09:26:44.741 INFO:tasks.workunit.client.1.smithi134.stdout: Size: 5 Blocks: 1 IO Block: 4194304 regular file 2024-03-09T09:26:44.741 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep asdf .snap/2/a 2024-03-09T09:26:44.745 INFO:tasks.workunit.client.1.smithi134.stdout:asdf 2024-03-09T09:26:44.747 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/2/a 2024-03-09T09:26:44.748 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'Size: 5' 2024-03-09T09:26:44.751 INFO:tasks.workunit.client.1.smithi134.stdout: Size: 5 Blocks: 1 IO Block: 4194304 regular file 2024-03-09T09:26:44.752 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/2/a 2024-03-09T09:26:44.753 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep -- -rwxrwxrwx 2024-03-09T09:26:44.756 INFO:tasks.workunit.client.1.smithi134.stdout:Access: (0777/-rwxrwxrwx) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-09T09:26:44.756 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep qwer .snap/3/a 2024-03-09T09:26:44.761 INFO:tasks.workunit.client.1.smithi134.stdout:qwer 2024-03-09T09:26:44.762 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/3/a 2024-03-09T09:26:44.767 INFO:tasks.workunit.client.1.smithi134.stdout: Size: 5 Blocks: 1 IO Block: 4194304 regular file 2024-03-09T09:26:44.768 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'Size: 5' 2024-03-09T09:26:44.769 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/3/a 2024-03-09T09:26:44.771 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep -- -rwxrwxrwx 2024-03-09T09:26:44.774 INFO:tasks.workunit.client.1.smithi134.stdout:Access: (0777/-rwxrwxrwx) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-09T09:26:44.774 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep qwer .snap/4/a 2024-03-09T09:26:44.783 INFO:tasks.workunit.client.1.smithi134.stdout:qwer 2024-03-09T09:26:44.785 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/4/a 2024-03-09T09:26:44.786 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'Size: 5' 2024-03-09T09:26:44.789 INFO:tasks.workunit.client.1.smithi134.stdout: Size: 5 Blocks: 1 IO Block: 4194304 regular file 2024-03-09T09:26:44.792 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep -- -rw-rw-rw- 2024-03-09T09:26:44.792 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/4/a 2024-03-09T09:26:44.796 INFO:tasks.workunit.client.1.smithi134.stdout:Access: (0666/-rw-rw-rw-) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-09T09:26:44.797 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep zxcv .snap/5/a 2024-03-09T09:26:44.800 INFO:tasks.workunit.client.1.smithi134.stdout:zxcv 2024-03-09T09:26:44.801 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/5/a 2024-03-09T09:26:44.802 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep 'Size: 5' 2024-03-09T09:26:44.804 INFO:tasks.workunit.client.1.smithi134.stdout: Size: 5 Blocks: 1 IO Block: 4194304 regular file 2024-03-09T09:26:44.805 INFO:tasks.workunit.client.1.smithi134.stderr:+ stat .snap/5/a 2024-03-09T09:26:44.806 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep -- -rw-rw-rw- 2024-03-09T09:26:44.809 INFO:tasks.workunit.client.1.smithi134.stdout:Access: (0666/-rw-rw-rw-) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-09T09:26:44.810 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/1 .snap/2 .snap/3 .snap/4 .snap/5 2024-03-09T09:26:44.855 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T09:26:44.856 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T09:26:44.856 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:44.857 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T09:26:44.895 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:44.895 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:44.955 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-name-limits.sh... 2024-03-09T09:26:44.955 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-name-limits.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-name-limits.sh 2024-03-09T09:26:45.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:45.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:45.011 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:45.034 INFO:tasks.workunit.client.1.smithi134.stderr:rmdir: failed to remove 'd1/.snap/*': No such file or directory 2024-03-09T09:26:45.039 INFO:tasks.workunit.client.0.smithi067.stdout:9 2024-03-09T09:26:45.040 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:45.040 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 9 2024-03-09T09:26:45.040 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:45.040 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T09:26:45.041 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:45.041 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T09:26:45.158 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:45.159 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-parents.sh... 2024-03-09T09:26:45.160 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-parents.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-parents.sh 2024-03-09T09:26:45.196 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:45.222 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'making directory tree and files' 2024-03-09T09:26:45.222 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir -p 1/a/b/c/ 2024-03-09T09:26:45.222 INFO:tasks.workunit.client.1.smithi134.stdout:making directory tree and files 2024-03-09T09:26:45.231 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'i'\''m file1' 2024-03-09T09:26:45.233 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'i'\''m file2' 2024-03-09T09:26:45.235 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'i'\''m file3' 2024-03-09T09:26:45.238 INFO:tasks.workunit.client.1.smithi134.stdout:snapshotting 2024-03-09T09:26:45.238 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo snapshotting 2024-03-09T09:26:45.238 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir 1/.snap/foosnap1 2024-03-09T09:26:45.288 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir 2 2024-03-09T09:26:45.293 INFO:tasks.workunit.client.1.smithi134.stdout:moving tree 2024-03-09T09:26:45.294 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'moving tree' 2024-03-09T09:26:45.294 INFO:tasks.workunit.client.1.smithi134.stderr:+ mv 1/a 2 2024-03-09T09:26:45.302 INFO:tasks.workunit.client.1.smithi134.stdout:checking snapshot contains tree... 2024-03-09T09:26:45.303 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'checking snapshot contains tree...' 2024-03-09T09:26:45.304 INFO:tasks.workunit.client.1.smithi134.stderr:++ find 1/.snap/foosnap1 2024-03-09T09:26:45.304 INFO:tasks.workunit.client.1.smithi134.stderr:++ wc -w 2024-03-09T09:26:45.341 INFO:tasks.workunit.client.1.smithi134.stderr:+ dir1=7 2024-03-09T09:26:45.343 INFO:tasks.workunit.client.1.smithi134.stderr:++ find 2/ 2024-03-09T09:26:45.344 INFO:tasks.workunit.client.1.smithi134.stderr:++ wc -w 2024-03-09T09:26:45.355 INFO:tasks.workunit.client.1.smithi134.stderr:+ dir2=7 2024-03-09T09:26:45.355 INFO:tasks.workunit.client.1.smithi134.stderr:+ test 7==7 2024-03-09T09:26:45.355 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'Success!' 2024-03-09T09:26:45.355 INFO:tasks.workunit.client.1.smithi134.stdout:Success! 2024-03-09T09:26:45.355 INFO:tasks.workunit.client.1.smithi134.stdout:adding folder and file to tree... 2024-03-09T09:26:45.355 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'adding folder and file to tree...' 2024-03-09T09:26:45.356 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir 2/a/b/c/d 2024-03-09T09:26:45.361 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'i'\''m file 4!' 2024-03-09T09:26:45.365 INFO:tasks.workunit.client.1.smithi134.stdout:snapshotting tree 2 2024-03-09T09:26:45.365 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'snapshotting tree 2' 2024-03-09T09:26:45.366 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir 2/.snap/barsnap2 2024-03-09T09:26:45.410 INFO:tasks.workunit.client.1.smithi134.stdout:comparing snapshots 2024-03-09T09:26:45.411 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'comparing snapshots' 2024-03-09T09:26:45.414 INFO:tasks.workunit.client.1.smithi134.stderr:++ find 1/.snap/foosnap1/ -maxdepth 2 2024-03-09T09:26:45.414 INFO:tasks.workunit.client.1.smithi134.stderr:++ wc -w 2024-03-09T09:26:45.427 INFO:tasks.workunit.client.1.smithi134.stderr:+ dir1=4 2024-03-09T09:26:45.428 INFO:tasks.workunit.client.1.smithi134.stderr:++ find 2/.snap/barsnap2/ -maxdepth 2 2024-03-09T09:26:45.428 INFO:tasks.workunit.client.1.smithi134.stderr:++ wc -w 2024-03-09T09:26:45.453 INFO:tasks.workunit.client.1.smithi134.stderr:+ dir2=4 2024-03-09T09:26:45.454 INFO:tasks.workunit.client.1.smithi134.stderr:+ test 4==4 2024-03-09T09:26:45.454 INFO:tasks.workunit.client.1.smithi134.stdout:Success! 2024-03-09T09:26:45.454 INFO:tasks.workunit.client.1.smithi134.stdout:moving subtree to first folder 2024-03-09T09:26:45.455 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'Success!' 2024-03-09T09:26:45.455 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'moving subtree to first folder' 2024-03-09T09:26:45.455 INFO:tasks.workunit.client.1.smithi134.stderr:+ mv 2/a/b/c 1 2024-03-09T09:26:45.464 INFO:tasks.workunit.client.1.smithi134.stdout:comparing snapshots and new tree 2024-03-09T09:26:45.465 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'comparing snapshots and new tree' 2024-03-09T09:26:45.466 INFO:tasks.workunit.client.1.smithi134.stderr:++ find 1/ 2024-03-09T09:26:45.467 INFO:tasks.workunit.client.1.smithi134.stderr:++ wc -w 2024-03-09T09:26:45.486 INFO:tasks.workunit.client.1.smithi134.stderr:+ dir1=5 2024-03-09T09:26:45.487 INFO:tasks.workunit.client.1.smithi134.stderr:++ find 2/.snap/barsnap2/a/b/c 2024-03-09T09:26:45.488 INFO:tasks.workunit.client.1.smithi134.stderr:++ wc -w 2024-03-09T09:26:45.505 INFO:tasks.workunit.client.1.smithi134.stderr:+ dir2=4 2024-03-09T09:26:45.506 INFO:tasks.workunit.client.1.smithi134.stderr:+ test 5==4 2024-03-09T09:26:45.506 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'Success!' 2024-03-09T09:26:45.506 INFO:tasks.workunit.client.1.smithi134.stdout:Success! 2024-03-09T09:26:45.507 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir 1/.snap/foosnap1 2024-03-09T09:26:45.524 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir 2/.snap/barsnap2 2024-03-09T09:26:45.539 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T09:26:45.540 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T09:26:45.541 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:45.541 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T09:26:45.827 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-realm-split.sh... 2024-03-09T09:26:45.827 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-realm-split.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-realm-split.sh 2024-03-09T09:26:45.865 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:45.904 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T09:26:45.904 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir -p 1/a 2024-03-09T09:26:46.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:26:46.176 INFO:tasks.workunit.client.1.smithi134.stderr:+ exec 2024-03-09T09:26:46.177 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo -n a 2024-03-09T09:26:46.177 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir 1/.snap/s1 2024-03-09T09:26:46.206 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo -n b 2024-03-09T09:26:46.211 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir 2 2024-03-09T09:26:46.216 INFO:tasks.workunit.client.1.smithi134.stderr:+ mv 1/a 2 2024-03-09T09:26:46.222 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir 2/.snap/s2 2024-03-09T09:26:46.248 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo -n c 2024-03-09T09:26:46.248 INFO:tasks.workunit.client.1.smithi134.stderr:+ exec 2024-03-09T09:26:46.248 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep '^a$' 1/.snap/s1/a/file1 2024-03-09T09:26:46.257 INFO:tasks.workunit.client.1.smithi134.stdout:a 2024-03-09T09:26:46.258 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep '^ab$' 2/.snap/s2/a/file1 2024-03-09T09:26:46.285 INFO:tasks.workunit.client.1.smithi134.stdout:ab 2024-03-09T09:26:46.285 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep '^abc$' 2/a/file1 2024-03-09T09:26:46.290 INFO:tasks.workunit.client.1.smithi134.stdout:abc 2024-03-09T09:26:46.290 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir 1/.snap/s1 2024-03-09T09:26:46.304 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir 2/.snap/s2 2024-03-09T09:26:46.313 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm -rf 1 2 2024-03-09T09:26:46.348 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T09:26:46.349 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T09:26:46.350 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:46.350 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T09:26:46.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.379+0000 7f5dd9a00640 1 Processor -- start 2024-03-09T09:26:46.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.379+0000 7f5dd9a00640 1 -- start start 2024-03-09T09:26:46.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.381+0000 7f5dd9a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd40ac730 0x7f5dd4071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:46.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.381+0000 7f5dd9a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5dd4157d60 0x7f5dd40acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:46.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.381+0000 7f5dd9a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd4153d40 con 0x7f5dd40c22a0 2024-03-09T09:26:46.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.381+0000 7f5dd9a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd4070ee0 con 0x7f5dd4157d60 2024-03-09T09:26:46.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.381+0000 7f5dd9a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd4154d00 con 0x7f5dd40ac730 2024-03-09T09:26:46.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.381+0000 7f5dd2000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5dd40c22a0 0x7f5dd4157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:59866/0 (socket says 172.21.15.67:59866) 2024-03-09T09:26:46.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.381+0000 7f5dd2000640 1 -- 172.21.15.67:0/2596092413 learned_addr learned my addr 172.21.15.67:0/2596092413 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:26:46.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.381+0000 7f5dd3400640 1 --2- 172.21.15.67:0/2596092413 >> v2:172.21.15.67:3300/0 conn(0x7f5dd4157d60 0x7f5dd40acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:46.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.382+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/2596092413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd40ac730 0x7f5dd4071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:46.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.382+0000 7f5dd3400640 1 -- 172.21.15.67:0/2596092413 >> v2:172.21.15.67:3300/0 conn(0x7f5dd4157d60 msgr2=0x7f5dd40acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:26:46.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.382+0000 7f5dd3400640 1 -- 172.21.15.67:0/2596092413 >> v2:172.21.15.67:3300/0 conn(0x7f5dd4157d60 msgr2=0x7f5dd40acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:46.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.382+0000 7f5dd3400640 1 --2- 172.21.15.67:0/2596092413 >> v2:172.21.15.67:3300/0 conn(0x7f5dd4157d60 0x7f5dd40acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:46.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.382+0000 7f5dd3400640 1 --2- 172.21.15.67:0/2596092413 >> v2:172.21.15.67:3300/0 conn(0x7f5dd4157d60 0x7f5dd40acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:46.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.383+0000 7f5dd2a00640 1 -- 172.21.15.67:0/2596092413 >> v2:172.21.15.67:3300/0 conn(0x7f5dd4157d60 msgr2=0x7f5dd40acb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:46.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.383+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/2596092413 >> v2:172.21.15.67:3300/0 conn(0x7f5dd4157d60 0x7f5dd40acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:46.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.383+0000 7f5dd2a00640 1 -- 172.21.15.67:0/2596092413 >> v1:172.21.15.67:6789/0 conn(0x7f5dd40c22a0 legacy=0x7f5dd4157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:46.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.383+0000 7f5dd2a00640 1 -- 172.21.15.67:0/2596092413 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40c2790 con 0x7f5dd40ac730 2024-03-09T09:26:46.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.385+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/2596092413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd40ac730 0x7f5dd4071190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40097a0 tx=0x7f5dc402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aaa596460feaebe server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:46.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.385+0000 7f5dd1600640 1 -- 172.21.15.67:0/2596092413 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40315a0 con 0x7f5dd40ac730 2024-03-09T09:26:46.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.385+0000 7f5dd1600640 1 -- 172.21.15.67:0/2596092413 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc4037070 con 0x7f5dd40ac730 2024-03-09T09:26:46.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.386+0000 7f5dd1600640 1 -- 172.21.15.67:0/2596092413 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4031960 con 0x7f5dd40ac730 2024-03-09T09:26:46.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.386+0000 7f5dd9a00640 1 -- 172.21.15.67:0/2596092413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd40ac730 msgr2=0x7f5dd4071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:46.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.386+0000 7f5dd9a00640 1 --2- 172.21.15.67:0/2596092413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd40ac730 0x7f5dd4071190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40097a0 tx=0x7f5dc402b840 comp rx=0 tx=0).stop 2024-03-09T09:26:46.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.386+0000 7f5dd9a00640 1 -- 172.21.15.67:0/2596092413 shutdown_connections 2024-03-09T09:26:46.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.386+0000 7f5dd9a00640 1 --2- 172.21.15.67:0/2596092413 >> v2:172.21.15.67:3300/0 conn(0x7f5dd4157d60 0x7f5dd40acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:46.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.386+0000 7f5dd9a00640 1 --2- 172.21.15.67:0/2596092413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd40ac730 0x7f5dd4071190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:46.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.386+0000 7f5dd9a00640 1 -- 172.21.15.67:0/2596092413 >> 172.21.15.67:0/2596092413 conn(0x7f5dd40c7370 msgr2=0x7f5dd40c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:46.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.387+0000 7f5dd9a00640 1 -- 172.21.15.67:0/2596092413 shutdown_connections 2024-03-09T09:26:46.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.387+0000 7f5dd9a00640 1 -- 172.21.15.67:0/2596092413 wait complete. 2024-03-09T09:26:46.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.389+0000 7f5dd9a00640 1 Processor -- start 2024-03-09T09:26:46.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.390+0000 7f5dd9a00640 1 -- start start 2024-03-09T09:26:46.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.391+0000 7f5dd9a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd40ac730 0x7f5dd40c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:46.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.391+0000 7f5dd9a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40c22a0 0x7f5dd41dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:46.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.391+0000 7f5dd9a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd4157d60 0x7f5dd41eec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:46.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.391+0000 7f5dd9a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd4151610 con 0x7f5dd40c22a0 2024-03-09T09:26:46.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.391+0000 7f5dd9a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd4070c00 con 0x7f5dd40ac730 2024-03-09T09:26:46.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.391+0000 7f5dd9a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd4151790 con 0x7f5dd4157d60 2024-03-09T09:26:46.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.392+0000 7f5dd2a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd40ac730 0x7f5dd40c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:46.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.392+0000 7f5dd2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40c22a0 0x7f5dd41dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:46.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.392+0000 7f5dd3400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd4157d60 0x7f5dd41eec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:46.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.392+0000 7f5dd3400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd4157d60 0x7f5dd41eec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:55238/0 (socket says 172.21.15.67:55238) 2024-03-09T09:26:46.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.392+0000 7f5dd3400640 1 -- 172.21.15.67:0/1421066431 learned_addr learned my addr 172.21.15.67:0/1421066431 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.392+0000 7f5dd2000640 1 -- 172.21.15.67:0/1421066431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40c22a0 msgr2=0x7f5dd41dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:26:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.392+0000 7f5dd2000640 1 -- 172.21.15.67:0/1421066431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40c22a0 msgr2=0x7f5dd41dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.392+0000 7f5dd2000640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40c22a0 0x7f5dd41dd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.392+0000 7f5dd2000640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40c22a0 0x7f5dd41dd2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:46.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.393+0000 7f5dd2a00640 1 -- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd4157d60 msgr2=0x7f5dd41eec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.393+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd4157d60 0x7f5dd41eec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.393+0000 7f5dd2a00640 1 -- 172.21.15.67:0/1421066431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40c22a0 msgr2=0x7f5dd41dd2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:26:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.393+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40c22a0 0x7f5dd41dd2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.393+0000 7f5dd2a00640 1 -- 172.21.15.67:0/1421066431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd4156cc0 con 0x7f5dd40ac730 2024-03-09T09:26:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.393+0000 7f5dd3400640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd4157d60 0x7f5dd41eec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:26:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.393+0000 7f5dd2a00640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd40ac730 0x7f5dd40c0a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4033000 tx=0x7f5dc40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.394+0000 7f5dcaa00640 1 -- 172.21.15.67:0/1421066431 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4032840 con 0x7f5dd40ac730 2024-03-09T09:26:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.394+0000 7f5dcaa00640 1 -- 172.21.15.67:0/1421066431 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc40329e0 con 0x7f5dd40ac730 2024-03-09T09:26:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.394+0000 7f5dcaa00640 1 -- 172.21.15.67:0/1421066431 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4032dd0 con 0x7f5dd40ac730 2024-03-09T09:26:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.394+0000 7f5dd9a00640 1 -- 172.21.15.67:0/1421066431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd4156fa0 con 0x7f5dd40ac730 2024-03-09T09:26:46.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.394+0000 7f5dd9a00640 1 -- 172.21.15.67:0/1421066431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd41ebbf0 con 0x7f5dd40ac730 2024-03-09T09:26:46.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.395+0000 7f5dd9a00640 1 -- 172.21.15.67:0/1421066431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd41eb720 con 0x7f5dd40ac730 2024-03-09T09:26:46.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.396+0000 7f5dcaa00640 1 -- 172.21.15.67:0/1421066431 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5dc4005990 con 0x7f5dd40ac730 2024-03-09T09:26:46.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.397+0000 7f5dcaa00640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5db406b4d0 0x7f5db406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:46.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.397+0000 7f5dcaa00640 1 -- 172.21.15.67:0/1421066431 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f5dc402f030 con 0x7f5dd40ac730 2024-03-09T09:26:46.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.398+0000 7f5dd2000640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5db406b4d0 0x7f5db406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:46.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.399+0000 7f5dd2000640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5db406b4d0 0x7f5db406d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40c11e0 tx=0x7f5dc0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:46.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.401+0000 7f5dcaa00640 1 -- 172.21.15.67:0/1421066431 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5dc4040070 con 0x7f5dd40ac730 2024-03-09T09:26:46.450 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-snap-rename.sh... 2024-03-09T09:26:46.451 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-snap-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-snap-rename.sh 2024-03-09T09:26:46.526 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:46.527 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:46.529 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T09:26:46.529 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir -p d1/d2 2024-03-09T09:26:46.534 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir -p d1/d3 2024-03-09T09:26:46.539 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir d1/.snap/foo 2024-03-09T09:26:46.568 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir d1/d2/.snap/foo 2024-03-09T09:26:46.589 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir d1/d3/.snap/foo 2024-03-09T09:26:46.610 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir d1/d3/.snap/bar 2024-03-09T09:26:46.647 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:46.647 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:46.647 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:46.696 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:46.696 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 10 2024-03-09T09:26:46.697 INFO:tasks.workunit.client.0.smithi067.stdout:10 2024-03-09T09:26:46.697 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:46.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.723+0000 7f5dd9a00640 1 -- 172.21.15.67:0/1421066431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5dd4075a00 con 0x7f5dd40ac730 2024-03-09T09:26:46.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.724+0000 7f5dcaa00640 1 -- 172.21.15.67:0/1421066431 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f5dc407b320 con 0x7f5dd40ac730 2024-03-09T09:26:46.731 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":264,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":33,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:26:46.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 -- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5db406b4d0 msgr2=0x7f5db406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5db406b4d0 0x7f5db406d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40c11e0 tx=0x7f5dc0009040 comp rx=0 tx=0).stop 2024-03-09T09:26:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 -- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd40ac730 msgr2=0x7f5dd40c0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd40ac730 0x7f5dd40c0a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4033000 tx=0x7f5dc40046c0 comp rx=0 tx=0).stop 2024-03-09T09:26:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 -- 172.21.15.67:0/1421066431 shutdown_connections 2024-03-09T09:26:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5db406b4d0 0x7f5db406d990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5dd4157d60 0x7f5dd41eec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5dd40c22a0 0x7f5dd41dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 --2- 172.21.15.67:0/1421066431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dd40ac730 0x7f5dd40c0a10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 -- 172.21.15.67:0/1421066431 >> 172.21.15.67:0/1421066431 conn(0x7f5dd40c7370 msgr2=0x7f5dd40c1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 -- 172.21.15.67:0/1421066431 shutdown_connections 2024-03-09T09:26:46.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:46.727+0000 7f5dd9a00640 1 -- 172.21.15.67:0/1421066431 wait complete. 2024-03-09T09:26:46.855 INFO:tasks.workunit.client.1.smithi134.stderr:+ mv d1/d2/.snap/foo d1/d2/.snap/bar 2024-03-09T09:26:46.873 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:46.877 INFO:tasks.workunit.client.1.smithi134.stderr:+ expect_failure mv d1/d2/.snap/bar d1/d2/.snap/_bar 2024-03-09T09:26:46.878 INFO:tasks.workunit.client.1.smithi134.stderr:+ mv d1/d2/.snap/bar d1/d2/.snap/_bar 2024-03-09T09:26:46.884 INFO:tasks.workunit.client.1.smithi134.stderr:mv: cannot move 'd1/d2/.snap/bar' to a subdirectory of itself, 'd1/d2/.snap/_bar' 2024-03-09T09:26:46.884 INFO:tasks.workunit.client.1.smithi134.stderr:+ return 0 2024-03-09T09:26:46.886 INFO:tasks.workunit.client.1.smithi134.stderr:+ expect_failure mv d1/d2/.snap/_foo_1099511669243 d1/d2/.snap/foo 2024-03-09T09:26:46.886 INFO:tasks.workunit.client.1.smithi134.stderr:+ mv d1/d2/.snap/_foo_1099511669243 d1/d2/.snap/foo 2024-03-09T09:26:46.894 INFO:tasks.workunit.client.1.smithi134.stderr:mv: cannot move 'd1/d2/.snap/_foo_1099511669243' to a subdirectory of itself, 'd1/d2/.snap/foo' 2024-03-09T09:26:46.895 INFO:tasks.workunit.client.1.smithi134.stderr:+ return 0 2024-03-09T09:26:46.896 INFO:tasks.workunit.client.1.smithi134.stderr:+ expect_failure mv d1/d2/.snap/_foo_1099511669243 d1/d2/.snap/_foo_1 2024-03-09T09:26:46.896 INFO:tasks.workunit.client.1.smithi134.stderr:+ mv d1/d2/.snap/_foo_1099511669243 d1/d2/.snap/_foo_1 2024-03-09T09:26:46.901 INFO:tasks.workunit.client.1.smithi134.stderr:mv: cannot move 'd1/d2/.snap/_foo_1099511669243' to a subdirectory of itself, 'd1/d2/.snap/_foo_1' 2024-03-09T09:26:46.902 INFO:tasks.workunit.client.1.smithi134.stderr:+ return 0 2024-03-09T09:26:46.902 INFO:tasks.workunit.client.1.smithi134.stderr:+ expect_failure mv d1/d2/.snap/bar d1/.snap/ 2024-03-09T09:26:46.902 INFO:tasks.workunit.client.1.smithi134.stderr:+ mv d1/d2/.snap/bar d1/.snap/ 2024-03-09T09:26:46.910 INFO:tasks.workunit.client.1.smithi134.stderr:mv: cannot move 'd1/d2/.snap/bar' to 'd1/.snap/bar': Read-only file system 2024-03-09T09:26:46.911 INFO:tasks.workunit.client.1.smithi134.stderr:+ return 0 2024-03-09T09:26:46.911 INFO:tasks.workunit.client.1.smithi134.stderr:+ expect_failure python3 -c 'import os; os.rename('\''d1/d3/.snap/foo'\'', '\''d1/d3/.snap/bar'\'')' 2024-03-09T09:26:46.911 INFO:tasks.workunit.client.1.smithi134.stderr:+ python3 -c 'import os; os.rename('\''d1/d3/.snap/foo'\'', '\''d1/d3/.snap/bar'\'')' 2024-03-09T09:26:46.941 INFO:tasks.workunit.client.1.smithi134.stderr:Traceback (most recent call last): 2024-03-09T09:26:46.941 INFO:tasks.workunit.client.1.smithi134.stderr: File "", line 1, in 2024-03-09T09:26:46.942 INFO:tasks.workunit.client.1.smithi134.stderr:FileExistsError: [Errno 17] File exists: 'd1/d3/.snap/foo' -> 'd1/d3/.snap/bar' 2024-03-09T09:26:46.945 INFO:tasks.workunit.client.1.smithi134.stderr:+ return 0 2024-03-09T09:26:46.946 INFO:tasks.workunit.client.1.smithi134.stderr:+ expect_failure python3 -c 'import os; os.rename('\''d1/.snap/foo'\'', '\''d1/foo'\'')' 2024-03-09T09:26:46.946 INFO:tasks.workunit.client.1.smithi134.stderr:+ python3 -c 'import os; os.rename('\''d1/.snap/foo'\'', '\''d1/foo'\'')' 2024-03-09T09:26:46.976 INFO:tasks.workunit.client.1.smithi134.stderr:Traceback (most recent call last): 2024-03-09T09:26:46.976 INFO:tasks.workunit.client.1.smithi134.stderr: File "", line 1, in 2024-03-09T09:26:46.976 INFO:tasks.workunit.client.1.smithi134.stderr:OSError: [Errno 18] Invalid cross-device link: 'd1/.snap/foo' -> 'd1/foo' 2024-03-09T09:26:46.979 INFO:tasks.workunit.client.1.smithi134.stderr:+ return 0 2024-03-09T09:26:46.979 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir d1/.snap/foo 2024-03-09T09:26:46.994 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir d1/d2/.snap/bar 2024-03-09T09:26:47.010 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir d1/d3/.snap/foo 2024-03-09T09:26:47.025 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir d1/d3/.snap/bar 2024-03-09T09:26:47.040 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm -rf d1 2024-03-09T09:26:47.088 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T09:26:47.088 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T09:26:47.090 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:26:47.090 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T09:26:47.198 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-snap-rm-cmp.sh... 2024-03-09T09:26:47.198 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-snap-rm-cmp.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-snap-rm-cmp.sh 2024-03-09T09:26:47.257 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T09:26:47.257 INFO:tasks.workunit.client.1.smithi134.stderr:+ file=linux-2.6.33.tar.bz2 2024-03-09T09:26:47.258 INFO:tasks.workunit.client.1.smithi134.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2024-03-09T09:26:47.686 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:47.695 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:48.301 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:48.302 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:48.412 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:48.412 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:48.412 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:48.454 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:48.455 INFO:tasks.workunit.client.0.smithi067.stdout:11 2024-03-09T09:26:48.455 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 11 2024-03-09T09:26:48.455 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:48.791 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:48.846 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:48.854 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:49.370 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:49.371 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:49.484 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:49.484 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:49.484 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:49.535 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:49.535 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 12 2024-03-09T09:26:49.535 INFO:tasks.workunit.client.0.smithi067.stdout:12 2024-03-09T09:26:49.535 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:49.627 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum linux-2.6.33.tar.bz2 2024-03-09T09:26:49.627 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:26:49.642 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:49.680 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:49.688 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:49.755 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:26:49.762 INFO:tasks.workunit.client.1.smithi134.stderr:+ real=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:49.763 INFO:tasks.workunit.client.1.smithi134.stderr:++ seq 1 20 2024-03-09T09:26:49.765 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:49.765 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 1 2024-03-09T09:26:49.765 INFO:tasks.workunit.client.1.smithi134.stdout:1 2024-03-09T09:26:49.765 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:49.934 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:26:49.980 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:26:49.986 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.033+0000 7f9114a00640 1 Processor -- start 2024-03-09T09:26:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.033+0000 7f9114a00640 1 -- start start 2024-03-09T09:26:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.036+0000 7f9114a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.036+0000 7f9114a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.036+0000 7f9114a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91100c4010 con 0x7f9110153370 2024-03-09T09:26:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.036+0000 7f9114a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9110156dc0 con 0x7f9110153750 2024-03-09T09:26:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.036+0000 7f9114a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91100c42f0 con 0x7f911006dc00 2024-03-09T09:26:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.036+0000 7f910de00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7f9110153370 0x7f91100c29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:43132/0 (socket says 172.21.15.67:43132) 2024-03-09T09:26:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.036+0000 7f910de00640 1 -- 172.21.15.67:0/1592625099 learned_addr learned my addr 172.21.15.67:0/1592625099 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:26:50.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.037+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:50.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.037+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.038+0000 7f910e800640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 msgr2=0x7f91100729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:26:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.038+0000 7f910e800640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 msgr2=0x7f91100729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.038+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:50.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.038+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:50.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.039+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f91100c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:50.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.039+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f91100c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:50.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.039+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:50.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.039+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:50.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.240+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:50.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.240+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:50.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.240+0000 7f910e800640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 msgr2=0x7f91100729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:50.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.240+0000 7f910e800640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 msgr2=0x7f91100729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:50.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.240+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:50.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.240+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:26:50.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.241+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f91100c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:26:50.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.241+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f91100c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:50.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.241+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:50.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.241+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:26:50.439 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:50.439 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:50.499 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:26:50.500 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:26:50.555 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:50.556 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:50.556 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:50.623 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:50.623 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:50.623 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:26:50.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.642+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:50.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.642+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:50.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.643+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f91100c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:50.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.643+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f91100c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:50.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.643+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:50.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.643+0000 7f910e800640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 msgr2=0x7f91100729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:26:50.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.643+0000 7f910e800640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 msgr2=0x7f91100729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:50.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.643+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:26:50.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.643+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:50.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:50.643+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:26:50.654 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:50.654 INFO:tasks.workunit.client.1.smithi134.stdout:2 2024-03-09T09:26:50.655 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 2 2024-03-09T09:26:50.655 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:50.761 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:26:50.792 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:50.793 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 13 2024-03-09T09:26:50.793 INFO:tasks.workunit.client.0.smithi067.stdout:13 2024-03-09T09:26:50.793 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:50.803 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:26:50.819 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:51.352 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:51.385 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:26:51.385 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:26:51.392 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:51.406 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:51.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:51.444+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:51.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:51.444+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:51.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:51.445+0000 7f910e800640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 msgr2=0x7f91100729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:26:51.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:51.445+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f91100c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:51.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:51.445+0000 7f910e800640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 msgr2=0x7f91100729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:51.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:51.445+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f91100c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:51.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:51.445+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:51.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:51.445+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:51.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:51.445+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:26:51.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:51.445+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:26:51.502 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:51.502 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:51.502 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:26:51.607 INFO:tasks.workunit.client.1.smithi134.stdout:3 2024-03-09T09:26:51.608 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:51.608 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 3 2024-03-09T09:26:51.608 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:51.984 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:26:52.012 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:52.016 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:52.032 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:26:52.038 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:52.149 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:52.149 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:52.149 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:52.184 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:52.185 INFO:tasks.workunit.client.0.smithi067.stdout:14 2024-03-09T09:26:52.185 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 14 2024-03-09T09:26:52.185 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:52.308 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:52.645 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:26:52.645 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:26:52.734 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:52.748 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:52.766 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:52.766 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:52.766 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:26:52.813 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:52.813 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 4 2024-03-09T09:26:52.813 INFO:tasks.workunit.client.1.smithi134.stdout:4 2024-03-09T09:26:52.814 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:52.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:52.853+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297719543 0 0) 0x7f91100c4010 con 0x7f9110153370 2024-03-09T09:26:52.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:52.853+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90f4003520 con 0x7f9110153370 2024-03-09T09:26:52.994 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:26:53.039 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:26:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.036+0000 7f9107400640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 msgr2=0x7f91100729c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.036+0000 7f9107400640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006dc00 0x7f91100729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.036+0000 7f9107400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f91100c30b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.036+0000 7f9107400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f91100c30b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:53.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.036+0000 7f9107400640 1 -- 172.21.15.67:0/1592625099 >> v1:172.21.15.134:6789/0 conn(0x7f9110153370 legacy=0x7f91100c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.036+0000 7f9107400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.036+0000 7f9107400640 1 -- 172.21.15.67:0/1592625099 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91100c5a50 con 0x7f90f00036c0 2024-03-09T09:26:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.036+0000 7f9107400640 1 -- 172.21.15.67:0/1592625099 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9110156dc0 con 0x7f90f0007370 2024-03-09T09:26:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.036+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.037+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 msgr2=0x7f90f00097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.037+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 msgr2=0x7f90f00097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.037+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.037+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:53.045 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.238+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.238+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 msgr2=0x7f90f00097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:53.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.238+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 msgr2=0x7f90f00097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:53.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.238+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:53.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.238+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:26:53.429 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:53.430 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:53.544 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:53.544 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:53.545 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:53.582 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:53.583 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 15 2024-03-09T09:26:53.583 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:53.583 INFO:tasks.workunit.client.0.smithi067.stdout:15 2024-03-09T09:26:53.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.639+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:53.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.640+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 msgr2=0x7f90f00097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:53.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.640+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 msgr2=0x7f90f00097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:53.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.640+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:53.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:53.640+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:26:53.691 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:53.732 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:53.738 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:54.058 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:26:54.058 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:26:54.174 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:54.174 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:54.174 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:26:54.220 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:54.220 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 5 2024-03-09T09:26:54.221 INFO:tasks.workunit.client.1.smithi134.stdout:5 2024-03-09T09:26:54.221 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:54.359 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:26:54.400 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:26:54.411 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:54.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:54.441+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:54.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:54.442+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 msgr2=0x7f90f00097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:54.442+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 msgr2=0x7f90f00097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:54.442+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:54.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:54.442+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:26:54.871 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:54.871 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:55.001 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:55.001 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:55.001 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:55.057 INFO:tasks.workunit.client.0.smithi067.stdout:16 2024-03-09T09:26:55.057 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:55.057 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 16 2024-03-09T09:26:55.057 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:55.089 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:26:55.089 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:26:55.163 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:55.209 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:55.220 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:55.220 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:55.220 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:26:55.222 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:55.260 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:55.260 INFO:tasks.workunit.client.1.smithi134.stdout:6 2024-03-09T09:26:55.260 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 6 2024-03-09T09:26:55.260 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:55.445 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:26:55.484 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:26:55.494 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:56.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.036+0000 7f9107400640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 msgr2=0x7f90f00097f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:56.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.036+0000 7f9107400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.67:3300/0 conn(0x7f90f0007370 0x7f90f00097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:56.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.036+0000 7f9107400640 1 -- 172.21.15.67:0/1592625099 >> v1:172.21.15.67:6789/0 conn(0x7f90f00036c0 legacy=0x7f90f0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:56.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.036+0000 7f910e800640 1 -- 172.21.15.67:0/1592625099 reap_dead start 2024-03-09T09:26:56.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.036+0000 7f9107400640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:56.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.036+0000 7f9107400640 1 -- 172.21.15.67:0/1592625099 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f000af00 con 0x7f90f00036c0 2024-03-09T09:26:56.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.036+0000 7f9107400640 1 -- 172.21.15.67:0/1592625099 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f000ab10 con 0x7f90f000a520 2024-03-09T09:26:56.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.036+0000 7f9107400640 1 -- 172.21.15.67:0/1592625099 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9110156dc0 con 0x7f9110153750 2024-03-09T09:26:56.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.037+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:56.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.037+0000 7f910de00640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f90f0009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:56.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.037+0000 7f910de00640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f90f0009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:56.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.037+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:56.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.037+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:56.058 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:26:56.058 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:26:56.171 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:56.171 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:56.171 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:26:56.207 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:56.207 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 7 2024-03-09T09:26:56.207 INFO:tasks.workunit.client.1.smithi134.stdout:7 2024-03-09T09:26:56.208 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.238+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.239+0000 7f910de00640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f90f0009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.239+0000 7f910de00640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f90f0009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.239+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:56.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.239+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:26:56.333 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:26:56.383 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:26:56.393 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:56.542 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:56.544 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.640+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.641+0000 7f910de00640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f90f0009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.641+0000 7f910de00640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f90f0009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.641+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:56.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:56.641+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:26:56.667 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:56.668 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:56.668 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:56.703 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:56.704 INFO:tasks.workunit.client.0.smithi067.stdout:17 2024-03-09T09:26:56.704 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 17 2024-03-09T09:26:56.704 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:56.819 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:56.866 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:57.002 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:26:57.002 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:26:57.124 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:57.124 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:57.124 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:26:57.132 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:57.153 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:57.153 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 8 2024-03-09T09:26:57.153 INFO:tasks.workunit.client.1.smithi134.stdout:8 2024-03-09T09:26:57.153 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:57.342 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:26:57.380 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:26:57.386 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:57.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.442+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:57.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.442+0000 7f910de00640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f90f0009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:26:57.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.442+0000 7f910de00640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f90f0009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:26:57.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.442+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:57.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.443+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:26:57.836 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:57.836 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:57.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.866+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714230241 0 0) 0x7f90f000ab10 con 0x7f90f000a520 2024-03-09T09:26:57.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.866+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91100c4010 con 0x7f90f000a520 2024-03-09T09:26:57.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 <== mon.1 v1:172.21.15.134:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1758715989 0 0) 0x7f91100c4010 con 0x7f90f000a520 2024-03-09T09:26:57.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 msgr2=0x7f90f0009f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:26:57.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 --2- 172.21.15.67:0/1592625099 >> v2:172.21.15.134:3300/0 conn(0x7f9110153750 0x7f90f0009f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:57.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 >> v1:172.21.15.67:6789/0 conn(0x7f90f00036c0 legacy=0x7f90f0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:57.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 --> v1:172.21.15.134:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f00097f0 con 0x7f90f000a520 2024-03-09T09:26:57.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 <== mon.1 v1:172.21.15.134:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f9108009950 con 0x7f90f000a520 2024-03-09T09:26:57.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 >> v1:172.21.15.134:6789/0 conn(0x7f90f000a520 legacy=0x7f90f000d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:57.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90f4003b40 0x7f90f4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:57.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90f4004800 0x7f90f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:57.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f40099b0 0x7f90f400bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:57.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9110156dc0 con 0x7f90f4004800 2024-03-09T09:26:57.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f400c3c0 con 0x7f90f40099b0 2024-03-09T09:26:57.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f400c5c0 con 0x7f90f4003b40 2024-03-09T09:26:57.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.868+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90f4004800 0x7f90f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:57.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.869+0000 7f910de00640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90f4003b40 0x7f90f4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:57.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.869+0000 7f910e800640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f40099b0 0x7f90f400bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:57.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.869+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90f4003b40 msgr2=0x7f90f4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:26:57.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.869+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90f4003b40 0x7f90f4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:57.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.869+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f40099b0 msgr2=0x7f90f400bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:26:57.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.869+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f40099b0 0x7f90f400bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:57.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.869+0000 7f910d400640 1 -- 172.21.15.67:0/1592625099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100c4010 con 0x7f90f4004800 2024-03-09T09:26:57.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.869+0000 7f910e800640 1 -- 172.21.15.67:0/1592625099 reap_dead start 2024-03-09T09:26:57.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.871+0000 7f910d400640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90f4004800 0x7f90f4004c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9100006fd0 tx=0x7f90f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e79ac779298b89 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:26:57.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.875+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100037070 con 0x7f90f4004800 2024-03-09T09:26:57.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.876+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9100033460 con 0x7f90f4004800 2024-03-09T09:26:57.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.876+0000 7f910ca00640 1 -- 172.21.15.67:0/1592625099 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000198e0 con 0x7f90f4004800 2024-03-09T09:26:57.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.877+0000 7f9114a00640 1 -- 172.21.15.67:0/1592625099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90f4004800 msgr2=0x7f90f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:26:57.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.877+0000 7f9114a00640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90f4004800 0x7f90f4004c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9100006fd0 tx=0x7f90f4005830 comp rx=0 tx=0).stop 2024-03-09T09:26:57.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.877+0000 7f9114a00640 1 -- 172.21.15.67:0/1592625099 shutdown_connections 2024-03-09T09:26:57.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.877+0000 7f9114a00640 1 --2- 172.21.15.67:0/1592625099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90f4004800 0x7f90f4004c80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:26:57.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.877+0000 7f9114a00640 1 -- 172.21.15.67:0/1592625099 >> 172.21.15.67:0/1592625099 conn(0x7f91100ab320 msgr2=0x7f91100ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:26:57.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.877+0000 7f9114a00640 1 -- 172.21.15.67:0/1592625099 shutdown_connections 2024-03-09T09:26:57.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.877+0000 7f9114a00640 1 -- 172.21.15.67:0/1592625099 wait complete. 2024-03-09T09:26:57.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.879+0000 7f9114a00640 1 Processor -- start 2024-03-09T09:26:57.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.879+0000 7f9114a00640 1 -- start start 2024-03-09T09:26:57.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.880+0000 7f9114a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:57.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.880+0000 7f9114a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:57.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.880+0000 7f9114a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:26:57.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.880+0000 7f9114a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100c5d30 con 0x7f9110070290 2024-03-09T09:26:57.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.880+0000 7f9114a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9110073dc0 con 0x7f90f4004800 2024-03-09T09:26:57.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.880+0000 7f9114a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91100c29a0 con 0x7f911006f6e0 2024-03-09T09:26:57.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.880+0000 7f910d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:57.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.880+0000 7f910d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44930/0 (socket says 172.21.15.67:44930) 2024-03-09T09:26:57.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.880+0000 7f910d400640 1 -- 172.21.15.67:0/3193989324 learned_addr learned my addr 172.21.15.67:0/3193989324 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.880+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.880+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910d400640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 msgr2=0x7f91101ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910d400640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 msgr2=0x7f91101ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910de00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 msgr2=0x7f9110154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910de00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 msgr2=0x7f9110154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910e800640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 msgr2=0x7f91101ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910e800640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 msgr2=0x7f91101ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:57.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:57.881+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:26:57.949 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:57.950 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:57.950 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:57.988 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:57.989 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 18 2024-03-09T09:26:57.989 INFO:tasks.workunit.client.0.smithi067.stdout:18 2024-03-09T09:26:57.990 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:58.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.082+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:58.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.082+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:58.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.082+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:58.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.082+0000 7f910e800640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 msgr2=0x7f91101ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:26:58.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.082+0000 7f910e800640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 msgr2=0x7f91101ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:58.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.082+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:58.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.082+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:26:58.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.083+0000 7f910d400640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 msgr2=0x7f91101ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:26:58.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.083+0000 7f910d400640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 msgr2=0x7f91101ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:58.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.083+0000 7f910de00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 msgr2=0x7f9110154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:26:58.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.083+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:58.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.083+0000 7f910de00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 msgr2=0x7f9110154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:58.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.083+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:58.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.083+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:26:58.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.083+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:26:58.093 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:26:58.093 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:26:58.105 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:58.139 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:58.146 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:58.216 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:58.216 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:58.216 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:26:58.253 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:58.253 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 9 2024-03-09T09:26:58.253 INFO:tasks.workunit.client.1.smithi134.stdout:9 2024-03-09T09:26:58.253 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:58.392 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:26:58.425 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:26:58.433 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:58.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.485+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:58.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.485+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:58.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.486+0000 7f910e800640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 msgr2=0x7f91101ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:26:58.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.486+0000 7f910e800640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 msgr2=0x7f91101ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:58.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.486+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:58.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.486+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:26:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.486+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.486+0000 7f910d400640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 msgr2=0x7f91101ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:26:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.486+0000 7f910d400640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 msgr2=0x7f91101ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.486+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.487+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:26:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.487+0000 7f910de00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 msgr2=0x7f9110154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-03-09T09:26:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.487+0000 7f910de00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 msgr2=0x7f9110154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.487+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:58.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:58.488+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:26:58.793 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:58.796 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:58.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:58.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:58.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:26:58.935 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:58.935 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 19 2024-03-09T09:26:58.935 INFO:tasks.workunit.client.0.smithi067.stdout:19 2024-03-09T09:26:58.935 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:58.961 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:26:58.961 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:26:59.065 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:26:59.080 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:59.080 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:59.080 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:26:59.102 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:26:59.109 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:59.123 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:26:59.123 INFO:tasks.workunit.client.1.smithi134.stdout:10 2024-03-09T09:26:59.123 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 10 2024-03-09T09:26:59.123 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:26:59.281 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:26:59.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.287+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:59.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.288+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.288+0000 7f910e800640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 msgr2=0x7f91101ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:26:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.288+0000 7f910e800640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 msgr2=0x7f91101ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.288+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.288+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:26:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.288+0000 7f910d400640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 msgr2=0x7f91101ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:26:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.288+0000 7f910d400640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 msgr2=0x7f91101ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.288+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:59.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.289+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:26:59.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.289+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:26:59.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.289+0000 7f910de00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 msgr2=0x7f9110154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:26:59.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.289+0000 7f910de00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 msgr2=0x7f9110154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:26:59.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.289+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:26:59.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:26:59.289+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:26:59.328 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:26:59.331 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:26:59.710 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:26:59.710 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:26:59.826 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:26:59.826 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:26:59.826 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:27:00.075 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in `seq 1 20` 2024-03-09T09:27:00.076 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 20 2024-03-09T09:27:00.076 INFO:tasks.workunit.client.0.smithi067.stdout:20 2024-03-09T09:27:00.076 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:27:00.186 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s 2024-03-09T09:27:00.228 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm a 2024-03-09T09:27:00.235 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:27:00.457 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:27:00.458 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:27:00.589 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:00.589 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:00.589 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:27:00.618 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:27:00.618 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 11 2024-03-09T09:27:00.619 INFO:tasks.workunit.client.1.smithi134.stdout:11 2024-03-09T09:27:00.619 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:27:00.724 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:27:00.761 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:27:00.767 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:27:00.833 INFO:tasks.workunit.client.0.smithi067.stderr:++ md5sum /tmp/a 2024-03-09T09:27:00.834 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-03-09T09:27:00.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 msgr2=0x7f91101ed200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:00.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f91101ed200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:00.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 msgr2=0x7f9110154eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:00.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f9110154eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:00.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 msgr2=0x7f91101ec710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:00.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9110070290 0x7f91101ec710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:00.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90f00099c0 0x7f90f0009f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f90f0005ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f90f0005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 -- 172.21.15.67:0/3193989324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100c5d30 con 0x7f90f00099c0 2024-03-09T09:27:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 -- 172.21.15.67:0/3193989324 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9110073dc0 con 0x7f90f4004800 2024-03-09T09:27:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.880+0000 7f9104c00640 1 -- 172.21.15.67:0/3193989324 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91100c29a0 con 0x7f911006f6e0 2024-03-09T09:27:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.881+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f90f0005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.881+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90f00099c0 0x7f90f0009f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:00.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.881+0000 7f910d400640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f90f0005ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:00.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.882+0000 7f910e800640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 msgr2=0x7f90f0005ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:00.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.882+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f90f0005ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:00.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.882+0000 7f910e800640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90f00099c0 msgr2=0x7f90f0009f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:00.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.882+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90f00099c0 0x7f90f0009f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:00.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.882+0000 7f910e800640 1 -- 172.21.15.67:0/3193989324 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110070790 con 0x7f911006f6e0 2024-03-09T09:27:00.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.882+0000 7f910e800640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f90f0005ff0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9110075040 tx=0x7f910801b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:00.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.883+0000 7f9106000640 1 -- 172.21.15.67:0/3193989324 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108029070 con 0x7f911006f6e0 2024-03-09T09:27:00.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.883+0000 7f9106000640 1 -- 172.21.15.67:0/3193989324 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9108024410 con 0x7f911006f6e0 2024-03-09T09:27:00.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.883+0000 7f9106000640 1 -- 172.21.15.67:0/3193989324 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91080245e0 con 0x7f911006f6e0 2024-03-09T09:27:00.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.883+0000 7f9114a00640 1 -- 172.21.15.67:0/3193989324 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911006ec00 con 0x7f911006f6e0 2024-03-09T09:27:00.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.883+0000 7f9114a00640 1 -- 172.21.15.67:0/3193989324 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911006f1c0 con 0x7f911006f6e0 2024-03-09T09:27:00.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.884+0000 7f9114a00640 1 -- 172.21.15.67:0/3193989324 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9110075a00 con 0x7f911006f6e0 2024-03-09T09:27:00.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.886+0000 7f9106000640 1 -- 172.21.15.67:0/3193989324 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9108005d10 con 0x7f911006f6e0 2024-03-09T09:27:00.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.887+0000 7f9106000640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e806b4d0 0x7f90e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:00.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.887+0000 7f9106000640 1 -- 172.21.15.67:0/3193989324 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f910808bc60 con 0x7f911006f6e0 2024-03-09T09:27:00.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.889+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e806b4d0 0x7f90e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:00.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.891+0000 7f910de00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e806b4d0 0x7f90e806d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f90f000aed0 tx=0x7f90fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:00.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:00.894+0000 7f9106000640 1 -- 172.21.15.67:0/3193989324 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f910801be20 con 0x7f911006f6e0 2024-03-09T09:27:00.972 INFO:tasks.workunit.client.0.smithi067.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:00.972 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:00.972 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s 2024-03-09T09:27:00.996 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm linux-2.6.33.tar.bz2 2024-03-09T09:27:01.002 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:27:01.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T09:27:01.115 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-upchildrealms.sh... 2024-03-09T09:27:01.115 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-upchildrealms.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-upchildrealms.sh 2024-03-09T09:27:01.184 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T09:27:01.185 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T09:27:01.189 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/b 2024-03-09T09:27:01.193 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/a1 2024-03-09T09:27:01.233 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/b/.snap/b1 2024-03-09T09:27:01.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.249+0000 7f9114a00640 1 -- 172.21.15.67:0/3193989324 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f911006c7a0 con 0x7f911006f6e0 2024-03-09T09:27:01.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.250+0000 7f9106000640 1 -- 172.21.15.67:0/3193989324 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f910805a1a0 con 0x7f911006f6e0 2024-03-09T09:27:01.257 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":276,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:27:01.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.253+0000 7f9114a00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e806b4d0 msgr2=0x7f90e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.253+0000 7f9114a00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e806b4d0 0x7f90e806d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f90f000aed0 tx=0x7f90fc009040 comp rx=0 tx=0).stop 2024-03-09T09:27:01.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.253+0000 7f9114a00640 1 -- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 msgr2=0x7f90f0005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.253+0000 7f9114a00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f90f0005ff0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9110075040 tx=0x7f910801b820 comp rx=0 tx=0).stop 2024-03-09T09:27:01.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.254+0000 7f9114a00640 1 -- 172.21.15.67:0/3193989324 shutdown_connections 2024-03-09T09:27:01.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.254+0000 7f9114a00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e806b4d0 0x7f90e806d990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.254+0000 7f9114a00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f911006f6e0 0x7f90f0005ff0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.254+0000 7f9114a00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f4004800 0x7f90f0005ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.254+0000 7f9114a00640 1 --2- 172.21.15.67:0/3193989324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90f00099c0 0x7f90f0009f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.254+0000 7f9114a00640 1 -- 172.21.15.67:0/3193989324 >> 172.21.15.67:0/3193989324 conn(0x7f91100ab320 msgr2=0x7f911006ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:01.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.254+0000 7f9114a00640 1 -- 172.21.15.67:0/3193989324 shutdown_connections 2024-03-09T09:27:01.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.254+0000 7f9114a00640 1 -- 172.21.15.67:0/3193989324 wait complete. 2024-03-09T09:27:01.267 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo asdf 2024-03-09T09:27:01.271 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/a2 2024-03-09T09:27:01.284 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:27:01.284 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:27:01.284 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:27:01.307 INFO:tasks.workunit.client.0.smithi067.stderr:+ ln a/b/foo a/b/bar 2024-03-09T09:27:01.314 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'this should not hang...' 2024-03-09T09:27:01.314 INFO:tasks.workunit.client.0.smithi067.stdout:this should not hang... 2024-03-09T09:27:01.333 INFO:tasks.workunit.client.0.smithi067.stderr:+ cat a/b/.snap/_a2_1099511669214/foo 2024-03-09T09:27:01.336 INFO:tasks.workunit.client.0.smithi067.stdout:asdf 2024-03-09T09:27:01.336 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'good, it did not hang.' 2024-03-09T09:27:01.337 INFO:tasks.workunit.client.0.smithi067.stdout:good, it did not hang. 2024-03-09T09:27:01.337 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/b/.snap/b1 2024-03-09T09:27:01.348 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/a1 2024-03-09T09:27:01.361 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/a2 2024-03-09T09:27:01.373 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -r a 2024-03-09T09:27:01.472 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T09:27:01.472 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T09:27:01.473 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:27:01.473 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T09:27:01.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.487+0000 7ff313e00640 1 Processor -- start 2024-03-09T09:27:01.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.487+0000 7ff313e00640 1 -- start start 2024-03-09T09:27:01.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.489+0000 7ff313e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff3140aab20 0x7ff3140aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.489+0000 7ff313e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3140c1c20 con 0x7ff3141575b0 2024-03-09T09:27:01.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.489+0000 7ff313e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31415a7f0 con 0x7ff3140c3700 2024-03-09T09:27:01.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.489+0000 7ff313e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3140aa3b0 con 0x7ff3140aab20 2024-03-09T09:27:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.489+0000 7ff312a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff3141575b0 0x7ff31415a010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55818/0 (socket says 172.21.15.67:55818) 2024-03-09T09:27:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.489+0000 7ff312a00640 1 -- 172.21.15.67:0/3970607137 learned_addr learned my addr 172.21.15.67:0/3970607137 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:27:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.489+0000 7ff313400640 1 --2- 172.21.15.67:0/3970607137 >> v2:172.21.15.134:3300/0 conn(0x7ff3140aab20 0x7ff3140aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.490+0000 7ff312000640 1 -- 172.21.15.67:0/3970607137 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978748399 0 0) 0x7ff3140c1c20 con 0x7ff3141575b0 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.490+0000 7ff312000640 1 -- 172.21.15.67:0/3970607137 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2f8003520 con 0x7ff3141575b0 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.490+0000 7ff313400640 1 -- 172.21.15.67:0/3970607137 >> v1:172.21.15.134:6789/0 conn(0x7ff3140c3700 legacy=0x7ff3140ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.490+0000 7ff312000640 1 -- 172.21.15.67:0/3970607137 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3230682082 0 0) 0x7ff2f8003520 con 0x7ff3141575b0 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.490+0000 7ff313400640 1 -- 172.21.15.67:0/3970607137 >> v1:172.21.15.67:6789/0 conn(0x7ff3141575b0 legacy=0x7ff31415a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.490+0000 7ff313400640 1 -- 172.21.15.67:0/3970607137 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31415aad0 con 0x7ff3140aab20 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.491+0000 7ff313400640 1 --2- 172.21.15.67:0/3970607137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3140aab20 0x7ff3140aaf70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009d90 tx=0x7ff30c02bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826cef25f1dfa2aa server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.492+0000 7ff312000640 1 -- 172.21.15.67:0/3970607137 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c039070 con 0x7ff3140aab20 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.492+0000 7ff312000640 1 -- 172.21.15.67:0/3970607137 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c02c8b0 con 0x7ff3140aab20 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.492+0000 7ff312000640 1 -- 172.21.15.67:0/3970607137 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304001670 con 0x7ff3140aab20 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.492+0000 7ff313e00640 1 -- 172.21.15.67:0/3970607137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3140aab20 msgr2=0x7ff3140aaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.492+0000 7ff313e00640 1 --2- 172.21.15.67:0/3970607137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3140aab20 0x7ff3140aaf70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009d90 tx=0x7ff30c02bd80 comp rx=0 tx=0).stop 2024-03-09T09:27:01.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.493+0000 7ff313e00640 1 -- 172.21.15.67:0/3970607137 shutdown_connections 2024-03-09T09:27:01.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.493+0000 7ff313e00640 1 --2- 172.21.15.67:0/3970607137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3140aab20 0x7ff3140aaf70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.493+0000 7ff313e00640 1 -- 172.21.15.67:0/3970607137 >> 172.21.15.67:0/3970607137 conn(0x7ff3140c7220 msgr2=0x7ff3140c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:01.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.493+0000 7ff313e00640 1 -- 172.21.15.67:0/3970607137 shutdown_connections 2024-03-09T09:27:01.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.493+0000 7ff313e00640 1 -- 172.21.15.67:0/3970607137 wait complete. 2024-03-09T09:27:01.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.495+0000 7ff313e00640 1 Processor -- start 2024-03-09T09:27:01.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.496+0000 7ff313e00640 1 -- start start 2024-03-09T09:27:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.497+0000 7ff313e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3140aab20 0x7ff314070930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.497+0000 7ff313e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3140c3700 0x7ff314151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.497+0000 7ff313e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141575b0 0x7ff31414fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.497+0000 7ff313e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314157990 con 0x7ff3141575b0 2024-03-09T09:27:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.497+0000 7ff313e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140aa600 con 0x7ff3140aab20 2024-03-09T09:27:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.497+0000 7ff313e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140abee0 con 0x7ff3140c3700 2024-03-09T09:27:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.497+0000 7ff313400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141575b0 0x7ff31414fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.498+0000 7ff313400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141575b0 0x7ff31414fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59312/0 (socket says 172.21.15.67:59312) 2024-03-09T09:27:01.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.498+0000 7ff313400640 1 -- 172.21.15.67:0/1661177787 learned_addr learned my addr 172.21.15.67:0/1661177787 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.498+0000 7ff30be00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3140c3700 0x7ff314151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.498+0000 7ff312a00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3140aab20 0x7ff314070930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.498+0000 7ff313400640 1 -- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3140c3700 msgr2=0x7ff314151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.498+0000 7ff313400640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3140c3700 0x7ff314151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.498+0000 7ff313400640 1 -- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3140aab20 msgr2=0x7ff314070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.498+0000 7ff313400640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3140aab20 0x7ff314070930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.498+0000 7ff313400640 1 -- 172.21.15.67:0/1661177787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314070fc0 con 0x7ff3141575b0 2024-03-09T09:27:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.498+0000 7ff312a00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3140aab20 0x7ff314070930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:27:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.499+0000 7ff313400640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141575b0 0x7ff31414fcd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009d90 tx=0x7ff30c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.499+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c044030 con 0x7ff3141575b0 2024-03-09T09:27:01.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.499+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c034af0 con 0x7ff3141575b0 2024-03-09T09:27:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.499+0000 7ff313e00640 1 -- 172.21.15.67:0/1661177787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff314073d40 con 0x7ff3141575b0 2024-03-09T09:27:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.499+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c039040 con 0x7ff3141575b0 2024-03-09T09:27:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.499+0000 7ff313e00640 1 -- 172.21.15.67:0/1661177787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31415c4e0 con 0x7ff3141575b0 2024-03-09T09:27:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.500+0000 7ff313e00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff314072f60 0x7ff314150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.501+0000 7ff313e00640 1 -- 172.21.15.67:0/1661177787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140ab7d0 con 0x7ff314072f60 2024-03-09T09:27:01.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.501+0000 7ff312a00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff314072f60 0x7ff314150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.501+0000 7ff312a00640 1 -- 172.21.15.67:0/1661177787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff31404f3c0 con 0x7ff314072f60 2024-03-09T09:27:01.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.502+0000 7ff312a00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff314072f60 0x7ff314150210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff304005880 tx=0x7ff30400db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:01.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.502+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304005ad0 con 0x7ff314072f60 2024-03-09T09:27:01.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.502+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff30c047070 con 0x7ff3141575b0 2024-03-09T09:27:01.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.504+0000 7ff30aa00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2f006b3d0 0x7ff2f006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.504+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6721+0+0 (secure 0 0 0) 0x7ff30c0ac9e0 con 0x7ff3141575b0 2024-03-09T09:27:01.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.504+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff304017070 con 0x7ff314072f60 2024-03-09T09:27:01.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.504+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff314072f60 msgr2=0x7ff314150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.504+0000 7ff30aa00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff314072f60 0x7ff314150210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff304005880 tx=0x7ff30400db80 comp rx=0 tx=0).stop 2024-03-09T09:27:01.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.504+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff314072f60 msgr2=0x7ff314150210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:01.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.504+0000 7ff30aa00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff314072f60 0x7ff314150210 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.507+0000 7ff30be00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2f006b3d0 0x7ff2f006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.508+0000 7ff30be00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2f006b3d0 0x7ff2f006d890 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff3141f0d70 tx=0x7ff2fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:01.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.512+0000 7ff313e00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141f31b0 0x7ff3141f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.512+0000 7ff313e00640 1 -- 172.21.15.67:0/1661177787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140c1290 con 0x7ff3141f31b0 2024-03-09T09:27:01.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.513+0000 7ff312a00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141f31b0 0x7ff3141f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.513+0000 7ff312a00640 1 -- 172.21.15.67:0/1661177787 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff3140c1900 con 0x7ff3141f31b0 2024-03-09T09:27:01.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.513+0000 7ff312a00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141f31b0 0x7ff3141f35c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff304005880 tx=0x7ff304000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:01.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.514+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304007500 con 0x7ff3141f31b0 2024-03-09T09:27:01.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.514+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4536 (secure 0 0 0) 0x7ff304007500 con 0x7ff3141f31b0 2024-03-09T09:27:01.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.514+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141f31b0 msgr2=0x7ff3141f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.514+0000 7ff30aa00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141f31b0 0x7ff3141f35c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff304005880 tx=0x7ff304000ed0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.514+0000 7ff30aa00640 1 -- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141f31b0 msgr2=0x7ff3141f35c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:01.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.514+0000 7ff30aa00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141f31b0 0x7ff3141f35c0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff304005880 tx=0x7ff304000ed0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.523 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:27:01.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:01.523 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:01.524 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:27:01.524 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 276, 2024-03-09T09:27:01.524 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:27:01.525 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:27:01.525 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:27:01.525 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:27:01.525 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:01.525 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:27:01.525 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:01.526 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:27:01.526 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:27:01.526 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:01.526 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:01.527 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:01.527 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:01.527 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:01.527 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:01.527 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:01.527 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:01.527 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:01.528 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:01.528 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:01.528 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:01.528 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:27:01.528 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:27:01.528 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:01.528 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:01.529 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:01.529 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:01.529 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:01.529 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:01.529 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:01.529 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:01.529 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:01.529 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:01.529 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:01.529 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:01.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.530 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:27:01.530 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:27:01.530 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:27:01.530 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:27:01.530 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:27:01.530 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:27:01.530 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:01.530 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:01.530 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:27:01.531 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:27:01.531 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:27:01.531 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:27:01.531 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:27:01.531 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:27:01.531 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:27:01.531 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:01.531 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:27:01.531 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:01.532 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:01.532 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:01.532 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:01.532 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:01.532 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:01.533 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:01.533 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:01.533 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:01.534 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:01.534 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:01.534 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:01.534 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:27:01.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.535 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:27:01.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:01.535 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:01.535 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:01.535 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:01.535 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:01.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:01.535 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:01.535 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:27:01.536 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:01.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:01.536 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:01.536 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:27:01.536 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:01.536 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:01.536 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:01.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.537 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:27:01.537 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:27:01.537 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:01.537 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:01.537 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:01.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:01.538 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:01.538 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:01.538 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:01.538 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:01.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:01.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:01.538 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:27:01.538 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:01.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:01.539 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:01.539 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:27:01.539 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:01.539 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:01.539 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:01.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.539 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:27:01.541 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:01.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:01.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:01.542 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:27:01.542 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:01.542 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:01.542 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:01.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.543 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:27:01.543 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:27:01.543 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:01.543 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:01.543 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:01.543 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:01.544 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:01.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.544 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:27:01.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:27:01.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:01.544 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:01.544 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:01.544 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:01.544 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:01.545 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:01.545 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:27:01.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:01.545 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:01.545 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:01.545 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:27:01.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:01.546 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:01.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:01.546 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:27:01.546 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:27:01.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.515+0000 7ff313e00640 1 -- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2f006b3d0 msgr2=0x7ff2f006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.515+0000 7ff313e00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff2f006b3d0 0x7ff2f006d890 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff3141f0d70 tx=0x7ff2fc009040 comp rx=0 tx=0).stop 2024-03-09T09:27:01.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.515+0000 7ff313e00640 1 -- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141575b0 msgr2=0x7ff31414fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.515+0000 7ff313e00640 1 --2- 172.21.15.67:0/1661177787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3141575b0 0x7ff31414fcd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009d90 tx=0x7ff30c004a40 comp rx=0 tx=0).stop 2024-03-09T09:27:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.515+0000 7ff313400640 1 -- 172.21.15.67:0/1661177787 reap_dead start 2024-03-09T09:27:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.515+0000 7ff313400640 1 -- 172.21.15.67:0/1661177787 reap_dead start 2024-03-09T09:27:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.515+0000 7ff313e00640 1 -- 172.21.15.67:0/1661177787 shutdown_connections 2024-03-09T09:27:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.515+0000 7ff313e00640 1 -- 172.21.15.67:0/1661177787 >> 172.21.15.67:0/1661177787 conn(0x7ff3140c7220 msgr2=0x7ff3140c4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.515+0000 7ff313e00640 1 -- 172.21.15.67:0/1661177787 shutdown_connections 2024-03-09T09:27:01.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.515+0000 7ff313e00640 1 -- 172.21.15.67:0/1661177787 wait complete. 2024-03-09T09:27:01.550 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:27:01.551 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:27:01.551 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:27:01.588 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-xattrwb.sh... 2024-03-09T09:27:01.588 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/snaptest-xattrwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-xattrwb.sh 2024-03-09T09:27:01.742 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2024-03-09T09:27:01.742 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'testing simple xattr wb' 2024-03-09T09:27:01.743 INFO:tasks.workunit.client.0.smithi067.stdout:testing simple xattr wb 2024-03-09T09:27:01.743 INFO:tasks.workunit.client.0.smithi067.stderr:+ touch x 2024-03-09T09:27:01.750 INFO:tasks.workunit.client.0.smithi067.stderr:+ setfattr -n user.foo x 2024-03-09T09:27:01.777 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/s1 2024-03-09T09:27:01.880 INFO:tasks.workunit.client.0.smithi067.stderr:+ getfattr -n user.foo .snap/s1/x 2024-03-09T09:27:01.881 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep user.foo 2024-03-09T09:27:01.896 INFO:tasks.workunit.client.0.smithi067.stdout:user.foo="" 2024-03-09T09:27:01.896 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm x 2024-03-09T09:27:01.916 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir .snap/s1 2024-03-09T09:27:01.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.975+0000 7f3991800640 1 Processor -- start 2024-03-09T09:27:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.975+0000 7f3991800640 1 -- start start 2024-03-09T09:27:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.977+0000 7f3991800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f398c0c8c60 0x7f398c0c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.977+0000 7f3991800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f398c0c9d50 0x7f398c0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.977+0000 7f3991800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c06b040 con 0x7f398c153460 2024-03-09T09:27:01.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.979+0000 7f398aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f398c153460 0x7f398c155ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55830/0 (socket says 172.21.15.67:55830) 2024-03-09T09:27:01.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.979+0000 7f398a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f398c0c8c60 0x7f398c0c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.980+0000 7f398aa00640 1 -- 172.21.15.67:0/966811555 learned_addr learned my addr 172.21.15.67:0/966811555 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:27:01.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.980+0000 7f398b400640 1 --2- 172.21.15.67:0/966811555 >> v2:172.21.15.134:3300/0 conn(0x7f398c0c9d50 0x7f398c0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.981+0000 7f3991800640 1 -- 172.21.15.67:0/966811555 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c0ca590 con 0x7f398c0c8c60 2024-03-09T09:27:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.982+0000 7f3991800640 1 -- 172.21.15.67:0/966811555 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c156710 con 0x7f398c0c9d50 2024-03-09T09:27:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.983+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15746799 0 0) 0x7f397c001110 con 0x7f398c153460 2024-03-09T09:27:01.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.983+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3970003520 con 0x7f398c153460 2024-03-09T09:27:01.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.984+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1556089348 0 0) 0x7f3970003520 con 0x7f398c153460 2024-03-09T09:27:01.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.984+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 >> v2:172.21.15.134:3300/0 conn(0x7f398c0c9d50 msgr2=0x7f398c0caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.984+0000 7f3989600640 1 --2- 172.21.15.67:0/966811555 >> v2:172.21.15.134:3300/0 conn(0x7f398c0c9d50 0x7f398c0caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.984+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 >> v2:172.21.15.67:3300/0 conn(0x7f398c0c8c60 msgr2=0x7f398c0c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.984+0000 7f3989600640 1 --2- 172.21.15.67:0/966811555 >> v2:172.21.15.67:3300/0 conn(0x7f398c0c8c60 0x7f398c0c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.985+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c1569f0 con 0x7f398c153460 2024-03-09T09:27:01.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.985+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f397c001840 con 0x7f398c153460 2024-03-09T09:27:01.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.985+0000 7f398a000640 1 --2- 172.21.15.67:0/966811555 >> v2:172.21.15.67:3300/0 conn(0x7f398c0c8c60 0x7f398c0c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:27:01.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.985+0000 7f398b400640 1 --2- 172.21.15.67:0/966811555 >> v2:172.21.15.134:3300/0 conn(0x7f398c0c9d50 0x7f398c0caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:27:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.985+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 >> v1:172.21.15.67:6789/0 conn(0x7f398c153460 legacy=0x7f398c155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.986+0000 7f3989600640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3970003a80 0x7f3970003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.986+0000 7f3989600640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3970004800 0x7f3970004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.986+0000 7f3989600640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3970005210 0x7f397000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.986+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0ca590 con 0x7f3970005210 2024-03-09T09:27:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.986+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c156710 con 0x7f3970003a80 2024-03-09T09:27:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.986+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397000c1e0 con 0x7f3970004800 2024-03-09T09:27:01.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.987+0000 7f398aa00640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3970003a80 0x7f3970003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.987+0000 7f398a000640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3970004800 0x7f3970004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.988+0000 7f398b400640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3970005210 0x7f397000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:01.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.989+0000 7f398a000640 1 -- 172.21.15.67:0/966811555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3970003a80 msgr2=0x7f3970003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:01.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.989+0000 7f398a000640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3970003a80 0x7f3970003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.989+0000 7f398a000640 1 -- 172.21.15.67:0/966811555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3970005210 msgr2=0x7f397000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:01.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.989+0000 7f398a000640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3970005210 0x7f397000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:01.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.989+0000 7f398a000640 1 -- 172.21.15.67:0/966811555 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3970003520 con 0x7f3970004800 2024-03-09T09:27:01.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.989+0000 7f398b400640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3970005210 0x7f397000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:27:01.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.989+0000 7f398b400640 1 -- 172.21.15.67:0/966811555 reap_dead start 2024-03-09T09:27:01.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.991+0000 7f398a000640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3970004800 0x7f3970004c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f39780098a0 tx=0x7f3970005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6755cfacc8419e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:01.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.992+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397800f040 con 0x7f3970004800 2024-03-09T09:27:02.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.992+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3978004230 con 0x7f3970004800 2024-03-09T09:27:02.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.992+0000 7f3989600640 1 -- 172.21.15.67:0/966811555 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978003610 con 0x7f3970004800 2024-03-09T09:27:02.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.994+0000 7f3991800640 1 -- 172.21.15.67:0/966811555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3970004800 msgr2=0x7f3970004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.994+0000 7f3991800640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3970004800 0x7f3970004c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f39780098a0 tx=0x7f3970005830 comp rx=0 tx=0).stop 2024-03-09T09:27:02.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.994+0000 7f3991800640 1 -- 172.21.15.67:0/966811555 shutdown_connections 2024-03-09T09:27:02.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.994+0000 7f3991800640 1 --2- 172.21.15.67:0/966811555 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3970004800 0x7f3970004c80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.994+0000 7f3991800640 1 -- 172.21.15.67:0/966811555 >> 172.21.15.67:0/966811555 conn(0x7f398c0c8170 msgr2=0x7f398c0c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:02.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.995+0000 7f3991800640 1 -- 172.21.15.67:0/966811555 shutdown_connections 2024-03-09T09:27:02.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.995+0000 7f3991800640 1 -- 172.21.15.67:0/966811555 wait complete. 2024-03-09T09:27:02.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.998+0000 7f3991800640 1 Processor -- start 2024-03-09T09:27:02.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:01.998+0000 7f3991800640 1 -- start start 2024-03-09T09:27:02.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.001+0000 7f3991800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3970004800 0x7f398c0c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.001+0000 7f3991800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f398c1f3140 0x7f398c158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.001+0000 7f3991800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398c158b50 0x7f398c14b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.001+0000 7f3991800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0c9810 con 0x7f398c1f3140 2024-03-09T09:27:02.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.001+0000 7f3991800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c153840 con 0x7f3970004800 2024-03-09T09:27:02.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.001+0000 7f3991800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0c9990 con 0x7f398c158b50 2024-03-09T09:27:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.002+0000 7f398aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3970004800 0x7f398c0c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.002+0000 7f398aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3970004800 0x7f398c0c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:58800/0 (socket says 172.21.15.67:58800) 2024-03-09T09:27:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.002+0000 7f398aa00640 1 -- 172.21.15.67:0/3735825860 learned_addr learned my addr 172.21.15.67:0/3735825860 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.002+0000 7f398b400640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398c158b50 0x7f398c14b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.002+0000 7f398a000640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f398c1f3140 0x7f398c158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.003+0000 7f398b400640 1 -- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3970004800 msgr2=0x7f398c0c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.003+0000 7f398b400640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3970004800 0x7f398c0c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.003+0000 7f398b400640 1 -- 172.21.15.67:0/3735825860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f398c1f3140 msgr2=0x7f398c158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.003+0000 7f398b400640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f398c1f3140 0x7f398c158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.003+0000 7f398b400640 1 -- 172.21.15.67:0/3735825860 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c1479c0 con 0x7f398c158b50 2024-03-09T09:27:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.009+0000 7f398b400640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398c158b50 0x7f398c14b5a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f398c06ad60 tx=0x7f3984004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.012+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984007c10 con 0x7f398c158b50 2024-03-09T09:27:02.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.012+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3984019070 con 0x7f398c158b50 2024-03-09T09:27:02.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.012+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984014410 con 0x7f398c158b50 2024-03-09T09:27:02.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.013+0000 7f3991800640 1 -- 172.21.15.67:0/3735825860 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c147ca0 con 0x7f398c158b50 2024-03-09T09:27:02.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.013+0000 7f3991800640 1 -- 172.21.15.67:0/3735825860 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c148260 con 0x7f398c158b50 2024-03-09T09:27:02.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.018+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f39840145d0 con 0x7f398c158b50 2024-03-09T09:27:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.019+0000 7f3991800640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bae0 0x7f398c1498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.019+0000 7f3991800640 1 -- 172.21.15.67:0/3735825860 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0752b0 con 0x7f398c14bae0 2024-03-09T09:27:02.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.021+0000 7f3982a00640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f396c06b5d0 0x7f396c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.021+0000 7f398aa00640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bae0 0x7f398c1498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.021+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f398408a980 con 0x7f398c158b50 2024-03-09T09:27:02.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.022+0000 7f398a000640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f396c06b5d0 0x7f396c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.022+0000 7f398aa00640 1 -- 172.21.15.67:0/3735825860 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f398c158fd0 con 0x7f398c14bae0 2024-03-09T09:27:02.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.024+0000 7f398aa00640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bae0 0x7f398c1498e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f397c0012b0 tx=0x7f397c012620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.027+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c01b070 con 0x7f398c14bae0 2024-03-09T09:27:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.027+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f397c012e80 con 0x7f398c14bae0 2024-03-09T09:27:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.027+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bae0 msgr2=0x7f398c1498e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.027+0000 7f3982a00640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bae0 0x7f398c1498e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f397c0012b0 tx=0x7f397c012620 comp rx=0 tx=0).stop 2024-03-09T09:27:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.027+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bae0 msgr2=0x7f398c1498e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:02.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.027+0000 7f3982a00640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bae0 0x7f398c1498e0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f397c0012b0 tx=0x7f397c012620 comp rx=0 tx=0).stop 2024-03-09T09:27:02.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.031+0000 7f398a000640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f396c06b5d0 0x7f396c06da90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3978035000 tx=0x7f397803a040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.044 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'testing wb with pre-wb server cow' 2024-03-09T09:27:02.044 INFO:tasks.workunit.client.0.smithi067.stdout:testing wb with pre-wb server cow 2024-03-09T09:27:02.045 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a 2024-03-09T09:27:02.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.047+0000 7f3991800640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bec0 0x7f398c149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.047+0000 7f3991800640 1 -- 172.21.15.67:0/3735825860 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c075a00 con 0x7f398c14bec0 2024-03-09T09:27:02.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.049+0000 7f398aa00640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bec0 0x7f398c149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.057 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/b 2024-03-09T09:27:02.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.054+0000 7f398aa00640 1 -- 172.21.15.67:0/3735825860 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f398c0c95f0 con 0x7f398c14bec0 2024-03-09T09:27:02.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.055+0000 7f398aa00640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bec0 0x7f398c149e20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f397c018000 tx=0x7f397c008d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.063 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/b/c 2024-03-09T09:27:02.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.056+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c01b040 con 0x7f398c14bec0 2024-03-09T09:27:02.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.056+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4874 (secure 0 0 0) 0x7f397c0167d0 con 0x7f398c14bec0 2024-03-09T09:27:02.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.056+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bec0 msgr2=0x7f398c149e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.064 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:27:02.064 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 276, 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:27:02.065 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:27:02.066 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.067 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:27:02.068 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:02.069 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:02.070 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:02.072 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:02.073 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:02.075 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:02.076 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:27:02.077 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:27:02.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.056+0000 7f3982a00640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bec0 0x7f398c149e20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f397c018000 tx=0x7f397c008d60 comp rx=0 tx=0).stop 2024-03-09T09:27:02.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.056+0000 7f3982a00640 1 -- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bec0 msgr2=0x7f398c149e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.056+0000 7f3982a00640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f398c14bec0 0x7f398c149e20 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f397c018000 tx=0x7f397c008d60 comp rx=0 tx=0).stop 2024-03-09T09:27:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.062+0000 7f3991800640 1 -- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f396c06b5d0 msgr2=0x7f396c06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.062+0000 7f3991800640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f396c06b5d0 0x7f396c06da90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3978035000 tx=0x7f397803a040 comp rx=0 tx=0).stop 2024-03-09T09:27:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.062+0000 7f3991800640 1 -- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398c158b50 msgr2=0x7f398c14b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.062+0000 7f3991800640 1 --2- 172.21.15.67:0/3735825860 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f398c158b50 0x7f398c14b5a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f398c06ad60 tx=0x7f3984004700 comp rx=0 tx=0).stop 2024-03-09T09:27:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.062+0000 7f398b400640 1 -- 172.21.15.67:0/3735825860 reap_dead start 2024-03-09T09:27:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.062+0000 7f398b400640 1 -- 172.21.15.67:0/3735825860 reap_dead start 2024-03-09T09:27:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.063+0000 7f3991800640 1 -- 172.21.15.67:0/3735825860 shutdown_connections 2024-03-09T09:27:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.063+0000 7f3991800640 1 -- 172.21.15.67:0/3735825860 >> 172.21.15.67:0/3735825860 conn(0x7f398c0c8170 msgr2=0x7f398c154510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.065+0000 7f3991800640 1 -- 172.21.15.67:0/3735825860 shutdown_connections 2024-03-09T09:27:02.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.065+0000 7f3991800640 1 -- 172.21.15.67:0/3735825860 wait complete. 2024-03-09T09:27:02.079 INFO:tasks.workunit.client.0.smithi067.stderr:+ setfattr -n user.foo a/b 2024-03-09T09:27:02.080 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:27:02.080 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:27:02.082 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/.snap/s 2024-03-09T09:27:02.105 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:27:02.105 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:27:02.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:27:02.187 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir a/b/cc 2024-03-09T09:27:02.198 INFO:tasks.workunit.client.0.smithi067.stderr:+ getfattr -n user.foo a/b 2024-03-09T09:27:02.200 INFO:tasks.workunit.client.0.smithi067.stdout:# file: a/b 2024-03-09T09:27:02.200 INFO:tasks.workunit.client.0.smithi067.stdout:user.foo="" 2024-03-09T09:27:02.200 INFO:tasks.workunit.client.0.smithi067.stdout: 2024-03-09T09:27:02.202 INFO:tasks.workunit.client.0.smithi067.stderr:+ grep user.foo 2024-03-09T09:27:02.204 INFO:tasks.workunit.client.0.smithi067.stderr:+ getfattr -n user.foo a/.snap/s/b 2024-03-09T09:27:02.214 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:02.214 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:02.214 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:27:02.254 INFO:tasks.workunit.client.0.smithi067.stdout:user.foo="" 2024-03-09T09:27:02.255 INFO:tasks.workunit.client.0.smithi067.stderr:+ rmdir a/.snap/s 2024-03-09T09:27:02.287 INFO:tasks.workunit.client.1.smithi134.stdout:12 2024-03-09T09:27:02.287 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:27:02.287 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 12 2024-03-09T09:27:02.287 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:27:02.291 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -r a 2024-03-09T09:27:02.363 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-03-09T09:27:02.364 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-03-09T09:27:02.365 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:27:02.365 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-09T09:27:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.421+0000 7ff65aa00640 1 Processor -- start 2024-03-09T09:27:02.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.422+0000 7ff65aa00640 1 -- start start 2024-03-09T09:27:02.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.426+0000 7ff65aa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff65414f4b0 0x7ff654155430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.428+0000 7ff653e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff65414f4b0 0x7ff654155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.428+0000 7ff653e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ff65414f4b0 0x7ff654155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:58822/0 (socket says 172.21.15.67:58822) 2024-03-09T09:27:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.428+0000 7ff653e00640 1 -- 172.21.15.67:0/4240260195 learned_addr learned my addr 172.21.15.67:0/4240260195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.429+0000 7ff65aa00640 1 -- 172.21.15.67:0/4240260195 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6540c2ec0 con 0x7ff6540aa420 2024-03-09T09:27:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.429+0000 7ff65aa00640 1 -- 172.21.15.67:0/4240260195 --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65414f890 con 0x7ff6540c32b0 2024-03-09T09:27:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.430+0000 7ff65aa00640 1 -- 172.21.15.67:0/4240260195 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff654155970 con 0x7ff65414f4b0 2024-03-09T09:27:02.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.432+0000 7ff653e00640 1 -- 172.21.15.67:0/4240260195 >> v1:172.21.15.134:6789/0 conn(0x7ff6540c32b0 legacy=0x7ff654152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.432+0000 7ff653e00640 1 -- 172.21.15.67:0/4240260195 >> v1:172.21.15.67:6789/0 conn(0x7ff6540aa420 legacy=0x7ff6540abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.434+0000 7ff653e00640 1 -- 172.21.15.67:0/4240260195 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff654155ba0 con 0x7ff65414f4b0 2024-03-09T09:27:02.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.437+0000 7ff653e00640 1 --2- 172.21.15.67:0/4240260195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65414f4b0 0x7ff654155430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff644007360 tx=0x7ff64402c770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8976f1eb9a67b08 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.438+0000 7ff652a00640 1 -- 172.21.15.67:0/4240260195 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644038070 con 0x7ff65414f4b0 2024-03-09T09:27:02.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.439+0000 7ff652a00640 1 -- 172.21.15.67:0/4240260195 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff644008df0 con 0x7ff65414f4b0 2024-03-09T09:27:02.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.440+0000 7ff65aa00640 1 -- 172.21.15.67:0/4240260195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65414f4b0 msgr2=0x7ff654155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.440+0000 7ff65aa00640 1 --2- 172.21.15.67:0/4240260195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65414f4b0 0x7ff654155430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff644007360 tx=0x7ff64402c770 comp rx=0 tx=0).stop 2024-03-09T09:27:02.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.442+0000 7ff65aa00640 1 -- 172.21.15.67:0/4240260195 shutdown_connections 2024-03-09T09:27:02.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.442+0000 7ff65aa00640 1 --2- 172.21.15.67:0/4240260195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff65414f4b0 0x7ff654155430 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.442+0000 7ff65aa00640 1 -- 172.21.15.67:0/4240260195 >> 172.21.15.67:0/4240260195 conn(0x7ff6540c7370 msgr2=0x7ff6540c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:02.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.443+0000 7ff65aa00640 1 -- 172.21.15.67:0/4240260195 shutdown_connections 2024-03-09T09:27:02.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.443+0000 7ff65aa00640 1 -- 172.21.15.67:0/4240260195 wait complete. 2024-03-09T09:27:02.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.448+0000 7ff65aa00640 1 Processor -- start 2024-03-09T09:27:02.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.448+0000 7ff65aa00640 1 -- start start 2024-03-09T09:27:02.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.450+0000 7ff65aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6540aa420 0x7ff654155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.450+0000 7ff65aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6540c32b0 0x7ff65413ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.450+0000 7ff65aa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff65414f4b0 0x7ff6541efc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.450+0000 7ff65aa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff654152050 con 0x7ff6540c32b0 2024-03-09T09:27:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.450+0000 7ff65aa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff654156160 con 0x7ff6540aa420 2024-03-09T09:27:02.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.450+0000 7ff65aa00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6540ac3b0 con 0x7ff65414f4b0 2024-03-09T09:27:02.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.451+0000 7ff653400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6540c32b0 0x7ff65413ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.451+0000 7ff653400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6540c32b0 0x7ff65413ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59370/0 (socket says 172.21.15.67:59370) 2024-03-09T09:27:02.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.451+0000 7ff653400640 1 -- 172.21.15.67:0/3889797556 learned_addr learned my addr 172.21.15.67:0/3889797556 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:02.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.452+0000 7ff653e00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6540aa420 0x7ff654155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.452+0000 7ff658a00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff65414f4b0 0x7ff6541efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.452+0000 7ff653400640 1 -- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff65414f4b0 msgr2=0x7ff6541efc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.453+0000 7ff653400640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff65414f4b0 0x7ff6541efc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.453+0000 7ff653400640 1 -- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6540aa420 msgr2=0x7ff654155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.453+0000 7ff653400640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6540aa420 0x7ff654155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.453+0000 7ff653400640 1 -- 172.21.15.67:0/3889797556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65413ef70 con 0x7ff6540c32b0 2024-03-09T09:27:02.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.454+0000 7ff658a00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff65414f4b0 0x7ff6541efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:27:02.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.454+0000 7ff653e00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6540aa420 0x7ff654155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:27:02.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.456+0000 7ff653400640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6540c32b0 0x7ff65413ea30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c570 tx=0x7ff64000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.458+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640017070 con 0x7ff6540c32b0 2024-03-09T09:27:02.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.458+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6400035f0 con 0x7ff6540c32b0 2024-03-09T09:27:02.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.460+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400039e0 con 0x7ff6540c32b0 2024-03-09T09:27:02.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.460+0000 7ff65aa00640 1 -- 172.21.15.67:0/3889797556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff654141f10 con 0x7ff6540c32b0 2024-03-09T09:27:02.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.462+0000 7ff65aa00640 1 -- 172.21.15.67:0/3889797556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff654154c00 con 0x7ff6540c32b0 2024-03-09T09:27:02.472 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:27:02.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.465+0000 7ff65aa00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654154f70 0x7ff654150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.465+0000 7ff65aa00640 1 -- 172.21.15.67:0/3889797556 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541549f0 con 0x7ff654154f70 2024-03-09T09:27:02.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.467+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff64001a070 con 0x7ff6540c32b0 2024-03-09T09:27:02.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.467+0000 7ff653e00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654154f70 0x7ff654150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.472+0000 7ff64be00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff63406b4d0 0x7ff63406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.472+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6749+0+0 (secure 0 0 0) 0x7ff64008e060 con 0x7ff6540c32b0 2024-03-09T09:27:02.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.474+0000 7ff653e00640 1 -- 172.21.15.67:0/3889797556 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff654075a00 con 0x7ff654154f70 2024-03-09T09:27:02.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.474+0000 7ff658a00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff63406b4d0 0x7ff63406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.478+0000 7ff658a00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff63406b4d0 0x7ff63406d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0012e0 tx=0x7ff64c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.481+0000 7ff653e00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654154f70 0x7ff654150fa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff6540c83d0 tx=0x7ff6440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.483+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644042030 con 0x7ff654154f70 2024-03-09T09:27:02.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.483+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff64400a0d0 con 0x7ff654154f70 2024-03-09T09:27:02.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.483+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654154f70 msgr2=0x7ff654150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.483+0000 7ff64be00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654154f70 0x7ff654150fa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff6540c83d0 tx=0x7ff6440046c0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.483+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654154f70 msgr2=0x7ff654150fa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:02.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.483+0000 7ff64be00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654154f70 0x7ff654150fa0 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff6540c83d0 tx=0x7ff6440046c0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.507+0000 7ff65aa00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654145920 0x7ff654140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.507+0000 7ff65aa00640 1 -- 172.21.15.67:0/3889797556 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6540682c0 con 0x7ff654145920 2024-03-09T09:27:02.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.508+0000 7ff653e00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654145920 0x7ff654140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.510+0000 7ff653e00640 1 -- 172.21.15.67:0/3889797556 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff6540c3090 con 0x7ff654145920 2024-03-09T09:27:02.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.511+0000 7ff653e00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654145920 0x7ff654140270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff64402c7b0 tx=0x7ff644031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.512+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644042040 con 0x7ff654145920 2024-03-09T09:27:02.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.514+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7ff644007000 con 0x7ff654145920 2024-03-09T09:27:02.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.514+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654145920 msgr2=0x7ff654140270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.514+0000 7ff64be00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654145920 0x7ff654140270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff64402c7b0 tx=0x7ff644031000 comp rx=0 tx=0).stop 2024-03-09T09:27:02.521 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:27:02.521 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:02.521 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 276, 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 1, 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:02.522 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:02.523 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:27:02.524 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:02.525 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:27:02.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:02.527 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:02.528 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.529 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:02.530 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:02.531 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.532 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.533 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:02.534 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:27:02.535 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:02.535 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:02.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:02.535 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:27:02.535 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:27:02.536 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:27:02.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.515+0000 7ff64be00640 1 -- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654145920 msgr2=0x7ff654140270 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.515+0000 7ff64be00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff654145920 0x7ff654140270 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.517+0000 7ff65aa00640 1 -- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff63406b4d0 msgr2=0x7ff63406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.517+0000 7ff65aa00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff63406b4d0 0x7ff63406d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0012e0 tx=0x7ff64c009040 comp rx=0 tx=0).stop 2024-03-09T09:27:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.517+0000 7ff65aa00640 1 -- 172.21.15.67:0/3889797556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6540c32b0 msgr2=0x7ff65413ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.517+0000 7ff65aa00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6540c32b0 0x7ff65413ea30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c570 tx=0x7ff64000ca30 comp rx=0 tx=0).stop 2024-03-09T09:27:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.517+0000 7ff658a00640 1 -- 172.21.15.67:0/3889797556 reap_dead start 2024-03-09T09:27:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.519+0000 7ff65aa00640 1 -- 172.21.15.67:0/3889797556 shutdown_connections 2024-03-09T09:27:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.519+0000 7ff65aa00640 1 --2- 172.21.15.67:0/3889797556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6540c32b0 0x7ff65413ea30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.519+0000 7ff65aa00640 1 -- 172.21.15.67:0/3889797556 >> 172.21.15.67:0/3889797556 conn(0x7ff6540c7370 msgr2=0x7ff65414c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.520+0000 7ff65aa00640 1 -- 172.21.15.67:0/3889797556 shutdown_connections 2024-03-09T09:27:02.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.520+0000 7ff65aa00640 1 -- 172.21.15.67:0/3889797556 wait complete. 2024-03-09T09:27:02.541 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:27:02.549 INFO:tasks.workunit:Running workunit fs/snaps/untar_snap_rm.sh... 2024-03-09T09:27:02.550 DEBUG:teuthology.orchestra.run.smithi067:workunit test fs/snaps/untar_snap_rm.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh 2024-03-09T09:27:02.559 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:27:02.559 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:27:02.559 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:27:02.658 INFO:tasks.workunit.client.0.smithi067.stderr:+ do_tarball coreutils_8.5.orig.tar.gz z coreutils-8.5 2024-03-09T09:27:02.659 INFO:tasks.workunit.client.0.smithi067.stderr:+ wget http://download.ceph.com/qa/coreutils_8.5.orig.tar.gz 2024-03-09T09:27:02.675 INFO:tasks.workunit.client.0.smithi067.stderr:--2024-03-09 09:27:02-- http://download.ceph.com/qa/coreutils_8.5.orig.tar.gz 2024-03-09T09:27:02.677 INFO:tasks.workunit.client.0.smithi067.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-09T09:27:02.746 INFO:tasks.workunit.client.0.smithi067.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-09T09:27:02.805 INFO:tasks.workunit.client.0.smithi067.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-09T09:27:02.805 INFO:tasks.workunit.client.0.smithi067.stderr:Length: 10740866 (10M) [application/octet-stream] 2024-03-09T09:27:02.807 INFO:tasks.workunit.client.0.smithi067.stderr:Saving to: ‘coreutils_8.5.orig.tar.gz’ 2024-03-09T09:27:02.807 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-09T09:27:02.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:27:02.863 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:27:02.927 INFO:tasks.workunit.client.0.smithi067.stderr: 0K .......... .......... .......... .......... .......... 0% 418K 25s 2024-03-09T09:27:02.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.941+0000 7fe8e6e00640 1 Processor -- start 2024-03-09T09:27:02.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.941+0000 7fe8e6e00640 1 -- start start 2024-03-09T09:27:02.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.943+0000 7fe8e6e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe8d8050720 0x7fe8d80062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.943+0000 7fe8e6e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8d800fcf0 0x7fe8d8006800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.943+0000 7fe8e6e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d80d7d00 con 0x7fe8d80d7920 2024-03-09T09:27:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.943+0000 7fe8e6e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d80db250 con 0x7fe8d8050720 2024-03-09T09:27:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.943+0000 7fe8e6e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d804ba00 con 0x7fe8d800fcf0 2024-03-09T09:27:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.944+0000 7fe8e5a00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fe8d80d7920 0x7fe8d800edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:44802/0 (socket says 172.21.15.67:44802) 2024-03-09T09:27:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.944+0000 7fe8e5a00640 1 -- 172.21.15.67:0/1187239658 learned_addr learned my addr 172.21.15.67:0/1187239658 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:27:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.944+0000 7fe8e5000640 1 --2- 172.21.15.67:0/1187239658 >> v2:172.21.15.67:3300/0 conn(0x7fe8d8050720 0x7fe8d80062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.944+0000 7fe8e6400640 1 --2- 172.21.15.67:0/1187239658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8d800fcf0 0x7fe8d8006800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.945+0000 7fe8e5000640 1 -- 172.21.15.67:0/1187239658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8d800fcf0 msgr2=0x7fe8d8006800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.945+0000 7fe8e5000640 1 --2- 172.21.15.67:0/1187239658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8d800fcf0 0x7fe8d8006800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.945+0000 7fe8e5000640 1 -- 172.21.15.67:0/1187239658 >> v1:172.21.15.134:6789/0 conn(0x7fe8d80d7920 legacy=0x7fe8d800edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.945+0000 7fe8dfe00640 1 -- 172.21.15.67:0/1187239658 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052581250 0 0) 0x7fe8d80d7d00 con 0x7fe8d80d7920 2024-03-09T09:27:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.945+0000 7fe8e5000640 1 -- 172.21.15.67:0/1187239658 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d804bce0 con 0x7fe8d8050720 2024-03-09T09:27:02.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.947+0000 7fe8e5000640 1 --2- 172.21.15.67:0/1187239658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d8050720 0x7fe8d80062c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40097a0 tx=0x7fe8d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750e07b884036525 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.948+0000 7fe8dfe00640 1 -- 172.21.15.67:0/1187239658 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40315a0 con 0x7fe8d8050720 2024-03-09T09:27:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.948+0000 7fe8dfe00640 1 -- 172.21.15.67:0/1187239658 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4037070 con 0x7fe8d8050720 2024-03-09T09:27:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.948+0000 7fe8dfe00640 1 -- 172.21.15.67:0/1187239658 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40319a0 con 0x7fe8d8050720 2024-03-09T09:27:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.949+0000 7fe8e6e00640 1 -- 172.21.15.67:0/1187239658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d8050720 msgr2=0x7fe8d80062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.949+0000 7fe8e6e00640 1 --2- 172.21.15.67:0/1187239658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d8050720 0x7fe8d80062c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40097a0 tx=0x7fe8d402b840 comp rx=0 tx=0).stop 2024-03-09T09:27:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.950+0000 7fe8e6e00640 1 -- 172.21.15.67:0/1187239658 shutdown_connections 2024-03-09T09:27:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.950+0000 7fe8e6e00640 1 --2- 172.21.15.67:0/1187239658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8d800fcf0 0x7fe8d8006800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.950+0000 7fe8e6e00640 1 --2- 172.21.15.67:0/1187239658 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d8050720 0x7fe8d80062c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.950+0000 7fe8e6e00640 1 -- 172.21.15.67:0/1187239658 >> 172.21.15.67:0/1187239658 conn(0x7fe8d804fc00 msgr2=0x7fe8d8050010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:02.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.950+0000 7fe8e6e00640 1 -- 172.21.15.67:0/1187239658 shutdown_connections 2024-03-09T09:27:02.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.950+0000 7fe8e6e00640 1 -- 172.21.15.67:0/1187239658 wait complete. 2024-03-09T09:27:02.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.953+0000 7fe8e6e00640 1 Processor -- start 2024-03-09T09:27:02.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.953+0000 7fe8e6e00640 1 -- start start 2024-03-09T09:27:02.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.955+0000 7fe8e6e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d800fcf0 0x7fe8d8174170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.955+0000 7fe8e6e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d8050720 0x7fe8d80d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.955+0000 7fe8e6e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8d80d7920 0x7fe8d80db6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.955+0000 7fe8e6e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d800f4c0 con 0x7fe8d800fcf0 2024-03-09T09:27:02.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.955+0000 7fe8e6e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d804c2a0 con 0x7fe8d8050720 2024-03-09T09:27:02.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.955+0000 7fe8e6e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d800f640 con 0x7fe8d80d7920 2024-03-09T09:27:02.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.958+0000 7fe8e5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d8050720 0x7fe8d80d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.958+0000 7fe8e5000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d8050720 0x7fe8d80d2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:58838/0 (socket says 172.21.15.67:58838) 2024-03-09T09:27:02.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.958+0000 7fe8e5000640 1 -- 172.21.15.67:0/882951265 learned_addr learned my addr 172.21.15.67:0/882951265 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:02.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.959+0000 7fe8e6400640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8d80d7920 0x7fe8d80db6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.959+0000 7fe8e5000640 1 -- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8d80d7920 msgr2=0x7fe8d80db6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:02.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.959+0000 7fe8e5000640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8d80d7920 0x7fe8d80db6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.959+0000 7fe8e5000640 1 -- 172.21.15.67:0/882951265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d800fcf0 msgr2=0x7fe8d8174170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:02.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.959+0000 7fe8e5a00640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d800fcf0 0x7fe8d8174170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.959+0000 7fe8e5000640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d800fcf0 0x7fe8d8174170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:02.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.959+0000 7fe8e5000640 1 -- 172.21.15.67:0/882951265 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d81746b0 con 0x7fe8d8050720 2024-03-09T09:27:02.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.959+0000 7fe8e5a00640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d800fcf0 0x7fe8d8174170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:27:02.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.963+0000 7fe8e5000640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d8050720 0x7fe8d80d2e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8d402b880 tx=0x7fe8d40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.964+0000 7fe8dd600640 1 -- 172.21.15.67:0/882951265 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4032630 con 0x7fe8d8050720 2024-03-09T09:27:02.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.964+0000 7fe8dd600640 1 -- 172.21.15.67:0/882951265 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d40327f0 con 0x7fe8d8050720 2024-03-09T09:27:02.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.964+0000 7fe8dd600640 1 -- 172.21.15.67:0/882951265 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4032be0 con 0x7fe8d8050720 2024-03-09T09:27:02.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.965+0000 7fe8e6e00640 1 -- 172.21.15.67:0/882951265 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d8034310 con 0x7fe8d8050720 2024-03-09T09:27:02.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.965+0000 7fe8e6e00640 1 -- 172.21.15.67:0/882951265 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d8170730 con 0x7fe8d8050720 2024-03-09T09:27:02.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.967+0000 7fe8e6e00640 1 -- 172.21.15.67:0/882951265 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d8170240 con 0x7fe8d8050720 2024-03-09T09:27:02.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.969+0000 7fe8dd600640 1 -- 172.21.15.67:0/882951265 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe8d404c020 con 0x7fe8d8050720 2024-03-09T09:27:02.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.971+0000 7fe8dd600640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe8c006b4d0 0x7fe8c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:02.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.971+0000 7fe8dd600640 1 -- 172.21.15.67:0/882951265 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fe8d402f030 con 0x7fe8d8050720 2024-03-09T09:27:02.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.975+0000 7fe8e5a00640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe8c006b4d0 0x7fe8c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:02.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.980+0000 7fe8e5a00640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe8c006b4d0 0x7fe8c006d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe8e002d5c0 tx=0x7fe8e0076980 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:02.992 INFO:tasks.workunit.client.0.smithi067.stderr: 50K .......... .......... .......... .......... .......... 0% 793K 19s 2024-03-09T09:27:02.992 INFO:tasks.workunit.client.0.smithi067.stderr: 100K .......... .......... .......... .......... .......... 1% 25.1M 13s 2024-03-09T09:27:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:02.985+0000 7fe8dd600640 1 -- 172.21.15.67:0/882951265 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe8d4040070 con 0x7fe8d8050720 2024-03-09T09:27:03.047 INFO:tasks.workunit.client.0.smithi067.stderr: 150K .......... .......... .......... .......... .......... 1% 936K 12s 2024-03-09T09:27:03.089 INFO:tasks.workunit.client.0.smithi067.stderr: 200K .......... .......... .......... .......... .......... 2% 1.12M 12s 2024-03-09T09:27:03.089 INFO:tasks.workunit.client.0.smithi067.stderr: 250K .......... .......... .......... .......... .......... 2% 114M 10s 2024-03-09T09:27:03.091 INFO:tasks.workunit.client.0.smithi067.stderr: 300K .......... .......... .......... .......... .......... 3% 29.9M 8s 2024-03-09T09:27:03.126 INFO:tasks.workunit.client.0.smithi067.stderr: 350K .......... .......... .......... .......... .......... 3% 1.44M 8s 2024-03-09T09:27:03.132 INFO:tasks.workunit.client.0.smithi067.stderr: 400K .......... .......... .......... .......... .......... 4% 33.7M 7s 2024-03-09T09:27:03.132 INFO:tasks.workunit.client.0.smithi067.stderr: 450K .......... .......... .......... .......... .......... 4% 112M 6s 2024-03-09T09:27:03.132 INFO:tasks.workunit.client.0.smithi067.stderr: 500K .......... .......... .......... .......... .......... 5% 108M 6s 2024-03-09T09:27:03.132 INFO:tasks.workunit.client.0.smithi067.stderr: 550K .......... .......... .......... .......... .......... 5% 102M 5s 2024-03-09T09:27:03.132 INFO:tasks.workunit.client.0.smithi067.stderr: 600K .......... .......... .......... .......... .......... 6% 116M 5s 2024-03-09T09:27:03.132 INFO:tasks.workunit.client.0.smithi067.stderr: 650K .......... .......... .......... .......... .......... 6% 98.3M 4s 2024-03-09T09:27:03.132 INFO:tasks.workunit.client.0.smithi067.stderr: 700K .......... .......... .......... .......... .......... 7% 16.2M 4s 2024-03-09T09:27:03.162 INFO:tasks.workunit.client.0.smithi067.stderr: 750K .......... .......... .......... .......... .......... 7% 1.62M 4s 2024-03-09T09:27:03.170 INFO:tasks.workunit.client.0.smithi067.stderr: 800K .......... .......... .......... .......... .......... 8% 7.96M 4s 2024-03-09T09:27:03.170 INFO:tasks.workunit.client.0.smithi067.stderr: 850K .......... .......... .......... .......... .......... 8% 111M 4s 2024-03-09T09:27:03.170 INFO:tasks.workunit.client.0.smithi067.stderr: 900K .......... .......... .......... .......... .......... 9% 129M 4s 2024-03-09T09:27:03.170 INFO:tasks.workunit.client.0.smithi067.stderr: 950K .......... .......... .......... .......... .......... 9% 126M 3s 2024-03-09T09:27:03.172 INFO:tasks.workunit.client.0.smithi067.stderr: 1000K .......... .......... .......... .......... .......... 10% 17.3M 3s 2024-03-09T09:27:03.172 INFO:tasks.workunit.client.0.smithi067.stderr: 1050K .......... .......... .......... .......... .......... 10% 96.2M 3s 2024-03-09T09:27:03.179 INFO:tasks.workunit.client.0.smithi067.stderr: 1100K .......... .......... .......... .......... .......... 10% 13.3M 3s 2024-03-09T09:27:03.179 INFO:tasks.workunit.client.0.smithi067.stderr: 1150K .......... .......... .......... .......... .......... 11% 35.4M 3s 2024-03-09T09:27:03.217 INFO:tasks.workunit.client.0.smithi067.stderr: 1200K .......... .......... .......... .......... .......... 11% 1.31M 3s 2024-03-09T09:27:03.219 INFO:tasks.workunit.client.0.smithi067.stderr: 1250K .......... .......... .......... .......... .......... 12% 15.2M 3s 2024-03-09T09:27:03.220 INFO:tasks.workunit.client.0.smithi067.stderr: 1300K .......... .......... .......... .......... .......... 12% 54.0M 3s 2024-03-09T09:27:03.221 INFO:tasks.workunit.client.0.smithi067.stderr: 1350K .......... .......... .......... .......... .......... 13% 74.0M 3s 2024-03-09T09:27:03.227 INFO:tasks.workunit.client.0.smithi067.stderr: 1400K .......... .......... .......... .......... .......... 13% 20.7M 3s 2024-03-09T09:27:03.227 INFO:tasks.workunit.client.0.smithi067.stderr: 1450K .......... .......... .......... .......... .......... 14% 118M 2s 2024-03-09T09:27:03.227 INFO:tasks.workunit.client.0.smithi067.stderr: 1500K .......... .......... .......... .......... .......... 14% 115M 2s 2024-03-09T09:27:03.227 INFO:tasks.workunit.client.0.smithi067.stderr: 1550K .......... .......... .......... .......... .......... 15% 26.4M 2s 2024-03-09T09:27:03.227 INFO:tasks.workunit.client.0.smithi067.stderr: 1600K .......... .......... .......... .......... .......... 15% 81.9M 2s 2024-03-09T09:27:03.260 INFO:tasks.workunit.client.0.smithi067.stderr: 1650K .......... .......... .......... .......... .......... 16% 1.46M 2s 2024-03-09T09:27:03.265 INFO:tasks.workunit.client.0.smithi067.stderr: 1700K .......... .......... .......... .......... .......... 16% 11.7M 2s 2024-03-09T09:27:03.265 INFO:tasks.workunit.client.0.smithi067.stderr: 1750K .......... .......... .......... .......... .......... 17% 123M 2s 2024-03-09T09:27:03.269 INFO:tasks.workunit.client.0.smithi067.stderr: 1800K .......... .......... .......... .......... .......... 17% 158M 2s 2024-03-09T09:27:03.269 INFO:tasks.workunit.client.0.smithi067.stderr: 1850K .......... .......... .......... .......... .......... 18% 128M 2s 2024-03-09T09:27:03.269 INFO:tasks.workunit.client.0.smithi067.stderr: 1900K .......... .......... .......... .......... .......... 18% 27.1M 2s 2024-03-09T09:27:03.270 INFO:tasks.workunit.client.0.smithi067.stderr: 1950K .......... .......... .......... .......... .......... 19% 25.7M 2s 2024-03-09T09:27:03.270 INFO:tasks.workunit.client.0.smithi067.stderr: 2000K .......... .......... .......... .......... .......... 19% 35.2M 2s 2024-03-09T09:27:03.274 INFO:tasks.workunit.client.0.smithi067.stderr: 2050K .......... .......... .......... .......... .......... 20% 13.2M 2s 2024-03-09T09:27:03.285 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:27:03.285 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:27:03.310 INFO:tasks.workunit.client.0.smithi067.stderr: 2100K .......... .......... .......... .......... .......... 20% 1.46M 2s 2024-03-09T09:27:03.310 INFO:tasks.workunit.client.0.smithi067.stderr: 2150K .......... .......... .......... .......... .......... 20% 25.1M 2s 2024-03-09T09:27:03.310 INFO:tasks.workunit.client.0.smithi067.stderr: 2200K .......... .......... .......... .......... .......... 21% 141M 2s 2024-03-09T09:27:03.311 INFO:tasks.workunit.client.0.smithi067.stderr: 2250K .......... .......... .......... .......... .......... 21% 21.2M 2s 2024-03-09T09:27:03.314 INFO:tasks.workunit.client.0.smithi067.stderr: 2300K .......... .......... .......... .......... .......... 22% 29.2M 2s 2024-03-09T09:27:03.314 INFO:tasks.workunit.client.0.smithi067.stderr: 2350K .......... .......... .......... .......... .......... 22% 46.9M 2s 2024-03-09T09:27:03.317 INFO:tasks.workunit.client.0.smithi067.stderr: 2400K .......... .......... .......... .......... .......... 23% 21.7M 2s 2024-03-09T09:27:03.320 INFO:tasks.workunit.client.0.smithi067.stderr: 2450K .......... .......... .......... .......... .......... 23% 16.8M 2s 2024-03-09T09:27:03.320 INFO:tasks.workunit.client.0.smithi067.stderr: 2500K .......... .......... .......... .......... .......... 24% 107M 2s 2024-03-09T09:27:03.322 INFO:tasks.workunit.client.0.smithi067.stderr: 2550K .......... .......... .......... .......... .......... 24% 27.0M 2s 2024-03-09T09:27:03.357 INFO:tasks.workunit.client.0.smithi067.stderr: 2600K .......... .......... .......... .......... .......... 25% 1.40M 2s 2024-03-09T09:27:03.357 INFO:tasks.workunit.client.0.smithi067.stderr: 2650K .......... .......... .......... .......... .......... 25% 90.2M 2s 2024-03-09T09:27:03.360 INFO:tasks.workunit.client.0.smithi067.stderr: 2700K .......... .......... .......... .......... .......... 26% 14.4M 2s 2024-03-09T09:27:03.366 INFO:tasks.workunit.client.0.smithi067.stderr: 2750K .......... .......... .......... .......... .......... 26% 16.0M 2s 2024-03-09T09:27:03.366 INFO:tasks.workunit.client.0.smithi067.stderr: 2800K .......... .......... .......... .......... .......... 27% 96.2M 1s 2024-03-09T09:27:03.366 INFO:tasks.workunit.client.0.smithi067.stderr: 2850K .......... .......... .......... .......... .......... 27% 110M 1s 2024-03-09T09:27:03.366 INFO:tasks.workunit.client.0.smithi067.stderr: 2900K .......... .......... .......... .......... .......... 28% 138M 1s 2024-03-09T09:27:03.369 INFO:tasks.workunit.client.0.smithi067.stderr: 2950K .......... .......... .......... .......... .......... 28% 12.4M 1s 2024-03-09T09:27:03.369 INFO:tasks.workunit.client.0.smithi067.stderr: 3000K .......... .......... .......... .......... .......... 29% 155M 1s 2024-03-09T09:27:03.370 INFO:tasks.workunit.client.0.smithi067.stderr: 3050K .......... .......... .......... .......... .......... 29% 30.9M 1s 2024-03-09T09:27:03.403 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:03.403 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:03.403 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:27:03.405 INFO:tasks.workunit.client.0.smithi067.stderr: 3100K .......... .......... .......... .......... .......... 30% 1.48M 1s 2024-03-09T09:27:03.405 INFO:tasks.workunit.client.0.smithi067.stderr: 3150K .......... .......... .......... .......... .......... 30% 26.5M 1s 2024-03-09T09:27:03.413 INFO:tasks.workunit.client.0.smithi067.stderr: 3200K .......... .......... .......... .......... .......... 30% 12.2M 1s 2024-03-09T09:27:03.413 INFO:tasks.workunit.client.0.smithi067.stderr: 3250K .......... .......... .......... .......... .......... 31% 17.8M 1s 2024-03-09T09:27:03.416 INFO:tasks.workunit.client.0.smithi067.stderr: 3300K .......... .......... .......... .......... .......... 31% 83.7M 1s 2024-03-09T09:27:03.416 INFO:tasks.workunit.client.0.smithi067.stderr: 3350K .......... .......... .......... .......... .......... 32% 116M 1s 2024-03-09T09:27:03.419 INFO:tasks.workunit.client.0.smithi067.stderr: 3400K .......... .......... .......... .......... .......... 32% 9.53M 1s 2024-03-09T09:27:03.419 INFO:tasks.workunit.client.0.smithi067.stderr: 3450K .......... .......... .......... .......... .......... 33% 83.4M 1s 2024-03-09T09:27:03.421 INFO:tasks.workunit.client.0.smithi067.stderr: 3500K .......... .......... .......... .......... .......... 33% 75.0M 1s 2024-03-09T09:27:03.422 INFO:tasks.workunit.client.0.smithi067.stderr: 3550K .......... .......... .......... .......... .......... 34% 120M 1s 2024-03-09T09:27:03.422 INFO:tasks.workunit.client.0.smithi067.stderr: 3600K .......... .......... .......... .......... .......... 34% 32.8M 1s 2024-03-09T09:27:03.457 INFO:tasks.workunit.client.0.smithi067.stderr: 3650K .......... .......... .......... .......... .......... 35% 1.38M 1s 2024-03-09T09:27:03.460 INFO:tasks.workunit.client.0.smithi067.stderr: 3700K .......... .......... .......... .......... .......... 35% 32.3M 1s 2024-03-09T09:27:03.461 INFO:tasks.workunit.client.0.smithi067.stderr: 3750K .......... .......... .......... .......... .......... 36% 17.0M 1s 2024-03-09T09:27:03.463 INFO:tasks.workunit.client.0.smithi067.stderr: 3800K .......... .......... .......... .......... .......... 36% 110M 1s 2024-03-09T09:27:03.463 INFO:tasks.workunit.client.0.smithi067.stderr: 3850K .......... .......... .......... .......... .......... 37% 124M 1s 2024-03-09T09:27:03.464 INFO:tasks.workunit.client.0.smithi067.stderr: 3900K .......... .......... .......... .......... .......... 37% 33.5M 1s 2024-03-09T09:27:03.469 INFO:tasks.workunit.client.0.smithi067.stderr: 3950K .......... .......... .......... .......... .......... 38% 10.4M 1s 2024-03-09T09:27:03.469 INFO:tasks.workunit.client.0.smithi067.stderr: 4000K .......... .......... .......... .......... .......... 38% 118M 1s 2024-03-09T09:27:03.477 INFO:tasks.workunit.client.1.smithi134.stdout:13 2024-03-09T09:27:03.477 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:27:03.478 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 13 2024-03-09T09:27:03.478 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:27:03.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.500+0000 7fe8e6e00640 1 -- 172.21.15.67:0/882951265 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fe8d8005390 con 0x7fe8d8050720 2024-03-09T09:27:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.507+0000 7fe8dd600640 1 -- 172.21.15.67:0/882951265 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fe8d407b390 con 0x7fe8d8050720 2024-03-09T09:27:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.511+0000 7fe8e6e00640 1 -- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe8c006b4d0 msgr2=0x7fe8c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:03.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.511+0000 7fe8e6e00640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe8c006b4d0 0x7fe8c006d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe8e002d5c0 tx=0x7fe8e0076980 comp rx=0 tx=0).stop 2024-03-09T09:27:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.511+0000 7fe8e6e00640 1 -- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d8050720 msgr2=0x7fe8d80d2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.511+0000 7fe8e6e00640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d8050720 0x7fe8d80d2e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8d402b880 tx=0x7fe8d40046c0 comp rx=0 tx=0).stop 2024-03-09T09:27:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.511+0000 7fe8e6e00640 1 -- 172.21.15.67:0/882951265 shutdown_connections 2024-03-09T09:27:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.511+0000 7fe8e6e00640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe8c006b4d0 0x7fe8c006d990 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.511+0000 7fe8e6e00640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8d80d7920 0x7fe8d80db6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.511+0000 7fe8e6e00640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8d8050720 0x7fe8d80d2e80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.511+0000 7fe8e6e00640 1 --2- 172.21.15.67:0/882951265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe8d800fcf0 0x7fe8d8174170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:03.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.511+0000 7fe8e6e00640 1 -- 172.21.15.67:0/882951265 >> 172.21.15.67:0/882951265 conn(0x7fe8d804fc00 msgr2=0x7fe8d80062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:03.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.512+0000 7fe8e6e00640 1 -- 172.21.15.67:0/882951265 shutdown_connections 2024-03-09T09:27:03.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:03.512+0000 7fe8e6e00640 1 -- 172.21.15.67:0/882951265 wait complete. 2024-03-09T09:27:03.547 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:27:03.658 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:27:03.725 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:27:03.738 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:27:03.771 INFO:tasks.workunit.client.0.smithi067.stderr: 4050K .......... .......... .......... .......... .......... 39% 167K 1s 2024-03-09T09:27:03.771 INFO:tasks.workunit.client.0.smithi067.stderr: 4100K .......... .......... .......... .......... .......... 39% 106M 1s 2024-03-09T09:27:03.772 INFO:tasks.workunit.client.0.smithi067.stderr: 4150K .......... .......... .......... .......... .......... 40% 142M 1s 2024-03-09T09:27:03.772 INFO:tasks.workunit.client.0.smithi067.stderr: 4200K .......... .......... .......... .......... .......... 40% 123M 1s 2024-03-09T09:27:03.772 INFO:tasks.workunit.client.0.smithi067.stderr: 4250K .......... .......... .......... .......... .......... 40% 148M 1s 2024-03-09T09:27:03.772 INFO:tasks.workunit.client.0.smithi067.stderr: 4300K .......... .......... .......... .......... .......... 41% 146M 1s 2024-03-09T09:27:03.772 INFO:tasks.workunit.client.0.smithi067.stderr: 4350K .......... .......... .......... .......... .......... 41% 138M 1s 2024-03-09T09:27:03.772 INFO:tasks.workunit.client.0.smithi067.stderr: 4400K .......... .......... .......... .......... .......... 42% 129M 1s 2024-03-09T09:27:03.772 INFO:tasks.workunit.client.0.smithi067.stderr: 4450K .......... .......... .......... .......... .......... 42% 150M 1s 2024-03-09T09:27:03.774 INFO:tasks.workunit.client.0.smithi067.stderr: 4500K .......... .......... .......... .......... .......... 43% 35.5M 1s 2024-03-09T09:27:03.774 INFO:tasks.workunit.client.0.smithi067.stderr: 4550K .......... .......... .......... .......... .......... 43% 123M 1s 2024-03-09T09:27:03.775 INFO:tasks.workunit.client.0.smithi067.stderr: 4600K .......... .......... .......... .......... .......... 44% 127M 1s 2024-03-09T09:27:03.775 INFO:tasks.workunit.client.0.smithi067.stderr: 4650K .......... .......... .......... .......... .......... 44% 138M 1s 2024-03-09T09:27:03.775 INFO:tasks.workunit.client.0.smithi067.stderr: 4700K .......... .......... .......... .......... .......... 45% 131M 1s 2024-03-09T09:27:03.775 INFO:tasks.workunit.client.0.smithi067.stderr: 4750K .......... .......... .......... .......... .......... 45% 129M 1s 2024-03-09T09:27:03.775 INFO:tasks.workunit.client.0.smithi067.stderr: 4800K .......... .......... .......... .......... .......... 46% 113M 1s 2024-03-09T09:27:03.775 INFO:tasks.workunit.client.0.smithi067.stderr: 4850K .......... .......... .......... .......... .......... 46% 130M 1s 2024-03-09T09:27:03.812 INFO:tasks.workunit.client.0.smithi067.stderr: 4900K .......... .......... .......... .......... .......... 47% 15.7M 1s 2024-03-09T09:27:03.812 INFO:tasks.workunit.client.0.smithi067.stderr: 4950K .......... .......... .......... .......... .......... 47% 98.1M 1s 2024-03-09T09:27:03.812 INFO:tasks.workunit.client.0.smithi067.stderr: 5000K .......... .......... .......... .......... .......... 48% 128M 1s 2024-03-09T09:27:03.812 INFO:tasks.workunit.client.0.smithi067.stderr: 5050K .......... .......... .......... .......... .......... 48% 92.5M 1s 2024-03-09T09:27:03.812 INFO:tasks.workunit.client.0.smithi067.stderr: 5100K .......... .......... .......... .......... .......... 49% 70.1M 1s 2024-03-09T09:27:03.812 INFO:tasks.workunit.client.0.smithi067.stderr: 5150K .......... .......... .......... .......... .......... 49% 157M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5200K .......... .......... .......... .......... .......... 50% 135M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5250K .......... .......... .......... .......... .......... 50% 35.0M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5300K .......... .......... .......... .......... .......... 51% 19.6M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5350K .......... .......... .......... .......... .......... 51% 154M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5400K .......... .......... .......... .......... .......... 51% 135M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5450K .......... .......... .......... .......... .......... 52% 164M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5500K .......... .......... .......... .......... .......... 52% 112M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5550K .......... .......... .......... .......... .......... 53% 160M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5600K .......... .......... .......... .......... .......... 53% 32.4M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5650K .......... .......... .......... .......... .......... 54% 66.6M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5700K .......... .......... .......... .......... .......... 54% 158M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5750K .......... .......... .......... .......... .......... 55% 154M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5800K .......... .......... .......... .......... .......... 55% 115M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5850K .......... .......... .......... .......... .......... 56% 42.0M 1s 2024-03-09T09:27:03.813 INFO:tasks.workunit.client.0.smithi067.stderr: 5900K .......... .......... .......... .......... .......... 56% 147M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 5950K .......... .......... .......... .......... .......... 57% 156M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6000K .......... .......... .......... .......... .......... 57% 137M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6050K .......... .......... .......... .......... .......... 58% 152M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6100K .......... .......... .......... .......... .......... 58% 103M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6150K .......... .......... .......... .......... .......... 59% 155M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6200K .......... .......... .......... .......... .......... 59% 18.7M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6250K .......... .......... .......... .......... .......... 60% 152M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6300K .......... .......... .......... .......... .......... 60% 106M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6350K .......... .......... .......... .......... .......... 61% 126M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6400K .......... .......... .......... .......... .......... 61% 129M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6450K .......... .......... .......... .......... .......... 61% 152M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6500K .......... .......... .......... .......... .......... 62% 156M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6550K .......... .......... .......... .......... .......... 62% 53.1M 1s 2024-03-09T09:27:03.814 INFO:tasks.workunit.client.0.smithi067.stderr: 6600K .......... .......... .......... .......... .......... 63% 24.6M 1s 2024-03-09T09:27:03.815 INFO:tasks.workunit.client.0.smithi067.stderr: 6650K .......... .......... .......... .......... .......... 63% 157M 1s 2024-03-09T09:27:03.815 INFO:tasks.workunit.client.0.smithi067.stderr: 6700K .......... .......... .......... .......... .......... 64% 156M 1s 2024-03-09T09:27:03.815 INFO:tasks.workunit.client.0.smithi067.stderr: 6750K .......... .......... .......... .......... .......... 64% 166M 1s 2024-03-09T09:27:03.815 INFO:tasks.workunit.client.0.smithi067.stderr: 6800K .......... .......... .......... .......... .......... 65% 13.4M 1s 2024-03-09T09:27:03.815 INFO:tasks.workunit.client.0.smithi067.stderr: 6850K .......... .......... .......... .......... .......... 65% 9.78M 1s 2024-03-09T09:27:03.815 INFO:tasks.workunit.client.0.smithi067.stderr: 6900K .......... .......... .......... .......... .......... 66% 107M 1s 2024-03-09T09:27:03.815 INFO:tasks.workunit.client.0.smithi067.stderr: 6950K .......... .......... .......... .......... .......... 66% 161M 1s 2024-03-09T09:27:03.815 INFO:tasks.workunit.client.0.smithi067.stderr: 7000K .......... .......... .......... .......... .......... 67% 75.2M 0s 2024-03-09T09:27:03.820 INFO:tasks.workunit.client.0.smithi067.stderr: 7050K .......... .......... .......... .......... .......... 67% 8.99M 0s 2024-03-09T09:27:03.820 INFO:tasks.workunit.client.0.smithi067.stderr: 7100K .......... .......... .......... .......... .......... 68% 123M 0s 2024-03-09T09:27:03.820 INFO:tasks.workunit.client.0.smithi067.stderr: 7150K .......... .......... .......... .......... .......... 68% 96.3M 0s 2024-03-09T09:27:03.820 INFO:tasks.workunit.client.0.smithi067.stderr: 7200K .......... .......... .......... .......... .......... 69% 90.5M 0s 2024-03-09T09:27:03.822 INFO:tasks.workunit.client.0.smithi067.stderr: 7250K .......... .......... .......... .......... .......... 69% 12.9M 0s 2024-03-09T09:27:03.832 INFO:tasks.workunit.client.0.smithi067.stderr: 7300K .......... .......... .......... .......... .......... 70% 7.74M 0s 2024-03-09T09:27:03.838 INFO:tasks.workunit.client.0.smithi067.stderr: 7350K .......... .......... .......... .......... .......... 70% 13.4M 0s 2024-03-09T09:27:03.839 INFO:tasks.workunit.client.0.smithi067.stderr: 7400K .......... .......... .......... .......... .......... 71% 62.0M 0s 2024-03-09T09:27:03.839 INFO:tasks.workunit.client.0.smithi067.stderr: 7450K .......... .......... .......... .......... .......... 71% 151M 0s 2024-03-09T09:27:03.840 INFO:tasks.workunit.client.0.smithi067.stderr: 7500K .......... .......... .......... .......... .......... 71% 7.07M 0s 2024-03-09T09:27:03.843 INFO:tasks.workunit.client.0.smithi067.stderr: 7550K .......... .......... .......... .......... .......... 72% 16.1M 0s 2024-03-09T09:27:03.847 INFO:tasks.workunit.client.0.smithi067.stderr: 7600K .......... .......... .......... .......... .......... 72% 16.1M 0s 2024-03-09T09:27:03.847 INFO:tasks.workunit.client.0.smithi067.stderr: 7650K .......... .......... .......... .......... .......... 73% 85.2M 0s 2024-03-09T09:27:03.847 INFO:tasks.workunit.client.0.smithi067.stderr: 7700K .......... .......... .......... .......... .......... 73% 128M 0s 2024-03-09T09:27:03.851 INFO:tasks.workunit.client.0.smithi067.stderr: 7750K .......... .......... .......... .......... .......... 74% 20.6M 0s 2024-03-09T09:27:03.851 INFO:tasks.workunit.client.0.smithi067.stderr: 7800K .......... .......... .......... .......... .......... 74% 125M 0s 2024-03-09T09:27:03.853 INFO:tasks.workunit.client.0.smithi067.stderr: 7850K .......... .......... .......... .......... .......... 75% 22.8M 0s 2024-03-09T09:27:03.857 INFO:tasks.workunit.client.0.smithi067.stderr: 7900K .......... .......... .......... .......... .......... 75% 12.3M 0s 2024-03-09T09:27:03.858 INFO:tasks.workunit.client.0.smithi067.stderr: 7950K .......... .......... .......... .......... .......... 76% 129M 0s 2024-03-09T09:27:03.858 INFO:tasks.workunit.client.0.smithi067.stderr: 8000K .......... .......... .......... .......... .......... 76% 124M 0s 2024-03-09T09:27:03.881 INFO:tasks.workunit.client.0.smithi067.stderr: 8050K .......... .......... .......... .......... .......... 77% 5.86M 0s 2024-03-09T09:27:03.881 INFO:tasks.workunit.client.0.smithi067.stderr: 8100K .......... .......... .......... .......... .......... 77% 20.6M 0s 2024-03-09T09:27:03.918 INFO:tasks.workunit.client.0.smithi067.stderr: 8150K .......... .......... .......... .......... .......... 78% 1.12M 0s 2024-03-09T09:27:03.918 INFO:tasks.workunit.client.0.smithi067.stderr: 8200K .......... .......... .......... .......... .......... 78% 120M 0s 2024-03-09T09:27:03.918 INFO:tasks.workunit.client.0.smithi067.stderr: 8250K .......... .......... .......... .......... .......... 79% 116M 0s 2024-03-09T09:27:03.918 INFO:tasks.workunit.client.0.smithi067.stderr: 8300K .......... .......... .......... .......... .......... 79% 146M 0s 2024-03-09T09:27:03.918 INFO:tasks.workunit.client.0.smithi067.stderr: 8350K .......... .......... .......... .......... .......... 80% 149M 0s 2024-03-09T09:27:03.918 INFO:tasks.workunit.client.0.smithi067.stderr: 8400K .......... .......... .......... .......... .......... 80% 144M 0s 2024-03-09T09:27:03.918 INFO:tasks.workunit.client.0.smithi067.stderr: 8450K .......... .......... .......... .......... .......... 81% 25.8M 0s 2024-03-09T09:27:03.918 INFO:tasks.workunit.client.0.smithi067.stderr: 8500K .......... .......... .......... .......... .......... 81% 151M 0s 2024-03-09T09:27:03.918 INFO:tasks.workunit.client.0.smithi067.stderr: 8550K .......... .......... .......... .......... .......... 81% 149M 0s 2024-03-09T09:27:03.920 INFO:tasks.workunit.client.0.smithi067.stderr: 8600K .......... .......... .......... .......... .......... 82% 32.5M 0s 2024-03-09T09:27:03.920 INFO:tasks.workunit.client.0.smithi067.stderr: 8650K .......... .......... .......... .......... .......... 82% 110M 0s 2024-03-09T09:27:03.920 INFO:tasks.workunit.client.0.smithi067.stderr: 8700K .......... .......... .......... .......... .......... 83% 149M 0s 2024-03-09T09:27:03.920 INFO:tasks.workunit.client.0.smithi067.stderr: 8750K .......... .......... .......... .......... .......... 83% 154M 0s 2024-03-09T09:27:03.921 INFO:tasks.workunit.client.0.smithi067.stderr: 8800K .......... .......... .......... .......... .......... 84% 31.9M 0s 2024-03-09T09:27:03.921 INFO:tasks.workunit.client.0.smithi067.stderr: 8850K .......... .......... .......... .......... .......... 84% 123M 0s 2024-03-09T09:27:03.921 INFO:tasks.workunit.client.0.smithi067.stderr: 8900K .......... .......... .......... .......... .......... 85% 148M 0s 2024-03-09T09:27:03.921 INFO:tasks.workunit.client.0.smithi067.stderr: 8950K .......... .......... .......... .......... .......... 85% 150M 0s 2024-03-09T09:27:03.939 INFO:tasks.workunit.client.0.smithi067.stderr: 9000K .......... .......... .......... .......... .......... 86% 3.17M 0s 2024-03-09T09:27:03.940 INFO:tasks.workunit.client.0.smithi067.stderr: 9050K .......... .......... .......... .......... .......... 86% 106M 0s 2024-03-09T09:27:03.940 INFO:tasks.workunit.client.0.smithi067.stderr: 9100K .......... .......... .......... .......... .......... 87% 116M 0s 2024-03-09T09:27:03.940 INFO:tasks.workunit.client.0.smithi067.stderr: 9150K .......... .......... .......... .......... .......... 87% 109M 0s 2024-03-09T09:27:03.940 INFO:tasks.workunit.client.0.smithi067.stderr: 9200K .......... .......... .......... .......... .......... 88% 128M 0s 2024-03-09T09:27:03.940 INFO:tasks.workunit.client.0.smithi067.stderr: 9250K .......... .......... .......... .......... .......... 88% 128M 0s 2024-03-09T09:27:03.940 INFO:tasks.workunit.client.0.smithi067.stderr: 9300K .......... .......... .......... .......... .......... 89% 137M 0s 2024-03-09T09:27:03.946 INFO:tasks.workunit.client.0.smithi067.stderr: 9350K .......... .......... .......... .......... .......... 89% 59.6M 0s 2024-03-09T09:27:03.946 INFO:tasks.workunit.client.0.smithi067.stderr: 9400K .......... .......... .......... .......... .......... 90% 45.4M 0s 2024-03-09T09:27:03.946 INFO:tasks.workunit.client.0.smithi067.stderr: 9450K .......... .......... .......... .......... .......... 90% 27.3M 0s 2024-03-09T09:27:03.946 INFO:tasks.workunit.client.0.smithi067.stderr: 9500K .......... .......... .......... .......... .......... 91% 24.7M 0s 2024-03-09T09:27:03.947 INFO:tasks.workunit.client.0.smithi067.stderr: 9550K .......... .......... .......... .......... .......... 91% 90.4M 0s 2024-03-09T09:27:03.948 INFO:tasks.workunit.client.0.smithi067.stderr: 9600K .......... .......... .......... .......... .......... 92% 19.5M 0s 2024-03-09T09:27:03.948 INFO:tasks.workunit.client.0.smithi067.stderr: 9650K .......... .......... .......... .......... .......... 92% 139M 0s 2024-03-09T09:27:03.951 INFO:tasks.workunit.client.0.smithi067.stderr: 9700K .......... .......... .......... .......... .......... 92% 25.6M 0s 2024-03-09T09:27:03.959 INFO:tasks.workunit.client.0.smithi067.stderr: 9750K .......... .......... .......... .......... .......... 93% 10.8M 0s 2024-03-09T09:27:03.960 INFO:tasks.workunit.client.0.smithi067.stderr: 9800K .......... .......... .......... .......... .......... 93% 10.5M 0s 2024-03-09T09:27:03.977 INFO:tasks.workunit.client.0.smithi067.stderr: 9850K .......... .......... .......... .......... .......... 94% 3.07M 0s 2024-03-09T09:27:03.978 INFO:tasks.workunit.client.0.smithi067.stderr: 9900K .......... .......... .......... .......... .......... 94% 20.3M 0s 2024-03-09T09:27:03.978 INFO:tasks.workunit.client.0.smithi067.stderr: 9950K .......... .......... .......... .......... .......... 95% 93.1M 0s 2024-03-09T09:27:03.979 INFO:tasks.workunit.client.0.smithi067.stderr: 10000K .......... .......... .......... .......... .......... 95% 136M 0s 2024-03-09T09:27:03.979 INFO:tasks.workunit.client.0.smithi067.stderr: 10050K .......... .......... .......... .......... .......... 96% 152M 0s 2024-03-09T09:27:03.979 INFO:tasks.workunit.client.0.smithi067.stderr: 10100K .......... .......... .......... .......... .......... 96% 115M 0s 2024-03-09T09:27:03.983 INFO:tasks.workunit.client.0.smithi067.stderr: 10150K .......... .......... .......... .......... .......... 97% 19.9M 0s 2024-03-09T09:27:03.984 INFO:tasks.workunit.client.0.smithi067.stderr: 10200K .......... .......... .......... .......... .......... 97% 158M 0s 2024-03-09T09:27:03.984 INFO:tasks.workunit.client.0.smithi067.stderr: 10250K .......... .......... .......... .......... .......... 98% 155M 0s 2024-03-09T09:27:03.984 INFO:tasks.workunit.client.0.smithi067.stderr: 10300K .......... .......... .......... .......... .......... 98% 133M 0s 2024-03-09T09:27:03.984 INFO:tasks.workunit.client.0.smithi067.stderr: 10350K .......... .......... .......... .......... .......... 99% 134M 0s 2024-03-09T09:27:03.984 INFO:tasks.workunit.client.0.smithi067.stderr: 10400K .......... .......... .......... .......... .......... 99% 140M 0s 2024-03-09T09:27:03.984 INFO:tasks.workunit.client.0.smithi067.stderr: 10450K .......... .......... .......... ......... 100% 124M=1.2s 2024-03-09T09:27:03.984 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-09T09:27:03.984 INFO:tasks.workunit.client.0.smithi067.stderr:2024-03-09 09:27:03 (8.71 MB/s) - ‘coreutils_8.5.orig.tar.gz’ saved [10740866/10740866] 2024-03-09T09:27:03.984 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-09T09:27:03.985 INFO:tasks.workunit.client.0.smithi067.stderr:+ tar xvfz coreutils_8.5.orig.tar.gz 2024-03-09T09:27:04.009 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/ 2024-03-09T09:27:04.010 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/maint.mk 2024-03-09T09:27:04.014 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/THANKS 2024-03-09T09:27:04.015 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/ABOUT-NLS 2024-03-09T09:27:04.016 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/ 2024-03-09T09:27:04.021 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/textutils/ 2024-03-09T09:27:04.028 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/textutils/ChangeLog 2024-03-09T09:27:04.032 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/textutils/NEWS 2024-03-09T09:27:04.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/fileutils/ 2024-03-09T09:27:04.037 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/fileutils/ChangeLog 2024-03-09T09:27:04.041 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/fileutils/ChangeLog-1997 2024-03-09T09:27:04.042 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/fileutils/NEWS 2024-03-09T09:27:04.043 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/sh-utils/ 2024-03-09T09:27:04.045 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/sh-utils/ChangeLog 2024-03-09T09:27:04.053 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/sh-utils/NEWS 2024-03-09T09:27:04.054 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/old/sh-utils/ChangeLog.0 2024-03-09T09:27:04.054 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_prohibit_strcmp 2024-03-09T09:27:04.055 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/ 2024-03-09T09:27:04.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/config.rpath 2024-03-09T09:27:04.331 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/cvsu 2024-03-09T09:27:04.331 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/mdate-sh 2024-03-09T09:27:04.331 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/gitlog-to-changelog 2024-03-09T09:27:04.331 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/gnu-web-doc-update 2024-03-09T09:27:04.332 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/useless-if-before-free 2024-03-09T09:27:04.332 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/config.sub 2024-03-09T09:27:04.333 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/missing 2024-03-09T09:27:04.333 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/compile 2024-03-09T09:27:04.333 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/install-sh 2024-03-09T09:27:04.333 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/gendocs.sh 2024-03-09T09:27:04.334 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/update-copyright 2024-03-09T09:27:04.334 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/texinfo.tex 2024-03-09T09:27:04.337 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/unused-parameter.h 2024-03-09T09:27:04.337 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/c++defs.h 2024-03-09T09:27:04.338 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/config.guess 2024-03-09T09:27:04.338 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/do-release-commit-and-tag 2024-03-09T09:27:04.338 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/git-version-gen 2024-03-09T09:27:04.339 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/arg-nonnull.h 2024-03-09T09:27:04.340 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/gnupload 2024-03-09T09:27:04.340 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/warn-on-use.h 2024-03-09T09:27:04.341 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/vc-list-files 2024-03-09T09:27:04.341 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/ylwrap 2024-03-09T09:27:04.342 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/ChangeLog-2007 2024-03-09T09:27:04.342 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/announce-gen 2024-03-09T09:27:04.342 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/build-aux/depcomp 2024-03-09T09:27:04.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_error_message_uppercase 2024-03-09T09:27:04.344 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_obsolete_symbols 2024-03-09T09:27:04.345 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/INSTALL 2024-03-09T09:27:04.346 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gl/ 2024-03-09T09:27:04.412 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gl/modules/ 2024-03-09T09:27:04.445 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gl/modules/getloadavg.diff 2024-03-09T09:27:04.452 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/configure.ac 2024-03-09T09:27:04.452 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_prohibit_tab_based_indentation 2024-03-09T09:27:04.452 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/GNUmakefile 2024-03-09T09:27:04.453 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/ChangeLog 2024-03-09T09:27:04.455 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/COPYING 2024-03-09T09:27:04.456 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/ 2024-03-09T09:27:04.472 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/closeout.m4 2024-03-09T09:27:04.484 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/dirent-safer.m4 2024-03-09T09:27:04.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/settime.m4 2024-03-09T09:27:04.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/readtokens.m4 2024-03-09T09:27:04.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/clock_time.m4 2024-03-09T09:27:04.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/config-h.m4 2024-03-09T09:27:04.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/euidaccess.m4 2024-03-09T09:27:04.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/printf-frexpl.m4 2024-03-09T09:27:04.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/posix-shell.m4 2024-03-09T09:27:04.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/servent.m4 2024-03-09T09:27:04.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/xstrtol.m4 2024-03-09T09:27:04.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mkdir-p.m4 2024-03-09T09:27:04.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/00gnulib.m4 2024-03-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mbsinit.m4 2024-03-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/dup2.m4 2024-03-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/eealloc.m4 2024-03-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/savewd.m4 2024-03-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/memrchr.m4 2024-03-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/d-ino.m4 2024-03-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/bison.m4 2024-03-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/isnanl.m4 2024-03-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/lib-ld.m4 2024-03-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stpcpy.m4 2024-03-09T09:27:04.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/progtest.m4 2024-03-09T09:27:04.491 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/unistd_h.m4 2024-03-09T09:27:04.492 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/time_h.m4 2024-03-09T09:27:04.492 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mkancesdirs.m4 2024-03-09T09:27:04.493 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/signalblocking.m4 2024-03-09T09:27:04.493 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/unlink.m4 2024-03-09T09:27:04.493 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/backupfile.m4 2024-03-09T09:27:04.493 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/remove.m4 2024-03-09T09:27:04.493 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fts.m4 2024-03-09T09:27:04.494 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/xalloc.m4 2024-03-09T09:27:04.494 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/inet_ntop.m4 2024-03-09T09:27:04.494 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/iconv_open.m4 2024-03-09T09:27:04.495 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/extensions.m4 2024-03-09T09:27:04.495 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mbrtowc.m4 2024-03-09T09:27:04.495 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/check-decl.m4 2024-03-09T09:27:04.496 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/cycle-check.m4 2024-03-09T09:27:04.496 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/signal_h.m4 2024-03-09T09:27:04.496 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sys_socket_h.m4 2024-03-09T09:27:04.497 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fprintftime.m4 2024-03-09T09:27:04.497 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/lib-check.m4 2024-03-09T09:27:04.497 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/isblank.m4 2024-03-09T09:27:04.499 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/xstrtod.m4 2024-03-09T09:27:04.499 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stdarg.m4 2024-03-09T09:27:04.499 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/selinux-selinux-h.m4 2024-03-09T09:27:04.499 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fstypename.m4 2024-03-09T09:27:04.499 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/link.m4 2024-03-09T09:27:04.500 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/md5.m4 2024-03-09T09:27:04.500 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/multiarch.m4 2024-03-09T09:27:04.500 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/ioctl.m4 2024-03-09T09:27:04.501 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/flexmember.m4 2024-03-09T09:27:04.501 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/locale-fr.m4 2024-03-09T09:27:04.501 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fdopendir.m4 2024-03-09T09:27:04.502 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/exponentf.m4 2024-03-09T09:27:04.502 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/utimecmp.m4 2024-03-09T09:27:04.502 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/human.m4 2024-03-09T09:27:04.502 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strerror.m4 2024-03-09T09:27:04.504 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/socklen.m4 2024-03-09T09:27:04.504 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/dirent_h.m4 2024-03-09T09:27:04.504 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/locale-zh.m4 2024-03-09T09:27:04.504 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getdate.m4 2024-03-09T09:27:04.504 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/cloexec.m4 2024-03-09T09:27:04.505 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/i-ring.m4 2024-03-09T09:27:04.505 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/group-member.m4 2024-03-09T09:27:04.505 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strtod.m4 2024-03-09T09:27:04.505 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/chdir-long.m4 2024-03-09T09:27:04.505 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strnlen.m4 2024-03-09T09:27:04.505 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strtoimax.m4 2024-03-09T09:27:04.505 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getgroups.m4 2024-03-09T09:27:04.505 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/yield.m4 2024-03-09T09:27:04.505 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/write.m4 2024-03-09T09:27:04.506 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/localcharset.m4 2024-03-09T09:27:04.507 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/usleep.m4 2024-03-09T09:27:04.507 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/memmove.m4 2024-03-09T09:27:04.507 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/close.m4 2024-03-09T09:27:04.507 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/memcmp.m4 2024-03-09T09:27:04.508 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/autobuild.m4 2024-03-09T09:27:04.508 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fchdir.m4 2024-03-09T09:27:04.508 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fd-reopen.m4 2024-03-09T09:27:04.509 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/jm-winsz2.m4 2024-03-09T09:27:04.509 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/posixver.m4 2024-03-09T09:27:04.509 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/locale-ja.m4 2024-03-09T09:27:04.510 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stat-prog.m4 2024-03-09T09:27:04.510 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/wint_t.m4 2024-03-09T09:27:04.511 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/st_dm_mode.m4 2024-03-09T09:27:04.511 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/tempname.m4 2024-03-09T09:27:04.511 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/frexp.m4 2024-03-09T09:27:04.512 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/dirfd.m4 2024-03-09T09:27:04.512 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sys_select_h.m4 2024-03-09T09:27:04.512 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/btowc.m4 2024-03-09T09:27:04.512 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/filenamecat.m4 2024-03-09T09:27:04.512 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/gmp.m4 2024-03-09T09:27:04.513 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/isapipe.m4 2024-03-09T09:27:04.513 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/wctype_h.m4 2024-03-09T09:27:04.514 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/ldexpl.m4 2024-03-09T09:27:04.514 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getloadavg.m4 2024-03-09T09:27:04.514 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mbchar.m4 2024-03-09T09:27:04.514 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/version-etc.m4 2024-03-09T09:27:04.514 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fcntl_h.m4 2024-03-09T09:27:04.514 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/gettime.m4 2024-03-09T09:27:04.514 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/wchar_h.m4 2024-03-09T09:27:04.515 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/vasprintf-posix.m4 2024-03-09T09:27:04.515 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/selinux-context-h.m4 2024-03-09T09:27:04.515 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getpass.m4 2024-03-09T09:27:04.515 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mempcpy.m4 2024-03-09T09:27:04.515 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/save-cwd.m4 2024-03-09T09:27:04.516 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fclose.m4 2024-03-09T09:27:04.516 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/modechange.m4 2024-03-09T09:27:04.516 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strstr.m4 2024-03-09T09:27:04.517 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getdtablesize.m4 2024-03-09T09:27:04.517 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/wchar_t.m4 2024-03-09T09:27:04.518 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mkdir.m4 2024-03-09T09:27:04.518 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/idcache.m4 2024-03-09T09:27:04.519 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/tm_gmtoff.m4 2024-03-09T09:27:04.521 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/dirname.m4 2024-03-09T09:27:04.522 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mbiter.m4 2024-03-09T09:27:04.522 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fcntl-safer.m4 2024-03-09T09:27:04.522 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/manywarnings.m4 2024-03-09T09:27:04.522 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/same.m4 2024-03-09T09:27:04.522 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getdelim.m4 2024-03-09T09:27:04.522 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mountlist.m4 2024-03-09T09:27:04.522 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/lchmod.m4 2024-03-09T09:27:04.522 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/readlink.m4 2024-03-09T09:27:04.522 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/gnu-make.m4 2024-03-09T09:27:04.524 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/thread.m4 2024-03-09T09:27:04.524 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/hostent.m4 2024-03-09T09:27:04.524 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/math_h.m4 2024-03-09T09:27:04.524 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fpending.m4 2024-03-09T09:27:04.525 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/xgetcwd.m4 2024-03-09T09:27:04.525 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/utimens.m4 2024-03-09T09:27:04.525 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/exponentd.m4 2024-03-09T09:27:04.525 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getaddrinfo.m4 2024-03-09T09:27:04.525 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sys_ioctl_h.m4 2024-03-09T09:27:04.525 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stdlib_h.m4 2024-03-09T09:27:04.525 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mgetgroups.m4 2024-03-09T09:27:04.525 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/open.m4 2024-03-09T09:27:04.526 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strpbrk.m4 2024-03-09T09:27:04.526 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/ssize_t.m4 2024-03-09T09:27:04.526 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/priv-set.m4 2024-03-09T09:27:04.526 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fcntl-o.m4 2024-03-09T09:27:04.527 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/inttostr.m4 2024-03-09T09:27:04.527 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/malloca.m4 2024-03-09T09:27:04.527 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/unlocked-io.m4 2024-03-09T09:27:04.527 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getugroups.m4 2024-03-09T09:27:04.528 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sha1.m4 2024-03-09T09:27:04.528 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fpurge.m4 2024-03-09T09:27:04.529 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/gnulib-comp.m4 2024-03-09T09:27:04.530 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/wcrtomb.m4 2024-03-09T09:27:04.530 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/putenv.m4 2024-03-09T09:27:04.531 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/lstat.m4 2024-03-09T09:27:04.531 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strsignal.m4 2024-03-09T09:27:04.531 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mbslen.m4 2024-03-09T09:27:04.531 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/inttypes.m4 2024-03-09T09:27:04.532 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stdlib-safer.m4 2024-03-09T09:27:04.532 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strtol.m4 2024-03-09T09:27:04.532 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/lib-ignore.m4 2024-03-09T09:27:04.533 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/readutmp.m4 2024-03-09T09:27:04.533 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strtoll.m4 2024-03-09T09:27:04.533 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stdbool.m4 2024-03-09T09:27:04.533 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/setenv.m4 2024-03-09T09:27:04.534 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/alloca.m4 2024-03-09T09:27:04.534 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/size_max.m4 2024-03-09T09:27:04.534 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/include_next.m4 2024-03-09T09:27:04.535 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sys_stat_h.m4 2024-03-09T09:27:04.535 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strndup.m4 2024-03-09T09:27:04.536 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/link-follow.m4 2024-03-09T09:27:04.536 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/ctype.m4 2024-03-09T09:27:04.536 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/timespec.m4 2024-03-09T09:27:04.536 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getline.m4 2024-03-09T09:27:04.537 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/calloc.m4 2024-03-09T09:27:04.537 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strftime.m4 2024-03-09T09:27:04.537 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/arpa_inet_h.m4 2024-03-09T09:27:04.537 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sig2str.m4 2024-03-09T09:27:04.537 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/printf.m4 2024-03-09T09:27:04.538 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/environ.m4 2024-03-09T09:27:04.539 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/vprintf-posix.m4 2024-03-09T09:27:04.540 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/intlmacosx.m4 2024-03-09T09:27:04.540 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/locale-tr.m4 2024-03-09T09:27:04.540 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mktime.m4 2024-03-09T09:27:04.540 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/iconv.m4 2024-03-09T09:27:04.541 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/pathmax.m4 2024-03-09T09:27:04.541 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/printf-frexp.m4 2024-03-09T09:27:04.542 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/userspec.m4 2024-03-09T09:27:04.542 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/host-os.m4 2024-03-09T09:27:04.544 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/file-type.m4 2024-03-09T09:27:04.544 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/ls-mntd-fs.m4 2024-03-09T09:27:04.544 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sha512.m4 2024-03-09T09:27:04.545 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stdio-safer.m4 2024-03-09T09:27:04.545 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/lock.m4 2024-03-09T09:27:04.545 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/c-strtod.m4 2024-03-09T09:27:04.545 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/closein.m4 2024-03-09T09:27:04.545 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/frexpl.m4 2024-03-09T09:27:04.552 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/posixtm.m4 2024-03-09T09:27:04.552 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/rmdir.m4 2024-03-09T09:27:04.552 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/chown.m4 2024-03-09T09:27:04.552 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stdint.m4 2024-03-09T09:27:04.554 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stddef_h.m4 2024-03-09T09:27:04.554 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sigaction.m4 2024-03-09T09:27:04.554 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/error.m4 2024-03-09T09:27:04.554 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/vasnprintf.m4 2024-03-09T09:27:04.554 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/utimbuf.m4 2024-03-09T09:27:04.554 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sockets.m4 2024-03-09T09:27:04.554 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/canonicalize.m4 2024-03-09T09:27:04.554 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sys_time_h.m4 2024-03-09T09:27:04.554 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/unlink-busy.m4 2024-03-09T09:27:04.556 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/gettimeofday.m4 2024-03-09T09:27:04.556 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/write-any-file.m4 2024-03-09T09:27:04.556 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strdup.m4 2024-03-09T09:27:04.556 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/regex.m4 2024-03-09T09:27:04.556 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sleep.m4 2024-03-09T09:27:04.556 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/float_h.m4 2024-03-09T09:27:04.556 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/warn-on-use.m4 2024-03-09T09:27:04.556 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/atexit.m4 2024-03-09T09:27:04.557 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getcwd-path-max.m4 2024-03-09T09:27:04.557 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fnmatch.m4 2024-03-09T09:27:04.557 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strnumcmp.m4 2024-03-09T09:27:04.557 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/gnulib-common.m4 2024-03-09T09:27:04.557 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/codeset.m4 2024-03-09T09:27:04.557 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/time_r.m4 2024-03-09T09:27:04.557 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/memcoll.m4 2024-03-09T09:27:04.557 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/string_h.m4 2024-03-09T09:27:04.558 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sys_utsname_h.m4 2024-03-09T09:27:04.558 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stdint_h.m4 2024-03-09T09:27:04.558 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/nproc.m4 2024-03-09T09:27:04.558 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/symlink.m4 2024-03-09T09:27:04.558 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/rename.m4 2024-03-09T09:27:04.558 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/read-file.m4 2024-03-09T09:27:04.561 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strtoumax.m4 2024-03-09T09:27:04.561 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getopt.m4 2024-03-09T09:27:04.561 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fopen.m4 2024-03-09T09:27:04.561 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/ungetc.m4 2024-03-09T09:27:04.561 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/xsize.m4 2024-03-09T09:27:04.561 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fflush.m4 2024-03-09T09:27:04.561 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/acl.m4 2024-03-09T09:27:04.561 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fsusage.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/gethrxtime.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/nls.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/lib-link.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mpsort.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/netinet_in_h.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fseeko.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/inet_pton.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/yesno.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/physmem.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/intmax_t.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/inttypes-pri.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/malloc.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/jm-winsz1.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/close-stream.m4 2024-03-09T09:27:04.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/nocrash.m4 2024-03-09T09:27:04.563 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/realloc.m4 2024-03-09T09:27:04.569 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/canon-host.m4 2024-03-09T09:27:04.569 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/memcpy.m4 2024-03-09T09:27:04.569 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/tls.m4 2024-03-09T09:27:04.569 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/perl.m4 2024-03-09T09:27:04.569 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/signbit.m4 2024-03-09T09:27:04.569 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/memchr.m4 2024-03-09T09:27:04.569 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/d-type.m4 2024-03-09T09:27:04.570 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/lchown.m4 2024-03-09T09:27:04.570 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/lib-prefix.m4 2024-03-09T09:27:04.570 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getcwd.m4 2024-03-09T09:27:04.570 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/locale_h.m4 2024-03-09T09:27:04.570 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/memxfrm.m4 2024-03-09T09:27:04.570 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mbstate_t.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/filemode.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/memset.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/gethostname.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/vasprintf.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/unistd-safer.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/xfts.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/uptime.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/prereq.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/symlinkat.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getusershell.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/isnand.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/select.m4 2024-03-09T09:27:04.571 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/wctob.m4 2024-03-09T09:27:04.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/double-slash-root.m4 2024-03-09T09:27:04.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/long-options.m4 2024-03-09T09:27:04.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/glibc21.m4 2024-03-09T09:27:04.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stat-time.m4 2024-03-09T09:27:04.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/rpmatch.m4 2024-03-09T09:27:04.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/xattr.m4 2024-03-09T09:27:04.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/gettext.m4 2024-03-09T09:27:04.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fsync.m4 2024-03-09T09:27:04.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/exclude.m4 2024-03-09T09:27:04.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/freopen.m4 2024-03-09T09:27:04.573 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strtoul.m4 2024-03-09T09:27:04.573 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/netdb_h.m4 2024-03-09T09:27:04.573 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/snprintf.m4 2024-03-09T09:27:04.573 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stpncpy.m4 2024-03-09T09:27:04.573 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/xnanosleep.m4 2024-03-09T09:27:04.573 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/warnings.m4 2024-03-09T09:27:04.573 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/lseek.m4 2024-03-09T09:27:04.573 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/hard-locale.m4 2024-03-09T09:27:04.575 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/include-exclude-prog.m4 2024-03-09T09:27:04.575 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mmap-anon.m4 2024-03-09T09:27:04.575 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mkstemp.m4 2024-03-09T09:27:04.575 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/openat.m4 2024-03-09T09:27:04.580 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mbrlen.m4 2024-03-09T09:27:04.580 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/jm-macros.m4 2024-03-09T09:27:04.580 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/ftello.m4 2024-03-09T09:27:04.580 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stdio_h.m4 2024-03-09T09:27:04.580 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/exponentl.m4 2024-03-09T09:27:04.580 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getpagesize.m4 2024-03-09T09:27:04.580 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/argmatch.m4 2024-03-09T09:27:04.580 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/perror.m4 2024-03-09T09:27:04.580 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/inline.m4 2024-03-09T09:27:04.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/inttypes_h.m4 2024-03-09T09:27:04.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/iconv_h.m4 2024-03-09T09:27:04.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/base64.m4 2024-03-09T09:27:04.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/stat.m4 2024-03-09T09:27:04.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/tzset.m4 2024-03-09T09:27:04.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/unicodeio.m4 2024-03-09T09:27:04.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sha256.m4 2024-03-09T09:27:04.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/errno_h.m4 2024-03-09T09:27:04.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/unlinkdir.m4 2024-03-09T09:27:04.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/boottime.m4 2024-03-09T09:27:04.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/freading.m4 2024-03-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mode_t.m4 2024-03-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getndelim2.m4 2024-03-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/xvasprintf.m4 2024-03-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mbsrtowcs.m4 2024-03-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/hash.m4 2024-03-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/safe-write.m4 2024-03-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/po.m4 2024-03-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/isnanf.m4 2024-03-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/longlong.m4 2024-03-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/xstrndup.m4 2024-03-09T09:27:04.582 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/euidaccess-stat.m4 2024-03-09T09:27:04.583 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/getcwd-abort-bug.m4 2024-03-09T09:27:04.583 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/faccessat.m4 2024-03-09T09:27:04.583 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/nanosleep.m4 2024-03-09T09:27:04.583 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/ChangeLog-2007 2024-03-09T09:27:04.584 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/utimes.m4 2024-03-09T09:27:04.584 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fileblocks.m4 2024-03-09T09:27:04.584 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/sockpfaf.m4 2024-03-09T09:27:04.584 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strtoull.m4 2024-03-09T09:27:04.585 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/assert.m4 2024-03-09T09:27:04.585 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/quotearg.m4 2024-03-09T09:27:04.585 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/uname.m4 2024-03-09T09:27:04.585 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/savedir.m4 2024-03-09T09:27:04.585 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/threadlib.m4 2024-03-09T09:27:04.585 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/langinfo_h.m4 2024-03-09T09:27:04.585 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/linkat.m4 2024-03-09T09:27:05.306 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fpieee.m4 2024-03-09T09:27:05.985 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:27:05.985 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:27:06.154 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:06.155 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:06.155 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:27:06.229 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:27:06.229 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 14 2024-03-09T09:27:06.229 INFO:tasks.workunit.client.1.smithi134.stdout:14 2024-03-09T09:27:06.230 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:27:06.474 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:27:06.541 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:27:06.555 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:27:07.601 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:27:07.602 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:27:07.753 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:07.753 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:07.754 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:27:07.818 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:27:07.818 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 15 2024-03-09T09:27:07.818 INFO:tasks.workunit.client.1.smithi134.stdout:15 2024-03-09T09:27:07.819 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:27:08.307 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:27:08.340 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/memcasecmp.m4 2024-03-09T09:27:08.340 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/vfprintf-posix.m4 2024-03-09T09:27:08.340 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/dos.m4 2024-03-09T09:27:08.341 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/root-dev-ino.m4 2024-03-09T09:27:08.341 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/quote.m4 2024-03-09T09:27:08.342 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/mbswidth.m4 2024-03-09T09:27:08.342 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/nl_langinfo.m4 2024-03-09T09:27:08.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/fcntl.m4 2024-03-09T09:27:08.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/strcspn.m4 2024-03-09T09:27:08.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/safe-read.m4 2024-03-09T09:27:08.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/m4/wcwidth.m4 2024-03-09T09:27:08.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/Makefile.am 2024-03-09T09:27:08.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_po_check 2024-03-09T09:27:08.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/dist-check.mk 2024-03-09T09:27:08.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_GPL_version 2024-03-09T09:27:08.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_unmarked_diagnostics 2024-03-09T09:27:08.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.prev-version 2024-03-09T09:27:08.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/bootstrap.conf 2024-03-09T09:27:08.344 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/ChangeLog-2005 2024-03-09T09:27:08.348 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/aclocal.m4 2024-03-09T09:27:08.348 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/AUTHORS 2024-03-09T09:27:08.348 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/cfg.mk 2024-03-09T09:27:08.348 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_file_system 2024-03-09T09:27:08.348 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.version 2024-03-09T09:27:08.348 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/TODO 2024-03-09T09:27:08.348 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_space_tab 2024-03-09T09:27:08.348 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_prohibit_empty_lines_at_EOF 2024-03-09T09:27:08.349 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/Makefile.in 2024-03-09T09:27:08.349 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:27:08.349 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/ 2024-03-09T09:27:08.363 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:27:08.372 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/cat.x 2024-03-09T09:27:08.386 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/date.x 2024-03-09T09:27:08.391 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/basename.1 2024-03-09T09:27:08.391 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/pathchk.x 2024-03-09T09:27:08.391 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tail.x 2024-03-09T09:27:08.391 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/yes.x 2024-03-09T09:27:08.391 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/pinky.1 2024-03-09T09:27:08.391 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/kill.x 2024-03-09T09:27:08.391 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/pathchk.1 2024-03-09T09:27:08.392 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sync.1 2024-03-09T09:27:08.392 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/dir.1 2024-03-09T09:27:08.392 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/mv.1 2024-03-09T09:27:08.392 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/rm.1 2024-03-09T09:27:08.393 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/dircolors.x 2024-03-09T09:27:08.394 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/rmdir.1 2024-03-09T09:27:08.394 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tty.1 2024-03-09T09:27:08.394 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/fold.x 2024-03-09T09:27:08.394 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/fold.1 2024-03-09T09:27:08.394 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/dir.x 2024-03-09T09:27:08.394 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/fmt.x 2024-03-09T09:27:08.394 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/paste.1 2024-03-09T09:27:08.395 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/chmod.x 2024-03-09T09:27:08.395 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/od.1 2024-03-09T09:27:08.395 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/ptx.x 2024-03-09T09:27:08.395 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/nice.1 2024-03-09T09:27:08.395 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/install.1 2024-03-09T09:27:08.395 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/dd.x 2024-03-09T09:27:08.395 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/cksum.1 2024-03-09T09:27:08.395 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/test.1 2024-03-09T09:27:08.396 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/chgrp.1 2024-03-09T09:27:08.396 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/unlink.1 2024-03-09T09:27:08.396 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/chmod.1 2024-03-09T09:27:08.396 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/dd.1 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/du.1 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tsort.1 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/ls.1 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/split.x 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/nohup.x 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sha1sum.1 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tr.1 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/head.x 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/false.1 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/pwd.x 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/logname.x 2024-03-09T09:27:08.397 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/paste.x 2024-03-09T09:27:08.398 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/nl.x 2024-03-09T09:27:08.398 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sha224sum.1 2024-03-09T09:27:08.398 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/mkdir.1 2024-03-09T09:27:08.398 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/nl.1 2024-03-09T09:27:08.400 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sha256sum.1 2024-03-09T09:27:08.400 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/printf.x 2024-03-09T09:27:08.400 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/md5sum.1 2024-03-09T09:27:08.400 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/Makefile.am 2024-03-09T09:27:08.401 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/dirname.1 2024-03-09T09:27:08.401 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/shuf.x 2024-03-09T09:27:08.401 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/echo.1 2024-03-09T09:27:08.401 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/chcon.x 2024-03-09T09:27:08.401 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/chroot.1 2024-03-09T09:27:08.401 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sha384sum.x 2024-03-09T09:27:08.401 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/stdbuf.1 2024-03-09T09:27:08.401 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/users.1 2024-03-09T09:27:08.401 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/readlink.x 2024-03-09T09:27:08.402 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/cut.x 2024-03-09T09:27:08.402 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/groups.1 2024-03-09T09:27:08.402 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/seq.1 2024-03-09T09:27:08.404 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/help2man 2024-03-09T09:27:08.404 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/dirname.x 2024-03-09T09:27:08.405 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/uname.x 2024-03-09T09:27:08.405 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/uptime.x 2024-03-09T09:27:08.405 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/cut.1 2024-03-09T09:27:08.405 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/cksum.x 2024-03-09T09:27:08.405 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/nohup.1 2024-03-09T09:27:08.405 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/mv.x 2024-03-09T09:27:08.405 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sort.1 2024-03-09T09:27:08.405 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/chown.1 2024-03-09T09:27:08.405 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/pr.x 2024-03-09T09:27:08.405 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/true.x 2024-03-09T09:27:08.405 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/base64.1 2024-03-09T09:27:08.406 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/factor.1 2024-03-09T09:27:08.406 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/ln.x 2024-03-09T09:27:08.406 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/head.1 2024-03-09T09:27:08.406 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/unexpand.1 2024-03-09T09:27:08.406 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/df.x 2024-03-09T09:27:08.406 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/join.x 2024-03-09T09:27:08.406 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/du.x 2024-03-09T09:27:08.406 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/dircolors.1 2024-03-09T09:27:08.408 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/uniq.x 2024-03-09T09:27:08.408 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/split.1 2024-03-09T09:27:08.408 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tty.x 2024-03-09T09:27:08.408 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/uname.1 2024-03-09T09:27:08.408 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/uptime.1 2024-03-09T09:27:08.408 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sort.x 2024-03-09T09:27:08.408 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sha256sum.x 2024-03-09T09:27:08.409 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sha512sum.x 2024-03-09T09:27:08.409 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/nice.x 2024-03-09T09:27:08.409 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/pinky.x 2024-03-09T09:27:08.409 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/fmt.1 2024-03-09T09:27:08.409 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/expr.1 2024-03-09T09:27:08.410 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/wc.1 2024-03-09T09:27:08.410 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/true.1 2024-03-09T09:27:08.410 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/comm.x 2024-03-09T09:27:08.410 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/truncate.x 2024-03-09T09:27:08.410 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/install.x 2024-03-09T09:27:08.410 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/wc.x 2024-03-09T09:27:08.410 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/uniq.1 2024-03-09T09:27:08.410 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/arch.x 2024-03-09T09:27:08.411 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sum.x 2024-03-09T09:27:08.411 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/timeout.x 2024-03-09T09:27:08.412 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/link.x 2024-03-09T09:27:08.412 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tac.1 2024-03-09T09:27:08.412 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sync.x 2024-03-09T09:27:08.412 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/Makefile.in 2024-03-09T09:27:08.413 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/mkfifo.1 2024-03-09T09:27:08.414 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/od.x 2024-03-09T09:27:08.414 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/vdir.1 2024-03-09T09:27:08.414 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/timeout.1 2024-03-09T09:27:08.414 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/mkfifo.x 2024-03-09T09:27:08.415 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/basename.x 2024-03-09T09:27:08.415 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/expr.x 2024-03-09T09:27:08.416 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/mktemp.x 2024-03-09T09:27:08.416 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/vdir.x 2024-03-09T09:27:08.417 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/su.x 2024-03-09T09:27:08.417 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/env.1 2024-03-09T09:27:08.417 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/who.x 2024-03-09T09:27:08.417 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/hostid.1 2024-03-09T09:27:08.417 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tee.x 2024-03-09T09:27:08.418 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/rmdir.x 2024-03-09T09:27:08.418 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/hostid.x 2024-03-09T09:27:08.418 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/id.x 2024-03-09T09:27:08.418 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/unexpand.x 2024-03-09T09:27:08.418 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/pr.1 2024-03-09T09:27:08.419 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/csplit.1 2024-03-09T09:27:08.419 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/truncate.1 2024-03-09T09:27:08.420 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tee.1 2024-03-09T09:27:08.420 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/env.x 2024-03-09T09:27:08.420 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/chroot.x 2024-03-09T09:27:08.420 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/cp.x 2024-03-09T09:27:08.420 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/whoami.x 2024-03-09T09:27:08.422 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/logname.1 2024-03-09T09:27:08.422 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/mknod.1 2024-03-09T09:27:08.423 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/groups.x 2024-03-09T09:27:08.423 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/cp.1 2024-03-09T09:27:08.424 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/runcon.x 2024-03-09T09:27:08.424 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/yes.1 2024-03-09T09:27:08.424 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/stat.1 2024-03-09T09:27:08.425 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/nproc.x 2024-03-09T09:27:08.425 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/unlink.x 2024-03-09T09:27:08.426 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sha384sum.1 2024-03-09T09:27:08.426 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/nproc.1 2024-03-09T09:27:08.427 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/seq.x 2024-03-09T09:27:08.427 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sha512sum.1 2024-03-09T09:27:08.430 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/stat.x 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sleep.x 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/printenv.1 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tr.x 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tail.1 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/md5sum.x 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/link.1 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/echo.x 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/ptx.1 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/factor.x 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/who.1 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/mktemp.1 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/shred.1 2024-03-09T09:27:08.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/runcon.1 2024-03-09T09:27:08.432 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/stty.x 2024-03-09T09:27:08.432 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/comm.1 2024-03-09T09:27:08.432 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/shuf.1 2024-03-09T09:27:08.432 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/stty.1 2024-03-09T09:27:08.432 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/printenv.x 2024-03-09T09:27:08.432 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/join.1 2024-03-09T09:27:08.432 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/hostname.x 2024-03-09T09:27:08.432 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/ln.1 2024-03-09T09:27:08.432 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/false.x 2024-03-09T09:27:08.433 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/cat.1 2024-03-09T09:27:08.433 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/users.x 2024-03-09T09:27:08.433 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/printf.1 2024-03-09T09:27:08.433 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/ls.x 2024-03-09T09:27:08.433 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sha224sum.x 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/whoami.1 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/expand.1 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/chgrp.x 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sleep.1 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/csplit.x 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/pwd.1 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/kill.1 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/touch.x 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/df.1 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/stdbuf.x 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/mkdir.x 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/id.1 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/mknod.x 2024-03-09T09:27:08.437 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sum.1 2024-03-09T09:27:08.438 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/rm.x 2024-03-09T09:27:08.438 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/date.1 2024-03-09T09:27:08.438 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/chown.x 2024-03-09T09:27:08.438 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/touch.1 2024-03-09T09:27:08.438 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tac.x 2024-03-09T09:27:08.438 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/base64.x 2024-03-09T09:27:08.439 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/tsort.x 2024-03-09T09:27:08.439 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/shred.x 2024-03-09T09:27:08.439 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/readlink.1 2024-03-09T09:27:08.439 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/sha1sum.x 2024-03-09T09:27:08.439 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/expand.x 2024-03-09T09:27:08.439 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/test.x 2024-03-09T09:27:08.439 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/man/chcon.1 2024-03-09T09:27:08.440 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.tarball-version 2024-03-09T09:27:08.440 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/NEWS 2024-03-09T09:27:08.443 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_sun_os_names 2024-03-09T09:27:08.443 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/bootstrap 2024-03-09T09:27:08.444 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/ 2024-03-09T09:27:08.859 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbsalign.h 2024-03-09T09:27:08.887 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/streq.h 2024-03-09T09:27:08.888 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/asprintf.c 2024-03-09T09:27:08.888 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strnlen1.c 2024-03-09T09:27:08.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dirfd.c 2024-03-09T09:27:08.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/safe-write.h 2024-03-09T09:27:08.890 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/snprintf.c 2024-03-09T09:27:08.890 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbiter.h 2024-03-09T09:27:08.891 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mountlist.c 2024-03-09T09:27:08.892 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/propername.h 2024-03-09T09:27:08.893 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memset.c 2024-03-09T09:27:08.893 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/nproc.c 2024-03-09T09:27:08.894 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getdelim.c 2024-03-09T09:27:08.894 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/signbitl.c 2024-03-09T09:27:08.895 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/group-member.h 2024-03-09T09:27:08.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/long-options.c 2024-03-09T09:27:08.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fts_.h 2024-03-09T09:27:08.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fts.c 2024-03-09T09:27:08.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/verify.h 2024-03-09T09:27:08.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/str-two-way.h 2024-03-09T09:27:08.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/areadlink.c 2024-03-09T09:27:08.901 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/c-strtod.h 2024-03-09T09:27:08.901 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sha256.h 2024-03-09T09:27:08.901 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stddef.in.h 2024-03-09T09:27:08.901 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/randint.c 2024-03-09T09:27:08.901 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dup-safer.c 2024-03-09T09:27:08.901 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getpass.h 2024-03-09T09:27:08.901 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/gettext.h 2024-03-09T09:27:08.901 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/randint.h 2024-03-09T09:27:08.902 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stdio-write.c 2024-03-09T09:27:08.902 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xgethostname.h 2024-03-09T09:27:08.902 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/asnprintf.c 2024-03-09T09:27:08.902 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv_open-solaris.gperf 2024-03-09T09:27:08.902 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stdio-impl.h 2024-03-09T09:27:08.902 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mpsort.h 2024-03-09T09:27:08.906 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/closein.c 2024-03-09T09:27:08.918 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/hard-locale.c 2024-03-09T09:27:08.918 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/utimecmp.c 2024-03-09T09:27:08.919 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/vasnprintf.c 2024-03-09T09:27:08.921 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/chown.c 2024-03-09T09:27:08.922 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/printf-args.c 2024-03-09T09:27:08.923 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/netdb.in.h 2024-03-09T09:27:08.923 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strnlen.c 2024-03-09T09:27:08.924 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/errno.in.h 2024-03-09T09:27:08.924 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbslen.c 2024-03-09T09:27:08.925 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/obstack.h 2024-03-09T09:27:08.925 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/file-type.h 2024-03-09T09:27:08.926 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbsrtowcs.c 2024-03-09T09:27:08.926 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fnmatch_loop.c 2024-03-09T09:27:08.927 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/safe-write.c 2024-03-09T09:27:08.928 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mpsort.c 2024-03-09T09:27:08.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/netinet_in.in.h 2024-03-09T09:27:08.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xfreopen.h 2024-03-09T09:27:08.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/wctype.in.h 2024-03-09T09:27:08.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stripslash.c 2024-03-09T09:27:08.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/closein.h 2024-03-09T09:27:08.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memchr2.c 2024-03-09T09:27:08.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memcoll.c 2024-03-09T09:27:08.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/malloca.h 2024-03-09T09:27:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stpcpy.c 2024-03-09T09:27:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/striconv.c 2024-03-09T09:27:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/basename.c 2024-03-09T09:27:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sha256.c 2024-03-09T09:27:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stdint.in.h 2024-03-09T09:27:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/acl-internal.h 2024-03-09T09:27:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/atexit.c 2024-03-09T09:27:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/size_max.h 2024-03-09T09:27:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/buffer-lcm.h 2024-03-09T09:27:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/idcache.h 2024-03-09T09:27:08.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/root-dev-ino.c 2024-03-09T09:27:08.933 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/physmem.c 2024-03-09T09:27:08.933 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/glthread/ 2024-03-09T09:27:09.254 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/glthread/lock.h 2024-03-09T09:27:09.318 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/glthread/threadlib.c 2024-03-09T09:27:09.320 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/glthread/lock.c 2024-03-09T09:27:09.321 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/glthread/tls.h 2024-03-09T09:27:09.321 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/glthread/tls.c 2024-03-09T09:27:09.321 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unistr/ 2024-03-09T09:27:09.328 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unistr/u8-uctomb.c 2024-03-09T09:27:09.333 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unistr/u8-uctomb-aux.c 2024-03-09T09:27:09.333 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mkancesdirs.c 2024-03-09T09:27:09.333 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/readtokens0.c 2024-03-09T09:27:09.333 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fnmatch.in.h 2024-03-09T09:27:09.334 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dirname.h 2024-03-09T09:27:09.334 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/regex_internal.c 2024-03-09T09:27:09.334 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strcspn.c 2024-03-09T09:27:09.335 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getugroups.h 2024-03-09T09:27:09.335 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/file-set.c 2024-03-09T09:27:09.335 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sys_socket.in.h 2024-03-09T09:27:09.335 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/inttostr.h 2024-03-09T09:27:09.336 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mkdirat.c 2024-03-09T09:27:09.336 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/readtokens.h 2024-03-09T09:27:09.336 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fd-reopen.h 2024-03-09T09:27:09.336 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/w32sock.h 2024-03-09T09:27:09.336 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memcasecmp.c 2024-03-09T09:27:09.336 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/at-func.c 2024-03-09T09:27:09.337 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unistd-safer.h 2024-03-09T09:27:09.338 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/filevercmp.h 2024-03-09T09:27:09.338 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/c-strcasecmp.c 2024-03-09T09:27:09.338 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fopen-safer.c 2024-03-09T09:27:09.339 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/modechange.h 2024-03-09T09:27:09.339 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sys_stat.in.h 2024-03-09T09:27:09.339 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/freading.c 2024-03-09T09:27:09.339 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/utimens.h 2024-03-09T09:27:09.339 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/md5.h 2024-03-09T09:27:09.340 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/isnanf.c 2024-03-09T09:27:09.340 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sha1.c 2024-03-09T09:27:09.340 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/filenamecat-lgpl.c 2024-03-09T09:27:09.340 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/config.hin 2024-03-09T09:27:09.341 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/ignore-value.h 2024-03-09T09:27:09.341 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strstr.c 2024-03-09T09:27:09.341 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/verror.c 2024-03-09T09:27:09.342 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/randperm.h 2024-03-09T09:27:09.342 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/vasprintf.c 2024-03-09T09:27:09.342 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/c-strncasecmp.c 2024-03-09T09:27:09.342 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/basename-lgpl.c 2024-03-09T09:27:09.342 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/human.h 2024-03-09T09:27:09.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/regex.h 2024-03-09T09:27:09.343 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/regcomp.c 2024-03-09T09:27:09.344 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/freadahead.c 2024-03-09T09:27:09.344 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/trim.h 2024-03-09T09:27:09.345 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getgroups.c 2024-03-09T09:27:09.345 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xprintf.c 2024-03-09T09:27:09.346 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/euidaccess-stat.h 2024-03-09T09:27:09.346 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/rand-isaac.h 2024-03-09T09:27:09.346 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mgetgroups.h 2024-03-09T09:27:09.346 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/selinux-at.h 2024-03-09T09:27:09.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv_open-solaris.h 2024-03-09T09:27:09.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/setenv.c 2024-03-09T09:27:09.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/locale.in.h 2024-03-09T09:27:09.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/base64.h 2024-03-09T09:27:09.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/open-safer.c 2024-03-09T09:27:09.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/exitfail.c 2024-03-09T09:27:09.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/closeout.h 2024-03-09T09:27:09.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/idcache.c 2024-03-09T09:27:09.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unlink.c 2024-03-09T09:27:09.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xfts.c 2024-03-09T09:27:09.351 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memrchr.c 2024-03-09T09:27:09.351 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/Makefile.am 2024-03-09T09:27:09.351 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstrtol.h 2024-03-09T09:27:09.351 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/gettimeofday.c 2024-03-09T09:27:09.351 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/isnanf-nolibm.h 2024-03-09T09:27:09.351 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/chdir-long.c 2024-03-09T09:27:09.352 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbchar.c 2024-03-09T09:27:09.352 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/canon-host.h 2024-03-09T09:27:09.352 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unsetenv.c 2024-03-09T09:27:09.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/freadseek.c 2024-03-09T09:27:09.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xnanosleep.c 2024-03-09T09:27:09.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/nanosleep.c 2024-03-09T09:27:09.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/ctype.in.h 2024-03-09T09:27:09.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xmalloc.c 2024-03-09T09:27:09.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strtoul.c 2024-03-09T09:27:09.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sha512.h 2024-03-09T09:27:09.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dirchownmod.h 2024-03-09T09:27:09.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/freadptr.h 2024-03-09T09:27:09.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getfilecon.c 2024-03-09T09:27:09.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/alignof.h 2024-03-09T09:27:09.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mempcpy.c 2024-03-09T09:27:09.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/savedir.h 2024-03-09T09:27:09.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/areadlink-with-size.c 2024-03-09T09:27:09.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/inttypes.in.h 2024-03-09T09:27:09.355 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/config.charset 2024-03-09T09:27:09.355 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstrtold.c 2024-03-09T09:27:09.355 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fcntl-safer.h 2024-03-09T09:27:09.356 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstrndup.c 2024-03-09T09:27:09.356 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/write.c 2024-03-09T09:27:09.356 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/math.in.h 2024-03-09T09:27:09.357 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/openat-priv.h 2024-03-09T09:27:09.357 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/vasnprintf.h 2024-03-09T09:27:09.358 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getcwd.c 2024-03-09T09:27:09.358 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stdbool.in.h 2024-03-09T09:27:09.358 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strtol.c 2024-03-09T09:27:09.359 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/argmatch.c 2024-03-09T09:27:09.359 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/acl_entries.c 2024-03-09T09:27:09.359 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/progname.c 2024-03-09T09:27:09.359 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mktime-internal.h 2024-03-09T09:27:09.360 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/printf-frexpl.c 2024-03-09T09:27:09.360 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/offtostr.c 2024-03-09T09:27:09.360 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memchr2.h 2024-03-09T09:27:09.361 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstrtoumax.c 2024-03-09T09:27:09.361 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getdate.y 2024-03-09T09:27:09.361 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/close-hook.h 2024-03-09T09:27:09.361 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/printf-parse.c 2024-03-09T09:27:09.362 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/isnand.c 2024-03-09T09:27:09.362 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/timespec.h 2024-03-09T09:27:09.362 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getusershell.c 2024-03-09T09:27:09.362 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstrtod.c 2024-03-09T09:27:09.362 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/verror.h 2024-03-09T09:27:09.364 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/same.c 2024-03-09T09:27:09.364 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/quote.c 2024-03-09T09:27:09.364 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/raise.c 2024-03-09T09:27:09.364 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/link.c 2024-03-09T09:27:09.364 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/cloexec.h 2024-03-09T09:27:09.364 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sys_utsname.in.h 2024-03-09T09:27:09.364 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/cloexec.c 2024-03-09T09:27:09.365 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/vfprintf.c 2024-03-09T09:27:09.365 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/uniwidth/ 2024-03-09T09:27:09.567 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:27:09.568 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:27:09.688 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:09.688 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:09.688 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:27:09.812 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:27:09.812 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 16 2024-03-09T09:27:09.812 INFO:tasks.workunit.client.1.smithi134.stdout:16 2024-03-09T09:27:09.813 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:27:09.967 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:27:10.000 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:27:10.006 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:27:10.726 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:27:10.727 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:27:10.842 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:10.842 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:10.843 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:27:10.885 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:27:10.885 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 17 2024-03-09T09:27:10.886 INFO:tasks.workunit.client.1.smithi134.stdout:17 2024-03-09T09:27:10.886 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:27:11.080 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:27:11.325 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:27:11.335 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:27:13.006 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:27:13.006 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:27:13.289 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:13.289 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:13.289 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:27:13.334 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:27:13.334 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 18 2024-03-09T09:27:13.335 INFO:tasks.workunit.client.1.smithi134.stdout:18 2024-03-09T09:27:13.335 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:27:13.460 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:27:13.501 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:27:13.513 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:27:13.548 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:27:13.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:27:13.627 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/uniwidth/cjk.h 2024-03-09T09:27:13.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.784+0000 7fe509800640 1 Processor -- start 2024-03-09T09:27:13.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.784+0000 7fe509800640 1 -- start start 2024-03-09T09:27:13.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.786+0000 7fe509800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe5040c30f0 0x7fe504152a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:13.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.786+0000 7fe509800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5040ab3c0 con 0x7fe50406dbe0 2024-03-09T09:27:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.786+0000 7fe509800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5040aa8a0 con 0x7fe5040aa420 2024-03-09T09:27:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.786+0000 7fe509800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5040c82d0 con 0x7fe5040c30f0 2024-03-09T09:27:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.786+0000 7fe502a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe5040c30f0 0x7fe504152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.787+0000 7fe503400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe50406dbe0 0x7fe504072940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47174/0 (socket says 172.21.15.67:47174) 2024-03-09T09:27:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.787+0000 7fe503400640 1 -- 172.21.15.67:0/3522016884 learned_addr learned my addr 172.21.15.67:0/3522016884 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:27:13.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.787+0000 7fe502a00640 1 -- 172.21.15.67:0/3522016884 >> v1:172.21.15.134:6789/0 conn(0x7fe5040aa420 legacy=0x7fe5041490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:13.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.787+0000 7fe502a00640 1 -- 172.21.15.67:0/3522016884 >> v1:172.21.15.67:6789/0 conn(0x7fe50406dbe0 legacy=0x7fe504072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:13.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.787+0000 7fe502a00640 1 -- 172.21.15.67:0/3522016884 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5040aab20 con 0x7fe5040c30f0 2024-03-09T09:27:13.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.787+0000 7fe501600640 1 -- 172.21.15.67:0/3522016884 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226195934 0 0) 0x7fe4fc001490 con 0x7fe50406dbe0 2024-03-09T09:27:13.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.788+0000 7fe502a00640 1 --2- 172.21.15.67:0/3522016884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5040c30f0 0x7fe504152a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40097a0 tx=0x7fe4f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c7758ce6d2f107 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.788+0000 7fe501600640 1 -- 172.21.15.67:0/3522016884 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4035020 con 0x7fe5040c30f0 2024-03-09T09:27:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.788+0000 7fe501600640 1 -- 172.21.15.67:0/3522016884 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f4031830 con 0x7fe5040c30f0 2024-03-09T09:27:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.788+0000 7fe501600640 1 -- 172.21.15.67:0/3522016884 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4031c00 con 0x7fe5040c30f0 2024-03-09T09:27:13.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.789+0000 7fe509800640 1 -- 172.21.15.67:0/3522016884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5040c30f0 msgr2=0x7fe504152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.789+0000 7fe509800640 1 --2- 172.21.15.67:0/3522016884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5040c30f0 0x7fe504152a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40097a0 tx=0x7fe4f402b8d0 comp rx=0 tx=0).stop 2024-03-09T09:27:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.789+0000 7fe509800640 1 -- 172.21.15.67:0/3522016884 shutdown_connections 2024-03-09T09:27:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.789+0000 7fe509800640 1 --2- 172.21.15.67:0/3522016884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5040c30f0 0x7fe504152a20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.789+0000 7fe509800640 1 -- 172.21.15.67:0/3522016884 >> 172.21.15.67:0/3522016884 conn(0x7fe5040c7370 msgr2=0x7fe5040c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.789+0000 7fe509800640 1 -- 172.21.15.67:0/3522016884 shutdown_connections 2024-03-09T09:27:13.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.789+0000 7fe509800640 1 -- 172.21.15.67:0/3522016884 wait complete. 2024-03-09T09:27:13.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.791+0000 7fe509800640 1 Processor -- start 2024-03-09T09:27:13.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.791+0000 7fe509800640 1 -- start start 2024-03-09T09:27:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.792+0000 7fe509800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe50406dbe0 0x7fe5041e2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.792+0000 7fe509800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5040aa420 0x7fe5041480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.792+0000 7fe509800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5040c30f0 0x7fe504148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.792+0000 7fe509800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50406d6c0 con 0x7fe50406dbe0 2024-03-09T09:27:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.792+0000 7fe509800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5040ab0e0 con 0x7fe5040c30f0 2024-03-09T09:27:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.792+0000 7fe509800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe504149800 con 0x7fe5040aa420 2024-03-09T09:27:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.792+0000 7fe502a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe50406dbe0 0x7fe5041e2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.792+0000 7fe502a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe50406dbe0 0x7fe5041e2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51836/0 (socket says 172.21.15.67:51836) 2024-03-09T09:27:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.792+0000 7fe502a00640 1 -- 172.21.15.67:0/2409325723 learned_addr learned my addr 172.21.15.67:0/2409325723 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.793+0000 7fe503400640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5040c30f0 0x7fe504148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:13.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.793+0000 7fe502000640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5040aa420 0x7fe5041480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.793+0000 7fe502a00640 1 -- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5040aa420 msgr2=0x7fe5041480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.793+0000 7fe502a00640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5040aa420 0x7fe5041480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.793+0000 7fe502a00640 1 -- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5040c30f0 msgr2=0x7fe504148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.793+0000 7fe502a00640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5040c30f0 0x7fe504148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.793+0000 7fe502a00640 1 -- 172.21.15.67:0/2409325723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50414a140 con 0x7fe50406dbe0 2024-03-09T09:27:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.793+0000 7fe502a00640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe50406dbe0 0x7fe5041e2700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40098c0 tx=0x7fe4f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.794+0000 7fe4faa00640 1 -- 172.21.15.67:0/2409325723 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4041070 con 0x7fe50406dbe0 2024-03-09T09:27:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.794+0000 7fe4faa00640 1 -- 172.21.15.67:0/2409325723 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f403aa30 con 0x7fe50406dbe0 2024-03-09T09:27:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.794+0000 7fe4faa00640 1 -- 172.21.15.67:0/2409325723 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4035040 con 0x7fe50406dbe0 2024-03-09T09:27:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.794+0000 7fe509800640 1 -- 172.21.15.67:0/2409325723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5041ebb50 con 0x7fe50406dbe0 2024-03-09T09:27:13.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.794+0000 7fe509800640 1 -- 172.21.15.67:0/2409325723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5041ec110 con 0x7fe50406dbe0 2024-03-09T09:27:13.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.795+0000 7fe509800640 1 -- 172.21.15.67:0/2409325723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe504075a00 con 0x7fe50406dbe0 2024-03-09T09:27:13.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.796+0000 7fe4faa00640 1 -- 172.21.15.67:0/2409325723 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe4f404c060 con 0x7fe50406dbe0 2024-03-09T09:27:13.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.798+0000 7fe4faa00640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4e406b500 0x7fe4e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:13.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.798+0000 7fe4faa00640 1 -- 172.21.15.67:0/2409325723 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fe4f40abf80 con 0x7fe50406dbe0 2024-03-09T09:27:13.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.798+0000 7fe502000640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4e406b500 0x7fe4e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:13.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.799+0000 7fe502000640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4e406b500 0x7fe4e406d9c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe504153210 tx=0x7fe4f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:13.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:13.803+0000 7fe4faa00640 1 -- 172.21.15.67:0/2409325723 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe4f407a9f0 con 0x7fe50406dbe0 2024-03-09T09:27:13.895 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/uniwidth/width.c 2024-03-09T09:27:13.895 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strtoull.c 2024-03-09T09:27:13.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/opendir-safer.c 2024-03-09T09:27:13.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/hash-pjw.h 2024-03-09T09:27:13.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/uname.c 2024-03-09T09:27:13.901 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/full-read.c 2024-03-09T09:27:13.901 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/priv-set.h 2024-03-09T09:27:13.902 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/close.c 2024-03-09T09:27:13.902 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fpending.h 2024-03-09T09:27:13.902 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/i-ring.h 2024-03-09T09:27:13.902 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/nl_langinfo.c 2024-03-09T09:27:13.903 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xmemxfrm.h 2024-03-09T09:27:13.903 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/c-strcase.h 2024-03-09T09:27:13.903 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fchdir.c 2024-03-09T09:27:13.903 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/canon-host.c 2024-03-09T09:27:13.903 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/signbitf.c 2024-03-09T09:27:13.903 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xnanosleep.h 2024-03-09T09:27:13.904 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/tempname.c 2024-03-09T09:27:13.904 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xreadlink.c 2024-03-09T09:27:13.904 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xgetcwd.h 2024-03-09T09:27:13.904 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/regex.c 2024-03-09T09:27:13.904 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/diacrit.h 2024-03-09T09:27:13.905 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/posixver.c 2024-03-09T09:27:13.905 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getugroups.c 2024-03-09T09:27:13.905 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/readutmp.h 2024-03-09T09:27:13.906 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/quote.h 2024-03-09T09:27:13.906 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/quotearg.c 2024-03-09T09:27:13.906 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/filevercmp.c 2024-03-09T09:27:13.907 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/printf-args.h 2024-03-09T09:27:13.907 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mountlist.h 2024-03-09T09:27:13.907 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstrtol-error.c 2024-03-09T09:27:13.907 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/rename.c 2024-03-09T09:27:13.907 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sha1.h 2024-03-09T09:27:13.908 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strndup.c 2024-03-09T09:27:13.908 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/priv-set.c 2024-03-09T09:27:13.908 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xfts.h 2024-03-09T09:27:13.908 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/argv-iter.c 2024-03-09T09:27:13.908 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stdio--.h 2024-03-09T09:27:13.909 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/yesno.c 2024-03-09T09:27:13.909 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getopt.c 2024-03-09T09:27:13.910 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strpbrk.c 2024-03-09T09:27:13.910 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dirname-lgpl.c 2024-03-09T09:27:13.910 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/filemode.h 2024-03-09T09:27:13.910 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbrtowc.c 2024-03-09T09:27:13.911 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv_open.c 2024-03-09T09:27:13.911 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/euidaccess-stat.c 2024-03-09T09:27:13.911 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stat-time.h 2024-03-09T09:27:13.911 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/frexpl.c 2024-03-09T09:27:13.911 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sys_select.in.h 2024-03-09T09:27:13.912 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strftime.h 2024-03-09T09:27:13.912 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strftime.c 2024-03-09T09:27:13.912 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv_open-aix.gperf 2024-03-09T09:27:13.912 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/buffer-lcm.c 2024-03-09T09:27:13.913 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unistr.h 2024-03-09T09:27:13.913 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/modechange.c 2024-03-09T09:27:13.913 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/tempname.h 2024-03-09T09:27:13.913 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/faccessat.c 2024-03-09T09:27:13.913 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unicodeio.h 2024-03-09T09:27:13.914 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/filemode.c 2024-03-09T09:27:13.914 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strdup.c 2024-03-09T09:27:13.914 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strsignal.c 2024-03-09T09:27:13.914 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/euidaccess.c 2024-03-09T09:27:13.915 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/select.c 2024-03-09T09:27:13.915 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memcoll.h 2024-03-09T09:27:13.915 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fseeko.c 2024-03-09T09:27:13.917 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/langinfo.in.h 2024-03-09T09:27:13.918 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getopt.in.h 2024-03-09T09:27:13.918 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/localcharset.c 2024-03-09T09:27:13.918 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/isapipe.c 2024-03-09T09:27:13.918 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/settime.c 2024-03-09T09:27:13.918 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/bitrotate.h 2024-03-09T09:27:13.918 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/isnanl-nolibm.h 2024-03-09T09:27:13.918 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fcntl.c 2024-03-09T09:27:13.918 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/exclude.h 2024-03-09T09:27:13.919 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/symlinkat.c 2024-03-09T09:27:13.919 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fpending.c 2024-03-09T09:27:13.919 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/savewd.c 2024-03-09T09:27:13.919 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fchownat.c 2024-03-09T09:27:13.920 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/root-dev-ino.h 2024-03-09T09:27:13.920 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/randread.h 2024-03-09T09:27:13.920 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/freadseek.h 2024-03-09T09:27:13.920 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strtoll.c 2024-03-09T09:27:13.921 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stdio-safer.h 2024-03-09T09:27:13.921 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strtoumax.c 2024-03-09T09:27:13.921 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/argv-iter.h 2024-03-09T09:27:13.921 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dirent-safer.h 2024-03-09T09:27:13.921 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dirent.in.h 2024-03-09T09:27:13.921 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/malloca.valgrind 2024-03-09T09:27:13.921 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/savewd.h 2024-03-09T09:27:13.921 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/uinttostr.c 2024-03-09T09:27:13.922 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getline.c 2024-03-09T09:27:13.922 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/chdir-long.h 2024-03-09T09:27:13.922 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/cycle-check.h 2024-03-09T09:27:13.922 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fts-cycle.c 2024-03-09T09:27:13.923 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/file-has-acl.c 2024-03-09T09:27:13.923 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/areadlink.h 2024-03-09T09:27:13.923 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sig2str.h 2024-03-09T09:27:13.923 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memcpy.c 2024-03-09T09:27:13.924 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/freading.h 2024-03-09T09:27:13.924 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/full-write.h 2024-03-09T09:27:13.924 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unistd.in.h 2024-03-09T09:27:13.924 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/symlink.c 2024-03-09T09:27:13.924 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/se-selinux.in.h 2024-03-09T09:27:13.924 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/write-any-file.h 2024-03-09T09:27:13.925 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/printf-frexpl.h 2024-03-09T09:27:13.925 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/Makefile.in 2024-03-09T09:27:13.927 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/filenamecat.h 2024-03-09T09:27:13.928 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/malloca.c 2024-03-09T09:27:13.928 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/utimecmp.h 2024-03-09T09:27:13.928 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stdlib.in.h 2024-03-09T09:27:13.928 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stdio.in.h 2024-03-09T09:27:13.929 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbsinit.c 2024-03-09T09:27:13.929 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/diacrit.c 2024-03-09T09:27:13.929 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/se-context.in.h 2024-03-09T09:27:13.929 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xvasprintf.c 2024-03-09T09:27:13.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xgetcwd.c 2024-03-09T09:27:13.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sockets.h 2024-03-09T09:27:13.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/striconv.h 2024-03-09T09:27:13.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mkstemp-safer.c 2024-03-09T09:27:13.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/u64.h 2024-03-09T09:27:13.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstrtoimax.c 2024-03-09T09:27:13.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/time_r.c 2024-03-09T09:27:13.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbchar.h 2024-03-09T09:27:13.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xtime.h 2024-03-09T09:27:13.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/set-mode-acl.c 2024-03-09T09:27:13.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/version-etc.c 2024-03-09T09:27:13.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dup2.c 2024-03-09T09:27:13.935 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unlinkat.c 2024-03-09T09:27:13.935 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sigprocmask.c 2024-03-09T09:27:13.935 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/hash-triple.h 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fprintftime.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unistd--.h 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/umaxtostr.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/i-ring.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/signbitd.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/ftello.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/lchown.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/lstat.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fpurge.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fstatat.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/lseek.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fchmodat.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/alloca.c 2024-03-09T09:27:13.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memcmp.c 2024-03-09T09:27:13.937 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstrndup.h 2024-03-09T09:27:13.937 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/frexp.c 2024-03-09T09:27:13.937 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/nproc.h 2024-03-09T09:27:13.937 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fsusage.h 2024-03-09T09:27:13.937 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fd-safer-flag.c 2024-03-09T09:27:13.937 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getdtablesize.c 2024-03-09T09:27:13.937 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/savedir.c 2024-03-09T09:27:13.943 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/wcwidth.c 2024-03-09T09:27:13.943 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbsstr.c 2024-03-09T09:27:13.943 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fpucw.h 2024-03-09T09:27:13.943 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv_open-hpux.h 2024-03-09T09:27:13.944 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/readtokens0.h 2024-03-09T09:27:13.944 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getdate.c 2024-03-09T09:27:13.944 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/utimens.c 2024-03-09T09:27:13.945 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getndelim2.c 2024-03-09T09:27:13.945 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fflush.c 2024-03-09T09:27:13.945 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/version-etc-fsf.c 2024-03-09T09:27:13.945 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xsize.h 2024-03-09T09:27:13.945 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xgethostname.c 2024-03-09T09:27:13.945 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strnlen1.h 2024-03-09T09:27:13.945 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/isnanl.c 2024-03-09T09:27:13.946 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/same-inode.h 2024-03-09T09:27:13.947 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/regex_internal.h 2024-03-09T09:27:13.947 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/full-read.h 2024-03-09T09:27:13.947 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/intprops.h 2024-03-09T09:27:13.947 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fsusage.c 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/save-cwd.c 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memchr.valgrind 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/exitfail.h 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/printf-frexp.h 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/isblank.c 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/arpa_inet.in.h 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strerror.c 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/freadptr.c 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/openat.h 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv_open-irix.h 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/open.c 2024-03-09T09:27:13.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/vprintf.c 2024-03-09T09:27:13.950 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/same.h 2024-03-09T09:27:13.950 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/gai_strerror.c 2024-03-09T09:27:13.950 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/imaxtostr.c 2024-03-09T09:27:13.950 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memcmp2.h 2024-03-09T09:27:13.950 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xfreopen.c 2024-03-09T09:27:13.950 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/rand-isaac.c 2024-03-09T09:27:13.950 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/full-write.c 2024-03-09T09:27:13.951 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/rmdir.c 2024-03-09T09:27:13.951 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/save-cwd.h 2024-03-09T09:27:13.951 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/at-func2.c 2024-03-09T09:27:13.951 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/gethrxtime.c 2024-03-09T09:27:13.951 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strtod.c 2024-03-09T09:27:13.951 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbsalign.c 2024-03-09T09:27:13.951 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv_open-osf.gperf 2024-03-09T09:27:13.951 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/ref-add.sin 2024-03-09T09:27:13.951 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memcmp2.c 2024-03-09T09:27:13.951 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/gnulib.mk 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/ref-del.sin 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stpncpy.c 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv.in.h 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/posixtm.h 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstrtol.c 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getopt1.c 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/quotearg.h 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/cycle-check.c 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/calloc.c 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/areadlinkat.c 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/hash-triple.c 2024-03-09T09:27:13.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getpass.c 2024-03-09T09:27:13.954 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstriconv.c 2024-03-09T09:27:13.954 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dirent--.h 2024-03-09T09:27:13.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sigaction.c 2024-03-09T09:27:13.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/linkat.c 2024-03-09T09:27:13.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unitypes.h 2024-03-09T09:27:13.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/linebuffer.c 2024-03-09T09:27:13.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fnmatch.c 2024-03-09T09:27:13.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xasprintf.c 2024-03-09T09:27:13.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/printf-parse.h 2024-03-09T09:27:13.956 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/string.in.h 2024-03-09T09:27:13.956 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mkdir.c 2024-03-09T09:27:13.956 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fchown-stub.c 2024-03-09T09:27:13.956 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/human.c 2024-03-09T09:27:13.956 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sockets.c 2024-03-09T09:27:13.956 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/canonicalize.h 2024-03-09T09:27:13.963 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getopt_int.h 2024-03-09T09:27:13.963 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/version-etc.h 2024-03-09T09:27:13.963 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stdlib--.h 2024-03-09T09:27:13.963 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/backupfile.h 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/file-type.c 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/file-set.h 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbuiter.h 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fcntl.in.h 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/inet_ntop.c 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fseterr.c 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/close-stream.h 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/freopen-safer.c 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stat-macros.h 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dirname.c 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dev-ino.h 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xalloc.h 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memcasecmp.h 2024-03-09T09:27:13.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/hash.c 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/userspec.c 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/hard-locale.h 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xvasprintf.h 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/wcrtomb.c 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/gethrxtime.h 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fprintftime.h 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mgetgroups.c 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/randread.c 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstriconv.h 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dup-safer-flag.c 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/yesno.h 2024-03-09T09:27:13.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbscasecmp.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strtoimax.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbsrtowcs-state.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mkstemp.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xmemcoll.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strintcmp.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fclose.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fd-reopen.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/inttostr.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/c-ctype.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/openat-die.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/error.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sig2str.c 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unlocked-io.h 2024-03-09T09:27:13.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memxfrm.c 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/pipe-safer.c 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/backupfile.c 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv_open-osf.h 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/linebuffer.h 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xprintf.h 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/exclude.c 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/userspec.h 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mkancesdirs.h 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/hash-pjw.c 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fd-safer.c 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/long-options.h 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/freadahead.h 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/gethostname.c 2024-03-09T09:27:13.967 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/putenv.c 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbrlen.c 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/group-member.c 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sha512.c 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/signal.in.h 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mktime.c 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/readlink.c 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mkdir-p.c 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getndelim2.h 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memxfrm.h 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/c-strtod.c 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fseterr.h 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv_open-hpux.gperf 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/filenamecat.c 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memmove.c 2024-03-09T09:27:13.968 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/str-kmp.h 2024-03-09T09:27:13.969 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/base64.c 2024-03-09T09:27:13.969 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/close-hook.c 2024-03-09T09:27:13.969 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/dirchownmod.c 2024-03-09T09:27:13.969 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/openat-safer.c 2024-03-09T09:27:13.969 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/posixtm.c 2024-03-09T09:27:13.969 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/printf-frexp.c 2024-03-09T09:27:13.969 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/safe-read.h 2024-03-09T09:27:13.969 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/regexec.c 2024-03-09T09:27:13.971 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/uniwidth.h 2024-03-09T09:27:13.971 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/write-any-file.c 2024-03-09T09:27:13.971 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv_open-aix.h 2024-03-09T09:27:13.971 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/unicodeio.c 2024-03-09T09:27:13.971 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/gettime.c 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strnumcmp.h 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/btowc.c 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sys_time.in.h 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/isnan.c 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getaddrinfo.c 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getpagesize.c 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/posixver.h 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/openat.c 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/closeout.c 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fopen.c 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memchr2.valgrind 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/time.in.h 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/physmem.h 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/iconv_open-irix.gperf 2024-03-09T09:27:13.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stat.c 2024-03-09T09:27:14.148 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:27:14.148 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:27:14.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.164+0000 7fe509800640 1 -- 172.21.15.67:0/2409325723 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe5040abd90 con 0x7fe50406dbe0 2024-03-09T09:27:14.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.164+0000 7fe4faa00640 1 -- 172.21.15.67:0/2409325723 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fe4f407a2f0 con 0x7fe50406dbe0 2024-03-09T09:27:14.172 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":276,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:27:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.168+0000 7fe509800640 1 -- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4e406b500 msgr2=0x7fe4e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.168+0000 7fe509800640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4e406b500 0x7fe4e406d9c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe504153210 tx=0x7fe4f0009040 comp rx=0 tx=0).stop 2024-03-09T09:27:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.168+0000 7fe509800640 1 -- 172.21.15.67:0/2409325723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe50406dbe0 msgr2=0x7fe5041e2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.168+0000 7fe509800640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe50406dbe0 0x7fe5041e2700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40098c0 tx=0x7fe4f40046c0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.168+0000 7fe509800640 1 -- 172.21.15.67:0/2409325723 shutdown_connections 2024-03-09T09:27:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.168+0000 7fe509800640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe4e406b500 0x7fe4e406d9c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.168+0000 7fe509800640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5040c30f0 0x7fe504148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.168+0000 7fe509800640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5040aa420 0x7fe5041480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.168+0000 7fe509800640 1 --2- 172.21.15.67:0/2409325723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe50406dbe0 0x7fe5041e2700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.169+0000 7fe509800640 1 -- 172.21.15.67:0/2409325723 >> 172.21.15.67:0/2409325723 conn(0x7fe5040c7370 msgr2=0x7fe504071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.169+0000 7fe509800640 1 -- 172.21.15.67:0/2409325723 shutdown_connections 2024-03-09T09:27:14.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.169+0000 7fe509800640 1 -- 172.21.15.67:0/2409325723 wait complete. 2024-03-09T09:27:14.200 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:27:14.200 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:27:14.200 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:27:14.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:27:14.268 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:14.268 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:14.269 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:27:14.310 INFO:tasks.workunit.client.1.smithi134.stdout:19 2024-03-09T09:27:14.311 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:27:14.311 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 19 2024-03-09T09:27:14.311 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/siglist.h 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbswidth.h 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstrtod.h 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/md5.c 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getloadavg.c 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/getdate.h 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stdlib-safer.h 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/remove.c 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/malloc.c 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/progname.h 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/argmatch.h 2024-03-09T09:27:14.347 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/ChangeLog-2007 2024-03-09T09:27:14.349 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mbswidth.c 2024-03-09T09:27:14.349 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/error.h 2024-03-09T09:27:14.349 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/obstack.c 2024-03-09T09:27:14.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/acl.h 2024-03-09T09:27:14.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strnumcmp-in.h 2024-03-09T09:27:14.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xstrtoul.c 2024-03-09T09:27:14.350 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/float+.h 2024-03-09T09:27:14.351 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/canonicalize.c 2024-03-09T09:27:14.351 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fcntl--.h 2024-03-09T09:27:14.351 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/wchar.in.h 2024-03-09T09:27:14.351 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/trim.c 2024-03-09T09:27:14.352 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/rpmatch.c 2024-03-09T09:27:14.352 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/isapipe.h 2024-03-09T09:27:14.352 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/memchr.c 2024-03-09T09:27:14.352 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/localcharset.h 2024-03-09T09:27:14.352 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/sig-handler.h 2024-03-09T09:27:14.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/openat-proc.c 2024-03-09T09:27:14.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/pathmax.h 2024-03-09T09:27:14.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/close-stream.c 2024-03-09T09:27:14.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/readutmp.c 2024-03-09T09:27:14.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/float.in.h 2024-03-09T09:27:14.353 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/creat-safer.c 2024-03-09T09:27:14.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/randperm.c 2024-03-09T09:27:14.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/readtokens.c 2024-03-09T09:27:14.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/safe-read.c 2024-03-09T09:27:14.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xreadlink.h 2024-03-09T09:27:14.354 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/stdarg.in.h 2024-03-09T09:27:14.355 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xalloc-die.c 2024-03-09T09:27:14.355 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/copy-acl.c 2024-03-09T09:27:14.355 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fsync.c 2024-03-09T09:27:14.355 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/strnumcmp.c 2024-03-09T09:27:14.355 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/selinux-at.c 2024-03-09T09:27:14.356 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/realloc.c 2024-03-09T09:27:14.356 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fdopendir.c 2024-03-09T09:27:14.356 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/alloca.in.h 2024-03-09T09:27:14.357 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/freopen.c 2024-03-09T09:27:14.357 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/mkdir-p.h 2024-03-09T09:27:14.357 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/isnand-nolibm.h 2024-03-09T09:27:14.357 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/propername.c 2024-03-09T09:27:14.358 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/fileblocks.c 2024-03-09T09:27:14.358 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xmemcoll.h 2024-03-09T09:27:14.358 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/hash.h 2024-03-09T09:27:14.358 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/c-strtold.c 2024-03-09T09:27:14.358 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/c-ctype.h 2024-03-09T09:27:14.358 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/lib/xmemxfrm.c 2024-03-09T09:27:14.358 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_require_config_h_first 2024-03-09T09:27:14.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.417+0000 7fa832000640 1 Processor -- start 2024-03-09T09:27:14.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.417+0000 7fa832000640 1 -- start start 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.419+0000 7fa832000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa82c0c5f50 0x7fa82c06b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.419+0000 7fa832000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa82c0c6330 0x7fa82c0cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.419+0000 7fa832000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c0c9f30 con 0x7fa82c0c96f0 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.419+0000 7fa832000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c06cfa0 con 0x7fa82c0c6330 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.419+0000 7fa832000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0ca820 con 0x7fa82c0c5f50 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.419+0000 7fa82aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fa82c0c96f0 0x7fa82c1567a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47186/0 (socket says 172.21.15.67:47186) 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.419+0000 7fa82aa00640 1 -- 172.21.15.67:0/1678849531 learned_addr learned my addr 172.21.15.67:0/1678849531 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.419+0000 7fa82be00640 1 --2- 172.21.15.67:0/1678849531 >> v2:172.21.15.67:3300/0 conn(0x7fa82c0c6330 0x7fa82c0cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.420+0000 7fa82a000640 1 -- 172.21.15.67:0/1678849531 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839531236 0 0) 0x7fa82c0c9f30 con 0x7fa82c0c96f0 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.420+0000 7fa82a000640 1 -- 172.21.15.67:0/1678849531 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa810003520 con 0x7fa82c0c96f0 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.420+0000 7fa82be00640 1 -- 172.21.15.67:0/1678849531 >> v2:172.21.15.134:3300/0 conn(0x7fa82c0c5f50 msgr2=0x7fa82c06b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.420+0000 7fa82be00640 1 --2- 172.21.15.67:0/1678849531 >> v2:172.21.15.134:3300/0 conn(0x7fa82c0c5f50 0x7fa82c06b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.420+0000 7fa82be00640 1 -- 172.21.15.67:0/1678849531 >> v1:172.21.15.67:6789/0 conn(0x7fa82c0c96f0 legacy=0x7fa82c1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.420+0000 7fa82be00640 1 -- 172.21.15.67:0/1678849531 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0ca1d0 con 0x7fa82c0c6330 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.421+0000 7fa82be00640 1 --2- 172.21.15.67:0/1678849531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0c6330 0x7fa82c0cacb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0753c0 tx=0x7fa82402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950c42575b805839 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.422+0000 7fa82a000640 1 -- 172.21.15.67:0/1678849531 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824039070 con 0x7fa82c0c6330 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.422+0000 7fa82a000640 1 -- 172.21.15.67:0/1678849531 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82402cc80 con 0x7fa82c0c6330 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.422+0000 7fa82a000640 1 -- 172.21.15.67:0/1678849531 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824034660 con 0x7fa82c0c6330 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.422+0000 7fa832000640 1 -- 172.21.15.67:0/1678849531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0c6330 msgr2=0x7fa82c0cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.422+0000 7fa832000640 1 --2- 172.21.15.67:0/1678849531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0c6330 0x7fa82c0cacb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0753c0 tx=0x7fa82402bed0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.422+0000 7fa832000640 1 -- 172.21.15.67:0/1678849531 shutdown_connections 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.422+0000 7fa832000640 1 --2- 172.21.15.67:0/1678849531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0c6330 0x7fa82c0cacb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.422+0000 7fa832000640 1 --2- 172.21.15.67:0/1678849531 >> v2:172.21.15.134:3300/0 conn(0x7fa82c0c5f50 0x7fa82c06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.422+0000 7fa832000640 1 -- 172.21.15.67:0/1678849531 >> 172.21.15.67:0/1678849531 conn(0x7fa82c0c8050 msgr2=0x7fa82c0c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.423+0000 7fa832000640 1 -- 172.21.15.67:0/1678849531 shutdown_connections 2024-03-09T09:27:14.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.423+0000 7fa832000640 1 -- 172.21.15.67:0/1678849531 wait complete. 2024-03-09T09:27:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.424+0000 7fa832000640 1 Processor -- start 2024-03-09T09:27:14.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.424+0000 7fa832000640 1 -- start start 2024-03-09T09:27:14.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.425+0000 7fa832000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa82c0c5f50 0x7fa82c0ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.425+0000 7fa832000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0c6330 0x7fa82c1ef100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.425+0000 7fa832000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa82c0c96f0 0x7fa82c149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.425+0000 7fa832000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0c9bf0 con 0x7fa82c0c6330 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.425+0000 7fa832000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0c67a0 con 0x7fa82c0c96f0 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.425+0000 7fa832000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0cb1f0 con 0x7fa82c0c5f50 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.425+0000 7fa82aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0c6330 0x7fa82c1ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.426+0000 7fa82aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0c6330 0x7fa82c1ef100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51858/0 (socket says 172.21.15.67:51858) 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.426+0000 7fa82aa00640 1 -- 172.21.15.67:0/3727198097 learned_addr learned my addr 172.21.15.67:0/3727198097 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.426+0000 7fa82be00640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa82c0c96f0 0x7fa82c149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.426+0000 7fa82aa00640 1 -- 172.21.15.67:0/3727198097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa82c0c5f50 msgr2=0x7fa82c0ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.426+0000 7fa82aa00640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa82c0c5f50 0x7fa82c0ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.426+0000 7fa82aa00640 1 -- 172.21.15.67:0/3727198097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa82c0c96f0 msgr2=0x7fa82c149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.426+0000 7fa82aa00640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa82c0c96f0 0x7fa82c149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.426+0000 7fa82aa00640 1 -- 172.21.15.67:0/3727198097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0c17c0 con 0x7fa82c0c6330 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.426+0000 7fa82be00640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa82c0c96f0 0x7fa82c149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:27:14.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.426+0000 7fa82aa00640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0c6330 0x7fa82c1ef100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c430 tx=0x7fa81800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.427+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818017070 con 0x7fa82c0c6330 2024-03-09T09:27:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.427+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa818005d00 con 0x7fa82c0c6330 2024-03-09T09:27:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.427+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818007310 con 0x7fa82c0c6330 2024-03-09T09:27:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.427+0000 7fa832000640 1 -- 172.21.15.67:0/3727198097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c0c1200 con 0x7fa82c0c6330 2024-03-09T09:27:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.427+0000 7fa832000640 1 -- 172.21.15.67:0/3727198097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c0c2c60 con 0x7fa82c0c6330 2024-03-09T09:27:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.428+0000 7fa832000640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c1ef640 0x7fa82c14a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.428+0000 7fa832000640 1 -- 172.21.15.67:0/3727198097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0ab980 con 0x7fa82c1ef640 2024-03-09T09:27:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.428+0000 7fa82b400640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c1ef640 0x7fa82c14a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.429+0000 7fa82b400640 1 -- 172.21.15.67:0/3727198097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa82c075a00 con 0x7fa82c1ef640 2024-03-09T09:27:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.429+0000 7fa82b400640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c1ef640 0x7fa82c14a3f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00b340 tx=0x7fa81c00b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.429+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c015020 con 0x7fa82c1ef640 2024-03-09T09:27:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.430+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fa81c011830 con 0x7fa82c1ef640 2024-03-09T09:27:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.430+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c1ef640 msgr2=0x7fa82c14a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.430+0000 7fa823400640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c1ef640 0x7fa82c14a3f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00b340 tx=0x7fa81c00b800 comp rx=0 tx=0).stop 2024-03-09T09:27:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.430+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c1ef640 msgr2=0x7fa82c14a3f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.430+0000 7fa823400640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c1ef640 0x7fa82c14a3f0 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00b340 tx=0x7fa81c00b800 comp rx=0 tx=0).stop 2024-03-09T09:27:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.430+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa81801a070 con 0x7fa82c0c6330 2024-03-09T09:27:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.431+0000 7fa823400640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa80c06af40 0x7fa80c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.431+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fa818089a60 con 0x7fa82c0c6330 2024-03-09T09:27:14.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.431+0000 7fa82b400640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa80c06af40 0x7fa80c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.432+0000 7fa82b400640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa80c06af40 0x7fa80c06d400 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00bda0 tx=0x7fa81c01e040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.441+0000 7fa832000640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0ac560 0x7fa82c1eb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.441+0000 7fa832000640 1 -- 172.21.15.67:0/3727198097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c06cd10 con 0x7fa82c0ac560 2024-03-09T09:27:14.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.441+0000 7fa82be00640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0ac560 0x7fa82c1eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.442+0000 7fa82be00640 1 -- 172.21.15.67:0/3727198097 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa82c0c94d0 con 0x7fa82c0ac560 2024-03-09T09:27:14.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.442+0000 7fa82be00640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0ac560 0x7fa82c1eb920 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa82c073bd0 tx=0x7fa824004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.442+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824045070 con 0x7fa82c0ac560 2024-03-09T09:27:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4537 (secure 0 0 0) 0x7fa824034d60 con 0x7fa82c0ac560 2024-03-09T09:27:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0ac560 msgr2=0x7fa82c1eb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa823400640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0ac560 0x7fa82c1eb920 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa82c073bd0 tx=0x7fa824004a40 comp rx=0 tx=0).stop 2024-03-09T09:27:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa823400640 1 -- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0ac560 msgr2=0x7fa82c1eb920 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:14.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa823400640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0ac560 0x7fa82c1eb920 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa82c073bd0 tx=0x7fa824004a40 comp rx=0 tx=0).stop 2024-03-09T09:27:14.451 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:27:14.451 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:14.451 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:14.451 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:27:14.451 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 276, 2024-03-09T09:27:14.451 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:27:14.451 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:27:14.451 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:27:14.451 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 13, 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:14.452 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:14.453 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:14.454 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:14.455 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:14.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:27:14.457 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:14.459 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.460 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:14.461 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.462 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:14.462 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:27:14.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.462 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:14.462 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:14.462 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:27:14.462 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.462 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.462 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:27:14.462 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:27:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa832000640 1 -- 172.21.15.67:0/3727198097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa80c06af40 msgr2=0x7fa80c06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa832000640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa80c06af40 0x7fa80c06d400 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00bda0 tx=0x7fa81c01e040 comp rx=0 tx=0).stop 2024-03-09T09:27:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa832000640 1 -- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0c6330 msgr2=0x7fa82c1ef100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa832000640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0c6330 0x7fa82c1ef100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c430 tx=0x7fa81800c8f0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa82be00640 1 -- 172.21.15.67:0/3727198097 reap_dead start 2024-03-09T09:27:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa832000640 1 -- 172.21.15.67:0/3727198097 shutdown_connections 2024-03-09T09:27:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa832000640 1 --2- 172.21.15.67:0/3727198097 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa82c0c6330 0x7fa82c1ef100 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.443+0000 7fa832000640 1 -- 172.21.15.67:0/3727198097 >> 172.21.15.67:0/3727198097 conn(0x7fa82c0c8050 msgr2=0x7fa82c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.444+0000 7fa832000640 1 -- 172.21.15.67:0/3727198097 shutdown_connections 2024-03-09T09:27:14.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.444+0000 7fa832000640 1 -- 172.21.15.67:0/3727198097 wait complete. 2024-03-09T09:27:14.464 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:27:14.472 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:27:14.472 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:27:14.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:27:14.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.691+0000 7fcc2de00640 1 Processor -- start 2024-03-09T09:27:14.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.691+0000 7fcc2de00640 1 -- start start 2024-03-09T09:27:14.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.693+0000 7fcc2de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcc280c9380 0x7fcc280acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.693+0000 7fcc2de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc280ad440 0x7fcc2806c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.693+0000 7fcc2de00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc280c9890 con 0x7fcc2814fd90 2024-03-09T09:27:14.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.693+0000 7fcc2de00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2806ba60 con 0x7fcc280c9380 2024-03-09T09:27:14.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.693+0000 7fcc2de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280c5f80 con 0x7fcc280ad440 2024-03-09T09:27:14.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.693+0000 7fcc27e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcc280c9380 0x7fcc280acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.693+0000 7fcc27e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcc280c9380 0x7fcc280acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51892/0 (socket says 172.21.15.67:51892) 2024-03-09T09:27:14.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.693+0000 7fcc27e00640 1 -- 172.21.15.67:0/3314768509 learned_addr learned my addr 172.21.15.67:0/3314768509 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:14.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.694+0000 7fcc27e00640 1 -- 172.21.15.67:0/3314768509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc280ad440 msgr2=0x7fcc2806c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:14.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.694+0000 7fcc27e00640 1 --2- 172.21.15.67:0/3314768509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc280ad440 0x7fcc2806c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.694+0000 7fcc27e00640 1 -- 172.21.15.67:0/3314768509 >> v1:172.21.15.134:6789/0 conn(0x7fcc2814fd90 legacy=0x7fcc28156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.694+0000 7fcc27e00640 1 -- 172.21.15.67:0/3314768509 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280c6240 con 0x7fcc280c9380 2024-03-09T09:27:14.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.695+0000 7fcc27e00640 1 --2- 172.21.15.67:0/3314768509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280c9380 0x7fcc280acf00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009870 tx=0x7fcc1402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27947a5a6bc3131 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.695+0000 7fcc27400640 1 -- 172.21.15.67:0/3314768509 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc140317c0 con 0x7fcc280c9380 2024-03-09T09:27:14.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.695+0000 7fcc27400640 1 -- 172.21.15.67:0/3314768509 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc14037070 con 0x7fcc280c9380 2024-03-09T09:27:14.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.695+0000 7fcc27400640 1 -- 172.21.15.67:0/3314768509 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14031b80 con 0x7fcc280c9380 2024-03-09T09:27:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.696+0000 7fcc2de00640 1 -- 172.21.15.67:0/3314768509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280c9380 msgr2=0x7fcc280acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.696+0000 7fcc2de00640 1 --2- 172.21.15.67:0/3314768509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280c9380 0x7fcc280acf00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009870 tx=0x7fcc1402b860 comp rx=0 tx=0).stop 2024-03-09T09:27:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.696+0000 7fcc2de00640 1 -- 172.21.15.67:0/3314768509 shutdown_connections 2024-03-09T09:27:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.696+0000 7fcc2de00640 1 --2- 172.21.15.67:0/3314768509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc280ad440 0x7fcc2806c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.696+0000 7fcc2de00640 1 --2- 172.21.15.67:0/3314768509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280c9380 0x7fcc280acf00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.696+0000 7fcc2de00640 1 -- 172.21.15.67:0/3314768509 >> 172.21.15.67:0/3314768509 conn(0x7fcc280cb480 msgr2=0x7fcc280cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:14.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.696+0000 7fcc2de00640 1 -- 172.21.15.67:0/3314768509 shutdown_connections 2024-03-09T09:27:14.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.696+0000 7fcc2de00640 1 -- 172.21.15.67:0/3314768509 wait complete. 2024-03-09T09:27:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.698+0000 7fcc2de00640 1 Processor -- start 2024-03-09T09:27:14.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.698+0000 7fcc2de00640 1 -- start start 2024-03-09T09:27:14.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.698+0000 7fcc2de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc280ad440 0x7fcc281ef140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.698+0000 7fcc2de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280c9380 0x7fcc281dfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.698+0000 7fcc2de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc2814fd90 0x7fcc281e01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc2de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2806c710 con 0x7fcc280c9380 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc2de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2806bcb0 con 0x7fcc2814fd90 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc2de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2806c890 con 0x7fcc280ad440 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280c9380 0x7fcc281dfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc27e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280c9380 0x7fcc281dfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51900/0 (socket says 172.21.15.67:51900) 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc27e00640 1 -- 172.21.15.67:0/1431126884 learned_addr learned my addr 172.21.15.67:0/1431126884 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc2ca00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc280ad440 0x7fcc281ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc2d400640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc2814fd90 0x7fcc281e01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc27e00640 1 -- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc280ad440 msgr2=0x7fcc281ef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc27e00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcc280ad440 0x7fcc281ef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc27e00640 1 -- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc2814fd90 msgr2=0x7fcc281e01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc27e00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc2814fd90 0x7fcc281e01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.699+0000 7fcc27e00640 1 -- 172.21.15.67:0/1431126884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc281efe00 con 0x7fcc280c9380 2024-03-09T09:27:14.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.700+0000 7fcc27e00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280c9380 0x7fcc281dfc70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009870 tx=0x7fcc140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.700+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14032860 con 0x7fcc280c9380 2024-03-09T09:27:14.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.700+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc14032a20 con 0x7fcc280c9380 2024-03-09T09:27:14.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.700+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14032e10 con 0x7fcc280c9380 2024-03-09T09:27:14.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.700+0000 7fcc2de00640 1 -- 172.21.15.67:0/1431126884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2814a660 con 0x7fcc280c9380 2024-03-09T09:27:14.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.701+0000 7fcc2de00640 1 -- 172.21.15.67:0/1431126884 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2814afd0 con 0x7fcc280c9380 2024-03-09T09:27:14.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.702+0000 7fcc2de00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc281e1d40 0x7fcc281590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.702+0000 7fcc2de00640 1 -- 172.21.15.67:0/1431126884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2804f3c0 con 0x7fcc281e1d40 2024-03-09T09:27:14.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.702+0000 7fcc2ca00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc281e1d40 0x7fcc281590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.703+0000 7fcc2ca00640 1 -- 172.21.15.67:0/1431126884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc281e1aa0 con 0x7fcc281e1d40 2024-03-09T09:27:14.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.703+0000 7fcc2ca00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc281e1d40 0x7fcc281590f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000f60 tx=0x7fcc1c00f9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.703+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcc1404c060 con 0x7fcc280c9380 2024-03-09T09:27:14.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.704+0000 7fcc24c00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc0806b4d0 0x7fcc0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.704+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6721+0+0 (secure 0 0 0) 0x7fcc140ac000 con 0x7fcc280c9380 2024-03-09T09:27:14.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.704+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0052e0 con 0x7fcc281e1d40 2024-03-09T09:27:14.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.704+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fcc1c005480 con 0x7fcc281e1d40 2024-03-09T09:27:14.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.704+0000 7fcc2d400640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc0806b4d0 0x7fcc0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.704+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc281e1d40 msgr2=0x7fcc281590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.704+0000 7fcc24c00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc281e1d40 0x7fcc281590f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000f60 tx=0x7fcc1c00f9d0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.704+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc281e1d40 msgr2=0x7fcc281590f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:14.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.705+0000 7fcc24c00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc281e1d40 0x7fcc281590f0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000f60 tx=0x7fcc1c00f9d0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.705+0000 7fcc2d400640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc0806b4d0 0x7fcc0806d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009f60 tx=0x7fcc2000d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.712+0000 7fcc2de00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc28159e80 0x7fcc28159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.712+0000 7fcc2de00640 1 -- 172.21.15.67:0/1431126884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2806d2d0 con 0x7fcc28159e80 2024-03-09T09:27:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.712+0000 7fcc2ca00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc28159e80 0x7fcc28159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.713+0000 7fcc2ca00640 1 -- 172.21.15.67:0/1431126884 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcc2814f120 con 0x7fcc28159e80 2024-03-09T09:27:14.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.713+0000 7fcc2ca00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc28159e80 0x7fcc28159630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000f60 tx=0x7fcc1c001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.713+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c006c60 con 0x7fcc28159e80 2024-03-09T09:27:14.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4875 (secure 0 0 0) 0x7fcc1c006c60 con 0x7fcc28159e80 2024-03-09T09:27:14.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc28159e80 msgr2=0x7fcc28159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc24c00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc28159e80 0x7fcc28159630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000f60 tx=0x7fcc1c001080 comp rx=0 tx=0).stop 2024-03-09T09:27:14.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc24c00640 1 -- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc28159e80 msgr2=0x7fcc28159630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:14.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc24c00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc28159e80 0x7fcc28159630 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000f60 tx=0x7fcc1c001080 comp rx=0 tx=0).stop 2024-03-09T09:27:14.721 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:27:14.721 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:14.722 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:14.722 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:27:14.722 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 276, 2024-03-09T09:27:14.722 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:27:14.722 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:27:14.722 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:27:14.722 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:27:14.722 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:14.722 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:27:14.722 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:14.723 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:14.724 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:27:14.725 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:14.726 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:14.727 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:14.728 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:14.729 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:14.730 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.731 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:14.732 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:14.733 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:27:14.734 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:27:14.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc2de00640 1 -- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc0806b4d0 msgr2=0x7fcc0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc2de00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcc0806b4d0 0x7fcc0806d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009f60 tx=0x7fcc2000d040 comp rx=0 tx=0).stop 2024-03-09T09:27:14.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc2de00640 1 -- 172.21.15.67:0/1431126884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280c9380 msgr2=0x7fcc281dfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc2de00640 1 --2- 172.21.15.67:0/1431126884 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc280c9380 0x7fcc281dfc70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009870 tx=0x7fcc140046c0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc2d400640 1 -- 172.21.15.67:0/1431126884 reap_dead start 2024-03-09T09:27:14.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc2d400640 1 -- 172.21.15.67:0/1431126884 reap_dead start 2024-03-09T09:27:14.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc2de00640 1 -- 172.21.15.67:0/1431126884 shutdown_connections 2024-03-09T09:27:14.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc2de00640 1 -- 172.21.15.67:0/1431126884 >> 172.21.15.67:0/1431126884 conn(0x7fcc280cb480 msgr2=0x7fcc280c6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:14.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc2de00640 1 -- 172.21.15.67:0/1431126884 shutdown_connections 2024-03-09T09:27:14.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.714+0000 7fcc2de00640 1 -- 172.21.15.67:0/1431126884 wait complete. 2024-03-09T09:27:14.739 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:27:14.739 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:27:14.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:27:14.799 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ 2024-03-09T09:27:14.803 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/pt.po 2024-03-09T09:27:14.810 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/sl.po 2024-03-09T09:27:14.818 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/zh_TW.po 2024-03-09T09:27:14.823 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/sk.po 2024-03-09T09:27:14.832 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/el.gmo 2024-03-09T09:27:14.833 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/en.gmo 2024-03-09T09:27:14.833 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/lg.po 2024-03-09T09:27:14.840 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/sv.gmo 2024-03-09T09:27:14.844 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/pl.gmo 2024-03-09T09:27:14.848 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/insert-header.sin 2024-03-09T09:27:14.848 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/sk.gmo 2024-03-09T09:27:14.849 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ja.gmo 2024-03-09T09:27:14.850 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ko.po 2024-03-09T09:27:14.856 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/da.gmo 2024-03-09T09:27:14.861 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/LINGUAS 2024-03-09T09:27:14.861 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/zh_TW.gmo 2024-03-09T09:27:14.862 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/tr.gmo 2024-03-09T09:27:14.865 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/gl.gmo 2024-03-09T09:27:14.865 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/fi.po 2024-03-09T09:27:14.871 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/da.po 2024-03-09T09:27:14.877 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/fi.gmo 2024-03-09T09:27:14.879 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/quot.sed 2024-03-09T09:27:14.879 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/stamp-po 2024-03-09T09:27:14.879 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ms.gmo 2024-03-09T09:27:14.879 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/pt_BR.po 2024-03-09T09:27:14.882 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/Makevars 2024-03-09T09:27:14.883 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/boldquot.sed 2024-03-09T09:27:14.883 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/coreutils.pot 2024-03-09T09:27:14.885 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/de.po 2024-03-09T09:27:14.890 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/pt.gmo 2024-03-09T09:27:14.892 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/Makefile.in.in 2024-03-09T09:27:14.893 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/nl.gmo 2024-03-09T09:27:14.896 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/fr.gmo 2024-03-09T09:27:14.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ga.gmo 2024-03-09T09:27:14.902 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/cs.po 2024-03-09T09:27:14.907 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ru.po 2024-03-09T09:27:14.911 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/hu.po 2024-03-09T09:27:14.915 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/id.po 2024-03-09T09:27:14.919 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ru.gmo 2024-03-09T09:27:14.922 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/sl.gmo 2024-03-09T09:27:14.925 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/remove-potcdate.sin 2024-03-09T09:27:14.925 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/hu.gmo 2024-03-09T09:27:14.927 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/et.po 2024-03-09T09:27:14.929 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/gl.po 2024-03-09T09:27:14.933 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/vi.gmo 2024-03-09T09:27:14.935 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/bg.po 2024-03-09T09:27:14.939 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/pl.po 2024-03-09T09:27:14.942 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/nl.po 2024-03-09T09:27:14.945 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/pt_BR.gmo 2024-03-09T09:27:14.947 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ko.gmo 2024-03-09T09:27:14.947 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/de.gmo 2024-03-09T09:27:14.949 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/POTFILES.in 2024-03-09T09:27:14.950 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/sv.po 2024-03-09T09:27:14.952 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/it.po 2024-03-09T09:27:14.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/eu.gmo 2024-03-09T09:27:14.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/en.po 2024-03-09T09:27:14.957 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/it.gmo 2024-03-09T09:27:14.959 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/en@boldquot.header 2024-03-09T09:27:14.959 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/et.gmo 2024-03-09T09:27:14.960 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ja.po 2024-03-09T09:27:14.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/zh_CN.gmo 2024-03-09T09:27:14.966 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/uk.po 2024-03-09T09:27:14.969 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ca.gmo 2024-03-09T09:27:14.971 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/Rules-quot 2024-03-09T09:27:14.972 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ro.po 2024-03-09T09:27:14.973 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/nb.gmo 2024-03-09T09:27:14.973 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/bg.gmo 2024-03-09T09:27:14.975 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/be.po 2024-03-09T09:27:14.977 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/vi.po 2024-03-09T09:27:14.979 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/lt.gmo 2024-03-09T09:27:14.980 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/uk.gmo 2024-03-09T09:27:14.981 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/id.gmo 2024-03-09T09:27:14.983 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/af.po 2024-03-09T09:27:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.976+0000 7efc4a000640 1 Processor -- start 2024-03-09T09:27:14.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.977+0000 7efc4a000640 1 -- start start 2024-03-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.978+0000 7efc4a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc440acf00 0x7efc440c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.978+0000 7efc4a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc440c9380 0x7efc44156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.978+0000 7efc4a000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc440c9810 con 0x7efc440c5e00 2024-03-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.978+0000 7efc4a000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4406d740 con 0x7efc440c9380 2024-03-09T09:27:14.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.978+0000 7efc4a000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc44153680 con 0x7efc440acf00 2024-03-09T09:27:14.986 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/eu.po 2024-03-09T09:27:14.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.978+0000 7efc49600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7efc440c9380 0x7efc44156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.978+0000 7efc48c00640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7efc440c5e00 0x7efc4406c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:40594/0 (socket says 172.21.15.67:40594) 2024-03-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.978+0000 7efc43e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc440acf00 0x7efc440c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.978+0000 7efc48c00640 1 -- 172.21.15.67:0/1899483076 learned_addr learned my addr 172.21.15.67:0/1899483076 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.979+0000 7efc49600640 1 -- 172.21.15.67:0/1899483076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc440acf00 msgr2=0x7efc440c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.979+0000 7efc49600640 1 --2- 172.21.15.67:0/1899483076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc440acf00 0x7efc440c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.979+0000 7efc49600640 1 -- 172.21.15.67:0/1899483076 >> v1:172.21.15.134:6789/0 conn(0x7efc440c5e00 legacy=0x7efc4406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.979+0000 7efc49600640 1 -- 172.21.15.67:0/1899483076 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc440ad7c0 con 0x7efc440c9380 2024-03-09T09:27:14.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.979+0000 7efc43400640 1 -- 172.21.15.67:0/1899483076 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944066676 0 0) 0x7efc440c9810 con 0x7efc440c5e00 2024-03-09T09:27:14.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.980+0000 7efc49600640 1 --2- 172.21.15.67:0/1899483076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc440c9380 0x7efc44156660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009dc0 tx=0x7efc3c02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c6dc70104c60a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.988 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ms.po 2024-03-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.980+0000 7efc43400640 1 -- 172.21.15.67:0/1899483076 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c039070 con 0x7efc440c9380 2024-03-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.980+0000 7efc43400640 1 -- 172.21.15.67:0/1899483076 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c02cc40 con 0x7efc440c9380 2024-03-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.980+0000 7efc43400640 1 -- 172.21.15.67:0/1899483076 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c034440 con 0x7efc440c9380 2024-03-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.981+0000 7efc4a000640 1 -- 172.21.15.67:0/1899483076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc440c9380 msgr2=0x7efc44156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.981+0000 7efc4a000640 1 --2- 172.21.15.67:0/1899483076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc440c9380 0x7efc44156660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009dc0 tx=0x7efc3c02be90 comp rx=0 tx=0).stop 2024-03-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.981+0000 7efc4a000640 1 -- 172.21.15.67:0/1899483076 shutdown_connections 2024-03-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.981+0000 7efc4a000640 1 --2- 172.21.15.67:0/1899483076 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc440c9380 0x7efc44156660 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.981+0000 7efc4a000640 1 --2- 172.21.15.67:0/1899483076 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc440acf00 0x7efc440c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.981+0000 7efc4a000640 1 -- 172.21.15.67:0/1899483076 >> 172.21.15.67:0/1899483076 conn(0x7efc440cb480 msgr2=0x7efc440cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:14.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.982+0000 7efc4a000640 1 -- 172.21.15.67:0/1899483076 shutdown_connections 2024-03-09T09:27:14.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.982+0000 7efc4a000640 1 -- 172.21.15.67:0/1899483076 wait complete. 2024-03-09T09:27:14.991 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/en@quot.header 2024-03-09T09:27:14.992 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/af.gmo 2024-03-09T09:27:14.992 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/es.gmo 2024-03-09T09:27:14.992 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ga.po 2024-03-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.984+0000 7efc4a000640 1 Processor -- start 2024-03-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.984+0000 7efc4a000640 1 -- start start 2024-03-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.985+0000 7efc4a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc440acf00 0x7efc440c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.985+0000 7efc4a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc440c5e00 0x7efc440c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.985+0000 7efc4a000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc440c9380 0x7efc440bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc4a000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4406bde0 con 0x7efc440c5e00 2024-03-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc4a000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4406bb00 con 0x7efc440c9380 2024-03-09T09:27:14.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc4a000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4406c0c0 con 0x7efc440acf00 2024-03-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc440c5e00 0x7efc440c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc43e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc440c5e00 0x7efc440c14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51930/0 (socket says 172.21.15.67:51930) 2024-03-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc48c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc440acf00 0x7efc440c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc43e00640 1 -- 172.21.15.67:0/2863313483 learned_addr learned my addr 172.21.15.67:0/2863313483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc49600640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc440c9380 0x7efc440bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.995 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ro.gmo 2024-03-09T09:27:14.996 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ca.po 2024-03-09T09:27:14.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc43e00640 1 -- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc440acf00 msgr2=0x7efc440c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc43e00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc440acf00 0x7efc440c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc43e00640 1 -- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc440c9380 msgr2=0x7efc440bf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc43e00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc440c9380 0x7efc440bf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.986+0000 7efc43e00640 1 -- 172.21.15.67:0/2863313483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc440c5850 con 0x7efc440c5e00 2024-03-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.987+0000 7efc43e00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc440c5e00 0x7efc440c14f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc3000e9e0 tx=0x7efc3000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.987+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30017070 con 0x7efc440c5e00 2024-03-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.987+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3000c9d0 con 0x7efc440c5e00 2024-03-09T09:27:14.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.987+0000 7efc4a000640 1 -- 172.21.15.67:0/2863313483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc441588c0 con 0x7efc440c5e00 2024-03-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.988+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3000cdc0 con 0x7efc440c5e00 2024-03-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.988+0000 7efc4a000640 1 -- 172.21.15.67:0/2863313483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc44158e80 con 0x7efc440c5e00 2024-03-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.989+0000 7efc4a000640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159e30 0x7efc440bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.989+0000 7efc4a000640 1 -- 172.21.15.67:0/2863313483 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc440c61e0 con 0x7efc44159e30 2024-03-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.989+0000 7efc48c00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159e30 0x7efc440bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:14.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.990+0000 7efc48c00640 1 -- 172.21.15.67:0/2863313483 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc4404f3c0 con 0x7efc44159e30 2024-03-09T09:27:14.999 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/lg.gmo 2024-03-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.990+0000 7efc48c00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159e30 0x7efc440bfa20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efc380011e0 tx=0x7efc38001990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.991+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3800f600 con 0x7efc44159e30 2024-03-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.991+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7efc3001a070 con 0x7efc440c5e00 2024-03-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.992+0000 7efc40c00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efc2406b420 0x7efc2406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:15.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.992+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7efc38018070 con 0x7efc44159e30 2024-03-09T09:27:15.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.992+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159e30 msgr2=0x7efc440bfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:15.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.992+0000 7efc40c00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159e30 0x7efc440bfa20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efc380011e0 tx=0x7efc38001990 comp rx=0 tx=0).stop 2024-03-09T09:27:15.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.992+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159e30 msgr2=0x7efc440bfa20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.992+0000 7efc40c00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159e30 0x7efc440bfa20 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efc380011e0 tx=0x7efc38001990 comp rx=0 tx=0).stop 2024-03-09T09:27:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.992+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6721+0+0 (secure 0 0 0) 0x7efc3008a810 con 0x7efc440c5e00 2024-03-09T09:27:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.992+0000 7efc49600640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efc2406b420 0x7efc2406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:15.003 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/zh_CN.po 2024-03-09T09:27:15.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:14.993+0000 7efc49600640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efc2406b420 0x7efc2406d8e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7efc3c036000 tx=0x7efc3c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:15.004 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/cs.gmo 2024-03-09T09:27:15.005 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/nb.po 2024-03-09T09:27:15.009 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/ChangeLog-2007 2024-03-09T09:27:15.009 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/tr.po 2024-03-09T09:27:15.010 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/be.gmo 2024-03-09T09:27:15.011 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/fr.po 2024-03-09T09:27:15.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.005+0000 7efc4a000640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159050 0x7efc441ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:15.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.005+0000 7efc4a000640 1 -- 172.21.15.67:0/2863313483 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc440cb900 con 0x7efc44159050 2024-03-09T09:27:15.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.006+0000 7efc48c00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159050 0x7efc441ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:15.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.007+0000 7efc48c00640 1 -- 172.21.15.67:0/2863313483 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efc441e8d80 con 0x7efc44159050 2024-03-09T09:27:15.014 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/el.po 2024-03-09T09:27:15.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.007+0000 7efc48c00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159050 0x7efc441ee7c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc380011e0 tx=0x7efc38001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.008+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3801dd90 con 0x7efc44159050 2024-03-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.008+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7efc380133f0 con 0x7efc44159050 2024-03-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.008+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159050 msgr2=0x7efc441ee7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.008+0000 7efc40c00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159050 0x7efc441ee7c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc380011e0 tx=0x7efc38001080 comp rx=0 tx=0).stop 2024-03-09T09:27:15.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.008+0000 7efc40c00640 1 -- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159050 msgr2=0x7efc441ee7c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.008+0000 7efc40c00640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc44159050 0x7efc441ee7c0 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc380011e0 tx=0x7efc38001080 comp rx=0 tx=0).stop 2024-03-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 276, 2024-03-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:27:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 14, 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:27:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:27:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.024 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:15.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:15.026 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:27:15.027 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:27:15.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:27:15.029 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.009+0000 7efc4a000640 1 -- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efc2406b420 msgr2=0x7efc2406d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.009+0000 7efc4a000640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7efc2406b420 0x7efc2406d8e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7efc3c036000 tx=0x7efc3c030000 comp rx=0 tx=0).stop 2024-03-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.009+0000 7efc4a000640 1 -- 172.21.15.67:0/2863313483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc440c5e00 msgr2=0x7efc440c14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.009+0000 7efc4a000640 1 --2- 172.21.15.67:0/2863313483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc440c5e00 0x7efc440c14f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc3000e9e0 tx=0x7efc3000eea0 comp rx=0 tx=0).stop 2024-03-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.009+0000 7efc49600640 1 -- 172.21.15.67:0/2863313483 reap_dead start 2024-03-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.009+0000 7efc49600640 1 -- 172.21.15.67:0/2863313483 reap_dead start 2024-03-09T09:27:15.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.009+0000 7efc4a000640 1 -- 172.21.15.67:0/2863313483 shutdown_connections 2024-03-09T09:27:15.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.009+0000 7efc4a000640 1 -- 172.21.15.67:0/2863313483 >> 172.21.15.67:0/2863313483 conn(0x7efc440cb480 msgr2=0x7efc440ac6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:15.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.009+0000 7efc4a000640 1 -- 172.21.15.67:0/2863313483 shutdown_connections 2024-03-09T09:27:15.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.010+0000 7efc4a000640 1 -- 172.21.15.67:0/2863313483 wait complete. 2024-03-09T09:27:15.032 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/lt.po 2024-03-09T09:27:15.032 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/po/es.po 2024-03-09T09:27:15.032 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/ 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/skip-seek 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/unblock-sync 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/skip-seek2 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/unblock 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/stderr 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/direct 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/skip-seek-past-file 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/not-rewound 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/skip-seek-past-dev 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/misc 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/dd/reblock 2024-03-09T09:27:15.033 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ 2024-03-09T09:27:15.034 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:27:15.039 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-03-09T09:27:15.039 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:27:15.039 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-03-09T09:27:15.040 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:27:15.040 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:27:15.040 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:27:15.040 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tt-FF 2024-03-09T09:27:15.050 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/t_tab_ 2024-03-09T09:27:15.050 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/FF 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/FnFn 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/o3ml24f-bl-tn 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+5-8b3l17f-FF 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+3ml24f-bl-tn 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/FFtn 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/a2l15-FF 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tFnFt 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tn2e5o3-t_tab 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3ml24f-t 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/nSml20-t-tFFFF 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tn2e8-t_tab 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/b3f-FF 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/2w60f-t_notab 2024-03-09T09:27:15.051 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/FFn 2024-03-09T09:27:15.052 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/t_tab 2024-03-09T09:27:15.052 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/b3-0F 2024-03-09T09:27:15.052 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+4-8a2l17-FF 2024-03-09T09:27:15.052 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+4b2l17f-0FF 2024-03-09T09:27:15.052 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3a3l15-t 2024-03-09T09:27:15.052 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tFFn 2024-03-09T09:27:15.052 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3l24f-t 2024-03-09T09:27:15.052 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/0FnFt 2024-03-09T09:27:15.052 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3-FF 2024-03-09T09:27:15.052 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/t-FF 2024-03-09T09:27:15.055 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/nSml24-bl-FF 2024-03-09T09:27:15.055 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+3l24f-bl 2024-03-09T09:27:15.055 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/b3l15-t 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3b3f-FF 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3b3f-0F 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W35b3l24f-lm 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/0Ft 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tt-bl 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3ml24-t-FF 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tFFt-lm 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/nSml20-bl-FF 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tn2e8o3-t_tab 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W28l24f-ll 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/nSml24-t-tFFFF 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/t-0FF 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tb3-FF 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/o3b3Sl24f-tn 2024-03-09T09:27:15.056 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/o3mSl24f-bl-tn 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3b2l17-FF 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/2f-t_notab 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/b2l15-FF 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3b3f-0FF 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/a3l15-t 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/o3Jml24f-lm-lo 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tb3-0FF 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/l24-t 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W35ml24f-lm-lo 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/a3-0F 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ml24f-0F 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/loli 2024-03-09T09:27:15.057 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3ml24-t 2024-03-09T09:27:15.058 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/a3l24f-lm 2024-03-09T09:27:15.058 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/Jb3l24f-lm 2024-03-09T09:27:15.058 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/t-t 2024-03-09T09:27:15.058 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/0F 2024-03-09T09:27:15.059 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/w72l24f-ll 2024-03-09T09:27:15.059 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ml24f-t 2024-03-09T09:27:15.059 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/b3l15f-t 2024-03-09T09:27:15.059 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/0FnFnt 2024-03-09T09:27:15.060 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/b3l24f-lm 2024-03-09T09:27:15.060 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/o3mSnl24fbltn 2024-03-09T09:27:15.060 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/nJml24f-lmlolm 2024-03-09T09:27:15.060 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/o3a3Sl24f-tn 2024-03-09T09:27:15.060 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tne8-t_tab 2024-03-09T09:27:15.061 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3ml24-FF 2024-03-09T09:27:15.061 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/a3f-0FF 2024-03-09T09:27:15.061 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ml20-FF-t 2024-03-09T09:27:15.061 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/pr-tests 2024-03-09T09:27:15.062 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W35Jb3l24f-lm 2024-03-09T09:27:15.062 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ttb3-0FF 2024-03-09T09:27:15.062 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/nl24f-bl 2024-03-09T09:27:15.065 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/nSml24-t-t-FF 2024-03-09T09:27:15.065 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3b3l15f-t 2024-03-09T09:27:15.065 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/0FF 2024-03-09T09:27:15.065 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3f-0F 2024-03-09T09:27:15.066 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/Ja3l24f-lm 2024-03-09T09:27:15.066 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3a2l17-FF 2024-03-09T09:27:15.066 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ml24f-t-0F 2024-03-09T09:27:15.066 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tn_2e8-t_tab 2024-03-09T09:27:15.066 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/o3a3Snl24f-tn 2024-03-09T09:27:15.066 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/2s_w60f-t_nota 2024-03-09T09:27:15.066 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/t-bl 2024-03-09T09:27:15.066 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ml24-t-FF 2024-03-09T09:27:15.066 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+3-7l24-FF 2024-03-09T09:27:15.066 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/t_notab 2024-03-09T09:27:15.066 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W72Jl24f-ll 2024-03-09T09:27:15.067 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/a2l17-FF 2024-03-09T09:27:15.067 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ml24-FF 2024-03-09T09:27:15.067 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/b3f-0F 2024-03-09T09:27:15.067 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+3l24f-0FF 2024-03-09T09:27:15.067 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+3ml20f-bl-FF 2024-03-09T09:27:15.067 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/a3f-0F 2024-03-09T09:27:15.068 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W35Jml24f-lmlo 2024-03-09T09:27:15.068 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W27l24f-ll 2024-03-09T09:27:15.068 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ml24-t 2024-03-09T09:27:15.068 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/a3f-FF 2024-03-09T09:27:15.069 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tFFt-bl 2024-03-09T09:27:15.069 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tta3-FF 2024-03-09T09:27:15.069 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3-0F 2024-03-09T09:27:15.069 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/2sf-t_notab 2024-03-09T09:27:15.076 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3a3l15f-t 2024-03-09T09:27:15.076 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W26l24f-ll 2024-03-09T09:27:15.076 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+7l24-FF 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ml24f-lm-lo 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/l24-FF 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+2-5l24f-0FF 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3f-FF 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+3ml24f-tn-bl 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ta3-0FF 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3b3l15-t 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+2l24f-0FF 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tn_2e8S-t_tab 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ta3-FF 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tFFt-ll 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+6b3l13f-FF 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/2-Sf-t_notab 2024-03-09T09:27:15.077 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/0FFnt 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W20l24f-ll 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/Jml24f-lm-lo 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3l24-t 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/o3b3l24f-tn 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W35a3l24f-lm 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/2-S_f-t_notab 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3-5l24f-t 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/o3b3Snl24f-tn 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+2l24f-bl 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W35Ja3l24f-lm 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+8l20-FF 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/0FFt 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/o3a3l24f-tn 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/2s_f-t_notab 2024-03-09T09:27:15.078 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/nN15l24f-bl 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/0Fnt 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/4-7l24-FF 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tn 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/b2l17-FF 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/nJml24f-lmlmlo 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tta3-0FF 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+5a3l13f-0FF 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/nN1+3l24f-bl 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/2sw60f-t_notab 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/W-72l24f-ll 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tt-0FF 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/4l24-FF 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/3a3f-0F 2024-03-09T09:27:15.079 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tFFt 2024-03-09T09:27:15.080 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/n+6a2l17-FF 2024-03-09T09:27:15.080 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/a3l15f-t 2024-03-09T09:27:15.080 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/l24f-t 2024-03-09T09:27:15.080 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/b3f-0FF 2024-03-09T09:27:15.080 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tt-t 2024-03-09T09:27:15.080 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/ttb3-FF 2024-03-09T09:27:15.080 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/nSml20-t-t-FF 2024-03-09T09:27:15.080 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/pr/tne8o3-t_tab 2024-03-09T09:27:15.080 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/Coreutils.pm 2024-03-09T09:27:15.081 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:27:15.081 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:27:15.081 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:27:15.148 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:27:15.156 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:27:15.275 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rmdir/ 2024-03-09T09:27:15.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.315+0000 7f0ee4000640 1 Processor -- start 2024-03-09T09:27:15.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.315+0000 7f0ee4000640 1 -- start start 2024-03-09T09:27:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.316+0000 7f0ee4000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc0c1d70 0x7f0edc0ac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.316+0000 7f0ee4000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0edc0c2400 0x7f0edc0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.316+0000 7f0ee4000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0edc156d50 con 0x7f0edc153370 2024-03-09T09:27:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.316+0000 7f0ee4000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0edc0aadf0 con 0x7f0edc0c2400 2024-03-09T09:27:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.316+0000 7f0ee4000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc0aa4c0 con 0x7f0edc0c1d70 2024-03-09T09:27:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.316+0000 7f0ee0c00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc0c1d70 msgr2=0x7f0edc0ac610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.317+0000 7f0ee0c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc0c1d70 0x7f0edc0ac610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.317+0000 7f0ee2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0edc0c2400 0x7f0edc0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.317+0000 7f0ee2000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0edc0c2400 0x7f0edc0acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52002/0 (socket says 172.21.15.67:52002) 2024-03-09T09:27:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.317+0000 7f0ee1600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0edc153370 0x7f0edc0c07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:47192/0 (socket says 172.21.15.67:47192) 2024-03-09T09:27:15.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.317+0000 7f0ee2000640 1 -- 172.21.15.67:0/301794565 learned_addr learned my addr 172.21.15.67:0/301794565 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.317+0000 7f0ee2000640 1 -- 172.21.15.67:0/301794565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc0c1d70 msgr2=0x7f0edc0ac610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:27:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.317+0000 7f0ee2000640 1 --2- 172.21.15.67:0/301794565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc0c1d70 0x7f0edc0ac610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.317+0000 7f0ee2000640 1 -- 172.21.15.67:0/301794565 >> v1:172.21.15.67:6789/0 conn(0x7f0edc153370 legacy=0x7f0edc0c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.317+0000 7f0ee2000640 1 -- 172.21.15.67:0/301794565 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc0aa7a0 con 0x7f0edc0c2400 2024-03-09T09:27:15.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.318+0000 7f0ee2000640 1 --2- 172.21.15.67:0/301794565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0edc0c2400 0x7f0edc0acb50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009e50 tx=0x7f0ed402bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83652162bb60b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.318+0000 7f0edbe00640 1 -- 172.21.15.67:0/301794565 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed402caa0 con 0x7f0edc0c2400 2024-03-09T09:27:15.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.319+0000 7f0edbe00640 1 -- 172.21.15.67:0/301794565 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed402cc40 con 0x7f0edc0c2400 2024-03-09T09:27:15.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.319+0000 7f0edbe00640 1 -- 172.21.15.67:0/301794565 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40343f0 con 0x7f0edc0c2400 2024-03-09T09:27:15.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.319+0000 7f0ee4000640 1 -- 172.21.15.67:0/301794565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0edc0c2400 msgr2=0x7f0edc0acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:15.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.319+0000 7f0ee4000640 1 --2- 172.21.15.67:0/301794565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0edc0c2400 0x7f0edc0acb50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009e50 tx=0x7f0ed402bf00 comp rx=0 tx=0).stop 2024-03-09T09:27:15.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.319+0000 7f0ee4000640 1 -- 172.21.15.67:0/301794565 shutdown_connections 2024-03-09T09:27:15.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.319+0000 7f0ee4000640 1 --2- 172.21.15.67:0/301794565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0edc0c2400 0x7f0edc0acb50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:15.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.319+0000 7f0ee4000640 1 --2- 172.21.15.67:0/301794565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc0c1d70 0x7f0edc0ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:15.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.319+0000 7f0ee4000640 1 -- 172.21.15.67:0/301794565 >> 172.21.15.67:0/301794565 conn(0x7f0edc0c7370 msgr2=0x7f0edc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:15.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.319+0000 7f0ee4000640 1 -- 172.21.15.67:0/301794565 shutdown_connections 2024-03-09T09:27:15.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.319+0000 7f0ee4000640 1 -- 172.21.15.67:0/301794565 wait complete. 2024-03-09T09:27:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.321+0000 7f0ee4000640 1 Processor -- start 2024-03-09T09:27:15.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.321+0000 7f0ee4000640 1 -- start start 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.322+0000 7f0ee4000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0edc0c1d70 0x7f0edc14e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.322+0000 7f0ee4000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0edc0c2400 0x7f0edc14c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.322+0000 7f0ee4000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc153370 0x7f0edc14cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.322+0000 7f0ee4000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc0c1030 con 0x7f0edc0c2400 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.322+0000 7f0ee4000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc0c0eb0 con 0x7f0edc0c1d70 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.322+0000 7f0ee4000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc153750 con 0x7f0edc153370 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.322+0000 7f0ee2000640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc153370 msgr2=0x7f0edc14cbb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.322+0000 7f0ee0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0edc0c2400 0x7f0edc14c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.323+0000 7f0ee2000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc153370 0x7f0edc14cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.323+0000 7f0ee0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0edc0c2400 0x7f0edc14c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52004/0 (socket says 172.21.15.67:52004) 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.323+0000 7f0ee0c00640 1 -- 172.21.15.67:0/4273284091 learned_addr learned my addr 172.21.15.67:0/4273284091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.323+0000 7f0ee1600640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0edc0c1d70 0x7f0edc14e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:15.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.323+0000 7f0ee0c00640 1 -- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc153370 msgr2=0x7f0edc14cbb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:15.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.323+0000 7f0ee0c00640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc153370 0x7f0edc14cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:15.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.323+0000 7f0ee0c00640 1 -- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0edc0c1d70 msgr2=0x7f0edc14e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:15.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.323+0000 7f0ee0c00640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0edc0c1d70 0x7f0edc14e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:15.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.323+0000 7f0ee0c00640 1 -- 172.21.15.67:0/4273284091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc159410 con 0x7f0edc0c2400 2024-03-09T09:27:15.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.323+0000 7f0ee0c00640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0edc0c2400 0x7f0edc14c670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800b530 tx=0x7f0ec800b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:15.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.324+0000 7f0ed9600640 1 -- 172.21.15.67:0/4273284091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8013020 con 0x7f0edc0c2400 2024-03-09T09:27:15.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.324+0000 7f0ed9600640 1 -- 172.21.15.67:0/4273284091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8003dc0 con 0x7f0edc0c2400 2024-03-09T09:27:15.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.324+0000 7f0ed9600640 1 -- 172.21.15.67:0/4273284091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8005290 con 0x7f0edc0c2400 2024-03-09T09:27:15.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.324+0000 7f0ee4000640 1 -- 172.21.15.67:0/4273284091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc071fa0 con 0x7f0edc0c2400 2024-03-09T09:27:15.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.324+0000 7f0ee4000640 1 -- 172.21.15.67:0/4273284091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc072520 con 0x7f0edc0c2400 2024-03-09T09:27:15.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.325+0000 7f0ee4000640 1 -- 172.21.15.67:0/4273284091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0edc075a00 con 0x7f0edc0c2400 2024-03-09T09:27:15.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.326+0000 7f0ed9600640 1 -- 172.21.15.67:0/4273284091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0ec8016020 con 0x7f0edc0c2400 2024-03-09T09:27:15.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.327+0000 7f0ed9600640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ebc06b480 0x7f0ebc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:15.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.327+0000 7f0ed9600640 1 -- 172.21.15.67:0/4273284091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f0ec808a080 con 0x7f0edc0c2400 2024-03-09T09:27:15.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.328+0000 7f0ee1600640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ebc06b480 0x7f0ebc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:15.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.330+0000 7f0ee1600640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ebc06b480 0x7f0ebc06d940 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0730d0 tx=0x7f0ed0007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:15.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.333+0000 7f0ed9600640 1 -- 172.21.15.67:0/4273284091 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0ec801d020 con 0x7f0edc0c2400 2024-03-09T09:27:15.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.681+0000 7f0ee4000640 1 -- 172.21.15.67:0/4273284091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0edc151d20 con 0x7f0edc0c2400 2024-03-09T09:27:15.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.682+0000 7f0ed9600640 1 -- 172.21.15.67:0/4273284091 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f0ec8058b90 con 0x7f0edc0c2400 2024-03-09T09:27:15.689 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":276,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:27:15.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 -- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ebc06b480 msgr2=0x7f0ebc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:15.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ebc06b480 0x7f0ebc06d940 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0730d0 tx=0x7f0ed0007e80 comp rx=0 tx=0).stop 2024-03-09T09:27:15.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 -- 172.21.15.67:0/4273284091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0edc0c2400 msgr2=0x7f0edc14c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:15.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0edc0c2400 0x7f0edc14c670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800b530 tx=0x7f0ec800b9f0 comp rx=0 tx=0).stop 2024-03-09T09:27:15.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 -- 172.21.15.67:0/4273284091 shutdown_connections 2024-03-09T09:27:15.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0ebc06b480 0x7f0ebc06d940 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:15.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0edc153370 0x7f0edc14cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:15.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0edc0c2400 0x7f0edc14c670 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:15.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 --2- 172.21.15.67:0/4273284091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0edc0c1d70 0x7f0edc14e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:15.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 -- 172.21.15.67:0/4273284091 >> 172.21.15.67:0/4273284091 conn(0x7f0edc0c7370 msgr2=0x7f0edc150a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:15.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 -- 172.21.15.67:0/4273284091 shutdown_connections 2024-03-09T09:27:15.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:15.685+0000 7f0ee4000640 1 -- 172.21.15.67:0/4273284091 wait complete. 2024-03-09T09:27:15.863 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rmdir/t-slash 2024-03-09T09:27:15.870 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rmdir/ignore 2024-03-09T09:27:15.870 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rmdir/fail-perm 2024-03-09T09:27:15.870 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/ 2024-03-09T09:27:15.877 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/ignorable 2024-03-09T09:27:15.882 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/r-2 2024-03-09T09:27:15.883 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/fail-2eperm 2024-03-09T09:27:15.883 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/empty-inacc 2024-03-09T09:27:15.883 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/cycle 2024-03-09T09:27:15.884 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/sunos-1 2024-03-09T09:27:15.884 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/v-slash 2024-03-09T09:27:15.884 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/no-give-up 2024-03-09T09:27:15.884 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/dot-rel 2024-03-09T09:27:15.885 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/one-file-system 2024-03-09T09:27:15.885 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/i-never 2024-03-09T09:27:15.885 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/rm1 2024-03-09T09:27:15.886 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/unread2 2024-03-09T09:27:15.887 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/read-only 2024-03-09T09:27:15.887 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/deep-2 2024-03-09T09:27:15.887 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/r-4 2024-03-09T09:27:15.888 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/dir-no-w 2024-03-09T09:27:15.888 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/interactive-always 2024-03-09T09:27:15.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/one-file-system2 2024-03-09T09:27:15.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/unreadable 2024-03-09T09:27:15.890 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/r-1 2024-03-09T09:27:15.890 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/fail-eacces 2024-03-09T09:27:15.891 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/inaccessible 2024-03-09T09:27:15.891 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/unread3 2024-03-09T09:27:15.891 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/ir-1 2024-03-09T09:27:15.892 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/rm3 2024-03-09T09:27:15.892 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/rm5 2024-03-09T09:27:15.892 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/deep-1 2024-03-09T09:27:15.893 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/ext3-perf 2024-03-09T09:27:15.893 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/dangling-symlink 2024-03-09T09:27:15.894 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/isatty 2024-03-09T09:27:15.894 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/i-1 2024-03-09T09:27:15.895 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/dir-nonrecur 2024-03-09T09:27:15.895 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/interactive-once 2024-03-09T09:27:15.896 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/f-1 2024-03-09T09:27:15.896 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/fail-eperm 2024-03-09T09:27:15.896 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/empty-name 2024-03-09T09:27:15.897 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/hash 2024-03-09T09:27:15.897 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/i-no-r 2024-03-09T09:27:15.897 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/r-3 2024-03-09T09:27:15.898 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/readdir-bug 2024-03-09T09:27:15.898 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/rm2 2024-03-09T09:27:15.898 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/rm/rm4 2024-03-09T09:27:15.899 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/test-lib.sh 2024-03-09T09:27:15.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/sample-test 2024-03-09T09:27:15.900 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chown/ 2024-03-09T09:27:15.940 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chown/basic 2024-03-09T09:27:16.300 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:27:16.300 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:27:16.413 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:16.413 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:16.413 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:27:16.444 INFO:tasks.workunit.client.1.smithi134.stderr:+ for f in `seq 1 20` 2024-03-09T09:27:16.444 INFO:tasks.workunit.client.1.smithi134.stdout:20 2024-03-09T09:27:16.445 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 20 2024-03-09T09:27:16.445 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp linux-2.6.33.tar.bz2 a 2024-03-09T09:27:16.542 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s 2024-03-09T09:27:16.583 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm a 2024-03-09T09:27:16.592 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp .snap/s/a /tmp/a 2024-03-09T09:27:16.692 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chown/separator 2024-03-09T09:27:16.693 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chown/deref 2024-03-09T09:27:16.693 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chown/preserve-root 2024-03-09T09:27:16.693 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/id/ 2024-03-09T09:27:16.707 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/id/no-context 2024-03-09T09:27:16.709 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/CuTmpdir.pm 2024-03-09T09:27:16.710 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/Makefile.am 2024-03-09T09:27:16.710 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/ 2024-03-09T09:27:16.713 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/dir-slash 2024-03-09T09:27:16.715 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/preserve-2 2024-03-09T09:27:16.716 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/dir-vs-file 2024-03-09T09:27:16.716 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/preserve-slink-time 2024-03-09T09:27:16.717 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/same-file 2024-03-09T09:27:16.719 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/src-base-dot 2024-03-09T09:27:16.719 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/cp-a-selinux 2024-03-09T09:27:16.719 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/r-vs-symlink 2024-03-09T09:27:16.719 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/capability 2024-03-09T09:27:16.719 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/existing-perm-race 2024-03-09T09:27:16.719 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/parent-perm-race 2024-03-09T09:27:16.719 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/reflink-auto 2024-03-09T09:27:16.719 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/cp-mv-enotsup-xattr 2024-03-09T09:27:16.720 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/no-deref-link2 2024-03-09T09:27:16.720 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/cp-mv-backup 2024-03-09T09:27:16.720 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/link-no-deref 2024-03-09T09:27:16.721 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/file-perm-race 2024-03-09T09:27:16.721 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/abuse 2024-03-09T09:27:16.721 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/link-heap 2024-03-09T09:27:16.722 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/backup-dir 2024-03-09T09:27:16.722 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/preserve-gid 2024-03-09T09:27:16.722 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/link 2024-03-09T09:27:16.722 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/cp-parents 2024-03-09T09:27:16.723 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/no-deref-link1 2024-03-09T09:27:16.723 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/into-self 2024-03-09T09:27:16.724 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/parent-perm 2024-03-09T09:27:16.724 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/cp-deref 2024-03-09T09:27:16.725 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/proc-short-read 2024-03-09T09:27:16.725 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/backup-is-src 2024-03-09T09:27:16.726 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/cp-i 2024-03-09T09:27:16.726 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/special-f 2024-03-09T09:27:16.726 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/proc-zero-len 2024-03-09T09:27:16.727 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/slink-2-slink 2024-03-09T09:27:16.727 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/special-bits 2024-03-09T09:27:16.728 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/fail-perm 2024-03-09T09:27:16.728 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/link-preserve 2024-03-09T09:27:16.729 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/dir-rm-dest 2024-03-09T09:27:16.730 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/backup-1 2024-03-09T09:27:16.730 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/cp-HL 2024-03-09T09:27:16.730 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/thru-dangling 2024-03-09T09:27:16.730 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/sparse 2024-03-09T09:27:16.731 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/deref-slink 2024-03-09T09:27:16.731 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/symlink-slash 2024-03-09T09:27:16.731 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/acl 2024-03-09T09:27:16.731 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/perm 2024-03-09T09:27:16.732 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/reflink-perm 2024-03-09T09:27:16.732 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/cp/no-deref-link3 2024-03-09T09:27:16.733 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/ 2024-03-09T09:27:16.758 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/t-slash 2024-03-09T09:27:16.886 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/p-v 2024-03-09T09:27:16.887 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/p-3 2024-03-09T09:27:16.887 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/writable-under-readonly 2024-03-09T09:27:16.887 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/p-slashdot 2024-03-09T09:27:16.888 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/selinux 2024-03-09T09:27:16.888 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/p-1 2024-03-09T09:27:16.888 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/p-2 2024-03-09T09:27:16.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/parents 2024-03-09T09:27:16.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/special-1 2024-03-09T09:27:16.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/perm 2024-03-09T09:27:16.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mkdir/p-thru-slink 2024-03-09T09:27:16.891 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/require-perl 2024-03-09T09:27:16.897 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ln/ 2024-03-09T09:27:17.268 INFO:tasks.workunit.client.1.smithi134.stderr:++ md5sum /tmp/a 2024-03-09T09:27:17.268 INFO:tasks.workunit.client.1.smithi134.stderr:++ awk '{print $1}' 2024-03-09T09:27:17.384 INFO:tasks.workunit.client.1.smithi134.stderr:+ cur=c3883760b18d50e8d78819c54d579b00 2024-03-09T09:27:17.385 INFO:tasks.workunit.client.1.smithi134.stderr:+ '[' c3883760b18d50e8d78819c54d579b00 '!=' c3883760b18d50e8d78819c54d579b00 ']' 2024-03-09T09:27:17.385 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s 2024-03-09T09:27:17.422 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm linux-2.6.33.tar.bz2 2024-03-09T09:27:17.430 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:27:17.430 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T09:27:17.431 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ln/target-1 2024-03-09T09:27:17.445 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ln/hard-to-sym 2024-03-09T09:27:17.445 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ln/hard-backup 2024-03-09T09:27:17.446 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ln/sf-1 2024-03-09T09:27:17.446 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ln/backup-1 2024-03-09T09:27:17.446 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ln/slash-decorated-nonexistent-dest 2024-03-09T09:27:17.447 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ln/misc 2024-03-09T09:27:17.447 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/lang-default 2024-03-09T09:27:17.448 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/check.mk 2024-03-09T09:27:17.448 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/Makefile.in 2024-03-09T09:27:17.448 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/install/ 2024-03-09T09:27:17.454 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/install/create-leading 2024-03-09T09:27:17.457 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/install/install-C-selinux 2024-03-09T09:27:17.458 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/install/trap 2024-03-09T09:27:17.458 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/install/basic-1 2024-03-09T09:27:17.459 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/install/strip-program 2024-03-09T09:27:17.460 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/install/d-slashdot 2024-03-09T09:27:17.461 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/install/install-C 2024-03-09T09:27:17.462 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/install/install-C-root 2024-03-09T09:27:17.462 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/ 2024-03-09T09:27:17.464 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/c-option 2024-03-09T09:27:17.466 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/octal 2024-03-09T09:27:17.466 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/umask-x 2024-03-09T09:27:17.466 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/inaccessible 2024-03-09T09:27:17.466 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/equal-x 2024-03-09T09:27:17.467 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/silent 2024-03-09T09:27:17.467 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/equals 2024-03-09T09:27:17.467 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/usage 2024-03-09T09:27:17.467 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/thru-dangling 2024-03-09T09:27:17.468 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/setgid 2024-03-09T09:27:17.468 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chmod/no-x 2024-03-09T09:27:17.468 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/df/ 2024-03-09T09:27:17.476 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/df/unreadable 2024-03-09T09:27:17.487 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/df/total-verify 2024-03-09T09:27:17.487 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/other-fs-tmpdir 2024-03-09T09:27:17.487 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/ 2024-03-09T09:27:17.487 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/basic 2024-03-09T09:27:17.487 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/exclude 2024-03-09T09:27:17.487 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/one-file-system 2024-03-09T09:27:17.487 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/long-sloop 2024-03-09T09:27:17.488 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/slink 2024-03-09T09:27:17.488 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/inacc-dir 2024-03-09T09:27:17.488 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/inaccessible-cwd 2024-03-09T09:27:17.488 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/inacc-dest 2024-03-09T09:27:17.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/deref-args 2024-03-09T09:27:17.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/deref 2024-03-09T09:27:17.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/trailing-slash 2024-03-09T09:27:17.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/files0-from 2024-03-09T09:27:17.489 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/two-args 2024-03-09T09:27:17.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/hard-link 2024-03-09T09:27:17.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/long-from-unreadable 2024-03-09T09:27:17.490 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/8gb 2024-03-09T09:27:17.491 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/2g 2024-03-09T09:27:17.491 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/restore-wd 2024-03-09T09:27:17.491 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/slash 2024-03-09T09:27:17.492 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/no-x 2024-03-09T09:27:17.492 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/fd-leak 2024-03-09T09:27:17.492 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/du/no-deref 2024-03-09T09:27:17.492 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/ 2024-03-09T09:27:17.506 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/recursive 2024-03-09T09:27:17.511 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/readdir-mountpoint-inode 2024-03-09T09:27:17.512 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/multihardlink 2024-03-09T09:27:17.512 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/color-dtype-dir 2024-03-09T09:27:17.512 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/file-type 2024-03-09T09:27:17.513 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/rt-1 2024-03-09T09:27:17.513 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/stat-free-symlinks 2024-03-09T09:27:17.513 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/no-cap 2024-03-09T09:27:17.514 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/capability 2024-03-09T09:27:17.514 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/inode 2024-03-09T09:27:17.514 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/color-norm 2024-03-09T09:27:17.515 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/abmon-align 2024-03-09T09:27:17.516 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/dangle 2024-03-09T09:27:17.516 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/dired 2024-03-09T09:27:17.517 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/m-option 2024-03-09T09:27:17.517 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/nameless-uid 2024-03-09T09:27:17.518 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/proc-selinux-segfault 2024-03-09T09:27:17.518 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/stat-vs-dirent 2024-03-09T09:27:17.519 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/color-clear-to-eol 2024-03-09T09:27:17.519 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/no-arg 2024-03-09T09:27:17.520 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/follow-slink 2024-03-09T09:27:17.520 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/symlink-slash 2024-03-09T09:27:17.520 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/stat-dtype 2024-03-09T09:27:17.521 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/x-option 2024-03-09T09:27:17.521 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/infloop 2024-03-09T09:27:17.521 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/ls/stat-failed 2024-03-09T09:27:17.522 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/ 2024-03-09T09:27:17.537 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/hard-3 2024-03-09T09:27:17.545 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/mv-n 2024-03-09T09:27:17.545 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/to-symlink 2024-03-09T09:27:17.546 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/part-fail 2024-03-09T09:27:17.546 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/hard-4 2024-03-09T09:27:17.547 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/leak-fd 2024-03-09T09:27:17.547 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/no-target-dir 2024-03-09T09:27:17.547 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/part-rename 2024-03-09T09:27:17.548 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/dir-file 2024-03-09T09:27:17.548 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/hard-2 2024-03-09T09:27:17.548 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/part-hardlink 2024-03-09T09:27:17.550 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/sticky-to-xpart 2024-03-09T09:27:17.550 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/mv-special-1 2024-03-09T09:27:17.551 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/trailing-slash 2024-03-09T09:27:17.551 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/dir2dir 2024-03-09T09:27:17.551 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/into-self-2 2024-03-09T09:27:17.552 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/backup-dir 2024-03-09T09:27:17.552 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/diag 2024-03-09T09:27:17.552 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/atomic 2024-03-09T09:27:17.553 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/force 2024-03-09T09:27:17.554 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/into-self-3 2024-03-09T09:27:17.555 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/into-self 2024-03-09T09:27:17.555 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/i-2 2024-03-09T09:27:17.556 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/childproof 2024-03-09T09:27:17.556 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/perm-1 2024-03-09T09:27:17.557 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/i-3 2024-03-09T09:27:17.557 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/partition-perm 2024-03-09T09:27:17.557 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/part-symlink 2024-03-09T09:27:17.558 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/backup-is-src 2024-03-09T09:27:17.558 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/i-1 2024-03-09T09:27:17.558 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/update 2024-03-09T09:27:17.559 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/i-4 2024-03-09T09:27:17.559 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/hard-verbose 2024-03-09T09:27:17.559 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/dup-source 2024-03-09T09:27:17.561 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/i-link-no 2024-03-09T09:27:17.561 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/atomic2 2024-03-09T09:27:17.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/into-self-4 2024-03-09T09:27:17.563 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/hard-link-1 2024-03-09T09:27:17.563 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/i-5 2024-03-09T09:27:17.564 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/mv/acl 2024-03-09T09:27:17.564 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/ 2024-03-09T09:27:17.567 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-upchildrealms.sh... 2024-03-09T09:27:17.567 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-upchildrealms.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-upchildrealms.sh 2024-03-09T09:27:17.578 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/fifo 2024-03-09T09:27:17.590 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/empty-file 2024-03-09T09:27:17.590 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/read-only 2024-03-09T09:27:17.590 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/fail-diag 2024-03-09T09:27:17.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/trailing-slash 2024-03-09T09:27:17.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/no-dereference 2024-03-09T09:27:17.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/dir-1 2024-03-09T09:27:17.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/dangling-symlink 2024-03-09T09:27:17.592 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/now-owned-by-other 2024-03-09T09:27:17.592 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/no-create-missing 2024-03-09T09:27:17.592 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/no-rights 2024-03-09T09:27:17.592 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/obsolescent 2024-03-09T09:27:17.592 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/60-seconds 2024-03-09T09:27:17.592 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/not-owner 2024-03-09T09:27:17.593 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/touch/relative 2024-03-09T09:27:17.593 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/ 2024-03-09T09:27:17.603 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sort-merge-fdlimit 2024-03-09T09:27:17.625 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T09:27:17.625 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T09:27:17.631 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/b 2024-03-09T09:27:17.636 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/a1 2024-03-09T09:27:17.686 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/b/.snap/b1 2024-03-09T09:27:17.723 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo asdf 2024-03-09T09:27:17.725 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/a2 2024-03-09T09:27:17.766 INFO:tasks.workunit.client.1.smithi134.stderr:+ ln a/b/foo a/b/bar 2024-03-09T09:27:17.771 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'this should not hang...' 2024-03-09T09:27:17.772 INFO:tasks.workunit.client.1.smithi134.stdout:this should not hang... 2024-03-09T09:27:17.799 INFO:tasks.workunit.client.1.smithi134.stderr:+ cat a/b/.snap/_a2_1099511669268/foo 2024-03-09T09:27:17.805 INFO:tasks.workunit.client.1.smithi134.stdout:asdf 2024-03-09T09:27:17.805 INFO:tasks.workunit.client.1.smithi134.stdout:good, it did not hang. 2024-03-09T09:27:17.805 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'good, it did not hang.' 2024-03-09T09:27:17.805 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/b/.snap/b1 2024-03-09T09:27:17.823 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/a1 2024-03-09T09:27:17.838 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/a2 2024-03-09T09:27:17.854 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm -r a 2024-03-09T09:27:17.903 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T09:27:17.903 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T09:27:17.904 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:27:17.904 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T09:27:17.910 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/basename 2024-03-09T09:27:17.911 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/stat-fmt 2024-03-09T09:27:17.911 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/pr 2024-03-09T09:27:17.912 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/dirname 2024-03-09T09:27:17.912 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/chcon-fail 2024-03-09T09:27:17.912 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/head-elide-tail 2024-03-09T09:27:17.913 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/chroot-credentials 2024-03-09T09:27:17.913 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/nl 2024-03-09T09:27:17.913 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/tee-dash 2024-03-09T09:27:17.913 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/stat-hyphen 2024-03-09T09:27:17.913 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sort-rand 2024-03-09T09:27:17.914 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/expr 2024-03-09T09:27:17.915 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/fmt-long-line 2024-03-09T09:27:17.915 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/split-fail 2024-03-09T09:27:17.915 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/truncate-relative 2024-03-09T09:27:17.916 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/truncate-dir-fail 2024-03-09T09:27:17.916 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/csplit 2024-03-09T09:27:17.917 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/od-N 2024-03-09T09:27:17.918 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/xattr 2024-03-09T09:27:17.919 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/mktemp 2024-03-09T09:27:17.920 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/cut 2024-03-09T09:27:17.920 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/cat-proc 2024-03-09T09:27:17.921 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/comm 2024-03-09T09:27:17.922 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/timeout-parameters 2024-03-09T09:27:17.922 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/tac-continue 2024-03-09T09:27:17.923 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/tty-eof 2024-03-09T09:27:17.923 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/df-P 2024-03-09T09:27:17.923 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/shuf 2024-03-09T09:27:17.923 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/env 2024-03-09T09:27:17.924 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/od 2024-03-09T09:27:17.925 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/split-a 2024-03-09T09:27:17.925 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/tail 2024-03-09T09:27:17.926 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/stat-slash 2024-03-09T09:27:17.926 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/timeout 2024-03-09T09:27:17.926 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/close-stdout 2024-03-09T09:27:17.927 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/wc-files0-from 2024-03-09T09:27:17.927 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sum 2024-03-09T09:27:17.928 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/expand 2024-03-09T09:27:17.928 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sum-sysv 2024-03-09T09:27:17.928 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/wc-parallel 2024-03-09T09:27:17.929 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/md5sum-newline 2024-03-09T09:27:17.929 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sha224sum 2024-03-09T09:27:17.930 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/stty 2024-03-09T09:27:17.930 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/printf-hex 2024-03-09T09:27:17.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/id-context 2024-03-09T09:27:17.931 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/stat-printf 2024-03-09T09:27:17.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/pwd-option 2024-03-09T09:27:17.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sha256sum 2024-03-09T09:27:17.932 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sort-version 2024-03-09T09:27:17.933 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/groups-dash 2024-03-09T09:27:17.933 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/printenv 2024-03-09T09:27:17.933 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/seq 2024-03-09T09:27:17.934 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/fmt 2024-03-09T09:27:17.935 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/cat-buf 2024-03-09T09:27:17.935 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/dircolors 2024-03-09T09:27:17.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/fold 2024-03-09T09:27:17.936 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/selinux 2024-03-09T09:27:17.937 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/md5sum-parallel 2024-03-09T09:27:17.937 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/md5sum 2024-03-09T09:27:17.938 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/nice-fail 2024-03-09T09:27:17.938 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/groups-version 2024-03-09T09:27:17.938 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/date-sec 2024-03-09T09:27:17.939 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/split-l 2024-03-09T09:27:17.939 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/false-status 2024-03-09T09:27:17.939 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sha384sum 2024-03-09T09:27:17.939 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/truncate-dangling-symlink 2024-03-09T09:27:17.940 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/od-multiple-t 2024-03-09T09:27:17.940 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/wc-files0 2024-03-09T09:27:17.940 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/nohup 2024-03-09T09:27:17.941 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/truncate-no-create-missing 2024-03-09T09:27:17.941 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/head-pos 2024-03-09T09:27:17.941 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/uniq 2024-03-09T09:27:17.942 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/test-diag 2024-03-09T09:27:17.942 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/tac 2024-03-09T09:27:17.942 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sort-continue 2024-03-09T09:27:17.943 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/head 2024-03-09T09:27:17.943 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/ptx 2024-03-09T09:27:17.944 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/nproc-avail 2024-03-09T09:27:17.944 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/su-fail 2024-03-09T09:27:17.945 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/od-x8 2024-03-09T09:27:17.945 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/nice 2024-03-09T09:27:17.945 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/chcon 2024-03-09T09:27:17.946 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/head-c 2024-03-09T09:27:17.946 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sort-compress 2024-03-09T09:27:17.946 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/df 2024-03-09T09:27:17.947 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/chroot-fail 2024-03-09T09:27:17.947 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/date-next-dow 2024-03-09T09:27:17.947 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sha1sum 2024-03-09T09:27:17.948 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sha1sum-vec 2024-03-09T09:27:17.950 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/pathchk1 2024-03-09T09:27:17.951 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/arch 2024-03-09T09:27:17.952 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sort-month 2024-03-09T09:27:17.952 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/shred-passes 2024-03-09T09:27:17.952 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sha512sum 2024-03-09T09:27:17.952 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/paste 2024-03-09T09:27:17.952 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/wc 2024-03-09T09:27:17.953 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/ls-time 2024-03-09T09:27:17.954 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/pwd-unreadable-parent 2024-03-09T09:27:17.954 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/test 2024-03-09T09:27:17.954 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/truncate-parameters 2024-03-09T09:27:17.954 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sort-files0-from 2024-03-09T09:27:17.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/base64 2024-03-09T09:27:17.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/ptx-overrun 2024-03-09T09:27:17.955 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/printf-cov 2024-03-09T09:27:17.956 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/tee 2024-03-09T09:27:17.956 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/mknod 2024-03-09T09:27:17.956 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/printf-surprise 2024-03-09T09:27:17.957 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/ls-misc 2024-03-09T09:27:17.957 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/nproc-positive 2024-03-09T09:27:17.957 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/join 2024-03-09T09:27:17.958 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sort-merge 2024-03-09T09:27:17.958 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/factor 2024-03-09T09:27:17.958 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/sort 2024-03-09T09:27:17.958 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/truncate-fifo 2024-03-09T09:27:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/truncate-fail-diag 2024-03-09T09:27:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/stty-invalid 2024-03-09T09:27:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/env-null 2024-03-09T09:27:17.959 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/printf 2024-03-09T09:27:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/seq-long-double 2024-03-09T09:27:17.960 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/pwd-long 2024-03-09T09:27:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/tsort 2024-03-09T09:27:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/runcon-no-reorder 2024-03-09T09:27:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/date 2024-03-09T09:27:17.961 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/shred-exact 2024-03-09T09:27:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/shred-remove 2024-03-09T09:27:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/stdbuf 2024-03-09T09:27:17.962 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/truncate-overflow 2024-03-09T09:27:17.963 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/stty-row-col 2024-03-09T09:27:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/truncate-owned-by-other 2024-03-09T09:27:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/xstrtol 2024-03-09T09:27:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/invalid-opt 2024-03-09T09:27:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/readlink-fp-loop 2024-03-09T09:27:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/tr 2024-03-09T09:27:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/help-version 2024-03-09T09:27:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/id-groups 2024-03-09T09:27:17.964 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/misc/unexpand 2024-03-09T09:27:17.965 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chgrp/ 2024-03-09T09:27:18.010 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-xattrwb.sh... 2024-03-09T09:27:18.010 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/snaptest-xattrwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-xattrwb.sh 2024-03-09T09:27:18.028 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chgrp/basic 2024-03-09T09:27:18.072 INFO:tasks.workunit.client.1.smithi134.stderr:+ set -e 2024-03-09T09:27:18.073 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'testing simple xattr wb' 2024-03-09T09:27:18.073 INFO:tasks.workunit.client.1.smithi134.stdout:testing simple xattr wb 2024-03-09T09:27:18.073 INFO:tasks.workunit.client.1.smithi134.stderr:+ touch x 2024-03-09T09:27:18.078 INFO:tasks.workunit.client.1.smithi134.stderr:+ setfattr -n user.foo x 2024-03-09T09:27:18.098 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/s1 2024-03-09T09:27:18.155 INFO:tasks.workunit.client.1.smithi134.stderr:+ getfattr -n user.foo .snap/s1/x 2024-03-09T09:27:18.155 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep user.foo 2024-03-09T09:27:18.426 INFO:tasks.workunit.client.1.smithi134.stdout:user.foo="" 2024-03-09T09:27:18.427 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm x 2024-03-09T09:27:18.434 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir .snap/s1 2024-03-09T09:27:18.460 INFO:tasks.workunit.client.1.smithi134.stdout:testing wb with pre-wb server cow 2024-03-09T09:27:18.461 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo 'testing wb with pre-wb server cow' 2024-03-09T09:27:18.461 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a 2024-03-09T09:27:18.466 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/b 2024-03-09T09:27:18.472 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/b/c 2024-03-09T09:27:18.478 INFO:tasks.workunit.client.1.smithi134.stderr:+ setfattr -n user.foo a/b 2024-03-09T09:27:18.482 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/.snap/s 2024-03-09T09:27:18.528 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir a/b/cc 2024-03-09T09:27:18.534 INFO:tasks.workunit.client.1.smithi134.stderr:+ getfattr -n user.foo a/b 2024-03-09T09:27:18.537 INFO:tasks.workunit.client.1.smithi134.stdout:# file: a/b 2024-03-09T09:27:18.537 INFO:tasks.workunit.client.1.smithi134.stdout:user.foo="" 2024-03-09T09:27:18.537 INFO:tasks.workunit.client.1.smithi134.stdout: 2024-03-09T09:27:18.538 INFO:tasks.workunit.client.1.smithi134.stderr:+ getfattr -n user.foo a/.snap/s/b 2024-03-09T09:27:18.539 INFO:tasks.workunit.client.1.smithi134.stderr:+ grep user.foo 2024-03-09T09:27:18.550 INFO:tasks.workunit.client.1.smithi134.stdout:user.foo="" 2024-03-09T09:27:18.551 INFO:tasks.workunit.client.1.smithi134.stderr:+ rmdir a/.snap/s 2024-03-09T09:27:18.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chgrp/default-no-deref 2024-03-09T09:27:18.562 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chgrp/deref 2024-03-09T09:27:18.563 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chgrp/posix-H 2024-03-09T09:27:18.563 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chgrp/recurse 2024-03-09T09:27:18.563 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/chgrp/no-x 2024-03-09T09:27:18.563 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/ 2024-03-09T09:27:18.570 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm -r a 2024-03-09T09:27:18.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/big-4gb 2024-03-09T09:27:18.628 INFO:tasks.workunit.client.1.smithi134.stdout:OK 2024-03-09T09:27:18.629 INFO:tasks.workunit.client.1.smithi134.stderr:+ echo OK 2024-03-09T09:27:18.630 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-09T09:27:18.630 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-09T09:27:18.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:27:18.741 INFO:tasks.workunit:Running workunit fs/snaps/untar_snap_rm.sh... 2024-03-09T09:27:18.742 DEBUG:teuthology.orchestra.run.smithi134:workunit test fs/snaps/untar_snap_rm.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/untar_snap_rm.sh 2024-03-09T09:27:18.774 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/follow-stdin 2024-03-09T09:27:18.774 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/F-vs-rename 2024-03-09T09:27:18.774 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/inotify-rotate 2024-03-09T09:27:18.775 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/pid 2024-03-09T09:27:18.775 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/infloop-1 2024-03-09T09:27:18.775 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/assert-2 2024-03-09T09:27:18.775 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/pipe-f 2024-03-09T09:27:18.775 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/pipe-f2 2024-03-09T09:27:18.775 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/start-middle 2024-03-09T09:27:18.775 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/inotify-hash-abuse2 2024-03-09T09:27:18.776 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/inotify-hash-abuse 2024-03-09T09:27:18.777 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/inotify-race 2024-03-09T09:27:18.777 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/append-only 2024-03-09T09:27:18.777 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/tail-n0f 2024-03-09T09:27:18.777 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/flush-initial 2024-03-09T09:27:18.777 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/assert 2024-03-09T09:27:18.777 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/wait 2024-03-09T09:27:18.777 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/tail-2/proc-ksyms 2024-03-09T09:27:18.777 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/envvar-check 2024-03-09T09:27:18.777 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/readlink/ 2024-03-09T09:27:18.786 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/readlink/can-f 2024-03-09T09:27:18.793 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/readlink/can-m 2024-03-09T09:27:18.794 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/readlink/rl-1 2024-03-09T09:27:18.794 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/tests/readlink/can-e 2024-03-09T09:27:18.795 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_trailing_blank 2024-03-09T09:27:18.795 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/ 2024-03-09T09:27:18.798 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/coreutils.texi 2024-03-09T09:27:18.805 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/perm.texi 2024-03-09T09:27:18.806 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/version.texi 2024-03-09T09:27:18.806 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/Makefile.am 2024-03-09T09:27:18.806 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/coreutils.info 2024-03-09T09:27:18.812 INFO:tasks.workunit.client.1.smithi134.stderr:+ do_tarball coreutils_8.5.orig.tar.gz z coreutils-8.5 2024-03-09T09:27:18.812 INFO:tasks.workunit.client.1.smithi134.stderr:+ wget http://download.ceph.com/qa/coreutils_8.5.orig.tar.gz 2024-03-09T09:27:18.813 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/stamp-vti 2024-03-09T09:27:18.813 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/fdl.texi 2024-03-09T09:27:18.813 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/Makefile.in 2024-03-09T09:27:18.814 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/getdate.texi 2024-03-09T09:27:18.815 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/constants.texi 2024-03-09T09:27:18.815 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/doc/ChangeLog-2007 2024-03-09T09:27:18.817 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/ 2024-03-09T09:27:18.819 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/split.c 2024-03-09T09:27:18.821 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/dircolors.hin 2024-03-09T09:27:18.821 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/test.c 2024-03-09T09:27:18.822 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/nproc.c 2024-03-09T09:27:18.822 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/cp-hash.c 2024-03-09T09:27:18.823 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/rm.c 2024-03-09T09:27:18.823 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/uname-arch.c 2024-03-09T09:27:18.823 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/chroot.c 2024-03-09T09:27:18.823 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/remove.h 2024-03-09T09:27:18.823 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/users.c 2024-03-09T09:27:18.823 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/dircolors.c 2024-03-09T09:27:18.823 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/join.c 2024-03-09T09:27:18.825 INFO:tasks.workunit.client.1.smithi134.stderr:--2024-03-09 09:27:18-- http://download.ceph.com/qa/coreutils_8.5.orig.tar.gz 2024-03-09T09:27:18.825 INFO:tasks.workunit.client.1.smithi134.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-09T09:27:18.827 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/chown.c 2024-03-09T09:27:18.827 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/shuf.c 2024-03-09T09:27:18.827 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/sum.c 2024-03-09T09:27:18.827 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/csplit.c 2024-03-09T09:27:18.827 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/libstdbuf.c 2024-03-09T09:27:18.827 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/fold.c 2024-03-09T09:27:18.827 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/basename.c 2024-03-09T09:27:18.827 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/prog-fprintf.h 2024-03-09T09:27:18.827 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/sync.c 2024-03-09T09:27:18.827 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/lbracket.c 2024-03-09T09:27:18.828 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/ln.c 2024-03-09T09:27:18.828 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/tsort.c 2024-03-09T09:27:18.828 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/wc.c 2024-03-09T09:27:18.828 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/extract-magic 2024-03-09T09:27:18.828 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/hostid.c 2024-03-09T09:27:18.828 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/dd.c 2024-03-09T09:27:18.828 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/shred.c 2024-03-09T09:27:18.829 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/uname.h 2024-03-09T09:27:18.829 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/unlink.c 2024-03-09T09:27:18.829 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/Makefile.am 2024-03-09T09:27:18.829 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/seq.c 2024-03-09T09:27:18.830 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/uname-uname.c 2024-03-09T09:27:18.830 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/ptx.c 2024-03-09T09:27:18.831 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/wheel-size.h 2024-03-09T09:27:18.831 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/stty.c 2024-03-09T09:27:18.832 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/chcon.c 2024-03-09T09:27:18.832 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/dcgen 2024-03-09T09:27:18.832 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/uptime.c 2024-03-09T09:27:18.833 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/link.c 2024-03-09T09:27:18.833 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/install.c 2024-03-09T09:27:18.833 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/uname.c 2024-03-09T09:27:18.834 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/mknod.c 2024-03-09T09:27:18.835 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/su.c 2024-03-09T09:27:18.835 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/expand.c 2024-03-09T09:27:18.835 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/mv.c 2024-03-09T09:27:18.836 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/ls.c 2024-03-09T09:27:18.837 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/whoami.c 2024-03-09T09:27:18.837 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/ls-ls.c 2024-03-09T09:27:18.837 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/ls.h 2024-03-09T09:27:18.837 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/id.c 2024-03-09T09:27:18.838 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/Makefile.in 2024-03-09T09:27:18.838 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/du.c 2024-03-09T09:27:18.839 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/cut.c 2024-03-09T09:27:18.839 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/comm.c 2024-03-09T09:27:18.839 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/od.c 2024-03-09T09:27:18.840 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/expr.c 2024-03-09T09:27:18.840 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/nohup.c 2024-03-09T09:27:18.840 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/system.h 2024-03-09T09:27:18.840 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/cat.c 2024-03-09T09:27:18.841 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/mktemp.c 2024-03-09T09:27:18.841 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/tac.c 2024-03-09T09:27:18.841 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/ls-dir.c 2024-03-09T09:27:18.841 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/tac-pipe.c 2024-03-09T09:27:18.841 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/touch.c 2024-03-09T09:27:18.842 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/false.c 2024-03-09T09:27:18.842 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/echo.c 2024-03-09T09:27:18.842 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/printenv.c 2024-03-09T09:27:18.843 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/logname.c 2024-03-09T09:27:18.843 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/group-list.c 2024-03-09T09:27:18.843 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/pr.c 2024-03-09T09:27:18.843 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/chgrp.c 2024-03-09T09:27:18.844 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/cp-hash.h 2024-03-09T09:27:18.845 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/getlimits.c 2024-03-09T09:27:18.845 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/rmdir.c 2024-03-09T09:27:18.845 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/tail.c 2024-03-09T09:27:18.846 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/timeout.c 2024-03-09T09:27:18.847 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/sort.c 2024-03-09T09:27:18.849 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/operand2sig.c 2024-03-09T09:27:18.849 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/factor.c 2024-03-09T09:27:18.850 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/paste.c 2024-03-09T09:27:18.851 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/chmod.c 2024-03-09T09:27:18.851 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/pinky.c 2024-03-09T09:27:18.851 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/unexpand.c 2024-03-09T09:27:18.851 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/cksum.c 2024-03-09T09:27:18.851 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/date.c 2024-03-09T09:27:18.851 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/mkdir.c 2024-03-09T09:27:18.852 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/printf.c 2024-03-09T09:27:18.852 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/true.c 2024-03-09T09:27:18.853 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/c99-to-c89.diff 2024-03-09T09:27:18.853 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/groups.c 2024-03-09T09:27:18.853 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/dirname.c 2024-03-09T09:27:18.853 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/chown-core.h 2024-03-09T09:27:18.853 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/fs.h 2024-03-09T09:27:18.853 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/nl.c 2024-03-09T09:27:18.853 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/tee.c 2024-03-09T09:27:18.853 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/dircolors.h 2024-03-09T09:27:18.854 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/head.c 2024-03-09T09:27:18.854 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/uniq.c 2024-03-09T09:27:18.855 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/chown-core.c 2024-03-09T09:27:18.855 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/runcon.c 2024-03-09T09:27:18.855 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/sleep.c 2024-03-09T09:27:18.856 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/md5sum.c 2024-03-09T09:27:18.856 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/yes.c 2024-03-09T09:27:18.857 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/readlink.c 2024-03-09T09:27:18.857 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/env.c 2024-03-09T09:27:18.858 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/df.c 2024-03-09T09:27:18.858 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/nice.c 2024-03-09T09:27:18.859 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/fmt.c 2024-03-09T09:27:18.859 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/base64.c 2024-03-09T09:27:18.860 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/tty.c 2024-03-09T09:27:18.860 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/truncate.c 2024-03-09T09:27:18.860 INFO:tasks.workunit.client.1.smithi134.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-09T09:27:18.861 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/mkfifo.c 2024-03-09T09:27:18.861 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/operand2sig.h 2024-03-09T09:27:18.861 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/copy.h 2024-03-09T09:27:18.861 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/kill.c 2024-03-09T09:27:18.862 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/who.c 2024-03-09T09:27:18.862 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/stat.c 2024-03-09T09:27:18.862 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/remove.c 2024-03-09T09:27:18.863 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/pwd.c 2024-03-09T09:27:18.863 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/prog-fprintf.c 2024-03-09T09:27:18.863 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/ls-vdir.c 2024-03-09T09:27:18.863 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/copy.c 2024-03-09T09:27:18.863 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/hostname.c 2024-03-09T09:27:18.864 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/wheel-gen.pl 2024-03-09T09:27:18.865 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/group-list.h 2024-03-09T09:27:18.865 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/cp.c 2024-03-09T09:27:18.865 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/setuidgid.c 2024-03-09T09:27:18.866 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/tr.c 2024-03-09T09:27:18.866 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/stdbuf.c 2024-03-09T09:27:18.866 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/pathchk.c 2024-03-09T09:27:18.866 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/src/wheel.h 2024-03-09T09:27:18.867 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/configure 2024-03-09T09:27:18.881 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/README 2024-03-09T09:27:18.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.vg-suppressions 2024-03-09T09:27:18.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_prohibit_atoi_atof 2024-03-09T09:27:18.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_prohibit_stat_st_blocks 2024-03-09T09:27:18.889 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/ChangeLog-2006 2024-03-09T09:27:18.891 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_program_name 2024-03-09T09:27:18.891 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/THANKStt.in 2024-03-09T09:27:18.891 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/ChangeLog-2008 2024-03-09T09:27:18.891 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/ChangeLog-2007 2024-03-09T09:27:18.893 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/ 2024-03-09T09:27:18.896 INFO:tasks.workunit.client.1.smithi134.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-09T09:27:18.896 INFO:tasks.workunit.client.1.smithi134.stderr:Length: 10740866 (10M) [application/octet-stream] 2024-03-09T09:27:18.898 INFO:tasks.workunit.client.1.smithi134.stderr:Saving to: ‘coreutils_8.5.orig.tar.gz’ 2024-03-09T09:27:18.898 INFO:tasks.workunit.client.1.smithi134.stderr: 2024-03-09T09:27:18.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.937+0000 7f4b85200640 1 Processor -- start 2024-03-09T09:27:18.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.937+0000 7f4b85200640 1 -- start start 2024-03-09T09:27:18.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.938+0000 7f4b85200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b800c40a0 0x7f4b8006e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:18.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.939+0000 7f4b85200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b800c3000 con 0x7f4b80150220 2024-03-09T09:27:18.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.939+0000 7f4b85200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b800c32e0 con 0x7f4b800c4a50 2024-03-09T09:27:18.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.939+0000 7f4b85200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b800c3c10 con 0x7f4b800c40a0 2024-03-09T09:27:18.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.939+0000 7f4b7e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b800c40a0 0x7f4b8006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:18.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.939+0000 7f4b7de00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4b80150220 0x7f4b80153460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:49138/0 (socket says 172.21.15.67:49138) 2024-03-09T09:27:18.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.939+0000 7f4b7de00640 1 -- 172.21.15.67:0/3665201443 learned_addr learned my addr 172.21.15.67:0/3665201443 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:27:18.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.939+0000 7f4b7e800640 1 -- 172.21.15.67:0/3665201443 >> v1:172.21.15.134:6789/0 conn(0x7f4b800c4a50 legacy=0x7f4b800718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:18.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.939+0000 7f4b7e800640 1 -- 172.21.15.67:0/3665201443 >> v1:172.21.15.67:6789/0 conn(0x7f4b80150220 legacy=0x7f4b80153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:18.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.939+0000 7f4b7e800640 1 -- 172.21.15.67:0/3665201443 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b800c35c0 con 0x7f4b800c40a0 2024-03-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.941+0000 7f4b7e800640 1 --2- 172.21.15.67:0/3665201443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800c40a0 0x7f4b8006e870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009d90 tx=0x7f4b7802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57db990344c8663b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.941+0000 7f4b7ca00640 1 -- 172.21.15.67:0/3665201443 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78039070 con 0x7f4b800c40a0 2024-03-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.941+0000 7f4b7ca00640 1 -- 172.21.15.67:0/3665201443 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7802cc60 con 0x7f4b800c40a0 2024-03-09T09:27:18.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.941+0000 7f4b7ca00640 1 -- 172.21.15.67:0/3665201443 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78034660 con 0x7f4b800c40a0 2024-03-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.942+0000 7f4b85200640 1 -- 172.21.15.67:0/3665201443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800c40a0 msgr2=0x7f4b8006e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.942+0000 7f4b85200640 1 --2- 172.21.15.67:0/3665201443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800c40a0 0x7f4b8006e870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009d90 tx=0x7f4b7802beb0 comp rx=0 tx=0).stop 2024-03-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.942+0000 7f4b85200640 1 -- 172.21.15.67:0/3665201443 shutdown_connections 2024-03-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.942+0000 7f4b85200640 1 --2- 172.21.15.67:0/3665201443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800c40a0 0x7f4b8006e870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.942+0000 7f4b85200640 1 -- 172.21.15.67:0/3665201443 >> 172.21.15.67:0/3665201443 conn(0x7f4b800c8b20 msgr2=0x7f4b8006c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:18.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.942+0000 7f4b85200640 1 -- 172.21.15.67:0/3665201443 shutdown_connections 2024-03-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.942+0000 7f4b85200640 1 -- 172.21.15.67:0/3665201443 wait complete. 2024-03-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.943+0000 7f4b85200640 1 Processor -- start 2024-03-09T09:27:18.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.944+0000 7f4b85200640 1 -- start start 2024-03-09T09:27:18.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.944+0000 7f4b85200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b800c40a0 0x7f4b80154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.944+0000 7f4b85200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800c4a50 0x7f4b80148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b85200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b80150220 0x7f4b800c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b85200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b800c4550 con 0x7f4b800c4a50 2024-03-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b85200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b80150600 con 0x7f4b800c40a0 2024-03-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b85200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8006edb0 con 0x7f4b80150220 2024-03-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800c4a50 0x7f4b80148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7e800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b80150220 msgr2=0x7f4b800c6d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7e800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b80150220 0x7f4b800c6d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7d400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800c4a50 0x7f4b80148350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39528/0 (socket says 172.21.15.67:39528) 2024-03-09T09:27:18.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7d400640 1 -- 172.21.15.67:0/1962018123 learned_addr learned my addr 172.21.15.67:0/1962018123 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7de00640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b800c40a0 0x7f4b80154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7d400640 1 -- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b80150220 msgr2=0x7f4b800c6d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7d400640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b80150220 0x7f4b800c6d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7d400640 1 -- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b800c40a0 msgr2=0x7f4b80154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7d400640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b800c40a0 0x7f4b80154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7d400640 1 -- 172.21.15.67:0/1962018123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80155110 con 0x7f4b800c4a50 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.945+0000 7f4b7de00640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b800c40a0 0x7f4b80154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.946+0000 7f4b7d400640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800c4a50 0x7f4b80148350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00c440 tx=0x7f4b6c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.946+0000 7f4b76000640 1 -- 172.21.15.67:0/1962018123 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0052d0 con 0x7f4b800c4a50 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.946+0000 7f4b76000640 1 -- 172.21.15.67:0/1962018123 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c005470 con 0x7f4b800c4a50 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.946+0000 7f4b85200640 1 -- 172.21.15.67:0/1962018123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b800c6260 con 0x7f4b800c4a50 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.946+0000 7f4b76000640 1 -- 172.21.15.67:0/1962018123 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c007260 con 0x7f4b800c4a50 2024-03-09T09:27:18.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.946+0000 7f4b85200640 1 -- 172.21.15.67:0/1962018123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b80148890 con 0x7f4b800c4a50 2024-03-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.947+0000 7f4b85200640 1 -- 172.21.15.67:0/1962018123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b800c5dc0 con 0x7f4b800c4a50 2024-03-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.948+0000 7f4b76000640 1 -- 172.21.15.67:0/1962018123 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4b6c007400 con 0x7f4b800c4a50 2024-03-09T09:27:18.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.949+0000 7f4b76000640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6006b4b0 0x7f4b6006d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.949+0000 7f4b76000640 1 -- 172.21.15.67:0/1962018123 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f4b6c089b80 con 0x7f4b800c4a50 2024-03-09T09:27:18.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.950+0000 7f4b7de00640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6006b4b0 0x7f4b6006d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:18.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.951+0000 7f4b7de00640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6006b4b0 0x7f4b6006d970 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4b70005880 tx=0x7f4b70009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:18.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:18.954+0000 7f4b76000640 1 -- 172.21.15.67:0/1962018123 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4b800c5dc0 con 0x7f4b800c4a50 2024-03-09T09:27:18.968 INFO:tasks.workunit.client.1.smithi134.stderr: 0K .......... .......... .......... .......... .......... 0% 717K 15s 2024-03-09T09:27:19.006 INFO:tasks.workunit.client.1.smithi134.stderr: 50K .......... .......... .......... .......... .......... 0% 1.36M 11s 2024-03-09T09:27:19.006 INFO:tasks.workunit.client.1.smithi134.stderr: 100K .......... .......... .......... .......... .......... 1% 77.0M 7s 2024-03-09T09:27:19.006 INFO:tasks.workunit.client.1.smithi134.stderr: 150K .......... .......... .......... .......... .......... 1% 96.6M 5s 2024-03-09T09:27:19.045 INFO:tasks.workunit.client.1.smithi134.stderr: 200K .......... .......... .......... .......... .......... 2% 1.39M 6s 2024-03-09T09:27:19.045 INFO:tasks.workunit.client.1.smithi134.stderr: 250K .......... .......... .......... .......... .......... 2% 40.0M 5s 2024-03-09T09:27:19.045 INFO:tasks.workunit.client.1.smithi134.stderr: 300K .......... .......... .......... .......... .......... 3% 88.2M 4s 2024-03-09T09:27:19.045 INFO:tasks.workunit.client.1.smithi134.stderr: 350K .......... .......... .......... .......... .......... 3% 86.1M 4s 2024-03-09T09:27:19.084 INFO:tasks.workunit.client.1.smithi134.stderr: 400K .......... .......... .......... .......... .......... 4% 1.46M 4s 2024-03-09T09:27:19.084 INFO:tasks.workunit.client.1.smithi134.stderr: 450K .......... .......... .......... .......... .......... 4% 27.0M 4s 2024-03-09T09:27:19.084 INFO:tasks.workunit.client.1.smithi134.stderr: 500K .......... .......... .......... .......... .......... 5% 47.6M 3s 2024-03-09T09:27:19.085 INFO:tasks.workunit.client.1.smithi134.stderr: 550K .......... .......... .......... .......... .......... 5% 65.9M 3s 2024-03-09T09:27:19.085 INFO:tasks.workunit.client.1.smithi134.stderr: 600K .......... .......... .......... .......... .......... 6% 81.4M 3s 2024-03-09T09:27:19.085 INFO:tasks.workunit.client.1.smithi134.stderr: 650K .......... .......... .......... .......... .......... 6% 77.1M 3s 2024-03-09T09:27:19.085 INFO:tasks.workunit.client.1.smithi134.stderr: 700K .......... .......... .......... .......... .......... 7% 88.6M 2s 2024-03-09T09:27:19.085 INFO:tasks.workunit.client.1.smithi134.stderr: 750K .......... .......... .......... .......... .......... 7% 88.6M 2s 2024-03-09T09:27:19.085 INFO:tasks.workunit.client.1.smithi134.stderr: 800K .......... .......... .......... .......... .......... 8% 86.0M 2s 2024-03-09T09:27:19.121 INFO:tasks.workunit.client.1.smithi134.stderr: 850K .......... .......... .......... .......... .......... 8% 1.67M 2s 2024-03-09T09:27:19.121 INFO:tasks.workunit.client.1.smithi134.stderr: 900K .......... .......... .......... .......... .......... 9% 28.9M 2s 2024-03-09T09:27:19.121 INFO:tasks.workunit.client.1.smithi134.stderr: 950K .......... .......... .......... .......... .......... 9% 43.6M 2s 2024-03-09T09:27:19.121 INFO:tasks.workunit.client.1.smithi134.stderr: 1000K .......... .......... .......... .......... .......... 10% 82.6M 2s 2024-03-09T09:27:19.122 INFO:tasks.workunit.client.1.smithi134.stderr: 1050K .......... .......... .......... .......... .......... 10% 78.2M 2s 2024-03-09T09:27:19.122 INFO:tasks.workunit.client.1.smithi134.stderr: 1100K .......... .......... .......... .......... .......... 10% 70.8M 2s 2024-03-09T09:27:19.122 INFO:tasks.workunit.client.1.smithi134.stderr: 1150K .......... .......... .......... .......... .......... 11% 84.9M 2s 2024-03-09T09:27:19.122 INFO:tasks.workunit.client.1.smithi134.stderr: 1200K .......... .......... .......... .......... .......... 11% 85.6M 2s 2024-03-09T09:27:19.122 INFO:tasks.workunit.client.1.smithi134.stderr: 1250K .......... .......... .......... .......... .......... 12% 87.8M 2s 2024-03-09T09:27:19.152 INFO:tasks.workunit.client.1.smithi134.stderr: 1300K .......... .......... .......... .......... .......... 12% 1.57M 2s 2024-03-09T09:27:19.153 INFO:tasks.workunit.client.1.smithi134.stderr: 1350K .......... .......... .......... .......... .......... 13% 27.6M 2s 2024-03-09T09:27:19.153 INFO:tasks.workunit.client.1.smithi134.stderr: 1400K .......... .......... .......... .......... .......... 13% 63.3M 2s 2024-03-09T09:27:19.153 INFO:tasks.workunit.client.1.smithi134.stderr: 1450K .......... .......... .......... .......... .......... 14% 55.5M 2s 2024-03-09T09:27:19.156 INFO:tasks.workunit.client.1.smithi134.stderr: 1500K .......... .......... .......... .......... .......... 14% 26.4M 1s 2024-03-09T09:27:19.156 INFO:tasks.workunit.client.1.smithi134.stderr: 1550K .......... .......... .......... .......... .......... 15% 60.6M 1s 2024-03-09T09:27:19.156 INFO:tasks.workunit.client.1.smithi134.stderr: 1600K .......... .......... .......... .......... .......... 15% 56.7M 1s 2024-03-09T09:27:19.157 INFO:tasks.workunit.client.1.smithi134.stderr: 1650K .......... .......... .......... .......... .......... 16% 69.8M 1s 2024-03-09T09:27:19.159 INFO:tasks.workunit.client.1.smithi134.stderr: 1700K .......... .......... .......... .......... .......... 16% 38.5M 1s 2024-03-09T09:27:19.159 INFO:tasks.workunit.client.1.smithi134.stderr: 1750K .......... .......... .......... .......... .......... 17% 42.7M 1s 2024-03-09T09:27:19.160 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-isnand.h 2024-03-09T09:27:19.179 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-ftello.sh 2024-03-09T09:27:19.179 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-snprintf.c 2024-03-09T09:27:19.180 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-openat.c 2024-03-09T09:27:19.180 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-dirent-safer.c 2024-03-09T09:27:19.181 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-cloexec.c 2024-03-09T09:27:19.181 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fstatat.c 2024-03-09T09:27:19.181 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fnmatch.c 2024-03-09T09:27:19.181 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-getndelim2.c 2024-03-09T09:27:19.181 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-wcrtomb.sh 2024-03-09T09:27:19.182 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-filenamecat.c 2024-03-09T09:27:19.182 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fchownat.c 2024-03-09T09:27:19.182 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-stdlib.c 2024-03-09T09:27:19.182 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-read-file.c 2024-03-09T09:27:19.182 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-exclude7.sh 2024-03-09T09:27:19.184 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-closein.c 2024-03-09T09:27:19.184 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/connect.c 2024-03-09T09:27:19.184 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-rename.h 2024-03-09T09:27:19.184 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-memchr2.c 2024-03-09T09:27:19.184 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-sys_time.c 2024-03-09T09:27:19.184 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/read-file.h 2024-03-09T09:27:19.184 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mkdirat.c 2024-03-09T09:27:19.195 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-isnand-nolibm.c 2024-03-09T09:27:19.195 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fopen-safer.c 2024-03-09T09:27:19.195 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-stdio.c 2024-03-09T09:27:19.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fseeko.c 2024-03-09T09:27:19.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-arpa_inet.c 2024-03-09T09:27:19.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-freadahead.sh 2024-03-09T09:27:19.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-openat-safer.c 2024-03-09T09:27:19.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-unlink.c 2024-03-09T09:27:19.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsstr2.sh 2024-03-09T09:27:19.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-lchown.c 2024-03-09T09:27:19.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-nl_langinfo.c 2024-03-09T09:27:19.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-open.h 2024-03-09T09:27:19.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-memcmp.c 2024-03-09T09:27:19.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-vfprintf-posix.c 2024-03-09T09:27:19.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-freadahead.c 2024-03-09T09:27:19.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/ioctl.c 2024-03-09T09:27:19.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-iconv-h.c 2024-03-09T09:27:19.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mkdir.c 2024-03-09T09:27:19.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-posixtm.c 2024-03-09T09:27:19.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/wctob.c 2024-03-09T09:27:19.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/glthread/ 2024-03-09T09:27:19.204 INFO:tasks.workunit.client.1.smithi134.stderr: 1800K .......... .......... .......... .......... .......... 17% 2.11M 1s 2024-03-09T09:27:19.204 INFO:tasks.workunit.client.1.smithi134.stderr: 1850K .......... .......... .......... .......... .......... 18% 147M 1s 2024-03-09T09:27:19.204 INFO:tasks.workunit.client.1.smithi134.stderr: 1900K .......... .......... .......... .......... .......... 18% 94.0M 1s 2024-03-09T09:27:19.204 INFO:tasks.workunit.client.1.smithi134.stderr: 1950K .......... .......... .......... .......... .......... 19% 105M 1s 2024-03-09T09:27:19.204 INFO:tasks.workunit.client.1.smithi134.stderr: 2000K .......... .......... .......... .......... .......... 19% 70.9M 1s 2024-03-09T09:27:19.204 INFO:tasks.workunit.client.1.smithi134.stderr: 2050K .......... .......... .......... .......... .......... 20% 92.7M 1s 2024-03-09T09:27:19.204 INFO:tasks.workunit.client.1.smithi134.stderr: 2100K .......... .......... .......... .......... .......... 20% 73.9M 1s 2024-03-09T09:27:19.205 INFO:tasks.workunit.client.1.smithi134.stderr: 2150K .......... .......... .......... .......... .......... 20% 101M 1s 2024-03-09T09:27:19.205 INFO:tasks.workunit.client.1.smithi134.stderr: 2200K .......... .......... .......... .......... .......... 21% 63.5M 1s 2024-03-09T09:27:19.205 INFO:tasks.workunit.client.1.smithi134.stderr: 2250K .......... .......... .......... .......... .......... 21% 43.6M 1s 2024-03-09T09:27:19.205 INFO:tasks.workunit.client.1.smithi134.stderr: 2300K .......... .......... .......... .......... .......... 22% 26.4M 1s 2024-03-09T09:27:19.223 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/glthread/thread.c 2024-03-09T09:27:19.232 INFO:tasks.workunit.client.1.smithi134.stderr: 2350K .......... .......... .......... .......... .......... 22% 1.72M 1s 2024-03-09T09:27:19.232 INFO:tasks.workunit.client.1.smithi134.stderr: 2400K .......... .......... .......... .......... .......... 23% 58.7M 1s 2024-03-09T09:27:19.233 INFO:tasks.workunit.client.1.smithi134.stderr: 2450K .......... .......... .......... .......... .......... 23% 63.0M 1s 2024-03-09T09:27:19.233 INFO:tasks.workunit.client.1.smithi134.stderr: 2500K .......... .......... .......... .......... .......... 24% 63.6M 1s 2024-03-09T09:27:19.233 INFO:tasks.workunit.client.1.smithi134.stderr: 2550K .......... .......... .......... .......... .......... 24% 76.3M 1s 2024-03-09T09:27:19.233 INFO:tasks.workunit.client.1.smithi134.stderr: 2600K .......... .......... .......... .......... .......... 25% 65.2M 1s 2024-03-09T09:27:19.233 INFO:tasks.workunit.client.1.smithi134.stderr: 2650K .......... .......... .......... .......... .......... 25% 54.7M 1s 2024-03-09T09:27:19.233 INFO:tasks.workunit.client.1.smithi134.stderr: 2700K .......... .......... .......... .......... .......... 26% 103M 1s 2024-03-09T09:27:19.233 INFO:tasks.workunit.client.1.smithi134.stderr: 2750K .......... .......... .......... .......... .......... 26% 106M 1s 2024-03-09T09:27:19.233 INFO:tasks.workunit.client.1.smithi134.stderr: 2800K .......... .......... .......... .......... .......... 27% 91.3M 1s 2024-03-09T09:27:19.233 INFO:tasks.workunit.client.1.smithi134.stderr: 2850K .......... .......... .......... .......... .......... 27% 32.9M 1s 2024-03-09T09:27:19.244 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/glthread/yield.h 2024-03-09T09:27:19.244 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/glthread/thread.h 2024-03-09T09:27:19.245 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/unistr/ 2024-03-09T09:27:19.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.238+0000 7f4b85200640 1 -- 172.21.15.67:0/1962018123 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4b80075a00 con 0x7f4b800c4a50 2024-03-09T09:27:19.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.239+0000 7f4b76000640 1 -- 172.21.15.67:0/1962018123 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f4b6c058510 con 0x7f4b800c4a50 2024-03-09T09:27:19.246 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":276,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:27:19.248 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/unistr/test-u8-uctomb.c 2024-03-09T09:27:19.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.243+0000 7f4b85200640 1 -- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6006b4b0 msgr2=0x7f4b6006d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:19.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.243+0000 7f4b85200640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6006b4b0 0x7f4b6006d970 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4b70005880 tx=0x7f4b70009040 comp rx=0 tx=0).stop 2024-03-09T09:27:19.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.243+0000 7f4b85200640 1 -- 172.21.15.67:0/1962018123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800c4a50 msgr2=0x7f4b80148350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:19.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.243+0000 7f4b85200640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800c4a50 0x7f4b80148350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00c440 tx=0x7f4b6c001e70 comp rx=0 tx=0).stop 2024-03-09T09:27:19.251 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-exclude5.sh 2024-03-09T09:27:19.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.243+0000 7f4b85200640 1 -- 172.21.15.67:0/1962018123 shutdown_connections 2024-03-09T09:27:19.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.243+0000 7f4b85200640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6006b4b0 0x7f4b6006d970 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.243+0000 7f4b85200640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b80150220 0x7f4b800c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.243+0000 7f4b85200640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b800c4a50 0x7f4b80148350 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.243+0000 7f4b85200640 1 --2- 172.21.15.67:0/1962018123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b800c40a0 0x7f4b80154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.243+0000 7f4b85200640 1 -- 172.21.15.67:0/1962018123 >> 172.21.15.67:0/1962018123 conn(0x7f4b800c8b20 msgr2=0x7f4b800a7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:19.252 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-rmdir.h 2024-03-09T09:27:19.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.243+0000 7f4b85200640 1 -- 172.21.15.67:0/1962018123 shutdown_connections 2024-03-09T09:27:19.253 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-symlink.h 2024-03-09T09:27:19.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:19.244+0000 7f4b85200640 1 -- 172.21.15.67:0/1962018123 wait complete. 2024-03-09T09:27:19.266 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-unsetenv.c 2024-03-09T09:27:19.266 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-exclude4.sh 2024-03-09T09:27:19.266 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/listen.c 2024-03-09T09:27:19.266 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-printf-posix.output 2024-03-09T09:27:19.266 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbrtowc3.sh 2024-03-09T09:27:19.266 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-dirent.c 2024-03-09T09:27:19.266 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-utimens-common.h 2024-03-09T09:27:19.266 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsinit.c 2024-03-09T09:27:19.266 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/w32sock.h 2024-03-09T09:27:19.266 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-printf-posix.h 2024-03-09T09:27:19.267 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fchdir.c 2024-03-09T09:27:19.267 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/read-file.c 2024-03-09T09:27:19.267 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-ctype.c 2024-03-09T09:27:19.267 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/binary-io.h 2024-03-09T09:27:19.267 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/nap.h 2024-03-09T09:27:19.267 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/sys_ioctl.in.h 2024-03-09T09:27:19.267 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-ftello2.sh 2024-03-09T09:27:19.267 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-memchr.c 2024-03-09T09:27:19.267 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsrtowcs3.sh 2024-03-09T09:27:19.267 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fseeko.sh 2024-03-09T09:27:19.267 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-select-out.sh 2024-03-09T09:27:19.268 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/inet_pton.c 2024-03-09T09:27:19.268 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-stdbool.c 2024-03-09T09:27:19.268 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-nanosleep.c 2024-03-09T09:27:19.268 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/bind.c 2024-03-09T09:27:19.268 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/unlinkdir.h 2024-03-09T09:27:19.268 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-getline.c 2024-03-09T09:27:19.268 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-md5.c 2024-03-09T09:27:19.268 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/Makefile.am 2024-03-09T09:27:19.268 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-select-in.sh 2024-03-09T09:27:19.268 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-uname.c 2024-03-09T09:27:19.268 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-readlink.h 2024-03-09T09:27:19.269 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-string.c 2024-03-09T09:27:19.269 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-argmatch.c 2024-03-09T09:27:19.269 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-nl_langinfo.sh 2024-03-09T09:27:19.269 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-sys_stat.c 2024-03-09T09:27:19.269 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-version-etc.c 2024-03-09T09:27:19.269 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-areadlink.h 2024-03-09T09:27:19.269 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-version-etc.sh 2024-03-09T09:27:19.269 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-binary-io.c 2024-03-09T09:27:19.269 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-exclude2.sh 2024-03-09T09:27:19.269 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xfprintf-posix.c 2024-03-09T09:27:19.269 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-set-mode-acl.c 2024-03-09T09:27:19.270 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-wchar.c 2024-03-09T09:27:19.270 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-vasprintf.c 2024-03-09T09:27:19.270 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xstrtoimax.sh 2024-03-09T09:27:19.270 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-freadseek.sh 2024-03-09T09:27:19.270 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-isblank.c 2024-03-09T09:27:19.270 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fpending.sh 2024-03-09T09:27:19.270 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-lstat.h 2024-03-09T09:27:19.270 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-chown.c 2024-03-09T09:27:19.270 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xstrtoumax.sh 2024-03-09T09:27:19.270 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/nan.h 2024-03-09T09:27:19.270 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xprintf-posix.c 2024-03-09T09:27:19.271 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-inet_pton.c 2024-03-09T09:27:19.271 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-dup-safer.c 2024-03-09T09:27:19.271 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-closein.sh 2024-03-09T09:27:19.271 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-canonicalize.c 2024-03-09T09:27:19.271 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-isnanl-nolibm.c 2024-03-09T09:27:19.271 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xstrtol.c 2024-03-09T09:27:19.271 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-open.c 2024-03-09T09:27:19.271 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fprintf-posix.h 2024-03-09T09:27:19.271 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbrtowc1.sh 2024-03-09T09:27:19.271 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-errno.c 2024-03-09T09:27:19.271 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-netdb.c 2024-03-09T09:27:19.272 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-wctype.c 2024-03-09T09:27:19.272 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-strsignal.c 2024-03-09T09:27:19.272 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsstr3.sh 2024-03-09T09:27:19.272 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-yesno.c 2024-03-09T09:27:19.272 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-userspec.c 2024-03-09T09:27:19.272 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-freopen.c 2024-03-09T09:27:19.272 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/uniwidth/ 2024-03-09T09:27:19.283 INFO:tasks.workunit.client.1.smithi134.stderr: 2900K .......... .......... .......... .......... .......... 28% 1.75M 1s 2024-03-09T09:27:19.284 INFO:tasks.workunit.client.1.smithi134.stderr: 2950K .......... .......... .......... .......... .......... 28% 69.0M 1s 2024-03-09T09:27:19.284 INFO:tasks.workunit.client.1.smithi134.stderr: 3000K .......... .......... .......... .......... .......... 29% 74.8M 1s 2024-03-09T09:27:19.284 INFO:tasks.workunit.client.1.smithi134.stderr: 3050K .......... .......... .......... .......... .......... 29% 71.1M 1s 2024-03-09T09:27:19.284 INFO:tasks.workunit.client.1.smithi134.stderr: 3100K .......... .......... .......... .......... .......... 30% 74.2M 1s 2024-03-09T09:27:19.284 INFO:tasks.workunit.client.1.smithi134.stderr: 3150K .......... .......... .......... .......... .......... 30% 87.5M 1s 2024-03-09T09:27:19.284 INFO:tasks.workunit.client.1.smithi134.stderr: 3200K .......... .......... .......... .......... .......... 30% 91.9M 1s 2024-03-09T09:27:19.284 INFO:tasks.workunit.client.1.smithi134.stderr: 3250K .......... .......... .......... .......... .......... 31% 99.0M 1s 2024-03-09T09:27:19.284 INFO:tasks.workunit.client.1.smithi134.stderr: 3300K .......... .......... .......... .......... .......... 31% 80.0M 1s 2024-03-09T09:27:19.284 INFO:tasks.workunit.client.1.smithi134.stderr: 3350K .......... .......... .......... .......... .......... 32% 57.5M 1s 2024-03-09T09:27:19.284 INFO:tasks.workunit.client.1.smithi134.stderr: 3400K .......... .......... .......... .......... .......... 32% 25.5M 1s 2024-03-09T09:27:19.284 INFO:tasks.workunit.client.1.smithi134.stderr: 3450K .......... .......... .......... .......... .......... 33% 31.0M 1s 2024-03-09T09:27:19.302 INFO:tasks.workunit.client.1.smithi134.stderr: 3500K .......... .......... .......... .......... .......... 33% 1.84M 1s 2024-03-09T09:27:19.302 INFO:tasks.workunit.client.1.smithi134.stderr: 3550K .......... .......... .......... .......... .......... 34% 133M 1s 2024-03-09T09:27:19.303 INFO:tasks.workunit.client.1.smithi134.stderr: 3600K .......... .......... .......... .......... .......... 34% 93.5M 1s 2024-03-09T09:27:19.303 INFO:tasks.workunit.client.1.smithi134.stderr: 3650K .......... .......... .......... .......... .......... 35% 94.5M 1s 2024-03-09T09:27:19.303 INFO:tasks.workunit.client.1.smithi134.stderr: 3700K .......... .......... .......... .......... .......... 35% 93.8M 1s 2024-03-09T09:27:19.303 INFO:tasks.workunit.client.1.smithi134.stderr: 3750K .......... .......... .......... .......... .......... 36% 94.6M 1s 2024-03-09T09:27:19.303 INFO:tasks.workunit.client.1.smithi134.stderr: 3800K .......... .......... .......... .......... .......... 36% 78.3M 1s 2024-03-09T09:27:19.303 INFO:tasks.workunit.client.1.smithi134.stderr: 3850K .......... .......... .......... .......... .......... 37% 110M 1s 2024-03-09T09:27:19.303 INFO:tasks.workunit.client.1.smithi134.stderr: 3900K .......... .......... .......... .......... .......... 37% 29.4M 1s 2024-03-09T09:27:19.303 INFO:tasks.workunit.client.1.smithi134.stderr: 3950K .......... .......... .......... .......... .......... 38% 39.5M 1s 2024-03-09T09:27:19.303 INFO:tasks.workunit.client.1.smithi134.stderr: 4000K .......... .......... .......... .......... .......... 38% 31.5M 1s 2024-03-09T09:27:19.303 INFO:tasks.workunit.client.1.smithi134.stderr: 4050K .......... .......... .......... .......... .......... 39% 25.3M 1s 2024-03-09T09:27:19.342 INFO:tasks.workunit.client.1.smithi134.stderr: 4100K .......... .......... .......... .......... .......... 39% 1.91M 1s 2024-03-09T09:27:19.342 INFO:tasks.workunit.client.1.smithi134.stderr: 4150K .......... .......... .......... .......... .......... 40% 60.3M 1s 2024-03-09T09:27:19.342 INFO:tasks.workunit.client.1.smithi134.stderr: 4200K .......... .......... .......... .......... .......... 40% 73.4M 1s 2024-03-09T09:27:19.342 INFO:tasks.workunit.client.1.smithi134.stderr: 4250K .......... .......... .......... .......... .......... 40% 76.4M 1s 2024-03-09T09:27:19.343 INFO:tasks.workunit.client.1.smithi134.stderr: 4300K .......... .......... .......... .......... .......... 41% 72.3M 1s 2024-03-09T09:27:19.343 INFO:tasks.workunit.client.1.smithi134.stderr: 4350K .......... .......... .......... .......... .......... 41% 66.3M 1s 2024-03-09T09:27:19.343 INFO:tasks.workunit.client.1.smithi134.stderr: 4400K .......... .......... .......... .......... .......... 42% 87.0M 1s 2024-03-09T09:27:19.343 INFO:tasks.workunit.client.1.smithi134.stderr: 4450K .......... .......... .......... .......... .......... 42% 77.2M 1s 2024-03-09T09:27:19.343 INFO:tasks.workunit.client.1.smithi134.stderr: 4500K .......... .......... .......... .......... .......... 43% 82.0M 1s 2024-03-09T09:27:19.343 INFO:tasks.workunit.client.1.smithi134.stderr: 4550K .......... .......... .......... .......... .......... 43% 42.4M 1s 2024-03-09T09:27:19.343 INFO:tasks.workunit.client.1.smithi134.stderr: 4600K .......... .......... .......... .......... .......... 44% 35.6M 1s 2024-03-09T09:27:19.343 INFO:tasks.workunit.client.1.smithi134.stderr: 4650K .......... .......... .......... .......... .......... 44% 25.1M 1s 2024-03-09T09:27:19.343 INFO:tasks.workunit.client.1.smithi134.stderr: 4700K .......... .......... .......... .......... .......... 45% 33.9M 1s 2024-03-09T09:27:19.379 INFO:tasks.workunit.client.1.smithi134.stderr: 4750K .......... .......... .......... .......... .......... 45% 1.98M 1s 2024-03-09T09:27:19.380 INFO:tasks.workunit.client.1.smithi134.stderr: 4800K .......... .......... .......... .......... .......... 46% 71.8M 1s 2024-03-09T09:27:19.380 INFO:tasks.workunit.client.1.smithi134.stderr: 4850K .......... .......... .......... .......... .......... 46% 73.5M 1s 2024-03-09T09:27:19.380 INFO:tasks.workunit.client.1.smithi134.stderr: 4900K .......... .......... .......... .......... .......... 47% 63.3M 1s 2024-03-09T09:27:19.380 INFO:tasks.workunit.client.1.smithi134.stderr: 4950K .......... .......... .......... .......... .......... 47% 72.2M 1s 2024-03-09T09:27:19.380 INFO:tasks.workunit.client.1.smithi134.stderr: 5000K .......... .......... .......... .......... .......... 48% 85.1M 1s 2024-03-09T09:27:19.380 INFO:tasks.workunit.client.1.smithi134.stderr: 5050K .......... .......... .......... .......... .......... 48% 81.7M 0s 2024-03-09T09:27:19.380 INFO:tasks.workunit.client.1.smithi134.stderr: 5100K .......... .......... .......... .......... .......... 49% 64.4M 0s 2024-03-09T09:27:19.380 INFO:tasks.workunit.client.1.smithi134.stderr: 5150K .......... .......... .......... .......... .......... 49% 85.8M 0s 2024-03-09T09:27:19.380 INFO:tasks.workunit.client.1.smithi134.stderr: 5200K .......... .......... .......... .......... .......... 50% 50.1M 0s 2024-03-09T09:27:19.381 INFO:tasks.workunit.client.1.smithi134.stderr: 5250K .......... .......... .......... .......... .......... 50% 34.2M 0s 2024-03-09T09:27:19.381 INFO:tasks.workunit.client.1.smithi134.stderr: 5300K .......... .......... .......... .......... .......... 51% 25.0M 0s 2024-03-09T09:27:19.381 INFO:tasks.workunit.client.1.smithi134.stderr: 5350K .......... .......... .......... .......... .......... 51% 32.8M 0s 2024-03-09T09:27:19.415 INFO:tasks.workunit.client.1.smithi134.stderr: 5400K .......... .......... .......... .......... .......... 51% 2.01M 0s 2024-03-09T09:27:19.415 INFO:tasks.workunit.client.1.smithi134.stderr: 5450K .......... .......... .......... .......... .......... 52% 65.8M 0s 2024-03-09T09:27:19.415 INFO:tasks.workunit.client.1.smithi134.stderr: 5500K .......... .......... .......... .......... .......... 52% 72.1M 0s 2024-03-09T09:27:19.416 INFO:tasks.workunit.client.1.smithi134.stderr: 5550K .......... .......... .......... .......... .......... 53% 75.1M 0s 2024-03-09T09:27:19.416 INFO:tasks.workunit.client.1.smithi134.stderr: 5600K .......... .......... .......... .......... .......... 53% 59.6M 0s 2024-03-09T09:27:19.416 INFO:tasks.workunit.client.1.smithi134.stderr: 5650K .......... .......... .......... .......... .......... 54% 76.4M 0s 2024-03-09T09:27:19.416 INFO:tasks.workunit.client.1.smithi134.stderr: 5700K .......... .......... .......... .......... .......... 54% 82.0M 0s 2024-03-09T09:27:19.416 INFO:tasks.workunit.client.1.smithi134.stderr: 5750K .......... .......... .......... .......... .......... 55% 65.6M 0s 2024-03-09T09:27:19.416 INFO:tasks.workunit.client.1.smithi134.stderr: 5800K .......... .......... .......... .......... .......... 55% 61.2M 0s 2024-03-09T09:27:19.416 INFO:tasks.workunit.client.1.smithi134.stderr: 5850K .......... .......... .......... .......... .......... 56% 67.8M 0s 2024-03-09T09:27:19.416 INFO:tasks.workunit.client.1.smithi134.stderr: 5900K .......... .......... .......... .......... .......... 56% 47.4M 0s 2024-03-09T09:27:19.416 INFO:tasks.workunit.client.1.smithi134.stderr: 5950K .......... .......... .......... .......... .......... 57% 28.9M 0s 2024-03-09T09:27:19.416 INFO:tasks.workunit.client.1.smithi134.stderr: 6000K .......... .......... .......... .......... .......... 57% 33.8M 0s 2024-03-09T09:27:19.417 INFO:tasks.workunit.client.1.smithi134.stderr: 6050K .......... .......... .......... .......... .......... 58% 26.8M 0s 2024-03-09T09:27:19.417 INFO:tasks.workunit.client.1.smithi134.stderr: 6100K .......... .......... .......... .......... .......... 58% 35.3M 0s 2024-03-09T09:27:19.451 INFO:tasks.workunit.client.1.smithi134.stderr: 6150K .......... .......... .......... .......... .......... 59% 2.20M 0s 2024-03-09T09:27:19.451 INFO:tasks.workunit.client.1.smithi134.stderr: 6200K .......... .......... .......... .......... .......... 59% 64.2M 0s 2024-03-09T09:27:19.451 INFO:tasks.workunit.client.1.smithi134.stderr: 6250K .......... .......... .......... .......... .......... 60% 61.8M 0s 2024-03-09T09:27:19.451 INFO:tasks.workunit.client.1.smithi134.stderr: 6300K .......... .......... .......... .......... .......... 60% 73.1M 0s 2024-03-09T09:27:19.451 INFO:tasks.workunit.client.1.smithi134.stderr: 6350K .......... .......... .......... .......... .......... 61% 74.5M 0s 2024-03-09T09:27:19.451 INFO:tasks.workunit.client.1.smithi134.stderr: 6400K .......... .......... .......... .......... .......... 61% 71.1M 0s 2024-03-09T09:27:19.452 INFO:tasks.workunit.client.1.smithi134.stderr: 6450K .......... .......... .......... .......... .......... 61% 67.3M 0s 2024-03-09T09:27:19.452 INFO:tasks.workunit.client.1.smithi134.stderr: 6500K .......... .......... .......... .......... .......... 62% 83.2M 0s 2024-03-09T09:27:19.452 INFO:tasks.workunit.client.1.smithi134.stderr: 6550K .......... .......... .......... .......... .......... 62% 80.8M 0s 2024-03-09T09:27:19.452 INFO:tasks.workunit.client.1.smithi134.stderr: 6600K .......... .......... .......... .......... .......... 63% 42.2M 0s 2024-03-09T09:27:19.452 INFO:tasks.workunit.client.1.smithi134.stderr: 6650K .......... .......... .......... .......... .......... 63% 25.5M 0s 2024-03-09T09:27:19.452 INFO:tasks.workunit.client.1.smithi134.stderr: 6700K .......... .......... .......... .......... .......... 64% 26.6M 0s 2024-03-09T09:27:19.452 INFO:tasks.workunit.client.1.smithi134.stderr: 6750K .......... .......... .......... .......... .......... 64% 34.0M 0s 2024-03-09T09:27:19.452 INFO:tasks.workunit.client.1.smithi134.stderr: 6800K .......... .......... .......... .......... .......... 65% 24.8M 0s 2024-03-09T09:27:19.488 INFO:tasks.workunit.client.1.smithi134.stderr: 6850K .......... .......... .......... .......... .......... 65% 2.28M 0s 2024-03-09T09:27:19.489 INFO:tasks.workunit.client.1.smithi134.stderr: 6900K .......... .......... .......... .......... .......... 66% 65.9M 0s 2024-03-09T09:27:19.489 INFO:tasks.workunit.client.1.smithi134.stderr: 6950K .......... .......... .......... .......... .......... 66% 69.4M 0s 2024-03-09T09:27:19.489 INFO:tasks.workunit.client.1.smithi134.stderr: 7000K .......... .......... .......... .......... .......... 67% 61.9M 0s 2024-03-09T09:27:19.489 INFO:tasks.workunit.client.1.smithi134.stderr: 7050K .......... .......... .......... .......... .......... 67% 75.5M 0s 2024-03-09T09:27:19.489 INFO:tasks.workunit.client.1.smithi134.stderr: 7100K .......... .......... .......... .......... .......... 68% 71.9M 0s 2024-03-09T09:27:19.489 INFO:tasks.workunit.client.1.smithi134.stderr: 7150K .......... .......... .......... .......... .......... 68% 74.5M 0s 2024-03-09T09:27:19.489 INFO:tasks.workunit.client.1.smithi134.stderr: 7200K .......... .......... .......... .......... .......... 69% 71.6M 0s 2024-03-09T09:27:19.489 INFO:tasks.workunit.client.1.smithi134.stderr: 7250K .......... .......... .......... .......... .......... 69% 79.0M 0s 2024-03-09T09:27:19.489 INFO:tasks.workunit.client.1.smithi134.stderr: 7300K .......... .......... .......... .......... .......... 70% 68.7M 0s 2024-03-09T09:27:19.490 INFO:tasks.workunit.client.1.smithi134.stderr: 7350K .......... .......... .......... .......... .......... 70% 39.8M 0s 2024-03-09T09:27:19.490 INFO:tasks.workunit.client.1.smithi134.stderr: 7400K .......... .......... .......... .......... .......... 71% 25.6M 0s 2024-03-09T09:27:19.490 INFO:tasks.workunit.client.1.smithi134.stderr: 7450K .......... .......... .......... .......... .......... 71% 24.8M 0s 2024-03-09T09:27:19.490 INFO:tasks.workunit.client.1.smithi134.stderr: 7500K .......... .......... .......... .......... .......... 71% 35.7M 0s 2024-03-09T09:27:19.490 INFO:tasks.workunit.client.1.smithi134.stderr: 7550K .......... .......... .......... .......... .......... 72% 24.4M 0s 2024-03-09T09:27:19.490 INFO:tasks.workunit.client.1.smithi134.stderr: 7600K .......... .......... .......... .......... .......... 72% 41.2M 0s 2024-03-09T09:27:19.527 INFO:tasks.workunit.client.1.smithi134.stderr: 7650K .......... .......... .......... .......... .......... 73% 2.42M 0s 2024-03-09T09:27:19.527 INFO:tasks.workunit.client.1.smithi134.stderr: 7700K .......... .......... .......... .......... .......... 73% 72.7M 0s 2024-03-09T09:27:19.527 INFO:tasks.workunit.client.1.smithi134.stderr: 7750K .......... .......... .......... .......... .......... 74% 63.0M 0s 2024-03-09T09:27:19.527 INFO:tasks.workunit.client.1.smithi134.stderr: 7800K .......... .......... .......... .......... .......... 74% 72.4M 0s 2024-03-09T09:27:19.528 INFO:tasks.workunit.client.1.smithi134.stderr: 7850K .......... .......... .......... .......... .......... 75% 74.4M 0s 2024-03-09T09:27:19.528 INFO:tasks.workunit.client.1.smithi134.stderr: 7900K .......... .......... .......... .......... .......... 75% 78.2M 0s 2024-03-09T09:27:19.528 INFO:tasks.workunit.client.1.smithi134.stderr: 7950K .......... .......... .......... .......... .......... 76% 73.1M 0s 2024-03-09T09:27:19.528 INFO:tasks.workunit.client.1.smithi134.stderr: 8000K .......... .......... .......... .......... .......... 76% 80.4M 0s 2024-03-09T09:27:19.528 INFO:tasks.workunit.client.1.smithi134.stderr: 8050K .......... .......... .......... .......... .......... 77% 78.0M 0s 2024-03-09T09:27:19.528 INFO:tasks.workunit.client.1.smithi134.stderr: 8100K .......... .......... .......... .......... .......... 77% 84.6M 0s 2024-03-09T09:27:19.528 INFO:tasks.workunit.client.1.smithi134.stderr: 8150K .......... .......... .......... .......... .......... 78% 32.2M 0s 2024-03-09T09:27:19.528 INFO:tasks.workunit.client.1.smithi134.stderr: 8200K .......... .......... .......... .......... .......... 78% 25.4M 0s 2024-03-09T09:27:19.528 INFO:tasks.workunit.client.1.smithi134.stderr: 8250K .......... .......... .......... .......... .......... 79% 31.6M 0s 2024-03-09T09:27:19.528 INFO:tasks.workunit.client.1.smithi134.stderr: 8300K .......... .......... .......... .......... .......... 79% 26.1M 0s 2024-03-09T09:27:19.529 INFO:tasks.workunit.client.1.smithi134.stderr: 8350K .......... .......... .......... .......... .......... 80% 25.1M 0s 2024-03-09T09:27:19.529 INFO:tasks.workunit.client.1.smithi134.stderr: 8400K .......... .......... .......... .......... .......... 80% 39.3M 0s 2024-03-09T09:27:19.529 INFO:tasks.workunit.client.1.smithi134.stderr: 8450K .......... .......... .......... .......... .......... 81% 35.5M 0s 2024-03-09T09:27:19.569 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.c 2024-03-09T09:27:19.569 INFO:tasks.workunit.client.1.smithi134.stderr: 8500K .......... .......... .......... .......... .......... 81% 2.59M 0s 2024-03-09T09:27:19.569 INFO:tasks.workunit.client.1.smithi134.stderr: 8550K .......... .......... .......... .......... .......... 81% 73.9M 0s 2024-03-09T09:27:19.569 INFO:tasks.workunit.client.1.smithi134.stderr: 8600K .......... .......... .......... .......... .......... 82% 71.7M 0s 2024-03-09T09:27:19.569 INFO:tasks.workunit.client.1.smithi134.stderr: 8650K .......... .......... .......... .......... .......... 82% 72.3M 0s 2024-03-09T09:27:19.569 INFO:tasks.workunit.client.1.smithi134.stderr: 8700K .......... .......... .......... .......... .......... 83% 54.7M 0s 2024-03-09T09:27:19.570 INFO:tasks.workunit.client.1.smithi134.stderr: 8750K .......... .......... .......... .......... .......... 83% 85.8M 0s 2024-03-09T09:27:19.570 INFO:tasks.workunit.client.1.smithi134.stderr: 8800K .......... .......... .......... .......... .......... 84% 73.9M 0s 2024-03-09T09:27:19.570 INFO:tasks.workunit.client.1.smithi134.stderr: 8850K .......... .......... .......... .......... .......... 84% 75.8M 0s 2024-03-09T09:27:19.570 INFO:tasks.workunit.client.1.smithi134.stderr: 8900K .......... .......... .......... .......... .......... 85% 53.8M 0s 2024-03-09T09:27:19.570 INFO:tasks.workunit.client.1.smithi134.stderr: 8950K .......... .......... .......... .......... .......... 85% 46.9M 0s 2024-03-09T09:27:19.570 INFO:tasks.workunit.client.1.smithi134.stderr: 9000K .......... .......... .......... .......... .......... 86% 39.3M 0s 2024-03-09T09:27:19.570 INFO:tasks.workunit.client.1.smithi134.stderr: 9050K .......... .......... .......... .......... .......... 86% 24.9M 0s 2024-03-09T09:27:19.570 INFO:tasks.workunit.client.1.smithi134.stderr: 9100K .......... .......... .......... .......... .......... 87% 25.6M 0s 2024-03-09T09:27:19.570 INFO:tasks.workunit.client.1.smithi134.stderr: 9150K .......... .......... .......... .......... .......... 87% 38.1M 0s 2024-03-09T09:27:19.570 INFO:tasks.workunit.client.1.smithi134.stderr: 9200K .......... .......... .......... .......... .......... 88% 25.5M 0s 2024-03-09T09:27:19.570 INFO:tasks.workunit.client.1.smithi134.stderr: 9250K .......... .......... .......... .......... .......... 88% 24.9M 0s 2024-03-09T09:27:19.571 INFO:tasks.workunit.client.1.smithi134.stderr: 9300K .......... .......... .......... .......... .......... 89% 37.0M 0s 2024-03-09T09:27:19.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/uniwidth/test-uc_width.c 2024-03-09T09:27:19.572 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.sh 2024-03-09T09:27:19.573 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-sockets.c 2024-03-09T09:27:19.573 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-link.h 2024-03-09T09:27:19.573 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-select-fd.c 2024-03-09T09:27:19.575 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xstrtoimax.c 2024-03-09T09:27:19.575 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xvasprintf.c 2024-03-09T09:27:19.575 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-yesno.sh 2024-03-09T09:27:19.575 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fflush.c 2024-03-09T09:27:19.575 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-vc-list-files-cvs.sh 2024-03-09T09:27:19.575 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbscasecmp.sh 2024-03-09T09:27:19.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-lseek.sh 2024-03-09T09:27:19.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/usleep.c 2024-03-09T09:27:19.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-sleep.c 2024-03-09T09:27:19.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-link.c 2024-03-09T09:27:19.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-perror.c 2024-03-09T09:27:19.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsrtowcs.c 2024-03-09T09:27:19.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xalloc-die.sh 2024-03-09T09:27:19.581 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-i-ring.c 2024-03-09T09:27:19.590 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-priv-set.c 2024-03-09T09:27:19.590 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-btowc.c 2024-03-09T09:27:19.590 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-sys_select.c 2024-03-09T09:27:19.590 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbrtowc.c 2024-03-09T09:27:19.590 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-strerror.c 2024-03-09T09:27:19.590 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-update-copyright.sh 2024-03-09T09:27:19.590 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-btowc2.sh 2024-03-09T09:27:19.590 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-getcwd.c 2024-03-09T09:27:19.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/socket.c 2024-03-09T09:27:19.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-set-mode-acl.sh 2024-03-09T09:27:19.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-linkat.c 2024-03-09T09:27:19.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/perror.c 2024-03-09T09:27:19.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-wcrtomb.c 2024-03-09T09:27:19.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-vasnprintf.c 2024-03-09T09:27:19.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-memrchr.c 2024-03-09T09:27:19.591 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-ftello.c 2024-03-09T09:27:19.592 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-chown.h 2024-03-09T09:27:19.592 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-stat.h 2024-03-09T09:27:19.592 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/Makefile.in 2024-03-09T09:27:19.597 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-getdelim.c 2024-03-09T09:27:19.597 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-vprintf-posix.sh 2024-03-09T09:27:19.598 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fseeko2.sh 2024-03-09T09:27:19.598 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-c-strcase.sh 2024-03-09T09:27:19.598 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsalign.c 2024-03-09T09:27:19.598 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fcntl-h.c 2024-03-09T09:27:19.598 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-vprintf-posix.c 2024-03-09T09:27:19.598 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-rename.c 2024-03-09T09:27:19.598 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-lstat.c 2024-03-09T09:27:19.598 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-malloca.c 2024-03-09T09:27:19.598 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xstrtoul.c 2024-03-09T09:27:19.598 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-perror.sh 2024-03-09T09:27:19.599 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-vasprintf-posix.c 2024-03-09T09:27:19.599 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-frexp.c 2024-03-09T09:27:19.599 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-environ.c 2024-03-09T09:27:19.599 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-stdint.c 2024-03-09T09:27:19.607 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-isnanl.h 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fcntl-safer.c 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-locale.c 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbrtowc4.sh 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-argv-iter.c 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-utimens.h 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xalloc-die.c 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fcntl.c 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-alloca-opt.c 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-dirname.c 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-setenv.c 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-printf-frexpl.c 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-atexit.sh 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-stddef.c 2024-03-09T09:27:19.608 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-c-ctype.c 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xstrtol.sh 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-getopt_long.h 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-quotearg.sh 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-langinfo.c 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-inttypes.c 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-filevercmp.c 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-rmdir.c 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-futimens.h 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/setsockopt.c 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-exclude1.sh 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-freopen-safer.c 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-getaddrinfo.c 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-iconv.c 2024-03-09T09:27:19.609 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/zerosize-ptr.h 2024-03-09T09:27:19.614 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-lseek.c 2024-03-09T09:27:19.614 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-usleep.c 2024-03-09T09:27:19.614 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/gnulib.mk 2024-03-09T09:27:19.621 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-unlinkat.c 2024-03-09T09:27:19.622 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-quotearg.c 2024-03-09T09:27:19.622 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-base64.c 2024-03-09T09:27:19.622 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-frexpl.c 2024-03-09T09:27:19.622 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-gethostname.c 2024-03-09T09:27:19.622 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mkdir.h 2024-03-09T09:27:19.622 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-binary-io.sh 2024-03-09T09:27:19.622 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-sys_ioctl.c 2024-03-09T09:27:19.622 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-time.c 2024-03-09T09:27:19.623 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/signature.h 2024-03-09T09:27:19.623 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-isnanf.h 2024-03-09T09:27:19.623 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-isnanf-nolibm.c 2024-03-09T09:27:19.629 INFO:tasks.workunit.client.1.smithi134.stderr: 9350K .......... .......... .......... .......... .......... 89% 2.88M 0s 2024-03-09T09:27:19.629 INFO:tasks.workunit.client.1.smithi134.stderr: 9400K .......... .......... .......... .......... .......... 90% 64.3M 0s 2024-03-09T09:27:19.630 INFO:tasks.workunit.client.1.smithi134.stderr: 9450K .......... .......... .......... .......... .......... 90% 69.9M 0s 2024-03-09T09:27:19.630 INFO:tasks.workunit.client.1.smithi134.stderr: 9500K .......... .......... .......... .......... .......... 91% 62.5M 0s 2024-03-09T09:27:19.630 INFO:tasks.workunit.client.1.smithi134.stderr: 9550K .......... .......... .......... .......... .......... 91% 66.1M 0s 2024-03-09T09:27:19.630 INFO:tasks.workunit.client.1.smithi134.stderr: 9600K .......... .......... .......... .......... .......... 92% 42.3M 0s 2024-03-09T09:27:19.630 INFO:tasks.workunit.client.1.smithi134.stderr: 9650K .......... .......... .......... .......... .......... 92% 42.4M 0s 2024-03-09T09:27:19.630 INFO:tasks.workunit.client.1.smithi134.stderr: 9700K .......... .......... .......... .......... .......... 92% 35.8M 0s 2024-03-09T09:27:19.630 INFO:tasks.workunit.client.1.smithi134.stderr: 9750K .......... .......... .......... .......... .......... 93% 41.3M 0s 2024-03-09T09:27:19.630 INFO:tasks.workunit.client.1.smithi134.stderr: 9800K .......... .......... .......... .......... .......... 93% 44.2M 0s 2024-03-09T09:27:19.630 INFO:tasks.workunit.client.1.smithi134.stderr: 9850K .......... .......... .......... .......... .......... 94% 43.5M 0s 2024-03-09T09:27:19.630 INFO:tasks.workunit.client.1.smithi134.stderr: 9900K .......... .......... .......... .......... .......... 94% 36.1M 0s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 9950K .......... .......... .......... .......... .......... 95% 44.2M 0s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 10000K .......... .......... .......... .......... .......... 95% 43.3M 0s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 10050K .......... .......... .......... .......... .......... 96% 35.7M 0s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 10100K .......... .......... .......... .......... .......... 96% 28.8M 0s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 10150K .......... .......... .......... .......... .......... 97% 32.2M 0s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 10200K .......... .......... .......... .......... .......... 97% 27.5M 0s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 10250K .......... .......... .......... .......... .......... 98% 3.09M 0s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 10300K .......... .......... .......... .......... .......... 98% 35.3M 0s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 10350K .......... .......... .......... .......... .......... 99% 44.8M 0s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 10400K .......... .......... .......... .......... .......... 99% 47.2M 0s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 10450K .......... .......... .......... ......... 100% 42.2M=0.7s 2024-03-09T09:27:19.631 INFO:tasks.workunit.client.1.smithi134.stderr: 2024-03-09T09:27:19.632 INFO:tasks.workunit.client.1.smithi134.stderr:2024-03-09 09:27:19 (14.3 MB/s) - ‘coreutils_8.5.orig.tar.gz’ saved [10740866/10740866] 2024-03-09T09:27:19.632 INFO:tasks.workunit.client.1.smithi134.stderr: 2024-03-09T09:27:19.632 INFO:tasks.workunit.client.1.smithi134.stderr:+ tar xvfz coreutils_8.5.orig.tar.gz 2024-03-09T09:27:19.642 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/ 2024-03-09T09:27:19.644 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/maint.mk 2024-03-09T09:27:19.647 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/THANKS 2024-03-09T09:27:19.647 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/ABOUT-NLS 2024-03-09T09:27:19.648 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/ 2024-03-09T09:27:19.652 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/textutils/ 2024-03-09T09:27:19.655 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/textutils/ChangeLog 2024-03-09T09:27:19.660 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/textutils/NEWS 2024-03-09T09:27:19.661 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/fileutils/ 2024-03-09T09:27:19.664 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/fileutils/ChangeLog 2024-03-09T09:27:19.669 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/fileutils/ChangeLog-1997 2024-03-09T09:27:19.672 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/fileutils/NEWS 2024-03-09T09:27:19.672 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/sh-utils/ 2024-03-09T09:27:19.675 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/sh-utils/ChangeLog 2024-03-09T09:27:19.679 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/sh-utils/NEWS 2024-03-09T09:27:19.679 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/old/sh-utils/ChangeLog.0 2024-03-09T09:27:19.680 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_prohibit_strcmp 2024-03-09T09:27:19.680 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/ 2024-03-09T09:27:19.684 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/config.rpath 2024-03-09T09:27:19.688 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/cvsu 2024-03-09T09:27:19.689 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/mdate-sh 2024-03-09T09:27:19.689 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/gitlog-to-changelog 2024-03-09T09:27:19.690 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/gnu-web-doc-update 2024-03-09T09:27:19.690 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/useless-if-before-free 2024-03-09T09:27:19.690 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/config.sub 2024-03-09T09:27:19.691 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/missing 2024-03-09T09:27:19.691 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/compile 2024-03-09T09:27:19.691 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/install-sh 2024-03-09T09:27:19.691 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/gendocs.sh 2024-03-09T09:27:19.691 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/update-copyright 2024-03-09T09:27:19.693 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/texinfo.tex 2024-03-09T09:27:19.696 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/unused-parameter.h 2024-03-09T09:27:19.696 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/c++defs.h 2024-03-09T09:27:19.697 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/config.guess 2024-03-09T09:27:19.697 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/do-release-commit-and-tag 2024-03-09T09:27:19.697 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/git-version-gen 2024-03-09T09:27:19.699 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/arg-nonnull.h 2024-03-09T09:27:19.699 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/gnupload 2024-03-09T09:27:19.699 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/warn-on-use.h 2024-03-09T09:27:19.699 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/vc-list-files 2024-03-09T09:27:19.699 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/ylwrap 2024-03-09T09:27:19.699 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/ChangeLog-2007 2024-03-09T09:27:19.700 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/announce-gen 2024-03-09T09:27:19.701 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/build-aux/depcomp 2024-03-09T09:27:19.701 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_error_message_uppercase 2024-03-09T09:27:19.701 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_obsolete_symbols 2024-03-09T09:27:19.701 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/INSTALL 2024-03-09T09:27:19.701 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gl/ 2024-03-09T09:27:19.726 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gl/modules/ 2024-03-09T09:27:19.805 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gl/modules/getloadavg.diff 2024-03-09T09:27:19.880 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/configure.ac 2024-03-09T09:27:19.880 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_prohibit_tab_based_indentation 2024-03-09T09:27:19.881 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/GNUmakefile 2024-03-09T09:27:19.881 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/ChangeLog 2024-03-09T09:27:19.891 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/COPYING 2024-03-09T09:27:19.891 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/ 2024-03-09T09:27:19.893 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-wcwidth.c 2024-03-09T09:27:19.896 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/closeout.m4 2024-03-09T09:27:19.899 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/dirent-safer.m4 2024-03-09T09:27:19.900 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/settime.m4 2024-03-09T09:27:19.900 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/readtokens.m4 2024-03-09T09:27:19.901 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/clock_time.m4 2024-03-09T09:27:19.901 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/config-h.m4 2024-03-09T09:27:19.901 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/euidaccess.m4 2024-03-09T09:27:19.902 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/printf-frexpl.m4 2024-03-09T09:27:19.902 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/posix-shell.m4 2024-03-09T09:27:19.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/servent.m4 2024-03-09T09:27:19.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/xstrtol.m4 2024-03-09T09:27:19.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mkdir-p.m4 2024-03-09T09:27:19.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/00gnulib.m4 2024-03-09T09:27:19.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mbsinit.m4 2024-03-09T09:27:19.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/dup2.m4 2024-03-09T09:27:19.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/eealloc.m4 2024-03-09T09:27:19.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/savewd.m4 2024-03-09T09:27:19.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/memrchr.m4 2024-03-09T09:27:19.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/d-ino.m4 2024-03-09T09:27:19.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/bison.m4 2024-03-09T09:27:19.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/isnanl.m4 2024-03-09T09:27:19.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/lib-ld.m4 2024-03-09T09:27:19.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stpcpy.m4 2024-03-09T09:27:19.909 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/progtest.m4 2024-03-09T09:27:19.909 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/unistd_h.m4 2024-03-09T09:27:19.909 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/time_h.m4 2024-03-09T09:27:19.910 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mkancesdirs.m4 2024-03-09T09:27:19.910 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/signalblocking.m4 2024-03-09T09:27:19.910 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/unlink.m4 2024-03-09T09:27:19.910 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/backupfile.m4 2024-03-09T09:27:19.910 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/remove.m4 2024-03-09T09:27:19.910 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fts.m4 2024-03-09T09:27:19.912 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/xalloc.m4 2024-03-09T09:27:19.912 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/inet_ntop.m4 2024-03-09T09:27:19.912 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/iconv_open.m4 2024-03-09T09:27:19.913 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/extensions.m4 2024-03-09T09:27:19.913 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mbrtowc.m4 2024-03-09T09:27:19.914 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/check-decl.m4 2024-03-09T09:27:19.914 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/cycle-check.m4 2024-03-09T09:27:19.914 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/signal_h.m4 2024-03-09T09:27:19.914 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sys_socket_h.m4 2024-03-09T09:27:19.915 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fprintftime.m4 2024-03-09T09:27:19.916 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/lib-check.m4 2024-03-09T09:27:19.916 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/isblank.m4 2024-03-09T09:27:19.916 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/xstrtod.m4 2024-03-09T09:27:19.916 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stdarg.m4 2024-03-09T09:27:19.917 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/selinux-selinux-h.m4 2024-03-09T09:27:19.917 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fstypename.m4 2024-03-09T09:27:19.917 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/link.m4 2024-03-09T09:27:19.917 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/md5.m4 2024-03-09T09:27:19.918 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/multiarch.m4 2024-03-09T09:27:19.918 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/ioctl.m4 2024-03-09T09:27:19.918 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/flexmember.m4 2024-03-09T09:27:19.919 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/locale-fr.m4 2024-03-09T09:27:19.919 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fdopendir.m4 2024-03-09T09:27:19.920 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/exponentf.m4 2024-03-09T09:27:19.920 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/utimecmp.m4 2024-03-09T09:27:19.921 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/human.m4 2024-03-09T09:27:19.921 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strerror.m4 2024-03-09T09:27:19.921 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/socklen.m4 2024-03-09T09:27:19.922 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/dirent_h.m4 2024-03-09T09:27:19.922 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/locale-zh.m4 2024-03-09T09:27:19.923 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getdate.m4 2024-03-09T09:27:19.924 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/cloexec.m4 2024-03-09T09:27:19.924 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/i-ring.m4 2024-03-09T09:27:19.924 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/group-member.m4 2024-03-09T09:27:19.925 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strtod.m4 2024-03-09T09:27:19.925 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/chdir-long.m4 2024-03-09T09:27:19.926 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strnlen.m4 2024-03-09T09:27:19.926 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strtoimax.m4 2024-03-09T09:27:19.926 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getgroups.m4 2024-03-09T09:27:19.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/yield.m4 2024-03-09T09:27:19.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/write.m4 2024-03-09T09:27:19.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/localcharset.m4 2024-03-09T09:27:19.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/usleep.m4 2024-03-09T09:27:19.928 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/memmove.m4 2024-03-09T09:27:19.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/close.m4 2024-03-09T09:27:19.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/memcmp.m4 2024-03-09T09:27:19.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/autobuild.m4 2024-03-09T09:27:19.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fchdir.m4 2024-03-09T09:27:19.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fd-reopen.m4 2024-03-09T09:27:19.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/jm-winsz2.m4 2024-03-09T09:27:19.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/posixver.m4 2024-03-09T09:27:19.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/locale-ja.m4 2024-03-09T09:27:19.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stat-prog.m4 2024-03-09T09:27:19.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/wint_t.m4 2024-03-09T09:27:19.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/st_dm_mode.m4 2024-03-09T09:27:19.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/tempname.m4 2024-03-09T09:27:19.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/frexp.m4 2024-03-09T09:27:19.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/dirfd.m4 2024-03-09T09:27:19.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sys_select_h.m4 2024-03-09T09:27:19.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/btowc.m4 2024-03-09T09:27:19.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/filenamecat.m4 2024-03-09T09:27:19.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/gmp.m4 2024-03-09T09:27:19.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/isapipe.m4 2024-03-09T09:27:19.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/wctype_h.m4 2024-03-09T09:27:19.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/ldexpl.m4 2024-03-09T09:27:19.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getloadavg.m4 2024-03-09T09:27:19.938 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mbchar.m4 2024-03-09T09:27:19.938 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/version-etc.m4 2024-03-09T09:27:19.938 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fcntl_h.m4 2024-03-09T09:27:19.938 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/gettime.m4 2024-03-09T09:27:19.939 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/wchar_h.m4 2024-03-09T09:27:19.939 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/vasprintf-posix.m4 2024-03-09T09:27:19.940 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/selinux-context-h.m4 2024-03-09T09:27:19.940 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getpass.m4 2024-03-09T09:27:19.940 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mempcpy.m4 2024-03-09T09:27:19.940 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/save-cwd.m4 2024-03-09T09:27:19.941 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fclose.m4 2024-03-09T09:27:19.942 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/modechange.m4 2024-03-09T09:27:19.942 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strstr.m4 2024-03-09T09:27:19.942 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getdtablesize.m4 2024-03-09T09:27:19.942 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/wchar_t.m4 2024-03-09T09:27:19.943 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mkdir.m4 2024-03-09T09:27:19.943 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/idcache.m4 2024-03-09T09:27:19.943 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/tm_gmtoff.m4 2024-03-09T09:27:19.944 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/dirname.m4 2024-03-09T09:27:19.944 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mbiter.m4 2024-03-09T09:27:19.944 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fcntl-safer.m4 2024-03-09T09:27:19.944 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/manywarnings.m4 2024-03-09T09:27:19.944 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/same.m4 2024-03-09T09:27:19.944 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getdelim.m4 2024-03-09T09:27:19.944 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mountlist.m4 2024-03-09T09:27:19.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/lchmod.m4 2024-03-09T09:27:19.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/readlink.m4 2024-03-09T09:27:19.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/gnu-make.m4 2024-03-09T09:27:19.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/thread.m4 2024-03-09T09:27:19.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/hostent.m4 2024-03-09T09:27:19.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/math_h.m4 2024-03-09T09:27:19.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fpending.m4 2024-03-09T09:27:19.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/xgetcwd.m4 2024-03-09T09:27:19.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/utimens.m4 2024-03-09T09:27:19.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/exponentd.m4 2024-03-09T09:27:19.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getaddrinfo.m4 2024-03-09T09:27:19.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sys_ioctl_h.m4 2024-03-09T09:27:19.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stdlib_h.m4 2024-03-09T09:27:19.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mgetgroups.m4 2024-03-09T09:27:19.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/open.m4 2024-03-09T09:27:19.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strpbrk.m4 2024-03-09T09:27:19.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/ssize_t.m4 2024-03-09T09:27:19.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/priv-set.m4 2024-03-09T09:27:19.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fcntl-o.m4 2024-03-09T09:27:19.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/inttostr.m4 2024-03-09T09:27:19.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/malloca.m4 2024-03-09T09:27:19.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/unlocked-io.m4 2024-03-09T09:27:19.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getugroups.m4 2024-03-09T09:27:19.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sha1.m4 2024-03-09T09:27:19.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fpurge.m4 2024-03-09T09:27:19.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/gnulib-comp.m4 2024-03-09T09:27:19.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/wcrtomb.m4 2024-03-09T09:27:19.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/putenv.m4 2024-03-09T09:27:19.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/lstat.m4 2024-03-09T09:27:19.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strsignal.m4 2024-03-09T09:27:19.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mbslen.m4 2024-03-09T09:27:19.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/inttypes.m4 2024-03-09T09:27:19.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stdlib-safer.m4 2024-03-09T09:27:19.953 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strtol.m4 2024-03-09T09:27:19.953 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/lib-ignore.m4 2024-03-09T09:27:19.953 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/readutmp.m4 2024-03-09T09:27:19.953 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strtoll.m4 2024-03-09T09:27:19.954 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stdbool.m4 2024-03-09T09:27:19.954 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/setenv.m4 2024-03-09T09:27:19.954 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/alloca.m4 2024-03-09T09:27:19.954 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/size_max.m4 2024-03-09T09:27:19.954 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/include_next.m4 2024-03-09T09:27:19.954 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sys_stat_h.m4 2024-03-09T09:27:19.956 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strndup.m4 2024-03-09T09:27:19.956 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/link-follow.m4 2024-03-09T09:27:19.957 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/ctype.m4 2024-03-09T09:27:19.957 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/timespec.m4 2024-03-09T09:27:19.957 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getline.m4 2024-03-09T09:27:19.957 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/calloc.m4 2024-03-09T09:27:19.957 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strftime.m4 2024-03-09T09:27:19.957 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/arpa_inet_h.m4 2024-03-09T09:27:19.957 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sig2str.m4 2024-03-09T09:27:19.957 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/printf.m4 2024-03-09T09:27:19.957 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/environ.m4 2024-03-09T09:27:19.958 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/vprintf-posix.m4 2024-03-09T09:27:19.958 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/intlmacosx.m4 2024-03-09T09:27:19.962 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/locale-tr.m4 2024-03-09T09:27:19.962 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mktime.m4 2024-03-09T09:27:19.962 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/iconv.m4 2024-03-09T09:27:19.962 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/pathmax.m4 2024-03-09T09:27:19.962 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/printf-frexp.m4 2024-03-09T09:27:19.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/userspec.m4 2024-03-09T09:27:19.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/host-os.m4 2024-03-09T09:27:19.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/file-type.m4 2024-03-09T09:27:19.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/ls-mntd-fs.m4 2024-03-09T09:27:19.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sha512.m4 2024-03-09T09:27:19.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stdio-safer.m4 2024-03-09T09:27:19.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/lock.m4 2024-03-09T09:27:19.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/c-strtod.m4 2024-03-09T09:27:19.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/closein.m4 2024-03-09T09:27:19.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/frexpl.m4 2024-03-09T09:27:19.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/posixtm.m4 2024-03-09T09:27:19.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/rmdir.m4 2024-03-09T09:27:19.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/chown.m4 2024-03-09T09:27:19.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stdint.m4 2024-03-09T09:27:19.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stddef_h.m4 2024-03-09T09:27:19.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sigaction.m4 2024-03-09T09:27:19.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/error.m4 2024-03-09T09:27:19.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/vasnprintf.m4 2024-03-09T09:27:19.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/utimbuf.m4 2024-03-09T09:27:19.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sockets.m4 2024-03-09T09:27:19.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/canonicalize.m4 2024-03-09T09:27:19.974 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sys_time_h.m4 2024-03-09T09:27:19.974 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/unlink-busy.m4 2024-03-09T09:27:19.974 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/gettimeofday.m4 2024-03-09T09:27:19.974 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/write-any-file.m4 2024-03-09T09:27:19.974 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strdup.m4 2024-03-09T09:27:19.974 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/regex.m4 2024-03-09T09:27:19.974 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sleep.m4 2024-03-09T09:27:19.974 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/float_h.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/warn-on-use.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/atexit.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getcwd-path-max.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fnmatch.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strnumcmp.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/gnulib-common.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/codeset.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/time_r.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/memcoll.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/string_h.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sys_utsname_h.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stdint_h.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/nproc.m4 2024-03-09T09:27:19.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/symlink.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/rename.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/read-file.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strtoumax.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getopt.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fopen.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/ungetc.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/xsize.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fflush.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/acl.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fsusage.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/gethrxtime.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/nls.m4 2024-03-09T09:27:19.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/lib-link.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mpsort.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/netinet_in_h.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fseeko.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/inet_pton.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/yesno.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/physmem.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/intmax_t.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/inttypes-pri.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/malloc.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/jm-winsz1.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/close-stream.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/nocrash.m4 2024-03-09T09:27:19.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/realloc.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/canon-host.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/memcpy.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/tls.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/perl.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/signbit.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/memchr.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/d-type.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/lchown.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/lib-prefix.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getcwd.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/locale_h.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/memxfrm.m4 2024-03-09T09:27:19.978 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mbstate_t.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/filemode.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/memset.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/gethostname.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/vasprintf.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/unistd-safer.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/xfts.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/uptime.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/prereq.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/symlinkat.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getusershell.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/isnand.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/select.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/wctob.m4 2024-03-09T09:27:19.979 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/double-slash-root.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/long-options.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/glibc21.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stat-time.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/rpmatch.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/xattr.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/gettext.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fsync.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/exclude.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/freopen.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strtoul.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/netdb_h.m4 2024-03-09T09:27:19.980 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/snprintf.m4 2024-03-09T09:27:19.987 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stpncpy.m4 2024-03-09T09:27:19.988 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/xnanosleep.m4 2024-03-09T09:27:19.988 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/warnings.m4 2024-03-09T09:27:19.988 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/lseek.m4 2024-03-09T09:27:19.988 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/hard-locale.m4 2024-03-09T09:27:19.988 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/include-exclude-prog.m4 2024-03-09T09:27:19.988 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mmap-anon.m4 2024-03-09T09:27:19.988 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mkstemp.m4 2024-03-09T09:27:19.988 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/openat.m4 2024-03-09T09:27:19.988 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mbrlen.m4 2024-03-09T09:27:19.989 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/jm-macros.m4 2024-03-09T09:27:19.989 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/ftello.m4 2024-03-09T09:27:19.989 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stdio_h.m4 2024-03-09T09:27:19.989 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/exponentl.m4 2024-03-09T09:27:19.989 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getpagesize.m4 2024-03-09T09:27:19.989 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/argmatch.m4 2024-03-09T09:27:19.989 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/perror.m4 2024-03-09T09:27:19.989 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/inline.m4 2024-03-09T09:27:19.989 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/inttypes_h.m4 2024-03-09T09:27:19.990 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/iconv_h.m4 2024-03-09T09:27:19.990 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/base64.m4 2024-03-09T09:27:19.990 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/stat.m4 2024-03-09T09:27:19.990 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/tzset.m4 2024-03-09T09:27:19.990 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/unicodeio.m4 2024-03-09T09:27:19.990 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sha256.m4 2024-03-09T09:27:19.990 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/errno_h.m4 2024-03-09T09:27:19.990 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/unlinkdir.m4 2024-03-09T09:27:19.990 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/boottime.m4 2024-03-09T09:27:19.991 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/freading.m4 2024-03-09T09:27:19.991 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mode_t.m4 2024-03-09T09:27:19.991 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getndelim2.m4 2024-03-09T09:27:19.991 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/xvasprintf.m4 2024-03-09T09:27:19.991 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mbsrtowcs.m4 2024-03-09T09:27:19.991 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/hash.m4 2024-03-09T09:27:19.991 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/safe-write.m4 2024-03-09T09:27:19.991 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/po.m4 2024-03-09T09:27:19.991 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/isnanf.m4 2024-03-09T09:27:19.991 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/longlong.m4 2024-03-09T09:27:19.992 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/xstrndup.m4 2024-03-09T09:27:19.992 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/euidaccess-stat.m4 2024-03-09T09:27:19.992 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/getcwd-abort-bug.m4 2024-03-09T09:27:19.992 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/faccessat.m4 2024-03-09T09:27:19.992 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/nanosleep.m4 2024-03-09T09:27:19.992 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/ChangeLog-2007 2024-03-09T09:27:19.992 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/utimes.m4 2024-03-09T09:27:19.992 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fileblocks.m4 2024-03-09T09:27:19.992 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/sockpfaf.m4 2024-03-09T09:27:19.993 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strtoull.m4 2024-03-09T09:27:19.993 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/assert.m4 2024-03-09T09:27:19.993 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/quotearg.m4 2024-03-09T09:27:19.993 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/uname.m4 2024-03-09T09:27:19.993 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/savedir.m4 2024-03-09T09:27:19.993 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/threadlib.m4 2024-03-09T09:27:19.993 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/langinfo_h.m4 2024-03-09T09:27:19.993 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/linkat.m4 2024-03-09T09:27:19.993 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fpieee.m4 2024-03-09T09:27:19.994 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/memcasecmp.m4 2024-03-09T09:27:19.994 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/vfprintf-posix.m4 2024-03-09T09:27:19.994 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/dos.m4 2024-03-09T09:27:19.996 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/root-dev-ino.m4 2024-03-09T09:27:19.997 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/quote.m4 2024-03-09T09:27:19.997 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/mbswidth.m4 2024-03-09T09:27:19.997 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/nl_langinfo.m4 2024-03-09T09:27:19.997 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/fcntl.m4 2024-03-09T09:27:19.997 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/strcspn.m4 2024-03-09T09:27:19.998 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/safe-read.m4 2024-03-09T09:27:19.998 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/m4/wcwidth.m4 2024-03-09T09:27:19.998 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/Makefile.am 2024-03-09T09:27:19.999 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_po_check 2024-03-09T09:27:19.999 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/dist-check.mk 2024-03-09T09:27:19.999 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_GPL_version 2024-03-09T09:27:20.000 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_unmarked_diagnostics 2024-03-09T09:27:20.000 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.prev-version 2024-03-09T09:27:20.000 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/bootstrap.conf 2024-03-09T09:27:20.000 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/ChangeLog-2005 2024-03-09T09:27:20.003 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/aclocal.m4 2024-03-09T09:27:20.003 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/AUTHORS 2024-03-09T09:27:20.003 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/cfg.mk 2024-03-09T09:27:20.003 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_file_system 2024-03-09T09:27:20.003 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.version 2024-03-09T09:27:20.003 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/TODO 2024-03-09T09:27:20.003 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_space_tab 2024-03-09T09:27:20.003 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_prohibit_empty_lines_at_EOF 2024-03-09T09:27:20.004 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/Makefile.in 2024-03-09T09:27:20.004 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/ 2024-03-09T09:27:20.369 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/cat.x 2024-03-09T09:27:20.555 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/date.x 2024-03-09T09:27:20.556 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/basename.1 2024-03-09T09:27:20.556 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/pathchk.x 2024-03-09T09:27:20.556 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tail.x 2024-03-09T09:27:20.556 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/yes.x 2024-03-09T09:27:20.557 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/pinky.1 2024-03-09T09:27:20.557 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/kill.x 2024-03-09T09:27:20.557 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/pathchk.1 2024-03-09T09:27:20.557 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sync.1 2024-03-09T09:27:20.557 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/dir.1 2024-03-09T09:27:20.558 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/mv.1 2024-03-09T09:27:20.558 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/rm.1 2024-03-09T09:27:20.559 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/dircolors.x 2024-03-09T09:27:20.559 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/rmdir.1 2024-03-09T09:27:20.559 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tty.1 2024-03-09T09:27:20.559 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/fold.x 2024-03-09T09:27:20.559 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/fold.1 2024-03-09T09:27:20.559 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/dir.x 2024-03-09T09:27:20.560 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/fmt.x 2024-03-09T09:27:20.560 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/paste.1 2024-03-09T09:27:20.560 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/chmod.x 2024-03-09T09:27:20.560 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/od.1 2024-03-09T09:27:20.561 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/ptx.x 2024-03-09T09:27:20.561 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/nice.1 2024-03-09T09:27:20.561 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/install.1 2024-03-09T09:27:20.562 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/dd.x 2024-03-09T09:27:20.562 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/cksum.1 2024-03-09T09:27:20.562 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/test.1 2024-03-09T09:27:20.563 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/chgrp.1 2024-03-09T09:27:20.563 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/unlink.1 2024-03-09T09:27:20.563 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/chmod.1 2024-03-09T09:27:20.563 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/dd.1 2024-03-09T09:27:20.564 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/du.1 2024-03-09T09:27:20.564 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tsort.1 2024-03-09T09:27:20.564 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/ls.1 2024-03-09T09:27:20.564 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/split.x 2024-03-09T09:27:20.565 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/nohup.x 2024-03-09T09:27:20.565 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sha1sum.1 2024-03-09T09:27:20.565 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tr.1 2024-03-09T09:27:20.565 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/head.x 2024-03-09T09:27:20.565 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/false.1 2024-03-09T09:27:20.565 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/pwd.x 2024-03-09T09:27:20.565 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/logname.x 2024-03-09T09:27:20.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/paste.x 2024-03-09T09:27:20.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/nl.x 2024-03-09T09:27:20.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sha224sum.1 2024-03-09T09:27:20.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/mkdir.1 2024-03-09T09:27:20.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/nl.1 2024-03-09T09:27:20.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sha256sum.1 2024-03-09T09:27:20.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/printf.x 2024-03-09T09:27:20.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/md5sum.1 2024-03-09T09:27:20.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/Makefile.am 2024-03-09T09:27:20.568 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/dirname.1 2024-03-09T09:27:20.568 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/shuf.x 2024-03-09T09:27:20.568 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/echo.1 2024-03-09T09:27:20.569 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/chcon.x 2024-03-09T09:27:20.570 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/chroot.1 2024-03-09T09:27:20.570 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sha384sum.x 2024-03-09T09:27:20.571 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/stdbuf.1 2024-03-09T09:27:20.571 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/users.1 2024-03-09T09:27:20.571 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/readlink.x 2024-03-09T09:27:20.571 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/cut.x 2024-03-09T09:27:20.572 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/groups.1 2024-03-09T09:27:20.572 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/seq.1 2024-03-09T09:27:20.572 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/help2man 2024-03-09T09:27:20.572 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/dirname.x 2024-03-09T09:27:20.572 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/uname.x 2024-03-09T09:27:20.574 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/uptime.x 2024-03-09T09:27:20.574 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/cut.1 2024-03-09T09:27:20.574 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/cksum.x 2024-03-09T09:27:20.574 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/nohup.1 2024-03-09T09:27:20.575 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/mv.x 2024-03-09T09:27:20.575 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sort.1 2024-03-09T09:27:20.575 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/chown.1 2024-03-09T09:27:20.576 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/pr.x 2024-03-09T09:27:20.576 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/true.x 2024-03-09T09:27:20.576 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/base64.1 2024-03-09T09:27:20.576 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/factor.1 2024-03-09T09:27:20.576 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/ln.x 2024-03-09T09:27:20.577 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/head.1 2024-03-09T09:27:20.577 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/unexpand.1 2024-03-09T09:27:20.577 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/df.x 2024-03-09T09:27:20.577 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/join.x 2024-03-09T09:27:20.577 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/du.x 2024-03-09T09:27:20.578 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/dircolors.1 2024-03-09T09:27:20.578 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/uniq.x 2024-03-09T09:27:20.579 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/split.1 2024-03-09T09:27:20.579 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tty.x 2024-03-09T09:27:20.579 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/uname.1 2024-03-09T09:27:20.579 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/uptime.1 2024-03-09T09:27:20.579 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sort.x 2024-03-09T09:27:20.579 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sha256sum.x 2024-03-09T09:27:20.581 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sha512sum.x 2024-03-09T09:27:20.581 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/nice.x 2024-03-09T09:27:20.581 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/pinky.x 2024-03-09T09:27:20.581 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/fmt.1 2024-03-09T09:27:20.581 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/expr.1 2024-03-09T09:27:20.581 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/wc.1 2024-03-09T09:27:20.581 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/true.1 2024-03-09T09:27:20.581 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/comm.x 2024-03-09T09:27:20.582 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/truncate.x 2024-03-09T09:27:20.582 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/install.x 2024-03-09T09:27:20.583 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/wc.x 2024-03-09T09:27:20.583 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/uniq.1 2024-03-09T09:27:20.583 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/arch.x 2024-03-09T09:27:20.583 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sum.x 2024-03-09T09:27:20.583 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/timeout.x 2024-03-09T09:27:20.583 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/link.x 2024-03-09T09:27:20.584 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tac.1 2024-03-09T09:27:20.585 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sync.x 2024-03-09T09:27:20.585 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/Makefile.in 2024-03-09T09:27:20.586 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/mkfifo.1 2024-03-09T09:27:20.586 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/od.x 2024-03-09T09:27:20.587 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/vdir.1 2024-03-09T09:27:20.587 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/timeout.1 2024-03-09T09:27:20.587 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/mkfifo.x 2024-03-09T09:27:20.588 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/basename.x 2024-03-09T09:27:20.588 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/expr.x 2024-03-09T09:27:20.588 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/mktemp.x 2024-03-09T09:27:20.588 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/vdir.x 2024-03-09T09:27:20.588 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/su.x 2024-03-09T09:27:20.589 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/env.1 2024-03-09T09:27:20.589 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/who.x 2024-03-09T09:27:20.589 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/hostid.1 2024-03-09T09:27:20.589 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tee.x 2024-03-09T09:27:20.590 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/rmdir.x 2024-03-09T09:27:20.590 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/hostid.x 2024-03-09T09:27:20.590 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/id.x 2024-03-09T09:27:20.591 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/unexpand.x 2024-03-09T09:27:20.591 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/pr.1 2024-03-09T09:27:20.592 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/csplit.1 2024-03-09T09:27:20.592 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/truncate.1 2024-03-09T09:27:20.592 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tee.1 2024-03-09T09:27:20.592 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/env.x 2024-03-09T09:27:20.592 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/chroot.x 2024-03-09T09:27:20.592 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/cp.x 2024-03-09T09:27:20.593 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/whoami.x 2024-03-09T09:27:20.594 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/logname.1 2024-03-09T09:27:20.594 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/mknod.1 2024-03-09T09:27:20.594 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/groups.x 2024-03-09T09:27:20.595 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/cp.1 2024-03-09T09:27:20.595 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/runcon.x 2024-03-09T09:27:20.595 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/yes.1 2024-03-09T09:27:20.595 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/stat.1 2024-03-09T09:27:20.596 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/nproc.x 2024-03-09T09:27:20.596 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/unlink.x 2024-03-09T09:27:20.596 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sha384sum.1 2024-03-09T09:27:20.596 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/nproc.1 2024-03-09T09:27:20.596 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/seq.x 2024-03-09T09:27:20.596 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sha512sum.1 2024-03-09T09:27:20.596 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/stat.x 2024-03-09T09:27:20.596 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sleep.x 2024-03-09T09:27:20.597 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/printenv.1 2024-03-09T09:27:20.597 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tr.x 2024-03-09T09:27:20.597 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tail.1 2024-03-09T09:27:20.598 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/md5sum.x 2024-03-09T09:27:20.598 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/link.1 2024-03-09T09:27:20.598 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/echo.x 2024-03-09T09:27:20.599 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/ptx.1 2024-03-09T09:27:20.599 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/factor.x 2024-03-09T09:27:20.599 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/who.1 2024-03-09T09:27:20.599 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/mktemp.1 2024-03-09T09:27:20.599 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/shred.1 2024-03-09T09:27:20.600 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/runcon.1 2024-03-09T09:27:20.600 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/stty.x 2024-03-09T09:27:20.600 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/comm.1 2024-03-09T09:27:20.600 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/shuf.1 2024-03-09T09:27:20.601 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/stty.1 2024-03-09T09:27:20.601 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/printenv.x 2024-03-09T09:27:20.601 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/join.1 2024-03-09T09:27:20.601 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/hostname.x 2024-03-09T09:27:20.602 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/ln.1 2024-03-09T09:27:20.602 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/false.x 2024-03-09T09:27:20.602 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/cat.1 2024-03-09T09:27:20.603 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/users.x 2024-03-09T09:27:20.603 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/printf.1 2024-03-09T09:27:20.603 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/ls.x 2024-03-09T09:27:20.603 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sha224sum.x 2024-03-09T09:27:20.604 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/whoami.1 2024-03-09T09:27:20.604 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/expand.1 2024-03-09T09:27:20.604 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/chgrp.x 2024-03-09T09:27:20.604 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sleep.1 2024-03-09T09:27:20.604 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/csplit.x 2024-03-09T09:27:20.605 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/pwd.1 2024-03-09T09:27:20.605 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/kill.1 2024-03-09T09:27:20.605 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/touch.x 2024-03-09T09:27:20.605 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/df.1 2024-03-09T09:27:20.605 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/stdbuf.x 2024-03-09T09:27:20.606 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/mkdir.x 2024-03-09T09:27:20.606 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/id.1 2024-03-09T09:27:20.606 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/mknod.x 2024-03-09T09:27:20.606 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sum.1 2024-03-09T09:27:20.606 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/rm.x 2024-03-09T09:27:20.606 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/date.1 2024-03-09T09:27:20.607 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/chown.x 2024-03-09T09:27:20.607 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/touch.1 2024-03-09T09:27:20.607 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tac.x 2024-03-09T09:27:20.607 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/base64.x 2024-03-09T09:27:20.607 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/tsort.x 2024-03-09T09:27:20.608 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/shred.x 2024-03-09T09:27:20.609 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/readlink.1 2024-03-09T09:27:20.609 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/sha1sum.x 2024-03-09T09:27:20.609 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/expand.x 2024-03-09T09:27:20.609 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/test.x 2024-03-09T09:27:20.609 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/man/chcon.1 2024-03-09T09:27:20.609 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.tarball-version 2024-03-09T09:27:20.609 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/NEWS 2024-03-09T09:27:20.610 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_sun_os_names 2024-03-09T09:27:20.610 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/bootstrap 2024-03-09T09:27:20.611 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/ 2024-03-09T09:27:20.717 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbsalign.h 2024-03-09T09:27:20.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/streq.h 2024-03-09T09:27:20.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/asprintf.c 2024-03-09T09:27:20.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strnlen1.c 2024-03-09T09:27:20.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dirfd.c 2024-03-09T09:27:20.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/safe-write.h 2024-03-09T09:27:20.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/snprintf.c 2024-03-09T09:27:20.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbiter.h 2024-03-09T09:27:20.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mountlist.c 2024-03-09T09:27:20.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/propername.h 2024-03-09T09:27:20.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memset.c 2024-03-09T09:27:20.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/nproc.c 2024-03-09T09:27:20.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getdelim.c 2024-03-09T09:27:20.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/signbitl.c 2024-03-09T09:27:20.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/group-member.h 2024-03-09T09:27:20.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/long-options.c 2024-03-09T09:27:20.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fts_.h 2024-03-09T09:27:20.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fts.c 2024-03-09T09:27:20.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/verify.h 2024-03-09T09:27:20.950 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/str-two-way.h 2024-03-09T09:27:20.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/areadlink.c 2024-03-09T09:27:20.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/c-strtod.h 2024-03-09T09:27:20.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sha256.h 2024-03-09T09:27:20.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stddef.in.h 2024-03-09T09:27:20.951 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/randint.c 2024-03-09T09:27:20.952 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dup-safer.c 2024-03-09T09:27:20.952 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getpass.h 2024-03-09T09:27:20.952 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/gettext.h 2024-03-09T09:27:20.952 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/randint.h 2024-03-09T09:27:20.952 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stdio-write.c 2024-03-09T09:27:20.953 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xgethostname.h 2024-03-09T09:27:20.953 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/asnprintf.c 2024-03-09T09:27:20.955 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv_open-solaris.gperf 2024-03-09T09:27:20.956 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stdio-impl.h 2024-03-09T09:27:20.956 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mpsort.h 2024-03-09T09:27:20.957 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/closein.c 2024-03-09T09:27:20.958 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/hard-locale.c 2024-03-09T09:27:20.958 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/utimecmp.c 2024-03-09T09:27:20.959 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/vasnprintf.c 2024-03-09T09:27:20.961 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/chown.c 2024-03-09T09:27:20.962 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/printf-args.c 2024-03-09T09:27:20.962 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/netdb.in.h 2024-03-09T09:27:20.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strnlen.c 2024-03-09T09:27:20.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/errno.in.h 2024-03-09T09:27:20.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbslen.c 2024-03-09T09:27:20.965 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/obstack.h 2024-03-09T09:27:20.965 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/file-type.h 2024-03-09T09:27:20.965 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbsrtowcs.c 2024-03-09T09:27:20.965 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fnmatch_loop.c 2024-03-09T09:27:20.967 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/safe-write.c 2024-03-09T09:27:20.967 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mpsort.c 2024-03-09T09:27:20.968 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/netinet_in.in.h 2024-03-09T09:27:20.968 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xfreopen.h 2024-03-09T09:27:20.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/wctype.in.h 2024-03-09T09:27:20.970 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stripslash.c 2024-03-09T09:27:20.970 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/closein.h 2024-03-09T09:27:20.970 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memchr2.c 2024-03-09T09:27:20.971 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memcoll.c 2024-03-09T09:27:20.971 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/malloca.h 2024-03-09T09:27:20.971 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stpcpy.c 2024-03-09T09:27:20.972 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/striconv.c 2024-03-09T09:27:20.972 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/basename.c 2024-03-09T09:27:20.972 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sha256.c 2024-03-09T09:27:20.972 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stdint.in.h 2024-03-09T09:27:20.973 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/acl-internal.h 2024-03-09T09:27:20.973 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/atexit.c 2024-03-09T09:27:20.973 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/size_max.h 2024-03-09T09:27:20.973 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/buffer-lcm.h 2024-03-09T09:27:20.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/idcache.h 2024-03-09T09:27:20.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/root-dev-ino.c 2024-03-09T09:27:20.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/physmem.c 2024-03-09T09:27:20.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/glthread/ 2024-03-09T09:27:21.003 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/glthread/lock.h 2024-03-09T09:27:21.149 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-freadseek.c 2024-03-09T09:27:21.149 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-areadlink-with-size.c 2024-03-09T09:27:21.149 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-select-stdin.c 2024-03-09T09:27:21.149 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsstr1.c 2024-03-09T09:27:21.149 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-exclude3.sh 2024-03-09T09:27:21.150 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fopen.c 2024-03-09T09:27:21.150 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-lock.c 2024-03-09T09:27:21.150 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/locale/ 2024-03-09T09:27:21.163 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/locale/fr/ 2024-03-09T09:27:21.171 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/ 2024-03-09T09:27:21.174 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.mo 2024-03-09T09:27:21.178 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.po 2024-03-09T09:27:21.178 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fpending.c 2024-03-09T09:27:21.179 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-netinet_in.c 2024-03-09T09:27:21.179 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-bitrotate.c 2024-03-09T09:27:21.180 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-c-strcasecmp.c 2024-03-09T09:27:21.180 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsstr3.c 2024-03-09T09:27:21.180 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-hash.c 2024-03-09T09:27:21.180 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-copy-acl.c 2024-03-09T09:27:21.181 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-sigaction.c 2024-03-09T09:27:21.181 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xstrtoumax.c 2024-03-09T09:27:21.181 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fseterr.c 2024-03-09T09:27:21.182 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-areadlink.c 2024-03-09T09:27:21.183 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-signbit.c 2024-03-09T09:27:21.184 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-btowc1.sh 2024-03-09T09:27:21.184 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-strtod.c 2024-03-09T09:27:21.184 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-atexit.c 2024-03-09T09:27:21.184 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-getdate.c 2024-03-09T09:27:21.185 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fpurge.c 2024-03-09T09:27:21.185 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-select.c 2024-03-09T09:27:21.187 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-tls.c 2024-03-09T09:27:21.187 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/sleep.c 2024-03-09T09:27:21.187 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbrtowc2.sh 2024-03-09T09:27:21.188 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-stat.c 2024-03-09T09:27:21.189 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-freadptr.sh 2024-03-09T09:27:21.189 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/accept.c 2024-03-09T09:27:21.191 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-unistd.c 2024-03-09T09:27:21.191 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fopen.h 2024-03-09T09:27:21.191 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-math.c 2024-03-09T09:27:21.191 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-printf-frexp.c 2024-03-09T09:27:21.191 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-vfprintf-posix.sh 2024-03-09T09:27:21.192 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-exclude.c 2024-03-09T09:27:21.193 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-freading.c 2024-03-09T09:27:21.193 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-getdtablesize.c 2024-03-09T09:27:21.193 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/unlinkdir.c 2024-03-09T09:27:21.193 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fflush2.c 2024-03-09T09:27:21.193 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-stat-time.c 2024-03-09T09:27:21.194 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-inet_ntop.c 2024-03-09T09:27:21.194 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-vc-list-files-git.sh 2024-03-09T09:27:21.195 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-getopt.c 2024-03-09T09:27:21.195 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-alignof.c 2024-03-09T09:27:21.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/macros.h 2024-03-09T09:27:21.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-symlinkat.c 2024-03-09T09:27:21.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-getgroups.c 2024-03-09T09:27:21.196 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-xprintf-posix.sh 2024-03-09T09:27:21.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fdopendir.c 2024-03-09T09:27:21.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-sys_socket.c 2024-03-09T09:27:21.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbscasecmp.c 2024-03-09T09:27:21.197 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-getopt.h 2024-03-09T09:27:21.198 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsrtowcs1.sh 2024-03-09T09:27:21.199 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsinit.sh 2024-03-09T09:27:21.199 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-gettimeofday.c 2024-03-09T09:27:21.200 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsstr2.c 2024-03-09T09:27:21.200 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/init.sh 2024-03-09T09:27:21.201 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-symlink.c 2024-03-09T09:27:21.201 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-utimens.c 2024-03-09T09:27:21.201 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-readlink.c 2024-03-09T09:27:21.202 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-lchown.h 2024-03-09T09:27:21.203 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-c-strncasecmp.c 2024-03-09T09:27:21.203 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-lutimens.h 2024-03-09T09:27:21.203 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-copy-acl.sh 2024-03-09T09:27:21.203 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-exclude6.sh 2024-03-09T09:27:21.204 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-freadptr2.c 2024-03-09T09:27:21.204 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fflush2.sh 2024-03-09T09:27:21.204 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-sys_utsname.c 2024-03-09T09:27:21.204 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsrtowcs2.sh 2024-03-09T09:27:21.204 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-file-has-acl.c 2024-03-09T09:27:21.204 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-signal.c 2024-03-09T09:27:21.206 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-sha1.c 2024-03-09T09:27:21.206 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-freadptr2.sh 2024-03-09T09:27:21.206 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-areadlinkat.c 2024-03-09T09:27:21.206 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-unlink.h 2024-03-09T09:27:21.206 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-dup2.c 2024-03-09T09:27:21.207 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-freadptr.c 2024-03-09T09:27:21.207 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-file-has-acl.sh 2024-03-09T09:27:21.208 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-u64.c 2024-03-09T09:27:21.208 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-striconv.c 2024-03-09T09:27:21.209 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-sameacls.c 2024-03-09T09:27:21.209 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-remove.c 2024-03-09T09:27:21.209 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-fsync.c 2024-03-09T09:27:21.209 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/gnulib-tests/test-mbsrtowcs4.sh 2024-03-09T09:27:21.209 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_require_config_h 2024-03-09T09:27:21.210 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_system_h_headers 2024-03-09T09:27:21.210 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_prohibit_fail_0 2024-03-09T09:27:21.210 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_prohibit_magic_number_exit 2024-03-09T09:27:21.210 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/THANKS-to-translators 2024-03-09T09:27:21.210 INFO:tasks.workunit.client.0.smithi067.stdout:coreutils-8.5/.x-sc_useless_cpp_parens 2024-03-09T09:27:21.212 INFO:tasks.workunit.client.0.smithi067.stderr:+ mkdir .snap/k 2024-03-09T09:27:21.478 INFO:tasks.workunit.client.0.smithi067.stderr:+ sync 2024-03-09T09:27:21.795 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/glthread/threadlib.c 2024-03-09T09:27:21.795 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/glthread/lock.c 2024-03-09T09:27:21.796 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/glthread/tls.h 2024-03-09T09:27:21.796 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/glthread/tls.c 2024-03-09T09:27:21.798 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unistr/ 2024-03-09T09:27:21.820 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unistr/u8-uctomb.c 2024-03-09T09:27:21.839 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unistr/u8-uctomb-aux.c 2024-03-09T09:27:21.839 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mkancesdirs.c 2024-03-09T09:27:21.840 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/readtokens0.c 2024-03-09T09:27:21.840 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fnmatch.in.h 2024-03-09T09:27:21.840 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dirname.h 2024-03-09T09:27:21.840 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/regex_internal.c 2024-03-09T09:27:21.840 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strcspn.c 2024-03-09T09:27:21.842 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getugroups.h 2024-03-09T09:27:21.843 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/file-set.c 2024-03-09T09:27:21.843 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sys_socket.in.h 2024-03-09T09:27:21.843 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/inttostr.h 2024-03-09T09:27:21.844 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mkdirat.c 2024-03-09T09:27:21.844 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/readtokens.h 2024-03-09T09:27:21.846 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fd-reopen.h 2024-03-09T09:27:21.846 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/w32sock.h 2024-03-09T09:27:21.846 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memcasecmp.c 2024-03-09T09:27:21.847 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/at-func.c 2024-03-09T09:27:21.847 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unistd-safer.h 2024-03-09T09:27:21.847 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/filevercmp.h 2024-03-09T09:27:21.848 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/c-strcasecmp.c 2024-03-09T09:27:21.848 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fopen-safer.c 2024-03-09T09:27:21.848 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/modechange.h 2024-03-09T09:27:21.848 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sys_stat.in.h 2024-03-09T09:27:21.851 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/freading.c 2024-03-09T09:27:21.851 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/utimens.h 2024-03-09T09:27:21.851 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/md5.h 2024-03-09T09:27:21.851 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/isnanf.c 2024-03-09T09:27:21.852 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sha1.c 2024-03-09T09:27:21.853 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/filenamecat-lgpl.c 2024-03-09T09:27:21.854 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/config.hin 2024-03-09T09:27:21.856 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/ignore-value.h 2024-03-09T09:27:21.856 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strstr.c 2024-03-09T09:27:21.856 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/verror.c 2024-03-09T09:27:21.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/randperm.h 2024-03-09T09:27:21.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/vasprintf.c 2024-03-09T09:27:21.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/c-strncasecmp.c 2024-03-09T09:27:21.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/basename-lgpl.c 2024-03-09T09:27:21.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/human.h 2024-03-09T09:27:21.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/regex.h 2024-03-09T09:27:21.859 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/regcomp.c 2024-03-09T09:27:21.863 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/freadahead.c 2024-03-09T09:27:21.863 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/trim.h 2024-03-09T09:27:21.863 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getgroups.c 2024-03-09T09:27:21.864 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xprintf.c 2024-03-09T09:27:21.864 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/euidaccess-stat.h 2024-03-09T09:27:21.864 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/rand-isaac.h 2024-03-09T09:27:21.864 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mgetgroups.h 2024-03-09T09:27:21.865 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/selinux-at.h 2024-03-09T09:27:21.865 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv_open-solaris.h 2024-03-09T09:27:21.865 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/setenv.c 2024-03-09T09:27:21.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/locale.in.h 2024-03-09T09:27:21.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/base64.h 2024-03-09T09:27:21.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/open-safer.c 2024-03-09T09:27:21.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/exitfail.c 2024-03-09T09:27:21.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/closeout.h 2024-03-09T09:27:21.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/idcache.c 2024-03-09T09:27:21.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unlink.c 2024-03-09T09:27:21.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xfts.c 2024-03-09T09:27:21.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memrchr.c 2024-03-09T09:27:21.870 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/Makefile.am 2024-03-09T09:27:21.870 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstrtol.h 2024-03-09T09:27:21.870 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/gettimeofday.c 2024-03-09T09:27:21.870 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/isnanf-nolibm.h 2024-03-09T09:27:21.870 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/chdir-long.c 2024-03-09T09:27:21.870 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbchar.c 2024-03-09T09:27:21.871 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/canon-host.h 2024-03-09T09:27:21.871 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unsetenv.c 2024-03-09T09:27:21.872 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/freadseek.c 2024-03-09T09:27:21.872 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xnanosleep.c 2024-03-09T09:27:21.873 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/nanosleep.c 2024-03-09T09:27:21.873 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/ctype.in.h 2024-03-09T09:27:21.873 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xmalloc.c 2024-03-09T09:27:21.874 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strtoul.c 2024-03-09T09:27:21.874 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sha512.h 2024-03-09T09:27:21.875 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dirchownmod.h 2024-03-09T09:27:21.876 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/freadptr.h 2024-03-09T09:27:21.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getfilecon.c 2024-03-09T09:27:21.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/alignof.h 2024-03-09T09:27:21.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mempcpy.c 2024-03-09T09:27:21.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/savedir.h 2024-03-09T09:27:21.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/areadlink-with-size.c 2024-03-09T09:27:21.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/inttypes.in.h 2024-03-09T09:27:21.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/config.charset 2024-03-09T09:27:21.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstrtold.c 2024-03-09T09:27:21.879 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fcntl-safer.h 2024-03-09T09:27:21.880 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstrndup.c 2024-03-09T09:27:21.880 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/write.c 2024-03-09T09:27:21.880 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/math.in.h 2024-03-09T09:27:21.881 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/openat-priv.h 2024-03-09T09:27:21.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/vasnprintf.h 2024-03-09T09:27:21.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getcwd.c 2024-03-09T09:27:21.883 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stdbool.in.h 2024-03-09T09:27:21.883 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strtol.c 2024-03-09T09:27:21.883 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/argmatch.c 2024-03-09T09:27:21.886 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/acl_entries.c 2024-03-09T09:27:21.886 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/progname.c 2024-03-09T09:27:21.886 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mktime-internal.h 2024-03-09T09:27:21.887 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/printf-frexpl.c 2024-03-09T09:27:21.887 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/offtostr.c 2024-03-09T09:27:21.887 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memchr2.h 2024-03-09T09:27:21.887 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstrtoumax.c 2024-03-09T09:27:21.887 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getdate.y 2024-03-09T09:27:21.891 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/close-hook.h 2024-03-09T09:27:21.891 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/printf-parse.c 2024-03-09T09:27:21.892 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/isnand.c 2024-03-09T09:27:21.892 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/timespec.h 2024-03-09T09:27:21.892 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getusershell.c 2024-03-09T09:27:21.892 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstrtod.c 2024-03-09T09:27:21.892 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/verror.h 2024-03-09T09:27:21.892 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/same.c 2024-03-09T09:27:21.892 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/quote.c 2024-03-09T09:27:21.893 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/raise.c 2024-03-09T09:27:21.893 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/link.c 2024-03-09T09:27:21.893 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/cloexec.h 2024-03-09T09:27:21.894 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sys_utsname.in.h 2024-03-09T09:27:21.902 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/cloexec.c 2024-03-09T09:27:21.902 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/vfprintf.c 2024-03-09T09:27:21.902 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/uniwidth/ 2024-03-09T09:27:22.281 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:27:22.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.606+0000 7f8f36c00640 1 Processor -- start 2024-03-09T09:27:22.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.607+0000 7f8f36c00640 1 -- start start 2024-03-09T09:27:22.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.608+0000 7f8f36c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f300710a0 0x7f8f30153970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:22.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.608+0000 7f8f36c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8f30151360 0x7f8f30153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:22.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.608+0000 7f8f36c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f300c81e0 con 0x7f8f300c1f70 2024-03-09T09:27:22.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.609+0000 7f8f36c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f300c32b0 con 0x7f8f300710a0 2024-03-09T09:27:22.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.609+0000 7f8f36c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f300c2fd0 con 0x7f8f30151360 2024-03-09T09:27:22.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.609+0000 7f8f34e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f300710a0 0x7f8f30153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:22.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.609+0000 7f8f34e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8f300710a0 0x7f8f30153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39532/0 (socket says 172.21.15.67:39532) 2024-03-09T09:27:22.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.609+0000 7f8f34e00640 1 -- 172.21.15.67:0/2173492450 learned_addr learned my addr 172.21.15.67:0/2173492450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:22.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.609+0000 7f8f36200640 1 --2- 172.21.15.67:0/2173492450 >> v2:172.21.15.134:3300/0 conn(0x7f8f30151360 0x7f8f30153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:22.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.609+0000 7f8f34e00640 1 -- 172.21.15.67:0/2173492450 >> v2:172.21.15.134:3300/0 conn(0x7f8f30151360 msgr2=0x7f8f30153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:22.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.609+0000 7f8f34e00640 1 --2- 172.21.15.67:0/2173492450 >> v2:172.21.15.134:3300/0 conn(0x7f8f30151360 0x7f8f30153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:22.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.609+0000 7f8f34e00640 1 -- 172.21.15.67:0/2173492450 >> v1:172.21.15.134:6789/0 conn(0x7f8f300c1f70 legacy=0x7f8f30150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:22.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.609+0000 7f8f34e00640 1 -- 172.21.15.67:0/2173492450 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f300c3700 con 0x7f8f300710a0 2024-03-09T09:27:22.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.611+0000 7f8f34e00640 1 --2- 172.21.15.67:0/2173492450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f300710a0 0x7f8f30153970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f30073ef0 tx=0x7f8f1c02fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4092f17fdc44eec server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:22.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.611+0000 7f8f2fe00640 1 -- 172.21.15.67:0/2173492450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c03d070 con 0x7f8f300710a0 2024-03-09T09:27:22.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.611+0000 7f8f2fe00640 1 -- 172.21.15.67:0/2173492450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c030ad0 con 0x7f8f300710a0 2024-03-09T09:27:22.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.611+0000 7f8f2fe00640 1 -- 172.21.15.67:0/2173492450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c038440 con 0x7f8f300710a0 2024-03-09T09:27:22.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.612+0000 7f8f36c00640 1 -- 172.21.15.67:0/2173492450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f300710a0 msgr2=0x7f8f30153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:22.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.612+0000 7f8f36c00640 1 --2- 172.21.15.67:0/2173492450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f300710a0 0x7f8f30153970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f30073ef0 tx=0x7f8f1c02fd20 comp rx=0 tx=0).stop 2024-03-09T09:27:22.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.613+0000 7f8f36c00640 1 -- 172.21.15.67:0/2173492450 shutdown_connections 2024-03-09T09:27:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.613+0000 7f8f36c00640 1 --2- 172.21.15.67:0/2173492450 >> v2:172.21.15.134:3300/0 conn(0x7f8f30151360 0x7f8f30153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.613+0000 7f8f36c00640 1 --2- 172.21.15.67:0/2173492450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f300710a0 0x7f8f30153970 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.613+0000 7f8f36c00640 1 -- 172.21.15.67:0/2173492450 >> 172.21.15.67:0/2173492450 conn(0x7f8f300c7300 msgr2=0x7f8f300c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.613+0000 7f8f36c00640 1 -- 172.21.15.67:0/2173492450 shutdown_connections 2024-03-09T09:27:22.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.613+0000 7f8f36c00640 1 -- 172.21.15.67:0/2173492450 wait complete. 2024-03-09T09:27:22.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.614+0000 7f8f36c00640 1 Processor -- start 2024-03-09T09:27:22.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.615+0000 7f8f36c00640 1 -- start start 2024-03-09T09:27:22.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.615+0000 7f8f36c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f300710a0 0x7f8f3014a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.615+0000 7f8f36c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f300c1f70 0x7f8f3014ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.615+0000 7f8f36c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f30151360 0x7f8f30147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f36c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f30070d50 con 0x7f8f300c1f70 2024-03-09T09:27:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f36c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f30070830 con 0x7f8f30151360 2024-03-09T09:27:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f36c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f30070ed0 con 0x7f8f300710a0 2024-03-09T09:27:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f35800640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f300710a0 msgr2=0x7f8f3014a1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f35800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f300710a0 0x7f8f3014a1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f34e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f300c1f70 0x7f8f3014ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:22.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f34e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f300c1f70 0x7f8f3014ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39544/0 (socket says 172.21.15.67:39544) 2024-03-09T09:27:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f34e00640 1 -- 172.21.15.67:0/2989829047 learned_addr learned my addr 172.21.15.67:0/2989829047 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f36200640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f30151360 0x7f8f30147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f34e00640 1 -- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f300710a0 msgr2=0x7f8f3014a1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f34e00640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f300710a0 0x7f8f3014a1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f34e00640 1 -- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f30151360 msgr2=0x7f8f30147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f34e00640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f30151360 0x7f8f30147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.616+0000 7f8f34e00640 1 -- 172.21.15.67:0/2989829047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f301eaf40 con 0x7f8f300c1f70 2024-03-09T09:27:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.617+0000 7f8f34e00640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f300c1f70 0x7f8f3014ab30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009960 tx=0x7f8f1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.617+0000 7f8f2d600640 1 -- 172.21.15.67:0/2989829047 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c03d040 con 0x7f8f300c1f70 2024-03-09T09:27:22.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.617+0000 7f8f2d600640 1 -- 172.21.15.67:0/2989829047 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c048070 con 0x7f8f300c1f70 2024-03-09T09:27:22.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.617+0000 7f8f36c00640 1 -- 172.21.15.67:0/2989829047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f301eb200 con 0x7f8f300c1f70 2024-03-09T09:27:22.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.617+0000 7f8f2d600640 1 -- 172.21.15.67:0/2989829047 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c043680 con 0x7f8f300c1f70 2024-03-09T09:27:22.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.617+0000 7f8f36c00640 1 -- 172.21.15.67:0/2989829047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3014b070 con 0x7f8f300c1f70 2024-03-09T09:27:22.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.619+0000 7f8f36c00640 1 -- 172.21.15.67:0/2989829047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f300ac2c0 con 0x7f8f300c1f70 2024-03-09T09:27:22.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.620+0000 7f8f2d600640 1 -- 172.21.15.67:0/2989829047 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8f1c043820 con 0x7f8f300c1f70 2024-03-09T09:27:22.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.622+0000 7f8f2d600640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f1006b4b0 0x7f8f1006d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:22.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.622+0000 7f8f2d600640 1 -- 172.21.15.67:0/2989829047 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6721+0+0 (secure 0 0 0) 0x7f8f1c0b0ae0 con 0x7f8f300c1f70 2024-03-09T09:27:22.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.622+0000 7f8f35800640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f1006b4b0 0x7f8f1006d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:22.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.623+0000 7f8f35800640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f1006b4b0 0x7f8f1006d970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8f24001030 tx=0x7f8f24007a30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:22.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.627+0000 7f8f2d600640 1 -- 172.21.15.67:0/2989829047 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8f1c07f680 con 0x7f8f300c1f70 2024-03-09T09:27:22.852 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/uniwidth/cjk.h 2024-03-09T09:27:22.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/uniwidth/width.c 2024-03-09T09:27:22.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strtoull.c 2024-03-09T09:27:22.859 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/opendir-safer.c 2024-03-09T09:27:22.860 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/hash-pjw.h 2024-03-09T09:27:22.860 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/uname.c 2024-03-09T09:27:22.860 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/full-read.c 2024-03-09T09:27:22.860 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/priv-set.h 2024-03-09T09:27:22.861 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/close.c 2024-03-09T09:27:22.861 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fpending.h 2024-03-09T09:27:22.861 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/i-ring.h 2024-03-09T09:27:22.862 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/nl_langinfo.c 2024-03-09T09:27:22.862 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xmemxfrm.h 2024-03-09T09:27:22.862 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/c-strcase.h 2024-03-09T09:27:22.862 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fchdir.c 2024-03-09T09:27:22.862 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/canon-host.c 2024-03-09T09:27:22.864 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/signbitf.c 2024-03-09T09:27:22.864 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xnanosleep.h 2024-03-09T09:27:22.864 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/tempname.c 2024-03-09T09:27:22.865 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xreadlink.c 2024-03-09T09:27:22.865 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xgetcwd.h 2024-03-09T09:27:22.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/regex.c 2024-03-09T09:27:22.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/diacrit.h 2024-03-09T09:27:22.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/posixver.c 2024-03-09T09:27:22.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getugroups.c 2024-03-09T09:27:22.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/readutmp.h 2024-03-09T09:27:22.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/quote.h 2024-03-09T09:27:22.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/quotearg.c 2024-03-09T09:27:22.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/filevercmp.c 2024-03-09T09:27:22.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/printf-args.h 2024-03-09T09:27:22.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mountlist.h 2024-03-09T09:27:22.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstrtol-error.c 2024-03-09T09:27:22.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/rename.c 2024-03-09T09:27:22.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sha1.h 2024-03-09T09:27:22.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strndup.c 2024-03-09T09:27:22.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/priv-set.c 2024-03-09T09:27:22.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xfts.h 2024-03-09T09:27:22.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/argv-iter.c 2024-03-09T09:27:22.870 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stdio--.h 2024-03-09T09:27:22.870 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/yesno.c 2024-03-09T09:27:22.870 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getopt.c 2024-03-09T09:27:22.870 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strpbrk.c 2024-03-09T09:27:22.870 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dirname-lgpl.c 2024-03-09T09:27:22.871 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/filemode.h 2024-03-09T09:27:22.871 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbrtowc.c 2024-03-09T09:27:22.871 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv_open.c 2024-03-09T09:27:22.871 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/euidaccess-stat.c 2024-03-09T09:27:22.871 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stat-time.h 2024-03-09T09:27:22.871 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/frexpl.c 2024-03-09T09:27:22.872 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sys_select.in.h 2024-03-09T09:27:22.872 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strftime.h 2024-03-09T09:27:22.872 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strftime.c 2024-03-09T09:27:22.872 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv_open-aix.gperf 2024-03-09T09:27:22.872 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/buffer-lcm.c 2024-03-09T09:27:22.872 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unistr.h 2024-03-09T09:27:22.873 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/modechange.c 2024-03-09T09:27:22.873 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/tempname.h 2024-03-09T09:27:22.873 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/faccessat.c 2024-03-09T09:27:22.874 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unicodeio.h 2024-03-09T09:27:22.874 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/filemode.c 2024-03-09T09:27:22.874 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strdup.c 2024-03-09T09:27:22.874 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strsignal.c 2024-03-09T09:27:22.874 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/euidaccess.c 2024-03-09T09:27:22.875 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/select.c 2024-03-09T09:27:22.875 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memcoll.h 2024-03-09T09:27:22.875 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fseeko.c 2024-03-09T09:27:22.875 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/langinfo.in.h 2024-03-09T09:27:22.876 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getopt.in.h 2024-03-09T09:27:22.876 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/localcharset.c 2024-03-09T09:27:22.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/isapipe.c 2024-03-09T09:27:22.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/settime.c 2024-03-09T09:27:22.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/bitrotate.h 2024-03-09T09:27:22.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/isnanl-nolibm.h 2024-03-09T09:27:22.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fcntl.c 2024-03-09T09:27:22.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/exclude.h 2024-03-09T09:27:22.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/symlinkat.c 2024-03-09T09:27:22.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fpending.c 2024-03-09T09:27:22.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/savewd.c 2024-03-09T09:27:22.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fchownat.c 2024-03-09T09:27:22.878 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/root-dev-ino.h 2024-03-09T09:27:22.879 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/randread.h 2024-03-09T09:27:22.879 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/freadseek.h 2024-03-09T09:27:22.879 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strtoll.c 2024-03-09T09:27:22.879 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stdio-safer.h 2024-03-09T09:27:22.879 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strtoumax.c 2024-03-09T09:27:22.879 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/argv-iter.h 2024-03-09T09:27:22.881 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dirent-safer.h 2024-03-09T09:27:22.881 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dirent.in.h 2024-03-09T09:27:22.881 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/malloca.valgrind 2024-03-09T09:27:22.881 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/savewd.h 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/uinttostr.c 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getline.c 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/chdir-long.h 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/cycle-check.h 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fts-cycle.c 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/file-has-acl.c 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/areadlink.h 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sig2str.h 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memcpy.c 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/freading.h 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/full-write.h 2024-03-09T09:27:22.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unistd.in.h 2024-03-09T09:27:22.883 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/symlink.c 2024-03-09T09:27:22.883 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/se-selinux.in.h 2024-03-09T09:27:22.883 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/write-any-file.h 2024-03-09T09:27:22.883 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/printf-frexpl.h 2024-03-09T09:27:22.883 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/Makefile.in 2024-03-09T09:27:22.884 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/filenamecat.h 2024-03-09T09:27:22.890 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/malloca.c 2024-03-09T09:27:22.890 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/utimecmp.h 2024-03-09T09:27:22.890 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stdlib.in.h 2024-03-09T09:27:22.890 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stdio.in.h 2024-03-09T09:27:22.902 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbsinit.c 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/diacrit.c 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/se-context.in.h 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xvasprintf.c 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xgetcwd.c 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sockets.h 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/striconv.h 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mkstemp-safer.c 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/u64.h 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstrtoimax.c 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/time_r.c 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbchar.h 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xtime.h 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/set-mode-acl.c 2024-03-09T09:27:22.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/version-etc.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dup2.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unlinkat.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sigprocmask.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/hash-triple.h 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fprintftime.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unistd--.h 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/umaxtostr.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/i-ring.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/signbitd.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/ftello.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/lchown.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/lstat.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fpurge.c 2024-03-09T09:27:22.904 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fstatat.c 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/lseek.c 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fchmodat.c 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/alloca.c 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memcmp.c 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstrndup.h 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/frexp.c 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/nproc.h 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fsusage.h 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fd-safer-flag.c 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getdtablesize.c 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/savedir.c 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/wcwidth.c 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbsstr.c 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fpucw.h 2024-03-09T09:27:22.905 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv_open-hpux.h 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/readtokens0.h 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getdate.c 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/utimens.c 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getndelim2.c 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fflush.c 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/version-etc-fsf.c 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xsize.h 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xgethostname.c 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strnlen1.h 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/isnanl.c 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/same-inode.h 2024-03-09T09:27:22.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/regex_internal.h 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/full-read.h 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/intprops.h 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fsusage.c 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/save-cwd.c 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memchr.valgrind 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/exitfail.h 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/printf-frexp.h 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/isblank.c 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/arpa_inet.in.h 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strerror.c 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/freadptr.c 2024-03-09T09:27:22.907 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/openat.h 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv_open-irix.h 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/open.c 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/vprintf.c 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/same.h 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/gai_strerror.c 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/imaxtostr.c 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memcmp2.h 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xfreopen.c 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/rand-isaac.c 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/full-write.c 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/rmdir.c 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/save-cwd.h 2024-03-09T09:27:22.908 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/at-func2.c 2024-03-09T09:27:22.909 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/gethrxtime.c 2024-03-09T09:27:22.909 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strtod.c 2024-03-09T09:27:22.909 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbsalign.c 2024-03-09T09:27:22.909 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv_open-osf.gperf 2024-03-09T09:27:22.909 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/ref-add.sin 2024-03-09T09:27:22.909 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memcmp2.c 2024-03-09T09:27:22.909 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/gnulib.mk 2024-03-09T09:27:22.925 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/ref-del.sin 2024-03-09T09:27:22.925 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stpncpy.c 2024-03-09T09:27:22.925 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv.in.h 2024-03-09T09:27:22.925 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/posixtm.h 2024-03-09T09:27:22.925 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstrtol.c 2024-03-09T09:27:22.925 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getopt1.c 2024-03-09T09:27:22.926 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/quotearg.h 2024-03-09T09:27:22.926 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/cycle-check.c 2024-03-09T09:27:22.926 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/calloc.c 2024-03-09T09:27:22.926 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/areadlinkat.c 2024-03-09T09:27:22.926 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/hash-triple.c 2024-03-09T09:27:22.926 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getpass.c 2024-03-09T09:27:22.926 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstriconv.c 2024-03-09T09:27:22.926 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dirent--.h 2024-03-09T09:27:22.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sigaction.c 2024-03-09T09:27:22.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/linkat.c 2024-03-09T09:27:22.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unitypes.h 2024-03-09T09:27:22.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/linebuffer.c 2024-03-09T09:27:22.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fnmatch.c 2024-03-09T09:27:22.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xasprintf.c 2024-03-09T09:27:22.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/printf-parse.h 2024-03-09T09:27:22.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/string.in.h 2024-03-09T09:27:22.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mkdir.c 2024-03-09T09:27:22.928 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fchown-stub.c 2024-03-09T09:27:22.928 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/human.c 2024-03-09T09:27:22.928 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sockets.c 2024-03-09T09:27:22.928 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/canonicalize.h 2024-03-09T09:27:22.928 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getopt_int.h 2024-03-09T09:27:22.928 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/version-etc.h 2024-03-09T09:27:22.928 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stdlib--.h 2024-03-09T09:27:22.928 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/backupfile.h 2024-03-09T09:27:22.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/file-type.c 2024-03-09T09:27:22.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/file-set.h 2024-03-09T09:27:22.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbuiter.h 2024-03-09T09:27:22.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fcntl.in.h 2024-03-09T09:27:22.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/inet_ntop.c 2024-03-09T09:27:22.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fseterr.c 2024-03-09T09:27:22.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/close-stream.h 2024-03-09T09:27:22.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/freopen-safer.c 2024-03-09T09:27:22.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stat-macros.h 2024-03-09T09:27:22.929 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dirname.c 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dev-ino.h 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xalloc.h 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memcasecmp.h 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/hash.c 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/userspec.c 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/hard-locale.h 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xvasprintf.h 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/wcrtomb.c 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/gethrxtime.h 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fprintftime.h 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mgetgroups.c 2024-03-09T09:27:22.930 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/randread.c 2024-03-09T09:27:22.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstriconv.h 2024-03-09T09:27:22.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dup-safer-flag.c 2024-03-09T09:27:22.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/yesno.h 2024-03-09T09:27:22.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbscasecmp.c 2024-03-09T09:27:22.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strtoimax.c 2024-03-09T09:27:22.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbsrtowcs-state.c 2024-03-09T09:27:22.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mkstemp.c 2024-03-09T09:27:22.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xmemcoll.c 2024-03-09T09:27:22.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strintcmp.c 2024-03-09T09:27:22.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fclose.c 2024-03-09T09:27:22.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fd-reopen.c 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/inttostr.c 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/c-ctype.c 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/openat-die.c 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/error.c 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sig2str.c 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unlocked-io.h 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memxfrm.c 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/pipe-safer.c 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/backupfile.c 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv_open-osf.h 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/linebuffer.h 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xprintf.h 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/exclude.c 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/userspec.h 2024-03-09T09:27:22.932 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mkancesdirs.h 2024-03-09T09:27:22.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/hash-pjw.c 2024-03-09T09:27:22.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fd-safer.c 2024-03-09T09:27:22.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/long-options.h 2024-03-09T09:27:22.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/freadahead.h 2024-03-09T09:27:22.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/gethostname.c 2024-03-09T09:27:22.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/putenv.c 2024-03-09T09:27:22.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbrlen.c 2024-03-09T09:27:22.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/group-member.c 2024-03-09T09:27:22.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sha512.c 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/signal.in.h 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mktime.c 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/readlink.c 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mkdir-p.c 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getndelim2.h 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memxfrm.h 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/c-strtod.c 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fseterr.h 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv_open-hpux.gperf 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/filenamecat.c 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memmove.c 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/str-kmp.h 2024-03-09T09:27:22.934 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/base64.c 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/close-hook.c 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/dirchownmod.c 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/openat-safer.c 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/posixtm.c 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/printf-frexp.c 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/safe-read.h 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/regexec.c 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/uniwidth.h 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/write-any-file.c 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv_open-aix.h 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/unicodeio.c 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/gettime.c 2024-03-09T09:27:22.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strnumcmp.h 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/btowc.c 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sys_time.in.h 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/isnan.c 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getaddrinfo.c 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getpagesize.c 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/posixver.h 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/openat.c 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/closeout.c 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fopen.c 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memchr2.valgrind 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/time.in.h 2024-03-09T09:27:22.936 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/physmem.h 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/iconv_open-irix.gperf 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stat.c 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/siglist.h 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbswidth.h 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstrtod.h 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/md5.c 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getloadavg.c 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/getdate.h 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stdlib-safer.h 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/remove.c 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/malloc.c 2024-03-09T09:27:22.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/progname.h 2024-03-09T09:27:22.938 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/argmatch.h 2024-03-09T09:27:22.938 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/ChangeLog-2007 2024-03-09T09:27:22.938 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mbswidth.c 2024-03-09T09:27:22.938 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/error.h 2024-03-09T09:27:22.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.926+0000 7f8f36c00640 1 -- 172.21.15.67:0/2989829047 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8f3004f3c0 con 0x7f8f300c1f70 2024-03-09T09:27:22.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.927+0000 7f8f2d600640 1 -- 172.21.15.67:0/2989829047 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f8f1c07ef80 con 0x7f8f300c1f70 2024-03-09T09:27:22.939 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":276,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.929+0000 7f8f36c00640 1 -- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f1006b4b0 msgr2=0x7f8f1006d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.929+0000 7f8f36c00640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f1006b4b0 0x7f8f1006d970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8f24001030 tx=0x7f8f24007a30 comp rx=0 tx=0).stop 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.929+0000 7f8f36c00640 1 -- 172.21.15.67:0/2989829047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f300c1f70 msgr2=0x7f8f3014ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.929+0000 7f8f36c00640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f300c1f70 0x7f8f3014ab30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009960 tx=0x7f8f1c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.930+0000 7f8f36c00640 1 -- 172.21.15.67:0/2989829047 shutdown_connections 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.930+0000 7f8f36c00640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8f1006b4b0 0x7f8f1006d970 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.930+0000 7f8f36c00640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f30151360 0x7f8f30147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.930+0000 7f8f36c00640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f300c1f70 0x7f8f3014ab30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.930+0000 7f8f36c00640 1 --2- 172.21.15.67:0/2989829047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f300710a0 0x7f8f3014a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.930+0000 7f8f36c00640 1 -- 172.21.15.67:0/2989829047 >> 172.21.15.67:0/2989829047 conn(0x7f8f300c7300 msgr2=0x7f8f3014eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.930+0000 7f8f36c00640 1 -- 172.21.15.67:0/2989829047 shutdown_connections 2024-03-09T09:27:22.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:22.930+0000 7f8f36c00640 1 -- 172.21.15.67:0/2989829047 wait complete. 2024-03-09T09:27:22.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/obstack.c 2024-03-09T09:27:22.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/acl.h 2024-03-09T09:27:22.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strnumcmp-in.h 2024-03-09T09:27:22.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xstrtoul.c 2024-03-09T09:27:22.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/float+.h 2024-03-09T09:27:22.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/canonicalize.c 2024-03-09T09:27:22.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fcntl--.h 2024-03-09T09:27:22.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/wchar.in.h 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/trim.c 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/rpmatch.c 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/isapipe.h 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/memchr.c 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/localcharset.h 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/sig-handler.h 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/openat-proc.c 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/pathmax.h 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/close-stream.c 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/readutmp.c 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/float.in.h 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/creat-safer.c 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/randperm.c 2024-03-09T09:27:22.946 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/readtokens.c 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/safe-read.c 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xreadlink.h 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/stdarg.in.h 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xalloc-die.c 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/copy-acl.c 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fsync.c 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/strnumcmp.c 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/selinux-at.c 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/realloc.c 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fdopendir.c 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/alloca.in.h 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/freopen.c 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/mkdir-p.h 2024-03-09T09:27:22.947 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/isnand-nolibm.h 2024-03-09T09:27:22.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/propername.c 2024-03-09T09:27:22.948 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/fileblocks.c 2024-03-09T09:27:23.833 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xmemcoll.h 2024-03-09T09:27:23.834 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/hash.h 2024-03-09T09:27:23.835 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/c-strtold.c 2024-03-09T09:27:23.835 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/c-ctype.h 2024-03-09T09:27:23.835 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/lib/xmemxfrm.c 2024-03-09T09:27:23.836 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_require_config_h_first 2024-03-09T09:27:23.836 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ 2024-03-09T09:27:23.854 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/pt.po 2024-03-09T09:27:23.863 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/sl.po 2024-03-09T09:27:23.871 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/zh_TW.po 2024-03-09T09:27:23.879 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/sk.po 2024-03-09T09:27:23.888 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/el.gmo 2024-03-09T09:27:23.888 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/en.gmo 2024-03-09T09:27:23.888 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/lg.po 2024-03-09T09:27:23.894 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/sv.gmo 2024-03-09T09:27:23.899 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/pl.gmo 2024-03-09T09:27:23.901 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/insert-header.sin 2024-03-09T09:27:23.902 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/sk.gmo 2024-03-09T09:27:23.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ja.gmo 2024-03-09T09:27:23.903 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ko.po 2024-03-09T09:27:23.906 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/da.gmo 2024-03-09T09:27:23.909 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/LINGUAS 2024-03-09T09:27:23.910 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/zh_TW.gmo 2024-03-09T09:27:23.912 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/tr.gmo 2024-03-09T09:27:23.913 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/gl.gmo 2024-03-09T09:27:23.913 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/fi.po 2024-03-09T09:27:23.914 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/da.po 2024-03-09T09:27:23.918 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/fi.gmo 2024-03-09T09:27:23.919 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/quot.sed 2024-03-09T09:27:23.919 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/stamp-po 2024-03-09T09:27:23.920 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ms.gmo 2024-03-09T09:27:23.920 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/pt_BR.po 2024-03-09T09:27:23.923 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/Makevars 2024-03-09T09:27:23.924 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/boldquot.sed 2024-03-09T09:27:23.924 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/coreutils.pot 2024-03-09T09:27:23.927 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/de.po 2024-03-09T09:27:23.931 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/pt.gmo 2024-03-09T09:27:23.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/Makefile.in.in 2024-03-09T09:27:23.933 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/nl.gmo 2024-03-09T09:27:23.937 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/fr.gmo 2024-03-09T09:27:23.940 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ga.gmo 2024-03-09T09:27:23.942 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/cs.po 2024-03-09T09:27:23.945 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ru.po 2024-03-09T09:27:23.949 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/hu.po 2024-03-09T09:27:23.953 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/id.po 2024-03-09T09:27:23.957 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ru.gmo 2024-03-09T09:27:23.961 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/sl.gmo 2024-03-09T09:27:23.968 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/remove-potcdate.sin 2024-03-09T09:27:23.968 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/hu.gmo 2024-03-09T09:27:23.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/et.po 2024-03-09T09:27:23.972 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/gl.po 2024-03-09T09:27:23.975 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/vi.gmo 2024-03-09T09:27:23.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/bg.po 2024-03-09T09:27:23.981 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/pl.po 2024-03-09T09:27:23.983 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/nl.po 2024-03-09T09:27:23.986 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/pt_BR.gmo 2024-03-09T09:27:23.988 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ko.gmo 2024-03-09T09:27:23.988 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/de.gmo 2024-03-09T09:27:23.990 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/POTFILES.in 2024-03-09T09:27:23.990 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/sv.po 2024-03-09T09:27:23.992 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/it.po 2024-03-09T09:27:23.994 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/eu.gmo 2024-03-09T09:27:23.995 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/en.po 2024-03-09T09:27:23.996 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/it.gmo 2024-03-09T09:27:23.998 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/en@boldquot.header 2024-03-09T09:27:23.998 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/et.gmo 2024-03-09T09:27:23.999 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ja.po 2024-03-09T09:27:24.002 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/zh_CN.gmo 2024-03-09T09:27:24.004 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/uk.po 2024-03-09T09:27:24.006 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ca.gmo 2024-03-09T09:27:24.009 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/Rules-quot 2024-03-09T09:27:24.009 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ro.po 2024-03-09T09:27:24.010 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/nb.gmo 2024-03-09T09:27:24.011 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/bg.gmo 2024-03-09T09:27:24.013 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/be.po 2024-03-09T09:27:24.015 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/vi.po 2024-03-09T09:27:24.018 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/lt.gmo 2024-03-09T09:27:24.019 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/uk.gmo 2024-03-09T09:27:24.021 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/id.gmo 2024-03-09T09:27:24.024 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/af.po 2024-03-09T09:27:24.026 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/eu.po 2024-03-09T09:27:24.028 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ms.po 2024-03-09T09:27:24.031 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/en@quot.header 2024-03-09T09:27:24.031 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/af.gmo 2024-03-09T09:27:24.031 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/es.gmo 2024-03-09T09:27:24.031 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ga.po 2024-03-09T09:27:24.035 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ro.gmo 2024-03-09T09:27:24.035 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ca.po 2024-03-09T09:27:24.039 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/lg.gmo 2024-03-09T09:27:24.042 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/zh_CN.po 2024-03-09T09:27:24.045 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/cs.gmo 2024-03-09T09:27:24.048 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/nb.po 2024-03-09T09:27:24.050 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/ChangeLog-2007 2024-03-09T09:27:24.050 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/tr.po 2024-03-09T09:27:24.052 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/be.gmo 2024-03-09T09:27:24.053 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/fr.po 2024-03-09T09:27:24.055 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/el.po 2024-03-09T09:27:24.058 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/lt.po 2024-03-09T09:27:24.060 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/po/es.po 2024-03-09T09:27:24.063 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ 2024-03-09T09:27:24.332 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/ 2024-03-09T09:27:24.535 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/skip-seek 2024-03-09T09:27:24.535 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/unblock-sync 2024-03-09T09:27:24.535 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/skip-seek2 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/unblock 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/stderr 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/direct 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/skip-seek-past-file 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/not-rewound 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/skip-seek-past-dev 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/misc 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/dd/reblock 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tt-FF 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/t_tab_ 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/FF 2024-03-09T09:27:24.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/FnFn 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/o3ml24f-bl-tn 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+5-8b3l17f-FF 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+3ml24f-bl-tn 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/FFtn 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/a2l15-FF 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tFnFt 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tn2e5o3-t_tab 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3ml24f-t 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/nSml20-t-tFFFF 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tn2e8-t_tab 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/b3f-FF 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/2w60f-t_notab 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/FFn 2024-03-09T09:27:24.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/t_tab 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/b3-0F 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+4-8a2l17-FF 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+4b2l17f-0FF 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3a3l15-t 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tFFn 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3l24f-t 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/0FnFt 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3-FF 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/t-FF 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/nSml24-bl-FF 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+3l24f-bl 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/b3l15-t 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3b3f-FF 2024-03-09T09:27:24.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3b3f-0F 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W35b3l24f-lm 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/0Ft 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tt-bl 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3ml24-t-FF 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tFFt-lm 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/nSml20-bl-FF 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tn2e8o3-t_tab 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W28l24f-ll 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/nSml24-t-tFFFF 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/t-0FF 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tb3-FF 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/o3b3Sl24f-tn 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/o3mSl24f-bl-tn 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3b2l17-FF 2024-03-09T09:27:24.539 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/2f-t_notab 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/b2l15-FF 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3b3f-0FF 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/a3l15-t 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/o3Jml24f-lm-lo 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tb3-0FF 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/l24-t 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W35ml24f-lm-lo 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/a3-0F 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ml24f-0F 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/loli 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3ml24-t 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/a3l24f-lm 2024-03-09T09:27:24.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/Jb3l24f-lm 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/t-t 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/0F 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/w72l24f-ll 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ml24f-t 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/b3l15f-t 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/0FnFnt 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/b3l24f-lm 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/o3mSnl24fbltn 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/nJml24f-lmlolm 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/o3a3Sl24f-tn 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tne8-t_tab 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3ml24-FF 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/a3f-0FF 2024-03-09T09:27:24.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ml20-FF-t 2024-03-09T09:27:24.542 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/pr-tests 2024-03-09T09:27:24.542 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W35Jb3l24f-lm 2024-03-09T09:27:24.542 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ttb3-0FF 2024-03-09T09:27:24.542 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/nl24f-bl 2024-03-09T09:27:24.542 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/nSml24-t-t-FF 2024-03-09T09:27:24.542 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3b3l15f-t 2024-03-09T09:27:24.542 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/0FF 2024-03-09T09:27:24.542 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3f-0F 2024-03-09T09:27:24.542 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/Ja3l24f-lm 2024-03-09T09:27:24.542 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3a2l17-FF 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ml24f-t-0F 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tn_2e8-t_tab 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/o3a3Snl24f-tn 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/2s_w60f-t_nota 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/t-bl 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ml24-t-FF 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+3-7l24-FF 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/t_notab 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W72Jl24f-ll 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/a2l17-FF 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ml24-FF 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/b3f-0F 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+3l24f-0FF 2024-03-09T09:27:24.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+3ml20f-bl-FF 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/a3f-0F 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W35Jml24f-lmlo 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W27l24f-ll 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ml24-t 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/a3f-FF 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tFFt-bl 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tta3-FF 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3-0F 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/2sf-t_notab 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3a3l15f-t 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W26l24f-ll 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+7l24-FF 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ml24f-lm-lo 2024-03-09T09:27:24.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/l24-FF 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+2-5l24f-0FF 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3f-FF 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+3ml24f-tn-bl 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ta3-0FF 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3b3l15-t 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+2l24f-0FF 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tn_2e8S-t_tab 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ta3-FF 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tFFt-ll 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+6b3l13f-FF 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/2-Sf-t_notab 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/0FFnt 2024-03-09T09:27:24.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W20l24f-ll 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/Jml24f-lm-lo 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3l24-t 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/o3b3l24f-tn 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W35a3l24f-lm 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/2-S_f-t_notab 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3-5l24f-t 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/o3b3Snl24f-tn 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+2l24f-bl 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W35Ja3l24f-lm 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+8l20-FF 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/0FFt 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/o3a3l24f-tn 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/2s_f-t_notab 2024-03-09T09:27:24.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/nN15l24f-bl 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/0Fnt 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/4-7l24-FF 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tn 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/b2l17-FF 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/nJml24f-lmlmlo 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tta3-0FF 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+5a3l13f-0FF 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/nN1+3l24f-bl 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/2sw60f-t_notab 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/W-72l24f-ll 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tt-0FF 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/4l24-FF 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/3a3f-0F 2024-03-09T09:27:24.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tFFt 2024-03-09T09:27:24.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/n+6a2l17-FF 2024-03-09T09:27:24.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/a3l15f-t 2024-03-09T09:27:24.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/l24f-t 2024-03-09T09:27:24.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/b3f-0FF 2024-03-09T09:27:24.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tt-t 2024-03-09T09:27:24.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/ttb3-FF 2024-03-09T09:27:24.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/nSml20-t-t-FF 2024-03-09T09:27:24.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/pr/tne8o3-t_tab 2024-03-09T09:27:24.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/Coreutils.pm 2024-03-09T09:27:25.321 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rmdir/ 2024-03-09T09:27:25.377 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rmdir/t-slash 2024-03-09T09:27:25.379 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rmdir/ignore 2024-03-09T09:27:25.380 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rmdir/fail-perm 2024-03-09T09:27:25.380 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/ 2024-03-09T09:27:25.386 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/ignorable 2024-03-09T09:27:25.390 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/r-2 2024-03-09T09:27:25.390 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/fail-2eperm 2024-03-09T09:27:25.391 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/empty-inacc 2024-03-09T09:27:25.391 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/cycle 2024-03-09T09:27:25.392 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/sunos-1 2024-03-09T09:27:25.392 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/v-slash 2024-03-09T09:27:25.393 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/no-give-up 2024-03-09T09:27:25.393 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/dot-rel 2024-03-09T09:27:25.393 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/one-file-system 2024-03-09T09:27:25.393 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/i-never 2024-03-09T09:27:25.394 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/rm1 2024-03-09T09:27:25.395 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/unread2 2024-03-09T09:27:25.395 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/read-only 2024-03-09T09:27:25.396 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/deep-2 2024-03-09T09:27:25.396 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/r-4 2024-03-09T09:27:25.396 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/dir-no-w 2024-03-09T09:27:25.396 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/interactive-always 2024-03-09T09:27:25.396 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/one-file-system2 2024-03-09T09:27:25.397 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/unreadable 2024-03-09T09:27:25.397 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/r-1 2024-03-09T09:27:25.397 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/fail-eacces 2024-03-09T09:27:25.398 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/inaccessible 2024-03-09T09:27:25.399 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/unread3 2024-03-09T09:27:25.399 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/ir-1 2024-03-09T09:27:25.400 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/rm3 2024-03-09T09:27:25.401 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/rm5 2024-03-09T09:27:25.401 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/deep-1 2024-03-09T09:27:25.402 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/ext3-perf 2024-03-09T09:27:25.402 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/dangling-symlink 2024-03-09T09:27:25.402 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/isatty 2024-03-09T09:27:25.403 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/i-1 2024-03-09T09:27:25.403 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/dir-nonrecur 2024-03-09T09:27:25.403 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/interactive-once 2024-03-09T09:27:25.404 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/f-1 2024-03-09T09:27:25.404 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/fail-eperm 2024-03-09T09:27:25.404 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/empty-name 2024-03-09T09:27:25.405 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/hash 2024-03-09T09:27:25.405 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/i-no-r 2024-03-09T09:27:25.406 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/r-3 2024-03-09T09:27:25.406 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/readdir-bug 2024-03-09T09:27:25.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/rm2 2024-03-09T09:27:25.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/rm/rm4 2024-03-09T09:27:25.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/test-lib.sh 2024-03-09T09:27:25.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/sample-test 2024-03-09T09:27:25.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chown/ 2024-03-09T09:27:25.436 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chown/basic 2024-03-09T09:27:25.454 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chown/separator 2024-03-09T09:27:25.454 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chown/deref 2024-03-09T09:27:25.454 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chown/preserve-root 2024-03-09T09:27:25.454 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/id/ 2024-03-09T09:27:25.457 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/id/no-context 2024-03-09T09:27:25.460 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/CuTmpdir.pm 2024-03-09T09:27:25.460 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/Makefile.am 2024-03-09T09:27:25.461 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/ 2024-03-09T09:27:25.464 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/dir-slash 2024-03-09T09:27:25.466 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/preserve-2 2024-03-09T09:27:25.466 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/dir-vs-file 2024-03-09T09:27:25.466 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/preserve-slink-time 2024-03-09T09:27:25.467 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/same-file 2024-03-09T09:27:25.467 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/src-base-dot 2024-03-09T09:27:25.468 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/cp-a-selinux 2024-03-09T09:27:25.468 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/r-vs-symlink 2024-03-09T09:27:25.468 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/capability 2024-03-09T09:27:25.468 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/existing-perm-race 2024-03-09T09:27:25.469 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/parent-perm-race 2024-03-09T09:27:25.469 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/reflink-auto 2024-03-09T09:27:25.469 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/cp-mv-enotsup-xattr 2024-03-09T09:27:25.469 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/no-deref-link2 2024-03-09T09:27:25.469 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/cp-mv-backup 2024-03-09T09:27:25.470 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/link-no-deref 2024-03-09T09:27:25.471 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/file-perm-race 2024-03-09T09:27:25.471 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/abuse 2024-03-09T09:27:25.472 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/link-heap 2024-03-09T09:27:25.472 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/backup-dir 2024-03-09T09:27:25.473 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/preserve-gid 2024-03-09T09:27:25.473 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/link 2024-03-09T09:27:25.473 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/cp-parents 2024-03-09T09:27:25.473 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/no-deref-link1 2024-03-09T09:27:25.473 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/into-self 2024-03-09T09:27:25.474 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/parent-perm 2024-03-09T09:27:25.474 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/cp-deref 2024-03-09T09:27:25.475 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/proc-short-read 2024-03-09T09:27:25.475 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/backup-is-src 2024-03-09T09:27:25.475 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/cp-i 2024-03-09T09:27:25.476 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/special-f 2024-03-09T09:27:25.476 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/proc-zero-len 2024-03-09T09:27:25.477 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/slink-2-slink 2024-03-09T09:27:25.477 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/special-bits 2024-03-09T09:27:25.477 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/fail-perm 2024-03-09T09:27:25.477 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/link-preserve 2024-03-09T09:27:25.478 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/dir-rm-dest 2024-03-09T09:27:25.478 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/backup-1 2024-03-09T09:27:25.479 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/cp-HL 2024-03-09T09:27:25.479 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/thru-dangling 2024-03-09T09:27:25.479 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/sparse 2024-03-09T09:27:25.480 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/deref-slink 2024-03-09T09:27:25.480 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/symlink-slash 2024-03-09T09:27:25.480 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/acl 2024-03-09T09:27:25.480 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/perm 2024-03-09T09:27:25.481 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/reflink-perm 2024-03-09T09:27:25.481 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/cp/no-deref-link3 2024-03-09T09:27:25.481 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/ 2024-03-09T09:27:25.507 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/t-slash 2024-03-09T09:27:25.737 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/p-v 2024-03-09T09:27:25.737 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/p-3 2024-03-09T09:27:25.738 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/writable-under-readonly 2024-03-09T09:27:25.739 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/p-slashdot 2024-03-09T09:27:25.739 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/selinux 2024-03-09T09:27:25.739 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/p-1 2024-03-09T09:27:25.739 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/p-2 2024-03-09T09:27:25.740 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/parents 2024-03-09T09:27:25.740 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/special-1 2024-03-09T09:27:25.740 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/perm 2024-03-09T09:27:25.740 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mkdir/p-thru-slink 2024-03-09T09:27:25.740 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/require-perl 2024-03-09T09:27:25.740 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ln/ 2024-03-09T09:27:25.751 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ln/target-1 2024-03-09T09:27:25.758 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ln/hard-to-sym 2024-03-09T09:27:25.758 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ln/hard-backup 2024-03-09T09:27:25.759 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ln/sf-1 2024-03-09T09:27:25.759 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ln/backup-1 2024-03-09T09:27:25.759 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ln/slash-decorated-nonexistent-dest 2024-03-09T09:27:25.759 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ln/misc 2024-03-09T09:27:25.760 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/lang-default 2024-03-09T09:27:25.760 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/check.mk 2024-03-09T09:27:25.760 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/Makefile.in 2024-03-09T09:27:25.762 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/install/ 2024-03-09T09:27:25.770 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/install/create-leading 2024-03-09T09:27:25.785 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/install/install-C-selinux 2024-03-09T09:27:25.785 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/install/trap 2024-03-09T09:27:25.785 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/install/basic-1 2024-03-09T09:27:25.785 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/install/strip-program 2024-03-09T09:27:25.786 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/install/d-slashdot 2024-03-09T09:27:25.786 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/install/install-C 2024-03-09T09:27:25.786 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/install/install-C-root 2024-03-09T09:27:25.786 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/ 2024-03-09T09:27:25.793 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/c-option 2024-03-09T09:27:25.797 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/octal 2024-03-09T09:27:25.798 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/umask-x 2024-03-09T09:27:25.798 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/inaccessible 2024-03-09T09:27:25.799 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/equal-x 2024-03-09T09:27:25.799 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/silent 2024-03-09T09:27:25.800 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/equals 2024-03-09T09:27:25.800 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/usage 2024-03-09T09:27:25.800 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/thru-dangling 2024-03-09T09:27:25.800 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/setgid 2024-03-09T09:27:25.802 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chmod/no-x 2024-03-09T09:27:25.802 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/df/ 2024-03-09T09:27:25.814 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/df/unreadable 2024-03-09T09:27:25.833 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/df/total-verify 2024-03-09T09:27:25.833 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/other-fs-tmpdir 2024-03-09T09:27:25.834 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/ 2024-03-09T09:27:25.840 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/basic 2024-03-09T09:27:25.846 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/exclude 2024-03-09T09:27:25.846 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/one-file-system 2024-03-09T09:27:25.846 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/long-sloop 2024-03-09T09:27:25.847 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/slink 2024-03-09T09:27:25.847 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/inacc-dir 2024-03-09T09:27:25.848 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/inaccessible-cwd 2024-03-09T09:27:25.848 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/inacc-dest 2024-03-09T09:27:25.848 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/deref-args 2024-03-09T09:27:25.849 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/deref 2024-03-09T09:27:25.849 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/trailing-slash 2024-03-09T09:27:25.849 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/files0-from 2024-03-09T09:27:25.849 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/two-args 2024-03-09T09:27:25.849 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/hard-link 2024-03-09T09:27:25.851 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/long-from-unreadable 2024-03-09T09:27:25.851 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/8gb 2024-03-09T09:27:25.851 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/2g 2024-03-09T09:27:25.852 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/restore-wd 2024-03-09T09:27:25.852 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/slash 2024-03-09T09:27:25.853 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/no-x 2024-03-09T09:27:25.853 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/fd-leak 2024-03-09T09:27:25.853 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/du/no-deref 2024-03-09T09:27:25.853 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/ 2024-03-09T09:27:25.895 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/recursive 2024-03-09T09:27:25.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:27:26.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.150+0000 7ff8d6a00640 1 Processor -- start 2024-03-09T09:27:26.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.150+0000 7ff8d6a00640 1 -- start start 2024-03-09T09:27:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.151+0000 7ff8d6a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.151+0000 7ff8d6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 0x7ff8d01562d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.151+0000 7ff8d6a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d00729e0 con 0x7ff8d01532c0 2024-03-09T09:27:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.152+0000 7ff8d6a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d0073310 con 0x7ff8d00c2e80 2024-03-09T09:27:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.152+0000 7ff8d6a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00c2ab0 con 0x7ff8d0155e40 2024-03-09T09:27:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.152+0000 7ff8d4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.152+0000 7ff8d4c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39558/0 (socket says 172.21.15.67:39558) 2024-03-09T09:27:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.152+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 learned_addr learned my addr 172.21.15.67:0/3038045459 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.152+0000 7ff8d6000640 1 -- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 msgr2=0x7ff8d01562d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.152+0000 7ff8d6000640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 0x7ff8d01562d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.152+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 msgr2=0x7ff8d00c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:27:26.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.152+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 msgr2=0x7ff8d00c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:27:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.152+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:26.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.152+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:26.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.353+0000 7ff8d6000640 1 -- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 msgr2=0x7ff8d01562d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:26.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.353+0000 7ff8d6000640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 0x7ff8d01562d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:27:26.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.353+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:26.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.354+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 msgr2=0x7ff8d00c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:27:26.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.354+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 msgr2=0x7ff8d00c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:27:26.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.354+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:26.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.354+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:27:26.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.754+0000 7ff8d6000640 1 -- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 msgr2=0x7ff8d01562d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.754+0000 7ff8d6000640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 0x7ff8d01562d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:27:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.755+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.756+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 msgr2=0x7ff8d00c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:27:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.756+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 msgr2=0x7ff8d00c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:27:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.756+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:26.756+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:27:26.850 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/readdir-mountpoint-inode 2024-03-09T09:27:26.850 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/multihardlink 2024-03-09T09:27:26.850 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/color-dtype-dir 2024-03-09T09:27:26.851 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/file-type 2024-03-09T09:27:26.851 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/rt-1 2024-03-09T09:27:26.852 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/stat-free-symlinks 2024-03-09T09:27:26.852 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/no-cap 2024-03-09T09:27:26.852 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/capability 2024-03-09T09:27:26.852 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/inode 2024-03-09T09:27:26.853 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/color-norm 2024-03-09T09:27:26.853 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/abmon-align 2024-03-09T09:27:26.853 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/dangle 2024-03-09T09:27:26.854 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/dired 2024-03-09T09:27:26.854 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/m-option 2024-03-09T09:27:26.855 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/nameless-uid 2024-03-09T09:27:26.855 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/proc-selinux-segfault 2024-03-09T09:27:26.856 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/stat-vs-dirent 2024-03-09T09:27:26.856 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/color-clear-to-eol 2024-03-09T09:27:26.856 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/no-arg 2024-03-09T09:27:26.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/follow-slink 2024-03-09T09:27:26.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/symlink-slash 2024-03-09T09:27:26.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/stat-dtype 2024-03-09T09:27:26.857 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/x-option 2024-03-09T09:27:26.858 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/infloop 2024-03-09T09:27:26.858 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/ls/stat-failed 2024-03-09T09:27:26.858 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/ 2024-03-09T09:27:26.935 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/hard-3 2024-03-09T09:27:26.961 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/mv-n 2024-03-09T09:27:26.961 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/to-symlink 2024-03-09T09:27:26.961 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/part-fail 2024-03-09T09:27:26.962 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/hard-4 2024-03-09T09:27:26.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/leak-fd 2024-03-09T09:27:26.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/no-target-dir 2024-03-09T09:27:26.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/part-rename 2024-03-09T09:27:26.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/dir-file 2024-03-09T09:27:26.963 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/hard-2 2024-03-09T09:27:26.964 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/part-hardlink 2024-03-09T09:27:26.968 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/sticky-to-xpart 2024-03-09T09:27:26.968 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/mv-special-1 2024-03-09T09:27:26.968 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/trailing-slash 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/dir2dir 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/into-self-2 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/backup-dir 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/diag 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/atomic 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/force 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/into-self-3 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/into-self 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/i-2 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/childproof 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/perm-1 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/i-3 2024-03-09T09:27:26.969 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/partition-perm 2024-03-09T09:27:26.972 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/part-symlink 2024-03-09T09:27:26.972 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/backup-is-src 2024-03-09T09:27:26.972 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/i-1 2024-03-09T09:27:26.973 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/update 2024-03-09T09:27:26.973 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/i-4 2024-03-09T09:27:26.973 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/hard-verbose 2024-03-09T09:27:26.973 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/dup-source 2024-03-09T09:27:26.973 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/i-link-no 2024-03-09T09:27:26.973 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/atomic2 2024-03-09T09:27:26.974 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/into-self-4 2024-03-09T09:27:26.974 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/hard-link-1 2024-03-09T09:27:26.976 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/i-5 2024-03-09T09:27:26.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/mv/acl 2024-03-09T09:27:26.977 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/ 2024-03-09T09:27:27.013 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/fifo 2024-03-09T09:27:27.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:27.555+0000 7ff8d6000640 1 -- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 msgr2=0x7ff8d01562d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:27.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:27.555+0000 7ff8d6000640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 0x7ff8d01562d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:27:27.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:27.557+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:27.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:27.558+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 msgr2=0x7ff8d00c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:27:27.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:27.558+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 msgr2=0x7ff8d00c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:27:27.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:27.558+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:27.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:27.558+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:27:27.687 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/empty-file 2024-03-09T09:27:27.687 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/read-only 2024-03-09T09:27:27.687 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/fail-diag 2024-03-09T09:27:27.688 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/trailing-slash 2024-03-09T09:27:27.688 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/no-dereference 2024-03-09T09:27:27.688 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/dir-1 2024-03-09T09:27:27.688 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/dangling-symlink 2024-03-09T09:27:27.689 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/now-owned-by-other 2024-03-09T09:27:27.689 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/no-create-missing 2024-03-09T09:27:27.689 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/no-rights 2024-03-09T09:27:27.689 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/obsolescent 2024-03-09T09:27:27.690 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/60-seconds 2024-03-09T09:27:27.690 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/not-owner 2024-03-09T09:27:27.691 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/touch/relative 2024-03-09T09:27:27.691 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/ 2024-03-09T09:27:27.719 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sort-merge-fdlimit 2024-03-09T09:27:27.742 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/basename 2024-03-09T09:27:27.742 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/stat-fmt 2024-03-09T09:27:27.743 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/pr 2024-03-09T09:27:27.743 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/dirname 2024-03-09T09:27:27.743 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/chcon-fail 2024-03-09T09:27:27.743 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/head-elide-tail 2024-03-09T09:27:27.743 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/chroot-credentials 2024-03-09T09:27:27.743 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/nl 2024-03-09T09:27:27.743 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/tee-dash 2024-03-09T09:27:27.743 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/stat-hyphen 2024-03-09T09:27:27.746 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sort-rand 2024-03-09T09:27:27.746 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/expr 2024-03-09T09:27:27.746 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/fmt-long-line 2024-03-09T09:27:27.746 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/split-fail 2024-03-09T09:27:27.746 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/truncate-relative 2024-03-09T09:27:27.746 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/truncate-dir-fail 2024-03-09T09:27:27.746 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/csplit 2024-03-09T09:27:27.746 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/od-N 2024-03-09T09:27:27.747 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/xattr 2024-03-09T09:27:27.747 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/mktemp 2024-03-09T09:27:27.748 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/cut 2024-03-09T09:27:27.748 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/cat-proc 2024-03-09T09:27:27.749 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/comm 2024-03-09T09:27:27.749 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/timeout-parameters 2024-03-09T09:27:27.749 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/tac-continue 2024-03-09T09:27:27.750 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/tty-eof 2024-03-09T09:27:27.750 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/df-P 2024-03-09T09:27:27.751 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/shuf 2024-03-09T09:27:27.751 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/env 2024-03-09T09:27:27.751 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/od 2024-03-09T09:27:27.751 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/split-a 2024-03-09T09:27:27.751 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/tail 2024-03-09T09:27:27.753 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/stat-slash 2024-03-09T09:27:27.753 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/timeout 2024-03-09T09:27:27.755 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/close-stdout 2024-03-09T09:27:27.755 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/wc-files0-from 2024-03-09T09:27:27.755 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sum 2024-03-09T09:27:27.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/expand 2024-03-09T09:27:27.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sum-sysv 2024-03-09T09:27:27.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/wc-parallel 2024-03-09T09:27:27.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/md5sum-newline 2024-03-09T09:27:27.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sha224sum 2024-03-09T09:27:27.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/stty 2024-03-09T09:27:27.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/printf-hex 2024-03-09T09:27:27.758 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/id-context 2024-03-09T09:27:27.758 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/stat-printf 2024-03-09T09:27:27.758 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/pwd-option 2024-03-09T09:27:27.758 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sha256sum 2024-03-09T09:27:27.758 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sort-version 2024-03-09T09:27:27.759 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/groups-dash 2024-03-09T09:27:27.759 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/printenv 2024-03-09T09:27:27.760 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/seq 2024-03-09T09:27:27.760 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/fmt 2024-03-09T09:27:27.760 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/cat-buf 2024-03-09T09:27:27.761 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/dircolors 2024-03-09T09:27:27.761 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/fold 2024-03-09T09:27:27.762 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/selinux 2024-03-09T09:27:27.763 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/md5sum-parallel 2024-03-09T09:27:27.763 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/md5sum 2024-03-09T09:27:27.763 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/nice-fail 2024-03-09T09:27:27.764 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/groups-version 2024-03-09T09:27:27.766 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/date-sec 2024-03-09T09:27:27.766 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/split-l 2024-03-09T09:27:27.767 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/false-status 2024-03-09T09:27:27.767 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sha384sum 2024-03-09T09:27:27.767 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/truncate-dangling-symlink 2024-03-09T09:27:27.767 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/od-multiple-t 2024-03-09T09:27:27.767 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/wc-files0 2024-03-09T09:27:27.767 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/nohup 2024-03-09T09:27:27.768 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/truncate-no-create-missing 2024-03-09T09:27:27.768 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/head-pos 2024-03-09T09:27:27.769 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/uniq 2024-03-09T09:27:27.769 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/test-diag 2024-03-09T09:27:27.769 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/tac 2024-03-09T09:27:27.772 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sort-continue 2024-03-09T09:27:27.772 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/head 2024-03-09T09:27:27.772 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/ptx 2024-03-09T09:27:27.773 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/nproc-avail 2024-03-09T09:27:27.773 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/su-fail 2024-03-09T09:27:27.774 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/od-x8 2024-03-09T09:27:27.774 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/nice 2024-03-09T09:27:27.774 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/chcon 2024-03-09T09:27:27.775 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/head-c 2024-03-09T09:27:27.776 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sort-compress 2024-03-09T09:27:27.777 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/df 2024-03-09T09:27:27.777 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/chroot-fail 2024-03-09T09:27:27.777 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/date-next-dow 2024-03-09T09:27:27.777 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sha1sum 2024-03-09T09:27:27.778 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sha1sum-vec 2024-03-09T09:27:27.783 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/pathchk1 2024-03-09T09:27:27.783 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/arch 2024-03-09T09:27:27.783 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sort-month 2024-03-09T09:27:27.783 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/shred-passes 2024-03-09T09:27:27.784 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sha512sum 2024-03-09T09:27:27.784 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/paste 2024-03-09T09:27:27.785 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/wc 2024-03-09T09:27:27.785 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/ls-time 2024-03-09T09:27:27.785 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/pwd-unreadable-parent 2024-03-09T09:27:27.786 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/test 2024-03-09T09:27:27.786 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/truncate-parameters 2024-03-09T09:27:27.787 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sort-files0-from 2024-03-09T09:27:27.794 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/base64 2024-03-09T09:27:27.794 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/ptx-overrun 2024-03-09T09:27:27.794 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/printf-cov 2024-03-09T09:27:27.796 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/tee 2024-03-09T09:27:27.797 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/mknod 2024-03-09T09:27:27.797 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/printf-surprise 2024-03-09T09:27:27.797 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/ls-misc 2024-03-09T09:27:27.797 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/nproc-positive 2024-03-09T09:27:27.797 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/join 2024-03-09T09:27:27.797 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sort-merge 2024-03-09T09:27:27.797 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/factor 2024-03-09T09:27:27.797 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/sort 2024-03-09T09:27:27.799 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/truncate-fifo 2024-03-09T09:27:27.799 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/truncate-fail-diag 2024-03-09T09:27:27.799 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/stty-invalid 2024-03-09T09:27:27.799 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/env-null 2024-03-09T09:27:27.799 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/printf 2024-03-09T09:27:27.799 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/seq-long-double 2024-03-09T09:27:27.803 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/pwd-long 2024-03-09T09:27:27.803 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/tsort 2024-03-09T09:27:27.803 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/runcon-no-reorder 2024-03-09T09:27:27.803 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/date 2024-03-09T09:27:27.803 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/shred-exact 2024-03-09T09:27:27.804 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/shred-remove 2024-03-09T09:27:27.804 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/stdbuf 2024-03-09T09:27:27.804 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/truncate-overflow 2024-03-09T09:27:27.804 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/stty-row-col 2024-03-09T09:27:27.804 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/truncate-owned-by-other 2024-03-09T09:27:27.804 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/xstrtol 2024-03-09T09:27:27.804 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/invalid-opt 2024-03-09T09:27:27.804 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/readlink-fp-loop 2024-03-09T09:27:27.804 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/tr 2024-03-09T09:27:27.804 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/help-version 2024-03-09T09:27:27.804 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/id-groups 2024-03-09T09:27:27.805 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/misc/unexpand 2024-03-09T09:27:27.805 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chgrp/ 2024-03-09T09:27:27.805 INFO:tasks.workunit.client.0.smithi067.stderr:+ rm -rv coreutils-8.5 2024-03-09T09:27:28.090 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chgrp/basic 2024-03-09T09:27:28.143 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chgrp/default-no-deref 2024-03-09T09:27:28.144 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chgrp/deref 2024-03-09T09:27:28.144 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chgrp/posix-H 2024-03-09T09:27:28.145 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chgrp/recurse 2024-03-09T09:27:28.145 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/chgrp/no-x 2024-03-09T09:27:28.146 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/ 2024-03-09T09:27:28.153 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/big-4gb 2024-03-09T09:27:28.156 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/follow-stdin 2024-03-09T09:27:28.158 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/F-vs-rename 2024-03-09T09:27:28.158 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/inotify-rotate 2024-03-09T09:27:28.158 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/pid 2024-03-09T09:27:28.158 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/infloop-1 2024-03-09T09:27:28.158 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/assert-2 2024-03-09T09:27:28.158 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/pipe-f 2024-03-09T09:27:28.159 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/pipe-f2 2024-03-09T09:27:28.159 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/start-middle 2024-03-09T09:27:28.159 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/inotify-hash-abuse2 2024-03-09T09:27:28.159 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/inotify-hash-abuse 2024-03-09T09:27:28.159 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/inotify-race 2024-03-09T09:27:28.159 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/append-only 2024-03-09T09:27:28.159 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/tail-n0f 2024-03-09T09:27:28.159 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/flush-initial 2024-03-09T09:27:28.159 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/assert 2024-03-09T09:27:28.160 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/wait 2024-03-09T09:27:28.160 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/tail-2/proc-ksyms 2024-03-09T09:27:28.160 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/envvar-check 2024-03-09T09:27:28.160 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/readlink/ 2024-03-09T09:27:28.174 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/readlink/can-f 2024-03-09T09:27:28.182 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/readlink/can-m 2024-03-09T09:27:28.183 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/readlink/rl-1 2024-03-09T09:27:28.183 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/tests/readlink/can-e 2024-03-09T09:27:28.183 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_trailing_blank 2024-03-09T09:27:28.183 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/ 2024-03-09T09:27:28.191 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/coreutils.texi 2024-03-09T09:27:28.209 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/perm.texi 2024-03-09T09:27:28.209 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/version.texi 2024-03-09T09:27:28.210 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/Makefile.am 2024-03-09T09:27:28.210 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/coreutils.info 2024-03-09T09:27:28.222 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/stamp-vti 2024-03-09T09:27:28.222 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/fdl.texi 2024-03-09T09:27:28.223 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/Makefile.in 2024-03-09T09:27:28.223 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/getdate.texi 2024-03-09T09:27:28.224 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/constants.texi 2024-03-09T09:27:28.224 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/doc/ChangeLog-2007 2024-03-09T09:27:28.226 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/ 2024-03-09T09:27:28.509 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/split.c 2024-03-09T09:27:28.517 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/dircolors.hin 2024-03-09T09:27:28.517 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/test.c 2024-03-09T09:27:28.518 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/nproc.c 2024-03-09T09:27:28.518 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/cp-hash.c 2024-03-09T09:27:28.518 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/rm.c 2024-03-09T09:27:28.518 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/uname-arch.c 2024-03-09T09:27:28.518 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/chroot.c 2024-03-09T09:27:28.518 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/remove.h 2024-03-09T09:27:28.518 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/users.c 2024-03-09T09:27:28.519 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/dircolors.c 2024-03-09T09:27:28.519 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/join.c 2024-03-09T09:27:28.520 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/chown.c 2024-03-09T09:27:28.520 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/shuf.c 2024-03-09T09:27:28.520 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/sum.c 2024-03-09T09:27:28.521 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/csplit.c 2024-03-09T09:27:28.521 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/libstdbuf.c 2024-03-09T09:27:28.523 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/fold.c 2024-03-09T09:27:28.523 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/basename.c 2024-03-09T09:27:28.523 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/prog-fprintf.h 2024-03-09T09:27:28.524 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/sync.c 2024-03-09T09:27:28.524 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/lbracket.c 2024-03-09T09:27:28.525 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/ln.c 2024-03-09T09:27:28.525 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/tsort.c 2024-03-09T09:27:28.526 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/wc.c 2024-03-09T09:27:28.526 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/extract-magic 2024-03-09T09:27:28.526 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/hostid.c 2024-03-09T09:27:28.527 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/dd.c 2024-03-09T09:27:28.528 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/shred.c 2024-03-09T09:27:28.529 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/uname.h 2024-03-09T09:27:28.530 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/unlink.c 2024-03-09T09:27:28.530 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/Makefile.am 2024-03-09T09:27:28.531 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/seq.c 2024-03-09T09:27:28.531 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/uname-uname.c 2024-03-09T09:27:28.532 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/ptx.c 2024-03-09T09:27:28.533 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/wheel-size.h 2024-03-09T09:27:28.533 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/stty.c 2024-03-09T09:27:28.534 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/chcon.c 2024-03-09T09:27:28.534 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/dcgen 2024-03-09T09:27:28.535 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/uptime.c 2024-03-09T09:27:28.535 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/link.c 2024-03-09T09:27:28.535 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/install.c 2024-03-09T09:27:28.535 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/uname.c 2024-03-09T09:27:28.535 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/mknod.c 2024-03-09T09:27:28.535 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/su.c 2024-03-09T09:27:28.536 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/expand.c 2024-03-09T09:27:28.537 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/mv.c 2024-03-09T09:27:28.538 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/ls.c 2024-03-09T09:27:28.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/whoami.c 2024-03-09T09:27:28.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/ls-ls.c 2024-03-09T09:27:28.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/ls.h 2024-03-09T09:27:28.540 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/id.c 2024-03-09T09:27:28.541 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/Makefile.in 2024-03-09T09:27:28.542 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/du.c 2024-03-09T09:27:28.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/cut.c 2024-03-09T09:27:28.543 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/comm.c 2024-03-09T09:27:28.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/od.c 2024-03-09T09:27:28.544 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/expr.c 2024-03-09T09:27:28.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/nohup.c 2024-03-09T09:27:28.545 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/system.h 2024-03-09T09:27:28.546 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/cat.c 2024-03-09T09:27:28.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/mktemp.c 2024-03-09T09:27:28.547 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/tac.c 2024-03-09T09:27:28.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/ls-dir.c 2024-03-09T09:27:28.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/tac-pipe.c 2024-03-09T09:27:28.548 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/touch.c 2024-03-09T09:27:28.549 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/false.c 2024-03-09T09:27:28.549 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/echo.c 2024-03-09T09:27:28.549 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/printenv.c 2024-03-09T09:27:28.549 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/logname.c 2024-03-09T09:27:28.550 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/group-list.c 2024-03-09T09:27:28.550 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/pr.c 2024-03-09T09:27:28.551 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/chgrp.c 2024-03-09T09:27:28.551 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/cp-hash.h 2024-03-09T09:27:28.552 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/getlimits.c 2024-03-09T09:27:28.552 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/rmdir.c 2024-03-09T09:27:28.553 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/tail.c 2024-03-09T09:27:28.554 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/timeout.c 2024-03-09T09:27:28.554 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/sort.c 2024-03-09T09:27:28.555 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/operand2sig.c 2024-03-09T09:27:28.555 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/factor.c 2024-03-09T09:27:28.556 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/paste.c 2024-03-09T09:27:28.556 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/chmod.c 2024-03-09T09:27:28.557 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/pinky.c 2024-03-09T09:27:28.557 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/unexpand.c 2024-03-09T09:27:28.558 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/cksum.c 2024-03-09T09:27:28.558 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/date.c 2024-03-09T09:27:28.558 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/mkdir.c 2024-03-09T09:27:28.559 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/printf.c 2024-03-09T09:27:28.559 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/true.c 2024-03-09T09:27:28.559 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/c99-to-c89.diff 2024-03-09T09:27:28.560 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/groups.c 2024-03-09T09:27:28.560 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/dirname.c 2024-03-09T09:27:28.561 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/chown-core.h 2024-03-09T09:27:28.561 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/fs.h 2024-03-09T09:27:28.561 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/nl.c 2024-03-09T09:27:28.561 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/tee.c 2024-03-09T09:27:28.562 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/dircolors.h 2024-03-09T09:27:28.562 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/head.c 2024-03-09T09:27:28.563 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/uniq.c 2024-03-09T09:27:28.564 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/chown-core.c 2024-03-09T09:27:28.564 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/runcon.c 2024-03-09T09:27:28.564 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/sleep.c 2024-03-09T09:27:28.564 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/md5sum.c 2024-03-09T09:27:28.565 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/yes.c 2024-03-09T09:27:28.565 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/readlink.c 2024-03-09T09:27:28.565 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/env.c 2024-03-09T09:27:28.566 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/df.c 2024-03-09T09:27:28.566 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/nice.c 2024-03-09T09:27:28.566 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/fmt.c 2024-03-09T09:27:28.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/base64.c 2024-03-09T09:27:28.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/tty.c 2024-03-09T09:27:28.567 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/truncate.c 2024-03-09T09:27:28.568 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/mkfifo.c 2024-03-09T09:27:28.568 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/operand2sig.h 2024-03-09T09:27:28.568 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/copy.h 2024-03-09T09:27:28.569 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/kill.c 2024-03-09T09:27:28.569 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/who.c 2024-03-09T09:27:28.569 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/stat.c 2024-03-09T09:27:28.570 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/remove.c 2024-03-09T09:27:28.570 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/pwd.c 2024-03-09T09:27:28.571 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/prog-fprintf.c 2024-03-09T09:27:28.571 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/ls-vdir.c 2024-03-09T09:27:28.571 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/copy.c 2024-03-09T09:27:28.572 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/hostname.c 2024-03-09T09:27:28.572 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/wheel-gen.pl 2024-03-09T09:27:28.572 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/group-list.h 2024-03-09T09:27:28.572 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/cp.c 2024-03-09T09:27:28.573 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/setuidgid.c 2024-03-09T09:27:28.574 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/tr.c 2024-03-09T09:27:28.575 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/stdbuf.c 2024-03-09T09:27:28.575 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/pathchk.c 2024-03-09T09:27:28.575 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/src/wheel.h 2024-03-09T09:27:28.576 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/configure 2024-03-09T09:27:28.672 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/README 2024-03-09T09:27:28.672 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.vg-suppressions 2024-03-09T09:27:28.672 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_prohibit_atoi_atof 2024-03-09T09:27:28.672 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_prohibit_stat_st_blocks 2024-03-09T09:27:28.672 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/ChangeLog-2006 2024-03-09T09:27:28.674 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_program_name 2024-03-09T09:27:28.675 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/THANKStt.in 2024-03-09T09:27:28.675 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/ChangeLog-2008 2024-03-09T09:27:28.675 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/ChangeLog-2007 2024-03-09T09:27:28.678 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/ 2024-03-09T09:27:28.688 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-isnand.h 2024-03-09T09:27:28.693 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-ftello.sh 2024-03-09T09:27:28.693 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-snprintf.c 2024-03-09T09:27:28.693 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-openat.c 2024-03-09T09:27:28.694 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-dirent-safer.c 2024-03-09T09:27:28.694 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-cloexec.c 2024-03-09T09:27:28.694 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fstatat.c 2024-03-09T09:27:28.694 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fnmatch.c 2024-03-09T09:27:28.694 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-getndelim2.c 2024-03-09T09:27:28.695 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-wcrtomb.sh 2024-03-09T09:27:28.695 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-filenamecat.c 2024-03-09T09:27:28.695 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fchownat.c 2024-03-09T09:27:28.695 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-stdlib.c 2024-03-09T09:27:28.696 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-read-file.c 2024-03-09T09:27:28.697 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-exclude7.sh 2024-03-09T09:27:28.698 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-closein.c 2024-03-09T09:27:28.699 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/connect.c 2024-03-09T09:27:28.700 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-rename.h 2024-03-09T09:27:28.701 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-memchr2.c 2024-03-09T09:27:28.701 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-sys_time.c 2024-03-09T09:27:28.702 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/read-file.h 2024-03-09T09:27:28.702 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mkdirat.c 2024-03-09T09:27:28.702 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-isnand-nolibm.c 2024-03-09T09:27:28.702 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fopen-safer.c 2024-03-09T09:27:28.703 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-stdio.c 2024-03-09T09:27:28.703 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fseeko.c 2024-03-09T09:27:28.704 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-arpa_inet.c 2024-03-09T09:27:28.704 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-freadahead.sh 2024-03-09T09:27:28.705 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-openat-safer.c 2024-03-09T09:27:28.705 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-unlink.c 2024-03-09T09:27:28.705 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsstr2.sh 2024-03-09T09:27:28.705 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-lchown.c 2024-03-09T09:27:28.706 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-nl_langinfo.c 2024-03-09T09:27:28.706 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-open.h 2024-03-09T09:27:28.706 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-memcmp.c 2024-03-09T09:27:28.707 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-vfprintf-posix.c 2024-03-09T09:27:28.707 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-freadahead.c 2024-03-09T09:27:28.707 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/ioctl.c 2024-03-09T09:27:28.707 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-iconv-h.c 2024-03-09T09:27:28.707 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mkdir.c 2024-03-09T09:27:28.708 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-posixtm.c 2024-03-09T09:27:28.708 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/wctob.c 2024-03-09T09:27:28.708 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/glthread/ 2024-03-09T09:27:28.725 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/glthread/thread.c 2024-03-09T09:27:28.734 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/glthread/yield.h 2024-03-09T09:27:28.734 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/glthread/thread.h 2024-03-09T09:27:28.735 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/unistr/ 2024-03-09T09:27:28.737 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/unistr/test-u8-uctomb.c 2024-03-09T09:27:28.739 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-exclude5.sh 2024-03-09T09:27:28.739 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-rmdir.h 2024-03-09T09:27:28.739 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-symlink.h 2024-03-09T09:27:28.739 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-unsetenv.c 2024-03-09T09:27:28.740 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-exclude4.sh 2024-03-09T09:27:28.740 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/listen.c 2024-03-09T09:27:28.740 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-printf-posix.output 2024-03-09T09:27:28.741 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbrtowc3.sh 2024-03-09T09:27:28.741 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-dirent.c 2024-03-09T09:27:28.741 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-utimens-common.h 2024-03-09T09:27:28.742 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsinit.c 2024-03-09T09:27:28.743 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/w32sock.h 2024-03-09T09:27:28.743 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-printf-posix.h 2024-03-09T09:27:28.744 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fchdir.c 2024-03-09T09:27:28.744 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/read-file.c 2024-03-09T09:27:28.745 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-ctype.c 2024-03-09T09:27:28.745 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/binary-io.h 2024-03-09T09:27:28.746 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/nap.h 2024-03-09T09:27:28.746 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/sys_ioctl.in.h 2024-03-09T09:27:28.748 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-ftello2.sh 2024-03-09T09:27:28.748 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-memchr.c 2024-03-09T09:27:28.749 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsrtowcs3.sh 2024-03-09T09:27:28.749 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fseeko.sh 2024-03-09T09:27:28.750 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-select-out.sh 2024-03-09T09:27:28.750 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/inet_pton.c 2024-03-09T09:27:28.750 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-stdbool.c 2024-03-09T09:27:28.750 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-nanosleep.c 2024-03-09T09:27:28.750 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/bind.c 2024-03-09T09:27:28.751 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/unlinkdir.h 2024-03-09T09:27:28.751 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-getline.c 2024-03-09T09:27:28.752 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-md5.c 2024-03-09T09:27:28.754 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/Makefile.am 2024-03-09T09:27:28.754 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-select-in.sh 2024-03-09T09:27:28.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-uname.c 2024-03-09T09:27:28.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-readlink.h 2024-03-09T09:27:28.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-string.c 2024-03-09T09:27:28.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-argmatch.c 2024-03-09T09:27:28.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-nl_langinfo.sh 2024-03-09T09:27:28.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-sys_stat.c 2024-03-09T09:27:28.756 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-version-etc.c 2024-03-09T09:27:28.757 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-areadlink.h 2024-03-09T09:27:28.759 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-version-etc.sh 2024-03-09T09:27:28.759 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-binary-io.c 2024-03-09T09:27:28.760 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-exclude2.sh 2024-03-09T09:27:28.760 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xfprintf-posix.c 2024-03-09T09:27:28.760 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-set-mode-acl.c 2024-03-09T09:27:28.761 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-wchar.c 2024-03-09T09:27:28.761 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-vasprintf.c 2024-03-09T09:27:28.761 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xstrtoimax.sh 2024-03-09T09:27:28.763 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-freadseek.sh 2024-03-09T09:27:28.763 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-isblank.c 2024-03-09T09:27:28.763 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fpending.sh 2024-03-09T09:27:28.764 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-lstat.h 2024-03-09T09:27:28.764 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-chown.c 2024-03-09T09:27:28.764 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xstrtoumax.sh 2024-03-09T09:27:28.764 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/nan.h 2024-03-09T09:27:28.765 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xprintf-posix.c 2024-03-09T09:27:28.765 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-inet_pton.c 2024-03-09T09:27:28.765 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-dup-safer.c 2024-03-09T09:27:28.767 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-closein.sh 2024-03-09T09:27:28.768 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-canonicalize.c 2024-03-09T09:27:28.768 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-isnanl-nolibm.c 2024-03-09T09:27:28.768 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xstrtol.c 2024-03-09T09:27:28.768 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-open.c 2024-03-09T09:27:28.768 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fprintf-posix.h 2024-03-09T09:27:28.769 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbrtowc1.sh 2024-03-09T09:27:28.769 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-errno.c 2024-03-09T09:27:28.769 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-netdb.c 2024-03-09T09:27:28.769 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-wctype.c 2024-03-09T09:27:28.769 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-strsignal.c 2024-03-09T09:27:28.770 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsstr3.sh 2024-03-09T09:27:28.770 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-yesno.c 2024-03-09T09:27:28.770 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-userspec.c 2024-03-09T09:27:28.770 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-freopen.c 2024-03-09T09:27:28.770 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/uniwidth/ 2024-03-09T09:27:28.787 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.c 2024-03-09T09:27:28.830 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/uniwidth/test-uc_width.c 2024-03-09T09:27:28.830 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.sh 2024-03-09T09:27:28.830 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-sockets.c 2024-03-09T09:27:28.830 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-link.h 2024-03-09T09:27:28.831 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-select-fd.c 2024-03-09T09:27:28.832 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xstrtoimax.c 2024-03-09T09:27:28.865 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xvasprintf.c 2024-03-09T09:27:28.865 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-yesno.sh 2024-03-09T09:27:28.865 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fflush.c 2024-03-09T09:27:28.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-vc-list-files-cvs.sh 2024-03-09T09:27:28.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbscasecmp.sh 2024-03-09T09:27:28.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-lseek.sh 2024-03-09T09:27:28.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/usleep.c 2024-03-09T09:27:28.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-sleep.c 2024-03-09T09:27:28.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-link.c 2024-03-09T09:27:28.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-perror.c 2024-03-09T09:27:28.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsrtowcs.c 2024-03-09T09:27:28.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xalloc-die.sh 2024-03-09T09:27:28.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-i-ring.c 2024-03-09T09:27:28.866 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-priv-set.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-btowc.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-sys_select.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbrtowc.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-strerror.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-update-copyright.sh 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-btowc2.sh 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-getcwd.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/socket.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-set-mode-acl.sh 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-linkat.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/perror.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-wcrtomb.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-vasnprintf.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-memrchr.c 2024-03-09T09:27:28.867 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-ftello.c 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-chown.h 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-stat.h 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/Makefile.in 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-getdelim.c 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-vprintf-posix.sh 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fseeko2.sh 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-c-strcase.sh 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsalign.c 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fcntl-h.c 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-vprintf-posix.c 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-rename.c 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-lstat.c 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-malloca.c 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xstrtoul.c 2024-03-09T09:27:28.868 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-perror.sh 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-vasprintf-posix.c 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-frexp.c 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-environ.c 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-stdint.c 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-isnanl.h 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fcntl-safer.c 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-locale.c 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbrtowc4.sh 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-argv-iter.c 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-utimens.h 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xalloc-die.c 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fcntl.c 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-alloca-opt.c 2024-03-09T09:27:28.869 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-dirname.c 2024-03-09T09:27:28.882 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-setenv.c 2024-03-09T09:27:29.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.153+0000 7ff8cea00640 1 -- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 msgr2=0x7ff8d01562d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:27:29.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.153+0000 7ff8cea00640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 0x7ff8d01562d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.153+0000 7ff8cea00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 msgr2=0x7ff8d00c3510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:27:29.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.153+0000 7ff8cea00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.67:3300/0 conn(0x7ff8d00c2e80 0x7ff8d00c3510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.153+0000 7ff8cea00640 1 -- 172.21.15.67:0/3038045459 >> v1:172.21.15.134:6789/0 conn(0x7ff8d01532c0 legacy=0x7ff8d006fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.153+0000 7ff8cea00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.134:3300/0 conn(0x7ff8b0007370 0x7ff8b0009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.153+0000 7ff8cea00640 1 -- 172.21.15.67:0/3038045459 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d0072cc0 con 0x7ff8b00036e0 2024-03-09T09:27:29.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.153+0000 7ff8cea00640 1 -- 172.21.15.67:0/3038045459 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d0073310 con 0x7ff8b0007370 2024-03-09T09:27:29.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.154+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.134:3300/0 conn(0x7ff8b0007370 0x7ff8b0009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:29.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.155+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.134:3300/0 conn(0x7ff8b0007370 msgr2=0x7ff8b0009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:27:29.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.155+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.134:3300/0 conn(0x7ff8b0007370 msgr2=0x7ff8b0009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:27:29.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.155+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.134:3300/0 conn(0x7ff8b0007370 0x7ff8b0009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:29.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.155+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.134:3300/0 conn(0x7ff8b0007370 0x7ff8b0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:29.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.219+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702019021 0 0) 0x7ff8d0072cc0 con 0x7ff8b00036e0 2024-03-09T09:27:29.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.219+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b8003520 con 0x7ff8b00036e0 2024-03-09T09:27:29.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.220+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 162534038 0 0) 0x7ff8b8003520 con 0x7ff8b00036e0 2024-03-09T09:27:29.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.220+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 >> v2:172.21.15.134:3300/0 conn(0x7ff8b0007370 msgr2=0x7ff8b0009830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:27:29.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.220+0000 7ff8cfe00640 1 --2- 172.21.15.67:0/3038045459 >> v2:172.21.15.134:3300/0 conn(0x7ff8b0007370 0x7ff8b0009830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.220+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b000af40 con 0x7ff8b00036e0 2024-03-09T09:27:29.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.220+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7ff8c40016b0 con 0x7ff8b00036e0 2024-03-09T09:27:29.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.220+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 >> v1:172.21.15.67:6789/0 conn(0x7ff8b00036e0 legacy=0x7ff8b0005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.220+0000 7ff8cfe00640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 0x7ff8b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.221+0000 7ff8cfe00640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 0x7ff8b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.221+0000 7ff8cfe00640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8b8004d40 0x7ff8b80051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.221+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0073310 con 0x7ff8b8004d40 2024-03-09T09:27:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.221+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b800bbd0 con 0x7ff8b8003a20 2024-03-09T09:27:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.221+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b800bdd0 con 0x7ff8d0155e40 2024-03-09T09:27:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.221+0000 7ff8d6000640 1 -- 172.21.15.67:0/3038045459 reap_dead start 2024-03-09T09:27:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.221+0000 7ff8d4c00640 1 -- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 msgr2=0x7ff8b8004800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.221+0000 7ff8d4c00640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 0x7ff8b8004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:29.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.222+0000 7ff8d5600640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 0x7ff8b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:29.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.225+0000 7ff8d5600640 1 -- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 msgr2=0x7ff8b8004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:27:29.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.225+0000 7ff8d5600640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 0x7ff8b8004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.225+0000 7ff8d5600640 1 -- 172.21.15.67:0/3038045459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8b8004d40 msgr2=0x7ff8b80051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:29.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.225+0000 7ff8d5600640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8b8004d40 0x7ff8b80051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.225+0000 7ff8d5600640 1 -- 172.21.15.67:0/3038045459 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8003520 con 0x7ff8b8003a20 2024-03-09T09:27:29.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.228+0000 7ff8d5600640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 0x7ff8b8003e20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4007eb0 tx=0x7ff8b8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1da55d8eb0ad9866 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:29.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.228+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4039070 con 0x7ff8b8003a20 2024-03-09T09:27:29.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.228+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c402c900 con 0x7ff8b8003a20 2024-03-09T09:27:29.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.228+0000 7ff8cfe00640 1 -- 172.21.15.67:0/3038045459 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4007ad0 con 0x7ff8b8003a20 2024-03-09T09:27:29.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.229+0000 7ff8d6a00640 1 -- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 msgr2=0x7ff8b8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.229+0000 7ff8d6a00640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 0x7ff8b8003e20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4007eb0 tx=0x7ff8b8005830 comp rx=0 tx=0).stop 2024-03-09T09:27:29.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.229+0000 7ff8d6a00640 1 -- 172.21.15.67:0/3038045459 shutdown_connections 2024-03-09T09:27:29.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.230+0000 7ff8d6a00640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8b8004d40 0x7ff8b80051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.230+0000 7ff8d6a00640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8d0155e40 0x7ff8b8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.230+0000 7ff8d6a00640 1 --2- 172.21.15.67:0/3038045459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 0x7ff8b8003e20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.230+0000 7ff8d6a00640 1 -- 172.21.15.67:0/3038045459 >> 172.21.15.67:0/3038045459 conn(0x7ff8d0066d90 msgr2=0x7ff8d0067170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:29.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.230+0000 7ff8d6a00640 1 -- 172.21.15.67:0/3038045459 shutdown_connections 2024-03-09T09:27:29.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.230+0000 7ff8d6a00640 1 -- 172.21.15.67:0/3038045459 wait complete. 2024-03-09T09:27:29.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.233+0000 7ff8d6a00640 1 Processor -- start 2024-03-09T09:27:29.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.233+0000 7ff8d6a00640 1 -- start start 2024-03-09T09:27:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d6a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 0x7ff8d01f0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d6a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b8004d40 0x7ff8d014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8d0155e40 0x7ff8d01593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d6a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00c3690 con 0x7ff8d0155e40 2024-03-09T09:27:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d6a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00c3510 con 0x7ff8b8003a20 2024-03-09T09:27:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d6a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00c3810 con 0x7ff8b8004d40 2024-03-09T09:27:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d4c00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b8004d40 msgr2=0x7ff8d014e2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8d0155e40 0x7ff8d01593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d4c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b8004d40 0x7ff8d014e2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d5600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 0x7ff8d01f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8d0155e40 0x7ff8d01593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35822/0 (socket says 172.21.15.67:35822) 2024-03-09T09:27:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.235+0000 7ff8d6000640 1 -- 172.21.15.67:0/1276990316 learned_addr learned my addr 172.21.15.67:0/1276990316 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.236+0000 7ff8d6000640 1 -- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b8004d40 msgr2=0x7ff8d014e2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.236+0000 7ff8d6000640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b8004d40 0x7ff8d014e2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.236+0000 7ff8d6000640 1 -- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 msgr2=0x7ff8d01f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.236+0000 7ff8d6000640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 0x7ff8d01f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.236+0000 7ff8d6000640 1 -- 172.21.15.67:0/1276990316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d014b6e0 con 0x7ff8d0155e40 2024-03-09T09:27:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.236+0000 7ff8d5600640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 0x7ff8d01f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:27:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.237+0000 7ff8d6000640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8d0155e40 0x7ff8d01593d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00072e0 tx=0x7ff8c800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.237+0000 7ff8cd600640 1 -- 172.21.15.67:0/1276990316 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800a8d0 con 0x7ff8d0155e40 2024-03-09T09:27:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.237+0000 7ff8cd600640 1 -- 172.21.15.67:0/1276990316 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c800aa70 con 0x7ff8d0155e40 2024-03-09T09:27:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.237+0000 7ff8cd600640 1 -- 172.21.15.67:0/1276990316 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800ae60 con 0x7ff8d0155e40 2024-03-09T09:27:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.237+0000 7ff8d6a00640 1 -- 172.21.15.67:0/1276990316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d01f38c0 con 0x7ff8d0155e40 2024-03-09T09:27:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.237+0000 7ff8d6a00640 1 -- 172.21.15.67:0/1276990316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d014b9c0 con 0x7ff8d0155e40 2024-03-09T09:27:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.239+0000 7ff8d6a00640 1 -- 172.21.15.67:0/1276990316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d004f3c0 con 0x7ff8d0155e40 2024-03-09T09:27:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.241+0000 7ff8cd600640 1 -- 172.21.15.67:0/1276990316 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff8c801c470 con 0x7ff8d0155e40 2024-03-09T09:27:29.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.242+0000 7ff8cd600640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8b006c170 0x7ff8b006e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.242+0000 7ff8cd600640 1 -- 172.21.15.67:0/1276990316 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6749+0+0 (secure 0 0 0) 0x7ff8c808b4a0 con 0x7ff8d0155e40 2024-03-09T09:27:29.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.243+0000 7ff8d5600640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8b006c170 0x7ff8b006e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:29.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.244+0000 7ff8d5600640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8b006c170 0x7ff8b006e630 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4036000 tx=0x7ff8c4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:29.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.246+0000 7ff8cd600640 1 -- 172.21.15.67:0/1276990316 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff8c8059f80 con 0x7ff8d0155e40 2024-03-09T09:27:29.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.557+0000 7ff8d6a00640 1 -- 172.21.15.67:0/1276990316 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff8d006fcd0 con 0x7ff8d0155e40 2024-03-09T09:27:29.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.557+0000 7ff8cd600640 1 -- 172.21.15.67:0/1276990316 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7ff8c8059880 con 0x7ff8d0155e40 2024-03-09T09:27:29.564 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":280,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:27:29.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 -- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8b006c170 msgr2=0x7ff8b006e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8b006c170 0x7ff8b006e630 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4036000 tx=0x7ff8c4009030 comp rx=0 tx=0).stop 2024-03-09T09:27:29.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 -- 172.21.15.67:0/1276990316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8d0155e40 msgr2=0x7ff8d01593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8d0155e40 0x7ff8d01593d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00072e0 tx=0x7ff8c800cbe0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 -- 172.21.15.67:0/1276990316 shutdown_connections 2024-03-09T09:27:29.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff8b006c170 0x7ff8b006e630 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff8d0155e40 0x7ff8d01593d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff8b8004d40 0x7ff8d014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 --2- 172.21.15.67:0/1276990316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b8003a20 0x7ff8d01f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 -- 172.21.15.67:0/1276990316 >> 172.21.15.67:0/1276990316 conn(0x7ff8d0066d90 msgr2=0x7ff8d0152540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:29.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 -- 172.21.15.67:0/1276990316 shutdown_connections 2024-03-09T09:27:29.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.560+0000 7ff8d6a00640 1 -- 172.21.15.67:0/1276990316 wait complete. 2024-03-09T09:27:29.593 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:27:29.593 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:27:29.594 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:27:29.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.809+0000 7fcd6ca00640 1 Processor -- start 2024-03-09T09:27:29.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.809+0000 7fcd6ca00640 1 -- start start 2024-03-09T09:27:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.811+0000 7fcd6ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd6814fd90 0x7fcd68156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.811+0000 7fcd6ca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd68156bc0 con 0x7fcd6806c1d0 2024-03-09T09:27:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.811+0000 7fcd6ca00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd680c9890 con 0x7fcd680c9380 2024-03-09T09:27:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.811+0000 7fcd6ca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6806cd00 con 0x7fcd6814fd90 2024-03-09T09:27:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.811+0000 7fcd66800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fcd6814fd90 0x7fcd68156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.811+0000 7fcd67200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcd6806c1d0 0x7fcd6806c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36156/0 (socket says 172.21.15.67:36156) 2024-03-09T09:27:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.811+0000 7fcd67200640 1 -- 172.21.15.67:0/4156103234 learned_addr learned my addr 172.21.15.67:0/4156103234 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:27:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.812+0000 7fcd66800640 1 -- 172.21.15.67:0/4156103234 >> v1:172.21.15.134:6789/0 conn(0x7fcd680c9380 legacy=0x7fcd680acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:29.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.812+0000 7fcd66800640 1 -- 172.21.15.67:0/4156103234 >> v1:172.21.15.67:6789/0 conn(0x7fcd6806c1d0 legacy=0x7fcd6806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.812+0000 7fcd66800640 1 -- 172.21.15.67:0/4156103234 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680c5f80 con 0x7fcd6814fd90 2024-03-09T09:27:29.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.812+0000 7fcd65400640 1 -- 172.21.15.67:0/4156103234 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007876333 0 0) 0x7fcd60001490 con 0x7fcd6806c1d0 2024-03-09T09:27:29.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.813+0000 7fcd66800640 1 --2- 172.21.15.67:0/4156103234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd6814fd90 0x7fcd68156680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd580097a0 tx=0x7fcd5802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adefe7e00b03665b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.813+0000 7fcd65400640 1 -- 172.21.15.67:0/4156103234 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58038070 con 0x7fcd6814fd90 2024-03-09T09:27:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.813+0000 7fcd65400640 1 -- 172.21.15.67:0/4156103234 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58031700 con 0x7fcd6814fd90 2024-03-09T09:27:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.813+0000 7fcd65400640 1 -- 172.21.15.67:0/4156103234 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58031ad0 con 0x7fcd6814fd90 2024-03-09T09:27:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.814+0000 7fcd6ca00640 1 -- 172.21.15.67:0/4156103234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd6814fd90 msgr2=0x7fcd68156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.814+0000 7fcd6ca00640 1 --2- 172.21.15.67:0/4156103234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd6814fd90 0x7fcd68156680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd580097a0 tx=0x7fcd5802b7a0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.814+0000 7fcd6ca00640 1 -- 172.21.15.67:0/4156103234 shutdown_connections 2024-03-09T09:27:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.814+0000 7fcd6ca00640 1 --2- 172.21.15.67:0/4156103234 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd6814fd90 0x7fcd68156680 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.814+0000 7fcd6ca00640 1 -- 172.21.15.67:0/4156103234 >> 172.21.15.67:0/4156103234 conn(0x7fcd680cb480 msgr2=0x7fcd680cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.814+0000 7fcd6ca00640 1 -- 172.21.15.67:0/4156103234 shutdown_connections 2024-03-09T09:27:29.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.814+0000 7fcd6ca00640 1 -- 172.21.15.67:0/4156103234 wait complete. 2024-03-09T09:27:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.816+0000 7fcd6ca00640 1 Processor -- start 2024-03-09T09:27:29.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.816+0000 7fcd6ca00640 1 -- start start 2024-03-09T09:27:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd6ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd6806c1d0 0x7fcd681ec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd6ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd680c9380 0x7fcd680ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd6ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd6814fd90 0x7fcd68145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd6ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680c6520 con 0x7fcd6806c1d0 2024-03-09T09:27:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd6ca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6806ba60 con 0x7fcd680c9380 2024-03-09T09:27:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd6ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6806bcb0 con 0x7fcd6814fd90 2024-03-09T09:27:29.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd66800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd6806c1d0 0x7fcd681ec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd67200640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd6814fd90 msgr2=0x7fcd68145450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd67200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd6814fd90 0x7fcd68145450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd66800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd6806c1d0 0x7fcd681ec8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35848/0 (socket says 172.21.15.67:35848) 2024-03-09T09:27:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd66800640 1 -- 172.21.15.67:0/3535480397 learned_addr learned my addr 172.21.15.67:0/3535480397 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.817+0000 7fcd65e00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd680c9380 0x7fcd680ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.818+0000 7fcd66800640 1 -- 172.21.15.67:0/3535480397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd6814fd90 msgr2=0x7fcd68145450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.818+0000 7fcd66800640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd6814fd90 0x7fcd68145450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.818+0000 7fcd66800640 1 -- 172.21.15.67:0/3535480397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd680c9380 msgr2=0x7fcd680ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.818+0000 7fcd66800640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd680c9380 0x7fcd680ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.818+0000 7fcd66800640 1 -- 172.21.15.67:0/3535480397 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680af410 con 0x7fcd6806c1d0 2024-03-09T09:27:29.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.818+0000 7fcd66800640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd6806c1d0 0x7fcd681ec8b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd580097a0 tx=0x7fcd580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.818+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58043070 con 0x7fcd6806c1d0 2024-03-09T09:27:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.818+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58033a30 con 0x7fcd6806c1d0 2024-03-09T09:27:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.818+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58038040 con 0x7fcd6806c1d0 2024-03-09T09:27:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.818+0000 7fcd6ca00640 1 -- 172.21.15.67:0/3535480397 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd681ef3a0 con 0x7fcd6806c1d0 2024-03-09T09:27:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.819+0000 7fcd6ca00640 1 -- 172.21.15.67:0/3535480397 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd681ef960 con 0x7fcd6806c1d0 2024-03-09T09:27:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.819+0000 7fcd6ca00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680ace10 0x7fcd68145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.820+0000 7fcd6ca00640 1 -- 172.21.15.67:0/3535480397 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680c4450 con 0x7fcd680ace10 2024-03-09T09:27:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.820+0000 7fcd65e00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680ace10 0x7fcd68145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.820+0000 7fcd65e00640 1 -- 172.21.15.67:0/3535480397 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd68068be0 con 0x7fcd680ace10 2024-03-09T09:27:29.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.821+0000 7fcd65e00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680ace10 0x7fcd68145990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400d4d0 tx=0x7fcd5400d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:29.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.821+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd54018020 con 0x7fcd680ace10 2024-03-09T09:27:29.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.821+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcd58046070 con 0x7fcd6806c1d0 2024-03-09T09:27:29.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.822+0000 7fcd5ea00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd4806b3d0 0x7fcd4806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.822+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fcd580ac510 con 0x7fcd6806c1d0 2024-03-09T09:27:29.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.822+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fcd54014620 con 0x7fcd680ace10 2024-03-09T09:27:29.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.822+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680ace10 msgr2=0x7fcd68145990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.822+0000 7fcd5ea00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680ace10 0x7fcd68145990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400d4d0 tx=0x7fcd5400d990 comp rx=0 tx=0).stop 2024-03-09T09:27:29.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.822+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680ace10 msgr2=0x7fcd68145990 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:29.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.822+0000 7fcd5ea00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680ace10 0x7fcd68145990 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400d4d0 tx=0x7fcd5400d990 comp rx=0 tx=0).stop 2024-03-09T09:27:29.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.822+0000 7fcd67200640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd4806b3d0 0x7fcd4806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:29.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.823+0000 7fcd67200640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd4806b3d0 0x7fcd4806d890 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcd60004b90 tx=0x7fcd600015b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:29.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.830+0000 7fcd6ca00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680aff80 0x7fcd680b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:29.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.830+0000 7fcd6ca00640 1 -- 172.21.15.67:0/3535480397 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6804f3c0 con 0x7fcd680aff80 2024-03-09T09:27:29.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.830+0000 7fcd65e00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680aff80 0x7fcd680b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:29.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.830+0000 7fcd65e00640 1 -- 172.21.15.67:0/3535480397 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcd68146060 con 0x7fcd680aff80 2024-03-09T09:27:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.831+0000 7fcd65e00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680aff80 0x7fcd680b0390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400d4d0 tx=0x7fcd54004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:29.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.831+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd54025070 con 0x7fcd680aff80 2024-03-09T09:27:29.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.831+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7fcd54004020 con 0x7fcd680aff80 2024-03-09T09:27:29.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.831+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680aff80 msgr2=0x7fcd680b0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.831+0000 7fcd5ea00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680aff80 0x7fcd680b0390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400d4d0 tx=0x7fcd54004680 comp rx=0 tx=0).stop 2024-03-09T09:27:29.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.831+0000 7fcd5ea00640 1 -- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680aff80 msgr2=0x7fcd680b0390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:29.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.831+0000 7fcd5ea00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd680aff80 0x7fcd680b0390 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400d4d0 tx=0x7fcd54004680 comp rx=0 tx=0).stop 2024-03-09T09:27:29.838 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 280, 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:29.839 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:29.840 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:29.840 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:29.840 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:29.840 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:29.840 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:29.840 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:29.840 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:29.840 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:29.840 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:29.840 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:29.841 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:29.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.842 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:27:29.842 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:27:29.842 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:27:29.842 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:27:29.842 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:27:29.842 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:27:29.842 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:29.842 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:29.842 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:27:29.842 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:29.843 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:29.844 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:29.844 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:29.844 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:29.844 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:29.844 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:29.844 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:29.844 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:27:29.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.844 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:27:29.844 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:29.845 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.846 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:27:29.847 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:29.848 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:27:29.849 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:27:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:27:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:27:29.850 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:29.851 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:29.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:27:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:27:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:29.853 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:29.854 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:27:29.854 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:27:29.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.832+0000 7fcd6ca00640 1 -- 172.21.15.67:0/3535480397 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd4806b3d0 msgr2=0x7fcd4806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.832+0000 7fcd6ca00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcd4806b3d0 0x7fcd4806d890 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcd60004b90 tx=0x7fcd600015b0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.832+0000 7fcd6ca00640 1 -- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd6806c1d0 msgr2=0x7fcd681ec8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:29.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.832+0000 7fcd6ca00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd6806c1d0 0x7fcd681ec8b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd580097a0 tx=0x7fcd580046c0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.832+0000 7fcd67200640 1 -- 172.21.15.67:0/3535480397 reap_dead start 2024-03-09T09:27:29.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.832+0000 7fcd6ca00640 1 -- 172.21.15.67:0/3535480397 shutdown_connections 2024-03-09T09:27:29.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.832+0000 7fcd6ca00640 1 --2- 172.21.15.67:0/3535480397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd6806c1d0 0x7fcd681ec8b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:29.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.832+0000 7fcd6ca00640 1 -- 172.21.15.67:0/3535480397 >> 172.21.15.67:0/3535480397 conn(0x7fcd680cb480 msgr2=0x7fcd6814c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:29.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.832+0000 7fcd6ca00640 1 -- 172.21.15.67:0/3535480397 shutdown_connections 2024-03-09T09:27:29.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:29.832+0000 7fcd6ca00640 1 -- 172.21.15.67:0/3535480397 wait complete. 2024-03-09T09:27:29.857 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:27:29.857 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-03-09T09:27:29.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:27:30.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.054+0000 7f59c7000640 1 Processor -- start 2024-03-09T09:27:30.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.054+0000 7f59c7000640 1 -- start start 2024-03-09T09:27:30.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.056+0000 7f59c7000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c014fd90 0x7f59c0156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:30.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.056+0000 7f59c7000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f59c006c1d0 0x7f59c006c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:30.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.056+0000 7f59c7000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c0156bc0 con 0x7f59c00c9380 2024-03-09T09:27:30.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.056+0000 7f59c7000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c006ba60 con 0x7f59c006c1d0 2024-03-09T09:27:30.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.056+0000 7f59c7000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c00c9890 con 0x7f59c014fd90 2024-03-09T09:27:30.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.056+0000 7f59c5c00640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c014fd90 msgr2=0x7f59c0156680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:30.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.056+0000 7f59c5c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c014fd90 0x7f59c0156680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:30.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.056+0000 7f59c6600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f59c006c1d0 0x7f59c006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:30.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.056+0000 7f59c6600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f59c006c1d0 0x7f59c006c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:59720/0 (socket says 172.21.15.67:59720) 2024-03-09T09:27:30.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.056+0000 7f59c6600640 1 -- 172.21.15.67:0/3078089320 learned_addr learned my addr 172.21.15.67:0/3078089320 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:30.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.057+0000 7f59c6600640 1 -- 172.21.15.67:0/3078089320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c014fd90 msgr2=0x7f59c0156680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:27:30.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.057+0000 7f59c6600640 1 --2- 172.21.15.67:0/3078089320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c014fd90 0x7f59c0156680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:30.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.057+0000 7f59c6600640 1 -- 172.21.15.67:0/3078089320 >> v1:172.21.15.134:6789/0 conn(0x7f59c00c9380 legacy=0x7f59c00acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:30.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.057+0000 7f59c6600640 1 -- 172.21.15.67:0/3078089320 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c00c5f80 con 0x7f59c006c1d0 2024-03-09T09:27:30.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.058+0000 7f59c6600640 1 --2- 172.21.15.67:0/3078089320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c006c1d0 0x7f59c006c600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59b8009e50 tx=0x7f59b802be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177e4b9aeb32858e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:30.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.059+0000 7f59bfe00640 1 -- 172.21.15.67:0/3078089320 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b802ca10 con 0x7f59c006c1d0 2024-03-09T09:27:30.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.059+0000 7f59bfe00640 1 -- 172.21.15.67:0/3078089320 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b802cbb0 con 0x7f59c006c1d0 2024-03-09T09:27:30.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.059+0000 7f59bfe00640 1 -- 172.21.15.67:0/3078089320 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b80343f0 con 0x7f59c006c1d0 2024-03-09T09:27:30.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.060+0000 7f59c7000640 1 -- 172.21.15.67:0/3078089320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c006c1d0 msgr2=0x7f59c006c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:30.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.060+0000 7f59c7000640 1 --2- 172.21.15.67:0/3078089320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c006c1d0 0x7f59c006c600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59b8009e50 tx=0x7f59b802be70 comp rx=0 tx=0).stop 2024-03-09T09:27:30.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.060+0000 7f59c7000640 1 -- 172.21.15.67:0/3078089320 shutdown_connections 2024-03-09T09:27:30.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.060+0000 7f59c7000640 1 --2- 172.21.15.67:0/3078089320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c006c1d0 0x7f59c006c600 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:30.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.060+0000 7f59c7000640 1 --2- 172.21.15.67:0/3078089320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c014fd90 0x7f59c0156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:30.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.060+0000 7f59c7000640 1 -- 172.21.15.67:0/3078089320 >> 172.21.15.67:0/3078089320 conn(0x7f59c00cb480 msgr2=0x7f59c00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:30.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.060+0000 7f59c7000640 1 -- 172.21.15.67:0/3078089320 shutdown_connections 2024-03-09T09:27:30.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.060+0000 7f59c7000640 1 -- 172.21.15.67:0/3078089320 wait complete. 2024-03-09T09:27:30.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.061+0000 7f59c7000640 1 Processor -- start 2024-03-09T09:27:30.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.062+0000 7f59c7000640 1 -- start start 2024-03-09T09:27:30.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.062+0000 7f59c7000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c006c1d0 0x7f59c01514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:30.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.062+0000 7f59c7000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c00c9380 0x7f59c01519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:30.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.062+0000 7f59c7000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c014fd90 0x7f59c01ef090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:30.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.062+0000 7f59c7000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c006bcb0 con 0x7f59c006c1d0 2024-03-09T09:27:30.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.062+0000 7f59c7000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c00c6520 con 0x7f59c014fd90 2024-03-09T09:27:30.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.062+0000 7f59c7000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c00ad610 con 0x7f59c00c9380 2024-03-09T09:27:30.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.063+0000 7f59c5200640 1 -- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c00c9380 msgr2=0x7f59c01519e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3301/0 2024-03-09T09:27:30.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.063+0000 7f59c5200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c00c9380 0x7f59c01519e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:30.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.063+0000 7f59c5c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c006c1d0 0x7f59c01514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:30.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.063+0000 7f59c5c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c006c1d0 0x7f59c01514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35864/0 (socket says 172.21.15.67:35864) 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.063+0000 7f59c5c00640 1 -- 172.21.15.67:0/3924821675 learned_addr learned my addr 172.21.15.67:0/3924821675 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.063+0000 7f59c6600640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c014fd90 0x7f59c01ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.063+0000 7f59c5c00640 1 -- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c00c9380 msgr2=0x7f59c01519e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.063+0000 7f59c5c00640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f59c00c9380 0x7f59c01519e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.063+0000 7f59c5c00640 1 -- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c014fd90 msgr2=0x7f59c01ef090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.063+0000 7f59c5c00640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c014fd90 0x7f59c01ef090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.063+0000 7f59c5c00640 1 -- 172.21.15.67:0/3924821675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c0156f40 con 0x7f59c006c1d0 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.064+0000 7f59c5c00640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c006c1d0 0x7f59c01514a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59b400b550 tx=0x7f59b400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.064+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4013020 con 0x7f59c006c1d0 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.064+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b4003de0 con 0x7f59c006c1d0 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.064+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40052b0 con 0x7f59c006c1d0 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.064+0000 7f59c7000640 1 -- 172.21.15.67:0/3924821675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c01efc10 con 0x7f59c006c1d0 2024-03-09T09:27:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.064+0000 7f59c7000640 1 -- 172.21.15.67:0/3924821675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c0150f80 con 0x7f59c006c1d0 2024-03-09T09:27:30.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.065+0000 7f59c7000640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c014a4a0 0x7f59c01ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:30.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.065+0000 7f59c7000640 1 -- 172.21.15.67:0/3924821675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0150de0 con 0x7f59c014a4a0 2024-03-09T09:27:30.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.066+0000 7f59c5200640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c014a4a0 0x7f59c01ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:30.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.067+0000 7f59c5200640 1 -- 172.21.15.67:0/3924821675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59c0151180 con 0x7f59c014a4a0 2024-03-09T09:27:30.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.067+0000 7f59c5200640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c014a4a0 0x7f59c01ef5d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f59c00b0f00 tx=0x7f59ac00c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:30.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.068+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0075a0 con 0x7f59c014a4a0 2024-03-09T09:27:30.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.068+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f59b4016020 con 0x7f59c006c1d0 2024-03-09T09:27:30.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.068+0000 7f59bd600640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59a006b3d0 0x7f59a006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:30.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.069+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f59b408a340 con 0x7f59c006c1d0 2024-03-09T09:27:30.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.069+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f59ac004020 con 0x7f59c014a4a0 2024-03-09T09:27:30.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.069+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c014a4a0 msgr2=0x7f59c01ef5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:30.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.069+0000 7f59bd600640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c014a4a0 0x7f59c01ef5d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f59c00b0f00 tx=0x7f59ac00c7a0 comp rx=0 tx=0).stop 2024-03-09T09:27:30.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.069+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c014a4a0 msgr2=0x7f59c01ef5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:30.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.069+0000 7f59bd600640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c014a4a0 0x7f59c01ef5d0 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f59c00b0f00 tx=0x7f59ac00c7a0 comp rx=0 tx=0).stop 2024-03-09T09:27:30.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.069+0000 7f59c6600640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59a006b3d0 0x7f59a006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:30.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.070+0000 7f59c6600640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59a006b3d0 0x7f59a006d890 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f59b8035000 tx=0x7f59b8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:30.078 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-printf-frexpl.c 2024-03-09T09:27:30.079 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-atexit.sh 2024-03-09T09:27:30.079 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-stddef.c 2024-03-09T09:27:30.079 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-c-ctype.c 2024-03-09T09:27:30.080 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xstrtol.sh 2024-03-09T09:27:30.080 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-getopt_long.h 2024-03-09T09:27:30.081 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-quotearg.sh 2024-03-09T09:27:30.082 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-langinfo.c 2024-03-09T09:27:30.082 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-inttypes.c 2024-03-09T09:27:30.083 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-filevercmp.c 2024-03-09T09:27:30.083 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-rmdir.c 2024-03-09T09:27:30.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.077+0000 7f59c7000640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c00aefe0 0x7f59c01493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:30.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.077+0000 7f59c7000640 1 -- 172.21.15.67:0/3924821675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c00c8dc0 con 0x7f59c00aefe0 2024-03-09T09:27:30.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.078+0000 7f59c5200640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c00aefe0 0x7f59c01493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:30.086 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-futimens.h 2024-03-09T09:27:30.086 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/setsockopt.c 2024-03-09T09:27:30.086 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-exclude1.sh 2024-03-09T09:27:30.086 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-freopen-safer.c 2024-03-09T09:27:30.086 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-getaddrinfo.c 2024-03-09T09:27:30.086 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-iconv.c 2024-03-09T09:27:30.086 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/zerosize-ptr.h 2024-03-09T09:27:30.087 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-lseek.c 2024-03-09T09:27:30.087 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-usleep.c 2024-03-09T09:27:30.087 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/gnulib.mk 2024-03-09T09:27:30.088 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-unlinkat.c 2024-03-09T09:27:30.088 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-quotearg.c 2024-03-09T09:27:30.088 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-base64.c 2024-03-09T09:27:30.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.079+0000 7f59c5200640 1 -- 172.21.15.67:0/3924821675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f59c004f3c0 con 0x7f59c00aefe0 2024-03-09T09:27:30.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.080+0000 7f59c5200640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c00aefe0 0x7f59c01493b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f59c014a450 tx=0x7f59ac000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:30.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.080+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac019880 con 0x7f59c00aefe0 2024-03-09T09:27:30.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.081+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5176 (secure 0 0 0) 0x7f59ac019880 con 0x7f59c00aefe0 2024-03-09T09:27:30.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.081+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c00aefe0 msgr2=0x7f59c01493b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:30.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.081+0000 7f59bd600640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c00aefe0 0x7f59c01493b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f59c014a450 tx=0x7f59ac000ed0 comp rx=0 tx=0).stop 2024-03-09T09:27:30.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.081+0000 7f59bd600640 1 -- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c00aefe0 msgr2=0x7f59c01493b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:30.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.081+0000 7f59bd600640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f59c00aefe0 0x7f59c01493b0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f59c014a450 tx=0x7f59ac000ed0 comp rx=0 tx=0).stop 2024-03-09T09:27:30.090 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:27:30.090 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:30.091 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:30.091 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:27:30.091 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 280, 2024-03-09T09:27:30.091 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:27:30.091 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:27:30.091 INFO:teuthology.orchestra.run.smithi067.stdout: 1 2024-03-09T09:27:30.091 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:30.091 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:27:30.092 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:30.092 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:27:30.092 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:27:30.092 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:30.092 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:30.092 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:30.092 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:30.092 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:30.092 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:30.093 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:30.093 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:30.093 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:30.093 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:30.093 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:30.093 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:30.093 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:27:30.094 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:27:30.094 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:30.094 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:30.094 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:30.094 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:30.094 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:30.094 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:30.095 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:30.095 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:30.095 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:30.095 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:30.095 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:30.095 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:30.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.096 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:27:30.096 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:27:30.096 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:27:30.096 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:27:30.096 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:27:30.096 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:27:30.097 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:30.097 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:30.097 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:27:30.097 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:27:30.097 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:27:30.097 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:27:30.097 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:27:30.097 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:27:30.098 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:27:30.098 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:30.098 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:27:30.098 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:30.098 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:30.098 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:30.098 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:30.098 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:30.098 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:30.098 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:30.098 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:30.099 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:30.099 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:30.099 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:30.099 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:30.099 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:27:30.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.099 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:27:30.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.099 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:30.099 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:30.100 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:30.100 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:30.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.100 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:30.100 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:27:30.100 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:30.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.100 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:30.100 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:27:30.100 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:30.101 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:30.101 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:30.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.101 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:27:30.101 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:27:30.101 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:30.101 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:30.101 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:30.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.101 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:30.102 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:30.102 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:30.102 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:30.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.102 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:30.102 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:27:30.102 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:30.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.102 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:30.102 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:27:30.103 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:30.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:30.103 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:30.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.103 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:27:30.103 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:27:30.103 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:30.103 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:30.103 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:30.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.104 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:30.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:30.104 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:30.104 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:30.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.104 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:30.104 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:27:30.104 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:30.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:30.105 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:27:30.105 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:30.105 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:30.105 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:30.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.105 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:27:30.105 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:27:30.105 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:30.105 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:30.106 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:30.106 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:30.106 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:30.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.106 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:27:30.106 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:27:30.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.106 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:30.106 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:30.106 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:30.106 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:30.107 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:30.107 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:27:30.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.107 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:30.107 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:30.107 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:30.107 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:30.107 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:30.107 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:27:30.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.108 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:30.108 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:30.108 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:27:30.108 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:30.108 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:30.108 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:27:30.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.108 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:27:30.108 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:30.109 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:30.110 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:30.110 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:30.110 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:30.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:30.110 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:27:30.110 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:27:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.082+0000 7f59c7000640 1 -- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59a006b3d0 msgr2=0x7f59a006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.082+0000 7f59c7000640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f59a006b3d0 0x7f59a006d890 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f59b8035000 tx=0x7f59b8030000 comp rx=0 tx=0).stop 2024-03-09T09:27:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.082+0000 7f59c7000640 1 -- 172.21.15.67:0/3924821675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c006c1d0 msgr2=0x7f59c01514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.082+0000 7f59c7000640 1 --2- 172.21.15.67:0/3924821675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59c006c1d0 0x7f59c01514a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59b400b550 tx=0x7f59b400ba10 comp rx=0 tx=0).stop 2024-03-09T09:27:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.082+0000 7f59c6600640 1 -- 172.21.15.67:0/3924821675 reap_dead start 2024-03-09T09:27:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.082+0000 7f59c6600640 1 -- 172.21.15.67:0/3924821675 reap_dead start 2024-03-09T09:27:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.082+0000 7f59c7000640 1 -- 172.21.15.67:0/3924821675 shutdown_connections 2024-03-09T09:27:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.082+0000 7f59c7000640 1 -- 172.21.15.67:0/3924821675 >> 172.21.15.67:0/3924821675 conn(0x7f59c00cb480 msgr2=0x7f59c0153a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.082+0000 7f59c7000640 1 -- 172.21.15.67:0/3924821675 shutdown_connections 2024-03-09T09:27:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:30.082+0000 7f59c7000640 1 -- 172.21.15.67:0/3924821675 wait complete. 2024-03-09T09:27:30.113 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:27:30.113 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:27:30.114 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-frexpl.c 2024-03-09T09:27:30.114 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-gethostname.c 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mkdir.h 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-binary-io.sh 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-sys_ioctl.c 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-time.c 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/signature.h 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-isnanf.h 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-isnanf-nolibm.c 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-wcwidth.c 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-freadseek.c 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-areadlink-with-size.c 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-select-stdin.c 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsstr1.c 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-exclude3.sh 2024-03-09T09:27:30.115 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fopen.c 2024-03-09T09:27:30.116 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-lock.c 2024-03-09T09:27:30.116 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/locale/ 2024-03-09T09:27:30.360 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/locale/fr/ 2024-03-09T09:27:30.377 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/ 2024-03-09T09:27:30.380 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.mo 2024-03-09T09:27:30.384 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.po 2024-03-09T09:27:30.385 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fpending.c 2024-03-09T09:27:30.385 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-netinet_in.c 2024-03-09T09:27:30.387 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-bitrotate.c 2024-03-09T09:27:30.387 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-c-strcasecmp.c 2024-03-09T09:27:30.387 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsstr3.c 2024-03-09T09:27:30.387 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-hash.c 2024-03-09T09:27:30.387 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-copy-acl.c 2024-03-09T09:27:30.387 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-sigaction.c 2024-03-09T09:27:30.387 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xstrtoumax.c 2024-03-09T09:27:30.387 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fseterr.c 2024-03-09T09:27:30.387 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-areadlink.c 2024-03-09T09:27:30.387 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-signbit.c 2024-03-09T09:27:30.389 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-btowc1.sh 2024-03-09T09:27:30.389 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-strtod.c 2024-03-09T09:27:30.389 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-atexit.c 2024-03-09T09:27:30.389 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-getdate.c 2024-03-09T09:27:30.389 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fpurge.c 2024-03-09T09:27:30.389 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-select.c 2024-03-09T09:27:30.389 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-tls.c 2024-03-09T09:27:30.390 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/sleep.c 2024-03-09T09:27:30.390 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbrtowc2.sh 2024-03-09T09:27:30.390 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-stat.c 2024-03-09T09:27:30.391 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-freadptr.sh 2024-03-09T09:27:30.391 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/accept.c 2024-03-09T09:27:30.391 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-unistd.c 2024-03-09T09:27:30.392 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fopen.h 2024-03-09T09:27:30.392 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-math.c 2024-03-09T09:27:30.392 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-printf-frexp.c 2024-03-09T09:27:30.392 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-vfprintf-posix.sh 2024-03-09T09:27:30.392 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-exclude.c 2024-03-09T09:27:30.392 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-freading.c 2024-03-09T09:27:30.392 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-getdtablesize.c 2024-03-09T09:27:30.393 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/unlinkdir.c 2024-03-09T09:27:30.394 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fflush2.c 2024-03-09T09:27:30.394 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-stat-time.c 2024-03-09T09:27:30.394 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-inet_ntop.c 2024-03-09T09:27:30.395 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-vc-list-files-git.sh 2024-03-09T09:27:30.395 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-getopt.c 2024-03-09T09:27:30.395 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-alignof.c 2024-03-09T09:27:30.395 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/macros.h 2024-03-09T09:27:30.395 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-symlinkat.c 2024-03-09T09:27:30.395 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-getgroups.c 2024-03-09T09:27:30.396 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-xprintf-posix.sh 2024-03-09T09:27:30.396 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fdopendir.c 2024-03-09T09:27:30.396 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-sys_socket.c 2024-03-09T09:27:30.396 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbscasecmp.c 2024-03-09T09:27:30.396 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-getopt.h 2024-03-09T09:27:30.397 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsrtowcs1.sh 2024-03-09T09:27:30.397 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsinit.sh 2024-03-09T09:27:30.398 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-gettimeofday.c 2024-03-09T09:27:30.398 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsstr2.c 2024-03-09T09:27:30.399 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/init.sh 2024-03-09T09:27:30.399 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-symlink.c 2024-03-09T09:27:30.399 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-utimens.c 2024-03-09T09:27:30.400 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-readlink.c 2024-03-09T09:27:30.400 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-lchown.h 2024-03-09T09:27:30.400 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-c-strncasecmp.c 2024-03-09T09:27:30.400 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-lutimens.h 2024-03-09T09:27:30.401 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-copy-acl.sh 2024-03-09T09:27:30.401 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-exclude6.sh 2024-03-09T09:27:30.401 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-freadptr2.c 2024-03-09T09:27:30.402 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fflush2.sh 2024-03-09T09:27:30.402 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-sys_utsname.c 2024-03-09T09:27:30.403 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsrtowcs2.sh 2024-03-09T09:27:30.403 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-file-has-acl.c 2024-03-09T09:27:30.406 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-signal.c 2024-03-09T09:27:30.406 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-sha1.c 2024-03-09T09:27:30.406 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-freadptr2.sh 2024-03-09T09:27:30.406 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-areadlinkat.c 2024-03-09T09:27:30.406 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-unlink.h 2024-03-09T09:27:30.406 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-dup2.c 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-freadptr.c 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-file-has-acl.sh 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-u64.c 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-striconv.c 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-sameacls.c 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-remove.c 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-fsync.c 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/gnulib-tests/test-mbsrtowcs4.sh 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_require_config_h 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_system_h_headers 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_prohibit_fail_0 2024-03-09T09:27:30.407 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_prohibit_magic_number_exit 2024-03-09T09:27:30.408 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/THANKS-to-translators 2024-03-09T09:27:30.408 INFO:tasks.workunit.client.1.smithi134.stdout:coreutils-8.5/.x-sc_useless_cpp_parens 2024-03-09T09:27:30.408 INFO:tasks.workunit.client.1.smithi134.stderr:+ mkdir .snap/k 2024-03-09T09:27:31.012 INFO:tasks.workunit.client.1.smithi134.stderr:+ sync 2024-03-09T09:27:32.962 INFO:tasks.workunit.client.1.smithi134.stderr:+ rm -rv coreutils-8.5 2024-03-09T09:27:33.030 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:27:33.036 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:27:33.988 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_prohibit_strcmp' 2024-03-09T09:27:33.988 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_GPL_version' 2024-03-09T09:27:33.988 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.prev-version' 2024-03-09T09:27:33.988 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_program_name' 2024-03-09T09:27:33.988 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/expr.x' 2024-03-09T09:27:33.988 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/hostid.1' 2024-03-09T09:27:33.988 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/seq.x' 2024-03-09T09:27:33.988 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/yes.1' 2024-03-09T09:27:33.988 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tee.x' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/cat.x' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/true.x' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/dirname.1' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tail.1' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/timeout.1' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/uname.1' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/unlink.1' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/expr.1' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/printf.1' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/logname.x' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/expand.1' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sum.x' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/md5sum.x' 2024-03-09T09:27:33.989 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/env.x' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/nice.x' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/date.1' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tr.1' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/join.x' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/stty.1' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/seq.1' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/pinky.x' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/hostname.x' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/install.1' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sha512sum.x' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/install.x' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tail.x' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/dircolors.1' 2024-03-09T09:27:33.990 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/hostid.x' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/md5sum.1' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/split.x' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/mv.x' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/users.x' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/od.1' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/ln.x' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/kill.x' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/date.x' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/chown.1' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/paste.x' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/expand.x' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/rm.x' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sha384sum.1' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sleep.1' 2024-03-09T09:27:33.991 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sha512sum.1' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/shuf.x' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/join.1' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/cp.x' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/mktemp.1' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tsort.x' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/factor.1' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/ptx.x' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/who.x' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/touch.x' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/logname.1' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/dd.1' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sleep.x' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/base64.1' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/vdir.x' 2024-03-09T09:27:33.992 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/printenv.x' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sha224sum.1' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/chcon.1' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/whoami.1' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/unlink.x' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/runcon.1' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tty.x' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/id.1' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/printf.x' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/rmdir.1' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/od.x' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/rmdir.x' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/pathchk.x' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sort.1' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/chown.x' 2024-03-09T09:27:33.993 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/df.x' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/mkfifo.x' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/test.x' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/vdir.1' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/mknod.x' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/who.1' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/arch.x' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/stty.x' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/pr.1' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/test.1' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/uniq.x' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/stat.x' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/paste.1' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/cksum.x' 2024-03-09T09:27:33.994 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/ptx.1' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/Makefile.in' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/dircolors.x' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/uptime.x' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/chgrp.x' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/basename.1' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/mktemp.x' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sum.1' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sha1sum.x' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/groups.x' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/mknod.1' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/help2man' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/uname.x' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/ls.1' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/du.1' 2024-03-09T09:27:33.995 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/timeout.x' 2024-03-09T09:27:33.996 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/groups.1' 2024-03-09T09:27:33.996 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/dir.1' 2024-03-09T09:27:34.176 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_prohibit_strcmp' 2024-03-09T09:27:34.176 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_GPL_version' 2024-03-09T09:27:34.176 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.prev-version' 2024-03-09T09:27:34.176 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_program_name' 2024-03-09T09:27:34.176 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/expr.x' 2024-03-09T09:27:34.176 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/hostid.1' 2024-03-09T09:27:34.176 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/seq.x' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/yes.1' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tee.x' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/cat.x' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/true.x' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/dirname.1' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tail.1' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/timeout.1' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/uname.1' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/unlink.1' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/expr.1' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/printf.1' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/logname.x' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/expand.1' 2024-03-09T09:27:34.177 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sum.x' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/md5sum.x' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/env.x' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/nice.x' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/date.1' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tr.1' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/join.x' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/stty.1' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/seq.1' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/pinky.x' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/hostname.x' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/install.1' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sha512sum.x' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/install.x' 2024-03-09T09:27:34.178 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tail.x' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/dircolors.1' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/hostid.x' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/md5sum.1' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/split.x' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/mv.x' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/users.x' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/od.1' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/ln.x' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/kill.x' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/date.x' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/chown.1' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/paste.x' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/expand.x' 2024-03-09T09:27:34.179 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/rm.x' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sha384sum.1' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sleep.1' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sha512sum.1' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/shuf.x' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/join.1' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/cp.x' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/mktemp.1' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tsort.x' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/factor.1' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/ptx.x' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/who.x' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/touch.x' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/logname.1' 2024-03-09T09:27:34.180 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/dd.1' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sleep.x' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/base64.1' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/vdir.x' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/printenv.x' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sha224sum.1' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/chcon.1' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/whoami.1' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/unlink.x' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/runcon.1' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tty.x' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/id.1' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/printf.x' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/rmdir.1' 2024-03-09T09:27:34.181 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/od.x' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/rmdir.x' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/pathchk.x' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sort.1' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/chown.x' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/df.x' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/mkfifo.x' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/test.x' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/vdir.1' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/mknod.x' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/who.1' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/arch.x' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/stty.x' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/pr.1' 2024-03-09T09:27:34.182 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/test.1' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/uniq.x' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/stat.x' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/paste.1' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/cksum.x' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/ptx.1' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/Makefile.in' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/dircolors.x' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/uptime.x' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/chgrp.x' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/basename.1' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/mktemp.x' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sum.1' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sha1sum.x' 2024-03-09T09:27:34.183 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/groups.x' 2024-03-09T09:27:34.184 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/mknod.1' 2024-03-09T09:27:34.184 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/help2man' 2024-03-09T09:27:34.184 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/uname.x' 2024-03-09T09:27:34.184 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/ls.1' 2024-03-09T09:27:34.184 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/du.1' 2024-03-09T09:27:34.184 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/timeout.x' 2024-03-09T09:27:34.184 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/groups.1' 2024-03-09T09:27:34.184 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/dir.1' 2024-03-09T09:27:34.940 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sync.x' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/cat.1' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/comm.x' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sha256sum.1' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/comm.1' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/chroot.1' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/csplit.x' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sync.1' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sort.x' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/chmod.x' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/Makefile.am' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/readlink.1' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/uptime.1' 2024-03-09T09:27:34.941 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/pwd.x' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tac.1' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/dir.x' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/mkdir.x' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/split.1' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/chroot.x' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/false.x' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/df.1' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tac.x' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/du.x' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/rm.1' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/head.x' 2024-03-09T09:27:34.942 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/mv.1' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sha384sum.x' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/echo.1' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/su.x' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/pwd.1' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/shred.1' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/fold.x' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/touch.1' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/printenv.1' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/uniq.1' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/id.x' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/chmod.1' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/link.x' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/dirname.x' 2024-03-09T09:27:34.943 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/nproc.1' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/nice.1' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/shred.x' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/fold.1' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sha224sum.x' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/pinky.1' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/runcon.x' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/unexpand.x' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/readlink.x' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/stdbuf.x' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/pr.x' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/whoami.x' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/nl.x' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/base64.x' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tsort.1' 2024-03-09T09:27:34.944 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tty.1' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/kill.1' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/shuf.1' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/csplit.1' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/users.1' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/basename.x' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/stat.1' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sha1sum.1' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/fmt.1' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/unexpand.1' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/mkfifo.1' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/truncate.1' 2024-03-09T09:27:34.945 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/wc.1' 2024-03-09T09:27:34.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/dd.x' 2024-03-09T09:27:34.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/cp.1' 2024-03-09T09:27:34.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/chcon.x' 2024-03-09T09:27:34.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/sha256sum.x' 2024-03-09T09:27:34.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/nproc.x' 2024-03-09T09:27:34.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/wc.x' 2024-03-09T09:27:34.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/yes.x' 2024-03-09T09:27:34.946 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/true.1' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/mkdir.1' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/factor.x' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/cksum.1' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/pathchk.1' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/nohup.x' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/echo.x' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tee.1' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/nohup.1' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/false.1' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/chgrp.1' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/cut.1' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/cut.x' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/fmt.x' 2024-03-09T09:27:34.947 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/stdbuf.1' 2024-03-09T09:27:34.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/nl.1' 2024-03-09T09:27:34.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/env.1' 2024-03-09T09:27:34.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/truncate.x' 2024-03-09T09:27:34.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/ls.x' 2024-03-09T09:27:34.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/ln.1' 2024-03-09T09:27:34.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/link.1' 2024-03-09T09:27:34.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/head.1' 2024-03-09T09:27:34.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/man/tr.x' 2024-03-09T09:27:34.948 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/man' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/TODO' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_prohibit_magic_number_exit' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/who.c' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/chown-core.h' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/expr.c' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/ls-ls.c' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/cp-hash.h' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/chgrp.c' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/ptx.c' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/nohup.c' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/ls.h' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/chroot.c' 2024-03-09T09:27:34.949 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/tail.c' 2024-03-09T09:27:34.950 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/nice.c' 2024-03-09T09:27:34.950 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/fold.c' 2024-03-09T09:27:34.951 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/comm.c' 2024-03-09T09:27:34.969 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sync.x' 2024-03-09T09:27:34.969 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/cat.1' 2024-03-09T09:27:34.969 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/comm.x' 2024-03-09T09:27:34.969 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sha256sum.1' 2024-03-09T09:27:34.969 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/comm.1' 2024-03-09T09:27:34.969 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/chroot.1' 2024-03-09T09:27:34.970 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/csplit.x' 2024-03-09T09:27:34.970 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sync.1' 2024-03-09T09:27:34.970 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sort.x' 2024-03-09T09:27:34.970 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/chmod.x' 2024-03-09T09:27:34.970 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/Makefile.am' 2024-03-09T09:27:34.970 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/readlink.1' 2024-03-09T09:27:34.970 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/uptime.1' 2024-03-09T09:27:34.970 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/pwd.x' 2024-03-09T09:27:34.970 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tac.1' 2024-03-09T09:27:34.970 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/dir.x' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/mkdir.x' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/split.1' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/chroot.x' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/false.x' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/df.1' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tac.x' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/du.x' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/rm.1' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/head.x' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/mv.1' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sha384sum.x' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/echo.1' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/su.x' 2024-03-09T09:27:34.971 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/pwd.1' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/shred.1' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/fold.x' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/touch.1' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/printenv.1' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/uniq.1' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/id.x' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/chmod.1' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/link.x' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/dirname.x' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/nproc.1' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/nice.1' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/shred.x' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/fold.1' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sha224sum.x' 2024-03-09T09:27:34.972 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/pinky.1' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/runcon.x' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/unexpand.x' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/readlink.x' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/stdbuf.x' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/pr.x' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/whoami.x' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/nl.x' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/base64.x' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tsort.1' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tty.1' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/kill.1' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/shuf.1' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/csplit.1' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/users.1' 2024-03-09T09:27:34.973 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/basename.x' 2024-03-09T09:27:34.974 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/stat.1' 2024-03-09T09:27:34.974 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sha1sum.1' 2024-03-09T09:27:34.974 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/fmt.1' 2024-03-09T09:27:34.974 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/unexpand.1' 2024-03-09T09:27:34.975 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/mkfifo.1' 2024-03-09T09:27:34.975 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/truncate.1' 2024-03-09T09:27:34.975 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/wc.1' 2024-03-09T09:27:34.975 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/dd.x' 2024-03-09T09:27:34.975 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/cp.1' 2024-03-09T09:27:34.975 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/chcon.x' 2024-03-09T09:27:34.975 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/sha256sum.x' 2024-03-09T09:27:34.975 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/nproc.x' 2024-03-09T09:27:34.975 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/wc.x' 2024-03-09T09:27:34.975 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/yes.x' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/true.1' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/mkdir.1' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/factor.x' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/cksum.1' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/pathchk.1' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/nohup.x' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/echo.x' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tee.1' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/nohup.1' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/false.1' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/chgrp.1' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/cut.1' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/cut.x' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/fmt.x' 2024-03-09T09:27:34.976 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/stdbuf.1' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/nl.1' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/env.1' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/truncate.x' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/ls.x' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/ln.1' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/link.1' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/head.1' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/man/tr.x' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/man' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/TODO' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_prohibit_magic_number_exit' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/who.c' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/chown-core.h' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/expr.c' 2024-03-09T09:27:34.977 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/ls-ls.c' 2024-03-09T09:27:34.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/cp-hash.h' 2024-03-09T09:27:34.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/chgrp.c' 2024-03-09T09:27:34.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/ptx.c' 2024-03-09T09:27:34.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/nohup.c' 2024-03-09T09:27:34.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/ls.h' 2024-03-09T09:27:34.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/chroot.c' 2024-03-09T09:27:34.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/tail.c' 2024-03-09T09:27:34.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/nice.c' 2024-03-09T09:27:34.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/fold.c' 2024-03-09T09:27:34.978 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/comm.c' 2024-03-09T09:27:35.403 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/chown-core.c' 2024-03-09T09:27:35.403 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/tac.c' 2024-03-09T09:27:35.403 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/prog-fprintf.c' 2024-03-09T09:27:35.403 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/tr.c' 2024-03-09T09:27:35.404 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/tac-pipe.c' 2024-03-09T09:27:35.404 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/groups.c' 2024-03-09T09:27:35.404 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/nproc.c' 2024-03-09T09:27:35.404 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/operand2sig.c' 2024-03-09T09:27:35.404 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/libstdbuf.c' 2024-03-09T09:27:35.404 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/nl.c' 2024-03-09T09:27:35.404 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/env.c' 2024-03-09T09:27:35.404 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/dcgen' 2024-03-09T09:27:35.404 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/stat.c' 2024-03-09T09:27:35.405 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/base64.c' 2024-03-09T09:27:35.405 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/chown.c' 2024-03-09T09:27:35.405 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/od.c' 2024-03-09T09:27:35.405 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/mkdir.c' 2024-03-09T09:27:35.405 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/logname.c' 2024-03-09T09:27:35.405 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/extract-magic' 2024-03-09T09:27:35.405 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/wheel-gen.pl' 2024-03-09T09:27:35.405 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/chcon.c' 2024-03-09T09:27:35.405 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/printenv.c' 2024-03-09T09:27:35.405 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/copy.c' 2024-03-09T09:27:35.405 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/ls-dir.c' 2024-03-09T09:27:35.406 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/cut.c' 2024-03-09T09:27:35.406 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/uptime.c' 2024-03-09T09:27:35.406 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/hostname.c' 2024-03-09T09:27:35.406 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/dirname.c' 2024-03-09T09:27:35.406 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/sleep.c' 2024-03-09T09:27:35.407 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/stdbuf.c' 2024-03-09T09:27:35.407 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/true.c' 2024-03-09T09:27:35.407 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/df.c' 2024-03-09T09:27:35.407 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/rmdir.c' 2024-03-09T09:27:35.407 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/copy.h' 2024-03-09T09:27:35.407 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/group-list.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/unexpand.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/stty.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/chmod.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/fmt.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/csplit.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/expand.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/kill.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/factor.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/wheel-size.h' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/pinky.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/yes.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/remove.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/Makefile.in' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/uname-uname.c' 2024-03-09T09:27:35.408 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/md5sum.c' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/ln.c' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/su.c' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/cksum.c' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/dircolors.h' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/group-list.h' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/false.c' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/operand2sig.h' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/id.c' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/rm.c' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/shred.c' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/printf.c' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/cp.c' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/uname-arch.c' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/dircolors.hin' 2024-03-09T09:27:35.409 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/setuidgid.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/cp-hash.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/tee.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/Makefile.am' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/head.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/users.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/runcon.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/du.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/date.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/unlink.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/echo.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/pathchk.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/tsort.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/touch.c' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/c99-to-c89.diff' 2024-03-09T09:27:35.410 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/join.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/cat.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/sync.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/shuf.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/truncate.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/ls-vdir.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/test.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/remove.h' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/seq.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/readlink.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/uname.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/sum.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/paste.c' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/uname.h' 2024-03-09T09:27:35.411 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/tty.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/fs.h' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/basename.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/dd.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/split.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/whoami.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/timeout.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/getlimits.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/dircolors.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/link.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/sort.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/wc.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/prog-fprintf.h' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/uniq.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/pr.c' 2024-03-09T09:27:35.412 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/system.h' 2024-03-09T09:27:35.413 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/pwd.c' 2024-03-09T09:27:35.413 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/hostid.c' 2024-03-09T09:27:35.413 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/install.c' 2024-03-09T09:27:36.161 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/wheel.h' 2024-03-09T09:27:36.161 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/mkfifo.c' 2024-03-09T09:27:36.161 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/mv.c' 2024-03-09T09:27:36.161 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/lbracket.c' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/mknod.c' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/mktemp.c' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/src/ls.c' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/src' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ru.gmo' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/zh_TW.gmo' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ko.gmo' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ro.po' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/vi.gmo' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/vi.po' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/da.gmo' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/fi.gmo' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/en.po' 2024-03-09T09:27:36.162 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/sl.po' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/fr.gmo' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/it.gmo' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/en@quot.header' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/el.po' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/tr.po' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/it.po' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/stamp-po' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/sk.gmo' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/tr.gmo' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ga.gmo' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ja.po' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/pl.po' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/pt.po' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/de.po' 2024-03-09T09:27:36.163 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/sv.gmo' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/Makevars' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/et.po' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ChangeLog-2007' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/fr.po' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/pt_BR.po' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ga.po' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/cs.po' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/pl.gmo' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/af.gmo' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/lt.po' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/cs.gmo' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/en.gmo' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ca.gmo' 2024-03-09T09:27:36.164 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/sl.gmo' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/eu.gmo' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/gl.gmo' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/uk.gmo' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/bg.gmo' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ro.gmo' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/hu.gmo' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/nl.po' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/el.gmo' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/LINGUAS' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/de.gmo' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ms.po' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/remove-potcdate.sin' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/da.po' 2024-03-09T09:27:36.165 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/en@boldquot.header' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/be.po' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/Makefile.in.in' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ca.po' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/id.po' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/hu.po' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/zh_CN.po' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/nb.gmo' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/nb.po' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/coreutils.pot' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/bg.po' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/id.gmo' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/sv.po' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/af.po' 2024-03-09T09:27:36.166 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/Rules-quot' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/eu.po' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/es.gmo' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/quot.sed' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ru.po' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ms.gmo' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/zh_TW.po' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/nl.gmo' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/zh_CN.gmo' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/pt.gmo' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/sk.po' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/lg.po' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ko.po' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/pt_BR.gmo' 2024-03-09T09:27:36.167 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/fi.po' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/es.po' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/lt.gmo' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/be.gmo' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/ja.gmo' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/lg.gmo' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/gl.po' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/boldquot.sed' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/et.gmo' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/insert-header.sin' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/POTFILES.in' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/po/uk.po' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/po' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_prohibit_empty_lines_at_EOF' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/GNUmakefile' 2024-03-09T09:27:36.168 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_useless_cpp_parens' 2024-03-09T09:27:36.169 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/ChangeLog' 2024-03-09T09:27:36.169 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_require_config_h' 2024-03-09T09:27:36.169 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/test-lib.sh' 2024-03-09T09:27:36.169 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/require-perl' 2024-03-09T09:27:36.169 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chmod/octal' 2024-03-09T09:27:36.169 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chmod/umask-x' 2024-03-09T09:27:36.169 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chmod/equal-x' 2024-03-09T09:27:36.169 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chmod/thru-dangling' 2024-03-09T09:27:36.169 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chmod/equals' 2024-03-09T09:27:36.169 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chmod/no-x' 2024-03-09T09:27:36.584 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chmod/inaccessible' 2024-03-09T09:27:36.584 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chmod/usage' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chmod/c-option' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chmod/silent' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chmod/setgid' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/chmod' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rmdir/t-slash' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rmdir/fail-perm' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rmdir/ignore' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/rmdir' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chgrp/posix-H' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chgrp/default-no-deref' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chgrp/no-x' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chgrp/basic' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chgrp/deref' 2024-03-09T09:27:36.585 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chgrp/recurse' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/chgrp' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/envvar-check' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chown/basic' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chown/deref' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chown/separator' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/chown/preserve-root' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/chown' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/acl' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/i-2' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/dup-source' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/to-symlink' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/childproof' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/i-5' 2024-03-09T09:27:36.586 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/mv-n' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/part-hardlink' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/dir2dir' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/part-rename' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/into-self-3' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/hard-2' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/trailing-slash' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/hard-verbose' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/force' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/i-link-no' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/i-3' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/backup-dir' 2024-03-09T09:27:36.587 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/i-4' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/into-self' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/diag' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/into-self-2' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/mv-special-1' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/update' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/atomic2' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/no-target-dir' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/hard-4' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/sticky-to-xpart' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/dir-file' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/i-1' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/hard-3' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/part-symlink' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/hard-link-1' 2024-03-09T09:27:36.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/backup-is-src' 2024-03-09T09:27:36.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/perm-1' 2024-03-09T09:27:36.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/partition-perm' 2024-03-09T09:27:36.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/part-fail' 2024-03-09T09:27:36.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/leak-fd' 2024-03-09T09:27:36.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/atomic' 2024-03-09T09:27:36.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mv/into-self-4' 2024-03-09T09:27:36.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/mv' 2024-03-09T09:27:36.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/multihardlink' 2024-03-09T09:27:36.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/stat-failed' 2024-03-09T09:27:36.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/stat-free-symlinks' 2024-03-09T09:27:36.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/color-clear-to-eol' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/rt-1' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/color-dtype-dir' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/symlink-slash' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/infloop' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/follow-slink' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/no-arg' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/inode' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/color-norm' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/no-cap' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/x-option' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/proc-selinux-segfault' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/abmon-align' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/file-type' 2024-03-09T09:27:36.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/stat-vs-dirent' 2024-03-09T09:27:36.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/nameless-uid' 2024-03-09T09:27:36.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/dangle' 2024-03-09T09:27:36.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/readdir-mountpoint-inode' 2024-03-09T09:27:36.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/stat-dtype' 2024-03-09T09:27:36.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/dired' 2024-03-09T09:27:36.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/capability' 2024-03-09T09:27:36.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/m-option' 2024-03-09T09:27:36.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ls/recursive' 2024-03-09T09:27:36.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/ls' 2024-03-09T09:27:36.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/p-slashdot' 2024-03-09T09:27:36.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/t-slash' 2024-03-09T09:27:37.140 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/perm' 2024-03-09T09:27:37.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/p-thru-slink' 2024-03-09T09:27:37.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/p-3' 2024-03-09T09:27:37.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/p-2' 2024-03-09T09:27:37.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/writable-under-readonly' 2024-03-09T09:27:37.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/selinux' 2024-03-09T09:27:37.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/p-1' 2024-03-09T09:27:37.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/p-v' 2024-03-09T09:27:37.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/parents' 2024-03-09T09:27:37.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/mkdir/special-1' 2024-03-09T09:27:37.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/mkdir' 2024-03-09T09:27:37.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/install/create-leading' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/install/install-C-root' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/install/install-C' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/install/basic-1' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/install/install-C-selinux' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/install/d-slashdot' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/install/strip-program' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/install/trap' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/install' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/readlink/rl-1' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/readlink/can-m' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/readlink/can-f' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/readlink/can-e' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/readlink' 2024-03-09T09:27:37.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/2g' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/8gb' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/restore-wd' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/long-from-unreadable' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/two-args' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/slink' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/inaccessible-cwd' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/no-x' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/fd-leak' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/basic' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/one-file-system' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/trailing-slash' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/no-deref' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/long-sloop' 2024-03-09T09:27:37.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/inacc-dir' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/deref' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/inacc-dest' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/files0-from' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/slash' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/exclude' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/hard-link' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/du/deref-args' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/du' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/id/no-context' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/id' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/not-owner' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/fifo' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/obsolescent' 2024-03-09T09:27:37.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/dangling-symlink' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/trailing-slash' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/no-create-missing' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/no-rights' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/relative' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/no-dereference' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/empty-file' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/dir-1' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/60-seconds' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/now-owned-by-other' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/fail-diag' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/touch/read-only' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/touch' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/flush-initial' 2024-03-09T09:27:37.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/F-vs-rename' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/assert-2' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/inotify-rotate' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/proc-ksyms' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/infloop-1' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/start-middle' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/inotify-hash-abuse' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/pipe-f2' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/tail-n0f' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/append-only' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/pipe-f' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/wait' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/assert' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/inotify-race' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/follow-stdin' 2024-03-09T09:27:37.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/inotify-hash-abuse2' 2024-03-09T09:27:37.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/big-4gb' 2024-03-09T09:27:37.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/tail-2/pid' 2024-03-09T09:27:37.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/tail-2' 2024-03-09T09:27:37.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/lang-default' 2024-03-09T09:27:37.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/Makefile.in' 2024-03-09T09:27:37.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/Coreutils.pm' 2024-03-09T09:27:37.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/Makefile.am' 2024-03-09T09:27:37.560 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/chown-core.c' 2024-03-09T09:27:37.560 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/tac.c' 2024-03-09T09:27:37.560 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/prog-fprintf.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/tr.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/tac-pipe.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/groups.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/nproc.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/operand2sig.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/libstdbuf.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/nl.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/env.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/dcgen' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/stat.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/base64.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/chown.c' 2024-03-09T09:27:37.561 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/od.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/mkdir.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/logname.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/extract-magic' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/wheel-gen.pl' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/chcon.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/printenv.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/copy.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/ls-dir.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/cut.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/uptime.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/hostname.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/dirname.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/sleep.c' 2024-03-09T09:27:37.562 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/stdbuf.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/true.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/df.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/rmdir.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/copy.h' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/group-list.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/unexpand.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/stty.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/chmod.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/fmt.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/csplit.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/expand.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/kill.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/factor.c' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/wheel-size.h' 2024-03-09T09:27:37.563 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/pinky.c' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/yes.c' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/remove.c' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/Makefile.in' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/uname-uname.c' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/md5sum.c' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/ln.c' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/su.c' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/cksum.c' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/dircolors.h' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/group-list.h' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/false.c' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/operand2sig.h' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/id.c' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/rm.c' 2024-03-09T09:27:37.564 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/shred.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/printf.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/cp.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/uname-arch.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/dircolors.hin' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/setuidgid.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/cp-hash.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/tee.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/Makefile.am' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/head.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/users.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/runcon.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/du.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/date.c' 2024-03-09T09:27:37.565 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/unlink.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/echo.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/pathchk.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/tsort.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/touch.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/c99-to-c89.diff' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/join.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/cat.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/sync.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/shuf.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/truncate.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/ls-vdir.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/test.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/remove.h' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/seq.c' 2024-03-09T09:27:37.566 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/readlink.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/uname.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/sum.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/paste.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/uname.h' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/tty.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/fs.h' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/basename.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/dd.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/split.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/whoami.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/timeout.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/getlimits.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/dircolors.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/link.c' 2024-03-09T09:27:37.567 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/sort.c' 2024-03-09T09:27:37.568 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/wc.c' 2024-03-09T09:27:37.568 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/prog-fprintf.h' 2024-03-09T09:27:37.568 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/uniq.c' 2024-03-09T09:27:37.568 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/pr.c' 2024-03-09T09:27:37.568 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/system.h' 2024-03-09T09:27:37.568 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/pwd.c' 2024-03-09T09:27:37.568 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/hostid.c' 2024-03-09T09:27:37.568 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/install.c' 2024-03-09T09:27:37.784 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/hash' 2024-03-09T09:27:37.784 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/rm1' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/empty-inacc' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/r-3' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/one-file-system2' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/readdir-bug' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/r-1' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/f-1' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/i-no-r' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/ir-1' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/fail-eacces' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/i-never' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/r-4' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/dir-no-w' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/no-give-up' 2024-03-09T09:27:37.785 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/dangling-symlink' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/unreadable' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/dot-rel' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/ignorable' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/one-file-system' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/interactive-once' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/unread2' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/inaccessible' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/isatty' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/rm5' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/interactive-always' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/unread3' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/fail-eperm' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/dir-nonrecur' 2024-03-09T09:27:37.786 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/empty-name' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/rm4' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/cycle' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/v-slash' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/deep-1' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/rm2' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/rm3' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/sunos-1' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/deep-2' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/i-1' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/r-2' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/ext3-perf' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/fail-2eperm' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/rm/read-only' 2024-03-09T09:27:37.787 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/rm' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/check.mk' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/pr-tests' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/o3ml24f-bl-tn' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+2l24f-bl' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/b2l17-FF' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+3ml24f-tn-bl' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/l24f-t' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3b3l15f-t' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/4-7l24-FF' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+6a2l17-FF' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/nl24f-bl' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/nSml24-bl-FF' 2024-03-09T09:27:37.788 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/a3l24f-lm' 2024-03-09T09:27:37.789 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+3ml20f-bl-FF' 2024-03-09T09:27:37.789 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3b3f-0F' 2024-03-09T09:27:37.789 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/FFn' 2024-03-09T09:27:37.789 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W27l24f-ll' 2024-03-09T09:27:37.789 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tne8o3-t_tab' 2024-03-09T09:27:37.789 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/a2l17-FF' 2024-03-09T09:27:37.789 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/a2l15-FF' 2024-03-09T09:27:37.789 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ml24f-0F' 2024-03-09T09:27:37.789 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W72Jl24f-ll' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/o3b3Snl24f-tn' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/nSml24-t-tFFFF' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W26l24f-ll' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3b3l15-t' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ml24-t-FF' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ttb3-0FF' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+4-8a2l17-FF' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tFFt-ll' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/b3-0F' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+3-7l24-FF' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tt-0FF' 2024-03-09T09:27:37.790 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/Ja3l24f-lm' 2024-03-09T09:27:37.791 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/w72l24f-ll' 2024-03-09T09:27:37.791 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W35b3l24f-lm' 2024-03-09T09:27:37.791 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/0FF' 2024-03-09T09:27:37.791 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/0Ft' 2024-03-09T09:27:37.791 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/2-S_f-t_notab' 2024-03-09T09:27:37.791 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/2sw60f-t_notab' 2024-03-09T09:27:37.791 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3b2l17-FF' 2024-03-09T09:27:37.791 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tFFt-lm' 2024-03-09T09:27:37.791 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3-FF' 2024-03-09T09:27:37.791 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/t-t' 2024-03-09T09:27:37.792 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/Jml24f-lm-lo' 2024-03-09T09:27:37.792 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/nN1+3l24f-bl' 2024-03-09T09:27:37.792 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/0FFnt' 2024-03-09T09:27:37.792 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tta3-0FF' 2024-03-09T09:27:37.792 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3-0F' 2024-03-09T09:27:37.792 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+6b3l13f-FF' 2024-03-09T09:27:37.792 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3ml24-t' 2024-03-09T09:27:37.792 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/t-0FF' 2024-03-09T09:27:38.681 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/o3Jml24f-lm-lo' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tn_2e8-t_tab' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+8l20-FF' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W35Ja3l24f-lm' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ml24-t' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W35ml24f-lm-lo' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W-72l24f-ll' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/o3mSl24f-bl-tn' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/t-FF' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3ml24-t-FF' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3b3f-FF' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tta3-FF' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W35Jml24f-lmlo' 2024-03-09T09:27:38.682 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tFnFt' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/0FnFnt' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/0FnFt' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/nSml24-t-t-FF' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/b3f-0FF' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tn' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tt-FF' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W28l24f-ll' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tb3-FF' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tn2e8-t_tab' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/l24-FF' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+4b2l17f-0FF' 2024-03-09T09:27:38.683 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3a3f-0F' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/FFtn' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/a3f-FF' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+2l24f-0FF' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/2s_f-t_notab' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/a3l15-t' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3b3f-0FF' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/nJml24f-lmlolm' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ttb3-FF' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/loli' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/b3l15-t' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3-5l24f-t' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ta3-0FF' 2024-03-09T09:27:38.684 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/0F' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/FF' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ml24f-t' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/t_tab_' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/o3b3l24f-tn' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/4l24-FF' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/o3mSnl24fbltn' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tFFn' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+5-8b3l17f-FF' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/nSml20-bl-FF' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/o3a3Sl24f-tn' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/nJml24f-lmlmlo' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W35a3l24f-lm' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/nN15l24f-bl' 2024-03-09T09:27:38.685 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/a3f-0F' 2024-03-09T09:27:38.686 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/b3f-0F' 2024-03-09T09:27:38.686 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/t_notab' 2024-03-09T09:27:38.686 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+5a3l13f-0FF' 2024-03-09T09:27:38.686 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/nSml20-t-t-FF' 2024-03-09T09:27:38.686 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tn2e5o3-t_tab' 2024-03-09T09:27:38.686 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ml24f-lm-lo' 2024-03-09T09:27:38.686 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3f-FF' 2024-03-09T09:27:38.686 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/t-bl' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3ml24f-t' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/b3f-FF' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/FnFn' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+3ml24f-bl-tn' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/0FFt' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tb3-0FF' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tt-t' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/2-Sf-t_notab' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ta3-FF' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W20l24f-ll' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tn_2e8S-t_tab' 2024-03-09T09:27:38.687 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/b2l15-FF' 2024-03-09T09:27:38.688 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/l24-t' 2024-03-09T09:27:38.688 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3l24f-t' 2024-03-09T09:27:38.688 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tn2e8o3-t_tab' 2024-03-09T09:27:38.688 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/a3f-0FF' 2024-03-09T09:27:38.688 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/a3l15f-t' 2024-03-09T09:27:38.688 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/Jb3l24f-lm' 2024-03-09T09:27:38.688 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3a2l17-FF' 2024-03-09T09:27:38.688 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3f-0F' 2024-03-09T09:27:38.688 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/t_tab' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3l24-t' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/W35Jb3l24f-lm' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tFFt-bl' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tFFt' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+2-5l24f-0FF' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+3l24f-bl' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+7l24-FF' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3a3l15-t' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ml24f-t-0F' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ml24-FF' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/0Fnt' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/2f-t_notab' 2024-03-09T09:27:38.689 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/2s_w60f-t_nota' 2024-03-09T09:27:38.690 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/ml20-FF-t' 2024-03-09T09:27:38.690 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tt-bl' 2024-03-09T09:27:39.087 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/wheel.h' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/mkfifo.c' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/mv.c' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/lbracket.c' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/mknod.c' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/mktemp.c' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/src/ls.c' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/src' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ru.gmo' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/zh_TW.gmo' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ko.gmo' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ro.po' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/vi.gmo' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/vi.po' 2024-03-09T09:27:39.088 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/da.gmo' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/fi.gmo' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/en.po' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/sl.po' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/fr.gmo' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/it.gmo' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/en@quot.header' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/el.po' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/tr.po' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/it.po' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/stamp-po' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/sk.gmo' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/tr.gmo' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ga.gmo' 2024-03-09T09:27:39.089 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ja.po' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/pl.po' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/pt.po' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/de.po' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/sv.gmo' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/Makevars' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/et.po' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ChangeLog-2007' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/fr.po' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/pt_BR.po' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ga.po' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/cs.po' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/pl.gmo' 2024-03-09T09:27:39.090 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/af.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/lt.po' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/cs.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/en.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ca.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/sl.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/eu.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/gl.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/uk.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/bg.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ro.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/hu.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/nl.po' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/el.gmo' 2024-03-09T09:27:39.091 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/LINGUAS' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/de.gmo' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ms.po' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/remove-potcdate.sin' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/da.po' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/en@boldquot.header' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/be.po' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/Makefile.in.in' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ca.po' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/id.po' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/hu.po' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/zh_CN.po' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/nb.gmo' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/nb.po' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/coreutils.pot' 2024-03-09T09:27:39.092 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/bg.po' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/id.gmo' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/sv.po' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/af.po' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/Rules-quot' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/eu.po' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/es.gmo' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/quot.sed' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ru.po' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ms.gmo' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/zh_TW.po' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/nl.gmo' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/zh_CN.gmo' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/pt.gmo' 2024-03-09T09:27:39.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/sk.po' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/lg.po' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ko.po' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/pt_BR.gmo' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/fi.po' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/es.po' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/lt.gmo' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/be.gmo' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/ja.gmo' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/lg.gmo' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/gl.po' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/boldquot.sed' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/et.gmo' 2024-03-09T09:27:39.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/insert-header.sin' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/POTFILES.in' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/po/uk.po' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/po' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_prohibit_empty_lines_at_EOF' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/GNUmakefile' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_useless_cpp_parens' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/ChangeLog' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_require_config_h' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/test-lib.sh' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/require-perl' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chmod/octal' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chmod/umask-x' 2024-03-09T09:27:39.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chmod/equal-x' 2024-03-09T09:27:39.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chmod/thru-dangling' 2024-03-09T09:27:39.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chmod/equals' 2024-03-09T09:27:39.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chmod/no-x' 2024-03-09T09:27:39.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3a3l15f-t' 2024-03-09T09:27:39.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/o3a3l24f-tn' 2024-03-09T09:27:39.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/o3b3Sl24f-tn' 2024-03-09T09:27:39.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/tne8-t_tab' 2024-03-09T09:27:39.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/a3-0F' 2024-03-09T09:27:39.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/nSml20-t-tFFFF' 2024-03-09T09:27:39.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/b3l24f-lm' 2024-03-09T09:27:39.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/2w60f-t_notab' 2024-03-09T09:27:39.588 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/2sf-t_notab' 2024-03-09T09:27:39.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/b3l15f-t' 2024-03-09T09:27:39.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/3ml24-FF' 2024-03-09T09:27:39.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/n+3l24f-0FF' 2024-03-09T09:27:39.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/pr/o3a3Snl24f-tn' 2024-03-09T09:27:39.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/pr' 2024-03-09T09:27:39.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/CuTmpdir.pm' 2024-03-09T09:27:39.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/df/unreadable' 2024-03-09T09:27:39.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/df/total-verify' 2024-03-09T09:27:39.589 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/df' 2024-03-09T09:27:39.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ln/sf-1' 2024-03-09T09:27:39.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ln/hard-backup' 2024-03-09T09:27:39.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ln/slash-decorated-nonexistent-dest' 2024-03-09T09:27:39.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ln/hard-to-sym' 2024-03-09T09:27:39.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ln/target-1' 2024-03-09T09:27:39.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ln/backup-1' 2024-03-09T09:27:39.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/ln/misc' 2024-03-09T09:27:39.590 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/ln' 2024-03-09T09:27:39.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/cp-mv-backup' 2024-03-09T09:27:39.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/acl' 2024-03-09T09:27:39.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/perm' 2024-03-09T09:27:39.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/cp-deref' 2024-03-09T09:27:39.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/special-bits' 2024-03-09T09:27:39.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/link' 2024-03-09T09:27:39.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/preserve-slink-time' 2024-03-09T09:27:39.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/symlink-slash' 2024-03-09T09:27:39.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/cp-mv-enotsup-xattr' 2024-03-09T09:27:39.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/thru-dangling' 2024-03-09T09:27:39.591 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/reflink-perm' 2024-03-09T09:27:39.592 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/r-vs-symlink' 2024-03-09T09:27:39.592 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/preserve-2' 2024-03-09T09:27:39.592 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/special-f' 2024-03-09T09:27:39.592 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/file-perm-race' 2024-03-09T09:27:39.592 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/slink-2-slink' 2024-03-09T09:27:39.592 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/proc-zero-len' 2024-03-09T09:27:39.592 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/reflink-auto' 2024-03-09T09:27:39.592 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/src-base-dot' 2024-03-09T09:27:39.592 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/link-preserve' 2024-03-09T09:27:39.592 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/cp-i' 2024-03-09T09:27:39.592 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/cp-a-selinux' 2024-03-09T09:27:39.593 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/preserve-gid' 2024-03-09T09:27:39.593 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/no-deref-link3' 2024-03-09T09:27:39.593 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/backup-dir' 2024-03-09T09:27:39.593 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/into-self' 2024-03-09T09:27:39.593 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/same-file' 2024-03-09T09:27:39.594 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/cp-HL' 2024-03-09T09:27:39.594 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/link-no-deref' 2024-03-09T09:27:39.594 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/parent-perm-race' 2024-03-09T09:27:39.594 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/proc-short-read' 2024-03-09T09:27:39.594 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/no-deref-link1' 2024-03-09T09:27:39.594 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/parent-perm' 2024-03-09T09:27:39.594 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/dir-vs-file' 2024-03-09T09:27:39.594 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/sparse' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/link-heap' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/no-deref-link2' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/dir-rm-dest' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/dir-slash' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/fail-perm' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/deref-slink' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/existing-perm-race' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/abuse' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/cp-parents' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/backup-1' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/backup-is-src' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/cp/capability' 2024-03-09T09:27:39.595 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/cp' 2024-03-09T09:27:39.596 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/dd/not-rewound' 2024-03-09T09:27:39.596 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/dd/skip-seek2' 2024-03-09T09:27:39.596 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/dd/unblock' 2024-03-09T09:27:39.596 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/dd/skip-seek-past-dev' 2024-03-09T09:27:39.596 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/dd/direct' 2024-03-09T09:27:39.596 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/dd/unblock-sync' 2024-03-09T09:27:39.596 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/dd/skip-seek-past-file' 2024-03-09T09:27:39.596 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/dd/stderr' 2024-03-09T09:27:39.596 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/dd/reblock' 2024-03-09T09:27:39.596 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/dd/misc' 2024-03-09T09:27:39.596 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/dd/skip-seek' 2024-03-09T09:27:39.597 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/dd' 2024-03-09T09:27:39.597 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/fold' 2024-03-09T09:27:39.597 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/tty-eof' 2024-03-09T09:27:39.597 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sort-rand' 2024-03-09T09:27:39.597 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/seq-long-double' 2024-03-09T09:27:39.597 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/shred-remove' 2024-03-09T09:27:40.378 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/nice' 2024-03-09T09:27:40.379 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/tee' 2024-03-09T09:27:40.379 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/truncate-owned-by-other' 2024-03-09T09:27:40.379 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/mknod' 2024-03-09T09:27:40.379 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/expand' 2024-03-09T09:27:40.379 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/timeout-parameters' 2024-03-09T09:27:40.379 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sum' 2024-03-09T09:27:40.379 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/readlink-fp-loop' 2024-03-09T09:27:40.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/printenv' 2024-03-09T09:27:40.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/nohup' 2024-03-09T09:27:40.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/printf-hex' 2024-03-09T09:27:40.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/ptx' 2024-03-09T09:27:40.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/wc-files0' 2024-03-09T09:27:40.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/tac-continue' 2024-03-09T09:27:40.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sort-merge-fdlimit' 2024-03-09T09:27:40.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/truncate-dir-fail' 2024-03-09T09:27:40.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/env-null' 2024-03-09T09:27:40.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/xstrtol' 2024-03-09T09:27:40.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/tr' 2024-03-09T09:27:40.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/paste' 2024-03-09T09:27:40.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/printf' 2024-03-09T09:27:40.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/comm' 2024-03-09T09:27:40.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/id-context' 2024-03-09T09:27:40.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/chcon' 2024-03-09T09:27:40.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/shred-passes' 2024-03-09T09:27:40.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/stat-fmt' 2024-03-09T09:27:40.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/truncate-parameters' 2024-03-09T09:27:40.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/date-sec' 2024-03-09T09:27:40.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/tsort' 2024-03-09T09:27:40.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sha1sum-vec' 2024-03-09T09:27:40.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/stat-hyphen' 2024-03-09T09:27:40.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/od-multiple-t' 2024-03-09T09:27:40.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/xattr' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/od' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sum-sysv' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/date-next-dow' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/nproc-avail' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/split-a' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/unexpand' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/pathchk1' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/md5sum' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/groups-version' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/head' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/stty-invalid' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/factor' 2024-03-09T09:27:40.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/timeout' 2024-03-09T09:27:40.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/su-fail' 2024-03-09T09:27:40.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/wc-parallel' 2024-03-09T09:27:40.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/md5sum-newline' 2024-03-09T09:27:40.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/tee-dash' 2024-03-09T09:27:40.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/runcon-no-reorder' 2024-03-09T09:27:40.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/dircolors' 2024-03-09T09:27:40.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/basename' 2024-03-09T09:27:40.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/selinux' 2024-03-09T09:27:40.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/head-elide-tail' 2024-03-09T09:27:40.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/help-version' 2024-03-09T09:27:40.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/stdbuf' 2024-03-09T09:27:40.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/expr' 2024-03-09T09:27:40.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/groups-dash' 2024-03-09T09:27:40.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/shuf' 2024-03-09T09:27:40.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sort-version' 2024-03-09T09:27:40.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/pwd-long' 2024-03-09T09:27:40.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/env' 2024-03-09T09:27:40.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sort-continue' 2024-03-09T09:27:40.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/od-x8' 2024-03-09T09:27:40.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/arch' 2024-03-09T09:27:40.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/chcon-fail' 2024-03-09T09:27:40.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/tac' 2024-03-09T09:27:40.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/chroot-fail' 2024-03-09T09:27:40.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/cat-proc' 2024-03-09T09:27:40.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/truncate-fail-diag' 2024-03-09T09:27:40.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/ptx-overrun' 2024-03-09T09:27:40.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sort-month' 2024-03-09T09:27:40.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/date' 2024-03-09T09:27:40.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/ls-time' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/stat-printf' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/printf-surprise' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/pwd-option' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/id-groups' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/stat-slash' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/ls-misc' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/fmt-long-line' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/split-l' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/uniq' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/truncate-relative' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/mktemp' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/nproc-positive' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/truncate-dangling-symlink' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/dirname' 2024-03-09T09:27:40.388 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/wc' 2024-03-09T09:27:40.389 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/pr' 2024-03-09T09:27:40.857 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chmod/inaccessible' 2024-03-09T09:27:40.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chmod/usage' 2024-03-09T09:27:40.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chmod/c-option' 2024-03-09T09:27:40.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chmod/silent' 2024-03-09T09:27:40.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chmod/setgid' 2024-03-09T09:27:40.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/chmod' 2024-03-09T09:27:40.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rmdir/t-slash' 2024-03-09T09:27:40.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rmdir/fail-perm' 2024-03-09T09:27:40.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rmdir/ignore' 2024-03-09T09:27:40.858 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/rmdir' 2024-03-09T09:27:40.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chgrp/posix-H' 2024-03-09T09:27:40.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chgrp/default-no-deref' 2024-03-09T09:27:40.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chgrp/no-x' 2024-03-09T09:27:40.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chgrp/basic' 2024-03-09T09:27:40.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chgrp/deref' 2024-03-09T09:27:40.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chgrp/recurse' 2024-03-09T09:27:40.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/chgrp' 2024-03-09T09:27:40.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/envvar-check' 2024-03-09T09:27:40.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chown/basic' 2024-03-09T09:27:40.859 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chown/deref' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chown/separator' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/chown/preserve-root' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/chown' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/acl' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/i-2' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/dup-source' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/to-symlink' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/childproof' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/i-5' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/mv-n' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/part-hardlink' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/dir2dir' 2024-03-09T09:27:40.860 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/part-rename' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/into-self-3' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/hard-2' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/trailing-slash' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/hard-verbose' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/force' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/i-link-no' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/i-3' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/backup-dir' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/i-4' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/into-self' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/diag' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/into-self-2' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/mv-special-1' 2024-03-09T09:27:40.861 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/update' 2024-03-09T09:27:40.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/atomic2' 2024-03-09T09:27:40.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/no-target-dir' 2024-03-09T09:27:40.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/hard-4' 2024-03-09T09:27:40.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/sticky-to-xpart' 2024-03-09T09:27:40.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/dir-file' 2024-03-09T09:27:40.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/i-1' 2024-03-09T09:27:40.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/hard-3' 2024-03-09T09:27:40.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/part-symlink' 2024-03-09T09:27:40.862 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/hard-link-1' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/backup-is-src' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/perm-1' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/partition-perm' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/part-fail' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/leak-fd' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/atomic' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mv/into-self-4' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/mv' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/multihardlink' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/stat-failed' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/stat-free-symlinks' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/color-clear-to-eol' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/rt-1' 2024-03-09T09:27:40.863 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/color-dtype-dir' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/symlink-slash' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/infloop' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/follow-slink' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/no-arg' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/inode' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/color-norm' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/no-cap' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/x-option' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/proc-selinux-segfault' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/abmon-align' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/file-type' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/stat-vs-dirent' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/nameless-uid' 2024-03-09T09:27:40.864 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/dangle' 2024-03-09T09:27:40.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/readdir-mountpoint-inode' 2024-03-09T09:27:40.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/stat-dtype' 2024-03-09T09:27:40.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/dired' 2024-03-09T09:27:40.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/capability' 2024-03-09T09:27:40.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/m-option' 2024-03-09T09:27:40.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ls/recursive' 2024-03-09T09:27:40.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/ls' 2024-03-09T09:27:40.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/p-slashdot' 2024-03-09T09:27:40.865 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/t-slash' 2024-03-09T09:27:42.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/perm' 2024-03-09T09:27:42.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/p-thru-slink' 2024-03-09T09:27:42.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/p-3' 2024-03-09T09:27:42.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/p-2' 2024-03-09T09:27:42.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/writable-under-readonly' 2024-03-09T09:27:42.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/selinux' 2024-03-09T09:27:42.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/p-1' 2024-03-09T09:27:42.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/p-v' 2024-03-09T09:27:42.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/parents' 2024-03-09T09:27:42.116 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/mkdir/special-1' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/mkdir' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/install/create-leading' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/install/install-C-root' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/install/install-C' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/install/basic-1' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/install/install-C-selinux' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/install/d-slashdot' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/install/strip-program' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/install/trap' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/install' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/readlink/rl-1' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/readlink/can-m' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/readlink/can-f' 2024-03-09T09:27:42.117 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/readlink/can-e' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/readlink' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/2g' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/8gb' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/restore-wd' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/long-from-unreadable' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/two-args' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/slink' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/inaccessible-cwd' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/no-x' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/fd-leak' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/basic' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/one-file-system' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/trailing-slash' 2024-03-09T09:27:42.118 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/no-deref' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/long-sloop' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/inacc-dir' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/deref' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/inacc-dest' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/files0-from' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/slash' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/exclude' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/hard-link' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/du/deref-args' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/du' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/id/no-context' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/id' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/not-owner' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/fifo' 2024-03-09T09:27:42.119 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/obsolescent' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/dangling-symlink' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/trailing-slash' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/no-create-missing' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/no-rights' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/relative' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/no-dereference' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/empty-file' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/dir-1' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/60-seconds' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/now-owned-by-other' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/fail-diag' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/touch/read-only' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/touch' 2024-03-09T09:27:42.120 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/flush-initial' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/F-vs-rename' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/assert-2' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/inotify-rotate' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/proc-ksyms' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/infloop-1' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/start-middle' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/inotify-hash-abuse' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/pipe-f2' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/tail-n0f' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/append-only' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/pipe-f' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/wait' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/assert' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/inotify-race' 2024-03-09T09:27:42.121 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/follow-stdin' 2024-03-09T09:27:42.122 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/inotify-hash-abuse2' 2024-03-09T09:27:42.122 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/big-4gb' 2024-03-09T09:27:42.122 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/tail-2/pid' 2024-03-09T09:27:42.122 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/tail-2' 2024-03-09T09:27:42.122 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/lang-default' 2024-03-09T09:27:42.122 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/Makefile.in' 2024-03-09T09:27:42.122 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/Coreutils.pm' 2024-03-09T09:27:42.122 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/Makefile.am' 2024-03-09T09:27:42.705 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sort' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/chroot-credentials' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/base64' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/test-diag' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/od-N' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/invalid-opt' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sha224sum' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/split-fail' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/tail' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/shred-exact' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/df' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/seq' 2024-03-09T09:27:42.706 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/truncate-no-create-missing' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sha256sum' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sha1sum' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/nl' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/fmt' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/stty-row-col' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/close-stdout' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/nice-fail' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/truncate-overflow' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sort-compress' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/false-status' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/test' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sort-files0-from' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/df-P' 2024-03-09T09:27:42.707 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/md5sum-parallel' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/cut' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sort-merge' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/csplit' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/wc-files0-from' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/truncate-fifo' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sha512sum' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/head-c' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/join' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/sha384sum' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/cat-buf' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/printf-cov' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/head-pos' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/stty' 2024-03-09T09:27:42.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/misc/pwd-unreadable-parent' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests/misc' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/other-fs-tmpdir' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/tests/sample-test' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/tests' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/doc/getdate.texi' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/doc/perm.texi' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/doc/coreutils.info' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/doc/stamp-vti' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/doc/ChangeLog-2007' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/doc/Makefile.in' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/doc/constants.texi' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/doc/Makefile.am' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/doc/fdl.texi' 2024-03-09T09:27:42.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/doc/coreutils.texi' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/doc/version.texi' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/doc' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_prohibit_tab_based_indentation' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gl/modules/getloadavg.diff' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/gl/modules' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/gl' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_prohibit_stat_st_blocks' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/THANKS' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/hash.h' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fts.c' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/full-read.c' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/error.h' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/idcache.h' 2024-03-09T09:27:42.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/faccessat.c' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fcntl.in.h' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/lchown.c' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sys_stat.in.h' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/openat-safer.c' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstrtol.c' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/verror.c' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/root-dev-ino.c' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstrtod.c' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/printf-args.h' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/vasprintf.c' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/randint.c' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/progname.c' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/progname.h' 2024-03-09T09:27:42.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/se-context.in.h' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv_open.c' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/file-has-acl.c' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/trim.h' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbswidth.h' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/isnan.c' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/euidaccess.c' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/physmem.h' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/areadlink.c' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbuiter.h' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xmemxfrm.h' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fd-reopen.c' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/rand-isaac.h' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/savewd.c' 2024-03-09T09:27:42.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xfts.c' 2024-03-09T09:27:43.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/hash' 2024-03-09T09:27:43.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/rm1' 2024-03-09T09:27:43.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/empty-inacc' 2024-03-09T09:27:43.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/r-3' 2024-03-09T09:27:43.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/one-file-system2' 2024-03-09T09:27:43.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/readdir-bug' 2024-03-09T09:27:43.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/r-1' 2024-03-09T09:27:43.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/f-1' 2024-03-09T09:27:43.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/i-no-r' 2024-03-09T09:27:43.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/ir-1' 2024-03-09T09:27:43.108 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/fail-eacces' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/i-never' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/r-4' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/dir-no-w' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/no-give-up' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/dangling-symlink' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/unreadable' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/dot-rel' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/ignorable' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/one-file-system' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/interactive-once' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/unread2' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/inaccessible' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/isatty' 2024-03-09T09:27:43.109 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/rm5' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/interactive-always' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/unread3' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/fail-eperm' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/dir-nonrecur' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/empty-name' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/rm4' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/cycle' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/v-slash' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/deep-1' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/rm2' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/rm3' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/sunos-1' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/deep-2' 2024-03-09T09:27:43.110 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/i-1' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/r-2' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/ext3-perf' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/fail-2eperm' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/rm/read-only' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/rm' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/check.mk' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/pr-tests' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/o3ml24f-bl-tn' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+2l24f-bl' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/b2l17-FF' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+3ml24f-tn-bl' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/l24f-t' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3b3l15f-t' 2024-03-09T09:27:43.111 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/4-7l24-FF' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+6a2l17-FF' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/nl24f-bl' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/nSml24-bl-FF' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/a3l24f-lm' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+3ml20f-bl-FF' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3b3f-0F' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/FFn' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W27l24f-ll' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tne8o3-t_tab' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/a2l17-FF' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/a2l15-FF' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ml24f-0F' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W72Jl24f-ll' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/o3b3Snl24f-tn' 2024-03-09T09:27:43.112 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/nSml24-t-tFFFF' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W26l24f-ll' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3b3l15-t' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ml24-t-FF' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ttb3-0FF' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+4-8a2l17-FF' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tFFt-ll' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/b3-0F' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+3-7l24-FF' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tt-0FF' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/Ja3l24f-lm' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/w72l24f-ll' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W35b3l24f-lm' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/0FF' 2024-03-09T09:27:43.113 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/0Ft' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/2-S_f-t_notab' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/2sw60f-t_notab' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3b2l17-FF' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tFFt-lm' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3-FF' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/t-t' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/Jml24f-lm-lo' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/nN1+3l24f-bl' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/0FFnt' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tta3-0FF' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3-0F' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+6b3l13f-FF' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3ml24-t' 2024-03-09T09:27:43.114 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/t-0FF' 2024-03-09T09:27:43.457 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getdate.c' 2024-03-09T09:27:43.457 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/diacrit.h' 2024-03-09T09:27:43.457 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xmemcoll.h' 2024-03-09T09:27:43.457 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/isnanf.c' 2024-03-09T09:27:43.457 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/savewd.h' 2024-03-09T09:27:43.457 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/gethostname.c' 2024-03-09T09:27:43.457 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/alloca.in.h' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getdelim.c' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mpsort.c' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/config.hin' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/close-stream.h' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/savedir.h' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/hash-triple.h' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/trim.c' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/physmem.c' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fchdir.c' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stdarg.in.h' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strintcmp.c' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stpncpy.c' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xalloc.h' 2024-03-09T09:27:43.458 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/timespec.h' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/readtokens.h' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fsusage.h' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/linebuffer.c' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memcoll.h' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/string.in.h' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/quote.h' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strstr.c' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/realloc.c' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stdlib-safer.h' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/userspec.c' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/siglist.h' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/safe-read.h' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/filenamecat.c' 2024-03-09T09:27:43.459 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbrtowc.c' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getndelim2.h' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/argv-iter.h' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/openat-proc.c' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stdio-safer.h' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv_open-osf.gperf' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getugroups.h' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stdio.in.h' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memcmp.c' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/tempname.c' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/inttostr.h' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/yesno.c' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/full-write.c' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strnumcmp.c' 2024-03-09T09:27:43.460 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sys_socket.in.h' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/arpa_inet.in.h' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unlocked-io.h' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unitypes.h' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/canon-host.h' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/euidaccess-stat.c' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/isapipe.c' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/quotearg.h' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/ref-add.sin' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstriconv.h' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fprintftime.h' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/c-ctype.c' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/float.in.h' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/cloexec.c' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/offtostr.c' 2024-03-09T09:27:43.461 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv_open-aix.gperf' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/wcwidth.c' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/nproc.c' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/gai_strerror.c' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fcntl.c' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getndelim2.c' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/w32sock.h' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/printf-frexpl.h' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getopt.in.h' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/asprintf.c' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/filemode.c' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strndup.c' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/uniwidth/cjk.h' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/uniwidth/width.c' 2024-03-09T09:27:43.462 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/lib/uniwidth' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fseterr.h' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/pipe-safer.c' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/freading.c' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/diacrit.c' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/quotearg.c' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/randint.h' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/inttypes.in.h' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/regex_internal.h' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dirent--.h' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stdio-write.c' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/hash.c' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/atexit.c' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stat.c' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv_open-aix.h' 2024-03-09T09:27:43.463 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/posixtm.c' 2024-03-09T09:27:43.464 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/obstack.c' 2024-03-09T09:27:43.464 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/readutmp.h' 2024-03-09T09:27:43.464 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sig2str.h' 2024-03-09T09:27:43.464 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/str-two-way.h' 2024-03-09T09:27:43.464 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/copy-acl.c' 2024-03-09T09:27:43.464 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/canonicalize.c' 2024-03-09T09:27:43.464 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/userspec.h' 2024-03-09T09:27:43.464 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbsalign.h' 2024-03-09T09:27:43.464 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/wchar.in.h' 2024-03-09T09:27:43.464 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/filevercmp.h' 2024-03-09T09:27:44.463 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/o3Jml24f-lm-lo' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tn_2e8-t_tab' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+8l20-FF' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W35Ja3l24f-lm' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ml24-t' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W35ml24f-lm-lo' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W-72l24f-ll' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/o3mSl24f-bl-tn' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/t-FF' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3ml24-t-FF' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3b3f-FF' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tta3-FF' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W35Jml24f-lmlo' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tFnFt' 2024-03-09T09:27:44.464 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/0FnFnt' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/0FnFt' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/nSml24-t-t-FF' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/b3f-0FF' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tn' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tt-FF' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W28l24f-ll' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tb3-FF' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tn2e8-t_tab' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/l24-FF' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+4b2l17f-0FF' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3a3f-0F' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/FFtn' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/a3f-FF' 2024-03-09T09:27:44.465 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+2l24f-0FF' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/2s_f-t_notab' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/a3l15-t' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3b3f-0FF' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/nJml24f-lmlolm' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ttb3-FF' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/loli' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/b3l15-t' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3-5l24f-t' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ta3-0FF' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/0F' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/FF' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ml24f-t' 2024-03-09T09:27:44.466 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/t_tab_' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/o3b3l24f-tn' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/4l24-FF' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/o3mSnl24fbltn' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tFFn' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+5-8b3l17f-FF' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/nSml20-bl-FF' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/o3a3Sl24f-tn' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/nJml24f-lmlmlo' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W35a3l24f-lm' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/nN15l24f-bl' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/a3f-0F' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/b3f-0F' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/t_notab' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+5a3l13f-0FF' 2024-03-09T09:27:44.467 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/nSml20-t-t-FF' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tn2e5o3-t_tab' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ml24f-lm-lo' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3f-FF' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/t-bl' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3ml24f-t' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/b3f-FF' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/FnFn' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+3ml24f-bl-tn' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/0FFt' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tb3-0FF' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tt-t' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/2-Sf-t_notab' 2024-03-09T09:27:44.468 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ta3-FF' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W20l24f-ll' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tn_2e8S-t_tab' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/b2l15-FF' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/l24-t' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3l24f-t' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tn2e8o3-t_tab' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/a3f-0FF' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/a3l15f-t' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/Jb3l24f-lm' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3a2l17-FF' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3f-0F' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/t_tab' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3l24-t' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/W35Jb3l24f-lm' 2024-03-09T09:27:44.469 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tFFt-bl' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tFFt' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+2-5l24f-0FF' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+3l24f-bl' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+7l24-FF' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3a3l15-t' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ml24f-t-0F' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ml24-FF' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/0Fnt' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/2f-t_notab' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/2s_w60f-t_nota' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/ml20-FF-t' 2024-03-09T09:27:44.470 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tt-bl' 2024-03-09T09:27:45.185 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/regex_internal.c' 2024-03-09T09:27:45.185 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memcpy.c' 2024-03-09T09:27:45.185 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/base64.c' 2024-03-09T09:27:45.185 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/exclude.h' 2024-03-09T09:27:45.185 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/chown.c' 2024-03-09T09:27:45.185 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/alignof.h' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/signbitd.c' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/at-func.c' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/asnprintf.c' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sig2str.c' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mkdir.c' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/savedir.c' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strtoull.c' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/cycle-check.c' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/version-etc.h' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mkancesdirs.c' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unistr/u8-uctomb-aux.c' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unistr/u8-uctomb.c' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/lib/unistr' 2024-03-09T09:27:45.186 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/priv-set.h' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/time_r.c' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/at-func2.c' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv_open-solaris.h' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/freopen-safer.c' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xnanosleep.h' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/verify.h' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/vasnprintf.c' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getgroups.c' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/errno.in.h' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/long-options.h' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/posixver.c' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/netdb.in.h' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stdlib--.h' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/same.h' 2024-03-09T09:27:45.187 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/gethrxtime.h' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/version-etc-fsf.c' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/printf-frexp.h' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/linebuffer.h' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xnanosleep.c' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/openat.c' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dup2.c' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/obstack.h' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/safe-read.c' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memchr2.valgrind' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/putenv.c' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/ctype.in.h' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unlinkat.c' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/gettime.c' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sha1.h' 2024-03-09T09:27:45.188 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/yesno.h' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/regexec.c' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/closeout.c' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/str-kmp.h' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getpass.h' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/cloexec.h' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv_open-hpux.h' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/close-hook.c' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fchmodat.c' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sig-handler.h' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/glthread/tls.h' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/glthread/tls.c' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/glthread/threadlib.c' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/glthread/lock.c' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/glthread/lock.h' 2024-03-09T09:27:45.189 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/lib/glthread' 2024-03-09T09:27:45.190 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getopt.c' 2024-03-09T09:27:45.190 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/readtokens0.c' 2024-03-09T09:27:45.190 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/gnulib.mk' 2024-03-09T09:27:45.190 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strtoimax.c' 2024-03-09T09:27:45.190 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/se-selinux.in.h' 2024-03-09T09:27:45.190 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/argv-iter.c' 2024-03-09T09:27:45.190 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/canonicalize.h' 2024-03-09T09:27:45.190 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fpending.c' 2024-03-09T09:27:45.190 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/openat-priv.h' 2024-03-09T09:27:45.190 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/closeout.h' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unistd--.h' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/close.c' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memchr.valgrind' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/ignore-value.h' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getcwd.c' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mkancesdirs.h' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbrlen.c' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/randread.c' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dirname.c' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/chdir-long.h' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/areadlink.h' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stat-time.h' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/save-cwd.h' 2024-03-09T09:27:45.191 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbsstr.c' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/open-safer.c' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mkdirat.c' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xgethostname.h' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/langinfo.in.h' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/rmdir.c' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fopen.c' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/close-stream.c' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unicodeio.c' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv_open-hpux.gperf' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/wctype.in.h' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbsinit.c' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/streq.h' 2024-03-09T09:27:45.192 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/open.c' 2024-03-09T09:27:45.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/freopen.c' 2024-03-09T09:27:45.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/posixtm.h' 2024-03-09T09:27:45.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dirfd.c' 2024-03-09T09:27:45.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fcntl-safer.h' 2024-03-09T09:27:45.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/isapipe.h' 2024-03-09T09:27:45.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstrtold.c' 2024-03-09T09:27:45.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mgetgroups.h' 2024-03-09T09:27:45.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/quote.c' 2024-03-09T09:27:45.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strcspn.c' 2024-03-09T09:27:45.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/ChangeLog-2007' 2024-03-09T09:27:45.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mpsort.h' 2024-03-09T09:27:45.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/inttostr.c' 2024-03-09T09:27:45.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stpcpy.c' 2024-03-09T09:27:45.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/umaxtostr.c' 2024-03-09T09:27:45.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/printf-frexp.c' 2024-03-09T09:27:45.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/buffer-lcm.h' 2024-03-09T09:27:45.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/regcomp.c' 2024-03-09T09:27:45.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/openat-die.c' 2024-03-09T09:27:45.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/tempname.h' 2024-03-09T09:27:45.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getdtablesize.c' 2024-03-09T09:27:45.496 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strsignal.c' 2024-03-09T09:27:45.496 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getfilecon.c' 2024-03-09T09:27:45.496 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/utimens.h' 2024-03-09T09:27:45.496 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/snprintf.c' 2024-03-09T09:27:45.496 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fts-cycle.c' 2024-03-09T09:27:45.496 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fpending.h' 2024-03-09T09:27:45.497 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/group-member.h' 2024-03-09T09:27:45.497 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstrtod.h' 2024-03-09T09:27:45.497 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getusershell.c' 2024-03-09T09:27:45.497 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/ref-del.sin' 2024-03-09T09:27:45.497 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/frexp.c' 2024-03-09T09:27:45.498 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getpagesize.c' 2024-03-09T09:27:45.498 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dirname.h' 2024-03-09T09:27:45.498 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sockets.c' 2024-03-09T09:27:45.498 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getopt_int.h' 2024-03-09T09:27:45.498 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/freadahead.h' 2024-03-09T09:27:45.498 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strftime.h' 2024-03-09T09:27:45.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/close-hook.h' 2024-03-09T09:27:45.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/raise.c' 2024-03-09T09:27:45.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fpucw.h' 2024-03-09T09:27:45.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/localcharset.c' 2024-03-09T09:27:45.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/lstat.c' 2024-03-09T09:27:45.499 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/rename.c' 2024-03-09T09:27:45.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/freadahead.c' 2024-03-09T09:27:45.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/argmatch.h' 2024-03-09T09:27:45.500 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fchown-stub.c' 2024-03-09T09:27:45.501 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/creat-safer.c' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strtoul.c' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xprintf.c' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mountlist.c' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mkdir-p.c' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/utimecmp.c' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mountlist.h' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/isnanl.c' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/randperm.h' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/remove.c' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dirent.in.h' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/malloc.c' 2024-03-09T09:27:45.502 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memrchr.c' 2024-03-09T09:27:45.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dirchownmod.h' 2024-03-09T09:27:45.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/argmatch.c' 2024-03-09T09:27:45.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/Makefile.in' 2024-03-09T09:27:45.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strnumcmp.h' 2024-03-09T09:27:45.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/c-strtold.c' 2024-03-09T09:27:45.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/freadptr.c' 2024-03-09T09:27:45.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/version-etc.c' 2024-03-09T09:27:45.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/file-type.h' 2024-03-09T09:27:45.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/same-inode.h' 2024-03-09T09:27:45.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xvasprintf.h' 2024-03-09T09:27:45.503 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/i-ring.c' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/c-strncasecmp.c' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/posixver.h' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/locale.in.h' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unistr.h' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xreadlink.c' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/human.c' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sha512.c' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv_open-solaris.gperf' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fdopendir.c' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/regex.c' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getaddrinfo.c' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/acl_entries.c' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getopt1.c' 2024-03-09T09:27:45.504 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/selinux-at.h' 2024-03-09T09:27:45.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strnlen1.c' 2024-03-09T09:27:45.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/readtokens0.h' 2024-03-09T09:27:45.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/symlink.c' 2024-03-09T09:27:45.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fclose.c' 2024-03-09T09:27:45.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/freadseek.h' 2024-03-09T09:27:45.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/safe-write.h' 2024-03-09T09:27:45.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fsusage.c' 2024-03-09T09:27:45.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/basename-lgpl.c' 2024-03-09T09:27:45.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/nproc.h' 2024-03-09T09:27:45.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stat-macros.h' 2024-03-09T09:27:45.505 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstrtoimax.c' 2024-03-09T09:27:45.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xmemcoll.c' 2024-03-09T09:27:45.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dirent-safer.h' 2024-03-09T09:27:45.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbchar.c' 2024-03-09T09:27:45.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memchr2.h' 2024-03-09T09:27:45.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strdup.c' 2024-03-09T09:27:45.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strnlen.c' 2024-03-09T09:27:45.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/c-strtod.h' 2024-03-09T09:27:45.506 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getloadavg.c' 2024-03-09T09:27:45.528 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3a3l15f-t' 2024-03-09T09:27:45.528 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/o3a3l24f-tn' 2024-03-09T09:27:45.528 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/o3b3Sl24f-tn' 2024-03-09T09:27:45.528 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/tne8-t_tab' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/a3-0F' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/nSml20-t-tFFFF' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/b3l24f-lm' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/2w60f-t_notab' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/2sf-t_notab' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/b3l15f-t' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/3ml24-FF' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/n+3l24f-0FF' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/pr/o3a3Snl24f-tn' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/pr' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/CuTmpdir.pm' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/df/unreadable' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/df/total-verify' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/df' 2024-03-09T09:27:45.529 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ln/sf-1' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ln/hard-backup' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ln/slash-decorated-nonexistent-dest' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ln/hard-to-sym' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ln/target-1' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ln/backup-1' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/ln/misc' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/ln' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/cp-mv-backup' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/acl' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/perm' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/cp-deref' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/special-bits' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/link' 2024-03-09T09:27:45.530 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/preserve-slink-time' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/symlink-slash' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/cp-mv-enotsup-xattr' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/thru-dangling' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/reflink-perm' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/r-vs-symlink' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/preserve-2' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/special-f' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/file-perm-race' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/slink-2-slink' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/proc-zero-len' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/reflink-auto' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/src-base-dot' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/link-preserve' 2024-03-09T09:27:45.531 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/cp-i' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/cp-a-selinux' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/preserve-gid' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/no-deref-link3' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/backup-dir' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/into-self' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/same-file' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/cp-HL' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/link-no-deref' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/parent-perm-race' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/proc-short-read' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/no-deref-link1' 2024-03-09T09:27:45.532 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/parent-perm' 2024-03-09T09:27:45.533 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/dir-vs-file' 2024-03-09T09:27:45.533 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/sparse' 2024-03-09T09:27:45.533 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/link-heap' 2024-03-09T09:27:45.533 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/no-deref-link2' 2024-03-09T09:27:45.533 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/dir-rm-dest' 2024-03-09T09:27:45.533 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/dir-slash' 2024-03-09T09:27:45.533 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/fail-perm' 2024-03-09T09:27:45.533 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/deref-slink' 2024-03-09T09:27:45.533 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/existing-perm-race' 2024-03-09T09:27:45.533 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/abuse' 2024-03-09T09:27:45.533 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/cp-parents' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/backup-1' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/backup-is-src' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/cp/capability' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/cp' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/dd/not-rewound' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/dd/skip-seek2' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/dd/unblock' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/dd/skip-seek-past-dev' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/dd/direct' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/dd/unblock-sync' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/dd/skip-seek-past-file' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/dd/stderr' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/dd/reblock' 2024-03-09T09:27:45.534 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/dd/misc' 2024-03-09T09:27:45.535 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/dd/skip-seek' 2024-03-09T09:27:45.535 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/dd' 2024-03-09T09:27:45.535 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/fold' 2024-03-09T09:27:45.535 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/tty-eof' 2024-03-09T09:27:45.535 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sort-rand' 2024-03-09T09:27:45.535 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/seq-long-double' 2024-03-09T09:27:45.535 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/shred-remove' 2024-03-09T09:27:45.846 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/nice' 2024-03-09T09:27:45.846 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/tee' 2024-03-09T09:27:45.846 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/truncate-owned-by-other' 2024-03-09T09:27:45.846 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/mknod' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/expand' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/timeout-parameters' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sum' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/readlink-fp-loop' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/printenv' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/nohup' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/printf-hex' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/ptx' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/wc-files0' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/tac-continue' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sort-merge-fdlimit' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/truncate-dir-fail' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/env-null' 2024-03-09T09:27:45.847 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/xstrtol' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/tr' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/paste' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/printf' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/comm' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/id-context' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/chcon' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/shred-passes' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/stat-fmt' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/truncate-parameters' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/date-sec' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/tsort' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sha1sum-vec' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/stat-hyphen' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/od-multiple-t' 2024-03-09T09:27:45.848 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/xattr' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/od' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sum-sysv' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/date-next-dow' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/nproc-avail' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/split-a' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/unexpand' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/pathchk1' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/md5sum' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/groups-version' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/head' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/stty-invalid' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/factor' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/timeout' 2024-03-09T09:27:45.849 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/su-fail' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/wc-parallel' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/md5sum-newline' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/tee-dash' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/runcon-no-reorder' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/dircolors' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/basename' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/selinux' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/head-elide-tail' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/help-version' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/stdbuf' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/expr' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/groups-dash' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/shuf' 2024-03-09T09:27:45.850 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sort-version' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/pwd-long' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/env' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sort-continue' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/od-x8' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/arch' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/chcon-fail' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/tac' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/chroot-fail' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/cat-proc' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/truncate-fail-diag' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/ptx-overrun' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sort-month' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/date' 2024-03-09T09:27:45.851 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/ls-time' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/stat-printf' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/printf-surprise' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/pwd-option' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/id-groups' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/stat-slash' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/ls-misc' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/fmt-long-line' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/split-l' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/uniq' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/truncate-relative' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/mktemp' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/nproc-positive' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/truncate-dangling-symlink' 2024-03-09T09:27:45.852 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/dirname' 2024-03-09T09:27:45.853 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/wc' 2024-03-09T09:27:45.853 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/pr' 2024-03-09T09:27:46.138 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/file-type.c' 2024-03-09T09:27:46.138 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/i-ring.h' 2024-03-09T09:27:46.138 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fstatat.c' 2024-03-09T09:27:46.138 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sha512.h' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/gettimeofday.c' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbsrtowcs-state.c' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xmalloc.c' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memcoll.c' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstrtoul.c' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fts_.h' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/c-ctype.h' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stripslash.c' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv_open-osf.h' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/wcrtomb.c' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/utimecmp.h' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stdio--.h' 2024-03-09T09:27:46.139 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/randperm.c' 2024-03-09T09:27:46.140 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/areadlinkat.c' 2024-03-09T09:27:46.140 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/readutmp.c' 2024-03-09T09:27:46.140 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unicodeio.h' 2024-03-09T09:27:46.140 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/full-read.h' 2024-03-09T09:27:46.140 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fseterr.c' 2024-03-09T09:27:46.140 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/intprops.h' 2024-03-09T09:27:46.140 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/pathmax.h' 2024-03-09T09:27:46.140 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/utimens.c' 2024-03-09T09:27:46.140 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/propername.h' 2024-03-09T09:27:46.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/acl-internal.h' 2024-03-09T09:27:46.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/safe-write.c' 2024-03-09T09:27:46.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strtoll.c' 2024-03-09T09:27:46.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mkstemp.c' 2024-03-09T09:27:46.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/Makefile.am' 2024-03-09T09:27:46.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/file-set.h' 2024-03-09T09:27:46.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/md5.c' 2024-03-09T09:27:46.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/base64.h' 2024-03-09T09:27:46.141 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv_open-irix.gperf' 2024-03-09T09:27:46.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/vasnprintf.h' 2024-03-09T09:27:46.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstrndup.h' 2024-03-09T09:27:46.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/hash-pjw.c' 2024-03-09T09:27:46.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sys_time.in.h' 2024-03-09T09:27:46.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strtol.c' 2024-03-09T09:27:46.142 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/backupfile.h' 2024-03-09T09:27:46.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/setenv.c' 2024-03-09T09:27:46.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sys_select.in.h' 2024-03-09T09:27:46.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/config.charset' 2024-03-09T09:27:46.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xgetcwd.h' 2024-03-09T09:27:46.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fnmatch_loop.c' 2024-03-09T09:27:46.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/hash-triple.c' 2024-03-09T09:27:46.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/hash-pjw.h' 2024-03-09T09:27:46.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memcmp2.c' 2024-03-09T09:27:46.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/write.c' 2024-03-09T09:27:46.143 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/modechange.h' 2024-03-09T09:27:46.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unlink.c' 2024-03-09T09:27:46.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stdbool.in.h' 2024-03-09T09:27:46.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/time.in.h' 2024-03-09T09:27:46.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/freadptr.h' 2024-03-09T09:27:46.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/isblank.c' 2024-03-09T09:27:46.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getline.c' 2024-03-09T09:27:46.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/math.in.h' 2024-03-09T09:27:46.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/buffer-lcm.c' 2024-03-09T09:27:46.144 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/exclude.c' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/filevercmp.c' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/closein.c' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/closein.h' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xfts.h' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/file-set.c' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unistd-safer.h' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/uniwidth.h' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/symlinkat.c' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/readtokens.c' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/exitfail.h' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/malloca.c' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fd-safer.c' 2024-03-09T09:27:46.145 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xsize.h' 2024-03-09T09:27:46.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strnumcmp-in.h' 2024-03-09T09:27:46.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbswidth.c' 2024-03-09T09:27:46.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fopen-safer.c' 2024-03-09T09:27:46.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstrndup.c' 2024-03-09T09:27:46.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/human.h' 2024-03-09T09:27:46.146 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/gettext.h' 2024-03-09T09:27:46.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mktime-internal.h' 2024-03-09T09:27:46.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fchownat.c' 2024-03-09T09:27:46.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dup-safer.c' 2024-03-09T09:27:46.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstrtol-error.c' 2024-03-09T09:27:46.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/full-write.h' 2024-03-09T09:27:46.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dup-safer-flag.c' 2024-03-09T09:27:46.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/float+.h' 2024-03-09T09:27:46.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sigprocmask.c' 2024-03-09T09:27:46.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/printf-args.c' 2024-03-09T09:27:46.147 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/gethrxtime.c' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/randread.h' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strpbrk.c' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getugroups.c' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/chdir-long.c' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fcntl--.h' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mkstemp-safer.c' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbslen.c' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/bitrotate.h' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dev-ino.h' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xasprintf.c' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sockets.h' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/inet_ntop.c' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getdate.y' 2024-03-09T09:27:46.148 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stddef.in.h' 2024-03-09T09:27:46.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/isnand-nolibm.h' 2024-03-09T09:27:46.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbsrtowcs.c' 2024-03-09T09:27:46.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/cycle-check.h' 2024-03-09T09:27:46.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sha256.c' 2024-03-09T09:27:46.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/readlink.c' 2024-03-09T09:27:46.366 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/uname.c' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/printf-parse.h' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/printf-frexpl.c' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/exitfail.c' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/linkat.c' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/same.c' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/btowc.c' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memcasecmp.h' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sys_utsname.in.h' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/filenamecat-lgpl.c' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstriconv.c' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/rand-isaac.c' 2024-03-09T09:27:46.367 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstrtol.h' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strtod.c' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/select.c' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/ftello.c' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strnlen1.h' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xtime.h' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/save-cwd.c' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sha256.h' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memxfrm.c' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/opendir-safer.c' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/size_max.h' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbscasecmp.c' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fflush.c' 2024-03-09T09:27:46.368 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mgetgroups.c' 2024-03-09T09:27:46.369 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getdate.h' 2024-03-09T09:27:46.369 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/malloca.valgrind' 2024-03-09T09:27:46.369 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv_open-irix.h' 2024-03-09T09:27:46.369 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xgetcwd.c' 2024-03-09T09:27:46.369 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xalloc-die.c' 2024-03-09T09:27:46.369 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/long-options.c' 2024-03-09T09:27:46.369 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/basename.c' 2024-03-09T09:27:46.369 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/error.c' 2024-03-09T09:27:46.369 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memcmp2.h' 2024-03-09T09:27:46.369 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/regex.h' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mkdir-p.h' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xstrtoumax.c' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dirchownmod.c' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/isnanl-nolibm.h' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/areadlink-with-size.c' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/printf-parse.c' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unistd.in.h' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strtoumax.c' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mempcpy.c' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xfreopen.c' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memmove.c' 2024-03-09T09:27:46.370 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xmemxfrm.c' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/link.c' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/priv-set.c' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/signbitl.c' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fnmatch.c' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/backupfile.c' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fd-safer-flag.c' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/signal.in.h' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/filenamecat.h' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xvasprintf.c' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/euidaccess-stat.h' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fsync.c' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/filemode.h' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/write-any-file.h' 2024-03-09T09:27:46.371 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stdlib.in.h' 2024-03-09T09:27:46.372 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/selinux-at.c' 2024-03-09T09:27:46.372 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/netinet_in.in.h' 2024-03-09T09:27:46.372 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/u64.h' 2024-03-09T09:27:46.372 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/unsetenv.c' 2024-03-09T09:27:46.372 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memchr.c' 2024-03-09T09:27:46.372 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/verror.h' 2024-03-09T09:27:46.372 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/freading.h' 2024-03-09T09:27:46.372 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xgethostname.c' 2024-03-09T09:27:46.372 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/nanosleep.c' 2024-03-09T09:27:46.372 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xprintf.h' 2024-03-09T09:27:46.373 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/malloca.h' 2024-03-09T09:27:46.373 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/settime.c' 2024-03-09T09:27:46.373 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memchr2.c' 2024-03-09T09:27:46.373 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memcasecmp.c' 2024-03-09T09:27:46.373 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/frexpl.c' 2024-03-09T09:27:46.373 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/root-dev-ino.h' 2024-03-09T09:27:46.373 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/hard-locale.c' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/localcharset.h' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stdio-impl.h' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/set-mode-acl.c' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/imaxtostr.c' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/freadseek.c' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/md5.h' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/idcache.c' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/calloc.c' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fseeko.c' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/propername.c' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/vprintf.c' 2024-03-09T09:27:46.374 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fileblocks.c' 2024-03-09T09:27:46.375 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memxfrm.h' 2024-03-09T09:27:46.375 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fprintftime.c' 2024-03-09T09:27:46.375 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/alloca.c' 2024-03-09T09:27:46.375 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/group-member.c' 2024-03-09T09:27:46.375 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/striconv.h' 2024-03-09T09:27:46.375 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbchar.h' 2024-03-09T09:27:46.375 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xreadlink.h' 2024-03-09T09:27:46.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/c-strcase.h' 2024-03-09T09:27:46.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/acl.h' 2024-03-09T09:27:46.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/modechange.c' 2024-03-09T09:27:46.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fpurge.c' 2024-03-09T09:27:46.708 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/openat.h' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/signbitf.c' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mktime.c' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbsalign.c' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/xfreopen.h' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/iconv.in.h' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/hard-locale.h' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/nl_langinfo.c' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/striconv.c' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sigaction.c' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/memset.c' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fd-reopen.h' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/mbiter.h' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strftime.c' 2024-03-09T09:27:46.709 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/uinttostr.c' 2024-03-09T09:27:46.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/canon-host.c' 2024-03-09T09:27:46.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/rpmatch.c' 2024-03-09T09:27:46.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/sha1.c' 2024-03-09T09:27:46.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/vfprintf.c' 2024-03-09T09:27:46.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/isnanf-nolibm.h' 2024-03-09T09:27:46.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/c-strtod.c' 2024-03-09T09:27:46.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/getpass.c' 2024-03-09T09:27:46.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/fnmatch.in.h' 2024-03-09T09:27:46.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/isnand.c' 2024-03-09T09:27:46.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/write-any-file.c' 2024-03-09T09:27:46.710 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/c-strcasecmp.c' 2024-03-09T09:27:46.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/stdint.in.h' 2024-03-09T09:27:46.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/dirname-lgpl.c' 2024-03-09T09:27:46.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/lseek.c' 2024-03-09T09:27:46.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/lib/strerror.c' 2024-03-09T09:27:46.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/lib' 2024-03-09T09:27:46.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.tarball-version' 2024-03-09T09:27:46.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_error_message_uppercase' 2024-03-09T09:27:46.711 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_space_tab' 2024-03-09T09:27:46.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/ChangeLog-2007' 2024-03-09T09:27:46.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_require_config_h_first' 2024-03-09T09:27:46.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/ABOUT-NLS' 2024-03-09T09:27:46.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/aclocal.m4' 2024-03-09T09:27:46.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/bootstrap' 2024-03-09T09:27:46.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/NEWS' 2024-03-09T09:27:46.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/ChangeLog-2006' 2024-03-09T09:27:46.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_unmarked_diagnostics' 2024-03-09T09:27:46.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/Makefile.in' 2024-03-09T09:27:46.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/ChangeLog-2005' 2024-03-09T09:27:46.712 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/timespec.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/xfts.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/jm-macros.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getugroups.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/unicodeio.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/lib-link.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stpcpy.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/math_h.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/printf.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/locale-fr.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sha1.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/safe-write.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/double-slash-root.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/close-stream.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/linkat.m4' 2024-03-09T09:27:46.713 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/nl_langinfo.m4' 2024-03-09T09:27:46.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/host-os.m4' 2024-03-09T09:27:46.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/priv-set.m4' 2024-03-09T09:27:46.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/write.m4' 2024-03-09T09:27:46.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/lchmod.m4' 2024-03-09T09:27:46.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/read-file.m4' 2024-03-09T09:27:46.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/netinet_in_h.m4' 2024-03-09T09:27:46.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/00gnulib.m4' 2024-03-09T09:27:46.714 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/utimbuf.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/euidaccess.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/cloexec.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/argmatch.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mountlist.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/string_h.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/utimens.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/warn-on-use.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/codeset.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/unlink.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fclose.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/vfprintf-posix.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/yield.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/localcharset.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/isnanf.m4' 2024-03-09T09:27:46.715 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/idcache.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/readlink.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mbrlen.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getgroups.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getopt.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getndelim2.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strnlen.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mgetgroups.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/signalblocking.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stat.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/gettext.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/uname.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/xnanosleep.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strtoimax.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/locale-zh.m4' 2024-03-09T09:27:46.716 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mode_t.m4' 2024-03-09T09:27:46.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/printf-frexp.m4' 2024-03-09T09:27:46.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strcspn.m4' 2024-03-09T09:27:46.717 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/wchar_h.m4' 2024-03-09T09:27:47.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/file-type.m4' 2024-03-09T09:27:47.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/usleep.m4' 2024-03-09T09:27:47.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stdlib-safer.m4' 2024-03-09T09:27:47.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/setenv.m4' 2024-03-09T09:27:47.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stat-prog.m4' 2024-03-09T09:27:47.380 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/write-any-file.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/yesno.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sys_ioctl_h.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/lchown.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/intlmacosx.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strsignal.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mbsinit.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/memcasecmp.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/freopen.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/iconv.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mbrtowc.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/ioctl.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sockpfaf.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/thread.m4' 2024-03-09T09:27:47.381 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fflush.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/assert.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stpncpy.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strftime.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/wcrtomb.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/langinfo_h.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/safe-read.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/clock_time.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/tls.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/memcoll.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/readutmp.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stdio_h.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/ftello.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/memcmp.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/canon-host.m4' 2024-03-09T09:27:47.382 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/gmp.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/environ.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fdopendir.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stdarg.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stdbool.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/intmax_t.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/isapipe.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/ldexpl.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/remove.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fpurge.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getusershell.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/memcpy.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fcntl.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/tempname.m4' 2024-03-09T09:27:47.383 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/xstrtol.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fseeko.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fnmatch.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/save-cwd.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/config-h.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/filemode.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/errno_h.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/eealloc.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strdup.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/signal_h.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/regex.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mbchar.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strtoull.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stdint.m4' 2024-03-09T09:27:47.384 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/wctype_h.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/realloc.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/printf-frexpl.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/open.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/inline.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/warnings.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/vasprintf-posix.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/xstrndup.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sys_utsname_h.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/locale-ja.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/snprintf.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/po.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/hash.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sig2str.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/ctype.m4' 2024-03-09T09:27:47.385 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strnumcmp.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/rpmatch.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/md5.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/boottime.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/unistd_h.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fd-reopen.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/cycle-check.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/ls-mntd-fs.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mkancesdirs.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/manywarnings.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/ChangeLog-2007' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getcwd-path-max.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/modechange.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stdlib_h.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/gethostname.m4' 2024-03-09T09:27:47.386 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/physmem.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fileblocks.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/socklen.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/st_dm_mode.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/ungetc.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/posixver.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getdtablesize.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/check-decl.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/threadlib.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sys_socket_h.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getaddrinfo.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/float_h.m4' 2024-03-09T09:27:47.387 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/btowc.m4' 2024-03-09T09:27:47.779 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sort' 2024-03-09T09:27:47.779 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/chroot-credentials' 2024-03-09T09:27:47.779 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/base64' 2024-03-09T09:27:47.779 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/test-diag' 2024-03-09T09:27:47.779 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/od-N' 2024-03-09T09:27:47.779 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/invalid-opt' 2024-03-09T09:27:47.779 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sha224sum' 2024-03-09T09:27:47.779 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/split-fail' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/tail' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/shred-exact' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/df' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/seq' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/truncate-no-create-missing' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sha256sum' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sha1sum' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/nl' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/fmt' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/stty-row-col' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/close-stdout' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/nice-fail' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/truncate-overflow' 2024-03-09T09:27:47.780 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sort-compress' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/false-status' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/test' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sort-files0-from' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/df-P' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/md5sum-parallel' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/cut' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sort-merge' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/csplit' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/wc-files0-from' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/truncate-fifo' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sha512sum' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/head-c' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/join' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/sha384sum' 2024-03-09T09:27:47.781 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/cat-buf' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/printf-cov' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/head-pos' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/stty' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/misc/pwd-unreadable-parent' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests/misc' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/other-fs-tmpdir' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/tests/sample-test' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/tests' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/doc/getdate.texi' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/doc/perm.texi' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/doc/coreutils.info' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/doc/stamp-vti' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/doc/ChangeLog-2007' 2024-03-09T09:27:47.782 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/doc/Makefile.in' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/doc/constants.texi' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/doc/Makefile.am' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/doc/fdl.texi' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/doc/coreutils.texi' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/doc/version.texi' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/doc' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_prohibit_tab_based_indentation' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gl/modules/getloadavg.diff' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/gl/modules' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/gl' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_prohibit_stat_st_blocks' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/THANKS' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/hash.h' 2024-03-09T09:27:47.783 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fts.c' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/full-read.c' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/error.h' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/idcache.h' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/faccessat.c' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fcntl.in.h' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/lchown.c' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sys_stat.in.h' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/openat-safer.c' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstrtol.c' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/verror.c' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/root-dev-ino.c' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstrtod.c' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/printf-args.h' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/vasprintf.c' 2024-03-09T09:27:47.784 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/randint.c' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/progname.c' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/progname.h' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/se-context.in.h' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv_open.c' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/file-has-acl.c' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/trim.h' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbswidth.h' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/isnan.c' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/euidaccess.c' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/physmem.h' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/areadlink.c' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbuiter.h' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xmemxfrm.h' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fd-reopen.c' 2024-03-09T09:27:47.785 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/rand-isaac.h' 2024-03-09T09:27:47.786 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/savewd.c' 2024-03-09T09:27:47.786 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xfts.c' 2024-03-09T09:27:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/userspec.m4' 2024-03-09T09:27:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/posixtm.m4' 2024-03-09T09:27:48.047 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/alloca.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/lstat.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/lock.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/readtokens.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/locale-tr.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/wctob.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/canonicalize.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/servent.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mbswidth.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/memrchr.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/autobuild.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strtol.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/closein.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sha256.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mbslen.m4' 2024-03-09T09:27:48.048 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/lseek.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/base64.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/iconv_h.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/longlong.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/dirfd.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/d-ino.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/ssize_t.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/gnulib-comp.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/memset.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getcwd-abort-bug.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/c-strtod.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fts.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/lib-prefix.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mmap-anon.m4' 2024-03-09T09:27:48.049 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/close.m4' 2024-03-09T09:27:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/lib-ignore.m4' 2024-03-09T09:27:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/gnulib-common.m4' 2024-03-09T09:27:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/time_h.m4' 2024-03-09T09:27:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/nls.m4' 2024-03-09T09:27:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/dirname.m4' 2024-03-09T09:27:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fcntl_h.m4' 2024-03-09T09:27:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stddef_h.m4' 2024-03-09T09:27:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mkstemp.m4' 2024-03-09T09:27:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/link-follow.m4' 2024-03-09T09:27:48.050 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/wcwidth.m4' 2024-03-09T09:27:48.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/memchr.m4' 2024-03-09T09:27:48.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strerror.m4' 2024-03-09T09:27:48.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/iconv_open.m4' 2024-03-09T09:27:48.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/exponentf.m4' 2024-03-09T09:27:48.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/pathmax.m4' 2024-03-09T09:27:48.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stdio-safer.m4' 2024-03-09T09:27:48.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/acl.m4' 2024-03-09T09:27:48.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fpending.m4' 2024-03-09T09:27:48.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/putenv.m4' 2024-03-09T09:27:48.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/include-exclude-prog.m4' 2024-03-09T09:27:48.051 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mktime.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getdate.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fsync.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/symlink.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/quote.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getloadavg.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/gettime.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/quotearg.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/inttypes.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/dos.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/gettimeofday.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/hostent.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/chdir-long.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/selinux-selinux-h.m4' 2024-03-09T09:27:48.052 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fcntl-safer.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/inttostr.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getline.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/posix-shell.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/d-type.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/inet_pton.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sigaction.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mkdir.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getdelim.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/xsize.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/symlinkat.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/dirent-safer.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/include_next.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getcwd.m4' 2024-03-09T09:27:48.053 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sys_time_h.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fprintftime.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/prereq.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/netdb_h.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/perl.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/extensions.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/closeout.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/jm-winsz2.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/settime.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/progtest.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/atexit.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/savedir.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/isnand.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/arpa_inet_h.m4' 2024-03-09T09:27:48.054 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/root-dev-ino.m4' 2024-03-09T09:27:48.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/hard-locale.m4' 2024-03-09T09:27:48.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/multiarch.m4' 2024-03-09T09:27:48.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/gnu-make.m4' 2024-03-09T09:27:48.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stdint_h.m4' 2024-03-09T09:27:48.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/openat.m4' 2024-03-09T09:27:48.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/nproc.m4' 2024-03-09T09:27:48.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/link.m4' 2024-03-09T09:27:48.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/faccessat.m4' 2024-03-09T09:27:48.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/uptime.m4' 2024-03-09T09:27:48.055 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fpieee.m4' 2024-03-09T09:27:49.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getdate.c' 2024-03-09T09:27:49.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/diacrit.h' 2024-03-09T09:27:49.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xmemcoll.h' 2024-03-09T09:27:49.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/isnanf.c' 2024-03-09T09:27:49.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/savewd.h' 2024-03-09T09:27:49.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/gethostname.c' 2024-03-09T09:27:49.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/alloca.in.h' 2024-03-09T09:27:49.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getdelim.c' 2024-03-09T09:27:49.153 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mpsort.c' 2024-03-09T09:27:49.154 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/config.hin' 2024-03-09T09:27:49.154 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/close-stream.h' 2024-03-09T09:27:49.154 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/savedir.h' 2024-03-09T09:27:49.154 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/hash-triple.h' 2024-03-09T09:27:49.154 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/trim.c' 2024-03-09T09:27:49.154 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/physmem.c' 2024-03-09T09:27:49.154 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fchdir.c' 2024-03-09T09:27:49.154 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stdarg.in.h' 2024-03-09T09:27:49.154 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strintcmp.c' 2024-03-09T09:27:49.154 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stpncpy.c' 2024-03-09T09:27:49.157 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xalloc.h' 2024-03-09T09:27:49.157 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/timespec.h' 2024-03-09T09:27:49.157 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/readtokens.h' 2024-03-09T09:27:49.157 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fsusage.h' 2024-03-09T09:27:49.157 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/linebuffer.c' 2024-03-09T09:27:49.157 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memcoll.h' 2024-03-09T09:27:49.157 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/string.in.h' 2024-03-09T09:27:49.157 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/quote.h' 2024-03-09T09:27:49.157 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strstr.c' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/realloc.c' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stdlib-safer.h' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/userspec.c' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/siglist.h' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/safe-read.h' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/filenamecat.c' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbrtowc.c' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getndelim2.h' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/argv-iter.h' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/openat-proc.c' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stdio-safer.h' 2024-03-09T09:27:49.158 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv_open-osf.gperf' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getugroups.h' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stdio.in.h' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memcmp.c' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/tempname.c' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/inttostr.h' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/yesno.c' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/full-write.c' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strnumcmp.c' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sys_socket.in.h' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/arpa_inet.in.h' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unlocked-io.h' 2024-03-09T09:27:49.159 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unitypes.h' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/canon-host.h' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/euidaccess-stat.c' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/isapipe.c' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/quotearg.h' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/ref-add.sin' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstriconv.h' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fprintftime.h' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/c-ctype.c' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/float.in.h' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/cloexec.c' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/offtostr.c' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv_open-aix.gperf' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/wcwidth.c' 2024-03-09T09:27:49.160 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/nproc.c' 2024-03-09T09:27:49.161 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/gai_strerror.c' 2024-03-09T09:27:49.161 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fcntl.c' 2024-03-09T09:27:49.161 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getndelim2.c' 2024-03-09T09:27:49.161 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/w32sock.h' 2024-03-09T09:27:49.161 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/printf-frexpl.h' 2024-03-09T09:27:49.161 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getopt.in.h' 2024-03-09T09:27:49.161 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/asprintf.c' 2024-03-09T09:27:49.161 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/filemode.c' 2024-03-09T09:27:49.161 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strndup.c' 2024-03-09T09:27:49.161 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/uniwidth/cjk.h' 2024-03-09T09:27:49.161 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/uniwidth/width.c' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/lib/uniwidth' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fseterr.h' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/pipe-safer.c' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/freading.c' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/diacrit.c' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/quotearg.c' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/randint.h' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/inttypes.in.h' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/regex_internal.h' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dirent--.h' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stdio-write.c' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/hash.c' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/atexit.c' 2024-03-09T09:27:49.162 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stat.c' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv_open-aix.h' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/posixtm.c' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/obstack.c' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/readutmp.h' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sig2str.h' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/str-two-way.h' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/copy-acl.c' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/canonicalize.c' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/userspec.h' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbsalign.h' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/wchar.in.h' 2024-03-09T09:27:49.163 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/filevercmp.h' 2024-03-09T09:27:49.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mkdir-p.m4' 2024-03-09T09:27:49.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/xvasprintf.m4' 2024-03-09T09:27:49.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strpbrk.m4' 2024-03-09T09:27:49.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/rename.m4' 2024-03-09T09:27:49.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/wchar_t.m4' 2024-03-09T09:27:49.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/glibc21.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/perror.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/backupfile.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/xstrtod.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/xattr.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fsusage.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/human.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mbsrtowcs.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/tm_gmtoff.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strtod.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/selinux-context-h.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strtoumax.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/inttypes-pri.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/bison.m4' 2024-03-09T09:27:49.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/filenamecat.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/i-ring.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strstr.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/calloc.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getpass.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/error.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/vasnprintf.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/exponentd.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/unlocked-io.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/nocrash.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fstypename.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/gethrxtime.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/vasprintf.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/isblank.m4' 2024-03-09T09:27:49.314 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/jm-winsz1.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/memmove.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/unlinkdir.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sockets.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/savewd.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/frexpl.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fopen.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/unistd-safer.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/utimecmp.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mempcpy.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mpsort.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sys_stat_h.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/exponentl.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sha512.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/time_r.m4' 2024-03-09T09:27:49.315 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sys_select_h.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/vprintf-posix.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/malloc.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/tzset.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/memxfrm.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/flexmember.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fcntl-o.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/getpagesize.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/unlink-busy.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/fchdir.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/malloca.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/lib-ld.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/euidaccess-stat.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/xgetcwd.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/dirent_h.m4' 2024-03-09T09:27:49.316 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/dup2.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/locale_h.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/stat-time.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/select.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/same.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/freading.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strtoll.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/group-member.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/inttypes_h.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/long-options.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/chown.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/xalloc.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/nanosleep.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/utimes.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/size_max.m4' 2024-03-09T09:27:49.317 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/frexp.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/exclude.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/wint_t.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strndup.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/lib-check.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mbiter.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/mbstate_t.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/rmdir.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/inet_ntop.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/version-etc.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/isnanl.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/signbit.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/strtoul.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/m4/sleep.m4' 2024-03-09T09:27:49.318 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/m4' 2024-03-09T09:27:49.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.version' 2024-03-09T09:27:49.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_prohibit_fail_0' 2024-03-09T09:27:49.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/Makefile.am' 2024-03-09T09:27:49.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/INSTALL' 2024-03-09T09:27:49.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/maint.mk' 2024-03-09T09:27:49.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/bootstrap.conf' 2024-03-09T09:27:49.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtoimax.c' 2024-03-09T09:27:49.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-update-copyright.sh' 2024-03-09T09:27:49.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/connect.c' 2024-03-09T09:27:49.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-signbit.c' 2024-03-09T09:27:49.319 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-stdlib.c' 2024-03-09T09:27:49.507 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/regex_internal.c' 2024-03-09T09:27:49.507 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memcpy.c' 2024-03-09T09:27:49.508 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/base64.c' 2024-03-09T09:27:49.508 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/exclude.h' 2024-03-09T09:27:49.508 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/chown.c' 2024-03-09T09:27:49.508 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/alignof.h' 2024-03-09T09:27:49.508 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/signbitd.c' 2024-03-09T09:27:49.508 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/at-func.c' 2024-03-09T09:27:49.508 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/asnprintf.c' 2024-03-09T09:27:49.508 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sig2str.c' 2024-03-09T09:27:49.508 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mkdir.c' 2024-03-09T09:27:49.508 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/savedir.c' 2024-03-09T09:27:49.508 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strtoull.c' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/cycle-check.c' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/version-etc.h' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mkancesdirs.c' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unistr/u8-uctomb-aux.c' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unistr/u8-uctomb.c' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/lib/unistr' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/priv-set.h' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/time_r.c' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/at-func2.c' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv_open-solaris.h' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/freopen-safer.c' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xnanosleep.h' 2024-03-09T09:27:49.509 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/verify.h' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/vasnprintf.c' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getgroups.c' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/errno.in.h' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/long-options.h' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/posixver.c' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/netdb.in.h' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stdlib--.h' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/same.h' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/gethrxtime.h' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/version-etc-fsf.c' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/printf-frexp.h' 2024-03-09T09:27:49.510 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/linebuffer.h' 2024-03-09T09:27:49.511 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xnanosleep.c' 2024-03-09T09:27:49.511 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/openat.c' 2024-03-09T09:27:49.511 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dup2.c' 2024-03-09T09:27:49.511 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/obstack.h' 2024-03-09T09:27:49.511 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/safe-read.c' 2024-03-09T09:27:49.511 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memchr2.valgrind' 2024-03-09T09:27:49.511 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/putenv.c' 2024-03-09T09:27:49.511 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/ctype.in.h' 2024-03-09T09:27:49.511 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unlinkat.c' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/gettime.c' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sha1.h' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/yesno.h' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/regexec.c' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/closeout.c' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/str-kmp.h' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getpass.h' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/cloexec.h' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv_open-hpux.h' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/close-hook.c' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fchmodat.c' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sig-handler.h' 2024-03-09T09:27:49.512 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/glthread/tls.h' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/glthread/tls.c' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/glthread/threadlib.c' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/glthread/lock.c' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/glthread/lock.h' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/lib/glthread' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getopt.c' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/readtokens0.c' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/gnulib.mk' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strtoimax.c' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/se-selinux.in.h' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/argv-iter.c' 2024-03-09T09:27:49.513 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/canonicalize.h' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fpending.c' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/openat-priv.h' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/closeout.h' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unistd--.h' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/close.c' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memchr.valgrind' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/ignore-value.h' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getcwd.c' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mkancesdirs.h' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbrlen.c' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/randread.c' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dirname.c' 2024-03-09T09:27:49.514 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/chdir-long.h' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/areadlink.h' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stat-time.h' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/save-cwd.h' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbsstr.c' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/open-safer.c' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mkdirat.c' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xgethostname.h' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/langinfo.in.h' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/rmdir.c' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fopen.c' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/close-stream.c' 2024-03-09T09:27:49.515 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unicodeio.c' 2024-03-09T09:27:49.516 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv_open-hpux.gperf' 2024-03-09T09:27:49.516 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/wctype.in.h' 2024-03-09T09:27:49.516 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbsinit.c' 2024-03-09T09:27:49.516 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/streq.h' 2024-03-09T09:27:49.516 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/open.c' 2024-03-09T09:27:49.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/freopen.c' 2024-03-09T09:27:49.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/posixtm.h' 2024-03-09T09:27:49.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dirfd.c' 2024-03-09T09:27:49.723 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fcntl-safer.h' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/isapipe.h' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstrtold.c' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mgetgroups.h' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/quote.c' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strcspn.c' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/ChangeLog-2007' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mpsort.h' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/inttostr.c' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stpcpy.c' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/umaxtostr.c' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/printf-frexp.c' 2024-03-09T09:27:49.724 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/buffer-lcm.h' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/regcomp.c' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/openat-die.c' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/tempname.h' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getdtablesize.c' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strsignal.c' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getfilecon.c' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/utimens.h' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/snprintf.c' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fts-cycle.c' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fpending.h' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/group-member.h' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstrtod.h' 2024-03-09T09:27:49.725 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getusershell.c' 2024-03-09T09:27:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/ref-del.sin' 2024-03-09T09:27:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/frexp.c' 2024-03-09T09:27:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getpagesize.c' 2024-03-09T09:27:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dirname.h' 2024-03-09T09:27:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sockets.c' 2024-03-09T09:27:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getopt_int.h' 2024-03-09T09:27:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/freadahead.h' 2024-03-09T09:27:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strftime.h' 2024-03-09T09:27:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/close-hook.h' 2024-03-09T09:27:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/raise.c' 2024-03-09T09:27:49.726 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fpucw.h' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/localcharset.c' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/lstat.c' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/rename.c' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/freadahead.c' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/argmatch.h' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fchown-stub.c' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/creat-safer.c' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strtoul.c' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xprintf.c' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mountlist.c' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mkdir-p.c' 2024-03-09T09:27:49.727 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/utimecmp.c' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mountlist.h' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/isnanl.c' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/randperm.h' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/remove.c' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dirent.in.h' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/malloc.c' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memrchr.c' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dirchownmod.h' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/argmatch.c' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/Makefile.in' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strnumcmp.h' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/c-strtold.c' 2024-03-09T09:27:49.728 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/freadptr.c' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/version-etc.c' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/file-type.h' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/same-inode.h' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xvasprintf.h' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/i-ring.c' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/c-strncasecmp.c' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/posixver.h' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/locale.in.h' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unistr.h' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xreadlink.c' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/human.c' 2024-03-09T09:27:49.729 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sha512.c' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv_open-solaris.gperf' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fdopendir.c' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/regex.c' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getaddrinfo.c' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/acl_entries.c' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getopt1.c' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/selinux-at.h' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strnlen1.c' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/readtokens0.h' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/symlink.c' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fclose.c' 2024-03-09T09:27:49.730 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/freadseek.h' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/safe-write.h' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fsusage.c' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/basename-lgpl.c' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/nproc.h' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stat-macros.h' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstrtoimax.c' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xmemcoll.c' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dirent-safer.h' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbchar.c' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memchr2.h' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strdup.c' 2024-03-09T09:27:49.731 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strnlen.c' 2024-03-09T09:27:49.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/c-strtod.h' 2024-03-09T09:27:49.732 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getloadavg.c' 2024-03-09T09:27:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-wcrtomb.c' 2024-03-09T09:27:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-sameacls.c' 2024-03-09T09:27:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnanf-nolibm.c' 2024-03-09T09:27:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-wcwidth.c' 2024-03-09T09:27:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-argmatch.c' 2024-03-09T09:27:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-dirname.c' 2024-03-09T09:27:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fseeko.c' 2024-03-09T09:27:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_select.c' 2024-03-09T09:27:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-freopen-safer.c' 2024-03-09T09:27:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbrtowc.c' 2024-03-09T09:27:50.101 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fseterr.c' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-vasprintf.c' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fopen.h' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-ftello.sh' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsinit.sh' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-dup-safer.c' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsstr3.c' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-vprintf-posix.sh' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-userspec.c' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnand.h' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-freopen.c' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-file-has-acl.sh' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbrtowc4.sh' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsrtowcs2.sh' 2024-03-09T09:27:50.102 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fseeko.sh' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/nan.h' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_ioctl.c' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-btowc.c' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-getopt.h' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mkdirat.c' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-stdint.c' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-c-strcasecmp.c' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsstr3.sh' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-readlink.h' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbrtowc1.sh' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude.c' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadptr.c' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fflush2.sh' 2024-03-09T09:27:50.103 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-inttypes.c' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_utsname.c' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-math.c' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/signature.h' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/w32sock.h' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-wchar.c' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.c' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/uniwidth/test-uc_width.c' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.sh' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/gnulib-tests/uniwidth' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-lstat.c' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/unlinkdir.h' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbscasecmp.c' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-netdb.c' 2024-03-09T09:27:50.104 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-stat.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-open.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-strsignal.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/accept.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xfprintf-posix.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-sockets.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xalloc-die.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fchdir.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-sigaction.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-i-ring.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-read-file.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-unlinkat.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-stddef.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-linkat.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-openat-safer.c' 2024-03-09T09:27:50.105 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-gettimeofday.c' 2024-03-09T09:27:50.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-remove.c' 2024-03-09T09:27:50.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-strtod.c' 2024-03-09T09:27:50.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-unlink.h' 2024-03-09T09:27:50.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude6.sh' 2024-03-09T09:27:50.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-argv-iter.c' 2024-03-09T09:27:50.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-lstat.h' 2024-03-09T09:27:50.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/unistr/test-u8-uctomb.c' 2024-03-09T09:27:50.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/gnulib-tests/unistr' 2024-03-09T09:27:50.106 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/setsockopt.c' 2024-03-09T09:27:50.113 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-03-09T09:27:50.114 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-03-09T09:27:50.114 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-09T09:27:50.114 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-09T09:27:50.124 INFO:tasks.ceph.mon.c:Started 2024-03-09T09:27:50.124 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:27:50.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:27:50.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.325+0000 7f1f11600640 1 Processor -- start 2024-03-09T09:27:50.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.325+0000 7f1f11600640 1 -- start start 2024-03-09T09:27:50.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.326+0000 7f1f11600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1f0c0ac730 0x7f1f0c071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:50.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.326+0000 7f1f11600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:50.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.326+0000 7f1f11600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c153d40 con 0x7f1f0c157650 2024-03-09T09:27:50.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f11600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c070ee0 con 0x7f1f0c0ac730 2024-03-09T09:27:50.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f11600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c154d00 con 0x7f1f0c0c22a0 2024-03-09T09:27:50.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f0aa00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1f0c0ac730 0x7f1f0c071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:50.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f0a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f0a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c0acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:50932/0 (socket says 172.21.15.67:50932) 2024-03-09T09:27:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f0a000640 1 -- 172.21.15.67:0/212104432 learned_addr learned my addr 172.21.15.67:0/212104432 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f0a000640 1 -- 172.21.15.67:0/212104432 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 msgr2=0x7f1f0c0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:27:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f0a000640 1 -- 172.21.15.67:0/212104432 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 msgr2=0x7f1f0c0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:27:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f0a000640 1 --2- 172.21.15.67:0/212104432 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c0acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f0a000640 1 --2- 172.21.15.67:0/212104432 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c0acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f0aa00640 1 -- 172.21.15.67:0/212104432 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 msgr2=0x7f1f0c0acb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:27:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f0aa00640 1 --2- 172.21.15.67:0/212104432 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c0acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.327+0000 7f1f0aa00640 1 -- 172.21.15.67:0/212104432 >> v1:172.21.15.134:6789/0 conn(0x7f1f0c157650 legacy=0x7f1f0c157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:50.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.328+0000 7f1f0aa00640 1 -- 172.21.15.67:0/212104432 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0c2790 con 0x7f1f0c0ac730 2024-03-09T09:27:50.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.329+0000 7f1f0aa00640 1 --2- 172.21.15.67:0/212104432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f0c0ac730 0x7f1f0c071190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0097a0 tx=0x7f1efc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84fc49dd8af1e0b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.329+0000 7f1f09600640 1 -- 172.21.15.67:0/212104432 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0315a0 con 0x7f1f0c0ac730 2024-03-09T09:27:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.329+0000 7f1f09600640 1 -- 172.21.15.67:0/212104432 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc037070 con 0x7f1f0c0ac730 2024-03-09T09:27:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.329+0000 7f1f09600640 1 -- 172.21.15.67:0/212104432 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc031960 con 0x7f1f0c0ac730 2024-03-09T09:27:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.330+0000 7f1f11600640 1 -- 172.21.15.67:0/212104432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f0c0ac730 msgr2=0x7f1f0c071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.330+0000 7f1f11600640 1 --2- 172.21.15.67:0/212104432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f0c0ac730 0x7f1f0c071190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0097a0 tx=0x7f1efc02b840 comp rx=0 tx=0).stop 2024-03-09T09:27:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.330+0000 7f1f11600640 1 -- 172.21.15.67:0/212104432 shutdown_connections 2024-03-09T09:27:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.330+0000 7f1f11600640 1 --2- 172.21.15.67:0/212104432 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.330+0000 7f1f11600640 1 --2- 172.21.15.67:0/212104432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f0c0ac730 0x7f1f0c071190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.330+0000 7f1f11600640 1 -- 172.21.15.67:0/212104432 >> 172.21.15.67:0/212104432 conn(0x7f1f0c0c7370 msgr2=0x7f1f0c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.330+0000 7f1f11600640 1 -- 172.21.15.67:0/212104432 shutdown_connections 2024-03-09T09:27:50.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.330+0000 7f1f11600640 1 -- 172.21.15.67:0/212104432 wait complete. 2024-03-09T09:27:50.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.333+0000 7f1f11600640 1 Processor -- start 2024-03-09T09:27:50.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.333+0000 7f1f11600640 1 -- start start 2024-03-09T09:27:50.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.334+0000 7f1f11600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f0c0ac730 0x7f1f0c1dda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:50.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.334+0000 7f1f11600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c1df560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.334+0000 7f1f11600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f0c157650 0x7f1f0c1dfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.334+0000 7f1f11600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c151610 con 0x7f1f0c157650 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.334+0000 7f1f11600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c070c00 con 0x7f1f0c0ac730 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.334+0000 7f1f11600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c151790 con 0x7f1f0c0c22a0 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c1df560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f0c0ac730 0x7f1f0c1dda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0a000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c1df560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:50940/0 (socket says 172.21.15.67:50940) 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0a000640 1 -- 172.21.15.67:0/2366269294 learned_addr learned my addr 172.21.15.67:0/2366269294 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0b400640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f0c157650 0x7f1f0c1dfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0a000640 1 -- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 msgr2=0x7f1f0c1df560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0a000640 1 -- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 msgr2=0x7f1f0c1df560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0a000640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c1df560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:50.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0a000640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c1df560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:50.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0aa00640 1 -- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 msgr2=0x7f1f0c1df560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:50.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0aa00640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c1df560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:50.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0aa00640 1 -- 172.21.15.67:0/2366269294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f0c157650 msgr2=0x7f1f0c1dfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:50.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0aa00640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f0c157650 0x7f1f0c1dfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:50.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.335+0000 7f1f0aa00640 1 -- 172.21.15.67:0/2366269294 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c1deac0 con 0x7f1f0c0ac730 2024-03-09T09:27:50.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.336+0000 7f1f0aa00640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f0c0ac730 0x7f1f0c1dda40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c073b20 tx=0x7f1efc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:50.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.336+0000 7f1f02a00640 1 -- 172.21.15.67:0/2366269294 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc032840 con 0x7f1f0c0ac730 2024-03-09T09:27:50.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.336+0000 7f1f02a00640 1 -- 172.21.15.67:0/2366269294 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc0329e0 con 0x7f1f0c0ac730 2024-03-09T09:27:50.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.336+0000 7f1f11600640 1 -- 172.21.15.67:0/2366269294 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c1ea820 con 0x7f1f0c0ac730 2024-03-09T09:27:50.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.336+0000 7f1f02a00640 1 -- 172.21.15.67:0/2366269294 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc032db0 con 0x7f1f0c0ac730 2024-03-09T09:27:50.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.336+0000 7f1f11600640 1 -- 172.21.15.67:0/2366269294 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c1ddf80 con 0x7f1f0c0ac730 2024-03-09T09:27:50.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.338+0000 7f1f11600640 1 -- 172.21.15.67:0/2366269294 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c075a00 con 0x7f1f0c0ac730 2024-03-09T09:27:50.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.339+0000 7f1f02a00640 1 -- 172.21.15.67:0/2366269294 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1efc03a380 con 0x7f1f0c0ac730 2024-03-09T09:27:50.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.340+0000 7f1f02a00640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1eec06b4d0 0x7f1eec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:50.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.341+0000 7f1f02a00640 1 -- 172.21.15.67:0/2366269294 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f1efc02f030 con 0x7f1f0c0ac730 2024-03-09T09:27:50.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.342+0000 7f1f0a000640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1eec06b4d0 0x7f1eec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:50.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.343+0000 7f1f0a000640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1eec06b4d0 0x7f1eec06d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c1e7a60 tx=0x7f1ef8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:50.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.347+0000 7f1f02a00640 1 -- 172.21.15.67:0/2366269294 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1efc04a070 con 0x7f1f0c0ac730 2024-03-09T09:27:50.375 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/file-type.c' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/i-ring.h' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fstatat.c' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sha512.h' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/gettimeofday.c' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbsrtowcs-state.c' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xmalloc.c' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memcoll.c' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstrtoul.c' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fts_.h' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/c-ctype.h' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stripslash.c' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv_open-osf.h' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/wcrtomb.c' 2024-03-09T09:27:50.376 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/utimecmp.h' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stdio--.h' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/randperm.c' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/areadlinkat.c' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/readutmp.c' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unicodeio.h' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/full-read.h' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fseterr.c' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/intprops.h' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/pathmax.h' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/utimens.c' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/propername.h' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/acl-internal.h' 2024-03-09T09:27:50.377 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/safe-write.c' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strtoll.c' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mkstemp.c' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/Makefile.am' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/file-set.h' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/md5.c' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/base64.h' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv_open-irix.gperf' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/vasnprintf.h' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstrndup.h' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/hash-pjw.c' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sys_time.in.h' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strtol.c' 2024-03-09T09:27:50.378 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/backupfile.h' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/setenv.c' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sys_select.in.h' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/config.charset' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xgetcwd.h' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fnmatch_loop.c' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/hash-triple.c' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/hash-pjw.h' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memcmp2.c' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/write.c' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/modechange.h' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unlink.c' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stdbool.in.h' 2024-03-09T09:27:50.379 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/time.in.h' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/freadptr.h' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/isblank.c' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getline.c' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/math.in.h' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/buffer-lcm.c' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/exclude.c' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/filevercmp.c' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/closein.c' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/closein.h' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xfts.h' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/file-set.c' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unistd-safer.h' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/uniwidth.h' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/symlinkat.c' 2024-03-09T09:27:50.380 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/readtokens.c' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/exitfail.h' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/malloca.c' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fd-safer.c' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xsize.h' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strnumcmp-in.h' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbswidth.c' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fopen-safer.c' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstrndup.c' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/human.h' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/gettext.h' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mktime-internal.h' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fchownat.c' 2024-03-09T09:27:50.381 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dup-safer.c' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstrtol-error.c' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/full-write.h' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dup-safer-flag.c' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/float+.h' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sigprocmask.c' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/printf-args.c' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/gethrxtime.c' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/randread.h' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strpbrk.c' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getugroups.c' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/chdir-long.c' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fcntl--.h' 2024-03-09T09:27:50.382 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mkstemp-safer.c' 2024-03-09T09:27:50.383 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbslen.c' 2024-03-09T09:27:50.383 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/bitrotate.h' 2024-03-09T09:27:50.383 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dev-ino.h' 2024-03-09T09:27:50.383 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xasprintf.c' 2024-03-09T09:27:50.383 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sockets.h' 2024-03-09T09:27:50.383 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/inet_ntop.c' 2024-03-09T09:27:50.383 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getdate.y' 2024-03-09T09:27:50.383 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stddef.in.h' 2024-03-09T09:27:50.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.748+0000 7f1f11600640 1 -- 172.21.15.67:0/2366269294 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1f0c0c1d70 con 0x7f1f0c0ac730 2024-03-09T09:27:50.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.749+0000 7f1f02a00640 1 -- 172.21.15.67:0/2366269294 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f1efc040070 con 0x7f1f0c0ac730 2024-03-09T09:27:50.756 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":280,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:27:50.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.752+0000 7f1f11600640 1 -- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1eec06b4d0 msgr2=0x7f1eec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.752+0000 7f1f11600640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1eec06b4d0 0x7f1eec06d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c1e7a60 tx=0x7f1ef8008040 comp rx=0 tx=0).stop 2024-03-09T09:27:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.752+0000 7f1f11600640 1 -- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f0c0ac730 msgr2=0x7f1f0c1dda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.752+0000 7f1f11600640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f0c0ac730 0x7f1f0c1dda40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c073b20 tx=0x7f1efc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:27:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.752+0000 7f1f11600640 1 -- 172.21.15.67:0/2366269294 shutdown_connections 2024-03-09T09:27:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.752+0000 7f1f11600640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1eec06b4d0 0x7f1eec06d990 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.752+0000 7f1f11600640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f0c157650 0x7f1f0c1dfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.752+0000 7f1f11600640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f0c0c22a0 0x7f1f0c1df560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.752+0000 7f1f11600640 1 --2- 172.21.15.67:0/2366269294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f0c0ac730 0x7f1f0c1dda40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:50.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.752+0000 7f1f11600640 1 -- 172.21.15.67:0/2366269294 >> 172.21.15.67:0/2366269294 conn(0x7f1f0c0c7370 msgr2=0x7f1f0c1531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.753+0000 7f1f11600640 1 -- 172.21.15.67:0/2366269294 shutdown_connections 2024-03-09T09:27:50.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:50.753+0000 7f1f11600640 1 -- 172.21.15.67:0/2366269294 wait complete. 2024-03-09T09:27:50.959 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/isnand-nolibm.h' 2024-03-09T09:27:50.959 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbsrtowcs.c' 2024-03-09T09:27:50.959 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/cycle-check.h' 2024-03-09T09:27:50.959 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sha256.c' 2024-03-09T09:27:50.959 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/readlink.c' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/uname.c' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/printf-parse.h' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/printf-frexpl.c' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/exitfail.c' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/linkat.c' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/same.c' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/btowc.c' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memcasecmp.h' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sys_utsname.in.h' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/filenamecat-lgpl.c' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstriconv.c' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/rand-isaac.c' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstrtol.h' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strtod.c' 2024-03-09T09:27:50.960 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/select.c' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/ftello.c' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strnlen1.h' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xtime.h' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/save-cwd.c' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sha256.h' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memxfrm.c' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/opendir-safer.c' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/size_max.h' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbscasecmp.c' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fflush.c' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mgetgroups.c' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getdate.h' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/malloca.valgrind' 2024-03-09T09:27:50.961 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv_open-irix.h' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xgetcwd.c' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xalloc-die.c' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/long-options.c' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/basename.c' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/error.c' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memcmp2.h' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/regex.h' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mkdir-p.h' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xstrtoumax.c' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dirchownmod.c' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/isnanl-nolibm.h' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/areadlink-with-size.c' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/printf-parse.c' 2024-03-09T09:27:50.962 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unistd.in.h' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strtoumax.c' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mempcpy.c' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xfreopen.c' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memmove.c' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xmemxfrm.c' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/link.c' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/priv-set.c' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/signbitl.c' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fnmatch.c' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/backupfile.c' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fd-safer-flag.c' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/signal.in.h' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/filenamecat.h' 2024-03-09T09:27:50.963 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xvasprintf.c' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/euidaccess-stat.h' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fsync.c' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/filemode.h' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/write-any-file.h' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stdlib.in.h' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/selinux-at.c' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/netinet_in.in.h' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/u64.h' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/unsetenv.c' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memchr.c' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/verror.h' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/freading.h' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xgethostname.c' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/nanosleep.c' 2024-03-09T09:27:50.964 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xprintf.h' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/malloca.h' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/settime.c' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memchr2.c' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memcasecmp.c' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/frexpl.c' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/root-dev-ino.h' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/hard-locale.c' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/localcharset.h' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stdio-impl.h' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/set-mode-acl.c' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/imaxtostr.c' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/freadseek.c' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/md5.h' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/idcache.c' 2024-03-09T09:27:50.965 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/calloc.c' 2024-03-09T09:27:50.966 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fseeko.c' 2024-03-09T09:27:50.966 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/propername.c' 2024-03-09T09:27:50.966 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/vprintf.c' 2024-03-09T09:27:50.966 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fileblocks.c' 2024-03-09T09:27:50.966 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memxfrm.h' 2024-03-09T09:27:50.966 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fprintftime.c' 2024-03-09T09:27:50.966 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/alloca.c' 2024-03-09T09:27:50.966 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/group-member.c' 2024-03-09T09:27:50.966 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/striconv.h' 2024-03-09T09:27:50.966 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbchar.h' 2024-03-09T09:27:50.966 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xreadlink.h' 2024-03-09T09:27:51.249 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude3.sh' 2024-03-09T09:27:51.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/ioctl.c' 2024-03-09T09:27:51.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.po' 2024-03-09T09:27:51.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.mo' 2024-03-09T09:27:51.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES' 2024-03-09T09:27:51.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/gnulib-tests/locale/fr' 2024-03-09T09:27:51.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/gnulib-tests/locale' 2024-03-09T09:27:51.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-c-strncasecmp.c' 2024-03-09T09:27:51.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fpurge.c' 2024-03-09T09:27:51.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude5.sh' 2024-03-09T09:27:51.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-arpa_inet.c' 2024-03-09T09:27:51.250 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fopen-safer.c' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-nl_langinfo.c' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-time.c' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-netinet_in.c' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fopen.c' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fnmatch.c' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/bind.c' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadahead.sh' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-lutimens.h' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-getndelim2.c' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-printf-posix.h' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xprintf-posix.c' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-environ.c' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-uname.c' 2024-03-09T09:27:51.251 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/glthread/thread.h' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/glthread/thread.c' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/glthread/yield.h' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/gnulib-tests/glthread' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/gnulib.mk' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-areadlink.c' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-link.h' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-nl_langinfo.sh' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-quotearg.c' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-vasnprintf.c' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-atexit.sh' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/sleep.c' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fpending.c' 2024-03-09T09:27:51.252 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-isblank.c' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xalloc-die.sh' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-wctype.c' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-utimens.h' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-file-has-acl.c' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-set-mode-acl.sh' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-btowc2.sh' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_time.c' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-areadlink.h' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadptr2.sh' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsrtowcs3.sh' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-vfprintf-posix.c' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fsync.c' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-vc-list-files-cvs.sh' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-alloca-opt.c' 2024-03-09T09:27:51.253 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-yesno.sh' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-memrchr.c' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-c-strcase.sh' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-select-out.sh' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-binary-io.sh' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-wcrtomb.sh' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-ftello2.sh' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mkdir.h' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtoul.c' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-printf-frexp.c' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-snprintf.c' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnand-nolibm.c' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-readlink.c' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-sleep.c' 2024-03-09T09:27:51.254 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/usleep.c' 2024-03-09T09:27:51.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtoimax.sh' 2024-03-09T09:27:51.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-stdbool.c' 2024-03-09T09:27:51.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-frexpl.c' 2024-03-09T09:27:51.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-alignof.c' 2024-03-09T09:27:51.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fflush.c' 2024-03-09T09:27:51.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-ctype.c' 2024-03-09T09:27:51.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/socket.c' 2024-03-09T09:27:51.255 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-yesno.c' 2024-03-09T09:27:51.419 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/c-strcase.h' 2024-03-09T09:27:51.419 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/acl.h' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/modechange.c' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fpurge.c' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/openat.h' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/signbitf.c' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mktime.c' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbsalign.c' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/xfreopen.h' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/iconv.in.h' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/hard-locale.h' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/nl_langinfo.c' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/striconv.c' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sigaction.c' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/memset.c' 2024-03-09T09:27:51.420 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fd-reopen.h' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/mbiter.h' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strftime.c' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/uinttostr.c' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/canon-host.c' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/rpmatch.c' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/sha1.c' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/vfprintf.c' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/isnanf-nolibm.h' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/c-strtod.c' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/getpass.c' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/fnmatch.in.h' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/isnand.c' 2024-03-09T09:27:51.421 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/write-any-file.c' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/c-strcasecmp.c' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/stdint.in.h' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/dirname-lgpl.c' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/lseek.c' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/lib/strerror.c' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/lib' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.tarball-version' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_error_message_uppercase' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_space_tab' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/ChangeLog-2007' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_require_config_h_first' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/ABOUT-NLS' 2024-03-09T09:27:51.422 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/aclocal.m4' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/bootstrap' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/NEWS' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/ChangeLog-2006' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_unmarked_diagnostics' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/Makefile.in' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/ChangeLog-2005' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/timespec.m4' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/xfts.m4' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/jm-macros.m4' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getugroups.m4' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/unicodeio.m4' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/lib-link.m4' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stpcpy.m4' 2024-03-09T09:27:51.423 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/math_h.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/printf.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/locale-fr.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sha1.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/safe-write.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/double-slash-root.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/close-stream.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/linkat.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/nl_langinfo.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/host-os.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/priv-set.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/write.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/lchmod.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/read-file.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/netinet_in_h.m4' 2024-03-09T09:27:51.424 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/00gnulib.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/utimbuf.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/euidaccess.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/cloexec.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/argmatch.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mountlist.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/string_h.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/utimens.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/warn-on-use.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/codeset.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/unlink.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fclose.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/vfprintf-posix.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/yield.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/localcharset.m4' 2024-03-09T09:27:51.425 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/isnanf.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/idcache.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/readlink.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mbrlen.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getgroups.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getopt.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getndelim2.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strnlen.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mgetgroups.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/signalblocking.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stat.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/gettext.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/uname.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/xnanosleep.m4' 2024-03-09T09:27:51.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strtoimax.m4' 2024-03-09T09:27:51.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/locale-zh.m4' 2024-03-09T09:27:51.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mode_t.m4' 2024-03-09T09:27:51.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/printf-frexp.m4' 2024-03-09T09:27:51.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strcspn.m4' 2024-03-09T09:27:51.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/wchar_h.m4' 2024-03-09T09:27:51.490 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/Makefile.in' 2024-03-09T09:27:51.490 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-filenamecat.c' 2024-03-09T09:27:51.490 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-link.c' 2024-03-09T09:27:51.490 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-nanosleep.c' 2024-03-09T09:27:51.490 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_socket.c' 2024-03-09T09:27:51.490 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsrtowcs4.sh' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-copy-acl.c' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-rmdir.h' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnanl.h' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-signal.c' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbrtowc2.sh' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-iconv-h.c' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude2.sh' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-perror.sh' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-langinfo.c' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-vprintf-posix.c' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-rmdir.c' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fcntl.c' 2024-03-09T09:27:51.491 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-dirent.c' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-chown.h' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/read-file.c' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-usleep.c' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-base64.c' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-atexit.c' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-memchr2.c' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fdopendir.c' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-stdio.c' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-vfprintf-posix.sh' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-getdtablesize.c' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtoumax.c' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fpending.sh' 2024-03-09T09:27:51.492 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-version-etc.c' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-getgroups.c' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadptr2.c' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-select-fd.c' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fcntl-safer.c' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fflush2.c' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-select-in.sh' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-dup2.c' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-frexp.c' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsstr2.sh' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/Makefile.am' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-select-stdin.c' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-getdelim.c' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-btowc1.sh' 2024-03-09T09:27:51.493 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-locale.c' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-md5.c' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-rename.h' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-utimens.c' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-strerror.c' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-inet_pton.c' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-priv-set.c' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-open.h' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-unlink.c' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/macros.h' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-rename.c' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtoumax.sh' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/inet_pton.c' 2024-03-09T09:27:51.494 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-memcmp.c' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xvasprintf.c' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadahead.c' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-printf-posix.output' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-openat.c' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-c-ctype.c' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-chown.c' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_stat.c' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-closein.sh' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/init.sh' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-filevercmp.c' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-getaddrinfo.c' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-areadlinkat.c' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnanl-nolibm.c' 2024-03-09T09:27:51.495 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-lchown.h' 2024-03-09T09:27:51.496 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-getline.c' 2024-03-09T09:27:51.496 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-hash.c' 2024-03-09T09:27:51.496 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-canonicalize.c' 2024-03-09T09:27:51.496 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fchownat.c' 2024-03-09T09:27:51.496 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-striconv.c' 2024-03-09T09:27:51.906 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-binary-io.c' 2024-03-09T09:27:51.906 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-malloca.c' 2024-03-09T09:27:51.906 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadseek.c' 2024-03-09T09:27:51.906 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-closein.c' 2024-03-09T09:27:51.906 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-unistd.c' 2024-03-09T09:27:51.906 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/listen.c' 2024-03-09T09:27:51.906 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-posixtm.c' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbrtowc3.sh' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsstr2.c' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/unlinkdir.c' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-version-etc.sh' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-lseek.sh' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/nap.h' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-gethostname.c' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-freading.c' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-areadlink-with-size.c' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadseek.sh' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude7.sh' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsalign.c' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/wctob.c' 2024-03-09T09:27:51.907 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-quotearg.sh' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbscasecmp.sh' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/binary-io.h' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-getopt.c' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-select.c' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-u64.c' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-stat.h' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-cloexec.c' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadptr.sh' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-lseek.c' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-perror.c' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-bitrotate.c' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-stat-time.c' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-ftello.c' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-getcwd.c' 2024-03-09T09:27:51.908 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-symlink.h' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-inet_ntop.c' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/zerosize-ptr.h' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-sha1.c' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fcntl-h.c' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fseeko2.sh' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-printf-frexpl.c' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsrtowcs.c' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-futimens.h' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mkdir.c' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsrtowcs1.sh' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude4.sh' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude1.sh' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-symlink.c' 2024-03-09T09:27:51.909 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-unsetenv.c' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtol.c' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsstr1.c' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-lchown.c' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-symlinkat.c' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-set-mode-acl.c' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xprintf-posix.sh' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-lock.c' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-copy-acl.sh' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/sys_ioctl.in.h' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/read-file.h' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fstatat.c' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtol.sh' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnanf.h' 2024-03-09T09:27:51.910 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-getopt_long.h' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-memchr.c' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-fprintf-posix.h' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-setenv.c' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-iconv.c' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-getdate.c' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-dirent-safer.c' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-vasprintf-posix.c' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-errno.c' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-vc-list-files-git.sh' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-tls.c' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsinit.c' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-string.c' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/perror.c' 2024-03-09T09:27:51.911 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/gnulib-tests/test-utimens-common.h' 2024-03-09T09:27:52.308 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/gnulib-tests' 2024-03-09T09:27:52.308 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/AUTHORS' 2024-03-09T09:27:52.308 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_po_check' 2024-03-09T09:27:52.308 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/dist-check.mk' 2024-03-09T09:27:52.308 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/ChangeLog-2008' 2024-03-09T09:27:52.308 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_trailing_blank' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_file_system' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/README' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/configure.ac' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_system_h_headers' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/THANKStt.in' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/old/textutils/ChangeLog' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/old/textutils/NEWS' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/old/textutils' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/old/sh-utils/ChangeLog' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/old/sh-utils/NEWS' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/old/sh-utils/ChangeLog.0' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/old/sh-utils' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/old/fileutils/ChangeLog' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/old/fileutils/NEWS' 2024-03-09T09:27:52.309 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/old/fileutils/ChangeLog-1997' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/old/fileutils' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/old' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_sun_os_names' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/THANKS-to-translators' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/gitlog-to-changelog' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/config.rpath' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/depcomp' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/gendocs.sh' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/compile' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/useless-if-before-free' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/do-release-commit-and-tag' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/vc-list-files' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/texinfo.tex' 2024-03-09T09:27:52.310 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/config.guess' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/update-copyright' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/announce-gen' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/ChangeLog-2007' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/cvsu' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/config.sub' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/install-sh' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/ylwrap' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/git-version-gen' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/arg-nonnull.h' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/mdate-sh' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/gnupload' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/unused-parameter.h' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/warn-on-use.h' 2024-03-09T09:27:52.311 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/missing' 2024-03-09T09:27:52.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/c++defs.h' 2024-03-09T09:27:52.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/build-aux/gnu-web-doc-update' 2024-03-09T09:27:52.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5/build-aux' 2024-03-09T09:27:52.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/configure' 2024-03-09T09:27:52.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.vg-suppressions' 2024-03-09T09:27:52.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_obsolete_symbols' 2024-03-09T09:27:52.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/.x-sc_prohibit_atoi_atof' 2024-03-09T09:27:52.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/COPYING' 2024-03-09T09:27:52.312 INFO:tasks.workunit.client.0.smithi067.stdout:removed 'coreutils-8.5/cfg.mk' 2024-03-09T09:27:52.313 INFO:tasks.workunit.client.0.smithi067.stdout:removed directory 'coreutils-8.5' 2024-03-09T09:27:52.313 INFO:tasks.workunit.client.0.smithi067.stderr:+ cp -av .snap/k . 2024-03-09T09:27:52.366 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k' -> './k' 2024-03-09T09:27:52.367 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils_8.5.orig.tar.gz' -> './k/coreutils_8.5.orig.tar.gz' 2024-03-09T09:27:52.367 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils_8.5.orig.tar.gz’: No data available 2024-03-09T09:27:52.383 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5' -> './k/coreutils-8.5' 2024-03-09T09:27:52.384 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/THANKS' -> './k/coreutils-8.5/THANKS' 2024-03-09T09:27:52.384 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/THANKS’: No data available 2024-03-09T09:27:52.387 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/ABOUT-NLS' -> './k/coreutils-8.5/ABOUT-NLS' 2024-03-09T09:27:52.387 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ABOUT-NLS’: No data available 2024-03-09T09:27:52.389 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_strcmp' -> './k/coreutils-8.5/.x-sc_prohibit_strcmp' 2024-03-09T09:27:52.389 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_strcmp’: No data available 2024-03-09T09:27:52.392 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_error_message_uppercase' -> './k/coreutils-8.5/.x-sc_error_message_uppercase' 2024-03-09T09:27:52.392 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_error_message_uppercase’: No data available 2024-03-09T09:27:52.393 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_obsolete_symbols' -> './k/coreutils-8.5/.x-sc_obsolete_symbols' 2024-03-09T09:27:52.394 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_obsolete_symbols’: No data available 2024-03-09T09:27:52.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/INSTALL' -> './k/coreutils-8.5/INSTALL' 2024-03-09T09:27:52.395 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/INSTALL’: No data available 2024-03-09T09:27:52.396 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/configure.ac' -> './k/coreutils-8.5/configure.ac' 2024-03-09T09:27:52.396 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/configure.ac’: No data available 2024-03-09T09:27:52.397 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_tab_based_indentation' -> './k/coreutils-8.5/.x-sc_prohibit_tab_based_indentation' 2024-03-09T09:27:52.397 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_tab_based_indentation’: No data available 2024-03-09T09:27:52.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/GNUmakefile' -> './k/coreutils-8.5/GNUmakefile' 2024-03-09T09:27:52.398 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/GNUmakefile’: No data available 2024-03-09T09:27:52.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/ChangeLog' -> './k/coreutils-8.5/ChangeLog' 2024-03-09T09:27:52.403 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ChangeLog’: No data available 2024-03-09T09:27:52.403 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/COPYING' -> './k/coreutils-8.5/COPYING' 2024-03-09T09:27:52.404 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/COPYING’: No data available 2024-03-09T09:27:52.405 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/maint.mk' -> './k/coreutils-8.5/maint.mk' 2024-03-09T09:27:52.405 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/maint.mk’: No data available 2024-03-09T09:27:52.412 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old' -> './k/coreutils-8.5/old' 2024-03-09T09:27:52.412 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old/textutils' -> './k/coreutils-8.5/old/textutils' 2024-03-09T09:27:52.412 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old/textutils/NEWS' -> './k/coreutils-8.5/old/textutils/NEWS' 2024-03-09T09:27:52.412 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/textutils/NEWS’: No data available 2024-03-09T09:27:52.415 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old/textutils/ChangeLog' -> './k/coreutils-8.5/old/textutils/ChangeLog' 2024-03-09T09:27:52.415 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/textutils/ChangeLog’: No data available 2024-03-09T09:27:52.437 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old/fileutils' -> './k/coreutils-8.5/old/fileutils' 2024-03-09T09:27:52.438 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old/fileutils/ChangeLog-1997' -> './k/coreutils-8.5/old/fileutils/ChangeLog-1997' 2024-03-09T09:27:52.438 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/fileutils/ChangeLog-1997’: No data available 2024-03-09T09:27:52.445 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old/fileutils/NEWS' -> './k/coreutils-8.5/old/fileutils/NEWS' 2024-03-09T09:27:52.445 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/fileutils/NEWS’: No data available 2024-03-09T09:27:52.448 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old/fileutils/ChangeLog' -> './k/coreutils-8.5/old/fileutils/ChangeLog' 2024-03-09T09:27:52.449 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/fileutils/ChangeLog’: No data available 2024-03-09T09:27:52.494 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old/sh-utils' -> './k/coreutils-8.5/old/sh-utils' 2024-03-09T09:27:52.494 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old/sh-utils/NEWS' -> './k/coreutils-8.5/old/sh-utils/NEWS' 2024-03-09T09:27:52.494 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/sh-utils/NEWS’: No data available 2024-03-09T09:27:52.501 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old/sh-utils/ChangeLog.0' -> './k/coreutils-8.5/old/sh-utils/ChangeLog.0' 2024-03-09T09:27:52.501 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/sh-utils/ChangeLog.0’: No data available 2024-03-09T09:27:52.507 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/old/sh-utils/ChangeLog' -> './k/coreutils-8.5/old/sh-utils/ChangeLog' 2024-03-09T09:27:52.507 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/sh-utils/ChangeLog’: No data available 2024-03-09T09:27:52.527 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux' -> './k/coreutils-8.5/build-aux' 2024-03-09T09:27:52.528 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/cvsu' -> './k/coreutils-8.5/build-aux/cvsu' 2024-03-09T09:27:52.528 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/cvsu’: No data available 2024-03-09T09:27:52.529 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/mdate-sh' -> './k/coreutils-8.5/build-aux/mdate-sh' 2024-03-09T09:27:52.529 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/mdate-sh’: No data available 2024-03-09T09:27:52.530 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/gitlog-to-changelog' -> './k/coreutils-8.5/build-aux/gitlog-to-changelog' 2024-03-09T09:27:52.530 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/gitlog-to-changelog’: No data available 2024-03-09T09:27:52.531 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/gnu-web-doc-update' -> './k/coreutils-8.5/build-aux/gnu-web-doc-update' 2024-03-09T09:27:52.531 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/gnu-web-doc-update’: No data available 2024-03-09T09:27:52.532 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/useless-if-before-free' -> './k/coreutils-8.5/build-aux/useless-if-before-free' 2024-03-09T09:27:52.533 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/useless-if-before-free’: No data available 2024-03-09T09:27:52.534 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/config.sub' -> './k/coreutils-8.5/build-aux/config.sub' 2024-03-09T09:27:52.534 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/config.sub’: No data available 2024-03-09T09:27:52.535 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/missing' -> './k/coreutils-8.5/build-aux/missing' 2024-03-09T09:27:52.535 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/missing’: No data available 2024-03-09T09:27:52.536 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/compile' -> './k/coreutils-8.5/build-aux/compile' 2024-03-09T09:27:52.536 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/compile’: No data available 2024-03-09T09:27:52.537 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/install-sh' -> './k/coreutils-8.5/build-aux/install-sh' 2024-03-09T09:27:52.537 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/install-sh’: No data available 2024-03-09T09:27:52.538 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/gendocs.sh' -> './k/coreutils-8.5/build-aux/gendocs.sh' 2024-03-09T09:27:52.538 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/gendocs.sh’: No data available 2024-03-09T09:27:52.539 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/update-copyright' -> './k/coreutils-8.5/build-aux/update-copyright' 2024-03-09T09:27:52.540 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/update-copyright’: No data available 2024-03-09T09:27:52.542 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/texinfo.tex' -> './k/coreutils-8.5/build-aux/texinfo.tex' 2024-03-09T09:27:52.542 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/texinfo.tex’: No data available 2024-03-09T09:27:52.543 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/unused-parameter.h' -> './k/coreutils-8.5/build-aux/unused-parameter.h' 2024-03-09T09:27:52.543 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/unused-parameter.h’: No data available 2024-03-09T09:27:52.544 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/c++defs.h' -> './k/coreutils-8.5/build-aux/c++defs.h' 2024-03-09T09:27:52.544 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/c++defs.h’: No data available 2024-03-09T09:27:52.545 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/config.guess' -> './k/coreutils-8.5/build-aux/config.guess' 2024-03-09T09:27:52.545 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/config.guess’: No data available 2024-03-09T09:27:52.546 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/do-release-commit-and-tag' -> './k/coreutils-8.5/build-aux/do-release-commit-and-tag' 2024-03-09T09:27:52.546 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/do-release-commit-and-tag’: No data available 2024-03-09T09:27:52.547 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/git-version-gen' -> './k/coreutils-8.5/build-aux/git-version-gen' 2024-03-09T09:27:52.547 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/git-version-gen’: No data available 2024-03-09T09:27:52.548 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/arg-nonnull.h' -> './k/coreutils-8.5/build-aux/arg-nonnull.h' 2024-03-09T09:27:52.548 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/arg-nonnull.h’: No data available 2024-03-09T09:27:52.549 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/gnupload' -> './k/coreutils-8.5/build-aux/gnupload' 2024-03-09T09:27:52.549 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/gnupload’: No data available 2024-03-09T09:27:52.550 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/warn-on-use.h' -> './k/coreutils-8.5/build-aux/warn-on-use.h' 2024-03-09T09:27:52.550 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/warn-on-use.h’: No data available 2024-03-09T09:27:52.551 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/vc-list-files' -> './k/coreutils-8.5/build-aux/vc-list-files' 2024-03-09T09:27:52.551 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/vc-list-files’: No data available 2024-03-09T09:27:52.552 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/ylwrap' -> './k/coreutils-8.5/build-aux/ylwrap' 2024-03-09T09:27:52.552 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/ylwrap’: No data available 2024-03-09T09:27:52.553 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/ChangeLog-2007' -> './k/coreutils-8.5/build-aux/ChangeLog-2007' 2024-03-09T09:27:52.553 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/ChangeLog-2007’: No data available 2024-03-09T09:27:52.554 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/announce-gen' -> './k/coreutils-8.5/build-aux/announce-gen' 2024-03-09T09:27:52.554 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/announce-gen’: No data available 2024-03-09T09:27:52.555 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/depcomp' -> './k/coreutils-8.5/build-aux/depcomp' 2024-03-09T09:27:52.556 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/depcomp’: No data available 2024-03-09T09:27:52.557 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/build-aux/config.rpath' -> './k/coreutils-8.5/build-aux/config.rpath' 2024-03-09T09:27:52.557 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/config.rpath’: No data available 2024-03-09T09:27:52.583 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gl' -> './k/coreutils-8.5/gl' 2024-03-09T09:27:52.583 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gl/modules' -> './k/coreutils-8.5/gl/modules' 2024-03-09T09:27:52.583 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gl/modules/getloadavg.diff' -> './k/coreutils-8.5/gl/modules/getloadavg.diff' 2024-03-09T09:27:52.584 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gl/modules/getloadavg.diff’: No data available 2024-03-09T09:27:52.625 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4' -> './k/coreutils-8.5/m4' 2024-03-09T09:27:52.626 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/dirent-safer.m4' -> './k/coreutils-8.5/m4/dirent-safer.m4' 2024-03-09T09:27:52.626 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dirent-safer.m4’: No data available 2024-03-09T09:27:52.630 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/settime.m4' -> './k/coreutils-8.5/m4/settime.m4' 2024-03-09T09:27:52.630 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/settime.m4’: No data available 2024-03-09T09:27:52.631 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/readtokens.m4' -> './k/coreutils-8.5/m4/readtokens.m4' 2024-03-09T09:27:52.631 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/readtokens.m4’: No data available 2024-03-09T09:27:52.631 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/clock_time.m4' -> './k/coreutils-8.5/m4/clock_time.m4' 2024-03-09T09:27:52.632 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/clock_time.m4’: No data available 2024-03-09T09:27:52.633 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/config-h.m4' -> './k/coreutils-8.5/m4/config-h.m4' 2024-03-09T09:27:52.633 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/config-h.m4’: No data available 2024-03-09T09:27:52.635 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/euidaccess.m4' -> './k/coreutils-8.5/m4/euidaccess.m4' 2024-03-09T09:27:52.635 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/euidaccess.m4’: No data available 2024-03-09T09:27:52.637 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/printf-frexpl.m4' -> './k/coreutils-8.5/m4/printf-frexpl.m4' 2024-03-09T09:27:52.638 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/printf-frexpl.m4’: No data available 2024-03-09T09:27:52.639 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/posix-shell.m4' -> './k/coreutils-8.5/m4/posix-shell.m4' 2024-03-09T09:27:52.640 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/posix-shell.m4’: No data available 2024-03-09T09:27:52.642 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/servent.m4' -> './k/coreutils-8.5/m4/servent.m4' 2024-03-09T09:27:52.642 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/servent.m4’: No data available 2024-03-09T09:27:52.644 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/xstrtol.m4' -> './k/coreutils-8.5/m4/xstrtol.m4' 2024-03-09T09:27:52.644 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xstrtol.m4’: No data available 2024-03-09T09:27:52.646 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mkdir-p.m4' -> './k/coreutils-8.5/m4/mkdir-p.m4' 2024-03-09T09:27:52.646 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mkdir-p.m4’: No data available 2024-03-09T09:27:52.648 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/00gnulib.m4' -> './k/coreutils-8.5/m4/00gnulib.m4' 2024-03-09T09:27:52.648 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/00gnulib.m4’: No data available 2024-03-09T09:27:52.650 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mbsinit.m4' -> './k/coreutils-8.5/m4/mbsinit.m4' 2024-03-09T09:27:52.650 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbsinit.m4’: No data available 2024-03-09T09:27:52.652 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/dup2.m4' -> './k/coreutils-8.5/m4/dup2.m4' 2024-03-09T09:27:52.652 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dup2.m4’: No data available 2024-03-09T09:27:52.654 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/eealloc.m4' -> './k/coreutils-8.5/m4/eealloc.m4' 2024-03-09T09:27:52.654 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/eealloc.m4’: No data available 2024-03-09T09:27:52.656 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/savewd.m4' -> './k/coreutils-8.5/m4/savewd.m4' 2024-03-09T09:27:52.656 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/savewd.m4’: No data available 2024-03-09T09:27:52.942 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/memrchr.m4' -> './k/coreutils-8.5/m4/memrchr.m4' 2024-03-09T09:27:52.943 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memrchr.m4’: No data available 2024-03-09T09:27:52.945 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/d-ino.m4' -> './k/coreutils-8.5/m4/d-ino.m4' 2024-03-09T09:27:52.945 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/d-ino.m4’: No data available 2024-03-09T09:27:52.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/bison.m4' -> './k/coreutils-8.5/m4/bison.m4' 2024-03-09T09:27:52.948 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/bison.m4’: No data available 2024-03-09T09:27:52.951 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/isnanl.m4' -> './k/coreutils-8.5/m4/isnanl.m4' 2024-03-09T09:27:52.951 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/isnanl.m4’: No data available 2024-03-09T09:27:52.953 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/lib-ld.m4' -> './k/coreutils-8.5/m4/lib-ld.m4' 2024-03-09T09:27:52.953 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lib-ld.m4’: No data available 2024-03-09T09:27:52.956 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stpcpy.m4' -> './k/coreutils-8.5/m4/stpcpy.m4' 2024-03-09T09:27:52.956 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stpcpy.m4’: No data available 2024-03-09T09:27:52.958 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/progtest.m4' -> './k/coreutils-8.5/m4/progtest.m4' 2024-03-09T09:27:52.959 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/progtest.m4’: No data available 2024-03-09T09:27:52.961 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/unistd_h.m4' -> './k/coreutils-8.5/m4/unistd_h.m4' 2024-03-09T09:27:52.962 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unistd_h.m4’: No data available 2024-03-09T09:27:52.964 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/time_h.m4' -> './k/coreutils-8.5/m4/time_h.m4' 2024-03-09T09:27:52.965 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/time_h.m4’: No data available 2024-03-09T09:27:52.968 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mkancesdirs.m4' -> './k/coreutils-8.5/m4/mkancesdirs.m4' 2024-03-09T09:27:52.968 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mkancesdirs.m4’: No data available 2024-03-09T09:27:52.970 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/signalblocking.m4' -> './k/coreutils-8.5/m4/signalblocking.m4' 2024-03-09T09:27:52.970 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/signalblocking.m4’: No data available 2024-03-09T09:27:52.972 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/unlink.m4' -> './k/coreutils-8.5/m4/unlink.m4' 2024-03-09T09:27:52.973 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unlink.m4’: No data available 2024-03-09T09:27:52.975 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/backupfile.m4' -> './k/coreutils-8.5/m4/backupfile.m4' 2024-03-09T09:27:52.975 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/backupfile.m4’: No data available 2024-03-09T09:27:52.977 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/remove.m4' -> './k/coreutils-8.5/m4/remove.m4' 2024-03-09T09:27:52.977 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/remove.m4’: No data available 2024-03-09T09:27:52.979 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fts.m4' -> './k/coreutils-8.5/m4/fts.m4' 2024-03-09T09:27:52.979 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fts.m4’: No data available 2024-03-09T09:27:52.981 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/xalloc.m4' -> './k/coreutils-8.5/m4/xalloc.m4' 2024-03-09T09:27:52.982 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xalloc.m4’: No data available 2024-03-09T09:27:52.984 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/inet_ntop.m4' -> './k/coreutils-8.5/m4/inet_ntop.m4' 2024-03-09T09:27:52.984 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inet_ntop.m4’: No data available 2024-03-09T09:27:52.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/iconv_open.m4' -> './k/coreutils-8.5/m4/iconv_open.m4' 2024-03-09T09:27:52.987 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/iconv_open.m4’: No data available 2024-03-09T09:27:52.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/extensions.m4' -> './k/coreutils-8.5/m4/extensions.m4' 2024-03-09T09:27:52.989 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/extensions.m4’: No data available 2024-03-09T09:27:52.991 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mbrtowc.m4' -> './k/coreutils-8.5/m4/mbrtowc.m4' 2024-03-09T09:27:52.992 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbrtowc.m4’: No data available 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/file-type.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/usleep.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stdlib-safer.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/setenv.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stat-prog.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/write-any-file.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/yesno.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sys_ioctl_h.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/lchown.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/intlmacosx.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strsignal.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mbsinit.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/memcasecmp.m4' 2024-03-09T09:27:52.993 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/freopen.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/iconv.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mbrtowc.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/ioctl.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sockpfaf.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/thread.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fflush.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/assert.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stpncpy.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strftime.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/wcrtomb.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/langinfo_h.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/safe-read.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/clock_time.m4' 2024-03-09T09:27:52.994 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/tls.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/memcoll.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/readutmp.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stdio_h.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/ftello.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/memcmp.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/canon-host.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/gmp.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/environ.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fdopendir.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stdarg.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stdbool.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/intmax_t.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/isapipe.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/ldexpl.m4' 2024-03-09T09:27:52.995 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/remove.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fpurge.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getusershell.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/memcpy.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fcntl.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/tempname.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/xstrtol.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fseeko.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fnmatch.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/save-cwd.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/config-h.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/filemode.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/errno_h.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/eealloc.m4' 2024-03-09T09:27:52.996 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strdup.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/signal_h.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/regex.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mbchar.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strtoull.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stdint.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/wctype_h.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/realloc.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/printf-frexpl.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/open.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/inline.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/warnings.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/vasprintf-posix.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/xstrndup.m4' 2024-03-09T09:27:52.997 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sys_utsname_h.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/locale-ja.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/snprintf.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/po.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/hash.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sig2str.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/ctype.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strnumcmp.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/rpmatch.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/md5.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/boottime.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/unistd_h.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fd-reopen.m4' 2024-03-09T09:27:52.998 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/cycle-check.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/ls-mntd-fs.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mkancesdirs.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/manywarnings.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/ChangeLog-2007' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getcwd-path-max.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/modechange.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stdlib_h.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/gethostname.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/physmem.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fileblocks.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/socklen.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/st_dm_mode.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/ungetc.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/posixver.m4' 2024-03-09T09:27:52.999 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getdtablesize.m4' 2024-03-09T09:27:53.000 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/check-decl.m4' 2024-03-09T09:27:53.000 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/threadlib.m4' 2024-03-09T09:27:53.000 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sys_socket_h.m4' 2024-03-09T09:27:53.000 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getaddrinfo.m4' 2024-03-09T09:27:53.000 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/float_h.m4' 2024-03-09T09:27:53.000 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/btowc.m4' 2024-03-09T09:27:53.000 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/check-decl.m4' -> './k/coreutils-8.5/m4/check-decl.m4' 2024-03-09T09:27:53.001 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/check-decl.m4’: No data available 2024-03-09T09:27:53.001 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/cycle-check.m4' -> './k/coreutils-8.5/m4/cycle-check.m4' 2024-03-09T09:27:53.001 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/cycle-check.m4’: No data available 2024-03-09T09:27:53.001 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/signal_h.m4' -> './k/coreutils-8.5/m4/signal_h.m4' 2024-03-09T09:27:53.002 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/signal_h.m4’: No data available 2024-03-09T09:27:53.002 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sys_socket_h.m4' -> './k/coreutils-8.5/m4/sys_socket_h.m4' 2024-03-09T09:27:53.002 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_socket_h.m4’: No data available 2024-03-09T09:27:53.003 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fprintftime.m4' -> './k/coreutils-8.5/m4/fprintftime.m4' 2024-03-09T09:27:53.003 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fprintftime.m4’: No data available 2024-03-09T09:27:53.004 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/lib-check.m4' -> './k/coreutils-8.5/m4/lib-check.m4' 2024-03-09T09:27:53.005 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lib-check.m4’: No data available 2024-03-09T09:27:53.006 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/isblank.m4' -> './k/coreutils-8.5/m4/isblank.m4' 2024-03-09T09:27:53.006 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/isblank.m4’: No data available 2024-03-09T09:27:53.008 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/xstrtod.m4' -> './k/coreutils-8.5/m4/xstrtod.m4' 2024-03-09T09:27:53.008 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xstrtod.m4’: No data available 2024-03-09T09:27:53.010 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stdarg.m4' -> './k/coreutils-8.5/m4/stdarg.m4' 2024-03-09T09:27:53.010 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdarg.m4’: No data available 2024-03-09T09:27:53.011 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/selinux-selinux-h.m4' -> './k/coreutils-8.5/m4/selinux-selinux-h.m4' 2024-03-09T09:27:53.011 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/selinux-selinux-h.m4’: No data available 2024-03-09T09:27:53.013 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fstypename.m4' -> './k/coreutils-8.5/m4/fstypename.m4' 2024-03-09T09:27:53.013 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fstypename.m4’: No data available 2024-03-09T09:27:53.015 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/link.m4' -> './k/coreutils-8.5/m4/link.m4' 2024-03-09T09:27:53.015 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/link.m4’: No data available 2024-03-09T09:27:53.016 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/md5.m4' -> './k/coreutils-8.5/m4/md5.m4' 2024-03-09T09:27:53.016 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/md5.m4’: No data available 2024-03-09T09:27:53.017 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/multiarch.m4' -> './k/coreutils-8.5/m4/multiarch.m4' 2024-03-09T09:27:53.017 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/multiarch.m4’: No data available 2024-03-09T09:27:53.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/ioctl.m4' -> './k/coreutils-8.5/m4/ioctl.m4' 2024-03-09T09:27:53.020 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ioctl.m4’: No data available 2024-03-09T09:27:53.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/flexmember.m4' -> './k/coreutils-8.5/m4/flexmember.m4' 2024-03-09T09:27:53.021 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/flexmember.m4’: No data available 2024-03-09T09:27:53.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/locale-fr.m4' -> './k/coreutils-8.5/m4/locale-fr.m4' 2024-03-09T09:27:53.023 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/locale-fr.m4’: No data available 2024-03-09T09:27:53.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fdopendir.m4' -> './k/coreutils-8.5/m4/fdopendir.m4' 2024-03-09T09:27:53.025 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fdopendir.m4’: No data available 2024-03-09T09:27:53.026 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/exponentf.m4' -> './k/coreutils-8.5/m4/exponentf.m4' 2024-03-09T09:27:53.026 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/exponentf.m4’: No data available 2024-03-09T09:27:53.028 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/utimecmp.m4' -> './k/coreutils-8.5/m4/utimecmp.m4' 2024-03-09T09:27:53.028 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/utimecmp.m4’: No data available 2024-03-09T09:27:53.029 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/human.m4' -> './k/coreutils-8.5/m4/human.m4' 2024-03-09T09:27:53.029 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/human.m4’: No data available 2024-03-09T09:27:53.030 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strerror.m4' -> './k/coreutils-8.5/m4/strerror.m4' 2024-03-09T09:27:53.030 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strerror.m4’: No data available 2024-03-09T09:27:53.032 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/socklen.m4' -> './k/coreutils-8.5/m4/socklen.m4' 2024-03-09T09:27:53.032 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/socklen.m4’: No data available 2024-03-09T09:27:53.034 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/dirent_h.m4' -> './k/coreutils-8.5/m4/dirent_h.m4' 2024-03-09T09:27:53.034 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dirent_h.m4’: No data available 2024-03-09T09:27:53.036 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/locale-zh.m4' -> './k/coreutils-8.5/m4/locale-zh.m4' 2024-03-09T09:27:53.036 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/locale-zh.m4’: No data available 2024-03-09T09:27:53.038 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getdate.m4' -> './k/coreutils-8.5/m4/getdate.m4' 2024-03-09T09:27:53.038 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getdate.m4’: No data available 2024-03-09T09:27:53.041 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/cloexec.m4' -> './k/coreutils-8.5/m4/cloexec.m4' 2024-03-09T09:27:53.042 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/cloexec.m4’: No data available 2024-03-09T09:27:53.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/userspec.m4' 2024-03-09T09:27:53.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/posixtm.m4' 2024-03-09T09:27:53.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/alloca.m4' 2024-03-09T09:27:53.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/lstat.m4' 2024-03-09T09:27:53.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/lock.m4' 2024-03-09T09:27:53.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/readtokens.m4' 2024-03-09T09:27:53.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/locale-tr.m4' 2024-03-09T09:27:53.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/wctob.m4' 2024-03-09T09:27:53.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/canonicalize.m4' 2024-03-09T09:27:53.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/servent.m4' 2024-03-09T09:27:53.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mbswidth.m4' 2024-03-09T09:27:53.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/memrchr.m4' 2024-03-09T09:27:53.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/autobuild.m4' 2024-03-09T09:27:53.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strtol.m4' 2024-03-09T09:27:53.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/closein.m4' 2024-03-09T09:27:53.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sha256.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mbslen.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/lseek.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/base64.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/iconv_h.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/longlong.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/dirfd.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/d-ino.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/ssize_t.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/gnulib-comp.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/memset.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getcwd-abort-bug.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/c-strtod.m4' 2024-03-09T09:27:53.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fts.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/lib-prefix.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mmap-anon.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/close.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/lib-ignore.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/gnulib-common.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/time_h.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/nls.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/dirname.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fcntl_h.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stddef_h.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mkstemp.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/link-follow.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/wcwidth.m4' 2024-03-09T09:27:53.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/memchr.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strerror.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/iconv_open.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/exponentf.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/pathmax.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stdio-safer.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/acl.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fpending.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/putenv.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/include-exclude-prog.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mktime.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getdate.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fsync.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/symlink.m4' 2024-03-09T09:27:53.280 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/quote.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getloadavg.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/gettime.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/quotearg.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/inttypes.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/dos.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/gettimeofday.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/hostent.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/chdir-long.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/selinux-selinux-h.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fcntl-safer.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/inttostr.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getline.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/posix-shell.m4' 2024-03-09T09:27:53.281 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/d-type.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/inet_pton.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sigaction.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mkdir.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getdelim.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/xsize.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/symlinkat.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/dirent-safer.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/include_next.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getcwd.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sys_time_h.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fprintftime.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/prereq.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/netdb_h.m4' 2024-03-09T09:27:53.282 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/perl.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/extensions.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/closeout.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/jm-winsz2.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/settime.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/progtest.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/atexit.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/savedir.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/isnand.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/arpa_inet_h.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/root-dev-ino.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/hard-locale.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/multiarch.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/gnu-make.m4' 2024-03-09T09:27:53.283 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stdint_h.m4' 2024-03-09T09:27:53.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/openat.m4' 2024-03-09T09:27:53.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/nproc.m4' 2024-03-09T09:27:53.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/link.m4' 2024-03-09T09:27:53.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/faccessat.m4' 2024-03-09T09:27:53.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/uptime.m4' 2024-03-09T09:27:53.284 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fpieee.m4' 2024-03-09T09:27:53.439 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/i-ring.m4' -> './k/coreutils-8.5/m4/i-ring.m4' 2024-03-09T09:27:53.439 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/i-ring.m4’: No data available 2024-03-09T09:27:53.443 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/group-member.m4' -> './k/coreutils-8.5/m4/group-member.m4' 2024-03-09T09:27:53.444 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/group-member.m4’: No data available 2024-03-09T09:27:53.449 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strtod.m4' -> './k/coreutils-8.5/m4/strtod.m4' 2024-03-09T09:27:53.450 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtod.m4’: No data available 2024-03-09T09:27:53.459 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/chdir-long.m4' -> './k/coreutils-8.5/m4/chdir-long.m4' 2024-03-09T09:27:53.460 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/chdir-long.m4’: No data available 2024-03-09T09:27:53.462 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strnlen.m4' -> './k/coreutils-8.5/m4/strnlen.m4' 2024-03-09T09:27:53.463 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strnlen.m4’: No data available 2024-03-09T09:27:53.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strtoimax.m4' -> './k/coreutils-8.5/m4/strtoimax.m4' 2024-03-09T09:27:53.468 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtoimax.m4’: No data available 2024-03-09T09:27:53.473 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getgroups.m4' -> './k/coreutils-8.5/m4/getgroups.m4' 2024-03-09T09:27:53.473 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getgroups.m4’: No data available 2024-03-09T09:27:53.474 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/yield.m4' -> './k/coreutils-8.5/m4/yield.m4' 2024-03-09T09:27:53.475 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/yield.m4’: No data available 2024-03-09T09:27:53.476 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/write.m4' -> './k/coreutils-8.5/m4/write.m4' 2024-03-09T09:27:53.477 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/write.m4’: No data available 2024-03-09T09:27:53.478 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/localcharset.m4' -> './k/coreutils-8.5/m4/localcharset.m4' 2024-03-09T09:27:53.479 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/localcharset.m4’: No data available 2024-03-09T09:27:53.481 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/usleep.m4' -> './k/coreutils-8.5/m4/usleep.m4' 2024-03-09T09:27:53.482 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/usleep.m4’: No data available 2024-03-09T09:27:53.485 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/memmove.m4' -> './k/coreutils-8.5/m4/memmove.m4' 2024-03-09T09:27:53.485 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memmove.m4’: No data available 2024-03-09T09:27:53.487 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/close.m4' -> './k/coreutils-8.5/m4/close.m4' 2024-03-09T09:27:53.488 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/close.m4’: No data available 2024-03-09T09:27:53.491 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/memcmp.m4' -> './k/coreutils-8.5/m4/memcmp.m4' 2024-03-09T09:27:53.492 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memcmp.m4’: No data available 2024-03-09T09:27:53.493 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/autobuild.m4' -> './k/coreutils-8.5/m4/autobuild.m4' 2024-03-09T09:27:53.494 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/autobuild.m4’: No data available 2024-03-09T09:27:53.499 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fchdir.m4' -> './k/coreutils-8.5/m4/fchdir.m4' 2024-03-09T09:27:53.499 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fchdir.m4’: No data available 2024-03-09T09:27:53.504 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fd-reopen.m4' -> './k/coreutils-8.5/m4/fd-reopen.m4' 2024-03-09T09:27:53.504 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fd-reopen.m4’: No data available 2024-03-09T09:27:53.509 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/jm-winsz2.m4' -> './k/coreutils-8.5/m4/jm-winsz2.m4' 2024-03-09T09:27:53.509 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/jm-winsz2.m4’: No data available 2024-03-09T09:27:53.514 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/posixver.m4' -> './k/coreutils-8.5/m4/posixver.m4' 2024-03-09T09:27:53.514 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/posixver.m4’: No data available 2024-03-09T09:27:53.515 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/locale-ja.m4' -> './k/coreutils-8.5/m4/locale-ja.m4' 2024-03-09T09:27:53.516 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/locale-ja.m4’: No data available 2024-03-09T09:27:53.517 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stat-prog.m4' -> './k/coreutils-8.5/m4/stat-prog.m4' 2024-03-09T09:27:53.518 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stat-prog.m4’: No data available 2024-03-09T09:27:53.518 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/wint_t.m4' -> './k/coreutils-8.5/m4/wint_t.m4' 2024-03-09T09:27:53.519 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wint_t.m4’: No data available 2024-03-09T09:27:53.519 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/st_dm_mode.m4' -> './k/coreutils-8.5/m4/st_dm_mode.m4' 2024-03-09T09:27:53.520 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/st_dm_mode.m4’: No data available 2024-03-09T09:27:53.520 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/tempname.m4' -> './k/coreutils-8.5/m4/tempname.m4' 2024-03-09T09:27:53.521 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/tempname.m4’: No data available 2024-03-09T09:27:53.521 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/frexp.m4' -> './k/coreutils-8.5/m4/frexp.m4' 2024-03-09T09:27:53.522 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/frexp.m4’: No data available 2024-03-09T09:27:53.522 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/dirfd.m4' -> './k/coreutils-8.5/m4/dirfd.m4' 2024-03-09T09:27:53.523 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dirfd.m4’: No data available 2024-03-09T09:27:53.523 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sys_select_h.m4' -> './k/coreutils-8.5/m4/sys_select_h.m4' 2024-03-09T09:27:53.524 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_select_h.m4’: No data available 2024-03-09T09:27:53.524 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/btowc.m4' -> './k/coreutils-8.5/m4/btowc.m4' 2024-03-09T09:27:53.524 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/btowc.m4’: No data available 2024-03-09T09:27:53.525 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/filenamecat.m4' -> './k/coreutils-8.5/m4/filenamecat.m4' 2024-03-09T09:27:53.525 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/filenamecat.m4’: No data available 2024-03-09T09:27:53.526 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/gmp.m4' -> './k/coreutils-8.5/m4/gmp.m4' 2024-03-09T09:27:53.526 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gmp.m4’: No data available 2024-03-09T09:27:53.527 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/isapipe.m4' -> './k/coreutils-8.5/m4/isapipe.m4' 2024-03-09T09:27:53.527 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/isapipe.m4’: No data available 2024-03-09T09:27:53.528 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/wctype_h.m4' -> './k/coreutils-8.5/m4/wctype_h.m4' 2024-03-09T09:27:53.528 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wctype_h.m4’: No data available 2024-03-09T09:27:53.529 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/ldexpl.m4' -> './k/coreutils-8.5/m4/ldexpl.m4' 2024-03-09T09:27:53.529 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ldexpl.m4’: No data available 2024-03-09T09:27:53.530 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getloadavg.m4' -> './k/coreutils-8.5/m4/getloadavg.m4' 2024-03-09T09:27:53.530 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getloadavg.m4’: No data available 2024-03-09T09:27:53.530 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mbchar.m4' -> './k/coreutils-8.5/m4/mbchar.m4' 2024-03-09T09:27:53.531 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbchar.m4’: No data available 2024-03-09T09:27:53.531 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/version-etc.m4' -> './k/coreutils-8.5/m4/version-etc.m4' 2024-03-09T09:27:53.532 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/version-etc.m4’: No data available 2024-03-09T09:27:53.532 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fcntl_h.m4' -> './k/coreutils-8.5/m4/fcntl_h.m4' 2024-03-09T09:27:53.533 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fcntl_h.m4’: No data available 2024-03-09T09:27:53.533 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/gettime.m4' -> './k/coreutils-8.5/m4/gettime.m4' 2024-03-09T09:27:53.534 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gettime.m4’: No data available 2024-03-09T09:27:53.534 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/wchar_h.m4' -> './k/coreutils-8.5/m4/wchar_h.m4' 2024-03-09T09:27:53.534 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wchar_h.m4’: No data available 2024-03-09T09:27:53.535 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/vasprintf-posix.m4' -> './k/coreutils-8.5/m4/vasprintf-posix.m4' 2024-03-09T09:27:53.535 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/vasprintf-posix.m4’: No data available 2024-03-09T09:27:53.536 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/selinux-context-h.m4' -> './k/coreutils-8.5/m4/selinux-context-h.m4' 2024-03-09T09:27:53.536 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/selinux-context-h.m4’: No data available 2024-03-09T09:27:53.537 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getpass.m4' -> './k/coreutils-8.5/m4/getpass.m4' 2024-03-09T09:27:53.537 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getpass.m4’: No data available 2024-03-09T09:27:53.538 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mempcpy.m4' -> './k/coreutils-8.5/m4/mempcpy.m4' 2024-03-09T09:27:53.538 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mempcpy.m4’: No data available 2024-03-09T09:27:53.539 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/save-cwd.m4' -> './k/coreutils-8.5/m4/save-cwd.m4' 2024-03-09T09:27:53.540 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/save-cwd.m4’: No data available 2024-03-09T09:27:53.540 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fclose.m4' -> './k/coreutils-8.5/m4/fclose.m4' 2024-03-09T09:27:53.540 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fclose.m4’: No data available 2024-03-09T09:27:53.541 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/modechange.m4' -> './k/coreutils-8.5/m4/modechange.m4' 2024-03-09T09:27:53.541 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/modechange.m4’: No data available 2024-03-09T09:27:53.542 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strstr.m4' -> './k/coreutils-8.5/m4/strstr.m4' 2024-03-09T09:27:53.542 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strstr.m4’: No data available 2024-03-09T09:27:53.543 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getdtablesize.m4' -> './k/coreutils-8.5/m4/getdtablesize.m4' 2024-03-09T09:27:53.543 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getdtablesize.m4’: No data available 2024-03-09T09:27:53.544 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/wchar_t.m4' -> './k/coreutils-8.5/m4/wchar_t.m4' 2024-03-09T09:27:53.544 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wchar_t.m4’: No data available 2024-03-09T09:27:53.545 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mkdir.m4' -> './k/coreutils-8.5/m4/mkdir.m4' 2024-03-09T09:27:53.545 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mkdir.m4’: No data available 2024-03-09T09:27:53.546 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/idcache.m4' -> './k/coreutils-8.5/m4/idcache.m4' 2024-03-09T09:27:53.546 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/idcache.m4’: No data available 2024-03-09T09:27:53.546 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/tm_gmtoff.m4' -> './k/coreutils-8.5/m4/tm_gmtoff.m4' 2024-03-09T09:27:53.547 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/tm_gmtoff.m4’: No data available 2024-03-09T09:27:53.547 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/dirname.m4' -> './k/coreutils-8.5/m4/dirname.m4' 2024-03-09T09:27:53.548 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dirname.m4’: No data available 2024-03-09T09:27:53.548 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mbiter.m4' -> './k/coreutils-8.5/m4/mbiter.m4' 2024-03-09T09:27:53.549 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbiter.m4’: No data available 2024-03-09T09:27:53.549 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fcntl-safer.m4' -> './k/coreutils-8.5/m4/fcntl-safer.m4' 2024-03-09T09:27:53.549 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fcntl-safer.m4’: No data available 2024-03-09T09:27:53.550 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/manywarnings.m4' -> './k/coreutils-8.5/m4/manywarnings.m4' 2024-03-09T09:27:53.550 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/manywarnings.m4’: No data available 2024-03-09T09:27:53.551 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/same.m4' -> './k/coreutils-8.5/m4/same.m4' 2024-03-09T09:27:53.551 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/same.m4’: No data available 2024-03-09T09:27:53.552 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getdelim.m4' -> './k/coreutils-8.5/m4/getdelim.m4' 2024-03-09T09:27:53.552 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getdelim.m4’: No data available 2024-03-09T09:27:53.553 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mountlist.m4' -> './k/coreutils-8.5/m4/mountlist.m4' 2024-03-09T09:27:53.553 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mountlist.m4’: No data available 2024-03-09T09:27:53.554 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/lchmod.m4' -> './k/coreutils-8.5/m4/lchmod.m4' 2024-03-09T09:27:53.554 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lchmod.m4’: No data available 2024-03-09T09:27:53.555 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/readlink.m4' -> './k/coreutils-8.5/m4/readlink.m4' 2024-03-09T09:27:53.555 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/readlink.m4’: No data available 2024-03-09T09:27:53.556 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/gnu-make.m4' -> './k/coreutils-8.5/m4/gnu-make.m4' 2024-03-09T09:27:53.556 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gnu-make.m4’: No data available 2024-03-09T09:27:53.557 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/thread.m4' -> './k/coreutils-8.5/m4/thread.m4' 2024-03-09T09:27:53.557 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/thread.m4’: No data available 2024-03-09T09:27:53.558 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/hostent.m4' -> './k/coreutils-8.5/m4/hostent.m4' 2024-03-09T09:27:53.558 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/hostent.m4’: No data available 2024-03-09T09:27:53.559 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/math_h.m4' -> './k/coreutils-8.5/m4/math_h.m4' 2024-03-09T09:27:53.559 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/math_h.m4’: No data available 2024-03-09T09:27:53.560 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fpending.m4' -> './k/coreutils-8.5/m4/fpending.m4' 2024-03-09T09:27:53.560 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fpending.m4’: No data available 2024-03-09T09:27:53.561 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/xgetcwd.m4' -> './k/coreutils-8.5/m4/xgetcwd.m4' 2024-03-09T09:27:53.561 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xgetcwd.m4’: No data available 2024-03-09T09:27:53.562 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/utimens.m4' -> './k/coreutils-8.5/m4/utimens.m4' 2024-03-09T09:27:53.562 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/utimens.m4’: No data available 2024-03-09T09:27:53.563 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/exponentd.m4' -> './k/coreutils-8.5/m4/exponentd.m4' 2024-03-09T09:27:53.563 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/exponentd.m4’: No data available 2024-03-09T09:27:53.564 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getaddrinfo.m4' -> './k/coreutils-8.5/m4/getaddrinfo.m4' 2024-03-09T09:27:53.564 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getaddrinfo.m4’: No data available 2024-03-09T09:27:53.565 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sys_ioctl_h.m4' -> './k/coreutils-8.5/m4/sys_ioctl_h.m4' 2024-03-09T09:27:53.565 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_ioctl_h.m4’: No data available 2024-03-09T09:27:53.566 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stdlib_h.m4' -> './k/coreutils-8.5/m4/stdlib_h.m4' 2024-03-09T09:27:53.567 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdlib_h.m4’: No data available 2024-03-09T09:27:53.567 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mgetgroups.m4' -> './k/coreutils-8.5/m4/mgetgroups.m4' 2024-03-09T09:27:53.567 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mgetgroups.m4’: No data available 2024-03-09T09:27:53.568 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/open.m4' -> './k/coreutils-8.5/m4/open.m4' 2024-03-09T09:27:53.568 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/open.m4’: No data available 2024-03-09T09:27:53.569 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strpbrk.m4' -> './k/coreutils-8.5/m4/strpbrk.m4' 2024-03-09T09:27:53.569 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strpbrk.m4’: No data available 2024-03-09T09:27:53.570 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/ssize_t.m4' -> './k/coreutils-8.5/m4/ssize_t.m4' 2024-03-09T09:27:53.570 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ssize_t.m4’: No data available 2024-03-09T09:27:53.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/priv-set.m4' -> './k/coreutils-8.5/m4/priv-set.m4' 2024-03-09T09:27:53.571 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/priv-set.m4’: No data available 2024-03-09T09:27:53.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fcntl-o.m4' -> './k/coreutils-8.5/m4/fcntl-o.m4' 2024-03-09T09:27:53.572 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fcntl-o.m4’: No data available 2024-03-09T09:27:53.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/inttostr.m4' -> './k/coreutils-8.5/m4/inttostr.m4' 2024-03-09T09:27:53.573 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inttostr.m4’: No data available 2024-03-09T09:27:53.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/malloca.m4' -> './k/coreutils-8.5/m4/malloca.m4' 2024-03-09T09:27:53.574 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/malloca.m4’: No data available 2024-03-09T09:27:53.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/unlocked-io.m4' -> './k/coreutils-8.5/m4/unlocked-io.m4' 2024-03-09T09:27:53.575 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unlocked-io.m4’: No data available 2024-03-09T09:27:53.576 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getugroups.m4' -> './k/coreutils-8.5/m4/getugroups.m4' 2024-03-09T09:27:53.576 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getugroups.m4’: No data available 2024-03-09T09:27:53.577 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sha1.m4' -> './k/coreutils-8.5/m4/sha1.m4' 2024-03-09T09:27:53.577 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sha1.m4’: No data available 2024-03-09T09:27:53.578 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fpurge.m4' -> './k/coreutils-8.5/m4/fpurge.m4' 2024-03-09T09:27:53.578 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fpurge.m4’: No data available 2024-03-09T09:27:53.579 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/gnulib-comp.m4' -> './k/coreutils-8.5/m4/gnulib-comp.m4' 2024-03-09T09:27:53.580 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gnulib-comp.m4’: No data available 2024-03-09T09:27:53.580 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/wcrtomb.m4' -> './k/coreutils-8.5/m4/wcrtomb.m4' 2024-03-09T09:27:53.581 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wcrtomb.m4’: No data available 2024-03-09T09:27:53.582 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/putenv.m4' -> './k/coreutils-8.5/m4/putenv.m4' 2024-03-09T09:27:53.582 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/putenv.m4’: No data available 2024-03-09T09:27:53.582 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/lstat.m4' -> './k/coreutils-8.5/m4/lstat.m4' 2024-03-09T09:27:53.583 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lstat.m4’: No data available 2024-03-09T09:27:53.584 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strsignal.m4' -> './k/coreutils-8.5/m4/strsignal.m4' 2024-03-09T09:27:53.584 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strsignal.m4’: No data available 2024-03-09T09:27:53.586 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mbslen.m4' -> './k/coreutils-8.5/m4/mbslen.m4' 2024-03-09T09:27:53.586 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbslen.m4’: No data available 2024-03-09T09:27:53.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:27:53.881 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/inttypes.m4' -> './k/coreutils-8.5/m4/inttypes.m4' 2024-03-09T09:27:53.882 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inttypes.m4’: No data available 2024-03-09T09:27:53.883 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stdlib-safer.m4' -> './k/coreutils-8.5/m4/stdlib-safer.m4' 2024-03-09T09:27:53.883 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdlib-safer.m4’: No data available 2024-03-09T09:27:53.884 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strtol.m4' -> './k/coreutils-8.5/m4/strtol.m4' 2024-03-09T09:27:53.884 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtol.m4’: No data available 2024-03-09T09:27:53.886 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/lib-ignore.m4' -> './k/coreutils-8.5/m4/lib-ignore.m4' 2024-03-09T09:27:53.886 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lib-ignore.m4’: No data available 2024-03-09T09:27:53.887 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/readutmp.m4' -> './k/coreutils-8.5/m4/readutmp.m4' 2024-03-09T09:27:53.888 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/readutmp.m4’: No data available 2024-03-09T09:27:53.889 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strtoll.m4' -> './k/coreutils-8.5/m4/strtoll.m4' 2024-03-09T09:27:53.889 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtoll.m4’: No data available 2024-03-09T09:27:53.891 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stdbool.m4' -> './k/coreutils-8.5/m4/stdbool.m4' 2024-03-09T09:27:53.891 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdbool.m4’: No data available 2024-03-09T09:27:53.893 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/setenv.m4' -> './k/coreutils-8.5/m4/setenv.m4' 2024-03-09T09:27:53.893 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/setenv.m4’: No data available 2024-03-09T09:27:53.894 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/alloca.m4' -> './k/coreutils-8.5/m4/alloca.m4' 2024-03-09T09:27:53.895 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/alloca.m4’: No data available 2024-03-09T09:27:53.896 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/size_max.m4' -> './k/coreutils-8.5/m4/size_max.m4' 2024-03-09T09:27:53.896 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/size_max.m4’: No data available 2024-03-09T09:27:53.898 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/include_next.m4' -> './k/coreutils-8.5/m4/include_next.m4' 2024-03-09T09:27:53.898 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/include_next.m4’: No data available 2024-03-09T09:27:53.900 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sys_stat_h.m4' -> './k/coreutils-8.5/m4/sys_stat_h.m4' 2024-03-09T09:27:53.900 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_stat_h.m4’: No data available 2024-03-09T09:27:53.902 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strndup.m4' -> './k/coreutils-8.5/m4/strndup.m4' 2024-03-09T09:27:53.902 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strndup.m4’: No data available 2024-03-09T09:27:53.903 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/link-follow.m4' -> './k/coreutils-8.5/m4/link-follow.m4' 2024-03-09T09:27:53.904 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/link-follow.m4’: No data available 2024-03-09T09:27:53.905 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/ctype.m4' -> './k/coreutils-8.5/m4/ctype.m4' 2024-03-09T09:27:53.905 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ctype.m4’: No data available 2024-03-09T09:27:53.907 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/timespec.m4' -> './k/coreutils-8.5/m4/timespec.m4' 2024-03-09T09:27:53.907 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/timespec.m4’: No data available 2024-03-09T09:27:53.909 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getline.m4' -> './k/coreutils-8.5/m4/getline.m4' 2024-03-09T09:27:53.909 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getline.m4’: No data available 2024-03-09T09:27:53.910 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/calloc.m4' -> './k/coreutils-8.5/m4/calloc.m4' 2024-03-09T09:27:53.911 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/calloc.m4’: No data available 2024-03-09T09:27:53.912 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strftime.m4' -> './k/coreutils-8.5/m4/strftime.m4' 2024-03-09T09:27:53.912 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strftime.m4’: No data available 2024-03-09T09:27:53.913 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/arpa_inet_h.m4' -> './k/coreutils-8.5/m4/arpa_inet_h.m4' 2024-03-09T09:27:53.913 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/arpa_inet_h.m4’: No data available 2024-03-09T09:27:53.915 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sig2str.m4' -> './k/coreutils-8.5/m4/sig2str.m4' 2024-03-09T09:27:53.915 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sig2str.m4’: No data available 2024-03-09T09:27:53.917 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/printf.m4' -> './k/coreutils-8.5/m4/printf.m4' 2024-03-09T09:27:53.918 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/printf.m4’: No data available 2024-03-09T09:27:53.919 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/environ.m4' -> './k/coreutils-8.5/m4/environ.m4' 2024-03-09T09:27:53.919 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/environ.m4’: No data available 2024-03-09T09:27:53.921 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/vprintf-posix.m4' -> './k/coreutils-8.5/m4/vprintf-posix.m4' 2024-03-09T09:27:53.921 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/vprintf-posix.m4’: No data available 2024-03-09T09:27:53.923 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/intlmacosx.m4' -> './k/coreutils-8.5/m4/intlmacosx.m4' 2024-03-09T09:27:53.923 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/intlmacosx.m4’: No data available 2024-03-09T09:27:53.924 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/locale-tr.m4' -> './k/coreutils-8.5/m4/locale-tr.m4' 2024-03-09T09:27:53.924 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/locale-tr.m4’: No data available 2024-03-09T09:27:53.926 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mktime.m4' -> './k/coreutils-8.5/m4/mktime.m4' 2024-03-09T09:27:53.926 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mktime.m4’: No data available 2024-03-09T09:27:53.928 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/iconv.m4' -> './k/coreutils-8.5/m4/iconv.m4' 2024-03-09T09:27:53.928 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/iconv.m4’: No data available 2024-03-09T09:27:53.929 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/pathmax.m4' -> './k/coreutils-8.5/m4/pathmax.m4' 2024-03-09T09:27:53.929 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/pathmax.m4’: No data available 2024-03-09T09:27:53.931 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/printf-frexp.m4' -> './k/coreutils-8.5/m4/printf-frexp.m4' 2024-03-09T09:27:53.931 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/printf-frexp.m4’: No data available 2024-03-09T09:27:53.933 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/userspec.m4' -> './k/coreutils-8.5/m4/userspec.m4' 2024-03-09T09:27:53.933 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/userspec.m4’: No data available 2024-03-09T09:27:53.934 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/host-os.m4' -> './k/coreutils-8.5/m4/host-os.m4' 2024-03-09T09:27:53.934 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/host-os.m4’: No data available 2024-03-09T09:27:53.935 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/file-type.m4' -> './k/coreutils-8.5/m4/file-type.m4' 2024-03-09T09:27:53.935 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/file-type.m4’: No data available 2024-03-09T09:27:53.937 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/ls-mntd-fs.m4' -> './k/coreutils-8.5/m4/ls-mntd-fs.m4' 2024-03-09T09:27:53.937 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ls-mntd-fs.m4’: No data available 2024-03-09T09:27:53.939 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sha512.m4' -> './k/coreutils-8.5/m4/sha512.m4' 2024-03-09T09:27:53.939 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sha512.m4’: No data available 2024-03-09T09:27:53.941 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stdio-safer.m4' -> './k/coreutils-8.5/m4/stdio-safer.m4' 2024-03-09T09:27:53.941 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdio-safer.m4’: No data available 2024-03-09T09:27:53.943 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/lock.m4' -> './k/coreutils-8.5/m4/lock.m4' 2024-03-09T09:27:53.943 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lock.m4’: No data available 2024-03-09T09:27:53.944 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/c-strtod.m4' -> './k/coreutils-8.5/m4/c-strtod.m4' 2024-03-09T09:27:53.945 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/c-strtod.m4’: No data available 2024-03-09T09:27:53.946 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/closein.m4' -> './k/coreutils-8.5/m4/closein.m4' 2024-03-09T09:27:53.947 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/closein.m4’: No data available 2024-03-09T09:27:53.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/frexpl.m4' -> './k/coreutils-8.5/m4/frexpl.m4' 2024-03-09T09:27:53.948 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/frexpl.m4’: No data available 2024-03-09T09:27:53.949 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/posixtm.m4' -> './k/coreutils-8.5/m4/posixtm.m4' 2024-03-09T09:27:53.949 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/posixtm.m4’: No data available 2024-03-09T09:27:53.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/rmdir.m4' -> './k/coreutils-8.5/m4/rmdir.m4' 2024-03-09T09:27:53.950 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/rmdir.m4’: No data available 2024-03-09T09:27:53.954 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/chown.m4' -> './k/coreutils-8.5/m4/chown.m4' 2024-03-09T09:27:53.954 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/chown.m4’: No data available 2024-03-09T09:27:53.955 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stdint.m4' -> './k/coreutils-8.5/m4/stdint.m4' 2024-03-09T09:27:53.955 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdint.m4’: No data available 2024-03-09T09:27:53.957 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stddef_h.m4' -> './k/coreutils-8.5/m4/stddef_h.m4' 2024-03-09T09:27:53.957 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stddef_h.m4’: No data available 2024-03-09T09:27:53.958 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sigaction.m4' -> './k/coreutils-8.5/m4/sigaction.m4' 2024-03-09T09:27:53.959 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sigaction.m4’: No data available 2024-03-09T09:27:53.960 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/error.m4' -> './k/coreutils-8.5/m4/error.m4' 2024-03-09T09:27:53.960 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/error.m4’: No data available 2024-03-09T09:27:53.962 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/vasnprintf.m4' -> './k/coreutils-8.5/m4/vasnprintf.m4' 2024-03-09T09:27:53.962 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/vasnprintf.m4’: No data available 2024-03-09T09:27:53.963 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/utimbuf.m4' -> './k/coreutils-8.5/m4/utimbuf.m4' 2024-03-09T09:27:53.963 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/utimbuf.m4’: No data available 2024-03-09T09:27:53.965 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sockets.m4' -> './k/coreutils-8.5/m4/sockets.m4' 2024-03-09T09:27:53.965 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sockets.m4’: No data available 2024-03-09T09:27:53.966 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/canonicalize.m4' -> './k/coreutils-8.5/m4/canonicalize.m4' 2024-03-09T09:27:53.966 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/canonicalize.m4’: No data available 2024-03-09T09:27:53.967 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sys_time_h.m4' -> './k/coreutils-8.5/m4/sys_time_h.m4' 2024-03-09T09:27:53.967 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_time_h.m4’: No data available 2024-03-09T09:27:53.969 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/unlink-busy.m4' -> './k/coreutils-8.5/m4/unlink-busy.m4' 2024-03-09T09:27:53.969 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unlink-busy.m4’: No data available 2024-03-09T09:27:53.971 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/gettimeofday.m4' -> './k/coreutils-8.5/m4/gettimeofday.m4' 2024-03-09T09:27:53.971 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gettimeofday.m4’: No data available 2024-03-09T09:27:53.973 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/write-any-file.m4' -> './k/coreutils-8.5/m4/write-any-file.m4' 2024-03-09T09:27:53.974 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/write-any-file.m4’: No data available 2024-03-09T09:27:53.975 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strdup.m4' -> './k/coreutils-8.5/m4/strdup.m4' 2024-03-09T09:27:53.975 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strdup.m4’: No data available 2024-03-09T09:27:53.977 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/regex.m4' -> './k/coreutils-8.5/m4/regex.m4' 2024-03-09T09:27:53.978 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/regex.m4’: No data available 2024-03-09T09:27:53.978 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sleep.m4' -> './k/coreutils-8.5/m4/sleep.m4' 2024-03-09T09:27:53.978 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sleep.m4’: No data available 2024-03-09T09:27:53.979 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/float_h.m4' -> './k/coreutils-8.5/m4/float_h.m4' 2024-03-09T09:27:53.979 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/float_h.m4’: No data available 2024-03-09T09:27:53.980 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/warn-on-use.m4' -> './k/coreutils-8.5/m4/warn-on-use.m4' 2024-03-09T09:27:53.980 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/warn-on-use.m4’: No data available 2024-03-09T09:27:53.982 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/atexit.m4' -> './k/coreutils-8.5/m4/atexit.m4' 2024-03-09T09:27:53.982 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/atexit.m4’: No data available 2024-03-09T09:27:53.983 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getcwd-path-max.m4' -> './k/coreutils-8.5/m4/getcwd-path-max.m4' 2024-03-09T09:27:53.983 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getcwd-path-max.m4’: No data available 2024-03-09T09:27:53.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fnmatch.m4' -> './k/coreutils-8.5/m4/fnmatch.m4' 2024-03-09T09:27:53.985 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fnmatch.m4’: No data available 2024-03-09T09:27:53.986 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strnumcmp.m4' -> './k/coreutils-8.5/m4/strnumcmp.m4' 2024-03-09T09:27:53.986 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strnumcmp.m4’: No data available 2024-03-09T09:27:53.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/gnulib-common.m4' -> './k/coreutils-8.5/m4/gnulib-common.m4' 2024-03-09T09:27:53.990 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gnulib-common.m4’: No data available 2024-03-09T09:27:53.991 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/codeset.m4' -> './k/coreutils-8.5/m4/codeset.m4' 2024-03-09T09:27:53.991 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/codeset.m4’: No data available 2024-03-09T09:27:53.992 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/time_r.m4' -> './k/coreutils-8.5/m4/time_r.m4' 2024-03-09T09:27:53.993 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/time_r.m4’: No data available 2024-03-09T09:27:53.994 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/memcoll.m4' -> './k/coreutils-8.5/m4/memcoll.m4' 2024-03-09T09:27:53.994 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memcoll.m4’: No data available 2024-03-09T09:27:53.995 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/string_h.m4' -> './k/coreutils-8.5/m4/string_h.m4' 2024-03-09T09:27:53.996 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/string_h.m4’: No data available 2024-03-09T09:27:53.996 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sys_utsname_h.m4' -> './k/coreutils-8.5/m4/sys_utsname_h.m4' 2024-03-09T09:27:53.996 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_utsname_h.m4’: No data available 2024-03-09T09:27:53.997 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stdint_h.m4' -> './k/coreutils-8.5/m4/stdint_h.m4' 2024-03-09T09:27:53.997 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdint_h.m4’: No data available 2024-03-09T09:27:53.999 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/nproc.m4' -> './k/coreutils-8.5/m4/nproc.m4' 2024-03-09T09:27:53.999 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/nproc.m4’: No data available 2024-03-09T09:27:54.000 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/symlink.m4' -> './k/coreutils-8.5/m4/symlink.m4' 2024-03-09T09:27:54.001 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/symlink.m4’: No data available 2024-03-09T09:27:54.002 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/rename.m4' -> './k/coreutils-8.5/m4/rename.m4' 2024-03-09T09:27:54.002 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/rename.m4’: No data available 2024-03-09T09:27:54.003 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/read-file.m4' -> './k/coreutils-8.5/m4/read-file.m4' 2024-03-09T09:27:54.003 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/read-file.m4’: No data available 2024-03-09T09:27:54.004 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strtoumax.m4' -> './k/coreutils-8.5/m4/strtoumax.m4' 2024-03-09T09:27:54.004 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtoumax.m4’: No data available 2024-03-09T09:27:54.005 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getopt.m4' -> './k/coreutils-8.5/m4/getopt.m4' 2024-03-09T09:27:54.006 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getopt.m4’: No data available 2024-03-09T09:27:54.007 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fopen.m4' -> './k/coreutils-8.5/m4/fopen.m4' 2024-03-09T09:27:54.007 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fopen.m4’: No data available 2024-03-09T09:27:54.008 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/ungetc.m4' -> './k/coreutils-8.5/m4/ungetc.m4' 2024-03-09T09:27:54.009 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ungetc.m4’: No data available 2024-03-09T09:27:54.010 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/xsize.m4' -> './k/coreutils-8.5/m4/xsize.m4' 2024-03-09T09:27:54.010 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xsize.m4’: No data available 2024-03-09T09:27:54.012 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fflush.m4' -> './k/coreutils-8.5/m4/fflush.m4' 2024-03-09T09:27:54.012 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fflush.m4’: No data available 2024-03-09T09:27:54.013 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/acl.m4' -> './k/coreutils-8.5/m4/acl.m4' 2024-03-09T09:27:54.014 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/acl.m4’: No data available 2024-03-09T09:27:54.015 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fsusage.m4' -> './k/coreutils-8.5/m4/fsusage.m4' 2024-03-09T09:27:54.015 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fsusage.m4’: No data available 2024-03-09T09:27:54.017 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/gethrxtime.m4' -> './k/coreutils-8.5/m4/gethrxtime.m4' 2024-03-09T09:27:54.017 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gethrxtime.m4’: No data available 2024-03-09T09:27:54.018 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/nls.m4' -> './k/coreutils-8.5/m4/nls.m4' 2024-03-09T09:27:54.018 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/nls.m4’: No data available 2024-03-09T09:27:54.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/lib-link.m4' -> './k/coreutils-8.5/m4/lib-link.m4' 2024-03-09T09:27:54.021 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lib-link.m4’: No data available 2024-03-09T09:27:54.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mpsort.m4' -> './k/coreutils-8.5/m4/mpsort.m4' 2024-03-09T09:27:54.023 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mpsort.m4’: No data available 2024-03-09T09:27:54.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/netinet_in_h.m4' -> './k/coreutils-8.5/m4/netinet_in_h.m4' 2024-03-09T09:27:54.024 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/netinet_in_h.m4’: No data available 2024-03-09T09:27:54.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fseeko.m4' -> './k/coreutils-8.5/m4/fseeko.m4' 2024-03-09T09:27:54.026 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fseeko.m4’: No data available 2024-03-09T09:27:54.027 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/inet_pton.m4' -> './k/coreutils-8.5/m4/inet_pton.m4' 2024-03-09T09:27:54.027 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inet_pton.m4’: No data available 2024-03-09T09:27:54.029 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/yesno.m4' -> './k/coreutils-8.5/m4/yesno.m4' 2024-03-09T09:27:54.029 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/yesno.m4’: No data available 2024-03-09T09:27:54.030 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/physmem.m4' -> './k/coreutils-8.5/m4/physmem.m4' 2024-03-09T09:27:54.030 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/physmem.m4’: No data available 2024-03-09T09:27:54.031 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/intmax_t.m4' -> './k/coreutils-8.5/m4/intmax_t.m4' 2024-03-09T09:27:54.031 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/intmax_t.m4’: No data available 2024-03-09T09:27:54.033 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/inttypes-pri.m4' -> './k/coreutils-8.5/m4/inttypes-pri.m4' 2024-03-09T09:27:54.033 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inttypes-pri.m4’: No data available 2024-03-09T09:27:54.034 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/malloc.m4' -> './k/coreutils-8.5/m4/malloc.m4' 2024-03-09T09:27:54.034 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/malloc.m4’: No data available 2024-03-09T09:27:54.035 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/jm-winsz1.m4' -> './k/coreutils-8.5/m4/jm-winsz1.m4' 2024-03-09T09:27:54.035 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/jm-winsz1.m4’: No data available 2024-03-09T09:27:54.036 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/close-stream.m4' -> './k/coreutils-8.5/m4/close-stream.m4' 2024-03-09T09:27:54.036 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/close-stream.m4’: No data available 2024-03-09T09:27:54.037 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/nocrash.m4' -> './k/coreutils-8.5/m4/nocrash.m4' 2024-03-09T09:27:54.037 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/nocrash.m4’: No data available 2024-03-09T09:27:54.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.031+0000 7fa3fa000640 1 Processor -- start 2024-03-09T09:27:54.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.031+0000 7fa3fa000640 1 -- start start 2024-03-09T09:27:54.039 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/realloc.m4' -> './k/coreutils-8.5/m4/realloc.m4' 2024-03-09T09:27:54.039 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/realloc.m4’: No data available 2024-03-09T09:27:54.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.032+0000 7fa3fa000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa3f40c7bc0 0x7fa3f4161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:54.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.032+0000 7fa3fa000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f415a2a0 con 0x7fa3f40c6630 2024-03-09T09:27:54.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.032+0000 7fa3fa000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f406bb30 con 0x7fa3f4153500 2024-03-09T09:27:54.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.033+0000 7fa3fa000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f4159d70 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.040 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/canon-host.m4' -> './k/coreutils-8.5/m4/canon-host.m4' 2024-03-09T09:27:54.041 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/canon-host.m4’: No data available 2024-03-09T09:27:54.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.033+0000 7fa3f3400640 1 --1- >> v1:172.21.15.134:6789/0 conn(0x7fa3f4153500 0x7fa3f406b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:38162/0 (socket says 172.21.15.67:38162) 2024-03-09T09:27:54.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.033+0000 7fa3f3400640 1 -- 172.21.15.67:0/918945374 learned_addr learned my addr 172.21.15.67:0/918945374 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:27:54.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.033+0000 7fa3f3e00640 1 --2- 172.21.15.67:0/918945374 >> v2:172.21.15.134:3300/0 conn(0x7fa3f40c7bc0 0x7fa3f4161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:54.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.034+0000 7fa3f2000640 1 -- 172.21.15.67:0/918945374 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687582663 0 0) 0x7fa3f406bb30 con 0x7fa3f4153500 2024-03-09T09:27:54.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.034+0000 7fa3f3e00640 1 -- 172.21.15.67:0/918945374 >> v1:172.21.15.134:6789/0 conn(0x7fa3f4153500 legacy=0x7fa3f406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:54.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.034+0000 7fa3f3e00640 1 -- 172.21.15.67:0/918945374 >> v1:172.21.15.67:6789/0 conn(0x7fa3f40c6630 legacy=0x7fa3f40c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:54.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.034+0000 7fa3f3e00640 1 -- 172.21.15.67:0/918945374 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f4159fc0 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.042 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/memcpy.m4' -> './k/coreutils-8.5/m4/memcpy.m4' 2024-03-09T09:27:54.042 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memcpy.m4’: No data available 2024-03-09T09:27:54.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.035+0000 7fa3f3e00640 1 --2- 172.21.15.67:0/918945374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3f40c7bc0 0x7fa3f4161700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009d90 tx=0x7fa3ec02bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c06033e0d65f7e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:54.043 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/tls.m4' -> './k/coreutils-8.5/m4/tls.m4' 2024-03-09T09:27:54.043 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/tls.m4’: No data available 2024-03-09T09:27:54.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.036+0000 7fa3f2000640 1 -- 172.21.15.67:0/918945374 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec039070 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.036+0000 7fa3f2000640 1 -- 172.21.15.67:0/918945374 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec02c9a0 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.036+0000 7fa3f2000640 1 -- 172.21.15.67:0/918945374 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec02cd70 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.037+0000 7fa3fa000640 1 -- 172.21.15.67:0/918945374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3f40c7bc0 msgr2=0x7fa3f4161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:54.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.037+0000 7fa3fa000640 1 --2- 172.21.15.67:0/918945374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3f40c7bc0 0x7fa3f4161700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009d90 tx=0x7fa3ec02bc70 comp rx=0 tx=0).stop 2024-03-09T09:27:54.045 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/perl.m4' -> './k/coreutils-8.5/m4/perl.m4' 2024-03-09T09:27:54.045 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/perl.m4’: No data available 2024-03-09T09:27:54.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.037+0000 7fa3fa000640 1 -- 172.21.15.67:0/918945374 shutdown_connections 2024-03-09T09:27:54.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.037+0000 7fa3fa000640 1 --2- 172.21.15.67:0/918945374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3f40c7bc0 0x7fa3f4161700 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:54.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.037+0000 7fa3fa000640 1 -- 172.21.15.67:0/918945374 >> 172.21.15.67:0/918945374 conn(0x7fa3f40c8060 msgr2=0x7fa3f40c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:54.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.037+0000 7fa3fa000640 1 -- 172.21.15.67:0/918945374 shutdown_connections 2024-03-09T09:27:54.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.037+0000 7fa3fa000640 1 -- 172.21.15.67:0/918945374 wait complete. 2024-03-09T09:27:54.046 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/signbit.m4' -> './k/coreutils-8.5/m4/signbit.m4' 2024-03-09T09:27:54.046 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/signbit.m4’: No data available 2024-03-09T09:27:54.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.040+0000 7fa3fa000640 1 Processor -- start 2024-03-09T09:27:54.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.040+0000 7fa3fa000640 1 -- start start 2024-03-09T09:27:54.047 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/memchr.m4' -> './k/coreutils-8.5/m4/memchr.m4' 2024-03-09T09:27:54.048 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memchr.m4’: No data available 2024-03-09T09:27:54.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.041+0000 7fa3fa000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f40c7bc0 0x7fa3f41e2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:54.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.041+0000 7fa3fa000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3f4153500 0x7fa3f40ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:54.049 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/d-type.m4' -> './k/coreutils-8.5/m4/d-type.m4' 2024-03-09T09:27:54.049 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/d-type.m4’: No data available 2024-03-09T09:27:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.041+0000 7fa3fa000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3f414bae0 0x7fa3f40ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.041+0000 7fa3fa000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40bed90 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.041+0000 7fa3fa000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40c6f90 con 0x7fa3f414bae0 2024-03-09T09:27:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.041+0000 7fa3fa000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f406b750 con 0x7fa3f4153500 2024-03-09T09:27:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.041+0000 7fa3f3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f40c7bc0 0x7fa3f41e2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:54.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.041+0000 7fa3f3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f40c7bc0 0x7fa3f41e2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49688/0 (socket says 172.21.15.67:49688) 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f3400640 1 -- 172.21.15.67:0/3533254825 learned_addr learned my addr 172.21.15.67:0/3533254825 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f2a00640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3f4153500 0x7fa3f40ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f3e00640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3f414bae0 0x7fa3f40ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f2a00640 1 -- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3f4153500 msgr2=0x7fa3f40ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f2a00640 1 -- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3f4153500 msgr2=0x7fa3f40ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f2a00640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3f4153500 0x7fa3f40ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f2a00640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3f4153500 0x7fa3f40ac3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f3400640 1 -- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3f4153500 msgr2=0x7fa3f40ac3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f3400640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3f4153500 0x7fa3f40ac3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f3400640 1 -- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3f414bae0 msgr2=0x7fa3f40ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f3400640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3f414bae0 0x7fa3f40ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.042+0000 7fa3f3400640 1 -- 172.21.15.67:0/3533254825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f40ace70 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.043+0000 7fa3f3400640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f40c7bc0 0x7fa3f41e2e70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4001c40 tx=0x7fa3e4014420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:54.052 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/lchown.m4' -> './k/coreutils-8.5/m4/lchown.m4' 2024-03-09T09:27:54.052 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lchown.m4’: No data available 2024-03-09T09:27:54.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.043+0000 7fa3eb400640 1 -- 172.21.15.67:0/3533254825 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4014d40 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.043+0000 7fa3eb400640 1 -- 172.21.15.67:0/3533254825 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e40070d0 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.043+0000 7fa3fa000640 1 -- 172.21.15.67:0/3533254825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f40abf30 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.043+0000 7fa3eb400640 1 -- 172.21.15.67:0/3533254825 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40074c0 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.044+0000 7fa3fa000640 1 -- 172.21.15.67:0/3533254825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f40ab830 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.053 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/lib-prefix.m4' -> './k/coreutils-8.5/m4/lib-prefix.m4' 2024-03-09T09:27:54.053 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lib-prefix.m4’: No data available 2024-03-09T09:27:54.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.045+0000 7fa3fa000640 1 -- 172.21.15.67:0/3533254825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f4075a00 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.054 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getcwd.m4' -> './k/coreutils-8.5/m4/getcwd.m4' 2024-03-09T09:27:54.054 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getcwd.m4’: No data available 2024-03-09T09:27:54.055 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/locale_h.m4' -> './k/coreutils-8.5/m4/locale_h.m4' 2024-03-09T09:27:54.055 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/locale_h.m4’: No data available 2024-03-09T09:27:54.055 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/memxfrm.m4' -> './k/coreutils-8.5/m4/memxfrm.m4' 2024-03-09T09:27:54.056 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memxfrm.m4’: No data available 2024-03-09T09:27:54.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.048+0000 7fa3eb400640 1 -- 172.21.15.67:0/3533254825 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa3e4010850 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.056 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mbstate_t.m4' -> './k/coreutils-8.5/m4/mbstate_t.m4' 2024-03-09T09:27:54.057 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbstate_t.m4’: No data available 2024-03-09T09:27:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.050+0000 7fa3eb400640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3d406b5a0 0x7fa3d406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.050+0000 7fa3eb400640 1 -- 172.21.15.67:0/3533254825 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fa3e4092ae0 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.050+0000 7fa3f2a00640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3d406b5a0 0x7fa3d406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:54.058 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/filemode.m4' -> './k/coreutils-8.5/m4/filemode.m4' 2024-03-09T09:27:54.058 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/filemode.m4’: No data available 2024-03-09T09:27:54.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.051+0000 7fa3f2a00640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3d406b5a0 0x7fa3d406da60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40abd70 tx=0x7fa3e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:54.062 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/memset.m4' -> './k/coreutils-8.5/m4/memset.m4' 2024-03-09T09:27:54.062 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memset.m4’: No data available 2024-03-09T09:27:54.063 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/gethostname.m4' -> './k/coreutils-8.5/m4/gethostname.m4' 2024-03-09T09:27:54.063 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gethostname.m4’: No data available 2024-03-09T09:27:54.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.056+0000 7fa3eb400640 1 -- 172.21.15.67:0/3533254825 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa3f40abf30 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.064 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/vasprintf.m4' -> './k/coreutils-8.5/m4/vasprintf.m4' 2024-03-09T09:27:54.064 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/vasprintf.m4’: No data available 2024-03-09T09:27:54.064 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/unistd-safer.m4' -> './k/coreutils-8.5/m4/unistd-safer.m4' 2024-03-09T09:27:54.064 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unistd-safer.m4’: No data available 2024-03-09T09:27:54.065 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/xfts.m4' -> './k/coreutils-8.5/m4/xfts.m4' 2024-03-09T09:27:54.065 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xfts.m4’: No data available 2024-03-09T09:27:54.067 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/uptime.m4' -> './k/coreutils-8.5/m4/uptime.m4' 2024-03-09T09:27:54.067 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/uptime.m4’: No data available 2024-03-09T09:27:54.068 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/prereq.m4' -> './k/coreutils-8.5/m4/prereq.m4' 2024-03-09T09:27:54.068 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/prereq.m4’: No data available 2024-03-09T09:27:54.069 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/symlinkat.m4' -> './k/coreutils-8.5/m4/symlinkat.m4' 2024-03-09T09:27:54.069 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/symlinkat.m4’: No data available 2024-03-09T09:27:54.070 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getusershell.m4' -> './k/coreutils-8.5/m4/getusershell.m4' 2024-03-09T09:27:54.070 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getusershell.m4’: No data available 2024-03-09T09:27:54.071 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/isnand.m4' -> './k/coreutils-8.5/m4/isnand.m4' 2024-03-09T09:27:54.071 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/isnand.m4’: No data available 2024-03-09T09:27:54.072 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/select.m4' -> './k/coreutils-8.5/m4/select.m4' 2024-03-09T09:27:54.072 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/select.m4’: No data available 2024-03-09T09:27:54.074 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/wctob.m4' -> './k/coreutils-8.5/m4/wctob.m4' 2024-03-09T09:27:54.074 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wctob.m4’: No data available 2024-03-09T09:27:54.075 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/double-slash-root.m4' -> './k/coreutils-8.5/m4/double-slash-root.m4' 2024-03-09T09:27:54.075 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/double-slash-root.m4’: No data available 2024-03-09T09:27:54.076 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/long-options.m4' -> './k/coreutils-8.5/m4/long-options.m4' 2024-03-09T09:27:54.076 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/long-options.m4’: No data available 2024-03-09T09:27:54.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/glibc21.m4' -> './k/coreutils-8.5/m4/glibc21.m4' 2024-03-09T09:27:54.078 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/glibc21.m4’: No data available 2024-03-09T09:27:54.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stat-time.m4' -> './k/coreutils-8.5/m4/stat-time.m4' 2024-03-09T09:27:54.079 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stat-time.m4’: No data available 2024-03-09T09:27:54.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/rpmatch.m4' -> './k/coreutils-8.5/m4/rpmatch.m4' 2024-03-09T09:27:54.081 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/rpmatch.m4’: No data available 2024-03-09T09:27:54.082 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/xattr.m4' -> './k/coreutils-8.5/m4/xattr.m4' 2024-03-09T09:27:54.082 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xattr.m4’: No data available 2024-03-09T09:27:54.083 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/gettext.m4' -> './k/coreutils-8.5/m4/gettext.m4' 2024-03-09T09:27:54.084 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gettext.m4’: No data available 2024-03-09T09:27:54.084 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fsync.m4' -> './k/coreutils-8.5/m4/fsync.m4' 2024-03-09T09:27:54.085 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fsync.m4’: No data available 2024-03-09T09:27:54.086 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/exclude.m4' -> './k/coreutils-8.5/m4/exclude.m4' 2024-03-09T09:27:54.086 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/exclude.m4’: No data available 2024-03-09T09:27:54.087 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/freopen.m4' -> './k/coreutils-8.5/m4/freopen.m4' 2024-03-09T09:27:54.087 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/freopen.m4’: No data available 2024-03-09T09:27:54.088 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strtoul.m4' -> './k/coreutils-8.5/m4/strtoul.m4' 2024-03-09T09:27:54.089 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtoul.m4’: No data available 2024-03-09T09:27:54.090 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/netdb_h.m4' -> './k/coreutils-8.5/m4/netdb_h.m4' 2024-03-09T09:27:54.090 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/netdb_h.m4’: No data available 2024-03-09T09:27:54.095 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/snprintf.m4' -> './k/coreutils-8.5/m4/snprintf.m4' 2024-03-09T09:27:54.095 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/snprintf.m4’: No data available 2024-03-09T09:27:54.096 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stpncpy.m4' -> './k/coreutils-8.5/m4/stpncpy.m4' 2024-03-09T09:27:54.097 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stpncpy.m4’: No data available 2024-03-09T09:27:54.098 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/xnanosleep.m4' -> './k/coreutils-8.5/m4/xnanosleep.m4' 2024-03-09T09:27:54.098 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xnanosleep.m4’: No data available 2024-03-09T09:27:54.099 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/warnings.m4' -> './k/coreutils-8.5/m4/warnings.m4' 2024-03-09T09:27:54.099 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/warnings.m4’: No data available 2024-03-09T09:27:54.100 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/lseek.m4' -> './k/coreutils-8.5/m4/lseek.m4' 2024-03-09T09:27:54.101 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lseek.m4’: No data available 2024-03-09T09:27:54.106 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/hard-locale.m4' -> './k/coreutils-8.5/m4/hard-locale.m4' 2024-03-09T09:27:54.106 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/hard-locale.m4’: No data available 2024-03-09T09:27:54.108 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/include-exclude-prog.m4' -> './k/coreutils-8.5/m4/include-exclude-prog.m4' 2024-03-09T09:27:54.108 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/include-exclude-prog.m4’: No data available 2024-03-09T09:27:54.109 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mmap-anon.m4' -> './k/coreutils-8.5/m4/mmap-anon.m4' 2024-03-09T09:27:54.109 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mmap-anon.m4’: No data available 2024-03-09T09:27:54.110 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mkstemp.m4' -> './k/coreutils-8.5/m4/mkstemp.m4' 2024-03-09T09:27:54.110 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mkstemp.m4’: No data available 2024-03-09T09:27:54.113 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/openat.m4' -> './k/coreutils-8.5/m4/openat.m4' 2024-03-09T09:27:54.113 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/openat.m4’: No data available 2024-03-09T09:27:54.113 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mbrlen.m4' -> './k/coreutils-8.5/m4/mbrlen.m4' 2024-03-09T09:27:54.114 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbrlen.m4’: No data available 2024-03-09T09:27:54.115 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/jm-macros.m4' -> './k/coreutils-8.5/m4/jm-macros.m4' 2024-03-09T09:27:54.115 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/jm-macros.m4’: No data available 2024-03-09T09:27:54.116 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/ftello.m4' -> './k/coreutils-8.5/m4/ftello.m4' 2024-03-09T09:27:54.116 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ftello.m4’: No data available 2024-03-09T09:27:54.117 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stdio_h.m4' -> './k/coreutils-8.5/m4/stdio_h.m4' 2024-03-09T09:27:54.117 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdio_h.m4’: No data available 2024-03-09T09:27:54.118 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/exponentl.m4' -> './k/coreutils-8.5/m4/exponentl.m4' 2024-03-09T09:27:54.118 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/exponentl.m4’: No data available 2024-03-09T09:27:54.119 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getpagesize.m4' -> './k/coreutils-8.5/m4/getpagesize.m4' 2024-03-09T09:27:54.120 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getpagesize.m4’: No data available 2024-03-09T09:27:54.131 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/argmatch.m4' -> './k/coreutils-8.5/m4/argmatch.m4' 2024-03-09T09:27:54.131 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/argmatch.m4’: No data available 2024-03-09T09:27:54.132 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/perror.m4' -> './k/coreutils-8.5/m4/perror.m4' 2024-03-09T09:27:54.132 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/perror.m4’: No data available 2024-03-09T09:27:54.134 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/inline.m4' -> './k/coreutils-8.5/m4/inline.m4' 2024-03-09T09:27:54.134 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inline.m4’: No data available 2024-03-09T09:27:54.135 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/inttypes_h.m4' -> './k/coreutils-8.5/m4/inttypes_h.m4' 2024-03-09T09:27:54.135 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inttypes_h.m4’: No data available 2024-03-09T09:27:54.136 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/iconv_h.m4' -> './k/coreutils-8.5/m4/iconv_h.m4' 2024-03-09T09:27:54.136 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/iconv_h.m4’: No data available 2024-03-09T09:27:54.137 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/base64.m4' -> './k/coreutils-8.5/m4/base64.m4' 2024-03-09T09:27:54.137 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/base64.m4’: No data available 2024-03-09T09:27:54.138 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/stat.m4' -> './k/coreutils-8.5/m4/stat.m4' 2024-03-09T09:27:54.138 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stat.m4’: No data available 2024-03-09T09:27:54.139 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/tzset.m4' -> './k/coreutils-8.5/m4/tzset.m4' 2024-03-09T09:27:54.139 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/tzset.m4’: No data available 2024-03-09T09:27:54.140 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/unicodeio.m4' -> './k/coreutils-8.5/m4/unicodeio.m4' 2024-03-09T09:27:54.140 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unicodeio.m4’: No data available 2024-03-09T09:27:54.141 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sha256.m4' -> './k/coreutils-8.5/m4/sha256.m4' 2024-03-09T09:27:54.141 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sha256.m4’: No data available 2024-03-09T09:27:54.142 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/errno_h.m4' -> './k/coreutils-8.5/m4/errno_h.m4' 2024-03-09T09:27:54.142 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/errno_h.m4’: No data available 2024-03-09T09:27:54.143 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/unlinkdir.m4' -> './k/coreutils-8.5/m4/unlinkdir.m4' 2024-03-09T09:27:54.143 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unlinkdir.m4’: No data available 2024-03-09T09:27:54.145 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/boottime.m4' -> './k/coreutils-8.5/m4/boottime.m4' 2024-03-09T09:27:54.145 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/boottime.m4’: No data available 2024-03-09T09:27:54.146 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/freading.m4' -> './k/coreutils-8.5/m4/freading.m4' 2024-03-09T09:27:54.146 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/freading.m4’: No data available 2024-03-09T09:27:54.147 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mode_t.m4' -> './k/coreutils-8.5/m4/mode_t.m4' 2024-03-09T09:27:54.148 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mode_t.m4’: No data available 2024-03-09T09:27:54.149 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getndelim2.m4' -> './k/coreutils-8.5/m4/getndelim2.m4' 2024-03-09T09:27:54.149 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getndelim2.m4’: No data available 2024-03-09T09:27:54.150 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/xvasprintf.m4' -> './k/coreutils-8.5/m4/xvasprintf.m4' 2024-03-09T09:27:54.150 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xvasprintf.m4’: No data available 2024-03-09T09:27:54.151 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mbsrtowcs.m4' -> './k/coreutils-8.5/m4/mbsrtowcs.m4' 2024-03-09T09:27:54.151 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbsrtowcs.m4’: No data available 2024-03-09T09:27:54.152 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/hash.m4' -> './k/coreutils-8.5/m4/hash.m4' 2024-03-09T09:27:54.152 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/hash.m4’: No data available 2024-03-09T09:27:54.153 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/safe-write.m4' -> './k/coreutils-8.5/m4/safe-write.m4' 2024-03-09T09:27:54.154 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/safe-write.m4’: No data available 2024-03-09T09:27:54.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/po.m4' -> './k/coreutils-8.5/m4/po.m4' 2024-03-09T09:27:54.155 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/po.m4’: No data available 2024-03-09T09:27:54.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/isnanf.m4' -> './k/coreutils-8.5/m4/isnanf.m4' 2024-03-09T09:27:54.157 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/isnanf.m4’: No data available 2024-03-09T09:27:54.158 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/longlong.m4' -> './k/coreutils-8.5/m4/longlong.m4' 2024-03-09T09:27:54.158 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/longlong.m4’: No data available 2024-03-09T09:27:54.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/xstrndup.m4' -> './k/coreutils-8.5/m4/xstrndup.m4' 2024-03-09T09:27:54.159 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xstrndup.m4’: No data available 2024-03-09T09:27:54.161 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/euidaccess-stat.m4' -> './k/coreutils-8.5/m4/euidaccess-stat.m4' 2024-03-09T09:27:54.161 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/euidaccess-stat.m4’: No data available 2024-03-09T09:27:54.162 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/getcwd-abort-bug.m4' -> './k/coreutils-8.5/m4/getcwd-abort-bug.m4' 2024-03-09T09:27:54.162 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getcwd-abort-bug.m4’: No data available 2024-03-09T09:27:54.163 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/faccessat.m4' -> './k/coreutils-8.5/m4/faccessat.m4' 2024-03-09T09:27:54.163 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/faccessat.m4’: No data available 2024-03-09T09:27:54.164 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/nanosleep.m4' -> './k/coreutils-8.5/m4/nanosleep.m4' 2024-03-09T09:27:54.164 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/nanosleep.m4’: No data available 2024-03-09T09:27:54.167 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/ChangeLog-2007' -> './k/coreutils-8.5/m4/ChangeLog-2007' 2024-03-09T09:27:54.167 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ChangeLog-2007’: No data available 2024-03-09T09:27:54.168 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/utimes.m4' -> './k/coreutils-8.5/m4/utimes.m4' 2024-03-09T09:27:54.169 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/utimes.m4’: No data available 2024-03-09T09:27:54.170 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fileblocks.m4' -> './k/coreutils-8.5/m4/fileblocks.m4' 2024-03-09T09:27:54.170 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fileblocks.m4’: No data available 2024-03-09T09:27:54.171 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/sockpfaf.m4' -> './k/coreutils-8.5/m4/sockpfaf.m4' 2024-03-09T09:27:54.171 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sockpfaf.m4’: No data available 2024-03-09T09:27:54.172 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strtoull.m4' -> './k/coreutils-8.5/m4/strtoull.m4' 2024-03-09T09:27:54.172 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtoull.m4’: No data available 2024-03-09T09:27:54.174 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/assert.m4' -> './k/coreutils-8.5/m4/assert.m4' 2024-03-09T09:27:54.174 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/assert.m4’: No data available 2024-03-09T09:27:54.175 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/quotearg.m4' -> './k/coreutils-8.5/m4/quotearg.m4' 2024-03-09T09:27:54.175 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/quotearg.m4’: No data available 2024-03-09T09:27:54.176 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/uname.m4' -> './k/coreutils-8.5/m4/uname.m4' 2024-03-09T09:27:54.177 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/uname.m4’: No data available 2024-03-09T09:27:54.178 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/savedir.m4' -> './k/coreutils-8.5/m4/savedir.m4' 2024-03-09T09:27:54.178 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/savedir.m4’: No data available 2024-03-09T09:27:54.179 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/threadlib.m4' -> './k/coreutils-8.5/m4/threadlib.m4' 2024-03-09T09:27:54.180 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/threadlib.m4’: No data available 2024-03-09T09:27:54.181 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/langinfo_h.m4' -> './k/coreutils-8.5/m4/langinfo_h.m4' 2024-03-09T09:27:54.181 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/langinfo_h.m4’: No data available 2024-03-09T09:27:54.182 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/closeout.m4' -> './k/coreutils-8.5/m4/closeout.m4' 2024-03-09T09:27:54.182 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/closeout.m4’: No data available 2024-03-09T09:27:54.183 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/memcasecmp.m4' -> './k/coreutils-8.5/m4/memcasecmp.m4' 2024-03-09T09:27:54.183 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memcasecmp.m4’: No data available 2024-03-09T09:27:54.185 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/vfprintf-posix.m4' -> './k/coreutils-8.5/m4/vfprintf-posix.m4' 2024-03-09T09:27:54.185 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/vfprintf-posix.m4’: No data available 2024-03-09T09:27:54.186 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/dos.m4' -> './k/coreutils-8.5/m4/dos.m4' 2024-03-09T09:27:54.186 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dos.m4’: No data available 2024-03-09T09:27:54.187 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/root-dev-ino.m4' -> './k/coreutils-8.5/m4/root-dev-ino.m4' 2024-03-09T09:27:54.188 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/root-dev-ino.m4’: No data available 2024-03-09T09:27:54.189 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/quote.m4' -> './k/coreutils-8.5/m4/quote.m4' 2024-03-09T09:27:54.189 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/quote.m4’: No data available 2024-03-09T09:27:54.191 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/mbswidth.m4' -> './k/coreutils-8.5/m4/mbswidth.m4' 2024-03-09T09:27:54.191 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbswidth.m4’: No data available 2024-03-09T09:27:54.192 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/nl_langinfo.m4' -> './k/coreutils-8.5/m4/nl_langinfo.m4' 2024-03-09T09:27:54.192 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/nl_langinfo.m4’: No data available 2024-03-09T09:27:54.193 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fcntl.m4' -> './k/coreutils-8.5/m4/fcntl.m4' 2024-03-09T09:27:54.193 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fcntl.m4’: No data available 2024-03-09T09:27:54.194 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/strcspn.m4' -> './k/coreutils-8.5/m4/strcspn.m4' 2024-03-09T09:27:54.195 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strcspn.m4’: No data available 2024-03-09T09:27:54.195 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/safe-read.m4' -> './k/coreutils-8.5/m4/safe-read.m4' 2024-03-09T09:27:54.196 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/safe-read.m4’: No data available 2024-03-09T09:27:54.197 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/wcwidth.m4' -> './k/coreutils-8.5/m4/wcwidth.m4' 2024-03-09T09:27:54.197 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wcwidth.m4’: No data available 2024-03-09T09:27:54.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/linkat.m4' -> './k/coreutils-8.5/m4/linkat.m4' 2024-03-09T09:27:54.198 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/linkat.m4’: No data available 2024-03-09T09:27:54.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/m4/fpieee.m4' -> './k/coreutils-8.5/m4/fpieee.m4' 2024-03-09T09:27:54.199 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fpieee.m4’: No data available 2024-03-09T09:27:54.280 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/Makefile.am' -> './k/coreutils-8.5/Makefile.am' 2024-03-09T09:27:54.280 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/Makefile.am’: No data available 2024-03-09T09:27:54.281 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_po_check' -> './k/coreutils-8.5/.x-sc_po_check' 2024-03-09T09:27:54.282 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_po_check’: No data available 2024-03-09T09:27:54.284 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/dist-check.mk' -> './k/coreutils-8.5/dist-check.mk' 2024-03-09T09:27:54.284 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/dist-check.mk’: No data available 2024-03-09T09:27:54.285 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_GPL_version' -> './k/coreutils-8.5/.x-sc_GPL_version' 2024-03-09T09:27:54.286 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_GPL_version’: No data available 2024-03-09T09:27:54.287 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_unmarked_diagnostics' -> './k/coreutils-8.5/.x-sc_unmarked_diagnostics' 2024-03-09T09:27:54.288 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_unmarked_diagnostics’: No data available 2024-03-09T09:27:54.289 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.prev-version' -> './k/coreutils-8.5/.prev-version' 2024-03-09T09:27:54.289 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.prev-version’: No data available 2024-03-09T09:27:54.291 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/bootstrap.conf' -> './k/coreutils-8.5/bootstrap.conf' 2024-03-09T09:27:54.291 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/bootstrap.conf’: No data available 2024-03-09T09:27:54.298 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/ChangeLog-2005' -> './k/coreutils-8.5/ChangeLog-2005' 2024-03-09T09:27:54.298 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ChangeLog-2005’: No data available 2024-03-09T09:27:54.301 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/aclocal.m4' -> './k/coreutils-8.5/aclocal.m4' 2024-03-09T09:27:54.301 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/aclocal.m4’: No data available 2024-03-09T09:27:54.303 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/AUTHORS' -> './k/coreutils-8.5/AUTHORS' 2024-03-09T09:27:54.303 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/AUTHORS’: No data available 2024-03-09T09:27:54.305 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/cfg.mk' -> './k/coreutils-8.5/cfg.mk' 2024-03-09T09:27:54.305 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/cfg.mk’: No data available 2024-03-09T09:27:54.307 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_file_system' -> './k/coreutils-8.5/.x-sc_file_system' 2024-03-09T09:27:54.307 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_file_system’: No data available 2024-03-09T09:27:54.309 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.version' -> './k/coreutils-8.5/.version' 2024-03-09T09:27:54.309 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.version’: No data available 2024-03-09T09:27:54.310 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/TODO' -> './k/coreutils-8.5/TODO' 2024-03-09T09:27:54.311 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/TODO’: No data available 2024-03-09T09:27:54.313 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_space_tab' -> './k/coreutils-8.5/.x-sc_space_tab' 2024-03-09T09:27:54.313 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_space_tab’: No data available 2024-03-09T09:27:54.315 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_empty_lines_at_EOF' -> './k/coreutils-8.5/.x-sc_prohibit_empty_lines_at_EOF' 2024-03-09T09:27:54.315 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_empty_lines_at_EOF’: No data available 2024-03-09T09:27:54.317 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/Makefile.in' -> './k/coreutils-8.5/Makefile.in' 2024-03-09T09:27:54.317 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/Makefile.in’: No data available 2024-03-09T09:27:54.319 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.tarball-version' -> './k/coreutils-8.5/.tarball-version' 2024-03-09T09:27:54.319 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.tarball-version’: No data available 2024-03-09T09:27:54.322 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/NEWS' -> './k/coreutils-8.5/NEWS' 2024-03-09T09:27:54.322 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/NEWS’: No data available 2024-03-09T09:27:54.324 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_sun_os_names' -> './k/coreutils-8.5/.x-sc_sun_os_names' 2024-03-09T09:27:54.324 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_sun_os_names’: No data available 2024-03-09T09:27:54.326 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/bootstrap' -> './k/coreutils-8.5/bootstrap' 2024-03-09T09:27:54.326 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/bootstrap’: No data available 2024-03-09T09:27:54.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.382+0000 7fa3fa000640 1 -- 172.21.15.67:0/3533254825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa3f40cafb0 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.382+0000 7fa3eb400640 1 -- 172.21.15.67:0/3533254825 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fa3e4061610 con 0x7fa3f40c7bc0 2024-03-09T09:27:54.390 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":280,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:27:54.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.385+0000 7fa3fa000640 1 -- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3d406b5a0 msgr2=0x7fa3d406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:54.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.385+0000 7fa3fa000640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3d406b5a0 0x7fa3d406da60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40abd70 tx=0x7fa3e0009040 comp rx=0 tx=0).stop 2024-03-09T09:27:54.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.385+0000 7fa3fa000640 1 -- 172.21.15.67:0/3533254825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f40c7bc0 msgr2=0x7fa3f41e2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.385+0000 7fa3fa000640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f40c7bc0 0x7fa3f41e2e70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4001c40 tx=0x7fa3e4014420 comp rx=0 tx=0).stop 2024-03-09T09:27:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.386+0000 7fa3fa000640 1 -- 172.21.15.67:0/3533254825 shutdown_connections 2024-03-09T09:27:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.386+0000 7fa3fa000640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa3d406b5a0 0x7fa3d406da60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.386+0000 7fa3fa000640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa3f414bae0 0x7fa3f40ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.386+0000 7fa3fa000640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa3f4153500 0x7fa3f40ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.386+0000 7fa3fa000640 1 --2- 172.21.15.67:0/3533254825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa3f40c7bc0 0x7fa3f41e2e70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.386+0000 7fa3fa000640 1 -- 172.21.15.67:0/3533254825 >> 172.21.15.67:0/3533254825 conn(0x7fa3f40c8060 msgr2=0x7fa3f40c5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.386+0000 7fa3fa000640 1 -- 172.21.15.67:0/3533254825 shutdown_connections 2024-03-09T09:27:54.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:54.386+0000 7fa3fa000640 1 -- 172.21.15.67:0/3533254825 wait complete. 2024-03-09T09:27:54.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mkdir-p.m4' 2024-03-09T09:27:54.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/xvasprintf.m4' 2024-03-09T09:27:54.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strpbrk.m4' 2024-03-09T09:27:54.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/rename.m4' 2024-03-09T09:27:54.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/wchar_t.m4' 2024-03-09T09:27:54.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/glibc21.m4' 2024-03-09T09:27:54.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/perror.m4' 2024-03-09T09:27:54.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/backupfile.m4' 2024-03-09T09:27:54.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/xstrtod.m4' 2024-03-09T09:27:54.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/xattr.m4' 2024-03-09T09:27:54.433 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fsusage.m4' 2024-03-09T09:27:54.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/human.m4' 2024-03-09T09:27:54.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mbsrtowcs.m4' 2024-03-09T09:27:54.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/tm_gmtoff.m4' 2024-03-09T09:27:54.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strtod.m4' 2024-03-09T09:27:54.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/selinux-context-h.m4' 2024-03-09T09:27:54.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strtoumax.m4' 2024-03-09T09:27:54.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/inttypes-pri.m4' 2024-03-09T09:27:54.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/bison.m4' 2024-03-09T09:27:54.434 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/filenamecat.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/i-ring.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strstr.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/calloc.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getpass.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/error.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/vasnprintf.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/exponentd.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/unlocked-io.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/nocrash.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fstypename.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/gethrxtime.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/vasprintf.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/isblank.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/jm-winsz1.m4' 2024-03-09T09:27:54.435 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/memmove.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/unlinkdir.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sockets.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/savewd.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/frexpl.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fopen.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/unistd-safer.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/utimecmp.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mempcpy.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mpsort.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sys_stat_h.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/exponentl.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sha512.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/time_r.m4' 2024-03-09T09:27:54.436 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sys_select_h.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/vprintf-posix.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/malloc.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/tzset.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/memxfrm.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/flexmember.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fcntl-o.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/getpagesize.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/unlink-busy.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/fchdir.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/malloca.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/lib-ld.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/euidaccess-stat.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/xgetcwd.m4' 2024-03-09T09:27:54.437 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/dirent_h.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/dup2.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/locale_h.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/stat-time.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/select.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/same.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/freading.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strtoll.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/group-member.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/inttypes_h.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/long-options.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/chown.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/xalloc.m4' 2024-03-09T09:27:54.438 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/nanosleep.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/utimes.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/size_max.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/frexp.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/exclude.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/wint_t.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strndup.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/lib-check.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mbiter.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/mbstate_t.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/rmdir.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/inet_ntop.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/version-etc.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/isnanl.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/signbit.m4' 2024-03-09T09:27:54.439 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/strtoul.m4' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/m4/sleep.m4' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/m4' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.version' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_prohibit_fail_0' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/Makefile.am' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/INSTALL' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/maint.mk' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/bootstrap.conf' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtoimax.c' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-update-copyright.sh' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/connect.c' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-signbit.c' 2024-03-09T09:27:54.440 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-stdlib.c' 2024-03-09T09:27:54.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-wcrtomb.c' 2024-03-09T09:27:54.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-sameacls.c' 2024-03-09T09:27:54.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnanf-nolibm.c' 2024-03-09T09:27:54.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-wcwidth.c' 2024-03-09T09:27:54.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-argmatch.c' 2024-03-09T09:27:54.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-dirname.c' 2024-03-09T09:27:54.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fseeko.c' 2024-03-09T09:27:54.836 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_select.c' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-freopen-safer.c' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbrtowc.c' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fseterr.c' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-vasprintf.c' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fopen.h' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-ftello.sh' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsinit.sh' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-dup-safer.c' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsstr3.c' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-vprintf-posix.sh' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-userspec.c' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnand.h' 2024-03-09T09:27:54.837 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-freopen.c' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-file-has-acl.sh' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbrtowc4.sh' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsrtowcs2.sh' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fseeko.sh' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/nan.h' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_ioctl.c' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-btowc.c' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-getopt.h' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mkdirat.c' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-stdint.c' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-c-strcasecmp.c' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsstr3.sh' 2024-03-09T09:27:54.838 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-readlink.h' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbrtowc1.sh' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude.c' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadptr.c' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fflush2.sh' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-inttypes.c' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_utsname.c' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-math.c' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/signature.h' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/w32sock.h' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-wchar.c' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.c' 2024-03-09T09:27:54.839 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/uniwidth/test-uc_width.c' 2024-03-09T09:27:54.840 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.sh' 2024-03-09T09:27:54.840 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/gnulib-tests/uniwidth' 2024-03-09T09:27:54.840 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-lstat.c' 2024-03-09T09:27:54.840 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/unlinkdir.h' 2024-03-09T09:27:54.840 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbscasecmp.c' 2024-03-09T09:27:54.840 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-netdb.c' 2024-03-09T09:27:54.840 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-stat.c' 2024-03-09T09:27:54.840 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-open.c' 2024-03-09T09:27:54.840 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-strsignal.c' 2024-03-09T09:27:54.840 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/accept.c' 2024-03-09T09:27:54.840 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xfprintf-posix.c' 2024-03-09T09:27:54.841 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-sockets.c' 2024-03-09T09:27:54.841 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xalloc-die.c' 2024-03-09T09:27:54.841 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fchdir.c' 2024-03-09T09:27:54.841 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-sigaction.c' 2024-03-09T09:27:54.841 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-i-ring.c' 2024-03-09T09:27:54.841 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-read-file.c' 2024-03-09T09:27:54.841 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-unlinkat.c' 2024-03-09T09:27:54.841 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-stddef.c' 2024-03-09T09:27:54.841 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-linkat.c' 2024-03-09T09:27:54.841 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-openat-safer.c' 2024-03-09T09:27:54.841 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-gettimeofday.c' 2024-03-09T09:27:54.842 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-remove.c' 2024-03-09T09:27:54.842 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-strtod.c' 2024-03-09T09:27:54.842 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-unlink.h' 2024-03-09T09:27:54.842 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude6.sh' 2024-03-09T09:27:54.842 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-argv-iter.c' 2024-03-09T09:27:54.842 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-lstat.h' 2024-03-09T09:27:54.842 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/unistr/test-u8-uctomb.c' 2024-03-09T09:27:54.842 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/gnulib-tests/unistr' 2024-03-09T09:27:54.842 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/setsockopt.c' 2024-03-09T09:27:55.235 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man' -> './k/coreutils-8.5/man' 2024-03-09T09:27:55.235 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/date.x' -> './k/coreutils-8.5/man/date.x' 2024-03-09T09:27:55.236 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/date.x’: No data available 2024-03-09T09:27:55.238 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/basename.1' -> './k/coreutils-8.5/man/basename.1' 2024-03-09T09:27:55.238 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/basename.1’: No data available 2024-03-09T09:27:55.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/pathchk.x' -> './k/coreutils-8.5/man/pathchk.x' 2024-03-09T09:27:55.240 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pathchk.x’: No data available 2024-03-09T09:27:55.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tail.x' -> './k/coreutils-8.5/man/tail.x' 2024-03-09T09:27:55.243 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tail.x’: No data available 2024-03-09T09:27:55.245 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/yes.x' -> './k/coreutils-8.5/man/yes.x' 2024-03-09T09:27:55.245 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/yes.x’: No data available 2024-03-09T09:27:55.248 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/pinky.1' -> './k/coreutils-8.5/man/pinky.1' 2024-03-09T09:27:55.248 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pinky.1’: No data available 2024-03-09T09:27:55.250 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/kill.x' -> './k/coreutils-8.5/man/kill.x' 2024-03-09T09:27:55.250 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/kill.x’: No data available 2024-03-09T09:27:55.253 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/pathchk.1' -> './k/coreutils-8.5/man/pathchk.1' 2024-03-09T09:27:55.253 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pathchk.1’: No data available 2024-03-09T09:27:55.257 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sync.1' -> './k/coreutils-8.5/man/sync.1' 2024-03-09T09:27:55.257 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sync.1’: No data available 2024-03-09T09:27:55.260 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/dir.1' -> './k/coreutils-8.5/man/dir.1' 2024-03-09T09:27:55.260 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dir.1’: No data available 2024-03-09T09:27:55.262 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/mv.1' -> './k/coreutils-8.5/man/mv.1' 2024-03-09T09:27:55.263 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mv.1’: No data available 2024-03-09T09:27:55.270 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/rm.1' -> './k/coreutils-8.5/man/rm.1' 2024-03-09T09:27:55.271 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/rm.1’: No data available 2024-03-09T09:27:55.272 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/dircolors.x' -> './k/coreutils-8.5/man/dircolors.x' 2024-03-09T09:27:55.273 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dircolors.x’: No data available 2024-03-09T09:27:55.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude3.sh' 2024-03-09T09:27:55.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/ioctl.c' 2024-03-09T09:27:55.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.po' 2024-03-09T09:27:55.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.mo' 2024-03-09T09:27:55.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES' 2024-03-09T09:27:55.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/gnulib-tests/locale/fr' 2024-03-09T09:27:55.273 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/gnulib-tests/locale' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-c-strncasecmp.c' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fpurge.c' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude5.sh' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-arpa_inet.c' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fopen-safer.c' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-nl_langinfo.c' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-time.c' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-netinet_in.c' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fopen.c' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fnmatch.c' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/bind.c' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadahead.sh' 2024-03-09T09:27:55.274 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-lutimens.h' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-getndelim2.c' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-printf-posix.h' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xprintf-posix.c' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-environ.c' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-uname.c' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/glthread/thread.h' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/glthread/thread.c' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/glthread/yield.h' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/gnulib-tests/glthread' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/gnulib.mk' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-areadlink.c' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-link.h' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-nl_langinfo.sh' 2024-03-09T09:27:55.275 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-quotearg.c' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-vasnprintf.c' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-atexit.sh' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/sleep.c' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fpending.c' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-isblank.c' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xalloc-die.sh' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-wctype.c' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-utimens.h' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-file-has-acl.c' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-set-mode-acl.sh' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-btowc2.sh' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_time.c' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-areadlink.h' 2024-03-09T09:27:55.276 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadptr2.sh' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsrtowcs3.sh' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-vfprintf-posix.c' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fsync.c' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-vc-list-files-cvs.sh' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-alloca-opt.c' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-yesno.sh' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-memrchr.c' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-c-strcase.sh' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-select-out.sh' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-binary-io.sh' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-wcrtomb.sh' 2024-03-09T09:27:55.277 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-ftello2.sh' 2024-03-09T09:27:55.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mkdir.h' 2024-03-09T09:27:55.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtoul.c' 2024-03-09T09:27:55.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-printf-frexp.c' 2024-03-09T09:27:55.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-snprintf.c' 2024-03-09T09:27:55.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnand-nolibm.c' 2024-03-09T09:27:55.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-readlink.c' 2024-03-09T09:27:55.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-sleep.c' 2024-03-09T09:27:55.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/usleep.c' 2024-03-09T09:27:55.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtoimax.sh' 2024-03-09T09:27:55.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-stdbool.c' 2024-03-09T09:27:55.278 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-frexpl.c' 2024-03-09T09:27:55.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-alignof.c' 2024-03-09T09:27:55.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fflush.c' 2024-03-09T09:27:55.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-ctype.c' 2024-03-09T09:27:55.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/socket.c' 2024-03-09T09:27:55.279 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-yesno.c' 2024-03-09T09:27:55.279 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/rmdir.1' -> './k/coreutils-8.5/man/rmdir.1' 2024-03-09T09:27:55.280 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/rmdir.1’: No data available 2024-03-09T09:27:55.280 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tty.1' -> './k/coreutils-8.5/man/tty.1' 2024-03-09T09:27:55.280 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tty.1’: No data available 2024-03-09T09:27:55.281 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/fold.x' -> './k/coreutils-8.5/man/fold.x' 2024-03-09T09:27:55.281 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/fold.x’: No data available 2024-03-09T09:27:55.286 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/fold.1' -> './k/coreutils-8.5/man/fold.1' 2024-03-09T09:27:55.286 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/fold.1’: No data available 2024-03-09T09:27:55.287 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/dir.x' -> './k/coreutils-8.5/man/dir.x' 2024-03-09T09:27:55.287 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dir.x’: No data available 2024-03-09T09:27:55.287 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/fmt.x' -> './k/coreutils-8.5/man/fmt.x' 2024-03-09T09:27:55.287 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/fmt.x’: No data available 2024-03-09T09:27:55.289 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/paste.1' -> './k/coreutils-8.5/man/paste.1' 2024-03-09T09:27:55.289 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/paste.1’: No data available 2024-03-09T09:27:55.291 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/chmod.x' -> './k/coreutils-8.5/man/chmod.x' 2024-03-09T09:27:55.291 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chmod.x’: No data available 2024-03-09T09:27:55.294 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/od.1' -> './k/coreutils-8.5/man/od.1' 2024-03-09T09:27:55.294 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/od.1’: No data available 2024-03-09T09:27:55.296 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/ptx.x' -> './k/coreutils-8.5/man/ptx.x' 2024-03-09T09:27:55.296 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ptx.x’: No data available 2024-03-09T09:27:55.299 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/nice.1' -> './k/coreutils-8.5/man/nice.1' 2024-03-09T09:27:55.299 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nice.1’: No data available 2024-03-09T09:27:55.301 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/install.1' -> './k/coreutils-8.5/man/install.1' 2024-03-09T09:27:55.301 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/install.1’: No data available 2024-03-09T09:27:55.303 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/dd.x' -> './k/coreutils-8.5/man/dd.x' 2024-03-09T09:27:55.303 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dd.x’: No data available 2024-03-09T09:27:55.305 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/cksum.1' -> './k/coreutils-8.5/man/cksum.1' 2024-03-09T09:27:55.305 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cksum.1’: No data available 2024-03-09T09:27:55.307 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/test.1' -> './k/coreutils-8.5/man/test.1' 2024-03-09T09:27:55.308 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/test.1’: No data available 2024-03-09T09:27:55.310 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/chgrp.1' -> './k/coreutils-8.5/man/chgrp.1' 2024-03-09T09:27:55.310 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chgrp.1’: No data available 2024-03-09T09:27:55.312 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/unlink.1' -> './k/coreutils-8.5/man/unlink.1' 2024-03-09T09:27:55.312 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/unlink.1’: No data available 2024-03-09T09:27:55.315 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/chmod.1' -> './k/coreutils-8.5/man/chmod.1' 2024-03-09T09:27:55.315 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chmod.1’: No data available 2024-03-09T09:27:55.317 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/dd.1' -> './k/coreutils-8.5/man/dd.1' 2024-03-09T09:27:55.317 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dd.1’: No data available 2024-03-09T09:27:55.319 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/du.1' -> './k/coreutils-8.5/man/du.1' 2024-03-09T09:27:55.319 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/du.1’: No data available 2024-03-09T09:27:55.321 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tsort.1' -> './k/coreutils-8.5/man/tsort.1' 2024-03-09T09:27:55.321 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tsort.1’: No data available 2024-03-09T09:27:55.323 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/ls.1' -> './k/coreutils-8.5/man/ls.1' 2024-03-09T09:27:55.323 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ls.1’: No data available 2024-03-09T09:27:55.325 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/split.x' -> './k/coreutils-8.5/man/split.x' 2024-03-09T09:27:55.325 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/split.x’: No data available 2024-03-09T09:27:55.326 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/nohup.x' -> './k/coreutils-8.5/man/nohup.x' 2024-03-09T09:27:55.326 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nohup.x’: No data available 2024-03-09T09:27:55.328 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sha1sum.1' -> './k/coreutils-8.5/man/sha1sum.1' 2024-03-09T09:27:55.329 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha1sum.1’: No data available 2024-03-09T09:27:55.330 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tr.1' -> './k/coreutils-8.5/man/tr.1' 2024-03-09T09:27:55.330 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tr.1’: No data available 2024-03-09T09:27:55.332 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/head.x' -> './k/coreutils-8.5/man/head.x' 2024-03-09T09:27:55.332 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/head.x’: No data available 2024-03-09T09:27:55.334 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/false.1' -> './k/coreutils-8.5/man/false.1' 2024-03-09T09:27:55.335 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/false.1’: No data available 2024-03-09T09:27:55.336 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/pwd.x' -> './k/coreutils-8.5/man/pwd.x' 2024-03-09T09:27:55.337 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pwd.x’: No data available 2024-03-09T09:27:55.339 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/logname.x' -> './k/coreutils-8.5/man/logname.x' 2024-03-09T09:27:55.339 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/logname.x’: No data available 2024-03-09T09:27:55.341 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/paste.x' -> './k/coreutils-8.5/man/paste.x' 2024-03-09T09:27:55.341 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/paste.x’: No data available 2024-03-09T09:27:55.344 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/nl.x' -> './k/coreutils-8.5/man/nl.x' 2024-03-09T09:27:55.344 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nl.x’: No data available 2024-03-09T09:27:55.346 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sha224sum.1' -> './k/coreutils-8.5/man/sha224sum.1' 2024-03-09T09:27:55.346 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha224sum.1’: No data available 2024-03-09T09:27:55.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/mkdir.1' -> './k/coreutils-8.5/man/mkdir.1' 2024-03-09T09:27:55.349 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mkdir.1’: No data available 2024-03-09T09:27:55.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/nl.1' -> './k/coreutils-8.5/man/nl.1' 2024-03-09T09:27:55.351 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nl.1’: No data available 2024-03-09T09:27:55.353 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sha256sum.1' -> './k/coreutils-8.5/man/sha256sum.1' 2024-03-09T09:27:55.353 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha256sum.1’: No data available 2024-03-09T09:27:55.355 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/printf.x' -> './k/coreutils-8.5/man/printf.x' 2024-03-09T09:27:55.355 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/printf.x’: No data available 2024-03-09T09:27:55.358 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/md5sum.1' -> './k/coreutils-8.5/man/md5sum.1' 2024-03-09T09:27:55.358 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/md5sum.1’: No data available 2024-03-09T09:27:55.360 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/Makefile.am' -> './k/coreutils-8.5/man/Makefile.am' 2024-03-09T09:27:55.360 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/Makefile.am’: No data available 2024-03-09T09:27:55.362 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/dirname.1' -> './k/coreutils-8.5/man/dirname.1' 2024-03-09T09:27:55.363 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dirname.1’: No data available 2024-03-09T09:27:55.364 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/shuf.x' -> './k/coreutils-8.5/man/shuf.x' 2024-03-09T09:27:55.365 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/shuf.x’: No data available 2024-03-09T09:27:55.367 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/echo.1' -> './k/coreutils-8.5/man/echo.1' 2024-03-09T09:27:55.367 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/echo.1’: No data available 2024-03-09T09:27:55.369 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/chcon.x' -> './k/coreutils-8.5/man/chcon.x' 2024-03-09T09:27:55.370 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chcon.x’: No data available 2024-03-09T09:27:55.371 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/chroot.1' -> './k/coreutils-8.5/man/chroot.1' 2024-03-09T09:27:55.372 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chroot.1’: No data available 2024-03-09T09:27:55.374 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sha384sum.x' -> './k/coreutils-8.5/man/sha384sum.x' 2024-03-09T09:27:55.374 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha384sum.x’: No data available 2024-03-09T09:27:55.375 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/stdbuf.1' -> './k/coreutils-8.5/man/stdbuf.1' 2024-03-09T09:27:55.376 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stdbuf.1’: No data available 2024-03-09T09:27:55.378 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/users.1' -> './k/coreutils-8.5/man/users.1' 2024-03-09T09:27:55.378 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/users.1’: No data available 2024-03-09T09:27:55.380 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/readlink.x' -> './k/coreutils-8.5/man/readlink.x' 2024-03-09T09:27:55.380 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/readlink.x’: No data available 2024-03-09T09:27:55.382 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/cut.x' -> './k/coreutils-8.5/man/cut.x' 2024-03-09T09:27:55.382 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cut.x’: No data available 2024-03-09T09:27:55.385 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/groups.1' -> './k/coreutils-8.5/man/groups.1' 2024-03-09T09:27:55.385 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/groups.1’: No data available 2024-03-09T09:27:55.387 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/seq.1' -> './k/coreutils-8.5/man/seq.1' 2024-03-09T09:27:55.387 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/seq.1’: No data available 2024-03-09T09:27:55.389 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/help2man' -> './k/coreutils-8.5/man/help2man' 2024-03-09T09:27:55.389 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/help2man’: No data available 2024-03-09T09:27:55.391 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/dirname.x' -> './k/coreutils-8.5/man/dirname.x' 2024-03-09T09:27:55.391 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dirname.x’: No data available 2024-03-09T09:27:55.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/uname.x' -> './k/coreutils-8.5/man/uname.x' 2024-03-09T09:27:55.394 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uname.x’: No data available 2024-03-09T09:27:55.396 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/uptime.x' -> './k/coreutils-8.5/man/uptime.x' 2024-03-09T09:27:55.396 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uptime.x’: No data available 2024-03-09T09:27:55.400 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/cut.1' -> './k/coreutils-8.5/man/cut.1' 2024-03-09T09:27:55.400 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cut.1’: No data available 2024-03-09T09:27:55.403 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/cksum.x' -> './k/coreutils-8.5/man/cksum.x' 2024-03-09T09:27:55.403 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cksum.x’: No data available 2024-03-09T09:27:55.406 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/nohup.1' -> './k/coreutils-8.5/man/nohup.1' 2024-03-09T09:27:55.407 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nohup.1’: No data available 2024-03-09T09:27:55.408 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/mv.x' -> './k/coreutils-8.5/man/mv.x' 2024-03-09T09:27:55.409 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mv.x’: No data available 2024-03-09T09:27:55.411 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sort.1' -> './k/coreutils-8.5/man/sort.1' 2024-03-09T09:27:55.411 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sort.1’: No data available 2024-03-09T09:27:55.413 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/chown.1' -> './k/coreutils-8.5/man/chown.1' 2024-03-09T09:27:55.413 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chown.1’: No data available 2024-03-09T09:27:55.413 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/pr.x' -> './k/coreutils-8.5/man/pr.x' 2024-03-09T09:27:55.414 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pr.x’: No data available 2024-03-09T09:27:55.416 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/true.x' -> './k/coreutils-8.5/man/true.x' 2024-03-09T09:27:55.416 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/true.x’: No data available 2024-03-09T09:27:55.418 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/base64.1' -> './k/coreutils-8.5/man/base64.1' 2024-03-09T09:27:55.418 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/base64.1’: No data available 2024-03-09T09:27:55.422 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/factor.1' -> './k/coreutils-8.5/man/factor.1' 2024-03-09T09:27:55.422 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/factor.1’: No data available 2024-03-09T09:27:55.423 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/ln.x' -> './k/coreutils-8.5/man/ln.x' 2024-03-09T09:27:55.424 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ln.x’: No data available 2024-03-09T09:27:55.426 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/head.1' -> './k/coreutils-8.5/man/head.1' 2024-03-09T09:27:55.426 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/head.1’: No data available 2024-03-09T09:27:55.428 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/unexpand.1' -> './k/coreutils-8.5/man/unexpand.1' 2024-03-09T09:27:55.428 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/unexpand.1’: No data available 2024-03-09T09:27:55.429 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/df.x' -> './k/coreutils-8.5/man/df.x' 2024-03-09T09:27:55.429 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/df.x’: No data available 2024-03-09T09:27:55.430 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/join.x' -> './k/coreutils-8.5/man/join.x' 2024-03-09T09:27:55.430 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/join.x’: No data available 2024-03-09T09:27:55.431 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/du.x' -> './k/coreutils-8.5/man/du.x' 2024-03-09T09:27:55.431 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/du.x’: No data available 2024-03-09T09:27:55.433 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/dircolors.1' -> './k/coreutils-8.5/man/dircolors.1' 2024-03-09T09:27:55.433 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dircolors.1’: No data available 2024-03-09T09:27:55.434 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/uniq.x' -> './k/coreutils-8.5/man/uniq.x' 2024-03-09T09:27:55.434 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uniq.x’: No data available 2024-03-09T09:27:55.435 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/split.1' -> './k/coreutils-8.5/man/split.1' 2024-03-09T09:27:55.435 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/split.1’: No data available 2024-03-09T09:27:55.436 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tty.x' -> './k/coreutils-8.5/man/tty.x' 2024-03-09T09:27:55.436 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tty.x’: No data available 2024-03-09T09:27:55.437 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/uname.1' -> './k/coreutils-8.5/man/uname.1' 2024-03-09T09:27:55.438 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uname.1’: No data available 2024-03-09T09:27:55.439 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/uptime.1' -> './k/coreutils-8.5/man/uptime.1' 2024-03-09T09:27:55.439 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uptime.1’: No data available 2024-03-09T09:27:55.440 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sort.x' -> './k/coreutils-8.5/man/sort.x' 2024-03-09T09:27:55.440 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sort.x’: No data available 2024-03-09T09:27:55.441 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sha256sum.x' -> './k/coreutils-8.5/man/sha256sum.x' 2024-03-09T09:27:55.442 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha256sum.x’: No data available 2024-03-09T09:27:55.442 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sha512sum.x' -> './k/coreutils-8.5/man/sha512sum.x' 2024-03-09T09:27:55.443 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha512sum.x’: No data available 2024-03-09T09:27:55.444 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/nice.x' -> './k/coreutils-8.5/man/nice.x' 2024-03-09T09:27:55.444 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nice.x’: No data available 2024-03-09T09:27:55.446 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/pinky.x' -> './k/coreutils-8.5/man/pinky.x' 2024-03-09T09:27:55.446 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pinky.x’: No data available 2024-03-09T09:27:55.449 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/fmt.1' -> './k/coreutils-8.5/man/fmt.1' 2024-03-09T09:27:55.449 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/fmt.1’: No data available 2024-03-09T09:27:55.451 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/expr.1' -> './k/coreutils-8.5/man/expr.1' 2024-03-09T09:27:55.451 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/expr.1’: No data available 2024-03-09T09:27:55.453 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/wc.1' -> './k/coreutils-8.5/man/wc.1' 2024-03-09T09:27:55.453 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/wc.1’: No data available 2024-03-09T09:27:55.455 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/true.1' -> './k/coreutils-8.5/man/true.1' 2024-03-09T09:27:55.455 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/true.1’: No data available 2024-03-09T09:27:55.457 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/comm.x' -> './k/coreutils-8.5/man/comm.x' 2024-03-09T09:27:55.457 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/comm.x’: No data available 2024-03-09T09:27:55.458 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/truncate.x' -> './k/coreutils-8.5/man/truncate.x' 2024-03-09T09:27:55.458 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/truncate.x’: No data available 2024-03-09T09:27:55.459 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/install.x' -> './k/coreutils-8.5/man/install.x' 2024-03-09T09:27:55.459 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/install.x’: No data available 2024-03-09T09:27:55.461 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/wc.x' -> './k/coreutils-8.5/man/wc.x' 2024-03-09T09:27:55.461 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/wc.x’: No data available 2024-03-09T09:27:55.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/uniq.1' -> './k/coreutils-8.5/man/uniq.1' 2024-03-09T09:27:55.464 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uniq.1’: No data available 2024-03-09T09:27:55.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/arch.x' -> './k/coreutils-8.5/man/arch.x' 2024-03-09T09:27:55.464 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/arch.x’: No data available 2024-03-09T09:27:55.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sum.x' -> './k/coreutils-8.5/man/sum.x' 2024-03-09T09:27:55.465 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sum.x’: No data available 2024-03-09T09:27:55.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/timeout.x' -> './k/coreutils-8.5/man/timeout.x' 2024-03-09T09:27:55.466 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/timeout.x’: No data available 2024-03-09T09:27:55.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/link.x' -> './k/coreutils-8.5/man/link.x' 2024-03-09T09:27:55.467 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/link.x’: No data available 2024-03-09T09:27:55.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tac.1' -> './k/coreutils-8.5/man/tac.1' 2024-03-09T09:27:55.468 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tac.1’: No data available 2024-03-09T09:27:55.470 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sync.x' -> './k/coreutils-8.5/man/sync.x' 2024-03-09T09:27:55.470 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sync.x’: No data available 2024-03-09T09:27:55.472 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/Makefile.in' -> './k/coreutils-8.5/man/Makefile.in' 2024-03-09T09:27:55.472 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/Makefile.in’: No data available 2024-03-09T09:27:55.473 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/mkfifo.1' -> './k/coreutils-8.5/man/mkfifo.1' 2024-03-09T09:27:55.474 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mkfifo.1’: No data available 2024-03-09T09:27:55.475 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/od.x' -> './k/coreutils-8.5/man/od.x' 2024-03-09T09:27:55.475 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/od.x’: No data available 2024-03-09T09:27:55.476 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/vdir.1' -> './k/coreutils-8.5/man/vdir.1' 2024-03-09T09:27:55.476 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/vdir.1’: No data available 2024-03-09T09:27:55.478 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/timeout.1' -> './k/coreutils-8.5/man/timeout.1' 2024-03-09T09:27:55.478 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/timeout.1’: No data available 2024-03-09T09:27:55.479 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/mkfifo.x' -> './k/coreutils-8.5/man/mkfifo.x' 2024-03-09T09:27:55.479 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mkfifo.x’: No data available 2024-03-09T09:27:55.481 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/basename.x' -> './k/coreutils-8.5/man/basename.x' 2024-03-09T09:27:55.481 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/basename.x’: No data available 2024-03-09T09:27:55.482 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/expr.x' -> './k/coreutils-8.5/man/expr.x' 2024-03-09T09:27:55.482 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/expr.x’: No data available 2024-03-09T09:27:55.483 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/mktemp.x' -> './k/coreutils-8.5/man/mktemp.x' 2024-03-09T09:27:55.484 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mktemp.x’: No data available 2024-03-09T09:27:55.485 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/vdir.x' -> './k/coreutils-8.5/man/vdir.x' 2024-03-09T09:27:55.485 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/vdir.x’: No data available 2024-03-09T09:27:55.486 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/su.x' -> './k/coreutils-8.5/man/su.x' 2024-03-09T09:27:55.486 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/su.x’: No data available 2024-03-09T09:27:55.490 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/env.1' -> './k/coreutils-8.5/man/env.1' 2024-03-09T09:27:55.490 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/env.1’: No data available 2024-03-09T09:27:55.490 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/who.x' -> './k/coreutils-8.5/man/who.x' 2024-03-09T09:27:55.490 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/who.x’: No data available 2024-03-09T09:27:55.491 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/hostid.1' -> './k/coreutils-8.5/man/hostid.1' 2024-03-09T09:27:55.491 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/hostid.1’: No data available 2024-03-09T09:27:55.492 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tee.x' -> './k/coreutils-8.5/man/tee.x' 2024-03-09T09:27:55.492 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tee.x’: No data available 2024-03-09T09:27:55.493 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/rmdir.x' -> './k/coreutils-8.5/man/rmdir.x' 2024-03-09T09:27:55.493 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/rmdir.x’: No data available 2024-03-09T09:27:55.494 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/hostid.x' -> './k/coreutils-8.5/man/hostid.x' 2024-03-09T09:27:55.494 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/hostid.x’: No data available 2024-03-09T09:27:55.495 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/id.x' -> './k/coreutils-8.5/man/id.x' 2024-03-09T09:27:55.495 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/id.x’: No data available 2024-03-09T09:27:55.496 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/unexpand.x' -> './k/coreutils-8.5/man/unexpand.x' 2024-03-09T09:27:55.496 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/unexpand.x’: No data available 2024-03-09T09:27:55.498 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/pr.1' -> './k/coreutils-8.5/man/pr.1' 2024-03-09T09:27:55.498 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pr.1’: No data available 2024-03-09T09:27:55.499 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/csplit.1' -> './k/coreutils-8.5/man/csplit.1' 2024-03-09T09:27:55.499 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/csplit.1’: No data available 2024-03-09T09:27:55.501 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/truncate.1' -> './k/coreutils-8.5/man/truncate.1' 2024-03-09T09:27:55.501 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/truncate.1’: No data available 2024-03-09T09:27:55.502 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tee.1' -> './k/coreutils-8.5/man/tee.1' 2024-03-09T09:27:55.503 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tee.1’: No data available 2024-03-09T09:27:55.503 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/env.x' -> './k/coreutils-8.5/man/env.x' 2024-03-09T09:27:55.504 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/env.x’: No data available 2024-03-09T09:27:55.505 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/chroot.x' -> './k/coreutils-8.5/man/chroot.x' 2024-03-09T09:27:55.505 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chroot.x’: No data available 2024-03-09T09:27:55.506 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/cp.x' -> './k/coreutils-8.5/man/cp.x' 2024-03-09T09:27:55.507 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cp.x’: No data available 2024-03-09T09:27:55.508 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/whoami.x' -> './k/coreutils-8.5/man/whoami.x' 2024-03-09T09:27:55.508 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/whoami.x’: No data available 2024-03-09T09:27:55.509 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/logname.1' -> './k/coreutils-8.5/man/logname.1' 2024-03-09T09:27:55.509 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/logname.1’: No data available 2024-03-09T09:27:55.511 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/mknod.1' -> './k/coreutils-8.5/man/mknod.1' 2024-03-09T09:27:55.511 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mknod.1’: No data available 2024-03-09T09:27:55.512 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/groups.x' -> './k/coreutils-8.5/man/groups.x' 2024-03-09T09:27:55.512 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/groups.x’: No data available 2024-03-09T09:27:55.514 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/cp.1' -> './k/coreutils-8.5/man/cp.1' 2024-03-09T09:27:55.514 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cp.1’: No data available 2024-03-09T09:27:55.515 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/runcon.x' -> './k/coreutils-8.5/man/runcon.x' 2024-03-09T09:27:55.516 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/runcon.x’: No data available 2024-03-09T09:27:55.517 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/yes.1' -> './k/coreutils-8.5/man/yes.1' 2024-03-09T09:27:55.517 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/yes.1’: No data available 2024-03-09T09:27:55.518 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/stat.1' -> './k/coreutils-8.5/man/stat.1' 2024-03-09T09:27:55.518 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stat.1’: No data available 2024-03-09T09:27:55.520 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/nproc.x' -> './k/coreutils-8.5/man/nproc.x' 2024-03-09T09:27:55.520 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nproc.x’: No data available 2024-03-09T09:27:55.521 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/unlink.x' -> './k/coreutils-8.5/man/unlink.x' 2024-03-09T09:27:55.522 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/unlink.x’: No data available 2024-03-09T09:27:55.523 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sha384sum.1' -> './k/coreutils-8.5/man/sha384sum.1' 2024-03-09T09:27:55.523 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha384sum.1’: No data available 2024-03-09T09:27:55.525 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/nproc.1' -> './k/coreutils-8.5/man/nproc.1' 2024-03-09T09:27:55.525 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nproc.1’: No data available 2024-03-09T09:27:55.526 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/seq.x' -> './k/coreutils-8.5/man/seq.x' 2024-03-09T09:27:55.526 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/seq.x’: No data available 2024-03-09T09:27:55.527 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sha512sum.1' -> './k/coreutils-8.5/man/sha512sum.1' 2024-03-09T09:27:55.528 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha512sum.1’: No data available 2024-03-09T09:27:55.540 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/stat.x' -> './k/coreutils-8.5/man/stat.x' 2024-03-09T09:27:55.541 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stat.x’: No data available 2024-03-09T09:27:55.543 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sleep.x' -> './k/coreutils-8.5/man/sleep.x' 2024-03-09T09:27:55.543 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sleep.x’: No data available 2024-03-09T09:27:55.545 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/printenv.1' -> './k/coreutils-8.5/man/printenv.1' 2024-03-09T09:27:55.545 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/printenv.1’: No data available 2024-03-09T09:27:55.546 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tr.x' -> './k/coreutils-8.5/man/tr.x' 2024-03-09T09:27:55.546 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tr.x’: No data available 2024-03-09T09:27:55.548 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tail.1' -> './k/coreutils-8.5/man/tail.1' 2024-03-09T09:27:55.549 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tail.1’: No data available 2024-03-09T09:27:55.550 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/md5sum.x' -> './k/coreutils-8.5/man/md5sum.x' 2024-03-09T09:27:55.551 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/md5sum.x’: No data available 2024-03-09T09:27:55.552 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/link.1' -> './k/coreutils-8.5/man/link.1' 2024-03-09T09:27:55.553 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/link.1’: No data available 2024-03-09T09:27:55.555 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/echo.x' -> './k/coreutils-8.5/man/echo.x' 2024-03-09T09:27:55.555 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/echo.x’: No data available 2024-03-09T09:27:55.556 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/ptx.1' -> './k/coreutils-8.5/man/ptx.1' 2024-03-09T09:27:55.556 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ptx.1’: No data available 2024-03-09T09:27:55.558 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/factor.x' -> './k/coreutils-8.5/man/factor.x' 2024-03-09T09:27:55.558 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/factor.x’: No data available 2024-03-09T09:27:55.560 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/who.1' -> './k/coreutils-8.5/man/who.1' 2024-03-09T09:27:55.560 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/who.1’: No data available 2024-03-09T09:27:55.561 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/mktemp.1' -> './k/coreutils-8.5/man/mktemp.1' 2024-03-09T09:27:55.561 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mktemp.1’: No data available 2024-03-09T09:27:55.562 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/shred.1' -> './k/coreutils-8.5/man/shred.1' 2024-03-09T09:27:55.562 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/shred.1’: No data available 2024-03-09T09:27:55.563 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/runcon.1' -> './k/coreutils-8.5/man/runcon.1' 2024-03-09T09:27:55.563 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/runcon.1’: No data available 2024-03-09T09:27:55.565 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/stty.x' -> './k/coreutils-8.5/man/stty.x' 2024-03-09T09:27:55.565 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stty.x’: No data available 2024-03-09T09:27:55.566 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/comm.1' -> './k/coreutils-8.5/man/comm.1' 2024-03-09T09:27:55.566 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/comm.1’: No data available 2024-03-09T09:27:55.568 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/shuf.1' -> './k/coreutils-8.5/man/shuf.1' 2024-03-09T09:27:55.568 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/shuf.1’: No data available 2024-03-09T09:27:55.570 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/stty.1' -> './k/coreutils-8.5/man/stty.1' 2024-03-09T09:27:55.570 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stty.1’: No data available 2024-03-09T09:27:55.577 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/printenv.x' -> './k/coreutils-8.5/man/printenv.x' 2024-03-09T09:27:55.578 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/printenv.x’: No data available 2024-03-09T09:27:55.579 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/join.1' -> './k/coreutils-8.5/man/join.1' 2024-03-09T09:27:55.579 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/join.1’: No data available 2024-03-09T09:27:55.580 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/hostname.x' -> './k/coreutils-8.5/man/hostname.x' 2024-03-09T09:27:55.581 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/hostname.x’: No data available 2024-03-09T09:27:55.583 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/ln.1' -> './k/coreutils-8.5/man/ln.1' 2024-03-09T09:27:55.583 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ln.1’: No data available 2024-03-09T09:27:55.585 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/false.x' -> './k/coreutils-8.5/man/false.x' 2024-03-09T09:27:55.585 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/false.x’: No data available 2024-03-09T09:27:55.587 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/cat.1' -> './k/coreutils-8.5/man/cat.1' 2024-03-09T09:27:55.587 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cat.1’: No data available 2024-03-09T09:27:55.589 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/users.x' -> './k/coreutils-8.5/man/users.x' 2024-03-09T09:27:55.589 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/users.x’: No data available 2024-03-09T09:27:55.590 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/printf.1' -> './k/coreutils-8.5/man/printf.1' 2024-03-09T09:27:55.590 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/printf.1’: No data available 2024-03-09T09:27:55.591 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/ls.x' -> './k/coreutils-8.5/man/ls.x' 2024-03-09T09:27:55.591 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ls.x’: No data available 2024-03-09T09:27:55.592 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sha224sum.x' -> './k/coreutils-8.5/man/sha224sum.x' 2024-03-09T09:27:55.593 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha224sum.x’: No data available 2024-03-09T09:27:55.594 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/whoami.1' -> './k/coreutils-8.5/man/whoami.1' 2024-03-09T09:27:55.594 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/whoami.1’: No data available 2024-03-09T09:27:55.595 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/expand.1' -> './k/coreutils-8.5/man/expand.1' 2024-03-09T09:27:55.595 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/expand.1’: No data available 2024-03-09T09:27:55.597 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/chgrp.x' -> './k/coreutils-8.5/man/chgrp.x' 2024-03-09T09:27:55.597 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chgrp.x’: No data available 2024-03-09T09:27:55.598 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sleep.1' -> './k/coreutils-8.5/man/sleep.1' 2024-03-09T09:27:55.598 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sleep.1’: No data available 2024-03-09T09:27:55.600 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/csplit.x' -> './k/coreutils-8.5/man/csplit.x' 2024-03-09T09:27:55.600 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/csplit.x’: No data available 2024-03-09T09:27:55.601 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/pwd.1' -> './k/coreutils-8.5/man/pwd.1' 2024-03-09T09:27:55.601 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pwd.1’: No data available 2024-03-09T09:27:55.603 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/kill.1' -> './k/coreutils-8.5/man/kill.1' 2024-03-09T09:27:55.604 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/kill.1’: No data available 2024-03-09T09:27:55.605 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/touch.x' -> './k/coreutils-8.5/man/touch.x' 2024-03-09T09:27:55.605 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/touch.x’: No data available 2024-03-09T09:27:55.606 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/df.1' -> './k/coreutils-8.5/man/df.1' 2024-03-09T09:27:55.606 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/df.1’: No data available 2024-03-09T09:27:55.607 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/stdbuf.x' -> './k/coreutils-8.5/man/stdbuf.x' 2024-03-09T09:27:55.607 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stdbuf.x’: No data available 2024-03-09T09:27:55.609 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/mkdir.x' -> './k/coreutils-8.5/man/mkdir.x' 2024-03-09T09:27:55.609 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mkdir.x’: No data available 2024-03-09T09:27:55.610 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/id.1' -> './k/coreutils-8.5/man/id.1' 2024-03-09T09:27:55.610 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/id.1’: No data available 2024-03-09T09:27:55.611 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/mknod.x' -> './k/coreutils-8.5/man/mknod.x' 2024-03-09T09:27:55.611 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mknod.x’: No data available 2024-03-09T09:27:55.613 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sum.1' -> './k/coreutils-8.5/man/sum.1' 2024-03-09T09:27:55.613 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sum.1’: No data available 2024-03-09T09:27:55.615 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/rm.x' -> './k/coreutils-8.5/man/rm.x' 2024-03-09T09:27:55.615 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/rm.x’: No data available 2024-03-09T09:27:55.616 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/date.1' -> './k/coreutils-8.5/man/date.1' 2024-03-09T09:27:55.617 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/date.1’: No data available 2024-03-09T09:27:55.618 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/chown.x' -> './k/coreutils-8.5/man/chown.x' 2024-03-09T09:27:55.618 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chown.x’: No data available 2024-03-09T09:27:55.619 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/touch.1' -> './k/coreutils-8.5/man/touch.1' 2024-03-09T09:27:55.619 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/touch.1’: No data available 2024-03-09T09:27:55.621 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tac.x' -> './k/coreutils-8.5/man/tac.x' 2024-03-09T09:27:55.621 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tac.x’: No data available 2024-03-09T09:27:55.624 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/base64.x' -> './k/coreutils-8.5/man/base64.x' 2024-03-09T09:27:55.624 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/base64.x’: No data available 2024-03-09T09:27:55.625 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/tsort.x' -> './k/coreutils-8.5/man/tsort.x' 2024-03-09T09:27:55.625 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tsort.x’: No data available 2024-03-09T09:27:55.627 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/shred.x' -> './k/coreutils-8.5/man/shred.x' 2024-03-09T09:27:55.627 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/shred.x’: No data available 2024-03-09T09:27:55.629 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/readlink.1' -> './k/coreutils-8.5/man/readlink.1' 2024-03-09T09:27:55.630 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/readlink.1’: No data available 2024-03-09T09:27:55.631 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/sha1sum.x' -> './k/coreutils-8.5/man/sha1sum.x' 2024-03-09T09:27:55.631 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha1sum.x’: No data available 2024-03-09T09:27:55.632 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/expand.x' -> './k/coreutils-8.5/man/expand.x' 2024-03-09T09:27:55.633 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/expand.x’: No data available 2024-03-09T09:27:55.634 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/test.x' -> './k/coreutils-8.5/man/test.x' 2024-03-09T09:27:55.634 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/test.x’: No data available 2024-03-09T09:27:55.635 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/chcon.1' -> './k/coreutils-8.5/man/chcon.1' 2024-03-09T09:27:55.636 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chcon.1’: No data available 2024-03-09T09:27:55.637 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/man/cat.x' -> './k/coreutils-8.5/man/cat.x' 2024-03-09T09:27:55.637 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cat.x’: No data available 2024-03-09T09:27:56.395 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/Makefile.in' 2024-03-09T09:27:56.395 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-filenamecat.c' 2024-03-09T09:27:56.395 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-link.c' 2024-03-09T09:27:56.395 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-nanosleep.c' 2024-03-09T09:27:56.395 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_socket.c' 2024-03-09T09:27:56.395 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsrtowcs4.sh' 2024-03-09T09:27:56.395 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-copy-acl.c' 2024-03-09T09:27:56.395 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-rmdir.h' 2024-03-09T09:27:56.396 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnanl.h' 2024-03-09T09:27:56.396 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-signal.c' 2024-03-09T09:27:56.396 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbrtowc2.sh' 2024-03-09T09:27:56.396 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-iconv-h.c' 2024-03-09T09:27:56.396 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude2.sh' 2024-03-09T09:27:56.396 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-perror.sh' 2024-03-09T09:27:56.396 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-langinfo.c' 2024-03-09T09:27:56.396 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-vprintf-posix.c' 2024-03-09T09:27:56.396 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-rmdir.c' 2024-03-09T09:27:56.396 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fcntl.c' 2024-03-09T09:27:56.397 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-dirent.c' 2024-03-09T09:27:56.397 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-chown.h' 2024-03-09T09:27:56.397 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/read-file.c' 2024-03-09T09:27:56.397 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-usleep.c' 2024-03-09T09:27:56.397 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-base64.c' 2024-03-09T09:27:56.397 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-atexit.c' 2024-03-09T09:27:56.397 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-memchr2.c' 2024-03-09T09:27:56.397 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fdopendir.c' 2024-03-09T09:27:56.397 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-stdio.c' 2024-03-09T09:27:56.397 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-vfprintf-posix.sh' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-getdtablesize.c' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtoumax.c' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fpending.sh' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-version-etc.c' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-getgroups.c' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadptr2.c' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-select-fd.c' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fcntl-safer.c' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fflush2.c' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-select-in.sh' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-dup2.c' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-frexp.c' 2024-03-09T09:27:56.398 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsstr2.sh' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/Makefile.am' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-select-stdin.c' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-getdelim.c' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-btowc1.sh' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-locale.c' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-md5.c' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-rename.h' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-utimens.c' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-strerror.c' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-inet_pton.c' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-priv-set.c' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-open.h' 2024-03-09T09:27:56.399 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-unlink.c' 2024-03-09T09:27:56.400 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/macros.h' 2024-03-09T09:27:56.400 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-rename.c' 2024-03-09T09:27:56.400 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtoumax.sh' 2024-03-09T09:27:56.400 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/inet_pton.c' 2024-03-09T09:27:56.400 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-memcmp.c' 2024-03-09T09:27:56.400 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xvasprintf.c' 2024-03-09T09:27:56.400 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadahead.c' 2024-03-09T09:27:56.400 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-printf-posix.output' 2024-03-09T09:27:56.400 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-openat.c' 2024-03-09T09:27:56.400 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-c-ctype.c' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-chown.c' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-sys_stat.c' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-closein.sh' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/init.sh' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-filevercmp.c' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-getaddrinfo.c' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-areadlinkat.c' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnanl-nolibm.c' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-lchown.h' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-getline.c' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-hash.c' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-canonicalize.c' 2024-03-09T09:27:56.401 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fchownat.c' 2024-03-09T09:27:56.402 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-striconv.c' 2024-03-09T09:27:56.471 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib' -> './k/coreutils-8.5/lib' 2024-03-09T09:27:56.471 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/streq.h' -> './k/coreutils-8.5/lib/streq.h' 2024-03-09T09:27:56.472 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/streq.h’: No data available 2024-03-09T09:27:56.472 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/asprintf.c' -> './k/coreutils-8.5/lib/asprintf.c' 2024-03-09T09:27:56.472 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/asprintf.c’: No data available 2024-03-09T09:27:56.474 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strnlen1.c' -> './k/coreutils-8.5/lib/strnlen1.c' 2024-03-09T09:27:56.474 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnlen1.c’: No data available 2024-03-09T09:27:56.476 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dirfd.c' -> './k/coreutils-8.5/lib/dirfd.c' 2024-03-09T09:27:56.476 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirfd.c’: No data available 2024-03-09T09:27:56.478 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/safe-write.h' -> './k/coreutils-8.5/lib/safe-write.h' 2024-03-09T09:27:56.478 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/safe-write.h’: No data available 2024-03-09T09:27:56.479 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/snprintf.c' -> './k/coreutils-8.5/lib/snprintf.c' 2024-03-09T09:27:56.479 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/snprintf.c’: No data available 2024-03-09T09:27:56.481 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbiter.h' -> './k/coreutils-8.5/lib/mbiter.h' 2024-03-09T09:27:56.482 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbiter.h’: No data available 2024-03-09T09:27:56.483 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mountlist.c' -> './k/coreutils-8.5/lib/mountlist.c' 2024-03-09T09:27:56.484 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mountlist.c’: No data available 2024-03-09T09:27:56.485 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/propername.h' -> './k/coreutils-8.5/lib/propername.h' 2024-03-09T09:27:56.485 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/propername.h’: No data available 2024-03-09T09:27:56.487 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memset.c' -> './k/coreutils-8.5/lib/memset.c' 2024-03-09T09:27:56.487 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memset.c’: No data available 2024-03-09T09:27:56.490 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/nproc.c' -> './k/coreutils-8.5/lib/nproc.c' 2024-03-09T09:27:56.490 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/nproc.c’: No data available 2024-03-09T09:27:56.492 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getdelim.c' -> './k/coreutils-8.5/lib/getdelim.c' 2024-03-09T09:27:56.492 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getdelim.c’: No data available 2024-03-09T09:27:56.494 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/signbitl.c' -> './k/coreutils-8.5/lib/signbitl.c' 2024-03-09T09:27:56.495 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/signbitl.c’: No data available 2024-03-09T09:27:56.497 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/group-member.h' -> './k/coreutils-8.5/lib/group-member.h' 2024-03-09T09:27:56.497 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/group-member.h’: No data available 2024-03-09T09:27:56.499 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/long-options.c' -> './k/coreutils-8.5/lib/long-options.c' 2024-03-09T09:27:56.499 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/long-options.c’: No data available 2024-03-09T09:27:56.502 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fts_.h' -> './k/coreutils-8.5/lib/fts_.h' 2024-03-09T09:27:56.502 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fts_.h’: No data available 2024-03-09T09:27:56.504 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fts.c' -> './k/coreutils-8.5/lib/fts.c' 2024-03-09T09:27:56.505 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fts.c’: No data available 2024-03-09T09:27:56.506 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/verify.h' -> './k/coreutils-8.5/lib/verify.h' 2024-03-09T09:27:56.506 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/verify.h’: No data available 2024-03-09T09:27:56.508 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/str-two-way.h' -> './k/coreutils-8.5/lib/str-two-way.h' 2024-03-09T09:27:56.508 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/str-two-way.h’: No data available 2024-03-09T09:27:56.509 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/areadlink.c' -> './k/coreutils-8.5/lib/areadlink.c' 2024-03-09T09:27:56.509 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/areadlink.c’: No data available 2024-03-09T09:27:56.511 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/c-strtod.h' -> './k/coreutils-8.5/lib/c-strtod.h' 2024-03-09T09:27:56.511 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strtod.h’: No data available 2024-03-09T09:27:56.512 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sha256.h' -> './k/coreutils-8.5/lib/sha256.h' 2024-03-09T09:27:56.512 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha256.h’: No data available 2024-03-09T09:27:56.514 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stddef.in.h' -> './k/coreutils-8.5/lib/stddef.in.h' 2024-03-09T09:27:56.514 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stddef.in.h’: No data available 2024-03-09T09:27:56.516 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/randint.c' -> './k/coreutils-8.5/lib/randint.c' 2024-03-09T09:27:56.516 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randint.c’: No data available 2024-03-09T09:27:56.518 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dup-safer.c' -> './k/coreutils-8.5/lib/dup-safer.c' 2024-03-09T09:27:56.518 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dup-safer.c’: No data available 2024-03-09T09:27:56.519 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getpass.h' -> './k/coreutils-8.5/lib/getpass.h' 2024-03-09T09:27:56.519 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getpass.h’: No data available 2024-03-09T09:27:56.521 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/gettext.h' -> './k/coreutils-8.5/lib/gettext.h' 2024-03-09T09:27:56.521 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gettext.h’: No data available 2024-03-09T09:27:56.523 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/randint.h' -> './k/coreutils-8.5/lib/randint.h' 2024-03-09T09:27:56.523 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randint.h’: No data available 2024-03-09T09:27:56.525 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stdio-write.c' -> './k/coreutils-8.5/lib/stdio-write.c' 2024-03-09T09:27:56.526 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdio-write.c’: No data available 2024-03-09T09:27:56.527 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xgethostname.h' -> './k/coreutils-8.5/lib/xgethostname.h' 2024-03-09T09:27:56.527 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xgethostname.h’: No data available 2024-03-09T09:27:56.529 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/asnprintf.c' -> './k/coreutils-8.5/lib/asnprintf.c' 2024-03-09T09:27:56.529 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/asnprintf.c’: No data available 2024-03-09T09:27:56.532 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-solaris.gperf' -> './k/coreutils-8.5/lib/iconv_open-solaris.gperf' 2024-03-09T09:27:56.532 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-solaris.gperf’: No data available 2024-03-09T09:27:56.537 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stdio-impl.h' -> './k/coreutils-8.5/lib/stdio-impl.h' 2024-03-09T09:27:56.537 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdio-impl.h’: No data available 2024-03-09T09:27:56.541 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mpsort.h' -> './k/coreutils-8.5/lib/mpsort.h' 2024-03-09T09:27:56.541 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mpsort.h’: No data available 2024-03-09T09:27:56.542 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/closein.c' -> './k/coreutils-8.5/lib/closein.c' 2024-03-09T09:27:56.543 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/closein.c’: No data available 2024-03-09T09:27:56.544 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/hard-locale.c' -> './k/coreutils-8.5/lib/hard-locale.c' 2024-03-09T09:27:56.544 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hard-locale.c’: No data available 2024-03-09T09:27:56.551 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/utimecmp.c' -> './k/coreutils-8.5/lib/utimecmp.c' 2024-03-09T09:27:56.551 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/utimecmp.c’: No data available 2024-03-09T09:27:56.555 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/vasnprintf.c' -> './k/coreutils-8.5/lib/vasnprintf.c' 2024-03-09T09:27:56.555 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/vasnprintf.c’: No data available 2024-03-09T09:27:56.559 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/chown.c' -> './k/coreutils-8.5/lib/chown.c' 2024-03-09T09:27:56.559 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/chown.c’: No data available 2024-03-09T09:27:56.563 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/printf-args.c' -> './k/coreutils-8.5/lib/printf-args.c' 2024-03-09T09:27:56.563 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-args.c’: No data available 2024-03-09T09:27:56.565 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/netdb.in.h' -> './k/coreutils-8.5/lib/netdb.in.h' 2024-03-09T09:27:56.566 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/netdb.in.h’: No data available 2024-03-09T09:27:56.570 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strnlen.c' -> './k/coreutils-8.5/lib/strnlen.c' 2024-03-09T09:27:56.570 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnlen.c’: No data available 2024-03-09T09:27:56.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/errno.in.h' -> './k/coreutils-8.5/lib/errno.in.h' 2024-03-09T09:27:56.575 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/errno.in.h’: No data available 2024-03-09T09:27:56.577 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbslen.c' -> './k/coreutils-8.5/lib/mbslen.c' 2024-03-09T09:27:56.577 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbslen.c’: No data available 2024-03-09T09:27:56.580 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/obstack.h' -> './k/coreutils-8.5/lib/obstack.h' 2024-03-09T09:27:56.580 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/obstack.h’: No data available 2024-03-09T09:27:56.583 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/file-type.h' -> './k/coreutils-8.5/lib/file-type.h' 2024-03-09T09:27:56.583 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/file-type.h’: No data available 2024-03-09T09:27:56.590 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbsrtowcs.c' -> './k/coreutils-8.5/lib/mbsrtowcs.c' 2024-03-09T09:27:56.591 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsrtowcs.c’: No data available 2024-03-09T09:27:56.594 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fnmatch_loop.c' -> './k/coreutils-8.5/lib/fnmatch_loop.c' 2024-03-09T09:27:56.595 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fnmatch_loop.c’: No data available 2024-03-09T09:27:56.597 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/safe-write.c' -> './k/coreutils-8.5/lib/safe-write.c' 2024-03-09T09:27:56.597 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/safe-write.c’: No data available 2024-03-09T09:27:56.599 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mpsort.c' -> './k/coreutils-8.5/lib/mpsort.c' 2024-03-09T09:27:56.599 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mpsort.c’: No data available 2024-03-09T09:27:56.601 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/netinet_in.in.h' -> './k/coreutils-8.5/lib/netinet_in.in.h' 2024-03-09T09:27:56.601 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/netinet_in.in.h’: No data available 2024-03-09T09:27:56.601 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xfreopen.h' -> './k/coreutils-8.5/lib/xfreopen.h' 2024-03-09T09:27:56.602 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xfreopen.h’: No data available 2024-03-09T09:27:56.606 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/wctype.in.h' -> './k/coreutils-8.5/lib/wctype.in.h' 2024-03-09T09:27:56.606 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/wctype.in.h’: No data available 2024-03-09T09:27:56.608 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stripslash.c' -> './k/coreutils-8.5/lib/stripslash.c' 2024-03-09T09:27:56.608 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stripslash.c’: No data available 2024-03-09T09:27:56.609 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/closein.h' -> './k/coreutils-8.5/lib/closein.h' 2024-03-09T09:27:56.609 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/closein.h’: No data available 2024-03-09T09:27:56.612 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memchr2.c' -> './k/coreutils-8.5/lib/memchr2.c' 2024-03-09T09:27:56.612 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memchr2.c’: No data available 2024-03-09T09:27:56.614 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memcoll.c' -> './k/coreutils-8.5/lib/memcoll.c' 2024-03-09T09:27:56.615 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcoll.c’: No data available 2024-03-09T09:27:56.616 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/malloca.h' -> './k/coreutils-8.5/lib/malloca.h' 2024-03-09T09:27:56.616 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/malloca.h’: No data available 2024-03-09T09:27:56.619 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stpcpy.c' -> './k/coreutils-8.5/lib/stpcpy.c' 2024-03-09T09:27:56.620 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stpcpy.c’: No data available 2024-03-09T09:27:56.625 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/striconv.c' -> './k/coreutils-8.5/lib/striconv.c' 2024-03-09T09:27:56.625 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/striconv.c’: No data available 2024-03-09T09:27:56.633 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/basename.c' -> './k/coreutils-8.5/lib/basename.c' 2024-03-09T09:27:56.634 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/basename.c’: No data available 2024-03-09T09:27:56.636 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sha256.c' -> './k/coreutils-8.5/lib/sha256.c' 2024-03-09T09:27:56.636 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha256.c’: No data available 2024-03-09T09:27:56.642 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stdint.in.h' -> './k/coreutils-8.5/lib/stdint.in.h' 2024-03-09T09:27:56.642 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdint.in.h’: No data available 2024-03-09T09:27:56.644 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/acl-internal.h' -> './k/coreutils-8.5/lib/acl-internal.h' 2024-03-09T09:27:56.644 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/acl-internal.h’: No data available 2024-03-09T09:27:57.414 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:27:57.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-binary-io.c' 2024-03-09T09:27:57.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-malloca.c' 2024-03-09T09:27:57.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadseek.c' 2024-03-09T09:27:57.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-closein.c' 2024-03-09T09:27:57.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-unistd.c' 2024-03-09T09:27:57.426 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/listen.c' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-posixtm.c' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbrtowc3.sh' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsstr2.c' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/unlinkdir.c' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-version-etc.sh' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-lseek.sh' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/nap.h' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-gethostname.c' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-freading.c' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-areadlink-with-size.c' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadseek.sh' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude7.sh' 2024-03-09T09:27:57.427 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsalign.c' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/wctob.c' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-quotearg.sh' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbscasecmp.sh' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/binary-io.h' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-getopt.c' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-select.c' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-u64.c' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-stat.h' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-cloexec.c' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-freadptr.sh' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-lseek.c' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-perror.c' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-bitrotate.c' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-stat-time.c' 2024-03-09T09:27:57.428 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-ftello.c' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-getcwd.c' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-symlink.h' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-inet_ntop.c' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/zerosize-ptr.h' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-sha1.c' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fcntl-h.c' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fseeko2.sh' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-printf-frexpl.c' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsrtowcs.c' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-futimens.h' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mkdir.c' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsrtowcs1.sh' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude4.sh' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-exclude1.sh' 2024-03-09T09:27:57.429 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-symlink.c' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-unsetenv.c' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtol.c' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsstr1.c' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-lchown.c' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-symlinkat.c' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-set-mode-acl.c' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xprintf-posix.sh' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-lock.c' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-copy-acl.sh' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/sys_ioctl.in.h' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/read-file.h' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fstatat.c' 2024-03-09T09:27:57.430 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-xstrtol.sh' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-isnanf.h' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-getopt_long.h' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-memchr.c' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-fprintf-posix.h' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-setenv.c' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-iconv.c' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-getdate.c' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-dirent-safer.c' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-vasprintf-posix.c' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-errno.c' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-vc-list-files-git.sh' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-tls.c' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-mbsinit.c' 2024-03-09T09:27:57.431 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-string.c' 2024-03-09T09:27:57.432 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/perror.c' 2024-03-09T09:27:57.432 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/gnulib-tests/test-utimens-common.h' 2024-03-09T09:27:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/atexit.c' -> './k/coreutils-8.5/lib/atexit.c' 2024-03-09T09:27:57.531 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/atexit.c’: No data available 2024-03-09T09:27:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/size_max.h' -> './k/coreutils-8.5/lib/size_max.h' 2024-03-09T09:27:57.533 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/size_max.h’: No data available 2024-03-09T09:27:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/buffer-lcm.h' -> './k/coreutils-8.5/lib/buffer-lcm.h' 2024-03-09T09:27:57.535 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/buffer-lcm.h’: No data available 2024-03-09T09:27:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/idcache.h' -> './k/coreutils-8.5/lib/idcache.h' 2024-03-09T09:27:57.536 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/idcache.h’: No data available 2024-03-09T09:27:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/root-dev-ino.c' -> './k/coreutils-8.5/lib/root-dev-ino.c' 2024-03-09T09:27:57.538 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/root-dev-ino.c’: No data available 2024-03-09T09:27:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/physmem.c' -> './k/coreutils-8.5/lib/physmem.c' 2024-03-09T09:27:57.540 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/physmem.c’: No data available 2024-03-09T09:27:57.545 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mkancesdirs.c' -> './k/coreutils-8.5/lib/mkancesdirs.c' 2024-03-09T09:27:57.545 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkancesdirs.c’: No data available 2024-03-09T09:27:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/readtokens0.c' -> './k/coreutils-8.5/lib/readtokens0.c' 2024-03-09T09:27:57.548 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readtokens0.c’: No data available 2024-03-09T09:27:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fnmatch.in.h' -> './k/coreutils-8.5/lib/fnmatch.in.h' 2024-03-09T09:27:57.558 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fnmatch.in.h’: No data available 2024-03-09T09:27:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dirname.h' -> './k/coreutils-8.5/lib/dirname.h' 2024-03-09T09:27:57.559 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirname.h’: No data available 2024-03-09T09:27:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/regex_internal.c' -> './k/coreutils-8.5/lib/regex_internal.c' 2024-03-09T09:27:57.561 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regex_internal.c’: No data available 2024-03-09T09:27:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strcspn.c' -> './k/coreutils-8.5/lib/strcspn.c' 2024-03-09T09:27:57.562 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strcspn.c’: No data available 2024-03-09T09:27:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getugroups.h' -> './k/coreutils-8.5/lib/getugroups.h' 2024-03-09T09:27:57.563 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getugroups.h’: No data available 2024-03-09T09:27:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/file-set.c' -> './k/coreutils-8.5/lib/file-set.c' 2024-03-09T09:27:57.567 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/file-set.c’: No data available 2024-03-09T09:27:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sys_socket.in.h' -> './k/coreutils-8.5/lib/sys_socket.in.h' 2024-03-09T09:27:57.568 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sys_socket.in.h’: No data available 2024-03-09T09:27:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/inttostr.h' -> './k/coreutils-8.5/lib/inttostr.h' 2024-03-09T09:27:57.569 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/inttostr.h’: No data available 2024-03-09T09:27:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mkdirat.c' -> './k/coreutils-8.5/lib/mkdirat.c' 2024-03-09T09:27:57.570 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkdirat.c’: No data available 2024-03-09T09:27:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/readtokens.h' -> './k/coreutils-8.5/lib/readtokens.h' 2024-03-09T09:27:57.571 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readtokens.h’: No data available 2024-03-09T09:27:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fd-reopen.h' -> './k/coreutils-8.5/lib/fd-reopen.h' 2024-03-09T09:27:57.572 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fd-reopen.h’: No data available 2024-03-09T09:27:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/w32sock.h' -> './k/coreutils-8.5/lib/w32sock.h' 2024-03-09T09:27:57.573 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/w32sock.h’: No data available 2024-03-09T09:27:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memcasecmp.c' -> './k/coreutils-8.5/lib/memcasecmp.c' 2024-03-09T09:27:57.574 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcasecmp.c’: No data available 2024-03-09T09:27:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/at-func.c' -> './k/coreutils-8.5/lib/at-func.c' 2024-03-09T09:27:57.575 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/at-func.c’: No data available 2024-03-09T09:27:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unistd-safer.h' -> './k/coreutils-8.5/lib/unistd-safer.h' 2024-03-09T09:27:57.576 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistd-safer.h’: No data available 2024-03-09T09:27:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/filevercmp.h' -> './k/coreutils-8.5/lib/filevercmp.h' 2024-03-09T09:27:57.577 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filevercmp.h’: No data available 2024-03-09T09:27:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/c-strcasecmp.c' -> './k/coreutils-8.5/lib/c-strcasecmp.c' 2024-03-09T09:27:57.579 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strcasecmp.c’: No data available 2024-03-09T09:27:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fopen-safer.c' -> './k/coreutils-8.5/lib/fopen-safer.c' 2024-03-09T09:27:57.579 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fopen-safer.c’: No data available 2024-03-09T09:27:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/modechange.h' -> './k/coreutils-8.5/lib/modechange.h' 2024-03-09T09:27:57.581 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/modechange.h’: No data available 2024-03-09T09:27:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sys_stat.in.h' -> './k/coreutils-8.5/lib/sys_stat.in.h' 2024-03-09T09:27:57.582 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sys_stat.in.h’: No data available 2024-03-09T09:27:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/freading.c' -> './k/coreutils-8.5/lib/freading.c' 2024-03-09T09:27:57.587 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freading.c’: No data available 2024-03-09T09:27:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/utimens.h' -> './k/coreutils-8.5/lib/utimens.h' 2024-03-09T09:27:57.588 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/utimens.h’: No data available 2024-03-09T09:27:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/md5.h' -> './k/coreutils-8.5/lib/md5.h' 2024-03-09T09:27:57.589 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/md5.h’: No data available 2024-03-09T09:27:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/isnanf.c' -> './k/coreutils-8.5/lib/isnanf.c' 2024-03-09T09:27:57.589 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnanf.c’: No data available 2024-03-09T09:27:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sha1.c' -> './k/coreutils-8.5/lib/sha1.c' 2024-03-09T09:27:57.590 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha1.c’: No data available 2024-03-09T09:27:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/filenamecat-lgpl.c' -> './k/coreutils-8.5/lib/filenamecat-lgpl.c' 2024-03-09T09:27:57.591 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filenamecat-lgpl.c’: No data available 2024-03-09T09:27:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/config.hin' -> './k/coreutils-8.5/lib/config.hin' 2024-03-09T09:27:57.592 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/config.hin’: No data available 2024-03-09T09:27:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/ignore-value.h' -> './k/coreutils-8.5/lib/ignore-value.h' 2024-03-09T09:27:57.593 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ignore-value.h’: No data available 2024-03-09T09:27:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strstr.c' -> './k/coreutils-8.5/lib/strstr.c' 2024-03-09T09:27:57.593 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strstr.c’: No data available 2024-03-09T09:27:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/verror.c' -> './k/coreutils-8.5/lib/verror.c' 2024-03-09T09:27:57.594 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/verror.c’: No data available 2024-03-09T09:27:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/randperm.h' -> './k/coreutils-8.5/lib/randperm.h' 2024-03-09T09:27:57.595 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randperm.h’: No data available 2024-03-09T09:27:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/vasprintf.c' -> './k/coreutils-8.5/lib/vasprintf.c' 2024-03-09T09:27:57.595 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/vasprintf.c’: No data available 2024-03-09T09:27:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/c-strncasecmp.c' -> './k/coreutils-8.5/lib/c-strncasecmp.c' 2024-03-09T09:27:57.597 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strncasecmp.c’: No data available 2024-03-09T09:27:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/basename-lgpl.c' -> './k/coreutils-8.5/lib/basename-lgpl.c' 2024-03-09T09:27:57.598 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/basename-lgpl.c’: No data available 2024-03-09T09:27:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/human.h' -> './k/coreutils-8.5/lib/human.h' 2024-03-09T09:27:57.599 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/human.h’: No data available 2024-03-09T09:27:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/regex.h' -> './k/coreutils-8.5/lib/regex.h' 2024-03-09T09:27:57.600 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regex.h’: No data available 2024-03-09T09:27:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/regcomp.c' -> './k/coreutils-8.5/lib/regcomp.c' 2024-03-09T09:27:57.602 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regcomp.c’: No data available 2024-03-09T09:27:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/freadahead.c' -> './k/coreutils-8.5/lib/freadahead.c' 2024-03-09T09:27:57.603 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadahead.c’: No data available 2024-03-09T09:27:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/trim.h' -> './k/coreutils-8.5/lib/trim.h' 2024-03-09T09:27:57.604 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/trim.h’: No data available 2024-03-09T09:27:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getgroups.c' -> './k/coreutils-8.5/lib/getgroups.c' 2024-03-09T09:27:57.605 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getgroups.c’: No data available 2024-03-09T09:27:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xprintf.c' -> './k/coreutils-8.5/lib/xprintf.c' 2024-03-09T09:27:57.606 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xprintf.c’: No data available 2024-03-09T09:27:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/euidaccess-stat.h' -> './k/coreutils-8.5/lib/euidaccess-stat.h' 2024-03-09T09:27:57.607 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/euidaccess-stat.h’: No data available 2024-03-09T09:27:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/rand-isaac.h' -> './k/coreutils-8.5/lib/rand-isaac.h' 2024-03-09T09:27:57.608 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/rand-isaac.h’: No data available 2024-03-09T09:27:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mgetgroups.h' -> './k/coreutils-8.5/lib/mgetgroups.h' 2024-03-09T09:27:57.609 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mgetgroups.h’: No data available 2024-03-09T09:27:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/selinux-at.h' -> './k/coreutils-8.5/lib/selinux-at.h' 2024-03-09T09:27:57.610 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/selinux-at.h’: No data available 2024-03-09T09:27:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-solaris.h' -> './k/coreutils-8.5/lib/iconv_open-solaris.h' 2024-03-09T09:27:57.611 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-solaris.h’: No data available 2024-03-09T09:27:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/setenv.c' -> './k/coreutils-8.5/lib/setenv.c' 2024-03-09T09:27:57.612 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/setenv.c’: No data available 2024-03-09T09:27:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/locale.in.h' -> './k/coreutils-8.5/lib/locale.in.h' 2024-03-09T09:27:57.613 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/locale.in.h’: No data available 2024-03-09T09:27:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/base64.h' -> './k/coreutils-8.5/lib/base64.h' 2024-03-09T09:27:57.614 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/base64.h’: No data available 2024-03-09T09:27:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/open-safer.c' -> './k/coreutils-8.5/lib/open-safer.c' 2024-03-09T09:27:57.615 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/open-safer.c’: No data available 2024-03-09T09:27:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/exitfail.c' -> './k/coreutils-8.5/lib/exitfail.c' 2024-03-09T09:27:57.616 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/exitfail.c’: No data available 2024-03-09T09:27:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/closeout.h' -> './k/coreutils-8.5/lib/closeout.h' 2024-03-09T09:27:57.617 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/closeout.h’: No data available 2024-03-09T09:27:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/idcache.c' -> './k/coreutils-8.5/lib/idcache.c' 2024-03-09T09:27:57.618 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/idcache.c’: No data available 2024-03-09T09:27:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unlink.c' -> './k/coreutils-8.5/lib/unlink.c' 2024-03-09T09:27:57.619 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unlink.c’: No data available 2024-03-09T09:27:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xfts.c' -> './k/coreutils-8.5/lib/xfts.c' 2024-03-09T09:27:57.620 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xfts.c’: No data available 2024-03-09T09:27:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memrchr.c' -> './k/coreutils-8.5/lib/memrchr.c' 2024-03-09T09:27:57.621 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memrchr.c’: No data available 2024-03-09T09:27:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/Makefile.am' -> './k/coreutils-8.5/lib/Makefile.am' 2024-03-09T09:27:57.623 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/Makefile.am’: No data available 2024-03-09T09:27:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstrtol.h' -> './k/coreutils-8.5/lib/xstrtol.h' 2024-03-09T09:27:57.624 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtol.h’: No data available 2024-03-09T09:27:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/gettimeofday.c' -> './k/coreutils-8.5/lib/gettimeofday.c' 2024-03-09T09:27:57.625 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gettimeofday.c’: No data available 2024-03-09T09:27:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/isnanf-nolibm.h' -> './k/coreutils-8.5/lib/isnanf-nolibm.h' 2024-03-09T09:27:57.625 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnanf-nolibm.h’: No data available 2024-03-09T09:27:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/chdir-long.c' -> './k/coreutils-8.5/lib/chdir-long.c' 2024-03-09T09:27:57.626 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/chdir-long.c’: No data available 2024-03-09T09:27:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbchar.c' -> './k/coreutils-8.5/lib/mbchar.c' 2024-03-09T09:27:57.627 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbchar.c’: No data available 2024-03-09T09:27:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/canon-host.h' -> './k/coreutils-8.5/lib/canon-host.h' 2024-03-09T09:27:57.628 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/canon-host.h’: No data available 2024-03-09T09:27:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unsetenv.c' -> './k/coreutils-8.5/lib/unsetenv.c' 2024-03-09T09:27:57.629 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unsetenv.c’: No data available 2024-03-09T09:27:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/freadseek.c' -> './k/coreutils-8.5/lib/freadseek.c' 2024-03-09T09:27:57.630 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadseek.c’: No data available 2024-03-09T09:27:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xnanosleep.c' -> './k/coreutils-8.5/lib/xnanosleep.c' 2024-03-09T09:27:57.631 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xnanosleep.c’: No data available 2024-03-09T09:27:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/nanosleep.c' -> './k/coreutils-8.5/lib/nanosleep.c' 2024-03-09T09:27:57.632 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/nanosleep.c’: No data available 2024-03-09T09:27:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/ctype.in.h' -> './k/coreutils-8.5/lib/ctype.in.h' 2024-03-09T09:27:57.633 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ctype.in.h’: No data available 2024-03-09T09:27:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xmalloc.c' -> './k/coreutils-8.5/lib/xmalloc.c' 2024-03-09T09:27:57.634 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xmalloc.c’: No data available 2024-03-09T09:27:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strtoul.c' -> './k/coreutils-8.5/lib/strtoul.c' 2024-03-09T09:27:57.635 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtoul.c’: No data available 2024-03-09T09:27:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sha512.h' -> './k/coreutils-8.5/lib/sha512.h' 2024-03-09T09:27:57.636 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha512.h’: No data available 2024-03-09T09:27:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dirchownmod.h' -> './k/coreutils-8.5/lib/dirchownmod.h' 2024-03-09T09:27:57.637 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirchownmod.h’: No data available 2024-03-09T09:27:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/freadptr.h' -> './k/coreutils-8.5/lib/freadptr.h' 2024-03-09T09:27:57.638 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadptr.h’: No data available 2024-03-09T09:27:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getfilecon.c' -> './k/coreutils-8.5/lib/getfilecon.c' 2024-03-09T09:27:57.639 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getfilecon.c’: No data available 2024-03-09T09:27:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/alignof.h' -> './k/coreutils-8.5/lib/alignof.h' 2024-03-09T09:27:57.640 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/alignof.h’: No data available 2024-03-09T09:27:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mempcpy.c' -> './k/coreutils-8.5/lib/mempcpy.c' 2024-03-09T09:27:57.641 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mempcpy.c’: No data available 2024-03-09T09:27:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/savedir.h' -> './k/coreutils-8.5/lib/savedir.h' 2024-03-09T09:27:57.642 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/savedir.h’: No data available 2024-03-09T09:27:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/areadlink-with-size.c' -> './k/coreutils-8.5/lib/areadlink-with-size.c' 2024-03-09T09:27:57.643 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/areadlink-with-size.c’: No data available 2024-03-09T09:27:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/inttypes.in.h' -> './k/coreutils-8.5/lib/inttypes.in.h' 2024-03-09T09:27:57.644 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/inttypes.in.h’: No data available 2024-03-09T09:27:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/config.charset' -> './k/coreutils-8.5/lib/config.charset' 2024-03-09T09:27:57.646 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/config.charset’: No data available 2024-03-09T09:27:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstrtold.c' -> './k/coreutils-8.5/lib/xstrtold.c' 2024-03-09T09:27:57.647 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtold.c’: No data available 2024-03-09T09:27:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fcntl-safer.h' -> './k/coreutils-8.5/lib/fcntl-safer.h' 2024-03-09T09:27:57.648 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fcntl-safer.h’: No data available 2024-03-09T09:27:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstrndup.c' -> './k/coreutils-8.5/lib/xstrndup.c' 2024-03-09T09:27:57.657 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrndup.c’: No data available 2024-03-09T09:27:57.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.683+0000 7f4009800640 1 Processor -- start 2024-03-09T09:27:57.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.683+0000 7f4009800640 1 -- start start 2024-03-09T09:27:57.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.685+0000 7f4009800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f400406b040 0x7f400406b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.685+0000 7f4009800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f400414a270 0x7f400414a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.685+0000 7f4009800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4004154610 con 0x7f4004157b50 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.685+0000 7f4009800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40041576c0 con 0x7f400406b040 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.685+0000 7f4009800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40041548f0 con 0x7f400414a270 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.686+0000 7f4002a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f400406b040 0x7f400406b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.686+0000 7f4002a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f400406b040 0x7f400406b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49702/0 (socket says 172.21.15.67:49702) 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.686+0000 7f4002a00640 1 -- 172.21.15.67:0/3399118562 learned_addr learned my addr 172.21.15.67:0/3399118562 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.686+0000 7f4002000640 1 --1- 172.21.15.67:0/3399118562 >> v1:172.21.15.134:6789/0 conn(0x7f4004157b50 0x7f4004153bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.134:6789/0 says I am v1:172.21.15.67:38178/0 (socket says 172.21.15.67:38178) 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.686+0000 7f4003400640 1 --2- 172.21.15.67:0/3399118562 >> v2:172.21.15.134:3300/0 conn(0x7f400414a270 0x7f400414a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.686+0000 7f4002a00640 1 -- 172.21.15.67:0/3399118562 >> v2:172.21.15.134:3300/0 conn(0x7f400414a270 msgr2=0x7f400414a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.686+0000 7f4002a00640 1 --2- 172.21.15.67:0/3399118562 >> v2:172.21.15.134:3300/0 conn(0x7f400414a270 0x7f400414a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:57.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.686+0000 7f4002a00640 1 -- 172.21.15.67:0/3399118562 >> v1:172.21.15.134:6789/0 conn(0x7f4004157b50 legacy=0x7f4004153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:57.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.686+0000 7f4002a00640 1 -- 172.21.15.67:0/3399118562 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004157350 con 0x7f400406b040 2024-03-09T09:27:57.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.688+0000 7f4002a00640 1 --2- 172.21.15.67:0/3399118562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400406b040 0x7f400406b420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f400406f980 tx=0x7f3ff402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8783ce179e91efd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.689+0000 7f4001600640 1 -- 172.21.15.67:0/3399118562 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4034070 con 0x7f400406b040 2024-03-09T09:27:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.689+0000 7f4001600640 1 -- 172.21.15.67:0/3399118562 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff4030860 con 0x7f400406b040 2024-03-09T09:27:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.689+0000 7f4001600640 1 -- 172.21.15.67:0/3399118562 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4030c30 con 0x7f400406b040 2024-03-09T09:27:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.689+0000 7f4009800640 1 -- 172.21.15.67:0/3399118562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400406b040 msgr2=0x7f400406b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.689+0000 7f4009800640 1 --2- 172.21.15.67:0/3399118562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400406b040 0x7f400406b420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f400406f980 tx=0x7f3ff402b9a0 comp rx=0 tx=0).stop 2024-03-09T09:27:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.689+0000 7f4009800640 1 -- 172.21.15.67:0/3399118562 shutdown_connections 2024-03-09T09:27:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.689+0000 7f4009800640 1 --2- 172.21.15.67:0/3399118562 >> v2:172.21.15.134:3300/0 conn(0x7f400414a270 0x7f400414a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.689+0000 7f4009800640 1 --2- 172.21.15.67:0/3399118562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400406b040 0x7f400406b420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.689+0000 7f4009800640 1 -- 172.21.15.67:0/3399118562 >> 172.21.15.67:0/3399118562 conn(0x7f40040c8b20 msgr2=0x7f400406c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.690+0000 7f4009800640 1 -- 172.21.15.67:0/3399118562 shutdown_connections 2024-03-09T09:27:57.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.690+0000 7f4009800640 1 -- 172.21.15.67:0/3399118562 wait complete. 2024-03-09T09:27:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.692+0000 7f4009800640 1 Processor -- start 2024-03-09T09:27:57.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.692+0000 7f4009800640 1 -- start start 2024-03-09T09:27:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.693+0000 7f4009800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400406b040 0x7f40041ec1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.693+0000 7f4009800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400414a270 0x7f4004152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.693+0000 7f4009800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4004157b50 0x7f4004148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.693+0000 7f4009800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40040c3010 con 0x7f4004157b50 2024-03-09T09:27:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.693+0000 7f4009800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4004157fd0 con 0x7f400406b040 2024-03-09T09:27:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.693+0000 7f4009800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40040c3190 con 0x7f400414a270 2024-03-09T09:27:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400406b040 0x7f40041ec1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400406b040 0x7f40041ec1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:54758/0 (socket says 172.21.15.67:54758) 2024-03-09T09:27:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002a00640 1 -- 172.21.15.67:0/3494676065 learned_addr learned my addr 172.21.15.67:0/3494676065 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4003400640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4004157b50 0x7f4004148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:57.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002000640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400414a270 0x7f4004152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002000640 1 -- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400414a270 msgr2=0x7f4004152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:27:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002000640 1 -- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400414a270 msgr2=0x7f4004152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:27:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002000640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400414a270 0x7f4004152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002000640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400414a270 0x7f4004152940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002a00640 1 -- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400414a270 msgr2=0x7f4004152940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:27:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002a00640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400414a270 0x7f4004152940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002a00640 1 -- 172.21.15.67:0/3494676065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4004157b50 msgr2=0x7f4004148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002a00640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4004157b50 0x7f4004148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4002a00640 1 -- 172.21.15.67:0/3494676065 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40041ec6f0 con 0x7f400406b040 2024-03-09T09:27:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.694+0000 7f4003400640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4004157b50 0x7f4004148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:27:57.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.695+0000 7f4002a00640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400406b040 0x7f40041ec1b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4000c00 tx=0x7f3ff4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.695+0000 7f3ffaa00640 1 -- 172.21.15.67:0/3494676065 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4048070 con 0x7f400406b040 2024-03-09T09:27:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.695+0000 7f3ffaa00640 1 -- 172.21.15.67:0/3494676065 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff4030660 con 0x7f400406b040 2024-03-09T09:27:57.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.695+0000 7f3ffaa00640 1 -- 172.21.15.67:0/3494676065 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4034040 con 0x7f400406b040 2024-03-09T09:27:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.695+0000 7f4009800640 1 -- 172.21.15.67:0/3494676065 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4004148400 con 0x7f400406b040 2024-03-09T09:27:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.695+0000 7f4009800640 1 -- 172.21.15.67:0/3494676065 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40040bdd20 con 0x7f400406b040 2024-03-09T09:27:57.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.696+0000 7f4009800640 1 -- 172.21.15.67:0/3494676065 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4004075a00 con 0x7f400406b040 2024-03-09T09:27:57.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.699+0000 7f3ffaa00640 1 -- 172.21.15.67:0/3494676065 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3ff404b070 con 0x7f400406b040 2024-03-09T09:27:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.700+0000 7f3ffaa00640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe406b4d0 0x7f3fe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.700+0000 7f3ffaa00640 1 -- 172.21.15.67:0/3494676065 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f3ff40ac850 con 0x7f400406b040 2024-03-09T09:27:57.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.700+0000 7f4002000640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe406b4d0 0x7f3fe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:57.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.701+0000 7f4002000640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe406b4d0 0x7f3fe406d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4004151200 tx=0x7f3ff0007cc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:57.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:57.703+0000 7f3ffaa00640 1 -- 172.21.15.67:0/3494676065 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3ff407ae50 con 0x7f400406b040 2024-03-09T09:27:58.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.098+0000 7f4009800640 1 -- 172.21.15.67:0/3494676065 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f400406cb00 con 0x7f400406b040 2024-03-09T09:27:58.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.098+0000 7f3ffaa00640 1 -- 172.21.15.67:0/3494676065 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f3ff407eef0 con 0x7f400406b040 2024-03-09T09:27:58.106 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":282,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:27:58.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 -- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe406b4d0 msgr2=0x7f3fe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe406b4d0 0x7f3fe406d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4004151200 tx=0x7f3ff0007cc0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 -- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400406b040 msgr2=0x7f40041ec1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400406b040 0x7f40041ec1b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4000c00 tx=0x7f3ff4004730 comp rx=0 tx=0).stop 2024-03-09T09:27:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 -- 172.21.15.67:0/3494676065 shutdown_connections 2024-03-09T09:27:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe406b4d0 0x7f3fe406d990 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4004157b50 0x7f4004148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400414a270 0x7f4004152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 --2- 172.21.15.67:0/3494676065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400406b040 0x7f40041ec1b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 -- 172.21.15.67:0/3494676065 >> 172.21.15.67:0/3494676065 conn(0x7f40040c8b20 msgr2=0x7f40040c7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 -- 172.21.15.67:0/3494676065 shutdown_connections 2024-03-09T09:27:58.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.101+0000 7f4009800640 1 -- 172.21.15.67:0/3494676065 wait complete. 2024-03-09T09:27:58.129 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:27:58.130 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:27:58.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:27:58.246 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/write.c' -> './k/coreutils-8.5/lib/write.c' 2024-03-09T09:27:58.246 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/write.c’: No data available 2024-03-09T09:27:58.248 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/math.in.h' -> './k/coreutils-8.5/lib/math.in.h' 2024-03-09T09:27:58.248 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/math.in.h’: No data available 2024-03-09T09:27:58.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.345+0000 7ff28f200640 1 Processor -- start 2024-03-09T09:27:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.345+0000 7ff28f200640 1 -- start start 2024-03-09T09:27:58.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.347+0000 7ff28f200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff29006c1d0 0x7ff29006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.347+0000 7ff28f200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2900c9890 con 0x7ff29014fd90 2024-03-09T09:27:58.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.347+0000 7ff28f200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2900c5f80 con 0x7ff2900c9380 2024-03-09T09:27:58.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.347+0000 7ff28f200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29006ba60 con 0x7ff29006c1d0 2024-03-09T09:27:58.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.347+0000 7ff28de00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ff29014fd90 0x7ff290156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:44524/0 (socket says 172.21.15.67:44524) 2024-03-09T09:27:58.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.347+0000 7ff28de00640 1 -- 172.21.15.67:0/1811495836 learned_addr learned my addr 172.21.15.67:0/1811495836 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:27:58.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.347+0000 7ff28e800640 1 --2- 172.21.15.67:0/1811495836 >> v2:172.21.15.67:3300/0 conn(0x7ff29006c1d0 0x7ff29006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.348+0000 7ff28e800640 1 -- 172.21.15.67:0/1811495836 >> v1:172.21.15.134:6789/0 conn(0x7ff2900c9380 legacy=0x7ff2900acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.348+0000 7ff28e800640 1 -- 172.21.15.67:0/1811495836 >> v1:172.21.15.67:6789/0 conn(0x7ff29014fd90 legacy=0x7ff290156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.348+0000 7ff28e800640 1 -- 172.21.15.67:0/1811495836 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2900c6240 con 0x7ff29006c1d0 2024-03-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.348+0000 7ff28d400640 1 -- 172.21.15.67:0/1811495836 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528222442 0 0) 0x7ff2900c9890 con 0x7ff29014fd90 2024-03-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.350+0000 7ff28e800640 1 --2- 172.21.15.67:0/1811495836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff29006c1d0 0x7ff29006c5f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff29006bbe0 tx=0x7ff28802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a3d69f9633695f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.350+0000 7ff28d400640 1 -- 172.21.15.67:0/1811495836 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288039070 con 0x7ff29006c1d0 2024-03-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.350+0000 7ff28d400640 1 -- 172.21.15.67:0/1811495836 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28802cbd0 con 0x7ff29006c1d0 2024-03-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.350+0000 7ff28d400640 1 -- 172.21.15.67:0/1811495836 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288034440 con 0x7ff29006c1d0 2024-03-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.351+0000 7ff28f200640 1 -- 172.21.15.67:0/1811495836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff29006c1d0 msgr2=0x7ff29006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.351+0000 7ff28f200640 1 --2- 172.21.15.67:0/1811495836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff29006c1d0 0x7ff29006c5f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff29006bbe0 tx=0x7ff28802be20 comp rx=0 tx=0).stop 2024-03-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.351+0000 7ff28f200640 1 -- 172.21.15.67:0/1811495836 shutdown_connections 2024-03-09T09:27:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.351+0000 7ff28f200640 1 --2- 172.21.15.67:0/1811495836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff29006c1d0 0x7ff29006c5f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.351+0000 7ff28f200640 1 -- 172.21.15.67:0/1811495836 >> 172.21.15.67:0/1811495836 conn(0x7ff2900cb480 msgr2=0x7ff2900cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.351+0000 7ff28f200640 1 -- 172.21.15.67:0/1811495836 shutdown_connections 2024-03-09T09:27:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.351+0000 7ff28f200640 1 -- 172.21.15.67:0/1811495836 wait complete. 2024-03-09T09:27:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.353+0000 7ff28f200640 1 Processor -- start 2024-03-09T09:27:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.353+0000 7ff28f200640 1 -- start start 2024-03-09T09:27:58.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.354+0000 7ff28f200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff29006c1d0 0x7ff2900ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.354+0000 7ff28f200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2900c9380 0x7ff2901ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.354+0000 7ff28f200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff29014fd90 0x7ff2901ec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.354+0000 7ff28f200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900ad610 con 0x7ff29014fd90 2024-03-09T09:27:58.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.354+0000 7ff28f200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29006bcb0 con 0x7ff29006c1d0 2024-03-09T09:27:58.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.354+0000 7ff28f200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900ad790 con 0x7ff2900c9380 2024-03-09T09:27:58.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.354+0000 7ff28e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff29014fd90 0x7ff2901ec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.354+0000 7ff28e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff29014fd90 0x7ff2901ec0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49728/0 (socket says 172.21.15.67:49728) 2024-03-09T09:27:58.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff28e800640 1 -- 172.21.15.67:0/2657009721 learned_addr learned my addr 172.21.15.67:0/2657009721 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:58.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff287e00640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2900c9380 0x7ff2901ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff28de00640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff29006c1d0 0x7ff2900ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff287e00640 1 -- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2900c9380 msgr2=0x7ff2901ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-03-09T09:27:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff287e00640 1 -- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2900c9380 msgr2=0x7ff2901ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:27:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff287e00640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2900c9380 0x7ff2901ebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff28e800640 1 -- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2900c9380 msgr2=0x7ff2901ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff287e00640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2900c9380 0x7ff2901ebb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff28e800640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff2900c9380 0x7ff2901ebb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff28e800640 1 -- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff29006c1d0 msgr2=0x7ff2900ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff28e800640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff29006c1d0 0x7ff2900ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff28e800640 1 -- 172.21.15.67:0/2657009721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2900c1e30 con 0x7ff29014fd90 2024-03-09T09:27:58.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.355+0000 7ff28e800640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff29014fd90 0x7ff2901ec0b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff288037040 tx=0x7ff288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.356+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288039050 con 0x7ff29014fd90 2024-03-09T09:27:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.356+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff288045070 con 0x7ff29014fd90 2024-03-09T09:27:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.356+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28803e810 con 0x7ff29014fd90 2024-03-09T09:27:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.356+0000 7ff28f200640 1 -- 172.21.15.67:0/2657009721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2901ef330 con 0x7ff29014fd90 2024-03-09T09:27:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.356+0000 7ff28f200640 1 -- 172.21.15.67:0/2657009721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2901ef8f0 con 0x7ff29014fd90 2024-03-09T09:27:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.358+0000 7ff28f200640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2900c1930 0x7ff290148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.358+0000 7ff28f200640 1 -- 172.21.15.67:0/2657009721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2901efaf0 con 0x7ff2900c1930 2024-03-09T09:27:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.358+0000 7ff28de00640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2900c1930 0x7ff290148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.359+0000 7ff28de00640 1 -- 172.21.15.67:0/2657009721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff29004f3c0 con 0x7ff2900c1930 2024-03-09T09:27:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.359+0000 7ff28de00640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2900c1930 0x7ff290148210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2901ecca0 tx=0x7ff280001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.359+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff28803e9b0 con 0x7ff29014fd90 2024-03-09T09:27:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.361+0000 7ff286000640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff26c06b4d0 0x7ff26c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.361+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7ff2880ad440 con 0x7ff29014fd90 2024-03-09T09:27:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.361+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280012590 con 0x7ff2900c1930 2024-03-09T09:27:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.361+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7ff280018070 con 0x7ff2900c1930 2024-03-09T09:27:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.361+0000 7ff287e00640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff26c06b4d0 0x7ff26c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.361+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2900c1930 msgr2=0x7ff290148210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.361+0000 7ff286000640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2900c1930 0x7ff290148210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2901ecca0 tx=0x7ff280001970 comp rx=0 tx=0).stop 2024-03-09T09:27:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.361+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2900c1930 msgr2=0x7ff290148210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.361+0000 7ff286000640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2900c1930 0x7ff290148210 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2901ecca0 tx=0x7ff280001970 comp rx=0 tx=0).stop 2024-03-09T09:27:58.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.363+0000 7ff287e00640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff26c06b4d0 0x7ff26c06d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff2901e99b0 tx=0x7ff278009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.373+0000 7ff28f200640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff290158e10 0x7ff290148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.373+0000 7ff28f200640 1 -- 172.21.15.67:0/2657009721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29014c460 con 0x7ff290158e10 2024-03-09T09:27:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.373+0000 7ff28de00640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff290158e10 0x7ff290148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.374+0000 7ff28de00640 1 -- 172.21.15.67:0/2657009721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff290155f10 con 0x7ff290158e10 2024-03-09T09:27:58.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.374+0000 7ff28de00640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff290158e10 0x7ff290148750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff280017000 tx=0x7ff280000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.375+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280012a80 con 0x7ff290158e10 2024-03-09T09:27:58.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.375+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7ff280012a80 con 0x7ff290158e10 2024-03-09T09:27:58.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.375+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff290158e10 msgr2=0x7ff290148750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.375+0000 7ff286000640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff290158e10 0x7ff290148750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff280017000 tx=0x7ff280000ed0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.375+0000 7ff286000640 1 -- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff290158e10 msgr2=0x7ff290148750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:58.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.375+0000 7ff286000640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff290158e10 0x7ff290148750 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff280017000 tx=0x7ff280000ed0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.383 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:27:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 282, 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:27:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:27:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:27:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:27:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:27:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:27:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:27:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:27:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:27:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:27:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:27:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:27:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:27:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:27:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:27:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:27:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:27:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:27:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:27:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:27:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:27:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:27:58.400 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:27:58.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.376+0000 7ff28f200640 1 -- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff26c06b4d0 msgr2=0x7ff26c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.376+0000 7ff28f200640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff26c06b4d0 0x7ff26c06d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff2901e99b0 tx=0x7ff278009040 comp rx=0 tx=0).stop 2024-03-09T09:27:58.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.376+0000 7ff28f200640 1 -- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff29014fd90 msgr2=0x7ff2901ec0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.376+0000 7ff28f200640 1 --2- 172.21.15.67:0/2657009721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff29014fd90 0x7ff2901ec0b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff288037040 tx=0x7ff288004a40 comp rx=0 tx=0).stop 2024-03-09T09:27:58.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.376+0000 7ff28e800640 1 -- 172.21.15.67:0/2657009721 reap_dead start 2024-03-09T09:27:58.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.376+0000 7ff28e800640 1 -- 172.21.15.67:0/2657009721 reap_dead start 2024-03-09T09:27:58.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.376+0000 7ff28f200640 1 -- 172.21.15.67:0/2657009721 shutdown_connections 2024-03-09T09:27:58.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.376+0000 7ff28f200640 1 -- 172.21.15.67:0/2657009721 >> 172.21.15.67:0/2657009721 conn(0x7ff2900cb480 msgr2=0x7ff29014f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:58.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.376+0000 7ff28f200640 1 -- 172.21.15.67:0/2657009721 shutdown_connections 2024-03-09T09:27:58.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.376+0000 7ff28f200640 1 -- 172.21.15.67:0/2657009721 wait complete. 2024-03-09T09:27:58.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/openat-priv.h' -> './k/coreutils-8.5/lib/openat-priv.h' 2024-03-09T09:27:58.403 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat-priv.h’: No data available 2024-03-09T09:27:58.403 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/vasnprintf.h' -> './k/coreutils-8.5/lib/vasnprintf.h' 2024-03-09T09:27:58.404 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/vasnprintf.h’: No data available 2024-03-09T09:27:58.404 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getcwd.c' -> './k/coreutils-8.5/lib/getcwd.c' 2024-03-09T09:27:58.404 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getcwd.c’: No data available 2024-03-09T09:27:58.405 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stdbool.in.h' -> './k/coreutils-8.5/lib/stdbool.in.h' 2024-03-09T09:27:58.405 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdbool.in.h’: No data available 2024-03-09T09:27:58.405 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strtol.c' -> './k/coreutils-8.5/lib/strtol.c' 2024-03-09T09:27:58.406 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtol.c’: No data available 2024-03-09T09:27:58.406 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/argmatch.c' -> './k/coreutils-8.5/lib/argmatch.c' 2024-03-09T09:27:58.406 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/argmatch.c’: No data available 2024-03-09T09:27:58.407 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:27:58.408 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:27:58.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:27:58.435 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/acl_entries.c' -> './k/coreutils-8.5/lib/acl_entries.c' 2024-03-09T09:27:58.436 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/acl_entries.c’: No data available 2024-03-09T09:27:58.437 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/progname.c' -> './k/coreutils-8.5/lib/progname.c' 2024-03-09T09:27:58.437 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/progname.c’: No data available 2024-03-09T09:27:58.438 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mktime-internal.h' -> './k/coreutils-8.5/lib/mktime-internal.h' 2024-03-09T09:27:58.438 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mktime-internal.h’: No data available 2024-03-09T09:27:58.440 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/printf-frexpl.c' -> './k/coreutils-8.5/lib/printf-frexpl.c' 2024-03-09T09:27:58.440 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-frexpl.c’: No data available 2024-03-09T09:27:58.441 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/offtostr.c' -> './k/coreutils-8.5/lib/offtostr.c' 2024-03-09T09:27:58.441 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/offtostr.c’: No data available 2024-03-09T09:27:58.442 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memchr2.h' -> './k/coreutils-8.5/lib/memchr2.h' 2024-03-09T09:27:58.443 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memchr2.h’: No data available 2024-03-09T09:27:58.443 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstrtoumax.c' -> './k/coreutils-8.5/lib/xstrtoumax.c' 2024-03-09T09:27:58.444 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtoumax.c’: No data available 2024-03-09T09:27:58.445 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getdate.y' -> './k/coreutils-8.5/lib/getdate.y' 2024-03-09T09:27:58.446 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getdate.y’: No data available 2024-03-09T09:27:58.447 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/close-hook.h' -> './k/coreutils-8.5/lib/close-hook.h' 2024-03-09T09:27:58.447 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/close-hook.h’: No data available 2024-03-09T09:27:58.448 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/printf-parse.c' -> './k/coreutils-8.5/lib/printf-parse.c' 2024-03-09T09:27:58.448 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-parse.c’: No data available 2024-03-09T09:27:58.450 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/isnand.c' -> './k/coreutils-8.5/lib/isnand.c' 2024-03-09T09:27:58.450 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnand.c’: No data available 2024-03-09T09:27:58.452 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/timespec.h' -> './k/coreutils-8.5/lib/timespec.h' 2024-03-09T09:27:58.452 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/timespec.h’: No data available 2024-03-09T09:27:58.453 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getusershell.c' -> './k/coreutils-8.5/lib/getusershell.c' 2024-03-09T09:27:58.453 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getusershell.c’: No data available 2024-03-09T09:27:58.454 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstrtod.c' -> './k/coreutils-8.5/lib/xstrtod.c' 2024-03-09T09:27:58.455 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtod.c’: No data available 2024-03-09T09:27:58.455 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/verror.h' -> './k/coreutils-8.5/lib/verror.h' 2024-03-09T09:27:58.456 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/verror.h’: No data available 2024-03-09T09:27:58.457 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/same.c' -> './k/coreutils-8.5/lib/same.c' 2024-03-09T09:27:58.457 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/same.c’: No data available 2024-03-09T09:27:58.459 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/quote.c' -> './k/coreutils-8.5/lib/quote.c' 2024-03-09T09:27:58.459 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/quote.c’: No data available 2024-03-09T09:27:58.460 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/raise.c' -> './k/coreutils-8.5/lib/raise.c' 2024-03-09T09:27:58.460 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/raise.c’: No data available 2024-03-09T09:27:58.462 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/link.c' -> './k/coreutils-8.5/lib/link.c' 2024-03-09T09:27:58.462 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/link.c’: No data available 2024-03-09T09:27:58.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/cloexec.h' -> './k/coreutils-8.5/lib/cloexec.h' 2024-03-09T09:27:58.463 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/cloexec.h’: No data available 2024-03-09T09:27:58.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sys_utsname.in.h' -> './k/coreutils-8.5/lib/sys_utsname.in.h' 2024-03-09T09:27:58.465 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sys_utsname.in.h’: No data available 2024-03-09T09:27:58.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/cloexec.c' -> './k/coreutils-8.5/lib/cloexec.c' 2024-03-09T09:27:58.466 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/cloexec.c’: No data available 2024-03-09T09:27:58.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/vfprintf.c' -> './k/coreutils-8.5/lib/vfprintf.c' 2024-03-09T09:27:58.468 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/vfprintf.c’: No data available 2024-03-09T09:27:58.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/opendir-safer.c' -> './k/coreutils-8.5/lib/opendir-safer.c' 2024-03-09T09:27:58.469 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/opendir-safer.c’: No data available 2024-03-09T09:27:58.470 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/hash-pjw.h' -> './k/coreutils-8.5/lib/hash-pjw.h' 2024-03-09T09:27:58.471 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash-pjw.h’: No data available 2024-03-09T09:27:58.472 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/uname.c' -> './k/coreutils-8.5/lib/uname.c' 2024-03-09T09:27:58.472 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/uname.c’: No data available 2024-03-09T09:27:58.473 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/full-read.c' -> './k/coreutils-8.5/lib/full-read.c' 2024-03-09T09:27:58.473 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/full-read.c’: No data available 2024-03-09T09:27:58.474 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/priv-set.h' -> './k/coreutils-8.5/lib/priv-set.h' 2024-03-09T09:27:58.474 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/priv-set.h’: No data available 2024-03-09T09:27:58.475 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/close.c' -> './k/coreutils-8.5/lib/close.c' 2024-03-09T09:27:58.475 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/close.c’: No data available 2024-03-09T09:27:58.476 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fpending.h' -> './k/coreutils-8.5/lib/fpending.h' 2024-03-09T09:27:58.477 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fpending.h’: No data available 2024-03-09T09:27:58.478 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/i-ring.h' -> './k/coreutils-8.5/lib/i-ring.h' 2024-03-09T09:27:58.478 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/i-ring.h’: No data available 2024-03-09T09:27:58.480 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/nl_langinfo.c' -> './k/coreutils-8.5/lib/nl_langinfo.c' 2024-03-09T09:27:58.480 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/nl_langinfo.c’: No data available 2024-03-09T09:27:58.482 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xmemxfrm.h' -> './k/coreutils-8.5/lib/xmemxfrm.h' 2024-03-09T09:27:58.482 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xmemxfrm.h’: No data available 2024-03-09T09:27:58.484 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/c-strcase.h' -> './k/coreutils-8.5/lib/c-strcase.h' 2024-03-09T09:27:58.484 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strcase.h’: No data available 2024-03-09T09:27:58.485 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fchdir.c' -> './k/coreutils-8.5/lib/fchdir.c' 2024-03-09T09:27:58.486 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fchdir.c’: No data available 2024-03-09T09:27:58.488 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/canon-host.c' -> './k/coreutils-8.5/lib/canon-host.c' 2024-03-09T09:27:58.489 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/canon-host.c’: No data available 2024-03-09T09:27:58.490 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/signbitf.c' -> './k/coreutils-8.5/lib/signbitf.c' 2024-03-09T09:27:58.490 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/signbitf.c’: No data available 2024-03-09T09:27:58.490 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xnanosleep.h' -> './k/coreutils-8.5/lib/xnanosleep.h' 2024-03-09T09:27:58.491 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xnanosleep.h’: No data available 2024-03-09T09:27:58.492 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/tempname.c' -> './k/coreutils-8.5/lib/tempname.c' 2024-03-09T09:27:58.492 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/tempname.c’: No data available 2024-03-09T09:27:58.493 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xreadlink.c' -> './k/coreutils-8.5/lib/xreadlink.c' 2024-03-09T09:27:58.493 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xreadlink.c’: No data available 2024-03-09T09:27:58.494 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xgetcwd.h' -> './k/coreutils-8.5/lib/xgetcwd.h' 2024-03-09T09:27:58.494 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xgetcwd.h’: No data available 2024-03-09T09:27:58.495 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/regex.c' -> './k/coreutils-8.5/lib/regex.c' 2024-03-09T09:27:58.496 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regex.c’: No data available 2024-03-09T09:27:58.496 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/diacrit.h' -> './k/coreutils-8.5/lib/diacrit.h' 2024-03-09T09:27:58.497 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/diacrit.h’: No data available 2024-03-09T09:27:58.498 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/posixver.c' -> './k/coreutils-8.5/lib/posixver.c' 2024-03-09T09:27:58.498 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/posixver.c’: No data available 2024-03-09T09:27:58.499 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getugroups.c' -> './k/coreutils-8.5/lib/getugroups.c' 2024-03-09T09:27:58.500 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getugroups.c’: No data available 2024-03-09T09:27:58.501 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/readutmp.h' -> './k/coreutils-8.5/lib/readutmp.h' 2024-03-09T09:27:58.501 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readutmp.h’: No data available 2024-03-09T09:27:58.503 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/quote.h' -> './k/coreutils-8.5/lib/quote.h' 2024-03-09T09:27:58.503 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/quote.h’: No data available 2024-03-09T09:27:58.504 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/quotearg.c' -> './k/coreutils-8.5/lib/quotearg.c' 2024-03-09T09:27:58.504 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/quotearg.c’: No data available 2024-03-09T09:27:58.506 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/filevercmp.c' -> './k/coreutils-8.5/lib/filevercmp.c' 2024-03-09T09:27:58.506 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filevercmp.c’: No data available 2024-03-09T09:27:58.508 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/printf-args.h' -> './k/coreutils-8.5/lib/printf-args.h' 2024-03-09T09:27:58.508 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-args.h’: No data available 2024-03-09T09:27:58.510 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mountlist.h' -> './k/coreutils-8.5/lib/mountlist.h' 2024-03-09T09:27:58.510 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mountlist.h’: No data available 2024-03-09T09:27:58.511 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstrtol-error.c' -> './k/coreutils-8.5/lib/xstrtol-error.c' 2024-03-09T09:27:58.512 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtol-error.c’: No data available 2024-03-09T09:27:58.513 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/rename.c' -> './k/coreutils-8.5/lib/rename.c' 2024-03-09T09:27:58.514 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/rename.c’: No data available 2024-03-09T09:27:58.515 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sha1.h' -> './k/coreutils-8.5/lib/sha1.h' 2024-03-09T09:27:58.515 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha1.h’: No data available 2024-03-09T09:27:58.517 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strndup.c' -> './k/coreutils-8.5/lib/strndup.c' 2024-03-09T09:27:58.517 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strndup.c’: No data available 2024-03-09T09:27:58.518 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/priv-set.c' -> './k/coreutils-8.5/lib/priv-set.c' 2024-03-09T09:27:58.519 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/priv-set.c’: No data available 2024-03-09T09:27:58.520 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xfts.h' -> './k/coreutils-8.5/lib/xfts.h' 2024-03-09T09:27:58.520 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xfts.h’: No data available 2024-03-09T09:27:58.522 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/argv-iter.c' -> './k/coreutils-8.5/lib/argv-iter.c' 2024-03-09T09:27:58.522 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/argv-iter.c’: No data available 2024-03-09T09:27:58.523 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stdio--.h' -> './k/coreutils-8.5/lib/stdio--.h' 2024-03-09T09:27:58.524 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdio--.h’: No data available 2024-03-09T09:27:58.525 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/yesno.c' -> './k/coreutils-8.5/lib/yesno.c' 2024-03-09T09:27:58.525 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/yesno.c’: No data available 2024-03-09T09:27:58.528 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getopt.c' -> './k/coreutils-8.5/lib/getopt.c' 2024-03-09T09:27:58.528 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getopt.c’: No data available 2024-03-09T09:27:58.529 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strpbrk.c' -> './k/coreutils-8.5/lib/strpbrk.c' 2024-03-09T09:27:58.530 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strpbrk.c’: No data available 2024-03-09T09:27:58.531 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dirname-lgpl.c' -> './k/coreutils-8.5/lib/dirname-lgpl.c' 2024-03-09T09:27:58.531 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirname-lgpl.c’: No data available 2024-03-09T09:27:58.533 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/filemode.h' -> './k/coreutils-8.5/lib/filemode.h' 2024-03-09T09:27:58.533 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filemode.h’: No data available 2024-03-09T09:27:58.534 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbrtowc.c' -> './k/coreutils-8.5/lib/mbrtowc.c' 2024-03-09T09:27:58.535 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbrtowc.c’: No data available 2024-03-09T09:27:58.537 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv_open.c' -> './k/coreutils-8.5/lib/iconv_open.c' 2024-03-09T09:27:58.537 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open.c’: No data available 2024-03-09T09:27:58.539 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/euidaccess-stat.c' -> './k/coreutils-8.5/lib/euidaccess-stat.c' 2024-03-09T09:27:58.539 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/euidaccess-stat.c’: No data available 2024-03-09T09:27:58.541 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stat-time.h' -> './k/coreutils-8.5/lib/stat-time.h' 2024-03-09T09:27:58.541 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stat-time.h’: No data available 2024-03-09T09:27:58.543 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/frexpl.c' -> './k/coreutils-8.5/lib/frexpl.c' 2024-03-09T09:27:58.543 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/frexpl.c’: No data available 2024-03-09T09:27:58.545 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sys_select.in.h' -> './k/coreutils-8.5/lib/sys_select.in.h' 2024-03-09T09:27:58.545 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sys_select.in.h’: No data available 2024-03-09T09:27:58.553 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strftime.h' -> './k/coreutils-8.5/lib/strftime.h' 2024-03-09T09:27:58.553 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strftime.h’: No data available 2024-03-09T09:27:58.555 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strftime.c' -> './k/coreutils-8.5/lib/strftime.c' 2024-03-09T09:27:58.555 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strftime.c’: No data available 2024-03-09T09:27:58.556 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-aix.gperf' -> './k/coreutils-8.5/lib/iconv_open-aix.gperf' 2024-03-09T09:27:58.557 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-aix.gperf’: No data available 2024-03-09T09:27:58.558 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/buffer-lcm.c' -> './k/coreutils-8.5/lib/buffer-lcm.c' 2024-03-09T09:27:58.558 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/buffer-lcm.c’: No data available 2024-03-09T09:27:58.562 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unistr.h' -> './k/coreutils-8.5/lib/unistr.h' 2024-03-09T09:27:58.562 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistr.h’: No data available 2024-03-09T09:27:58.562 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/modechange.c' -> './k/coreutils-8.5/lib/modechange.c' 2024-03-09T09:27:58.563 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/modechange.c’: No data available 2024-03-09T09:27:58.564 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/tempname.h' -> './k/coreutils-8.5/lib/tempname.h' 2024-03-09T09:27:58.565 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/tempname.h’: No data available 2024-03-09T09:27:58.565 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/faccessat.c' -> './k/coreutils-8.5/lib/faccessat.c' 2024-03-09T09:27:58.565 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/faccessat.c’: No data available 2024-03-09T09:27:58.566 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unicodeio.h' -> './k/coreutils-8.5/lib/unicodeio.h' 2024-03-09T09:27:58.567 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unicodeio.h’: No data available 2024-03-09T09:27:58.568 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/filemode.c' -> './k/coreutils-8.5/lib/filemode.c' 2024-03-09T09:27:58.568 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filemode.c’: No data available 2024-03-09T09:27:58.570 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strdup.c' -> './k/coreutils-8.5/lib/strdup.c' 2024-03-09T09:27:58.570 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strdup.c’: No data available 2024-03-09T09:27:58.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strsignal.c' -> './k/coreutils-8.5/lib/strsignal.c' 2024-03-09T09:27:58.572 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strsignal.c’: No data available 2024-03-09T09:27:58.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/euidaccess.c' -> './k/coreutils-8.5/lib/euidaccess.c' 2024-03-09T09:27:58.573 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/euidaccess.c’: No data available 2024-03-09T09:27:58.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/select.c' -> './k/coreutils-8.5/lib/select.c' 2024-03-09T09:27:58.575 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/select.c’: No data available 2024-03-09T09:27:58.577 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memcoll.h' -> './k/coreutils-8.5/lib/memcoll.h' 2024-03-09T09:27:58.577 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcoll.h’: No data available 2024-03-09T09:27:58.579 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fseeko.c' -> './k/coreutils-8.5/lib/fseeko.c' 2024-03-09T09:27:58.579 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fseeko.c’: No data available 2024-03-09T09:27:58.581 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/langinfo.in.h' -> './k/coreutils-8.5/lib/langinfo.in.h' 2024-03-09T09:27:58.581 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/langinfo.in.h’: No data available 2024-03-09T09:27:58.583 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getopt.in.h' -> './k/coreutils-8.5/lib/getopt.in.h' 2024-03-09T09:27:58.583 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getopt.in.h’: No data available 2024-03-09T09:27:58.585 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/localcharset.c' -> './k/coreutils-8.5/lib/localcharset.c' 2024-03-09T09:27:58.585 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/localcharset.c’: No data available 2024-03-09T09:27:58.586 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/isapipe.c' -> './k/coreutils-8.5/lib/isapipe.c' 2024-03-09T09:27:58.587 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isapipe.c’: No data available 2024-03-09T09:27:58.588 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/settime.c' -> './k/coreutils-8.5/lib/settime.c' 2024-03-09T09:27:58.588 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/settime.c’: No data available 2024-03-09T09:27:58.590 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/bitrotate.h' -> './k/coreutils-8.5/lib/bitrotate.h' 2024-03-09T09:27:58.590 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/bitrotate.h’: No data available 2024-03-09T09:27:58.591 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/isnanl-nolibm.h' -> './k/coreutils-8.5/lib/isnanl-nolibm.h' 2024-03-09T09:27:58.591 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnanl-nolibm.h’: No data available 2024-03-09T09:27:58.593 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fcntl.c' -> './k/coreutils-8.5/lib/fcntl.c' 2024-03-09T09:27:58.593 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fcntl.c’: No data available 2024-03-09T09:27:58.594 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/exclude.h' -> './k/coreutils-8.5/lib/exclude.h' 2024-03-09T09:27:58.595 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/exclude.h’: No data available 2024-03-09T09:27:58.600 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/symlinkat.c' -> './k/coreutils-8.5/lib/symlinkat.c' 2024-03-09T09:27:58.600 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/symlinkat.c’: No data available 2024-03-09T09:27:58.601 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbsalign.h' -> './k/coreutils-8.5/lib/mbsalign.h' 2024-03-09T09:27:58.602 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsalign.h’: No data available 2024-03-09T09:27:58.603 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fpending.c' -> './k/coreutils-8.5/lib/fpending.c' 2024-03-09T09:27:58.603 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fpending.c’: No data available 2024-03-09T09:27:58.604 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/savewd.c' -> './k/coreutils-8.5/lib/savewd.c' 2024-03-09T09:27:58.605 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/savewd.c’: No data available 2024-03-09T09:27:58.606 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fchownat.c' -> './k/coreutils-8.5/lib/fchownat.c' 2024-03-09T09:27:58.606 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fchownat.c’: No data available 2024-03-09T09:27:58.608 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/root-dev-ino.h' -> './k/coreutils-8.5/lib/root-dev-ino.h' 2024-03-09T09:27:58.608 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/root-dev-ino.h’: No data available 2024-03-09T09:27:58.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.601+0000 7f4e23200640 1 Processor -- start 2024-03-09T09:27:58.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.601+0000 7f4e23200640 1 -- start start 2024-03-09T09:27:58.609 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/randread.h' -> './k/coreutils-8.5/lib/randread.h' 2024-03-09T09:27:58.609 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randread.h’: No data available 2024-03-09T09:27:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.602+0000 7f4e23200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e2414fd90 0x7f4e24156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.602+0000 7f4e23200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4e240c9380 0x7f4e240acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.602+0000 7f4e23200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240ad440 0x7f4e2406c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.602+0000 7f4e23200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2406ba60 con 0x7f4e2414fd90 2024-03-09T09:27:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.602+0000 7f4e23200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e24156bc0 con 0x7f4e240c9380 2024-03-09T09:27:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.602+0000 7f4e23200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240c9890 con 0x7f4e240ad440 2024-03-09T09:27:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.602+0000 7f4e21e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e2414fd90 0x7f4e24156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.603+0000 7f4e21e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4e2414fd90 0x7f4e24156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49750/0 (socket says 172.21.15.67:49750) 2024-03-09T09:27:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.603+0000 7f4e21e00640 1 -- 172.21.15.67:0/726262019 learned_addr learned my addr 172.21.15.67:0/726262019 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.603+0000 7f4e21400640 1 --2- 172.21.15.67:0/726262019 >> v2:172.21.15.134:3300/0 conn(0x7f4e240c9380 0x7f4e240acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.603+0000 7f4e22800640 1 --2- 172.21.15.67:0/726262019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240ad440 0x7f4e2406c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.603+0000 7f4e21e00640 1 -- 172.21.15.67:0/726262019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240ad440 msgr2=0x7f4e2406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.603+0000 7f4e21e00640 1 --2- 172.21.15.67:0/726262019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240ad440 0x7f4e2406c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.603+0000 7f4e21e00640 1 -- 172.21.15.67:0/726262019 >> v2:172.21.15.134:3300/0 conn(0x7f4e240c9380 msgr2=0x7f4e240acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:58.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.603+0000 7f4e21e00640 1 --2- 172.21.15.67:0/726262019 >> v2:172.21.15.134:3300/0 conn(0x7f4e240c9380 0x7f4e240acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.603+0000 7f4e21e00640 1 -- 172.21.15.67:0/726262019 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240c5f80 con 0x7f4e2414fd90 2024-03-09T09:27:58.613 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/freadseek.h' -> './k/coreutils-8.5/lib/freadseek.h' 2024-03-09T09:27:58.613 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadseek.h’: No data available 2024-03-09T09:27:58.614 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strtoll.c' -> './k/coreutils-8.5/lib/strtoll.c' 2024-03-09T09:27:58.614 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtoll.c’: No data available 2024-03-09T09:27:58.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.605+0000 7f4e21e00640 1 --2- 172.21.15.67:0/726262019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e2414fd90 0x7f4e24156680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4e240ae040 tx=0x7f4e1402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33a96ea2a2d18db server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.606+0000 7f4e20a00640 1 -- 172.21.15.67:0/726262019 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14038070 con 0x7f4e2414fd90 2024-03-09T09:27:58.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.606+0000 7f4e20a00640 1 -- 172.21.15.67:0/726262019 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1402f050 con 0x7f4e2414fd90 2024-03-09T09:27:58.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.606+0000 7f4e20a00640 1 -- 172.21.15.67:0/726262019 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14031970 con 0x7f4e2414fd90 2024-03-09T09:27:58.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.606+0000 7f4e23200640 1 -- 172.21.15.67:0/726262019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e2414fd90 msgr2=0x7f4e24156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.606+0000 7f4e23200640 1 --2- 172.21.15.67:0/726262019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e2414fd90 0x7f4e24156680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4e240ae040 tx=0x7f4e1402b7f0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.606+0000 7f4e23200640 1 -- 172.21.15.67:0/726262019 shutdown_connections 2024-03-09T09:27:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.606+0000 7f4e23200640 1 --2- 172.21.15.67:0/726262019 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240ad440 0x7f4e2406c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.616 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stdio-safer.h' -> './k/coreutils-8.5/lib/stdio-safer.h' 2024-03-09T09:27:58.616 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdio-safer.h’: No data available 2024-03-09T09:27:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.606+0000 7f4e23200640 1 --2- 172.21.15.67:0/726262019 >> v2:172.21.15.134:3300/0 conn(0x7f4e240c9380 0x7f4e240acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.606+0000 7f4e23200640 1 --2- 172.21.15.67:0/726262019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e2414fd90 0x7f4e24156680 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.606+0000 7f4e23200640 1 -- 172.21.15.67:0/726262019 >> 172.21.15.67:0/726262019 conn(0x7f4e240cb480 msgr2=0x7f4e240cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.607+0000 7f4e23200640 1 -- 172.21.15.67:0/726262019 shutdown_connections 2024-03-09T09:27:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.607+0000 7f4e23200640 1 -- 172.21.15.67:0/726262019 wait complete. 2024-03-09T09:27:58.617 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strtoumax.c' -> './k/coreutils-8.5/lib/strtoumax.c' 2024-03-09T09:27:58.618 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtoumax.c’: No data available 2024-03-09T09:27:58.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.608+0000 7f4e23200640 1 Processor -- start 2024-03-09T09:27:58.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.609+0000 7f4e23200640 1 -- start start 2024-03-09T09:27:58.619 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/argv-iter.h' -> './k/coreutils-8.5/lib/argv-iter.h' 2024-03-09T09:27:58.619 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/argv-iter.h’: No data available 2024-03-09T09:27:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.609+0000 7f4e23200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240ad440 0x7f4e24148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.609+0000 7f4e23200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240c9380 0x7f4e24148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.609+0000 7f4e23200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e2414fd90 0x7f4e240aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e23200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2406bcb0 con 0x7f4e2414fd90 2024-03-09T09:27:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e23200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240c6520 con 0x7f4e240ad440 2024-03-09T09:27:58.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e23200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2406c710 con 0x7f4e240c9380 2024-03-09T09:27:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e22800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e2414fd90 0x7f4e240aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e22800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e2414fd90 0x7f4e240aaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49760/0 (socket says 172.21.15.67:49760) 2024-03-09T09:27:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e22800640 1 -- 172.21.15.67:0/1514210523 learned_addr learned my addr 172.21.15.67:0/1514210523 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e21400640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240c9380 0x7f4e24148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e21e00640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240ad440 0x7f4e24148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e22800640 1 -- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240c9380 msgr2=0x7f4e24148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e22800640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240c9380 0x7f4e24148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e22800640 1 -- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240ad440 msgr2=0x7f4e24148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e22800640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240ad440 0x7f4e24148210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.610+0000 7f4e22800640 1 -- 172.21.15.67:0/1514210523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e241ec310 con 0x7f4e2414fd90 2024-03-09T09:27:58.623 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dirent-safer.h' -> './k/coreutils-8.5/lib/dirent-safer.h' 2024-03-09T09:27:58.623 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirent-safer.h’: No data available 2024-03-09T09:27:58.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.611+0000 7f4e22800640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e2414fd90 0x7f4e240aaf90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c007e60 tx=0x7f4e1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.611+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c014620 con 0x7f4e2414fd90 2024-03-09T09:27:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.611+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c019070 con 0x7f4e2414fd90 2024-03-09T09:27:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.611+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c014a20 con 0x7f4e2414fd90 2024-03-09T09:27:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.611+0000 7f4e23200640 1 -- 172.21.15.67:0/1514210523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e241ec5f0 con 0x7f4e2414fd90 2024-03-09T09:27:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.611+0000 7f4e23200640 1 -- 172.21.15.67:0/1514210523 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e24148c90 con 0x7f4e2414fd90 2024-03-09T09:27:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.612+0000 7f4e23200640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240ab4d0 0x7f4e240ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.612+0000 7f4e23200640 1 -- 172.21.15.67:0/1514210523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e24153a70 con 0x7f4e240ab4d0 2024-03-09T09:27:58.625 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dirent.in.h' -> './k/coreutils-8.5/lib/dirent.in.h' 2024-03-09T09:27:58.626 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirent.in.h’: No data available 2024-03-09T09:27:58.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.613+0000 7f4e21e00640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240ab4d0 0x7f4e240ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.614+0000 7f4e21e00640 1 -- 172.21.15.67:0/1514210523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e2404f3c0 con 0x7f4e240ab4d0 2024-03-09T09:27:58.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.614+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4e1c014bc0 con 0x7f4e2414fd90 2024-03-09T09:27:58.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.614+0000 7f4e21e00640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240ab4d0 0x7f4e240ab8b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4e240ada40 tx=0x7f4e140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.627 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/malloca.valgrind' -> './k/coreutils-8.5/lib/malloca.valgrind' 2024-03-09T09:27:58.627 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/malloca.valgrind’: No data available 2024-03-09T09:27:58.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.615+0000 7f4e1a000640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e0406b4d0 0x7f4e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.615+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f4e1c08ae50 con 0x7f4e2414fd90 2024-03-09T09:27:58.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.615+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14038040 con 0x7f4e240ab4d0 2024-03-09T09:27:58.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.615+0000 7f4e21400640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e0406b4d0 0x7f4e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.616+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4e14009490 con 0x7f4e240ab4d0 2024-03-09T09:27:58.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.616+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240ab4d0 msgr2=0x7f4e240ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.616+0000 7f4e1a000640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240ab4d0 0x7f4e240ab8b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4e240ada40 tx=0x7f4e140046c0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.616+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240ab4d0 msgr2=0x7f4e240ab8b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:58.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.616+0000 7f4e1a000640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240ab4d0 0x7f4e240ab8b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.630 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/savewd.h' -> './k/coreutils-8.5/lib/savewd.h' 2024-03-09T09:27:58.630 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/savewd.h’: No data available 2024-03-09T09:27:58.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.617+0000 7f4e21400640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e0406b4d0 0x7f4e0406d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4e2414cf40 tx=0x7f4e10008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.631 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/uinttostr.c' -> './k/coreutils-8.5/lib/uinttostr.c' 2024-03-09T09:27:58.631 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/uinttostr.c’: No data available 2024-03-09T09:27:58.631 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getline.c' -> './k/coreutils-8.5/lib/getline.c' 2024-03-09T09:27:58.632 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getline.c’: No data available 2024-03-09T09:27:58.632 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/chdir-long.h' -> './k/coreutils-8.5/lib/chdir-long.h' 2024-03-09T09:27:58.632 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/chdir-long.h’: No data available 2024-03-09T09:27:58.633 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/cycle-check.h' -> './k/coreutils-8.5/lib/cycle-check.h' 2024-03-09T09:27:58.633 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/cycle-check.h’: No data available 2024-03-09T09:27:58.633 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fts-cycle.c' -> './k/coreutils-8.5/lib/fts-cycle.c' 2024-03-09T09:27:58.634 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fts-cycle.c’: No data available 2024-03-09T09:27:58.634 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/file-has-acl.c' -> './k/coreutils-8.5/lib/file-has-acl.c' 2024-03-09T09:27:58.635 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/file-has-acl.c’: No data available 2024-03-09T09:27:58.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.626+0000 7f4e23200640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e241518a0 0x7f4e24151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.626+0000 7f4e23200640 1 -- 172.21.15.67:0/1514210523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e24149070 con 0x7f4e241518a0 2024-03-09T09:27:58.636 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/areadlink.h' -> './k/coreutils-8.5/lib/areadlink.h' 2024-03-09T09:27:58.636 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/areadlink.h’: No data available 2024-03-09T09:27:58.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.626+0000 7f4e21e00640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e241518a0 0x7f4e24151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.627+0000 7f4e21e00640 1 -- 172.21.15.67:0/1514210523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4e2406d2d0 con 0x7f4e241518a0 2024-03-09T09:27:58.637 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sig2str.h' -> './k/coreutils-8.5/lib/sig2str.h' 2024-03-09T09:27:58.637 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sig2str.h’: No data available 2024-03-09T09:27:58.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.628+0000 7f4e21e00640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e241518a0 0x7f4e24151c80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4e24153bf0 tx=0x7f4e14031000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.628+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1403dc30 con 0x7f4e241518a0 2024-03-09T09:27:58.638 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memcpy.c' -> './k/coreutils-8.5/lib/memcpy.c' 2024-03-09T09:27:58.639 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcpy.c’: No data available 2024-03-09T09:27:58.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.629+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5187 (secure 0 0 0) 0x7f4e1402f040 con 0x7f4e241518a0 2024-03-09T09:27:58.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.629+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e241518a0 msgr2=0x7f4e24151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.629+0000 7f4e1a000640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e241518a0 0x7f4e24151c80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4e24153bf0 tx=0x7f4e14031000 comp rx=0 tx=0).stop 2024-03-09T09:27:58.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.629+0000 7f4e1a000640 1 -- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e241518a0 msgr2=0x7f4e24151c80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:27:58.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.629+0000 7f4e1a000640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e241518a0 0x7f4e24151c80 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4e24153bf0 tx=0x7f4e14031000 comp rx=0 tx=0).stop 2024-03-09T09:27:58.640 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:27:58.640 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:58.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:58.640 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:27:58.640 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 282, 2024-03-09T09:27:58.640 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:27:58.641 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:27:58.641 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:27:58.641 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:27:58.641 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.641 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:27:58.641 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:58.641 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:27:58.641 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:27:58.641 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:58.641 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:58.642 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:58.642 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:58.642 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:58.642 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:58.642 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:58.642 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:58.642 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:58.642 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:58.642 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:58.642 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.643 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:27:58.643 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:27:58.643 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:58.643 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:58.643 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:58.643 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:58.643 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:58.643 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:58.643 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:58.643 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:58.643 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:58.644 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:58.644 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:58.644 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.644 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:27:58.644 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:27:58.644 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:27:58.644 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:27:58.644 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:27:58.644 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:27:58.644 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:27:58.645 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:27:58.646 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:27:58.646 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:27:58.646 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:27:58.646 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:27:58.646 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:27:58.646 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:27:58.646 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:27:58.646 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:27:58.646 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:27:58.646 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.646 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:27:58.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.647 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:27:58.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:27:58.647 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:27:58.647 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:58.647 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:58.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.647 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:58.647 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:27:58.647 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.648 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:58.648 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:27:58.648 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.648 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.648 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.648 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:27:58.648 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:27:58.648 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:58.649 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:58.649 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:58.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.649 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:27:58.649 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:27:58.649 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:58.649 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:58.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:58.649 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:27:58.650 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.650 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.650 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:58.650 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:27:58.650 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.650 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.650 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.650 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:27:58.651 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:27:58.651 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:58.651 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:58.651 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:58.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.651 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:27:58.651 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:27:58.651 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:27:58.651 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:27:58.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.652 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:27:58.652 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:27:58.652 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.652 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.652 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:27:58.652 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:27:58.652 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:27:58.652 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.652 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.653 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:58.654 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.654 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:58.654 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.654 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.654 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:27:58.654 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.654 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:58.654 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.654 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:58.654 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.654 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:27:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:27:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:27:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.655 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:27:58.656 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.656 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:58.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.656 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.656 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:58.656 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.656 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:27:58.656 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.656 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:27:58.656 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:27:58.656 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:27:58.657 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:27:58.657 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:27:58.657 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:27:58.657 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:27:58.657 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:27:58.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:27:58.657 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:27:58.657 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:27:58.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.630+0000 7f4e23200640 1 -- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e0406b4d0 msgr2=0x7f4e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.630+0000 7f4e23200640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4e0406b4d0 0x7f4e0406d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4e2414cf40 tx=0x7f4e10008040 comp rx=0 tx=0).stop 2024-03-09T09:27:58.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.630+0000 7f4e23200640 1 -- 172.21.15.67:0/1514210523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e2414fd90 msgr2=0x7f4e240aaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.630+0000 7f4e22800640 1 -- 172.21.15.67:0/1514210523 reap_dead start 2024-03-09T09:27:58.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.630+0000 7f4e23200640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e2414fd90 0x7f4e240aaf90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c007e60 tx=0x7f4e1c00e4b0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.630+0000 7f4e23200640 1 -- 172.21.15.67:0/1514210523 shutdown_connections 2024-03-09T09:27:58.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.630+0000 7f4e23200640 1 --2- 172.21.15.67:0/1514210523 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4e2414fd90 0x7f4e240aaf90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.630+0000 7f4e23200640 1 -- 172.21.15.67:0/1514210523 >> 172.21.15.67:0/1514210523 conn(0x7f4e240cb480 msgr2=0x7f4e2406cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:58.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.630+0000 7f4e23200640 1 -- 172.21.15.67:0/1514210523 shutdown_connections 2024-03-09T09:27:58.660 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/freading.h' -> './k/coreutils-8.5/lib/freading.h' 2024-03-09T09:27:58.660 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freading.h’: No data available 2024-03-09T09:27:58.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.630+0000 7f4e23200640 1 -- 172.21.15.67:0/1514210523 wait complete. 2024-03-09T09:27:58.661 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/full-write.h' -> './k/coreutils-8.5/lib/full-write.h' 2024-03-09T09:27:58.661 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/full-write.h’: No data available 2024-03-09T09:27:58.662 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unistd.in.h' -> './k/coreutils-8.5/lib/unistd.in.h' 2024-03-09T09:27:58.662 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistd.in.h’: No data available 2024-03-09T09:27:58.663 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/symlink.c' -> './k/coreutils-8.5/lib/symlink.c' 2024-03-09T09:27:58.663 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/symlink.c’: No data available 2024-03-09T09:27:58.663 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/se-selinux.in.h' -> './k/coreutils-8.5/lib/se-selinux.in.h' 2024-03-09T09:27:58.664 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/se-selinux.in.h’: No data available 2024-03-09T09:27:58.664 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/write-any-file.h' -> './k/coreutils-8.5/lib/write-any-file.h' 2024-03-09T09:27:58.664 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/write-any-file.h’: No data available 2024-03-09T09:27:58.665 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/printf-frexpl.h' -> './k/coreutils-8.5/lib/printf-frexpl.h' 2024-03-09T09:27:58.665 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-frexpl.h’: No data available 2024-03-09T09:27:58.666 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/Makefile.in' -> './k/coreutils-8.5/lib/Makefile.in' 2024-03-09T09:27:58.666 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/Makefile.in’: No data available 2024-03-09T09:27:58.666 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/filenamecat.h' -> './k/coreutils-8.5/lib/filenamecat.h' 2024-03-09T09:27:58.667 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filenamecat.h’: No data available 2024-03-09T09:27:58.667 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/malloca.c' -> './k/coreutils-8.5/lib/malloca.c' 2024-03-09T09:27:58.667 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/malloca.c’: No data available 2024-03-09T09:27:58.669 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:27:58.669 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:27:58.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:27:58.669 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/utimecmp.h' -> './k/coreutils-8.5/lib/utimecmp.h' 2024-03-09T09:27:58.670 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/utimecmp.h’: No data available 2024-03-09T09:27:58.670 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stdlib.in.h' -> './k/coreutils-8.5/lib/stdlib.in.h' 2024-03-09T09:27:58.670 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdlib.in.h’: No data available 2024-03-09T09:27:58.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stdio.in.h' -> './k/coreutils-8.5/lib/stdio.in.h' 2024-03-09T09:27:58.671 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdio.in.h’: No data available 2024-03-09T09:27:58.671 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbsinit.c' -> './k/coreutils-8.5/lib/mbsinit.c' 2024-03-09T09:27:58.672 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsinit.c’: No data available 2024-03-09T09:27:58.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/diacrit.c' -> './k/coreutils-8.5/lib/diacrit.c' 2024-03-09T09:27:58.673 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/diacrit.c’: No data available 2024-03-09T09:27:58.673 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/se-context.in.h' -> './k/coreutils-8.5/lib/se-context.in.h' 2024-03-09T09:27:58.673 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/se-context.in.h’: No data available 2024-03-09T09:27:58.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xvasprintf.c' -> './k/coreutils-8.5/lib/xvasprintf.c' 2024-03-09T09:27:58.674 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xvasprintf.c’: No data available 2024-03-09T09:27:58.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xgetcwd.c' -> './k/coreutils-8.5/lib/xgetcwd.c' 2024-03-09T09:27:58.675 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xgetcwd.c’: No data available 2024-03-09T09:27:58.675 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sockets.h' -> './k/coreutils-8.5/lib/sockets.h' 2024-03-09T09:27:58.676 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sockets.h’: No data available 2024-03-09T09:27:58.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/striconv.h' -> './k/coreutils-8.5/lib/striconv.h' 2024-03-09T09:27:58.676 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/striconv.h’: No data available 2024-03-09T09:27:58.677 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mkstemp-safer.c' -> './k/coreutils-8.5/lib/mkstemp-safer.c' 2024-03-09T09:27:58.677 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkstemp-safer.c’: No data available 2024-03-09T09:27:58.677 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/u64.h' -> './k/coreutils-8.5/lib/u64.h' 2024-03-09T09:27:58.678 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/u64.h’: No data available 2024-03-09T09:27:58.678 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstrtoimax.c' -> './k/coreutils-8.5/lib/xstrtoimax.c' 2024-03-09T09:27:58.678 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtoimax.c’: No data available 2024-03-09T09:27:58.679 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/time_r.c' -> './k/coreutils-8.5/lib/time_r.c' 2024-03-09T09:27:58.679 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/time_r.c’: No data available 2024-03-09T09:27:58.680 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbchar.h' -> './k/coreutils-8.5/lib/mbchar.h' 2024-03-09T09:27:58.680 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbchar.h’: No data available 2024-03-09T09:27:58.680 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xtime.h' -> './k/coreutils-8.5/lib/xtime.h' 2024-03-09T09:27:58.681 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xtime.h’: No data available 2024-03-09T09:27:58.681 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/set-mode-acl.c' -> './k/coreutils-8.5/lib/set-mode-acl.c' 2024-03-09T09:27:58.682 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/set-mode-acl.c’: No data available 2024-03-09T09:27:58.682 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/version-etc.c' -> './k/coreutils-8.5/lib/version-etc.c' 2024-03-09T09:27:58.682 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/version-etc.c’: No data available 2024-03-09T09:27:58.683 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dup2.c' -> './k/coreutils-8.5/lib/dup2.c' 2024-03-09T09:27:58.683 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dup2.c’: No data available 2024-03-09T09:27:58.683 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unlinkat.c' -> './k/coreutils-8.5/lib/unlinkat.c' 2024-03-09T09:27:58.684 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unlinkat.c’: No data available 2024-03-09T09:27:58.684 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sigprocmask.c' -> './k/coreutils-8.5/lib/sigprocmask.c' 2024-03-09T09:27:58.684 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sigprocmask.c’: No data available 2024-03-09T09:27:58.685 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/hash-triple.h' -> './k/coreutils-8.5/lib/hash-triple.h' 2024-03-09T09:27:58.685 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash-triple.h’: No data available 2024-03-09T09:27:58.685 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fprintftime.c' -> './k/coreutils-8.5/lib/fprintftime.c' 2024-03-09T09:27:58.686 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fprintftime.c’: No data available 2024-03-09T09:27:58.687 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unistd--.h' -> './k/coreutils-8.5/lib/unistd--.h' 2024-03-09T09:27:58.687 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/umaxtostr.c' -> './k/coreutils-8.5/lib/umaxtostr.c' 2024-03-09T09:27:58.687 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/i-ring.c' -> './k/coreutils-8.5/lib/i-ring.c' 2024-03-09T09:27:58.687 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/signbitd.c' -> './k/coreutils-8.5/lib/signbitd.c' 2024-03-09T09:27:58.687 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistd--.h’: No data available 2024-03-09T09:27:58.687 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/umaxtostr.c’: No data available 2024-03-09T09:27:58.687 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/i-ring.c’: No data available 2024-03-09T09:27:58.687 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/signbitd.c’: No data available 2024-03-09T09:27:58.688 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/ftello.c' -> './k/coreutils-8.5/lib/ftello.c' 2024-03-09T09:27:58.688 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ftello.c’: No data available 2024-03-09T09:27:58.689 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/lchown.c' -> './k/coreutils-8.5/lib/lchown.c' 2024-03-09T09:27:58.689 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/lchown.c’: No data available 2024-03-09T09:27:58.689 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/lstat.c' -> './k/coreutils-8.5/lib/lstat.c' 2024-03-09T09:27:58.690 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/lstat.c’: No data available 2024-03-09T09:27:58.690 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fpurge.c' -> './k/coreutils-8.5/lib/fpurge.c' 2024-03-09T09:27:58.691 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fpurge.c’: No data available 2024-03-09T09:27:58.692 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fstatat.c' -> './k/coreutils-8.5/lib/fstatat.c' 2024-03-09T09:27:58.692 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fstatat.c’: No data available 2024-03-09T09:27:58.695 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/lseek.c' -> './k/coreutils-8.5/lib/lseek.c' 2024-03-09T09:27:58.695 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/lseek.c’: No data available 2024-03-09T09:27:58.696 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fchmodat.c' -> './k/coreutils-8.5/lib/fchmodat.c' 2024-03-09T09:27:58.696 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fchmodat.c’: No data available 2024-03-09T09:27:58.697 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/alloca.c' -> './k/coreutils-8.5/lib/alloca.c' 2024-03-09T09:27:58.697 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/alloca.c’: No data available 2024-03-09T09:27:58.698 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memcmp.c' -> './k/coreutils-8.5/lib/memcmp.c' 2024-03-09T09:27:58.698 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcmp.c’: No data available 2024-03-09T09:27:58.699 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstrndup.h' -> './k/coreutils-8.5/lib/xstrndup.h' 2024-03-09T09:27:58.699 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrndup.h’: No data available 2024-03-09T09:27:58.700 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/frexp.c' -> './k/coreutils-8.5/lib/frexp.c' 2024-03-09T09:27:58.700 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/frexp.c’: No data available 2024-03-09T09:27:58.701 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/nproc.h' -> './k/coreutils-8.5/lib/nproc.h' 2024-03-09T09:27:58.701 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/nproc.h’: No data available 2024-03-09T09:27:58.703 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fsusage.h' -> './k/coreutils-8.5/lib/fsusage.h' 2024-03-09T09:27:58.703 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fsusage.h’: No data available 2024-03-09T09:27:58.704 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fd-safer-flag.c' -> './k/coreutils-8.5/lib/fd-safer-flag.c' 2024-03-09T09:27:58.704 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fd-safer-flag.c’: No data available 2024-03-09T09:27:58.705 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getdtablesize.c' -> './k/coreutils-8.5/lib/getdtablesize.c' 2024-03-09T09:27:58.705 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getdtablesize.c’: No data available 2024-03-09T09:27:58.706 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/savedir.c' -> './k/coreutils-8.5/lib/savedir.c' 2024-03-09T09:27:58.706 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/savedir.c’: No data available 2024-03-09T09:27:58.707 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/wcwidth.c' -> './k/coreutils-8.5/lib/wcwidth.c' 2024-03-09T09:27:58.707 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/wcwidth.c’: No data available 2024-03-09T09:27:58.708 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbsstr.c' -> './k/coreutils-8.5/lib/mbsstr.c' 2024-03-09T09:27:58.708 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsstr.c’: No data available 2024-03-09T09:27:58.709 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fpucw.h' -> './k/coreutils-8.5/lib/fpucw.h' 2024-03-09T09:27:58.709 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fpucw.h’: No data available 2024-03-09T09:27:58.710 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-hpux.h' -> './k/coreutils-8.5/lib/iconv_open-hpux.h' 2024-03-09T09:27:58.711 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-hpux.h’: No data available 2024-03-09T09:27:58.711 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/readtokens0.h' -> './k/coreutils-8.5/lib/readtokens0.h' 2024-03-09T09:27:58.711 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readtokens0.h’: No data available 2024-03-09T09:27:58.713 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getdate.c' -> './k/coreutils-8.5/lib/getdate.c' 2024-03-09T09:27:58.713 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getdate.c’: No data available 2024-03-09T09:27:58.714 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/utimens.c' -> './k/coreutils-8.5/lib/utimens.c' 2024-03-09T09:27:58.714 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/utimens.c’: No data available 2024-03-09T09:27:58.716 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getndelim2.c' -> './k/coreutils-8.5/lib/getndelim2.c' 2024-03-09T09:27:58.716 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getndelim2.c’: No data available 2024-03-09T09:27:58.717 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fflush.c' -> './k/coreutils-8.5/lib/fflush.c' 2024-03-09T09:27:58.717 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fflush.c’: No data available 2024-03-09T09:27:58.718 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/version-etc-fsf.c' -> './k/coreutils-8.5/lib/version-etc-fsf.c' 2024-03-09T09:27:58.718 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/version-etc-fsf.c’: No data available 2024-03-09T09:27:58.719 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xsize.h' -> './k/coreutils-8.5/lib/xsize.h' 2024-03-09T09:27:58.719 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xsize.h’: No data available 2024-03-09T09:27:58.720 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xgethostname.c' -> './k/coreutils-8.5/lib/xgethostname.c' 2024-03-09T09:27:58.720 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xgethostname.c’: No data available 2024-03-09T09:27:58.721 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strnlen1.h' -> './k/coreutils-8.5/lib/strnlen1.h' 2024-03-09T09:27:58.721 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnlen1.h’: No data available 2024-03-09T09:27:58.722 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/isnanl.c' -> './k/coreutils-8.5/lib/isnanl.c' 2024-03-09T09:27:58.722 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnanl.c’: No data available 2024-03-09T09:27:58.731 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/same-inode.h' -> './k/coreutils-8.5/lib/same-inode.h' 2024-03-09T09:27:58.731 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/same-inode.h’: No data available 2024-03-09T09:27:58.732 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/regex_internal.h' -> './k/coreutils-8.5/lib/regex_internal.h' 2024-03-09T09:27:58.733 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regex_internal.h’: No data available 2024-03-09T09:27:58.735 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/full-read.h' -> './k/coreutils-8.5/lib/full-read.h' 2024-03-09T09:27:58.735 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/full-read.h’: No data available 2024-03-09T09:27:58.736 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/intprops.h' -> './k/coreutils-8.5/lib/intprops.h' 2024-03-09T09:27:58.737 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/intprops.h’: No data available 2024-03-09T09:27:58.738 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fsusage.c' -> './k/coreutils-8.5/lib/fsusage.c' 2024-03-09T09:27:58.738 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fsusage.c’: No data available 2024-03-09T09:27:58.739 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/save-cwd.c' -> './k/coreutils-8.5/lib/save-cwd.c' 2024-03-09T09:27:58.739 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/save-cwd.c’: No data available 2024-03-09T09:27:58.740 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memchr.valgrind' -> './k/coreutils-8.5/lib/memchr.valgrind' 2024-03-09T09:27:58.740 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memchr.valgrind’: No data available 2024-03-09T09:27:58.741 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/exitfail.h' -> './k/coreutils-8.5/lib/exitfail.h' 2024-03-09T09:27:58.741 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/exitfail.h’: No data available 2024-03-09T09:27:58.743 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/printf-frexp.h' -> './k/coreutils-8.5/lib/printf-frexp.h' 2024-03-09T09:27:58.743 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-frexp.h’: No data available 2024-03-09T09:27:58.743 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/isblank.c' -> './k/coreutils-8.5/lib/isblank.c' 2024-03-09T09:27:58.744 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isblank.c’: No data available 2024-03-09T09:27:58.745 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/arpa_inet.in.h' -> './k/coreutils-8.5/lib/arpa_inet.in.h' 2024-03-09T09:27:58.746 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/arpa_inet.in.h’: No data available 2024-03-09T09:27:58.746 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strerror.c' -> './k/coreutils-8.5/lib/strerror.c' 2024-03-09T09:27:58.747 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strerror.c’: No data available 2024-03-09T09:27:58.748 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/freadptr.c' -> './k/coreutils-8.5/lib/freadptr.c' 2024-03-09T09:27:58.748 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadptr.c’: No data available 2024-03-09T09:27:58.752 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/openat.h' -> './k/coreutils-8.5/lib/openat.h' 2024-03-09T09:27:58.752 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat.h’: No data available 2024-03-09T09:27:58.753 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-irix.h' -> './k/coreutils-8.5/lib/iconv_open-irix.h' 2024-03-09T09:27:58.754 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-irix.h’: No data available 2024-03-09T09:27:58.755 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/open.c' -> './k/coreutils-8.5/lib/open.c' 2024-03-09T09:27:58.755 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/open.c’: No data available 2024-03-09T09:27:58.757 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/vprintf.c' -> './k/coreutils-8.5/lib/vprintf.c' 2024-03-09T09:27:58.757 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/vprintf.c’: No data available 2024-03-09T09:27:58.758 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/same.h' -> './k/coreutils-8.5/lib/same.h' 2024-03-09T09:27:58.759 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/same.h’: No data available 2024-03-09T09:27:58.760 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/gai_strerror.c' -> './k/coreutils-8.5/lib/gai_strerror.c' 2024-03-09T09:27:58.760 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gai_strerror.c’: No data available 2024-03-09T09:27:58.761 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/imaxtostr.c' -> './k/coreutils-8.5/lib/imaxtostr.c' 2024-03-09T09:27:58.761 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/imaxtostr.c’: No data available 2024-03-09T09:27:58.762 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memcmp2.h' -> './k/coreutils-8.5/lib/memcmp2.h' 2024-03-09T09:27:58.763 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcmp2.h’: No data available 2024-03-09T09:27:58.765 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xfreopen.c' -> './k/coreutils-8.5/lib/xfreopen.c' 2024-03-09T09:27:58.766 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xfreopen.c’: No data available 2024-03-09T09:27:58.766 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/rand-isaac.c' -> './k/coreutils-8.5/lib/rand-isaac.c' 2024-03-09T09:27:58.766 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/rand-isaac.c’: No data available 2024-03-09T09:27:58.767 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/full-write.c' -> './k/coreutils-8.5/lib/full-write.c' 2024-03-09T09:27:58.767 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/full-write.c’: No data available 2024-03-09T09:27:58.768 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/rmdir.c' -> './k/coreutils-8.5/lib/rmdir.c' 2024-03-09T09:27:58.769 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/rmdir.c’: No data available 2024-03-09T09:27:58.769 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/save-cwd.h' -> './k/coreutils-8.5/lib/save-cwd.h' 2024-03-09T09:27:58.770 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/save-cwd.h’: No data available 2024-03-09T09:27:58.771 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/at-func2.c' -> './k/coreutils-8.5/lib/at-func2.c' 2024-03-09T09:27:58.771 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/at-func2.c’: No data available 2024-03-09T09:27:58.773 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/gethrxtime.c' -> './k/coreutils-8.5/lib/gethrxtime.c' 2024-03-09T09:27:58.773 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gethrxtime.c’: No data available 2024-03-09T09:27:58.775 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strtod.c' -> './k/coreutils-8.5/lib/strtod.c' 2024-03-09T09:27:58.776 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtod.c’: No data available 2024-03-09T09:27:58.777 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbsalign.c' -> './k/coreutils-8.5/lib/mbsalign.c' 2024-03-09T09:27:58.778 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsalign.c’: No data available 2024-03-09T09:27:58.779 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-osf.gperf' -> './k/coreutils-8.5/lib/iconv_open-osf.gperf' 2024-03-09T09:27:58.780 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-osf.gperf’: No data available 2024-03-09T09:27:58.781 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/ref-add.sin' -> './k/coreutils-8.5/lib/ref-add.sin' 2024-03-09T09:27:58.781 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ref-add.sin’: No data available 2024-03-09T09:27:58.782 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memcmp2.c' -> './k/coreutils-8.5/lib/memcmp2.c' 2024-03-09T09:27:58.783 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcmp2.c’: No data available 2024-03-09T09:27:58.785 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/gnulib.mk' -> './k/coreutils-8.5/lib/gnulib.mk' 2024-03-09T09:27:58.785 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gnulib.mk’: No data available 2024-03-09T09:27:58.786 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/ref-del.sin' -> './k/coreutils-8.5/lib/ref-del.sin' 2024-03-09T09:27:58.787 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ref-del.sin’: No data available 2024-03-09T09:27:58.788 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stpncpy.c' -> './k/coreutils-8.5/lib/stpncpy.c' 2024-03-09T09:27:58.788 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stpncpy.c’: No data available 2024-03-09T09:27:58.789 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv.in.h' -> './k/coreutils-8.5/lib/iconv.in.h' 2024-03-09T09:27:58.789 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv.in.h’: No data available 2024-03-09T09:27:58.790 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/posixtm.h' -> './k/coreutils-8.5/lib/posixtm.h' 2024-03-09T09:27:58.791 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/posixtm.h’: No data available 2024-03-09T09:27:58.792 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstrtol.c' -> './k/coreutils-8.5/lib/xstrtol.c' 2024-03-09T09:27:58.792 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtol.c’: No data available 2024-03-09T09:27:58.794 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getopt1.c' -> './k/coreutils-8.5/lib/getopt1.c' 2024-03-09T09:27:58.794 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getopt1.c’: No data available 2024-03-09T09:27:58.795 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/quotearg.h' -> './k/coreutils-8.5/lib/quotearg.h' 2024-03-09T09:27:58.795 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/quotearg.h’: No data available 2024-03-09T09:27:58.797 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/cycle-check.c' -> './k/coreutils-8.5/lib/cycle-check.c' 2024-03-09T09:27:58.797 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/cycle-check.c’: No data available 2024-03-09T09:27:58.799 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/calloc.c' -> './k/coreutils-8.5/lib/calloc.c' 2024-03-09T09:27:58.799 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/calloc.c’: No data available 2024-03-09T09:27:58.801 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/areadlinkat.c' -> './k/coreutils-8.5/lib/areadlinkat.c' 2024-03-09T09:27:58.801 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/areadlinkat.c’: No data available 2024-03-09T09:27:58.802 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/hash-triple.c' -> './k/coreutils-8.5/lib/hash-triple.c' 2024-03-09T09:27:58.803 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash-triple.c’: No data available 2024-03-09T09:27:58.803 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getpass.c' -> './k/coreutils-8.5/lib/getpass.c' 2024-03-09T09:27:58.804 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getpass.c’: No data available 2024-03-09T09:27:58.805 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstriconv.c' -> './k/coreutils-8.5/lib/xstriconv.c' 2024-03-09T09:27:58.806 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstriconv.c’: No data available 2024-03-09T09:27:58.807 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dirent--.h' -> './k/coreutils-8.5/lib/dirent--.h' 2024-03-09T09:27:58.807 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirent--.h’: No data available 2024-03-09T09:27:58.808 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sigaction.c' -> './k/coreutils-8.5/lib/sigaction.c' 2024-03-09T09:27:58.808 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sigaction.c’: No data available 2024-03-09T09:27:58.814 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/linkat.c' -> './k/coreutils-8.5/lib/linkat.c' 2024-03-09T09:27:58.814 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/linkat.c’: No data available 2024-03-09T09:27:58.816 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unitypes.h' -> './k/coreutils-8.5/lib/unitypes.h' 2024-03-09T09:27:58.816 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unitypes.h’: No data available 2024-03-09T09:27:58.817 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/linebuffer.c' -> './k/coreutils-8.5/lib/linebuffer.c' 2024-03-09T09:27:58.817 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/linebuffer.c’: No data available 2024-03-09T09:27:58.819 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fnmatch.c' -> './k/coreutils-8.5/lib/fnmatch.c' 2024-03-09T09:27:58.819 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fnmatch.c’: No data available 2024-03-09T09:27:58.820 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xasprintf.c' -> './k/coreutils-8.5/lib/xasprintf.c' 2024-03-09T09:27:58.820 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xasprintf.c’: No data available 2024-03-09T09:27:58.822 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/printf-parse.h' -> './k/coreutils-8.5/lib/printf-parse.h' 2024-03-09T09:27:58.822 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-parse.h’: No data available 2024-03-09T09:27:58.824 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/string.in.h' -> './k/coreutils-8.5/lib/string.in.h' 2024-03-09T09:27:58.825 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/string.in.h’: No data available 2024-03-09T09:27:58.827 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mkdir.c' -> './k/coreutils-8.5/lib/mkdir.c' 2024-03-09T09:27:58.827 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkdir.c’: No data available 2024-03-09T09:27:58.829 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fchown-stub.c' -> './k/coreutils-8.5/lib/fchown-stub.c' 2024-03-09T09:27:58.829 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fchown-stub.c’: No data available 2024-03-09T09:27:58.831 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/human.c' -> './k/coreutils-8.5/lib/human.c' 2024-03-09T09:27:58.831 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/human.c’: No data available 2024-03-09T09:27:58.832 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sockets.c' -> './k/coreutils-8.5/lib/sockets.c' 2024-03-09T09:27:58.833 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sockets.c’: No data available 2024-03-09T09:27:58.835 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/canonicalize.h' -> './k/coreutils-8.5/lib/canonicalize.h' 2024-03-09T09:27:58.835 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/canonicalize.h’: No data available 2024-03-09T09:27:58.836 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getopt_int.h' -> './k/coreutils-8.5/lib/getopt_int.h' 2024-03-09T09:27:58.837 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getopt_int.h’: No data available 2024-03-09T09:27:58.838 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/version-etc.h' -> './k/coreutils-8.5/lib/version-etc.h' 2024-03-09T09:27:58.838 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/version-etc.h’: No data available 2024-03-09T09:27:58.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.877+0000 7f9132c00640 1 Processor -- start 2024-03-09T09:27:58.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.877+0000 7f9132c00640 1 -- start start 2024-03-09T09:27:58.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.878+0000 7f9132c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f912c0730c0 0x7f912c0aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.878+0000 7f9132c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f912c06d830 0x7f912c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.879+0000 7f9132c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912c152e00 con 0x7f912c0734a0 2024-03-09T09:27:58.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.879+0000 7f9132c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912c0aa960 con 0x7f912c06d830 2024-03-09T09:27:58.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.879+0000 7f9132c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912c1530e0 con 0x7f912c0730c0 2024-03-09T09:27:58.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.879+0000 7f9130c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f912c06d830 0x7f912c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.879+0000 7f9130c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f912c06d830 0x7f912c06dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40986/0 (socket says 172.21.15.67:40986) 2024-03-09T09:27:58.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.879+0000 7f9130c00640 1 -- 172.21.15.67:0/750749633 learned_addr learned my addr 172.21.15.67:0/750749633 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:58.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.879+0000 7f912b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f912c0734a0 0x7f912c0c2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36314/0 (socket says 172.21.15.67:36314) 2024-03-09T09:27:58.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.879+0000 7f912be00640 1 --2- 172.21.15.67:0/750749633 >> v2:172.21.15.134:3300/0 conn(0x7f912c0730c0 0x7f912c0aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.879+0000 7f9130c00640 1 -- 172.21.15.67:0/750749633 >> v2:172.21.15.134:3300/0 conn(0x7f912c0730c0 msgr2=0x7f912c0aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:27:58.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.879+0000 7f9130c00640 1 --2- 172.21.15.67:0/750749633 >> v2:172.21.15.134:3300/0 conn(0x7f912c0730c0 0x7f912c0aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.880+0000 7f9130c00640 1 -- 172.21.15.67:0/750749633 >> v1:172.21.15.67:6789/0 conn(0x7f912c0734a0 legacy=0x7f912c0c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.880+0000 7f9130c00640 1 -- 172.21.15.67:0/750749633 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c1533c0 con 0x7f912c06d830 2024-03-09T09:27:58.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.880+0000 7f912aa00640 1 -- 172.21.15.67:0/750749633 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037959482 0 0) 0x7f912c152e00 con 0x7f912c0734a0 2024-03-09T09:27:58.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.881+0000 7f9130c00640 1 --2- 172.21.15.67:0/750749633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f912c06d830 0x7f912c06dc60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f912c06c090 tx=0x7f912402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c464b3abcd100c03 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.881+0000 7f912aa00640 1 -- 172.21.15.67:0/750749633 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912402ca80 con 0x7f912c06d830 2024-03-09T09:27:58.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.881+0000 7f912aa00640 1 -- 172.21.15.67:0/750749633 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912402cc20 con 0x7f912c06d830 2024-03-09T09:27:58.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.881+0000 7f912aa00640 1 -- 172.21.15.67:0/750749633 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91240343f0 con 0x7f912c06d830 2024-03-09T09:27:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.882+0000 7f9132c00640 1 -- 172.21.15.67:0/750749633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f912c06d830 msgr2=0x7f912c06dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.882+0000 7f9132c00640 1 --2- 172.21.15.67:0/750749633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f912c06d830 0x7f912c06dc60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f912c06c090 tx=0x7f912402bee0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.882+0000 7f9132c00640 1 -- 172.21.15.67:0/750749633 shutdown_connections 2024-03-09T09:27:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.882+0000 7f9132c00640 1 --2- 172.21.15.67:0/750749633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f912c06d830 0x7f912c06dc60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.882+0000 7f9132c00640 1 --2- 172.21.15.67:0/750749633 >> v2:172.21.15.134:3300/0 conn(0x7f912c0730c0 0x7f912c0aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.882+0000 7f9132c00640 1 -- 172.21.15.67:0/750749633 >> 172.21.15.67:0/750749633 conn(0x7f912c0c7370 msgr2=0x7f912c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:27:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.882+0000 7f9132c00640 1 -- 172.21.15.67:0/750749633 shutdown_connections 2024-03-09T09:27:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.882+0000 7f9132c00640 1 -- 172.21.15.67:0/750749633 wait complete. 2024-03-09T09:27:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.884+0000 7f9132c00640 1 Processor -- start 2024-03-09T09:27:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.884+0000 7f9132c00640 1 -- start start 2024-03-09T09:27:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f9132c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c06d830 0x7f912c14da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f9132c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f912c0730c0 0x7f912c1ee930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f9132c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c0734a0 0x7f912c154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f9132c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0c3590 con 0x7f912c0730c0 2024-03-09T09:27:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f9132c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c153980 con 0x7f912c06d830 2024-03-09T09:27:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f9132c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0c3710 con 0x7f912c0734a0 2024-03-09T09:27:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f912b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f912c0730c0 0x7f912c1ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f912b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f912c0730c0 0x7f912c1ee930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40988/0 (socket says 172.21.15.67:40988) 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f912b400640 1 -- 172.21.15.67:0/1905258548 learned_addr learned my addr 172.21.15.67:0/1905258548 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f9130c00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c0734a0 0x7f912c154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c06d830 0x7f912c14da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f912b400640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c0734a0 msgr2=0x7f912c154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f912b400640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c0734a0 0x7f912c154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f912b400640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c06d830 msgr2=0x7f912c14da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.885+0000 7f912b400640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c06d830 0x7f912c14da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.886+0000 7f912b400640 1 -- 172.21.15.67:0/1905258548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c0aa6c0 con 0x7f912c0730c0 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.886+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f912c06d830 0x7f912c14da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.886+0000 7f912b400640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f912c0730c0 0x7f912c1ee930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f911800c440 tx=0x7f911800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.886+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118018070 con 0x7f912c0730c0 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.886+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91180074e0 con 0x7f912c0730c0 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.886+0000 7f9132c00640 1 -- 172.21.15.67:0/1905258548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c155910 con 0x7f912c0730c0 2024-03-09T09:27:58.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.886+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118004a20 con 0x7f912c0730c0 2024-03-09T09:27:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.886+0000 7f9132c00640 1 -- 172.21.15.67:0/1905258548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c1ef250 con 0x7f912c0730c0 2024-03-09T09:27:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.887+0000 7f9132c00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.887+0000 7f9132c00640 1 -- 172.21.15.67:0/1905258548 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c155ae0 con 0x7f912c1eee70 2024-03-09T09:27:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.888+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.888+0000 7f912be00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 msgr2=0x7f912c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-03-09T09:27:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.888+0000 7f912be00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 msgr2=0x7f912c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:27:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.888+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.888+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:27:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.889+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f911801b070 con 0x7f912c0730c0 2024-03-09T09:27:58.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.890+0000 7f9123e00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f910c06b4d0 0x7f910c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:27:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.890+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f9118089810 con 0x7f912c0730c0 2024-03-09T09:27:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.890+0000 7f9130c00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f910c06b4d0 0x7f910c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:58.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:58.891+0000 7f9130c00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f910c06b4d0 0x7f910c06d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9124035000 tx=0x7f9124030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:27:59.005 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stdlib--.h' -> './k/coreutils-8.5/lib/stdlib--.h' 2024-03-09T09:27:59.006 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdlib--.h’: No data available 2024-03-09T09:27:59.008 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/backupfile.h' -> './k/coreutils-8.5/lib/backupfile.h' 2024-03-09T09:27:59.008 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/backupfile.h’: No data available 2024-03-09T09:27:59.010 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/file-type.c' -> './k/coreutils-8.5/lib/file-type.c' 2024-03-09T09:27:59.010 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/file-type.c’: No data available 2024-03-09T09:27:59.013 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/file-set.h' -> './k/coreutils-8.5/lib/file-set.h' 2024-03-09T09:27:59.013 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/file-set.h’: No data available 2024-03-09T09:27:59.015 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbuiter.h' -> './k/coreutils-8.5/lib/mbuiter.h' 2024-03-09T09:27:59.015 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbuiter.h’: No data available 2024-03-09T09:27:59.016 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fcntl.in.h' -> './k/coreutils-8.5/lib/fcntl.in.h' 2024-03-09T09:27:59.017 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fcntl.in.h’: No data available 2024-03-09T09:27:59.018 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/inet_ntop.c' -> './k/coreutils-8.5/lib/inet_ntop.c' 2024-03-09T09:27:59.018 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/inet_ntop.c’: No data available 2024-03-09T09:27:59.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fseterr.c' -> './k/coreutils-8.5/lib/fseterr.c' 2024-03-09T09:27:59.020 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fseterr.c’: No data available 2024-03-09T09:27:59.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/close-stream.h' -> './k/coreutils-8.5/lib/close-stream.h' 2024-03-09T09:27:59.023 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/close-stream.h’: No data available 2024-03-09T09:27:59.024 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/freopen-safer.c' -> './k/coreutils-8.5/lib/freopen-safer.c' 2024-03-09T09:27:59.024 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freopen-safer.c’: No data available 2024-03-09T09:27:59.026 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stat-macros.h' -> './k/coreutils-8.5/lib/stat-macros.h' 2024-03-09T09:27:59.026 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stat-macros.h’: No data available 2024-03-09T09:27:59.027 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dirname.c' -> './k/coreutils-8.5/lib/dirname.c' 2024-03-09T09:27:59.027 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirname.c’: No data available 2024-03-09T09:27:59.029 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dev-ino.h' -> './k/coreutils-8.5/lib/dev-ino.h' 2024-03-09T09:27:59.029 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dev-ino.h’: No data available 2024-03-09T09:27:59.034 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xalloc.h' -> './k/coreutils-8.5/lib/xalloc.h' 2024-03-09T09:27:59.034 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xalloc.h’: No data available 2024-03-09T09:27:59.035 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memcasecmp.h' -> './k/coreutils-8.5/lib/memcasecmp.h' 2024-03-09T09:27:59.036 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcasecmp.h’: No data available 2024-03-09T09:27:59.038 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/hash.c' -> './k/coreutils-8.5/lib/hash.c' 2024-03-09T09:27:59.038 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash.c’: No data available 2024-03-09T09:27:59.041 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/userspec.c' -> './k/coreutils-8.5/lib/userspec.c' 2024-03-09T09:27:59.041 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/userspec.c’: No data available 2024-03-09T09:27:59.043 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/hard-locale.h' -> './k/coreutils-8.5/lib/hard-locale.h' 2024-03-09T09:27:59.044 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hard-locale.h’: No data available 2024-03-09T09:27:59.046 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xvasprintf.h' -> './k/coreutils-8.5/lib/xvasprintf.h' 2024-03-09T09:27:59.046 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xvasprintf.h’: No data available 2024-03-09T09:27:59.048 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/wcrtomb.c' -> './k/coreutils-8.5/lib/wcrtomb.c' 2024-03-09T09:27:59.048 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/wcrtomb.c’: No data available 2024-03-09T09:27:59.050 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/gethrxtime.h' -> './k/coreutils-8.5/lib/gethrxtime.h' 2024-03-09T09:27:59.050 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gethrxtime.h’: No data available 2024-03-09T09:27:59.052 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fprintftime.h' -> './k/coreutils-8.5/lib/fprintftime.h' 2024-03-09T09:27:59.053 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fprintftime.h’: No data available 2024-03-09T09:27:59.055 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mgetgroups.c' -> './k/coreutils-8.5/lib/mgetgroups.c' 2024-03-09T09:27:59.055 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mgetgroups.c’: No data available 2024-03-09T09:27:59.056 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/randread.c' -> './k/coreutils-8.5/lib/randread.c' 2024-03-09T09:27:59.057 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randread.c’: No data available 2024-03-09T09:27:59.059 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstriconv.h' -> './k/coreutils-8.5/lib/xstriconv.h' 2024-03-09T09:27:59.059 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstriconv.h’: No data available 2024-03-09T09:27:59.061 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dup-safer-flag.c' -> './k/coreutils-8.5/lib/dup-safer-flag.c' 2024-03-09T09:27:59.061 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dup-safer-flag.c’: No data available 2024-03-09T09:27:59.062 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/yesno.h' -> './k/coreutils-8.5/lib/yesno.h' 2024-03-09T09:27:59.062 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/yesno.h’: No data available 2024-03-09T09:27:59.063 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbscasecmp.c' -> './k/coreutils-8.5/lib/mbscasecmp.c' 2024-03-09T09:27:59.064 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbscasecmp.c’: No data available 2024-03-09T09:27:59.065 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strtoimax.c' -> './k/coreutils-8.5/lib/strtoimax.c' 2024-03-09T09:27:59.065 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtoimax.c’: No data available 2024-03-09T09:27:59.066 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbsrtowcs-state.c' -> './k/coreutils-8.5/lib/mbsrtowcs-state.c' 2024-03-09T09:27:59.066 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsrtowcs-state.c’: No data available 2024-03-09T09:27:59.067 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mkstemp.c' -> './k/coreutils-8.5/lib/mkstemp.c' 2024-03-09T09:27:59.067 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkstemp.c’: No data available 2024-03-09T09:27:59.068 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xmemcoll.c' -> './k/coreutils-8.5/lib/xmemcoll.c' 2024-03-09T09:27:59.069 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xmemcoll.c’: No data available 2024-03-09T09:27:59.070 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strintcmp.c' -> './k/coreutils-8.5/lib/strintcmp.c' 2024-03-09T09:27:59.070 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strintcmp.c’: No data available 2024-03-09T09:27:59.071 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fclose.c' -> './k/coreutils-8.5/lib/fclose.c' 2024-03-09T09:27:59.071 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fclose.c’: No data available 2024-03-09T09:27:59.072 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fd-reopen.c' -> './k/coreutils-8.5/lib/fd-reopen.c' 2024-03-09T09:27:59.073 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fd-reopen.c’: No data available 2024-03-09T09:27:59.074 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/inttostr.c' -> './k/coreutils-8.5/lib/inttostr.c' 2024-03-09T09:27:59.074 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/inttostr.c’: No data available 2024-03-09T09:27:59.076 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/c-ctype.c' -> './k/coreutils-8.5/lib/c-ctype.c' 2024-03-09T09:27:59.076 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-ctype.c’: No data available 2024-03-09T09:27:59.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/openat-die.c' -> './k/coreutils-8.5/lib/openat-die.c' 2024-03-09T09:27:59.078 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat-die.c’: No data available 2024-03-09T09:27:59.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/error.c' -> './k/coreutils-8.5/lib/error.c' 2024-03-09T09:27:59.080 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/error.c’: No data available 2024-03-09T09:27:59.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sig2str.c' -> './k/coreutils-8.5/lib/sig2str.c' 2024-03-09T09:27:59.082 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sig2str.c’: No data available 2024-03-09T09:27:59.083 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unlocked-io.h' -> './k/coreutils-8.5/lib/unlocked-io.h' 2024-03-09T09:27:59.083 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unlocked-io.h’: No data available 2024-03-09T09:27:59.085 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memxfrm.c' -> './k/coreutils-8.5/lib/memxfrm.c' 2024-03-09T09:27:59.085 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memxfrm.c’: No data available 2024-03-09T09:27:59.087 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/pipe-safer.c' -> './k/coreutils-8.5/lib/pipe-safer.c' 2024-03-09T09:27:59.087 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/pipe-safer.c’: No data available 2024-03-09T09:27:59.088 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/backupfile.c' -> './k/coreutils-8.5/lib/backupfile.c' 2024-03-09T09:27:59.088 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/backupfile.c’: No data available 2024-03-09T09:27:59.090 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-osf.h' -> './k/coreutils-8.5/lib/iconv_open-osf.h' 2024-03-09T09:27:59.090 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-osf.h’: No data available 2024-03-09T09:27:59.091 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/linebuffer.h' -> './k/coreutils-8.5/lib/linebuffer.h' 2024-03-09T09:27:59.092 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/linebuffer.h’: No data available 2024-03-09T09:27:59.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/gnulib-tests' 2024-03-09T09:27:59.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/AUTHORS' 2024-03-09T09:27:59.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_po_check' 2024-03-09T09:27:59.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/dist-check.mk' 2024-03-09T09:27:59.093 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/ChangeLog-2008' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_trailing_blank' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_file_system' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/README' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/configure.ac' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_system_h_headers' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/THANKStt.in' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/old/textutils/ChangeLog' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/old/textutils/NEWS' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/old/textutils' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/old/sh-utils/ChangeLog' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/old/sh-utils/NEWS' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/old/sh-utils/ChangeLog.0' 2024-03-09T09:27:59.094 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/old/sh-utils' 2024-03-09T09:27:59.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/old/fileutils/ChangeLog' 2024-03-09T09:27:59.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/old/fileutils/NEWS' 2024-03-09T09:27:59.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/old/fileutils/ChangeLog-1997' 2024-03-09T09:27:59.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/old/fileutils' 2024-03-09T09:27:59.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/old' 2024-03-09T09:27:59.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_sun_os_names' 2024-03-09T09:27:59.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/THANKS-to-translators' 2024-03-09T09:27:59.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/gitlog-to-changelog' 2024-03-09T09:27:59.095 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/config.rpath' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/depcomp' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/gendocs.sh' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/compile' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/useless-if-before-free' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/do-release-commit-and-tag' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/vc-list-files' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/texinfo.tex' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/config.guess' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/update-copyright' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/announce-gen' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/ChangeLog-2007' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/cvsu' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/config.sub' 2024-03-09T09:27:59.096 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/install-sh' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/ylwrap' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/git-version-gen' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/arg-nonnull.h' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/mdate-sh' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/gnupload' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/unused-parameter.h' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/warn-on-use.h' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/missing' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/c++defs.h' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/build-aux/gnu-web-doc-update' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5/build-aux' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/configure' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.vg-suppressions' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_obsolete_symbols' 2024-03-09T09:27:59.097 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/.x-sc_prohibit_atoi_atof' 2024-03-09T09:27:59.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/COPYING' 2024-03-09T09:27:59.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed 'coreutils-8.5/cfg.mk' 2024-03-09T09:27:59.098 INFO:tasks.workunit.client.1.smithi134.stdout:removed directory 'coreutils-8.5' 2024-03-09T09:27:59.099 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xprintf.h' -> './k/coreutils-8.5/lib/xprintf.h' 2024-03-09T09:27:59.099 INFO:tasks.workunit.client.1.smithi134.stderr:+ cp -av .snap/k . 2024-03-09T09:27:59.100 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xprintf.h’: No data available 2024-03-09T09:27:59.100 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/exclude.c' -> './k/coreutils-8.5/lib/exclude.c' 2024-03-09T09:27:59.100 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/exclude.c’: No data available 2024-03-09T09:27:59.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:59.089+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:59.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:59.090+0000 7f912be00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 msgr2=0x7f912c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:27:59.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:59.090+0000 7f912be00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 msgr2=0x7f912c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:27:59.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:59.090+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:59.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:59.090+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:27:59.101 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/userspec.h' -> './k/coreutils-8.5/lib/userspec.h' 2024-03-09T09:27:59.102 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/userspec.h’: No data available 2024-03-09T09:27:59.102 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mkancesdirs.h' -> './k/coreutils-8.5/lib/mkancesdirs.h' 2024-03-09T09:27:59.102 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkancesdirs.h’: No data available 2024-03-09T09:27:59.103 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/hash-pjw.c' -> './k/coreutils-8.5/lib/hash-pjw.c' 2024-03-09T09:27:59.103 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash-pjw.c’: No data available 2024-03-09T09:27:59.103 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fd-safer.c' -> './k/coreutils-8.5/lib/fd-safer.c' 2024-03-09T09:27:59.103 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fd-safer.c’: No data available 2024-03-09T09:27:59.105 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/long-options.h' -> './k/coreutils-8.5/lib/long-options.h' 2024-03-09T09:27:59.105 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/long-options.h’: No data available 2024-03-09T09:27:59.106 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/freadahead.h' -> './k/coreutils-8.5/lib/freadahead.h' 2024-03-09T09:27:59.107 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadahead.h’: No data available 2024-03-09T09:27:59.108 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/gethostname.c' -> './k/coreutils-8.5/lib/gethostname.c' 2024-03-09T09:27:59.108 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gethostname.c’: No data available 2024-03-09T09:27:59.109 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/putenv.c' -> './k/coreutils-8.5/lib/putenv.c' 2024-03-09T09:27:59.109 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/putenv.c’: No data available 2024-03-09T09:27:59.111 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbrlen.c' -> './k/coreutils-8.5/lib/mbrlen.c' 2024-03-09T09:27:59.111 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbrlen.c’: No data available 2024-03-09T09:27:59.113 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/group-member.c' -> './k/coreutils-8.5/lib/group-member.c' 2024-03-09T09:27:59.114 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/group-member.c’: No data available 2024-03-09T09:27:59.115 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sha512.c' -> './k/coreutils-8.5/lib/sha512.c' 2024-03-09T09:27:59.116 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha512.c’: No data available 2024-03-09T09:27:59.117 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/signal.in.h' -> './k/coreutils-8.5/lib/signal.in.h' 2024-03-09T09:27:59.117 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/signal.in.h’: No data available 2024-03-09T09:27:59.119 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mktime.c' -> './k/coreutils-8.5/lib/mktime.c' 2024-03-09T09:27:59.119 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mktime.c’: No data available 2024-03-09T09:27:59.120 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/readlink.c' -> './k/coreutils-8.5/lib/readlink.c' 2024-03-09T09:27:59.120 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readlink.c’: No data available 2024-03-09T09:27:59.121 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mkdir-p.c' -> './k/coreutils-8.5/lib/mkdir-p.c' 2024-03-09T09:27:59.122 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkdir-p.c’: No data available 2024-03-09T09:27:59.123 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getndelim2.h' -> './k/coreutils-8.5/lib/getndelim2.h' 2024-03-09T09:27:59.123 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getndelim2.h’: No data available 2024-03-09T09:27:59.125 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memxfrm.h' -> './k/coreutils-8.5/lib/memxfrm.h' 2024-03-09T09:27:59.125 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memxfrm.h’: No data available 2024-03-09T09:27:59.126 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/c-strtod.c' -> './k/coreutils-8.5/lib/c-strtod.c' 2024-03-09T09:27:59.126 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strtod.c’: No data available 2024-03-09T09:27:59.127 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fseterr.h' -> './k/coreutils-8.5/lib/fseterr.h' 2024-03-09T09:27:59.128 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fseterr.h’: No data available 2024-03-09T09:27:59.129 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-hpux.gperf' -> './k/coreutils-8.5/lib/iconv_open-hpux.gperf' 2024-03-09T09:27:59.129 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-hpux.gperf’: No data available 2024-03-09T09:27:59.131 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/filenamecat.c' -> './k/coreutils-8.5/lib/filenamecat.c' 2024-03-09T09:27:59.131 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filenamecat.c’: No data available 2024-03-09T09:27:59.132 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memmove.c' -> './k/coreutils-8.5/lib/memmove.c' 2024-03-09T09:27:59.132 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memmove.c’: No data available 2024-03-09T09:27:59.134 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/str-kmp.h' -> './k/coreutils-8.5/lib/str-kmp.h' 2024-03-09T09:27:59.134 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/str-kmp.h’: No data available 2024-03-09T09:27:59.136 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/base64.c' -> './k/coreutils-8.5/lib/base64.c' 2024-03-09T09:27:59.136 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/base64.c’: No data available 2024-03-09T09:27:59.137 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/close-hook.c' -> './k/coreutils-8.5/lib/close-hook.c' 2024-03-09T09:27:59.137 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/close-hook.c’: No data available 2024-03-09T09:27:59.139 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/dirchownmod.c' -> './k/coreutils-8.5/lib/dirchownmod.c' 2024-03-09T09:27:59.139 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirchownmod.c’: No data available 2024-03-09T09:27:59.141 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/openat-safer.c' -> './k/coreutils-8.5/lib/openat-safer.c' 2024-03-09T09:27:59.141 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat-safer.c’: No data available 2024-03-09T09:27:59.144 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/posixtm.c' -> './k/coreutils-8.5/lib/posixtm.c' 2024-03-09T09:27:59.144 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/posixtm.c’: No data available 2024-03-09T09:27:59.146 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/printf-frexp.c' -> './k/coreutils-8.5/lib/printf-frexp.c' 2024-03-09T09:27:59.146 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-frexp.c’: No data available 2024-03-09T09:27:59.147 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/safe-read.h' -> './k/coreutils-8.5/lib/safe-read.h' 2024-03-09T09:27:59.148 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/safe-read.h’: No data available 2024-03-09T09:27:59.150 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/regexec.c' -> './k/coreutils-8.5/lib/regexec.c' 2024-03-09T09:27:59.151 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regexec.c’: No data available 2024-03-09T09:27:59.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/uniwidth.h' -> './k/coreutils-8.5/lib/uniwidth.h' 2024-03-09T09:27:59.154 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/uniwidth.h’: No data available 2024-03-09T09:27:59.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/write-any-file.c' -> './k/coreutils-8.5/lib/write-any-file.c' 2024-03-09T09:27:59.156 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/write-any-file.c’: No data available 2024-03-09T09:27:59.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-aix.h' -> './k/coreutils-8.5/lib/iconv_open-aix.h' 2024-03-09T09:27:59.158 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-aix.h’: No data available 2024-03-09T09:27:59.160 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unicodeio.c' -> './k/coreutils-8.5/lib/unicodeio.c' 2024-03-09T09:27:59.160 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unicodeio.c’: No data available 2024-03-09T09:27:59.162 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/gettime.c' -> './k/coreutils-8.5/lib/gettime.c' 2024-03-09T09:27:59.162 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gettime.c’: No data available 2024-03-09T09:27:59.164 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strnumcmp.h' -> './k/coreutils-8.5/lib/strnumcmp.h' 2024-03-09T09:27:59.164 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnumcmp.h’: No data available 2024-03-09T09:27:59.166 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/btowc.c' -> './k/coreutils-8.5/lib/btowc.c' 2024-03-09T09:27:59.166 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/btowc.c’: No data available 2024-03-09T09:27:59.168 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sys_time.in.h' -> './k/coreutils-8.5/lib/sys_time.in.h' 2024-03-09T09:27:59.168 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sys_time.in.h’: No data available 2024-03-09T09:27:59.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k' -> './k' 2024-03-09T09:27:59.169 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils_8.5.orig.tar.gz' -> './k/coreutils_8.5.orig.tar.gz' 2024-03-09T09:27:59.169 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils_8.5.orig.tar.gz’: No data available 2024-03-09T09:27:59.170 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/isnan.c' -> './k/coreutils-8.5/lib/isnan.c' 2024-03-09T09:27:59.170 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnan.c’: No data available 2024-03-09T09:27:59.174 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getaddrinfo.c' -> './k/coreutils-8.5/lib/getaddrinfo.c' 2024-03-09T09:27:59.175 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getaddrinfo.c’: No data available 2024-03-09T09:27:59.175 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getpagesize.c' -> './k/coreutils-8.5/lib/getpagesize.c' 2024-03-09T09:27:59.175 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getpagesize.c’: No data available 2024-03-09T09:27:59.176 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/posixver.h' -> './k/coreutils-8.5/lib/posixver.h' 2024-03-09T09:27:59.176 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/posixver.h’: No data available 2024-03-09T09:27:59.177 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/openat.c' -> './k/coreutils-8.5/lib/openat.c' 2024-03-09T09:27:59.177 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat.c’: No data available 2024-03-09T09:27:59.179 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/closeout.c' -> './k/coreutils-8.5/lib/closeout.c' 2024-03-09T09:27:59.179 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/closeout.c’: No data available 2024-03-09T09:27:59.181 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fopen.c' -> './k/coreutils-8.5/lib/fopen.c' 2024-03-09T09:27:59.181 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fopen.c’: No data available 2024-03-09T09:27:59.182 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memchr2.valgrind' -> './k/coreutils-8.5/lib/memchr2.valgrind' 2024-03-09T09:27:59.183 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memchr2.valgrind’: No data available 2024-03-09T09:27:59.184 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/time.in.h' -> './k/coreutils-8.5/lib/time.in.h' 2024-03-09T09:27:59.184 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/time.in.h’: No data available 2024-03-09T09:27:59.186 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/physmem.h' -> './k/coreutils-8.5/lib/physmem.h' 2024-03-09T09:27:59.186 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/physmem.h’: No data available 2024-03-09T09:27:59.187 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-irix.gperf' -> './k/coreutils-8.5/lib/iconv_open-irix.gperf' 2024-03-09T09:27:59.187 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-irix.gperf’: No data available 2024-03-09T09:27:59.190 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5' -> './k/coreutils-8.5' 2024-03-09T09:27:59.190 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/THANKS' -> './k/coreutils-8.5/THANKS' 2024-03-09T09:27:59.190 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/THANKS’: No data available 2024-03-09T09:27:59.192 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/ABOUT-NLS' -> './k/coreutils-8.5/ABOUT-NLS' 2024-03-09T09:27:59.193 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ABOUT-NLS’: No data available 2024-03-09T09:27:59.194 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_strcmp' -> './k/coreutils-8.5/.x-sc_prohibit_strcmp' 2024-03-09T09:27:59.194 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_strcmp’: No data available 2024-03-09T09:27:59.195 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/glthread' -> './k/coreutils-8.5/lib/glthread' 2024-03-09T09:27:59.195 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/glthread/threadlib.c' -> './k/coreutils-8.5/lib/glthread/threadlib.c' 2024-03-09T09:27:59.195 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/glthread/threadlib.c’: No data available 2024-03-09T09:27:59.196 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_error_message_uppercase' -> './k/coreutils-8.5/.x-sc_error_message_uppercase' 2024-03-09T09:27:59.196 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_error_message_uppercase’: No data available 2024-03-09T09:27:59.196 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/glthread/lock.c' -> './k/coreutils-8.5/lib/glthread/lock.c' 2024-03-09T09:27:59.196 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/glthread/lock.c’: No data available 2024-03-09T09:27:59.197 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_obsolete_symbols' -> './k/coreutils-8.5/.x-sc_obsolete_symbols' 2024-03-09T09:27:59.197 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_obsolete_symbols’: No data available 2024-03-09T09:27:59.198 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/glthread/tls.h' -> './k/coreutils-8.5/lib/glthread/tls.h' 2024-03-09T09:27:59.198 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/glthread/tls.h’: No data available 2024-03-09T09:27:59.198 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/INSTALL' -> './k/coreutils-8.5/INSTALL' 2024-03-09T09:27:59.198 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/INSTALL’: No data available 2024-03-09T09:27:59.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/glthread/tls.c' -> './k/coreutils-8.5/lib/glthread/tls.c' 2024-03-09T09:27:59.199 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/glthread/tls.c’: No data available 2024-03-09T09:27:59.200 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/configure.ac' -> './k/coreutils-8.5/configure.ac' 2024-03-09T09:27:59.200 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/configure.ac’: No data available 2024-03-09T09:27:59.201 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_tab_based_indentation' -> './k/coreutils-8.5/.x-sc_prohibit_tab_based_indentation' 2024-03-09T09:27:59.201 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_tab_based_indentation’: No data available 2024-03-09T09:27:59.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/glthread/lock.h' -> './k/coreutils-8.5/lib/glthread/lock.h' 2024-03-09T09:27:59.202 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/glthread/lock.h’: No data available 2024-03-09T09:27:59.202 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/GNUmakefile' -> './k/coreutils-8.5/GNUmakefile' 2024-03-09T09:27:59.203 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/GNUmakefile’: No data available 2024-03-09T09:27:59.206 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/ChangeLog' -> './k/coreutils-8.5/ChangeLog' 2024-03-09T09:27:59.207 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ChangeLog’: No data available 2024-03-09T09:27:59.208 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/COPYING' -> './k/coreutils-8.5/COPYING' 2024-03-09T09:27:59.208 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/COPYING’: No data available 2024-03-09T09:27:59.209 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/Makefile.am' -> './k/coreutils-8.5/Makefile.am' 2024-03-09T09:27:59.209 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/Makefile.am’: No data available 2024-03-09T09:27:59.211 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_po_check' -> './k/coreutils-8.5/.x-sc_po_check' 2024-03-09T09:27:59.211 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_po_check’: No data available 2024-03-09T09:27:59.212 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/dist-check.mk' -> './k/coreutils-8.5/dist-check.mk' 2024-03-09T09:27:59.212 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/dist-check.mk’: No data available 2024-03-09T09:27:59.214 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_GPL_version' -> './k/coreutils-8.5/.x-sc_GPL_version' 2024-03-09T09:27:59.214 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_GPL_version’: No data available 2024-03-09T09:27:59.215 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_unmarked_diagnostics' -> './k/coreutils-8.5/.x-sc_unmarked_diagnostics' 2024-03-09T09:27:59.215 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_unmarked_diagnostics’: No data available 2024-03-09T09:27:59.216 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.prev-version' -> './k/coreutils-8.5/.prev-version' 2024-03-09T09:27:59.216 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.prev-version’: No data available 2024-03-09T09:27:59.217 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/bootstrap.conf' -> './k/coreutils-8.5/bootstrap.conf' 2024-03-09T09:27:59.217 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/bootstrap.conf’: No data available 2024-03-09T09:27:59.221 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/ChangeLog-2005' -> './k/coreutils-8.5/ChangeLog-2005' 2024-03-09T09:27:59.221 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ChangeLog-2005’: No data available 2024-03-09T09:27:59.223 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/aclocal.m4' -> './k/coreutils-8.5/aclocal.m4' 2024-03-09T09:27:59.223 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/aclocal.m4’: No data available 2024-03-09T09:27:59.224 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/AUTHORS' -> './k/coreutils-8.5/AUTHORS' 2024-03-09T09:27:59.224 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/AUTHORS’: No data available 2024-03-09T09:27:59.226 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/cfg.mk' -> './k/coreutils-8.5/cfg.mk' 2024-03-09T09:27:59.226 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/cfg.mk’: No data available 2024-03-09T09:27:59.227 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_file_system' -> './k/coreutils-8.5/.x-sc_file_system' 2024-03-09T09:27:59.227 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_file_system’: No data available 2024-03-09T09:27:59.228 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.version' -> './k/coreutils-8.5/.version' 2024-03-09T09:27:59.228 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.version’: No data available 2024-03-09T09:27:59.230 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/TODO' -> './k/coreutils-8.5/TODO' 2024-03-09T09:27:59.230 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/TODO’: No data available 2024-03-09T09:27:59.231 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_space_tab' -> './k/coreutils-8.5/.x-sc_space_tab' 2024-03-09T09:27:59.232 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_space_tab’: No data available 2024-03-09T09:27:59.232 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_empty_lines_at_EOF' -> './k/coreutils-8.5/.x-sc_prohibit_empty_lines_at_EOF' 2024-03-09T09:27:59.233 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_empty_lines_at_EOF’: No data available 2024-03-09T09:27:59.239 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/Makefile.in' -> './k/coreutils-8.5/Makefile.in' 2024-03-09T09:27:59.239 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/Makefile.in’: No data available 2024-03-09T09:27:59.241 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/maint.mk' -> './k/coreutils-8.5/maint.mk' 2024-03-09T09:27:59.241 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/maint.mk’: No data available 2024-03-09T09:27:59.284 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old' -> './k/coreutils-8.5/old' 2024-03-09T09:27:59.284 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old/textutils' -> './k/coreutils-8.5/old/textutils' 2024-03-09T09:27:59.284 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old/textutils/NEWS' -> './k/coreutils-8.5/old/textutils/NEWS' 2024-03-09T09:27:59.284 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/textutils/NEWS’: No data available 2024-03-09T09:27:59.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old/textutils/ChangeLog' -> './k/coreutils-8.5/old/textutils/ChangeLog' 2024-03-09T09:27:59.289 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/textutils/ChangeLog’: No data available 2024-03-09T09:27:59.339 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old/fileutils' -> './k/coreutils-8.5/old/fileutils' 2024-03-09T09:27:59.339 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old/fileutils/ChangeLog-1997' -> './k/coreutils-8.5/old/fileutils/ChangeLog-1997' 2024-03-09T09:27:59.340 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/fileutils/ChangeLog-1997’: No data available 2024-03-09T09:27:59.341 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old/fileutils/NEWS' -> './k/coreutils-8.5/old/fileutils/NEWS' 2024-03-09T09:27:59.341 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/fileutils/NEWS’: No data available 2024-03-09T09:27:59.344 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old/fileutils/ChangeLog' -> './k/coreutils-8.5/old/fileutils/ChangeLog' 2024-03-09T09:27:59.344 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/fileutils/ChangeLog’: No data available 2024-03-09T09:27:59.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:59.492+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:27:59.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:59.492+0000 7f912be00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 msgr2=0x7f912c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:27:59.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:59.492+0000 7f912be00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 msgr2=0x7f912c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:27:59.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:59.492+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:27:59.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:27:59.492+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:27:59.644 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old/sh-utils' -> './k/coreutils-8.5/old/sh-utils' 2024-03-09T09:27:59.644 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old/sh-utils/NEWS' -> './k/coreutils-8.5/old/sh-utils/NEWS' 2024-03-09T09:27:59.644 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/sh-utils/NEWS’: No data available 2024-03-09T09:27:59.647 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old/sh-utils/ChangeLog.0' -> './k/coreutils-8.5/old/sh-utils/ChangeLog.0' 2024-03-09T09:27:59.647 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/sh-utils/ChangeLog.0’: No data available 2024-03-09T09:27:59.651 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/old/sh-utils/ChangeLog' -> './k/coreutils-8.5/old/sh-utils/ChangeLog' 2024-03-09T09:27:59.651 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/old/sh-utils/ChangeLog’: No data available 2024-03-09T09:27:59.919 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unistr' -> './k/coreutils-8.5/lib/unistr' 2024-03-09T09:27:59.919 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unistr/u8-uctomb-aux.c' -> './k/coreutils-8.5/lib/unistr/u8-uctomb-aux.c' 2024-03-09T09:27:59.920 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistr/u8-uctomb-aux.c’: No data available 2024-03-09T09:27:59.922 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/unistr/u8-uctomb.c' -> './k/coreutils-8.5/lib/unistr/u8-uctomb.c' 2024-03-09T09:27:59.922 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistr/u8-uctomb.c’: No data available 2024-03-09T09:27:59.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/uniwidth' -> './k/coreutils-8.5/lib/uniwidth' 2024-03-09T09:27:59.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/uniwidth/width.c' -> './k/coreutils-8.5/lib/uniwidth/width.c' 2024-03-09T09:27:59.952 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux' -> './k/coreutils-8.5/build-aux' 2024-03-09T09:27:59.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/cvsu' -> './k/coreutils-8.5/build-aux/cvsu' 2024-03-09T09:27:59.953 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/uniwidth/width.c’: No data available 2024-03-09T09:27:59.953 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/cvsu’: No data available 2024-03-09T09:27:59.954 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/uniwidth/cjk.h' -> './k/coreutils-8.5/lib/uniwidth/cjk.h' 2024-03-09T09:27:59.955 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/uniwidth/cjk.h’: No data available 2024-03-09T09:27:59.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/mdate-sh' -> './k/coreutils-8.5/build-aux/mdate-sh' 2024-03-09T09:27:59.955 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/mdate-sh’: No data available 2024-03-09T09:27:59.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/gitlog-to-changelog' -> './k/coreutils-8.5/build-aux/gitlog-to-changelog' 2024-03-09T09:27:59.958 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/gitlog-to-changelog’: No data available 2024-03-09T09:27:59.960 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/gnu-web-doc-update' -> './k/coreutils-8.5/build-aux/gnu-web-doc-update' 2024-03-09T09:27:59.960 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/gnu-web-doc-update’: No data available 2024-03-09T09:27:59.963 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/useless-if-before-free' -> './k/coreutils-8.5/build-aux/useless-if-before-free' 2024-03-09T09:27:59.963 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/useless-if-before-free’: No data available 2024-03-09T09:27:59.967 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/config.sub' -> './k/coreutils-8.5/build-aux/config.sub' 2024-03-09T09:27:59.967 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/config.sub’: No data available 2024-03-09T09:27:59.970 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/missing' -> './k/coreutils-8.5/build-aux/missing' 2024-03-09T09:27:59.970 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/missing’: No data available 2024-03-09T09:27:59.974 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/compile' -> './k/coreutils-8.5/build-aux/compile' 2024-03-09T09:27:59.974 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/compile’: No data available 2024-03-09T09:27:59.976 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/install-sh' -> './k/coreutils-8.5/build-aux/install-sh' 2024-03-09T09:27:59.976 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/install-sh’: No data available 2024-03-09T09:27:59.979 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/gendocs.sh' -> './k/coreutils-8.5/build-aux/gendocs.sh' 2024-03-09T09:27:59.979 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/gendocs.sh’: No data available 2024-03-09T09:27:59.981 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/update-copyright' -> './k/coreutils-8.5/build-aux/update-copyright' 2024-03-09T09:27:59.982 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/update-copyright’: No data available 2024-03-09T09:27:59.988 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/texinfo.tex' -> './k/coreutils-8.5/build-aux/texinfo.tex' 2024-03-09T09:27:59.989 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/texinfo.tex’: No data available 2024-03-09T09:27:59.991 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/unused-parameter.h' -> './k/coreutils-8.5/build-aux/unused-parameter.h' 2024-03-09T09:27:59.991 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/unused-parameter.h’: No data available 2024-03-09T09:27:59.991 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strtoull.c' -> './k/coreutils-8.5/lib/strtoull.c' 2024-03-09T09:27:59.991 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtoull.c’: No data available 2024-03-09T09:27:59.993 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/c++defs.h' -> './k/coreutils-8.5/build-aux/c++defs.h' 2024-03-09T09:27:59.993 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/c++defs.h’: No data available 2024-03-09T09:27:59.994 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/siglist.h' -> './k/coreutils-8.5/lib/siglist.h' 2024-03-09T09:27:59.995 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/siglist.h’: No data available 2024-03-09T09:27:59.996 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/config.guess' -> './k/coreutils-8.5/build-aux/config.guess' 2024-03-09T09:27:59.996 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/config.guess’: No data available 2024-03-09T09:27:59.996 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbswidth.h' -> './k/coreutils-8.5/lib/mbswidth.h' 2024-03-09T09:27:59.997 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbswidth.h’: No data available 2024-03-09T09:27:59.998 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/do-release-commit-and-tag' -> './k/coreutils-8.5/build-aux/do-release-commit-and-tag' 2024-03-09T09:27:59.998 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/do-release-commit-and-tag’: No data available 2024-03-09T09:27:59.999 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstrtod.h' -> './k/coreutils-8.5/lib/xstrtod.h' 2024-03-09T09:27:59.999 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtod.h’: No data available 2024-03-09T09:28:00.000 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/git-version-gen' -> './k/coreutils-8.5/build-aux/git-version-gen' 2024-03-09T09:28:00.000 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/git-version-gen’: No data available 2024-03-09T09:28:00.001 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/arg-nonnull.h' -> './k/coreutils-8.5/build-aux/arg-nonnull.h' 2024-03-09T09:28:00.002 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/md5.c' -> './k/coreutils-8.5/lib/md5.c' 2024-03-09T09:28:00.002 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/arg-nonnull.h’: No data available 2024-03-09T09:28:00.002 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/md5.c’: No data available 2024-03-09T09:28:00.003 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/gnupload' -> './k/coreutils-8.5/build-aux/gnupload' 2024-03-09T09:28:00.003 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/gnupload’: No data available 2024-03-09T09:28:00.004 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getloadavg.c' -> './k/coreutils-8.5/lib/getloadavg.c' 2024-03-09T09:28:00.004 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getloadavg.c’: No data available 2024-03-09T09:28:00.005 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/warn-on-use.h' -> './k/coreutils-8.5/build-aux/warn-on-use.h' 2024-03-09T09:28:00.005 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/warn-on-use.h’: No data available 2024-03-09T09:28:00.006 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/vc-list-files' -> './k/coreutils-8.5/build-aux/vc-list-files' 2024-03-09T09:28:00.006 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/vc-list-files’: No data available 2024-03-09T09:28:00.006 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/getdate.h' -> './k/coreutils-8.5/lib/getdate.h' 2024-03-09T09:28:00.007 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getdate.h’: No data available 2024-03-09T09:28:00.007 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/ylwrap' -> './k/coreutils-8.5/build-aux/ylwrap' 2024-03-09T09:28:00.007 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/ylwrap’: No data available 2024-03-09T09:28:00.008 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stdlib-safer.h' -> './k/coreutils-8.5/lib/stdlib-safer.h' 2024-03-09T09:28:00.009 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdlib-safer.h’: No data available 2024-03-09T09:28:00.009 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/ChangeLog-2007' -> './k/coreutils-8.5/build-aux/ChangeLog-2007' 2024-03-09T09:28:00.009 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/ChangeLog-2007’: No data available 2024-03-09T09:28:00.010 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/announce-gen' -> './k/coreutils-8.5/build-aux/announce-gen' 2024-03-09T09:28:00.010 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/announce-gen’: No data available 2024-03-09T09:28:00.011 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/remove.c' -> './k/coreutils-8.5/lib/remove.c' 2024-03-09T09:28:00.012 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/remove.c’: No data available 2024-03-09T09:28:00.012 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/depcomp' -> './k/coreutils-8.5/build-aux/depcomp' 2024-03-09T09:28:00.012 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/depcomp’: No data available 2024-03-09T09:28:00.014 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/build-aux/config.rpath' -> './k/coreutils-8.5/build-aux/config.rpath' 2024-03-09T09:28:00.014 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/build-aux/config.rpath’: No data available 2024-03-09T09:28:00.015 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/malloc.c' -> './k/coreutils-8.5/lib/malloc.c' 2024-03-09T09:28:00.015 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/malloc.c’: No data available 2024-03-09T09:28:00.016 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/progname.h' -> './k/coreutils-8.5/lib/progname.h' 2024-03-09T09:28:00.017 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/progname.h’: No data available 2024-03-09T09:28:00.019 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/argmatch.h' -> './k/coreutils-8.5/lib/argmatch.h' 2024-03-09T09:28:00.019 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/argmatch.h’: No data available 2024-03-09T09:28:00.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/ChangeLog-2007' -> './k/coreutils-8.5/lib/ChangeLog-2007' 2024-03-09T09:28:00.025 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ChangeLog-2007’: No data available 2024-03-09T09:28:00.028 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mbswidth.c' -> './k/coreutils-8.5/lib/mbswidth.c' 2024-03-09T09:28:00.029 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbswidth.c’: No data available 2024-03-09T09:28:00.030 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/error.h' -> './k/coreutils-8.5/lib/error.h' 2024-03-09T09:28:00.031 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/error.h’: No data available 2024-03-09T09:28:00.033 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/obstack.c' -> './k/coreutils-8.5/lib/obstack.c' 2024-03-09T09:28:00.033 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/obstack.c’: No data available 2024-03-09T09:28:00.035 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/acl.h' -> './k/coreutils-8.5/lib/acl.h' 2024-03-09T09:28:00.035 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/acl.h’: No data available 2024-03-09T09:28:00.037 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strnumcmp-in.h' -> './k/coreutils-8.5/lib/strnumcmp-in.h' 2024-03-09T09:28:00.037 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnumcmp-in.h’: No data available 2024-03-09T09:28:00.039 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xstrtoul.c' -> './k/coreutils-8.5/lib/xstrtoul.c' 2024-03-09T09:28:00.039 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtoul.c’: No data available 2024-03-09T09:28:00.041 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/float+.h' -> './k/coreutils-8.5/lib/float+.h' 2024-03-09T09:28:00.041 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/float+.h’: No data available 2024-03-09T09:28:00.043 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/canonicalize.c' -> './k/coreutils-8.5/lib/canonicalize.c' 2024-03-09T09:28:00.044 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/canonicalize.c’: No data available 2024-03-09T09:28:00.045 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fcntl--.h' -> './k/coreutils-8.5/lib/fcntl--.h' 2024-03-09T09:28:00.045 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fcntl--.h’: No data available 2024-03-09T09:28:00.047 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/wchar.in.h' -> './k/coreutils-8.5/lib/wchar.in.h' 2024-03-09T09:28:00.047 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/wchar.in.h’: No data available 2024-03-09T09:28:00.050 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/trim.c' -> './k/coreutils-8.5/lib/trim.c' 2024-03-09T09:28:00.050 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/trim.c’: No data available 2024-03-09T09:28:00.052 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/rpmatch.c' -> './k/coreutils-8.5/lib/rpmatch.c' 2024-03-09T09:28:00.052 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/rpmatch.c’: No data available 2024-03-09T09:28:00.054 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/isapipe.h' -> './k/coreutils-8.5/lib/isapipe.h' 2024-03-09T09:28:00.054 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isapipe.h’: No data available 2024-03-09T09:28:00.056 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/memchr.c' -> './k/coreutils-8.5/lib/memchr.c' 2024-03-09T09:28:00.056 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memchr.c’: No data available 2024-03-09T09:28:00.058 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/localcharset.h' -> './k/coreutils-8.5/lib/localcharset.h' 2024-03-09T09:28:00.059 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/localcharset.h’: No data available 2024-03-09T09:28:00.060 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/sig-handler.h' -> './k/coreutils-8.5/lib/sig-handler.h' 2024-03-09T09:28:00.060 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sig-handler.h’: No data available 2024-03-09T09:28:00.062 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/openat-proc.c' -> './k/coreutils-8.5/lib/openat-proc.c' 2024-03-09T09:28:00.062 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat-proc.c’: No data available 2024-03-09T09:28:00.064 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/pathmax.h' -> './k/coreutils-8.5/lib/pathmax.h' 2024-03-09T09:28:00.064 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/pathmax.h’: No data available 2024-03-09T09:28:00.066 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/close-stream.c' -> './k/coreutils-8.5/lib/close-stream.c' 2024-03-09T09:28:00.066 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/close-stream.c’: No data available 2024-03-09T09:28:00.068 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/readutmp.c' -> './k/coreutils-8.5/lib/readutmp.c' 2024-03-09T09:28:00.068 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readutmp.c’: No data available 2024-03-09T09:28:00.070 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/float.in.h' -> './k/coreutils-8.5/lib/float.in.h' 2024-03-09T09:28:00.070 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/float.in.h’: No data available 2024-03-09T09:28:00.071 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/creat-safer.c' -> './k/coreutils-8.5/lib/creat-safer.c' 2024-03-09T09:28:00.071 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/creat-safer.c’: No data available 2024-03-09T09:28:00.073 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/randperm.c' -> './k/coreutils-8.5/lib/randperm.c' 2024-03-09T09:28:00.073 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randperm.c’: No data available 2024-03-09T09:28:00.074 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/readtokens.c' -> './k/coreutils-8.5/lib/readtokens.c' 2024-03-09T09:28:00.075 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readtokens.c’: No data available 2024-03-09T09:28:00.076 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/safe-read.c' -> './k/coreutils-8.5/lib/safe-read.c' 2024-03-09T09:28:00.076 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/safe-read.c’: No data available 2024-03-09T09:28:00.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xreadlink.h' -> './k/coreutils-8.5/lib/xreadlink.h' 2024-03-09T09:28:00.078 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xreadlink.h’: No data available 2024-03-09T09:28:00.079 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stdarg.in.h' -> './k/coreutils-8.5/lib/stdarg.in.h' 2024-03-09T09:28:00.079 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdarg.in.h’: No data available 2024-03-09T09:28:00.089 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xalloc-die.c' -> './k/coreutils-8.5/lib/xalloc-die.c' 2024-03-09T09:28:00.089 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xalloc-die.c’: No data available 2024-03-09T09:28:00.091 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/copy-acl.c' -> './k/coreutils-8.5/lib/copy-acl.c' 2024-03-09T09:28:00.091 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/copy-acl.c’: No data available 2024-03-09T09:28:00.092 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fsync.c' -> './k/coreutils-8.5/lib/fsync.c' 2024-03-09T09:28:00.092 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fsync.c’: No data available 2024-03-09T09:28:00.093 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/strnumcmp.c' -> './k/coreutils-8.5/lib/strnumcmp.c' 2024-03-09T09:28:00.093 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnumcmp.c’: No data available 2024-03-09T09:28:00.095 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/selinux-at.c' -> './k/coreutils-8.5/lib/selinux-at.c' 2024-03-09T09:28:00.095 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/selinux-at.c’: No data available 2024-03-09T09:28:00.096 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/realloc.c' -> './k/coreutils-8.5/lib/realloc.c' 2024-03-09T09:28:00.097 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/realloc.c’: No data available 2024-03-09T09:28:00.098 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gl' -> './k/coreutils-8.5/gl' 2024-03-09T09:28:00.098 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gl/modules' -> './k/coreutils-8.5/gl/modules' 2024-03-09T09:28:00.098 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gl/modules/getloadavg.diff' -> './k/coreutils-8.5/gl/modules/getloadavg.diff' 2024-03-09T09:28:00.098 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gl/modules/getloadavg.diff’: No data available 2024-03-09T09:28:00.099 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fdopendir.c' -> './k/coreutils-8.5/lib/fdopendir.c' 2024-03-09T09:28:00.099 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fdopendir.c’: No data available 2024-03-09T09:28:00.101 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/alloca.in.h' -> './k/coreutils-8.5/lib/alloca.in.h' 2024-03-09T09:28:00.101 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/alloca.in.h’: No data available 2024-03-09T09:28:00.103 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/freopen.c' -> './k/coreutils-8.5/lib/freopen.c' 2024-03-09T09:28:00.103 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freopen.c’: No data available 2024-03-09T09:28:00.105 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/mkdir-p.h' -> './k/coreutils-8.5/lib/mkdir-p.h' 2024-03-09T09:28:00.105 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkdir-p.h’: No data available 2024-03-09T09:28:00.106 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/isnand-nolibm.h' -> './k/coreutils-8.5/lib/isnand-nolibm.h' 2024-03-09T09:28:00.107 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnand-nolibm.h’: No data available 2024-03-09T09:28:00.108 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/propername.c' -> './k/coreutils-8.5/lib/propername.c' 2024-03-09T09:28:00.108 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/propername.c’: No data available 2024-03-09T09:28:00.110 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/fileblocks.c' -> './k/coreutils-8.5/lib/fileblocks.c' 2024-03-09T09:28:00.110 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fileblocks.c’: No data available 2024-03-09T09:28:00.111 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xmemcoll.h' -> './k/coreutils-8.5/lib/xmemcoll.h' 2024-03-09T09:28:00.111 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xmemcoll.h’: No data available 2024-03-09T09:28:00.112 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/hash.h' -> './k/coreutils-8.5/lib/hash.h' 2024-03-09T09:28:00.112 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash.h’: No data available 2024-03-09T09:28:00.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:00.294+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:00.294+0000 7f912be00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 msgr2=0x7f912c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-03-09T09:28:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:00.294+0000 7f912be00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 msgr2=0x7f912c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:28:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:00.294+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:00.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:00.294+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:28:00.409 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/c-strtold.c' -> './k/coreutils-8.5/lib/c-strtold.c' 2024-03-09T09:28:00.410 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strtold.c’: No data available 2024-03-09T09:28:00.411 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/c-ctype.h' -> './k/coreutils-8.5/lib/c-ctype.h' 2024-03-09T09:28:00.411 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-ctype.h’: No data available 2024-03-09T09:28:00.412 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/xmemxfrm.c' -> './k/coreutils-8.5/lib/xmemxfrm.c' 2024-03-09T09:28:00.412 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xmemxfrm.c’: No data available 2024-03-09T09:28:00.414 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/lib/stat.c' -> './k/coreutils-8.5/lib/stat.c' 2024-03-09T09:28:00.414 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stat.c’: No data available 2024-03-09T09:28:00.437 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4' -> './k/coreutils-8.5/m4' 2024-03-09T09:28:00.437 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/dirent-safer.m4' -> './k/coreutils-8.5/m4/dirent-safer.m4' 2024-03-09T09:28:00.437 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dirent-safer.m4’: No data available 2024-03-09T09:28:00.439 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/settime.m4' -> './k/coreutils-8.5/m4/settime.m4' 2024-03-09T09:28:00.439 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/settime.m4’: No data available 2024-03-09T09:28:00.442 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/readtokens.m4' -> './k/coreutils-8.5/m4/readtokens.m4' 2024-03-09T09:28:00.442 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/readtokens.m4’: No data available 2024-03-09T09:28:00.445 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/clock_time.m4' -> './k/coreutils-8.5/m4/clock_time.m4' 2024-03-09T09:28:00.445 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/clock_time.m4’: No data available 2024-03-09T09:28:00.447 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/config-h.m4' -> './k/coreutils-8.5/m4/config-h.m4' 2024-03-09T09:28:00.447 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/config-h.m4’: No data available 2024-03-09T09:28:00.452 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/euidaccess.m4' -> './k/coreutils-8.5/m4/euidaccess.m4' 2024-03-09T09:28:00.452 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/euidaccess.m4’: No data available 2024-03-09T09:28:00.454 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/printf-frexpl.m4' -> './k/coreutils-8.5/m4/printf-frexpl.m4' 2024-03-09T09:28:00.454 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/printf-frexpl.m4’: No data available 2024-03-09T09:28:00.456 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/posix-shell.m4' -> './k/coreutils-8.5/m4/posix-shell.m4' 2024-03-09T09:28:00.456 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/posix-shell.m4’: No data available 2024-03-09T09:28:00.458 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/servent.m4' -> './k/coreutils-8.5/m4/servent.m4' 2024-03-09T09:28:00.458 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/servent.m4’: No data available 2024-03-09T09:28:00.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/xstrtol.m4' -> './k/coreutils-8.5/m4/xstrtol.m4' 2024-03-09T09:28:00.460 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xstrtol.m4’: No data available 2024-03-09T09:28:00.461 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mkdir-p.m4' -> './k/coreutils-8.5/m4/mkdir-p.m4' 2024-03-09T09:28:00.461 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mkdir-p.m4’: No data available 2024-03-09T09:28:00.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/00gnulib.m4' -> './k/coreutils-8.5/m4/00gnulib.m4' 2024-03-09T09:28:00.463 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/00gnulib.m4’: No data available 2024-03-09T09:28:00.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mbsinit.m4' -> './k/coreutils-8.5/m4/mbsinit.m4' 2024-03-09T09:28:00.465 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbsinit.m4’: No data available 2024-03-09T09:28:00.466 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/dup2.m4' -> './k/coreutils-8.5/m4/dup2.m4' 2024-03-09T09:28:00.466 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dup2.m4’: No data available 2024-03-09T09:28:00.468 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/eealloc.m4' -> './k/coreutils-8.5/m4/eealloc.m4' 2024-03-09T09:28:00.469 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/eealloc.m4’: No data available 2024-03-09T09:28:00.470 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/savewd.m4' -> './k/coreutils-8.5/m4/savewd.m4' 2024-03-09T09:28:00.470 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/savewd.m4’: No data available 2024-03-09T09:28:00.472 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/memrchr.m4' -> './k/coreutils-8.5/m4/memrchr.m4' 2024-03-09T09:28:00.472 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memrchr.m4’: No data available 2024-03-09T09:28:00.473 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/d-ino.m4' -> './k/coreutils-8.5/m4/d-ino.m4' 2024-03-09T09:28:00.474 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/d-ino.m4’: No data available 2024-03-09T09:28:00.475 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/bison.m4' -> './k/coreutils-8.5/m4/bison.m4' 2024-03-09T09:28:00.475 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/bison.m4’: No data available 2024-03-09T09:28:00.478 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/isnanl.m4' -> './k/coreutils-8.5/m4/isnanl.m4' 2024-03-09T09:28:00.478 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/isnanl.m4’: No data available 2024-03-09T09:28:00.480 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/lib-ld.m4' -> './k/coreutils-8.5/m4/lib-ld.m4' 2024-03-09T09:28:00.480 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lib-ld.m4’: No data available 2024-03-09T09:28:00.481 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stpcpy.m4' -> './k/coreutils-8.5/m4/stpcpy.m4' 2024-03-09T09:28:00.482 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stpcpy.m4’: No data available 2024-03-09T09:28:00.483 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/progtest.m4' -> './k/coreutils-8.5/m4/progtest.m4' 2024-03-09T09:28:00.483 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/progtest.m4’: No data available 2024-03-09T09:28:00.485 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/unistd_h.m4' -> './k/coreutils-8.5/m4/unistd_h.m4' 2024-03-09T09:28:00.485 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unistd_h.m4’: No data available 2024-03-09T09:28:00.486 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/time_h.m4' -> './k/coreutils-8.5/m4/time_h.m4' 2024-03-09T09:28:00.487 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/time_h.m4’: No data available 2024-03-09T09:28:00.488 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mkancesdirs.m4' -> './k/coreutils-8.5/m4/mkancesdirs.m4' 2024-03-09T09:28:00.488 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mkancesdirs.m4’: No data available 2024-03-09T09:28:00.490 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/signalblocking.m4' -> './k/coreutils-8.5/m4/signalblocking.m4' 2024-03-09T09:28:00.490 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/signalblocking.m4’: No data available 2024-03-09T09:28:00.492 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/unlink.m4' -> './k/coreutils-8.5/m4/unlink.m4' 2024-03-09T09:28:00.492 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unlink.m4’: No data available 2024-03-09T09:28:00.494 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/backupfile.m4' -> './k/coreutils-8.5/m4/backupfile.m4' 2024-03-09T09:28:00.494 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/backupfile.m4’: No data available 2024-03-09T09:28:00.496 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/remove.m4' -> './k/coreutils-8.5/m4/remove.m4' 2024-03-09T09:28:00.496 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/remove.m4’: No data available 2024-03-09T09:28:00.497 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fts.m4' -> './k/coreutils-8.5/m4/fts.m4' 2024-03-09T09:28:00.497 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fts.m4’: No data available 2024-03-09T09:28:00.499 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/xalloc.m4' -> './k/coreutils-8.5/m4/xalloc.m4' 2024-03-09T09:28:00.499 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xalloc.m4’: No data available 2024-03-09T09:28:00.500 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/inet_ntop.m4' -> './k/coreutils-8.5/m4/inet_ntop.m4' 2024-03-09T09:28:00.500 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inet_ntop.m4’: No data available 2024-03-09T09:28:00.501 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/iconv_open.m4' -> './k/coreutils-8.5/m4/iconv_open.m4' 2024-03-09T09:28:00.502 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/iconv_open.m4’: No data available 2024-03-09T09:28:00.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/extensions.m4' -> './k/coreutils-8.5/m4/extensions.m4' 2024-03-09T09:28:00.503 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/extensions.m4’: No data available 2024-03-09T09:28:00.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mbrtowc.m4' -> './k/coreutils-8.5/m4/mbrtowc.m4' 2024-03-09T09:28:00.505 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbrtowc.m4’: No data available 2024-03-09T09:28:00.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/check-decl.m4' -> './k/coreutils-8.5/m4/check-decl.m4' 2024-03-09T09:28:00.507 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/check-decl.m4’: No data available 2024-03-09T09:28:00.508 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/cycle-check.m4' -> './k/coreutils-8.5/m4/cycle-check.m4' 2024-03-09T09:28:00.508 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/cycle-check.m4’: No data available 2024-03-09T09:28:00.510 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/signal_h.m4' -> './k/coreutils-8.5/m4/signal_h.m4' 2024-03-09T09:28:00.510 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/signal_h.m4’: No data available 2024-03-09T09:28:00.512 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sys_socket_h.m4' -> './k/coreutils-8.5/m4/sys_socket_h.m4' 2024-03-09T09:28:00.512 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_socket_h.m4’: No data available 2024-03-09T09:28:00.513 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fprintftime.m4' -> './k/coreutils-8.5/m4/fprintftime.m4' 2024-03-09T09:28:00.514 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fprintftime.m4’: No data available 2024-03-09T09:28:00.515 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/lib-check.m4' -> './k/coreutils-8.5/m4/lib-check.m4' 2024-03-09T09:28:00.515 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lib-check.m4’: No data available 2024-03-09T09:28:00.517 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/isblank.m4' -> './k/coreutils-8.5/m4/isblank.m4' 2024-03-09T09:28:00.517 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/isblank.m4’: No data available 2024-03-09T09:28:00.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/xstrtod.m4' -> './k/coreutils-8.5/m4/xstrtod.m4' 2024-03-09T09:28:00.518 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xstrtod.m4’: No data available 2024-03-09T09:28:00.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stdarg.m4' -> './k/coreutils-8.5/m4/stdarg.m4' 2024-03-09T09:28:00.519 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdarg.m4’: No data available 2024-03-09T09:28:00.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/selinux-selinux-h.m4' -> './k/coreutils-8.5/m4/selinux-selinux-h.m4' 2024-03-09T09:28:00.520 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/selinux-selinux-h.m4’: No data available 2024-03-09T09:28:00.522 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fstypename.m4' -> './k/coreutils-8.5/m4/fstypename.m4' 2024-03-09T09:28:00.522 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fstypename.m4’: No data available 2024-03-09T09:28:00.523 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/link.m4' -> './k/coreutils-8.5/m4/link.m4' 2024-03-09T09:28:00.523 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/link.m4’: No data available 2024-03-09T09:28:00.524 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/md5.m4' -> './k/coreutils-8.5/m4/md5.m4' 2024-03-09T09:28:00.525 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/md5.m4’: No data available 2024-03-09T09:28:00.526 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/multiarch.m4' -> './k/coreutils-8.5/m4/multiarch.m4' 2024-03-09T09:28:00.527 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/multiarch.m4’: No data available 2024-03-09T09:28:00.528 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/ioctl.m4' -> './k/coreutils-8.5/m4/ioctl.m4' 2024-03-09T09:28:00.528 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ioctl.m4’: No data available 2024-03-09T09:28:00.529 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/flexmember.m4' -> './k/coreutils-8.5/m4/flexmember.m4' 2024-03-09T09:28:00.529 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/flexmember.m4’: No data available 2024-03-09T09:28:00.530 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/locale-fr.m4' -> './k/coreutils-8.5/m4/locale-fr.m4' 2024-03-09T09:28:00.531 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/locale-fr.m4’: No data available 2024-03-09T09:28:00.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fdopendir.m4' -> './k/coreutils-8.5/m4/fdopendir.m4' 2024-03-09T09:28:00.532 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fdopendir.m4’: No data available 2024-03-09T09:28:00.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/exponentf.m4' -> './k/coreutils-8.5/m4/exponentf.m4' 2024-03-09T09:28:00.533 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/exponentf.m4’: No data available 2024-03-09T09:28:00.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/utimecmp.m4' -> './k/coreutils-8.5/m4/utimecmp.m4' 2024-03-09T09:28:00.534 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/utimecmp.m4’: No data available 2024-03-09T09:28:00.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/human.m4' -> './k/coreutils-8.5/m4/human.m4' 2024-03-09T09:28:00.536 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/human.m4’: No data available 2024-03-09T09:28:00.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strerror.m4' -> './k/coreutils-8.5/m4/strerror.m4' 2024-03-09T09:28:00.537 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strerror.m4’: No data available 2024-03-09T09:28:00.537 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/socklen.m4' -> './k/coreutils-8.5/m4/socklen.m4' 2024-03-09T09:28:00.538 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/socklen.m4’: No data available 2024-03-09T09:28:00.658 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/dirent_h.m4' -> './k/coreutils-8.5/m4/dirent_h.m4' 2024-03-09T09:28:00.658 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dirent_h.m4’: No data available 2024-03-09T09:28:00.660 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/locale-zh.m4' -> './k/coreutils-8.5/m4/locale-zh.m4' 2024-03-09T09:28:00.660 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/locale-zh.m4’: No data available 2024-03-09T09:28:00.662 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getdate.m4' -> './k/coreutils-8.5/m4/getdate.m4' 2024-03-09T09:28:00.663 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getdate.m4’: No data available 2024-03-09T09:28:00.664 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/cloexec.m4' -> './k/coreutils-8.5/m4/cloexec.m4' 2024-03-09T09:28:00.664 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/cloexec.m4’: No data available 2024-03-09T09:28:00.666 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/i-ring.m4' -> './k/coreutils-8.5/m4/i-ring.m4' 2024-03-09T09:28:00.667 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/i-ring.m4’: No data available 2024-03-09T09:28:00.669 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/group-member.m4' -> './k/coreutils-8.5/m4/group-member.m4' 2024-03-09T09:28:00.669 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/group-member.m4’: No data available 2024-03-09T09:28:00.671 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strtod.m4' -> './k/coreutils-8.5/m4/strtod.m4' 2024-03-09T09:28:00.671 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtod.m4’: No data available 2024-03-09T09:28:00.676 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/chdir-long.m4' -> './k/coreutils-8.5/m4/chdir-long.m4' 2024-03-09T09:28:00.676 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/chdir-long.m4’: No data available 2024-03-09T09:28:00.678 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strnlen.m4' -> './k/coreutils-8.5/m4/strnlen.m4' 2024-03-09T09:28:00.678 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strnlen.m4’: No data available 2024-03-09T09:28:00.680 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strtoimax.m4' -> './k/coreutils-8.5/m4/strtoimax.m4' 2024-03-09T09:28:00.680 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtoimax.m4’: No data available 2024-03-09T09:28:00.683 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getgroups.m4' -> './k/coreutils-8.5/m4/getgroups.m4' 2024-03-09T09:28:00.684 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getgroups.m4’: No data available 2024-03-09T09:28:00.685 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/yield.m4' -> './k/coreutils-8.5/m4/yield.m4' 2024-03-09T09:28:00.685 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/yield.m4’: No data available 2024-03-09T09:28:00.688 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/write.m4' -> './k/coreutils-8.5/m4/write.m4' 2024-03-09T09:28:00.688 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/write.m4’: No data available 2024-03-09T09:28:00.689 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/localcharset.m4' -> './k/coreutils-8.5/m4/localcharset.m4' 2024-03-09T09:28:00.690 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/localcharset.m4’: No data available 2024-03-09T09:28:00.691 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/usleep.m4' -> './k/coreutils-8.5/m4/usleep.m4' 2024-03-09T09:28:00.691 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/usleep.m4’: No data available 2024-03-09T09:28:00.692 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/memmove.m4' -> './k/coreutils-8.5/m4/memmove.m4' 2024-03-09T09:28:00.692 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memmove.m4’: No data available 2024-03-09T09:28:00.694 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/close.m4' -> './k/coreutils-8.5/m4/close.m4' 2024-03-09T09:28:00.695 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/close.m4’: No data available 2024-03-09T09:28:00.696 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/memcmp.m4' -> './k/coreutils-8.5/m4/memcmp.m4' 2024-03-09T09:28:00.696 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memcmp.m4’: No data available 2024-03-09T09:28:00.698 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/autobuild.m4' -> './k/coreutils-8.5/m4/autobuild.m4' 2024-03-09T09:28:00.698 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/autobuild.m4’: No data available 2024-03-09T09:28:00.700 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fchdir.m4' -> './k/coreutils-8.5/m4/fchdir.m4' 2024-03-09T09:28:00.700 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fchdir.m4’: No data available 2024-03-09T09:28:00.701 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fd-reopen.m4' -> './k/coreutils-8.5/m4/fd-reopen.m4' 2024-03-09T09:28:00.702 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fd-reopen.m4’: No data available 2024-03-09T09:28:00.703 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/jm-winsz2.m4' -> './k/coreutils-8.5/m4/jm-winsz2.m4' 2024-03-09T09:28:00.704 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/jm-winsz2.m4’: No data available 2024-03-09T09:28:00.705 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/posixver.m4' -> './k/coreutils-8.5/m4/posixver.m4' 2024-03-09T09:28:00.705 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/posixver.m4’: No data available 2024-03-09T09:28:00.707 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/locale-ja.m4' -> './k/coreutils-8.5/m4/locale-ja.m4' 2024-03-09T09:28:00.707 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/locale-ja.m4’: No data available 2024-03-09T09:28:00.709 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stat-prog.m4' -> './k/coreutils-8.5/m4/stat-prog.m4' 2024-03-09T09:28:00.710 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stat-prog.m4’: No data available 2024-03-09T09:28:00.712 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/wint_t.m4' -> './k/coreutils-8.5/m4/wint_t.m4' 2024-03-09T09:28:00.712 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wint_t.m4’: No data available 2024-03-09T09:28:00.714 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/st_dm_mode.m4' -> './k/coreutils-8.5/m4/st_dm_mode.m4' 2024-03-09T09:28:00.715 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/st_dm_mode.m4’: No data available 2024-03-09T09:28:00.717 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/tempname.m4' -> './k/coreutils-8.5/m4/tempname.m4' 2024-03-09T09:28:00.717 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/tempname.m4’: No data available 2024-03-09T09:28:00.719 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/frexp.m4' -> './k/coreutils-8.5/m4/frexp.m4' 2024-03-09T09:28:00.720 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/frexp.m4’: No data available 2024-03-09T09:28:00.723 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/dirfd.m4' -> './k/coreutils-8.5/m4/dirfd.m4' 2024-03-09T09:28:00.723 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dirfd.m4’: No data available 2024-03-09T09:28:00.725 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sys_select_h.m4' -> './k/coreutils-8.5/m4/sys_select_h.m4' 2024-03-09T09:28:00.725 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_select_h.m4’: No data available 2024-03-09T09:28:00.726 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/btowc.m4' -> './k/coreutils-8.5/m4/btowc.m4' 2024-03-09T09:28:00.726 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/btowc.m4’: No data available 2024-03-09T09:28:00.728 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/filenamecat.m4' -> './k/coreutils-8.5/m4/filenamecat.m4' 2024-03-09T09:28:00.728 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/filenamecat.m4’: No data available 2024-03-09T09:28:00.730 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/gmp.m4' -> './k/coreutils-8.5/m4/gmp.m4' 2024-03-09T09:28:00.730 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gmp.m4’: No data available 2024-03-09T09:28:00.733 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/isapipe.m4' -> './k/coreutils-8.5/m4/isapipe.m4' 2024-03-09T09:28:00.733 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/isapipe.m4’: No data available 2024-03-09T09:28:00.735 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/wctype_h.m4' -> './k/coreutils-8.5/m4/wctype_h.m4' 2024-03-09T09:28:00.736 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wctype_h.m4’: No data available 2024-03-09T09:28:00.737 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/ldexpl.m4' -> './k/coreutils-8.5/m4/ldexpl.m4' 2024-03-09T09:28:00.737 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ldexpl.m4’: No data available 2024-03-09T09:28:00.739 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getloadavg.m4' -> './k/coreutils-8.5/m4/getloadavg.m4' 2024-03-09T09:28:00.739 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getloadavg.m4’: No data available 2024-03-09T09:28:00.741 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mbchar.m4' -> './k/coreutils-8.5/m4/mbchar.m4' 2024-03-09T09:28:00.741 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbchar.m4’: No data available 2024-03-09T09:28:00.744 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/version-etc.m4' -> './k/coreutils-8.5/m4/version-etc.m4' 2024-03-09T09:28:00.744 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/version-etc.m4’: No data available 2024-03-09T09:28:00.746 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fcntl_h.m4' -> './k/coreutils-8.5/m4/fcntl_h.m4' 2024-03-09T09:28:00.746 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fcntl_h.m4’: No data available 2024-03-09T09:28:00.748 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/gettime.m4' -> './k/coreutils-8.5/m4/gettime.m4' 2024-03-09T09:28:00.749 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gettime.m4’: No data available 2024-03-09T09:28:00.750 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/wchar_h.m4' -> './k/coreutils-8.5/m4/wchar_h.m4' 2024-03-09T09:28:00.750 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wchar_h.m4’: No data available 2024-03-09T09:28:00.752 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/vasprintf-posix.m4' -> './k/coreutils-8.5/m4/vasprintf-posix.m4' 2024-03-09T09:28:00.753 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/vasprintf-posix.m4’: No data available 2024-03-09T09:28:00.755 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/selinux-context-h.m4' -> './k/coreutils-8.5/m4/selinux-context-h.m4' 2024-03-09T09:28:00.755 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/selinux-context-h.m4’: No data available 2024-03-09T09:28:00.756 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getpass.m4' -> './k/coreutils-8.5/m4/getpass.m4' 2024-03-09T09:28:00.757 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getpass.m4’: No data available 2024-03-09T09:28:00.759 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mempcpy.m4' -> './k/coreutils-8.5/m4/mempcpy.m4' 2024-03-09T09:28:00.759 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mempcpy.m4’: No data available 2024-03-09T09:28:00.761 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/save-cwd.m4' -> './k/coreutils-8.5/m4/save-cwd.m4' 2024-03-09T09:28:00.762 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/save-cwd.m4’: No data available 2024-03-09T09:28:00.768 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fclose.m4' -> './k/coreutils-8.5/m4/fclose.m4' 2024-03-09T09:28:00.768 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fclose.m4’: No data available 2024-03-09T09:28:00.770 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/modechange.m4' -> './k/coreutils-8.5/m4/modechange.m4' 2024-03-09T09:28:00.770 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/modechange.m4’: No data available 2024-03-09T09:28:00.772 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strstr.m4' -> './k/coreutils-8.5/m4/strstr.m4' 2024-03-09T09:28:00.773 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strstr.m4’: No data available 2024-03-09T09:28:00.775 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getdtablesize.m4' -> './k/coreutils-8.5/m4/getdtablesize.m4' 2024-03-09T09:28:00.775 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getdtablesize.m4’: No data available 2024-03-09T09:28:00.777 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/wchar_t.m4' -> './k/coreutils-8.5/m4/wchar_t.m4' 2024-03-09T09:28:00.777 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wchar_t.m4’: No data available 2024-03-09T09:28:00.779 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mkdir.m4' -> './k/coreutils-8.5/m4/mkdir.m4' 2024-03-09T09:28:00.779 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mkdir.m4’: No data available 2024-03-09T09:28:00.781 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/idcache.m4' -> './k/coreutils-8.5/m4/idcache.m4' 2024-03-09T09:28:00.781 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/idcache.m4’: No data available 2024-03-09T09:28:00.783 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/tm_gmtoff.m4' -> './k/coreutils-8.5/m4/tm_gmtoff.m4' 2024-03-09T09:28:00.783 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/tm_gmtoff.m4’: No data available 2024-03-09T09:28:00.785 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/dirname.m4' -> './k/coreutils-8.5/m4/dirname.m4' 2024-03-09T09:28:00.785 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dirname.m4’: No data available 2024-03-09T09:28:00.787 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mbiter.m4' -> './k/coreutils-8.5/m4/mbiter.m4' 2024-03-09T09:28:00.787 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbiter.m4’: No data available 2024-03-09T09:28:00.790 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fcntl-safer.m4' -> './k/coreutils-8.5/m4/fcntl-safer.m4' 2024-03-09T09:28:00.790 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fcntl-safer.m4’: No data available 2024-03-09T09:28:00.792 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/manywarnings.m4' -> './k/coreutils-8.5/m4/manywarnings.m4' 2024-03-09T09:28:00.793 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/manywarnings.m4’: No data available 2024-03-09T09:28:00.795 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/same.m4' -> './k/coreutils-8.5/m4/same.m4' 2024-03-09T09:28:00.795 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/same.m4’: No data available 2024-03-09T09:28:00.798 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getdelim.m4' -> './k/coreutils-8.5/m4/getdelim.m4' 2024-03-09T09:28:00.798 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getdelim.m4’: No data available 2024-03-09T09:28:00.801 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mountlist.m4' -> './k/coreutils-8.5/m4/mountlist.m4' 2024-03-09T09:28:00.801 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mountlist.m4’: No data available 2024-03-09T09:28:00.803 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/lchmod.m4' -> './k/coreutils-8.5/m4/lchmod.m4' 2024-03-09T09:28:00.806 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lchmod.m4’: No data available 2024-03-09T09:28:00.807 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/readlink.m4' -> './k/coreutils-8.5/m4/readlink.m4' 2024-03-09T09:28:00.807 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/readlink.m4’: No data available 2024-03-09T09:28:00.808 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/gnu-make.m4' -> './k/coreutils-8.5/m4/gnu-make.m4' 2024-03-09T09:28:00.808 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gnu-make.m4’: No data available 2024-03-09T09:28:00.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/thread.m4' -> './k/coreutils-8.5/m4/thread.m4' 2024-03-09T09:28:00.811 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/thread.m4’: No data available 2024-03-09T09:28:00.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/hostent.m4' -> './k/coreutils-8.5/m4/hostent.m4' 2024-03-09T09:28:00.813 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/hostent.m4’: No data available 2024-03-09T09:28:00.815 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/math_h.m4' -> './k/coreutils-8.5/m4/math_h.m4' 2024-03-09T09:28:00.815 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/math_h.m4’: No data available 2024-03-09T09:28:00.818 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fpending.m4' -> './k/coreutils-8.5/m4/fpending.m4' 2024-03-09T09:28:00.818 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fpending.m4’: No data available 2024-03-09T09:28:00.820 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/xgetcwd.m4' -> './k/coreutils-8.5/m4/xgetcwd.m4' 2024-03-09T09:28:00.821 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xgetcwd.m4’: No data available 2024-03-09T09:28:00.823 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/utimens.m4' -> './k/coreutils-8.5/m4/utimens.m4' 2024-03-09T09:28:00.823 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/utimens.m4’: No data available 2024-03-09T09:28:00.825 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/exponentd.m4' -> './k/coreutils-8.5/m4/exponentd.m4' 2024-03-09T09:28:00.825 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/exponentd.m4’: No data available 2024-03-09T09:28:00.827 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getaddrinfo.m4' -> './k/coreutils-8.5/m4/getaddrinfo.m4' 2024-03-09T09:28:00.828 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getaddrinfo.m4’: No data available 2024-03-09T09:28:00.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sys_ioctl_h.m4' -> './k/coreutils-8.5/m4/sys_ioctl_h.m4' 2024-03-09T09:28:00.830 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_ioctl_h.m4’: No data available 2024-03-09T09:28:00.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stdlib_h.m4' -> './k/coreutils-8.5/m4/stdlib_h.m4' 2024-03-09T09:28:00.832 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdlib_h.m4’: No data available 2024-03-09T09:28:00.834 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mgetgroups.m4' -> './k/coreutils-8.5/m4/mgetgroups.m4' 2024-03-09T09:28:00.835 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mgetgroups.m4’: No data available 2024-03-09T09:28:00.837 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/open.m4' -> './k/coreutils-8.5/m4/open.m4' 2024-03-09T09:28:00.837 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/open.m4’: No data available 2024-03-09T09:28:00.839 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strpbrk.m4' -> './k/coreutils-8.5/m4/strpbrk.m4' 2024-03-09T09:28:00.839 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strpbrk.m4’: No data available 2024-03-09T09:28:00.841 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/ssize_t.m4' -> './k/coreutils-8.5/m4/ssize_t.m4' 2024-03-09T09:28:00.841 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ssize_t.m4’: No data available 2024-03-09T09:28:00.843 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/priv-set.m4' -> './k/coreutils-8.5/m4/priv-set.m4' 2024-03-09T09:28:00.843 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/priv-set.m4’: No data available 2024-03-09T09:28:00.845 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fcntl-o.m4' -> './k/coreutils-8.5/m4/fcntl-o.m4' 2024-03-09T09:28:00.845 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fcntl-o.m4’: No data available 2024-03-09T09:28:00.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/inttostr.m4' -> './k/coreutils-8.5/m4/inttostr.m4' 2024-03-09T09:28:00.848 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inttostr.m4’: No data available 2024-03-09T09:28:00.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/malloca.m4' -> './k/coreutils-8.5/m4/malloca.m4' 2024-03-09T09:28:00.850 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/malloca.m4’: No data available 2024-03-09T09:28:00.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/unlocked-io.m4' -> './k/coreutils-8.5/m4/unlocked-io.m4' 2024-03-09T09:28:00.853 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unlocked-io.m4’: No data available 2024-03-09T09:28:00.855 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getugroups.m4' -> './k/coreutils-8.5/m4/getugroups.m4' 2024-03-09T09:28:00.855 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getugroups.m4’: No data available 2024-03-09T09:28:00.856 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sha1.m4' -> './k/coreutils-8.5/m4/sha1.m4' 2024-03-09T09:28:00.856 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sha1.m4’: No data available 2024-03-09T09:28:00.858 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fpurge.m4' -> './k/coreutils-8.5/m4/fpurge.m4' 2024-03-09T09:28:00.859 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fpurge.m4’: No data available 2024-03-09T09:28:00.860 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/gnulib-comp.m4' -> './k/coreutils-8.5/m4/gnulib-comp.m4' 2024-03-09T09:28:00.860 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gnulib-comp.m4’: No data available 2024-03-09T09:28:00.862 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/wcrtomb.m4' -> './k/coreutils-8.5/m4/wcrtomb.m4' 2024-03-09T09:28:00.863 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wcrtomb.m4’: No data available 2024-03-09T09:28:00.863 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/putenv.m4' -> './k/coreutils-8.5/m4/putenv.m4' 2024-03-09T09:28:00.863 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/putenv.m4’: No data available 2024-03-09T09:28:00.864 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/lstat.m4' -> './k/coreutils-8.5/m4/lstat.m4' 2024-03-09T09:28:00.865 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lstat.m4’: No data available 2024-03-09T09:28:00.866 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strsignal.m4' -> './k/coreutils-8.5/m4/strsignal.m4' 2024-03-09T09:28:00.867 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strsignal.m4’: No data available 2024-03-09T09:28:00.868 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mbslen.m4' -> './k/coreutils-8.5/m4/mbslen.m4' 2024-03-09T09:28:00.868 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbslen.m4’: No data available 2024-03-09T09:28:00.870 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/inttypes.m4' -> './k/coreutils-8.5/m4/inttypes.m4' 2024-03-09T09:28:00.871 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inttypes.m4’: No data available 2024-03-09T09:28:00.877 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stdlib-safer.m4' -> './k/coreutils-8.5/m4/stdlib-safer.m4' 2024-03-09T09:28:00.877 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdlib-safer.m4’: No data available 2024-03-09T09:28:00.955 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_require_config_h_first' -> './k/coreutils-8.5/.x-sc_require_config_h_first' 2024-03-09T09:28:00.955 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_require_config_h_first’: No data available 2024-03-09T09:28:00.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po' -> './k/coreutils-8.5/po' 2024-03-09T09:28:00.990 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/sl.po' -> './k/coreutils-8.5/po/sl.po' 2024-03-09T09:28:00.990 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sl.po’: No data available 2024-03-09T09:28:00.996 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/zh_TW.po' -> './k/coreutils-8.5/po/zh_TW.po' 2024-03-09T09:28:00.997 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/zh_TW.po’: No data available 2024-03-09T09:28:01.004 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/sk.po' -> './k/coreutils-8.5/po/sk.po' 2024-03-09T09:28:01.005 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sk.po’: No data available 2024-03-09T09:28:01.008 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/el.gmo' -> './k/coreutils-8.5/po/el.gmo' 2024-03-09T09:28:01.008 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/el.gmo’: No data available 2024-03-09T09:28:01.010 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/en.gmo' -> './k/coreutils-8.5/po/en.gmo' 2024-03-09T09:28:01.010 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/en.gmo’: No data available 2024-03-09T09:28:01.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/lg.po' -> './k/coreutils-8.5/po/lg.po' 2024-03-09T09:28:01.020 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/lg.po’: No data available 2024-03-09T09:28:01.026 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/sv.gmo' -> './k/coreutils-8.5/po/sv.gmo' 2024-03-09T09:28:01.026 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sv.gmo’: No data available 2024-03-09T09:28:01.032 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/pl.gmo' -> './k/coreutils-8.5/po/pl.gmo' 2024-03-09T09:28:01.032 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pl.gmo’: No data available 2024-03-09T09:28:01.034 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/insert-header.sin' -> './k/coreutils-8.5/po/insert-header.sin' 2024-03-09T09:28:01.034 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/insert-header.sin’: No data available 2024-03-09T09:28:01.037 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/sk.gmo' -> './k/coreutils-8.5/po/sk.gmo' 2024-03-09T09:28:01.037 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sk.gmo’: No data available 2024-03-09T09:28:01.041 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ja.gmo' -> './k/coreutils-8.5/po/ja.gmo' 2024-03-09T09:28:01.042 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ja.gmo’: No data available 2024-03-09T09:28:01.049 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ko.po' -> './k/coreutils-8.5/po/ko.po' 2024-03-09T09:28:01.049 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ko.po’: No data available 2024-03-09T09:28:01.054 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/da.gmo' -> './k/coreutils-8.5/po/da.gmo' 2024-03-09T09:28:01.054 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/da.gmo’: No data available 2024-03-09T09:28:01.057 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/LINGUAS' -> './k/coreutils-8.5/po/LINGUAS' 2024-03-09T09:28:01.057 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/LINGUAS’: No data available 2024-03-09T09:28:01.061 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/zh_TW.gmo' -> './k/coreutils-8.5/po/zh_TW.gmo' 2024-03-09T09:28:01.061 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/zh_TW.gmo’: No data available 2024-03-09T09:28:01.065 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/tr.gmo' -> './k/coreutils-8.5/po/tr.gmo' 2024-03-09T09:28:01.065 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/tr.gmo’: No data available 2024-03-09T09:28:01.068 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/gl.gmo' -> './k/coreutils-8.5/po/gl.gmo' 2024-03-09T09:28:01.068 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/gl.gmo’: No data available 2024-03-09T09:28:01.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/fi.po' -> './k/coreutils-8.5/po/fi.po' 2024-03-09T09:28:01.078 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/fi.po’: No data available 2024-03-09T09:28:01.085 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/da.po' -> './k/coreutils-8.5/po/da.po' 2024-03-09T09:28:01.085 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/da.po’: No data available 2024-03-09T09:28:01.089 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/fi.gmo' -> './k/coreutils-8.5/po/fi.gmo' 2024-03-09T09:28:01.090 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/fi.gmo’: No data available 2024-03-09T09:28:01.092 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/quot.sed' -> './k/coreutils-8.5/po/quot.sed' 2024-03-09T09:28:01.092 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/quot.sed’: No data available 2024-03-09T09:28:01.094 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/stamp-po' -> './k/coreutils-8.5/po/stamp-po' 2024-03-09T09:28:01.095 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/stamp-po’: No data available 2024-03-09T09:28:01.097 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ms.gmo' -> './k/coreutils-8.5/po/ms.gmo' 2024-03-09T09:28:01.097 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ms.gmo’: No data available 2024-03-09T09:28:01.104 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/pt_BR.po' -> './k/coreutils-8.5/po/pt_BR.po' 2024-03-09T09:28:01.104 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pt_BR.po’: No data available 2024-03-09T09:28:01.106 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/Makevars' -> './k/coreutils-8.5/po/Makevars' 2024-03-09T09:28:01.107 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/Makevars’: No data available 2024-03-09T09:28:01.109 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/boldquot.sed' -> './k/coreutils-8.5/po/boldquot.sed' 2024-03-09T09:28:01.109 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/boldquot.sed’: No data available 2024-03-09T09:28:01.114 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/coreutils.pot' -> './k/coreutils-8.5/po/coreutils.pot' 2024-03-09T09:28:01.115 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/coreutils.pot’: No data available 2024-03-09T09:28:01.122 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/de.po' -> './k/coreutils-8.5/po/de.po' 2024-03-09T09:28:01.122 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/de.po’: No data available 2024-03-09T09:28:01.127 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/pt.gmo' -> './k/coreutils-8.5/po/pt.gmo' 2024-03-09T09:28:01.128 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pt.gmo’: No data available 2024-03-09T09:28:01.130 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/Makefile.in.in' -> './k/coreutils-8.5/po/Makefile.in.in' 2024-03-09T09:28:01.130 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/Makefile.in.in’: No data available 2024-03-09T09:28:01.137 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/nl.gmo' -> './k/coreutils-8.5/po/nl.gmo' 2024-03-09T09:28:01.137 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/nl.gmo’: No data available 2024-03-09T09:28:01.143 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/fr.gmo' -> './k/coreutils-8.5/po/fr.gmo' 2024-03-09T09:28:01.143 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/fr.gmo’: No data available 2024-03-09T09:28:01.148 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ga.gmo' -> './k/coreutils-8.5/po/ga.gmo' 2024-03-09T09:28:01.148 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ga.gmo’: No data available 2024-03-09T09:28:01.158 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/cs.po' -> './k/coreutils-8.5/po/cs.po' 2024-03-09T09:28:01.158 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/cs.po’: No data available 2024-03-09T09:28:01.166 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ru.po' -> './k/coreutils-8.5/po/ru.po' 2024-03-09T09:28:01.166 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ru.po’: No data available 2024-03-09T09:28:01.172 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/hu.po' -> './k/coreutils-8.5/po/hu.po' 2024-03-09T09:28:01.173 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/hu.po’: No data available 2024-03-09T09:28:01.179 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/id.po' -> './k/coreutils-8.5/po/id.po' 2024-03-09T09:28:01.180 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/id.po’: No data available 2024-03-09T09:28:01.187 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ru.gmo' -> './k/coreutils-8.5/po/ru.gmo' 2024-03-09T09:28:01.187 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ru.gmo’: No data available 2024-03-09T09:28:01.194 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/sl.gmo' -> './k/coreutils-8.5/po/sl.gmo' 2024-03-09T09:28:01.194 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sl.gmo’: No data available 2024-03-09T09:28:01.196 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/remove-potcdate.sin' -> './k/coreutils-8.5/po/remove-potcdate.sin' 2024-03-09T09:28:01.196 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/remove-potcdate.sin’: No data available 2024-03-09T09:28:01.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/hu.gmo' -> './k/coreutils-8.5/po/hu.gmo' 2024-03-09T09:28:01.201 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/hu.gmo’: No data available 2024-03-09T09:28:01.207 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/et.po' -> './k/coreutils-8.5/po/et.po' 2024-03-09T09:28:01.208 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/et.po’: No data available 2024-03-09T09:28:01.216 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/gl.po' -> './k/coreutils-8.5/po/gl.po' 2024-03-09T09:28:01.216 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/gl.po’: No data available 2024-03-09T09:28:01.223 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/vi.gmo' -> './k/coreutils-8.5/po/vi.gmo' 2024-03-09T09:28:01.223 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/vi.gmo’: No data available 2024-03-09T09:28:01.232 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/bg.po' -> './k/coreutils-8.5/po/bg.po' 2024-03-09T09:28:01.232 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/bg.po’: No data available 2024-03-09T09:28:01.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/pl.po' -> './k/coreutils-8.5/po/pl.po' 2024-03-09T09:28:01.240 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pl.po’: No data available 2024-03-09T09:28:01.249 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/nl.po' -> './k/coreutils-8.5/po/nl.po' 2024-03-09T09:28:01.249 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/nl.po’: No data available 2024-03-09T09:28:01.254 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/pt_BR.gmo' -> './k/coreutils-8.5/po/pt_BR.gmo' 2024-03-09T09:28:01.254 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pt_BR.gmo’: No data available 2024-03-09T09:28:01.257 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ko.gmo' -> './k/coreutils-8.5/po/ko.gmo' 2024-03-09T09:28:01.258 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ko.gmo’: No data available 2024-03-09T09:28:01.263 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/de.gmo' -> './k/coreutils-8.5/po/de.gmo' 2024-03-09T09:28:01.264 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/de.gmo’: No data available 2024-03-09T09:28:01.266 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/POTFILES.in' -> './k/coreutils-8.5/po/POTFILES.in' 2024-03-09T09:28:01.266 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/POTFILES.in’: No data available 2024-03-09T09:28:01.274 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/sv.po' -> './k/coreutils-8.5/po/sv.po' 2024-03-09T09:28:01.274 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sv.po’: No data available 2024-03-09T09:28:01.282 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/it.po' -> './k/coreutils-8.5/po/it.po' 2024-03-09T09:28:01.282 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/it.po’: No data available 2024-03-09T09:28:01.285 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/eu.gmo' -> './k/coreutils-8.5/po/eu.gmo' 2024-03-09T09:28:01.285 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/eu.gmo’: No data available 2024-03-09T09:28:01.290 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/en.po' -> './k/coreutils-8.5/po/en.po' 2024-03-09T09:28:01.291 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/en.po’: No data available 2024-03-09T09:28:01.297 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/it.gmo' -> './k/coreutils-8.5/po/it.gmo' 2024-03-09T09:28:01.297 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/it.gmo’: No data available 2024-03-09T09:28:01.299 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/en@boldquot.header' -> './k/coreutils-8.5/po/en@boldquot.header' 2024-03-09T09:28:01.300 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/en@boldquot.header’: No data available 2024-03-09T09:28:01.304 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/et.gmo' -> './k/coreutils-8.5/po/et.gmo' 2024-03-09T09:28:01.304 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/et.gmo’: No data available 2024-03-09T09:28:01.312 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ja.po' -> './k/coreutils-8.5/po/ja.po' 2024-03-09T09:28:01.312 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ja.po’: No data available 2024-03-09T09:28:01.318 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/zh_CN.gmo' -> './k/coreutils-8.5/po/zh_CN.gmo' 2024-03-09T09:28:01.318 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/zh_CN.gmo’: No data available 2024-03-09T09:28:01.326 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/uk.po' -> './k/coreutils-8.5/po/uk.po' 2024-03-09T09:28:01.326 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/uk.po’: No data available 2024-03-09T09:28:01.333 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ca.gmo' -> './k/coreutils-8.5/po/ca.gmo' 2024-03-09T09:28:01.333 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ca.gmo’: No data available 2024-03-09T09:28:01.335 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/Rules-quot' -> './k/coreutils-8.5/po/Rules-quot' 2024-03-09T09:28:01.335 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/Rules-quot’: No data available 2024-03-09T09:28:01.342 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ro.po' -> './k/coreutils-8.5/po/ro.po' 2024-03-09T09:28:01.342 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ro.po’: No data available 2024-03-09T09:28:01.344 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/nb.gmo' -> './k/coreutils-8.5/po/nb.gmo' 2024-03-09T09:28:01.344 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/nb.gmo’: No data available 2024-03-09T09:28:01.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/bg.gmo' -> './k/coreutils-8.5/po/bg.gmo' 2024-03-09T09:28:01.348 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/bg.gmo’: No data available 2024-03-09T09:28:01.354 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/be.po' -> './k/coreutils-8.5/po/be.po' 2024-03-09T09:28:01.355 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/be.po’: No data available 2024-03-09T09:28:01.362 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/vi.po' -> './k/coreutils-8.5/po/vi.po' 2024-03-09T09:28:01.363 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/vi.po’: No data available 2024-03-09T09:28:01.366 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/lt.gmo' -> './k/coreutils-8.5/po/lt.gmo' 2024-03-09T09:28:01.366 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/lt.gmo’: No data available 2024-03-09T09:28:01.372 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/uk.gmo' -> './k/coreutils-8.5/po/uk.gmo' 2024-03-09T09:28:01.372 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/uk.gmo’: No data available 2024-03-09T09:28:01.378 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/id.gmo' -> './k/coreutils-8.5/po/id.gmo' 2024-03-09T09:28:01.378 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/id.gmo’: No data available 2024-03-09T09:28:01.384 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/af.po' -> './k/coreutils-8.5/po/af.po' 2024-03-09T09:28:01.385 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/af.po’: No data available 2024-03-09T09:28:01.389 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/eu.po' -> './k/coreutils-8.5/po/eu.po' 2024-03-09T09:28:01.389 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/eu.po’: No data available 2024-03-09T09:28:01.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ms.po' -> './k/coreutils-8.5/po/ms.po' 2024-03-09T09:28:01.394 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ms.po’: No data available 2024-03-09T09:28:01.396 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/en@quot.header' -> './k/coreutils-8.5/po/en@quot.header' 2024-03-09T09:28:01.396 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/en@quot.header’: No data available 2024-03-09T09:28:01.399 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/af.gmo' -> './k/coreutils-8.5/po/af.gmo' 2024-03-09T09:28:01.399 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/af.gmo’: No data available 2024-03-09T09:28:01.403 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/es.gmo' -> './k/coreutils-8.5/po/es.gmo' 2024-03-09T09:28:01.403 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/es.gmo’: No data available 2024-03-09T09:28:01.409 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ga.po' -> './k/coreutils-8.5/po/ga.po' 2024-03-09T09:28:01.409 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ga.po’: No data available 2024-03-09T09:28:01.412 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ro.gmo' -> './k/coreutils-8.5/po/ro.gmo' 2024-03-09T09:28:01.412 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ro.gmo’: No data available 2024-03-09T09:28:01.419 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ca.po' -> './k/coreutils-8.5/po/ca.po' 2024-03-09T09:28:01.419 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ca.po’: No data available 2024-03-09T09:28:01.429 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/lg.gmo' -> './k/coreutils-8.5/po/lg.gmo' 2024-03-09T09:28:01.429 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/lg.gmo’: No data available 2024-03-09T09:28:01.438 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/zh_CN.po' -> './k/coreutils-8.5/po/zh_CN.po' 2024-03-09T09:28:01.439 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/zh_CN.po’: No data available 2024-03-09T09:28:01.444 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/cs.gmo' -> './k/coreutils-8.5/po/cs.gmo' 2024-03-09T09:28:01.444 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/cs.gmo’: No data available 2024-03-09T09:28:01.450 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/nb.po' -> './k/coreutils-8.5/po/nb.po' 2024-03-09T09:28:01.450 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/nb.po’: No data available 2024-03-09T09:28:01.452 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/ChangeLog-2007' -> './k/coreutils-8.5/po/ChangeLog-2007' 2024-03-09T09:28:01.452 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ChangeLog-2007’: No data available 2024-03-09T09:28:01.460 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/tr.po' -> './k/coreutils-8.5/po/tr.po' 2024-03-09T09:28:01.460 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/tr.po’: No data available 2024-03-09T09:28:01.464 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/be.gmo' -> './k/coreutils-8.5/po/be.gmo' 2024-03-09T09:28:01.464 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/be.gmo’: No data available 2024-03-09T09:28:01.473 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/fr.po' -> './k/coreutils-8.5/po/fr.po' 2024-03-09T09:28:01.473 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/fr.po’: No data available 2024-03-09T09:28:01.482 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/el.po' -> './k/coreutils-8.5/po/el.po' 2024-03-09T09:28:01.483 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/el.po’: No data available 2024-03-09T09:28:01.488 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/lt.po' -> './k/coreutils-8.5/po/lt.po' 2024-03-09T09:28:01.488 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/lt.po’: No data available 2024-03-09T09:28:01.497 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/es.po' -> './k/coreutils-8.5/po/es.po' 2024-03-09T09:28:01.497 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/es.po’: No data available 2024-03-09T09:28:01.504 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/po/pt.po' -> './k/coreutils-8.5/po/pt.po' 2024-03-09T09:28:01.504 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pt.po’: No data available 2024-03-09T09:28:01.621 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests' -> './k/coreutils-8.5/tests' 2024-03-09T09:28:01.621 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/test-lib.sh' -> './k/coreutils-8.5/tests/test-lib.sh' 2024-03-09T09:28:01.621 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/test-lib.sh’: No data available 2024-03-09T09:28:01.623 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/sample-test' -> './k/coreutils-8.5/tests/sample-test' 2024-03-09T09:28:01.623 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/sample-test’: No data available 2024-03-09T09:28:01.625 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/CuTmpdir.pm' -> './k/coreutils-8.5/tests/CuTmpdir.pm' 2024-03-09T09:28:01.625 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/CuTmpdir.pm’: No data available 2024-03-09T09:28:01.626 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/Makefile.am' -> './k/coreutils-8.5/tests/Makefile.am' 2024-03-09T09:28:01.627 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/Makefile.am’: No data available 2024-03-09T09:28:01.628 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/require-perl' -> './k/coreutils-8.5/tests/require-perl' 2024-03-09T09:28:01.629 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/require-perl’: No data available 2024-03-09T09:28:01.631 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/lang-default' -> './k/coreutils-8.5/tests/lang-default' 2024-03-09T09:28:01.631 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/lang-default’: No data available 2024-03-09T09:28:01.633 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/check.mk' -> './k/coreutils-8.5/tests/check.mk' 2024-03-09T09:28:01.633 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/check.mk’: No data available 2024-03-09T09:28:01.637 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/Makefile.in' -> './k/coreutils-8.5/tests/Makefile.in' 2024-03-09T09:28:01.638 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/Makefile.in’: No data available 2024-03-09T09:28:01.642 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/other-fs-tmpdir' -> './k/coreutils-8.5/tests/other-fs-tmpdir' 2024-03-09T09:28:01.642 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/other-fs-tmpdir’: No data available 2024-03-09T09:28:01.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.897+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:01.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.898+0000 7f912be00640 1 -- 172.21.15.67:0/1905258548 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f912c075a00 con 0x7f912c1eee70 2024-03-09T09:28:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.899+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f912c155800 tx=0x7f911c016da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.899+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c01c070 con 0x7f912c1eee70 2024-03-09T09:28:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.901+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f911c012d30 con 0x7f912c1eee70 2024-03-09T09:28:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.901+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 msgr2=0x7f912c154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.901+0000 7f9123e00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f912c155800 tx=0x7f911c016da0 comp rx=0 tx=0).stop 2024-03-09T09:28:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.901+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 msgr2=0x7f912c154d60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.901+0000 7f9123e00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c1eee70 0x7f912c154d60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:01.916 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd' -> './k/coreutils-8.5/tests/dd' 2024-03-09T09:28:01.917 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd/unblock-sync' -> './k/coreutils-8.5/tests/dd/unblock-sync' 2024-03-09T09:28:01.917 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/unblock-sync’: No data available 2024-03-09T09:28:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.911+0000 7f9132c00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c155df0 0x7f912c1e22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:01.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.911+0000 7f9132c00640 1 -- 172.21.15.67:0/1905258548 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c072470 con 0x7f912c155df0 2024-03-09T09:28:01.919 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd/skip-seek2' -> './k/coreutils-8.5/tests/dd/skip-seek2' 2024-03-09T09:28:01.919 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/skip-seek2’: No data available 2024-03-09T09:28:01.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.912+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c155df0 0x7f912c1e22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:01.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.913+0000 7f912be00640 1 -- 172.21.15.67:0/1905258548 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f912c067dd0 con 0x7f912c155df0 2024-03-09T09:28:01.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.914+0000 7f912be00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c155df0 0x7f912c1e22a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f912c155800 tx=0x7f911c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:01.921 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd/unblock' -> './k/coreutils-8.5/tests/dd/unblock' 2024-03-09T09:28:01.922 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/unblock’: No data available 2024-03-09T09:28:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.914+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c00f040 con 0x7f912c155df0 2024-03-09T09:28:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.915+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f911c025570 con 0x7f912c155df0 2024-03-09T09:28:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.915+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c155df0 msgr2=0x7f912c1e22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.915+0000 7f9123e00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c155df0 0x7f912c1e22a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f912c155800 tx=0x7f911c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:28:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.915+0000 7f9123e00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c155df0 msgr2=0x7f912c1e22a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.915+0000 7f9123e00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f912c155df0 0x7f912c1e22a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:01.923 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:28:01.923 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:01.923 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 282, 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 6, 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:01.924 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:01.925 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:28:01.926 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:01.927 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:28:01.928 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:01.929 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:01.930 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:28:01.931 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:01.932 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:01.932 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:01.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.932 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:28:01.933 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:28:01.934 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:01.935 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:28:01.936 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:28:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.916+0000 7f9132c00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f910c06b4d0 msgr2=0x7f910c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.916+0000 7f9132c00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f910c06b4d0 0x7f910c06d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9124035000 tx=0x7f9124030000 comp rx=0 tx=0).stop 2024-03-09T09:28:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.916+0000 7f9132c00640 1 -- 172.21.15.67:0/1905258548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f912c0730c0 msgr2=0x7f912c1ee930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.916+0000 7f9132c00640 1 --2- 172.21.15.67:0/1905258548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f912c0730c0 0x7f912c1ee930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f911800c440 tx=0x7f911800c900 comp rx=0 tx=0).stop 2024-03-09T09:28:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.916+0000 7f9130c00640 1 -- 172.21.15.67:0/1905258548 reap_dead start 2024-03-09T09:28:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.916+0000 7f9130c00640 1 -- 172.21.15.67:0/1905258548 reap_dead start 2024-03-09T09:28:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.917+0000 7f9132c00640 1 -- 172.21.15.67:0/1905258548 shutdown_connections 2024-03-09T09:28:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.917+0000 7f9132c00640 1 -- 172.21.15.67:0/1905258548 >> 172.21.15.67:0/1905258548 conn(0x7f912c0c7370 msgr2=0x7f912c0aaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.917+0000 7f9132c00640 1 -- 172.21.15.67:0/1905258548 shutdown_connections 2024-03-09T09:28:01.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:01.917+0000 7f9132c00640 1 -- 172.21.15.67:0/1905258548 wait complete. 2024-03-09T09:28:01.937 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd/stderr' -> './k/coreutils-8.5/tests/dd/stderr' 2024-03-09T09:28:01.938 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/stderr’: No data available 2024-03-09T09:28:01.938 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd/direct' -> './k/coreutils-8.5/tests/dd/direct' 2024-03-09T09:28:01.938 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/direct’: No data available 2024-03-09T09:28:01.939 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd/skip-seek-past-file' -> './k/coreutils-8.5/tests/dd/skip-seek-past-file' 2024-03-09T09:28:01.939 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/skip-seek-past-file’: No data available 2024-03-09T09:28:01.939 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd/not-rewound' -> './k/coreutils-8.5/tests/dd/not-rewound' 2024-03-09T09:28:01.940 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/not-rewound’: No data available 2024-03-09T09:28:01.940 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd/skip-seek-past-dev' -> './k/coreutils-8.5/tests/dd/skip-seek-past-dev' 2024-03-09T09:28:01.940 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/skip-seek-past-dev’: No data available 2024-03-09T09:28:01.940 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd/misc' -> './k/coreutils-8.5/tests/dd/misc' 2024-03-09T09:28:01.941 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/misc’: No data available 2024-03-09T09:28:01.942 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd/reblock' -> './k/coreutils-8.5/tests/dd/reblock' 2024-03-09T09:28:01.942 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/reblock’: No data available 2024-03-09T09:28:01.942 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/dd/skip-seek' -> './k/coreutils-8.5/tests/dd/skip-seek' 2024-03-09T09:28:01.943 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/skip-seek’: No data available 2024-03-09T09:28:01.956 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:28:01.956 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:28:01.956 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:28:02.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.208+0000 7fd36d600640 1 Processor -- start 2024-03-09T09:28:02.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.208+0000 7fd36d600640 1 -- start start 2024-03-09T09:28:02.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.210+0000 7fd36d600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd3680c22a0 0x7fd368071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:02.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.210+0000 7fd36d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd368157d60 0x7fd3680acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.210+0000 7fd36d600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3681581a0 con 0x7fd3680ac730 2024-03-09T09:28:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.210+0000 7fd36d600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd368070ee0 con 0x7fd368157d60 2024-03-09T09:28:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.210+0000 7fd36d600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd368154d00 con 0x7fd3680c22a0 2024-03-09T09:28:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.210+0000 7fd367400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd368157d60 0x7fd3680acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.210+0000 7fd366a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fd3680ac730 0x7fd368157650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36316/0 (socket says 172.21.15.67:36316) 2024-03-09T09:28:02.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.210+0000 7fd366a00640 1 -- 172.21.15.67:0/1714883860 learned_addr learned my addr 172.21.15.67:0/1714883860 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:28:02.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.211+0000 7fd367400640 1 -- 172.21.15.67:0/1714883860 >> v2:172.21.15.134:3300/0 conn(0x7fd3680c22a0 msgr2=0x7fd368071190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-09T09:28:02.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.211+0000 7fd367400640 1 --2- 172.21.15.67:0/1714883860 >> v2:172.21.15.134:3300/0 conn(0x7fd3680c22a0 0x7fd368071190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:02.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.211+0000 7fd367400640 1 -- 172.21.15.67:0/1714883860 >> v1:172.21.15.67:6789/0 conn(0x7fd3680ac730 legacy=0x7fd368157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:02.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.211+0000 7fd366000640 1 -- 172.21.15.67:0/1714883860 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041591352 0 0) 0x7fd3681581a0 con 0x7fd3680ac730 2024-03-09T09:28:02.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.211+0000 7fd367400640 1 -- 172.21.15.67:0/1714883860 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd368153d40 con 0x7fd368157d60 2024-03-09T09:28:02.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.212+0000 7fd367400640 1 --2- 172.21.15.67:0/1714883860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd368157d60 0x7fd3680acb10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd3680c1a30 tx=0x7fd36002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94207aa5f028c978 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:02.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.212+0000 7fd366000640 1 -- 172.21.15.67:0/1714883860 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360039070 con 0x7fd368157d60 2024-03-09T09:28:02.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.212+0000 7fd366000640 1 -- 172.21.15.67:0/1714883860 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36002cc00 con 0x7fd368157d60 2024-03-09T09:28:02.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.212+0000 7fd366000640 1 -- 172.21.15.67:0/1714883860 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360034440 con 0x7fd368157d60 2024-03-09T09:28:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.213+0000 7fd36d600640 1 -- 172.21.15.67:0/1714883860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd368157d60 msgr2=0x7fd3680acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.213+0000 7fd36d600640 1 --2- 172.21.15.67:0/1714883860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd368157d60 0x7fd3680acb10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd3680c1a30 tx=0x7fd36002be50 comp rx=0 tx=0).stop 2024-03-09T09:28:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.213+0000 7fd36d600640 1 -- 172.21.15.67:0/1714883860 shutdown_connections 2024-03-09T09:28:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.213+0000 7fd36d600640 1 --2- 172.21.15.67:0/1714883860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd368157d60 0x7fd3680acb10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.213+0000 7fd36d600640 1 --2- 172.21.15.67:0/1714883860 >> v2:172.21.15.134:3300/0 conn(0x7fd3680c22a0 0x7fd368071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.213+0000 7fd36d600640 1 -- 172.21.15.67:0/1714883860 >> 172.21.15.67:0/1714883860 conn(0x7fd3680c7370 msgr2=0x7fd3680c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.213+0000 7fd36d600640 1 -- 172.21.15.67:0/1714883860 shutdown_connections 2024-03-09T09:28:02.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.213+0000 7fd36d600640 1 -- 172.21.15.67:0/1714883860 wait complete. 2024-03-09T09:28:02.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.215+0000 7fd36d600640 1 Processor -- start 2024-03-09T09:28:02.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.216+0000 7fd36d600640 1 -- start start 2024-03-09T09:28:02.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.217+0000 7fd36d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3680ac730 0x7fd36814e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:02.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.217+0000 7fd36d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3680c22a0 0x7fd3680abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:02.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.217+0000 7fd36d600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd368157d60 0x7fd3681ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.217+0000 7fd36d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd368070c00 con 0x7fd3680ac730 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.217+0000 7fd36d600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd368070920 con 0x7fd3680c22a0 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.217+0000 7fd36d600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd368151610 con 0x7fd368157d60 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.217+0000 7fd366a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3680ac730 0x7fd36814e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.218+0000 7fd366a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3680ac730 0x7fd36814e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41000/0 (socket says 172.21.15.67:41000) 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.218+0000 7fd366a00640 1 -- 172.21.15.67:0/812422578 learned_addr learned my addr 172.21.15.67:0/812422578 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.218+0000 7fd35e000640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3680c22a0 0x7fd3680abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.218+0000 7fd366a00640 1 -- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd368157d60 msgr2=0x7fd3681ef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.218+0000 7fd366a00640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd368157d60 0x7fd3681ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.218+0000 7fd366a00640 1 -- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3680c22a0 msgr2=0x7fd3680abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.218+0000 7fd366a00640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3680c22a0 0x7fd3680abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.218+0000 7fd366a00640 1 -- 172.21.15.67:0/812422578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3681ebd00 con 0x7fd3680ac730 2024-03-09T09:28:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.218+0000 7fd35e000640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3680c22a0 0x7fd3680abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:28:02.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.218+0000 7fd366a00640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3680ac730 0x7fd36814e790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd35800c4d0 tx=0x7fd35800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:02.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.219+0000 7fd35f400640 1 -- 172.21.15.67:0/812422578 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358018070 con 0x7fd3680ac730 2024-03-09T09:28:02.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.219+0000 7fd35f400640 1 -- 172.21.15.67:0/812422578 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd358007500 con 0x7fd3680ac730 2024-03-09T09:28:02.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.219+0000 7fd35f400640 1 -- 172.21.15.67:0/812422578 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358004a20 con 0x7fd3680ac730 2024-03-09T09:28:02.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.219+0000 7fd36d600640 1 -- 172.21.15.67:0/812422578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36814ef00 con 0x7fd3680ac730 2024-03-09T09:28:02.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.219+0000 7fd36d600640 1 -- 172.21.15.67:0/812422578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3680ac020 con 0x7fd3680ac730 2024-03-09T09:28:02.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.220+0000 7fd36d600640 1 -- 172.21.15.67:0/812422578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3681ebaf0 con 0x7fd3680ac730 2024-03-09T09:28:02.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.221+0000 7fd35f400640 1 -- 172.21.15.67:0/812422578 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd35801b070 con 0x7fd3680ac730 2024-03-09T09:28:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.222+0000 7fd35f400640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd34406b500 0x7fd34406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.222+0000 7fd35f400640 1 -- 172.21.15.67:0/812422578 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fd3580898e0 con 0x7fd3680ac730 2024-03-09T09:28:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.222+0000 7fd35e000640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd34406b500 0x7fd34406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:02.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.224+0000 7fd35e000640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd34406b500 0x7fd34406d9c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd3680c1500 tx=0x7fd350008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.227+0000 7fd35f400640 1 -- 172.21.15.67:0/812422578 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd358058390 con 0x7fd3680ac730 2024-03-09T09:28:02.277 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr' -> './k/coreutils-8.5/tests/pr' 2024-03-09T09:28:02.277 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/t_tab_' -> './k/coreutils-8.5/tests/pr/t_tab_' 2024-03-09T09:28:02.278 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t_tab_’: No data available 2024-03-09T09:28:02.279 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/FF' -> './k/coreutils-8.5/tests/pr/FF' 2024-03-09T09:28:02.280 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/FF’: No data available 2024-03-09T09:28:02.282 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/FnFn' -> './k/coreutils-8.5/tests/pr/FnFn' 2024-03-09T09:28:02.283 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/FnFn’: No data available 2024-03-09T09:28:02.284 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/o3ml24f-bl-tn' -> './k/coreutils-8.5/tests/pr/o3ml24f-bl-tn' 2024-03-09T09:28:02.285 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3ml24f-bl-tn’: No data available 2024-03-09T09:28:02.287 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+5-8b3l17f-FF' -> './k/coreutils-8.5/tests/pr/n+5-8b3l17f-FF' 2024-03-09T09:28:02.287 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+5-8b3l17f-FF’: No data available 2024-03-09T09:28:02.289 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3ml24f-bl-tn' -> './k/coreutils-8.5/tests/pr/n+3ml24f-bl-tn' 2024-03-09T09:28:02.290 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3ml24f-bl-tn’: No data available 2024-03-09T09:28:02.291 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/FFtn' -> './k/coreutils-8.5/tests/pr/FFtn' 2024-03-09T09:28:02.292 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/FFtn’: No data available 2024-03-09T09:28:02.293 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/a2l15-FF' -> './k/coreutils-8.5/tests/pr/a2l15-FF' 2024-03-09T09:28:02.293 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a2l15-FF’: No data available 2024-03-09T09:28:02.295 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tFnFt' -> './k/coreutils-8.5/tests/pr/tFnFt' 2024-03-09T09:28:02.295 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFnFt’: No data available 2024-03-09T09:28:02.297 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tn2e5o3-t_tab' -> './k/coreutils-8.5/tests/pr/tn2e5o3-t_tab' 2024-03-09T09:28:02.298 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn2e5o3-t_tab’: No data available 2024-03-09T09:28:02.299 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3ml24f-t' -> './k/coreutils-8.5/tests/pr/3ml24f-t' 2024-03-09T09:28:02.299 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3ml24f-t’: No data available 2024-03-09T09:28:02.302 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml20-t-tFFFF' -> './k/coreutils-8.5/tests/pr/nSml20-t-tFFFF' 2024-03-09T09:28:02.302 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml20-t-tFFFF’: No data available 2024-03-09T09:28:02.304 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tn2e8-t_tab' -> './k/coreutils-8.5/tests/pr/tn2e8-t_tab' 2024-03-09T09:28:02.305 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn2e8-t_tab’: No data available 2024-03-09T09:28:02.306 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/b3f-FF' -> './k/coreutils-8.5/tests/pr/b3f-FF' 2024-03-09T09:28:02.306 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3f-FF’: No data available 2024-03-09T09:28:02.308 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/2w60f-t_notab' -> './k/coreutils-8.5/tests/pr/2w60f-t_notab' 2024-03-09T09:28:02.308 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2w60f-t_notab’: No data available 2024-03-09T09:28:02.311 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/FFn' -> './k/coreutils-8.5/tests/pr/FFn' 2024-03-09T09:28:02.311 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/FFn’: No data available 2024-03-09T09:28:02.314 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/t_tab' -> './k/coreutils-8.5/tests/pr/t_tab' 2024-03-09T09:28:02.314 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t_tab’: No data available 2024-03-09T09:28:02.316 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/b3-0F' -> './k/coreutils-8.5/tests/pr/b3-0F' 2024-03-09T09:28:02.316 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3-0F’: No data available 2024-03-09T09:28:02.318 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+4-8a2l17-FF' -> './k/coreutils-8.5/tests/pr/n+4-8a2l17-FF' 2024-03-09T09:28:02.318 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+4-8a2l17-FF’: No data available 2024-03-09T09:28:02.321 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+4b2l17f-0FF' -> './k/coreutils-8.5/tests/pr/n+4b2l17f-0FF' 2024-03-09T09:28:02.321 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+4b2l17f-0FF’: No data available 2024-03-09T09:28:02.323 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3a3l15-t' -> './k/coreutils-8.5/tests/pr/3a3l15-t' 2024-03-09T09:28:02.323 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3a3l15-t’: No data available 2024-03-09T09:28:02.325 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tFFn' -> './k/coreutils-8.5/tests/pr/tFFn' 2024-03-09T09:28:02.325 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFFn’: No data available 2024-03-09T09:28:02.326 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3l24f-t' -> './k/coreutils-8.5/tests/pr/3l24f-t' 2024-03-09T09:28:02.326 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3l24f-t’: No data available 2024-03-09T09:28:02.327 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/0FnFt' -> './k/coreutils-8.5/tests/pr/0FnFt' 2024-03-09T09:28:02.328 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0FnFt’: No data available 2024-03-09T09:28:02.328 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3-FF' -> './k/coreutils-8.5/tests/pr/3-FF' 2024-03-09T09:28:02.329 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3-FF’: No data available 2024-03-09T09:28:02.331 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/t-FF' -> './k/coreutils-8.5/tests/pr/t-FF' 2024-03-09T09:28:02.331 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t-FF’: No data available 2024-03-09T09:28:02.332 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml24-bl-FF' -> './k/coreutils-8.5/tests/pr/nSml24-bl-FF' 2024-03-09T09:28:02.333 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml24-bl-FF’: No data available 2024-03-09T09:28:02.333 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3l24f-bl' -> './k/coreutils-8.5/tests/pr/n+3l24f-bl' 2024-03-09T09:28:02.334 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3l24f-bl’: No data available 2024-03-09T09:28:02.335 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/b3l15-t' -> './k/coreutils-8.5/tests/pr/b3l15-t' 2024-03-09T09:28:02.335 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3l15-t’: No data available 2024-03-09T09:28:02.336 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3b3f-FF' -> './k/coreutils-8.5/tests/pr/3b3f-FF' 2024-03-09T09:28:02.337 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b3f-FF’: No data available 2024-03-09T09:28:02.338 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3b3f-0F' -> './k/coreutils-8.5/tests/pr/3b3f-0F' 2024-03-09T09:28:02.338 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b3f-0F’: No data available 2024-03-09T09:28:02.339 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W35b3l24f-lm' -> './k/coreutils-8.5/tests/pr/W35b3l24f-lm' 2024-03-09T09:28:02.339 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35b3l24f-lm’: No data available 2024-03-09T09:28:02.341 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/0Ft' -> './k/coreutils-8.5/tests/pr/0Ft' 2024-03-09T09:28:02.341 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0Ft’: No data available 2024-03-09T09:28:02.342 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tt-bl' -> './k/coreutils-8.5/tests/pr/tt-bl' 2024-03-09T09:28:02.342 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tt-bl’: No data available 2024-03-09T09:28:02.344 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3ml24-t-FF' -> './k/coreutils-8.5/tests/pr/3ml24-t-FF' 2024-03-09T09:28:02.344 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3ml24-t-FF’: No data available 2024-03-09T09:28:02.345 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tFFt-lm' -> './k/coreutils-8.5/tests/pr/tFFt-lm' 2024-03-09T09:28:02.345 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFFt-lm’: No data available 2024-03-09T09:28:02.347 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml20-bl-FF' -> './k/coreutils-8.5/tests/pr/nSml20-bl-FF' 2024-03-09T09:28:02.348 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml20-bl-FF’: No data available 2024-03-09T09:28:02.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tn2e8o3-t_tab' -> './k/coreutils-8.5/tests/pr/tn2e8o3-t_tab' 2024-03-09T09:28:02.350 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn2e8o3-t_tab’: No data available 2024-03-09T09:28:02.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W28l24f-ll' -> './k/coreutils-8.5/tests/pr/W28l24f-ll' 2024-03-09T09:28:02.353 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W28l24f-ll’: No data available 2024-03-09T09:28:02.355 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml24-t-tFFFF' -> './k/coreutils-8.5/tests/pr/nSml24-t-tFFFF' 2024-03-09T09:28:02.355 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml24-t-tFFFF’: No data available 2024-03-09T09:28:02.356 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/t-0FF' -> './k/coreutils-8.5/tests/pr/t-0FF' 2024-03-09T09:28:02.356 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t-0FF’: No data available 2024-03-09T09:28:02.357 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tb3-FF' -> './k/coreutils-8.5/tests/pr/tb3-FF' 2024-03-09T09:28:02.358 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tb3-FF’: No data available 2024-03-09T09:28:02.360 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/o3b3Sl24f-tn' -> './k/coreutils-8.5/tests/pr/o3b3Sl24f-tn' 2024-03-09T09:28:02.360 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3b3Sl24f-tn’: No data available 2024-03-09T09:28:02.361 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/o3mSl24f-bl-tn' -> './k/coreutils-8.5/tests/pr/o3mSl24f-bl-tn' 2024-03-09T09:28:02.361 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3mSl24f-bl-tn’: No data available 2024-03-09T09:28:02.363 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3b2l17-FF' -> './k/coreutils-8.5/tests/pr/3b2l17-FF' 2024-03-09T09:28:02.363 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b2l17-FF’: No data available 2024-03-09T09:28:02.365 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/2f-t_notab' -> './k/coreutils-8.5/tests/pr/2f-t_notab' 2024-03-09T09:28:02.365 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2f-t_notab’: No data available 2024-03-09T09:28:02.366 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/b2l15-FF' -> './k/coreutils-8.5/tests/pr/b2l15-FF' 2024-03-09T09:28:02.366 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b2l15-FF’: No data available 2024-03-09T09:28:02.368 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3b3f-0FF' -> './k/coreutils-8.5/tests/pr/3b3f-0FF' 2024-03-09T09:28:02.368 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b3f-0FF’: No data available 2024-03-09T09:28:02.369 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/a3l15-t' -> './k/coreutils-8.5/tests/pr/a3l15-t' 2024-03-09T09:28:02.369 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3l15-t’: No data available 2024-03-09T09:28:02.371 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/o3Jml24f-lm-lo' -> './k/coreutils-8.5/tests/pr/o3Jml24f-lm-lo' 2024-03-09T09:28:02.371 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3Jml24f-lm-lo’: No data available 2024-03-09T09:28:02.373 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tb3-0FF' -> './k/coreutils-8.5/tests/pr/tb3-0FF' 2024-03-09T09:28:02.373 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tb3-0FF’: No data available 2024-03-09T09:28:02.375 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/l24-t' -> './k/coreutils-8.5/tests/pr/l24-t' 2024-03-09T09:28:02.375 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/l24-t’: No data available 2024-03-09T09:28:02.377 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W35ml24f-lm-lo' -> './k/coreutils-8.5/tests/pr/W35ml24f-lm-lo' 2024-03-09T09:28:02.377 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35ml24f-lm-lo’: No data available 2024-03-09T09:28:02.379 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/a3-0F' -> './k/coreutils-8.5/tests/pr/a3-0F' 2024-03-09T09:28:02.379 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3-0F’: No data available 2024-03-09T09:28:02.380 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24f-0F' -> './k/coreutils-8.5/tests/pr/ml24f-0F' 2024-03-09T09:28:02.381 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24f-0F’: No data available 2024-03-09T09:28:02.382 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/loli' -> './k/coreutils-8.5/tests/pr/loli' 2024-03-09T09:28:02.383 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/loli’: No data available 2024-03-09T09:28:02.384 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3ml24-t' -> './k/coreutils-8.5/tests/pr/3ml24-t' 2024-03-09T09:28:02.384 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3ml24-t’: No data available 2024-03-09T09:28:02.386 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/a3l24f-lm' -> './k/coreutils-8.5/tests/pr/a3l24f-lm' 2024-03-09T09:28:02.386 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3l24f-lm’: No data available 2024-03-09T09:28:02.389 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/Jb3l24f-lm' -> './k/coreutils-8.5/tests/pr/Jb3l24f-lm' 2024-03-09T09:28:02.389 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/Jb3l24f-lm’: No data available 2024-03-09T09:28:02.390 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/t-t' -> './k/coreutils-8.5/tests/pr/t-t' 2024-03-09T09:28:02.390 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t-t’: No data available 2024-03-09T09:28:02.392 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/0F' -> './k/coreutils-8.5/tests/pr/0F' 2024-03-09T09:28:02.392 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0F’: No data available 2024-03-09T09:28:02.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/w72l24f-ll' -> './k/coreutils-8.5/tests/pr/w72l24f-ll' 2024-03-09T09:28:02.394 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/w72l24f-ll’: No data available 2024-03-09T09:28:02.395 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24f-t' -> './k/coreutils-8.5/tests/pr/ml24f-t' 2024-03-09T09:28:02.395 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24f-t’: No data available 2024-03-09T09:28:02.397 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/b3l15f-t' -> './k/coreutils-8.5/tests/pr/b3l15f-t' 2024-03-09T09:28:02.397 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3l15f-t’: No data available 2024-03-09T09:28:02.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/0FnFnt' -> './k/coreutils-8.5/tests/pr/0FnFnt' 2024-03-09T09:28:02.399 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0FnFnt’: No data available 2024-03-09T09:28:02.400 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/b3l24f-lm' -> './k/coreutils-8.5/tests/pr/b3l24f-lm' 2024-03-09T09:28:02.400 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3l24f-lm’: No data available 2024-03-09T09:28:02.402 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/o3mSnl24fbltn' -> './k/coreutils-8.5/tests/pr/o3mSnl24fbltn' 2024-03-09T09:28:02.402 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3mSnl24fbltn’: No data available 2024-03-09T09:28:02.405 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/nJml24f-lmlolm' -> './k/coreutils-8.5/tests/pr/nJml24f-lmlolm' 2024-03-09T09:28:02.405 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nJml24f-lmlolm’: No data available 2024-03-09T09:28:02.406 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/o3a3Sl24f-tn' -> './k/coreutils-8.5/tests/pr/o3a3Sl24f-tn' 2024-03-09T09:28:02.406 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3a3Sl24f-tn’: No data available 2024-03-09T09:28:02.408 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tne8-t_tab' -> './k/coreutils-8.5/tests/pr/tne8-t_tab' 2024-03-09T09:28:02.408 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tne8-t_tab’: No data available 2024-03-09T09:28:02.409 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3ml24-FF' -> './k/coreutils-8.5/tests/pr/3ml24-FF' 2024-03-09T09:28:02.409 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3ml24-FF’: No data available 2024-03-09T09:28:02.411 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/a3f-0FF' -> './k/coreutils-8.5/tests/pr/a3f-0FF' 2024-03-09T09:28:02.411 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3f-0FF’: No data available 2024-03-09T09:28:02.412 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ml20-FF-t' -> './k/coreutils-8.5/tests/pr/ml20-FF-t' 2024-03-09T09:28:02.413 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml20-FF-t’: No data available 2024-03-09T09:28:02.415 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/pr-tests' -> './k/coreutils-8.5/tests/pr/pr-tests' 2024-03-09T09:28:02.415 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/pr-tests’: No data available 2024-03-09T09:28:02.417 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W35Jb3l24f-lm' -> './k/coreutils-8.5/tests/pr/W35Jb3l24f-lm' 2024-03-09T09:28:02.417 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35Jb3l24f-lm’: No data available 2024-03-09T09:28:02.419 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ttb3-0FF' -> './k/coreutils-8.5/tests/pr/ttb3-0FF' 2024-03-09T09:28:02.419 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ttb3-0FF’: No data available 2024-03-09T09:28:02.421 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/nl24f-bl' -> './k/coreutils-8.5/tests/pr/nl24f-bl' 2024-03-09T09:28:02.421 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nl24f-bl’: No data available 2024-03-09T09:28:02.423 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml24-t-t-FF' -> './k/coreutils-8.5/tests/pr/nSml24-t-t-FF' 2024-03-09T09:28:02.423 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml24-t-t-FF’: No data available 2024-03-09T09:28:02.425 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3b3l15f-t' -> './k/coreutils-8.5/tests/pr/3b3l15f-t' 2024-03-09T09:28:02.425 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b3l15f-t’: No data available 2024-03-09T09:28:02.427 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/0FF' -> './k/coreutils-8.5/tests/pr/0FF' 2024-03-09T09:28:02.427 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0FF’: No data available 2024-03-09T09:28:02.429 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3f-0F' -> './k/coreutils-8.5/tests/pr/3f-0F' 2024-03-09T09:28:02.429 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3f-0F’: No data available 2024-03-09T09:28:02.431 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/Ja3l24f-lm' -> './k/coreutils-8.5/tests/pr/Ja3l24f-lm' 2024-03-09T09:28:02.431 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/Ja3l24f-lm’: No data available 2024-03-09T09:28:02.432 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3a2l17-FF' -> './k/coreutils-8.5/tests/pr/3a2l17-FF' 2024-03-09T09:28:02.433 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3a2l17-FF’: No data available 2024-03-09T09:28:02.434 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24f-t-0F' -> './k/coreutils-8.5/tests/pr/ml24f-t-0F' 2024-03-09T09:28:02.434 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24f-t-0F’: No data available 2024-03-09T09:28:02.436 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tn_2e8-t_tab' -> './k/coreutils-8.5/tests/pr/tn_2e8-t_tab' 2024-03-09T09:28:02.437 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn_2e8-t_tab’: No data available 2024-03-09T09:28:02.438 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/o3a3Snl24f-tn' -> './k/coreutils-8.5/tests/pr/o3a3Snl24f-tn' 2024-03-09T09:28:02.438 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3a3Snl24f-tn’: No data available 2024-03-09T09:28:02.440 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/2s_w60f-t_nota' -> './k/coreutils-8.5/tests/pr/2s_w60f-t_nota' 2024-03-09T09:28:02.440 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2s_w60f-t_nota’: No data available 2024-03-09T09:28:02.442 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/t-bl' -> './k/coreutils-8.5/tests/pr/t-bl' 2024-03-09T09:28:02.442 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t-bl’: No data available 2024-03-09T09:28:02.444 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24-t-FF' -> './k/coreutils-8.5/tests/pr/ml24-t-FF' 2024-03-09T09:28:02.444 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24-t-FF’: No data available 2024-03-09T09:28:02.446 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3-7l24-FF' -> './k/coreutils-8.5/tests/pr/n+3-7l24-FF' 2024-03-09T09:28:02.446 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3-7l24-FF’: No data available 2024-03-09T09:28:02.447 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/t_notab' -> './k/coreutils-8.5/tests/pr/t_notab' 2024-03-09T09:28:02.448 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t_notab’: No data available 2024-03-09T09:28:02.449 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W72Jl24f-ll' -> './k/coreutils-8.5/tests/pr/W72Jl24f-ll' 2024-03-09T09:28:02.449 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W72Jl24f-ll’: No data available 2024-03-09T09:28:02.451 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/a2l17-FF' -> './k/coreutils-8.5/tests/pr/a2l17-FF' 2024-03-09T09:28:02.451 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a2l17-FF’: No data available 2024-03-09T09:28:02.453 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24-FF' -> './k/coreutils-8.5/tests/pr/ml24-FF' 2024-03-09T09:28:02.453 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24-FF’: No data available 2024-03-09T09:28:02.455 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/b3f-0F' -> './k/coreutils-8.5/tests/pr/b3f-0F' 2024-03-09T09:28:02.455 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3f-0F’: No data available 2024-03-09T09:28:02.457 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3l24f-0FF' -> './k/coreutils-8.5/tests/pr/n+3l24f-0FF' 2024-03-09T09:28:02.457 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3l24f-0FF’: No data available 2024-03-09T09:28:02.459 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3ml20f-bl-FF' -> './k/coreutils-8.5/tests/pr/n+3ml20f-bl-FF' 2024-03-09T09:28:02.459 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3ml20f-bl-FF’: No data available 2024-03-09T09:28:02.461 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/a3f-0F' -> './k/coreutils-8.5/tests/pr/a3f-0F' 2024-03-09T09:28:02.461 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3f-0F’: No data available 2024-03-09T09:28:02.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W35Jml24f-lmlo' -> './k/coreutils-8.5/tests/pr/W35Jml24f-lmlo' 2024-03-09T09:28:02.463 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35Jml24f-lmlo’: No data available 2024-03-09T09:28:02.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W27l24f-ll' -> './k/coreutils-8.5/tests/pr/W27l24f-ll' 2024-03-09T09:28:02.465 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W27l24f-ll’: No data available 2024-03-09T09:28:02.466 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24-t' -> './k/coreutils-8.5/tests/pr/ml24-t' 2024-03-09T09:28:02.466 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24-t’: No data available 2024-03-09T09:28:02.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/a3f-FF' -> './k/coreutils-8.5/tests/pr/a3f-FF' 2024-03-09T09:28:02.468 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3f-FF’: No data available 2024-03-09T09:28:02.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tFFt-bl' -> './k/coreutils-8.5/tests/pr/tFFt-bl' 2024-03-09T09:28:02.469 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFFt-bl’: No data available 2024-03-09T09:28:02.471 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tta3-FF' -> './k/coreutils-8.5/tests/pr/tta3-FF' 2024-03-09T09:28:02.471 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tta3-FF’: No data available 2024-03-09T09:28:02.472 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3-0F' -> './k/coreutils-8.5/tests/pr/3-0F' 2024-03-09T09:28:02.472 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3-0F’: No data available 2024-03-09T09:28:02.474 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/2sf-t_notab' -> './k/coreutils-8.5/tests/pr/2sf-t_notab' 2024-03-09T09:28:02.474 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2sf-t_notab’: No data available 2024-03-09T09:28:02.476 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3a3l15f-t' -> './k/coreutils-8.5/tests/pr/3a3l15f-t' 2024-03-09T09:28:02.476 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3a3l15f-t’: No data available 2024-03-09T09:28:02.478 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W26l24f-ll' -> './k/coreutils-8.5/tests/pr/W26l24f-ll' 2024-03-09T09:28:02.478 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W26l24f-ll’: No data available 2024-03-09T09:28:02.479 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+7l24-FF' -> './k/coreutils-8.5/tests/pr/n+7l24-FF' 2024-03-09T09:28:02.480 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+7l24-FF’: No data available 2024-03-09T09:28:02.481 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24f-lm-lo' -> './k/coreutils-8.5/tests/pr/ml24f-lm-lo' 2024-03-09T09:28:02.481 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24f-lm-lo’: No data available 2024-03-09T09:28:02.482 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/l24-FF' -> './k/coreutils-8.5/tests/pr/l24-FF' 2024-03-09T09:28:02.482 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/l24-FF’: No data available 2024-03-09T09:28:02.484 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+2-5l24f-0FF' -> './k/coreutils-8.5/tests/pr/n+2-5l24f-0FF' 2024-03-09T09:28:02.484 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+2-5l24f-0FF’: No data available 2024-03-09T09:28:02.485 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3f-FF' -> './k/coreutils-8.5/tests/pr/3f-FF' 2024-03-09T09:28:02.485 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3f-FF’: No data available 2024-03-09T09:28:02.487 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3ml24f-tn-bl' -> './k/coreutils-8.5/tests/pr/n+3ml24f-tn-bl' 2024-03-09T09:28:02.488 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3ml24f-tn-bl’: No data available 2024-03-09T09:28:02.489 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ta3-0FF' -> './k/coreutils-8.5/tests/pr/ta3-0FF' 2024-03-09T09:28:02.489 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ta3-0FF’: No data available 2024-03-09T09:28:02.491 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3b3l15-t' -> './k/coreutils-8.5/tests/pr/3b3l15-t' 2024-03-09T09:28:02.492 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b3l15-t’: No data available 2024-03-09T09:28:02.493 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+2l24f-0FF' -> './k/coreutils-8.5/tests/pr/n+2l24f-0FF' 2024-03-09T09:28:02.493 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+2l24f-0FF’: No data available 2024-03-09T09:28:02.494 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tn_2e8S-t_tab' -> './k/coreutils-8.5/tests/pr/tn_2e8S-t_tab' 2024-03-09T09:28:02.495 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn_2e8S-t_tab’: No data available 2024-03-09T09:28:02.496 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ta3-FF' -> './k/coreutils-8.5/tests/pr/ta3-FF' 2024-03-09T09:28:02.496 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ta3-FF’: No data available 2024-03-09T09:28:02.498 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tFFt-ll' -> './k/coreutils-8.5/tests/pr/tFFt-ll' 2024-03-09T09:28:02.498 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFFt-ll’: No data available 2024-03-09T09:28:02.500 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+6b3l13f-FF' -> './k/coreutils-8.5/tests/pr/n+6b3l13f-FF' 2024-03-09T09:28:02.500 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+6b3l13f-FF’: No data available 2024-03-09T09:28:02.501 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/2-Sf-t_notab' -> './k/coreutils-8.5/tests/pr/2-Sf-t_notab' 2024-03-09T09:28:02.502 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2-Sf-t_notab’: No data available 2024-03-09T09:28:02.503 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/0FFnt' -> './k/coreutils-8.5/tests/pr/0FFnt' 2024-03-09T09:28:02.504 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0FFnt’: No data available 2024-03-09T09:28:02.505 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W20l24f-ll' -> './k/coreutils-8.5/tests/pr/W20l24f-ll' 2024-03-09T09:28:02.506 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W20l24f-ll’: No data available 2024-03-09T09:28:02.507 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/Jml24f-lm-lo' -> './k/coreutils-8.5/tests/pr/Jml24f-lm-lo' 2024-03-09T09:28:02.507 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/Jml24f-lm-lo’: No data available 2024-03-09T09:28:02.509 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3l24-t' -> './k/coreutils-8.5/tests/pr/3l24-t' 2024-03-09T09:28:02.509 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3l24-t’: No data available 2024-03-09T09:28:02.511 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/o3b3l24f-tn' -> './k/coreutils-8.5/tests/pr/o3b3l24f-tn' 2024-03-09T09:28:02.511 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3b3l24f-tn’: No data available 2024-03-09T09:28:02.513 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W35a3l24f-lm' -> './k/coreutils-8.5/tests/pr/W35a3l24f-lm' 2024-03-09T09:28:02.513 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35a3l24f-lm’: No data available 2024-03-09T09:28:02.515 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/2-S_f-t_notab' -> './k/coreutils-8.5/tests/pr/2-S_f-t_notab' 2024-03-09T09:28:02.515 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2-S_f-t_notab’: No data available 2024-03-09T09:28:02.516 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3-5l24f-t' -> './k/coreutils-8.5/tests/pr/3-5l24f-t' 2024-03-09T09:28:02.516 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3-5l24f-t’: No data available 2024-03-09T09:28:02.518 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/o3b3Snl24f-tn' -> './k/coreutils-8.5/tests/pr/o3b3Snl24f-tn' 2024-03-09T09:28:02.518 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3b3Snl24f-tn’: No data available 2024-03-09T09:28:02.520 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+2l24f-bl' -> './k/coreutils-8.5/tests/pr/n+2l24f-bl' 2024-03-09T09:28:02.520 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+2l24f-bl’: No data available 2024-03-09T09:28:02.522 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W35Ja3l24f-lm' -> './k/coreutils-8.5/tests/pr/W35Ja3l24f-lm' 2024-03-09T09:28:02.522 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35Ja3l24f-lm’: No data available 2024-03-09T09:28:02.524 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+8l20-FF' -> './k/coreutils-8.5/tests/pr/n+8l20-FF' 2024-03-09T09:28:02.524 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+8l20-FF’: No data available 2024-03-09T09:28:02.526 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/0FFt' -> './k/coreutils-8.5/tests/pr/0FFt' 2024-03-09T09:28:02.527 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0FFt’: No data available 2024-03-09T09:28:02.531 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/o3a3l24f-tn' -> './k/coreutils-8.5/tests/pr/o3a3l24f-tn' 2024-03-09T09:28:02.531 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3a3l24f-tn’: No data available 2024-03-09T09:28:02.531 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/2s_f-t_notab' -> './k/coreutils-8.5/tests/pr/2s_f-t_notab' 2024-03-09T09:28:02.532 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2s_f-t_notab’: No data available 2024-03-09T09:28:02.532 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/nN15l24f-bl' -> './k/coreutils-8.5/tests/pr/nN15l24f-bl' 2024-03-09T09:28:02.533 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nN15l24f-bl’: No data available 2024-03-09T09:28:02.533 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/0Fnt' -> './k/coreutils-8.5/tests/pr/0Fnt' 2024-03-09T09:28:02.534 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0Fnt’: No data available 2024-03-09T09:28:02.535 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/4-7l24-FF' -> './k/coreutils-8.5/tests/pr/4-7l24-FF' 2024-03-09T09:28:02.535 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/4-7l24-FF’: No data available 2024-03-09T09:28:02.537 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tn' -> './k/coreutils-8.5/tests/pr/tn' 2024-03-09T09:28:02.538 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn’: No data available 2024-03-09T09:28:02.539 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/b2l17-FF' -> './k/coreutils-8.5/tests/pr/b2l17-FF' 2024-03-09T09:28:02.539 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b2l17-FF’: No data available 2024-03-09T09:28:02.541 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/nJml24f-lmlmlo' -> './k/coreutils-8.5/tests/pr/nJml24f-lmlmlo' 2024-03-09T09:28:02.541 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nJml24f-lmlmlo’: No data available 2024-03-09T09:28:02.543 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tta3-0FF' -> './k/coreutils-8.5/tests/pr/tta3-0FF' 2024-03-09T09:28:02.543 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tta3-0FF’: No data available 2024-03-09T09:28:02.544 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+5a3l13f-0FF' -> './k/coreutils-8.5/tests/pr/n+5a3l13f-0FF' 2024-03-09T09:28:02.545 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+5a3l13f-0FF’: No data available 2024-03-09T09:28:02.546 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/nN1+3l24f-bl' -> './k/coreutils-8.5/tests/pr/nN1+3l24f-bl' 2024-03-09T09:28:02.546 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nN1+3l24f-bl’: No data available 2024-03-09T09:28:02.548 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/2sw60f-t_notab' -> './k/coreutils-8.5/tests/pr/2sw60f-t_notab' 2024-03-09T09:28:02.548 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2sw60f-t_notab’: No data available 2024-03-09T09:28:02.550 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/W-72l24f-ll' -> './k/coreutils-8.5/tests/pr/W-72l24f-ll' 2024-03-09T09:28:02.550 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W-72l24f-ll’: No data available 2024-03-09T09:28:02.552 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tt-0FF' -> './k/coreutils-8.5/tests/pr/tt-0FF' 2024-03-09T09:28:02.552 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tt-0FF’: No data available 2024-03-09T09:28:02.555 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/4l24-FF' -> './k/coreutils-8.5/tests/pr/4l24-FF' 2024-03-09T09:28:02.555 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/4l24-FF’: No data available 2024-03-09T09:28:02.556 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/3a3f-0F' -> './k/coreutils-8.5/tests/pr/3a3f-0F' 2024-03-09T09:28:02.557 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3a3f-0F’: No data available 2024-03-09T09:28:02.558 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tFFt' -> './k/coreutils-8.5/tests/pr/tFFt' 2024-03-09T09:28:02.558 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFFt’: No data available 2024-03-09T09:28:02.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.556+0000 7fd36d600640 1 -- 172.21.15.67:0/812422578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fd368075a00 con 0x7fd3680ac730 2024-03-09T09:28:02.564 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/n+6a2l17-FF' -> './k/coreutils-8.5/tests/pr/n+6a2l17-FF' 2024-03-09T09:28:02.564 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+6a2l17-FF’: No data available 2024-03-09T09:28:02.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.557+0000 7fd35f400640 1 -- 172.21.15.67:0/812422578 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fd358057c90 con 0x7fd3680ac730 2024-03-09T09:28:02.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.560+0000 7fd36d600640 1 -- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd34406b500 msgr2=0x7fd34406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:02.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.560+0000 7fd36d600640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd34406b500 0x7fd34406d9c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd3680c1500 tx=0x7fd350008040 comp rx=0 tx=0).stop 2024-03-09T09:28:02.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.560+0000 7fd36d600640 1 -- 172.21.15.67:0/812422578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3680ac730 msgr2=0x7fd36814e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:02.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.560+0000 7fd36d600640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3680ac730 0x7fd36814e790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd35800c4d0 tx=0x7fd35800c990 comp rx=0 tx=0).stop 2024-03-09T09:28:02.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.561+0000 7fd36d600640 1 -- 172.21.15.67:0/812422578 shutdown_connections 2024-03-09T09:28:02.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.561+0000 7fd36d600640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd34406b500 0x7fd34406d9c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:02.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.561+0000 7fd36d600640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd368157d60 0x7fd3681ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:02.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.561+0000 7fd36d600640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3680c22a0 0x7fd3680abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:02.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.561+0000 7fd36d600640 1 --2- 172.21.15.67:0/812422578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3680ac730 0x7fd36814e790 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:02.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.561+0000 7fd36d600640 1 -- 172.21.15.67:0/812422578 >> 172.21.15.67:0/812422578 conn(0x7fd3680c7370 msgr2=0x7fd368153040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:02.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.561+0000 7fd36d600640 1 -- 172.21.15.67:0/812422578 shutdown_connections 2024-03-09T09:28:02.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:02.561+0000 7fd36d600640 1 -- 172.21.15.67:0/812422578 wait complete. 2024-03-09T09:28:02.586 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:28:03.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:28:03.191 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:28:03.549 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/a3l15f-t' -> './k/coreutils-8.5/tests/pr/a3l15f-t' 2024-03-09T09:28:03.550 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3l15f-t’: No data available 2024-03-09T09:28:03.551 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/l24f-t' -> './k/coreutils-8.5/tests/pr/l24f-t' 2024-03-09T09:28:03.551 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/l24f-t’: No data available 2024-03-09T09:28:03.552 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/b3f-0FF' -> './k/coreutils-8.5/tests/pr/b3f-0FF' 2024-03-09T09:28:03.553 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3f-0FF’: No data available 2024-03-09T09:28:03.555 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tt-t' -> './k/coreutils-8.5/tests/pr/tt-t' 2024-03-09T09:28:03.555 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tt-t’: No data available 2024-03-09T09:28:03.557 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/ttb3-FF' -> './k/coreutils-8.5/tests/pr/ttb3-FF' 2024-03-09T09:28:03.557 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ttb3-FF’: No data available 2024-03-09T09:28:03.559 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml20-t-t-FF' -> './k/coreutils-8.5/tests/pr/nSml20-t-t-FF' 2024-03-09T09:28:03.559 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml20-t-t-FF’: No data available 2024-03-09T09:28:03.561 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tne8o3-t_tab' -> './k/coreutils-8.5/tests/pr/tne8o3-t_tab' 2024-03-09T09:28:03.561 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tne8o3-t_tab’: No data available 2024-03-09T09:28:03.563 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/pr/tt-FF' -> './k/coreutils-8.5/tests/pr/tt-FF' 2024-03-09T09:28:03.563 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tt-FF’: No data available 2024-03-09T09:28:03.955 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/envvar-check' -> './k/coreutils-8.5/tests/envvar-check' 2024-03-09T09:28:03.955 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/envvar-check’: No data available 2024-03-09T09:28:03.958 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/Coreutils.pm' -> './k/coreutils-8.5/tests/Coreutils.pm' 2024-03-09T09:28:03.958 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/Coreutils.pm’: No data available 2024-03-09T09:28:03.969 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rmdir' -> './k/coreutils-8.5/tests/rmdir' 2024-03-09T09:28:03.969 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rmdir/ignore' -> './k/coreutils-8.5/tests/rmdir/ignore' 2024-03-09T09:28:03.969 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rmdir/ignore’: No data available 2024-03-09T09:28:03.972 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rmdir/fail-perm' -> './k/coreutils-8.5/tests/rmdir/fail-perm' 2024-03-09T09:28:03.972 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rmdir/fail-perm’: No data available 2024-03-09T09:28:03.974 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rmdir/t-slash' -> './k/coreutils-8.5/tests/rmdir/t-slash' 2024-03-09T09:28:03.974 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rmdir/t-slash’: No data available 2024-03-09T09:28:04.009 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm' -> './k/coreutils-8.5/tests/rm' 2024-03-09T09:28:04.009 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/r-2' -> './k/coreutils-8.5/tests/rm/r-2' 2024-03-09T09:28:04.009 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/r-2’: No data available 2024-03-09T09:28:04.011 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/fail-2eperm' -> './k/coreutils-8.5/tests/rm/fail-2eperm' 2024-03-09T09:28:04.011 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/fail-2eperm’: No data available 2024-03-09T09:28:04.014 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/empty-inacc' -> './k/coreutils-8.5/tests/rm/empty-inacc' 2024-03-09T09:28:04.014 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/empty-inacc’: No data available 2024-03-09T09:28:04.016 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/cycle' -> './k/coreutils-8.5/tests/rm/cycle' 2024-03-09T09:28:04.016 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/cycle’: No data available 2024-03-09T09:28:04.018 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/sunos-1' -> './k/coreutils-8.5/tests/rm/sunos-1' 2024-03-09T09:28:04.018 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/sunos-1’: No data available 2024-03-09T09:28:04.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/v-slash' -> './k/coreutils-8.5/tests/rm/v-slash' 2024-03-09T09:28:04.021 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/v-slash’: No data available 2024-03-09T09:28:04.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/no-give-up' -> './k/coreutils-8.5/tests/rm/no-give-up' 2024-03-09T09:28:04.023 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/no-give-up’: No data available 2024-03-09T09:28:04.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/dot-rel' -> './k/coreutils-8.5/tests/rm/dot-rel' 2024-03-09T09:28:04.026 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/dot-rel’: No data available 2024-03-09T09:28:04.027 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/one-file-system' -> './k/coreutils-8.5/tests/rm/one-file-system' 2024-03-09T09:28:04.028 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/one-file-system’: No data available 2024-03-09T09:28:04.030 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/i-never' -> './k/coreutils-8.5/tests/rm/i-never' 2024-03-09T09:28:04.030 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/i-never’: No data available 2024-03-09T09:28:04.032 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/rm1' -> './k/coreutils-8.5/tests/rm/rm1' 2024-03-09T09:28:04.033 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/rm1’: No data available 2024-03-09T09:28:04.035 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/unread2' -> './k/coreutils-8.5/tests/rm/unread2' 2024-03-09T09:28:04.035 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/unread2’: No data available 2024-03-09T09:28:04.037 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/read-only' -> './k/coreutils-8.5/tests/rm/read-only' 2024-03-09T09:28:04.038 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/read-only’: No data available 2024-03-09T09:28:04.040 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/deep-2' -> './k/coreutils-8.5/tests/rm/deep-2' 2024-03-09T09:28:04.040 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/deep-2’: No data available 2024-03-09T09:28:04.042 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/r-4' -> './k/coreutils-8.5/tests/rm/r-4' 2024-03-09T09:28:04.043 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/r-4’: No data available 2024-03-09T09:28:04.045 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/dir-no-w' -> './k/coreutils-8.5/tests/rm/dir-no-w' 2024-03-09T09:28:04.046 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/dir-no-w’: No data available 2024-03-09T09:28:04.048 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/interactive-always' -> './k/coreutils-8.5/tests/rm/interactive-always' 2024-03-09T09:28:04.049 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/interactive-always’: No data available 2024-03-09T09:28:04.051 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/one-file-system2' -> './k/coreutils-8.5/tests/rm/one-file-system2' 2024-03-09T09:28:04.051 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/one-file-system2’: No data available 2024-03-09T09:28:04.054 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/unreadable' -> './k/coreutils-8.5/tests/rm/unreadable' 2024-03-09T09:28:04.054 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/unreadable’: No data available 2024-03-09T09:28:04.056 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/r-1' -> './k/coreutils-8.5/tests/rm/r-1' 2024-03-09T09:28:04.056 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/r-1’: No data available 2024-03-09T09:28:04.057 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strtol.m4' -> './k/coreutils-8.5/m4/strtol.m4' 2024-03-09T09:28:04.058 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtol.m4’: No data available 2024-03-09T09:28:04.059 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/fail-eacces' -> './k/coreutils-8.5/tests/rm/fail-eacces' 2024-03-09T09:28:04.059 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/fail-eacces’: No data available 2024-03-09T09:28:04.060 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/lib-ignore.m4' -> './k/coreutils-8.5/m4/lib-ignore.m4' 2024-03-09T09:28:04.061 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lib-ignore.m4’: No data available 2024-03-09T09:28:04.061 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/inaccessible' -> './k/coreutils-8.5/tests/rm/inaccessible' 2024-03-09T09:28:04.062 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/inaccessible’: No data available 2024-03-09T09:28:04.063 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/readutmp.m4' -> './k/coreutils-8.5/m4/readutmp.m4' 2024-03-09T09:28:04.063 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/readutmp.m4’: No data available 2024-03-09T09:28:04.064 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/unread3' -> './k/coreutils-8.5/tests/rm/unread3' 2024-03-09T09:28:04.064 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/unread3’: No data available 2024-03-09T09:28:04.065 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strtoll.m4' -> './k/coreutils-8.5/m4/strtoll.m4' 2024-03-09T09:28:04.066 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtoll.m4’: No data available 2024-03-09T09:28:04.068 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/ir-1' -> './k/coreutils-8.5/tests/rm/ir-1' 2024-03-09T09:28:04.068 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/ir-1’: No data available 2024-03-09T09:28:04.068 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stdbool.m4' -> './k/coreutils-8.5/m4/stdbool.m4' 2024-03-09T09:28:04.069 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdbool.m4’: No data available 2024-03-09T09:28:04.070 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/setenv.m4' -> './k/coreutils-8.5/m4/setenv.m4' 2024-03-09T09:28:04.070 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/setenv.m4’: No data available 2024-03-09T09:28:04.071 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/rm3' -> './k/coreutils-8.5/tests/rm/rm3' 2024-03-09T09:28:04.071 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/rm3’: No data available 2024-03-09T09:28:04.072 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/alloca.m4' -> './k/coreutils-8.5/m4/alloca.m4' 2024-03-09T09:28:04.072 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/alloca.m4’: No data available 2024-03-09T09:28:04.073 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/rm5' -> './k/coreutils-8.5/tests/rm/rm5' 2024-03-09T09:28:04.073 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/rm5’: No data available 2024-03-09T09:28:04.074 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/size_max.m4' -> './k/coreutils-8.5/m4/size_max.m4' 2024-03-09T09:28:04.074 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/size_max.m4’: No data available 2024-03-09T09:28:04.075 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/deep-1' -> './k/coreutils-8.5/tests/rm/deep-1' 2024-03-09T09:28:04.075 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/deep-1’: No data available 2024-03-09T09:28:04.076 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/include_next.m4' -> './k/coreutils-8.5/m4/include_next.m4' 2024-03-09T09:28:04.077 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/include_next.m4’: No data available 2024-03-09T09:28:04.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/ext3-perf' -> './k/coreutils-8.5/tests/rm/ext3-perf' 2024-03-09T09:28:04.078 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/ext3-perf’: No data available 2024-03-09T09:28:04.078 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sys_stat_h.m4' -> './k/coreutils-8.5/m4/sys_stat_h.m4' 2024-03-09T09:28:04.079 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_stat_h.m4’: No data available 2024-03-09T09:28:04.080 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strndup.m4' -> './k/coreutils-8.5/m4/strndup.m4' 2024-03-09T09:28:04.080 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strndup.m4’: No data available 2024-03-09T09:28:04.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/dangling-symlink' -> './k/coreutils-8.5/tests/rm/dangling-symlink' 2024-03-09T09:28:04.081 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/dangling-symlink’: No data available 2024-03-09T09:28:04.082 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/link-follow.m4' -> './k/coreutils-8.5/m4/link-follow.m4' 2024-03-09T09:28:04.082 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/link-follow.m4’: No data available 2024-03-09T09:28:04.083 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/isatty' -> './k/coreutils-8.5/tests/rm/isatty' 2024-03-09T09:28:04.083 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/isatty’: No data available 2024-03-09T09:28:04.083 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/ctype.m4' -> './k/coreutils-8.5/m4/ctype.m4' 2024-03-09T09:28:04.084 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ctype.m4’: No data available 2024-03-09T09:28:04.084 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/timespec.m4' -> './k/coreutils-8.5/m4/timespec.m4' 2024-03-09T09:28:04.085 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/timespec.m4’: No data available 2024-03-09T09:28:04.085 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/i-1' -> './k/coreutils-8.5/tests/rm/i-1' 2024-03-09T09:28:04.085 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/i-1’: No data available 2024-03-09T09:28:04.086 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getline.m4' -> './k/coreutils-8.5/m4/getline.m4' 2024-03-09T09:28:04.086 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getline.m4’: No data available 2024-03-09T09:28:04.087 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/dir-nonrecur' -> './k/coreutils-8.5/tests/rm/dir-nonrecur' 2024-03-09T09:28:04.088 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/dir-nonrecur’: No data available 2024-03-09T09:28:04.089 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/calloc.m4' -> './k/coreutils-8.5/m4/calloc.m4' 2024-03-09T09:28:04.089 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/calloc.m4’: No data available 2024-03-09T09:28:04.090 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/interactive-once' -> './k/coreutils-8.5/tests/rm/interactive-once' 2024-03-09T09:28:04.090 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/interactive-once’: No data available 2024-03-09T09:28:04.091 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strftime.m4' -> './k/coreutils-8.5/m4/strftime.m4' 2024-03-09T09:28:04.091 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strftime.m4’: No data available 2024-03-09T09:28:04.091 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/f-1' -> './k/coreutils-8.5/tests/rm/f-1' 2024-03-09T09:28:04.092 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/f-1’: No data available 2024-03-09T09:28:04.094 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/fail-eperm' -> './k/coreutils-8.5/tests/rm/fail-eperm' 2024-03-09T09:28:04.094 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/arpa_inet_h.m4' -> './k/coreutils-8.5/m4/arpa_inet_h.m4' 2024-03-09T09:28:04.095 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/fail-eperm’: No data available 2024-03-09T09:28:04.095 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/arpa_inet_h.m4’: No data available 2024-03-09T09:28:04.095 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sig2str.m4' -> './k/coreutils-8.5/m4/sig2str.m4' 2024-03-09T09:28:04.095 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sig2str.m4’: No data available 2024-03-09T09:28:04.096 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/printf.m4' -> './k/coreutils-8.5/m4/printf.m4' 2024-03-09T09:28:04.097 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/printf.m4’: No data available 2024-03-09T09:28:04.097 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/empty-name' -> './k/coreutils-8.5/tests/rm/empty-name' 2024-03-09T09:28:04.097 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/empty-name’: No data available 2024-03-09T09:28:04.098 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/environ.m4' -> './k/coreutils-8.5/m4/environ.m4' 2024-03-09T09:28:04.098 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/environ.m4’: No data available 2024-03-09T09:28:04.099 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/vprintf-posix.m4' -> './k/coreutils-8.5/m4/vprintf-posix.m4' 2024-03-09T09:28:04.099 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/vprintf-posix.m4’: No data available 2024-03-09T09:28:04.100 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/hash' -> './k/coreutils-8.5/tests/rm/hash' 2024-03-09T09:28:04.100 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/hash’: No data available 2024-03-09T09:28:04.100 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/intlmacosx.m4' -> './k/coreutils-8.5/m4/intlmacosx.m4' 2024-03-09T09:28:04.101 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/intlmacosx.m4’: No data available 2024-03-09T09:28:04.102 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/i-no-r' -> './k/coreutils-8.5/tests/rm/i-no-r' 2024-03-09T09:28:04.103 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/i-no-r’: No data available 2024-03-09T09:28:04.103 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/locale-tr.m4' -> './k/coreutils-8.5/m4/locale-tr.m4' 2024-03-09T09:28:04.104 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/locale-tr.m4’: No data available 2024-03-09T09:28:04.104 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mktime.m4' -> './k/coreutils-8.5/m4/mktime.m4' 2024-03-09T09:28:04.104 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mktime.m4’: No data available 2024-03-09T09:28:04.105 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/r-3' -> './k/coreutils-8.5/tests/rm/r-3' 2024-03-09T09:28:04.106 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/r-3’: No data available 2024-03-09T09:28:04.106 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/iconv.m4' -> './k/coreutils-8.5/m4/iconv.m4' 2024-03-09T09:28:04.107 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/iconv.m4’: No data available 2024-03-09T09:28:04.108 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/pathmax.m4' -> './k/coreutils-8.5/m4/pathmax.m4' 2024-03-09T09:28:04.108 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/pathmax.m4’: No data available 2024-03-09T09:28:04.109 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/readdir-bug' -> './k/coreutils-8.5/tests/rm/readdir-bug' 2024-03-09T09:28:04.109 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/readdir-bug’: No data available 2024-03-09T09:28:04.110 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/printf-frexp.m4' -> './k/coreutils-8.5/m4/printf-frexp.m4' 2024-03-09T09:28:04.110 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/printf-frexp.m4’: No data available 2024-03-09T09:28:04.110 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/userspec.m4' -> './k/coreutils-8.5/m4/userspec.m4' 2024-03-09T09:28:04.111 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/userspec.m4’: No data available 2024-03-09T09:28:04.111 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/rm2' -> './k/coreutils-8.5/tests/rm/rm2' 2024-03-09T09:28:04.112 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/rm2’: No data available 2024-03-09T09:28:04.112 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/host-os.m4' -> './k/coreutils-8.5/m4/host-os.m4' 2024-03-09T09:28:04.113 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/host-os.m4’: No data available 2024-03-09T09:28:04.113 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/file-type.m4' -> './k/coreutils-8.5/m4/file-type.m4' 2024-03-09T09:28:04.113 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/file-type.m4’: No data available 2024-03-09T09:28:04.114 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/rm4' -> './k/coreutils-8.5/tests/rm/rm4' 2024-03-09T09:28:04.114 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/rm4’: No data available 2024-03-09T09:28:04.115 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/ls-mntd-fs.m4' -> './k/coreutils-8.5/m4/ls-mntd-fs.m4' 2024-03-09T09:28:04.115 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ls-mntd-fs.m4’: No data available 2024-03-09T09:28:04.116 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sha512.m4' -> './k/coreutils-8.5/m4/sha512.m4' 2024-03-09T09:28:04.116 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sha512.m4’: No data available 2024-03-09T09:28:04.117 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/rm/ignorable' -> './k/coreutils-8.5/tests/rm/ignorable' 2024-03-09T09:28:04.117 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/ignorable’: No data available 2024-03-09T09:28:04.118 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stdio-safer.m4' -> './k/coreutils-8.5/m4/stdio-safer.m4' 2024-03-09T09:28:04.118 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdio-safer.m4’: No data available 2024-03-09T09:28:04.119 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/lock.m4' -> './k/coreutils-8.5/m4/lock.m4' 2024-03-09T09:28:04.120 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lock.m4’: No data available 2024-03-09T09:28:04.120 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/c-strtod.m4' -> './k/coreutils-8.5/m4/c-strtod.m4' 2024-03-09T09:28:04.121 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/c-strtod.m4’: No data available 2024-03-09T09:28:04.121 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/closein.m4' -> './k/coreutils-8.5/m4/closein.m4' 2024-03-09T09:28:04.122 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/closein.m4’: No data available 2024-03-09T09:28:04.123 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/frexpl.m4' -> './k/coreutils-8.5/m4/frexpl.m4' 2024-03-09T09:28:04.124 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/frexpl.m4’: No data available 2024-03-09T09:28:04.125 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/posixtm.m4' -> './k/coreutils-8.5/m4/posixtm.m4' 2024-03-09T09:28:04.125 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/posixtm.m4’: No data available 2024-03-09T09:28:04.127 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/rmdir.m4' -> './k/coreutils-8.5/m4/rmdir.m4' 2024-03-09T09:28:04.127 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/rmdir.m4’: No data available 2024-03-09T09:28:04.129 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/chown.m4' -> './k/coreutils-8.5/m4/chown.m4' 2024-03-09T09:28:04.129 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/chown.m4’: No data available 2024-03-09T09:28:04.131 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stdint.m4' -> './k/coreutils-8.5/m4/stdint.m4' 2024-03-09T09:28:04.131 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdint.m4’: No data available 2024-03-09T09:28:04.132 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stddef_h.m4' -> './k/coreutils-8.5/m4/stddef_h.m4' 2024-03-09T09:28:04.133 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stddef_h.m4’: No data available 2024-03-09T09:28:04.134 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sigaction.m4' -> './k/coreutils-8.5/m4/sigaction.m4' 2024-03-09T09:28:04.134 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sigaction.m4’: No data available 2024-03-09T09:28:04.135 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/error.m4' -> './k/coreutils-8.5/m4/error.m4' 2024-03-09T09:28:04.135 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/error.m4’: No data available 2024-03-09T09:28:04.136 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/vasnprintf.m4' -> './k/coreutils-8.5/m4/vasnprintf.m4' 2024-03-09T09:28:04.137 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/vasnprintf.m4’: No data available 2024-03-09T09:28:04.138 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/utimbuf.m4' -> './k/coreutils-8.5/m4/utimbuf.m4' 2024-03-09T09:28:04.138 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/utimbuf.m4’: No data available 2024-03-09T09:28:04.139 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sockets.m4' -> './k/coreutils-8.5/m4/sockets.m4' 2024-03-09T09:28:04.139 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sockets.m4’: No data available 2024-03-09T09:28:04.140 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/canonicalize.m4' -> './k/coreutils-8.5/m4/canonicalize.m4' 2024-03-09T09:28:04.140 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/canonicalize.m4’: No data available 2024-03-09T09:28:04.142 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sys_time_h.m4' -> './k/coreutils-8.5/m4/sys_time_h.m4' 2024-03-09T09:28:04.142 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_time_h.m4’: No data available 2024-03-09T09:28:04.143 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/unlink-busy.m4' -> './k/coreutils-8.5/m4/unlink-busy.m4' 2024-03-09T09:28:04.144 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unlink-busy.m4’: No data available 2024-03-09T09:28:04.145 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/gettimeofday.m4' -> './k/coreutils-8.5/m4/gettimeofday.m4' 2024-03-09T09:28:04.145 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gettimeofday.m4’: No data available 2024-03-09T09:28:04.146 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/write-any-file.m4' -> './k/coreutils-8.5/m4/write-any-file.m4' 2024-03-09T09:28:04.146 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/write-any-file.m4’: No data available 2024-03-09T09:28:04.147 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strdup.m4' -> './k/coreutils-8.5/m4/strdup.m4' 2024-03-09T09:28:04.147 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strdup.m4’: No data available 2024-03-09T09:28:04.148 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/regex.m4' -> './k/coreutils-8.5/m4/regex.m4' 2024-03-09T09:28:04.148 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/regex.m4’: No data available 2024-03-09T09:28:04.149 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sleep.m4' -> './k/coreutils-8.5/m4/sleep.m4' 2024-03-09T09:28:04.150 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sleep.m4’: No data available 2024-03-09T09:28:04.150 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/float_h.m4' -> './k/coreutils-8.5/m4/float_h.m4' 2024-03-09T09:28:04.150 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/float_h.m4’: No data available 2024-03-09T09:28:04.152 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/warn-on-use.m4' -> './k/coreutils-8.5/m4/warn-on-use.m4' 2024-03-09T09:28:04.152 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/warn-on-use.m4’: No data available 2024-03-09T09:28:04.153 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/atexit.m4' -> './k/coreutils-8.5/m4/atexit.m4' 2024-03-09T09:28:04.153 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/atexit.m4’: No data available 2024-03-09T09:28:04.154 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getcwd-path-max.m4' -> './k/coreutils-8.5/m4/getcwd-path-max.m4' 2024-03-09T09:28:04.154 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getcwd-path-max.m4’: No data available 2024-03-09T09:28:04.155 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fnmatch.m4' -> './k/coreutils-8.5/m4/fnmatch.m4' 2024-03-09T09:28:04.155 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fnmatch.m4’: No data available 2024-03-09T09:28:04.156 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strnumcmp.m4' -> './k/coreutils-8.5/m4/strnumcmp.m4' 2024-03-09T09:28:04.156 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strnumcmp.m4’: No data available 2024-03-09T09:28:04.157 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/gnulib-common.m4' -> './k/coreutils-8.5/m4/gnulib-common.m4' 2024-03-09T09:28:04.157 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gnulib-common.m4’: No data available 2024-03-09T09:28:04.157 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/codeset.m4' -> './k/coreutils-8.5/m4/codeset.m4' 2024-03-09T09:28:04.158 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/codeset.m4’: No data available 2024-03-09T09:28:04.159 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/time_r.m4' -> './k/coreutils-8.5/m4/time_r.m4' 2024-03-09T09:28:04.159 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/time_r.m4’: No data available 2024-03-09T09:28:04.160 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/memcoll.m4' -> './k/coreutils-8.5/m4/memcoll.m4' 2024-03-09T09:28:04.160 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memcoll.m4’: No data available 2024-03-09T09:28:04.161 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/string_h.m4' -> './k/coreutils-8.5/m4/string_h.m4' 2024-03-09T09:28:04.161 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/string_h.m4’: No data available 2024-03-09T09:28:04.162 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sys_utsname_h.m4' -> './k/coreutils-8.5/m4/sys_utsname_h.m4' 2024-03-09T09:28:04.162 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sys_utsname_h.m4’: No data available 2024-03-09T09:28:04.163 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stdint_h.m4' -> './k/coreutils-8.5/m4/stdint_h.m4' 2024-03-09T09:28:04.163 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdint_h.m4’: No data available 2024-03-09T09:28:04.164 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/nproc.m4' -> './k/coreutils-8.5/m4/nproc.m4' 2024-03-09T09:28:04.165 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/nproc.m4’: No data available 2024-03-09T09:28:04.166 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/symlink.m4' -> './k/coreutils-8.5/m4/symlink.m4' 2024-03-09T09:28:04.166 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/symlink.m4’: No data available 2024-03-09T09:28:04.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/rename.m4' -> './k/coreutils-8.5/m4/rename.m4' 2024-03-09T09:28:04.167 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/rename.m4’: No data available 2024-03-09T09:28:04.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/read-file.m4' -> './k/coreutils-8.5/m4/read-file.m4' 2024-03-09T09:28:04.169 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/read-file.m4’: No data available 2024-03-09T09:28:04.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strtoumax.m4' -> './k/coreutils-8.5/m4/strtoumax.m4' 2024-03-09T09:28:04.170 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtoumax.m4’: No data available 2024-03-09T09:28:04.171 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getopt.m4' -> './k/coreutils-8.5/m4/getopt.m4' 2024-03-09T09:28:04.172 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getopt.m4’: No data available 2024-03-09T09:28:04.172 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fopen.m4' -> './k/coreutils-8.5/m4/fopen.m4' 2024-03-09T09:28:04.173 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fopen.m4’: No data available 2024-03-09T09:28:04.174 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/ungetc.m4' -> './k/coreutils-8.5/m4/ungetc.m4' 2024-03-09T09:28:04.174 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ungetc.m4’: No data available 2024-03-09T09:28:04.175 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/xsize.m4' -> './k/coreutils-8.5/m4/xsize.m4' 2024-03-09T09:28:04.175 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xsize.m4’: No data available 2024-03-09T09:28:04.176 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fflush.m4' -> './k/coreutils-8.5/m4/fflush.m4' 2024-03-09T09:28:04.176 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fflush.m4’: No data available 2024-03-09T09:28:04.177 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/acl.m4' -> './k/coreutils-8.5/m4/acl.m4' 2024-03-09T09:28:04.177 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/acl.m4’: No data available 2024-03-09T09:28:04.179 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fsusage.m4' -> './k/coreutils-8.5/m4/fsusage.m4' 2024-03-09T09:28:04.179 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fsusage.m4’: No data available 2024-03-09T09:28:04.180 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/gethrxtime.m4' -> './k/coreutils-8.5/m4/gethrxtime.m4' 2024-03-09T09:28:04.180 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gethrxtime.m4’: No data available 2024-03-09T09:28:04.181 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/nls.m4' -> './k/coreutils-8.5/m4/nls.m4' 2024-03-09T09:28:04.181 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/nls.m4’: No data available 2024-03-09T09:28:04.182 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/lib-link.m4' -> './k/coreutils-8.5/m4/lib-link.m4' 2024-03-09T09:28:04.182 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lib-link.m4’: No data available 2024-03-09T09:28:04.183 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mpsort.m4' -> './k/coreutils-8.5/m4/mpsort.m4' 2024-03-09T09:28:04.183 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mpsort.m4’: No data available 2024-03-09T09:28:04.184 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/netinet_in_h.m4' -> './k/coreutils-8.5/m4/netinet_in_h.m4' 2024-03-09T09:28:04.185 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/netinet_in_h.m4’: No data available 2024-03-09T09:28:04.185 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fseeko.m4' -> './k/coreutils-8.5/m4/fseeko.m4' 2024-03-09T09:28:04.185 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fseeko.m4’: No data available 2024-03-09T09:28:04.186 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/inet_pton.m4' -> './k/coreutils-8.5/m4/inet_pton.m4' 2024-03-09T09:28:04.186 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inet_pton.m4’: No data available 2024-03-09T09:28:04.187 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/yesno.m4' -> './k/coreutils-8.5/m4/yesno.m4' 2024-03-09T09:28:04.188 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/yesno.m4’: No data available 2024-03-09T09:28:04.189 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/physmem.m4' -> './k/coreutils-8.5/m4/physmem.m4' 2024-03-09T09:28:04.189 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/physmem.m4’: No data available 2024-03-09T09:28:04.191 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/intmax_t.m4' -> './k/coreutils-8.5/m4/intmax_t.m4' 2024-03-09T09:28:04.191 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/intmax_t.m4’: No data available 2024-03-09T09:28:04.192 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/inttypes-pri.m4' -> './k/coreutils-8.5/m4/inttypes-pri.m4' 2024-03-09T09:28:04.192 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inttypes-pri.m4’: No data available 2024-03-09T09:28:04.194 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/malloc.m4' -> './k/coreutils-8.5/m4/malloc.m4' 2024-03-09T09:28:04.194 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/malloc.m4’: No data available 2024-03-09T09:28:04.195 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/jm-winsz1.m4' -> './k/coreutils-8.5/m4/jm-winsz1.m4' 2024-03-09T09:28:04.195 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/jm-winsz1.m4’: No data available 2024-03-09T09:28:04.198 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/close-stream.m4' -> './k/coreutils-8.5/m4/close-stream.m4' 2024-03-09T09:28:04.198 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/close-stream.m4’: No data available 2024-03-09T09:28:04.199 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/nocrash.m4' -> './k/coreutils-8.5/m4/nocrash.m4' 2024-03-09T09:28:04.199 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/nocrash.m4’: No data available 2024-03-09T09:28:04.200 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/realloc.m4' -> './k/coreutils-8.5/m4/realloc.m4' 2024-03-09T09:28:04.200 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/realloc.m4’: No data available 2024-03-09T09:28:04.201 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/canon-host.m4' -> './k/coreutils-8.5/m4/canon-host.m4' 2024-03-09T09:28:04.201 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/canon-host.m4’: No data available 2024-03-09T09:28:04.203 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/memcpy.m4' -> './k/coreutils-8.5/m4/memcpy.m4' 2024-03-09T09:28:04.203 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memcpy.m4’: No data available 2024-03-09T09:28:04.204 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/tls.m4' -> './k/coreutils-8.5/m4/tls.m4' 2024-03-09T09:28:04.204 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/tls.m4’: No data available 2024-03-09T09:28:04.206 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/perl.m4' -> './k/coreutils-8.5/m4/perl.m4' 2024-03-09T09:28:04.206 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/perl.m4’: No data available 2024-03-09T09:28:04.208 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/signbit.m4' -> './k/coreutils-8.5/m4/signbit.m4' 2024-03-09T09:28:04.208 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/signbit.m4’: No data available 2024-03-09T09:28:04.210 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/memchr.m4' -> './k/coreutils-8.5/m4/memchr.m4' 2024-03-09T09:28:04.210 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memchr.m4’: No data available 2024-03-09T09:28:04.211 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/d-type.m4' -> './k/coreutils-8.5/m4/d-type.m4' 2024-03-09T09:28:04.211 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/d-type.m4’: No data available 2024-03-09T09:28:04.212 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/lchown.m4' -> './k/coreutils-8.5/m4/lchown.m4' 2024-03-09T09:28:04.212 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lchown.m4’: No data available 2024-03-09T09:28:04.214 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/lib-prefix.m4' -> './k/coreutils-8.5/m4/lib-prefix.m4' 2024-03-09T09:28:04.214 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lib-prefix.m4’: No data available 2024-03-09T09:28:04.215 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getcwd.m4' -> './k/coreutils-8.5/m4/getcwd.m4' 2024-03-09T09:28:04.215 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getcwd.m4’: No data available 2024-03-09T09:28:04.217 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/locale_h.m4' -> './k/coreutils-8.5/m4/locale_h.m4' 2024-03-09T09:28:04.217 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/locale_h.m4’: No data available 2024-03-09T09:28:04.218 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/memxfrm.m4' -> './k/coreutils-8.5/m4/memxfrm.m4' 2024-03-09T09:28:04.218 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memxfrm.m4’: No data available 2024-03-09T09:28:04.219 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mbstate_t.m4' -> './k/coreutils-8.5/m4/mbstate_t.m4' 2024-03-09T09:28:04.220 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbstate_t.m4’: No data available 2024-03-09T09:28:04.221 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/filemode.m4' -> './k/coreutils-8.5/m4/filemode.m4' 2024-03-09T09:28:04.221 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/filemode.m4’: No data available 2024-03-09T09:28:04.222 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/memset.m4' -> './k/coreutils-8.5/m4/memset.m4' 2024-03-09T09:28:04.223 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memset.m4’: No data available 2024-03-09T09:28:04.224 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/gethostname.m4' -> './k/coreutils-8.5/m4/gethostname.m4' 2024-03-09T09:28:04.224 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gethostname.m4’: No data available 2024-03-09T09:28:04.226 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/vasprintf.m4' -> './k/coreutils-8.5/m4/vasprintf.m4' 2024-03-09T09:28:04.226 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/vasprintf.m4’: No data available 2024-03-09T09:28:04.227 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/unistd-safer.m4' -> './k/coreutils-8.5/m4/unistd-safer.m4' 2024-03-09T09:28:04.228 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unistd-safer.m4’: No data available 2024-03-09T09:28:04.234 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/xfts.m4' -> './k/coreutils-8.5/m4/xfts.m4' 2024-03-09T09:28:04.235 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xfts.m4’: No data available 2024-03-09T09:28:04.236 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/uptime.m4' -> './k/coreutils-8.5/m4/uptime.m4' 2024-03-09T09:28:04.236 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/uptime.m4’: No data available 2024-03-09T09:28:04.237 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/prereq.m4' -> './k/coreutils-8.5/m4/prereq.m4' 2024-03-09T09:28:04.237 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/prereq.m4’: No data available 2024-03-09T09:28:04.238 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/symlinkat.m4' -> './k/coreutils-8.5/m4/symlinkat.m4' 2024-03-09T09:28:04.239 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/symlinkat.m4’: No data available 2024-03-09T09:28:04.240 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getusershell.m4' -> './k/coreutils-8.5/m4/getusershell.m4' 2024-03-09T09:28:04.240 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getusershell.m4’: No data available 2024-03-09T09:28:04.241 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/isnand.m4' -> './k/coreutils-8.5/m4/isnand.m4' 2024-03-09T09:28:04.241 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/isnand.m4’: No data available 2024-03-09T09:28:04.243 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/select.m4' -> './k/coreutils-8.5/m4/select.m4' 2024-03-09T09:28:04.243 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/select.m4’: No data available 2024-03-09T09:28:04.244 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/wctob.m4' -> './k/coreutils-8.5/m4/wctob.m4' 2024-03-09T09:28:04.244 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wctob.m4’: No data available 2024-03-09T09:28:04.245 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/double-slash-root.m4' -> './k/coreutils-8.5/m4/double-slash-root.m4' 2024-03-09T09:28:04.245 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/double-slash-root.m4’: No data available 2024-03-09T09:28:04.246 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/long-options.m4' -> './k/coreutils-8.5/m4/long-options.m4' 2024-03-09T09:28:04.247 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/long-options.m4’: No data available 2024-03-09T09:28:04.248 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/glibc21.m4' -> './k/coreutils-8.5/m4/glibc21.m4' 2024-03-09T09:28:04.248 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/glibc21.m4’: No data available 2024-03-09T09:28:04.249 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stat-time.m4' -> './k/coreutils-8.5/m4/stat-time.m4' 2024-03-09T09:28:04.249 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stat-time.m4’: No data available 2024-03-09T09:28:04.250 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/rpmatch.m4' -> './k/coreutils-8.5/m4/rpmatch.m4' 2024-03-09T09:28:04.250 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/rpmatch.m4’: No data available 2024-03-09T09:28:04.251 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/xattr.m4' -> './k/coreutils-8.5/m4/xattr.m4' 2024-03-09T09:28:04.251 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xattr.m4’: No data available 2024-03-09T09:28:04.252 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/gettext.m4' -> './k/coreutils-8.5/m4/gettext.m4' 2024-03-09T09:28:04.252 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/gettext.m4’: No data available 2024-03-09T09:28:04.253 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fsync.m4' -> './k/coreutils-8.5/m4/fsync.m4' 2024-03-09T09:28:04.253 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fsync.m4’: No data available 2024-03-09T09:28:04.254 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/exclude.m4' -> './k/coreutils-8.5/m4/exclude.m4' 2024-03-09T09:28:04.254 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/exclude.m4’: No data available 2024-03-09T09:28:04.255 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/freopen.m4' -> './k/coreutils-8.5/m4/freopen.m4' 2024-03-09T09:28:04.255 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/freopen.m4’: No data available 2024-03-09T09:28:04.256 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strtoul.m4' -> './k/coreutils-8.5/m4/strtoul.m4' 2024-03-09T09:28:04.256 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtoul.m4’: No data available 2024-03-09T09:28:04.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/netdb_h.m4' -> './k/coreutils-8.5/m4/netdb_h.m4' 2024-03-09T09:28:04.257 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/netdb_h.m4’: No data available 2024-03-09T09:28:04.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/snprintf.m4' -> './k/coreutils-8.5/m4/snprintf.m4' 2024-03-09T09:28:04.258 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/snprintf.m4’: No data available 2024-03-09T09:28:04.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stpncpy.m4' -> './k/coreutils-8.5/m4/stpncpy.m4' 2024-03-09T09:28:04.260 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stpncpy.m4’: No data available 2024-03-09T09:28:04.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/xnanosleep.m4' -> './k/coreutils-8.5/m4/xnanosleep.m4' 2024-03-09T09:28:04.261 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xnanosleep.m4’: No data available 2024-03-09T09:28:04.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/warnings.m4' -> './k/coreutils-8.5/m4/warnings.m4' 2024-03-09T09:28:04.262 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/warnings.m4’: No data available 2024-03-09T09:28:04.263 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/lseek.m4' -> './k/coreutils-8.5/m4/lseek.m4' 2024-03-09T09:28:04.264 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/lseek.m4’: No data available 2024-03-09T09:28:04.264 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/hard-locale.m4' -> './k/coreutils-8.5/m4/hard-locale.m4' 2024-03-09T09:28:04.265 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/hard-locale.m4’: No data available 2024-03-09T09:28:04.266 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/include-exclude-prog.m4' -> './k/coreutils-8.5/m4/include-exclude-prog.m4' 2024-03-09T09:28:04.266 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/include-exclude-prog.m4’: No data available 2024-03-09T09:28:04.267 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mmap-anon.m4' -> './k/coreutils-8.5/m4/mmap-anon.m4' 2024-03-09T09:28:04.267 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mmap-anon.m4’: No data available 2024-03-09T09:28:04.268 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mkstemp.m4' -> './k/coreutils-8.5/m4/mkstemp.m4' 2024-03-09T09:28:04.269 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mkstemp.m4’: No data available 2024-03-09T09:28:04.269 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/openat.m4' -> './k/coreutils-8.5/m4/openat.m4' 2024-03-09T09:28:04.269 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/openat.m4’: No data available 2024-03-09T09:28:04.270 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mbrlen.m4' -> './k/coreutils-8.5/m4/mbrlen.m4' 2024-03-09T09:28:04.271 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbrlen.m4’: No data available 2024-03-09T09:28:04.272 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/jm-macros.m4' -> './k/coreutils-8.5/m4/jm-macros.m4' 2024-03-09T09:28:04.272 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/jm-macros.m4’: No data available 2024-03-09T09:28:04.273 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/ftello.m4' -> './k/coreutils-8.5/m4/ftello.m4' 2024-03-09T09:28:04.273 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ftello.m4’: No data available 2024-03-09T09:28:04.274 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stdio_h.m4' -> './k/coreutils-8.5/m4/stdio_h.m4' 2024-03-09T09:28:04.274 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stdio_h.m4’: No data available 2024-03-09T09:28:04.276 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/exponentl.m4' -> './k/coreutils-8.5/m4/exponentl.m4' 2024-03-09T09:28:04.276 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/exponentl.m4’: No data available 2024-03-09T09:28:04.277 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getpagesize.m4' -> './k/coreutils-8.5/m4/getpagesize.m4' 2024-03-09T09:28:04.277 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getpagesize.m4’: No data available 2024-03-09T09:28:04.278 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/argmatch.m4' -> './k/coreutils-8.5/m4/argmatch.m4' 2024-03-09T09:28:04.278 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/argmatch.m4’: No data available 2024-03-09T09:28:04.279 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/perror.m4' -> './k/coreutils-8.5/m4/perror.m4' 2024-03-09T09:28:04.279 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/perror.m4’: No data available 2024-03-09T09:28:04.280 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/inline.m4' -> './k/coreutils-8.5/m4/inline.m4' 2024-03-09T09:28:04.280 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inline.m4’: No data available 2024-03-09T09:28:04.281 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/inttypes_h.m4' -> './k/coreutils-8.5/m4/inttypes_h.m4' 2024-03-09T09:28:04.282 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/inttypes_h.m4’: No data available 2024-03-09T09:28:04.283 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/iconv_h.m4' -> './k/coreutils-8.5/m4/iconv_h.m4' 2024-03-09T09:28:04.283 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/iconv_h.m4’: No data available 2024-03-09T09:28:04.284 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/base64.m4' -> './k/coreutils-8.5/m4/base64.m4' 2024-03-09T09:28:04.284 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/base64.m4’: No data available 2024-03-09T09:28:04.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/stat.m4' -> './k/coreutils-8.5/m4/stat.m4' 2024-03-09T09:28:04.286 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/stat.m4’: No data available 2024-03-09T09:28:04.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/tzset.m4' -> './k/coreutils-8.5/m4/tzset.m4' 2024-03-09T09:28:04.287 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/tzset.m4’: No data available 2024-03-09T09:28:04.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/unicodeio.m4' -> './k/coreutils-8.5/m4/unicodeio.m4' 2024-03-09T09:28:04.288 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unicodeio.m4’: No data available 2024-03-09T09:28:04.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sha256.m4' -> './k/coreutils-8.5/m4/sha256.m4' 2024-03-09T09:28:04.290 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sha256.m4’: No data available 2024-03-09T09:28:04.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/errno_h.m4' -> './k/coreutils-8.5/m4/errno_h.m4' 2024-03-09T09:28:04.291 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/errno_h.m4’: No data available 2024-03-09T09:28:04.292 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/unlinkdir.m4' -> './k/coreutils-8.5/m4/unlinkdir.m4' 2024-03-09T09:28:04.292 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/unlinkdir.m4’: No data available 2024-03-09T09:28:04.293 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/boottime.m4' -> './k/coreutils-8.5/m4/boottime.m4' 2024-03-09T09:28:04.293 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/boottime.m4’: No data available 2024-03-09T09:28:04.294 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/freading.m4' -> './k/coreutils-8.5/m4/freading.m4' 2024-03-09T09:28:04.294 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/freading.m4’: No data available 2024-03-09T09:28:04.298 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mode_t.m4' -> './k/coreutils-8.5/m4/mode_t.m4' 2024-03-09T09:28:04.298 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mode_t.m4’: No data available 2024-03-09T09:28:04.299 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getndelim2.m4' -> './k/coreutils-8.5/m4/getndelim2.m4' 2024-03-09T09:28:04.299 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getndelim2.m4’: No data available 2024-03-09T09:28:04.299 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/xvasprintf.m4' -> './k/coreutils-8.5/m4/xvasprintf.m4' 2024-03-09T09:28:04.300 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xvasprintf.m4’: No data available 2024-03-09T09:28:04.300 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mbsrtowcs.m4' -> './k/coreutils-8.5/m4/mbsrtowcs.m4' 2024-03-09T09:28:04.300 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbsrtowcs.m4’: No data available 2024-03-09T09:28:04.301 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/hash.m4' -> './k/coreutils-8.5/m4/hash.m4' 2024-03-09T09:28:04.301 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/hash.m4’: No data available 2024-03-09T09:28:04.302 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/safe-write.m4' -> './k/coreutils-8.5/m4/safe-write.m4' 2024-03-09T09:28:04.302 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/safe-write.m4’: No data available 2024-03-09T09:28:04.304 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/po.m4' -> './k/coreutils-8.5/m4/po.m4' 2024-03-09T09:28:04.304 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/po.m4’: No data available 2024-03-09T09:28:04.305 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/isnanf.m4' -> './k/coreutils-8.5/m4/isnanf.m4' 2024-03-09T09:28:04.305 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/isnanf.m4’: No data available 2024-03-09T09:28:04.306 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/longlong.m4' -> './k/coreutils-8.5/m4/longlong.m4' 2024-03-09T09:28:04.306 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/longlong.m4’: No data available 2024-03-09T09:28:04.307 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/xstrndup.m4' -> './k/coreutils-8.5/m4/xstrndup.m4' 2024-03-09T09:28:04.308 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/xstrndup.m4’: No data available 2024-03-09T09:28:04.308 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/euidaccess-stat.m4' -> './k/coreutils-8.5/m4/euidaccess-stat.m4' 2024-03-09T09:28:04.309 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/euidaccess-stat.m4’: No data available 2024-03-09T09:28:04.310 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/getcwd-abort-bug.m4' -> './k/coreutils-8.5/m4/getcwd-abort-bug.m4' 2024-03-09T09:28:04.310 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/getcwd-abort-bug.m4’: No data available 2024-03-09T09:28:04.311 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/faccessat.m4' -> './k/coreutils-8.5/m4/faccessat.m4' 2024-03-09T09:28:04.311 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/faccessat.m4’: No data available 2024-03-09T09:28:04.312 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/nanosleep.m4' -> './k/coreutils-8.5/m4/nanosleep.m4' 2024-03-09T09:28:04.312 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/nanosleep.m4’: No data available 2024-03-09T09:28:04.315 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/ChangeLog-2007' -> './k/coreutils-8.5/m4/ChangeLog-2007' 2024-03-09T09:28:04.315 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/ChangeLog-2007’: No data available 2024-03-09T09:28:04.316 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/utimes.m4' -> './k/coreutils-8.5/m4/utimes.m4' 2024-03-09T09:28:04.316 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/utimes.m4’: No data available 2024-03-09T09:28:04.317 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fileblocks.m4' -> './k/coreutils-8.5/m4/fileblocks.m4' 2024-03-09T09:28:04.318 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fileblocks.m4’: No data available 2024-03-09T09:28:04.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/sockpfaf.m4' -> './k/coreutils-8.5/m4/sockpfaf.m4' 2024-03-09T09:28:04.319 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/sockpfaf.m4’: No data available 2024-03-09T09:28:04.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strtoull.m4' -> './k/coreutils-8.5/m4/strtoull.m4' 2024-03-09T09:28:04.320 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strtoull.m4’: No data available 2024-03-09T09:28:04.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/assert.m4' -> './k/coreutils-8.5/m4/assert.m4' 2024-03-09T09:28:04.320 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/assert.m4’: No data available 2024-03-09T09:28:04.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/quotearg.m4' -> './k/coreutils-8.5/m4/quotearg.m4' 2024-03-09T09:28:04.322 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/quotearg.m4’: No data available 2024-03-09T09:28:04.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/uname.m4' -> './k/coreutils-8.5/m4/uname.m4' 2024-03-09T09:28:04.323 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/uname.m4’: No data available 2024-03-09T09:28:04.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/savedir.m4' -> './k/coreutils-8.5/m4/savedir.m4' 2024-03-09T09:28:04.325 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/savedir.m4’: No data available 2024-03-09T09:28:04.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/threadlib.m4' -> './k/coreutils-8.5/m4/threadlib.m4' 2024-03-09T09:28:04.326 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/threadlib.m4’: No data available 2024-03-09T09:28:04.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/langinfo_h.m4' -> './k/coreutils-8.5/m4/langinfo_h.m4' 2024-03-09T09:28:04.327 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/langinfo_h.m4’: No data available 2024-03-09T09:28:04.329 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/linkat.m4' -> './k/coreutils-8.5/m4/linkat.m4' 2024-03-09T09:28:04.329 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/linkat.m4’: No data available 2024-03-09T09:28:04.330 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fpieee.m4' -> './k/coreutils-8.5/m4/fpieee.m4' 2024-03-09T09:28:04.330 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fpieee.m4’: No data available 2024-03-09T09:28:04.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/memcasecmp.m4' -> './k/coreutils-8.5/m4/memcasecmp.m4' 2024-03-09T09:28:04.332 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/memcasecmp.m4’: No data available 2024-03-09T09:28:04.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/vfprintf-posix.m4' -> './k/coreutils-8.5/m4/vfprintf-posix.m4' 2024-03-09T09:28:04.333 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/vfprintf-posix.m4’: No data available 2024-03-09T09:28:04.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/dos.m4' -> './k/coreutils-8.5/m4/dos.m4' 2024-03-09T09:28:04.334 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/dos.m4’: No data available 2024-03-09T09:28:04.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/root-dev-ino.m4' -> './k/coreutils-8.5/m4/root-dev-ino.m4' 2024-03-09T09:28:04.336 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/root-dev-ino.m4’: No data available 2024-03-09T09:28:04.337 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/quote.m4' -> './k/coreutils-8.5/m4/quote.m4' 2024-03-09T09:28:04.337 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/quote.m4’: No data available 2024-03-09T09:28:04.338 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/mbswidth.m4' -> './k/coreutils-8.5/m4/mbswidth.m4' 2024-03-09T09:28:04.339 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/mbswidth.m4’: No data available 2024-03-09T09:28:04.339 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/nl_langinfo.m4' -> './k/coreutils-8.5/m4/nl_langinfo.m4' 2024-03-09T09:28:04.340 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/nl_langinfo.m4’: No data available 2024-03-09T09:28:04.340 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/fcntl.m4' -> './k/coreutils-8.5/m4/fcntl.m4' 2024-03-09T09:28:04.340 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/fcntl.m4’: No data available 2024-03-09T09:28:04.342 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/strcspn.m4' -> './k/coreutils-8.5/m4/strcspn.m4' 2024-03-09T09:28:04.342 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/strcspn.m4’: No data available 2024-03-09T09:28:04.343 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/safe-read.m4' -> './k/coreutils-8.5/m4/safe-read.m4' 2024-03-09T09:28:04.344 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/safe-read.m4’: No data available 2024-03-09T09:28:04.345 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/wcwidth.m4' -> './k/coreutils-8.5/m4/wcwidth.m4' 2024-03-09T09:28:04.345 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/wcwidth.m4’: No data available 2024-03-09T09:28:04.346 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/m4/closeout.m4' -> './k/coreutils-8.5/m4/closeout.m4' 2024-03-09T09:28:04.346 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/m4/closeout.m4’: No data available 2024-03-09T09:28:04.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.tarball-version' -> './k/coreutils-8.5/.tarball-version' 2024-03-09T09:28:04.853 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.tarball-version’: No data available 2024-03-09T09:28:05.170 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chown' -> './k/coreutils-8.5/tests/chown' 2024-03-09T09:28:05.170 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chown/separator' -> './k/coreutils-8.5/tests/chown/separator' 2024-03-09T09:28:05.170 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chown/separator’: No data available 2024-03-09T09:28:05.172 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chown/deref' -> './k/coreutils-8.5/tests/chown/deref' 2024-03-09T09:28:05.172 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chown/deref’: No data available 2024-03-09T09:28:05.173 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chown/preserve-root' -> './k/coreutils-8.5/tests/chown/preserve-root' 2024-03-09T09:28:05.174 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chown/preserve-root’: No data available 2024-03-09T09:28:05.175 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chown/basic' -> './k/coreutils-8.5/tests/chown/basic' 2024-03-09T09:28:05.176 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chown/basic’: No data available 2024-03-09T09:28:05.473 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/id' -> './k/coreutils-8.5/tests/id' 2024-03-09T09:28:05.473 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/id/no-context' -> './k/coreutils-8.5/tests/id/no-context' 2024-03-09T09:28:05.474 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/id/no-context’: No data available 2024-03-09T09:28:05.509 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp' -> './k/coreutils-8.5/tests/cp' 2024-03-09T09:28:05.509 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/preserve-2' -> './k/coreutils-8.5/tests/cp/preserve-2' 2024-03-09T09:28:05.509 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/preserve-2’: No data available 2024-03-09T09:28:05.511 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/dir-vs-file' -> './k/coreutils-8.5/tests/cp/dir-vs-file' 2024-03-09T09:28:05.511 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/dir-vs-file’: No data available 2024-03-09T09:28:05.513 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/preserve-slink-time' -> './k/coreutils-8.5/tests/cp/preserve-slink-time' 2024-03-09T09:28:05.513 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/preserve-slink-time’: No data available 2024-03-09T09:28:05.515 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/same-file' -> './k/coreutils-8.5/tests/cp/same-file' 2024-03-09T09:28:05.516 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/same-file’: No data available 2024-03-09T09:28:05.517 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/src-base-dot' -> './k/coreutils-8.5/tests/cp/src-base-dot' 2024-03-09T09:28:05.518 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/src-base-dot’: No data available 2024-03-09T09:28:05.520 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-a-selinux' -> './k/coreutils-8.5/tests/cp/cp-a-selinux' 2024-03-09T09:28:05.520 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-a-selinux’: No data available 2024-03-09T09:28:05.526 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/r-vs-symlink' -> './k/coreutils-8.5/tests/cp/r-vs-symlink' 2024-03-09T09:28:05.526 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/r-vs-symlink’: No data available 2024-03-09T09:28:05.527 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/capability' -> './k/coreutils-8.5/tests/cp/capability' 2024-03-09T09:28:05.527 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/capability’: No data available 2024-03-09T09:28:05.528 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/existing-perm-race' -> './k/coreutils-8.5/tests/cp/existing-perm-race' 2024-03-09T09:28:05.528 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/existing-perm-race’: No data available 2024-03-09T09:28:05.529 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/parent-perm-race' -> './k/coreutils-8.5/tests/cp/parent-perm-race' 2024-03-09T09:28:05.530 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/parent-perm-race’: No data available 2024-03-09T09:28:05.532 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/reflink-auto' -> './k/coreutils-8.5/tests/cp/reflink-auto' 2024-03-09T09:28:05.532 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/reflink-auto’: No data available 2024-03-09T09:28:05.534 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-mv-enotsup-xattr' -> './k/coreutils-8.5/tests/cp/cp-mv-enotsup-xattr' 2024-03-09T09:28:05.534 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-mv-enotsup-xattr’: No data available 2024-03-09T09:28:05.536 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/no-deref-link2' -> './k/coreutils-8.5/tests/cp/no-deref-link2' 2024-03-09T09:28:05.536 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/no-deref-link2’: No data available 2024-03-09T09:28:05.539 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-mv-backup' -> './k/coreutils-8.5/tests/cp/cp-mv-backup' 2024-03-09T09:28:05.539 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-mv-backup’: No data available 2024-03-09T09:28:05.541 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/link-no-deref' -> './k/coreutils-8.5/tests/cp/link-no-deref' 2024-03-09T09:28:05.541 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/link-no-deref’: No data available 2024-03-09T09:28:05.543 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/file-perm-race' -> './k/coreutils-8.5/tests/cp/file-perm-race' 2024-03-09T09:28:05.543 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/file-perm-race’: No data available 2024-03-09T09:28:05.546 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/abuse' -> './k/coreutils-8.5/tests/cp/abuse' 2024-03-09T09:28:05.546 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/abuse’: No data available 2024-03-09T09:28:05.548 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/link-heap' -> './k/coreutils-8.5/tests/cp/link-heap' 2024-03-09T09:28:05.548 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/link-heap’: No data available 2024-03-09T09:28:05.550 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/backup-dir' -> './k/coreutils-8.5/tests/cp/backup-dir' 2024-03-09T09:28:05.551 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/backup-dir’: No data available 2024-03-09T09:28:05.553 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/preserve-gid' -> './k/coreutils-8.5/tests/cp/preserve-gid' 2024-03-09T09:28:05.553 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/preserve-gid’: No data available 2024-03-09T09:28:05.555 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/NEWS' -> './k/coreutils-8.5/NEWS' 2024-03-09T09:28:05.555 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/NEWS’: No data available 2024-03-09T09:28:05.556 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/link' -> './k/coreutils-8.5/tests/cp/link' 2024-03-09T09:28:05.556 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/link’: No data available 2024-03-09T09:28:05.557 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_sun_os_names' -> './k/coreutils-8.5/.x-sc_sun_os_names' 2024-03-09T09:28:05.558 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_sun_os_names’: No data available 2024-03-09T09:28:05.559 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-parents' -> './k/coreutils-8.5/tests/cp/cp-parents' 2024-03-09T09:28:05.559 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-parents’: No data available 2024-03-09T09:28:05.561 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/bootstrap' -> './k/coreutils-8.5/bootstrap' 2024-03-09T09:28:05.561 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/bootstrap’: No data available 2024-03-09T09:28:05.562 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/no-deref-link1' -> './k/coreutils-8.5/tests/cp/no-deref-link1' 2024-03-09T09:28:05.562 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/no-deref-link1’: No data available 2024-03-09T09:28:05.564 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/into-self' -> './k/coreutils-8.5/tests/cp/into-self' 2024-03-09T09:28:05.564 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/into-self’: No data available 2024-03-09T09:28:05.566 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/parent-perm' -> './k/coreutils-8.5/tests/cp/parent-perm' 2024-03-09T09:28:05.566 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/parent-perm’: No data available 2024-03-09T09:28:05.569 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-deref' -> './k/coreutils-8.5/tests/cp/cp-deref' 2024-03-09T09:28:05.569 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-deref’: No data available 2024-03-09T09:28:05.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/proc-short-read' -> './k/coreutils-8.5/tests/cp/proc-short-read' 2024-03-09T09:28:05.572 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/proc-short-read’: No data available 2024-03-09T09:28:05.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/backup-is-src' -> './k/coreutils-8.5/tests/cp/backup-is-src' 2024-03-09T09:28:05.575 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/backup-is-src’: No data available 2024-03-09T09:28:05.578 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-i' -> './k/coreutils-8.5/tests/cp/cp-i' 2024-03-09T09:28:05.579 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-i’: No data available 2024-03-09T09:28:05.581 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/special-f' -> './k/coreutils-8.5/tests/cp/special-f' 2024-03-09T09:28:05.581 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/special-f’: No data available 2024-03-09T09:28:05.583 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/proc-zero-len' -> './k/coreutils-8.5/tests/cp/proc-zero-len' 2024-03-09T09:28:05.584 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/proc-zero-len’: No data available 2024-03-09T09:28:05.586 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/slink-2-slink' -> './k/coreutils-8.5/tests/cp/slink-2-slink' 2024-03-09T09:28:05.586 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/slink-2-slink’: No data available 2024-03-09T09:28:05.588 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/special-bits' -> './k/coreutils-8.5/tests/cp/special-bits' 2024-03-09T09:28:05.588 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/special-bits’: No data available 2024-03-09T09:28:05.591 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/fail-perm' -> './k/coreutils-8.5/tests/cp/fail-perm' 2024-03-09T09:28:05.591 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/fail-perm’: No data available 2024-03-09T09:28:05.593 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/link-preserve' -> './k/coreutils-8.5/tests/cp/link-preserve' 2024-03-09T09:28:05.593 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/link-preserve’: No data available 2024-03-09T09:28:05.595 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man' -> './k/coreutils-8.5/man' 2024-03-09T09:28:05.595 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/date.x' -> './k/coreutils-8.5/man/date.x' 2024-03-09T09:28:05.595 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/date.x’: No data available 2024-03-09T09:28:05.596 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/dir-rm-dest' -> './k/coreutils-8.5/tests/cp/dir-rm-dest' 2024-03-09T09:28:05.596 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/dir-rm-dest’: No data available 2024-03-09T09:28:05.597 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/backup-1' -> './k/coreutils-8.5/tests/cp/backup-1' 2024-03-09T09:28:05.597 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/backup-1’: No data available 2024-03-09T09:28:05.598 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/basename.1' -> './k/coreutils-8.5/man/basename.1' 2024-03-09T09:28:05.598 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/basename.1’: No data available 2024-03-09T09:28:05.600 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/pathchk.x' -> './k/coreutils-8.5/man/pathchk.x' 2024-03-09T09:28:05.600 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pathchk.x’: No data available 2024-03-09T09:28:05.600 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-HL' -> './k/coreutils-8.5/tests/cp/cp-HL' 2024-03-09T09:28:05.600 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-HL’: No data available 2024-03-09T09:28:05.602 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tail.x' -> './k/coreutils-8.5/man/tail.x' 2024-03-09T09:28:05.603 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tail.x’: No data available 2024-03-09T09:28:05.603 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/thru-dangling' -> './k/coreutils-8.5/tests/cp/thru-dangling' 2024-03-09T09:28:05.603 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/thru-dangling’: No data available 2024-03-09T09:28:05.604 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/sparse' -> './k/coreutils-8.5/tests/cp/sparse' 2024-03-09T09:28:05.605 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/sparse’: No data available 2024-03-09T09:28:05.605 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/yes.x' -> './k/coreutils-8.5/man/yes.x' 2024-03-09T09:28:05.605 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/yes.x’: No data available 2024-03-09T09:28:05.606 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/deref-slink' -> './k/coreutils-8.5/tests/cp/deref-slink' 2024-03-09T09:28:05.607 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/deref-slink’: No data available 2024-03-09T09:28:05.607 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/pinky.1' -> './k/coreutils-8.5/man/pinky.1' 2024-03-09T09:28:05.607 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pinky.1’: No data available 2024-03-09T09:28:05.609 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/symlink-slash' -> './k/coreutils-8.5/tests/cp/symlink-slash' 2024-03-09T09:28:05.609 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/symlink-slash’: No data available 2024-03-09T09:28:05.610 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/kill.x' -> './k/coreutils-8.5/man/kill.x' 2024-03-09T09:28:05.610 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/kill.x’: No data available 2024-03-09T09:28:05.612 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/acl' -> './k/coreutils-8.5/tests/cp/acl' 2024-03-09T09:28:05.612 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/acl’: No data available 2024-03-09T09:28:05.612 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/pathchk.1' -> './k/coreutils-8.5/man/pathchk.1' 2024-03-09T09:28:05.612 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pathchk.1’: No data available 2024-03-09T09:28:05.613 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/perm' -> './k/coreutils-8.5/tests/cp/perm' 2024-03-09T09:28:05.613 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/perm’: No data available 2024-03-09T09:28:05.614 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sync.1' -> './k/coreutils-8.5/man/sync.1' 2024-03-09T09:28:05.614 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sync.1’: No data available 2024-03-09T09:28:05.615 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/reflink-perm' -> './k/coreutils-8.5/tests/cp/reflink-perm' 2024-03-09T09:28:05.615 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/reflink-perm’: No data available 2024-03-09T09:28:05.616 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/dir.1' -> './k/coreutils-8.5/man/dir.1' 2024-03-09T09:28:05.616 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dir.1’: No data available 2024-03-09T09:28:05.617 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/no-deref-link3' -> './k/coreutils-8.5/tests/cp/no-deref-link3' 2024-03-09T09:28:05.617 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/no-deref-link3’: No data available 2024-03-09T09:28:05.619 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/mv.1' -> './k/coreutils-8.5/man/mv.1' 2024-03-09T09:28:05.619 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mv.1’: No data available 2024-03-09T09:28:05.619 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/cp/dir-slash' -> './k/coreutils-8.5/tests/cp/dir-slash' 2024-03-09T09:28:05.620 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/dir-slash’: No data available 2024-03-09T09:28:05.620 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/rm.1' -> './k/coreutils-8.5/man/rm.1' 2024-03-09T09:28:05.620 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/rm.1’: No data available 2024-03-09T09:28:05.622 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/dircolors.x' -> './k/coreutils-8.5/man/dircolors.x' 2024-03-09T09:28:05.622 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dircolors.x’: No data available 2024-03-09T09:28:05.623 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/rmdir.1' -> './k/coreutils-8.5/man/rmdir.1' 2024-03-09T09:28:05.624 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/rmdir.1’: No data available 2024-03-09T09:28:05.625 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tty.1' -> './k/coreutils-8.5/man/tty.1' 2024-03-09T09:28:05.625 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tty.1’: No data available 2024-03-09T09:28:05.626 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/fold.x' -> './k/coreutils-8.5/man/fold.x' 2024-03-09T09:28:05.626 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/fold.x’: No data available 2024-03-09T09:28:05.627 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/fold.1' -> './k/coreutils-8.5/man/fold.1' 2024-03-09T09:28:05.628 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/fold.1’: No data available 2024-03-09T09:28:05.629 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/dir.x' -> './k/coreutils-8.5/man/dir.x' 2024-03-09T09:28:05.629 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dir.x’: No data available 2024-03-09T09:28:05.631 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/fmt.x' -> './k/coreutils-8.5/man/fmt.x' 2024-03-09T09:28:05.631 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/fmt.x’: No data available 2024-03-09T09:28:05.632 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/paste.1' -> './k/coreutils-8.5/man/paste.1' 2024-03-09T09:28:05.633 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/paste.1’: No data available 2024-03-09T09:28:05.634 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/chmod.x' -> './k/coreutils-8.5/man/chmod.x' 2024-03-09T09:28:05.634 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chmod.x’: No data available 2024-03-09T09:28:05.636 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/od.1' -> './k/coreutils-8.5/man/od.1' 2024-03-09T09:28:05.636 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/od.1’: No data available 2024-03-09T09:28:05.637 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/ptx.x' -> './k/coreutils-8.5/man/ptx.x' 2024-03-09T09:28:05.637 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ptx.x’: No data available 2024-03-09T09:28:05.639 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/nice.1' -> './k/coreutils-8.5/man/nice.1' 2024-03-09T09:28:05.639 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nice.1’: No data available 2024-03-09T09:28:05.640 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/install.1' -> './k/coreutils-8.5/man/install.1' 2024-03-09T09:28:05.641 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/install.1’: No data available 2024-03-09T09:28:05.642 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/dd.x' -> './k/coreutils-8.5/man/dd.x' 2024-03-09T09:28:05.642 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dd.x’: No data available 2024-03-09T09:28:05.643 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/cksum.1' -> './k/coreutils-8.5/man/cksum.1' 2024-03-09T09:28:05.644 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cksum.1’: No data available 2024-03-09T09:28:05.645 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/test.1' -> './k/coreutils-8.5/man/test.1' 2024-03-09T09:28:05.645 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/test.1’: No data available 2024-03-09T09:28:05.646 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/chgrp.1' -> './k/coreutils-8.5/man/chgrp.1' 2024-03-09T09:28:05.646 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chgrp.1’: No data available 2024-03-09T09:28:05.648 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/unlink.1' -> './k/coreutils-8.5/man/unlink.1' 2024-03-09T09:28:05.648 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/unlink.1’: No data available 2024-03-09T09:28:05.651 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/chmod.1' -> './k/coreutils-8.5/man/chmod.1' 2024-03-09T09:28:05.651 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chmod.1’: No data available 2024-03-09T09:28:05.652 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/dd.1' -> './k/coreutils-8.5/man/dd.1' 2024-03-09T09:28:05.653 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dd.1’: No data available 2024-03-09T09:28:05.655 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/du.1' -> './k/coreutils-8.5/man/du.1' 2024-03-09T09:28:05.655 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/du.1’: No data available 2024-03-09T09:28:05.657 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tsort.1' -> './k/coreutils-8.5/man/tsort.1' 2024-03-09T09:28:05.657 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tsort.1’: No data available 2024-03-09T09:28:05.659 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/ls.1' -> './k/coreutils-8.5/man/ls.1' 2024-03-09T09:28:05.659 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ls.1’: No data available 2024-03-09T09:28:05.661 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/split.x' -> './k/coreutils-8.5/man/split.x' 2024-03-09T09:28:05.661 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/split.x’: No data available 2024-03-09T09:28:05.663 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/nohup.x' -> './k/coreutils-8.5/man/nohup.x' 2024-03-09T09:28:05.663 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nohup.x’: No data available 2024-03-09T09:28:05.664 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sha1sum.1' -> './k/coreutils-8.5/man/sha1sum.1' 2024-03-09T09:28:05.664 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha1sum.1’: No data available 2024-03-09T09:28:05.666 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tr.1' -> './k/coreutils-8.5/man/tr.1' 2024-03-09T09:28:05.666 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tr.1’: No data available 2024-03-09T09:28:05.668 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/head.x' -> './k/coreutils-8.5/man/head.x' 2024-03-09T09:28:05.668 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/head.x’: No data available 2024-03-09T09:28:05.669 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/false.1' -> './k/coreutils-8.5/man/false.1' 2024-03-09T09:28:05.670 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/false.1’: No data available 2024-03-09T09:28:05.671 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/pwd.x' -> './k/coreutils-8.5/man/pwd.x' 2024-03-09T09:28:05.671 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pwd.x’: No data available 2024-03-09T09:28:05.673 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/logname.x' -> './k/coreutils-8.5/man/logname.x' 2024-03-09T09:28:05.673 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/logname.x’: No data available 2024-03-09T09:28:05.683 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/paste.x' -> './k/coreutils-8.5/man/paste.x' 2024-03-09T09:28:05.683 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/paste.x’: No data available 2024-03-09T09:28:05.685 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/nl.x' -> './k/coreutils-8.5/man/nl.x' 2024-03-09T09:28:05.686 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nl.x’: No data available 2024-03-09T09:28:05.688 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sha224sum.1' -> './k/coreutils-8.5/man/sha224sum.1' 2024-03-09T09:28:05.688 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha224sum.1’: No data available 2024-03-09T09:28:05.689 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/mkdir.1' -> './k/coreutils-8.5/man/mkdir.1' 2024-03-09T09:28:05.690 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mkdir.1’: No data available 2024-03-09T09:28:05.691 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/nl.1' -> './k/coreutils-8.5/man/nl.1' 2024-03-09T09:28:05.691 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nl.1’: No data available 2024-03-09T09:28:05.693 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sha256sum.1' -> './k/coreutils-8.5/man/sha256sum.1' 2024-03-09T09:28:05.694 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha256sum.1’: No data available 2024-03-09T09:28:05.695 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/printf.x' -> './k/coreutils-8.5/man/printf.x' 2024-03-09T09:28:05.695 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/printf.x’: No data available 2024-03-09T09:28:05.697 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/md5sum.1' -> './k/coreutils-8.5/man/md5sum.1' 2024-03-09T09:28:05.697 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/md5sum.1’: No data available 2024-03-09T09:28:05.699 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/Makefile.am' -> './k/coreutils-8.5/man/Makefile.am' 2024-03-09T09:28:05.699 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/Makefile.am’: No data available 2024-03-09T09:28:05.701 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/dirname.1' -> './k/coreutils-8.5/man/dirname.1' 2024-03-09T09:28:05.701 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dirname.1’: No data available 2024-03-09T09:28:05.702 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/shuf.x' -> './k/coreutils-8.5/man/shuf.x' 2024-03-09T09:28:05.702 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/shuf.x’: No data available 2024-03-09T09:28:05.705 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/echo.1' -> './k/coreutils-8.5/man/echo.1' 2024-03-09T09:28:05.705 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/echo.1’: No data available 2024-03-09T09:28:05.705 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/chcon.x' -> './k/coreutils-8.5/man/chcon.x' 2024-03-09T09:28:05.705 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chcon.x’: No data available 2024-03-09T09:28:05.707 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/chroot.1' -> './k/coreutils-8.5/man/chroot.1' 2024-03-09T09:28:05.708 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chroot.1’: No data available 2024-03-09T09:28:05.709 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sha384sum.x' -> './k/coreutils-8.5/man/sha384sum.x' 2024-03-09T09:28:05.709 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha384sum.x’: No data available 2024-03-09T09:28:05.710 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/stdbuf.1' -> './k/coreutils-8.5/man/stdbuf.1' 2024-03-09T09:28:05.710 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stdbuf.1’: No data available 2024-03-09T09:28:05.711 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/users.1' -> './k/coreutils-8.5/man/users.1' 2024-03-09T09:28:05.712 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/users.1’: No data available 2024-03-09T09:28:05.713 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/readlink.x' -> './k/coreutils-8.5/man/readlink.x' 2024-03-09T09:28:05.713 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/readlink.x’: No data available 2024-03-09T09:28:05.714 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/cut.x' -> './k/coreutils-8.5/man/cut.x' 2024-03-09T09:28:05.715 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cut.x’: No data available 2024-03-09T09:28:05.716 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/groups.1' -> './k/coreutils-8.5/man/groups.1' 2024-03-09T09:28:05.716 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/groups.1’: No data available 2024-03-09T09:28:05.717 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/seq.1' -> './k/coreutils-8.5/man/seq.1' 2024-03-09T09:28:05.717 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/seq.1’: No data available 2024-03-09T09:28:05.719 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/help2man' -> './k/coreutils-8.5/man/help2man' 2024-03-09T09:28:05.719 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/help2man’: No data available 2024-03-09T09:28:05.721 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/dirname.x' -> './k/coreutils-8.5/man/dirname.x' 2024-03-09T09:28:05.722 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dirname.x’: No data available 2024-03-09T09:28:05.723 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/uname.x' -> './k/coreutils-8.5/man/uname.x' 2024-03-09T09:28:05.723 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uname.x’: No data available 2024-03-09T09:28:05.725 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/uptime.x' -> './k/coreutils-8.5/man/uptime.x' 2024-03-09T09:28:05.725 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uptime.x’: No data available 2024-03-09T09:28:05.726 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/cut.1' -> './k/coreutils-8.5/man/cut.1' 2024-03-09T09:28:05.729 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cut.1’: No data available 2024-03-09T09:28:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/cksum.x' -> './k/coreutils-8.5/man/cksum.x' 2024-03-09T09:28:05.730 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cksum.x’: No data available 2024-03-09T09:28:05.730 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/nohup.1' -> './k/coreutils-8.5/man/nohup.1' 2024-03-09T09:28:05.731 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nohup.1’: No data available 2024-03-09T09:28:05.731 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/mv.x' -> './k/coreutils-8.5/man/mv.x' 2024-03-09T09:28:05.731 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mv.x’: No data available 2024-03-09T09:28:05.732 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sort.1' -> './k/coreutils-8.5/man/sort.1' 2024-03-09T09:28:05.732 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sort.1’: No data available 2024-03-09T09:28:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/chown.1' -> './k/coreutils-8.5/man/chown.1' 2024-03-09T09:28:05.733 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chown.1’: No data available 2024-03-09T09:28:05.733 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/pr.x' -> './k/coreutils-8.5/man/pr.x' 2024-03-09T09:28:05.734 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pr.x’: No data available 2024-03-09T09:28:05.735 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/true.x' -> './k/coreutils-8.5/man/true.x' 2024-03-09T09:28:05.735 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/true.x’: No data available 2024-03-09T09:28:05.736 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/base64.1' -> './k/coreutils-8.5/man/base64.1' 2024-03-09T09:28:05.736 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/base64.1’: No data available 2024-03-09T09:28:06.260 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/factor.1' -> './k/coreutils-8.5/man/factor.1' 2024-03-09T09:28:06.260 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/factor.1’: No data available 2024-03-09T09:28:06.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/ln.x' -> './k/coreutils-8.5/man/ln.x' 2024-03-09T09:28:06.261 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ln.x’: No data available 2024-03-09T09:28:06.263 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/head.1' -> './k/coreutils-8.5/man/head.1' 2024-03-09T09:28:06.263 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/head.1’: No data available 2024-03-09T09:28:06.265 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/unexpand.1' -> './k/coreutils-8.5/man/unexpand.1' 2024-03-09T09:28:06.266 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/unexpand.1’: No data available 2024-03-09T09:28:06.267 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/df.x' -> './k/coreutils-8.5/man/df.x' 2024-03-09T09:28:06.268 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/df.x’: No data available 2024-03-09T09:28:06.270 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/join.x' -> './k/coreutils-8.5/man/join.x' 2024-03-09T09:28:06.270 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/join.x’: No data available 2024-03-09T09:28:06.271 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/du.x' -> './k/coreutils-8.5/man/du.x' 2024-03-09T09:28:06.272 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/du.x’: No data available 2024-03-09T09:28:06.274 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/dircolors.1' -> './k/coreutils-8.5/man/dircolors.1' 2024-03-09T09:28:06.274 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/dircolors.1’: No data available 2024-03-09T09:28:06.275 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/uniq.x' -> './k/coreutils-8.5/man/uniq.x' 2024-03-09T09:28:06.275 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uniq.x’: No data available 2024-03-09T09:28:06.277 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/split.1' -> './k/coreutils-8.5/man/split.1' 2024-03-09T09:28:06.278 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/split.1’: No data available 2024-03-09T09:28:06.279 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tty.x' -> './k/coreutils-8.5/man/tty.x' 2024-03-09T09:28:06.279 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tty.x’: No data available 2024-03-09T09:28:06.282 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/uname.1' -> './k/coreutils-8.5/man/uname.1' 2024-03-09T09:28:06.283 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uname.1’: No data available 2024-03-09T09:28:06.283 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/uptime.1' -> './k/coreutils-8.5/man/uptime.1' 2024-03-09T09:28:06.284 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uptime.1’: No data available 2024-03-09T09:28:06.285 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sort.x' -> './k/coreutils-8.5/man/sort.x' 2024-03-09T09:28:06.285 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sort.x’: No data available 2024-03-09T09:28:06.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sha256sum.x' -> './k/coreutils-8.5/man/sha256sum.x' 2024-03-09T09:28:06.287 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha256sum.x’: No data available 2024-03-09T09:28:06.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sha512sum.x' -> './k/coreutils-8.5/man/sha512sum.x' 2024-03-09T09:28:06.289 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha512sum.x’: No data available 2024-03-09T09:28:06.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/nice.x' -> './k/coreutils-8.5/man/nice.x' 2024-03-09T09:28:06.291 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nice.x’: No data available 2024-03-09T09:28:06.292 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/pinky.x' -> './k/coreutils-8.5/man/pinky.x' 2024-03-09T09:28:06.292 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pinky.x’: No data available 2024-03-09T09:28:06.294 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/fmt.1' -> './k/coreutils-8.5/man/fmt.1' 2024-03-09T09:28:06.294 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/fmt.1’: No data available 2024-03-09T09:28:06.295 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/expr.1' -> './k/coreutils-8.5/man/expr.1' 2024-03-09T09:28:06.296 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/expr.1’: No data available 2024-03-09T09:28:06.297 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/wc.1' -> './k/coreutils-8.5/man/wc.1' 2024-03-09T09:28:06.298 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/wc.1’: No data available 2024-03-09T09:28:06.299 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/true.1' -> './k/coreutils-8.5/man/true.1' 2024-03-09T09:28:06.300 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/true.1’: No data available 2024-03-09T09:28:06.301 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/comm.x' -> './k/coreutils-8.5/man/comm.x' 2024-03-09T09:28:06.301 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/comm.x’: No data available 2024-03-09T09:28:06.303 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/truncate.x' -> './k/coreutils-8.5/man/truncate.x' 2024-03-09T09:28:06.303 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/truncate.x’: No data available 2024-03-09T09:28:06.305 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/install.x' -> './k/coreutils-8.5/man/install.x' 2024-03-09T09:28:06.305 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/install.x’: No data available 2024-03-09T09:28:06.307 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/wc.x' -> './k/coreutils-8.5/man/wc.x' 2024-03-09T09:28:06.307 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/wc.x’: No data available 2024-03-09T09:28:06.308 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/uniq.1' -> './k/coreutils-8.5/man/uniq.1' 2024-03-09T09:28:06.308 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/uniq.1’: No data available 2024-03-09T09:28:06.310 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/arch.x' -> './k/coreutils-8.5/man/arch.x' 2024-03-09T09:28:06.310 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/arch.x’: No data available 2024-03-09T09:28:06.311 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sum.x' -> './k/coreutils-8.5/man/sum.x' 2024-03-09T09:28:06.311 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sum.x’: No data available 2024-03-09T09:28:06.313 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/timeout.x' -> './k/coreutils-8.5/man/timeout.x' 2024-03-09T09:28:06.313 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/timeout.x’: No data available 2024-03-09T09:28:06.314 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/link.x' -> './k/coreutils-8.5/man/link.x' 2024-03-09T09:28:06.314 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/link.x’: No data available 2024-03-09T09:28:06.316 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tac.1' -> './k/coreutils-8.5/man/tac.1' 2024-03-09T09:28:06.316 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tac.1’: No data available 2024-03-09T09:28:06.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sync.x' -> './k/coreutils-8.5/man/sync.x' 2024-03-09T09:28:06.318 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sync.x’: No data available 2024-03-09T09:28:06.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/Makefile.in' -> './k/coreutils-8.5/man/Makefile.in' 2024-03-09T09:28:06.320 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/Makefile.in’: No data available 2024-03-09T09:28:06.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/mkfifo.1' -> './k/coreutils-8.5/man/mkfifo.1' 2024-03-09T09:28:06.322 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mkfifo.1’: No data available 2024-03-09T09:28:06.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/od.x' -> './k/coreutils-8.5/man/od.x' 2024-03-09T09:28:06.324 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/od.x’: No data available 2024-03-09T09:28:06.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/vdir.1' -> './k/coreutils-8.5/man/vdir.1' 2024-03-09T09:28:06.326 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/vdir.1’: No data available 2024-03-09T09:28:06.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/timeout.1' -> './k/coreutils-8.5/man/timeout.1' 2024-03-09T09:28:06.327 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/timeout.1’: No data available 2024-03-09T09:28:06.329 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/mkfifo.x' -> './k/coreutils-8.5/man/mkfifo.x' 2024-03-09T09:28:06.329 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mkfifo.x’: No data available 2024-03-09T09:28:06.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/basename.x' -> './k/coreutils-8.5/man/basename.x' 2024-03-09T09:28:06.331 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/basename.x’: No data available 2024-03-09T09:28:06.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/expr.x' -> './k/coreutils-8.5/man/expr.x' 2024-03-09T09:28:06.333 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/expr.x’: No data available 2024-03-09T09:28:06.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/mktemp.x' -> './k/coreutils-8.5/man/mktemp.x' 2024-03-09T09:28:06.335 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mktemp.x’: No data available 2024-03-09T09:28:06.336 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/vdir.x' -> './k/coreutils-8.5/man/vdir.x' 2024-03-09T09:28:06.336 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/vdir.x’: No data available 2024-03-09T09:28:06.337 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/su.x' -> './k/coreutils-8.5/man/su.x' 2024-03-09T09:28:06.337 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/su.x’: No data available 2024-03-09T09:28:06.339 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/env.1' -> './k/coreutils-8.5/man/env.1' 2024-03-09T09:28:06.342 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/env.1’: No data available 2024-03-09T09:28:06.342 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/who.x' -> './k/coreutils-8.5/man/who.x' 2024-03-09T09:28:06.342 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/who.x’: No data available 2024-03-09T09:28:06.343 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/hostid.1' -> './k/coreutils-8.5/man/hostid.1' 2024-03-09T09:28:06.343 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/hostid.1’: No data available 2024-03-09T09:28:06.344 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tee.x' -> './k/coreutils-8.5/man/tee.x' 2024-03-09T09:28:06.344 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tee.x’: No data available 2024-03-09T09:28:06.345 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/rmdir.x' -> './k/coreutils-8.5/man/rmdir.x' 2024-03-09T09:28:06.345 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/rmdir.x’: No data available 2024-03-09T09:28:06.346 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/hostid.x' -> './k/coreutils-8.5/man/hostid.x' 2024-03-09T09:28:06.347 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/hostid.x’: No data available 2024-03-09T09:28:06.348 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/id.x' -> './k/coreutils-8.5/man/id.x' 2024-03-09T09:28:06.348 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/id.x’: No data available 2024-03-09T09:28:06.350 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/unexpand.x' -> './k/coreutils-8.5/man/unexpand.x' 2024-03-09T09:28:06.350 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/unexpand.x’: No data available 2024-03-09T09:28:06.352 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/pr.1' -> './k/coreutils-8.5/man/pr.1' 2024-03-09T09:28:06.352 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pr.1’: No data available 2024-03-09T09:28:06.353 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/csplit.1' -> './k/coreutils-8.5/man/csplit.1' 2024-03-09T09:28:06.353 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/csplit.1’: No data available 2024-03-09T09:28:06.354 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/truncate.1' -> './k/coreutils-8.5/man/truncate.1' 2024-03-09T09:28:06.355 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/truncate.1’: No data available 2024-03-09T09:28:06.356 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tee.1' -> './k/coreutils-8.5/man/tee.1' 2024-03-09T09:28:06.356 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tee.1’: No data available 2024-03-09T09:28:06.367 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/env.x' -> './k/coreutils-8.5/man/env.x' 2024-03-09T09:28:06.367 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/env.x’: No data available 2024-03-09T09:28:06.368 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/chroot.x' -> './k/coreutils-8.5/man/chroot.x' 2024-03-09T09:28:06.369 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chroot.x’: No data available 2024-03-09T09:28:06.370 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/cp.x' -> './k/coreutils-8.5/man/cp.x' 2024-03-09T09:28:06.370 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cp.x’: No data available 2024-03-09T09:28:06.372 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/whoami.x' -> './k/coreutils-8.5/man/whoami.x' 2024-03-09T09:28:06.372 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/whoami.x’: No data available 2024-03-09T09:28:06.379 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/logname.1' -> './k/coreutils-8.5/man/logname.1' 2024-03-09T09:28:06.379 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/logname.1’: No data available 2024-03-09T09:28:06.381 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/mknod.1' -> './k/coreutils-8.5/man/mknod.1' 2024-03-09T09:28:06.381 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mknod.1’: No data available 2024-03-09T09:28:06.382 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/groups.x' -> './k/coreutils-8.5/man/groups.x' 2024-03-09T09:28:06.382 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/groups.x’: No data available 2024-03-09T09:28:06.384 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/cp.1' -> './k/coreutils-8.5/man/cp.1' 2024-03-09T09:28:06.384 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cp.1’: No data available 2024-03-09T09:28:06.386 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/runcon.x' -> './k/coreutils-8.5/man/runcon.x' 2024-03-09T09:28:06.386 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/runcon.x’: No data available 2024-03-09T09:28:06.388 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/yes.1' -> './k/coreutils-8.5/man/yes.1' 2024-03-09T09:28:06.388 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/yes.1’: No data available 2024-03-09T09:28:06.390 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/stat.1' -> './k/coreutils-8.5/man/stat.1' 2024-03-09T09:28:06.390 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stat.1’: No data available 2024-03-09T09:28:06.391 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/nproc.x' -> './k/coreutils-8.5/man/nproc.x' 2024-03-09T09:28:06.391 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nproc.x’: No data available 2024-03-09T09:28:06.393 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/unlink.x' -> './k/coreutils-8.5/man/unlink.x' 2024-03-09T09:28:06.393 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/unlink.x’: No data available 2024-03-09T09:28:06.395 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sha384sum.1' -> './k/coreutils-8.5/man/sha384sum.1' 2024-03-09T09:28:06.395 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha384sum.1’: No data available 2024-03-09T09:28:06.397 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/nproc.1' -> './k/coreutils-8.5/man/nproc.1' 2024-03-09T09:28:06.397 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/nproc.1’: No data available 2024-03-09T09:28:06.399 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/seq.x' -> './k/coreutils-8.5/man/seq.x' 2024-03-09T09:28:06.399 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/seq.x’: No data available 2024-03-09T09:28:06.400 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sha512sum.1' -> './k/coreutils-8.5/man/sha512sum.1' 2024-03-09T09:28:06.400 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha512sum.1’: No data available 2024-03-09T09:28:06.401 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/stat.x' -> './k/coreutils-8.5/man/stat.x' 2024-03-09T09:28:06.402 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stat.x’: No data available 2024-03-09T09:28:06.403 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sleep.x' -> './k/coreutils-8.5/man/sleep.x' 2024-03-09T09:28:06.403 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sleep.x’: No data available 2024-03-09T09:28:06.405 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/printenv.1' -> './k/coreutils-8.5/man/printenv.1' 2024-03-09T09:28:06.405 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/printenv.1’: No data available 2024-03-09T09:28:06.407 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tr.x' -> './k/coreutils-8.5/man/tr.x' 2024-03-09T09:28:06.407 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tr.x’: No data available 2024-03-09T09:28:06.408 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tail.1' -> './k/coreutils-8.5/man/tail.1' 2024-03-09T09:28:06.408 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tail.1’: No data available 2024-03-09T09:28:06.410 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/md5sum.x' -> './k/coreutils-8.5/man/md5sum.x' 2024-03-09T09:28:06.410 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/md5sum.x’: No data available 2024-03-09T09:28:06.411 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/link.1' -> './k/coreutils-8.5/man/link.1' 2024-03-09T09:28:06.412 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/link.1’: No data available 2024-03-09T09:28:06.413 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/echo.x' -> './k/coreutils-8.5/man/echo.x' 2024-03-09T09:28:06.413 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/echo.x’: No data available 2024-03-09T09:28:06.414 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/ptx.1' -> './k/coreutils-8.5/man/ptx.1' 2024-03-09T09:28:06.415 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ptx.1’: No data available 2024-03-09T09:28:06.416 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/factor.x' -> './k/coreutils-8.5/man/factor.x' 2024-03-09T09:28:06.416 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/factor.x’: No data available 2024-03-09T09:28:06.418 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/who.1' -> './k/coreutils-8.5/man/who.1' 2024-03-09T09:28:06.418 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/who.1’: No data available 2024-03-09T09:28:06.419 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/mktemp.1' -> './k/coreutils-8.5/man/mktemp.1' 2024-03-09T09:28:06.419 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mktemp.1’: No data available 2024-03-09T09:28:06.421 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/shred.1' -> './k/coreutils-8.5/man/shred.1' 2024-03-09T09:28:06.421 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/shred.1’: No data available 2024-03-09T09:28:06.423 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/runcon.1' -> './k/coreutils-8.5/man/runcon.1' 2024-03-09T09:28:06.423 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/runcon.1’: No data available 2024-03-09T09:28:06.424 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/stty.x' -> './k/coreutils-8.5/man/stty.x' 2024-03-09T09:28:06.425 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stty.x’: No data available 2024-03-09T09:28:06.426 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/comm.1' -> './k/coreutils-8.5/man/comm.1' 2024-03-09T09:28:06.427 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/comm.1’: No data available 2024-03-09T09:28:06.428 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/shuf.1' -> './k/coreutils-8.5/man/shuf.1' 2024-03-09T09:28:06.428 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/shuf.1’: No data available 2024-03-09T09:28:06.430 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/stty.1' -> './k/coreutils-8.5/man/stty.1' 2024-03-09T09:28:06.430 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stty.1’: No data available 2024-03-09T09:28:06.432 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/printenv.x' -> './k/coreutils-8.5/man/printenv.x' 2024-03-09T09:28:06.432 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/printenv.x’: No data available 2024-03-09T09:28:06.434 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/join.1' -> './k/coreutils-8.5/man/join.1' 2024-03-09T09:28:06.434 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/join.1’: No data available 2024-03-09T09:28:06.435 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/hostname.x' -> './k/coreutils-8.5/man/hostname.x' 2024-03-09T09:28:06.435 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/hostname.x’: No data available 2024-03-09T09:28:06.438 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/ln.1' -> './k/coreutils-8.5/man/ln.1' 2024-03-09T09:28:06.438 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ln.1’: No data available 2024-03-09T09:28:06.440 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/false.x' -> './k/coreutils-8.5/man/false.x' 2024-03-09T09:28:06.441 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/false.x’: No data available 2024-03-09T09:28:06.442 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/cat.1' -> './k/coreutils-8.5/man/cat.1' 2024-03-09T09:28:06.442 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cat.1’: No data available 2024-03-09T09:28:06.443 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/users.x' -> './k/coreutils-8.5/man/users.x' 2024-03-09T09:28:06.444 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/users.x’: No data available 2024-03-09T09:28:06.445 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/printf.1' -> './k/coreutils-8.5/man/printf.1' 2024-03-09T09:28:06.445 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/printf.1’: No data available 2024-03-09T09:28:06.446 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/ls.x' -> './k/coreutils-8.5/man/ls.x' 2024-03-09T09:28:06.447 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/ls.x’: No data available 2024-03-09T09:28:06.448 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sha224sum.x' -> './k/coreutils-8.5/man/sha224sum.x' 2024-03-09T09:28:06.448 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha224sum.x’: No data available 2024-03-09T09:28:06.450 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/whoami.1' -> './k/coreutils-8.5/man/whoami.1' 2024-03-09T09:28:06.450 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/whoami.1’: No data available 2024-03-09T09:28:06.451 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/expand.1' -> './k/coreutils-8.5/man/expand.1' 2024-03-09T09:28:06.451 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/expand.1’: No data available 2024-03-09T09:28:06.453 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/chgrp.x' -> './k/coreutils-8.5/man/chgrp.x' 2024-03-09T09:28:06.453 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chgrp.x’: No data available 2024-03-09T09:28:06.455 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sleep.1' -> './k/coreutils-8.5/man/sleep.1' 2024-03-09T09:28:06.455 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sleep.1’: No data available 2024-03-09T09:28:06.457 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/csplit.x' -> './k/coreutils-8.5/man/csplit.x' 2024-03-09T09:28:06.457 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/csplit.x’: No data available 2024-03-09T09:28:06.458 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/pwd.1' -> './k/coreutils-8.5/man/pwd.1' 2024-03-09T09:28:06.459 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/pwd.1’: No data available 2024-03-09T09:28:06.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/kill.1' -> './k/coreutils-8.5/man/kill.1' 2024-03-09T09:28:06.460 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/kill.1’: No data available 2024-03-09T09:28:06.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/touch.x' -> './k/coreutils-8.5/man/touch.x' 2024-03-09T09:28:06.462 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/touch.x’: No data available 2024-03-09T09:28:06.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir' -> './k/coreutils-8.5/tests/mkdir' 2024-03-09T09:28:06.463 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-v' -> './k/coreutils-8.5/tests/mkdir/p-v' 2024-03-09T09:28:06.463 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-v’: No data available 2024-03-09T09:28:06.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/df.1' -> './k/coreutils-8.5/man/df.1' 2024-03-09T09:28:06.465 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/df.1’: No data available 2024-03-09T09:28:06.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-3' -> './k/coreutils-8.5/tests/mkdir/p-3' 2024-03-09T09:28:06.465 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-3’: No data available 2024-03-09T09:28:06.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/stdbuf.x' -> './k/coreutils-8.5/man/stdbuf.x' 2024-03-09T09:28:06.467 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/stdbuf.x’: No data available 2024-03-09T09:28:06.467 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/writable-under-readonly' -> './k/coreutils-8.5/tests/mkdir/writable-under-readonly' 2024-03-09T09:28:06.468 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/writable-under-readonly’: No data available 2024-03-09T09:28:06.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-slashdot' -> './k/coreutils-8.5/tests/mkdir/p-slashdot' 2024-03-09T09:28:06.469 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-slashdot’: No data available 2024-03-09T09:28:06.469 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/mkdir.x' -> './k/coreutils-8.5/man/mkdir.x' 2024-03-09T09:28:06.469 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mkdir.x’: No data available 2024-03-09T09:28:06.470 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/selinux' -> './k/coreutils-8.5/tests/mkdir/selinux' 2024-03-09T09:28:06.471 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/selinux’: No data available 2024-03-09T09:28:06.472 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-1' -> './k/coreutils-8.5/tests/mkdir/p-1' 2024-03-09T09:28:06.473 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-1’: No data available 2024-03-09T09:28:06.473 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/id.1' -> './k/coreutils-8.5/man/id.1' 2024-03-09T09:28:06.473 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/id.1’: No data available 2024-03-09T09:28:06.474 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-2' -> './k/coreutils-8.5/tests/mkdir/p-2' 2024-03-09T09:28:06.474 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-2’: No data available 2024-03-09T09:28:06.475 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/mknod.x' -> './k/coreutils-8.5/man/mknod.x' 2024-03-09T09:28:06.475 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/mknod.x’: No data available 2024-03-09T09:28:06.476 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/parents' -> './k/coreutils-8.5/tests/mkdir/parents' 2024-03-09T09:28:06.477 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/parents’: No data available 2024-03-09T09:28:06.477 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sum.1' -> './k/coreutils-8.5/man/sum.1' 2024-03-09T09:28:06.477 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sum.1’: No data available 2024-03-09T09:28:06.479 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/rm.x' -> './k/coreutils-8.5/man/rm.x' 2024-03-09T09:28:06.479 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/rm.x’: No data available 2024-03-09T09:28:06.479 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/special-1' -> './k/coreutils-8.5/tests/mkdir/special-1' 2024-03-09T09:28:06.479 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/special-1’: No data available 2024-03-09T09:28:06.480 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/perm' -> './k/coreutils-8.5/tests/mkdir/perm' 2024-03-09T09:28:06.480 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/perm’: No data available 2024-03-09T09:28:06.481 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/date.1' -> './k/coreutils-8.5/man/date.1' 2024-03-09T09:28:06.481 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/date.1’: No data available 2024-03-09T09:28:06.482 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-thru-slink' -> './k/coreutils-8.5/tests/mkdir/p-thru-slink' 2024-03-09T09:28:06.482 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-thru-slink’: No data available 2024-03-09T09:28:06.483 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/chown.x' -> './k/coreutils-8.5/man/chown.x' 2024-03-09T09:28:06.483 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chown.x’: No data available 2024-03-09T09:28:06.484 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/touch.1' -> './k/coreutils-8.5/man/touch.1' 2024-03-09T09:28:06.484 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/touch.1’: No data available 2024-03-09T09:28:06.485 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mkdir/t-slash' -> './k/coreutils-8.5/tests/mkdir/t-slash' 2024-03-09T09:28:06.485 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/t-slash’: No data available 2024-03-09T09:28:06.486 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tac.x' -> './k/coreutils-8.5/man/tac.x' 2024-03-09T09:28:06.486 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tac.x’: No data available 2024-03-09T09:28:06.487 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/base64.x' -> './k/coreutils-8.5/man/base64.x' 2024-03-09T09:28:06.488 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/base64.x’: No data available 2024-03-09T09:28:06.490 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/tsort.x' -> './k/coreutils-8.5/man/tsort.x' 2024-03-09T09:28:06.491 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/tsort.x’: No data available 2024-03-09T09:28:06.492 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/shred.x' -> './k/coreutils-8.5/man/shred.x' 2024-03-09T09:28:06.492 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/shred.x’: No data available 2024-03-09T09:28:06.494 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/readlink.1' -> './k/coreutils-8.5/man/readlink.1' 2024-03-09T09:28:06.494 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/readlink.1’: No data available 2024-03-09T09:28:06.495 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/sha1sum.x' -> './k/coreutils-8.5/man/sha1sum.x' 2024-03-09T09:28:06.495 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/sha1sum.x’: No data available 2024-03-09T09:28:06.496 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/expand.x' -> './k/coreutils-8.5/man/expand.x' 2024-03-09T09:28:06.497 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/expand.x’: No data available 2024-03-09T09:28:06.498 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/test.x' -> './k/coreutils-8.5/man/test.x' 2024-03-09T09:28:06.499 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/test.x’: No data available 2024-03-09T09:28:06.500 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/chcon.1' -> './k/coreutils-8.5/man/chcon.1' 2024-03-09T09:28:06.500 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/chcon.1’: No data available 2024-03-09T09:28:06.501 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/man/cat.x' -> './k/coreutils-8.5/man/cat.x' 2024-03-09T09:28:06.501 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/man/cat.x’: No data available 2024-03-09T09:28:06.520 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ln' -> './k/coreutils-8.5/tests/ln' 2024-03-09T09:28:06.520 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ln/hard-to-sym' -> './k/coreutils-8.5/tests/ln/hard-to-sym' 2024-03-09T09:28:06.520 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/hard-to-sym’: No data available 2024-03-09T09:28:06.521 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ln/hard-backup' -> './k/coreutils-8.5/tests/ln/hard-backup' 2024-03-09T09:28:06.522 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/hard-backup’: No data available 2024-03-09T09:28:06.524 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ln/sf-1' -> './k/coreutils-8.5/tests/ln/sf-1' 2024-03-09T09:28:06.524 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/sf-1’: No data available 2024-03-09T09:28:06.526 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ln/backup-1' -> './k/coreutils-8.5/tests/ln/backup-1' 2024-03-09T09:28:06.526 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/backup-1’: No data available 2024-03-09T09:28:06.528 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ln/slash-decorated-nonexistent-dest' -> './k/coreutils-8.5/tests/ln/slash-decorated-nonexistent-dest' 2024-03-09T09:28:06.528 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/slash-decorated-nonexistent-dest’: No data available 2024-03-09T09:28:06.529 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ln/misc' -> './k/coreutils-8.5/tests/ln/misc' 2024-03-09T09:28:06.530 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/misc’: No data available 2024-03-09T09:28:06.531 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ln/target-1' -> './k/coreutils-8.5/tests/ln/target-1' 2024-03-09T09:28:06.532 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/target-1’: No data available 2024-03-09T09:28:06.565 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/install' -> './k/coreutils-8.5/tests/install' 2024-03-09T09:28:06.565 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/install/install-C-selinux' -> './k/coreutils-8.5/tests/install/install-C-selinux' 2024-03-09T09:28:06.565 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/install-C-selinux’: No data available 2024-03-09T09:28:06.566 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/install/trap' -> './k/coreutils-8.5/tests/install/trap' 2024-03-09T09:28:06.567 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/trap’: No data available 2024-03-09T09:28:06.568 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/install/basic-1' -> './k/coreutils-8.5/tests/install/basic-1' 2024-03-09T09:28:06.569 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/basic-1’: No data available 2024-03-09T09:28:06.570 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/install/strip-program' -> './k/coreutils-8.5/tests/install/strip-program' 2024-03-09T09:28:06.571 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/strip-program’: No data available 2024-03-09T09:28:06.572 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/install/d-slashdot' -> './k/coreutils-8.5/tests/install/d-slashdot' 2024-03-09T09:28:06.573 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/d-slashdot’: No data available 2024-03-09T09:28:06.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/install/install-C' -> './k/coreutils-8.5/tests/install/install-C' 2024-03-09T09:28:06.575 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/install-C’: No data available 2024-03-09T09:28:06.577 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/install/install-C-root' -> './k/coreutils-8.5/tests/install/install-C-root' 2024-03-09T09:28:06.577 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/install-C-root’: No data available 2024-03-09T09:28:06.579 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/install/create-leading' -> './k/coreutils-8.5/tests/install/create-leading' 2024-03-09T09:28:06.579 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/create-leading’: No data available 2024-03-09T09:28:06.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod' -> './k/coreutils-8.5/tests/chmod' 2024-03-09T09:28:06.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod/octal' -> './k/coreutils-8.5/tests/chmod/octal' 2024-03-09T09:28:06.872 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/octal’: No data available 2024-03-09T09:28:06.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod/umask-x' -> './k/coreutils-8.5/tests/chmod/umask-x' 2024-03-09T09:28:06.873 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/umask-x’: No data available 2024-03-09T09:28:06.875 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod/inaccessible' -> './k/coreutils-8.5/tests/chmod/inaccessible' 2024-03-09T09:28:06.876 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/inaccessible’: No data available 2024-03-09T09:28:06.877 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod/equal-x' -> './k/coreutils-8.5/tests/chmod/equal-x' 2024-03-09T09:28:06.878 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/equal-x’: No data available 2024-03-09T09:28:06.879 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod/silent' -> './k/coreutils-8.5/tests/chmod/silent' 2024-03-09T09:28:06.880 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/silent’: No data available 2024-03-09T09:28:06.882 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod/equals' -> './k/coreutils-8.5/tests/chmod/equals' 2024-03-09T09:28:06.882 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/equals’: No data available 2024-03-09T09:28:06.889 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod/usage' -> './k/coreutils-8.5/tests/chmod/usage' 2024-03-09T09:28:06.890 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/usage’: No data available 2024-03-09T09:28:06.892 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod/thru-dangling' -> './k/coreutils-8.5/tests/chmod/thru-dangling' 2024-03-09T09:28:06.892 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/thru-dangling’: No data available 2024-03-09T09:28:06.894 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod/setgid' -> './k/coreutils-8.5/tests/chmod/setgid' 2024-03-09T09:28:06.894 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/setgid’: No data available 2024-03-09T09:28:06.896 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod/no-x' -> './k/coreutils-8.5/tests/chmod/no-x' 2024-03-09T09:28:06.896 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/no-x’: No data available 2024-03-09T09:28:06.899 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chmod/c-option' -> './k/coreutils-8.5/tests/chmod/c-option' 2024-03-09T09:28:06.899 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/c-option’: No data available 2024-03-09T09:28:06.941 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib' -> './k/coreutils-8.5/lib' 2024-03-09T09:28:06.941 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/streq.h' -> './k/coreutils-8.5/lib/streq.h' 2024-03-09T09:28:06.941 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/streq.h’: No data available 2024-03-09T09:28:06.943 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/asprintf.c' -> './k/coreutils-8.5/lib/asprintf.c' 2024-03-09T09:28:06.943 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/asprintf.c’: No data available 2024-03-09T09:28:06.945 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strnlen1.c' -> './k/coreutils-8.5/lib/strnlen1.c' 2024-03-09T09:28:06.945 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnlen1.c’: No data available 2024-03-09T09:28:06.947 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dirfd.c' -> './k/coreutils-8.5/lib/dirfd.c' 2024-03-09T09:28:06.947 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirfd.c’: No data available 2024-03-09T09:28:06.949 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/safe-write.h' -> './k/coreutils-8.5/lib/safe-write.h' 2024-03-09T09:28:06.950 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/safe-write.h’: No data available 2024-03-09T09:28:06.952 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/snprintf.c' -> './k/coreutils-8.5/lib/snprintf.c' 2024-03-09T09:28:06.952 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/snprintf.c’: No data available 2024-03-09T09:28:06.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbiter.h' -> './k/coreutils-8.5/lib/mbiter.h' 2024-03-09T09:28:06.955 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbiter.h’: No data available 2024-03-09T09:28:06.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mountlist.c' -> './k/coreutils-8.5/lib/mountlist.c' 2024-03-09T09:28:06.957 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mountlist.c’: No data available 2024-03-09T09:28:06.959 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/propername.h' -> './k/coreutils-8.5/lib/propername.h' 2024-03-09T09:28:06.959 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/propername.h’: No data available 2024-03-09T09:28:06.961 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memset.c' -> './k/coreutils-8.5/lib/memset.c' 2024-03-09T09:28:06.962 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memset.c’: No data available 2024-03-09T09:28:06.963 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/nproc.c' -> './k/coreutils-8.5/lib/nproc.c' 2024-03-09T09:28:06.964 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/nproc.c’: No data available 2024-03-09T09:28:06.966 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getdelim.c' -> './k/coreutils-8.5/lib/getdelim.c' 2024-03-09T09:28:06.966 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getdelim.c’: No data available 2024-03-09T09:28:06.968 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/signbitl.c' -> './k/coreutils-8.5/lib/signbitl.c' 2024-03-09T09:28:06.968 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/signbitl.c’: No data available 2024-03-09T09:28:06.970 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/group-member.h' -> './k/coreutils-8.5/lib/group-member.h' 2024-03-09T09:28:06.970 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/group-member.h’: No data available 2024-03-09T09:28:06.972 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/long-options.c' -> './k/coreutils-8.5/lib/long-options.c' 2024-03-09T09:28:06.972 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/long-options.c’: No data available 2024-03-09T09:28:06.974 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fts_.h' -> './k/coreutils-8.5/lib/fts_.h' 2024-03-09T09:28:06.975 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fts_.h’: No data available 2024-03-09T09:28:06.978 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fts.c' -> './k/coreutils-8.5/lib/fts.c' 2024-03-09T09:28:06.978 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fts.c’: No data available 2024-03-09T09:28:06.980 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/verify.h' -> './k/coreutils-8.5/lib/verify.h' 2024-03-09T09:28:06.980 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/verify.h’: No data available 2024-03-09T09:28:06.983 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/str-two-way.h' -> './k/coreutils-8.5/lib/str-two-way.h' 2024-03-09T09:28:06.983 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/str-two-way.h’: No data available 2024-03-09T09:28:06.985 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/areadlink.c' -> './k/coreutils-8.5/lib/areadlink.c' 2024-03-09T09:28:06.985 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/areadlink.c’: No data available 2024-03-09T09:28:06.987 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/c-strtod.h' -> './k/coreutils-8.5/lib/c-strtod.h' 2024-03-09T09:28:06.987 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strtod.h’: No data available 2024-03-09T09:28:06.989 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sha256.h' -> './k/coreutils-8.5/lib/sha256.h' 2024-03-09T09:28:06.989 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha256.h’: No data available 2024-03-09T09:28:06.992 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stddef.in.h' -> './k/coreutils-8.5/lib/stddef.in.h' 2024-03-09T09:28:06.992 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stddef.in.h’: No data available 2024-03-09T09:28:06.994 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/randint.c' -> './k/coreutils-8.5/lib/randint.c' 2024-03-09T09:28:06.994 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randint.c’: No data available 2024-03-09T09:28:06.996 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dup-safer.c' -> './k/coreutils-8.5/lib/dup-safer.c' 2024-03-09T09:28:06.996 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dup-safer.c’: No data available 2024-03-09T09:28:06.999 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getpass.h' -> './k/coreutils-8.5/lib/getpass.h' 2024-03-09T09:28:06.999 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getpass.h’: No data available 2024-03-09T09:28:07.001 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/gettext.h' -> './k/coreutils-8.5/lib/gettext.h' 2024-03-09T09:28:07.001 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gettext.h’: No data available 2024-03-09T09:28:07.003 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/randint.h' -> './k/coreutils-8.5/lib/randint.h' 2024-03-09T09:28:07.003 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randint.h’: No data available 2024-03-09T09:28:07.005 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stdio-write.c' -> './k/coreutils-8.5/lib/stdio-write.c' 2024-03-09T09:28:07.006 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdio-write.c’: No data available 2024-03-09T09:28:07.008 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xgethostname.h' -> './k/coreutils-8.5/lib/xgethostname.h' 2024-03-09T09:28:07.008 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xgethostname.h’: No data available 2024-03-09T09:28:07.010 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/asnprintf.c' -> './k/coreutils-8.5/lib/asnprintf.c' 2024-03-09T09:28:07.010 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/asnprintf.c’: No data available 2024-03-09T09:28:07.012 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-solaris.gperf' -> './k/coreutils-8.5/lib/iconv_open-solaris.gperf' 2024-03-09T09:28:07.013 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-solaris.gperf’: No data available 2024-03-09T09:28:07.015 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stdio-impl.h' -> './k/coreutils-8.5/lib/stdio-impl.h' 2024-03-09T09:28:07.015 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdio-impl.h’: No data available 2024-03-09T09:28:07.023 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mpsort.h' -> './k/coreutils-8.5/lib/mpsort.h' 2024-03-09T09:28:07.024 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mpsort.h’: No data available 2024-03-09T09:28:07.025 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/closein.c' -> './k/coreutils-8.5/lib/closein.c' 2024-03-09T09:28:07.026 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/closein.c’: No data available 2024-03-09T09:28:07.028 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/hard-locale.c' -> './k/coreutils-8.5/lib/hard-locale.c' 2024-03-09T09:28:07.028 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hard-locale.c’: No data available 2024-03-09T09:28:07.030 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/utimecmp.c' -> './k/coreutils-8.5/lib/utimecmp.c' 2024-03-09T09:28:07.030 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/utimecmp.c’: No data available 2024-03-09T09:28:07.036 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/vasnprintf.c' -> './k/coreutils-8.5/lib/vasnprintf.c' 2024-03-09T09:28:07.036 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/vasnprintf.c’: No data available 2024-03-09T09:28:07.038 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/chown.c' -> './k/coreutils-8.5/lib/chown.c' 2024-03-09T09:28:07.038 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/chown.c’: No data available 2024-03-09T09:28:07.040 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/printf-args.c' -> './k/coreutils-8.5/lib/printf-args.c' 2024-03-09T09:28:07.040 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-args.c’: No data available 2024-03-09T09:28:07.042 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/netdb.in.h' -> './k/coreutils-8.5/lib/netdb.in.h' 2024-03-09T09:28:07.042 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/netdb.in.h’: No data available 2024-03-09T09:28:07.044 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strnlen.c' -> './k/coreutils-8.5/lib/strnlen.c' 2024-03-09T09:28:07.044 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnlen.c’: No data available 2024-03-09T09:28:07.046 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/errno.in.h' -> './k/coreutils-8.5/lib/errno.in.h' 2024-03-09T09:28:07.046 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/errno.in.h’: No data available 2024-03-09T09:28:07.048 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbslen.c' -> './k/coreutils-8.5/lib/mbslen.c' 2024-03-09T09:28:07.048 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbslen.c’: No data available 2024-03-09T09:28:07.049 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/obstack.h' -> './k/coreutils-8.5/lib/obstack.h' 2024-03-09T09:28:07.050 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/obstack.h’: No data available 2024-03-09T09:28:07.051 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/file-type.h' -> './k/coreutils-8.5/lib/file-type.h' 2024-03-09T09:28:07.052 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/file-type.h’: No data available 2024-03-09T09:28:07.053 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbsrtowcs.c' -> './k/coreutils-8.5/lib/mbsrtowcs.c' 2024-03-09T09:28:07.053 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsrtowcs.c’: No data available 2024-03-09T09:28:07.056 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fnmatch_loop.c' -> './k/coreutils-8.5/lib/fnmatch_loop.c' 2024-03-09T09:28:07.056 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fnmatch_loop.c’: No data available 2024-03-09T09:28:07.057 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/safe-write.c' -> './k/coreutils-8.5/lib/safe-write.c' 2024-03-09T09:28:07.057 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/safe-write.c’: No data available 2024-03-09T09:28:07.058 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mpsort.c' -> './k/coreutils-8.5/lib/mpsort.c' 2024-03-09T09:28:07.059 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mpsort.c’: No data available 2024-03-09T09:28:07.060 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/netinet_in.in.h' -> './k/coreutils-8.5/lib/netinet_in.in.h' 2024-03-09T09:28:07.060 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/netinet_in.in.h’: No data available 2024-03-09T09:28:07.062 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xfreopen.h' -> './k/coreutils-8.5/lib/xfreopen.h' 2024-03-09T09:28:07.062 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xfreopen.h’: No data available 2024-03-09T09:28:07.063 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/wctype.in.h' -> './k/coreutils-8.5/lib/wctype.in.h' 2024-03-09T09:28:07.063 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/wctype.in.h’: No data available 2024-03-09T09:28:07.065 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stripslash.c' -> './k/coreutils-8.5/lib/stripslash.c' 2024-03-09T09:28:07.065 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stripslash.c’: No data available 2024-03-09T09:28:07.066 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/closein.h' -> './k/coreutils-8.5/lib/closein.h' 2024-03-09T09:28:07.066 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/closein.h’: No data available 2024-03-09T09:28:07.067 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memchr2.c' -> './k/coreutils-8.5/lib/memchr2.c' 2024-03-09T09:28:07.068 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memchr2.c’: No data available 2024-03-09T09:28:07.069 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memcoll.c' -> './k/coreutils-8.5/lib/memcoll.c' 2024-03-09T09:28:07.070 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcoll.c’: No data available 2024-03-09T09:28:07.071 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/malloca.h' -> './k/coreutils-8.5/lib/malloca.h' 2024-03-09T09:28:07.071 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/malloca.h’: No data available 2024-03-09T09:28:07.073 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stpcpy.c' -> './k/coreutils-8.5/lib/stpcpy.c' 2024-03-09T09:28:07.074 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stpcpy.c’: No data available 2024-03-09T09:28:07.074 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/striconv.c' -> './k/coreutils-8.5/lib/striconv.c' 2024-03-09T09:28:07.075 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/striconv.c’: No data available 2024-03-09T09:28:07.076 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/basename.c' -> './k/coreutils-8.5/lib/basename.c' 2024-03-09T09:28:07.077 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/basename.c’: No data available 2024-03-09T09:28:07.079 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sha256.c' -> './k/coreutils-8.5/lib/sha256.c' 2024-03-09T09:28:07.079 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha256.c’: No data available 2024-03-09T09:28:07.081 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stdint.in.h' -> './k/coreutils-8.5/lib/stdint.in.h' 2024-03-09T09:28:07.081 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdint.in.h’: No data available 2024-03-09T09:28:07.083 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/acl-internal.h' -> './k/coreutils-8.5/lib/acl-internal.h' 2024-03-09T09:28:07.084 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/acl-internal.h’: No data available 2024-03-09T09:28:07.085 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/atexit.c' -> './k/coreutils-8.5/lib/atexit.c' 2024-03-09T09:28:07.086 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/atexit.c’: No data available 2024-03-09T09:28:07.087 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/size_max.h' -> './k/coreutils-8.5/lib/size_max.h' 2024-03-09T09:28:07.087 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/size_max.h’: No data available 2024-03-09T09:28:07.089 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/buffer-lcm.h' -> './k/coreutils-8.5/lib/buffer-lcm.h' 2024-03-09T09:28:07.089 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/buffer-lcm.h’: No data available 2024-03-09T09:28:07.092 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/idcache.h' -> './k/coreutils-8.5/lib/idcache.h' 2024-03-09T09:28:07.092 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/idcache.h’: No data available 2024-03-09T09:28:07.094 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/root-dev-ino.c' -> './k/coreutils-8.5/lib/root-dev-ino.c' 2024-03-09T09:28:07.094 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/root-dev-ino.c’: No data available 2024-03-09T09:28:07.096 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/physmem.c' -> './k/coreutils-8.5/lib/physmem.c' 2024-03-09T09:28:07.096 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/physmem.c’: No data available 2024-03-09T09:28:07.098 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mkancesdirs.c' -> './k/coreutils-8.5/lib/mkancesdirs.c' 2024-03-09T09:28:07.098 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkancesdirs.c’: No data available 2024-03-09T09:28:07.100 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/readtokens0.c' -> './k/coreutils-8.5/lib/readtokens0.c' 2024-03-09T09:28:07.100 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readtokens0.c’: No data available 2024-03-09T09:28:07.102 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fnmatch.in.h' -> './k/coreutils-8.5/lib/fnmatch.in.h' 2024-03-09T09:28:07.103 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fnmatch.in.h’: No data available 2024-03-09T09:28:07.104 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dirname.h' -> './k/coreutils-8.5/lib/dirname.h' 2024-03-09T09:28:07.105 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirname.h’: No data available 2024-03-09T09:28:07.107 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/regex_internal.c' -> './k/coreutils-8.5/lib/regex_internal.c' 2024-03-09T09:28:07.107 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regex_internal.c’: No data available 2024-03-09T09:28:07.109 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strcspn.c' -> './k/coreutils-8.5/lib/strcspn.c' 2024-03-09T09:28:07.110 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strcspn.c’: No data available 2024-03-09T09:28:07.112 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getugroups.h' -> './k/coreutils-8.5/lib/getugroups.h' 2024-03-09T09:28:07.112 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getugroups.h’: No data available 2024-03-09T09:28:07.114 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/file-set.c' -> './k/coreutils-8.5/lib/file-set.c' 2024-03-09T09:28:07.114 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/file-set.c’: No data available 2024-03-09T09:28:07.116 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sys_socket.in.h' -> './k/coreutils-8.5/lib/sys_socket.in.h' 2024-03-09T09:28:07.117 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sys_socket.in.h’: No data available 2024-03-09T09:28:07.119 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/inttostr.h' -> './k/coreutils-8.5/lib/inttostr.h' 2024-03-09T09:28:07.119 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/inttostr.h’: No data available 2024-03-09T09:28:07.121 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mkdirat.c' -> './k/coreutils-8.5/lib/mkdirat.c' 2024-03-09T09:28:07.121 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkdirat.c’: No data available 2024-03-09T09:28:07.123 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/readtokens.h' -> './k/coreutils-8.5/lib/readtokens.h' 2024-03-09T09:28:07.123 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readtokens.h’: No data available 2024-03-09T09:28:07.125 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fd-reopen.h' -> './k/coreutils-8.5/lib/fd-reopen.h' 2024-03-09T09:28:07.126 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fd-reopen.h’: No data available 2024-03-09T09:28:07.128 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/w32sock.h' -> './k/coreutils-8.5/lib/w32sock.h' 2024-03-09T09:28:07.128 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/w32sock.h’: No data available 2024-03-09T09:28:07.130 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memcasecmp.c' -> './k/coreutils-8.5/lib/memcasecmp.c' 2024-03-09T09:28:07.130 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcasecmp.c’: No data available 2024-03-09T09:28:07.132 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/at-func.c' -> './k/coreutils-8.5/lib/at-func.c' 2024-03-09T09:28:07.132 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/at-func.c’: No data available 2024-03-09T09:28:07.134 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unistd-safer.h' -> './k/coreutils-8.5/lib/unistd-safer.h' 2024-03-09T09:28:07.134 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistd-safer.h’: No data available 2024-03-09T09:28:07.136 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/filevercmp.h' -> './k/coreutils-8.5/lib/filevercmp.h' 2024-03-09T09:28:07.136 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filevercmp.h’: No data available 2024-03-09T09:28:07.139 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/c-strcasecmp.c' -> './k/coreutils-8.5/lib/c-strcasecmp.c' 2024-03-09T09:28:07.139 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strcasecmp.c’: No data available 2024-03-09T09:28:07.140 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fopen-safer.c' -> './k/coreutils-8.5/lib/fopen-safer.c' 2024-03-09T09:28:07.141 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fopen-safer.c’: No data available 2024-03-09T09:28:07.143 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/modechange.h' -> './k/coreutils-8.5/lib/modechange.h' 2024-03-09T09:28:07.143 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/modechange.h’: No data available 2024-03-09T09:28:07.145 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sys_stat.in.h' -> './k/coreutils-8.5/lib/sys_stat.in.h' 2024-03-09T09:28:07.145 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sys_stat.in.h’: No data available 2024-03-09T09:28:07.147 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/freading.c' -> './k/coreutils-8.5/lib/freading.c' 2024-03-09T09:28:07.147 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freading.c’: No data available 2024-03-09T09:28:07.148 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/utimens.h' -> './k/coreutils-8.5/lib/utimens.h' 2024-03-09T09:28:07.149 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/utimens.h’: No data available 2024-03-09T09:28:07.151 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/md5.h' -> './k/coreutils-8.5/lib/md5.h' 2024-03-09T09:28:07.151 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/md5.h’: No data available 2024-03-09T09:28:07.154 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/isnanf.c' -> './k/coreutils-8.5/lib/isnanf.c' 2024-03-09T09:28:07.154 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnanf.c’: No data available 2024-03-09T09:28:07.156 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sha1.c' -> './k/coreutils-8.5/lib/sha1.c' 2024-03-09T09:28:07.158 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha1.c’: No data available 2024-03-09T09:28:07.158 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/filenamecat-lgpl.c' -> './k/coreutils-8.5/lib/filenamecat-lgpl.c' 2024-03-09T09:28:07.158 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filenamecat-lgpl.c’: No data available 2024-03-09T09:28:07.162 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/config.hin' -> './k/coreutils-8.5/lib/config.hin' 2024-03-09T09:28:07.162 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/config.hin’: No data available 2024-03-09T09:28:07.164 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/ignore-value.h' -> './k/coreutils-8.5/lib/ignore-value.h' 2024-03-09T09:28:07.164 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ignore-value.h’: No data available 2024-03-09T09:28:07.165 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strstr.c' -> './k/coreutils-8.5/lib/strstr.c' 2024-03-09T09:28:07.166 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strstr.c’: No data available 2024-03-09T09:28:07.167 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/verror.c' -> './k/coreutils-8.5/lib/verror.c' 2024-03-09T09:28:07.168 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/verror.c’: No data available 2024-03-09T09:28:07.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/randperm.h' -> './k/coreutils-8.5/lib/randperm.h' 2024-03-09T09:28:07.171 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randperm.h’: No data available 2024-03-09T09:28:07.172 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/vasprintf.c' -> './k/coreutils-8.5/lib/vasprintf.c' 2024-03-09T09:28:07.172 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/vasprintf.c’: No data available 2024-03-09T09:28:07.174 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/c-strncasecmp.c' -> './k/coreutils-8.5/lib/c-strncasecmp.c' 2024-03-09T09:28:07.174 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strncasecmp.c’: No data available 2024-03-09T09:28:07.175 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/basename-lgpl.c' -> './k/coreutils-8.5/lib/basename-lgpl.c' 2024-03-09T09:28:07.175 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/basename-lgpl.c’: No data available 2024-03-09T09:28:07.177 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/human.h' -> './k/coreutils-8.5/lib/human.h' 2024-03-09T09:28:07.177 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/human.h’: No data available 2024-03-09T09:28:07.179 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/regex.h' -> './k/coreutils-8.5/lib/regex.h' 2024-03-09T09:28:07.179 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regex.h’: No data available 2024-03-09T09:28:07.180 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/df' -> './k/coreutils-8.5/tests/df' 2024-03-09T09:28:07.180 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/df/total-verify' -> './k/coreutils-8.5/tests/df/total-verify' 2024-03-09T09:28:07.180 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/df/total-verify’: No data available 2024-03-09T09:28:07.182 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/regcomp.c' -> './k/coreutils-8.5/lib/regcomp.c' 2024-03-09T09:28:07.182 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regcomp.c’: No data available 2024-03-09T09:28:07.183 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/df/unreadable' -> './k/coreutils-8.5/tests/df/unreadable' 2024-03-09T09:28:07.183 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/df/unreadable’: No data available 2024-03-09T09:28:07.184 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/freadahead.c' -> './k/coreutils-8.5/lib/freadahead.c' 2024-03-09T09:28:07.185 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadahead.c’: No data available 2024-03-09T09:28:07.186 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/trim.h' -> './k/coreutils-8.5/lib/trim.h' 2024-03-09T09:28:07.186 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/trim.h’: No data available 2024-03-09T09:28:07.188 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getgroups.c' -> './k/coreutils-8.5/lib/getgroups.c' 2024-03-09T09:28:07.188 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getgroups.c’: No data available 2024-03-09T09:28:07.190 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xprintf.c' -> './k/coreutils-8.5/lib/xprintf.c' 2024-03-09T09:28:07.190 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xprintf.c’: No data available 2024-03-09T09:28:07.193 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/euidaccess-stat.h' -> './k/coreutils-8.5/lib/euidaccess-stat.h' 2024-03-09T09:28:07.194 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/euidaccess-stat.h’: No data available 2024-03-09T09:28:07.197 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/rand-isaac.h' -> './k/coreutils-8.5/lib/rand-isaac.h' 2024-03-09T09:28:07.197 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/rand-isaac.h’: No data available 2024-03-09T09:28:07.199 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mgetgroups.h' -> './k/coreutils-8.5/lib/mgetgroups.h' 2024-03-09T09:28:07.199 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mgetgroups.h’: No data available 2024-03-09T09:28:07.201 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/selinux-at.h' -> './k/coreutils-8.5/lib/selinux-at.h' 2024-03-09T09:28:07.201 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/selinux-at.h’: No data available 2024-03-09T09:28:07.203 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-solaris.h' -> './k/coreutils-8.5/lib/iconv_open-solaris.h' 2024-03-09T09:28:07.204 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-solaris.h’: No data available 2024-03-09T09:28:07.206 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/setenv.c' -> './k/coreutils-8.5/lib/setenv.c' 2024-03-09T09:28:07.206 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/setenv.c’: No data available 2024-03-09T09:28:07.208 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/locale.in.h' -> './k/coreutils-8.5/lib/locale.in.h' 2024-03-09T09:28:07.208 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/locale.in.h’: No data available 2024-03-09T09:28:07.210 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du' -> './k/coreutils-8.5/tests/du' 2024-03-09T09:28:07.210 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/exclude' -> './k/coreutils-8.5/tests/du/exclude' 2024-03-09T09:28:07.210 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/exclude’: No data available 2024-03-09T09:28:07.211 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/base64.h' -> './k/coreutils-8.5/lib/base64.h' 2024-03-09T09:28:07.211 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/base64.h’: No data available 2024-03-09T09:28:07.212 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/open-safer.c' -> './k/coreutils-8.5/lib/open-safer.c' 2024-03-09T09:28:07.212 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/open-safer.c’: No data available 2024-03-09T09:28:07.213 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/one-file-system' -> './k/coreutils-8.5/tests/du/one-file-system' 2024-03-09T09:28:07.213 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/one-file-system’: No data available 2024-03-09T09:28:07.215 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/exitfail.c' -> './k/coreutils-8.5/lib/exitfail.c' 2024-03-09T09:28:07.215 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/exitfail.c’: No data available 2024-03-09T09:28:07.216 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/long-sloop' -> './k/coreutils-8.5/tests/du/long-sloop' 2024-03-09T09:28:07.216 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/long-sloop’: No data available 2024-03-09T09:28:07.216 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/closeout.h' -> './k/coreutils-8.5/lib/closeout.h' 2024-03-09T09:28:07.216 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/closeout.h’: No data available 2024-03-09T09:28:07.217 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/slink' -> './k/coreutils-8.5/tests/du/slink' 2024-03-09T09:28:07.218 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/slink’: No data available 2024-03-09T09:28:07.219 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/idcache.c' -> './k/coreutils-8.5/lib/idcache.c' 2024-03-09T09:28:07.219 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/idcache.c’: No data available 2024-03-09T09:28:07.220 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/inacc-dir' -> './k/coreutils-8.5/tests/du/inacc-dir' 2024-03-09T09:28:07.221 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/inacc-dir’: No data available 2024-03-09T09:28:07.221 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unlink.c' -> './k/coreutils-8.5/lib/unlink.c' 2024-03-09T09:28:07.221 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unlink.c’: No data available 2024-03-09T09:28:07.223 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/inaccessible-cwd' -> './k/coreutils-8.5/tests/du/inaccessible-cwd' 2024-03-09T09:28:07.223 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/inaccessible-cwd’: No data available 2024-03-09T09:28:07.223 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xfts.c' -> './k/coreutils-8.5/lib/xfts.c' 2024-03-09T09:28:07.224 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xfts.c’: No data available 2024-03-09T09:28:07.225 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memrchr.c' -> './k/coreutils-8.5/lib/memrchr.c' 2024-03-09T09:28:07.225 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memrchr.c’: No data available 2024-03-09T09:28:07.226 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/inacc-dest' -> './k/coreutils-8.5/tests/du/inacc-dest' 2024-03-09T09:28:07.226 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/inacc-dest’: No data available 2024-03-09T09:28:07.227 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/Makefile.am' -> './k/coreutils-8.5/lib/Makefile.am' 2024-03-09T09:28:07.227 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/Makefile.am’: No data available 2024-03-09T09:28:07.228 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/deref-args' -> './k/coreutils-8.5/tests/du/deref-args' 2024-03-09T09:28:07.228 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/deref-args’: No data available 2024-03-09T09:28:07.229 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstrtol.h' -> './k/coreutils-8.5/lib/xstrtol.h' 2024-03-09T09:28:07.229 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtol.h’: No data available 2024-03-09T09:28:07.231 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/deref' -> './k/coreutils-8.5/tests/du/deref' 2024-03-09T09:28:07.231 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/deref’: No data available 2024-03-09T09:28:07.231 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/gettimeofday.c' -> './k/coreutils-8.5/lib/gettimeofday.c' 2024-03-09T09:28:07.232 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gettimeofday.c’: No data available 2024-03-09T09:28:07.233 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/trailing-slash' -> './k/coreutils-8.5/tests/du/trailing-slash' 2024-03-09T09:28:07.233 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/trailing-slash’: No data available 2024-03-09T09:28:07.233 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/isnanf-nolibm.h' -> './k/coreutils-8.5/lib/isnanf-nolibm.h' 2024-03-09T09:28:07.234 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnanf-nolibm.h’: No data available 2024-03-09T09:28:07.235 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/files0-from' -> './k/coreutils-8.5/tests/du/files0-from' 2024-03-09T09:28:07.235 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/files0-from’: No data available 2024-03-09T09:28:07.236 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/chdir-long.c' -> './k/coreutils-8.5/lib/chdir-long.c' 2024-03-09T09:28:07.236 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/chdir-long.c’: No data available 2024-03-09T09:28:07.237 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/two-args' -> './k/coreutils-8.5/tests/du/two-args' 2024-03-09T09:28:07.237 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/two-args’: No data available 2024-03-09T09:28:07.238 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbchar.c' -> './k/coreutils-8.5/lib/mbchar.c' 2024-03-09T09:28:07.238 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbchar.c’: No data available 2024-03-09T09:28:07.240 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/canon-host.h' -> './k/coreutils-8.5/lib/canon-host.h' 2024-03-09T09:28:07.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/hard-link' -> './k/coreutils-8.5/tests/du/hard-link' 2024-03-09T09:28:07.240 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/canon-host.h’: No data available 2024-03-09T09:28:07.241 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/hard-link’: No data available 2024-03-09T09:28:07.242 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unsetenv.c' -> './k/coreutils-8.5/lib/unsetenv.c' 2024-03-09T09:28:07.242 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unsetenv.c’: No data available 2024-03-09T09:28:07.243 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/long-from-unreadable' -> './k/coreutils-8.5/tests/du/long-from-unreadable' 2024-03-09T09:28:07.243 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/long-from-unreadable’: No data available 2024-03-09T09:28:07.244 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/freadseek.c' -> './k/coreutils-8.5/lib/freadseek.c' 2024-03-09T09:28:07.244 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadseek.c’: No data available 2024-03-09T09:28:07.245 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/8gb' -> './k/coreutils-8.5/tests/du/8gb' 2024-03-09T09:28:07.245 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/8gb’: No data available 2024-03-09T09:28:07.246 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xnanosleep.c' -> './k/coreutils-8.5/lib/xnanosleep.c' 2024-03-09T09:28:07.246 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xnanosleep.c’: No data available 2024-03-09T09:28:07.247 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/2g' -> './k/coreutils-8.5/tests/du/2g' 2024-03-09T09:28:07.247 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/2g’: No data available 2024-03-09T09:28:07.248 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/nanosleep.c' -> './k/coreutils-8.5/lib/nanosleep.c' 2024-03-09T09:28:07.248 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/nanosleep.c’: No data available 2024-03-09T09:28:07.249 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/restore-wd' -> './k/coreutils-8.5/tests/du/restore-wd' 2024-03-09T09:28:07.250 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/restore-wd’: No data available 2024-03-09T09:28:07.250 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/ctype.in.h' -> './k/coreutils-8.5/lib/ctype.in.h' 2024-03-09T09:28:07.250 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ctype.in.h’: No data available 2024-03-09T09:28:07.252 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xmalloc.c' -> './k/coreutils-8.5/lib/xmalloc.c' 2024-03-09T09:28:07.252 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xmalloc.c’: No data available 2024-03-09T09:28:07.253 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/slash' -> './k/coreutils-8.5/tests/du/slash' 2024-03-09T09:28:07.253 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/slash’: No data available 2024-03-09T09:28:07.255 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/no-x' -> './k/coreutils-8.5/tests/du/no-x' 2024-03-09T09:28:07.255 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/no-x’: No data available 2024-03-09T09:28:07.257 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/fd-leak' -> './k/coreutils-8.5/tests/du/fd-leak' 2024-03-09T09:28:07.257 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/fd-leak’: No data available 2024-03-09T09:28:07.267 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/no-deref' -> './k/coreutils-8.5/tests/du/no-deref' 2024-03-09T09:28:07.267 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/no-deref’: No data available 2024-03-09T09:28:07.269 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/du/basic' -> './k/coreutils-8.5/tests/du/basic' 2024-03-09T09:28:07.269 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/basic’: No data available 2024-03-09T09:28:07.323 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls' -> './k/coreutils-8.5/tests/ls' 2024-03-09T09:28:07.323 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/readdir-mountpoint-inode' -> './k/coreutils-8.5/tests/ls/readdir-mountpoint-inode' 2024-03-09T09:28:07.324 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/readdir-mountpoint-inode’: No data available 2024-03-09T09:28:07.326 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/multihardlink' -> './k/coreutils-8.5/tests/ls/multihardlink' 2024-03-09T09:28:07.326 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/multihardlink’: No data available 2024-03-09T09:28:07.329 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/color-dtype-dir' -> './k/coreutils-8.5/tests/ls/color-dtype-dir' 2024-03-09T09:28:07.329 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/color-dtype-dir’: No data available 2024-03-09T09:28:07.331 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/file-type' -> './k/coreutils-8.5/tests/ls/file-type' 2024-03-09T09:28:07.332 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/file-type’: No data available 2024-03-09T09:28:07.334 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/rt-1' -> './k/coreutils-8.5/tests/ls/rt-1' 2024-03-09T09:28:07.335 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/rt-1’: No data available 2024-03-09T09:28:07.337 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/stat-free-symlinks' -> './k/coreutils-8.5/tests/ls/stat-free-symlinks' 2024-03-09T09:28:07.337 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/stat-free-symlinks’: No data available 2024-03-09T09:28:07.340 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/no-cap' -> './k/coreutils-8.5/tests/ls/no-cap' 2024-03-09T09:28:07.340 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/no-cap’: No data available 2024-03-09T09:28:07.343 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/capability' -> './k/coreutils-8.5/tests/ls/capability' 2024-03-09T09:28:07.343 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/capability’: No data available 2024-03-09T09:28:07.346 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/inode' -> './k/coreutils-8.5/tests/ls/inode' 2024-03-09T09:28:07.346 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/inode’: No data available 2024-03-09T09:28:07.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/color-norm' -> './k/coreutils-8.5/tests/ls/color-norm' 2024-03-09T09:28:07.348 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/color-norm’: No data available 2024-03-09T09:28:07.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/abmon-align' -> './k/coreutils-8.5/tests/ls/abmon-align' 2024-03-09T09:28:07.350 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/abmon-align’: No data available 2024-03-09T09:28:07.351 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/dangle' -> './k/coreutils-8.5/tests/ls/dangle' 2024-03-09T09:28:07.352 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/dangle’: No data available 2024-03-09T09:28:07.354 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/dired' -> './k/coreutils-8.5/tests/ls/dired' 2024-03-09T09:28:07.354 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/dired’: No data available 2024-03-09T09:28:07.356 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/m-option' -> './k/coreutils-8.5/tests/ls/m-option' 2024-03-09T09:28:07.356 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/m-option’: No data available 2024-03-09T09:28:07.358 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/nameless-uid' -> './k/coreutils-8.5/tests/ls/nameless-uid' 2024-03-09T09:28:07.358 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/nameless-uid’: No data available 2024-03-09T09:28:07.362 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/proc-selinux-segfault' -> './k/coreutils-8.5/tests/ls/proc-selinux-segfault' 2024-03-09T09:28:07.362 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/proc-selinux-segfault’: No data available 2024-03-09T09:28:07.362 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/stat-vs-dirent' -> './k/coreutils-8.5/tests/ls/stat-vs-dirent' 2024-03-09T09:28:07.363 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/stat-vs-dirent’: No data available 2024-03-09T09:28:07.364 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/color-clear-to-eol' -> './k/coreutils-8.5/tests/ls/color-clear-to-eol' 2024-03-09T09:28:07.365 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/color-clear-to-eol’: No data available 2024-03-09T09:28:07.366 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/no-arg' -> './k/coreutils-8.5/tests/ls/no-arg' 2024-03-09T09:28:07.367 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/no-arg’: No data available 2024-03-09T09:28:07.369 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/follow-slink' -> './k/coreutils-8.5/tests/ls/follow-slink' 2024-03-09T09:28:07.369 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/follow-slink’: No data available 2024-03-09T09:28:07.371 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/symlink-slash' -> './k/coreutils-8.5/tests/ls/symlink-slash' 2024-03-09T09:28:07.371 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/symlink-slash’: No data available 2024-03-09T09:28:07.372 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/stat-dtype' -> './k/coreutils-8.5/tests/ls/stat-dtype' 2024-03-09T09:28:07.372 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/stat-dtype’: No data available 2024-03-09T09:28:07.374 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/x-option' -> './k/coreutils-8.5/tests/ls/x-option' 2024-03-09T09:28:07.374 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/x-option’: No data available 2024-03-09T09:28:07.376 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/infloop' -> './k/coreutils-8.5/tests/ls/infloop' 2024-03-09T09:28:07.376 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/infloop’: No data available 2024-03-09T09:28:07.378 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/stat-failed' -> './k/coreutils-8.5/tests/ls/stat-failed' 2024-03-09T09:28:07.378 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/stat-failed’: No data available 2024-03-09T09:28:07.380 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/ls/recursive' -> './k/coreutils-8.5/tests/ls/recursive' 2024-03-09T09:28:07.380 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/recursive’: No data available 2024-03-09T09:28:07.434 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv' -> './k/coreutils-8.5/tests/mv' 2024-03-09T09:28:07.434 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/mv-n' -> './k/coreutils-8.5/tests/mv/mv-n' 2024-03-09T09:28:07.435 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/mv-n’: No data available 2024-03-09T09:28:07.435 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/to-symlink' -> './k/coreutils-8.5/tests/mv/to-symlink' 2024-03-09T09:28:07.435 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/to-symlink’: No data available 2024-03-09T09:28:07.437 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/part-fail' -> './k/coreutils-8.5/tests/mv/part-fail' 2024-03-09T09:28:07.437 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/part-fail’: No data available 2024-03-09T09:28:07.439 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/hard-4' -> './k/coreutils-8.5/tests/mv/hard-4' 2024-03-09T09:28:07.439 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/hard-4’: No data available 2024-03-09T09:28:07.442 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/leak-fd' -> './k/coreutils-8.5/tests/mv/leak-fd' 2024-03-09T09:28:07.442 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/leak-fd’: No data available 2024-03-09T09:28:07.443 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/no-target-dir' -> './k/coreutils-8.5/tests/mv/no-target-dir' 2024-03-09T09:28:07.443 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/no-target-dir’: No data available 2024-03-09T09:28:07.445 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/part-rename' -> './k/coreutils-8.5/tests/mv/part-rename' 2024-03-09T09:28:07.445 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/part-rename’: No data available 2024-03-09T09:28:07.446 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/dir-file' -> './k/coreutils-8.5/tests/mv/dir-file' 2024-03-09T09:28:07.447 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/dir-file’: No data available 2024-03-09T09:28:07.448 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/hard-2' -> './k/coreutils-8.5/tests/mv/hard-2' 2024-03-09T09:28:07.448 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/hard-2’: No data available 2024-03-09T09:28:07.449 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/part-hardlink' -> './k/coreutils-8.5/tests/mv/part-hardlink' 2024-03-09T09:28:07.449 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/part-hardlink’: No data available 2024-03-09T09:28:07.451 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/sticky-to-xpart' -> './k/coreutils-8.5/tests/mv/sticky-to-xpart' 2024-03-09T09:28:07.451 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/sticky-to-xpart’: No data available 2024-03-09T09:28:07.453 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/mv-special-1' -> './k/coreutils-8.5/tests/mv/mv-special-1' 2024-03-09T09:28:07.453 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/mv-special-1’: No data available 2024-03-09T09:28:07.454 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/trailing-slash' -> './k/coreutils-8.5/tests/mv/trailing-slash' 2024-03-09T09:28:07.454 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/trailing-slash’: No data available 2024-03-09T09:28:07.456 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/dir2dir' -> './k/coreutils-8.5/tests/mv/dir2dir' 2024-03-09T09:28:07.456 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/dir2dir’: No data available 2024-03-09T09:28:07.457 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/into-self-2' -> './k/coreutils-8.5/tests/mv/into-self-2' 2024-03-09T09:28:07.457 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/into-self-2’: No data available 2024-03-09T09:28:07.458 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/backup-dir' -> './k/coreutils-8.5/tests/mv/backup-dir' 2024-03-09T09:28:07.458 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/backup-dir’: No data available 2024-03-09T09:28:07.460 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/diag' -> './k/coreutils-8.5/tests/mv/diag' 2024-03-09T09:28:07.460 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/diag’: No data available 2024-03-09T09:28:07.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/atomic' -> './k/coreutils-8.5/tests/mv/atomic' 2024-03-09T09:28:07.468 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/atomic’: No data available 2024-03-09T09:28:07.469 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/force' -> './k/coreutils-8.5/tests/mv/force' 2024-03-09T09:28:07.469 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/force’: No data available 2024-03-09T09:28:07.470 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/into-self-3' -> './k/coreutils-8.5/tests/mv/into-self-3' 2024-03-09T09:28:07.471 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/into-self-3’: No data available 2024-03-09T09:28:07.472 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/into-self' -> './k/coreutils-8.5/tests/mv/into-self' 2024-03-09T09:28:07.472 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/into-self’: No data available 2024-03-09T09:28:07.473 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/i-2' -> './k/coreutils-8.5/tests/mv/i-2' 2024-03-09T09:28:07.473 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-2’: No data available 2024-03-09T09:28:07.474 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/childproof' -> './k/coreutils-8.5/tests/mv/childproof' 2024-03-09T09:28:07.474 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/childproof’: No data available 2024-03-09T09:28:07.476 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/perm-1' -> './k/coreutils-8.5/tests/mv/perm-1' 2024-03-09T09:28:07.476 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/perm-1’: No data available 2024-03-09T09:28:07.477 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/i-3' -> './k/coreutils-8.5/tests/mv/i-3' 2024-03-09T09:28:07.478 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-3’: No data available 2024-03-09T09:28:07.479 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/partition-perm' -> './k/coreutils-8.5/tests/mv/partition-perm' 2024-03-09T09:28:07.479 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/partition-perm’: No data available 2024-03-09T09:28:07.481 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/part-symlink' -> './k/coreutils-8.5/tests/mv/part-symlink' 2024-03-09T09:28:07.481 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/part-symlink’: No data available 2024-03-09T09:28:07.482 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/backup-is-src' -> './k/coreutils-8.5/tests/mv/backup-is-src' 2024-03-09T09:28:07.483 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/backup-is-src’: No data available 2024-03-09T09:28:07.483 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/i-1' -> './k/coreutils-8.5/tests/mv/i-1' 2024-03-09T09:28:07.484 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-1’: No data available 2024-03-09T09:28:07.485 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/update' -> './k/coreutils-8.5/tests/mv/update' 2024-03-09T09:28:07.485 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/update’: No data available 2024-03-09T09:28:07.486 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/i-4' -> './k/coreutils-8.5/tests/mv/i-4' 2024-03-09T09:28:07.486 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-4’: No data available 2024-03-09T09:28:07.487 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/hard-verbose' -> './k/coreutils-8.5/tests/mv/hard-verbose' 2024-03-09T09:28:07.488 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/hard-verbose’: No data available 2024-03-09T09:28:07.489 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/dup-source' -> './k/coreutils-8.5/tests/mv/dup-source' 2024-03-09T09:28:07.489 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/dup-source’: No data available 2024-03-09T09:28:07.491 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/i-link-no' -> './k/coreutils-8.5/tests/mv/i-link-no' 2024-03-09T09:28:07.491 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-link-no’: No data available 2024-03-09T09:28:07.492 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/atomic2' -> './k/coreutils-8.5/tests/mv/atomic2' 2024-03-09T09:28:07.493 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/atomic2’: No data available 2024-03-09T09:28:07.494 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/into-self-4' -> './k/coreutils-8.5/tests/mv/into-self-4' 2024-03-09T09:28:07.494 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/into-self-4’: No data available 2024-03-09T09:28:07.496 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/hard-link-1' -> './k/coreutils-8.5/tests/mv/hard-link-1' 2024-03-09T09:28:07.496 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/hard-link-1’: No data available 2024-03-09T09:28:07.498 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/i-5' -> './k/coreutils-8.5/tests/mv/i-5' 2024-03-09T09:28:07.498 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-5’: No data available 2024-03-09T09:28:07.499 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/acl' -> './k/coreutils-8.5/tests/mv/acl' 2024-03-09T09:28:07.500 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/acl’: No data available 2024-03-09T09:28:07.502 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/mv/hard-3' -> './k/coreutils-8.5/tests/mv/hard-3' 2024-03-09T09:28:07.502 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/hard-3’: No data available 2024-03-09T09:28:07.846 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch' -> './k/coreutils-8.5/tests/touch' 2024-03-09T09:28:07.846 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/empty-file' -> './k/coreutils-8.5/tests/touch/empty-file' 2024-03-09T09:28:07.846 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/empty-file’: No data available 2024-03-09T09:28:07.848 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/read-only' -> './k/coreutils-8.5/tests/touch/read-only' 2024-03-09T09:28:07.849 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/read-only’: No data available 2024-03-09T09:28:07.851 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/fail-diag' -> './k/coreutils-8.5/tests/touch/fail-diag' 2024-03-09T09:28:07.851 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/fail-diag’: No data available 2024-03-09T09:28:07.853 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/trailing-slash' -> './k/coreutils-8.5/tests/touch/trailing-slash' 2024-03-09T09:28:07.854 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/trailing-slash’: No data available 2024-03-09T09:28:07.855 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/no-dereference' -> './k/coreutils-8.5/tests/touch/no-dereference' 2024-03-09T09:28:07.855 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/no-dereference’: No data available 2024-03-09T09:28:07.858 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/dir-1' -> './k/coreutils-8.5/tests/touch/dir-1' 2024-03-09T09:28:07.859 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/dir-1’: No data available 2024-03-09T09:28:07.861 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/dangling-symlink' -> './k/coreutils-8.5/tests/touch/dangling-symlink' 2024-03-09T09:28:07.861 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/dangling-symlink’: No data available 2024-03-09T09:28:07.863 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/now-owned-by-other' -> './k/coreutils-8.5/tests/touch/now-owned-by-other' 2024-03-09T09:28:07.864 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/now-owned-by-other’: No data available 2024-03-09T09:28:07.865 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/no-create-missing' -> './k/coreutils-8.5/tests/touch/no-create-missing' 2024-03-09T09:28:07.866 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/no-create-missing’: No data available 2024-03-09T09:28:07.868 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/no-rights' -> './k/coreutils-8.5/tests/touch/no-rights' 2024-03-09T09:28:07.868 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/no-rights’: No data available 2024-03-09T09:28:07.871 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/obsolescent' -> './k/coreutils-8.5/tests/touch/obsolescent' 2024-03-09T09:28:07.871 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/obsolescent’: No data available 2024-03-09T09:28:07.873 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/60-seconds' -> './k/coreutils-8.5/tests/touch/60-seconds' 2024-03-09T09:28:07.873 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/60-seconds’: No data available 2024-03-09T09:28:07.877 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/not-owner' -> './k/coreutils-8.5/tests/touch/not-owner' 2024-03-09T09:28:07.880 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/not-owner’: No data available 2024-03-09T09:28:07.880 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/relative' -> './k/coreutils-8.5/tests/touch/relative' 2024-03-09T09:28:07.881 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/relative’: No data available 2024-03-09T09:28:07.881 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/touch/fifo' -> './k/coreutils-8.5/tests/touch/fifo' 2024-03-09T09:28:07.881 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/fifo’: No data available 2024-03-09T09:28:07.941 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc' -> './k/coreutils-8.5/tests/misc' 2024-03-09T09:28:07.941 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/basename' -> './k/coreutils-8.5/tests/misc/basename' 2024-03-09T09:28:07.942 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/basename’: No data available 2024-03-09T09:28:07.944 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/stat-fmt' -> './k/coreutils-8.5/tests/misc/stat-fmt' 2024-03-09T09:28:07.944 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stat-fmt’: No data available 2024-03-09T09:28:07.946 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/pr' -> './k/coreutils-8.5/tests/misc/pr' 2024-03-09T09:28:07.946 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/pr’: No data available 2024-03-09T09:28:07.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/dirname' -> './k/coreutils-8.5/tests/misc/dirname' 2024-03-09T09:28:07.951 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/dirname’: No data available 2024-03-09T09:28:07.952 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/chcon-fail' -> './k/coreutils-8.5/tests/misc/chcon-fail' 2024-03-09T09:28:07.952 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/chcon-fail’: No data available 2024-03-09T09:28:07.954 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/head-elide-tail' -> './k/coreutils-8.5/tests/misc/head-elide-tail' 2024-03-09T09:28:07.954 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/head-elide-tail’: No data available 2024-03-09T09:28:07.956 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/chroot-credentials' -> './k/coreutils-8.5/tests/misc/chroot-credentials' 2024-03-09T09:28:07.956 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/chroot-credentials’: No data available 2024-03-09T09:28:08.228 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strtoul.c' -> './k/coreutils-8.5/lib/strtoul.c' 2024-03-09T09:28:08.229 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtoul.c’: No data available 2024-03-09T09:28:08.232 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sha512.h' -> './k/coreutils-8.5/lib/sha512.h' 2024-03-09T09:28:08.233 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha512.h’: No data available 2024-03-09T09:28:08.235 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dirchownmod.h' -> './k/coreutils-8.5/lib/dirchownmod.h' 2024-03-09T09:28:08.235 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirchownmod.h’: No data available 2024-03-09T09:28:08.236 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/freadptr.h' -> './k/coreutils-8.5/lib/freadptr.h' 2024-03-09T09:28:08.236 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadptr.h’: No data available 2024-03-09T09:28:08.237 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getfilecon.c' -> './k/coreutils-8.5/lib/getfilecon.c' 2024-03-09T09:28:08.238 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getfilecon.c’: No data available 2024-03-09T09:28:08.240 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/alignof.h' -> './k/coreutils-8.5/lib/alignof.h' 2024-03-09T09:28:08.241 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/alignof.h’: No data available 2024-03-09T09:28:08.242 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mempcpy.c' -> './k/coreutils-8.5/lib/mempcpy.c' 2024-03-09T09:28:08.242 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mempcpy.c’: No data available 2024-03-09T09:28:08.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/savedir.h' -> './k/coreutils-8.5/lib/savedir.h' 2024-03-09T09:28:08.257 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/savedir.h’: No data available 2024-03-09T09:28:08.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/areadlink-with-size.c' -> './k/coreutils-8.5/lib/areadlink-with-size.c' 2024-03-09T09:28:08.258 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/areadlink-with-size.c’: No data available 2024-03-09T09:28:08.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/inttypes.in.h' -> './k/coreutils-8.5/lib/inttypes.in.h' 2024-03-09T09:28:08.259 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/inttypes.in.h’: No data available 2024-03-09T09:28:08.424 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/config.charset' -> './k/coreutils-8.5/lib/config.charset' 2024-03-09T09:28:08.425 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/config.charset’: No data available 2024-03-09T09:28:08.431 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstrtold.c' -> './k/coreutils-8.5/lib/xstrtold.c' 2024-03-09T09:28:08.431 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtold.c’: No data available 2024-03-09T09:28:08.431 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fcntl-safer.h' -> './k/coreutils-8.5/lib/fcntl-safer.h' 2024-03-09T09:28:08.432 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fcntl-safer.h’: No data available 2024-03-09T09:28:08.433 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstrndup.c' -> './k/coreutils-8.5/lib/xstrndup.c' 2024-03-09T09:28:08.433 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrndup.c’: No data available 2024-03-09T09:28:08.434 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/write.c' -> './k/coreutils-8.5/lib/write.c' 2024-03-09T09:28:08.434 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/write.c’: No data available 2024-03-09T09:28:08.435 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/math.in.h' -> './k/coreutils-8.5/lib/math.in.h' 2024-03-09T09:28:08.435 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/math.in.h’: No data available 2024-03-09T09:28:08.438 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/openat-priv.h' -> './k/coreutils-8.5/lib/openat-priv.h' 2024-03-09T09:28:08.438 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat-priv.h’: No data available 2024-03-09T09:28:08.439 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/vasnprintf.h' -> './k/coreutils-8.5/lib/vasnprintf.h' 2024-03-09T09:28:08.440 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/vasnprintf.h’: No data available 2024-03-09T09:28:08.443 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getcwd.c' -> './k/coreutils-8.5/lib/getcwd.c' 2024-03-09T09:28:08.443 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getcwd.c’: No data available 2024-03-09T09:28:08.444 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stdbool.in.h' -> './k/coreutils-8.5/lib/stdbool.in.h' 2024-03-09T09:28:08.445 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdbool.in.h’: No data available 2024-03-09T09:28:08.446 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strtol.c' -> './k/coreutils-8.5/lib/strtol.c' 2024-03-09T09:28:08.446 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtol.c’: No data available 2024-03-09T09:28:08.448 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/argmatch.c' -> './k/coreutils-8.5/lib/argmatch.c' 2024-03-09T09:28:08.448 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/argmatch.c’: No data available 2024-03-09T09:28:08.450 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/acl_entries.c' -> './k/coreutils-8.5/lib/acl_entries.c' 2024-03-09T09:28:08.450 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/acl_entries.c’: No data available 2024-03-09T09:28:08.453 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/progname.c' -> './k/coreutils-8.5/lib/progname.c' 2024-03-09T09:28:08.453 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/progname.c’: No data available 2024-03-09T09:28:08.455 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mktime-internal.h' -> './k/coreutils-8.5/lib/mktime-internal.h' 2024-03-09T09:28:08.455 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mktime-internal.h’: No data available 2024-03-09T09:28:08.457 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/printf-frexpl.c' -> './k/coreutils-8.5/lib/printf-frexpl.c' 2024-03-09T09:28:08.457 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-frexpl.c’: No data available 2024-03-09T09:28:08.459 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/offtostr.c' -> './k/coreutils-8.5/lib/offtostr.c' 2024-03-09T09:28:08.460 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/offtostr.c’: No data available 2024-03-09T09:28:08.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memchr2.h' -> './k/coreutils-8.5/lib/memchr2.h' 2024-03-09T09:28:08.462 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memchr2.h’: No data available 2024-03-09T09:28:08.464 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstrtoumax.c' -> './k/coreutils-8.5/lib/xstrtoumax.c' 2024-03-09T09:28:08.464 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtoumax.c’: No data available 2024-03-09T09:28:08.468 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getdate.y' -> './k/coreutils-8.5/lib/getdate.y' 2024-03-09T09:28:08.468 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getdate.y’: No data available 2024-03-09T09:28:08.470 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/close-hook.h' -> './k/coreutils-8.5/lib/close-hook.h' 2024-03-09T09:28:08.470 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/close-hook.h’: No data available 2024-03-09T09:28:08.473 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/printf-parse.c' -> './k/coreutils-8.5/lib/printf-parse.c' 2024-03-09T09:28:08.473 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-parse.c’: No data available 2024-03-09T09:28:08.474 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/isnand.c' -> './k/coreutils-8.5/lib/isnand.c' 2024-03-09T09:28:08.475 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnand.c’: No data available 2024-03-09T09:28:08.477 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/timespec.h' -> './k/coreutils-8.5/lib/timespec.h' 2024-03-09T09:28:08.477 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/timespec.h’: No data available 2024-03-09T09:28:08.479 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getusershell.c' -> './k/coreutils-8.5/lib/getusershell.c' 2024-03-09T09:28:08.479 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getusershell.c’: No data available 2024-03-09T09:28:08.481 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstrtod.c' -> './k/coreutils-8.5/lib/xstrtod.c' 2024-03-09T09:28:08.481 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtod.c’: No data available 2024-03-09T09:28:08.483 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/verror.h' -> './k/coreutils-8.5/lib/verror.h' 2024-03-09T09:28:08.484 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/verror.h’: No data available 2024-03-09T09:28:08.486 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/same.c' -> './k/coreutils-8.5/lib/same.c' 2024-03-09T09:28:08.486 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/same.c’: No data available 2024-03-09T09:28:08.488 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/quote.c' -> './k/coreutils-8.5/lib/quote.c' 2024-03-09T09:28:08.488 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/quote.c’: No data available 2024-03-09T09:28:08.490 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/raise.c' -> './k/coreutils-8.5/lib/raise.c' 2024-03-09T09:28:08.490 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/raise.c’: No data available 2024-03-09T09:28:08.492 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/link.c' -> './k/coreutils-8.5/lib/link.c' 2024-03-09T09:28:08.492 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/link.c’: No data available 2024-03-09T09:28:08.494 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/cloexec.h' -> './k/coreutils-8.5/lib/cloexec.h' 2024-03-09T09:28:08.494 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/cloexec.h’: No data available 2024-03-09T09:28:08.496 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sys_utsname.in.h' -> './k/coreutils-8.5/lib/sys_utsname.in.h' 2024-03-09T09:28:08.497 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sys_utsname.in.h’: No data available 2024-03-09T09:28:08.506 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/cloexec.c' -> './k/coreutils-8.5/lib/cloexec.c' 2024-03-09T09:28:08.506 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/cloexec.c’: No data available 2024-03-09T09:28:08.508 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/vfprintf.c' -> './k/coreutils-8.5/lib/vfprintf.c' 2024-03-09T09:28:08.508 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/vfprintf.c’: No data available 2024-03-09T09:28:08.510 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/opendir-safer.c' -> './k/coreutils-8.5/lib/opendir-safer.c' 2024-03-09T09:28:08.510 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/opendir-safer.c’: No data available 2024-03-09T09:28:08.512 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/hash-pjw.h' -> './k/coreutils-8.5/lib/hash-pjw.h' 2024-03-09T09:28:08.512 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash-pjw.h’: No data available 2024-03-09T09:28:08.515 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/uname.c' -> './k/coreutils-8.5/lib/uname.c' 2024-03-09T09:28:08.515 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/uname.c’: No data available 2024-03-09T09:28:08.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/full-read.c' -> './k/coreutils-8.5/lib/full-read.c' 2024-03-09T09:28:08.516 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/full-read.c’: No data available 2024-03-09T09:28:08.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/priv-set.h' -> './k/coreutils-8.5/lib/priv-set.h' 2024-03-09T09:28:08.518 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/priv-set.h’: No data available 2024-03-09T09:28:08.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/close.c' -> './k/coreutils-8.5/lib/close.c' 2024-03-09T09:28:08.520 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/close.c’: No data available 2024-03-09T09:28:08.522 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fpending.h' -> './k/coreutils-8.5/lib/fpending.h' 2024-03-09T09:28:08.522 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fpending.h’: No data available 2024-03-09T09:28:08.524 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/i-ring.h' -> './k/coreutils-8.5/lib/i-ring.h' 2024-03-09T09:28:08.524 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/i-ring.h’: No data available 2024-03-09T09:28:08.527 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/nl_langinfo.c' -> './k/coreutils-8.5/lib/nl_langinfo.c' 2024-03-09T09:28:08.527 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/nl_langinfo.c’: No data available 2024-03-09T09:28:08.529 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xmemxfrm.h' -> './k/coreutils-8.5/lib/xmemxfrm.h' 2024-03-09T09:28:08.529 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xmemxfrm.h’: No data available 2024-03-09T09:28:08.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/c-strcase.h' -> './k/coreutils-8.5/lib/c-strcase.h' 2024-03-09T09:28:08.532 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strcase.h’: No data available 2024-03-09T09:28:08.533 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fchdir.c' -> './k/coreutils-8.5/lib/fchdir.c' 2024-03-09T09:28:08.534 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fchdir.c’: No data available 2024-03-09T09:28:08.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/canon-host.c' -> './k/coreutils-8.5/lib/canon-host.c' 2024-03-09T09:28:08.536 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/canon-host.c’: No data available 2024-03-09T09:28:08.538 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/signbitf.c' -> './k/coreutils-8.5/lib/signbitf.c' 2024-03-09T09:28:08.538 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/signbitf.c’: No data available 2024-03-09T09:28:08.541 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xnanosleep.h' -> './k/coreutils-8.5/lib/xnanosleep.h' 2024-03-09T09:28:08.541 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xnanosleep.h’: No data available 2024-03-09T09:28:08.543 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/tempname.c' -> './k/coreutils-8.5/lib/tempname.c' 2024-03-09T09:28:08.543 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/tempname.c’: No data available 2024-03-09T09:28:08.545 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xreadlink.c' -> './k/coreutils-8.5/lib/xreadlink.c' 2024-03-09T09:28:08.545 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xreadlink.c’: No data available 2024-03-09T09:28:08.547 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xgetcwd.h' -> './k/coreutils-8.5/lib/xgetcwd.h' 2024-03-09T09:28:08.548 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xgetcwd.h’: No data available 2024-03-09T09:28:08.550 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/regex.c' -> './k/coreutils-8.5/lib/regex.c' 2024-03-09T09:28:08.550 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regex.c’: No data available 2024-03-09T09:28:08.552 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/diacrit.h' -> './k/coreutils-8.5/lib/diacrit.h' 2024-03-09T09:28:08.552 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/diacrit.h’: No data available 2024-03-09T09:28:08.554 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/posixver.c' -> './k/coreutils-8.5/lib/posixver.c' 2024-03-09T09:28:08.554 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/posixver.c’: No data available 2024-03-09T09:28:08.556 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getugroups.c' -> './k/coreutils-8.5/lib/getugroups.c' 2024-03-09T09:28:08.556 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getugroups.c’: No data available 2024-03-09T09:28:08.559 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/readutmp.h' -> './k/coreutils-8.5/lib/readutmp.h' 2024-03-09T09:28:08.559 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readutmp.h’: No data available 2024-03-09T09:28:08.561 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/quote.h' -> './k/coreutils-8.5/lib/quote.h' 2024-03-09T09:28:08.561 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/quote.h’: No data available 2024-03-09T09:28:08.564 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/quotearg.c' -> './k/coreutils-8.5/lib/quotearg.c' 2024-03-09T09:28:08.564 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/quotearg.c’: No data available 2024-03-09T09:28:08.566 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/filevercmp.c' -> './k/coreutils-8.5/lib/filevercmp.c' 2024-03-09T09:28:08.566 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filevercmp.c’: No data available 2024-03-09T09:28:08.568 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/printf-args.h' -> './k/coreutils-8.5/lib/printf-args.h' 2024-03-09T09:28:08.569 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-args.h’: No data available 2024-03-09T09:28:08.570 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mountlist.h' -> './k/coreutils-8.5/lib/mountlist.h' 2024-03-09T09:28:08.570 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mountlist.h’: No data available 2024-03-09T09:28:08.573 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstrtol-error.c' -> './k/coreutils-8.5/lib/xstrtol-error.c' 2024-03-09T09:28:08.573 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtol-error.c’: No data available 2024-03-09T09:28:08.576 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/rename.c' -> './k/coreutils-8.5/lib/rename.c' 2024-03-09T09:28:08.576 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/rename.c’: No data available 2024-03-09T09:28:08.577 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sha1.h' -> './k/coreutils-8.5/lib/sha1.h' 2024-03-09T09:28:08.578 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha1.h’: No data available 2024-03-09T09:28:08.580 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strndup.c' -> './k/coreutils-8.5/lib/strndup.c' 2024-03-09T09:28:08.580 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strndup.c’: No data available 2024-03-09T09:28:08.581 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/nl' -> './k/coreutils-8.5/tests/misc/nl' 2024-03-09T09:28:08.581 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nl’: No data available 2024-03-09T09:28:08.582 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/priv-set.c' -> './k/coreutils-8.5/lib/priv-set.c' 2024-03-09T09:28:08.582 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/priv-set.c’: No data available 2024-03-09T09:28:08.586 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xfts.h' -> './k/coreutils-8.5/lib/xfts.h' 2024-03-09T09:28:08.586 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xfts.h’: No data available 2024-03-09T09:28:08.587 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/tee-dash' -> './k/coreutils-8.5/tests/misc/tee-dash' 2024-03-09T09:28:08.587 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tee-dash’: No data available 2024-03-09T09:28:08.589 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/argv-iter.c' -> './k/coreutils-8.5/lib/argv-iter.c' 2024-03-09T09:28:08.589 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/argv-iter.c’: No data available 2024-03-09T09:28:08.590 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/stat-hyphen' -> './k/coreutils-8.5/tests/misc/stat-hyphen' 2024-03-09T09:28:08.590 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stat-hyphen’: No data available 2024-03-09T09:28:08.590 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-rand' -> './k/coreutils-8.5/tests/misc/sort-rand' 2024-03-09T09:28:08.590 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-rand’: No data available 2024-03-09T09:28:08.592 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/expr' -> './k/coreutils-8.5/tests/misc/expr' 2024-03-09T09:28:08.592 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/expr’: No data available 2024-03-09T09:28:08.592 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stdio--.h' -> './k/coreutils-8.5/lib/stdio--.h' 2024-03-09T09:28:08.593 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdio--.h’: No data available 2024-03-09T09:28:08.593 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/fmt-long-line' -> './k/coreutils-8.5/tests/misc/fmt-long-line' 2024-03-09T09:28:08.594 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/fmt-long-line’: No data available 2024-03-09T09:28:08.594 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/yesno.c' -> './k/coreutils-8.5/lib/yesno.c' 2024-03-09T09:28:08.594 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/yesno.c’: No data available 2024-03-09T09:28:08.596 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/split-fail' -> './k/coreutils-8.5/tests/misc/split-fail' 2024-03-09T09:28:08.596 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/split-fail’: No data available 2024-03-09T09:28:08.597 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getopt.c' -> './k/coreutils-8.5/lib/getopt.c' 2024-03-09T09:28:08.597 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getopt.c’: No data available 2024-03-09T09:28:08.598 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-relative' -> './k/coreutils-8.5/tests/misc/truncate-relative' 2024-03-09T09:28:08.598 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-relative’: No data available 2024-03-09T09:28:08.599 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-dir-fail' -> './k/coreutils-8.5/tests/misc/truncate-dir-fail' 2024-03-09T09:28:08.599 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-dir-fail’: No data available 2024-03-09T09:28:08.600 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strpbrk.c' -> './k/coreutils-8.5/lib/strpbrk.c' 2024-03-09T09:28:08.600 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strpbrk.c’: No data available 2024-03-09T09:28:08.601 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/csplit' -> './k/coreutils-8.5/tests/misc/csplit' 2024-03-09T09:28:08.601 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/csplit’: No data available 2024-03-09T09:28:08.602 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dirname-lgpl.c' -> './k/coreutils-8.5/lib/dirname-lgpl.c' 2024-03-09T09:28:08.603 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirname-lgpl.c’: No data available 2024-03-09T09:28:08.603 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/od-N' -> './k/coreutils-8.5/tests/misc/od-N' 2024-03-09T09:28:08.603 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/od-N’: No data available 2024-03-09T09:28:08.604 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/filemode.h' -> './k/coreutils-8.5/lib/filemode.h' 2024-03-09T09:28:08.604 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filemode.h’: No data available 2024-03-09T09:28:08.605 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/xattr' -> './k/coreutils-8.5/tests/misc/xattr' 2024-03-09T09:28:08.606 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/xattr’: No data available 2024-03-09T09:28:08.607 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/mktemp' -> './k/coreutils-8.5/tests/misc/mktemp' 2024-03-09T09:28:08.607 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/mktemp’: No data available 2024-03-09T09:28:08.608 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbrtowc.c' -> './k/coreutils-8.5/lib/mbrtowc.c' 2024-03-09T09:28:08.608 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbrtowc.c’: No data available 2024-03-09T09:28:08.609 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv_open.c' -> './k/coreutils-8.5/lib/iconv_open.c' 2024-03-09T09:28:08.609 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open.c’: No data available 2024-03-09T09:28:08.609 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/cut' -> './k/coreutils-8.5/tests/misc/cut' 2024-03-09T09:28:08.609 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/cut’: No data available 2024-03-09T09:28:08.610 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/cat-proc' -> './k/coreutils-8.5/tests/misc/cat-proc' 2024-03-09T09:28:08.611 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/cat-proc’: No data available 2024-03-09T09:28:08.611 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/euidaccess-stat.c' -> './k/coreutils-8.5/lib/euidaccess-stat.c' 2024-03-09T09:28:08.611 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/euidaccess-stat.c’: No data available 2024-03-09T09:28:08.613 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/comm' -> './k/coreutils-8.5/tests/misc/comm' 2024-03-09T09:28:08.613 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/comm’: No data available 2024-03-09T09:28:08.614 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stat-time.h' -> './k/coreutils-8.5/lib/stat-time.h' 2024-03-09T09:28:08.614 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stat-time.h’: No data available 2024-03-09T09:28:08.615 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/timeout-parameters' -> './k/coreutils-8.5/tests/misc/timeout-parameters' 2024-03-09T09:28:08.615 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/timeout-parameters’: No data available 2024-03-09T09:28:08.616 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/frexpl.c' -> './k/coreutils-8.5/lib/frexpl.c' 2024-03-09T09:28:08.616 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/frexpl.c’: No data available 2024-03-09T09:28:08.617 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/tac-continue' -> './k/coreutils-8.5/tests/misc/tac-continue' 2024-03-09T09:28:08.617 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tac-continue’: No data available 2024-03-09T09:28:08.618 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/tty-eof' -> './k/coreutils-8.5/tests/misc/tty-eof' 2024-03-09T09:28:08.619 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tty-eof’: No data available 2024-03-09T09:28:08.619 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sys_select.in.h' -> './k/coreutils-8.5/lib/sys_select.in.h' 2024-03-09T09:28:08.619 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sys_select.in.h’: No data available 2024-03-09T09:28:08.621 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/df-P' -> './k/coreutils-8.5/tests/misc/df-P' 2024-03-09T09:28:08.621 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/df-P’: No data available 2024-03-09T09:28:08.621 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strftime.h' -> './k/coreutils-8.5/lib/strftime.h' 2024-03-09T09:28:08.622 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strftime.h’: No data available 2024-03-09T09:28:08.623 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/shuf' -> './k/coreutils-8.5/tests/misc/shuf' 2024-03-09T09:28:08.624 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strftime.c' -> './k/coreutils-8.5/lib/strftime.c' 2024-03-09T09:28:08.624 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/shuf’: No data available 2024-03-09T09:28:08.624 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strftime.c’: No data available 2024-03-09T09:28:08.625 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-aix.gperf' -> './k/coreutils-8.5/lib/iconv_open-aix.gperf' 2024-03-09T09:28:08.625 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-aix.gperf’: No data available 2024-03-09T09:28:08.626 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/env' -> './k/coreutils-8.5/tests/misc/env' 2024-03-09T09:28:08.626 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/env’: No data available 2024-03-09T09:28:08.626 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/od' -> './k/coreutils-8.5/tests/misc/od' 2024-03-09T09:28:08.627 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/od’: No data available 2024-03-09T09:28:08.627 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/buffer-lcm.c' -> './k/coreutils-8.5/lib/buffer-lcm.c' 2024-03-09T09:28:08.627 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/buffer-lcm.c’: No data available 2024-03-09T09:28:08.628 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/split-a' -> './k/coreutils-8.5/tests/misc/split-a' 2024-03-09T09:28:08.628 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/split-a’: No data available 2024-03-09T09:28:08.629 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unistr.h' -> './k/coreutils-8.5/lib/unistr.h' 2024-03-09T09:28:08.629 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistr.h’: No data available 2024-03-09T09:28:08.631 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/tail' -> './k/coreutils-8.5/tests/misc/tail' 2024-03-09T09:28:08.631 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tail’: No data available 2024-03-09T09:28:08.632 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/modechange.c' -> './k/coreutils-8.5/lib/modechange.c' 2024-03-09T09:28:08.632 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/modechange.c’: No data available 2024-03-09T09:28:08.632 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/stat-slash' -> './k/coreutils-8.5/tests/misc/stat-slash' 2024-03-09T09:28:08.633 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stat-slash’: No data available 2024-03-09T09:28:08.634 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/timeout' -> './k/coreutils-8.5/tests/misc/timeout' 2024-03-09T09:28:08.634 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/tempname.h' -> './k/coreutils-8.5/lib/tempname.h' 2024-03-09T09:28:08.634 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/timeout’: No data available 2024-03-09T09:28:08.634 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/tempname.h’: No data available 2024-03-09T09:28:08.635 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/faccessat.c' -> './k/coreutils-8.5/lib/faccessat.c' 2024-03-09T09:28:08.636 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/faccessat.c’: No data available 2024-03-09T09:28:08.636 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/close-stdout' -> './k/coreutils-8.5/tests/misc/close-stdout' 2024-03-09T09:28:08.636 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/close-stdout’: No data available 2024-03-09T09:28:08.637 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/wc-files0-from' -> './k/coreutils-8.5/tests/misc/wc-files0-from' 2024-03-09T09:28:08.637 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/wc-files0-from’: No data available 2024-03-09T09:28:08.637 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unicodeio.h' -> './k/coreutils-8.5/lib/unicodeio.h' 2024-03-09T09:28:08.638 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unicodeio.h’: No data available 2024-03-09T09:28:08.639 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sum' -> './k/coreutils-8.5/tests/misc/sum' 2024-03-09T09:28:08.639 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sum’: No data available 2024-03-09T09:28:08.639 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/filemode.c' -> './k/coreutils-8.5/lib/filemode.c' 2024-03-09T09:28:08.640 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filemode.c’: No data available 2024-03-09T09:28:08.641 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/expand' -> './k/coreutils-8.5/tests/misc/expand' 2024-03-09T09:28:08.641 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/expand’: No data available 2024-03-09T09:28:08.641 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strdup.c' -> './k/coreutils-8.5/lib/strdup.c' 2024-03-09T09:28:08.642 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strdup.c’: No data available 2024-03-09T09:28:08.642 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sum-sysv' -> './k/coreutils-8.5/tests/misc/sum-sysv' 2024-03-09T09:28:08.642 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sum-sysv’: No data available 2024-03-09T09:28:08.643 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/wc-parallel' -> './k/coreutils-8.5/tests/misc/wc-parallel' 2024-03-09T09:28:08.643 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/wc-parallel’: No data available 2024-03-09T09:28:08.644 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strsignal.c' -> './k/coreutils-8.5/lib/strsignal.c' 2024-03-09T09:28:08.644 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strsignal.c’: No data available 2024-03-09T09:28:08.646 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/euidaccess.c' -> './k/coreutils-8.5/lib/euidaccess.c' 2024-03-09T09:28:08.646 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/euidaccess.c’: No data available 2024-03-09T09:28:08.648 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/select.c' -> './k/coreutils-8.5/lib/select.c' 2024-03-09T09:28:08.649 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/select.c’: No data available 2024-03-09T09:28:08.650 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memcoll.h' -> './k/coreutils-8.5/lib/memcoll.h' 2024-03-09T09:28:08.651 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcoll.h’: No data available 2024-03-09T09:28:08.653 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fseeko.c' -> './k/coreutils-8.5/lib/fseeko.c' 2024-03-09T09:28:08.653 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fseeko.c’: No data available 2024-03-09T09:28:08.654 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/langinfo.in.h' -> './k/coreutils-8.5/lib/langinfo.in.h' 2024-03-09T09:28:08.655 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/langinfo.in.h’: No data available 2024-03-09T09:28:08.656 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getopt.in.h' -> './k/coreutils-8.5/lib/getopt.in.h' 2024-03-09T09:28:08.656 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getopt.in.h’: No data available 2024-03-09T09:28:08.659 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/localcharset.c' -> './k/coreutils-8.5/lib/localcharset.c' 2024-03-09T09:28:08.659 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/localcharset.c’: No data available 2024-03-09T09:28:08.660 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/isapipe.c' -> './k/coreutils-8.5/lib/isapipe.c' 2024-03-09T09:28:08.661 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isapipe.c’: No data available 2024-03-09T09:28:08.662 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/settime.c' -> './k/coreutils-8.5/lib/settime.c' 2024-03-09T09:28:08.662 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/settime.c’: No data available 2024-03-09T09:28:08.664 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/bitrotate.h' -> './k/coreutils-8.5/lib/bitrotate.h' 2024-03-09T09:28:08.664 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/bitrotate.h’: No data available 2024-03-09T09:28:08.665 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/isnanl-nolibm.h' -> './k/coreutils-8.5/lib/isnanl-nolibm.h' 2024-03-09T09:28:08.666 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnanl-nolibm.h’: No data available 2024-03-09T09:28:08.668 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fcntl.c' -> './k/coreutils-8.5/lib/fcntl.c' 2024-03-09T09:28:08.668 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fcntl.c’: No data available 2024-03-09T09:28:08.669 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/exclude.h' -> './k/coreutils-8.5/lib/exclude.h' 2024-03-09T09:28:08.669 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/exclude.h’: No data available 2024-03-09T09:28:08.671 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/symlinkat.c' -> './k/coreutils-8.5/lib/symlinkat.c' 2024-03-09T09:28:08.671 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/symlinkat.c’: No data available 2024-03-09T09:28:08.673 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fpending.c' -> './k/coreutils-8.5/lib/fpending.c' 2024-03-09T09:28:08.673 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fpending.c’: No data available 2024-03-09T09:28:08.675 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/savewd.c' -> './k/coreutils-8.5/lib/savewd.c' 2024-03-09T09:28:08.675 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/savewd.c’: No data available 2024-03-09T09:28:08.677 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fchownat.c' -> './k/coreutils-8.5/lib/fchownat.c' 2024-03-09T09:28:08.677 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fchownat.c’: No data available 2024-03-09T09:28:08.679 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/root-dev-ino.h' -> './k/coreutils-8.5/lib/root-dev-ino.h' 2024-03-09T09:28:08.679 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/root-dev-ino.h’: No data available 2024-03-09T09:28:08.680 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/randread.h' -> './k/coreutils-8.5/lib/randread.h' 2024-03-09T09:28:08.681 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randread.h’: No data available 2024-03-09T09:28:08.682 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/freadseek.h' -> './k/coreutils-8.5/lib/freadseek.h' 2024-03-09T09:28:08.683 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadseek.h’: No data available 2024-03-09T09:28:08.684 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strtoll.c' -> './k/coreutils-8.5/lib/strtoll.c' 2024-03-09T09:28:08.685 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtoll.c’: No data available 2024-03-09T09:28:08.687 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stdio-safer.h' -> './k/coreutils-8.5/lib/stdio-safer.h' 2024-03-09T09:28:08.687 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdio-safer.h’: No data available 2024-03-09T09:28:08.689 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strtoumax.c' -> './k/coreutils-8.5/lib/strtoumax.c' 2024-03-09T09:28:08.689 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtoumax.c’: No data available 2024-03-09T09:28:08.690 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/argv-iter.h' -> './k/coreutils-8.5/lib/argv-iter.h' 2024-03-09T09:28:08.691 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/argv-iter.h’: No data available 2024-03-09T09:28:08.692 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dirent-safer.h' -> './k/coreutils-8.5/lib/dirent-safer.h' 2024-03-09T09:28:08.693 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirent-safer.h’: No data available 2024-03-09T09:28:08.694 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dirent.in.h' -> './k/coreutils-8.5/lib/dirent.in.h' 2024-03-09T09:28:08.695 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirent.in.h’: No data available 2024-03-09T09:28:08.696 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/malloca.valgrind' -> './k/coreutils-8.5/lib/malloca.valgrind' 2024-03-09T09:28:08.696 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/malloca.valgrind’: No data available 2024-03-09T09:28:08.697 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/savewd.h' -> './k/coreutils-8.5/lib/savewd.h' 2024-03-09T09:28:08.697 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/savewd.h’: No data available 2024-03-09T09:28:08.699 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/uinttostr.c' -> './k/coreutils-8.5/lib/uinttostr.c' 2024-03-09T09:28:08.699 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/uinttostr.c’: No data available 2024-03-09T09:28:08.701 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getline.c' -> './k/coreutils-8.5/lib/getline.c' 2024-03-09T09:28:08.701 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getline.c’: No data available 2024-03-09T09:28:08.703 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/chdir-long.h' -> './k/coreutils-8.5/lib/chdir-long.h' 2024-03-09T09:28:08.703 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/chdir-long.h’: No data available 2024-03-09T09:28:08.705 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/cycle-check.h' -> './k/coreutils-8.5/lib/cycle-check.h' 2024-03-09T09:28:08.705 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/cycle-check.h’: No data available 2024-03-09T09:28:08.707 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fts-cycle.c' -> './k/coreutils-8.5/lib/fts-cycle.c' 2024-03-09T09:28:08.707 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fts-cycle.c’: No data available 2024-03-09T09:28:08.709 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/file-has-acl.c' -> './k/coreutils-8.5/lib/file-has-acl.c' 2024-03-09T09:28:08.710 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/file-has-acl.c’: No data available 2024-03-09T09:28:08.711 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/areadlink.h' -> './k/coreutils-8.5/lib/areadlink.h' 2024-03-09T09:28:08.711 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/areadlink.h’: No data available 2024-03-09T09:28:08.713 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sig2str.h' -> './k/coreutils-8.5/lib/sig2str.h' 2024-03-09T09:28:08.713 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sig2str.h’: No data available 2024-03-09T09:28:08.715 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memcpy.c' -> './k/coreutils-8.5/lib/memcpy.c' 2024-03-09T09:28:08.716 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcpy.c’: No data available 2024-03-09T09:28:08.717 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/freading.h' -> './k/coreutils-8.5/lib/freading.h' 2024-03-09T09:28:08.718 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freading.h’: No data available 2024-03-09T09:28:08.720 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/full-write.h' -> './k/coreutils-8.5/lib/full-write.h' 2024-03-09T09:28:08.720 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/full-write.h’: No data available 2024-03-09T09:28:08.722 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unistd.in.h' -> './k/coreutils-8.5/lib/unistd.in.h' 2024-03-09T09:28:08.723 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistd.in.h’: No data available 2024-03-09T09:28:08.725 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/symlink.c' -> './k/coreutils-8.5/lib/symlink.c' 2024-03-09T09:28:08.725 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/symlink.c’: No data available 2024-03-09T09:28:08.727 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/se-selinux.in.h' -> './k/coreutils-8.5/lib/se-selinux.in.h' 2024-03-09T09:28:08.727 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/se-selinux.in.h’: No data available 2024-03-09T09:28:08.729 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/write-any-file.h' -> './k/coreutils-8.5/lib/write-any-file.h' 2024-03-09T09:28:08.729 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/write-any-file.h’: No data available 2024-03-09T09:28:08.732 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/printf-frexpl.h' -> './k/coreutils-8.5/lib/printf-frexpl.h' 2024-03-09T09:28:08.732 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-frexpl.h’: No data available 2024-03-09T09:28:08.735 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/Makefile.in' -> './k/coreutils-8.5/lib/Makefile.in' 2024-03-09T09:28:08.736 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/Makefile.in’: No data available 2024-03-09T09:28:08.738 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/filenamecat.h' -> './k/coreutils-8.5/lib/filenamecat.h' 2024-03-09T09:28:08.738 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filenamecat.h’: No data available 2024-03-09T09:28:08.740 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/malloca.c' -> './k/coreutils-8.5/lib/malloca.c' 2024-03-09T09:28:08.740 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/malloca.c’: No data available 2024-03-09T09:28:08.743 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/utimecmp.h' -> './k/coreutils-8.5/lib/utimecmp.h' 2024-03-09T09:28:08.743 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/utimecmp.h’: No data available 2024-03-09T09:28:08.745 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stdlib.in.h' -> './k/coreutils-8.5/lib/stdlib.in.h' 2024-03-09T09:28:08.745 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdlib.in.h’: No data available 2024-03-09T09:28:08.747 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stdio.in.h' -> './k/coreutils-8.5/lib/stdio.in.h' 2024-03-09T09:28:08.748 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdio.in.h’: No data available 2024-03-09T09:28:08.749 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbsinit.c' -> './k/coreutils-8.5/lib/mbsinit.c' 2024-03-09T09:28:08.749 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsinit.c’: No data available 2024-03-09T09:28:08.751 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/diacrit.c' -> './k/coreutils-8.5/lib/diacrit.c' 2024-03-09T09:28:08.752 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/diacrit.c’: No data available 2024-03-09T09:28:08.753 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/se-context.in.h' -> './k/coreutils-8.5/lib/se-context.in.h' 2024-03-09T09:28:08.753 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/se-context.in.h’: No data available 2024-03-09T09:28:08.755 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xvasprintf.c' -> './k/coreutils-8.5/lib/xvasprintf.c' 2024-03-09T09:28:08.755 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xvasprintf.c’: No data available 2024-03-09T09:28:08.756 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xgetcwd.c' -> './k/coreutils-8.5/lib/xgetcwd.c' 2024-03-09T09:28:08.756 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xgetcwd.c’: No data available 2024-03-09T09:28:08.759 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sockets.h' -> './k/coreutils-8.5/lib/sockets.h' 2024-03-09T09:28:08.759 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sockets.h’: No data available 2024-03-09T09:28:08.766 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/striconv.h' -> './k/coreutils-8.5/lib/striconv.h' 2024-03-09T09:28:08.766 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/striconv.h’: No data available 2024-03-09T09:28:08.768 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mkstemp-safer.c' -> './k/coreutils-8.5/lib/mkstemp-safer.c' 2024-03-09T09:28:08.768 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkstemp-safer.c’: No data available 2024-03-09T09:28:08.770 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/u64.h' -> './k/coreutils-8.5/lib/u64.h' 2024-03-09T09:28:08.770 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/u64.h’: No data available 2024-03-09T09:28:08.771 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstrtoimax.c' -> './k/coreutils-8.5/lib/xstrtoimax.c' 2024-03-09T09:28:08.771 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtoimax.c’: No data available 2024-03-09T09:28:08.772 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/time_r.c' -> './k/coreutils-8.5/lib/time_r.c' 2024-03-09T09:28:08.772 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/time_r.c’: No data available 2024-03-09T09:28:08.775 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbchar.h' -> './k/coreutils-8.5/lib/mbchar.h' 2024-03-09T09:28:08.775 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbchar.h’: No data available 2024-03-09T09:28:08.776 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xtime.h' -> './k/coreutils-8.5/lib/xtime.h' 2024-03-09T09:28:08.777 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xtime.h’: No data available 2024-03-09T09:28:08.778 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/set-mode-acl.c' -> './k/coreutils-8.5/lib/set-mode-acl.c' 2024-03-09T09:28:08.778 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/set-mode-acl.c’: No data available 2024-03-09T09:28:08.779 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/version-etc.c' -> './k/coreutils-8.5/lib/version-etc.c' 2024-03-09T09:28:08.779 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/version-etc.c’: No data available 2024-03-09T09:28:08.781 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dup2.c' -> './k/coreutils-8.5/lib/dup2.c' 2024-03-09T09:28:08.782 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dup2.c’: No data available 2024-03-09T09:28:08.783 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbsalign.h' -> './k/coreutils-8.5/lib/mbsalign.h' 2024-03-09T09:28:08.783 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsalign.h’: No data available 2024-03-09T09:28:08.785 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unlinkat.c' -> './k/coreutils-8.5/lib/unlinkat.c' 2024-03-09T09:28:08.785 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unlinkat.c’: No data available 2024-03-09T09:28:08.789 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sigprocmask.c' -> './k/coreutils-8.5/lib/sigprocmask.c' 2024-03-09T09:28:08.790 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sigprocmask.c’: No data available 2024-03-09T09:28:08.790 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/hash-triple.h' -> './k/coreutils-8.5/lib/hash-triple.h' 2024-03-09T09:28:08.791 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash-triple.h’: No data available 2024-03-09T09:28:08.791 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fprintftime.c' -> './k/coreutils-8.5/lib/fprintftime.c' 2024-03-09T09:28:08.791 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fprintftime.c’: No data available 2024-03-09T09:28:08.794 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unistd--.h' -> './k/coreutils-8.5/lib/unistd--.h' 2024-03-09T09:28:08.794 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistd--.h’: No data available 2024-03-09T09:28:08.795 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/umaxtostr.c' -> './k/coreutils-8.5/lib/umaxtostr.c' 2024-03-09T09:28:08.795 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/umaxtostr.c’: No data available 2024-03-09T09:28:08.796 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/i-ring.c' -> './k/coreutils-8.5/lib/i-ring.c' 2024-03-09T09:28:08.796 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/i-ring.c’: No data available 2024-03-09T09:28:08.797 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/signbitd.c' -> './k/coreutils-8.5/lib/signbitd.c' 2024-03-09T09:28:08.798 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/signbitd.c’: No data available 2024-03-09T09:28:08.799 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/ftello.c' -> './k/coreutils-8.5/lib/ftello.c' 2024-03-09T09:28:08.799 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ftello.c’: No data available 2024-03-09T09:28:08.801 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/lchown.c' -> './k/coreutils-8.5/lib/lchown.c' 2024-03-09T09:28:08.801 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/lchown.c’: No data available 2024-03-09T09:28:08.803 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/lstat.c' -> './k/coreutils-8.5/lib/lstat.c' 2024-03-09T09:28:08.803 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/lstat.c’: No data available 2024-03-09T09:28:08.808 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fpurge.c' -> './k/coreutils-8.5/lib/fpurge.c' 2024-03-09T09:28:08.808 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fpurge.c’: No data available 2024-03-09T09:28:08.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fstatat.c' -> './k/coreutils-8.5/lib/fstatat.c' 2024-03-09T09:28:08.814 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fstatat.c’: No data available 2024-03-09T09:28:08.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/lseek.c' -> './k/coreutils-8.5/lib/lseek.c' 2024-03-09T09:28:08.815 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/lseek.c’: No data available 2024-03-09T09:28:08.815 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fchmodat.c' -> './k/coreutils-8.5/lib/fchmodat.c' 2024-03-09T09:28:08.815 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fchmodat.c’: No data available 2024-03-09T09:28:08.817 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/alloca.c' -> './k/coreutils-8.5/lib/alloca.c' 2024-03-09T09:28:08.817 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/alloca.c’: No data available 2024-03-09T09:28:08.820 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memcmp.c' -> './k/coreutils-8.5/lib/memcmp.c' 2024-03-09T09:28:08.820 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcmp.c’: No data available 2024-03-09T09:28:08.821 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstrndup.h' -> './k/coreutils-8.5/lib/xstrndup.h' 2024-03-09T09:28:08.822 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrndup.h’: No data available 2024-03-09T09:28:08.823 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/frexp.c' -> './k/coreutils-8.5/lib/frexp.c' 2024-03-09T09:28:08.823 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/frexp.c’: No data available 2024-03-09T09:28:08.825 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/nproc.h' -> './k/coreutils-8.5/lib/nproc.h' 2024-03-09T09:28:08.825 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/nproc.h’: No data available 2024-03-09T09:28:08.827 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fsusage.h' -> './k/coreutils-8.5/lib/fsusage.h' 2024-03-09T09:28:08.827 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fsusage.h’: No data available 2024-03-09T09:28:08.829 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fd-safer-flag.c' -> './k/coreutils-8.5/lib/fd-safer-flag.c' 2024-03-09T09:28:08.829 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fd-safer-flag.c’: No data available 2024-03-09T09:28:08.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getdtablesize.c' -> './k/coreutils-8.5/lib/getdtablesize.c' 2024-03-09T09:28:08.832 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getdtablesize.c’: No data available 2024-03-09T09:28:08.833 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/savedir.c' -> './k/coreutils-8.5/lib/savedir.c' 2024-03-09T09:28:08.833 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/savedir.c’: No data available 2024-03-09T09:28:08.835 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/wcwidth.c' -> './k/coreutils-8.5/lib/wcwidth.c' 2024-03-09T09:28:08.835 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/wcwidth.c’: No data available 2024-03-09T09:28:08.837 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbsstr.c' -> './k/coreutils-8.5/lib/mbsstr.c' 2024-03-09T09:28:08.837 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsstr.c’: No data available 2024-03-09T09:28:08.839 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fpucw.h' -> './k/coreutils-8.5/lib/fpucw.h' 2024-03-09T09:28:08.839 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fpucw.h’: No data available 2024-03-09T09:28:08.840 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-hpux.h' -> './k/coreutils-8.5/lib/iconv_open-hpux.h' 2024-03-09T09:28:08.841 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-hpux.h’: No data available 2024-03-09T09:28:08.842 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/readtokens0.h' -> './k/coreutils-8.5/lib/readtokens0.h' 2024-03-09T09:28:08.842 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readtokens0.h’: No data available 2024-03-09T09:28:08.845 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getdate.c' -> './k/coreutils-8.5/lib/getdate.c' 2024-03-09T09:28:08.845 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getdate.c’: No data available 2024-03-09T09:28:08.846 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/utimens.c' -> './k/coreutils-8.5/lib/utimens.c' 2024-03-09T09:28:08.846 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/utimens.c’: No data available 2024-03-09T09:28:08.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getndelim2.c' -> './k/coreutils-8.5/lib/getndelim2.c' 2024-03-09T09:28:08.848 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getndelim2.c’: No data available 2024-03-09T09:28:08.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fflush.c' -> './k/coreutils-8.5/lib/fflush.c' 2024-03-09T09:28:08.850 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fflush.c’: No data available 2024-03-09T09:28:08.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/version-etc-fsf.c' -> './k/coreutils-8.5/lib/version-etc-fsf.c' 2024-03-09T09:28:08.852 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/version-etc-fsf.c’: No data available 2024-03-09T09:28:08.853 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xsize.h' -> './k/coreutils-8.5/lib/xsize.h' 2024-03-09T09:28:08.854 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xsize.h’: No data available 2024-03-09T09:28:08.855 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xgethostname.c' -> './k/coreutils-8.5/lib/xgethostname.c' 2024-03-09T09:28:08.856 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xgethostname.c’: No data available 2024-03-09T09:28:08.858 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strnlen1.h' -> './k/coreutils-8.5/lib/strnlen1.h' 2024-03-09T09:28:08.858 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnlen1.h’: No data available 2024-03-09T09:28:08.859 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/isnanl.c' -> './k/coreutils-8.5/lib/isnanl.c' 2024-03-09T09:28:08.860 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnanl.c’: No data available 2024-03-09T09:28:08.862 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/same-inode.h' -> './k/coreutils-8.5/lib/same-inode.h' 2024-03-09T09:28:08.862 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/same-inode.h’: No data available 2024-03-09T09:28:08.864 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/regex_internal.h' -> './k/coreutils-8.5/lib/regex_internal.h' 2024-03-09T09:28:08.864 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regex_internal.h’: No data available 2024-03-09T09:28:08.865 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/full-read.h' -> './k/coreutils-8.5/lib/full-read.h' 2024-03-09T09:28:08.865 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/full-read.h’: No data available 2024-03-09T09:28:08.867 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/intprops.h' -> './k/coreutils-8.5/lib/intprops.h' 2024-03-09T09:28:08.867 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/intprops.h’: No data available 2024-03-09T09:28:08.869 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fsusage.c' -> './k/coreutils-8.5/lib/fsusage.c' 2024-03-09T09:28:08.869 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fsusage.c’: No data available 2024-03-09T09:28:08.870 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/save-cwd.c' -> './k/coreutils-8.5/lib/save-cwd.c' 2024-03-09T09:28:08.871 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/save-cwd.c’: No data available 2024-03-09T09:28:08.872 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memchr.valgrind' -> './k/coreutils-8.5/lib/memchr.valgrind' 2024-03-09T09:28:08.872 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memchr.valgrind’: No data available 2024-03-09T09:28:08.874 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/exitfail.h' -> './k/coreutils-8.5/lib/exitfail.h' 2024-03-09T09:28:08.874 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/exitfail.h’: No data available 2024-03-09T09:28:08.875 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/printf-frexp.h' -> './k/coreutils-8.5/lib/printf-frexp.h' 2024-03-09T09:28:08.876 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-frexp.h’: No data available 2024-03-09T09:28:08.879 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/isblank.c' -> './k/coreutils-8.5/lib/isblank.c' 2024-03-09T09:28:08.879 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isblank.c’: No data available 2024-03-09T09:28:08.880 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/arpa_inet.in.h' -> './k/coreutils-8.5/lib/arpa_inet.in.h' 2024-03-09T09:28:08.880 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/arpa_inet.in.h’: No data available 2024-03-09T09:28:08.884 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strerror.c' -> './k/coreutils-8.5/lib/strerror.c' 2024-03-09T09:28:08.884 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strerror.c’: No data available 2024-03-09T09:28:08.885 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/freadptr.c' -> './k/coreutils-8.5/lib/freadptr.c' 2024-03-09T09:28:08.885 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadptr.c’: No data available 2024-03-09T09:28:08.886 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/openat.h' -> './k/coreutils-8.5/lib/openat.h' 2024-03-09T09:28:08.886 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat.h’: No data available 2024-03-09T09:28:08.887 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-irix.h' -> './k/coreutils-8.5/lib/iconv_open-irix.h' 2024-03-09T09:28:08.887 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-irix.h’: No data available 2024-03-09T09:28:08.890 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/open.c' -> './k/coreutils-8.5/lib/open.c' 2024-03-09T09:28:08.890 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/open.c’: No data available 2024-03-09T09:28:08.891 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/vprintf.c' -> './k/coreutils-8.5/lib/vprintf.c' 2024-03-09T09:28:08.892 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/vprintf.c’: No data available 2024-03-09T09:28:08.893 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/same.h' -> './k/coreutils-8.5/lib/same.h' 2024-03-09T09:28:08.894 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/same.h’: No data available 2024-03-09T09:28:08.896 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/gai_strerror.c' -> './k/coreutils-8.5/lib/gai_strerror.c' 2024-03-09T09:28:08.896 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gai_strerror.c’: No data available 2024-03-09T09:28:08.905 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/imaxtostr.c' -> './k/coreutils-8.5/lib/imaxtostr.c' 2024-03-09T09:28:08.906 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/imaxtostr.c’: No data available 2024-03-09T09:28:08.906 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memcmp2.h' -> './k/coreutils-8.5/lib/memcmp2.h' 2024-03-09T09:28:08.906 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcmp2.h’: No data available 2024-03-09T09:28:08.907 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xfreopen.c' -> './k/coreutils-8.5/lib/xfreopen.c' 2024-03-09T09:28:08.907 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xfreopen.c’: No data available 2024-03-09T09:28:08.907 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/rand-isaac.c' -> './k/coreutils-8.5/lib/rand-isaac.c' 2024-03-09T09:28:08.907 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/rand-isaac.c’: No data available 2024-03-09T09:28:08.908 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/full-write.c' -> './k/coreutils-8.5/lib/full-write.c' 2024-03-09T09:28:08.908 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/full-write.c’: No data available 2024-03-09T09:28:08.908 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/rmdir.c' -> './k/coreutils-8.5/lib/rmdir.c' 2024-03-09T09:28:08.909 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/rmdir.c’: No data available 2024-03-09T09:28:08.910 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/save-cwd.h' -> './k/coreutils-8.5/lib/save-cwd.h' 2024-03-09T09:28:08.910 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/save-cwd.h’: No data available 2024-03-09T09:28:08.911 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/at-func2.c' -> './k/coreutils-8.5/lib/at-func2.c' 2024-03-09T09:28:08.912 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/at-func2.c’: No data available 2024-03-09T09:28:08.913 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/gethrxtime.c' -> './k/coreutils-8.5/lib/gethrxtime.c' 2024-03-09T09:28:08.913 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gethrxtime.c’: No data available 2024-03-09T09:28:08.915 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strtod.c' -> './k/coreutils-8.5/lib/strtod.c' 2024-03-09T09:28:08.915 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtod.c’: No data available 2024-03-09T09:28:08.917 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbsalign.c' -> './k/coreutils-8.5/lib/mbsalign.c' 2024-03-09T09:28:08.918 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsalign.c’: No data available 2024-03-09T09:28:08.919 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-osf.gperf' -> './k/coreutils-8.5/lib/iconv_open-osf.gperf' 2024-03-09T09:28:08.919 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-osf.gperf’: No data available 2024-03-09T09:28:08.921 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/ref-add.sin' -> './k/coreutils-8.5/lib/ref-add.sin' 2024-03-09T09:28:08.921 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ref-add.sin’: No data available 2024-03-09T09:28:08.923 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memcmp2.c' -> './k/coreutils-8.5/lib/memcmp2.c' 2024-03-09T09:28:08.923 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcmp2.c’: No data available 2024-03-09T09:28:08.926 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/gnulib.mk' -> './k/coreutils-8.5/lib/gnulib.mk' 2024-03-09T09:28:08.926 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gnulib.mk’: No data available 2024-03-09T09:28:08.929 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/ref-del.sin' -> './k/coreutils-8.5/lib/ref-del.sin' 2024-03-09T09:28:08.929 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ref-del.sin’: No data available 2024-03-09T09:28:08.930 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stpncpy.c' -> './k/coreutils-8.5/lib/stpncpy.c' 2024-03-09T09:28:08.930 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stpncpy.c’: No data available 2024-03-09T09:28:08.932 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv.in.h' -> './k/coreutils-8.5/lib/iconv.in.h' 2024-03-09T09:28:08.932 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv.in.h’: No data available 2024-03-09T09:28:08.933 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/posixtm.h' -> './k/coreutils-8.5/lib/posixtm.h' 2024-03-09T09:28:08.934 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/posixtm.h’: No data available 2024-03-09T09:28:08.935 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstrtol.c' -> './k/coreutils-8.5/lib/xstrtol.c' 2024-03-09T09:28:08.935 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtol.c’: No data available 2024-03-09T09:28:08.938 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getopt1.c' -> './k/coreutils-8.5/lib/getopt1.c' 2024-03-09T09:28:08.938 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getopt1.c’: No data available 2024-03-09T09:28:08.940 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/quotearg.h' -> './k/coreutils-8.5/lib/quotearg.h' 2024-03-09T09:28:08.940 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/quotearg.h’: No data available 2024-03-09T09:28:08.942 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/cycle-check.c' -> './k/coreutils-8.5/lib/cycle-check.c' 2024-03-09T09:28:08.942 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/cycle-check.c’: No data available 2024-03-09T09:28:08.944 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/calloc.c' -> './k/coreutils-8.5/lib/calloc.c' 2024-03-09T09:28:08.944 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/calloc.c’: No data available 2024-03-09T09:28:08.947 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/areadlinkat.c' -> './k/coreutils-8.5/lib/areadlinkat.c' 2024-03-09T09:28:08.947 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/areadlinkat.c’: No data available 2024-03-09T09:28:08.948 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/hash-triple.c' -> './k/coreutils-8.5/lib/hash-triple.c' 2024-03-09T09:28:08.949 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash-triple.c’: No data available 2024-03-09T09:28:08.951 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getpass.c' -> './k/coreutils-8.5/lib/getpass.c' 2024-03-09T09:28:08.951 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getpass.c’: No data available 2024-03-09T09:28:08.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstriconv.c' -> './k/coreutils-8.5/lib/xstriconv.c' 2024-03-09T09:28:08.953 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstriconv.c’: No data available 2024-03-09T09:28:08.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dirent--.h' -> './k/coreutils-8.5/lib/dirent--.h' 2024-03-09T09:28:08.956 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirent--.h’: No data available 2024-03-09T09:28:08.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sigaction.c' -> './k/coreutils-8.5/lib/sigaction.c' 2024-03-09T09:28:08.958 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sigaction.c’: No data available 2024-03-09T09:28:08.961 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/linkat.c' -> './k/coreutils-8.5/lib/linkat.c' 2024-03-09T09:28:08.961 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/linkat.c’: No data available 2024-03-09T09:28:08.963 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unitypes.h' -> './k/coreutils-8.5/lib/unitypes.h' 2024-03-09T09:28:08.963 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unitypes.h’: No data available 2024-03-09T09:28:08.965 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/linebuffer.c' -> './k/coreutils-8.5/lib/linebuffer.c' 2024-03-09T09:28:08.965 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/linebuffer.c’: No data available 2024-03-09T09:28:08.967 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fnmatch.c' -> './k/coreutils-8.5/lib/fnmatch.c' 2024-03-09T09:28:08.968 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fnmatch.c’: No data available 2024-03-09T09:28:08.969 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xasprintf.c' -> './k/coreutils-8.5/lib/xasprintf.c' 2024-03-09T09:28:08.969 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xasprintf.c’: No data available 2024-03-09T09:28:08.972 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/printf-parse.h' -> './k/coreutils-8.5/lib/printf-parse.h' 2024-03-09T09:28:08.972 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-parse.h’: No data available 2024-03-09T09:28:08.974 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/string.in.h' -> './k/coreutils-8.5/lib/string.in.h' 2024-03-09T09:28:08.975 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/string.in.h’: No data available 2024-03-09T09:28:08.976 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mkdir.c' -> './k/coreutils-8.5/lib/mkdir.c' 2024-03-09T09:28:08.976 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkdir.c’: No data available 2024-03-09T09:28:08.978 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fchown-stub.c' -> './k/coreutils-8.5/lib/fchown-stub.c' 2024-03-09T09:28:08.978 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fchown-stub.c’: No data available 2024-03-09T09:28:08.982 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/human.c' -> './k/coreutils-8.5/lib/human.c' 2024-03-09T09:28:08.982 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/human.c’: No data available 2024-03-09T09:28:08.984 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sockets.c' -> './k/coreutils-8.5/lib/sockets.c' 2024-03-09T09:28:08.984 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sockets.c’: No data available 2024-03-09T09:28:08.985 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/canonicalize.h' -> './k/coreutils-8.5/lib/canonicalize.h' 2024-03-09T09:28:08.985 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/canonicalize.h’: No data available 2024-03-09T09:28:08.988 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getopt_int.h' -> './k/coreutils-8.5/lib/getopt_int.h' 2024-03-09T09:28:08.988 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getopt_int.h’: No data available 2024-03-09T09:28:08.989 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/version-etc.h' -> './k/coreutils-8.5/lib/version-etc.h' 2024-03-09T09:28:08.989 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/version-etc.h’: No data available 2024-03-09T09:28:08.995 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stdlib--.h' -> './k/coreutils-8.5/lib/stdlib--.h' 2024-03-09T09:28:08.995 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdlib--.h’: No data available 2024-03-09T09:28:08.996 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/backupfile.h' -> './k/coreutils-8.5/lib/backupfile.h' 2024-03-09T09:28:08.996 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/backupfile.h’: No data available 2024-03-09T09:28:08.996 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/file-type.c' -> './k/coreutils-8.5/lib/file-type.c' 2024-03-09T09:28:08.996 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/file-type.c’: No data available 2024-03-09T09:28:08.998 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/file-set.h' -> './k/coreutils-8.5/lib/file-set.h' 2024-03-09T09:28:08.998 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/file-set.h’: No data available 2024-03-09T09:28:09.000 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbuiter.h' -> './k/coreutils-8.5/lib/mbuiter.h' 2024-03-09T09:28:09.001 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbuiter.h’: No data available 2024-03-09T09:28:09.003 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fcntl.in.h' -> './k/coreutils-8.5/lib/fcntl.in.h' 2024-03-09T09:28:09.003 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fcntl.in.h’: No data available 2024-03-09T09:28:09.005 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/inet_ntop.c' -> './k/coreutils-8.5/lib/inet_ntop.c' 2024-03-09T09:28:09.005 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/inet_ntop.c’: No data available 2024-03-09T09:28:09.007 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fseterr.c' -> './k/coreutils-8.5/lib/fseterr.c' 2024-03-09T09:28:09.007 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fseterr.c’: No data available 2024-03-09T09:28:09.009 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/close-stream.h' -> './k/coreutils-8.5/lib/close-stream.h' 2024-03-09T09:28:09.009 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/close-stream.h’: No data available 2024-03-09T09:28:09.011 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/freopen-safer.c' -> './k/coreutils-8.5/lib/freopen-safer.c' 2024-03-09T09:28:09.011 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freopen-safer.c’: No data available 2024-03-09T09:28:09.013 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stat-macros.h' -> './k/coreutils-8.5/lib/stat-macros.h' 2024-03-09T09:28:09.014 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stat-macros.h’: No data available 2024-03-09T09:28:09.016 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dirname.c' -> './k/coreutils-8.5/lib/dirname.c' 2024-03-09T09:28:09.016 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirname.c’: No data available 2024-03-09T09:28:09.018 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dev-ino.h' -> './k/coreutils-8.5/lib/dev-ino.h' 2024-03-09T09:28:09.018 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dev-ino.h’: No data available 2024-03-09T09:28:09.020 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xalloc.h' -> './k/coreutils-8.5/lib/xalloc.h' 2024-03-09T09:28:09.020 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xalloc.h’: No data available 2024-03-09T09:28:09.022 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memcasecmp.h' -> './k/coreutils-8.5/lib/memcasecmp.h' 2024-03-09T09:28:09.022 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memcasecmp.h’: No data available 2024-03-09T09:28:09.025 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/hash.c' -> './k/coreutils-8.5/lib/hash.c' 2024-03-09T09:28:09.025 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash.c’: No data available 2024-03-09T09:28:09.027 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/userspec.c' -> './k/coreutils-8.5/lib/userspec.c' 2024-03-09T09:28:09.028 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/userspec.c’: No data available 2024-03-09T09:28:09.029 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/hard-locale.h' -> './k/coreutils-8.5/lib/hard-locale.h' 2024-03-09T09:28:09.030 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hard-locale.h’: No data available 2024-03-09T09:28:09.032 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xvasprintf.h' -> './k/coreutils-8.5/lib/xvasprintf.h' 2024-03-09T09:28:09.032 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xvasprintf.h’: No data available 2024-03-09T09:28:09.034 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/wcrtomb.c' -> './k/coreutils-8.5/lib/wcrtomb.c' 2024-03-09T09:28:09.034 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/wcrtomb.c’: No data available 2024-03-09T09:28:09.036 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/gethrxtime.h' -> './k/coreutils-8.5/lib/gethrxtime.h' 2024-03-09T09:28:09.036 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gethrxtime.h’: No data available 2024-03-09T09:28:09.038 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fprintftime.h' -> './k/coreutils-8.5/lib/fprintftime.h' 2024-03-09T09:28:09.039 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fprintftime.h’: No data available 2024-03-09T09:28:09.041 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mgetgroups.c' -> './k/coreutils-8.5/lib/mgetgroups.c' 2024-03-09T09:28:09.041 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mgetgroups.c’: No data available 2024-03-09T09:28:09.043 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/randread.c' -> './k/coreutils-8.5/lib/randread.c' 2024-03-09T09:28:09.043 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randread.c’: No data available 2024-03-09T09:28:09.046 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstriconv.h' -> './k/coreutils-8.5/lib/xstriconv.h' 2024-03-09T09:28:09.046 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstriconv.h’: No data available 2024-03-09T09:28:09.048 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dup-safer-flag.c' -> './k/coreutils-8.5/lib/dup-safer-flag.c' 2024-03-09T09:28:09.048 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dup-safer-flag.c’: No data available 2024-03-09T09:28:09.050 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/yesno.h' -> './k/coreutils-8.5/lib/yesno.h' 2024-03-09T09:28:09.051 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/yesno.h’: No data available 2024-03-09T09:28:09.052 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbscasecmp.c' -> './k/coreutils-8.5/lib/mbscasecmp.c' 2024-03-09T09:28:09.053 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbscasecmp.c’: No data available 2024-03-09T09:28:09.055 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strtoimax.c' -> './k/coreutils-8.5/lib/strtoimax.c' 2024-03-09T09:28:09.055 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtoimax.c’: No data available 2024-03-09T09:28:09.058 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbsrtowcs-state.c' -> './k/coreutils-8.5/lib/mbsrtowcs-state.c' 2024-03-09T09:28:09.058 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbsrtowcs-state.c’: No data available 2024-03-09T09:28:09.060 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mkstemp.c' -> './k/coreutils-8.5/lib/mkstemp.c' 2024-03-09T09:28:09.060 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkstemp.c’: No data available 2024-03-09T09:28:09.062 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xmemcoll.c' -> './k/coreutils-8.5/lib/xmemcoll.c' 2024-03-09T09:28:09.062 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xmemcoll.c’: No data available 2024-03-09T09:28:09.064 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strintcmp.c' -> './k/coreutils-8.5/lib/strintcmp.c' 2024-03-09T09:28:09.064 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strintcmp.c’: No data available 2024-03-09T09:28:09.065 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fclose.c' -> './k/coreutils-8.5/lib/fclose.c' 2024-03-09T09:28:09.066 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fclose.c’: No data available 2024-03-09T09:28:09.067 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fd-reopen.c' -> './k/coreutils-8.5/lib/fd-reopen.c' 2024-03-09T09:28:09.067 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fd-reopen.c’: No data available 2024-03-09T09:28:09.069 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/inttostr.c' -> './k/coreutils-8.5/lib/inttostr.c' 2024-03-09T09:28:09.069 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/inttostr.c’: No data available 2024-03-09T09:28:09.071 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/c-ctype.c' -> './k/coreutils-8.5/lib/c-ctype.c' 2024-03-09T09:28:09.071 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-ctype.c’: No data available 2024-03-09T09:28:09.080 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/openat-die.c' -> './k/coreutils-8.5/lib/openat-die.c' 2024-03-09T09:28:09.080 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat-die.c’: No data available 2024-03-09T09:28:09.082 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/error.c' -> './k/coreutils-8.5/lib/error.c' 2024-03-09T09:28:09.082 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/error.c’: No data available 2024-03-09T09:28:09.084 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sig2str.c' -> './k/coreutils-8.5/lib/sig2str.c' 2024-03-09T09:28:09.084 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sig2str.c’: No data available 2024-03-09T09:28:09.086 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unlocked-io.h' -> './k/coreutils-8.5/lib/unlocked-io.h' 2024-03-09T09:28:09.086 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unlocked-io.h’: No data available 2024-03-09T09:28:09.088 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memxfrm.c' -> './k/coreutils-8.5/lib/memxfrm.c' 2024-03-09T09:28:09.088 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memxfrm.c’: No data available 2024-03-09T09:28:09.090 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/pipe-safer.c' -> './k/coreutils-8.5/lib/pipe-safer.c' 2024-03-09T09:28:09.090 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/pipe-safer.c’: No data available 2024-03-09T09:28:09.093 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/backupfile.c' -> './k/coreutils-8.5/lib/backupfile.c' 2024-03-09T09:28:09.093 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/backupfile.c’: No data available 2024-03-09T09:28:09.095 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-osf.h' -> './k/coreutils-8.5/lib/iconv_open-osf.h' 2024-03-09T09:28:09.095 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-osf.h’: No data available 2024-03-09T09:28:09.097 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/linebuffer.h' -> './k/coreutils-8.5/lib/linebuffer.h' 2024-03-09T09:28:09.097 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/linebuffer.h’: No data available 2024-03-09T09:28:09.099 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xprintf.h' -> './k/coreutils-8.5/lib/xprintf.h' 2024-03-09T09:28:09.099 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xprintf.h’: No data available 2024-03-09T09:28:09.101 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/exclude.c' -> './k/coreutils-8.5/lib/exclude.c' 2024-03-09T09:28:09.101 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/exclude.c’: No data available 2024-03-09T09:28:09.102 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/userspec.h' -> './k/coreutils-8.5/lib/userspec.h' 2024-03-09T09:28:09.102 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/userspec.h’: No data available 2024-03-09T09:28:09.103 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mkancesdirs.h' -> './k/coreutils-8.5/lib/mkancesdirs.h' 2024-03-09T09:28:09.103 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkancesdirs.h’: No data available 2024-03-09T09:28:09.105 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/hash-pjw.c' -> './k/coreutils-8.5/lib/hash-pjw.c' 2024-03-09T09:28:09.105 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash-pjw.c’: No data available 2024-03-09T09:28:09.107 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fd-safer.c' -> './k/coreutils-8.5/lib/fd-safer.c' 2024-03-09T09:28:09.107 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fd-safer.c’: No data available 2024-03-09T09:28:09.108 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/long-options.h' -> './k/coreutils-8.5/lib/long-options.h' 2024-03-09T09:28:09.108 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/long-options.h’: No data available 2024-03-09T09:28:09.110 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/freadahead.h' -> './k/coreutils-8.5/lib/freadahead.h' 2024-03-09T09:28:09.110 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freadahead.h’: No data available 2024-03-09T09:28:09.112 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/gethostname.c' -> './k/coreutils-8.5/lib/gethostname.c' 2024-03-09T09:28:09.112 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gethostname.c’: No data available 2024-03-09T09:28:09.114 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/putenv.c' -> './k/coreutils-8.5/lib/putenv.c' 2024-03-09T09:28:09.114 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/putenv.c’: No data available 2024-03-09T09:28:09.115 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbrlen.c' -> './k/coreutils-8.5/lib/mbrlen.c' 2024-03-09T09:28:09.116 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbrlen.c’: No data available 2024-03-09T09:28:09.117 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/group-member.c' -> './k/coreutils-8.5/lib/group-member.c' 2024-03-09T09:28:09.117 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/group-member.c’: No data available 2024-03-09T09:28:09.119 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sha512.c' -> './k/coreutils-8.5/lib/sha512.c' 2024-03-09T09:28:09.119 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sha512.c’: No data available 2024-03-09T09:28:09.120 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/signal.in.h' -> './k/coreutils-8.5/lib/signal.in.h' 2024-03-09T09:28:09.120 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/signal.in.h’: No data available 2024-03-09T09:28:09.423 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mktime.c' -> './k/coreutils-8.5/lib/mktime.c' 2024-03-09T09:28:09.423 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mktime.c’: No data available 2024-03-09T09:28:09.425 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/readlink.c' -> './k/coreutils-8.5/lib/readlink.c' 2024-03-09T09:28:09.425 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readlink.c’: No data available 2024-03-09T09:28:09.428 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mkdir-p.c' -> './k/coreutils-8.5/lib/mkdir-p.c' 2024-03-09T09:28:09.428 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkdir-p.c’: No data available 2024-03-09T09:28:09.430 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getndelim2.h' -> './k/coreutils-8.5/lib/getndelim2.h' 2024-03-09T09:28:09.430 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getndelim2.h’: No data available 2024-03-09T09:28:09.432 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memxfrm.h' -> './k/coreutils-8.5/lib/memxfrm.h' 2024-03-09T09:28:09.432 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memxfrm.h’: No data available 2024-03-09T09:28:09.434 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/c-strtod.c' -> './k/coreutils-8.5/lib/c-strtod.c' 2024-03-09T09:28:09.434 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strtod.c’: No data available 2024-03-09T09:28:09.436 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fseterr.h' -> './k/coreutils-8.5/lib/fseterr.h' 2024-03-09T09:28:09.436 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fseterr.h’: No data available 2024-03-09T09:28:09.439 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-hpux.gperf' -> './k/coreutils-8.5/lib/iconv_open-hpux.gperf' 2024-03-09T09:28:09.439 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-hpux.gperf’: No data available 2024-03-09T09:28:09.441 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/filenamecat.c' -> './k/coreutils-8.5/lib/filenamecat.c' 2024-03-09T09:28:09.441 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/filenamecat.c’: No data available 2024-03-09T09:28:09.444 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memmove.c' -> './k/coreutils-8.5/lib/memmove.c' 2024-03-09T09:28:09.444 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memmove.c’: No data available 2024-03-09T09:28:09.446 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/str-kmp.h' -> './k/coreutils-8.5/lib/str-kmp.h' 2024-03-09T09:28:09.446 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/str-kmp.h’: No data available 2024-03-09T09:28:09.449 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/base64.c' -> './k/coreutils-8.5/lib/base64.c' 2024-03-09T09:28:09.449 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/base64.c’: No data available 2024-03-09T09:28:09.451 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/close-hook.c' -> './k/coreutils-8.5/lib/close-hook.c' 2024-03-09T09:28:09.451 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/close-hook.c’: No data available 2024-03-09T09:28:09.453 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/dirchownmod.c' -> './k/coreutils-8.5/lib/dirchownmod.c' 2024-03-09T09:28:09.453 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/dirchownmod.c’: No data available 2024-03-09T09:28:09.455 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/openat-safer.c' -> './k/coreutils-8.5/lib/openat-safer.c' 2024-03-09T09:28:09.456 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat-safer.c’: No data available 2024-03-09T09:28:09.458 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/posixtm.c' -> './k/coreutils-8.5/lib/posixtm.c' 2024-03-09T09:28:09.458 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/posixtm.c’: No data available 2024-03-09T09:28:09.460 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/printf-frexp.c' -> './k/coreutils-8.5/lib/printf-frexp.c' 2024-03-09T09:28:09.460 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/printf-frexp.c’: No data available 2024-03-09T09:28:09.462 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/safe-read.h' -> './k/coreutils-8.5/lib/safe-read.h' 2024-03-09T09:28:09.463 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/safe-read.h’: No data available 2024-03-09T09:28:09.465 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/md5sum-newline' -> './k/coreutils-8.5/tests/misc/md5sum-newline' 2024-03-09T09:28:09.466 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/md5sum-newline’: No data available 2024-03-09T09:28:09.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/regexec.c' -> './k/coreutils-8.5/lib/regexec.c' 2024-03-09T09:28:09.467 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/regexec.c’: No data available 2024-03-09T09:28:09.468 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sha224sum' -> './k/coreutils-8.5/tests/misc/sha224sum' 2024-03-09T09:28:09.469 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha224sum’: No data available 2024-03-09T09:28:09.469 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/uniwidth.h' -> './k/coreutils-8.5/lib/uniwidth.h' 2024-03-09T09:28:09.470 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/uniwidth.h’: No data available 2024-03-09T09:28:09.471 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/stty' -> './k/coreutils-8.5/tests/misc/stty' 2024-03-09T09:28:09.471 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stty’: No data available 2024-03-09T09:28:09.472 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/write-any-file.c' -> './k/coreutils-8.5/lib/write-any-file.c' 2024-03-09T09:28:09.472 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/write-any-file.c’: No data available 2024-03-09T09:28:09.474 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/printf-hex' -> './k/coreutils-8.5/tests/misc/printf-hex' 2024-03-09T09:28:09.474 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/printf-hex’: No data available 2024-03-09T09:28:09.474 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-aix.h' -> './k/coreutils-8.5/lib/iconv_open-aix.h' 2024-03-09T09:28:09.474 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-aix.h’: No data available 2024-03-09T09:28:09.476 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/id-context' -> './k/coreutils-8.5/tests/misc/id-context' 2024-03-09T09:28:09.476 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/id-context’: No data available 2024-03-09T09:28:09.476 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unicodeio.c' -> './k/coreutils-8.5/lib/unicodeio.c' 2024-03-09T09:28:09.477 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unicodeio.c’: No data available 2024-03-09T09:28:09.478 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/stat-printf' -> './k/coreutils-8.5/tests/misc/stat-printf' 2024-03-09T09:28:09.478 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stat-printf’: No data available 2024-03-09T09:28:09.479 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/gettime.c' -> './k/coreutils-8.5/lib/gettime.c' 2024-03-09T09:28:09.479 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/gettime.c’: No data available 2024-03-09T09:28:09.481 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/pwd-option' -> './k/coreutils-8.5/tests/misc/pwd-option' 2024-03-09T09:28:09.481 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/pwd-option’: No data available 2024-03-09T09:28:09.482 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strnumcmp.h' -> './k/coreutils-8.5/lib/strnumcmp.h' 2024-03-09T09:28:09.482 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnumcmp.h’: No data available 2024-03-09T09:28:09.484 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/btowc.c' -> './k/coreutils-8.5/lib/btowc.c' 2024-03-09T09:28:09.484 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/btowc.c’: No data available 2024-03-09T09:28:09.484 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sha256sum' -> './k/coreutils-8.5/tests/misc/sha256sum' 2024-03-09T09:28:09.484 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha256sum’: No data available 2024-03-09T09:28:09.485 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sys_time.in.h' -> './k/coreutils-8.5/lib/sys_time.in.h' 2024-03-09T09:28:09.486 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sys_time.in.h’: No data available 2024-03-09T09:28:09.487 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-version' -> './k/coreutils-8.5/tests/misc/sort-version' 2024-03-09T09:28:09.487 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-version’: No data available 2024-03-09T09:28:09.488 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/isnan.c' -> './k/coreutils-8.5/lib/isnan.c' 2024-03-09T09:28:09.488 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnan.c’: No data available 2024-03-09T09:28:09.489 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/groups-dash' -> './k/coreutils-8.5/tests/misc/groups-dash' 2024-03-09T09:28:09.489 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/groups-dash’: No data available 2024-03-09T09:28:09.490 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getaddrinfo.c' -> './k/coreutils-8.5/lib/getaddrinfo.c' 2024-03-09T09:28:09.490 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getaddrinfo.c’: No data available 2024-03-09T09:28:09.491 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/printenv' -> './k/coreutils-8.5/tests/misc/printenv' 2024-03-09T09:28:09.491 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/printenv’: No data available 2024-03-09T09:28:09.492 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getpagesize.c' -> './k/coreutils-8.5/lib/getpagesize.c' 2024-03-09T09:28:09.492 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getpagesize.c’: No data available 2024-03-09T09:28:09.493 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/posixver.h' -> './k/coreutils-8.5/lib/posixver.h' 2024-03-09T09:28:09.493 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/posixver.h’: No data available 2024-03-09T09:28:09.494 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/seq' -> './k/coreutils-8.5/tests/misc/seq' 2024-03-09T09:28:09.494 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/seq’: No data available 2024-03-09T09:28:09.495 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/openat.c' -> './k/coreutils-8.5/lib/openat.c' 2024-03-09T09:28:09.495 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat.c’: No data available 2024-03-09T09:28:09.496 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/fmt' -> './k/coreutils-8.5/tests/misc/fmt' 2024-03-09T09:28:09.496 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/fmt’: No data available 2024-03-09T09:28:09.498 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/closeout.c' -> './k/coreutils-8.5/lib/closeout.c' 2024-03-09T09:28:09.498 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/cat-buf' -> './k/coreutils-8.5/tests/misc/cat-buf' 2024-03-09T09:28:09.498 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/closeout.c’: No data available 2024-03-09T09:28:09.498 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/cat-buf’: No data available 2024-03-09T09:28:09.500 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fopen.c' -> './k/coreutils-8.5/lib/fopen.c' 2024-03-09T09:28:09.500 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fopen.c’: No data available 2024-03-09T09:28:09.500 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/dircolors' -> './k/coreutils-8.5/tests/misc/dircolors' 2024-03-09T09:28:09.500 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/dircolors’: No data available 2024-03-09T09:28:09.501 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/fold' -> './k/coreutils-8.5/tests/misc/fold' 2024-03-09T09:28:09.502 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/fold’: No data available 2024-03-09T09:28:09.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memchr2.valgrind' -> './k/coreutils-8.5/lib/memchr2.valgrind' 2024-03-09T09:28:09.502 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memchr2.valgrind’: No data available 2024-03-09T09:28:09.504 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/selinux' -> './k/coreutils-8.5/tests/misc/selinux' 2024-03-09T09:28:09.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/time.in.h' -> './k/coreutils-8.5/lib/time.in.h' 2024-03-09T09:28:09.504 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/selinux’: No data available 2024-03-09T09:28:09.505 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/time.in.h’: No data available 2024-03-09T09:28:09.506 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/md5sum-parallel' -> './k/coreutils-8.5/tests/misc/md5sum-parallel' 2024-03-09T09:28:09.507 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/md5sum-parallel’: No data available 2024-03-09T09:28:09.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/physmem.h' -> './k/coreutils-8.5/lib/physmem.h' 2024-03-09T09:28:09.507 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/physmem.h’: No data available 2024-03-09T09:28:09.508 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/md5sum' -> './k/coreutils-8.5/tests/misc/md5sum' 2024-03-09T09:28:09.508 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/md5sum’: No data available 2024-03-09T09:28:09.509 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/iconv_open-irix.gperf' -> './k/coreutils-8.5/lib/iconv_open-irix.gperf' 2024-03-09T09:28:09.509 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/iconv_open-irix.gperf’: No data available 2024-03-09T09:28:09.510 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/nice-fail' -> './k/coreutils-8.5/tests/misc/nice-fail' 2024-03-09T09:28:09.510 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nice-fail’: No data available 2024-03-09T09:28:09.511 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stat.c' -> './k/coreutils-8.5/lib/stat.c' 2024-03-09T09:28:09.511 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stat.c’: No data available 2024-03-09T09:28:09.512 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/siglist.h' -> './k/coreutils-8.5/lib/siglist.h' 2024-03-09T09:28:09.512 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/siglist.h’: No data available 2024-03-09T09:28:09.513 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/groups-version' -> './k/coreutils-8.5/tests/misc/groups-version' 2024-03-09T09:28:09.513 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/groups-version’: No data available 2024-03-09T09:28:09.514 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/date-sec' -> './k/coreutils-8.5/tests/misc/date-sec' 2024-03-09T09:28:09.514 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/date-sec’: No data available 2024-03-09T09:28:09.515 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/split-l' -> './k/coreutils-8.5/tests/misc/split-l' 2024-03-09T09:28:09.516 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/split-l’: No data available 2024-03-09T09:28:09.517 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/false-status' -> './k/coreutils-8.5/tests/misc/false-status' 2024-03-09T09:28:09.518 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/false-status’: No data available 2024-03-09T09:28:09.519 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sha384sum' -> './k/coreutils-8.5/tests/misc/sha384sum' 2024-03-09T09:28:09.519 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha384sum’: No data available 2024-03-09T09:28:09.521 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-dangling-symlink' -> './k/coreutils-8.5/tests/misc/truncate-dangling-symlink' 2024-03-09T09:28:09.521 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-dangling-symlink’: No data available 2024-03-09T09:28:09.522 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/od-multiple-t' -> './k/coreutils-8.5/tests/misc/od-multiple-t' 2024-03-09T09:28:09.523 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/od-multiple-t’: No data available 2024-03-09T09:28:09.524 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/wc-files0' -> './k/coreutils-8.5/tests/misc/wc-files0' 2024-03-09T09:28:09.524 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/wc-files0’: No data available 2024-03-09T09:28:09.526 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/nohup' -> './k/coreutils-8.5/tests/misc/nohup' 2024-03-09T09:28:09.527 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nohup’: No data available 2024-03-09T09:28:09.528 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-no-create-missing' -> './k/coreutils-8.5/tests/misc/truncate-no-create-missing' 2024-03-09T09:28:09.528 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-no-create-missing’: No data available 2024-03-09T09:28:09.529 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/head-pos' -> './k/coreutils-8.5/tests/misc/head-pos' 2024-03-09T09:28:09.529 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/head-pos’: No data available 2024-03-09T09:28:09.531 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/uniq' -> './k/coreutils-8.5/tests/misc/uniq' 2024-03-09T09:28:09.531 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/uniq’: No data available 2024-03-09T09:28:09.533 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/test-diag' -> './k/coreutils-8.5/tests/misc/test-diag' 2024-03-09T09:28:09.533 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/test-diag’: No data available 2024-03-09T09:28:09.535 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/tac' -> './k/coreutils-8.5/tests/misc/tac' 2024-03-09T09:28:09.535 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tac’: No data available 2024-03-09T09:28:09.537 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-continue' -> './k/coreutils-8.5/tests/misc/sort-continue' 2024-03-09T09:28:09.537 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-continue’: No data available 2024-03-09T09:28:09.538 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/head' -> './k/coreutils-8.5/tests/misc/head' 2024-03-09T09:28:09.538 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/head’: No data available 2024-03-09T09:28:09.540 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/ptx' -> './k/coreutils-8.5/tests/misc/ptx' 2024-03-09T09:28:09.540 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/ptx’: No data available 2024-03-09T09:28:09.542 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/nproc-avail' -> './k/coreutils-8.5/tests/misc/nproc-avail' 2024-03-09T09:28:09.542 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nproc-avail’: No data available 2024-03-09T09:28:09.543 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/su-fail' -> './k/coreutils-8.5/tests/misc/su-fail' 2024-03-09T09:28:09.544 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/su-fail’: No data available 2024-03-09T09:28:09.545 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/od-x8' -> './k/coreutils-8.5/tests/misc/od-x8' 2024-03-09T09:28:09.545 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/od-x8’: No data available 2024-03-09T09:28:09.547 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/nice' -> './k/coreutils-8.5/tests/misc/nice' 2024-03-09T09:28:09.547 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nice’: No data available 2024-03-09T09:28:09.549 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/chcon' -> './k/coreutils-8.5/tests/misc/chcon' 2024-03-09T09:28:09.549 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/chcon’: No data available 2024-03-09T09:28:09.551 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/head-c' -> './k/coreutils-8.5/tests/misc/head-c' 2024-03-09T09:28:09.551 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/head-c’: No data available 2024-03-09T09:28:09.552 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-compress' -> './k/coreutils-8.5/tests/misc/sort-compress' 2024-03-09T09:28:09.553 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-compress’: No data available 2024-03-09T09:28:09.554 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/df' -> './k/coreutils-8.5/tests/misc/df' 2024-03-09T09:28:09.554 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/df’: No data available 2024-03-09T09:28:09.556 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/chroot-fail' -> './k/coreutils-8.5/tests/misc/chroot-fail' 2024-03-09T09:28:09.556 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/chroot-fail’: No data available 2024-03-09T09:28:09.558 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/date-next-dow' -> './k/coreutils-8.5/tests/misc/date-next-dow' 2024-03-09T09:28:09.558 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/date-next-dow’: No data available 2024-03-09T09:28:09.560 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sha1sum' -> './k/coreutils-8.5/tests/misc/sha1sum' 2024-03-09T09:28:09.560 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha1sum’: No data available 2024-03-09T09:28:09.563 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sha1sum-vec' -> './k/coreutils-8.5/tests/misc/sha1sum-vec' 2024-03-09T09:28:09.563 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha1sum-vec’: No data available 2024-03-09T09:28:09.566 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/pathchk1' -> './k/coreutils-8.5/tests/misc/pathchk1' 2024-03-09T09:28:09.566 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/pathchk1’: No data available 2024-03-09T09:28:09.567 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/arch' -> './k/coreutils-8.5/tests/misc/arch' 2024-03-09T09:28:09.568 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/arch’: No data available 2024-03-09T09:28:09.569 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-month' -> './k/coreutils-8.5/tests/misc/sort-month' 2024-03-09T09:28:09.569 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-month’: No data available 2024-03-09T09:28:09.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/shred-passes' -> './k/coreutils-8.5/tests/misc/shred-passes' 2024-03-09T09:28:09.571 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/shred-passes’: No data available 2024-03-09T09:28:09.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sha512sum' -> './k/coreutils-8.5/tests/misc/sha512sum' 2024-03-09T09:28:09.573 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha512sum’: No data available 2024-03-09T09:28:09.575 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/paste' -> './k/coreutils-8.5/tests/misc/paste' 2024-03-09T09:28:09.575 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/paste’: No data available 2024-03-09T09:28:09.577 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/wc' -> './k/coreutils-8.5/tests/misc/wc' 2024-03-09T09:28:09.577 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/wc’: No data available 2024-03-09T09:28:09.578 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/ls-time' -> './k/coreutils-8.5/tests/misc/ls-time' 2024-03-09T09:28:09.579 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/ls-time’: No data available 2024-03-09T09:28:09.580 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/pwd-unreadable-parent' -> './k/coreutils-8.5/tests/misc/pwd-unreadable-parent' 2024-03-09T09:28:09.581 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/pwd-unreadable-parent’: No data available 2024-03-09T09:28:09.582 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/test' -> './k/coreutils-8.5/tests/misc/test' 2024-03-09T09:28:09.582 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/test’: No data available 2024-03-09T09:28:09.584 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-parameters' -> './k/coreutils-8.5/tests/misc/truncate-parameters' 2024-03-09T09:28:09.585 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-parameters’: No data available 2024-03-09T09:28:09.587 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-files0-from' -> './k/coreutils-8.5/tests/misc/sort-files0-from' 2024-03-09T09:28:09.587 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-files0-from’: No data available 2024-03-09T09:28:09.589 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/base64' -> './k/coreutils-8.5/tests/misc/base64' 2024-03-09T09:28:09.590 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/base64’: No data available 2024-03-09T09:28:09.592 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/ptx-overrun' -> './k/coreutils-8.5/tests/misc/ptx-overrun' 2024-03-09T09:28:09.593 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/ptx-overrun’: No data available 2024-03-09T09:28:09.595 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/printf-cov' -> './k/coreutils-8.5/tests/misc/printf-cov' 2024-03-09T09:28:09.596 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/printf-cov’: No data available 2024-03-09T09:28:09.597 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/tee' -> './k/coreutils-8.5/tests/misc/tee' 2024-03-09T09:28:09.598 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tee’: No data available 2024-03-09T09:28:09.600 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/mknod' -> './k/coreutils-8.5/tests/misc/mknod' 2024-03-09T09:28:09.600 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/mknod’: No data available 2024-03-09T09:28:09.605 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/printf-surprise' -> './k/coreutils-8.5/tests/misc/printf-surprise' 2024-03-09T09:28:09.606 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/printf-surprise’: No data available 2024-03-09T09:28:09.610 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/ls-misc' -> './k/coreutils-8.5/tests/misc/ls-misc' 2024-03-09T09:28:09.610 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/ls-misc’: No data available 2024-03-09T09:28:09.613 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/nproc-positive' -> './k/coreutils-8.5/tests/misc/nproc-positive' 2024-03-09T09:28:09.613 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nproc-positive’: No data available 2024-03-09T09:28:09.619 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/join' -> './k/coreutils-8.5/tests/misc/join' 2024-03-09T09:28:09.619 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/join’: No data available 2024-03-09T09:28:09.622 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-merge' -> './k/coreutils-8.5/tests/misc/sort-merge' 2024-03-09T09:28:09.622 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-merge’: No data available 2024-03-09T09:28:09.625 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/factor' -> './k/coreutils-8.5/tests/misc/factor' 2024-03-09T09:28:09.625 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/factor’: No data available 2024-03-09T09:28:09.628 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sort' -> './k/coreutils-8.5/tests/misc/sort' 2024-03-09T09:28:09.628 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort’: No data available 2024-03-09T09:28:09.630 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-fifo' -> './k/coreutils-8.5/tests/misc/truncate-fifo' 2024-03-09T09:28:09.631 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-fifo’: No data available 2024-03-09T09:28:09.633 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-fail-diag' -> './k/coreutils-8.5/tests/misc/truncate-fail-diag' 2024-03-09T09:28:09.633 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-fail-diag’: No data available 2024-03-09T09:28:09.635 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/stty-invalid' -> './k/coreutils-8.5/tests/misc/stty-invalid' 2024-03-09T09:28:09.635 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stty-invalid’: No data available 2024-03-09T09:28:09.637 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/env-null' -> './k/coreutils-8.5/tests/misc/env-null' 2024-03-09T09:28:09.638 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/env-null’: No data available 2024-03-09T09:28:09.640 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/printf' -> './k/coreutils-8.5/tests/misc/printf' 2024-03-09T09:28:09.640 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/printf’: No data available 2024-03-09T09:28:09.642 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/seq-long-double' -> './k/coreutils-8.5/tests/misc/seq-long-double' 2024-03-09T09:28:09.643 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/seq-long-double’: No data available 2024-03-09T09:28:09.644 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/pwd-long' -> './k/coreutils-8.5/tests/misc/pwd-long' 2024-03-09T09:28:09.645 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/pwd-long’: No data available 2024-03-09T09:28:09.647 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/tsort' -> './k/coreutils-8.5/tests/misc/tsort' 2024-03-09T09:28:09.647 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tsort’: No data available 2024-03-09T09:28:09.650 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/runcon-no-reorder' -> './k/coreutils-8.5/tests/misc/runcon-no-reorder' 2024-03-09T09:28:09.650 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/runcon-no-reorder’: No data available 2024-03-09T09:28:09.653 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/date' -> './k/coreutils-8.5/tests/misc/date' 2024-03-09T09:28:09.654 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/date’: No data available 2024-03-09T09:28:09.656 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/shred-exact' -> './k/coreutils-8.5/tests/misc/shred-exact' 2024-03-09T09:28:09.656 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/shred-exact’: No data available 2024-03-09T09:28:09.658 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/shred-remove' -> './k/coreutils-8.5/tests/misc/shred-remove' 2024-03-09T09:28:09.658 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/shred-remove’: No data available 2024-03-09T09:28:09.660 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/stdbuf' -> './k/coreutils-8.5/tests/misc/stdbuf' 2024-03-09T09:28:09.660 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stdbuf’: No data available 2024-03-09T09:28:09.661 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-overflow' -> './k/coreutils-8.5/tests/misc/truncate-overflow' 2024-03-09T09:28:09.662 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-overflow’: No data available 2024-03-09T09:28:09.663 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/stty-row-col' -> './k/coreutils-8.5/tests/misc/stty-row-col' 2024-03-09T09:28:09.664 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stty-row-col’: No data available 2024-03-09T09:28:09.666 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-owned-by-other' -> './k/coreutils-8.5/tests/misc/truncate-owned-by-other' 2024-03-09T09:28:09.666 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-owned-by-other’: No data available 2024-03-09T09:28:09.668 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/xstrtol' -> './k/coreutils-8.5/tests/misc/xstrtol' 2024-03-09T09:28:09.668 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/xstrtol’: No data available 2024-03-09T09:28:09.670 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/invalid-opt' -> './k/coreutils-8.5/tests/misc/invalid-opt' 2024-03-09T09:28:09.670 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/invalid-opt’: No data available 2024-03-09T09:28:09.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/readlink-fp-loop' -> './k/coreutils-8.5/tests/misc/readlink-fp-loop' 2024-03-09T09:28:09.672 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/readlink-fp-loop’: No data available 2024-03-09T09:28:09.674 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/tr' -> './k/coreutils-8.5/tests/misc/tr' 2024-03-09T09:28:09.674 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tr’: No data available 2024-03-09T09:28:09.676 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/help-version' -> './k/coreutils-8.5/tests/misc/help-version' 2024-03-09T09:28:09.677 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/help-version’: No data available 2024-03-09T09:28:09.679 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/id-groups' -> './k/coreutils-8.5/tests/misc/id-groups' 2024-03-09T09:28:09.680 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/id-groups’: No data available 2024-03-09T09:28:09.682 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/unexpand' -> './k/coreutils-8.5/tests/misc/unexpand' 2024-03-09T09:28:09.682 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/unexpand’: No data available 2024-03-09T09:28:09.684 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-merge-fdlimit' -> './k/coreutils-8.5/tests/misc/sort-merge-fdlimit' 2024-03-09T09:28:09.684 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-merge-fdlimit’: No data available 2024-03-09T09:28:09.952 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbswidth.h' -> './k/coreutils-8.5/lib/mbswidth.h' 2024-03-09T09:28:09.952 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbswidth.h’: No data available 2024-03-09T09:28:09.960 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstrtod.h' -> './k/coreutils-8.5/lib/xstrtod.h' 2024-03-09T09:28:09.960 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtod.h’: No data available 2024-03-09T09:28:09.962 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/md5.c' -> './k/coreutils-8.5/lib/md5.c' 2024-03-09T09:28:09.963 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/md5.c’: No data available 2024-03-09T09:28:09.965 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getloadavg.c' -> './k/coreutils-8.5/lib/getloadavg.c' 2024-03-09T09:28:09.965 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getloadavg.c’: No data available 2024-03-09T09:28:09.967 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/getdate.h' -> './k/coreutils-8.5/lib/getdate.h' 2024-03-09T09:28:09.967 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/getdate.h’: No data available 2024-03-09T09:28:09.969 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stdlib-safer.h' -> './k/coreutils-8.5/lib/stdlib-safer.h' 2024-03-09T09:28:09.969 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdlib-safer.h’: No data available 2024-03-09T09:28:09.971 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/remove.c' -> './k/coreutils-8.5/lib/remove.c' 2024-03-09T09:28:09.971 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/remove.c’: No data available 2024-03-09T09:28:09.974 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/malloc.c' -> './k/coreutils-8.5/lib/malloc.c' 2024-03-09T09:28:09.974 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/malloc.c’: No data available 2024-03-09T09:28:09.975 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/progname.h' -> './k/coreutils-8.5/lib/progname.h' 2024-03-09T09:28:09.976 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/progname.h’: No data available 2024-03-09T09:28:09.978 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/argmatch.h' -> './k/coreutils-8.5/lib/argmatch.h' 2024-03-09T09:28:09.978 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/argmatch.h’: No data available 2024-03-09T09:28:09.984 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/ChangeLog-2007' -> './k/coreutils-8.5/lib/ChangeLog-2007' 2024-03-09T09:28:09.984 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/ChangeLog-2007’: No data available 2024-03-09T09:28:09.986 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mbswidth.c' -> './k/coreutils-8.5/lib/mbswidth.c' 2024-03-09T09:28:09.986 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mbswidth.c’: No data available 2024-03-09T09:28:09.989 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/error.h' -> './k/coreutils-8.5/lib/error.h' 2024-03-09T09:28:09.989 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/error.h’: No data available 2024-03-09T09:28:09.991 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/obstack.c' -> './k/coreutils-8.5/lib/obstack.c' 2024-03-09T09:28:09.991 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/obstack.c’: No data available 2024-03-09T09:28:09.994 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/acl.h' -> './k/coreutils-8.5/lib/acl.h' 2024-03-09T09:28:09.994 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/acl.h’: No data available 2024-03-09T09:28:09.996 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strnumcmp-in.h' -> './k/coreutils-8.5/lib/strnumcmp-in.h' 2024-03-09T09:28:09.996 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnumcmp-in.h’: No data available 2024-03-09T09:28:09.998 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xstrtoul.c' -> './k/coreutils-8.5/lib/xstrtoul.c' 2024-03-09T09:28:09.999 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xstrtoul.c’: No data available 2024-03-09T09:28:10.001 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/float+.h' -> './k/coreutils-8.5/lib/float+.h' 2024-03-09T09:28:10.001 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/float+.h’: No data available 2024-03-09T09:28:10.003 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/canonicalize.c' -> './k/coreutils-8.5/lib/canonicalize.c' 2024-03-09T09:28:10.003 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/canonicalize.c’: No data available 2024-03-09T09:28:10.006 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fcntl--.h' -> './k/coreutils-8.5/lib/fcntl--.h' 2024-03-09T09:28:10.006 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fcntl--.h’: No data available 2024-03-09T09:28:10.008 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/wchar.in.h' -> './k/coreutils-8.5/lib/wchar.in.h' 2024-03-09T09:28:10.008 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/wchar.in.h’: No data available 2024-03-09T09:28:10.010 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/trim.c' -> './k/coreutils-8.5/lib/trim.c' 2024-03-09T09:28:10.010 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/trim.c’: No data available 2024-03-09T09:28:10.013 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/rpmatch.c' -> './k/coreutils-8.5/lib/rpmatch.c' 2024-03-09T09:28:10.013 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/rpmatch.c’: No data available 2024-03-09T09:28:10.015 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/isapipe.h' -> './k/coreutils-8.5/lib/isapipe.h' 2024-03-09T09:28:10.015 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isapipe.h’: No data available 2024-03-09T09:28:10.017 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/memchr.c' -> './k/coreutils-8.5/lib/memchr.c' 2024-03-09T09:28:10.017 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/memchr.c’: No data available 2024-03-09T09:28:10.020 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/localcharset.h' -> './k/coreutils-8.5/lib/localcharset.h' 2024-03-09T09:28:10.020 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/localcharset.h’: No data available 2024-03-09T09:28:10.022 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/sig-handler.h' -> './k/coreutils-8.5/lib/sig-handler.h' 2024-03-09T09:28:10.023 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/sig-handler.h’: No data available 2024-03-09T09:28:10.025 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/openat-proc.c' -> './k/coreutils-8.5/lib/openat-proc.c' 2024-03-09T09:28:10.025 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/openat-proc.c’: No data available 2024-03-09T09:28:10.027 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/pathmax.h' -> './k/coreutils-8.5/lib/pathmax.h' 2024-03-09T09:28:10.027 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/pathmax.h’: No data available 2024-03-09T09:28:10.029 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/close-stream.c' -> './k/coreutils-8.5/lib/close-stream.c' 2024-03-09T09:28:10.029 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/close-stream.c’: No data available 2024-03-09T09:28:10.032 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/readutmp.c' -> './k/coreutils-8.5/lib/readutmp.c' 2024-03-09T09:28:10.032 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readutmp.c’: No data available 2024-03-09T09:28:10.034 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/float.in.h' -> './k/coreutils-8.5/lib/float.in.h' 2024-03-09T09:28:10.034 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/float.in.h’: No data available 2024-03-09T09:28:10.036 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/creat-safer.c' -> './k/coreutils-8.5/lib/creat-safer.c' 2024-03-09T09:28:10.036 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/creat-safer.c’: No data available 2024-03-09T09:28:10.038 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/randperm.c' -> './k/coreutils-8.5/lib/randperm.c' 2024-03-09T09:28:10.038 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/randperm.c’: No data available 2024-03-09T09:28:10.041 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/readtokens.c' -> './k/coreutils-8.5/lib/readtokens.c' 2024-03-09T09:28:10.041 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/readtokens.c’: No data available 2024-03-09T09:28:10.043 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/safe-read.c' -> './k/coreutils-8.5/lib/safe-read.c' 2024-03-09T09:28:10.043 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/safe-read.c’: No data available 2024-03-09T09:28:10.045 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xreadlink.h' -> './k/coreutils-8.5/lib/xreadlink.h' 2024-03-09T09:28:10.045 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xreadlink.h’: No data available 2024-03-09T09:28:10.047 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/stdarg.in.h' -> './k/coreutils-8.5/lib/stdarg.in.h' 2024-03-09T09:28:10.048 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/stdarg.in.h’: No data available 2024-03-09T09:28:10.049 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xalloc-die.c' -> './k/coreutils-8.5/lib/xalloc-die.c' 2024-03-09T09:28:10.049 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xalloc-die.c’: No data available 2024-03-09T09:28:10.051 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/copy-acl.c' -> './k/coreutils-8.5/lib/copy-acl.c' 2024-03-09T09:28:10.051 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/copy-acl.c’: No data available 2024-03-09T09:28:10.053 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fsync.c' -> './k/coreutils-8.5/lib/fsync.c' 2024-03-09T09:28:10.053 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fsync.c’: No data available 2024-03-09T09:28:10.055 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strnumcmp.c' -> './k/coreutils-8.5/lib/strnumcmp.c' 2024-03-09T09:28:10.055 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strnumcmp.c’: No data available 2024-03-09T09:28:10.057 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/selinux-at.c' -> './k/coreutils-8.5/lib/selinux-at.c' 2024-03-09T09:28:10.057 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/selinux-at.c’: No data available 2024-03-09T09:28:10.059 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/realloc.c' -> './k/coreutils-8.5/lib/realloc.c' 2024-03-09T09:28:10.059 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/realloc.c’: No data available 2024-03-09T09:28:10.060 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fdopendir.c' -> './k/coreutils-8.5/lib/fdopendir.c' 2024-03-09T09:28:10.061 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fdopendir.c’: No data available 2024-03-09T09:28:10.063 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/alloca.in.h' -> './k/coreutils-8.5/lib/alloca.in.h' 2024-03-09T09:28:10.063 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/alloca.in.h’: No data available 2024-03-09T09:28:10.065 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/freopen.c' -> './k/coreutils-8.5/lib/freopen.c' 2024-03-09T09:28:10.065 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/freopen.c’: No data available 2024-03-09T09:28:10.067 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/mkdir-p.h' -> './k/coreutils-8.5/lib/mkdir-p.h' 2024-03-09T09:28:10.067 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/mkdir-p.h’: No data available 2024-03-09T09:28:10.068 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/isnand-nolibm.h' -> './k/coreutils-8.5/lib/isnand-nolibm.h' 2024-03-09T09:28:10.069 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/isnand-nolibm.h’: No data available 2024-03-09T09:28:10.069 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/propername.c' -> './k/coreutils-8.5/lib/propername.c' 2024-03-09T09:28:10.070 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/propername.c’: No data available 2024-03-09T09:28:10.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chgrp' -> './k/coreutils-8.5/tests/chgrp' 2024-03-09T09:28:10.571 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chgrp/default-no-deref' -> './k/coreutils-8.5/tests/chgrp/default-no-deref' 2024-03-09T09:28:10.571 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/default-no-deref’: No data available 2024-03-09T09:28:10.573 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chgrp/deref' -> './k/coreutils-8.5/tests/chgrp/deref' 2024-03-09T09:28:10.574 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/deref’: No data available 2024-03-09T09:28:10.576 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chgrp/posix-H' -> './k/coreutils-8.5/tests/chgrp/posix-H' 2024-03-09T09:28:10.577 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/posix-H’: No data available 2024-03-09T09:28:10.579 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chgrp/recurse' -> './k/coreutils-8.5/tests/chgrp/recurse' 2024-03-09T09:28:10.579 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/recurse’: No data available 2024-03-09T09:28:10.582 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chgrp/no-x' -> './k/coreutils-8.5/tests/chgrp/no-x' 2024-03-09T09:28:10.582 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/no-x’: No data available 2024-03-09T09:28:10.585 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/chgrp/basic' -> './k/coreutils-8.5/tests/chgrp/basic' 2024-03-09T09:28:10.585 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/basic’: No data available 2024-03-09T09:28:10.628 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2' -> './k/coreutils-8.5/tests/tail-2' 2024-03-09T09:28:10.629 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/follow-stdin' -> './k/coreutils-8.5/tests/tail-2/follow-stdin' 2024-03-09T09:28:10.629 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/follow-stdin’: No data available 2024-03-09T09:28:10.631 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/F-vs-rename' -> './k/coreutils-8.5/tests/tail-2/F-vs-rename' 2024-03-09T09:28:10.631 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/F-vs-rename’: No data available 2024-03-09T09:28:10.633 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/inotify-rotate' -> './k/coreutils-8.5/tests/tail-2/inotify-rotate' 2024-03-09T09:28:10.634 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/inotify-rotate’: No data available 2024-03-09T09:28:10.636 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/pid' -> './k/coreutils-8.5/tests/tail-2/pid' 2024-03-09T09:28:10.636 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/pid’: No data available 2024-03-09T09:28:10.638 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/infloop-1' -> './k/coreutils-8.5/tests/tail-2/infloop-1' 2024-03-09T09:28:10.638 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/infloop-1’: No data available 2024-03-09T09:28:10.641 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/assert-2' -> './k/coreutils-8.5/tests/tail-2/assert-2' 2024-03-09T09:28:10.641 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/assert-2’: No data available 2024-03-09T09:28:10.643 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/pipe-f' -> './k/coreutils-8.5/tests/tail-2/pipe-f' 2024-03-09T09:28:10.643 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/pipe-f’: No data available 2024-03-09T09:28:10.646 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/pipe-f2' -> './k/coreutils-8.5/tests/tail-2/pipe-f2' 2024-03-09T09:28:10.646 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/pipe-f2’: No data available 2024-03-09T09:28:10.648 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/start-middle' -> './k/coreutils-8.5/tests/tail-2/start-middle' 2024-03-09T09:28:10.648 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/start-middle’: No data available 2024-03-09T09:28:10.651 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/inotify-hash-abuse2' -> './k/coreutils-8.5/tests/tail-2/inotify-hash-abuse2' 2024-03-09T09:28:10.651 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/inotify-hash-abuse2’: No data available 2024-03-09T09:28:10.653 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/inotify-hash-abuse' -> './k/coreutils-8.5/tests/tail-2/inotify-hash-abuse' 2024-03-09T09:28:10.653 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/inotify-hash-abuse’: No data available 2024-03-09T09:28:10.656 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/inotify-race' -> './k/coreutils-8.5/tests/tail-2/inotify-race' 2024-03-09T09:28:10.656 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/inotify-race’: No data available 2024-03-09T09:28:10.658 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/append-only' -> './k/coreutils-8.5/tests/tail-2/append-only' 2024-03-09T09:28:10.658 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/append-only’: No data available 2024-03-09T09:28:10.660 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/tail-n0f' -> './k/coreutils-8.5/tests/tail-2/tail-n0f' 2024-03-09T09:28:10.660 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/tail-n0f’: No data available 2024-03-09T09:28:10.662 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/flush-initial' -> './k/coreutils-8.5/tests/tail-2/flush-initial' 2024-03-09T09:28:10.663 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/flush-initial’: No data available 2024-03-09T09:28:10.665 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/assert' -> './k/coreutils-8.5/tests/tail-2/assert' 2024-03-09T09:28:10.665 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/assert’: No data available 2024-03-09T09:28:10.668 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/wait' -> './k/coreutils-8.5/tests/tail-2/wait' 2024-03-09T09:28:10.668 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/wait’: No data available 2024-03-09T09:28:10.670 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/proc-ksyms' -> './k/coreutils-8.5/tests/tail-2/proc-ksyms' 2024-03-09T09:28:10.671 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/proc-ksyms’: No data available 2024-03-09T09:28:10.672 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/tail-2/big-4gb' -> './k/coreutils-8.5/tests/tail-2/big-4gb' 2024-03-09T09:28:10.673 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/big-4gb’: No data available 2024-03-09T09:28:10.719 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/readlink' -> './k/coreutils-8.5/tests/readlink' 2024-03-09T09:28:10.720 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/readlink/can-m' -> './k/coreutils-8.5/tests/readlink/can-m' 2024-03-09T09:28:10.720 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/readlink/can-m’: No data available 2024-03-09T09:28:10.722 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/readlink/rl-1' -> './k/coreutils-8.5/tests/readlink/rl-1' 2024-03-09T09:28:10.722 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/readlink/rl-1’: No data available 2024-03-09T09:28:10.724 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/readlink/can-e' -> './k/coreutils-8.5/tests/readlink/can-e' 2024-03-09T09:28:10.725 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/readlink/can-e’: No data available 2024-03-09T09:28:10.727 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/tests/readlink/can-f' -> './k/coreutils-8.5/tests/readlink/can-f' 2024-03-09T09:28:10.727 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/readlink/can-f’: No data available 2024-03-09T09:28:10.763 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_trailing_blank' -> './k/coreutils-8.5/.x-sc_trailing_blank' 2024-03-09T09:28:10.764 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_trailing_blank’: No data available 2024-03-09T09:28:10.782 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/configure' -> './k/coreutils-8.5/configure' 2024-03-09T09:28:10.783 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/configure’: No data available 2024-03-09T09:28:10.785 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/README' -> './k/coreutils-8.5/README' 2024-03-09T09:28:10.786 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/README’: No data available 2024-03-09T09:28:10.788 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.vg-suppressions' -> './k/coreutils-8.5/.vg-suppressions' 2024-03-09T09:28:10.788 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.vg-suppressions’: No data available 2024-03-09T09:28:10.790 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_atoi_atof' -> './k/coreutils-8.5/.x-sc_prohibit_atoi_atof' 2024-03-09T09:28:10.790 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_atoi_atof’: No data available 2024-03-09T09:28:10.793 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_stat_st_blocks' -> './k/coreutils-8.5/.x-sc_prohibit_stat_st_blocks' 2024-03-09T09:28:10.793 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_stat_st_blocks’: No data available 2024-03-09T09:28:10.797 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/ChangeLog-2006' -> './k/coreutils-8.5/ChangeLog-2006' 2024-03-09T09:28:10.797 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ChangeLog-2006’: No data available 2024-03-09T09:28:10.799 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_program_name' -> './k/coreutils-8.5/.x-sc_program_name' 2024-03-09T09:28:10.800 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_program_name’: No data available 2024-03-09T09:28:10.802 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/THANKStt.in' -> './k/coreutils-8.5/THANKStt.in' 2024-03-09T09:28:10.802 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/THANKStt.in’: No data available 2024-03-09T09:28:10.804 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/ChangeLog-2008' -> './k/coreutils-8.5/ChangeLog-2008' 2024-03-09T09:28:10.804 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ChangeLog-2008’: No data available 2024-03-09T09:28:10.809 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/ChangeLog-2007' -> './k/coreutils-8.5/ChangeLog-2007' 2024-03-09T09:28:10.809 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ChangeLog-2007’: No data available 2024-03-09T09:28:10.823 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc' -> './k/coreutils-8.5/doc' 2024-03-09T09:28:10.823 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc/perm.texi' -> './k/coreutils-8.5/doc/perm.texi' 2024-03-09T09:28:10.823 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/perm.texi’: No data available 2024-03-09T09:28:10.825 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc/version.texi' -> './k/coreutils-8.5/doc/version.texi' 2024-03-09T09:28:10.826 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/version.texi’: No data available 2024-03-09T09:28:10.830 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc/Makefile.am' -> './k/coreutils-8.5/doc/Makefile.am' 2024-03-09T09:28:10.830 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/Makefile.am’: No data available 2024-03-09T09:28:10.843 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc/coreutils.info' -> './k/coreutils-8.5/doc/coreutils.info' 2024-03-09T09:28:10.844 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/coreutils.info’: No data available 2024-03-09T09:28:10.845 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc/stamp-vti' -> './k/coreutils-8.5/doc/stamp-vti' 2024-03-09T09:28:10.846 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/stamp-vti’: No data available 2024-03-09T09:28:10.848 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc/fdl.texi' -> './k/coreutils-8.5/doc/fdl.texi' 2024-03-09T09:28:10.849 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/fdl.texi’: No data available 2024-03-09T09:28:10.851 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc/Makefile.in' -> './k/coreutils-8.5/doc/Makefile.in' 2024-03-09T09:28:10.852 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/Makefile.in’: No data available 2024-03-09T09:28:10.854 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc/getdate.texi' -> './k/coreutils-8.5/doc/getdate.texi' 2024-03-09T09:28:10.855 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/getdate.texi’: No data available 2024-03-09T09:28:10.857 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc/constants.texi' -> './k/coreutils-8.5/doc/constants.texi' 2024-03-09T09:28:10.858 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/constants.texi’: No data available 2024-03-09T09:28:10.860 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc/ChangeLog-2007' -> './k/coreutils-8.5/doc/ChangeLog-2007' 2024-03-09T09:28:10.861 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/ChangeLog-2007’: No data available 2024-03-09T09:28:10.869 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/doc/coreutils.texi' -> './k/coreutils-8.5/doc/coreutils.texi' 2024-03-09T09:28:10.869 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/coreutils.texi’: No data available 2024-03-09T09:28:10.945 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src' -> './k/coreutils-8.5/src' 2024-03-09T09:28:10.945 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/dircolors.hin' -> './k/coreutils-8.5/src/dircolors.hin' 2024-03-09T09:28:10.946 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dircolors.hin’: No data available 2024-03-09T09:28:10.948 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/test.c' -> './k/coreutils-8.5/src/test.c' 2024-03-09T09:28:10.948 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/test.c’: No data available 2024-03-09T09:28:10.950 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/nproc.c' -> './k/coreutils-8.5/src/nproc.c' 2024-03-09T09:28:10.950 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/nproc.c’: No data available 2024-03-09T09:28:10.953 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/cp-hash.c' -> './k/coreutils-8.5/src/cp-hash.c' 2024-03-09T09:28:10.953 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cp-hash.c’: No data available 2024-03-09T09:28:10.955 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/rm.c' -> './k/coreutils-8.5/src/rm.c' 2024-03-09T09:28:10.956 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/rm.c’: No data available 2024-03-09T09:28:10.957 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/uname-arch.c' -> './k/coreutils-8.5/src/uname-arch.c' 2024-03-09T09:28:10.958 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uname-arch.c’: No data available 2024-03-09T09:28:10.960 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/chroot.c' -> './k/coreutils-8.5/src/chroot.c' 2024-03-09T09:28:10.960 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chroot.c’: No data available 2024-03-09T09:28:10.962 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/remove.h' -> './k/coreutils-8.5/src/remove.h' 2024-03-09T09:28:10.963 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/remove.h’: No data available 2024-03-09T09:28:10.965 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/users.c' -> './k/coreutils-8.5/src/users.c' 2024-03-09T09:28:10.965 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/users.c’: No data available 2024-03-09T09:28:10.967 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/dircolors.c' -> './k/coreutils-8.5/src/dircolors.c' 2024-03-09T09:28:10.968 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dircolors.c’: No data available 2024-03-09T09:28:10.970 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/join.c' -> './k/coreutils-8.5/src/join.c' 2024-03-09T09:28:10.971 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/join.c’: No data available 2024-03-09T09:28:10.973 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/chown.c' -> './k/coreutils-8.5/src/chown.c' 2024-03-09T09:28:10.974 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chown.c’: No data available 2024-03-09T09:28:10.976 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/glthread' -> './k/coreutils-8.5/lib/glthread' 2024-03-09T09:28:10.976 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/glthread/threadlib.c' -> './k/coreutils-8.5/lib/glthread/threadlib.c' 2024-03-09T09:28:10.976 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/glthread/threadlib.c’: No data available 2024-03-09T09:28:10.977 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/shuf.c' -> './k/coreutils-8.5/src/shuf.c' 2024-03-09T09:28:10.977 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/shuf.c’: No data available 2024-03-09T09:28:10.979 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/glthread/lock.c' -> './k/coreutils-8.5/lib/glthread/lock.c' 2024-03-09T09:28:10.979 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/glthread/lock.c’: No data available 2024-03-09T09:28:10.980 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/sum.c' -> './k/coreutils-8.5/src/sum.c' 2024-03-09T09:28:10.980 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/sum.c’: No data available 2024-03-09T09:28:10.983 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/csplit.c' -> './k/coreutils-8.5/src/csplit.c' 2024-03-09T09:28:10.983 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/csplit.c’: No data available 2024-03-09T09:28:10.985 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/libstdbuf.c' -> './k/coreutils-8.5/src/libstdbuf.c' 2024-03-09T09:28:10.986 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/libstdbuf.c’: No data available 2024-03-09T09:28:10.988 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/fold.c' -> './k/coreutils-8.5/src/fold.c' 2024-03-09T09:28:10.989 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/fold.c’: No data available 2024-03-09T09:28:10.991 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/basename.c' -> './k/coreutils-8.5/src/basename.c' 2024-03-09T09:28:10.991 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/basename.c’: No data available 2024-03-09T09:28:10.993 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/prog-fprintf.h' -> './k/coreutils-8.5/src/prog-fprintf.h' 2024-03-09T09:28:10.993 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/prog-fprintf.h’: No data available 2024-03-09T09:28:10.995 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/sync.c' -> './k/coreutils-8.5/src/sync.c' 2024-03-09T09:28:10.996 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/sync.c’: No data available 2024-03-09T09:28:10.997 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/lbracket.c' -> './k/coreutils-8.5/src/lbracket.c' 2024-03-09T09:28:10.997 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/lbracket.c’: No data available 2024-03-09T09:28:10.999 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/ln.c' -> './k/coreutils-8.5/src/ln.c' 2024-03-09T09:28:11.000 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ln.c’: No data available 2024-03-09T09:28:11.001 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/tsort.c' -> './k/coreutils-8.5/src/tsort.c' 2024-03-09T09:28:11.002 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tsort.c’: No data available 2024-03-09T09:28:11.003 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/wc.c' -> './k/coreutils-8.5/src/wc.c' 2024-03-09T09:28:11.004 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/wc.c’: No data available 2024-03-09T09:28:11.005 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/extract-magic' -> './k/coreutils-8.5/src/extract-magic' 2024-03-09T09:28:11.006 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/extract-magic’: No data available 2024-03-09T09:28:11.007 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/hostid.c' -> './k/coreutils-8.5/src/hostid.c' 2024-03-09T09:28:11.007 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/hostid.c’: No data available 2024-03-09T09:28:11.010 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/dd.c' -> './k/coreutils-8.5/src/dd.c' 2024-03-09T09:28:11.010 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dd.c’: No data available 2024-03-09T09:28:11.012 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/shred.c' -> './k/coreutils-8.5/src/shred.c' 2024-03-09T09:28:11.012 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/shred.c’: No data available 2024-03-09T09:28:11.014 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/uname.h' -> './k/coreutils-8.5/src/uname.h' 2024-03-09T09:28:11.014 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uname.h’: No data available 2024-03-09T09:28:11.016 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/unlink.c' -> './k/coreutils-8.5/src/unlink.c' 2024-03-09T09:28:11.016 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/unlink.c’: No data available 2024-03-09T09:28:11.018 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/Makefile.am' -> './k/coreutils-8.5/src/Makefile.am' 2024-03-09T09:28:11.018 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/Makefile.am’: No data available 2024-03-09T09:28:11.020 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/seq.c' -> './k/coreutils-8.5/src/seq.c' 2024-03-09T09:28:11.020 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/seq.c’: No data available 2024-03-09T09:28:11.022 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/uname-uname.c' -> './k/coreutils-8.5/src/uname-uname.c' 2024-03-09T09:28:11.022 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uname-uname.c’: No data available 2024-03-09T09:28:11.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/ptx.c' -> './k/coreutils-8.5/src/ptx.c' 2024-03-09T09:28:11.025 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ptx.c’: No data available 2024-03-09T09:28:11.026 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/wheel-size.h' -> './k/coreutils-8.5/src/wheel-size.h' 2024-03-09T09:28:11.027 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/wheel-size.h’: No data available 2024-03-09T09:28:11.029 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/stty.c' -> './k/coreutils-8.5/src/stty.c' 2024-03-09T09:28:11.029 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/stty.c’: No data available 2024-03-09T09:28:11.031 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/chcon.c' -> './k/coreutils-8.5/src/chcon.c' 2024-03-09T09:28:11.034 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chcon.c’: No data available 2024-03-09T09:28:11.034 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/dcgen' -> './k/coreutils-8.5/src/dcgen' 2024-03-09T09:28:11.035 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dcgen’: No data available 2024-03-09T09:28:11.035 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/uptime.c' -> './k/coreutils-8.5/src/uptime.c' 2024-03-09T09:28:11.035 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uptime.c’: No data available 2024-03-09T09:28:11.037 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/link.c' -> './k/coreutils-8.5/src/link.c' 2024-03-09T09:28:11.037 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/link.c’: No data available 2024-03-09T09:28:11.039 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/install.c' -> './k/coreutils-8.5/src/install.c' 2024-03-09T09:28:11.039 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/install.c’: No data available 2024-03-09T09:28:11.041 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/uname.c' -> './k/coreutils-8.5/src/uname.c' 2024-03-09T09:28:11.041 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uname.c’: No data available 2024-03-09T09:28:11.043 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/mknod.c' -> './k/coreutils-8.5/src/mknod.c' 2024-03-09T09:28:11.043 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/mknod.c’: No data available 2024-03-09T09:28:11.045 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/su.c' -> './k/coreutils-8.5/src/su.c' 2024-03-09T09:28:11.045 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/su.c’: No data available 2024-03-09T09:28:11.047 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/expand.c' -> './k/coreutils-8.5/src/expand.c' 2024-03-09T09:28:11.047 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/expand.c’: No data available 2024-03-09T09:28:11.049 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/mv.c' -> './k/coreutils-8.5/src/mv.c' 2024-03-09T09:28:11.049 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/mv.c’: No data available 2024-03-09T09:28:11.053 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/ls.c' -> './k/coreutils-8.5/src/ls.c' 2024-03-09T09:28:11.053 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ls.c’: No data available 2024-03-09T09:28:11.055 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/whoami.c' -> './k/coreutils-8.5/src/whoami.c' 2024-03-09T09:28:11.055 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/whoami.c’: No data available 2024-03-09T09:28:11.058 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/ls-ls.c' -> './k/coreutils-8.5/src/ls-ls.c' 2024-03-09T09:28:11.058 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ls-ls.c’: No data available 2024-03-09T09:28:11.060 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/ls.h' -> './k/coreutils-8.5/src/ls.h' 2024-03-09T09:28:11.060 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ls.h’: No data available 2024-03-09T09:28:11.062 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/id.c' -> './k/coreutils-8.5/src/id.c' 2024-03-09T09:28:11.063 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/id.c’: No data available 2024-03-09T09:28:11.067 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/Makefile.in' -> './k/coreutils-8.5/src/Makefile.in' 2024-03-09T09:28:11.067 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/Makefile.in’: No data available 2024-03-09T09:28:11.069 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/du.c' -> './k/coreutils-8.5/src/du.c' 2024-03-09T09:28:11.070 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/du.c’: No data available 2024-03-09T09:28:11.072 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/cut.c' -> './k/coreutils-8.5/src/cut.c' 2024-03-09T09:28:11.073 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cut.c’: No data available 2024-03-09T09:28:11.075 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/comm.c' -> './k/coreutils-8.5/src/comm.c' 2024-03-09T09:28:11.075 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/comm.c’: No data available 2024-03-09T09:28:11.078 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/od.c' -> './k/coreutils-8.5/src/od.c' 2024-03-09T09:28:11.078 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/od.c’: No data available 2024-03-09T09:28:11.081 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/expr.c' -> './k/coreutils-8.5/src/expr.c' 2024-03-09T09:28:11.081 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/expr.c’: No data available 2024-03-09T09:28:11.083 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/nohup.c' -> './k/coreutils-8.5/src/nohup.c' 2024-03-09T09:28:11.083 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/nohup.c’: No data available 2024-03-09T09:28:11.086 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/system.h' -> './k/coreutils-8.5/src/system.h' 2024-03-09T09:28:11.086 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/system.h’: No data available 2024-03-09T09:28:11.089 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/cat.c' -> './k/coreutils-8.5/src/cat.c' 2024-03-09T09:28:11.089 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cat.c’: No data available 2024-03-09T09:28:11.092 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/mktemp.c' -> './k/coreutils-8.5/src/mktemp.c' 2024-03-09T09:28:11.092 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/mktemp.c’: No data available 2024-03-09T09:28:11.095 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/tac.c' -> './k/coreutils-8.5/src/tac.c' 2024-03-09T09:28:11.095 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tac.c’: No data available 2024-03-09T09:28:11.097 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/ls-dir.c' -> './k/coreutils-8.5/src/ls-dir.c' 2024-03-09T09:28:11.098 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ls-dir.c’: No data available 2024-03-09T09:28:11.100 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/tac-pipe.c' -> './k/coreutils-8.5/src/tac-pipe.c' 2024-03-09T09:28:11.100 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tac-pipe.c’: No data available 2024-03-09T09:28:11.103 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/touch.c' -> './k/coreutils-8.5/src/touch.c' 2024-03-09T09:28:11.103 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/touch.c’: No data available 2024-03-09T09:28:11.105 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/false.c' -> './k/coreutils-8.5/src/false.c' 2024-03-09T09:28:11.106 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/false.c’: No data available 2024-03-09T09:28:11.108 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/echo.c' -> './k/coreutils-8.5/src/echo.c' 2024-03-09T09:28:11.108 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/echo.c’: No data available 2024-03-09T09:28:11.110 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/printenv.c' -> './k/coreutils-8.5/src/printenv.c' 2024-03-09T09:28:11.110 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/printenv.c’: No data available 2024-03-09T09:28:11.113 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/logname.c' -> './k/coreutils-8.5/src/logname.c' 2024-03-09T09:28:11.113 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/logname.c’: No data available 2024-03-09T09:28:11.115 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/group-list.c' -> './k/coreutils-8.5/src/group-list.c' 2024-03-09T09:28:11.115 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/group-list.c’: No data available 2024-03-09T09:28:11.119 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/pr.c' -> './k/coreutils-8.5/src/pr.c' 2024-03-09T09:28:11.119 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/pr.c’: No data available 2024-03-09T09:28:11.122 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/chgrp.c' -> './k/coreutils-8.5/src/chgrp.c' 2024-03-09T09:28:11.122 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chgrp.c’: No data available 2024-03-09T09:28:11.124 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/cp-hash.h' -> './k/coreutils-8.5/src/cp-hash.h' 2024-03-09T09:28:11.124 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cp-hash.h’: No data available 2024-03-09T09:28:11.127 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/getlimits.c' -> './k/coreutils-8.5/src/getlimits.c' 2024-03-09T09:28:11.127 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/getlimits.c’: No data available 2024-03-09T09:28:11.129 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/rmdir.c' -> './k/coreutils-8.5/src/rmdir.c' 2024-03-09T09:28:11.130 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/rmdir.c’: No data available 2024-03-09T09:28:11.132 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/tail.c' -> './k/coreutils-8.5/src/tail.c' 2024-03-09T09:28:11.133 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tail.c’: No data available 2024-03-09T09:28:11.135 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/timeout.c' -> './k/coreutils-8.5/src/timeout.c' 2024-03-09T09:28:11.135 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/timeout.c’: No data available 2024-03-09T09:28:11.139 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/sort.c' -> './k/coreutils-8.5/src/sort.c' 2024-03-09T09:28:11.139 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/sort.c’: No data available 2024-03-09T09:28:11.142 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/operand2sig.c' -> './k/coreutils-8.5/src/operand2sig.c' 2024-03-09T09:28:11.142 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/operand2sig.c’: No data available 2024-03-09T09:28:11.145 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/factor.c' -> './k/coreutils-8.5/src/factor.c' 2024-03-09T09:28:11.145 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/factor.c’: No data available 2024-03-09T09:28:11.148 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/paste.c' -> './k/coreutils-8.5/src/paste.c' 2024-03-09T09:28:11.148 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/paste.c’: No data available 2024-03-09T09:28:11.151 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/chmod.c' -> './k/coreutils-8.5/src/chmod.c' 2024-03-09T09:28:11.151 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chmod.c’: No data available 2024-03-09T09:28:11.154 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/pinky.c' -> './k/coreutils-8.5/src/pinky.c' 2024-03-09T09:28:11.154 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/pinky.c’: No data available 2024-03-09T09:28:11.157 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/unexpand.c' -> './k/coreutils-8.5/src/unexpand.c' 2024-03-09T09:28:11.157 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/unexpand.c’: No data available 2024-03-09T09:28:11.159 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/cksum.c' -> './k/coreutils-8.5/src/cksum.c' 2024-03-09T09:28:11.159 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cksum.c’: No data available 2024-03-09T09:28:11.162 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/date.c' -> './k/coreutils-8.5/src/date.c' 2024-03-09T09:28:11.162 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/date.c’: No data available 2024-03-09T09:28:11.164 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/mkdir.c' -> './k/coreutils-8.5/src/mkdir.c' 2024-03-09T09:28:11.165 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/mkdir.c’: No data available 2024-03-09T09:28:11.167 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/printf.c' -> './k/coreutils-8.5/src/printf.c' 2024-03-09T09:28:11.167 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/printf.c’: No data available 2024-03-09T09:28:11.169 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/true.c' -> './k/coreutils-8.5/src/true.c' 2024-03-09T09:28:11.170 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/true.c’: No data available 2024-03-09T09:28:11.172 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/c99-to-c89.diff' -> './k/coreutils-8.5/src/c99-to-c89.diff' 2024-03-09T09:28:11.172 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/c99-to-c89.diff’: No data available 2024-03-09T09:28:11.175 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/groups.c' -> './k/coreutils-8.5/src/groups.c' 2024-03-09T09:28:11.175 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/groups.c’: No data available 2024-03-09T09:28:11.177 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/dirname.c' -> './k/coreutils-8.5/src/dirname.c' 2024-03-09T09:28:11.177 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dirname.c’: No data available 2024-03-09T09:28:11.180 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/chown-core.h' -> './k/coreutils-8.5/src/chown-core.h' 2024-03-09T09:28:11.180 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chown-core.h’: No data available 2024-03-09T09:28:11.182 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/fs.h' -> './k/coreutils-8.5/src/fs.h' 2024-03-09T09:28:11.182 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/fs.h’: No data available 2024-03-09T09:28:11.185 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/nl.c' -> './k/coreutils-8.5/src/nl.c' 2024-03-09T09:28:11.185 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/nl.c’: No data available 2024-03-09T09:28:11.188 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/tee.c' -> './k/coreutils-8.5/src/tee.c' 2024-03-09T09:28:11.188 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tee.c’: No data available 2024-03-09T09:28:11.190 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/dircolors.h' -> './k/coreutils-8.5/src/dircolors.h' 2024-03-09T09:28:11.191 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dircolors.h’: No data available 2024-03-09T09:28:11.193 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/head.c' -> './k/coreutils-8.5/src/head.c' 2024-03-09T09:28:11.194 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/head.c’: No data available 2024-03-09T09:28:11.196 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/uniq.c' -> './k/coreutils-8.5/src/uniq.c' 2024-03-09T09:28:11.196 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uniq.c’: No data available 2024-03-09T09:28:11.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/chown-core.c' -> './k/coreutils-8.5/src/chown-core.c' 2024-03-09T09:28:11.199 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chown-core.c’: No data available 2024-03-09T09:28:11.201 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/runcon.c' -> './k/coreutils-8.5/src/runcon.c' 2024-03-09T09:28:11.201 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/runcon.c’: No data available 2024-03-09T09:28:11.204 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/sleep.c' -> './k/coreutils-8.5/src/sleep.c' 2024-03-09T09:28:11.204 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/sleep.c’: No data available 2024-03-09T09:28:11.206 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/md5sum.c' -> './k/coreutils-8.5/src/md5sum.c' 2024-03-09T09:28:11.206 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/md5sum.c’: No data available 2024-03-09T09:28:11.208 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/yes.c' -> './k/coreutils-8.5/src/yes.c' 2024-03-09T09:28:11.209 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/yes.c’: No data available 2024-03-09T09:28:11.211 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/readlink.c' -> './k/coreutils-8.5/src/readlink.c' 2024-03-09T09:28:11.211 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/readlink.c’: No data available 2024-03-09T09:28:11.213 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/env.c' -> './k/coreutils-8.5/src/env.c' 2024-03-09T09:28:11.213 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/env.c’: No data available 2024-03-09T09:28:11.216 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/df.c' -> './k/coreutils-8.5/src/df.c' 2024-03-09T09:28:11.216 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/df.c’: No data available 2024-03-09T09:28:11.218 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/nice.c' -> './k/coreutils-8.5/src/nice.c' 2024-03-09T09:28:11.218 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/nice.c’: No data available 2024-03-09T09:28:11.221 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/fmt.c' -> './k/coreutils-8.5/src/fmt.c' 2024-03-09T09:28:11.221 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/fmt.c’: No data available 2024-03-09T09:28:11.223 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/base64.c' -> './k/coreutils-8.5/src/base64.c' 2024-03-09T09:28:11.223 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/base64.c’: No data available 2024-03-09T09:28:11.225 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/tty.c' -> './k/coreutils-8.5/src/tty.c' 2024-03-09T09:28:11.225 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tty.c’: No data available 2024-03-09T09:28:11.228 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/truncate.c' -> './k/coreutils-8.5/src/truncate.c' 2024-03-09T09:28:11.228 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/truncate.c’: No data available 2024-03-09T09:28:11.230 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/mkfifo.c' -> './k/coreutils-8.5/src/mkfifo.c' 2024-03-09T09:28:11.230 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/mkfifo.c’: No data available 2024-03-09T09:28:11.232 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/operand2sig.h' -> './k/coreutils-8.5/src/operand2sig.h' 2024-03-09T09:28:11.232 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/operand2sig.h’: No data available 2024-03-09T09:28:11.240 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/copy.h' -> './k/coreutils-8.5/src/copy.h' 2024-03-09T09:28:11.240 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/copy.h’: No data available 2024-03-09T09:28:11.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/kill.c' -> './k/coreutils-8.5/src/kill.c' 2024-03-09T09:28:11.243 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/kill.c’: No data available 2024-03-09T09:28:11.245 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/who.c' -> './k/coreutils-8.5/src/who.c' 2024-03-09T09:28:11.245 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/who.c’: No data available 2024-03-09T09:28:11.247 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/stat.c' -> './k/coreutils-8.5/src/stat.c' 2024-03-09T09:28:11.248 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/stat.c’: No data available 2024-03-09T09:28:11.249 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/remove.c' -> './k/coreutils-8.5/src/remove.c' 2024-03-09T09:28:11.250 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/remove.c’: No data available 2024-03-09T09:28:11.252 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/pwd.c' -> './k/coreutils-8.5/src/pwd.c' 2024-03-09T09:28:11.252 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/pwd.c’: No data available 2024-03-09T09:28:11.254 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/prog-fprintf.c' -> './k/coreutils-8.5/src/prog-fprintf.c' 2024-03-09T09:28:11.254 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/prog-fprintf.c’: No data available 2024-03-09T09:28:11.255 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/ls-vdir.c' -> './k/coreutils-8.5/src/ls-vdir.c' 2024-03-09T09:28:11.255 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ls-vdir.c’: No data available 2024-03-09T09:28:11.258 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/copy.c' -> './k/coreutils-8.5/src/copy.c' 2024-03-09T09:28:11.259 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/copy.c’: No data available 2024-03-09T09:28:11.260 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/hostname.c' -> './k/coreutils-8.5/src/hostname.c' 2024-03-09T09:28:11.261 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/hostname.c’: No data available 2024-03-09T09:28:11.263 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/wheel-gen.pl' -> './k/coreutils-8.5/src/wheel-gen.pl' 2024-03-09T09:28:11.263 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/wheel-gen.pl’: No data available 2024-03-09T09:28:11.265 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/group-list.h' -> './k/coreutils-8.5/src/group-list.h' 2024-03-09T09:28:11.265 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/group-list.h’: No data available 2024-03-09T09:28:11.267 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/cp.c' -> './k/coreutils-8.5/src/cp.c' 2024-03-09T09:28:11.268 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cp.c’: No data available 2024-03-09T09:28:11.270 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/setuidgid.c' -> './k/coreutils-8.5/src/setuidgid.c' 2024-03-09T09:28:11.270 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/setuidgid.c’: No data available 2024-03-09T09:28:11.273 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/tr.c' -> './k/coreutils-8.5/src/tr.c' 2024-03-09T09:28:11.273 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tr.c’: No data available 2024-03-09T09:28:11.275 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/stdbuf.c' -> './k/coreutils-8.5/src/stdbuf.c' 2024-03-09T09:28:11.275 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/stdbuf.c’: No data available 2024-03-09T09:28:11.277 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/pathchk.c' -> './k/coreutils-8.5/src/pathchk.c' 2024-03-09T09:28:11.277 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/pathchk.c’: No data available 2024-03-09T09:28:11.279 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/wheel.h' -> './k/coreutils-8.5/src/wheel.h' 2024-03-09T09:28:11.279 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/wheel.h’: No data available 2024-03-09T09:28:11.281 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/src/split.c' -> './k/coreutils-8.5/src/split.c' 2024-03-09T09:28:11.281 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/split.c’: No data available 2024-03-09T09:28:11.747 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/glthread/tls.h' -> './k/coreutils-8.5/lib/glthread/tls.h' 2024-03-09T09:28:11.747 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/glthread/tls.h’: No data available 2024-03-09T09:28:11.749 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/glthread/tls.c' -> './k/coreutils-8.5/lib/glthread/tls.c' 2024-03-09T09:28:11.749 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/glthread/tls.c’: No data available 2024-03-09T09:28:11.751 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/glthread/lock.h' -> './k/coreutils-8.5/lib/glthread/lock.h' 2024-03-09T09:28:11.751 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/glthread/lock.h’: No data available 2024-03-09T09:28:11.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unistr' -> './k/coreutils-8.5/lib/unistr' 2024-03-09T09:28:11.812 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unistr/u8-uctomb-aux.c' -> './k/coreutils-8.5/lib/unistr/u8-uctomb-aux.c' 2024-03-09T09:28:11.813 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistr/u8-uctomb-aux.c’: No data available 2024-03-09T09:28:11.814 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/unistr/u8-uctomb.c' -> './k/coreutils-8.5/lib/unistr/u8-uctomb.c' 2024-03-09T09:28:11.815 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/unistr/u8-uctomb.c’: No data available 2024-03-09T09:28:11.857 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/uniwidth' -> './k/coreutils-8.5/lib/uniwidth' 2024-03-09T09:28:11.857 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/uniwidth/width.c' -> './k/coreutils-8.5/lib/uniwidth/width.c' 2024-03-09T09:28:11.857 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/uniwidth/width.c’: No data available 2024-03-09T09:28:11.859 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/uniwidth/cjk.h' -> './k/coreutils-8.5/lib/uniwidth/cjk.h' 2024-03-09T09:28:11.859 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/uniwidth/cjk.h’: No data available 2024-03-09T09:28:11.886 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/strtoull.c' -> './k/coreutils-8.5/lib/strtoull.c' 2024-03-09T09:28:11.887 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/strtoull.c’: No data available 2024-03-09T09:28:11.888 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xmemcoll.h' -> './k/coreutils-8.5/lib/xmemcoll.h' 2024-03-09T09:28:11.889 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xmemcoll.h’: No data available 2024-03-09T09:28:11.891 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/hash.h' -> './k/coreutils-8.5/lib/hash.h' 2024-03-09T09:28:11.891 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/hash.h’: No data available 2024-03-09T09:28:12.174 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_require_config_h' -> './k/coreutils-8.5/.x-sc_require_config_h' 2024-03-09T09:28:12.174 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_require_config_h’: No data available 2024-03-09T09:28:12.176 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_system_h_headers' -> './k/coreutils-8.5/.x-sc_system_h_headers' 2024-03-09T09:28:12.176 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_system_h_headers’: No data available 2024-03-09T09:28:12.178 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_fail_0' -> './k/coreutils-8.5/.x-sc_prohibit_fail_0' 2024-03-09T09:28:12.178 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_fail_0’: No data available 2024-03-09T09:28:12.181 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_magic_number_exit' -> './k/coreutils-8.5/.x-sc_prohibit_magic_number_exit' 2024-03-09T09:28:12.181 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_magic_number_exit’: No data available 2024-03-09T09:28:12.183 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/THANKS-to-translators' -> './k/coreutils-8.5/THANKS-to-translators' 2024-03-09T09:28:12.183 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/THANKS-to-translators’: No data available 2024-03-09T09:28:12.186 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/.x-sc_useless_cpp_parens' -> './k/coreutils-8.5/.x-sc_useless_cpp_parens' 2024-03-09T09:28:12.186 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_useless_cpp_parens’: No data available 2024-03-09T09:28:12.227 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests' -> './k/coreutils-8.5/gnulib-tests' 2024-03-09T09:28:12.227 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-ftello.sh' -> './k/coreutils-8.5/gnulib-tests/test-ftello.sh' 2024-03-09T09:28:12.228 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-ftello.sh’: No data available 2024-03-09T09:28:12.229 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-snprintf.c' -> './k/coreutils-8.5/gnulib-tests/test-snprintf.c' 2024-03-09T09:28:12.230 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-snprintf.c’: No data available 2024-03-09T09:28:12.232 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-openat.c' -> './k/coreutils-8.5/gnulib-tests/test-openat.c' 2024-03-09T09:28:12.232 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-openat.c’: No data available 2024-03-09T09:28:12.234 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-dirent-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-dirent-safer.c' 2024-03-09T09:28:12.234 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-dirent-safer.c’: No data available 2024-03-09T09:28:12.237 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-cloexec.c' -> './k/coreutils-8.5/gnulib-tests/test-cloexec.c' 2024-03-09T09:28:12.237 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-cloexec.c’: No data available 2024-03-09T09:28:12.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fstatat.c' -> './k/coreutils-8.5/gnulib-tests/test-fstatat.c' 2024-03-09T09:28:12.239 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fstatat.c’: No data available 2024-03-09T09:28:12.242 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fnmatch.c' -> './k/coreutils-8.5/gnulib-tests/test-fnmatch.c' 2024-03-09T09:28:12.242 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fnmatch.c’: No data available 2024-03-09T09:28:12.244 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getndelim2.c' -> './k/coreutils-8.5/gnulib-tests/test-getndelim2.c' 2024-03-09T09:28:12.245 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getndelim2.c’: No data available 2024-03-09T09:28:12.247 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-wcrtomb.sh' -> './k/coreutils-8.5/gnulib-tests/test-wcrtomb.sh' 2024-03-09T09:28:12.247 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-wcrtomb.sh’: No data available 2024-03-09T09:28:12.249 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-filenamecat.c' -> './k/coreutils-8.5/gnulib-tests/test-filenamecat.c' 2024-03-09T09:28:12.249 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-filenamecat.c’: No data available 2024-03-09T09:28:12.251 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fchownat.c' -> './k/coreutils-8.5/gnulib-tests/test-fchownat.c' 2024-03-09T09:28:12.252 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fchownat.c’: No data available 2024-03-09T09:28:12.253 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stdlib.c' -> './k/coreutils-8.5/gnulib-tests/test-stdlib.c' 2024-03-09T09:28:12.254 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stdlib.c’: No data available 2024-03-09T09:28:12.256 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-read-file.c' -> './k/coreutils-8.5/gnulib-tests/test-read-file.c' 2024-03-09T09:28:12.256 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-read-file.c’: No data available 2024-03-09T09:28:12.258 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude7.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude7.sh' 2024-03-09T09:28:12.258 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude7.sh’: No data available 2024-03-09T09:28:12.261 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-closein.c' -> './k/coreutils-8.5/gnulib-tests/test-closein.c' 2024-03-09T09:28:12.261 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-closein.c’: No data available 2024-03-09T09:28:12.263 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/connect.c' -> './k/coreutils-8.5/gnulib-tests/connect.c' 2024-03-09T09:28:12.263 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/connect.c’: No data available 2024-03-09T09:28:12.266 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-rename.h' -> './k/coreutils-8.5/gnulib-tests/test-rename.h' 2024-03-09T09:28:12.266 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-rename.h’: No data available 2024-03-09T09:28:12.268 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-memchr2.c' -> './k/coreutils-8.5/gnulib-tests/test-memchr2.c' 2024-03-09T09:28:12.268 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-memchr2.c’: No data available 2024-03-09T09:28:12.270 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_time.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_time.c' 2024-03-09T09:28:12.271 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_time.c’: No data available 2024-03-09T09:28:12.273 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/read-file.h' -> './k/coreutils-8.5/gnulib-tests/read-file.h' 2024-03-09T09:28:12.273 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/read-file.h’: No data available 2024-03-09T09:28:12.275 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mkdirat.c' -> './k/coreutils-8.5/gnulib-tests/test-mkdirat.c' 2024-03-09T09:28:12.275 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mkdirat.c’: No data available 2024-03-09T09:28:12.278 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnand-nolibm.c' -> './k/coreutils-8.5/gnulib-tests/test-isnand-nolibm.c' 2024-03-09T09:28:12.278 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnand-nolibm.c’: No data available 2024-03-09T09:28:12.280 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fopen-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-fopen-safer.c' 2024-03-09T09:28:12.280 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fopen-safer.c’: No data available 2024-03-09T09:28:12.282 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stdio.c' -> './k/coreutils-8.5/gnulib-tests/test-stdio.c' 2024-03-09T09:28:12.283 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stdio.c’: No data available 2024-03-09T09:28:12.285 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fseeko.c' -> './k/coreutils-8.5/gnulib-tests/test-fseeko.c' 2024-03-09T09:28:12.285 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fseeko.c’: No data available 2024-03-09T09:28:12.287 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-arpa_inet.c' -> './k/coreutils-8.5/gnulib-tests/test-arpa_inet.c' 2024-03-09T09:28:12.288 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-arpa_inet.c’: No data available 2024-03-09T09:28:12.290 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadahead.sh' -> './k/coreutils-8.5/gnulib-tests/test-freadahead.sh' 2024-03-09T09:28:12.290 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadahead.sh’: No data available 2024-03-09T09:28:12.292 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-openat-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-openat-safer.c' 2024-03-09T09:28:12.293 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-openat-safer.c’: No data available 2024-03-09T09:28:12.295 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-unlink.c' -> './k/coreutils-8.5/gnulib-tests/test-unlink.c' 2024-03-09T09:28:12.295 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-unlink.c’: No data available 2024-03-09T09:28:12.297 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsstr2.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsstr2.sh' 2024-03-09T09:28:12.298 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsstr2.sh’: No data available 2024-03-09T09:28:12.299 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lchown.c' -> './k/coreutils-8.5/gnulib-tests/test-lchown.c' 2024-03-09T09:28:12.300 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lchown.c’: No data available 2024-03-09T09:28:12.302 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-nl_langinfo.c' -> './k/coreutils-8.5/gnulib-tests/test-nl_langinfo.c' 2024-03-09T09:28:12.302 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-nl_langinfo.c’: No data available 2024-03-09T09:28:12.304 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-open.h' -> './k/coreutils-8.5/gnulib-tests/test-open.h' 2024-03-09T09:28:12.304 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-open.h’: No data available 2024-03-09T09:28:12.306 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-memcmp.c' -> './k/coreutils-8.5/gnulib-tests/test-memcmp.c' 2024-03-09T09:28:12.307 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-memcmp.c’: No data available 2024-03-09T09:28:12.309 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.c' -> './k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.c' 2024-03-09T09:28:12.309 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.c’: No data available 2024-03-09T09:28:12.311 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadahead.c' -> './k/coreutils-8.5/gnulib-tests/test-freadahead.c' 2024-03-09T09:28:12.311 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadahead.c’: No data available 2024-03-09T09:28:12.314 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/ioctl.c' -> './k/coreutils-8.5/gnulib-tests/ioctl.c' 2024-03-09T09:28:12.314 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/ioctl.c’: No data available 2024-03-09T09:28:12.317 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-iconv-h.c' -> './k/coreutils-8.5/gnulib-tests/test-iconv-h.c' 2024-03-09T09:28:12.317 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-iconv-h.c’: No data available 2024-03-09T09:28:12.319 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mkdir.c' -> './k/coreutils-8.5/gnulib-tests/test-mkdir.c' 2024-03-09T09:28:12.319 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mkdir.c’: No data available 2024-03-09T09:28:12.322 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-posixtm.c' -> './k/coreutils-8.5/gnulib-tests/test-posixtm.c' 2024-03-09T09:28:12.322 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-posixtm.c’: No data available 2024-03-09T09:28:12.324 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/wctob.c' -> './k/coreutils-8.5/gnulib-tests/wctob.c' 2024-03-09T09:28:12.325 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/wctob.c’: No data available 2024-03-09T09:28:12.327 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude5.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude5.sh' 2024-03-09T09:28:12.327 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude5.sh’: No data available 2024-03-09T09:28:12.330 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-rmdir.h' -> './k/coreutils-8.5/gnulib-tests/test-rmdir.h' 2024-03-09T09:28:12.330 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-rmdir.h’: No data available 2024-03-09T09:28:12.332 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-symlink.h' -> './k/coreutils-8.5/gnulib-tests/test-symlink.h' 2024-03-09T09:28:12.332 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-symlink.h’: No data available 2024-03-09T09:28:12.335 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-unsetenv.c' -> './k/coreutils-8.5/gnulib-tests/test-unsetenv.c' 2024-03-09T09:28:12.335 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-unsetenv.c’: No data available 2024-03-09T09:28:12.343 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude4.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude4.sh' 2024-03-09T09:28:12.343 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude4.sh’: No data available 2024-03-09T09:28:12.345 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/listen.c' -> './k/coreutils-8.5/gnulib-tests/listen.c' 2024-03-09T09:28:12.346 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/listen.c’: No data available 2024-03-09T09:28:12.587 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:28:12.587 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:28:12.595 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/c-strtold.c' -> './k/coreutils-8.5/lib/c-strtold.c' 2024-03-09T09:28:12.595 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-strtold.c’: No data available 2024-03-09T09:28:12.598 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/c-ctype.h' -> './k/coreutils-8.5/lib/c-ctype.h' 2024-03-09T09:28:12.598 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/c-ctype.h’: No data available 2024-03-09T09:28:12.600 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/xmemxfrm.c' -> './k/coreutils-8.5/lib/xmemxfrm.c' 2024-03-09T09:28:12.601 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/xmemxfrm.c’: No data available 2024-03-09T09:28:12.603 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/lib/fileblocks.c' -> './k/coreutils-8.5/lib/fileblocks.c' 2024-03-09T09:28:12.603 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/lib/fileblocks.c’: No data available 2024-03-09T09:28:12.627 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_require_config_h_first' -> './k/coreutils-8.5/.x-sc_require_config_h_first' 2024-03-09T09:28:12.627 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_require_config_h_first’: No data available 2024-03-09T09:28:12.643 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po' -> './k/coreutils-8.5/po' 2024-03-09T09:28:12.643 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/sl.po' -> './k/coreutils-8.5/po/sl.po' 2024-03-09T09:28:12.644 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sl.po’: No data available 2024-03-09T09:28:12.649 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/zh_TW.po' -> './k/coreutils-8.5/po/zh_TW.po' 2024-03-09T09:28:12.649 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/zh_TW.po’: No data available 2024-03-09T09:28:12.657 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/sk.po' -> './k/coreutils-8.5/po/sk.po' 2024-03-09T09:28:12.657 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sk.po’: No data available 2024-03-09T09:28:12.660 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/el.gmo' -> './k/coreutils-8.5/po/el.gmo' 2024-03-09T09:28:12.660 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/el.gmo’: No data available 2024-03-09T09:28:12.662 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/en.gmo' -> './k/coreutils-8.5/po/en.gmo' 2024-03-09T09:28:12.662 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/en.gmo’: No data available 2024-03-09T09:28:12.671 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/lg.po' -> './k/coreutils-8.5/po/lg.po' 2024-03-09T09:28:12.672 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/lg.po’: No data available 2024-03-09T09:28:12.677 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/sv.gmo' -> './k/coreutils-8.5/po/sv.gmo' 2024-03-09T09:28:12.677 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sv.gmo’: No data available 2024-03-09T09:28:12.682 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/pl.gmo' -> './k/coreutils-8.5/po/pl.gmo' 2024-03-09T09:28:12.683 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pl.gmo’: No data available 2024-03-09T09:28:12.684 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/insert-header.sin' -> './k/coreutils-8.5/po/insert-header.sin' 2024-03-09T09:28:12.684 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/insert-header.sin’: No data available 2024-03-09T09:28:12.688 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/sk.gmo' -> './k/coreutils-8.5/po/sk.gmo' 2024-03-09T09:28:12.688 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sk.gmo’: No data available 2024-03-09T09:28:12.692 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ja.gmo' -> './k/coreutils-8.5/po/ja.gmo' 2024-03-09T09:28:12.692 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ja.gmo’: No data available 2024-03-09T09:28:12.699 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ko.po' -> './k/coreutils-8.5/po/ko.po' 2024-03-09T09:28:12.699 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ko.po’: No data available 2024-03-09T09:28:12.704 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/da.gmo' -> './k/coreutils-8.5/po/da.gmo' 2024-03-09T09:28:12.704 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/da.gmo’: No data available 2024-03-09T09:28:12.706 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/LINGUAS' -> './k/coreutils-8.5/po/LINGUAS' 2024-03-09T09:28:12.706 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/LINGUAS’: No data available 2024-03-09T09:28:12.709 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/zh_TW.gmo' -> './k/coreutils-8.5/po/zh_TW.gmo' 2024-03-09T09:28:12.709 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/zh_TW.gmo’: No data available 2024-03-09T09:28:12.713 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/tr.gmo' -> './k/coreutils-8.5/po/tr.gmo' 2024-03-09T09:28:12.713 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/tr.gmo’: No data available 2024-03-09T09:28:12.716 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/gl.gmo' -> './k/coreutils-8.5/po/gl.gmo' 2024-03-09T09:28:12.716 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/gl.gmo’: No data available 2024-03-09T09:28:12.722 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/fi.po' -> './k/coreutils-8.5/po/fi.po' 2024-03-09T09:28:12.723 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/fi.po’: No data available 2024-03-09T09:28:12.731 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/da.po' -> './k/coreutils-8.5/po/da.po' 2024-03-09T09:28:12.732 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/da.po’: No data available 2024-03-09T09:28:12.736 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/fi.gmo' -> './k/coreutils-8.5/po/fi.gmo' 2024-03-09T09:28:12.736 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/fi.gmo’: No data available 2024-03-09T09:28:12.739 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/quot.sed' -> './k/coreutils-8.5/po/quot.sed' 2024-03-09T09:28:12.739 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/quot.sed’: No data available 2024-03-09T09:28:12.741 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/stamp-po' -> './k/coreutils-8.5/po/stamp-po' 2024-03-09T09:28:12.741 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/stamp-po’: No data available 2024-03-09T09:28:12.743 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ms.gmo' -> './k/coreutils-8.5/po/ms.gmo' 2024-03-09T09:28:12.743 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ms.gmo’: No data available 2024-03-09T09:28:12.751 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/pt_BR.po' -> './k/coreutils-8.5/po/pt_BR.po' 2024-03-09T09:28:12.751 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pt_BR.po’: No data available 2024-03-09T09:28:12.753 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/Makevars' -> './k/coreutils-8.5/po/Makevars' 2024-03-09T09:28:12.754 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/Makevars’: No data available 2024-03-09T09:28:12.756 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/boldquot.sed' -> './k/coreutils-8.5/po/boldquot.sed' 2024-03-09T09:28:12.756 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/boldquot.sed’: No data available 2024-03-09T09:28:12.762 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/coreutils.pot' -> './k/coreutils-8.5/po/coreutils.pot' 2024-03-09T09:28:12.762 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/coreutils.pot’: No data available 2024-03-09T09:28:12.769 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/de.po' -> './k/coreutils-8.5/po/de.po' 2024-03-09T09:28:12.769 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/de.po’: No data available 2024-03-09T09:28:12.773 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/pt.gmo' -> './k/coreutils-8.5/po/pt.gmo' 2024-03-09T09:28:12.774 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pt.gmo’: No data available 2024-03-09T09:28:12.775 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/Makefile.in.in' -> './k/coreutils-8.5/po/Makefile.in.in' 2024-03-09T09:28:12.776 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/Makefile.in.in’: No data available 2024-03-09T09:28:12.783 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/nl.gmo' -> './k/coreutils-8.5/po/nl.gmo' 2024-03-09T09:28:12.783 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/nl.gmo’: No data available 2024-03-09T09:28:12.789 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/fr.gmo' -> './k/coreutils-8.5/po/fr.gmo' 2024-03-09T09:28:12.789 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/fr.gmo’: No data available 2024-03-09T09:28:12.794 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ga.gmo' -> './k/coreutils-8.5/po/ga.gmo' 2024-03-09T09:28:12.794 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ga.gmo’: No data available 2024-03-09T09:28:12.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.818+0000 7fe67a800640 1 Processor -- start 2024-03-09T09:28:12.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.819+0000 7fe67a800640 1 -- start start 2024-03-09T09:28:12.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.821+0000 7fe67a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6740c5e40 0x7fe674156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:12.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.821+0000 7fe67a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6740cadb0 0x7fe674156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:12.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.821+0000 7fe67a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe674156a90 con 0x7fe674153300 2024-03-09T09:28:12.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.821+0000 7fe67a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe674157000 con 0x7fe6740c5e40 2024-03-09T09:28:12.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.821+0000 7fe67a800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6740ae430 con 0x7fe6740cadb0 2024-03-09T09:28:12.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.821+0000 7fe678a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fe6740c5e40 0x7fe674156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:12.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.821+0000 7fe679400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fe674153300 0x7fe6740b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50704/0 (socket says 172.21.15.67:50704) 2024-03-09T09:28:12.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.821+0000 7fe679400640 1 -- 172.21.15.67:0/992967900 learned_addr learned my addr 172.21.15.67:0/992967900 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:28:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.822+0000 7fe679e00640 1 --2- 172.21.15.67:0/992967900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6740cadb0 0x7fe674156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.822+0000 7fe678a00640 1 -- 172.21.15.67:0/992967900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6740cadb0 msgr2=0x7fe674156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.822+0000 7fe678a00640 1 --2- 172.21.15.67:0/992967900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6740cadb0 0x7fe674156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.822+0000 7fe678a00640 1 -- 172.21.15.67:0/992967900 >> v1:172.21.15.67:6789/0 conn(0x7fe674153300 legacy=0x7fe6740b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.822+0000 7fe678a00640 1 -- 172.21.15.67:0/992967900 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6740ae6f0 con 0x7fe6740c5e40 2024-03-09T09:28:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.822+0000 7fe673e00640 1 -- 172.21.15.67:0/992967900 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82018098 0 0) 0x7fe674156a90 con 0x7fe674153300 2024-03-09T09:28:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.822+0000 7fe679e00640 1 --2- 172.21.15.67:0/992967900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6740cadb0 0x7fe674156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:28:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.823+0000 7fe678a00640 1 --2- 172.21.15.67:0/992967900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6740c5e40 0x7fe674156010 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe6600097a0 tx=0x7fe66002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3eac34fa404b64 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.823+0000 7fe673e00640 1 -- 172.21.15.67:0/992967900 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660035020 con 0x7fe6740c5e40 2024-03-09T09:28:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.823+0000 7fe673e00640 1 -- 172.21.15.67:0/992967900 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe660031810 con 0x7fe6740c5e40 2024-03-09T09:28:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.823+0000 7fe673e00640 1 -- 172.21.15.67:0/992967900 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660031be0 con 0x7fe6740c5e40 2024-03-09T09:28:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.824+0000 7fe67a800640 1 -- 172.21.15.67:0/992967900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6740c5e40 msgr2=0x7fe674156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.824+0000 7fe67a800640 1 --2- 172.21.15.67:0/992967900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6740c5e40 0x7fe674156010 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe6600097a0 tx=0x7fe66002b8b0 comp rx=0 tx=0).stop 2024-03-09T09:28:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.824+0000 7fe67a800640 1 -- 172.21.15.67:0/992967900 shutdown_connections 2024-03-09T09:28:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.824+0000 7fe67a800640 1 --2- 172.21.15.67:0/992967900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6740cadb0 0x7fe674156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.824+0000 7fe67a800640 1 --2- 172.21.15.67:0/992967900 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6740c5e40 0x7fe674156010 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.824+0000 7fe67a800640 1 -- 172.21.15.67:0/992967900 >> 172.21.15.67:0/992967900 conn(0x7fe6740732d0 msgr2=0x7fe6740ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.824+0000 7fe67a800640 1 -- 172.21.15.67:0/992967900 shutdown_connections 2024-03-09T09:28:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.824+0000 7fe67a800640 1 -- 172.21.15.67:0/992967900 wait complete. 2024-03-09T09:28:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.826+0000 7fe67a800640 1 Processor -- start 2024-03-09T09:28:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.826+0000 7fe67a800640 1 -- start start 2024-03-09T09:28:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.827+0000 7fe67a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6740c5e40 0x7fe6741eff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.827+0000 7fe67a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6740cadb0 0x7fe6740c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:12.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.827+0000 7fe67a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe674153300 0x7fe67406b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.827+0000 7fe67a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674159e90 con 0x7fe6740cadb0 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.827+0000 7fe67a800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674159bb0 con 0x7fe6740c5e40 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.827+0000 7fe67a800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6740c59e0 con 0x7fe674153300 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.828+0000 7fe678a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6740cadb0 0x7fe6740c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.828+0000 7fe678a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6740cadb0 0x7fe6740c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37460/0 (socket says 172.21.15.67:37460) 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.828+0000 7fe678a00640 1 -- 172.21.15.67:0/3216103492 learned_addr learned my addr 172.21.15.67:0/3216103492 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.828+0000 7fe679400640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6740c5e40 0x7fe6741eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.828+0000 7fe678a00640 1 -- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe674153300 msgr2=0x7fe67406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.828+0000 7fe678a00640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe674153300 0x7fe67406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.828+0000 7fe678a00640 1 -- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6740c5e40 msgr2=0x7fe6741eff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.828+0000 7fe678a00640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6740c5e40 0x7fe6741eff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:12.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.828+0000 7fe678a00640 1 -- 172.21.15.67:0/3216103492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6740c63e0 con 0x7fe6740cadb0 2024-03-09T09:28:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.829+0000 7fe678a00640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6740cadb0 0x7fe6740c7f00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe6600097a0 tx=0x7fe6600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.829+0000 7fe671600640 1 -- 172.21.15.67:0/3216103492 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660041070 con 0x7fe6740cadb0 2024-03-09T09:28:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.830+0000 7fe671600640 1 -- 172.21.15.67:0/3216103492 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66003ad00 con 0x7fe6740cadb0 2024-03-09T09:28:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.830+0000 7fe67a800640 1 -- 172.21.15.67:0/3216103492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67414c300 con 0x7fe6740cadb0 2024-03-09T09:28:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.830+0000 7fe671600640 1 -- 172.21.15.67:0/3216103492 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660035040 con 0x7fe6740cadb0 2024-03-09T09:28:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.830+0000 7fe67a800640 1 -- 172.21.15.67:0/3216103492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6741ef780 con 0x7fe6740cadb0 2024-03-09T09:28:12.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.831+0000 7fe67a800640 1 -- 172.21.15.67:0/3216103492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67404f3c0 con 0x7fe6740cadb0 2024-03-09T09:28:12.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.832+0000 7fe671600640 1 -- 172.21.15.67:0/3216103492 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe660038020 con 0x7fe6740cadb0 2024-03-09T09:28:12.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.833+0000 7fe671600640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe65406b500 0x7fe65406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:12.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.833+0000 7fe671600640 1 -- 172.21.15.67:0/3216103492 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fe6600ac2b0 con 0x7fe6740cadb0 2024-03-09T09:28:12.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.834+0000 7fe679400640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe65406b500 0x7fe65406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:12.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.835+0000 7fe679400640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe65406b500 0x7fe65406d9c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe668000f60 tx=0x7fe668007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:12.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:12.839+0000 7fe671600640 1 -- 172.21.15.67:0/3216103492 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe66007ae00 con 0x7fe6740cadb0 2024-03-09T09:28:12.987 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-printf-posix.output' -> './k/coreutils-8.5/gnulib-tests/test-printf-posix.output' 2024-03-09T09:28:12.987 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-printf-posix.output’: No data available 2024-03-09T09:28:12.989 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbrtowc3.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbrtowc3.sh' 2024-03-09T09:28:12.989 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbrtowc3.sh’: No data available 2024-03-09T09:28:12.991 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-dirent.c' -> './k/coreutils-8.5/gnulib-tests/test-dirent.c' 2024-03-09T09:28:12.991 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-dirent.c’: No data available 2024-03-09T09:28:12.992 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-utimens-common.h' -> './k/coreutils-8.5/gnulib-tests/test-utimens-common.h' 2024-03-09T09:28:12.993 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-utimens-common.h’: No data available 2024-03-09T09:28:12.995 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsinit.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsinit.c' 2024-03-09T09:28:12.995 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsinit.c’: No data available 2024-03-09T09:28:12.996 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/w32sock.h' -> './k/coreutils-8.5/gnulib-tests/w32sock.h' 2024-03-09T09:28:12.996 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/w32sock.h’: No data available 2024-03-09T09:28:12.998 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-printf-posix.h' -> './k/coreutils-8.5/gnulib-tests/test-printf-posix.h' 2024-03-09T09:28:12.999 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-printf-posix.h’: No data available 2024-03-09T09:28:13.000 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fchdir.c' -> './k/coreutils-8.5/gnulib-tests/test-fchdir.c' 2024-03-09T09:28:13.000 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fchdir.c’: No data available 2024-03-09T09:28:13.002 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/read-file.c' -> './k/coreutils-8.5/gnulib-tests/read-file.c' 2024-03-09T09:28:13.002 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/read-file.c’: No data available 2024-03-09T09:28:13.004 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-ctype.c' -> './k/coreutils-8.5/gnulib-tests/test-ctype.c' 2024-03-09T09:28:13.004 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-ctype.c’: No data available 2024-03-09T09:28:13.007 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/binary-io.h' -> './k/coreutils-8.5/gnulib-tests/binary-io.h' 2024-03-09T09:28:13.007 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/binary-io.h’: No data available 2024-03-09T09:28:13.009 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/nap.h' -> './k/coreutils-8.5/gnulib-tests/nap.h' 2024-03-09T09:28:13.010 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/nap.h’: No data available 2024-03-09T09:28:13.011 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/sys_ioctl.in.h' -> './k/coreutils-8.5/gnulib-tests/sys_ioctl.in.h' 2024-03-09T09:28:13.011 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/sys_ioctl.in.h’: No data available 2024-03-09T09:28:13.012 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-ftello2.sh' -> './k/coreutils-8.5/gnulib-tests/test-ftello2.sh' 2024-03-09T09:28:13.012 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-ftello2.sh’: No data available 2024-03-09T09:28:13.014 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-memchr.c' -> './k/coreutils-8.5/gnulib-tests/test-memchr.c' 2024-03-09T09:28:13.014 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-memchr.c’: No data available 2024-03-09T09:28:13.016 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsrtowcs3.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsrtowcs3.sh' 2024-03-09T09:28:13.016 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsrtowcs3.sh’: No data available 2024-03-09T09:28:13.017 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fseeko.sh' -> './k/coreutils-8.5/gnulib-tests/test-fseeko.sh' 2024-03-09T09:28:13.017 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fseeko.sh’: No data available 2024-03-09T09:28:13.019 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-select-out.sh' -> './k/coreutils-8.5/gnulib-tests/test-select-out.sh' 2024-03-09T09:28:13.019 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-select-out.sh’: No data available 2024-03-09T09:28:13.021 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/inet_pton.c' -> './k/coreutils-8.5/gnulib-tests/inet_pton.c' 2024-03-09T09:28:13.021 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/inet_pton.c’: No data available 2024-03-09T09:28:13.023 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stdbool.c' -> './k/coreutils-8.5/gnulib-tests/test-stdbool.c' 2024-03-09T09:28:13.023 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stdbool.c’: No data available 2024-03-09T09:28:13.025 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-nanosleep.c' -> './k/coreutils-8.5/gnulib-tests/test-nanosleep.c' 2024-03-09T09:28:13.025 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-nanosleep.c’: No data available 2024-03-09T09:28:13.026 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/bind.c' -> './k/coreutils-8.5/gnulib-tests/bind.c' 2024-03-09T09:28:13.027 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/bind.c’: No data available 2024-03-09T09:28:13.028 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/unlinkdir.h' -> './k/coreutils-8.5/gnulib-tests/unlinkdir.h' 2024-03-09T09:28:13.029 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/unlinkdir.h’: No data available 2024-03-09T09:28:13.030 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getline.c' -> './k/coreutils-8.5/gnulib-tests/test-getline.c' 2024-03-09T09:28:13.031 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getline.c’: No data available 2024-03-09T09:28:13.032 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-md5.c' -> './k/coreutils-8.5/gnulib-tests/test-md5.c' 2024-03-09T09:28:13.032 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-md5.c’: No data available 2024-03-09T09:28:13.034 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/Makefile.am' -> './k/coreutils-8.5/gnulib-tests/Makefile.am' 2024-03-09T09:28:13.034 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/Makefile.am’: No data available 2024-03-09T09:28:13.036 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-select-in.sh' -> './k/coreutils-8.5/gnulib-tests/test-select-in.sh' 2024-03-09T09:28:13.036 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-select-in.sh’: No data available 2024-03-09T09:28:13.038 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-uname.c' -> './k/coreutils-8.5/gnulib-tests/test-uname.c' 2024-03-09T09:28:13.038 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-uname.c’: No data available 2024-03-09T09:28:13.040 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-readlink.h' -> './k/coreutils-8.5/gnulib-tests/test-readlink.h' 2024-03-09T09:28:13.040 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-readlink.h’: No data available 2024-03-09T09:28:13.042 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-string.c' -> './k/coreutils-8.5/gnulib-tests/test-string.c' 2024-03-09T09:28:13.042 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-string.c’: No data available 2024-03-09T09:28:13.043 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-argmatch.c' -> './k/coreutils-8.5/gnulib-tests/test-argmatch.c' 2024-03-09T09:28:13.044 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-argmatch.c’: No data available 2024-03-09T09:28:13.045 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-nl_langinfo.sh' -> './k/coreutils-8.5/gnulib-tests/test-nl_langinfo.sh' 2024-03-09T09:28:13.046 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-nl_langinfo.sh’: No data available 2024-03-09T09:28:13.047 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_stat.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_stat.c' 2024-03-09T09:28:13.047 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_stat.c’: No data available 2024-03-09T09:28:13.049 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-version-etc.c' -> './k/coreutils-8.5/gnulib-tests/test-version-etc.c' 2024-03-09T09:28:13.049 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-version-etc.c’: No data available 2024-03-09T09:28:13.051 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-areadlink.h' -> './k/coreutils-8.5/gnulib-tests/test-areadlink.h' 2024-03-09T09:28:13.051 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-areadlink.h’: No data available 2024-03-09T09:28:13.053 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-version-etc.sh' -> './k/coreutils-8.5/gnulib-tests/test-version-etc.sh' 2024-03-09T09:28:13.053 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-version-etc.sh’: No data available 2024-03-09T09:28:13.055 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-binary-io.c' -> './k/coreutils-8.5/gnulib-tests/test-binary-io.c' 2024-03-09T09:28:13.055 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-binary-io.c’: No data available 2024-03-09T09:28:13.057 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude2.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude2.sh' 2024-03-09T09:28:13.057 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude2.sh’: No data available 2024-03-09T09:28:13.058 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xfprintf-posix.c' -> './k/coreutils-8.5/gnulib-tests/test-xfprintf-posix.c' 2024-03-09T09:28:13.059 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xfprintf-posix.c’: No data available 2024-03-09T09:28:13.060 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-set-mode-acl.c' -> './k/coreutils-8.5/gnulib-tests/test-set-mode-acl.c' 2024-03-09T09:28:13.060 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-set-mode-acl.c’: No data available 2024-03-09T09:28:13.063 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-wchar.c' -> './k/coreutils-8.5/gnulib-tests/test-wchar.c' 2024-03-09T09:28:13.063 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-wchar.c’: No data available 2024-03-09T09:28:13.065 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vasprintf.c' -> './k/coreutils-8.5/gnulib-tests/test-vasprintf.c' 2024-03-09T09:28:13.065 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vasprintf.c’: No data available 2024-03-09T09:28:13.066 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtoimax.sh' -> './k/coreutils-8.5/gnulib-tests/test-xstrtoimax.sh' 2024-03-09T09:28:13.067 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtoimax.sh’: No data available 2024-03-09T09:28:13.068 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadseek.sh' -> './k/coreutils-8.5/gnulib-tests/test-freadseek.sh' 2024-03-09T09:28:13.068 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadseek.sh’: No data available 2024-03-09T09:28:13.074 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isblank.c' -> './k/coreutils-8.5/gnulib-tests/test-isblank.c' 2024-03-09T09:28:13.074 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isblank.c’: No data available 2024-03-09T09:28:13.074 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fpending.sh' -> './k/coreutils-8.5/gnulib-tests/test-fpending.sh' 2024-03-09T09:28:13.075 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fpending.sh’: No data available 2024-03-09T09:28:13.075 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lstat.h' -> './k/coreutils-8.5/gnulib-tests/test-lstat.h' 2024-03-09T09:28:13.075 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lstat.h’: No data available 2024-03-09T09:28:13.075 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-chown.c' -> './k/coreutils-8.5/gnulib-tests/test-chown.c' 2024-03-09T09:28:13.076 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-chown.c’: No data available 2024-03-09T09:28:13.077 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtoumax.sh' -> './k/coreutils-8.5/gnulib-tests/test-xstrtoumax.sh' 2024-03-09T09:28:13.078 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtoumax.sh’: No data available 2024-03-09T09:28:13.080 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/nan.h' -> './k/coreutils-8.5/gnulib-tests/nan.h' 2024-03-09T09:28:13.080 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/nan.h’: No data available 2024-03-09T09:28:13.082 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xprintf-posix.c' -> './k/coreutils-8.5/gnulib-tests/test-xprintf-posix.c' 2024-03-09T09:28:13.082 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xprintf-posix.c’: No data available 2024-03-09T09:28:13.084 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-inet_pton.c' -> './k/coreutils-8.5/gnulib-tests/test-inet_pton.c' 2024-03-09T09:28:13.084 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-inet_pton.c’: No data available 2024-03-09T09:28:13.086 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-dup-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-dup-safer.c' 2024-03-09T09:28:13.086 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-dup-safer.c’: No data available 2024-03-09T09:28:13.088 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-closein.sh' -> './k/coreutils-8.5/gnulib-tests/test-closein.sh' 2024-03-09T09:28:13.089 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-closein.sh’: No data available 2024-03-09T09:28:13.091 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-canonicalize.c' -> './k/coreutils-8.5/gnulib-tests/test-canonicalize.c' 2024-03-09T09:28:13.091 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-canonicalize.c’: No data available 2024-03-09T09:28:13.093 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnanl-nolibm.c' -> './k/coreutils-8.5/gnulib-tests/test-isnanl-nolibm.c' 2024-03-09T09:28:13.094 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnanl-nolibm.c’: No data available 2024-03-09T09:28:13.096 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtol.c' -> './k/coreutils-8.5/gnulib-tests/test-xstrtol.c' 2024-03-09T09:28:13.096 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtol.c’: No data available 2024-03-09T09:28:13.098 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-open.c' -> './k/coreutils-8.5/gnulib-tests/test-open.c' 2024-03-09T09:28:13.098 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-open.c’: No data available 2024-03-09T09:28:13.100 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fprintf-posix.h' -> './k/coreutils-8.5/gnulib-tests/test-fprintf-posix.h' 2024-03-09T09:28:13.100 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fprintf-posix.h’: No data available 2024-03-09T09:28:13.103 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbrtowc1.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbrtowc1.sh' 2024-03-09T09:28:13.103 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbrtowc1.sh’: No data available 2024-03-09T09:28:13.105 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-errno.c' -> './k/coreutils-8.5/gnulib-tests/test-errno.c' 2024-03-09T09:28:13.105 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-errno.c’: No data available 2024-03-09T09:28:13.107 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-netdb.c' -> './k/coreutils-8.5/gnulib-tests/test-netdb.c' 2024-03-09T09:28:13.107 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-netdb.c’: No data available 2024-03-09T09:28:13.109 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-wctype.c' -> './k/coreutils-8.5/gnulib-tests/test-wctype.c' 2024-03-09T09:28:13.109 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-wctype.c’: No data available 2024-03-09T09:28:13.112 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-strsignal.c' -> './k/coreutils-8.5/gnulib-tests/test-strsignal.c' 2024-03-09T09:28:13.112 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-strsignal.c’: No data available 2024-03-09T09:28:13.114 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsstr3.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsstr3.sh' 2024-03-09T09:28:13.114 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsstr3.sh’: No data available 2024-03-09T09:28:13.116 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-yesno.c' -> './k/coreutils-8.5/gnulib-tests/test-yesno.c' 2024-03-09T09:28:13.116 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-yesno.c’: No data available 2024-03-09T09:28:13.117 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-userspec.c' -> './k/coreutils-8.5/gnulib-tests/test-userspec.c' 2024-03-09T09:28:13.117 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-userspec.c’: No data available 2024-03-09T09:28:13.118 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freopen.c' -> './k/coreutils-8.5/gnulib-tests/test-freopen.c' 2024-03-09T09:28:13.118 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freopen.c’: No data available 2024-03-09T09:28:13.120 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sockets.c' -> './k/coreutils-8.5/gnulib-tests/test-sockets.c' 2024-03-09T09:28:13.120 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sockets.c’: No data available 2024-03-09T09:28:13.122 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-link.h' -> './k/coreutils-8.5/gnulib-tests/test-link.h' 2024-03-09T09:28:13.122 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-link.h’: No data available 2024-03-09T09:28:13.124 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-select-fd.c' -> './k/coreutils-8.5/gnulib-tests/test-select-fd.c' 2024-03-09T09:28:13.124 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-select-fd.c’: No data available 2024-03-09T09:28:13.125 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtoimax.c' -> './k/coreutils-8.5/gnulib-tests/test-xstrtoimax.c' 2024-03-09T09:28:13.126 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtoimax.c’: No data available 2024-03-09T09:28:13.127 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xvasprintf.c' -> './k/coreutils-8.5/gnulib-tests/test-xvasprintf.c' 2024-03-09T09:28:13.127 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xvasprintf.c’: No data available 2024-03-09T09:28:13.128 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-yesno.sh' -> './k/coreutils-8.5/gnulib-tests/test-yesno.sh' 2024-03-09T09:28:13.128 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-yesno.sh’: No data available 2024-03-09T09:28:13.130 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fflush.c' -> './k/coreutils-8.5/gnulib-tests/test-fflush.c' 2024-03-09T09:28:13.130 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fflush.c’: No data available 2024-03-09T09:28:13.131 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vc-list-files-cvs.sh' -> './k/coreutils-8.5/gnulib-tests/test-vc-list-files-cvs.sh' 2024-03-09T09:28:13.131 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vc-list-files-cvs.sh’: No data available 2024-03-09T09:28:13.132 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbscasecmp.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbscasecmp.sh' 2024-03-09T09:28:13.133 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbscasecmp.sh’: No data available 2024-03-09T09:28:13.133 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lseek.sh' -> './k/coreutils-8.5/gnulib-tests/test-lseek.sh' 2024-03-09T09:28:13.134 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lseek.sh’: No data available 2024-03-09T09:28:13.135 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/usleep.c' -> './k/coreutils-8.5/gnulib-tests/usleep.c' 2024-03-09T09:28:13.135 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/usleep.c’: No data available 2024-03-09T09:28:13.137 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sleep.c' -> './k/coreutils-8.5/gnulib-tests/test-sleep.c' 2024-03-09T09:28:13.137 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sleep.c’: No data available 2024-03-09T09:28:13.138 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-link.c' -> './k/coreutils-8.5/gnulib-tests/test-link.c' 2024-03-09T09:28:13.138 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-link.c’: No data available 2024-03-09T09:28:13.139 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-perror.c' -> './k/coreutils-8.5/gnulib-tests/test-perror.c' 2024-03-09T09:28:13.140 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-perror.c’: No data available 2024-03-09T09:28:13.141 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsrtowcs.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsrtowcs.c' 2024-03-09T09:28:13.142 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsrtowcs.c’: No data available 2024-03-09T09:28:13.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.142+0000 7fe67a800640 1 -- 172.21.15.67:0/3216103492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe674156320 con 0x7fe6740cadb0 2024-03-09T09:28:13.149 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xalloc-die.sh' -> './k/coreutils-8.5/gnulib-tests/test-xalloc-die.sh' 2024-03-09T09:28:13.150 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xalloc-die.sh’: No data available 2024-03-09T09:28:13.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.143+0000 7fe671600640 1 -- 172.21.15.67:0/3216103492 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7fe66004de80 con 0x7fe6740cadb0 2024-03-09T09:28:13.150 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":282,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:28:13.151 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-i-ring.c' -> './k/coreutils-8.5/gnulib-tests/test-i-ring.c' 2024-03-09T09:28:13.151 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-i-ring.c’: No data available 2024-03-09T09:28:13.152 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-priv-set.c' -> './k/coreutils-8.5/gnulib-tests/test-priv-set.c' 2024-03-09T09:28:13.153 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-priv-set.c’: No data available 2024-03-09T09:28:13.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 -- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe65406b500 msgr2=0x7fe65406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe65406b500 0x7fe65406d9c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe668000f60 tx=0x7fe668007b50 comp rx=0 tx=0).stop 2024-03-09T09:28:13.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 -- 172.21.15.67:0/3216103492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6740cadb0 msgr2=0x7fe6740c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6740cadb0 0x7fe6740c7f00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe6600097a0 tx=0x7fe6600046c0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 -- 172.21.15.67:0/3216103492 shutdown_connections 2024-03-09T09:28:13.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fe65406b500 0x7fe65406d9c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe674153300 0x7fe67406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe6740cadb0 0x7fe6740c7f00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 --2- 172.21.15.67:0/3216103492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6740c5e40 0x7fe6741eff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 -- 172.21.15.67:0/3216103492 >> 172.21.15.67:0/3216103492 conn(0x7fe6740732d0 msgr2=0x7fe6741518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:13.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 -- 172.21.15.67:0/3216103492 shutdown_connections 2024-03-09T09:28:13.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.146+0000 7fe67a800640 1 -- 172.21.15.67:0/3216103492 wait complete. 2024-03-09T09:28:13.155 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-btowc.c' -> './k/coreutils-8.5/gnulib-tests/test-btowc.c' 2024-03-09T09:28:13.156 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-btowc.c’: No data available 2024-03-09T09:28:13.156 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_select.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_select.c' 2024-03-09T09:28:13.156 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_select.c’: No data available 2024-03-09T09:28:13.166 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbrtowc.c' -> './k/coreutils-8.5/gnulib-tests/test-mbrtowc.c' 2024-03-09T09:28:13.166 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbrtowc.c’: No data available 2024-03-09T09:28:13.167 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-strerror.c' -> './k/coreutils-8.5/gnulib-tests/test-strerror.c' 2024-03-09T09:28:13.168 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-strerror.c’: No data available 2024-03-09T09:28:13.169 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-update-copyright.sh' -> './k/coreutils-8.5/gnulib-tests/test-update-copyright.sh' 2024-03-09T09:28:13.169 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-update-copyright.sh’: No data available 2024-03-09T09:28:13.170 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-btowc2.sh' -> './k/coreutils-8.5/gnulib-tests/test-btowc2.sh' 2024-03-09T09:28:13.170 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-btowc2.sh’: No data available 2024-03-09T09:28:13.171 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:28:13.171 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-03-09T09:28:13.172 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:28:13.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:28:13.172 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getcwd.c' -> './k/coreutils-8.5/gnulib-tests/test-getcwd.c' 2024-03-09T09:28:13.172 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getcwd.c’: No data available 2024-03-09T09:28:13.173 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/socket.c' -> './k/coreutils-8.5/gnulib-tests/socket.c' 2024-03-09T09:28:13.173 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/socket.c’: No data available 2024-03-09T09:28:13.176 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-set-mode-acl.sh' -> './k/coreutils-8.5/gnulib-tests/test-set-mode-acl.sh' 2024-03-09T09:28:13.176 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-set-mode-acl.sh’: No data available 2024-03-09T09:28:13.177 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-linkat.c' -> './k/coreutils-8.5/gnulib-tests/test-linkat.c' 2024-03-09T09:28:13.177 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-linkat.c’: No data available 2024-03-09T09:28:13.178 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/perror.c' -> './k/coreutils-8.5/gnulib-tests/perror.c' 2024-03-09T09:28:13.179 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/perror.c’: No data available 2024-03-09T09:28:13.179 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-wcrtomb.c' -> './k/coreutils-8.5/gnulib-tests/test-wcrtomb.c' 2024-03-09T09:28:13.180 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-wcrtomb.c’: No data available 2024-03-09T09:28:13.181 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vasnprintf.c' -> './k/coreutils-8.5/gnulib-tests/test-vasnprintf.c' 2024-03-09T09:28:13.181 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vasnprintf.c’: No data available 2024-03-09T09:28:13.182 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-memrchr.c' -> './k/coreutils-8.5/gnulib-tests/test-memrchr.c' 2024-03-09T09:28:13.182 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-memrchr.c’: No data available 2024-03-09T09:28:13.183 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-ftello.c' -> './k/coreutils-8.5/gnulib-tests/test-ftello.c' 2024-03-09T09:28:13.184 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-ftello.c’: No data available 2024-03-09T09:28:13.185 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-chown.h' -> './k/coreutils-8.5/gnulib-tests/test-chown.h' 2024-03-09T09:28:13.185 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-chown.h’: No data available 2024-03-09T09:28:13.187 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stat.h' -> './k/coreutils-8.5/gnulib-tests/test-stat.h' 2024-03-09T09:28:13.187 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stat.h’: No data available 2024-03-09T09:28:13.190 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/Makefile.in' -> './k/coreutils-8.5/gnulib-tests/Makefile.in' 2024-03-09T09:28:13.190 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/Makefile.in’: No data available 2024-03-09T09:28:13.191 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getdelim.c' -> './k/coreutils-8.5/gnulib-tests/test-getdelim.c' 2024-03-09T09:28:13.191 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getdelim.c’: No data available 2024-03-09T09:28:13.192 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vprintf-posix.sh' -> './k/coreutils-8.5/gnulib-tests/test-vprintf-posix.sh' 2024-03-09T09:28:13.193 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vprintf-posix.sh’: No data available 2024-03-09T09:28:13.193 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fseeko2.sh' -> './k/coreutils-8.5/gnulib-tests/test-fseeko2.sh' 2024-03-09T09:28:13.194 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fseeko2.sh’: No data available 2024-03-09T09:28:13.195 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-c-strcase.sh' -> './k/coreutils-8.5/gnulib-tests/test-c-strcase.sh' 2024-03-09T09:28:13.195 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-c-strcase.sh’: No data available 2024-03-09T09:28:13.196 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsalign.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsalign.c' 2024-03-09T09:28:13.196 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsalign.c’: No data available 2024-03-09T09:28:13.197 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fcntl-h.c' -> './k/coreutils-8.5/gnulib-tests/test-fcntl-h.c' 2024-03-09T09:28:13.197 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fcntl-h.c’: No data available 2024-03-09T09:28:13.199 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vprintf-posix.c' -> './k/coreutils-8.5/gnulib-tests/test-vprintf-posix.c' 2024-03-09T09:28:13.199 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vprintf-posix.c’: No data available 2024-03-09T09:28:13.200 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-rename.c' -> './k/coreutils-8.5/gnulib-tests/test-rename.c' 2024-03-09T09:28:13.200 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-rename.c’: No data available 2024-03-09T09:28:13.202 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lstat.c' -> './k/coreutils-8.5/gnulib-tests/test-lstat.c' 2024-03-09T09:28:13.202 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lstat.c’: No data available 2024-03-09T09:28:13.203 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-malloca.c' -> './k/coreutils-8.5/gnulib-tests/test-malloca.c' 2024-03-09T09:28:13.203 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-malloca.c’: No data available 2024-03-09T09:28:13.204 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtoul.c' -> './k/coreutils-8.5/gnulib-tests/test-xstrtoul.c' 2024-03-09T09:28:13.204 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtoul.c’: No data available 2024-03-09T09:28:13.205 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-perror.sh' -> './k/coreutils-8.5/gnulib-tests/test-perror.sh' 2024-03-09T09:28:13.205 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-perror.sh’: No data available 2024-03-09T09:28:13.207 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vasprintf-posix.c' -> './k/coreutils-8.5/gnulib-tests/test-vasprintf-posix.c' 2024-03-09T09:28:13.207 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vasprintf-posix.c’: No data available 2024-03-09T09:28:13.208 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-frexp.c' -> './k/coreutils-8.5/gnulib-tests/test-frexp.c' 2024-03-09T09:28:13.208 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-frexp.c’: No data available 2024-03-09T09:28:13.209 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-environ.c' -> './k/coreutils-8.5/gnulib-tests/test-environ.c' 2024-03-09T09:28:13.209 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-environ.c’: No data available 2024-03-09T09:28:13.210 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stdint.c' -> './k/coreutils-8.5/gnulib-tests/test-stdint.c' 2024-03-09T09:28:13.211 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stdint.c’: No data available 2024-03-09T09:28:13.212 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnanl.h' -> './k/coreutils-8.5/gnulib-tests/test-isnanl.h' 2024-03-09T09:28:13.212 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnanl.h’: No data available 2024-03-09T09:28:13.213 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fcntl-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-fcntl-safer.c' 2024-03-09T09:28:13.214 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fcntl-safer.c’: No data available 2024-03-09T09:28:13.215 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-locale.c' -> './k/coreutils-8.5/gnulib-tests/test-locale.c' 2024-03-09T09:28:13.215 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-locale.c’: No data available 2024-03-09T09:28:13.217 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbrtowc4.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbrtowc4.sh' 2024-03-09T09:28:13.217 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbrtowc4.sh’: No data available 2024-03-09T09:28:13.219 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-argv-iter.c' -> './k/coreutils-8.5/gnulib-tests/test-argv-iter.c' 2024-03-09T09:28:13.219 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-argv-iter.c’: No data available 2024-03-09T09:28:13.220 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-utimens.h' -> './k/coreutils-8.5/gnulib-tests/test-utimens.h' 2024-03-09T09:28:13.221 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-utimens.h’: No data available 2024-03-09T09:28:13.222 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xalloc-die.c' -> './k/coreutils-8.5/gnulib-tests/test-xalloc-die.c' 2024-03-09T09:28:13.222 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xalloc-die.c’: No data available 2024-03-09T09:28:13.223 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fcntl.c' -> './k/coreutils-8.5/gnulib-tests/test-fcntl.c' 2024-03-09T09:28:13.224 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fcntl.c’: No data available 2024-03-09T09:28:13.224 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-alloca-opt.c' -> './k/coreutils-8.5/gnulib-tests/test-alloca-opt.c' 2024-03-09T09:28:13.225 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-alloca-opt.c’: No data available 2024-03-09T09:28:13.226 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-dirname.c' -> './k/coreutils-8.5/gnulib-tests/test-dirname.c' 2024-03-09T09:28:13.226 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-dirname.c’: No data available 2024-03-09T09:28:13.227 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-setenv.c' -> './k/coreutils-8.5/gnulib-tests/test-setenv.c' 2024-03-09T09:28:13.227 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-setenv.c’: No data available 2024-03-09T09:28:13.229 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-printf-frexpl.c' -> './k/coreutils-8.5/gnulib-tests/test-printf-frexpl.c' 2024-03-09T09:28:13.229 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-printf-frexpl.c’: No data available 2024-03-09T09:28:13.231 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-atexit.sh' -> './k/coreutils-8.5/gnulib-tests/test-atexit.sh' 2024-03-09T09:28:13.232 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-atexit.sh’: No data available 2024-03-09T09:28:13.233 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stddef.c' -> './k/coreutils-8.5/gnulib-tests/test-stddef.c' 2024-03-09T09:28:13.233 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stddef.c’: No data available 2024-03-09T09:28:13.234 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-c-ctype.c' -> './k/coreutils-8.5/gnulib-tests/test-c-ctype.c' 2024-03-09T09:28:13.235 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-c-ctype.c’: No data available 2024-03-09T09:28:13.236 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtol.sh' -> './k/coreutils-8.5/gnulib-tests/test-xstrtol.sh' 2024-03-09T09:28:13.236 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtol.sh’: No data available 2024-03-09T09:28:13.238 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getopt_long.h' -> './k/coreutils-8.5/gnulib-tests/test-getopt_long.h' 2024-03-09T09:28:13.238 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getopt_long.h’: No data available 2024-03-09T09:28:13.239 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-quotearg.sh' -> './k/coreutils-8.5/gnulib-tests/test-quotearg.sh' 2024-03-09T09:28:13.239 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-quotearg.sh’: No data available 2024-03-09T09:28:13.244 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-langinfo.c' -> './k/coreutils-8.5/gnulib-tests/test-langinfo.c' 2024-03-09T09:28:13.244 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-langinfo.c’: No data available 2024-03-09T09:28:13.247 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-inttypes.c' -> './k/coreutils-8.5/gnulib-tests/test-inttypes.c' 2024-03-09T09:28:13.247 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-inttypes.c’: No data available 2024-03-09T09:28:13.248 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-filevercmp.c' -> './k/coreutils-8.5/gnulib-tests/test-filevercmp.c' 2024-03-09T09:28:13.248 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-filevercmp.c’: No data available 2024-03-09T09:28:13.250 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-rmdir.c' -> './k/coreutils-8.5/gnulib-tests/test-rmdir.c' 2024-03-09T09:28:13.250 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-rmdir.c’: No data available 2024-03-09T09:28:13.251 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-futimens.h' -> './k/coreutils-8.5/gnulib-tests/test-futimens.h' 2024-03-09T09:28:13.252 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-futimens.h’: No data available 2024-03-09T09:28:13.253 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/setsockopt.c' -> './k/coreutils-8.5/gnulib-tests/setsockopt.c' 2024-03-09T09:28:13.253 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/setsockopt.c’: No data available 2024-03-09T09:28:13.255 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude1.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude1.sh' 2024-03-09T09:28:13.255 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude1.sh’: No data available 2024-03-09T09:28:13.256 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freopen-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-freopen-safer.c' 2024-03-09T09:28:13.257 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freopen-safer.c’: No data available 2024-03-09T09:28:13.258 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getaddrinfo.c' -> './k/coreutils-8.5/gnulib-tests/test-getaddrinfo.c' 2024-03-09T09:28:13.259 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getaddrinfo.c’: No data available 2024-03-09T09:28:13.260 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-iconv.c' -> './k/coreutils-8.5/gnulib-tests/test-iconv.c' 2024-03-09T09:28:13.260 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-iconv.c’: No data available 2024-03-09T09:28:13.261 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/zerosize-ptr.h' -> './k/coreutils-8.5/gnulib-tests/zerosize-ptr.h' 2024-03-09T09:28:13.262 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/zerosize-ptr.h’: No data available 2024-03-09T09:28:13.262 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/cs.po' -> './k/coreutils-8.5/po/cs.po' 2024-03-09T09:28:13.263 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/cs.po’: No data available 2024-03-09T09:28:13.263 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lseek.c' -> './k/coreutils-8.5/gnulib-tests/test-lseek.c' 2024-03-09T09:28:13.263 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lseek.c’: No data available 2024-03-09T09:28:13.265 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-usleep.c' -> './k/coreutils-8.5/gnulib-tests/test-usleep.c' 2024-03-09T09:28:13.265 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-usleep.c’: No data available 2024-03-09T09:28:13.267 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/gnulib.mk' -> './k/coreutils-8.5/gnulib-tests/gnulib.mk' 2024-03-09T09:28:13.268 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/gnulib.mk’: No data available 2024-03-09T09:28:13.269 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-unlinkat.c' -> './k/coreutils-8.5/gnulib-tests/test-unlinkat.c' 2024-03-09T09:28:13.269 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-unlinkat.c’: No data available 2024-03-09T09:28:13.270 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-quotearg.c' -> './k/coreutils-8.5/gnulib-tests/test-quotearg.c' 2024-03-09T09:28:13.271 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-quotearg.c’: No data available 2024-03-09T09:28:13.272 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ru.po' -> './k/coreutils-8.5/po/ru.po' 2024-03-09T09:28:13.272 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ru.po’: No data available 2024-03-09T09:28:13.272 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-base64.c' -> './k/coreutils-8.5/gnulib-tests/test-base64.c' 2024-03-09T09:28:13.273 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-base64.c’: No data available 2024-03-09T09:28:13.273 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-frexpl.c' -> './k/coreutils-8.5/gnulib-tests/test-frexpl.c' 2024-03-09T09:28:13.273 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-frexpl.c’: No data available 2024-03-09T09:28:13.275 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-gethostname.c' -> './k/coreutils-8.5/gnulib-tests/test-gethostname.c' 2024-03-09T09:28:13.278 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-gethostname.c’: No data available 2024-03-09T09:28:13.278 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mkdir.h' -> './k/coreutils-8.5/gnulib-tests/test-mkdir.h' 2024-03-09T09:28:13.278 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mkdir.h’: No data available 2024-03-09T09:28:13.279 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-binary-io.sh' -> './k/coreutils-8.5/gnulib-tests/test-binary-io.sh' 2024-03-09T09:28:13.279 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-binary-io.sh’: No data available 2024-03-09T09:28:13.279 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_ioctl.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_ioctl.c' 2024-03-09T09:28:13.279 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_ioctl.c’: No data available 2024-03-09T09:28:13.280 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-time.c' -> './k/coreutils-8.5/gnulib-tests/test-time.c' 2024-03-09T09:28:13.280 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-time.c’: No data available 2024-03-09T09:28:13.281 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/hu.po' -> './k/coreutils-8.5/po/hu.po' 2024-03-09T09:28:13.281 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/signature.h' -> './k/coreutils-8.5/gnulib-tests/signature.h' 2024-03-09T09:28:13.281 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/hu.po’: No data available 2024-03-09T09:28:13.281 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/signature.h’: No data available 2024-03-09T09:28:13.282 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnanf.h' -> './k/coreutils-8.5/gnulib-tests/test-isnanf.h' 2024-03-09T09:28:13.282 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnanf.h’: No data available 2024-03-09T09:28:13.283 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadseek.c' -> './k/coreutils-8.5/gnulib-tests/test-freadseek.c' 2024-03-09T09:28:13.283 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadseek.c’: No data available 2024-03-09T09:28:13.284 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-areadlink-with-size.c' -> './k/coreutils-8.5/gnulib-tests/test-areadlink-with-size.c' 2024-03-09T09:28:13.285 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-areadlink-with-size.c’: No data available 2024-03-09T09:28:13.286 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-select-stdin.c' -> './k/coreutils-8.5/gnulib-tests/test-select-stdin.c' 2024-03-09T09:28:13.286 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-select-stdin.c’: No data available 2024-03-09T09:28:13.287 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsstr1.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsstr1.c' 2024-03-09T09:28:13.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/id.po' -> './k/coreutils-8.5/po/id.po' 2024-03-09T09:28:13.287 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsstr1.c’: No data available 2024-03-09T09:28:13.288 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/id.po’: No data available 2024-03-09T09:28:13.288 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude3.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude3.sh' 2024-03-09T09:28:13.288 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude3.sh’: No data available 2024-03-09T09:28:13.289 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fopen.c' -> './k/coreutils-8.5/gnulib-tests/test-fopen.c' 2024-03-09T09:28:13.290 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fopen.c’: No data available 2024-03-09T09:28:13.291 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lock.c' -> './k/coreutils-8.5/gnulib-tests/test-lock.c' 2024-03-09T09:28:13.291 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lock.c’: No data available 2024-03-09T09:28:13.292 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fpending.c' -> './k/coreutils-8.5/gnulib-tests/test-fpending.c' 2024-03-09T09:28:13.292 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fpending.c’: No data available 2024-03-09T09:28:13.292 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ru.gmo' -> './k/coreutils-8.5/po/ru.gmo' 2024-03-09T09:28:13.293 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ru.gmo’: No data available 2024-03-09T09:28:13.293 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-netinet_in.c' -> './k/coreutils-8.5/gnulib-tests/test-netinet_in.c' 2024-03-09T09:28:13.293 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-netinet_in.c’: No data available 2024-03-09T09:28:13.295 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-bitrotate.c' -> './k/coreutils-8.5/gnulib-tests/test-bitrotate.c' 2024-03-09T09:28:13.295 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-bitrotate.c’: No data available 2024-03-09T09:28:13.296 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-c-strcasecmp.c' -> './k/coreutils-8.5/gnulib-tests/test-c-strcasecmp.c' 2024-03-09T09:28:13.297 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-c-strcasecmp.c’: No data available 2024-03-09T09:28:13.298 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/sl.gmo' -> './k/coreutils-8.5/po/sl.gmo' 2024-03-09T09:28:13.298 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sl.gmo’: No data available 2024-03-09T09:28:13.298 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsstr3.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsstr3.c' 2024-03-09T09:28:13.299 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsstr3.c’: No data available 2024-03-09T09:28:13.299 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-hash.c' -> './k/coreutils-8.5/gnulib-tests/test-hash.c' 2024-03-09T09:28:13.299 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-hash.c’: No data available 2024-03-09T09:28:13.300 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/remove-potcdate.sin' -> './k/coreutils-8.5/po/remove-potcdate.sin' 2024-03-09T09:28:13.300 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/remove-potcdate.sin’: No data available 2024-03-09T09:28:13.300 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-copy-acl.c' -> './k/coreutils-8.5/gnulib-tests/test-copy-acl.c' 2024-03-09T09:28:13.301 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-copy-acl.c’: No data available 2024-03-09T09:28:13.302 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sigaction.c' -> './k/coreutils-8.5/gnulib-tests/test-sigaction.c' 2024-03-09T09:28:13.302 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sigaction.c’: No data available 2024-03-09T09:28:13.303 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtoumax.c' -> './k/coreutils-8.5/gnulib-tests/test-xstrtoumax.c' 2024-03-09T09:28:13.303 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtoumax.c’: No data available 2024-03-09T09:28:13.304 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/hu.gmo' -> './k/coreutils-8.5/po/hu.gmo' 2024-03-09T09:28:13.304 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/hu.gmo’: No data available 2024-03-09T09:28:13.304 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fseterr.c' -> './k/coreutils-8.5/gnulib-tests/test-fseterr.c' 2024-03-09T09:28:13.304 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fseterr.c’: No data available 2024-03-09T09:28:13.305 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-areadlink.c' -> './k/coreutils-8.5/gnulib-tests/test-areadlink.c' 2024-03-09T09:28:13.306 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-areadlink.c’: No data available 2024-03-09T09:28:13.307 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-signbit.c' -> './k/coreutils-8.5/gnulib-tests/test-signbit.c' 2024-03-09T09:28:13.307 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-signbit.c’: No data available 2024-03-09T09:28:13.308 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-btowc1.sh' -> './k/coreutils-8.5/gnulib-tests/test-btowc1.sh' 2024-03-09T09:28:13.308 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-btowc1.sh’: No data available 2024-03-09T09:28:13.310 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-strtod.c' -> './k/coreutils-8.5/gnulib-tests/test-strtod.c' 2024-03-09T09:28:13.310 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-strtod.c’: No data available 2024-03-09T09:28:13.310 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/et.po' -> './k/coreutils-8.5/po/et.po' 2024-03-09T09:28:13.311 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/et.po’: No data available 2024-03-09T09:28:13.311 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-atexit.c' -> './k/coreutils-8.5/gnulib-tests/test-atexit.c' 2024-03-09T09:28:13.311 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-atexit.c’: No data available 2024-03-09T09:28:13.313 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getdate.c' -> './k/coreutils-8.5/gnulib-tests/test-getdate.c' 2024-03-09T09:28:13.313 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getdate.c’: No data available 2024-03-09T09:28:13.314 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fpurge.c' -> './k/coreutils-8.5/gnulib-tests/test-fpurge.c' 2024-03-09T09:28:13.314 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fpurge.c’: No data available 2024-03-09T09:28:13.315 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-select.c' -> './k/coreutils-8.5/gnulib-tests/test-select.c' 2024-03-09T09:28:13.315 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-select.c’: No data available 2024-03-09T09:28:13.317 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-tls.c' -> './k/coreutils-8.5/gnulib-tests/test-tls.c' 2024-03-09T09:28:13.317 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-tls.c’: No data available 2024-03-09T09:28:13.319 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/sleep.c' -> './k/coreutils-8.5/gnulib-tests/sleep.c' 2024-03-09T09:28:13.319 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/sleep.c’: No data available 2024-03-09T09:28:13.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/gl.po' -> './k/coreutils-8.5/po/gl.po' 2024-03-09T09:28:13.319 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/gl.po’: No data available 2024-03-09T09:28:13.320 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbrtowc2.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbrtowc2.sh' 2024-03-09T09:28:13.320 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbrtowc2.sh’: No data available 2024-03-09T09:28:13.321 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stat.c' -> './k/coreutils-8.5/gnulib-tests/test-stat.c' 2024-03-09T09:28:13.322 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stat.c’: No data available 2024-03-09T09:28:13.323 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadptr.sh' -> './k/coreutils-8.5/gnulib-tests/test-freadptr.sh' 2024-03-09T09:28:13.323 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadptr.sh’: No data available 2024-03-09T09:28:13.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/vi.gmo' -> './k/coreutils-8.5/po/vi.gmo' 2024-03-09T09:28:13.324 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/vi.gmo’: No data available 2024-03-09T09:28:13.325 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/accept.c' -> './k/coreutils-8.5/gnulib-tests/accept.c' 2024-03-09T09:28:13.325 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/accept.c’: No data available 2024-03-09T09:28:13.326 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-unistd.c' -> './k/coreutils-8.5/gnulib-tests/test-unistd.c' 2024-03-09T09:28:13.326 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-unistd.c’: No data available 2024-03-09T09:28:13.328 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fopen.h' -> './k/coreutils-8.5/gnulib-tests/test-fopen.h' 2024-03-09T09:28:13.328 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fopen.h’: No data available 2024-03-09T09:28:13.329 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-math.c' -> './k/coreutils-8.5/gnulib-tests/test-math.c' 2024-03-09T09:28:13.329 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-math.c’: No data available 2024-03-09T09:28:13.331 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-printf-frexp.c' -> './k/coreutils-8.5/gnulib-tests/test-printf-frexp.c' 2024-03-09T09:28:13.331 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-printf-frexp.c’: No data available 2024-03-09T09:28:13.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/bg.po' -> './k/coreutils-8.5/po/bg.po' 2024-03-09T09:28:13.332 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/bg.po’: No data available 2024-03-09T09:28:13.332 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.sh' -> './k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.sh' 2024-03-09T09:28:13.332 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.sh’: No data available 2024-03-09T09:28:13.334 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude.c' -> './k/coreutils-8.5/gnulib-tests/test-exclude.c' 2024-03-09T09:28:13.334 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude.c’: No data available 2024-03-09T09:28:13.336 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freading.c' -> './k/coreutils-8.5/gnulib-tests/test-freading.c' 2024-03-09T09:28:13.336 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freading.c’: No data available 2024-03-09T09:28:13.337 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getdtablesize.c' -> './k/coreutils-8.5/gnulib-tests/test-getdtablesize.c' 2024-03-09T09:28:13.337 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getdtablesize.c’: No data available 2024-03-09T09:28:13.338 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/pl.po' -> './k/coreutils-8.5/po/pl.po' 2024-03-09T09:28:13.339 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pl.po’: No data available 2024-03-09T09:28:13.339 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/unlinkdir.c' -> './k/coreutils-8.5/gnulib-tests/unlinkdir.c' 2024-03-09T09:28:13.339 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/unlinkdir.c’: No data available 2024-03-09T09:28:13.340 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fflush2.c' -> './k/coreutils-8.5/gnulib-tests/test-fflush2.c' 2024-03-09T09:28:13.340 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fflush2.c’: No data available 2024-03-09T09:28:13.342 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stat-time.c' -> './k/coreutils-8.5/gnulib-tests/test-stat-time.c' 2024-03-09T09:28:13.342 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stat-time.c’: No data available 2024-03-09T09:28:13.345 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-inet_ntop.c' -> './k/coreutils-8.5/gnulib-tests/test-inet_ntop.c' 2024-03-09T09:28:13.345 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-inet_ntop.c’: No data available 2024-03-09T09:28:13.346 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vc-list-files-git.sh' -> './k/coreutils-8.5/gnulib-tests/test-vc-list-files-git.sh' 2024-03-09T09:28:13.347 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vc-list-files-git.sh’: No data available 2024-03-09T09:28:13.347 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/nl.po' -> './k/coreutils-8.5/po/nl.po' 2024-03-09T09:28:13.347 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/nl.po’: No data available 2024-03-09T09:28:13.348 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getopt.c' -> './k/coreutils-8.5/gnulib-tests/test-getopt.c' 2024-03-09T09:28:13.349 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getopt.c’: No data available 2024-03-09T09:28:13.350 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-alignof.c' -> './k/coreutils-8.5/gnulib-tests/test-alignof.c' 2024-03-09T09:28:13.350 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-alignof.c’: No data available 2024-03-09T09:28:13.352 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/macros.h' -> './k/coreutils-8.5/gnulib-tests/macros.h' 2024-03-09T09:28:13.352 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/macros.h’: No data available 2024-03-09T09:28:13.352 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/pt_BR.gmo' -> './k/coreutils-8.5/po/pt_BR.gmo' 2024-03-09T09:28:13.353 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pt_BR.gmo’: No data available 2024-03-09T09:28:13.354 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-symlinkat.c' -> './k/coreutils-8.5/gnulib-tests/test-symlinkat.c' 2024-03-09T09:28:13.354 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-symlinkat.c’: No data available 2024-03-09T09:28:13.356 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ko.gmo' -> './k/coreutils-8.5/po/ko.gmo' 2024-03-09T09:28:13.356 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getgroups.c' -> './k/coreutils-8.5/gnulib-tests/test-getgroups.c' 2024-03-09T09:28:13.356 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ko.gmo’: No data available 2024-03-09T09:28:13.356 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getgroups.c’: No data available 2024-03-09T09:28:13.357 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xprintf-posix.sh' -> './k/coreutils-8.5/gnulib-tests/test-xprintf-posix.sh' 2024-03-09T09:28:13.357 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xprintf-posix.sh’: No data available 2024-03-09T09:28:13.359 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fdopendir.c' -> './k/coreutils-8.5/gnulib-tests/test-fdopendir.c' 2024-03-09T09:28:13.359 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fdopendir.c’: No data available 2024-03-09T09:28:13.360 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_socket.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_socket.c' 2024-03-09T09:28:13.360 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_socket.c’: No data available 2024-03-09T09:28:13.361 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/de.gmo' -> './k/coreutils-8.5/po/de.gmo' 2024-03-09T09:28:13.361 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/de.gmo’: No data available 2024-03-09T09:28:13.362 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbscasecmp.c' -> './k/coreutils-8.5/gnulib-tests/test-mbscasecmp.c' 2024-03-09T09:28:13.362 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbscasecmp.c’: No data available 2024-03-09T09:28:13.363 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/POTFILES.in' -> './k/coreutils-8.5/po/POTFILES.in' 2024-03-09T09:28:13.363 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/POTFILES.in’: No data available 2024-03-09T09:28:13.364 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getopt.h' -> './k/coreutils-8.5/gnulib-tests/test-getopt.h' 2024-03-09T09:28:13.364 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getopt.h’: No data available 2024-03-09T09:28:13.365 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsrtowcs1.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsrtowcs1.sh' 2024-03-09T09:28:13.366 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsrtowcs1.sh’: No data available 2024-03-09T09:28:13.367 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsinit.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsinit.sh' 2024-03-09T09:28:13.367 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsinit.sh’: No data available 2024-03-09T09:28:13.368 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-gettimeofday.c' -> './k/coreutils-8.5/gnulib-tests/test-gettimeofday.c' 2024-03-09T09:28:13.369 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-gettimeofday.c’: No data available 2024-03-09T09:28:13.369 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/sv.po' -> './k/coreutils-8.5/po/sv.po' 2024-03-09T09:28:13.369 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/sv.po’: No data available 2024-03-09T09:28:13.370 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsstr2.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsstr2.c' 2024-03-09T09:28:13.370 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsstr2.c’: No data available 2024-03-09T09:28:13.371 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/init.sh' -> './k/coreutils-8.5/gnulib-tests/init.sh' 2024-03-09T09:28:13.371 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/init.sh’: No data available 2024-03-09T09:28:13.372 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-symlink.c' -> './k/coreutils-8.5/gnulib-tests/test-symlink.c' 2024-03-09T09:28:13.372 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-symlink.c’: No data available 2024-03-09T09:28:13.374 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-utimens.c' -> './k/coreutils-8.5/gnulib-tests/test-utimens.c' 2024-03-09T09:28:13.374 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-utimens.c’: No data available 2024-03-09T09:28:13.375 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-readlink.c' -> './k/coreutils-8.5/gnulib-tests/test-readlink.c' 2024-03-09T09:28:13.375 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-readlink.c’: No data available 2024-03-09T09:28:13.375 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/it.po' -> './k/coreutils-8.5/po/it.po' 2024-03-09T09:28:13.376 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/it.po’: No data available 2024-03-09T09:28:13.376 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lchown.h' -> './k/coreutils-8.5/gnulib-tests/test-lchown.h' 2024-03-09T09:28:13.377 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lchown.h’: No data available 2024-03-09T09:28:13.377 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-c-strncasecmp.c' -> './k/coreutils-8.5/gnulib-tests/test-c-strncasecmp.c' 2024-03-09T09:28:13.377 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-c-strncasecmp.c’: No data available 2024-03-09T09:28:13.378 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/eu.gmo' -> './k/coreutils-8.5/po/eu.gmo' 2024-03-09T09:28:13.378 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/eu.gmo’: No data available 2024-03-09T09:28:13.379 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lutimens.h' -> './k/coreutils-8.5/gnulib-tests/test-lutimens.h' 2024-03-09T09:28:13.379 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lutimens.h’: No data available 2024-03-09T09:28:13.381 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-copy-acl.sh' -> './k/coreutils-8.5/gnulib-tests/test-copy-acl.sh' 2024-03-09T09:28:13.381 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-copy-acl.sh’: No data available 2024-03-09T09:28:13.381 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/en.po' -> './k/coreutils-8.5/po/en.po' 2024-03-09T09:28:13.381 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/en.po’: No data available 2024-03-09T09:28:13.382 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude6.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude6.sh' 2024-03-09T09:28:13.382 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude6.sh’: No data available 2024-03-09T09:28:13.383 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadptr2.c' -> './k/coreutils-8.5/gnulib-tests/test-freadptr2.c' 2024-03-09T09:28:13.383 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadptr2.c’: No data available 2024-03-09T09:28:13.385 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fflush2.sh' -> './k/coreutils-8.5/gnulib-tests/test-fflush2.sh' 2024-03-09T09:28:13.385 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fflush2.sh’: No data available 2024-03-09T09:28:13.385 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/it.gmo' -> './k/coreutils-8.5/po/it.gmo' 2024-03-09T09:28:13.385 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/it.gmo’: No data available 2024-03-09T09:28:13.386 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_utsname.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_utsname.c' 2024-03-09T09:28:13.386 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_utsname.c’: No data available 2024-03-09T09:28:13.387 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/en@boldquot.header' -> './k/coreutils-8.5/po/en@boldquot.header' 2024-03-09T09:28:13.387 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/en@boldquot.header’: No data available 2024-03-09T09:28:13.388 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsrtowcs2.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsrtowcs2.sh' 2024-03-09T09:28:13.388 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsrtowcs2.sh’: No data available 2024-03-09T09:28:13.389 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-file-has-acl.c' -> './k/coreutils-8.5/gnulib-tests/test-file-has-acl.c' 2024-03-09T09:28:13.389 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-file-has-acl.c’: No data available 2024-03-09T09:28:13.391 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-signal.c' -> './k/coreutils-8.5/gnulib-tests/test-signal.c' 2024-03-09T09:28:13.391 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/et.gmo' -> './k/coreutils-8.5/po/et.gmo' 2024-03-09T09:28:13.391 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-signal.c’: No data available 2024-03-09T09:28:13.391 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/et.gmo’: No data available 2024-03-09T09:28:13.392 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sha1.c' -> './k/coreutils-8.5/gnulib-tests/test-sha1.c' 2024-03-09T09:28:13.392 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sha1.c’: No data available 2024-03-09T09:28:13.393 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadptr2.sh' -> './k/coreutils-8.5/gnulib-tests/test-freadptr2.sh' 2024-03-09T09:28:13.393 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadptr2.sh’: No data available 2024-03-09T09:28:13.394 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-areadlinkat.c' -> './k/coreutils-8.5/gnulib-tests/test-areadlinkat.c' 2024-03-09T09:28:13.394 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-areadlinkat.c’: No data available 2024-03-09T09:28:13.395 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-unlink.h' -> './k/coreutils-8.5/gnulib-tests/test-unlink.h' 2024-03-09T09:28:13.395 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-unlink.h’: No data available 2024-03-09T09:28:13.396 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-dup2.c' -> './k/coreutils-8.5/gnulib-tests/test-dup2.c' 2024-03-09T09:28:13.396 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-dup2.c’: No data available 2024-03-09T09:28:13.397 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ja.po' -> './k/coreutils-8.5/po/ja.po' 2024-03-09T09:28:13.397 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ja.po’: No data available 2024-03-09T09:28:13.398 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadptr.c' -> './k/coreutils-8.5/gnulib-tests/test-freadptr.c' 2024-03-09T09:28:13.398 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadptr.c’: No data available 2024-03-09T09:28:13.400 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-file-has-acl.sh' -> './k/coreutils-8.5/gnulib-tests/test-file-has-acl.sh' 2024-03-09T09:28:13.400 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-file-has-acl.sh’: No data available 2024-03-09T09:28:13.401 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/zh_CN.gmo' -> './k/coreutils-8.5/po/zh_CN.gmo' 2024-03-09T09:28:13.402 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/zh_CN.gmo’: No data available 2024-03-09T09:28:13.403 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-u64.c' -> './k/coreutils-8.5/gnulib-tests/test-u64.c' 2024-03-09T09:28:13.403 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-u64.c’: No data available 2024-03-09T09:28:13.405 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-striconv.c' -> './k/coreutils-8.5/gnulib-tests/test-striconv.c' 2024-03-09T09:28:13.405 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-striconv.c’: No data available 2024-03-09T09:28:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.399+0000 7fae02400640 1 Processor -- start 2024-03-09T09:28:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.399+0000 7fae02400640 1 -- start start 2024-03-09T09:28:13.407 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sameacls.c' -> './k/coreutils-8.5/gnulib-tests/test-sameacls.c' 2024-03-09T09:28:13.407 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sameacls.c’: No data available 2024-03-09T09:28:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.400+0000 7fae02400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fadfc0ac9a0 0x7fadfc1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.400+0000 7fae02400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadfc071190 0x7fadfc151620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.400+0000 7fae02400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc0c8200 con 0x7fadfc0c2780 2024-03-09T09:28:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.400+0000 7fae02400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadfc06d590 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.400+0000 7fae02400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0c3740 con 0x7fadfc071190 2024-03-09T09:28:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.400+0000 7fadfb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fadfc0ac9a0 0x7fadfc1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.401+0000 7fadfb400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fadfc0ac9a0 0x7fadfc1510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37462/0 (socket says 172.21.15.67:37462) 2024-03-09T09:28:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.401+0000 7fadfb400640 1 -- 172.21.15.67:0/3328908122 learned_addr learned my addr 172.21.15.67:0/3328908122 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:13.409 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/uk.po' -> './k/coreutils-8.5/po/uk.po' 2024-03-09T09:28:13.409 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/uk.po’: No data available 2024-03-09T09:28:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.401+0000 7fadfbe00640 1 --2- 172.21.15.67:0/3328908122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadfc071190 0x7fadfc151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.401+0000 7fadfb400640 1 -- 172.21.15.67:0/3328908122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadfc071190 msgr2=0x7fadfc151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.401+0000 7fadfb400640 1 --2- 172.21.15.67:0/3328908122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadfc071190 0x7fadfc151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.401+0000 7fadfb400640 1 -- 172.21.15.67:0/3328908122 >> v1:172.21.15.134:6789/0 conn(0x7fadfc0c2780 legacy=0x7fadfc154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.401+0000 7fadfb400640 1 -- 172.21.15.67:0/3328908122 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc0c3990 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.410 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-remove.c' -> './k/coreutils-8.5/gnulib-tests/test-remove.c' 2024-03-09T09:28:13.410 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-remove.c’: No data available 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.403+0000 7fadfb400640 1 --2- 172.21.15.67:0/3328908122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc0ac9a0 0x7fadfc1510e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fadec009870 tx=0x7fadec02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cea43715c55f02 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.403+0000 7fadfa000640 1 -- 172.21.15.67:0/3328908122 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec0317c0 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.403+0000 7fadfa000640 1 -- 172.21.15.67:0/3328908122 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec037070 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.403+0000 7fadfa000640 1 -- 172.21.15.67:0/3328908122 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec031b80 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.404+0000 7fae02400640 1 -- 172.21.15.67:0/3328908122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc0ac9a0 msgr2=0x7fadfc1510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.404+0000 7fae02400640 1 --2- 172.21.15.67:0/3328908122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc0ac9a0 0x7fadfc1510e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fadec009870 tx=0x7fadec02b860 comp rx=0 tx=0).stop 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.404+0000 7fae02400640 1 -- 172.21.15.67:0/3328908122 shutdown_connections 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.404+0000 7fae02400640 1 --2- 172.21.15.67:0/3328908122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadfc071190 0x7fadfc151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.404+0000 7fae02400640 1 --2- 172.21.15.67:0/3328908122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc0ac9a0 0x7fadfc1510e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.404+0000 7fae02400640 1 -- 172.21.15.67:0/3328908122 >> 172.21.15.67:0/3328908122 conn(0x7fadfc0c7370 msgr2=0x7fadfc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.404+0000 7fae02400640 1 -- 172.21.15.67:0/3328908122 shutdown_connections 2024-03-09T09:28:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.404+0000 7fae02400640 1 -- 172.21.15.67:0/3328908122 wait complete. 2024-03-09T09:28:13.413 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fsync.c' -> './k/coreutils-8.5/gnulib-tests/test-fsync.c' 2024-03-09T09:28:13.413 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fsync.c’: No data available 2024-03-09T09:28:13.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.406+0000 7fae02400640 1 Processor -- start 2024-03-09T09:28:13.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.406+0000 7fae02400640 1 -- start start 2024-03-09T09:28:13.414 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ca.gmo' -> './k/coreutils-8.5/po/ca.gmo' 2024-03-09T09:28:13.414 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ca.gmo’: No data available 2024-03-09T09:28:13.415 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsrtowcs4.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsrtowcs4.sh' 2024-03-09T09:28:13.415 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsrtowcs4.sh’: No data available 2024-03-09T09:28:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.407+0000 7fae02400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadfc071190 0x7fadfc1ebbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.407+0000 7fae02400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc0ac9a0 0x7fadfc14cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.407+0000 7fae02400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadfc0c2780 0x7fadfc150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.407+0000 7fae02400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0acd80 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.407+0000 7fae02400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc070c00 con 0x7fadfc071190 2024-03-09T09:28:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.407+0000 7fae02400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0acf00 con 0x7fadfc0c2780 2024-03-09T09:28:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.408+0000 7fadfaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc0ac9a0 0x7fadfc14cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.408+0000 7fadfaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc0ac9a0 0x7fadfc14cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37476/0 (socket says 172.21.15.67:37476) 2024-03-09T09:28:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.408+0000 7fadfaa00640 1 -- 172.21.15.67:0/3275813983 learned_addr learned my addr 172.21.15.67:0/3275813983 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.408+0000 7fadfb400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadfc071190 0x7fadfc1ebbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.408+0000 7fadfbe00640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadfc0c2780 0x7fadfc150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.408+0000 7fadfaa00640 1 -- 172.21.15.67:0/3275813983 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadfc0c2780 msgr2=0x7fadfc150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.408+0000 7fadfaa00640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fadfc0c2780 0x7fadfc150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.409+0000 7fadfaa00640 1 -- 172.21.15.67:0/3275813983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadfc071190 msgr2=0x7fadfc1ebbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.409+0000 7fadfaa00640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fadfc071190 0x7fadfc1ebbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.409+0000 7fadfaa00640 1 -- 172.21.15.67:0/3275813983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc14d440 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.417 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnand.h' -> './k/coreutils-8.5/gnulib-tests/test-isnand.h' 2024-03-09T09:28:13.417 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnand.h’: No data available 2024-03-09T09:28:13.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.409+0000 7fadfaa00640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc0ac9a0 0x7fadfc14cae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fade800b3c0 tx=0x7fade800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.409+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade8015020 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.409+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade80118f0 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.409+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade8011d00 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.410+0000 7fae02400640 1 -- 172.21.15.67:0/3275813983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc14eb70 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.410+0000 7fae02400640 1 -- 172.21.15.67:0/3275813983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc0abf70 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.411+0000 7fae02400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc1efb20 0x7fadfc0c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.411+0000 7fae02400640 1 -- 172.21.15.67:0/3275813983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1eff30 con 0x7fadfc1efb20 2024-03-09T09:28:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.411+0000 7fadfb400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc1efb20 0x7fadfc0c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.420 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/Rules-quot' -> './k/coreutils-8.5/po/Rules-quot' 2024-03-09T09:28:13.420 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/Rules-quot’: No data available 2024-03-09T09:28:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.412+0000 7fadfb400640 1 -- 172.21.15.67:0/3275813983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fadfc075a00 con 0x7fadfc1efb20 2024-03-09T09:28:13.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.412+0000 7fadfb400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc1efb20 0x7fadfc0c68a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fadfc070320 tx=0x7fadec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.413+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec032650 con 0x7fadfc1efb20 2024-03-09T09:28:13.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.413+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fade8018020 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.421 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ro.po' -> './k/coreutils-8.5/po/ro.po' 2024-03-09T09:28:13.421 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ro.po’: No data available 2024-03-09T09:28:13.421 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/nb.gmo' -> './k/coreutils-8.5/po/nb.gmo' 2024-03-09T09:28:13.422 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/nb.gmo’: No data available 2024-03-09T09:28:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.414+0000 7fadf3400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faddc06b3d0 0x7faddc06d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.415+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fade808a5a0 con 0x7fadfc0ac9a0 2024-03-09T09:28:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.415+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fadec04a070 con 0x7fadfc1efb20 2024-03-09T09:28:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.415+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc1efb20 msgr2=0x7fadfc0c68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.415+0000 7fadf3400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc1efb20 0x7fadfc0c68a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fadfc070320 tx=0x7fadec004730 comp rx=0 tx=0).stop 2024-03-09T09:28:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.415+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc1efb20 msgr2=0x7fadfc0c68a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.415+0000 7fadf3400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc1efb20 0x7fadfc0c68a0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fadfc070320 tx=0x7fadec004730 comp rx=0 tx=0).stop 2024-03-09T09:28:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.415+0000 7fadfbe00640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faddc06b3d0 0x7faddc06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.424 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/bg.gmo' -> './k/coreutils-8.5/po/bg.gmo' 2024-03-09T09:28:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.416+0000 7fadfbe00640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faddc06b3d0 0x7faddc06d890 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fadf4007f20 tx=0x7fadf40074b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.424 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/bg.gmo’: No data available 2024-03-09T09:28:13.429 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/be.po' -> './k/coreutils-8.5/po/be.po' 2024-03-09T09:28:13.429 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/be.po’: No data available 2024-03-09T09:28:13.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.423+0000 7fae02400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc145ce0 0x7fadfc0c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.423+0000 7fae02400640 1 -- 172.21.15.67:0/3275813983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1ec110 con 0x7fadfc145ce0 2024-03-09T09:28:13.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.423+0000 7fadfb400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc145ce0 0x7fadfc0c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.423+0000 7fadfb400640 1 -- 172.21.15.67:0/3275813983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fadfc072b80 con 0x7fadfc145ce0 2024-03-09T09:28:13.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.424+0000 7fadfb400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc145ce0 0x7fadfc0c6de0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fadec000c00 tx=0x7fadec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.424+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec04fbd0 con 0x7fadfc145ce0 2024-03-09T09:28:13.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.424+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5190 (secure 0 0 0) 0x7fadec037040 con 0x7fadfc145ce0 2024-03-09T09:28:13.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.424+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc145ce0 msgr2=0x7fadfc0c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.424+0000 7fadf3400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc145ce0 0x7fadfc0c6de0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fadec000c00 tx=0x7fadec0046c0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.424+0000 7fadf3400640 1 -- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc145ce0 msgr2=0x7fadfc0c6de0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:13.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.424+0000 7fadf3400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc145ce0 0x7fadfc0c6de0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.432 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:28:13.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:13.432 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 282, 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 17, 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:13.433 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:13.434 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:28:13.435 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:13.436 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:28:13.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:13.438 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:28:13.439 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.440 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:13.441 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:13.442 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:28:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.444 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 5 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.445 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:13.446 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.446 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:13.446 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.446 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.446 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:28:13.446 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:28:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.425+0000 7fae02400640 1 -- 172.21.15.67:0/3275813983 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faddc06b3d0 msgr2=0x7faddc06d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.425+0000 7fae02400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faddc06b3d0 0x7faddc06d890 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fadf4007f20 tx=0x7fadf40074b0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.425+0000 7fae02400640 1 -- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc0ac9a0 msgr2=0x7fadfc14cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.425+0000 7fae02400640 1 --2- 172.21.15.67:0/3275813983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fadfc0ac9a0 0x7fadfc14cae0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fade800b3c0 tx=0x7fade800b880 comp rx=0 tx=0).stop 2024-03-09T09:28:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.425+0000 7fadfbe00640 1 -- 172.21.15.67:0/3275813983 reap_dead start 2024-03-09T09:28:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.425+0000 7fadfbe00640 1 -- 172.21.15.67:0/3275813983 reap_dead start 2024-03-09T09:28:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.425+0000 7fae02400640 1 -- 172.21.15.67:0/3275813983 shutdown_connections 2024-03-09T09:28:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.425+0000 7fae02400640 1 -- 172.21.15.67:0/3275813983 >> 172.21.15.67:0/3275813983 conn(0x7fadfc0c7370 msgr2=0x7fadfc0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.425+0000 7fae02400640 1 -- 172.21.15.67:0/3275813983 shutdown_connections 2024-03-09T09:28:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.425+0000 7fae02400640 1 -- 172.21.15.67:0/3275813983 wait complete. 2024-03-09T09:28:13.448 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/vi.po' -> './k/coreutils-8.5/po/vi.po' 2024-03-09T09:28:13.448 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/vi.po’: No data available 2024-03-09T09:28:13.448 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/lt.gmo' -> './k/coreutils-8.5/po/lt.gmo' 2024-03-09T09:28:13.449 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/lt.gmo’: No data available 2024-03-09T09:28:13.450 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:28:13.450 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:28:13.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:28:13.452 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/uk.gmo' -> './k/coreutils-8.5/po/uk.gmo' 2024-03-09T09:28:13.452 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/uk.gmo’: No data available 2024-03-09T09:28:13.458 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/id.gmo' -> './k/coreutils-8.5/po/id.gmo' 2024-03-09T09:28:13.458 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/id.gmo’: No data available 2024-03-09T09:28:13.463 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/af.po' -> './k/coreutils-8.5/po/af.po' 2024-03-09T09:28:13.463 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/af.po’: No data available 2024-03-09T09:28:13.467 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/eu.po' -> './k/coreutils-8.5/po/eu.po' 2024-03-09T09:28:13.467 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/eu.po’: No data available 2024-03-09T09:28:13.471 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ms.po' -> './k/coreutils-8.5/po/ms.po' 2024-03-09T09:28:13.471 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ms.po’: No data available 2024-03-09T09:28:13.473 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/en@quot.header' -> './k/coreutils-8.5/po/en@quot.header' 2024-03-09T09:28:13.473 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/en@quot.header’: No data available 2024-03-09T09:28:13.475 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/af.gmo' -> './k/coreutils-8.5/po/af.gmo' 2024-03-09T09:28:13.475 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/af.gmo’: No data available 2024-03-09T09:28:13.478 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/es.gmo' -> './k/coreutils-8.5/po/es.gmo' 2024-03-09T09:28:13.478 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/es.gmo’: No data available 2024-03-09T09:28:13.484 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ga.po' -> './k/coreutils-8.5/po/ga.po' 2024-03-09T09:28:13.484 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ga.po’: No data available 2024-03-09T09:28:13.487 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ro.gmo' -> './k/coreutils-8.5/po/ro.gmo' 2024-03-09T09:28:13.487 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ro.gmo’: No data available 2024-03-09T09:28:13.492 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ca.po' -> './k/coreutils-8.5/po/ca.po' 2024-03-09T09:28:13.492 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ca.po’: No data available 2024-03-09T09:28:13.497 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/lg.gmo' -> './k/coreutils-8.5/po/lg.gmo' 2024-03-09T09:28:13.497 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/lg.gmo’: No data available 2024-03-09T09:28:13.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/zh_CN.po' -> './k/coreutils-8.5/po/zh_CN.po' 2024-03-09T09:28:13.502 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/zh_CN.po’: No data available 2024-03-09T09:28:13.508 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/cs.gmo' -> './k/coreutils-8.5/po/cs.gmo' 2024-03-09T09:28:13.509 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/cs.gmo’: No data available 2024-03-09T09:28:13.512 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/nb.po' -> './k/coreutils-8.5/po/nb.po' 2024-03-09T09:28:13.512 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/nb.po’: No data available 2024-03-09T09:28:13.514 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/ChangeLog-2007' -> './k/coreutils-8.5/po/ChangeLog-2007' 2024-03-09T09:28:13.514 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/ChangeLog-2007’: No data available 2024-03-09T09:28:13.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/tr.po' -> './k/coreutils-8.5/po/tr.po' 2024-03-09T09:28:13.518 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/tr.po’: No data available 2024-03-09T09:28:13.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/be.gmo' -> './k/coreutils-8.5/po/be.gmo' 2024-03-09T09:28:13.521 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/be.gmo’: No data available 2024-03-09T09:28:13.525 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/fr.po' -> './k/coreutils-8.5/po/fr.po' 2024-03-09T09:28:13.525 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/fr.po’: No data available 2024-03-09T09:28:13.531 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/el.po' -> './k/coreutils-8.5/po/el.po' 2024-03-09T09:28:13.531 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/el.po’: No data available 2024-03-09T09:28:13.535 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/lt.po' -> './k/coreutils-8.5/po/lt.po' 2024-03-09T09:28:13.535 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/lt.po’: No data available 2024-03-09T09:28:13.540 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/es.po' -> './k/coreutils-8.5/po/es.po' 2024-03-09T09:28:13.540 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/es.po’: No data available 2024-03-09T09:28:13.545 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/po/pt.po' -> './k/coreutils-8.5/po/pt.po' 2024-03-09T09:28:13.545 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/po/pt.po’: No data available 2024-03-09T09:28:13.582 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests' -> './k/coreutils-8.5/tests' 2024-03-09T09:28:13.583 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/test-lib.sh' -> './k/coreutils-8.5/tests/test-lib.sh' 2024-03-09T09:28:13.583 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/test-lib.sh’: No data available 2024-03-09T09:28:13.584 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/sample-test' -> './k/coreutils-8.5/tests/sample-test' 2024-03-09T09:28:13.584 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/sample-test’: No data available 2024-03-09T09:28:13.585 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/CuTmpdir.pm' -> './k/coreutils-8.5/tests/CuTmpdir.pm' 2024-03-09T09:28:13.585 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/CuTmpdir.pm’: No data available 2024-03-09T09:28:13.586 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/Makefile.am' -> './k/coreutils-8.5/tests/Makefile.am' 2024-03-09T09:28:13.587 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/Makefile.am’: No data available 2024-03-09T09:28:13.587 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/require-perl' -> './k/coreutils-8.5/tests/require-perl' 2024-03-09T09:28:13.588 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/require-perl’: No data available 2024-03-09T09:28:13.588 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/lang-default' -> './k/coreutils-8.5/tests/lang-default' 2024-03-09T09:28:13.589 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/lang-default’: No data available 2024-03-09T09:28:13.590 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/check.mk' -> './k/coreutils-8.5/tests/check.mk' 2024-03-09T09:28:13.590 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/check.mk’: No data available 2024-03-09T09:28:13.592 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/Makefile.in' -> './k/coreutils-8.5/tests/Makefile.in' 2024-03-09T09:28:13.592 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/Makefile.in’: No data available 2024-03-09T09:28:13.594 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/other-fs-tmpdir' -> './k/coreutils-8.5/tests/other-fs-tmpdir' 2024-03-09T09:28:13.594 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/other-fs-tmpdir’: No data available 2024-03-09T09:28:13.598 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd' -> './k/coreutils-8.5/tests/dd' 2024-03-09T09:28:13.598 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd/unblock-sync' -> './k/coreutils-8.5/tests/dd/unblock-sync' 2024-03-09T09:28:13.598 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/unblock-sync’: No data available 2024-03-09T09:28:13.599 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd/skip-seek2' -> './k/coreutils-8.5/tests/dd/skip-seek2' 2024-03-09T09:28:13.599 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/skip-seek2’: No data available 2024-03-09T09:28:13.600 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd/unblock' -> './k/coreutils-8.5/tests/dd/unblock' 2024-03-09T09:28:13.601 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/unblock’: No data available 2024-03-09T09:28:13.602 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd/stderr' -> './k/coreutils-8.5/tests/dd/stderr' 2024-03-09T09:28:13.602 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/stderr’: No data available 2024-03-09T09:28:13.603 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd/direct' -> './k/coreutils-8.5/tests/dd/direct' 2024-03-09T09:28:13.603 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/direct’: No data available 2024-03-09T09:28:13.604 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd/skip-seek-past-file' -> './k/coreutils-8.5/tests/dd/skip-seek-past-file' 2024-03-09T09:28:13.605 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/skip-seek-past-file’: No data available 2024-03-09T09:28:13.606 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd/not-rewound' -> './k/coreutils-8.5/tests/dd/not-rewound' 2024-03-09T09:28:13.606 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/not-rewound’: No data available 2024-03-09T09:28:13.607 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd/skip-seek-past-dev' -> './k/coreutils-8.5/tests/dd/skip-seek-past-dev' 2024-03-09T09:28:13.607 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/skip-seek-past-dev’: No data available 2024-03-09T09:28:13.608 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd/misc' -> './k/coreutils-8.5/tests/dd/misc' 2024-03-09T09:28:13.608 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/misc’: No data available 2024-03-09T09:28:13.609 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd/reblock' -> './k/coreutils-8.5/tests/dd/reblock' 2024-03-09T09:28:13.609 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/reblock’: No data available 2024-03-09T09:28:13.610 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/dd/skip-seek' -> './k/coreutils-8.5/tests/dd/skip-seek' 2024-03-09T09:28:13.610 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/dd/skip-seek’: No data available 2024-03-09T09:28:13.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.720+0000 7f46ad200640 1 Processor -- start 2024-03-09T09:28:13.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.720+0000 7f46ad200640 1 -- start start 2024-03-09T09:28:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.720+0000 7f46ad200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a814fd90 0x7f46a8156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.720+0000 7f46ad200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46a806c1d0 0x7f46a806c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.720+0000 7f46ad200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a8156bc0 con 0x7f46a80c9380 2024-03-09T09:28:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.720+0000 7f46ad200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a806ba60 con 0x7f46a806c1d0 2024-03-09T09:28:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.720+0000 7f46ad200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80c9890 con 0x7f46a814fd90 2024-03-09T09:28:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.721+0000 7f46a7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46a806c1d0 0x7f46a806c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.721+0000 7f46a7400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a814fd90 0x7f46a8156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.721+0000 7f46a7e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f46a806c1d0 0x7f46a806c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37496/0 (socket says 172.21.15.67:37496) 2024-03-09T09:28:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.721+0000 7f46a7e00640 1 -- 172.21.15.67:0/3175405664 learned_addr learned my addr 172.21.15.67:0/3175405664 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.721+0000 7f46a7e00640 1 -- 172.21.15.67:0/3175405664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a814fd90 msgr2=0x7f46a8156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.721+0000 7f46a7e00640 1 --2- 172.21.15.67:0/3175405664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a814fd90 0x7f46a8156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.721+0000 7f46a7e00640 1 -- 172.21.15.67:0/3175405664 >> v1:172.21.15.134:6789/0 conn(0x7f46a80c9380 legacy=0x7f46a80acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.721+0000 7f46a7e00640 1 -- 172.21.15.67:0/3175405664 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a80c5f80 con 0x7f46a806c1d0 2024-03-09T09:28:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.722+0000 7f46a7e00640 1 --2- 172.21.15.67:0/3175405664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a806c1d0 0x7f46a806c600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009e50 tx=0x7f46a002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38aba103350212f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.723+0000 7f46a6000640 1 -- 172.21.15.67:0/3175405664 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a002c9f0 con 0x7f46a806c1d0 2024-03-09T09:28:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.723+0000 7f46a6000640 1 -- 172.21.15.67:0/3175405664 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a002cb90 con 0x7f46a806c1d0 2024-03-09T09:28:13.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.723+0000 7f46a6000640 1 -- 172.21.15.67:0/3175405664 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a00343f0 con 0x7f46a806c1d0 2024-03-09T09:28:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.723+0000 7f46ad200640 1 -- 172.21.15.67:0/3175405664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a806c1d0 msgr2=0x7f46a806c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.723+0000 7f46ad200640 1 --2- 172.21.15.67:0/3175405664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a806c1d0 0x7f46a806c600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009e50 tx=0x7f46a002be50 comp rx=0 tx=0).stop 2024-03-09T09:28:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.724+0000 7f46ad200640 1 -- 172.21.15.67:0/3175405664 shutdown_connections 2024-03-09T09:28:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.724+0000 7f46ad200640 1 --2- 172.21.15.67:0/3175405664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a806c1d0 0x7f46a806c600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.724+0000 7f46ad200640 1 --2- 172.21.15.67:0/3175405664 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a814fd90 0x7f46a8156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.724+0000 7f46ad200640 1 -- 172.21.15.67:0/3175405664 >> 172.21.15.67:0/3175405664 conn(0x7f46a80cb480 msgr2=0x7f46a80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.724+0000 7f46ad200640 1 -- 172.21.15.67:0/3175405664 shutdown_connections 2024-03-09T09:28:13.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.724+0000 7f46ad200640 1 -- 172.21.15.67:0/3175405664 wait complete. 2024-03-09T09:28:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.726+0000 7f46ad200640 1 Processor -- start 2024-03-09T09:28:13.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.726+0000 7f46ad200640 1 -- start start 2024-03-09T09:28:13.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.727+0000 7f46ad200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a806c1d0 0x7f46a8148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.727+0000 7f46ad200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a80c9380 0x7f46a80ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.727+0000 7f46ad200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a814fd90 0x7f46a80aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.727+0000 7f46ad200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a806bcb0 con 0x7f46a80c9380 2024-03-09T09:28:13.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.727+0000 7f46ad200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80c6520 con 0x7f46a814fd90 2024-03-09T09:28:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.727+0000 7f46ad200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a80ad610 con 0x7f46a806c1d0 2024-03-09T09:28:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.727+0000 7f46a7400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a806c1d0 0x7f46a8148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.727+0000 7f46a6a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a80c9380 0x7f46a80ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.727+0000 7f46a7400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a806c1d0 0x7f46a8148500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:35350/0 (socket says 172.21.15.67:35350) 2024-03-09T09:28:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.727+0000 7f46a7400640 1 -- 172.21.15.67:0/1770025035 learned_addr learned my addr 172.21.15.67:0/1770025035 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.727+0000 7f46a7e00640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a814fd90 0x7f46a80aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.728+0000 7f46a7400640 1 -- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a814fd90 msgr2=0x7f46a80aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.728+0000 7f46a7400640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a814fd90 0x7f46a80aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.728+0000 7f46a7400640 1 -- 172.21.15.67:0/1770025035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a80c9380 msgr2=0x7f46a80ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.728+0000 7f46a7400640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46a80c9380 0x7f46a80ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.728+0000 7f46a7400640 1 -- 172.21.15.67:0/1770025035 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a81542c0 con 0x7f46a806c1d0 2024-03-09T09:28:13.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.728+0000 7f46a7400640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a806c1d0 0x7f46a8148500 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f469800e8a0 tx=0x7f469800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.728+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698017070 con 0x7f46a806c1d0 2024-03-09T09:28:13.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.728+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469800c940 con 0x7f46a806c1d0 2024-03-09T09:28:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.728+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800cd30 con 0x7f46a806c1d0 2024-03-09T09:28:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.729+0000 7f46ad200640 1 -- 172.21.15.67:0/1770025035 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a81ebbd0 con 0x7f46a806c1d0 2024-03-09T09:28:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.729+0000 7f46ad200640 1 -- 172.21.15.67:0/1770025035 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a81ec190 con 0x7f46a806c1d0 2024-03-09T09:28:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.730+0000 7f46ad200640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a81433f0 0x7f46a81437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.730+0000 7f46ad200640 1 -- 172.21.15.67:0/1770025035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a81ec970 con 0x7f46a81433f0 2024-03-09T09:28:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.730+0000 7f46a6a00640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a81433f0 0x7f46a81437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.730+0000 7f46a6a00640 1 -- 172.21.15.67:0/1770025035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46a80c2180 con 0x7f46a81433f0 2024-03-09T09:28:13.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.731+0000 7f46a6a00640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a81433f0 0x7f46a81437d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f46a8148f90 tx=0x7f469400f900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.731+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694019070 con 0x7f46a81433f0 2024-03-09T09:28:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.731+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f4694005320 con 0x7f46a81433f0 2024-03-09T09:28:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.731+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a81433f0 msgr2=0x7f46a81437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.731+0000 7f469f400640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a81433f0 0x7f46a81437d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f46a8148f90 tx=0x7f469400f900 comp rx=0 tx=0).stop 2024-03-09T09:28:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.731+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a81433f0 msgr2=0x7f46a81437d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.731+0000 7f469f400640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a81433f0 0x7f46a81437d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.732+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f469801a070 con 0x7f46a806c1d0 2024-03-09T09:28:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.732+0000 7f469f400640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f468806af40 0x7f468806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.733+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f469808a2d0 con 0x7f46a806c1d0 2024-03-09T09:28:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.733+0000 7f46a6a00640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f468806af40 0x7f468806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.734+0000 7f46a6a00640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f468806af40 0x7f468806d400 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4694013000 tx=0x7f4694004000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.739+0000 7f46ad200640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a8143d10 0x7f46a8152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.739+0000 7f46ad200640 1 -- 172.21.15.67:0/1770025035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a804f3c0 con 0x7f46a8143d10 2024-03-09T09:28:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.740+0000 7f46a7e00640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a8143d10 0x7f46a8152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.740+0000 7f46a7e00640 1 -- 172.21.15.67:0/1770025035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f46a806d2d0 con 0x7f46a8143d10 2024-03-09T09:28:13.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.741+0000 7f46a7e00640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a8143d10 0x7f46a8152290 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f46a80aad00 tx=0x7f46a0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.741+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0009510 con 0x7f46a8143d10 2024-03-09T09:28:13.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5188 (secure 0 0 0) 0x7f46a003f610 con 0x7f46a8143d10 2024-03-09T09:28:13.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a8143d10 msgr2=0x7f46a8152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f469f400640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a8143d10 0x7f46a8152290 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f46a80aad00 tx=0x7f46a0004a40 comp rx=0 tx=0).stop 2024-03-09T09:28:13.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f469f400640 1 -- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a8143d10 msgr2=0x7f46a8152290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:13.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f469f400640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46a8143d10 0x7f46a8152290 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f46a80aad00 tx=0x7f46a0004a40 comp rx=0 tx=0).stop 2024-03-09T09:28:13.749 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 282, 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 18, 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:28:13.750 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:28:13.751 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.752 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:28:13.753 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:13.754 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.756 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.757 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.758 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:13.759 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:28:13.760 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:28:13.761 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:13.762 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.763 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:28:13.763 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:28:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f46ad200640 1 -- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f468806af40 msgr2=0x7f468806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f46ad200640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f468806af40 0x7f468806d400 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4694013000 tx=0x7f4694004000 comp rx=0 tx=0).stop 2024-03-09T09:28:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f46ad200640 1 -- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a806c1d0 msgr2=0x7f46a8148500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f46ad200640 1 --2- 172.21.15.67:0/1770025035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46a806c1d0 0x7f46a8148500 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f469800e8a0 tx=0x7f469800ed60 comp rx=0 tx=0).stop 2024-03-09T09:28:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f46a7e00640 1 -- 172.21.15.67:0/1770025035 reap_dead start 2024-03-09T09:28:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f46ad200640 1 -- 172.21.15.67:0/1770025035 shutdown_connections 2024-03-09T09:28:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f46ad200640 1 -- 172.21.15.67:0/1770025035 >> 172.21.15.67:0/1770025035 conn(0x7f46a80cb480 msgr2=0x7f46a80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f46ad200640 1 -- 172.21.15.67:0/1770025035 shutdown_connections 2024-03-09T09:28:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.742+0000 7f46ad200640 1 -- 172.21.15.67:0/1770025035 wait complete. 2024-03-09T09:28:13.766 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:28:13.766 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:28:13.766 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:28:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.964+0000 7f163c000640 1 Processor -- start 2024-03-09T09:28:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.965+0000 7f163c000640 1 -- start start 2024-03-09T09:28:13.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.966+0000 7f163c000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1634071190 0x7f16341548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.966+0000 7f163c000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340c8200 con 0x7f16340ac9a0 2024-03-09T09:28:13.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.966+0000 7f163c000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340c3990 con 0x7f16340c2780 2024-03-09T09:28:13.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.966+0000 7f163c000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163406d590 con 0x7f1634071190 2024-03-09T09:28:13.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.966+0000 7f1639600640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f16340ac9a0 0x7f16341510e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50712/0 (socket says 172.21.15.67:50712) 2024-03-09T09:28:13.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.966+0000 7f1639600640 1 -- 172.21.15.67:0/84849728 learned_addr learned my addr 172.21.15.67:0/84849728 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f163a000640 1 --2- 172.21.15.67:0/84849728 >> v2:172.21.15.134:3300/0 conn(0x7f1634071190 0x7f16341548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259359051 0 0) 0x7f16340c8200 con 0x7f16340ac9a0 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f161c003520 con 0x7f16340ac9a0 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2077611590 0 0) 0x7f161c003520 con 0x7f16340ac9a0 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 >> v2:172.21.15.134:3300/0 conn(0x7f1634071190 msgr2=0x7f16341548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 --2- 172.21.15.67:0/84849728 >> v2:172.21.15.134:3300/0 conn(0x7f1634071190 0x7f16341548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 >> v1:172.21.15.134:6789/0 conn(0x7f16340c2780 legacy=0x7f1634154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1634070920 con 0x7f16340ac9a0 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f1628001670 con 0x7f16340ac9a0 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 >> v1:172.21.15.67:6789/0 conn(0x7f16340ac9a0 legacy=0x7f16341510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c003a00 0x7f161c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c004800 0x7f161c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.967+0000 7f1633e00640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f161c005210 0x7f161c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c00c130 con 0x7f161c005210 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c00c2e0 con 0x7f161c003a00 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c00c4e0 con 0x7f161c004800 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f163a000640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f161c005210 0x7f161c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f1638c00640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c004800 0x7f161c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f163a000640 1 -- 172.21.15.67:0/84849728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c004800 msgr2=0x7f161c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f1639600640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c003a00 0x7f161c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f163a000640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f161c004800 0x7f161c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f163a000640 1 -- 172.21.15.67:0/84849728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c003a00 msgr2=0x7f161c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f163a000640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c003a00 0x7f161c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f163a000640 1 -- 172.21.15.67:0/84849728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c003520 con 0x7f161c005210 2024-03-09T09:28:13.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.968+0000 7f1639600640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c003a00 0x7f161c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.969+0000 7f163a000640 1 -- 172.21.15.67:0/84849728 reap_dead start 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.969+0000 7f163a000640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f161c005210 0x7f161c00b9c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f162c009f90 tx=0x7f161c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b9d50ba1be2556 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.970+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c009c10 con 0x7f161c005210 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.970+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c02cbf0 con 0x7f161c005210 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.970+0000 7f1633e00640 1 -- 172.21.15.67:0/84849728 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c003990 con 0x7f161c005210 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.970+0000 7f163c000640 1 -- 172.21.15.67:0/84849728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f161c005210 msgr2=0x7f161c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.970+0000 7f163c000640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f161c005210 0x7f161c00b9c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f162c009f90 tx=0x7f161c005830 comp rx=0 tx=0).stop 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.970+0000 7f163c000640 1 -- 172.21.15.67:0/84849728 shutdown_connections 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.970+0000 7f163c000640 1 --2- 172.21.15.67:0/84849728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f161c005210 0x7f161c00b9c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.970+0000 7f163c000640 1 -- 172.21.15.67:0/84849728 >> 172.21.15.67:0/84849728 conn(0x7f16340c7370 msgr2=0x7f16340c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.970+0000 7f163c000640 1 -- 172.21.15.67:0/84849728 shutdown_connections 2024-03-09T09:28:13.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.970+0000 7f163c000640 1 -- 172.21.15.67:0/84849728 wait complete. 2024-03-09T09:28:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.972+0000 7f163c000640 1 Processor -- start 2024-03-09T09:28:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.972+0000 7f163c000640 1 -- start start 2024-03-09T09:28:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.973+0000 7f163c000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c005210 0x7f16341ef690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.973+0000 7f163c000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16340c57a0 0x7f163414c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.973+0000 7f163c000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16341ebe90 0x7f16341ec2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.973+0000 7f163c000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16340acd80 con 0x7f16340c57a0 2024-03-09T09:28:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.973+0000 7f163c000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16340c3740 con 0x7f161c005210 2024-03-09T09:28:13.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.973+0000 7f163c000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16340acf00 con 0x7f16341ebe90 2024-03-09T09:28:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.973+0000 7f1638c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16340c57a0 0x7f163414c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.973+0000 7f1638c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16340c57a0 0x7f163414c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37516/0 (socket says 172.21.15.67:37516) 2024-03-09T09:28:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.973+0000 7f1638c00640 1 -- 172.21.15.67:0/3189284716 learned_addr learned my addr 172.21.15.67:0/3189284716 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.974+0000 7f163a000640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16341ebe90 0x7f16341ec2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.974+0000 7f1639600640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c005210 0x7f16341ef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.974+0000 7f1638c00640 1 -- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16341ebe90 msgr2=0x7f16341ec2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.974+0000 7f1638c00640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16341ebe90 0x7f16341ec2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.974+0000 7f1638c00640 1 -- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c005210 msgr2=0x7f16341ef690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.974+0000 7f1638c00640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f161c005210 0x7f16341ef690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.974+0000 7f1638c00640 1 -- 172.21.15.67:0/3189284716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340ab6d0 con 0x7f16340c57a0 2024-03-09T09:28:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.974+0000 7f1638c00640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16340c57a0 0x7f163414c9b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f162000d4e0 tx=0x7f162000d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.974+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620005480 con 0x7f16340c57a0 2024-03-09T09:28:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.975+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1620017070 con 0x7f16340c57a0 2024-03-09T09:28:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.975+0000 7f163c000640 1 -- 172.21.15.67:0/3189284716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16341efbd0 con 0x7f16340c57a0 2024-03-09T09:28:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.975+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620003830 con 0x7f16340c57a0 2024-03-09T09:28:13.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.975+0000 7f163c000640 1 -- 172.21.15.67:0/3189284716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16340ab110 con 0x7f16340c57a0 2024-03-09T09:28:13.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.976+0000 7f163c000640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340ab2e0 0x7f1634145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.976+0000 7f163c000640 1 -- 172.21.15.67:0/3189284716 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16341eb760 con 0x7f16340ab2e0 2024-03-09T09:28:13.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.976+0000 7f1639600640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340ab2e0 0x7f1634145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.977+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f16200039d0 con 0x7f16340c57a0 2024-03-09T09:28:13.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.977+0000 7f1639600640 1 -- 172.21.15.67:0/3189284716 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1634075a00 con 0x7f16340ab2e0 2024-03-09T09:28:13.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.978+0000 7f1639600640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340ab2e0 0x7f1634145f80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f16340c1890 tx=0x7f162800fb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.979+0000 7f1631600640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f161406b500 0x7f161406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.979+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f162008a2d0 con 0x7f16340c57a0 2024-03-09T09:28:13.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.979+0000 7f163a000640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f161406b500 0x7f161406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.979+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628017020 con 0x7f16340ab2e0 2024-03-09T09:28:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.979+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f162800d720 con 0x7f16340ab2e0 2024-03-09T09:28:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.979+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340ab2e0 msgr2=0x7f1634145f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.979+0000 7f1631600640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340ab2e0 0x7f1634145f80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f16340c1890 tx=0x7f162800fb50 comp rx=0 tx=0).stop 2024-03-09T09:28:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.979+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340ab2e0 msgr2=0x7f1634145f80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.979+0000 7f1631600640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340ab2e0 0x7f1634145f80 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f16340c1890 tx=0x7f162800fb50 comp rx=0 tx=0).stop 2024-03-09T09:28:13.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.980+0000 7f163a000640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f161406b500 0x7f161406d9c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f161c00c2b0 tx=0x7f162c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.986+0000 7f163c000640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16341464c0 0x7f16341468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:13.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.986+0000 7f163c000640 1 -- 172.21.15.67:0/3189284716 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1634146e10 con 0x7f16341464c0 2024-03-09T09:28:13.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.987+0000 7f1639600640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16341464c0 0x7f16341468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:13.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.988+0000 7f1639600640 1 -- 172.21.15.67:0/3189284716 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1634068360 con 0x7f16341464c0 2024-03-09T09:28:13.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.988+0000 7f1639600640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16341464c0 0x7f16341468d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16340c1890 tx=0x7f16280011c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:13.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.989+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628017040 con 0x7f16341464c0 2024-03-09T09:28:13.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.989+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4535 (secure 0 0 0) 0x7f162801e220 con 0x7f16341464c0 2024-03-09T09:28:13.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.989+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16341464c0 msgr2=0x7f16341468d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:13.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.989+0000 7f1631600640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16341464c0 0x7f16341468d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16340c1890 tx=0x7f16280011c0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.989+0000 7f1631600640 1 -- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16341464c0 msgr2=0x7f16341468d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:13.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.989+0000 7f1631600640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16341464c0 0x7f16341468d0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16340c1890 tx=0x7f16280011c0 comp rx=0 tx=0).stop 2024-03-09T09:28:13.996 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:28:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:28:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 282, 2024-03-09T09:28:13.996 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 18, 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:13.997 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:13.998 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:28:13.999 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:14.000 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:14.001 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:14.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:14.003 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:14.004 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:14.005 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:14.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:14.008 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:28:14.008 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:28:14.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.990+0000 7f163c000640 1 -- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f161406b500 msgr2=0x7f161406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:14.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.990+0000 7f163c000640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f161406b500 0x7f161406d9c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f161c00c2b0 tx=0x7f162c02e000 comp rx=0 tx=0).stop 2024-03-09T09:28:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.990+0000 7f163c000640 1 -- 172.21.15.67:0/3189284716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16340c57a0 msgr2=0x7f163414c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.990+0000 7f163c000640 1 --2- 172.21.15.67:0/3189284716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16340c57a0 0x7f163414c9b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f162000d4e0 tx=0x7f162000d9a0 comp rx=0 tx=0).stop 2024-03-09T09:28:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.990+0000 7f163a000640 1 -- 172.21.15.67:0/3189284716 reap_dead start 2024-03-09T09:28:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.990+0000 7f163a000640 1 -- 172.21.15.67:0/3189284716 reap_dead start 2024-03-09T09:28:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.990+0000 7f163c000640 1 -- 172.21.15.67:0/3189284716 shutdown_connections 2024-03-09T09:28:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.990+0000 7f163c000640 1 -- 172.21.15.67:0/3189284716 >> 172.21.15.67:0/3189284716 conn(0x7f16340c7370 msgr2=0x7f163414de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.990+0000 7f163c000640 1 -- 172.21.15.67:0/3189284716 shutdown_connections 2024-03-09T09:28:14.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:13.990+0000 7f163c000640 1 -- 172.21.15.67:0/3189284716 wait complete. 2024-03-09T09:28:14.013 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:28:14.013 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-03-09T09:28:14.013 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-03-09T09:28:14.013 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-03-09T09:28:14.013 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:28:14.013 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-09T09:28:14.013 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:28:14.050 INFO:tasks.ceph.mon.b:Stopped 2024-03-09T09:28:14.050 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-03-09T09:28:14.051 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:28:14.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.256+0000 7f7403000640 1 Processor -- start 2024-03-09T09:28:14.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.257+0000 7f7403000640 1 -- start start 2024-03-09T09:28:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.258+0000 7f7403000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc06d420 0x7f73fc1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:14.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.258+0000 7f7403000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc06d920 con 0x7f73fc0c3080 2024-03-09T09:28:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.258+0000 7f7403000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc06dc00 con 0x7f73fc14c9f0 2024-03-09T09:28:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.258+0000 7f7403000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc155ba0 con 0x7f73fc06d420 2024-03-09T09:28:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.258+0000 7f7402600640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f73fc14c9f0 legacy=0x7f73fc1540a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:28:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.258+0000 7f7401c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f73fc0c3080 0x7f73fc14c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50720/0 (socket says 172.21.15.67:50720) 2024-03-09T09:28:14.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.258+0000 7f7401c00640 1 -- 172.21.15.67:0/1367122863 learned_addr learned my addr 172.21.15.67:0/1367122863 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:28:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.259+0000 7f7401200640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc06d420 0x7f73fc1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.259+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57877879 0 0) 0x7f73fc06d920 con 0x7f73fc0c3080 2024-03-09T09:28:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.259+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e4003520 con 0x7f73fc0c3080 2024-03-09T09:28:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.259+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 997664869 0 0) 0x7f73e4003520 con 0x7f73fc0c3080 2024-03-09T09:28:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.259+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc06d420 msgr2=0x7f73fc1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.259+0000 7f73fbe00640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc06d420 0x7f73fc1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.259+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 >> v1:172.21.15.134:6789/0 conn(0x7f73fc14c9f0 legacy=0x7f73fc1540a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.259+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc06dee0 con 0x7f73fc0c3080 2024-03-09T09:28:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.259+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f73f0001670 con 0x7f73fc0c3080 2024-03-09T09:28:14.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.259+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 >> v1:172.21.15.67:6789/0 conn(0x7f73fc0c3080 legacy=0x7f73fc14c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f73fbe00640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc06d420 0x7f73e4003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f73fbe00640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73e4003fb0 0x7f73e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f73fbe00640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73e4004d40 0x7f73e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc155ba0 con 0x7f73e4004d40 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e400bbb0 con 0x7f73e4003fb0 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e400bdb0 con 0x7f73fc06d420 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f7401200640 1 -- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73e4003fb0 msgr2=0x7f73e4004800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f7401200640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73e4003fb0 0x7f73e4004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f7402600640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73e4004d40 0x7f73e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f7401c00640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc06d420 0x7f73e4003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f7402600640 1 -- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc06d420 msgr2=0x7f73e4003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f7402600640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc06d420 0x7f73e4003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f7402600640 1 -- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73e4003fb0 msgr2=0x7f73e4004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:14.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f7402600640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73e4003fb0 0x7f73e4004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:14.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.260+0000 7f7402600640 1 -- 172.21.15.67:0/1367122863 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e4003520 con 0x7f73e4004d40 2024-03-09T09:28:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.262+0000 7f7402600640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73e4004d40 0x7f73e40051c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f73f4003cb0 tx=0x7f73e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e833d1cda35d1ec server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.262+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4009a50 con 0x7f73e4004d40 2024-03-09T09:28:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.263+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f4030070 con 0x7f73e4004d40 2024-03-09T09:28:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.263+0000 7f73fbe00640 1 -- 172.21.15.67:0/1367122863 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4035470 con 0x7f73e4004d40 2024-03-09T09:28:14.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.263+0000 7f7403000640 1 -- 172.21.15.67:0/1367122863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73e4004d40 msgr2=0x7f73e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.263+0000 7f7403000640 1 --2- 172.21.15.67:0/1367122863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73e4004d40 0x7f73e40051c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f73f4003cb0 tx=0x7f73e4005830 comp rx=0 tx=0).stop 2024-03-09T09:28:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.263+0000 7f7402600640 1 -- 172.21.15.67:0/1367122863 reap_dead start 2024-03-09T09:28:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.263+0000 7f7403000640 1 -- 172.21.15.67:0/1367122863 shutdown_connections 2024-03-09T09:28:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.263+0000 7f7403000640 1 -- 172.21.15.67:0/1367122863 >> 172.21.15.67:0/1367122863 conn(0x7f73fc0c7300 msgr2=0x7f73fc0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.263+0000 7f7403000640 1 -- 172.21.15.67:0/1367122863 shutdown_connections 2024-03-09T09:28:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.263+0000 7f7403000640 1 -- 172.21.15.67:0/1367122863 wait complete. 2024-03-09T09:28:14.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.265+0000 7f7403000640 1 Processor -- start 2024-03-09T09:28:14.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.265+0000 7f7403000640 1 -- start start 2024-03-09T09:28:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.266+0000 7f7403000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73e4004d40 0x7f73fc1eb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.266+0000 7f7403000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73fc0acf40 0x7f73fc0c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.266+0000 7f7403000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc0ab180 0x7f73fc0ab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.266+0000 7f7403000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0c8030 con 0x7f73fc0acf40 2024-03-09T09:28:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.266+0000 7f7403000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0c3460 con 0x7f73e4004d40 2024-03-09T09:28:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.266+0000 7f7403000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc14b260 con 0x7f73fc0ab180 2024-03-09T09:28:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.266+0000 7f7401c00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73e4004d40 msgr2=0x7f73fc1eb240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7401c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73e4004d40 0x7f73fc1eb240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7401200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73fc0acf40 0x7f73fc0c09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7401200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73fc0acf40 0x7f73fc0c09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37532/0 (socket says 172.21.15.67:37532) 2024-03-09T09:28:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7401200640 1 -- 172.21.15.67:0/1238804656 learned_addr learned my addr 172.21.15.67:0/1238804656 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:14.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7402600640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc0ab180 0x7f73fc0ab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7401200640 1 -- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc0ab180 msgr2=0x7f73fc0ab600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7401200640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc0ab180 0x7f73fc0ab600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7401200640 1 -- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73e4004d40 msgr2=0x7f73fc1eb240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7401200640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73e4004d40 0x7f73fc1eb240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7401200640 1 -- 172.21.15.67:0/1238804656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc14a630 con 0x7f73fc0acf40 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7402600640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc0ab180 0x7f73fc0ab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.267+0000 7f7401200640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73fc0acf40 0x7f73fc0c09a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f73e800e900 tx=0x7f73e800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.268+0000 7f73f9600640 1 -- 172.21.15.67:0/1238804656 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8017070 con 0x7f73fc0acf40 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.268+0000 7f73f9600640 1 -- 172.21.15.67:0/1238804656 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e800ca00 con 0x7f73fc0acf40 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.268+0000 7f7403000640 1 -- 172.21.15.67:0/1238804656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc14b4b0 con 0x7f73fc0acf40 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.268+0000 7f73f9600640 1 -- 172.21.15.67:0/1238804656 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e800cdf0 con 0x7f73fc0acf40 2024-03-09T09:28:14.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.268+0000 7f7403000640 1 -- 172.21.15.67:0/1238804656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc0c0ee0 con 0x7f73fc0acf40 2024-03-09T09:28:14.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.269+0000 7f7403000640 1 -- 172.21.15.67:0/1238804656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc04f3c0 con 0x7f73fc0acf40 2024-03-09T09:28:14.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.270+0000 7f73f9600640 1 -- 172.21.15.67:0/1238804656 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f73e801a070 con 0x7f73fc0acf40 2024-03-09T09:28:14.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.271+0000 7f73f9600640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73dc06b480 0x7f73dc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:14.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.271+0000 7f73f9600640 1 -- 172.21.15.67:0/1238804656 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f73e808a9d0 con 0x7f73fc0acf40 2024-03-09T09:28:14.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.272+0000 7f7401c00640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73dc06b480 0x7f73dc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:14.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.273+0000 7f7401c00640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73dc06b480 0x7f73dc06d940 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f73f0001050 tx=0x7f73f0001b80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:14.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.275+0000 7f73f9600640 1 -- 172.21.15.67:0/1238804656 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f73e80593f0 con 0x7f73fc0acf40 2024-03-09T09:28:14.308 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr' -> './k/coreutils-8.5/tests/pr' 2024-03-09T09:28:14.308 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/t_tab_' -> './k/coreutils-8.5/tests/pr/t_tab_' 2024-03-09T09:28:14.308 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t_tab_’: No data available 2024-03-09T09:28:14.310 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/FF' -> './k/coreutils-8.5/tests/pr/FF' 2024-03-09T09:28:14.310 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/FF’: No data available 2024-03-09T09:28:14.311 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/FnFn' -> './k/coreutils-8.5/tests/pr/FnFn' 2024-03-09T09:28:14.312 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/FnFn’: No data available 2024-03-09T09:28:14.314 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/o3ml24f-bl-tn' -> './k/coreutils-8.5/tests/pr/o3ml24f-bl-tn' 2024-03-09T09:28:14.314 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3ml24f-bl-tn’: No data available 2024-03-09T09:28:14.315 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+5-8b3l17f-FF' -> './k/coreutils-8.5/tests/pr/n+5-8b3l17f-FF' 2024-03-09T09:28:14.316 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+5-8b3l17f-FF’: No data available 2024-03-09T09:28:14.317 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3ml24f-bl-tn' -> './k/coreutils-8.5/tests/pr/n+3ml24f-bl-tn' 2024-03-09T09:28:14.317 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3ml24f-bl-tn’: No data available 2024-03-09T09:28:14.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/FFtn' -> './k/coreutils-8.5/tests/pr/FFtn' 2024-03-09T09:28:14.319 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/FFtn’: No data available 2024-03-09T09:28:14.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/a2l15-FF' -> './k/coreutils-8.5/tests/pr/a2l15-FF' 2024-03-09T09:28:14.320 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a2l15-FF’: No data available 2024-03-09T09:28:14.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tFnFt' -> './k/coreutils-8.5/tests/pr/tFnFt' 2024-03-09T09:28:14.322 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFnFt’: No data available 2024-03-09T09:28:14.323 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tn2e5o3-t_tab' -> './k/coreutils-8.5/tests/pr/tn2e5o3-t_tab' 2024-03-09T09:28:14.323 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn2e5o3-t_tab’: No data available 2024-03-09T09:28:14.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3ml24f-t' -> './k/coreutils-8.5/tests/pr/3ml24f-t' 2024-03-09T09:28:14.325 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3ml24f-t’: No data available 2024-03-09T09:28:14.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml20-t-tFFFF' -> './k/coreutils-8.5/tests/pr/nSml20-t-tFFFF' 2024-03-09T09:28:14.327 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml20-t-tFFFF’: No data available 2024-03-09T09:28:14.328 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tn2e8-t_tab' -> './k/coreutils-8.5/tests/pr/tn2e8-t_tab' 2024-03-09T09:28:14.328 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn2e8-t_tab’: No data available 2024-03-09T09:28:14.329 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/b3f-FF' -> './k/coreutils-8.5/tests/pr/b3f-FF' 2024-03-09T09:28:14.330 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3f-FF’: No data available 2024-03-09T09:28:14.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/2w60f-t_notab' -> './k/coreutils-8.5/tests/pr/2w60f-t_notab' 2024-03-09T09:28:14.331 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2w60f-t_notab’: No data available 2024-03-09T09:28:14.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/FFn' -> './k/coreutils-8.5/tests/pr/FFn' 2024-03-09T09:28:14.333 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/FFn’: No data available 2024-03-09T09:28:14.334 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/t_tab' -> './k/coreutils-8.5/tests/pr/t_tab' 2024-03-09T09:28:14.335 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t_tab’: No data available 2024-03-09T09:28:14.336 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/b3-0F' -> './k/coreutils-8.5/tests/pr/b3-0F' 2024-03-09T09:28:14.336 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3-0F’: No data available 2024-03-09T09:28:14.338 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+4-8a2l17-FF' -> './k/coreutils-8.5/tests/pr/n+4-8a2l17-FF' 2024-03-09T09:28:14.338 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+4-8a2l17-FF’: No data available 2024-03-09T09:28:14.340 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+4b2l17f-0FF' -> './k/coreutils-8.5/tests/pr/n+4b2l17f-0FF' 2024-03-09T09:28:14.340 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+4b2l17f-0FF’: No data available 2024-03-09T09:28:14.342 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3a3l15-t' -> './k/coreutils-8.5/tests/pr/3a3l15-t' 2024-03-09T09:28:14.342 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3a3l15-t’: No data available 2024-03-09T09:28:14.344 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tFFn' -> './k/coreutils-8.5/tests/pr/tFFn' 2024-03-09T09:28:14.344 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFFn’: No data available 2024-03-09T09:28:14.346 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3l24f-t' -> './k/coreutils-8.5/tests/pr/3l24f-t' 2024-03-09T09:28:14.346 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3l24f-t’: No data available 2024-03-09T09:28:14.348 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/0FnFt' -> './k/coreutils-8.5/tests/pr/0FnFt' 2024-03-09T09:28:14.348 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0FnFt’: No data available 2024-03-09T09:28:14.350 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3-FF' -> './k/coreutils-8.5/tests/pr/3-FF' 2024-03-09T09:28:14.350 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3-FF’: No data available 2024-03-09T09:28:14.352 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/t-FF' -> './k/coreutils-8.5/tests/pr/t-FF' 2024-03-09T09:28:14.353 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t-FF’: No data available 2024-03-09T09:28:14.355 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml24-bl-FF' -> './k/coreutils-8.5/tests/pr/nSml24-bl-FF' 2024-03-09T09:28:14.355 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml24-bl-FF’: No data available 2024-03-09T09:28:14.357 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3l24f-bl' -> './k/coreutils-8.5/tests/pr/n+3l24f-bl' 2024-03-09T09:28:14.357 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3l24f-bl’: No data available 2024-03-09T09:28:14.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.597+0000 7f7403000640 1 -- 172.21.15.67:0/1238804656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f73fc14bbd0 con 0x7f73fc0acf40 2024-03-09T09:28:14.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.598+0000 7f73f9600640 1 -- 172.21.15.67:0/1238804656 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f73e8058cf0 con 0x7f73fc0acf40 2024-03-09T09:28:14.605 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":282,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:28:14.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.601+0000 7f7403000640 1 -- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73dc06b480 msgr2=0x7f73dc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:14.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.601+0000 7f7403000640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73dc06b480 0x7f73dc06d940 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f73f0001050 tx=0x7f73f0001b80 comp rx=0 tx=0).stop 2024-03-09T09:28:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.601+0000 7f7403000640 1 -- 172.21.15.67:0/1238804656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73fc0acf40 msgr2=0x7f73fc0c09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.601+0000 7f7403000640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73fc0acf40 0x7f73fc0c09a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f73e800e900 tx=0x7f73e800edc0 comp rx=0 tx=0).stop 2024-03-09T09:28:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.602+0000 7f7403000640 1 -- 172.21.15.67:0/1238804656 shutdown_connections 2024-03-09T09:28:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.602+0000 7f7403000640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73dc06b480 0x7f73dc06d940 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.602+0000 7f7403000640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73fc0ab180 0x7f73fc0ab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.602+0000 7f7403000640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f73fc0acf40 0x7f73fc0c09a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.602+0000 7f7403000640 1 --2- 172.21.15.67:0/1238804656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73e4004d40 0x7f73fc1eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.602+0000 7f7403000640 1 -- 172.21.15.67:0/1238804656 >> 172.21.15.67:0/1238804656 conn(0x7f73fc0c7300 msgr2=0x7f73fc0c37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.602+0000 7f7403000640 1 -- 172.21.15.67:0/1238804656 shutdown_connections 2024-03-09T09:28:14.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:14.602+0000 7f7403000640 1 -- 172.21.15.67:0/1238804656 wait complete. 2024-03-09T09:28:14.791 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/b3l15-t' -> './k/coreutils-8.5/tests/pr/b3l15-t' 2024-03-09T09:28:14.792 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3l15-t’: No data available 2024-03-09T09:28:14.793 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3b3f-FF' -> './k/coreutils-8.5/tests/pr/3b3f-FF' 2024-03-09T09:28:14.794 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b3f-FF’: No data available 2024-03-09T09:28:14.798 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3b3f-0F' -> './k/coreutils-8.5/tests/pr/3b3f-0F' 2024-03-09T09:28:14.799 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b3f-0F’: No data available 2024-03-09T09:28:14.800 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W35b3l24f-lm' -> './k/coreutils-8.5/tests/pr/W35b3l24f-lm' 2024-03-09T09:28:14.801 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35b3l24f-lm’: No data available 2024-03-09T09:28:14.802 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/0Ft' -> './k/coreutils-8.5/tests/pr/0Ft' 2024-03-09T09:28:14.802 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0Ft’: No data available 2024-03-09T09:28:14.804 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tt-bl' -> './k/coreutils-8.5/tests/pr/tt-bl' 2024-03-09T09:28:14.804 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tt-bl’: No data available 2024-03-09T09:28:14.806 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3ml24-t-FF' -> './k/coreutils-8.5/tests/pr/3ml24-t-FF' 2024-03-09T09:28:14.806 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3ml24-t-FF’: No data available 2024-03-09T09:28:14.807 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tFFt-lm' -> './k/coreutils-8.5/tests/pr/tFFt-lm' 2024-03-09T09:28:14.808 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFFt-lm’: No data available 2024-03-09T09:28:14.809 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml20-bl-FF' -> './k/coreutils-8.5/tests/pr/nSml20-bl-FF' 2024-03-09T09:28:14.809 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml20-bl-FF’: No data available 2024-03-09T09:28:14.810 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tn2e8o3-t_tab' -> './k/coreutils-8.5/tests/pr/tn2e8o3-t_tab' 2024-03-09T09:28:14.811 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn2e8o3-t_tab’: No data available 2024-03-09T09:28:14.813 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W28l24f-ll' -> './k/coreutils-8.5/tests/pr/W28l24f-ll' 2024-03-09T09:28:14.813 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W28l24f-ll’: No data available 2024-03-09T09:28:14.819 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml24-t-tFFFF' -> './k/coreutils-8.5/tests/pr/nSml24-t-tFFFF' 2024-03-09T09:28:14.819 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml24-t-tFFFF’: No data available 2024-03-09T09:28:14.820 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/t-0FF' -> './k/coreutils-8.5/tests/pr/t-0FF' 2024-03-09T09:28:14.820 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t-0FF’: No data available 2024-03-09T09:28:14.823 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tb3-FF' -> './k/coreutils-8.5/tests/pr/tb3-FF' 2024-03-09T09:28:14.823 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tb3-FF’: No data available 2024-03-09T09:28:14.825 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/o3b3Sl24f-tn' -> './k/coreutils-8.5/tests/pr/o3b3Sl24f-tn' 2024-03-09T09:28:14.825 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3b3Sl24f-tn’: No data available 2024-03-09T09:28:14.826 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/o3mSl24f-bl-tn' -> './k/coreutils-8.5/tests/pr/o3mSl24f-bl-tn' 2024-03-09T09:28:14.826 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3mSl24f-bl-tn’: No data available 2024-03-09T09:28:14.827 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3b2l17-FF' -> './k/coreutils-8.5/tests/pr/3b2l17-FF' 2024-03-09T09:28:14.827 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b2l17-FF’: No data available 2024-03-09T09:28:14.828 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/2f-t_notab' -> './k/coreutils-8.5/tests/pr/2f-t_notab' 2024-03-09T09:28:14.828 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2f-t_notab’: No data available 2024-03-09T09:28:14.830 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/b2l15-FF' -> './k/coreutils-8.5/tests/pr/b2l15-FF' 2024-03-09T09:28:14.830 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b2l15-FF’: No data available 2024-03-09T09:28:14.831 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3b3f-0FF' -> './k/coreutils-8.5/tests/pr/3b3f-0FF' 2024-03-09T09:28:14.831 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b3f-0FF’: No data available 2024-03-09T09:28:14.832 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/a3l15-t' -> './k/coreutils-8.5/tests/pr/a3l15-t' 2024-03-09T09:28:14.832 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3l15-t’: No data available 2024-03-09T09:28:14.835 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/o3Jml24f-lm-lo' -> './k/coreutils-8.5/tests/pr/o3Jml24f-lm-lo' 2024-03-09T09:28:14.835 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3Jml24f-lm-lo’: No data available 2024-03-09T09:28:14.836 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tb3-0FF' -> './k/coreutils-8.5/tests/pr/tb3-0FF' 2024-03-09T09:28:14.836 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tb3-0FF’: No data available 2024-03-09T09:28:14.838 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/l24-t' -> './k/coreutils-8.5/tests/pr/l24-t' 2024-03-09T09:28:14.838 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/l24-t’: No data available 2024-03-09T09:28:14.839 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W35ml24f-lm-lo' -> './k/coreutils-8.5/tests/pr/W35ml24f-lm-lo' 2024-03-09T09:28:14.840 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35ml24f-lm-lo’: No data available 2024-03-09T09:28:14.840 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/a3-0F' -> './k/coreutils-8.5/tests/pr/a3-0F' 2024-03-09T09:28:14.840 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3-0F’: No data available 2024-03-09T09:28:14.842 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24f-0F' -> './k/coreutils-8.5/tests/pr/ml24f-0F' 2024-03-09T09:28:14.842 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24f-0F’: No data available 2024-03-09T09:28:14.843 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/loli' -> './k/coreutils-8.5/tests/pr/loli' 2024-03-09T09:28:14.843 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/loli’: No data available 2024-03-09T09:28:14.845 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3ml24-t' -> './k/coreutils-8.5/tests/pr/3ml24-t' 2024-03-09T09:28:14.845 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3ml24-t’: No data available 2024-03-09T09:28:14.846 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/a3l24f-lm' -> './k/coreutils-8.5/tests/pr/a3l24f-lm' 2024-03-09T09:28:14.846 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3l24f-lm’: No data available 2024-03-09T09:28:14.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/Jb3l24f-lm' -> './k/coreutils-8.5/tests/pr/Jb3l24f-lm' 2024-03-09T09:28:14.847 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/Jb3l24f-lm’: No data available 2024-03-09T09:28:14.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/t-t' -> './k/coreutils-8.5/tests/pr/t-t' 2024-03-09T09:28:14.848 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t-t’: No data available 2024-03-09T09:28:14.849 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/0F' -> './k/coreutils-8.5/tests/pr/0F' 2024-03-09T09:28:14.849 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0F’: No data available 2024-03-09T09:28:14.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/w72l24f-ll' -> './k/coreutils-8.5/tests/pr/w72l24f-ll' 2024-03-09T09:28:14.850 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/w72l24f-ll’: No data available 2024-03-09T09:28:14.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24f-t' -> './k/coreutils-8.5/tests/pr/ml24f-t' 2024-03-09T09:28:14.851 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24f-t’: No data available 2024-03-09T09:28:14.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/b3l15f-t' -> './k/coreutils-8.5/tests/pr/b3l15f-t' 2024-03-09T09:28:14.852 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3l15f-t’: No data available 2024-03-09T09:28:14.854 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/0FnFnt' -> './k/coreutils-8.5/tests/pr/0FnFnt' 2024-03-09T09:28:14.854 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0FnFnt’: No data available 2024-03-09T09:28:14.855 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/b3l24f-lm' -> './k/coreutils-8.5/tests/pr/b3l24f-lm' 2024-03-09T09:28:14.855 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3l24f-lm’: No data available 2024-03-09T09:28:14.856 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/o3mSnl24fbltn' -> './k/coreutils-8.5/tests/pr/o3mSnl24fbltn' 2024-03-09T09:28:14.857 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3mSnl24fbltn’: No data available 2024-03-09T09:28:14.858 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/nJml24f-lmlolm' -> './k/coreutils-8.5/tests/pr/nJml24f-lmlolm' 2024-03-09T09:28:14.858 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nJml24f-lmlolm’: No data available 2024-03-09T09:28:14.859 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/o3a3Sl24f-tn' -> './k/coreutils-8.5/tests/pr/o3a3Sl24f-tn' 2024-03-09T09:28:14.860 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3a3Sl24f-tn’: No data available 2024-03-09T09:28:14.861 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tne8-t_tab' -> './k/coreutils-8.5/tests/pr/tne8-t_tab' 2024-03-09T09:28:14.861 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tne8-t_tab’: No data available 2024-03-09T09:28:14.862 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3ml24-FF' -> './k/coreutils-8.5/tests/pr/3ml24-FF' 2024-03-09T09:28:14.862 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3ml24-FF’: No data available 2024-03-09T09:28:14.863 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/a3f-0FF' -> './k/coreutils-8.5/tests/pr/a3f-0FF' 2024-03-09T09:28:14.863 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3f-0FF’: No data available 2024-03-09T09:28:14.865 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ml20-FF-t' -> './k/coreutils-8.5/tests/pr/ml20-FF-t' 2024-03-09T09:28:14.865 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml20-FF-t’: No data available 2024-03-09T09:28:14.867 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/pr-tests' -> './k/coreutils-8.5/tests/pr/pr-tests' 2024-03-09T09:28:14.867 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/pr-tests’: No data available 2024-03-09T09:28:14.869 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W35Jb3l24f-lm' -> './k/coreutils-8.5/tests/pr/W35Jb3l24f-lm' 2024-03-09T09:28:14.869 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35Jb3l24f-lm’: No data available 2024-03-09T09:28:14.870 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ttb3-0FF' -> './k/coreutils-8.5/tests/pr/ttb3-0FF' 2024-03-09T09:28:14.870 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ttb3-0FF’: No data available 2024-03-09T09:28:14.872 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/nl24f-bl' -> './k/coreutils-8.5/tests/pr/nl24f-bl' 2024-03-09T09:28:14.872 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nl24f-bl’: No data available 2024-03-09T09:28:14.874 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml24-t-t-FF' -> './k/coreutils-8.5/tests/pr/nSml24-t-t-FF' 2024-03-09T09:28:14.874 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml24-t-t-FF’: No data available 2024-03-09T09:28:14.875 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3b3l15f-t' -> './k/coreutils-8.5/tests/pr/3b3l15f-t' 2024-03-09T09:28:14.876 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b3l15f-t’: No data available 2024-03-09T09:28:14.877 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/0FF' -> './k/coreutils-8.5/tests/pr/0FF' 2024-03-09T09:28:14.877 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0FF’: No data available 2024-03-09T09:28:14.879 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3f-0F' -> './k/coreutils-8.5/tests/pr/3f-0F' 2024-03-09T09:28:14.879 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3f-0F’: No data available 2024-03-09T09:28:14.880 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/Ja3l24f-lm' -> './k/coreutils-8.5/tests/pr/Ja3l24f-lm' 2024-03-09T09:28:14.881 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/Ja3l24f-lm’: No data available 2024-03-09T09:28:14.882 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3a2l17-FF' -> './k/coreutils-8.5/tests/pr/3a2l17-FF' 2024-03-09T09:28:14.883 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3a2l17-FF’: No data available 2024-03-09T09:28:14.884 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24f-t-0F' -> './k/coreutils-8.5/tests/pr/ml24f-t-0F' 2024-03-09T09:28:14.884 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24f-t-0F’: No data available 2024-03-09T09:28:14.886 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tn_2e8-t_tab' -> './k/coreutils-8.5/tests/pr/tn_2e8-t_tab' 2024-03-09T09:28:14.886 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn_2e8-t_tab’: No data available 2024-03-09T09:28:14.888 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/o3a3Snl24f-tn' -> './k/coreutils-8.5/tests/pr/o3a3Snl24f-tn' 2024-03-09T09:28:14.888 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3a3Snl24f-tn’: No data available 2024-03-09T09:28:14.889 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/2s_w60f-t_nota' -> './k/coreutils-8.5/tests/pr/2s_w60f-t_nota' 2024-03-09T09:28:14.889 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2s_w60f-t_nota’: No data available 2024-03-09T09:28:14.891 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/t-bl' -> './k/coreutils-8.5/tests/pr/t-bl' 2024-03-09T09:28:14.891 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t-bl’: No data available 2024-03-09T09:28:14.893 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/glthread' -> './k/coreutils-8.5/gnulib-tests/glthread' 2024-03-09T09:28:14.893 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/glthread/yield.h' -> './k/coreutils-8.5/gnulib-tests/glthread/yield.h' 2024-03-09T09:28:14.893 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/glthread/yield.h’: No data available 2024-03-09T09:28:14.893 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24-t-FF' -> './k/coreutils-8.5/tests/pr/ml24-t-FF' 2024-03-09T09:28:14.894 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24-t-FF’: No data available 2024-03-09T09:28:14.895 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/glthread/thread.h' -> './k/coreutils-8.5/gnulib-tests/glthread/thread.h' 2024-03-09T09:28:14.895 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/glthread/thread.h’: No data available 2024-03-09T09:28:14.896 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3-7l24-FF' -> './k/coreutils-8.5/tests/pr/n+3-7l24-FF' 2024-03-09T09:28:14.896 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3-7l24-FF’: No data available 2024-03-09T09:28:14.897 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/glthread/thread.c' -> './k/coreutils-8.5/gnulib-tests/glthread/thread.c' 2024-03-09T09:28:14.898 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/glthread/thread.c’: No data available 2024-03-09T09:28:14.898 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/t_notab' -> './k/coreutils-8.5/tests/pr/t_notab' 2024-03-09T09:28:14.898 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/t_notab’: No data available 2024-03-09T09:28:14.899 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W72Jl24f-ll' -> './k/coreutils-8.5/tests/pr/W72Jl24f-ll' 2024-03-09T09:28:14.899 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W72Jl24f-ll’: No data available 2024-03-09T09:28:14.901 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/a2l17-FF' -> './k/coreutils-8.5/tests/pr/a2l17-FF' 2024-03-09T09:28:14.901 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a2l17-FF’: No data available 2024-03-09T09:28:14.903 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24-FF' -> './k/coreutils-8.5/tests/pr/ml24-FF' 2024-03-09T09:28:14.903 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24-FF’: No data available 2024-03-09T09:28:14.904 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/b3f-0F' -> './k/coreutils-8.5/tests/pr/b3f-0F' 2024-03-09T09:28:14.904 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3f-0F’: No data available 2024-03-09T09:28:14.905 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3l24f-0FF' -> './k/coreutils-8.5/tests/pr/n+3l24f-0FF' 2024-03-09T09:28:14.905 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3l24f-0FF’: No data available 2024-03-09T09:28:14.906 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3ml20f-bl-FF' -> './k/coreutils-8.5/tests/pr/n+3ml20f-bl-FF' 2024-03-09T09:28:14.907 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3ml20f-bl-FF’: No data available 2024-03-09T09:28:14.908 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/a3f-0F' -> './k/coreutils-8.5/tests/pr/a3f-0F' 2024-03-09T09:28:14.908 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3f-0F’: No data available 2024-03-09T09:28:14.909 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W35Jml24f-lmlo' -> './k/coreutils-8.5/tests/pr/W35Jml24f-lmlo' 2024-03-09T09:28:14.909 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35Jml24f-lmlo’: No data available 2024-03-09T09:28:14.911 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W27l24f-ll' -> './k/coreutils-8.5/tests/pr/W27l24f-ll' 2024-03-09T09:28:14.911 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W27l24f-ll’: No data available 2024-03-09T09:28:14.912 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24-t' -> './k/coreutils-8.5/tests/pr/ml24-t' 2024-03-09T09:28:14.912 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24-t’: No data available 2024-03-09T09:28:14.913 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/a3f-FF' -> './k/coreutils-8.5/tests/pr/a3f-FF' 2024-03-09T09:28:14.914 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3f-FF’: No data available 2024-03-09T09:28:14.915 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tFFt-bl' -> './k/coreutils-8.5/tests/pr/tFFt-bl' 2024-03-09T09:28:14.915 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFFt-bl’: No data available 2024-03-09T09:28:14.917 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tta3-FF' -> './k/coreutils-8.5/tests/pr/tta3-FF' 2024-03-09T09:28:14.917 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tta3-FF’: No data available 2024-03-09T09:28:14.919 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3-0F' -> './k/coreutils-8.5/tests/pr/3-0F' 2024-03-09T09:28:14.919 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3-0F’: No data available 2024-03-09T09:28:14.920 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/2sf-t_notab' -> './k/coreutils-8.5/tests/pr/2sf-t_notab' 2024-03-09T09:28:14.920 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2sf-t_notab’: No data available 2024-03-09T09:28:14.922 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3a3l15f-t' -> './k/coreutils-8.5/tests/pr/3a3l15f-t' 2024-03-09T09:28:14.922 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3a3l15f-t’: No data available 2024-03-09T09:28:14.923 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W26l24f-ll' -> './k/coreutils-8.5/tests/pr/W26l24f-ll' 2024-03-09T09:28:14.923 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W26l24f-ll’: No data available 2024-03-09T09:28:14.925 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+7l24-FF' -> './k/coreutils-8.5/tests/pr/n+7l24-FF' 2024-03-09T09:28:14.925 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+7l24-FF’: No data available 2024-03-09T09:28:14.927 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ml24f-lm-lo' -> './k/coreutils-8.5/tests/pr/ml24f-lm-lo' 2024-03-09T09:28:14.927 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ml24f-lm-lo’: No data available 2024-03-09T09:28:14.928 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/l24-FF' -> './k/coreutils-8.5/tests/pr/l24-FF' 2024-03-09T09:28:14.929 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/l24-FF’: No data available 2024-03-09T09:28:14.930 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+2-5l24f-0FF' -> './k/coreutils-8.5/tests/pr/n+2-5l24f-0FF' 2024-03-09T09:28:14.930 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+2-5l24f-0FF’: No data available 2024-03-09T09:28:14.932 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3f-FF' -> './k/coreutils-8.5/tests/pr/3f-FF' 2024-03-09T09:28:14.932 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3f-FF’: No data available 2024-03-09T09:28:14.933 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+3ml24f-tn-bl' -> './k/coreutils-8.5/tests/pr/n+3ml24f-tn-bl' 2024-03-09T09:28:14.933 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+3ml24f-tn-bl’: No data available 2024-03-09T09:28:14.935 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ta3-0FF' -> './k/coreutils-8.5/tests/pr/ta3-0FF' 2024-03-09T09:28:14.935 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ta3-0FF’: No data available 2024-03-09T09:28:14.937 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3b3l15-t' -> './k/coreutils-8.5/tests/pr/3b3l15-t' 2024-03-09T09:28:14.937 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3b3l15-t’: No data available 2024-03-09T09:28:14.938 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+2l24f-0FF' -> './k/coreutils-8.5/tests/pr/n+2l24f-0FF' 2024-03-09T09:28:14.938 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+2l24f-0FF’: No data available 2024-03-09T09:28:14.939 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tn_2e8S-t_tab' -> './k/coreutils-8.5/tests/pr/tn_2e8S-t_tab' 2024-03-09T09:28:14.940 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn_2e8S-t_tab’: No data available 2024-03-09T09:28:14.940 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ta3-FF' -> './k/coreutils-8.5/tests/pr/ta3-FF' 2024-03-09T09:28:14.941 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ta3-FF’: No data available 2024-03-09T09:28:14.942 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tFFt-ll' -> './k/coreutils-8.5/tests/pr/tFFt-ll' 2024-03-09T09:28:14.942 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFFt-ll’: No data available 2024-03-09T09:28:14.943 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+6b3l13f-FF' -> './k/coreutils-8.5/tests/pr/n+6b3l13f-FF' 2024-03-09T09:28:14.943 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+6b3l13f-FF’: No data available 2024-03-09T09:28:14.944 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/2-Sf-t_notab' -> './k/coreutils-8.5/tests/pr/2-Sf-t_notab' 2024-03-09T09:28:14.944 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2-Sf-t_notab’: No data available 2024-03-09T09:28:14.946 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/0FFnt' -> './k/coreutils-8.5/tests/pr/0FFnt' 2024-03-09T09:28:14.946 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0FFnt’: No data available 2024-03-09T09:28:14.947 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W20l24f-ll' -> './k/coreutils-8.5/tests/pr/W20l24f-ll' 2024-03-09T09:28:14.948 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W20l24f-ll’: No data available 2024-03-09T09:28:14.949 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/Jml24f-lm-lo' -> './k/coreutils-8.5/tests/pr/Jml24f-lm-lo' 2024-03-09T09:28:14.949 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/Jml24f-lm-lo’: No data available 2024-03-09T09:28:14.950 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3l24-t' -> './k/coreutils-8.5/tests/pr/3l24-t' 2024-03-09T09:28:14.950 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3l24-t’: No data available 2024-03-09T09:28:14.951 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/o3b3l24f-tn' -> './k/coreutils-8.5/tests/pr/o3b3l24f-tn' 2024-03-09T09:28:14.951 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3b3l24f-tn’: No data available 2024-03-09T09:28:14.952 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W35a3l24f-lm' -> './k/coreutils-8.5/tests/pr/W35a3l24f-lm' 2024-03-09T09:28:14.953 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35a3l24f-lm’: No data available 2024-03-09T09:28:14.954 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/2-S_f-t_notab' -> './k/coreutils-8.5/tests/pr/2-S_f-t_notab' 2024-03-09T09:28:14.954 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2-S_f-t_notab’: No data available 2024-03-09T09:28:14.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3-5l24f-t' -> './k/coreutils-8.5/tests/pr/3-5l24f-t' 2024-03-09T09:28:14.955 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3-5l24f-t’: No data available 2024-03-09T09:28:14.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/o3b3Snl24f-tn' -> './k/coreutils-8.5/tests/pr/o3b3Snl24f-tn' 2024-03-09T09:28:14.957 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3b3Snl24f-tn’: No data available 2024-03-09T09:28:14.958 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+2l24f-bl' -> './k/coreutils-8.5/tests/pr/n+2l24f-bl' 2024-03-09T09:28:14.958 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+2l24f-bl’: No data available 2024-03-09T09:28:14.959 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W35Ja3l24f-lm' -> './k/coreutils-8.5/tests/pr/W35Ja3l24f-lm' 2024-03-09T09:28:14.960 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W35Ja3l24f-lm’: No data available 2024-03-09T09:28:14.960 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+8l20-FF' -> './k/coreutils-8.5/tests/pr/n+8l20-FF' 2024-03-09T09:28:14.961 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+8l20-FF’: No data available 2024-03-09T09:28:14.962 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/0FFt' -> './k/coreutils-8.5/tests/pr/0FFt' 2024-03-09T09:28:14.962 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0FFt’: No data available 2024-03-09T09:28:14.964 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/o3a3l24f-tn' -> './k/coreutils-8.5/tests/pr/o3a3l24f-tn' 2024-03-09T09:28:14.964 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/o3a3l24f-tn’: No data available 2024-03-09T09:28:14.965 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/2s_f-t_notab' -> './k/coreutils-8.5/tests/pr/2s_f-t_notab' 2024-03-09T09:28:14.965 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2s_f-t_notab’: No data available 2024-03-09T09:28:14.967 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/nN15l24f-bl' -> './k/coreutils-8.5/tests/pr/nN15l24f-bl' 2024-03-09T09:28:14.967 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nN15l24f-bl’: No data available 2024-03-09T09:28:14.969 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/0Fnt' -> './k/coreutils-8.5/tests/pr/0Fnt' 2024-03-09T09:28:14.969 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/0Fnt’: No data available 2024-03-09T09:28:14.970 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/4-7l24-FF' -> './k/coreutils-8.5/tests/pr/4-7l24-FF' 2024-03-09T09:28:14.970 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/4-7l24-FF’: No data available 2024-03-09T09:28:14.971 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tn' -> './k/coreutils-8.5/tests/pr/tn' 2024-03-09T09:28:14.972 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tn’: No data available 2024-03-09T09:28:14.972 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/b2l17-FF' -> './k/coreutils-8.5/tests/pr/b2l17-FF' 2024-03-09T09:28:14.973 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b2l17-FF’: No data available 2024-03-09T09:28:14.974 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/nJml24f-lmlmlo' -> './k/coreutils-8.5/tests/pr/nJml24f-lmlmlo' 2024-03-09T09:28:14.974 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nJml24f-lmlmlo’: No data available 2024-03-09T09:28:14.974 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tta3-0FF' -> './k/coreutils-8.5/tests/pr/tta3-0FF' 2024-03-09T09:28:14.975 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tta3-0FF’: No data available 2024-03-09T09:28:14.976 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+5a3l13f-0FF' -> './k/coreutils-8.5/tests/pr/n+5a3l13f-0FF' 2024-03-09T09:28:14.976 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+5a3l13f-0FF’: No data available 2024-03-09T09:28:14.977 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/nN1+3l24f-bl' -> './k/coreutils-8.5/tests/pr/nN1+3l24f-bl' 2024-03-09T09:28:14.977 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nN1+3l24f-bl’: No data available 2024-03-09T09:28:14.978 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/2sw60f-t_notab' -> './k/coreutils-8.5/tests/pr/2sw60f-t_notab' 2024-03-09T09:28:14.979 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/2sw60f-t_notab’: No data available 2024-03-09T09:28:14.979 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/W-72l24f-ll' -> './k/coreutils-8.5/tests/pr/W-72l24f-ll' 2024-03-09T09:28:14.979 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/W-72l24f-ll’: No data available 2024-03-09T09:28:14.980 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tt-0FF' -> './k/coreutils-8.5/tests/pr/tt-0FF' 2024-03-09T09:28:14.981 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tt-0FF’: No data available 2024-03-09T09:28:14.981 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/4l24-FF' -> './k/coreutils-8.5/tests/pr/4l24-FF' 2024-03-09T09:28:14.982 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/4l24-FF’: No data available 2024-03-09T09:28:14.982 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/3a3f-0F' -> './k/coreutils-8.5/tests/pr/3a3f-0F' 2024-03-09T09:28:14.983 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/3a3f-0F’: No data available 2024-03-09T09:28:14.984 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tFFt' -> './k/coreutils-8.5/tests/pr/tFFt' 2024-03-09T09:28:14.984 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tFFt’: No data available 2024-03-09T09:28:14.987 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/n+6a2l17-FF' -> './k/coreutils-8.5/tests/pr/n+6a2l17-FF' 2024-03-09T09:28:14.987 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/n+6a2l17-FF’: No data available 2024-03-09T09:28:15.194 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/a3l15f-t' -> './k/coreutils-8.5/tests/pr/a3l15f-t' 2024-03-09T09:28:15.195 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/a3l15f-t’: No data available 2024-03-09T09:28:15.196 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/l24f-t' -> './k/coreutils-8.5/tests/pr/l24f-t' 2024-03-09T09:28:15.196 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/l24f-t’: No data available 2024-03-09T09:28:15.198 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/b3f-0FF' -> './k/coreutils-8.5/tests/pr/b3f-0FF' 2024-03-09T09:28:15.199 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/b3f-0FF’: No data available 2024-03-09T09:28:15.200 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tt-t' -> './k/coreutils-8.5/tests/pr/tt-t' 2024-03-09T09:28:15.200 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tt-t’: No data available 2024-03-09T09:28:15.202 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/ttb3-FF' -> './k/coreutils-8.5/tests/pr/ttb3-FF' 2024-03-09T09:28:15.203 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/ttb3-FF’: No data available 2024-03-09T09:28:15.205 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/nSml20-t-t-FF' -> './k/coreutils-8.5/tests/pr/nSml20-t-t-FF' 2024-03-09T09:28:15.205 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/nSml20-t-t-FF’: No data available 2024-03-09T09:28:15.207 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tne8o3-t_tab' -> './k/coreutils-8.5/tests/pr/tne8o3-t_tab' 2024-03-09T09:28:15.207 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tne8o3-t_tab’: No data available 2024-03-09T09:28:15.209 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/pr/tt-FF' -> './k/coreutils-8.5/tests/pr/tt-FF' 2024-03-09T09:28:15.209 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/pr/tt-FF’: No data available 2024-03-09T09:28:15.229 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/envvar-check' -> './k/coreutils-8.5/tests/envvar-check' 2024-03-09T09:28:15.230 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/envvar-check’: No data available 2024-03-09T09:28:15.232 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/Coreutils.pm' -> './k/coreutils-8.5/tests/Coreutils.pm' 2024-03-09T09:28:15.232 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/Coreutils.pm’: No data available 2024-03-09T09:28:15.242 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rmdir' -> './k/coreutils-8.5/tests/rmdir' 2024-03-09T09:28:15.242 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rmdir/ignore' -> './k/coreutils-8.5/tests/rmdir/ignore' 2024-03-09T09:28:15.242 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rmdir/ignore’: No data available 2024-03-09T09:28:15.244 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rmdir/fail-perm' -> './k/coreutils-8.5/tests/rmdir/fail-perm' 2024-03-09T09:28:15.245 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rmdir/fail-perm’: No data available 2024-03-09T09:28:15.247 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rmdir/t-slash' -> './k/coreutils-8.5/tests/rmdir/t-slash' 2024-03-09T09:28:15.247 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rmdir/t-slash’: No data available 2024-03-09T09:28:15.278 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm' -> './k/coreutils-8.5/tests/rm' 2024-03-09T09:28:15.278 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/r-2' -> './k/coreutils-8.5/tests/rm/r-2' 2024-03-09T09:28:15.278 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/r-2’: No data available 2024-03-09T09:28:15.280 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/fail-2eperm' -> './k/coreutils-8.5/tests/rm/fail-2eperm' 2024-03-09T09:28:15.280 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/fail-2eperm’: No data available 2024-03-09T09:28:15.282 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/empty-inacc' -> './k/coreutils-8.5/tests/rm/empty-inacc' 2024-03-09T09:28:15.283 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/empty-inacc’: No data available 2024-03-09T09:28:15.285 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/cycle' -> './k/coreutils-8.5/tests/rm/cycle' 2024-03-09T09:28:15.285 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/cycle’: No data available 2024-03-09T09:28:15.287 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/sunos-1' -> './k/coreutils-8.5/tests/rm/sunos-1' 2024-03-09T09:28:15.287 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/sunos-1’: No data available 2024-03-09T09:28:15.289 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/v-slash' -> './k/coreutils-8.5/tests/rm/v-slash' 2024-03-09T09:28:15.289 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/v-slash’: No data available 2024-03-09T09:28:15.291 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/no-give-up' -> './k/coreutils-8.5/tests/rm/no-give-up' 2024-03-09T09:28:15.292 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/no-give-up’: No data available 2024-03-09T09:28:15.294 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/dot-rel' -> './k/coreutils-8.5/tests/rm/dot-rel' 2024-03-09T09:28:15.294 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/dot-rel’: No data available 2024-03-09T09:28:15.296 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/one-file-system' -> './k/coreutils-8.5/tests/rm/one-file-system' 2024-03-09T09:28:15.296 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/one-file-system’: No data available 2024-03-09T09:28:15.298 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/i-never' -> './k/coreutils-8.5/tests/rm/i-never' 2024-03-09T09:28:15.298 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/i-never’: No data available 2024-03-09T09:28:15.300 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/rm1' -> './k/coreutils-8.5/tests/rm/rm1' 2024-03-09T09:28:15.300 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/rm1’: No data available 2024-03-09T09:28:15.302 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/unread2' -> './k/coreutils-8.5/tests/rm/unread2' 2024-03-09T09:28:15.302 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/unread2’: No data available 2024-03-09T09:28:15.304 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/read-only' -> './k/coreutils-8.5/tests/rm/read-only' 2024-03-09T09:28:15.305 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/read-only’: No data available 2024-03-09T09:28:15.306 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/deep-2' -> './k/coreutils-8.5/tests/rm/deep-2' 2024-03-09T09:28:15.307 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/deep-2’: No data available 2024-03-09T09:28:15.309 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/r-4' -> './k/coreutils-8.5/tests/rm/r-4' 2024-03-09T09:28:15.309 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/r-4’: No data available 2024-03-09T09:28:15.311 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/dir-no-w' -> './k/coreutils-8.5/tests/rm/dir-no-w' 2024-03-09T09:28:15.311 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/dir-no-w’: No data available 2024-03-09T09:28:15.313 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/interactive-always' -> './k/coreutils-8.5/tests/rm/interactive-always' 2024-03-09T09:28:15.313 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/interactive-always’: No data available 2024-03-09T09:28:15.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/one-file-system2' -> './k/coreutils-8.5/tests/rm/one-file-system2' 2024-03-09T09:28:15.319 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/one-file-system2’: No data available 2024-03-09T09:28:15.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/unreadable' -> './k/coreutils-8.5/tests/rm/unreadable' 2024-03-09T09:28:15.319 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/unreadable’: No data available 2024-03-09T09:28:15.319 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/r-1' -> './k/coreutils-8.5/tests/rm/r-1' 2024-03-09T09:28:15.320 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/r-1’: No data available 2024-03-09T09:28:15.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/fail-eacces' -> './k/coreutils-8.5/tests/rm/fail-eacces' 2024-03-09T09:28:15.322 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/fail-eacces’: No data available 2024-03-09T09:28:15.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/inaccessible' -> './k/coreutils-8.5/tests/rm/inaccessible' 2024-03-09T09:28:15.324 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/inaccessible’: No data available 2024-03-09T09:28:15.325 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/unread3' -> './k/coreutils-8.5/tests/rm/unread3' 2024-03-09T09:28:15.326 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/unread3’: No data available 2024-03-09T09:28:15.327 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/ir-1' -> './k/coreutils-8.5/tests/rm/ir-1' 2024-03-09T09:28:15.327 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/ir-1’: No data available 2024-03-09T09:28:15.329 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/rm3' -> './k/coreutils-8.5/tests/rm/rm3' 2024-03-09T09:28:15.329 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/rm3’: No data available 2024-03-09T09:28:15.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/rm5' -> './k/coreutils-8.5/tests/rm/rm5' 2024-03-09T09:28:15.331 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/rm5’: No data available 2024-03-09T09:28:15.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/deep-1' -> './k/coreutils-8.5/tests/rm/deep-1' 2024-03-09T09:28:15.333 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/deep-1’: No data available 2024-03-09T09:28:15.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/ext3-perf' -> './k/coreutils-8.5/tests/rm/ext3-perf' 2024-03-09T09:28:15.335 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/ext3-perf’: No data available 2024-03-09T09:28:15.337 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/dangling-symlink' -> './k/coreutils-8.5/tests/rm/dangling-symlink' 2024-03-09T09:28:15.338 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/dangling-symlink’: No data available 2024-03-09T09:28:15.340 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/isatty' -> './k/coreutils-8.5/tests/rm/isatty' 2024-03-09T09:28:15.340 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/isatty’: No data available 2024-03-09T09:28:15.342 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/i-1' -> './k/coreutils-8.5/tests/rm/i-1' 2024-03-09T09:28:15.342 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/i-1’: No data available 2024-03-09T09:28:15.344 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/dir-nonrecur' -> './k/coreutils-8.5/tests/rm/dir-nonrecur' 2024-03-09T09:28:15.344 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/dir-nonrecur’: No data available 2024-03-09T09:28:15.352 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/interactive-once' -> './k/coreutils-8.5/tests/rm/interactive-once' 2024-03-09T09:28:15.352 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/interactive-once’: No data available 2024-03-09T09:28:15.354 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/f-1' -> './k/coreutils-8.5/tests/rm/f-1' 2024-03-09T09:28:15.354 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/f-1’: No data available 2024-03-09T09:28:15.356 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/fail-eperm' -> './k/coreutils-8.5/tests/rm/fail-eperm' 2024-03-09T09:28:15.356 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/fail-eperm’: No data available 2024-03-09T09:28:15.359 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/empty-name' -> './k/coreutils-8.5/tests/rm/empty-name' 2024-03-09T09:28:15.359 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/empty-name’: No data available 2024-03-09T09:28:15.361 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/hash' -> './k/coreutils-8.5/tests/rm/hash' 2024-03-09T09:28:15.361 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/hash’: No data available 2024-03-09T09:28:15.363 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/i-no-r' -> './k/coreutils-8.5/tests/rm/i-no-r' 2024-03-09T09:28:15.363 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/i-no-r’: No data available 2024-03-09T09:28:15.365 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/r-3' -> './k/coreutils-8.5/tests/rm/r-3' 2024-03-09T09:28:15.365 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/r-3’: No data available 2024-03-09T09:28:15.367 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/readdir-bug' -> './k/coreutils-8.5/tests/rm/readdir-bug' 2024-03-09T09:28:15.367 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/readdir-bug’: No data available 2024-03-09T09:28:15.369 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/rm2' -> './k/coreutils-8.5/tests/rm/rm2' 2024-03-09T09:28:15.370 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/rm2’: No data available 2024-03-09T09:28:15.371 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/rm4' -> './k/coreutils-8.5/tests/rm/rm4' 2024-03-09T09:28:15.372 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/rm4’: No data available 2024-03-09T09:28:15.374 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/rm/ignorable' -> './k/coreutils-8.5/tests/rm/ignorable' 2024-03-09T09:28:15.374 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/rm/ignorable’: No data available 2024-03-09T09:28:15.425 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/unistr' -> './k/coreutils-8.5/gnulib-tests/unistr' 2024-03-09T09:28:15.425 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/unistr/test-u8-uctomb.c' -> './k/coreutils-8.5/gnulib-tests/unistr/test-u8-uctomb.c' 2024-03-09T09:28:15.425 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/unistr/test-u8-uctomb.c’: No data available 2024-03-09T09:28:15.700 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/uniwidth' -> './k/coreutils-8.5/gnulib-tests/uniwidth' 2024-03-09T09:28:15.700 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width.c' -> './k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width.c' 2024-03-09T09:28:15.700 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width.c’: No data available 2024-03-09T09:28:15.703 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.sh' -> './k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.sh' 2024-03-09T09:28:15.703 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.sh’: No data available 2024-03-09T09:28:15.706 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.c' -> './k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.c' 2024-03-09T09:28:15.706 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.c’: No data available 2024-03-09T09:28:15.736 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnanf-nolibm.c' -> './k/coreutils-8.5/gnulib-tests/test-isnanf-nolibm.c' 2024-03-09T09:28:15.736 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnanf-nolibm.c’: No data available 2024-03-09T09:28:15.738 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-wcwidth.c' -> './k/coreutils-8.5/gnulib-tests/test-wcwidth.c' 2024-03-09T09:28:15.738 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-wcwidth.c’: No data available 2024-03-09T09:28:15.763 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/locale' -> './k/coreutils-8.5/gnulib-tests/locale' 2024-03-09T09:28:15.763 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/locale/fr' -> './k/coreutils-8.5/gnulib-tests/locale/fr' 2024-03-09T09:28:15.764 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES' -> './k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES' 2024-03-09T09:28:15.764 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.po' -> './k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.po' 2024-03-09T09:28:15.764 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.po’: No data available 2024-03-09T09:28:15.767 INFO:tasks.workunit.client.0.smithi067.stdout:'.snap/k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.mo' -> './k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.mo' 2024-03-09T09:28:15.767 INFO:tasks.workunit.client.0.smithi067.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.mo’: No data available 2024-03-09T09:28:15.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T09:28:15.851 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2024-03-09T09:28:15.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-09T09:28:16.429 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chown' -> './k/coreutils-8.5/tests/chown' 2024-03-09T09:28:16.429 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chown/separator' -> './k/coreutils-8.5/tests/chown/separator' 2024-03-09T09:28:16.429 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chown/separator’: No data available 2024-03-09T09:28:16.431 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chown/deref' -> './k/coreutils-8.5/tests/chown/deref' 2024-03-09T09:28:16.431 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chown/deref’: No data available 2024-03-09T09:28:16.433 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chown/preserve-root' -> './k/coreutils-8.5/tests/chown/preserve-root' 2024-03-09T09:28:16.433 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chown/preserve-root’: No data available 2024-03-09T09:28:16.436 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chown/basic' -> './k/coreutils-8.5/tests/chown/basic' 2024-03-09T09:28:16.436 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chown/basic’: No data available 2024-03-09T09:28:16.468 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/id' -> './k/coreutils-8.5/tests/id' 2024-03-09T09:28:16.468 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/id/no-context' -> './k/coreutils-8.5/tests/id/no-context' 2024-03-09T09:28:16.469 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/id/no-context’: No data available 2024-03-09T09:28:16.493 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp' -> './k/coreutils-8.5/tests/cp' 2024-03-09T09:28:16.493 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/preserve-2' -> './k/coreutils-8.5/tests/cp/preserve-2' 2024-03-09T09:28:16.493 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/preserve-2’: No data available 2024-03-09T09:28:16.495 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/dir-vs-file' -> './k/coreutils-8.5/tests/cp/dir-vs-file' 2024-03-09T09:28:16.495 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/dir-vs-file’: No data available 2024-03-09T09:28:16.498 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/preserve-slink-time' -> './k/coreutils-8.5/tests/cp/preserve-slink-time' 2024-03-09T09:28:16.498 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/preserve-slink-time’: No data available 2024-03-09T09:28:16.500 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/same-file' -> './k/coreutils-8.5/tests/cp/same-file' 2024-03-09T09:28:16.500 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/same-file’: No data available 2024-03-09T09:28:16.502 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/src-base-dot' -> './k/coreutils-8.5/tests/cp/src-base-dot' 2024-03-09T09:28:16.502 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/src-base-dot’: No data available 2024-03-09T09:28:16.504 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-a-selinux' -> './k/coreutils-8.5/tests/cp/cp-a-selinux' 2024-03-09T09:28:16.505 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-a-selinux’: No data available 2024-03-09T09:28:16.507 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/r-vs-symlink' -> './k/coreutils-8.5/tests/cp/r-vs-symlink' 2024-03-09T09:28:16.507 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/r-vs-symlink’: No data available 2024-03-09T09:28:16.509 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/capability' -> './k/coreutils-8.5/tests/cp/capability' 2024-03-09T09:28:16.509 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/capability’: No data available 2024-03-09T09:28:16.511 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/existing-perm-race' -> './k/coreutils-8.5/tests/cp/existing-perm-race' 2024-03-09T09:28:16.511 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/existing-perm-race’: No data available 2024-03-09T09:28:16.514 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/parent-perm-race' -> './k/coreutils-8.5/tests/cp/parent-perm-race' 2024-03-09T09:28:16.514 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/parent-perm-race’: No data available 2024-03-09T09:28:16.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/reflink-auto' -> './k/coreutils-8.5/tests/cp/reflink-auto' 2024-03-09T09:28:16.516 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/reflink-auto’: No data available 2024-03-09T09:28:16.518 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-mv-enotsup-xattr' -> './k/coreutils-8.5/tests/cp/cp-mv-enotsup-xattr' 2024-03-09T09:28:16.518 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-mv-enotsup-xattr’: No data available 2024-03-09T09:28:16.520 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/no-deref-link2' -> './k/coreutils-8.5/tests/cp/no-deref-link2' 2024-03-09T09:28:16.521 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/no-deref-link2’: No data available 2024-03-09T09:28:16.523 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-mv-backup' -> './k/coreutils-8.5/tests/cp/cp-mv-backup' 2024-03-09T09:28:16.523 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-mv-backup’: No data available 2024-03-09T09:28:16.525 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/link-no-deref' -> './k/coreutils-8.5/tests/cp/link-no-deref' 2024-03-09T09:28:16.525 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/link-no-deref’: No data available 2024-03-09T09:28:16.527 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/file-perm-race' -> './k/coreutils-8.5/tests/cp/file-perm-race' 2024-03-09T09:28:16.527 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/file-perm-race’: No data available 2024-03-09T09:28:16.529 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/abuse' -> './k/coreutils-8.5/tests/cp/abuse' 2024-03-09T09:28:16.530 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/abuse’: No data available 2024-03-09T09:28:16.532 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/link-heap' -> './k/coreutils-8.5/tests/cp/link-heap' 2024-03-09T09:28:16.532 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/link-heap’: No data available 2024-03-09T09:28:16.534 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/backup-dir' -> './k/coreutils-8.5/tests/cp/backup-dir' 2024-03-09T09:28:16.535 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/backup-dir’: No data available 2024-03-09T09:28:16.536 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/preserve-gid' -> './k/coreutils-8.5/tests/cp/preserve-gid' 2024-03-09T09:28:16.537 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/preserve-gid’: No data available 2024-03-09T09:28:16.539 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/link' -> './k/coreutils-8.5/tests/cp/link' 2024-03-09T09:28:16.539 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/link’: No data available 2024-03-09T09:28:16.541 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-parents' -> './k/coreutils-8.5/tests/cp/cp-parents' 2024-03-09T09:28:16.541 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-parents’: No data available 2024-03-09T09:28:16.543 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/no-deref-link1' -> './k/coreutils-8.5/tests/cp/no-deref-link1' 2024-03-09T09:28:16.543 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/no-deref-link1’: No data available 2024-03-09T09:28:16.545 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/into-self' -> './k/coreutils-8.5/tests/cp/into-self' 2024-03-09T09:28:16.545 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/into-self’: No data available 2024-03-09T09:28:16.547 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/parent-perm' -> './k/coreutils-8.5/tests/cp/parent-perm' 2024-03-09T09:28:16.548 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/parent-perm’: No data available 2024-03-09T09:28:16.550 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-deref' -> './k/coreutils-8.5/tests/cp/cp-deref' 2024-03-09T09:28:16.550 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-deref’: No data available 2024-03-09T09:28:16.552 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/proc-short-read' -> './k/coreutils-8.5/tests/cp/proc-short-read' 2024-03-09T09:28:16.552 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/proc-short-read’: No data available 2024-03-09T09:28:16.554 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/backup-is-src' -> './k/coreutils-8.5/tests/cp/backup-is-src' 2024-03-09T09:28:16.554 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/backup-is-src’: No data available 2024-03-09T09:28:16.556 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-i' -> './k/coreutils-8.5/tests/cp/cp-i' 2024-03-09T09:28:16.557 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-i’: No data available 2024-03-09T09:28:16.559 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/special-f' -> './k/coreutils-8.5/tests/cp/special-f' 2024-03-09T09:28:16.559 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/special-f’: No data available 2024-03-09T09:28:16.561 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/proc-zero-len' -> './k/coreutils-8.5/tests/cp/proc-zero-len' 2024-03-09T09:28:16.561 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/proc-zero-len’: No data available 2024-03-09T09:28:16.563 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/slink-2-slink' -> './k/coreutils-8.5/tests/cp/slink-2-slink' 2024-03-09T09:28:16.563 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/slink-2-slink’: No data available 2024-03-09T09:28:16.565 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/special-bits' -> './k/coreutils-8.5/tests/cp/special-bits' 2024-03-09T09:28:16.565 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/special-bits’: No data available 2024-03-09T09:28:16.567 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/fail-perm' -> './k/coreutils-8.5/tests/cp/fail-perm' 2024-03-09T09:28:16.567 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/fail-perm’: No data available 2024-03-09T09:28:16.569 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/link-preserve' -> './k/coreutils-8.5/tests/cp/link-preserve' 2024-03-09T09:28:16.570 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/link-preserve’: No data available 2024-03-09T09:28:16.571 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/dir-rm-dest' -> './k/coreutils-8.5/tests/cp/dir-rm-dest' 2024-03-09T09:28:16.571 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/dir-rm-dest’: No data available 2024-03-09T09:28:16.573 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/backup-1' -> './k/coreutils-8.5/tests/cp/backup-1' 2024-03-09T09:28:16.574 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/backup-1’: No data available 2024-03-09T09:28:16.575 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/cp-HL' -> './k/coreutils-8.5/tests/cp/cp-HL' 2024-03-09T09:28:16.576 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/cp-HL’: No data available 2024-03-09T09:28:16.578 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/thru-dangling' -> './k/coreutils-8.5/tests/cp/thru-dangling' 2024-03-09T09:28:16.578 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/thru-dangling’: No data available 2024-03-09T09:28:16.580 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/sparse' -> './k/coreutils-8.5/tests/cp/sparse' 2024-03-09T09:28:16.580 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/sparse’: No data available 2024-03-09T09:28:16.582 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/deref-slink' -> './k/coreutils-8.5/tests/cp/deref-slink' 2024-03-09T09:28:16.582 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/deref-slink’: No data available 2024-03-09T09:28:16.584 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/symlink-slash' -> './k/coreutils-8.5/tests/cp/symlink-slash' 2024-03-09T09:28:16.584 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/symlink-slash’: No data available 2024-03-09T09:28:16.586 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/acl' -> './k/coreutils-8.5/tests/cp/acl' 2024-03-09T09:28:16.587 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/acl’: No data available 2024-03-09T09:28:16.588 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/perm' -> './k/coreutils-8.5/tests/cp/perm' 2024-03-09T09:28:16.589 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/perm’: No data available 2024-03-09T09:28:16.590 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/reflink-perm' -> './k/coreutils-8.5/tests/cp/reflink-perm' 2024-03-09T09:28:16.590 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/reflink-perm’: No data available 2024-03-09T09:28:16.592 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/no-deref-link3' -> './k/coreutils-8.5/tests/cp/no-deref-link3' 2024-03-09T09:28:16.592 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/no-deref-link3’: No data available 2024-03-09T09:28:16.595 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/cp/dir-slash' -> './k/coreutils-8.5/tests/cp/dir-slash' 2024-03-09T09:28:16.595 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/cp/dir-slash’: No data available 2024-03-09T09:28:16.847 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir' -> './k/coreutils-8.5/tests/mkdir' 2024-03-09T09:28:16.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-v' -> './k/coreutils-8.5/tests/mkdir/p-v' 2024-03-09T09:28:16.848 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-v’: No data available 2024-03-09T09:28:16.850 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-3' -> './k/coreutils-8.5/tests/mkdir/p-3' 2024-03-09T09:28:16.850 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-3’: No data available 2024-03-09T09:28:16.852 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/writable-under-readonly' -> './k/coreutils-8.5/tests/mkdir/writable-under-readonly' 2024-03-09T09:28:16.852 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/writable-under-readonly’: No data available 2024-03-09T09:28:16.854 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-slashdot' -> './k/coreutils-8.5/tests/mkdir/p-slashdot' 2024-03-09T09:28:16.854 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-slashdot’: No data available 2024-03-09T09:28:16.857 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/selinux' -> './k/coreutils-8.5/tests/mkdir/selinux' 2024-03-09T09:28:16.857 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/selinux’: No data available 2024-03-09T09:28:16.859 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-1' -> './k/coreutils-8.5/tests/mkdir/p-1' 2024-03-09T09:28:16.860 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-1’: No data available 2024-03-09T09:28:16.862 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-2' -> './k/coreutils-8.5/tests/mkdir/p-2' 2024-03-09T09:28:16.862 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-2’: No data available 2024-03-09T09:28:16.864 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/parents' -> './k/coreutils-8.5/tests/mkdir/parents' 2024-03-09T09:28:16.864 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/parents’: No data available 2024-03-09T09:28:16.866 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/special-1' -> './k/coreutils-8.5/tests/mkdir/special-1' 2024-03-09T09:28:16.867 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/special-1’: No data available 2024-03-09T09:28:16.869 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/perm' -> './k/coreutils-8.5/tests/mkdir/perm' 2024-03-09T09:28:16.869 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/perm’: No data available 2024-03-09T09:28:16.872 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/p-thru-slink' -> './k/coreutils-8.5/tests/mkdir/p-thru-slink' 2024-03-09T09:28:16.872 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/p-thru-slink’: No data available 2024-03-09T09:28:16.874 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mkdir/t-slash' -> './k/coreutils-8.5/tests/mkdir/t-slash' 2024-03-09T09:28:16.874 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mkdir/t-slash’: No data available 2024-03-09T09:28:16.907 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ln' -> './k/coreutils-8.5/tests/ln' 2024-03-09T09:28:16.907 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ln/hard-to-sym' -> './k/coreutils-8.5/tests/ln/hard-to-sym' 2024-03-09T09:28:16.907 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/hard-to-sym’: No data available 2024-03-09T09:28:16.909 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ln/hard-backup' -> './k/coreutils-8.5/tests/ln/hard-backup' 2024-03-09T09:28:16.909 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/hard-backup’: No data available 2024-03-09T09:28:16.911 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ln/sf-1' -> './k/coreutils-8.5/tests/ln/sf-1' 2024-03-09T09:28:16.911 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/sf-1’: No data available 2024-03-09T09:28:16.913 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ln/backup-1' -> './k/coreutils-8.5/tests/ln/backup-1' 2024-03-09T09:28:16.913 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/backup-1’: No data available 2024-03-09T09:28:16.915 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ln/slash-decorated-nonexistent-dest' -> './k/coreutils-8.5/tests/ln/slash-decorated-nonexistent-dest' 2024-03-09T09:28:16.915 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/slash-decorated-nonexistent-dest’: No data available 2024-03-09T09:28:16.916 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ln/misc' -> './k/coreutils-8.5/tests/ln/misc' 2024-03-09T09:28:16.917 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/misc’: No data available 2024-03-09T09:28:16.918 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ln/target-1' -> './k/coreutils-8.5/tests/ln/target-1' 2024-03-09T09:28:16.918 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ln/target-1’: No data available 2024-03-09T09:28:16.986 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi067 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' 2024-03-09T09:28:17.226 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9229f5ab0faa4aaba8f4efbf6f4a4b0f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi067 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' 2024-03-09T09:28:17.229 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-03-09T09:28:17.240 INFO:tasks.mon_thrash:joining mon_thrasher 2024-03-09T09:28:17.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:28:17.790 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/install' -> './k/coreutils-8.5/tests/install' 2024-03-09T09:28:17.791 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/install/install-C-selinux' -> './k/coreutils-8.5/tests/install/install-C-selinux' 2024-03-09T09:28:17.791 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/install-C-selinux’: No data available 2024-03-09T09:28:17.792 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/install/trap' -> './k/coreutils-8.5/tests/install/trap' 2024-03-09T09:28:17.792 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/trap’: No data available 2024-03-09T09:28:17.795 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/install/basic-1' -> './k/coreutils-8.5/tests/install/basic-1' 2024-03-09T09:28:17.795 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/basic-1’: No data available 2024-03-09T09:28:17.797 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/install/strip-program' -> './k/coreutils-8.5/tests/install/strip-program' 2024-03-09T09:28:17.797 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/strip-program’: No data available 2024-03-09T09:28:17.799 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/install/d-slashdot' -> './k/coreutils-8.5/tests/install/d-slashdot' 2024-03-09T09:28:17.799 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/d-slashdot’: No data available 2024-03-09T09:28:17.802 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/install/install-C' -> './k/coreutils-8.5/tests/install/install-C' 2024-03-09T09:28:17.802 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/install-C’: No data available 2024-03-09T09:28:17.804 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/install/install-C-root' -> './k/coreutils-8.5/tests/install/install-C-root' 2024-03-09T09:28:17.804 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/install-C-root’: No data available 2024-03-09T09:28:17.806 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/install/create-leading' -> './k/coreutils-8.5/tests/install/create-leading' 2024-03-09T09:28:17.806 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/install/create-leading’: No data available 2024-03-09T09:28:17.842 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod' -> './k/coreutils-8.5/tests/chmod' 2024-03-09T09:28:17.842 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod/octal' -> './k/coreutils-8.5/tests/chmod/octal' 2024-03-09T09:28:17.842 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/octal’: No data available 2024-03-09T09:28:17.844 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod/umask-x' -> './k/coreutils-8.5/tests/chmod/umask-x' 2024-03-09T09:28:17.844 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/umask-x’: No data available 2024-03-09T09:28:17.846 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod/inaccessible' -> './k/coreutils-8.5/tests/chmod/inaccessible' 2024-03-09T09:28:17.846 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/inaccessible’: No data available 2024-03-09T09:28:17.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod/equal-x' -> './k/coreutils-8.5/tests/chmod/equal-x' 2024-03-09T09:28:17.849 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/equal-x’: No data available 2024-03-09T09:28:17.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod/silent' -> './k/coreutils-8.5/tests/chmod/silent' 2024-03-09T09:28:17.851 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/silent’: No data available 2024-03-09T09:28:17.854 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod/equals' -> './k/coreutils-8.5/tests/chmod/equals' 2024-03-09T09:28:17.854 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/equals’: No data available 2024-03-09T09:28:17.856 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod/usage' -> './k/coreutils-8.5/tests/chmod/usage' 2024-03-09T09:28:17.856 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/usage’: No data available 2024-03-09T09:28:17.858 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod/thru-dangling' -> './k/coreutils-8.5/tests/chmod/thru-dangling' 2024-03-09T09:28:17.858 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/thru-dangling’: No data available 2024-03-09T09:28:17.860 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod/setgid' -> './k/coreutils-8.5/tests/chmod/setgid' 2024-03-09T09:28:17.861 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/setgid’: No data available 2024-03-09T09:28:17.862 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod/no-x' -> './k/coreutils-8.5/tests/chmod/no-x' 2024-03-09T09:28:17.863 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/no-x’: No data available 2024-03-09T09:28:17.865 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chmod/c-option' -> './k/coreutils-8.5/tests/chmod/c-option' 2024-03-09T09:28:17.865 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chmod/c-option’: No data available 2024-03-09T09:28:17.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.875+0000 7f0f93200640 1 Processor -- start 2024-03-09T09:28:17.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.875+0000 7f0f93200640 1 -- start start 2024-03-09T09:28:17.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.877+0000 7f0f93200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f94151050 0x7f0f9406f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.877+0000 7f0f93200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0f9406e640 0x7f0f940c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.877+0000 7f0f93200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f940c4220 0x7f0f94153e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.877+0000 7f0f93200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f940c2d80 con 0x7f0f940c4220 2024-03-09T09:28:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.877+0000 7f0f93200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f94156f30 con 0x7f0f9406e640 2024-03-09T09:28:17.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.877+0000 7f0f93200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94153940 con 0x7f0f94151050 2024-03-09T09:28:17.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.877+0000 7f0f91400640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f0f9406e640 msgr2=0x7f0f940c3ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:17.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.877+0000 7f0f91400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0f9406e640 0x7f0f940c3ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:17.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.877+0000 7f0f92800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f940c4220 0x7f0f94153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:17.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.878+0000 7f0f92800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f940c4220 0x7f0f94153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37542/0 (socket says 172.21.15.67:37542) 2024-03-09T09:28:17.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.878+0000 7f0f92800640 1 -- 172.21.15.67:0/1770595777 learned_addr learned my addr 172.21.15.67:0/1770595777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:17.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.878+0000 7f0f91e00640 1 --2- 172.21.15.67:0/1770595777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f94151050 0x7f0f9406f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:17.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.878+0000 7f0f92800640 1 -- 172.21.15.67:0/1770595777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f94151050 msgr2=0x7f0f9406f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:17.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.878+0000 7f0f92800640 1 --2- 172.21.15.67:0/1770595777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f94151050 0x7f0f9406f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:17.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.878+0000 7f0f92800640 1 -- 172.21.15.67:0/1770595777 >> v2:172.21.15.134:3300/0 conn(0x7f0f9406e640 msgr2=0x7f0f940c3ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:17.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.878+0000 7f0f92800640 1 --2- 172.21.15.67:0/1770595777 >> v2:172.21.15.134:3300/0 conn(0x7f0f9406e640 0x7f0f940c3ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:17.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.878+0000 7f0f92800640 1 -- 172.21.15.67:0/1770595777 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f940c2fd0 con 0x7f0f940c4220 2024-03-09T09:28:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.880+0000 7f0f92800640 1 --2- 172.21.15.67:0/1770595777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f940c4220 0x7f0f94153e40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009870 tx=0x7f0f8402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4f28e3e4cf30b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:17.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.880+0000 7f0f90a00640 1 -- 172.21.15.67:0/1770595777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8402e070 con 0x7f0f940c4220 2024-03-09T09:28:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.880+0000 7f0f90a00640 1 -- 172.21.15.67:0/1770595777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84037070 con 0x7f0f940c4220 2024-03-09T09:28:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.880+0000 7f0f90a00640 1 -- 172.21.15.67:0/1770595777 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84031a00 con 0x7f0f940c4220 2024-03-09T09:28:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.881+0000 7f0f93200640 1 -- 172.21.15.67:0/1770595777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f940c4220 msgr2=0x7f0f94153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.881+0000 7f0f93200640 1 --2- 172.21.15.67:0/1770595777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f940c4220 0x7f0f94153e40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009870 tx=0x7f0f8402b880 comp rx=0 tx=0).stop 2024-03-09T09:28:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.881+0000 7f0f93200640 1 -- 172.21.15.67:0/1770595777 shutdown_connections 2024-03-09T09:28:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.881+0000 7f0f93200640 1 --2- 172.21.15.67:0/1770595777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f940c4220 0x7f0f94153e40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.881+0000 7f0f93200640 1 --2- 172.21.15.67:0/1770595777 >> v2:172.21.15.134:3300/0 conn(0x7f0f9406e640 0x7f0f940c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:17.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.881+0000 7f0f93200640 1 --2- 172.21.15.67:0/1770595777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f94151050 0x7f0f9406f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.881+0000 7f0f93200640 1 -- 172.21.15.67:0/1770595777 >> 172.21.15.67:0/1770595777 conn(0x7f0f9406c150 msgr2=0x7f0f9406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.881+0000 7f0f93200640 1 -- 172.21.15.67:0/1770595777 shutdown_connections 2024-03-09T09:28:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.881+0000 7f0f93200640 1 -- 172.21.15.67:0/1770595777 wait complete. 2024-03-09T09:28:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.883+0000 7f0f93200640 1 Processor -- start 2024-03-09T09:28:17.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.883+0000 7f0f93200640 1 -- start start 2024-03-09T09:28:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.884+0000 7f0f93200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9406e640 0x7f0f9414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.884+0000 7f0f93200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f940c4220 0x7f0f940adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.884+0000 7f0f93200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f94151050 0x7f0f94156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.884+0000 7f0f93200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940c3870 con 0x7f0f9406e640 2024-03-09T09:28:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.884+0000 7f0f93200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940c3590 con 0x7f0f94151050 2024-03-09T09:28:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.884+0000 7f0f93200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94154380 con 0x7f0f940c4220 2024-03-09T09:28:17.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.884+0000 7f0f91e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9406e640 0x7f0f9414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:17.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.884+0000 7f0f92800640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f94151050 msgr2=0x7f0f94156000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:17.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.884+0000 7f0f92800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f94151050 0x7f0f94156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:17.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.885+0000 7f0f91e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9406e640 0x7f0f9414bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37556/0 (socket says 172.21.15.67:37556) 2024-03-09T09:28:17.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.885+0000 7f0f91e00640 1 -- 172.21.15.67:0/1332317663 learned_addr learned my addr 172.21.15.67:0/1332317663 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:17.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.885+0000 7f0f91400640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f940c4220 0x7f0f940adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:17.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.885+0000 7f0f91e00640 1 -- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f940c4220 msgr2=0x7f0f940adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:17.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.885+0000 7f0f91e00640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f940c4220 0x7f0f940adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:17.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.885+0000 7f0f91e00640 1 -- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f94151050 msgr2=0x7f0f94156000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:28:17.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.885+0000 7f0f91e00640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f94151050 0x7f0f94156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:17.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.885+0000 7f0f91e00640 1 -- 172.21.15.67:0/1332317663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f941ed5f0 con 0x7f0f9406e640 2024-03-09T09:28:17.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.885+0000 7f0f91e00640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9406e640 0x7f0f9414bcc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c007e40 tx=0x7f0f8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:17.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.886+0000 7f0f8a000640 1 -- 172.21.15.67:0/1332317663 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c014620 con 0x7f0f9406e640 2024-03-09T09:28:17.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.886+0000 7f0f8a000640 1 -- 172.21.15.67:0/1332317663 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c019070 con 0x7f0f9406e640 2024-03-09T09:28:17.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.886+0000 7f0f8a000640 1 -- 172.21.15.67:0/1332317663 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c014a00 con 0x7f0f9406e640 2024-03-09T09:28:17.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.886+0000 7f0f93200640 1 -- 172.21.15.67:0/1332317663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f940c7d80 con 0x7f0f9406e640 2024-03-09T09:28:17.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.886+0000 7f0f93200640 1 -- 172.21.15.67:0/1332317663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f940c8060 con 0x7f0f9406e640 2024-03-09T09:28:17.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.887+0000 7f0f93200640 1 -- 172.21.15.67:0/1332317663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9404f3c0 con 0x7f0f9406e640 2024-03-09T09:28:17.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.888+0000 7f0f8a000640 1 -- 172.21.15.67:0/1332317663 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0f8c014ba0 con 0x7f0f9406e640 2024-03-09T09:28:17.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.889+0000 7f0f8a000640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f7006b480 0x7f0f7006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.889+0000 7f0f8a000640 1 -- 172.21.15.67:0/1332317663 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f0f8c08ae70 con 0x7f0f9406e640 2024-03-09T09:28:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.890+0000 7f0f91400640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f7006b480 0x7f0f7006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:17.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.891+0000 7f0f91400640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f7006b480 0x7f0f7006d940 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0f940c7610 tx=0x7f0f7c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:17.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:17.894+0000 7f0f8a000640 1 -- 172.21.15.67:0/1332317663 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0f8c059890 con 0x7f0f9406e640 2024-03-09T09:28:18.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.181+0000 7f0f93200640 1 -- 172.21.15.67:0/1332317663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0f94073e20 con 0x7f0f9406e640 2024-03-09T09:28:18.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.181+0000 7f0f8a000640 1 -- 172.21.15.67:0/1332317663 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f0f8c059190 con 0x7f0f9406e640 2024-03-09T09:28:18.189 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":282,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:28:18.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 -- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f7006b480 msgr2=0x7f0f7006d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:18.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f7006b480 0x7f0f7006d940 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0f940c7610 tx=0x7f0f7c008040 comp rx=0 tx=0).stop 2024-03-09T09:28:18.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 -- 172.21.15.67:0/1332317663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9406e640 msgr2=0x7f0f9414bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:18.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9406e640 0x7f0f9414bcc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c007e40 tx=0x7f0f8c00e4b0 comp rx=0 tx=0).stop 2024-03-09T09:28:18.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 -- 172.21.15.67:0/1332317663 shutdown_connections 2024-03-09T09:28:18.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f7006b480 0x7f0f7006d940 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:18.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f94151050 0x7f0f94156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:18.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f940c4220 0x7f0f940adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:18.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 --2- 172.21.15.67:0/1332317663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f9406e640 0x7f0f9414bcc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:18.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 -- 172.21.15.67:0/1332317663 >> 172.21.15.67:0/1332317663 conn(0x7f0f9406c150 msgr2=0x7f0f9406c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:18.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 -- 172.21.15.67:0/1332317663 shutdown_connections 2024-03-09T09:28:18.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:18.184+0000 7f0f93200640 1 -- 172.21.15.67:0/1332317663 wait complete. 2024-03-09T09:28:18.838 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/df' -> './k/coreutils-8.5/tests/df' 2024-03-09T09:28:18.838 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/df/total-verify' -> './k/coreutils-8.5/tests/df/total-verify' 2024-03-09T09:28:18.839 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/df/total-verify’: No data available 2024-03-09T09:28:18.839 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/df/unreadable' -> './k/coreutils-8.5/tests/df/unreadable' 2024-03-09T09:28:18.839 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/df/unreadable’: No data available 2024-03-09T09:28:18.868 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du' -> './k/coreutils-8.5/tests/du' 2024-03-09T09:28:18.868 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/exclude' -> './k/coreutils-8.5/tests/du/exclude' 2024-03-09T09:28:18.869 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/exclude’: No data available 2024-03-09T09:28:18.870 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/one-file-system' -> './k/coreutils-8.5/tests/du/one-file-system' 2024-03-09T09:28:18.871 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/one-file-system’: No data available 2024-03-09T09:28:18.872 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/long-sloop' -> './k/coreutils-8.5/tests/du/long-sloop' 2024-03-09T09:28:18.872 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/long-sloop’: No data available 2024-03-09T09:28:18.873 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/slink' -> './k/coreutils-8.5/tests/du/slink' 2024-03-09T09:28:18.874 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/slink’: No data available 2024-03-09T09:28:18.875 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/inacc-dir' -> './k/coreutils-8.5/tests/du/inacc-dir' 2024-03-09T09:28:18.875 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/inacc-dir’: No data available 2024-03-09T09:28:18.877 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/inaccessible-cwd' -> './k/coreutils-8.5/tests/du/inaccessible-cwd' 2024-03-09T09:28:18.877 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/inaccessible-cwd’: No data available 2024-03-09T09:28:18.878 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/inacc-dest' -> './k/coreutils-8.5/tests/du/inacc-dest' 2024-03-09T09:28:18.878 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/inacc-dest’: No data available 2024-03-09T09:28:18.880 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/deref-args' -> './k/coreutils-8.5/tests/du/deref-args' 2024-03-09T09:28:18.880 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/deref-args’: No data available 2024-03-09T09:28:18.881 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/deref' -> './k/coreutils-8.5/tests/du/deref' 2024-03-09T09:28:18.881 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/deref’: No data available 2024-03-09T09:28:18.883 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/trailing-slash' -> './k/coreutils-8.5/tests/du/trailing-slash' 2024-03-09T09:28:18.883 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/trailing-slash’: No data available 2024-03-09T09:28:18.884 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/files0-from' -> './k/coreutils-8.5/tests/du/files0-from' 2024-03-09T09:28:18.884 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/files0-from’: No data available 2024-03-09T09:28:18.885 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/two-args' -> './k/coreutils-8.5/tests/du/two-args' 2024-03-09T09:28:18.886 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/two-args’: No data available 2024-03-09T09:28:18.887 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/hard-link' -> './k/coreutils-8.5/tests/du/hard-link' 2024-03-09T09:28:18.887 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/hard-link’: No data available 2024-03-09T09:28:18.888 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/long-from-unreadable' -> './k/coreutils-8.5/tests/du/long-from-unreadable' 2024-03-09T09:28:18.888 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/long-from-unreadable’: No data available 2024-03-09T09:28:18.890 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/8gb' -> './k/coreutils-8.5/tests/du/8gb' 2024-03-09T09:28:18.891 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/8gb’: No data available 2024-03-09T09:28:18.892 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/2g' -> './k/coreutils-8.5/tests/du/2g' 2024-03-09T09:28:18.893 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/2g’: No data available 2024-03-09T09:28:18.893 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/restore-wd' -> './k/coreutils-8.5/tests/du/restore-wd' 2024-03-09T09:28:18.893 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/restore-wd’: No data available 2024-03-09T09:28:18.894 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/slash' -> './k/coreutils-8.5/tests/du/slash' 2024-03-09T09:28:18.894 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/slash’: No data available 2024-03-09T09:28:18.895 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/no-x' -> './k/coreutils-8.5/tests/du/no-x' 2024-03-09T09:28:18.895 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/no-x’: No data available 2024-03-09T09:28:18.897 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/fd-leak' -> './k/coreutils-8.5/tests/du/fd-leak' 2024-03-09T09:28:18.897 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/fd-leak’: No data available 2024-03-09T09:28:18.898 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/no-deref' -> './k/coreutils-8.5/tests/du/no-deref' 2024-03-09T09:28:18.899 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/no-deref’: No data available 2024-03-09T09:28:18.900 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/du/basic' -> './k/coreutils-8.5/tests/du/basic' 2024-03-09T09:28:18.900 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/du/basic’: No data available 2024-03-09T09:28:18.965 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls' -> './k/coreutils-8.5/tests/ls' 2024-03-09T09:28:18.965 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/readdir-mountpoint-inode' -> './k/coreutils-8.5/tests/ls/readdir-mountpoint-inode' 2024-03-09T09:28:18.966 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/readdir-mountpoint-inode’: No data available 2024-03-09T09:28:18.967 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/multihardlink' -> './k/coreutils-8.5/tests/ls/multihardlink' 2024-03-09T09:28:18.967 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/multihardlink’: No data available 2024-03-09T09:28:18.968 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/color-dtype-dir' -> './k/coreutils-8.5/tests/ls/color-dtype-dir' 2024-03-09T09:28:18.969 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/color-dtype-dir’: No data available 2024-03-09T09:28:19.591 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/file-type' -> './k/coreutils-8.5/tests/ls/file-type' 2024-03-09T09:28:19.591 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/file-type’: No data available 2024-03-09T09:28:19.592 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/rt-1' -> './k/coreutils-8.5/tests/ls/rt-1' 2024-03-09T09:28:19.592 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/rt-1’: No data available 2024-03-09T09:28:19.594 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/stat-free-symlinks' -> './k/coreutils-8.5/tests/ls/stat-free-symlinks' 2024-03-09T09:28:19.594 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/stat-free-symlinks’: No data available 2024-03-09T09:28:19.596 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/no-cap' -> './k/coreutils-8.5/tests/ls/no-cap' 2024-03-09T09:28:19.596 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/no-cap’: No data available 2024-03-09T09:28:19.598 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/capability' -> './k/coreutils-8.5/tests/ls/capability' 2024-03-09T09:28:19.598 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/capability’: No data available 2024-03-09T09:28:19.600 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/inode' -> './k/coreutils-8.5/tests/ls/inode' 2024-03-09T09:28:19.600 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/inode’: No data available 2024-03-09T09:28:19.603 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/color-norm' -> './k/coreutils-8.5/tests/ls/color-norm' 2024-03-09T09:28:19.603 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/color-norm’: No data available 2024-03-09T09:28:19.605 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/abmon-align' -> './k/coreutils-8.5/tests/ls/abmon-align' 2024-03-09T09:28:19.605 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/abmon-align’: No data available 2024-03-09T09:28:19.607 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/dangle' -> './k/coreutils-8.5/tests/ls/dangle' 2024-03-09T09:28:19.607 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/dangle’: No data available 2024-03-09T09:28:19.609 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/dired' -> './k/coreutils-8.5/tests/ls/dired' 2024-03-09T09:28:19.609 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/dired’: No data available 2024-03-09T09:28:19.611 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/m-option' -> './k/coreutils-8.5/tests/ls/m-option' 2024-03-09T09:28:19.611 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/m-option’: No data available 2024-03-09T09:28:19.613 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/nameless-uid' -> './k/coreutils-8.5/tests/ls/nameless-uid' 2024-03-09T09:28:19.613 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/nameless-uid’: No data available 2024-03-09T09:28:19.616 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/proc-selinux-segfault' -> './k/coreutils-8.5/tests/ls/proc-selinux-segfault' 2024-03-09T09:28:19.616 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/proc-selinux-segfault’: No data available 2024-03-09T09:28:19.618 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/stat-vs-dirent' -> './k/coreutils-8.5/tests/ls/stat-vs-dirent' 2024-03-09T09:28:19.619 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/stat-vs-dirent’: No data available 2024-03-09T09:28:19.621 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/color-clear-to-eol' -> './k/coreutils-8.5/tests/ls/color-clear-to-eol' 2024-03-09T09:28:19.621 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/color-clear-to-eol’: No data available 2024-03-09T09:28:19.623 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/no-arg' -> './k/coreutils-8.5/tests/ls/no-arg' 2024-03-09T09:28:19.624 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/no-arg’: No data available 2024-03-09T09:28:19.626 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/follow-slink' -> './k/coreutils-8.5/tests/ls/follow-slink' 2024-03-09T09:28:19.626 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/follow-slink’: No data available 2024-03-09T09:28:19.628 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/symlink-slash' -> './k/coreutils-8.5/tests/ls/symlink-slash' 2024-03-09T09:28:19.629 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/symlink-slash’: No data available 2024-03-09T09:28:19.631 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/stat-dtype' -> './k/coreutils-8.5/tests/ls/stat-dtype' 2024-03-09T09:28:19.631 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/stat-dtype’: No data available 2024-03-09T09:28:19.633 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/x-option' -> './k/coreutils-8.5/tests/ls/x-option' 2024-03-09T09:28:19.633 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/x-option’: No data available 2024-03-09T09:28:19.636 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/infloop' -> './k/coreutils-8.5/tests/ls/infloop' 2024-03-09T09:28:19.636 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/infloop’: No data available 2024-03-09T09:28:19.638 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/stat-failed' -> './k/coreutils-8.5/tests/ls/stat-failed' 2024-03-09T09:28:19.638 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/stat-failed’: No data available 2024-03-09T09:28:19.641 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/ls/recursive' -> './k/coreutils-8.5/tests/ls/recursive' 2024-03-09T09:28:19.641 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/ls/recursive’: No data available 2024-03-09T09:28:19.845 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv' -> './k/coreutils-8.5/tests/mv' 2024-03-09T09:28:19.845 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/mv-n' -> './k/coreutils-8.5/tests/mv/mv-n' 2024-03-09T09:28:19.846 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/mv-n’: No data available 2024-03-09T09:28:19.848 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/to-symlink' -> './k/coreutils-8.5/tests/mv/to-symlink' 2024-03-09T09:28:19.848 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/to-symlink’: No data available 2024-03-09T09:28:19.851 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/part-fail' -> './k/coreutils-8.5/tests/mv/part-fail' 2024-03-09T09:28:19.851 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/part-fail’: No data available 2024-03-09T09:28:19.853 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/hard-4' -> './k/coreutils-8.5/tests/mv/hard-4' 2024-03-09T09:28:19.854 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/hard-4’: No data available 2024-03-09T09:28:19.855 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/leak-fd' -> './k/coreutils-8.5/tests/mv/leak-fd' 2024-03-09T09:28:19.856 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/leak-fd’: No data available 2024-03-09T09:28:19.858 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/no-target-dir' -> './k/coreutils-8.5/tests/mv/no-target-dir' 2024-03-09T09:28:19.859 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/no-target-dir’: No data available 2024-03-09T09:28:19.861 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/part-rename' -> './k/coreutils-8.5/tests/mv/part-rename' 2024-03-09T09:28:19.861 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/part-rename’: No data available 2024-03-09T09:28:19.863 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/dir-file' -> './k/coreutils-8.5/tests/mv/dir-file' 2024-03-09T09:28:19.864 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/dir-file’: No data available 2024-03-09T09:28:19.866 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/hard-2' -> './k/coreutils-8.5/tests/mv/hard-2' 2024-03-09T09:28:19.866 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/hard-2’: No data available 2024-03-09T09:28:19.868 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/part-hardlink' -> './k/coreutils-8.5/tests/mv/part-hardlink' 2024-03-09T09:28:19.868 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/part-hardlink’: No data available 2024-03-09T09:28:19.871 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/sticky-to-xpart' -> './k/coreutils-8.5/tests/mv/sticky-to-xpart' 2024-03-09T09:28:19.871 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/sticky-to-xpart’: No data available 2024-03-09T09:28:19.873 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/mv-special-1' -> './k/coreutils-8.5/tests/mv/mv-special-1' 2024-03-09T09:28:19.873 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/mv-special-1’: No data available 2024-03-09T09:28:19.875 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/trailing-slash' -> './k/coreutils-8.5/tests/mv/trailing-slash' 2024-03-09T09:28:19.876 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/trailing-slash’: No data available 2024-03-09T09:28:19.878 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/dir2dir' -> './k/coreutils-8.5/tests/mv/dir2dir' 2024-03-09T09:28:19.878 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/dir2dir’: No data available 2024-03-09T09:28:19.880 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/into-self-2' -> './k/coreutils-8.5/tests/mv/into-self-2' 2024-03-09T09:28:19.881 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/into-self-2’: No data available 2024-03-09T09:28:19.883 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/backup-dir' -> './k/coreutils-8.5/tests/mv/backup-dir' 2024-03-09T09:28:19.883 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/backup-dir’: No data available 2024-03-09T09:28:19.885 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/diag' -> './k/coreutils-8.5/tests/mv/diag' 2024-03-09T09:28:19.886 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/diag’: No data available 2024-03-09T09:28:19.888 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/atomic' -> './k/coreutils-8.5/tests/mv/atomic' 2024-03-09T09:28:19.888 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/atomic’: No data available 2024-03-09T09:28:19.890 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/force' -> './k/coreutils-8.5/tests/mv/force' 2024-03-09T09:28:19.891 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/force’: No data available 2024-03-09T09:28:19.892 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/into-self-3' -> './k/coreutils-8.5/tests/mv/into-self-3' 2024-03-09T09:28:19.893 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/into-self-3’: No data available 2024-03-09T09:28:19.895 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/into-self' -> './k/coreutils-8.5/tests/mv/into-self' 2024-03-09T09:28:19.895 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/into-self’: No data available 2024-03-09T09:28:19.897 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/i-2' -> './k/coreutils-8.5/tests/mv/i-2' 2024-03-09T09:28:19.898 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-2’: No data available 2024-03-09T09:28:19.900 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/childproof' -> './k/coreutils-8.5/tests/mv/childproof' 2024-03-09T09:28:19.900 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/childproof’: No data available 2024-03-09T09:28:19.903 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/perm-1' -> './k/coreutils-8.5/tests/mv/perm-1' 2024-03-09T09:28:19.903 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/perm-1’: No data available 2024-03-09T09:28:19.905 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/i-3' -> './k/coreutils-8.5/tests/mv/i-3' 2024-03-09T09:28:19.905 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-3’: No data available 2024-03-09T09:28:19.907 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/partition-perm' -> './k/coreutils-8.5/tests/mv/partition-perm' 2024-03-09T09:28:19.907 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/partition-perm’: No data available 2024-03-09T09:28:19.910 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/part-symlink' -> './k/coreutils-8.5/tests/mv/part-symlink' 2024-03-09T09:28:19.910 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/part-symlink’: No data available 2024-03-09T09:28:19.912 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/backup-is-src' -> './k/coreutils-8.5/tests/mv/backup-is-src' 2024-03-09T09:28:19.912 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/backup-is-src’: No data available 2024-03-09T09:28:19.915 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/i-1' -> './k/coreutils-8.5/tests/mv/i-1' 2024-03-09T09:28:19.915 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-1’: No data available 2024-03-09T09:28:19.917 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/update' -> './k/coreutils-8.5/tests/mv/update' 2024-03-09T09:28:19.918 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/update’: No data available 2024-03-09T09:28:19.920 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/i-4' -> './k/coreutils-8.5/tests/mv/i-4' 2024-03-09T09:28:19.920 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-4’: No data available 2024-03-09T09:28:19.922 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/hard-verbose' -> './k/coreutils-8.5/tests/mv/hard-verbose' 2024-03-09T09:28:19.923 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/hard-verbose’: No data available 2024-03-09T09:28:19.925 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/dup-source' -> './k/coreutils-8.5/tests/mv/dup-source' 2024-03-09T09:28:19.925 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/dup-source’: No data available 2024-03-09T09:28:19.927 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/i-link-no' -> './k/coreutils-8.5/tests/mv/i-link-no' 2024-03-09T09:28:19.928 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-link-no’: No data available 2024-03-09T09:28:19.930 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/atomic2' -> './k/coreutils-8.5/tests/mv/atomic2' 2024-03-09T09:28:19.930 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/atomic2’: No data available 2024-03-09T09:28:19.932 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/into-self-4' -> './k/coreutils-8.5/tests/mv/into-self-4' 2024-03-09T09:28:19.933 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/into-self-4’: No data available 2024-03-09T09:28:19.935 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/hard-link-1' -> './k/coreutils-8.5/tests/mv/hard-link-1' 2024-03-09T09:28:19.935 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/hard-link-1’: No data available 2024-03-09T09:28:19.937 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/i-5' -> './k/coreutils-8.5/tests/mv/i-5' 2024-03-09T09:28:19.938 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/i-5’: No data available 2024-03-09T09:28:19.940 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/acl' -> './k/coreutils-8.5/tests/mv/acl' 2024-03-09T09:28:19.940 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/acl’: No data available 2024-03-09T09:28:20.082 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/mv/hard-3' -> './k/coreutils-8.5/tests/mv/hard-3' 2024-03-09T09:28:20.082 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/mv/hard-3’: No data available 2024-03-09T09:28:20.492 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch' -> './k/coreutils-8.5/tests/touch' 2024-03-09T09:28:20.493 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/empty-file' -> './k/coreutils-8.5/tests/touch/empty-file' 2024-03-09T09:28:20.493 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/empty-file’: No data available 2024-03-09T09:28:20.495 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/read-only' -> './k/coreutils-8.5/tests/touch/read-only' 2024-03-09T09:28:20.495 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/read-only’: No data available 2024-03-09T09:28:20.497 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/fail-diag' -> './k/coreutils-8.5/tests/touch/fail-diag' 2024-03-09T09:28:20.498 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/fail-diag’: No data available 2024-03-09T09:28:20.500 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/trailing-slash' -> './k/coreutils-8.5/tests/touch/trailing-slash' 2024-03-09T09:28:20.501 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/trailing-slash’: No data available 2024-03-09T09:28:20.503 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/no-dereference' -> './k/coreutils-8.5/tests/touch/no-dereference' 2024-03-09T09:28:20.503 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/no-dereference’: No data available 2024-03-09T09:28:20.505 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/dir-1' -> './k/coreutils-8.5/tests/touch/dir-1' 2024-03-09T09:28:20.506 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/dir-1’: No data available 2024-03-09T09:28:20.508 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/dangling-symlink' -> './k/coreutils-8.5/tests/touch/dangling-symlink' 2024-03-09T09:28:20.508 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/dangling-symlink’: No data available 2024-03-09T09:28:20.511 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/now-owned-by-other' -> './k/coreutils-8.5/tests/touch/now-owned-by-other' 2024-03-09T09:28:20.511 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/now-owned-by-other’: No data available 2024-03-09T09:28:20.513 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/no-create-missing' -> './k/coreutils-8.5/tests/touch/no-create-missing' 2024-03-09T09:28:20.514 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/no-create-missing’: No data available 2024-03-09T09:28:20.516 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/no-rights' -> './k/coreutils-8.5/tests/touch/no-rights' 2024-03-09T09:28:20.517 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/no-rights’: No data available 2024-03-09T09:28:20.519 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/obsolescent' -> './k/coreutils-8.5/tests/touch/obsolescent' 2024-03-09T09:28:20.519 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/obsolescent’: No data available 2024-03-09T09:28:20.522 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/60-seconds' -> './k/coreutils-8.5/tests/touch/60-seconds' 2024-03-09T09:28:20.522 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/60-seconds’: No data available 2024-03-09T09:28:20.524 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/not-owner' -> './k/coreutils-8.5/tests/touch/not-owner' 2024-03-09T09:28:20.524 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/not-owner’: No data available 2024-03-09T09:28:20.527 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/relative' -> './k/coreutils-8.5/tests/touch/relative' 2024-03-09T09:28:20.528 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/relative’: No data available 2024-03-09T09:28:20.530 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/touch/fifo' -> './k/coreutils-8.5/tests/touch/fifo' 2024-03-09T09:28:20.530 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/touch/fifo’: No data available 2024-03-09T09:28:20.854 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc' -> './k/coreutils-8.5/tests/misc' 2024-03-09T09:28:20.854 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/basename' -> './k/coreutils-8.5/tests/misc/basename' 2024-03-09T09:28:20.855 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/basename’: No data available 2024-03-09T09:28:20.857 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/stat-fmt' -> './k/coreutils-8.5/tests/misc/stat-fmt' 2024-03-09T09:28:20.857 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stat-fmt’: No data available 2024-03-09T09:28:20.859 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/pr' -> './k/coreutils-8.5/tests/misc/pr' 2024-03-09T09:28:20.859 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/pr’: No data available 2024-03-09T09:28:20.863 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/dirname' -> './k/coreutils-8.5/tests/misc/dirname' 2024-03-09T09:28:20.863 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/dirname’: No data available 2024-03-09T09:28:20.865 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/chcon-fail' -> './k/coreutils-8.5/tests/misc/chcon-fail' 2024-03-09T09:28:20.866 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/chcon-fail’: No data available 2024-03-09T09:28:20.868 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/head-elide-tail' -> './k/coreutils-8.5/tests/misc/head-elide-tail' 2024-03-09T09:28:20.868 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/head-elide-tail’: No data available 2024-03-09T09:28:20.870 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/chroot-credentials' -> './k/coreutils-8.5/tests/misc/chroot-credentials' 2024-03-09T09:28:20.871 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/chroot-credentials’: No data available 2024-03-09T09:28:20.873 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/nl' -> './k/coreutils-8.5/tests/misc/nl' 2024-03-09T09:28:20.873 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nl’: No data available 2024-03-09T09:28:20.875 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/tee-dash' -> './k/coreutils-8.5/tests/misc/tee-dash' 2024-03-09T09:28:20.875 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tee-dash’: No data available 2024-03-09T09:28:20.878 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/stat-hyphen' -> './k/coreutils-8.5/tests/misc/stat-hyphen' 2024-03-09T09:28:20.878 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stat-hyphen’: No data available 2024-03-09T09:28:20.880 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-rand' -> './k/coreutils-8.5/tests/misc/sort-rand' 2024-03-09T09:28:20.881 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-rand’: No data available 2024-03-09T09:28:20.882 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/expr' -> './k/coreutils-8.5/tests/misc/expr' 2024-03-09T09:28:20.883 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/expr’: No data available 2024-03-09T09:28:20.885 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/fmt-long-line' -> './k/coreutils-8.5/tests/misc/fmt-long-line' 2024-03-09T09:28:20.885 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/fmt-long-line’: No data available 2024-03-09T09:28:20.888 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/split-fail' -> './k/coreutils-8.5/tests/misc/split-fail' 2024-03-09T09:28:20.889 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/split-fail’: No data available 2024-03-09T09:28:20.891 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-relative' -> './k/coreutils-8.5/tests/misc/truncate-relative' 2024-03-09T09:28:20.891 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-relative’: No data available 2024-03-09T09:28:20.893 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-dir-fail' -> './k/coreutils-8.5/tests/misc/truncate-dir-fail' 2024-03-09T09:28:20.894 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-dir-fail’: No data available 2024-03-09T09:28:20.895 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/csplit' -> './k/coreutils-8.5/tests/misc/csplit' 2024-03-09T09:28:20.896 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/csplit’: No data available 2024-03-09T09:28:20.898 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/od-N' -> './k/coreutils-8.5/tests/misc/od-N' 2024-03-09T09:28:20.898 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/od-N’: No data available 2024-03-09T09:28:20.900 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/xattr' -> './k/coreutils-8.5/tests/misc/xattr' 2024-03-09T09:28:20.901 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/xattr’: No data available 2024-03-09T09:28:20.903 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/mktemp' -> './k/coreutils-8.5/tests/misc/mktemp' 2024-03-09T09:28:20.903 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/mktemp’: No data available 2024-03-09T09:28:20.905 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/cut' -> './k/coreutils-8.5/tests/misc/cut' 2024-03-09T09:28:20.905 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/cut’: No data available 2024-03-09T09:28:20.908 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/cat-proc' -> './k/coreutils-8.5/tests/misc/cat-proc' 2024-03-09T09:28:20.908 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/cat-proc’: No data available 2024-03-09T09:28:20.911 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/comm' -> './k/coreutils-8.5/tests/misc/comm' 2024-03-09T09:28:20.911 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/comm’: No data available 2024-03-09T09:28:20.913 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/timeout-parameters' -> './k/coreutils-8.5/tests/misc/timeout-parameters' 2024-03-09T09:28:20.914 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/timeout-parameters’: No data available 2024-03-09T09:28:20.916 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/tac-continue' -> './k/coreutils-8.5/tests/misc/tac-continue' 2024-03-09T09:28:20.916 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tac-continue’: No data available 2024-03-09T09:28:20.918 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/tty-eof' -> './k/coreutils-8.5/tests/misc/tty-eof' 2024-03-09T09:28:20.919 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tty-eof’: No data available 2024-03-09T09:28:20.921 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/df-P' -> './k/coreutils-8.5/tests/misc/df-P' 2024-03-09T09:28:20.921 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/df-P’: No data available 2024-03-09T09:28:20.924 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/shuf' -> './k/coreutils-8.5/tests/misc/shuf' 2024-03-09T09:28:20.924 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/shuf’: No data available 2024-03-09T09:28:20.926 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/env' -> './k/coreutils-8.5/tests/misc/env' 2024-03-09T09:28:20.926 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/env’: No data available 2024-03-09T09:28:20.929 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/od' -> './k/coreutils-8.5/tests/misc/od' 2024-03-09T09:28:20.929 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/od’: No data available 2024-03-09T09:28:20.931 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/split-a' -> './k/coreutils-8.5/tests/misc/split-a' 2024-03-09T09:28:20.931 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/split-a’: No data available 2024-03-09T09:28:20.933 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/tail' -> './k/coreutils-8.5/tests/misc/tail' 2024-03-09T09:28:20.934 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tail’: No data available 2024-03-09T09:28:20.936 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/stat-slash' -> './k/coreutils-8.5/tests/misc/stat-slash' 2024-03-09T09:28:20.936 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stat-slash’: No data available 2024-03-09T09:28:20.940 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/timeout' -> './k/coreutils-8.5/tests/misc/timeout' 2024-03-09T09:28:20.940 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/timeout’: No data available 2024-03-09T09:28:20.942 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/close-stdout' -> './k/coreutils-8.5/tests/misc/close-stdout' 2024-03-09T09:28:20.942 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/close-stdout’: No data available 2024-03-09T09:28:20.945 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/wc-files0-from' -> './k/coreutils-8.5/tests/misc/wc-files0-from' 2024-03-09T09:28:20.945 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/wc-files0-from’: No data available 2024-03-09T09:28:20.947 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sum' -> './k/coreutils-8.5/tests/misc/sum' 2024-03-09T09:28:20.947 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sum’: No data available 2024-03-09T09:28:20.949 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/expand' -> './k/coreutils-8.5/tests/misc/expand' 2024-03-09T09:28:20.949 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/expand’: No data available 2024-03-09T09:28:20.951 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sum-sysv' -> './k/coreutils-8.5/tests/misc/sum-sysv' 2024-03-09T09:28:20.951 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sum-sysv’: No data available 2024-03-09T09:28:20.953 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/wc-parallel' -> './k/coreutils-8.5/tests/misc/wc-parallel' 2024-03-09T09:28:20.953 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/wc-parallel’: No data available 2024-03-09T09:28:20.955 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/md5sum-newline' -> './k/coreutils-8.5/tests/misc/md5sum-newline' 2024-03-09T09:28:20.955 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/md5sum-newline’: No data available 2024-03-09T09:28:20.957 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sha224sum' -> './k/coreutils-8.5/tests/misc/sha224sum' 2024-03-09T09:28:20.957 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha224sum’: No data available 2024-03-09T09:28:20.959 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/stty' -> './k/coreutils-8.5/tests/misc/stty' 2024-03-09T09:28:20.963 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stty’: No data available 2024-03-09T09:28:20.964 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/printf-hex' -> './k/coreutils-8.5/tests/misc/printf-hex' 2024-03-09T09:28:20.964 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/printf-hex’: No data available 2024-03-09T09:28:20.965 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/id-context' -> './k/coreutils-8.5/tests/misc/id-context' 2024-03-09T09:28:20.965 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/id-context’: No data available 2024-03-09T09:28:20.966 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/stat-printf' -> './k/coreutils-8.5/tests/misc/stat-printf' 2024-03-09T09:28:20.967 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stat-printf’: No data available 2024-03-09T09:28:20.969 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/pwd-option' -> './k/coreutils-8.5/tests/misc/pwd-option' 2024-03-09T09:28:20.969 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/pwd-option’: No data available 2024-03-09T09:28:20.971 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sha256sum' -> './k/coreutils-8.5/tests/misc/sha256sum' 2024-03-09T09:28:20.971 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha256sum’: No data available 2024-03-09T09:28:20.974 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-version' -> './k/coreutils-8.5/tests/misc/sort-version' 2024-03-09T09:28:20.974 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-version’: No data available 2024-03-09T09:28:20.976 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/groups-dash' -> './k/coreutils-8.5/tests/misc/groups-dash' 2024-03-09T09:28:20.976 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/groups-dash’: No data available 2024-03-09T09:28:20.978 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/printenv' -> './k/coreutils-8.5/tests/misc/printenv' 2024-03-09T09:28:20.978 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/printenv’: No data available 2024-03-09T09:28:20.980 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/seq' -> './k/coreutils-8.5/tests/misc/seq' 2024-03-09T09:28:20.981 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/seq’: No data available 2024-03-09T09:28:20.983 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/fmt' -> './k/coreutils-8.5/tests/misc/fmt' 2024-03-09T09:28:20.983 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/fmt’: No data available 2024-03-09T09:28:20.985 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/cat-buf' -> './k/coreutils-8.5/tests/misc/cat-buf' 2024-03-09T09:28:20.985 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/cat-buf’: No data available 2024-03-09T09:28:20.987 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/dircolors' -> './k/coreutils-8.5/tests/misc/dircolors' 2024-03-09T09:28:20.987 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/dircolors’: No data available 2024-03-09T09:28:20.989 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/fold' -> './k/coreutils-8.5/tests/misc/fold' 2024-03-09T09:28:20.990 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/fold’: No data available 2024-03-09T09:28:20.992 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/selinux' -> './k/coreutils-8.5/tests/misc/selinux' 2024-03-09T09:28:20.992 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/selinux’: No data available 2024-03-09T09:28:20.994 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/md5sum-parallel' -> './k/coreutils-8.5/tests/misc/md5sum-parallel' 2024-03-09T09:28:20.995 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/md5sum-parallel’: No data available 2024-03-09T09:28:20.996 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/md5sum' -> './k/coreutils-8.5/tests/misc/md5sum' 2024-03-09T09:28:20.997 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/md5sum’: No data available 2024-03-09T09:28:20.999 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/nice-fail' -> './k/coreutils-8.5/tests/misc/nice-fail' 2024-03-09T09:28:20.999 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nice-fail’: No data available 2024-03-09T09:28:21.001 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/groups-version' -> './k/coreutils-8.5/tests/misc/groups-version' 2024-03-09T09:28:21.002 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/groups-version’: No data available 2024-03-09T09:28:21.003 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/date-sec' -> './k/coreutils-8.5/tests/misc/date-sec' 2024-03-09T09:28:21.003 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/date-sec’: No data available 2024-03-09T09:28:21.006 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/split-l' -> './k/coreutils-8.5/tests/misc/split-l' 2024-03-09T09:28:21.006 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/split-l’: No data available 2024-03-09T09:28:21.008 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/false-status' -> './k/coreutils-8.5/tests/misc/false-status' 2024-03-09T09:28:21.008 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/false-status’: No data available 2024-03-09T09:28:21.010 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sha384sum' -> './k/coreutils-8.5/tests/misc/sha384sum' 2024-03-09T09:28:21.010 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha384sum’: No data available 2024-03-09T09:28:21.012 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-dangling-symlink' -> './k/coreutils-8.5/tests/misc/truncate-dangling-symlink' 2024-03-09T09:28:21.012 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-dangling-symlink’: No data available 2024-03-09T09:28:21.014 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/od-multiple-t' -> './k/coreutils-8.5/tests/misc/od-multiple-t' 2024-03-09T09:28:21.014 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/od-multiple-t’: No data available 2024-03-09T09:28:21.016 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/wc-files0' -> './k/coreutils-8.5/tests/misc/wc-files0' 2024-03-09T09:28:21.016 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/wc-files0’: No data available 2024-03-09T09:28:21.018 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/nohup' -> './k/coreutils-8.5/tests/misc/nohup' 2024-03-09T09:28:21.018 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nohup’: No data available 2024-03-09T09:28:21.020 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-no-create-missing' -> './k/coreutils-8.5/tests/misc/truncate-no-create-missing' 2024-03-09T09:28:21.020 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-no-create-missing’: No data available 2024-03-09T09:28:21.022 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/head-pos' -> './k/coreutils-8.5/tests/misc/head-pos' 2024-03-09T09:28:21.023 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/head-pos’: No data available 2024-03-09T09:28:21.025 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/uniq' -> './k/coreutils-8.5/tests/misc/uniq' 2024-03-09T09:28:21.025 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/uniq’: No data available 2024-03-09T09:28:21.027 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/test-diag' -> './k/coreutils-8.5/tests/misc/test-diag' 2024-03-09T09:28:21.027 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/test-diag’: No data available 2024-03-09T09:28:21.029 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/tac' -> './k/coreutils-8.5/tests/misc/tac' 2024-03-09T09:28:21.030 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tac’: No data available 2024-03-09T09:28:21.032 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-continue' -> './k/coreutils-8.5/tests/misc/sort-continue' 2024-03-09T09:28:21.032 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-continue’: No data available 2024-03-09T09:28:21.034 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/head' -> './k/coreutils-8.5/tests/misc/head' 2024-03-09T09:28:21.034 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/head’: No data available 2024-03-09T09:28:21.036 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/ptx' -> './k/coreutils-8.5/tests/misc/ptx' 2024-03-09T09:28:21.036 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/ptx’: No data available 2024-03-09T09:28:21.038 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/nproc-avail' -> './k/coreutils-8.5/tests/misc/nproc-avail' 2024-03-09T09:28:21.039 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nproc-avail’: No data available 2024-03-09T09:28:21.041 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/su-fail' -> './k/coreutils-8.5/tests/misc/su-fail' 2024-03-09T09:28:21.041 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/su-fail’: No data available 2024-03-09T09:28:21.043 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/od-x8' -> './k/coreutils-8.5/tests/misc/od-x8' 2024-03-09T09:28:21.043 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/od-x8’: No data available 2024-03-09T09:28:21.045 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/nice' -> './k/coreutils-8.5/tests/misc/nice' 2024-03-09T09:28:21.046 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nice’: No data available 2024-03-09T09:28:21.047 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/chcon' -> './k/coreutils-8.5/tests/misc/chcon' 2024-03-09T09:28:21.048 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/chcon’: No data available 2024-03-09T09:28:21.050 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/head-c' -> './k/coreutils-8.5/tests/misc/head-c' 2024-03-09T09:28:21.050 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/head-c’: No data available 2024-03-09T09:28:21.052 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-compress' -> './k/coreutils-8.5/tests/misc/sort-compress' 2024-03-09T09:28:21.052 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-compress’: No data available 2024-03-09T09:28:21.054 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/df' -> './k/coreutils-8.5/tests/misc/df' 2024-03-09T09:28:21.054 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/df’: No data available 2024-03-09T09:28:21.057 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/chroot-fail' -> './k/coreutils-8.5/tests/misc/chroot-fail' 2024-03-09T09:28:21.057 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/chroot-fail’: No data available 2024-03-09T09:28:21.059 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/date-next-dow' -> './k/coreutils-8.5/tests/misc/date-next-dow' 2024-03-09T09:28:21.059 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/date-next-dow’: No data available 2024-03-09T09:28:21.061 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sha1sum' -> './k/coreutils-8.5/tests/misc/sha1sum' 2024-03-09T09:28:21.061 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha1sum’: No data available 2024-03-09T09:28:21.066 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sha1sum-vec' -> './k/coreutils-8.5/tests/misc/sha1sum-vec' 2024-03-09T09:28:21.066 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha1sum-vec’: No data available 2024-03-09T09:28:21.068 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/pathchk1' -> './k/coreutils-8.5/tests/misc/pathchk1' 2024-03-09T09:28:21.069 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/pathchk1’: No data available 2024-03-09T09:28:21.071 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/arch' -> './k/coreutils-8.5/tests/misc/arch' 2024-03-09T09:28:21.071 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/arch’: No data available 2024-03-09T09:28:21.073 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-month' -> './k/coreutils-8.5/tests/misc/sort-month' 2024-03-09T09:28:21.074 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-month’: No data available 2024-03-09T09:28:21.076 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/shred-passes' -> './k/coreutils-8.5/tests/misc/shred-passes' 2024-03-09T09:28:21.076 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/shred-passes’: No data available 2024-03-09T09:28:21.079 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sha512sum' -> './k/coreutils-8.5/tests/misc/sha512sum' 2024-03-09T09:28:21.079 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sha512sum’: No data available 2024-03-09T09:28:21.081 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/paste' -> './k/coreutils-8.5/tests/misc/paste' 2024-03-09T09:28:21.082 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/paste’: No data available 2024-03-09T09:28:21.084 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/wc' -> './k/coreutils-8.5/tests/misc/wc' 2024-03-09T09:28:21.084 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/wc’: No data available 2024-03-09T09:28:21.086 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/ls-time' -> './k/coreutils-8.5/tests/misc/ls-time' 2024-03-09T09:28:21.086 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/ls-time’: No data available 2024-03-09T09:28:21.088 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/pwd-unreadable-parent' -> './k/coreutils-8.5/tests/misc/pwd-unreadable-parent' 2024-03-09T09:28:21.089 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/pwd-unreadable-parent’: No data available 2024-03-09T09:28:21.090 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/test' -> './k/coreutils-8.5/tests/misc/test' 2024-03-09T09:28:21.091 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/test’: No data available 2024-03-09T09:28:21.093 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-parameters' -> './k/coreutils-8.5/tests/misc/truncate-parameters' 2024-03-09T09:28:21.093 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-parameters’: No data available 2024-03-09T09:28:21.095 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-files0-from' -> './k/coreutils-8.5/tests/misc/sort-files0-from' 2024-03-09T09:28:21.095 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-files0-from’: No data available 2024-03-09T09:28:21.098 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/base64' -> './k/coreutils-8.5/tests/misc/base64' 2024-03-09T09:28:21.098 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/base64’: No data available 2024-03-09T09:28:21.100 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/ptx-overrun' -> './k/coreutils-8.5/tests/misc/ptx-overrun' 2024-03-09T09:28:21.100 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/ptx-overrun’: No data available 2024-03-09T09:28:21.102 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/printf-cov' -> './k/coreutils-8.5/tests/misc/printf-cov' 2024-03-09T09:28:21.103 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/printf-cov’: No data available 2024-03-09T09:28:21.105 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/tee' -> './k/coreutils-8.5/tests/misc/tee' 2024-03-09T09:28:21.105 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tee’: No data available 2024-03-09T09:28:21.107 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/mknod' -> './k/coreutils-8.5/tests/misc/mknod' 2024-03-09T09:28:21.107 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/mknod’: No data available 2024-03-09T09:28:21.109 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/printf-surprise' -> './k/coreutils-8.5/tests/misc/printf-surprise' 2024-03-09T09:28:21.110 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/printf-surprise’: No data available 2024-03-09T09:28:21.112 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/ls-misc' -> './k/coreutils-8.5/tests/misc/ls-misc' 2024-03-09T09:28:21.112 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/ls-misc’: No data available 2024-03-09T09:28:21.114 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/nproc-positive' -> './k/coreutils-8.5/tests/misc/nproc-positive' 2024-03-09T09:28:21.114 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/nproc-positive’: No data available 2024-03-09T09:28:21.117 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/join' -> './k/coreutils-8.5/tests/misc/join' 2024-03-09T09:28:21.117 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/join’: No data available 2024-03-09T09:28:21.124 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-merge' -> './k/coreutils-8.5/tests/misc/sort-merge' 2024-03-09T09:28:21.124 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-merge’: No data available 2024-03-09T09:28:21.126 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/factor' -> './k/coreutils-8.5/tests/misc/factor' 2024-03-09T09:28:21.126 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/factor’: No data available 2024-03-09T09:28:21.129 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sort' -> './k/coreutils-8.5/tests/misc/sort' 2024-03-09T09:28:21.129 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort’: No data available 2024-03-09T09:28:21.131 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-fifo' -> './k/coreutils-8.5/tests/misc/truncate-fifo' 2024-03-09T09:28:21.131 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-fifo’: No data available 2024-03-09T09:28:21.133 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-fail-diag' -> './k/coreutils-8.5/tests/misc/truncate-fail-diag' 2024-03-09T09:28:21.134 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-fail-diag’: No data available 2024-03-09T09:28:21.136 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/stty-invalid' -> './k/coreutils-8.5/tests/misc/stty-invalid' 2024-03-09T09:28:21.136 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stty-invalid’: No data available 2024-03-09T09:28:21.138 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/env-null' -> './k/coreutils-8.5/tests/misc/env-null' 2024-03-09T09:28:21.139 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/env-null’: No data available 2024-03-09T09:28:21.140 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/printf' -> './k/coreutils-8.5/tests/misc/printf' 2024-03-09T09:28:21.141 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/printf’: No data available 2024-03-09T09:28:21.143 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/seq-long-double' -> './k/coreutils-8.5/tests/misc/seq-long-double' 2024-03-09T09:28:21.143 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/seq-long-double’: No data available 2024-03-09T09:28:21.145 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/pwd-long' -> './k/coreutils-8.5/tests/misc/pwd-long' 2024-03-09T09:28:21.145 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/pwd-long’: No data available 2024-03-09T09:28:21.147 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/tsort' -> './k/coreutils-8.5/tests/misc/tsort' 2024-03-09T09:28:21.147 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tsort’: No data available 2024-03-09T09:28:21.149 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/runcon-no-reorder' -> './k/coreutils-8.5/tests/misc/runcon-no-reorder' 2024-03-09T09:28:21.150 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/runcon-no-reorder’: No data available 2024-03-09T09:28:21.152 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/date' -> './k/coreutils-8.5/tests/misc/date' 2024-03-09T09:28:21.153 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/date’: No data available 2024-03-09T09:28:21.155 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/shred-exact' -> './k/coreutils-8.5/tests/misc/shred-exact' 2024-03-09T09:28:21.155 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/shred-exact’: No data available 2024-03-09T09:28:21.157 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/shred-remove' -> './k/coreutils-8.5/tests/misc/shred-remove' 2024-03-09T09:28:21.157 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/shred-remove’: No data available 2024-03-09T09:28:21.159 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/stdbuf' -> './k/coreutils-8.5/tests/misc/stdbuf' 2024-03-09T09:28:21.159 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stdbuf’: No data available 2024-03-09T09:28:21.161 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-overflow' -> './k/coreutils-8.5/tests/misc/truncate-overflow' 2024-03-09T09:28:21.162 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-overflow’: No data available 2024-03-09T09:28:21.164 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/stty-row-col' -> './k/coreutils-8.5/tests/misc/stty-row-col' 2024-03-09T09:28:21.164 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/stty-row-col’: No data available 2024-03-09T09:28:21.166 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/truncate-owned-by-other' -> './k/coreutils-8.5/tests/misc/truncate-owned-by-other' 2024-03-09T09:28:21.166 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/truncate-owned-by-other’: No data available 2024-03-09T09:28:21.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/xstrtol' -> './k/coreutils-8.5/tests/misc/xstrtol' 2024-03-09T09:28:21.168 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/xstrtol’: No data available 2024-03-09T09:28:21.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/invalid-opt' -> './k/coreutils-8.5/tests/misc/invalid-opt' 2024-03-09T09:28:21.171 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/invalid-opt’: No data available 2024-03-09T09:28:21.173 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/readlink-fp-loop' -> './k/coreutils-8.5/tests/misc/readlink-fp-loop' 2024-03-09T09:28:21.173 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/readlink-fp-loop’: No data available 2024-03-09T09:28:21.175 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/tr' -> './k/coreutils-8.5/tests/misc/tr' 2024-03-09T09:28:21.175 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/tr’: No data available 2024-03-09T09:28:21.177 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/help-version' -> './k/coreutils-8.5/tests/misc/help-version' 2024-03-09T09:28:21.178 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/help-version’: No data available 2024-03-09T09:28:21.179 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/id-groups' -> './k/coreutils-8.5/tests/misc/id-groups' 2024-03-09T09:28:21.180 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/id-groups’: No data available 2024-03-09T09:28:21.182 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/unexpand' -> './k/coreutils-8.5/tests/misc/unexpand' 2024-03-09T09:28:21.182 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/unexpand’: No data available 2024-03-09T09:28:21.184 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/misc/sort-merge-fdlimit' -> './k/coreutils-8.5/tests/misc/sort-merge-fdlimit' 2024-03-09T09:28:21.184 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/misc/sort-merge-fdlimit’: No data available 2024-03-09T09:28:21.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:28:21.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.438+0000 7f6755600640 1 Processor -- start 2024-03-09T09:28:21.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.438+0000 7f6755600640 1 -- start start 2024-03-09T09:28:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.439+0000 7f6755600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f67500aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.439+0000 7f6755600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67501579e0 con 0x7f67500c3600 2024-03-09T09:28:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.439+0000 7f6755600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675006d4c0 con 0x7f6750157600 2024-03-09T09:28:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.439+0000 7f6755600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675006d7a0 con 0x7f67500aa800 2024-03-09T09:28:21.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.440+0000 7f674fe00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f6750157600 legacy=0x7f675015aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:28:21.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.440+0000 7f6754c00640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 msgr2=0x7f67500aac90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:21.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.440+0000 7f674f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f67500c3600 0x7f67500ab760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:48698/0 (socket says 172.21.15.67:48698) 2024-03-09T09:28:21.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.440+0000 7f674f400640 1 -- 172.21.15.67:0/2130994072 learned_addr learned my addr 172.21.15.67:0/2130994072 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:28:21.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.440+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f67500aac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:21.553 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chgrp' -> './k/coreutils-8.5/tests/chgrp' 2024-03-09T09:28:21.553 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chgrp/default-no-deref' -> './k/coreutils-8.5/tests/chgrp/default-no-deref' 2024-03-09T09:28:21.554 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/default-no-deref’: No data available 2024-03-09T09:28:21.555 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chgrp/deref' -> './k/coreutils-8.5/tests/chgrp/deref' 2024-03-09T09:28:21.555 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/deref’: No data available 2024-03-09T09:28:21.557 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chgrp/posix-H' -> './k/coreutils-8.5/tests/chgrp/posix-H' 2024-03-09T09:28:21.557 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/posix-H’: No data available 2024-03-09T09:28:21.559 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chgrp/recurse' -> './k/coreutils-8.5/tests/chgrp/recurse' 2024-03-09T09:28:21.560 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/recurse’: No data available 2024-03-09T09:28:21.562 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chgrp/no-x' -> './k/coreutils-8.5/tests/chgrp/no-x' 2024-03-09T09:28:21.562 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/no-x’: No data available 2024-03-09T09:28:21.564 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/chgrp/basic' -> './k/coreutils-8.5/tests/chgrp/basic' 2024-03-09T09:28:21.564 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/chgrp/basic’: No data available 2024-03-09T09:28:21.594 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2' -> './k/coreutils-8.5/tests/tail-2' 2024-03-09T09:28:21.595 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/follow-stdin' -> './k/coreutils-8.5/tests/tail-2/follow-stdin' 2024-03-09T09:28:21.595 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/follow-stdin’: No data available 2024-03-09T09:28:21.597 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/F-vs-rename' -> './k/coreutils-8.5/tests/tail-2/F-vs-rename' 2024-03-09T09:28:21.597 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/F-vs-rename’: No data available 2024-03-09T09:28:21.599 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/inotify-rotate' -> './k/coreutils-8.5/tests/tail-2/inotify-rotate' 2024-03-09T09:28:21.600 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/inotify-rotate’: No data available 2024-03-09T09:28:21.602 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/pid' -> './k/coreutils-8.5/tests/tail-2/pid' 2024-03-09T09:28:21.602 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/pid’: No data available 2024-03-09T09:28:21.604 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/infloop-1' -> './k/coreutils-8.5/tests/tail-2/infloop-1' 2024-03-09T09:28:21.604 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/infloop-1’: No data available 2024-03-09T09:28:21.606 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/assert-2' -> './k/coreutils-8.5/tests/tail-2/assert-2' 2024-03-09T09:28:21.606 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/assert-2’: No data available 2024-03-09T09:28:21.608 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/pipe-f' -> './k/coreutils-8.5/tests/tail-2/pipe-f' 2024-03-09T09:28:21.609 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/pipe-f’: No data available 2024-03-09T09:28:21.611 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/pipe-f2' -> './k/coreutils-8.5/tests/tail-2/pipe-f2' 2024-03-09T09:28:21.611 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/pipe-f2’: No data available 2024-03-09T09:28:21.613 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/start-middle' -> './k/coreutils-8.5/tests/tail-2/start-middle' 2024-03-09T09:28:21.613 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/start-middle’: No data available 2024-03-09T09:28:21.615 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/inotify-hash-abuse2' -> './k/coreutils-8.5/tests/tail-2/inotify-hash-abuse2' 2024-03-09T09:28:21.615 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/inotify-hash-abuse2’: No data available 2024-03-09T09:28:21.617 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/inotify-hash-abuse' -> './k/coreutils-8.5/tests/tail-2/inotify-hash-abuse' 2024-03-09T09:28:21.618 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/inotify-hash-abuse’: No data available 2024-03-09T09:28:21.619 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/inotify-race' -> './k/coreutils-8.5/tests/tail-2/inotify-race' 2024-03-09T09:28:21.620 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/inotify-race’: No data available 2024-03-09T09:28:21.621 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/append-only' -> './k/coreutils-8.5/tests/tail-2/append-only' 2024-03-09T09:28:21.621 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/append-only’: No data available 2024-03-09T09:28:21.623 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/tail-n0f' -> './k/coreutils-8.5/tests/tail-2/tail-n0f' 2024-03-09T09:28:21.624 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/tail-n0f’: No data available 2024-03-09T09:28:21.625 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/flush-initial' -> './k/coreutils-8.5/tests/tail-2/flush-initial' 2024-03-09T09:28:21.626 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/flush-initial’: No data available 2024-03-09T09:28:21.628 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/assert' -> './k/coreutils-8.5/tests/tail-2/assert' 2024-03-09T09:28:21.628 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/assert’: No data available 2024-03-09T09:28:21.630 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/wait' -> './k/coreutils-8.5/tests/tail-2/wait' 2024-03-09T09:28:21.630 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/wait’: No data available 2024-03-09T09:28:21.632 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/proc-ksyms' -> './k/coreutils-8.5/tests/tail-2/proc-ksyms' 2024-03-09T09:28:21.632 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/proc-ksyms’: No data available 2024-03-09T09:28:21.634 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/tail-2/big-4gb' -> './k/coreutils-8.5/tests/tail-2/big-4gb' 2024-03-09T09:28:21.634 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/tail-2/big-4gb’: No data available 2024-03-09T09:28:21.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.641+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v1:172.21.15.134:6789/0 conn(0x7f6750157600 legacy=0x7f675015aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:28:21.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.641+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 msgr2=0x7f67500aac90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:21.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:21.641+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f67500aac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:28:21.944 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/readlink' -> './k/coreutils-8.5/tests/readlink' 2024-03-09T09:28:21.944 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/readlink/can-m' -> './k/coreutils-8.5/tests/readlink/can-m' 2024-03-09T09:28:21.944 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/readlink/can-m’: No data available 2024-03-09T09:28:21.947 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/readlink/rl-1' -> './k/coreutils-8.5/tests/readlink/rl-1' 2024-03-09T09:28:21.947 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/readlink/rl-1’: No data available 2024-03-09T09:28:21.950 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/readlink/can-e' -> './k/coreutils-8.5/tests/readlink/can-e' 2024-03-09T09:28:21.950 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/readlink/can-e’: No data available 2024-03-09T09:28:21.952 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/tests/readlink/can-f' -> './k/coreutils-8.5/tests/readlink/can-f' 2024-03-09T09:28:21.952 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/tests/readlink/can-f’: No data available 2024-03-09T09:28:21.990 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_trailing_blank' -> './k/coreutils-8.5/.x-sc_trailing_blank' 2024-03-09T09:28:21.991 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_trailing_blank’: No data available 2024-03-09T09:28:21.992 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/README' -> './k/coreutils-8.5/README' 2024-03-09T09:28:21.992 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/README’: No data available 2024-03-09T09:28:21.994 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.vg-suppressions' -> './k/coreutils-8.5/.vg-suppressions' 2024-03-09T09:28:21.995 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.vg-suppressions’: No data available 2024-03-09T09:28:21.996 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_atoi_atof' -> './k/coreutils-8.5/.x-sc_prohibit_atoi_atof' 2024-03-09T09:28:21.997 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_atoi_atof’: No data available 2024-03-09T09:28:21.999 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_stat_st_blocks' -> './k/coreutils-8.5/.x-sc_prohibit_stat_st_blocks' 2024-03-09T09:28:21.999 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_stat_st_blocks’: No data available 2024-03-09T09:28:22.002 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/ChangeLog-2006' -> './k/coreutils-8.5/ChangeLog-2006' 2024-03-09T09:28:22.003 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ChangeLog-2006’: No data available 2024-03-09T09:28:22.005 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_program_name' -> './k/coreutils-8.5/.x-sc_program_name' 2024-03-09T09:28:22.005 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_program_name’: No data available 2024-03-09T09:28:22.008 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/THANKStt.in' -> './k/coreutils-8.5/THANKStt.in' 2024-03-09T09:28:22.008 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/THANKStt.in’: No data available 2024-03-09T09:28:22.010 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/ChangeLog-2008' -> './k/coreutils-8.5/ChangeLog-2008' 2024-03-09T09:28:22.011 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ChangeLog-2008’: No data available 2024-03-09T09:28:22.015 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/ChangeLog-2007' -> './k/coreutils-8.5/ChangeLog-2007' 2024-03-09T09:28:22.016 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/ChangeLog-2007’: No data available 2024-03-09T09:28:22.026 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc' -> './k/coreutils-8.5/doc' 2024-03-09T09:28:22.026 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc/perm.texi' -> './k/coreutils-8.5/doc/perm.texi' 2024-03-09T09:28:22.027 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/perm.texi’: No data available 2024-03-09T09:28:22.029 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc/version.texi' -> './k/coreutils-8.5/doc/version.texi' 2024-03-09T09:28:22.029 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/version.texi’: No data available 2024-03-09T09:28:22.031 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc/Makefile.am' -> './k/coreutils-8.5/doc/Makefile.am' 2024-03-09T09:28:22.032 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/Makefile.am’: No data available 2024-03-09T09:28:22.043 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc/coreutils.info' -> './k/coreutils-8.5/doc/coreutils.info' 2024-03-09T09:28:22.043 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/coreutils.info’: No data available 2024-03-09T09:28:22.045 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc/stamp-vti' -> './k/coreutils-8.5/doc/stamp-vti' 2024-03-09T09:28:22.045 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/stamp-vti’: No data available 2024-03-09T09:28:22.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:22.042+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v1:172.21.15.134:6789/0 conn(0x7f6750157600 legacy=0x7f675015aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:28:22.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:22.042+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 msgr2=0x7f67500aac90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:22.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:22.042+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f67500aac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:28:22.051 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc/fdl.texi' -> './k/coreutils-8.5/doc/fdl.texi' 2024-03-09T09:28:22.051 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/fdl.texi’: No data available 2024-03-09T09:28:22.052 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc/Makefile.in' -> './k/coreutils-8.5/doc/Makefile.in' 2024-03-09T09:28:22.052 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/Makefile.in’: No data available 2024-03-09T09:28:22.054 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc/getdate.texi' -> './k/coreutils-8.5/doc/getdate.texi' 2024-03-09T09:28:22.054 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/getdate.texi’: No data available 2024-03-09T09:28:22.056 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc/constants.texi' -> './k/coreutils-8.5/doc/constants.texi' 2024-03-09T09:28:22.057 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/constants.texi’: No data available 2024-03-09T09:28:22.060 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc/ChangeLog-2007' -> './k/coreutils-8.5/doc/ChangeLog-2007' 2024-03-09T09:28:22.060 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/ChangeLog-2007’: No data available 2024-03-09T09:28:22.069 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/doc/coreutils.texi' -> './k/coreutils-8.5/doc/coreutils.texi' 2024-03-09T09:28:22.069 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/doc/coreutils.texi’: No data available 2024-03-09T09:28:22.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:22.843+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v1:172.21.15.134:6789/0 conn(0x7f6750157600 legacy=0x7f675015aa00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:28:22.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:22.843+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 msgr2=0x7f67500aac90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:23.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:22.843+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f67500aac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:28:23.082 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src' -> './k/coreutils-8.5/src' 2024-03-09T09:28:23.082 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/dircolors.hin' -> './k/coreutils-8.5/src/dircolors.hin' 2024-03-09T09:28:23.083 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dircolors.hin’: No data available 2024-03-09T09:28:23.085 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/test.c' -> './k/coreutils-8.5/src/test.c' 2024-03-09T09:28:23.085 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/test.c’: No data available 2024-03-09T09:28:23.087 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/nproc.c' -> './k/coreutils-8.5/src/nproc.c' 2024-03-09T09:28:23.088 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/nproc.c’: No data available 2024-03-09T09:28:23.090 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/cp-hash.c' -> './k/coreutils-8.5/src/cp-hash.c' 2024-03-09T09:28:23.090 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cp-hash.c’: No data available 2024-03-09T09:28:23.092 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/rm.c' -> './k/coreutils-8.5/src/rm.c' 2024-03-09T09:28:23.093 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/rm.c’: No data available 2024-03-09T09:28:23.095 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/uname-arch.c' -> './k/coreutils-8.5/src/uname-arch.c' 2024-03-09T09:28:23.095 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uname-arch.c’: No data available 2024-03-09T09:28:23.098 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/chroot.c' -> './k/coreutils-8.5/src/chroot.c' 2024-03-09T09:28:23.098 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chroot.c’: No data available 2024-03-09T09:28:23.100 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/remove.h' -> './k/coreutils-8.5/src/remove.h' 2024-03-09T09:28:23.101 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/remove.h’: No data available 2024-03-09T09:28:23.102 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/users.c' -> './k/coreutils-8.5/src/users.c' 2024-03-09T09:28:23.103 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/users.c’: No data available 2024-03-09T09:28:23.105 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/dircolors.c' -> './k/coreutils-8.5/src/dircolors.c' 2024-03-09T09:28:23.106 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dircolors.c’: No data available 2024-03-09T09:28:23.108 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/join.c' -> './k/coreutils-8.5/src/join.c' 2024-03-09T09:28:23.108 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/join.c’: No data available 2024-03-09T09:28:23.110 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/chown.c' -> './k/coreutils-8.5/src/chown.c' 2024-03-09T09:28:23.111 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chown.c’: No data available 2024-03-09T09:28:23.113 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/shuf.c' -> './k/coreutils-8.5/src/shuf.c' 2024-03-09T09:28:23.114 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/shuf.c’: No data available 2024-03-09T09:28:23.116 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/sum.c' -> './k/coreutils-8.5/src/sum.c' 2024-03-09T09:28:23.116 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/sum.c’: No data available 2024-03-09T09:28:23.119 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/csplit.c' -> './k/coreutils-8.5/src/csplit.c' 2024-03-09T09:28:23.119 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/csplit.c’: No data available 2024-03-09T09:28:23.121 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/libstdbuf.c' -> './k/coreutils-8.5/src/libstdbuf.c' 2024-03-09T09:28:23.121 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/libstdbuf.c’: No data available 2024-03-09T09:28:23.123 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/fold.c' -> './k/coreutils-8.5/src/fold.c' 2024-03-09T09:28:23.124 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/fold.c’: No data available 2024-03-09T09:28:23.126 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/basename.c' -> './k/coreutils-8.5/src/basename.c' 2024-03-09T09:28:23.126 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/basename.c’: No data available 2024-03-09T09:28:23.128 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/prog-fprintf.h' -> './k/coreutils-8.5/src/prog-fprintf.h' 2024-03-09T09:28:23.128 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/prog-fprintf.h’: No data available 2024-03-09T09:28:23.130 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/sync.c' -> './k/coreutils-8.5/src/sync.c' 2024-03-09T09:28:23.131 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/sync.c’: No data available 2024-03-09T09:28:23.133 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/lbracket.c' -> './k/coreutils-8.5/src/lbracket.c' 2024-03-09T09:28:23.133 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/lbracket.c’: No data available 2024-03-09T09:28:23.136 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/ln.c' -> './k/coreutils-8.5/src/ln.c' 2024-03-09T09:28:23.136 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ln.c’: No data available 2024-03-09T09:28:23.139 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/tsort.c' -> './k/coreutils-8.5/src/tsort.c' 2024-03-09T09:28:23.139 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tsort.c’: No data available 2024-03-09T09:28:23.141 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/wc.c' -> './k/coreutils-8.5/src/wc.c' 2024-03-09T09:28:23.142 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/wc.c’: No data available 2024-03-09T09:28:23.144 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/extract-magic' -> './k/coreutils-8.5/src/extract-magic' 2024-03-09T09:28:23.144 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/extract-magic’: No data available 2024-03-09T09:28:23.146 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/hostid.c' -> './k/coreutils-8.5/src/hostid.c' 2024-03-09T09:28:23.146 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/hostid.c’: No data available 2024-03-09T09:28:23.150 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/dd.c' -> './k/coreutils-8.5/src/dd.c' 2024-03-09T09:28:23.150 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dd.c’: No data available 2024-03-09T09:28:23.153 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/shred.c' -> './k/coreutils-8.5/src/shred.c' 2024-03-09T09:28:23.153 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/shred.c’: No data available 2024-03-09T09:28:23.155 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/uname.h' -> './k/coreutils-8.5/src/uname.h' 2024-03-09T09:28:23.156 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uname.h’: No data available 2024-03-09T09:28:23.158 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/unlink.c' -> './k/coreutils-8.5/src/unlink.c' 2024-03-09T09:28:23.158 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/unlink.c’: No data available 2024-03-09T09:28:23.161 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/Makefile.am' -> './k/coreutils-8.5/src/Makefile.am' 2024-03-09T09:28:23.161 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/Makefile.am’: No data available 2024-03-09T09:28:23.163 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/seq.c' -> './k/coreutils-8.5/src/seq.c' 2024-03-09T09:28:23.163 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/seq.c’: No data available 2024-03-09T09:28:23.165 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/uname-uname.c' -> './k/coreutils-8.5/src/uname-uname.c' 2024-03-09T09:28:23.165 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uname-uname.c’: No data available 2024-03-09T09:28:23.168 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/ptx.c' -> './k/coreutils-8.5/src/ptx.c' 2024-03-09T09:28:23.168 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ptx.c’: No data available 2024-03-09T09:28:23.170 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/wheel-size.h' -> './k/coreutils-8.5/src/wheel-size.h' 2024-03-09T09:28:23.170 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/wheel-size.h’: No data available 2024-03-09T09:28:23.172 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/stty.c' -> './k/coreutils-8.5/src/stty.c' 2024-03-09T09:28:23.173 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/stty.c’: No data available 2024-03-09T09:28:23.175 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/chcon.c' -> './k/coreutils-8.5/src/chcon.c' 2024-03-09T09:28:23.175 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chcon.c’: No data available 2024-03-09T09:28:23.178 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/dcgen' -> './k/coreutils-8.5/src/dcgen' 2024-03-09T09:28:23.179 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dcgen’: No data available 2024-03-09T09:28:23.181 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/uptime.c' -> './k/coreutils-8.5/src/uptime.c' 2024-03-09T09:28:23.181 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uptime.c’: No data available 2024-03-09T09:28:23.183 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/link.c' -> './k/coreutils-8.5/src/link.c' 2024-03-09T09:28:23.183 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/link.c’: No data available 2024-03-09T09:28:23.186 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/install.c' -> './k/coreutils-8.5/src/install.c' 2024-03-09T09:28:23.186 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/install.c’: No data available 2024-03-09T09:28:23.188 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/uname.c' -> './k/coreutils-8.5/src/uname.c' 2024-03-09T09:28:23.188 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uname.c’: No data available 2024-03-09T09:28:23.190 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/mknod.c' -> './k/coreutils-8.5/src/mknod.c' 2024-03-09T09:28:23.190 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/mknod.c’: No data available 2024-03-09T09:28:23.192 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/su.c' -> './k/coreutils-8.5/src/su.c' 2024-03-09T09:28:23.193 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/su.c’: No data available 2024-03-09T09:28:23.195 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/expand.c' -> './k/coreutils-8.5/src/expand.c' 2024-03-09T09:28:23.195 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/expand.c’: No data available 2024-03-09T09:28:23.197 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/mv.c' -> './k/coreutils-8.5/src/mv.c' 2024-03-09T09:28:23.197 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/mv.c’: No data available 2024-03-09T09:28:23.201 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/ls.c' -> './k/coreutils-8.5/src/ls.c' 2024-03-09T09:28:23.201 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ls.c’: No data available 2024-03-09T09:28:23.203 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/whoami.c' -> './k/coreutils-8.5/src/whoami.c' 2024-03-09T09:28:23.203 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/whoami.c’: No data available 2024-03-09T09:28:23.204 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/ls-ls.c' -> './k/coreutils-8.5/src/ls-ls.c' 2024-03-09T09:28:23.205 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ls-ls.c’: No data available 2024-03-09T09:28:23.207 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/ls.h' -> './k/coreutils-8.5/src/ls.h' 2024-03-09T09:28:23.207 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ls.h’: No data available 2024-03-09T09:28:23.209 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/id.c' -> './k/coreutils-8.5/src/id.c' 2024-03-09T09:28:23.209 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/id.c’: No data available 2024-03-09T09:28:23.213 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/Makefile.in' -> './k/coreutils-8.5/src/Makefile.in' 2024-03-09T09:28:23.213 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/Makefile.in’: No data available 2024-03-09T09:28:23.215 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/du.c' -> './k/coreutils-8.5/src/du.c' 2024-03-09T09:28:23.216 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/du.c’: No data available 2024-03-09T09:28:23.217 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/cut.c' -> './k/coreutils-8.5/src/cut.c' 2024-03-09T09:28:23.217 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cut.c’: No data available 2024-03-09T09:28:23.220 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/comm.c' -> './k/coreutils-8.5/src/comm.c' 2024-03-09T09:28:23.220 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/comm.c’: No data available 2024-03-09T09:28:23.222 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/od.c' -> './k/coreutils-8.5/src/od.c' 2024-03-09T09:28:23.223 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/od.c’: No data available 2024-03-09T09:28:23.224 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/expr.c' -> './k/coreutils-8.5/src/expr.c' 2024-03-09T09:28:23.224 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/expr.c’: No data available 2024-03-09T09:28:23.227 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/nohup.c' -> './k/coreutils-8.5/src/nohup.c' 2024-03-09T09:28:23.227 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/nohup.c’: No data available 2024-03-09T09:28:23.228 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/system.h' -> './k/coreutils-8.5/src/system.h' 2024-03-09T09:28:23.229 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/system.h’: No data available 2024-03-09T09:28:23.231 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/cat.c' -> './k/coreutils-8.5/src/cat.c' 2024-03-09T09:28:23.231 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cat.c’: No data available 2024-03-09T09:28:23.233 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/mktemp.c' -> './k/coreutils-8.5/src/mktemp.c' 2024-03-09T09:28:23.233 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/mktemp.c’: No data available 2024-03-09T09:28:23.234 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/tac.c' -> './k/coreutils-8.5/src/tac.c' 2024-03-09T09:28:23.235 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tac.c’: No data available 2024-03-09T09:28:23.237 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/ls-dir.c' -> './k/coreutils-8.5/src/ls-dir.c' 2024-03-09T09:28:23.237 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ls-dir.c’: No data available 2024-03-09T09:28:23.239 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/tac-pipe.c' -> './k/coreutils-8.5/src/tac-pipe.c' 2024-03-09T09:28:23.239 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tac-pipe.c’: No data available 2024-03-09T09:28:23.242 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/touch.c' -> './k/coreutils-8.5/src/touch.c' 2024-03-09T09:28:23.242 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/touch.c’: No data available 2024-03-09T09:28:23.244 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/false.c' -> './k/coreutils-8.5/src/false.c' 2024-03-09T09:28:23.244 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/false.c’: No data available 2024-03-09T09:28:23.246 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/echo.c' -> './k/coreutils-8.5/src/echo.c' 2024-03-09T09:28:23.246 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/echo.c’: No data available 2024-03-09T09:28:23.248 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/printenv.c' -> './k/coreutils-8.5/src/printenv.c' 2024-03-09T09:28:23.248 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/printenv.c’: No data available 2024-03-09T09:28:23.250 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/logname.c' -> './k/coreutils-8.5/src/logname.c' 2024-03-09T09:28:23.250 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/logname.c’: No data available 2024-03-09T09:28:23.251 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/group-list.c' -> './k/coreutils-8.5/src/group-list.c' 2024-03-09T09:28:23.252 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/group-list.c’: No data available 2024-03-09T09:28:23.254 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/pr.c' -> './k/coreutils-8.5/src/pr.c' 2024-03-09T09:28:23.254 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/pr.c’: No data available 2024-03-09T09:28:23.256 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/chgrp.c' -> './k/coreutils-8.5/src/chgrp.c' 2024-03-09T09:28:23.256 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chgrp.c’: No data available 2024-03-09T09:28:23.258 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/cp-hash.h' -> './k/coreutils-8.5/src/cp-hash.h' 2024-03-09T09:28:23.259 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cp-hash.h’: No data available 2024-03-09T09:28:23.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/getlimits.c' -> './k/coreutils-8.5/src/getlimits.c' 2024-03-09T09:28:23.261 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/getlimits.c’: No data available 2024-03-09T09:28:23.263 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/rmdir.c' -> './k/coreutils-8.5/src/rmdir.c' 2024-03-09T09:28:23.264 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/rmdir.c’: No data available 2024-03-09T09:28:23.266 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/tail.c' -> './k/coreutils-8.5/src/tail.c' 2024-03-09T09:28:23.267 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tail.c’: No data available 2024-03-09T09:28:23.269 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/timeout.c' -> './k/coreutils-8.5/src/timeout.c' 2024-03-09T09:28:23.269 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/timeout.c’: No data available 2024-03-09T09:28:23.272 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/sort.c' -> './k/coreutils-8.5/src/sort.c' 2024-03-09T09:28:23.272 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/sort.c’: No data available 2024-03-09T09:28:23.275 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/operand2sig.c' -> './k/coreutils-8.5/src/operand2sig.c' 2024-03-09T09:28:23.276 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/operand2sig.c’: No data available 2024-03-09T09:28:23.278 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/factor.c' -> './k/coreutils-8.5/src/factor.c' 2024-03-09T09:28:23.278 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/factor.c’: No data available 2024-03-09T09:28:23.280 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/paste.c' -> './k/coreutils-8.5/src/paste.c' 2024-03-09T09:28:23.281 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/paste.c’: No data available 2024-03-09T09:28:23.282 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/chmod.c' -> './k/coreutils-8.5/src/chmod.c' 2024-03-09T09:28:23.283 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chmod.c’: No data available 2024-03-09T09:28:23.285 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/pinky.c' -> './k/coreutils-8.5/src/pinky.c' 2024-03-09T09:28:23.285 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/pinky.c’: No data available 2024-03-09T09:28:23.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/unexpand.c' -> './k/coreutils-8.5/src/unexpand.c' 2024-03-09T09:28:23.287 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/unexpand.c’: No data available 2024-03-09T09:28:23.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/cksum.c' -> './k/coreutils-8.5/src/cksum.c' 2024-03-09T09:28:23.289 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cksum.c’: No data available 2024-03-09T09:28:23.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/date.c' -> './k/coreutils-8.5/src/date.c' 2024-03-09T09:28:23.291 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/date.c’: No data available 2024-03-09T09:28:23.292 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/mkdir.c' -> './k/coreutils-8.5/src/mkdir.c' 2024-03-09T09:28:23.292 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/mkdir.c’: No data available 2024-03-09T09:28:23.294 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/printf.c' -> './k/coreutils-8.5/src/printf.c' 2024-03-09T09:28:23.294 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/printf.c’: No data available 2024-03-09T09:28:23.296 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/true.c' -> './k/coreutils-8.5/src/true.c' 2024-03-09T09:28:23.296 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/true.c’: No data available 2024-03-09T09:28:23.298 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/c99-to-c89.diff' -> './k/coreutils-8.5/src/c99-to-c89.diff' 2024-03-09T09:28:23.299 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/c99-to-c89.diff’: No data available 2024-03-09T09:28:23.301 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/groups.c' -> './k/coreutils-8.5/src/groups.c' 2024-03-09T09:28:23.301 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/groups.c’: No data available 2024-03-09T09:28:23.304 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/dirname.c' -> './k/coreutils-8.5/src/dirname.c' 2024-03-09T09:28:23.304 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dirname.c’: No data available 2024-03-09T09:28:23.305 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/chown-core.h' -> './k/coreutils-8.5/src/chown-core.h' 2024-03-09T09:28:23.306 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chown-core.h’: No data available 2024-03-09T09:28:24.059 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/fs.h' -> './k/coreutils-8.5/src/fs.h' 2024-03-09T09:28:24.059 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/fs.h’: No data available 2024-03-09T09:28:24.061 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/nl.c' -> './k/coreutils-8.5/src/nl.c' 2024-03-09T09:28:24.061 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/nl.c’: No data available 2024-03-09T09:28:24.064 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/tee.c' -> './k/coreutils-8.5/src/tee.c' 2024-03-09T09:28:24.064 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tee.c’: No data available 2024-03-09T09:28:24.067 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/dircolors.h' -> './k/coreutils-8.5/src/dircolors.h' 2024-03-09T09:28:24.067 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/dircolors.h’: No data available 2024-03-09T09:28:24.069 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/head.c' -> './k/coreutils-8.5/src/head.c' 2024-03-09T09:28:24.070 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/head.c’: No data available 2024-03-09T09:28:24.072 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/uniq.c' -> './k/coreutils-8.5/src/uniq.c' 2024-03-09T09:28:24.072 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/uniq.c’: No data available 2024-03-09T09:28:24.075 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/chown-core.c' -> './k/coreutils-8.5/src/chown-core.c' 2024-03-09T09:28:24.076 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/chown-core.c’: No data available 2024-03-09T09:28:24.078 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/runcon.c' -> './k/coreutils-8.5/src/runcon.c' 2024-03-09T09:28:24.078 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/runcon.c’: No data available 2024-03-09T09:28:24.080 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/sleep.c' -> './k/coreutils-8.5/src/sleep.c' 2024-03-09T09:28:24.080 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/sleep.c’: No data available 2024-03-09T09:28:24.083 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/md5sum.c' -> './k/coreutils-8.5/src/md5sum.c' 2024-03-09T09:28:24.083 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/md5sum.c’: No data available 2024-03-09T09:28:24.085 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/yes.c' -> './k/coreutils-8.5/src/yes.c' 2024-03-09T09:28:24.085 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/yes.c’: No data available 2024-03-09T09:28:24.091 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/readlink.c' -> './k/coreutils-8.5/src/readlink.c' 2024-03-09T09:28:24.091 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/readlink.c’: No data available 2024-03-09T09:28:24.091 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/env.c' -> './k/coreutils-8.5/src/env.c' 2024-03-09T09:28:24.092 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/env.c’: No data available 2024-03-09T09:28:24.094 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/df.c' -> './k/coreutils-8.5/src/df.c' 2024-03-09T09:28:24.094 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/df.c’: No data available 2024-03-09T09:28:24.097 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/nice.c' -> './k/coreutils-8.5/src/nice.c' 2024-03-09T09:28:24.098 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/nice.c’: No data available 2024-03-09T09:28:24.101 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/fmt.c' -> './k/coreutils-8.5/src/fmt.c' 2024-03-09T09:28:24.102 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/fmt.c’: No data available 2024-03-09T09:28:24.104 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/base64.c' -> './k/coreutils-8.5/src/base64.c' 2024-03-09T09:28:24.104 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/base64.c’: No data available 2024-03-09T09:28:24.106 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/tty.c' -> './k/coreutils-8.5/src/tty.c' 2024-03-09T09:28:24.106 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tty.c’: No data available 2024-03-09T09:28:24.108 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/truncate.c' -> './k/coreutils-8.5/src/truncate.c' 2024-03-09T09:28:24.108 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/truncate.c’: No data available 2024-03-09T09:28:24.110 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/mkfifo.c' -> './k/coreutils-8.5/src/mkfifo.c' 2024-03-09T09:28:24.111 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/mkfifo.c’: No data available 2024-03-09T09:28:24.113 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/operand2sig.h' -> './k/coreutils-8.5/src/operand2sig.h' 2024-03-09T09:28:24.113 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/operand2sig.h’: No data available 2024-03-09T09:28:24.115 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/copy.h' -> './k/coreutils-8.5/src/copy.h' 2024-03-09T09:28:24.115 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/copy.h’: No data available 2024-03-09T09:28:24.117 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/kill.c' -> './k/coreutils-8.5/src/kill.c' 2024-03-09T09:28:24.117 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/kill.c’: No data available 2024-03-09T09:28:24.120 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/who.c' -> './k/coreutils-8.5/src/who.c' 2024-03-09T09:28:24.120 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/who.c’: No data available 2024-03-09T09:28:24.122 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/stat.c' -> './k/coreutils-8.5/src/stat.c' 2024-03-09T09:28:24.123 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/stat.c’: No data available 2024-03-09T09:28:24.124 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/remove.c' -> './k/coreutils-8.5/src/remove.c' 2024-03-09T09:28:24.125 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/remove.c’: No data available 2024-03-09T09:28:24.129 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/pwd.c' -> './k/coreutils-8.5/src/pwd.c' 2024-03-09T09:28:24.129 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/pwd.c’: No data available 2024-03-09T09:28:24.130 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/prog-fprintf.c' -> './k/coreutils-8.5/src/prog-fprintf.c' 2024-03-09T09:28:24.131 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/prog-fprintf.c’: No data available 2024-03-09T09:28:24.132 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/ls-vdir.c' -> './k/coreutils-8.5/src/ls-vdir.c' 2024-03-09T09:28:24.132 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/ls-vdir.c’: No data available 2024-03-09T09:28:24.135 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/copy.c' -> './k/coreutils-8.5/src/copy.c' 2024-03-09T09:28:24.135 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/copy.c’: No data available 2024-03-09T09:28:24.137 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/hostname.c' -> './k/coreutils-8.5/src/hostname.c' 2024-03-09T09:28:24.137 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/hostname.c’: No data available 2024-03-09T09:28:24.139 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/wheel-gen.pl' -> './k/coreutils-8.5/src/wheel-gen.pl' 2024-03-09T09:28:24.139 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/wheel-gen.pl’: No data available 2024-03-09T09:28:24.142 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/group-list.h' -> './k/coreutils-8.5/src/group-list.h' 2024-03-09T09:28:24.142 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/group-list.h’: No data available 2024-03-09T09:28:24.144 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/cp.c' -> './k/coreutils-8.5/src/cp.c' 2024-03-09T09:28:24.144 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/cp.c’: No data available 2024-03-09T09:28:24.146 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/setuidgid.c' -> './k/coreutils-8.5/src/setuidgid.c' 2024-03-09T09:28:24.146 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/setuidgid.c’: No data available 2024-03-09T09:28:24.150 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/tr.c' -> './k/coreutils-8.5/src/tr.c' 2024-03-09T09:28:24.150 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/tr.c’: No data available 2024-03-09T09:28:24.152 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/stdbuf.c' -> './k/coreutils-8.5/src/stdbuf.c' 2024-03-09T09:28:24.152 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/stdbuf.c’: No data available 2024-03-09T09:28:24.153 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/pathchk.c' -> './k/coreutils-8.5/src/pathchk.c' 2024-03-09T09:28:24.154 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/pathchk.c’: No data available 2024-03-09T09:28:24.156 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/wheel.h' -> './k/coreutils-8.5/src/wheel.h' 2024-03-09T09:28:24.156 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/wheel.h’: No data available 2024-03-09T09:28:24.158 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/src/split.c' -> './k/coreutils-8.5/src/split.c' 2024-03-09T09:28:24.158 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/src/split.c’: No data available 2024-03-09T09:28:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.440+0000 7f674d600640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 msgr2=0x7f67500aac90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.440+0000 7f674d600640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f67500aac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.440+0000 7f674d600640 1 -- 172.21.15.67:0/2130994072 >> v1:172.21.15.134:6789/0 conn(0x7f6750157600 legacy=0x7f675015aa00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.440+0000 7f674d600640 1 -- 172.21.15.67:0/2130994072 >> v1:172.21.15.67:6789/0 conn(0x7f67500c3600 legacy=0x7f67500ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.440+0000 7f674d600640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.440+0000 7f674d600640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.440+0000 7f674d600640 1 -- 172.21.15.67:0/2130994072 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675006d7a0 con 0x7f67300036b0 2024-03-09T09:28:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.440+0000 7f674d600640 1 -- 172.21.15.67:0/2130994072 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6730008ce0 con 0x7f6730006120 2024-03-09T09:28:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.441+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.441+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.441+0000 7f674f400640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 msgr2=0x7f67300085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:28:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.441+0000 7f674f400640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 msgr2=0x7f67300085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.441+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.441+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.442+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f6730005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:28:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.442+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f6730005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:24.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.442+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:24.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.442+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:24.560 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_require_config_h' -> './k/coreutils-8.5/.x-sc_require_config_h' 2024-03-09T09:28:24.561 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_require_config_h’: No data available 2024-03-09T09:28:24.562 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_system_h_headers' -> './k/coreutils-8.5/.x-sc_system_h_headers' 2024-03-09T09:28:24.563 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_system_h_headers’: No data available 2024-03-09T09:28:24.565 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_fail_0' -> './k/coreutils-8.5/.x-sc_prohibit_fail_0' 2024-03-09T09:28:24.565 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_fail_0’: No data available 2024-03-09T09:28:24.567 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_prohibit_magic_number_exit' -> './k/coreutils-8.5/.x-sc_prohibit_magic_number_exit' 2024-03-09T09:28:24.568 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_prohibit_magic_number_exit’: No data available 2024-03-09T09:28:24.570 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/THANKS-to-translators' -> './k/coreutils-8.5/THANKS-to-translators' 2024-03-09T09:28:24.570 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/THANKS-to-translators’: No data available 2024-03-09T09:28:24.572 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/.x-sc_useless_cpp_parens' -> './k/coreutils-8.5/.x-sc_useless_cpp_parens' 2024-03-09T09:28:24.572 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/.x-sc_useless_cpp_parens’: No data available 2024-03-09T09:28:24.589 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/configure' -> './k/coreutils-8.5/configure' 2024-03-09T09:28:24.590 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/configure’: No data available 2024-03-09T09:28:24.630 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests' -> './k/coreutils-8.5/gnulib-tests' 2024-03-09T09:28:24.631 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-ftello.sh' -> './k/coreutils-8.5/gnulib-tests/test-ftello.sh' 2024-03-09T09:28:24.631 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-ftello.sh’: No data available 2024-03-09T09:28:24.633 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-snprintf.c' -> './k/coreutils-8.5/gnulib-tests/test-snprintf.c' 2024-03-09T09:28:24.633 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-snprintf.c’: No data available 2024-03-09T09:28:24.636 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-openat.c' -> './k/coreutils-8.5/gnulib-tests/test-openat.c' 2024-03-09T09:28:24.636 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-openat.c’: No data available 2024-03-09T09:28:24.638 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-dirent-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-dirent-safer.c' 2024-03-09T09:28:24.638 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-dirent-safer.c’: No data available 2024-03-09T09:28:24.641 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-cloexec.c' -> './k/coreutils-8.5/gnulib-tests/test-cloexec.c' 2024-03-09T09:28:24.641 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-cloexec.c’: No data available 2024-03-09T09:28:24.643 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fstatat.c' -> './k/coreutils-8.5/gnulib-tests/test-fstatat.c' 2024-03-09T09:28:24.644 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fstatat.c’: No data available 2024-03-09T09:28:24.646 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fnmatch.c' -> './k/coreutils-8.5/gnulib-tests/test-fnmatch.c' 2024-03-09T09:28:24.646 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fnmatch.c’: No data available 2024-03-09T09:28:24.648 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getndelim2.c' -> './k/coreutils-8.5/gnulib-tests/test-getndelim2.c' 2024-03-09T09:28:24.649 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getndelim2.c’: No data available 2024-03-09T09:28:24.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.643+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:24.651 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-wcrtomb.sh' -> './k/coreutils-8.5/gnulib-tests/test-wcrtomb.sh' 2024-03-09T09:28:24.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.643+0000 7f674f400640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 msgr2=0x7f67300085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:28:24.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.643+0000 7f674f400640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 msgr2=0x7f67300085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:24.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.643+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:24.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.643+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:28:24.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.643+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:24.652 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-wcrtomb.sh’: No data available 2024-03-09T09:28:24.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.644+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f6730005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:28:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.644+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f6730005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.644+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:24.644+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:28:24.653 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-filenamecat.c' -> './k/coreutils-8.5/gnulib-tests/test-filenamecat.c' 2024-03-09T09:28:24.654 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-filenamecat.c’: No data available 2024-03-09T09:28:24.656 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fchownat.c' -> './k/coreutils-8.5/gnulib-tests/test-fchownat.c' 2024-03-09T09:28:24.656 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fchownat.c’: No data available 2024-03-09T09:28:24.658 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stdlib.c' -> './k/coreutils-8.5/gnulib-tests/test-stdlib.c' 2024-03-09T09:28:24.658 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stdlib.c’: No data available 2024-03-09T09:28:24.666 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-read-file.c' -> './k/coreutils-8.5/gnulib-tests/test-read-file.c' 2024-03-09T09:28:24.666 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-read-file.c’: No data available 2024-03-09T09:28:24.668 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude7.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude7.sh' 2024-03-09T09:28:24.669 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude7.sh’: No data available 2024-03-09T09:28:24.671 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-closein.c' -> './k/coreutils-8.5/gnulib-tests/test-closein.c' 2024-03-09T09:28:24.671 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-closein.c’: No data available 2024-03-09T09:28:24.674 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/connect.c' -> './k/coreutils-8.5/gnulib-tests/connect.c' 2024-03-09T09:28:24.674 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/connect.c’: No data available 2024-03-09T09:28:24.676 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-rename.h' -> './k/coreutils-8.5/gnulib-tests/test-rename.h' 2024-03-09T09:28:24.677 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-rename.h’: No data available 2024-03-09T09:28:24.679 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-memchr2.c' -> './k/coreutils-8.5/gnulib-tests/test-memchr2.c' 2024-03-09T09:28:24.679 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-memchr2.c’: No data available 2024-03-09T09:28:24.681 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_time.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_time.c' 2024-03-09T09:28:24.681 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_time.c’: No data available 2024-03-09T09:28:24.683 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/read-file.h' -> './k/coreutils-8.5/gnulib-tests/read-file.h' 2024-03-09T09:28:24.683 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/read-file.h’: No data available 2024-03-09T09:28:24.686 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mkdirat.c' -> './k/coreutils-8.5/gnulib-tests/test-mkdirat.c' 2024-03-09T09:28:24.686 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mkdirat.c’: No data available 2024-03-09T09:28:24.687 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnand-nolibm.c' -> './k/coreutils-8.5/gnulib-tests/test-isnand-nolibm.c' 2024-03-09T09:28:24.688 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnand-nolibm.c’: No data available 2024-03-09T09:28:24.690 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fopen-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-fopen-safer.c' 2024-03-09T09:28:24.690 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fopen-safer.c’: No data available 2024-03-09T09:28:24.692 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stdio.c' -> './k/coreutils-8.5/gnulib-tests/test-stdio.c' 2024-03-09T09:28:24.692 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stdio.c’: No data available 2024-03-09T09:28:24.694 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fseeko.c' -> './k/coreutils-8.5/gnulib-tests/test-fseeko.c' 2024-03-09T09:28:24.695 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fseeko.c’: No data available 2024-03-09T09:28:24.697 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-arpa_inet.c' -> './k/coreutils-8.5/gnulib-tests/test-arpa_inet.c' 2024-03-09T09:28:24.697 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-arpa_inet.c’: No data available 2024-03-09T09:28:24.700 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadahead.sh' -> './k/coreutils-8.5/gnulib-tests/test-freadahead.sh' 2024-03-09T09:28:24.700 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadahead.sh’: No data available 2024-03-09T09:28:24.702 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-openat-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-openat-safer.c' 2024-03-09T09:28:24.703 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-openat-safer.c’: No data available 2024-03-09T09:28:24.705 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-unlink.c' -> './k/coreutils-8.5/gnulib-tests/test-unlink.c' 2024-03-09T09:28:24.705 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-unlink.c’: No data available 2024-03-09T09:28:24.707 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsstr2.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsstr2.sh' 2024-03-09T09:28:24.707 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsstr2.sh’: No data available 2024-03-09T09:28:24.709 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lchown.c' -> './k/coreutils-8.5/gnulib-tests/test-lchown.c' 2024-03-09T09:28:24.710 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lchown.c’: No data available 2024-03-09T09:28:24.712 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-nl_langinfo.c' -> './k/coreutils-8.5/gnulib-tests/test-nl_langinfo.c' 2024-03-09T09:28:24.712 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-nl_langinfo.c’: No data available 2024-03-09T09:28:24.714 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-open.h' -> './k/coreutils-8.5/gnulib-tests/test-open.h' 2024-03-09T09:28:24.714 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-open.h’: No data available 2024-03-09T09:28:24.717 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-memcmp.c' -> './k/coreutils-8.5/gnulib-tests/test-memcmp.c' 2024-03-09T09:28:24.717 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-memcmp.c’: No data available 2024-03-09T09:28:24.719 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.c' -> './k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.c' 2024-03-09T09:28:24.719 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.c’: No data available 2024-03-09T09:28:24.721 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadahead.c' -> './k/coreutils-8.5/gnulib-tests/test-freadahead.c' 2024-03-09T09:28:24.722 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadahead.c’: No data available 2024-03-09T09:28:24.724 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/ioctl.c' -> './k/coreutils-8.5/gnulib-tests/ioctl.c' 2024-03-09T09:28:24.724 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/ioctl.c’: No data available 2024-03-09T09:28:24.727 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-iconv-h.c' -> './k/coreutils-8.5/gnulib-tests/test-iconv-h.c' 2024-03-09T09:28:24.727 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-iconv-h.c’: No data available 2024-03-09T09:28:24.729 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mkdir.c' -> './k/coreutils-8.5/gnulib-tests/test-mkdir.c' 2024-03-09T09:28:24.729 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mkdir.c’: No data available 2024-03-09T09:28:24.731 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-posixtm.c' -> './k/coreutils-8.5/gnulib-tests/test-posixtm.c' 2024-03-09T09:28:24.732 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-posixtm.c’: No data available 2024-03-09T09:28:24.734 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/wctob.c' -> './k/coreutils-8.5/gnulib-tests/wctob.c' 2024-03-09T09:28:24.734 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/wctob.c’: No data available 2024-03-09T09:28:24.736 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude5.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude5.sh' 2024-03-09T09:28:24.737 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude5.sh’: No data available 2024-03-09T09:28:24.739 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-rmdir.h' -> './k/coreutils-8.5/gnulib-tests/test-rmdir.h' 2024-03-09T09:28:24.739 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-rmdir.h’: No data available 2024-03-09T09:28:24.741 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-symlink.h' -> './k/coreutils-8.5/gnulib-tests/test-symlink.h' 2024-03-09T09:28:24.742 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-symlink.h’: No data available 2024-03-09T09:28:24.744 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-unsetenv.c' -> './k/coreutils-8.5/gnulib-tests/test-unsetenv.c' 2024-03-09T09:28:24.744 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-unsetenv.c’: No data available 2024-03-09T09:28:24.746 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude4.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude4.sh' 2024-03-09T09:28:24.746 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude4.sh’: No data available 2024-03-09T09:28:24.748 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/listen.c' -> './k/coreutils-8.5/gnulib-tests/listen.c' 2024-03-09T09:28:24.749 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/listen.c’: No data available 2024-03-09T09:28:24.750 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-printf-posix.output' -> './k/coreutils-8.5/gnulib-tests/test-printf-posix.output' 2024-03-09T09:28:24.751 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-printf-posix.output’: No data available 2024-03-09T09:28:24.753 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbrtowc3.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbrtowc3.sh' 2024-03-09T09:28:24.754 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbrtowc3.sh’: No data available 2024-03-09T09:28:25.004 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-dirent.c' -> './k/coreutils-8.5/gnulib-tests/test-dirent.c' 2024-03-09T09:28:25.005 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-dirent.c’: No data available 2024-03-09T09:28:25.006 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-utimens-common.h' -> './k/coreutils-8.5/gnulib-tests/test-utimens-common.h' 2024-03-09T09:28:25.006 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-utimens-common.h’: No data available 2024-03-09T09:28:25.008 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsinit.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsinit.c' 2024-03-09T09:28:25.008 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsinit.c’: No data available 2024-03-09T09:28:25.010 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/w32sock.h' -> './k/coreutils-8.5/gnulib-tests/w32sock.h' 2024-03-09T09:28:25.010 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/w32sock.h’: No data available 2024-03-09T09:28:25.012 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-printf-posix.h' -> './k/coreutils-8.5/gnulib-tests/test-printf-posix.h' 2024-03-09T09:28:25.012 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-printf-posix.h’: No data available 2024-03-09T09:28:25.014 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fchdir.c' -> './k/coreutils-8.5/gnulib-tests/test-fchdir.c' 2024-03-09T09:28:25.014 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fchdir.c’: No data available 2024-03-09T09:28:25.016 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/read-file.c' -> './k/coreutils-8.5/gnulib-tests/read-file.c' 2024-03-09T09:28:25.017 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/read-file.c’: No data available 2024-03-09T09:28:25.019 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-ctype.c' -> './k/coreutils-8.5/gnulib-tests/test-ctype.c' 2024-03-09T09:28:25.019 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-ctype.c’: No data available 2024-03-09T09:28:25.021 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/binary-io.h' -> './k/coreutils-8.5/gnulib-tests/binary-io.h' 2024-03-09T09:28:25.021 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/binary-io.h’: No data available 2024-03-09T09:28:25.023 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/nap.h' -> './k/coreutils-8.5/gnulib-tests/nap.h' 2024-03-09T09:28:25.023 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/nap.h’: No data available 2024-03-09T09:28:25.025 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/sys_ioctl.in.h' -> './k/coreutils-8.5/gnulib-tests/sys_ioctl.in.h' 2024-03-09T09:28:25.025 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/sys_ioctl.in.h’: No data available 2024-03-09T09:28:25.027 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-ftello2.sh' -> './k/coreutils-8.5/gnulib-tests/test-ftello2.sh' 2024-03-09T09:28:25.027 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-ftello2.sh’: No data available 2024-03-09T09:28:25.028 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-memchr.c' -> './k/coreutils-8.5/gnulib-tests/test-memchr.c' 2024-03-09T09:28:25.029 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-memchr.c’: No data available 2024-03-09T09:28:25.030 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsrtowcs3.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsrtowcs3.sh' 2024-03-09T09:28:25.030 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsrtowcs3.sh’: No data available 2024-03-09T09:28:25.033 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fseeko.sh' -> './k/coreutils-8.5/gnulib-tests/test-fseeko.sh' 2024-03-09T09:28:25.033 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fseeko.sh’: No data available 2024-03-09T09:28:25.034 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-select-out.sh' -> './k/coreutils-8.5/gnulib-tests/test-select-out.sh' 2024-03-09T09:28:25.035 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-select-out.sh’: No data available 2024-03-09T09:28:25.037 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/inet_pton.c' -> './k/coreutils-8.5/gnulib-tests/inet_pton.c' 2024-03-09T09:28:25.037 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/inet_pton.c’: No data available 2024-03-09T09:28:25.039 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stdbool.c' -> './k/coreutils-8.5/gnulib-tests/test-stdbool.c' 2024-03-09T09:28:25.040 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stdbool.c’: No data available 2024-03-09T09:28:25.041 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-nanosleep.c' -> './k/coreutils-8.5/gnulib-tests/test-nanosleep.c' 2024-03-09T09:28:25.042 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-nanosleep.c’: No data available 2024-03-09T09:28:25.043 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/bind.c' -> './k/coreutils-8.5/gnulib-tests/bind.c' 2024-03-09T09:28:25.044 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/bind.c’: No data available 2024-03-09T09:28:25.046 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/unlinkdir.h' -> './k/coreutils-8.5/gnulib-tests/unlinkdir.h' 2024-03-09T09:28:25.046 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/unlinkdir.h’: No data available 2024-03-09T09:28:25.048 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getline.c' -> './k/coreutils-8.5/gnulib-tests/test-getline.c' 2024-03-09T09:28:25.048 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getline.c’: No data available 2024-03-09T09:28:25.050 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-md5.c' -> './k/coreutils-8.5/gnulib-tests/test-md5.c' 2024-03-09T09:28:25.050 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-md5.c’: No data available 2024-03-09T09:28:25.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.044+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:25.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.045+0000 7f674f400640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 msgr2=0x7f67300085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:28:25.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.045+0000 7f674f400640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 msgr2=0x7f67300085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:25.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.045+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:25.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.045+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:28:25.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.045+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.046+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f6730005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-09T09:28:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.046+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f6730005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.046+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.046+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:28:25.058 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/Makefile.am' -> './k/coreutils-8.5/gnulib-tests/Makefile.am' 2024-03-09T09:28:25.058 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/Makefile.am’: No data available 2024-03-09T09:28:25.060 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-select-in.sh' -> './k/coreutils-8.5/gnulib-tests/test-select-in.sh' 2024-03-09T09:28:25.060 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-select-in.sh’: No data available 2024-03-09T09:28:25.062 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-uname.c' -> './k/coreutils-8.5/gnulib-tests/test-uname.c' 2024-03-09T09:28:25.062 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-uname.c’: No data available 2024-03-09T09:28:25.064 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-readlink.h' -> './k/coreutils-8.5/gnulib-tests/test-readlink.h' 2024-03-09T09:28:25.064 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-readlink.h’: No data available 2024-03-09T09:28:25.066 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-string.c' -> './k/coreutils-8.5/gnulib-tests/test-string.c' 2024-03-09T09:28:25.066 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-string.c’: No data available 2024-03-09T09:28:25.068 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-argmatch.c' -> './k/coreutils-8.5/gnulib-tests/test-argmatch.c' 2024-03-09T09:28:25.068 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-argmatch.c’: No data available 2024-03-09T09:28:25.069 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-nl_langinfo.sh' -> './k/coreutils-8.5/gnulib-tests/test-nl_langinfo.sh' 2024-03-09T09:28:25.069 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-nl_langinfo.sh’: No data available 2024-03-09T09:28:25.072 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_stat.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_stat.c' 2024-03-09T09:28:25.072 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_stat.c’: No data available 2024-03-09T09:28:25.073 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-version-etc.c' -> './k/coreutils-8.5/gnulib-tests/test-version-etc.c' 2024-03-09T09:28:25.074 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-version-etc.c’: No data available 2024-03-09T09:28:25.075 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-areadlink.h' -> './k/coreutils-8.5/gnulib-tests/test-areadlink.h' 2024-03-09T09:28:25.076 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-areadlink.h’: No data available 2024-03-09T09:28:25.078 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-version-etc.sh' -> './k/coreutils-8.5/gnulib-tests/test-version-etc.sh' 2024-03-09T09:28:25.078 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-version-etc.sh’: No data available 2024-03-09T09:28:25.080 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-binary-io.c' -> './k/coreutils-8.5/gnulib-tests/test-binary-io.c' 2024-03-09T09:28:25.080 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-binary-io.c’: No data available 2024-03-09T09:28:25.081 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude2.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude2.sh' 2024-03-09T09:28:25.082 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude2.sh’: No data available 2024-03-09T09:28:25.083 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xfprintf-posix.c' -> './k/coreutils-8.5/gnulib-tests/test-xfprintf-posix.c' 2024-03-09T09:28:25.084 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xfprintf-posix.c’: No data available 2024-03-09T09:28:25.085 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-set-mode-acl.c' -> './k/coreutils-8.5/gnulib-tests/test-set-mode-acl.c' 2024-03-09T09:28:25.086 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-set-mode-acl.c’: No data available 2024-03-09T09:28:25.087 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-wchar.c' -> './k/coreutils-8.5/gnulib-tests/test-wchar.c' 2024-03-09T09:28:25.087 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-wchar.c’: No data available 2024-03-09T09:28:25.089 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vasprintf.c' -> './k/coreutils-8.5/gnulib-tests/test-vasprintf.c' 2024-03-09T09:28:25.089 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vasprintf.c’: No data available 2024-03-09T09:28:25.091 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtoimax.sh' -> './k/coreutils-8.5/gnulib-tests/test-xstrtoimax.sh' 2024-03-09T09:28:25.091 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtoimax.sh’: No data available 2024-03-09T09:28:25.093 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadseek.sh' -> './k/coreutils-8.5/gnulib-tests/test-freadseek.sh' 2024-03-09T09:28:25.093 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadseek.sh’: No data available 2024-03-09T09:28:25.095 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isblank.c' -> './k/coreutils-8.5/gnulib-tests/test-isblank.c' 2024-03-09T09:28:25.095 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isblank.c’: No data available 2024-03-09T09:28:25.097 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fpending.sh' -> './k/coreutils-8.5/gnulib-tests/test-fpending.sh' 2024-03-09T09:28:25.097 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fpending.sh’: No data available 2024-03-09T09:28:25.099 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lstat.h' -> './k/coreutils-8.5/gnulib-tests/test-lstat.h' 2024-03-09T09:28:25.099 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lstat.h’: No data available 2024-03-09T09:28:25.101 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-chown.c' -> './k/coreutils-8.5/gnulib-tests/test-chown.c' 2024-03-09T09:28:25.101 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-chown.c’: No data available 2024-03-09T09:28:25.103 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtoumax.sh' -> './k/coreutils-8.5/gnulib-tests/test-xstrtoumax.sh' 2024-03-09T09:28:25.103 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtoumax.sh’: No data available 2024-03-09T09:28:25.105 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/nan.h' -> './k/coreutils-8.5/gnulib-tests/nan.h' 2024-03-09T09:28:25.105 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/nan.h’: No data available 2024-03-09T09:28:25.106 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xprintf-posix.c' -> './k/coreutils-8.5/gnulib-tests/test-xprintf-posix.c' 2024-03-09T09:28:25.107 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xprintf-posix.c’: No data available 2024-03-09T09:28:25.108 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-inet_pton.c' -> './k/coreutils-8.5/gnulib-tests/test-inet_pton.c' 2024-03-09T09:28:25.108 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-inet_pton.c’: No data available 2024-03-09T09:28:25.110 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-dup-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-dup-safer.c' 2024-03-09T09:28:25.110 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-dup-safer.c’: No data available 2024-03-09T09:28:25.112 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-closein.sh' -> './k/coreutils-8.5/gnulib-tests/test-closein.sh' 2024-03-09T09:28:25.112 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-closein.sh’: No data available 2024-03-09T09:28:25.114 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-canonicalize.c' -> './k/coreutils-8.5/gnulib-tests/test-canonicalize.c' 2024-03-09T09:28:25.115 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-canonicalize.c’: No data available 2024-03-09T09:28:25.116 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnanl-nolibm.c' -> './k/coreutils-8.5/gnulib-tests/test-isnanl-nolibm.c' 2024-03-09T09:28:25.116 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnanl-nolibm.c’: No data available 2024-03-09T09:28:25.118 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtol.c' -> './k/coreutils-8.5/gnulib-tests/test-xstrtol.c' 2024-03-09T09:28:25.118 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtol.c’: No data available 2024-03-09T09:28:25.120 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-open.c' -> './k/coreutils-8.5/gnulib-tests/test-open.c' 2024-03-09T09:28:25.120 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-open.c’: No data available 2024-03-09T09:28:25.122 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fprintf-posix.h' -> './k/coreutils-8.5/gnulib-tests/test-fprintf-posix.h' 2024-03-09T09:28:25.122 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fprintf-posix.h’: No data available 2024-03-09T09:28:25.124 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbrtowc1.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbrtowc1.sh' 2024-03-09T09:28:25.124 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbrtowc1.sh’: No data available 2024-03-09T09:28:25.126 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-errno.c' -> './k/coreutils-8.5/gnulib-tests/test-errno.c' 2024-03-09T09:28:25.126 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-errno.c’: No data available 2024-03-09T09:28:25.212 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-netdb.c' -> './k/coreutils-8.5/gnulib-tests/test-netdb.c' 2024-03-09T09:28:25.212 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-netdb.c’: No data available 2024-03-09T09:28:25.214 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-wctype.c' -> './k/coreutils-8.5/gnulib-tests/test-wctype.c' 2024-03-09T09:28:25.214 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-wctype.c’: No data available 2024-03-09T09:28:25.216 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-strsignal.c' -> './k/coreutils-8.5/gnulib-tests/test-strsignal.c' 2024-03-09T09:28:25.216 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-strsignal.c’: No data available 2024-03-09T09:28:25.218 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsstr3.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsstr3.sh' 2024-03-09T09:28:25.218 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsstr3.sh’: No data available 2024-03-09T09:28:25.220 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-yesno.c' -> './k/coreutils-8.5/gnulib-tests/test-yesno.c' 2024-03-09T09:28:25.220 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-yesno.c’: No data available 2024-03-09T09:28:25.223 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-userspec.c' -> './k/coreutils-8.5/gnulib-tests/test-userspec.c' 2024-03-09T09:28:25.223 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-userspec.c’: No data available 2024-03-09T09:28:25.225 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freopen.c' -> './k/coreutils-8.5/gnulib-tests/test-freopen.c' 2024-03-09T09:28:25.225 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freopen.c’: No data available 2024-03-09T09:28:25.227 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sockets.c' -> './k/coreutils-8.5/gnulib-tests/test-sockets.c' 2024-03-09T09:28:25.228 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sockets.c’: No data available 2024-03-09T09:28:25.230 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-link.h' -> './k/coreutils-8.5/gnulib-tests/test-link.h' 2024-03-09T09:28:25.230 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-link.h’: No data available 2024-03-09T09:28:25.232 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-select-fd.c' -> './k/coreutils-8.5/gnulib-tests/test-select-fd.c' 2024-03-09T09:28:25.232 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-select-fd.c’: No data available 2024-03-09T09:28:25.234 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtoimax.c' -> './k/coreutils-8.5/gnulib-tests/test-xstrtoimax.c' 2024-03-09T09:28:25.235 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtoimax.c’: No data available 2024-03-09T09:28:25.236 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xvasprintf.c' -> './k/coreutils-8.5/gnulib-tests/test-xvasprintf.c' 2024-03-09T09:28:25.236 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xvasprintf.c’: No data available 2024-03-09T09:28:25.239 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-yesno.sh' -> './k/coreutils-8.5/gnulib-tests/test-yesno.sh' 2024-03-09T09:28:25.239 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-yesno.sh’: No data available 2024-03-09T09:28:25.241 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fflush.c' -> './k/coreutils-8.5/gnulib-tests/test-fflush.c' 2024-03-09T09:28:25.241 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fflush.c’: No data available 2024-03-09T09:28:25.243 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vc-list-files-cvs.sh' -> './k/coreutils-8.5/gnulib-tests/test-vc-list-files-cvs.sh' 2024-03-09T09:28:25.243 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vc-list-files-cvs.sh’: No data available 2024-03-09T09:28:25.245 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbscasecmp.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbscasecmp.sh' 2024-03-09T09:28:25.246 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbscasecmp.sh’: No data available 2024-03-09T09:28:25.248 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lseek.sh' -> './k/coreutils-8.5/gnulib-tests/test-lseek.sh' 2024-03-09T09:28:25.248 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lseek.sh’: No data available 2024-03-09T09:28:25.250 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/usleep.c' -> './k/coreutils-8.5/gnulib-tests/usleep.c' 2024-03-09T09:28:25.250 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/usleep.c’: No data available 2024-03-09T09:28:25.252 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sleep.c' -> './k/coreutils-8.5/gnulib-tests/test-sleep.c' 2024-03-09T09:28:25.252 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sleep.c’: No data available 2024-03-09T09:28:25.254 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-link.c' -> './k/coreutils-8.5/gnulib-tests/test-link.c' 2024-03-09T09:28:25.255 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-link.c’: No data available 2024-03-09T09:28:25.257 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-perror.c' -> './k/coreutils-8.5/gnulib-tests/test-perror.c' 2024-03-09T09:28:25.257 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-perror.c’: No data available 2024-03-09T09:28:25.259 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsrtowcs.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsrtowcs.c' 2024-03-09T09:28:25.259 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsrtowcs.c’: No data available 2024-03-09T09:28:25.261 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xalloc-die.sh' -> './k/coreutils-8.5/gnulib-tests/test-xalloc-die.sh' 2024-03-09T09:28:25.261 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xalloc-die.sh’: No data available 2024-03-09T09:28:25.263 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-i-ring.c' -> './k/coreutils-8.5/gnulib-tests/test-i-ring.c' 2024-03-09T09:28:25.264 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-i-ring.c’: No data available 2024-03-09T09:28:25.266 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-priv-set.c' -> './k/coreutils-8.5/gnulib-tests/test-priv-set.c' 2024-03-09T09:28:25.266 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-priv-set.c’: No data available 2024-03-09T09:28:25.268 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-btowc.c' -> './k/coreutils-8.5/gnulib-tests/test-btowc.c' 2024-03-09T09:28:25.268 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-btowc.c’: No data available 2024-03-09T09:28:25.270 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_select.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_select.c' 2024-03-09T09:28:25.270 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_select.c’: No data available 2024-03-09T09:28:25.272 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbrtowc.c' -> './k/coreutils-8.5/gnulib-tests/test-mbrtowc.c' 2024-03-09T09:28:25.273 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbrtowc.c’: No data available 2024-03-09T09:28:25.275 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-strerror.c' -> './k/coreutils-8.5/gnulib-tests/test-strerror.c' 2024-03-09T09:28:25.275 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-strerror.c’: No data available 2024-03-09T09:28:25.277 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-update-copyright.sh' -> './k/coreutils-8.5/gnulib-tests/test-update-copyright.sh' 2024-03-09T09:28:25.277 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-update-copyright.sh’: No data available 2024-03-09T09:28:25.279 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-btowc2.sh' -> './k/coreutils-8.5/gnulib-tests/test-btowc2.sh' 2024-03-09T09:28:25.279 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-btowc2.sh’: No data available 2024-03-09T09:28:25.281 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getcwd.c' -> './k/coreutils-8.5/gnulib-tests/test-getcwd.c' 2024-03-09T09:28:25.281 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getcwd.c’: No data available 2024-03-09T09:28:25.283 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/socket.c' -> './k/coreutils-8.5/gnulib-tests/socket.c' 2024-03-09T09:28:25.284 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/socket.c’: No data available 2024-03-09T09:28:25.286 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-set-mode-acl.sh' -> './k/coreutils-8.5/gnulib-tests/test-set-mode-acl.sh' 2024-03-09T09:28:25.286 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-set-mode-acl.sh’: No data available 2024-03-09T09:28:25.288 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-linkat.c' -> './k/coreutils-8.5/gnulib-tests/test-linkat.c' 2024-03-09T09:28:25.288 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-linkat.c’: No data available 2024-03-09T09:28:25.290 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/perror.c' -> './k/coreutils-8.5/gnulib-tests/perror.c' 2024-03-09T09:28:25.290 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/perror.c’: No data available 2024-03-09T09:28:25.292 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-wcrtomb.c' -> './k/coreutils-8.5/gnulib-tests/test-wcrtomb.c' 2024-03-09T09:28:25.292 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-wcrtomb.c’: No data available 2024-03-09T09:28:25.294 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vasnprintf.c' -> './k/coreutils-8.5/gnulib-tests/test-vasnprintf.c' 2024-03-09T09:28:25.294 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vasnprintf.c’: No data available 2024-03-09T09:28:25.297 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-memrchr.c' -> './k/coreutils-8.5/gnulib-tests/test-memrchr.c' 2024-03-09T09:28:25.297 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-memrchr.c’: No data available 2024-03-09T09:28:25.299 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-ftello.c' -> './k/coreutils-8.5/gnulib-tests/test-ftello.c' 2024-03-09T09:28:25.299 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-ftello.c’: No data available 2024-03-09T09:28:25.301 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-chown.h' -> './k/coreutils-8.5/gnulib-tests/test-chown.h' 2024-03-09T09:28:25.302 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-chown.h’: No data available 2024-03-09T09:28:25.304 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stat.h' -> './k/coreutils-8.5/gnulib-tests/test-stat.h' 2024-03-09T09:28:25.304 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stat.h’: No data available 2024-03-09T09:28:25.309 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/Makefile.in' -> './k/coreutils-8.5/gnulib-tests/Makefile.in' 2024-03-09T09:28:25.309 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/Makefile.in’: No data available 2024-03-09T09:28:25.314 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getdelim.c' -> './k/coreutils-8.5/gnulib-tests/test-getdelim.c' 2024-03-09T09:28:25.315 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getdelim.c’: No data available 2024-03-09T09:28:25.315 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vprintf-posix.sh' -> './k/coreutils-8.5/gnulib-tests/test-vprintf-posix.sh' 2024-03-09T09:28:25.316 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vprintf-posix.sh’: No data available 2024-03-09T09:28:25.318 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fseeko2.sh' -> './k/coreutils-8.5/gnulib-tests/test-fseeko2.sh' 2024-03-09T09:28:25.318 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fseeko2.sh’: No data available 2024-03-09T09:28:25.320 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-c-strcase.sh' -> './k/coreutils-8.5/gnulib-tests/test-c-strcase.sh' 2024-03-09T09:28:25.320 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-c-strcase.sh’: No data available 2024-03-09T09:28:25.322 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsalign.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsalign.c' 2024-03-09T09:28:25.323 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsalign.c’: No data available 2024-03-09T09:28:25.324 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fcntl-h.c' -> './k/coreutils-8.5/gnulib-tests/test-fcntl-h.c' 2024-03-09T09:28:25.325 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fcntl-h.c’: No data available 2024-03-09T09:28:25.326 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vprintf-posix.c' -> './k/coreutils-8.5/gnulib-tests/test-vprintf-posix.c' 2024-03-09T09:28:25.327 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vprintf-posix.c’: No data available 2024-03-09T09:28:25.329 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-rename.c' -> './k/coreutils-8.5/gnulib-tests/test-rename.c' 2024-03-09T09:28:25.329 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-rename.c’: No data available 2024-03-09T09:28:25.331 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lstat.c' -> './k/coreutils-8.5/gnulib-tests/test-lstat.c' 2024-03-09T09:28:25.331 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lstat.c’: No data available 2024-03-09T09:28:25.333 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-malloca.c' -> './k/coreutils-8.5/gnulib-tests/test-malloca.c' 2024-03-09T09:28:25.334 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-malloca.c’: No data available 2024-03-09T09:28:25.335 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtoul.c' -> './k/coreutils-8.5/gnulib-tests/test-xstrtoul.c' 2024-03-09T09:28:25.335 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtoul.c’: No data available 2024-03-09T09:28:25.338 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-perror.sh' -> './k/coreutils-8.5/gnulib-tests/test-perror.sh' 2024-03-09T09:28:25.338 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-perror.sh’: No data available 2024-03-09T09:28:25.341 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vasprintf-posix.c' -> './k/coreutils-8.5/gnulib-tests/test-vasprintf-posix.c' 2024-03-09T09:28:25.342 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vasprintf-posix.c’: No data available 2024-03-09T09:28:25.344 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-frexp.c' -> './k/coreutils-8.5/gnulib-tests/test-frexp.c' 2024-03-09T09:28:25.344 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-frexp.c’: No data available 2024-03-09T09:28:25.346 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-environ.c' -> './k/coreutils-8.5/gnulib-tests/test-environ.c' 2024-03-09T09:28:25.346 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-environ.c’: No data available 2024-03-09T09:28:25.348 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stdint.c' -> './k/coreutils-8.5/gnulib-tests/test-stdint.c' 2024-03-09T09:28:25.349 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stdint.c’: No data available 2024-03-09T09:28:25.351 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnanl.h' -> './k/coreutils-8.5/gnulib-tests/test-isnanl.h' 2024-03-09T09:28:25.351 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnanl.h’: No data available 2024-03-09T09:28:25.353 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fcntl-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-fcntl-safer.c' 2024-03-09T09:28:25.353 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fcntl-safer.c’: No data available 2024-03-09T09:28:25.355 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-locale.c' -> './k/coreutils-8.5/gnulib-tests/test-locale.c' 2024-03-09T09:28:25.355 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-locale.c’: No data available 2024-03-09T09:28:25.357 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbrtowc4.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbrtowc4.sh' 2024-03-09T09:28:25.357 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbrtowc4.sh’: No data available 2024-03-09T09:28:25.359 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-argv-iter.c' -> './k/coreutils-8.5/gnulib-tests/test-argv-iter.c' 2024-03-09T09:28:25.359 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-argv-iter.c’: No data available 2024-03-09T09:28:25.362 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-utimens.h' -> './k/coreutils-8.5/gnulib-tests/test-utimens.h' 2024-03-09T09:28:25.362 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-utimens.h’: No data available 2024-03-09T09:28:25.364 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xalloc-die.c' -> './k/coreutils-8.5/gnulib-tests/test-xalloc-die.c' 2024-03-09T09:28:25.364 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xalloc-die.c’: No data available 2024-03-09T09:28:25.367 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fcntl.c' -> './k/coreutils-8.5/gnulib-tests/test-fcntl.c' 2024-03-09T09:28:25.367 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fcntl.c’: No data available 2024-03-09T09:28:25.370 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-alloca-opt.c' -> './k/coreutils-8.5/gnulib-tests/test-alloca-opt.c' 2024-03-09T09:28:25.370 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-alloca-opt.c’: No data available 2024-03-09T09:28:25.371 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-printf-frexpl.c' -> './k/coreutils-8.5/gnulib-tests/test-printf-frexpl.c' 2024-03-09T09:28:25.372 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-printf-frexpl.c’: No data available 2024-03-09T09:28:25.374 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-atexit.sh' -> './k/coreutils-8.5/gnulib-tests/test-atexit.sh' 2024-03-09T09:28:25.374 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-atexit.sh’: No data available 2024-03-09T09:28:25.376 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stddef.c' -> './k/coreutils-8.5/gnulib-tests/test-stddef.c' 2024-03-09T09:28:25.376 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stddef.c’: No data available 2024-03-09T09:28:25.379 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-c-ctype.c' -> './k/coreutils-8.5/gnulib-tests/test-c-ctype.c' 2024-03-09T09:28:25.379 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-c-ctype.c’: No data available 2024-03-09T09:28:25.381 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtol.sh' -> './k/coreutils-8.5/gnulib-tests/test-xstrtol.sh' 2024-03-09T09:28:25.382 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtol.sh’: No data available 2024-03-09T09:28:25.384 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getopt_long.h' -> './k/coreutils-8.5/gnulib-tests/test-getopt_long.h' 2024-03-09T09:28:25.384 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getopt_long.h’: No data available 2024-03-09T09:28:25.386 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-quotearg.sh' -> './k/coreutils-8.5/gnulib-tests/test-quotearg.sh' 2024-03-09T09:28:25.386 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-quotearg.sh’: No data available 2024-03-09T09:28:25.388 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-langinfo.c' -> './k/coreutils-8.5/gnulib-tests/test-langinfo.c' 2024-03-09T09:28:25.388 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-langinfo.c’: No data available 2024-03-09T09:28:25.390 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-inttypes.c' -> './k/coreutils-8.5/gnulib-tests/test-inttypes.c' 2024-03-09T09:28:25.390 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-inttypes.c’: No data available 2024-03-09T09:28:25.392 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-filevercmp.c' -> './k/coreutils-8.5/gnulib-tests/test-filevercmp.c' 2024-03-09T09:28:25.392 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-filevercmp.c’: No data available 2024-03-09T09:28:25.394 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-rmdir.c' -> './k/coreutils-8.5/gnulib-tests/test-rmdir.c' 2024-03-09T09:28:25.395 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-rmdir.c’: No data available 2024-03-09T09:28:25.396 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-futimens.h' -> './k/coreutils-8.5/gnulib-tests/test-futimens.h' 2024-03-09T09:28:25.397 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-futimens.h’: No data available 2024-03-09T09:28:25.398 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/setsockopt.c' -> './k/coreutils-8.5/gnulib-tests/setsockopt.c' 2024-03-09T09:28:25.399 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/setsockopt.c’: No data available 2024-03-09T09:28:25.400 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude1.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude1.sh' 2024-03-09T09:28:25.401 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude1.sh’: No data available 2024-03-09T09:28:25.402 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freopen-safer.c' -> './k/coreutils-8.5/gnulib-tests/test-freopen-safer.c' 2024-03-09T09:28:25.402 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freopen-safer.c’: No data available 2024-03-09T09:28:25.403 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getaddrinfo.c' -> './k/coreutils-8.5/gnulib-tests/test-getaddrinfo.c' 2024-03-09T09:28:25.403 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getaddrinfo.c’: No data available 2024-03-09T09:28:25.405 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-iconv.c' -> './k/coreutils-8.5/gnulib-tests/test-iconv.c' 2024-03-09T09:28:25.405 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-iconv.c’: No data available 2024-03-09T09:28:25.407 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/zerosize-ptr.h' -> './k/coreutils-8.5/gnulib-tests/zerosize-ptr.h' 2024-03-09T09:28:25.407 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/zerosize-ptr.h’: No data available 2024-03-09T09:28:25.408 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lseek.c' -> './k/coreutils-8.5/gnulib-tests/test-lseek.c' 2024-03-09T09:28:25.408 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lseek.c’: No data available 2024-03-09T09:28:25.410 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-usleep.c' -> './k/coreutils-8.5/gnulib-tests/test-usleep.c' 2024-03-09T09:28:25.410 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-usleep.c’: No data available 2024-03-09T09:28:25.412 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/gnulib.mk' -> './k/coreutils-8.5/gnulib-tests/gnulib.mk' 2024-03-09T09:28:25.412 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/gnulib.mk’: No data available 2024-03-09T09:28:25.414 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-unlinkat.c' -> './k/coreutils-8.5/gnulib-tests/test-unlinkat.c' 2024-03-09T09:28:25.414 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-unlinkat.c’: No data available 2024-03-09T09:28:25.416 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-quotearg.c' -> './k/coreutils-8.5/gnulib-tests/test-quotearg.c' 2024-03-09T09:28:25.417 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-quotearg.c’: No data available 2024-03-09T09:28:25.561 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-base64.c' -> './k/coreutils-8.5/gnulib-tests/test-base64.c' 2024-03-09T09:28:25.561 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-base64.c’: No data available 2024-03-09T09:28:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.846+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.846+0000 7f674f400640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 msgr2=0x7f67300085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:28:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.846+0000 7f674f400640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 msgr2=0x7f67300085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.846+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.846+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:28:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.847+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.847+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f6730005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:28:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.847+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f6730005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:25.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.847+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:25.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:25.847+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:28:26.575 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-frexpl.c' -> './k/coreutils-8.5/gnulib-tests/test-frexpl.c' 2024-03-09T09:28:26.576 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-frexpl.c’: No data available 2024-03-09T09:28:26.577 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-gethostname.c' -> './k/coreutils-8.5/gnulib-tests/test-gethostname.c' 2024-03-09T09:28:26.577 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-gethostname.c’: No data available 2024-03-09T09:28:26.579 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mkdir.h' -> './k/coreutils-8.5/gnulib-tests/test-mkdir.h' 2024-03-09T09:28:26.579 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mkdir.h’: No data available 2024-03-09T09:28:26.580 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-binary-io.sh' -> './k/coreutils-8.5/gnulib-tests/test-binary-io.sh' 2024-03-09T09:28:26.580 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-binary-io.sh’: No data available 2024-03-09T09:28:26.582 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_ioctl.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_ioctl.c' 2024-03-09T09:28:26.582 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_ioctl.c’: No data available 2024-03-09T09:28:26.583 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-time.c' -> './k/coreutils-8.5/gnulib-tests/test-time.c' 2024-03-09T09:28:26.583 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-time.c’: No data available 2024-03-09T09:28:26.585 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/signature.h' -> './k/coreutils-8.5/gnulib-tests/signature.h' 2024-03-09T09:28:26.585 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/signature.h’: No data available 2024-03-09T09:28:26.586 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnanf.h' -> './k/coreutils-8.5/gnulib-tests/test-isnanf.h' 2024-03-09T09:28:26.586 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnanf.h’: No data available 2024-03-09T09:28:26.597 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnanf-nolibm.c' -> './k/coreutils-8.5/gnulib-tests/test-isnanf-nolibm.c' 2024-03-09T09:28:26.597 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnanf-nolibm.c’: No data available 2024-03-09T09:28:26.598 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-wcwidth.c' -> './k/coreutils-8.5/gnulib-tests/test-wcwidth.c' 2024-03-09T09:28:26.598 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-wcwidth.c’: No data available 2024-03-09T09:28:26.600 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadseek.c' -> './k/coreutils-8.5/gnulib-tests/test-freadseek.c' 2024-03-09T09:28:26.600 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadseek.c’: No data available 2024-03-09T09:28:26.602 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-areadlink-with-size.c' -> './k/coreutils-8.5/gnulib-tests/test-areadlink-with-size.c' 2024-03-09T09:28:26.602 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-areadlink-with-size.c’: No data available 2024-03-09T09:28:26.603 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-select-stdin.c' -> './k/coreutils-8.5/gnulib-tests/test-select-stdin.c' 2024-03-09T09:28:26.604 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-select-stdin.c’: No data available 2024-03-09T09:28:26.605 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsstr1.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsstr1.c' 2024-03-09T09:28:26.605 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsstr1.c’: No data available 2024-03-09T09:28:26.607 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude3.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude3.sh' 2024-03-09T09:28:26.607 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude3.sh’: No data available 2024-03-09T09:28:26.609 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fopen.c' -> './k/coreutils-8.5/gnulib-tests/test-fopen.c' 2024-03-09T09:28:26.609 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fopen.c’: No data available 2024-03-09T09:28:26.611 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lock.c' -> './k/coreutils-8.5/gnulib-tests/test-lock.c' 2024-03-09T09:28:26.611 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lock.c’: No data available 2024-03-09T09:28:26.613 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fpending.c' -> './k/coreutils-8.5/gnulib-tests/test-fpending.c' 2024-03-09T09:28:26.613 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fpending.c’: No data available 2024-03-09T09:28:26.614 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-netinet_in.c' -> './k/coreutils-8.5/gnulib-tests/test-netinet_in.c' 2024-03-09T09:28:26.615 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-netinet_in.c’: No data available 2024-03-09T09:28:26.617 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-bitrotate.c' -> './k/coreutils-8.5/gnulib-tests/test-bitrotate.c' 2024-03-09T09:28:26.617 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-bitrotate.c’: No data available 2024-03-09T09:28:26.618 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-c-strcasecmp.c' -> './k/coreutils-8.5/gnulib-tests/test-c-strcasecmp.c' 2024-03-09T09:28:26.618 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-c-strcasecmp.c’: No data available 2024-03-09T09:28:26.620 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsstr3.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsstr3.c' 2024-03-09T09:28:26.620 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsstr3.c’: No data available 2024-03-09T09:28:26.622 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-hash.c' -> './k/coreutils-8.5/gnulib-tests/test-hash.c' 2024-03-09T09:28:26.622 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-hash.c’: No data available 2024-03-09T09:28:26.624 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-copy-acl.c' -> './k/coreutils-8.5/gnulib-tests/test-copy-acl.c' 2024-03-09T09:28:26.624 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-copy-acl.c’: No data available 2024-03-09T09:28:26.626 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sigaction.c' -> './k/coreutils-8.5/gnulib-tests/test-sigaction.c' 2024-03-09T09:28:26.626 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sigaction.c’: No data available 2024-03-09T09:28:26.628 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xstrtoumax.c' -> './k/coreutils-8.5/gnulib-tests/test-xstrtoumax.c' 2024-03-09T09:28:26.628 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xstrtoumax.c’: No data available 2024-03-09T09:28:26.630 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fseterr.c' -> './k/coreutils-8.5/gnulib-tests/test-fseterr.c' 2024-03-09T09:28:26.630 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fseterr.c’: No data available 2024-03-09T09:28:26.631 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-areadlink.c' -> './k/coreutils-8.5/gnulib-tests/test-areadlink.c' 2024-03-09T09:28:26.632 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-areadlink.c’: No data available 2024-03-09T09:28:26.633 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-signbit.c' -> './k/coreutils-8.5/gnulib-tests/test-signbit.c' 2024-03-09T09:28:26.634 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-signbit.c’: No data available 2024-03-09T09:28:26.635 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-btowc1.sh' -> './k/coreutils-8.5/gnulib-tests/test-btowc1.sh' 2024-03-09T09:28:26.635 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-btowc1.sh’: No data available 2024-03-09T09:28:26.637 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-strtod.c' -> './k/coreutils-8.5/gnulib-tests/test-strtod.c' 2024-03-09T09:28:26.638 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-strtod.c’: No data available 2024-03-09T09:28:26.639 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-atexit.c' -> './k/coreutils-8.5/gnulib-tests/test-atexit.c' 2024-03-09T09:28:26.640 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-atexit.c’: No data available 2024-03-09T09:28:26.641 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getdate.c' -> './k/coreutils-8.5/gnulib-tests/test-getdate.c' 2024-03-09T09:28:26.642 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getdate.c’: No data available 2024-03-09T09:28:26.643 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fpurge.c' -> './k/coreutils-8.5/gnulib-tests/test-fpurge.c' 2024-03-09T09:28:26.643 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fpurge.c’: No data available 2024-03-09T09:28:26.645 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-select.c' -> './k/coreutils-8.5/gnulib-tests/test-select.c' 2024-03-09T09:28:26.645 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-select.c’: No data available 2024-03-09T09:28:26.647 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-tls.c' -> './k/coreutils-8.5/gnulib-tests/test-tls.c' 2024-03-09T09:28:26.647 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-tls.c’: No data available 2024-03-09T09:28:26.648 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/sleep.c' -> './k/coreutils-8.5/gnulib-tests/sleep.c' 2024-03-09T09:28:26.648 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/sleep.c’: No data available 2024-03-09T09:28:26.649 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbrtowc2.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbrtowc2.sh' 2024-03-09T09:28:26.650 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbrtowc2.sh’: No data available 2024-03-09T09:28:26.651 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stat.c' -> './k/coreutils-8.5/gnulib-tests/test-stat.c' 2024-03-09T09:28:26.651 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stat.c’: No data available 2024-03-09T09:28:26.653 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadptr.sh' -> './k/coreutils-8.5/gnulib-tests/test-freadptr.sh' 2024-03-09T09:28:26.653 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadptr.sh’: No data available 2024-03-09T09:28:26.654 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/accept.c' -> './k/coreutils-8.5/gnulib-tests/accept.c' 2024-03-09T09:28:26.655 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/accept.c’: No data available 2024-03-09T09:28:26.657 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-unistd.c' -> './k/coreutils-8.5/gnulib-tests/test-unistd.c' 2024-03-09T09:28:26.657 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-unistd.c’: No data available 2024-03-09T09:28:26.659 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fopen.h' -> './k/coreutils-8.5/gnulib-tests/test-fopen.h' 2024-03-09T09:28:26.660 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fopen.h’: No data available 2024-03-09T09:28:26.662 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-math.c' -> './k/coreutils-8.5/gnulib-tests/test-math.c' 2024-03-09T09:28:26.662 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-math.c’: No data available 2024-03-09T09:28:26.663 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-printf-frexp.c' -> './k/coreutils-8.5/gnulib-tests/test-printf-frexp.c' 2024-03-09T09:28:26.664 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-printf-frexp.c’: No data available 2024-03-09T09:28:26.665 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.sh' -> './k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.sh' 2024-03-09T09:28:26.665 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vfprintf-posix.sh’: No data available 2024-03-09T09:28:26.666 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude.c' -> './k/coreutils-8.5/gnulib-tests/test-exclude.c' 2024-03-09T09:28:26.666 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude.c’: No data available 2024-03-09T09:28:26.667 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freading.c' -> './k/coreutils-8.5/gnulib-tests/test-freading.c' 2024-03-09T09:28:26.668 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freading.c’: No data available 2024-03-09T09:28:26.669 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getdtablesize.c' -> './k/coreutils-8.5/gnulib-tests/test-getdtablesize.c' 2024-03-09T09:28:26.669 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getdtablesize.c’: No data available 2024-03-09T09:28:26.670 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/unlinkdir.c' -> './k/coreutils-8.5/gnulib-tests/unlinkdir.c' 2024-03-09T09:28:26.670 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/unlinkdir.c’: No data available 2024-03-09T09:28:26.672 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fflush2.c' -> './k/coreutils-8.5/gnulib-tests/test-fflush2.c' 2024-03-09T09:28:26.672 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fflush2.c’: No data available 2024-03-09T09:28:26.673 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-stat-time.c' -> './k/coreutils-8.5/gnulib-tests/test-stat-time.c' 2024-03-09T09:28:26.673 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-stat-time.c’: No data available 2024-03-09T09:28:26.674 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-inet_ntop.c' -> './k/coreutils-8.5/gnulib-tests/test-inet_ntop.c' 2024-03-09T09:28:26.675 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-inet_ntop.c’: No data available 2024-03-09T09:28:26.676 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-vc-list-files-git.sh' -> './k/coreutils-8.5/gnulib-tests/test-vc-list-files-git.sh' 2024-03-09T09:28:26.676 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-vc-list-files-git.sh’: No data available 2024-03-09T09:28:26.678 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getopt.c' -> './k/coreutils-8.5/gnulib-tests/test-getopt.c' 2024-03-09T09:28:26.678 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getopt.c’: No data available 2024-03-09T09:28:26.679 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-alignof.c' -> './k/coreutils-8.5/gnulib-tests/test-alignof.c' 2024-03-09T09:28:26.680 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-alignof.c’: No data available 2024-03-09T09:28:26.681 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/macros.h' -> './k/coreutils-8.5/gnulib-tests/macros.h' 2024-03-09T09:28:26.681 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/macros.h’: No data available 2024-03-09T09:28:26.683 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-symlinkat.c' -> './k/coreutils-8.5/gnulib-tests/test-symlinkat.c' 2024-03-09T09:28:26.683 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-symlinkat.c’: No data available 2024-03-09T09:28:26.685 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getgroups.c' -> './k/coreutils-8.5/gnulib-tests/test-getgroups.c' 2024-03-09T09:28:26.685 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getgroups.c’: No data available 2024-03-09T09:28:26.687 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-xprintf-posix.sh' -> './k/coreutils-8.5/gnulib-tests/test-xprintf-posix.sh' 2024-03-09T09:28:26.687 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-xprintf-posix.sh’: No data available 2024-03-09T09:28:26.689 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fdopendir.c' -> './k/coreutils-8.5/gnulib-tests/test-fdopendir.c' 2024-03-09T09:28:26.689 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fdopendir.c’: No data available 2024-03-09T09:28:26.690 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_socket.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_socket.c' 2024-03-09T09:28:26.690 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_socket.c’: No data available 2024-03-09T09:28:26.691 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbscasecmp.c' -> './k/coreutils-8.5/gnulib-tests/test-mbscasecmp.c' 2024-03-09T09:28:26.692 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbscasecmp.c’: No data available 2024-03-09T09:28:26.694 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-getopt.h' -> './k/coreutils-8.5/gnulib-tests/test-getopt.h' 2024-03-09T09:28:26.694 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-getopt.h’: No data available 2024-03-09T09:28:26.695 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsrtowcs1.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsrtowcs1.sh' 2024-03-09T09:28:26.695 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsrtowcs1.sh’: No data available 2024-03-09T09:28:26.696 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsinit.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsinit.sh' 2024-03-09T09:28:26.697 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsinit.sh’: No data available 2024-03-09T09:28:26.698 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-gettimeofday.c' -> './k/coreutils-8.5/gnulib-tests/test-gettimeofday.c' 2024-03-09T09:28:26.698 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-gettimeofday.c’: No data available 2024-03-09T09:28:26.699 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsstr2.c' -> './k/coreutils-8.5/gnulib-tests/test-mbsstr2.c' 2024-03-09T09:28:26.700 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsstr2.c’: No data available 2024-03-09T09:28:26.701 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/init.sh' -> './k/coreutils-8.5/gnulib-tests/init.sh' 2024-03-09T09:28:26.701 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/init.sh’: No data available 2024-03-09T09:28:26.702 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-symlink.c' -> './k/coreutils-8.5/gnulib-tests/test-symlink.c' 2024-03-09T09:28:26.702 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-symlink.c’: No data available 2024-03-09T09:28:26.704 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-utimens.c' -> './k/coreutils-8.5/gnulib-tests/test-utimens.c' 2024-03-09T09:28:26.704 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-utimens.c’: No data available 2024-03-09T09:28:26.705 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-readlink.c' -> './k/coreutils-8.5/gnulib-tests/test-readlink.c' 2024-03-09T09:28:26.705 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-readlink.c’: No data available 2024-03-09T09:28:26.707 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lchown.h' -> './k/coreutils-8.5/gnulib-tests/test-lchown.h' 2024-03-09T09:28:26.707 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lchown.h’: No data available 2024-03-09T09:28:26.708 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-c-strncasecmp.c' -> './k/coreutils-8.5/gnulib-tests/test-c-strncasecmp.c' 2024-03-09T09:28:26.709 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-c-strncasecmp.c’: No data available 2024-03-09T09:28:26.711 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-lutimens.h' -> './k/coreutils-8.5/gnulib-tests/test-lutimens.h' 2024-03-09T09:28:26.711 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-lutimens.h’: No data available 2024-03-09T09:28:26.713 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-copy-acl.sh' -> './k/coreutils-8.5/gnulib-tests/test-copy-acl.sh' 2024-03-09T09:28:26.713 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-copy-acl.sh’: No data available 2024-03-09T09:28:26.714 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-exclude6.sh' -> './k/coreutils-8.5/gnulib-tests/test-exclude6.sh' 2024-03-09T09:28:26.715 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-exclude6.sh’: No data available 2024-03-09T09:28:26.716 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadptr2.c' -> './k/coreutils-8.5/gnulib-tests/test-freadptr2.c' 2024-03-09T09:28:26.716 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadptr2.c’: No data available 2024-03-09T09:28:26.718 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fflush2.sh' -> './k/coreutils-8.5/gnulib-tests/test-fflush2.sh' 2024-03-09T09:28:26.718 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fflush2.sh’: No data available 2024-03-09T09:28:26.719 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sys_utsname.c' -> './k/coreutils-8.5/gnulib-tests/test-sys_utsname.c' 2024-03-09T09:28:26.719 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sys_utsname.c’: No data available 2024-03-09T09:28:26.721 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsrtowcs2.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsrtowcs2.sh' 2024-03-09T09:28:26.721 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsrtowcs2.sh’: No data available 2024-03-09T09:28:26.723 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-file-has-acl.c' -> './k/coreutils-8.5/gnulib-tests/test-file-has-acl.c' 2024-03-09T09:28:26.723 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-file-has-acl.c’: No data available 2024-03-09T09:28:26.724 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-signal.c' -> './k/coreutils-8.5/gnulib-tests/test-signal.c' 2024-03-09T09:28:26.724 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-signal.c’: No data available 2024-03-09T09:28:26.726 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sha1.c' -> './k/coreutils-8.5/gnulib-tests/test-sha1.c' 2024-03-09T09:28:26.726 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sha1.c’: No data available 2024-03-09T09:28:26.728 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadptr2.sh' -> './k/coreutils-8.5/gnulib-tests/test-freadptr2.sh' 2024-03-09T09:28:26.728 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadptr2.sh’: No data available 2024-03-09T09:28:26.730 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-areadlinkat.c' -> './k/coreutils-8.5/gnulib-tests/test-areadlinkat.c' 2024-03-09T09:28:26.730 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-areadlinkat.c’: No data available 2024-03-09T09:28:26.731 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-unlink.h' -> './k/coreutils-8.5/gnulib-tests/test-unlink.h' 2024-03-09T09:28:26.731 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-unlink.h’: No data available 2024-03-09T09:28:26.733 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-dup2.c' -> './k/coreutils-8.5/gnulib-tests/test-dup2.c' 2024-03-09T09:28:26.733 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-dup2.c’: No data available 2024-03-09T09:28:26.734 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-freadptr.c' -> './k/coreutils-8.5/gnulib-tests/test-freadptr.c' 2024-03-09T09:28:26.735 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-freadptr.c’: No data available 2024-03-09T09:28:26.736 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-file-has-acl.sh' -> './k/coreutils-8.5/gnulib-tests/test-file-has-acl.sh' 2024-03-09T09:28:26.736 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-file-has-acl.sh’: No data available 2024-03-09T09:28:26.738 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-u64.c' -> './k/coreutils-8.5/gnulib-tests/test-u64.c' 2024-03-09T09:28:26.738 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-u64.c’: No data available 2024-03-09T09:28:26.743 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-striconv.c' -> './k/coreutils-8.5/gnulib-tests/test-striconv.c' 2024-03-09T09:28:26.743 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-striconv.c’: No data available 2024-03-09T09:28:26.744 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-sameacls.c' -> './k/coreutils-8.5/gnulib-tests/test-sameacls.c' 2024-03-09T09:28:26.744 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-sameacls.c’: No data available 2024-03-09T09:28:26.745 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-remove.c' -> './k/coreutils-8.5/gnulib-tests/test-remove.c' 2024-03-09T09:28:26.745 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-remove.c’: No data available 2024-03-09T09:28:26.746 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-fsync.c' -> './k/coreutils-8.5/gnulib-tests/test-fsync.c' 2024-03-09T09:28:26.746 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-fsync.c’: No data available 2024-03-09T09:28:26.747 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-mbsrtowcs4.sh' -> './k/coreutils-8.5/gnulib-tests/test-mbsrtowcs4.sh' 2024-03-09T09:28:26.747 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-mbsrtowcs4.sh’: No data available 2024-03-09T09:28:26.748 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-isnand.h' -> './k/coreutils-8.5/gnulib-tests/test-isnand.h' 2024-03-09T09:28:26.748 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-isnand.h’: No data available 2024-03-09T09:28:27.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.440+0000 7f674d600640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 msgr2=0x7f67300085e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:27.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.440+0000 7f674d600640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6730006120 0x7f67300085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:27.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.440+0000 7f674d600640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f6730005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:27.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.440+0000 7f674d600640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f6730005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:27.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.440+0000 7f674d600640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f673000b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:27.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.440+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 reap_dead start 2024-03-09T09:28:27.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.441+0000 7f674d600640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:27.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.441+0000 7f674d600640 1 -- 172.21.15.67:0/2130994072 --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675006da80 con 0x7f67500c3600 2024-03-09T09:28:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.441+0000 7f674d600640 1 -- 172.21.15.67:0/2130994072 --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675006d7a0 con 0x7f67300036b0 2024-03-09T09:28:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.441+0000 7f674d600640 1 -- 172.21.15.67:0/2130994072 --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6730008ce0 con 0x7f67500aa800 2024-03-09T09:28:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.441+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 msgr2=0x7f673000b1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.441+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f673000b1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:27.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.441+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:27.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.442+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f673000ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-09T09:28:27.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.442+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f673000ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:27.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.442+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:27.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.442+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:27.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.642+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 msgr2=0x7f673000b1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:28.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.642+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f673000b1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:28:28.012 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/glthread' -> './k/coreutils-8.5/gnulib-tests/glthread' 2024-03-09T09:28:28.012 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/glthread/yield.h' -> './k/coreutils-8.5/gnulib-tests/glthread/yield.h' 2024-03-09T09:28:28.013 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/glthread/yield.h’: No data available 2024-03-09T09:28:28.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.643+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:28.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.644+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f673000ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:28:28.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.644+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f673000ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:28.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.644+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:28.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:27.644+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-09T09:28:28.014 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/glthread/thread.h' -> './k/coreutils-8.5/gnulib-tests/glthread/thread.h' 2024-03-09T09:28:28.014 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/glthread/thread.h’: No data available 2024-03-09T09:28:28.015 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/glthread/thread.c' -> './k/coreutils-8.5/gnulib-tests/glthread/thread.c' 2024-03-09T09:28:28.015 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/glthread/thread.c’: No data available 2024-03-09T09:28:28.016 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/unistr' -> './k/coreutils-8.5/gnulib-tests/unistr' 2024-03-09T09:28:28.016 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/unistr/test-u8-uctomb.c' -> './k/coreutils-8.5/gnulib-tests/unistr/test-u8-uctomb.c' 2024-03-09T09:28:28.016 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/unistr/test-u8-uctomb.c’: No data available 2024-03-09T09:28:28.017 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/uniwidth' -> './k/coreutils-8.5/gnulib-tests/uniwidth' 2024-03-09T09:28:28.017 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width.c' -> './k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width.c' 2024-03-09T09:28:28.017 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width.c’: No data available 2024-03-09T09:28:28.018 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.sh' -> './k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.sh' 2024-03-09T09:28:28.018 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.sh’: No data available 2024-03-09T09:28:28.018 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.c' -> './k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.c' 2024-03-09T09:28:28.019 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/uniwidth/test-uc_width2.c’: No data available 2024-03-09T09:28:28.019 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-dirname.c' -> './k/coreutils-8.5/gnulib-tests/test-dirname.c' 2024-03-09T09:28:28.020 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-dirname.c’: No data available 2024-03-09T09:28:28.020 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/test-setenv.c' -> './k/coreutils-8.5/gnulib-tests/test-setenv.c' 2024-03-09T09:28:28.020 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/test-setenv.c’: No data available 2024-03-09T09:28:28.021 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/locale' -> './k/coreutils-8.5/gnulib-tests/locale' 2024-03-09T09:28:28.021 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/locale/fr' -> './k/coreutils-8.5/gnulib-tests/locale/fr' 2024-03-09T09:28:28.021 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES' -> './k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES' 2024-03-09T09:28:28.021 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.po' -> './k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.po' 2024-03-09T09:28:28.022 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.po’: No data available 2024-03-09T09:28:28.022 INFO:tasks.workunit.client.1.smithi134.stdout:'.snap/k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.mo' -> './k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.mo' 2024-03-09T09:28:28.022 INFO:tasks.workunit.client.1.smithi134.stderr:cp: preserving permissions for ‘./k/coreutils-8.5/gnulib-tests/locale/fr/LC_MESSAGES/test-quotearg.mo’: No data available 2024-03-09T09:28:28.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.043+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 msgr2=0x7f673000b1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:28.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.043+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f673000b1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:28:28.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.045+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:28.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.046+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f673000ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:28:28.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.046+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f673000ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:28.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.046+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:28.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.046+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-09T09:28:28.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T09:28:28.681 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.1... 2024-03-09T09:28:28.681 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-09T09:28:28.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.844+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 msgr2=0x7f673000b1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:28.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.844+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f673000b1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:28:28.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.847+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:28.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.848+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f673000ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:28:28.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.848+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f673000ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:28.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.848+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:28.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.848+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-09T09:28:28.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.886+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020830461 0 0) 0x7f675006da80 con 0x7f67500c3600 2024-03-09T09:28:28.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.887+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6734003520 con 0x7f67500c3600 2024-03-09T09:28:28.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.888+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 204480329 0 0) 0x7f6734003520 con 0x7f67500c3600 2024-03-09T09:28:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.888+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 msgr2=0x7f673000b1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.888+0000 7f674ea00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.134:3300/0 conn(0x7f67500aa800 0x7f673000b1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.888+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 msgr2=0x7f673000ee50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.888+0000 7f674ea00640 1 --2- 172.21.15.67:0/2130994072 >> v2:172.21.15.67:3300/0 conn(0x7f67300036b0 0x7f673000ee50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.888+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675006d4c0 con 0x7f67500c3600 2024-03-09T09:28:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.888+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f673c001620 con 0x7f67500c3600 2024-03-09T09:28:28.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.889+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 >> v1:172.21.15.67:6789/0 conn(0x7f67500c3600 legacy=0x7f673000d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:28.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.889+0000 7f674ea00640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 0x7f6734003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:28.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.889+0000 7f674ea00640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6734004800 0x7f6734004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:28.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.889+0000 7f674ea00640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67340052e0 0x7f673400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:28.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.889+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675006d7a0 con 0x7f6734003a20 2024-03-09T09:28:28.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.889+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6730008ce0 con 0x7f6734004800 2024-03-09T09:28:28.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.889+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673400c260 con 0x7f67340052e0 2024-03-09T09:28:28.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.890+0000 7f674f400640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6734004800 msgr2=0x7f6734004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:28.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.890+0000 7f674f400640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6734004800 0x7f6734004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:28.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.890+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67340052e0 0x7f673400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:28.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.890+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 0x7f6734003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:28.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.891+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67340052e0 msgr2=0x7f673400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:28.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.891+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67340052e0 0x7f673400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:28.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.891+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6734004800 msgr2=0x7f6734004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:28.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.891+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6734004800 0x7f6734004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:28.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.891+0000 7f674fe00640 1 -- 172.21.15.67:0/2130994072 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6734003520 con 0x7f6734003a20 2024-03-09T09:28:28.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.891+0000 7f6754c00640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67340052e0 0x7f673400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:28:28.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.891+0000 7f6754c00640 1 -- 172.21.15.67:0/2130994072 reap_dead start 2024-03-09T09:28:28.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.895+0000 7f674fe00640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 0x7f6734003e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6740009520 tx=0x7f6734005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2939f2dd92083a1d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:28.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.895+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740009ae0 con 0x7f6734003a20 2024-03-09T09:28:28.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.895+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6740006e50 con 0x7f6734003a20 2024-03-09T09:28:28.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.896+0000 7f674ea00640 1 -- 172.21.15.67:0/2130994072 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674002e730 con 0x7f6734003a20 2024-03-09T09:28:28.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.897+0000 7f6755600640 1 -- 172.21.15.67:0/2130994072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 msgr2=0x7f6734003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:28.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.897+0000 7f6755600640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 0x7f6734003e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6740009520 tx=0x7f6734005830 comp rx=0 tx=0).stop 2024-03-09T09:28:28.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.897+0000 7f6755600640 1 -- 172.21.15.67:0/2130994072 shutdown_connections 2024-03-09T09:28:28.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.897+0000 7f6755600640 1 --2- 172.21.15.67:0/2130994072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 0x7f6734003e40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:28.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.897+0000 7f6755600640 1 -- 172.21.15.67:0/2130994072 >> 172.21.15.67:0/2130994072 conn(0x7f67500c7300 msgr2=0x7f67500c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:28.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.897+0000 7f6755600640 1 -- 172.21.15.67:0/2130994072 shutdown_connections 2024-03-09T09:28:28.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.897+0000 7f6755600640 1 -- 172.21.15.67:0/2130994072 wait complete. 2024-03-09T09:28:28.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.902+0000 7f6755600640 1 Processor -- start 2024-03-09T09:28:28.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.903+0000 7f6755600640 1 -- start start 2024-03-09T09:28:28.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f6755600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 0x7f67501f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f6755600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f675015dd80 0x7f6750070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f6755600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67501ec8a0 0x7f67501ecd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f6755600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675015ab80 con 0x7f6734003a20 2024-03-09T09:28:28.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f6755600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675015aa00 con 0x7f67501ec8a0 2024-03-09T09:28:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f6755600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675015ad00 con 0x7f675015dd80 2024-03-09T09:28:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f6754c00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67501ec8a0 msgr2=0x7f67501ecd20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f674fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 0x7f67501f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f6754c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67501ec8a0 0x7f67501ecd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f674fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 0x7f67501f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49472/0 (socket says 172.21.15.67:49472) 2024-03-09T09:28:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f674fe00640 1 -- 172.21.15.67:0/123710724 learned_addr learned my addr 172.21.15.67:0/123710724 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:28.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.905+0000 7f674f400640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f675015dd80 0x7f6750070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.906+0000 7f674fe00640 1 -- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f675015dd80 msgr2=0x7f6750070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.906+0000 7f674fe00640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f675015dd80 0x7f6750070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.906+0000 7f674fe00640 1 -- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67501ec8a0 msgr2=0x7f67501ecd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:28:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.906+0000 7f674fe00640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67501ec8a0 0x7f67501ecd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:28.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.906+0000 7f674fe00640 1 -- 172.21.15.67:0/123710724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675015d1f0 con 0x7f6734003a20 2024-03-09T09:28:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.906+0000 7f674fe00640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 0x7f67501f35c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6740003930 tx=0x7f6740005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.907+0000 7f6747e00640 1 -- 172.21.15.67:0/123710724 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740046020 con 0x7f6734003a20 2024-03-09T09:28:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.907+0000 7f6747e00640 1 -- 172.21.15.67:0/123710724 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6740009ac0 con 0x7f6734003a20 2024-03-09T09:28:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.907+0000 7f6747e00640 1 -- 172.21.15.67:0/123710724 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740042610 con 0x7f6734003a20 2024-03-09T09:28:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.907+0000 7f6755600640 1 -- 172.21.15.67:0/123710724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67501ed260 con 0x7f6734003a20 2024-03-09T09:28:28.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.907+0000 7f6755600640 1 -- 172.21.15.67:0/123710724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675015e280 con 0x7f6734003a20 2024-03-09T09:28:28.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.909+0000 7f6755600640 1 -- 172.21.15.67:0/123710724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675004f3c0 con 0x7f6734003a20 2024-03-09T09:28:28.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.910+0000 7f6747e00640 1 -- 172.21.15.67:0/123710724 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6740049020 con 0x7f6734003a20 2024-03-09T09:28:28.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.912+0000 7f6747e00640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f673006c040 0x7f673006e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:28.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.912+0000 7f6747e00640 1 -- 172.21.15.67:0/123710724 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f67400ad4d0 con 0x7f6734003a20 2024-03-09T09:28:28.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.912+0000 7f674f400640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f673006c040 0x7f673006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:28.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.913+0000 7f674f400640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f673006c040 0x7f673006e500 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f67500a9bf0 tx=0x7f673c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:28.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:28.919+0000 7f6747e00640 1 -- 172.21.15.67:0/123710724 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6740050020 con 0x7f6734003a20 2024-03-09T09:28:29.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.199+0000 7f6755600640 1 -- 172.21.15.67:0/123710724 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f67500c19e0 con 0x7f6734003a20 2024-03-09T09:28:29.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.200+0000 7f6747e00640 1 -- 172.21.15.67:0/123710724 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1559 (secure 0 0 0) 0x7f675015d1f0 con 0x7f6734003a20 2024-03-09T09:28:29.207 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":286,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:28:29.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.202+0000 7f6755600640 1 -- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f673006c040 msgr2=0x7f673006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.202+0000 7f6755600640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f673006c040 0x7f673006e500 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f67500a9bf0 tx=0x7f673c009040 comp rx=0 tx=0).stop 2024-03-09T09:28:29.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.203+0000 7f6755600640 1 -- 172.21.15.67:0/123710724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 msgr2=0x7f67501f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.203+0000 7f6755600640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 0x7f67501f35c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6740003930 tx=0x7f6740005d00 comp rx=0 tx=0).stop 2024-03-09T09:28:29.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.203+0000 7f6755600640 1 -- 172.21.15.67:0/123710724 shutdown_connections 2024-03-09T09:28:29.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.203+0000 7f6755600640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f673006c040 0x7f673006e500 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.203+0000 7f6755600640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67501ec8a0 0x7f67501ecd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.203+0000 7f6755600640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f675015dd80 0x7f6750070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.203+0000 7f6755600640 1 --2- 172.21.15.67:0/123710724 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6734003a20 0x7f67501f35c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.203+0000 7f6755600640 1 -- 172.21.15.67:0/123710724 >> 172.21.15.67:0/123710724 conn(0x7f67500c7300 msgr2=0x7f6750157df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:29.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.203+0000 7f6755600640 1 -- 172.21.15.67:0/123710724 shutdown_connections 2024-03-09T09:28:29.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.203+0000 7f6755600640 1 -- 172.21.15.67:0/123710724 wait complete. 2024-03-09T09:28:29.230 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-03-09T09:28:29.230 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-03-09T09:28:29.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:28:29.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.444+0000 7fb67dc00640 1 Processor -- start 2024-03-09T09:28:29.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.444+0000 7fb67dc00640 1 -- start start 2024-03-09T09:28:29.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.445+0000 7fb67dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb67814fd90 0x7fb678156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.446+0000 7fb67dc00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb678156bc0 con 0x7fb6780c9380 2024-03-09T09:28:29.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.446+0000 7fb67dc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6780c9890 con 0x7fb67806c1d0 2024-03-09T09:28:29.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.446+0000 7fb67dc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67806cd10 con 0x7fb67814fd90 2024-03-09T09:28:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.446+0000 7fb66f400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb6780c9380 0x7fb6780acf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:39450/0 (socket says 172.21.15.67:39450) 2024-03-09T09:28:29.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.446+0000 7fb66f400640 1 -- 172.21.15.67:0/1156515033 learned_addr learned my addr 172.21.15.67:0/1156515033 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:28:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.446+0000 7fb67d200640 1 -- 172.21.15.67:0/1156515033 >> v1:172.21.15.134:6789/0 conn(0x7fb67806c1d0 legacy=0x7fb67806c600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:28:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.446+0000 7fb677e00640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb67814fd90 0x7fb678156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.446+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319520265 0 0) 0x7fb660001110 con 0x7fb6780c9380 2024-03-09T09:28:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.446+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb65c003520 con 0x7fb6780c9380 2024-03-09T09:28:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3827621569 0 0) 0x7fb65c003520 con 0x7fb6780c9380 2024-03-09T09:28:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb67814fd90 msgr2=0x7fb678156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb67814fd90 0x7fb678156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 >> v1:172.21.15.134:6789/0 conn(0x7fb67806c1d0 legacy=0x7fb67806c600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780c5f80 con 0x7fb6780c9380 2024-03-09T09:28:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb660001840 con 0x7fb6780c9380 2024-03-09T09:28:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 >> v1:172.21.15.67:6789/0 conn(0x7fb6780c9380 legacy=0x7fb6780acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb65c003a50 0x7fb65c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb65c004800 0x7fb65c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb67814fd90 0x7fb65c005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c00bb50 con 0x7fb65c003a50 2024-03-09T09:28:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c00bd00 con 0x7fb65c004800 2024-03-09T09:28:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c00bf00 con 0x7fb67814fd90 2024-03-09T09:28:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb66f400640 1 -- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb65c004800 msgr2=0x7fb65c004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb66f400640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb65c004800 0x7fb65c004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.447+0000 7fb677e00640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb65c003a50 0x7fb65c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.448+0000 7fb67d200640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb67814fd90 0x7fb65c005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.448+0000 7fb677e00640 1 -- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb67814fd90 msgr2=0x7fb65c005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.448+0000 7fb677e00640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb67814fd90 0x7fb65c005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.448+0000 7fb677e00640 1 -- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb65c004800 msgr2=0x7fb65c004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.448+0000 7fb677e00640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb65c004800 0x7fb65c004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.448+0000 7fb677e00640 1 -- 172.21.15.67:0/1156515033 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c003520 con 0x7fb65c003a50 2024-03-09T09:28:29.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.449+0000 7fb677e00640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb65c003a50 0x7fb65c003e50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb668009ac0 tx=0x7fb65c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec7dadd8ac5f3ef server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:29.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.449+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668039070 con 0x7fb65c003a50 2024-03-09T09:28:29.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.449+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66802c6b0 con 0x7fb65c003a50 2024-03-09T09:28:29.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.449+0000 7fb677400640 1 -- 172.21.15.67:0/1156515033 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668009690 con 0x7fb65c003a50 2024-03-09T09:28:29.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.450+0000 7fb67dc00640 1 -- 172.21.15.67:0/1156515033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb65c003a50 msgr2=0x7fb65c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.450+0000 7fb67dc00640 1 --2- 172.21.15.67:0/1156515033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb65c003a50 0x7fb65c003e50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb668009ac0 tx=0x7fb65c005830 comp rx=0 tx=0).stop 2024-03-09T09:28:29.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.450+0000 7fb67d200640 1 -- 172.21.15.67:0/1156515033 reap_dead start 2024-03-09T09:28:29.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.450+0000 7fb67dc00640 1 -- 172.21.15.67:0/1156515033 shutdown_connections 2024-03-09T09:28:29.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.450+0000 7fb67dc00640 1 -- 172.21.15.67:0/1156515033 >> 172.21.15.67:0/1156515033 conn(0x7fb6780cb480 msgr2=0x7fb6780cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:29.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.450+0000 7fb67dc00640 1 -- 172.21.15.67:0/1156515033 shutdown_connections 2024-03-09T09:28:29.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.450+0000 7fb67dc00640 1 -- 172.21.15.67:0/1156515033 wait complete. 2024-03-09T09:28:29.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.453+0000 7fb67dc00640 1 Processor -- start 2024-03-09T09:28:29.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.453+0000 7fb67dc00640 1 -- start start 2024-03-09T09:28:29.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb67dc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb65c003a50 0x7fb6780c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb67dc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c5240 0x7fb678148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb67dc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb678148980 0x7fb6780c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb67dc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6780c6520 con 0x7fb6780c5240 2024-03-09T09:28:29.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb67dc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67806ba60 con 0x7fb65c003a50 2024-03-09T09:28:29.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb67dc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67806bcb0 con 0x7fb678148980 2024-03-09T09:28:29.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb677e00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb65c003a50 msgr2=0x7fb6780c9d20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:29.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb677e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb65c003a50 0x7fb6780c9d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:29.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb66f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c5240 0x7fb678148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb66f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c5240 0x7fb678148440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49488/0 (socket says 172.21.15.67:49488) 2024-03-09T09:28:29.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb66f400640 1 -- 172.21.15.67:0/811231635 learned_addr learned my addr 172.21.15.67:0/811231635 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:29.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.455+0000 7fb67d200640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb678148980 0x7fb6780c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.456+0000 7fb66f400640 1 -- 172.21.15.67:0/811231635 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb678148980 msgr2=0x7fb6780c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.456+0000 7fb66f400640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb678148980 0x7fb6780c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.456+0000 7fb66f400640 1 -- 172.21.15.67:0/811231635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb65c003a50 msgr2=0x7fb6780c9d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:28:29.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.456+0000 7fb66f400640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb65c003a50 0x7fb6780c9d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.456+0000 7fb66f400640 1 -- 172.21.15.67:0/811231635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6781ec880 con 0x7fb6780c5240 2024-03-09T09:28:29.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.456+0000 7fb66f400640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c5240 0x7fb678148440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c7a0 tx=0x7fb66000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.456+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660018070 con 0x7fb6780c5240 2024-03-09T09:28:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.456+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66000ce50 con 0x7fb6780c5240 2024-03-09T09:28:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.456+0000 7fb67dc00640 1 -- 172.21.15.67:0/811231635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6780c4900 con 0x7fb6780c5240 2024-03-09T09:28:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.456+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6600047e0 con 0x7fb6780c5240 2024-03-09T09:28:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.457+0000 7fb67dc00640 1 -- 172.21.15.67:0/811231635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6780ca260 con 0x7fb6780c5240 2024-03-09T09:28:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.458+0000 7fb67dc00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c3cf0 0x7fb6780c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.458+0000 7fb67dc00640 1 -- 172.21.15.67:0/811231635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6781ec370 con 0x7fb6780c3cf0 2024-03-09T09:28:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.458+0000 7fb677e00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c3cf0 0x7fb6780c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.458+0000 7fb677e00640 1 -- 172.21.15.67:0/811231635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6780c46b0 con 0x7fb6780c3cf0 2024-03-09T09:28:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.459+0000 7fb677e00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c3cf0 0x7fb6780c40d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb668009a90 tx=0x7fb668004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.459+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668039040 con 0x7fb6780c3cf0 2024-03-09T09:28:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.459+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb66001b070 con 0x7fb6780c5240 2024-03-09T09:28:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.461+0000 7fb674c00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb65406b3d0 0x7fb65406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.461+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fb660089960 con 0x7fb6780c5240 2024-03-09T09:28:29.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.461+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7fb66803f470 con 0x7fb6780c3cf0 2024-03-09T09:28:29.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.461+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c3cf0 msgr2=0x7fb6780c40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.461+0000 7fb674c00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c3cf0 0x7fb6780c40d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb668009a90 tx=0x7fb668004730 comp rx=0 tx=0).stop 2024-03-09T09:28:29.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.461+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c3cf0 msgr2=0x7fb6780c40d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:29.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.461+0000 7fb674c00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c3cf0 0x7fb6780c40d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.461+0000 7fb67d200640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb65406b3d0 0x7fb65406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.463+0000 7fb67d200640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb65406b3d0 0x7fb65406d890 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb6780aacd0 tx=0x7fb6700095d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:29.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.475+0000 7fb67dc00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6781eee60 0x7fb6781ef240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.475+0000 7fb67dc00640 1 -- 172.21.15.67:0/811231635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6780c3510 con 0x7fb6781eee60 2024-03-09T09:28:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.475+0000 7fb677e00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6781eee60 0x7fb6781ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.476+0000 7fb677e00640 1 -- 172.21.15.67:0/811231635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb6780c57b0 con 0x7fb6781eee60 2024-03-09T09:28:29.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.476+0000 7fb677e00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6781eee60 0x7fb6781ef240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb668009ac0 tx=0x7fb668009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:29.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.477+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66803fb60 con 0x7fb6781eee60 2024-03-09T09:28:29.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.477+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5178 (secure 0 0 0) 0x7fb66803fb60 con 0x7fb6781eee60 2024-03-09T09:28:29.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.477+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6781eee60 msgr2=0x7fb6781ef240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.477+0000 7fb674c00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6781eee60 0x7fb6781ef240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb668009ac0 tx=0x7fb668009190 comp rx=0 tx=0).stop 2024-03-09T09:28:29.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.477+0000 7fb674c00640 1 -- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6781eee60 msgr2=0x7fb6781ef240 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:29.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.477+0000 7fb674c00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6781eee60 0x7fb6781ef240 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb668009ac0 tx=0x7fb668009190 comp rx=0 tx=0).stop 2024-03-09T09:28:29.485 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:28:29.486 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:29.486 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:29.486 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:28:29.486 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 286, 2024-03-09T09:28:29.486 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:28:29.486 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:28:29.486 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:28:29.487 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.487 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:28:29.487 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:29.487 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:28:29.487 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:28:29.487 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:29.487 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:29.488 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:29.488 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:29.488 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:29.488 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:29.488 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:29.488 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:29.488 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:29.489 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:29.489 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:29.489 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.489 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:28:29.489 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:28:29.490 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:29.490 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:29.490 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:29.490 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:29.490 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:29.490 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:29.490 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:29.490 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:29.491 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:29.491 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:29.491 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:29.491 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.491 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:28:29.491 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:28:29.491 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:28:29.492 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:28:29.492 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:28:29.492 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:28:29.492 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:29.493 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:29.493 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:28:29.493 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:28:29.493 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:28:29.493 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:28:29.493 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:28:29.493 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:28:29.493 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:28:29.494 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:29.494 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:28:29.494 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:29.494 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:29.494 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:29.494 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:29.494 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:29.494 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:29.495 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:29.495 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:29.495 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:29.495 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:29.495 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:29.495 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.495 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:28:29.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.496 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:28:29.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.496 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:29.496 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:29.496 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:29.497 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:29.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.497 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:29.497 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:28:29.497 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.498 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:29.498 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:28:29.498 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.498 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.499 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:28:29.499 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:28:29.499 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:29.499 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:29.499 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:29.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.500 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:28:29.500 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:28:29.500 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:29.500 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:29.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:29.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:28:29.501 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:29.501 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:28:29.502 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.502 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.502 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.502 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:28:29.502 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:28:29.503 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:29.503 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:29.503 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:29.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.503 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:29.503 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:29.504 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:29.504 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:29.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.504 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:29.504 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:28:29.504 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.505 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:29.506 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:28:29.506 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.506 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.506 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.506 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:28:29.507 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:28:29.507 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:29.507 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:29.507 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:29.508 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.508 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.508 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:28:29.508 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:28:29.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.509 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:29.509 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:29.509 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:29.509 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.509 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.509 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:28:29.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.510 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:29.510 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:29.510 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:28:29.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.511 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.511 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:28:29.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.511 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:29.512 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:29.512 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:28:29.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.512 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.512 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:28:29.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.512 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:28:29.512 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:29.513 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:28:29.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.513 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:29.513 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:29.513 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:28:29.513 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.513 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.514 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:28:29.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.514 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:29.514 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:29.514 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:28:29.514 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.514 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.515 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:28:29.515 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:28:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.477+0000 7fb67dc00640 1 -- 172.21.15.67:0/811231635 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb65406b3d0 msgr2=0x7fb65406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.477+0000 7fb67dc00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb65406b3d0 0x7fb65406d890 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb6780aacd0 tx=0x7fb6700095d0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.477+0000 7fb67dc00640 1 -- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c5240 msgr2=0x7fb678148440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.477+0000 7fb67dc00640 1 --2- 172.21.15.67:0/811231635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6780c5240 0x7fb678148440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c7a0 tx=0x7fb66000cc60 comp rx=0 tx=0).stop 2024-03-09T09:28:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.477+0000 7fb67d200640 1 -- 172.21.15.67:0/811231635 reap_dead start 2024-03-09T09:28:29.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.478+0000 7fb67d200640 1 -- 172.21.15.67:0/811231635 reap_dead start 2024-03-09T09:28:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.478+0000 7fb67dc00640 1 -- 172.21.15.67:0/811231635 shutdown_connections 2024-03-09T09:28:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.478+0000 7fb67dc00640 1 -- 172.21.15.67:0/811231635 >> 172.21.15.67:0/811231635 conn(0x7fb6780cb480 msgr2=0x7fb67814e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.478+0000 7fb67dc00640 1 -- 172.21.15.67:0/811231635 shutdown_connections 2024-03-09T09:28:29.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.478+0000 7fb67dc00640 1 -- 172.21.15.67:0/811231635 wait complete. 2024-03-09T09:28:29.518 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:28:29.518 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-03-09T09:28:29.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:28:29.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.731+0000 7f3272c00640 1 Processor -- start 2024-03-09T09:28:29.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.731+0000 7f3272c00640 1 -- start start 2024-03-09T09:28:29.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.732+0000 7f3272c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f326c1532d0 0x7f326c159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.732+0000 7f3272c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c1561f0 0x7f326c156680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.732+0000 7f3272c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326c159820 con 0x7f326c0a9e10 2024-03-09T09:28:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.732+0000 7f3272c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326c06f4b0 con 0x7f326c1532d0 2024-03-09T09:28:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.732+0000 7f3272c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c15a0c0 con 0x7f326c1561f0 2024-03-09T09:28:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.733+0000 7f3271800640 1 -- >> v2:172.21.15.134:3300/0 conn(0x7f326c1532d0 msgr2=0x7f326c159120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.733+0000 7f3270e00640 1 -- >> v1:172.21.15.134:6789/0 conn(0x7f326c0a9e10 legacy=0x7f326c0c0b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.134:6789/0 2024-03-09T09:28:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.733+0000 7f3271800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f326c1532d0 0x7f326c159120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.733+0000 7f3272200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c1561f0 0x7f326c156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.733+0000 7f3272200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c1561f0 0x7f326c156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:36170/0 (socket says 172.21.15.67:36170) 2024-03-09T09:28:29.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.733+0000 7f3272200640 1 -- 172.21.15.67:0/108729812 learned_addr learned my addr 172.21.15.67:0/108729812 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:29.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.733+0000 7f3272200640 1 -- 172.21.15.67:0/108729812 >> v2:172.21.15.134:3300/0 conn(0x7f326c1532d0 msgr2=0x7f326c159120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:29.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.733+0000 7f3272200640 1 --2- 172.21.15.67:0/108729812 >> v2:172.21.15.134:3300/0 conn(0x7f326c1532d0 0x7f326c159120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.734+0000 7f3272200640 1 -- 172.21.15.67:0/108729812 >> v1:172.21.15.134:6789/0 conn(0x7f326c0a9e10 legacy=0x7f326c0c0b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.734+0000 7f3272200640 1 -- 172.21.15.67:0/108729812 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c159b00 con 0x7f326c1561f0 2024-03-09T09:28:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.735+0000 7f3272200640 1 --2- 172.21.15.67:0/108729812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c1561f0 0x7f326c156680 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f325c009870 tx=0x7f325c02b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e658b7b3fbbeba0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.735+0000 7f326be00640 1 -- 172.21.15.67:0/108729812 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c02e070 con 0x7f326c1561f0 2024-03-09T09:28:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.735+0000 7f326be00640 1 -- 172.21.15.67:0/108729812 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c037070 con 0x7f326c1561f0 2024-03-09T09:28:29.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.735+0000 7f326be00640 1 -- 172.21.15.67:0/108729812 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c031a00 con 0x7f326c1561f0 2024-03-09T09:28:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.736+0000 7f3272c00640 1 -- 172.21.15.67:0/108729812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c1561f0 msgr2=0x7f326c156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.736+0000 7f3272c00640 1 --2- 172.21.15.67:0/108729812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c1561f0 0x7f326c156680 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f325c009870 tx=0x7f325c02b880 comp rx=0 tx=0).stop 2024-03-09T09:28:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.736+0000 7f3272c00640 1 -- 172.21.15.67:0/108729812 shutdown_connections 2024-03-09T09:28:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.736+0000 7f3272c00640 1 --2- 172.21.15.67:0/108729812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c1561f0 0x7f326c156680 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.736+0000 7f3272c00640 1 --2- 172.21.15.67:0/108729812 >> v2:172.21.15.134:3300/0 conn(0x7f326c1532d0 0x7f326c159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.736+0000 7f3272c00640 1 -- 172.21.15.67:0/108729812 >> 172.21.15.67:0/108729812 conn(0x7f326c0c76f0 msgr2=0x7f326c0c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.736+0000 7f3272c00640 1 -- 172.21.15.67:0/108729812 shutdown_connections 2024-03-09T09:28:29.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.736+0000 7f3272c00640 1 -- 172.21.15.67:0/108729812 wait complete. 2024-03-09T09:28:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.738+0000 7f3272c00640 1 Processor -- start 2024-03-09T09:28:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.738+0000 7f3272c00640 1 -- start start 2024-03-09T09:28:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3272c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f326c0a9e10 0x7f326c155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3272c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f326c1532d0 0x7f326c158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3272c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c1561f0 0x7f326c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3272c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1536b0 con 0x7f326c0a9e10 2024-03-09T09:28:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3272c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c155d60 con 0x7f326c1532d0 2024-03-09T09:28:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3272c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c0c1230 con 0x7f326c1561f0 2024-03-09T09:28:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3270e00640 1 -- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f326c1532d0 msgr2=0x7f326c158330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.134:3300/0 2024-03-09T09:28:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3270e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f326c1532d0 0x7f326c158330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3271800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f326c0a9e10 0x7f326c155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3271800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f326c0a9e10 0x7f326c155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49528/0 (socket says 172.21.15.67:49528) 2024-03-09T09:28:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3271800640 1 -- 172.21.15.67:0/2351136947 learned_addr learned my addr 172.21.15.67:0/2351136947 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.739+0000 7f3272200640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c1561f0 0x7f326c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.740+0000 7f3271800640 1 -- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c1561f0 msgr2=0x7f326c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.740+0000 7f3271800640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c1561f0 0x7f326c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.740+0000 7f3271800640 1 -- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f326c1532d0 msgr2=0x7f326c158330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:28:29.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.740+0000 7f3271800640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f326c1532d0 0x7f326c158330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.740+0000 7f3271800640 1 -- 172.21.15.67:0/2351136947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c0c5400 con 0x7f326c0a9e10 2024-03-09T09:28:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.740+0000 7f3271800640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f326c0a9e10 0x7f326c155330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f326400ba10 tx=0x7f326400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.740+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400cbd0 con 0x7f326c0a9e10 2024-03-09T09:28:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.740+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3264010070 con 0x7f326c0a9e10 2024-03-09T09:28:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.740+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264015470 con 0x7f326c0a9e10 2024-03-09T09:28:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.740+0000 7f3272c00640 1 -- 172.21.15.67:0/2351136947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c1ec1b0 con 0x7f326c0a9e10 2024-03-09T09:28:29.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.741+0000 7f3272c00640 1 -- 172.21.15.67:0/2351136947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c0ab030 con 0x7f326c0a9e10 2024-03-09T09:28:29.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.742+0000 7f3272c00640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c158db0 0x7f326c1ee930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.742+0000 7f3272c00640 1 -- 172.21.15.67:0/2351136947 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c04f3c0 con 0x7f326c158db0 2024-03-09T09:28:29.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.742+0000 7f3270e00640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c158db0 0x7f326c1ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.743+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f326401e3f0 con 0x7f326c0a9e10 2024-03-09T09:28:29.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.743+0000 7f3270e00640 1 -- 172.21.15.67:0/2351136947 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f326c157f50 con 0x7f326c158db0 2024-03-09T09:28:29.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.743+0000 7f3270e00640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c158db0 0x7f326c1ee930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f326c151520 tx=0x7f325800b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:29.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.744+0000 7f3269600640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f324c06b480 0x7f324c06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.744+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f326408a6b0 con 0x7f326c0a9e10 2024-03-09T09:28:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.744+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258016070 con 0x7f326c158db0 2024-03-09T09:28:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.744+0000 7f3272200640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f324c06b480 0x7f324c06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.744+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f3258003dc0 con 0x7f326c158db0 2024-03-09T09:28:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.744+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c158db0 msgr2=0x7f326c1ee930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.744+0000 7f3269600640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c158db0 0x7f326c1ee930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f326c151520 tx=0x7f325800b8e0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.744+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c158db0 msgr2=0x7f326c1ee930 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:29.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.744+0000 7f3269600640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c158db0 0x7f326c1ee930 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f326c151520 tx=0x7f325800b8e0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.746+0000 7f3272200640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f324c06b480 0x7f324c06d940 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f325c036000 tx=0x7f325c03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.752+0000 7f3272c00640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c0c40d0 0x7f326c1eee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:29.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.752+0000 7f3272c00640 1 -- 172.21.15.67:0/2351136947 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c0c7ad0 con 0x7f326c0c40d0 2024-03-09T09:28:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.753+0000 7f3270e00640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c0c40d0 0x7f326c1eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:29.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.753+0000 7f3270e00640 1 -- 172.21.15.67:0/2351136947 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f326c0c0ef0 con 0x7f326c0c40d0 2024-03-09T09:28:29.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.754+0000 7f3270e00640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c0c40d0 0x7f326c1eee70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3258011000 tx=0x7f32580059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:29.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.754+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258016040 con 0x7f326c0c40d0 2024-03-09T09:28:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4863 (secure 0 0 0) 0x7f325801a900 con 0x7f326c0c40d0 2024-03-09T09:28:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c0c40d0 msgr2=0x7f326c1eee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3269600640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c0c40d0 0x7f326c1eee70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3258011000 tx=0x7f32580059d0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3269600640 1 -- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c0c40d0 msgr2=0x7f326c1eee70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3269600640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f326c0c40d0 0x7f326c1eee70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.764 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:28:29.764 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:29.764 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:29.764 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:28:29.764 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 286, 2024-03-09T09:28:29.764 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:28:29.764 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:28:29.765 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:28:29.765 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.765 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 0, 2024-03-09T09:28:29.765 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:29.765 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:28:29.766 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:28:29.766 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:29.766 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:29.766 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:29.766 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:29.766 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:29.766 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:29.766 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:29.767 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:29.767 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:29.767 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:29.767 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:29.767 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.767 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:28:29.767 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:28:29.767 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:29.768 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:29.768 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:29.768 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:29.768 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:29.768 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:29.768 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:29.768 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:29.768 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:29.769 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:29.769 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:29.769 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.769 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:28:29.769 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:28:29.769 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:28:29.769 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:28:29.770 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:28:29.770 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:28:29.770 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:29.770 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:29.770 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:28:29.770 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:28:29.770 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:28:29.770 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:28:29.771 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:28:29.771 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:28:29.771 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:28:29.771 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:29.771 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:28:29.771 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:29.771 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:29.771 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:29.772 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:29.772 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:29.772 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:29.772 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:29.772 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:29.772 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:29.772 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:29.772 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:29.773 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.773 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:28:29.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.773 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:28:29.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.773 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:29.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:29.773 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:29.774 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:29.774 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:29.774 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:28:29.774 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.774 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:29.775 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:28:29.775 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.775 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.775 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.775 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:28:29.775 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:28:29.775 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:29.776 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:29.776 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:29.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.776 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.776 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:28:29.776 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:28:29.777 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:29.777 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:29.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.777 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:29.777 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:28:29.777 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.778 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.778 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:29.778 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:28:29.778 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.778 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.778 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.779 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:28:29.779 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:28:29.779 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:29.779 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:29.779 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:29.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.780 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:29.780 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:29.780 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:29.780 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:29.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.780 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:29.781 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:28:29.781 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:29.782 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:28:29.782 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:29.782 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.782 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.782 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:28:29.782 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:28:29.783 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:29.783 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:29.783 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:29.783 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.783 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.783 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:28:29.783 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:28:29.784 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.784 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:29.784 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:29.784 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:29.784 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.784 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.784 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:28:29.784 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.784 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:29.785 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:29.785 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:29.785 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.785 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.785 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:28:29.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.785 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:29.785 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:29.786 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:29.786 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.786 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:29.786 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:28:29.786 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:29.786 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:29.786 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:29.787 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:29.787 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:29.787 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:29.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:29.787 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:28:29.787 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:28:29.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3272c00640 1 -- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f324c06b480 msgr2=0x7f324c06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3272c00640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f324c06b480 0x7f324c06d940 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f325c036000 tx=0x7f325c03d040 comp rx=0 tx=0).stop 2024-03-09T09:28:29.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3272c00640 1 -- 172.21.15.67:0/2351136947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f326c0a9e10 msgr2=0x7f326c155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:29.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3272c00640 1 --2- 172.21.15.67:0/2351136947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f326c0a9e10 0x7f326c155330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f326400ba10 tx=0x7f326400bed0 comp rx=0 tx=0).stop 2024-03-09T09:28:29.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3272200640 1 -- 172.21.15.67:0/2351136947 reap_dead start 2024-03-09T09:28:29.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3272200640 1 -- 172.21.15.67:0/2351136947 reap_dead start 2024-03-09T09:28:29.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3272c00640 1 -- 172.21.15.67:0/2351136947 shutdown_connections 2024-03-09T09:28:29.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3272c00640 1 -- 172.21.15.67:0/2351136947 >> 172.21.15.67:0/2351136947 conn(0x7f326c0c76f0 msgr2=0x7f326c14ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:29.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3272c00640 1 -- 172.21.15.67:0/2351136947 shutdown_connections 2024-03-09T09:28:29.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:29.755+0000 7f3272c00640 1 -- 172.21.15.67:0/2351136947 wait complete. 2024-03-09T09:28:29.790 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-03-09T09:28:29.790 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-03-09T09:28:33.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:28:33.426 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:28:49.791 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-03-09T09:28:49.792 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-03-09T09:28:49.792 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-09T09:28:49.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-09T09:28:49.800 INFO:tasks.ceph.mon.b:Started 2024-03-09T09:28:49.800 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:28:49.801 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:28:50.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.021+0000 7f2c8be00640 1 Processor -- start 2024-03-09T09:28:50.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.021+0000 7f2c8be00640 1 -- start start 2024-03-09T09:28:50.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.024+0000 7f2c8be00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2c8c072770 0x7f2c8c072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:50.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.024+0000 7f2c8be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c8c0bebc0 0x7f2c8c156650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:50.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.024+0000 7f2c8be00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8c06cd30 con 0x7f2c8c06b690 2024-03-09T09:28:50.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.024+0000 7f2c8be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8c06b200 con 0x7f2c8c0bebc0 2024-03-09T09:28:50.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.024+0000 7f2c8be00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8c0c5eb0 con 0x7f2c8c072770 2024-03-09T09:28:50.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.024+0000 7f2c8a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c8c0bebc0 0x7f2c8c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:50.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.024+0000 7f2c8a000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f2c8c0bebc0 0x7f2c8c156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34334/0 (socket says 172.21.15.67:34334) 2024-03-09T09:28:50.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.024+0000 7f2c8a000640 1 -- 172.21.15.67:0/2351795823 learned_addr learned my addr 172.21.15.67:0/2351795823 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.024+0000 7f2c8aa00640 1 --2- 172.21.15.67:0/2351795823 >> v2:172.21.15.134:3300/0 conn(0x7f2c8c072770 0x7f2c8c072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.025+0000 7f2c8aa00640 1 -- 172.21.15.67:0/2351795823 >> v2:172.21.15.134:3300/0 conn(0x7f2c8c072770 msgr2=0x7f2c8c072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-09T09:28:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.025+0000 7f2c8aa00640 1 -- 172.21.15.67:0/2351795823 >> v2:172.21.15.134:3300/0 conn(0x7f2c8c072770 msgr2=0x7f2c8c072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-09T09:28:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.025+0000 7f2c8aa00640 1 --2- 172.21.15.67:0/2351795823 >> v2:172.21.15.134:3300/0 conn(0x7f2c8c072770 0x7f2c8c072b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.025+0000 7f2c8aa00640 1 --2- 172.21.15.67:0/2351795823 >> v2:172.21.15.134:3300/0 conn(0x7f2c8c072770 0x7f2c8c072b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.025+0000 7f2c8a000640 1 -- 172.21.15.67:0/2351795823 >> v2:172.21.15.134:3300/0 conn(0x7f2c8c072770 msgr2=0x7f2c8c072b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-09T09:28:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.025+0000 7f2c8a000640 1 --2- 172.21.15.67:0/2351795823 >> v2:172.21.15.134:3300/0 conn(0x7f2c8c072770 0x7f2c8c072b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:50.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.025+0000 7f2c8a000640 1 -- 172.21.15.67:0/2351795823 >> v1:172.21.15.134:6789/0 conn(0x7f2c8c06b690 legacy=0x7f2c8c06bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:50.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.025+0000 7f2c8a000640 1 -- 172.21.15.67:0/2351795823 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0c6150 con 0x7f2c8c0bebc0 2024-03-09T09:28:50.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.026+0000 7f2c8a000640 1 --2- 172.21.15.67:0/2351795823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c8c0bebc0 0x7f2c8c156650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0aae90 tx=0x7f2c7802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c0c506b68cfc19 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:50.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.026+0000 7f2c89600640 1 -- 172.21.15.67:0/2351795823 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7802e070 con 0x7f2c8c0bebc0 2024-03-09T09:28:50.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.027+0000 7f2c89600640 1 -- 172.21.15.67:0/2351795823 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c78037070 con 0x7f2c8c0bebc0 2024-03-09T09:28:50.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.027+0000 7f2c89600640 1 -- 172.21.15.67:0/2351795823 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78031a00 con 0x7f2c8c0bebc0 2024-03-09T09:28:50.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.027+0000 7f2c8be00640 1 -- 172.21.15.67:0/2351795823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c8c0bebc0 msgr2=0x7f2c8c156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:50.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.027+0000 7f2c8be00640 1 --2- 172.21.15.67:0/2351795823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c8c0bebc0 0x7f2c8c156650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0aae90 tx=0x7f2c7802b880 comp rx=0 tx=0).stop 2024-03-09T09:28:50.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.027+0000 7f2c8be00640 1 -- 172.21.15.67:0/2351795823 shutdown_connections 2024-03-09T09:28:50.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.027+0000 7f2c8be00640 1 --2- 172.21.15.67:0/2351795823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c8c0bebc0 0x7f2c8c156650 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:50.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.027+0000 7f2c8be00640 1 --2- 172.21.15.67:0/2351795823 >> v2:172.21.15.134:3300/0 conn(0x7f2c8c072770 0x7f2c8c072b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:50.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.027+0000 7f2c8be00640 1 -- 172.21.15.67:0/2351795823 >> 172.21.15.67:0/2351795823 conn(0x7f2c8c0c7fb0 msgr2=0x7f2c8c0c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:50.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.027+0000 7f2c8be00640 1 -- 172.21.15.67:0/2351795823 shutdown_connections 2024-03-09T09:28:50.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.027+0000 7f2c8be00640 1 -- 172.21.15.67:0/2351795823 wait complete. 2024-03-09T09:28:50.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.029+0000 7f2c8be00640 1 Processor -- start 2024-03-09T09:28:50.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.029+0000 7f2c8be00640 1 -- start start 2024-03-09T09:28:50.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.030+0000 7f2c8be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c8c06b690 0x7f2c8c0c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:50.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.030+0000 7f2c8be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c8c072770 0x7f2c8c0ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:50.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.030+0000 7f2c8be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c8c0bebc0 0x7f2c8c157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:50.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.030+0000 7f2c8be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c0c9430 con 0x7f2c8c072770 2024-03-09T09:28:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.030+0000 7f2c8be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c0c6710 con 0x7f2c8c06b690 2024-03-09T09:28:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.030+0000 7f2c8be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c0c5150 con 0x7f2c8c0bebc0 2024-03-09T09:28:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c8c072770 0x7f2c8c0ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c8c06b690 0x7f2c8c0c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8b400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c8c0bebc0 0x7f2c8c157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c8c06b690 0x7f2c8c0c3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:39284/0 (socket says 172.21.15.67:39284) 2024-03-09T09:28:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8b400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c8c0bebc0 0x7f2c8c157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:54636/0 (socket says 172.21.15.67:54636) 2024-03-09T09:28:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8aa00640 1 -- 172.21.15.67:0/661494483 learned_addr learned my addr 172.21.15.67:0/661494483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:50.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8aa00640 1 -- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c8c06b690 msgr2=0x7f2c8c0c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-09T09:28:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8aa00640 1 -- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c8c06b690 msgr2=0x7f2c8c0c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-09T09:28:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8aa00640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c8c06b690 0x7f2c8c0c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-09T09:28:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8aa00640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c8c06b690 0x7f2c8c0c3ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-09T09:28:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8a000640 1 -- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c8c0bebc0 msgr2=0x7f2c8c157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8a000640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c8c0bebc0 0x7f2c8c157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8a000640 1 -- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c8c06b690 msgr2=0x7f2c8c0c3ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-09T09:28:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8a000640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c8c06b690 0x7f2c8c0c3ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:50.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.031+0000 7f2c8a000640 1 -- 172.21.15.67:0/661494483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0c53a0 con 0x7f2c8c072770 2024-03-09T09:28:50.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.032+0000 7f2c8a000640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c8c072770 0x7f2c8c0ad310 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0c7e90 tx=0x7f2c780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.032+0000 7f2c82a00640 1 -- 172.21.15.67:0/661494483 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78032670 con 0x7f2c8c072770 2024-03-09T09:28:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.032+0000 7f2c82a00640 1 -- 172.21.15.67:0/661494483 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c78032830 con 0x7f2c8c072770 2024-03-09T09:28:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.032+0000 7f2c8be00640 1 -- 172.21.15.67:0/661494483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8c0c3610 con 0x7f2c8c072770 2024-03-09T09:28:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.032+0000 7f2c82a00640 1 -- 172.21.15.67:0/661494483 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78032c20 con 0x7f2c8c072770 2024-03-09T09:28:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.032+0000 7f2c8be00640 1 -- 172.21.15.67:0/661494483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8c157b50 con 0x7f2c8c072770 2024-03-09T09:28:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.033+0000 7f2c8be00640 1 -- 172.21.15.67:0/661494483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8c0c3400 con 0x7f2c8c072770 2024-03-09T09:28:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.034+0000 7f2c82a00640 1 -- 172.21.15.67:0/661494483 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2c780453f0 con 0x7f2c8c072770 2024-03-09T09:28:50.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.035+0000 7f2c82a00640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c6c06b500 0x7f2c6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:50.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.035+0000 7f2c82a00640 1 -- 172.21.15.67:0/661494483 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f2c780ac480 con 0x7f2c8c072770 2024-03-09T09:28:50.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.036+0000 7f2c8aa00640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c6c06b500 0x7f2c6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:50.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.037+0000 7f2c8aa00640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c6c06b500 0x7f2c6c06d9c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c153400 tx=0x7f2c7c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:50.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.040+0000 7f2c82a00640 1 -- 172.21.15.67:0/661494483 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2c78040070 con 0x7f2c8c072770 2024-03-09T09:28:50.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.316+0000 7f2c8be00640 1 -- 172.21.15.67:0/661494483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2c8c04f3c0 con 0x7f2c8c072770 2024-03-09T09:28:50.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.317+0000 7f2c82a00640 1 -- 172.21.15.67:0/661494483 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7f2c7807af30 con 0x7f2c8c072770 2024-03-09T09:28:50.325 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":286,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:28:50.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.320+0000 7f2c8be00640 1 -- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c6c06b500 msgr2=0x7f2c6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:50.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.320+0000 7f2c8be00640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c6c06b500 0x7f2c6c06d9c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c153400 tx=0x7f2c7c008040 comp rx=0 tx=0).stop 2024-03-09T09:28:50.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.320+0000 7f2c8be00640 1 -- 172.21.15.67:0/661494483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c8c072770 msgr2=0x7f2c8c0ad310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:50.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.320+0000 7f2c8be00640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c8c072770 0x7f2c8c0ad310 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0c7e90 tx=0x7f2c780046c0 comp rx=0 tx=0).stop 2024-03-09T09:28:50.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.320+0000 7f2c8be00640 1 -- 172.21.15.67:0/661494483 shutdown_connections 2024-03-09T09:28:50.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.320+0000 7f2c8be00640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f2c6c06b500 0x7f2c6c06d9c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:50.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.320+0000 7f2c8be00640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c8c0bebc0 0x7f2c8c157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:50.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.320+0000 7f2c8be00640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2c8c072770 0x7f2c8c0ad310 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:50.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.320+0000 7f2c8be00640 1 --2- 172.21.15.67:0/661494483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c8c06b690 0x7f2c8c0c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:50.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.320+0000 7f2c8be00640 1 -- 172.21.15.67:0/661494483 >> 172.21.15.67:0/661494483 conn(0x7f2c8c0c7fb0 msgr2=0x7f2c8c0c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:50.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.321+0000 7f2c8be00640 1 -- 172.21.15.67:0/661494483 shutdown_connections 2024-03-09T09:28:50.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:50.321+0000 7f2c8be00640 1 -- 172.21.15.67:0/661494483 wait complete. 2024-03-09T09:28:53.349 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:28:53.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.568+0000 7fbf11c00640 1 Processor -- start 2024-03-09T09:28:53.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.569+0000 7fbf11c00640 1 -- start start 2024-03-09T09:28:53.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.570+0000 7fbf11c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbf0c0c2630 0x7fbf0c06e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:53.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.570+0000 7fbf11c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0c151b50 con 0x7fbf0c0a73e0 2024-03-09T09:28:53.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.570+0000 7fbf11c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0c154dc0 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.570+0000 7fbf11c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0c154930 con 0x7fbf0c0c2630 2024-03-09T09:28:53.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.570+0000 7fbf02000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbf0c0c2630 0x7fbf0c06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:53.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.570+0000 7fbf0aa00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbf0c0a73e0 0x7fbf0c066d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34744/0 (socket says 172.21.15.67:34744) 2024-03-09T09:28:53.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.570+0000 7fbf0aa00640 1 -- 172.21.15.67:0/862763228 learned_addr learned my addr 172.21.15.67:0/862763228 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:28:53.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.571+0000 7fbf02000640 1 -- 172.21.15.67:0/862763228 >> v1:172.21.15.134:6789/0 conn(0x7fbf0c0c2a80 legacy=0x7fbf0c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:53.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.571+0000 7fbf02000640 1 -- 172.21.15.67:0/862763228 >> v1:172.21.15.67:6789/0 conn(0x7fbf0c0a73e0 legacy=0x7fbf0c066d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:53.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.571+0000 7fbf0a000640 1 -- 172.21.15.67:0/862763228 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2081972074 0 0) 0x7fbf0c151b50 con 0x7fbf0c0a73e0 2024-03-09T09:28:53.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.571+0000 7fbf02000640 1 -- 172.21.15.67:0/862763228 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c0734b0 con 0x7fbf0c0c2630 2024-03-09T09:28:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.573+0000 7fbf02000640 1 --2- 172.21.15.67:0/862763228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf0c0c2630 0x7fbf0c06e630 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbef40097a0 tx=0x7fbef402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df61d0419ff17c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.573+0000 7fbf0a000640 1 -- 172.21.15.67:0/862763228 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4035020 con 0x7fbf0c0c2630 2024-03-09T09:28:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.573+0000 7fbf0a000640 1 -- 172.21.15.67:0/862763228 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef4031830 con 0x7fbf0c0c2630 2024-03-09T09:28:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.573+0000 7fbf0a000640 1 -- 172.21.15.67:0/862763228 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4031c00 con 0x7fbf0c0c2630 2024-03-09T09:28:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.574+0000 7fbf11c00640 1 -- 172.21.15.67:0/862763228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf0c0c2630 msgr2=0x7fbf0c06e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.574+0000 7fbf11c00640 1 --2- 172.21.15.67:0/862763228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf0c0c2630 0x7fbf0c06e630 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbef40097a0 tx=0x7fbef402b8d0 comp rx=0 tx=0).stop 2024-03-09T09:28:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.574+0000 7fbf11c00640 1 -- 172.21.15.67:0/862763228 shutdown_connections 2024-03-09T09:28:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.574+0000 7fbf11c00640 1 --2- 172.21.15.67:0/862763228 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf0c0c2630 0x7fbf0c06e630 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.574+0000 7fbf11c00640 1 -- 172.21.15.67:0/862763228 >> 172.21.15.67:0/862763228 conn(0x7fbf0c0a9950 msgr2=0x7fbf0c0a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.574+0000 7fbf11c00640 1 -- 172.21.15.67:0/862763228 shutdown_connections 2024-03-09T09:28:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.574+0000 7fbf11c00640 1 -- 172.21.15.67:0/862763228 wait complete. 2024-03-09T09:28:53.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.575+0000 7fbf11c00640 1 Processor -- start 2024-03-09T09:28:53.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.576+0000 7fbf11c00640 1 -- start start 2024-03-09T09:28:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.576+0000 7fbf11c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c0a73e0 0x7fbf0c0c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.576+0000 7fbf11c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf0c0c2630 0x7fbf0c1ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.576+0000 7fbf11c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf0c0c2a80 0x7fbf0c1ec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.576+0000 7fbf11c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c06f450 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf11c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c06f170 con 0x7fbf0c0a73e0 2024-03-09T09:28:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf11c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c073720 con 0x7fbf0c0c2630 2024-03-09T09:28:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf0b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf0c0c2a80 0x7fbf0c1ec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf0b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf0c0c2a80 0x7fbf0c1ec1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34364/0 (socket says 172.21.15.67:34364) 2024-03-09T09:28:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf0b400640 1 -- 172.21.15.67:0/103289085 learned_addr learned my addr 172.21.15.67:0/103289085 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf0aa00640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c0a73e0 0x7fbf0c0c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf02000640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf0c0c2630 0x7fbf0c1ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf0b400640 1 -- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf0c0c2630 msgr2=0x7fbf0c1ebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf0b400640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf0c0c2630 0x7fbf0c1ebc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf0b400640 1 -- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c0a73e0 msgr2=0x7fbf0c0c0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf0b400640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c0a73e0 0x7fbf0c0c0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.577+0000 7fbf0b400640 1 -- 172.21.15.67:0/103289085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c1e2210 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.578+0000 7fbf0b400640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf0c0c2a80 0x7fbf0c1ec1d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b970 tx=0x7fbf0400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.578+0000 7fbf03400640 1 -- 172.21.15.67:0/103289085 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04019070 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.578+0000 7fbf03400640 1 -- 172.21.15.67:0/103289085 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0400cae0 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.578+0000 7fbf03400640 1 -- 172.21.15.67:0/103289085 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04014440 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.578+0000 7fbf11c00640 1 -- 172.21.15.67:0/103289085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0c1ef380 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.578+0000 7fbf11c00640 1 -- 172.21.15.67:0/103289085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0c1efe60 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.579+0000 7fbf11c00640 1 -- 172.21.15.67:0/103289085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0c1f00f0 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.581+0000 7fbf03400640 1 -- 172.21.15.67:0/103289085 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbf0401c070 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.581+0000 7fbf03400640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbee806b4d0 0x7fbee806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:53.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.582+0000 7fbf03400640 1 -- 172.21.15.67:0/103289085 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fbf0408a120 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.582+0000 7fbf0aa00640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbee806b4d0 0x7fbee806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:53.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.583+0000 7fbf0aa00640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbee806b4d0 0x7fbee806d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbefc000f60 tx=0x7fbefc007b70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.586+0000 7fbf03400640 1 -- 172.21.15.67:0/103289085 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbf04058c70 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.917+0000 7fbf11c00640 1 -- 172.21.15.67:0/103289085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbf0c075a00 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.918+0000 7fbf03400640 1 -- 172.21.15.67:0/103289085 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1560 (secure 0 0 0) 0x7fbf04058570 con 0x7fbf0c0c2a80 2024-03-09T09:28:53.925 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":286,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:28:53.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.920+0000 7fbf11c00640 1 -- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbee806b4d0 msgr2=0x7fbee806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.920+0000 7fbf11c00640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbee806b4d0 0x7fbee806d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbefc000f60 tx=0x7fbefc007b70 comp rx=0 tx=0).stop 2024-03-09T09:28:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.921+0000 7fbf11c00640 1 -- 172.21.15.67:0/103289085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf0c0c2a80 msgr2=0x7fbf0c1ec1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.921+0000 7fbf11c00640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf0c0c2a80 0x7fbf0c1ec1d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b970 tx=0x7fbf0400be30 comp rx=0 tx=0).stop 2024-03-09T09:28:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.921+0000 7fbf11c00640 1 -- 172.21.15.67:0/103289085 shutdown_connections 2024-03-09T09:28:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.921+0000 7fbf11c00640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbee806b4d0 0x7fbee806d990 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.921+0000 7fbf11c00640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf0c0c2a80 0x7fbf0c1ec1d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:53.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.921+0000 7fbf11c00640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf0c0c2630 0x7fbf0c1ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.921+0000 7fbf11c00640 1 --2- 172.21.15.67:0/103289085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf0c0a73e0 0x7fbf0c0c0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.921+0000 7fbf11c00640 1 -- 172.21.15.67:0/103289085 >> 172.21.15.67:0/103289085 conn(0x7fbf0c0a9950 msgr2=0x7fbf0c0c6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.921+0000 7fbf11c00640 1 -- 172.21.15.67:0/103289085 shutdown_connections 2024-03-09T09:28:53.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:53.921+0000 7fbf11c00640 1 -- 172.21.15.67:0/103289085 wait complete. 2024-03-09T09:28:56.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:28:57.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.153+0000 7f1cef200640 1 Processor -- start 2024-03-09T09:28:57.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.154+0000 7f1cef200640 1 -- start start 2024-03-09T09:28:57.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.155+0000 7f1cef200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cf01531c0 0x7f1cf01570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.155+0000 7f1cef200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf0156700 con 0x7f1cf00cae50 2024-03-09T09:28:57.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.155+0000 7f1cef200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf01569a0 con 0x7f1cf00c9440 2024-03-09T09:28:57.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.155+0000 7f1cef200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf00afba0 con 0x7f1cf01531c0 2024-03-09T09:28:57.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1ce5400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1cf00cae50 0x7f1cf00ae590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34760/0 (socket says 172.21.15.67:34760) 2024-03-09T09:28:57.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1ce5400640 1 -- 172.21.15.67:0/922427177 learned_addr learned my addr 172.21.15.67:0/922427177 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:28:57.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1cede00640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cf01531c0 0x7f1cf01570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039056888 0 0) 0x7f1cf0156700 con 0x7f1cf00cae50 2024-03-09T09:28:57.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cd4003520 con 0x7f1cf00cae50 2024-03-09T09:28:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1668377822 0 0) 0x7f1cd4003520 con 0x7f1cf00cae50 2024-03-09T09:28:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cf01531c0 msgr2=0x7f1cf01570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1ced400640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cf01531c0 0x7f1cf01570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 >> v1:172.21.15.134:6789/0 conn(0x7f1cf00c9440 legacy=0x7f1cf00c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf00b0410 con 0x7f1cf00cae50 2024-03-09T09:28:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f1cd8001690 con 0x7f1cf00cae50 2024-03-09T09:28:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.156+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 >> v1:172.21.15.67:6789/0 conn(0x7f1cf00cae50 legacy=0x7f1cf00ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ced400640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cf01531c0 0x7f1cd4003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ced400640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cd4004020 0x7f1cd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ced400640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd4004d40 0x7f1cd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd400bb50 con 0x7f1cd4004020 2024-03-09T09:28:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd400bd00 con 0x7f1cd4004d40 2024-03-09T09:28:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd400bf00 con 0x7f1cf01531c0 2024-03-09T09:28:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ce5400640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cd4004020 0x7f1cd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1cede00640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cf01531c0 0x7f1cd4003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1cee800640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd4004d40 0x7f1cd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ce5400640 1 -- 172.21.15.67:0/922427177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cf01531c0 msgr2=0x7f1cd4003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ce5400640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cf01531c0 0x7f1cd4003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ce5400640 1 -- 172.21.15.67:0/922427177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd4004d40 msgr2=0x7f1cd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ce5400640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cd4004d40 0x7f1cd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.157+0000 7f1ce5400640 1 -- 172.21.15.67:0/922427177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd4003520 con 0x7f1cd4004020 2024-03-09T09:28:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.159+0000 7f1ce5400640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cd4004020 0x7f1cd4004800 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800c160 tx=0x7f1cd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54091bc8f3db3fe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:57.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.159+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8007c60 con 0x7f1cd4004020 2024-03-09T09:28:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.159+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd8030070 con 0x7f1cd4004020 2024-03-09T09:28:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.159+0000 7f1ced400640 1 -- 172.21.15.67:0/922427177 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8005d10 con 0x7f1cd4004020 2024-03-09T09:28:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.160+0000 7f1cef200640 1 -- 172.21.15.67:0/922427177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cd4004020 msgr2=0x7f1cd4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.160+0000 7f1cef200640 1 --2- 172.21.15.67:0/922427177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cd4004020 0x7f1cd4004800 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800c160 tx=0x7f1cd4005830 comp rx=0 tx=0).stop 2024-03-09T09:28:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.160+0000 7f1cee800640 1 -- 172.21.15.67:0/922427177 reap_dead start 2024-03-09T09:28:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.160+0000 7f1cef200640 1 -- 172.21.15.67:0/922427177 shutdown_connections 2024-03-09T09:28:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.160+0000 7f1cef200640 1 -- 172.21.15.67:0/922427177 >> 172.21.15.67:0/922427177 conn(0x7f1cf00ad190 msgr2=0x7f1cf00ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:57.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.160+0000 7f1cef200640 1 -- 172.21.15.67:0/922427177 shutdown_connections 2024-03-09T09:28:57.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.160+0000 7f1cef200640 1 -- 172.21.15.67:0/922427177 wait complete. 2024-03-09T09:28:57.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.162+0000 7f1cef200640 1 Processor -- start 2024-03-09T09:28:57.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.162+0000 7f1cef200640 1 -- start start 2024-03-09T09:28:57.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.163+0000 7f1cef200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd4004020 0x7f1cf00ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.163+0000 7f1cef200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cf006ce30 0x7f1cf01ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.163+0000 7f1cef200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00c70b0 0x7f1cf00c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.163+0000 7f1cef200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf01535a0 con 0x7f1cf00c70b0 2024-03-09T09:28:57.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.163+0000 7f1cef200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf00b01c0 con 0x7f1cf006ce30 2024-03-09T09:28:57.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.163+0000 7f1cef200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf00cb2b0 con 0x7f1cd4004020 2024-03-09T09:28:57.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.163+0000 7f1cee800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00c70b0 0x7f1cf00c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.163+0000 7f1cee800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00c70b0 0x7f1cf00c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34392/0 (socket says 172.21.15.67:34392) 2024-03-09T09:28:57.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.163+0000 7f1cee800640 1 -- 172.21.15.67:0/1304795208 learned_addr learned my addr 172.21.15.67:0/1304795208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.163+0000 7f1cede00640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd4004020 0x7f1cf00ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.163+0000 7f1ce5400640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cf006ce30 0x7f1cf01ebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1cee800640 1 -- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd4004020 msgr2=0x7f1cf00ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1cee800640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd4004020 0x7f1cf00ab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1cee800640 1 -- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cf006ce30 msgr2=0x7f1cf01ebeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1cee800640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cf006ce30 0x7f1cf01ebeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1cee800640 1 -- 172.21.15.67:0/1304795208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf00c8000 con 0x7f1cf00c70b0 2024-03-09T09:28:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1ce5400640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cf006ce30 0x7f1cf01ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:28:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1cee800640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00c70b0 0x7f1cf00c7530 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00adf50 tx=0x7f1ce800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1ce6a00640 1 -- 172.21.15.67:0/1304795208 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8003970 con 0x7f1cf00c70b0 2024-03-09T09:28:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1ce6a00640 1 -- 172.21.15.67:0/1304795208 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce8018070 con 0x7f1cf00c70b0 2024-03-09T09:28:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1ce6a00640 1 -- 172.21.15.67:0/1304795208 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8003d70 con 0x7f1cf00c70b0 2024-03-09T09:28:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1cef200640 1 -- 172.21.15.67:0/1304795208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf00ac430 con 0x7f1cf00c70b0 2024-03-09T09:28:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.164+0000 7f1cef200640 1 -- 172.21.15.67:0/1304795208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf01eb6b0 con 0x7f1cf00c70b0 2024-03-09T09:28:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.166+0000 7f1cef200640 1 -- 172.21.15.67:0/1304795208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf0075a00 con 0x7f1cf00c70b0 2024-03-09T09:28:57.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.166+0000 7f1ce6a00640 1 -- 172.21.15.67:0/1304795208 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1ce80050d0 con 0x7f1cf00c70b0 2024-03-09T09:28:57.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.167+0000 7f1ce6a00640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ccc06b500 0x7f1ccc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.167+0000 7f1ce6a00640 1 -- 172.21.15.67:0/1304795208 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f1ce808a330 con 0x7f1cf00c70b0 2024-03-09T09:28:57.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.168+0000 7f1cede00640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ccc06b500 0x7f1ccc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.169+0000 7f1cede00640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ccc06b500 0x7f1ccc06d9c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1cf014bbc0 tx=0x7f1ce0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:57.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.172+0000 7f1ce6a00640 1 -- 172.21.15.67:0/1304795208 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1cf0075a00 con 0x7f1cf00c70b0 2024-03-09T09:28:57.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.505+0000 7f1cef200640 1 -- 172.21.15.67:0/1304795208 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1cf004f3c0 con 0x7f1cf00c70b0 2024-03-09T09:28:57.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.506+0000 7f1ce6a00640 1 -- 172.21.15.67:0/1304795208 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1565 (secure 0 0 0) 0x7f1ce8058de0 con 0x7f1cf00c70b0 2024-03-09T09:28:57.514 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":288,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:28:57.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.508+0000 7f1cef200640 1 -- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ccc06b500 msgr2=0x7f1ccc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.508+0000 7f1cef200640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ccc06b500 0x7f1ccc06d9c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1cf014bbc0 tx=0x7f1ce0008040 comp rx=0 tx=0).stop 2024-03-09T09:28:57.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.508+0000 7f1cef200640 1 -- 172.21.15.67:0/1304795208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00c70b0 msgr2=0x7f1cf00c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.508+0000 7f1cef200640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00c70b0 0x7f1cf00c7530 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00adf50 tx=0x7f1ce800ec10 comp rx=0 tx=0).stop 2024-03-09T09:28:57.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.509+0000 7f1cef200640 1 -- 172.21.15.67:0/1304795208 shutdown_connections 2024-03-09T09:28:57.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.509+0000 7f1cef200640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ccc06b500 0x7f1ccc06d9c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.509+0000 7f1cef200640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1cf00c70b0 0x7f1cf00c7530 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.509+0000 7f1cef200640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cf006ce30 0x7f1cf01ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.509+0000 7f1cef200640 1 --2- 172.21.15.67:0/1304795208 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1cd4004020 0x7f1cf00ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.509+0000 7f1cef200640 1 -- 172.21.15.67:0/1304795208 >> 172.21.15.67:0/1304795208 conn(0x7f1cf00ad190 msgr2=0x7f1cf00c59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:57.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.509+0000 7f1cef200640 1 -- 172.21.15.67:0/1304795208 shutdown_connections 2024-03-09T09:28:57.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.509+0000 7f1cef200640 1 -- 172.21.15.67:0/1304795208 wait complete. 2024-03-09T09:28:57.535 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:28:57.535 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-03-09T09:28:57.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-03-09T09:28:57.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.738+0000 7f7415000640 1 Processor -- start 2024-03-09T09:28:57.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.738+0000 7f7415000640 1 -- start start 2024-03-09T09:28:57.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.739+0000 7f7415000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f741006cdb0 0x7f74100a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.739+0000 7f7415000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f74100a7360 0x7f7410073140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.739+0000 7f7415000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7410154e20 con 0x7f741006e950 2024-03-09T09:28:57.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.739+0000 7f7415000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7410154900 con 0x7f74100a7360 2024-03-09T09:28:57.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.739+0000 7f7415000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7410067280 con 0x7f741006cdb0 2024-03-09T09:28:57.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.739+0000 7f740e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f74100a7360 0x7f7410073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.739+0000 7f740e800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f74100a7360 0x7f7410073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34406/0 (socket says 172.21.15.67:34406) 2024-03-09T09:28:57.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.739+0000 7f740e800640 1 -- 172.21.15.67:0/3690487272 learned_addr learned my addr 172.21.15.67:0/3690487272 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:57.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.740+0000 7f7405400640 1 --2- 172.21.15.67:0/3690487272 >> v2:172.21.15.134:3300/0 conn(0x7f741006cdb0 0x7f74100a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.740+0000 7f740e800640 1 -- 172.21.15.67:0/3690487272 >> v2:172.21.15.134:3300/0 conn(0x7f741006cdb0 msgr2=0x7f74100a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:57.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.740+0000 7f740e800640 1 --2- 172.21.15.67:0/3690487272 >> v2:172.21.15.134:3300/0 conn(0x7f741006cdb0 0x7f74100a6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.740+0000 7f740e800640 1 -- 172.21.15.67:0/3690487272 >> v1:172.21.15.134:6789/0 conn(0x7f741006e950 legacy=0x7f741006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:57.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.740+0000 7f740e800640 1 -- 172.21.15.67:0/3690487272 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7410066f50 con 0x7f74100a7360 2024-03-09T09:28:57.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.740+0000 7f7405400640 1 --2- 172.21.15.67:0/3690487272 >> v2:172.21.15.134:3300/0 conn(0x7f741006cdb0 0x7f74100a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:28:57.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.741+0000 7f740e800640 1 --2- 172.21.15.67:0/3690487272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100a7360 0x7f7410073140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74100c5710 tx=0x7f740802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7474638f3898f5ae server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:57.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.741+0000 7f740d400640 1 -- 172.21.15.67:0/3690487272 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740802e070 con 0x7f74100a7360 2024-03-09T09:28:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.741+0000 7f740d400640 1 -- 172.21.15.67:0/3690487272 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7408037070 con 0x7f74100a7360 2024-03-09T09:28:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.742+0000 7f740d400640 1 -- 172.21.15.67:0/3690487272 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408031a00 con 0x7f74100a7360 2024-03-09T09:28:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.742+0000 7f7415000640 1 -- 172.21.15.67:0/3690487272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100a7360 msgr2=0x7f7410073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.742+0000 7f7415000640 1 --2- 172.21.15.67:0/3690487272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100a7360 0x7f7410073140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74100c5710 tx=0x7f740802b880 comp rx=0 tx=0).stop 2024-03-09T09:28:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.742+0000 7f7415000640 1 -- 172.21.15.67:0/3690487272 shutdown_connections 2024-03-09T09:28:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.742+0000 7f7415000640 1 --2- 172.21.15.67:0/3690487272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100a7360 0x7f7410073140 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.742+0000 7f7415000640 1 --2- 172.21.15.67:0/3690487272 >> v2:172.21.15.134:3300/0 conn(0x7f741006cdb0 0x7f74100a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.742+0000 7f7415000640 1 -- 172.21.15.67:0/3690487272 >> 172.21.15.67:0/3690487272 conn(0x7f74100a9950 msgr2=0x7f74100a9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.742+0000 7f7415000640 1 -- 172.21.15.67:0/3690487272 shutdown_connections 2024-03-09T09:28:57.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.742+0000 7f7415000640 1 -- 172.21.15.67:0/3690487272 wait complete. 2024-03-09T09:28:57.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.744+0000 7f7415000640 1 Processor -- start 2024-03-09T09:28:57.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.744+0000 7f7415000640 1 -- start start 2024-03-09T09:28:57.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.745+0000 7f7415000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f741006cdb0 0x7f7410152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.745+0000 7f7415000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f741006e950 0x7f74101df600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.745+0000 7f7415000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100a7360 0x7f74101dfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.745+0000 7f7415000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74100c2200 con 0x7f74100a7360 2024-03-09T09:28:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.745+0000 7f7415000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7410154b40 con 0x7f741006e950 2024-03-09T09:28:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.745+0000 7f7415000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7410073680 con 0x7f741006cdb0 2024-03-09T09:28:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.745+0000 7f740e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100a7360 0x7f74101dfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.745+0000 7f740e800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100a7360 0x7f74101dfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34412/0 (socket says 172.21.15.67:34412) 2024-03-09T09:28:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.745+0000 7f740e800640 1 -- 172.21.15.67:0/2467384329 learned_addr learned my addr 172.21.15.67:0/2467384329 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f7405400640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f741006e950 0x7f74101df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f740e800640 1 -- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f741006cdb0 msgr2=0x7f7410152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f740de00640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f741006cdb0 0x7f7410152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f740e800640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f741006cdb0 0x7f7410152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f740e800640 1 -- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f741006e950 msgr2=0x7f74101df600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f740e800640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f741006e950 0x7f74101df600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f740e800640 1 -- 172.21.15.67:0/2467384329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74101ec800 con 0x7f74100a7360 2024-03-09T09:28:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f7405400640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f741006e950 0x7f74101df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:28:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f740de00640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f741006cdb0 0x7f7410152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:28:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f740e800640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100a7360 0x7f74101dfb40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f74080090b0 tx=0x7f74080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:57.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408032880 con 0x7f74100a7360 2024-03-09T09:28:57.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7408032a40 con 0x7f74100a7360 2024-03-09T09:28:57.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408032e30 con 0x7f74100a7360 2024-03-09T09:28:57.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.746+0000 7f7415000640 1 -- 172.21.15.67:0/2467384329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74100c41b0 con 0x7f74100a7360 2024-03-09T09:28:57.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.747+0000 7f7415000640 1 -- 172.21.15.67:0/2467384329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74100ab790 con 0x7f74100a7360 2024-03-09T09:28:57.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.748+0000 7f7415000640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c2f60 0x7f74100c3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.748+0000 7f7415000640 1 -- 172.21.15.67:0/2467384329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741004f3c0 con 0x7f74100c2f60 2024-03-09T09:28:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.748+0000 7f740de00640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c2f60 0x7f74100c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.748+0000 7f740de00640 1 -- 172.21.15.67:0/2467384329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7410071470 con 0x7f74100c2f60 2024-03-09T09:28:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.749+0000 7f740de00640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c2f60 0x7f74100c3340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f740000d830 tx=0x7f740000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.749+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400003b80 con 0x7f74100c2f60 2024-03-09T09:28:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.749+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f7400003d20 con 0x7f74100c2f60 2024-03-09T09:28:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.749+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c2f60 msgr2=0x7f74100c3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.749+0000 7f7406a00640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c2f60 0x7f74100c3340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f740000d830 tx=0x7f740000dcf0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.749+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c2f60 msgr2=0x7f74100c3340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.749+0000 7f7406a00640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c2f60 0x7f74100c3340 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f740000d830 tx=0x7f740000dcf0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.750+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f74080453f0 con 0x7f74100a7360 2024-03-09T09:28:57.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.751+0000 7f7406a00640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73ec06afb0 0x7f73ec06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.752+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f74080ac690 con 0x7f74100a7360 2024-03-09T09:28:57.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.752+0000 7f740de00640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73ec06afb0 0x7f73ec06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.753+0000 7f740de00640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73ec06afb0 0x7f73ec06d470 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f740000b540 tx=0x7f7400011000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:57.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.758+0000 7f7415000640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c3880 0x7f74101eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:57.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.758+0000 7f7415000640 1 -- 172.21.15.67:0/2467384329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74100c2ca0 con 0x7f74100c3880 2024-03-09T09:28:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.758+0000 7f7405400640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c3880 0x7f74101eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.758+0000 7f7405400640 1 -- 172.21.15.67:0/2467384329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f74100c3fa0 con 0x7f74100c3880 2024-03-09T09:28:57.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.759+0000 7f7405400640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c3880 0x7f74101eefa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74100ac940 tx=0x7f73f800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.759+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8016070 con 0x7f74100c3880 2024-03-09T09:28:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.759+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 <== mon.0 v2:172.21.15.67:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+5189 (secure 0 0 0) 0x7f73f80088d0 con 0x7f74100c3880 2024-03-09T09:28:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.759+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c3880 msgr2=0x7f74101eefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.759+0000 7f7406a00640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c3880 0x7f74101eefa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74100ac940 tx=0x7f73f800dbf0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.759+0000 7f7406a00640 1 -- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c3880 msgr2=0x7f74101eefa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.759+0000 7f7406a00640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100c3880 0x7f74101eefa0 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f74100ac940 tx=0x7f73f800dbf0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.768 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:28:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "leader", 2024-03-09T09:28:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 288, 2024-03-09T09:28:57.768 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:28:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:28:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:28:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:28:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:28:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:28:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:28:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:57.770 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:28:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:28:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:57.771 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:57.772 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:28:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:28:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:28:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:28:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:28:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:28:57.773 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:28:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:28:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:28:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:28:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:28:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:28:57.774 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:28:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:28:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:57.775 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:28:57.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:28:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.777 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:57.778 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:28:57.778 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:57.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.778 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.778 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:57.778 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:28:57.778 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:57.778 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:57.779 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:57.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.779 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:28:57.779 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:28:57.779 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:57.779 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:57.779 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:57.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:28:57.780 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:57.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.782 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:57.783 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:57.783 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:57.783 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:57.783 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.783 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:57.783 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:28:57.783 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:57.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.783 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:28:57.784 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:28:57.785 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:57.785 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:57.785 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:57.785 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:57.785 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:57.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.785 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:28:57.785 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:28:57.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.786 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:57.786 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:57.786 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:57.786 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:57.786 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:57.786 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:28:57.786 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.787 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:57.787 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:57.787 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 3 2024-03-09T09:28:57.787 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:57.787 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:57.787 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:28:57.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.787 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:57.787 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:57.788 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 7 2024-03-09T09:28:57.788 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:57.788 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:57.788 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:28:57.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.788 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x2f018fb87aa4aafe", 2024-03-09T09:28:57.788 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:57.788 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:28:57.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.789 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:57.789 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:57.789 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 6 2024-03-09T09:28:57.789 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:57.789 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:57.789 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr": [ 2024-03-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 2 2024-03-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:57.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:57.791 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:28:57.791 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:28:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.760+0000 7f7415000640 1 -- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73ec06afb0 msgr2=0x7f73ec06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.760+0000 7f7415000640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f73ec06afb0 0x7f73ec06d470 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f740000b540 tx=0x7f7400011000 comp rx=0 tx=0).stop 2024-03-09T09:28:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.760+0000 7f7415000640 1 -- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100a7360 msgr2=0x7f74101dfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:57.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.760+0000 7f7415000640 1 --2- 172.21.15.67:0/2467384329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74100a7360 0x7f74101dfb40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f74080090b0 tx=0x7f74080046c0 comp rx=0 tx=0).stop 2024-03-09T09:28:57.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.760+0000 7f740e800640 1 -- 172.21.15.67:0/2467384329 reap_dead start 2024-03-09T09:28:57.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.760+0000 7f740e800640 1 -- 172.21.15.67:0/2467384329 reap_dead start 2024-03-09T09:28:57.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.760+0000 7f7415000640 1 -- 172.21.15.67:0/2467384329 shutdown_connections 2024-03-09T09:28:57.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.760+0000 7f7415000640 1 -- 172.21.15.67:0/2467384329 >> 172.21.15.67:0/2467384329 conn(0x7f74100a9950 msgr2=0x7f741006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:57.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.760+0000 7f7415000640 1 -- 172.21.15.67:0/2467384329 shutdown_connections 2024-03-09T09:28:57.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:57.760+0000 7f7415000640 1 -- 172.21.15.67:0/2467384329 wait complete. 2024-03-09T09:28:57.794 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:28:57.794 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-03-09T09:28:57.794 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-03-09T09:28:58.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.019+0000 7f5794e00640 1 Processor -- start 2024-03-09T09:28:58.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.020+0000 7f5794e00640 1 -- start start 2024-03-09T09:28:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.021+0000 7f5794e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57900730c0 0x7f57900aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.021+0000 7f5794e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5790152e00 con 0x7f579006d830 2024-03-09T09:28:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.021+0000 7f5794e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57901530e0 con 0x7f57900734a0 2024-03-09T09:28:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.021+0000 7f5794e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57900aa960 con 0x7f57900730c0 2024-03-09T09:28:58.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.022+0000 7f578de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57900730c0 0x7f57900aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.022+0000 7f578e800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f579006d830 0x7f579006dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34776/0 (socket says 172.21.15.67:34776) 2024-03-09T09:28:58.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.022+0000 7f578e800640 1 -- 172.21.15.67:0/1809244796 learned_addr learned my addr 172.21.15.67:0/1809244796 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:28:58.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.022+0000 7f578d400640 1 -- 172.21.15.67:0/1809244796 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464943723 0 0) 0x7f5790152e00 con 0x7f579006d830 2024-03-09T09:28:58.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.022+0000 7f578de00640 1 -- 172.21.15.67:0/1809244796 >> v1:172.21.15.134:6789/0 conn(0x7f57900734a0 legacy=0x7f57900c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.022+0000 7f578de00640 1 -- 172.21.15.67:0/1809244796 >> v1:172.21.15.67:6789/0 conn(0x7f579006d830 legacy=0x7f579006dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.023+0000 7f578de00640 1 -- 172.21.15.67:0/1809244796 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57901533c0 con 0x7f57900730c0 2024-03-09T09:28:58.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.024+0000 7f578de00640 1 --2- 172.21.15.67:0/1809244796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57900730c0 0x7f57900aa420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f57800097d0 tx=0x7f578002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ddd09f8ddffff52 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.024+0000 7f578d400640 1 -- 172.21.15.67:0/1809244796 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780035020 con 0x7f57900730c0 2024-03-09T09:28:58.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.024+0000 7f578d400640 1 -- 172.21.15.67:0/1809244796 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5780031800 con 0x7f57900730c0 2024-03-09T09:28:58.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.024+0000 7f578d400640 1 -- 172.21.15.67:0/1809244796 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780031bd0 con 0x7f57900730c0 2024-03-09T09:28:58.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.025+0000 7f5794e00640 1 -- 172.21.15.67:0/1809244796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57900730c0 msgr2=0x7f57900aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.025+0000 7f5794e00640 1 --2- 172.21.15.67:0/1809244796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57900730c0 0x7f57900aa420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f57800097d0 tx=0x7f578002b8a0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.025+0000 7f5794e00640 1 -- 172.21.15.67:0/1809244796 shutdown_connections 2024-03-09T09:28:58.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.025+0000 7f5794e00640 1 --2- 172.21.15.67:0/1809244796 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57900730c0 0x7f57900aa420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.025+0000 7f5794e00640 1 -- 172.21.15.67:0/1809244796 >> 172.21.15.67:0/1809244796 conn(0x7f57900c7370 msgr2=0x7f57900c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:58.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.025+0000 7f5794e00640 1 -- 172.21.15.67:0/1809244796 shutdown_connections 2024-03-09T09:28:58.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.025+0000 7f5794e00640 1 -- 172.21.15.67:0/1809244796 wait complete. 2024-03-09T09:28:58.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.027+0000 7f5794e00640 1 Processor -- start 2024-03-09T09:28:58.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.027+0000 7f5794e00640 1 -- start start 2024-03-09T09:28:58.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.028+0000 7f5794e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579006d830 0x7f5790156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.028+0000 7f5794e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57900730c0 0x7f579014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.028+0000 7f5794e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900734a0 0x7f579014e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.028+0000 7f5794e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57900c3590 con 0x7f579006d830 2024-03-09T09:28:58.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.028+0000 7f5794e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790153980 con 0x7f57900734a0 2024-03-09T09:28:58.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.028+0000 7f5794e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57900c3710 con 0x7f57900730c0 2024-03-09T09:28:58.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.028+0000 7f578de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579006d830 0x7f5790156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.028+0000 7f578de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579006d830 0x7f5790156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34440/0 (socket says 172.21.15.67:34440) 2024-03-09T09:28:58.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.028+0000 7f578de00640 1 -- 172.21.15.67:0/1025713856 learned_addr learned my addr 172.21.15.67:0/1025713856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:58.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.028+0000 7f5787e00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57900730c0 0x7f579014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f578e800640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900734a0 0x7f579014e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f578de00640 1 -- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57900730c0 msgr2=0x7f579014dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f578de00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57900730c0 0x7f579014dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f578de00640 1 -- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900734a0 msgr2=0x7f579014e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f578de00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900734a0 0x7f579014e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f578de00640 1 -- 172.21.15.67:0/1025713856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57901ec310 con 0x7f579006d830 2024-03-09T09:28:58.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f578e800640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900734a0 0x7f579014e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:28:58.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f578de00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579006d830 0x7f5790156910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5780009720 tx=0x7f57800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780035050 con 0x7f579006d830 2024-03-09T09:28:58.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578003bcf0 con 0x7f579006d830 2024-03-09T09:28:58.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780004270 con 0x7f579006d830 2024-03-09T09:28:58.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.029+0000 7f5794e00640 1 -- 172.21.15.67:0/1025713856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5790153e60 con 0x7f579006d830 2024-03-09T09:28:58.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.030+0000 7f5794e00640 1 -- 172.21.15.67:0/1025713856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5790154420 con 0x7f579006d830 2024-03-09T09:28:58.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.031+0000 7f5794e00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579009ba40 0x7f5790154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.031+0000 7f5794e00640 1 -- 172.21.15.67:0/1025713856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790075a00 con 0x7f579009ba40 2024-03-09T09:28:58.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.031+0000 7f5787e00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579009ba40 0x7f5790154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.032+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5780038020 con 0x7f579006d830 2024-03-09T09:28:58.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.032+0000 7f5786000640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f576c06b4d0 0x7f576c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.032+0000 7f5787e00640 1 -- 172.21.15.67:0/1025713856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57900c3150 con 0x7f579009ba40 2024-03-09T09:28:58.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.033+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f57800ac720 con 0x7f579006d830 2024-03-09T09:28:58.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.033+0000 7f578e800640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f576c06b4d0 0x7f576c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.033+0000 7f5787e00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579009ba40 0x7f5790154940 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5790150050 tx=0x7f5778001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.034+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778018070 con 0x7f579009ba40 2024-03-09T09:28:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.034+0000 7f578e800640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f576c06b4d0 0x7f576c06d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f57880012e0 tx=0x7f5788009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.035+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f57780052e0 con 0x7f579009ba40 2024-03-09T09:28:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.035+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579009ba40 msgr2=0x7f5790154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.035+0000 7f5786000640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579009ba40 0x7f5790154940 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5790150050 tx=0x7f5778001e70 comp rx=0 tx=0).stop 2024-03-09T09:28:58.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.035+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579009ba40 msgr2=0x7f5790154940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:58.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.035+0000 7f5786000640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f579009ba40 0x7f5790154940 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5790150050 tx=0x7f5778001e70 comp rx=0 tx=0).stop 2024-03-09T09:28:58.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.044+0000 7f5794e00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900a9960 0x7f5790154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.044+0000 7f5794e00640 1 -- 172.21.15.67:0/1025713856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57900c2e70 con 0x7f57900a9960 2024-03-09T09:28:58.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.046+0000 7f5787e00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900a9960 0x7f5790154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.047+0000 7f5787e00640 1 -- 172.21.15.67:0/1025713856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f57900c2b90 con 0x7f57900a9960 2024-03-09T09:28:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.047+0000 7f5787e00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900a9960 0x7f5790154e80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5778007000 tx=0x7f5778001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.047+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778018030 con 0x7f57900a9960 2024-03-09T09:28:58.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.048+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 <== mon.1 v2:172.21.15.134:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4534 (secure 0 0 0) 0x7f57780069e0 con 0x7f57900a9960 2024-03-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.048+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900a9960 msgr2=0x7f5790154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.048+0000 7f5786000640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900a9960 0x7f5790154e80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5778007000 tx=0x7f5778001080 comp rx=0 tx=0).stop 2024-03-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.048+0000 7f5786000640 1 -- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900a9960 msgr2=0x7f5790154e80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.048+0000 7f5786000640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57900a9960 0x7f5790154e80 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5778007000 tx=0x7f5778001080 comp rx=0 tx=0).stop 2024-03-09T09:28:58.056 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 288, 2024-03-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:28:58.057 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:28:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 2, 2024-03-09T09:28:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:28:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:28:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:58.058 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:58.059 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:28:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:28:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:58.060 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.061 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:28:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:28:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:28:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:28:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:28:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:28:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:58.062 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:28:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:28:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:28:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:28:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:28:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:28:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:28:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:58.063 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:28:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:58.064 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:28:58.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:28:58.066 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:28:58.068 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:28:58.070 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:58.071 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:28:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:58.072 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:28:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:28:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:28:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:58.073 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:28:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:28:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.074 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:28:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:58.075 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:58.076 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.076 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.076 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:28:58.077 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:28:58.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.049+0000 7f5794e00640 1 -- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f576c06b4d0 msgr2=0x7f576c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.049+0000 7f5794e00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f576c06b4d0 0x7f576c06d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f57880012e0 tx=0x7f5788009040 comp rx=0 tx=0).stop 2024-03-09T09:28:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.049+0000 7f5794e00640 1 -- 172.21.15.67:0/1025713856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579006d830 msgr2=0x7f5790156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.049+0000 7f5794e00640 1 --2- 172.21.15.67:0/1025713856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f579006d830 0x7f5790156910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5780009720 tx=0x7f57800046c0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.049+0000 7f578e800640 1 -- 172.21.15.67:0/1025713856 reap_dead start 2024-03-09T09:28:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.049+0000 7f578e800640 1 -- 172.21.15.67:0/1025713856 reap_dead start 2024-03-09T09:28:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.049+0000 7f5794e00640 1 -- 172.21.15.67:0/1025713856 shutdown_connections 2024-03-09T09:28:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.049+0000 7f5794e00640 1 -- 172.21.15.67:0/1025713856 >> 172.21.15.67:0/1025713856 conn(0x7f57900c7370 msgr2=0x7f57900aaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.049+0000 7f5794e00640 1 -- 172.21.15.67:0/1025713856 shutdown_connections 2024-03-09T09:28:58.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.049+0000 7f5794e00640 1 -- 172.21.15.67:0/1025713856 wait complete. 2024-03-09T09:28:58.080 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:28:58.080 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-03-09T09:28:58.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-03-09T09:28:58.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.289+0000 7f5677e00640 1 Processor -- start 2024-03-09T09:28:58.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.289+0000 7f5677e00640 1 -- start start 2024-03-09T09:28:58.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.291+0000 7f5677e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f56780c5e00 0x7f56780acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.291+0000 7f5677e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56780c97d0 con 0x7f56780c61e0 2024-03-09T09:28:58.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.291+0000 7f5677e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56780ad7c0 con 0x7f56780c9380 2024-03-09T09:28:58.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.291+0000 7f5677e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56780c6690 con 0x7f56780c5e00 2024-03-09T09:28:58.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.291+0000 7f5676a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f56780c5e00 0x7f56780acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.291+0000 7f5676a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f56780c5e00 0x7f56780acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34454/0 (socket says 172.21.15.67:34454) 2024-03-09T09:28:58.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.291+0000 7f5676a00640 1 -- 172.21.15.67:0/4200447056 learned_addr learned my addr 172.21.15.67:0/4200447056 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:58.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.291+0000 7f5676000640 1 --1- 172.21.15.67:0/4200447056 >> v1:172.21.15.67:6789/0 conn(0x7f56780c61e0 0x7f567806c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:34786/0 (socket says 172.21.15.67:34786) 2024-03-09T09:28:58.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.292+0000 7f5676a00640 1 -- 172.21.15.67:0/4200447056 >> v1:172.21.15.134:6789/0 conn(0x7f56780c9380 legacy=0x7f5678160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:58.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.292+0000 7f5676a00640 1 -- 172.21.15.67:0/4200447056 >> v1:172.21.15.67:6789/0 conn(0x7f56780c61e0 legacy=0x7f567806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.292+0000 7f5676a00640 1 -- 172.21.15.67:0/4200447056 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56780ad4e0 con 0x7f56780c5e00 2024-03-09T09:28:58.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.292+0000 7f5675600640 1 -- 172.21.15.67:0/4200447056 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163320941 0 0) 0x7f56780c97d0 con 0x7f56780c61e0 2024-03-09T09:28:58.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.293+0000 7f5676a00640 1 --2- 172.21.15.67:0/4200447056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56780c5e00 0x7f56780acf00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5668009e80 tx=0x7f566802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66311e32aca8432a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.293+0000 7f5675600640 1 -- 172.21.15.67:0/4200447056 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566802cb00 con 0x7f56780c5e00 2024-03-09T09:28:58.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.293+0000 7f5675600640 1 -- 172.21.15.67:0/4200447056 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566802cca0 con 0x7f56780c5e00 2024-03-09T09:28:58.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.293+0000 7f5675600640 1 -- 172.21.15.67:0/4200447056 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668034610 con 0x7f56780c5e00 2024-03-09T09:28:58.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.294+0000 7f5677e00640 1 -- 172.21.15.67:0/4200447056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56780c5e00 msgr2=0x7f56780acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.294+0000 7f5677e00640 1 --2- 172.21.15.67:0/4200447056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56780c5e00 0x7f56780acf00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5668009e80 tx=0x7f566802bf60 comp rx=0 tx=0).stop 2024-03-09T09:28:58.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.294+0000 7f5677e00640 1 -- 172.21.15.67:0/4200447056 shutdown_connections 2024-03-09T09:28:58.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.294+0000 7f5677e00640 1 --2- 172.21.15.67:0/4200447056 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56780c5e00 0x7f56780acf00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.294+0000 7f5677e00640 1 -- 172.21.15.67:0/4200447056 >> 172.21.15.67:0/4200447056 conn(0x7f56780cb480 msgr2=0x7f56780cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:58.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.294+0000 7f5677e00640 1 -- 172.21.15.67:0/4200447056 shutdown_connections 2024-03-09T09:28:58.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.294+0000 7f5677e00640 1 -- 172.21.15.67:0/4200447056 wait complete. 2024-03-09T09:28:58.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.297+0000 7f5677e00640 1 Processor -- start 2024-03-09T09:28:58.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.297+0000 7f5677e00640 1 -- start start 2024-03-09T09:28:58.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5677e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56780c5e00 0x7f5678158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5677e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56780c61e0 0x7f56780abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5677e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56780c9380 0x7f56781ebbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5677e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5678153680 con 0x7f56780c5e00 2024-03-09T09:28:58.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5677e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567806bde0 con 0x7f56780c61e0 2024-03-09T09:28:58.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5677e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567806c0c0 con 0x7f56780c9380 2024-03-09T09:28:58.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5676a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56780c5e00 0x7f5678158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5676a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56780c5e00 0x7f5678158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34462/0 (socket says 172.21.15.67:34462) 2024-03-09T09:28:58.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5676a00640 1 -- 172.21.15.67:0/1766995776 learned_addr learned my addr 172.21.15.67:0/1766995776 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:58.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5676000640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56780c61e0 0x7f56780abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5677400640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56780c9380 0x7f56781ebbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5676a00640 1 -- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56780c9380 msgr2=0x7f56781ebbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5676a00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56780c9380 0x7f56781ebbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5676a00640 1 -- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56780c61e0 msgr2=0x7f56780abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5676a00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56780c61e0 0x7f56780abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5676a00640 1 -- 172.21.15.67:0/1766995776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56780c4db0 con 0x7f56780c5e00 2024-03-09T09:28:58.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.299+0000 7f5676000640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56780c61e0 0x7f56780abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:28:58.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.300+0000 7f5676a00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56780c5e00 0x7f5678158f80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5668036040 tx=0x7f5668004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.300+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668043020 con 0x7f56780c5e00 2024-03-09T09:28:58.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.300+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566802ce50 con 0x7f56780c5e00 2024-03-09T09:28:58.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.300+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668009510 con 0x7f56780c5e00 2024-03-09T09:28:58.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.300+0000 7f5677e00640 1 -- 172.21.15.67:0/1766995776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56781594c0 con 0x7f56780c5e00 2024-03-09T09:28:58.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.301+0000 7f5677e00640 1 -- 172.21.15.67:0/1766995776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56780c5330 con 0x7f56780c5e00 2024-03-09T09:28:58.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.302+0000 7f5667e00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648002a70 0x7f5648004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.302+0000 7f5667e00640 1 -- 172.21.15.67:0/1766995776 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5648007ec0 con 0x7f5648002a70 2024-03-09T09:28:58.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.302+0000 7f5676000640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648002a70 0x7f5648004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.303+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5668046020 con 0x7f56780c5e00 2024-03-09T09:28:58.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.303+0000 7f5676000640 1 -- 172.21.15.67:0/1766995776 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56480080c0 con 0x7f5648002a70 2024-03-09T09:28:58.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.304+0000 7f5676000640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648002a70 0x7f5648004f30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f56780bec40 tx=0x7f566000d950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.304+0000 7f566ea00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f565406b4d0 0x7f565406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.304+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f56680acde0 con 0x7f56780c5e00 2024-03-09T09:28:58.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.304+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660018070 con 0x7f5648002a70 2024-03-09T09:28:58.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.305+0000 7f5677400640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f565406b4d0 0x7f565406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.305+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9324 (secure 0 0 0) 0x7f56600074e0 con 0x7f5648002a70 2024-03-09T09:28:58.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.305+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648002a70 msgr2=0x7f5648004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.305+0000 7f566ea00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648002a70 0x7f5648004f30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f56780bec40 tx=0x7f566000d950 comp rx=0 tx=0).stop 2024-03-09T09:28:58.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.305+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648002a70 msgr2=0x7f5648004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:58.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.305+0000 7f566ea00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648002a70 0x7f5648004f30 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f56780bec40 tx=0x7f566000d950 comp rx=0 tx=0).stop 2024-03-09T09:28:58.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.306+0000 7f5677400640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f565406b4d0 0x7f565406d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f56780c44c0 tx=0x7f5670009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.312+0000 7f5667e00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648006f00 0x7f5648007390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.313+0000 7f5667e00640 1 -- 172.21.15.67:0/1766995776 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564800a4a0 con 0x7f5648006f00 2024-03-09T09:28:58.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.313+0000 7f5676000640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648006f00 0x7f5648007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.314+0000 7f5676000640 1 -- 172.21.15.67:0/1766995776 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f56480078d0 con 0x7f5648006f00 2024-03-09T09:28:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.315+0000 7f5676000640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648006f00 0x7f5648007390 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5660015000 tx=0x7f5660001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.315+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660018050 con 0x7f5648006f00 2024-03-09T09:28:58.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 <== mon.2 v2:172.21.15.134:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4874 (secure 0 0 0) 0x7f5660019040 con 0x7f5648006f00 2024-03-09T09:28:58.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648006f00 msgr2=0x7f5648007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f566ea00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648006f00 0x7f5648007390 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5660015000 tx=0x7f5660001080 comp rx=0 tx=0).stop 2024-03-09T09:28:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f566ea00640 1 -- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648006f00 msgr2=0x7f5648007390 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-09T09:28:58.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f566ea00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5648006f00 0x7f5648007390 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.323 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:28:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "peon", 2024-03-09T09:28:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "election_epoch": 288, 2024-03-09T09:28:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum": [ 2024-03-09T09:28:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: 0, 2024-03-09T09:28:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: 1, 2024-03-09T09:28:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: 2 2024-03-09T09:28:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_age": 3, 2024-03-09T09:28:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "required_con": "2450521708007866372", 2024-03-09T09:28:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "required_mon": [ 2024-03-09T09:28:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_con": "4540701547738038271", 2024-03-09T09:28:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "quorum_mon": [ 2024-03-09T09:28:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "outside_quorum": [], 2024-03-09T09:28:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "extra_probe_peers": [], 2024-03-09T09:28:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "sync_provider": [], 2024-03-09T09:28:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": { 2024-03-09T09:28:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 1, 2024-03-09T09:28:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "aded9f77-d43e-4509-848a-632eae6a080f", 2024-03-09T09:28:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: "modified": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: "created": "2024-03-09T08:51:10.767075Z", 2024-03-09T09:28:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release": 19, 2024-03-09T09:28:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: "min_mon_release_name": "squid", 2024-03-09T09:28:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: "election_strategy": 1, 2024-03-09T09:28:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: "disallowed_leaders: ": "", 2024-03-09T09:28:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false, 2024-03-09T09:28:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "tiebreaker_mon": "", 2024-03-09T09:28:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "removed_ranks: ": "", 2024-03-09T09:28:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "features": { 2024-03-09T09:28:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "persistent": [ 2024-03-09T09:28:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "kraken", 2024-03-09T09:28:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "luminous", 2024-03-09T09:28:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "mimic", 2024-03-09T09:28:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap-prune", 2024-03-09T09:28:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "nautilus", 2024-03-09T09:28:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus", 2024-03-09T09:28:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific", 2024-03-09T09:28:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "elector-pinging", 2024-03-09T09:28:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy", 2024-03-09T09:28:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "reef", 2024-03-09T09:28:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "squid" 2024-03-09T09:28:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "optional": [] 2024-03-09T09:28:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "mons": [ 2024-03-09T09:28:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 0, 2024-03-09T09:28:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "a", 2024-03-09T09:28:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:3300", 2024-03-09T09:28:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789", 2024-03-09T09:28:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6789/0", 2024-03-09T09:28:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.67:6789/0", 2024-03-09T09:28:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 1, 2024-03-09T09:28:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "b", 2024-03-09T09:28:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3300", 2024-03-09T09:28:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789", 2024-03-09T09:28:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6789/0", 2024-03-09T09:28:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6789/0", 2024-03-09T09:28:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "rank": 2, 2024-03-09T09:28:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "c", 2024-03-09T09:28:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrs": { 2024-03-09T09:28:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-09T09:28:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-09T09:28:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:3301", 2024-03-09T09:28:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-09T09:28:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790", 2024-03-09T09:28:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 0 2024-03-09T09:28:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.134:6790/0", 2024-03-09T09:28:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "172.21.15.134:6790/0", 2024-03-09T09:28:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "priority": 0, 2024-03-09T09:28:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "weight": 0, 2024-03-09T09:28:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "{}" 2024-03-09T09:28:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "feature_map": { 2024-03-09T09:28:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "mon": [ 2024-03-09T09:28:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": [ 2024-03-09T09:28:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "osd": [ 2024-03-09T09:28:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-09T09:28:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: "client": [ 2024-03-09T09:28:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-09T09:28:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "features": "0x3f03cffffffdffff", 2024-03-09T09:28:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "release": "luminous", 2024-03-09T09:28:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 1 2024-03-09T09:28:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-09T09:28:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-09T09:28:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-09T09:28:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "stretch_mode": false 2024-03-09T09:28:58.344 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:28:58.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f5667e00640 1 -- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f565406b4d0 msgr2=0x7f565406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f5667e00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f565406b4d0 0x7f565406d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f56780c44c0 tx=0x7f5670009040 comp rx=0 tx=0).stop 2024-03-09T09:28:58.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f5667e00640 1 -- 172.21.15.67:0/1766995776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56780c5e00 msgr2=0x7f5678158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f5667e00640 1 --2- 172.21.15.67:0/1766995776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56780c5e00 0x7f5678158f80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5668036040 tx=0x7f5668004a40 comp rx=0 tx=0).stop 2024-03-09T09:28:58.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f5677400640 1 -- 172.21.15.67:0/1766995776 reap_dead start 2024-03-09T09:28:58.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f5677400640 1 -- 172.21.15.67:0/1766995776 reap_dead start 2024-03-09T09:28:58.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f5667e00640 1 -- 172.21.15.67:0/1766995776 shutdown_connections 2024-03-09T09:28:58.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.316+0000 7f5667e00640 1 -- 172.21.15.67:0/1766995776 >> 172.21.15.67:0/1766995776 conn(0x7f56780cb480 msgr2=0x7f56780ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:58.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.317+0000 7f5667e00640 1 -- 172.21.15.67:0/1766995776 shutdown_connections 2024-03-09T09:28:58.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.317+0000 7f5667e00640 1 -- 172.21.15.67:0/1766995776 wait complete. 2024-03-09T09:28:58.347 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-03-09T09:28:58.347 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-03-09T09:28:58.348 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-03-09T09:28:58.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.557+0000 7f590b200640 1 Processor -- start 2024-03-09T09:28:58.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.557+0000 7f590b200640 1 -- start start 2024-03-09T09:28:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.559+0000 7f590b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f590c0c1f70 0x7f590c150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.559+0000 7f590b200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f590c0710a0 0x7f590c151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.559+0000 7f590b200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c0c81e0 con 0x7f590c0c27a0 2024-03-09T09:28:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.559+0000 7f590b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590c0c32b0 con 0x7f590c0c1f70 2024-03-09T09:28:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.559+0000 7f590b200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590c0c2fd0 con 0x7f590c0710a0 2024-03-09T09:28:58.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.559+0000 7f5909e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f590c0c1f70 0x7f590c150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.559+0000 7f5909e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f590c0c1f70 0x7f590c150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34470/0 (socket says 172.21.15.67:34470) 2024-03-09T09:28:58.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.559+0000 7f5909e00640 1 -- 172.21.15.67:0/39730608 learned_addr learned my addr 172.21.15.67:0/39730608 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:58.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.560+0000 7f5903e00640 1 --2- 172.21.15.67:0/39730608 >> v2:172.21.15.134:3300/0 conn(0x7f590c0710a0 0x7f590c151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.560+0000 7f5909e00640 1 -- 172.21.15.67:0/39730608 >> v2:172.21.15.134:3300/0 conn(0x7f590c0710a0 msgr2=0x7f590c151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.560+0000 7f5909e00640 1 --2- 172.21.15.67:0/39730608 >> v2:172.21.15.134:3300/0 conn(0x7f590c0710a0 0x7f590c151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.560+0000 7f5909e00640 1 -- 172.21.15.67:0/39730608 >> v1:172.21.15.134:6789/0 conn(0x7f590c0c27a0 legacy=0x7f590c153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:28:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.560+0000 7f5909e00640 1 -- 172.21.15.67:0/39730608 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c0c3700 con 0x7f590c0c1f70 2024-03-09T09:28:58.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.562+0000 7f5909e00640 1 --2- 172.21.15.67:0/39730608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0c1f70 0x7f590c150c50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f590c06bf40 tx=0x7f58fc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd27c72ca889a86 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.563+0000 7f5909400640 1 -- 172.21.15.67:0/39730608 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc02e070 con 0x7f590c0c1f70 2024-03-09T09:28:58.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.563+0000 7f5909400640 1 -- 172.21.15.67:0/39730608 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc037070 con 0x7f590c0c1f70 2024-03-09T09:28:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.563+0000 7f5909400640 1 -- 172.21.15.67:0/39730608 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc031a00 con 0x7f590c0c1f70 2024-03-09T09:28:58.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.563+0000 7f590b200640 1 -- 172.21.15.67:0/39730608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0c1f70 msgr2=0x7f590c150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.563+0000 7f590b200640 1 --2- 172.21.15.67:0/39730608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0c1f70 0x7f590c150c50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f590c06bf40 tx=0x7f58fc02b880 comp rx=0 tx=0).stop 2024-03-09T09:28:58.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.564+0000 7f590b200640 1 -- 172.21.15.67:0/39730608 shutdown_connections 2024-03-09T09:28:58.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.564+0000 7f590b200640 1 --2- 172.21.15.67:0/39730608 >> v2:172.21.15.134:3300/0 conn(0x7f590c0710a0 0x7f590c151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.564+0000 7f590b200640 1 --2- 172.21.15.67:0/39730608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0c1f70 0x7f590c150c50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.564+0000 7f590b200640 1 -- 172.21.15.67:0/39730608 >> 172.21.15.67:0/39730608 conn(0x7f590c0c7300 msgr2=0x7f590c0c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:58.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.564+0000 7f590b200640 1 -- 172.21.15.67:0/39730608 shutdown_connections 2024-03-09T09:28:58.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.564+0000 7f590b200640 1 -- 172.21.15.67:0/39730608 wait complete. 2024-03-09T09:28:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.565+0000 7f590b200640 1 Processor -- start 2024-03-09T09:28:58.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.566+0000 7f590b200640 1 -- start start 2024-03-09T09:28:58.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.566+0000 7f590b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0710a0 0x7f590c14da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.566+0000 7f590b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0c1f70 0x7f590c1ea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.566+0000 7f590b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0c27a0 0x7f590c1eac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f590b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c070d50 con 0x7f590c0c27a0 2024-03-09T09:28:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f590b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c070830 con 0x7f590c0c1f70 2024-03-09T09:28:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f590b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c070ed0 con 0x7f590c0710a0 2024-03-09T09:28:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f590a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0c27a0 0x7f590c1eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f590a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0c27a0 0x7f590c1eac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34484/0 (socket says 172.21.15.67:34484) 2024-03-09T09:28:58.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f590a800640 1 -- 172.21.15.67:0/2749573764 learned_addr learned my addr 172.21.15.67:0/2749573764 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:28:58.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f5909e00640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0710a0 0x7f590c14da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f5903e00640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0c1f70 0x7f590c1ea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f590a800640 1 -- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0710a0 msgr2=0x7f590c14da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f590a800640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0710a0 0x7f590c14da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f590a800640 1 -- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0c1f70 msgr2=0x7f590c1ea6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f590a800640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0c1f70 0x7f590c1ea6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.567+0000 7f590a800640 1 -- 172.21.15.67:0/2749573764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c1e7340 con 0x7f590c0c27a0 2024-03-09T09:28:58.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.568+0000 7f590a800640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0c27a0 0x7f590c1eac00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f590400b970 tx=0x7f590400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.568+0000 7f5902000640 1 -- 172.21.15.67:0/2749573764 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904019070 con 0x7f590c0c27a0 2024-03-09T09:28:58.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.568+0000 7f5902000640 1 -- 172.21.15.67:0/2749573764 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590400cb40 con 0x7f590c0c27a0 2024-03-09T09:28:58.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.568+0000 7f5902000640 1 -- 172.21.15.67:0/2749573764 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904014440 con 0x7f590c0c27a0 2024-03-09T09:28:58.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.568+0000 7f590b200640 1 -- 172.21.15.67:0/2749573764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c1e7620 con 0x7f590c0c27a0 2024-03-09T09:28:58.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.568+0000 7f590b200640 1 -- 172.21.15.67:0/2749573764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c1e7be0 con 0x7f590c0c27a0 2024-03-09T09:28:58.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.569+0000 7f590b200640 1 -- 172.21.15.67:0/2749573764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c04f3c0 con 0x7f590c0c27a0 2024-03-09T09:28:58.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.570+0000 7f5902000640 1 -- 172.21.15.67:0/2749573764 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f590401c070 con 0x7f590c0c27a0 2024-03-09T09:28:58.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.571+0000 7f5902000640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58e806b4d0 0x7f58e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:28:58.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.571+0000 7f5902000640 1 -- 172.21.15.67:0/2749573764 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f590408a270 con 0x7f590c0c27a0 2024-03-09T09:28:58.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.572+0000 7f5909e00640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58e806b4d0 0x7f58e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:28:58.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.573+0000 7f5909e00640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58e806b4d0 0x7f58e806d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f58fc036000 tx=0x7f58fc03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:28:58.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.576+0000 7f5902000640 1 -- 172.21.15.67:0/2749573764 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5904058ca0 con 0x7f590c0c27a0 2024-03-09T09:28:58.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.852+0000 7f590b200640 1 -- 172.21.15.67:0/2749573764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f590c1506e0 con 0x7f590c0c27a0 2024-03-09T09:28:58.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.854+0000 7f5902000640 1 -- 172.21.15.67:0/2749573764 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f59040585a0 con 0x7f590c0c27a0 2024-03-09T09:28:58.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 -- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58e806b4d0 msgr2=0x7f58e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58e806b4d0 0x7f58e806d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f58fc036000 tx=0x7f58fc03d040 comp rx=0 tx=0).stop 2024-03-09T09:28:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 -- 172.21.15.67:0/2749573764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0c27a0 msgr2=0x7f590c1eac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:28:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0c27a0 0x7f590c1eac00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f590400b970 tx=0x7f590400be30 comp rx=0 tx=0).stop 2024-03-09T09:28:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 -- 172.21.15.67:0/2749573764 shutdown_connections 2024-03-09T09:28:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f58e806b4d0 0x7f58e806d990 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f590c0c27a0 0x7f590c1eac00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f590c0c1f70 0x7f590c1ea6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 --2- 172.21.15.67:0/2749573764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f590c0710a0 0x7f590c14da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:28:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 -- 172.21.15.67:0/2749573764 >> 172.21.15.67:0/2749573764 conn(0x7f590c0c7300 msgr2=0x7f590c0733a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:28:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 -- 172.21.15.67:0/2749573764 shutdown_connections 2024-03-09T09:28:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:28:58.858+0000 7f590b200640 1 -- 172.21.15.67:0/2749573764 wait complete. 2024-03-09T09:28:58.883 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-03-09T09:29:04.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:29:04.528 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:29:08.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T09:29:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.107+0000 7feb3b200640 1 Processor -- start 2024-03-09T09:29:09.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.107+0000 7feb3b200640 1 -- start start 2024-03-09T09:29:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.109+0000 7feb3b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7feb3c0c5e40 0x7feb3c156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.109+0000 7feb3b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3c157000 con 0x7feb3c0cadb0 2024-03-09T09:29:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.110+0000 7feb3b200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3c0ae430 con 0x7feb3c153300 2024-03-09T09:29:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.110+0000 7feb3b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3c0ae6f0 con 0x7feb3c0c5e40 2024-03-09T09:29:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.110+0000 7feb3a800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7feb3c0cadb0 0x7feb3c156550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55648/0 (socket says 172.21.15.67:55648) 2024-03-09T09:29:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.110+0000 7feb3a800640 1 -- 172.21.15.67:0/1737617996 learned_addr learned my addr 172.21.15.67:0/1737617996 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.110+0000 7feb33e00640 1 --2- 172.21.15.67:0/1737617996 >> v2:172.21.15.67:3300/0 conn(0x7feb3c0c5e40 0x7feb3c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.110+0000 7feb39400640 1 -- 172.21.15.67:0/1737617996 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099294669 0 0) 0x7feb3c157000 con 0x7feb3c0cadb0 2024-03-09T09:29:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.110+0000 7feb39400640 1 -- 172.21.15.67:0/1737617996 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb20003520 con 0x7feb3c0cadb0 2024-03-09T09:29:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.111+0000 7feb33e00640 1 -- 172.21.15.67:0/1737617996 >> v1:172.21.15.134:6789/0 conn(0x7feb3c153300 legacy=0x7feb3c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.111+0000 7feb33e00640 1 -- 172.21.15.67:0/1737617996 >> v1:172.21.15.67:6789/0 conn(0x7feb3c0cadb0 legacy=0x7feb3c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.111+0000 7feb33e00640 1 -- 172.21.15.67:0/1737617996 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c155b10 con 0x7feb3c0c5e40 2024-03-09T09:29:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.112+0000 7feb33e00640 1 --2- 172.21.15.67:0/1737617996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb3c0c5e40 0x7feb3c156010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb240097a0 tx=0x7feb2402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3d4491f7cfec3e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.112+0000 7feb39400640 1 -- 172.21.15.67:0/1737617996 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24034070 con 0x7feb3c0c5e40 2024-03-09T09:29:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.112+0000 7feb39400640 1 -- 172.21.15.67:0/1737617996 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb24030610 con 0x7feb3c0c5e40 2024-03-09T09:29:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.112+0000 7feb39400640 1 -- 172.21.15.67:0/1737617996 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb240309e0 con 0x7feb3c0c5e40 2024-03-09T09:29:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.113+0000 7feb3b200640 1 -- 172.21.15.67:0/1737617996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb3c0c5e40 msgr2=0x7feb3c156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.113+0000 7feb3b200640 1 --2- 172.21.15.67:0/1737617996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb3c0c5e40 0x7feb3c156010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb240097a0 tx=0x7feb2402b960 comp rx=0 tx=0).stop 2024-03-09T09:29:09.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.113+0000 7feb3b200640 1 -- 172.21.15.67:0/1737617996 shutdown_connections 2024-03-09T09:29:09.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.113+0000 7feb3b200640 1 --2- 172.21.15.67:0/1737617996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb3c0c5e40 0x7feb3c156010 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.113+0000 7feb3b200640 1 -- 172.21.15.67:0/1737617996 >> 172.21.15.67:0/1737617996 conn(0x7feb3c0732d0 msgr2=0x7feb3c0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:09.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.113+0000 7feb3b200640 1 -- 172.21.15.67:0/1737617996 shutdown_connections 2024-03-09T09:29:09.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.113+0000 7feb3b200640 1 -- 172.21.15.67:0/1737617996 wait complete. 2024-03-09T09:29:09.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.115+0000 7feb3b200640 1 Processor -- start 2024-03-09T09:29:09.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.115+0000 7feb3b200640 1 -- start start 2024-03-09T09:29:09.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.116+0000 7feb3b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb3c0c5e40 0x7feb3c0c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:09.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.116+0000 7feb3b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb3c0cadb0 0x7feb3c152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:09.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.116+0000 7feb3b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb3c153300 0x7feb3c14dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:09.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.116+0000 7feb3b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c0c59e0 con 0x7feb3c153300 2024-03-09T09:29:09.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.116+0000 7feb3b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c159e90 con 0x7feb3c0cadb0 2024-03-09T09:29:09.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.116+0000 7feb3b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c0b0710 con 0x7feb3c0c5e40 2024-03-09T09:29:09.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.116+0000 7feb3a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb3c153300 0x7feb3c14dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:09.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.116+0000 7feb3a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb3c153300 0x7feb3c14dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46606/0 (socket says 172.21.15.67:46606) 2024-03-09T09:29:09.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.116+0000 7feb3a800640 1 -- 172.21.15.67:0/2594712824 learned_addr learned my addr 172.21.15.67:0/2594712824 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:09.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.116+0000 7feb33e00640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb3c0cadb0 0x7feb3c152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:09.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.116+0000 7feb39e00640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb3c0c5e40 0x7feb3c0c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:09.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb3a800640 1 -- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb3c0c5e40 msgr2=0x7feb3c0c23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb3a800640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb3c0c5e40 0x7feb3c0c23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb3a800640 1 -- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb3c0cadb0 msgr2=0x7feb3c152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb3a800640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb3c0cadb0 0x7feb3c152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb3a800640 1 -- 172.21.15.67:0/2594712824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c06c4b0 con 0x7feb3c153300 2024-03-09T09:29:09.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb39e00640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb3c0c5e40 0x7feb3c0c23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:09.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb3a800640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb3c153300 0x7feb3c14dd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feb3400cb40 tx=0x7feb3400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:09.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb32000640 1 -- 172.21.15.67:0/2594712824 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34017070 con 0x7feb3c153300 2024-03-09T09:29:09.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb32000640 1 -- 172.21.15.67:0/2594712824 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3400aca0 con 0x7feb3c153300 2024-03-09T09:29:09.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb32000640 1 -- 172.21.15.67:0/2594712824 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34012620 con 0x7feb3c153300 2024-03-09T09:29:09.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb3b200640 1 -- 172.21.15.67:0/2594712824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3c1eca30 con 0x7feb3c153300 2024-03-09T09:29:09.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.117+0000 7feb3b200640 1 -- 172.21.15.67:0/2594712824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3c06c1d0 con 0x7feb3c153300 2024-03-09T09:29:09.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.119+0000 7feb3b200640 1 -- 172.21.15.67:0/2594712824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3c04f3c0 con 0x7feb3c153300 2024-03-09T09:29:09.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.120+0000 7feb32000640 1 -- 172.21.15.67:0/2594712824 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7feb3401a070 con 0x7feb3c153300 2024-03-09T09:29:09.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.121+0000 7feb32000640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb1806b4d0 0x7feb1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:09.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.121+0000 7feb32000640 1 -- 172.21.15.67:0/2594712824 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7feb3408a430 con 0x7feb3c153300 2024-03-09T09:29:09.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.121+0000 7feb39e00640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb1806b4d0 0x7feb1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:09.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.123+0000 7feb39e00640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb1806b4d0 0x7feb1806d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0ad090 tx=0x7feb2c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:09.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.126+0000 7feb32000640 1 -- 172.21.15.67:0/2594712824 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feb34058ee0 con 0x7feb3c153300 2024-03-09T09:29:09.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.365+0000 7feb3b200640 1 -- 172.21.15.67:0/2594712824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb3c0cab90 con 0x7feb3c153300 2024-03-09T09:29:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.367+0000 7feb32000640 1 -- 172.21.15.67:0/2594712824 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6247 (secure 0 0 0) 0x7feb340587e0 con 0x7feb3c153300 2024-03-09T09:29:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:29:09.374 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:17.736973+0000","modified":"2024-03-09T08:52:21.931108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T09:29:09.375 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 14 2024-03-09T09:29:09.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 -- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb1806b4d0 msgr2=0x7feb1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb1806b4d0 0x7feb1806d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0ad090 tx=0x7feb2c009040 comp rx=0 tx=0).stop 2024-03-09T09:29:09.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 -- 172.21.15.67:0/2594712824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb3c153300 msgr2=0x7feb3c14dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb3c153300 0x7feb3c14dd80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feb3400cb40 tx=0x7feb3400a060 comp rx=0 tx=0).stop 2024-03-09T09:29:09.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 -- 172.21.15.67:0/2594712824 shutdown_connections 2024-03-09T09:29:09.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7feb1806b4d0 0x7feb1806d990 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb3c153300 0x7feb3c14dd80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb3c0cadb0 0x7feb3c152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 --2- 172.21.15.67:0/2594712824 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb3c0c5e40 0x7feb3c0c23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 -- 172.21.15.67:0/2594712824 >> 172.21.15.67:0/2594712824 conn(0x7feb3c0732d0 msgr2=0x7feb3c1511a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 -- 172.21.15.67:0/2594712824 shutdown_connections 2024-03-09T09:29:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.370+0000 7feb3b200640 1 -- 172.21.15.67:0/2594712824 wait complete. 2024-03-09T09:29:09.397 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-03-09T09:29:09.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-03-09T09:29:09.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.596+0000 7f615de00640 1 Processor -- start 2024-03-09T09:29:09.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.597+0000 7f615de00640 1 -- start start 2024-03-09T09:29:09.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.598+0000 7f615de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f61580c5e40 0x7f6158156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:09.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.598+0000 7f615de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61580cadb0 0x7f6158156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:09.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.598+0000 7f615de00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6158156a90 con 0x7f6158153300 2024-03-09T09:29:09.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.598+0000 7f615de00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6158157000 con 0x7f61580c5e40 2024-03-09T09:29:09.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.598+0000 7f615de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61580ae430 con 0x7f61580cadb0 2024-03-09T09:29:09.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.598+0000 7f6157e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f61580c5e40 0x7f6158156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:09.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.598+0000 7f6157e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f61580c5e40 0x7f6158156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46608/0 (socket says 172.21.15.67:46608) 2024-03-09T09:29:09.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.598+0000 7f6157e00640 1 -- 172.21.15.67:0/2017031518 learned_addr learned my addr 172.21.15.67:0/2017031518 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.599+0000 7f615d400640 1 --2- 172.21.15.67:0/2017031518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61580cadb0 0x7f6158156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.599+0000 7f6157e00640 1 -- 172.21.15.67:0/2017031518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61580cadb0 msgr2=0x7f6158156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.599+0000 7f6157e00640 1 --2- 172.21.15.67:0/2017031518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61580cadb0 0x7f6158156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.599+0000 7f6157e00640 1 -- 172.21.15.67:0/2017031518 >> v1:172.21.15.134:6789/0 conn(0x7f6158153300 legacy=0x7f61580b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.599+0000 7f6157e00640 1 -- 172.21.15.67:0/2017031518 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61580ae6f0 con 0x7f61580c5e40 2024-03-09T09:29:09.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.600+0000 7f6157e00640 1 --2- 172.21.15.67:0/2017031518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61580c5e40 0x7f6158156010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6144009870 tx=0x7f614402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fddee88477372b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:09.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.601+0000 7f6157400640 1 -- 172.21.15.67:0/2017031518 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144034070 con 0x7f61580c5e40 2024-03-09T09:29:09.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.601+0000 7f6157400640 1 -- 172.21.15.67:0/2017031518 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6144030840 con 0x7f61580c5e40 2024-03-09T09:29:09.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.601+0000 7f6157400640 1 -- 172.21.15.67:0/2017031518 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144030c10 con 0x7f61580c5e40 2024-03-09T09:29:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.601+0000 7f615de00640 1 -- 172.21.15.67:0/2017031518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61580c5e40 msgr2=0x7f6158156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.601+0000 7f615de00640 1 --2- 172.21.15.67:0/2017031518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61580c5e40 0x7f6158156010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6144009870 tx=0x7f614402b980 comp rx=0 tx=0).stop 2024-03-09T09:29:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.601+0000 7f615de00640 1 -- 172.21.15.67:0/2017031518 shutdown_connections 2024-03-09T09:29:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.601+0000 7f615de00640 1 --2- 172.21.15.67:0/2017031518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61580cadb0 0x7f6158156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.601+0000 7f615de00640 1 --2- 172.21.15.67:0/2017031518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61580c5e40 0x7f6158156010 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.601+0000 7f615de00640 1 -- 172.21.15.67:0/2017031518 >> 172.21.15.67:0/2017031518 conn(0x7f61580732d0 msgr2=0x7f61580ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:09.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.602+0000 7f615de00640 1 -- 172.21.15.67:0/2017031518 shutdown_connections 2024-03-09T09:29:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.602+0000 7f615de00640 1 -- 172.21.15.67:0/2017031518 wait complete. 2024-03-09T09:29:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.604+0000 7f615de00640 1 Processor -- start 2024-03-09T09:29:09.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.604+0000 7f615de00640 1 -- start start 2024-03-09T09:29:09.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.606+0000 7f615de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61580c5e40 0x7f615814b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:09.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.606+0000 7f615de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61580cadb0 0x7f6158153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:09.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.606+0000 7f615de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6158153300 0x7f6158154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:09.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.606+0000 7f615de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6158159e90 con 0x7f61580c5e40 2024-03-09T09:29:09.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.606+0000 7f615de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6158159bb0 con 0x7f6158153300 2024-03-09T09:29:09.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.606+0000 7f615de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61580c59e0 con 0x7f61580cadb0 2024-03-09T09:29:09.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.606+0000 7f615ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61580c5e40 0x7f615814b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.606+0000 7f615ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61580c5e40 0x7f615814b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46618/0 (socket says 172.21.15.67:46618) 2024-03-09T09:29:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.606+0000 7f615ca00640 1 -- 172.21.15.67:0/1555354593 learned_addr learned my addr 172.21.15.67:0/1555354593 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.607+0000 7f6157e00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61580cadb0 0x7f6158153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.607+0000 7f615d400640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6158153300 0x7f6158154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.607+0000 7f615ca00640 1 -- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61580cadb0 msgr2=0x7f6158153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.607+0000 7f615ca00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61580cadb0 0x7f6158153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.607+0000 7f615ca00640 1 -- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6158153300 msgr2=0x7f6158154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.607+0000 7f615ca00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6158153300 0x7f6158154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.607+0000 7f615ca00640 1 -- 172.21.15.67:0/1555354593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615814bd80 con 0x7f61580c5e40 2024-03-09T09:29:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.607+0000 7f615ca00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61580c5e40 0x7f615814b410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f614c00c400 tx=0x7f614c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.607+0000 7f6154c00640 1 -- 172.21.15.67:0/1555354593 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c0052d0 con 0x7f61580c5e40 2024-03-09T09:29:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.608+0000 7f6154c00640 1 -- 172.21.15.67:0/1555354593 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c005490 con 0x7f61580c5e40 2024-03-09T09:29:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.608+0000 7f6154c00640 1 -- 172.21.15.67:0/1555354593 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c007260 con 0x7f61580c5e40 2024-03-09T09:29:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.608+0000 7f615de00640 1 -- 172.21.15.67:0/1555354593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61581ebf30 con 0x7f61580c5e40 2024-03-09T09:29:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.608+0000 7f615de00640 1 -- 172.21.15.67:0/1555354593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615814e4e0 con 0x7f61580c5e40 2024-03-09T09:29:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.609+0000 7f615de00640 1 -- 172.21.15.67:0/1555354593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615804f3c0 con 0x7f61580c5e40 2024-03-09T09:29:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.610+0000 7f6154c00640 1 -- 172.21.15.67:0/1555354593 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f614c007400 con 0x7f61580c5e40 2024-03-09T09:29:09.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.611+0000 7f6154c00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f613806b4d0 0x7f613806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:09.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.611+0000 7f6154c00640 1 -- 172.21.15.67:0/1555354593 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f614c089aa0 con 0x7f61580c5e40 2024-03-09T09:29:09.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.612+0000 7f6157e00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f613806b4d0 0x7f613806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:09.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.613+0000 7f6157e00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f613806b4d0 0x7f613806d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6144036000 tx=0x7f614403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:09.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.616+0000 7f6154c00640 1 -- 172.21.15.67:0/1555354593 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f614c0585d0 con 0x7f61580c5e40 2024-03-09T09:29:09.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.900+0000 7f615de00640 1 -- 172.21.15.67:0/1555354593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f61580cab90 con 0x7f61580c5e40 2024-03-09T09:29:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.901+0000 7f6154c00640 1 -- 172.21.15.67:0/1555354593 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1566 (secure 0 0 0) 0x7f614c057ed0 con 0x7f61580c5e40 2024-03-09T09:29:09.909 INFO:teuthology.orchestra.run.smithi067.stdout:{"election_epoch":288,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","modified":"2024-03-09T08:51:10.767075Z","created":"2024-03-09T08:51:10.767075Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3301","nonce":0},{"type":"v1","addr":"172.21.15.134:6790","nonce":0}]},"addr":"172.21.15.134:6790/0","public_addr":"172.21.15.134:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-03-09T09:29:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.903+0000 7f615de00640 1 -- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f613806b4d0 msgr2=0x7f613806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.903+0000 7f615de00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f613806b4d0 0x7f613806d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6144036000 tx=0x7f614403b040 comp rx=0 tx=0).stop 2024-03-09T09:29:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.903+0000 7f615de00640 1 -- 172.21.15.67:0/1555354593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61580c5e40 msgr2=0x7f615814b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.903+0000 7f615de00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61580c5e40 0x7f615814b410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f614c00c400 tx=0x7f614c00c8c0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.904+0000 7f615de00640 1 -- 172.21.15.67:0/1555354593 shutdown_connections 2024-03-09T09:29:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.904+0000 7f615de00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f613806b4d0 0x7f613806d990 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.904+0000 7f615de00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6158153300 0x7f6158154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.904+0000 7f615de00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61580cadb0 0x7f6158153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.904+0000 7f615de00640 1 --2- 172.21.15.67:0/1555354593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61580c5e40 0x7f615814b410 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:09.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.904+0000 7f615de00640 1 -- 172.21.15.67:0/1555354593 >> 172.21.15.67:0/1555354593 conn(0x7f61580732d0 msgr2=0x7f61581513b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:09.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.904+0000 7f615de00640 1 -- 172.21.15.67:0/1555354593 shutdown_connections 2024-03-09T09:29:09.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:09.904+0000 7f615de00640 1 -- 172.21.15.67:0/1555354593 wait complete. 2024-03-09T09:29:09.930 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-03-09T09:29:09.930 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-09T09:29:10.387 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-09T09:29:10.387 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T09:29:10.388 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-09T09:29:10.452 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-09T09:29:10.453 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-09T09:29:10.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T09:29:10.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.754+0000 7f910a600640 1 Processor -- start 2024-03-09T09:29:10.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.754+0000 7f910a600640 1 -- start start 2024-03-09T09:29:10.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.755+0000 7f910a600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f91040c5e40 0x7f9104156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:10.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.756+0000 7f910a600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9104157000 con 0x7f9104153300 2024-03-09T09:29:10.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.756+0000 7f910a600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91040ae430 con 0x7f91040cadb0 2024-03-09T09:29:10.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.756+0000 7f910a600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91040ae6f0 con 0x7f91040c5e40 2024-03-09T09:29:10.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.756+0000 7f9109200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f9104153300 0x7f91040b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55652/0 (socket says 172.21.15.67:55652) 2024-03-09T09:29:10.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.756+0000 7f9109200640 1 -- 172.21.15.67:0/1084052308 learned_addr learned my addr 172.21.15.67:0/1084052308 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:10.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.756+0000 7f9103e00640 1 --2- 172.21.15.67:0/1084052308 >> v2:172.21.15.67:3300/0 conn(0x7f91040c5e40 0x7f9104156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:10.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.756+0000 7f9103400640 1 -- 172.21.15.67:0/1084052308 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084061923 0 0) 0x7f9104157000 con 0x7f9104153300 2024-03-09T09:29:10.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.756+0000 7f9103400640 1 -- 172.21.15.67:0/1084052308 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e8003520 con 0x7f9104153300 2024-03-09T09:29:10.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.757+0000 7f9103e00640 1 -- 172.21.15.67:0/1084052308 >> v1:172.21.15.134:6789/0 conn(0x7f91040cadb0 legacy=0x7f9104156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:10.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.757+0000 7f9103e00640 1 -- 172.21.15.67:0/1084052308 >> v1:172.21.15.67:6789/0 conn(0x7f9104153300 legacy=0x7f91040b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:10.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.757+0000 7f9103400640 1 -- 172.21.15.67:0/1084052308 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2083715051 0 0) 0x7f90e8003520 con 0x7f9104153300 2024-03-09T09:29:10.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.757+0000 7f9103e00640 1 -- 172.21.15.67:0/1084052308 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9104155b10 con 0x7f91040c5e40 2024-03-09T09:29:10.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.758+0000 7f9103e00640 1 --2- 172.21.15.67:0/1084052308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91040c5e40 0x7f9104156010 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0097a0 tx=0x7f90ec02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737fc162c2e54e56 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:10.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.758+0000 7f9103400640 1 -- 172.21.15.67:0/1084052308 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec038070 con 0x7f91040c5e40 2024-03-09T09:29:10.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.758+0000 7f9103400640 1 -- 172.21.15.67:0/1084052308 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec031500 con 0x7f91040c5e40 2024-03-09T09:29:10.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.758+0000 7f9103400640 1 -- 172.21.15.67:0/1084052308 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4001670 con 0x7f91040c5e40 2024-03-09T09:29:10.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.759+0000 7f910a600640 1 -- 172.21.15.67:0/1084052308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91040c5e40 msgr2=0x7f9104156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:10.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.759+0000 7f910a600640 1 --2- 172.21.15.67:0/1084052308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91040c5e40 0x7f9104156010 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0097a0 tx=0x7f90ec02b820 comp rx=0 tx=0).stop 2024-03-09T09:29:10.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.759+0000 7f910a600640 1 -- 172.21.15.67:0/1084052308 shutdown_connections 2024-03-09T09:29:10.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.759+0000 7f910a600640 1 --2- 172.21.15.67:0/1084052308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91040c5e40 0x7f9104156010 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:10.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.759+0000 7f910a600640 1 -- 172.21.15.67:0/1084052308 >> 172.21.15.67:0/1084052308 conn(0x7f91040732d0 msgr2=0x7f91040ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:10.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.759+0000 7f910a600640 1 -- 172.21.15.67:0/1084052308 shutdown_connections 2024-03-09T09:29:10.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.759+0000 7f910a600640 1 -- 172.21.15.67:0/1084052308 wait complete. 2024-03-09T09:29:10.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.760+0000 7f910a600640 1 Processor -- start 2024-03-09T09:29:10.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.760+0000 7f910a600640 1 -- start start 2024-03-09T09:29:10.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.761+0000 7f910a600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91040c5e40 0x7f91040ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:10.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.761+0000 7f910a600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91040cadb0 0x7f91041ee440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:10.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.761+0000 7f910a600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9104153300 0x7f910414bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:10.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.761+0000 7f910a600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91040c59e0 con 0x7f9104153300 2024-03-09T09:29:10.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.761+0000 7f910a600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9104159e90 con 0x7f91040cadb0 2024-03-09T09:29:10.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.761+0000 7f910a600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91040b0710 con 0x7f91040c5e40 2024-03-09T09:29:10.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.762+0000 7f9109c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9104153300 0x7f910414bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:10.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.762+0000 7f9109c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9104153300 0x7f910414bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46650/0 (socket says 172.21.15.67:46650) 2024-03-09T09:29:10.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.762+0000 7f9109c00640 1 -- 172.21.15.67:0/1994963912 learned_addr learned my addr 172.21.15.67:0/1994963912 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:10.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.762+0000 7f9109200640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91040c5e40 0x7f91040ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:10.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.762+0000 7f9103e00640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91040cadb0 0x7f91041ee440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:10.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.762+0000 7f9109c00640 1 -- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91040c5e40 msgr2=0x7f91040ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:10.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.762+0000 7f9109c00640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91040c5e40 0x7f91040ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:10.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.762+0000 7f9109c00640 1 -- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91040cadb0 msgr2=0x7f91041ee440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:10.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.762+0000 7f9109c00640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91040cadb0 0x7f91041ee440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:10.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.762+0000 7f9109c00640 1 -- 172.21.15.67:0/1994963912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91040c6980 con 0x7f9104153300 2024-03-09T09:29:10.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.763+0000 7f9109c00640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9104153300 0x7f910414bcd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c7d0 tx=0x7f90fc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:10.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.763+0000 7f9100c00640 1 -- 172.21.15.67:0/1994963912 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc003b70 con 0x7f9104153300 2024-03-09T09:29:10.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.763+0000 7f9100c00640 1 -- 172.21.15.67:0/1994963912 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc003d30 con 0x7f9104153300 2024-03-09T09:29:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.763+0000 7f910a600640 1 -- 172.21.15.67:0/1994963912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91040c6c60 con 0x7f9104153300 2024-03-09T09:29:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.763+0000 7f9100c00640 1 -- 172.21.15.67:0/1994963912 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc00aa30 con 0x7f9104153300 2024-03-09T09:29:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.763+0000 7f910a600640 1 -- 172.21.15.67:0/1994963912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910406b3c0 con 0x7f9104153300 2024-03-09T09:29:10.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.764+0000 7f910a600640 1 -- 172.21.15.67:0/1994963912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910404f3c0 con 0x7f9104153300 2024-03-09T09:29:10.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.766+0000 7f9100c00640 1 -- 172.21.15.67:0/1994963912 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f90fc00abd0 con 0x7f9104153300 2024-03-09T09:29:10.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.767+0000 7f9100c00640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e006b4d0 0x7f90e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:10.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.767+0000 7f9100c00640 1 -- 172.21.15.67:0/1994963912 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f90fc08a300 con 0x7f9104153300 2024-03-09T09:29:10.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.767+0000 7f9109200640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e006b4d0 0x7f90e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:10.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.768+0000 7f9109200640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e006b4d0 0x7f90e006d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f90f40011e0 tx=0x7f90f40016f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:10.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.777+0000 7f9100c00640 1 -- 172.21.15.67:0/1994963912 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f90fc0587e0 con 0x7f9104153300 2024-03-09T09:29:10.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.971+0000 7f910a600640 1 -- 172.21.15.67:0/1994963912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91040ac8d0 con 0x7f9104153300 2024-03-09T09:29:10.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.975+0000 7f9100c00640 1 -- 172.21.15.67:0/1994963912 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+20649 (secure 0 0 0) 0x7f90fc05c880 con 0x7f9104153300 2024-03-09T09:29:10.984 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:29:10.984 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T09:27:40.166424+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":765,"snap_epoch":79,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":765,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":748,"length":18}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T09:29:10.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 -- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e006b4d0 msgr2=0x7f90e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:10.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e006b4d0 0x7f90e006d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f90f40011e0 tx=0x7f90f40016f0 comp rx=0 tx=0).stop 2024-03-09T09:29:10.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 -- 172.21.15.67:0/1994963912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9104153300 msgr2=0x7f910414bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:10.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9104153300 0x7f910414bcd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c7d0 tx=0x7f90fc00cc90 comp rx=0 tx=0).stop 2024-03-09T09:29:10.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 -- 172.21.15.67:0/1994963912 shutdown_connections 2024-03-09T09:29:10.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f90e006b4d0 0x7f90e006d990 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:10.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9104153300 0x7f910414bcd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:10.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91040cadb0 0x7f91041ee440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:10.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 --2- 172.21.15.67:0/1994963912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91040c5e40 0x7f91040ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:10.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 -- 172.21.15.67:0/1994963912 >> 172.21.15.67:0/1994963912 conn(0x7f91040732d0 msgr2=0x7f91041582d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:10.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 -- 172.21.15.67:0/1994963912 shutdown_connections 2024-03-09T09:29:10.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:10.978+0000 7f910a600640 1 -- 172.21.15.67:0/1994963912 wait complete. 2024-03-09T09:29:11.003 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-09T08:52:06.749641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-03-09T08:52:16.185356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-03-09T08:52:16.756829+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 765, 'snap_epoch': 79, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-03-09T08:52:27.484297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-03-09T08:52:28.060754+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 765, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-09T09:29:11.004 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T09:29:11.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.217+0000 7fccf7e00640 1 Processor -- start 2024-03-09T09:29:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.217+0000 7fccf7e00640 1 -- start start 2024-03-09T09:29:11.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.218+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccf81531e0 0x7fccf80c4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.218+0000 7fccf7e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fccf80aa860 0x7fccf80aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.219+0000 7fccf7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf81535c0 con 0x7fccf8072830 2024-03-09T09:29:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.219+0000 7fccf7e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf8073e60 con 0x7fccf80aa860 2024-03-09T09:29:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.219+0000 7fccf7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8073710 con 0x7fccf81531e0 2024-03-09T09:29:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.219+0000 7fccf6000640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fccf8072830 0x7fccf80c4dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55656/0 (socket says 172.21.15.67:55656) 2024-03-09T09:29:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.219+0000 7fccf6000640 1 -- 172.21.15.67:0/3147162087 learned_addr learned my addr 172.21.15.67:0/3147162087 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.219+0000 7fccf6a00640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccf81531e0 0x7fccf80c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.219+0000 7fccf7400640 1 --2- 172.21.15.67:0/3147162087 >> v2:172.21.15.134:3300/0 conn(0x7fccf80aa860 0x7fccf80aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.219+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798619603 0 0) 0x7fccf81535c0 con 0x7fccf8072830 2024-03-09T09:29:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.219+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccdc003520 con 0x7fccf8072830 2024-03-09T09:29:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4217948205 0 0) 0x7fccdc003520 con 0x7fccf8072830 2024-03-09T09:29:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccf81531e0 msgr2=0x7fccf80c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccf81531e0 0x7fccf80c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 >> v2:172.21.15.134:3300/0 conn(0x7fccf80aa860 msgr2=0x7fccf80aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 --2- 172.21.15.67:0/3147162087 >> v2:172.21.15.134:3300/0 conn(0x7fccf80aa860 0x7fccf80aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf8072ec0 con 0x7fccf8072830 2024-03-09T09:29:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fcce4001670 con 0x7fccf8072830 2024-03-09T09:29:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf6a00640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccf81531e0 0x7fccf80c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:29:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 >> v1:172.21.15.67:6789/0 conn(0x7fccf8072830 legacy=0x7fccf80c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccdc0039b0 0x7fccdc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccf81531e0 0x7fccdc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc004d40 0x7fccdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8073e60 con 0x7fccdc004d40 2024-03-09T09:29:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8073710 con 0x7fccdc0039b0 2024-03-09T09:29:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc00ba10 con 0x7fccf81531e0 2024-03-09T09:29:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.220+0000 7fccf7400640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc004d40 0x7fccdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.221+0000 7fccf6000640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccf81531e0 0x7fccdc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.221+0000 7fccf6a00640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccdc0039b0 0x7fccdc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.221+0000 7fccf7400640 1 -- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccf81531e0 msgr2=0x7fccdc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.221+0000 7fccf7400640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccf81531e0 0x7fccdc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.221+0000 7fccf7400640 1 -- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccdc0039b0 msgr2=0x7fccdc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.221+0000 7fccf7400640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccdc0039b0 0x7fccdc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.221+0000 7fccf7400640 1 -- 172.21.15.67:0/3147162087 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc003520 con 0x7fccdc004d40 2024-03-09T09:29:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.223+0000 7fccf7400640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc004d40 0x7fccdc0051c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009af0 tx=0x7fccdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b68d56dd1b25ae server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.223+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0039070 con 0x7fccdc004d40 2024-03-09T09:29:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.223+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf002c710 con 0x7fccdc004d40 2024-03-09T09:29:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.223+0000 7fccf5600640 1 -- 172.21.15.67:0/3147162087 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0009690 con 0x7fccdc004d40 2024-03-09T09:29:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.224+0000 7fccf7e00640 1 -- 172.21.15.67:0/3147162087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc004d40 msgr2=0x7fccdc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.224+0000 7fccf7e00640 1 --2- 172.21.15.67:0/3147162087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc004d40 0x7fccdc0051c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009af0 tx=0x7fccdc005830 comp rx=0 tx=0).stop 2024-03-09T09:29:11.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.224+0000 7fccf7400640 1 -- 172.21.15.67:0/3147162087 reap_dead start 2024-03-09T09:29:11.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.224+0000 7fccf7e00640 1 -- 172.21.15.67:0/3147162087 shutdown_connections 2024-03-09T09:29:11.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.224+0000 7fccf7e00640 1 -- 172.21.15.67:0/3147162087 >> 172.21.15.67:0/3147162087 conn(0x7fccf80673c0 msgr2=0x7fccf80c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:11.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.224+0000 7fccf7e00640 1 -- 172.21.15.67:0/3147162087 shutdown_connections 2024-03-09T09:29:11.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.224+0000 7fccf7e00640 1 -- 172.21.15.67:0/3147162087 wait complete. 2024-03-09T09:29:11.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.226+0000 7fccf7e00640 1 Processor -- start 2024-03-09T09:29:11.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.226+0000 7fccf7e00640 1 -- start start 2024-03-09T09:29:11.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccdc004d40 0x7fccf806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf814bc40 0x7fccf814c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81ee350 0x7fccf81ee7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80ab210 con 0x7fccf814bc40 2024-03-09T09:29:11.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8073b30 con 0x7fccf81ee350 2024-03-09T09:29:11.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80ab390 con 0x7fccdc004d40 2024-03-09T09:29:11.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf814bc40 0x7fccf814c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf814bc40 0x7fccf814c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46668/0 (socket says 172.21.15.67:46668) 2024-03-09T09:29:11.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf6000640 1 -- 172.21.15.67:0/1557369981 learned_addr learned my addr 172.21.15.67:0/1557369981 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:11.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf6a00640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccdc004d40 0x7fccf806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf7400640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81ee350 0x7fccf81ee7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf6000640 1 -- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccdc004d40 msgr2=0x7fccf806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf6000640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccdc004d40 0x7fccf806d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf6000640 1 -- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81ee350 msgr2=0x7fccf81ee7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf6000640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81ee350 0x7fccf81ee7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf6000640 1 -- 172.21.15.67:0/1557369981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf80ac340 con 0x7fccf814bc40 2024-03-09T09:29:11.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf6a00640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccdc004d40 0x7fccf806d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:11.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.227+0000 7fccf7400640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81ee350 0x7fccf81ee7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:11.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.228+0000 7fccf6000640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf814bc40 0x7fccf814c0c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcce4007310 tx=0x7fcce40077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:11.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.228+0000 7fcceea00640 1 -- 172.21.15.67:0/1557369981 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce401a070 con 0x7fccf814bc40 2024-03-09T09:29:11.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.228+0000 7fcceea00640 1 -- 172.21.15.67:0/1557369981 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce400f9a0 con 0x7fccf814bc40 2024-03-09T09:29:11.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.228+0000 7fcceea00640 1 -- 172.21.15.67:0/1557369981 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce400fd90 con 0x7fccf814bc40 2024-03-09T09:29:11.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.228+0000 7fccf7e00640 1 -- 172.21.15.67:0/1557369981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf80ac620 con 0x7fccf814bc40 2024-03-09T09:29:11.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.228+0000 7fccf7e00640 1 -- 172.21.15.67:0/1557369981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf81efbe0 con 0x7fccf814bc40 2024-03-09T09:29:11.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.229+0000 7fccf7e00640 1 -- 172.21.15.67:0/1557369981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf804f3c0 con 0x7fccf814bc40 2024-03-09T09:29:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.230+0000 7fcceea00640 1 -- 172.21.15.67:0/1557369981 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcce4014450 con 0x7fccf814bc40 2024-03-09T09:29:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.231+0000 7fcceea00640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fccd806b500 0x7fccd806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.231+0000 7fcceea00640 1 -- 172.21.15.67:0/1557369981 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fcce4089ea0 con 0x7fccf814bc40 2024-03-09T09:29:11.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.232+0000 7fccf6a00640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fccd806b500 0x7fccd806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.233+0000 7fccf6a00640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fccd806b500 0x7fccd806d9c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcce8007440 tx=0x7fcce800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:11.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.236+0000 7fcceea00640 1 -- 172.21.15.67:0/1557369981 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcce40589f0 con 0x7fccf814bc40 2024-03-09T09:29:11.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.439+0000 7fccf7e00640 1 -- 172.21.15.67:0/1557369981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fccf80abcf0 con 0x7fccf814bc40 2024-03-09T09:29:11.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.440+0000 7fcceea00640 1 -- 172.21.15.67:0/1557369981 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcce40582f0 con 0x7fccf814bc40 2024-03-09T09:29:11.448 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-09T09:29:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.442+0000 7fccf7e00640 1 -- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fccd806b500 msgr2=0x7fccd806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.442+0000 7fccf7e00640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fccd806b500 0x7fccd806d9c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcce8007440 tx=0x7fcce800b040 comp rx=0 tx=0).stop 2024-03-09T09:29:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.442+0000 7fccf7e00640 1 -- 172.21.15.67:0/1557369981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf814bc40 msgr2=0x7fccf814c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.442+0000 7fccf7e00640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf814bc40 0x7fccf814c0c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcce4007310 tx=0x7fcce40077d0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.442+0000 7fccf7e00640 1 -- 172.21.15.67:0/1557369981 shutdown_connections 2024-03-09T09:29:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.442+0000 7fccf7e00640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fccd806b500 0x7fccd806d9c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.442+0000 7fccf7e00640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccf81ee350 0x7fccf81ee7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.442+0000 7fccf7e00640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccf814bc40 0x7fccf814c0c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.442+0000 7fccf7e00640 1 --2- 172.21.15.67:0/1557369981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fccdc004d40 0x7fccf806d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.442+0000 7fccf7e00640 1 -- 172.21.15.67:0/1557369981 >> 172.21.15.67:0/1557369981 conn(0x7fccf80673c0 msgr2=0x7fccf80c6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:11.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.443+0000 7fccf7e00640 1 -- 172.21.15.67:0/1557369981 shutdown_connections 2024-03-09T09:29:11.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.443+0000 7fccf7e00640 1 -- 172.21.15.67:0/1557369981 wait complete. 2024-03-09T09:29:11.471 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-03-09T09:29:11.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.680+0000 7ffbc2000640 1 Processor -- start 2024-03-09T09:29:11.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.681+0000 7ffbc2000640 1 -- start start 2024-03-09T09:29:11.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.682+0000 7ffbc2000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7ffbbc1532a0 0x7ffbbc1563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.682+0000 7ffbc2000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1568e0 0x7ffbbc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.682+0000 7ffbc2000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbbc073d40 con 0x7ffbbc06da80 2024-03-09T09:29:11.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.682+0000 7ffbc2000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbbc15a270 con 0x7ffbbc1532a0 2024-03-09T09:29:11.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.682+0000 7ffbc2000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc159090 con 0x7ffbbc1568e0 2024-03-09T09:29:11.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.683+0000 7ffbbbe00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7ffbbc06da80 0x7ffbbc0ab800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55672/0 (socket says 172.21.15.67:55672) 2024-03-09T09:29:11.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.683+0000 7ffbbbe00640 1 -- 172.21.15.67:0/2584119178 learned_addr learned my addr 172.21.15.67:0/2584119178 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:11.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.683+0000 7ffbbb400640 1 --2- 172.21.15.67:0/2584119178 >> v2:172.21.15.134:3300/0 conn(0x7ffbbc1532a0 0x7ffbbc1563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.683+0000 7ffbbaa00640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1568e0 0x7ffbbc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.683+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460892571 0 0) 0x7ffbbc073d40 con 0x7ffbbc06da80 2024-03-09T09:29:11.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.683+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba0003520 con 0x7ffbbc06da80 2024-03-09T09:29:11.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.683+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2964072311 0 0) 0x7ffba0003520 con 0x7ffbbc06da80 2024-03-09T09:29:11.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.683+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1568e0 msgr2=0x7ffbbc06d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:11.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.683+0000 7ffbba000640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1568e0 0x7ffbbc06d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.683+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 >> v2:172.21.15.134:3300/0 conn(0x7ffbbc1532a0 msgr2=0x7ffbbc1563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:11.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.683+0000 7ffbba000640 1 --2- 172.21.15.67:0/2584119178 >> v2:172.21.15.134:3300/0 conn(0x7ffbbc1532a0 0x7ffbbc1563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc159cb0 con 0x7ffbbc06da80 2024-03-09T09:29:11.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7ffbb40019f0 con 0x7ffbbc06da80 2024-03-09T09:29:11.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 >> v1:172.21.15.67:6789/0 conn(0x7ffbbc06da80 legacy=0x7ffbbc0ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbba000640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1568e0 0x7ffba00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbba000640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba0003f80 0x7ffba0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbba000640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0004d40 0x7ffba00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc15a270 con 0x7ffba0003f80 2024-03-09T09:29:11.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc159090 con 0x7ffba0004d40 2024-03-09T09:29:11.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba000b9c0 con 0x7ffbbc1568e0 2024-03-09T09:29:11.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbbaa00640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba0003f80 0x7ffba0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbbb400640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1568e0 0x7ffba00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbbbe00640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0004d40 0x7ffba00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbbaa00640 1 -- 172.21.15.67:0/2584119178 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1568e0 msgr2=0x7ffba00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:11.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbbaa00640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1568e0 0x7ffba00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbbaa00640 1 -- 172.21.15.67:0/2584119178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0004d40 msgr2=0x7ffba00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:11.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbbaa00640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0004d40 0x7ffba00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.684+0000 7ffbbaa00640 1 -- 172.21.15.67:0/2584119178 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba0003520 con 0x7ffba0003f80 2024-03-09T09:29:11.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.686+0000 7ffbbaa00640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba0003f80 0x7ffba0004800 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009af0 tx=0x7ffba0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2af8e9e3cbdbdeb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:11.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.686+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8039070 con 0x7ffba0003f80 2024-03-09T09:29:11.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.687+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba802c6b0 con 0x7ffba0003f80 2024-03-09T09:29:11.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.687+0000 7ffbba000640 1 -- 172.21.15.67:0/2584119178 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8009690 con 0x7ffba0003f80 2024-03-09T09:29:11.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.687+0000 7ffbc2000640 1 -- 172.21.15.67:0/2584119178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba0003f80 msgr2=0x7ffba0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.687+0000 7ffbc2000640 1 --2- 172.21.15.67:0/2584119178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffba0003f80 0x7ffba0004800 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009af0 tx=0x7ffba0005830 comp rx=0 tx=0).stop 2024-03-09T09:29:11.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.687+0000 7ffbbbe00640 1 -- 172.21.15.67:0/2584119178 reap_dead start 2024-03-09T09:29:11.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.687+0000 7ffbc2000640 1 -- 172.21.15.67:0/2584119178 shutdown_connections 2024-03-09T09:29:11.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.687+0000 7ffbc2000640 1 -- 172.21.15.67:0/2584119178 >> 172.21.15.67:0/2584119178 conn(0x7ffbbc06b040 msgr2=0x7ffbbc06b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:11.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.687+0000 7ffbc2000640 1 -- 172.21.15.67:0/2584119178 shutdown_connections 2024-03-09T09:29:11.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.687+0000 7ffbc2000640 1 -- 172.21.15.67:0/2584119178 wait complete. 2024-03-09T09:29:11.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.689+0000 7ffbc2000640 1 Processor -- start 2024-03-09T09:29:11.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.689+0000 7ffbc2000640 1 -- start start 2024-03-09T09:29:11.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.690+0000 7ffbc2000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0003f80 0x7ffbbc1ef420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.690+0000 7ffbc2000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbbc1df310 0x7ffbbc1ef960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.690+0000 7ffbc2000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1e0ff0 0x7ffbbc1e1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.690+0000 7ffbc2000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0c0f30 con 0x7ffbbc1df310 2024-03-09T09:29:11.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.690+0000 7ffbc2000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc153680 con 0x7ffba0003f80 2024-03-09T09:29:11.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.690+0000 7ffbc2000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0c10b0 con 0x7ffbbc1e0ff0 2024-03-09T09:29:11.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.690+0000 7ffbbaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbbc1df310 0x7ffbbc1ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.690+0000 7ffbbaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbbc1df310 0x7ffbbc1ef960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46680/0 (socket says 172.21.15.67:46680) 2024-03-09T09:29:11.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.690+0000 7ffbbaa00640 1 -- 172.21.15.67:0/1179875805 learned_addr learned my addr 172.21.15.67:0/1179875805 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:11.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.691+0000 7ffbbb400640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0003f80 0x7ffbbc1ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.691+0000 7ffbbaa00640 1 -- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1e0ff0 msgr2=0x7ffbbc1e1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.691+0000 7ffbbbe00640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1e0ff0 0x7ffbbc1e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.691+0000 7ffbbaa00640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1e0ff0 0x7ffbbc1e1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.691+0000 7ffbbaa00640 1 -- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0003f80 msgr2=0x7ffbbc1ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.691+0000 7ffbbaa00640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0003f80 0x7ffbbc1ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.691+0000 7ffbbaa00640 1 -- 172.21.15.67:0/1179875805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc1efea0 con 0x7ffbbc1df310 2024-03-09T09:29:11.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.691+0000 7ffbbaa00640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbbc1df310 0x7ffbbc1ef960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffba8037040 tx=0x7ffba8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:11.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.691+0000 7ffbb3400640 1 -- 172.21.15.67:0/1179875805 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8042020 con 0x7ffbbc1df310 2024-03-09T09:29:11.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.692+0000 7ffbb3400640 1 -- 172.21.15.67:0/1179875805 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba803a040 con 0x7ffbbc1df310 2024-03-09T09:29:11.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.692+0000 7ffbc2000640 1 -- 172.21.15.67:0/1179875805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc1df7a0 con 0x7ffbbc1df310 2024-03-09T09:29:11.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.692+0000 7ffbb3400640 1 -- 172.21.15.67:0/1179875805 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8039070 con 0x7ffbbc1df310 2024-03-09T09:29:11.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.692+0000 7ffbc2000640 1 -- 172.21.15.67:0/1179875805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc1e07f0 con 0x7ffbbc1df310 2024-03-09T09:29:11.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.693+0000 7ffbc2000640 1 -- 172.21.15.67:0/1179875805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc0c80f0 con 0x7ffbbc1df310 2024-03-09T09:29:11.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.694+0000 7ffbb3400640 1 -- 172.21.15.67:0/1179875805 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ffba8045020 con 0x7ffbbc1df310 2024-03-09T09:29:11.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.695+0000 7ffbb3400640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:11.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.695+0000 7ffbb3400640 1 -- 172.21.15.67:0/1179875805 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7ffba80ab760 con 0x7ffbbc1df310 2024-03-09T09:29:11.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.696+0000 7ffbbb400640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:11.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.697+0000 7ffbbb400640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009a90 tx=0x7ffbac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:11.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.699+0000 7ffbb3400640 1 -- 172.21.15.67:0/1179875805 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffba804c020 con 0x7ffbbc1df310 2024-03-09T09:29:11.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.902+0000 7ffbc2000640 1 -- 172.21.15.67:0/1179875805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffbbc075a00 con 0x7ffbbc1df310 2024-03-09T09:29:11.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.903+0000 7ffbb3400640 1 -- 172.21.15.67:0/1179875805 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v89) v1 ==== 99+0+11 (secure 0 0 0) 0x7ffba807a180 con 0x7ffbbc1df310 2024-03-09T09:29:11.911 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T09:29:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.905+0000 7ffbc2000640 1 -- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb9c06b4d0 msgr2=0x7ffb9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.905+0000 7ffbc2000640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009a90 tx=0x7ffbac008040 comp rx=0 tx=0).stop 2024-03-09T09:29:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.905+0000 7ffbc2000640 1 -- 172.21.15.67:0/1179875805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbbc1df310 msgr2=0x7ffbbc1ef960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.905+0000 7ffbc2000640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbbc1df310 0x7ffbbc1ef960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffba8037040 tx=0x7ffba8004730 comp rx=0 tx=0).stop 2024-03-09T09:29:11.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.905+0000 7ffbc2000640 1 -- 172.21.15.67:0/1179875805 shutdown_connections 2024-03-09T09:29:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.905+0000 7ffbc2000640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.905+0000 7ffbc2000640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffbbc1e0ff0 0x7ffbbc1e1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.905+0000 7ffbc2000640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbbc1df310 0x7ffbbc1ef960 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.905+0000 7ffbc2000640 1 --2- 172.21.15.67:0/1179875805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffba0003f80 0x7ffbbc1ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.905+0000 7ffbc2000640 1 -- 172.21.15.67:0/1179875805 >> 172.21.15.67:0/1179875805 conn(0x7ffbbc06b040 msgr2=0x7ffbbc1570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.905+0000 7ffbc2000640 1 -- 172.21.15.67:0/1179875805 shutdown_connections 2024-03-09T09:29:11.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:11.906+0000 7ffbc2000640 1 -- 172.21.15.67:0/1179875805 wait complete. 2024-03-09T09:29:11.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-03-09T09:29:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.139+0000 7fc26e400640 1 Processor -- start 2024-03-09T09:29:12.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.139+0000 7fc26e400640 1 -- start start 2024-03-09T09:29:12.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.141+0000 7fc26e400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc2680c5f10 0x7fc2680c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.141+0000 7fc26e400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc2680c9490 0x7fc2680cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.141+0000 7fc26e400640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2680c9a30 con 0x7fc26806b040 2024-03-09T09:29:12.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.141+0000 7fc26e400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2680c9d10 con 0x7fc2680c9490 2024-03-09T09:29:12.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.141+0000 7fc26e400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2680ca340 con 0x7fc2680c5f10 2024-03-09T09:29:12.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.141+0000 7fc26da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc2680c9490 0x7fc2680cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.141+0000 7fc26da00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fc2680c9490 0x7fc2680cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46684/0 (socket says 172.21.15.67:46684) 2024-03-09T09:29:12.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.141+0000 7fc26da00640 1 -- 172.21.15.67:0/3761573263 learned_addr learned my addr 172.21.15.67:0/3761573263 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:12.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.142+0000 7fc26da00640 1 -- 172.21.15.67:0/3761573263 >> v2:172.21.15.134:3300/0 conn(0x7fc2680c5f10 msgr2=0x7fc2680c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:12.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.141+0000 7fc26d000640 1 --2- 172.21.15.67:0/3761573263 >> v2:172.21.15.134:3300/0 conn(0x7fc2680c5f10 0x7fc2680c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.142+0000 7fc26da00640 1 --2- 172.21.15.67:0/3761573263 >> v2:172.21.15.134:3300/0 conn(0x7fc2680c5f10 0x7fc2680c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.142+0000 7fc26da00640 1 -- 172.21.15.67:0/3761573263 >> v1:172.21.15.134:6789/0 conn(0x7fc26806b040 legacy=0x7fc2681567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:12.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.142+0000 7fc26da00640 1 -- 172.21.15.67:0/3761573263 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2680ca620 con 0x7fc2680c9490 2024-03-09T09:29:12.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.142+0000 7fc26d000640 1 --2- 172.21.15.67:0/3761573263 >> v2:172.21.15.134:3300/0 conn(0x7fc2680c5f10 0x7fc2680c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:12.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.143+0000 7fc26da00640 1 --2- 172.21.15.67:0/3761573263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2680c9490 0x7fc2680cab40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc26808d2e0 tx=0x7fc26002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a734841634e2d48 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.143+0000 7fc267400640 1 -- 172.21.15.67:0/3761573263 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26002c9f0 con 0x7fc2680c9490 2024-03-09T09:29:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.143+0000 7fc267400640 1 -- 172.21.15.67:0/3761573263 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26002cb90 con 0x7fc2680c9490 2024-03-09T09:29:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.143+0000 7fc267400640 1 -- 172.21.15.67:0/3761573263 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2600343f0 con 0x7fc2680c9490 2024-03-09T09:29:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.144+0000 7fc26e400640 1 -- 172.21.15.67:0/3761573263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2680c9490 msgr2=0x7fc2680cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.144+0000 7fc26e400640 1 --2- 172.21.15.67:0/3761573263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2680c9490 0x7fc2680cab40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc26808d2e0 tx=0x7fc26002be50 comp rx=0 tx=0).stop 2024-03-09T09:29:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.144+0000 7fc26e400640 1 -- 172.21.15.67:0/3761573263 shutdown_connections 2024-03-09T09:29:12.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.144+0000 7fc26e400640 1 --2- 172.21.15.67:0/3761573263 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc2680c9490 0x7fc2680cab40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.144+0000 7fc26e400640 1 --2- 172.21.15.67:0/3761573263 >> v2:172.21.15.134:3300/0 conn(0x7fc2680c5f10 0x7fc2680c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.144+0000 7fc26e400640 1 -- 172.21.15.67:0/3761573263 >> 172.21.15.67:0/3761573263 conn(0x7fc2680cb590 msgr2=0x7fc2680cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:12.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.144+0000 7fc26e400640 1 -- 172.21.15.67:0/3761573263 shutdown_connections 2024-03-09T09:29:12.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.144+0000 7fc26e400640 1 -- 172.21.15.67:0/3761573263 wait complete. 2024-03-09T09:29:12.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.147+0000 7fc26e400640 1 Processor -- start 2024-03-09T09:29:12.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.147+0000 7fc26e400640 1 -- start start 2024-03-09T09:29:12.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.148+0000 7fc26e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc26806b040 0x7fc26814a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26e400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c5f10 0x7fc268148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26e400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2680c9490 0x7fc268146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26e400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2681537a0 con 0x7fc26806b040 2024-03-09T09:29:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26e400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc268153620 con 0x7fc2680c9490 2024-03-09T09:29:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26e400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2680cb080 con 0x7fc2680c5f10 2024-03-09T09:29:12.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc26806b040 0x7fc26814a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc26806b040 0x7fc26814a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46686/0 (socket says 172.21.15.67:46686) 2024-03-09T09:29:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26d000640 1 -- 172.21.15.67:0/653897792 learned_addr learned my addr 172.21.15.67:0/653897792 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26da00640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2680c9490 0x7fc268146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc267e00640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c5f10 0x7fc268148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26d000640 1 -- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c5f10 msgr2=0x7fc268148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:12.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26d000640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c5f10 0x7fc268148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26d000640 1 -- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2680c9490 msgr2=0x7fc268146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26d000640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2680c9490 0x7fc268146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.149+0000 7fc26d000640 1 -- 172.21.15.67:0/653897792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2681542e0 con 0x7fc26806b040 2024-03-09T09:29:12.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.150+0000 7fc26d000640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc26806b040 0x7fc26814a340 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00c7f0 tx=0x7fc25c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:12.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.150+0000 7fc264c00640 1 -- 172.21.15.67:0/653897792 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c015070 con 0x7fc26806b040 2024-03-09T09:29:12.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.150+0000 7fc264c00640 1 -- 172.21.15.67:0/653897792 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c007a30 con 0x7fc26806b040 2024-03-09T09:29:12.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.150+0000 7fc264c00640 1 -- 172.21.15.67:0/653897792 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c007e00 con 0x7fc26806b040 2024-03-09T09:29:12.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.150+0000 7fc26e400640 1 -- 172.21.15.67:0/653897792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2681558e0 con 0x7fc26806b040 2024-03-09T09:29:12.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.150+0000 7fc26e400640 1 -- 172.21.15.67:0/653897792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc268154860 con 0x7fc26806b040 2024-03-09T09:29:12.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.151+0000 7fc26e400640 1 -- 172.21.15.67:0/653897792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26804f3c0 con 0x7fc26806b040 2024-03-09T09:29:12.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.153+0000 7fc264c00640 1 -- 172.21.15.67:0/653897792 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc25c018070 con 0x7fc26806b040 2024-03-09T09:29:12.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.153+0000 7fc264c00640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc24806b4d0 0x7fc24806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.154+0000 7fc264c00640 1 -- 172.21.15.67:0/653897792 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fc25c08a540 con 0x7fc26806b040 2024-03-09T09:29:12.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.154+0000 7fc267e00640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc24806b4d0 0x7fc24806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.155+0000 7fc267e00640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc24806b4d0 0x7fc24806d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc2681478d0 tx=0x7fc254009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:12.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.158+0000 7fc264c00640 1 -- 172.21.15.67:0/653897792 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc25c058ff0 con 0x7fc26806b040 2024-03-09T09:29:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.357+0000 7fc26e400640 1 -- 172.21.15.67:0/653897792 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7fc2680cc260 con 0x7fc26806b040 2024-03-09T09:29:12.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.358+0000 7fc264c00640 1 -- 172.21.15.67:0/653897792 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v89) v1 ==== 95+0+11 (secure 0 0 0) 0x7fc25c0588f0 con 0x7fc26806b040 2024-03-09T09:29:12.366 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T09:29:12.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 -- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc24806b4d0 msgr2=0x7fc24806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:12.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc24806b4d0 0x7fc24806d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc2681478d0 tx=0x7fc254009040 comp rx=0 tx=0).stop 2024-03-09T09:29:12.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 -- 172.21.15.67:0/653897792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc26806b040 msgr2=0x7fc26814a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:12.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc26806b040 0x7fc26814a340 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00c7f0 tx=0x7fc25c00cbf0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 -- 172.21.15.67:0/653897792 shutdown_connections 2024-03-09T09:29:12.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc24806b4d0 0x7fc24806d990 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2680c9490 0x7fc268146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2680c5f10 0x7fc268148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 --2- 172.21.15.67:0/653897792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc26806b040 0x7fc26814a340 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 -- 172.21.15.67:0/653897792 >> 172.21.15.67:0/653897792 conn(0x7fc2680cb590 msgr2=0x7fc268151850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:12.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 -- 172.21.15.67:0/653897792 shutdown_connections 2024-03-09T09:29:12.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.360+0000 7fc26e400640 1 -- 172.21.15.67:0/653897792 wait complete. 2024-03-09T09:29:12.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-03-09T09:29:12.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.600+0000 7fb4af400640 1 Processor -- start 2024-03-09T09:29:12.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.600+0000 7fb4af400640 1 -- start start 2024-03-09T09:29:12.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.601+0000 7fb4af400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a80c84f0 0x7fb4a8073a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.601+0000 7fb4af400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb4a80c7500 0x7fb4a80c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.601+0000 7fb4af400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4a80a6af0 0x7fb4a80a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.601+0000 7fb4af400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a81568c0 con 0x7fb4a80a6af0 2024-03-09T09:29:12.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.601+0000 7fb4af400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a80c2870 con 0x7fb4a80c7500 2024-03-09T09:29:12.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.601+0000 7fb4af400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80c2590 con 0x7fb4a80c84f0 2024-03-09T09:29:12.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.602+0000 7fb4ad400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4a80a6af0 0x7fb4a80a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.602+0000 7fb4ad400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4a80a6af0 0x7fb4a80a6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46694/0 (socket says 172.21.15.67:46694) 2024-03-09T09:29:12.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.602+0000 7fb4ad400640 1 -- 172.21.15.67:0/547219501 learned_addr learned my addr 172.21.15.67:0/547219501 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:12.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.602+0000 7fb49fe00640 1 --2- 172.21.15.67:0/547219501 >> v2:172.21.15.134:3300/0 conn(0x7fb4a80c7500 0x7fb4a80c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.602+0000 7fb4aca00640 1 --2- 172.21.15.67:0/547219501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a80c84f0 0x7fb4a8073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.602+0000 7fb4ad400640 1 -- 172.21.15.67:0/547219501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a80c84f0 msgr2=0x7fb4a8073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:12.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.602+0000 7fb4ad400640 1 --2- 172.21.15.67:0/547219501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a80c84f0 0x7fb4a8073a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.602+0000 7fb4ad400640 1 -- 172.21.15.67:0/547219501 >> v2:172.21.15.134:3300/0 conn(0x7fb4a80c7500 msgr2=0x7fb4a80c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:12.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.602+0000 7fb4ad400640 1 --2- 172.21.15.67:0/547219501 >> v2:172.21.15.134:3300/0 conn(0x7fb4a80c7500 0x7fb4a80c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.603+0000 7fb4ad400640 1 -- 172.21.15.67:0/547219501 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a806ed70 con 0x7fb4a80a6af0 2024-03-09T09:29:12.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.604+0000 7fb4ad400640 1 --2- 172.21.15.67:0/547219501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4a80a6af0 0x7fb4a80a6f60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb4a806c110 tx=0x7fb4a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1056a638101b0699 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:12.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.604+0000 7fb4a7e00640 1 -- 172.21.15.67:0/547219501 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a002ca70 con 0x7fb4a80a6af0 2024-03-09T09:29:12.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.604+0000 7fb4a7e00640 1 -- 172.21.15.67:0/547219501 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a002cc10 con 0x7fb4a80a6af0 2024-03-09T09:29:12.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.604+0000 7fb4a7e00640 1 -- 172.21.15.67:0/547219501 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00343f0 con 0x7fb4a80a6af0 2024-03-09T09:29:12.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.605+0000 7fb4af400640 1 -- 172.21.15.67:0/547219501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4a80a6af0 msgr2=0x7fb4a80a6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:12.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.605+0000 7fb4af400640 1 --2- 172.21.15.67:0/547219501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4a80a6af0 0x7fb4a80a6f60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb4a806c110 tx=0x7fb4a002bed0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.605+0000 7fb4af400640 1 -- 172.21.15.67:0/547219501 shutdown_connections 2024-03-09T09:29:12.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.605+0000 7fb4af400640 1 --2- 172.21.15.67:0/547219501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4a80a6af0 0x7fb4a80a6f60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.605+0000 7fb4af400640 1 --2- 172.21.15.67:0/547219501 >> v2:172.21.15.134:3300/0 conn(0x7fb4a80c7500 0x7fb4a80c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.605+0000 7fb4af400640 1 --2- 172.21.15.67:0/547219501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a80c84f0 0x7fb4a8073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.605+0000 7fb4af400640 1 -- 172.21.15.67:0/547219501 >> 172.21.15.67:0/547219501 conn(0x7fb4a80a9fd0 msgr2=0x7fb4a80c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:12.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.605+0000 7fb4af400640 1 -- 172.21.15.67:0/547219501 shutdown_connections 2024-03-09T09:29:12.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.605+0000 7fb4af400640 1 -- 172.21.15.67:0/547219501 wait complete. 2024-03-09T09:29:12.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.607+0000 7fb4af400640 1 Processor -- start 2024-03-09T09:29:12.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.607+0000 7fb4af400640 1 -- start start 2024-03-09T09:29:12.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb4af400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a80a6af0 0x7fb4a81e8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb4af400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4a80c7500 0x7fb4a8154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb4af400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a80c84f0 0x7fb4a81548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb4af400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a8156d50 con 0x7fb4a80c7500 2024-03-09T09:29:12.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb4af400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80c4390 con 0x7fb4a80c84f0 2024-03-09T09:29:12.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb4af400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80c17d0 con 0x7fb4a80a6af0 2024-03-09T09:29:12.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb49fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4a80c7500 0x7fb4a8154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb49fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4a80c7500 0x7fb4a8154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46706/0 (socket says 172.21.15.67:46706) 2024-03-09T09:29:12.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb49fe00640 1 -- 172.21.15.67:0/3979930734 learned_addr learned my addr 172.21.15.67:0/3979930734 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:12.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb4ad400640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a80c84f0 0x7fb4a81548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb4aca00640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a80a6af0 0x7fb4a81e8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb49fe00640 1 -- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a80a6af0 msgr2=0x7fb4a81e8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:12.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb49fe00640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a80a6af0 0x7fb4a81e8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb49fe00640 1 -- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a80c84f0 msgr2=0x7fb4a81548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:12.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb49fe00640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a80c84f0 0x7fb4a81548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.608+0000 7fb49fe00640 1 -- 172.21.15.67:0/3979930734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a80a8c30 con 0x7fb4a80c7500 2024-03-09T09:29:12.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.609+0000 7fb49fe00640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4a80c7500 0x7fb4a8154390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb49800d800 tx=0x7fb49800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:12.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.609+0000 7fb4a5600640 1 -- 172.21.15.67:0/3979930734 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498018020 con 0x7fb4a80c7500 2024-03-09T09:29:12.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.609+0000 7fb4a5600640 1 -- 172.21.15.67:0/3979930734 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4980096a0 con 0x7fb4a80c7500 2024-03-09T09:29:12.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.609+0000 7fb4a5600640 1 -- 172.21.15.67:0/3979930734 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498009a70 con 0x7fb4a80c7500 2024-03-09T09:29:12.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.609+0000 7fb4af400640 1 -- 172.21.15.67:0/3979930734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a81e9010 con 0x7fb4a80c7500 2024-03-09T09:29:12.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.609+0000 7fb4af400640 1 -- 172.21.15.67:0/3979930734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a81560d0 con 0x7fb4a80c7500 2024-03-09T09:29:12.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.611+0000 7fb4af400640 1 -- 172.21.15.67:0/3979930734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a8075a00 con 0x7fb4a80c7500 2024-03-09T09:29:12.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.611+0000 7fb4a5600640 1 -- 172.21.15.67:0/3979930734 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb49801b020 con 0x7fb4a80c7500 2024-03-09T09:29:12.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.612+0000 7fb4a5600640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb48406b520 0x7fb48406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:12.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.612+0000 7fb4a5600640 1 -- 172.21.15.67:0/3979930734 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fb49808aa70 con 0x7fb4a80c7500 2024-03-09T09:29:12.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.613+0000 7fb4aca00640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb48406b520 0x7fb48406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:12.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.614+0000 7fb4aca00640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb48406b520 0x7fb48406d9e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb490006fd0 tx=0x7fb490008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:12.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.617+0000 7fb4a5600640 1 -- 172.21.15.67:0/3979930734 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb498022020 con 0x7fb4a80c7500 2024-03-09T09:29:12.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.820+0000 7fb4af400640 1 -- 172.21.15.67:0/3979930734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4a80c1ae0 con 0x7fb4a80c7500 2024-03-09T09:29:12.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.821+0000 7fb4a5600640 1 -- 172.21.15.67:0/3979930734 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v89) v1 ==== 99+0+11 (secure 0 0 0) 0x7fb498058f50 con 0x7fb4a80c7500 2024-03-09T09:29:12.829 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T09:29:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 -- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb48406b520 msgr2=0x7fb48406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb48406b520 0x7fb48406d9e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb490006fd0 tx=0x7fb490008040 comp rx=0 tx=0).stop 2024-03-09T09:29:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 -- 172.21.15.67:0/3979930734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4a80c7500 msgr2=0x7fb4a8154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4a80c7500 0x7fb4a8154390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb49800d800 tx=0x7fb49800dcc0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 -- 172.21.15.67:0/3979930734 shutdown_connections 2024-03-09T09:29:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb48406b520 0x7fb48406d9e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a80c84f0 0x7fb4a81548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4a80c7500 0x7fb4a8154390 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 --2- 172.21.15.67:0/3979930734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a80a6af0 0x7fb4a81e8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 -- 172.21.15.67:0/3979930734 >> 172.21.15.67:0/3979930734 conn(0x7fb4a80a9fd0 msgr2=0x7fb4a80c7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 -- 172.21.15.67:0/3979930734 shutdown_connections 2024-03-09T09:29:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:12.824+0000 7fb4af400640 1 -- 172.21.15.67:0/3979930734 wait complete. 2024-03-09T09:29:12.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-03-09T09:29:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.058+0000 7fa811600640 1 Processor -- start 2024-03-09T09:29:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.058+0000 7fa811600640 1 -- start start 2024-03-09T09:29:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.060+0000 7fa811600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa80c0c3010 0x7fa80c0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.060+0000 7fa811600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa80c156980 0x7fa80c14fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.060+0000 7fa811600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa80c153950 con 0x7fa80c150180 2024-03-09T09:29:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.060+0000 7fa811600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80c1505f0 con 0x7fa80c156980 2024-03-09T09:29:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.060+0000 7fa811600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c152b90 con 0x7fa80c0c3010 2024-03-09T09:29:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.061+0000 7fa80b400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa80c156980 0x7fa80c14fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.061+0000 7fa80be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa80c0c3010 0x7fa80c0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.061+0000 7fa80b400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa80c156980 0x7fa80c14fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:60308/0 (socket says 172.21.15.67:60308) 2024-03-09T09:29:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.061+0000 7fa80b400640 1 -- 172.21.15.67:0/3927584985 learned_addr learned my addr 172.21.15.67:0/3927584985 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.062+0000 7fa80b400640 1 -- 172.21.15.67:0/3927584985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa80c0c3010 msgr2=0x7fa80c0ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:13.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.062+0000 7fa80b400640 1 --2- 172.21.15.67:0/3927584985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa80c0c3010 0x7fa80c0ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.062+0000 7fa80b400640 1 -- 172.21.15.67:0/3927584985 >> v1:172.21.15.134:6789/0 conn(0x7fa80c150180 legacy=0x7fa80c06d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.062+0000 7fa80b400640 1 -- 172.21.15.67:0/3927584985 --> v2:172.21.15.134:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c152e70 con 0x7fa80c156980 2024-03-09T09:29:13.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.062+0000 7fa80be00640 1 --2- 172.21.15.67:0/3927584985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa80c0c3010 0x7fa80c0ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-09T09:29:13.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.063+0000 7fa80b400640 1 --2- 172.21.15.67:0/3927584985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80c156980 0x7fa80c14fc40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800bb10 tx=0x7fa7f802e030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7636c55832d5a72 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:13.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.063+0000 7fa80aa00640 1 -- 172.21.15.67:0/3927584985 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f803b070 con 0x7fa80c156980 2024-03-09T09:29:13.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.063+0000 7fa80aa00640 1 -- 172.21.15.67:0/3927584985 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f802ede0 con 0x7fa80c156980 2024-03-09T09:29:13.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.063+0000 7fa80aa00640 1 -- 172.21.15.67:0/3927584985 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8036660 con 0x7fa80c156980 2024-03-09T09:29:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.064+0000 7fa811600640 1 -- 172.21.15.67:0/3927584985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80c156980 msgr2=0x7fa80c14fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.064+0000 7fa811600640 1 --2- 172.21.15.67:0/3927584985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80c156980 0x7fa80c14fc40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800bb10 tx=0x7fa7f802e030 comp rx=0 tx=0).stop 2024-03-09T09:29:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.064+0000 7fa811600640 1 -- 172.21.15.67:0/3927584985 shutdown_connections 2024-03-09T09:29:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.064+0000 7fa811600640 1 --2- 172.21.15.67:0/3927584985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80c156980 0x7fa80c14fc40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.064+0000 7fa811600640 1 --2- 172.21.15.67:0/3927584985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa80c0c3010 0x7fa80c0ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.064+0000 7fa811600640 1 -- 172.21.15.67:0/3927584985 >> 172.21.15.67:0/3927584985 conn(0x7fa80c06b8a0 msgr2=0x7fa80c0c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.064+0000 7fa811600640 1 -- 172.21.15.67:0/3927584985 shutdown_connections 2024-03-09T09:29:13.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.064+0000 7fa811600640 1 -- 172.21.15.67:0/3927584985 wait complete. 2024-03-09T09:29:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.066+0000 7fa811600640 1 Processor -- start 2024-03-09T09:29:13.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.066+0000 7fa811600640 1 -- start start 2024-03-09T09:29:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.067+0000 7fa811600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80c0c3010 0x7fa80c1ebfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.067+0000 7fa811600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c150180 0x7fa80c0c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.067+0000 7fa811600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa80c156980 0x7fa80c1521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.067+0000 7fa811600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c0ac9a0 con 0x7fa80c150180 2024-03-09T09:29:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.067+0000 7fa811600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c153430 con 0x7fa80c0c3010 2024-03-09T09:29:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.067+0000 7fa811600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c0acb20 con 0x7fa80c156980 2024-03-09T09:29:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.067+0000 7fa80b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c150180 0x7fa80c0c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.068+0000 7fa80b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c150180 0x7fa80c0c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46720/0 (socket says 172.21.15.67:46720) 2024-03-09T09:29:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.068+0000 7fa80b400640 1 -- 172.21.15.67:0/3806379931 learned_addr learned my addr 172.21.15.67:0/3806379931 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:13.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.068+0000 7fa80be00640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80c0c3010 0x7fa80c1ebfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.068+0000 7fa810c00640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa80c156980 0x7fa80c1521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.068+0000 7fa80b400640 1 -- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa80c156980 msgr2=0x7fa80c1521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.068+0000 7fa80b400640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa80c156980 0x7fa80c1521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.068+0000 7fa80b400640 1 -- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80c0c3010 msgr2=0x7fa80c1ebfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.068+0000 7fa80b400640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80c0c3010 0x7fa80c1ebfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.068+0000 7fa80b400640 1 -- 172.21.15.67:0/3806379931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c0c6620 con 0x7fa80c150180 2024-03-09T09:29:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.068+0000 7fa80b400640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c150180 0x7fa80c0c5ed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa7f802e070 tx=0x7fa7f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:13.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.069+0000 7fa803e00640 1 -- 172.21.15.67:0/3806379931 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f803b030 con 0x7fa80c150180 2024-03-09T09:29:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.069+0000 7fa803e00640 1 -- 172.21.15.67:0/3806379931 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f803d070 con 0x7fa80c150180 2024-03-09T09:29:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.069+0000 7fa803e00640 1 -- 172.21.15.67:0/3806379931 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8034550 con 0x7fa80c150180 2024-03-09T09:29:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.069+0000 7fa811600640 1 -- 172.21.15.67:0/3806379931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80c0c6900 con 0x7fa80c150180 2024-03-09T09:29:13.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.069+0000 7fa811600640 1 -- 172.21.15.67:0/3806379931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c152700 con 0x7fa80c150180 2024-03-09T09:29:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.070+0000 7fa811600640 1 -- 172.21.15.67:0/3806379931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80c04f3c0 con 0x7fa80c150180 2024-03-09T09:29:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.071+0000 7fa803e00640 1 -- 172.21.15.67:0/3806379931 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa7f8048060 con 0x7fa80c150180 2024-03-09T09:29:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.072+0000 7fa803e00640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7ec06b500 0x7fa7ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.073+0000 7fa803e00640 1 -- 172.21.15.67:0/3806379931 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fa7f80abbf0 con 0x7fa80c150180 2024-03-09T09:29:13.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.073+0000 7fa80be00640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7ec06b500 0x7fa7ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.074+0000 7fa80be00640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7ec06b500 0x7fa7ec06d9c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009560 tx=0x7fa7fc009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:13.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.077+0000 7fa803e00640 1 -- 172.21.15.67:0/3806379931 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa7f807a6c0 con 0x7fa80c150180 2024-03-09T09:29:13.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.291+0000 7fa811600640 1 -- 172.21.15.67:0/3806379931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7fa80c0ac1b0 con 0x7fa80c150180 2024-03-09T09:29:13.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.292+0000 7fa803e00640 1 -- 172.21.15.67:0/3806379931 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v89) v1 ==== 95+0+11 (secure 0 0 0) 0x7fa7f8079fc0 con 0x7fa80c150180 2024-03-09T09:29:13.300 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T09:29:13.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.294+0000 7fa811600640 1 -- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7ec06b500 msgr2=0x7fa7ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.294+0000 7fa811600640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7ec06b500 0x7fa7ec06d9c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009560 tx=0x7fa7fc009190 comp rx=0 tx=0).stop 2024-03-09T09:29:13.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.294+0000 7fa811600640 1 -- 172.21.15.67:0/3806379931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c150180 msgr2=0x7fa80c0c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.294+0000 7fa811600640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c150180 0x7fa80c0c5ed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa7f802e070 tx=0x7fa7f8004730 comp rx=0 tx=0).stop 2024-03-09T09:29:13.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.295+0000 7fa811600640 1 -- 172.21.15.67:0/3806379931 shutdown_connections 2024-03-09T09:29:13.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.295+0000 7fa811600640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa7ec06b500 0x7fa7ec06d9c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.295+0000 7fa811600640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa80c156980 0x7fa80c1521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.295+0000 7fa811600640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c150180 0x7fa80c0c5ed0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.295+0000 7fa811600640 1 --2- 172.21.15.67:0/3806379931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa80c0c3010 0x7fa80c1ebfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.295+0000 7fa811600640 1 -- 172.21.15.67:0/3806379931 >> 172.21.15.67:0/3806379931 conn(0x7fa80c06b8a0 msgr2=0x7fa80c0c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:13.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.295+0000 7fa811600640 1 -- 172.21.15.67:0/3806379931 shutdown_connections 2024-03-09T09:29:13.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.295+0000 7fa811600640 1 -- 172.21.15.67:0/3806379931 wait complete. 2024-03-09T09:29:13.321 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-09T09:29:13.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.531+0000 7f1b0b000640 1 Processor -- start 2024-03-09T09:29:13.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.532+0000 7f1b0b000640 1 -- start start 2024-03-09T09:29:13.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.533+0000 7f1b0b000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b040734a0 0x7f1b040aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.533+0000 7f1b0b000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1b0406d830 0x7f1b0406dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.533+0000 7f1b0b000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b04152e00 con 0x7f1b040730c0 2024-03-09T09:29:13.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.533+0000 7f1b0b000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b040aa960 con 0x7f1b0406d830 2024-03-09T09:29:13.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.533+0000 7f1b0b000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b041530e0 con 0x7f1b040734a0 2024-03-09T09:29:13.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.533+0000 7f1b03e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1b040730c0 0x7f1b040c2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55688/0 (socket says 172.21.15.67:55688) 2024-03-09T09:29:13.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.533+0000 7f1b03e00640 1 -- 172.21.15.67:0/500169427 learned_addr learned my addr 172.21.15.67:0/500169427 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:13.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.534+0000 7f1b03400640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b040734a0 0x7f1b040aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.534+0000 7f1b09000640 1 --2- 172.21.15.67:0/500169427 >> v2:172.21.15.134:3300/0 conn(0x7f1b0406d830 0x7f1b0406dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.534+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245317745 0 0) 0x7f1b04152e00 con 0x7f1b040730c0 2024-03-09T09:29:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.534+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ae8005730 con 0x7f1b040730c0 2024-03-09T09:29:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.534+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4099841424 0 0) 0x7f1ae8005730 con 0x7f1b040730c0 2024-03-09T09:29:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.534+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b040734a0 msgr2=0x7f1b040aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.534+0000 7f1b02a00640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b040734a0 0x7f1b040aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.534+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 >> v2:172.21.15.134:3300/0 conn(0x7f1b0406d830 msgr2=0x7f1b0406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.534+0000 7f1b02a00640 1 --2- 172.21.15.67:0/500169427 >> v2:172.21.15.134:3300/0 conn(0x7f1b0406d830 0x7f1b0406dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.534+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b041533c0 con 0x7f1b040730c0 2024-03-09T09:29:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f1af4001670 con 0x7f1b040730c0 2024-03-09T09:29:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 >> v1:172.21.15.67:6789/0 conn(0x7f1b040730c0 legacy=0x7f1b040c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b02a00640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 0x7f1ae8005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b02a00640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ae8006a10 0x7f1ae8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b02a00640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b040734a0 0x7f1ae80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040aa960 con 0x7f1ae8005bc0 2024-03-09T09:29:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b041530e0 con 0x7f1ae8006a10 2024-03-09T09:29:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae800dbd0 con 0x7f1b040734a0 2024-03-09T09:29:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b03e00640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 0x7f1ae8005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b03400640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ae8006a10 0x7f1ae8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b09000640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b040734a0 0x7f1ae80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.535+0000 7f1b03e00640 1 -- 172.21.15.67:0/500169427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b040734a0 msgr2=0x7f1ae80073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.536+0000 7f1b03e00640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b040734a0 0x7f1ae80073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.536+0000 7f1b03e00640 1 -- 172.21.15.67:0/500169427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ae8006a10 msgr2=0x7f1ae8006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:13.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.536+0000 7f1b03e00640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ae8006a10 0x7f1ae8006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.536+0000 7f1b03e00640 1 -- 172.21.15.67:0/500169427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae8005730 con 0x7f1ae8005bc0 2024-03-09T09:29:13.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.537+0000 7f1b03e00640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 0x7f1ae8005fe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1af400c210 tx=0x7f1ae8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a5dd578a3786d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:13.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.538+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4007d50 con 0x7f1ae8005bc0 2024-03-09T09:29:13.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.538+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af4030070 con 0x7f1ae8005bc0 2024-03-09T09:29:13.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.538+0000 7f1b02a00640 1 -- 172.21.15.67:0/500169427 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af402cd20 con 0x7f1ae8005bc0 2024-03-09T09:29:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.538+0000 7f1b0b000640 1 -- 172.21.15.67:0/500169427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 msgr2=0x7f1ae8005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.538+0000 7f1b0b000640 1 --2- 172.21.15.67:0/500169427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 0x7f1ae8005fe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1af400c210 tx=0x7f1ae8007a40 comp rx=0 tx=0).stop 2024-03-09T09:29:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.538+0000 7f1b09000640 1 -- 172.21.15.67:0/500169427 reap_dead start 2024-03-09T09:29:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.539+0000 7f1b0b000640 1 -- 172.21.15.67:0/500169427 shutdown_connections 2024-03-09T09:29:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.539+0000 7f1b0b000640 1 -- 172.21.15.67:0/500169427 >> 172.21.15.67:0/500169427 conn(0x7f1b040c7370 msgr2=0x7f1b040c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.539+0000 7f1b0b000640 1 -- 172.21.15.67:0/500169427 shutdown_connections 2024-03-09T09:29:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.539+0000 7f1b0b000640 1 -- 172.21.15.67:0/500169427 wait complete. 2024-03-09T09:29:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.540+0000 7f1b0b000640 1 Processor -- start 2024-03-09T09:29:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.540+0000 7f1b0b000640 1 -- start start 2024-03-09T09:29:13.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.541+0000 7f1b0b000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 0x7f1b041ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.541+0000 7f1b0b000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b04152680 0x7f1b040aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.541+0000 7f1b0b000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b04156aa0 0x7f1b04153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.541+0000 7f1b0b000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040c3590 con 0x7f1ae8005bc0 2024-03-09T09:29:13.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.541+0000 7f1b0b000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b04153980 con 0x7f1b04156aa0 2024-03-09T09:29:13.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.541+0000 7f1b0b000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040c3710 con 0x7f1b04152680 2024-03-09T09:29:13.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.542+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 0x7f1b041ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.542+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 0x7f1b041ebe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46742/0 (socket says 172.21.15.67:46742) 2024-03-09T09:29:13.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.542+0000 7f1b03e00640 1 -- 172.21.15.67:0/4283541255 learned_addr learned my addr 172.21.15.67:0/4283541255 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:13.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.542+0000 7f1b03400640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b04152680 0x7f1b040aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.542+0000 7f1b03e00640 1 -- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b04152680 msgr2=0x7f1b040aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.542+0000 7f1b03e00640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b04152680 0x7f1b040aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.542+0000 7f1b09000640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b04156aa0 0x7f1b04153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.542+0000 7f1b03e00640 1 -- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b04156aa0 msgr2=0x7f1b04153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.542+0000 7f1b03e00640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b04156aa0 0x7f1b04153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.542+0000 7f1b03e00640 1 -- 172.21.15.67:0/4283541255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b041eb3a0 con 0x7f1ae8005bc0 2024-03-09T09:29:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.543+0000 7f1b03e00640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 0x7f1b041ebe20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1af400c330 tx=0x7f1b04152fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.543+0000 7f1afbe00640 1 -- 172.21.15.67:0/4283541255 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4001cb0 con 0x7f1ae8005bc0 2024-03-09T09:29:13.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.543+0000 7f1afbe00640 1 -- 172.21.15.67:0/4283541255 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af4004bf0 con 0x7f1ae8005bc0 2024-03-09T09:29:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.543+0000 7f1afbe00640 1 -- 172.21.15.67:0/4283541255 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4007070 con 0x7f1ae8005bc0 2024-03-09T09:29:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.543+0000 7f1b0b000640 1 -- 172.21.15.67:0/4283541255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b04147570 con 0x7f1ae8005bc0 2024-03-09T09:29:13.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.543+0000 7f1b0b000640 1 -- 172.21.15.67:0/4283541255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b040ab290 con 0x7f1ae8005bc0 2024-03-09T09:29:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.544+0000 7f1b0b000640 1 -- 172.21.15.67:0/4283541255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b04075a00 con 0x7f1ae8005bc0 2024-03-09T09:29:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.546+0000 7f1afbe00640 1 -- 172.21.15.67:0/4283541255 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1af4007210 con 0x7f1ae8005bc0 2024-03-09T09:29:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.547+0000 7f1afbe00640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ae406b4d0 0x7f1ae406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.547+0000 7f1afbe00640 1 -- 172.21.15.67:0/4283541255 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f1af40ab590 con 0x7f1ae8005bc0 2024-03-09T09:29:13.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.548+0000 7f1b03400640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ae406b4d0 0x7f1ae406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.548+0000 7f1b03400640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ae406b4d0 0x7f1ae406d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1b04152a60 tx=0x7f1af0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:13.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.554+0000 7f1afbe00640 1 -- 172.21.15.67:0/4283541255 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1b041eb3a0 con 0x7f1ae8005bc0 2024-03-09T09:29:13.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.768+0000 7f1b0b000640 1 -- 172.21.15.67:0/4283541255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1b040723e0 con 0x7f1ae8005bc0 2024-03-09T09:29:13.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.769+0000 7f1afbe00640 1 -- 172.21.15.67:0/4283541255 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1af407a0e0 con 0x7f1ae8005bc0 2024-03-09T09:29:13.776 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-03-09T09:29:13.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.771+0000 7f1b0b000640 1 -- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ae406b4d0 msgr2=0x7f1ae406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.771+0000 7f1b0b000640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ae406b4d0 0x7f1ae406d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1b04152a60 tx=0x7f1af0008040 comp rx=0 tx=0).stop 2024-03-09T09:29:13.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.771+0000 7f1b0b000640 1 -- 172.21.15.67:0/4283541255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 msgr2=0x7f1b041ebe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:13.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.771+0000 7f1b0b000640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 0x7f1b041ebe20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1af400c330 tx=0x7f1b04152fd0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.771+0000 7f1b0b000640 1 -- 172.21.15.67:0/4283541255 shutdown_connections 2024-03-09T09:29:13.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.771+0000 7f1b0b000640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1ae406b4d0 0x7f1ae406d990 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.772+0000 7f1b0b000640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b04156aa0 0x7f1b04153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.772+0000 7f1b0b000640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1b04152680 0x7f1b040aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.772+0000 7f1b0b000640 1 --2- 172.21.15.67:0/4283541255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ae8005bc0 0x7f1b041ebe20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:13.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.772+0000 7f1b0b000640 1 -- 172.21.15.67:0/4283541255 >> 172.21.15.67:0/4283541255 conn(0x7f1b040c7370 msgr2=0x7f1b040c1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:13.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.772+0000 7f1b0b000640 1 -- 172.21.15.67:0/4283541255 shutdown_connections 2024-03-09T09:29:13.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:13.772+0000 7f1b0b000640 1 -- 172.21.15.67:0/4283541255 wait complete. 2024-03-09T09:29:13.797 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-09T09:29:13.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T09:29:13.797 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-09T09:29:14.138 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T09:29:14.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T09:29:14.139 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-09T09:29:14.181 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T09:29:14.181 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-09T09:29:14.219 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-03-09T09:29:14.219 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-09T09:29:14.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T09:29:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.518+0000 7ff117200640 1 Processor -- start 2024-03-09T09:29:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.518+0000 7ff117200640 1 -- start start 2024-03-09T09:29:14.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff117200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff11806ffc0 0x7ff118159110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:14.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff117200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff11806fa40 0x7ff118159650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:14.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff117200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1181535f0 con 0x7ff118153210 2024-03-09T09:29:14.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff117200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1180c4520 con 0x7ff11806ffc0 2024-03-09T09:29:14.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff117200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1180c4dc0 con 0x7ff11806fa40 2024-03-09T09:29:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff10fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff11806ffc0 0x7ff118159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff10fe00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7ff11806ffc0 0x7ff118159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46754/0 (socket says 172.21.15.67:46754) 2024-03-09T09:29:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff10fe00640 1 -- 172.21.15.67:0/1028204105 learned_addr learned my addr 172.21.15.67:0/1028204105 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff116800640 1 --2- 172.21.15.67:0/1028204105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff11806fa40 0x7ff118159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff10fe00640 1 -- 172.21.15.67:0/1028204105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff11806fa40 msgr2=0x7ff118159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff10fe00640 1 --2- 172.21.15.67:0/1028204105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff11806fa40 0x7ff118159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.520+0000 7ff10fe00640 1 -- 172.21.15.67:0/1028204105 >> v1:172.21.15.134:6789/0 conn(0x7ff118153210 legacy=0x7ff1180c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.521+0000 7ff10fe00640 1 -- 172.21.15.67:0/1028204105 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11806ddb0 con 0x7ff11806ffc0 2024-03-09T09:29:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.522+0000 7ff10fe00640 1 --2- 172.21.15.67:0/1028204105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff11806ffc0 0x7ff118159110 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff100009870 tx=0x7ff10002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db98235a542bcd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.522+0000 7ff115400640 1 -- 172.21.15.67:0/1028204105 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000317c0 con 0x7ff11806ffc0 2024-03-09T09:29:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.522+0000 7ff115400640 1 -- 172.21.15.67:0/1028204105 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff100037070 con 0x7ff11806ffc0 2024-03-09T09:29:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.522+0000 7ff115400640 1 -- 172.21.15.67:0/1028204105 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100031b80 con 0x7ff11806ffc0 2024-03-09T09:29:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.523+0000 7ff117200640 1 -- 172.21.15.67:0/1028204105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff11806ffc0 msgr2=0x7ff118159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.523+0000 7ff117200640 1 --2- 172.21.15.67:0/1028204105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff11806ffc0 0x7ff118159110 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff100009870 tx=0x7ff10002b860 comp rx=0 tx=0).stop 2024-03-09T09:29:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.523+0000 7ff117200640 1 -- 172.21.15.67:0/1028204105 shutdown_connections 2024-03-09T09:29:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.523+0000 7ff117200640 1 --2- 172.21.15.67:0/1028204105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff11806fa40 0x7ff118159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.523+0000 7ff117200640 1 --2- 172.21.15.67:0/1028204105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff11806ffc0 0x7ff118159110 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.523+0000 7ff117200640 1 -- 172.21.15.67:0/1028204105 >> 172.21.15.67:0/1028204105 conn(0x7ff1180c9710 msgr2=0x7ff1180c9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.523+0000 7ff117200640 1 -- 172.21.15.67:0/1028204105 shutdown_connections 2024-03-09T09:29:14.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.523+0000 7ff117200640 1 -- 172.21.15.67:0/1028204105 wait complete. 2024-03-09T09:29:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.525+0000 7ff117200640 1 Processor -- start 2024-03-09T09:29:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.525+0000 7ff117200640 1 -- start start 2024-03-09T09:29:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff117200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11806fa40 0x7ff1181581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff117200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff11806ffc0 0x7ff1181df120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff117200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff118153210 0x7ff11814c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:14.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff117200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1180c3250 con 0x7ff118153210 2024-03-09T09:29:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff117200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff118071390 con 0x7ff11806fa40 2024-03-09T09:29:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff117200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1180c33d0 con 0x7ff11806ffc0 2024-03-09T09:29:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff116800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff118153210 0x7ff11814c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff116800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff118153210 0x7ff11814c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46758/0 (socket says 172.21.15.67:46758) 2024-03-09T09:29:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff116800640 1 -- 172.21.15.67:0/2553421614 learned_addr learned my addr 172.21.15.67:0/2553421614 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff10fe00640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff11806ffc0 0x7ff1181df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff115e00640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11806fa40 0x7ff1181581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:14.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff116800640 1 -- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff11806ffc0 msgr2=0x7ff1181df120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff116800640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff11806ffc0 0x7ff1181df120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff116800640 1 -- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11806fa40 msgr2=0x7ff1181581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff116800640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11806fa40 0x7ff1181581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff116800640 1 -- 172.21.15.67:0/2553421614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1181e1cd0 con 0x7ff118153210 2024-03-09T09:29:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.526+0000 7ff10fe00640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff11806ffc0 0x7ff1181df120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.527+0000 7ff116800640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff118153210 0x7ff11814c9f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff110007e30 tx=0x7ff11000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.527+0000 7ff10e000640 1 -- 172.21.15.67:0/2553421614 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110014620 con 0x7ff118153210 2024-03-09T09:29:14.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.527+0000 7ff10e000640 1 -- 172.21.15.67:0/2553421614 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff110019070 con 0x7ff118153210 2024-03-09T09:29:14.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.527+0000 7ff117200640 1 -- 172.21.15.67:0/2553421614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1181e28a0 con 0x7ff118153210 2024-03-09T09:29:14.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.527+0000 7ff10e000640 1 -- 172.21.15.67:0/2553421614 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110014a00 con 0x7ff118153210 2024-03-09T09:29:14.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.527+0000 7ff117200640 1 -- 172.21.15.67:0/2553421614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1181e2250 con 0x7ff118153210 2024-03-09T09:29:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.529+0000 7ff117200640 1 -- 172.21.15.67:0/2553421614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1180c5780 con 0x7ff118153210 2024-03-09T09:29:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.529+0000 7ff10e000640 1 -- 172.21.15.67:0/2553421614 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff110014ba0 con 0x7ff118153210 2024-03-09T09:29:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.531+0000 7ff10e000640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0f406b4d0 0x7ff0f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:14.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.531+0000 7ff10e000640 1 -- 172.21.15.67:0/2553421614 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7ff11008aeb0 con 0x7ff118153210 2024-03-09T09:29:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.531+0000 7ff115e00640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0f406b4d0 0x7ff0f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.533+0000 7ff115e00640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0f406b4d0 0x7ff0f406d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff10800a730 tx=0x7ff108008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:14.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.535+0000 7ff10e000640 1 -- 172.21.15.67:0/2553421614 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff1180c5780 con 0x7ff118153210 2024-03-09T09:29:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.800+0000 7ff117200640 1 -- 172.21.15.67:0/2553421614 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff11804f3c0 con 0x7ff118153210 2024-03-09T09:29:14.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.803+0000 7ff10e000640 1 -- 172.21.15.67:0/2553421614 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+20649 (secure 0 0 0) 0x7ff110059960 con 0x7ff118153210 2024-03-09T09:29:14.811 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:29:14.811 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T09:27:40.166424+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":765,"snap_epoch":79,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":765,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":748,"length":18}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T09:29:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 -- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0f406b4d0 msgr2=0x7ff0f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0f406b4d0 0x7ff0f406d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff10800a730 tx=0x7ff108008040 comp rx=0 tx=0).stop 2024-03-09T09:29:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 -- 172.21.15.67:0/2553421614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff118153210 msgr2=0x7ff11814c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff118153210 0x7ff11814c9f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff110007e30 tx=0x7ff11000e4b0 comp rx=0 tx=0).stop 2024-03-09T09:29:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 -- 172.21.15.67:0/2553421614 shutdown_connections 2024-03-09T09:29:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7ff0f406b4d0 0x7ff0f406d990 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:14.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff118153210 0x7ff11814c9f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff11806ffc0 0x7ff1181df120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 --2- 172.21.15.67:0/2553421614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff11806fa40 0x7ff1181581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 -- 172.21.15.67:0/2553421614 >> 172.21.15.67:0/2553421614 conn(0x7ff1180c9710 msgr2=0x7ff118070610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 -- 172.21.15.67:0/2553421614 shutdown_connections 2024-03-09T09:29:14.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:14.806+0000 7ff117200640 1 -- 172.21.15.67:0/2553421614 wait complete. 2024-03-09T09:29:14.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-09T08:52:06.749641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-03-09T08:52:16.185356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-03-09T08:52:16.756829+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 765, 'snap_epoch': 79, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-03-09T08:52:27.484297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-03-09T08:52:28.060754+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 765, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-09T09:29:14.836 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-09T09:29:15.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.046+0000 7fbd97e00640 1 Processor -- start 2024-03-09T09:29:15.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.046+0000 7fbd97e00640 1 -- start start 2024-03-09T09:29:15.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.047+0000 7fbd97e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbd980c4370 0x7fbd981544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.047+0000 7fbd97e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbd98072aa0 0x7fbd98072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.047+0000 7fbd97e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9814a9c0 con 0x7fbd9814a5e0 2024-03-09T09:29:15.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.047+0000 7fbd97e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd98157010 con 0x7fbd980c4370 2024-03-09T09:29:15.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.047+0000 7fbd97e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd981572b0 con 0x7fbd98072aa0 2024-03-09T09:29:15.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.048+0000 7fbd96000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbd980c4370 0x7fbd981544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.048+0000 7fbd96000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbd980c4370 0x7fbd981544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46764/0 (socket says 172.21.15.67:46764) 2024-03-09T09:29:15.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.048+0000 7fbd96000640 1 -- 172.21.15.67:0/702685350 learned_addr learned my addr 172.21.15.67:0/702685350 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:15.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.048+0000 7fbd96000640 1 -- 172.21.15.67:0/702685350 >> v2:172.21.15.134:3300/0 conn(0x7fbd98072aa0 msgr2=0x7fbd98072ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:15.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.048+0000 7fbd96000640 1 --2- 172.21.15.67:0/702685350 >> v2:172.21.15.134:3300/0 conn(0x7fbd98072aa0 0x7fbd98072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.048+0000 7fbd96000640 1 -- 172.21.15.67:0/702685350 >> v1:172.21.15.134:6789/0 conn(0x7fbd9814a5e0 legacy=0x7fbd9806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:15.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.048+0000 7fbd96000640 1 -- 172.21.15.67:0/702685350 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd98157590 con 0x7fbd980c4370 2024-03-09T09:29:15.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.049+0000 7fbd96000640 1 --2- 172.21.15.67:0/702685350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd980c4370 0x7fbd981544f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbd980c9330 tx=0x7fbd8402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4af9bd5978412be server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:15.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.050+0000 7fbd95600640 1 -- 172.21.15.67:0/702685350 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84035020 con 0x7fbd980c4370 2024-03-09T09:29:15.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.050+0000 7fbd95600640 1 -- 172.21.15.67:0/702685350 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd84031890 con 0x7fbd980c4370 2024-03-09T09:29:15.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.050+0000 7fbd95600640 1 -- 172.21.15.67:0/702685350 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84031c60 con 0x7fbd980c4370 2024-03-09T09:29:15.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.050+0000 7fbd97e00640 1 -- 172.21.15.67:0/702685350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd980c4370 msgr2=0x7fbd981544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.050+0000 7fbd97e00640 1 --2- 172.21.15.67:0/702685350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd980c4370 0x7fbd981544f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbd980c9330 tx=0x7fbd8402b930 comp rx=0 tx=0).stop 2024-03-09T09:29:15.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.050+0000 7fbd97e00640 1 -- 172.21.15.67:0/702685350 shutdown_connections 2024-03-09T09:29:15.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.050+0000 7fbd97e00640 1 --2- 172.21.15.67:0/702685350 >> v2:172.21.15.134:3300/0 conn(0x7fbd98072aa0 0x7fbd98072ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.050+0000 7fbd97e00640 1 --2- 172.21.15.67:0/702685350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd980c4370 0x7fbd981544f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.050+0000 7fbd97e00640 1 -- 172.21.15.67:0/702685350 >> 172.21.15.67:0/702685350 conn(0x7fbd980c82a0 msgr2=0x7fbd980c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:15.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.050+0000 7fbd97e00640 1 -- 172.21.15.67:0/702685350 shutdown_connections 2024-03-09T09:29:15.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.050+0000 7fbd97e00640 1 -- 172.21.15.67:0/702685350 wait complete. 2024-03-09T09:29:15.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.052+0000 7fbd97e00640 1 Processor -- start 2024-03-09T09:29:15.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.053+0000 7fbd97e00640 1 -- start start 2024-03-09T09:29:15.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.054+0000 7fbd97e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd98072aa0 0x7fbd9806dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.054+0000 7fbd97e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd980c4370 0x7fbd9813f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.054+0000 7fbd97e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd9814a5e0 0x7fbd9813ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.054+0000 7fbd97e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd98157e30 con 0x7fbd9814a5e0 2024-03-09T09:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.054+0000 7fbd97e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd98157b50 con 0x7fbd98072aa0 2024-03-09T09:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.054+0000 7fbd97e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9806b750 con 0x7fbd980c4370 2024-03-09T09:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.054+0000 7fbd97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd9814a5e0 0x7fbd9813ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.054+0000 7fbd97400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd9814a5e0 0x7fbd9813ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46766/0 (socket says 172.21.15.67:46766) 2024-03-09T09:29:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.054+0000 7fbd97400640 1 -- 172.21.15.67:0/1621023220 learned_addr learned my addr 172.21.15.67:0/1621023220 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.054+0000 7fbd96000640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd980c4370 0x7fbd9813f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.054+0000 7fbd96a00640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd98072aa0 0x7fbd9806dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.055+0000 7fbd97400640 1 -- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd980c4370 msgr2=0x7fbd9813f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.055+0000 7fbd97400640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd980c4370 0x7fbd9813f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.055+0000 7fbd97400640 1 -- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd98072aa0 msgr2=0x7fbd9806dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.055+0000 7fbd97400640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd98072aa0 0x7fbd9806dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.055+0000 7fbd97400640 1 -- 172.21.15.67:0/1621023220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd98140440 con 0x7fbd9814a5e0 2024-03-09T09:29:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.055+0000 7fbd97400640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd9814a5e0 0x7fbd9813ff00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000b9a0 tx=0x7fbd9000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.055+0000 7fbd8ea00640 1 -- 172.21.15.67:0/1621023220 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90019070 con 0x7fbd9814a5e0 2024-03-09T09:29:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.056+0000 7fbd8ea00640 1 -- 172.21.15.67:0/1621023220 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9000cbc0 con 0x7fbd9814a5e0 2024-03-09T09:29:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.056+0000 7fbd8ea00640 1 -- 172.21.15.67:0/1621023220 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90014440 con 0x7fbd9814a5e0 2024-03-09T09:29:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.056+0000 7fbd97e00640 1 -- 172.21.15.67:0/1621023220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd981428e0 con 0x7fbd9814a5e0 2024-03-09T09:29:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.056+0000 7fbd97e00640 1 -- 172.21.15.67:0/1621023220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd98156480 con 0x7fbd9814a5e0 2024-03-09T09:29:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.057+0000 7fbd97e00640 1 -- 172.21.15.67:0/1621023220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9804f3c0 con 0x7fbd9814a5e0 2024-03-09T09:29:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.058+0000 7fbd8ea00640 1 -- 172.21.15.67:0/1621023220 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbd9001c070 con 0x7fbd9814a5e0 2024-03-09T09:29:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.059+0000 7fbd8ea00640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbd7806b4d0 0x7fbd7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.059+0000 7fbd8ea00640 1 -- 172.21.15.67:0/1621023220 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fbd9008a350 con 0x7fbd9814a5e0 2024-03-09T09:29:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.059+0000 7fbd96a00640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbd7806b4d0 0x7fbd7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.061+0000 7fbd96a00640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbd7806b4d0 0x7fbd7806d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbd88006fd0 tx=0x7fbd88008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:15.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.063+0000 7fbd8ea00640 1 -- 172.21.15.67:0/1621023220 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbd90058e00 con 0x7fbd9814a5e0 2024-03-09T09:29:15.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.262+0000 7fbd97e00640 1 -- 172.21.15.67:0/1621023220 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd980c40e0 con 0x7fbd9814a5e0 2024-03-09T09:29:15.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.263+0000 7fbd8ea00640 1 -- 172.21.15.67:0/1621023220 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd90058700 con 0x7fbd9814a5e0 2024-03-09T09:29:15.270 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-09T09:29:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 -- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbd7806b4d0 msgr2=0x7fbd7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbd7806b4d0 0x7fbd7806d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbd88006fd0 tx=0x7fbd88008040 comp rx=0 tx=0).stop 2024-03-09T09:29:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 -- 172.21.15.67:0/1621023220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd9814a5e0 msgr2=0x7fbd9813ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd9814a5e0 0x7fbd9813ff00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000b9a0 tx=0x7fbd9000be60 comp rx=0 tx=0).stop 2024-03-09T09:29:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 -- 172.21.15.67:0/1621023220 shutdown_connections 2024-03-09T09:29:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbd7806b4d0 0x7fbd7806d990 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd9814a5e0 0x7fbd9813ff00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd980c4370 0x7fbd9813f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 --2- 172.21.15.67:0/1621023220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd98072aa0 0x7fbd9806dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 -- 172.21.15.67:0/1621023220 >> 172.21.15.67:0/1621023220 conn(0x7fbd980c82a0 msgr2=0x7fbd98142d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:15.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 -- 172.21.15.67:0/1621023220 shutdown_connections 2024-03-09T09:29:15.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.266+0000 7fbd97e00640 1 -- 172.21.15.67:0/1621023220 wait complete. 2024-03-09T09:29:15.301 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-03-09T09:29:15.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.509+0000 7f0a2aa00640 1 Processor -- start 2024-03-09T09:29:15.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.509+0000 7f0a2aa00640 1 -- start start 2024-03-09T09:29:15.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.511+0000 7f0a2aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0a240c5f10 0x7f0a240c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.511+0000 7f0a2aa00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a240c9a30 con 0x7f0a2406b040 2024-03-09T09:29:15.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.511+0000 7f0a2aa00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a240c9d10 con 0x7f0a240c9490 2024-03-09T09:29:15.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.511+0000 7f0a2aa00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a240ca340 con 0x7f0a240c5f10 2024-03-09T09:29:15.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.511+0000 7f0a29600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0a240c5f10 0x7f0a240c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.511+0000 7f0a28c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f0a2406b040 0x7f0a241567d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55690/0 (socket says 172.21.15.67:55690) 2024-03-09T09:29:15.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.511+0000 7f0a28c00640 1 -- 172.21.15.67:0/3253220173 learned_addr learned my addr 172.21.15.67:0/3253220173 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:15.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.512+0000 7f0a29600640 1 -- 172.21.15.67:0/3253220173 >> v1:172.21.15.134:6789/0 conn(0x7f0a240c9490 legacy=0x7f0a240cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:15.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.512+0000 7f0a29600640 1 -- 172.21.15.67:0/3253220173 >> v1:172.21.15.67:6789/0 conn(0x7f0a2406b040 legacy=0x7f0a241567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.512+0000 7f0a23e00640 1 -- 172.21.15.67:0/3253220173 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414106236 0 0) 0x7f0a240c9a30 con 0x7f0a2406b040 2024-03-09T09:29:15.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.512+0000 7f0a29600640 1 -- 172.21.15.67:0/3253220173 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240ca620 con 0x7f0a240c5f10 2024-03-09T09:29:15.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.513+0000 7f0a29600640 1 --2- 172.21.15.67:0/3253220173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a240c5f10 0x7f0a240c62f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009d90 tx=0x7f0a1802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c5fdaa520af057 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:15.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.513+0000 7f0a23e00640 1 -- 172.21.15.67:0/3253220173 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18039070 con 0x7f0a240c5f10 2024-03-09T09:29:15.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.514+0000 7f0a23e00640 1 -- 172.21.15.67:0/3253220173 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1802cc60 con 0x7f0a240c5f10 2024-03-09T09:29:15.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.514+0000 7f0a23e00640 1 -- 172.21.15.67:0/3253220173 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18034660 con 0x7f0a240c5f10 2024-03-09T09:29:15.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.514+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3253220173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a240c5f10 msgr2=0x7f0a240c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.514+0000 7f0a2aa00640 1 --2- 172.21.15.67:0/3253220173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a240c5f10 0x7f0a240c62f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009d90 tx=0x7f0a1802beb0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.514+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3253220173 shutdown_connections 2024-03-09T09:29:15.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.514+0000 7f0a2aa00640 1 --2- 172.21.15.67:0/3253220173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a240c5f10 0x7f0a240c62f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.514+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3253220173 >> 172.21.15.67:0/3253220173 conn(0x7f0a240cb590 msgr2=0x7f0a240cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:15.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.514+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3253220173 shutdown_connections 2024-03-09T09:29:15.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.514+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3253220173 wait complete. 2024-03-09T09:29:15.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.516+0000 7f0a2aa00640 1 Processor -- start 2024-03-09T09:29:15.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.516+0000 7f0a2aa00640 1 -- start start 2024-03-09T09:29:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.517+0000 7f0a2aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a2406b040 0x7f0a24157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.517+0000 7f0a2aa00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a240c5f10 0x7f0a24157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.517+0000 7f0a2aa00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a240c9490 0x7f0a2414f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.517+0000 7f0a2aa00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240cb3d0 con 0x7f0a2406b040 2024-03-09T09:29:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.517+0000 7f0a2aa00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240cb250 con 0x7f0a240c5f10 2024-03-09T09:29:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.517+0000 7f0a2aa00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a24153620 con 0x7f0a240c9490 2024-03-09T09:29:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.517+0000 7f0a29600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a2406b040 0x7f0a24157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.517+0000 7f0a29600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a2406b040 0x7f0a24157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46798/0 (socket says 172.21.15.67:46798) 2024-03-09T09:29:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.517+0000 7f0a29600640 1 -- 172.21.15.67:0/3654608035 learned_addr learned my addr 172.21.15.67:0/3654608035 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.518+0000 7f0a29600640 1 -- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a240c9490 msgr2=0x7f0a2414f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.518+0000 7f0a29600640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a240c9490 0x7f0a2414f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.518+0000 7f0a29600640 1 -- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a240c5f10 msgr2=0x7f0a24157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.518+0000 7f0a29600640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a240c5f10 0x7f0a24157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.518+0000 7f0a29600640 1 -- 172.21.15.67:0/3654608035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a241ec560 con 0x7f0a2406b040 2024-03-09T09:29:15.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.518+0000 7f0a29600640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a2406b040 0x7f0a24157e70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0a18037040 tx=0x7f0a18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:15.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.518+0000 7f0a21600640 1 -- 172.21.15.67:0/3654608035 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18039050 con 0x7f0a2406b040 2024-03-09T09:29:15.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.518+0000 7f0a21600640 1 -- 172.21.15.67:0/3654608035 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a18045070 con 0x7f0a2406b040 2024-03-09T09:29:15.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.518+0000 7f0a21600640 1 -- 172.21.15.67:0/3654608035 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1803ea50 con 0x7f0a2406b040 2024-03-09T09:29:15.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.518+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3654608035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a241ec840 con 0x7f0a2406b040 2024-03-09T09:29:15.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.519+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3654608035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a24159ea0 con 0x7f0a2406b040 2024-03-09T09:29:15.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.520+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3654608035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2404f3c0 con 0x7f0a2406b040 2024-03-09T09:29:15.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.521+0000 7f0a21600640 1 -- 172.21.15.67:0/3654608035 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0a1802cb20 con 0x7f0a2406b040 2024-03-09T09:29:15.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.522+0000 7f0a21600640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a0406b500 0x7f0a0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.522+0000 7f0a21600640 1 -- 172.21.15.67:0/3654608035 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f0a180ace30 con 0x7f0a2406b040 2024-03-09T09:29:15.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.522+0000 7f0a28c00640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a0406b500 0x7f0a0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.523+0000 7f0a28c00640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a0406b500 0x7f0a0406d9c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000c00 tx=0x7f0a100079c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:15.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.527+0000 7f0a21600640 1 -- 172.21.15.67:0/3654608035 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0a1804e020 con 0x7f0a2406b040 2024-03-09T09:29:15.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.726+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3654608035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a240cc260 con 0x7f0a2406b040 2024-03-09T09:29:15.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.727+0000 7f0a21600640 1 -- 172.21.15.67:0/3654608035 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v89) v1 ==== 99+0+11 (secure 0 0 0) 0x7f0a1807b850 con 0x7f0a2406b040 2024-03-09T09:29:15.734 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T09:29:15.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a0406b500 msgr2=0x7f0a0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a0406b500 0x7f0a0406d9c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000c00 tx=0x7f0a100079c0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3654608035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a2406b040 msgr2=0x7f0a24157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a2406b040 0x7f0a24157e70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0a18037040 tx=0x7f0a18004a40 comp rx=0 tx=0).stop 2024-03-09T09:29:15.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3654608035 shutdown_connections 2024-03-09T09:29:15.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0a0406b500 0x7f0a0406d9c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a240c9490 0x7f0a2414f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a240c5f10 0x7f0a24157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 --2- 172.21.15.67:0/3654608035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a2406b040 0x7f0a24157e70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3654608035 >> 172.21.15.67:0/3654608035 conn(0x7f0a240cb590 msgr2=0x7f0a2406c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:15.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3654608035 shutdown_connections 2024-03-09T09:29:15.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.729+0000 7f0a2aa00640 1 -- 172.21.15.67:0/3654608035 wait complete. 2024-03-09T09:29:15.759 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-03-09T09:29:15.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.970+0000 7f1df2600640 1 Processor -- start 2024-03-09T09:29:15.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.970+0000 7f1df2600640 1 -- start start 2024-03-09T09:29:15.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.971+0000 7f1df2600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1dec0c5f10 0x7f1dec0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.971+0000 7f1df2600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1dec06b040 0x7f1dec1567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.971+0000 7f1df2600640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dec156d10 con 0x7f1dec0c9490 2024-03-09T09:29:15.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.971+0000 7f1df2600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dec0c9a30 con 0x7f1dec0c5f10 2024-03-09T09:29:15.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.971+0000 7f1df2600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dec0c9d10 con 0x7f1dec06b040 2024-03-09T09:29:15.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.972+0000 7f1df1200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1dec0c5f10 0x7f1dec0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.972+0000 7f1df1c00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f1dec0c9490 0x7f1dec0cab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55700/0 (socket says 172.21.15.67:55700) 2024-03-09T09:29:15.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.972+0000 7f1df1c00640 1 -- 172.21.15.67:0/3382749506 learned_addr learned my addr 172.21.15.67:0/3382749506 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:15.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.972+0000 7f1debe00640 1 --2- 172.21.15.67:0/3382749506 >> v2:172.21.15.134:3300/0 conn(0x7f1dec06b040 0x7f1dec1567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.972+0000 7f1deb400640 1 -- 172.21.15.67:0/3382749506 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113950955 0 0) 0x7f1de4001490 con 0x7f1dec0c9490 2024-03-09T09:29:15.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.973+0000 7f1deb400640 1 -- 172.21.15.67:0/3382749506 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd0003520 con 0x7f1dec0c9490 2024-03-09T09:29:15.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.973+0000 7f1df1200640 1 -- 172.21.15.67:0/3382749506 >> v2:172.21.15.134:3300/0 conn(0x7f1dec06b040 msgr2=0x7f1dec1567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:15.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.973+0000 7f1df1200640 1 --2- 172.21.15.67:0/3382749506 >> v2:172.21.15.134:3300/0 conn(0x7f1dec06b040 0x7f1dec1567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.973+0000 7f1df1200640 1 -- 172.21.15.67:0/3382749506 >> v1:172.21.15.67:6789/0 conn(0x7f1dec0c9490 legacy=0x7f1dec0cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.973+0000 7f1df1200640 1 -- 172.21.15.67:0/3382749506 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec0ca340 con 0x7f1dec0c5f10 2024-03-09T09:29:15.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.973+0000 7f1debe00640 1 --2- 172.21.15.67:0/3382749506 >> v2:172.21.15.134:3300/0 conn(0x7f1dec06b040 0x7f1dec1567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:29:15.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.975+0000 7f1df1200640 1 --2- 172.21.15.67:0/3382749506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dec0c5f10 0x7f1dec0c62f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0c7330 tx=0x7f1ddc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8ace08d191ffcf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:15.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.975+0000 7f1deb400640 1 -- 172.21.15.67:0/3382749506 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc0315a0 con 0x7f1dec0c5f10 2024-03-09T09:29:15.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.976+0000 7f1deb400640 1 -- 172.21.15.67:0/3382749506 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ddc037070 con 0x7f1dec0c5f10 2024-03-09T09:29:15.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.976+0000 7f1deb400640 1 -- 172.21.15.67:0/3382749506 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc031940 con 0x7f1dec0c5f10 2024-03-09T09:29:15.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.976+0000 7f1df2600640 1 -- 172.21.15.67:0/3382749506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dec0c5f10 msgr2=0x7f1dec0c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.976+0000 7f1df2600640 1 --2- 172.21.15.67:0/3382749506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dec0c5f10 0x7f1dec0c62f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0c7330 tx=0x7f1ddc02b840 comp rx=0 tx=0).stop 2024-03-09T09:29:15.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.976+0000 7f1df2600640 1 -- 172.21.15.67:0/3382749506 shutdown_connections 2024-03-09T09:29:15.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.976+0000 7f1df2600640 1 --2- 172.21.15.67:0/3382749506 >> v2:172.21.15.134:3300/0 conn(0x7f1dec06b040 0x7f1dec1567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.976+0000 7f1df2600640 1 --2- 172.21.15.67:0/3382749506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dec0c5f10 0x7f1dec0c62f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.976+0000 7f1df2600640 1 -- 172.21.15.67:0/3382749506 >> 172.21.15.67:0/3382749506 conn(0x7f1dec0cb590 msgr2=0x7f1dec0cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:15.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.976+0000 7f1df2600640 1 -- 172.21.15.67:0/3382749506 shutdown_connections 2024-03-09T09:29:15.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.976+0000 7f1df2600640 1 -- 172.21.15.67:0/3382749506 wait complete. 2024-03-09T09:29:15.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.978+0000 7f1df2600640 1 Processor -- start 2024-03-09T09:29:15.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.979+0000 7f1df2600640 1 -- start start 2024-03-09T09:29:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.980+0000 7f1df2600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dec06b040 0x7f1dec14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.980+0000 7f1df2600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1dec0c5f10 0x7f1dec154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.980+0000 7f1df2600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dec0c9490 0x7f1dec155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.980+0000 7f1df2600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec0cb250 con 0x7f1dec06b040 2024-03-09T09:29:15.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.980+0000 7f1df2600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec0c9ff0 con 0x7f1dec0c5f10 2024-03-09T09:29:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.980+0000 7f1df2600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec0cb3d0 con 0x7f1dec0c9490 2024-03-09T09:29:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.980+0000 7f1df1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dec06b040 0x7f1dec14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.981+0000 7f1df1200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dec06b040 0x7f1dec14cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46810/0 (socket says 172.21.15.67:46810) 2024-03-09T09:29:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.981+0000 7f1df1200640 1 -- 172.21.15.67:0/2259452707 learned_addr learned my addr 172.21.15.67:0/2259452707 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.981+0000 7f1debe00640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1dec0c5f10 0x7f1dec154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.981+0000 7f1df1c00640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dec0c9490 0x7f1dec155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.981+0000 7f1df1200640 1 -- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dec0c9490 msgr2=0x7f1dec155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.981+0000 7f1df1200640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dec0c9490 0x7f1dec155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.981+0000 7f1df1200640 1 -- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1dec0c5f10 msgr2=0x7f1dec154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.981+0000 7f1df1200640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1dec0c5f10 0x7f1dec154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.981+0000 7f1df1200640 1 -- 172.21.15.67:0/2259452707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec14da70 con 0x7f1dec06b040 2024-03-09T09:29:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.982+0000 7f1df1200640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dec06b040 0x7f1dec14cf60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009890 tx=0x7f1ddc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.982+0000 7f1de8c00640 1 -- 172.21.15.67:0/2259452707 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc032630 con 0x7f1dec06b040 2024-03-09T09:29:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.982+0000 7f1de8c00640 1 -- 172.21.15.67:0/2259452707 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ddc0327d0 con 0x7f1dec06b040 2024-03-09T09:29:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.982+0000 7f1de8c00640 1 -- 172.21.15.67:0/2259452707 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc032b80 con 0x7f1dec06b040 2024-03-09T09:29:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.982+0000 7f1df2600640 1 -- 172.21.15.67:0/2259452707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dec1ecb00 con 0x7f1dec06b040 2024-03-09T09:29:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.982+0000 7f1df2600640 1 -- 172.21.15.67:0/2259452707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dec154610 con 0x7f1dec06b040 2024-03-09T09:29:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.983+0000 7f1df2600640 1 -- 172.21.15.67:0/2259452707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dec04f3c0 con 0x7f1dec06b040 2024-03-09T09:29:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.985+0000 7f1de8c00640 1 -- 172.21.15.67:0/2259452707 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1ddc04c020 con 0x7f1dec06b040 2024-03-09T09:29:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.986+0000 7f1de8c00640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dcc06b4d0 0x7f1dcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.986+0000 7f1de8c00640 1 -- 172.21.15.67:0/2259452707 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f1ddc02f020 con 0x7f1dec06b040 2024-03-09T09:29:15.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.987+0000 7f1debe00640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dcc06b4d0 0x7f1dcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.988+0000 7f1debe00640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dcc06b4d0 0x7f1dcc06d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1dec15a0f0 tx=0x7f1dd8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:15.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:15.992+0000 7f1de8c00640 1 -- 172.21.15.67:0/2259452707 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1ddc040070 con 0x7f1dec06b040 2024-03-09T09:29:16.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.188+0000 7f1df2600640 1 -- 172.21.15.67:0/2259452707 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f1dec155890 con 0x7f1dec06b040 2024-03-09T09:29:16.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.189+0000 7f1de8c00640 1 -- 172.21.15.67:0/2259452707 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v89) v1 ==== 95+0+11 (secure 0 0 0) 0x7f1ddc07ace0 con 0x7f1dec06b040 2024-03-09T09:29:16.197 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T09:29:16.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 -- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dcc06b4d0 msgr2=0x7f1dcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:16.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dcc06b4d0 0x7f1dcc06d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1dec15a0f0 tx=0x7f1dd8008040 comp rx=0 tx=0).stop 2024-03-09T09:29:16.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 -- 172.21.15.67:0/2259452707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dec06b040 msgr2=0x7f1dec14cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:16.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dec06b040 0x7f1dec14cf60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009890 tx=0x7f1ddc0046c0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 -- 172.21.15.67:0/2259452707 shutdown_connections 2024-03-09T09:29:16.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1dcc06b4d0 0x7f1dcc06d990 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1dec0c9490 0x7f1dec155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1dec0c5f10 0x7f1dec154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 --2- 172.21.15.67:0/2259452707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dec06b040 0x7f1dec14cf60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 -- 172.21.15.67:0/2259452707 >> 172.21.15.67:0/2259452707 conn(0x7f1dec0cb590 msgr2=0x7f1dec06c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:16.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 -- 172.21.15.67:0/2259452707 shutdown_connections 2024-03-09T09:29:16.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.192+0000 7f1df2600640 1 -- 172.21.15.67:0/2259452707 wait complete. 2024-03-09T09:29:16.218 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-03-09T09:29:16.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.442+0000 7fa5b6000640 1 Processor -- start 2024-03-09T09:29:16.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.443+0000 7fa5b6000640 1 -- start start 2024-03-09T09:29:16.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.444+0000 7fa5b6000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b00c73d0 0x7fa5b01547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.444+0000 7fa5b6000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa5b00710a0 0x7fa5b0151010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.444+0000 7fa5b6000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fa5b0151550 0x7fa5b0072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.444+0000 7fa5b6000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b006e480 con 0x7fa5b00710a0 2024-03-09T09:29:16.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.444+0000 7fa5b6000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b006dff0 con 0x7fa5b0151550 2024-03-09T09:29:16.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.444+0000 7fa5b6000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b006d6c0 con 0x7fa5b00c73d0 2024-03-09T09:29:16.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.445+0000 7fa5aea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa5b00710a0 0x7fa5b0151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.445+0000 7fa5aea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa5b00710a0 0x7fa5b0151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46816/0 (socket says 172.21.15.67:46816) 2024-03-09T09:29:16.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.445+0000 7fa5aea00640 1 -- 172.21.15.67:0/2160153333 learned_addr learned my addr 172.21.15.67:0/2160153333 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:16.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.445+0000 7fa5afe00640 1 --2- 172.21.15.67:0/2160153333 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0151550 0x7fa5b0072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.445+0000 7fa5af400640 1 --2- 172.21.15.67:0/2160153333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b00c73d0 0x7fa5b01547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.445+0000 7fa5aea00640 1 -- 172.21.15.67:0/2160153333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b00c73d0 msgr2=0x7fa5b01547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:16.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.445+0000 7fa5aea00640 1 --2- 172.21.15.67:0/2160153333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b00c73d0 0x7fa5b01547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.445+0000 7fa5aea00640 1 -- 172.21.15.67:0/2160153333 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0151550 msgr2=0x7fa5b0072cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:16.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.445+0000 7fa5aea00640 1 --2- 172.21.15.67:0/2160153333 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0151550 0x7fa5b0072cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.445+0000 7fa5aea00640 1 -- 172.21.15.67:0/2160153333 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b006d9a0 con 0x7fa5b00710a0 2024-03-09T09:29:16.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.447+0000 7fa5aea00640 1 --2- 172.21.15.67:0/2160153333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b00710a0 0x7fa5b0151010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0070b90 tx=0x7fa59c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2552851e03f97e41 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:16.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.447+0000 7fa5ae000640 1 -- 172.21.15.67:0/2160153333 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c035020 con 0x7fa5b00710a0 2024-03-09T09:29:16.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.447+0000 7fa5ae000640 1 -- 172.21.15.67:0/2160153333 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c031870 con 0x7fa5b00710a0 2024-03-09T09:29:16.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.447+0000 7fa5ae000640 1 -- 172.21.15.67:0/2160153333 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c031c40 con 0x7fa5b00710a0 2024-03-09T09:29:16.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.448+0000 7fa5b6000640 1 -- 172.21.15.67:0/2160153333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b00710a0 msgr2=0x7fa5b0151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:16.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.448+0000 7fa5b6000640 1 --2- 172.21.15.67:0/2160153333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b00710a0 0x7fa5b0151010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0070b90 tx=0x7fa59c02b910 comp rx=0 tx=0).stop 2024-03-09T09:29:16.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.448+0000 7fa5b6000640 1 -- 172.21.15.67:0/2160153333 shutdown_connections 2024-03-09T09:29:16.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.448+0000 7fa5b6000640 1 --2- 172.21.15.67:0/2160153333 >> v2:172.21.15.134:3300/0 conn(0x7fa5b0151550 0x7fa5b0072cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.448+0000 7fa5b6000640 1 --2- 172.21.15.67:0/2160153333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b00710a0 0x7fa5b0151010 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.448+0000 7fa5b6000640 1 --2- 172.21.15.67:0/2160153333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b00c73d0 0x7fa5b01547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.448+0000 7fa5b6000640 1 -- 172.21.15.67:0/2160153333 >> 172.21.15.67:0/2160153333 conn(0x7fa5b006b040 msgr2=0x7fa5b006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:16.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.448+0000 7fa5b6000640 1 -- 172.21.15.67:0/2160153333 shutdown_connections 2024-03-09T09:29:16.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.448+0000 7fa5b6000640 1 -- 172.21.15.67:0/2160153333 wait complete. 2024-03-09T09:29:16.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.451+0000 7fa5b6000640 1 Processor -- start 2024-03-09T09:29:16.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.451+0000 7fa5b6000640 1 -- start start 2024-03-09T09:29:16.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.452+0000 7fa5b6000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b00710a0 0x7fa5b014f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.452+0000 7fa5b6000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b00c73d0 0x7fa5b00c2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.452+0000 7fa5b6000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b0151550 0x7fa5b01ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.452+0000 7fa5b6000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00c77b0 con 0x7fa5b00c73d0 2024-03-09T09:29:16.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.452+0000 7fa5b6000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b0071500 con 0x7fa5b00710a0 2024-03-09T09:29:16.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.452+0000 7fa5b6000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00c7930 con 0x7fa5b0151550 2024-03-09T09:29:16.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.452+0000 7fa5aea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b00c73d0 0x7fa5b00c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.452+0000 7fa5aea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b00c73d0 0x7fa5b00c2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46822/0 (socket says 172.21.15.67:46822) 2024-03-09T09:29:16.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.452+0000 7fa5aea00640 1 -- 172.21.15.67:0/1323036084 learned_addr learned my addr 172.21.15.67:0/1323036084 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:16.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.453+0000 7fa5af400640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b00710a0 0x7fa5b014f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.453+0000 7fa5afe00640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b0151550 0x7fa5b01ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.453+0000 7fa5aea00640 1 -- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b0151550 msgr2=0x7fa5b01ecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:16.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.453+0000 7fa5aea00640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b0151550 0x7fa5b01ecb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.453+0000 7fa5aea00640 1 -- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b00710a0 msgr2=0x7fa5b014f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:16.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.453+0000 7fa5aea00640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b00710a0 0x7fa5b014f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.453+0000 7fa5aea00640 1 -- 172.21.15.67:0/1323036084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b00ab130 con 0x7fa5b00c73d0 2024-03-09T09:29:16.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.453+0000 7fa5aea00640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b00c73d0 0x7fa5b00c2210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa59c02b950 tx=0x7fa59c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:16.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.454+0000 7fa5a7400640 1 -- 172.21.15.67:0/1323036084 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c03ad60 con 0x7fa5b00c73d0 2024-03-09T09:29:16.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.454+0000 7fa5a7400640 1 -- 172.21.15.67:0/1323036084 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c0443f0 con 0x7fa5b00c73d0 2024-03-09T09:29:16.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.454+0000 7fa5a7400640 1 -- 172.21.15.67:0/1323036084 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0447e0 con 0x7fa5b00c73d0 2024-03-09T09:29:16.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.454+0000 7fa5b6000640 1 -- 172.21.15.67:0/1323036084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b00abe10 con 0x7fa5b00c73d0 2024-03-09T09:29:16.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.454+0000 7fa5b6000640 1 -- 172.21.15.67:0/1323036084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b00c2750 con 0x7fa5b00c73d0 2024-03-09T09:29:16.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.455+0000 7fa5b6000640 1 -- 172.21.15.67:0/1323036084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b0075a00 con 0x7fa5b00c73d0 2024-03-09T09:29:16.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.456+0000 7fa5a7400640 1 -- 172.21.15.67:0/1323036084 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa59c038020 con 0x7fa5b00c73d0 2024-03-09T09:29:16.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.457+0000 7fa5a7400640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa59006b4d0 0x7fa59006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.457+0000 7fa5a7400640 1 -- 172.21.15.67:0/1323036084 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fa59c0ac660 con 0x7fa5b00c73d0 2024-03-09T09:29:16.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.457+0000 7fa5af400640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa59006b4d0 0x7fa59006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.458+0000 7fa5af400640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa59006b4d0 0x7fa59006d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0006fd0 tx=0x7fa5a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:16.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.461+0000 7fa5a7400640 1 -- 172.21.15.67:0/1323036084 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa59c03f070 con 0x7fa5b00c73d0 2024-03-09T09:29:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.694+0000 7fa5b6000640 1 -- 172.21.15.67:0/1323036084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5b00c2ae0 con 0x7fa5b00c73d0 2024-03-09T09:29:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.695+0000 7fa5a7400640 1 -- 172.21.15.67:0/1323036084 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v89) v1 ==== 99+0+11 (secure 0 0 0) 0x7fa59c07b1b0 con 0x7fa5b00c73d0 2024-03-09T09:29:16.703 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-09T09:29:16.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 -- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa59006b4d0 msgr2=0x7fa59006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa59006b4d0 0x7fa59006d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0006fd0 tx=0x7fa5a0008040 comp rx=0 tx=0).stop 2024-03-09T09:29:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 -- 172.21.15.67:0/1323036084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b00c73d0 msgr2=0x7fa5b00c2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b00c73d0 0x7fa5b00c2210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa59c02b950 tx=0x7fa59c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 -- 172.21.15.67:0/1323036084 shutdown_connections 2024-03-09T09:29:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa59006b4d0 0x7fa59006d990 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5b0151550 0x7fa5b01ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5b00c73d0 0x7fa5b00c2210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 --2- 172.21.15.67:0/1323036084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5b00710a0 0x7fa5b014f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 -- 172.21.15.67:0/1323036084 >> 172.21.15.67:0/1323036084 conn(0x7fa5b006b040 msgr2=0x7fa5b00c43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 -- 172.21.15.67:0/1323036084 shutdown_connections 2024-03-09T09:29:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.698+0000 7fa5b6000640 1 -- 172.21.15.67:0/1323036084 wait complete. 2024-03-09T09:29:16.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-03-09T09:29:16.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.942+0000 7fbe1de00640 1 Processor -- start 2024-03-09T09:29:16.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.942+0000 7fbe1de00640 1 -- start start 2024-03-09T09:29:16.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.943+0000 7fbe1de00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbe180c8190 0x7fbe180c9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.944+0000 7fbe1de00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe181537b0 con 0x7fbe18153380 2024-03-09T09:29:16.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.944+0000 7fbe1de00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe18155d70 con 0x7fbe18152f20 2024-03-09T09:29:16.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.944+0000 7fbe1de00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe18156c40 con 0x7fbe180c8190 2024-03-09T09:29:16.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.944+0000 7fbe1ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbe180c8190 0x7fbe180c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.944+0000 7fbe1ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbe180c8190 0x7fbe180c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46836/0 (socket says 172.21.15.67:46836) 2024-03-09T09:29:16.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.944+0000 7fbe1ca00640 1 -- 172.21.15.67:0/2384417575 learned_addr learned my addr 172.21.15.67:0/2384417575 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:16.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.944+0000 7fbe1d400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbe18153380 0x7fbe180ca9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:55714/0 (socket says 172.21.15.67:55714) 2024-03-09T09:29:16.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.944+0000 7fbe1ca00640 1 -- 172.21.15.67:0/2384417575 >> v1:172.21.15.134:6789/0 conn(0x7fbe18152f20 legacy=0x7fbe180ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:16.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.944+0000 7fbe1ca00640 1 -- 172.21.15.67:0/2384417575 >> v1:172.21.15.67:6789/0 conn(0x7fbe18153380 legacy=0x7fbe180ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:16.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.944+0000 7fbe1ca00640 1 -- 172.21.15.67:0/2384417575 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe18156030 con 0x7fbe180c8190 2024-03-09T09:29:16.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.946+0000 7fbe1ca00640 1 --2- 172.21.15.67:0/2384417575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe180c8190 0x7fbe180c9640 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe18156dc0 tx=0x7fbe0002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6e4471fae9e295 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:16.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.946+0000 7fbe17e00640 1 -- 172.21.15.67:0/2384417575 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000317c0 con 0x7fbe180c8190 2024-03-09T09:29:16.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.946+0000 7fbe17e00640 1 -- 172.21.15.67:0/2384417575 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe00037070 con 0x7fbe180c8190 2024-03-09T09:29:16.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.946+0000 7fbe17e00640 1 -- 172.21.15.67:0/2384417575 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00031b80 con 0x7fbe180c8190 2024-03-09T09:29:16.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.946+0000 7fbe1de00640 1 -- 172.21.15.67:0/2384417575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe180c8190 msgr2=0x7fbe180c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:16.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.947+0000 7fbe1de00640 1 --2- 172.21.15.67:0/2384417575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe180c8190 0x7fbe180c9640 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe18156dc0 tx=0x7fbe0002b860 comp rx=0 tx=0).stop 2024-03-09T09:29:16.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.947+0000 7fbe1de00640 1 -- 172.21.15.67:0/2384417575 shutdown_connections 2024-03-09T09:29:16.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.947+0000 7fbe1de00640 1 --2- 172.21.15.67:0/2384417575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe180c8190 0x7fbe180c9640 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.947+0000 7fbe1de00640 1 -- 172.21.15.67:0/2384417575 >> 172.21.15.67:0/2384417575 conn(0x7fbe180ad060 msgr2=0x7fbe180ad440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.947+0000 7fbe1de00640 1 -- 172.21.15.67:0/2384417575 shutdown_connections 2024-03-09T09:29:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.947+0000 7fbe1de00640 1 -- 172.21.15.67:0/2384417575 wait complete. 2024-03-09T09:29:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.948+0000 7fbe1de00640 1 Processor -- start 2024-03-09T09:29:16.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.949+0000 7fbe1de00640 1 -- start start 2024-03-09T09:29:16.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.949+0000 7fbe1de00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe180c8190 0x7fbe181596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe1de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe18152f20 0x7fbe180ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe1de00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe18153380 0x7fbe180ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe1de00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe181568d0 con 0x7fbe18152f20 2024-03-09T09:29:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe1de00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe181565f0 con 0x7fbe18153380 2024-03-09T09:29:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe1de00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180c9b80 con 0x7fbe180c8190 2024-03-09T09:29:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe18152f20 0x7fbe180ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe18152f20 0x7fbe180ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46850/0 (socket says 172.21.15.67:46850) 2024-03-09T09:29:16.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe0fe00640 1 -- 172.21.15.67:0/1537893690 learned_addr learned my addr 172.21.15.67:0/1537893690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe1ca00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe180c8190 0x7fbe181596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe1d400640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe18153380 0x7fbe180ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe0fe00640 1 -- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe180c8190 msgr2=0x7fbe181596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe0fe00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe180c8190 0x7fbe181596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe0fe00640 1 -- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe18153380 msgr2=0x7fbe180ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:16.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe0fe00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe18153380 0x7fbe180ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:16.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.950+0000 7fbe0fe00640 1 -- 172.21.15.67:0/1537893690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe180c5940 con 0x7fbe18152f20 2024-03-09T09:29:16.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.951+0000 7fbe0fe00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe18152f20 0x7fbe180ae480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c430 tx=0x7fbe0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:16.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.951+0000 7fbe15600640 1 -- 172.21.15.67:0/1537893690 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08011790 con 0x7fbe18152f20 2024-03-09T09:29:16.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.951+0000 7fbe15600640 1 -- 172.21.15.67:0/1537893690 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe08017070 con 0x7fbe18152f20 2024-03-09T09:29:16.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.951+0000 7fbe15600640 1 -- 172.21.15.67:0/1537893690 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08011bb0 con 0x7fbe18152f20 2024-03-09T09:29:16.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.951+0000 7fbe1de00640 1 -- 172.21.15.67:0/1537893690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe18157bf0 con 0x7fbe18152f20 2024-03-09T09:29:16.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.951+0000 7fbe1de00640 1 -- 172.21.15.67:0/1537893690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe181581b0 con 0x7fbe18152f20 2024-03-09T09:29:16.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.952+0000 7fbe1de00640 1 -- 172.21.15.67:0/1537893690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe181583b0 con 0x7fbe18152f20 2024-03-09T09:29:16.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.953+0000 7fbe15600640 1 -- 172.21.15.67:0/1537893690 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbe0801c460 con 0x7fbe18152f20 2024-03-09T09:29:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.955+0000 7fbe15600640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbdf406b500 0x7fbdf406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.955+0000 7fbe15600640 1 -- 172.21.15.67:0/1537893690 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fbe0808a360 con 0x7fbe18152f20 2024-03-09T09:29:16.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.955+0000 7fbe1ca00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbdf406b500 0x7fbdf406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:16.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.957+0000 7fbe1ca00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbdf406b500 0x7fbdf406d9c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbe00036000 tx=0x7fbe00008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:16.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:16.959+0000 7fbe15600640 1 -- 172.21.15.67:0/1537893690 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbe181583b0 con 0x7fbe18152f20 2024-03-09T09:29:17.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.157+0000 7fbe1de00640 1 -- 172.21.15.67:0/1537893690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7fbe1804f3c0 con 0x7fbe18152f20 2024-03-09T09:29:17.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.158+0000 7fbe15600640 1 -- 172.21.15.67:0/1537893690 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v89) v1 ==== 95+0+11 (secure 0 0 0) 0x7fbe08058eb0 con 0x7fbe18152f20 2024-03-09T09:29:17.166 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-09T09:29:17.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.161+0000 7fbe1de00640 1 -- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbdf406b500 msgr2=0x7fbdf406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:17.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.161+0000 7fbe1de00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbdf406b500 0x7fbdf406d9c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbe00036000 tx=0x7fbe00008030 comp rx=0 tx=0).stop 2024-03-09T09:29:17.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.161+0000 7fbe1de00640 1 -- 172.21.15.67:0/1537893690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe18152f20 msgr2=0x7fbe180ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:17.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.161+0000 7fbe1de00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe18152f20 0x7fbe180ae480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c430 tx=0x7fbe0800c8f0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.162+0000 7fbe1de00640 1 -- 172.21.15.67:0/1537893690 shutdown_connections 2024-03-09T09:29:17.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.162+0000 7fbe1de00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbdf406b500 0x7fbdf406d9c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.162+0000 7fbe1de00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe18153380 0x7fbe180ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.162+0000 7fbe1de00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe18152f20 0x7fbe180ae480 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.162+0000 7fbe1de00640 1 --2- 172.21.15.67:0/1537893690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe180c8190 0x7fbe181596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.162+0000 7fbe1de00640 1 -- 172.21.15.67:0/1537893690 >> 172.21.15.67:0/1537893690 conn(0x7fbe180ad060 msgr2=0x7fbe1806b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:17.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.162+0000 7fbe1de00640 1 -- 172.21.15.67:0/1537893690 shutdown_connections 2024-03-09T09:29:17.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.162+0000 7fbe1de00640 1 -- 172.21.15.67:0/1537893690 wait complete. 2024-03-09T09:29:17.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-09T09:29:17.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.397+0000 7f269d600640 1 Processor -- start 2024-03-09T09:29:17.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.397+0000 7f269d600640 1 -- start start 2024-03-09T09:29:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f269d600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f26980c2780 0x7f2698154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f269d600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f2698071190 0x7f26981546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f269d600640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26980c8200 con 0x7f26980ac9a0 2024-03-09T09:29:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f269d600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269806d590 con 0x7f26980c2780 2024-03-09T09:29:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f269d600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26980c3740 con 0x7f2698071190 2024-03-09T09:29:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f2696000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f26980c2780 0x7f2698154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:17.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f2696000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f26980c2780 0x7f2698154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46858/0 (socket says 172.21.15.67:46858) 2024-03-09T09:29:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f2696000640 1 -- 172.21.15.67:0/2481399058 learned_addr learned my addr 172.21.15.67:0/2481399058 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f2696000640 1 -- 172.21.15.67:0/2481399058 >> v2:172.21.15.134:3300/0 conn(0x7f2698071190 msgr2=0x7f26981546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f2696000640 1 --2- 172.21.15.67:0/2481399058 >> v2:172.21.15.134:3300/0 conn(0x7f2698071190 0x7f26981546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f2696000640 1 -- 172.21.15.67:0/2481399058 >> v1:172.21.15.134:6789/0 conn(0x7f26980ac9a0 legacy=0x7f26981510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:17.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.399+0000 7f2696000640 1 -- 172.21.15.67:0/2481399058 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26980c3990 con 0x7f26980c2780 2024-03-09T09:29:17.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.401+0000 7f2696000640 1 --2- 172.21.15.67:0/2481399058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26980c2780 0x7f2698154190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f26980c3190 tx=0x7f268402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9647854609e1bded server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:17.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.401+0000 7f2695600640 1 -- 172.21.15.67:0/2481399058 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268402e070 con 0x7f26980c2780 2024-03-09T09:29:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.401+0000 7f2695600640 1 -- 172.21.15.67:0/2481399058 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684037070 con 0x7f26980c2780 2024-03-09T09:29:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.401+0000 7f2695600640 1 -- 172.21.15.67:0/2481399058 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684031a00 con 0x7f26980c2780 2024-03-09T09:29:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.402+0000 7f269d600640 1 -- 172.21.15.67:0/2481399058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26980c2780 msgr2=0x7f2698154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.402+0000 7f269d600640 1 --2- 172.21.15.67:0/2481399058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26980c2780 0x7f2698154190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f26980c3190 tx=0x7f268402b880 comp rx=0 tx=0).stop 2024-03-09T09:29:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.402+0000 7f269d600640 1 -- 172.21.15.67:0/2481399058 shutdown_connections 2024-03-09T09:29:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.402+0000 7f269d600640 1 --2- 172.21.15.67:0/2481399058 >> v2:172.21.15.134:3300/0 conn(0x7f2698071190 0x7f26981546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.402+0000 7f269d600640 1 --2- 172.21.15.67:0/2481399058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26980c2780 0x7f2698154190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.402+0000 7f269d600640 1 -- 172.21.15.67:0/2481399058 >> 172.21.15.67:0/2481399058 conn(0x7f26980c7370 msgr2=0x7f26980c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:17.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.402+0000 7f269d600640 1 -- 172.21.15.67:0/2481399058 shutdown_connections 2024-03-09T09:29:17.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.402+0000 7f269d600640 1 -- 172.21.15.67:0/2481399058 wait complete. 2024-03-09T09:29:17.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.403+0000 7f269d600640 1 Processor -- start 2024-03-09T09:29:17.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.404+0000 7f269d600640 1 -- start start 2024-03-09T09:29:17.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.404+0000 7f269d600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2698071190 0x7f26981df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:17.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.404+0000 7f269d600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26980ac9a0 0x7f26981eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:17.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.404+0000 7f269d600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26980c2780 0x7f26981eb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:17.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.404+0000 7f269d600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2698154c10 con 0x7f26980ac9a0 2024-03-09T09:29:17.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.404+0000 7f269d600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2698070c00 con 0x7f26980c2780 2024-03-09T09:29:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.404+0000 7f269d600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26980acd80 con 0x7f2698071190 2024-03-09T09:29:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.405+0000 7f2696000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26980ac9a0 0x7f26981eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.405+0000 7f2696000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26980ac9a0 0x7f26981eabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46868/0 (socket says 172.21.15.67:46868) 2024-03-09T09:29:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.405+0000 7f2696000640 1 -- 172.21.15.67:0/120925881 learned_addr learned my addr 172.21.15.67:0/120925881 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.405+0000 7f2696a00640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2698071190 0x7f26981df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.405+0000 7f2697400640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26980c2780 0x7f26981eb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.405+0000 7f2696000640 1 -- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2698071190 msgr2=0x7f26981df4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.405+0000 7f2696000640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2698071190 0x7f26981df4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.405+0000 7f2696000640 1 -- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26980c2780 msgr2=0x7f26981eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:17.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.405+0000 7f2696000640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26980c2780 0x7f26981eb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.405+0000 7f2696000640 1 -- 172.21.15.67:0/120925881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269814c1a0 con 0x7f26980ac9a0 2024-03-09T09:29:17.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.406+0000 7f2696000640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26980ac9a0 0x7f26981eabf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f268402b8c0 tx=0x7f26840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:17.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.406+0000 7f268ea00640 1 -- 172.21.15.67:0/120925881 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684032880 con 0x7f26980ac9a0 2024-03-09T09:29:17.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.406+0000 7f268ea00640 1 -- 172.21.15.67:0/120925881 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684032a40 con 0x7f26980ac9a0 2024-03-09T09:29:17.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.406+0000 7f268ea00640 1 -- 172.21.15.67:0/120925881 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684032e30 con 0x7f26980ac9a0 2024-03-09T09:29:17.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.406+0000 7f269d600640 1 -- 172.21.15.67:0/120925881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26981eb880 con 0x7f26980ac9a0 2024-03-09T09:29:17.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.406+0000 7f269d600640 1 -- 172.21.15.67:0/120925881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2698151230 con 0x7f26980ac9a0 2024-03-09T09:29:17.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.407+0000 7f269d600640 1 -- 172.21.15.67:0/120925881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2698075a00 con 0x7f26980ac9a0 2024-03-09T09:29:17.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.408+0000 7f268ea00640 1 -- 172.21.15.67:0/120925881 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f268404c060 con 0x7f26980ac9a0 2024-03-09T09:29:17.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.409+0000 7f268ea00640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f267806b4d0 0x7f267806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:17.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.410+0000 7f268ea00640 1 -- 172.21.15.67:0/120925881 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f26840abeb0 con 0x7f26980ac9a0 2024-03-09T09:29:17.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.410+0000 7f2696a00640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f267806b4d0 0x7f267806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:17.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.411+0000 7f2696a00640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f267806b4d0 0x7f267806d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2688004ea0 tx=0x7f2688009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:17.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.414+0000 7f268ea00640 1 -- 172.21.15.67:0/120925881 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2684040070 con 0x7f26980ac9a0 2024-03-09T09:29:17.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.627+0000 7f269d600640 1 -- 172.21.15.67:0/120925881 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f26980c4330 con 0x7f26980ac9a0 2024-03-09T09:29:17.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.627+0000 7f268ea00640 1 -- 172.21.15.67:0/120925881 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) v1 ==== 80+0+0 (secure 0 0 0) 0x7f268407a8d0 con 0x7f26980ac9a0 2024-03-09T09:29:17.635 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2024-03-09T09:29:17.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 -- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f267806b4d0 msgr2=0x7f267806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:17.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f267806b4d0 0x7f267806d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2688004ea0 tx=0x7f2688009040 comp rx=0 tx=0).stop 2024-03-09T09:29:17.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 -- 172.21.15.67:0/120925881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26980ac9a0 msgr2=0x7f26981eabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:17.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26980ac9a0 0x7f26981eabf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f268402b8c0 tx=0x7f26840046c0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 -- 172.21.15.67:0/120925881 shutdown_connections 2024-03-09T09:29:17.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f267806b4d0 0x7f267806d990 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f26980c2780 0x7f26981eb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f26980ac9a0 0x7f26981eabf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 --2- 172.21.15.67:0/120925881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2698071190 0x7f26981df4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:17.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 -- 172.21.15.67:0/120925881 >> 172.21.15.67:0/120925881 conn(0x7f26980c7370 msgr2=0x7f269806d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:17.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 -- 172.21.15.67:0/120925881 shutdown_connections 2024-03-09T09:29:17.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:17.630+0000 7f269d600640 1 -- 172.21.15.67:0/120925881 wait complete. 2024-03-09T09:29:17.664 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-09T09:29:17.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T09:29:17.664 DEBUG:teuthology.orchestra.run.smithi134:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-09T09:29:18.161 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T09:29:18.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T09:29:18.162 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-09T09:29:18.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T09:29:18.207 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-03-09T09:29:18.260 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-09T09:29:18.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T09:29:18.261 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-09T09:29:18.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T09:29:18.351 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-03-09T09:29:18.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T09:29:18.485 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-03-09T09:29:18.538 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-09T09:29:18.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T09:29:18.539 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-09T09:29:18.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-09T09:29:18.625 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-03-09T09:29:18.742 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-09T09:29:18.753 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-09T09:29:18.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T09:29:18.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.969+0000 7f8e16a00640 1 Processor -- start 2024-03-09T09:29:18.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.970+0000 7f8e16a00640 1 -- start start 2024-03-09T09:29:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.971+0000 7f8e16a00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8e100ac730 0x7f8e10071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.971+0000 7f8e16a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e100c22a0 0x7f8e100acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.971+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e10157650 0x7f8e10157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:18.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.971+0000 7f8e16a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e10070ee0 con 0x7f8e100c22a0 2024-03-09T09:29:18.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.971+0000 7f8e16a00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e10154d00 con 0x7f8e100ac730 2024-03-09T09:29:18.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.971+0000 7f8e16a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e10153d40 con 0x7f8e10157650 2024-03-09T09:29:18.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.972+0000 7f8e07e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e100c22a0 0x7f8e100acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:18.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.972+0000 7f8e0fe00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8e100ac730 0x7f8e10071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:18.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.972+0000 7f8e07e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e100c22a0 0x7f8e100acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45014/0 (socket says 172.21.15.67:45014) 2024-03-09T09:29:18.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.972+0000 7f8e07e00640 1 -- 172.21.15.67:0/4285655947 learned_addr learned my addr 172.21.15.67:0/4285655947 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:18.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.972+0000 7f8e14a00640 1 --2- 172.21.15.67:0/4285655947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e10157650 0x7f8e10157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:18.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.973+0000 7f8e07e00640 1 -- 172.21.15.67:0/4285655947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e10157650 msgr2=0x7f8e10157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.973+0000 7f8e07e00640 1 --2- 172.21.15.67:0/4285655947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e10157650 0x7f8e10157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.973+0000 7f8e07e00640 1 -- 172.21.15.67:0/4285655947 >> v2:172.21.15.134:3300/0 conn(0x7f8e100ac730 msgr2=0x7f8e10071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.973+0000 7f8e07e00640 1 --2- 172.21.15.67:0/4285655947 >> v2:172.21.15.134:3300/0 conn(0x7f8e100ac730 0x7f8e10071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.973+0000 7f8e07e00640 1 -- 172.21.15.67:0/4285655947 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e100c2790 con 0x7f8e100c22a0 2024-03-09T09:29:18.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.975+0000 7f8e07e00640 1 --2- 172.21.15.67:0/4285655947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e100c22a0 0x7f8e100acb10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8e100ad380 tx=0x7f8df802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50253008b25594c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.976+0000 7f8e0f400640 1 -- 172.21.15.67:0/4285655947 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8035020 con 0x7f8e100c22a0 2024-03-09T09:29:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.976+0000 7f8e0f400640 1 -- 172.21.15.67:0/4285655947 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df8031870 con 0x7f8e100c22a0 2024-03-09T09:29:18.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.976+0000 7f8e0f400640 1 -- 172.21.15.67:0/4285655947 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8031c40 con 0x7f8e100c22a0 2024-03-09T09:29:18.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.976+0000 7f8e16a00640 1 -- 172.21.15.67:0/4285655947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e100c22a0 msgr2=0x7f8e100acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.976+0000 7f8e16a00640 1 --2- 172.21.15.67:0/4285655947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e100c22a0 0x7f8e100acb10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8e100ad380 tx=0x7f8df802b910 comp rx=0 tx=0).stop 2024-03-09T09:29:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.977+0000 7f8e16a00640 1 -- 172.21.15.67:0/4285655947 shutdown_connections 2024-03-09T09:29:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.977+0000 7f8e16a00640 1 --2- 172.21.15.67:0/4285655947 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e10157650 0x7f8e10157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.977+0000 7f8e16a00640 1 --2- 172.21.15.67:0/4285655947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e100c22a0 0x7f8e100acb10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.977+0000 7f8e16a00640 1 --2- 172.21.15.67:0/4285655947 >> v2:172.21.15.134:3300/0 conn(0x7f8e100ac730 0x7f8e10071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.977+0000 7f8e16a00640 1 -- 172.21.15.67:0/4285655947 >> 172.21.15.67:0/4285655947 conn(0x7f8e100c7370 msgr2=0x7f8e100c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.977+0000 7f8e16a00640 1 -- 172.21.15.67:0/4285655947 shutdown_connections 2024-03-09T09:29:18.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.977+0000 7f8e16a00640 1 -- 172.21.15.67:0/4285655947 wait complete. 2024-03-09T09:29:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.979+0000 7f8e16a00640 1 Processor -- start 2024-03-09T09:29:18.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.979+0000 7f8e16a00640 1 -- start start 2024-03-09T09:29:18.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.980+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e100ac730 0x7f8e100abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.980+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e100c22a0 0x7f8e10155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.980+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e10157650 0x7f8e1014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.980+0000 7f8e16a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e10158020 con 0x7f8e100c22a0 2024-03-09T09:29:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.980+0000 7f8e16a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e10070c00 con 0x7f8e10157650 2024-03-09T09:29:18.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.980+0000 7f8e16a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e101581a0 con 0x7f8e100ac730 2024-03-09T09:29:18.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.980+0000 7f8e07e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e100c22a0 0x7f8e10155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.980+0000 7f8e07e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e100c22a0 0x7f8e10155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45026/0 (socket says 172.21.15.67:45026) 2024-03-09T09:29:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.980+0000 7f8e07e00640 1 -- 172.21.15.67:0/2825906841 learned_addr learned my addr 172.21.15.67:0/2825906841 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.981+0000 7f8e0fe00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e100ac730 0x7f8e100abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.981+0000 7f8e14a00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e10157650 0x7f8e1014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.981+0000 7f8e07e00640 1 -- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e100ac730 msgr2=0x7f8e100abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.981+0000 7f8e07e00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e100ac730 0x7f8e100abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.981+0000 7f8e07e00640 1 -- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e10157650 msgr2=0x7f8e1014d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:18.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.981+0000 7f8e07e00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e10157650 0x7f8e1014d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.981+0000 7f8e07e00640 1 -- 172.21.15.67:0/2825906841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1009be10 con 0x7f8e100c22a0 2024-03-09T09:29:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.981+0000 7f8e14a00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e10157650 0x7f8e1014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:29:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.981+0000 7f8e07e00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e100c22a0 0x7f8e10155ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8df8000c00 tx=0x7f8df80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.982+0000 7f8e0cc00640 1 -- 172.21.15.67:0/2825906841 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8035040 con 0x7f8e100c22a0 2024-03-09T09:29:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.982+0000 7f8e0cc00640 1 -- 172.21.15.67:0/2825906841 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df8030d00 con 0x7f8e100c22a0 2024-03-09T09:29:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.982+0000 7f8e0cc00640 1 -- 172.21.15.67:0/2825906841 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df80456a0 con 0x7f8e100c22a0 2024-03-09T09:29:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.982+0000 7f8e16a00640 1 -- 172.21.15.67:0/2825906841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e100ac1f0 con 0x7f8e100c22a0 2024-03-09T09:29:18.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.982+0000 7f8e16a00640 1 -- 172.21.15.67:0/2825906841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1009c390 con 0x7f8e100c22a0 2024-03-09T09:29:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.983+0000 7f8e16a00640 1 -- 172.21.15.67:0/2825906841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e10075a00 con 0x7f8e100c22a0 2024-03-09T09:29:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.984+0000 7f8e0cc00640 1 -- 172.21.15.67:0/2825906841 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8df8038020 con 0x7f8e100c22a0 2024-03-09T09:29:18.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.985+0000 7f8e0cc00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dec06b4d0 0x7f8dec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:18.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.985+0000 7f8e0cc00640 1 -- 172.21.15.67:0/2825906841 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f8df80ac8a0 con 0x7f8e100c22a0 2024-03-09T09:29:18.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.985+0000 7f8e0fe00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dec06b4d0 0x7f8dec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:18.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.986+0000 7f8e0fe00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dec06b4d0 0x7f8dec06d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009a20 tx=0x7f8e00009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:18.994+0000 7f8e0cc00640 1 -- 172.21.15.67:0/2825906841 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8df807b350 con 0x7f8e100c22a0 2024-03-09T09:29:19.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.191+0000 7f8e16a00640 1 -- 172.21.15.67:0/2825906841 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e1009c5d0 con 0x7f8dec06b4d0 2024-03-09T09:29:19.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.234+0000 7f8e0cc00640 1 -- 172.21.15.67:0/2825906841 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+499763 (secure 0 0 0) 0x7f8e00011940 con 0x7f8dec06b4d0 2024-03-09T09:29:19.250 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:29:19.252 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T09:29:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.243+0000 7f8e16a00640 1 -- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dec06b4d0 msgr2=0x7f8dec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.243+0000 7f8e16a00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dec06b4d0 0x7f8dec06d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009a20 tx=0x7f8e00009190 comp rx=0 tx=0).stop 2024-03-09T09:29:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.243+0000 7f8e16a00640 1 -- 172.21.15.67:0/2825906841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e100c22a0 msgr2=0x7f8e10155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.243+0000 7f8e16a00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e100c22a0 0x7f8e10155ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8df8000c00 tx=0x7f8df80046c0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.243+0000 7f8e16a00640 1 -- 172.21.15.67:0/2825906841 shutdown_connections 2024-03-09T09:29:19.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.243+0000 7f8e16a00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8dec06b4d0 0x7f8dec06d990 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.243+0000 7f8e16a00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e10157650 0x7f8e1014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.243+0000 7f8e16a00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e100c22a0 0x7f8e10155ae0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.243+0000 7f8e16a00640 1 --2- 172.21.15.67:0/2825906841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e100ac730 0x7f8e100abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.243+0000 7f8e16a00640 1 -- 172.21.15.67:0/2825906841 >> 172.21.15.67:0/2825906841 conn(0x7f8e100c7370 msgr2=0x7f8e10073110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:19.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.244+0000 7f8e16a00640 1 -- 172.21.15.67:0/2825906841 shutdown_connections 2024-03-09T09:29:19.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.244+0000 7f8e16a00640 1 -- 172.21.15.67:0/2825906841 wait complete. 2024-03-09T09:29:19.273 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":1197,"stamp":"2024-03-09T09:29:17.710364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1877224629,"num_objects":17180,"num_object_clones":4728,"num_object_copies":34360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17180,"num_whiteouts":4728,"num_read":20747,"num_read_kb":3057165,"num_write":223823,"num_write_kb":12116397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2438,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":332679,"ondisk_log_size":332679,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6205708,"kb_used_data":3720588,"kb_used_omap":12,"kb_used_meta":2485043,"kb_avail":748769012,"statfs":{"total":773094113280,"available":766739468288,"internally_reserved":0,"allocated":3809882112,"data_stored":3758988958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2544684594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":964688,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":16,"num_write_kb":950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005483"},"pg_stats":[{"pgid":"5.18","version":"89'2465","reported_seq":5338,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:47.659926+0000","last_change":"2024-03-09T09:27:34.120297+0000","last_active":"2024-03-09T09:28:47.659926+0000","last_peered":"2024-03-09T09:28:47.659926+0000","last_clean":"2024-03-09T09:28:47.659926+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T09:28:47.659926+0000","last_undegraded":"2024-03-09T09:28:47.659926+0000","last_fullsized":"2024-03-09T09:28:47.659926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2465,"log_dups_size":0,"ondisk_log_size":2465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:09:33.307167+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":3.854819714,"stat_sum":{"num_bytes":1582215,"num_objects":90,"num_object_clones":27,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":27,"num_read":122,"num_read_kb":30313,"num_write":1455,"num_write_kb":61176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"89'449","reported_seq":618,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:51.137870+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T09:27:51.137870+0000","last_peered":"2024-03-09T09:27:51.137870+0000","last_clean":"2024-03-09T09:27:51.137870+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T09:27:51.137870+0000","last_undegraded":"2024-03-09T09:27:51.137870+0000","last_fullsized":"2024-03-09T09:27:51.137870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:41:40.332687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":462,"num_write_kb":82632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"89'2178","reported_seq":5158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.532867+0000","last_change":"2024-03-09T09:27:12.157369+0000","last_active":"2024-03-09T09:28:23.532867+0000","last_peered":"2024-03-09T09:28:23.532867+0000","last_clean":"2024-03-09T09:28:23.532867+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T09:28:23.532867+0000","last_undegraded":"2024-03-09T09:28:23.532867+0000","last_fullsized":"2024-03-09T09:28:23.532867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2178,"log_dups_size":0,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:03:38.261625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.47085273999999999,"stat_sum":{"num_bytes":1473782,"num_objects":111,"num_object_clones":38,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":38,"num_read":99,"num_read_kb":25344,"num_write":1311,"num_write_kb":54508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"89'405","reported_seq":563,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:59.412978+0000","last_change":"2024-03-09T08:52:28.571905+0000","last_active":"2024-03-09T09:27:59.412978+0000","last_peered":"2024-03-09T09:27:59.412978+0000","last_clean":"2024-03-09T09:27:59.412978+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T09:27:59.412978+0000","last_undegraded":"2024-03-09T09:27:59.412978+0000","last_fullsized":"2024-03-09T09:27:59.412978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:01.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":445,"num_write_kb":78871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"89'2252","reported_seq":5164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:52.564577+0000","last_change":"2024-03-09T09:27:32.072161+0000","last_active":"2024-03-09T09:28:52.564577+0000","last_peered":"2024-03-09T09:28:52.564577+0000","last_clean":"2024-03-09T09:28:52.564577+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T09:28:52.564577+0000","last_undegraded":"2024-03-09T09:28:52.564577+0000","last_fullsized":"2024-03-09T09:28:52.564577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2252,"log_dups_size":0,"ondisk_log_size":2252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:24.242000+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.703796324,"stat_sum":{"num_bytes":1252433,"num_objects":118,"num_object_clones":38,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":38,"num_read":118,"num_read_kb":38794,"num_write":1352,"num_write_kb":71340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"89'744","reported_seq":913,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.820185+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T09:28:14.820185+0000","last_peered":"2024-03-09T09:28:14.820185+0000","last_clean":"2024-03-09T09:28:14.820185+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T09:28:14.820185+0000","last_undegraded":"2024-03-09T09:28:14.820185+0000","last_fullsized":"2024-03-09T09:28:14.820185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":50,"num_read_kb":0,"num_write":805,"num_write_kb":91455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'2158","reported_seq":5144,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.591443+0000","last_change":"2024-03-09T09:27:09.692723+0000","last_active":"2024-03-09T09:28:23.591443+0000","last_peered":"2024-03-09T09:28:23.591443+0000","last_clean":"2024-03-09T09:28:23.591443+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T09:28:23.591443+0000","last_undegraded":"2024-03-09T09:28:23.591443+0000","last_fullsized":"2024-03-09T09:28:23.591443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2158,"log_dups_size":0,"ondisk_log_size":2158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:29:50.385379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.48044756999999999,"stat_sum":{"num_bytes":498978,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":114,"num_read_kb":18251,"num_write":1268,"num_write_kb":37672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"89'892","reported_seq":1051,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.657948+0000","last_change":"2024-03-09T08:52:28.573074+0000","last_active":"2024-03-09T09:28:26.657948+0000","last_peered":"2024-03-09T09:28:26.657948+0000","last_clean":"2024-03-09T09:28:26.657948+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T09:28:26.657948+0000","last_undegraded":"2024-03-09T09:28:26.657948+0000","last_fullsized":"2024-03-09T09:28:26.657948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:50.600408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":939,"num_write_kb":120134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"89'2262","reported_seq":5165,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.740407+0000","last_change":"2024-03-09T09:27:34.614436+0000","last_active":"2024-03-09T09:28:56.740407+0000","last_peered":"2024-03-09T09:28:56.740407+0000","last_clean":"2024-03-09T09:28:56.740407+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T09:28:56.740407+0000","last_undegraded":"2024-03-09T09:28:56.740407+0000","last_fullsized":"2024-03-09T09:28:56.740407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2262,"log_dups_size":0,"ondisk_log_size":2262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:18:23.796765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.3826170339999999,"stat_sum":{"num_bytes":1068833,"num_objects":92,"num_object_clones":30,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":30,"num_read":126,"num_read_kb":14571,"num_write":1308,"num_write_kb":45011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"89'674","reported_seq":840,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:58.111396+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T09:27:58.111396+0000","last_peered":"2024-03-09T09:27:58.111396+0000","last_clean":"2024-03-09T09:27:58.111396+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T09:27:58.111396+0000","last_undegraded":"2024-03-09T09:27:58.111396+0000","last_fullsized":"2024-03-09T09:27:58.111396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":67,"num_read_kb":0,"num_write":775,"num_write_kb":140765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"89'2379","reported_seq":5342,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.635615+0000","last_change":"2024-03-09T09:27:14.117823+0000","last_active":"2024-03-09T09:28:23.635615+0000","last_peered":"2024-03-09T09:28:23.635615+0000","last_clean":"2024-03-09T09:28:23.635615+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T09:28:23.635615+0000","last_undegraded":"2024-03-09T09:28:23.635615+0000","last_fullsized":"2024-03-09T09:28:23.635615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2379,"log_dups_size":0,"ondisk_log_size":2379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:49:44.890858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.653915281,"stat_sum":{"num_bytes":2793747,"num_objects":148,"num_object_clones":53,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":53,"num_read":138,"num_read_kb":27086,"num_write":1428,"num_write_kb":43660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"89'531","reported_seq":689,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.454336+0000","last_change":"2024-03-09T08:52:28.577128+0000","last_active":"2024-03-09T09:28:14.454336+0000","last_peered":"2024-03-09T09:28:14.454336+0000","last_clean":"2024-03-09T09:28:14.454336+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T09:28:14.454336+0000","last_undegraded":"2024-03-09T09:28:14.454336+0000","last_fullsized":"2024-03-09T09:28:14.454336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:41.801752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":543,"num_write_kb":111605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"89'2269","reported_seq":5167,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.750594+0000","last_change":"2024-03-09T09:27:34.731982+0000","last_active":"2024-03-09T09:28:56.750594+0000","last_peered":"2024-03-09T09:28:56.750594+0000","last_clean":"2024-03-09T09:28:56.750594+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T09:28:56.750594+0000","last_undegraded":"2024-03-09T09:28:56.750594+0000","last_fullsized":"2024-03-09T09:28:56.750594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2269,"log_dups_size":0,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:34:13.555244+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4571779879999998,"stat_sum":{"num_bytes":3086231,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":121,"num_read_kb":9749,"num_write":1336,"num_write_kb":28981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"89'459","reported_seq":625,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:52.092030+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T09:27:52.092030+0000","last_peered":"2024-03-09T09:27:52.092030+0000","last_clean":"2024-03-09T09:27:52.092030+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T09:27:52.092030+0000","last_undegraded":"2024-03-09T09:27:52.092030+0000","last_fullsized":"2024-03-09T09:27:52.092030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":55,"num_read_kb":0,"num_write":535,"num_write_kb":83331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'2153","reported_seq":5147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.664103+0000","last_change":"2024-03-09T09:27:13.836207+0000","last_active":"2024-03-09T09:28:23.664103+0000","last_peered":"2024-03-09T09:28:23.664103+0000","last_clean":"2024-03-09T09:28:23.664103+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T09:28:23.664103+0000","last_undegraded":"2024-03-09T09:28:23.664103+0000","last_fullsized":"2024-03-09T09:28:23.664103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2153,"log_dups_size":0,"ondisk_log_size":2153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:20:22.564401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.3715434019999999,"stat_sum":{"num_bytes":1052383,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":116,"num_read_kb":6444,"num_write":1289,"num_write_kb":22757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"89'569","reported_seq":727,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:51.106524+0000","last_change":"2024-03-09T08:52:28.577069+0000","last_active":"2024-03-09T09:27:51.106524+0000","last_peered":"2024-03-09T09:27:51.106524+0000","last_clean":"2024-03-09T09:27:51.106524+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T09:27:51.106524+0000","last_undegraded":"2024-03-09T09:27:51.106524+0000","last_fullsized":"2024-03-09T09:27:51.106524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:09:02.410915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":598,"num_write_kb":95071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"89'2154","reported_seq":5089,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.560248+0000","last_change":"2024-03-09T09:27:32.079362+0000","last_active":"2024-03-09T09:28:32.560248+0000","last_peered":"2024-03-09T09:28:32.560248+0000","last_clean":"2024-03-09T09:28:32.560248+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T09:28:32.560248+0000","last_undegraded":"2024-03-09T09:28:32.560248+0000","last_fullsized":"2024-03-09T09:28:32.560248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2154,"log_dups_size":0,"ondisk_log_size":2154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:14:45.458685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.709485658,"stat_sum":{"num_bytes":1197226,"num_objects":92,"num_object_clones":31,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":31,"num_read":105,"num_read_kb":17945,"num_write":1287,"num_write_kb":45184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"89'2226","reported_seq":5194,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.062389+0000","last_change":"2024-03-09T09:27:07.936292+0000","last_active":"2024-03-09T09:29:14.062389+0000","last_peered":"2024-03-09T09:29:14.062389+0000","last_clean":"2024-03-09T09:29:14.062389+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T09:29:14.062389+0000","last_undegraded":"2024-03-09T09:29:14.062389+0000","last_fullsized":"2024-03-09T09:29:14.062389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2226,"log_dups_size":0,"ondisk_log_size":2226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:15:17.296354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.2630355259999999,"stat_sum":{"num_bytes":1174090,"num_objects":98,"num_object_clones":35,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":35,"num_read":98,"num_read_kb":18109,"num_write":1319,"num_write_kb":36158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"89'817","reported_seq":985,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.795764+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T09:28:14.795764+0000","last_peered":"2024-03-09T09:28:14.795764+0000","last_clean":"2024-03-09T09:28:14.795764+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T09:28:14.795764+0000","last_undegraded":"2024-03-09T09:28:14.795764+0000","last_fullsized":"2024-03-09T09:28:14.795764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":48,"num_read_kb":0,"num_write":879,"num_write_kb":99632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"89'743","reported_seq":903,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.514530+0000","last_change":"2024-03-09T08:52:28.570631+0000","last_active":"2024-03-09T09:28:31.514530+0000","last_peered":"2024-03-09T09:28:31.514530+0000","last_clean":"2024-03-09T09:28:31.514530+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T09:28:31.514530+0000","last_undegraded":"2024-03-09T09:28:31.514530+0000","last_fullsized":"2024-03-09T09:28:31.514530+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:06.630704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971838,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":820,"num_write_kb":111945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"89'2187","reported_seq":5097,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.736598+0000","last_change":"2024-03-09T09:27:30.893758+0000","last_active":"2024-03-09T09:28:56.736598+0000","last_peered":"2024-03-09T09:28:56.736598+0000","last_clean":"2024-03-09T09:28:56.736598+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T09:28:56.736598+0000","last_undegraded":"2024-03-09T09:28:56.736598+0000","last_fullsized":"2024-03-09T09:28:56.736598+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2187,"log_dups_size":0,"ondisk_log_size":2187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:16.469362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.63600609399999997,"stat_sum":{"num_bytes":1331264,"num_objects":108,"num_object_clones":38,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":38,"num_read":110,"num_read_kb":9978,"num_write":1295,"num_write_kb":30848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'2259","reported_seq":5235,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.564562+0000","last_change":"2024-03-09T09:27:12.385283+0000","last_active":"2024-03-09T09:28:23.564562+0000","last_peered":"2024-03-09T09:28:23.564562+0000","last_clean":"2024-03-09T09:28:23.564562+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T09:28:23.564562+0000","last_undegraded":"2024-03-09T09:28:23.564562+0000","last_fullsized":"2024-03-09T09:28:23.564562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2259,"log_dups_size":0,"ondisk_log_size":2259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:24:11.227698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.69905185199999997,"stat_sum":{"num_bytes":1031167,"num_objects":108,"num_object_clones":39,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":39,"num_read":109,"num_read_kb":22315,"num_write":1384,"num_write_kb":51621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'445","reported_seq":614,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.253653+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T09:28:16.253653+0000","last_peered":"2024-03-09T09:28:16.253653+0000","last_clean":"2024-03-09T09:28:16.253653+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T09:28:16.253653+0000","last_undegraded":"2024-03-09T09:28:16.253653+0000","last_fullsized":"2024-03-09T09:28:16.253653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:45:45.612953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":467,"num_write_kb":115624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"89'633","reported_seq":793,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.652095+0000","last_change":"2024-03-09T08:52:28.577110+0000","last_active":"2024-03-09T09:28:26.652095+0000","last_peered":"2024-03-09T09:28:26.652095+0000","last_clean":"2024-03-09T09:28:26.652095+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T09:28:26.652095+0000","last_undegraded":"2024-03-09T09:28:26.652095+0000","last_fullsized":"2024-03-09T09:28:26.652095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:57.197373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":674,"num_write_kb":99647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"89'2250","reported_seq":5168,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.732049+0000","last_change":"2024-03-09T09:27:34.263088+0000","last_active":"2024-03-09T09:28:56.732049+0000","last_peered":"2024-03-09T09:28:56.732049+0000","last_clean":"2024-03-09T09:28:56.732049+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T09:28:56.732049+0000","last_undegraded":"2024-03-09T09:28:56.732049+0000","last_fullsized":"2024-03-09T09:28:56.732049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2250,"log_dups_size":0,"ondisk_log_size":2250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:15:00.105256+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":3.9938263410000001,"stat_sum":{"num_bytes":1661343,"num_objects":107,"num_object_clones":33,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":33,"num_read":118,"num_read_kb":35803,"num_write":1365,"num_write_kb":51532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'2347","reported_seq":5330,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.606885+0000","last_change":"2024-03-09T09:27:12.068457+0000","last_active":"2024-03-09T09:28:23.606885+0000","last_peered":"2024-03-09T09:28:23.606885+0000","last_clean":"2024-03-09T09:28:23.606885+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T09:28:23.606885+0000","last_undegraded":"2024-03-09T09:28:23.606885+0000","last_fullsized":"2024-03-09T09:28:23.606885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2347,"log_dups_size":0,"ondisk_log_size":2347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:13.795599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.380889329,"stat_sum":{"num_bytes":1156693,"num_objects":132,"num_object_clones":46,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":46,"num_read":122,"num_read_kb":38030,"num_write":1453,"num_write_kb":61557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"89'644","reported_seq":813,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.817502+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T09:28:14.817502+0000","last_peered":"2024-03-09T09:28:14.817502+0000","last_clean":"2024-03-09T09:28:14.817502+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T09:28:14.817502+0000","last_undegraded":"2024-03-09T09:28:14.817502+0000","last_fullsized":"2024-03-09T09:28:14.817502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:21:59.990396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":715,"num_write_kb":95464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"89'626","reported_seq":786,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:27.476620+0000","last_change":"2024-03-09T08:52:28.576254+0000","last_active":"2024-03-09T09:28:27.476620+0000","last_peered":"2024-03-09T09:28:27.476620+0000","last_clean":"2024-03-09T09:28:27.476620+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T09:28:27.476620+0000","last_undegraded":"2024-03-09T09:28:27.476620+0000","last_fullsized":"2024-03-09T09:28:27.476620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:21:56.330955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748758,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":685,"num_write_kb":95366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"89'2197","reported_seq":5123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.730637+0000","last_change":"2024-03-09T09:27:34.731118+0000","last_active":"2024-03-09T09:28:56.730637+0000","last_peered":"2024-03-09T09:28:56.730637+0000","last_clean":"2024-03-09T09:28:56.730637+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T09:28:56.730637+0000","last_undegraded":"2024-03-09T09:28:56.730637+0000","last_fullsized":"2024-03-09T09:28:56.730637+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2197,"log_dups_size":0,"ondisk_log_size":2197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:12.794501+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4556873890000004,"stat_sum":{"num_bytes":4719568,"num_objects":111,"num_object_clones":34,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":34,"num_read":112,"num_read_kb":26370,"num_write":1323,"num_write_kb":45566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"89'2296","reported_seq":5256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.039509+0000","last_change":"2024-03-09T09:27:08.625397+0000","last_active":"2024-03-09T09:29:14.039509+0000","last_peered":"2024-03-09T09:29:14.039509+0000","last_clean":"2024-03-09T09:29:14.039509+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T09:29:14.039509+0000","last_undegraded":"2024-03-09T09:29:14.039509+0000","last_fullsized":"2024-03-09T09:29:14.039509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2296,"log_dups_size":0,"ondisk_log_size":2296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:26:08.206478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0901323600000001,"stat_sum":{"num_bytes":867761,"num_objects":98,"num_object_clones":31,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":31,"num_read":98,"num_read_kb":37765,"num_write":1389,"num_write_kb":74512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'434","reported_seq":603,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.795677+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T09:28:14.795677+0000","last_peered":"2024-03-09T09:28:14.795677+0000","last_clean":"2024-03-09T09:28:14.795677+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T09:28:14.795677+0000","last_undegraded":"2024-03-09T09:28:14.795677+0000","last_fullsized":"2024-03-09T09:28:14.795677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":69,"num_read_kb":0,"num_write":510,"num_write_kb":91498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"89'614","reported_seq":773,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.546373+0000","last_change":"2024-03-09T08:52:28.573638+0000","last_active":"2024-03-09T09:28:26.546373+0000","last_peered":"2024-03-09T09:28:26.546373+0000","last_clean":"2024-03-09T09:28:26.546373+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T09:28:26.546373+0000","last_undegraded":"2024-03-09T09:28:26.546373+0000","last_fullsized":"2024-03-09T09:28:26.546373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:02.478112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":666,"num_write_kb":107635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"89'2327","reported_seq":5251,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.754491+0000","last_change":"2024-03-09T09:27:34.495452+0000","last_active":"2024-03-09T09:28:56.754491+0000","last_peered":"2024-03-09T09:28:56.754491+0000","last_clean":"2024-03-09T09:28:56.754491+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T09:28:56.754491+0000","last_undegraded":"2024-03-09T09:28:56.754491+0000","last_fullsized":"2024-03-09T09:28:56.754491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2327,"log_dups_size":0,"ondisk_log_size":2327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:06.642191+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.1575832950000002,"stat_sum":{"num_bytes":392237,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":136,"num_read_kb":53607,"num_write":1404,"num_write_kb":81678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"89'2171","reported_seq":5263,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.579428+0000","last_change":"2024-03-09T09:27:09.510565+0000","last_active":"2024-03-09T09:28:23.579428+0000","last_peered":"2024-03-09T09:28:23.579428+0000","last_clean":"2024-03-09T09:28:23.579428+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T09:28:23.579428+0000","last_undegraded":"2024-03-09T09:28:23.579428+0000","last_fullsized":"2024-03-09T09:28:23.579428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2171,"log_dups_size":0,"ondisk_log_size":2171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:04:13.602716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.38281627099999999,"stat_sum":{"num_bytes":951718,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":197,"num_read_kb":25950,"num_write":1332,"num_write_kb":45421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'345","reported_seq":514,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:58.111717+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T09:27:58.111717+0000","last_peered":"2024-03-09T09:27:58.111717+0000","last_clean":"2024-03-09T09:27:58.111717+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T09:27:58.111717+0000","last_undegraded":"2024-03-09T09:27:58.111717+0000","last_fullsized":"2024-03-09T09:27:58.111717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:35:38.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":397,"num_write_kb":78578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"89'1700","reported_seq":2916,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.642425+0000","last_change":"2024-03-09T08:52:28.575630+0000","last_active":"2024-03-09T09:28:29.642425+0000","last_peered":"2024-03-09T09:28:29.642425+0000","last_clean":"2024-03-09T09:28:29.642425+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T09:28:29.642425+0000","last_undegraded":"2024-03-09T09:28:29.642425+0000","last_fullsized":"2024-03-09T09:28:29.642425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1700,"log_dups_size":0,"ondisk_log_size":1700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:48.210921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":1117,"num_read_kb":4814,"num_write":1762,"num_write_kb":112708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"89'2107","reported_seq":5058,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.735789+0000","last_change":"2024-03-09T09:27:34.095045+0000","last_active":"2024-03-09T09:28:56.735789+0000","last_peered":"2024-03-09T09:28:56.735789+0000","last_clean":"2024-03-09T09:28:56.735789+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T09:28:56.735789+0000","last_undegraded":"2024-03-09T09:28:56.735789+0000","last_fullsized":"2024-03-09T09:28:56.735789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2107,"log_dups_size":0,"ondisk_log_size":2107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:05:04.126062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.8297685179999998,"stat_sum":{"num_bytes":268964,"num_objects":107,"num_object_clones":32,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":32,"num_read":107,"num_read_kb":24622,"num_write":1287,"num_write_kb":41444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"89'2254","reported_seq":5300,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.043906+0000","last_change":"2024-03-09T09:27:06.300751+0000","last_active":"2024-03-09T09:29:14.043906+0000","last_peered":"2024-03-09T09:29:14.043906+0000","last_clean":"2024-03-09T09:29:14.043906+0000","last_became_active":"2024-03-09T08:53:00.240844+0000","last_became_peered":"2024-03-09T08:53:00.240844+0000","last_unstale":"2024-03-09T09:29:14.043906+0000","last_undegraded":"2024-03-09T09:29:14.043906+0000","last_fullsized":"2024-03-09T09:29:14.043906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2254,"log_dups_size":0,"ondisk_log_size":2254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:21:13.503778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.78540855600000004,"stat_sum":{"num_bytes":1854558,"num_objects":126,"num_object_clones":46,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":46,"num_read":216,"num_read_kb":39239,"num_write":1332,"num_write_kb":71797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"89'1558","reported_seq":1727,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:07.567056+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T09:29:07.567056+0000","last_peered":"2024-03-09T09:29:07.567056+0000","last_clean":"2024-03-09T09:29:07.567056+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T09:29:07.567056+0000","last_undegraded":"2024-03-09T09:29:07.567056+0000","last_fullsized":"2024-03-09T09:29:07.567056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1558,"log_dups_size":0,"ondisk_log_size":1558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:30.201471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":57,"num_read_kb":0,"num_write":3420,"num_write_kb":84853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"89'672","reported_seq":833,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.478737+0000","last_change":"2024-03-09T08:52:28.586702+0000","last_active":"2024-03-09T09:28:31.478737+0000","last_peered":"2024-03-09T09:28:31.478737+0000","last_clean":"2024-03-09T09:28:31.478737+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T09:28:31.478737+0000","last_undegraded":"2024-03-09T09:28:31.478737+0000","last_fullsized":"2024-03-09T09:28:31.478737+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:09.201224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":71,"num_read_kb":0,"num_write":764,"num_write_kb":87842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"89'2333","reported_seq":5217,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.713975+0000","last_change":"2024-03-09T09:27:30.887844+0000","last_active":"2024-03-09T09:28:56.713975+0000","last_peered":"2024-03-09T09:28:56.713975+0000","last_clean":"2024-03-09T09:28:56.713975+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T09:28:56.713975+0000","last_undegraded":"2024-03-09T09:28:56.713975+0000","last_fullsized":"2024-03-09T09:28:56.713975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2333,"log_dups_size":0,"ondisk_log_size":2333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:11:46.526489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65806592200000003,"stat_sum":{"num_bytes":2259601,"num_objects":134,"num_object_clones":47,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":47,"num_read":108,"num_read_kb":22295,"num_write":1398,"num_write_kb":43486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"89'2181","reported_seq":5178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.005893+0000","last_change":"2024-03-09T09:27:09.896878+0000","last_active":"2024-03-09T09:29:14.005893+0000","last_peered":"2024-03-09T09:29:14.005893+0000","last_clean":"2024-03-09T09:29:14.005893+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T09:29:14.005893+0000","last_undegraded":"2024-03-09T09:29:14.005893+0000","last_fullsized":"2024-03-09T09:29:14.005893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2181,"log_dups_size":0,"ondisk_log_size":2181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:38:09.744644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.96663918400000004,"stat_sum":{"num_bytes":2499803,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":112,"num_read_kb":5934,"num_write":1320,"num_write_kb":25592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"89'641","reported_seq":810,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:10.237036+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T09:28:10.237036+0000","last_peered":"2024-03-09T09:28:10.237036+0000","last_clean":"2024-03-09T09:28:10.237036+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T09:28:10.237036+0000","last_undegraded":"2024-03-09T09:28:10.237036+0000","last_fullsized":"2024-03-09T09:28:10.237036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":74,"num_read_kb":0,"num_write":723,"num_write_kb":96132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"89'571","reported_seq":732,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:08.281289+0000","last_change":"2024-03-09T08:52:28.580879+0000","last_active":"2024-03-09T09:29:08.281289+0000","last_peered":"2024-03-09T09:29:08.281289+0000","last_clean":"2024-03-09T09:29:08.281289+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T09:29:08.281289+0000","last_undegraded":"2024-03-09T09:29:08.281289+0000","last_fullsized":"2024-03-09T09:29:08.281289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:19:44.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6098017,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":612,"num_write_kb":72495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"89'2273","reported_seq":5171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.939855+0000","last_change":"2024-03-09T09:27:31.029951+0000","last_active":"2024-03-09T09:28:30.939855+0000","last_peered":"2024-03-09T09:28:30.939855+0000","last_clean":"2024-03-09T09:28:30.939855+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T09:28:30.939855+0000","last_undegraded":"2024-03-09T09:28:30.939855+0000","last_fullsized":"2024-03-09T09:28:30.939855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2273,"log_dups_size":0,"ondisk_log_size":2273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:38.142661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.71811667999999995,"stat_sum":{"num_bytes":1846155,"num_objects":126,"num_object_clones":42,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":42,"num_read":111,"num_read_kb":21516,"num_write":1347,"num_write_kb":38338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"89'2281","reported_seq":5228,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.661831+0000","last_change":"2024-03-09T09:27:14.280348+0000","last_active":"2024-03-09T09:28:23.661831+0000","last_peered":"2024-03-09T09:28:23.661831+0000","last_clean":"2024-03-09T09:28:23.661831+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T09:28:23.661831+0000","last_undegraded":"2024-03-09T09:28:23.661831+0000","last_fullsized":"2024-03-09T09:28:23.661831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2281,"log_dups_size":0,"ondisk_log_size":2281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:42:30.439855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.815577539,"stat_sum":{"num_bytes":1586992,"num_objects":105,"num_object_clones":35,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":35,"num_read":97,"num_read_kb":33815,"num_write":1338,"num_write_kb":74580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"89'546","reported_seq":715,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:03.302448+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T09:28:03.302448+0000","last_peered":"2024-03-09T09:28:03.302448+0000","last_clean":"2024-03-09T09:28:03.302448+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T09:28:03.302448+0000","last_undegraded":"2024-03-09T09:28:03.302448+0000","last_fullsized":"2024-03-09T09:28:03.302448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":620,"num_write_kb":120324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"89'552","reported_seq":713,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:56.663234+0000","last_change":"2024-03-09T08:52:28.584457+0000","last_active":"2024-03-09T09:27:56.663234+0000","last_peered":"2024-03-09T09:27:56.663234+0000","last_clean":"2024-03-09T09:27:56.663234+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T09:27:56.663234+0000","last_undegraded":"2024-03-09T09:27:56.663234+0000","last_fullsized":"2024-03-09T09:27:56.663234+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:45.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":563,"num_write_kb":103368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"89'2260","reported_seq":5160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.938153+0000","last_change":"2024-03-09T09:27:31.026391+0000","last_active":"2024-03-09T09:28:30.938153+0000","last_peered":"2024-03-09T09:28:30.938153+0000","last_clean":"2024-03-09T09:28:30.938153+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T09:28:30.938153+0000","last_undegraded":"2024-03-09T09:28:30.938153+0000","last_fullsized":"2024-03-09T09:28:30.938153+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2260,"log_dups_size":0,"ondisk_log_size":2260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:49:57.313638+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.71599741500000003,"stat_sum":{"num_bytes":1873672,"num_objects":101,"num_object_clones":34,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":34,"num_read":103,"num_read_kb":19624,"num_write":1351,"num_write_kb":43427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"89'2260","reported_seq":5236,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.039799+0000","last_change":"2024-03-09T09:27:08.957379+0000","last_active":"2024-03-09T09:29:14.039799+0000","last_peered":"2024-03-09T09:29:14.039799+0000","last_clean":"2024-03-09T09:29:14.039799+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T09:29:14.039799+0000","last_undegraded":"2024-03-09T09:29:14.039799+0000","last_fullsized":"2024-03-09T09:29:14.039799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2260,"log_dups_size":0,"ondisk_log_size":2260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:57:34.155091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.4919903310000002,"stat_sum":{"num_bytes":2681803,"num_objects":128,"num_object_clones":43,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":43,"num_read":104,"num_read_kb":25781,"num_write":1386,"num_write_kb":55300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"89'736","reported_seq":905,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:03.425815+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T09:28:03.425815+0000","last_peered":"2024-03-09T09:28:03.425815+0000","last_clean":"2024-03-09T09:28:03.425815+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T09:28:03.425815+0000","last_undegraded":"2024-03-09T09:28:03.425815+0000","last_fullsized":"2024-03-09T09:28:03.425815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":62,"num_read_kb":0,"num_write":830,"num_write_kb":103820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'561","reported_seq":722,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.554450+0000","last_change":"2024-03-09T08:52:28.578096+0000","last_active":"2024-03-09T09:27:57.554450+0000","last_peered":"2024-03-09T09:27:57.554450+0000","last_clean":"2024-03-09T09:27:57.554450+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T09:27:57.554450+0000","last_undegraded":"2024-03-09T09:27:57.554450+0000","last_fullsized":"2024-03-09T09:27:57.554450+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:42.645517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":52,"num_read_kb":0,"num_write":616,"num_write_kb":119874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"89'2163","reported_seq":5096,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.976905+0000","last_change":"2024-03-09T09:27:30.887413+0000","last_active":"2024-03-09T09:28:30.976905+0000","last_peered":"2024-03-09T09:28:30.976905+0000","last_clean":"2024-03-09T09:28:30.976905+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T09:28:30.976905+0000","last_undegraded":"2024-03-09T09:28:30.976905+0000","last_fullsized":"2024-03-09T09:28:30.976905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2163,"log_dups_size":0,"ondisk_log_size":2163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:41.847948+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.63049624100000001,"stat_sum":{"num_bytes":833172,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":114,"num_read_kb":17857,"num_write":1302,"num_write_kb":44488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"89'2250","reported_seq":5226,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.030973+0000","last_change":"2024-03-09T09:27:10.015883+0000","last_active":"2024-03-09T09:29:14.030973+0000","last_peered":"2024-03-09T09:29:14.030973+0000","last_clean":"2024-03-09T09:29:14.030973+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T09:29:14.030973+0000","last_undegraded":"2024-03-09T09:29:14.030973+0000","last_fullsized":"2024-03-09T09:29:14.030973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2250,"log_dups_size":0,"ondisk_log_size":2250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:23:02.726704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.18447864,"stat_sum":{"num_bytes":1122276,"num_objects":117,"num_object_clones":43,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":43,"num_read":101,"num_read_kb":18409,"num_write":1364,"num_write_kb":39400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"89'1830","reported_seq":3014,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.251658+0000","last_change":"2024-03-09T08:52:17.271847+0000","last_active":"2024-03-09T09:28:16.251658+0000","last_peered":"2024-03-09T09:28:16.251658+0000","last_clean":"2024-03-09T09:28:16.251658+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T09:28:16.251658+0000","last_undegraded":"2024-03-09T09:28:16.251658+0000","last_fullsized":"2024-03-09T09:28:16.251658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1830,"log_dups_size":0,"ondisk_log_size":1830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:27.418674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":1037,"num_read_kb":4764,"num_write":1840,"num_write_kb":108217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"89'404","reported_seq":564,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.641753+0000","last_change":"2024-03-09T08:52:28.588636+0000","last_active":"2024-03-09T09:28:29.641753+0000","last_peered":"2024-03-09T09:28:29.641753+0000","last_clean":"2024-03-09T09:28:29.641753+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T09:28:29.641753+0000","last_undegraded":"2024-03-09T09:28:29.641753+0000","last_fullsized":"2024-03-09T09:28:29.641753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:21.601182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":44,"num_read_kb":0,"num_write":445,"num_write_kb":78896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"89'2178","reported_seq":5120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.016676+0000","last_change":"2024-03-09T09:27:34.760081+0000","last_active":"2024-03-09T09:28:31.016676+0000","last_peered":"2024-03-09T09:28:31.016676+0000","last_clean":"2024-03-09T09:28:31.016676+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T09:28:31.016676+0000","last_undegraded":"2024-03-09T09:28:31.016676+0000","last_fullsized":"2024-03-09T09:28:31.016676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2178,"log_dups_size":0,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:30:07.164361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4843044489999997,"stat_sum":{"num_bytes":1636574,"num_objects":123,"num_object_clones":44,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":44,"num_read":134,"num_read_kb":27719,"num_write":1302,"num_write_kb":59461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'2121","reported_seq":5114,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.622622+0000","last_change":"2024-03-09T09:27:09.413913+0000","last_active":"2024-03-09T09:28:23.622622+0000","last_peered":"2024-03-09T09:28:23.622622+0000","last_clean":"2024-03-09T09:28:23.622622+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T09:28:23.622622+0000","last_undegraded":"2024-03-09T09:28:23.622622+0000","last_fullsized":"2024-03-09T09:28:23.622622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2121,"log_dups_size":0,"ondisk_log_size":2121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:36:18.637176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.289202336,"stat_sum":{"num_bytes":424328,"num_objects":113,"num_object_clones":35,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":35,"num_read":94,"num_read_kb":17266,"num_write":1284,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"89'579","reported_seq":749,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:03.425304+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T09:28:03.425304+0000","last_peered":"2024-03-09T09:28:03.425304+0000","last_clean":"2024-03-09T09:28:03.425304+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T09:28:03.425304+0000","last_undegraded":"2024-03-09T09:28:03.425304+0000","last_fullsized":"2024-03-09T09:28:03.425304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":598,"num_write_kb":82993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"89'1592","reported_seq":1752,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:48.494834+0000","last_change":"2024-03-09T08:52:28.578599+0000","last_active":"2024-03-09T09:28:48.494834+0000","last_peered":"2024-03-09T09:28:48.494834+0000","last_clean":"2024-03-09T09:28:48.494834+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T09:28:48.494834+0000","last_undegraded":"2024-03-09T09:28:48.494834+0000","last_fullsized":"2024-03-09T09:28:48.494834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1592,"log_dups_size":0,"ondisk_log_size":1592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:52:31.328248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":3473,"num_write_kb":84746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"89'2230","reported_seq":5161,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.024843+0000","last_change":"2024-03-09T09:27:34.687418+0000","last_active":"2024-03-09T09:28:31.024843+0000","last_peered":"2024-03-09T09:28:31.024843+0000","last_clean":"2024-03-09T09:28:31.024843+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T09:28:31.024843+0000","last_undegraded":"2024-03-09T09:28:31.024843+0000","last_fullsized":"2024-03-09T09:28:31.024843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2230,"log_dups_size":0,"ondisk_log_size":2230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:18:43.296322+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4503168610000001,"stat_sum":{"num_bytes":1665642,"num_objects":121,"num_object_clones":41,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":41,"num_read":126,"num_read_kb":42837,"num_write":1341,"num_write_kb":67584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'2305","reported_seq":5266,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.607547+0000","last_change":"2024-03-09T09:27:12.261576+0000","last_active":"2024-03-09T09:28:23.607547+0000","last_peered":"2024-03-09T09:28:23.607547+0000","last_clean":"2024-03-09T09:28:23.607547+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T09:28:23.607547+0000","last_undegraded":"2024-03-09T09:28:23.607547+0000","last_fullsized":"2024-03-09T09:28:23.607547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2305,"log_dups_size":0,"ondisk_log_size":2305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:07:46.894530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.56599842099999997,"stat_sum":{"num_bytes":1847778,"num_objects":137,"num_object_clones":49,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":49,"num_read":115,"num_read_kb":13750,"num_write":1371,"num_write_kb":55061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'626","reported_seq":795,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:11.134820+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T09:28:11.134820+0000","last_peered":"2024-03-09T09:28:11.134820+0000","last_clean":"2024-03-09T09:28:11.134820+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T09:28:11.134820+0000","last_undegraded":"2024-03-09T09:28:11.134820+0000","last_fullsized":"2024-03-09T09:28:11.134820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":698,"num_write_kb":70663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":326,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:40.174178+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T09:27:40.174178+0000","last_peered":"2024-03-09T09:27:40.174178+0000","last_clean":"2024-03-09T09:27:40.174178+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T09:27:40.174178+0000","last_undegraded":"2024-03-09T09:27:40.174178+0000","last_fullsized":"2024-03-09T09:27:40.174178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'680","reported_seq":841,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:02.565362+0000","last_change":"2024-03-09T08:52:28.577973+0000","last_active":"2024-03-09T09:28:02.565362+0000","last_peered":"2024-03-09T09:28:02.565362+0000","last_clean":"2024-03-09T09:28:02.565362+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T09:28:02.565362+0000","last_undegraded":"2024-03-09T09:28:02.565362+0000","last_fullsized":"2024-03-09T09:28:02.565362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:13.291622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":743,"num_write_kb":95766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"89'2243","reported_seq":5171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.944320+0000","last_change":"2024-03-09T09:27:30.919984+0000","last_active":"2024-03-09T09:28:30.944320+0000","last_peered":"2024-03-09T09:28:30.944320+0000","last_clean":"2024-03-09T09:28:30.944320+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T09:28:30.944320+0000","last_undegraded":"2024-03-09T09:28:30.944320+0000","last_fullsized":"2024-03-09T09:28:30.944320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2243,"log_dups_size":0,"ondisk_log_size":2243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:42:58.142985+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.66264078000000004,"stat_sum":{"num_bytes":792938,"num_objects":106,"num_object_clones":38,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":38,"num_read":125,"num_read_kb":17823,"num_write":1366,"num_write_kb":27629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'2242","reported_seq":5186,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.060199+0000","last_change":"2024-03-09T09:27:06.276304+0000","last_active":"2024-03-09T09:29:14.060199+0000","last_peered":"2024-03-09T09:29:14.060199+0000","last_clean":"2024-03-09T09:29:14.060199+0000","last_became_active":"2024-03-09T08:53:00.437859+0000","last_became_peered":"2024-03-09T08:53:00.437859+0000","last_unstale":"2024-03-09T09:29:14.060199+0000","last_undegraded":"2024-03-09T09:29:14.060199+0000","last_fullsized":"2024-03-09T09:29:14.060199+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2242,"log_dups_size":0,"ondisk_log_size":2242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:07:36.832520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.79260830400000004,"stat_sum":{"num_bytes":1456716,"num_objects":115,"num_object_clones":40,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":40,"num_read":109,"num_read_kb":18518,"num_write":1301,"num_write_kb":37816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"89'501","reported_seq":669,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.901581+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T09:28:14.901581+0000","last_peered":"2024-03-09T09:28:14.901581+0000","last_clean":"2024-03-09T09:28:14.901581+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T09:28:14.901581+0000","last_undegraded":"2024-03-09T09:28:14.901581+0000","last_fullsized":"2024-03-09T09:28:14.901581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:27.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":39,"num_read_kb":0,"num_write":538,"num_write_kb":99417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"89'703","reported_seq":863,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.545223+0000","last_change":"2024-03-09T08:52:28.589278+0000","last_active":"2024-03-09T09:28:26.545223+0000","last_peered":"2024-03-09T09:28:26.545223+0000","last_clean":"2024-03-09T09:28:26.545223+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T09:28:26.545223+0000","last_undegraded":"2024-03-09T09:28:26.545223+0000","last_fullsized":"2024-03-09T09:28:26.545223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:16:26.449275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":780,"num_write_kb":116155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"89'2189","reported_seq":5136,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.910057+0000","last_change":"2024-03-09T09:27:30.904584+0000","last_active":"2024-03-09T09:28:30.910057+0000","last_peered":"2024-03-09T09:28:30.910057+0000","last_clean":"2024-03-09T09:28:30.910057+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T09:28:30.910057+0000","last_undegraded":"2024-03-09T09:28:30.910057+0000","last_fullsized":"2024-03-09T09:28:30.910057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2189,"log_dups_size":0,"ondisk_log_size":2189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:42:21.944851+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.647865472,"stat_sum":{"num_bytes":999257,"num_objects":92,"num_object_clones":32,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":32,"num_read":116,"num_read_kb":42999,"num_write":1346,"num_write_kb":55653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"89'2182","reported_seq":5176,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.600289+0000","last_change":"2024-03-09T09:27:14.415883+0000","last_active":"2024-03-09T09:28:23.600289+0000","last_peered":"2024-03-09T09:28:23.600289+0000","last_clean":"2024-03-09T09:28:23.600289+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T09:28:23.600289+0000","last_undegraded":"2024-03-09T09:28:23.600289+0000","last_fullsized":"2024-03-09T09:28:23.600289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2182,"log_dups_size":0,"ondisk_log_size":2182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:34:33.950832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.951760857,"stat_sum":{"num_bytes":1333229,"num_objects":97,"num_object_clones":33,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":33,"num_read":100,"num_read_kb":17450,"num_write":1313,"num_write_kb":41729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"89'845","reported_seq":1013,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.817246+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T09:28:14.817246+0000","last_peered":"2024-03-09T09:28:14.817246+0000","last_clean":"2024-03-09T09:28:14.817246+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T09:28:14.817246+0000","last_undegraded":"2024-03-09T09:28:14.817246+0000","last_fullsized":"2024-03-09T09:28:14.817246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:26:35.574903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971641,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":912,"num_write_kb":111865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"89'796","reported_seq":956,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:06.427393+0000","last_change":"2024-03-09T08:52:28.583181+0000","last_active":"2024-03-09T09:28:06.427393+0000","last_peered":"2024-03-09T09:28:06.427393+0000","last_clean":"2024-03-09T09:28:06.427393+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T09:28:06.427393+0000","last_undegraded":"2024-03-09T09:28:06.427393+0000","last_fullsized":"2024-03-09T09:28:06.427393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:58:50.061324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":51,"num_read_kb":0,"num_write":872,"num_write_kb":99652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"89'2234","reported_seq":5129,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.000542+0000","last_change":"2024-03-09T09:27:34.640947+0000","last_active":"2024-03-09T09:28:31.000542+0000","last_peered":"2024-03-09T09:28:31.000542+0000","last_clean":"2024-03-09T09:28:31.000542+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T09:28:31.000542+0000","last_undegraded":"2024-03-09T09:28:31.000542+0000","last_fullsized":"2024-03-09T09:28:31.000542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:51:18.312332+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4098675719999996,"stat_sum":{"num_bytes":603203,"num_objects":100,"num_object_clones":32,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":32,"num_read":95,"num_read_kb":13936,"num_write":1338,"num_write_kb":41727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"89'2195","reported_seq":5188,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.643732+0000","last_change":"2024-03-09T09:27:13.656604+0000","last_active":"2024-03-09T09:28:23.643732+0000","last_peered":"2024-03-09T09:28:23.643732+0000","last_clean":"2024-03-09T09:28:23.643732+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T09:28:23.643732+0000","last_undegraded":"2024-03-09T09:28:23.643732+0000","last_fullsized":"2024-03-09T09:28:23.643732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2195,"log_dups_size":0,"ondisk_log_size":2195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:53:35.952871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.91280589400000001,"stat_sum":{"num_bytes":4608718,"num_objects":119,"num_object_clones":37,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":37,"num_read":119,"num_read_kb":26183,"num_write":1317,"num_write_kb":37436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'672","reported_seq":840,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:17.301970+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T09:28:17.301970+0000","last_peered":"2024-03-09T09:28:17.301970+0000","last_clean":"2024-03-09T09:28:17.301970+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T09:28:17.301970+0000","last_undegraded":"2024-03-09T09:28:17.301970+0000","last_fullsized":"2024-03-09T09:28:17.301970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:16.138216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":724,"num_write_kb":103814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"89'713","reported_seq":873,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:38.294529+0000","last_change":"2024-03-09T08:52:28.587637+0000","last_active":"2024-03-09T09:28:38.294529+0000","last_peered":"2024-03-09T09:28:38.294529+0000","last_clean":"2024-03-09T09:28:38.294529+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T09:28:38.294529+0000","last_undegraded":"2024-03-09T09:28:38.294529+0000","last_fullsized":"2024-03-09T09:28:38.294529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:31:32.362061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":59,"num_read_kb":0,"num_write":793,"num_write_kb":95673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"89'2354","reported_seq":5251,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.737741+0000","last_change":"2024-03-09T09:27:30.891372+0000","last_active":"2024-03-09T09:28:56.737741+0000","last_peered":"2024-03-09T09:28:56.737741+0000","last_clean":"2024-03-09T09:28:56.737741+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T09:28:56.737741+0000","last_undegraded":"2024-03-09T09:28:56.737741+0000","last_fullsized":"2024-03-09T09:28:56.737741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2354,"log_dups_size":0,"ondisk_log_size":2354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:45:35.757078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.634097468,"stat_sum":{"num_bytes":572008,"num_objects":135,"num_object_clones":47,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":47,"num_read":117,"num_read_kb":9742,"num_write":1431,"num_write_kb":32918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"89'2237","reported_seq":5216,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.532142+0000","last_change":"2024-03-09T09:27:13.609914+0000","last_active":"2024-03-09T09:28:23.532142+0000","last_peered":"2024-03-09T09:28:23.532142+0000","last_clean":"2024-03-09T09:28:23.532142+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T09:28:23.532142+0000","last_undegraded":"2024-03-09T09:28:23.532142+0000","last_fullsized":"2024-03-09T09:28:23.532142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2237,"log_dups_size":0,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:55:21.022399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.81694827699999994,"stat_sum":{"num_bytes":851628,"num_objects":105,"num_object_clones":33,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":33,"num_read":114,"num_read_kb":26314,"num_write":1339,"num_write_kb":37517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"89'520","reported_seq":687,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:58.112217+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T09:27:58.112217+0000","last_peered":"2024-03-09T09:27:58.112217+0000","last_clean":"2024-03-09T09:27:58.112217+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T09:27:58.112217+0000","last_undegraded":"2024-03-09T09:27:58.112217+0000","last_fullsized":"2024-03-09T09:27:58.112217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:10:34.698235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554454,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":604,"num_write_kb":91576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"89'683","reported_seq":843,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.545188+0000","last_change":"2024-03-09T08:52:28.582308+0000","last_active":"2024-03-09T09:28:26.545188+0000","last_peered":"2024-03-09T09:28:26.545188+0000","last_clean":"2024-03-09T09:28:26.545188+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T09:28:26.545188+0000","last_undegraded":"2024-03-09T09:28:26.545188+0000","last_fullsized":"2024-03-09T09:28:26.545188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:04.918893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":783,"num_write_kb":95503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"89'2300","reported_seq":5191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:48.600809+0000","last_change":"2024-03-09T09:27:34.538055+0000","last_active":"2024-03-09T09:28:48.600809+0000","last_peered":"2024-03-09T09:28:48.600809+0000","last_clean":"2024-03-09T09:28:48.600809+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T09:28:48.600809+0000","last_undegraded":"2024-03-09T09:28:48.600809+0000","last_fullsized":"2024-03-09T09:28:48.600809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2300,"log_dups_size":0,"ondisk_log_size":2300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:30:20.481016+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.1951373749999998,"stat_sum":{"num_bytes":1358361,"num_objects":133,"num_object_clones":45,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":45,"num_read":125,"num_read_kb":25752,"num_write":1343,"num_write_kb":45310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"89'2282","reported_seq":5254,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.591429+0000","last_change":"2024-03-09T09:27:13.043792+0000","last_active":"2024-03-09T09:28:23.591429+0000","last_peered":"2024-03-09T09:28:23.591429+0000","last_clean":"2024-03-09T09:28:23.591429+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T09:28:23.591429+0000","last_undegraded":"2024-03-09T09:28:23.591429+0000","last_fullsized":"2024-03-09T09:28:23.591429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2282,"log_dups_size":0,"ondisk_log_size":2282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:24:58.260830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30155770100000001,"stat_sum":{"num_bytes":730111,"num_objects":106,"num_object_clones":35,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":35,"num_read":106,"num_read_kb":32982,"num_write":1381,"num_write_kb":56548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'750","reported_seq":917,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.253028+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T09:28:16.253028+0000","last_peered":"2024-03-09T09:28:16.253028+0000","last_clean":"2024-03-09T09:28:16.253028+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T09:28:16.253028+0000","last_undegraded":"2024-03-09T09:28:16.253028+0000","last_fullsized":"2024-03-09T09:28:16.253028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:31.859924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":74,"num_read_kb":0,"num_write":870,"num_write_kb":87759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"89'602","reported_seq":761,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.544148+0000","last_change":"2024-03-09T08:52:28.575975+0000","last_active":"2024-03-09T09:28:26.544148+0000","last_peered":"2024-03-09T09:28:26.544148+0000","last_clean":"2024-03-09T09:28:26.544148+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T09:28:26.544148+0000","last_undegraded":"2024-03-09T09:28:26.544148+0000","last_fullsized":"2024-03-09T09:28:26.544148+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:51:24.405316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":669,"num_write_kb":112438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"89'2182","reported_seq":5113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.753962+0000","last_change":"2024-03-09T09:27:34.711852+0000","last_active":"2024-03-09T09:28:56.753962+0000","last_peered":"2024-03-09T09:28:56.753962+0000","last_clean":"2024-03-09T09:28:56.753962+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T09:28:56.753962+0000","last_undegraded":"2024-03-09T09:28:56.753962+0000","last_fullsized":"2024-03-09T09:28:56.753962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2182,"log_dups_size":0,"ondisk_log_size":2182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:07:14.254525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4375742420000002,"stat_sum":{"num_bytes":1378177,"num_objects":100,"num_object_clones":26,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":26,"num_read":107,"num_read_kb":18463,"num_write":1316,"num_write_kb":41976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"89'2181","reported_seq":5172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.034483+0000","last_change":"2024-03-09T09:27:07.994121+0000","last_active":"2024-03-09T09:29:14.034483+0000","last_peered":"2024-03-09T09:29:14.034483+0000","last_clean":"2024-03-09T09:29:14.034483+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T09:29:14.034483+0000","last_undegraded":"2024-03-09T09:29:14.034483+0000","last_fullsized":"2024-03-09T09:29:14.034483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2181,"log_dups_size":0,"ondisk_log_size":2181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:07:11.820222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.3444653409999998,"stat_sum":{"num_bytes":3375625,"num_objects":103,"num_object_clones":31,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":31,"num_read":106,"num_read_kb":36185,"num_write":1311,"num_write_kb":52081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"89'386","reported_seq":553,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.986701+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T09:27:57.986701+0000","last_peered":"2024-03-09T09:27:57.986701+0000","last_clean":"2024-03-09T09:27:57.986701+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T09:27:57.986701+0000","last_undegraded":"2024-03-09T09:27:57.986701+0000","last_fullsized":"2024-03-09T09:27:57.986701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:07:29.577461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":412,"num_write_kb":99378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"89'560","reported_seq":959,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.643843+0000","last_change":"2024-03-09T08:52:28.588312+0000","last_active":"2024-03-09T09:28:29.643843+0000","last_peered":"2024-03-09T09:28:29.643843+0000","last_clean":"2024-03-09T09:28:29.643843+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T09:28:29.643843+0000","last_undegraded":"2024-03-09T09:28:29.643843+0000","last_fullsized":"2024-03-09T09:28:29.643843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:14:52.895618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007622,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":290,"num_read_kb":993,"num_write":609,"num_write_kb":88247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"89'2213","reported_seq":5119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:43.680729+0000","last_change":"2024-03-09T09:27:34.708914+0000","last_active":"2024-03-09T09:28:43.680729+0000","last_peered":"2024-03-09T09:28:43.680729+0000","last_clean":"2024-03-09T09:28:43.680729+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T09:28:43.680729+0000","last_undegraded":"2024-03-09T09:28:43.680729+0000","last_fullsized":"2024-03-09T09:28:43.680729+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2213,"log_dups_size":0,"ondisk_log_size":2213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:43:47.385372+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4357263939999996,"stat_sum":{"num_bytes":1364696,"num_objects":131,"num_object_clones":50,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":50,"num_read":112,"num_read_kb":14161,"num_write":1321,"num_write_kb":29407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'2335","reported_seq":5294,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.571804+0000","last_change":"2024-03-09T09:27:13.390531+0000","last_active":"2024-03-09T09:28:23.571804+0000","last_peered":"2024-03-09T09:28:23.571804+0000","last_clean":"2024-03-09T09:28:23.571804+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T09:28:23.571804+0000","last_undegraded":"2024-03-09T09:28:23.571804+0000","last_fullsized":"2024-03-09T09:28:23.571804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2335,"log_dups_size":0,"ondisk_log_size":2335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:00:16.543112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64721804299999997,"stat_sum":{"num_bytes":2530176,"num_objects":100,"num_object_clones":36,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":36,"num_read":119,"num_read_kb":17537,"num_write":1383,"num_write_kb":31574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'267","reported_seq":434,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:40.238636+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T09:27:40.238636+0000","last_peered":"2024-03-09T09:27:40.238636+0000","last_clean":"2024-03-09T09:27:40.238636+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T09:27:40.238636+0000","last_undegraded":"2024-03-09T09:27:40.238636+0000","last_fullsized":"2024-03-09T09:27:40.238636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":313,"num_write_kb":62737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"89'829","reported_seq":987,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:48.273880+0000","last_change":"2024-03-09T08:52:28.585562+0000","last_active":"2024-03-09T09:28:48.273880+0000","last_peered":"2024-03-09T09:28:48.273880+0000","last_clean":"2024-03-09T09:28:48.273880+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T09:28:48.273880+0000","last_undegraded":"2024-03-09T09:28:48.273880+0000","last_fullsized":"2024-03-09T09:28:48.273880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:28:50.544795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":879,"num_write_kb":124191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"89'2220","reported_seq":5127,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.752544+0000","last_change":"2024-03-09T09:27:34.550507+0000","last_active":"2024-03-09T09:28:56.752544+0000","last_peered":"2024-03-09T09:28:56.752544+0000","last_clean":"2024-03-09T09:28:56.752544+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T09:28:56.752544+0000","last_undegraded":"2024-03-09T09:28:56.752544+0000","last_fullsized":"2024-03-09T09:28:56.752544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2220,"log_dups_size":0,"ondisk_log_size":2220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:36:18.970938+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.2113280319999999,"stat_sum":{"num_bytes":5678953,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":106,"num_read_kb":14364,"num_write":1330,"num_write_kb":47101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"89'2145","reported_seq":5151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.591372+0000","last_change":"2024-03-09T09:27:09.238850+0000","last_active":"2024-03-09T09:28:23.591372+0000","last_peered":"2024-03-09T09:28:23.591372+0000","last_clean":"2024-03-09T09:28:23.591372+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T09:28:23.591372+0000","last_undegraded":"2024-03-09T09:28:23.591372+0000","last_fullsized":"2024-03-09T09:28:23.591372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2145,"log_dups_size":0,"ondisk_log_size":2145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:13:15.580902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.096588679999999996,"stat_sum":{"num_bytes":1266796,"num_objects":124,"num_object_clones":39,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":39,"num_read":107,"num_read_kb":17887,"num_write":1323,"num_write_kb":41057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"89'594","reported_seq":761,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.117171+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T09:27:57.117171+0000","last_peered":"2024-03-09T09:27:57.117171+0000","last_clean":"2024-03-09T09:27:57.117171+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T09:27:57.117171+0000","last_undegraded":"2024-03-09T09:27:57.117171+0000","last_fullsized":"2024-03-09T09:27:57.117171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:22:05.504557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":687,"num_write_kb":95812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"89'1176","reported_seq":1335,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.644757+0000","last_change":"2024-03-09T08:52:28.585697+0000","last_active":"2024-03-09T09:28:29.644757+0000","last_peered":"2024-03-09T09:28:29.644757+0000","last_clean":"2024-03-09T09:28:29.644757+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T09:28:29.644757+0000","last_undegraded":"2024-03-09T09:28:29.644757+0000","last_fullsized":"2024-03-09T09:28:29.644757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1176,"log_dups_size":0,"ondisk_log_size":1176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:50:23.095008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777306,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":1250,"num_write_kb":83773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"89'2360","reported_seq":5262,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.752894+0000","last_change":"2024-03-09T09:27:34.539439+0000","last_active":"2024-03-09T09:28:56.752894+0000","last_peered":"2024-03-09T09:28:56.752894+0000","last_clean":"2024-03-09T09:28:56.752894+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T09:28:56.752894+0000","last_undegraded":"2024-03-09T09:28:56.752894+0000","last_fullsized":"2024-03-09T09:28:56.752894+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2360,"log_dups_size":0,"ondisk_log_size":2360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:04:32.627956+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2009582749999996,"stat_sum":{"num_bytes":273727,"num_objects":96,"num_object_clones":34,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":34,"num_read":130,"num_read_kb":37554,"num_write":1394,"num_write_kb":63694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"89'2218","reported_seq":5182,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.570650+0000","last_change":"2024-03-09T09:27:12.934974+0000","last_active":"2024-03-09T09:28:23.570650+0000","last_peered":"2024-03-09T09:28:23.570650+0000","last_clean":"2024-03-09T09:28:23.570650+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T09:28:23.570650+0000","last_undegraded":"2024-03-09T09:28:23.570650+0000","last_fullsized":"2024-03-09T09:28:23.570650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2218,"log_dups_size":0,"ondisk_log_size":2218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:00:09.930894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19064995900000001,"stat_sum":{"num_bytes":1055946,"num_objects":133,"num_object_clones":51,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":51,"num_read":117,"num_read_kb":18332,"num_write":1305,"num_write_kb":38006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"89'1040","reported_seq":1207,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.156049+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T09:28:23.156049+0000","last_peered":"2024-03-09T09:28:23.156049+0000","last_clean":"2024-03-09T09:28:23.156049+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T09:28:23.156049+0000","last_undegraded":"2024-03-09T09:28:23.156049+0000","last_fullsized":"2024-03-09T09:28:23.156049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1040,"log_dups_size":0,"ondisk_log_size":1040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:01.724043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":58,"num_read_kb":0,"num_write":1092,"num_write_kb":124321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"89'297","reported_seq":456,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.642610+0000","last_change":"2024-03-09T08:52:28.583685+0000","last_active":"2024-03-09T09:28:29.642610+0000","last_peered":"2024-03-09T09:28:29.642610+0000","last_clean":"2024-03-09T09:28:29.642610+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T09:28:29.642610+0000","last_undegraded":"2024-03-09T09:28:29.642610+0000","last_fullsized":"2024-03-09T09:28:29.642610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:58:00.899620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":328,"num_write_kb":62283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"89'2169","reported_seq":5098,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.750241+0000","last_change":"2024-03-09T09:27:32.064467+0000","last_active":"2024-03-09T09:28:56.750241+0000","last_peered":"2024-03-09T09:28:56.750241+0000","last_clean":"2024-03-09T09:28:56.750241+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T09:28:56.750241+0000","last_undegraded":"2024-03-09T09:28:56.750241+0000","last_fullsized":"2024-03-09T09:28:56.750241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2169,"log_dups_size":0,"ondisk_log_size":2169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:45:48.687413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.695386042,"stat_sum":{"num_bytes":3385389,"num_objects":130,"num_object_clones":42,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":42,"num_read":111,"num_read_kb":40727,"num_write":1306,"num_write_kb":67855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"89'2237","reported_seq":5201,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.072301+0000","last_change":"2024-03-09T09:27:08.841098+0000","last_active":"2024-03-09T09:29:14.072301+0000","last_peered":"2024-03-09T09:29:14.072301+0000","last_clean":"2024-03-09T09:29:14.072301+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T09:29:14.072301+0000","last_undegraded":"2024-03-09T09:29:14.072301+0000","last_fullsized":"2024-03-09T09:29:14.072301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2237,"log_dups_size":0,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:41:56.840106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.3666159750000002,"stat_sum":{"num_bytes":799596,"num_objects":94,"num_object_clones":30,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":30,"num_read":103,"num_read_kb":34868,"num_write":1326,"num_write_kb":46782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'1298","reported_seq":1465,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:18.712625+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T09:28:18.712625+0000","last_peered":"2024-03-09T09:28:18.712625+0000","last_clean":"2024-03-09T09:28:18.712625+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T09:28:18.712625+0000","last_undegraded":"2024-03-09T09:28:18.712625+0000","last_fullsized":"2024-03-09T09:28:18.712625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1298,"log_dups_size":0,"ondisk_log_size":1298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:41:10.578669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":80,"num_read_kb":0,"num_write":1400,"num_write_kb":80149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"89'411","reported_seq":570,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:08.402304+0000","last_change":"2024-03-09T08:52:28.568794+0000","last_active":"2024-03-09T09:28:08.402304+0000","last_peered":"2024-03-09T09:28:08.402304+0000","last_clean":"2024-03-09T09:28:08.402304+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T09:28:08.402304+0000","last_undegraded":"2024-03-09T09:28:08.402304+0000","last_fullsized":"2024-03-09T09:28:08.402304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:44:21.721511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":462,"num_write_kb":95092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"89'2267","reported_seq":5188,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.727032+0000","last_change":"2024-03-09T09:27:30.874038+0000","last_active":"2024-03-09T09:28:56.727032+0000","last_peered":"2024-03-09T09:28:56.727032+0000","last_clean":"2024-03-09T09:28:56.727032+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T09:28:56.727032+0000","last_undegraded":"2024-03-09T09:28:56.727032+0000","last_fullsized":"2024-03-09T09:28:56.727032+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2267,"log_dups_size":0,"ondisk_log_size":2267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:11:48.337332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64490501099999997,"stat_sum":{"num_bytes":1735737,"num_objects":103,"num_object_clones":32,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":32,"num_read":113,"num_read_kb":37575,"num_write":1379,"num_write_kb":57434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"89'2288","reported_seq":5239,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.590144+0000","last_change":"2024-03-09T09:27:09.683757+0000","last_active":"2024-03-09T09:28:23.590144+0000","last_peered":"2024-03-09T09:28:23.590144+0000","last_clean":"2024-03-09T09:28:23.590144+0000","last_became_active":"2024-03-09T08:53:00.254589+0000","last_became_peered":"2024-03-09T08:53:00.254589+0000","last_unstale":"2024-03-09T09:28:23.590144+0000","last_undegraded":"2024-03-09T09:28:23.590144+0000","last_fullsized":"2024-03-09T09:28:23.590144+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2288,"log_dups_size":0,"ondisk_log_size":2288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:40:21.450942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.471373141,"stat_sum":{"num_bytes":5513596,"num_objects":123,"num_object_clones":39,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":39,"num_read":114,"num_read_kb":29387,"num_write":1331,"num_write_kb":57801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'590","reported_seq":757,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.986880+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T09:27:57.986880+0000","last_peered":"2024-03-09T09:27:57.986880+0000","last_clean":"2024-03-09T09:27:57.986880+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T09:27:57.986880+0000","last_undegraded":"2024-03-09T09:27:57.986880+0000","last_fullsized":"2024-03-09T09:27:57.986880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":598,"num_write_kb":111422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"89'729","reported_seq":887,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.544483+0000","last_change":"2024-03-09T08:52:28.583697+0000","last_active":"2024-03-09T09:28:26.544483+0000","last_peered":"2024-03-09T09:28:26.544483+0000","last_clean":"2024-03-09T09:28:26.544483+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T09:28:26.544483+0000","last_undegraded":"2024-03-09T09:28:26.544483+0000","last_fullsized":"2024-03-09T09:28:26.544483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:41.552156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":81,"num_read_kb":0,"num_write":849,"num_write_kb":91941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"89'2523","reported_seq":5327,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.740822+0000","last_change":"2024-03-09T09:27:34.669817+0000","last_active":"2024-03-09T09:28:56.740822+0000","last_peered":"2024-03-09T09:28:56.740822+0000","last_clean":"2024-03-09T09:28:56.740822+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T09:28:56.740822+0000","last_undegraded":"2024-03-09T09:28:56.740822+0000","last_fullsized":"2024-03-09T09:28:56.740822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2523,"log_dups_size":0,"ondisk_log_size":2523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:42:51.478391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4377723019999999,"stat_sum":{"num_bytes":1160060,"num_objects":111,"num_object_clones":35,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":35,"num_read":113,"num_read_kb":9969,"num_write":1381,"num_write_kb":17733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"89'2163","reported_seq":5159,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.663315+0000","last_change":"2024-03-09T09:27:14.293488+0000","last_active":"2024-03-09T09:28:23.663315+0000","last_peered":"2024-03-09T09:28:23.663315+0000","last_clean":"2024-03-09T09:28:23.663315+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T09:28:23.663315+0000","last_undegraded":"2024-03-09T09:28:23.663315+0000","last_fullsized":"2024-03-09T09:28:23.663315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2163,"log_dups_size":0,"ondisk_log_size":2163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:12:46.882947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.82863014,"stat_sum":{"num_bytes":1951923,"num_objects":107,"num_object_clones":29,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":29,"num_read":96,"num_read_kb":37066,"num_write":1331,"num_write_kb":53456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"89'554","reported_seq":950,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:38.153367+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T09:28:38.153367+0000","last_peered":"2024-03-09T09:28:38.153367+0000","last_clean":"2024-03-09T09:28:38.153367+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T09:28:38.153367+0000","last_undegraded":"2024-03-09T09:28:38.153367+0000","last_fullsized":"2024-03-09T09:28:38.153367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:31.731626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007933,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":267,"num_read_kb":985,"num_write":604,"num_write_kb":87856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"89'461","reported_seq":619,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:55.398848+0000","last_change":"2024-03-09T08:52:28.586606+0000","last_active":"2024-03-09T09:27:55.398848+0000","last_peered":"2024-03-09T09:27:55.398848+0000","last_clean":"2024-03-09T09:27:55.398848+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T09:27:55.398848+0000","last_undegraded":"2024-03-09T09:27:55.398848+0000","last_fullsized":"2024-03-09T09:27:55.398848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:17:40.902403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":506,"num_write_kb":99411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"89'2186","reported_seq":5109,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.729816+0000","last_change":"2024-03-09T09:27:30.880846+0000","last_active":"2024-03-09T09:28:56.729816+0000","last_peered":"2024-03-09T09:28:56.729816+0000","last_clean":"2024-03-09T09:28:56.729816+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T09:28:56.729816+0000","last_undegraded":"2024-03-09T09:28:56.729816+0000","last_fullsized":"2024-03-09T09:28:56.729816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2186,"log_dups_size":0,"ondisk_log_size":2186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:47:22.327088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64610520400000004,"stat_sum":{"num_bytes":1318782,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":124,"num_read_kb":9990,"num_write":1296,"num_write_kb":22209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"89'667","reported_seq":829,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:27.475346+0000","last_change":"2024-03-09T08:52:28.586201+0000","last_active":"2024-03-09T09:28:27.475346+0000","last_peered":"2024-03-09T09:28:27.475346+0000","last_clean":"2024-03-09T09:28:27.475346+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T09:28:27.475346+0000","last_undegraded":"2024-03-09T09:28:27.475346+0000","last_fullsized":"2024-03-09T09:28:27.475346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:28:03.471646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":72,"num_read_kb":0,"num_write":777,"num_write_kb":92056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'2210","reported_seq":5186,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.003403+0000","last_change":"2024-03-09T09:27:10.069765+0000","last_active":"2024-03-09T09:29:14.003403+0000","last_peered":"2024-03-09T09:29:14.003403+0000","last_clean":"2024-03-09T09:29:14.003403+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T09:29:14.003403+0000","last_undegraded":"2024-03-09T09:29:14.003403+0000","last_fullsized":"2024-03-09T09:29:14.003403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2210,"log_dups_size":0,"ondisk_log_size":2210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:30:38.949657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.23951623,"stat_sum":{"num_bytes":1295292,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":13266,"num_write":1323,"num_write_kb":26325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"89'861","reported_seq":1028,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.251620+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T09:28:16.251620+0000","last_peered":"2024-03-09T09:28:16.251620+0000","last_clean":"2024-03-09T09:28:16.251620+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T09:28:16.251620+0000","last_undegraded":"2024-03-09T09:28:16.251620+0000","last_fullsized":"2024-03-09T09:28:16.251620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":45,"num_read_kb":0,"num_write":880,"num_write_kb":119947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"89'2142","reported_seq":5080,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.790657+0000","last_change":"2024-03-09T09:27:31.015518+0000","last_active":"2024-03-09T09:28:32.790657+0000","last_peered":"2024-03-09T09:28:32.790657+0000","last_clean":"2024-03-09T09:28:32.790657+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T09:28:32.790657+0000","last_undegraded":"2024-03-09T09:28:32.790657+0000","last_fullsized":"2024-03-09T09:28:32.790657+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2142,"log_dups_size":0,"ondisk_log_size":2142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:20:21.154856+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.70511981300000004,"stat_sum":{"num_bytes":1915718,"num_objects":107,"num_object_clones":37,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":37,"num_read":111,"num_read_kb":24546,"num_write":1285,"num_write_kb":44187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"89'545","reported_seq":706,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:15.180548+0000","last_change":"2024-03-09T08:52:28.585631+0000","last_active":"2024-03-09T09:28:15.180548+0000","last_peered":"2024-03-09T09:28:15.180548+0000","last_clean":"2024-03-09T09:28:15.180548+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T09:28:15.180548+0000","last_undegraded":"2024-03-09T09:28:15.180548+0000","last_fullsized":"2024-03-09T09:28:15.180548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:06:03.569759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":589,"num_write_kb":82799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'2467","reported_seq":5388,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.608252+0000","last_change":"2024-03-09T09:27:12.483134+0000","last_active":"2024-03-09T09:28:23.608252+0000","last_peered":"2024-03-09T09:28:23.608252+0000","last_clean":"2024-03-09T09:28:23.608252+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T09:28:23.608252+0000","last_undegraded":"2024-03-09T09:28:23.608252+0000","last_fullsized":"2024-03-09T09:28:23.608252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2467,"log_dups_size":0,"ondisk_log_size":2467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:53:15.813937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.79565864399999997,"stat_sum":{"num_bytes":2512650,"num_objects":93,"num_object_clones":29,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":29,"num_read":114,"num_read_kb":34867,"num_write":1447,"num_write_kb":63344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'400","reported_seq":566,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:04.123705+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T09:28:04.123705+0000","last_peered":"2024-03-09T09:28:04.123705+0000","last_clean":"2024-03-09T09:28:04.123705+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T09:28:04.123705+0000","last_undegraded":"2024-03-09T09:28:04.123705+0000","last_fullsized":"2024-03-09T09:28:04.123705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":480,"num_write_kb":79281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"89'2172","reported_seq":5110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.758184+0000","last_change":"2024-03-09T09:27:32.094130+0000","last_active":"2024-03-09T09:28:56.758184+0000","last_peered":"2024-03-09T09:28:56.758184+0000","last_clean":"2024-03-09T09:28:56.758184+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T09:28:56.758184+0000","last_undegraded":"2024-03-09T09:28:56.758184+0000","last_fullsized":"2024-03-09T09:28:56.758184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2172,"log_dups_size":0,"ondisk_log_size":2172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:02:28.116630+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.72318536,"stat_sum":{"num_bytes":768948,"num_objects":112,"num_object_clones":37,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":37,"num_read":126,"num_read_kb":26327,"num_write":1312,"num_write_kb":46360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"89'739","reported_seq":905,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.656661+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T09:27:57.656661+0000","last_peered":"2024-03-09T09:27:57.656661+0000","last_clean":"2024-03-09T09:27:57.656661+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T09:27:57.656661+0000","last_undegraded":"2024-03-09T09:27:57.656661+0000","last_fullsized":"2024-03-09T09:27:57.656661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":777,"num_write_kb":95156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"89'2301","reported_seq":5286,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.619389+0000","last_change":"2024-03-09T09:27:09.781774+0000","last_active":"2024-03-09T09:28:23.619389+0000","last_peered":"2024-03-09T09:28:23.619389+0000","last_clean":"2024-03-09T09:28:23.619389+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T09:28:23.619389+0000","last_undegraded":"2024-03-09T09:28:23.619389+0000","last_fullsized":"2024-03-09T09:28:23.619389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2301,"log_dups_size":0,"ondisk_log_size":2301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:44:18.347873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65597337700000002,"stat_sum":{"num_bytes":5623986,"num_objects":128,"num_object_clones":44,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":44,"num_read":148,"num_read_kb":35274,"num_write":1372,"num_write_kb":72006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"89'471","reported_seq":629,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.563819+0000","last_change":"2024-03-09T08:52:28.579315+0000","last_active":"2024-03-09T09:28:26.563819+0000","last_peered":"2024-03-09T09:28:26.563819+0000","last_clean":"2024-03-09T09:28:26.563819+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T09:28:26.563819+0000","last_undegraded":"2024-03-09T09:28:26.563819+0000","last_fullsized":"2024-03-09T09:28:26.563819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:57:17.301677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":514,"num_write_kb":86887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"89'2419","reported_seq":5301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.005816+0000","last_change":"2024-03-09T09:27:34.193425+0000","last_active":"2024-03-09T09:28:31.005816+0000","last_peered":"2024-03-09T09:28:31.005816+0000","last_clean":"2024-03-09T09:28:31.005816+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T09:28:31.005816+0000","last_undegraded":"2024-03-09T09:28:31.005816+0000","last_fullsized":"2024-03-09T09:28:31.005816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2419,"log_dups_size":0,"ondisk_log_size":2419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:25:58.792996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.9237505820000003,"stat_sum":{"num_bytes":5533714,"num_objects":147,"num_object_clones":51,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":51,"num_read":136,"num_read_kb":30612,"num_write":1437,"num_write_kb":51393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"89'613","reported_seq":778,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.112406+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T09:27:57.112406+0000","last_peered":"2024-03-09T09:27:57.112406+0000","last_clean":"2024-03-09T09:27:57.112406+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T09:27:57.112406+0000","last_undegraded":"2024-03-09T09:27:57.112406+0000","last_fullsized":"2024-03-09T09:27:57.112406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":61,"num_read_kb":0,"num_write":693,"num_write_kb":107810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"89'2233","reported_seq":5212,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.032993+0000","last_change":"2024-03-09T09:27:08.660892+0000","last_active":"2024-03-09T09:29:14.032993+0000","last_peered":"2024-03-09T09:29:14.032993+0000","last_clean":"2024-03-09T09:29:14.032993+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T09:29:14.032993+0000","last_undegraded":"2024-03-09T09:29:14.032993+0000","last_fullsized":"2024-03-09T09:29:14.032993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2233,"log_dups_size":0,"ondisk_log_size":2233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:58:11.216375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.1639723850000001,"stat_sum":{"num_bytes":538194,"num_objects":89,"num_object_clones":30,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":30,"num_read":112,"num_read_kb":30041,"num_write":1325,"num_write_kb":47681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"89'768","reported_seq":925,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:08.255955+0000","last_change":"2024-03-09T08:52:28.585877+0000","last_active":"2024-03-09T09:28:08.255955+0000","last_peered":"2024-03-09T09:28:08.255955+0000","last_clean":"2024-03-09T09:28:08.255955+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T09:28:08.255955+0000","last_undegraded":"2024-03-09T09:28:08.255955+0000","last_fullsized":"2024-03-09T09:28:08.255955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:30:20.036385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":79,"num_read_kb":0,"num_write":862,"num_write_kb":140804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"89'2373","reported_seq":5288,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.764408+0000","last_change":"2024-03-09T09:27:34.560887+0000","last_active":"2024-03-09T09:28:56.764408+0000","last_peered":"2024-03-09T09:28:56.764408+0000","last_clean":"2024-03-09T09:28:56.764408+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T09:28:56.764408+0000","last_undegraded":"2024-03-09T09:28:56.764408+0000","last_fullsized":"2024-03-09T09:28:56.764408+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2373,"log_dups_size":0,"ondisk_log_size":2373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:20:20.590247+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2173252530000003,"stat_sum":{"num_bytes":794839,"num_objects":125,"num_object_clones":42,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":42,"num_read":135,"num_read_kb":30019,"num_write":1425,"num_write_kb":47795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"89'2255","reported_seq":5234,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.034255+0000","last_change":"2024-03-09T09:27:07.515170+0000","last_active":"2024-03-09T09:29:14.034255+0000","last_peered":"2024-03-09T09:29:14.034255+0000","last_clean":"2024-03-09T09:29:14.034255+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T09:29:14.034255+0000","last_undegraded":"2024-03-09T09:29:14.034255+0000","last_fullsized":"2024-03-09T09:29:14.034255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2255,"log_dups_size":0,"ondisk_log_size":2255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:08.036556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.8676528270000001,"stat_sum":{"num_bytes":501236,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":17494,"num_write":1356,"num_write_kb":32845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"89'2131","reported_seq":5059,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.745807+0000","last_change":"2024-03-09T09:27:32.084492+0000","last_active":"2024-03-09T09:28:56.745807+0000","last_peered":"2024-03-09T09:28:56.745807+0000","last_clean":"2024-03-09T09:28:56.745807+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T09:28:56.745807+0000","last_undegraded":"2024-03-09T09:28:56.745807+0000","last_fullsized":"2024-03-09T09:28:56.745807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2131,"log_dups_size":0,"ondisk_log_size":2131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:22.480872+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":1.7132004649999999,"stat_sum":{"num_bytes":1622605,"num_objects":113,"num_object_clones":36,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":36,"num_read":103,"num_read_kb":9361,"num_write":1283,"num_write_kb":37703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"89'2356","reported_seq":5312,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.037599+0000","last_change":"2024-03-09T09:27:06.219951+0000","last_active":"2024-03-09T09:29:14.037599+0000","last_peered":"2024-03-09T09:29:14.037599+0000","last_clean":"2024-03-09T09:29:14.037599+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T09:29:14.037599+0000","last_undegraded":"2024-03-09T09:29:14.037599+0000","last_fullsized":"2024-03-09T09:29:14.037599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:15.950822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.73410826500000004,"stat_sum":{"num_bytes":968837,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":110,"num_read_kb":13537,"num_write":1403,"num_write_kb":47183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"89'2243","reported_seq":5128,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.038397+0000","last_change":"2024-03-09T09:27:34.720190+0000","last_active":"2024-03-09T09:28:31.038397+0000","last_peered":"2024-03-09T09:28:31.038397+0000","last_clean":"2024-03-09T09:28:31.038397+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T09:28:31.038397+0000","last_undegraded":"2024-03-09T09:28:31.038397+0000","last_fullsized":"2024-03-09T09:28:31.038397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2243,"log_dups_size":0,"ondisk_log_size":2243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:58:23.785495+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.446018724,"stat_sum":{"num_bytes":5408806,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":112,"num_read_kb":29960,"num_write":1313,"num_write_kb":48116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"89'2234","reported_seq":5231,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.661461+0000","last_change":"2024-03-09T09:27:14.175435+0000","last_active":"2024-03-09T09:28:23.661461+0000","last_peered":"2024-03-09T09:28:23.661461+0000","last_clean":"2024-03-09T09:28:23.661461+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T09:28:23.661461+0000","last_undegraded":"2024-03-09T09:28:23.661461+0000","last_fullsized":"2024-03-09T09:28:23.661461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:29.862968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.706678991,"stat_sum":{"num_bytes":887348,"num_objects":110,"num_object_clones":37,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":37,"num_read":123,"num_read_kb":42348,"num_write":1334,"num_write_kb":66020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"89'2116","reported_seq":5064,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.060507+0000","last_change":"2024-03-09T09:27:32.079328+0000","last_active":"2024-03-09T09:28:31.060507+0000","last_peered":"2024-03-09T09:28:31.060507+0000","last_clean":"2024-03-09T09:28:31.060507+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T09:28:31.060507+0000","last_undegraded":"2024-03-09T09:28:31.060507+0000","last_fullsized":"2024-03-09T09:28:31.060507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2116,"log_dups_size":0,"ondisk_log_size":2116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:18.607812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.708282305,"stat_sum":{"num_bytes":596753,"num_objects":112,"num_object_clones":36,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":36,"num_read":111,"num_read_kb":13558,"num_write":1290,"num_write_kb":25078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"89'2388","reported_seq":5308,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.601723+0000","last_change":"2024-03-09T09:27:09.523471+0000","last_active":"2024-03-09T09:28:23.601723+0000","last_peered":"2024-03-09T09:28:23.601723+0000","last_clean":"2024-03-09T09:28:23.601723+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T09:28:23.601723+0000","last_undegraded":"2024-03-09T09:28:23.601723+0000","last_fullsized":"2024-03-09T09:28:23.601723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2388,"log_dups_size":0,"ondisk_log_size":2388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:57:42.927188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.39778581299999999,"stat_sum":{"num_bytes":1231522,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":112,"num_read_kb":16729,"num_write":1390,"num_write_kb":42674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"89'2178","reported_seq":5123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.970887+0000","last_change":"2024-03-09T09:27:34.631961+0000","last_active":"2024-03-09T09:28:30.970887+0000","last_peered":"2024-03-09T09:28:30.970887+0000","last_clean":"2024-03-09T09:28:30.970887+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T09:28:30.970887+0000","last_undegraded":"2024-03-09T09:28:30.970887+0000","last_fullsized":"2024-03-09T09:28:30.970887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2178,"log_dups_size":0,"ondisk_log_size":2178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:11:08.985120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4007626340000003,"stat_sum":{"num_bytes":767001,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":115,"num_read_kb":17556,"num_write":1334,"num_write_kb":40745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"89'2219","reported_seq":5183,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.631592+0000","last_change":"2024-03-09T09:27:12.489198+0000","last_active":"2024-03-09T09:28:23.631592+0000","last_peered":"2024-03-09T09:28:23.631592+0000","last_clean":"2024-03-09T09:28:23.631592+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T09:28:23.631592+0000","last_undegraded":"2024-03-09T09:28:23.631592+0000","last_fullsized":"2024-03-09T09:28:23.631592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2219,"log_dups_size":0,"ondisk_log_size":2219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:48:35.331040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.80249947799999999,"stat_sum":{"num_bytes":1587591,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":108,"num_read_kb":24972,"num_write":1321,"num_write_kb":50275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"89'2414","reported_seq":5260,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.985366+0000","last_change":"2024-03-09T09:27:33.932429+0000","last_active":"2024-03-09T09:28:30.985366+0000","last_peered":"2024-03-09T09:28:30.985366+0000","last_clean":"2024-03-09T09:28:30.985366+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T09:28:30.985366+0000","last_undegraded":"2024-03-09T09:28:30.985366+0000","last_fullsized":"2024-03-09T09:28:30.985366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2414,"log_dups_size":0,"ondisk_log_size":2414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:24.675125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.6671456120000001,"stat_sum":{"num_bytes":827266,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":108,"num_read_kb":21834,"num_write":1396,"num_write_kb":37656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"89'2190","reported_seq":5177,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.014853+0000","last_change":"2024-03-09T09:27:10.243190+0000","last_active":"2024-03-09T09:29:14.014853+0000","last_peered":"2024-03-09T09:29:14.014853+0000","last_clean":"2024-03-09T09:29:14.014853+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T09:29:14.014853+0000","last_undegraded":"2024-03-09T09:29:14.014853+0000","last_fullsized":"2024-03-09T09:29:14.014853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2190,"log_dups_size":0,"ondisk_log_size":2190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:08.187277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.410735039,"stat_sum":{"num_bytes":1374461,"num_objects":110,"num_object_clones":35,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":35,"num_read":104,"num_read_kb":29127,"num_write":1325,"num_write_kb":54062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"89'2217","reported_seq":5145,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.042515+0000","last_change":"2024-03-09T09:27:34.622798+0000","last_active":"2024-03-09T09:28:31.042515+0000","last_peered":"2024-03-09T09:28:31.042515+0000","last_clean":"2024-03-09T09:28:31.042515+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T09:28:31.042515+0000","last_undegraded":"2024-03-09T09:28:31.042515+0000","last_fullsized":"2024-03-09T09:28:31.042515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2217,"log_dups_size":0,"ondisk_log_size":2217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:16:59.598968+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.393423125,"stat_sum":{"num_bytes":646888,"num_objects":117,"num_object_clones":37,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":37,"num_read":111,"num_read_kb":21616,"num_write":1343,"num_write_kb":41561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"89'2380","reported_seq":5353,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.053034+0000","last_change":"2024-03-09T09:27:08.090545+0000","last_active":"2024-03-09T09:29:14.053034+0000","last_peered":"2024-03-09T09:29:14.053034+0000","last_clean":"2024-03-09T09:29:14.053034+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T09:29:14.053034+0000","last_undegraded":"2024-03-09T09:29:14.053034+0000","last_fullsized":"2024-03-09T09:29:14.053034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2380,"log_dups_size":0,"ondisk_log_size":2380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:38:53.276797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.440348755,"stat_sum":{"num_bytes":1448398,"num_objects":130,"num_object_clones":44,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":44,"num_read":125,"num_read_kb":18804,"num_write":1435,"num_write_kb":32798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"89'2303","reported_seq":5205,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.859353+0000","last_change":"2024-03-09T09:27:30.874126+0000","last_active":"2024-03-09T09:28:30.859353+0000","last_peered":"2024-03-09T09:28:30.859353+0000","last_clean":"2024-03-09T09:28:30.859353+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T09:28:30.859353+0000","last_undegraded":"2024-03-09T09:28:30.859353+0000","last_fullsized":"2024-03-09T09:28:30.859353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2303,"log_dups_size":0,"ondisk_log_size":2303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:12:39.325227+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.64500296700000004,"stat_sum":{"num_bytes":1066603,"num_objects":98,"num_object_clones":32,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":32,"num_read":104,"num_read_kb":18198,"num_write":1399,"num_write_kb":47978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"89'2173","reported_seq":5134,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.524023+0000","last_change":"2024-03-09T09:27:12.333957+0000","last_active":"2024-03-09T09:28:23.524023+0000","last_peered":"2024-03-09T09:28:23.524023+0000","last_clean":"2024-03-09T09:28:23.524023+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T09:28:23.524023+0000","last_undegraded":"2024-03-09T09:28:23.524023+0000","last_fullsized":"2024-03-09T09:28:23.524023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2173,"log_dups_size":0,"ondisk_log_size":2173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:59:27.720581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.62298635300000005,"stat_sum":{"num_bytes":1371558,"num_objects":107,"num_object_clones":34,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":34,"num_read":92,"num_read_kb":30679,"num_write":1263,"num_write_kb":63255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"89'2202","reported_seq":5096,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.042030+0000","last_change":"2024-03-09T09:27:34.527780+0000","last_active":"2024-03-09T09:28:31.042030+0000","last_peered":"2024-03-09T09:28:31.042030+0000","last_clean":"2024-03-09T09:28:31.042030+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T09:28:31.042030+0000","last_undegraded":"2024-03-09T09:28:31.042030+0000","last_fullsized":"2024-03-09T09:28:31.042030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2202,"log_dups_size":0,"ondisk_log_size":2202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:33:42.613616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.1845584630000001,"stat_sum":{"num_bytes":1760124,"num_objects":108,"num_object_clones":35,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":35,"num_read":107,"num_read_kb":13622,"num_write":1279,"num_write_kb":27255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'2197","reported_seq":5178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.600035+0000","last_change":"2024-03-09T09:27:13.741409+0000","last_active":"2024-03-09T09:28:23.600035+0000","last_peered":"2024-03-09T09:28:23.600035+0000","last_clean":"2024-03-09T09:28:23.600035+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T09:28:23.600035+0000","last_undegraded":"2024-03-09T09:28:23.600035+0000","last_fullsized":"2024-03-09T09:28:23.600035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2197,"log_dups_size":0,"ondisk_log_size":2197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:03.762335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.2778432559999999,"stat_sum":{"num_bytes":1308027,"num_objects":118,"num_object_clones":42,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":42,"num_read":112,"num_read_kb":14330,"num_write":1310,"num_write_kb":25655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"89'2327","reported_seq":5223,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.734708+0000","last_change":"2024-03-09T09:27:34.655594+0000","last_active":"2024-03-09T09:28:56.734708+0000","last_peered":"2024-03-09T09:28:56.734708+0000","last_clean":"2024-03-09T09:28:56.734708+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T09:28:56.734708+0000","last_undegraded":"2024-03-09T09:28:56.734708+0000","last_fullsized":"2024-03-09T09:28:56.734708+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2327,"log_dups_size":0,"ondisk_log_size":2327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:01:41.618756+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4171876489999997,"stat_sum":{"num_bytes":1480146,"num_objects":115,"num_object_clones":41,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":41,"num_read":120,"num_read_kb":31406,"num_write":1383,"num_write_kb":52041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"89'2170","reported_seq":5168,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.531990+0000","last_change":"2024-03-09T09:27:12.405710+0000","last_active":"2024-03-09T09:28:23.531990+0000","last_peered":"2024-03-09T09:28:23.531990+0000","last_clean":"2024-03-09T09:28:23.531990+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T09:28:23.531990+0000","last_undegraded":"2024-03-09T09:28:23.531990+0000","last_fullsized":"2024-03-09T09:28:23.531990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2170,"log_dups_size":0,"ondisk_log_size":2170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:12:10.053606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.72006586699999997,"stat_sum":{"num_bytes":650831,"num_objects":114,"num_object_clones":39,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":39,"num_read":114,"num_read_kb":46474,"num_write":1326,"num_write_kb":53541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"89'2284","reported_seq":5214,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.304433+0000","last_change":"2024-03-09T09:27:30.902599+0000","last_active":"2024-03-09T09:28:32.304433+0000","last_peered":"2024-03-09T09:28:32.304433+0000","last_clean":"2024-03-09T09:28:32.304433+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T09:28:32.304433+0000","last_undegraded":"2024-03-09T09:28:32.304433+0000","last_fullsized":"2024-03-09T09:28:32.304433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:54:27.990313+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.64248945300000004,"stat_sum":{"num_bytes":509009,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":145,"num_read_kb":34396,"num_write":1348,"num_write_kb":44845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"89'2306","reported_seq":5252,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.665235+0000","last_change":"2024-03-09T09:27:13.760270+0000","last_active":"2024-03-09T09:28:23.665235+0000","last_peered":"2024-03-09T09:28:23.665235+0000","last_clean":"2024-03-09T09:28:23.665235+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T09:28:23.665235+0000","last_undegraded":"2024-03-09T09:28:23.665235+0000","last_fullsized":"2024-03-09T09:28:23.665235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2306,"log_dups_size":0,"ondisk_log_size":2306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:54:17.411237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.291579509,"stat_sum":{"num_bytes":917223,"num_objects":100,"num_object_clones":30,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":30,"num_read":95,"num_read_kb":13353,"num_write":1387,"num_write_kb":42086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"89'2316","reported_seq":5206,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.758307+0000","last_change":"2024-03-09T09:27:34.549097+0000","last_active":"2024-03-09T09:28:56.758307+0000","last_peered":"2024-03-09T09:28:56.758307+0000","last_clean":"2024-03-09T09:28:56.758307+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T09:28:56.758307+0000","last_undegraded":"2024-03-09T09:28:56.758307+0000","last_fullsized":"2024-03-09T09:28:56.758307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2316,"log_dups_size":0,"ondisk_log_size":2316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:56:26.561322+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.2066621079999997,"stat_sum":{"num_bytes":1356079,"num_objects":97,"num_object_clones":28,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":28,"num_read":106,"num_read_kb":29896,"num_write":1386,"num_write_kb":51017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"89'2221","reported_seq":5211,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.626848+0000","last_change":"2024-03-09T09:27:09.439110+0000","last_active":"2024-03-09T09:28:23.626848+0000","last_peered":"2024-03-09T09:28:23.626848+0000","last_clean":"2024-03-09T09:28:23.626848+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T09:28:23.626848+0000","last_undegraded":"2024-03-09T09:28:23.626848+0000","last_fullsized":"2024-03-09T09:28:23.626848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2221,"log_dups_size":0,"ondisk_log_size":2221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:14:38.198734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.313168905,"stat_sum":{"num_bytes":1092268,"num_objects":121,"num_object_clones":43,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":43,"num_read":123,"num_read_kb":25979,"num_write":1341,"num_write_kb":58516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"89'2297","reported_seq":5196,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.750823+0000","last_change":"2024-03-09T09:27:34.188251+0000","last_active":"2024-03-09T09:28:56.750823+0000","last_peered":"2024-03-09T09:28:56.750823+0000","last_clean":"2024-03-09T09:28:56.750823+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T09:28:56.750823+0000","last_undegraded":"2024-03-09T09:28:56.750823+0000","last_fullsized":"2024-03-09T09:28:56.750823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2297,"log_dups_size":0,"ondisk_log_size":2297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:07:37.642402+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":3.9232747450000001,"stat_sum":{"num_bytes":5183706,"num_objects":95,"num_object_clones":27,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":111,"num_read_kb":34166,"num_write":1371,"num_write_kb":49768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"89'2247","reported_seq":5207,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.063774+0000","last_change":"2024-03-09T09:27:07.884719+0000","last_active":"2024-03-09T09:29:14.063774+0000","last_peered":"2024-03-09T09:29:14.063774+0000","last_clean":"2024-03-09T09:29:14.063774+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T09:29:14.063774+0000","last_undegraded":"2024-03-09T09:29:14.063774+0000","last_fullsized":"2024-03-09T09:29:14.063774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2247,"log_dups_size":0,"ondisk_log_size":2247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:38.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.1326348209999999,"stat_sum":{"num_bytes":1866711,"num_objects":117,"num_object_clones":40,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":40,"num_read":109,"num_read_kb":17692,"num_write":1315,"num_write_kb":41853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"89'2193","reported_seq":5129,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.657070+0000","last_change":"2024-03-09T09:27:33.862027+0000","last_active":"2024-03-09T09:28:32.657070+0000","last_peered":"2024-03-09T09:28:32.657070+0000","last_clean":"2024-03-09T09:28:32.657070+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T09:28:32.657070+0000","last_undegraded":"2024-03-09T09:28:32.657070+0000","last_fullsized":"2024-03-09T09:28:32.657070+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2193,"log_dups_size":0,"ondisk_log_size":2193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:24:42.648859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.5970959150000001,"stat_sum":{"num_bytes":1348477,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":126,"num_read_kb":8825,"num_write":1341,"num_write_kb":33612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"89'2669","reported_seq":5491,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.529787+0000","last_change":"2024-03-09T09:27:13.243644+0000","last_active":"2024-03-09T09:28:23.529787+0000","last_peered":"2024-03-09T09:28:23.529787+0000","last_clean":"2024-03-09T09:28:23.529787+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T09:28:23.529787+0000","last_undegraded":"2024-03-09T09:28:23.529787+0000","last_fullsized":"2024-03-09T09:28:23.529787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2669,"log_dups_size":0,"ondisk_log_size":2669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:48.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.49950316500000003,"stat_sum":{"num_bytes":924300,"num_objects":92,"num_object_clones":25,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":25,"num_read":107,"num_read_kb":17449,"num_write":1480,"num_write_kb":44901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"89'2173","reported_seq":5118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.923833+0000","last_change":"2024-03-09T09:27:34.679372+0000","last_active":"2024-03-09T09:28:30.923833+0000","last_peered":"2024-03-09T09:28:30.923833+0000","last_clean":"2024-03-09T09:28:30.923833+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T09:28:30.923833+0000","last_undegraded":"2024-03-09T09:28:30.923833+0000","last_fullsized":"2024-03-09T09:28:30.923833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2173,"log_dups_size":0,"ondisk_log_size":2173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:56.141041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4099735320000004,"stat_sum":{"num_bytes":773517,"num_objects":127,"num_object_clones":46,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":46,"num_read":120,"num_read_kb":5950,"num_write":1331,"num_write_kb":13298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"89'2162","reported_seq":5147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.536262+0000","last_change":"2024-03-09T09:27:06.385525+0000","last_active":"2024-03-09T09:28:23.536262+0000","last_peered":"2024-03-09T09:28:23.536262+0000","last_clean":"2024-03-09T09:28:23.536262+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T09:28:23.536262+0000","last_undegraded":"2024-03-09T09:28:23.536262+0000","last_fullsized":"2024-03-09T09:28:23.536262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2162,"log_dups_size":0,"ondisk_log_size":2162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:33:27.669672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.89838860799999998,"stat_sum":{"num_bytes":6604987,"num_objects":101,"num_object_clones":30,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":30,"num_read":109,"num_read_kb":14504,"num_write":1281,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"89'2152","reported_seq":5182,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.885774+0000","last_change":"2024-03-09T09:27:31.020668+0000","last_active":"2024-03-09T09:28:30.885774+0000","last_peered":"2024-03-09T09:28:30.885774+0000","last_clean":"2024-03-09T09:28:30.885774+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T09:28:30.885774+0000","last_undegraded":"2024-03-09T09:28:30.885774+0000","last_fullsized":"2024-03-09T09:28:30.885774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2152,"log_dups_size":0,"ondisk_log_size":2152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:50:23.678039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.71096335899999996,"stat_sum":{"num_bytes":637081,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":204,"num_read_kb":24036,"num_write":1319,"num_write_kb":30441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"89'2336","reported_seq":5279,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.555277+0000","last_change":"2024-03-09T09:27:06.358258+0000","last_active":"2024-03-09T09:28:23.555277+0000","last_peered":"2024-03-09T09:28:23.555277+0000","last_clean":"2024-03-09T09:28:23.555277+0000","last_became_active":"2024-03-09T08:53:00.240319+0000","last_became_peered":"2024-03-09T08:53:00.240319+0000","last_unstale":"2024-03-09T09:28:23.555277+0000","last_undegraded":"2024-03-09T09:28:23.555277+0000","last_fullsized":"2024-03-09T09:28:23.555277+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2336,"log_dups_size":0,"ondisk_log_size":2336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:31:17.488290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.86880120699999996,"stat_sum":{"num_bytes":1614659,"num_objects":116,"num_object_clones":42,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":42,"num_read":120,"num_read_kb":18915,"num_write":1382,"num_write_kb":39172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"89'2125","reported_seq":5073,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.759491+0000","last_change":"2024-03-09T09:27:32.089100+0000","last_active":"2024-03-09T09:28:56.759491+0000","last_peered":"2024-03-09T09:28:56.759491+0000","last_clean":"2024-03-09T09:28:56.759491+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T09:28:56.759491+0000","last_undegraded":"2024-03-09T09:28:56.759491+0000","last_fullsized":"2024-03-09T09:28:56.759491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2125,"log_dups_size":0,"ondisk_log_size":2125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:42.901325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.7177660380000002,"stat_sum":{"num_bytes":924110,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":129,"num_read_kb":10365,"num_write":1260,"num_write_kb":33897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"89'2224","reported_seq":5201,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.636921+0000","last_change":"2024-03-09T09:27:13.674044+0000","last_active":"2024-03-09T09:28:23.636921+0000","last_peered":"2024-03-09T09:28:23.636921+0000","last_clean":"2024-03-09T09:28:23.636921+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T09:28:23.636921+0000","last_undegraded":"2024-03-09T09:28:23.636921+0000","last_fullsized":"2024-03-09T09:28:23.636921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2224,"log_dups_size":0,"ondisk_log_size":2224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:32.494482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.2105350770000001,"stat_sum":{"num_bytes":1326786,"num_objects":103,"num_object_clones":34,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":34,"num_read":111,"num_read_kb":18735,"num_write":1322,"num_write_kb":41042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"89'2350","reported_seq":5252,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:52.661098+0000","last_change":"2024-03-09T09:27:34.276120+0000","last_active":"2024-03-09T09:28:52.661098+0000","last_peered":"2024-03-09T09:28:52.661098+0000","last_clean":"2024-03-09T09:28:52.661098+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T09:28:52.661098+0000","last_undegraded":"2024-03-09T09:28:52.661098+0000","last_fullsized":"2024-03-09T09:28:52.661098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2350,"log_dups_size":0,"ondisk_log_size":2350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:34:48.217825+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.0059579840000001,"stat_sum":{"num_bytes":936699,"num_objects":111,"num_object_clones":39,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":39,"num_read":129,"num_read_kb":34540,"num_write":1403,"num_write_kb":57337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"89'2277","reported_seq":5248,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048137+0000","last_change":"2024-03-09T09:27:09.746181+0000","last_active":"2024-03-09T09:29:14.048137+0000","last_peered":"2024-03-09T09:29:14.048137+0000","last_clean":"2024-03-09T09:29:14.048137+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T09:29:14.048137+0000","last_undegraded":"2024-03-09T09:29:14.048137+0000","last_fullsized":"2024-03-09T09:29:14.048137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2277,"log_dups_size":0,"ondisk_log_size":2277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:47:00.612196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.91582108900000003,"stat_sum":{"num_bytes":1759504,"num_objects":101,"num_object_clones":31,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":31,"num_read":108,"num_read_kb":5845,"num_write":1367,"num_write_kb":19152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"89'2343","reported_seq":5227,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.752770+0000","last_change":"2024-03-09T09:27:34.759707+0000","last_active":"2024-03-09T09:28:56.752770+0000","last_peered":"2024-03-09T09:28:56.752770+0000","last_clean":"2024-03-09T09:28:56.752770+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T09:28:56.752770+0000","last_undegraded":"2024-03-09T09:28:56.752770+0000","last_fullsized":"2024-03-09T09:28:56.752770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2343,"log_dups_size":0,"ondisk_log_size":2343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:27:24.016743+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.4857478520000003,"stat_sum":{"num_bytes":2540930,"num_objects":122,"num_object_clones":40,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":40,"num_read":127,"num_read_kb":51469,"num_write":1352,"num_write_kb":80787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"89'2214","reported_seq":5190,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.573623+0000","last_change":"2024-03-09T09:27:09.024910+0000","last_active":"2024-03-09T09:28:23.573623+0000","last_peered":"2024-03-09T09:28:23.573623+0000","last_clean":"2024-03-09T09:28:23.573623+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T09:28:23.573623+0000","last_undegraded":"2024-03-09T09:28:23.573623+0000","last_fullsized":"2024-03-09T09:28:23.573623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2214,"log_dups_size":0,"ondisk_log_size":2214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:02:10.004295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.1939797309999998,"stat_sum":{"num_bytes":1585107,"num_objects":96,"num_object_clones":31,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":31,"num_read":96,"num_read_kb":18307,"num_write":1352,"num_write_kb":50489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"89'2673","reported_seq":5425,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.754624+0000","last_change":"2024-03-09T09:27:34.659119+0000","last_active":"2024-03-09T09:28:56.754624+0000","last_peered":"2024-03-09T09:28:56.754624+0000","last_clean":"2024-03-09T09:28:56.754624+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T09:28:56.754624+0000","last_undegraded":"2024-03-09T09:28:56.754624+0000","last_fullsized":"2024-03-09T09:28:56.754624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2673,"log_dups_size":0,"ondisk_log_size":2673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:45:21.245870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4197025170000002,"stat_sum":{"num_bytes":1559225,"num_objects":122,"num_object_clones":42,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":42,"num_read":122,"num_read_kb":6495,"num_write":1460,"num_write_kb":38738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"89'2223","reported_seq":5195,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.545518+0000","last_change":"2024-03-09T09:27:12.627461+0000","last_active":"2024-03-09T09:28:23.545518+0000","last_peered":"2024-03-09T09:28:23.545518+0000","last_clean":"2024-03-09T09:28:23.545518+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T09:28:23.545518+0000","last_undegraded":"2024-03-09T09:28:23.545518+0000","last_fullsized":"2024-03-09T09:28:23.545518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2223,"log_dups_size":0,"ondisk_log_size":2223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:03:50.159643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.94133736700000004,"stat_sum":{"num_bytes":314683,"num_objects":117,"num_object_clones":39,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":39,"num_read":116,"num_read_kb":13620,"num_write":1329,"num_write_kb":42753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"89'2194","reported_seq":5204,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.014352+0000","last_change":"2024-03-09T09:27:34.517246+0000","last_active":"2024-03-09T09:28:31.014352+0000","last_peered":"2024-03-09T09:28:31.014352+0000","last_clean":"2024-03-09T09:28:31.014352+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T09:28:31.014352+0000","last_undegraded":"2024-03-09T09:28:31.014352+0000","last_fullsized":"2024-03-09T09:28:31.014352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2194,"log_dups_size":0,"ondisk_log_size":2194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:23:51.461775+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.1785381859999999,"stat_sum":{"num_bytes":1532500,"num_objects":95,"num_object_clones":30,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":30,"num_read":195,"num_read_kb":18044,"num_write":1327,"num_write_kb":39414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"89'2266","reported_seq":5235,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.043614+0000","last_change":"2024-03-09T09:27:08.772433+0000","last_active":"2024-03-09T09:29:14.043614+0000","last_peered":"2024-03-09T09:29:14.043614+0000","last_clean":"2024-03-09T09:29:14.043614+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T09:29:14.043614+0000","last_undegraded":"2024-03-09T09:29:14.043614+0000","last_fullsized":"2024-03-09T09:29:14.043614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2266,"log_dups_size":0,"ondisk_log_size":2266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:09:40.668035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.298540461,"stat_sum":{"num_bytes":2039183,"num_objects":130,"num_object_clones":43,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":43,"num_read":126,"num_read_kb":25377,"num_write":1340,"num_write_kb":50110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"89'2204","reported_seq":5123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:47.307121+0000","last_change":"2024-03-09T09:27:30.886579+0000","last_active":"2024-03-09T09:28:47.307121+0000","last_peered":"2024-03-09T09:28:47.307121+0000","last_clean":"2024-03-09T09:28:47.307121+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T09:28:47.307121+0000","last_undegraded":"2024-03-09T09:28:47.307121+0000","last_fullsized":"2024-03-09T09:28:47.307121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2204,"log_dups_size":0,"ondisk_log_size":2204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:31:23.131967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.62967200199999995,"stat_sum":{"num_bytes":347019,"num_objects":120,"num_object_clones":41,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":41,"num_read":119,"num_read_kb":17844,"num_write":1302,"num_write_kb":51829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"89'2658","reported_seq":5496,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.057096+0000","last_change":"2024-03-09T09:27:07.715360+0000","last_active":"2024-03-09T09:29:14.057096+0000","last_peered":"2024-03-09T09:29:14.057096+0000","last_clean":"2024-03-09T09:29:14.057096+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T09:29:14.057096+0000","last_undegraded":"2024-03-09T09:29:14.057096+0000","last_fullsized":"2024-03-09T09:29:14.057096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2658,"log_dups_size":0,"ondisk_log_size":2658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:37:57.318273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.0660642299999998,"stat_sum":{"num_bytes":1365000,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":127,"num_read_kb":33469,"num_write":1475,"num_write_kb":57156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"89'2226","reported_seq":5152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.035736+0000","last_change":"2024-03-09T09:27:34.649642+0000","last_active":"2024-03-09T09:28:31.035736+0000","last_peered":"2024-03-09T09:28:31.035736+0000","last_clean":"2024-03-09T09:28:31.035736+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T09:28:31.035736+0000","last_undegraded":"2024-03-09T09:28:31.035736+0000","last_fullsized":"2024-03-09T09:28:31.035736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2226,"log_dups_size":0,"ondisk_log_size":2226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:53.123132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4162208879999998,"stat_sum":{"num_bytes":1481141,"num_objects":109,"num_object_clones":37,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":37,"num_read":120,"num_read_kb":25693,"num_write":1333,"num_write_kb":50297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'2192","reported_seq":5171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.049795+0000","last_change":"2024-03-09T09:27:08.029887+0000","last_active":"2024-03-09T09:29:14.049795+0000","last_peered":"2024-03-09T09:29:14.049795+0000","last_clean":"2024-03-09T09:29:14.049795+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T09:29:14.049795+0000","last_undegraded":"2024-03-09T09:29:14.049795+0000","last_fullsized":"2024-03-09T09:29:14.049795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2192,"log_dups_size":0,"ondisk_log_size":2192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:57:17.824590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.379810049,"stat_sum":{"num_bytes":472147,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":115,"num_read_kb":25809,"num_write":1277,"num_write_kb":34061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"89'2305","reported_seq":5227,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.736916+0000","last_change":"2024-03-09T09:27:34.694598+0000","last_active":"2024-03-09T09:28:56.736916+0000","last_peered":"2024-03-09T09:28:56.736916+0000","last_clean":"2024-03-09T09:28:56.736916+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T09:28:56.736916+0000","last_undegraded":"2024-03-09T09:28:56.736916+0000","last_fullsized":"2024-03-09T09:28:56.736916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2305,"log_dups_size":0,"ondisk_log_size":2305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:24:52.867402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4238349550000002,"stat_sum":{"num_bytes":1560364,"num_objects":99,"num_object_clones":31,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":31,"num_read":120,"num_read_kb":23106,"num_write":1406,"num_write_kb":36148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"89'2350","reported_seq":5311,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.659030+0000","last_change":"2024-03-09T09:27:13.474431+0000","last_active":"2024-03-09T09:28:23.659030+0000","last_peered":"2024-03-09T09:28:23.659030+0000","last_clean":"2024-03-09T09:28:23.659030+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T09:28:23.659030+0000","last_undegraded":"2024-03-09T09:28:23.659030+0000","last_fullsized":"2024-03-09T09:28:23.659030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2350,"log_dups_size":0,"ondisk_log_size":2350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:53.961909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.73072901000000001,"stat_sum":{"num_bytes":1439516,"num_objects":106,"num_object_clones":37,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":37,"num_read":117,"num_read_kb":43194,"num_write":1390,"num_write_kb":63251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"89'2108","reported_seq":5031,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.764089+0000","last_change":"2024-03-09T09:27:34.559512+0000","last_active":"2024-03-09T09:28:56.764089+0000","last_peered":"2024-03-09T09:28:56.764089+0000","last_clean":"2024-03-09T09:28:56.764089+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T09:28:56.764089+0000","last_undegraded":"2024-03-09T09:28:56.764089+0000","last_fullsized":"2024-03-09T09:28:56.764089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2108,"log_dups_size":0,"ondisk_log_size":2108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:12:19.227405+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2163343510000004,"stat_sum":{"num_bytes":526581,"num_objects":91,"num_object_clones":29,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":29,"num_read":100,"num_read_kb":25429,"num_write":1233,"num_write_kb":64615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"89'2155","reported_seq":5129,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.610096+0000","last_change":"2024-03-09T09:27:12.157156+0000","last_active":"2024-03-09T09:28:23.610096+0000","last_peered":"2024-03-09T09:28:23.610096+0000","last_clean":"2024-03-09T09:28:23.610096+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T09:28:23.610096+0000","last_undegraded":"2024-03-09T09:28:23.610096+0000","last_fullsized":"2024-03-09T09:28:23.610096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2155,"log_dups_size":0,"ondisk_log_size":2155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:45:37.914970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.46979842799999999,"stat_sum":{"num_bytes":631595,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":98,"num_read_kb":25968,"num_write":1286,"num_write_kb":41427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"89'2132","reported_seq":5071,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.889842+0000","last_change":"2024-03-09T09:27:30.899203+0000","last_active":"2024-03-09T09:28:30.889842+0000","last_peered":"2024-03-09T09:28:30.889842+0000","last_clean":"2024-03-09T09:28:30.889842+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T09:28:30.889842+0000","last_undegraded":"2024-03-09T09:28:30.889842+0000","last_fullsized":"2024-03-09T09:28:30.889842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2132,"log_dups_size":0,"ondisk_log_size":2132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:29:36.464781+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.66506321800000001,"stat_sum":{"num_bytes":592790,"num_objects":103,"num_object_clones":35,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":35,"num_read":105,"num_read_kb":21771,"num_write":1301,"num_write_kb":41126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"89'2198","reported_seq":5154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.571132+0000","last_change":"2024-03-09T09:27:13.380861+0000","last_active":"2024-03-09T09:28:23.571132+0000","last_peered":"2024-03-09T09:28:23.571132+0000","last_clean":"2024-03-09T09:28:23.571132+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T09:28:23.571132+0000","last_undegraded":"2024-03-09T09:28:23.571132+0000","last_fullsized":"2024-03-09T09:28:23.571132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2198,"log_dups_size":0,"ondisk_log_size":2198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:02:11.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.638707739,"stat_sum":{"num_bytes":1008318,"num_objects":81,"num_object_clones":24,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":24,"num_read":86,"num_read_kb":17604,"num_write":1317,"num_write_kb":38401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"89'2293","reported_seq":5184,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.754410+0000","last_change":"2024-03-09T09:27:32.075213+0000","last_active":"2024-03-09T09:28:56.754410+0000","last_peered":"2024-03-09T09:28:56.754410+0000","last_clean":"2024-03-09T09:28:56.754410+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T09:28:56.754410+0000","last_undegraded":"2024-03-09T09:28:56.754410+0000","last_fullsized":"2024-03-09T09:28:56.754410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2293,"log_dups_size":0,"ondisk_log_size":2293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:17:17.703718+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.705573912,"stat_sum":{"num_bytes":534756,"num_objects":106,"num_object_clones":34,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":34,"num_read":108,"num_read_kb":25061,"num_write":1338,"num_write_kb":44408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"89'2108","reported_seq":5111,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.601144+0000","last_change":"2024-03-09T09:27:13.610566+0000","last_active":"2024-03-09T09:28:23.601144+0000","last_peered":"2024-03-09T09:28:23.601144+0000","last_clean":"2024-03-09T09:28:23.601144+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T09:28:23.601144+0000","last_undegraded":"2024-03-09T09:28:23.601144+0000","last_fullsized":"2024-03-09T09:28:23.601144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2108,"log_dups_size":0,"ondisk_log_size":2108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:17:33.229502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.1420017650000001,"stat_sum":{"num_bytes":1651219,"num_objects":118,"num_object_clones":44,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":44,"num_read":114,"num_read_kb":27021,"num_write":1266,"num_write_kb":36457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"89'2271","reported_seq":5195,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.743120+0000","last_change":"2024-03-09T09:27:34.669037+0000","last_active":"2024-03-09T09:28:56.743120+0000","last_peered":"2024-03-09T09:28:56.743120+0000","last_clean":"2024-03-09T09:28:56.743120+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T09:28:56.743120+0000","last_undegraded":"2024-03-09T09:28:56.743120+0000","last_fullsized":"2024-03-09T09:28:56.743120+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2271,"log_dups_size":0,"ondisk_log_size":2271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:54:35.897455+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.3954307559999997,"stat_sum":{"num_bytes":1015705,"num_objects":121,"num_object_clones":38,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":38,"num_read":132,"num_read_kb":25133,"num_write":1351,"num_write_kb":59371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"89'2354","reported_seq":5307,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.571815+0000","last_change":"2024-03-09T09:27:13.261980+0000","last_active":"2024-03-09T09:28:23.571815+0000","last_peered":"2024-03-09T09:28:23.571815+0000","last_clean":"2024-03-09T09:28:23.571815+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T09:28:23.571815+0000","last_undegraded":"2024-03-09T09:28:23.571815+0000","last_fullsized":"2024-03-09T09:28:23.571815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2354,"log_dups_size":0,"ondisk_log_size":2354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:13:56.252007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.51876684699999998,"stat_sum":{"num_bytes":1407157,"num_objects":117,"num_object_clones":38,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":38,"num_read":113,"num_read_kb":14121,"num_write":1423,"num_write_kb":30929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"89'2142","reported_seq":5072,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.010893+0000","last_change":"2024-03-09T09:27:34.506277+0000","last_active":"2024-03-09T09:28:31.010893+0000","last_peered":"2024-03-09T09:28:31.010893+0000","last_clean":"2024-03-09T09:28:31.010893+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T09:28:31.010893+0000","last_undegraded":"2024-03-09T09:28:31.010893+0000","last_fullsized":"2024-03-09T09:28:31.010893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2142,"log_dups_size":0,"ondisk_log_size":2142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:27.408652+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.1683436250000003,"stat_sum":{"num_bytes":2080769,"num_objects":116,"num_object_clones":43,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":43,"num_read":114,"num_read_kb":27228,"num_write":1275,"num_write_kb":54719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"89'2222","reported_seq":5193,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.644428+0000","last_change":"2024-03-09T09:27:13.892143+0000","last_active":"2024-03-09T09:28:23.644428+0000","last_peered":"2024-03-09T09:28:23.644428+0000","last_clean":"2024-03-09T09:28:23.644428+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T09:28:23.644428+0000","last_undegraded":"2024-03-09T09:28:23.644428+0000","last_fullsized":"2024-03-09T09:28:23.644428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2222,"log_dups_size":0,"ondisk_log_size":2222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:36:18.464529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.4282657350000001,"stat_sum":{"num_bytes":1076453,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":106,"num_read_kb":19463,"num_write":1318,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"89'2408","reported_seq":5285,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.733743+0000","last_change":"2024-03-09T09:27:30.899184+0000","last_active":"2024-03-09T09:28:56.733743+0000","last_peered":"2024-03-09T09:28:56.733743+0000","last_clean":"2024-03-09T09:28:56.733743+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T09:28:56.733743+0000","last_undegraded":"2024-03-09T09:28:56.733743+0000","last_fullsized":"2024-03-09T09:28:56.733743+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2408,"log_dups_size":0,"ondisk_log_size":2408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:47.160761+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.63967549000000001,"stat_sum":{"num_bytes":3580963,"num_objects":116,"num_object_clones":37,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":37,"num_read":116,"num_read_kb":32310,"num_write":1440,"num_write_kb":66236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"89'2243","reported_seq":5223,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.539723+0000","last_change":"2024-03-09T09:27:09.597741+0000","last_active":"2024-03-09T09:28:23.539723+0000","last_peered":"2024-03-09T09:28:23.539723+0000","last_clean":"2024-03-09T09:28:23.539723+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T09:28:23.539723+0000","last_undegraded":"2024-03-09T09:28:23.539723+0000","last_fullsized":"2024-03-09T09:28:23.539723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2243,"log_dups_size":0,"ondisk_log_size":2243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:29:26.509228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.47268980999999999,"stat_sum":{"num_bytes":1488567,"num_objects":112,"num_object_clones":34,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":34,"num_read":108,"num_read_kb":30029,"num_write":1350,"num_write_kb":42424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"89'2138","reported_seq":5068,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.747878+0000","last_change":"2024-03-09T09:27:34.605106+0000","last_active":"2024-03-09T09:28:56.747878+0000","last_peered":"2024-03-09T09:28:56.747878+0000","last_clean":"2024-03-09T09:28:56.747878+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T09:28:56.747878+0000","last_undegraded":"2024-03-09T09:28:56.747878+0000","last_fullsized":"2024-03-09T09:28:56.747878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2138,"log_dups_size":0,"ondisk_log_size":2138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:04:07.487361+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.3763956530000003,"stat_sum":{"num_bytes":1440979,"num_objects":98,"num_object_clones":30,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":30,"num_read":104,"num_read_kb":22606,"num_write":1302,"num_write_kb":46651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"89'2066","reported_seq":5058,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.003676+0000","last_change":"2024-03-09T09:27:10.176463+0000","last_active":"2024-03-09T09:29:14.003676+0000","last_peered":"2024-03-09T09:29:14.003676+0000","last_clean":"2024-03-09T09:29:14.003676+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T09:29:14.003676+0000","last_undegraded":"2024-03-09T09:29:14.003676+0000","last_fullsized":"2024-03-09T09:29:14.003676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2066,"log_dups_size":0,"ondisk_log_size":2066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:32:22.191345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.344208157,"stat_sum":{"num_bytes":485403,"num_objects":85,"num_object_clones":27,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":27,"num_read":96,"num_read_kb":5635,"num_write":1204,"num_write_kb":28568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"89'2209","reported_seq":5121,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.734713+0000","last_change":"2024-03-09T09:27:34.669203+0000","last_active":"2024-03-09T09:28:56.734713+0000","last_peered":"2024-03-09T09:28:56.734713+0000","last_clean":"2024-03-09T09:28:56.734713+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T09:28:56.734713+0000","last_undegraded":"2024-03-09T09:28:56.734713+0000","last_fullsized":"2024-03-09T09:28:56.734713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2209,"log_dups_size":0,"ondisk_log_size":2209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:04:18.586097+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.4378791440000001,"stat_sum":{"num_bytes":906976,"num_objects":87,"num_object_clones":27,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":27,"num_read":110,"num_read_kb":26703,"num_write":1307,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"89'2141","reported_seq":5137,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.600768+0000","last_change":"2024-03-09T09:27:13.826300+0000","last_active":"2024-03-09T09:28:23.600768+0000","last_peered":"2024-03-09T09:28:23.600768+0000","last_clean":"2024-03-09T09:28:23.600768+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T09:28:23.600768+0000","last_undegraded":"2024-03-09T09:28:23.600768+0000","last_fullsized":"2024-03-09T09:28:23.600768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2141,"log_dups_size":0,"ondisk_log_size":2141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:23:44.879214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.0826407490000001,"stat_sum":{"num_bytes":992571,"num_objects":97,"num_object_clones":32,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":32,"num_read":100,"num_read_kb":32945,"num_write":1310,"num_write_kb":54952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7626,"num_read_kb":1526336,"num_write":85965,"num_write_kb":2933421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144120,"ondisk_log_size":144120,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":875162461,"num_objects":1506,"num_object_clones":0,"num_object_copies":3012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1506,"num_whiteouts":0,"num_read":2924,"num_read_kb":5807,"num_write":25870,"num_write_kb":3142874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1225,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1750593536,"data_stored":1750532723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22156,"ondisk_log_size":22156,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7230,"num_read_kb":1519173,"num_write":85971,"num_write_kb":2933405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144129,"ondisk_log_size":144129,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797761390,"num_objects":1474,"num_object_clones":0,"num_object_copies":2948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1474,"num_whiteouts":0,"num_read":2851,"num_read_kb":5749,"num_write":25858,"num_write_kb":3104281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1213,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1595379712,"data_stored":1595320496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22157,"ondisk_log_size":22157,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575297,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":851424,"kb_used_data":501016,"kb_used_omap":1,"kb_used_meta":350398,"kb_avail":93520416,"statfs":{"total":96636764160,"available":95764905984,"internally_reserved":0,"allocated":513040384,"data_stored":506264834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":358808010},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575299,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699740,"kb_used_data":456276,"kb_used_omap":1,"kb_used_meta":243454,"kb_avail":93672100,"statfs":{"total":96636764160,"available":95920230400,"internally_reserved":0,"allocated":467226624,"data_stored":462408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":249297350},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575296,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794200,"kb_used_data":452112,"kb_used_omap":1,"kb_used_meta":342078,"kb_avail":93577640,"statfs":{"total":96636764160,"available":95823503360,"internally_reserved":0,"allocated":462962688,"data_stored":455766994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":350288325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575298,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":853316,"kb_used_data":551484,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":93518524,"statfs":{"total":96636764160,"available":95762968576,"internally_reserved":0,"allocated":564719616,"data_stored":557367936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309066181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575298,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":917592,"kb_used_data":543632,"kb_used_omap":1,"kb_used_meta":373950,"kb_avail":93454248,"statfs":{"total":96636764160,"available":95697149952,"internally_reserved":0,"allocated":556679168,"data_stored":549175915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":382925255},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575298,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450472,"kb_used_data":225056,"kb_used_omap":1,"kb_used_meta":225406,"kb_avail":93921368,"statfs":{"total":96636764160,"available":96175480832,"internally_reserved":0,"allocated":230457344,"data_stored":225591238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":230816197},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575298,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":968836,"kb_used_data":612924,"kb_used_omap":1,"kb_used_meta":355902,"kb_avail":93403004,"statfs":{"total":96636764160,"available":95644676096,"internally_reserved":0,"allocated":627634176,"data_stored":621572263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":364444101},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575299,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670128,"kb_used_data":378088,"kb_used_omap":1,"kb_used_meta":292030,"kb_avail":93701712,"statfs":{"total":96636764160,"available":95950553088,"internally_reserved":0,"allocated":387162112,"data_stored":380840943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":299039175},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":226504704,"data_stored":226493054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":218112000,"data_stored":218104019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92274688,"data_stored":92274688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":171974656,"data_stored":171967136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":201338880,"data_stored":201326769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":197779456,"data_stored":197772203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":223150080,"data_stored":223140893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":264245248,"data_stored":264241734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37781504,"data_stored":34251073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20246528,"data_stored":18282047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22822912,"data_stored":20809621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27308032,"data_stored":23727598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37437440,"data_stored":33951899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":27983872,"data_stored":24565897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23818240,"data_stored":21628413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":29773824,"data_stored":26493862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98373632,"data_stored":98372705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":360722432,"data_stored":360710838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96473088,"data_stored":96469310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":315428864,"data_stored":315415292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":289415168,"data_stored":289407328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":199254016,"data_stored":199244474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":201334784,"data_stored":201327208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":189591552,"data_stored":189585568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22835200,"data_stored":20540995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27480064,"data_stored":23882611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17813504,"data_stored":15444871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40894464,"data_stored":37473141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35454976,"data_stored":32089192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":37081088,"data_stored":33799473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17850368,"data_stored":15719573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":27762688,"data_stored":24760554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T09:29:19.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T09:29:19.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.525+0000 7fd07b000640 1 Processor -- start 2024-03-09T09:29:19.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.525+0000 7fd07b000640 1 -- start start 2024-03-09T09:29:19.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.526+0000 7fd07b000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fd074153300 0x7fd0740b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:19.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.527+0000 7fd07b000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0740c5e40 0x7fd0740b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:19.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.527+0000 7fd07b000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0740cadb0 0x7fd074156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:19.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.527+0000 7fd07b000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd074157000 con 0x7fd0740c5e40 2024-03-09T09:29:19.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.527+0000 7fd07b000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0740ae430 con 0x7fd074153300 2024-03-09T09:29:19.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.527+0000 7fd07b000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740ae6f0 con 0x7fd0740cadb0 2024-03-09T09:29:19.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.527+0000 7fd079200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0740c5e40 0x7fd0740b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:19.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.527+0000 7fd079200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fd0740c5e40 0x7fd0740b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45034/0 (socket says 172.21.15.67:45034) 2024-03-09T09:29:19.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.527+0000 7fd079200640 1 -- 172.21.15.67:0/3114485288 learned_addr learned my addr 172.21.15.67:0/3114485288 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:19.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.527+0000 7fd07a600640 1 --2- 172.21.15.67:0/3114485288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0740cadb0 0x7fd074156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:19.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.527+0000 7fd079c00640 1 --2- 172.21.15.67:0/3114485288 >> v2:172.21.15.134:3300/0 conn(0x7fd074153300 0x7fd0740b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:19.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.528+0000 7fd079200640 1 -- 172.21.15.67:0/3114485288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0740cadb0 msgr2=0x7fd074156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:19.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.528+0000 7fd079200640 1 --2- 172.21.15.67:0/3114485288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0740cadb0 0x7fd074156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.528+0000 7fd079200640 1 -- 172.21.15.67:0/3114485288 >> v2:172.21.15.134:3300/0 conn(0x7fd074153300 msgr2=0x7fd0740b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:19.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.528+0000 7fd079200640 1 --2- 172.21.15.67:0/3114485288 >> v2:172.21.15.134:3300/0 conn(0x7fd074153300 0x7fd0740b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.528+0000 7fd079200640 1 -- 172.21.15.67:0/3114485288 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd074155b10 con 0x7fd0740c5e40 2024-03-09T09:29:19.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.529+0000 7fd079200640 1 --2- 172.21.15.67:0/3114485288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0740c5e40 0x7fd0740b0540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd0740ade40 tx=0x7fd06002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4314d5a48d5a3a75 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:19.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.529+0000 7fd073e00640 1 -- 172.21.15.67:0/3114485288 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060035020 con 0x7fd0740c5e40 2024-03-09T09:29:19.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.529+0000 7fd073e00640 1 -- 172.21.15.67:0/3114485288 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06002d040 con 0x7fd0740c5e40 2024-03-09T09:29:19.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.529+0000 7fd073e00640 1 -- 172.21.15.67:0/3114485288 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060031a80 con 0x7fd0740c5e40 2024-03-09T09:29:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.530+0000 7fd07b000640 1 -- 172.21.15.67:0/3114485288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0740c5e40 msgr2=0x7fd0740b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.530+0000 7fd07b000640 1 --2- 172.21.15.67:0/3114485288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0740c5e40 0x7fd0740b0540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd0740ade40 tx=0x7fd06002b900 comp rx=0 tx=0).stop 2024-03-09T09:29:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.530+0000 7fd07b000640 1 -- 172.21.15.67:0/3114485288 shutdown_connections 2024-03-09T09:29:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.530+0000 7fd07b000640 1 --2- 172.21.15.67:0/3114485288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0740cadb0 0x7fd074156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.530+0000 7fd07b000640 1 --2- 172.21.15.67:0/3114485288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0740c5e40 0x7fd0740b0540 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.530+0000 7fd07b000640 1 --2- 172.21.15.67:0/3114485288 >> v2:172.21.15.134:3300/0 conn(0x7fd074153300 0x7fd0740b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.530+0000 7fd07b000640 1 -- 172.21.15.67:0/3114485288 >> 172.21.15.67:0/3114485288 conn(0x7fd0740732d0 msgr2=0x7fd0740ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.530+0000 7fd07b000640 1 -- 172.21.15.67:0/3114485288 shutdown_connections 2024-03-09T09:29:19.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.530+0000 7fd07b000640 1 -- 172.21.15.67:0/3114485288 wait complete. 2024-03-09T09:29:19.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.532+0000 7fd07b000640 1 Processor -- start 2024-03-09T09:29:19.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.532+0000 7fd07b000640 1 -- start start 2024-03-09T09:29:19.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.533+0000 7fd07b000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0740c5e40 0x7fd0740c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:19.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.533+0000 7fd07b000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0740cadb0 0x7fd0740abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:19.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.533+0000 7fd07b000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd074153300 0x7fd07414b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:19.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.533+0000 7fd07b000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740c59e0 con 0x7fd0740c5e40 2024-03-09T09:29:19.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.533+0000 7fd07b000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd074159e90 con 0x7fd074153300 2024-03-09T09:29:19.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.533+0000 7fd07b000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd074156550 con 0x7fd0740cadb0 2024-03-09T09:29:19.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.533+0000 7fd079c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0740c5e40 0x7fd0740c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:19.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.533+0000 7fd079c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0740c5e40 0x7fd0740c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45042/0 (socket says 172.21.15.67:45042) 2024-03-09T09:29:19.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.533+0000 7fd079c00640 1 -- 172.21.15.67:0/2015390734 learned_addr learned my addr 172.21.15.67:0/2015390734 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:19.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd079200640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0740cadb0 0x7fd0740abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:19.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd07a600640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd074153300 0x7fd07414b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:19.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd079c00640 1 -- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0740cadb0 msgr2=0x7fd0740abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:19.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd079c00640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0740cadb0 0x7fd0740abbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd079c00640 1 -- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd074153300 msgr2=0x7fd07414b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:19.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd079c00640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd074153300 0x7fd07414b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd079c00640 1 -- 172.21.15.67:0/2015390734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0740c7520 con 0x7fd0740c5e40 2024-03-09T09:29:19.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd079c00640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0740c5e40 0x7fd0740c6340 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd06800d620 tx=0x7fd06800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd071600640 1 -- 172.21.15.67:0/2015390734 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06800b620 con 0x7fd0740c5e40 2024-03-09T09:29:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd071600640 1 -- 172.21.15.67:0/2015390734 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd068019070 con 0x7fd0740c5e40 2024-03-09T09:29:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd071600640 1 -- 172.21.15.67:0/2015390734 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06800ba00 con 0x7fd0740c5e40 2024-03-09T09:29:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.534+0000 7fd07b000640 1 -- 172.21.15.67:0/2015390734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0740ac810 con 0x7fd0740c5e40 2024-03-09T09:29:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.535+0000 7fd07b000640 1 -- 172.21.15.67:0/2015390734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0740ab3b0 con 0x7fd0740c5e40 2024-03-09T09:29:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.536+0000 7fd07b000640 1 -- 172.21.15.67:0/2015390734 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07404f3c0 con 0x7fd0740c5e40 2024-03-09T09:29:19.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.537+0000 7fd071600640 1 -- 172.21.15.67:0/2015390734 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd06800bba0 con 0x7fd0740c5e40 2024-03-09T09:29:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.537+0000 7fd071600640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd05406b500 0x7fd05406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.538+0000 7fd071600640 1 -- 172.21.15.67:0/2015390734 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fd06808a9b0 con 0x7fd0740c5e40 2024-03-09T09:29:19.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.538+0000 7fd079200640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd05406b500 0x7fd05406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:19.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.539+0000 7fd079200640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd05406b500 0x7fd05406d9c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd060036000 tx=0x7fd06003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:19.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.542+0000 7fd071600640 1 -- 172.21.15.67:0/2015390734 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0740c7520 con 0x7fd0740c5e40 2024-03-09T09:29:19.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.740+0000 7fd07b000640 1 -- 172.21.15.67:0/2015390734 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0740cab90 con 0x7fd05406b500 2024-03-09T09:29:19.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.795+0000 7fd071600640 1 -- 172.21.15.67:0/2015390734 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+499794 (secure 0 0 0) 0x7fd0740cab90 con 0x7fd05406b500 2024-03-09T09:29:19.809 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:29:19.813 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T09:29:19.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.802+0000 7fd07b000640 1 -- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd05406b500 msgr2=0x7fd05406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:19.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.802+0000 7fd07b000640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd05406b500 0x7fd05406d9c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd060036000 tx=0x7fd06003c040 comp rx=0 tx=0).stop 2024-03-09T09:29:19.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.802+0000 7fd07b000640 1 -- 172.21.15.67:0/2015390734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0740c5e40 msgr2=0x7fd0740c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:19.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.802+0000 7fd07b000640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0740c5e40 0x7fd0740c6340 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd06800d620 tx=0x7fd06800dae0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.803+0000 7fd07b000640 1 -- 172.21.15.67:0/2015390734 shutdown_connections 2024-03-09T09:29:19.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.803+0000 7fd07b000640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fd05406b500 0x7fd05406d9c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.803+0000 7fd07b000640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd074153300 0x7fd07414b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.803+0000 7fd07b000640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0740cadb0 0x7fd0740abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.803+0000 7fd07b000640 1 --2- 172.21.15.67:0/2015390734 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0740c5e40 0x7fd0740c6340 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:19.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.803+0000 7fd07b000640 1 -- 172.21.15.67:0/2015390734 >> 172.21.15.67:0/2015390734 conn(0x7fd0740732d0 msgr2=0x7fd0741568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:19.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.803+0000 7fd07b000640 1 -- 172.21.15.67:0/2015390734 shutdown_connections 2024-03-09T09:29:19.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:19.803+0000 7fd07b000640 1 -- 172.21.15.67:0/2015390734 wait complete. 2024-03-09T09:29:19.833 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":1198,"stamp":"2024-03-09T09:29:19.711022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1878194240,"num_objects":17181,"num_object_clones":4728,"num_object_copies":34362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17181,"num_whiteouts":4728,"num_read":20750,"num_read_kb":3057165,"num_write":223838,"num_write_kb":12117351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2438,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":333014,"ondisk_log_size":333014,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6207596,"kb_used_data":3722476,"kb_used_omap":12,"kb_used_meta":2485043,"kb_avail":748767124,"statfs":{"total":773094113280,"available":766737534976,"internally_reserved":0,"allocated":3811815424,"data_stored":3760920592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2544684594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1934299,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":31,"num_write_kb":1904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005497"},"pg_stats":[{"pgid":"5.18","version":"89'2465","reported_seq":5338,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:47.659926+0000","last_change":"2024-03-09T09:27:34.120297+0000","last_active":"2024-03-09T09:28:47.659926+0000","last_peered":"2024-03-09T09:28:47.659926+0000","last_clean":"2024-03-09T09:28:47.659926+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T09:28:47.659926+0000","last_undegraded":"2024-03-09T09:28:47.659926+0000","last_fullsized":"2024-03-09T09:28:47.659926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2465,"log_dups_size":0,"ondisk_log_size":2465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:09:33.307167+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":3.854819714,"stat_sum":{"num_bytes":1582215,"num_objects":90,"num_object_clones":27,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":27,"num_read":122,"num_read_kb":30313,"num_write":1455,"num_write_kb":61176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"89'449","reported_seq":618,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:51.137870+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T09:27:51.137870+0000","last_peered":"2024-03-09T09:27:51.137870+0000","last_clean":"2024-03-09T09:27:51.137870+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T09:27:51.137870+0000","last_undegraded":"2024-03-09T09:27:51.137870+0000","last_fullsized":"2024-03-09T09:27:51.137870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:41:40.332687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":462,"num_write_kb":82632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"89'2179","reported_seq":5159,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:28.596602+0000","last_change":"2024-03-09T09:27:12.157369+0000","last_active":"2024-03-09T09:28:28.596602+0000","last_peered":"2024-03-09T09:28:28.596602+0000","last_clean":"2024-03-09T09:28:28.596602+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T09:28:28.596602+0000","last_undegraded":"2024-03-09T09:28:28.596602+0000","last_fullsized":"2024-03-09T09:28:28.596602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2179,"log_dups_size":0,"ondisk_log_size":2179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:03:38.261625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.47085273999999999,"stat_sum":{"num_bytes":1473782,"num_objects":111,"num_object_clones":38,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":38,"num_read":99,"num_read_kb":25344,"num_write":1311,"num_write_kb":54508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"89'405","reported_seq":563,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:59.412978+0000","last_change":"2024-03-09T08:52:28.571905+0000","last_active":"2024-03-09T09:27:59.412978+0000","last_peered":"2024-03-09T09:27:59.412978+0000","last_clean":"2024-03-09T09:27:59.412978+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T09:27:59.412978+0000","last_undegraded":"2024-03-09T09:27:59.412978+0000","last_fullsized":"2024-03-09T09:27:59.412978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:01.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":445,"num_write_kb":78871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"89'2252","reported_seq":5164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:52.564577+0000","last_change":"2024-03-09T09:27:32.072161+0000","last_active":"2024-03-09T09:28:52.564577+0000","last_peered":"2024-03-09T09:28:52.564577+0000","last_clean":"2024-03-09T09:28:52.564577+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T09:28:52.564577+0000","last_undegraded":"2024-03-09T09:28:52.564577+0000","last_fullsized":"2024-03-09T09:28:52.564577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2252,"log_dups_size":0,"ondisk_log_size":2252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:24.242000+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.703796324,"stat_sum":{"num_bytes":1252433,"num_objects":118,"num_object_clones":38,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":38,"num_read":118,"num_read_kb":38794,"num_write":1352,"num_write_kb":71340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"89'745","reported_seq":914,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.595752+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T09:28:23.595752+0000","last_peered":"2024-03-09T09:28:23.595752+0000","last_clean":"2024-03-09T09:28:23.595752+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T09:28:23.595752+0000","last_undegraded":"2024-03-09T09:28:23.595752+0000","last_fullsized":"2024-03-09T09:28:23.595752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":51,"num_read_kb":0,"num_write":808,"num_write_kb":91456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'2160","reported_seq":5146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.038761+0000","last_change":"2024-03-09T09:27:09.692723+0000","last_active":"2024-03-09T09:29:14.038761+0000","last_peered":"2024-03-09T09:29:14.038761+0000","last_clean":"2024-03-09T09:29:14.038761+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T09:29:14.038761+0000","last_undegraded":"2024-03-09T09:29:14.038761+0000","last_fullsized":"2024-03-09T09:29:14.038761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2160,"log_dups_size":0,"ondisk_log_size":2160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:29:50.385379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.48044756999999999,"stat_sum":{"num_bytes":498978,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":114,"num_read_kb":18251,"num_write":1268,"num_write_kb":37672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"89'892","reported_seq":1051,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.657948+0000","last_change":"2024-03-09T08:52:28.573074+0000","last_active":"2024-03-09T09:28:26.657948+0000","last_peered":"2024-03-09T09:28:26.657948+0000","last_clean":"2024-03-09T09:28:26.657948+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T09:28:26.657948+0000","last_undegraded":"2024-03-09T09:28:26.657948+0000","last_fullsized":"2024-03-09T09:28:26.657948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:50.600408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":939,"num_write_kb":120134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"89'2266","reported_seq":5169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924644+0000","last_change":"2024-03-09T09:27:34.614436+0000","last_active":"2024-03-09T09:29:17.924644+0000","last_peered":"2024-03-09T09:29:17.924644+0000","last_clean":"2024-03-09T09:29:17.924644+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T09:29:17.924644+0000","last_undegraded":"2024-03-09T09:29:17.924644+0000","last_fullsized":"2024-03-09T09:29:17.924644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2266,"log_dups_size":0,"ondisk_log_size":2266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:18:23.796765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.3826170339999999,"stat_sum":{"num_bytes":1068833,"num_objects":92,"num_object_clones":30,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":30,"num_read":126,"num_read_kb":14571,"num_write":1308,"num_write_kb":45011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"89'674","reported_seq":840,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:58.111396+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T09:27:58.111396+0000","last_peered":"2024-03-09T09:27:58.111396+0000","last_clean":"2024-03-09T09:27:58.111396+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T09:27:58.111396+0000","last_undegraded":"2024-03-09T09:27:58.111396+0000","last_fullsized":"2024-03-09T09:27:58.111396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":67,"num_read_kb":0,"num_write":775,"num_write_kb":140765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"89'2379","reported_seq":5342,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.635615+0000","last_change":"2024-03-09T09:27:14.117823+0000","last_active":"2024-03-09T09:28:23.635615+0000","last_peered":"2024-03-09T09:28:23.635615+0000","last_clean":"2024-03-09T09:28:23.635615+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T09:28:23.635615+0000","last_undegraded":"2024-03-09T09:28:23.635615+0000","last_fullsized":"2024-03-09T09:28:23.635615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2379,"log_dups_size":0,"ondisk_log_size":2379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:49:44.890858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.653915281,"stat_sum":{"num_bytes":2793747,"num_objects":148,"num_object_clones":53,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":53,"num_read":138,"num_read_kb":27086,"num_write":1428,"num_write_kb":43660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"89'531","reported_seq":689,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.454336+0000","last_change":"2024-03-09T08:52:28.577128+0000","last_active":"2024-03-09T09:28:14.454336+0000","last_peered":"2024-03-09T09:28:14.454336+0000","last_clean":"2024-03-09T09:28:14.454336+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T09:28:14.454336+0000","last_undegraded":"2024-03-09T09:28:14.454336+0000","last_fullsized":"2024-03-09T09:28:14.454336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:41.801752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":543,"num_write_kb":111605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"89'2275","reported_seq":5173,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.920377+0000","last_change":"2024-03-09T09:27:34.731982+0000","last_active":"2024-03-09T09:29:17.920377+0000","last_peered":"2024-03-09T09:29:17.920377+0000","last_clean":"2024-03-09T09:29:17.920377+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T09:29:17.920377+0000","last_undegraded":"2024-03-09T09:29:17.920377+0000","last_fullsized":"2024-03-09T09:29:17.920377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2275,"log_dups_size":0,"ondisk_log_size":2275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:34:13.555244+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4571779879999998,"stat_sum":{"num_bytes":3086231,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":121,"num_read_kb":9749,"num_write":1336,"num_write_kb":28981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"89'459","reported_seq":625,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:52.092030+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T09:27:52.092030+0000","last_peered":"2024-03-09T09:27:52.092030+0000","last_clean":"2024-03-09T09:27:52.092030+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T09:27:52.092030+0000","last_undegraded":"2024-03-09T09:27:52.092030+0000","last_fullsized":"2024-03-09T09:27:52.092030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":55,"num_read_kb":0,"num_write":535,"num_write_kb":83331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'2158","reported_seq":5152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.068307+0000","last_change":"2024-03-09T09:27:13.836207+0000","last_active":"2024-03-09T09:29:14.068307+0000","last_peered":"2024-03-09T09:29:14.068307+0000","last_clean":"2024-03-09T09:29:14.068307+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T09:29:14.068307+0000","last_undegraded":"2024-03-09T09:29:14.068307+0000","last_fullsized":"2024-03-09T09:29:14.068307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2158,"log_dups_size":0,"ondisk_log_size":2158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:20:22.564401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.3715434019999999,"stat_sum":{"num_bytes":1052383,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":116,"num_read_kb":6444,"num_write":1289,"num_write_kb":22757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"89'569","reported_seq":727,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:51.106524+0000","last_change":"2024-03-09T08:52:28.577069+0000","last_active":"2024-03-09T09:27:51.106524+0000","last_peered":"2024-03-09T09:27:51.106524+0000","last_clean":"2024-03-09T09:27:51.106524+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T09:27:51.106524+0000","last_undegraded":"2024-03-09T09:27:51.106524+0000","last_fullsized":"2024-03-09T09:27:51.106524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:09:02.410915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":598,"num_write_kb":95071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"89'2154","reported_seq":5089,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.560248+0000","last_change":"2024-03-09T09:27:32.079362+0000","last_active":"2024-03-09T09:28:32.560248+0000","last_peered":"2024-03-09T09:28:32.560248+0000","last_clean":"2024-03-09T09:28:32.560248+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T09:28:32.560248+0000","last_undegraded":"2024-03-09T09:28:32.560248+0000","last_fullsized":"2024-03-09T09:28:32.560248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2154,"log_dups_size":0,"ondisk_log_size":2154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:14:45.458685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.709485658,"stat_sum":{"num_bytes":1197226,"num_objects":92,"num_object_clones":31,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":31,"num_read":105,"num_read_kb":17945,"num_write":1287,"num_write_kb":45184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"89'2226","reported_seq":5194,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.062389+0000","last_change":"2024-03-09T09:27:07.936292+0000","last_active":"2024-03-09T09:29:14.062389+0000","last_peered":"2024-03-09T09:29:14.062389+0000","last_clean":"2024-03-09T09:29:14.062389+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T09:29:14.062389+0000","last_undegraded":"2024-03-09T09:29:14.062389+0000","last_fullsized":"2024-03-09T09:29:14.062389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2226,"log_dups_size":0,"ondisk_log_size":2226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:15:17.296354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.2630355259999999,"stat_sum":{"num_bytes":1174090,"num_objects":98,"num_object_clones":35,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":35,"num_read":98,"num_read_kb":18109,"num_write":1319,"num_write_kb":36158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"89'817","reported_seq":985,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.795764+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T09:28:14.795764+0000","last_peered":"2024-03-09T09:28:14.795764+0000","last_clean":"2024-03-09T09:28:14.795764+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T09:28:14.795764+0000","last_undegraded":"2024-03-09T09:28:14.795764+0000","last_fullsized":"2024-03-09T09:28:14.795764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":48,"num_read_kb":0,"num_write":879,"num_write_kb":99632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"89'743","reported_seq":903,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.514530+0000","last_change":"2024-03-09T08:52:28.570631+0000","last_active":"2024-03-09T09:28:31.514530+0000","last_peered":"2024-03-09T09:28:31.514530+0000","last_clean":"2024-03-09T09:28:31.514530+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T09:28:31.514530+0000","last_undegraded":"2024-03-09T09:28:31.514530+0000","last_fullsized":"2024-03-09T09:28:31.514530+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:06.630704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971838,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":820,"num_write_kb":111945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"89'2187","reported_seq":5097,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.736598+0000","last_change":"2024-03-09T09:27:30.893758+0000","last_active":"2024-03-09T09:28:56.736598+0000","last_peered":"2024-03-09T09:28:56.736598+0000","last_clean":"2024-03-09T09:28:56.736598+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T09:28:56.736598+0000","last_undegraded":"2024-03-09T09:28:56.736598+0000","last_fullsized":"2024-03-09T09:28:56.736598+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2187,"log_dups_size":0,"ondisk_log_size":2187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:16.469362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.63600609399999997,"stat_sum":{"num_bytes":1331264,"num_objects":108,"num_object_clones":38,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":38,"num_read":110,"num_read_kb":9978,"num_write":1295,"num_write_kb":30848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'2264","reported_seq":5240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.036656+0000","last_change":"2024-03-09T09:27:12.385283+0000","last_active":"2024-03-09T09:29:14.036656+0000","last_peered":"2024-03-09T09:29:14.036656+0000","last_clean":"2024-03-09T09:29:14.036656+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T09:29:14.036656+0000","last_undegraded":"2024-03-09T09:29:14.036656+0000","last_fullsized":"2024-03-09T09:29:14.036656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2264,"log_dups_size":0,"ondisk_log_size":2264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:24:11.227698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.69905185199999997,"stat_sum":{"num_bytes":1031167,"num_objects":108,"num_object_clones":39,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":39,"num_read":109,"num_read_kb":22315,"num_write":1384,"num_write_kb":51621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'445","reported_seq":614,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.253653+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T09:28:16.253653+0000","last_peered":"2024-03-09T09:28:16.253653+0000","last_clean":"2024-03-09T09:28:16.253653+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T09:28:16.253653+0000","last_undegraded":"2024-03-09T09:28:16.253653+0000","last_fullsized":"2024-03-09T09:28:16.253653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:45:45.612953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":467,"num_write_kb":115624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"89'633","reported_seq":793,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.652095+0000","last_change":"2024-03-09T08:52:28.577110+0000","last_active":"2024-03-09T09:28:26.652095+0000","last_peered":"2024-03-09T09:28:26.652095+0000","last_clean":"2024-03-09T09:28:26.652095+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T09:28:26.652095+0000","last_undegraded":"2024-03-09T09:28:26.652095+0000","last_fullsized":"2024-03-09T09:28:26.652095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:57.197373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":674,"num_write_kb":99647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"89'2250","reported_seq":5168,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.732049+0000","last_change":"2024-03-09T09:27:34.263088+0000","last_active":"2024-03-09T09:28:56.732049+0000","last_peered":"2024-03-09T09:28:56.732049+0000","last_clean":"2024-03-09T09:28:56.732049+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T09:28:56.732049+0000","last_undegraded":"2024-03-09T09:28:56.732049+0000","last_fullsized":"2024-03-09T09:28:56.732049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2250,"log_dups_size":0,"ondisk_log_size":2250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:15:00.105256+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":3.9938263410000001,"stat_sum":{"num_bytes":1661343,"num_objects":107,"num_object_clones":33,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":33,"num_read":118,"num_read_kb":35803,"num_write":1365,"num_write_kb":51532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'2351","reported_seq":5334,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.039556+0000","last_change":"2024-03-09T09:27:12.068457+0000","last_active":"2024-03-09T09:29:14.039556+0000","last_peered":"2024-03-09T09:29:14.039556+0000","last_clean":"2024-03-09T09:29:14.039556+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T09:29:14.039556+0000","last_undegraded":"2024-03-09T09:29:14.039556+0000","last_fullsized":"2024-03-09T09:29:14.039556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2351,"log_dups_size":0,"ondisk_log_size":2351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:13.795599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.380889329,"stat_sum":{"num_bytes":1156693,"num_objects":132,"num_object_clones":46,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":46,"num_read":122,"num_read_kb":38030,"num_write":1453,"num_write_kb":61557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"89'644","reported_seq":813,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.817502+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T09:28:14.817502+0000","last_peered":"2024-03-09T09:28:14.817502+0000","last_clean":"2024-03-09T09:28:14.817502+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T09:28:14.817502+0000","last_undegraded":"2024-03-09T09:28:14.817502+0000","last_fullsized":"2024-03-09T09:28:14.817502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:21:59.990396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":715,"num_write_kb":95464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"89'626","reported_seq":786,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:27.476620+0000","last_change":"2024-03-09T08:52:28.576254+0000","last_active":"2024-03-09T09:28:27.476620+0000","last_peered":"2024-03-09T09:28:27.476620+0000","last_clean":"2024-03-09T09:28:27.476620+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T09:28:27.476620+0000","last_undegraded":"2024-03-09T09:28:27.476620+0000","last_fullsized":"2024-03-09T09:28:27.476620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:21:56.330955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748758,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":685,"num_write_kb":95366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"89'2201","reported_seq":5127,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.920162+0000","last_change":"2024-03-09T09:27:34.731118+0000","last_active":"2024-03-09T09:29:17.920162+0000","last_peered":"2024-03-09T09:29:17.920162+0000","last_clean":"2024-03-09T09:29:17.920162+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T09:29:17.920162+0000","last_undegraded":"2024-03-09T09:29:17.920162+0000","last_fullsized":"2024-03-09T09:29:17.920162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2201,"log_dups_size":0,"ondisk_log_size":2201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:12.794501+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4556873890000004,"stat_sum":{"num_bytes":4719568,"num_objects":111,"num_object_clones":34,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":34,"num_read":112,"num_read_kb":26370,"num_write":1323,"num_write_kb":45566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"89'2296","reported_seq":5256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.039509+0000","last_change":"2024-03-09T09:27:08.625397+0000","last_active":"2024-03-09T09:29:14.039509+0000","last_peered":"2024-03-09T09:29:14.039509+0000","last_clean":"2024-03-09T09:29:14.039509+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T09:29:14.039509+0000","last_undegraded":"2024-03-09T09:29:14.039509+0000","last_fullsized":"2024-03-09T09:29:14.039509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2296,"log_dups_size":0,"ondisk_log_size":2296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:26:08.206478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0901323600000001,"stat_sum":{"num_bytes":867761,"num_objects":98,"num_object_clones":31,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":31,"num_read":98,"num_read_kb":37765,"num_write":1389,"num_write_kb":74512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'434","reported_seq":603,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.795677+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T09:28:14.795677+0000","last_peered":"2024-03-09T09:28:14.795677+0000","last_clean":"2024-03-09T09:28:14.795677+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T09:28:14.795677+0000","last_undegraded":"2024-03-09T09:28:14.795677+0000","last_fullsized":"2024-03-09T09:28:14.795677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":69,"num_read_kb":0,"num_write":510,"num_write_kb":91498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"89'614","reported_seq":773,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.546373+0000","last_change":"2024-03-09T08:52:28.573638+0000","last_active":"2024-03-09T09:28:26.546373+0000","last_peered":"2024-03-09T09:28:26.546373+0000","last_clean":"2024-03-09T09:28:26.546373+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T09:28:26.546373+0000","last_undegraded":"2024-03-09T09:28:26.546373+0000","last_fullsized":"2024-03-09T09:28:26.546373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:02.478112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":666,"num_write_kb":107635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"89'2336","reported_seq":5260,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.951528+0000","last_change":"2024-03-09T09:27:34.495452+0000","last_active":"2024-03-09T09:29:17.951528+0000","last_peered":"2024-03-09T09:29:17.951528+0000","last_clean":"2024-03-09T09:29:17.951528+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T09:29:17.951528+0000","last_undegraded":"2024-03-09T09:29:17.951528+0000","last_fullsized":"2024-03-09T09:29:17.951528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2336,"log_dups_size":0,"ondisk_log_size":2336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:06.642191+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.1575832950000002,"stat_sum":{"num_bytes":392237,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":136,"num_read_kb":53607,"num_write":1404,"num_write_kb":81678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"89'2175","reported_seq":5267,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.022103+0000","last_change":"2024-03-09T09:27:09.510565+0000","last_active":"2024-03-09T09:29:14.022103+0000","last_peered":"2024-03-09T09:29:14.022103+0000","last_clean":"2024-03-09T09:29:14.022103+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T09:29:14.022103+0000","last_undegraded":"2024-03-09T09:29:14.022103+0000","last_fullsized":"2024-03-09T09:29:14.022103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:04:13.602716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.38281627099999999,"stat_sum":{"num_bytes":951718,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":197,"num_read_kb":25950,"num_write":1332,"num_write_kb":45421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'345","reported_seq":514,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:58.111717+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T09:27:58.111717+0000","last_peered":"2024-03-09T09:27:58.111717+0000","last_clean":"2024-03-09T09:27:58.111717+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T09:27:58.111717+0000","last_undegraded":"2024-03-09T09:27:58.111717+0000","last_fullsized":"2024-03-09T09:27:58.111717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:35:38.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":397,"num_write_kb":78578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"89'1700","reported_seq":2916,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.642425+0000","last_change":"2024-03-09T08:52:28.575630+0000","last_active":"2024-03-09T09:28:29.642425+0000","last_peered":"2024-03-09T09:28:29.642425+0000","last_clean":"2024-03-09T09:28:29.642425+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T09:28:29.642425+0000","last_undegraded":"2024-03-09T09:28:29.642425+0000","last_fullsized":"2024-03-09T09:28:29.642425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1700,"log_dups_size":0,"ondisk_log_size":1700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:48.210921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":1117,"num_read_kb":4814,"num_write":1762,"num_write_kb":112708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"89'2107","reported_seq":5058,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.735789+0000","last_change":"2024-03-09T09:27:34.095045+0000","last_active":"2024-03-09T09:28:56.735789+0000","last_peered":"2024-03-09T09:28:56.735789+0000","last_clean":"2024-03-09T09:28:56.735789+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T09:28:56.735789+0000","last_undegraded":"2024-03-09T09:28:56.735789+0000","last_fullsized":"2024-03-09T09:28:56.735789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2107,"log_dups_size":0,"ondisk_log_size":2107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:05:04.126062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.8297685179999998,"stat_sum":{"num_bytes":268964,"num_objects":107,"num_object_clones":32,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":32,"num_read":107,"num_read_kb":24622,"num_write":1287,"num_write_kb":41444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"89'2254","reported_seq":5300,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.043906+0000","last_change":"2024-03-09T09:27:06.300751+0000","last_active":"2024-03-09T09:29:14.043906+0000","last_peered":"2024-03-09T09:29:14.043906+0000","last_clean":"2024-03-09T09:29:14.043906+0000","last_became_active":"2024-03-09T08:53:00.240844+0000","last_became_peered":"2024-03-09T08:53:00.240844+0000","last_unstale":"2024-03-09T09:29:14.043906+0000","last_undegraded":"2024-03-09T09:29:14.043906+0000","last_fullsized":"2024-03-09T09:29:14.043906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2254,"log_dups_size":0,"ondisk_log_size":2254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:21:13.503778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.78540855600000004,"stat_sum":{"num_bytes":1854558,"num_objects":126,"num_object_clones":46,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":46,"num_read":216,"num_read_kb":39239,"num_write":1332,"num_write_kb":71797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"89'1558","reported_seq":1727,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:07.567056+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T09:29:07.567056+0000","last_peered":"2024-03-09T09:29:07.567056+0000","last_clean":"2024-03-09T09:29:07.567056+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T09:29:07.567056+0000","last_undegraded":"2024-03-09T09:29:07.567056+0000","last_fullsized":"2024-03-09T09:29:07.567056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1558,"log_dups_size":0,"ondisk_log_size":1558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:30.201471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":57,"num_read_kb":0,"num_write":3420,"num_write_kb":84853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"89'672","reported_seq":833,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.478737+0000","last_change":"2024-03-09T08:52:28.586702+0000","last_active":"2024-03-09T09:28:31.478737+0000","last_peered":"2024-03-09T09:28:31.478737+0000","last_clean":"2024-03-09T09:28:31.478737+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T09:28:31.478737+0000","last_undegraded":"2024-03-09T09:28:31.478737+0000","last_fullsized":"2024-03-09T09:28:31.478737+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:09.201224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":71,"num_read_kb":0,"num_write":764,"num_write_kb":87842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"89'2333","reported_seq":5217,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.713975+0000","last_change":"2024-03-09T09:27:30.887844+0000","last_active":"2024-03-09T09:28:56.713975+0000","last_peered":"2024-03-09T09:28:56.713975+0000","last_clean":"2024-03-09T09:28:56.713975+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T09:28:56.713975+0000","last_undegraded":"2024-03-09T09:28:56.713975+0000","last_fullsized":"2024-03-09T09:28:56.713975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2333,"log_dups_size":0,"ondisk_log_size":2333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:11:46.526489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65806592200000003,"stat_sum":{"num_bytes":2259601,"num_objects":134,"num_object_clones":47,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":47,"num_read":108,"num_read_kb":22295,"num_write":1398,"num_write_kb":43486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"89'2181","reported_seq":5178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.005893+0000","last_change":"2024-03-09T09:27:09.896878+0000","last_active":"2024-03-09T09:29:14.005893+0000","last_peered":"2024-03-09T09:29:14.005893+0000","last_clean":"2024-03-09T09:29:14.005893+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T09:29:14.005893+0000","last_undegraded":"2024-03-09T09:29:14.005893+0000","last_fullsized":"2024-03-09T09:29:14.005893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2181,"log_dups_size":0,"ondisk_log_size":2181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:38:09.744644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.96663918400000004,"stat_sum":{"num_bytes":2499803,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":112,"num_read_kb":5934,"num_write":1320,"num_write_kb":25592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"89'642","reported_seq":811,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.817501+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T09:28:14.817501+0000","last_peered":"2024-03-09T09:28:14.817501+0000","last_clean":"2024-03-09T09:28:14.817501+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T09:28:14.817501+0000","last_undegraded":"2024-03-09T09:28:14.817501+0000","last_fullsized":"2024-03-09T09:28:14.817501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":75,"num_read_kb":0,"num_write":726,"num_write_kb":96134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"89'573","reported_seq":734,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:13.273576+0000","last_change":"2024-03-09T08:52:28.580879+0000","last_active":"2024-03-09T09:29:13.273576+0000","last_peered":"2024-03-09T09:29:13.273576+0000","last_clean":"2024-03-09T09:29:13.273576+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T09:29:13.273576+0000","last_undegraded":"2024-03-09T09:29:13.273576+0000","last_fullsized":"2024-03-09T09:29:13.273576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:19:44.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6515690,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":614,"num_write_kb":72903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"89'2273","reported_seq":5171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.939855+0000","last_change":"2024-03-09T09:27:31.029951+0000","last_active":"2024-03-09T09:28:30.939855+0000","last_peered":"2024-03-09T09:28:30.939855+0000","last_clean":"2024-03-09T09:28:30.939855+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T09:28:30.939855+0000","last_undegraded":"2024-03-09T09:28:30.939855+0000","last_fullsized":"2024-03-09T09:28:30.939855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2273,"log_dups_size":0,"ondisk_log_size":2273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:38.142661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.71811667999999995,"stat_sum":{"num_bytes":1846155,"num_objects":126,"num_object_clones":42,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":42,"num_read":111,"num_read_kb":21516,"num_write":1347,"num_write_kb":38338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"89'2284","reported_seq":5231,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.029735+0000","last_change":"2024-03-09T09:27:14.280348+0000","last_active":"2024-03-09T09:29:14.029735+0000","last_peered":"2024-03-09T09:29:14.029735+0000","last_clean":"2024-03-09T09:29:14.029735+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T09:29:14.029735+0000","last_undegraded":"2024-03-09T09:29:14.029735+0000","last_fullsized":"2024-03-09T09:29:14.029735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:42:30.439855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.815577539,"stat_sum":{"num_bytes":1586992,"num_objects":105,"num_object_clones":35,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":35,"num_read":97,"num_read_kb":33815,"num_write":1338,"num_write_kb":74580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"89'546","reported_seq":715,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:03.302448+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T09:28:03.302448+0000","last_peered":"2024-03-09T09:28:03.302448+0000","last_clean":"2024-03-09T09:28:03.302448+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T09:28:03.302448+0000","last_undegraded":"2024-03-09T09:28:03.302448+0000","last_fullsized":"2024-03-09T09:28:03.302448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":620,"num_write_kb":120324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"89'552","reported_seq":713,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:56.663234+0000","last_change":"2024-03-09T08:52:28.584457+0000","last_active":"2024-03-09T09:27:56.663234+0000","last_peered":"2024-03-09T09:27:56.663234+0000","last_clean":"2024-03-09T09:27:56.663234+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T09:27:56.663234+0000","last_undegraded":"2024-03-09T09:27:56.663234+0000","last_fullsized":"2024-03-09T09:27:56.663234+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:45.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":563,"num_write_kb":103368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"89'2260","reported_seq":5160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.938153+0000","last_change":"2024-03-09T09:27:31.026391+0000","last_active":"2024-03-09T09:28:30.938153+0000","last_peered":"2024-03-09T09:28:30.938153+0000","last_clean":"2024-03-09T09:28:30.938153+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T09:28:30.938153+0000","last_undegraded":"2024-03-09T09:28:30.938153+0000","last_fullsized":"2024-03-09T09:28:30.938153+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2260,"log_dups_size":0,"ondisk_log_size":2260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:49:57.313638+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.71599741500000003,"stat_sum":{"num_bytes":1873672,"num_objects":101,"num_object_clones":34,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":34,"num_read":103,"num_read_kb":19624,"num_write":1351,"num_write_kb":43427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"89'2260","reported_seq":5236,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.039799+0000","last_change":"2024-03-09T09:27:08.957379+0000","last_active":"2024-03-09T09:29:14.039799+0000","last_peered":"2024-03-09T09:29:14.039799+0000","last_clean":"2024-03-09T09:29:14.039799+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T09:29:14.039799+0000","last_undegraded":"2024-03-09T09:29:14.039799+0000","last_fullsized":"2024-03-09T09:29:14.039799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2260,"log_dups_size":0,"ondisk_log_size":2260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:57:34.155091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.4919903310000002,"stat_sum":{"num_bytes":2681803,"num_objects":128,"num_object_clones":43,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":43,"num_read":104,"num_read_kb":25781,"num_write":1386,"num_write_kb":55300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"89'737","reported_seq":906,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.252650+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T09:28:16.252650+0000","last_peered":"2024-03-09T09:28:16.252650+0000","last_clean":"2024-03-09T09:28:16.252650+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T09:28:16.252650+0000","last_undegraded":"2024-03-09T09:28:16.252650+0000","last_fullsized":"2024-03-09T09:28:16.252650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":833,"num_write_kb":103821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'561","reported_seq":722,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.554450+0000","last_change":"2024-03-09T08:52:28.578096+0000","last_active":"2024-03-09T09:27:57.554450+0000","last_peered":"2024-03-09T09:27:57.554450+0000","last_clean":"2024-03-09T09:27:57.554450+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T09:27:57.554450+0000","last_undegraded":"2024-03-09T09:27:57.554450+0000","last_fullsized":"2024-03-09T09:27:57.554450+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:42.645517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":52,"num_read_kb":0,"num_write":616,"num_write_kb":119874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"89'2163","reported_seq":5096,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.976905+0000","last_change":"2024-03-09T09:27:30.887413+0000","last_active":"2024-03-09T09:28:30.976905+0000","last_peered":"2024-03-09T09:28:30.976905+0000","last_clean":"2024-03-09T09:28:30.976905+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T09:28:30.976905+0000","last_undegraded":"2024-03-09T09:28:30.976905+0000","last_fullsized":"2024-03-09T09:28:30.976905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2163,"log_dups_size":0,"ondisk_log_size":2163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:41.847948+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.63049624100000001,"stat_sum":{"num_bytes":833172,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":114,"num_read_kb":17857,"num_write":1302,"num_write_kb":44488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"89'2250","reported_seq":5226,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.030973+0000","last_change":"2024-03-09T09:27:10.015883+0000","last_active":"2024-03-09T09:29:14.030973+0000","last_peered":"2024-03-09T09:29:14.030973+0000","last_clean":"2024-03-09T09:29:14.030973+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T09:29:14.030973+0000","last_undegraded":"2024-03-09T09:29:14.030973+0000","last_fullsized":"2024-03-09T09:29:14.030973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2250,"log_dups_size":0,"ondisk_log_size":2250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:23:02.726704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.18447864,"stat_sum":{"num_bytes":1122276,"num_objects":117,"num_object_clones":43,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":43,"num_read":101,"num_read_kb":18409,"num_write":1364,"num_write_kb":39400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"89'1830","reported_seq":3014,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.251658+0000","last_change":"2024-03-09T08:52:17.271847+0000","last_active":"2024-03-09T09:28:16.251658+0000","last_peered":"2024-03-09T09:28:16.251658+0000","last_clean":"2024-03-09T09:28:16.251658+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T09:28:16.251658+0000","last_undegraded":"2024-03-09T09:28:16.251658+0000","last_fullsized":"2024-03-09T09:28:16.251658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1830,"log_dups_size":0,"ondisk_log_size":1830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:27.418674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":1037,"num_read_kb":4764,"num_write":1840,"num_write_kb":108217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"89'404","reported_seq":564,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.641753+0000","last_change":"2024-03-09T08:52:28.588636+0000","last_active":"2024-03-09T09:28:29.641753+0000","last_peered":"2024-03-09T09:28:29.641753+0000","last_clean":"2024-03-09T09:28:29.641753+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T09:28:29.641753+0000","last_undegraded":"2024-03-09T09:28:29.641753+0000","last_fullsized":"2024-03-09T09:28:29.641753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:21.601182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":44,"num_read_kb":0,"num_write":445,"num_write_kb":78896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"89'2184","reported_seq":5126,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.923858+0000","last_change":"2024-03-09T09:27:34.760081+0000","last_active":"2024-03-09T09:29:17.923858+0000","last_peered":"2024-03-09T09:29:17.923858+0000","last_clean":"2024-03-09T09:29:17.923858+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T09:29:17.923858+0000","last_undegraded":"2024-03-09T09:29:17.923858+0000","last_fullsized":"2024-03-09T09:29:17.923858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2184,"log_dups_size":0,"ondisk_log_size":2184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:30:07.164361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4843044489999997,"stat_sum":{"num_bytes":1636574,"num_objects":123,"num_object_clones":44,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":44,"num_read":134,"num_read_kb":27719,"num_write":1302,"num_write_kb":59461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'2127","reported_seq":5120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.054947+0000","last_change":"2024-03-09T09:27:09.413913+0000","last_active":"2024-03-09T09:29:14.054947+0000","last_peered":"2024-03-09T09:29:14.054947+0000","last_clean":"2024-03-09T09:29:14.054947+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T09:29:14.054947+0000","last_undegraded":"2024-03-09T09:29:14.054947+0000","last_fullsized":"2024-03-09T09:29:14.054947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2127,"log_dups_size":0,"ondisk_log_size":2127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:36:18.637176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.289202336,"stat_sum":{"num_bytes":424328,"num_objects":113,"num_object_clones":35,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":35,"num_read":94,"num_read_kb":17266,"num_write":1284,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"89'580","reported_seq":750,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.074969+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T09:28:23.074969+0000","last_peered":"2024-03-09T09:28:23.074969+0000","last_clean":"2024-03-09T09:28:23.074969+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T09:28:23.074969+0000","last_undegraded":"2024-03-09T09:28:23.074969+0000","last_fullsized":"2024-03-09T09:28:23.074969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":598,"num_write_kb":82993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"89'1592","reported_seq":1752,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:48.494834+0000","last_change":"2024-03-09T08:52:28.578599+0000","last_active":"2024-03-09T09:28:48.494834+0000","last_peered":"2024-03-09T09:28:48.494834+0000","last_clean":"2024-03-09T09:28:48.494834+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T09:28:48.494834+0000","last_undegraded":"2024-03-09T09:28:48.494834+0000","last_fullsized":"2024-03-09T09:28:48.494834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1592,"log_dups_size":0,"ondisk_log_size":1592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:52:31.328248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":3473,"num_write_kb":84746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"89'2236","reported_seq":5167,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.928826+0000","last_change":"2024-03-09T09:27:34.687418+0000","last_active":"2024-03-09T09:29:17.928826+0000","last_peered":"2024-03-09T09:29:17.928826+0000","last_clean":"2024-03-09T09:29:17.928826+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T09:29:17.928826+0000","last_undegraded":"2024-03-09T09:29:17.928826+0000","last_fullsized":"2024-03-09T09:29:17.928826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2236,"log_dups_size":0,"ondisk_log_size":2236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:18:43.296322+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4503168610000001,"stat_sum":{"num_bytes":1665642,"num_objects":121,"num_object_clones":41,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":41,"num_read":126,"num_read_kb":42837,"num_write":1341,"num_write_kb":67584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'2308","reported_seq":5269,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.052272+0000","last_change":"2024-03-09T09:27:12.261576+0000","last_active":"2024-03-09T09:29:14.052272+0000","last_peered":"2024-03-09T09:29:14.052272+0000","last_clean":"2024-03-09T09:29:14.052272+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T09:29:14.052272+0000","last_undegraded":"2024-03-09T09:29:14.052272+0000","last_fullsized":"2024-03-09T09:29:14.052272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2308,"log_dups_size":0,"ondisk_log_size":2308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:07:46.894530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.56599842099999997,"stat_sum":{"num_bytes":1847778,"num_objects":137,"num_object_clones":49,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":49,"num_read":115,"num_read_kb":13750,"num_write":1371,"num_write_kb":55061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'626","reported_seq":795,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:11.134820+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T09:28:11.134820+0000","last_peered":"2024-03-09T09:28:11.134820+0000","last_clean":"2024-03-09T09:28:11.134820+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T09:28:11.134820+0000","last_undegraded":"2024-03-09T09:28:11.134820+0000","last_fullsized":"2024-03-09T09:28:11.134820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":698,"num_write_kb":70663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":326,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:40.174178+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T09:27:40.174178+0000","last_peered":"2024-03-09T09:27:40.174178+0000","last_clean":"2024-03-09T09:27:40.174178+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T09:27:40.174178+0000","last_undegraded":"2024-03-09T09:27:40.174178+0000","last_fullsized":"2024-03-09T09:27:40.174178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'680","reported_seq":841,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:02.565362+0000","last_change":"2024-03-09T08:52:28.577973+0000","last_active":"2024-03-09T09:28:02.565362+0000","last_peered":"2024-03-09T09:28:02.565362+0000","last_clean":"2024-03-09T09:28:02.565362+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T09:28:02.565362+0000","last_undegraded":"2024-03-09T09:28:02.565362+0000","last_fullsized":"2024-03-09T09:28:02.565362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:13.291622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":743,"num_write_kb":95766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"89'2243","reported_seq":5171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.944320+0000","last_change":"2024-03-09T09:27:30.919984+0000","last_active":"2024-03-09T09:28:30.944320+0000","last_peered":"2024-03-09T09:28:30.944320+0000","last_clean":"2024-03-09T09:28:30.944320+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T09:28:30.944320+0000","last_undegraded":"2024-03-09T09:28:30.944320+0000","last_fullsized":"2024-03-09T09:28:30.944320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2243,"log_dups_size":0,"ondisk_log_size":2243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:42:58.142985+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.66264078000000004,"stat_sum":{"num_bytes":792938,"num_objects":106,"num_object_clones":38,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":38,"num_read":125,"num_read_kb":17823,"num_write":1366,"num_write_kb":27629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'2242","reported_seq":5186,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.060199+0000","last_change":"2024-03-09T09:27:06.276304+0000","last_active":"2024-03-09T09:29:14.060199+0000","last_peered":"2024-03-09T09:29:14.060199+0000","last_clean":"2024-03-09T09:29:14.060199+0000","last_became_active":"2024-03-09T08:53:00.437859+0000","last_became_peered":"2024-03-09T08:53:00.437859+0000","last_unstale":"2024-03-09T09:29:14.060199+0000","last_undegraded":"2024-03-09T09:29:14.060199+0000","last_fullsized":"2024-03-09T09:29:14.060199+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2242,"log_dups_size":0,"ondisk_log_size":2242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:07:36.832520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.79260830400000004,"stat_sum":{"num_bytes":1456716,"num_objects":115,"num_object_clones":40,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":40,"num_read":109,"num_read_kb":18518,"num_write":1301,"num_write_kb":37816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"89'501","reported_seq":669,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.901581+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T09:28:14.901581+0000","last_peered":"2024-03-09T09:28:14.901581+0000","last_clean":"2024-03-09T09:28:14.901581+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T09:28:14.901581+0000","last_undegraded":"2024-03-09T09:28:14.901581+0000","last_fullsized":"2024-03-09T09:28:14.901581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:27.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":39,"num_read_kb":0,"num_write":538,"num_write_kb":99417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"89'703","reported_seq":863,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.545223+0000","last_change":"2024-03-09T08:52:28.589278+0000","last_active":"2024-03-09T09:28:26.545223+0000","last_peered":"2024-03-09T09:28:26.545223+0000","last_clean":"2024-03-09T09:28:26.545223+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T09:28:26.545223+0000","last_undegraded":"2024-03-09T09:28:26.545223+0000","last_fullsized":"2024-03-09T09:28:26.545223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:16:26.449275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":780,"num_write_kb":116155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"89'2189","reported_seq":5136,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.910057+0000","last_change":"2024-03-09T09:27:30.904584+0000","last_active":"2024-03-09T09:28:30.910057+0000","last_peered":"2024-03-09T09:28:30.910057+0000","last_clean":"2024-03-09T09:28:30.910057+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T09:28:30.910057+0000","last_undegraded":"2024-03-09T09:28:30.910057+0000","last_fullsized":"2024-03-09T09:28:30.910057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2189,"log_dups_size":0,"ondisk_log_size":2189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:42:21.944851+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.647865472,"stat_sum":{"num_bytes":999257,"num_objects":92,"num_object_clones":32,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":32,"num_read":116,"num_read_kb":42999,"num_write":1346,"num_write_kb":55653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"89'2187","reported_seq":5181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048275+0000","last_change":"2024-03-09T09:27:14.415883+0000","last_active":"2024-03-09T09:29:14.048275+0000","last_peered":"2024-03-09T09:29:14.048275+0000","last_clean":"2024-03-09T09:29:14.048275+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T09:29:14.048275+0000","last_undegraded":"2024-03-09T09:29:14.048275+0000","last_fullsized":"2024-03-09T09:29:14.048275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2187,"log_dups_size":0,"ondisk_log_size":2187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:34:33.950832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.951760857,"stat_sum":{"num_bytes":1333229,"num_objects":97,"num_object_clones":33,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":33,"num_read":100,"num_read_kb":17450,"num_write":1313,"num_write_kb":41729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"89'845","reported_seq":1013,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.817246+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T09:28:14.817246+0000","last_peered":"2024-03-09T09:28:14.817246+0000","last_clean":"2024-03-09T09:28:14.817246+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T09:28:14.817246+0000","last_undegraded":"2024-03-09T09:28:14.817246+0000","last_fullsized":"2024-03-09T09:28:14.817246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:26:35.574903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971641,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":912,"num_write_kb":111865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"89'796","reported_seq":956,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:06.427393+0000","last_change":"2024-03-09T08:52:28.583181+0000","last_active":"2024-03-09T09:28:06.427393+0000","last_peered":"2024-03-09T09:28:06.427393+0000","last_clean":"2024-03-09T09:28:06.427393+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T09:28:06.427393+0000","last_undegraded":"2024-03-09T09:28:06.427393+0000","last_fullsized":"2024-03-09T09:28:06.427393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:58:50.061324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":51,"num_read_kb":0,"num_write":872,"num_write_kb":99652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"89'2237","reported_seq":5132,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.931245+0000","last_change":"2024-03-09T09:27:34.640947+0000","last_active":"2024-03-09T09:29:17.931245+0000","last_peered":"2024-03-09T09:29:17.931245+0000","last_clean":"2024-03-09T09:29:17.931245+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T09:29:17.931245+0000","last_undegraded":"2024-03-09T09:29:17.931245+0000","last_fullsized":"2024-03-09T09:29:17.931245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2237,"log_dups_size":0,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:51:18.312332+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4098675719999996,"stat_sum":{"num_bytes":603203,"num_objects":100,"num_object_clones":32,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":32,"num_read":95,"num_read_kb":13936,"num_write":1338,"num_write_kb":41727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"89'2197","reported_seq":5190,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.040606+0000","last_change":"2024-03-09T09:27:13.656604+0000","last_active":"2024-03-09T09:29:14.040606+0000","last_peered":"2024-03-09T09:29:14.040606+0000","last_clean":"2024-03-09T09:29:14.040606+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T09:29:14.040606+0000","last_undegraded":"2024-03-09T09:29:14.040606+0000","last_fullsized":"2024-03-09T09:29:14.040606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2197,"log_dups_size":0,"ondisk_log_size":2197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:53:35.952871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.91280589400000001,"stat_sum":{"num_bytes":4608718,"num_objects":119,"num_object_clones":37,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":37,"num_read":119,"num_read_kb":26183,"num_write":1317,"num_write_kb":37436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'672","reported_seq":840,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:17.301970+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T09:28:17.301970+0000","last_peered":"2024-03-09T09:28:17.301970+0000","last_clean":"2024-03-09T09:28:17.301970+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T09:28:17.301970+0000","last_undegraded":"2024-03-09T09:28:17.301970+0000","last_fullsized":"2024-03-09T09:28:17.301970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:16.138216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":724,"num_write_kb":103814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"89'713","reported_seq":873,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:38.294529+0000","last_change":"2024-03-09T08:52:28.587637+0000","last_active":"2024-03-09T09:28:38.294529+0000","last_peered":"2024-03-09T09:28:38.294529+0000","last_clean":"2024-03-09T09:28:38.294529+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T09:28:38.294529+0000","last_undegraded":"2024-03-09T09:28:38.294529+0000","last_fullsized":"2024-03-09T09:28:38.294529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:31:32.362061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":59,"num_read_kb":0,"num_write":793,"num_write_kb":95673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"89'2354","reported_seq":5251,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.737741+0000","last_change":"2024-03-09T09:27:30.891372+0000","last_active":"2024-03-09T09:28:56.737741+0000","last_peered":"2024-03-09T09:28:56.737741+0000","last_clean":"2024-03-09T09:28:56.737741+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T09:28:56.737741+0000","last_undegraded":"2024-03-09T09:28:56.737741+0000","last_fullsized":"2024-03-09T09:28:56.737741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2354,"log_dups_size":0,"ondisk_log_size":2354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:45:35.757078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.634097468,"stat_sum":{"num_bytes":572008,"num_objects":135,"num_object_clones":47,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":47,"num_read":117,"num_read_kb":9742,"num_write":1431,"num_write_kb":32918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"89'2248","reported_seq":5227,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.069831+0000","last_change":"2024-03-09T09:27:13.609914+0000","last_active":"2024-03-09T09:29:14.069831+0000","last_peered":"2024-03-09T09:29:14.069831+0000","last_clean":"2024-03-09T09:29:14.069831+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T09:29:14.069831+0000","last_undegraded":"2024-03-09T09:29:14.069831+0000","last_fullsized":"2024-03-09T09:29:14.069831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2248,"log_dups_size":0,"ondisk_log_size":2248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:55:21.022399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.81694827699999994,"stat_sum":{"num_bytes":851628,"num_objects":105,"num_object_clones":33,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":33,"num_read":114,"num_read_kb":26314,"num_write":1339,"num_write_kb":37517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"89'520","reported_seq":687,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:58.112217+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T09:27:58.112217+0000","last_peered":"2024-03-09T09:27:58.112217+0000","last_clean":"2024-03-09T09:27:58.112217+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T09:27:58.112217+0000","last_undegraded":"2024-03-09T09:27:58.112217+0000","last_fullsized":"2024-03-09T09:27:58.112217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:10:34.698235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554454,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":604,"num_write_kb":91576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"89'683","reported_seq":843,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.545188+0000","last_change":"2024-03-09T08:52:28.582308+0000","last_active":"2024-03-09T09:28:26.545188+0000","last_peered":"2024-03-09T09:28:26.545188+0000","last_clean":"2024-03-09T09:28:26.545188+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T09:28:26.545188+0000","last_undegraded":"2024-03-09T09:28:26.545188+0000","last_fullsized":"2024-03-09T09:28:26.545188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:04.918893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":783,"num_write_kb":95503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"89'2304","reported_seq":5195,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.952036+0000","last_change":"2024-03-09T09:27:34.538055+0000","last_active":"2024-03-09T09:29:17.952036+0000","last_peered":"2024-03-09T09:29:17.952036+0000","last_clean":"2024-03-09T09:29:17.952036+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T09:29:17.952036+0000","last_undegraded":"2024-03-09T09:29:17.952036+0000","last_fullsized":"2024-03-09T09:29:17.952036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2304,"log_dups_size":0,"ondisk_log_size":2304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:30:20.481016+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.1951373749999998,"stat_sum":{"num_bytes":1358361,"num_objects":133,"num_object_clones":45,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":45,"num_read":125,"num_read_kb":25752,"num_write":1343,"num_write_kb":45310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"89'2285","reported_seq":5257,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.067779+0000","last_change":"2024-03-09T09:27:13.043792+0000","last_active":"2024-03-09T09:29:14.067779+0000","last_peered":"2024-03-09T09:29:14.067779+0000","last_clean":"2024-03-09T09:29:14.067779+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T09:29:14.067779+0000","last_undegraded":"2024-03-09T09:29:14.067779+0000","last_fullsized":"2024-03-09T09:29:14.067779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:24:58.260830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30155770100000001,"stat_sum":{"num_bytes":730111,"num_objects":106,"num_object_clones":35,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":35,"num_read":106,"num_read_kb":32982,"num_write":1381,"num_write_kb":56548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'750","reported_seq":917,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.253028+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T09:28:16.253028+0000","last_peered":"2024-03-09T09:28:16.253028+0000","last_clean":"2024-03-09T09:28:16.253028+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T09:28:16.253028+0000","last_undegraded":"2024-03-09T09:28:16.253028+0000","last_fullsized":"2024-03-09T09:28:16.253028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:31.859924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":74,"num_read_kb":0,"num_write":870,"num_write_kb":87759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"89'602","reported_seq":761,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.544148+0000","last_change":"2024-03-09T08:52:28.575975+0000","last_active":"2024-03-09T09:28:26.544148+0000","last_peered":"2024-03-09T09:28:26.544148+0000","last_clean":"2024-03-09T09:28:26.544148+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T09:28:26.544148+0000","last_undegraded":"2024-03-09T09:28:26.544148+0000","last_fullsized":"2024-03-09T09:28:26.544148+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:51:24.405316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":669,"num_write_kb":112438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"89'2184","reported_seq":5115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.919953+0000","last_change":"2024-03-09T09:27:34.711852+0000","last_active":"2024-03-09T09:29:17.919953+0000","last_peered":"2024-03-09T09:29:17.919953+0000","last_clean":"2024-03-09T09:29:17.919953+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T09:29:17.919953+0000","last_undegraded":"2024-03-09T09:29:17.919953+0000","last_fullsized":"2024-03-09T09:29:17.919953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2184,"log_dups_size":0,"ondisk_log_size":2184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:07:14.254525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4375742420000002,"stat_sum":{"num_bytes":1378177,"num_objects":100,"num_object_clones":26,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":26,"num_read":107,"num_read_kb":18463,"num_write":1316,"num_write_kb":41976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"89'2181","reported_seq":5172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.034483+0000","last_change":"2024-03-09T09:27:07.994121+0000","last_active":"2024-03-09T09:29:14.034483+0000","last_peered":"2024-03-09T09:29:14.034483+0000","last_clean":"2024-03-09T09:29:14.034483+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T09:29:14.034483+0000","last_undegraded":"2024-03-09T09:29:14.034483+0000","last_fullsized":"2024-03-09T09:29:14.034483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2181,"log_dups_size":0,"ondisk_log_size":2181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:07:11.820222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.3444653409999998,"stat_sum":{"num_bytes":3375625,"num_objects":103,"num_object_clones":31,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":31,"num_read":106,"num_read_kb":36185,"num_write":1311,"num_write_kb":52081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"89'386","reported_seq":553,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.986701+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T09:27:57.986701+0000","last_peered":"2024-03-09T09:27:57.986701+0000","last_clean":"2024-03-09T09:27:57.986701+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T09:27:57.986701+0000","last_undegraded":"2024-03-09T09:27:57.986701+0000","last_fullsized":"2024-03-09T09:27:57.986701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:07:29.577461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":412,"num_write_kb":99378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"89'560","reported_seq":959,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.643843+0000","last_change":"2024-03-09T08:52:28.588312+0000","last_active":"2024-03-09T09:28:29.643843+0000","last_peered":"2024-03-09T09:28:29.643843+0000","last_clean":"2024-03-09T09:28:29.643843+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T09:28:29.643843+0000","last_undegraded":"2024-03-09T09:28:29.643843+0000","last_fullsized":"2024-03-09T09:28:29.643843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:14:52.895618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007622,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":290,"num_read_kb":993,"num_write":609,"num_write_kb":88247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"89'2217","reported_seq":5123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.922142+0000","last_change":"2024-03-09T09:27:34.708914+0000","last_active":"2024-03-09T09:29:17.922142+0000","last_peered":"2024-03-09T09:29:17.922142+0000","last_clean":"2024-03-09T09:29:17.922142+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T09:29:17.922142+0000","last_undegraded":"2024-03-09T09:29:17.922142+0000","last_fullsized":"2024-03-09T09:29:17.922142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2217,"log_dups_size":0,"ondisk_log_size":2217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:43:47.385372+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4357263939999996,"stat_sum":{"num_bytes":1364696,"num_objects":131,"num_object_clones":50,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":50,"num_read":112,"num_read_kb":14161,"num_write":1321,"num_write_kb":29407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'2338","reported_seq":5297,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.068878+0000","last_change":"2024-03-09T09:27:13.390531+0000","last_active":"2024-03-09T09:29:14.068878+0000","last_peered":"2024-03-09T09:29:14.068878+0000","last_clean":"2024-03-09T09:29:14.068878+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T09:29:14.068878+0000","last_undegraded":"2024-03-09T09:29:14.068878+0000","last_fullsized":"2024-03-09T09:29:14.068878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2338,"log_dups_size":0,"ondisk_log_size":2338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:00:16.543112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64721804299999997,"stat_sum":{"num_bytes":2530176,"num_objects":100,"num_object_clones":36,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":36,"num_read":119,"num_read_kb":17537,"num_write":1383,"num_write_kb":31574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'267","reported_seq":434,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:40.238636+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T09:27:40.238636+0000","last_peered":"2024-03-09T09:27:40.238636+0000","last_clean":"2024-03-09T09:27:40.238636+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T09:27:40.238636+0000","last_undegraded":"2024-03-09T09:27:40.238636+0000","last_fullsized":"2024-03-09T09:27:40.238636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":313,"num_write_kb":62737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"89'830","reported_seq":988,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:03.286424+0000","last_change":"2024-03-09T08:52:28.585562+0000","last_active":"2024-03-09T09:29:03.286424+0000","last_peered":"2024-03-09T09:29:03.286424+0000","last_clean":"2024-03-09T09:29:03.286424+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T09:29:03.286424+0000","last_undegraded":"2024-03-09T09:29:03.286424+0000","last_fullsized":"2024-03-09T09:29:03.286424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:28:50.544795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":880,"num_write_kb":124192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"89'2228","reported_seq":5135,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.941262+0000","last_change":"2024-03-09T09:27:34.550507+0000","last_active":"2024-03-09T09:29:17.941262+0000","last_peered":"2024-03-09T09:29:17.941262+0000","last_clean":"2024-03-09T09:29:17.941262+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T09:29:17.941262+0000","last_undegraded":"2024-03-09T09:29:17.941262+0000","last_fullsized":"2024-03-09T09:29:17.941262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2228,"log_dups_size":0,"ondisk_log_size":2228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:36:18.970938+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.2113280319999999,"stat_sum":{"num_bytes":5678953,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":106,"num_read_kb":14364,"num_write":1330,"num_write_kb":47101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"89'2149","reported_seq":5155,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.041773+0000","last_change":"2024-03-09T09:27:09.238850+0000","last_active":"2024-03-09T09:29:14.041773+0000","last_peered":"2024-03-09T09:29:14.041773+0000","last_clean":"2024-03-09T09:29:14.041773+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T09:29:14.041773+0000","last_undegraded":"2024-03-09T09:29:14.041773+0000","last_fullsized":"2024-03-09T09:29:14.041773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2149,"log_dups_size":0,"ondisk_log_size":2149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:13:15.580902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.096588679999999996,"stat_sum":{"num_bytes":1266796,"num_objects":124,"num_object_clones":39,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":39,"num_read":107,"num_read_kb":17887,"num_write":1323,"num_write_kb":41057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"89'594","reported_seq":761,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.117171+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T09:27:57.117171+0000","last_peered":"2024-03-09T09:27:57.117171+0000","last_clean":"2024-03-09T09:27:57.117171+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T09:27:57.117171+0000","last_undegraded":"2024-03-09T09:27:57.117171+0000","last_fullsized":"2024-03-09T09:27:57.117171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:22:05.504557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":687,"num_write_kb":95812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"89'1176","reported_seq":1335,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.644757+0000","last_change":"2024-03-09T08:52:28.585697+0000","last_active":"2024-03-09T09:28:29.644757+0000","last_peered":"2024-03-09T09:28:29.644757+0000","last_clean":"2024-03-09T09:28:29.644757+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T09:28:29.644757+0000","last_undegraded":"2024-03-09T09:28:29.644757+0000","last_fullsized":"2024-03-09T09:28:29.644757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1176,"log_dups_size":0,"ondisk_log_size":1176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:50:23.095008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777306,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":1250,"num_write_kb":83773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"89'2364","reported_seq":5266,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.935770+0000","last_change":"2024-03-09T09:27:34.539439+0000","last_active":"2024-03-09T09:29:17.935770+0000","last_peered":"2024-03-09T09:29:17.935770+0000","last_clean":"2024-03-09T09:29:17.935770+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T09:29:17.935770+0000","last_undegraded":"2024-03-09T09:29:17.935770+0000","last_fullsized":"2024-03-09T09:29:17.935770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2364,"log_dups_size":0,"ondisk_log_size":2364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:04:32.627956+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2009582749999996,"stat_sum":{"num_bytes":273727,"num_objects":96,"num_object_clones":34,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":34,"num_read":130,"num_read_kb":37554,"num_write":1394,"num_write_kb":63694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"89'2221","reported_seq":5185,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.038375+0000","last_change":"2024-03-09T09:27:12.934974+0000","last_active":"2024-03-09T09:29:14.038375+0000","last_peered":"2024-03-09T09:29:14.038375+0000","last_clean":"2024-03-09T09:29:14.038375+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T09:29:14.038375+0000","last_undegraded":"2024-03-09T09:29:14.038375+0000","last_fullsized":"2024-03-09T09:29:14.038375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2221,"log_dups_size":0,"ondisk_log_size":2221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:00:09.930894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19064995900000001,"stat_sum":{"num_bytes":1055946,"num_objects":133,"num_object_clones":51,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":51,"num_read":117,"num_read_kb":18332,"num_write":1305,"num_write_kb":38006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"89'1040","reported_seq":1207,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.156049+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T09:28:23.156049+0000","last_peered":"2024-03-09T09:28:23.156049+0000","last_clean":"2024-03-09T09:28:23.156049+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T09:28:23.156049+0000","last_undegraded":"2024-03-09T09:28:23.156049+0000","last_fullsized":"2024-03-09T09:28:23.156049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1040,"log_dups_size":0,"ondisk_log_size":1040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:01.724043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":58,"num_read_kb":0,"num_write":1092,"num_write_kb":124321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"89'297","reported_seq":456,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.642610+0000","last_change":"2024-03-09T08:52:28.583685+0000","last_active":"2024-03-09T09:28:29.642610+0000","last_peered":"2024-03-09T09:28:29.642610+0000","last_clean":"2024-03-09T09:28:29.642610+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T09:28:29.642610+0000","last_undegraded":"2024-03-09T09:28:29.642610+0000","last_fullsized":"2024-03-09T09:28:29.642610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:58:00.899620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":328,"num_write_kb":62283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"89'2169","reported_seq":5098,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.750241+0000","last_change":"2024-03-09T09:27:32.064467+0000","last_active":"2024-03-09T09:28:56.750241+0000","last_peered":"2024-03-09T09:28:56.750241+0000","last_clean":"2024-03-09T09:28:56.750241+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T09:28:56.750241+0000","last_undegraded":"2024-03-09T09:28:56.750241+0000","last_fullsized":"2024-03-09T09:28:56.750241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2169,"log_dups_size":0,"ondisk_log_size":2169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:45:48.687413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.695386042,"stat_sum":{"num_bytes":3385389,"num_objects":130,"num_object_clones":42,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":42,"num_read":111,"num_read_kb":40727,"num_write":1306,"num_write_kb":67855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"89'2237","reported_seq":5201,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.072301+0000","last_change":"2024-03-09T09:27:08.841098+0000","last_active":"2024-03-09T09:29:14.072301+0000","last_peered":"2024-03-09T09:29:14.072301+0000","last_clean":"2024-03-09T09:29:14.072301+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T09:29:14.072301+0000","last_undegraded":"2024-03-09T09:29:14.072301+0000","last_fullsized":"2024-03-09T09:29:14.072301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2237,"log_dups_size":0,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:41:56.840106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.3666159750000002,"stat_sum":{"num_bytes":799596,"num_objects":94,"num_object_clones":30,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":30,"num_read":103,"num_read_kb":34868,"num_write":1326,"num_write_kb":46782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'1298","reported_seq":1465,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:18.712625+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T09:28:18.712625+0000","last_peered":"2024-03-09T09:28:18.712625+0000","last_clean":"2024-03-09T09:28:18.712625+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T09:28:18.712625+0000","last_undegraded":"2024-03-09T09:28:18.712625+0000","last_fullsized":"2024-03-09T09:28:18.712625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1298,"log_dups_size":0,"ondisk_log_size":1298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:41:10.578669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":80,"num_read_kb":0,"num_write":1400,"num_write_kb":80149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"89'411","reported_seq":570,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:08.402304+0000","last_change":"2024-03-09T08:52:28.568794+0000","last_active":"2024-03-09T09:28:08.402304+0000","last_peered":"2024-03-09T09:28:08.402304+0000","last_clean":"2024-03-09T09:28:08.402304+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T09:28:08.402304+0000","last_undegraded":"2024-03-09T09:28:08.402304+0000","last_fullsized":"2024-03-09T09:28:08.402304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:44:21.721511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":462,"num_write_kb":95092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"89'2267","reported_seq":5188,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.727032+0000","last_change":"2024-03-09T09:27:30.874038+0000","last_active":"2024-03-09T09:28:56.727032+0000","last_peered":"2024-03-09T09:28:56.727032+0000","last_clean":"2024-03-09T09:28:56.727032+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T09:28:56.727032+0000","last_undegraded":"2024-03-09T09:28:56.727032+0000","last_fullsized":"2024-03-09T09:28:56.727032+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2267,"log_dups_size":0,"ondisk_log_size":2267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:11:48.337332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64490501099999997,"stat_sum":{"num_bytes":1735737,"num_objects":103,"num_object_clones":32,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":32,"num_read":113,"num_read_kb":37575,"num_write":1379,"num_write_kb":57434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"89'2291","reported_seq":5242,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.022970+0000","last_change":"2024-03-09T09:27:09.683757+0000","last_active":"2024-03-09T09:29:14.022970+0000","last_peered":"2024-03-09T09:29:14.022970+0000","last_clean":"2024-03-09T09:29:14.022970+0000","last_became_active":"2024-03-09T08:53:00.254589+0000","last_became_peered":"2024-03-09T08:53:00.254589+0000","last_unstale":"2024-03-09T09:29:14.022970+0000","last_undegraded":"2024-03-09T09:29:14.022970+0000","last_fullsized":"2024-03-09T09:29:14.022970+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2291,"log_dups_size":0,"ondisk_log_size":2291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:40:21.450942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.471373141,"stat_sum":{"num_bytes":5513596,"num_objects":123,"num_object_clones":39,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":39,"num_read":114,"num_read_kb":29387,"num_write":1331,"num_write_kb":57801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'590","reported_seq":757,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.986880+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T09:27:57.986880+0000","last_peered":"2024-03-09T09:27:57.986880+0000","last_clean":"2024-03-09T09:27:57.986880+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T09:27:57.986880+0000","last_undegraded":"2024-03-09T09:27:57.986880+0000","last_fullsized":"2024-03-09T09:27:57.986880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":598,"num_write_kb":111422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"89'729","reported_seq":887,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.544483+0000","last_change":"2024-03-09T08:52:28.583697+0000","last_active":"2024-03-09T09:28:26.544483+0000","last_peered":"2024-03-09T09:28:26.544483+0000","last_clean":"2024-03-09T09:28:26.544483+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T09:28:26.544483+0000","last_undegraded":"2024-03-09T09:28:26.544483+0000","last_fullsized":"2024-03-09T09:28:26.544483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:41.552156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":81,"num_read_kb":0,"num_write":849,"num_write_kb":91941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"89'2528","reported_seq":5332,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.925475+0000","last_change":"2024-03-09T09:27:34.669817+0000","last_active":"2024-03-09T09:29:17.925475+0000","last_peered":"2024-03-09T09:29:17.925475+0000","last_clean":"2024-03-09T09:29:17.925475+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T09:29:17.925475+0000","last_undegraded":"2024-03-09T09:29:17.925475+0000","last_fullsized":"2024-03-09T09:29:17.925475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:42:51.478391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4377723019999999,"stat_sum":{"num_bytes":1160060,"num_objects":111,"num_object_clones":35,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":35,"num_read":113,"num_read_kb":9969,"num_write":1381,"num_write_kb":17733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"89'2165","reported_seq":5161,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.053013+0000","last_change":"2024-03-09T09:27:14.293488+0000","last_active":"2024-03-09T09:29:14.053013+0000","last_peered":"2024-03-09T09:29:14.053013+0000","last_clean":"2024-03-09T09:29:14.053013+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T09:29:14.053013+0000","last_undegraded":"2024-03-09T09:29:14.053013+0000","last_fullsized":"2024-03-09T09:29:14.053013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2165,"log_dups_size":0,"ondisk_log_size":2165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:12:46.882947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.82863014,"stat_sum":{"num_bytes":1951923,"num_objects":107,"num_object_clones":29,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":29,"num_read":96,"num_read_kb":37066,"num_write":1331,"num_write_kb":53456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"89'554","reported_seq":950,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:38.153367+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T09:28:38.153367+0000","last_peered":"2024-03-09T09:28:38.153367+0000","last_clean":"2024-03-09T09:28:38.153367+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T09:28:38.153367+0000","last_undegraded":"2024-03-09T09:28:38.153367+0000","last_fullsized":"2024-03-09T09:28:38.153367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:31.731626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007933,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":267,"num_read_kb":985,"num_write":604,"num_write_kb":87856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"89'461","reported_seq":619,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:55.398848+0000","last_change":"2024-03-09T08:52:28.586606+0000","last_active":"2024-03-09T09:27:55.398848+0000","last_peered":"2024-03-09T09:27:55.398848+0000","last_clean":"2024-03-09T09:27:55.398848+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T09:27:55.398848+0000","last_undegraded":"2024-03-09T09:27:55.398848+0000","last_fullsized":"2024-03-09T09:27:55.398848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:17:40.902403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":506,"num_write_kb":99411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"89'2186","reported_seq":5109,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.729816+0000","last_change":"2024-03-09T09:27:30.880846+0000","last_active":"2024-03-09T09:28:56.729816+0000","last_peered":"2024-03-09T09:28:56.729816+0000","last_clean":"2024-03-09T09:28:56.729816+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T09:28:56.729816+0000","last_undegraded":"2024-03-09T09:28:56.729816+0000","last_fullsized":"2024-03-09T09:28:56.729816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2186,"log_dups_size":0,"ondisk_log_size":2186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:47:22.327088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64610520400000004,"stat_sum":{"num_bytes":1318782,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":124,"num_read_kb":9990,"num_write":1296,"num_write_kb":22209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"89'667","reported_seq":829,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:27.475346+0000","last_change":"2024-03-09T08:52:28.586201+0000","last_active":"2024-03-09T09:28:27.475346+0000","last_peered":"2024-03-09T09:28:27.475346+0000","last_clean":"2024-03-09T09:28:27.475346+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T09:28:27.475346+0000","last_undegraded":"2024-03-09T09:28:27.475346+0000","last_fullsized":"2024-03-09T09:28:27.475346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:28:03.471646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":72,"num_read_kb":0,"num_write":777,"num_write_kb":92056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'2210","reported_seq":5186,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.003403+0000","last_change":"2024-03-09T09:27:10.069765+0000","last_active":"2024-03-09T09:29:14.003403+0000","last_peered":"2024-03-09T09:29:14.003403+0000","last_clean":"2024-03-09T09:29:14.003403+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T09:29:14.003403+0000","last_undegraded":"2024-03-09T09:29:14.003403+0000","last_fullsized":"2024-03-09T09:29:14.003403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2210,"log_dups_size":0,"ondisk_log_size":2210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:30:38.949657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.23951623,"stat_sum":{"num_bytes":1295292,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":13266,"num_write":1323,"num_write_kb":26325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"89'861","reported_seq":1028,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.251620+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T09:28:16.251620+0000","last_peered":"2024-03-09T09:28:16.251620+0000","last_clean":"2024-03-09T09:28:16.251620+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T09:28:16.251620+0000","last_undegraded":"2024-03-09T09:28:16.251620+0000","last_fullsized":"2024-03-09T09:28:16.251620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":45,"num_read_kb":0,"num_write":880,"num_write_kb":119947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"89'2142","reported_seq":5080,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.790657+0000","last_change":"2024-03-09T09:27:31.015518+0000","last_active":"2024-03-09T09:28:32.790657+0000","last_peered":"2024-03-09T09:28:32.790657+0000","last_clean":"2024-03-09T09:28:32.790657+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T09:28:32.790657+0000","last_undegraded":"2024-03-09T09:28:32.790657+0000","last_fullsized":"2024-03-09T09:28:32.790657+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2142,"log_dups_size":0,"ondisk_log_size":2142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:20:21.154856+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.70511981300000004,"stat_sum":{"num_bytes":1915718,"num_objects":107,"num_object_clones":37,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":37,"num_read":111,"num_read_kb":24546,"num_write":1285,"num_write_kb":44187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"89'545","reported_seq":706,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:15.180548+0000","last_change":"2024-03-09T08:52:28.585631+0000","last_active":"2024-03-09T09:28:15.180548+0000","last_peered":"2024-03-09T09:28:15.180548+0000","last_clean":"2024-03-09T09:28:15.180548+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T09:28:15.180548+0000","last_undegraded":"2024-03-09T09:28:15.180548+0000","last_fullsized":"2024-03-09T09:28:15.180548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:06:03.569759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":589,"num_write_kb":82799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'2475","reported_seq":5396,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.045469+0000","last_change":"2024-03-09T09:27:12.483134+0000","last_active":"2024-03-09T09:29:14.045469+0000","last_peered":"2024-03-09T09:29:14.045469+0000","last_clean":"2024-03-09T09:29:14.045469+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T09:29:14.045469+0000","last_undegraded":"2024-03-09T09:29:14.045469+0000","last_fullsized":"2024-03-09T09:29:14.045469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2475,"log_dups_size":0,"ondisk_log_size":2475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:53:15.813937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.79565864399999997,"stat_sum":{"num_bytes":2512650,"num_objects":93,"num_object_clones":29,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":29,"num_read":114,"num_read_kb":34867,"num_write":1447,"num_write_kb":63344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'400","reported_seq":566,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:04.123705+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T09:28:04.123705+0000","last_peered":"2024-03-09T09:28:04.123705+0000","last_clean":"2024-03-09T09:28:04.123705+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T09:28:04.123705+0000","last_undegraded":"2024-03-09T09:28:04.123705+0000","last_fullsized":"2024-03-09T09:28:04.123705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":480,"num_write_kb":79281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"89'2172","reported_seq":5110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.758184+0000","last_change":"2024-03-09T09:27:32.094130+0000","last_active":"2024-03-09T09:28:56.758184+0000","last_peered":"2024-03-09T09:28:56.758184+0000","last_clean":"2024-03-09T09:28:56.758184+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T09:28:56.758184+0000","last_undegraded":"2024-03-09T09:28:56.758184+0000","last_fullsized":"2024-03-09T09:28:56.758184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2172,"log_dups_size":0,"ondisk_log_size":2172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:02:28.116630+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.72318536,"stat_sum":{"num_bytes":768948,"num_objects":112,"num_object_clones":37,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":37,"num_read":126,"num_read_kb":26327,"num_write":1312,"num_write_kb":46360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"89'739","reported_seq":905,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.656661+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T09:27:57.656661+0000","last_peered":"2024-03-09T09:27:57.656661+0000","last_clean":"2024-03-09T09:27:57.656661+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T09:27:57.656661+0000","last_undegraded":"2024-03-09T09:27:57.656661+0000","last_fullsized":"2024-03-09T09:27:57.656661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":777,"num_write_kb":95156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"89'2306","reported_seq":5291,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.064156+0000","last_change":"2024-03-09T09:27:09.781774+0000","last_active":"2024-03-09T09:29:14.064156+0000","last_peered":"2024-03-09T09:29:14.064156+0000","last_clean":"2024-03-09T09:29:14.064156+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T09:29:14.064156+0000","last_undegraded":"2024-03-09T09:29:14.064156+0000","last_fullsized":"2024-03-09T09:29:14.064156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2306,"log_dups_size":0,"ondisk_log_size":2306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:44:18.347873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65597337700000002,"stat_sum":{"num_bytes":5623986,"num_objects":128,"num_object_clones":44,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":44,"num_read":148,"num_read_kb":35274,"num_write":1372,"num_write_kb":72006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"89'471","reported_seq":629,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.563819+0000","last_change":"2024-03-09T08:52:28.579315+0000","last_active":"2024-03-09T09:28:26.563819+0000","last_peered":"2024-03-09T09:28:26.563819+0000","last_clean":"2024-03-09T09:28:26.563819+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T09:28:26.563819+0000","last_undegraded":"2024-03-09T09:28:26.563819+0000","last_fullsized":"2024-03-09T09:28:26.563819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:57:17.301677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":514,"num_write_kb":86887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"89'2419","reported_seq":5301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.005816+0000","last_change":"2024-03-09T09:27:34.193425+0000","last_active":"2024-03-09T09:28:31.005816+0000","last_peered":"2024-03-09T09:28:31.005816+0000","last_clean":"2024-03-09T09:28:31.005816+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T09:28:31.005816+0000","last_undegraded":"2024-03-09T09:28:31.005816+0000","last_fullsized":"2024-03-09T09:28:31.005816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2419,"log_dups_size":0,"ondisk_log_size":2419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:25:58.792996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.9237505820000003,"stat_sum":{"num_bytes":5533714,"num_objects":147,"num_object_clones":51,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":51,"num_read":136,"num_read_kb":30612,"num_write":1437,"num_write_kb":51393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"89'616","reported_seq":781,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:13.970713+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T09:29:13.970713+0000","last_peered":"2024-03-09T09:29:13.970713+0000","last_clean":"2024-03-09T09:29:13.970713+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T09:29:13.970713+0000","last_undegraded":"2024-03-09T09:29:13.970713+0000","last_fullsized":"2024-03-09T09:29:13.970713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17329154,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":61,"num_read_kb":0,"num_write":696,"num_write_kb":108351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"89'2233","reported_seq":5212,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.032993+0000","last_change":"2024-03-09T09:27:08.660892+0000","last_active":"2024-03-09T09:29:14.032993+0000","last_peered":"2024-03-09T09:29:14.032993+0000","last_clean":"2024-03-09T09:29:14.032993+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T09:29:14.032993+0000","last_undegraded":"2024-03-09T09:29:14.032993+0000","last_fullsized":"2024-03-09T09:29:14.032993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2233,"log_dups_size":0,"ondisk_log_size":2233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:58:11.216375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.1639723850000001,"stat_sum":{"num_bytes":538194,"num_objects":89,"num_object_clones":30,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":30,"num_read":112,"num_read_kb":30041,"num_write":1325,"num_write_kb":47681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"89'768","reported_seq":925,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:08.255955+0000","last_change":"2024-03-09T08:52:28.585877+0000","last_active":"2024-03-09T09:28:08.255955+0000","last_peered":"2024-03-09T09:28:08.255955+0000","last_clean":"2024-03-09T09:28:08.255955+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T09:28:08.255955+0000","last_undegraded":"2024-03-09T09:28:08.255955+0000","last_fullsized":"2024-03-09T09:28:08.255955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:30:20.036385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":79,"num_read_kb":0,"num_write":862,"num_write_kb":140804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"89'2380","reported_seq":5295,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.944878+0000","last_change":"2024-03-09T09:27:34.560887+0000","last_active":"2024-03-09T09:29:17.944878+0000","last_peered":"2024-03-09T09:29:17.944878+0000","last_clean":"2024-03-09T09:29:17.944878+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T09:29:17.944878+0000","last_undegraded":"2024-03-09T09:29:17.944878+0000","last_fullsized":"2024-03-09T09:29:17.944878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2380,"log_dups_size":0,"ondisk_log_size":2380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:20:20.590247+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2173252530000003,"stat_sum":{"num_bytes":794839,"num_objects":125,"num_object_clones":42,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":42,"num_read":135,"num_read_kb":30019,"num_write":1425,"num_write_kb":47795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"89'2255","reported_seq":5234,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.034255+0000","last_change":"2024-03-09T09:27:07.515170+0000","last_active":"2024-03-09T09:29:14.034255+0000","last_peered":"2024-03-09T09:29:14.034255+0000","last_clean":"2024-03-09T09:29:14.034255+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T09:29:14.034255+0000","last_undegraded":"2024-03-09T09:29:14.034255+0000","last_fullsized":"2024-03-09T09:29:14.034255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2255,"log_dups_size":0,"ondisk_log_size":2255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:08.036556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.8676528270000001,"stat_sum":{"num_bytes":501236,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":17494,"num_write":1356,"num_write_kb":32845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"89'2131","reported_seq":5059,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.745807+0000","last_change":"2024-03-09T09:27:32.084492+0000","last_active":"2024-03-09T09:28:56.745807+0000","last_peered":"2024-03-09T09:28:56.745807+0000","last_clean":"2024-03-09T09:28:56.745807+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T09:28:56.745807+0000","last_undegraded":"2024-03-09T09:28:56.745807+0000","last_fullsized":"2024-03-09T09:28:56.745807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2131,"log_dups_size":0,"ondisk_log_size":2131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:22.480872+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":1.7132004649999999,"stat_sum":{"num_bytes":1622605,"num_objects":113,"num_object_clones":36,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":36,"num_read":103,"num_read_kb":9361,"num_write":1283,"num_write_kb":37703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"89'2356","reported_seq":5312,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.037599+0000","last_change":"2024-03-09T09:27:06.219951+0000","last_active":"2024-03-09T09:29:14.037599+0000","last_peered":"2024-03-09T09:29:14.037599+0000","last_clean":"2024-03-09T09:29:14.037599+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T09:29:14.037599+0000","last_undegraded":"2024-03-09T09:29:14.037599+0000","last_fullsized":"2024-03-09T09:29:14.037599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:15.950822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.73410826500000004,"stat_sum":{"num_bytes":968837,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":110,"num_read_kb":13537,"num_write":1403,"num_write_kb":47183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"89'2251","reported_seq":5137,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.925278+0000","last_change":"2024-03-09T09:27:34.720190+0000","last_active":"2024-03-09T09:29:17.925278+0000","last_peered":"2024-03-09T09:29:17.925278+0000","last_clean":"2024-03-09T09:29:17.925278+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T09:29:17.925278+0000","last_undegraded":"2024-03-09T09:29:17.925278+0000","last_fullsized":"2024-03-09T09:29:17.925278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2251,"log_dups_size":0,"ondisk_log_size":2251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:58:23.785495+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.446018724,"stat_sum":{"num_bytes":5408806,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":112,"num_read_kb":29960,"num_write":1313,"num_write_kb":48116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"89'2237","reported_seq":5234,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048940+0000","last_change":"2024-03-09T09:27:14.175435+0000","last_active":"2024-03-09T09:29:14.048940+0000","last_peered":"2024-03-09T09:29:14.048940+0000","last_clean":"2024-03-09T09:29:14.048940+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T09:29:14.048940+0000","last_undegraded":"2024-03-09T09:29:14.048940+0000","last_fullsized":"2024-03-09T09:29:14.048940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2237,"log_dups_size":0,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:29.862968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.706678991,"stat_sum":{"num_bytes":887348,"num_objects":110,"num_object_clones":37,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":37,"num_read":123,"num_read_kb":42348,"num_write":1334,"num_write_kb":66020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"89'2116","reported_seq":5064,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.060507+0000","last_change":"2024-03-09T09:27:32.079328+0000","last_active":"2024-03-09T09:28:31.060507+0000","last_peered":"2024-03-09T09:28:31.060507+0000","last_clean":"2024-03-09T09:28:31.060507+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T09:28:31.060507+0000","last_undegraded":"2024-03-09T09:28:31.060507+0000","last_fullsized":"2024-03-09T09:28:31.060507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2116,"log_dups_size":0,"ondisk_log_size":2116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:18.607812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.708282305,"stat_sum":{"num_bytes":596753,"num_objects":112,"num_object_clones":36,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":36,"num_read":111,"num_read_kb":13558,"num_write":1290,"num_write_kb":25078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"89'2395","reported_seq":5315,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.036196+0000","last_change":"2024-03-09T09:27:09.523471+0000","last_active":"2024-03-09T09:29:14.036196+0000","last_peered":"2024-03-09T09:29:14.036196+0000","last_clean":"2024-03-09T09:29:14.036196+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T09:29:14.036196+0000","last_undegraded":"2024-03-09T09:29:14.036196+0000","last_fullsized":"2024-03-09T09:29:14.036196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2395,"log_dups_size":0,"ondisk_log_size":2395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:57:42.927188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.39778581299999999,"stat_sum":{"num_bytes":1231522,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":112,"num_read_kb":16729,"num_write":1390,"num_write_kb":42674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"89'2182","reported_seq":5127,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.920533+0000","last_change":"2024-03-09T09:27:34.631961+0000","last_active":"2024-03-09T09:29:17.920533+0000","last_peered":"2024-03-09T09:29:17.920533+0000","last_clean":"2024-03-09T09:29:17.920533+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T09:29:17.920533+0000","last_undegraded":"2024-03-09T09:29:17.920533+0000","last_fullsized":"2024-03-09T09:29:17.920533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2182,"log_dups_size":0,"ondisk_log_size":2182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:11:08.985120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4007626340000003,"stat_sum":{"num_bytes":767001,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":115,"num_read_kb":17556,"num_write":1334,"num_write_kb":40745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"89'2227","reported_seq":5191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.070029+0000","last_change":"2024-03-09T09:27:12.489198+0000","last_active":"2024-03-09T09:29:14.070029+0000","last_peered":"2024-03-09T09:29:14.070029+0000","last_clean":"2024-03-09T09:29:14.070029+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T09:29:14.070029+0000","last_undegraded":"2024-03-09T09:29:14.070029+0000","last_fullsized":"2024-03-09T09:29:14.070029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2227,"log_dups_size":0,"ondisk_log_size":2227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:48:35.331040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.80249947799999999,"stat_sum":{"num_bytes":1587591,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":108,"num_read_kb":24972,"num_write":1321,"num_write_kb":50275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"89'2414","reported_seq":5260,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.985366+0000","last_change":"2024-03-09T09:27:33.932429+0000","last_active":"2024-03-09T09:28:30.985366+0000","last_peered":"2024-03-09T09:28:30.985366+0000","last_clean":"2024-03-09T09:28:30.985366+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T09:28:30.985366+0000","last_undegraded":"2024-03-09T09:28:30.985366+0000","last_fullsized":"2024-03-09T09:28:30.985366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2414,"log_dups_size":0,"ondisk_log_size":2414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:24.675125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.6671456120000001,"stat_sum":{"num_bytes":827266,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":108,"num_read_kb":21834,"num_write":1396,"num_write_kb":37656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"89'2190","reported_seq":5177,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.014853+0000","last_change":"2024-03-09T09:27:10.243190+0000","last_active":"2024-03-09T09:29:14.014853+0000","last_peered":"2024-03-09T09:29:14.014853+0000","last_clean":"2024-03-09T09:29:14.014853+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T09:29:14.014853+0000","last_undegraded":"2024-03-09T09:29:14.014853+0000","last_fullsized":"2024-03-09T09:29:14.014853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2190,"log_dups_size":0,"ondisk_log_size":2190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:08.187277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.410735039,"stat_sum":{"num_bytes":1374461,"num_objects":110,"num_object_clones":35,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":35,"num_read":104,"num_read_kb":29127,"num_write":1325,"num_write_kb":54062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"89'2222","reported_seq":5150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.925368+0000","last_change":"2024-03-09T09:27:34.622798+0000","last_active":"2024-03-09T09:29:17.925368+0000","last_peered":"2024-03-09T09:29:17.925368+0000","last_clean":"2024-03-09T09:29:17.925368+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T09:29:17.925368+0000","last_undegraded":"2024-03-09T09:29:17.925368+0000","last_fullsized":"2024-03-09T09:29:17.925368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2222,"log_dups_size":0,"ondisk_log_size":2222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:16:59.598968+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.393423125,"stat_sum":{"num_bytes":646888,"num_objects":117,"num_object_clones":37,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":37,"num_read":111,"num_read_kb":21616,"num_write":1343,"num_write_kb":41561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"89'2380","reported_seq":5353,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.053034+0000","last_change":"2024-03-09T09:27:08.090545+0000","last_active":"2024-03-09T09:29:14.053034+0000","last_peered":"2024-03-09T09:29:14.053034+0000","last_clean":"2024-03-09T09:29:14.053034+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T09:29:14.053034+0000","last_undegraded":"2024-03-09T09:29:14.053034+0000","last_fullsized":"2024-03-09T09:29:14.053034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2380,"log_dups_size":0,"ondisk_log_size":2380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:38:53.276797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.440348755,"stat_sum":{"num_bytes":1448398,"num_objects":130,"num_object_clones":44,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":44,"num_read":125,"num_read_kb":18804,"num_write":1435,"num_write_kb":32798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"89'2303","reported_seq":5205,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.859353+0000","last_change":"2024-03-09T09:27:30.874126+0000","last_active":"2024-03-09T09:28:30.859353+0000","last_peered":"2024-03-09T09:28:30.859353+0000","last_clean":"2024-03-09T09:28:30.859353+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T09:28:30.859353+0000","last_undegraded":"2024-03-09T09:28:30.859353+0000","last_fullsized":"2024-03-09T09:28:30.859353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2303,"log_dups_size":0,"ondisk_log_size":2303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:12:39.325227+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.64500296700000004,"stat_sum":{"num_bytes":1066603,"num_objects":98,"num_object_clones":32,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":32,"num_read":104,"num_read_kb":18198,"num_write":1399,"num_write_kb":47978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"89'2175","reported_seq":5136,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.046381+0000","last_change":"2024-03-09T09:27:12.333957+0000","last_active":"2024-03-09T09:29:14.046381+0000","last_peered":"2024-03-09T09:29:14.046381+0000","last_clean":"2024-03-09T09:29:14.046381+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T09:29:14.046381+0000","last_undegraded":"2024-03-09T09:29:14.046381+0000","last_fullsized":"2024-03-09T09:29:14.046381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:59:27.720581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.62298635300000005,"stat_sum":{"num_bytes":1371558,"num_objects":107,"num_object_clones":34,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":34,"num_read":92,"num_read_kb":30679,"num_write":1263,"num_write_kb":63255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"89'2203","reported_seq":5097,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.761136+0000","last_change":"2024-03-09T09:27:34.527780+0000","last_active":"2024-03-09T09:28:56.761136+0000","last_peered":"2024-03-09T09:28:56.761136+0000","last_clean":"2024-03-09T09:28:56.761136+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T09:28:56.761136+0000","last_undegraded":"2024-03-09T09:28:56.761136+0000","last_fullsized":"2024-03-09T09:28:56.761136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2203,"log_dups_size":0,"ondisk_log_size":2203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:33:42.613616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.1845584630000001,"stat_sum":{"num_bytes":1760124,"num_objects":108,"num_object_clones":35,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":35,"num_read":107,"num_read_kb":13622,"num_write":1279,"num_write_kb":27255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'2199","reported_seq":5181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.036836+0000","last_change":"2024-03-09T09:27:13.741409+0000","last_active":"2024-03-09T09:29:14.036836+0000","last_peered":"2024-03-09T09:29:14.036836+0000","last_clean":"2024-03-09T09:29:14.036836+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T09:29:14.036836+0000","last_undegraded":"2024-03-09T09:29:14.036836+0000","last_fullsized":"2024-03-09T09:29:14.036836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2199,"log_dups_size":0,"ondisk_log_size":2199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:03.762335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.2778432559999999,"stat_sum":{"num_bytes":1308027,"num_objects":118,"num_object_clones":42,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":42,"num_read":112,"num_read_kb":14330,"num_write":1310,"num_write_kb":25655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"89'2329","reported_seq":5225,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.875787+0000","last_change":"2024-03-09T09:27:34.655594+0000","last_active":"2024-03-09T09:29:17.875787+0000","last_peered":"2024-03-09T09:29:17.875787+0000","last_clean":"2024-03-09T09:29:17.875787+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T09:29:17.875787+0000","last_undegraded":"2024-03-09T09:29:17.875787+0000","last_fullsized":"2024-03-09T09:29:17.875787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2329,"log_dups_size":0,"ondisk_log_size":2329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:01:41.618756+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4171876489999997,"stat_sum":{"num_bytes":1480146,"num_objects":115,"num_object_clones":41,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":41,"num_read":120,"num_read_kb":31406,"num_write":1383,"num_write_kb":52041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"89'2175","reported_seq":5173,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.056157+0000","last_change":"2024-03-09T09:27:12.405710+0000","last_active":"2024-03-09T09:29:14.056157+0000","last_peered":"2024-03-09T09:29:14.056157+0000","last_clean":"2024-03-09T09:29:14.056157+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T09:29:14.056157+0000","last_undegraded":"2024-03-09T09:29:14.056157+0000","last_fullsized":"2024-03-09T09:29:14.056157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:12:10.053606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.72006586699999997,"stat_sum":{"num_bytes":650831,"num_objects":114,"num_object_clones":39,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":39,"num_read":114,"num_read_kb":46474,"num_write":1326,"num_write_kb":53541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"89'2284","reported_seq":5214,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.304433+0000","last_change":"2024-03-09T09:27:30.902599+0000","last_active":"2024-03-09T09:28:32.304433+0000","last_peered":"2024-03-09T09:28:32.304433+0000","last_clean":"2024-03-09T09:28:32.304433+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T09:28:32.304433+0000","last_undegraded":"2024-03-09T09:28:32.304433+0000","last_fullsized":"2024-03-09T09:28:32.304433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:54:27.990313+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.64248945300000004,"stat_sum":{"num_bytes":509009,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":145,"num_read_kb":34396,"num_write":1348,"num_write_kb":44845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"89'2311","reported_seq":5257,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.050683+0000","last_change":"2024-03-09T09:27:13.760270+0000","last_active":"2024-03-09T09:29:14.050683+0000","last_peered":"2024-03-09T09:29:14.050683+0000","last_clean":"2024-03-09T09:29:14.050683+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T09:29:14.050683+0000","last_undegraded":"2024-03-09T09:29:14.050683+0000","last_fullsized":"2024-03-09T09:29:14.050683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2311,"log_dups_size":0,"ondisk_log_size":2311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:54:17.411237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.291579509,"stat_sum":{"num_bytes":917223,"num_objects":100,"num_object_clones":30,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":30,"num_read":95,"num_read_kb":13353,"num_write":1387,"num_write_kb":42086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"89'2321","reported_seq":5211,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.932450+0000","last_change":"2024-03-09T09:27:34.549097+0000","last_active":"2024-03-09T09:29:17.932450+0000","last_peered":"2024-03-09T09:29:17.932450+0000","last_clean":"2024-03-09T09:29:17.932450+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T09:29:17.932450+0000","last_undegraded":"2024-03-09T09:29:17.932450+0000","last_fullsized":"2024-03-09T09:29:17.932450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2321,"log_dups_size":0,"ondisk_log_size":2321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:56:26.561322+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.2066621079999997,"stat_sum":{"num_bytes":1356079,"num_objects":97,"num_object_clones":28,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":28,"num_read":106,"num_read_kb":29896,"num_write":1386,"num_write_kb":51017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"89'2224","reported_seq":5214,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.042583+0000","last_change":"2024-03-09T09:27:09.439110+0000","last_active":"2024-03-09T09:29:14.042583+0000","last_peered":"2024-03-09T09:29:14.042583+0000","last_clean":"2024-03-09T09:29:14.042583+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T09:29:14.042583+0000","last_undegraded":"2024-03-09T09:29:14.042583+0000","last_fullsized":"2024-03-09T09:29:14.042583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2224,"log_dups_size":0,"ondisk_log_size":2224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:14:38.198734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.313168905,"stat_sum":{"num_bytes":1092268,"num_objects":121,"num_object_clones":43,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":43,"num_read":123,"num_read_kb":25979,"num_write":1341,"num_write_kb":58516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"89'2297","reported_seq":5196,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.750823+0000","last_change":"2024-03-09T09:27:34.188251+0000","last_active":"2024-03-09T09:28:56.750823+0000","last_peered":"2024-03-09T09:28:56.750823+0000","last_clean":"2024-03-09T09:28:56.750823+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T09:28:56.750823+0000","last_undegraded":"2024-03-09T09:28:56.750823+0000","last_fullsized":"2024-03-09T09:28:56.750823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2297,"log_dups_size":0,"ondisk_log_size":2297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:07:37.642402+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":3.9232747450000001,"stat_sum":{"num_bytes":5183706,"num_objects":95,"num_object_clones":27,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":111,"num_read_kb":34166,"num_write":1371,"num_write_kb":49768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"89'2247","reported_seq":5207,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.063774+0000","last_change":"2024-03-09T09:27:07.884719+0000","last_active":"2024-03-09T09:29:14.063774+0000","last_peered":"2024-03-09T09:29:14.063774+0000","last_clean":"2024-03-09T09:29:14.063774+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T09:29:14.063774+0000","last_undegraded":"2024-03-09T09:29:14.063774+0000","last_fullsized":"2024-03-09T09:29:14.063774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2247,"log_dups_size":0,"ondisk_log_size":2247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:38.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.1326348209999999,"stat_sum":{"num_bytes":1866711,"num_objects":117,"num_object_clones":40,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":40,"num_read":109,"num_read_kb":17692,"num_write":1315,"num_write_kb":41853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"89'2193","reported_seq":5129,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.657070+0000","last_change":"2024-03-09T09:27:33.862027+0000","last_active":"2024-03-09T09:28:32.657070+0000","last_peered":"2024-03-09T09:28:32.657070+0000","last_clean":"2024-03-09T09:28:32.657070+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T09:28:32.657070+0000","last_undegraded":"2024-03-09T09:28:32.657070+0000","last_fullsized":"2024-03-09T09:28:32.657070+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2193,"log_dups_size":0,"ondisk_log_size":2193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:24:42.648859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.5970959150000001,"stat_sum":{"num_bytes":1348477,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":126,"num_read_kb":8825,"num_write":1341,"num_write_kb":33612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"89'2672","reported_seq":5494,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.040015+0000","last_change":"2024-03-09T09:27:13.243644+0000","last_active":"2024-03-09T09:29:14.040015+0000","last_peered":"2024-03-09T09:29:14.040015+0000","last_clean":"2024-03-09T09:29:14.040015+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T09:29:14.040015+0000","last_undegraded":"2024-03-09T09:29:14.040015+0000","last_fullsized":"2024-03-09T09:29:14.040015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2672,"log_dups_size":0,"ondisk_log_size":2672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:48.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.49950316500000003,"stat_sum":{"num_bytes":924300,"num_objects":92,"num_object_clones":25,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":25,"num_read":107,"num_read_kb":17449,"num_write":1480,"num_write_kb":44901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"89'2175","reported_seq":5120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.873811+0000","last_change":"2024-03-09T09:27:34.679372+0000","last_active":"2024-03-09T09:29:17.873811+0000","last_peered":"2024-03-09T09:29:17.873811+0000","last_clean":"2024-03-09T09:29:17.873811+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T09:29:17.873811+0000","last_undegraded":"2024-03-09T09:29:17.873811+0000","last_fullsized":"2024-03-09T09:29:17.873811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:56.141041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4099735320000004,"stat_sum":{"num_bytes":773517,"num_objects":127,"num_object_clones":46,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":46,"num_read":120,"num_read_kb":5950,"num_write":1331,"num_write_kb":13298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"89'2162","reported_seq":5147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.536262+0000","last_change":"2024-03-09T09:27:06.385525+0000","last_active":"2024-03-09T09:28:23.536262+0000","last_peered":"2024-03-09T09:28:23.536262+0000","last_clean":"2024-03-09T09:28:23.536262+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T09:28:23.536262+0000","last_undegraded":"2024-03-09T09:28:23.536262+0000","last_fullsized":"2024-03-09T09:28:23.536262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2162,"log_dups_size":0,"ondisk_log_size":2162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:33:27.669672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.89838860799999998,"stat_sum":{"num_bytes":6604987,"num_objects":101,"num_object_clones":30,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":30,"num_read":109,"num_read_kb":14504,"num_write":1281,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"89'2152","reported_seq":5182,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.885774+0000","last_change":"2024-03-09T09:27:31.020668+0000","last_active":"2024-03-09T09:28:30.885774+0000","last_peered":"2024-03-09T09:28:30.885774+0000","last_clean":"2024-03-09T09:28:30.885774+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T09:28:30.885774+0000","last_undegraded":"2024-03-09T09:28:30.885774+0000","last_fullsized":"2024-03-09T09:28:30.885774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2152,"log_dups_size":0,"ondisk_log_size":2152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:50:23.678039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.71096335899999996,"stat_sum":{"num_bytes":637081,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":204,"num_read_kb":24036,"num_write":1319,"num_write_kb":30441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"89'2336","reported_seq":5279,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.555277+0000","last_change":"2024-03-09T09:27:06.358258+0000","last_active":"2024-03-09T09:28:23.555277+0000","last_peered":"2024-03-09T09:28:23.555277+0000","last_clean":"2024-03-09T09:28:23.555277+0000","last_became_active":"2024-03-09T08:53:00.240319+0000","last_became_peered":"2024-03-09T08:53:00.240319+0000","last_unstale":"2024-03-09T09:28:23.555277+0000","last_undegraded":"2024-03-09T09:28:23.555277+0000","last_fullsized":"2024-03-09T09:28:23.555277+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2336,"log_dups_size":0,"ondisk_log_size":2336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:31:17.488290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.86880120699999996,"stat_sum":{"num_bytes":1614659,"num_objects":116,"num_object_clones":42,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":42,"num_read":120,"num_read_kb":18915,"num_write":1382,"num_write_kb":39172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"89'2125","reported_seq":5073,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.759491+0000","last_change":"2024-03-09T09:27:32.089100+0000","last_active":"2024-03-09T09:28:56.759491+0000","last_peered":"2024-03-09T09:28:56.759491+0000","last_clean":"2024-03-09T09:28:56.759491+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T09:28:56.759491+0000","last_undegraded":"2024-03-09T09:28:56.759491+0000","last_fullsized":"2024-03-09T09:28:56.759491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2125,"log_dups_size":0,"ondisk_log_size":2125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:42.901325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.7177660380000002,"stat_sum":{"num_bytes":924110,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":129,"num_read_kb":10365,"num_write":1260,"num_write_kb":33897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"89'2227","reported_seq":5204,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.029490+0000","last_change":"2024-03-09T09:27:13.674044+0000","last_active":"2024-03-09T09:29:14.029490+0000","last_peered":"2024-03-09T09:29:14.029490+0000","last_clean":"2024-03-09T09:29:14.029490+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T09:29:14.029490+0000","last_undegraded":"2024-03-09T09:29:14.029490+0000","last_fullsized":"2024-03-09T09:29:14.029490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2227,"log_dups_size":0,"ondisk_log_size":2227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:32.494482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.2105350770000001,"stat_sum":{"num_bytes":1326786,"num_objects":103,"num_object_clones":34,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":34,"num_read":111,"num_read_kb":18735,"num_write":1322,"num_write_kb":41042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"89'2350","reported_seq":5252,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:52.661098+0000","last_change":"2024-03-09T09:27:34.276120+0000","last_active":"2024-03-09T09:28:52.661098+0000","last_peered":"2024-03-09T09:28:52.661098+0000","last_clean":"2024-03-09T09:28:52.661098+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T09:28:52.661098+0000","last_undegraded":"2024-03-09T09:28:52.661098+0000","last_fullsized":"2024-03-09T09:28:52.661098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2350,"log_dups_size":0,"ondisk_log_size":2350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:34:48.217825+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.0059579840000001,"stat_sum":{"num_bytes":936699,"num_objects":111,"num_object_clones":39,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":39,"num_read":129,"num_read_kb":34540,"num_write":1403,"num_write_kb":57337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"89'2277","reported_seq":5248,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048137+0000","last_change":"2024-03-09T09:27:09.746181+0000","last_active":"2024-03-09T09:29:14.048137+0000","last_peered":"2024-03-09T09:29:14.048137+0000","last_clean":"2024-03-09T09:29:14.048137+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T09:29:14.048137+0000","last_undegraded":"2024-03-09T09:29:14.048137+0000","last_fullsized":"2024-03-09T09:29:14.048137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2277,"log_dups_size":0,"ondisk_log_size":2277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:47:00.612196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.91582108900000003,"stat_sum":{"num_bytes":1759504,"num_objects":101,"num_object_clones":31,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":31,"num_read":108,"num_read_kb":5845,"num_write":1367,"num_write_kb":19152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"89'2349","reported_seq":5233,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.940855+0000","last_change":"2024-03-09T09:27:34.759707+0000","last_active":"2024-03-09T09:29:17.940855+0000","last_peered":"2024-03-09T09:29:17.940855+0000","last_clean":"2024-03-09T09:29:17.940855+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T09:29:17.940855+0000","last_undegraded":"2024-03-09T09:29:17.940855+0000","last_fullsized":"2024-03-09T09:29:17.940855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2349,"log_dups_size":0,"ondisk_log_size":2349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:27:24.016743+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.4857478520000003,"stat_sum":{"num_bytes":2540930,"num_objects":122,"num_object_clones":40,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":40,"num_read":127,"num_read_kb":51469,"num_write":1352,"num_write_kb":80787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"89'2218","reported_seq":5194,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.022677+0000","last_change":"2024-03-09T09:27:09.024910+0000","last_active":"2024-03-09T09:29:14.022677+0000","last_peered":"2024-03-09T09:29:14.022677+0000","last_clean":"2024-03-09T09:29:14.022677+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T09:29:14.022677+0000","last_undegraded":"2024-03-09T09:29:14.022677+0000","last_fullsized":"2024-03-09T09:29:14.022677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2218,"log_dups_size":0,"ondisk_log_size":2218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:02:10.004295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.1939797309999998,"stat_sum":{"num_bytes":1585107,"num_objects":96,"num_object_clones":31,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":31,"num_read":96,"num_read_kb":18307,"num_write":1352,"num_write_kb":50489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"89'2682","reported_seq":5434,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.949755+0000","last_change":"2024-03-09T09:27:34.659119+0000","last_active":"2024-03-09T09:29:17.949755+0000","last_peered":"2024-03-09T09:29:17.949755+0000","last_clean":"2024-03-09T09:29:17.949755+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T09:29:17.949755+0000","last_undegraded":"2024-03-09T09:29:17.949755+0000","last_fullsized":"2024-03-09T09:29:17.949755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2682,"log_dups_size":0,"ondisk_log_size":2682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:45:21.245870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4197025170000002,"stat_sum":{"num_bytes":1559225,"num_objects":122,"num_object_clones":42,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":42,"num_read":122,"num_read_kb":6495,"num_write":1460,"num_write_kb":38738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"89'2225","reported_seq":5197,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.005606+0000","last_change":"2024-03-09T09:27:12.627461+0000","last_active":"2024-03-09T09:29:14.005606+0000","last_peered":"2024-03-09T09:29:14.005606+0000","last_clean":"2024-03-09T09:29:14.005606+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T09:29:14.005606+0000","last_undegraded":"2024-03-09T09:29:14.005606+0000","last_fullsized":"2024-03-09T09:29:14.005606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2225,"log_dups_size":0,"ondisk_log_size":2225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:03:50.159643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.94133736700000004,"stat_sum":{"num_bytes":314683,"num_objects":117,"num_object_clones":39,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":39,"num_read":116,"num_read_kb":13620,"num_write":1329,"num_write_kb":42753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"89'2198","reported_seq":5208,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.950711+0000","last_change":"2024-03-09T09:27:34.517246+0000","last_active":"2024-03-09T09:29:17.950711+0000","last_peered":"2024-03-09T09:29:17.950711+0000","last_clean":"2024-03-09T09:29:17.950711+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T09:29:17.950711+0000","last_undegraded":"2024-03-09T09:29:17.950711+0000","last_fullsized":"2024-03-09T09:29:17.950711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2198,"log_dups_size":0,"ondisk_log_size":2198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:23:51.461775+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.1785381859999999,"stat_sum":{"num_bytes":1532500,"num_objects":95,"num_object_clones":30,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":30,"num_read":195,"num_read_kb":18044,"num_write":1327,"num_write_kb":39414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"89'2266","reported_seq":5235,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.043614+0000","last_change":"2024-03-09T09:27:08.772433+0000","last_active":"2024-03-09T09:29:14.043614+0000","last_peered":"2024-03-09T09:29:14.043614+0000","last_clean":"2024-03-09T09:29:14.043614+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T09:29:14.043614+0000","last_undegraded":"2024-03-09T09:29:14.043614+0000","last_fullsized":"2024-03-09T09:29:14.043614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2266,"log_dups_size":0,"ondisk_log_size":2266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:09:40.668035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.298540461,"stat_sum":{"num_bytes":2039183,"num_objects":130,"num_object_clones":43,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":43,"num_read":126,"num_read_kb":25377,"num_write":1340,"num_write_kb":50110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"89'2204","reported_seq":5123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:47.307121+0000","last_change":"2024-03-09T09:27:30.886579+0000","last_active":"2024-03-09T09:28:47.307121+0000","last_peered":"2024-03-09T09:28:47.307121+0000","last_clean":"2024-03-09T09:28:47.307121+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T09:28:47.307121+0000","last_undegraded":"2024-03-09T09:28:47.307121+0000","last_fullsized":"2024-03-09T09:28:47.307121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2204,"log_dups_size":0,"ondisk_log_size":2204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:31:23.131967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.62967200199999995,"stat_sum":{"num_bytes":347019,"num_objects":120,"num_object_clones":41,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":41,"num_read":119,"num_read_kb":17844,"num_write":1302,"num_write_kb":51829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"89'2658","reported_seq":5496,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.057096+0000","last_change":"2024-03-09T09:27:07.715360+0000","last_active":"2024-03-09T09:29:14.057096+0000","last_peered":"2024-03-09T09:29:14.057096+0000","last_clean":"2024-03-09T09:29:14.057096+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T09:29:14.057096+0000","last_undegraded":"2024-03-09T09:29:14.057096+0000","last_fullsized":"2024-03-09T09:29:14.057096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2658,"log_dups_size":0,"ondisk_log_size":2658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:37:57.318273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.0660642299999998,"stat_sum":{"num_bytes":1365000,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":127,"num_read_kb":33469,"num_write":1475,"num_write_kb":57156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"89'2231","reported_seq":5157,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.942922+0000","last_change":"2024-03-09T09:27:34.649642+0000","last_active":"2024-03-09T09:29:17.942922+0000","last_peered":"2024-03-09T09:29:17.942922+0000","last_clean":"2024-03-09T09:29:17.942922+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T09:29:17.942922+0000","last_undegraded":"2024-03-09T09:29:17.942922+0000","last_fullsized":"2024-03-09T09:29:17.942922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2231,"log_dups_size":0,"ondisk_log_size":2231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:53.123132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4162208879999998,"stat_sum":{"num_bytes":1481141,"num_objects":109,"num_object_clones":37,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":37,"num_read":120,"num_read_kb":25693,"num_write":1333,"num_write_kb":50297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'2192","reported_seq":5171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.049795+0000","last_change":"2024-03-09T09:27:08.029887+0000","last_active":"2024-03-09T09:29:14.049795+0000","last_peered":"2024-03-09T09:29:14.049795+0000","last_clean":"2024-03-09T09:29:14.049795+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T09:29:14.049795+0000","last_undegraded":"2024-03-09T09:29:14.049795+0000","last_fullsized":"2024-03-09T09:29:14.049795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2192,"log_dups_size":0,"ondisk_log_size":2192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:57:17.824590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.379810049,"stat_sum":{"num_bytes":472147,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":115,"num_read_kb":25809,"num_write":1277,"num_write_kb":34061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"89'2310","reported_seq":5232,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924102+0000","last_change":"2024-03-09T09:27:34.694598+0000","last_active":"2024-03-09T09:29:17.924102+0000","last_peered":"2024-03-09T09:29:17.924102+0000","last_clean":"2024-03-09T09:29:17.924102+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T09:29:17.924102+0000","last_undegraded":"2024-03-09T09:29:17.924102+0000","last_fullsized":"2024-03-09T09:29:17.924102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2310,"log_dups_size":0,"ondisk_log_size":2310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:24:52.867402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4238349550000002,"stat_sum":{"num_bytes":1560364,"num_objects":99,"num_object_clones":31,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":31,"num_read":120,"num_read_kb":23106,"num_write":1406,"num_write_kb":36148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"89'2355","reported_seq":5316,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.061430+0000","last_change":"2024-03-09T09:27:13.474431+0000","last_active":"2024-03-09T09:29:14.061430+0000","last_peered":"2024-03-09T09:29:14.061430+0000","last_clean":"2024-03-09T09:29:14.061430+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T09:29:14.061430+0000","last_undegraded":"2024-03-09T09:29:14.061430+0000","last_fullsized":"2024-03-09T09:29:14.061430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2355,"log_dups_size":0,"ondisk_log_size":2355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:53.961909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.73072901000000001,"stat_sum":{"num_bytes":1439516,"num_objects":106,"num_object_clones":37,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":37,"num_read":117,"num_read_kb":43194,"num_write":1390,"num_write_kb":63251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"89'2112","reported_seq":5035,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.954079+0000","last_change":"2024-03-09T09:27:34.559512+0000","last_active":"2024-03-09T09:29:17.954079+0000","last_peered":"2024-03-09T09:29:17.954079+0000","last_clean":"2024-03-09T09:29:17.954079+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T09:29:17.954079+0000","last_undegraded":"2024-03-09T09:29:17.954079+0000","last_fullsized":"2024-03-09T09:29:17.954079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2112,"log_dups_size":0,"ondisk_log_size":2112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:12:19.227405+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2163343510000004,"stat_sum":{"num_bytes":526581,"num_objects":91,"num_object_clones":29,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":29,"num_read":100,"num_read_kb":25429,"num_write":1233,"num_write_kb":64615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"89'2159","reported_seq":5133,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.052874+0000","last_change":"2024-03-09T09:27:12.157156+0000","last_active":"2024-03-09T09:29:14.052874+0000","last_peered":"2024-03-09T09:29:14.052874+0000","last_clean":"2024-03-09T09:29:14.052874+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T09:29:14.052874+0000","last_undegraded":"2024-03-09T09:29:14.052874+0000","last_fullsized":"2024-03-09T09:29:14.052874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2159,"log_dups_size":0,"ondisk_log_size":2159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:45:37.914970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.46979842799999999,"stat_sum":{"num_bytes":631595,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":98,"num_read_kb":25968,"num_write":1286,"num_write_kb":41427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"89'2132","reported_seq":5071,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.889842+0000","last_change":"2024-03-09T09:27:30.899203+0000","last_active":"2024-03-09T09:28:30.889842+0000","last_peered":"2024-03-09T09:28:30.889842+0000","last_clean":"2024-03-09T09:28:30.889842+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T09:28:30.889842+0000","last_undegraded":"2024-03-09T09:28:30.889842+0000","last_fullsized":"2024-03-09T09:28:30.889842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2132,"log_dups_size":0,"ondisk_log_size":2132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:29:36.464781+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.66506321800000001,"stat_sum":{"num_bytes":592790,"num_objects":103,"num_object_clones":35,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":35,"num_read":105,"num_read_kb":21771,"num_write":1301,"num_write_kb":41126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"89'2204","reported_seq":5160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048992+0000","last_change":"2024-03-09T09:27:13.380861+0000","last_active":"2024-03-09T09:29:14.048992+0000","last_peered":"2024-03-09T09:29:14.048992+0000","last_clean":"2024-03-09T09:29:14.048992+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T09:29:14.048992+0000","last_undegraded":"2024-03-09T09:29:14.048992+0000","last_fullsized":"2024-03-09T09:29:14.048992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2204,"log_dups_size":0,"ondisk_log_size":2204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:02:11.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.638707739,"stat_sum":{"num_bytes":1008318,"num_objects":81,"num_object_clones":24,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":24,"num_read":86,"num_read_kb":17604,"num_write":1317,"num_write_kb":38401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"89'2293","reported_seq":5184,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.754410+0000","last_change":"2024-03-09T09:27:32.075213+0000","last_active":"2024-03-09T09:28:56.754410+0000","last_peered":"2024-03-09T09:28:56.754410+0000","last_clean":"2024-03-09T09:28:56.754410+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T09:28:56.754410+0000","last_undegraded":"2024-03-09T09:28:56.754410+0000","last_fullsized":"2024-03-09T09:28:56.754410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2293,"log_dups_size":0,"ondisk_log_size":2293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:17:17.703718+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.705573912,"stat_sum":{"num_bytes":534756,"num_objects":106,"num_object_clones":34,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":34,"num_read":108,"num_read_kb":25061,"num_write":1338,"num_write_kb":44408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"89'2111","reported_seq":5114,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.032543+0000","last_change":"2024-03-09T09:27:13.610566+0000","last_active":"2024-03-09T09:29:14.032543+0000","last_peered":"2024-03-09T09:29:14.032543+0000","last_clean":"2024-03-09T09:29:14.032543+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T09:29:14.032543+0000","last_undegraded":"2024-03-09T09:29:14.032543+0000","last_fullsized":"2024-03-09T09:29:14.032543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2111,"log_dups_size":0,"ondisk_log_size":2111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:17:33.229502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.1420017650000001,"stat_sum":{"num_bytes":1651219,"num_objects":118,"num_object_clones":44,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":44,"num_read":114,"num_read_kb":27021,"num_write":1266,"num_write_kb":36457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"89'2279","reported_seq":5203,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924096+0000","last_change":"2024-03-09T09:27:34.669037+0000","last_active":"2024-03-09T09:29:17.924096+0000","last_peered":"2024-03-09T09:29:17.924096+0000","last_clean":"2024-03-09T09:29:17.924096+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T09:29:17.924096+0000","last_undegraded":"2024-03-09T09:29:17.924096+0000","last_fullsized":"2024-03-09T09:29:17.924096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2279,"log_dups_size":0,"ondisk_log_size":2279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:54:35.897455+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.3954307559999997,"stat_sum":{"num_bytes":1015705,"num_objects":121,"num_object_clones":38,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":38,"num_read":132,"num_read_kb":25133,"num_write":1351,"num_write_kb":59371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"89'2361","reported_seq":5314,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.067546+0000","last_change":"2024-03-09T09:27:13.261980+0000","last_active":"2024-03-09T09:29:14.067546+0000","last_peered":"2024-03-09T09:29:14.067546+0000","last_clean":"2024-03-09T09:29:14.067546+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T09:29:14.067546+0000","last_undegraded":"2024-03-09T09:29:14.067546+0000","last_fullsized":"2024-03-09T09:29:14.067546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2361,"log_dups_size":0,"ondisk_log_size":2361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:13:56.252007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.51876684699999998,"stat_sum":{"num_bytes":1407157,"num_objects":117,"num_object_clones":38,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":38,"num_read":113,"num_read_kb":14121,"num_write":1423,"num_write_kb":30929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"89'2145","reported_seq":5075,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.939348+0000","last_change":"2024-03-09T09:27:34.506277+0000","last_active":"2024-03-09T09:29:17.939348+0000","last_peered":"2024-03-09T09:29:17.939348+0000","last_clean":"2024-03-09T09:29:17.939348+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T09:29:17.939348+0000","last_undegraded":"2024-03-09T09:29:17.939348+0000","last_fullsized":"2024-03-09T09:29:17.939348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2145,"log_dups_size":0,"ondisk_log_size":2145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:27.408652+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.1683436250000003,"stat_sum":{"num_bytes":2080769,"num_objects":116,"num_object_clones":43,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":43,"num_read":114,"num_read_kb":27228,"num_write":1275,"num_write_kb":54719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"89'2227","reported_seq":5198,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.040802+0000","last_change":"2024-03-09T09:27:13.892143+0000","last_active":"2024-03-09T09:29:14.040802+0000","last_peered":"2024-03-09T09:29:14.040802+0000","last_clean":"2024-03-09T09:29:14.040802+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T09:29:14.040802+0000","last_undegraded":"2024-03-09T09:29:14.040802+0000","last_fullsized":"2024-03-09T09:29:14.040802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2227,"log_dups_size":0,"ondisk_log_size":2227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:36:18.464529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.4282657350000001,"stat_sum":{"num_bytes":1076453,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":106,"num_read_kb":19463,"num_write":1318,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"89'2408","reported_seq":5285,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.733743+0000","last_change":"2024-03-09T09:27:30.899184+0000","last_active":"2024-03-09T09:28:56.733743+0000","last_peered":"2024-03-09T09:28:56.733743+0000","last_clean":"2024-03-09T09:28:56.733743+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T09:28:56.733743+0000","last_undegraded":"2024-03-09T09:28:56.733743+0000","last_fullsized":"2024-03-09T09:28:56.733743+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2408,"log_dups_size":0,"ondisk_log_size":2408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:47.160761+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.63967549000000001,"stat_sum":{"num_bytes":3580963,"num_objects":116,"num_object_clones":37,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":37,"num_read":116,"num_read_kb":32310,"num_write":1440,"num_write_kb":66236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"89'2251","reported_seq":5231,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.030277+0000","last_change":"2024-03-09T09:27:09.597741+0000","last_active":"2024-03-09T09:29:14.030277+0000","last_peered":"2024-03-09T09:29:14.030277+0000","last_clean":"2024-03-09T09:29:14.030277+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T09:29:14.030277+0000","last_undegraded":"2024-03-09T09:29:14.030277+0000","last_fullsized":"2024-03-09T09:29:14.030277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2251,"log_dups_size":0,"ondisk_log_size":2251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:29:26.509228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.47268980999999999,"stat_sum":{"num_bytes":1488567,"num_objects":112,"num_object_clones":34,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":34,"num_read":108,"num_read_kb":30029,"num_write":1350,"num_write_kb":42424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"89'2142","reported_seq":5072,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.933559+0000","last_change":"2024-03-09T09:27:34.605106+0000","last_active":"2024-03-09T09:29:17.933559+0000","last_peered":"2024-03-09T09:29:17.933559+0000","last_clean":"2024-03-09T09:29:17.933559+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T09:29:17.933559+0000","last_undegraded":"2024-03-09T09:29:17.933559+0000","last_fullsized":"2024-03-09T09:29:17.933559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2142,"log_dups_size":0,"ondisk_log_size":2142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:04:07.487361+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.3763956530000003,"stat_sum":{"num_bytes":1440979,"num_objects":98,"num_object_clones":30,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":30,"num_read":104,"num_read_kb":22606,"num_write":1302,"num_write_kb":46651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"89'2066","reported_seq":5058,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.003676+0000","last_change":"2024-03-09T09:27:10.176463+0000","last_active":"2024-03-09T09:29:14.003676+0000","last_peered":"2024-03-09T09:29:14.003676+0000","last_clean":"2024-03-09T09:29:14.003676+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T09:29:14.003676+0000","last_undegraded":"2024-03-09T09:29:14.003676+0000","last_fullsized":"2024-03-09T09:29:14.003676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2066,"log_dups_size":0,"ondisk_log_size":2066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:32:22.191345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.344208157,"stat_sum":{"num_bytes":485403,"num_objects":85,"num_object_clones":27,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":27,"num_read":96,"num_read_kb":5635,"num_write":1204,"num_write_kb":28568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"89'2216","reported_seq":5128,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.931323+0000","last_change":"2024-03-09T09:27:34.669203+0000","last_active":"2024-03-09T09:29:17.931323+0000","last_peered":"2024-03-09T09:29:17.931323+0000","last_clean":"2024-03-09T09:29:17.931323+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T09:29:17.931323+0000","last_undegraded":"2024-03-09T09:29:17.931323+0000","last_fullsized":"2024-03-09T09:29:17.931323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2216,"log_dups_size":0,"ondisk_log_size":2216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:04:18.586097+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.4378791440000001,"stat_sum":{"num_bytes":906976,"num_objects":87,"num_object_clones":27,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":27,"num_read":110,"num_read_kb":26703,"num_write":1307,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"89'2145","reported_seq":5141,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.051697+0000","last_change":"2024-03-09T09:27:13.826300+0000","last_active":"2024-03-09T09:29:14.051697+0000","last_peered":"2024-03-09T09:29:14.051697+0000","last_clean":"2024-03-09T09:29:14.051697+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T09:29:14.051697+0000","last_undegraded":"2024-03-09T09:29:14.051697+0000","last_fullsized":"2024-03-09T09:29:14.051697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2145,"log_dups_size":0,"ondisk_log_size":2145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:23:44.879214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.0826407490000001,"stat_sum":{"num_bytes":992571,"num_objects":97,"num_object_clones":32,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":32,"num_read":100,"num_read_kb":32945,"num_write":1310,"num_write_kb":54952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7626,"num_read_kb":1526336,"num_write":85965,"num_write_kb":2933421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144274,"ondisk_log_size":144274,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":875580134,"num_objects":1506,"num_object_clones":0,"num_object_copies":3012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1506,"num_whiteouts":0,"num_read":2924,"num_read_kb":5807,"num_write":25873,"num_write_kb":3143283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1225,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1752338432,"data_stored":1752278262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22159,"ondisk_log_size":22159,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7230,"num_read_kb":1519173,"num_write":85971,"num_write_kb":2933405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144300,"ondisk_log_size":144300,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798313328,"num_objects":1475,"num_object_clones":0,"num_object_copies":2950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1475,"num_whiteouts":0,"num_read":2854,"num_read_kb":5749,"num_write":25870,"num_write_kb":3104826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1213,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1596686336,"data_stored":1596626656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22164,"ondisk_log_size":22164,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575298,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":851964,"kb_used_data":501556,"kb_used_omap":1,"kb_used_meta":350398,"kb_avail":93519876,"statfs":{"total":96636764160,"available":95764353024,"internally_reserved":0,"allocated":513593344,"data_stored":506816772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":358808010},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575299,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699740,"kb_used_data":456276,"kb_used_omap":1,"kb_used_meta":243454,"kb_avail":93672100,"statfs":{"total":96636764160,"available":95920230400,"internally_reserved":0,"allocated":467226624,"data_stored":462408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":249297350},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575297,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794804,"kb_used_data":452716,"kb_used_omap":1,"kb_used_meta":342078,"kb_avail":93577036,"statfs":{"total":96636764160,"available":95822884864,"internally_reserved":0,"allocated":463581184,"data_stored":456386951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":350288325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575298,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":853316,"kb_used_data":551484,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":93518524,"statfs":{"total":96636764160,"available":95762968576,"internally_reserved":0,"allocated":564719616,"data_stored":557367936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309066181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575299,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":918132,"kb_used_data":544172,"kb_used_omap":1,"kb_used_meta":373950,"kb_avail":93453708,"statfs":{"total":96636764160,"available":95696596992,"internally_reserved":0,"allocated":557232128,"data_stored":549727853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":382925255},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575298,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450472,"kb_used_data":225056,"kb_used_omap":1,"kb_used_meta":225406,"kb_avail":93921368,"statfs":{"total":96636764160,"available":96175480832,"internally_reserved":0,"allocated":230457344,"data_stored":225591238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":230816197},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575298,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":968836,"kb_used_data":612924,"kb_used_omap":1,"kb_used_meta":355902,"kb_avail":93403004,"statfs":{"total":96636764160,"available":95644676096,"internally_reserved":0,"allocated":627634176,"data_stored":621572263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":364444101},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575300,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670332,"kb_used_data":378292,"kb_used_omap":1,"kb_used_meta":292030,"kb_avail":93701508,"statfs":{"total":96636764160,"available":95950344192,"internally_reserved":0,"allocated":387371008,"data_stored":381048744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":299039175},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":226504704,"data_stored":226493054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":218112000,"data_stored":218104019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92274688,"data_stored":92274688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":172527616,"data_stored":172519074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":201338880,"data_stored":201326769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":197980160,"data_stored":197974487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":223150080,"data_stored":223140893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":264798208,"data_stored":264793672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37781504,"data_stored":34251073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20246528,"data_stored":18282047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22822912,"data_stored":20809621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27308032,"data_stored":23727598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37437440,"data_stored":33951899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":27983872,"data_stored":24565897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23818240,"data_stored":21628413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":29773824,"data_stored":26493862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98791424,"data_stored":98790378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":360722432,"data_stored":360710838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96473088,"data_stored":96469310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":315428864,"data_stored":315415292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":289415168,"data_stored":289407328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":200581120,"data_stored":200572340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":201334784,"data_stored":201327208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":189591552,"data_stored":189585568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22835200,"data_stored":20540995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27480064,"data_stored":23882611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17813504,"data_stored":15444871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40894464,"data_stored":37473141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35454976,"data_stored":32089192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":37081088,"data_stored":33799473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17850368,"data_stored":15719573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":27762688,"data_stored":24760554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T09:29:19.839 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-09T09:29:19.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T09:29:20.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.096+0000 7f1ea0400640 1 Processor -- start 2024-03-09T09:29:20.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.097+0000 7f1ea0400640 1 -- start start 2024-03-09T09:29:20.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.099+0000 7f1ea0400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1e98153750 0x7f1e980c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.099+0000 7f1ea0400640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f1e9806dc00 0x7f1e980729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.099+0000 7f1ea0400640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e980c4010 con 0x7f1e98153370 2024-03-09T09:29:20.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.099+0000 7f1ea0400640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e98156dc0 con 0x7f1e98153750 2024-03-09T09:29:20.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.099+0000 7f1ea0400640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e980c42f0 con 0x7f1e9806dc00 2024-03-09T09:29:20.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.100+0000 7f1e9d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1e98153750 0x7f1e980c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:20.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.100+0000 7f1e9d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1e98153750 0x7f1e980c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45058/0 (socket says 172.21.15.67:45058) 2024-03-09T09:29:20.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.100+0000 7f1e9d000640 1 -- 172.21.15.67:0/809364593 learned_addr learned my addr 172.21.15.67:0/809364593 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:20.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.100+0000 7f1e9e400640 1 --2- 172.21.15.67:0/809364593 >> v2:172.21.15.134:3300/0 conn(0x7f1e9806dc00 0x7f1e980729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:20.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.100+0000 7f1e9d000640 1 -- 172.21.15.67:0/809364593 >> v2:172.21.15.134:3300/0 conn(0x7f1e9806dc00 msgr2=0x7f1e980729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:20.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.100+0000 7f1e9d000640 1 --2- 172.21.15.67:0/809364593 >> v2:172.21.15.134:3300/0 conn(0x7f1e9806dc00 0x7f1e980729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.100+0000 7f1e9d000640 1 -- 172.21.15.67:0/809364593 >> v1:172.21.15.67:6789/0 conn(0x7f1e98153370 legacy=0x7f1e980c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:20.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.100+0000 7f1e9d000640 1 -- 172.21.15.67:0/809364593 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e980c5a50 con 0x7f1e98153750 2024-03-09T09:29:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.100+0000 7f1e97e00640 1 -- 172.21.15.67:0/809364593 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716387452 0 0) 0x7f1e980c4010 con 0x7f1e98153370 2024-03-09T09:29:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.101+0000 7f1e9d000640 1 --2- 172.21.15.67:0/809364593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e98153750 0x7f1e980c30b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1e9806e620 tx=0x7f1e8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67bd4e21dbdf7b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.102+0000 7f1e97e00640 1 -- 172.21.15.67:0/809364593 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84035020 con 0x7f1e98153750 2024-03-09T09:29:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.102+0000 7f1e97e00640 1 -- 172.21.15.67:0/809364593 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e84031870 con 0x7f1e98153750 2024-03-09T09:29:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.102+0000 7f1e97e00640 1 -- 172.21.15.67:0/809364593 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84031c40 con 0x7f1e98153750 2024-03-09T09:29:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.102+0000 7f1ea0400640 1 -- 172.21.15.67:0/809364593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e98153750 msgr2=0x7f1e980c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.102+0000 7f1ea0400640 1 --2- 172.21.15.67:0/809364593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e98153750 0x7f1e980c30b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1e9806e620 tx=0x7f1e8402b910 comp rx=0 tx=0).stop 2024-03-09T09:29:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.102+0000 7f1ea0400640 1 -- 172.21.15.67:0/809364593 shutdown_connections 2024-03-09T09:29:20.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.102+0000 7f1ea0400640 1 --2- 172.21.15.67:0/809364593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e98153750 0x7f1e980c30b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.102+0000 7f1ea0400640 1 --2- 172.21.15.67:0/809364593 >> v2:172.21.15.134:3300/0 conn(0x7f1e9806dc00 0x7f1e980729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.102+0000 7f1ea0400640 1 -- 172.21.15.67:0/809364593 >> 172.21.15.67:0/809364593 conn(0x7f1e980ab320 msgr2=0x7f1e980ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:20.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.103+0000 7f1ea0400640 1 -- 172.21.15.67:0/809364593 shutdown_connections 2024-03-09T09:29:20.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.103+0000 7f1ea0400640 1 -- 172.21.15.67:0/809364593 wait complete. 2024-03-09T09:29:20.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.106+0000 7f1ea0400640 1 Processor -- start 2024-03-09T09:29:20.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.106+0000 7f1ea0400640 1 -- start start 2024-03-09T09:29:20.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.107+0000 7f1ea0400640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9806dc00 0x7f1e980aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1ea0400640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e98153370 0x7f1e98153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1ea0400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e98153750 0x7f1e981eeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1ea0400640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980c5d30 con 0x7f1e98153750 2024-03-09T09:29:20.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1ea0400640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e98073dc0 con 0x7f1e9806dc00 2024-03-09T09:29:20.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1ea0400640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e98072f00 con 0x7f1e98153370 2024-03-09T09:29:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1e9e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e98153750 0x7f1e981eeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1e9e400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e98153750 0x7f1e981eeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45060/0 (socket says 172.21.15.67:45060) 2024-03-09T09:29:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1e9e400640 1 -- 172.21.15.67:0/3196620516 learned_addr learned my addr 172.21.15.67:0/3196620516 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1e9da00640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9806dc00 0x7f1e980aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1e9d000640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e98153370 0x7f1e98153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1e9e400640 1 -- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e98153370 msgr2=0x7f1e98153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1e9e400640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e98153370 0x7f1e98153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1e9e400640 1 -- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9806dc00 msgr2=0x7f1e980aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:20.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1e9e400640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9806dc00 0x7f1e980aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.108+0000 7f1e9e400640 1 -- 172.21.15.67:0/3196620516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e98157eb0 con 0x7f1e98153750 2024-03-09T09:29:20.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.109+0000 7f1e9e400640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e98153750 0x7f1e981eeb60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1e90007e60 tx=0x7f1e9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:20.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.109+0000 7f1e95600640 1 -- 172.21.15.67:0/3196620516 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90014420 con 0x7f1e98153750 2024-03-09T09:29:20.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.109+0000 7f1e95600640 1 -- 172.21.15.67:0/3196620516 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e90019070 con 0x7f1e98153750 2024-03-09T09:29:20.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.109+0000 7f1e95600640 1 -- 172.21.15.67:0/3196620516 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90014820 con 0x7f1e98153750 2024-03-09T09:29:20.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.109+0000 7f1ea0400640 1 -- 172.21.15.67:0/3196620516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e980ac750 con 0x7f1e98153750 2024-03-09T09:29:20.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.109+0000 7f1ea0400640 1 -- 172.21.15.67:0/3196620516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e980aded0 con 0x7f1e98153750 2024-03-09T09:29:20.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.110+0000 7f1ea0400640 1 -- 172.21.15.67:0/3196620516 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e98075a00 con 0x7f1e98153750 2024-03-09T09:29:20.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.111+0000 7f1e95600640 1 -- 172.21.15.67:0/3196620516 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1e900149c0 con 0x7f1e98153750 2024-03-09T09:29:20.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.112+0000 7f1e95600640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e7806b4d0 0x7f1e7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.112+0000 7f1e95600640 1 -- 172.21.15.67:0/3196620516 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f1e9008ae30 con 0x7f1e98153750 2024-03-09T09:29:20.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.113+0000 7f1e9da00640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e7806b4d0 0x7f1e7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:20.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.114+0000 7f1e9da00640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e7806b4d0 0x7f1e7806d990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c000f60 tx=0x7f1e8c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:20.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.117+0000 7f1e95600640 1 -- 172.21.15.67:0/3196620516 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1e900598e0 con 0x7f1e98153750 2024-03-09T09:29:20.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.312+0000 7f1ea0400640 1 -- 172.21.15.67:0/3196620516 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e98152e80 con 0x7f1e7806b4d0 2024-03-09T09:29:20.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.348+0000 7f1e95600640 1 -- 172.21.15.67:0/3196620516 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+499794 (secure 0 0 0) 0x7f1e98152e80 con 0x7f1e7806b4d0 2024-03-09T09:29:20.359 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:29:20.368 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T09:29:20.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.358+0000 7f1ea0400640 1 -- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e7806b4d0 msgr2=0x7f1e7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:20.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.358+0000 7f1ea0400640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e7806b4d0 0x7f1e7806d990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c000f60 tx=0x7f1e8c009040 comp rx=0 tx=0).stop 2024-03-09T09:29:20.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.359+0000 7f1ea0400640 1 -- 172.21.15.67:0/3196620516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e98153750 msgr2=0x7f1e981eeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:20.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.359+0000 7f1ea0400640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e98153750 0x7f1e981eeb60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1e90007e60 tx=0x7f1e9000e4b0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.359+0000 7f1ea0400640 1 -- 172.21.15.67:0/3196620516 shutdown_connections 2024-03-09T09:29:20.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.359+0000 7f1ea0400640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1e7806b4d0 0x7f1e7806d990 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.359+0000 7f1ea0400640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e98153750 0x7f1e981eeb60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.359+0000 7f1ea0400640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1e98153370 0x7f1e98153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.359+0000 7f1ea0400640 1 --2- 172.21.15.67:0/3196620516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9806dc00 0x7f1e980aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.359+0000 7f1ea0400640 1 -- 172.21.15.67:0/3196620516 >> 172.21.15.67:0/3196620516 conn(0x7f1e980ab320 msgr2=0x7f1e98151ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:20.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.359+0000 7f1ea0400640 1 -- 172.21.15.67:0/3196620516 shutdown_connections 2024-03-09T09:29:20.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.359+0000 7f1ea0400640 1 -- 172.21.15.67:0/3196620516 wait complete. 2024-03-09T09:29:20.387 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":1198,"stamp":"2024-03-09T09:29:19.711022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1878194240,"num_objects":17181,"num_object_clones":4728,"num_object_copies":34362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17181,"num_whiteouts":4728,"num_read":20750,"num_read_kb":3057165,"num_write":223838,"num_write_kb":12117351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2438,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":333014,"ondisk_log_size":333014,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6207596,"kb_used_data":3722476,"kb_used_omap":12,"kb_used_meta":2485043,"kb_avail":748767124,"statfs":{"total":773094113280,"available":766737534976,"internally_reserved":0,"allocated":3811815424,"data_stored":3760920592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2544684594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1934299,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":31,"num_write_kb":1904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005497"},"pg_stats":[{"pgid":"5.18","version":"89'2465","reported_seq":5338,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:47.659926+0000","last_change":"2024-03-09T09:27:34.120297+0000","last_active":"2024-03-09T09:28:47.659926+0000","last_peered":"2024-03-09T09:28:47.659926+0000","last_clean":"2024-03-09T09:28:47.659926+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T09:28:47.659926+0000","last_undegraded":"2024-03-09T09:28:47.659926+0000","last_fullsized":"2024-03-09T09:28:47.659926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2465,"log_dups_size":0,"ondisk_log_size":2465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:09:33.307167+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":3.854819714,"stat_sum":{"num_bytes":1582215,"num_objects":90,"num_object_clones":27,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":27,"num_read":122,"num_read_kb":30313,"num_write":1455,"num_write_kb":61176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"89'449","reported_seq":618,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:51.137870+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T09:27:51.137870+0000","last_peered":"2024-03-09T09:27:51.137870+0000","last_clean":"2024-03-09T09:27:51.137870+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T09:27:51.137870+0000","last_undegraded":"2024-03-09T09:27:51.137870+0000","last_fullsized":"2024-03-09T09:27:51.137870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:41:40.332687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":462,"num_write_kb":82632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"89'2179","reported_seq":5159,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:28.596602+0000","last_change":"2024-03-09T09:27:12.157369+0000","last_active":"2024-03-09T09:28:28.596602+0000","last_peered":"2024-03-09T09:28:28.596602+0000","last_clean":"2024-03-09T09:28:28.596602+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T09:28:28.596602+0000","last_undegraded":"2024-03-09T09:28:28.596602+0000","last_fullsized":"2024-03-09T09:28:28.596602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2179,"log_dups_size":0,"ondisk_log_size":2179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:03:38.261625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.47085273999999999,"stat_sum":{"num_bytes":1473782,"num_objects":111,"num_object_clones":38,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":38,"num_read":99,"num_read_kb":25344,"num_write":1311,"num_write_kb":54508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"89'405","reported_seq":563,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:59.412978+0000","last_change":"2024-03-09T08:52:28.571905+0000","last_active":"2024-03-09T09:27:59.412978+0000","last_peered":"2024-03-09T09:27:59.412978+0000","last_clean":"2024-03-09T09:27:59.412978+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T09:27:59.412978+0000","last_undegraded":"2024-03-09T09:27:59.412978+0000","last_fullsized":"2024-03-09T09:27:59.412978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:01.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":445,"num_write_kb":78871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"89'2252","reported_seq":5164,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:52.564577+0000","last_change":"2024-03-09T09:27:32.072161+0000","last_active":"2024-03-09T09:28:52.564577+0000","last_peered":"2024-03-09T09:28:52.564577+0000","last_clean":"2024-03-09T09:28:52.564577+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T09:28:52.564577+0000","last_undegraded":"2024-03-09T09:28:52.564577+0000","last_fullsized":"2024-03-09T09:28:52.564577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2252,"log_dups_size":0,"ondisk_log_size":2252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:24.242000+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.703796324,"stat_sum":{"num_bytes":1252433,"num_objects":118,"num_object_clones":38,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":38,"num_read":118,"num_read_kb":38794,"num_write":1352,"num_write_kb":71340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"89'745","reported_seq":914,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.595752+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T09:28:23.595752+0000","last_peered":"2024-03-09T09:28:23.595752+0000","last_clean":"2024-03-09T09:28:23.595752+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T09:28:23.595752+0000","last_undegraded":"2024-03-09T09:28:23.595752+0000","last_fullsized":"2024-03-09T09:28:23.595752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":51,"num_read_kb":0,"num_write":808,"num_write_kb":91456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'2160","reported_seq":5146,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.038761+0000","last_change":"2024-03-09T09:27:09.692723+0000","last_active":"2024-03-09T09:29:14.038761+0000","last_peered":"2024-03-09T09:29:14.038761+0000","last_clean":"2024-03-09T09:29:14.038761+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T09:29:14.038761+0000","last_undegraded":"2024-03-09T09:29:14.038761+0000","last_fullsized":"2024-03-09T09:29:14.038761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2160,"log_dups_size":0,"ondisk_log_size":2160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:29:50.385379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.48044756999999999,"stat_sum":{"num_bytes":498978,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":114,"num_read_kb":18251,"num_write":1268,"num_write_kb":37672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"89'892","reported_seq":1051,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.657948+0000","last_change":"2024-03-09T08:52:28.573074+0000","last_active":"2024-03-09T09:28:26.657948+0000","last_peered":"2024-03-09T09:28:26.657948+0000","last_clean":"2024-03-09T09:28:26.657948+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T09:28:26.657948+0000","last_undegraded":"2024-03-09T09:28:26.657948+0000","last_fullsized":"2024-03-09T09:28:26.657948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:39:50.600408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":939,"num_write_kb":120134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"89'2266","reported_seq":5169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924644+0000","last_change":"2024-03-09T09:27:34.614436+0000","last_active":"2024-03-09T09:29:17.924644+0000","last_peered":"2024-03-09T09:29:17.924644+0000","last_clean":"2024-03-09T09:29:17.924644+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T09:29:17.924644+0000","last_undegraded":"2024-03-09T09:29:17.924644+0000","last_fullsized":"2024-03-09T09:29:17.924644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2266,"log_dups_size":0,"ondisk_log_size":2266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:18:23.796765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.3826170339999999,"stat_sum":{"num_bytes":1068833,"num_objects":92,"num_object_clones":30,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":30,"num_read":126,"num_read_kb":14571,"num_write":1308,"num_write_kb":45011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"89'674","reported_seq":840,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:58.111396+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T09:27:58.111396+0000","last_peered":"2024-03-09T09:27:58.111396+0000","last_clean":"2024-03-09T09:27:58.111396+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T09:27:58.111396+0000","last_undegraded":"2024-03-09T09:27:58.111396+0000","last_fullsized":"2024-03-09T09:27:58.111396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":67,"num_read_kb":0,"num_write":775,"num_write_kb":140765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"89'2379","reported_seq":5342,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.635615+0000","last_change":"2024-03-09T09:27:14.117823+0000","last_active":"2024-03-09T09:28:23.635615+0000","last_peered":"2024-03-09T09:28:23.635615+0000","last_clean":"2024-03-09T09:28:23.635615+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T09:28:23.635615+0000","last_undegraded":"2024-03-09T09:28:23.635615+0000","last_fullsized":"2024-03-09T09:28:23.635615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2379,"log_dups_size":0,"ondisk_log_size":2379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:49:44.890858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.653915281,"stat_sum":{"num_bytes":2793747,"num_objects":148,"num_object_clones":53,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":53,"num_read":138,"num_read_kb":27086,"num_write":1428,"num_write_kb":43660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"89'531","reported_seq":689,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.454336+0000","last_change":"2024-03-09T08:52:28.577128+0000","last_active":"2024-03-09T09:28:14.454336+0000","last_peered":"2024-03-09T09:28:14.454336+0000","last_clean":"2024-03-09T09:28:14.454336+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T09:28:14.454336+0000","last_undegraded":"2024-03-09T09:28:14.454336+0000","last_fullsized":"2024-03-09T09:28:14.454336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:41.801752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":543,"num_write_kb":111605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"89'2275","reported_seq":5173,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.920377+0000","last_change":"2024-03-09T09:27:34.731982+0000","last_active":"2024-03-09T09:29:17.920377+0000","last_peered":"2024-03-09T09:29:17.920377+0000","last_clean":"2024-03-09T09:29:17.920377+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T09:29:17.920377+0000","last_undegraded":"2024-03-09T09:29:17.920377+0000","last_fullsized":"2024-03-09T09:29:17.920377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2275,"log_dups_size":0,"ondisk_log_size":2275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:34:13.555244+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4571779879999998,"stat_sum":{"num_bytes":3086231,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":121,"num_read_kb":9749,"num_write":1336,"num_write_kb":28981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"89'459","reported_seq":625,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:52.092030+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T09:27:52.092030+0000","last_peered":"2024-03-09T09:27:52.092030+0000","last_clean":"2024-03-09T09:27:52.092030+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T09:27:52.092030+0000","last_undegraded":"2024-03-09T09:27:52.092030+0000","last_fullsized":"2024-03-09T09:27:52.092030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":55,"num_read_kb":0,"num_write":535,"num_write_kb":83331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'2158","reported_seq":5152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.068307+0000","last_change":"2024-03-09T09:27:13.836207+0000","last_active":"2024-03-09T09:29:14.068307+0000","last_peered":"2024-03-09T09:29:14.068307+0000","last_clean":"2024-03-09T09:29:14.068307+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T09:29:14.068307+0000","last_undegraded":"2024-03-09T09:29:14.068307+0000","last_fullsized":"2024-03-09T09:29:14.068307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2158,"log_dups_size":0,"ondisk_log_size":2158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:20:22.564401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.3715434019999999,"stat_sum":{"num_bytes":1052383,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":116,"num_read_kb":6444,"num_write":1289,"num_write_kb":22757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"89'569","reported_seq":727,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:51.106524+0000","last_change":"2024-03-09T08:52:28.577069+0000","last_active":"2024-03-09T09:27:51.106524+0000","last_peered":"2024-03-09T09:27:51.106524+0000","last_clean":"2024-03-09T09:27:51.106524+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T09:27:51.106524+0000","last_undegraded":"2024-03-09T09:27:51.106524+0000","last_fullsized":"2024-03-09T09:27:51.106524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:09:02.410915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":598,"num_write_kb":95071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"89'2154","reported_seq":5089,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.560248+0000","last_change":"2024-03-09T09:27:32.079362+0000","last_active":"2024-03-09T09:28:32.560248+0000","last_peered":"2024-03-09T09:28:32.560248+0000","last_clean":"2024-03-09T09:28:32.560248+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T09:28:32.560248+0000","last_undegraded":"2024-03-09T09:28:32.560248+0000","last_fullsized":"2024-03-09T09:28:32.560248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2154,"log_dups_size":0,"ondisk_log_size":2154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:14:45.458685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.709485658,"stat_sum":{"num_bytes":1197226,"num_objects":92,"num_object_clones":31,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":31,"num_read":105,"num_read_kb":17945,"num_write":1287,"num_write_kb":45184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"89'2226","reported_seq":5194,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.062389+0000","last_change":"2024-03-09T09:27:07.936292+0000","last_active":"2024-03-09T09:29:14.062389+0000","last_peered":"2024-03-09T09:29:14.062389+0000","last_clean":"2024-03-09T09:29:14.062389+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T09:29:14.062389+0000","last_undegraded":"2024-03-09T09:29:14.062389+0000","last_fullsized":"2024-03-09T09:29:14.062389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2226,"log_dups_size":0,"ondisk_log_size":2226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:15:17.296354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.2630355259999999,"stat_sum":{"num_bytes":1174090,"num_objects":98,"num_object_clones":35,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":35,"num_read":98,"num_read_kb":18109,"num_write":1319,"num_write_kb":36158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"89'817","reported_seq":985,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.795764+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T09:28:14.795764+0000","last_peered":"2024-03-09T09:28:14.795764+0000","last_clean":"2024-03-09T09:28:14.795764+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T09:28:14.795764+0000","last_undegraded":"2024-03-09T09:28:14.795764+0000","last_fullsized":"2024-03-09T09:28:14.795764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":48,"num_read_kb":0,"num_write":879,"num_write_kb":99632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"89'743","reported_seq":903,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.514530+0000","last_change":"2024-03-09T08:52:28.570631+0000","last_active":"2024-03-09T09:28:31.514530+0000","last_peered":"2024-03-09T09:28:31.514530+0000","last_clean":"2024-03-09T09:28:31.514530+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T09:28:31.514530+0000","last_undegraded":"2024-03-09T09:28:31.514530+0000","last_fullsized":"2024-03-09T09:28:31.514530+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:06.630704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971838,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":820,"num_write_kb":111945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"89'2187","reported_seq":5097,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.736598+0000","last_change":"2024-03-09T09:27:30.893758+0000","last_active":"2024-03-09T09:28:56.736598+0000","last_peered":"2024-03-09T09:28:56.736598+0000","last_clean":"2024-03-09T09:28:56.736598+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T09:28:56.736598+0000","last_undegraded":"2024-03-09T09:28:56.736598+0000","last_fullsized":"2024-03-09T09:28:56.736598+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2187,"log_dups_size":0,"ondisk_log_size":2187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:16.469362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.63600609399999997,"stat_sum":{"num_bytes":1331264,"num_objects":108,"num_object_clones":38,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":38,"num_read":110,"num_read_kb":9978,"num_write":1295,"num_write_kb":30848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'2264","reported_seq":5240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.036656+0000","last_change":"2024-03-09T09:27:12.385283+0000","last_active":"2024-03-09T09:29:14.036656+0000","last_peered":"2024-03-09T09:29:14.036656+0000","last_clean":"2024-03-09T09:29:14.036656+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T09:29:14.036656+0000","last_undegraded":"2024-03-09T09:29:14.036656+0000","last_fullsized":"2024-03-09T09:29:14.036656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2264,"log_dups_size":0,"ondisk_log_size":2264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:24:11.227698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.69905185199999997,"stat_sum":{"num_bytes":1031167,"num_objects":108,"num_object_clones":39,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":39,"num_read":109,"num_read_kb":22315,"num_write":1384,"num_write_kb":51621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'445","reported_seq":614,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.253653+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T09:28:16.253653+0000","last_peered":"2024-03-09T09:28:16.253653+0000","last_clean":"2024-03-09T09:28:16.253653+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T09:28:16.253653+0000","last_undegraded":"2024-03-09T09:28:16.253653+0000","last_fullsized":"2024-03-09T09:28:16.253653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:45:45.612953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":467,"num_write_kb":115624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"89'633","reported_seq":793,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.652095+0000","last_change":"2024-03-09T08:52:28.577110+0000","last_active":"2024-03-09T09:28:26.652095+0000","last_peered":"2024-03-09T09:28:26.652095+0000","last_clean":"2024-03-09T09:28:26.652095+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T09:28:26.652095+0000","last_undegraded":"2024-03-09T09:28:26.652095+0000","last_fullsized":"2024-03-09T09:28:26.652095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:10:57.197373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":674,"num_write_kb":99647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"89'2250","reported_seq":5168,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.732049+0000","last_change":"2024-03-09T09:27:34.263088+0000","last_active":"2024-03-09T09:28:56.732049+0000","last_peered":"2024-03-09T09:28:56.732049+0000","last_clean":"2024-03-09T09:28:56.732049+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T09:28:56.732049+0000","last_undegraded":"2024-03-09T09:28:56.732049+0000","last_fullsized":"2024-03-09T09:28:56.732049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2250,"log_dups_size":0,"ondisk_log_size":2250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:15:00.105256+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":3.9938263410000001,"stat_sum":{"num_bytes":1661343,"num_objects":107,"num_object_clones":33,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":33,"num_read":118,"num_read_kb":35803,"num_write":1365,"num_write_kb":51532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'2351","reported_seq":5334,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.039556+0000","last_change":"2024-03-09T09:27:12.068457+0000","last_active":"2024-03-09T09:29:14.039556+0000","last_peered":"2024-03-09T09:29:14.039556+0000","last_clean":"2024-03-09T09:29:14.039556+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T09:29:14.039556+0000","last_undegraded":"2024-03-09T09:29:14.039556+0000","last_fullsized":"2024-03-09T09:29:14.039556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2351,"log_dups_size":0,"ondisk_log_size":2351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:13.795599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.380889329,"stat_sum":{"num_bytes":1156693,"num_objects":132,"num_object_clones":46,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":46,"num_read":122,"num_read_kb":38030,"num_write":1453,"num_write_kb":61557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"89'644","reported_seq":813,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.817502+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T09:28:14.817502+0000","last_peered":"2024-03-09T09:28:14.817502+0000","last_clean":"2024-03-09T09:28:14.817502+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T09:28:14.817502+0000","last_undegraded":"2024-03-09T09:28:14.817502+0000","last_fullsized":"2024-03-09T09:28:14.817502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:21:59.990396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":715,"num_write_kb":95464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"89'626","reported_seq":786,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:27.476620+0000","last_change":"2024-03-09T08:52:28.576254+0000","last_active":"2024-03-09T09:28:27.476620+0000","last_peered":"2024-03-09T09:28:27.476620+0000","last_clean":"2024-03-09T09:28:27.476620+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T09:28:27.476620+0000","last_undegraded":"2024-03-09T09:28:27.476620+0000","last_fullsized":"2024-03-09T09:28:27.476620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:21:56.330955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748758,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":685,"num_write_kb":95366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"89'2201","reported_seq":5127,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.920162+0000","last_change":"2024-03-09T09:27:34.731118+0000","last_active":"2024-03-09T09:29:17.920162+0000","last_peered":"2024-03-09T09:29:17.920162+0000","last_clean":"2024-03-09T09:29:17.920162+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T09:29:17.920162+0000","last_undegraded":"2024-03-09T09:29:17.920162+0000","last_fullsized":"2024-03-09T09:29:17.920162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2201,"log_dups_size":0,"ondisk_log_size":2201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:12.794501+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4556873890000004,"stat_sum":{"num_bytes":4719568,"num_objects":111,"num_object_clones":34,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":34,"num_read":112,"num_read_kb":26370,"num_write":1323,"num_write_kb":45566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"89'2296","reported_seq":5256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.039509+0000","last_change":"2024-03-09T09:27:08.625397+0000","last_active":"2024-03-09T09:29:14.039509+0000","last_peered":"2024-03-09T09:29:14.039509+0000","last_clean":"2024-03-09T09:29:14.039509+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T09:29:14.039509+0000","last_undegraded":"2024-03-09T09:29:14.039509+0000","last_fullsized":"2024-03-09T09:29:14.039509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2296,"log_dups_size":0,"ondisk_log_size":2296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:26:08.206478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0901323600000001,"stat_sum":{"num_bytes":867761,"num_objects":98,"num_object_clones":31,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":31,"num_read":98,"num_read_kb":37765,"num_write":1389,"num_write_kb":74512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'434","reported_seq":603,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.795677+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T09:28:14.795677+0000","last_peered":"2024-03-09T09:28:14.795677+0000","last_clean":"2024-03-09T09:28:14.795677+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T09:28:14.795677+0000","last_undegraded":"2024-03-09T09:28:14.795677+0000","last_fullsized":"2024-03-09T09:28:14.795677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":69,"num_read_kb":0,"num_write":510,"num_write_kb":91498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"89'614","reported_seq":773,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.546373+0000","last_change":"2024-03-09T08:52:28.573638+0000","last_active":"2024-03-09T09:28:26.546373+0000","last_peered":"2024-03-09T09:28:26.546373+0000","last_clean":"2024-03-09T09:28:26.546373+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T09:28:26.546373+0000","last_undegraded":"2024-03-09T09:28:26.546373+0000","last_fullsized":"2024-03-09T09:28:26.546373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:02:02.478112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":666,"num_write_kb":107635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"89'2336","reported_seq":5260,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.951528+0000","last_change":"2024-03-09T09:27:34.495452+0000","last_active":"2024-03-09T09:29:17.951528+0000","last_peered":"2024-03-09T09:29:17.951528+0000","last_clean":"2024-03-09T09:29:17.951528+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T09:29:17.951528+0000","last_undegraded":"2024-03-09T09:29:17.951528+0000","last_fullsized":"2024-03-09T09:29:17.951528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2336,"log_dups_size":0,"ondisk_log_size":2336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:06.642191+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.1575832950000002,"stat_sum":{"num_bytes":392237,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":136,"num_read_kb":53607,"num_write":1404,"num_write_kb":81678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"89'2175","reported_seq":5267,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.022103+0000","last_change":"2024-03-09T09:27:09.510565+0000","last_active":"2024-03-09T09:29:14.022103+0000","last_peered":"2024-03-09T09:29:14.022103+0000","last_clean":"2024-03-09T09:29:14.022103+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T09:29:14.022103+0000","last_undegraded":"2024-03-09T09:29:14.022103+0000","last_fullsized":"2024-03-09T09:29:14.022103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:04:13.602716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.38281627099999999,"stat_sum":{"num_bytes":951718,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":197,"num_read_kb":25950,"num_write":1332,"num_write_kb":45421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'345","reported_seq":514,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:58.111717+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T09:27:58.111717+0000","last_peered":"2024-03-09T09:27:58.111717+0000","last_clean":"2024-03-09T09:27:58.111717+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T09:27:58.111717+0000","last_undegraded":"2024-03-09T09:27:58.111717+0000","last_fullsized":"2024-03-09T09:27:58.111717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:35:38.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":397,"num_write_kb":78578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"89'1700","reported_seq":2916,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.642425+0000","last_change":"2024-03-09T08:52:28.575630+0000","last_active":"2024-03-09T09:28:29.642425+0000","last_peered":"2024-03-09T09:28:29.642425+0000","last_clean":"2024-03-09T09:28:29.642425+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T09:28:29.642425+0000","last_undegraded":"2024-03-09T09:28:29.642425+0000","last_fullsized":"2024-03-09T09:28:29.642425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1700,"log_dups_size":0,"ondisk_log_size":1700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:38:48.210921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":1117,"num_read_kb":4814,"num_write":1762,"num_write_kb":112708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"89'2107","reported_seq":5058,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.735789+0000","last_change":"2024-03-09T09:27:34.095045+0000","last_active":"2024-03-09T09:28:56.735789+0000","last_peered":"2024-03-09T09:28:56.735789+0000","last_clean":"2024-03-09T09:28:56.735789+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T09:28:56.735789+0000","last_undegraded":"2024-03-09T09:28:56.735789+0000","last_fullsized":"2024-03-09T09:28:56.735789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2107,"log_dups_size":0,"ondisk_log_size":2107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:05:04.126062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.8297685179999998,"stat_sum":{"num_bytes":268964,"num_objects":107,"num_object_clones":32,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":32,"num_read":107,"num_read_kb":24622,"num_write":1287,"num_write_kb":41444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"89'2254","reported_seq":5300,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.043906+0000","last_change":"2024-03-09T09:27:06.300751+0000","last_active":"2024-03-09T09:29:14.043906+0000","last_peered":"2024-03-09T09:29:14.043906+0000","last_clean":"2024-03-09T09:29:14.043906+0000","last_became_active":"2024-03-09T08:53:00.240844+0000","last_became_peered":"2024-03-09T08:53:00.240844+0000","last_unstale":"2024-03-09T09:29:14.043906+0000","last_undegraded":"2024-03-09T09:29:14.043906+0000","last_fullsized":"2024-03-09T09:29:14.043906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2254,"log_dups_size":0,"ondisk_log_size":2254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:21:13.503778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.78540855600000004,"stat_sum":{"num_bytes":1854558,"num_objects":126,"num_object_clones":46,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":46,"num_read":216,"num_read_kb":39239,"num_write":1332,"num_write_kb":71797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"89'1558","reported_seq":1727,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:07.567056+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T09:29:07.567056+0000","last_peered":"2024-03-09T09:29:07.567056+0000","last_clean":"2024-03-09T09:29:07.567056+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T09:29:07.567056+0000","last_undegraded":"2024-03-09T09:29:07.567056+0000","last_fullsized":"2024-03-09T09:29:07.567056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1558,"log_dups_size":0,"ondisk_log_size":1558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:30.201471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":57,"num_read_kb":0,"num_write":3420,"num_write_kb":84853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"89'672","reported_seq":833,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.478737+0000","last_change":"2024-03-09T08:52:28.586702+0000","last_active":"2024-03-09T09:28:31.478737+0000","last_peered":"2024-03-09T09:28:31.478737+0000","last_clean":"2024-03-09T09:28:31.478737+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T09:28:31.478737+0000","last_undegraded":"2024-03-09T09:28:31.478737+0000","last_fullsized":"2024-03-09T09:28:31.478737+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:09.201224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":71,"num_read_kb":0,"num_write":764,"num_write_kb":87842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"89'2333","reported_seq":5217,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.713975+0000","last_change":"2024-03-09T09:27:30.887844+0000","last_active":"2024-03-09T09:28:56.713975+0000","last_peered":"2024-03-09T09:28:56.713975+0000","last_clean":"2024-03-09T09:28:56.713975+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T09:28:56.713975+0000","last_undegraded":"2024-03-09T09:28:56.713975+0000","last_fullsized":"2024-03-09T09:28:56.713975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2333,"log_dups_size":0,"ondisk_log_size":2333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:11:46.526489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65806592200000003,"stat_sum":{"num_bytes":2259601,"num_objects":134,"num_object_clones":47,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":47,"num_read":108,"num_read_kb":22295,"num_write":1398,"num_write_kb":43486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"89'2181","reported_seq":5178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.005893+0000","last_change":"2024-03-09T09:27:09.896878+0000","last_active":"2024-03-09T09:29:14.005893+0000","last_peered":"2024-03-09T09:29:14.005893+0000","last_clean":"2024-03-09T09:29:14.005893+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T09:29:14.005893+0000","last_undegraded":"2024-03-09T09:29:14.005893+0000","last_fullsized":"2024-03-09T09:29:14.005893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2181,"log_dups_size":0,"ondisk_log_size":2181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:38:09.744644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.96663918400000004,"stat_sum":{"num_bytes":2499803,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":112,"num_read_kb":5934,"num_write":1320,"num_write_kb":25592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"89'642","reported_seq":811,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.817501+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T09:28:14.817501+0000","last_peered":"2024-03-09T09:28:14.817501+0000","last_clean":"2024-03-09T09:28:14.817501+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T09:28:14.817501+0000","last_undegraded":"2024-03-09T09:28:14.817501+0000","last_fullsized":"2024-03-09T09:28:14.817501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":75,"num_read_kb":0,"num_write":726,"num_write_kb":96134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"89'573","reported_seq":734,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:13.273576+0000","last_change":"2024-03-09T08:52:28.580879+0000","last_active":"2024-03-09T09:29:13.273576+0000","last_peered":"2024-03-09T09:29:13.273576+0000","last_clean":"2024-03-09T09:29:13.273576+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T09:29:13.273576+0000","last_undegraded":"2024-03-09T09:29:13.273576+0000","last_fullsized":"2024-03-09T09:29:13.273576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:19:44.652057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6515690,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":614,"num_write_kb":72903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"89'2273","reported_seq":5171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.939855+0000","last_change":"2024-03-09T09:27:31.029951+0000","last_active":"2024-03-09T09:28:30.939855+0000","last_peered":"2024-03-09T09:28:30.939855+0000","last_clean":"2024-03-09T09:28:30.939855+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T09:28:30.939855+0000","last_undegraded":"2024-03-09T09:28:30.939855+0000","last_fullsized":"2024-03-09T09:28:30.939855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2273,"log_dups_size":0,"ondisk_log_size":2273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:38.142661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.71811667999999995,"stat_sum":{"num_bytes":1846155,"num_objects":126,"num_object_clones":42,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":42,"num_read":111,"num_read_kb":21516,"num_write":1347,"num_write_kb":38338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"89'2284","reported_seq":5231,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.029735+0000","last_change":"2024-03-09T09:27:14.280348+0000","last_active":"2024-03-09T09:29:14.029735+0000","last_peered":"2024-03-09T09:29:14.029735+0000","last_clean":"2024-03-09T09:29:14.029735+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T09:29:14.029735+0000","last_undegraded":"2024-03-09T09:29:14.029735+0000","last_fullsized":"2024-03-09T09:29:14.029735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:42:30.439855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.815577539,"stat_sum":{"num_bytes":1586992,"num_objects":105,"num_object_clones":35,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":35,"num_read":97,"num_read_kb":33815,"num_write":1338,"num_write_kb":74580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"89'546","reported_seq":715,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:03.302448+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T09:28:03.302448+0000","last_peered":"2024-03-09T09:28:03.302448+0000","last_clean":"2024-03-09T09:28:03.302448+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T09:28:03.302448+0000","last_undegraded":"2024-03-09T09:28:03.302448+0000","last_fullsized":"2024-03-09T09:28:03.302448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":620,"num_write_kb":120324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"89'552","reported_seq":713,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:56.663234+0000","last_change":"2024-03-09T08:52:28.584457+0000","last_active":"2024-03-09T09:27:56.663234+0000","last_peered":"2024-03-09T09:27:56.663234+0000","last_clean":"2024-03-09T09:27:56.663234+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T09:27:56.663234+0000","last_undegraded":"2024-03-09T09:27:56.663234+0000","last_fullsized":"2024-03-09T09:27:56.663234+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:45.833876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":563,"num_write_kb":103368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"89'2260","reported_seq":5160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.938153+0000","last_change":"2024-03-09T09:27:31.026391+0000","last_active":"2024-03-09T09:28:30.938153+0000","last_peered":"2024-03-09T09:28:30.938153+0000","last_clean":"2024-03-09T09:28:30.938153+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T09:28:30.938153+0000","last_undegraded":"2024-03-09T09:28:30.938153+0000","last_fullsized":"2024-03-09T09:28:30.938153+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2260,"log_dups_size":0,"ondisk_log_size":2260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:49:57.313638+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.71599741500000003,"stat_sum":{"num_bytes":1873672,"num_objects":101,"num_object_clones":34,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":34,"num_read":103,"num_read_kb":19624,"num_write":1351,"num_write_kb":43427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"89'2260","reported_seq":5236,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.039799+0000","last_change":"2024-03-09T09:27:08.957379+0000","last_active":"2024-03-09T09:29:14.039799+0000","last_peered":"2024-03-09T09:29:14.039799+0000","last_clean":"2024-03-09T09:29:14.039799+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T09:29:14.039799+0000","last_undegraded":"2024-03-09T09:29:14.039799+0000","last_fullsized":"2024-03-09T09:29:14.039799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2260,"log_dups_size":0,"ondisk_log_size":2260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:57:34.155091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.4919903310000002,"stat_sum":{"num_bytes":2681803,"num_objects":128,"num_object_clones":43,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":43,"num_read":104,"num_read_kb":25781,"num_write":1386,"num_write_kb":55300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"89'737","reported_seq":906,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.252650+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T09:28:16.252650+0000","last_peered":"2024-03-09T09:28:16.252650+0000","last_clean":"2024-03-09T09:28:16.252650+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T09:28:16.252650+0000","last_undegraded":"2024-03-09T09:28:16.252650+0000","last_fullsized":"2024-03-09T09:28:16.252650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":833,"num_write_kb":103821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'561","reported_seq":722,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.554450+0000","last_change":"2024-03-09T08:52:28.578096+0000","last_active":"2024-03-09T09:27:57.554450+0000","last_peered":"2024-03-09T09:27:57.554450+0000","last_clean":"2024-03-09T09:27:57.554450+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T09:27:57.554450+0000","last_undegraded":"2024-03-09T09:27:57.554450+0000","last_fullsized":"2024-03-09T09:27:57.554450+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:19:42.645517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":52,"num_read_kb":0,"num_write":616,"num_write_kb":119874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"89'2163","reported_seq":5096,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.976905+0000","last_change":"2024-03-09T09:27:30.887413+0000","last_active":"2024-03-09T09:28:30.976905+0000","last_peered":"2024-03-09T09:28:30.976905+0000","last_clean":"2024-03-09T09:28:30.976905+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T09:28:30.976905+0000","last_undegraded":"2024-03-09T09:28:30.976905+0000","last_fullsized":"2024-03-09T09:28:30.976905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2163,"log_dups_size":0,"ondisk_log_size":2163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:17:41.847948+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.63049624100000001,"stat_sum":{"num_bytes":833172,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":114,"num_read_kb":17857,"num_write":1302,"num_write_kb":44488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"89'2250","reported_seq":5226,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.030973+0000","last_change":"2024-03-09T09:27:10.015883+0000","last_active":"2024-03-09T09:29:14.030973+0000","last_peered":"2024-03-09T09:29:14.030973+0000","last_clean":"2024-03-09T09:29:14.030973+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T09:29:14.030973+0000","last_undegraded":"2024-03-09T09:29:14.030973+0000","last_fullsized":"2024-03-09T09:29:14.030973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2250,"log_dups_size":0,"ondisk_log_size":2250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:23:02.726704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.18447864,"stat_sum":{"num_bytes":1122276,"num_objects":117,"num_object_clones":43,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":43,"num_read":101,"num_read_kb":18409,"num_write":1364,"num_write_kb":39400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"89'1830","reported_seq":3014,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.251658+0000","last_change":"2024-03-09T08:52:17.271847+0000","last_active":"2024-03-09T09:28:16.251658+0000","last_peered":"2024-03-09T09:28:16.251658+0000","last_clean":"2024-03-09T09:28:16.251658+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T09:28:16.251658+0000","last_undegraded":"2024-03-09T09:28:16.251658+0000","last_fullsized":"2024-03-09T09:28:16.251658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1830,"log_dups_size":0,"ondisk_log_size":1830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:27.418674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":1037,"num_read_kb":4764,"num_write":1840,"num_write_kb":108217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"89'404","reported_seq":564,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.641753+0000","last_change":"2024-03-09T08:52:28.588636+0000","last_active":"2024-03-09T09:28:29.641753+0000","last_peered":"2024-03-09T09:28:29.641753+0000","last_clean":"2024-03-09T09:28:29.641753+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T09:28:29.641753+0000","last_undegraded":"2024-03-09T09:28:29.641753+0000","last_fullsized":"2024-03-09T09:28:29.641753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:19:21.601182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":44,"num_read_kb":0,"num_write":445,"num_write_kb":78896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"89'2184","reported_seq":5126,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.923858+0000","last_change":"2024-03-09T09:27:34.760081+0000","last_active":"2024-03-09T09:29:17.923858+0000","last_peered":"2024-03-09T09:29:17.923858+0000","last_clean":"2024-03-09T09:29:17.923858+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T09:29:17.923858+0000","last_undegraded":"2024-03-09T09:29:17.923858+0000","last_fullsized":"2024-03-09T09:29:17.923858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2184,"log_dups_size":0,"ondisk_log_size":2184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:30:07.164361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4843044489999997,"stat_sum":{"num_bytes":1636574,"num_objects":123,"num_object_clones":44,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":44,"num_read":134,"num_read_kb":27719,"num_write":1302,"num_write_kb":59461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'2127","reported_seq":5120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.054947+0000","last_change":"2024-03-09T09:27:09.413913+0000","last_active":"2024-03-09T09:29:14.054947+0000","last_peered":"2024-03-09T09:29:14.054947+0000","last_clean":"2024-03-09T09:29:14.054947+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T09:29:14.054947+0000","last_undegraded":"2024-03-09T09:29:14.054947+0000","last_fullsized":"2024-03-09T09:29:14.054947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2127,"log_dups_size":0,"ondisk_log_size":2127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:36:18.637176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.289202336,"stat_sum":{"num_bytes":424328,"num_objects":113,"num_object_clones":35,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":35,"num_read":94,"num_read_kb":17266,"num_write":1284,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"89'580","reported_seq":750,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.074969+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T09:28:23.074969+0000","last_peered":"2024-03-09T09:28:23.074969+0000","last_clean":"2024-03-09T09:28:23.074969+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T09:28:23.074969+0000","last_undegraded":"2024-03-09T09:28:23.074969+0000","last_fullsized":"2024-03-09T09:28:23.074969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":598,"num_write_kb":82993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"89'1592","reported_seq":1752,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:48.494834+0000","last_change":"2024-03-09T08:52:28.578599+0000","last_active":"2024-03-09T09:28:48.494834+0000","last_peered":"2024-03-09T09:28:48.494834+0000","last_clean":"2024-03-09T09:28:48.494834+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T09:28:48.494834+0000","last_undegraded":"2024-03-09T09:28:48.494834+0000","last_fullsized":"2024-03-09T09:28:48.494834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1592,"log_dups_size":0,"ondisk_log_size":1592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:52:31.328248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":3473,"num_write_kb":84746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"89'2236","reported_seq":5167,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.928826+0000","last_change":"2024-03-09T09:27:34.687418+0000","last_active":"2024-03-09T09:29:17.928826+0000","last_peered":"2024-03-09T09:29:17.928826+0000","last_clean":"2024-03-09T09:29:17.928826+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T09:29:17.928826+0000","last_undegraded":"2024-03-09T09:29:17.928826+0000","last_fullsized":"2024-03-09T09:29:17.928826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2236,"log_dups_size":0,"ondisk_log_size":2236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:18:43.296322+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4503168610000001,"stat_sum":{"num_bytes":1665642,"num_objects":121,"num_object_clones":41,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":41,"num_read":126,"num_read_kb":42837,"num_write":1341,"num_write_kb":67584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'2308","reported_seq":5269,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.052272+0000","last_change":"2024-03-09T09:27:12.261576+0000","last_active":"2024-03-09T09:29:14.052272+0000","last_peered":"2024-03-09T09:29:14.052272+0000","last_clean":"2024-03-09T09:29:14.052272+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T09:29:14.052272+0000","last_undegraded":"2024-03-09T09:29:14.052272+0000","last_fullsized":"2024-03-09T09:29:14.052272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2308,"log_dups_size":0,"ondisk_log_size":2308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:07:46.894530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.56599842099999997,"stat_sum":{"num_bytes":1847778,"num_objects":137,"num_object_clones":49,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":49,"num_read":115,"num_read_kb":13750,"num_write":1371,"num_write_kb":55061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'626","reported_seq":795,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:11.134820+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T09:28:11.134820+0000","last_peered":"2024-03-09T09:28:11.134820+0000","last_clean":"2024-03-09T09:28:11.134820+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T09:28:11.134820+0000","last_undegraded":"2024-03-09T09:28:11.134820+0000","last_fullsized":"2024-03-09T09:28:11.134820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":698,"num_write_kb":70663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":326,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:40.174178+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T09:27:40.174178+0000","last_peered":"2024-03-09T09:27:40.174178+0000","last_clean":"2024-03-09T09:27:40.174178+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T09:27:40.174178+0000","last_undegraded":"2024-03-09T09:27:40.174178+0000","last_fullsized":"2024-03-09T09:27:40.174178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'680","reported_seq":841,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:02.565362+0000","last_change":"2024-03-09T08:52:28.577973+0000","last_active":"2024-03-09T09:28:02.565362+0000","last_peered":"2024-03-09T09:28:02.565362+0000","last_clean":"2024-03-09T09:28:02.565362+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T09:28:02.565362+0000","last_undegraded":"2024-03-09T09:28:02.565362+0000","last_fullsized":"2024-03-09T09:28:02.565362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:22:13.291622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":743,"num_write_kb":95766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"89'2243","reported_seq":5171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.944320+0000","last_change":"2024-03-09T09:27:30.919984+0000","last_active":"2024-03-09T09:28:30.944320+0000","last_peered":"2024-03-09T09:28:30.944320+0000","last_clean":"2024-03-09T09:28:30.944320+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T09:28:30.944320+0000","last_undegraded":"2024-03-09T09:28:30.944320+0000","last_fullsized":"2024-03-09T09:28:30.944320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2243,"log_dups_size":0,"ondisk_log_size":2243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:42:58.142985+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.66264078000000004,"stat_sum":{"num_bytes":792938,"num_objects":106,"num_object_clones":38,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":38,"num_read":125,"num_read_kb":17823,"num_write":1366,"num_write_kb":27629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'2242","reported_seq":5186,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.060199+0000","last_change":"2024-03-09T09:27:06.276304+0000","last_active":"2024-03-09T09:29:14.060199+0000","last_peered":"2024-03-09T09:29:14.060199+0000","last_clean":"2024-03-09T09:29:14.060199+0000","last_became_active":"2024-03-09T08:53:00.437859+0000","last_became_peered":"2024-03-09T08:53:00.437859+0000","last_unstale":"2024-03-09T09:29:14.060199+0000","last_undegraded":"2024-03-09T09:29:14.060199+0000","last_fullsized":"2024-03-09T09:29:14.060199+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2242,"log_dups_size":0,"ondisk_log_size":2242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:07:36.832520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.79260830400000004,"stat_sum":{"num_bytes":1456716,"num_objects":115,"num_object_clones":40,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":40,"num_read":109,"num_read_kb":18518,"num_write":1301,"num_write_kb":37816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"89'501","reported_seq":669,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.901581+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T09:28:14.901581+0000","last_peered":"2024-03-09T09:28:14.901581+0000","last_clean":"2024-03-09T09:28:14.901581+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T09:28:14.901581+0000","last_undegraded":"2024-03-09T09:28:14.901581+0000","last_fullsized":"2024-03-09T09:28:14.901581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:27.035280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":39,"num_read_kb":0,"num_write":538,"num_write_kb":99417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"89'703","reported_seq":863,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.545223+0000","last_change":"2024-03-09T08:52:28.589278+0000","last_active":"2024-03-09T09:28:26.545223+0000","last_peered":"2024-03-09T09:28:26.545223+0000","last_clean":"2024-03-09T09:28:26.545223+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T09:28:26.545223+0000","last_undegraded":"2024-03-09T09:28:26.545223+0000","last_fullsized":"2024-03-09T09:28:26.545223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:16:26.449275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":780,"num_write_kb":116155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"89'2189","reported_seq":5136,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.910057+0000","last_change":"2024-03-09T09:27:30.904584+0000","last_active":"2024-03-09T09:28:30.910057+0000","last_peered":"2024-03-09T09:28:30.910057+0000","last_clean":"2024-03-09T09:28:30.910057+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T09:28:30.910057+0000","last_undegraded":"2024-03-09T09:28:30.910057+0000","last_fullsized":"2024-03-09T09:28:30.910057+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2189,"log_dups_size":0,"ondisk_log_size":2189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:42:21.944851+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.647865472,"stat_sum":{"num_bytes":999257,"num_objects":92,"num_object_clones":32,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":32,"num_read":116,"num_read_kb":42999,"num_write":1346,"num_write_kb":55653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"89'2187","reported_seq":5181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048275+0000","last_change":"2024-03-09T09:27:14.415883+0000","last_active":"2024-03-09T09:29:14.048275+0000","last_peered":"2024-03-09T09:29:14.048275+0000","last_clean":"2024-03-09T09:29:14.048275+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T09:29:14.048275+0000","last_undegraded":"2024-03-09T09:29:14.048275+0000","last_fullsized":"2024-03-09T09:29:14.048275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2187,"log_dups_size":0,"ondisk_log_size":2187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:34:33.950832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.951760857,"stat_sum":{"num_bytes":1333229,"num_objects":97,"num_object_clones":33,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":33,"num_read":100,"num_read_kb":17450,"num_write":1313,"num_write_kb":41729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"89'845","reported_seq":1013,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.817246+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T09:28:14.817246+0000","last_peered":"2024-03-09T09:28:14.817246+0000","last_clean":"2024-03-09T09:28:14.817246+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T09:28:14.817246+0000","last_undegraded":"2024-03-09T09:28:14.817246+0000","last_fullsized":"2024-03-09T09:28:14.817246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:26:35.574903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971641,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":912,"num_write_kb":111865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"89'796","reported_seq":956,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:06.427393+0000","last_change":"2024-03-09T08:52:28.583181+0000","last_active":"2024-03-09T09:28:06.427393+0000","last_peered":"2024-03-09T09:28:06.427393+0000","last_clean":"2024-03-09T09:28:06.427393+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T09:28:06.427393+0000","last_undegraded":"2024-03-09T09:28:06.427393+0000","last_fullsized":"2024-03-09T09:28:06.427393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:58:50.061324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":51,"num_read_kb":0,"num_write":872,"num_write_kb":99652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"89'2237","reported_seq":5132,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.931245+0000","last_change":"2024-03-09T09:27:34.640947+0000","last_active":"2024-03-09T09:29:17.931245+0000","last_peered":"2024-03-09T09:29:17.931245+0000","last_clean":"2024-03-09T09:29:17.931245+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T09:29:17.931245+0000","last_undegraded":"2024-03-09T09:29:17.931245+0000","last_fullsized":"2024-03-09T09:29:17.931245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2237,"log_dups_size":0,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:51:18.312332+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4098675719999996,"stat_sum":{"num_bytes":603203,"num_objects":100,"num_object_clones":32,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":32,"num_read":95,"num_read_kb":13936,"num_write":1338,"num_write_kb":41727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"89'2197","reported_seq":5190,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.040606+0000","last_change":"2024-03-09T09:27:13.656604+0000","last_active":"2024-03-09T09:29:14.040606+0000","last_peered":"2024-03-09T09:29:14.040606+0000","last_clean":"2024-03-09T09:29:14.040606+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T09:29:14.040606+0000","last_undegraded":"2024-03-09T09:29:14.040606+0000","last_fullsized":"2024-03-09T09:29:14.040606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2197,"log_dups_size":0,"ondisk_log_size":2197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:53:35.952871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.91280589400000001,"stat_sum":{"num_bytes":4608718,"num_objects":119,"num_object_clones":37,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":37,"num_read":119,"num_read_kb":26183,"num_write":1317,"num_write_kb":37436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'672","reported_seq":840,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:17.301970+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T09:28:17.301970+0000","last_peered":"2024-03-09T09:28:17.301970+0000","last_clean":"2024-03-09T09:28:17.301970+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T09:28:17.301970+0000","last_undegraded":"2024-03-09T09:28:17.301970+0000","last_fullsized":"2024-03-09T09:28:17.301970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:16.138216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":724,"num_write_kb":103814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"89'713","reported_seq":873,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:38.294529+0000","last_change":"2024-03-09T08:52:28.587637+0000","last_active":"2024-03-09T09:28:38.294529+0000","last_peered":"2024-03-09T09:28:38.294529+0000","last_clean":"2024-03-09T09:28:38.294529+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T09:28:38.294529+0000","last_undegraded":"2024-03-09T09:28:38.294529+0000","last_fullsized":"2024-03-09T09:28:38.294529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:31:32.362061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":59,"num_read_kb":0,"num_write":793,"num_write_kb":95673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"89'2354","reported_seq":5251,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.737741+0000","last_change":"2024-03-09T09:27:30.891372+0000","last_active":"2024-03-09T09:28:56.737741+0000","last_peered":"2024-03-09T09:28:56.737741+0000","last_clean":"2024-03-09T09:28:56.737741+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T09:28:56.737741+0000","last_undegraded":"2024-03-09T09:28:56.737741+0000","last_fullsized":"2024-03-09T09:28:56.737741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2354,"log_dups_size":0,"ondisk_log_size":2354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:45:35.757078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.634097468,"stat_sum":{"num_bytes":572008,"num_objects":135,"num_object_clones":47,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":47,"num_read":117,"num_read_kb":9742,"num_write":1431,"num_write_kb":32918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"89'2248","reported_seq":5227,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.069831+0000","last_change":"2024-03-09T09:27:13.609914+0000","last_active":"2024-03-09T09:29:14.069831+0000","last_peered":"2024-03-09T09:29:14.069831+0000","last_clean":"2024-03-09T09:29:14.069831+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T09:29:14.069831+0000","last_undegraded":"2024-03-09T09:29:14.069831+0000","last_fullsized":"2024-03-09T09:29:14.069831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2248,"log_dups_size":0,"ondisk_log_size":2248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:55:21.022399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.81694827699999994,"stat_sum":{"num_bytes":851628,"num_objects":105,"num_object_clones":33,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":33,"num_read":114,"num_read_kb":26314,"num_write":1339,"num_write_kb":37517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"89'520","reported_seq":687,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:58.112217+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T09:27:58.112217+0000","last_peered":"2024-03-09T09:27:58.112217+0000","last_clean":"2024-03-09T09:27:58.112217+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T09:27:58.112217+0000","last_undegraded":"2024-03-09T09:27:58.112217+0000","last_fullsized":"2024-03-09T09:27:58.112217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:10:34.698235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554454,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":604,"num_write_kb":91576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"89'683","reported_seq":843,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.545188+0000","last_change":"2024-03-09T08:52:28.582308+0000","last_active":"2024-03-09T09:28:26.545188+0000","last_peered":"2024-03-09T09:28:26.545188+0000","last_clean":"2024-03-09T09:28:26.545188+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T09:28:26.545188+0000","last_undegraded":"2024-03-09T09:28:26.545188+0000","last_fullsized":"2024-03-09T09:28:26.545188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:01:04.918893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":783,"num_write_kb":95503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"89'2304","reported_seq":5195,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.952036+0000","last_change":"2024-03-09T09:27:34.538055+0000","last_active":"2024-03-09T09:29:17.952036+0000","last_peered":"2024-03-09T09:29:17.952036+0000","last_clean":"2024-03-09T09:29:17.952036+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T09:29:17.952036+0000","last_undegraded":"2024-03-09T09:29:17.952036+0000","last_fullsized":"2024-03-09T09:29:17.952036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2304,"log_dups_size":0,"ondisk_log_size":2304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:30:20.481016+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.1951373749999998,"stat_sum":{"num_bytes":1358361,"num_objects":133,"num_object_clones":45,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":45,"num_read":125,"num_read_kb":25752,"num_write":1343,"num_write_kb":45310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"89'2285","reported_seq":5257,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.067779+0000","last_change":"2024-03-09T09:27:13.043792+0000","last_active":"2024-03-09T09:29:14.067779+0000","last_peered":"2024-03-09T09:29:14.067779+0000","last_clean":"2024-03-09T09:29:14.067779+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T09:29:14.067779+0000","last_undegraded":"2024-03-09T09:29:14.067779+0000","last_fullsized":"2024-03-09T09:29:14.067779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:24:58.260830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30155770100000001,"stat_sum":{"num_bytes":730111,"num_objects":106,"num_object_clones":35,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":35,"num_read":106,"num_read_kb":32982,"num_write":1381,"num_write_kb":56548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'750","reported_seq":917,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.253028+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T09:28:16.253028+0000","last_peered":"2024-03-09T09:28:16.253028+0000","last_clean":"2024-03-09T09:28:16.253028+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T09:28:16.253028+0000","last_undegraded":"2024-03-09T09:28:16.253028+0000","last_fullsized":"2024-03-09T09:28:16.253028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:31.859924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":74,"num_read_kb":0,"num_write":870,"num_write_kb":87759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"89'602","reported_seq":761,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.544148+0000","last_change":"2024-03-09T08:52:28.575975+0000","last_active":"2024-03-09T09:28:26.544148+0000","last_peered":"2024-03-09T09:28:26.544148+0000","last_clean":"2024-03-09T09:28:26.544148+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T09:28:26.544148+0000","last_undegraded":"2024-03-09T09:28:26.544148+0000","last_fullsized":"2024-03-09T09:28:26.544148+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:51:24.405316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":669,"num_write_kb":112438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"89'2184","reported_seq":5115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.919953+0000","last_change":"2024-03-09T09:27:34.711852+0000","last_active":"2024-03-09T09:29:17.919953+0000","last_peered":"2024-03-09T09:29:17.919953+0000","last_clean":"2024-03-09T09:29:17.919953+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T09:29:17.919953+0000","last_undegraded":"2024-03-09T09:29:17.919953+0000","last_fullsized":"2024-03-09T09:29:17.919953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2184,"log_dups_size":0,"ondisk_log_size":2184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:07:14.254525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4375742420000002,"stat_sum":{"num_bytes":1378177,"num_objects":100,"num_object_clones":26,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":26,"num_read":107,"num_read_kb":18463,"num_write":1316,"num_write_kb":41976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"89'2181","reported_seq":5172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.034483+0000","last_change":"2024-03-09T09:27:07.994121+0000","last_active":"2024-03-09T09:29:14.034483+0000","last_peered":"2024-03-09T09:29:14.034483+0000","last_clean":"2024-03-09T09:29:14.034483+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T09:29:14.034483+0000","last_undegraded":"2024-03-09T09:29:14.034483+0000","last_fullsized":"2024-03-09T09:29:14.034483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2181,"log_dups_size":0,"ondisk_log_size":2181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:07:11.820222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.3444653409999998,"stat_sum":{"num_bytes":3375625,"num_objects":103,"num_object_clones":31,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":31,"num_read":106,"num_read_kb":36185,"num_write":1311,"num_write_kb":52081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"89'386","reported_seq":553,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.986701+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T09:27:57.986701+0000","last_peered":"2024-03-09T09:27:57.986701+0000","last_clean":"2024-03-09T09:27:57.986701+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T09:27:57.986701+0000","last_undegraded":"2024-03-09T09:27:57.986701+0000","last_fullsized":"2024-03-09T09:27:57.986701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:07:29.577461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":412,"num_write_kb":99378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"89'560","reported_seq":959,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.643843+0000","last_change":"2024-03-09T08:52:28.588312+0000","last_active":"2024-03-09T09:28:29.643843+0000","last_peered":"2024-03-09T09:28:29.643843+0000","last_clean":"2024-03-09T09:28:29.643843+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T09:28:29.643843+0000","last_undegraded":"2024-03-09T09:28:29.643843+0000","last_fullsized":"2024-03-09T09:28:29.643843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:14:52.895618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007622,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":290,"num_read_kb":993,"num_write":609,"num_write_kb":88247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"89'2217","reported_seq":5123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.922142+0000","last_change":"2024-03-09T09:27:34.708914+0000","last_active":"2024-03-09T09:29:17.922142+0000","last_peered":"2024-03-09T09:29:17.922142+0000","last_clean":"2024-03-09T09:29:17.922142+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T09:29:17.922142+0000","last_undegraded":"2024-03-09T09:29:17.922142+0000","last_fullsized":"2024-03-09T09:29:17.922142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2217,"log_dups_size":0,"ondisk_log_size":2217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:43:47.385372+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4357263939999996,"stat_sum":{"num_bytes":1364696,"num_objects":131,"num_object_clones":50,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":50,"num_read":112,"num_read_kb":14161,"num_write":1321,"num_write_kb":29407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'2338","reported_seq":5297,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.068878+0000","last_change":"2024-03-09T09:27:13.390531+0000","last_active":"2024-03-09T09:29:14.068878+0000","last_peered":"2024-03-09T09:29:14.068878+0000","last_clean":"2024-03-09T09:29:14.068878+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T09:29:14.068878+0000","last_undegraded":"2024-03-09T09:29:14.068878+0000","last_fullsized":"2024-03-09T09:29:14.068878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2338,"log_dups_size":0,"ondisk_log_size":2338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:00:16.543112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64721804299999997,"stat_sum":{"num_bytes":2530176,"num_objects":100,"num_object_clones":36,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":36,"num_read":119,"num_read_kb":17537,"num_write":1383,"num_write_kb":31574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'267","reported_seq":434,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:40.238636+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T09:27:40.238636+0000","last_peered":"2024-03-09T09:27:40.238636+0000","last_clean":"2024-03-09T09:27:40.238636+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T09:27:40.238636+0000","last_undegraded":"2024-03-09T09:27:40.238636+0000","last_fullsized":"2024-03-09T09:27:40.238636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":313,"num_write_kb":62737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"89'830","reported_seq":988,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:03.286424+0000","last_change":"2024-03-09T08:52:28.585562+0000","last_active":"2024-03-09T09:29:03.286424+0000","last_peered":"2024-03-09T09:29:03.286424+0000","last_clean":"2024-03-09T09:29:03.286424+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T09:29:03.286424+0000","last_undegraded":"2024-03-09T09:29:03.286424+0000","last_fullsized":"2024-03-09T09:29:03.286424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:28:50.544795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":880,"num_write_kb":124192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"89'2228","reported_seq":5135,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.941262+0000","last_change":"2024-03-09T09:27:34.550507+0000","last_active":"2024-03-09T09:29:17.941262+0000","last_peered":"2024-03-09T09:29:17.941262+0000","last_clean":"2024-03-09T09:29:17.941262+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T09:29:17.941262+0000","last_undegraded":"2024-03-09T09:29:17.941262+0000","last_fullsized":"2024-03-09T09:29:17.941262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2228,"log_dups_size":0,"ondisk_log_size":2228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:36:18.970938+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.2113280319999999,"stat_sum":{"num_bytes":5678953,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":106,"num_read_kb":14364,"num_write":1330,"num_write_kb":47101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"89'2149","reported_seq":5155,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.041773+0000","last_change":"2024-03-09T09:27:09.238850+0000","last_active":"2024-03-09T09:29:14.041773+0000","last_peered":"2024-03-09T09:29:14.041773+0000","last_clean":"2024-03-09T09:29:14.041773+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T09:29:14.041773+0000","last_undegraded":"2024-03-09T09:29:14.041773+0000","last_fullsized":"2024-03-09T09:29:14.041773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2149,"log_dups_size":0,"ondisk_log_size":2149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:13:15.580902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.096588679999999996,"stat_sum":{"num_bytes":1266796,"num_objects":124,"num_object_clones":39,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":39,"num_read":107,"num_read_kb":17887,"num_write":1323,"num_write_kb":41057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"89'594","reported_seq":761,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.117171+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T09:27:57.117171+0000","last_peered":"2024-03-09T09:27:57.117171+0000","last_clean":"2024-03-09T09:27:57.117171+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T09:27:57.117171+0000","last_undegraded":"2024-03-09T09:27:57.117171+0000","last_fullsized":"2024-03-09T09:27:57.117171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:22:05.504557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":687,"num_write_kb":95812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"89'1176","reported_seq":1335,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.644757+0000","last_change":"2024-03-09T08:52:28.585697+0000","last_active":"2024-03-09T09:28:29.644757+0000","last_peered":"2024-03-09T09:28:29.644757+0000","last_clean":"2024-03-09T09:28:29.644757+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T09:28:29.644757+0000","last_undegraded":"2024-03-09T09:28:29.644757+0000","last_fullsized":"2024-03-09T09:28:29.644757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1176,"log_dups_size":0,"ondisk_log_size":1176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:50:23.095008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777306,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":1250,"num_write_kb":83773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"89'2364","reported_seq":5266,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.935770+0000","last_change":"2024-03-09T09:27:34.539439+0000","last_active":"2024-03-09T09:29:17.935770+0000","last_peered":"2024-03-09T09:29:17.935770+0000","last_clean":"2024-03-09T09:29:17.935770+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T09:29:17.935770+0000","last_undegraded":"2024-03-09T09:29:17.935770+0000","last_fullsized":"2024-03-09T09:29:17.935770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2364,"log_dups_size":0,"ondisk_log_size":2364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:04:32.627956+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2009582749999996,"stat_sum":{"num_bytes":273727,"num_objects":96,"num_object_clones":34,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":34,"num_read":130,"num_read_kb":37554,"num_write":1394,"num_write_kb":63694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"89'2221","reported_seq":5185,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.038375+0000","last_change":"2024-03-09T09:27:12.934974+0000","last_active":"2024-03-09T09:29:14.038375+0000","last_peered":"2024-03-09T09:29:14.038375+0000","last_clean":"2024-03-09T09:29:14.038375+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T09:29:14.038375+0000","last_undegraded":"2024-03-09T09:29:14.038375+0000","last_fullsized":"2024-03-09T09:29:14.038375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2221,"log_dups_size":0,"ondisk_log_size":2221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:00:09.930894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19064995900000001,"stat_sum":{"num_bytes":1055946,"num_objects":133,"num_object_clones":51,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":51,"num_read":117,"num_read_kb":18332,"num_write":1305,"num_write_kb":38006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"89'1040","reported_seq":1207,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.156049+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T09:28:23.156049+0000","last_peered":"2024-03-09T09:28:23.156049+0000","last_clean":"2024-03-09T09:28:23.156049+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T09:28:23.156049+0000","last_undegraded":"2024-03-09T09:28:23.156049+0000","last_fullsized":"2024-03-09T09:28:23.156049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1040,"log_dups_size":0,"ondisk_log_size":1040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:01.724043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":58,"num_read_kb":0,"num_write":1092,"num_write_kb":124321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"89'297","reported_seq":456,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.642610+0000","last_change":"2024-03-09T08:52:28.583685+0000","last_active":"2024-03-09T09:28:29.642610+0000","last_peered":"2024-03-09T09:28:29.642610+0000","last_clean":"2024-03-09T09:28:29.642610+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T09:28:29.642610+0000","last_undegraded":"2024-03-09T09:28:29.642610+0000","last_fullsized":"2024-03-09T09:28:29.642610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:58:00.899620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":328,"num_write_kb":62283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"89'2169","reported_seq":5098,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.750241+0000","last_change":"2024-03-09T09:27:32.064467+0000","last_active":"2024-03-09T09:28:56.750241+0000","last_peered":"2024-03-09T09:28:56.750241+0000","last_clean":"2024-03-09T09:28:56.750241+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T09:28:56.750241+0000","last_undegraded":"2024-03-09T09:28:56.750241+0000","last_fullsized":"2024-03-09T09:28:56.750241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2169,"log_dups_size":0,"ondisk_log_size":2169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:45:48.687413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.695386042,"stat_sum":{"num_bytes":3385389,"num_objects":130,"num_object_clones":42,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":42,"num_read":111,"num_read_kb":40727,"num_write":1306,"num_write_kb":67855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"89'2237","reported_seq":5201,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.072301+0000","last_change":"2024-03-09T09:27:08.841098+0000","last_active":"2024-03-09T09:29:14.072301+0000","last_peered":"2024-03-09T09:29:14.072301+0000","last_clean":"2024-03-09T09:29:14.072301+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T09:29:14.072301+0000","last_undegraded":"2024-03-09T09:29:14.072301+0000","last_fullsized":"2024-03-09T09:29:14.072301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2237,"log_dups_size":0,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:41:56.840106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.3666159750000002,"stat_sum":{"num_bytes":799596,"num_objects":94,"num_object_clones":30,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":30,"num_read":103,"num_read_kb":34868,"num_write":1326,"num_write_kb":46782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'1298","reported_seq":1465,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:18.712625+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T09:28:18.712625+0000","last_peered":"2024-03-09T09:28:18.712625+0000","last_clean":"2024-03-09T09:28:18.712625+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T09:28:18.712625+0000","last_undegraded":"2024-03-09T09:28:18.712625+0000","last_fullsized":"2024-03-09T09:28:18.712625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1298,"log_dups_size":0,"ondisk_log_size":1298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:41:10.578669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":80,"num_read_kb":0,"num_write":1400,"num_write_kb":80149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"89'411","reported_seq":570,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:08.402304+0000","last_change":"2024-03-09T08:52:28.568794+0000","last_active":"2024-03-09T09:28:08.402304+0000","last_peered":"2024-03-09T09:28:08.402304+0000","last_clean":"2024-03-09T09:28:08.402304+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T09:28:08.402304+0000","last_undegraded":"2024-03-09T09:28:08.402304+0000","last_fullsized":"2024-03-09T09:28:08.402304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:44:21.721511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":462,"num_write_kb":95092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"89'2267","reported_seq":5188,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.727032+0000","last_change":"2024-03-09T09:27:30.874038+0000","last_active":"2024-03-09T09:28:56.727032+0000","last_peered":"2024-03-09T09:28:56.727032+0000","last_clean":"2024-03-09T09:28:56.727032+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T09:28:56.727032+0000","last_undegraded":"2024-03-09T09:28:56.727032+0000","last_fullsized":"2024-03-09T09:28:56.727032+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2267,"log_dups_size":0,"ondisk_log_size":2267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:11:48.337332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64490501099999997,"stat_sum":{"num_bytes":1735737,"num_objects":103,"num_object_clones":32,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":32,"num_read":113,"num_read_kb":37575,"num_write":1379,"num_write_kb":57434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"89'2291","reported_seq":5242,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.022970+0000","last_change":"2024-03-09T09:27:09.683757+0000","last_active":"2024-03-09T09:29:14.022970+0000","last_peered":"2024-03-09T09:29:14.022970+0000","last_clean":"2024-03-09T09:29:14.022970+0000","last_became_active":"2024-03-09T08:53:00.254589+0000","last_became_peered":"2024-03-09T08:53:00.254589+0000","last_unstale":"2024-03-09T09:29:14.022970+0000","last_undegraded":"2024-03-09T09:29:14.022970+0000","last_fullsized":"2024-03-09T09:29:14.022970+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2291,"log_dups_size":0,"ondisk_log_size":2291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:40:21.450942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.471373141,"stat_sum":{"num_bytes":5513596,"num_objects":123,"num_object_clones":39,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":39,"num_read":114,"num_read_kb":29387,"num_write":1331,"num_write_kb":57801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'590","reported_seq":757,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.986880+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T09:27:57.986880+0000","last_peered":"2024-03-09T09:27:57.986880+0000","last_clean":"2024-03-09T09:27:57.986880+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T09:27:57.986880+0000","last_undegraded":"2024-03-09T09:27:57.986880+0000","last_fullsized":"2024-03-09T09:27:57.986880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":598,"num_write_kb":111422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"89'729","reported_seq":887,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.544483+0000","last_change":"2024-03-09T08:52:28.583697+0000","last_active":"2024-03-09T09:28:26.544483+0000","last_peered":"2024-03-09T09:28:26.544483+0000","last_clean":"2024-03-09T09:28:26.544483+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T09:28:26.544483+0000","last_undegraded":"2024-03-09T09:28:26.544483+0000","last_fullsized":"2024-03-09T09:28:26.544483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:41.552156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":81,"num_read_kb":0,"num_write":849,"num_write_kb":91941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"89'2528","reported_seq":5332,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.925475+0000","last_change":"2024-03-09T09:27:34.669817+0000","last_active":"2024-03-09T09:29:17.925475+0000","last_peered":"2024-03-09T09:29:17.925475+0000","last_clean":"2024-03-09T09:29:17.925475+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T09:29:17.925475+0000","last_undegraded":"2024-03-09T09:29:17.925475+0000","last_fullsized":"2024-03-09T09:29:17.925475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:42:51.478391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4377723019999999,"stat_sum":{"num_bytes":1160060,"num_objects":111,"num_object_clones":35,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":35,"num_read":113,"num_read_kb":9969,"num_write":1381,"num_write_kb":17733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"89'2165","reported_seq":5161,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.053013+0000","last_change":"2024-03-09T09:27:14.293488+0000","last_active":"2024-03-09T09:29:14.053013+0000","last_peered":"2024-03-09T09:29:14.053013+0000","last_clean":"2024-03-09T09:29:14.053013+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T09:29:14.053013+0000","last_undegraded":"2024-03-09T09:29:14.053013+0000","last_fullsized":"2024-03-09T09:29:14.053013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2165,"log_dups_size":0,"ondisk_log_size":2165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:12:46.882947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.82863014,"stat_sum":{"num_bytes":1951923,"num_objects":107,"num_object_clones":29,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":29,"num_read":96,"num_read_kb":37066,"num_write":1331,"num_write_kb":53456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"89'554","reported_seq":950,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:38.153367+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T09:28:38.153367+0000","last_peered":"2024-03-09T09:28:38.153367+0000","last_clean":"2024-03-09T09:28:38.153367+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T09:28:38.153367+0000","last_undegraded":"2024-03-09T09:28:38.153367+0000","last_fullsized":"2024-03-09T09:28:38.153367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:04:31.731626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007933,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":267,"num_read_kb":985,"num_write":604,"num_write_kb":87856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"89'461","reported_seq":619,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:55.398848+0000","last_change":"2024-03-09T08:52:28.586606+0000","last_active":"2024-03-09T09:27:55.398848+0000","last_peered":"2024-03-09T09:27:55.398848+0000","last_clean":"2024-03-09T09:27:55.398848+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T09:27:55.398848+0000","last_undegraded":"2024-03-09T09:27:55.398848+0000","last_fullsized":"2024-03-09T09:27:55.398848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:17:40.902403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":506,"num_write_kb":99411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"89'2186","reported_seq":5109,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.729816+0000","last_change":"2024-03-09T09:27:30.880846+0000","last_active":"2024-03-09T09:28:56.729816+0000","last_peered":"2024-03-09T09:28:56.729816+0000","last_clean":"2024-03-09T09:28:56.729816+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T09:28:56.729816+0000","last_undegraded":"2024-03-09T09:28:56.729816+0000","last_fullsized":"2024-03-09T09:28:56.729816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2186,"log_dups_size":0,"ondisk_log_size":2186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:47:22.327088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64610520400000004,"stat_sum":{"num_bytes":1318782,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":124,"num_read_kb":9990,"num_write":1296,"num_write_kb":22209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"89'667","reported_seq":829,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:27.475346+0000","last_change":"2024-03-09T08:52:28.586201+0000","last_active":"2024-03-09T09:28:27.475346+0000","last_peered":"2024-03-09T09:28:27.475346+0000","last_clean":"2024-03-09T09:28:27.475346+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T09:28:27.475346+0000","last_undegraded":"2024-03-09T09:28:27.475346+0000","last_fullsized":"2024-03-09T09:28:27.475346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:28:03.471646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":72,"num_read_kb":0,"num_write":777,"num_write_kb":92056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'2210","reported_seq":5186,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.003403+0000","last_change":"2024-03-09T09:27:10.069765+0000","last_active":"2024-03-09T09:29:14.003403+0000","last_peered":"2024-03-09T09:29:14.003403+0000","last_clean":"2024-03-09T09:29:14.003403+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T09:29:14.003403+0000","last_undegraded":"2024-03-09T09:29:14.003403+0000","last_fullsized":"2024-03-09T09:29:14.003403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2210,"log_dups_size":0,"ondisk_log_size":2210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:30:38.949657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.23951623,"stat_sum":{"num_bytes":1295292,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":13266,"num_write":1323,"num_write_kb":26325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"89'861","reported_seq":1028,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.251620+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T09:28:16.251620+0000","last_peered":"2024-03-09T09:28:16.251620+0000","last_clean":"2024-03-09T09:28:16.251620+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T09:28:16.251620+0000","last_undegraded":"2024-03-09T09:28:16.251620+0000","last_fullsized":"2024-03-09T09:28:16.251620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":45,"num_read_kb":0,"num_write":880,"num_write_kb":119947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"89'2142","reported_seq":5080,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.790657+0000","last_change":"2024-03-09T09:27:31.015518+0000","last_active":"2024-03-09T09:28:32.790657+0000","last_peered":"2024-03-09T09:28:32.790657+0000","last_clean":"2024-03-09T09:28:32.790657+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T09:28:32.790657+0000","last_undegraded":"2024-03-09T09:28:32.790657+0000","last_fullsized":"2024-03-09T09:28:32.790657+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2142,"log_dups_size":0,"ondisk_log_size":2142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:20:21.154856+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.70511981300000004,"stat_sum":{"num_bytes":1915718,"num_objects":107,"num_object_clones":37,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":37,"num_read":111,"num_read_kb":24546,"num_write":1285,"num_write_kb":44187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"89'545","reported_seq":706,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:15.180548+0000","last_change":"2024-03-09T08:52:28.585631+0000","last_active":"2024-03-09T09:28:15.180548+0000","last_peered":"2024-03-09T09:28:15.180548+0000","last_clean":"2024-03-09T09:28:15.180548+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T09:28:15.180548+0000","last_undegraded":"2024-03-09T09:28:15.180548+0000","last_fullsized":"2024-03-09T09:28:15.180548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:06:03.569759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":589,"num_write_kb":82799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'2475","reported_seq":5396,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.045469+0000","last_change":"2024-03-09T09:27:12.483134+0000","last_active":"2024-03-09T09:29:14.045469+0000","last_peered":"2024-03-09T09:29:14.045469+0000","last_clean":"2024-03-09T09:29:14.045469+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T09:29:14.045469+0000","last_undegraded":"2024-03-09T09:29:14.045469+0000","last_fullsized":"2024-03-09T09:29:14.045469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2475,"log_dups_size":0,"ondisk_log_size":2475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:53:15.813937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.79565864399999997,"stat_sum":{"num_bytes":2512650,"num_objects":93,"num_object_clones":29,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":29,"num_read":114,"num_read_kb":34867,"num_write":1447,"num_write_kb":63344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'400","reported_seq":566,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:04.123705+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T09:28:04.123705+0000","last_peered":"2024-03-09T09:28:04.123705+0000","last_clean":"2024-03-09T09:28:04.123705+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T09:28:04.123705+0000","last_undegraded":"2024-03-09T09:28:04.123705+0000","last_fullsized":"2024-03-09T09:28:04.123705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":480,"num_write_kb":79281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"89'2172","reported_seq":5110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.758184+0000","last_change":"2024-03-09T09:27:32.094130+0000","last_active":"2024-03-09T09:28:56.758184+0000","last_peered":"2024-03-09T09:28:56.758184+0000","last_clean":"2024-03-09T09:28:56.758184+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T09:28:56.758184+0000","last_undegraded":"2024-03-09T09:28:56.758184+0000","last_fullsized":"2024-03-09T09:28:56.758184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2172,"log_dups_size":0,"ondisk_log_size":2172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:02:28.116630+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.72318536,"stat_sum":{"num_bytes":768948,"num_objects":112,"num_object_clones":37,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":37,"num_read":126,"num_read_kb":26327,"num_write":1312,"num_write_kb":46360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"89'739","reported_seq":905,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.656661+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T09:27:57.656661+0000","last_peered":"2024-03-09T09:27:57.656661+0000","last_clean":"2024-03-09T09:27:57.656661+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T09:27:57.656661+0000","last_undegraded":"2024-03-09T09:27:57.656661+0000","last_fullsized":"2024-03-09T09:27:57.656661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:32:44.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":777,"num_write_kb":95156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"89'2306","reported_seq":5291,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.064156+0000","last_change":"2024-03-09T09:27:09.781774+0000","last_active":"2024-03-09T09:29:14.064156+0000","last_peered":"2024-03-09T09:29:14.064156+0000","last_clean":"2024-03-09T09:29:14.064156+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T09:29:14.064156+0000","last_undegraded":"2024-03-09T09:29:14.064156+0000","last_fullsized":"2024-03-09T09:29:14.064156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2306,"log_dups_size":0,"ondisk_log_size":2306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:44:18.347873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65597337700000002,"stat_sum":{"num_bytes":5623986,"num_objects":128,"num_object_clones":44,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":44,"num_read":148,"num_read_kb":35274,"num_write":1372,"num_write_kb":72006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"89'471","reported_seq":629,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.563819+0000","last_change":"2024-03-09T08:52:28.579315+0000","last_active":"2024-03-09T09:28:26.563819+0000","last_peered":"2024-03-09T09:28:26.563819+0000","last_clean":"2024-03-09T09:28:26.563819+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T09:28:26.563819+0000","last_undegraded":"2024-03-09T09:28:26.563819+0000","last_fullsized":"2024-03-09T09:28:26.563819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:57:17.301677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":514,"num_write_kb":86887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"89'2419","reported_seq":5301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.005816+0000","last_change":"2024-03-09T09:27:34.193425+0000","last_active":"2024-03-09T09:28:31.005816+0000","last_peered":"2024-03-09T09:28:31.005816+0000","last_clean":"2024-03-09T09:28:31.005816+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T09:28:31.005816+0000","last_undegraded":"2024-03-09T09:28:31.005816+0000","last_fullsized":"2024-03-09T09:28:31.005816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2419,"log_dups_size":0,"ondisk_log_size":2419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:25:58.792996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.9237505820000003,"stat_sum":{"num_bytes":5533714,"num_objects":147,"num_object_clones":51,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":51,"num_read":136,"num_read_kb":30612,"num_write":1437,"num_write_kb":51393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"89'616","reported_seq":781,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:13.970713+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T09:29:13.970713+0000","last_peered":"2024-03-09T09:29:13.970713+0000","last_clean":"2024-03-09T09:29:13.970713+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T09:29:13.970713+0000","last_undegraded":"2024-03-09T09:29:13.970713+0000","last_fullsized":"2024-03-09T09:29:13.970713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17329154,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":61,"num_read_kb":0,"num_write":696,"num_write_kb":108351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"89'2233","reported_seq":5212,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.032993+0000","last_change":"2024-03-09T09:27:08.660892+0000","last_active":"2024-03-09T09:29:14.032993+0000","last_peered":"2024-03-09T09:29:14.032993+0000","last_clean":"2024-03-09T09:29:14.032993+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T09:29:14.032993+0000","last_undegraded":"2024-03-09T09:29:14.032993+0000","last_fullsized":"2024-03-09T09:29:14.032993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2233,"log_dups_size":0,"ondisk_log_size":2233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:58:11.216375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.1639723850000001,"stat_sum":{"num_bytes":538194,"num_objects":89,"num_object_clones":30,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":30,"num_read":112,"num_read_kb":30041,"num_write":1325,"num_write_kb":47681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"89'768","reported_seq":925,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:08.255955+0000","last_change":"2024-03-09T08:52:28.585877+0000","last_active":"2024-03-09T09:28:08.255955+0000","last_peered":"2024-03-09T09:28:08.255955+0000","last_clean":"2024-03-09T09:28:08.255955+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T09:28:08.255955+0000","last_undegraded":"2024-03-09T09:28:08.255955+0000","last_fullsized":"2024-03-09T09:28:08.255955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:30:20.036385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":79,"num_read_kb":0,"num_write":862,"num_write_kb":140804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"89'2380","reported_seq":5295,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.944878+0000","last_change":"2024-03-09T09:27:34.560887+0000","last_active":"2024-03-09T09:29:17.944878+0000","last_peered":"2024-03-09T09:29:17.944878+0000","last_clean":"2024-03-09T09:29:17.944878+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T09:29:17.944878+0000","last_undegraded":"2024-03-09T09:29:17.944878+0000","last_fullsized":"2024-03-09T09:29:17.944878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2380,"log_dups_size":0,"ondisk_log_size":2380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:20:20.590247+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2173252530000003,"stat_sum":{"num_bytes":794839,"num_objects":125,"num_object_clones":42,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":42,"num_read":135,"num_read_kb":30019,"num_write":1425,"num_write_kb":47795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"89'2255","reported_seq":5234,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.034255+0000","last_change":"2024-03-09T09:27:07.515170+0000","last_active":"2024-03-09T09:29:14.034255+0000","last_peered":"2024-03-09T09:29:14.034255+0000","last_clean":"2024-03-09T09:29:14.034255+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T09:29:14.034255+0000","last_undegraded":"2024-03-09T09:29:14.034255+0000","last_fullsized":"2024-03-09T09:29:14.034255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2255,"log_dups_size":0,"ondisk_log_size":2255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:08.036556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.8676528270000001,"stat_sum":{"num_bytes":501236,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":17494,"num_write":1356,"num_write_kb":32845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"89'2131","reported_seq":5059,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.745807+0000","last_change":"2024-03-09T09:27:32.084492+0000","last_active":"2024-03-09T09:28:56.745807+0000","last_peered":"2024-03-09T09:28:56.745807+0000","last_clean":"2024-03-09T09:28:56.745807+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T09:28:56.745807+0000","last_undegraded":"2024-03-09T09:28:56.745807+0000","last_fullsized":"2024-03-09T09:28:56.745807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2131,"log_dups_size":0,"ondisk_log_size":2131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:22.480872+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":1.7132004649999999,"stat_sum":{"num_bytes":1622605,"num_objects":113,"num_object_clones":36,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":36,"num_read":103,"num_read_kb":9361,"num_write":1283,"num_write_kb":37703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"89'2356","reported_seq":5312,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.037599+0000","last_change":"2024-03-09T09:27:06.219951+0000","last_active":"2024-03-09T09:29:14.037599+0000","last_peered":"2024-03-09T09:29:14.037599+0000","last_clean":"2024-03-09T09:29:14.037599+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T09:29:14.037599+0000","last_undegraded":"2024-03-09T09:29:14.037599+0000","last_fullsized":"2024-03-09T09:29:14.037599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:15.950822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.73410826500000004,"stat_sum":{"num_bytes":968837,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":110,"num_read_kb":13537,"num_write":1403,"num_write_kb":47183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"89'2251","reported_seq":5137,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.925278+0000","last_change":"2024-03-09T09:27:34.720190+0000","last_active":"2024-03-09T09:29:17.925278+0000","last_peered":"2024-03-09T09:29:17.925278+0000","last_clean":"2024-03-09T09:29:17.925278+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T09:29:17.925278+0000","last_undegraded":"2024-03-09T09:29:17.925278+0000","last_fullsized":"2024-03-09T09:29:17.925278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2251,"log_dups_size":0,"ondisk_log_size":2251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:58:23.785495+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.446018724,"stat_sum":{"num_bytes":5408806,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":112,"num_read_kb":29960,"num_write":1313,"num_write_kb":48116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"89'2237","reported_seq":5234,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048940+0000","last_change":"2024-03-09T09:27:14.175435+0000","last_active":"2024-03-09T09:29:14.048940+0000","last_peered":"2024-03-09T09:29:14.048940+0000","last_clean":"2024-03-09T09:29:14.048940+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T09:29:14.048940+0000","last_undegraded":"2024-03-09T09:29:14.048940+0000","last_fullsized":"2024-03-09T09:29:14.048940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2237,"log_dups_size":0,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:29.862968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.706678991,"stat_sum":{"num_bytes":887348,"num_objects":110,"num_object_clones":37,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":37,"num_read":123,"num_read_kb":42348,"num_write":1334,"num_write_kb":66020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"89'2116","reported_seq":5064,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.060507+0000","last_change":"2024-03-09T09:27:32.079328+0000","last_active":"2024-03-09T09:28:31.060507+0000","last_peered":"2024-03-09T09:28:31.060507+0000","last_clean":"2024-03-09T09:28:31.060507+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T09:28:31.060507+0000","last_undegraded":"2024-03-09T09:28:31.060507+0000","last_fullsized":"2024-03-09T09:28:31.060507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2116,"log_dups_size":0,"ondisk_log_size":2116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:18.607812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.708282305,"stat_sum":{"num_bytes":596753,"num_objects":112,"num_object_clones":36,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":36,"num_read":111,"num_read_kb":13558,"num_write":1290,"num_write_kb":25078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"89'2395","reported_seq":5315,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.036196+0000","last_change":"2024-03-09T09:27:09.523471+0000","last_active":"2024-03-09T09:29:14.036196+0000","last_peered":"2024-03-09T09:29:14.036196+0000","last_clean":"2024-03-09T09:29:14.036196+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T09:29:14.036196+0000","last_undegraded":"2024-03-09T09:29:14.036196+0000","last_fullsized":"2024-03-09T09:29:14.036196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2395,"log_dups_size":0,"ondisk_log_size":2395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:57:42.927188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.39778581299999999,"stat_sum":{"num_bytes":1231522,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":112,"num_read_kb":16729,"num_write":1390,"num_write_kb":42674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"89'2182","reported_seq":5127,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.920533+0000","last_change":"2024-03-09T09:27:34.631961+0000","last_active":"2024-03-09T09:29:17.920533+0000","last_peered":"2024-03-09T09:29:17.920533+0000","last_clean":"2024-03-09T09:29:17.920533+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T09:29:17.920533+0000","last_undegraded":"2024-03-09T09:29:17.920533+0000","last_fullsized":"2024-03-09T09:29:17.920533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2182,"log_dups_size":0,"ondisk_log_size":2182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:11:08.985120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4007626340000003,"stat_sum":{"num_bytes":767001,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":115,"num_read_kb":17556,"num_write":1334,"num_write_kb":40745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"89'2227","reported_seq":5191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.070029+0000","last_change":"2024-03-09T09:27:12.489198+0000","last_active":"2024-03-09T09:29:14.070029+0000","last_peered":"2024-03-09T09:29:14.070029+0000","last_clean":"2024-03-09T09:29:14.070029+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T09:29:14.070029+0000","last_undegraded":"2024-03-09T09:29:14.070029+0000","last_fullsized":"2024-03-09T09:29:14.070029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2227,"log_dups_size":0,"ondisk_log_size":2227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:48:35.331040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.80249947799999999,"stat_sum":{"num_bytes":1587591,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":108,"num_read_kb":24972,"num_write":1321,"num_write_kb":50275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"89'2414","reported_seq":5260,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.985366+0000","last_change":"2024-03-09T09:27:33.932429+0000","last_active":"2024-03-09T09:28:30.985366+0000","last_peered":"2024-03-09T09:28:30.985366+0000","last_clean":"2024-03-09T09:28:30.985366+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T09:28:30.985366+0000","last_undegraded":"2024-03-09T09:28:30.985366+0000","last_fullsized":"2024-03-09T09:28:30.985366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2414,"log_dups_size":0,"ondisk_log_size":2414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:33:24.675125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.6671456120000001,"stat_sum":{"num_bytes":827266,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":108,"num_read_kb":21834,"num_write":1396,"num_write_kb":37656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"89'2190","reported_seq":5177,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.014853+0000","last_change":"2024-03-09T09:27:10.243190+0000","last_active":"2024-03-09T09:29:14.014853+0000","last_peered":"2024-03-09T09:29:14.014853+0000","last_clean":"2024-03-09T09:29:14.014853+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T09:29:14.014853+0000","last_undegraded":"2024-03-09T09:29:14.014853+0000","last_fullsized":"2024-03-09T09:29:14.014853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2190,"log_dups_size":0,"ondisk_log_size":2190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:08.187277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.410735039,"stat_sum":{"num_bytes":1374461,"num_objects":110,"num_object_clones":35,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":35,"num_read":104,"num_read_kb":29127,"num_write":1325,"num_write_kb":54062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"89'2222","reported_seq":5150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.925368+0000","last_change":"2024-03-09T09:27:34.622798+0000","last_active":"2024-03-09T09:29:17.925368+0000","last_peered":"2024-03-09T09:29:17.925368+0000","last_clean":"2024-03-09T09:29:17.925368+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T09:29:17.925368+0000","last_undegraded":"2024-03-09T09:29:17.925368+0000","last_fullsized":"2024-03-09T09:29:17.925368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2222,"log_dups_size":0,"ondisk_log_size":2222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:16:59.598968+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.393423125,"stat_sum":{"num_bytes":646888,"num_objects":117,"num_object_clones":37,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":37,"num_read":111,"num_read_kb":21616,"num_write":1343,"num_write_kb":41561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"89'2380","reported_seq":5353,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.053034+0000","last_change":"2024-03-09T09:27:08.090545+0000","last_active":"2024-03-09T09:29:14.053034+0000","last_peered":"2024-03-09T09:29:14.053034+0000","last_clean":"2024-03-09T09:29:14.053034+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T09:29:14.053034+0000","last_undegraded":"2024-03-09T09:29:14.053034+0000","last_fullsized":"2024-03-09T09:29:14.053034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2380,"log_dups_size":0,"ondisk_log_size":2380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:38:53.276797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.440348755,"stat_sum":{"num_bytes":1448398,"num_objects":130,"num_object_clones":44,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":44,"num_read":125,"num_read_kb":18804,"num_write":1435,"num_write_kb":32798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"89'2303","reported_seq":5205,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.859353+0000","last_change":"2024-03-09T09:27:30.874126+0000","last_active":"2024-03-09T09:28:30.859353+0000","last_peered":"2024-03-09T09:28:30.859353+0000","last_clean":"2024-03-09T09:28:30.859353+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T09:28:30.859353+0000","last_undegraded":"2024-03-09T09:28:30.859353+0000","last_fullsized":"2024-03-09T09:28:30.859353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2303,"log_dups_size":0,"ondisk_log_size":2303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:12:39.325227+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.64500296700000004,"stat_sum":{"num_bytes":1066603,"num_objects":98,"num_object_clones":32,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":32,"num_read":104,"num_read_kb":18198,"num_write":1399,"num_write_kb":47978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"89'2175","reported_seq":5136,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.046381+0000","last_change":"2024-03-09T09:27:12.333957+0000","last_active":"2024-03-09T09:29:14.046381+0000","last_peered":"2024-03-09T09:29:14.046381+0000","last_clean":"2024-03-09T09:29:14.046381+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T09:29:14.046381+0000","last_undegraded":"2024-03-09T09:29:14.046381+0000","last_fullsized":"2024-03-09T09:29:14.046381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:59:27.720581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.62298635300000005,"stat_sum":{"num_bytes":1371558,"num_objects":107,"num_object_clones":34,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":34,"num_read":92,"num_read_kb":30679,"num_write":1263,"num_write_kb":63255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"89'2203","reported_seq":5097,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.761136+0000","last_change":"2024-03-09T09:27:34.527780+0000","last_active":"2024-03-09T09:28:56.761136+0000","last_peered":"2024-03-09T09:28:56.761136+0000","last_clean":"2024-03-09T09:28:56.761136+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T09:28:56.761136+0000","last_undegraded":"2024-03-09T09:28:56.761136+0000","last_fullsized":"2024-03-09T09:28:56.761136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2203,"log_dups_size":0,"ondisk_log_size":2203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:33:42.613616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.1845584630000001,"stat_sum":{"num_bytes":1760124,"num_objects":108,"num_object_clones":35,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":35,"num_read":107,"num_read_kb":13622,"num_write":1279,"num_write_kb":27255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'2199","reported_seq":5181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.036836+0000","last_change":"2024-03-09T09:27:13.741409+0000","last_active":"2024-03-09T09:29:14.036836+0000","last_peered":"2024-03-09T09:29:14.036836+0000","last_clean":"2024-03-09T09:29:14.036836+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T09:29:14.036836+0000","last_undegraded":"2024-03-09T09:29:14.036836+0000","last_fullsized":"2024-03-09T09:29:14.036836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2199,"log_dups_size":0,"ondisk_log_size":2199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:03.762335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.2778432559999999,"stat_sum":{"num_bytes":1308027,"num_objects":118,"num_object_clones":42,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":42,"num_read":112,"num_read_kb":14330,"num_write":1310,"num_write_kb":25655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"89'2329","reported_seq":5225,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.875787+0000","last_change":"2024-03-09T09:27:34.655594+0000","last_active":"2024-03-09T09:29:17.875787+0000","last_peered":"2024-03-09T09:29:17.875787+0000","last_clean":"2024-03-09T09:29:17.875787+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T09:29:17.875787+0000","last_undegraded":"2024-03-09T09:29:17.875787+0000","last_fullsized":"2024-03-09T09:29:17.875787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2329,"log_dups_size":0,"ondisk_log_size":2329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:01:41.618756+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4171876489999997,"stat_sum":{"num_bytes":1480146,"num_objects":115,"num_object_clones":41,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":41,"num_read":120,"num_read_kb":31406,"num_write":1383,"num_write_kb":52041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"89'2175","reported_seq":5173,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.056157+0000","last_change":"2024-03-09T09:27:12.405710+0000","last_active":"2024-03-09T09:29:14.056157+0000","last_peered":"2024-03-09T09:29:14.056157+0000","last_clean":"2024-03-09T09:29:14.056157+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T09:29:14.056157+0000","last_undegraded":"2024-03-09T09:29:14.056157+0000","last_fullsized":"2024-03-09T09:29:14.056157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:12:10.053606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.72006586699999997,"stat_sum":{"num_bytes":650831,"num_objects":114,"num_object_clones":39,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":39,"num_read":114,"num_read_kb":46474,"num_write":1326,"num_write_kb":53541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"89'2284","reported_seq":5214,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.304433+0000","last_change":"2024-03-09T09:27:30.902599+0000","last_active":"2024-03-09T09:28:32.304433+0000","last_peered":"2024-03-09T09:28:32.304433+0000","last_clean":"2024-03-09T09:28:32.304433+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T09:28:32.304433+0000","last_undegraded":"2024-03-09T09:28:32.304433+0000","last_fullsized":"2024-03-09T09:28:32.304433+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:54:27.990313+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.64248945300000004,"stat_sum":{"num_bytes":509009,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":145,"num_read_kb":34396,"num_write":1348,"num_write_kb":44845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"89'2311","reported_seq":5257,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.050683+0000","last_change":"2024-03-09T09:27:13.760270+0000","last_active":"2024-03-09T09:29:14.050683+0000","last_peered":"2024-03-09T09:29:14.050683+0000","last_clean":"2024-03-09T09:29:14.050683+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T09:29:14.050683+0000","last_undegraded":"2024-03-09T09:29:14.050683+0000","last_fullsized":"2024-03-09T09:29:14.050683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2311,"log_dups_size":0,"ondisk_log_size":2311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:54:17.411237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.291579509,"stat_sum":{"num_bytes":917223,"num_objects":100,"num_object_clones":30,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":30,"num_read":95,"num_read_kb":13353,"num_write":1387,"num_write_kb":42086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"89'2321","reported_seq":5211,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.932450+0000","last_change":"2024-03-09T09:27:34.549097+0000","last_active":"2024-03-09T09:29:17.932450+0000","last_peered":"2024-03-09T09:29:17.932450+0000","last_clean":"2024-03-09T09:29:17.932450+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T09:29:17.932450+0000","last_undegraded":"2024-03-09T09:29:17.932450+0000","last_fullsized":"2024-03-09T09:29:17.932450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2321,"log_dups_size":0,"ondisk_log_size":2321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:56:26.561322+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.2066621079999997,"stat_sum":{"num_bytes":1356079,"num_objects":97,"num_object_clones":28,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":28,"num_read":106,"num_read_kb":29896,"num_write":1386,"num_write_kb":51017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"89'2224","reported_seq":5214,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.042583+0000","last_change":"2024-03-09T09:27:09.439110+0000","last_active":"2024-03-09T09:29:14.042583+0000","last_peered":"2024-03-09T09:29:14.042583+0000","last_clean":"2024-03-09T09:29:14.042583+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T09:29:14.042583+0000","last_undegraded":"2024-03-09T09:29:14.042583+0000","last_fullsized":"2024-03-09T09:29:14.042583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2224,"log_dups_size":0,"ondisk_log_size":2224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:14:38.198734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.313168905,"stat_sum":{"num_bytes":1092268,"num_objects":121,"num_object_clones":43,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":43,"num_read":123,"num_read_kb":25979,"num_write":1341,"num_write_kb":58516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"89'2297","reported_seq":5196,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.750823+0000","last_change":"2024-03-09T09:27:34.188251+0000","last_active":"2024-03-09T09:28:56.750823+0000","last_peered":"2024-03-09T09:28:56.750823+0000","last_clean":"2024-03-09T09:28:56.750823+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T09:28:56.750823+0000","last_undegraded":"2024-03-09T09:28:56.750823+0000","last_fullsized":"2024-03-09T09:28:56.750823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2297,"log_dups_size":0,"ondisk_log_size":2297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:07:37.642402+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":3.9232747450000001,"stat_sum":{"num_bytes":5183706,"num_objects":95,"num_object_clones":27,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":111,"num_read_kb":34166,"num_write":1371,"num_write_kb":49768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"89'2247","reported_seq":5207,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.063774+0000","last_change":"2024-03-09T09:27:07.884719+0000","last_active":"2024-03-09T09:29:14.063774+0000","last_peered":"2024-03-09T09:29:14.063774+0000","last_clean":"2024-03-09T09:29:14.063774+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T09:29:14.063774+0000","last_undegraded":"2024-03-09T09:29:14.063774+0000","last_fullsized":"2024-03-09T09:29:14.063774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2247,"log_dups_size":0,"ondisk_log_size":2247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:38.238598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.1326348209999999,"stat_sum":{"num_bytes":1866711,"num_objects":117,"num_object_clones":40,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":40,"num_read":109,"num_read_kb":17692,"num_write":1315,"num_write_kb":41853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"89'2193","reported_seq":5129,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.657070+0000","last_change":"2024-03-09T09:27:33.862027+0000","last_active":"2024-03-09T09:28:32.657070+0000","last_peered":"2024-03-09T09:28:32.657070+0000","last_clean":"2024-03-09T09:28:32.657070+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T09:28:32.657070+0000","last_undegraded":"2024-03-09T09:28:32.657070+0000","last_fullsized":"2024-03-09T09:28:32.657070+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2193,"log_dups_size":0,"ondisk_log_size":2193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:24:42.648859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.5970959150000001,"stat_sum":{"num_bytes":1348477,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":126,"num_read_kb":8825,"num_write":1341,"num_write_kb":33612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"89'2672","reported_seq":5494,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.040015+0000","last_change":"2024-03-09T09:27:13.243644+0000","last_active":"2024-03-09T09:29:14.040015+0000","last_peered":"2024-03-09T09:29:14.040015+0000","last_clean":"2024-03-09T09:29:14.040015+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T09:29:14.040015+0000","last_undegraded":"2024-03-09T09:29:14.040015+0000","last_fullsized":"2024-03-09T09:29:14.040015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2672,"log_dups_size":0,"ondisk_log_size":2672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:48.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.49950316500000003,"stat_sum":{"num_bytes":924300,"num_objects":92,"num_object_clones":25,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":25,"num_read":107,"num_read_kb":17449,"num_write":1480,"num_write_kb":44901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"89'2175","reported_seq":5120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.873811+0000","last_change":"2024-03-09T09:27:34.679372+0000","last_active":"2024-03-09T09:29:17.873811+0000","last_peered":"2024-03-09T09:29:17.873811+0000","last_clean":"2024-03-09T09:29:17.873811+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T09:29:17.873811+0000","last_undegraded":"2024-03-09T09:29:17.873811+0000","last_fullsized":"2024-03-09T09:29:17.873811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:56.141041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4099735320000004,"stat_sum":{"num_bytes":773517,"num_objects":127,"num_object_clones":46,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":46,"num_read":120,"num_read_kb":5950,"num_write":1331,"num_write_kb":13298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"89'2162","reported_seq":5147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.536262+0000","last_change":"2024-03-09T09:27:06.385525+0000","last_active":"2024-03-09T09:28:23.536262+0000","last_peered":"2024-03-09T09:28:23.536262+0000","last_clean":"2024-03-09T09:28:23.536262+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T09:28:23.536262+0000","last_undegraded":"2024-03-09T09:28:23.536262+0000","last_fullsized":"2024-03-09T09:28:23.536262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2162,"log_dups_size":0,"ondisk_log_size":2162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:33:27.669672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.89838860799999998,"stat_sum":{"num_bytes":6604987,"num_objects":101,"num_object_clones":30,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":30,"num_read":109,"num_read_kb":14504,"num_write":1281,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"89'2152","reported_seq":5182,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.885774+0000","last_change":"2024-03-09T09:27:31.020668+0000","last_active":"2024-03-09T09:28:30.885774+0000","last_peered":"2024-03-09T09:28:30.885774+0000","last_clean":"2024-03-09T09:28:30.885774+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T09:28:30.885774+0000","last_undegraded":"2024-03-09T09:28:30.885774+0000","last_fullsized":"2024-03-09T09:28:30.885774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2152,"log_dups_size":0,"ondisk_log_size":2152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:50:23.678039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.71096335899999996,"stat_sum":{"num_bytes":637081,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":204,"num_read_kb":24036,"num_write":1319,"num_write_kb":30441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"89'2336","reported_seq":5279,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.555277+0000","last_change":"2024-03-09T09:27:06.358258+0000","last_active":"2024-03-09T09:28:23.555277+0000","last_peered":"2024-03-09T09:28:23.555277+0000","last_clean":"2024-03-09T09:28:23.555277+0000","last_became_active":"2024-03-09T08:53:00.240319+0000","last_became_peered":"2024-03-09T08:53:00.240319+0000","last_unstale":"2024-03-09T09:28:23.555277+0000","last_undegraded":"2024-03-09T09:28:23.555277+0000","last_fullsized":"2024-03-09T09:28:23.555277+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2336,"log_dups_size":0,"ondisk_log_size":2336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:31:17.488290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.86880120699999996,"stat_sum":{"num_bytes":1614659,"num_objects":116,"num_object_clones":42,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":42,"num_read":120,"num_read_kb":18915,"num_write":1382,"num_write_kb":39172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"89'2125","reported_seq":5073,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.759491+0000","last_change":"2024-03-09T09:27:32.089100+0000","last_active":"2024-03-09T09:28:56.759491+0000","last_peered":"2024-03-09T09:28:56.759491+0000","last_clean":"2024-03-09T09:28:56.759491+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T09:28:56.759491+0000","last_undegraded":"2024-03-09T09:28:56.759491+0000","last_fullsized":"2024-03-09T09:28:56.759491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2125,"log_dups_size":0,"ondisk_log_size":2125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:10:42.901325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.7177660380000002,"stat_sum":{"num_bytes":924110,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":129,"num_read_kb":10365,"num_write":1260,"num_write_kb":33897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"89'2227","reported_seq":5204,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.029490+0000","last_change":"2024-03-09T09:27:13.674044+0000","last_active":"2024-03-09T09:29:14.029490+0000","last_peered":"2024-03-09T09:29:14.029490+0000","last_clean":"2024-03-09T09:29:14.029490+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T09:29:14.029490+0000","last_undegraded":"2024-03-09T09:29:14.029490+0000","last_fullsized":"2024-03-09T09:29:14.029490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2227,"log_dups_size":0,"ondisk_log_size":2227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:32.494482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.2105350770000001,"stat_sum":{"num_bytes":1326786,"num_objects":103,"num_object_clones":34,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":34,"num_read":111,"num_read_kb":18735,"num_write":1322,"num_write_kb":41042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"89'2350","reported_seq":5252,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:52.661098+0000","last_change":"2024-03-09T09:27:34.276120+0000","last_active":"2024-03-09T09:28:52.661098+0000","last_peered":"2024-03-09T09:28:52.661098+0000","last_clean":"2024-03-09T09:28:52.661098+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T09:28:52.661098+0000","last_undegraded":"2024-03-09T09:28:52.661098+0000","last_fullsized":"2024-03-09T09:28:52.661098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2350,"log_dups_size":0,"ondisk_log_size":2350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:34:48.217825+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.0059579840000001,"stat_sum":{"num_bytes":936699,"num_objects":111,"num_object_clones":39,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":39,"num_read":129,"num_read_kb":34540,"num_write":1403,"num_write_kb":57337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"89'2277","reported_seq":5248,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048137+0000","last_change":"2024-03-09T09:27:09.746181+0000","last_active":"2024-03-09T09:29:14.048137+0000","last_peered":"2024-03-09T09:29:14.048137+0000","last_clean":"2024-03-09T09:29:14.048137+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T09:29:14.048137+0000","last_undegraded":"2024-03-09T09:29:14.048137+0000","last_fullsized":"2024-03-09T09:29:14.048137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2277,"log_dups_size":0,"ondisk_log_size":2277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:47:00.612196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.91582108900000003,"stat_sum":{"num_bytes":1759504,"num_objects":101,"num_object_clones":31,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":31,"num_read":108,"num_read_kb":5845,"num_write":1367,"num_write_kb":19152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"89'2349","reported_seq":5233,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.940855+0000","last_change":"2024-03-09T09:27:34.759707+0000","last_active":"2024-03-09T09:29:17.940855+0000","last_peered":"2024-03-09T09:29:17.940855+0000","last_clean":"2024-03-09T09:29:17.940855+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T09:29:17.940855+0000","last_undegraded":"2024-03-09T09:29:17.940855+0000","last_fullsized":"2024-03-09T09:29:17.940855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2349,"log_dups_size":0,"ondisk_log_size":2349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:27:24.016743+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.4857478520000003,"stat_sum":{"num_bytes":2540930,"num_objects":122,"num_object_clones":40,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":40,"num_read":127,"num_read_kb":51469,"num_write":1352,"num_write_kb":80787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"89'2218","reported_seq":5194,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.022677+0000","last_change":"2024-03-09T09:27:09.024910+0000","last_active":"2024-03-09T09:29:14.022677+0000","last_peered":"2024-03-09T09:29:14.022677+0000","last_clean":"2024-03-09T09:29:14.022677+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T09:29:14.022677+0000","last_undegraded":"2024-03-09T09:29:14.022677+0000","last_fullsized":"2024-03-09T09:29:14.022677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2218,"log_dups_size":0,"ondisk_log_size":2218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:02:10.004295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.1939797309999998,"stat_sum":{"num_bytes":1585107,"num_objects":96,"num_object_clones":31,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":31,"num_read":96,"num_read_kb":18307,"num_write":1352,"num_write_kb":50489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"89'2682","reported_seq":5434,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.949755+0000","last_change":"2024-03-09T09:27:34.659119+0000","last_active":"2024-03-09T09:29:17.949755+0000","last_peered":"2024-03-09T09:29:17.949755+0000","last_clean":"2024-03-09T09:29:17.949755+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T09:29:17.949755+0000","last_undegraded":"2024-03-09T09:29:17.949755+0000","last_fullsized":"2024-03-09T09:29:17.949755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2682,"log_dups_size":0,"ondisk_log_size":2682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:45:21.245870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4197025170000002,"stat_sum":{"num_bytes":1559225,"num_objects":122,"num_object_clones":42,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":42,"num_read":122,"num_read_kb":6495,"num_write":1460,"num_write_kb":38738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"89'2225","reported_seq":5197,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.005606+0000","last_change":"2024-03-09T09:27:12.627461+0000","last_active":"2024-03-09T09:29:14.005606+0000","last_peered":"2024-03-09T09:29:14.005606+0000","last_clean":"2024-03-09T09:29:14.005606+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T09:29:14.005606+0000","last_undegraded":"2024-03-09T09:29:14.005606+0000","last_fullsized":"2024-03-09T09:29:14.005606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2225,"log_dups_size":0,"ondisk_log_size":2225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:03:50.159643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.94133736700000004,"stat_sum":{"num_bytes":314683,"num_objects":117,"num_object_clones":39,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":39,"num_read":116,"num_read_kb":13620,"num_write":1329,"num_write_kb":42753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"89'2198","reported_seq":5208,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.950711+0000","last_change":"2024-03-09T09:27:34.517246+0000","last_active":"2024-03-09T09:29:17.950711+0000","last_peered":"2024-03-09T09:29:17.950711+0000","last_clean":"2024-03-09T09:29:17.950711+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T09:29:17.950711+0000","last_undegraded":"2024-03-09T09:29:17.950711+0000","last_fullsized":"2024-03-09T09:29:17.950711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2198,"log_dups_size":0,"ondisk_log_size":2198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:23:51.461775+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.1785381859999999,"stat_sum":{"num_bytes":1532500,"num_objects":95,"num_object_clones":30,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":30,"num_read":195,"num_read_kb":18044,"num_write":1327,"num_write_kb":39414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"89'2266","reported_seq":5235,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.043614+0000","last_change":"2024-03-09T09:27:08.772433+0000","last_active":"2024-03-09T09:29:14.043614+0000","last_peered":"2024-03-09T09:29:14.043614+0000","last_clean":"2024-03-09T09:29:14.043614+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T09:29:14.043614+0000","last_undegraded":"2024-03-09T09:29:14.043614+0000","last_fullsized":"2024-03-09T09:29:14.043614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2266,"log_dups_size":0,"ondisk_log_size":2266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:09:40.668035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.298540461,"stat_sum":{"num_bytes":2039183,"num_objects":130,"num_object_clones":43,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":43,"num_read":126,"num_read_kb":25377,"num_write":1340,"num_write_kb":50110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"89'2204","reported_seq":5123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:47.307121+0000","last_change":"2024-03-09T09:27:30.886579+0000","last_active":"2024-03-09T09:28:47.307121+0000","last_peered":"2024-03-09T09:28:47.307121+0000","last_clean":"2024-03-09T09:28:47.307121+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T09:28:47.307121+0000","last_undegraded":"2024-03-09T09:28:47.307121+0000","last_fullsized":"2024-03-09T09:28:47.307121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2204,"log_dups_size":0,"ondisk_log_size":2204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:31:23.131967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.62967200199999995,"stat_sum":{"num_bytes":347019,"num_objects":120,"num_object_clones":41,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":41,"num_read":119,"num_read_kb":17844,"num_write":1302,"num_write_kb":51829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"89'2658","reported_seq":5496,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.057096+0000","last_change":"2024-03-09T09:27:07.715360+0000","last_active":"2024-03-09T09:29:14.057096+0000","last_peered":"2024-03-09T09:29:14.057096+0000","last_clean":"2024-03-09T09:29:14.057096+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T09:29:14.057096+0000","last_undegraded":"2024-03-09T09:29:14.057096+0000","last_fullsized":"2024-03-09T09:29:14.057096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2658,"log_dups_size":0,"ondisk_log_size":2658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:37:57.318273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.0660642299999998,"stat_sum":{"num_bytes":1365000,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":127,"num_read_kb":33469,"num_write":1475,"num_write_kb":57156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"89'2231","reported_seq":5157,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.942922+0000","last_change":"2024-03-09T09:27:34.649642+0000","last_active":"2024-03-09T09:29:17.942922+0000","last_peered":"2024-03-09T09:29:17.942922+0000","last_clean":"2024-03-09T09:29:17.942922+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T09:29:17.942922+0000","last_undegraded":"2024-03-09T09:29:17.942922+0000","last_fullsized":"2024-03-09T09:29:17.942922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2231,"log_dups_size":0,"ondisk_log_size":2231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:53.123132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4162208879999998,"stat_sum":{"num_bytes":1481141,"num_objects":109,"num_object_clones":37,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":37,"num_read":120,"num_read_kb":25693,"num_write":1333,"num_write_kb":50297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'2192","reported_seq":5171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.049795+0000","last_change":"2024-03-09T09:27:08.029887+0000","last_active":"2024-03-09T09:29:14.049795+0000","last_peered":"2024-03-09T09:29:14.049795+0000","last_clean":"2024-03-09T09:29:14.049795+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T09:29:14.049795+0000","last_undegraded":"2024-03-09T09:29:14.049795+0000","last_fullsized":"2024-03-09T09:29:14.049795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2192,"log_dups_size":0,"ondisk_log_size":2192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:57:17.824590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.379810049,"stat_sum":{"num_bytes":472147,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":115,"num_read_kb":25809,"num_write":1277,"num_write_kb":34061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"89'2310","reported_seq":5232,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924102+0000","last_change":"2024-03-09T09:27:34.694598+0000","last_active":"2024-03-09T09:29:17.924102+0000","last_peered":"2024-03-09T09:29:17.924102+0000","last_clean":"2024-03-09T09:29:17.924102+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T09:29:17.924102+0000","last_undegraded":"2024-03-09T09:29:17.924102+0000","last_fullsized":"2024-03-09T09:29:17.924102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2310,"log_dups_size":0,"ondisk_log_size":2310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:24:52.867402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4238349550000002,"stat_sum":{"num_bytes":1560364,"num_objects":99,"num_object_clones":31,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":31,"num_read":120,"num_read_kb":23106,"num_write":1406,"num_write_kb":36148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"89'2355","reported_seq":5316,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.061430+0000","last_change":"2024-03-09T09:27:13.474431+0000","last_active":"2024-03-09T09:29:14.061430+0000","last_peered":"2024-03-09T09:29:14.061430+0000","last_clean":"2024-03-09T09:29:14.061430+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T09:29:14.061430+0000","last_undegraded":"2024-03-09T09:29:14.061430+0000","last_fullsized":"2024-03-09T09:29:14.061430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2355,"log_dups_size":0,"ondisk_log_size":2355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:53.961909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.73072901000000001,"stat_sum":{"num_bytes":1439516,"num_objects":106,"num_object_clones":37,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":37,"num_read":117,"num_read_kb":43194,"num_write":1390,"num_write_kb":63251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"89'2112","reported_seq":5035,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.954079+0000","last_change":"2024-03-09T09:27:34.559512+0000","last_active":"2024-03-09T09:29:17.954079+0000","last_peered":"2024-03-09T09:29:17.954079+0000","last_clean":"2024-03-09T09:29:17.954079+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T09:29:17.954079+0000","last_undegraded":"2024-03-09T09:29:17.954079+0000","last_fullsized":"2024-03-09T09:29:17.954079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2112,"log_dups_size":0,"ondisk_log_size":2112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:12:19.227405+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2163343510000004,"stat_sum":{"num_bytes":526581,"num_objects":91,"num_object_clones":29,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":29,"num_read":100,"num_read_kb":25429,"num_write":1233,"num_write_kb":64615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"89'2159","reported_seq":5133,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.052874+0000","last_change":"2024-03-09T09:27:12.157156+0000","last_active":"2024-03-09T09:29:14.052874+0000","last_peered":"2024-03-09T09:29:14.052874+0000","last_clean":"2024-03-09T09:29:14.052874+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T09:29:14.052874+0000","last_undegraded":"2024-03-09T09:29:14.052874+0000","last_fullsized":"2024-03-09T09:29:14.052874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2159,"log_dups_size":0,"ondisk_log_size":2159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:45:37.914970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.46979842799999999,"stat_sum":{"num_bytes":631595,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":98,"num_read_kb":25968,"num_write":1286,"num_write_kb":41427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"89'2132","reported_seq":5071,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:30.889842+0000","last_change":"2024-03-09T09:27:30.899203+0000","last_active":"2024-03-09T09:28:30.889842+0000","last_peered":"2024-03-09T09:28:30.889842+0000","last_clean":"2024-03-09T09:28:30.889842+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T09:28:30.889842+0000","last_undegraded":"2024-03-09T09:28:30.889842+0000","last_fullsized":"2024-03-09T09:28:30.889842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2132,"log_dups_size":0,"ondisk_log_size":2132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:29:36.464781+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.66506321800000001,"stat_sum":{"num_bytes":592790,"num_objects":103,"num_object_clones":35,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":35,"num_read":105,"num_read_kb":21771,"num_write":1301,"num_write_kb":41126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"89'2204","reported_seq":5160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048992+0000","last_change":"2024-03-09T09:27:13.380861+0000","last_active":"2024-03-09T09:29:14.048992+0000","last_peered":"2024-03-09T09:29:14.048992+0000","last_clean":"2024-03-09T09:29:14.048992+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T09:29:14.048992+0000","last_undegraded":"2024-03-09T09:29:14.048992+0000","last_fullsized":"2024-03-09T09:29:14.048992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2204,"log_dups_size":0,"ondisk_log_size":2204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:02:11.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.638707739,"stat_sum":{"num_bytes":1008318,"num_objects":81,"num_object_clones":24,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":24,"num_read":86,"num_read_kb":17604,"num_write":1317,"num_write_kb":38401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"89'2293","reported_seq":5184,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.754410+0000","last_change":"2024-03-09T09:27:32.075213+0000","last_active":"2024-03-09T09:28:56.754410+0000","last_peered":"2024-03-09T09:28:56.754410+0000","last_clean":"2024-03-09T09:28:56.754410+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T09:28:56.754410+0000","last_undegraded":"2024-03-09T09:28:56.754410+0000","last_fullsized":"2024-03-09T09:28:56.754410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2293,"log_dups_size":0,"ondisk_log_size":2293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:17:17.703718+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.705573912,"stat_sum":{"num_bytes":534756,"num_objects":106,"num_object_clones":34,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":34,"num_read":108,"num_read_kb":25061,"num_write":1338,"num_write_kb":44408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"89'2111","reported_seq":5114,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.032543+0000","last_change":"2024-03-09T09:27:13.610566+0000","last_active":"2024-03-09T09:29:14.032543+0000","last_peered":"2024-03-09T09:29:14.032543+0000","last_clean":"2024-03-09T09:29:14.032543+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T09:29:14.032543+0000","last_undegraded":"2024-03-09T09:29:14.032543+0000","last_fullsized":"2024-03-09T09:29:14.032543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2111,"log_dups_size":0,"ondisk_log_size":2111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:17:33.229502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.1420017650000001,"stat_sum":{"num_bytes":1651219,"num_objects":118,"num_object_clones":44,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":44,"num_read":114,"num_read_kb":27021,"num_write":1266,"num_write_kb":36457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"89'2279","reported_seq":5203,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924096+0000","last_change":"2024-03-09T09:27:34.669037+0000","last_active":"2024-03-09T09:29:17.924096+0000","last_peered":"2024-03-09T09:29:17.924096+0000","last_clean":"2024-03-09T09:29:17.924096+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T09:29:17.924096+0000","last_undegraded":"2024-03-09T09:29:17.924096+0000","last_fullsized":"2024-03-09T09:29:17.924096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2279,"log_dups_size":0,"ondisk_log_size":2279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:54:35.897455+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.3954307559999997,"stat_sum":{"num_bytes":1015705,"num_objects":121,"num_object_clones":38,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":38,"num_read":132,"num_read_kb":25133,"num_write":1351,"num_write_kb":59371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"89'2361","reported_seq":5314,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.067546+0000","last_change":"2024-03-09T09:27:13.261980+0000","last_active":"2024-03-09T09:29:14.067546+0000","last_peered":"2024-03-09T09:29:14.067546+0000","last_clean":"2024-03-09T09:29:14.067546+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T09:29:14.067546+0000","last_undegraded":"2024-03-09T09:29:14.067546+0000","last_fullsized":"2024-03-09T09:29:14.067546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2361,"log_dups_size":0,"ondisk_log_size":2361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:13:56.252007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.51876684699999998,"stat_sum":{"num_bytes":1407157,"num_objects":117,"num_object_clones":38,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":38,"num_read":113,"num_read_kb":14121,"num_write":1423,"num_write_kb":30929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"89'2145","reported_seq":5075,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.939348+0000","last_change":"2024-03-09T09:27:34.506277+0000","last_active":"2024-03-09T09:29:17.939348+0000","last_peered":"2024-03-09T09:29:17.939348+0000","last_clean":"2024-03-09T09:29:17.939348+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T09:29:17.939348+0000","last_undegraded":"2024-03-09T09:29:17.939348+0000","last_fullsized":"2024-03-09T09:29:17.939348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2145,"log_dups_size":0,"ondisk_log_size":2145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:27.408652+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.1683436250000003,"stat_sum":{"num_bytes":2080769,"num_objects":116,"num_object_clones":43,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":43,"num_read":114,"num_read_kb":27228,"num_write":1275,"num_write_kb":54719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"89'2227","reported_seq":5198,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.040802+0000","last_change":"2024-03-09T09:27:13.892143+0000","last_active":"2024-03-09T09:29:14.040802+0000","last_peered":"2024-03-09T09:29:14.040802+0000","last_clean":"2024-03-09T09:29:14.040802+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T09:29:14.040802+0000","last_undegraded":"2024-03-09T09:29:14.040802+0000","last_fullsized":"2024-03-09T09:29:14.040802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2227,"log_dups_size":0,"ondisk_log_size":2227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:36:18.464529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.4282657350000001,"stat_sum":{"num_bytes":1076453,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":106,"num_read_kb":19463,"num_write":1318,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"89'2408","reported_seq":5285,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.733743+0000","last_change":"2024-03-09T09:27:30.899184+0000","last_active":"2024-03-09T09:28:56.733743+0000","last_peered":"2024-03-09T09:28:56.733743+0000","last_clean":"2024-03-09T09:28:56.733743+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T09:28:56.733743+0000","last_undegraded":"2024-03-09T09:28:56.733743+0000","last_fullsized":"2024-03-09T09:28:56.733743+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2408,"log_dups_size":0,"ondisk_log_size":2408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:15:47.160761+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.63967549000000001,"stat_sum":{"num_bytes":3580963,"num_objects":116,"num_object_clones":37,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":37,"num_read":116,"num_read_kb":32310,"num_write":1440,"num_write_kb":66236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"89'2251","reported_seq":5231,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.030277+0000","last_change":"2024-03-09T09:27:09.597741+0000","last_active":"2024-03-09T09:29:14.030277+0000","last_peered":"2024-03-09T09:29:14.030277+0000","last_clean":"2024-03-09T09:29:14.030277+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T09:29:14.030277+0000","last_undegraded":"2024-03-09T09:29:14.030277+0000","last_fullsized":"2024-03-09T09:29:14.030277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2251,"log_dups_size":0,"ondisk_log_size":2251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:29:26.509228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.47268980999999999,"stat_sum":{"num_bytes":1488567,"num_objects":112,"num_object_clones":34,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":34,"num_read":108,"num_read_kb":30029,"num_write":1350,"num_write_kb":42424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"89'2142","reported_seq":5072,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.933559+0000","last_change":"2024-03-09T09:27:34.605106+0000","last_active":"2024-03-09T09:29:17.933559+0000","last_peered":"2024-03-09T09:29:17.933559+0000","last_clean":"2024-03-09T09:29:17.933559+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T09:29:17.933559+0000","last_undegraded":"2024-03-09T09:29:17.933559+0000","last_fullsized":"2024-03-09T09:29:17.933559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2142,"log_dups_size":0,"ondisk_log_size":2142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:04:07.487361+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.3763956530000003,"stat_sum":{"num_bytes":1440979,"num_objects":98,"num_object_clones":30,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":30,"num_read":104,"num_read_kb":22606,"num_write":1302,"num_write_kb":46651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"89'2066","reported_seq":5058,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.003676+0000","last_change":"2024-03-09T09:27:10.176463+0000","last_active":"2024-03-09T09:29:14.003676+0000","last_peered":"2024-03-09T09:29:14.003676+0000","last_clean":"2024-03-09T09:29:14.003676+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T09:29:14.003676+0000","last_undegraded":"2024-03-09T09:29:14.003676+0000","last_fullsized":"2024-03-09T09:29:14.003676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2066,"log_dups_size":0,"ondisk_log_size":2066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:32:22.191345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.344208157,"stat_sum":{"num_bytes":485403,"num_objects":85,"num_object_clones":27,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":27,"num_read":96,"num_read_kb":5635,"num_write":1204,"num_write_kb":28568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"89'2216","reported_seq":5128,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.931323+0000","last_change":"2024-03-09T09:27:34.669203+0000","last_active":"2024-03-09T09:29:17.931323+0000","last_peered":"2024-03-09T09:29:17.931323+0000","last_clean":"2024-03-09T09:29:17.931323+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T09:29:17.931323+0000","last_undegraded":"2024-03-09T09:29:17.931323+0000","last_fullsized":"2024-03-09T09:29:17.931323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2216,"log_dups_size":0,"ondisk_log_size":2216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:04:18.586097+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.4378791440000001,"stat_sum":{"num_bytes":906976,"num_objects":87,"num_object_clones":27,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":27,"num_read":110,"num_read_kb":26703,"num_write":1307,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"89'2145","reported_seq":5141,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.051697+0000","last_change":"2024-03-09T09:27:13.826300+0000","last_active":"2024-03-09T09:29:14.051697+0000","last_peered":"2024-03-09T09:29:14.051697+0000","last_clean":"2024-03-09T09:29:14.051697+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T09:29:14.051697+0000","last_undegraded":"2024-03-09T09:29:14.051697+0000","last_fullsized":"2024-03-09T09:29:14.051697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2145,"log_dups_size":0,"ondisk_log_size":2145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:23:44.879214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.0826407490000001,"stat_sum":{"num_bytes":992571,"num_objects":97,"num_object_clones":32,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":32,"num_read":100,"num_read_kb":32945,"num_write":1310,"num_write_kb":54952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7626,"num_read_kb":1526336,"num_write":85965,"num_write_kb":2933421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144274,"ondisk_log_size":144274,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":875580134,"num_objects":1506,"num_object_clones":0,"num_object_copies":3012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1506,"num_whiteouts":0,"num_read":2924,"num_read_kb":5807,"num_write":25873,"num_write_kb":3143283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1225,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1752338432,"data_stored":1752278262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22159,"ondisk_log_size":22159,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7230,"num_read_kb":1519173,"num_write":85971,"num_write_kb":2933405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144300,"ondisk_log_size":144300,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798313328,"num_objects":1475,"num_object_clones":0,"num_object_copies":2950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1475,"num_whiteouts":0,"num_read":2854,"num_read_kb":5749,"num_write":25870,"num_write_kb":3104826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1213,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1596686336,"data_stored":1596626656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22164,"ondisk_log_size":22164,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575298,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":851964,"kb_used_data":501556,"kb_used_omap":1,"kb_used_meta":350398,"kb_avail":93519876,"statfs":{"total":96636764160,"available":95764353024,"internally_reserved":0,"allocated":513593344,"data_stored":506816772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":358808010},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575299,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699740,"kb_used_data":456276,"kb_used_omap":1,"kb_used_meta":243454,"kb_avail":93672100,"statfs":{"total":96636764160,"available":95920230400,"internally_reserved":0,"allocated":467226624,"data_stored":462408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":249297350},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575297,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794804,"kb_used_data":452716,"kb_used_omap":1,"kb_used_meta":342078,"kb_avail":93577036,"statfs":{"total":96636764160,"available":95822884864,"internally_reserved":0,"allocated":463581184,"data_stored":456386951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":350288325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575298,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":853316,"kb_used_data":551484,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":93518524,"statfs":{"total":96636764160,"available":95762968576,"internally_reserved":0,"allocated":564719616,"data_stored":557367936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309066181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575299,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":918132,"kb_used_data":544172,"kb_used_omap":1,"kb_used_meta":373950,"kb_avail":93453708,"statfs":{"total":96636764160,"available":95696596992,"internally_reserved":0,"allocated":557232128,"data_stored":549727853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":382925255},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575298,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450472,"kb_used_data":225056,"kb_used_omap":1,"kb_used_meta":225406,"kb_avail":93921368,"statfs":{"total":96636764160,"available":96175480832,"internally_reserved":0,"allocated":230457344,"data_stored":225591238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":230816197},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575298,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":968836,"kb_used_data":612924,"kb_used_omap":1,"kb_used_meta":355902,"kb_avail":93403004,"statfs":{"total":96636764160,"available":95644676096,"internally_reserved":0,"allocated":627634176,"data_stored":621572263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":364444101},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575300,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670332,"kb_used_data":378292,"kb_used_omap":1,"kb_used_meta":292030,"kb_avail":93701508,"statfs":{"total":96636764160,"available":95950344192,"internally_reserved":0,"allocated":387371008,"data_stored":381048744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":299039175},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":226504704,"data_stored":226493054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":218112000,"data_stored":218104019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92274688,"data_stored":92274688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":172527616,"data_stored":172519074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":201338880,"data_stored":201326769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":197980160,"data_stored":197974487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":223150080,"data_stored":223140893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":264798208,"data_stored":264793672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37781504,"data_stored":34251073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20246528,"data_stored":18282047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22822912,"data_stored":20809621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27308032,"data_stored":23727598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37437440,"data_stored":33951899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":27983872,"data_stored":24565897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23818240,"data_stored":21628413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":29773824,"data_stored":26493862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98791424,"data_stored":98790378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":360722432,"data_stored":360710838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96473088,"data_stored":96469310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":315428864,"data_stored":315415292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":289415168,"data_stored":289407328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":200581120,"data_stored":200572340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":201334784,"data_stored":201327208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":189591552,"data_stored":189585568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22835200,"data_stored":20540995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27480064,"data_stored":23882611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17813504,"data_stored":15444871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40894464,"data_stored":37473141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35454976,"data_stored":32089192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":37081088,"data_stored":33799473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17850368,"data_stored":15719573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":27762688,"data_stored":24760554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T09:29:20.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T09:29:20.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.625+0000 7fbfa1000640 1 Processor -- start 2024-03-09T09:29:20.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.625+0000 7fbfa1000640 1 -- start start 2024-03-09T09:29:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.627+0000 7fbfa1000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf9c14c740 0x7fbf9c14cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.627+0000 7fbfa1000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbf9c153ef0 0x7fbf9c06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.627+0000 7fbfa1000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9c06f6b0 con 0x7fbf9c153260 2024-03-09T09:29:20.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.627+0000 7fbfa1000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c06f930 con 0x7fbf9c153ef0 2024-03-09T09:29:20.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.627+0000 7fbfa1000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c06fc10 con 0x7fbf9c14c740 2024-03-09T09:29:20.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.627+0000 7fbf9aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbf9c153ef0 0x7fbf9c06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:20.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.627+0000 7fbf9aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fbf9c153ef0 0x7fbf9c06ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45072/0 (socket says 172.21.15.67:45072) 2024-03-09T09:29:20.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.627+0000 7fbf9aa00640 1 -- 172.21.15.67:0/3185675155 learned_addr learned my addr 172.21.15.67:0/3185675155 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:20.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.628+0000 7fbf9b400640 1 --2- 172.21.15.67:0/3185675155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf9c14c740 0x7fbf9c14cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:20.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.628+0000 7fbf9aa00640 1 -- 172.21.15.67:0/3185675155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf9c14c740 msgr2=0x7fbf9c14cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:20.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.628+0000 7fbf9aa00640 1 --2- 172.21.15.67:0/3185675155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf9c14c740 0x7fbf9c14cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.628+0000 7fbf9aa00640 1 -- 172.21.15.67:0/3185675155 >> v1:172.21.15.134:6789/0 conn(0x7fbf9c153260 legacy=0x7fbf9c0acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:20.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.628+0000 7fbf9aa00640 1 -- 172.21.15.67:0/3185675155 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c06e660 con 0x7fbf9c153ef0 2024-03-09T09:29:20.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.629+0000 7fbf9aa00640 1 --2- 172.21.15.67:0/3185675155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9c153ef0 0x7fbf9c06ee60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800bb10 tx=0x7fbf8802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38d003b6de2f701 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:20.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.629+0000 7fbf9a000640 1 -- 172.21.15.67:0/3185675155 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8803b070 con 0x7fbf9c153ef0 2024-03-09T09:29:20.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.629+0000 7fbf9a000640 1 -- 172.21.15.67:0/3185675155 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf88036440 con 0x7fbf9c153ef0 2024-03-09T09:29:20.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.629+0000 7fbf9a000640 1 -- 172.21.15.67:0/3185675155 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88036810 con 0x7fbf9c153ef0 2024-03-09T09:29:20.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.630+0000 7fbfa1000640 1 -- 172.21.15.67:0/3185675155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9c153ef0 msgr2=0x7fbf9c06ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:20.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.630+0000 7fbfa1000640 1 --2- 172.21.15.67:0/3185675155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9c153ef0 0x7fbf9c06ee60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800bb10 tx=0x7fbf8802e0c0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.630+0000 7fbfa1000640 1 -- 172.21.15.67:0/3185675155 shutdown_connections 2024-03-09T09:29:20.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.630+0000 7fbfa1000640 1 --2- 172.21.15.67:0/3185675155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9c153ef0 0x7fbf9c06ee60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.630+0000 7fbfa1000640 1 --2- 172.21.15.67:0/3185675155 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf9c14c740 0x7fbf9c14cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.630+0000 7fbfa1000640 1 -- 172.21.15.67:0/3185675155 >> 172.21.15.67:0/3185675155 conn(0x7fbf9c0c8310 msgr2=0x7fbf9c0c86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:20.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.630+0000 7fbfa1000640 1 -- 172.21.15.67:0/3185675155 shutdown_connections 2024-03-09T09:29:20.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.630+0000 7fbfa1000640 1 -- 172.21.15.67:0/3185675155 wait complete. 2024-03-09T09:29:20.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.632+0000 7fbfa1000640 1 Processor -- start 2024-03-09T09:29:20.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.632+0000 7fbfa1000640 1 -- start start 2024-03-09T09:29:20.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbfa1000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf9c14c740 0x7fbf9c0c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbfa1000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9c153260 0x7fbf9c14dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbfa1000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf9c153ef0 0x7fbf9c14fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbfa1000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c06f3a0 con 0x7fbf9c153260 2024-03-09T09:29:20.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbfa1000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c154330 con 0x7fbf9c14c740 2024-03-09T09:29:20.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbfa1000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c0ad610 con 0x7fbf9c153ef0 2024-03-09T09:29:20.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbf9aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9c153260 0x7fbf9c14dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:20.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbf9aa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9c153260 0x7fbf9c14dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45076/0 (socket says 172.21.15.67:45076) 2024-03-09T09:29:20.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbf9aa00640 1 -- 172.21.15.67:0/1145699338 learned_addr learned my addr 172.21.15.67:0/1145699338 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:20.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbf9b400640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf9c14c740 0x7fbf9c0c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:20.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbf9aa00640 1 -- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf9c153ef0 msgr2=0x7fbf9c14fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:20.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.633+0000 7fbf9aa00640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf9c153ef0 0x7fbf9c14fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.634+0000 7fbf9aa00640 1 -- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf9c14c740 msgr2=0x7fbf9c0c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:20.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.634+0000 7fbf9aa00640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf9c14c740 0x7fbf9c0c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.634+0000 7fbf9aa00640 1 -- 172.21.15.67:0/1145699338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0736b0 con 0x7fbf9c153260 2024-03-09T09:29:20.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.634+0000 7fbf9b400640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf9c14c740 0x7fbf9c0c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:20.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.634+0000 7fbf9aa00640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9c153260 0x7fbf9c14dd50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800b2c0 tx=0x7fbf880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:20.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.634+0000 7fbf93400640 1 -- 172.21.15.67:0/1145699338 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88046070 con 0x7fbf9c153260 2024-03-09T09:29:20.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.634+0000 7fbf93400640 1 -- 172.21.15.67:0/1145699338 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8803d050 con 0x7fbf9c153260 2024-03-09T09:29:20.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.634+0000 7fbfa1000640 1 -- 172.21.15.67:0/1145699338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c14d270 con 0x7fbf9c153260 2024-03-09T09:29:20.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.634+0000 7fbf93400640 1 -- 172.21.15.67:0/1145699338 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8803b030 con 0x7fbf9c153260 2024-03-09T09:29:20.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.635+0000 7fbfa1000640 1 -- 172.21.15.67:0/1145699338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c0c7f40 con 0x7fbf9c153260 2024-03-09T09:29:20.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.636+0000 7fbfa1000640 1 -- 172.21.15.67:0/1145699338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c14d090 con 0x7fbf9c153260 2024-03-09T09:29:20.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.637+0000 7fbf93400640 1 -- 172.21.15.67:0/1145699338 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbf88049070 con 0x7fbf9c153260 2024-03-09T09:29:20.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.637+0000 7fbf93400640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbf7c06b4d0 0x7fbf7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:20.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.638+0000 7fbf93400640 1 -- 172.21.15.67:0/1145699338 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fbf880ac300 con 0x7fbf9c153260 2024-03-09T09:29:20.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.638+0000 7fbf9b400640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbf7c06b4d0 0x7fbf7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:20.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.639+0000 7fbf9b400640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbf7c06b4d0 0x7fbf7c06d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c008890 tx=0x7fbf8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:20.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.647+0000 7fbf93400640 1 -- 172.21.15.67:0/1145699338 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbf9c14d090 con 0x7fbf9c153260 2024-03-09T09:29:20.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.863+0000 7fbfa1000640 1 -- 172.21.15.67:0/1145699338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf9c04f3c0 con 0x7fbf9c153260 2024-03-09T09:29:20.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.866+0000 7fbf93400640 1 -- 172.21.15.67:0/1145699338 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+20649 (secure 0 0 0) 0x7fbf9c06f6b0 con 0x7fbf9c153260 2024-03-09T09:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:29:20.875 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T09:27:40.166424+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":765,"snap_epoch":79,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":765,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":748,"length":18}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T09:29:20.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.869+0000 7fbfa1000640 1 -- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbf7c06b4d0 msgr2=0x7fbf7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:20.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.869+0000 7fbfa1000640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbf7c06b4d0 0x7fbf7c06d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c008890 tx=0x7fbf8c008040 comp rx=0 tx=0).stop 2024-03-09T09:29:20.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.869+0000 7fbfa1000640 1 -- 172.21.15.67:0/1145699338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9c153260 msgr2=0x7fbf9c14dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.869+0000 7fbfa1000640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9c153260 0x7fbf9c14dd50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800b2c0 tx=0x7fbf880046c0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.869+0000 7fbfa1000640 1 -- 172.21.15.67:0/1145699338 shutdown_connections 2024-03-09T09:29:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.869+0000 7fbfa1000640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbf7c06b4d0 0x7fbf7c06d990 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.869+0000 7fbfa1000640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf9c153ef0 0x7fbf9c14fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.869+0000 7fbfa1000640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf9c153260 0x7fbf9c14dd50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.869+0000 7fbfa1000640 1 --2- 172.21.15.67:0/1145699338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf9c14c740 0x7fbf9c0c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.869+0000 7fbfa1000640 1 -- 172.21.15.67:0/1145699338 >> 172.21.15.67:0/1145699338 conn(0x7fbf9c0c8310 msgr2=0x7fbf9c1e1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:20.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.870+0000 7fbfa1000640 1 -- 172.21.15.67:0/1145699338 shutdown_connections 2024-03-09T09:29:20.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:20.870+0000 7fbfa1000640 1 -- 172.21.15.67:0/1145699338 wait complete. 2024-03-09T09:29:21.899 INFO:tasks.ceph:Scrubbing osd.0 2024-03-09T09:29:21.899 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-09T09:29:22.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.127+0000 7eff4cc00640 1 Processor -- start 2024-03-09T09:29:22.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.128+0000 7eff4cc00640 1 -- start start 2024-03-09T09:29:22.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.129+0000 7eff4cc00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7eff480ac310 0x7eff4806ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.129+0000 7eff4cc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff4806c460 0x7eff48152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.129+0000 7eff4cc00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff480c3900 con 0x7eff48153220 2024-03-09T09:29:22.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.129+0000 7eff4cc00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff480c36b0 con 0x7eff480ac310 2024-03-09T09:29:22.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.129+0000 7eff4cc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff480c3be0 con 0x7eff4806c460 2024-03-09T09:29:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.130+0000 7eff3ea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff4806c460 0x7eff48152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.130+0000 7eff3ea00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff4806c460 0x7eff48152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:54420/0 (socket says 172.21.15.67:54420) 2024-03-09T09:29:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.130+0000 7eff3ea00640 1 -- 172.21.15.67:0/145360755 learned_addr learned my addr 172.21.15.67:0/145360755 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.130+0000 7eff47400640 1 --2- 172.21.15.67:0/145360755 >> v2:172.21.15.134:3300/0 conn(0x7eff480ac310 0x7eff4806ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.131+0000 7eff3ea00640 1 -- 172.21.15.67:0/145360755 >> v2:172.21.15.134:3300/0 conn(0x7eff480ac310 msgr2=0x7eff4806ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:22.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.131+0000 7eff3ea00640 1 --2- 172.21.15.67:0/145360755 >> v2:172.21.15.134:3300/0 conn(0x7eff480ac310 0x7eff4806ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.131+0000 7eff3ea00640 1 -- 172.21.15.67:0/145360755 >> v1:172.21.15.134:6789/0 conn(0x7eff48153220 legacy=0x7eff481594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.131+0000 7eff3ea00640 1 -- 172.21.15.67:0/145360755 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff480aa4e0 con 0x7eff4806c460 2024-03-09T09:29:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.132+0000 7eff3ea00640 1 --2- 172.21.15.67:0/145360755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff4806c460 0x7eff48152ce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff30009870 tx=0x7eff3002b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2e2faf1bc46e18 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.133+0000 7eff46a00640 1 -- 172.21.15.67:0/145360755 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff300317d0 con 0x7eff4806c460 2024-03-09T09:29:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.133+0000 7eff46a00640 1 -- 172.21.15.67:0/145360755 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff30037070 con 0x7eff4806c460 2024-03-09T09:29:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.133+0000 7eff46a00640 1 -- 172.21.15.67:0/145360755 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30031b90 con 0x7eff4806c460 2024-03-09T09:29:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.133+0000 7eff4cc00640 1 -- 172.21.15.67:0/145360755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff4806c460 msgr2=0x7eff48152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.133+0000 7eff4cc00640 1 --2- 172.21.15.67:0/145360755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff4806c460 0x7eff48152ce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff30009870 tx=0x7eff3002b870 comp rx=0 tx=0).stop 2024-03-09T09:29:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.134+0000 7eff4cc00640 1 -- 172.21.15.67:0/145360755 shutdown_connections 2024-03-09T09:29:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.134+0000 7eff4cc00640 1 --2- 172.21.15.67:0/145360755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff4806c460 0x7eff48152ce0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.134+0000 7eff4cc00640 1 --2- 172.21.15.67:0/145360755 >> v2:172.21.15.134:3300/0 conn(0x7eff480ac310 0x7eff4806ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.134+0000 7eff4cc00640 1 -- 172.21.15.67:0/145360755 >> 172.21.15.67:0/145360755 conn(0x7eff480c3d90 msgr2=0x7eff480c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:22.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.134+0000 7eff4cc00640 1 -- 172.21.15.67:0/145360755 shutdown_connections 2024-03-09T09:29:22.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.134+0000 7eff4cc00640 1 -- 172.21.15.67:0/145360755 wait complete. 2024-03-09T09:29:22.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.135+0000 7eff4cc00640 1 Processor -- start 2024-03-09T09:29:22.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.136+0000 7eff4cc00640 1 -- start start 2024-03-09T09:29:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.136+0000 7eff4cc00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff4806c460 0x7eff48147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.136+0000 7eff4cc00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff480ac310 0x7eff481eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.137+0000 7eff4cc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff48153220 0x7eff481ef000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.137+0000 7eff4cc00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff48153800 con 0x7eff48153220 2024-03-09T09:29:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.137+0000 7eff4cc00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff48153680 con 0x7eff4806c460 2024-03-09T09:29:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.137+0000 7eff4cc00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff480ac6f0 con 0x7eff480ac310 2024-03-09T09:29:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.137+0000 7eff47e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff48153220 0x7eff481ef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.137+0000 7eff47e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff48153220 0x7eff481ef000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45080/0 (socket says 172.21.15.67:45080) 2024-03-09T09:29:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.137+0000 7eff47e00640 1 -- 172.21.15.67:0/2777596367 learned_addr learned my addr 172.21.15.67:0/2777596367 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.137+0000 7eff3ea00640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff480ac310 0x7eff481eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.137+0000 7eff47400640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff4806c460 0x7eff48147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.138+0000 7eff47e00640 1 -- 172.21.15.67:0/2777596367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff480ac310 msgr2=0x7eff481eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.138+0000 7eff47e00640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff480ac310 0x7eff481eeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.138+0000 7eff47e00640 1 -- 172.21.15.67:0/2777596367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff4806c460 msgr2=0x7eff48147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.138+0000 7eff47e00640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff4806c460 0x7eff48147890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.138+0000 7eff47e00640 1 -- 172.21.15.67:0/2777596367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff481ef540 con 0x7eff48153220 2024-03-09T09:29:22.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.138+0000 7eff47e00640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff48153220 0x7eff481ef000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c3c0 tx=0x7eff4000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:22.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.138+0000 7eff3fe00640 1 -- 172.21.15.67:0/2777596367 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40017070 con 0x7eff48153220 2024-03-09T09:29:22.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.138+0000 7eff3fe00640 1 -- 172.21.15.67:0/2777596367 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff400052d0 con 0x7eff48153220 2024-03-09T09:29:22.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.139+0000 7eff4cc00640 1 -- 172.21.15.67:0/2777596367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff48070d60 con 0x7eff48153220 2024-03-09T09:29:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.139+0000 7eff3fe00640 1 -- 172.21.15.67:0/2777596367 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40003810 con 0x7eff48153220 2024-03-09T09:29:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.139+0000 7eff4cc00640 1 -- 172.21.15.67:0/2777596367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff481500f0 con 0x7eff48153220 2024-03-09T09:29:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.140+0000 7eff4cc00640 1 -- 172.21.15.67:0/2777596367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff4804f430 con 0x7eff48153220 2024-03-09T09:29:22.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.141+0000 7eff3fe00640 1 -- 172.21.15.67:0/2777596367 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7eff4001a070 con 0x7eff48153220 2024-03-09T09:29:22.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.142+0000 7eff3fe00640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7eff2406b500 0x7eff2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.142+0000 7eff3fe00640 1 -- 172.21.15.67:0/2777596367 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7eff400898b0 con 0x7eff48153220 2024-03-09T09:29:22.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.142+0000 7eff3fe00640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] conn(0x7eff24075e80 0x7eff24078320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.142+0000 7eff3fe00640 1 -- 172.21.15.67:0/2777596367 --> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff240789f0 con 0x7eff24075e80 2024-03-09T09:29:22.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.142+0000 7eff3fe00640 1 -- 172.21.15.67:0/2777596367 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff4008e2a0 con 0x7eff48153220 2024-03-09T09:29:22.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.143+0000 7eff47400640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7eff2406b500 0x7eff2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.143+0000 7eff3ea00640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] conn(0x7eff24075e80 0x7eff24078320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.143+0000 7eff3ea00640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] conn(0x7eff24075e80 0x7eff24078320 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:22.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.144+0000 7eff47400640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7eff2406b500 0x7eff2406d9c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7eff380113e0 tx=0x7eff3800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:22.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.144+0000 7eff3fe00640 1 -- 172.21.15.67:0/2777596367 <== osd.0 v2:172.21.15.67:6805/3109561126 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7eff240789f0 con 0x7eff24075e80 2024-03-09T09:29:22.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.159+0000 7eff4cc00640 1 -- 172.21.15.67:0/2777596367 --> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7eff480c5890 con 0x7eff24075e80 2024-03-09T09:29:22.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.159+0000 7eff3fe00640 1 -- 172.21.15.67:0/2777596367 <== osd.0 v2:172.21.15.67:6805/3109561126 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7eff480c5890 con 0x7eff24075e80 2024-03-09T09:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-09T09:29:22.167 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:29:22.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.160+0000 7eff4cc00640 1 -- 172.21.15.67:0/2777596367 >> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] conn(0x7eff24075e80 msgr2=0x7eff24078320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.160+0000 7eff4cc00640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.67:6805/3109561126,v1:172.21.15.67:6806/3109561126] conn(0x7eff24075e80 0x7eff24078320 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.160+0000 7eff4cc00640 1 -- 172.21.15.67:0/2777596367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7eff2406b500 msgr2=0x7eff2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.160+0000 7eff4cc00640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7eff2406b500 0x7eff2406d9c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7eff380113e0 tx=0x7eff3800b040 comp rx=0 tx=0).stop 2024-03-09T09:29:22.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.160+0000 7eff4cc00640 1 -- 172.21.15.67:0/2777596367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff48153220 msgr2=0x7eff481ef000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.160+0000 7eff4cc00640 1 --2- 172.21.15.67:0/2777596367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff48153220 0x7eff481ef000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c3c0 tx=0x7eff4000c880 comp rx=0 tx=0).stop 2024-03-09T09:29:22.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.160+0000 7eff47e00640 1 -- 172.21.15.67:0/2777596367 reap_dead start 2024-03-09T09:29:22.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.160+0000 7eff4cc00640 1 -- 172.21.15.67:0/2777596367 shutdown_connections 2024-03-09T09:29:22.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.160+0000 7eff4cc00640 1 -- 172.21.15.67:0/2777596367 >> 172.21.15.67:0/2777596367 conn(0x7eff480c3d90 msgr2=0x7eff48073660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.160+0000 7eff4cc00640 1 -- 172.21.15.67:0/2777596367 shutdown_connections 2024-03-09T09:29:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.161+0000 7eff4cc00640 1 -- 172.21.15.67:0/2777596367 wait complete. 2024-03-09T09:29:22.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-09T09:29:22.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.409+0000 7fb1bc800640 1 Processor -- start 2024-03-09T09:29:22.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.409+0000 7fb1bc800640 1 -- start start 2024-03-09T09:29:22.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.410+0000 7fb1bc800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb1b40c96f0 0x7fb1b406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.410+0000 7fb1bc800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b40c6a70 con 0x7fb1b40c5f50 2024-03-09T09:29:22.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.411+0000 7fb1bc800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b406cf30 con 0x7fb1b4153560 2024-03-09T09:29:22.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.411+0000 7fb1bc800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b4159d80 con 0x7fb1b40c96f0 2024-03-09T09:29:22.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.411+0000 7fb1ba800640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fb1b40c5f50 0x7fb1b40c6360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36802/0 (socket says 172.21.15.67:36802) 2024-03-09T09:29:22.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.411+0000 7fb1ba800640 1 -- 172.21.15.67:0/2164424370 learned_addr learned my addr 172.21.15.67:0/2164424370 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:22.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.411+0000 7fb1b9400640 1 --2- 172.21.15.67:0/2164424370 >> v2:172.21.15.134:3300/0 conn(0x7fb1b40c96f0 0x7fb1b406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.411+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585515347 0 0) 0x7fb1b0001490 con 0x7fb1b40c5f50 2024-03-09T09:29:22.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.411+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb19c003520 con 0x7fb1b40c5f50 2024-03-09T09:29:22.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2105080559 0 0) 0x7fb19c003520 con 0x7fb1b40c5f50 2024-03-09T09:29:22.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 >> v2:172.21.15.134:3300/0 conn(0x7fb1b40c96f0 msgr2=0x7fb1b406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:22.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 --2- 172.21.15.67:0/2164424370 >> v2:172.21.15.134:3300/0 conn(0x7fb1b40c96f0 0x7fb1b406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 >> v1:172.21.15.134:6789/0 conn(0x7fb1b4153560 legacy=0x7fb1b40cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b415a060 con 0x7fb1b40c5f50 2024-03-09T09:29:22.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fb1b0001bc0 con 0x7fb1b40c5f50 2024-03-09T09:29:22.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 >> v1:172.21.15.67:6789/0 conn(0x7fb1b40c5f50 legacy=0x7fb1b40c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 --2- 172.21.15.67:0/2164424370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb19c003a00 0x7fb19c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 --2- 172.21.15.67:0/2164424370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb19c004800 0x7fb19c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 --2- 172.21.15.67:0/2164424370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 0x7fb19c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b4159d80 con 0x7fb19c005230 2024-03-09T09:29:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c00c190 con 0x7fb19c004800 2024-03-09T09:29:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c00c390 con 0x7fb19c003a00 2024-03-09T09:29:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.412+0000 7fb1ba800640 1 --2- 172.21.15.67:0/2164424370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 0x7fb19c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.413+0000 7fb1b9400640 1 --2- 172.21.15.67:0/2164424370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb19c004800 0x7fb19c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.413+0000 7fb1b9e00640 1 --2- 172.21.15.67:0/2164424370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb19c003a00 0x7fb19c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.413+0000 7fb1ba800640 1 -- 172.21.15.67:0/2164424370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb19c003a00 msgr2=0x7fb19c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:22.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.413+0000 7fb1ba800640 1 --2- 172.21.15.67:0/2164424370 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb19c003a00 0x7fb19c003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.413+0000 7fb1ba800640 1 -- 172.21.15.67:0/2164424370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb19c004800 msgr2=0x7fb19c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.413+0000 7fb1ba800640 1 --2- 172.21.15.67:0/2164424370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb19c004800 0x7fb19c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.413+0000 7fb1ba800640 1 -- 172.21.15.67:0/2164424370 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c003520 con 0x7fb19c005230 2024-03-09T09:29:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.414+0000 7fb1ba800640 1 -- 172.21.15.67:0/2164424370 reap_dead start 2024-03-09T09:29:22.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.414+0000 7fb1ba800640 1 --2- 172.21.15.67:0/2164424370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 0x7fb19c00b9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000a950 tx=0x7fb19c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7b6c7a7033cd91 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:22.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.414+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000a570 con 0x7fb19c005230 2024-03-09T09:29:22.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.414+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b0007310 con 0x7fb19c005230 2024-03-09T09:29:22.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.414+0000 7fb1b8a00640 1 -- 172.21.15.67:0/2164424370 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0007700 con 0x7fb19c005230 2024-03-09T09:29:22.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.415+0000 7fb1bc800640 1 -- 172.21.15.67:0/2164424370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 msgr2=0x7fb19c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.415+0000 7fb1bc800640 1 --2- 172.21.15.67:0/2164424370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 0x7fb19c00b9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000a950 tx=0x7fb19c005830 comp rx=0 tx=0).stop 2024-03-09T09:29:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.415+0000 7fb1bc800640 1 -- 172.21.15.67:0/2164424370 shutdown_connections 2024-03-09T09:29:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.415+0000 7fb1bc800640 1 --2- 172.21.15.67:0/2164424370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 0x7fb19c00b9c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.415+0000 7fb1bc800640 1 -- 172.21.15.67:0/2164424370 >> 172.21.15.67:0/2164424370 conn(0x7fb1b40c8050 msgr2=0x7fb1b40c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.415+0000 7fb1bc800640 1 -- 172.21.15.67:0/2164424370 shutdown_connections 2024-03-09T09:29:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.415+0000 7fb1bc800640 1 -- 172.21.15.67:0/2164424370 wait complete. 2024-03-09T09:29:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.417+0000 7fb1bc800640 1 Processor -- start 2024-03-09T09:29:22.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.417+0000 7fb1bc800640 1 -- start start 2024-03-09T09:29:22.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.418+0000 7fb1bc800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 0x7fb1b41ec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.418+0000 7fb1bc800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1b40c1f60 0x7fb1b406bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.418+0000 7fb1bc800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1b40c38e0 0x7fb1b40c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.418+0000 7fb1bc800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b40c9ff0 con 0x7fb19c005230 2024-03-09T09:29:22.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.418+0000 7fb1bc800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b40ca720 con 0x7fb1b40c38e0 2024-03-09T09:29:22.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.418+0000 7fb1bc800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b406b040 con 0x7fb1b40c1f60 2024-03-09T09:29:22.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.418+0000 7fb1b9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 0x7fb1b41ec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.418+0000 7fb1b9e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 0x7fb1b41ec8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45094/0 (socket says 172.21.15.67:45094) 2024-03-09T09:29:22.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.418+0000 7fb1b9e00640 1 -- 172.21.15.67:0/4138200726 learned_addr learned my addr 172.21.15.67:0/4138200726 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:22.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.419+0000 7fb1b9e00640 1 -- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1b40c1f60 msgr2=0x7fb1b406bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.419+0000 7fb1b9e00640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1b40c1f60 0x7fb1b406bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.419+0000 7fb1b9e00640 1 -- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1b40c38e0 msgr2=0x7fb1b40c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.419+0000 7fb1b9e00640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1b40c38e0 0x7fb1b40c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.419+0000 7fb1b9e00640 1 -- 172.21.15.67:0/4138200726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b41f0390 con 0x7fb19c005230 2024-03-09T09:29:22.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.419+0000 7fb1b9e00640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 0x7fb1b41ec8c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800d570 tx=0x7fb1a800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:22.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.419+0000 7fb1ae000640 1 -- 172.21.15.67:0/4138200726 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8005480 con 0x7fb19c005230 2024-03-09T09:29:22.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.419+0000 7fb1ae000640 1 -- 172.21.15.67:0/4138200726 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a8017070 con 0x7fb19c005230 2024-03-09T09:29:22.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.419+0000 7fb1ae000640 1 -- 172.21.15.67:0/4138200726 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8003810 con 0x7fb19c005230 2024-03-09T09:29:22.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.419+0000 7fb1bc800640 1 -- 172.21.15.67:0/4138200726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b41ef5d0 con 0x7fb19c005230 2024-03-09T09:29:22.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.420+0000 7fb1bc800640 1 -- 172.21.15.67:0/4138200726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b41efb90 con 0x7fb19c005230 2024-03-09T09:29:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.421+0000 7fb1bc800640 1 -- 172.21.15.67:0/4138200726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b4075a00 con 0x7fb19c005230 2024-03-09T09:29:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.423+0000 7fb1ae000640 1 -- 172.21.15.67:0/4138200726 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb1a80039b0 con 0x7fb19c005230 2024-03-09T09:29:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.424+0000 7fb1ae000640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb19806b4d0 0x7fb19806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.425+0000 7fb1ae000640 1 -- 172.21.15.67:0/4138200726 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fb1a808a3b0 con 0x7fb19c005230 2024-03-09T09:29:22.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.425+0000 7fb1b9400640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb19806b4d0 0x7fb19806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.426+0000 7fb1b9400640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb19806b4d0 0x7fb19806d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40c65c0 tx=0x7fb1a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:22.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.429+0000 7fb1ae000640 1 -- 172.21.15.67:0/4138200726 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb1a8058e60 con 0x7fb19c005230 2024-03-09T09:29:22.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.632+0000 7fb1bc800640 1 -- 172.21.15.67:0/4138200726 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1b406cd10 con 0x7fb19806b4d0 2024-03-09T09:29:22.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.633+0000 7fb1ae000640 1 -- 172.21.15.67:0/4138200726 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb1b406cd10 con 0x7fb19806b4d0 2024-03-09T09:29:22.641 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 0 to deep-scrub 2024-03-09T09:29:22.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.636+0000 7fb1bc800640 1 -- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb19806b4d0 msgr2=0x7fb19806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.636+0000 7fb1bc800640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb19806b4d0 0x7fb19806d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40c65c0 tx=0x7fb1a4008040 comp rx=0 tx=0).stop 2024-03-09T09:29:22.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.636+0000 7fb1bc800640 1 -- 172.21.15.67:0/4138200726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 msgr2=0x7fb1b41ec8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.636+0000 7fb1bc800640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 0x7fb1b41ec8c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800d570 tx=0x7fb1a800da30 comp rx=0 tx=0).stop 2024-03-09T09:29:22.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.636+0000 7fb1bc800640 1 -- 172.21.15.67:0/4138200726 shutdown_connections 2024-03-09T09:29:22.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.636+0000 7fb1bc800640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb19806b4d0 0x7fb19806d990 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.636+0000 7fb1bc800640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1b40c38e0 0x7fb1b40c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.636+0000 7fb1bc800640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1b40c1f60 0x7fb1b406bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.636+0000 7fb1bc800640 1 --2- 172.21.15.67:0/4138200726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb19c005230 0x7fb1b41ec8c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.636+0000 7fb1bc800640 1 -- 172.21.15.67:0/4138200726 >> 172.21.15.67:0/4138200726 conn(0x7fb1b40c8050 msgr2=0x7fb1b414e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:22.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.636+0000 7fb1bc800640 1 -- 172.21.15.67:0/4138200726 shutdown_connections 2024-03-09T09:29:22.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.637+0000 7fb1bc800640 1 -- 172.21.15.67:0/4138200726 wait complete. 2024-03-09T09:29:22.666 INFO:tasks.ceph:Scrubbing osd.1 2024-03-09T09:29:22.667 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-09T09:29:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.872+0000 7f68c7e00640 1 Processor -- start 2024-03-09T09:29:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.873+0000 7f68c7e00640 1 -- start start 2024-03-09T09:29:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.874+0000 7f68c7e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f68c0153510 0x7f68c00701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.874+0000 7f68c7e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c00c7640 con 0x7f68c0073830 2024-03-09T09:29:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.874+0000 7f68c7e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c00c7920 con 0x7f68c01568c0 2024-03-09T09:29:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.874+0000 7f68c7e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c00c7ee0 con 0x7f68c0153510 2024-03-09T09:29:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.875+0000 7f68c4a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f68c0073830 0x7f68c0159b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36814/0 (socket says 172.21.15.67:36814) 2024-03-09T09:29:22.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.875+0000 7f68c4a00640 1 -- 172.21.15.67:0/597223242 learned_addr learned my addr 172.21.15.67:0/597223242 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.875+0000 7f68c5400640 1 --2- 172.21.15.67:0/597223242 >> v2:172.21.15.134:3300/0 conn(0x7f68c0153510 0x7f68c00701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.875+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074416905 0 0) 0x7f68c00c7640 con 0x7f68c0073830 2024-03-09T09:29:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.875+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68a8003520 con 0x7f68c0073830 2024-03-09T09:29:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1599505766 0 0) 0x7f68a8003520 con 0x7f68c0073830 2024-03-09T09:29:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 >> v2:172.21.15.134:3300/0 conn(0x7f68c0153510 msgr2=0x7f68c00701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 --2- 172.21.15.67:0/597223242 >> v2:172.21.15.134:3300/0 conn(0x7f68c0153510 0x7f68c00701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 >> v1:172.21.15.134:6789/0 conn(0x7f68c01568c0 legacy=0x7f68c00c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c00c7c00 con 0x7f68c0073830 2024-03-09T09:29:22.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f68ac001670 con 0x7f68c0073830 2024-03-09T09:29:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 >> v1:172.21.15.67:6789/0 conn(0x7f68c0073830 legacy=0x7f68c0159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 --2- 172.21.15.67:0/597223242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68a80039b0 0x7f68a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 --2- 172.21.15.67:0/597223242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a8004800 0x7f68a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 --2- 172.21.15.67:0/597223242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68a80052e0 0x7f68a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a800c180 con 0x7f68a80052e0 2024-03-09T09:29:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a800c330 con 0x7f68a8004800 2024-03-09T09:29:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a800c530 con 0x7f68a80039b0 2024-03-09T09:29:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.876+0000 7f68c5e00640 1 --2- 172.21.15.67:0/597223242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68a80052e0 0x7f68a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.877+0000 7f68c5e00640 1 -- 172.21.15.67:0/597223242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68a80039b0 msgr2=0x7f68a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.877+0000 7f68c5e00640 1 --2- 172.21.15.67:0/597223242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68a80039b0 0x7f68a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.877+0000 7f68c4a00640 1 --2- 172.21.15.67:0/597223242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a8004800 0x7f68a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.877+0000 7f68c5e00640 1 -- 172.21.15.67:0/597223242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a8004800 msgr2=0x7f68a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.877+0000 7f68c5e00640 1 --2- 172.21.15.67:0/597223242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a8004800 0x7f68a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.877+0000 7f68c5e00640 1 -- 172.21.15.67:0/597223242 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a8003520 con 0x7f68a80052e0 2024-03-09T09:29:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.877+0000 7f68c4a00640 1 --2- 172.21.15.67:0/597223242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a8004800 0x7f68a8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.878+0000 7f68c5e00640 1 -- 172.21.15.67:0/597223242 reap_dead start 2024-03-09T09:29:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.878+0000 7f68c5e00640 1 --2- 172.21.15.67:0/597223242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68a80052e0 0x7f68a800b9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68b8001e20 tx=0x7f68a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cade31225772e74 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.878+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8001a70 con 0x7f68a80052e0 2024-03-09T09:29:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.878+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b8003b70 con 0x7f68a80052e0 2024-03-09T09:29:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.878+0000 7f68bfe00640 1 -- 172.21.15.67:0/597223242 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b80050d0 con 0x7f68a80052e0 2024-03-09T09:29:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.879+0000 7f68c7e00640 1 -- 172.21.15.67:0/597223242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68a80052e0 msgr2=0x7f68a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.879+0000 7f68c7e00640 1 --2- 172.21.15.67:0/597223242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68a80052e0 0x7f68a800b9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f68b8001e20 tx=0x7f68a8005830 comp rx=0 tx=0).stop 2024-03-09T09:29:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.879+0000 7f68c7e00640 1 -- 172.21.15.67:0/597223242 shutdown_connections 2024-03-09T09:29:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.879+0000 7f68c7e00640 1 --2- 172.21.15.67:0/597223242 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68a80052e0 0x7f68a800b9c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.879+0000 7f68c7e00640 1 -- 172.21.15.67:0/597223242 >> 172.21.15.67:0/597223242 conn(0x7f68c0067180 msgr2=0x7f68c0067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.879+0000 7f68c7e00640 1 -- 172.21.15.67:0/597223242 shutdown_connections 2024-03-09T09:29:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.879+0000 7f68c7e00640 1 -- 172.21.15.67:0/597223242 wait complete. 2024-03-09T09:29:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.881+0000 7f68c7e00640 1 Processor -- start 2024-03-09T09:29:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.881+0000 7f68c7e00640 1 -- start start 2024-03-09T09:29:22.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.882+0000 7f68c7e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a80052e0 0x7f68c0154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.882+0000 7f68c7e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68c01ec5b0 0x7f68c01585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.882+0000 7f68c7e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68c0157980 0x7f68c0141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.882+0000 7f68c7e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0159410 con 0x7f68c01ec5b0 2024-03-09T09:29:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.882+0000 7f68c7e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01538f0 con 0x7f68a80052e0 2024-03-09T09:29:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.882+0000 7f68c7e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01596f0 con 0x7f68c0157980 2024-03-09T09:29:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.882+0000 7f68c4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68c01ec5b0 0x7f68c01585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.882+0000 7f68c4a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68c01ec5b0 0x7f68c01585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45102/0 (socket says 172.21.15.67:45102) 2024-03-09T09:29:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.882+0000 7f68c4a00640 1 -- 172.21.15.67:0/3040290727 learned_addr learned my addr 172.21.15.67:0/3040290727 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.882+0000 7f68c5400640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a80052e0 0x7f68c0154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.883+0000 7f68c5e00640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68c0157980 0x7f68c0141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.883+0000 7f68c4a00640 1 -- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68c0157980 msgr2=0x7f68c0141d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.883+0000 7f68c4a00640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68c0157980 0x7f68c0141d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.883+0000 7f68c4a00640 1 -- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a80052e0 msgr2=0x7f68c0154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.883+0000 7f68c4a00640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a80052e0 0x7f68c0154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.883+0000 7f68c4a00640 1 -- 172.21.15.67:0/3040290727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c014df70 con 0x7f68c01ec5b0 2024-03-09T09:29:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.883+0000 7f68c5e00640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f68c0157980 0x7f68c0141d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.883+0000 7f68c4a00640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68c01ec5b0 0x7f68c01585e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d840 tx=0x7f68ac00dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.885+0000 7f68bd600640 1 -- 172.21.15.67:0/3040290727 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac019070 con 0x7f68c01ec5b0 2024-03-09T09:29:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.886+0000 7f68bd600640 1 -- 172.21.15.67:0/3040290727 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac0074e0 con 0x7f68c01ec5b0 2024-03-09T09:29:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.886+0000 7f68c7e00640 1 -- 172.21.15.67:0/3040290727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c014e250 con 0x7f68c01ec5b0 2024-03-09T09:29:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.886+0000 7f68bd600640 1 -- 172.21.15.67:0/3040290727 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0049e0 con 0x7f68c01ec5b0 2024-03-09T09:29:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.886+0000 7f68c7e00640 1 -- 172.21.15.67:0/3040290727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c0158b20 con 0x7f68c01ec5b0 2024-03-09T09:29:22.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.887+0000 7f68c7e00640 1 -- 172.21.15.67:0/3040290727 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68c0157560 con 0x7f68c01ec5b0 2024-03-09T09:29:22.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.888+0000 7f68bd600640 1 -- 172.21.15.67:0/3040290727 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f68ac01c070 con 0x7f68c01ec5b0 2024-03-09T09:29:22.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.889+0000 7f68bd600640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f68a006b500 0x7f68a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.889+0000 7f68bd600640 1 -- 172.21.15.67:0/3040290727 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f68ac089fe0 con 0x7f68c01ec5b0 2024-03-09T09:29:22.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.889+0000 7f68bd600640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] conn(0x7f68a0075ef0 0x7f68a0078390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:22.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.889+0000 7f68bd600640 1 -- 172.21.15.67:0/3040290727 --> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68a0078a60 con 0x7f68a0075ef0 2024-03-09T09:29:22.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.889+0000 7f68bd600640 1 -- 172.21.15.67:0/3040290727 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68ac091050 con 0x7f68c01ec5b0 2024-03-09T09:29:22.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.889+0000 7f68c5e00640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] conn(0x7f68a0075ef0 0x7f68a0078390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.890+0000 7f68c5400640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f68a006b500 0x7f68a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:22.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.890+0000 7f68c5e00640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] conn(0x7f68a0075ef0 0x7f68a0078390 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:22.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.891+0000 7f68bd600640 1 -- 172.21.15.67:0/3040290727 <== osd.1 v2:172.21.15.67:6815/1773797031 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f68a0078a60 con 0x7f68a0075ef0 2024-03-09T09:29:22.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.891+0000 7f68c5400640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f68a006b500 0x7f68a006d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f68c0155e80 tx=0x7f68b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:22.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.921+0000 7f68c7e00640 1 -- 172.21.15.67:0/3040290727 --> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f68c0075a00 con 0x7f68a0075ef0 2024-03-09T09:29:22.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.921+0000 7f68bd600640 1 -- 172.21.15.67:0/3040290727 <== osd.1 v2:172.21.15.67:6815/1773797031 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f68c0075a00 con 0x7f68a0075ef0 2024-03-09T09:29:22.929 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:29:22.929 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-09T09:29:22.929 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:29:22.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.922+0000 7f68c7e00640 1 -- 172.21.15.67:0/3040290727 >> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] conn(0x7f68a0075ef0 msgr2=0x7f68a0078390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.922+0000 7f68c7e00640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.67:6815/1773797031,v1:172.21.15.67:6817/1773797031] conn(0x7f68a0075ef0 0x7f68a0078390 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:22.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.922+0000 7f68c7e00640 1 -- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f68a006b500 msgr2=0x7f68a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.922+0000 7f68c7e00640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f68a006b500 0x7f68a006d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f68c0155e80 tx=0x7f68b4008040 comp rx=0 tx=0).stop 2024-03-09T09:29:22.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.922+0000 7f68c7e00640 1 -- 172.21.15.67:0/3040290727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68c01ec5b0 msgr2=0x7f68c01585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:22.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.922+0000 7f68c7e00640 1 --2- 172.21.15.67:0/3040290727 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f68c01ec5b0 0x7f68c01585e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00d840 tx=0x7f68ac00dd00 comp rx=0 tx=0).stop 2024-03-09T09:29:22.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.922+0000 7f68c5e00640 1 -- 172.21.15.67:0/3040290727 reap_dead start 2024-03-09T09:29:22.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.922+0000 7f68c7e00640 1 -- 172.21.15.67:0/3040290727 shutdown_connections 2024-03-09T09:29:22.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.922+0000 7f68c7e00640 1 -- 172.21.15.67:0/3040290727 >> 172.21.15.67:0/3040290727 conn(0x7f68c0067180 msgr2=0x7f68c0067560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:22.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.922+0000 7f68c7e00640 1 -- 172.21.15.67:0/3040290727 shutdown_connections 2024-03-09T09:29:22.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:22.922+0000 7f68c7e00640 1 -- 172.21.15.67:0/3040290727 wait complete. 2024-03-09T09:29:22.951 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-09T09:29:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.175+0000 7fa9e9200640 1 Processor -- start 2024-03-09T09:29:23.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.175+0000 7fa9e9200640 1 -- start start 2024-03-09T09:29:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.176+0000 7fa9e9200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9e406b880 0x7fa9e4159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.176+0000 7fa9e9200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9e40c6650 0x7fa9e4155e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.176+0000 7fa9e9200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e406c040 con 0x7fa9e40ad8d0 2024-03-09T09:29:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.176+0000 7fa9e9200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e40c61f0 con 0x7fa9e40c6650 2024-03-09T09:29:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.176+0000 7fa9e9200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e406bd60 con 0x7fa9e406b880 2024-03-09T09:29:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.177+0000 7fa9e3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9e40c6650 0x7fa9e4155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.177+0000 7fa9e3e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fa9e40c6650 0x7fa9e4155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45106/0 (socket says 172.21.15.67:45106) 2024-03-09T09:29:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.177+0000 7fa9e3e00640 1 -- 172.21.15.67:0/195424959 learned_addr learned my addr 172.21.15.67:0/195424959 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.177+0000 7fa9e2a00640 1 --2- 172.21.15.67:0/195424959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9e406b880 0x7fa9e4159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.177+0000 7fa9e3e00640 1 -- 172.21.15.67:0/195424959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9e406b880 msgr2=0x7fa9e4159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:23.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.177+0000 7fa9e3e00640 1 --2- 172.21.15.67:0/195424959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9e406b880 0x7fa9e4159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.177+0000 7fa9e3e00640 1 -- 172.21.15.67:0/195424959 >> v1:172.21.15.134:6789/0 conn(0x7fa9e40ad8d0 legacy=0x7fa9e4153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.177+0000 7fa9e3e00640 1 -- 172.21.15.67:0/195424959 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e40c5e80 con 0x7fa9e40c6650 2024-03-09T09:29:23.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.177+0000 7fa9e2a00640 1 --2- 172.21.15.67:0/195424959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9e406b880 0x7fa9e4159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:29:23.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.179+0000 7fa9e3e00640 1 --2- 172.21.15.67:0/195424959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e40c6650 0x7fa9e4155e40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009e50 tx=0x7fa9dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308cfa7facf05c74 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:23.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.179+0000 7fa9e2000640 1 -- 172.21.15.67:0/195424959 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc02ca70 con 0x7fa9e40c6650 2024-03-09T09:29:23.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.180+0000 7fa9e2000640 1 -- 172.21.15.67:0/195424959 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc02cc10 con 0x7fa9e40c6650 2024-03-09T09:29:23.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.180+0000 7fa9e2000640 1 -- 172.21.15.67:0/195424959 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0343f0 con 0x7fa9e40c6650 2024-03-09T09:29:23.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.180+0000 7fa9e9200640 1 -- 172.21.15.67:0/195424959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e40c6650 msgr2=0x7fa9e4155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.180+0000 7fa9e9200640 1 --2- 172.21.15.67:0/195424959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e40c6650 0x7fa9e4155e40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009e50 tx=0x7fa9dc02bed0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.180+0000 7fa9e9200640 1 -- 172.21.15.67:0/195424959 shutdown_connections 2024-03-09T09:29:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.180+0000 7fa9e9200640 1 --2- 172.21.15.67:0/195424959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e40c6650 0x7fa9e4155e40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.180+0000 7fa9e9200640 1 --2- 172.21.15.67:0/195424959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9e406b880 0x7fa9e4159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.180+0000 7fa9e9200640 1 -- 172.21.15.67:0/195424959 >> 172.21.15.67:0/195424959 conn(0x7fa9e40c8510 msgr2=0x7fa9e40acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.180+0000 7fa9e9200640 1 -- 172.21.15.67:0/195424959 shutdown_connections 2024-03-09T09:29:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.180+0000 7fa9e9200640 1 -- 172.21.15.67:0/195424959 wait complete. 2024-03-09T09:29:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.183+0000 7fa9e9200640 1 Processor -- start 2024-03-09T09:29:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.183+0000 7fa9e9200640 1 -- start start 2024-03-09T09:29:23.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.185+0000 7fa9e9200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406b880 0x7fa9e40ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.185+0000 7fa9e9200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e40ad8d0 0x7fa9e40ac420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.185+0000 7fa9e9200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9e40c6650 0x7fa9e40af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.185+0000 7fa9e9200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40adcb0 con 0x7fa9e406b880 2024-03-09T09:29:23.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.185+0000 7fa9e9200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e41539a0 con 0x7fa9e40ad8d0 2024-03-09T09:29:23.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.185+0000 7fa9e9200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40ade30 con 0x7fa9e40c6650 2024-03-09T09:29:23.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.185+0000 7fa9e3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406b880 0x7fa9e40ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.185+0000 7fa9e3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406b880 0x7fa9e40ae440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45114/0 (socket says 172.21.15.67:45114) 2024-03-09T09:29:23.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.185+0000 7fa9e3400640 1 -- 172.21.15.67:0/1189905522 learned_addr learned my addr 172.21.15.67:0/1189905522 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:23.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.185+0000 7fa9e2a00640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e40ad8d0 0x7fa9e40ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.185+0000 7fa9e3e00640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9e40c6650 0x7fa9e40af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.186+0000 7fa9e3400640 1 -- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9e40c6650 msgr2=0x7fa9e40af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.186+0000 7fa9e3400640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9e40c6650 0x7fa9e40af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.186+0000 7fa9e3400640 1 -- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e40ad8d0 msgr2=0x7fa9e40ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.186+0000 7fa9e3400640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e40ad8d0 0x7fa9e40ac420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.186+0000 7fa9e3400640 1 -- 172.21.15.67:0/1189905522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e40ac960 con 0x7fa9e406b880 2024-03-09T09:29:23.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.186+0000 7fa9e3400640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406b880 0x7fa9e40ae440 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400c430 tx=0x7fa9d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:23.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.186+0000 7fa9db400640 1 -- 172.21.15.67:0/1189905522 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40052d0 con 0x7fa9e406b880 2024-03-09T09:29:23.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.186+0000 7fa9db400640 1 -- 172.21.15.67:0/1189905522 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4005490 con 0x7fa9e406b880 2024-03-09T09:29:23.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.186+0000 7fa9e9200640 1 -- 172.21.15.67:0/1189905522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e40c1e40 con 0x7fa9e406b880 2024-03-09T09:29:23.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.186+0000 7fa9db400640 1 -- 172.21.15.67:0/1189905522 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4007260 con 0x7fa9e406b880 2024-03-09T09:29:23.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.186+0000 7fa9e9200640 1 -- 172.21.15.67:0/1189905522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e41ec130 con 0x7fa9e406b880 2024-03-09T09:29:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.188+0000 7fa9e9200640 1 -- 172.21.15.67:0/1189905522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e41ec510 con 0x7fa9e406b880 2024-03-09T09:29:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.188+0000 7fa9db400640 1 -- 172.21.15.67:0/1189905522 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa9d4007400 con 0x7fa9e406b880 2024-03-09T09:29:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.190+0000 7fa9db400640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa9c406b4d0 0x7fa9c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.190+0000 7fa9db400640 1 -- 172.21.15.67:0/1189905522 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fa9d4089a20 con 0x7fa9e406b880 2024-03-09T09:29:23.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.191+0000 7fa9e2a00640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa9c406b4d0 0x7fa9c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.192+0000 7fa9e2a00640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa9c406b4d0 0x7fa9c406d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4151ef0 tx=0x7fa9d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:23.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.193+0000 7fa9db400640 1 -- 172.21.15.67:0/1189905522 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa9e41ec510 con 0x7fa9e406b880 2024-03-09T09:29:23.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.404+0000 7fa9e9200640 1 -- 172.21.15.67:0/1189905522 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9e40ad4b0 con 0x7fa9c406b4d0 2024-03-09T09:29:23.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.406+0000 7fa9db400640 1 -- 172.21.15.67:0/1189905522 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa9e40ad4b0 con 0x7fa9c406b4d0 2024-03-09T09:29:23.414 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 1 to deep-scrub 2024-03-09T09:29:23.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 -- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa9c406b4d0 msgr2=0x7fa9c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa9c406b4d0 0x7fa9c406d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4151ef0 tx=0x7fa9d0008040 comp rx=0 tx=0).stop 2024-03-09T09:29:23.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 -- 172.21.15.67:0/1189905522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406b880 msgr2=0x7fa9e40ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406b880 0x7fa9e40ae440 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400c430 tx=0x7fa9d400c8f0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 -- 172.21.15.67:0/1189905522 shutdown_connections 2024-03-09T09:29:23.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fa9c406b4d0 0x7fa9c406d990 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9e40c6650 0x7fa9e40af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9e40ad8d0 0x7fa9e40ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 --2- 172.21.15.67:0/1189905522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9e406b880 0x7fa9e40ae440 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 -- 172.21.15.67:0/1189905522 >> 172.21.15.67:0/1189905522 conn(0x7fa9e40c8510 msgr2=0x7fa9e40c9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:23.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 -- 172.21.15.67:0/1189905522 shutdown_connections 2024-03-09T09:29:23.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.409+0000 7fa9e9200640 1 -- 172.21.15.67:0/1189905522 wait complete. 2024-03-09T09:29:23.437 INFO:tasks.ceph:Scrubbing osd.2 2024-03-09T09:29:23.438 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-09T09:29:23.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.657+0000 7fbd8b200640 1 Processor -- start 2024-03-09T09:29:23.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.657+0000 7fbd8b200640 1 -- start start 2024-03-09T09:29:23.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.659+0000 7fbd8b200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fbd8c153d30 0x7fbd8c1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.659+0000 7fbd8b200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c155eb0 con 0x7fbd8c14c9c0 2024-03-09T09:29:23.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.659+0000 7fbd8b200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c156190 con 0x7fbd8c0c37d0 2024-03-09T09:29:23.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.659+0000 7fbd8b200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd8c156ac0 con 0x7fbd8c153d30 2024-03-09T09:29:23.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.659+0000 7fbd89e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fbd8c14c9c0 0x7fbd8c153230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36820/0 (socket says 172.21.15.67:36820) 2024-03-09T09:29:23.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.659+0000 7fbd89e00640 1 -- 172.21.15.67:0/2804842912 learned_addr learned my addr 172.21.15.67:0/2804842912 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.659+0000 7fbd8a800640 1 --2- 172.21.15.67:0/2804842912 >> v2:172.21.15.134:3300/0 conn(0x7fbd8c153d30 0x7fbd8c1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830134147 0 0) 0x7fbd8c155eb0 con 0x7fbd8c14c9c0 2024-03-09T09:29:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd70005730 con 0x7fbd8c14c9c0 2024-03-09T09:29:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 431168450 0 0) 0x7fbd70005730 con 0x7fbd8c14c9c0 2024-03-09T09:29:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 >> v2:172.21.15.134:3300/0 conn(0x7fbd8c153d30 msgr2=0x7fbd8c1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 --2- 172.21.15.67:0/2804842912 >> v2:172.21.15.134:3300/0 conn(0x7fbd8c153d30 0x7fbd8c1541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 >> v1:172.21.15.134:6789/0 conn(0x7fbd8c0c37d0 legacy=0x7fbd8c0c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c156470 con 0x7fbd8c14c9c0 2024-03-09T09:29:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fbd7c0019f0 con 0x7fbd8c14c9c0 2024-03-09T09:29:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 >> v1:172.21.15.67:6789/0 conn(0x7fbd8c14c9c0 legacy=0x7fbd8c153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 --2- 172.21.15.67:0/2804842912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd70005c10 0x7fbd70006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 --2- 172.21.15.67:0/2804842912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd70006a10 0x7fbd70006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 --2- 172.21.15.67:0/2804842912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd70007420 0x7fbd7000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c156ac0 con 0x7fbd70007420 2024-03-09T09:29:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7000e370 con 0x7fbd70006a10 2024-03-09T09:29:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.660+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7000e570 con 0x7fbd70005c10 2024-03-09T09:29:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.661+0000 7fbd8a800640 1 --2- 172.21.15.67:0/2804842912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd70007420 0x7fbd7000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.661+0000 7fbd89400640 1 --2- 172.21.15.67:0/2804842912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd70006a10 0x7fbd70006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.661+0000 7fbd89e00640 1 --2- 172.21.15.67:0/2804842912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd70005c10 0x7fbd70006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.661+0000 7fbd8a800640 1 -- 172.21.15.67:0/2804842912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd70005c10 msgr2=0x7fbd70006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.661+0000 7fbd8a800640 1 --2- 172.21.15.67:0/2804842912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd70005c10 0x7fbd70006030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.661+0000 7fbd8a800640 1 -- 172.21.15.67:0/2804842912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd70006a10 msgr2=0x7fbd70006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:23.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.661+0000 7fbd8a800640 1 --2- 172.21.15.67:0/2804842912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd70006a10 0x7fbd70006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.661+0000 7fbd8a800640 1 -- 172.21.15.67:0/2804842912 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70005730 con 0x7fbd70007420 2024-03-09T09:29:23.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.662+0000 7fbd8a800640 1 -- 172.21.15.67:0/2804842912 reap_dead start 2024-03-09T09:29:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.662+0000 7fbd8a800640 1 --2- 172.21.15.67:0/2804842912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd70007420 0x7fbd7000dbd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009a90 tx=0x7fbd70007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab7563cea23850f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.662+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84009690 con 0x7fbd70007420 2024-03-09T09:29:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.662+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd84004020 con 0x7fbd70007420 2024-03-09T09:29:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.662+0000 7fbd88a00640 1 -- 172.21.15.67:0/2804842912 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd840043d0 con 0x7fbd70007420 2024-03-09T09:29:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.663+0000 7fbd8b200640 1 -- 172.21.15.67:0/2804842912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd70007420 msgr2=0x7fbd7000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.663+0000 7fbd8b200640 1 --2- 172.21.15.67:0/2804842912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd70007420 0x7fbd7000dbd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009a90 tx=0x7fbd70007a40 comp rx=0 tx=0).stop 2024-03-09T09:29:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.663+0000 7fbd8b200640 1 -- 172.21.15.67:0/2804842912 shutdown_connections 2024-03-09T09:29:23.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.663+0000 7fbd8b200640 1 --2- 172.21.15.67:0/2804842912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd70007420 0x7fbd7000dbd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.663+0000 7fbd8b200640 1 -- 172.21.15.67:0/2804842912 >> 172.21.15.67:0/2804842912 conn(0x7fbd8c0c3d90 msgr2=0x7fbd8c0c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.663+0000 7fbd8b200640 1 -- 172.21.15.67:0/2804842912 shutdown_connections 2024-03-09T09:29:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.663+0000 7fbd8b200640 1 -- 172.21.15.67:0/2804842912 wait complete. 2024-03-09T09:29:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.665+0000 7fbd8b200640 1 Processor -- start 2024-03-09T09:29:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.665+0000 7fbd8b200640 1 -- start start 2024-03-09T09:29:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.666+0000 7fbd8b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd70007420 0x7fbd8c1ebd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.666+0000 7fbd8b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd8c149560 0x7fbd8c1499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.666+0000 7fbd8b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd8c14ed10 0x7fbd8c14f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.666+0000 7fbd8b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c153230 con 0x7fbd8c14ed10 2024-03-09T09:29:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.666+0000 7fbd8b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c070860 con 0x7fbd8c149560 2024-03-09T09:29:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.666+0000 7fbd8b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c1533b0 con 0x7fbd70007420 2024-03-09T09:29:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.666+0000 7fbd8a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd8c14ed10 0x7fbd8c14f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.666+0000 7fbd8a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd8c14ed10 0x7fbd8c14f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45132/0 (socket says 172.21.15.67:45132) 2024-03-09T09:29:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.666+0000 7fbd8a800640 1 -- 172.21.15.67:0/4115774150 learned_addr learned my addr 172.21.15.67:0/4115774150 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.667+0000 7fbd89e00640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd70007420 0x7fbd8c1ebd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.667+0000 7fbd89400640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd8c149560 0x7fbd8c1499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.667+0000 7fbd8a800640 1 -- 172.21.15.67:0/4115774150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd70007420 msgr2=0x7fbd8c1ebd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.667+0000 7fbd8a800640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd70007420 0x7fbd8c1ebd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.667+0000 7fbd8a800640 1 -- 172.21.15.67:0/4115774150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd8c149560 msgr2=0x7fbd8c1499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.667+0000 7fbd8a800640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd8c149560 0x7fbd8c1499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.667+0000 7fbd8a800640 1 -- 172.21.15.67:0/4115774150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c1ec280 con 0x7fbd8c14ed10 2024-03-09T09:29:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.667+0000 7fbd8a800640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd8c14ed10 0x7fbd8c14f190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009b80 tx=0x7fbd84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:23.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.667+0000 7fbd82000640 1 -- 172.21.15.67:0/4115774150 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84009690 con 0x7fbd8c14ed10 2024-03-09T09:29:23.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.667+0000 7fbd82000640 1 -- 172.21.15.67:0/4115774150 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd84003610 con 0x7fbd8c14ed10 2024-03-09T09:29:23.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.668+0000 7fbd8b200640 1 -- 172.21.15.67:0/4115774150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8c150060 con 0x7fbd8c14ed10 2024-03-09T09:29:23.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.668+0000 7fbd82000640 1 -- 172.21.15.67:0/4115774150 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd840039e0 con 0x7fbd8c14ed10 2024-03-09T09:29:23.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.668+0000 7fbd8b200640 1 -- 172.21.15.67:0/4115774150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8c14ce40 con 0x7fbd8c14ed10 2024-03-09T09:29:23.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.669+0000 7fbd8b200640 1 -- 172.21.15.67:0/4115774150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd8c04f430 con 0x7fbd8c14ed10 2024-03-09T09:29:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.671+0000 7fbd82000640 1 -- 172.21.15.67:0/4115774150 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbd84003b80 con 0x7fbd8c14ed10 2024-03-09T09:29:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.672+0000 7fbd82000640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbd6806b4d0 0x7fbd6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.672+0000 7fbd82000640 1 -- 172.21.15.67:0/4115774150 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fbd840ab560 con 0x7fbd8c14ed10 2024-03-09T09:29:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.672+0000 7fbd82000640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] conn(0x7fbd68075ec0 0x7fbd68078360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.672+0000 7fbd82000640 1 -- 172.21.15.67:0/4115774150 --> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd84046ec0 con 0x7fbd68075ec0 2024-03-09T09:29:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.672+0000 7fbd82000640 1 -- 172.21.15.67:0/4115774150 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd840b2050 con 0x7fbd8c14ed10 2024-03-09T09:29:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.672+0000 7fbd89e00640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbd6806b4d0 0x7fbd6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.672+0000 7fbd89400640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] conn(0x7fbd68075ec0 0x7fbd68078360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.673+0000 7fbd89400640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] conn(0x7fbd68075ec0 0x7fbd68078360 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.673+0000 7fbd89e00640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbd6806b4d0 0x7fbd6806d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c001370 tx=0x7fbd7c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.674+0000 7fbd82000640 1 -- 172.21.15.67:0/4115774150 <== osd.2 v2:172.21.15.67:6800/74956722 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbd84046ec0 con 0x7fbd68075ec0 2024-03-09T09:29:23.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.689+0000 7fbd8b200640 1 -- 172.21.15.67:0/4115774150 --> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbd8c0aa4b0 con 0x7fbd68075ec0 2024-03-09T09:29:23.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.689+0000 7fbd82000640 1 -- 172.21.15.67:0/4115774150 <== osd.2 v2:172.21.15.67:6800/74956722 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbd8c0aa4b0 con 0x7fbd68075ec0 2024-03-09T09:29:23.697 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:29:23.697 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-09T09:29:23.697 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:29:23.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.689+0000 7fbd8b200640 1 -- 172.21.15.67:0/4115774150 >> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] conn(0x7fbd68075ec0 msgr2=0x7fbd68078360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.689+0000 7fbd8b200640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.67:6800/74956722,v1:172.21.15.67:6801/74956722] conn(0x7fbd68075ec0 0x7fbd68078360 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.690+0000 7fbd8b200640 1 -- 172.21.15.67:0/4115774150 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbd6806b4d0 msgr2=0x7fbd6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.690+0000 7fbd8b200640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fbd6806b4d0 0x7fbd6806d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c001370 tx=0x7fbd7c009040 comp rx=0 tx=0).stop 2024-03-09T09:29:23.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.690+0000 7fbd8b200640 1 -- 172.21.15.67:0/4115774150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd8c14ed10 msgr2=0x7fbd8c14f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.690+0000 7fbd8b200640 1 --2- 172.21.15.67:0/4115774150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbd8c14ed10 0x7fbd8c14f190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009b80 tx=0x7fbd84004730 comp rx=0 tx=0).stop 2024-03-09T09:29:23.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.690+0000 7fbd8a800640 1 -- 172.21.15.67:0/4115774150 reap_dead start 2024-03-09T09:29:23.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.690+0000 7fbd8b200640 1 -- 172.21.15.67:0/4115774150 shutdown_connections 2024-03-09T09:29:23.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.690+0000 7fbd8b200640 1 -- 172.21.15.67:0/4115774150 >> 172.21.15.67:0/4115774150 conn(0x7fbd8c0c3d90 msgr2=0x7fbd8c14a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:23.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.690+0000 7fbd8b200640 1 -- 172.21.15.67:0/4115774150 shutdown_connections 2024-03-09T09:29:23.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.690+0000 7fbd8b200640 1 -- 172.21.15.67:0/4115774150 wait complete. 2024-03-09T09:29:23.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-09T09:29:23.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.920+0000 7f4010c00640 1 Processor -- start 2024-03-09T09:29:23.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.920+0000 7f4010c00640 1 -- start start 2024-03-09T09:29:23.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.922+0000 7f4010c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f400c0734a0 0x7f400c0aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.922+0000 7f4010c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f400c06d830 0x7f400c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.922+0000 7f4010c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400c152e00 con 0x7f400c0730c0 2024-03-09T09:29:23.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.922+0000 7f4010c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400c0aa960 con 0x7f400c0734a0 2024-03-09T09:29:23.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.922+0000 7f4010c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400c1530e0 con 0x7f400c06d830 2024-03-09T09:29:23.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.922+0000 7f4003e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f400c0734a0 0x7f400c0aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.922+0000 7f4003e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f400c0734a0 0x7f400c0aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45140/0 (socket says 172.21.15.67:45140) 2024-03-09T09:29:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.922+0000 7f4003e00640 1 -- 172.21.15.67:0/2190176261 learned_addr learned my addr 172.21.15.67:0/2190176261 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.923+0000 7f400a800640 1 --2- 172.21.15.67:0/2190176261 >> v2:172.21.15.134:3300/0 conn(0x7f400c06d830 0x7f400c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.923+0000 7f4003e00640 1 -- 172.21.15.67:0/2190176261 >> v2:172.21.15.134:3300/0 conn(0x7f400c06d830 msgr2=0x7f400c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.923+0000 7f4003e00640 1 --2- 172.21.15.67:0/2190176261 >> v2:172.21.15.134:3300/0 conn(0x7f400c06d830 0x7f400c06dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.923+0000 7f4003e00640 1 -- 172.21.15.67:0/2190176261 >> v1:172.21.15.134:6789/0 conn(0x7f400c0730c0 legacy=0x7f400c0c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.923+0000 7f4003e00640 1 -- 172.21.15.67:0/2190176261 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c1533c0 con 0x7f400c0734a0 2024-03-09T09:29:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.924+0000 7f4003e00640 1 --2- 172.21.15.67:0/2190176261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c0734a0 0x7f400c0aa420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f400c0ac8a0 tx=0x7f3ff402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769ee0ff97118a46 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:23.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.924+0000 7f4009400640 1 -- 172.21.15.67:0/2190176261 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff402e070 con 0x7f400c0734a0 2024-03-09T09:29:23.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.924+0000 7f4009400640 1 -- 172.21.15.67:0/2190176261 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff4037070 con 0x7f400c0734a0 2024-03-09T09:29:23.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.924+0000 7f4009400640 1 -- 172.21.15.67:0/2190176261 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4031a00 con 0x7f400c0734a0 2024-03-09T09:29:23.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.925+0000 7f4010c00640 1 -- 172.21.15.67:0/2190176261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c0734a0 msgr2=0x7f400c0aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.925+0000 7f4010c00640 1 --2- 172.21.15.67:0/2190176261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c0734a0 0x7f400c0aa420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f400c0ac8a0 tx=0x7f3ff402b880 comp rx=0 tx=0).stop 2024-03-09T09:29:23.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.925+0000 7f4010c00640 1 -- 172.21.15.67:0/2190176261 shutdown_connections 2024-03-09T09:29:23.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.925+0000 7f4010c00640 1 --2- 172.21.15.67:0/2190176261 >> v2:172.21.15.134:3300/0 conn(0x7f400c06d830 0x7f400c06dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.925+0000 7f4010c00640 1 --2- 172.21.15.67:0/2190176261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c0734a0 0x7f400c0aa420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.925+0000 7f4010c00640 1 -- 172.21.15.67:0/2190176261 >> 172.21.15.67:0/2190176261 conn(0x7f400c0c7370 msgr2=0x7f400c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.925+0000 7f4010c00640 1 -- 172.21.15.67:0/2190176261 shutdown_connections 2024-03-09T09:29:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.925+0000 7f4010c00640 1 -- 172.21.15.67:0/2190176261 wait complete. 2024-03-09T09:29:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.927+0000 7f4010c00640 1 Processor -- start 2024-03-09T09:29:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.927+0000 7f4010c00640 1 -- start start 2024-03-09T09:29:23.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f4010c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400c06d830 0x7f400c1ee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f4010c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0730c0 0x7f400c151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f4010c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c0734a0 0x7f400c1dbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f4010c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c0c3590 con 0x7f400c0734a0 2024-03-09T09:29:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f4010c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c153980 con 0x7f400c0730c0 2024-03-09T09:29:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f4010c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400c0c3710 con 0x7f400c06d830 2024-03-09T09:29:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f400a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c0734a0 0x7f400c1dbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f400a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c0734a0 0x7f400c1dbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45144/0 (socket says 172.21.15.67:45144) 2024-03-09T09:29:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f400a800640 1 -- 172.21.15.67:0/193278300 learned_addr learned my addr 172.21.15.67:0/193278300 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f4003e00640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0730c0 0x7f400c151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f4009e00640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400c06d830 0x7f400c1ee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f400a800640 1 -- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400c06d830 msgr2=0x7f400c1ee9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.928+0000 7f400a800640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400c06d830 0x7f400c1ee9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.929+0000 7f400a800640 1 -- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0730c0 msgr2=0x7f400c151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.929+0000 7f400a800640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0730c0 0x7f400c151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.929+0000 7f400a800640 1 -- 172.21.15.67:0/193278300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c155750 con 0x7f400c0734a0 2024-03-09T09:29:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.929+0000 7f400a800640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c0734a0 0x7f400c1dbe50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f400400cda0 tx=0x7f400400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.929+0000 7f4002000640 1 -- 172.21.15.67:0/193278300 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4004003b90 con 0x7f400c0734a0 2024-03-09T09:29:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.929+0000 7f4002000640 1 -- 172.21.15.67:0/193278300 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4004003d30 con 0x7f400c0734a0 2024-03-09T09:29:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.929+0000 7f4002000640 1 -- 172.21.15.67:0/193278300 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4004004240 con 0x7f400c0734a0 2024-03-09T09:29:23.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.929+0000 7f4010c00640 1 -- 172.21.15.67:0/193278300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c155a30 con 0x7f400c0734a0 2024-03-09T09:29:23.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.929+0000 7f4010c00640 1 -- 172.21.15.67:0/193278300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c152460 con 0x7f400c0734a0 2024-03-09T09:29:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.931+0000 7f4010c00640 1 -- 172.21.15.67:0/193278300 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400c075a00 con 0x7f400c0734a0 2024-03-09T09:29:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.931+0000 7f4002000640 1 -- 172.21.15.67:0/193278300 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f40040043e0 con 0x7f400c0734a0 2024-03-09T09:29:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.932+0000 7f4002000640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe806b4d0 0x7f3fe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.933+0000 7f4002000640 1 -- 172.21.15.67:0/193278300 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f400408b150 con 0x7f400c0734a0 2024-03-09T09:29:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.933+0000 7f4009e00640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe806b4d0 0x7f3fe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:23.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.934+0000 7f4009e00640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe806b4d0 0x7f3fe806d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc007fd0 tx=0x7f3ffc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:23.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:23.936+0000 7f4002000640 1 -- 172.21.15.67:0/193278300 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f400401d340 con 0x7f400c0734a0 2024-03-09T09:29:24.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.202+0000 7f4010c00640 1 -- 172.21.15.67:0/193278300 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f400c072470 con 0x7f3fe806b4d0 2024-03-09T09:29:24.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.203+0000 7f4002000640 1 -- 172.21.15.67:0/193278300 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f400c072470 con 0x7f3fe806b4d0 2024-03-09T09:29:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 2 to deep-scrub 2024-03-09T09:29:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 -- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe806b4d0 msgr2=0x7f3fe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe806b4d0 0x7f3fe806d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc007fd0 tx=0x7f3ffc009040 comp rx=0 tx=0).stop 2024-03-09T09:29:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 -- 172.21.15.67:0/193278300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c0734a0 msgr2=0x7f400c1dbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c0734a0 0x7f400c1dbe50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f400400cda0 tx=0x7f400400a660 comp rx=0 tx=0).stop 2024-03-09T09:29:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 -- 172.21.15.67:0/193278300 shutdown_connections 2024-03-09T09:29:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3fe806b4d0 0x7f3fe806d990 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f400c0734a0 0x7f400c1dbe50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f400c0730c0 0x7f400c151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 --2- 172.21.15.67:0/193278300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f400c06d830 0x7f400c1ee9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 -- 172.21.15.67:0/193278300 >> 172.21.15.67:0/193278300 conn(0x7f400c0c7370 msgr2=0x7f400c0aaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 -- 172.21.15.67:0/193278300 shutdown_connections 2024-03-09T09:29:24.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.206+0000 7f4010c00640 1 -- 172.21.15.67:0/193278300 wait complete. 2024-03-09T09:29:24.234 INFO:tasks.ceph:Scrubbing osd.3 2024-03-09T09:29:24.234 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-09T09:29:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.482+0000 7f5aa3200640 1 Processor -- start 2024-03-09T09:29:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.482+0000 7f5aa3200640 1 -- start start 2024-03-09T09:29:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.484+0000 7f5aa3200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a9c14c9c0 0x7f5a9c153230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.484+0000 7f5aa3200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c153770 con 0x7f5a9c0c37d0 2024-03-09T09:29:24.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.484+0000 7f5aa3200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9c155eb0 con 0x7f5a9c153d30 2024-03-09T09:29:24.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.484+0000 7f5aa3200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a9c156ac0 con 0x7f5a9c14c9c0 2024-03-09T09:29:24.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.484+0000 7f5aa1e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a9c14c9c0 0x7f5a9c153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.484+0000 7f5aa1400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f5a9c0c37d0 0x7f5a9c0c2ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36822/0 (socket says 172.21.15.67:36822) 2024-03-09T09:29:24.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.484+0000 7f5aa1400640 1 -- 172.21.15.67:0/4288568151 learned_addr learned my addr 172.21.15.67:0/4288568151 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:24.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.484+0000 7f5aa1e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f5a9c14c9c0 0x7f5a9c153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45152/0 (socket says 172.21.15.67:45152) 2024-03-09T09:29:24.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.485+0000 7f5aa1e00640 1 -- 172.21.15.67:0/4288568151 >> v1:172.21.15.134:6789/0 conn(0x7f5a9c153d30 legacy=0x7f5a9c1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:24.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.485+0000 7f5aa1e00640 1 -- 172.21.15.67:0/4288568151 >> v1:172.21.15.67:6789/0 conn(0x7f5a9c0c37d0 legacy=0x7f5a9c0c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.485+0000 7f5aa1e00640 1 -- 172.21.15.67:0/4288568151 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c156190 con 0x7f5a9c14c9c0 2024-03-09T09:29:24.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.486+0000 7f5aa1e00640 1 --2- 172.21.15.67:0/4288568151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c14c9c0 0x7f5a9c153230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000ba40 tx=0x7f5a9002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67d40d91867363a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:24.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.486+0000 7f5aa0a00640 1 -- 172.21.15.67:0/4288568151 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9003b070 con 0x7f5a9c14c9c0 2024-03-09T09:29:24.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.486+0000 7f5aa0a00640 1 -- 172.21.15.67:0/4288568151 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9002edb0 con 0x7f5a9c14c9c0 2024-03-09T09:29:24.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.486+0000 7f5aa0a00640 1 -- 172.21.15.67:0/4288568151 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90036660 con 0x7f5a9c14c9c0 2024-03-09T09:29:24.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.487+0000 7f5aa3200640 1 -- 172.21.15.67:0/4288568151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c14c9c0 msgr2=0x7f5a9c153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.487+0000 7f5aa3200640 1 --2- 172.21.15.67:0/4288568151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c14c9c0 0x7f5a9c153230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000ba40 tx=0x7f5a9002e000 comp rx=0 tx=0).stop 2024-03-09T09:29:24.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.487+0000 7f5aa3200640 1 -- 172.21.15.67:0/4288568151 shutdown_connections 2024-03-09T09:29:24.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.487+0000 7f5aa3200640 1 --2- 172.21.15.67:0/4288568151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c14c9c0 0x7f5a9c153230 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.487+0000 7f5aa3200640 1 -- 172.21.15.67:0/4288568151 >> 172.21.15.67:0/4288568151 conn(0x7f5a9c0c3d90 msgr2=0x7f5a9c0c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:24.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.487+0000 7f5aa3200640 1 -- 172.21.15.67:0/4288568151 shutdown_connections 2024-03-09T09:29:24.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.487+0000 7f5aa3200640 1 -- 172.21.15.67:0/4288568151 wait complete. 2024-03-09T09:29:24.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.488+0000 7f5aa3200640 1 Processor -- start 2024-03-09T09:29:24.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.489+0000 7f5aa3200640 1 -- start start 2024-03-09T09:29:24.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.489+0000 7f5aa3200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a9c0c37d0 0x7f5a9c0c0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa3200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c14c9c0 0x7f5a9c1de550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa3200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c153d30 0x7f5a9c14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa3200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c156750 con 0x7f5a9c153d30 2024-03-09T09:29:24.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa3200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c070860 con 0x7f5a9c14c9c0 2024-03-09T09:29:24.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa3200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c0c33b0 con 0x7f5a9c0c37d0 2024-03-09T09:29:24.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c153d30 0x7f5a9c14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c153d30 0x7f5a9c14f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45156/0 (socket says 172.21.15.67:45156) 2024-03-09T09:29:24.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa2800640 1 -- 172.21.15.67:0/959646739 learned_addr learned my addr 172.21.15.67:0/959646739 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:24.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa1400640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c14c9c0 0x7f5a9c1de550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa1e00640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a9c0c37d0 0x7f5a9c0c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa2800640 1 -- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a9c0c37d0 msgr2=0x7f5a9c0c0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa2800640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a9c0c37d0 0x7f5a9c0c0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa2800640 1 -- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c14c9c0 msgr2=0x7f5a9c1de550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa2800640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a9c14c9c0 0x7f5a9c1de550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.490+0000 7f5aa2800640 1 -- 172.21.15.67:0/959646739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c1ef410 con 0x7f5a9c153d30 2024-03-09T09:29:24.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.491+0000 7f5aa1e00640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a9c0c37d0 0x7f5a9c0c0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.491+0000 7f5aa2800640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c153d30 0x7f5a9c14f210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800b9a0 tx=0x7f5a9800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.491+0000 7f5a96000640 1 -- 172.21.15.67:0/959646739 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98019070 con 0x7f5a9c153d30 2024-03-09T09:29:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.491+0000 7f5a96000640 1 -- 172.21.15.67:0/959646739 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9800cba0 con 0x7f5a9c153d30 2024-03-09T09:29:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.491+0000 7f5aa3200640 1 -- 172.21.15.67:0/959646739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c1ef6f0 con 0x7f5a9c153d30 2024-03-09T09:29:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.491+0000 7f5a96000640 1 -- 172.21.15.67:0/959646739 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98014440 con 0x7f5a9c153d30 2024-03-09T09:29:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.491+0000 7f5aa3200640 1 -- 172.21.15.67:0/959646739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c1efcb0 con 0x7f5a9c153d30 2024-03-09T09:29:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.492+0000 7f5aa3200640 1 -- 172.21.15.67:0/959646739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a9c04f430 con 0x7f5a9c153d30 2024-03-09T09:29:24.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.494+0000 7f5a96000640 1 -- 172.21.15.67:0/959646739 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5a9801c070 con 0x7f5a9c153d30 2024-03-09T09:29:24.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.494+0000 7f5a96000640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5a8006b500 0x7f5a8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.495+0000 7f5a96000640 1 -- 172.21.15.67:0/959646739 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f5a9808a4b0 con 0x7f5a9c153d30 2024-03-09T09:29:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.495+0000 7f5a96000640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] conn(0x7f5a80075f00 0x7f5a800783a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.495+0000 7f5a96000640 1 -- 172.21.15.67:0/959646739 --> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a80078a70 con 0x7f5a80075f00 2024-03-09T09:29:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.495+0000 7f5a96000640 1 -- 172.21.15.67:0/959646739 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a9808a940 con 0x7f5a9c153d30 2024-03-09T09:29:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.495+0000 7f5aa1e00640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5a8006b500 0x7f5a8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.495+0000 7f5aa1400640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] conn(0x7f5a80075f00 0x7f5a800783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.496+0000 7f5aa1400640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] conn(0x7f5a80075f00 0x7f5a800783a0 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:24.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.496+0000 7f5aa1e00640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5a8006b500 0x7f5a8006d9c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5a90038000 tx=0x7f5a90032000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:24.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.497+0000 7f5a96000640 1 -- 172.21.15.67:0/959646739 <== osd.3 v2:172.21.15.67:6824/1424647284 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5a80078a70 con 0x7f5a80075f00 2024-03-09T09:29:24.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.512+0000 7f5aa3200640 1 -- 172.21.15.67:0/959646739 --> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5a9c0739b0 con 0x7f5a80075f00 2024-03-09T09:29:24.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.513+0000 7f5a96000640 1 -- 172.21.15.67:0/959646739 <== osd.3 v2:172.21.15.67:6824/1424647284 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5a9c0739b0 con 0x7f5a80075f00 2024-03-09T09:29:24.521 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:29:24.521 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-09T09:29:24.521 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:29:24.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.513+0000 7f5aa3200640 1 -- 172.21.15.67:0/959646739 >> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] conn(0x7f5a80075f00 msgr2=0x7f5a800783a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.513+0000 7f5aa3200640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.67:6824/1424647284,v1:172.21.15.67:6825/1424647284] conn(0x7f5a80075f00 0x7f5a800783a0 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.514+0000 7f5aa3200640 1 -- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5a8006b500 msgr2=0x7f5a8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.514+0000 7f5aa3200640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f5a8006b500 0x7f5a8006d9c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5a90038000 tx=0x7f5a90032000 comp rx=0 tx=0).stop 2024-03-09T09:29:24.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.514+0000 7f5aa3200640 1 -- 172.21.15.67:0/959646739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c153d30 msgr2=0x7f5a9c14f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.514+0000 7f5aa3200640 1 --2- 172.21.15.67:0/959646739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a9c153d30 0x7f5a9c14f210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800b9a0 tx=0x7f5a9800be60 comp rx=0 tx=0).stop 2024-03-09T09:29:24.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.514+0000 7f5aa2800640 1 -- 172.21.15.67:0/959646739 reap_dead start 2024-03-09T09:29:24.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.514+0000 7f5aa3200640 1 -- 172.21.15.67:0/959646739 shutdown_connections 2024-03-09T09:29:24.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.514+0000 7f5aa3200640 1 -- 172.21.15.67:0/959646739 >> 172.21.15.67:0/959646739 conn(0x7f5a9c0c3d90 msgr2=0x7f5a9c155250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:24.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.514+0000 7f5aa3200640 1 -- 172.21.15.67:0/959646739 shutdown_connections 2024-03-09T09:29:24.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.514+0000 7f5aa3200640 1 -- 172.21.15.67:0/959646739 wait complete. 2024-03-09T09:29:24.541 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-09T09:29:24.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.739+0000 7f1229e00640 1 Processor -- start 2024-03-09T09:29:24.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.739+0000 7f1229e00640 1 -- start start 2024-03-09T09:29:24.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.740+0000 7f1229e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f12240c9380 0x7f12240acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.740+0000 7f1229e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f12240ad440 0x7f122406c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.741+0000 7f1229e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12240c9890 con 0x7f122414fd90 2024-03-09T09:29:24.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.741+0000 7f1229e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122406ba60 con 0x7f12240ad440 2024-03-09T09:29:24.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.741+0000 7f1229e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12240c5f80 con 0x7f12240c9380 2024-03-09T09:29:24.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.741+0000 7f1228a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f122414fd90 0x7f1224156680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36824/0 (socket says 172.21.15.67:36824) 2024-03-09T09:29:24.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.741+0000 7f1228a00640 1 -- 172.21.15.67:0/1633797388 learned_addr learned my addr 172.21.15.67:0/1633797388 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.741+0000 7f1223e00640 1 --2- 172.21.15.67:0/1633797388 >> v2:172.21.15.134:3300/0 conn(0x7f12240c9380 0x7f12240acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.741+0000 7f1229400640 1 --2- 172.21.15.67:0/1633797388 >> v2:172.21.15.67:3300/0 conn(0x7f12240ad440 0x7f122406c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.742+0000 7f1229400640 1 -- 172.21.15.67:0/1633797388 >> v2:172.21.15.134:3300/0 conn(0x7f12240c9380 msgr2=0x7f12240acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.742+0000 7f1229400640 1 --2- 172.21.15.67:0/1633797388 >> v2:172.21.15.134:3300/0 conn(0x7f12240c9380 0x7f12240acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.742+0000 7f1229400640 1 -- 172.21.15.67:0/1633797388 >> v1:172.21.15.67:6789/0 conn(0x7f122414fd90 legacy=0x7f1224156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.742+0000 7f1223400640 1 -- 172.21.15.67:0/1633797388 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460301625 0 0) 0x7f12240c9890 con 0x7f122414fd90 2024-03-09T09:29:24.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.742+0000 7f1229400640 1 -- 172.21.15.67:0/1633797388 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12240c6240 con 0x7f12240ad440 2024-03-09T09:29:24.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.743+0000 7f1229400640 1 --2- 172.21.15.67:0/1633797388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12240ad440 0x7f122406c1d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f12240ae040 tx=0x7f121c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05778165bbf4e24 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:24.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.744+0000 7f1223400640 1 -- 172.21.15.67:0/1633797388 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c039070 con 0x7f12240ad440 2024-03-09T09:29:24.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.744+0000 7f1223400640 1 -- 172.21.15.67:0/1633797388 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121c02cb60 con 0x7f12240ad440 2024-03-09T09:29:24.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.744+0000 7f1223400640 1 -- 172.21.15.67:0/1633797388 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c034440 con 0x7f12240ad440 2024-03-09T09:29:24.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.744+0000 7f1229e00640 1 -- 172.21.15.67:0/1633797388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12240ad440 msgr2=0x7f122406c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.744+0000 7f1229e00640 1 --2- 172.21.15.67:0/1633797388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12240ad440 0x7f122406c1d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f12240ae040 tx=0x7f121c02bdb0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.745+0000 7f1229e00640 1 -- 172.21.15.67:0/1633797388 shutdown_connections 2024-03-09T09:29:24.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.745+0000 7f1229e00640 1 --2- 172.21.15.67:0/1633797388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12240ad440 0x7f122406c1d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.745+0000 7f1229e00640 1 --2- 172.21.15.67:0/1633797388 >> v2:172.21.15.134:3300/0 conn(0x7f12240c9380 0x7f12240acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.745+0000 7f1229e00640 1 -- 172.21.15.67:0/1633797388 >> 172.21.15.67:0/1633797388 conn(0x7f12240cb480 msgr2=0x7f12240cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:24.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.745+0000 7f1229e00640 1 -- 172.21.15.67:0/1633797388 shutdown_connections 2024-03-09T09:29:24.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.745+0000 7f1229e00640 1 -- 172.21.15.67:0/1633797388 wait complete. 2024-03-09T09:29:24.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.746+0000 7f1229e00640 1 Processor -- start 2024-03-09T09:29:24.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.747+0000 7f1229e00640 1 -- start start 2024-03-09T09:29:24.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.747+0000 7f1229e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12240ad440 0x7f12241ef300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.747+0000 7f1229e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12240c9380 0x7f12240aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.747+0000 7f1229e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f122414fd90 0x7f12240af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.747+0000 7f1229e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122406c710 con 0x7f12240ad440 2024-03-09T09:29:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.747+0000 7f1229e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122406bcb0 con 0x7f12240c9380 2024-03-09T09:29:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.747+0000 7f1229e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122406c890 con 0x7f122414fd90 2024-03-09T09:29:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.748+0000 7f1228a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12240ad440 0x7f12241ef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.748+0000 7f1228a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12240ad440 0x7f12241ef300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45164/0 (socket says 172.21.15.67:45164) 2024-03-09T09:29:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.748+0000 7f1228a00640 1 -- 172.21.15.67:0/938958520 learned_addr learned my addr 172.21.15.67:0/938958520 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.748+0000 7f1223e00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12240c9380 0x7f12240aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.748+0000 7f1229400640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f122414fd90 0x7f12240af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.748+0000 7f1228a00640 1 -- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f122414fd90 msgr2=0x7f12240af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.748+0000 7f1228a00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f122414fd90 0x7f12240af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.748+0000 7f1228a00640 1 -- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12240c9380 msgr2=0x7f12240aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.748+0000 7f1228a00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12240c9380 0x7f12240aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.748+0000 7f1228a00640 1 -- 172.21.15.67:0/938958520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12240c2d00 con 0x7f12240ad440 2024-03-09T09:29:24.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.749+0000 7f1223e00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12240c9380 0x7f12240aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:29:24.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.749+0000 7f1228a00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12240ad440 0x7f12241ef300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f121800c460 tx=0x7f121800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:24.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.749+0000 7f1220c00640 1 -- 172.21.15.67:0/938958520 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218018070 con 0x7f12240ad440 2024-03-09T09:29:24.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.749+0000 7f1220c00640 1 -- 172.21.15.67:0/938958520 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1218007500 con 0x7f12240ad440 2024-03-09T09:29:24.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.749+0000 7f1220c00640 1 -- 172.21.15.67:0/938958520 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218004a20 con 0x7f12240ad440 2024-03-09T09:29:24.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.749+0000 7f1229e00640 1 -- 172.21.15.67:0/938958520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12240b0c00 con 0x7f12240ad440 2024-03-09T09:29:24.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.750+0000 7f1229e00640 1 -- 172.21.15.67:0/938958520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12241efd40 con 0x7f12240ad440 2024-03-09T09:29:24.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.751+0000 7f1229e00640 1 -- 172.21.15.67:0/938958520 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12240c2180 con 0x7f12240ad440 2024-03-09T09:29:24.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.752+0000 7f1220c00640 1 -- 172.21.15.67:0/938958520 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1218004bc0 con 0x7f12240ad440 2024-03-09T09:29:24.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.752+0000 7f1220c00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f120406b4d0 0x7f120406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:24.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.752+0000 7f1220c00640 1 -- 172.21.15.67:0/938958520 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f1218089840 con 0x7f12240ad440 2024-03-09T09:29:24.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.753+0000 7f1223e00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f120406b4d0 0x7f120406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:24.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.754+0000 7f1223e00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f120406b4d0 0x7f120406d990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f122414d330 tx=0x7f1210008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:24.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.761+0000 7f1220c00640 1 -- 172.21.15.67:0/938958520 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f12240c2180 con 0x7f12240ad440 2024-03-09T09:29:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.963+0000 7f1229e00640 1 -- 172.21.15.67:0/938958520 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f122414f8d0 con 0x7f120406b4d0 2024-03-09T09:29:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.965+0000 7f1220c00640 1 -- 172.21.15.67:0/938958520 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f122414f8d0 con 0x7f120406b4d0 2024-03-09T09:29:24.973 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 3 to deep-scrub 2024-03-09T09:29:24.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 -- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f120406b4d0 msgr2=0x7f120406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f120406b4d0 0x7f120406d990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f122414d330 tx=0x7f1210008040 comp rx=0 tx=0).stop 2024-03-09T09:29:24.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 -- 172.21.15.67:0/938958520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12240ad440 msgr2=0x7f12241ef300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:24.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12240ad440 0x7f12241ef300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f121800c460 tx=0x7f121800c920 comp rx=0 tx=0).stop 2024-03-09T09:29:24.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 -- 172.21.15.67:0/938958520 shutdown_connections 2024-03-09T09:29:24.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f120406b4d0 0x7f120406d990 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f122414fd90 0x7f12240af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12240c9380 0x7f12240aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 --2- 172.21.15.67:0/938958520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12240ad440 0x7f12241ef300 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 -- 172.21.15.67:0/938958520 >> 172.21.15.67:0/938958520 conn(0x7f12240cb480 msgr2=0x7f1224155500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 -- 172.21.15.67:0/938958520 shutdown_connections 2024-03-09T09:29:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:24.968+0000 7f1229e00640 1 -- 172.21.15.67:0/938958520 wait complete. 2024-03-09T09:29:24.996 INFO:tasks.ceph:Scrubbing osd.4 2024-03-09T09:29:24.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-09T09:29:25.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.231+0000 7f9002e00640 1 Processor -- start 2024-03-09T09:29:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.231+0000 7f9002e00640 1 -- start start 2024-03-09T09:29:25.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.232+0000 7f9002e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ffc0c73d0 0x7f8ffc1547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.232+0000 7f9002e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8ffc0710a0 0x7f8ffc151010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.232+0000 7f9002e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8ffc151550 0x7f8ffc072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.232+0000 7f9002e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ffc06e480 con 0x7f8ffc0710a0 2024-03-09T09:29:25.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.232+0000 7f9002e00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ffc06dff0 con 0x7f8ffc151550 2024-03-09T09:29:25.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.232+0000 7f9002e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc06d6c0 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.232+0000 7f8ff3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8ffc0710a0 0x7f8ffc151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.233+0000 7f8ff3400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8ffc0710a0 0x7f8ffc151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45176/0 (socket says 172.21.15.67:45176) 2024-03-09T09:29:25.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.233+0000 7f8ff3400640 1 -- 172.21.15.67:0/2924744684 learned_addr learned my addr 172.21.15.67:0/2924744684 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:25.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.233+0000 7f8ffbe00640 1 --2- 172.21.15.67:0/2924744684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ffc0c73d0 0x7f8ffc1547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.233+0000 7f9000e00640 1 --2- 172.21.15.67:0/2924744684 >> v2:172.21.15.134:3300/0 conn(0x7f8ffc151550 0x7f8ffc072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.233+0000 7f8ff3400640 1 -- 172.21.15.67:0/2924744684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ffc0c73d0 msgr2=0x7f8ffc1547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:25.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.233+0000 7f8ff3400640 1 --2- 172.21.15.67:0/2924744684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ffc0c73d0 0x7f8ffc1547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.233+0000 7f8ff3400640 1 -- 172.21.15.67:0/2924744684 >> v2:172.21.15.134:3300/0 conn(0x7f8ffc151550 msgr2=0x7f8ffc072cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:25.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.233+0000 7f8ff3400640 1 --2- 172.21.15.67:0/2924744684 >> v2:172.21.15.134:3300/0 conn(0x7f8ffc151550 0x7f8ffc072cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.233+0000 7f8ff3400640 1 -- 172.21.15.67:0/2924744684 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc06d9a0 con 0x7f8ffc0710a0 2024-03-09T09:29:25.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.233+0000 7f8ffbe00640 1 --2- 172.21.15.67:0/2924744684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ffc0c73d0 0x7f8ffc1547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:25.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.233+0000 7f9000e00640 1 --2- 172.21.15.67:0/2924744684 >> v2:172.21.15.134:3300/0 conn(0x7f8ffc151550 0x7f8ffc072cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:25.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.234+0000 7f8ff3400640 1 --2- 172.21.15.67:0/2924744684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ffc0710a0 0x7f8ffc151010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc070b90 tx=0x7f8fe402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2d84365bd284f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:25.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.234+0000 7f8ffb400640 1 -- 172.21.15.67:0/2924744684 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4035020 con 0x7f8ffc0710a0 2024-03-09T09:29:25.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.234+0000 7f8ffb400640 1 -- 172.21.15.67:0/2924744684 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe402d040 con 0x7f8ffc0710a0 2024-03-09T09:29:25.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.234+0000 7f8ffb400640 1 -- 172.21.15.67:0/2924744684 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4031a80 con 0x7f8ffc0710a0 2024-03-09T09:29:25.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.235+0000 7f9002e00640 1 -- 172.21.15.67:0/2924744684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ffc0710a0 msgr2=0x7f8ffc151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.235+0000 7f9002e00640 1 --2- 172.21.15.67:0/2924744684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ffc0710a0 0x7f8ffc151010 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc070b90 tx=0x7f8fe402b900 comp rx=0 tx=0).stop 2024-03-09T09:29:25.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.235+0000 7f9002e00640 1 -- 172.21.15.67:0/2924744684 shutdown_connections 2024-03-09T09:29:25.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.235+0000 7f9002e00640 1 --2- 172.21.15.67:0/2924744684 >> v2:172.21.15.134:3300/0 conn(0x7f8ffc151550 0x7f8ffc072cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.235+0000 7f9002e00640 1 --2- 172.21.15.67:0/2924744684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ffc0710a0 0x7f8ffc151010 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.235+0000 7f9002e00640 1 --2- 172.21.15.67:0/2924744684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ffc0c73d0 0x7f8ffc1547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.235+0000 7f9002e00640 1 -- 172.21.15.67:0/2924744684 >> 172.21.15.67:0/2924744684 conn(0x7f8ffc06b040 msgr2=0x7f8ffc06b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:25.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.235+0000 7f9002e00640 1 -- 172.21.15.67:0/2924744684 shutdown_connections 2024-03-09T09:29:25.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.235+0000 7f9002e00640 1 -- 172.21.15.67:0/2924744684 wait complete. 2024-03-09T09:29:25.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.237+0000 7f9002e00640 1 Processor -- start 2024-03-09T09:29:25.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.237+0000 7f9002e00640 1 -- start start 2024-03-09T09:29:25.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.238+0000 7f9002e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ffc0710a0 0x7f8ffc1dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.238+0000 7f9002e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ffc0c73d0 0x7f8ffc1df080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.238+0000 7f9002e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ffc151550 0x7f8ffc1eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.238+0000 7f9002e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0c77b0 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.238+0000 7f9002e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc071500 con 0x7f8ffc151550 2024-03-09T09:29:25.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.238+0000 7f9002e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0c7930 con 0x7f8ffc0710a0 2024-03-09T09:29:25.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.238+0000 7f8ff3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ffc0c73d0 0x7f8ffc1df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.238+0000 7f8ff3400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ffc0c73d0 0x7f8ffc1df080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45190/0 (socket says 172.21.15.67:45190) 2024-03-09T09:29:25.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.238+0000 7f8ff3400640 1 -- 172.21.15.67:0/2239835437 learned_addr learned my addr 172.21.15.67:0/2239835437 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:25.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.238+0000 7f8ffbe00640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ffc0710a0 0x7f8ffc1dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.238+0000 7f9000e00640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ffc151550 0x7f8ffc1eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.239+0000 7f8ff3400640 1 -- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ffc0710a0 msgr2=0x7f8ffc1dd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.239+0000 7f8ff3400640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ffc0710a0 0x7f8ffc1dd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.239+0000 7f8ff3400640 1 -- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ffc151550 msgr2=0x7f8ffc1eeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.239+0000 7f8ff3400640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ffc151550 0x7f8ffc1eeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.239+0000 7f8ff3400640 1 -- 172.21.15.67:0/2239835437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc1eb580 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.239+0000 7f8ff3400640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ffc0c73d0 0x7f8ffc1df080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4000c00 tx=0x7f8fe40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:25.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.239+0000 7f8ff8c00640 1 -- 172.21.15.67:0/2239835437 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe403ac60 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.239+0000 7f8ff8c00640 1 -- 172.21.15.67:0/2239835437 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe403ae00 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.239+0000 7f9002e00640 1 -- 172.21.15.67:0/2239835437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc1eb840 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.239+0000 7f8ff8c00640 1 -- 172.21.15.67:0/2239835437 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4044630 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.239+0000 7f9002e00640 1 -- 172.21.15.67:0/2239835437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc1ebe00 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.240+0000 7f9002e00640 1 -- 172.21.15.67:0/2239835437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ffc0682c0 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.242+0000 7f8ff8c00640 1 -- 172.21.15.67:0/2239835437 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8fe404b060 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.242+0000 7f8ff8c00640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8fd806b4d0 0x7f8fd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.243+0000 7f8ff8c00640 1 -- 172.21.15.67:0/2239835437 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f8fe40ac5a0 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.243+0000 7f8ff8c00640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] conn(0x7f8fd8075ed0 0x7f8fd8078370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.243+0000 7f8ff8c00640 1 -- 172.21.15.67:0/2239835437 --> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fe402de50 con 0x7f8fd8075ed0 2024-03-09T09:29:25.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.243+0000 7f8ff8c00640 1 -- 172.21.15.67:0/2239835437 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe40b3050 con 0x7f8ffc0c73d0 2024-03-09T09:29:25.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.243+0000 7f8ffbe00640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8fd806b4d0 0x7f8fd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.243+0000 7f9000e00640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] conn(0x7f8fd8075ed0 0x7f8fd8078370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.244+0000 7f8ffbe00640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8fd806b4d0 0x7f8fd806d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00a990 tx=0x7f8fec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:25.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.244+0000 7f9000e00640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] conn(0x7f8fd8075ed0 0x7f8fd8078370 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:25.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.247+0000 7f8ff8c00640 1 -- 172.21.15.67:0/2239835437 <== osd.4 v2:172.21.15.134:6824/294033811 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8fe402de50 con 0x7f8fd8075ed0 2024-03-09T09:29:25.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.265+0000 7f9002e00640 1 -- 172.21.15.67:0/2239835437 --> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8ffc1ec3c0 con 0x7f8fd8075ed0 2024-03-09T09:29:25.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.266+0000 7f8ff8c00640 1 -- 172.21.15.67:0/2239835437 <== osd.4 v2:172.21.15.134:6824/294033811 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8ffc1ec3c0 con 0x7f8fd8075ed0 2024-03-09T09:29:25.273 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:29:25.274 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-09T09:29:25.274 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:29:25.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.266+0000 7f9002e00640 1 -- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] conn(0x7f8fd8075ed0 msgr2=0x7f8fd8078370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.266+0000 7f9002e00640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:6824/294033811,v1:172.21.15.134:6825/294033811] conn(0x7f8fd8075ed0 0x7f8fd8078370 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.266+0000 7f9002e00640 1 -- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8fd806b4d0 msgr2=0x7f8fd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.266+0000 7f9002e00640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8fd806b4d0 0x7f8fd806d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00a990 tx=0x7f8fec008040 comp rx=0 tx=0).stop 2024-03-09T09:29:25.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.266+0000 7f9002e00640 1 -- 172.21.15.67:0/2239835437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ffc0c73d0 msgr2=0x7f8ffc1df080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.266+0000 7f9002e00640 1 --2- 172.21.15.67:0/2239835437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ffc0c73d0 0x7f8ffc1df080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4000c00 tx=0x7f8fe40046c0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.266+0000 7f9000e00640 1 -- 172.21.15.67:0/2239835437 reap_dead start 2024-03-09T09:29:25.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.267+0000 7f9002e00640 1 -- 172.21.15.67:0/2239835437 shutdown_connections 2024-03-09T09:29:25.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.267+0000 7f9002e00640 1 -- 172.21.15.67:0/2239835437 >> 172.21.15.67:0/2239835437 conn(0x7f8ffc06b040 msgr2=0x7f8ffc06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:25.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.267+0000 7f9002e00640 1 -- 172.21.15.67:0/2239835437 shutdown_connections 2024-03-09T09:29:25.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.267+0000 7f9002e00640 1 -- 172.21.15.67:0/2239835437 wait complete. 2024-03-09T09:29:25.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-09T09:29:25.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.483+0000 7f55f5a00640 1 Processor -- start 2024-03-09T09:29:25.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.483+0000 7f55f5a00640 1 -- start start 2024-03-09T09:29:25.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.484+0000 7f55f5a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f55f0070080 0x7f55f0157b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.484+0000 7f55f5a00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f0072d90 con 0x7f55f00c5100 2024-03-09T09:29:25.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.484+0000 7f55f5a00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f0073070 con 0x7f55f0071140 2024-03-09T09:29:25.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.484+0000 7f55f5a00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f0158040 con 0x7f55f0070080 2024-03-09T09:29:25.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.485+0000 7f55ee000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f55f0070080 0x7f55f0157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.485+0000 7f55ee000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f55f0070080 0x7f55f0157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45204/0 (socket says 172.21.15.67:45204) 2024-03-09T09:29:25.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.485+0000 7f55ee000640 1 -- 172.21.15.67:0/2834390639 learned_addr learned my addr 172.21.15.67:0/2834390639 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:25.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.485+0000 7f55ef400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f55f00c5100 0x7f55f015e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36834/0 (socket says 172.21.15.67:36834) 2024-03-09T09:29:25.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.485+0000 7f55ee000640 1 -- 172.21.15.67:0/2834390639 >> v1:172.21.15.134:6789/0 conn(0x7f55f0071140 legacy=0x7f55f00c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.485+0000 7f55ee000640 1 -- 172.21.15.67:0/2834390639 >> v1:172.21.15.67:6789/0 conn(0x7f55f00c5100 legacy=0x7f55f015e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.485+0000 7f55ee000640 1 -- 172.21.15.67:0/2834390639 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0073350 con 0x7f55f0070080 2024-03-09T09:29:25.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.485+0000 7f55ed600640 1 -- 172.21.15.67:0/2834390639 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666859542 0 0) 0x7f55f0072d90 con 0x7f55f00c5100 2024-03-09T09:29:25.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.486+0000 7f55ee000640 1 --2- 172.21.15.67:0/2834390639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55f0070080 0x7f55f0157b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009870 tx=0x7f55dc02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9a1b686a94d2e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:25.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.487+0000 7f55ed600640 1 -- 172.21.15.67:0/2834390639 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc02f840 con 0x7f55f0070080 2024-03-09T09:29:25.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.487+0000 7f55ed600640 1 -- 172.21.15.67:0/2834390639 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc02f9e0 con 0x7f55f0070080 2024-03-09T09:29:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.487+0000 7f55ed600640 1 -- 172.21.15.67:0/2834390639 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc02fdb0 con 0x7f55f0070080 2024-03-09T09:29:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.487+0000 7f55f5a00640 1 -- 172.21.15.67:0/2834390639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55f0070080 msgr2=0x7f55f0157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.487+0000 7f55f5a00640 1 --2- 172.21.15.67:0/2834390639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55f0070080 0x7f55f0157b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009870 tx=0x7f55dc02ba20 comp rx=0 tx=0).stop 2024-03-09T09:29:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.487+0000 7f55f5a00640 1 -- 172.21.15.67:0/2834390639 shutdown_connections 2024-03-09T09:29:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.487+0000 7f55f5a00640 1 --2- 172.21.15.67:0/2834390639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55f0070080 0x7f55f0157b00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.487+0000 7f55f5a00640 1 -- 172.21.15.67:0/2834390639 >> 172.21.15.67:0/2834390639 conn(0x7f55f00ab320 msgr2=0x7f55f00ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.488+0000 7f55f5a00640 1 -- 172.21.15.67:0/2834390639 shutdown_connections 2024-03-09T09:29:25.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.488+0000 7f55f5a00640 1 -- 172.21.15.67:0/2834390639 wait complete. 2024-03-09T09:29:25.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.490+0000 7f55f5a00640 1 Processor -- start 2024-03-09T09:29:25.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.490+0000 7f55f5a00640 1 -- start start 2024-03-09T09:29:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.491+0000 7f55f5a00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f0070080 0x7f55f0151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.491+0000 7f55f5a00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55f0071140 0x7f55f0152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.491+0000 7f55f5a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55f00c5100 0x7f55f01556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.491+0000 7f55f5a00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00704e0 con 0x7f55f0071140 2024-03-09T09:29:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.491+0000 7f55f5a00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00c42f0 con 0x7f55f0070080 2024-03-09T09:29:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.491+0000 7f55f5a00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00a8fd0 con 0x7f55f00c5100 2024-03-09T09:29:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.491+0000 7f55ee000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55f0071140 0x7f55f0152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.492+0000 7f55ee000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55f0071140 0x7f55f0152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45206/0 (socket says 172.21.15.67:45206) 2024-03-09T09:29:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.492+0000 7f55ee000640 1 -- 172.21.15.67:0/1592069526 learned_addr learned my addr 172.21.15.67:0/1592069526 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.492+0000 7f55eea00640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f0070080 0x7f55f0151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.492+0000 7f55ef400640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55f00c5100 0x7f55f01556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.492+0000 7f55ee000640 1 -- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55f00c5100 msgr2=0x7f55f01556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.492+0000 7f55ee000640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55f00c5100 0x7f55f01556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.492+0000 7f55ee000640 1 -- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f0070080 msgr2=0x7f55f0151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.492+0000 7f55ee000640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f0070080 0x7f55f0151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.492+0000 7f55ee000640 1 -- 172.21.15.67:0/1592069526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00af480 con 0x7f55f0071140 2024-03-09T09:29:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.493+0000 7f55ee000640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55f0071140 0x7f55f0152490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009870 tx=0x7f55dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.493+0000 7f55e6a00640 1 -- 172.21.15.67:0/1592069526 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc047070 con 0x7f55f0071140 2024-03-09T09:29:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.493+0000 7f55e6a00640 1 -- 172.21.15.67:0/1592069526 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc037cd0 con 0x7f55f0071140 2024-03-09T09:29:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.493+0000 7f55e6a00640 1 -- 172.21.15.67:0/1592069526 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc004270 con 0x7f55f0071140 2024-03-09T09:29:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.493+0000 7f55f5a00640 1 -- 172.21.15.67:0/1592069526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f0154550 con 0x7f55f0071140 2024-03-09T09:29:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.494+0000 7f55f5a00640 1 -- 172.21.15.67:0/1592069526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f00af110 con 0x7f55f0071140 2024-03-09T09:29:25.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.495+0000 7f55f5a00640 1 -- 172.21.15.67:0/1592069526 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f006c120 con 0x7f55f0071140 2024-03-09T09:29:25.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.496+0000 7f55e6a00640 1 -- 172.21.15.67:0/1592069526 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f55dc003610 con 0x7f55f0071140 2024-03-09T09:29:25.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.497+0000 7f55e6a00640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f55d006b500 0x7f55d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.498+0000 7f55e6a00640 1 -- 172.21.15.67:0/1592069526 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f55dc0ac6c0 con 0x7f55f0071140 2024-03-09T09:29:25.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.498+0000 7f55eea00640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f55d006b500 0x7f55d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.499+0000 7f55eea00640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f55d006b500 0x7f55d006d9c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f55f0157030 tx=0x7f55e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:25.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.503+0000 7f55e6a00640 1 -- 172.21.15.67:0/1592069526 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f55f006c120 con 0x7f55f0071140 2024-03-09T09:29:25.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.725+0000 7f55f5a00640 1 -- 172.21.15.67:0/1592069526 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f55f0075a00 con 0x7f55d006b500 2024-03-09T09:29:25.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.727+0000 7f55e6a00640 1 -- 172.21.15.67:0/1592069526 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f55f0075a00 con 0x7f55d006b500 2024-03-09T09:29:25.734 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 4 to deep-scrub 2024-03-09T09:29:25.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.729+0000 7f55f5a00640 1 -- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f55d006b500 msgr2=0x7f55d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.729+0000 7f55f5a00640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f55d006b500 0x7f55d006d9c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f55f0157030 tx=0x7f55e0009040 comp rx=0 tx=0).stop 2024-03-09T09:29:25.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.729+0000 7f55f5a00640 1 -- 172.21.15.67:0/1592069526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55f0071140 msgr2=0x7f55f0152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.729+0000 7f55f5a00640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55f0071140 0x7f55f0152490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009870 tx=0x7f55dc004730 comp rx=0 tx=0).stop 2024-03-09T09:29:25.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.730+0000 7f55f5a00640 1 -- 172.21.15.67:0/1592069526 shutdown_connections 2024-03-09T09:29:25.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.730+0000 7f55f5a00640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f55d006b500 0x7f55d006d9c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.730+0000 7f55f5a00640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55f00c5100 0x7f55f01556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.730+0000 7f55f5a00640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55f0071140 0x7f55f0152490 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.730+0000 7f55f5a00640 1 --2- 172.21.15.67:0/1592069526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55f0070080 0x7f55f0151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.730+0000 7f55f5a00640 1 -- 172.21.15.67:0/1592069526 >> 172.21.15.67:0/1592069526 conn(0x7f55f00ab320 msgr2=0x7f55f00aaaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:25.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.730+0000 7f55f5a00640 1 -- 172.21.15.67:0/1592069526 shutdown_connections 2024-03-09T09:29:25.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.730+0000 7f55f5a00640 1 -- 172.21.15.67:0/1592069526 wait complete. 2024-03-09T09:29:25.758 INFO:tasks.ceph:Scrubbing osd.5 2024-03-09T09:29:25.758 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-09T09:29:25.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.972+0000 7f6a5f800640 1 Processor -- start 2024-03-09T09:29:25.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.972+0000 7f6a5f800640 1 -- start start 2024-03-09T09:29:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.974+0000 7f6a5f800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f6a58153510 0x7f6a580701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.974+0000 7f6a5f800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a581568c0 0x7f6a580c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.974+0000 7f6a5f800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a58156d30 con 0x7f6a58073830 2024-03-09T09:29:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.974+0000 7f6a5f800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a580c7ee0 con 0x7f6a58153510 2024-03-09T09:29:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.974+0000 7f6a5f800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a581538f0 con 0x7f6a581568c0 2024-03-09T09:29:25.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.975+0000 7f6a5d800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a581568c0 0x7f6a580c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.975+0000 7f6a5d800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a581568c0 0x7f6a580c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:54532/0 (socket says 172.21.15.67:54532) 2024-03-09T09:29:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.975+0000 7f6a5d800640 1 -- 172.21.15.67:0/2707670116 learned_addr learned my addr 172.21.15.67:0/2707670116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.975+0000 7f6a5ce00640 1 --2- 172.21.15.67:0/2707670116 >> v2:172.21.15.134:3300/0 conn(0x7f6a58153510 0x7f6a580701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.975+0000 7f6a5d800640 1 -- 172.21.15.67:0/2707670116 >> v2:172.21.15.134:3300/0 conn(0x7f6a58153510 msgr2=0x7f6a580701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.975+0000 7f6a5d800640 1 --2- 172.21.15.67:0/2707670116 >> v2:172.21.15.134:3300/0 conn(0x7f6a58153510 0x7f6a580701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.975+0000 7f6a5d800640 1 -- 172.21.15.67:0/2707670116 >> v1:172.21.15.134:6789/0 conn(0x7f6a58073830 legacy=0x7f6a58159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.975+0000 7f6a5d800640 1 -- 172.21.15.67:0/2707670116 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580c7640 con 0x7f6a581568c0 2024-03-09T09:29:25.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.977+0000 7f6a5d800640 1 --2- 172.21.15.67:0/2707670116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a581568c0 0x7f6a580c6670 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009e50 tx=0x7f6a5002be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b92b8d20cfe473b server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.977+0000 7f6a57400640 1 -- 172.21.15.67:0/2707670116 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5002c9e0 con 0x7f6a581568c0 2024-03-09T09:29:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.977+0000 7f6a57400640 1 -- 172.21.15.67:0/2707670116 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5002cb80 con 0x7f6a581568c0 2024-03-09T09:29:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.977+0000 7f6a57400640 1 -- 172.21.15.67:0/2707670116 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a500343f0 con 0x7f6a581568c0 2024-03-09T09:29:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.978+0000 7f6a5f800640 1 -- 172.21.15.67:0/2707670116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a581568c0 msgr2=0x7f6a580c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.978+0000 7f6a5f800640 1 --2- 172.21.15.67:0/2707670116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a581568c0 0x7f6a580c6670 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009e50 tx=0x7f6a5002be40 comp rx=0 tx=0).stop 2024-03-09T09:29:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.978+0000 7f6a5f800640 1 -- 172.21.15.67:0/2707670116 shutdown_connections 2024-03-09T09:29:25.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.978+0000 7f6a5f800640 1 --2- 172.21.15.67:0/2707670116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a581568c0 0x7f6a580c6670 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.978+0000 7f6a5f800640 1 --2- 172.21.15.67:0/2707670116 >> v2:172.21.15.134:3300/0 conn(0x7f6a58153510 0x7f6a580701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.978+0000 7f6a5f800640 1 -- 172.21.15.67:0/2707670116 >> 172.21.15.67:0/2707670116 conn(0x7f6a58067180 msgr2=0x7f6a58067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:25.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.978+0000 7f6a5f800640 1 -- 172.21.15.67:0/2707670116 shutdown_connections 2024-03-09T09:29:25.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.978+0000 7f6a5f800640 1 -- 172.21.15.67:0/2707670116 wait complete. 2024-03-09T09:29:25.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.980+0000 7f6a5f800640 1 Processor -- start 2024-03-09T09:29:25.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.980+0000 7f6a5f800640 1 -- start start 2024-03-09T09:29:25.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.982+0000 7f6a5f800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a58073830 0x7f6a581e2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.982+0000 7f6a5f800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a58153510 0x7f6a581e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.982+0000 7f6a5f800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a581568c0 0x7f6a581550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.982+0000 7f6a5f800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a581596f0 con 0x7f6a581568c0 2024-03-09T09:29:25.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.982+0000 7f6a5f800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58159410 con 0x7f6a58073830 2024-03-09T09:29:25.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.982+0000 7f6a5f800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580c7c00 con 0x7f6a58153510 2024-03-09T09:29:25.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.982+0000 7f6a5d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a581568c0 0x7f6a581550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.982+0000 7f6a5d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a581568c0 0x7f6a581550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45210/0 (socket says 172.21.15.67:45210) 2024-03-09T09:29:25.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.982+0000 7f6a5d800640 1 -- 172.21.15.67:0/3574203417 learned_addr learned my addr 172.21.15.67:0/3574203417 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:25.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.982+0000 7f6a5ce00640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a58073830 0x7f6a581e2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.983+0000 7f6a57e00640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a58153510 0x7f6a581e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.983+0000 7f6a5d800640 1 -- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a58153510 msgr2=0x7f6a581e06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.983+0000 7f6a5d800640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a58153510 0x7f6a581e06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.983+0000 7f6a5d800640 1 -- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a58073830 msgr2=0x7f6a581e2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:25.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.983+0000 7f6a5d800640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a58073830 0x7f6a581e2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:25.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.983+0000 7f6a5d800640 1 -- 172.21.15.67:0/3574203417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a581e21a0 con 0x7f6a581568c0 2024-03-09T09:29:25.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.983+0000 7f6a5d800640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a581568c0 0x7f6a581550a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6a50036040 tx=0x7f6a50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:25.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.984+0000 7f6a54c00640 1 -- 172.21.15.67:0/3574203417 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50043020 con 0x7f6a581568c0 2024-03-09T09:29:25.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.984+0000 7f6a54c00640 1 -- 172.21.15.67:0/3574203417 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5003e450 con 0x7f6a581568c0 2024-03-09T09:29:25.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.984+0000 7f6a54c00640 1 -- 172.21.15.67:0/3574203417 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50009510 con 0x7f6a581568c0 2024-03-09T09:29:25.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.984+0000 7f6a5f800640 1 -- 172.21.15.67:0/3574203417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5814a6f0 con 0x7f6a581568c0 2024-03-09T09:29:25.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.984+0000 7f6a5f800640 1 -- 172.21.15.67:0/3574203417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a581e2460 con 0x7f6a581568c0 2024-03-09T09:29:25.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.985+0000 7f6a5f800640 1 -- 172.21.15.67:0/3574203417 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a5814ac10 con 0x7f6a581568c0 2024-03-09T09:29:25.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.986+0000 7f6a54c00640 1 -- 172.21.15.67:0/3574203417 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6a50046020 con 0x7f6a581568c0 2024-03-09T09:29:25.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.987+0000 7f6a54c00640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a3806b4d0 0x7f6a3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.988+0000 7f6a54c00640 1 -- 172.21.15.67:0/3574203417 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f6a500acb90 con 0x7f6a581568c0 2024-03-09T09:29:25.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.988+0000 7f6a54c00640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] conn(0x7f6a38075ec0 0x7f6a38078360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:25.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.988+0000 7f6a54c00640 1 -- 172.21.15.67:0/3574203417 --> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a5004ee50 con 0x7f6a38075ec0 2024-03-09T09:29:25.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.988+0000 7f6a54c00640 1 -- 172.21.15.67:0/3574203417 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a500af290 con 0x7f6a581568c0 2024-03-09T09:29:25.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.988+0000 7f6a5ce00640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a3806b4d0 0x7f6a3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.989+0000 7f6a57e00640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] conn(0x7f6a38075ec0 0x7f6a38078360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:25.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.989+0000 7f6a5ce00640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a3806b4d0 0x7f6a3806d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009b40 tx=0x7f6a4c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:25.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.990+0000 7f6a57e00640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] conn(0x7f6a38075ec0 0x7f6a38078360 crc :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:25.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:25.992+0000 7f6a54c00640 1 -- 172.21.15.67:0/3574203417 <== osd.5 v2:172.21.15.134:6800/4161897388 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6a5004ee50 con 0x7f6a38075ec0 2024-03-09T09:29:26.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.010+0000 7f6a5f800640 1 -- 172.21.15.67:0/3574203417 --> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6a58075a00 con 0x7f6a38075ec0 2024-03-09T09:29:26.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.011+0000 7f6a54c00640 1 -- 172.21.15.67:0/3574203417 <== osd.5 v2:172.21.15.134:6800/4161897388 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6a58075a00 con 0x7f6a38075ec0 2024-03-09T09:29:26.019 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:29:26.019 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-09T09:29:26.019 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:29:26.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.011+0000 7f6a5f800640 1 -- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] conn(0x7f6a38075ec0 msgr2=0x7f6a38078360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.011+0000 7f6a5f800640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:6800/4161897388,v1:172.21.15.134:6801/4161897388] conn(0x7f6a38075ec0 0x7f6a38078360 crc :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.011+0000 7f6a5f800640 1 -- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a3806b4d0 msgr2=0x7f6a3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.011+0000 7f6a5f800640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f6a3806b4d0 0x7f6a3806d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009b40 tx=0x7f6a4c009190 comp rx=0 tx=0).stop 2024-03-09T09:29:26.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.011+0000 7f6a5f800640 1 -- 172.21.15.67:0/3574203417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a581568c0 msgr2=0x7f6a581550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.011+0000 7f6a5f800640 1 --2- 172.21.15.67:0/3574203417 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a581568c0 0x7f6a581550a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6a50036040 tx=0x7f6a50004a40 comp rx=0 tx=0).stop 2024-03-09T09:29:26.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.011+0000 7f6a5d800640 1 -- 172.21.15.67:0/3574203417 reap_dead start 2024-03-09T09:29:26.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.012+0000 7f6a5f800640 1 -- 172.21.15.67:0/3574203417 shutdown_connections 2024-03-09T09:29:26.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.012+0000 7f6a5f800640 1 -- 172.21.15.67:0/3574203417 >> 172.21.15.67:0/3574203417 conn(0x7f6a58067180 msgr2=0x7f6a580c9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:26.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.012+0000 7f6a5f800640 1 -- 172.21.15.67:0/3574203417 shutdown_connections 2024-03-09T09:29:26.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.012+0000 7f6a5f800640 1 -- 172.21.15.67:0/3574203417 wait complete. 2024-03-09T09:29:26.038 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-09T09:29:26.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.266+0000 7f44b8e00640 1 Processor -- start 2024-03-09T09:29:26.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.267+0000 7f44b8e00640 1 -- start start 2024-03-09T09:29:26.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.268+0000 7f44b8e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f44b40ac9c0 0x7f44b40ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.268+0000 7f44b8e00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b40ac4e0 con 0x7f44b40c1d70 2024-03-09T09:29:26.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.268+0000 7f44b8e00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b40aa550 con 0x7f44b414c9a0 2024-03-09T09:29:26.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.268+0000 7f44b8e00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b406d590 con 0x7f44b40ac9c0 2024-03-09T09:29:26.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.268+0000 7f44b1400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f44b40c1d70 0x7f44b40c07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36842/0 (socket says 172.21.15.67:36842) 2024-03-09T09:29:26.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.268+0000 7f44b1400640 1 -- 172.21.15.67:0/1336830729 learned_addr learned my addr 172.21.15.67:0/1336830729 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:26.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.269+0000 7f44b2800640 1 --2- 172.21.15.67:0/1336830729 >> v2:172.21.15.67:3300/0 conn(0x7f44b40ac9c0 0x7f44b40ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.269+0000 7f44b0a00640 1 -- 172.21.15.67:0/1336830729 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578242631 0 0) 0x7f44b40ac4e0 con 0x7f44b40c1d70 2024-03-09T09:29:26.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.269+0000 7f44b0a00640 1 -- 172.21.15.67:0/1336830729 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4498003520 con 0x7f44b40c1d70 2024-03-09T09:29:26.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.269+0000 7f44b2800640 1 -- 172.21.15.67:0/1336830729 >> v1:172.21.15.134:6789/0 conn(0x7f44b414c9a0 legacy=0x7f44b41567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.269+0000 7f44b2800640 1 -- 172.21.15.67:0/1336830729 >> v1:172.21.15.67:6789/0 conn(0x7f44b40c1d70 legacy=0x7f44b40c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.269+0000 7f44b2800640 1 -- 172.21.15.67:0/1336830729 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b40aa830 con 0x7f44b40ac9c0 2024-03-09T09:29:26.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.271+0000 7f44b2800640 1 --2- 172.21.15.67:0/1336830729 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b40ac9c0 0x7f44b40ace50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009d90 tx=0x7f44ac02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fb6bbb98d06aa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:26.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.272+0000 7f44b0a00640 1 -- 172.21.15.67:0/1336830729 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac039070 con 0x7f44b40ac9c0 2024-03-09T09:29:26.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.272+0000 7f44b0a00640 1 -- 172.21.15.67:0/1336830729 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac02cc70 con 0x7f44b40ac9c0 2024-03-09T09:29:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.272+0000 7f44b0a00640 1 -- 172.21.15.67:0/1336830729 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac034660 con 0x7f44b40ac9c0 2024-03-09T09:29:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.273+0000 7f44b8e00640 1 -- 172.21.15.67:0/1336830729 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b40ac9c0 msgr2=0x7f44b40ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.273+0000 7f44b8e00640 1 --2- 172.21.15.67:0/1336830729 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b40ac9c0 0x7f44b40ace50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009d90 tx=0x7f44ac02bec0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.273+0000 7f44b8e00640 1 -- 172.21.15.67:0/1336830729 shutdown_connections 2024-03-09T09:29:26.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.273+0000 7f44b8e00640 1 --2- 172.21.15.67:0/1336830729 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b40ac9c0 0x7f44b40ace50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.273+0000 7f44b8e00640 1 -- 172.21.15.67:0/1336830729 >> 172.21.15.67:0/1336830729 conn(0x7f44b40c7370 msgr2=0x7f44b40c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:26.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.273+0000 7f44b8e00640 1 -- 172.21.15.67:0/1336830729 shutdown_connections 2024-03-09T09:29:26.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.273+0000 7f44b8e00640 1 -- 172.21.15.67:0/1336830729 wait complete. 2024-03-09T09:29:26.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.275+0000 7f44b8e00640 1 Processor -- start 2024-03-09T09:29:26.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.275+0000 7f44b8e00640 1 -- start start 2024-03-09T09:29:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.276+0000 7f44b8e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b40ac9c0 0x7f44b41ecd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.276+0000 7f44b8e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44b40c1d70 0x7f44b4148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.276+0000 7f44b8e00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44b414c9a0 0x7f44b40727a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.276+0000 7f44b8e00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40c0eb0 con 0x7f44b40ac9c0 2024-03-09T09:29:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.276+0000 7f44b8e00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40aadf0 con 0x7f44b414c9a0 2024-03-09T09:29:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.276+0000 7f44b8e00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40c1030 con 0x7f44b40c1d70 2024-03-09T09:29:26.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.277+0000 7f44b1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b40ac9c0 0x7f44b41ecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.277+0000 7f44b1e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b40ac9c0 0x7f44b41ecd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45238/0 (socket says 172.21.15.67:45238) 2024-03-09T09:29:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.277+0000 7f44b1e00640 1 -- 172.21.15.67:0/2566362156 learned_addr learned my addr 172.21.15.67:0/2566362156 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.277+0000 7f44b1400640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44b40c1d70 0x7f44b4148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.277+0000 7f44b2800640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44b414c9a0 0x7f44b40727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.277+0000 7f44b1e00640 1 -- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44b40c1d70 msgr2=0x7f44b4148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.277+0000 7f44b1e00640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44b40c1d70 0x7f44b4148350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.277+0000 7f44b1e00640 1 -- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44b414c9a0 msgr2=0x7f44b40727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.277+0000 7f44b1e00640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44b414c9a0 0x7f44b40727a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.277+0000 7f44b1e00640 1 -- 172.21.15.67:0/2566362156 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b4073710 con 0x7f44b40ac9c0 2024-03-09T09:29:26.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.278+0000 7f44b1e00640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b40ac9c0 0x7f44b41ecd60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f44a400c430 tx=0x7f44a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:26.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.278+0000 7f44aa000640 1 -- 172.21.15.67:0/2566362156 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4017070 con 0x7f44b40ac9c0 2024-03-09T09:29:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.278+0000 7f44aa000640 1 -- 172.21.15.67:0/2566362156 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a40052d0 con 0x7f44b40ac9c0 2024-03-09T09:29:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.278+0000 7f44aa000640 1 -- 172.21.15.67:0/2566362156 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4003830 con 0x7f44b40ac9c0 2024-03-09T09:29:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.278+0000 7f44b8e00640 1 -- 172.21.15.67:0/2566362156 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b41ef910 con 0x7f44b40ac9c0 2024-03-09T09:29:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.278+0000 7f44b8e00640 1 -- 172.21.15.67:0/2566362156 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b4148890 con 0x7f44b40ac9c0 2024-03-09T09:29:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.279+0000 7f44b8e00640 1 -- 172.21.15.67:0/2566362156 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b4075a00 con 0x7f44b40ac9c0 2024-03-09T09:29:26.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.281+0000 7f44aa000640 1 -- 172.21.15.67:0/2566362156 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f44a401a070 con 0x7f44b40ac9c0 2024-03-09T09:29:26.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.282+0000 7f44aa000640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f449406b4d0 0x7f449406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.283+0000 7f44aa000640 1 -- 172.21.15.67:0/2566362156 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f44a4089860 con 0x7f44b40ac9c0 2024-03-09T09:29:26.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.283+0000 7f44b1400640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f449406b4d0 0x7f449406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.284+0000 7f44b1400640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f449406b4d0 0x7f449406d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f44b4073930 tx=0x7f44a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:26.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.287+0000 7f44aa000640 1 -- 172.21.15.67:0/2566362156 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f44a40583b0 con 0x7f44b40ac9c0 2024-03-09T09:29:26.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.517+0000 7f44b8e00640 1 -- 172.21.15.67:0/2566362156 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f44b415a210 con 0x7f449406b4d0 2024-03-09T09:29:26.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.519+0000 7f44aa000640 1 -- 172.21.15.67:0/2566362156 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f44b415a210 con 0x7f449406b4d0 2024-03-09T09:29:26.526 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 5 to deep-scrub 2024-03-09T09:29:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.521+0000 7f44b8e00640 1 -- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f449406b4d0 msgr2=0x7f449406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.521+0000 7f44b8e00640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f449406b4d0 0x7f449406d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f44b4073930 tx=0x7f44a0009040 comp rx=0 tx=0).stop 2024-03-09T09:29:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.521+0000 7f44b8e00640 1 -- 172.21.15.67:0/2566362156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b40ac9c0 msgr2=0x7f44b41ecd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.521+0000 7f44b8e00640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b40ac9c0 0x7f44b41ecd60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f44a400c430 tx=0x7f44a400c8f0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.522+0000 7f44b8e00640 1 -- 172.21.15.67:0/2566362156 shutdown_connections 2024-03-09T09:29:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.522+0000 7f44b8e00640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f449406b4d0 0x7f449406d990 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.522+0000 7f44b8e00640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44b414c9a0 0x7f44b40727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.522+0000 7f44b8e00640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44b40c1d70 0x7f44b4148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.522+0000 7f44b8e00640 1 --2- 172.21.15.67:0/2566362156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b40ac9c0 0x7f44b41ecd60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.522+0000 7f44b8e00640 1 -- 172.21.15.67:0/2566362156 >> 172.21.15.67:0/2566362156 conn(0x7f44b40c7370 msgr2=0x7f44b40c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.522+0000 7f44b8e00640 1 -- 172.21.15.67:0/2566362156 shutdown_connections 2024-03-09T09:29:26.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.522+0000 7f44b8e00640 1 -- 172.21.15.67:0/2566362156 wait complete. 2024-03-09T09:29:26.551 INFO:tasks.ceph:Scrubbing osd.6 2024-03-09T09:29:26.552 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-09T09:29:26.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.750+0000 7f242be00640 1 Processor -- start 2024-03-09T09:29:26.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.751+0000 7f242be00640 1 -- start start 2024-03-09T09:29:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.752+0000 7f242be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242c153410 0x7f242c0c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.752+0000 7f242be00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f242c0c95d0 0x7f242c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.752+0000 7f242be00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242c06ce30 con 0x7f242c0bec10 2024-03-09T09:29:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.752+0000 7f242be00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242c156b00 con 0x7f242c0c95d0 2024-03-09T09:29:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.752+0000 7f242be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c155c30 con 0x7f242c153410 2024-03-09T09:29:26.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.752+0000 7f242b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f242c0c95d0 0x7f242c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.752+0000 7f242b400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f242c0c95d0 0x7f242c06b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45244/0 (socket says 172.21.15.67:45244) 2024-03-09T09:29:26.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.752+0000 7f242b400640 1 -- 172.21.15.67:0/4261788642 learned_addr learned my addr 172.21.15.67:0/4261788642 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:26.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.753+0000 7f242aa00640 1 --2- 172.21.15.67:0/4261788642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242c153410 0x7f242c0c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.753+0000 7f242b400640 1 -- 172.21.15.67:0/4261788642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242c153410 msgr2=0x7f242c0c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.753+0000 7f242b400640 1 --2- 172.21.15.67:0/4261788642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242c153410 0x7f242c0c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.753+0000 7f242b400640 1 -- 172.21.15.67:0/4261788642 >> v1:172.21.15.134:6789/0 conn(0x7f242c0bec10 legacy=0x7f242c06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.753+0000 7f242b400640 1 -- 172.21.15.67:0/4261788642 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c155ef0 con 0x7f242c0c95d0 2024-03-09T09:29:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.754+0000 7f242b400640 1 --2- 172.21.15.67:0/4261788642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242c0c95d0 0x7f242c06b900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2424009870 tx=0x7f242402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d197725a154e837e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.754+0000 7f2429600640 1 -- 172.21.15.67:0/4261788642 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2424035020 con 0x7f242c0c95d0 2024-03-09T09:29:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.754+0000 7f2429600640 1 -- 172.21.15.67:0/4261788642 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2424031640 con 0x7f242c0c95d0 2024-03-09T09:29:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.754+0000 7f2429600640 1 -- 172.21.15.67:0/4261788642 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242402d040 con 0x7f242c0c95d0 2024-03-09T09:29:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.755+0000 7f242be00640 1 -- 172.21.15.67:0/4261788642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242c0c95d0 msgr2=0x7f242c06b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.755+0000 7f242be00640 1 --2- 172.21.15.67:0/4261788642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242c0c95d0 0x7f242c06b900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2424009870 tx=0x7f242402b8f0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.755+0000 7f242be00640 1 -- 172.21.15.67:0/4261788642 shutdown_connections 2024-03-09T09:29:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.755+0000 7f242be00640 1 --2- 172.21.15.67:0/4261788642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242c0c95d0 0x7f242c06b900 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.755+0000 7f242be00640 1 --2- 172.21.15.67:0/4261788642 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242c153410 0x7f242c0c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.755+0000 7f242be00640 1 -- 172.21.15.67:0/4261788642 >> 172.21.15.67:0/4261788642 conn(0x7f242c0ad0d0 msgr2=0x7f242c0ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.755+0000 7f242be00640 1 -- 172.21.15.67:0/4261788642 shutdown_connections 2024-03-09T09:29:26.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.755+0000 7f242be00640 1 -- 172.21.15.67:0/4261788642 wait complete. 2024-03-09T09:29:26.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.757+0000 7f242be00640 1 Processor -- start 2024-03-09T09:29:26.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.757+0000 7f242be00640 1 -- start start 2024-03-09T09:29:26.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.758+0000 7f242be00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f242c0bec10 0x7f242c14c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.758+0000 7f242be00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242c0c95d0 0x7f242c14fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.758+0000 7f242be00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242c153410 0x7f242c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.758+0000 7f242be00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c156790 con 0x7f242c0c95d0 2024-03-09T09:29:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.758+0000 7f242be00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1564b0 con 0x7f242c0bec10 2024-03-09T09:29:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.758+0000 7f242be00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c06be40 con 0x7f242c153410 2024-03-09T09:29:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.758+0000 7f242a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242c0c95d0 0x7f242c14fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.758+0000 7f242a000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242c0c95d0 0x7f242c14fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45246/0 (socket says 172.21.15.67:45246) 2024-03-09T09:29:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.758+0000 7f242a000640 1 -- 172.21.15.67:0/3647988783 learned_addr learned my addr 172.21.15.67:0/3647988783 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.758+0000 7f242aa00640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f242c0bec10 0x7f242c14c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.759+0000 7f242b400640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242c153410 0x7f242c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.759+0000 7f242a000640 1 -- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242c153410 msgr2=0x7f242c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.759+0000 7f242a000640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f242c153410 0x7f242c158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.759+0000 7f242a000640 1 -- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f242c0bec10 msgr2=0x7f242c14c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.759+0000 7f242a000640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f242c0bec10 0x7f242c14c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.759+0000 7f242a000640 1 -- 172.21.15.67:0/3647988783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c14f2a0 con 0x7f242c0c95d0 2024-03-09T09:29:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.759+0000 7f242a000640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242c0c95d0 0x7f242c14fd40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f241800c440 tx=0x7f2418001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.759+0000 7f2422a00640 1 -- 172.21.15.67:0/3647988783 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24180052d0 con 0x7f242c0c95d0 2024-03-09T09:29:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.759+0000 7f2422a00640 1 -- 172.21.15.67:0/3647988783 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418005470 con 0x7f242c0c95d0 2024-03-09T09:29:26.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.759+0000 7f2422a00640 1 -- 172.21.15.67:0/3647988783 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418007260 con 0x7f242c0c95d0 2024-03-09T09:29:26.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.759+0000 7f242be00640 1 -- 172.21.15.67:0/3647988783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242c14d850 con 0x7f242c0c95d0 2024-03-09T09:29:26.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.760+0000 7f242be00640 1 -- 172.21.15.67:0/3647988783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242c14de10 con 0x7f242c0c95d0 2024-03-09T09:29:26.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.761+0000 7f242be00640 1 -- 172.21.15.67:0/3647988783 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f242c04f430 con 0x7f242c0c95d0 2024-03-09T09:29:26.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.762+0000 7f2422a00640 1 -- 172.21.15.67:0/3647988783 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2418007400 con 0x7f242c0c95d0 2024-03-09T09:29:26.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.763+0000 7f2422a00640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f240c06b4d0 0x7f240c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.763+0000 7f2422a00640 1 -- 172.21.15.67:0/3647988783 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f2418089a70 con 0x7f242c0c95d0 2024-03-09T09:29:26.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.763+0000 7f2422a00640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] conn(0x7f240c075ec0 0x7f240c078360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.763+0000 7f2422a00640 1 -- 172.21.15.67:0/3647988783 --> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2418009e50 con 0x7f240c075ec0 2024-03-09T09:29:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.763+0000 7f2422a00640 1 -- 172.21.15.67:0/3647988783 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f241808e2c0 con 0x7f242c0c95d0 2024-03-09T09:29:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.763+0000 7f242aa00640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f240c06b4d0 0x7f240c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.764+0000 7f242b400640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] conn(0x7f240c075ec0 0x7f240c078360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:26.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.765+0000 7f242aa00640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f240c06b4d0 0x7f240c06d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f241c0099c0 tx=0x7f241c003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:26.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.765+0000 7f242b400640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] conn(0x7f240c075ec0 0x7f240c078360 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:26.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.767+0000 7f2422a00640 1 -- 172.21.15.67:0/3647988783 <== osd.6 v2:172.21.15.134:6808/3579212498 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2418009e50 con 0x7f240c075ec0 2024-03-09T09:29:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.783+0000 7f242be00640 1 -- 172.21.15.67:0/3647988783 --> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f242c06ca80 con 0x7f240c075ec0 2024-03-09T09:29:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.784+0000 7f2422a00640 1 -- 172.21.15.67:0/3647988783 <== osd.6 v2:172.21.15.134:6808/3579212498 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f242c06ca80 con 0x7f240c075ec0 2024-03-09T09:29:26.791 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:29:26.792 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-09T09:29:26.792 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:29:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.784+0000 7f242be00640 1 -- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] conn(0x7f240c075ec0 msgr2=0x7f240c078360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.784+0000 7f242be00640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:6808/3579212498,v1:172.21.15.134:6809/3579212498] conn(0x7f240c075ec0 0x7f240c078360 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.785+0000 7f242be00640 1 -- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f240c06b4d0 msgr2=0x7f240c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.785+0000 7f242be00640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f240c06b4d0 0x7f240c06d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f241c0099c0 tx=0x7f241c003990 comp rx=0 tx=0).stop 2024-03-09T09:29:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.785+0000 7f242be00640 1 -- 172.21.15.67:0/3647988783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242c0c95d0 msgr2=0x7f242c14fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.785+0000 7f242be00640 1 --2- 172.21.15.67:0/3647988783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f242c0c95d0 0x7f242c14fd40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f241800c440 tx=0x7f2418001e70 comp rx=0 tx=0).stop 2024-03-09T09:29:26.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.785+0000 7f242b400640 1 -- 172.21.15.67:0/3647988783 reap_dead start 2024-03-09T09:29:26.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.785+0000 7f242be00640 1 -- 172.21.15.67:0/3647988783 shutdown_connections 2024-03-09T09:29:26.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.785+0000 7f242be00640 1 -- 172.21.15.67:0/3647988783 >> 172.21.15.67:0/3647988783 conn(0x7f242c0ad0d0 msgr2=0x7f242c0c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:26.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.785+0000 7f242be00640 1 -- 172.21.15.67:0/3647988783 shutdown_connections 2024-03-09T09:29:26.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:26.785+0000 7f242be00640 1 -- 172.21.15.67:0/3647988783 wait complete. 2024-03-09T09:29:26.817 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-09T09:29:27.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.041+0000 7f4d1a800640 1 Processor -- start 2024-03-09T09:29:27.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.041+0000 7f4d1a800640 1 -- start start 2024-03-09T09:29:27.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.042+0000 7f4d1a800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4d1406dbe0 0x7f4d140aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.042+0000 7f4d1a800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4d141637c0 0x7f4d140ab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.042+0000 7f4d1a800640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d14162d20 con 0x7f4d140aa960 2024-03-09T09:29:27.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.042+0000 7f4d1a800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d140c3cf0 con 0x7f4d141637c0 2024-03-09T09:29:27.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.042+0000 7f4d1a800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d140c3a10 con 0x7f4d1406dbe0 2024-03-09T09:29:27.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.042+0000 7f4d12a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4d141637c0 0x7f4d140ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.042+0000 7f4d12a00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4d141637c0 0x7f4d140ab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45250/0 (socket says 172.21.15.67:45250) 2024-03-09T09:29:27.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.042+0000 7f4d13e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f4d140aa960 0x7f4d140728a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36854/0 (socket says 172.21.15.67:36854) 2024-03-09T09:29:27.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.042+0000 7f4d12a00640 1 -- 172.21.15.67:0/4126990388 learned_addr learned my addr 172.21.15.67:0/4126990388 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:27.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.043+0000 7f4d13400640 1 --2- 172.21.15.67:0/4126990388 >> v2:172.21.15.134:3300/0 conn(0x7f4d1406dbe0 0x7f4d140aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.043+0000 7f4d12000640 1 -- 172.21.15.67:0/4126990388 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1372292343 0 0) 0x7f4d14162d20 con 0x7f4d140aa960 2024-03-09T09:29:27.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.088+0000 7f4d12000640 1 -- 172.21.15.67:0/4126990388 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cf8003520 con 0x7f4d140aa960 2024-03-09T09:29:27.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.088+0000 7f4d12a00640 1 -- 172.21.15.67:0/4126990388 >> v2:172.21.15.134:3300/0 conn(0x7f4d1406dbe0 msgr2=0x7f4d140aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:27.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.088+0000 7f4d12a00640 1 --2- 172.21.15.67:0/4126990388 >> v2:172.21.15.134:3300/0 conn(0x7f4d1406dbe0 0x7f4d140aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.088+0000 7f4d12a00640 1 -- 172.21.15.67:0/4126990388 >> v1:172.21.15.67:6789/0 conn(0x7f4d140aa960 legacy=0x7f4d140728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.088+0000 7f4d12a00640 1 -- 172.21.15.67:0/4126990388 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14162fc0 con 0x7f4d141637c0 2024-03-09T09:29:27.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.091+0000 7f4d12a00640 1 --2- 172.21.15.67:0/4126990388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d141637c0 0x7f4d140ab840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4d140c83b0 tx=0x7f4d0002cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a84c9fe1dc5e1e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:27.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.091+0000 7f4d12000640 1 -- 172.21.15.67:0/4126990388 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0002e900 con 0x7f4d141637c0 2024-03-09T09:29:27.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.092+0000 7f4d12000640 1 -- 172.21.15.67:0/4126990388 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0000ae50 con 0x7f4d141637c0 2024-03-09T09:29:27.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.092+0000 7f4d12000640 1 -- 172.21.15.67:0/4126990388 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0002eca0 con 0x7f4d141637c0 2024-03-09T09:29:27.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.093+0000 7f4d1a800640 1 -- 172.21.15.67:0/4126990388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d141637c0 msgr2=0x7f4d140ab840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.093+0000 7f4d1a800640 1 --2- 172.21.15.67:0/4126990388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d141637c0 0x7f4d140ab840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4d140c83b0 tx=0x7f4d0002cb40 comp rx=0 tx=0).stop 2024-03-09T09:29:27.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.093+0000 7f4d1a800640 1 -- 172.21.15.67:0/4126990388 shutdown_connections 2024-03-09T09:29:27.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.093+0000 7f4d1a800640 1 --2- 172.21.15.67:0/4126990388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d141637c0 0x7f4d140ab840 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.093+0000 7f4d1a800640 1 --2- 172.21.15.67:0/4126990388 >> v2:172.21.15.134:3300/0 conn(0x7f4d1406dbe0 0x7f4d140aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.093+0000 7f4d1a800640 1 -- 172.21.15.67:0/4126990388 >> 172.21.15.67:0/4126990388 conn(0x7f4d140c7370 msgr2=0x7f4d140c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:27.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.093+0000 7f4d1a800640 1 -- 172.21.15.67:0/4126990388 shutdown_connections 2024-03-09T09:29:27.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.093+0000 7f4d1a800640 1 -- 172.21.15.67:0/4126990388 wait complete. 2024-03-09T09:29:27.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.096+0000 7f4d1a800640 1 Processor -- start 2024-03-09T09:29:27.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.096+0000 7f4d1a800640 1 -- start start 2024-03-09T09:29:27.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.098+0000 7f4d1a800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d1406dbe0 0x7f4d141ed460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.098+0000 7f4d1a800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d140aa960 0x7f4d141ec300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.098+0000 7f4d1a800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d141637c0 0x7f4d141eeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.098+0000 7f4d1a800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d14073e20 con 0x7f4d141637c0 2024-03-09T09:29:27.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.098+0000 7f4d1a800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d14073ca0 con 0x7f4d140aa960 2024-03-09T09:29:27.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.098+0000 7f4d1a800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1406dfc0 con 0x7f4d1406dbe0 2024-03-09T09:29:27.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.098+0000 7f4d13e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d141637c0 0x7f4d141eeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.098+0000 7f4d13e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d141637c0 0x7f4d141eeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45262/0 (socket says 172.21.15.67:45262) 2024-03-09T09:29:27.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.098+0000 7f4d13e00640 1 -- 172.21.15.67:0/3861441687 learned_addr learned my addr 172.21.15.67:0/3861441687 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:27.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.098+0000 7f4d13400640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d1406dbe0 0x7f4d141ed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.099+0000 7f4d13e00640 1 -- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d1406dbe0 msgr2=0x7f4d141ed460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.099+0000 7f4d13e00640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d1406dbe0 0x7f4d141ed460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.099+0000 7f4d13e00640 1 -- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d140aa960 msgr2=0x7f4d141ec300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.099+0000 7f4d13e00640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d140aa960 0x7f4d141ec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.099+0000 7f4d13e00640 1 -- 172.21.15.67:0/3861441687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d141f92b0 con 0x7f4d141637c0 2024-03-09T09:29:27.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.100+0000 7f4d13e00640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d141637c0 0x7f4d141eeca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00c7e0 tx=0x7f4d0c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.100+0000 7f4d0b400640 1 -- 172.21.15.67:0/3861441687 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00a640 con 0x7f4d141637c0 2024-03-09T09:29:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.100+0000 7f4d0b400640 1 -- 172.21.15.67:0/3861441687 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c015030 con 0x7f4d141637c0 2024-03-09T09:29:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.100+0000 7f4d0b400640 1 -- 172.21.15.67:0/3861441687 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00a9e0 con 0x7f4d141637c0 2024-03-09T09:29:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.101+0000 7f4d1a800640 1 -- 172.21.15.67:0/3861441687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d141f9550 con 0x7f4d141637c0 2024-03-09T09:29:27.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.101+0000 7f4d1a800640 1 -- 172.21.15.67:0/3861441687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d140c7e10 con 0x7f4d141637c0 2024-03-09T09:29:27.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.104+0000 7f4d1a800640 1 -- 172.21.15.67:0/3861441687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d14075a00 con 0x7f4d141637c0 2024-03-09T09:29:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.105+0000 7f4d0b400640 1 -- 172.21.15.67:0/3861441687 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4d0c00ab80 con 0x7f4d141637c0 2024-03-09T09:29:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.106+0000 7f4d0b400640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4cf406b520 0x7f4cf406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.106+0000 7f4d0b400640 1 -- 172.21.15.67:0/3861441687 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f4d0c08e360 con 0x7f4d141637c0 2024-03-09T09:29:27.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.108+0000 7f4d13400640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4cf406b520 0x7f4cf406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.109+0000 7f4d13400640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4cf406b520 0x7f4cf406d9e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4d04006fd0 tx=0x7f4d04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:27.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.114+0000 7f4d0b400640 1 -- 172.21.15.67:0/3861441687 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4d0c05c7b0 con 0x7f4d141637c0 2024-03-09T09:29:27.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.441+0000 7f4d1a800640 1 -- 172.21.15.67:0/3861441687 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d141627f0 con 0x7f4cf406b520 2024-03-09T09:29:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.442+0000 7f4d0b400640 1 -- 172.21.15.67:0/3861441687 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4d141627f0 con 0x7f4cf406b520 2024-03-09T09:29:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 6 to deep-scrub 2024-03-09T09:29:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 -- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4cf406b520 msgr2=0x7f4cf406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4cf406b520 0x7f4cf406d9e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4d04006fd0 tx=0x7f4d04008040 comp rx=0 tx=0).stop 2024-03-09T09:29:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 -- 172.21.15.67:0/3861441687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d141637c0 msgr2=0x7f4d141eeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d141637c0 0x7f4d141eeca0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00c7e0 tx=0x7f4d0c00cca0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 -- 172.21.15.67:0/3861441687 shutdown_connections 2024-03-09T09:29:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4cf406b520 0x7f4cf406d9e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d141637c0 0x7f4d141eeca0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d140aa960 0x7f4d141ec300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 --2- 172.21.15.67:0/3861441687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4d1406dbe0 0x7f4d141ed460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 -- 172.21.15.67:0/3861441687 >> 172.21.15.67:0/3861441687 conn(0x7f4d140c7370 msgr2=0x7f4d140c1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:27.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 -- 172.21.15.67:0/3861441687 shutdown_connections 2024-03-09T09:29:27.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.445+0000 7f4d1a800640 1 -- 172.21.15.67:0/3861441687 wait complete. 2024-03-09T09:29:27.473 INFO:tasks.ceph:Scrubbing osd.7 2024-03-09T09:29:27.473 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-09T09:29:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.700+0000 7fb485600640 1 Processor -- start 2024-03-09T09:29:27.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.701+0000 7fb485600640 1 -- start start 2024-03-09T09:29:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.702+0000 7fb485600640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fb48014fdc0 0x7fb4800c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.702+0000 7fb485600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb48006e480 0x7fb4800c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.702+0000 7fb485600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4800c1b80 0x7fb48006b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb485600640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4800c42f0 con 0x7fb4800c1b80 2024-03-09T09:29:27.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb485600640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb480156870 con 0x7fb48014fdc0 2024-03-09T09:29:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb485600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800c4590 con 0x7fb48006e480 2024-03-09T09:29:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb484c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4800c1b80 0x7fb48006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb484c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7fb4800c1b80 0x7fb48006b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45272/0 (socket says 172.21.15.67:45272) 2024-03-09T09:29:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb484c00640 1 -- 172.21.15.67:0/3836327012 learned_addr learned my addr 172.21.15.67:0/3836327012 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb47f400640 1 --2- 172.21.15.67:0/3836327012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb48006e480 0x7fb4800c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb47fe00640 1 --2- 172.21.15.67:0/3836327012 >> v2:172.21.15.134:3300/0 conn(0x7fb48014fdc0 0x7fb4800c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb484c00640 1 -- 172.21.15.67:0/3836327012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb48006e480 msgr2=0x7fb4800c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:27.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb484c00640 1 --2- 172.21.15.67:0/3836327012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb48006e480 0x7fb4800c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb484c00640 1 -- 172.21.15.67:0/3836327012 >> v2:172.21.15.134:3300/0 conn(0x7fb48014fdc0 msgr2=0x7fb4800c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:27.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.703+0000 7fb484c00640 1 --2- 172.21.15.67:0/3836327012 >> v2:172.21.15.134:3300/0 conn(0x7fb48014fdc0 0x7fb4800c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.704+0000 7fb484c00640 1 -- 172.21.15.67:0/3836327012 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4800c4870 con 0x7fb4800c1b80 2024-03-09T09:29:27.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.705+0000 7fb484c00640 1 --2- 172.21.15.67:0/3836327012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4800c1b80 0x7fb48006b040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb4800c93d0 tx=0x7fb47802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d92f30d3edb90fc server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:27.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.706+0000 7fb47ea00640 1 -- 172.21.15.67:0/3836327012 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47802c950 con 0x7fb4800c1b80 2024-03-09T09:29:27.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.706+0000 7fb47ea00640 1 -- 172.21.15.67:0/3836327012 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47802caf0 con 0x7fb4800c1b80 2024-03-09T09:29:27.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.706+0000 7fb47ea00640 1 -- 172.21.15.67:0/3836327012 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4780343f0 con 0x7fb4800c1b80 2024-03-09T09:29:27.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.706+0000 7fb485600640 1 -- 172.21.15.67:0/3836327012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4800c1b80 msgr2=0x7fb48006b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.706+0000 7fb485600640 1 --2- 172.21.15.67:0/3836327012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4800c1b80 0x7fb48006b040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb4800c93d0 tx=0x7fb47802bdb0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.706+0000 7fb485600640 1 -- 172.21.15.67:0/3836327012 shutdown_connections 2024-03-09T09:29:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.707+0000 7fb485600640 1 --2- 172.21.15.67:0/3836327012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4800c1b80 0x7fb48006b040 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.707+0000 7fb485600640 1 --2- 172.21.15.67:0/3836327012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb48006e480 0x7fb4800c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.707+0000 7fb485600640 1 --2- 172.21.15.67:0/3836327012 >> v2:172.21.15.134:3300/0 conn(0x7fb48014fdc0 0x7fb4800c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.707+0000 7fb485600640 1 -- 172.21.15.67:0/3836327012 >> 172.21.15.67:0/3836327012 conn(0x7fb48006dc10 msgr2=0x7fb4800c8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:27.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.707+0000 7fb485600640 1 -- 172.21.15.67:0/3836327012 shutdown_connections 2024-03-09T09:29:27.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.707+0000 7fb485600640 1 -- 172.21.15.67:0/3836327012 wait complete. 2024-03-09T09:29:27.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.708+0000 7fb485600640 1 Processor -- start 2024-03-09T09:29:27.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.709+0000 7fb485600640 1 -- start start 2024-03-09T09:29:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.709+0000 7fb485600640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb48006e480 0x7fb4800c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb485600640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4800c1b80 0x7fb480154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb485600640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48014fdc0 0x7fb48014a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb485600640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48006b700 con 0x7fb48014fdc0 2024-03-09T09:29:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb485600640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48006b580 con 0x7fb48006e480 2024-03-09T09:29:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb485600640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48006b880 con 0x7fb4800c1b80 2024-03-09T09:29:27.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb484c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48014fdc0 0x7fb48014a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb484c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48014fdc0 0x7fb48014a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45280/0 (socket says 172.21.15.67:45280) 2024-03-09T09:29:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb484c00640 1 -- 172.21.15.67:0/565971671 learned_addr learned my addr 172.21.15.67:0/565971671 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb47f400640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4800c1b80 0x7fb480154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb47fe00640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb48006e480 0x7fb4800c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb484c00640 1 -- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4800c1b80 msgr2=0x7fb480154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb484c00640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4800c1b80 0x7fb480154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb484c00640 1 -- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb48006e480 msgr2=0x7fb4800c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb484c00640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb48006e480 0x7fb4800c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.710+0000 7fb484c00640 1 -- 172.21.15.67:0/565971671 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48014ad50 con 0x7fb48014fdc0 2024-03-09T09:29:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.711+0000 7fb484c00640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48014fdc0 0x7fb48014a0b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb47802c3c0 tx=0x7fb478004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.711+0000 7fb477e00640 1 -- 172.21.15.67:0/565971671 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478043020 con 0x7fb48014fdc0 2024-03-09T09:29:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.711+0000 7fb477e00640 1 -- 172.21.15.67:0/565971671 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47803e450 con 0x7fb48014fdc0 2024-03-09T09:29:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.711+0000 7fb477e00640 1 -- 172.21.15.67:0/565971671 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47803e820 con 0x7fb48014fdc0 2024-03-09T09:29:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.711+0000 7fb485600640 1 -- 172.21.15.67:0/565971671 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb480152d60 con 0x7fb48014fdc0 2024-03-09T09:29:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.711+0000 7fb485600640 1 -- 172.21.15.67:0/565971671 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb480154cd0 con 0x7fb48014fdc0 2024-03-09T09:29:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.712+0000 7fb485600640 1 -- 172.21.15.67:0/565971671 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb480159e40 con 0x7fb48014fdc0 2024-03-09T09:29:27.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.713+0000 7fb477e00640 1 -- 172.21.15.67:0/565971671 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb478046020 con 0x7fb48014fdc0 2024-03-09T09:29:27.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.714+0000 7fb477e00640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb46006b4d0 0x7fb46006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.714+0000 7fb477e00640 1 -- 172.21.15.67:0/565971671 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fb4780ad220 con 0x7fb48014fdc0 2024-03-09T09:29:27.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.714+0000 7fb477e00640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] conn(0x7fb460075ed0 0x7fb460078370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.714+0000 7fb477e00640 1 -- 172.21.15.67:0/565971671 --> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb478031e50 con 0x7fb460075ed0 2024-03-09T09:29:27.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.714+0000 7fb477e00640 1 -- 172.21.15.67:0/565971671 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4780b4050 con 0x7fb48014fdc0 2024-03-09T09:29:27.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.715+0000 7fb47fe00640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb46006b4d0 0x7fb46006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.715+0000 7fb47f400640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] conn(0x7fb460075ed0 0x7fb460078370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.716+0000 7fb47fe00640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb46006b4d0 0x7fb46006d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb470006fd0 tx=0x7fb470008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:27.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.716+0000 7fb47f400640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] conn(0x7fb460075ed0 0x7fb460078370 crc :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:27.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.718+0000 7fb477e00640 1 -- 172.21.15.67:0/565971671 <== osd.7 v2:172.21.15.134:6816/128562112 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb478031e50 con 0x7fb460075ed0 2024-03-09T09:29:27.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.732+0000 7fb485600640 1 -- 172.21.15.67:0/565971671 --> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb48004f430 con 0x7fb460075ed0 2024-03-09T09:29:27.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.733+0000 7fb477e00640 1 -- 172.21.15.67:0/565971671 <== osd.7 v2:172.21.15.134:6816/128562112 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb48004f430 con 0x7fb460075ed0 2024-03-09T09:29:27.741 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-09T09:29:27.741 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-09T09:29:27.741 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-09T09:29:27.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.734+0000 7fb485600640 1 -- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] conn(0x7fb460075ed0 msgr2=0x7fb460078370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.734+0000 7fb485600640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:6816/128562112,v1:172.21.15.134:6817/128562112] conn(0x7fb460075ed0 0x7fb460078370 crc :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.734+0000 7fb485600640 1 -- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb46006b4d0 msgr2=0x7fb46006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.734+0000 7fb485600640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fb46006b4d0 0x7fb46006d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb470006fd0 tx=0x7fb470008040 comp rx=0 tx=0).stop 2024-03-09T09:29:27.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.734+0000 7fb485600640 1 -- 172.21.15.67:0/565971671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48014fdc0 msgr2=0x7fb48014a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.734+0000 7fb485600640 1 --2- 172.21.15.67:0/565971671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb48014fdc0 0x7fb48014a0b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb47802c3c0 tx=0x7fb478004a40 comp rx=0 tx=0).stop 2024-03-09T09:29:27.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.734+0000 7fb484c00640 1 -- 172.21.15.67:0/565971671 reap_dead start 2024-03-09T09:29:27.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.734+0000 7fb485600640 1 -- 172.21.15.67:0/565971671 shutdown_connections 2024-03-09T09:29:27.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.734+0000 7fb485600640 1 -- 172.21.15.67:0/565971671 >> 172.21.15.67:0/565971671 conn(0x7fb48006dc10 msgr2=0x7fb480153830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:27.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.734+0000 7fb485600640 1 -- 172.21.15.67:0/565971671 shutdown_connections 2024-03-09T09:29:27.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.734+0000 7fb485600640 1 -- 172.21.15.67:0/565971671 wait complete. 2024-03-09T09:29:27.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-09T09:29:27.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.968+0000 7fcf20200640 1 Processor -- start 2024-03-09T09:29:27.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.968+0000 7fcf20200640 1 -- start start 2024-03-09T09:29:27.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.969+0000 7fcf20200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf180c4860 0x7fcf18072b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.969+0000 7fcf20200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcf1806f530 0x7fcf18154620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.969+0000 7fcf20200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf18154b60 con 0x7fcf18157990 2024-03-09T09:29:27.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.969+0000 7fcf20200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf18157500 con 0x7fcf1806f530 2024-03-09T09:29:27.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.969+0000 7fcf20200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1806fad0 con 0x7fcf180c4860 2024-03-09T09:29:27.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.970+0000 7fcf1e200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcf18157990 0x7fcf180c5410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:36864/0 (socket says 172.21.15.67:36864) 2024-03-09T09:29:27.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.970+0000 7fcf1e200640 1 -- 172.21.15.67:0/2054744180 learned_addr learned my addr 172.21.15.67:0/2054744180 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:29:27.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.970+0000 7fcf1d800640 1 --2- 172.21.15.67:0/2054744180 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf180c4860 0x7fcf18072b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.970+0000 7fcf1ce00640 1 --2- 172.21.15.67:0/2054744180 >> v2:172.21.15.134:3300/0 conn(0x7fcf1806f530 0x7fcf18154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.971+0000 7fcf17e00640 1 -- 172.21.15.67:0/2054744180 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153577006 0 0) 0x7fcf10001490 con 0x7fcf18157990 2024-03-09T09:29:27.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.971+0000 7fcf17e00640 1 -- 172.21.15.67:0/2054744180 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf00003520 con 0x7fcf18157990 2024-03-09T09:29:27.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.971+0000 7fcf1d800640 1 -- 172.21.15.67:0/2054744180 >> v2:172.21.15.134:3300/0 conn(0x7fcf1806f530 msgr2=0x7fcf18154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:27.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.971+0000 7fcf1d800640 1 --2- 172.21.15.67:0/2054744180 >> v2:172.21.15.134:3300/0 conn(0x7fcf1806f530 0x7fcf18154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.971+0000 7fcf1d800640 1 -- 172.21.15.67:0/2054744180 >> v1:172.21.15.67:6789/0 conn(0x7fcf18157990 legacy=0x7fcf180c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.971+0000 7fcf1d800640 1 -- 172.21.15.67:0/2054744180 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1806fdb0 con 0x7fcf180c4860 2024-03-09T09:29:27.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.972+0000 7fcf1d800640 1 --2- 172.21.15.67:0/2054744180 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf180c4860 0x7fcf18072b30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0097a0 tx=0x7fcf0c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90917a5cbbad1412 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:27.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.973+0000 7fcf17e00640 1 -- 172.21.15.67:0/2054744180 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c031780 con 0x7fcf180c4860 2024-03-09T09:29:27.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.973+0000 7fcf17e00640 1 -- 172.21.15.67:0/2054744180 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c037070 con 0x7fcf180c4860 2024-03-09T09:29:27.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.973+0000 7fcf17e00640 1 -- 172.21.15.67:0/2054744180 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c031b40 con 0x7fcf180c4860 2024-03-09T09:29:27.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.974+0000 7fcf20200640 1 -- 172.21.15.67:0/2054744180 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf180c4860 msgr2=0x7fcf18072b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.974+0000 7fcf20200640 1 --2- 172.21.15.67:0/2054744180 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf180c4860 0x7fcf18072b30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0097a0 tx=0x7fcf0c02b820 comp rx=0 tx=0).stop 2024-03-09T09:29:27.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.974+0000 7fcf20200640 1 -- 172.21.15.67:0/2054744180 shutdown_connections 2024-03-09T09:29:27.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.974+0000 7fcf20200640 1 --2- 172.21.15.67:0/2054744180 >> v2:172.21.15.134:3300/0 conn(0x7fcf1806f530 0x7fcf18154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.974+0000 7fcf20200640 1 --2- 172.21.15.67:0/2054744180 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf180c4860 0x7fcf18072b30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.974+0000 7fcf20200640 1 -- 172.21.15.67:0/2054744180 >> 172.21.15.67:0/2054744180 conn(0x7fcf1806c540 msgr2=0x7fcf1806c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:27.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.974+0000 7fcf20200640 1 -- 172.21.15.67:0/2054744180 shutdown_connections 2024-03-09T09:29:27.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.974+0000 7fcf20200640 1 -- 172.21.15.67:0/2054744180 wait complete. 2024-03-09T09:29:27.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.976+0000 7fcf20200640 1 Processor -- start 2024-03-09T09:29:27.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.976+0000 7fcf20200640 1 -- start start 2024-03-09T09:29:27.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.977+0000 7fcf20200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf1806f530 0x7fcf181eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.977+0000 7fcf20200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf180c4860 0x7fcf181ef110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.977+0000 7fcf20200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18157990 0x7fcf1814e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.977+0000 7fcf20200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180c4c40 con 0x7fcf18157990 2024-03-09T09:29:27.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.977+0000 7fcf20200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf18157190 con 0x7fcf180c4860 2024-03-09T09:29:27.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.977+0000 7fcf20200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180c5b20 con 0x7fcf1806f530 2024-03-09T09:29:27.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.977+0000 7fcf1e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18157990 0x7fcf1814e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.977+0000 7fcf1e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18157990 0x7fcf1814e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45290/0 (socket says 172.21.15.67:45290) 2024-03-09T09:29:27.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.977+0000 7fcf1e200640 1 -- 172.21.15.67:0/4011568379 learned_addr learned my addr 172.21.15.67:0/4011568379 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:27.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.978+0000 7fcf1ce00640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf180c4860 0x7fcf181ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.978+0000 7fcf1d800640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf1806f530 0x7fcf181eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.978+0000 7fcf1e200640 1 -- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf1806f530 msgr2=0x7fcf181eebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.978+0000 7fcf1e200640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf1806f530 0x7fcf181eebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.978+0000 7fcf1e200640 1 -- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf180c4860 msgr2=0x7fcf181ef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:27.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.978+0000 7fcf1e200640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf180c4860 0x7fcf181ef110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.978+0000 7fcf1e200640 1 -- 172.21.15.67:0/4011568379 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf181e3070 con 0x7fcf18157990 2024-03-09T09:29:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.978+0000 7fcf1e200640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18157990 0x7fcf1814e2b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000cb40 tx=0x7fcf1000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.979+0000 7fcf15600640 1 -- 172.21.15.67:0/4011568379 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10017070 con 0x7fcf18157990 2024-03-09T09:29:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.979+0000 7fcf15600640 1 -- 172.21.15.67:0/4011568379 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1000ac40 con 0x7fcf18157990 2024-03-09T09:29:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.979+0000 7fcf15600640 1 -- 172.21.15.67:0/4011568379 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10012400 con 0x7fcf18157990 2024-03-09T09:29:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.979+0000 7fcf20200640 1 -- 172.21.15.67:0/4011568379 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf181ef930 con 0x7fcf18157990 2024-03-09T09:29:27.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.979+0000 7fcf20200640 1 -- 172.21.15.67:0/4011568379 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf18153b80 con 0x7fcf18157990 2024-03-09T09:29:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.980+0000 7fcf20200640 1 -- 172.21.15.67:0/4011568379 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf18153690 con 0x7fcf18157990 2024-03-09T09:29:27.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.981+0000 7fcf15600640 1 -- 172.21.15.67:0/4011568379 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcf1001a070 con 0x7fcf18157990 2024-03-09T09:29:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.982+0000 7fcf15600640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcef806b4d0 0x7fcef806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.982+0000 7fcf15600640 1 -- 172.21.15.67:0/4011568379 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7fcf1008a290 con 0x7fcf18157990 2024-03-09T09:29:27.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.983+0000 7fcf1d800640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcef806b4d0 0x7fcef806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:27.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.984+0000 7fcf1d800640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcef806b4d0 0x7fcef806d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c036000 tx=0x7fcf0c03d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:27.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:27.987+0000 7fcf15600640 1 -- 172.21.15.67:0/4011568379 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcf18154b60 con 0x7fcf18157990 2024-03-09T09:29:28.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.271+0000 7fcf20200640 1 -- 172.21.15.67:0/4011568379 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf1814f170 con 0x7fcef806b4d0 2024-03-09T09:29:28.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.273+0000 7fcf15600640 1 -- 172.21.15.67:0/4011568379 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcf0c037040 con 0x7fcef806b4d0 2024-03-09T09:29:28.280 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 7 to deep-scrub 2024-03-09T09:29:28.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.275+0000 7fcf20200640 1 -- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcef806b4d0 msgr2=0x7fcef806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:28.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.275+0000 7fcf20200640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcef806b4d0 0x7fcef806d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c036000 tx=0x7fcf0c03d040 comp rx=0 tx=0).stop 2024-03-09T09:29:28.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.275+0000 7fcf20200640 1 -- 172.21.15.67:0/4011568379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18157990 msgr2=0x7fcf1814e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:28.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.275+0000 7fcf20200640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18157990 0x7fcf1814e2b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000cb40 tx=0x7fcf1000a060 comp rx=0 tx=0).stop 2024-03-09T09:29:28.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.275+0000 7fcf20200640 1 -- 172.21.15.67:0/4011568379 shutdown_connections 2024-03-09T09:29:28.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.275+0000 7fcf20200640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcef806b4d0 0x7fcef806d990 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:28.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.275+0000 7fcf20200640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18157990 0x7fcf1814e2b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:28.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.275+0000 7fcf20200640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf180c4860 0x7fcf181ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:28.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.275+0000 7fcf20200640 1 --2- 172.21.15.67:0/4011568379 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf1806f530 0x7fcf181eebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:28.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.276+0000 7fcf20200640 1 -- 172.21.15.67:0/4011568379 >> 172.21.15.67:0/4011568379 conn(0x7fcf1806c540 msgr2=0x7fcf1814fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:28.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.276+0000 7fcf20200640 1 -- 172.21.15.67:0/4011568379 shutdown_connections 2024-03-09T09:29:28.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.276+0000 7fcf20200640 1 -- 172.21.15.67:0/4011568379 wait complete. 2024-03-09T09:29:28.305 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T09:29:28.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.523+0000 7f4b8e200640 1 Processor -- start 2024-03-09T09:29:28.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.524+0000 7f4b8e200640 1 -- start start 2024-03-09T09:29:28.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.525+0000 7f4b8e200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b880c5e40 0x7f4b88156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:28.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.525+0000 7f4b8e200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4b880cadb0 0x7f4b88156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:28.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.525+0000 7f4b8e200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b88156a90 con 0x7f4b88153300 2024-03-09T09:29:28.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.525+0000 7f4b8e200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b88157000 con 0x7f4b880c5e40 2024-03-09T09:29:28.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.525+0000 7f4b8e200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b880ae430 con 0x7f4b880cadb0 2024-03-09T09:29:28.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.526+0000 7f4b87e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b880c5e40 0x7f4b88156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:28.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.526+0000 7f4b87e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f4b880c5e40 0x7f4b88156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45296/0 (socket says 172.21.15.67:45296) 2024-03-09T09:29:28.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.526+0000 7f4b87e00640 1 -- 172.21.15.67:0/1537095839 learned_addr learned my addr 172.21.15.67:0/1537095839 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:28.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.526+0000 7f4b8d800640 1 --2- 172.21.15.67:0/1537095839 >> v2:172.21.15.134:3300/0 conn(0x7f4b880cadb0 0x7f4b88156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:28.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.527+0000 7f4b87e00640 1 -- 172.21.15.67:0/1537095839 >> v2:172.21.15.134:3300/0 conn(0x7f4b880cadb0 msgr2=0x7f4b88156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:28.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.527+0000 7f4b87e00640 1 --2- 172.21.15.67:0/1537095839 >> v2:172.21.15.134:3300/0 conn(0x7f4b880cadb0 0x7f4b88156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:28.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.527+0000 7f4b87e00640 1 -- 172.21.15.67:0/1537095839 >> v1:172.21.15.134:6789/0 conn(0x7f4b88153300 legacy=0x7f4b880b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:28.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.527+0000 7f4b87e00640 1 -- 172.21.15.67:0/1537095839 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b880ae6f0 con 0x7f4b880c5e40 2024-03-09T09:29:28.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.527+0000 7f4b8d800640 1 --2- 172.21.15.67:0/1537095839 >> v2:172.21.15.134:3300/0 conn(0x7f4b880cadb0 0x7f4b88156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:28.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.528+0000 7f4b87e00640 1 --2- 172.21.15.67:0/1537095839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b880c5e40 0x7f4b88156010 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b880adc60 tx=0x7f4b7402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe455b9cfdf1a956 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:28.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.529+0000 7f4b87400640 1 -- 172.21.15.67:0/1537095839 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74035020 con 0x7f4b880c5e40 2024-03-09T09:29:28.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.529+0000 7f4b87400640 1 -- 172.21.15.67:0/1537095839 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b74031870 con 0x7f4b880c5e40 2024-03-09T09:29:28.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.529+0000 7f4b87400640 1 -- 172.21.15.67:0/1537095839 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74031c40 con 0x7f4b880c5e40 2024-03-09T09:29:28.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.530+0000 7f4b8e200640 1 -- 172.21.15.67:0/1537095839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b880c5e40 msgr2=0x7f4b88156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:28.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.530+0000 7f4b8e200640 1 --2- 172.21.15.67:0/1537095839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b880c5e40 0x7f4b88156010 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b880adc60 tx=0x7f4b7402b910 comp rx=0 tx=0).stop 2024-03-09T09:29:28.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.530+0000 7f4b8e200640 1 -- 172.21.15.67:0/1537095839 shutdown_connections 2024-03-09T09:29:28.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.530+0000 7f4b8e200640 1 --2- 172.21.15.67:0/1537095839 >> v2:172.21.15.134:3300/0 conn(0x7f4b880cadb0 0x7f4b88156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:28.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.530+0000 7f4b8e200640 1 --2- 172.21.15.67:0/1537095839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b880c5e40 0x7f4b88156010 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:28.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.530+0000 7f4b8e200640 1 -- 172.21.15.67:0/1537095839 >> 172.21.15.67:0/1537095839 conn(0x7f4b880732d0 msgr2=0x7f4b880ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:28.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.530+0000 7f4b8e200640 1 -- 172.21.15.67:0/1537095839 shutdown_connections 2024-03-09T09:29:28.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.531+0000 7f4b8e200640 1 -- 172.21.15.67:0/1537095839 wait complete. 2024-03-09T09:29:28.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.533+0000 7f4b8e200640 1 Processor -- start 2024-03-09T09:29:28.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.533+0000 7f4b8e200640 1 -- start start 2024-03-09T09:29:28.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.534+0000 7f4b8e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b880c5e40 0x7f4b881f0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:28.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.535+0000 7f4b8e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b880cadb0 0x7f4b881584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:28.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.535+0000 7f4b8e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b88153300 0x7f4b880aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:28.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.535+0000 7f4b8e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b88159e90 con 0x7f4b880c5e40 2024-03-09T09:29:28.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.535+0000 7f4b8e200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b88159bb0 con 0x7f4b880cadb0 2024-03-09T09:29:28.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.535+0000 7f4b8e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880c59e0 con 0x7f4b88153300 2024-03-09T09:29:28.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.535+0000 7f4b8ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b880c5e40 0x7f4b881f0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:28.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.535+0000 7f4b8ce00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b880c5e40 0x7f4b881f0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45306/0 (socket says 172.21.15.67:45306) 2024-03-09T09:29:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.535+0000 7f4b8ce00640 1 -- 172.21.15.67:0/3971493419 learned_addr learned my addr 172.21.15.67:0/3971493419 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.535+0000 7f4b8d800640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b88153300 0x7f4b880aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.536+0000 7f4b87e00640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b880cadb0 0x7f4b881584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.536+0000 7f4b8ce00640 1 -- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b88153300 msgr2=0x7f4b880aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:28.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.536+0000 7f4b8ce00640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b88153300 0x7f4b880aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:28.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.536+0000 7f4b8ce00640 1 -- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b880cadb0 msgr2=0x7f4b881584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:28.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.536+0000 7f4b8ce00640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b880cadb0 0x7f4b881584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:28.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.536+0000 7f4b8ce00640 1 -- 172.21.15.67:0/3971493419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b881ef780 con 0x7f4b880c5e40 2024-03-09T09:29:28.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.537+0000 7f4b8ce00640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b880c5e40 0x7f4b881f0240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00b420 tx=0x7f4b7c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:28.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.538+0000 7f4b84c00640 1 -- 172.21.15.67:0/3971493419 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c011830 con 0x7f4b880c5e40 2024-03-09T09:29:28.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.538+0000 7f4b84c00640 1 -- 172.21.15.67:0/3971493419 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c016070 con 0x7f4b880c5e40 2024-03-09T09:29:28.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.538+0000 7f4b84c00640 1 -- 172.21.15.67:0/3971493419 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c011c10 con 0x7f4b880c5e40 2024-03-09T09:29:28.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.538+0000 7f4b8e200640 1 -- 172.21.15.67:0/3971493419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b880c8350 con 0x7f4b880c5e40 2024-03-09T09:29:28.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.539+0000 7f4b8e200640 1 -- 172.21.15.67:0/3971493419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b881efd20 con 0x7f4b880c5e40 2024-03-09T09:29:28.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.540+0000 7f4b8e200640 1 -- 172.21.15.67:0/3971493419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8804f3c0 con 0x7f4b880c5e40 2024-03-09T09:29:28.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.543+0000 7f4b84c00640 1 -- 172.21.15.67:0/3971493419 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4b7c01b460 con 0x7f4b880c5e40 2024-03-09T09:29:28.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.544+0000 7f4b84c00640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6806b550 0x7f4b6806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:28.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.544+0000 7f4b84c00640 1 -- 172.21.15.67:0/3971493419 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f4b7c08a8b0 con 0x7f4b880c5e40 2024-03-09T09:29:28.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.545+0000 7f4b87e00640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6806b550 0x7f4b6806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:28.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.546+0000 7f4b84c00640 1 -- 172.21.15.67:0/3971493419 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4b7c020020 con 0x7f4b880c5e40 2024-03-09T09:29:28.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.547+0000 7f4b87e00640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6806b550 0x7f4b6806da10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4b74036000 tx=0x7f4b7403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:28.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.784+0000 7f4b8e200640 1 -- 172.21.15.67:0/3971493419 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b880cab90 con 0x7f4b6806b550 2024-03-09T09:29:29.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.817+0000 7f4b84c00640 1 -- 172.21.15.67:0/3971493419 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+496287 (secure 0 0 0) 0x7f4b880cab90 con 0x7f4b6806b550 2024-03-09T09:29:29.213 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:29:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T09:29:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.824+0000 7f4b8e200640 1 -- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6806b550 msgr2=0x7f4b6806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.824+0000 7f4b8e200640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6806b550 0x7f4b6806da10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4b74036000 tx=0x7f4b7403c040 comp rx=0 tx=0).stop 2024-03-09T09:29:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.824+0000 7f4b8e200640 1 -- 172.21.15.67:0/3971493419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b880c5e40 msgr2=0x7f4b881f0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.824+0000 7f4b8e200640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b880c5e40 0x7f4b881f0240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00b420 tx=0x7f4b7c00b8e0 comp rx=0 tx=0).stop 2024-03-09T09:29:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.824+0000 7f4b8e200640 1 -- 172.21.15.67:0/3971493419 shutdown_connections 2024-03-09T09:29:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.824+0000 7f4b8e200640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f4b6806b550 0x7f4b6806da10 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.824+0000 7f4b8e200640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4b88153300 0x7f4b880aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.824+0000 7f4b8e200640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b880cadb0 0x7f4b881584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.824+0000 7f4b8e200640 1 --2- 172.21.15.67:0/3971493419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b880c5e40 0x7f4b881f0240 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.824+0000 7f4b8e200640 1 -- 172.21.15.67:0/3971493419 >> 172.21.15.67:0/3971493419 conn(0x7f4b880732d0 msgr2=0x7f4b88152140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:29.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.825+0000 7f4b8e200640 1 -- 172.21.15.67:0/3971493419 shutdown_connections 2024-03-09T09:29:29.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:28.825+0000 7f4b8e200640 1 -- 172.21.15.67:0/3971493419 wait complete. 2024-03-09T09:29:29.221 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":1202,"stamp":"2024-03-09T09:29:27.714684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1879939357,"num_objects":17181,"num_object_clones":4728,"num_object_copies":34362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17181,"num_whiteouts":4728,"num_read":20750,"num_read_kb":3057165,"num_write":223849,"num_write_kb":12119060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2438,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2681252,"num_omap_keys":4452,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":333866,"ondisk_log_size":333866,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6218916,"kb_used_data":3725476,"kb_used_omap":12,"kb_used_meta":2493363,"kb_avail":748755804,"statfs":{"total":773094113280,"available":766725943296,"internally_reserved":0,"allocated":3814887424,"data_stored":3763993575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2553204274},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,5,5,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2917012,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":33,"num_write_kb":2864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2681252,"num_omap_keys":4452,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004884"},"pg_stats":[{"pgid":"5.18","version":"89'2474","reported_seq":5348,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.921398+0000","last_change":"2024-03-09T09:27:34.120297+0000","last_active":"2024-03-09T09:29:17.921398+0000","last_peered":"2024-03-09T09:29:17.921398+0000","last_clean":"2024-03-09T09:29:17.921398+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T09:29:17.921398+0000","last_undegraded":"2024-03-09T09:29:17.921398+0000","last_fullsized":"2024-03-09T09:29:17.921398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2474,"log_dups_size":0,"ondisk_log_size":2474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":3.854819714,"stat_sum":{"num_bytes":1582215,"num_objects":90,"num_object_clones":27,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":27,"num_read":122,"num_read_kb":30313,"num_write":1455,"num_write_kb":61176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"89'449","reported_seq":619,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:51.274382+0000","last_change":"2024-03-09T08:52:17.288653+0000","last_active":"2024-03-09T09:27:51.274382+0000","last_peered":"2024-03-09T09:27:51.274382+0000","last_clean":"2024-03-09T09:27:51.274382+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T09:27:51.274382+0000","last_undegraded":"2024-03-09T09:27:51.274382+0000","last_fullsized":"2024-03-09T09:27:51.274382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":462,"num_write_kb":82632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"89'2179","reported_seq":5159,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:28.596602+0000","last_change":"2024-03-09T09:27:12.157369+0000","last_active":"2024-03-09T09:28:28.596602+0000","last_peered":"2024-03-09T09:28:28.596602+0000","last_clean":"2024-03-09T09:28:28.596602+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T09:28:28.596602+0000","last_undegraded":"2024-03-09T09:28:28.596602+0000","last_fullsized":"2024-03-09T09:28:28.596602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2179,"log_dups_size":0,"ondisk_log_size":2179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:03:38.261625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.47085273999999999,"stat_sum":{"num_bytes":1473782,"num_objects":111,"num_object_clones":38,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":38,"num_read":99,"num_read_kb":25344,"num_write":1311,"num_write_kb":54508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"89'405","reported_seq":563,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:59.412978+0000","last_change":"2024-03-09T08:52:28.571905+0000","last_active":"2024-03-09T09:27:59.412978+0000","last_peered":"2024-03-09T09:27:59.412978+0000","last_clean":"2024-03-09T09:27:59.412978+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T09:27:59.412978+0000","last_undegraded":"2024-03-09T09:27:59.412978+0000","last_fullsized":"2024-03-09T09:27:59.412978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:42:01.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":445,"num_write_kb":78871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"89'2256","reported_seq":5169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.928000+0000","last_change":"2024-03-09T09:27:32.072161+0000","last_active":"2024-03-09T09:29:17.928000+0000","last_peered":"2024-03-09T09:29:17.928000+0000","last_clean":"2024-03-09T09:29:17.928000+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T09:29:17.928000+0000","last_undegraded":"2024-03-09T09:29:17.928000+0000","last_fullsized":"2024-03-09T09:29:17.928000+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2256,"log_dups_size":0,"ondisk_log_size":2256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.703796324,"stat_sum":{"num_bytes":1252433,"num_objects":118,"num_object_clones":38,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":38,"num_read":118,"num_read_kb":38794,"num_write":1352,"num_write_kb":71340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"89'745","reported_seq":914,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.595752+0000","last_change":"2024-03-09T08:52:17.270932+0000","last_active":"2024-03-09T09:28:23.595752+0000","last_peered":"2024-03-09T09:28:23.595752+0000","last_clean":"2024-03-09T09:28:23.595752+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T09:28:23.595752+0000","last_undegraded":"2024-03-09T09:28:23.595752+0000","last_fullsized":"2024-03-09T09:28:23.595752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:24:48.578171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":51,"num_read_kb":0,"num_write":808,"num_write_kb":91456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'2160","reported_seq":5147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.039424+0000","last_change":"2024-03-09T09:27:09.692723+0000","last_active":"2024-03-09T09:29:14.039424+0000","last_peered":"2024-03-09T09:29:14.039424+0000","last_clean":"2024-03-09T09:29:14.039424+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T09:29:14.039424+0000","last_undegraded":"2024-03-09T09:29:14.039424+0000","last_fullsized":"2024-03-09T09:29:14.039424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2160,"log_dups_size":0,"ondisk_log_size":2160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.48044756999999999,"stat_sum":{"num_bytes":498978,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":114,"num_read_kb":18251,"num_write":1268,"num_write_kb":37672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"89'892","reported_seq":1052,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:47.659991+0000","last_change":"2024-03-09T08:52:28.573074+0000","last_active":"2024-03-09T09:28:47.659991+0000","last_peered":"2024-03-09T09:28:47.659991+0000","last_clean":"2024-03-09T09:28:47.659991+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T09:28:47.659991+0000","last_undegraded":"2024-03-09T09:28:47.659991+0000","last_fullsized":"2024-03-09T09:28:47.659991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":939,"num_write_kb":120134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"89'2266","reported_seq":5169,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924644+0000","last_change":"2024-03-09T09:27:34.614436+0000","last_active":"2024-03-09T09:29:17.924644+0000","last_peered":"2024-03-09T09:29:17.924644+0000","last_clean":"2024-03-09T09:29:17.924644+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T09:29:17.924644+0000","last_undegraded":"2024-03-09T09:29:17.924644+0000","last_fullsized":"2024-03-09T09:29:17.924644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2266,"log_dups_size":0,"ondisk_log_size":2266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:18:23.796765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.3826170339999999,"stat_sum":{"num_bytes":1068833,"num_objects":92,"num_object_clones":30,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":30,"num_read":126,"num_read_kb":14571,"num_write":1308,"num_write_kb":45011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"89'674","reported_seq":840,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:58.111396+0000","last_change":"2024-03-09T08:52:17.295490+0000","last_active":"2024-03-09T09:27:58.111396+0000","last_peered":"2024-03-09T09:27:58.111396+0000","last_clean":"2024-03-09T09:27:58.111396+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T09:27:58.111396+0000","last_undegraded":"2024-03-09T09:27:58.111396+0000","last_fullsized":"2024-03-09T09:27:58.111396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:27:46.549460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":67,"num_read_kb":0,"num_write":775,"num_write_kb":140765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"89'2379","reported_seq":5342,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.635615+0000","last_change":"2024-03-09T09:27:14.117823+0000","last_active":"2024-03-09T09:28:23.635615+0000","last_peered":"2024-03-09T09:28:23.635615+0000","last_clean":"2024-03-09T09:28:23.635615+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T09:28:23.635615+0000","last_undegraded":"2024-03-09T09:28:23.635615+0000","last_fullsized":"2024-03-09T09:28:23.635615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2379,"log_dups_size":0,"ondisk_log_size":2379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:49:44.890858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.653915281,"stat_sum":{"num_bytes":2793747,"num_objects":148,"num_object_clones":53,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":53,"num_read":138,"num_read_kb":27086,"num_write":1428,"num_write_kb":43660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"89'531","reported_seq":690,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.545016+0000","last_change":"2024-03-09T08:52:28.577128+0000","last_active":"2024-03-09T09:28:26.545016+0000","last_peered":"2024-03-09T09:28:26.545016+0000","last_clean":"2024-03-09T09:28:26.545016+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T09:28:26.545016+0000","last_undegraded":"2024-03-09T09:28:26.545016+0000","last_fullsized":"2024-03-09T09:28:26.545016+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":543,"num_write_kb":111605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"89'2275","reported_seq":5173,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.920377+0000","last_change":"2024-03-09T09:27:34.731982+0000","last_active":"2024-03-09T09:29:17.920377+0000","last_peered":"2024-03-09T09:29:17.920377+0000","last_clean":"2024-03-09T09:29:17.920377+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T09:29:17.920377+0000","last_undegraded":"2024-03-09T09:29:17.920377+0000","last_fullsized":"2024-03-09T09:29:17.920377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2275,"log_dups_size":0,"ondisk_log_size":2275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:34:13.555244+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4571779879999998,"stat_sum":{"num_bytes":3086231,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":121,"num_read_kb":9749,"num_write":1336,"num_write_kb":28981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"89'459","reported_seq":625,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:52.092030+0000","last_change":"2024-03-09T08:52:17.294984+0000","last_active":"2024-03-09T09:27:52.092030+0000","last_peered":"2024-03-09T09:27:52.092030+0000","last_clean":"2024-03-09T09:27:52.092030+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T09:27:52.092030+0000","last_undegraded":"2024-03-09T09:27:52.092030+0000","last_fullsized":"2024-03-09T09:27:52.092030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:34:46.648792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":55,"num_read_kb":0,"num_write":535,"num_write_kb":83331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'2158","reported_seq":5152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.068307+0000","last_change":"2024-03-09T09:27:13.836207+0000","last_active":"2024-03-09T09:29:14.068307+0000","last_peered":"2024-03-09T09:29:14.068307+0000","last_clean":"2024-03-09T09:29:14.068307+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T09:29:14.068307+0000","last_undegraded":"2024-03-09T09:29:14.068307+0000","last_fullsized":"2024-03-09T09:29:14.068307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2158,"log_dups_size":0,"ondisk_log_size":2158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:20:22.564401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.3715434019999999,"stat_sum":{"num_bytes":1052383,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":116,"num_read_kb":6444,"num_write":1289,"num_write_kb":22757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"89'611","reported_seq":779,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.897913+0000","last_change":"2024-03-09T09:29:25.898016+0000","last_active":"2024-03-09T09:29:25.897913+0000","last_peered":"2024-03-09T09:29:25.897913+0000","last_clean":"2024-03-09T09:29:25.897913+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T09:29:25.897913+0000","last_undegraded":"2024-03-09T09:29:25.897913+0000","last_fullsized":"2024-03-09T09:29:25.897913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'611","last_scrub_stamp":"2024-03-09T09:29:25.897713+0000","last_deep_scrub":"89'611","last_deep_scrub_stamp":"2024-03-09T09:29:25.897713+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.897713+0000","objects_scrubbed":42,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:53:23.202311+0000","scrub_duration":135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":598,"num_write_kb":95071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":469023,"num_omap_keys":788,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"89'2159","reported_seq":5095,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924503+0000","last_change":"2024-03-09T09:27:32.079362+0000","last_active":"2024-03-09T09:29:17.924503+0000","last_peered":"2024-03-09T09:29:17.924503+0000","last_clean":"2024-03-09T09:29:17.924503+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T09:29:17.924503+0000","last_undegraded":"2024-03-09T09:29:17.924503+0000","last_fullsized":"2024-03-09T09:29:17.924503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2159,"log_dups_size":0,"ondisk_log_size":2159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.709485658,"stat_sum":{"num_bytes":1197226,"num_objects":92,"num_object_clones":31,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":31,"num_read":105,"num_read_kb":17945,"num_write":1287,"num_write_kb":45184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"89'2226","reported_seq":5195,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.578738+0000","last_change":"2024-03-09T09:27:07.936292+0000","last_active":"2024-03-09T09:29:17.578738+0000","last_peered":"2024-03-09T09:29:17.578738+0000","last_clean":"2024-03-09T09:29:17.578738+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T09:29:17.578738+0000","last_undegraded":"2024-03-09T09:29:17.578738+0000","last_fullsized":"2024-03-09T09:29:17.578738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2226,"log_dups_size":0,"ondisk_log_size":2226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.2630355259999999,"stat_sum":{"num_bytes":1174090,"num_objects":98,"num_object_clones":35,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":35,"num_read":98,"num_read_kb":18109,"num_write":1319,"num_write_kb":36158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"89'817","reported_seq":985,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.795764+0000","last_change":"2024-03-09T08:52:17.293041+0000","last_active":"2024-03-09T09:28:14.795764+0000","last_peered":"2024-03-09T09:28:14.795764+0000","last_clean":"2024-03-09T09:28:14.795764+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T09:28:14.795764+0000","last_undegraded":"2024-03-09T09:28:14.795764+0000","last_fullsized":"2024-03-09T09:28:14.795764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:16:11.673474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":48,"num_read_kb":0,"num_write":879,"num_write_kb":99632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"89'743","reported_seq":904,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.657136+0000","last_change":"2024-03-09T08:52:28.570631+0000","last_active":"2024-03-09T09:28:32.657136+0000","last_peered":"2024-03-09T09:28:32.657136+0000","last_clean":"2024-03-09T09:28:32.657136+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T09:28:32.657136+0000","last_undegraded":"2024-03-09T09:28:32.657136+0000","last_fullsized":"2024-03-09T09:28:32.657136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971838,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":820,"num_write_kb":111945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"89'2194","reported_seq":5105,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.931440+0000","last_change":"2024-03-09T09:27:30.893758+0000","last_active":"2024-03-09T09:29:17.931440+0000","last_peered":"2024-03-09T09:29:17.931440+0000","last_clean":"2024-03-09T09:29:17.931440+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T09:29:17.931440+0000","last_undegraded":"2024-03-09T09:29:17.931440+0000","last_fullsized":"2024-03-09T09:29:17.931440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2194,"log_dups_size":0,"ondisk_log_size":2194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.63600609399999997,"stat_sum":{"num_bytes":1331264,"num_objects":108,"num_object_clones":38,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":38,"num_read":110,"num_read_kb":9978,"num_write":1295,"num_write_kb":30848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'2264","reported_seq":5240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.036656+0000","last_change":"2024-03-09T09:27:12.385283+0000","last_active":"2024-03-09T09:29:14.036656+0000","last_peered":"2024-03-09T09:29:14.036656+0000","last_clean":"2024-03-09T09:29:14.036656+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T09:29:14.036656+0000","last_undegraded":"2024-03-09T09:29:14.036656+0000","last_fullsized":"2024-03-09T09:29:14.036656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2264,"log_dups_size":0,"ondisk_log_size":2264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:24:11.227698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.69905185199999997,"stat_sum":{"num_bytes":1031167,"num_objects":108,"num_object_clones":39,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":39,"num_read":109,"num_read_kb":22315,"num_write":1384,"num_write_kb":51621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'445","reported_seq":615,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:12.567724+0000","last_change":"2024-03-09T08:52:17.274952+0000","last_active":"2024-03-09T09:29:12.567724+0000","last_peered":"2024-03-09T09:29:12.567724+0000","last_clean":"2024-03-09T09:29:12.567724+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T09:29:12.567724+0000","last_undegraded":"2024-03-09T09:29:12.567724+0000","last_fullsized":"2024-03-09T09:29:12.567724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":467,"num_write_kb":115624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"89'670","reported_seq":840,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.910521+0000","last_change":"2024-03-09T09:29:26.910625+0000","last_active":"2024-03-09T09:29:26.910521+0000","last_peered":"2024-03-09T09:29:26.910521+0000","last_clean":"2024-03-09T09:29:26.910521+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T09:29:26.910521+0000","last_undegraded":"2024-03-09T09:29:26.910521+0000","last_fullsized":"2024-03-09T09:29:26.910521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'670","last_scrub_stamp":"2024-03-09T09:29:26.910351+0000","last_deep_scrub":"89'670","last_deep_scrub_stamp":"2024-03-09T09:29:26.910351+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.910351+0000","objects_scrubbed":37,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:30:18.446605+0000","scrub_duration":145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":674,"num_write_kb":99647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":379703,"num_omap_keys":669,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"89'2261","reported_seq":5180,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.928787+0000","last_change":"2024-03-09T09:27:34.263088+0000","last_active":"2024-03-09T09:29:17.928787+0000","last_peered":"2024-03-09T09:29:17.928787+0000","last_clean":"2024-03-09T09:29:17.928787+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T09:29:17.928787+0000","last_undegraded":"2024-03-09T09:29:17.928787+0000","last_fullsized":"2024-03-09T09:29:17.928787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2261,"log_dups_size":0,"ondisk_log_size":2261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":3.9938263410000001,"stat_sum":{"num_bytes":1661343,"num_objects":107,"num_object_clones":33,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":33,"num_read":118,"num_read_kb":35803,"num_write":1365,"num_write_kb":51532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'2351","reported_seq":5334,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.039556+0000","last_change":"2024-03-09T09:27:12.068457+0000","last_active":"2024-03-09T09:29:14.039556+0000","last_peered":"2024-03-09T09:29:14.039556+0000","last_clean":"2024-03-09T09:29:14.039556+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T09:29:14.039556+0000","last_undegraded":"2024-03-09T09:29:14.039556+0000","last_fullsized":"2024-03-09T09:29:14.039556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2351,"log_dups_size":0,"ondisk_log_size":2351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:06:13.795599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.380889329,"stat_sum":{"num_bytes":1156693,"num_objects":132,"num_object_clones":46,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":46,"num_read":122,"num_read_kb":38030,"num_write":1453,"num_write_kb":61557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"89'644","reported_seq":814,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:07.566939+0000","last_change":"2024-03-09T08:52:17.284329+0000","last_active":"2024-03-09T09:29:07.566939+0000","last_peered":"2024-03-09T09:29:07.566939+0000","last_clean":"2024-03-09T09:29:07.566939+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T09:29:07.566939+0000","last_undegraded":"2024-03-09T09:29:07.566939+0000","last_fullsized":"2024-03-09T09:29:07.566939+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":715,"num_write_kb":95464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"89'673","reported_seq":843,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:24.128278+0000","last_change":"2024-03-09T09:29:24.128343+0000","last_active":"2024-03-09T09:29:24.128278+0000","last_peered":"2024-03-09T09:29:24.128278+0000","last_clean":"2024-03-09T09:29:24.128278+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T09:29:24.128278+0000","last_undegraded":"2024-03-09T09:29:24.128278+0000","last_fullsized":"2024-03-09T09:29:24.128278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'673","last_scrub_stamp":"2024-03-09T09:29:24.128149+0000","last_deep_scrub":"89'673","last_deep_scrub_stamp":"2024-03-09T09:29:24.128149+0000","last_clean_scrub_stamp":"2024-03-09T09:29:24.128149+0000","objects_scrubbed":47,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:29:11.611027+0000","scrub_duration":125,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748758,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":685,"num_write_kb":95366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":455288,"num_omap_keys":782,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"89'2201","reported_seq":5127,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.920162+0000","last_change":"2024-03-09T09:27:34.731118+0000","last_active":"2024-03-09T09:29:17.920162+0000","last_peered":"2024-03-09T09:29:17.920162+0000","last_clean":"2024-03-09T09:29:17.920162+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T09:29:17.920162+0000","last_undegraded":"2024-03-09T09:29:17.920162+0000","last_fullsized":"2024-03-09T09:29:17.920162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2201,"log_dups_size":0,"ondisk_log_size":2201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:19:12.794501+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4556873890000004,"stat_sum":{"num_bytes":4719568,"num_objects":111,"num_object_clones":34,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":34,"num_read":112,"num_read_kb":26370,"num_write":1323,"num_write_kb":45566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"89'2296","reported_seq":5257,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.065682+0000","last_change":"2024-03-09T09:27:08.625397+0000","last_active":"2024-03-09T09:29:14.065682+0000","last_peered":"2024-03-09T09:29:14.065682+0000","last_clean":"2024-03-09T09:29:14.065682+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T09:29:14.065682+0000","last_undegraded":"2024-03-09T09:29:14.065682+0000","last_fullsized":"2024-03-09T09:29:14.065682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2296,"log_dups_size":0,"ondisk_log_size":2296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0901323600000001,"stat_sum":{"num_bytes":867761,"num_objects":98,"num_object_clones":31,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":31,"num_read":98,"num_read_kb":37765,"num_write":1389,"num_write_kb":74512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'434","reported_seq":603,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.795677+0000","last_change":"2024-03-09T08:52:17.293292+0000","last_active":"2024-03-09T09:28:14.795677+0000","last_peered":"2024-03-09T09:28:14.795677+0000","last_clean":"2024-03-09T09:28:14.795677+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T09:28:14.795677+0000","last_undegraded":"2024-03-09T09:28:14.795677+0000","last_fullsized":"2024-03-09T09:28:14.795677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:01:43.877286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":69,"num_read_kb":0,"num_write":510,"num_write_kb":91498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"89'614","reported_seq":774,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.657173+0000","last_change":"2024-03-09T08:52:28.573638+0000","last_active":"2024-03-09T09:28:32.657173+0000","last_peered":"2024-03-09T09:28:32.657173+0000","last_clean":"2024-03-09T09:28:32.657173+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T09:28:32.657173+0000","last_undegraded":"2024-03-09T09:28:32.657173+0000","last_fullsized":"2024-03-09T09:28:32.657173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":666,"num_write_kb":107635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"89'2336","reported_seq":5260,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.951528+0000","last_change":"2024-03-09T09:27:34.495452+0000","last_active":"2024-03-09T09:29:17.951528+0000","last_peered":"2024-03-09T09:29:17.951528+0000","last_clean":"2024-03-09T09:29:17.951528+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T09:29:17.951528+0000","last_undegraded":"2024-03-09T09:29:17.951528+0000","last_fullsized":"2024-03-09T09:29:17.951528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2336,"log_dups_size":0,"ondisk_log_size":2336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:06.642191+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.1575832950000002,"stat_sum":{"num_bytes":392237,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":136,"num_read_kb":53607,"num_write":1404,"num_write_kb":81678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"89'2175","reported_seq":5268,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.022216+0000","last_change":"2024-03-09T09:27:09.510565+0000","last_active":"2024-03-09T09:29:14.022216+0000","last_peered":"2024-03-09T09:29:14.022216+0000","last_clean":"2024-03-09T09:29:14.022216+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T09:29:14.022216+0000","last_undegraded":"2024-03-09T09:29:14.022216+0000","last_fullsized":"2024-03-09T09:29:14.022216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.38281627099999999,"stat_sum":{"num_bytes":951718,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":197,"num_read_kb":25950,"num_write":1332,"num_write_kb":45421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'345","reported_seq":515,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:07.577170+0000","last_change":"2024-03-09T08:52:17.291284+0000","last_active":"2024-03-09T09:29:07.577170+0000","last_peered":"2024-03-09T09:29:07.577170+0000","last_clean":"2024-03-09T09:29:07.577170+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T09:29:07.577170+0000","last_undegraded":"2024-03-09T09:29:07.577170+0000","last_fullsized":"2024-03-09T09:29:07.577170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":397,"num_write_kb":78578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"89'1700","reported_seq":2917,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.478531+0000","last_change":"2024-03-09T08:52:28.575630+0000","last_active":"2024-03-09T09:28:31.478531+0000","last_peered":"2024-03-09T09:28:31.478531+0000","last_clean":"2024-03-09T09:28:31.478531+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T09:28:31.478531+0000","last_undegraded":"2024-03-09T09:28:31.478531+0000","last_fullsized":"2024-03-09T09:28:31.478531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1700,"log_dups_size":0,"ondisk_log_size":1700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":1117,"num_read_kb":4814,"num_write":1762,"num_write_kb":112708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"89'2113","reported_seq":5065,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.937100+0000","last_change":"2024-03-09T09:27:34.095045+0000","last_active":"2024-03-09T09:29:17.937100+0000","last_peered":"2024-03-09T09:29:17.937100+0000","last_clean":"2024-03-09T09:29:17.937100+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T09:29:17.937100+0000","last_undegraded":"2024-03-09T09:29:17.937100+0000","last_fullsized":"2024-03-09T09:29:17.937100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2113,"log_dups_size":0,"ondisk_log_size":2113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.8297685179999998,"stat_sum":{"num_bytes":268964,"num_objects":107,"num_object_clones":32,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":32,"num_read":107,"num_read_kb":24622,"num_write":1287,"num_write_kb":41444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"89'2254","reported_seq":5301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.044000+0000","last_change":"2024-03-09T09:27:06.300751+0000","last_active":"2024-03-09T09:29:14.044000+0000","last_peered":"2024-03-09T09:29:14.044000+0000","last_clean":"2024-03-09T09:29:14.044000+0000","last_became_active":"2024-03-09T08:53:00.240844+0000","last_became_peered":"2024-03-09T08:53:00.240844+0000","last_unstale":"2024-03-09T09:29:14.044000+0000","last_undegraded":"2024-03-09T09:29:14.044000+0000","last_fullsized":"2024-03-09T09:29:14.044000+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2254,"log_dups_size":0,"ondisk_log_size":2254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.78540855600000004,"stat_sum":{"num_bytes":1854558,"num_objects":126,"num_object_clones":46,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":46,"num_read":216,"num_read_kb":39239,"num_write":1332,"num_write_kb":71797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"89'1558","reported_seq":1728,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.179789+0000","last_change":"2024-03-09T08:52:17.280987+0000","last_active":"2024-03-09T09:29:14.179789+0000","last_peered":"2024-03-09T09:29:14.179789+0000","last_clean":"2024-03-09T09:29:14.179789+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T09:29:14.179789+0000","last_undegraded":"2024-03-09T09:29:14.179789+0000","last_fullsized":"2024-03-09T09:29:14.179789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1558,"log_dups_size":0,"ondisk_log_size":1558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":57,"num_read_kb":0,"num_write":3420,"num_write_kb":84853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"89'672","reported_seq":833,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.478737+0000","last_change":"2024-03-09T08:52:28.586702+0000","last_active":"2024-03-09T09:28:31.478737+0000","last_peered":"2024-03-09T09:28:31.478737+0000","last_clean":"2024-03-09T09:28:31.478737+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T09:28:31.478737+0000","last_undegraded":"2024-03-09T09:28:31.478737+0000","last_fullsized":"2024-03-09T09:28:31.478737+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:08:09.201224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":71,"num_read_kb":0,"num_write":764,"num_write_kb":87842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"89'2339","reported_seq":5224,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.921493+0000","last_change":"2024-03-09T09:27:30.887844+0000","last_active":"2024-03-09T09:29:17.921493+0000","last_peered":"2024-03-09T09:29:17.921493+0000","last_clean":"2024-03-09T09:29:17.921493+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T09:29:17.921493+0000","last_undegraded":"2024-03-09T09:29:17.921493+0000","last_fullsized":"2024-03-09T09:29:17.921493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2339,"log_dups_size":0,"ondisk_log_size":2339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65806592200000003,"stat_sum":{"num_bytes":2259601,"num_objects":134,"num_object_clones":47,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":47,"num_read":108,"num_read_kb":22295,"num_write":1398,"num_write_kb":43486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"89'2181","reported_seq":5179,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.006304+0000","last_change":"2024-03-09T09:27:09.896878+0000","last_active":"2024-03-09T09:29:14.006304+0000","last_peered":"2024-03-09T09:29:14.006304+0000","last_clean":"2024-03-09T09:29:14.006304+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T09:29:14.006304+0000","last_undegraded":"2024-03-09T09:29:14.006304+0000","last_fullsized":"2024-03-09T09:29:14.006304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2181,"log_dups_size":0,"ondisk_log_size":2181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.96663918400000004,"stat_sum":{"num_bytes":2499803,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":112,"num_read_kb":5934,"num_write":1320,"num_write_kb":25592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"89'642","reported_seq":811,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:14.817501+0000","last_change":"2024-03-09T08:52:17.296650+0000","last_active":"2024-03-09T09:28:14.817501+0000","last_peered":"2024-03-09T09:28:14.817501+0000","last_clean":"2024-03-09T09:28:14.817501+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T09:28:14.817501+0000","last_undegraded":"2024-03-09T09:28:14.817501+0000","last_fullsized":"2024-03-09T09:28:14.817501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:27:27.996157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":75,"num_read_kb":0,"num_write":726,"num_write_kb":96134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"89'577","reported_seq":739,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:18.583993+0000","last_change":"2024-03-09T08:52:28.580879+0000","last_active":"2024-03-09T09:29:18.583993+0000","last_peered":"2024-03-09T09:29:18.583993+0000","last_clean":"2024-03-09T09:29:18.583993+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T09:29:18.583993+0000","last_undegraded":"2024-03-09T09:29:18.583993+0000","last_fullsized":"2024-03-09T09:29:18.583993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7633684,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":618,"num_write_kb":73997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"89'2274","reported_seq":5173,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.892585+0000","last_change":"2024-03-09T09:27:31.029951+0000","last_active":"2024-03-09T09:29:17.892585+0000","last_peered":"2024-03-09T09:29:17.892585+0000","last_clean":"2024-03-09T09:29:17.892585+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T09:29:17.892585+0000","last_undegraded":"2024-03-09T09:29:17.892585+0000","last_fullsized":"2024-03-09T09:29:17.892585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2274,"log_dups_size":0,"ondisk_log_size":2274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.71811667999999995,"stat_sum":{"num_bytes":1846155,"num_objects":126,"num_object_clones":42,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":42,"num_read":111,"num_read_kb":21516,"num_write":1347,"num_write_kb":38338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"89'2284","reported_seq":5231,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.029735+0000","last_change":"2024-03-09T09:27:14.280348+0000","last_active":"2024-03-09T09:29:14.029735+0000","last_peered":"2024-03-09T09:29:14.029735+0000","last_clean":"2024-03-09T09:29:14.029735+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T09:29:14.029735+0000","last_undegraded":"2024-03-09T09:29:14.029735+0000","last_fullsized":"2024-03-09T09:29:14.029735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:42:30.439855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.815577539,"stat_sum":{"num_bytes":1586992,"num_objects":105,"num_object_clones":35,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":35,"num_read":97,"num_read_kb":33815,"num_write":1338,"num_write_kb":74580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"89'546","reported_seq":716,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:03.370255+0000","last_change":"2024-03-09T08:52:17.275321+0000","last_active":"2024-03-09T09:28:03.370255+0000","last_peered":"2024-03-09T09:28:03.370255+0000","last_clean":"2024-03-09T09:28:03.370255+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T09:28:03.370255+0000","last_undegraded":"2024-03-09T09:28:03.370255+0000","last_fullsized":"2024-03-09T09:28:03.370255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":620,"num_write_kb":120324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"89'552","reported_seq":714,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.550241+0000","last_change":"2024-03-09T08:52:28.584457+0000","last_active":"2024-03-09T09:27:57.550241+0000","last_peered":"2024-03-09T09:27:57.550241+0000","last_clean":"2024-03-09T09:27:57.550241+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T09:27:57.550241+0000","last_undegraded":"2024-03-09T09:27:57.550241+0000","last_fullsized":"2024-03-09T09:27:57.550241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":563,"num_write_kb":103368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"89'2265","reported_seq":5166,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.926971+0000","last_change":"2024-03-09T09:27:31.026391+0000","last_active":"2024-03-09T09:29:17.926971+0000","last_peered":"2024-03-09T09:29:17.926971+0000","last_clean":"2024-03-09T09:29:17.926971+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T09:29:17.926971+0000","last_undegraded":"2024-03-09T09:29:17.926971+0000","last_fullsized":"2024-03-09T09:29:17.926971+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2265,"log_dups_size":0,"ondisk_log_size":2265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.71599741500000003,"stat_sum":{"num_bytes":1873672,"num_objects":101,"num_object_clones":34,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":34,"num_read":103,"num_read_kb":19624,"num_write":1351,"num_write_kb":43427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"89'2260","reported_seq":5237,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.060800+0000","last_change":"2024-03-09T09:27:08.957379+0000","last_active":"2024-03-09T09:29:14.060800+0000","last_peered":"2024-03-09T09:29:14.060800+0000","last_clean":"2024-03-09T09:29:14.060800+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T09:29:14.060800+0000","last_undegraded":"2024-03-09T09:29:14.060800+0000","last_fullsized":"2024-03-09T09:29:14.060800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2260,"log_dups_size":0,"ondisk_log_size":2260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.4919903310000002,"stat_sum":{"num_bytes":2681803,"num_objects":128,"num_object_clones":43,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":43,"num_read":104,"num_read_kb":25781,"num_write":1386,"num_write_kb":55300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"89'737","reported_seq":906,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.252650+0000","last_change":"2024-03-09T08:52:17.281921+0000","last_active":"2024-03-09T09:28:16.252650+0000","last_peered":"2024-03-09T09:28:16.252650+0000","last_clean":"2024-03-09T09:28:16.252650+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T09:28:16.252650+0000","last_undegraded":"2024-03-09T09:28:16.252650+0000","last_fullsized":"2024-03-09T09:28:16.252650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:57:04.219981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":833,"num_write_kb":103821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'609","reported_seq":780,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.150296+0000","last_change":"2024-03-09T09:29:25.150380+0000","last_active":"2024-03-09T09:29:25.150296+0000","last_peered":"2024-03-09T09:29:25.150296+0000","last_clean":"2024-03-09T09:29:25.150296+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T09:29:25.150296+0000","last_undegraded":"2024-03-09T09:29:25.150296+0000","last_fullsized":"2024-03-09T09:29:25.150296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'609","last_scrub_stamp":"2024-03-09T09:29:25.150212+0000","last_deep_scrub":"89'609","last_deep_scrub_stamp":"2024-03-09T09:29:25.150212+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.150212+0000","objects_scrubbed":48,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:04:01.814051+0000","scrub_duration":112,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":52,"num_read_kb":0,"num_write":616,"num_write_kb":119874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":438501,"num_omap_keys":702,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"89'2276","reported_seq":5223,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:24.750759+0000","last_change":"2024-03-09T09:29:24.750805+0000","last_active":"2024-03-09T09:29:24.750759+0000","last_peered":"2024-03-09T09:29:24.750759+0000","last_clean":"2024-03-09T09:29:24.750759+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T09:29:24.750759+0000","last_undegraded":"2024-03-09T09:29:24.750759+0000","last_fullsized":"2024-03-09T09:29:24.750759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2276","last_scrub_stamp":"2024-03-09T09:29:24.750679+0000","last_deep_scrub":"89'2276","last_deep_scrub_stamp":"2024-03-09T09:29:24.750679+0000","last_clean_scrub_stamp":"2024-03-09T09:29:24.750679+0000","objects_scrubbed":110,"log_size":2276,"log_dups_size":0,"ondisk_log_size":2276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:41:28.516964+0000","scrub_duration":247,"objects_trimmed":2,"snaptrim_duration":0.63049624100000001,"stat_sum":{"num_bytes":833172,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":114,"num_read_kb":17857,"num_write":1302,"num_write_kb":44488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"89'2250","reported_seq":5227,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.033364+0000","last_change":"2024-03-09T09:27:10.015883+0000","last_active":"2024-03-09T09:29:14.033364+0000","last_peered":"2024-03-09T09:29:14.033364+0000","last_clean":"2024-03-09T09:29:14.033364+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T09:29:14.033364+0000","last_undegraded":"2024-03-09T09:29:14.033364+0000","last_fullsized":"2024-03-09T09:29:14.033364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2250,"log_dups_size":0,"ondisk_log_size":2250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.18447864,"stat_sum":{"num_bytes":1122276,"num_objects":117,"num_object_clones":43,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":43,"num_read":101,"num_read_kb":18409,"num_write":1364,"num_write_kb":39400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"89'1869","reported_seq":3063,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.624099+0000","last_change":"2024-03-09T09:29:26.624152+0000","last_active":"2024-03-09T09:29:26.624099+0000","last_peered":"2024-03-09T09:29:26.624099+0000","last_clean":"2024-03-09T09:29:26.624099+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T09:29:26.624099+0000","last_undegraded":"2024-03-09T09:29:26.624099+0000","last_fullsized":"2024-03-09T09:29:26.624099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1869","last_scrub_stamp":"2024-03-09T09:29:26.624016+0000","last_deep_scrub":"89'1869","last_deep_scrub_stamp":"2024-03-09T09:29:26.624016+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.624016+0000","objects_scrubbed":39,"log_size":1869,"log_dups_size":0,"ondisk_log_size":1869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:29.163043+0000","scrub_duration":102,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":1037,"num_read_kb":4764,"num_write":1840,"num_write_kb":108217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":338965,"num_omap_keys":539,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"89'404","reported_seq":566,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:22.569287+0000","last_change":"2024-03-09T08:52:28.588636+0000","last_active":"2024-03-09T09:29:22.569287+0000","last_peered":"2024-03-09T09:29:22.569287+0000","last_clean":"2024-03-09T09:29:22.569287+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T09:29:22.569287+0000","last_undegraded":"2024-03-09T09:29:22.569287+0000","last_fullsized":"2024-03-09T09:29:22.569287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":44,"num_read_kb":0,"num_write":445,"num_write_kb":78896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"89'2184","reported_seq":5127,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924015+0000","last_change":"2024-03-09T09:27:34.760081+0000","last_active":"2024-03-09T09:29:17.924015+0000","last_peered":"2024-03-09T09:29:17.924015+0000","last_clean":"2024-03-09T09:29:17.924015+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T09:29:17.924015+0000","last_undegraded":"2024-03-09T09:29:17.924015+0000","last_fullsized":"2024-03-09T09:29:17.924015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2184,"log_dups_size":0,"ondisk_log_size":2184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:30:07.164361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4843044489999997,"stat_sum":{"num_bytes":1636574,"num_objects":123,"num_object_clones":44,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":44,"num_read":134,"num_read_kb":27719,"num_write":1302,"num_write_kb":59461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'2127","reported_seq":5121,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.055089+0000","last_change":"2024-03-09T09:27:09.413913+0000","last_active":"2024-03-09T09:29:14.055089+0000","last_peered":"2024-03-09T09:29:14.055089+0000","last_clean":"2024-03-09T09:29:14.055089+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T09:29:14.055089+0000","last_undegraded":"2024-03-09T09:29:14.055089+0000","last_fullsized":"2024-03-09T09:29:14.055089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2127,"log_dups_size":0,"ondisk_log_size":2127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.289202336,"stat_sum":{"num_bytes":424328,"num_objects":113,"num_object_clones":35,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":35,"num_read":94,"num_read_kb":17266,"num_write":1284,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"89'580","reported_seq":750,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:23.074969+0000","last_change":"2024-03-09T08:52:17.298159+0000","last_active":"2024-03-09T09:28:23.074969+0000","last_peered":"2024-03-09T09:28:23.074969+0000","last_clean":"2024-03-09T09:28:23.074969+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T09:28:23.074969+0000","last_undegraded":"2024-03-09T09:28:23.074969+0000","last_fullsized":"2024-03-09T09:28:23.074969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:15:23.532223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":598,"num_write_kb":82993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"89'1593","reported_seq":1755,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:22.635212+0000","last_change":"2024-03-09T08:52:28.578599+0000","last_active":"2024-03-09T09:29:22.635212+0000","last_peered":"2024-03-09T09:29:22.635212+0000","last_clean":"2024-03-09T09:29:22.635212+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T09:29:22.635212+0000","last_undegraded":"2024-03-09T09:29:22.635212+0000","last_fullsized":"2024-03-09T09:29:22.635212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1593,"log_dups_size":0,"ondisk_log_size":1593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":3475,"num_write_kb":84746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"89'2236","reported_seq":5167,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.928826+0000","last_change":"2024-03-09T09:27:34.687418+0000","last_active":"2024-03-09T09:29:17.928826+0000","last_peered":"2024-03-09T09:29:17.928826+0000","last_clean":"2024-03-09T09:29:17.928826+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T09:29:17.928826+0000","last_undegraded":"2024-03-09T09:29:17.928826+0000","last_fullsized":"2024-03-09T09:29:17.928826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2236,"log_dups_size":0,"ondisk_log_size":2236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:18:43.296322+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4503168610000001,"stat_sum":{"num_bytes":1665642,"num_objects":121,"num_object_clones":41,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":41,"num_read":126,"num_read_kb":42837,"num_write":1341,"num_write_kb":67584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'2308","reported_seq":5269,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.052272+0000","last_change":"2024-03-09T09:27:12.261576+0000","last_active":"2024-03-09T09:29:14.052272+0000","last_peered":"2024-03-09T09:29:14.052272+0000","last_clean":"2024-03-09T09:29:14.052272+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T09:29:14.052272+0000","last_undegraded":"2024-03-09T09:29:14.052272+0000","last_fullsized":"2024-03-09T09:29:14.052272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2308,"log_dups_size":0,"ondisk_log_size":2308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:07:46.894530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.56599842099999997,"stat_sum":{"num_bytes":1847778,"num_objects":137,"num_object_clones":49,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":49,"num_read":115,"num_read_kb":13750,"num_write":1371,"num_write_kb":55061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'626","reported_seq":795,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:11.134820+0000","last_change":"2024-03-09T08:52:17.278467+0000","last_active":"2024-03-09T09:28:11.134820+0000","last_peered":"2024-03-09T09:28:11.134820+0000","last_clean":"2024-03-09T09:28:11.134820+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T09:28:11.134820+0000","last_undegraded":"2024-03-09T09:28:11.134820+0000","last_fullsized":"2024-03-09T09:28:11.134820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:44:30.661207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":698,"num_write_kb":70663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":326,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:40.174178+0000","last_change":"2024-03-09T08:52:08.580559+0000","last_active":"2024-03-09T09:27:40.174178+0000","last_peered":"2024-03-09T09:27:40.174178+0000","last_clean":"2024-03-09T09:27:40.174178+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T09:27:40.174178+0000","last_undegraded":"2024-03-09T09:27:40.174178+0000","last_fullsized":"2024-03-09T09:27:40.174178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:06.914301+0000","last_clean_scrub_stamp":"2024-03-09T08:52:06.914301+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:02:04.356630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'680","reported_seq":842,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:31.478195+0000","last_change":"2024-03-09T08:52:28.577973+0000","last_active":"2024-03-09T09:28:31.478195+0000","last_peered":"2024-03-09T09:28:31.478195+0000","last_clean":"2024-03-09T09:28:31.478195+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T09:28:31.478195+0000","last_undegraded":"2024-03-09T09:28:31.478195+0000","last_fullsized":"2024-03-09T09:28:31.478195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":743,"num_write_kb":95766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"89'2248","reported_seq":5177,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.936635+0000","last_change":"2024-03-09T09:27:30.919984+0000","last_active":"2024-03-09T09:29:17.936635+0000","last_peered":"2024-03-09T09:29:17.936635+0000","last_clean":"2024-03-09T09:29:17.936635+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T09:29:17.936635+0000","last_undegraded":"2024-03-09T09:29:17.936635+0000","last_fullsized":"2024-03-09T09:29:17.936635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2248,"log_dups_size":0,"ondisk_log_size":2248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.66264078000000004,"stat_sum":{"num_bytes":792938,"num_objects":106,"num_object_clones":38,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":38,"num_read":125,"num_read_kb":17823,"num_write":1366,"num_write_kb":27629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'2242","reported_seq":5187,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.060277+0000","last_change":"2024-03-09T09:27:06.276304+0000","last_active":"2024-03-09T09:29:14.060277+0000","last_peered":"2024-03-09T09:29:14.060277+0000","last_clean":"2024-03-09T09:29:14.060277+0000","last_became_active":"2024-03-09T08:53:00.437859+0000","last_became_peered":"2024-03-09T08:53:00.437859+0000","last_unstale":"2024-03-09T09:29:14.060277+0000","last_undegraded":"2024-03-09T09:29:14.060277+0000","last_fullsized":"2024-03-09T09:29:14.060277+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2242,"log_dups_size":0,"ondisk_log_size":2242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.79260830400000004,"stat_sum":{"num_bytes":1456716,"num_objects":115,"num_object_clones":40,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":40,"num_read":109,"num_read_kb":18518,"num_write":1301,"num_write_kb":37816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"89'501","reported_seq":670,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:17.301885+0000","last_change":"2024-03-09T08:52:17.283654+0000","last_active":"2024-03-09T09:28:17.301885+0000","last_peered":"2024-03-09T09:28:17.301885+0000","last_clean":"2024-03-09T09:28:17.301885+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T09:28:17.301885+0000","last_undegraded":"2024-03-09T09:28:17.301885+0000","last_fullsized":"2024-03-09T09:28:17.301885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":39,"num_read_kb":0,"num_write":538,"num_write_kb":99417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"89'703","reported_seq":865,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.444243+0000","last_change":"2024-03-09T08:52:28.589278+0000","last_active":"2024-03-09T09:29:27.444243+0000","last_peered":"2024-03-09T09:29:27.444243+0000","last_clean":"2024-03-09T09:29:27.444243+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T09:29:27.444243+0000","last_undegraded":"2024-03-09T09:29:27.444243+0000","last_fullsized":"2024-03-09T09:29:27.444243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":780,"num_write_kb":116155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"89'2200","reported_seq":5149,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:24.205201+0000","last_change":"2024-03-09T09:27:30.904584+0000","last_active":"2024-03-09T09:29:24.205201+0000","last_peered":"2024-03-09T09:29:24.205201+0000","last_clean":"2024-03-09T09:29:24.205201+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T09:29:24.205201+0000","last_undegraded":"2024-03-09T09:29:24.205201+0000","last_fullsized":"2024-03-09T09:29:24.205201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2200,"log_dups_size":0,"ondisk_log_size":2200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.647865472,"stat_sum":{"num_bytes":999257,"num_objects":92,"num_object_clones":32,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":32,"num_read":116,"num_read_kb":42999,"num_write":1346,"num_write_kb":55653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"89'2187","reported_seq":5181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048275+0000","last_change":"2024-03-09T09:27:14.415883+0000","last_active":"2024-03-09T09:29:14.048275+0000","last_peered":"2024-03-09T09:29:14.048275+0000","last_clean":"2024-03-09T09:29:14.048275+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T09:29:14.048275+0000","last_undegraded":"2024-03-09T09:29:14.048275+0000","last_fullsized":"2024-03-09T09:29:14.048275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2187,"log_dups_size":0,"ondisk_log_size":2187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:34:33.950832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.951760857,"stat_sum":{"num_bytes":1333229,"num_objects":97,"num_object_clones":33,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":33,"num_read":100,"num_read_kb":17450,"num_write":1313,"num_write_kb":41729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"89'845","reported_seq":1014,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:17.557582+0000","last_change":"2024-03-09T08:52:17.286728+0000","last_active":"2024-03-09T09:28:17.557582+0000","last_peered":"2024-03-09T09:28:17.557582+0000","last_clean":"2024-03-09T09:28:17.557582+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T09:28:17.557582+0000","last_undegraded":"2024-03-09T09:28:17.557582+0000","last_fullsized":"2024-03-09T09:28:17.557582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971641,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":912,"num_write_kb":111865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"89'796","reported_seq":957,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.790607+0000","last_change":"2024-03-09T08:52:28.583181+0000","last_active":"2024-03-09T09:28:32.790607+0000","last_peered":"2024-03-09T09:28:32.790607+0000","last_clean":"2024-03-09T09:28:32.790607+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T09:28:32.790607+0000","last_undegraded":"2024-03-09T09:28:32.790607+0000","last_fullsized":"2024-03-09T09:28:32.790607+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":51,"num_read_kb":0,"num_write":872,"num_write_kb":99652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"89'2237","reported_seq":5133,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.931503+0000","last_change":"2024-03-09T09:27:34.640947+0000","last_active":"2024-03-09T09:29:17.931503+0000","last_peered":"2024-03-09T09:29:17.931503+0000","last_clean":"2024-03-09T09:29:17.931503+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T09:29:17.931503+0000","last_undegraded":"2024-03-09T09:29:17.931503+0000","last_fullsized":"2024-03-09T09:29:17.931503+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2237,"log_dups_size":0,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:51:18.312332+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4098675719999996,"stat_sum":{"num_bytes":603203,"num_objects":100,"num_object_clones":32,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":32,"num_read":95,"num_read_kb":13936,"num_write":1338,"num_write_kb":41727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"89'2197","reported_seq":5190,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.040606+0000","last_change":"2024-03-09T09:27:13.656604+0000","last_active":"2024-03-09T09:29:14.040606+0000","last_peered":"2024-03-09T09:29:14.040606+0000","last_clean":"2024-03-09T09:29:14.040606+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T09:29:14.040606+0000","last_undegraded":"2024-03-09T09:29:14.040606+0000","last_fullsized":"2024-03-09T09:29:14.040606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2197,"log_dups_size":0,"ondisk_log_size":2197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:53:35.952871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.91280589400000001,"stat_sum":{"num_bytes":4608718,"num_objects":119,"num_object_clones":37,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":37,"num_read":119,"num_read_kb":26183,"num_write":1317,"num_write_kb":37436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'672","reported_seq":841,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:18.707352+0000","last_change":"2024-03-09T08:52:17.272373+0000","last_active":"2024-03-09T09:28:18.707352+0000","last_peered":"2024-03-09T09:28:18.707352+0000","last_clean":"2024-03-09T09:28:18.707352+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T09:28:18.707352+0000","last_undegraded":"2024-03-09T09:28:18.707352+0000","last_fullsized":"2024-03-09T09:28:18.707352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":724,"num_write_kb":103814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"89'713","reported_seq":874,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:38.303247+0000","last_change":"2024-03-09T08:52:28.587637+0000","last_active":"2024-03-09T09:28:38.303247+0000","last_peered":"2024-03-09T09:28:38.303247+0000","last_clean":"2024-03-09T09:28:38.303247+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T09:28:38.303247+0000","last_undegraded":"2024-03-09T09:28:38.303247+0000","last_fullsized":"2024-03-09T09:28:38.303247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:31:32.362061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":59,"num_read_kb":0,"num_write":793,"num_write_kb":95673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"89'2359","reported_seq":5258,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:22.312448+0000","last_change":"2024-03-09T09:27:30.891372+0000","last_active":"2024-03-09T09:29:22.312448+0000","last_peered":"2024-03-09T09:29:22.312448+0000","last_clean":"2024-03-09T09:29:22.312448+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T09:29:22.312448+0000","last_undegraded":"2024-03-09T09:29:22.312448+0000","last_fullsized":"2024-03-09T09:29:22.312448+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2359,"log_dups_size":0,"ondisk_log_size":2359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.634097468,"stat_sum":{"num_bytes":572008,"num_objects":135,"num_object_clones":47,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":47,"num_read":117,"num_read_kb":9742,"num_write":1431,"num_write_kb":32918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"89'2248","reported_seq":5227,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.069831+0000","last_change":"2024-03-09T09:27:13.609914+0000","last_active":"2024-03-09T09:29:14.069831+0000","last_peered":"2024-03-09T09:29:14.069831+0000","last_clean":"2024-03-09T09:29:14.069831+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T09:29:14.069831+0000","last_undegraded":"2024-03-09T09:29:14.069831+0000","last_fullsized":"2024-03-09T09:29:14.069831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2248,"log_dups_size":0,"ondisk_log_size":2248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:55:21.022399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.81694827699999994,"stat_sum":{"num_bytes":851628,"num_objects":105,"num_object_clones":33,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":33,"num_read":114,"num_read_kb":26314,"num_write":1339,"num_write_kb":37517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"89'520","reported_seq":688,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:22.655355+0000","last_change":"2024-03-09T08:52:17.284403+0000","last_active":"2024-03-09T09:28:22.655355+0000","last_peered":"2024-03-09T09:28:22.655355+0000","last_clean":"2024-03-09T09:28:22.655355+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T09:28:22.655355+0000","last_undegraded":"2024-03-09T09:28:22.655355+0000","last_fullsized":"2024-03-09T09:28:22.655355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554454,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":604,"num_write_kb":91576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"89'683","reported_seq":845,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:22.666069+0000","last_change":"2024-03-09T08:52:28.582308+0000","last_active":"2024-03-09T09:29:22.666069+0000","last_peered":"2024-03-09T09:29:22.666069+0000","last_clean":"2024-03-09T09:29:22.666069+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T09:29:22.666069+0000","last_undegraded":"2024-03-09T09:29:22.666069+0000","last_fullsized":"2024-03-09T09:29:22.666069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":783,"num_write_kb":95503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"89'2304","reported_seq":5195,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.952036+0000","last_change":"2024-03-09T09:27:34.538055+0000","last_active":"2024-03-09T09:29:17.952036+0000","last_peered":"2024-03-09T09:29:17.952036+0000","last_clean":"2024-03-09T09:29:17.952036+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T09:29:17.952036+0000","last_undegraded":"2024-03-09T09:29:17.952036+0000","last_fullsized":"2024-03-09T09:29:17.952036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2304,"log_dups_size":0,"ondisk_log_size":2304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:30:20.481016+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.1951373749999998,"stat_sum":{"num_bytes":1358361,"num_objects":133,"num_object_clones":45,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":45,"num_read":125,"num_read_kb":25752,"num_write":1343,"num_write_kb":45310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"89'2285","reported_seq":5257,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.067779+0000","last_change":"2024-03-09T09:27:13.043792+0000","last_active":"2024-03-09T09:29:14.067779+0000","last_peered":"2024-03-09T09:29:14.067779+0000","last_clean":"2024-03-09T09:29:14.067779+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T09:29:14.067779+0000","last_undegraded":"2024-03-09T09:29:14.067779+0000","last_fullsized":"2024-03-09T09:29:14.067779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:24:58.260830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30155770100000001,"stat_sum":{"num_bytes":730111,"num_objects":106,"num_object_clones":35,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":35,"num_read":106,"num_read_kb":32982,"num_write":1381,"num_write_kb":56548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'750","reported_seq":918,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.179050+0000","last_change":"2024-03-09T08:52:17.278273+0000","last_active":"2024-03-09T09:29:14.179050+0000","last_peered":"2024-03-09T09:29:14.179050+0000","last_clean":"2024-03-09T09:29:14.179050+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T09:29:14.179050+0000","last_undegraded":"2024-03-09T09:29:14.179050+0000","last_fullsized":"2024-03-09T09:29:14.179050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":74,"num_read_kb":0,"num_write":870,"num_write_kb":87759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"89'602","reported_seq":762,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:47.793418+0000","last_change":"2024-03-09T08:52:28.575975+0000","last_active":"2024-03-09T09:28:47.793418+0000","last_peered":"2024-03-09T09:28:47.793418+0000","last_clean":"2024-03-09T09:28:47.793418+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T09:28:47.793418+0000","last_undegraded":"2024-03-09T09:28:47.793418+0000","last_fullsized":"2024-03-09T09:28:47.793418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":669,"num_write_kb":112438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"89'2184","reported_seq":5115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.919953+0000","last_change":"2024-03-09T09:27:34.711852+0000","last_active":"2024-03-09T09:29:17.919953+0000","last_peered":"2024-03-09T09:29:17.919953+0000","last_clean":"2024-03-09T09:29:17.919953+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T09:29:17.919953+0000","last_undegraded":"2024-03-09T09:29:17.919953+0000","last_fullsized":"2024-03-09T09:29:17.919953+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2184,"log_dups_size":0,"ondisk_log_size":2184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:07:14.254525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4375742420000002,"stat_sum":{"num_bytes":1378177,"num_objects":100,"num_object_clones":26,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":26,"num_read":107,"num_read_kb":18463,"num_write":1316,"num_write_kb":41976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"89'2254","reported_seq":5234,"reported_epoch":89,"state":"active+clean+scrubbing+deep","last_fresh":"2024-03-09T09:29:27.610827+0000","last_change":"2024-03-09T09:29:27.447669+0000","last_active":"2024-03-09T09:29:27.610827+0000","last_peered":"2024-03-09T09:29:27.610827+0000","last_clean":"2024-03-09T09:29:27.610827+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T09:29:27.610827+0000","last_undegraded":"2024-03-09T09:29:27.610827+0000","last_fullsized":"2024-03-09T09:29:27.610827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":73,"log_size":2254,"log_dups_size":0,"ondisk_log_size":2254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.3444653409999998,"stat_sum":{"num_bytes":3375625,"num_objects":103,"num_object_clones":31,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":31,"num_read":106,"num_read_kb":36185,"num_write":1311,"num_write_kb":52081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"89'386","reported_seq":554,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.987108+0000","last_change":"2024-03-09T08:52:17.291583+0000","last_active":"2024-03-09T09:27:57.987108+0000","last_peered":"2024-03-09T09:27:57.987108+0000","last_clean":"2024-03-09T09:27:57.987108+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T09:27:57.987108+0000","last_undegraded":"2024-03-09T09:27:57.987108+0000","last_fullsized":"2024-03-09T09:27:57.987108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":412,"num_write_kb":99378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"89'560","reported_seq":959,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.643843+0000","last_change":"2024-03-09T08:52:28.588312+0000","last_active":"2024-03-09T09:28:29.643843+0000","last_peered":"2024-03-09T09:28:29.643843+0000","last_clean":"2024-03-09T09:28:29.643843+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T09:28:29.643843+0000","last_undegraded":"2024-03-09T09:28:29.643843+0000","last_fullsized":"2024-03-09T09:28:29.643843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:14:52.895618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007622,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":290,"num_read_kb":993,"num_write":609,"num_write_kb":88247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"89'2217","reported_seq":5123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.922142+0000","last_change":"2024-03-09T09:27:34.708914+0000","last_active":"2024-03-09T09:29:17.922142+0000","last_peered":"2024-03-09T09:29:17.922142+0000","last_clean":"2024-03-09T09:29:17.922142+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T09:29:17.922142+0000","last_undegraded":"2024-03-09T09:29:17.922142+0000","last_fullsized":"2024-03-09T09:29:17.922142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2217,"log_dups_size":0,"ondisk_log_size":2217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:43:47.385372+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4357263939999996,"stat_sum":{"num_bytes":1364696,"num_objects":131,"num_object_clones":50,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":50,"num_read":112,"num_read_kb":14161,"num_write":1321,"num_write_kb":29407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'2338","reported_seq":5297,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.068878+0000","last_change":"2024-03-09T09:27:13.390531+0000","last_active":"2024-03-09T09:29:14.068878+0000","last_peered":"2024-03-09T09:29:14.068878+0000","last_clean":"2024-03-09T09:29:14.068878+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T09:29:14.068878+0000","last_undegraded":"2024-03-09T09:29:14.068878+0000","last_fullsized":"2024-03-09T09:29:14.068878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2338,"log_dups_size":0,"ondisk_log_size":2338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:00:16.543112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64721804299999997,"stat_sum":{"num_bytes":2530176,"num_objects":100,"num_object_clones":36,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":36,"num_read":119,"num_read_kb":17537,"num_write":1383,"num_write_kb":31574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'267","reported_seq":434,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:40.238636+0000","last_change":"2024-03-09T08:52:17.276879+0000","last_active":"2024-03-09T09:27:40.238636+0000","last_peered":"2024-03-09T09:27:40.238636+0000","last_clean":"2024-03-09T09:27:40.238636+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T09:27:40.238636+0000","last_undegraded":"2024-03-09T09:27:40.238636+0000","last_fullsized":"2024-03-09T09:27:40.238636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:22:38.624463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":313,"num_write_kb":62737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"89'830","reported_seq":988,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:03.286424+0000","last_change":"2024-03-09T08:52:28.585562+0000","last_active":"2024-03-09T09:29:03.286424+0000","last_peered":"2024-03-09T09:29:03.286424+0000","last_clean":"2024-03-09T09:29:03.286424+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T09:29:03.286424+0000","last_undegraded":"2024-03-09T09:29:03.286424+0000","last_fullsized":"2024-03-09T09:29:03.286424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:28:50.544795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":880,"num_write_kb":124192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"89'2228","reported_seq":5135,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.941262+0000","last_change":"2024-03-09T09:27:34.550507+0000","last_active":"2024-03-09T09:29:17.941262+0000","last_peered":"2024-03-09T09:29:17.941262+0000","last_clean":"2024-03-09T09:29:17.941262+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T09:29:17.941262+0000","last_undegraded":"2024-03-09T09:29:17.941262+0000","last_fullsized":"2024-03-09T09:29:17.941262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2228,"log_dups_size":0,"ondisk_log_size":2228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:36:18.970938+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.2113280319999999,"stat_sum":{"num_bytes":5678953,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":106,"num_read_kb":14364,"num_write":1330,"num_write_kb":47101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"89'2149","reported_seq":5156,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.042199+0000","last_change":"2024-03-09T09:27:09.238850+0000","last_active":"2024-03-09T09:29:14.042199+0000","last_peered":"2024-03-09T09:29:14.042199+0000","last_clean":"2024-03-09T09:29:14.042199+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T09:29:14.042199+0000","last_undegraded":"2024-03-09T09:29:14.042199+0000","last_fullsized":"2024-03-09T09:29:14.042199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2149,"log_dups_size":0,"ondisk_log_size":2149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.096588679999999996,"stat_sum":{"num_bytes":1266796,"num_objects":124,"num_object_clones":39,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":39,"num_read":107,"num_read_kb":17887,"num_write":1323,"num_write_kb":41057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"89'594","reported_seq":762,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:03.424325+0000","last_change":"2024-03-09T08:52:17.286821+0000","last_active":"2024-03-09T09:28:03.424325+0000","last_peered":"2024-03-09T09:28:03.424325+0000","last_clean":"2024-03-09T09:28:03.424325+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T09:28:03.424325+0000","last_undegraded":"2024-03-09T09:28:03.424325+0000","last_fullsized":"2024-03-09T09:28:03.424325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":687,"num_write_kb":95812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"89'1176","reported_seq":1335,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:29.644757+0000","last_change":"2024-03-09T08:52:28.585697+0000","last_active":"2024-03-09T09:28:29.644757+0000","last_peered":"2024-03-09T09:28:29.644757+0000","last_clean":"2024-03-09T09:28:29.644757+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T09:28:29.644757+0000","last_undegraded":"2024-03-09T09:28:29.644757+0000","last_fullsized":"2024-03-09T09:28:29.644757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":1176,"log_dups_size":0,"ondisk_log_size":1176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:50:23.095008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777306,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":1250,"num_write_kb":83773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"89'2364","reported_seq":5266,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.935770+0000","last_change":"2024-03-09T09:27:34.539439+0000","last_active":"2024-03-09T09:29:17.935770+0000","last_peered":"2024-03-09T09:29:17.935770+0000","last_clean":"2024-03-09T09:29:17.935770+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T09:29:17.935770+0000","last_undegraded":"2024-03-09T09:29:17.935770+0000","last_fullsized":"2024-03-09T09:29:17.935770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2364,"log_dups_size":0,"ondisk_log_size":2364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:04:32.627956+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2009582749999996,"stat_sum":{"num_bytes":273727,"num_objects":96,"num_object_clones":34,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":34,"num_read":130,"num_read_kb":37554,"num_write":1394,"num_write_kb":63694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"89'2221","reported_seq":5185,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.038375+0000","last_change":"2024-03-09T09:27:12.934974+0000","last_active":"2024-03-09T09:29:14.038375+0000","last_peered":"2024-03-09T09:29:14.038375+0000","last_clean":"2024-03-09T09:29:14.038375+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T09:29:14.038375+0000","last_undegraded":"2024-03-09T09:29:14.038375+0000","last_fullsized":"2024-03-09T09:29:14.038375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2221,"log_dups_size":0,"ondisk_log_size":2221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:00:09.930894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19064995900000001,"stat_sum":{"num_bytes":1055946,"num_objects":133,"num_object_clones":51,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":51,"num_read":117,"num_read_kb":18332,"num_write":1305,"num_write_kb":38006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"89'1040","reported_seq":1208,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:13.891168+0000","last_change":"2024-03-09T08:52:17.291153+0000","last_active":"2024-03-09T09:29:13.891168+0000","last_peered":"2024-03-09T09:29:13.891168+0000","last_clean":"2024-03-09T09:29:13.891168+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T09:29:13.891168+0000","last_undegraded":"2024-03-09T09:29:13.891168+0000","last_fullsized":"2024-03-09T09:29:13.891168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1040,"log_dups_size":0,"ondisk_log_size":1040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":58,"num_read_kb":0,"num_write":1092,"num_write_kb":124321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"89'340","reported_seq":509,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.177199+0000","last_change":"2024-03-09T09:29:27.177305+0000","last_active":"2024-03-09T09:29:27.177199+0000","last_peered":"2024-03-09T09:29:27.177199+0000","last_clean":"2024-03-09T09:29:27.177199+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T09:29:27.177199+0000","last_undegraded":"2024-03-09T09:29:27.177199+0000","last_fullsized":"2024-03-09T09:29:27.177199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'340","last_scrub_stamp":"2024-03-09T09:29:27.177105+0000","last_deep_scrub":"89'340","last_deep_scrub_stamp":"2024-03-09T09:29:27.177105+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.177105+0000","objects_scrubbed":43,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:44:35.621276+0000","scrub_duration":81,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":328,"num_write_kb":62283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":599772,"num_omap_keys":972,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"89'2174","reported_seq":5104,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.925089+0000","last_change":"2024-03-09T09:27:32.064467+0000","last_active":"2024-03-09T09:29:17.925089+0000","last_peered":"2024-03-09T09:29:17.925089+0000","last_clean":"2024-03-09T09:29:17.925089+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T09:29:17.925089+0000","last_undegraded":"2024-03-09T09:29:17.925089+0000","last_fullsized":"2024-03-09T09:29:17.925089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2174,"log_dups_size":0,"ondisk_log_size":2174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.695386042,"stat_sum":{"num_bytes":3385389,"num_objects":130,"num_object_clones":42,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":42,"num_read":111,"num_read_kb":40727,"num_write":1306,"num_write_kb":67855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"89'2237","reported_seq":5202,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.072392+0000","last_change":"2024-03-09T09:27:08.841098+0000","last_active":"2024-03-09T09:29:14.072392+0000","last_peered":"2024-03-09T09:29:14.072392+0000","last_clean":"2024-03-09T09:29:14.072392+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T09:29:14.072392+0000","last_undegraded":"2024-03-09T09:29:14.072392+0000","last_fullsized":"2024-03-09T09:29:14.072392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2237,"log_dups_size":0,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.3666159750000002,"stat_sum":{"num_bytes":799596,"num_objects":94,"num_object_clones":30,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":30,"num_read":103,"num_read_kb":34868,"num_write":1326,"num_write_kb":46782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'1298","reported_seq":1466,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.560338+0000","last_change":"2024-03-09T08:52:17.282006+0000","last_active":"2024-03-09T09:28:32.560338+0000","last_peered":"2024-03-09T09:28:32.560338+0000","last_clean":"2024-03-09T09:28:32.560338+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T09:28:32.560338+0000","last_undegraded":"2024-03-09T09:28:32.560338+0000","last_fullsized":"2024-03-09T09:28:32.560338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":1298,"log_dups_size":0,"ondisk_log_size":1298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":80,"num_read_kb":0,"num_write":1400,"num_write_kb":80149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"89'411","reported_seq":571,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.546412+0000","last_change":"2024-03-09T08:52:28.568794+0000","last_active":"2024-03-09T09:28:26.546412+0000","last_peered":"2024-03-09T09:28:26.546412+0000","last_clean":"2024-03-09T09:28:26.546412+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T09:28:26.546412+0000","last_undegraded":"2024-03-09T09:28:26.546412+0000","last_fullsized":"2024-03-09T09:28:26.546412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":462,"num_write_kb":95092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"89'2271","reported_seq":5193,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.904166+0000","last_change":"2024-03-09T09:27:30.874038+0000","last_active":"2024-03-09T09:29:17.904166+0000","last_peered":"2024-03-09T09:29:17.904166+0000","last_clean":"2024-03-09T09:29:17.904166+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T09:29:17.904166+0000","last_undegraded":"2024-03-09T09:29:17.904166+0000","last_fullsized":"2024-03-09T09:29:17.904166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2271,"log_dups_size":0,"ondisk_log_size":2271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64490501099999997,"stat_sum":{"num_bytes":1735737,"num_objects":103,"num_object_clones":32,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":32,"num_read":113,"num_read_kb":37575,"num_write":1379,"num_write_kb":57434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"89'2291","reported_seq":5243,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.023558+0000","last_change":"2024-03-09T09:27:09.683757+0000","last_active":"2024-03-09T09:29:14.023558+0000","last_peered":"2024-03-09T09:29:14.023558+0000","last_clean":"2024-03-09T09:29:14.023558+0000","last_became_active":"2024-03-09T08:53:00.254589+0000","last_became_peered":"2024-03-09T08:53:00.254589+0000","last_unstale":"2024-03-09T09:29:14.023558+0000","last_undegraded":"2024-03-09T09:29:14.023558+0000","last_fullsized":"2024-03-09T09:29:14.023558+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2291,"log_dups_size":0,"ondisk_log_size":2291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.471373141,"stat_sum":{"num_bytes":5513596,"num_objects":123,"num_object_clones":39,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":39,"num_read":114,"num_read_kb":29387,"num_write":1331,"num_write_kb":57801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'590","reported_seq":757,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.986880+0000","last_change":"2024-03-09T08:52:17.290509+0000","last_active":"2024-03-09T09:27:57.986880+0000","last_peered":"2024-03-09T09:27:57.986880+0000","last_clean":"2024-03-09T09:27:57.986880+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T09:27:57.986880+0000","last_undegraded":"2024-03-09T09:27:57.986880+0000","last_fullsized":"2024-03-09T09:27:57.986880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:11:47.922189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":598,"num_write_kb":111422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"89'729","reported_seq":887,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:26.544483+0000","last_change":"2024-03-09T08:52:28.583697+0000","last_active":"2024-03-09T09:28:26.544483+0000","last_peered":"2024-03-09T09:28:26.544483+0000","last_clean":"2024-03-09T09:28:26.544483+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T09:28:26.544483+0000","last_undegraded":"2024-03-09T09:28:26.544483+0000","last_fullsized":"2024-03-09T09:28:26.544483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:41.552156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":81,"num_read_kb":0,"num_write":849,"num_write_kb":91941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"89'2528","reported_seq":5332,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.925475+0000","last_change":"2024-03-09T09:27:34.669817+0000","last_active":"2024-03-09T09:29:17.925475+0000","last_peered":"2024-03-09T09:29:17.925475+0000","last_clean":"2024-03-09T09:29:17.925475+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T09:29:17.925475+0000","last_undegraded":"2024-03-09T09:29:17.925475+0000","last_fullsized":"2024-03-09T09:29:17.925475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:42:51.478391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4377723019999999,"stat_sum":{"num_bytes":1160060,"num_objects":111,"num_object_clones":35,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":35,"num_read":113,"num_read_kb":9969,"num_write":1381,"num_write_kb":17733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"89'2165","reported_seq":5161,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.053013+0000","last_change":"2024-03-09T09:27:14.293488+0000","last_active":"2024-03-09T09:29:14.053013+0000","last_peered":"2024-03-09T09:29:14.053013+0000","last_clean":"2024-03-09T09:29:14.053013+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T09:29:14.053013+0000","last_undegraded":"2024-03-09T09:29:14.053013+0000","last_fullsized":"2024-03-09T09:29:14.053013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2165,"log_dups_size":0,"ondisk_log_size":2165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:12:46.882947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.82863014,"stat_sum":{"num_bytes":1951923,"num_objects":107,"num_object_clones":29,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":29,"num_read":96,"num_read_kb":37066,"num_write":1331,"num_write_kb":53456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"89'554","reported_seq":951,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:13.891978+0000","last_change":"2024-03-09T08:52:17.294688+0000","last_active":"2024-03-09T09:29:13.891978+0000","last_peered":"2024-03-09T09:29:13.891978+0000","last_clean":"2024-03-09T09:29:13.891978+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T09:29:13.891978+0000","last_undegraded":"2024-03-09T09:29:13.891978+0000","last_fullsized":"2024-03-09T09:29:13.891978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007933,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":267,"num_read_kb":985,"num_write":604,"num_write_kb":87856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"89'461","reported_seq":619,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:55.398848+0000","last_change":"2024-03-09T08:52:28.586606+0000","last_active":"2024-03-09T09:27:55.398848+0000","last_peered":"2024-03-09T09:27:55.398848+0000","last_clean":"2024-03-09T09:27:55.398848+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T09:27:55.398848+0000","last_undegraded":"2024-03-09T09:27:55.398848+0000","last_fullsized":"2024-03-09T09:27:55.398848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:17:40.902403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":506,"num_write_kb":99411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"89'2189","reported_seq":5113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.918976+0000","last_change":"2024-03-09T09:27:30.880846+0000","last_active":"2024-03-09T09:29:17.918976+0000","last_peered":"2024-03-09T09:29:17.918976+0000","last_clean":"2024-03-09T09:29:17.918976+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T09:29:17.918976+0000","last_undegraded":"2024-03-09T09:29:17.918976+0000","last_fullsized":"2024-03-09T09:29:17.918976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2189,"log_dups_size":0,"ondisk_log_size":2189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.64610520400000004,"stat_sum":{"num_bytes":1318782,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":124,"num_read_kb":9990,"num_write":1296,"num_write_kb":22209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"89'667","reported_seq":830,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:32.790694+0000","last_change":"2024-03-09T08:52:28.586201+0000","last_active":"2024-03-09T09:28:32.790694+0000","last_peered":"2024-03-09T09:28:32.790694+0000","last_clean":"2024-03-09T09:28:32.790694+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T09:28:32.790694+0000","last_undegraded":"2024-03-09T09:28:32.790694+0000","last_fullsized":"2024-03-09T09:28:32.790694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":72,"num_read_kb":0,"num_write":777,"num_write_kb":92056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'2330","reported_seq":5319,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.253759+0000","last_change":"2024-03-09T09:29:26.253814+0000","last_active":"2024-03-09T09:29:26.253759+0000","last_peered":"2024-03-09T09:29:26.253759+0000","last_clean":"2024-03-09T09:29:26.253759+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T09:29:26.253759+0000","last_undegraded":"2024-03-09T09:29:26.253759+0000","last_fullsized":"2024-03-09T09:29:26.253759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2330","last_scrub_stamp":"2024-03-09T09:29:26.253681+0000","last_deep_scrub":"89'2330","last_deep_scrub_stamp":"2024-03-09T09:29:26.253681+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.253681+0000","objects_scrubbed":120,"log_size":2330,"log_dups_size":0,"ondisk_log_size":2330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:25:05.771093+0000","scrub_duration":172,"objects_trimmed":0,"snaptrim_duration":1.23951623,"stat_sum":{"num_bytes":1295292,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":13266,"num_write":1323,"num_write_kb":26325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"89'861","reported_seq":1028,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:16.251620+0000","last_change":"2024-03-09T08:52:17.281988+0000","last_active":"2024-03-09T09:28:16.251620+0000","last_peered":"2024-03-09T09:28:16.251620+0000","last_clean":"2024-03-09T09:28:16.251620+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T09:28:16.251620+0000","last_undegraded":"2024-03-09T09:28:16.251620+0000","last_fullsized":"2024-03-09T09:28:16.251620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:57:07.349190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":45,"num_read_kb":0,"num_write":880,"num_write_kb":119947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"89'2144","reported_seq":5083,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.921197+0000","last_change":"2024-03-09T09:27:31.015518+0000","last_active":"2024-03-09T09:29:17.921197+0000","last_peered":"2024-03-09T09:29:17.921197+0000","last_clean":"2024-03-09T09:29:17.921197+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T09:29:17.921197+0000","last_undegraded":"2024-03-09T09:29:17.921197+0000","last_fullsized":"2024-03-09T09:29:17.921197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2144,"log_dups_size":0,"ondisk_log_size":2144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.70511981300000004,"stat_sum":{"num_bytes":1915718,"num_objects":107,"num_object_clones":37,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":37,"num_read":111,"num_read_kb":24546,"num_write":1285,"num_write_kb":44187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"89'545","reported_seq":707,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:48.493055+0000","last_change":"2024-03-09T08:52:28.585631+0000","last_active":"2024-03-09T09:28:48.493055+0000","last_peered":"2024-03-09T09:28:48.493055+0000","last_clean":"2024-03-09T09:28:48.493055+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T09:28:48.493055+0000","last_undegraded":"2024-03-09T09:28:48.493055+0000","last_fullsized":"2024-03-09T09:28:48.493055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":589,"num_write_kb":82799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'2475","reported_seq":5396,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.045469+0000","last_change":"2024-03-09T09:27:12.483134+0000","last_active":"2024-03-09T09:29:14.045469+0000","last_peered":"2024-03-09T09:29:14.045469+0000","last_clean":"2024-03-09T09:29:14.045469+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T09:29:14.045469+0000","last_undegraded":"2024-03-09T09:29:14.045469+0000","last_fullsized":"2024-03-09T09:29:14.045469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2475,"log_dups_size":0,"ondisk_log_size":2475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:53:15.813937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.79565864399999997,"stat_sum":{"num_bytes":2512650,"num_objects":93,"num_object_clones":29,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":29,"num_read":114,"num_read_kb":34867,"num_write":1447,"num_write_kb":63344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'400","reported_seq":566,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:04.123705+0000","last_change":"2024-03-09T08:52:17.290634+0000","last_active":"2024-03-09T09:28:04.123705+0000","last_peered":"2024-03-09T09:28:04.123705+0000","last_clean":"2024-03-09T09:28:04.123705+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T09:28:04.123705+0000","last_undegraded":"2024-03-09T09:28:04.123705+0000","last_fullsized":"2024-03-09T09:28:04.123705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:31.159256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":480,"num_write_kb":79281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"89'2177","reported_seq":5116,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.951043+0000","last_change":"2024-03-09T09:27:32.094130+0000","last_active":"2024-03-09T09:29:17.951043+0000","last_peered":"2024-03-09T09:29:17.951043+0000","last_clean":"2024-03-09T09:29:17.951043+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T09:29:17.951043+0000","last_undegraded":"2024-03-09T09:29:17.951043+0000","last_fullsized":"2024-03-09T09:29:17.951043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2177,"log_dups_size":0,"ondisk_log_size":2177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.72318536,"stat_sum":{"num_bytes":768948,"num_objects":112,"num_object_clones":37,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":37,"num_read":126,"num_read_kb":26327,"num_write":1312,"num_write_kb":46360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"89'739","reported_seq":906,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:27:57.956420+0000","last_change":"2024-03-09T08:52:17.288234+0000","last_active":"2024-03-09T09:27:57.956420+0000","last_peered":"2024-03-09T09:27:57.956420+0000","last_clean":"2024-03-09T09:27:57.956420+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T09:27:57.956420+0000","last_undegraded":"2024-03-09T09:27:57.956420+0000","last_fullsized":"2024-03-09T09:27:57.956420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":777,"num_write_kb":95156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"89'2306","reported_seq":5292,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.064298+0000","last_change":"2024-03-09T09:27:09.781774+0000","last_active":"2024-03-09T09:29:14.064298+0000","last_peered":"2024-03-09T09:29:14.064298+0000","last_clean":"2024-03-09T09:29:14.064298+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T09:29:14.064298+0000","last_undegraded":"2024-03-09T09:29:14.064298+0000","last_fullsized":"2024-03-09T09:29:14.064298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2306,"log_dups_size":0,"ondisk_log_size":2306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65597337700000002,"stat_sum":{"num_bytes":5623986,"num_objects":128,"num_object_clones":44,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":44,"num_read":148,"num_read_kb":35274,"num_write":1372,"num_write_kb":72006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"89'471","reported_seq":630,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:48.494266+0000","last_change":"2024-03-09T08:52:28.579315+0000","last_active":"2024-03-09T09:28:48.494266+0000","last_peered":"2024-03-09T09:28:48.494266+0000","last_clean":"2024-03-09T09:28:48.494266+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T09:28:48.494266+0000","last_undegraded":"2024-03-09T09:28:48.494266+0000","last_fullsized":"2024-03-09T09:28:48.494266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":514,"num_write_kb":86887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"89'2420","reported_seq":5303,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.870852+0000","last_change":"2024-03-09T09:27:34.193425+0000","last_active":"2024-03-09T09:29:17.870852+0000","last_peered":"2024-03-09T09:29:17.870852+0000","last_clean":"2024-03-09T09:29:17.870852+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T09:29:17.870852+0000","last_undegraded":"2024-03-09T09:29:17.870852+0000","last_fullsized":"2024-03-09T09:29:17.870852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2420,"log_dups_size":0,"ondisk_log_size":2420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.9237505820000003,"stat_sum":{"num_bytes":5533714,"num_objects":147,"num_object_clones":51,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":51,"num_read":136,"num_read_kb":30612,"num_write":1437,"num_write_kb":51393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"89'621","reported_seq":786,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:23.162693+0000","last_change":"2024-03-09T08:52:17.284585+0000","last_active":"2024-03-09T09:29:23.162693+0000","last_peered":"2024-03-09T09:29:23.162693+0000","last_clean":"2024-03-09T09:29:23.162693+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T09:29:23.162693+0000","last_undegraded":"2024-03-09T09:29:23.162693+0000","last_fullsized":"2024-03-09T09:29:23.162693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:16.235613+0000","last_clean_scrub_stamp":"2024-03-09T08:52:16.235613+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:16:37.820327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17956277,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":61,"num_read_kb":0,"num_write":701,"num_write_kb":108966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"89'2233","reported_seq":5213,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.033081+0000","last_change":"2024-03-09T09:27:08.660892+0000","last_active":"2024-03-09T09:29:14.033081+0000","last_peered":"2024-03-09T09:29:14.033081+0000","last_clean":"2024-03-09T09:29:14.033081+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T09:29:14.033081+0000","last_undegraded":"2024-03-09T09:29:14.033081+0000","last_fullsized":"2024-03-09T09:29:14.033081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2233,"log_dups_size":0,"ondisk_log_size":2233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.1639723850000001,"stat_sum":{"num_bytes":538194,"num_objects":89,"num_object_clones":30,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":30,"num_read":112,"num_read_kb":30041,"num_write":1325,"num_write_kb":47681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"89'768","reported_seq":925,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:08.255955+0000","last_change":"2024-03-09T08:52:28.585877+0000","last_active":"2024-03-09T09:28:08.255955+0000","last_peered":"2024-03-09T09:28:08.255955+0000","last_clean":"2024-03-09T09:28:08.255955+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T09:28:08.255955+0000","last_undegraded":"2024-03-09T09:28:08.255955+0000","last_fullsized":"2024-03-09T09:28:08.255955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:27.534560+0000","last_clean_scrub_stamp":"2024-03-09T08:52:27.534560+0000","objects_scrubbed":0,"log_size":768,"log_dups_size":0,"ondisk_log_size":768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:30:20.036385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":79,"num_read_kb":0,"num_write":862,"num_write_kb":140804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"89'2380","reported_seq":5296,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.950338+0000","last_change":"2024-03-09T09:27:34.560887+0000","last_active":"2024-03-09T09:29:17.950338+0000","last_peered":"2024-03-09T09:29:17.950338+0000","last_clean":"2024-03-09T09:29:17.950338+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T09:29:17.950338+0000","last_undegraded":"2024-03-09T09:29:17.950338+0000","last_fullsized":"2024-03-09T09:29:17.950338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2380,"log_dups_size":0,"ondisk_log_size":2380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:20:20.590247+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2173252530000003,"stat_sum":{"num_bytes":794839,"num_objects":125,"num_object_clones":42,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":42,"num_read":135,"num_read_kb":30019,"num_write":1425,"num_write_kb":47795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"89'2255","reported_seq":5235,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.036800+0000","last_change":"2024-03-09T09:27:07.515170+0000","last_active":"2024-03-09T09:29:14.036800+0000","last_peered":"2024-03-09T09:29:14.036800+0000","last_clean":"2024-03-09T09:29:14.036800+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T09:29:14.036800+0000","last_undegraded":"2024-03-09T09:29:14.036800+0000","last_fullsized":"2024-03-09T09:29:14.036800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2255,"log_dups_size":0,"ondisk_log_size":2255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.8676528270000001,"stat_sum":{"num_bytes":501236,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":17494,"num_write":1356,"num_write_kb":32845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"89'2134","reported_seq":5064,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:22.569260+0000","last_change":"2024-03-09T09:27:32.084492+0000","last_active":"2024-03-09T09:29:22.569260+0000","last_peered":"2024-03-09T09:29:22.569260+0000","last_clean":"2024-03-09T09:29:22.569260+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T09:29:22.569260+0000","last_undegraded":"2024-03-09T09:29:22.569260+0000","last_fullsized":"2024-03-09T09:29:22.569260+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2134,"log_dups_size":0,"ondisk_log_size":2134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":1.7132004649999999,"stat_sum":{"num_bytes":1622605,"num_objects":113,"num_object_clones":36,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":36,"num_read":103,"num_read_kb":9361,"num_write":1283,"num_write_kb":37703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"89'2356","reported_seq":5313,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.037814+0000","last_change":"2024-03-09T09:27:06.219951+0000","last_active":"2024-03-09T09:29:14.037814+0000","last_peered":"2024-03-09T09:29:14.037814+0000","last_clean":"2024-03-09T09:29:14.037814+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T09:29:14.037814+0000","last_undegraded":"2024-03-09T09:29:14.037814+0000","last_fullsized":"2024-03-09T09:29:14.037814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.73410826500000004,"stat_sum":{"num_bytes":968837,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":110,"num_read_kb":13537,"num_write":1403,"num_write_kb":47183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"89'2251","reported_seq":5137,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.925278+0000","last_change":"2024-03-09T09:27:34.720190+0000","last_active":"2024-03-09T09:29:17.925278+0000","last_peered":"2024-03-09T09:29:17.925278+0000","last_clean":"2024-03-09T09:29:17.925278+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T09:29:17.925278+0000","last_undegraded":"2024-03-09T09:29:17.925278+0000","last_fullsized":"2024-03-09T09:29:17.925278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2251,"log_dups_size":0,"ondisk_log_size":2251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:58:23.785495+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.446018724,"stat_sum":{"num_bytes":5408806,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":112,"num_read_kb":29960,"num_write":1313,"num_write_kb":48116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"89'2237","reported_seq":5234,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048940+0000","last_change":"2024-03-09T09:27:14.175435+0000","last_active":"2024-03-09T09:29:14.048940+0000","last_peered":"2024-03-09T09:29:14.048940+0000","last_clean":"2024-03-09T09:29:14.048940+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T09:29:14.048940+0000","last_undegraded":"2024-03-09T09:29:14.048940+0000","last_fullsized":"2024-03-09T09:29:14.048940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2237,"log_dups_size":0,"ondisk_log_size":2237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:28:29.862968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.706678991,"stat_sum":{"num_bytes":887348,"num_objects":110,"num_object_clones":37,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":37,"num_read":123,"num_read_kb":42348,"num_write":1334,"num_write_kb":66020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"89'2119","reported_seq":5069,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:22.569184+0000","last_change":"2024-03-09T09:27:32.079328+0000","last_active":"2024-03-09T09:29:22.569184+0000","last_peered":"2024-03-09T09:29:22.569184+0000","last_clean":"2024-03-09T09:29:22.569184+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T09:29:22.569184+0000","last_undegraded":"2024-03-09T09:29:22.569184+0000","last_fullsized":"2024-03-09T09:29:22.569184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2119,"log_dups_size":0,"ondisk_log_size":2119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.708282305,"stat_sum":{"num_bytes":596753,"num_objects":112,"num_object_clones":36,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":36,"num_read":111,"num_read_kb":13558,"num_write":1290,"num_write_kb":25078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"89'2395","reported_seq":5316,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.036327+0000","last_change":"2024-03-09T09:27:09.523471+0000","last_active":"2024-03-09T09:29:14.036327+0000","last_peered":"2024-03-09T09:29:14.036327+0000","last_clean":"2024-03-09T09:29:14.036327+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T09:29:14.036327+0000","last_undegraded":"2024-03-09T09:29:14.036327+0000","last_fullsized":"2024-03-09T09:29:14.036327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2395,"log_dups_size":0,"ondisk_log_size":2395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.39778581299999999,"stat_sum":{"num_bytes":1231522,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":112,"num_read_kb":16729,"num_write":1390,"num_write_kb":42674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"89'2182","reported_seq":5127,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.920533+0000","last_change":"2024-03-09T09:27:34.631961+0000","last_active":"2024-03-09T09:29:17.920533+0000","last_peered":"2024-03-09T09:29:17.920533+0000","last_clean":"2024-03-09T09:29:17.920533+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T09:29:17.920533+0000","last_undegraded":"2024-03-09T09:29:17.920533+0000","last_fullsized":"2024-03-09T09:29:17.920533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2182,"log_dups_size":0,"ondisk_log_size":2182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:11:08.985120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4007626340000003,"stat_sum":{"num_bytes":767001,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":115,"num_read_kb":17556,"num_write":1334,"num_write_kb":40745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"89'2227","reported_seq":5191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.070029+0000","last_change":"2024-03-09T09:27:12.489198+0000","last_active":"2024-03-09T09:29:14.070029+0000","last_peered":"2024-03-09T09:29:14.070029+0000","last_clean":"2024-03-09T09:29:14.070029+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T09:29:14.070029+0000","last_undegraded":"2024-03-09T09:29:14.070029+0000","last_fullsized":"2024-03-09T09:29:14.070029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2227,"log_dups_size":0,"ondisk_log_size":2227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:48:35.331040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.80249947799999999,"stat_sum":{"num_bytes":1587591,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":108,"num_read_kb":24972,"num_write":1321,"num_write_kb":50275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"89'2422","reported_seq":5269,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.937927+0000","last_change":"2024-03-09T09:27:33.932429+0000","last_active":"2024-03-09T09:29:17.937927+0000","last_peered":"2024-03-09T09:29:17.937927+0000","last_clean":"2024-03-09T09:29:17.937927+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T09:29:17.937927+0000","last_undegraded":"2024-03-09T09:29:17.937927+0000","last_fullsized":"2024-03-09T09:29:17.937927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2422,"log_dups_size":0,"ondisk_log_size":2422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.6671456120000001,"stat_sum":{"num_bytes":827266,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":108,"num_read_kb":21834,"num_write":1396,"num_write_kb":37656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"89'2190","reported_seq":5178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.014978+0000","last_change":"2024-03-09T09:27:10.243190+0000","last_active":"2024-03-09T09:29:14.014978+0000","last_peered":"2024-03-09T09:29:14.014978+0000","last_clean":"2024-03-09T09:29:14.014978+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T09:29:14.014978+0000","last_undegraded":"2024-03-09T09:29:14.014978+0000","last_fullsized":"2024-03-09T09:29:14.014978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2190,"log_dups_size":0,"ondisk_log_size":2190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.410735039,"stat_sum":{"num_bytes":1374461,"num_objects":110,"num_object_clones":35,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":35,"num_read":104,"num_read_kb":29127,"num_write":1325,"num_write_kb":54062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"89'2222","reported_seq":5150,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.925368+0000","last_change":"2024-03-09T09:27:34.622798+0000","last_active":"2024-03-09T09:29:17.925368+0000","last_peered":"2024-03-09T09:29:17.925368+0000","last_clean":"2024-03-09T09:29:17.925368+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T09:29:17.925368+0000","last_undegraded":"2024-03-09T09:29:17.925368+0000","last_fullsized":"2024-03-09T09:29:17.925368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2222,"log_dups_size":0,"ondisk_log_size":2222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:16:59.598968+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.393423125,"stat_sum":{"num_bytes":646888,"num_objects":117,"num_object_clones":37,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":37,"num_read":111,"num_read_kb":21616,"num_write":1343,"num_write_kb":41561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"89'2380","reported_seq":5354,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.053348+0000","last_change":"2024-03-09T09:27:08.090545+0000","last_active":"2024-03-09T09:29:14.053348+0000","last_peered":"2024-03-09T09:29:14.053348+0000","last_clean":"2024-03-09T09:29:14.053348+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T09:29:14.053348+0000","last_undegraded":"2024-03-09T09:29:14.053348+0000","last_fullsized":"2024-03-09T09:29:14.053348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2380,"log_dups_size":0,"ondisk_log_size":2380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.440348755,"stat_sum":{"num_bytes":1448398,"num_objects":130,"num_object_clones":44,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":44,"num_read":125,"num_read_kb":18804,"num_write":1435,"num_write_kb":32798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"89'2308","reported_seq":5211,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.916162+0000","last_change":"2024-03-09T09:27:30.874126+0000","last_active":"2024-03-09T09:29:17.916162+0000","last_peered":"2024-03-09T09:29:17.916162+0000","last_clean":"2024-03-09T09:29:17.916162+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T09:29:17.916162+0000","last_undegraded":"2024-03-09T09:29:17.916162+0000","last_fullsized":"2024-03-09T09:29:17.916162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2308,"log_dups_size":0,"ondisk_log_size":2308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.64500296700000004,"stat_sum":{"num_bytes":1066603,"num_objects":98,"num_object_clones":32,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":32,"num_read":104,"num_read_kb":18198,"num_write":1399,"num_write_kb":47978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"89'2175","reported_seq":5136,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.046381+0000","last_change":"2024-03-09T09:27:12.333957+0000","last_active":"2024-03-09T09:29:14.046381+0000","last_peered":"2024-03-09T09:29:14.046381+0000","last_clean":"2024-03-09T09:29:14.046381+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T09:29:14.046381+0000","last_undegraded":"2024-03-09T09:29:14.046381+0000","last_fullsized":"2024-03-09T09:29:14.046381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:59:27.720581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.62298635300000005,"stat_sum":{"num_bytes":1371558,"num_objects":107,"num_object_clones":34,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":34,"num_read":92,"num_read_kb":30679,"num_write":1263,"num_write_kb":63255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"89'2203","reported_seq":5097,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:28:56.761136+0000","last_change":"2024-03-09T09:27:34.527780+0000","last_active":"2024-03-09T09:28:56.761136+0000","last_peered":"2024-03-09T09:28:56.761136+0000","last_clean":"2024-03-09T09:28:56.761136+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T09:28:56.761136+0000","last_undegraded":"2024-03-09T09:28:56.761136+0000","last_fullsized":"2024-03-09T09:28:56.761136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2203,"log_dups_size":0,"ondisk_log_size":2203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:33:42.613616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.1845584630000001,"stat_sum":{"num_bytes":1760124,"num_objects":108,"num_object_clones":35,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":35,"num_read":107,"num_read_kb":13622,"num_write":1279,"num_write_kb":27255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'2199","reported_seq":5181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.036836+0000","last_change":"2024-03-09T09:27:13.741409+0000","last_active":"2024-03-09T09:29:14.036836+0000","last_peered":"2024-03-09T09:29:14.036836+0000","last_clean":"2024-03-09T09:29:14.036836+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T09:29:14.036836+0000","last_undegraded":"2024-03-09T09:29:14.036836+0000","last_fullsized":"2024-03-09T09:29:14.036836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2199,"log_dups_size":0,"ondisk_log_size":2199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:46:03.762335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.2778432559999999,"stat_sum":{"num_bytes":1308027,"num_objects":118,"num_object_clones":42,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":42,"num_read":112,"num_read_kb":14330,"num_write":1310,"num_write_kb":25655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"89'2329","reported_seq":5225,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.875787+0000","last_change":"2024-03-09T09:27:34.655594+0000","last_active":"2024-03-09T09:29:17.875787+0000","last_peered":"2024-03-09T09:29:17.875787+0000","last_clean":"2024-03-09T09:29:17.875787+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T09:29:17.875787+0000","last_undegraded":"2024-03-09T09:29:17.875787+0000","last_fullsized":"2024-03-09T09:29:17.875787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2329,"log_dups_size":0,"ondisk_log_size":2329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:01:41.618756+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4171876489999997,"stat_sum":{"num_bytes":1480146,"num_objects":115,"num_object_clones":41,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":41,"num_read":120,"num_read_kb":31406,"num_write":1383,"num_write_kb":52041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"89'2175","reported_seq":5173,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.056157+0000","last_change":"2024-03-09T09:27:12.405710+0000","last_active":"2024-03-09T09:29:14.056157+0000","last_peered":"2024-03-09T09:29:14.056157+0000","last_clean":"2024-03-09T09:29:14.056157+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T09:29:14.056157+0000","last_undegraded":"2024-03-09T09:29:14.056157+0000","last_fullsized":"2024-03-09T09:29:14.056157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:12:10.053606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.72006586699999997,"stat_sum":{"num_bytes":650831,"num_objects":114,"num_object_clones":39,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":39,"num_read":114,"num_read_kb":46474,"num_write":1326,"num_write_kb":53541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"89'2398","reported_seq":5341,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.745167+0000","last_change":"2024-03-09T09:29:25.745227+0000","last_active":"2024-03-09T09:29:25.745167+0000","last_peered":"2024-03-09T09:29:25.745167+0000","last_clean":"2024-03-09T09:29:25.745167+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T09:29:25.745167+0000","last_undegraded":"2024-03-09T09:29:25.745167+0000","last_fullsized":"2024-03-09T09:29:25.745167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2398","last_scrub_stamp":"2024-03-09T09:29:25.745039+0000","last_deep_scrub":"89'2398","last_deep_scrub_stamp":"2024-03-09T09:29:25.745039+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.745039+0000","objects_scrubbed":112,"log_size":2398,"log_dups_size":0,"ondisk_log_size":2398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:21:28.894700+0000","scrub_duration":266,"objects_trimmed":2,"snaptrim_duration":0.64248945300000004,"stat_sum":{"num_bytes":509009,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":145,"num_read_kb":34396,"num_write":1348,"num_write_kb":44845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"89'2311","reported_seq":5257,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.050683+0000","last_change":"2024-03-09T09:27:13.760270+0000","last_active":"2024-03-09T09:29:14.050683+0000","last_peered":"2024-03-09T09:29:14.050683+0000","last_clean":"2024-03-09T09:29:14.050683+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T09:29:14.050683+0000","last_undegraded":"2024-03-09T09:29:14.050683+0000","last_fullsized":"2024-03-09T09:29:14.050683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2311,"log_dups_size":0,"ondisk_log_size":2311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:54:17.411237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.291579509,"stat_sum":{"num_bytes":917223,"num_objects":100,"num_object_clones":30,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":30,"num_read":95,"num_read_kb":13353,"num_write":1387,"num_write_kb":42086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"89'2321","reported_seq":5211,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.932450+0000","last_change":"2024-03-09T09:27:34.549097+0000","last_active":"2024-03-09T09:29:17.932450+0000","last_peered":"2024-03-09T09:29:17.932450+0000","last_clean":"2024-03-09T09:29:17.932450+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T09:29:17.932450+0000","last_undegraded":"2024-03-09T09:29:17.932450+0000","last_fullsized":"2024-03-09T09:29:17.932450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2321,"log_dups_size":0,"ondisk_log_size":2321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:56:26.561322+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.2066621079999997,"stat_sum":{"num_bytes":1356079,"num_objects":97,"num_object_clones":28,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":28,"num_read":106,"num_read_kb":29896,"num_write":1386,"num_write_kb":51017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"89'2224","reported_seq":5215,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.042715+0000","last_change":"2024-03-09T09:27:09.439110+0000","last_active":"2024-03-09T09:29:14.042715+0000","last_peered":"2024-03-09T09:29:14.042715+0000","last_clean":"2024-03-09T09:29:14.042715+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T09:29:14.042715+0000","last_undegraded":"2024-03-09T09:29:14.042715+0000","last_fullsized":"2024-03-09T09:29:14.042715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2224,"log_dups_size":0,"ondisk_log_size":2224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.313168905,"stat_sum":{"num_bytes":1092268,"num_objects":121,"num_object_clones":43,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":43,"num_read":123,"num_read_kb":25979,"num_write":1341,"num_write_kb":58516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"89'2303","reported_seq":5203,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.922701+0000","last_change":"2024-03-09T09:27:34.188251+0000","last_active":"2024-03-09T09:29:17.922701+0000","last_peered":"2024-03-09T09:29:17.922701+0000","last_clean":"2024-03-09T09:29:17.922701+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T09:29:17.922701+0000","last_undegraded":"2024-03-09T09:29:17.922701+0000","last_fullsized":"2024-03-09T09:29:17.922701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2303,"log_dups_size":0,"ondisk_log_size":2303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":3.9232747450000001,"stat_sum":{"num_bytes":5183706,"num_objects":95,"num_object_clones":27,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":111,"num_read_kb":34166,"num_write":1371,"num_write_kb":49768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"89'2247","reported_seq":5208,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.064234+0000","last_change":"2024-03-09T09:27:07.884719+0000","last_active":"2024-03-09T09:29:14.064234+0000","last_peered":"2024-03-09T09:29:14.064234+0000","last_clean":"2024-03-09T09:29:14.064234+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T09:29:14.064234+0000","last_undegraded":"2024-03-09T09:29:14.064234+0000","last_fullsized":"2024-03-09T09:29:14.064234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2247,"log_dups_size":0,"ondisk_log_size":2247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.1326348209999999,"stat_sum":{"num_bytes":1866711,"num_objects":117,"num_object_clones":40,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":40,"num_read":109,"num_read_kb":17692,"num_write":1315,"num_write_kb":41853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"89'2199","reported_seq":5136,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.935377+0000","last_change":"2024-03-09T09:27:33.862027+0000","last_active":"2024-03-09T09:29:17.935377+0000","last_peered":"2024-03-09T09:29:17.935377+0000","last_clean":"2024-03-09T09:29:17.935377+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T09:29:17.935377+0000","last_undegraded":"2024-03-09T09:29:17.935377+0000","last_fullsized":"2024-03-09T09:29:17.935377+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2199,"log_dups_size":0,"ondisk_log_size":2199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.5970959150000001,"stat_sum":{"num_bytes":1348477,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":126,"num_read_kb":8825,"num_write":1341,"num_write_kb":33612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"89'2672","reported_seq":5494,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.040015+0000","last_change":"2024-03-09T09:27:13.243644+0000","last_active":"2024-03-09T09:29:14.040015+0000","last_peered":"2024-03-09T09:29:14.040015+0000","last_clean":"2024-03-09T09:29:14.040015+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T09:29:14.040015+0000","last_undegraded":"2024-03-09T09:29:14.040015+0000","last_fullsized":"2024-03-09T09:29:14.040015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2672,"log_dups_size":0,"ondisk_log_size":2672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:32:48.806909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.49950316500000003,"stat_sum":{"num_bytes":924300,"num_objects":92,"num_object_clones":25,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":25,"num_read":107,"num_read_kb":17449,"num_write":1480,"num_write_kb":44901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"89'2175","reported_seq":5120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.873811+0000","last_change":"2024-03-09T09:27:34.679372+0000","last_active":"2024-03-09T09:29:17.873811+0000","last_peered":"2024-03-09T09:29:17.873811+0000","last_clean":"2024-03-09T09:29:17.873811+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T09:29:17.873811+0000","last_undegraded":"2024-03-09T09:29:17.873811+0000","last_fullsized":"2024-03-09T09:29:17.873811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:56.141041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4099735320000004,"stat_sum":{"num_bytes":773517,"num_objects":127,"num_object_clones":46,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":46,"num_read":120,"num_read_kb":5950,"num_write":1331,"num_write_kb":13298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"89'2162","reported_seq":5148,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.022602+0000","last_change":"2024-03-09T09:27:06.385525+0000","last_active":"2024-03-09T09:29:14.022602+0000","last_peered":"2024-03-09T09:29:14.022602+0000","last_clean":"2024-03-09T09:29:14.022602+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T09:29:14.022602+0000","last_undegraded":"2024-03-09T09:29:14.022602+0000","last_fullsized":"2024-03-09T09:29:14.022602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2162,"log_dups_size":0,"ondisk_log_size":2162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.89838860799999998,"stat_sum":{"num_bytes":6604987,"num_objects":101,"num_object_clones":30,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":30,"num_read":109,"num_read_kb":14504,"num_write":1281,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"89'2159","reported_seq":5190,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.929855+0000","last_change":"2024-03-09T09:27:31.020668+0000","last_active":"2024-03-09T09:29:17.929855+0000","last_peered":"2024-03-09T09:29:17.929855+0000","last_clean":"2024-03-09T09:29:17.929855+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T09:29:17.929855+0000","last_undegraded":"2024-03-09T09:29:17.929855+0000","last_fullsized":"2024-03-09T09:29:17.929855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2159,"log_dups_size":0,"ondisk_log_size":2159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.71096335899999996,"stat_sum":{"num_bytes":637081,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":204,"num_read_kb":24036,"num_write":1319,"num_write_kb":30441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"89'2336","reported_seq":5280,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.040723+0000","last_change":"2024-03-09T09:27:06.358258+0000","last_active":"2024-03-09T09:29:14.040723+0000","last_peered":"2024-03-09T09:29:14.040723+0000","last_clean":"2024-03-09T09:29:14.040723+0000","last_became_active":"2024-03-09T08:53:00.240319+0000","last_became_peered":"2024-03-09T08:53:00.240319+0000","last_unstale":"2024-03-09T09:29:14.040723+0000","last_undegraded":"2024-03-09T09:29:14.040723+0000","last_fullsized":"2024-03-09T09:29:14.040723+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2336,"log_dups_size":0,"ondisk_log_size":2336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.86880120699999996,"stat_sum":{"num_bytes":1614659,"num_objects":116,"num_object_clones":42,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":42,"num_read":120,"num_read_kb":18915,"num_write":1382,"num_write_kb":39172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"89'2129","reported_seq":5078,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.949389+0000","last_change":"2024-03-09T09:27:32.089100+0000","last_active":"2024-03-09T09:29:17.949389+0000","last_peered":"2024-03-09T09:29:17.949389+0000","last_clean":"2024-03-09T09:29:17.949389+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T09:29:17.949389+0000","last_undegraded":"2024-03-09T09:29:17.949389+0000","last_fullsized":"2024-03-09T09:29:17.949389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2129,"log_dups_size":0,"ondisk_log_size":2129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.7177660380000002,"stat_sum":{"num_bytes":924110,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":129,"num_read_kb":10365,"num_write":1260,"num_write_kb":33897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"89'2227","reported_seq":5204,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.029490+0000","last_change":"2024-03-09T09:27:13.674044+0000","last_active":"2024-03-09T09:29:14.029490+0000","last_peered":"2024-03-09T09:29:14.029490+0000","last_clean":"2024-03-09T09:29:14.029490+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T09:29:14.029490+0000","last_undegraded":"2024-03-09T09:29:14.029490+0000","last_fullsized":"2024-03-09T09:29:14.029490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2227,"log_dups_size":0,"ondisk_log_size":2227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:22:32.494482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.2105350770000001,"stat_sum":{"num_bytes":1326786,"num_objects":103,"num_object_clones":34,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":34,"num_read":111,"num_read_kb":18735,"num_write":1322,"num_write_kb":41042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"89'2356","reported_seq":5259,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.949219+0000","last_change":"2024-03-09T09:27:34.276120+0000","last_active":"2024-03-09T09:29:17.949219+0000","last_peered":"2024-03-09T09:29:17.949219+0000","last_clean":"2024-03-09T09:29:17.949219+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T09:29:17.949219+0000","last_undegraded":"2024-03-09T09:29:17.949219+0000","last_fullsized":"2024-03-09T09:29:17.949219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.0059579840000001,"stat_sum":{"num_bytes":936699,"num_objects":111,"num_object_clones":39,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":39,"num_read":129,"num_read_kb":34540,"num_write":1403,"num_write_kb":57337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"89'2277","reported_seq":5249,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.054685+0000","last_change":"2024-03-09T09:27:09.746181+0000","last_active":"2024-03-09T09:29:14.054685+0000","last_peered":"2024-03-09T09:29:14.054685+0000","last_clean":"2024-03-09T09:29:14.054685+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T09:29:14.054685+0000","last_undegraded":"2024-03-09T09:29:14.054685+0000","last_fullsized":"2024-03-09T09:29:14.054685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2277,"log_dups_size":0,"ondisk_log_size":2277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.91582108900000003,"stat_sum":{"num_bytes":1759504,"num_objects":101,"num_object_clones":31,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":31,"num_read":108,"num_read_kb":5845,"num_write":1367,"num_write_kb":19152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"89'2349","reported_seq":5233,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.940855+0000","last_change":"2024-03-09T09:27:34.759707+0000","last_active":"2024-03-09T09:29:17.940855+0000","last_peered":"2024-03-09T09:29:17.940855+0000","last_clean":"2024-03-09T09:29:17.940855+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T09:29:17.940855+0000","last_undegraded":"2024-03-09T09:29:17.940855+0000","last_fullsized":"2024-03-09T09:29:17.940855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2349,"log_dups_size":0,"ondisk_log_size":2349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:27:24.016743+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.4857478520000003,"stat_sum":{"num_bytes":2540930,"num_objects":122,"num_object_clones":40,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":40,"num_read":127,"num_read_kb":51469,"num_write":1352,"num_write_kb":80787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"89'2218","reported_seq":5195,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.023373+0000","last_change":"2024-03-09T09:27:09.024910+0000","last_active":"2024-03-09T09:29:14.023373+0000","last_peered":"2024-03-09T09:29:14.023373+0000","last_clean":"2024-03-09T09:29:14.023373+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T09:29:14.023373+0000","last_undegraded":"2024-03-09T09:29:14.023373+0000","last_fullsized":"2024-03-09T09:29:14.023373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2218,"log_dups_size":0,"ondisk_log_size":2218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.1939797309999998,"stat_sum":{"num_bytes":1585107,"num_objects":96,"num_object_clones":31,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":31,"num_read":96,"num_read_kb":18307,"num_write":1352,"num_write_kb":50489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"89'2682","reported_seq":5434,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.949755+0000","last_change":"2024-03-09T09:27:34.659119+0000","last_active":"2024-03-09T09:29:17.949755+0000","last_peered":"2024-03-09T09:29:17.949755+0000","last_clean":"2024-03-09T09:29:17.949755+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T09:29:17.949755+0000","last_undegraded":"2024-03-09T09:29:17.949755+0000","last_fullsized":"2024-03-09T09:29:17.949755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2682,"log_dups_size":0,"ondisk_log_size":2682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:45:21.245870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4197025170000002,"stat_sum":{"num_bytes":1559225,"num_objects":122,"num_object_clones":42,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":42,"num_read":122,"num_read_kb":6495,"num_write":1460,"num_write_kb":38738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"89'2225","reported_seq":5197,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.005606+0000","last_change":"2024-03-09T09:27:12.627461+0000","last_active":"2024-03-09T09:29:14.005606+0000","last_peered":"2024-03-09T09:29:14.005606+0000","last_clean":"2024-03-09T09:29:14.005606+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T09:29:14.005606+0000","last_undegraded":"2024-03-09T09:29:14.005606+0000","last_fullsized":"2024-03-09T09:29:14.005606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2225,"log_dups_size":0,"ondisk_log_size":2225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:03:50.159643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.94133736700000004,"stat_sum":{"num_bytes":314683,"num_objects":117,"num_object_clones":39,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":39,"num_read":116,"num_read_kb":13620,"num_write":1329,"num_write_kb":42753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"89'2198","reported_seq":5208,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.950711+0000","last_change":"2024-03-09T09:27:34.517246+0000","last_active":"2024-03-09T09:29:17.950711+0000","last_peered":"2024-03-09T09:29:17.950711+0000","last_clean":"2024-03-09T09:29:17.950711+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T09:29:17.950711+0000","last_undegraded":"2024-03-09T09:29:17.950711+0000","last_fullsized":"2024-03-09T09:29:17.950711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2198,"log_dups_size":0,"ondisk_log_size":2198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:23:51.461775+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.1785381859999999,"stat_sum":{"num_bytes":1532500,"num_objects":95,"num_object_clones":30,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":30,"num_read":195,"num_read_kb":18044,"num_write":1327,"num_write_kb":39414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"89'2266","reported_seq":5236,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.069275+0000","last_change":"2024-03-09T09:27:08.772433+0000","last_active":"2024-03-09T09:29:14.069275+0000","last_peered":"2024-03-09T09:29:14.069275+0000","last_clean":"2024-03-09T09:29:14.069275+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T09:29:14.069275+0000","last_undegraded":"2024-03-09T09:29:14.069275+0000","last_fullsized":"2024-03-09T09:29:14.069275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2266,"log_dups_size":0,"ondisk_log_size":2266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.298540461,"stat_sum":{"num_bytes":2039183,"num_objects":130,"num_object_clones":43,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":43,"num_read":126,"num_read_kb":25377,"num_write":1340,"num_write_kb":50110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"89'2209","reported_seq":5129,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.945010+0000","last_change":"2024-03-09T09:27:30.886579+0000","last_active":"2024-03-09T09:29:17.945010+0000","last_peered":"2024-03-09T09:29:17.945010+0000","last_clean":"2024-03-09T09:29:17.945010+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T09:29:17.945010+0000","last_undegraded":"2024-03-09T09:29:17.945010+0000","last_fullsized":"2024-03-09T09:29:17.945010+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2209,"log_dups_size":0,"ondisk_log_size":2209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.62967200199999995,"stat_sum":{"num_bytes":347019,"num_objects":120,"num_object_clones":41,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":41,"num_read":119,"num_read_kb":17844,"num_write":1302,"num_write_kb":51829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"89'2658","reported_seq":5497,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.057293+0000","last_change":"2024-03-09T09:27:07.715360+0000","last_active":"2024-03-09T09:29:14.057293+0000","last_peered":"2024-03-09T09:29:14.057293+0000","last_clean":"2024-03-09T09:29:14.057293+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T09:29:14.057293+0000","last_undegraded":"2024-03-09T09:29:14.057293+0000","last_fullsized":"2024-03-09T09:29:14.057293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2658,"log_dups_size":0,"ondisk_log_size":2658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.0660642299999998,"stat_sum":{"num_bytes":1365000,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":127,"num_read_kb":33469,"num_write":1475,"num_write_kb":57156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"89'2231","reported_seq":5157,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.942922+0000","last_change":"2024-03-09T09:27:34.649642+0000","last_active":"2024-03-09T09:29:17.942922+0000","last_peered":"2024-03-09T09:29:17.942922+0000","last_clean":"2024-03-09T09:29:17.942922+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T09:29:17.942922+0000","last_undegraded":"2024-03-09T09:29:17.942922+0000","last_fullsized":"2024-03-09T09:29:17.942922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2231,"log_dups_size":0,"ondisk_log_size":2231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:47:53.123132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4162208879999998,"stat_sum":{"num_bytes":1481141,"num_objects":109,"num_object_clones":37,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":37,"num_read":120,"num_read_kb":25693,"num_write":1333,"num_write_kb":50297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'2192","reported_seq":5172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.049921+0000","last_change":"2024-03-09T09:27:08.029887+0000","last_active":"2024-03-09T09:29:14.049921+0000","last_peered":"2024-03-09T09:29:14.049921+0000","last_clean":"2024-03-09T09:29:14.049921+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T09:29:14.049921+0000","last_undegraded":"2024-03-09T09:29:14.049921+0000","last_fullsized":"2024-03-09T09:29:14.049921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2192,"log_dups_size":0,"ondisk_log_size":2192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.379810049,"stat_sum":{"num_bytes":472147,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":115,"num_read_kb":25809,"num_write":1277,"num_write_kb":34061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"89'2310","reported_seq":5232,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924102+0000","last_change":"2024-03-09T09:27:34.694598+0000","last_active":"2024-03-09T09:29:17.924102+0000","last_peered":"2024-03-09T09:29:17.924102+0000","last_clean":"2024-03-09T09:29:17.924102+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T09:29:17.924102+0000","last_undegraded":"2024-03-09T09:29:17.924102+0000","last_fullsized":"2024-03-09T09:29:17.924102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2310,"log_dups_size":0,"ondisk_log_size":2310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:24:52.867402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4238349550000002,"stat_sum":{"num_bytes":1560364,"num_objects":99,"num_object_clones":31,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":31,"num_read":120,"num_read_kb":23106,"num_write":1406,"num_write_kb":36148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"89'2355","reported_seq":5316,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.061430+0000","last_change":"2024-03-09T09:27:13.474431+0000","last_active":"2024-03-09T09:29:14.061430+0000","last_peered":"2024-03-09T09:29:14.061430+0000","last_clean":"2024-03-09T09:29:14.061430+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T09:29:14.061430+0000","last_undegraded":"2024-03-09T09:29:14.061430+0000","last_fullsized":"2024-03-09T09:29:14.061430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2355,"log_dups_size":0,"ondisk_log_size":2355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:37:53.961909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.73072901000000001,"stat_sum":{"num_bytes":1439516,"num_objects":106,"num_object_clones":37,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":37,"num_read":117,"num_read_kb":43194,"num_write":1390,"num_write_kb":63251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"89'2112","reported_seq":5035,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.954079+0000","last_change":"2024-03-09T09:27:34.559512+0000","last_active":"2024-03-09T09:29:17.954079+0000","last_peered":"2024-03-09T09:29:17.954079+0000","last_clean":"2024-03-09T09:29:17.954079+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T09:29:17.954079+0000","last_undegraded":"2024-03-09T09:29:17.954079+0000","last_fullsized":"2024-03-09T09:29:17.954079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2112,"log_dups_size":0,"ondisk_log_size":2112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:12:19.227405+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2163343510000004,"stat_sum":{"num_bytes":526581,"num_objects":91,"num_object_clones":29,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":29,"num_read":100,"num_read_kb":25429,"num_write":1233,"num_write_kb":64615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"89'2159","reported_seq":5133,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.052874+0000","last_change":"2024-03-09T09:27:12.157156+0000","last_active":"2024-03-09T09:29:14.052874+0000","last_peered":"2024-03-09T09:29:14.052874+0000","last_clean":"2024-03-09T09:29:14.052874+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T09:29:14.052874+0000","last_undegraded":"2024-03-09T09:29:14.052874+0000","last_fullsized":"2024-03-09T09:29:14.052874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2159,"log_dups_size":0,"ondisk_log_size":2159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:45:37.914970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.46979842799999999,"stat_sum":{"num_bytes":631595,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":98,"num_read_kb":25968,"num_write":1286,"num_write_kb":41427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"89'2137","reported_seq":5077,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.927451+0000","last_change":"2024-03-09T09:27:30.899203+0000","last_active":"2024-03-09T09:29:17.927451+0000","last_peered":"2024-03-09T09:29:17.927451+0000","last_clean":"2024-03-09T09:29:17.927451+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T09:29:17.927451+0000","last_undegraded":"2024-03-09T09:29:17.927451+0000","last_fullsized":"2024-03-09T09:29:17.927451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2137,"log_dups_size":0,"ondisk_log_size":2137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.66506321800000001,"stat_sum":{"num_bytes":592790,"num_objects":103,"num_object_clones":35,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":35,"num_read":105,"num_read_kb":21771,"num_write":1301,"num_write_kb":41126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"89'2204","reported_seq":5160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.048992+0000","last_change":"2024-03-09T09:27:13.380861+0000","last_active":"2024-03-09T09:29:14.048992+0000","last_peered":"2024-03-09T09:29:14.048992+0000","last_clean":"2024-03-09T09:29:14.048992+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T09:29:14.048992+0000","last_undegraded":"2024-03-09T09:29:14.048992+0000","last_fullsized":"2024-03-09T09:29:14.048992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2204,"log_dups_size":0,"ondisk_log_size":2204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:02:11.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.638707739,"stat_sum":{"num_bytes":1008318,"num_objects":81,"num_object_clones":24,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":24,"num_read":86,"num_read_kb":17604,"num_write":1317,"num_write_kb":38401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"89'2298","reported_seq":5190,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.921307+0000","last_change":"2024-03-09T09:27:32.075213+0000","last_active":"2024-03-09T09:29:17.921307+0000","last_peered":"2024-03-09T09:29:17.921307+0000","last_clean":"2024-03-09T09:29:17.921307+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T09:29:17.921307+0000","last_undegraded":"2024-03-09T09:29:17.921307+0000","last_fullsized":"2024-03-09T09:29:17.921307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2298,"log_dups_size":0,"ondisk_log_size":2298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":1.705573912,"stat_sum":{"num_bytes":534756,"num_objects":106,"num_object_clones":34,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":34,"num_read":108,"num_read_kb":25061,"num_write":1338,"num_write_kb":44408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"89'2111","reported_seq":5114,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.032543+0000","last_change":"2024-03-09T09:27:13.610566+0000","last_active":"2024-03-09T09:29:14.032543+0000","last_peered":"2024-03-09T09:29:14.032543+0000","last_clean":"2024-03-09T09:29:14.032543+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T09:29:14.032543+0000","last_undegraded":"2024-03-09T09:29:14.032543+0000","last_fullsized":"2024-03-09T09:29:14.032543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2111,"log_dups_size":0,"ondisk_log_size":2111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:17:33.229502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.1420017650000001,"stat_sum":{"num_bytes":1651219,"num_objects":118,"num_object_clones":44,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":44,"num_read":114,"num_read_kb":27021,"num_write":1266,"num_write_kb":36457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"89'2279","reported_seq":5203,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924096+0000","last_change":"2024-03-09T09:27:34.669037+0000","last_active":"2024-03-09T09:29:17.924096+0000","last_peered":"2024-03-09T09:29:17.924096+0000","last_clean":"2024-03-09T09:29:17.924096+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T09:29:17.924096+0000","last_undegraded":"2024-03-09T09:29:17.924096+0000","last_fullsized":"2024-03-09T09:29:17.924096+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2279,"log_dups_size":0,"ondisk_log_size":2279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:54:35.897455+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.3954307559999997,"stat_sum":{"num_bytes":1015705,"num_objects":121,"num_object_clones":38,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":38,"num_read":132,"num_read_kb":25133,"num_write":1351,"num_write_kb":59371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"89'2361","reported_seq":5314,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.067546+0000","last_change":"2024-03-09T09:27:13.261980+0000","last_active":"2024-03-09T09:29:14.067546+0000","last_peered":"2024-03-09T09:29:14.067546+0000","last_clean":"2024-03-09T09:29:14.067546+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T09:29:14.067546+0000","last_undegraded":"2024-03-09T09:29:14.067546+0000","last_fullsized":"2024-03-09T09:29:14.067546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2361,"log_dups_size":0,"ondisk_log_size":2361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:13:56.252007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.51876684699999998,"stat_sum":{"num_bytes":1407157,"num_objects":117,"num_object_clones":38,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":38,"num_read":113,"num_read_kb":14121,"num_write":1423,"num_write_kb":30929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"89'2145","reported_seq":5075,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.939348+0000","last_change":"2024-03-09T09:27:34.506277+0000","last_active":"2024-03-09T09:29:17.939348+0000","last_peered":"2024-03-09T09:29:17.939348+0000","last_clean":"2024-03-09T09:29:17.939348+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T09:29:17.939348+0000","last_undegraded":"2024-03-09T09:29:17.939348+0000","last_fullsized":"2024-03-09T09:29:17.939348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2145,"log_dups_size":0,"ondisk_log_size":2145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:27.408652+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.1683436250000003,"stat_sum":{"num_bytes":2080769,"num_objects":116,"num_object_clones":43,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":43,"num_read":114,"num_read_kb":27228,"num_write":1275,"num_write_kb":54719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"89'2227","reported_seq":5198,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.040802+0000","last_change":"2024-03-09T09:27:13.892143+0000","last_active":"2024-03-09T09:29:14.040802+0000","last_peered":"2024-03-09T09:29:14.040802+0000","last_clean":"2024-03-09T09:29:14.040802+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T09:29:14.040802+0000","last_undegraded":"2024-03-09T09:29:14.040802+0000","last_fullsized":"2024-03-09T09:29:14.040802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2227,"log_dups_size":0,"ondisk_log_size":2227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:36:18.464529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.4282657350000001,"stat_sum":{"num_bytes":1076453,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":106,"num_read_kb":19463,"num_write":1318,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"89'2416","reported_seq":5294,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.940399+0000","last_change":"2024-03-09T09:27:30.899184+0000","last_active":"2024-03-09T09:29:17.940399+0000","last_peered":"2024-03-09T09:29:17.940399+0000","last_clean":"2024-03-09T09:29:17.940399+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T09:29:17.940399+0000","last_undegraded":"2024-03-09T09:29:17.940399+0000","last_fullsized":"2024-03-09T09:29:17.940399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2416,"log_dups_size":0,"ondisk_log_size":2416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.63967549000000001,"stat_sum":{"num_bytes":3580963,"num_objects":116,"num_object_clones":37,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":37,"num_read":116,"num_read_kb":32310,"num_write":1440,"num_write_kb":66236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"89'2251","reported_seq":5232,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.030395+0000","last_change":"2024-03-09T09:27:09.597741+0000","last_active":"2024-03-09T09:29:14.030395+0000","last_peered":"2024-03-09T09:29:14.030395+0000","last_clean":"2024-03-09T09:29:14.030395+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T09:29:14.030395+0000","last_undegraded":"2024-03-09T09:29:14.030395+0000","last_fullsized":"2024-03-09T09:29:14.030395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2251,"log_dups_size":0,"ondisk_log_size":2251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.47268980999999999,"stat_sum":{"num_bytes":1488567,"num_objects":112,"num_object_clones":34,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":34,"num_read":108,"num_read_kb":30029,"num_write":1350,"num_write_kb":42424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"89'2142","reported_seq":5072,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.933559+0000","last_change":"2024-03-09T09:27:34.605106+0000","last_active":"2024-03-09T09:29:17.933559+0000","last_peered":"2024-03-09T09:29:17.933559+0000","last_clean":"2024-03-09T09:29:17.933559+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T09:29:17.933559+0000","last_undegraded":"2024-03-09T09:29:17.933559+0000","last_fullsized":"2024-03-09T09:29:17.933559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2142,"log_dups_size":0,"ondisk_log_size":2142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:04:07.487361+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.3763956530000003,"stat_sum":{"num_bytes":1440979,"num_objects":98,"num_object_clones":30,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":30,"num_read":104,"num_read_kb":22606,"num_write":1302,"num_write_kb":46651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"89'2066","reported_seq":5059,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.021741+0000","last_change":"2024-03-09T09:27:10.176463+0000","last_active":"2024-03-09T09:29:14.021741+0000","last_peered":"2024-03-09T09:29:14.021741+0000","last_clean":"2024-03-09T09:29:14.021741+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T09:29:14.021741+0000","last_undegraded":"2024-03-09T09:29:14.021741+0000","last_fullsized":"2024-03-09T09:29:14.021741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2066,"log_dups_size":0,"ondisk_log_size":2066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.344208157,"stat_sum":{"num_bytes":485403,"num_objects":85,"num_object_clones":27,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":27,"num_read":96,"num_read_kb":5635,"num_write":1204,"num_write_kb":28568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"89'2216","reported_seq":5128,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.931323+0000","last_change":"2024-03-09T09:27:34.669203+0000","last_active":"2024-03-09T09:29:17.931323+0000","last_peered":"2024-03-09T09:29:17.931323+0000","last_clean":"2024-03-09T09:29:17.931323+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T09:29:17.931323+0000","last_undegraded":"2024-03-09T09:29:17.931323+0000","last_fullsized":"2024-03-09T09:29:17.931323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2216,"log_dups_size":0,"ondisk_log_size":2216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:04:18.586097+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.4378791440000001,"stat_sum":{"num_bytes":906976,"num_objects":87,"num_object_clones":27,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":27,"num_read":110,"num_read_kb":26703,"num_write":1307,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"89'2145","reported_seq":5141,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.051697+0000","last_change":"2024-03-09T09:27:13.826300+0000","last_active":"2024-03-09T09:29:14.051697+0000","last_peered":"2024-03-09T09:29:14.051697+0000","last_clean":"2024-03-09T09:29:14.051697+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T09:29:14.051697+0000","last_undegraded":"2024-03-09T09:29:14.051697+0000","last_fullsized":"2024-03-09T09:29:14.051697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2145,"log_dups_size":0,"ondisk_log_size":2145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:23:44.879214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.0826407490000001,"stat_sum":{"num_bytes":992571,"num_objects":97,"num_object_clones":32,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":32,"num_read":100,"num_read_kb":32945,"num_write":1310,"num_write_kb":54952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7626,"num_read_kb":1526336,"num_write":85965,"num_write_kb":2933421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144667,"ondisk_log_size":144667,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":876698128,"num_objects":1506,"num_object_clones":0,"num_object_copies":3012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1506,"num_whiteouts":0,"num_read":2924,"num_read_kb":5807,"num_write":25879,"num_write_kb":3144377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1225,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2342287,"num_omap_keys":3913,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1754079232,"data_stored":1754019992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22381,"ondisk_log_size":22381,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7230,"num_read_kb":1519173,"num_write":85971,"num_write_kb":2933405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144493,"ondisk_log_size":144493,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798940451,"num_objects":1475,"num_object_clones":0,"num_object_copies":2950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1475,"num_whiteouts":0,"num_read":2854,"num_read_kb":5749,"num_write":25875,"num_write_kb":3105441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1213,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":338965,"num_omap_keys":539,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597313024,"data_stored":1597253779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22208,"ondisk_log_size":22208,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575299,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":851964,"kb_used_data":501556,"kb_used_omap":1,"kb_used_meta":350398,"kb_avail":93519876,"statfs":{"total":96636764160,"available":95764353024,"internally_reserved":0,"allocated":513593344,"data_stored":506816772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":358808010},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575301,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699740,"kb_used_data":456276,"kb_used_omap":1,"kb_used_meta":243454,"kb_avail":93672100,"statfs":{"total":96636764160,"available":95920230400,"internally_reserved":0,"allocated":467226624,"data_stored":462408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":249297350},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575298,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":795896,"kb_used_data":453808,"kb_used_omap":1,"kb_used_meta":342078,"kb_avail":93575944,"statfs":{"total":96636764160,"available":95821766656,"internally_reserved":0,"allocated":464699392,"data_stored":457504945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":350288325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575300,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":853316,"kb_used_data":551484,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":93518524,"statfs":{"total":96636764160,"available":95762968576,"internally_reserved":0,"allocated":564719616,"data_stored":557367936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309066181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575300,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":918744,"kb_used_data":544784,"kb_used_omap":1,"kb_used_meta":373950,"kb_avail":93453096,"statfs":{"total":96636764160,"available":95695970304,"internally_reserved":0,"allocated":557858816,"data_stored":550354976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":382925255},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575300,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454632,"kb_used_data":225056,"kb_used_omap":1,"kb_used_meta":229566,"kb_avail":93917208,"statfs":{"total":96636764160,"available":96171220992,"internally_reserved":0,"allocated":230457344,"data_stored":225591238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235076037},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575300,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":972996,"kb_used_data":612924,"kb_used_omap":1,"kb_used_meta":360062,"kb_avail":93398844,"statfs":{"total":96636764160,"available":95640416256,"internally_reserved":0,"allocated":627634176,"data_stored":621572263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":368703941},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,5,5,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575301,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671628,"kb_used_data":379588,"kb_used_omap":1,"kb_used_meta":292030,"kb_avail":93700212,"statfs":{"total":96636764160,"available":95949017088,"internally_reserved":0,"allocated":388698112,"data_stored":382376610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":299039175},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":226504704,"data_stored":226493054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":218112000,"data_stored":218104019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92274688,"data_stored":92274688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":173154304,"data_stored":173146197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":201338880,"data_stored":201326769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":197980160,"data_stored":197974487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":223150080,"data_stored":223140893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":264798208,"data_stored":264793672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37781504,"data_stored":34251073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20246528,"data_stored":18282047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22822912,"data_stored":20809621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27308032,"data_stored":23727598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37437440,"data_stored":33951899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":27983872,"data_stored":24565897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23818240,"data_stored":21628413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":29773824,"data_stored":26493862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":99909632,"data_stored":99908372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":360722432,"data_stored":360710838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96473088,"data_stored":96469310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":315428864,"data_stored":315415292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":289415168,"data_stored":289407328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":201203712,"data_stored":201196076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":201334784,"data_stored":201327208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":189591552,"data_stored":189585568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22835200,"data_stored":20540995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27480064,"data_stored":23882611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17813504,"data_stored":15444871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40894464,"data_stored":37473141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35454976,"data_stored":32089192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":37081088,"data_stored":33799473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17850368,"data_stored":15719573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":27762688,"data_stored":24760554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T09:29:29.231 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.231 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.231 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.231 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.232 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.232 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.232 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.232 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.232 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.232 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.233 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.233 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.234 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.234 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.234 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.234 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.235 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.235 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.235 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.235 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.235 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.235 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.236 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.236 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.236 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.236 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.236 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.236 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.236 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.237 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.237 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.237 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.237 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.237 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.237 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.238 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.238 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.238 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.238 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.238 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.238 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.239 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.239 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.239 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.239 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.239 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.239 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.240 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.240 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.240 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.240 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.240 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.240 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.241 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.241 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.241 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.241 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.241 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-09T08:52:06.914301+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=6, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.242 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.242 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.242 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.242 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.242 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.242 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.242 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.243 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.243 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.243 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.243 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.244 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.244 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.244 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.244 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.244 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.244 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.245 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.245 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.245 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.245 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.245 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.245 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.246 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.246 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.246 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.246 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.247 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.247 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.247 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.247 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.247 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.247 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.248 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.248 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.248 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.248 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.248 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.248 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.249 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.249 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.249 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.249 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.249 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.249 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.250 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.250 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.250 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.250 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.250 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.250 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.251 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.251 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.251 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.251 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.251 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.251 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.252 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.252 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.252 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.252 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-03-09T08:52:16.235613+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=16, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.252 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.252 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-03-09T08:52:27.534560+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=27, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.253 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.253 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.253 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.253 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.253 INFO:tasks.ceph:pgid 5.24 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.253 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.253 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.254 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.254 INFO:tasks.ceph:pgid 5.22 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.254 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.254 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.254 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.255 INFO:tasks.ceph:pgid 5.20 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.255 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.255 INFO:tasks.ceph:pgid 5.21 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.255 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.256 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.256 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.256 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.256 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.256 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.256 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.257 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.257 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.257 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.257 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.257 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.257 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.258 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.258 INFO:tasks.ceph:pgid 5.29 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.258 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.258 INFO:tasks.ceph:pgid 5.36 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.258 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.259 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.259 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.259 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.259 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.259 INFO:tasks.ceph:pgid 5.35 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.259 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.260 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.260 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.260 INFO:tasks.ceph:pgid 5.33 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.260 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.260 INFO:tasks.ceph:pgid 5.30 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.260 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.260 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.261 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.261 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.261 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.261 INFO:tasks.ceph:pgid 5.3f last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.261 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.261 INFO:tasks.ceph:pgid 5.3c last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.262 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.262 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.262 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.262 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.262 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.262 INFO:tasks.ceph:pgid 5.3b last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.263 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.263 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.263 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.263 INFO:tasks.ceph:pgid 5.39 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.263 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:29.263 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-09T09:29:34.670 DEBUG:teuthology.orchestra.run.smithi067:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:29:34.680 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-09T09:29:34.787 INFO:tasks.ceph.mds.d.smithi134.stderr:2024-03-09T09:29:34.780+0000 7f9245e00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 49838) UID: 0 2024-03-09T09:29:34.788 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-03-09T09:29:34.780+0000 7efd53a00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 49838) UID: 0 2024-03-09T09:29:49.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T09:29:49.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.509+0000 7f3555800640 1 Processor -- start 2024-03-09T09:29:49.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.509+0000 7f3555800640 1 -- start start 2024-03-09T09:29:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.510+0000 7f3555800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f35500ac730 0x7f3550071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.510+0000 7f3555800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f35500c22a0 0x7f35500acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.510+0000 7f3555800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3550157650 0x7f3550157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.510+0000 7f3555800640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3550070ee0 con 0x7f35500c22a0 2024-03-09T09:29:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.510+0000 7f3555800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3550154d00 con 0x7f35500ac730 2024-03-09T09:29:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.510+0000 7f3555800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3550153d40 con 0x7f3550157650 2024-03-09T09:29:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.511+0000 7f3547e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f35500c22a0 0x7f35500acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.511+0000 7f3547e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f35500c22a0 0x7f35500acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38020/0 (socket says 172.21.15.67:38020) 2024-03-09T09:29:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.511+0000 7f3547e00640 1 -- 172.21.15.67:0/3073753484 learned_addr learned my addr 172.21.15.67:0/3073753484 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.511+0000 7f354ea00640 1 --2- 172.21.15.67:0/3073753484 >> v2:172.21.15.134:3300/0 conn(0x7f35500ac730 0x7f3550071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.511+0000 7f354f400640 1 --2- 172.21.15.67:0/3073753484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3550157650 0x7f3550157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.511+0000 7f3547e00640 1 -- 172.21.15.67:0/3073753484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3550157650 msgr2=0x7f3550157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.511+0000 7f3547e00640 1 --2- 172.21.15.67:0/3073753484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3550157650 0x7f3550157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.511+0000 7f3547e00640 1 -- 172.21.15.67:0/3073753484 >> v2:172.21.15.134:3300/0 conn(0x7f35500ac730 msgr2=0x7f3550071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:29:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.511+0000 7f3547e00640 1 --2- 172.21.15.67:0/3073753484 >> v2:172.21.15.134:3300/0 conn(0x7f35500ac730 0x7f3550071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.511+0000 7f3547e00640 1 -- 172.21.15.67:0/3073753484 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35500c2790 con 0x7f35500c22a0 2024-03-09T09:29:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.511+0000 7f354f400640 1 --2- 172.21.15.67:0/3073753484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3550157650 0x7f3550157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:29:49.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.512+0000 7f3547e00640 1 --2- 172.21.15.67:0/3073753484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35500c22a0 0x7f35500acb10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f35500ad380 tx=0x7f353802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a63de531bdc38f server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:49.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.513+0000 7f354e000640 1 -- 172.21.15.67:0/3073753484 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538035020 con 0x7f35500c22a0 2024-03-09T09:29:49.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.513+0000 7f354e000640 1 -- 172.21.15.67:0/3073753484 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3538031870 con 0x7f35500c22a0 2024-03-09T09:29:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.513+0000 7f354e000640 1 -- 172.21.15.67:0/3073753484 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538031c40 con 0x7f35500c22a0 2024-03-09T09:29:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.513+0000 7f3555800640 1 -- 172.21.15.67:0/3073753484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35500c22a0 msgr2=0x7f35500acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.513+0000 7f3555800640 1 --2- 172.21.15.67:0/3073753484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35500c22a0 0x7f35500acb10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f35500ad380 tx=0x7f353802b910 comp rx=0 tx=0).stop 2024-03-09T09:29:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.513+0000 7f3555800640 1 -- 172.21.15.67:0/3073753484 shutdown_connections 2024-03-09T09:29:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.513+0000 7f3555800640 1 --2- 172.21.15.67:0/3073753484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3550157650 0x7f3550157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.513+0000 7f3555800640 1 --2- 172.21.15.67:0/3073753484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35500c22a0 0x7f35500acb10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.513+0000 7f3555800640 1 --2- 172.21.15.67:0/3073753484 >> v2:172.21.15.134:3300/0 conn(0x7f35500ac730 0x7f3550071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.513+0000 7f3555800640 1 -- 172.21.15.67:0/3073753484 >> 172.21.15.67:0/3073753484 conn(0x7f35500c7370 msgr2=0x7f35500c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:49.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.513+0000 7f3555800640 1 -- 172.21.15.67:0/3073753484 shutdown_connections 2024-03-09T09:29:49.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.514+0000 7f3555800640 1 -- 172.21.15.67:0/3073753484 wait complete. 2024-03-09T09:29:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.515+0000 7f3555800640 1 Processor -- start 2024-03-09T09:29:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.515+0000 7f3555800640 1 -- start start 2024-03-09T09:29:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.516+0000 7f3555800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35500ac730 0x7f35500a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.516+0000 7f3555800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35500c22a0 0x7f355009d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.516+0000 7f3555800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3550157650 0x7f3550156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.516+0000 7f3555800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3550158020 con 0x7f3550157650 2024-03-09T09:29:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.516+0000 7f3555800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3550070c00 con 0x7f35500c22a0 2024-03-09T09:29:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.516+0000 7f3555800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35501581a0 con 0x7f35500ac730 2024-03-09T09:29:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.516+0000 7f354f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3550157650 0x7f3550156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.517+0000 7f354f400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3550157650 0x7f3550156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38034/0 (socket says 172.21.15.67:38034) 2024-03-09T09:29:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.517+0000 7f354f400640 1 -- 172.21.15.67:0/626734687 learned_addr learned my addr 172.21.15.67:0/626734687 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:29:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.517+0000 7f3547e00640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35500c22a0 0x7f355009d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.517+0000 7f354ea00640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35500ac730 0x7f35500a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.517+0000 7f354f400640 1 -- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35500ac730 msgr2=0x7f35500a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.517+0000 7f354f400640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35500ac730 0x7f35500a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.517+0000 7f354f400640 1 -- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35500c22a0 msgr2=0x7f355009d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.517+0000 7f354f400640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35500c22a0 0x7f355009d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.517+0000 7f354f400640 1 -- 172.21.15.67:0/626734687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3550155ac0 con 0x7f3550157650 2024-03-09T09:29:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.517+0000 7f354f400640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3550157650 0x7f3550156280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f354800cef0 tx=0x7f354800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.518+0000 7f3546a00640 1 -- 172.21.15.67:0/626734687 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354801a070 con 0x7f3550157650 2024-03-09T09:29:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.518+0000 7f3546a00640 1 -- 172.21.15.67:0/626734687 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354800d860 con 0x7f3550157650 2024-03-09T09:29:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.518+0000 7f3555800640 1 -- 172.21.15.67:0/626734687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355009c440 con 0x7f3550157650 2024-03-09T09:29:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.518+0000 7f3546a00640 1 -- 172.21.15.67:0/626734687 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354800dc50 con 0x7f3550157650 2024-03-09T09:29:49.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.518+0000 7f3555800640 1 -- 172.21.15.67:0/626734687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355009ca00 con 0x7f3550157650 2024-03-09T09:29:49.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.519+0000 7f3555800640 1 -- 172.21.15.67:0/626734687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3550075a00 con 0x7f3550157650 2024-03-09T09:29:49.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.521+0000 7f3546a00640 1 -- 172.21.15.67:0/626734687 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f354801d070 con 0x7f3550157650 2024-03-09T09:29:49.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.522+0000 7f3546a00640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f352c06b4d0 0x7f352c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:29:49.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.522+0000 7f3546a00640 1 -- 172.21.15.67:0/626734687 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f354808adf0 con 0x7f3550157650 2024-03-09T09:29:49.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.523+0000 7f354ea00640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f352c06b4d0 0x7f352c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:29:49.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.524+0000 7f354ea00640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f352c06b4d0 0x7f352c06d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3550157fd0 tx=0x7f3540009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:29:49.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.528+0000 7f3546a00640 1 -- 172.21.15.67:0/626734687 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3548059310 con 0x7f3550157650 2024-03-09T09:29:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.741+0000 7f3555800640 1 -- 172.21.15.67:0/626734687 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35500c1d70 con 0x7f352c06b4d0 2024-03-09T09:29:49.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.778+0000 7f3546a00640 1 -- 172.21.15.67:0/626734687 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+501198 (secure 0 0 0) 0x7f35500c1d70 con 0x7f352c06b4d0 2024-03-09T09:29:49.792 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:29:49.797 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T09:29:49.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 -- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f352c06b4d0 msgr2=0x7f352c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:49.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f352c06b4d0 0x7f352c06d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3550157fd0 tx=0x7f3540009190 comp rx=0 tx=0).stop 2024-03-09T09:29:49.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 -- 172.21.15.67:0/626734687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3550157650 msgr2=0x7f3550156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:29:49.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3550157650 0x7f3550156280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f354800cef0 tx=0x7f354800a710 comp rx=0 tx=0).stop 2024-03-09T09:29:49.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 -- 172.21.15.67:0/626734687 shutdown_connections 2024-03-09T09:29:49.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f352c06b4d0 0x7f352c06d990 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:49.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3550157650 0x7f3550156280 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:49.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35500c22a0 0x7f355009d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 --2- 172.21.15.67:0/626734687 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35500ac730 0x7f35500a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:29:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 -- 172.21.15.67:0/626734687 >> 172.21.15.67:0/626734687 conn(0x7f35500c7370 msgr2=0x7f35500c36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:29:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 -- 172.21.15.67:0/626734687 shutdown_connections 2024-03-09T09:29:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:29:49.789+0000 7f3555800640 1 -- 172.21.15.67:0/626734687 wait complete. 2024-03-09T09:29:49.821 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":1213,"stamp":"2024-03-09T09:29:49.724453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1880769962,"num_objects":17182,"num_object_clones":4728,"num_object_copies":34364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17182,"num_whiteouts":4728,"num_read":20750,"num_read_kb":3057165,"num_write":223858,"num_write_kb":12119877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2438,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":35042970,"num_omap_keys":55074,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347904,"ondisk_log_size":347904,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6265568,"kb_used_data":3727712,"kb_used_omap":12,"kb_used_meta":2537779,"kb_avail":748709152,"statfs":{"total":773094113280,"available":766678171648,"internally_reserved":0,"allocated":3817177088,"data_stored":3766281908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2598686258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":206869,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12415770,"num_omap_keys":19418,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004366"},"pg_stats":[{"pgid":"5.18","version":"89'2564","reported_seq":5449,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.802823+0000","last_change":"2024-03-09T09:29:40.802889+0000","last_active":"2024-03-09T09:29:40.802823+0000","last_peered":"2024-03-09T09:29:40.802823+0000","last_clean":"2024-03-09T09:29:40.802823+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T09:29:40.802823+0000","last_undegraded":"2024-03-09T09:29:40.802823+0000","last_fullsized":"2024-03-09T09:29:40.802823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2564","last_scrub_stamp":"2024-03-09T09:29:40.802693+0000","last_deep_scrub":"89'2564","last_deep_scrub_stamp":"2024-03-09T09:29:40.802693+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.802693+0000","objects_scrubbed":90,"log_size":2564,"log_dups_size":0,"ondisk_log_size":2564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:20:02.649663+0000","scrub_duration":130,"objects_trimmed":4,"snaptrim_duration":3.854819714,"stat_sum":{"num_bytes":1582215,"num_objects":90,"num_object_clones":27,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":27,"num_read":122,"num_read_kb":30313,"num_write":1455,"num_write_kb":61176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"89'487","reported_seq":667,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.108739+0000","last_change":"2024-03-09T09:29:29.108739+0000","last_active":"2024-03-09T09:29:29.108739+0000","last_peered":"2024-03-09T09:29:29.108739+0000","last_clean":"2024-03-09T09:29:29.108739+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T09:29:29.108739+0000","last_undegraded":"2024-03-09T09:29:29.108739+0000","last_fullsized":"2024-03-09T09:29:29.108739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'487","last_scrub_stamp":"2024-03-09T09:29:29.108443+0000","last_deep_scrub":"89'487","last_deep_scrub_stamp":"2024-03-09T09:29:29.108443+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.108443+0000","objects_scrubbed":38,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:47:26.177138+0000","scrub_duration":132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":462,"num_write_kb":82632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":421963,"num_omap_keys":720,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"89'2290","reported_seq":5283,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.579752+0000","last_change":"2024-03-09T09:29:44.579843+0000","last_active":"2024-03-09T09:29:44.579752+0000","last_peered":"2024-03-09T09:29:44.579752+0000","last_clean":"2024-03-09T09:29:44.579752+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T09:29:44.579752+0000","last_undegraded":"2024-03-09T09:29:44.579752+0000","last_fullsized":"2024-03-09T09:29:44.579752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2290","last_scrub_stamp":"2024-03-09T09:29:44.579609+0000","last_deep_scrub":"89'2290","last_deep_scrub_stamp":"2024-03-09T09:29:44.579609+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.579609+0000","objects_scrubbed":111,"log_size":2290,"log_dups_size":0,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:24:46.179406+0000","scrub_duration":302,"objects_trimmed":0,"snaptrim_duration":0.47085273999999999,"stat_sum":{"num_bytes":1473782,"num_objects":111,"num_object_clones":38,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":38,"num_read":99,"num_read_kb":25344,"num_write":1311,"num_write_kb":54508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"89'448","reported_seq":616,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.368720+0000","last_change":"2024-03-09T09:29:28.368794+0000","last_active":"2024-03-09T09:29:28.368720+0000","last_peered":"2024-03-09T09:29:28.368720+0000","last_clean":"2024-03-09T09:29:28.368720+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T09:29:28.368720+0000","last_undegraded":"2024-03-09T09:29:28.368720+0000","last_fullsized":"2024-03-09T09:29:28.368720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'448","last_scrub_stamp":"2024-03-09T09:29:28.368593+0000","last_deep_scrub":"89'448","last_deep_scrub_stamp":"2024-03-09T09:29:28.368593+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.368593+0000","objects_scrubbed":43,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:36:37.605429+0000","scrub_duration":83,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":445,"num_write_kb":78871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":504839,"num_omap_keys":800,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"89'2374","reported_seq":5299,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.106756+0000","last_change":"2024-03-09T09:29:41.106834+0000","last_active":"2024-03-09T09:29:41.106756+0000","last_peered":"2024-03-09T09:29:41.106756+0000","last_clean":"2024-03-09T09:29:41.106756+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T09:29:41.106756+0000","last_undegraded":"2024-03-09T09:29:41.106756+0000","last_fullsized":"2024-03-09T09:29:41.106756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2374","last_scrub_stamp":"2024-03-09T09:29:41.106670+0000","last_deep_scrub":"89'2374","last_deep_scrub_stamp":"2024-03-09T09:29:41.106670+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.106670+0000","objects_scrubbed":118,"log_size":2374,"log_dups_size":0,"ondisk_log_size":2374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:05:13.031039+0000","scrub_duration":227,"objects_trimmed":4,"snaptrim_duration":1.703796324,"stat_sum":{"num_bytes":1252433,"num_objects":118,"num_object_clones":38,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":38,"num_read":118,"num_read_kb":38794,"num_write":1352,"num_write_kb":71340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"89'791","reported_seq":970,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.456369+0000","last_change":"2024-03-09T09:29:39.456535+0000","last_active":"2024-03-09T09:29:39.456369+0000","last_peered":"2024-03-09T09:29:39.456369+0000","last_clean":"2024-03-09T09:29:39.456369+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T09:29:39.456369+0000","last_undegraded":"2024-03-09T09:29:39.456369+0000","last_fullsized":"2024-03-09T09:29:39.456369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'791","last_scrub_stamp":"2024-03-09T09:29:39.456268+0000","last_deep_scrub":"89'791","last_deep_scrub_stamp":"2024-03-09T09:29:39.456268+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.456268+0000","objects_scrubbed":47,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:12:28.443175+0000","scrub_duration":172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":51,"num_read_kb":0,"num_write":808,"num_write_kb":91456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":528589,"num_omap_keys":713,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'2268","reported_seq":5267,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.213457+0000","last_change":"2024-03-09T09:29:27.213491+0000","last_active":"2024-03-09T09:29:27.213457+0000","last_peered":"2024-03-09T09:29:27.213457+0000","last_clean":"2024-03-09T09:29:27.213457+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T09:29:27.213457+0000","last_undegraded":"2024-03-09T09:29:27.213457+0000","last_fullsized":"2024-03-09T09:29:27.213457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2268","last_scrub_stamp":"2024-03-09T09:29:27.213384+0000","last_deep_scrub":"89'2268","last_deep_scrub_stamp":"2024-03-09T09:29:27.213384+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.213384+0000","objects_scrubbed":108,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:16:20.677968+0000","scrub_duration":271,"objects_trimmed":0,"snaptrim_duration":0.48044756999999999,"stat_sum":{"num_bytes":498978,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":114,"num_read_kb":18251,"num_write":1268,"num_write_kb":37672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"89'943","reported_seq":1113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.925803+0000","last_change":"2024-03-09T09:29:31.925832+0000","last_active":"2024-03-09T09:29:31.925803+0000","last_peered":"2024-03-09T09:29:31.925803+0000","last_clean":"2024-03-09T09:29:31.925803+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T09:29:31.925803+0000","last_undegraded":"2024-03-09T09:29:31.925803+0000","last_fullsized":"2024-03-09T09:29:31.925803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'943","last_scrub_stamp":"2024-03-09T09:29:31.925738+0000","last_deep_scrub":"89'943","last_deep_scrub_stamp":"2024-03-09T09:29:31.925738+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.925738+0000","objects_scrubbed":51,"log_size":943,"log_dups_size":0,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:10:40.595121+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":939,"num_write_kb":120134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":539982,"num_omap_keys":943,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"89'2266","reported_seq":5170,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.927796+0000","last_change":"2024-03-09T09:27:34.614436+0000","last_active":"2024-03-09T09:29:17.927796+0000","last_peered":"2024-03-09T09:29:17.927796+0000","last_clean":"2024-03-09T09:29:17.927796+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T09:29:17.927796+0000","last_undegraded":"2024-03-09T09:29:17.927796+0000","last_fullsized":"2024-03-09T09:29:17.927796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2266,"log_dups_size":0,"ondisk_log_size":2266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.3826170339999999,"stat_sum":{"num_bytes":1068833,"num_objects":92,"num_object_clones":30,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":30,"num_read":126,"num_read_kb":14571,"num_write":1308,"num_write_kb":45011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"89'732","reported_seq":910,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.392894+0000","last_change":"2024-03-09T09:29:45.392894+0000","last_active":"2024-03-09T09:29:45.392894+0000","last_peered":"2024-03-09T09:29:45.392894+0000","last_clean":"2024-03-09T09:29:45.392894+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T09:29:45.392894+0000","last_undegraded":"2024-03-09T09:29:45.392894+0000","last_fullsized":"2024-03-09T09:29:45.392894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'732","last_scrub_stamp":"2024-03-09T09:29:45.392683+0000","last_deep_scrub":"89'732","last_deep_scrub_stamp":"2024-03-09T09:29:45.392683+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.392683+0000","objects_scrubbed":58,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:57:01.257852+0000","scrub_duration":161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":67,"num_read_kb":0,"num_write":775,"num_write_kb":140765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":732837,"num_omap_keys":1110,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"89'2527","reported_seq":5505,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.126493+0000","last_change":"2024-03-09T09:29:46.126540+0000","last_active":"2024-03-09T09:29:46.126493+0000","last_peered":"2024-03-09T09:29:46.126493+0000","last_clean":"2024-03-09T09:29:46.126493+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T09:29:46.126493+0000","last_undegraded":"2024-03-09T09:29:46.126493+0000","last_fullsized":"2024-03-09T09:29:46.126493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2527","last_scrub_stamp":"2024-03-09T09:29:46.126370+0000","last_deep_scrub":"89'2527","last_deep_scrub_stamp":"2024-03-09T09:29:46.126370+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.126370+0000","objects_scrubbed":148,"log_size":2527,"log_dups_size":0,"ondisk_log_size":2527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:41:22.100316+0000","scrub_duration":208,"objects_trimmed":0,"snaptrim_duration":1.653915281,"stat_sum":{"num_bytes":2793747,"num_objects":148,"num_object_clones":53,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":53,"num_read":138,"num_read_kb":27086,"num_write":1428,"num_write_kb":43660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"89'568","reported_seq":736,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.841589+0000","last_change":"2024-03-09T09:29:28.841659+0000","last_active":"2024-03-09T09:29:28.841589+0000","last_peered":"2024-03-09T09:29:28.841589+0000","last_clean":"2024-03-09T09:29:28.841589+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T09:29:28.841589+0000","last_undegraded":"2024-03-09T09:29:28.841589+0000","last_fullsized":"2024-03-09T09:29:28.841589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'568","last_scrub_stamp":"2024-03-09T09:29:28.841466+0000","last_deep_scrub":"89'568","last_deep_scrub_stamp":"2024-03-09T09:29:28.841466+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.841466+0000","objects_scrubbed":37,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:56:34.500330+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":543,"num_write_kb":111605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":558669,"num_omap_keys":858,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"89'2275","reported_seq":5174,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.921584+0000","last_change":"2024-03-09T09:27:34.731982+0000","last_active":"2024-03-09T09:29:17.921584+0000","last_peered":"2024-03-09T09:29:17.921584+0000","last_clean":"2024-03-09T09:29:17.921584+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T09:29:17.921584+0000","last_undegraded":"2024-03-09T09:29:17.921584+0000","last_fullsized":"2024-03-09T09:29:17.921584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2275,"log_dups_size":0,"ondisk_log_size":2275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4571779879999998,"stat_sum":{"num_bytes":3086231,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":121,"num_read_kb":9749,"num_write":1336,"num_write_kb":28981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"89'508","reported_seq":685,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.327430+0000","last_change":"2024-03-09T09:29:44.327430+0000","last_active":"2024-03-09T09:29:44.327430+0000","last_peered":"2024-03-09T09:29:44.327430+0000","last_clean":"2024-03-09T09:29:44.327430+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T09:29:44.327430+0000","last_undegraded":"2024-03-09T09:29:44.327430+0000","last_fullsized":"2024-03-09T09:29:44.327430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'508","last_scrub_stamp":"2024-03-09T09:29:44.327194+0000","last_deep_scrub":"89'508","last_deep_scrub_stamp":"2024-03-09T09:29:44.327194+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.327194+0000","objects_scrubbed":49,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:18:14.903140+0000","scrub_duration":110,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":55,"num_read_kb":0,"num_write":535,"num_write_kb":83331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":648870,"num_omap_keys":703,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'2269","reported_seq":5276,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.123887+0000","last_change":"2024-03-09T09:29:44.123958+0000","last_active":"2024-03-09T09:29:44.123887+0000","last_peered":"2024-03-09T09:29:44.123887+0000","last_clean":"2024-03-09T09:29:44.123887+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T09:29:44.123887+0000","last_undegraded":"2024-03-09T09:29:44.123887+0000","last_fullsized":"2024-03-09T09:29:44.123887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2269","last_scrub_stamp":"2024-03-09T09:29:44.123763+0000","last_deep_scrub":"89'2269","last_deep_scrub_stamp":"2024-03-09T09:29:44.123763+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.123763+0000","objects_scrubbed":111,"log_size":2269,"log_dups_size":0,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:07:13.959027+0000","scrub_duration":192,"objects_trimmed":0,"snaptrim_duration":1.3715434019999999,"stat_sum":{"num_bytes":1052383,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":116,"num_read_kb":6444,"num_write":1289,"num_write_kb":22757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"89'611","reported_seq":779,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.897913+0000","last_change":"2024-03-09T09:29:25.898016+0000","last_active":"2024-03-09T09:29:25.897913+0000","last_peered":"2024-03-09T09:29:25.897913+0000","last_clean":"2024-03-09T09:29:25.897913+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T09:29:25.897913+0000","last_undegraded":"2024-03-09T09:29:25.897913+0000","last_fullsized":"2024-03-09T09:29:25.897913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'611","last_scrub_stamp":"2024-03-09T09:29:25.897713+0000","last_deep_scrub":"89'611","last_deep_scrub_stamp":"2024-03-09T09:29:25.897713+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.897713+0000","objects_scrubbed":42,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:53:23.202311+0000","scrub_duration":135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":598,"num_write_kb":95071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":469023,"num_omap_keys":788,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"89'2251","reported_seq":5198,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.085473+0000","last_change":"2024-03-09T09:29:45.085541+0000","last_active":"2024-03-09T09:29:45.085473+0000","last_peered":"2024-03-09T09:29:45.085473+0000","last_clean":"2024-03-09T09:29:45.085473+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T09:29:45.085473+0000","last_undegraded":"2024-03-09T09:29:45.085473+0000","last_fullsized":"2024-03-09T09:29:45.085473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2251","last_scrub_stamp":"2024-03-09T09:29:45.085394+0000","last_deep_scrub":"89'2251","last_deep_scrub_stamp":"2024-03-09T09:29:45.085394+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.085394+0000","objects_scrubbed":92,"log_size":2251,"log_dups_size":0,"ondisk_log_size":2251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:42:36.622585+0000","scrub_duration":182,"objects_trimmed":0,"snaptrim_duration":1.709485658,"stat_sum":{"num_bytes":1197226,"num_objects":92,"num_object_clones":31,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":31,"num_read":105,"num_read_kb":17945,"num_write":1287,"num_write_kb":45184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"89'2324","reported_seq":5304,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.482498+0000","last_change":"2024-03-09T09:29:37.482591+0000","last_active":"2024-03-09T09:29:37.482498+0000","last_peered":"2024-03-09T09:29:37.482498+0000","last_clean":"2024-03-09T09:29:37.482498+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T09:29:37.482498+0000","last_undegraded":"2024-03-09T09:29:37.482498+0000","last_fullsized":"2024-03-09T09:29:37.482498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2324","last_scrub_stamp":"2024-03-09T09:29:37.482415+0000","last_deep_scrub":"89'2324","last_deep_scrub_stamp":"2024-03-09T09:29:37.482415+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.482415+0000","objects_scrubbed":98,"log_size":2324,"log_dups_size":0,"ondisk_log_size":2324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:49:14.376917+0000","scrub_duration":143,"objects_trimmed":0,"snaptrim_duration":2.2630355259999999,"stat_sum":{"num_bytes":1174090,"num_objects":98,"num_object_clones":35,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":35,"num_read":98,"num_read_kb":18109,"num_write":1319,"num_write_kb":36158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"89'864","reported_seq":1042,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.253889+0000","last_change":"2024-03-09T09:29:42.253951+0000","last_active":"2024-03-09T09:29:42.253889+0000","last_peered":"2024-03-09T09:29:42.253889+0000","last_clean":"2024-03-09T09:29:42.253889+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T09:29:42.253889+0000","last_undegraded":"2024-03-09T09:29:42.253889+0000","last_fullsized":"2024-03-09T09:29:42.253889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'864","last_scrub_stamp":"2024-03-09T09:29:42.253811+0000","last_deep_scrub":"89'864","last_deep_scrub_stamp":"2024-03-09T09:29:42.253811+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.253811+0000","objects_scrubbed":47,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:07:03.471127+0000","scrub_duration":94,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":48,"num_read_kb":0,"num_write":879,"num_write_kb":99632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":529913,"num_omap_keys":803,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"89'799","reported_seq":970,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.775492+0000","last_change":"2024-03-09T09:29:39.775537+0000","last_active":"2024-03-09T09:29:39.775492+0000","last_peered":"2024-03-09T09:29:39.775492+0000","last_clean":"2024-03-09T09:29:39.775492+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T09:29:39.775492+0000","last_undegraded":"2024-03-09T09:29:39.775492+0000","last_fullsized":"2024-03-09T09:29:39.775492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'799","last_scrub_stamp":"2024-03-09T09:29:39.775375+0000","last_deep_scrub":"89'799","last_deep_scrub_stamp":"2024-03-09T09:29:39.775375+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.775375+0000","objects_scrubbed":56,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:32:10.366074+0000","scrub_duration":115,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971838,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":820,"num_write_kb":111945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":502543,"num_omap_keys":872,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"89'2302","reported_seq":5225,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.744480+0000","last_change":"2024-03-09T09:29:34.744539+0000","last_active":"2024-03-09T09:29:34.744480+0000","last_peered":"2024-03-09T09:29:34.744480+0000","last_clean":"2024-03-09T09:29:34.744480+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T09:29:34.744480+0000","last_undegraded":"2024-03-09T09:29:34.744480+0000","last_fullsized":"2024-03-09T09:29:34.744480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2302","last_scrub_stamp":"2024-03-09T09:29:34.744358+0000","last_deep_scrub":"89'2302","last_deep_scrub_stamp":"2024-03-09T09:29:34.744358+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.744358+0000","objects_scrubbed":108,"log_size":2302,"log_dups_size":0,"ondisk_log_size":2302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:07:38.416270+0000","scrub_duration":208,"objects_trimmed":0,"snaptrim_duration":0.63600609399999997,"stat_sum":{"num_bytes":1331264,"num_objects":108,"num_object_clones":38,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":38,"num_read":110,"num_read_kb":9978,"num_write":1295,"num_write_kb":30848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'2372","reported_seq":5361,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.562406+0000","last_change":"2024-03-09T09:29:32.562470+0000","last_active":"2024-03-09T09:29:32.562406+0000","last_peered":"2024-03-09T09:29:32.562406+0000","last_clean":"2024-03-09T09:29:32.562406+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T09:29:32.562406+0000","last_undegraded":"2024-03-09T09:29:32.562406+0000","last_fullsized":"2024-03-09T09:29:32.562406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2372","last_scrub_stamp":"2024-03-09T09:29:32.562331+0000","last_deep_scrub":"89'2372","last_deep_scrub_stamp":"2024-03-09T09:29:32.562331+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.562331+0000","objects_scrubbed":108,"log_size":2372,"log_dups_size":0,"ondisk_log_size":2372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:16:09.656513+0000","scrub_duration":204,"objects_trimmed":0,"snaptrim_duration":0.69905185199999997,"stat_sum":{"num_bytes":1031167,"num_objects":108,"num_object_clones":39,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":39,"num_read":109,"num_read_kb":22315,"num_write":1384,"num_write_kb":51621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'490","reported_seq":669,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.108374+0000","last_change":"2024-03-09T09:29:33.108485+0000","last_active":"2024-03-09T09:29:33.108374+0000","last_peered":"2024-03-09T09:29:33.108374+0000","last_clean":"2024-03-09T09:29:33.108374+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T09:29:33.108374+0000","last_undegraded":"2024-03-09T09:29:33.108374+0000","last_fullsized":"2024-03-09T09:29:33.108374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'490","last_scrub_stamp":"2024-03-09T09:29:33.108300+0000","last_deep_scrub":"89'490","last_deep_scrub_stamp":"2024-03-09T09:29:33.108300+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.108300+0000","objects_scrubbed":45,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:23:03.366878+0000","scrub_duration":150,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":467,"num_write_kb":115624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":462381,"num_omap_keys":698,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"89'670","reported_seq":840,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.910521+0000","last_change":"2024-03-09T09:29:26.910625+0000","last_active":"2024-03-09T09:29:26.910521+0000","last_peered":"2024-03-09T09:29:26.910521+0000","last_clean":"2024-03-09T09:29:26.910521+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T09:29:26.910521+0000","last_undegraded":"2024-03-09T09:29:26.910521+0000","last_fullsized":"2024-03-09T09:29:26.910521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'670","last_scrub_stamp":"2024-03-09T09:29:26.910351+0000","last_deep_scrub":"89'670","last_deep_scrub_stamp":"2024-03-09T09:29:26.910351+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.910351+0000","objects_scrubbed":37,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:30:18.446605+0000","scrub_duration":145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":674,"num_write_kb":99647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":379703,"num_omap_keys":669,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"89'2368","reported_seq":5299,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.656669+0000","last_change":"2024-03-09T09:29:45.656725+0000","last_active":"2024-03-09T09:29:45.656669+0000","last_peered":"2024-03-09T09:29:45.656669+0000","last_clean":"2024-03-09T09:29:45.656669+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T09:29:45.656669+0000","last_undegraded":"2024-03-09T09:29:45.656669+0000","last_fullsized":"2024-03-09T09:29:45.656669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2368","last_scrub_stamp":"2024-03-09T09:29:45.656570+0000","last_deep_scrub":"89'2368","last_deep_scrub_stamp":"2024-03-09T09:29:45.656570+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.656570+0000","objects_scrubbed":107,"log_size":2368,"log_dups_size":0,"ondisk_log_size":2368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:26:53.091676+0000","scrub_duration":136,"objects_trimmed":4,"snaptrim_duration":3.9938263410000001,"stat_sum":{"num_bytes":1661343,"num_objects":107,"num_object_clones":33,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":33,"num_read":118,"num_read_kb":35803,"num_write":1365,"num_write_kb":51532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'2483","reported_seq":5480,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.520518+0000","last_change":"2024-03-09T09:29:42.520557+0000","last_active":"2024-03-09T09:29:42.520518+0000","last_peered":"2024-03-09T09:29:42.520518+0000","last_clean":"2024-03-09T09:29:42.520518+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T09:29:42.520518+0000","last_undegraded":"2024-03-09T09:29:42.520518+0000","last_fullsized":"2024-03-09T09:29:42.520518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2483","last_scrub_stamp":"2024-03-09T09:29:42.520449+0000","last_deep_scrub":"89'2483","last_deep_scrub_stamp":"2024-03-09T09:29:42.520449+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.520449+0000","objects_scrubbed":132,"log_size":2483,"log_dups_size":0,"ondisk_log_size":2483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:55:49.425811+0000","scrub_duration":222,"objects_trimmed":0,"snaptrim_duration":0.380889329,"stat_sum":{"num_bytes":1156693,"num_objects":132,"num_object_clones":46,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":46,"num_read":122,"num_read_kb":38030,"num_write":1453,"num_write_kb":61557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"89'690","reported_seq":869,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.979699+0000","last_change":"2024-03-09T09:29:36.979792+0000","last_active":"2024-03-09T09:29:36.979699+0000","last_peered":"2024-03-09T09:29:36.979699+0000","last_clean":"2024-03-09T09:29:36.979699+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T09:29:36.979699+0000","last_undegraded":"2024-03-09T09:29:36.979699+0000","last_fullsized":"2024-03-09T09:29:36.979699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'690","last_scrub_stamp":"2024-03-09T09:29:36.979617+0000","last_deep_scrub":"89'690","last_deep_scrub_stamp":"2024-03-09T09:29:36.979617+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.979617+0000","objects_scrubbed":46,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:35:33.065130+0000","scrub_duration":158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":715,"num_write_kb":95464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":598323,"num_omap_keys":1009,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"89'673","reported_seq":843,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:24.128278+0000","last_change":"2024-03-09T09:29:24.128343+0000","last_active":"2024-03-09T09:29:24.128278+0000","last_peered":"2024-03-09T09:29:24.128278+0000","last_clean":"2024-03-09T09:29:24.128278+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T09:29:24.128278+0000","last_undegraded":"2024-03-09T09:29:24.128278+0000","last_fullsized":"2024-03-09T09:29:24.128278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'673","last_scrub_stamp":"2024-03-09T09:29:24.128149+0000","last_deep_scrub":"89'673","last_deep_scrub_stamp":"2024-03-09T09:29:24.128149+0000","last_clean_scrub_stamp":"2024-03-09T09:29:24.128149+0000","objects_scrubbed":47,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:29:11.611027+0000","scrub_duration":125,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748758,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":685,"num_write_kb":95366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":455288,"num_omap_keys":782,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"89'2312","reported_seq":5251,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:48.119019+0000","last_change":"2024-03-09T09:29:48.119085+0000","last_active":"2024-03-09T09:29:48.119019+0000","last_peered":"2024-03-09T09:29:48.119019+0000","last_clean":"2024-03-09T09:29:48.119019+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T09:29:48.119019+0000","last_undegraded":"2024-03-09T09:29:48.119019+0000","last_fullsized":"2024-03-09T09:29:48.119019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2312","last_scrub_stamp":"2024-03-09T09:29:48.118943+0000","last_deep_scrub":"89'2312","last_deep_scrub_stamp":"2024-03-09T09:29:48.118943+0000","last_clean_scrub_stamp":"2024-03-09T09:29:48.118943+0000","objects_scrubbed":111,"log_size":2312,"log_dups_size":0,"ondisk_log_size":2312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:05.668155+0000","scrub_duration":173,"objects_trimmed":2,"snaptrim_duration":4.4556873890000004,"stat_sum":{"num_bytes":4719568,"num_objects":111,"num_object_clones":34,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":34,"num_read":112,"num_read_kb":26370,"num_write":1323,"num_write_kb":45566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"89'2394","reported_seq":5366,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.902991+0000","last_change":"2024-03-09T09:29:33.903075+0000","last_active":"2024-03-09T09:29:33.902991+0000","last_peered":"2024-03-09T09:29:33.902991+0000","last_clean":"2024-03-09T09:29:33.902991+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T09:29:33.902991+0000","last_undegraded":"2024-03-09T09:29:33.902991+0000","last_fullsized":"2024-03-09T09:29:33.902991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2394","last_scrub_stamp":"2024-03-09T09:29:33.902908+0000","last_deep_scrub":"89'2394","last_deep_scrub_stamp":"2024-03-09T09:29:33.902908+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.902908+0000","objects_scrubbed":98,"log_size":2394,"log_dups_size":0,"ondisk_log_size":2394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:41:52.741118+0000","scrub_duration":181,"objects_trimmed":0,"snaptrim_duration":3.0901323600000001,"stat_sum":{"num_bytes":867761,"num_objects":98,"num_object_clones":31,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":31,"num_read":98,"num_read_kb":37765,"num_write":1389,"num_write_kb":74512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'491","reported_seq":671,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.389166+0000","last_change":"2024-03-09T09:29:43.389237+0000","last_active":"2024-03-09T09:29:43.389166+0000","last_peered":"2024-03-09T09:29:43.389166+0000","last_clean":"2024-03-09T09:29:43.389166+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T09:29:43.389166+0000","last_undegraded":"2024-03-09T09:29:43.389166+0000","last_fullsized":"2024-03-09T09:29:43.389166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'491","last_scrub_stamp":"2024-03-09T09:29:43.388996+0000","last_deep_scrub":"89'491","last_deep_scrub_stamp":"2024-03-09T09:29:43.388996+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.388996+0000","objects_scrubbed":57,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:44:51.123622+0000","scrub_duration":207,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":69,"num_read_kb":0,"num_write":510,"num_write_kb":91498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":792682,"num_omap_keys":1280,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"89'655","reported_seq":824,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.853871+0000","last_change":"2024-03-09T09:29:29.853941+0000","last_active":"2024-03-09T09:29:29.853871+0000","last_peered":"2024-03-09T09:29:29.853871+0000","last_clean":"2024-03-09T09:29:29.853871+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T09:29:29.853871+0000","last_undegraded":"2024-03-09T09:29:29.853871+0000","last_fullsized":"2024-03-09T09:29:29.853871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'655","last_scrub_stamp":"2024-03-09T09:29:29.853754+0000","last_deep_scrub":"89'655","last_deep_scrub_stamp":"2024-03-09T09:29:29.853754+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.853754+0000","objects_scrubbed":41,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:25:33.750175+0000","scrub_duration":99,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":666,"num_write_kb":107635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":568275,"num_omap_keys":907,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"89'2336","reported_seq":5261,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.951872+0000","last_change":"2024-03-09T09:27:34.495452+0000","last_active":"2024-03-09T09:29:17.951872+0000","last_peered":"2024-03-09T09:29:17.951872+0000","last_clean":"2024-03-09T09:29:17.951872+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T09:29:17.951872+0000","last_undegraded":"2024-03-09T09:29:17.951872+0000","last_fullsized":"2024-03-09T09:29:17.951872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2336,"log_dups_size":0,"ondisk_log_size":2336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.1575832950000002,"stat_sum":{"num_bytes":392237,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":136,"num_read_kb":53607,"num_write":1404,"num_write_kb":81678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"89'2285","reported_seq":5390,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.148258+0000","last_change":"2024-03-09T09:29:32.148301+0000","last_active":"2024-03-09T09:29:32.148258+0000","last_peered":"2024-03-09T09:29:32.148258+0000","last_clean":"2024-03-09T09:29:32.148258+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T09:29:32.148258+0000","last_undegraded":"2024-03-09T09:29:32.148258+0000","last_fullsized":"2024-03-09T09:29:32.148258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2285","last_scrub_stamp":"2024-03-09T09:29:32.148181+0000","last_deep_scrub":"89'2285","last_deep_scrub_stamp":"2024-03-09T09:29:32.148181+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.148181+0000","objects_scrubbed":110,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:57:59.729700+0000","scrub_duration":179,"objects_trimmed":0,"snaptrim_duration":0.38281627099999999,"stat_sum":{"num_bytes":951718,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":197,"num_read_kb":25950,"num_write":1332,"num_write_kb":45421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'393","reported_seq":572,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.515060+0000","last_change":"2024-03-09T09:29:33.515152+0000","last_active":"2024-03-09T09:29:33.515060+0000","last_peered":"2024-03-09T09:29:33.515060+0000","last_clean":"2024-03-09T09:29:33.515060+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T09:29:33.515060+0000","last_undegraded":"2024-03-09T09:29:33.515060+0000","last_fullsized":"2024-03-09T09:29:33.515060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'393","last_scrub_stamp":"2024-03-09T09:29:33.514978+0000","last_deep_scrub":"89'393","last_deep_scrub_stamp":"2024-03-09T09:29:33.514978+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.514978+0000","objects_scrubbed":48,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:18:11.758277+0000","scrub_duration":104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":397,"num_write_kb":78578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":483803,"num_omap_keys":748,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"89'1750","reported_seq":2977,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.922706+0000","last_change":"2024-03-09T09:29:27.922756+0000","last_active":"2024-03-09T09:29:27.922706+0000","last_peered":"2024-03-09T09:29:27.922706+0000","last_clean":"2024-03-09T09:29:27.922706+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T09:29:27.922706+0000","last_undegraded":"2024-03-09T09:29:27.922706+0000","last_fullsized":"2024-03-09T09:29:27.922706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1750","last_scrub_stamp":"2024-03-09T09:29:27.922608+0000","last_deep_scrub":"89'1750","last_deep_scrub_stamp":"2024-03-09T09:29:27.922608+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.922608+0000","objects_scrubbed":50,"log_size":1750,"log_dups_size":0,"ondisk_log_size":1750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:05:35.558751+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":1117,"num_read_kb":4814,"num_write":1762,"num_write_kb":112708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":705687,"num_omap_keys":1069,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"89'2220","reported_seq":5184,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.789119+0000","last_change":"2024-03-09T09:29:41.789180+0000","last_active":"2024-03-09T09:29:41.789119+0000","last_peered":"2024-03-09T09:29:41.789119+0000","last_clean":"2024-03-09T09:29:41.789119+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T09:29:41.789119+0000","last_undegraded":"2024-03-09T09:29:41.789119+0000","last_fullsized":"2024-03-09T09:29:41.789119+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2220","last_scrub_stamp":"2024-03-09T09:29:41.788983+0000","last_deep_scrub":"89'2220","last_deep_scrub_stamp":"2024-03-09T09:29:41.788983+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.788983+0000","objects_scrubbed":107,"log_size":2220,"log_dups_size":0,"ondisk_log_size":2220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:08:31.651496+0000","scrub_duration":156,"objects_trimmed":0,"snaptrim_duration":3.8297685179999998,"stat_sum":{"num_bytes":268964,"num_objects":107,"num_object_clones":32,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":32,"num_read":107,"num_read_kb":24622,"num_write":1287,"num_write_kb":41444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"89'2380","reported_seq":5440,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.772126+0000","last_change":"2024-03-09T09:29:40.772206+0000","last_active":"2024-03-09T09:29:40.772126+0000","last_peered":"2024-03-09T09:29:40.772126+0000","last_clean":"2024-03-09T09:29:40.772126+0000","last_became_active":"2024-03-09T08:53:00.240844+0000","last_became_peered":"2024-03-09T08:53:00.240844+0000","last_unstale":"2024-03-09T09:29:40.772126+0000","last_undegraded":"2024-03-09T09:29:40.772126+0000","last_fullsized":"2024-03-09T09:29:40.772126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2380","last_scrub_stamp":"2024-03-09T09:29:40.771910+0000","last_deep_scrub":"89'2380","last_deep_scrub_stamp":"2024-03-09T09:29:40.771910+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.771910+0000","objects_scrubbed":126,"log_size":2380,"log_dups_size":0,"ondisk_log_size":2380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:27:55.399614+0000","scrub_duration":293,"objects_trimmed":0,"snaptrim_duration":0.78540855600000004,"stat_sum":{"num_bytes":1854558,"num_objects":126,"num_object_clones":46,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":46,"num_read":216,"num_read_kb":39239,"num_write":1332,"num_write_kb":71797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"89'1600","reported_seq":1779,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.020325+0000","last_change":"2024-03-09T09:29:35.020394+0000","last_active":"2024-03-09T09:29:35.020325+0000","last_peered":"2024-03-09T09:29:35.020325+0000","last_clean":"2024-03-09T09:29:35.020325+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T09:29:35.020325+0000","last_undegraded":"2024-03-09T09:29:35.020325+0000","last_fullsized":"2024-03-09T09:29:35.020325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1600","last_scrub_stamp":"2024-03-09T09:29:35.020224+0000","last_deep_scrub":"89'1600","last_deep_scrub_stamp":"2024-03-09T09:29:35.020224+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.020224+0000","objects_scrubbed":43,"log_size":1600,"log_dups_size":0,"ondisk_log_size":1600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:05:09.036176+0000","scrub_duration":140,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":57,"num_read_kb":0,"num_write":3420,"num_write_kb":84853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":559144,"num_omap_keys":941,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"89'726","reported_seq":898,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.482944+0000","last_change":"2024-03-09T09:29:31.483028+0000","last_active":"2024-03-09T09:29:31.482944+0000","last_peered":"2024-03-09T09:29:31.482944+0000","last_clean":"2024-03-09T09:29:31.482944+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T09:29:31.482944+0000","last_undegraded":"2024-03-09T09:29:31.482944+0000","last_fullsized":"2024-03-09T09:29:31.482944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'726","last_scrub_stamp":"2024-03-09T09:29:31.482759+0000","last_deep_scrub":"89'726","last_deep_scrub_stamp":"2024-03-09T09:29:31.482759+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.482759+0000","objects_scrubbed":54,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:10:04.708433+0000","scrub_duration":158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":71,"num_read_kb":0,"num_write":764,"num_write_kb":87842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":798407,"num_omap_keys":988,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"89'2473","reported_seq":5371,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.547309+0000","last_change":"2024-03-09T09:29:42.547368+0000","last_active":"2024-03-09T09:29:42.547309+0000","last_peered":"2024-03-09T09:29:42.547309+0000","last_clean":"2024-03-09T09:29:42.547309+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T09:29:42.547309+0000","last_undegraded":"2024-03-09T09:29:42.547309+0000","last_fullsized":"2024-03-09T09:29:42.547309+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2473","last_scrub_stamp":"2024-03-09T09:29:42.547184+0000","last_deep_scrub":"89'2473","last_deep_scrub_stamp":"2024-03-09T09:29:42.547184+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.547184+0000","objects_scrubbed":134,"log_size":2473,"log_dups_size":0,"ondisk_log_size":2473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:43:09.396144+0000","scrub_duration":238,"objects_trimmed":0,"snaptrim_duration":0.65806592200000003,"stat_sum":{"num_bytes":2259601,"num_objects":134,"num_object_clones":47,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":47,"num_read":108,"num_read_kb":22295,"num_write":1398,"num_write_kb":43486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"89'2292","reported_seq":5302,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.171581+0000","last_change":"2024-03-09T09:29:30.171649+0000","last_active":"2024-03-09T09:29:30.171581+0000","last_peered":"2024-03-09T09:29:30.171581+0000","last_clean":"2024-03-09T09:29:30.171581+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T09:29:30.171581+0000","last_undegraded":"2024-03-09T09:29:30.171581+0000","last_fullsized":"2024-03-09T09:29:30.171581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2292","last_scrub_stamp":"2024-03-09T09:29:30.171498+0000","last_deep_scrub":"89'2292","last_deep_scrub_stamp":"2024-03-09T09:29:30.171498+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.171498+0000","objects_scrubbed":111,"log_size":2292,"log_dups_size":0,"ondisk_log_size":2292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:16:25.210594+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0.96663918400000004,"stat_sum":{"num_bytes":2499803,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":112,"num_read_kb":5934,"num_write":1320,"num_write_kb":25592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"89'694","reported_seq":874,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.354608+0000","last_change":"2024-03-09T09:29:46.354649+0000","last_active":"2024-03-09T09:29:46.354608+0000","last_peered":"2024-03-09T09:29:46.354608+0000","last_clean":"2024-03-09T09:29:46.354608+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T09:29:46.354608+0000","last_undegraded":"2024-03-09T09:29:46.354608+0000","last_fullsized":"2024-03-09T09:29:46.354608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'694","last_scrub_stamp":"2024-03-09T09:29:46.354505+0000","last_deep_scrub":"89'694","last_deep_scrub_stamp":"2024-03-09T09:29:46.354505+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.354505+0000","objects_scrubbed":52,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:43:19.678339+0000","scrub_duration":136,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":75,"num_read_kb":0,"num_write":726,"num_write_kb":96134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":830736,"num_omap_keys":1184,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"89'617","reported_seq":788,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.084546+0000","last_change":"2024-03-09T09:29:36.084599+0000","last_active":"2024-03-09T09:29:36.084546+0000","last_peered":"2024-03-09T09:29:36.084546+0000","last_clean":"2024-03-09T09:29:36.084546+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T09:29:36.084546+0000","last_undegraded":"2024-03-09T09:29:36.084546+0000","last_fullsized":"2024-03-09T09:29:36.084546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'617","last_scrub_stamp":"2024-03-09T09:29:36.084430+0000","last_deep_scrub":"89'617","last_deep_scrub_stamp":"2024-03-09T09:29:36.084430+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.084430+0000","objects_scrubbed":33,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:55.329284+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":626,"num_write_kb":74740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":361223,"num_omap_keys":599,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"89'2400","reported_seq":5312,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.264848+0000","last_change":"2024-03-09T09:29:39.264915+0000","last_active":"2024-03-09T09:29:39.264848+0000","last_peered":"2024-03-09T09:29:39.264848+0000","last_clean":"2024-03-09T09:29:39.264848+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T09:29:39.264848+0000","last_undegraded":"2024-03-09T09:29:39.264848+0000","last_fullsized":"2024-03-09T09:29:39.264848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2400","last_scrub_stamp":"2024-03-09T09:29:39.264692+0000","last_deep_scrub":"89'2400","last_deep_scrub_stamp":"2024-03-09T09:29:39.264692+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.264692+0000","objects_scrubbed":126,"log_size":2400,"log_dups_size":0,"ondisk_log_size":2400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:31:01.872524+0000","scrub_duration":253,"objects_trimmed":0,"snaptrim_duration":0.71811667999999995,"stat_sum":{"num_bytes":1846155,"num_objects":126,"num_object_clones":42,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":42,"num_read":111,"num_read_kb":21516,"num_write":1347,"num_write_kb":38338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"89'2389","reported_seq":5349,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.168301+0000","last_change":"2024-03-09T09:29:27.168355+0000","last_active":"2024-03-09T09:29:27.168301+0000","last_peered":"2024-03-09T09:29:27.168301+0000","last_clean":"2024-03-09T09:29:27.168301+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T09:29:27.168301+0000","last_undegraded":"2024-03-09T09:29:27.168301+0000","last_fullsized":"2024-03-09T09:29:27.168301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2389","last_scrub_stamp":"2024-03-09T09:29:27.168219+0000","last_deep_scrub":"89'2389","last_deep_scrub_stamp":"2024-03-09T09:29:27.168219+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.168219+0000","objects_scrubbed":105,"log_size":2389,"log_dups_size":0,"ondisk_log_size":2389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:07:32.466927+0000","scrub_duration":328,"objects_trimmed":0,"snaptrim_duration":1.815577539,"stat_sum":{"num_bytes":1586992,"num_objects":105,"num_object_clones":35,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":35,"num_read":97,"num_read_kb":33815,"num_write":1338,"num_write_kb":74580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"89'594","reported_seq":774,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.056459+0000","last_change":"2024-03-09T09:29:34.056459+0000","last_active":"2024-03-09T09:29:34.056459+0000","last_peered":"2024-03-09T09:29:34.056459+0000","last_clean":"2024-03-09T09:29:34.056459+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T09:29:34.056459+0000","last_undegraded":"2024-03-09T09:29:34.056459+0000","last_fullsized":"2024-03-09T09:29:34.056459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'594","last_scrub_stamp":"2024-03-09T09:29:34.056310+0000","last_deep_scrub":"89'594","last_deep_scrub_stamp":"2024-03-09T09:29:34.056310+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.056310+0000","objects_scrubbed":48,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:43:54.072133+0000","scrub_duration":136,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":620,"num_write_kb":120324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":674477,"num_omap_keys":1093,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"89'590","reported_seq":761,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.201577+0000","last_change":"2024-03-09T09:29:29.201654+0000","last_active":"2024-03-09T09:29:29.201577+0000","last_peered":"2024-03-09T09:29:29.201577+0000","last_clean":"2024-03-09T09:29:29.201577+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T09:29:29.201577+0000","last_undegraded":"2024-03-09T09:29:29.201577+0000","last_fullsized":"2024-03-09T09:29:29.201577+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'590","last_scrub_stamp":"2024-03-09T09:29:29.201403+0000","last_deep_scrub":"89'590","last_deep_scrub_stamp":"2024-03-09T09:29:29.201403+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.201403+0000","objects_scrubbed":38,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:26:31.012880+0000","scrub_duration":156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":563,"num_write_kb":103368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":328063,"num_omap_keys":503,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"89'2366","reported_seq":5279,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.144774+0000","last_change":"2024-03-09T09:29:42.144804+0000","last_active":"2024-03-09T09:29:42.144774+0000","last_peered":"2024-03-09T09:29:42.144774+0000","last_clean":"2024-03-09T09:29:42.144774+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T09:29:42.144774+0000","last_undegraded":"2024-03-09T09:29:42.144774+0000","last_fullsized":"2024-03-09T09:29:42.144774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2366","last_scrub_stamp":"2024-03-09T09:29:42.144694+0000","last_deep_scrub":"89'2366","last_deep_scrub_stamp":"2024-03-09T09:29:42.144694+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.144694+0000","objects_scrubbed":101,"log_size":2366,"log_dups_size":0,"ondisk_log_size":2366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:44.531453+0000","scrub_duration":177,"objects_trimmed":4,"snaptrim_duration":0.71599741500000003,"stat_sum":{"num_bytes":1873672,"num_objects":101,"num_object_clones":34,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":34,"num_read":103,"num_read_kb":19624,"num_write":1351,"num_write_kb":43427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"89'2388","reported_seq":5378,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.908870+0000","last_change":"2024-03-09T09:29:34.908917+0000","last_active":"2024-03-09T09:29:34.908870+0000","last_peered":"2024-03-09T09:29:34.908870+0000","last_clean":"2024-03-09T09:29:34.908870+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T09:29:34.908870+0000","last_undegraded":"2024-03-09T09:29:34.908870+0000","last_fullsized":"2024-03-09T09:29:34.908870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2388","last_scrub_stamp":"2024-03-09T09:29:34.908766+0000","last_deep_scrub":"89'2388","last_deep_scrub_stamp":"2024-03-09T09:29:34.908766+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.908766+0000","objects_scrubbed":128,"log_size":2388,"log_dups_size":0,"ondisk_log_size":2388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:46:15.676520+0000","scrub_duration":201,"objects_trimmed":0,"snaptrim_duration":3.4919903310000002,"stat_sum":{"num_bytes":2681803,"num_objects":128,"num_object_clones":43,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":43,"num_read":104,"num_read_kb":25781,"num_write":1386,"num_write_kb":55300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"89'789","reported_seq":970,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.046778+0000","last_change":"2024-03-09T09:29:41.046778+0000","last_active":"2024-03-09T09:29:41.046778+0000","last_peered":"2024-03-09T09:29:41.046778+0000","last_clean":"2024-03-09T09:29:41.046778+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T09:29:41.046778+0000","last_undegraded":"2024-03-09T09:29:41.046778+0000","last_fullsized":"2024-03-09T09:29:41.046778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'789","last_scrub_stamp":"2024-03-09T09:29:41.046618+0000","last_deep_scrub":"89'789","last_deep_scrub_stamp":"2024-03-09T09:29:41.046618+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.046618+0000","objects_scrubbed":53,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:35.300265+0000","scrub_duration":202,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":833,"num_write_kb":103821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":595568,"num_omap_keys":1030,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'609","reported_seq":780,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.150296+0000","last_change":"2024-03-09T09:29:25.150380+0000","last_active":"2024-03-09T09:29:25.150296+0000","last_peered":"2024-03-09T09:29:25.150296+0000","last_clean":"2024-03-09T09:29:25.150296+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T09:29:25.150296+0000","last_undegraded":"2024-03-09T09:29:25.150296+0000","last_fullsized":"2024-03-09T09:29:25.150296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'609","last_scrub_stamp":"2024-03-09T09:29:25.150212+0000","last_deep_scrub":"89'609","last_deep_scrub_stamp":"2024-03-09T09:29:25.150212+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.150212+0000","objects_scrubbed":48,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:04:01.814051+0000","scrub_duration":112,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":52,"num_read_kb":0,"num_write":616,"num_write_kb":119874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":438501,"num_omap_keys":702,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"89'2276","reported_seq":5223,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:24.750759+0000","last_change":"2024-03-09T09:29:24.750805+0000","last_active":"2024-03-09T09:29:24.750759+0000","last_peered":"2024-03-09T09:29:24.750759+0000","last_clean":"2024-03-09T09:29:24.750759+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T09:29:24.750759+0000","last_undegraded":"2024-03-09T09:29:24.750759+0000","last_fullsized":"2024-03-09T09:29:24.750759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2276","last_scrub_stamp":"2024-03-09T09:29:24.750679+0000","last_deep_scrub":"89'2276","last_deep_scrub_stamp":"2024-03-09T09:29:24.750679+0000","last_clean_scrub_stamp":"2024-03-09T09:29:24.750679+0000","objects_scrubbed":110,"log_size":2276,"log_dups_size":0,"ondisk_log_size":2276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:41:28.516964+0000","scrub_duration":247,"objects_trimmed":2,"snaptrim_duration":0.63049624100000001,"stat_sum":{"num_bytes":833172,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":114,"num_read_kb":17857,"num_write":1302,"num_write_kb":44488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"89'2367","reported_seq":5356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.063265+0000","last_change":"2024-03-09T09:29:40.063339+0000","last_active":"2024-03-09T09:29:40.063265+0000","last_peered":"2024-03-09T09:29:40.063265+0000","last_clean":"2024-03-09T09:29:40.063265+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T09:29:40.063265+0000","last_undegraded":"2024-03-09T09:29:40.063265+0000","last_fullsized":"2024-03-09T09:29:40.063265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2367","last_scrub_stamp":"2024-03-09T09:29:40.063175+0000","last_deep_scrub":"89'2367","last_deep_scrub_stamp":"2024-03-09T09:29:40.063175+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.063175+0000","objects_scrubbed":117,"log_size":2367,"log_dups_size":0,"ondisk_log_size":2367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:42:37.267229+0000","scrub_duration":194,"objects_trimmed":0,"snaptrim_duration":1.18447864,"stat_sum":{"num_bytes":1122276,"num_objects":117,"num_object_clones":43,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":43,"num_read":101,"num_read_kb":18409,"num_write":1364,"num_write_kb":39400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"89'1869","reported_seq":3064,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.624152+0000","last_change":"2024-03-09T09:29:26.624152+0000","last_active":"2024-03-09T09:29:26.624152+0000","last_peered":"2024-03-09T09:29:26.624152+0000","last_clean":"2024-03-09T09:29:26.624152+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T09:29:26.624152+0000","last_undegraded":"2024-03-09T09:29:26.624152+0000","last_fullsized":"2024-03-09T09:29:26.624152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1869","last_scrub_stamp":"2024-03-09T09:29:26.624016+0000","last_deep_scrub":"89'1869","last_deep_scrub_stamp":"2024-03-09T09:29:26.624016+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.624016+0000","objects_scrubbed":39,"log_size":1869,"log_dups_size":0,"ondisk_log_size":1869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:29.163043+0000","scrub_duration":102,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":1037,"num_read_kb":4764,"num_write":1840,"num_write_kb":108217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":338965,"num_omap_keys":539,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"89'441","reported_seq":612,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.080999+0000","last_change":"2024-03-09T09:29:31.081118+0000","last_active":"2024-03-09T09:29:31.080999+0000","last_peered":"2024-03-09T09:29:31.080999+0000","last_clean":"2024-03-09T09:29:31.080999+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T09:29:31.080999+0000","last_undegraded":"2024-03-09T09:29:31.080999+0000","last_fullsized":"2024-03-09T09:29:31.080999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'441","last_scrub_stamp":"2024-03-09T09:29:31.080900+0000","last_deep_scrub":"89'441","last_deep_scrub_stamp":"2024-03-09T09:29:31.080900+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.080900+0000","objects_scrubbed":37,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:15:41.261309+0000","scrub_duration":127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":44,"num_read_kb":0,"num_write":445,"num_write_kb":78896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":431086,"num_omap_keys":609,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"89'2184","reported_seq":5128,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:23.687686+0000","last_change":"2024-03-09T09:27:34.760081+0000","last_active":"2024-03-09T09:29:23.687686+0000","last_peered":"2024-03-09T09:29:23.687686+0000","last_clean":"2024-03-09T09:29:23.687686+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T09:29:23.687686+0000","last_undegraded":"2024-03-09T09:29:23.687686+0000","last_fullsized":"2024-03-09T09:29:23.687686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2184,"log_dups_size":0,"ondisk_log_size":2184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4843044489999997,"stat_sum":{"num_bytes":1636574,"num_objects":123,"num_object_clones":44,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":44,"num_read":134,"num_read_kb":27719,"num_write":1302,"num_write_kb":59461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'2240","reported_seq":5246,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.182937+0000","last_change":"2024-03-09T09:29:30.183008+0000","last_active":"2024-03-09T09:29:30.182937+0000","last_peered":"2024-03-09T09:29:30.182937+0000","last_clean":"2024-03-09T09:29:30.182937+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T09:29:30.182937+0000","last_undegraded":"2024-03-09T09:29:30.182937+0000","last_fullsized":"2024-03-09T09:29:30.182937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2240","last_scrub_stamp":"2024-03-09T09:29:30.182824+0000","last_deep_scrub":"89'2240","last_deep_scrub_stamp":"2024-03-09T09:29:30.182824+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.182824+0000","objects_scrubbed":113,"log_size":2240,"log_dups_size":0,"ondisk_log_size":2240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:52:59.854784+0000","scrub_duration":187,"objects_trimmed":0,"snaptrim_duration":0.289202336,"stat_sum":{"num_bytes":424328,"num_objects":113,"num_object_clones":35,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":35,"num_read":94,"num_read_kb":17266,"num_write":1284,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"89'615","reported_seq":795,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:47.292509+0000","last_change":"2024-03-09T09:29:47.292559+0000","last_active":"2024-03-09T09:29:47.292509+0000","last_peered":"2024-03-09T09:29:47.292509+0000","last_clean":"2024-03-09T09:29:47.292509+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T09:29:47.292509+0000","last_undegraded":"2024-03-09T09:29:47.292509+0000","last_fullsized":"2024-03-09T09:29:47.292509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'615","last_scrub_stamp":"2024-03-09T09:29:47.292418+0000","last_deep_scrub":"89'615","last_deep_scrub_stamp":"2024-03-09T09:29:47.292418+0000","last_clean_scrub_stamp":"2024-03-09T09:29:47.292418+0000","objects_scrubbed":35,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:55.705231+0000","scrub_duration":76,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":598,"num_write_kb":82993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":415266,"num_omap_keys":726,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"89'1635","reported_seq":1806,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.112003+0000","last_change":"2024-03-09T09:29:26.112003+0000","last_active":"2024-03-09T09:29:26.112003+0000","last_peered":"2024-03-09T09:29:26.112003+0000","last_clean":"2024-03-09T09:29:26.112003+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T09:29:26.112003+0000","last_undegraded":"2024-03-09T09:29:26.112003+0000","last_fullsized":"2024-03-09T09:29:26.112003+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1634","last_scrub_stamp":"2024-03-09T09:29:26.111801+0000","last_deep_scrub":"89'1634","last_deep_scrub_stamp":"2024-03-09T09:29:26.111801+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.111801+0000","objects_scrubbed":42,"log_size":1635,"log_dups_size":0,"ondisk_log_size":1635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:02:01.981447+0000","scrub_duration":129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25241505,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":3476,"num_write_kb":84820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":357598,"num_omap_keys":628,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"89'2236","reported_seq":5168,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.931406+0000","last_change":"2024-03-09T09:27:34.687418+0000","last_active":"2024-03-09T09:29:17.931406+0000","last_peered":"2024-03-09T09:29:17.931406+0000","last_clean":"2024-03-09T09:29:17.931406+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T09:29:17.931406+0000","last_undegraded":"2024-03-09T09:29:17.931406+0000","last_fullsized":"2024-03-09T09:29:17.931406+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2236,"log_dups_size":0,"ondisk_log_size":2236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4503168610000001,"stat_sum":{"num_bytes":1665642,"num_objects":121,"num_object_clones":41,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":41,"num_read":126,"num_read_kb":42837,"num_write":1341,"num_write_kb":67584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'2445","reported_seq":5420,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.554785+0000","last_change":"2024-03-09T09:29:45.554842+0000","last_active":"2024-03-09T09:29:45.554785+0000","last_peered":"2024-03-09T09:29:45.554785+0000","last_clean":"2024-03-09T09:29:45.554785+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T09:29:45.554785+0000","last_undegraded":"2024-03-09T09:29:45.554785+0000","last_fullsized":"2024-03-09T09:29:45.554785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2445","last_scrub_stamp":"2024-03-09T09:29:45.554681+0000","last_deep_scrub":"89'2445","last_deep_scrub_stamp":"2024-03-09T09:29:45.554681+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.554681+0000","objects_scrubbed":137,"log_size":2445,"log_dups_size":0,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:29:22.203097+0000","scrub_duration":302,"objects_trimmed":0,"snaptrim_duration":0.56599842099999997,"stat_sum":{"num_bytes":1847778,"num_objects":137,"num_object_clones":49,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":49,"num_read":115,"num_read_kb":13750,"num_write":1371,"num_write_kb":55061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'665","reported_seq":845,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.902079+0000","last_change":"2024-03-09T09:29:39.902079+0000","last_active":"2024-03-09T09:29:39.902079+0000","last_peered":"2024-03-09T09:29:39.902079+0000","last_clean":"2024-03-09T09:29:39.902079+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T09:29:39.902079+0000","last_undegraded":"2024-03-09T09:29:39.902079+0000","last_fullsized":"2024-03-09T09:29:39.902079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'665","last_scrub_stamp":"2024-03-09T09:29:39.901928+0000","last_deep_scrub":"89'665","last_deep_scrub_stamp":"2024-03-09T09:29:39.901928+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.901928+0000","objects_scrubbed":39,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:49:03.340420+0000","scrub_duration":52,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":698,"num_write_kb":70663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":499002,"num_omap_keys":859,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"89'119","reported_seq":337,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.183267+0000","last_change":"2024-03-09T09:29:41.183293+0000","last_active":"2024-03-09T09:29:41.183267+0000","last_peered":"2024-03-09T09:29:41.183267+0000","last_clean":"2024-03-09T09:29:41.183267+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T09:29:41.183267+0000","last_undegraded":"2024-03-09T09:29:41.183267+0000","last_fullsized":"2024-03-09T09:29:41.183267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'119","last_scrub_stamp":"2024-03-09T09:29:41.183205+0000","last_deep_scrub":"89'119","last_deep_scrub_stamp":"2024-03-09T09:29:41.183205+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.183205+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:10:29.103603+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'726","reported_seq":897,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.473277+0000","last_change":"2024-03-09T09:29:39.473372+0000","last_active":"2024-03-09T09:29:39.473277+0000","last_peered":"2024-03-09T09:29:39.473277+0000","last_clean":"2024-03-09T09:29:39.473277+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T09:29:39.473277+0000","last_undegraded":"2024-03-09T09:29:39.473277+0000","last_fullsized":"2024-03-09T09:29:39.473277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'726","last_scrub_stamp":"2024-03-09T09:29:39.473216+0000","last_deep_scrub":"89'726","last_deep_scrub_stamp":"2024-03-09T09:29:39.473216+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.473216+0000","objects_scrubbed":46,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:30:31.060980+0000","scrub_duration":110,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":743,"num_write_kb":95766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":577964,"num_omap_keys":893,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"89'2354","reported_seq":5295,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.733098+0000","last_change":"2024-03-09T09:29:35.733160+0000","last_active":"2024-03-09T09:29:35.733098+0000","last_peered":"2024-03-09T09:29:35.733098+0000","last_clean":"2024-03-09T09:29:35.733098+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T09:29:35.733098+0000","last_undegraded":"2024-03-09T09:29:35.733098+0000","last_fullsized":"2024-03-09T09:29:35.733098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2354","last_scrub_stamp":"2024-03-09T09:29:35.733000+0000","last_deep_scrub":"89'2354","last_deep_scrub_stamp":"2024-03-09T09:29:35.733000+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.733000+0000","objects_scrubbed":106,"log_size":2354,"log_dups_size":0,"ondisk_log_size":2354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:51:37.891297+0000","scrub_duration":155,"objects_trimmed":4,"snaptrim_duration":0.66264078000000004,"stat_sum":{"num_bytes":792938,"num_objects":106,"num_object_clones":38,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":38,"num_read":125,"num_read_kb":17823,"num_write":1366,"num_write_kb":27629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'2357","reported_seq":5314,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.688984+0000","last_change":"2024-03-09T09:29:41.689028+0000","last_active":"2024-03-09T09:29:41.688984+0000","last_peered":"2024-03-09T09:29:41.688984+0000","last_clean":"2024-03-09T09:29:41.688984+0000","last_became_active":"2024-03-09T08:53:00.437859+0000","last_became_peered":"2024-03-09T08:53:00.437859+0000","last_unstale":"2024-03-09T09:29:41.688984+0000","last_undegraded":"2024-03-09T09:29:41.688984+0000","last_fullsized":"2024-03-09T09:29:41.688984+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2357","last_scrub_stamp":"2024-03-09T09:29:41.688920+0000","last_deep_scrub":"89'2357","last_deep_scrub_stamp":"2024-03-09T09:29:41.688920+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.688920+0000","objects_scrubbed":115,"log_size":2357,"log_dups_size":0,"ondisk_log_size":2357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:28:30.276654+0000","scrub_duration":224,"objects_trimmed":0,"snaptrim_duration":0.79260830400000004,"stat_sum":{"num_bytes":1456716,"num_objects":115,"num_object_clones":40,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":40,"num_read":109,"num_read_kb":18518,"num_write":1301,"num_write_kb":37816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"89'547","reported_seq":725,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.610930+0000","last_change":"2024-03-09T09:29:29.611054+0000","last_active":"2024-03-09T09:29:29.610930+0000","last_peered":"2024-03-09T09:29:29.610930+0000","last_clean":"2024-03-09T09:29:29.610930+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T09:29:29.610930+0000","last_undegraded":"2024-03-09T09:29:29.610930+0000","last_fullsized":"2024-03-09T09:29:29.610930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'547","last_scrub_stamp":"2024-03-09T09:29:29.610727+0000","last_deep_scrub":"89'547","last_deep_scrub_stamp":"2024-03-09T09:29:29.610727+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.610727+0000","objects_scrubbed":46,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:29:15.793829+0000","scrub_duration":161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":39,"num_read_kb":0,"num_write":538,"num_write_kb":99417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":457181,"num_omap_keys":720,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"89'758","reported_seq":930,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.479227+0000","last_change":"2024-03-09T09:29:41.479258+0000","last_active":"2024-03-09T09:29:41.479227+0000","last_peered":"2024-03-09T09:29:41.479227+0000","last_clean":"2024-03-09T09:29:41.479227+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T09:29:41.479227+0000","last_undegraded":"2024-03-09T09:29:41.479227+0000","last_fullsized":"2024-03-09T09:29:41.479227+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'758","last_scrub_stamp":"2024-03-09T09:29:41.479164+0000","last_deep_scrub":"89'758","last_deep_scrub_stamp":"2024-03-09T09:29:41.479164+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.479164+0000","objects_scrubbed":55,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:20.205971+0000","scrub_duration":156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":780,"num_write_kb":116155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":563342,"num_omap_keys":842,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"89'2292","reported_seq":5252,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.732442+0000","last_change":"2024-03-09T09:29:37.732525+0000","last_active":"2024-03-09T09:29:37.732442+0000","last_peered":"2024-03-09T09:29:37.732442+0000","last_clean":"2024-03-09T09:29:37.732442+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T09:29:37.732442+0000","last_undegraded":"2024-03-09T09:29:37.732442+0000","last_fullsized":"2024-03-09T09:29:37.732442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2292","last_scrub_stamp":"2024-03-09T09:29:37.732276+0000","last_deep_scrub":"89'2292","last_deep_scrub_stamp":"2024-03-09T09:29:37.732276+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.732276+0000","objects_scrubbed":92,"log_size":2292,"log_dups_size":0,"ondisk_log_size":2292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:37:49.725086+0000","scrub_duration":199,"objects_trimmed":2,"snaptrim_duration":0.647865472,"stat_sum":{"num_bytes":999257,"num_objects":92,"num_object_clones":32,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":32,"num_read":116,"num_read_kb":42999,"num_write":1346,"num_write_kb":55653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"89'2284","reported_seq":5290,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.156522+0000","last_change":"2024-03-09T09:29:28.156605+0000","last_active":"2024-03-09T09:29:28.156522+0000","last_peered":"2024-03-09T09:29:28.156522+0000","last_clean":"2024-03-09T09:29:28.156522+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T09:29:28.156522+0000","last_undegraded":"2024-03-09T09:29:28.156522+0000","last_fullsized":"2024-03-09T09:29:28.156522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2284","last_scrub_stamp":"2024-03-09T09:29:28.156442+0000","last_deep_scrub":"89'2284","last_deep_scrub_stamp":"2024-03-09T09:29:28.156442+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.156442+0000","objects_scrubbed":97,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:48:56.755552+0000","scrub_duration":364,"objects_trimmed":0,"snaptrim_duration":1.951760857,"stat_sum":{"num_bytes":1333229,"num_objects":97,"num_object_clones":33,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":33,"num_read":100,"num_read_kb":17450,"num_write":1313,"num_write_kb":41729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"89'888","reported_seq":1066,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.953183+0000","last_change":"2024-03-09T09:29:37.953238+0000","last_active":"2024-03-09T09:29:37.953183+0000","last_peered":"2024-03-09T09:29:37.953183+0000","last_clean":"2024-03-09T09:29:37.953183+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T09:29:37.953183+0000","last_undegraded":"2024-03-09T09:29:37.953183+0000","last_fullsized":"2024-03-09T09:29:37.953183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'888","last_scrub_stamp":"2024-03-09T09:29:37.953099+0000","last_deep_scrub":"89'888","last_deep_scrub_stamp":"2024-03-09T09:29:37.953099+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.953099+0000","objects_scrubbed":43,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:06:19.595033+0000","scrub_duration":131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971641,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":912,"num_write_kb":111865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":576689,"num_omap_keys":942,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"89'842","reported_seq":1012,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.077990+0000","last_change":"2024-03-09T09:29:37.078044+0000","last_active":"2024-03-09T09:29:37.077990+0000","last_peered":"2024-03-09T09:29:37.077990+0000","last_clean":"2024-03-09T09:29:37.077990+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T09:29:37.077990+0000","last_undegraded":"2024-03-09T09:29:37.077990+0000","last_fullsized":"2024-03-09T09:29:37.077990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'842","last_scrub_stamp":"2024-03-09T09:29:37.077914+0000","last_deep_scrub":"89'842","last_deep_scrub_stamp":"2024-03-09T09:29:37.077914+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.077914+0000","objects_scrubbed":46,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:20:48.969270+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":51,"num_read_kb":0,"num_write":872,"num_write_kb":99652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":580759,"num_omap_keys":912,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"89'2337","reported_seq":5246,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.409316+0000","last_change":"2024-03-09T09:29:37.409360+0000","last_active":"2024-03-09T09:29:37.409316+0000","last_peered":"2024-03-09T09:29:37.409316+0000","last_clean":"2024-03-09T09:29:37.409316+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T09:29:37.409316+0000","last_undegraded":"2024-03-09T09:29:37.409316+0000","last_fullsized":"2024-03-09T09:29:37.409316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2337","last_scrub_stamp":"2024-03-09T09:29:37.409205+0000","last_deep_scrub":"89'2337","last_deep_scrub_stamp":"2024-03-09T09:29:37.409205+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.409205+0000","objects_scrubbed":100,"log_size":2337,"log_dups_size":0,"ondisk_log_size":2337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:14:29.152094+0000","scrub_duration":183,"objects_trimmed":2,"snaptrim_duration":4.4098675719999996,"stat_sum":{"num_bytes":603203,"num_objects":100,"num_object_clones":32,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":32,"num_read":95,"num_read_kb":13936,"num_write":1338,"num_write_kb":41727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"89'2316","reported_seq":5322,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.413234+0000","last_change":"2024-03-09T09:29:31.413310+0000","last_active":"2024-03-09T09:29:31.413234+0000","last_peered":"2024-03-09T09:29:31.413234+0000","last_clean":"2024-03-09T09:29:31.413234+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T09:29:31.413234+0000","last_undegraded":"2024-03-09T09:29:31.413234+0000","last_fullsized":"2024-03-09T09:29:31.413234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2316","last_scrub_stamp":"2024-03-09T09:29:31.413148+0000","last_deep_scrub":"89'2316","last_deep_scrub_stamp":"2024-03-09T09:29:31.413148+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.413148+0000","objects_scrubbed":119,"log_size":2316,"log_dups_size":0,"ondisk_log_size":2316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:25:38.834796+0000","scrub_duration":212,"objects_trimmed":0,"snaptrim_duration":0.91280589400000001,"stat_sum":{"num_bytes":4608718,"num_objects":119,"num_object_clones":37,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":37,"num_read":119,"num_read_kb":26183,"num_write":1317,"num_write_kb":37436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'716","reported_seq":894,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.567246+0000","last_change":"2024-03-09T09:29:27.567312+0000","last_active":"2024-03-09T09:29:27.567246+0000","last_peered":"2024-03-09T09:29:27.567246+0000","last_clean":"2024-03-09T09:29:27.567246+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T09:29:27.567246+0000","last_undegraded":"2024-03-09T09:29:27.567246+0000","last_fullsized":"2024-03-09T09:29:27.567246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'716","last_scrub_stamp":"2024-03-09T09:29:27.567172+0000","last_deep_scrub":"89'716","last_deep_scrub_stamp":"2024-03-09T09:29:27.567172+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.567172+0000","objects_scrubbed":44,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:24:56.572979+0000","scrub_duration":87,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":724,"num_write_kb":103814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547853,"num_omap_keys":763,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"89'767","reported_seq":939,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.026274+0000","last_change":"2024-03-09T09:29:33.026328+0000","last_active":"2024-03-09T09:29:33.026274+0000","last_peered":"2024-03-09T09:29:33.026274+0000","last_clean":"2024-03-09T09:29:33.026274+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T09:29:33.026274+0000","last_undegraded":"2024-03-09T09:29:33.026274+0000","last_fullsized":"2024-03-09T09:29:33.026274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'767","last_scrub_stamp":"2024-03-09T09:29:33.026092+0000","last_deep_scrub":"89'767","last_deep_scrub_stamp":"2024-03-09T09:29:33.026092+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.026092+0000","objects_scrubbed":54,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:20:10.415787+0000","scrub_duration":182,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":59,"num_read_kb":0,"num_write":793,"num_write_kb":95673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":554506,"num_omap_keys":964,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"89'2494","reported_seq":5406,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.768604+0000","last_change":"2024-03-09T09:29:33.768673+0000","last_active":"2024-03-09T09:29:33.768604+0000","last_peered":"2024-03-09T09:29:33.768604+0000","last_clean":"2024-03-09T09:29:33.768604+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T09:29:33.768604+0000","last_undegraded":"2024-03-09T09:29:33.768604+0000","last_fullsized":"2024-03-09T09:29:33.768604+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2494","last_scrub_stamp":"2024-03-09T09:29:33.768471+0000","last_deep_scrub":"89'2494","last_deep_scrub_stamp":"2024-03-09T09:29:33.768471+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.768471+0000","objects_scrubbed":135,"log_size":2494,"log_dups_size":0,"ondisk_log_size":2494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:25:40.852627+0000","scrub_duration":235,"objects_trimmed":0,"snaptrim_duration":0.634097468,"stat_sum":{"num_bytes":572008,"num_objects":135,"num_object_clones":47,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":47,"num_read":117,"num_read_kb":9742,"num_write":1431,"num_write_kb":32918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"89'2353","reported_seq":5345,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.349389+0000","last_change":"2024-03-09T09:29:36.349440+0000","last_active":"2024-03-09T09:29:36.349389+0000","last_peered":"2024-03-09T09:29:36.349389+0000","last_clean":"2024-03-09T09:29:36.349389+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T09:29:36.349389+0000","last_undegraded":"2024-03-09T09:29:36.349389+0000","last_fullsized":"2024-03-09T09:29:36.349389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2353","last_scrub_stamp":"2024-03-09T09:29:36.349279+0000","last_deep_scrub":"89'2353","last_deep_scrub_stamp":"2024-03-09T09:29:36.349279+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.349279+0000","objects_scrubbed":105,"log_size":2353,"log_dups_size":0,"ondisk_log_size":2353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:48:34.518396+0000","scrub_duration":171,"objects_trimmed":0,"snaptrim_duration":0.81694827699999994,"stat_sum":{"num_bytes":851628,"num_objects":105,"num_object_clones":33,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":33,"num_read":114,"num_read_kb":26314,"num_write":1339,"num_write_kb":37517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"89'569","reported_seq":746,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.941125+0000","last_change":"2024-03-09T09:29:32.941257+0000","last_active":"2024-03-09T09:29:32.941125+0000","last_peered":"2024-03-09T09:29:32.941125+0000","last_clean":"2024-03-09T09:29:32.941125+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T09:29:32.941125+0000","last_undegraded":"2024-03-09T09:29:32.941125+0000","last_fullsized":"2024-03-09T09:29:32.941125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'569","last_scrub_stamp":"2024-03-09T09:29:32.941005+0000","last_deep_scrub":"89'569","last_deep_scrub_stamp":"2024-03-09T09:29:32.941005+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.941005+0000","objects_scrubbed":49,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:00:09.932346+0000","scrub_duration":177,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554454,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":604,"num_write_kb":91576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":684517,"num_omap_keys":1040,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"89'740","reported_seq":912,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.997100+0000","last_change":"2024-03-09T09:29:30.997177+0000","last_active":"2024-03-09T09:29:30.997100+0000","last_peered":"2024-03-09T09:29:30.997100+0000","last_clean":"2024-03-09T09:29:30.997100+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T09:29:30.997100+0000","last_undegraded":"2024-03-09T09:29:30.997100+0000","last_fullsized":"2024-03-09T09:29:30.997100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'740","last_scrub_stamp":"2024-03-09T09:29:30.996979+0000","last_deep_scrub":"89'740","last_deep_scrub_stamp":"2024-03-09T09:29:30.996979+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.996979+0000","objects_scrubbed":57,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:12:39.922259+0000","scrub_duration":202,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":783,"num_write_kb":95503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":674714,"num_omap_keys":1159,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"89'2304","reported_seq":5196,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.952186+0000","last_change":"2024-03-09T09:27:34.538055+0000","last_active":"2024-03-09T09:29:17.952186+0000","last_peered":"2024-03-09T09:29:17.952186+0000","last_clean":"2024-03-09T09:29:17.952186+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T09:29:17.952186+0000","last_undegraded":"2024-03-09T09:29:17.952186+0000","last_fullsized":"2024-03-09T09:29:17.952186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2304,"log_dups_size":0,"ondisk_log_size":2304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.1951373749999998,"stat_sum":{"num_bytes":1358361,"num_objects":133,"num_object_clones":45,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":45,"num_read":125,"num_read_kb":25752,"num_write":1343,"num_write_kb":45310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"89'2391","reported_seq":5377,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.368641+0000","last_change":"2024-03-09T09:29:34.368641+0000","last_active":"2024-03-09T09:29:34.368641+0000","last_peered":"2024-03-09T09:29:34.368641+0000","last_clean":"2024-03-09T09:29:34.368641+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T09:29:34.368641+0000","last_undegraded":"2024-03-09T09:29:34.368641+0000","last_fullsized":"2024-03-09T09:29:34.368641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2391","last_scrub_stamp":"2024-03-09T09:29:34.368459+0000","last_deep_scrub":"89'2391","last_deep_scrub_stamp":"2024-03-09T09:29:34.368459+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.368459+0000","objects_scrubbed":106,"log_size":2391,"log_dups_size":0,"ondisk_log_size":2391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:10:24.314293+0000","scrub_duration":184,"objects_trimmed":0,"snaptrim_duration":0.30155770100000001,"stat_sum":{"num_bytes":730111,"num_objects":106,"num_object_clones":35,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":35,"num_read":106,"num_read_kb":32982,"num_write":1381,"num_write_kb":56548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'813","reported_seq":992,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.585622+0000","last_change":"2024-03-09T09:29:28.585622+0000","last_active":"2024-03-09T09:29:28.585622+0000","last_peered":"2024-03-09T09:29:28.585622+0000","last_clean":"2024-03-09T09:29:28.585622+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T09:29:28.585622+0000","last_undegraded":"2024-03-09T09:29:28.585622+0000","last_fullsized":"2024-03-09T09:29:28.585622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'813","last_scrub_stamp":"2024-03-09T09:29:28.585444+0000","last_deep_scrub":"89'813","last_deep_scrub_stamp":"2024-03-09T09:29:28.585444+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.585444+0000","objects_scrubbed":64,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:35:23.927054+0000","scrub_duration":122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":74,"num_read_kb":0,"num_write":870,"num_write_kb":87759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":821123,"num_omap_keys":1317,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"89'656","reported_seq":826,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:24.066554+0000","last_change":"2024-03-09T09:29:24.066590+0000","last_active":"2024-03-09T09:29:24.066554+0000","last_peered":"2024-03-09T09:29:24.066554+0000","last_clean":"2024-03-09T09:29:24.066554+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T09:29:24.066554+0000","last_undegraded":"2024-03-09T09:29:24.066554+0000","last_fullsized":"2024-03-09T09:29:24.066554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'656","last_scrub_stamp":"2024-03-09T09:29:24.066487+0000","last_deep_scrub":"89'656","last_deep_scrub_stamp":"2024-03-09T09:29:24.066487+0000","last_clean_scrub_stamp":"2024-03-09T09:29:24.066487+0000","objects_scrubbed":54,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:27:17.167110+0000","scrub_duration":113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":669,"num_write_kb":112438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":801869,"num_omap_keys":1283,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"89'2284","reported_seq":5228,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:47.078831+0000","last_change":"2024-03-09T09:29:47.078874+0000","last_active":"2024-03-09T09:29:47.078831+0000","last_peered":"2024-03-09T09:29:47.078831+0000","last_clean":"2024-03-09T09:29:47.078831+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T09:29:47.078831+0000","last_undegraded":"2024-03-09T09:29:47.078831+0000","last_fullsized":"2024-03-09T09:29:47.078831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2284","last_scrub_stamp":"2024-03-09T09:29:47.078751+0000","last_deep_scrub":"89'2284","last_deep_scrub_stamp":"2024-03-09T09:29:47.078751+0000","last_clean_scrub_stamp":"2024-03-09T09:29:47.078751+0000","objects_scrubbed":100,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:41:16.716934+0000","scrub_duration":178,"objects_trimmed":0,"snaptrim_duration":4.4375742420000002,"stat_sum":{"num_bytes":1378177,"num_objects":100,"num_object_clones":26,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":26,"num_read":107,"num_read_kb":18463,"num_write":1316,"num_write_kb":41976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"89'2284","reported_seq":5289,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.668543+0000","last_change":"2024-03-09T09:29:27.668608+0000","last_active":"2024-03-09T09:29:27.668543+0000","last_peered":"2024-03-09T09:29:27.668543+0000","last_clean":"2024-03-09T09:29:27.668543+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T09:29:27.668543+0000","last_undegraded":"2024-03-09T09:29:27.668543+0000","last_fullsized":"2024-03-09T09:29:27.668543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2284","last_scrub_stamp":"2024-03-09T09:29:27.668439+0000","last_deep_scrub":"89'2284","last_deep_scrub_stamp":"2024-03-09T09:29:27.668439+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.668439+0000","objects_scrubbed":103,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:36:45.259300+0000","scrub_duration":219,"objects_trimmed":0,"snaptrim_duration":2.3444653409999998,"stat_sum":{"num_bytes":3375625,"num_objects":103,"num_object_clones":31,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":31,"num_read":106,"num_read_kb":36185,"num_write":1311,"num_write_kb":52081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"89'425","reported_seq":603,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.568640+0000","last_change":"2024-03-09T09:29:34.568640+0000","last_active":"2024-03-09T09:29:34.568640+0000","last_peered":"2024-03-09T09:29:34.568640+0000","last_clean":"2024-03-09T09:29:34.568640+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T09:29:34.568640+0000","last_undegraded":"2024-03-09T09:29:34.568640+0000","last_fullsized":"2024-03-09T09:29:34.568640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'425","last_scrub_stamp":"2024-03-09T09:29:34.568493+0000","last_deep_scrub":"89'425","last_deep_scrub_stamp":"2024-03-09T09:29:34.568493+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.568493+0000","objects_scrubbed":39,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:10.520113+0000","scrub_duration":154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":412,"num_write_kb":99378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":459011,"num_omap_keys":713,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"89'611","reported_seq":1021,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.520367+0000","last_change":"2024-03-09T09:29:33.520455+0000","last_active":"2024-03-09T09:29:33.520367+0000","last_peered":"2024-03-09T09:29:33.520367+0000","last_clean":"2024-03-09T09:29:33.520367+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T09:29:33.520367+0000","last_undegraded":"2024-03-09T09:29:33.520367+0000","last_fullsized":"2024-03-09T09:29:33.520367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'611","last_scrub_stamp":"2024-03-09T09:29:33.520170+0000","last_deep_scrub":"89'611","last_deep_scrub_stamp":"2024-03-09T09:29:33.520170+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.520170+0000","objects_scrubbed":51,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:30:44.823125+0000","scrub_duration":171,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007622,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":290,"num_read_kb":993,"num_write":609,"num_write_kb":88247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":534239,"num_omap_keys":903,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"89'2217","reported_seq":5124,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.928163+0000","last_change":"2024-03-09T09:27:34.708914+0000","last_active":"2024-03-09T09:29:17.928163+0000","last_peered":"2024-03-09T09:29:17.928163+0000","last_clean":"2024-03-09T09:29:17.928163+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T09:29:17.928163+0000","last_undegraded":"2024-03-09T09:29:17.928163+0000","last_fullsized":"2024-03-09T09:29:17.928163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2217,"log_dups_size":0,"ondisk_log_size":2217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4357263939999996,"stat_sum":{"num_bytes":1364696,"num_objects":131,"num_object_clones":50,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":50,"num_read":112,"num_read_kb":14161,"num_write":1321,"num_write_kb":29407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'2438","reported_seq":5411,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.285788+0000","last_change":"2024-03-09T09:29:29.285788+0000","last_active":"2024-03-09T09:29:29.285788+0000","last_peered":"2024-03-09T09:29:29.285788+0000","last_clean":"2024-03-09T09:29:29.285788+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T09:29:29.285788+0000","last_undegraded":"2024-03-09T09:29:29.285788+0000","last_fullsized":"2024-03-09T09:29:29.285788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2438","last_scrub_stamp":"2024-03-09T09:29:29.285678+0000","last_deep_scrub":"89'2438","last_deep_scrub_stamp":"2024-03-09T09:29:29.285678+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.285678+0000","objects_scrubbed":100,"log_size":2438,"log_dups_size":0,"ondisk_log_size":2438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:42:18.105349+0000","scrub_duration":147,"objects_trimmed":0,"snaptrim_duration":0.64721804299999997,"stat_sum":{"num_bytes":2530176,"num_objects":100,"num_object_clones":36,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":36,"num_read":119,"num_read_kb":17537,"num_write":1383,"num_write_kb":31574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'305","reported_seq":483,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.915455+0000","last_change":"2024-03-09T09:29:38.915505+0000","last_active":"2024-03-09T09:29:38.915455+0000","last_peered":"2024-03-09T09:29:38.915455+0000","last_clean":"2024-03-09T09:29:38.915455+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T09:29:38.915455+0000","last_undegraded":"2024-03-09T09:29:38.915455+0000","last_fullsized":"2024-03-09T09:29:38.915455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'305","last_scrub_stamp":"2024-03-09T09:29:38.915368+0000","last_deep_scrub":"89'305","last_deep_scrub_stamp":"2024-03-09T09:29:38.915368+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.915368+0000","objects_scrubbed":38,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:20:52.360008+0000","scrub_duration":69,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":313,"num_write_kb":62737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":405052,"num_omap_keys":616,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"89'875","reported_seq":1043,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.337271+0000","last_change":"2024-03-09T09:29:25.337360+0000","last_active":"2024-03-09T09:29:25.337271+0000","last_peered":"2024-03-09T09:29:25.337271+0000","last_clean":"2024-03-09T09:29:25.337271+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T09:29:25.337271+0000","last_undegraded":"2024-03-09T09:29:25.337271+0000","last_fullsized":"2024-03-09T09:29:25.337271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'875","last_scrub_stamp":"2024-03-09T09:29:25.337193+0000","last_deep_scrub":"89'875","last_deep_scrub_stamp":"2024-03-09T09:29:25.337193+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.337193+0000","objects_scrubbed":46,"log_size":875,"log_dups_size":0,"ondisk_log_size":875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:21:18.748241+0000","scrub_duration":89,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":880,"num_write_kb":124192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":605867,"num_omap_keys":715,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"89'2330","reported_seq":5250,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:48.498117+0000","last_change":"2024-03-09T09:29:48.498156+0000","last_active":"2024-03-09T09:29:48.498117+0000","last_peered":"2024-03-09T09:29:48.498117+0000","last_clean":"2024-03-09T09:29:48.498117+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T09:29:48.498117+0000","last_undegraded":"2024-03-09T09:29:48.498117+0000","last_fullsized":"2024-03-09T09:29:48.498117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2330","last_scrub_stamp":"2024-03-09T09:29:48.497986+0000","last_deep_scrub":"89'2330","last_deep_scrub_stamp":"2024-03-09T09:29:48.497986+0000","last_clean_scrub_stamp":"2024-03-09T09:29:48.497986+0000","objects_scrubbed":102,"log_size":2330,"log_dups_size":0,"ondisk_log_size":2330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:03:50.415412+0000","scrub_duration":197,"objects_trimmed":6,"snaptrim_duration":4.2113280319999999,"stat_sum":{"num_bytes":5678953,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":106,"num_read_kb":14364,"num_write":1330,"num_write_kb":47101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"89'2273","reported_seq":5294,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.197398+0000","last_change":"2024-03-09T09:29:35.197398+0000","last_active":"2024-03-09T09:29:35.197398+0000","last_peered":"2024-03-09T09:29:35.197398+0000","last_clean":"2024-03-09T09:29:35.197398+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T09:29:35.197398+0000","last_undegraded":"2024-03-09T09:29:35.197398+0000","last_fullsized":"2024-03-09T09:29:35.197398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2273","last_scrub_stamp":"2024-03-09T09:29:35.197297+0000","last_deep_scrub":"89'2273","last_deep_scrub_stamp":"2024-03-09T09:29:35.197297+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.197297+0000","objects_scrubbed":124,"log_size":2273,"log_dups_size":0,"ondisk_log_size":2273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:16:48.175211+0000","scrub_duration":177,"objects_trimmed":0,"snaptrim_duration":0.096588679999999996,"stat_sum":{"num_bytes":1266796,"num_objects":124,"num_object_clones":39,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":39,"num_read":107,"num_read_kb":17887,"num_write":1323,"num_write_kb":41057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"89'647","reported_seq":826,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.093883+0000","last_change":"2024-03-09T09:29:28.093883+0000","last_active":"2024-03-09T09:29:28.093883+0000","last_peered":"2024-03-09T09:29:28.093883+0000","last_clean":"2024-03-09T09:29:28.093883+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T09:29:28.093883+0000","last_undegraded":"2024-03-09T09:29:28.093883+0000","last_fullsized":"2024-03-09T09:29:28.093883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'647","last_scrub_stamp":"2024-03-09T09:29:28.093716+0000","last_deep_scrub":"89'647","last_deep_scrub_stamp":"2024-03-09T09:29:28.093716+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.093716+0000","objects_scrubbed":53,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:23:58.108279+0000","scrub_duration":131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":687,"num_write_kb":95812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":576914,"num_omap_keys":966,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"89'1219","reported_seq":1388,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.916765+0000","last_change":"2024-03-09T09:29:30.916912+0000","last_active":"2024-03-09T09:29:30.916765+0000","last_peered":"2024-03-09T09:29:30.916765+0000","last_clean":"2024-03-09T09:29:30.916765+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T09:29:30.916765+0000","last_undegraded":"2024-03-09T09:29:30.916765+0000","last_fullsized":"2024-03-09T09:29:30.916765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1219","last_scrub_stamp":"2024-03-09T09:29:30.916643+0000","last_deep_scrub":"89'1219","last_deep_scrub_stamp":"2024-03-09T09:29:30.916643+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.916643+0000","objects_scrubbed":43,"log_size":1219,"log_dups_size":0,"ondisk_log_size":1219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:53:23.719504+0000","scrub_duration":101,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777306,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":1250,"num_write_kb":83773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":524629,"num_omap_keys":853,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"89'2460","reported_seq":5374,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:47.421366+0000","last_change":"2024-03-09T09:29:47.421417+0000","last_active":"2024-03-09T09:29:47.421366+0000","last_peered":"2024-03-09T09:29:47.421366+0000","last_clean":"2024-03-09T09:29:47.421366+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T09:29:47.421366+0000","last_undegraded":"2024-03-09T09:29:47.421366+0000","last_fullsized":"2024-03-09T09:29:47.421366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2460","last_scrub_stamp":"2024-03-09T09:29:47.421304+0000","last_deep_scrub":"89'2460","last_deep_scrub_stamp":"2024-03-09T09:29:47.421304+0000","last_clean_scrub_stamp":"2024-03-09T09:29:47.421304+0000","objects_scrubbed":96,"log_size":2460,"log_dups_size":0,"ondisk_log_size":2460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:13:28.293465+0000","scrub_duration":168,"objects_trimmed":4,"snaptrim_duration":4.2009582749999996,"stat_sum":{"num_bytes":273727,"num_objects":96,"num_object_clones":34,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":34,"num_read":130,"num_read_kb":37554,"num_write":1394,"num_write_kb":63694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"89'2354","reported_seq":5332,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.440003+0000","last_change":"2024-03-09T09:29:38.440041+0000","last_active":"2024-03-09T09:29:38.440003+0000","last_peered":"2024-03-09T09:29:38.440003+0000","last_clean":"2024-03-09T09:29:38.440003+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T09:29:38.440003+0000","last_undegraded":"2024-03-09T09:29:38.440003+0000","last_fullsized":"2024-03-09T09:29:38.440003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2354","last_scrub_stamp":"2024-03-09T09:29:38.439939+0000","last_deep_scrub":"89'2354","last_deep_scrub_stamp":"2024-03-09T09:29:38.439939+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.439939+0000","objects_scrubbed":133,"log_size":2354,"log_dups_size":0,"ondisk_log_size":2354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:24:51.287488+0000","scrub_duration":260,"objects_trimmed":0,"snaptrim_duration":0.19064995900000001,"stat_sum":{"num_bytes":1055946,"num_objects":133,"num_object_clones":51,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":51,"num_read":117,"num_read_kb":18332,"num_write":1305,"num_write_kb":38006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"89'1087","reported_seq":1265,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.588370+0000","last_change":"2024-03-09T09:29:32.588370+0000","last_active":"2024-03-09T09:29:32.588370+0000","last_peered":"2024-03-09T09:29:32.588370+0000","last_clean":"2024-03-09T09:29:32.588370+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T09:29:32.588370+0000","last_undegraded":"2024-03-09T09:29:32.588370+0000","last_fullsized":"2024-03-09T09:29:32.588370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1087","last_scrub_stamp":"2024-03-09T09:29:32.588105+0000","last_deep_scrub":"89'1087","last_deep_scrub_stamp":"2024-03-09T09:29:32.588105+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.588105+0000","objects_scrubbed":48,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:10:42.375813+0000","scrub_duration":154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":58,"num_read_kb":0,"num_write":1092,"num_write_kb":124321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":613889,"num_omap_keys":1009,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"89'340","reported_seq":509,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.177199+0000","last_change":"2024-03-09T09:29:27.177305+0000","last_active":"2024-03-09T09:29:27.177199+0000","last_peered":"2024-03-09T09:29:27.177199+0000","last_clean":"2024-03-09T09:29:27.177199+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T09:29:27.177199+0000","last_undegraded":"2024-03-09T09:29:27.177199+0000","last_fullsized":"2024-03-09T09:29:27.177199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'340","last_scrub_stamp":"2024-03-09T09:29:27.177105+0000","last_deep_scrub":"89'340","last_deep_scrub_stamp":"2024-03-09T09:29:27.177105+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.177105+0000","objects_scrubbed":43,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:44:35.621276+0000","scrub_duration":81,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":328,"num_write_kb":62283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":599772,"num_omap_keys":972,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"89'2304","reported_seq":5247,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.141120+0000","last_change":"2024-03-09T09:29:42.141171+0000","last_active":"2024-03-09T09:29:42.141120+0000","last_peered":"2024-03-09T09:29:42.141120+0000","last_clean":"2024-03-09T09:29:42.141120+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T09:29:42.141120+0000","last_undegraded":"2024-03-09T09:29:42.141120+0000","last_fullsized":"2024-03-09T09:29:42.141120+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2304","last_scrub_stamp":"2024-03-09T09:29:42.140985+0000","last_deep_scrub":"89'2304","last_deep_scrub_stamp":"2024-03-09T09:29:42.140985+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.140985+0000","objects_scrubbed":130,"log_size":2304,"log_dups_size":0,"ondisk_log_size":2304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:56:57.948152+0000","scrub_duration":246,"objects_trimmed":0,"snaptrim_duration":1.695386042,"stat_sum":{"num_bytes":3385389,"num_objects":130,"num_object_clones":42,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":42,"num_read":111,"num_read_kb":40727,"num_write":1306,"num_write_kb":67855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"89'2331","reported_seq":5308,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.804448+0000","last_change":"2024-03-09T09:29:37.804518+0000","last_active":"2024-03-09T09:29:37.804448+0000","last_peered":"2024-03-09T09:29:37.804448+0000","last_clean":"2024-03-09T09:29:37.804448+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T09:29:37.804448+0000","last_undegraded":"2024-03-09T09:29:37.804448+0000","last_fullsized":"2024-03-09T09:29:37.804448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2331","last_scrub_stamp":"2024-03-09T09:29:37.804328+0000","last_deep_scrub":"89'2331","last_deep_scrub_stamp":"2024-03-09T09:29:37.804328+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.804328+0000","objects_scrubbed":94,"log_size":2331,"log_dups_size":0,"ondisk_log_size":2331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:18:40.635031+0000","scrub_duration":134,"objects_trimmed":0,"snaptrim_duration":3.3666159750000002,"stat_sum":{"num_bytes":799596,"num_objects":94,"num_object_clones":30,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":30,"num_read":103,"num_read_kb":34868,"num_write":1326,"num_write_kb":46782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'1346","reported_seq":1523,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.949434+0000","last_change":"2024-03-09T09:29:35.949536+0000","last_active":"2024-03-09T09:29:35.949434+0000","last_peered":"2024-03-09T09:29:35.949434+0000","last_clean":"2024-03-09T09:29:35.949434+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T09:29:35.949434+0000","last_undegraded":"2024-03-09T09:29:35.949434+0000","last_fullsized":"2024-03-09T09:29:35.949434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1346","last_scrub_stamp":"2024-03-09T09:29:35.949327+0000","last_deep_scrub":"89'1346","last_deep_scrub_stamp":"2024-03-09T09:29:35.949327+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.949327+0000","objects_scrubbed":48,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:59:02.731368+0000","scrub_duration":87,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":80,"num_read_kb":0,"num_write":1400,"num_write_kb":80149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852662,"num_omap_keys":1288,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"89'461","reported_seq":631,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.801109+0000","last_change":"2024-03-09T09:29:38.801135+0000","last_active":"2024-03-09T09:29:38.801109+0000","last_peered":"2024-03-09T09:29:38.801109+0000","last_clean":"2024-03-09T09:29:38.801109+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T09:29:38.801109+0000","last_undegraded":"2024-03-09T09:29:38.801109+0000","last_fullsized":"2024-03-09T09:29:38.801109+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'461","last_scrub_stamp":"2024-03-09T09:29:38.801026+0000","last_deep_scrub":"89'461","last_deep_scrub_stamp":"2024-03-09T09:29:38.801026+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.801026+0000","objects_scrubbed":50,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:46.713037+0000","scrub_duration":123,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":462,"num_write_kb":95092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":374202,"num_omap_keys":642,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"89'2374","reported_seq":5308,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.430413+0000","last_change":"2024-03-09T09:29:45.430466+0000","last_active":"2024-03-09T09:29:45.430413+0000","last_peered":"2024-03-09T09:29:45.430413+0000","last_clean":"2024-03-09T09:29:45.430413+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T09:29:45.430413+0000","last_undegraded":"2024-03-09T09:29:45.430413+0000","last_fullsized":"2024-03-09T09:29:45.430413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2374","last_scrub_stamp":"2024-03-09T09:29:45.430327+0000","last_deep_scrub":"89'2374","last_deep_scrub_stamp":"2024-03-09T09:29:45.430327+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.430327+0000","objects_scrubbed":103,"log_size":2374,"log_dups_size":0,"ondisk_log_size":2374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:27:17.101981+0000","scrub_duration":176,"objects_trimmed":0,"snaptrim_duration":0.64490501099999997,"stat_sum":{"num_bytes":1735737,"num_objects":103,"num_object_clones":32,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":32,"num_read":113,"num_read_kb":37575,"num_write":1379,"num_write_kb":57434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"89'2414","reported_seq":5378,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.180587+0000","last_change":"2024-03-09T09:29:43.180642+0000","last_active":"2024-03-09T09:29:43.180587+0000","last_peered":"2024-03-09T09:29:43.180587+0000","last_clean":"2024-03-09T09:29:43.180587+0000","last_became_active":"2024-03-09T08:53:00.254589+0000","last_became_peered":"2024-03-09T08:53:00.254589+0000","last_unstale":"2024-03-09T09:29:43.180587+0000","last_undegraded":"2024-03-09T09:29:43.180587+0000","last_fullsized":"2024-03-09T09:29:43.180587+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2414","last_scrub_stamp":"2024-03-09T09:29:43.180507+0000","last_deep_scrub":"89'2414","last_deep_scrub_stamp":"2024-03-09T09:29:43.180507+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.180507+0000","objects_scrubbed":123,"log_size":2414,"log_dups_size":0,"ondisk_log_size":2414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:11:55.562575+0000","scrub_duration":214,"objects_trimmed":0,"snaptrim_duration":0.471373141,"stat_sum":{"num_bytes":5513596,"num_objects":123,"num_object_clones":39,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":39,"num_read":114,"num_read_kb":29387,"num_write":1331,"num_write_kb":57801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'618","reported_seq":795,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.986665+0000","last_change":"2024-03-09T09:29:42.986696+0000","last_active":"2024-03-09T09:29:42.986665+0000","last_peered":"2024-03-09T09:29:42.986665+0000","last_clean":"2024-03-09T09:29:42.986665+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T09:29:42.986665+0000","last_undegraded":"2024-03-09T09:29:42.986665+0000","last_fullsized":"2024-03-09T09:29:42.986665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'618","last_scrub_stamp":"2024-03-09T09:29:42.986545+0000","last_deep_scrub":"89'618","last_deep_scrub_stamp":"2024-03-09T09:29:42.986545+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.986545+0000","objects_scrubbed":28,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:44:35.113222+0000","scrub_duration":92,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":598,"num_write_kb":111422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":173539,"num_omap_keys":299,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"89'785","reported_seq":955,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.372377+0000","last_change":"2024-03-09T09:29:26.372377+0000","last_active":"2024-03-09T09:29:26.372377+0000","last_peered":"2024-03-09T09:29:26.372377+0000","last_clean":"2024-03-09T09:29:26.372377+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T09:29:26.372377+0000","last_undegraded":"2024-03-09T09:29:26.372377+0000","last_fullsized":"2024-03-09T09:29:26.372377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'785","last_scrub_stamp":"2024-03-09T09:29:26.372276+0000","last_deep_scrub":"89'785","last_deep_scrub_stamp":"2024-03-09T09:29:26.372276+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.372276+0000","objects_scrubbed":56,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:43:47.007320+0000","scrub_duration":104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":81,"num_read_kb":0,"num_write":849,"num_write_kb":91941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":871124,"num_omap_keys":1334,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"89'2639","reported_seq":5456,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.323969+0000","last_change":"2024-03-09T09:29:39.324027+0000","last_active":"2024-03-09T09:29:39.323969+0000","last_peered":"2024-03-09T09:29:39.323969+0000","last_clean":"2024-03-09T09:29:39.323969+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T09:29:39.323969+0000","last_undegraded":"2024-03-09T09:29:39.323969+0000","last_fullsized":"2024-03-09T09:29:39.323969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2639","last_scrub_stamp":"2024-03-09T09:29:39.323851+0000","last_deep_scrub":"89'2639","last_deep_scrub_stamp":"2024-03-09T09:29:39.323851+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.323851+0000","objects_scrubbed":111,"log_size":2639,"log_dups_size":0,"ondisk_log_size":2639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:21:51.409246+0000","scrub_duration":160,"objects_trimmed":0,"snaptrim_duration":4.4377723019999999,"stat_sum":{"num_bytes":1160060,"num_objects":111,"num_object_clones":35,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":35,"num_read":113,"num_read_kb":9969,"num_write":1381,"num_write_kb":17733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"89'2272","reported_seq":5281,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.941475+0000","last_change":"2024-03-09T09:29:28.941508+0000","last_active":"2024-03-09T09:29:28.941475+0000","last_peered":"2024-03-09T09:29:28.941475+0000","last_clean":"2024-03-09T09:29:28.941475+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T09:29:28.941475+0000","last_undegraded":"2024-03-09T09:29:28.941475+0000","last_fullsized":"2024-03-09T09:29:28.941475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2272","last_scrub_stamp":"2024-03-09T09:29:28.941403+0000","last_deep_scrub":"89'2272","last_deep_scrub_stamp":"2024-03-09T09:29:28.941403+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.941403+0000","objects_scrubbed":107,"log_size":2272,"log_dups_size":0,"ondisk_log_size":2272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:10:44.701638+0000","scrub_duration":152,"objects_trimmed":0,"snaptrim_duration":1.82863014,"stat_sum":{"num_bytes":1951923,"num_objects":107,"num_object_clones":29,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":29,"num_read":96,"num_read_kb":37066,"num_write":1331,"num_write_kb":53456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"89'601","reported_seq":1007,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.480009+0000","last_change":"2024-03-09T09:29:35.480088+0000","last_active":"2024-03-09T09:29:35.480009+0000","last_peered":"2024-03-09T09:29:35.480009+0000","last_clean":"2024-03-09T09:29:35.480009+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T09:29:35.480009+0000","last_undegraded":"2024-03-09T09:29:35.480009+0000","last_fullsized":"2024-03-09T09:29:35.480009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'601","last_scrub_stamp":"2024-03-09T09:29:35.479886+0000","last_deep_scrub":"89'601","last_deep_scrub_stamp":"2024-03-09T09:29:35.479886+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.479886+0000","objects_scrubbed":48,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:59.508107+0000","scrub_duration":106,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007933,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":267,"num_read_kb":985,"num_write":604,"num_write_kb":87856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":409708,"num_omap_keys":587,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"89'504","reported_seq":672,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.475879+0000","last_change":"2024-03-09T09:29:29.476044+0000","last_active":"2024-03-09T09:29:29.475879+0000","last_peered":"2024-03-09T09:29:29.475879+0000","last_clean":"2024-03-09T09:29:29.475879+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T09:29:29.475879+0000","last_undegraded":"2024-03-09T09:29:29.475879+0000","last_fullsized":"2024-03-09T09:29:29.475879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'504","last_scrub_stamp":"2024-03-09T09:29:29.475650+0000","last_deep_scrub":"89'504","last_deep_scrub_stamp":"2024-03-09T09:29:29.475650+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.475650+0000","objects_scrubbed":43,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:28:49.825813+0000","scrub_duration":189,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":506,"num_write_kb":99411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":543720,"num_omap_keys":804,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"89'2299","reported_seq":5235,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.419515+0000","last_change":"2024-03-09T09:29:44.419568+0000","last_active":"2024-03-09T09:29:44.419515+0000","last_peered":"2024-03-09T09:29:44.419515+0000","last_clean":"2024-03-09T09:29:44.419515+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T09:29:44.419515+0000","last_undegraded":"2024-03-09T09:29:44.419515+0000","last_fullsized":"2024-03-09T09:29:44.419515+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2299","last_scrub_stamp":"2024-03-09T09:29:44.419446+0000","last_deep_scrub":"89'2299","last_deep_scrub_stamp":"2024-03-09T09:29:44.419446+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.419446+0000","objects_scrubbed":110,"log_size":2299,"log_dups_size":0,"ondisk_log_size":2299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:58:45.243233+0000","scrub_duration":160,"objects_trimmed":0,"snaptrim_duration":0.64610520400000004,"stat_sum":{"num_bytes":1318782,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":124,"num_read_kb":9990,"num_write":1296,"num_write_kb":22209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"89'721","reported_seq":894,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.127524+0000","last_change":"2024-03-09T09:29:38.127674+0000","last_active":"2024-03-09T09:29:38.127524+0000","last_peered":"2024-03-09T09:29:38.127524+0000","last_clean":"2024-03-09T09:29:38.127524+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T09:29:38.127524+0000","last_undegraded":"2024-03-09T09:29:38.127524+0000","last_fullsized":"2024-03-09T09:29:38.127524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'721","last_scrub_stamp":"2024-03-09T09:29:38.127212+0000","last_deep_scrub":"89'721","last_deep_scrub_stamp":"2024-03-09T09:29:38.127212+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.127212+0000","objects_scrubbed":54,"log_size":721,"log_dups_size":0,"ondisk_log_size":721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:49:46.042055+0000","scrub_duration":149,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":72,"num_read_kb":0,"num_write":777,"num_write_kb":92056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":568534,"num_omap_keys":965,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'2330","reported_seq":5319,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.253759+0000","last_change":"2024-03-09T09:29:26.253814+0000","last_active":"2024-03-09T09:29:26.253759+0000","last_peered":"2024-03-09T09:29:26.253759+0000","last_clean":"2024-03-09T09:29:26.253759+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T09:29:26.253759+0000","last_undegraded":"2024-03-09T09:29:26.253759+0000","last_fullsized":"2024-03-09T09:29:26.253759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2330","last_scrub_stamp":"2024-03-09T09:29:26.253681+0000","last_deep_scrub":"89'2330","last_deep_scrub_stamp":"2024-03-09T09:29:26.253681+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.253681+0000","objects_scrubbed":120,"log_size":2330,"log_dups_size":0,"ondisk_log_size":2330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:25:05.771093+0000","scrub_duration":172,"objects_trimmed":0,"snaptrim_duration":1.23951623,"stat_sum":{"num_bytes":1295292,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":13266,"num_write":1323,"num_write_kb":26325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"89'907","reported_seq":1084,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.004768+0000","last_change":"2024-03-09T09:29:42.004865+0000","last_active":"2024-03-09T09:29:42.004768+0000","last_peered":"2024-03-09T09:29:42.004768+0000","last_clean":"2024-03-09T09:29:42.004768+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T09:29:42.004768+0000","last_undegraded":"2024-03-09T09:29:42.004768+0000","last_fullsized":"2024-03-09T09:29:42.004768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'907","last_scrub_stamp":"2024-03-09T09:29:42.004658+0000","last_deep_scrub":"89'907","last_deep_scrub_stamp":"2024-03-09T09:29:42.004658+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.004658+0000","objects_scrubbed":46,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:27:34.410597+0000","scrub_duration":133,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":45,"num_read_kb":0,"num_write":880,"num_write_kb":119947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":513434,"num_omap_keys":845,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"89'2251","reported_seq":5202,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.149013+0000","last_change":"2024-03-09T09:29:41.149074+0000","last_active":"2024-03-09T09:29:41.149013+0000","last_peered":"2024-03-09T09:29:41.149013+0000","last_clean":"2024-03-09T09:29:41.149013+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T09:29:41.149013+0000","last_undegraded":"2024-03-09T09:29:41.149013+0000","last_fullsized":"2024-03-09T09:29:41.149013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2251","last_scrub_stamp":"2024-03-09T09:29:41.148890+0000","last_deep_scrub":"89'2251","last_deep_scrub_stamp":"2024-03-09T09:29:41.148890+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.148890+0000","objects_scrubbed":107,"log_size":2251,"log_dups_size":0,"ondisk_log_size":2251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:24:46.012715+0000","scrub_duration":179,"objects_trimmed":2,"snaptrim_duration":0.70511981300000004,"stat_sum":{"num_bytes":1915718,"num_objects":107,"num_object_clones":37,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":37,"num_read":111,"num_read_kb":24546,"num_write":1285,"num_write_kb":44187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"89'596","reported_seq":768,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.437187+0000","last_change":"2024-03-09T09:29:40.437230+0000","last_active":"2024-03-09T09:29:40.437187+0000","last_peered":"2024-03-09T09:29:40.437187+0000","last_clean":"2024-03-09T09:29:40.437187+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T09:29:40.437187+0000","last_undegraded":"2024-03-09T09:29:40.437187+0000","last_fullsized":"2024-03-09T09:29:40.437187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'596","last_scrub_stamp":"2024-03-09T09:29:40.437090+0000","last_deep_scrub":"89'596","last_deep_scrub_stamp":"2024-03-09T09:29:40.437090+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.437090+0000","objects_scrubbed":50,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:43:24.326589+0000","scrub_duration":114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":589,"num_write_kb":82799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":492001,"num_omap_keys":847,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'2568","reported_seq":5501,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.496342+0000","last_change":"2024-03-09T09:29:43.496392+0000","last_active":"2024-03-09T09:29:43.496342+0000","last_peered":"2024-03-09T09:29:43.496342+0000","last_clean":"2024-03-09T09:29:43.496342+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T09:29:43.496342+0000","last_undegraded":"2024-03-09T09:29:43.496342+0000","last_fullsized":"2024-03-09T09:29:43.496342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2568","last_scrub_stamp":"2024-03-09T09:29:43.496266+0000","last_deep_scrub":"89'2568","last_deep_scrub_stamp":"2024-03-09T09:29:43.496266+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.496266+0000","objects_scrubbed":93,"log_size":2568,"log_dups_size":0,"ondisk_log_size":2568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:04:36.024377+0000","scrub_duration":181,"objects_trimmed":0,"snaptrim_duration":0.79565864399999997,"stat_sum":{"num_bytes":2512650,"num_objects":93,"num_object_clones":29,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":29,"num_read":114,"num_read_kb":34867,"num_write":1447,"num_write_kb":63344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'447","reported_seq":623,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.448400+0000","last_change":"2024-03-09T09:29:41.448512+0000","last_active":"2024-03-09T09:29:41.448400+0000","last_peered":"2024-03-09T09:29:41.448400+0000","last_clean":"2024-03-09T09:29:41.448400+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T09:29:41.448400+0000","last_undegraded":"2024-03-09T09:29:41.448400+0000","last_fullsized":"2024-03-09T09:29:41.448400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'447","last_scrub_stamp":"2024-03-09T09:29:41.448251+0000","last_deep_scrub":"89'447","last_deep_scrub_stamp":"2024-03-09T09:29:41.448251+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.448251+0000","objects_scrubbed":47,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:48:44.843582+0000","scrub_duration":116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":480,"num_write_kb":79281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":589869,"num_omap_keys":880,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"89'2289","reported_seq":5240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.144833+0000","last_change":"2024-03-09T09:29:44.144881+0000","last_active":"2024-03-09T09:29:44.144833+0000","last_peered":"2024-03-09T09:29:44.144833+0000","last_clean":"2024-03-09T09:29:44.144833+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T09:29:44.144833+0000","last_undegraded":"2024-03-09T09:29:44.144833+0000","last_fullsized":"2024-03-09T09:29:44.144833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2289","last_scrub_stamp":"2024-03-09T09:29:44.144740+0000","last_deep_scrub":"89'2289","last_deep_scrub_stamp":"2024-03-09T09:29:44.144740+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.144740+0000","objects_scrubbed":112,"log_size":2289,"log_dups_size":0,"ondisk_log_size":2289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:19:24.277820+0000","scrub_duration":274,"objects_trimmed":4,"snaptrim_duration":1.72318536,"stat_sum":{"num_bytes":768948,"num_objects":112,"num_object_clones":37,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":37,"num_read":126,"num_read_kb":26327,"num_write":1312,"num_write_kb":46360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"89'779","reported_seq":956,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.511927+0000","last_change":"2024-03-09T09:29:31.511927+0000","last_active":"2024-03-09T09:29:31.511927+0000","last_peered":"2024-03-09T09:29:31.511927+0000","last_clean":"2024-03-09T09:29:31.511927+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T09:29:31.511927+0000","last_undegraded":"2024-03-09T09:29:31.511927+0000","last_fullsized":"2024-03-09T09:29:31.511927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'779","last_scrub_stamp":"2024-03-09T09:29:31.511787+0000","last_deep_scrub":"89'779","last_deep_scrub_stamp":"2024-03-09T09:29:31.511787+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.511787+0000","objects_scrubbed":40,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:36:36.049606+0000","scrub_duration":89,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":777,"num_write_kb":95156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":299260,"num_omap_keys":519,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"89'2434","reported_seq":5433,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.212547+0000","last_change":"2024-03-09T09:29:33.212607+0000","last_active":"2024-03-09T09:29:33.212547+0000","last_peered":"2024-03-09T09:29:33.212547+0000","last_clean":"2024-03-09T09:29:33.212547+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T09:29:33.212547+0000","last_undegraded":"2024-03-09T09:29:33.212547+0000","last_fullsized":"2024-03-09T09:29:33.212547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2434","last_scrub_stamp":"2024-03-09T09:29:33.212411+0000","last_deep_scrub":"89'2434","last_deep_scrub_stamp":"2024-03-09T09:29:33.212411+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.212411+0000","objects_scrubbed":128,"log_size":2434,"log_dups_size":0,"ondisk_log_size":2434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:40:06.160484+0000","scrub_duration":228,"objects_trimmed":0,"snaptrim_duration":0.65597337700000002,"stat_sum":{"num_bytes":5623986,"num_objects":128,"num_object_clones":44,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":44,"num_read":148,"num_read_kb":35274,"num_write":1372,"num_write_kb":72006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"89'513","reported_seq":681,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.596769+0000","last_change":"2024-03-09T09:29:32.596851+0000","last_active":"2024-03-09T09:29:32.596769+0000","last_peered":"2024-03-09T09:29:32.596769+0000","last_clean":"2024-03-09T09:29:32.596769+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T09:29:32.596769+0000","last_undegraded":"2024-03-09T09:29:32.596769+0000","last_fullsized":"2024-03-09T09:29:32.596769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'513","last_scrub_stamp":"2024-03-09T09:29:32.596705+0000","last_deep_scrub":"89'513","last_deep_scrub_stamp":"2024-03-09T09:29:32.596705+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.596705+0000","objects_scrubbed":42,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:55:34.301830+0000","scrub_duration":111,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":514,"num_write_kb":86887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":450365,"num_omap_keys":682,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"89'2567","reported_seq":5463,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.890370+0000","last_change":"2024-03-09T09:29:46.890621+0000","last_active":"2024-03-09T09:29:46.890370+0000","last_peered":"2024-03-09T09:29:46.890370+0000","last_clean":"2024-03-09T09:29:46.890370+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T09:29:46.890370+0000","last_undegraded":"2024-03-09T09:29:46.890370+0000","last_fullsized":"2024-03-09T09:29:46.890370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2567","last_scrub_stamp":"2024-03-09T09:29:46.890133+0000","last_deep_scrub":"89'2567","last_deep_scrub_stamp":"2024-03-09T09:29:46.890133+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.890133+0000","objects_scrubbed":147,"log_size":2567,"log_dups_size":0,"ondisk_log_size":2567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:58:06.268182+0000","scrub_duration":357,"objects_trimmed":0,"snaptrim_duration":3.9237505820000003,"stat_sum":{"num_bytes":5533714,"num_objects":147,"num_object_clones":51,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":51,"num_read":136,"num_read_kb":30612,"num_write":1437,"num_write_kb":51393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"89'670","reported_seq":847,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.424152+0000","last_change":"2024-03-09T09:29:40.424152+0000","last_active":"2024-03-09T09:29:40.424152+0000","last_peered":"2024-03-09T09:29:40.424152+0000","last_clean":"2024-03-09T09:29:40.424152+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T09:29:40.424152+0000","last_undegraded":"2024-03-09T09:29:40.424152+0000","last_fullsized":"2024-03-09T09:29:40.424152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'670","last_scrub_stamp":"2024-03-09T09:29:40.423973+0000","last_deep_scrub":"89'670","last_deep_scrub_stamp":"2024-03-09T09:29:40.423973+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.423973+0000","objects_scrubbed":50,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:53.222882+0000","scrub_duration":127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17956277,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":61,"num_read_kb":0,"num_write":701,"num_write_kb":108966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":563156,"num_omap_keys":856,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"89'2322","reported_seq":5313,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.810141+0000","last_change":"2024-03-09T09:29:35.810202+0000","last_active":"2024-03-09T09:29:35.810141+0000","last_peered":"2024-03-09T09:29:35.810141+0000","last_clean":"2024-03-09T09:29:35.810141+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T09:29:35.810141+0000","last_undegraded":"2024-03-09T09:29:35.810141+0000","last_fullsized":"2024-03-09T09:29:35.810141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2322","last_scrub_stamp":"2024-03-09T09:29:35.810044+0000","last_deep_scrub":"89'2322","last_deep_scrub_stamp":"2024-03-09T09:29:35.810044+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.810044+0000","objects_scrubbed":89,"log_size":2322,"log_dups_size":0,"ondisk_log_size":2322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:37:09.823607+0000","scrub_duration":129,"objects_trimmed":0,"snaptrim_duration":3.1639723850000001,"stat_sum":{"num_bytes":538194,"num_objects":89,"num_object_clones":30,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":30,"num_read":112,"num_read_kb":30041,"num_write":1325,"num_write_kb":47681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"89'828","reported_seq":996,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.420219+0000","last_change":"2024-03-09T09:29:27.420261+0000","last_active":"2024-03-09T09:29:27.420219+0000","last_peered":"2024-03-09T09:29:27.420219+0000","last_clean":"2024-03-09T09:29:27.420219+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T09:29:27.420219+0000","last_undegraded":"2024-03-09T09:29:27.420219+0000","last_fullsized":"2024-03-09T09:29:27.420219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'828","last_scrub_stamp":"2024-03-09T09:29:27.420151+0000","last_deep_scrub":"89'828","last_deep_scrub_stamp":"2024-03-09T09:29:27.420151+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.420151+0000","objects_scrubbed":60,"log_size":828,"log_dups_size":0,"ondisk_log_size":828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:46:33.569655+0000","scrub_duration":145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":79,"num_read_kb":0,"num_write":862,"num_write_kb":140804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":670100,"num_omap_keys":1071,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"89'2380","reported_seq":5297,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:23.606408+0000","last_change":"2024-03-09T09:27:34.560887+0000","last_active":"2024-03-09T09:29:23.606408+0000","last_peered":"2024-03-09T09:29:23.606408+0000","last_clean":"2024-03-09T09:29:23.606408+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T09:29:23.606408+0000","last_undegraded":"2024-03-09T09:29:23.606408+0000","last_fullsized":"2024-03-09T09:29:23.606408+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2380,"log_dups_size":0,"ondisk_log_size":2380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.2173252530000003,"stat_sum":{"num_bytes":794839,"num_objects":125,"num_object_clones":42,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":42,"num_read":135,"num_read_kb":30019,"num_write":1425,"num_write_kb":47795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"89'2375","reported_seq":5367,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.797726+0000","last_change":"2024-03-09T09:29:28.797820+0000","last_active":"2024-03-09T09:29:28.797726+0000","last_peered":"2024-03-09T09:29:28.797726+0000","last_clean":"2024-03-09T09:29:28.797726+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T09:29:28.797726+0000","last_undegraded":"2024-03-09T09:29:28.797726+0000","last_fullsized":"2024-03-09T09:29:28.797726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2375","last_scrub_stamp":"2024-03-09T09:29:28.797595+0000","last_deep_scrub":"89'2375","last_deep_scrub_stamp":"2024-03-09T09:29:28.797595+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.797595+0000","objects_scrubbed":120,"log_size":2375,"log_dups_size":0,"ondisk_log_size":2375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:49:05.083497+0000","scrub_duration":320,"objects_trimmed":0,"snaptrim_duration":1.8676528270000001,"stat_sum":{"num_bytes":501236,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":17494,"num_write":1356,"num_write_kb":32845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"89'2134","reported_seq":5064,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:22.569260+0000","last_change":"2024-03-09T09:27:32.084492+0000","last_active":"2024-03-09T09:29:22.569260+0000","last_peered":"2024-03-09T09:29:22.569260+0000","last_clean":"2024-03-09T09:29:22.569260+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T09:29:22.569260+0000","last_undegraded":"2024-03-09T09:29:22.569260+0000","last_fullsized":"2024-03-09T09:29:22.569260+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2134,"log_dups_size":0,"ondisk_log_size":2134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":1.7132004649999999,"stat_sum":{"num_bytes":1622605,"num_objects":113,"num_object_clones":36,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":36,"num_read":103,"num_read_kb":9361,"num_write":1283,"num_write_kb":37703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"89'2455","reported_seq":5424,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.616719+0000","last_change":"2024-03-09T09:29:30.616751+0000","last_active":"2024-03-09T09:29:30.616719+0000","last_peered":"2024-03-09T09:29:30.616719+0000","last_clean":"2024-03-09T09:29:30.616719+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T09:29:30.616719+0000","last_undegraded":"2024-03-09T09:29:30.616719+0000","last_fullsized":"2024-03-09T09:29:30.616719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2455","last_scrub_stamp":"2024-03-09T09:29:30.616633+0000","last_deep_scrub":"89'2455","last_deep_scrub_stamp":"2024-03-09T09:29:30.616633+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.616633+0000","objects_scrubbed":99,"log_size":2455,"log_dups_size":0,"ondisk_log_size":2455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:37:02.165755+0000","scrub_duration":180,"objects_trimmed":0,"snaptrim_duration":0.73410826500000004,"stat_sum":{"num_bytes":968837,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":110,"num_read_kb":13537,"num_write":1403,"num_write_kb":47183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"89'2377","reported_seq":5277,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.048548+0000","last_change":"2024-03-09T09:29:37.048589+0000","last_active":"2024-03-09T09:29:37.048548+0000","last_peered":"2024-03-09T09:29:37.048548+0000","last_clean":"2024-03-09T09:29:37.048548+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T09:29:37.048548+0000","last_undegraded":"2024-03-09T09:29:37.048548+0000","last_fullsized":"2024-03-09T09:29:37.048548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2377","last_scrub_stamp":"2024-03-09T09:29:37.048480+0000","last_deep_scrub":"89'2377","last_deep_scrub_stamp":"2024-03-09T09:29:37.048480+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.048480+0000","objects_scrubbed":126,"log_size":2377,"log_dups_size":0,"ondisk_log_size":2377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:30.619929+0000","scrub_duration":197,"objects_trimmed":2,"snaptrim_duration":4.446018724,"stat_sum":{"num_bytes":5408806,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":112,"num_read_kb":29960,"num_write":1313,"num_write_kb":48116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"89'2347","reported_seq":5357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.082159+0000","last_change":"2024-03-09T09:29:32.082223+0000","last_active":"2024-03-09T09:29:32.082159+0000","last_peered":"2024-03-09T09:29:32.082159+0000","last_clean":"2024-03-09T09:29:32.082159+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T09:29:32.082159+0000","last_undegraded":"2024-03-09T09:29:32.082159+0000","last_fullsized":"2024-03-09T09:29:32.082159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2347","last_scrub_stamp":"2024-03-09T09:29:32.082021+0000","last_deep_scrub":"89'2347","last_deep_scrub_stamp":"2024-03-09T09:29:32.082021+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.082021+0000","objects_scrubbed":110,"log_size":2347,"log_dups_size":0,"ondisk_log_size":2347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:38:10.823674+0000","scrub_duration":219,"objects_trimmed":0,"snaptrim_duration":1.706678991,"stat_sum":{"num_bytes":887348,"num_objects":110,"num_object_clones":37,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":37,"num_read":123,"num_read_kb":42348,"num_write":1334,"num_write_kb":66020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"89'2231","reported_seq":5193,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.130714+0000","last_change":"2024-03-09T09:29:43.130785+0000","last_active":"2024-03-09T09:29:43.130714+0000","last_peered":"2024-03-09T09:29:43.130714+0000","last_clean":"2024-03-09T09:29:43.130714+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T09:29:43.130714+0000","last_undegraded":"2024-03-09T09:29:43.130714+0000","last_fullsized":"2024-03-09T09:29:43.130714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2231","last_scrub_stamp":"2024-03-09T09:29:43.130621+0000","last_deep_scrub":"89'2231","last_deep_scrub_stamp":"2024-03-09T09:29:43.130621+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.130621+0000","objects_scrubbed":112,"log_size":2231,"log_dups_size":0,"ondisk_log_size":2231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:03.579083+0000","scrub_duration":222,"objects_trimmed":0,"snaptrim_duration":1.708282305,"stat_sum":{"num_bytes":596753,"num_objects":112,"num_object_clones":36,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":36,"num_read":111,"num_read_kb":13558,"num_write":1290,"num_write_kb":25078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"89'2507","reported_seq":5440,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:23.111168+0000","last_change":"2024-03-09T09:29:23.111217+0000","last_active":"2024-03-09T09:29:23.111168+0000","last_peered":"2024-03-09T09:29:23.111168+0000","last_clean":"2024-03-09T09:29:23.111168+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T09:29:23.111168+0000","last_undegraded":"2024-03-09T09:29:23.111168+0000","last_fullsized":"2024-03-09T09:29:23.111168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2507","last_scrub_stamp":"2024-03-09T09:29:23.111090+0000","last_deep_scrub":"89'2507","last_deep_scrub_stamp":"2024-03-09T09:29:23.111090+0000","last_clean_scrub_stamp":"2024-03-09T09:29:23.111090+0000","objects_scrubbed":112,"log_size":2507,"log_dups_size":0,"ondisk_log_size":2507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:16:12.677859+0000","scrub_duration":172,"objects_trimmed":0,"snaptrim_duration":0.39778581299999999,"stat_sum":{"num_bytes":1231522,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":112,"num_read_kb":16729,"num_write":1390,"num_write_kb":42674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"89'2182","reported_seq":5128,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.920719+0000","last_change":"2024-03-09T09:27:34.631961+0000","last_active":"2024-03-09T09:29:17.920719+0000","last_peered":"2024-03-09T09:29:17.920719+0000","last_clean":"2024-03-09T09:29:17.920719+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T09:29:17.920719+0000","last_undegraded":"2024-03-09T09:29:17.920719+0000","last_fullsized":"2024-03-09T09:29:17.920719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2182,"log_dups_size":0,"ondisk_log_size":2182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4007626340000003,"stat_sum":{"num_bytes":767001,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":115,"num_read_kb":17556,"num_write":1334,"num_write_kb":40745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"89'2353","reported_seq":5331,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.521386+0000","last_change":"2024-03-09T09:29:38.521419+0000","last_active":"2024-03-09T09:29:38.521386+0000","last_peered":"2024-03-09T09:29:38.521386+0000","last_clean":"2024-03-09T09:29:38.521386+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T09:29:38.521386+0000","last_undegraded":"2024-03-09T09:29:38.521386+0000","last_fullsized":"2024-03-09T09:29:38.521386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2353","last_scrub_stamp":"2024-03-09T09:29:38.521306+0000","last_deep_scrub":"89'2353","last_deep_scrub_stamp":"2024-03-09T09:29:38.521306+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.521306+0000","objects_scrubbed":126,"log_size":2353,"log_dups_size":0,"ondisk_log_size":2353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:24:47.266752+0000","scrub_duration":250,"objects_trimmed":0,"snaptrim_duration":0.80249947799999999,"stat_sum":{"num_bytes":1587591,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":108,"num_read_kb":24972,"num_write":1321,"num_write_kb":50275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"89'2530","reported_seq":5389,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.805348+0000","last_change":"2024-03-09T09:29:43.805397+0000","last_active":"2024-03-09T09:29:43.805348+0000","last_peered":"2024-03-09T09:29:43.805348+0000","last_clean":"2024-03-09T09:29:43.805348+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T09:29:43.805348+0000","last_undegraded":"2024-03-09T09:29:43.805348+0000","last_fullsized":"2024-03-09T09:29:43.805348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2530","last_scrub_stamp":"2024-03-09T09:29:43.805254+0000","last_deep_scrub":"89'2530","last_deep_scrub_stamp":"2024-03-09T09:29:43.805254+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.805254+0000","objects_scrubbed":108,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:53:27.143897+0000","scrub_duration":244,"objects_trimmed":0,"snaptrim_duration":3.6671456120000001,"stat_sum":{"num_bytes":827266,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":108,"num_read_kb":21834,"num_write":1396,"num_write_kb":37656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"89'2300","reported_seq":5300,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.242654+0000","last_change":"2024-03-09T09:29:28.242727+0000","last_active":"2024-03-09T09:29:28.242654+0000","last_peered":"2024-03-09T09:29:28.242654+0000","last_clean":"2024-03-09T09:29:28.242654+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T09:29:28.242654+0000","last_undegraded":"2024-03-09T09:29:28.242654+0000","last_fullsized":"2024-03-09T09:29:28.242654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2300","last_scrub_stamp":"2024-03-09T09:29:28.242562+0000","last_deep_scrub":"89'2300","last_deep_scrub_stamp":"2024-03-09T09:29:28.242562+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.242562+0000","objects_scrubbed":110,"log_size":2300,"log_dups_size":0,"ondisk_log_size":2300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:28:46.353806+0000","scrub_duration":160,"objects_trimmed":0,"snaptrim_duration":1.410735039,"stat_sum":{"num_bytes":1374461,"num_objects":110,"num_object_clones":35,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":35,"num_read":104,"num_read_kb":29127,"num_write":1325,"num_write_kb":54062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"89'2339","reported_seq":5280,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.341765+0000","last_change":"2024-03-09T09:29:35.341840+0000","last_active":"2024-03-09T09:29:35.341765+0000","last_peered":"2024-03-09T09:29:35.341765+0000","last_clean":"2024-03-09T09:29:35.341765+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T09:29:35.341765+0000","last_undegraded":"2024-03-09T09:29:35.341765+0000","last_fullsized":"2024-03-09T09:29:35.341765+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2339","last_scrub_stamp":"2024-03-09T09:29:35.341694+0000","last_deep_scrub":"89'2339","last_deep_scrub_stamp":"2024-03-09T09:29:35.341694+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.341694+0000","objects_scrubbed":117,"log_size":2339,"log_dups_size":0,"ondisk_log_size":2339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:43.563628+0000","scrub_duration":157,"objects_trimmed":4,"snaptrim_duration":4.393423125,"stat_sum":{"num_bytes":646888,"num_objects":117,"num_object_clones":37,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":37,"num_read":111,"num_read_kb":21616,"num_write":1343,"num_write_kb":41561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"89'2510","reported_seq":5497,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.644972+0000","last_change":"2024-03-09T09:29:29.645003+0000","last_active":"2024-03-09T09:29:29.644972+0000","last_peered":"2024-03-09T09:29:29.644972+0000","last_clean":"2024-03-09T09:29:29.644972+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T09:29:29.644972+0000","last_undegraded":"2024-03-09T09:29:29.644972+0000","last_fullsized":"2024-03-09T09:29:29.644972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2510","last_scrub_stamp":"2024-03-09T09:29:29.644909+0000","last_deep_scrub":"89'2510","last_deep_scrub_stamp":"2024-03-09T09:29:29.644909+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.644909+0000","objects_scrubbed":130,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:35:58.503655+0000","scrub_duration":189,"objects_trimmed":0,"snaptrim_duration":2.440348755,"stat_sum":{"num_bytes":1448398,"num_objects":130,"num_object_clones":44,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":44,"num_read":125,"num_read_kb":18804,"num_write":1435,"num_write_kb":32798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"89'2406","reported_seq":5320,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.416202+0000","last_change":"2024-03-09T09:29:46.416254+0000","last_active":"2024-03-09T09:29:46.416202+0000","last_peered":"2024-03-09T09:29:46.416202+0000","last_clean":"2024-03-09T09:29:46.416202+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T09:29:46.416202+0000","last_undegraded":"2024-03-09T09:29:46.416202+0000","last_fullsized":"2024-03-09T09:29:46.416202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2406","last_scrub_stamp":"2024-03-09T09:29:46.416131+0000","last_deep_scrub":"89'2406","last_deep_scrub_stamp":"2024-03-09T09:29:46.416131+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.416131+0000","objects_scrubbed":98,"log_size":2406,"log_dups_size":0,"ondisk_log_size":2406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:14:54.896801+0000","scrub_duration":139,"objects_trimmed":4,"snaptrim_duration":0.64500296700000004,"stat_sum":{"num_bytes":1066603,"num_objects":98,"num_object_clones":32,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":32,"num_read":104,"num_read_kb":18198,"num_write":1399,"num_write_kb":47978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"89'2282","reported_seq":5256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.491786+0000","last_change":"2024-03-09T09:29:30.491850+0000","last_active":"2024-03-09T09:29:30.491786+0000","last_peered":"2024-03-09T09:29:30.491786+0000","last_clean":"2024-03-09T09:29:30.491786+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T09:29:30.491786+0000","last_undegraded":"2024-03-09T09:29:30.491786+0000","last_fullsized":"2024-03-09T09:29:30.491786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2282","last_scrub_stamp":"2024-03-09T09:29:30.491684+0000","last_deep_scrub":"89'2282","last_deep_scrub_stamp":"2024-03-09T09:29:30.491684+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.491684+0000","objects_scrubbed":107,"log_size":2282,"log_dups_size":0,"ondisk_log_size":2282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:52:34.304376+0000","scrub_duration":186,"objects_trimmed":0,"snaptrim_duration":0.62298635300000005,"stat_sum":{"num_bytes":1371558,"num_objects":107,"num_object_clones":34,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":34,"num_read":92,"num_read_kb":30679,"num_write":1263,"num_write_kb":63255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"89'2203","reported_seq":5099,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:24.966871+0000","last_change":"2024-03-09T09:27:34.527780+0000","last_active":"2024-03-09T09:29:24.966871+0000","last_peered":"2024-03-09T09:29:24.966871+0000","last_clean":"2024-03-09T09:29:24.966871+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T09:29:24.966871+0000","last_undegraded":"2024-03-09T09:29:24.966871+0000","last_fullsized":"2024-03-09T09:29:24.966871+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2203,"log_dups_size":0,"ondisk_log_size":2203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.1845584630000001,"stat_sum":{"num_bytes":1760124,"num_objects":108,"num_object_clones":35,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":35,"num_read":107,"num_read_kb":13622,"num_write":1279,"num_write_kb":27255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'2317","reported_seq":5312,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.041719+0000","last_change":"2024-03-09T09:29:34.041794+0000","last_active":"2024-03-09T09:29:34.041719+0000","last_peered":"2024-03-09T09:29:34.041719+0000","last_clean":"2024-03-09T09:29:34.041719+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T09:29:34.041719+0000","last_undegraded":"2024-03-09T09:29:34.041719+0000","last_fullsized":"2024-03-09T09:29:34.041719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2317","last_scrub_stamp":"2024-03-09T09:29:34.041570+0000","last_deep_scrub":"89'2317","last_deep_scrub_stamp":"2024-03-09T09:29:34.041570+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.041570+0000","objects_scrubbed":118,"log_size":2317,"log_dups_size":0,"ondisk_log_size":2317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:43:34.420743+0000","scrub_duration":202,"objects_trimmed":0,"snaptrim_duration":1.2778432559999999,"stat_sum":{"num_bytes":1308027,"num_objects":118,"num_object_clones":42,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":42,"num_read":112,"num_read_kb":14330,"num_write":1310,"num_write_kb":25655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"89'2329","reported_seq":5226,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.885825+0000","last_change":"2024-03-09T09:27:34.655594+0000","last_active":"2024-03-09T09:29:17.885825+0000","last_peered":"2024-03-09T09:29:17.885825+0000","last_clean":"2024-03-09T09:29:17.885825+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T09:29:17.885825+0000","last_undegraded":"2024-03-09T09:29:17.885825+0000","last_fullsized":"2024-03-09T09:29:17.885825+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2329,"log_dups_size":0,"ondisk_log_size":2329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.4171876489999997,"stat_sum":{"num_bytes":1480146,"num_objects":115,"num_object_clones":41,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":41,"num_read":120,"num_read_kb":31406,"num_write":1383,"num_write_kb":52041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"89'2289","reported_seq":5300,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.401148+0000","last_change":"2024-03-09T09:29:46.401267+0000","last_active":"2024-03-09T09:29:46.401148+0000","last_peered":"2024-03-09T09:29:46.401148+0000","last_clean":"2024-03-09T09:29:46.401148+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T09:29:46.401148+0000","last_undegraded":"2024-03-09T09:29:46.401148+0000","last_fullsized":"2024-03-09T09:29:46.401148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2289","last_scrub_stamp":"2024-03-09T09:29:46.400913+0000","last_deep_scrub":"89'2289","last_deep_scrub_stamp":"2024-03-09T09:29:46.400913+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.400913+0000","objects_scrubbed":114,"log_size":2289,"log_dups_size":0,"ondisk_log_size":2289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:18:06.604961+0000","scrub_duration":192,"objects_trimmed":0,"snaptrim_duration":0.72006586699999997,"stat_sum":{"num_bytes":650831,"num_objects":114,"num_object_clones":39,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":39,"num_read":114,"num_read_kb":46474,"num_write":1326,"num_write_kb":53541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"89'2398","reported_seq":5341,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.745167+0000","last_change":"2024-03-09T09:29:25.745227+0000","last_active":"2024-03-09T09:29:25.745167+0000","last_peered":"2024-03-09T09:29:25.745167+0000","last_clean":"2024-03-09T09:29:25.745167+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T09:29:25.745167+0000","last_undegraded":"2024-03-09T09:29:25.745167+0000","last_fullsized":"2024-03-09T09:29:25.745167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2398","last_scrub_stamp":"2024-03-09T09:29:25.745039+0000","last_deep_scrub":"89'2398","last_deep_scrub_stamp":"2024-03-09T09:29:25.745039+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.745039+0000","objects_scrubbed":112,"log_size":2398,"log_dups_size":0,"ondisk_log_size":2398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:21:28.894700+0000","scrub_duration":266,"objects_trimmed":2,"snaptrim_duration":0.64248945300000004,"stat_sum":{"num_bytes":509009,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":145,"num_read_kb":34396,"num_write":1348,"num_write_kb":44845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"89'2411","reported_seq":5370,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.040932+0000","last_change":"2024-03-09T09:29:36.041005+0000","last_active":"2024-03-09T09:29:36.040932+0000","last_peered":"2024-03-09T09:29:36.040932+0000","last_clean":"2024-03-09T09:29:36.040932+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T09:29:36.040932+0000","last_undegraded":"2024-03-09T09:29:36.040932+0000","last_fullsized":"2024-03-09T09:29:36.040932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2411","last_scrub_stamp":"2024-03-09T09:29:36.040797+0000","last_deep_scrub":"89'2411","last_deep_scrub_stamp":"2024-03-09T09:29:36.040797+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.040797+0000","objects_scrubbed":100,"log_size":2411,"log_dups_size":0,"ondisk_log_size":2411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:12:02.963696+0000","scrub_duration":144,"objects_trimmed":0,"snaptrim_duration":1.291579509,"stat_sum":{"num_bytes":917223,"num_objects":100,"num_object_clones":30,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":30,"num_read":95,"num_read_kb":13353,"num_write":1387,"num_write_kb":42086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"89'2321","reported_seq":5212,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.944674+0000","last_change":"2024-03-09T09:27:34.549097+0000","last_active":"2024-03-09T09:29:17.944674+0000","last_peered":"2024-03-09T09:29:17.944674+0000","last_clean":"2024-03-09T09:29:17.944674+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T09:29:17.944674+0000","last_undegraded":"2024-03-09T09:29:17.944674+0000","last_fullsized":"2024-03-09T09:29:17.944674+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2321,"log_dups_size":0,"ondisk_log_size":2321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.2066621079999997,"stat_sum":{"num_bytes":1356079,"num_objects":97,"num_object_clones":28,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":28,"num_read":106,"num_read_kb":29896,"num_write":1386,"num_write_kb":51017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"89'2345","reported_seq":5348,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.220830+0000","last_change":"2024-03-09T09:29:25.220935+0000","last_active":"2024-03-09T09:29:25.220830+0000","last_peered":"2024-03-09T09:29:25.220830+0000","last_clean":"2024-03-09T09:29:25.220830+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T09:29:25.220830+0000","last_undegraded":"2024-03-09T09:29:25.220830+0000","last_fullsized":"2024-03-09T09:29:25.220830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2345","last_scrub_stamp":"2024-03-09T09:29:25.220712+0000","last_deep_scrub":"89'2345","last_deep_scrub_stamp":"2024-03-09T09:29:25.220712+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.220712+0000","objects_scrubbed":121,"log_size":2345,"log_dups_size":0,"ondisk_log_size":2345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:29:53.288594+0000","scrub_duration":262,"objects_trimmed":0,"snaptrim_duration":0.313168905,"stat_sum":{"num_bytes":1092268,"num_objects":121,"num_object_clones":43,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":43,"num_read":123,"num_read_kb":25979,"num_write":1341,"num_write_kb":58516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"89'2398","reported_seq":5310,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.808913+0000","last_change":"2024-03-09T09:29:42.808976+0000","last_active":"2024-03-09T09:29:42.808913+0000","last_peered":"2024-03-09T09:29:42.808913+0000","last_clean":"2024-03-09T09:29:42.808913+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T09:29:42.808913+0000","last_undegraded":"2024-03-09T09:29:42.808913+0000","last_fullsized":"2024-03-09T09:29:42.808913+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2398","last_scrub_stamp":"2024-03-09T09:29:42.808820+0000","last_deep_scrub":"89'2398","last_deep_scrub_stamp":"2024-03-09T09:29:42.808820+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.808820+0000","objects_scrubbed":95,"log_size":2398,"log_dups_size":0,"ondisk_log_size":2398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:58:20.828745+0000","scrub_duration":224,"objects_trimmed":6,"snaptrim_duration":3.9232747450000001,"stat_sum":{"num_bytes":5183706,"num_objects":95,"num_object_clones":27,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":111,"num_read_kb":34166,"num_write":1371,"num_write_kb":49768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"89'2364","reported_seq":5337,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.609331+0000","last_change":"2024-03-09T09:29:30.609381+0000","last_active":"2024-03-09T09:29:30.609331+0000","last_peered":"2024-03-09T09:29:30.609331+0000","last_clean":"2024-03-09T09:29:30.609331+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T09:29:30.609331+0000","last_undegraded":"2024-03-09T09:29:30.609331+0000","last_fullsized":"2024-03-09T09:29:30.609331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2364","last_scrub_stamp":"2024-03-09T09:29:30.609256+0000","last_deep_scrub":"89'2364","last_deep_scrub_stamp":"2024-03-09T09:29:30.609256+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.609256+0000","objects_scrubbed":117,"log_size":2364,"log_dups_size":0,"ondisk_log_size":2364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:52:13.442415+0000","scrub_duration":168,"objects_trimmed":0,"snaptrim_duration":2.1326348209999999,"stat_sum":{"num_bytes":1866711,"num_objects":117,"num_object_clones":40,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":40,"num_read":109,"num_read_kb":17692,"num_write":1315,"num_write_kb":41853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"89'2322","reported_seq":5272,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.834916+0000","last_change":"2024-03-09T09:29:44.835021+0000","last_active":"2024-03-09T09:29:44.834916+0000","last_peered":"2024-03-09T09:29:44.834916+0000","last_clean":"2024-03-09T09:29:44.834916+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T09:29:44.834916+0000","last_undegraded":"2024-03-09T09:29:44.834916+0000","last_fullsized":"2024-03-09T09:29:44.834916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2322","last_scrub_stamp":"2024-03-09T09:29:44.834720+0000","last_deep_scrub":"89'2322","last_deep_scrub_stamp":"2024-03-09T09:29:44.834720+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.834720+0000","objects_scrubbed":123,"log_size":2322,"log_dups_size":0,"ondisk_log_size":2322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:53:24.127536+0000","scrub_duration":316,"objects_trimmed":0,"snaptrim_duration":3.5970959150000001,"stat_sum":{"num_bytes":1348477,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":126,"num_read_kb":8825,"num_write":1341,"num_write_kb":33612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"89'2672","reported_seq":5495,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:14.045276+0000","last_change":"2024-03-09T09:27:13.243644+0000","last_active":"2024-03-09T09:29:14.045276+0000","last_peered":"2024-03-09T09:29:14.045276+0000","last_clean":"2024-03-09T09:29:14.045276+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T09:29:14.045276+0000","last_undegraded":"2024-03-09T09:29:14.045276+0000","last_fullsized":"2024-03-09T09:29:14.045276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:17.240377+0000","last_clean_scrub_stamp":"2024-03-09T08:52:17.240377+0000","objects_scrubbed":0,"log_size":2672,"log_dups_size":0,"ondisk_log_size":2672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.49950316500000003,"stat_sum":{"num_bytes":924300,"num_objects":92,"num_object_clones":25,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":25,"num_read":107,"num_read_kb":17449,"num_write":1480,"num_write_kb":44901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"89'2175","reported_seq":5122,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.521092+0000","last_change":"2024-03-09T09:27:34.679372+0000","last_active":"2024-03-09T09:29:26.521092+0000","last_peered":"2024-03-09T09:29:26.521092+0000","last_clean":"2024-03-09T09:29:26.521092+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T09:29:26.521092+0000","last_undegraded":"2024-03-09T09:29:26.521092+0000","last_fullsized":"2024-03-09T09:29:26.521092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2175,"log_dups_size":0,"ondisk_log_size":2175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4099735320000004,"stat_sum":{"num_bytes":773517,"num_objects":127,"num_object_clones":46,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":46,"num_read":120,"num_read_kb":5950,"num_write":1331,"num_write_kb":13298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"89'2263","reported_seq":5262,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.714031+0000","last_change":"2024-03-09T09:29:31.714031+0000","last_active":"2024-03-09T09:29:31.714031+0000","last_peered":"2024-03-09T09:29:31.714031+0000","last_clean":"2024-03-09T09:29:31.714031+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T09:29:31.714031+0000","last_undegraded":"2024-03-09T09:29:31.714031+0000","last_fullsized":"2024-03-09T09:29:31.714031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2263","last_scrub_stamp":"2024-03-09T09:29:31.713804+0000","last_deep_scrub":"89'2263","last_deep_scrub_stamp":"2024-03-09T09:29:31.713804+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.713804+0000","objects_scrubbed":101,"log_size":2263,"log_dups_size":0,"ondisk_log_size":2263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:38:28.086397+0000","scrub_duration":247,"objects_trimmed":0,"snaptrim_duration":0.89838860799999998,"stat_sum":{"num_bytes":6604987,"num_objects":101,"num_object_clones":30,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":30,"num_read":109,"num_read_kb":14504,"num_write":1281,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"89'2258","reported_seq":5301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.156349+0000","last_change":"2024-03-09T09:29:40.156383+0000","last_active":"2024-03-09T09:29:40.156349+0000","last_peered":"2024-03-09T09:29:40.156349+0000","last_clean":"2024-03-09T09:29:40.156349+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T09:29:40.156349+0000","last_undegraded":"2024-03-09T09:29:40.156349+0000","last_fullsized":"2024-03-09T09:29:40.156349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2258","last_scrub_stamp":"2024-03-09T09:29:40.156262+0000","last_deep_scrub":"89'2258","last_deep_scrub_stamp":"2024-03-09T09:29:40.156262+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.156262+0000","objects_scrubbed":99,"log_size":2258,"log_dups_size":0,"ondisk_log_size":2258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:12:59.205500+0000","scrub_duration":158,"objects_trimmed":0,"snaptrim_duration":0.71096335899999996,"stat_sum":{"num_bytes":637081,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":204,"num_read_kb":24036,"num_write":1319,"num_write_kb":30441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"89'2452","reported_seq":5408,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.725646+0000","last_change":"2024-03-09T09:29:39.725731+0000","last_active":"2024-03-09T09:29:39.725646+0000","last_peered":"2024-03-09T09:29:39.725646+0000","last_clean":"2024-03-09T09:29:39.725646+0000","last_became_active":"2024-03-09T08:53:00.240319+0000","last_became_peered":"2024-03-09T08:53:00.240319+0000","last_unstale":"2024-03-09T09:29:39.725646+0000","last_undegraded":"2024-03-09T09:29:39.725646+0000","last_fullsized":"2024-03-09T09:29:39.725646+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2452","last_scrub_stamp":"2024-03-09T09:29:39.725567+0000","last_deep_scrub":"89'2452","last_deep_scrub_stamp":"2024-03-09T09:29:39.725567+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.725567+0000","objects_scrubbed":116,"log_size":2452,"log_dups_size":0,"ondisk_log_size":2452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:51:10.590974+0000","scrub_duration":202,"objects_trimmed":0,"snaptrim_duration":0.86880120699999996,"stat_sum":{"num_bytes":1614659,"num_objects":116,"num_object_clones":42,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":42,"num_read":120,"num_read_kb":18915,"num_write":1382,"num_write_kb":39172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"89'2231","reported_seq":5192,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.141554+0000","last_change":"2024-03-09T09:29:46.141590+0000","last_active":"2024-03-09T09:29:46.141554+0000","last_peered":"2024-03-09T09:29:46.141554+0000","last_clean":"2024-03-09T09:29:46.141554+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T09:29:46.141554+0000","last_undegraded":"2024-03-09T09:29:46.141554+0000","last_fullsized":"2024-03-09T09:29:46.141554+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2231","last_scrub_stamp":"2024-03-09T09:29:46.141469+0000","last_deep_scrub":"89'2231","last_deep_scrub_stamp":"2024-03-09T09:29:46.141469+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.141469+0000","objects_scrubbed":102,"log_size":2231,"log_dups_size":0,"ondisk_log_size":2231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:30:37.575836+0000","scrub_duration":191,"objects_trimmed":0,"snaptrim_duration":1.7177660380000002,"stat_sum":{"num_bytes":924110,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":129,"num_read_kb":10365,"num_write":1260,"num_write_kb":33897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"89'2330","reported_seq":5321,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.101745+0000","last_change":"2024-03-09T09:29:45.101779+0000","last_active":"2024-03-09T09:29:45.101745+0000","last_peered":"2024-03-09T09:29:45.101745+0000","last_clean":"2024-03-09T09:29:45.101745+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T09:29:45.101745+0000","last_undegraded":"2024-03-09T09:29:45.101745+0000","last_fullsized":"2024-03-09T09:29:45.101745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2330","last_scrub_stamp":"2024-03-09T09:29:45.101686+0000","last_deep_scrub":"89'2330","last_deep_scrub_stamp":"2024-03-09T09:29:45.101686+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.101686+0000","objects_scrubbed":103,"log_size":2330,"log_dups_size":0,"ondisk_log_size":2330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:59:07.019534+0000","scrub_duration":190,"objects_trimmed":0,"snaptrim_duration":1.2105350770000001,"stat_sum":{"num_bytes":1326786,"num_objects":103,"num_object_clones":34,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":34,"num_read":111,"num_read_kb":18735,"num_write":1322,"num_write_kb":41042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"89'2454","reported_seq":5341,"reported_epoch":89,"state":"active+clean+scrubbing+deep","last_fresh":"2024-03-09T09:29:47.720949+0000","last_change":"2024-03-09T09:29:47.496651+0000","last_active":"2024-03-09T09:29:47.720949+0000","last_peered":"2024-03-09T09:29:47.720949+0000","last_clean":"2024-03-09T09:29:47.720949+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T09:29:47.720949+0000","last_undegraded":"2024-03-09T09:29:47.720949+0000","last_fullsized":"2024-03-09T09:29:47.720949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":98,"log_size":2454,"log_dups_size":0,"ondisk_log_size":2454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.0059579840000001,"stat_sum":{"num_bytes":936699,"num_objects":111,"num_object_clones":39,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":39,"num_read":129,"num_read_kb":34540,"num_write":1403,"num_write_kb":57337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"89'2378","reported_seq":5363,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.146738+0000","last_change":"2024-03-09T09:29:32.146738+0000","last_active":"2024-03-09T09:29:32.146738+0000","last_peered":"2024-03-09T09:29:32.146738+0000","last_clean":"2024-03-09T09:29:32.146738+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T09:29:32.146738+0000","last_undegraded":"2024-03-09T09:29:32.146738+0000","last_fullsized":"2024-03-09T09:29:32.146738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2378","last_scrub_stamp":"2024-03-09T09:29:32.146551+0000","last_deep_scrub":"89'2378","last_deep_scrub_stamp":"2024-03-09T09:29:32.146551+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.146551+0000","objects_scrubbed":101,"log_size":2378,"log_dups_size":0,"ondisk_log_size":2378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:49:53.410721+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0.91582108900000003,"stat_sum":{"num_bytes":1759504,"num_objects":101,"num_object_clones":31,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":31,"num_read":108,"num_read_kb":5845,"num_write":1367,"num_write_kb":19152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"89'2471","reported_seq":5368,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.085873+0000","last_change":"2024-03-09T09:29:35.085955+0000","last_active":"2024-03-09T09:29:35.085873+0000","last_peered":"2024-03-09T09:29:35.085873+0000","last_clean":"2024-03-09T09:29:35.085873+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T09:29:35.085873+0000","last_undegraded":"2024-03-09T09:29:35.085873+0000","last_fullsized":"2024-03-09T09:29:35.085873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2471","last_scrub_stamp":"2024-03-09T09:29:35.085785+0000","last_deep_scrub":"89'2471","last_deep_scrub_stamp":"2024-03-09T09:29:35.085785+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.085785+0000","objects_scrubbed":122,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:59:32.359356+0000","scrub_duration":207,"objects_trimmed":6,"snaptrim_duration":4.4857478520000003,"stat_sum":{"num_bytes":2540930,"num_objects":122,"num_object_clones":40,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":40,"num_read":127,"num_read_kb":51469,"num_write":1352,"num_write_kb":80787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"89'2314","reported_seq":5303,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.153266+0000","last_change":"2024-03-09T09:29:34.153266+0000","last_active":"2024-03-09T09:29:34.153266+0000","last_peered":"2024-03-09T09:29:34.153266+0000","last_clean":"2024-03-09T09:29:34.153266+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T09:29:34.153266+0000","last_undegraded":"2024-03-09T09:29:34.153266+0000","last_fullsized":"2024-03-09T09:29:34.153266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2314","last_scrub_stamp":"2024-03-09T09:29:34.153148+0000","last_deep_scrub":"89'2314","last_deep_scrub_stamp":"2024-03-09T09:29:34.153148+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.153148+0000","objects_scrubbed":96,"log_size":2314,"log_dups_size":0,"ondisk_log_size":2314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:05:43.864141+0000","scrub_duration":133,"objects_trimmed":0,"snaptrim_duration":3.1939797309999998,"stat_sum":{"num_bytes":1585107,"num_objects":96,"num_object_clones":31,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":31,"num_read":96,"num_read_kb":18307,"num_write":1352,"num_write_kb":50489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"89'2682","reported_seq":5435,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.949871+0000","last_change":"2024-03-09T09:27:34.659119+0000","last_active":"2024-03-09T09:29:17.949871+0000","last_peered":"2024-03-09T09:29:17.949871+0000","last_clean":"2024-03-09T09:29:17.949871+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T09:29:17.949871+0000","last_undegraded":"2024-03-09T09:29:17.949871+0000","last_fullsized":"2024-03-09T09:29:17.949871+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2682,"log_dups_size":0,"ondisk_log_size":2682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4197025170000002,"stat_sum":{"num_bytes":1559225,"num_objects":122,"num_object_clones":42,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":42,"num_read":122,"num_read_kb":6495,"num_write":1460,"num_write_kb":38738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"89'2342","reported_seq":5327,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.551463+0000","last_change":"2024-03-09T09:29:34.551559+0000","last_active":"2024-03-09T09:29:34.551463+0000","last_peered":"2024-03-09T09:29:34.551463+0000","last_clean":"2024-03-09T09:29:34.551463+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T09:29:34.551463+0000","last_undegraded":"2024-03-09T09:29:34.551463+0000","last_fullsized":"2024-03-09T09:29:34.551463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2342","last_scrub_stamp":"2024-03-09T09:29:34.551354+0000","last_deep_scrub":"89'2342","last_deep_scrub_stamp":"2024-03-09T09:29:34.551354+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.551354+0000","objects_scrubbed":117,"log_size":2342,"log_dups_size":0,"ondisk_log_size":2342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:08:23.847477+0000","scrub_duration":251,"objects_trimmed":0,"snaptrim_duration":0.94133736700000004,"stat_sum":{"num_bytes":314683,"num_objects":117,"num_object_clones":39,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":39,"num_read":116,"num_read_kb":13620,"num_write":1329,"num_write_kb":42753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"89'2198","reported_seq":5209,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.950842+0000","last_change":"2024-03-09T09:27:34.517246+0000","last_active":"2024-03-09T09:29:17.950842+0000","last_peered":"2024-03-09T09:29:17.950842+0000","last_clean":"2024-03-09T09:29:17.950842+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T09:29:17.950842+0000","last_undegraded":"2024-03-09T09:29:17.950842+0000","last_fullsized":"2024-03-09T09:29:17.950842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2198,"log_dups_size":0,"ondisk_log_size":2198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.1785381859999999,"stat_sum":{"num_bytes":1532500,"num_objects":95,"num_object_clones":30,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":30,"num_read":195,"num_read_kb":18044,"num_write":1327,"num_write_kb":39414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"89'2396","reported_seq":5379,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.852175+0000","last_change":"2024-03-09T09:29:36.852219+0000","last_active":"2024-03-09T09:29:36.852175+0000","last_peered":"2024-03-09T09:29:36.852175+0000","last_clean":"2024-03-09T09:29:36.852175+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T09:29:36.852175+0000","last_undegraded":"2024-03-09T09:29:36.852175+0000","last_fullsized":"2024-03-09T09:29:36.852175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2396","last_scrub_stamp":"2024-03-09T09:29:36.852066+0000","last_deep_scrub":"89'2396","last_deep_scrub_stamp":"2024-03-09T09:29:36.852066+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.852066+0000","objects_scrubbed":130,"log_size":2396,"log_dups_size":0,"ondisk_log_size":2396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:11:10.627586+0000","scrub_duration":176,"objects_trimmed":0,"snaptrim_duration":3.298540461,"stat_sum":{"num_bytes":2039183,"num_objects":130,"num_object_clones":43,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":43,"num_read":126,"num_read_kb":25377,"num_write":1340,"num_write_kb":50110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"89'2329","reported_seq":5261,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.731308+0000","last_change":"2024-03-09T09:29:36.731381+0000","last_active":"2024-03-09T09:29:36.731308+0000","last_peered":"2024-03-09T09:29:36.731308+0000","last_clean":"2024-03-09T09:29:36.731308+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T09:29:36.731308+0000","last_undegraded":"2024-03-09T09:29:36.731308+0000","last_fullsized":"2024-03-09T09:29:36.731308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2329","last_scrub_stamp":"2024-03-09T09:29:36.731193+0000","last_deep_scrub":"89'2329","last_deep_scrub_stamp":"2024-03-09T09:29:36.731193+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.731193+0000","objects_scrubbed":120,"log_size":2329,"log_dups_size":0,"ondisk_log_size":2329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:17:35.549238+0000","scrub_duration":183,"objects_trimmed":0,"snaptrim_duration":0.62967200199999995,"stat_sum":{"num_bytes":347019,"num_objects":120,"num_object_clones":41,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":41,"num_read":119,"num_read_kb":17844,"num_write":1302,"num_write_kb":51829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"89'2770","reported_seq":5621,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.538685+0000","last_change":"2024-03-09T09:29:36.538725+0000","last_active":"2024-03-09T09:29:36.538685+0000","last_peered":"2024-03-09T09:29:36.538685+0000","last_clean":"2024-03-09T09:29:36.538685+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T09:29:36.538685+0000","last_undegraded":"2024-03-09T09:29:36.538685+0000","last_fullsized":"2024-03-09T09:29:36.538685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2770","last_scrub_stamp":"2024-03-09T09:29:36.538615+0000","last_deep_scrub":"89'2770","last_deep_scrub_stamp":"2024-03-09T09:29:36.538615+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.538615+0000","objects_scrubbed":112,"log_size":2770,"log_dups_size":0,"ondisk_log_size":2770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:57:17.020890+0000","scrub_duration":157,"objects_trimmed":0,"snaptrim_duration":2.0660642299999998,"stat_sum":{"num_bytes":1365000,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":127,"num_read_kb":33469,"num_write":1475,"num_write_kb":57156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"89'2231","reported_seq":5158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.943634+0000","last_change":"2024-03-09T09:27:34.649642+0000","last_active":"2024-03-09T09:29:17.943634+0000","last_peered":"2024-03-09T09:29:17.943634+0000","last_clean":"2024-03-09T09:29:17.943634+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T09:29:17.943634+0000","last_undegraded":"2024-03-09T09:29:17.943634+0000","last_fullsized":"2024-03-09T09:29:17.943634+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2231,"log_dups_size":0,"ondisk_log_size":2231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4162208879999998,"stat_sum":{"num_bytes":1481141,"num_objects":109,"num_object_clones":37,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":37,"num_read":120,"num_read_kb":25693,"num_write":1333,"num_write_kb":50297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'2302","reported_seq":5294,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.522536+0000","last_change":"2024-03-09T09:29:38.522573+0000","last_active":"2024-03-09T09:29:38.522536+0000","last_peered":"2024-03-09T09:29:38.522536+0000","last_clean":"2024-03-09T09:29:38.522536+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T09:29:38.522536+0000","last_undegraded":"2024-03-09T09:29:38.522536+0000","last_fullsized":"2024-03-09T09:29:38.522536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2302","last_scrub_stamp":"2024-03-09T09:29:38.522458+0000","last_deep_scrub":"89'2302","last_deep_scrub_stamp":"2024-03-09T09:29:38.522458+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.522458+0000","objects_scrubbed":110,"log_size":2302,"log_dups_size":0,"ondisk_log_size":2302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:09:19.242940+0000","scrub_duration":165,"objects_trimmed":0,"snaptrim_duration":2.379810049,"stat_sum":{"num_bytes":472147,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":115,"num_read_kb":25809,"num_write":1277,"num_write_kb":34061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"89'2310","reported_seq":5233,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.924173+0000","last_change":"2024-03-09T09:27:34.694598+0000","last_active":"2024-03-09T09:29:17.924173+0000","last_peered":"2024-03-09T09:29:17.924173+0000","last_clean":"2024-03-09T09:29:17.924173+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T09:29:17.924173+0000","last_undegraded":"2024-03-09T09:29:17.924173+0000","last_fullsized":"2024-03-09T09:29:17.924173+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2310,"log_dups_size":0,"ondisk_log_size":2310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":4.4238349550000002,"stat_sum":{"num_bytes":1560364,"num_objects":99,"num_object_clones":31,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":31,"num_read":120,"num_read_kb":23106,"num_write":1406,"num_write_kb":36148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"89'2461","reported_seq":5435,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.333891+0000","last_change":"2024-03-09T09:29:33.333923+0000","last_active":"2024-03-09T09:29:33.333891+0000","last_peered":"2024-03-09T09:29:33.333891+0000","last_clean":"2024-03-09T09:29:33.333891+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T09:29:33.333891+0000","last_undegraded":"2024-03-09T09:29:33.333891+0000","last_fullsized":"2024-03-09T09:29:33.333891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2461","last_scrub_stamp":"2024-03-09T09:29:33.333818+0000","last_deep_scrub":"89'2461","last_deep_scrub_stamp":"2024-03-09T09:29:33.333818+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.333818+0000","objects_scrubbed":106,"log_size":2461,"log_dups_size":0,"ondisk_log_size":2461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:06:25.505859+0000","scrub_duration":154,"objects_trimmed":0,"snaptrim_duration":0.73072901000000001,"stat_sum":{"num_bytes":1439516,"num_objects":106,"num_object_clones":37,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":37,"num_read":117,"num_read_kb":43194,"num_write":1390,"num_write_kb":63251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"89'2203","reported_seq":5138,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.436754+0000","last_change":"2024-03-09T09:29:37.436790+0000","last_active":"2024-03-09T09:29:37.436754+0000","last_peered":"2024-03-09T09:29:37.436754+0000","last_clean":"2024-03-09T09:29:37.436754+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T09:29:37.436754+0000","last_undegraded":"2024-03-09T09:29:37.436754+0000","last_fullsized":"2024-03-09T09:29:37.436754+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2203","last_scrub_stamp":"2024-03-09T09:29:37.436692+0000","last_deep_scrub":"89'2203","last_deep_scrub_stamp":"2024-03-09T09:29:37.436692+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.436692+0000","objects_scrubbed":91,"log_size":2203,"log_dups_size":0,"ondisk_log_size":2203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:51:14.275290+0000","scrub_duration":154,"objects_trimmed":4,"snaptrim_duration":4.2163343510000004,"stat_sum":{"num_bytes":526581,"num_objects":91,"num_object_clones":29,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":29,"num_read":100,"num_read_kb":25429,"num_write":1233,"num_write_kb":64615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"89'2261","reported_seq":5249,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.400325+0000","last_change":"2024-03-09T09:29:36.400325+0000","last_active":"2024-03-09T09:29:36.400325+0000","last_peered":"2024-03-09T09:29:36.400325+0000","last_clean":"2024-03-09T09:29:36.400325+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T09:29:36.400325+0000","last_undegraded":"2024-03-09T09:29:36.400325+0000","last_fullsized":"2024-03-09T09:29:36.400325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2261","last_scrub_stamp":"2024-03-09T09:29:36.400199+0000","last_deep_scrub":"89'2261","last_deep_scrub_stamp":"2024-03-09T09:29:36.400199+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.400199+0000","objects_scrubbed":102,"log_size":2261,"log_dups_size":0,"ondisk_log_size":2261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:15:16.762722+0000","scrub_duration":152,"objects_trimmed":0,"snaptrim_duration":0.46979842799999999,"stat_sum":{"num_bytes":631595,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":98,"num_read_kb":25968,"num_write":1286,"num_write_kb":41427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"89'2240","reported_seq":5192,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.433571+0000","last_change":"2024-03-09T09:29:43.433616+0000","last_active":"2024-03-09T09:29:43.433571+0000","last_peered":"2024-03-09T09:29:43.433571+0000","last_clean":"2024-03-09T09:29:43.433571+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T09:29:43.433571+0000","last_undegraded":"2024-03-09T09:29:43.433571+0000","last_fullsized":"2024-03-09T09:29:43.433571+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2240","last_scrub_stamp":"2024-03-09T09:29:43.433506+0000","last_deep_scrub":"89'2240","last_deep_scrub_stamp":"2024-03-09T09:29:43.433506+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.433506+0000","objects_scrubbed":103,"log_size":2240,"log_dups_size":0,"ondisk_log_size":2240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:43:00.142548+0000","scrub_duration":145,"objects_trimmed":6,"snaptrim_duration":0.66506321800000001,"stat_sum":{"num_bytes":592790,"num_objects":103,"num_object_clones":35,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":35,"num_read":105,"num_read_kb":21771,"num_write":1301,"num_write_kb":41126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"89'2285","reported_seq":5253,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.343829+0000","last_change":"2024-03-09T09:29:30.343867+0000","last_active":"2024-03-09T09:29:30.343829+0000","last_peered":"2024-03-09T09:29:30.343829+0000","last_clean":"2024-03-09T09:29:30.343829+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T09:29:30.343829+0000","last_undegraded":"2024-03-09T09:29:30.343829+0000","last_fullsized":"2024-03-09T09:29:30.343829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2285","last_scrub_stamp":"2024-03-09T09:29:30.343749+0000","last_deep_scrub":"89'2285","last_deep_scrub_stamp":"2024-03-09T09:29:30.343749+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.343749+0000","objects_scrubbed":81,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:50:19.628014+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0.638707739,"stat_sum":{"num_bytes":1008318,"num_objects":81,"num_object_clones":24,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":24,"num_read":86,"num_read_kb":17604,"num_write":1317,"num_write_kb":38401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"89'2404","reported_seq":5308,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:47.125300+0000","last_change":"2024-03-09T09:29:47.125335+0000","last_active":"2024-03-09T09:29:47.125300+0000","last_peered":"2024-03-09T09:29:47.125300+0000","last_clean":"2024-03-09T09:29:47.125300+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T09:29:47.125300+0000","last_undegraded":"2024-03-09T09:29:47.125300+0000","last_fullsized":"2024-03-09T09:29:47.125300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2404","last_scrub_stamp":"2024-03-09T09:29:47.125222+0000","last_deep_scrub":"89'2404","last_deep_scrub_stamp":"2024-03-09T09:29:47.125222+0000","last_clean_scrub_stamp":"2024-03-09T09:29:47.125222+0000","objects_scrubbed":106,"log_size":2404,"log_dups_size":0,"ondisk_log_size":2404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:13:06.705903+0000","scrub_duration":210,"objects_trimmed":4,"snaptrim_duration":1.705573912,"stat_sum":{"num_bytes":534756,"num_objects":106,"num_object_clones":34,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":34,"num_read":108,"num_read_kb":25061,"num_write":1338,"num_write_kb":44408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"89'2229","reported_seq":5245,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.024309+0000","last_change":"2024-03-09T09:29:38.024384+0000","last_active":"2024-03-09T09:29:38.024309+0000","last_peered":"2024-03-09T09:29:38.024309+0000","last_clean":"2024-03-09T09:29:38.024309+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T09:29:38.024309+0000","last_undegraded":"2024-03-09T09:29:38.024309+0000","last_fullsized":"2024-03-09T09:29:38.024309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2229","last_scrub_stamp":"2024-03-09T09:29:38.024195+0000","last_deep_scrub":"89'2229","last_deep_scrub_stamp":"2024-03-09T09:29:38.024195+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.024195+0000","objects_scrubbed":118,"log_size":2229,"log_dups_size":0,"ondisk_log_size":2229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:50:01.538268+0000","scrub_duration":205,"objects_trimmed":0,"snaptrim_duration":1.1420017650000001,"stat_sum":{"num_bytes":1651219,"num_objects":118,"num_object_clones":44,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":44,"num_read":114,"num_read_kb":27021,"num_write":1266,"num_write_kb":36457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"89'2279","reported_seq":5204,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.928036+0000","last_change":"2024-03-09T09:27:34.669037+0000","last_active":"2024-03-09T09:29:17.928036+0000","last_peered":"2024-03-09T09:29:17.928036+0000","last_clean":"2024-03-09T09:29:17.928036+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T09:29:17.928036+0000","last_undegraded":"2024-03-09T09:29:17.928036+0000","last_fullsized":"2024-03-09T09:29:17.928036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2279,"log_dups_size":0,"ondisk_log_size":2279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":4.3954307559999997,"stat_sum":{"num_bytes":1015705,"num_objects":121,"num_object_clones":38,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":38,"num_read":132,"num_read_kb":25133,"num_write":1351,"num_write_kb":59371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"89'2478","reported_seq":5444,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.377809+0000","last_change":"2024-03-09T09:29:40.377903+0000","last_active":"2024-03-09T09:29:40.377809+0000","last_peered":"2024-03-09T09:29:40.377809+0000","last_clean":"2024-03-09T09:29:40.377809+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T09:29:40.377809+0000","last_undegraded":"2024-03-09T09:29:40.377809+0000","last_fullsized":"2024-03-09T09:29:40.377809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2478","last_scrub_stamp":"2024-03-09T09:29:40.377714+0000","last_deep_scrub":"89'2478","last_deep_scrub_stamp":"2024-03-09T09:29:40.377714+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.377714+0000","objects_scrubbed":117,"log_size":2478,"log_dups_size":0,"ondisk_log_size":2478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:37:06.154933+0000","scrub_duration":204,"objects_trimmed":0,"snaptrim_duration":0.51876684699999998,"stat_sum":{"num_bytes":1407157,"num_objects":117,"num_object_clones":38,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":38,"num_read":113,"num_read_kb":14121,"num_write":1423,"num_write_kb":30929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"89'2261","reported_seq":5204,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.500952+0000","last_change":"2024-03-09T09:29:35.501013+0000","last_active":"2024-03-09T09:29:35.500952+0000","last_peered":"2024-03-09T09:29:35.500952+0000","last_clean":"2024-03-09T09:29:35.500952+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T09:29:35.500952+0000","last_undegraded":"2024-03-09T09:29:35.500952+0000","last_fullsized":"2024-03-09T09:29:35.500952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2261","last_scrub_stamp":"2024-03-09T09:29:35.500858+0000","last_deep_scrub":"89'2261","last_deep_scrub_stamp":"2024-03-09T09:29:35.500858+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.500858+0000","objects_scrubbed":116,"log_size":2261,"log_dups_size":0,"ondisk_log_size":2261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:46:12.193654+0000","scrub_duration":204,"objects_trimmed":2,"snaptrim_duration":4.1683436250000003,"stat_sum":{"num_bytes":2080769,"num_objects":116,"num_object_clones":43,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":43,"num_read":114,"num_read_kb":27228,"num_write":1275,"num_write_kb":54719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"89'2329","reported_seq":5313,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.999608+0000","last_change":"2024-03-09T09:29:29.999663+0000","last_active":"2024-03-09T09:29:29.999608+0000","last_peered":"2024-03-09T09:29:29.999608+0000","last_clean":"2024-03-09T09:29:29.999608+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T09:29:29.999608+0000","last_undegraded":"2024-03-09T09:29:29.999608+0000","last_fullsized":"2024-03-09T09:29:29.999608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2329","last_scrub_stamp":"2024-03-09T09:29:29.999459+0000","last_deep_scrub":"89'2329","last_deep_scrub_stamp":"2024-03-09T09:29:29.999459+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.999459+0000","objects_scrubbed":102,"log_size":2329,"log_dups_size":0,"ondisk_log_size":2329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:41:58.957669+0000","scrub_duration":225,"objects_trimmed":0,"snaptrim_duration":1.4282657350000001,"stat_sum":{"num_bytes":1076453,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":106,"num_read_kb":19463,"num_write":1318,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"89'2532","reported_seq":5422,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.773711+0000","last_change":"2024-03-09T09:29:38.773752+0000","last_active":"2024-03-09T09:29:38.773711+0000","last_peered":"2024-03-09T09:29:38.773711+0000","last_clean":"2024-03-09T09:29:38.773711+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T09:29:38.773711+0000","last_undegraded":"2024-03-09T09:29:38.773711+0000","last_fullsized":"2024-03-09T09:29:38.773711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2532","last_scrub_stamp":"2024-03-09T09:29:38.773637+0000","last_deep_scrub":"89'2532","last_deep_scrub_stamp":"2024-03-09T09:29:38.773637+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.773637+0000","objects_scrubbed":116,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:29:01.867872+0000","scrub_duration":204,"objects_trimmed":2,"snaptrim_duration":0.63967549000000001,"stat_sum":{"num_bytes":3580963,"num_objects":116,"num_object_clones":37,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":37,"num_read":116,"num_read_kb":32310,"num_write":1440,"num_write_kb":66236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"89'2363","reported_seq":5356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.164477+0000","last_change":"2024-03-09T09:29:31.164540+0000","last_active":"2024-03-09T09:29:31.164477+0000","last_peered":"2024-03-09T09:29:31.164477+0000","last_clean":"2024-03-09T09:29:31.164477+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T09:29:31.164477+0000","last_undegraded":"2024-03-09T09:29:31.164477+0000","last_fullsized":"2024-03-09T09:29:31.164477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2363","last_scrub_stamp":"2024-03-09T09:29:31.164401+0000","last_deep_scrub":"89'2363","last_deep_scrub_stamp":"2024-03-09T09:29:31.164401+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.164401+0000","objects_scrubbed":112,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:04:08.484425+0000","scrub_duration":162,"objects_trimmed":0,"snaptrim_duration":0.47268980999999999,"stat_sum":{"num_bytes":1488567,"num_objects":112,"num_object_clones":34,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":34,"num_read":108,"num_read_kb":30029,"num_write":1350,"num_write_kb":42424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"89'2142","reported_seq":5073,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.933706+0000","last_change":"2024-03-09T09:27:34.605106+0000","last_active":"2024-03-09T09:29:17.933706+0000","last_peered":"2024-03-09T09:29:17.933706+0000","last_clean":"2024-03-09T09:29:17.933706+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T09:29:17.933706+0000","last_undegraded":"2024-03-09T09:29:17.933706+0000","last_fullsized":"2024-03-09T09:29:17.933706+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2142,"log_dups_size":0,"ondisk_log_size":2142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":4.3763956530000003,"stat_sum":{"num_bytes":1440979,"num_objects":98,"num_object_clones":30,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":30,"num_read":104,"num_read_kb":22606,"num_write":1302,"num_write_kb":46651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"89'2151","reported_seq":5155,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.982514+0000","last_change":"2024-03-09T09:29:38.982552+0000","last_active":"2024-03-09T09:29:38.982514+0000","last_peered":"2024-03-09T09:29:38.982514+0000","last_clean":"2024-03-09T09:29:38.982514+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T09:29:38.982514+0000","last_undegraded":"2024-03-09T09:29:38.982514+0000","last_fullsized":"2024-03-09T09:29:38.982514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2151","last_scrub_stamp":"2024-03-09T09:29:38.982431+0000","last_deep_scrub":"89'2151","last_deep_scrub_stamp":"2024-03-09T09:29:38.982431+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.982431+0000","objects_scrubbed":85,"log_size":2151,"log_dups_size":0,"ondisk_log_size":2151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:13:17.068773+0000","scrub_duration":127,"objects_trimmed":0,"snaptrim_duration":1.344208157,"stat_sum":{"num_bytes":485403,"num_objects":85,"num_object_clones":27,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":27,"num_read":96,"num_read_kb":5635,"num_write":1204,"num_write_kb":28568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"89'2216","reported_seq":5129,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:17.931730+0000","last_change":"2024-03-09T09:27:34.669203+0000","last_active":"2024-03-09T09:29:17.931730+0000","last_peered":"2024-03-09T09:29:17.931730+0000","last_clean":"2024-03-09T09:29:17.931730+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T09:29:17.931730+0000","last_undegraded":"2024-03-09T09:29:17.931730+0000","last_fullsized":"2024-03-09T09:29:17.931730+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-09T08:52:28.539137+0000","last_clean_scrub_stamp":"2024-03-09T08:52:28.539137+0000","objects_scrubbed":0,"log_size":2216,"log_dups_size":0,"ondisk_log_size":2216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":4.4378791440000001,"stat_sum":{"num_bytes":906976,"num_objects":87,"num_object_clones":27,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":27,"num_read":110,"num_read_kb":26703,"num_write":1307,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"89'2242","reported_seq":5250,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.337776+0000","last_change":"2024-03-09T09:29:32.337831+0000","last_active":"2024-03-09T09:29:32.337776+0000","last_peered":"2024-03-09T09:29:32.337776+0000","last_clean":"2024-03-09T09:29:32.337776+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T09:29:32.337776+0000","last_undegraded":"2024-03-09T09:29:32.337776+0000","last_fullsized":"2024-03-09T09:29:32.337776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2242","last_scrub_stamp":"2024-03-09T09:29:32.337704+0000","last_deep_scrub":"89'2242","last_deep_scrub_stamp":"2024-03-09T09:29:32.337704+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.337704+0000","objects_scrubbed":97,"log_size":2242,"log_dups_size":0,"ondisk_log_size":2242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:41:31.317870+0000","scrub_duration":144,"objects_trimmed":0,"snaptrim_duration":1.0826407490000001,"stat_sum":{"num_bytes":992571,"num_objects":97,"num_object_clones":32,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":32,"num_read":100,"num_read_kb":32945,"num_write":1310,"num_write_kb":54952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7626,"num_read_kb":1526336,"num_write":85965,"num_write_kb":2933421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149164,"ondisk_log_size":149164,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":877528733,"num_objects":1507,"num_object_clones":0,"num_object_copies":3014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1507,"num_whiteouts":0,"num_read":2924,"num_read_kb":5807,"num_write":25888,"num_write_kb":3145194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1225,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17386594,"num_omap_keys":27558,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1755119616,"data_stored":1755057466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23677,"ondisk_log_size":23677,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7230,"num_read_kb":1519173,"num_write":85971,"num_write_kb":2933405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151307,"ondisk_log_size":151307,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798940451,"num_objects":1475,"num_object_clones":0,"num_object_copies":2950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1475,"num_whiteouts":0,"num_read":2854,"num_read_kb":5749,"num_write":25875,"num_write_kb":3105441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1213,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17656376,"num_omap_keys":27516,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597939712,"data_stored":1597880902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23637,"ondisk_log_size":23637,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575304,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":848416,"kb_used_data":502168,"kb_used_omap":1,"kb_used_meta":346238,"kb_avail":93523424,"statfs":{"total":96636764160,"available":95767986176,"internally_reserved":0,"allocated":514220032,"data_stored":507443895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":354548170},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575305,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712220,"kb_used_data":456276,"kb_used_omap":1,"kb_used_meta":255934,"kb_avail":93659620,"statfs":{"total":96636764160,"available":95907450880,"internally_reserved":0,"allocated":467226624,"data_stored":462408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":262076870},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575303,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":810520,"kb_used_data":454544,"kb_used_omap":1,"kb_used_meta":355966,"kb_avail":93561320,"statfs":{"total":96636764160,"available":95806791680,"internally_reserved":0,"allocated":465453056,"data_stored":458259869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":364509637},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575304,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":853316,"kb_used_data":551484,"kb_used_omap":1,"kb_used_meta":301822,"kb_avail":93518524,"statfs":{"total":96636764160,"available":95762968576,"internally_reserved":0,"allocated":564719616,"data_stored":557367936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309066181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575305,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":940952,"kb_used_data":544784,"kb_used_omap":1,"kb_used_meta":396158,"kb_avail":93430888,"statfs":{"total":96636764160,"available":95673229312,"internally_reserved":0,"allocated":557858816,"data_stored":550354976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":405666247},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575304,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462952,"kb_used_data":225056,"kb_used_omap":1,"kb_used_meta":237886,"kb_avail":93908888,"statfs":{"total":96636764160,"available":96162701312,"internally_reserved":0,"allocated":230457344,"data_stored":225591238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":243595717},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575304,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":964752,"kb_used_data":613000,"kb_used_omap":1,"kb_used_meta":351742,"kb_avail":93407088,"statfs":{"total":96636764160,"available":95648858112,"internally_reserved":0,"allocated":627712000,"data_stored":621647944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":360184261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575306,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672440,"kb_used_data":380400,"kb_used_omap":1,"kb_used_meta":292030,"kb_avail":93699400,"statfs":{"total":96636764160,"available":95948185600,"internally_reserved":0,"allocated":389529600,"data_stored":383207215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":299039175},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":226504704,"data_stored":226493054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":218112000,"data_stored":218104019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92274688,"data_stored":92274688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":173154304,"data_stored":173146197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":201338880,"data_stored":201326769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":197980160,"data_stored":197974487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":223150080,"data_stored":223140893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":265424896,"data_stored":265420795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37781504,"data_stored":34251073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20246528,"data_stored":18282047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22822912,"data_stored":20809621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27308032,"data_stored":23727598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37437440,"data_stored":33951899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":27983872,"data_stored":24565897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23818240,"data_stored":21628413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":29773824,"data_stored":26493862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":100741120,"data_stored":100738977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":360800256,"data_stored":360786519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96473088,"data_stored":96469310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":315428864,"data_stored":315415292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":289415168,"data_stored":289407328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":201334784,"data_stored":201327264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":201334784,"data_stored":201327208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":189591552,"data_stored":189585568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22835200,"data_stored":20540995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27480064,"data_stored":23882611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17813504,"data_stored":15444871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40894464,"data_stored":37473141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35454976,"data_stored":32089192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":37081088,"data_stored":33799473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17850368,"data_stored":15719573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":27762688,"data_stored":24760554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T09:29:49.831 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.831 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.831 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.832 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.832 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.833 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.833 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.833 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.834 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.834 INFO:tasks.ceph:pgid 5.22 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.834 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.834 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.835 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.835 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-03-09T08:52:17.240377+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=17, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.835 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.835 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.835 INFO:tasks.ceph:pgid 5.35 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.836 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.836 INFO:tasks.ceph:pgid 5.30 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.836 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.836 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.836 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.837 INFO:tasks.ceph:pgid 5.39 last_scrub_stamp 2024-03-09T08:52:28.539137+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=8, tm_min=52, tm_sec=28, tm_wday=5, tm_yday=69, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=9, tm_hour=9, tm_min=29, tm_sec=20, tm_wday=5, tm_yday=69, tm_isdst=0) 2024-03-09T09:29:49.837 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-09T09:30:09.838 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-09T09:30:10.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.056+0000 7f0f4b200640 1 Processor -- start 2024-03-09T09:30:10.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.057+0000 7f0f4b200640 1 -- start start 2024-03-09T09:30:10.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f4b200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f4c0c3e00 0x7f0f4c0c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f4b200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f0f4c0aa9d0 0x7f0f4c1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f4b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f4c073900 0x7f0f4c06fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f4b200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4c06f6d0 con 0x7f0f4c0c3e00 2024-03-09T09:30:10.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f4b200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4c06f9b0 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f4b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c153e00 con 0x7f0f4c073900 2024-03-09T09:30:10.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f49e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f4c0c3e00 0x7f0f4c0c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f49e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f0f4c0c3e00 0x7f0f4c0c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38048/0 (socket says 172.21.15.67:38048) 2024-03-09T09:30:10.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f49e00640 1 -- 172.21.15.67:0/1153387613 learned_addr learned my addr 172.21.15.67:0/1153387613 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:10.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f4a800640 1 --2- 172.21.15.67:0/1153387613 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f4c073900 0x7f0f4c06fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f49400640 1 --2- 172.21.15.67:0/1153387613 >> v2:172.21.15.134:3300/0 conn(0x7f0f4c0aa9d0 0x7f0f4c1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f49e00640 1 -- 172.21.15.67:0/1153387613 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f4c073900 msgr2=0x7f0f4c06fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f49e00640 1 --2- 172.21.15.67:0/1153387613 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f4c073900 0x7f0f4c06fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.059+0000 7f0f49e00640 1 -- 172.21.15.67:0/1153387613 >> v2:172.21.15.134:3300/0 conn(0x7f0f4c0aa9d0 msgr2=0x7f0f4c1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.060+0000 7f0f49e00640 1 --2- 172.21.15.67:0/1153387613 >> v2:172.21.15.134:3300/0 conn(0x7f0f4c0aa9d0 0x7f0f4c1538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.060+0000 7f0f49e00640 1 -- 172.21.15.67:0/1153387613 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c1540d0 con 0x7f0f4c0c3e00 2024-03-09T09:30:10.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.060+0000 7f0f4a800640 1 --2- 172.21.15.67:0/1153387613 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f4c073900 0x7f0f4c06fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:30:10.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.060+0000 7f0f49400640 1 --2- 172.21.15.67:0/1153387613 >> v2:172.21.15.134:3300/0 conn(0x7f0f4c0aa9d0 0x7f0f4c1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:30:10.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.061+0000 7f0f49e00640 1 --2- 172.21.15.67:0/1153387613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f4c0c3e00 0x7f0f4c0c41e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009e50 tx=0x7f0f3c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d0493411ad772d server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.061+0000 7f0f48a00640 1 -- 172.21.15.67:0/1153387613 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c02c9f0 con 0x7f0f4c0c3e00 2024-03-09T09:30:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.061+0000 7f0f48a00640 1 -- 172.21.15.67:0/1153387613 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c02cb90 con 0x7f0f4c0c3e00 2024-03-09T09:30:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.061+0000 7f0f48a00640 1 -- 172.21.15.67:0/1153387613 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c0343f0 con 0x7f0f4c0c3e00 2024-03-09T09:30:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.062+0000 7f0f4b200640 1 -- 172.21.15.67:0/1153387613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f4c0c3e00 msgr2=0x7f0f4c0c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:10.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.062+0000 7f0f4b200640 1 --2- 172.21.15.67:0/1153387613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f4c0c3e00 0x7f0f4c0c41e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009e50 tx=0x7f0f3c02be50 comp rx=0 tx=0).stop 2024-03-09T09:30:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.062+0000 7f0f4b200640 1 -- 172.21.15.67:0/1153387613 shutdown_connections 2024-03-09T09:30:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.062+0000 7f0f4b200640 1 --2- 172.21.15.67:0/1153387613 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f4c073900 0x7f0f4c06fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.062+0000 7f0f4b200640 1 --2- 172.21.15.67:0/1153387613 >> v2:172.21.15.134:3300/0 conn(0x7f0f4c0aa9d0 0x7f0f4c1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.062+0000 7f0f4b200640 1 --2- 172.21.15.67:0/1153387613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f4c0c3e00 0x7f0f4c0c41e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.062+0000 7f0f4b200640 1 -- 172.21.15.67:0/1153387613 >> 172.21.15.67:0/1153387613 conn(0x7f0f4c067180 msgr2=0x7f0f4c067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:10.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.062+0000 7f0f4b200640 1 -- 172.21.15.67:0/1153387613 shutdown_connections 2024-03-09T09:30:10.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.062+0000 7f0f4b200640 1 -- 172.21.15.67:0/1153387613 wait complete. 2024-03-09T09:30:10.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.063+0000 7f0f4b200640 1 Processor -- start 2024-03-09T09:30:10.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.064+0000 7f0f4b200640 1 -- start start 2024-03-09T09:30:10.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.064+0000 7f0f4b200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f4c073900 0x7f0f4c1ece50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.064+0000 7f0f4b200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f4c0aa9d0 0x7f0f4c1ed390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.064+0000 7f0f4b200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c0c3e00 0x7f0f4c1439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f4b200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c14a6a0 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f4b200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c154690 con 0x7f0f4c0c3e00 2024-03-09T09:30:10.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f4b200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c14a820 con 0x7f0f4c073900 2024-03-09T09:30:10.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f49400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f4c0aa9d0 0x7f0f4c1ed390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f49400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f4c0aa9d0 0x7f0f4c1ed390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38052/0 (socket says 172.21.15.67:38052) 2024-03-09T09:30:10.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f49400640 1 -- 172.21.15.67:0/1438404589 learned_addr learned my addr 172.21.15.67:0/1438404589 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f49e00640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f4c073900 0x7f0f4c1ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f4a800640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c0c3e00 0x7f0f4c1439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f49400640 1 -- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f4c073900 msgr2=0x7f0f4c1ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f49400640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f4c073900 0x7f0f4c1ece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f49400640 1 -- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c0c3e00 msgr2=0x7f0f4c1439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f49400640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c0c3e00 0x7f0f4c1439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.065+0000 7f0f49400640 1 -- 172.21.15.67:0/1438404589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c14d9e0 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.066+0000 7f0f49400640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f4c0aa9d0 0x7f0f4c1ed390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400c7f0 tx=0x7f0f3400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.066+0000 7f0f42000640 1 -- 172.21.15.67:0/1438404589 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34015070 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.066+0000 7f0f42000640 1 -- 172.21.15.67:0/1438404589 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f34007a30 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.066+0000 7f0f42000640 1 -- 172.21.15.67:0/1438404589 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34007e00 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.066+0000 7f0f4b200640 1 -- 172.21.15.67:0/1438404589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4c14ae80 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.066+0000 7f0f4b200640 1 -- 172.21.15.67:0/1438404589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4c14cd30 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.067+0000 7f0f4b200640 1 -- 172.21.15.67:0/1438404589 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4c04f3c0 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.068+0000 7f0f42000640 1 -- 172.21.15.67:0/1438404589 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0f34018070 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.069+0000 7f0f42000640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f2806b4d0 0x7f0f2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.069+0000 7f0f42000640 1 -- 172.21.15.67:0/1438404589 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f0f3408a4d0 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.070+0000 7f0f49e00640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f2806b4d0 0x7f0f2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.071+0000 7f0f49e00640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f2806b4d0 0x7f0f2806d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c035000 tx=0x7f0f3c030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:10.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.074+0000 7f0f42000640 1 -- 172.21.15.67:0/1438404589 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0f34058930 con 0x7f0f4c0aa9d0 2024-03-09T09:30:10.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.272+0000 7f0f4b200640 1 -- 172.21.15.67:0/1438404589 --> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f4c145f90 con 0x7f0f2806b4d0 2024-03-09T09:30:10.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.295+0000 7f0f42000640 1 -- 172.21.15.67:0/1438404589 <== mgr.4106 v2:172.21.15.134:6832/1750216114 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+502260 (secure 0 0 0) 0x7f0f4c145f90 con 0x7f0f2806b4d0 2024-03-09T09:30:10.314 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:30:10.317 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-09T09:30:10.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 -- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f2806b4d0 msgr2=0x7f0f2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:10.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f2806b4d0 0x7f0f2806d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c035000 tx=0x7f0f3c030000 comp rx=0 tx=0).stop 2024-03-09T09:30:10.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 -- 172.21.15.67:0/1438404589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f4c0aa9d0 msgr2=0x7f0f4c1ed390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f4c0aa9d0 0x7f0f4c1ed390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400c7f0 tx=0x7f0f3400cbf0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 -- 172.21.15.67:0/1438404589 shutdown_connections 2024-03-09T09:30:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f0f2806b4d0 0x7f0f2806d990 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f4c0c3e00 0x7f0f4c1439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0f4c0aa9d0 0x7f0f4c1ed390 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 --2- 172.21.15.67:0/1438404589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f4c073900 0x7f0f4c1ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 -- 172.21.15.67:0/1438404589 >> 172.21.15.67:0/1438404589 conn(0x7f0f4c067180 msgr2=0x7f0f4c0aae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 -- 172.21.15.67:0/1438404589 shutdown_connections 2024-03-09T09:30:10.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.305+0000 7f0f4b200640 1 -- 172.21.15.67:0/1438404589 wait complete. 2024-03-09T09:30:10.335 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":1223,"stamp":"2024-03-09T09:30:09.732684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1880769962,"num_objects":17182,"num_object_clones":4728,"num_object_copies":34364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17182,"num_whiteouts":4728,"num_read":20750,"num_read_kb":3057165,"num_write":223858,"num_write_kb":12119877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2438,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":35042970,"num_omap_keys":55074,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350376,"ondisk_log_size":350376,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6273888,"kb_used_data":3727712,"kb_used_omap":12,"kb_used_meta":2546099,"kb_avail":748700832,"statfs":{"total":773094113280,"available":766669651968,"internally_reserved":0,"allocated":3817177088,"data_stored":3766281908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2607205938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004688"},"pg_stats":[{"pgid":"5.18","version":"89'2564","reported_seq":5449,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.802823+0000","last_change":"2024-03-09T09:29:40.802889+0000","last_active":"2024-03-09T09:29:40.802823+0000","last_peered":"2024-03-09T09:29:40.802823+0000","last_clean":"2024-03-09T09:29:40.802823+0000","last_became_active":"2024-03-09T08:52:29.573261+0000","last_became_peered":"2024-03-09T08:52:29.573261+0000","last_unstale":"2024-03-09T09:29:40.802823+0000","last_undegraded":"2024-03-09T09:29:40.802823+0000","last_fullsized":"2024-03-09T09:29:40.802823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2564","last_scrub_stamp":"2024-03-09T09:29:40.802693+0000","last_deep_scrub":"89'2564","last_deep_scrub_stamp":"2024-03-09T09:29:40.802693+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.802693+0000","objects_scrubbed":90,"log_size":2564,"log_dups_size":0,"ondisk_log_size":2564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:20:02.649663+0000","scrub_duration":130,"objects_trimmed":4,"snaptrim_duration":3.854819714,"stat_sum":{"num_bytes":1582215,"num_objects":90,"num_object_clones":27,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":27,"num_read":122,"num_read_kb":30313,"num_write":1455,"num_write_kb":61176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"89'487","reported_seq":667,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.108739+0000","last_change":"2024-03-09T09:29:29.108739+0000","last_active":"2024-03-09T09:29:29.108739+0000","last_peered":"2024-03-09T09:29:29.108739+0000","last_clean":"2024-03-09T09:29:29.108739+0000","last_became_active":"2024-03-09T08:52:17.288417+0000","last_became_peered":"2024-03-09T08:52:17.288417+0000","last_unstale":"2024-03-09T09:29:29.108739+0000","last_undegraded":"2024-03-09T09:29:29.108739+0000","last_fullsized":"2024-03-09T09:29:29.108739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'487","last_scrub_stamp":"2024-03-09T09:29:29.108443+0000","last_deep_scrub":"89'487","last_deep_scrub_stamp":"2024-03-09T09:29:29.108443+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.108443+0000","objects_scrubbed":38,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:47:26.177138+0000","scrub_duration":132,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":462,"num_write_kb":82632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":421963,"num_omap_keys":720,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"89'2290","reported_seq":5283,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.579752+0000","last_change":"2024-03-09T09:29:44.579843+0000","last_active":"2024-03-09T09:29:44.579752+0000","last_peered":"2024-03-09T09:29:44.579752+0000","last_clean":"2024-03-09T09:29:44.579752+0000","last_became_active":"2024-03-09T08:52:18.233300+0000","last_became_peered":"2024-03-09T08:52:18.233300+0000","last_unstale":"2024-03-09T09:29:44.579752+0000","last_undegraded":"2024-03-09T09:29:44.579752+0000","last_fullsized":"2024-03-09T09:29:44.579752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2290","last_scrub_stamp":"2024-03-09T09:29:44.579609+0000","last_deep_scrub":"89'2290","last_deep_scrub_stamp":"2024-03-09T09:29:44.579609+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.579609+0000","objects_scrubbed":111,"log_size":2290,"log_dups_size":0,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:24:46.179406+0000","scrub_duration":302,"objects_trimmed":0,"snaptrim_duration":0.47085273999999999,"stat_sum":{"num_bytes":1473782,"num_objects":111,"num_object_clones":38,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":38,"num_read":99,"num_read_kb":25344,"num_write":1311,"num_write_kb":54508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"89'448","reported_seq":616,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.368720+0000","last_change":"2024-03-09T09:29:28.368794+0000","last_active":"2024-03-09T09:29:28.368720+0000","last_peered":"2024-03-09T09:29:28.368720+0000","last_clean":"2024-03-09T09:29:28.368720+0000","last_became_active":"2024-03-09T08:52:28.571607+0000","last_became_peered":"2024-03-09T08:52:28.571607+0000","last_unstale":"2024-03-09T09:29:28.368720+0000","last_undegraded":"2024-03-09T09:29:28.368720+0000","last_fullsized":"2024-03-09T09:29:28.368720+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'448","last_scrub_stamp":"2024-03-09T09:29:28.368593+0000","last_deep_scrub":"89'448","last_deep_scrub_stamp":"2024-03-09T09:29:28.368593+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.368593+0000","objects_scrubbed":43,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:36:37.605429+0000","scrub_duration":83,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":445,"num_write_kb":78871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":504839,"num_omap_keys":800,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"89'2374","reported_seq":5299,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.106756+0000","last_change":"2024-03-09T09:29:41.106834+0000","last_active":"2024-03-09T09:29:41.106756+0000","last_peered":"2024-03-09T09:29:41.106756+0000","last_clean":"2024-03-09T09:29:41.106756+0000","last_became_active":"2024-03-09T08:52:29.578351+0000","last_became_peered":"2024-03-09T08:52:29.578351+0000","last_unstale":"2024-03-09T09:29:41.106756+0000","last_undegraded":"2024-03-09T09:29:41.106756+0000","last_fullsized":"2024-03-09T09:29:41.106756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2374","last_scrub_stamp":"2024-03-09T09:29:41.106670+0000","last_deep_scrub":"89'2374","last_deep_scrub_stamp":"2024-03-09T09:29:41.106670+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.106670+0000","objects_scrubbed":118,"log_size":2374,"log_dups_size":0,"ondisk_log_size":2374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:05:13.031039+0000","scrub_duration":227,"objects_trimmed":4,"snaptrim_duration":1.703796324,"stat_sum":{"num_bytes":1252433,"num_objects":118,"num_object_clones":38,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":38,"num_read":118,"num_read_kb":38794,"num_write":1352,"num_write_kb":71340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"89'791","reported_seq":970,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.456369+0000","last_change":"2024-03-09T09:29:39.456535+0000","last_active":"2024-03-09T09:29:39.456369+0000","last_peered":"2024-03-09T09:29:39.456369+0000","last_clean":"2024-03-09T09:29:39.456369+0000","last_became_active":"2024-03-09T08:52:17.270584+0000","last_became_peered":"2024-03-09T08:52:17.270584+0000","last_unstale":"2024-03-09T09:29:39.456369+0000","last_undegraded":"2024-03-09T09:29:39.456369+0000","last_fullsized":"2024-03-09T09:29:39.456369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'791","last_scrub_stamp":"2024-03-09T09:29:39.456268+0000","last_deep_scrub":"89'791","last_deep_scrub_stamp":"2024-03-09T09:29:39.456268+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.456268+0000","objects_scrubbed":47,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:12:28.443175+0000","scrub_duration":172,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":51,"num_read_kb":0,"num_write":808,"num_write_kb":91456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":528589,"num_omap_keys":713,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'2268","reported_seq":5268,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.213491+0000","last_change":"2024-03-09T09:29:27.213491+0000","last_active":"2024-03-09T09:29:27.213491+0000","last_peered":"2024-03-09T09:29:27.213491+0000","last_clean":"2024-03-09T09:29:27.213491+0000","last_became_active":"2024-03-09T08:52:18.236615+0000","last_became_peered":"2024-03-09T08:52:18.236615+0000","last_unstale":"2024-03-09T09:29:27.213491+0000","last_undegraded":"2024-03-09T09:29:27.213491+0000","last_fullsized":"2024-03-09T09:29:27.213491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2268","last_scrub_stamp":"2024-03-09T09:29:27.213384+0000","last_deep_scrub":"89'2268","last_deep_scrub_stamp":"2024-03-09T09:29:27.213384+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.213384+0000","objects_scrubbed":108,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:16:20.677968+0000","scrub_duration":271,"objects_trimmed":0,"snaptrim_duration":0.48044756999999999,"stat_sum":{"num_bytes":498978,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":114,"num_read_kb":18251,"num_write":1268,"num_write_kb":37672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"89'943","reported_seq":1113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.925803+0000","last_change":"2024-03-09T09:29:31.925832+0000","last_active":"2024-03-09T09:29:31.925803+0000","last_peered":"2024-03-09T09:29:31.925803+0000","last_clean":"2024-03-09T09:29:31.925803+0000","last_became_active":"2024-03-09T08:52:28.572789+0000","last_became_peered":"2024-03-09T08:52:28.572789+0000","last_unstale":"2024-03-09T09:29:31.925803+0000","last_undegraded":"2024-03-09T09:29:31.925803+0000","last_fullsized":"2024-03-09T09:29:31.925803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'943","last_scrub_stamp":"2024-03-09T09:29:31.925738+0000","last_deep_scrub":"89'943","last_deep_scrub_stamp":"2024-03-09T09:29:31.925738+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.925738+0000","objects_scrubbed":51,"log_size":943,"log_dups_size":0,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:10:40.595121+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":939,"num_write_kb":120134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":539982,"num_omap_keys":943,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"89'2358","reported_seq":5274,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:51.295746+0000","last_change":"2024-03-09T09:29:51.295746+0000","last_active":"2024-03-09T09:29:51.295746+0000","last_peered":"2024-03-09T09:29:51.295746+0000","last_clean":"2024-03-09T09:29:51.295746+0000","last_became_active":"2024-03-09T08:52:29.593726+0000","last_became_peered":"2024-03-09T08:52:29.593726+0000","last_unstale":"2024-03-09T09:29:51.295746+0000","last_undegraded":"2024-03-09T09:29:51.295746+0000","last_fullsized":"2024-03-09T09:29:51.295746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2358","last_scrub_stamp":"2024-03-09T09:29:51.295523+0000","last_deep_scrub":"89'2358","last_deep_scrub_stamp":"2024-03-09T09:29:51.295523+0000","last_clean_scrub_stamp":"2024-03-09T09:29:51.295523+0000","objects_scrubbed":92,"log_size":2358,"log_dups_size":0,"ondisk_log_size":2358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:59:20.593774+0000","scrub_duration":129,"objects_trimmed":0,"snaptrim_duration":4.3826170339999999,"stat_sum":{"num_bytes":1068833,"num_objects":92,"num_object_clones":30,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":30,"num_read":126,"num_read_kb":14571,"num_write":1308,"num_write_kb":45011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"89'732","reported_seq":910,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.392894+0000","last_change":"2024-03-09T09:29:45.392894+0000","last_active":"2024-03-09T09:29:45.392894+0000","last_peered":"2024-03-09T09:29:45.392894+0000","last_clean":"2024-03-09T09:29:45.392894+0000","last_became_active":"2024-03-09T08:52:17.295175+0000","last_became_peered":"2024-03-09T08:52:17.295175+0000","last_unstale":"2024-03-09T09:29:45.392894+0000","last_undegraded":"2024-03-09T09:29:45.392894+0000","last_fullsized":"2024-03-09T09:29:45.392894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'732","last_scrub_stamp":"2024-03-09T09:29:45.392683+0000","last_deep_scrub":"89'732","last_deep_scrub_stamp":"2024-03-09T09:29:45.392683+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.392683+0000","objects_scrubbed":58,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:57:01.257852+0000","scrub_duration":161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":67,"num_read_kb":0,"num_write":775,"num_write_kb":140765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":732837,"num_omap_keys":1110,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"89'2527","reported_seq":5506,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.126540+0000","last_change":"2024-03-09T09:29:46.126540+0000","last_active":"2024-03-09T09:29:46.126540+0000","last_peered":"2024-03-09T09:29:46.126540+0000","last_clean":"2024-03-09T09:29:46.126540+0000","last_became_active":"2024-03-09T08:52:18.230303+0000","last_became_peered":"2024-03-09T08:52:18.230303+0000","last_unstale":"2024-03-09T09:29:46.126540+0000","last_undegraded":"2024-03-09T09:29:46.126540+0000","last_fullsized":"2024-03-09T09:29:46.126540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2527","last_scrub_stamp":"2024-03-09T09:29:46.126370+0000","last_deep_scrub":"89'2527","last_deep_scrub_stamp":"2024-03-09T09:29:46.126370+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.126370+0000","objects_scrubbed":148,"log_size":2527,"log_dups_size":0,"ondisk_log_size":2527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:41:22.100316+0000","scrub_duration":208,"objects_trimmed":0,"snaptrim_duration":1.653915281,"stat_sum":{"num_bytes":2793747,"num_objects":148,"num_object_clones":53,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":53,"num_read":138,"num_read_kb":27086,"num_write":1428,"num_write_kb":43660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"89'568","reported_seq":737,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.841659+0000","last_change":"2024-03-09T09:29:28.841659+0000","last_active":"2024-03-09T09:29:28.841659+0000","last_peered":"2024-03-09T09:29:28.841659+0000","last_clean":"2024-03-09T09:29:28.841659+0000","last_became_active":"2024-03-09T08:52:28.576743+0000","last_became_peered":"2024-03-09T08:52:28.576743+0000","last_unstale":"2024-03-09T09:29:28.841659+0000","last_undegraded":"2024-03-09T09:29:28.841659+0000","last_fullsized":"2024-03-09T09:29:28.841659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'568","last_scrub_stamp":"2024-03-09T09:29:28.841466+0000","last_deep_scrub":"89'568","last_deep_scrub_stamp":"2024-03-09T09:29:28.841466+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.841466+0000","objects_scrubbed":37,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:56:34.500330+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":35,"num_read_kb":0,"num_write":543,"num_write_kb":111605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":558669,"num_omap_keys":858,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"89'2401","reported_seq":5314,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:50.291334+0000","last_change":"2024-03-09T09:29:50.291334+0000","last_active":"2024-03-09T09:29:50.291334+0000","last_peered":"2024-03-09T09:29:50.291334+0000","last_clean":"2024-03-09T09:29:50.291334+0000","last_became_active":"2024-03-09T08:52:29.581322+0000","last_became_peered":"2024-03-09T08:52:29.581322+0000","last_unstale":"2024-03-09T09:29:50.291334+0000","last_undegraded":"2024-03-09T09:29:50.291334+0000","last_fullsized":"2024-03-09T09:29:50.291334+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2401","last_scrub_stamp":"2024-03-09T09:29:50.291208+0000","last_deep_scrub":"89'2401","last_deep_scrub_stamp":"2024-03-09T09:29:50.291208+0000","last_clean_scrub_stamp":"2024-03-09T09:29:50.291208+0000","objects_scrubbed":126,"log_size":2401,"log_dups_size":0,"ondisk_log_size":2401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:19:05.631191+0000","scrub_duration":316,"objects_trimmed":2,"snaptrim_duration":4.4571779879999998,"stat_sum":{"num_bytes":3086231,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":121,"num_read_kb":9749,"num_write":1336,"num_write_kb":28981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"89'508","reported_seq":685,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.327430+0000","last_change":"2024-03-09T09:29:44.327430+0000","last_active":"2024-03-09T09:29:44.327430+0000","last_peered":"2024-03-09T09:29:44.327430+0000","last_clean":"2024-03-09T09:29:44.327430+0000","last_became_active":"2024-03-09T08:52:17.294720+0000","last_became_peered":"2024-03-09T08:52:17.294720+0000","last_unstale":"2024-03-09T09:29:44.327430+0000","last_undegraded":"2024-03-09T09:29:44.327430+0000","last_fullsized":"2024-03-09T09:29:44.327430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'508","last_scrub_stamp":"2024-03-09T09:29:44.327194+0000","last_deep_scrub":"89'508","last_deep_scrub_stamp":"2024-03-09T09:29:44.327194+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.327194+0000","objects_scrubbed":49,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:18:14.903140+0000","scrub_duration":110,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":55,"num_read_kb":0,"num_write":535,"num_write_kb":83331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":648870,"num_omap_keys":703,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"89'2269","reported_seq":5277,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.123958+0000","last_change":"2024-03-09T09:29:44.123958+0000","last_active":"2024-03-09T09:29:44.123958+0000","last_peered":"2024-03-09T09:29:44.123958+0000","last_clean":"2024-03-09T09:29:44.123958+0000","last_became_active":"2024-03-09T08:52:18.228034+0000","last_became_peered":"2024-03-09T08:52:18.228034+0000","last_unstale":"2024-03-09T09:29:44.123958+0000","last_undegraded":"2024-03-09T09:29:44.123958+0000","last_fullsized":"2024-03-09T09:29:44.123958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2269","last_scrub_stamp":"2024-03-09T09:29:44.123763+0000","last_deep_scrub":"89'2269","last_deep_scrub_stamp":"2024-03-09T09:29:44.123763+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.123763+0000","objects_scrubbed":111,"log_size":2269,"log_dups_size":0,"ondisk_log_size":2269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:07:13.959027+0000","scrub_duration":192,"objects_trimmed":0,"snaptrim_duration":1.3715434019999999,"stat_sum":{"num_bytes":1052383,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":116,"num_read_kb":6444,"num_write":1289,"num_write_kb":22757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"89'611","reported_seq":780,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.898016+0000","last_change":"2024-03-09T09:29:25.898016+0000","last_active":"2024-03-09T09:29:25.898016+0000","last_peered":"2024-03-09T09:29:25.898016+0000","last_clean":"2024-03-09T09:29:25.898016+0000","last_became_active":"2024-03-09T08:52:28.576715+0000","last_became_peered":"2024-03-09T08:52:28.576715+0000","last_unstale":"2024-03-09T09:29:25.898016+0000","last_undegraded":"2024-03-09T09:29:25.898016+0000","last_fullsized":"2024-03-09T09:29:25.898016+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'611","last_scrub_stamp":"2024-03-09T09:29:25.897713+0000","last_deep_scrub":"89'611","last_deep_scrub_stamp":"2024-03-09T09:29:25.897713+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.897713+0000","objects_scrubbed":42,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:53:23.202311+0000","scrub_duration":135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":598,"num_write_kb":95071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":469023,"num_omap_keys":788,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"89'2251","reported_seq":5198,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.085473+0000","last_change":"2024-03-09T09:29:45.085541+0000","last_active":"2024-03-09T09:29:45.085473+0000","last_peered":"2024-03-09T09:29:45.085473+0000","last_clean":"2024-03-09T09:29:45.085473+0000","last_became_active":"2024-03-09T08:52:29.588661+0000","last_became_peered":"2024-03-09T08:52:29.588661+0000","last_unstale":"2024-03-09T09:29:45.085473+0000","last_undegraded":"2024-03-09T09:29:45.085473+0000","last_fullsized":"2024-03-09T09:29:45.085473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2251","last_scrub_stamp":"2024-03-09T09:29:45.085394+0000","last_deep_scrub":"89'2251","last_deep_scrub_stamp":"2024-03-09T09:29:45.085394+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.085394+0000","objects_scrubbed":92,"log_size":2251,"log_dups_size":0,"ondisk_log_size":2251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:42:36.622585+0000","scrub_duration":182,"objects_trimmed":0,"snaptrim_duration":1.709485658,"stat_sum":{"num_bytes":1197226,"num_objects":92,"num_object_clones":31,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":31,"num_read":105,"num_read_kb":17945,"num_write":1287,"num_write_kb":45184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"89'2324","reported_seq":5304,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.482498+0000","last_change":"2024-03-09T09:29:37.482591+0000","last_active":"2024-03-09T09:29:37.482498+0000","last_peered":"2024-03-09T09:29:37.482498+0000","last_clean":"2024-03-09T09:29:37.482498+0000","last_became_active":"2024-03-09T08:52:18.233978+0000","last_became_peered":"2024-03-09T08:52:18.233978+0000","last_unstale":"2024-03-09T09:29:37.482498+0000","last_undegraded":"2024-03-09T09:29:37.482498+0000","last_fullsized":"2024-03-09T09:29:37.482498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2324","last_scrub_stamp":"2024-03-09T09:29:37.482415+0000","last_deep_scrub":"89'2324","last_deep_scrub_stamp":"2024-03-09T09:29:37.482415+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.482415+0000","objects_scrubbed":98,"log_size":2324,"log_dups_size":0,"ondisk_log_size":2324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:49:14.376917+0000","scrub_duration":143,"objects_trimmed":0,"snaptrim_duration":2.2630355259999999,"stat_sum":{"num_bytes":1174090,"num_objects":98,"num_object_clones":35,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":35,"num_read":98,"num_read_kb":18109,"num_write":1319,"num_write_kb":36158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"89'864","reported_seq":1042,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.253889+0000","last_change":"2024-03-09T09:29:42.253951+0000","last_active":"2024-03-09T09:29:42.253889+0000","last_peered":"2024-03-09T09:29:42.253889+0000","last_clean":"2024-03-09T09:29:42.253889+0000","last_became_active":"2024-03-09T08:52:17.292725+0000","last_became_peered":"2024-03-09T08:52:17.292725+0000","last_unstale":"2024-03-09T09:29:42.253889+0000","last_undegraded":"2024-03-09T09:29:42.253889+0000","last_fullsized":"2024-03-09T09:29:42.253889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'864","last_scrub_stamp":"2024-03-09T09:29:42.253811+0000","last_deep_scrub":"89'864","last_deep_scrub_stamp":"2024-03-09T09:29:42.253811+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.253811+0000","objects_scrubbed":47,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:07:03.471127+0000","scrub_duration":94,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":48,"num_read_kb":0,"num_write":879,"num_write_kb":99632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":529913,"num_omap_keys":803,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"89'799","reported_seq":970,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.775492+0000","last_change":"2024-03-09T09:29:39.775537+0000","last_active":"2024-03-09T09:29:39.775492+0000","last_peered":"2024-03-09T09:29:39.775492+0000","last_clean":"2024-03-09T09:29:39.775492+0000","last_became_active":"2024-03-09T08:52:28.570299+0000","last_became_peered":"2024-03-09T08:52:28.570299+0000","last_unstale":"2024-03-09T09:29:39.775492+0000","last_undegraded":"2024-03-09T09:29:39.775492+0000","last_fullsized":"2024-03-09T09:29:39.775492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'799","last_scrub_stamp":"2024-03-09T09:29:39.775375+0000","last_deep_scrub":"89'799","last_deep_scrub_stamp":"2024-03-09T09:29:39.775375+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.775375+0000","objects_scrubbed":56,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:32:10.366074+0000","scrub_duration":115,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971838,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":820,"num_write_kb":111945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":502543,"num_omap_keys":872,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"89'2302","reported_seq":5225,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.744480+0000","last_change":"2024-03-09T09:29:34.744539+0000","last_active":"2024-03-09T09:29:34.744480+0000","last_peered":"2024-03-09T09:29:34.744480+0000","last_clean":"2024-03-09T09:29:34.744480+0000","last_became_active":"2024-03-09T08:52:29.596585+0000","last_became_peered":"2024-03-09T08:52:29.596585+0000","last_unstale":"2024-03-09T09:29:34.744480+0000","last_undegraded":"2024-03-09T09:29:34.744480+0000","last_fullsized":"2024-03-09T09:29:34.744480+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2302","last_scrub_stamp":"2024-03-09T09:29:34.744358+0000","last_deep_scrub":"89'2302","last_deep_scrub_stamp":"2024-03-09T09:29:34.744358+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.744358+0000","objects_scrubbed":108,"log_size":2302,"log_dups_size":0,"ondisk_log_size":2302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:07:38.416270+0000","scrub_duration":208,"objects_trimmed":0,"snaptrim_duration":0.63600609399999997,"stat_sum":{"num_bytes":1331264,"num_objects":108,"num_object_clones":38,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":38,"num_read":110,"num_read_kb":9978,"num_write":1295,"num_write_kb":30848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'2372","reported_seq":5361,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.562406+0000","last_change":"2024-03-09T09:29:32.562470+0000","last_active":"2024-03-09T09:29:32.562406+0000","last_peered":"2024-03-09T09:29:32.562406+0000","last_clean":"2024-03-09T09:29:32.562406+0000","last_became_active":"2024-03-09T08:52:18.243989+0000","last_became_peered":"2024-03-09T08:52:18.243989+0000","last_unstale":"2024-03-09T09:29:32.562406+0000","last_undegraded":"2024-03-09T09:29:32.562406+0000","last_fullsized":"2024-03-09T09:29:32.562406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2372","last_scrub_stamp":"2024-03-09T09:29:32.562331+0000","last_deep_scrub":"89'2372","last_deep_scrub_stamp":"2024-03-09T09:29:32.562331+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.562331+0000","objects_scrubbed":108,"log_size":2372,"log_dups_size":0,"ondisk_log_size":2372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:16:09.656513+0000","scrub_duration":204,"objects_trimmed":0,"snaptrim_duration":0.69905185199999997,"stat_sum":{"num_bytes":1031167,"num_objects":108,"num_object_clones":39,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":39,"num_read":109,"num_read_kb":22315,"num_write":1384,"num_write_kb":51621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'490","reported_seq":669,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.108374+0000","last_change":"2024-03-09T09:29:33.108485+0000","last_active":"2024-03-09T09:29:33.108374+0000","last_peered":"2024-03-09T09:29:33.108374+0000","last_clean":"2024-03-09T09:29:33.108374+0000","last_became_active":"2024-03-09T08:52:17.274636+0000","last_became_peered":"2024-03-09T08:52:17.274636+0000","last_unstale":"2024-03-09T09:29:33.108374+0000","last_undegraded":"2024-03-09T09:29:33.108374+0000","last_fullsized":"2024-03-09T09:29:33.108374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'490","last_scrub_stamp":"2024-03-09T09:29:33.108300+0000","last_deep_scrub":"89'490","last_deep_scrub_stamp":"2024-03-09T09:29:33.108300+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.108300+0000","objects_scrubbed":45,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:23:03.366878+0000","scrub_duration":150,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":467,"num_write_kb":115624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":462381,"num_omap_keys":698,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"89'670","reported_seq":840,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.910521+0000","last_change":"2024-03-09T09:29:26.910625+0000","last_active":"2024-03-09T09:29:26.910521+0000","last_peered":"2024-03-09T09:29:26.910521+0000","last_clean":"2024-03-09T09:29:26.910521+0000","last_became_active":"2024-03-09T08:52:28.576768+0000","last_became_peered":"2024-03-09T08:52:28.576768+0000","last_unstale":"2024-03-09T09:29:26.910521+0000","last_undegraded":"2024-03-09T09:29:26.910521+0000","last_fullsized":"2024-03-09T09:29:26.910521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'670","last_scrub_stamp":"2024-03-09T09:29:26.910351+0000","last_deep_scrub":"89'670","last_deep_scrub_stamp":"2024-03-09T09:29:26.910351+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.910351+0000","objects_scrubbed":37,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:30:18.446605+0000","scrub_duration":145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":674,"num_write_kb":99647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":379703,"num_omap_keys":669,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"89'2368","reported_seq":5299,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.656669+0000","last_change":"2024-03-09T09:29:45.656725+0000","last_active":"2024-03-09T09:29:45.656669+0000","last_peered":"2024-03-09T09:29:45.656669+0000","last_clean":"2024-03-09T09:29:45.656669+0000","last_became_active":"2024-03-09T08:52:29.579653+0000","last_became_peered":"2024-03-09T08:52:29.579653+0000","last_unstale":"2024-03-09T09:29:45.656669+0000","last_undegraded":"2024-03-09T09:29:45.656669+0000","last_fullsized":"2024-03-09T09:29:45.656669+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2368","last_scrub_stamp":"2024-03-09T09:29:45.656570+0000","last_deep_scrub":"89'2368","last_deep_scrub_stamp":"2024-03-09T09:29:45.656570+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.656570+0000","objects_scrubbed":107,"log_size":2368,"log_dups_size":0,"ondisk_log_size":2368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:26:53.091676+0000","scrub_duration":136,"objects_trimmed":4,"snaptrim_duration":3.9938263410000001,"stat_sum":{"num_bytes":1661343,"num_objects":107,"num_object_clones":33,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":33,"num_read":118,"num_read_kb":35803,"num_write":1365,"num_write_kb":51532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'2483","reported_seq":5480,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.520518+0000","last_change":"2024-03-09T09:29:42.520557+0000","last_active":"2024-03-09T09:29:42.520518+0000","last_peered":"2024-03-09T09:29:42.520518+0000","last_clean":"2024-03-09T09:29:42.520518+0000","last_became_active":"2024-03-09T08:52:18.232638+0000","last_became_peered":"2024-03-09T08:52:18.232638+0000","last_unstale":"2024-03-09T09:29:42.520518+0000","last_undegraded":"2024-03-09T09:29:42.520518+0000","last_fullsized":"2024-03-09T09:29:42.520518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2483","last_scrub_stamp":"2024-03-09T09:29:42.520449+0000","last_deep_scrub":"89'2483","last_deep_scrub_stamp":"2024-03-09T09:29:42.520449+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.520449+0000","objects_scrubbed":132,"log_size":2483,"log_dups_size":0,"ondisk_log_size":2483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:55:49.425811+0000","scrub_duration":222,"objects_trimmed":0,"snaptrim_duration":0.380889329,"stat_sum":{"num_bytes":1156693,"num_objects":132,"num_object_clones":46,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":46,"num_read":122,"num_read_kb":38030,"num_write":1453,"num_write_kb":61557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"89'690","reported_seq":869,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.979699+0000","last_change":"2024-03-09T09:29:36.979792+0000","last_active":"2024-03-09T09:29:36.979699+0000","last_peered":"2024-03-09T09:29:36.979699+0000","last_clean":"2024-03-09T09:29:36.979699+0000","last_became_active":"2024-03-09T08:52:17.283436+0000","last_became_peered":"2024-03-09T08:52:17.283436+0000","last_unstale":"2024-03-09T09:29:36.979699+0000","last_undegraded":"2024-03-09T09:29:36.979699+0000","last_fullsized":"2024-03-09T09:29:36.979699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'690","last_scrub_stamp":"2024-03-09T09:29:36.979617+0000","last_deep_scrub":"89'690","last_deep_scrub_stamp":"2024-03-09T09:29:36.979617+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.979617+0000","objects_scrubbed":46,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:35:33.065130+0000","scrub_duration":158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":715,"num_write_kb":95464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":598323,"num_omap_keys":1009,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"89'673","reported_seq":843,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:24.128278+0000","last_change":"2024-03-09T09:29:24.128343+0000","last_active":"2024-03-09T09:29:24.128278+0000","last_peered":"2024-03-09T09:29:24.128278+0000","last_clean":"2024-03-09T09:29:24.128278+0000","last_became_active":"2024-03-09T08:52:28.576070+0000","last_became_peered":"2024-03-09T08:52:28.576070+0000","last_unstale":"2024-03-09T09:29:24.128278+0000","last_undegraded":"2024-03-09T09:29:24.128278+0000","last_fullsized":"2024-03-09T09:29:24.128278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'673","last_scrub_stamp":"2024-03-09T09:29:24.128149+0000","last_deep_scrub":"89'673","last_deep_scrub_stamp":"2024-03-09T09:29:24.128149+0000","last_clean_scrub_stamp":"2024-03-09T09:29:24.128149+0000","objects_scrubbed":47,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:29:11.611027+0000","scrub_duration":125,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748758,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":685,"num_write_kb":95366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":455288,"num_omap_keys":782,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"89'2312","reported_seq":5251,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:48.119019+0000","last_change":"2024-03-09T09:29:48.119085+0000","last_active":"2024-03-09T09:29:48.119019+0000","last_peered":"2024-03-09T09:29:48.119019+0000","last_clean":"2024-03-09T09:29:48.119019+0000","last_became_active":"2024-03-09T08:52:29.577968+0000","last_became_peered":"2024-03-09T08:52:29.577968+0000","last_unstale":"2024-03-09T09:29:48.119019+0000","last_undegraded":"2024-03-09T09:29:48.119019+0000","last_fullsized":"2024-03-09T09:29:48.119019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2312","last_scrub_stamp":"2024-03-09T09:29:48.118943+0000","last_deep_scrub":"89'2312","last_deep_scrub_stamp":"2024-03-09T09:29:48.118943+0000","last_clean_scrub_stamp":"2024-03-09T09:29:48.118943+0000","objects_scrubbed":111,"log_size":2312,"log_dups_size":0,"ondisk_log_size":2312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:05.668155+0000","scrub_duration":173,"objects_trimmed":2,"snaptrim_duration":4.4556873890000004,"stat_sum":{"num_bytes":4719568,"num_objects":111,"num_object_clones":34,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":34,"num_read":112,"num_read_kb":26370,"num_write":1323,"num_write_kb":45566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"89'2394","reported_seq":5366,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.902991+0000","last_change":"2024-03-09T09:29:33.903075+0000","last_active":"2024-03-09T09:29:33.902991+0000","last_peered":"2024-03-09T09:29:33.902991+0000","last_clean":"2024-03-09T09:29:33.902991+0000","last_became_active":"2024-03-09T08:52:18.216166+0000","last_became_peered":"2024-03-09T08:52:18.216166+0000","last_unstale":"2024-03-09T09:29:33.902991+0000","last_undegraded":"2024-03-09T09:29:33.902991+0000","last_fullsized":"2024-03-09T09:29:33.902991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2394","last_scrub_stamp":"2024-03-09T09:29:33.902908+0000","last_deep_scrub":"89'2394","last_deep_scrub_stamp":"2024-03-09T09:29:33.902908+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.902908+0000","objects_scrubbed":98,"log_size":2394,"log_dups_size":0,"ondisk_log_size":2394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:41:52.741118+0000","scrub_duration":181,"objects_trimmed":0,"snaptrim_duration":3.0901323600000001,"stat_sum":{"num_bytes":867761,"num_objects":98,"num_object_clones":31,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":31,"num_read":98,"num_read_kb":37765,"num_write":1389,"num_write_kb":74512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'491","reported_seq":671,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.389166+0000","last_change":"2024-03-09T09:29:43.389237+0000","last_active":"2024-03-09T09:29:43.389166+0000","last_peered":"2024-03-09T09:29:43.389166+0000","last_clean":"2024-03-09T09:29:43.389166+0000","last_became_active":"2024-03-09T08:52:17.293001+0000","last_became_peered":"2024-03-09T08:52:17.293001+0000","last_unstale":"2024-03-09T09:29:43.389166+0000","last_undegraded":"2024-03-09T09:29:43.389166+0000","last_fullsized":"2024-03-09T09:29:43.389166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'491","last_scrub_stamp":"2024-03-09T09:29:43.388996+0000","last_deep_scrub":"89'491","last_deep_scrub_stamp":"2024-03-09T09:29:43.388996+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.388996+0000","objects_scrubbed":57,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:44:51.123622+0000","scrub_duration":207,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":69,"num_read_kb":0,"num_write":510,"num_write_kb":91498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":792682,"num_omap_keys":1280,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"89'655","reported_seq":824,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.853871+0000","last_change":"2024-03-09T09:29:29.853941+0000","last_active":"2024-03-09T09:29:29.853871+0000","last_peered":"2024-03-09T09:29:29.853871+0000","last_clean":"2024-03-09T09:29:29.853871+0000","last_became_active":"2024-03-09T08:52:28.573337+0000","last_became_peered":"2024-03-09T08:52:28.573337+0000","last_unstale":"2024-03-09T09:29:29.853871+0000","last_undegraded":"2024-03-09T09:29:29.853871+0000","last_fullsized":"2024-03-09T09:29:29.853871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'655","last_scrub_stamp":"2024-03-09T09:29:29.853754+0000","last_deep_scrub":"89'655","last_deep_scrub_stamp":"2024-03-09T09:29:29.853754+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.853754+0000","objects_scrubbed":41,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:25:33.750175+0000","scrub_duration":99,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":46,"num_read_kb":0,"num_write":666,"num_write_kb":107635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":568275,"num_omap_keys":907,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"89'2459","reported_seq":5397,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:52.493244+0000","last_change":"2024-03-09T09:29:52.493270+0000","last_active":"2024-03-09T09:29:52.493244+0000","last_peered":"2024-03-09T09:29:52.493244+0000","last_clean":"2024-03-09T09:29:52.493244+0000","last_became_active":"2024-03-09T08:52:29.597382+0000","last_became_peered":"2024-03-09T08:52:29.597382+0000","last_unstale":"2024-03-09T09:29:52.493244+0000","last_undegraded":"2024-03-09T09:29:52.493244+0000","last_fullsized":"2024-03-09T09:29:52.493244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2459","last_scrub_stamp":"2024-03-09T09:29:52.493142+0000","last_deep_scrub":"89'2459","last_deep_scrub_stamp":"2024-03-09T09:29:52.493142+0000","last_clean_scrub_stamp":"2024-03-09T09:29:52.493142+0000","objects_scrubbed":123,"log_size":2459,"log_dups_size":0,"ondisk_log_size":2459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:34:18.198813+0000","scrub_duration":199,"objects_trimmed":2,"snaptrim_duration":4.1575832950000002,"stat_sum":{"num_bytes":392237,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":136,"num_read_kb":53607,"num_write":1404,"num_write_kb":81678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"89'2285","reported_seq":5390,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.148258+0000","last_change":"2024-03-09T09:29:32.148301+0000","last_active":"2024-03-09T09:29:32.148258+0000","last_peered":"2024-03-09T09:29:32.148258+0000","last_clean":"2024-03-09T09:29:32.148258+0000","last_became_active":"2024-03-09T08:52:18.227469+0000","last_became_peered":"2024-03-09T08:52:18.227469+0000","last_unstale":"2024-03-09T09:29:32.148258+0000","last_undegraded":"2024-03-09T09:29:32.148258+0000","last_fullsized":"2024-03-09T09:29:32.148258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2285","last_scrub_stamp":"2024-03-09T09:29:32.148181+0000","last_deep_scrub":"89'2285","last_deep_scrub_stamp":"2024-03-09T09:29:32.148181+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.148181+0000","objects_scrubbed":110,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:57:59.729700+0000","scrub_duration":179,"objects_trimmed":0,"snaptrim_duration":0.38281627099999999,"stat_sum":{"num_bytes":951718,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":197,"num_read_kb":25950,"num_write":1332,"num_write_kb":45421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'393","reported_seq":572,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.515060+0000","last_change":"2024-03-09T09:29:33.515152+0000","last_active":"2024-03-09T09:29:33.515060+0000","last_peered":"2024-03-09T09:29:33.515060+0000","last_clean":"2024-03-09T09:29:33.515060+0000","last_became_active":"2024-03-09T08:52:17.290985+0000","last_became_peered":"2024-03-09T08:52:17.290985+0000","last_unstale":"2024-03-09T09:29:33.515060+0000","last_undegraded":"2024-03-09T09:29:33.515060+0000","last_fullsized":"2024-03-09T09:29:33.515060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'393","last_scrub_stamp":"2024-03-09T09:29:33.514978+0000","last_deep_scrub":"89'393","last_deep_scrub_stamp":"2024-03-09T09:29:33.514978+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.514978+0000","objects_scrubbed":48,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:18:11.758277+0000","scrub_duration":104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":397,"num_write_kb":78578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":483803,"num_omap_keys":748,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"89'1750","reported_seq":2978,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.922756+0000","last_change":"2024-03-09T09:29:27.922756+0000","last_active":"2024-03-09T09:29:27.922756+0000","last_peered":"2024-03-09T09:29:27.922756+0000","last_clean":"2024-03-09T09:29:27.922756+0000","last_became_active":"2024-03-09T08:52:28.575064+0000","last_became_peered":"2024-03-09T08:52:28.575064+0000","last_unstale":"2024-03-09T09:29:27.922756+0000","last_undegraded":"2024-03-09T09:29:27.922756+0000","last_fullsized":"2024-03-09T09:29:27.922756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1750","last_scrub_stamp":"2024-03-09T09:29:27.922608+0000","last_deep_scrub":"89'1750","last_deep_scrub_stamp":"2024-03-09T09:29:27.922608+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.922608+0000","objects_scrubbed":50,"log_size":1750,"log_dups_size":0,"ondisk_log_size":1750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:05:35.558751+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":1117,"num_read_kb":4814,"num_write":1762,"num_write_kb":112708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":705687,"num_omap_keys":1069,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"89'2220","reported_seq":5185,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.789180+0000","last_change":"2024-03-09T09:29:41.789180+0000","last_active":"2024-03-09T09:29:41.789180+0000","last_peered":"2024-03-09T09:29:41.789180+0000","last_clean":"2024-03-09T09:29:41.789180+0000","last_became_active":"2024-03-09T08:52:29.576367+0000","last_became_peered":"2024-03-09T08:52:29.576367+0000","last_unstale":"2024-03-09T09:29:41.789180+0000","last_undegraded":"2024-03-09T09:29:41.789180+0000","last_fullsized":"2024-03-09T09:29:41.789180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2220","last_scrub_stamp":"2024-03-09T09:29:41.788983+0000","last_deep_scrub":"89'2220","last_deep_scrub_stamp":"2024-03-09T09:29:41.788983+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.788983+0000","objects_scrubbed":107,"log_size":2220,"log_dups_size":0,"ondisk_log_size":2220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:08:31.651496+0000","scrub_duration":156,"objects_trimmed":0,"snaptrim_duration":3.8297685179999998,"stat_sum":{"num_bytes":268964,"num_objects":107,"num_object_clones":32,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":32,"num_read":107,"num_read_kb":24622,"num_write":1287,"num_write_kb":41444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"89'2380","reported_seq":5441,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.772206+0000","last_change":"2024-03-09T09:29:40.772206+0000","last_active":"2024-03-09T09:29:40.772206+0000","last_peered":"2024-03-09T09:29:40.772206+0000","last_clean":"2024-03-09T09:29:40.772206+0000","last_became_active":"2024-03-09T08:53:00.240844+0000","last_became_peered":"2024-03-09T08:53:00.240844+0000","last_unstale":"2024-03-09T09:29:40.772206+0000","last_undegraded":"2024-03-09T09:29:40.772206+0000","last_fullsized":"2024-03-09T09:29:40.772206+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2380","last_scrub_stamp":"2024-03-09T09:29:40.771910+0000","last_deep_scrub":"89'2380","last_deep_scrub_stamp":"2024-03-09T09:29:40.771910+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.771910+0000","objects_scrubbed":126,"log_size":2380,"log_dups_size":0,"ondisk_log_size":2380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:27:55.399614+0000","scrub_duration":293,"objects_trimmed":0,"snaptrim_duration":0.78540855600000004,"stat_sum":{"num_bytes":1854558,"num_objects":126,"num_object_clones":46,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":46,"num_read":216,"num_read_kb":39239,"num_write":1332,"num_write_kb":71797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"89'1600","reported_seq":1779,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.020325+0000","last_change":"2024-03-09T09:29:35.020394+0000","last_active":"2024-03-09T09:29:35.020325+0000","last_peered":"2024-03-09T09:29:35.020325+0000","last_clean":"2024-03-09T09:29:35.020325+0000","last_became_active":"2024-03-09T08:52:17.280710+0000","last_became_peered":"2024-03-09T08:52:17.280710+0000","last_unstale":"2024-03-09T09:29:35.020325+0000","last_undegraded":"2024-03-09T09:29:35.020325+0000","last_fullsized":"2024-03-09T09:29:35.020325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1600","last_scrub_stamp":"2024-03-09T09:29:35.020224+0000","last_deep_scrub":"89'1600","last_deep_scrub_stamp":"2024-03-09T09:29:35.020224+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.020224+0000","objects_scrubbed":43,"log_size":1600,"log_dups_size":0,"ondisk_log_size":1600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:05:09.036176+0000","scrub_duration":140,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":57,"num_read_kb":0,"num_write":3420,"num_write_kb":84853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":559144,"num_omap_keys":941,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"89'726","reported_seq":898,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.482944+0000","last_change":"2024-03-09T09:29:31.483028+0000","last_active":"2024-03-09T09:29:31.482944+0000","last_peered":"2024-03-09T09:29:31.482944+0000","last_clean":"2024-03-09T09:29:31.482944+0000","last_became_active":"2024-03-09T08:52:28.586406+0000","last_became_peered":"2024-03-09T08:52:28.586406+0000","last_unstale":"2024-03-09T09:29:31.482944+0000","last_undegraded":"2024-03-09T09:29:31.482944+0000","last_fullsized":"2024-03-09T09:29:31.482944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'726","last_scrub_stamp":"2024-03-09T09:29:31.482759+0000","last_deep_scrub":"89'726","last_deep_scrub_stamp":"2024-03-09T09:29:31.482759+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.482759+0000","objects_scrubbed":54,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:10:04.708433+0000","scrub_duration":158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":71,"num_read_kb":0,"num_write":764,"num_write_kb":87842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":798407,"num_omap_keys":988,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"89'2473","reported_seq":5372,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.547368+0000","last_change":"2024-03-09T09:29:42.547368+0000","last_active":"2024-03-09T09:29:42.547368+0000","last_peered":"2024-03-09T09:29:42.547368+0000","last_clean":"2024-03-09T09:29:42.547368+0000","last_became_active":"2024-03-09T08:52:29.578324+0000","last_became_peered":"2024-03-09T08:52:29.578324+0000","last_unstale":"2024-03-09T09:29:42.547368+0000","last_undegraded":"2024-03-09T09:29:42.547368+0000","last_fullsized":"2024-03-09T09:29:42.547368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2473","last_scrub_stamp":"2024-03-09T09:29:42.547184+0000","last_deep_scrub":"89'2473","last_deep_scrub_stamp":"2024-03-09T09:29:42.547184+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.547184+0000","objects_scrubbed":134,"log_size":2473,"log_dups_size":0,"ondisk_log_size":2473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:43:09.396144+0000","scrub_duration":238,"objects_trimmed":0,"snaptrim_duration":0.65806592200000003,"stat_sum":{"num_bytes":2259601,"num_objects":134,"num_object_clones":47,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":47,"num_read":108,"num_read_kb":22295,"num_write":1398,"num_write_kb":43486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"89'2292","reported_seq":5303,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.171649+0000","last_change":"2024-03-09T09:29:30.171649+0000","last_active":"2024-03-09T09:29:30.171649+0000","last_peered":"2024-03-09T09:29:30.171649+0000","last_clean":"2024-03-09T09:29:30.171649+0000","last_became_active":"2024-03-09T08:52:18.234524+0000","last_became_peered":"2024-03-09T08:52:18.234524+0000","last_unstale":"2024-03-09T09:29:30.171649+0000","last_undegraded":"2024-03-09T09:29:30.171649+0000","last_fullsized":"2024-03-09T09:29:30.171649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2292","last_scrub_stamp":"2024-03-09T09:29:30.171498+0000","last_deep_scrub":"89'2292","last_deep_scrub_stamp":"2024-03-09T09:29:30.171498+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.171498+0000","objects_scrubbed":111,"log_size":2292,"log_dups_size":0,"ondisk_log_size":2292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:16:25.210594+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0.96663918400000004,"stat_sum":{"num_bytes":2499803,"num_objects":111,"num_object_clones":36,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":36,"num_read":112,"num_read_kb":5934,"num_write":1320,"num_write_kb":25592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"89'694","reported_seq":875,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.354649+0000","last_change":"2024-03-09T09:29:46.354649+0000","last_active":"2024-03-09T09:29:46.354649+0000","last_peered":"2024-03-09T09:29:46.354649+0000","last_clean":"2024-03-09T09:29:46.354649+0000","last_became_active":"2024-03-09T08:52:17.296421+0000","last_became_peered":"2024-03-09T08:52:17.296421+0000","last_unstale":"2024-03-09T09:29:46.354649+0000","last_undegraded":"2024-03-09T09:29:46.354649+0000","last_fullsized":"2024-03-09T09:29:46.354649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'694","last_scrub_stamp":"2024-03-09T09:29:46.354505+0000","last_deep_scrub":"89'694","last_deep_scrub_stamp":"2024-03-09T09:29:46.354505+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.354505+0000","objects_scrubbed":52,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:43:19.678339+0000","scrub_duration":136,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":75,"num_read_kb":0,"num_write":726,"num_write_kb":96134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":830736,"num_omap_keys":1184,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"89'617","reported_seq":789,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.084599+0000","last_change":"2024-03-09T09:29:36.084599+0000","last_active":"2024-03-09T09:29:36.084599+0000","last_peered":"2024-03-09T09:29:36.084599+0000","last_clean":"2024-03-09T09:29:36.084599+0000","last_became_active":"2024-03-09T08:52:28.580534+0000","last_became_peered":"2024-03-09T08:52:28.580534+0000","last_unstale":"2024-03-09T09:29:36.084599+0000","last_undegraded":"2024-03-09T09:29:36.084599+0000","last_fullsized":"2024-03-09T09:29:36.084599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'617","last_scrub_stamp":"2024-03-09T09:29:36.084430+0000","last_deep_scrub":"89'617","last_deep_scrub_stamp":"2024-03-09T09:29:36.084430+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.084430+0000","objects_scrubbed":33,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:59:55.329284+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":626,"num_write_kb":74740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":361223,"num_omap_keys":599,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"89'2400","reported_seq":5313,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.264915+0000","last_change":"2024-03-09T09:29:39.264915+0000","last_active":"2024-03-09T09:29:39.264915+0000","last_peered":"2024-03-09T09:29:39.264915+0000","last_clean":"2024-03-09T09:29:39.264915+0000","last_became_active":"2024-03-09T08:52:29.579523+0000","last_became_peered":"2024-03-09T08:52:29.579523+0000","last_unstale":"2024-03-09T09:29:39.264915+0000","last_undegraded":"2024-03-09T09:29:39.264915+0000","last_fullsized":"2024-03-09T09:29:39.264915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2400","last_scrub_stamp":"2024-03-09T09:29:39.264692+0000","last_deep_scrub":"89'2400","last_deep_scrub_stamp":"2024-03-09T09:29:39.264692+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.264692+0000","objects_scrubbed":126,"log_size":2400,"log_dups_size":0,"ondisk_log_size":2400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:31:01.872524+0000","scrub_duration":253,"objects_trimmed":0,"snaptrim_duration":0.71811667999999995,"stat_sum":{"num_bytes":1846155,"num_objects":126,"num_object_clones":42,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":42,"num_read":111,"num_read_kb":21516,"num_write":1347,"num_write_kb":38338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"89'2389","reported_seq":5350,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.168355+0000","last_change":"2024-03-09T09:29:27.168355+0000","last_active":"2024-03-09T09:29:27.168355+0000","last_peered":"2024-03-09T09:29:27.168355+0000","last_clean":"2024-03-09T09:29:27.168355+0000","last_became_active":"2024-03-09T08:52:18.249634+0000","last_became_peered":"2024-03-09T08:52:18.249634+0000","last_unstale":"2024-03-09T09:29:27.168355+0000","last_undegraded":"2024-03-09T09:29:27.168355+0000","last_fullsized":"2024-03-09T09:29:27.168355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2389","last_scrub_stamp":"2024-03-09T09:29:27.168219+0000","last_deep_scrub":"89'2389","last_deep_scrub_stamp":"2024-03-09T09:29:27.168219+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.168219+0000","objects_scrubbed":105,"log_size":2389,"log_dups_size":0,"ondisk_log_size":2389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:07:32.466927+0000","scrub_duration":328,"objects_trimmed":0,"snaptrim_duration":1.815577539,"stat_sum":{"num_bytes":1586992,"num_objects":105,"num_object_clones":35,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":35,"num_read":97,"num_read_kb":33815,"num_write":1338,"num_write_kb":74580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"89'594","reported_seq":774,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.056459+0000","last_change":"2024-03-09T09:29:34.056459+0000","last_active":"2024-03-09T09:29:34.056459+0000","last_peered":"2024-03-09T09:29:34.056459+0000","last_clean":"2024-03-09T09:29:34.056459+0000","last_became_active":"2024-03-09T08:52:17.274888+0000","last_became_peered":"2024-03-09T08:52:17.274888+0000","last_unstale":"2024-03-09T09:29:34.056459+0000","last_undegraded":"2024-03-09T09:29:34.056459+0000","last_fullsized":"2024-03-09T09:29:34.056459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'594","last_scrub_stamp":"2024-03-09T09:29:34.056310+0000","last_deep_scrub":"89'594","last_deep_scrub_stamp":"2024-03-09T09:29:34.056310+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.056310+0000","objects_scrubbed":48,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:43:54.072133+0000","scrub_duration":136,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":620,"num_write_kb":120324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":674477,"num_omap_keys":1093,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"89'590","reported_seq":762,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.201654+0000","last_change":"2024-03-09T09:29:29.201654+0000","last_active":"2024-03-09T09:29:29.201654+0000","last_peered":"2024-03-09T09:29:29.201654+0000","last_clean":"2024-03-09T09:29:29.201654+0000","last_became_active":"2024-03-09T08:52:28.584075+0000","last_became_peered":"2024-03-09T08:52:28.584075+0000","last_unstale":"2024-03-09T09:29:29.201654+0000","last_undegraded":"2024-03-09T09:29:29.201654+0000","last_fullsized":"2024-03-09T09:29:29.201654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'590","last_scrub_stamp":"2024-03-09T09:29:29.201403+0000","last_deep_scrub":"89'590","last_deep_scrub_stamp":"2024-03-09T09:29:29.201403+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.201403+0000","objects_scrubbed":38,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:26:31.012880+0000","scrub_duration":156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":563,"num_write_kb":103368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":328063,"num_omap_keys":503,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"89'2366","reported_seq":5280,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.144804+0000","last_change":"2024-03-09T09:29:42.144804+0000","last_active":"2024-03-09T09:29:42.144804+0000","last_peered":"2024-03-09T09:29:42.144804+0000","last_clean":"2024-03-09T09:29:42.144804+0000","last_became_active":"2024-03-09T08:52:29.596142+0000","last_became_peered":"2024-03-09T08:52:29.596142+0000","last_unstale":"2024-03-09T09:29:42.144804+0000","last_undegraded":"2024-03-09T09:29:42.144804+0000","last_fullsized":"2024-03-09T09:29:42.144804+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2366","last_scrub_stamp":"2024-03-09T09:29:42.144694+0000","last_deep_scrub":"89'2366","last_deep_scrub_stamp":"2024-03-09T09:29:42.144694+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.144694+0000","objects_scrubbed":101,"log_size":2366,"log_dups_size":0,"ondisk_log_size":2366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:44.531453+0000","scrub_duration":177,"objects_trimmed":4,"snaptrim_duration":0.71599741500000003,"stat_sum":{"num_bytes":1873672,"num_objects":101,"num_object_clones":34,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":34,"num_read":103,"num_read_kb":19624,"num_write":1351,"num_write_kb":43427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"89'2388","reported_seq":5379,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.908917+0000","last_change":"2024-03-09T09:29:34.908917+0000","last_active":"2024-03-09T09:29:34.908917+0000","last_peered":"2024-03-09T09:29:34.908917+0000","last_clean":"2024-03-09T09:29:34.908917+0000","last_became_active":"2024-03-09T08:52:18.227044+0000","last_became_peered":"2024-03-09T08:52:18.227044+0000","last_unstale":"2024-03-09T09:29:34.908917+0000","last_undegraded":"2024-03-09T09:29:34.908917+0000","last_fullsized":"2024-03-09T09:29:34.908917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2388","last_scrub_stamp":"2024-03-09T09:29:34.908766+0000","last_deep_scrub":"89'2388","last_deep_scrub_stamp":"2024-03-09T09:29:34.908766+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.908766+0000","objects_scrubbed":128,"log_size":2388,"log_dups_size":0,"ondisk_log_size":2388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:46:15.676520+0000","scrub_duration":201,"objects_trimmed":0,"snaptrim_duration":3.4919903310000002,"stat_sum":{"num_bytes":2681803,"num_objects":128,"num_object_clones":43,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":43,"num_read":104,"num_read_kb":25781,"num_write":1386,"num_write_kb":55300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"89'789","reported_seq":970,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.046778+0000","last_change":"2024-03-09T09:29:41.046778+0000","last_active":"2024-03-09T09:29:41.046778+0000","last_peered":"2024-03-09T09:29:41.046778+0000","last_clean":"2024-03-09T09:29:41.046778+0000","last_became_active":"2024-03-09T08:52:17.280431+0000","last_became_peered":"2024-03-09T08:52:17.280431+0000","last_unstale":"2024-03-09T09:29:41.046778+0000","last_undegraded":"2024-03-09T09:29:41.046778+0000","last_fullsized":"2024-03-09T09:29:41.046778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'789","last_scrub_stamp":"2024-03-09T09:29:41.046618+0000","last_deep_scrub":"89'789","last_deep_scrub_stamp":"2024-03-09T09:29:41.046618+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.046618+0000","objects_scrubbed":53,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:35.300265+0000","scrub_duration":202,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":833,"num_write_kb":103821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":595568,"num_omap_keys":1030,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'609","reported_seq":781,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.150380+0000","last_change":"2024-03-09T09:29:25.150380+0000","last_active":"2024-03-09T09:29:25.150380+0000","last_peered":"2024-03-09T09:29:25.150380+0000","last_clean":"2024-03-09T09:29:25.150380+0000","last_became_active":"2024-03-09T08:52:28.577747+0000","last_became_peered":"2024-03-09T08:52:28.577747+0000","last_unstale":"2024-03-09T09:29:25.150380+0000","last_undegraded":"2024-03-09T09:29:25.150380+0000","last_fullsized":"2024-03-09T09:29:25.150380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'609","last_scrub_stamp":"2024-03-09T09:29:25.150212+0000","last_deep_scrub":"89'609","last_deep_scrub_stamp":"2024-03-09T09:29:25.150212+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.150212+0000","objects_scrubbed":48,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:04:01.814051+0000","scrub_duration":112,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":52,"num_read_kb":0,"num_write":616,"num_write_kb":119874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":438501,"num_omap_keys":702,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"89'2276","reported_seq":5223,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:24.750759+0000","last_change":"2024-03-09T09:29:24.750805+0000","last_active":"2024-03-09T09:29:24.750759+0000","last_peered":"2024-03-09T09:29:24.750759+0000","last_clean":"2024-03-09T09:29:24.750759+0000","last_became_active":"2024-03-09T08:52:29.590517+0000","last_became_peered":"2024-03-09T08:52:29.590517+0000","last_unstale":"2024-03-09T09:29:24.750759+0000","last_undegraded":"2024-03-09T09:29:24.750759+0000","last_fullsized":"2024-03-09T09:29:24.750759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2276","last_scrub_stamp":"2024-03-09T09:29:24.750679+0000","last_deep_scrub":"89'2276","last_deep_scrub_stamp":"2024-03-09T09:29:24.750679+0000","last_clean_scrub_stamp":"2024-03-09T09:29:24.750679+0000","objects_scrubbed":110,"log_size":2276,"log_dups_size":0,"ondisk_log_size":2276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:41:28.516964+0000","scrub_duration":247,"objects_trimmed":2,"snaptrim_duration":0.63049624100000001,"stat_sum":{"num_bytes":833172,"num_objects":110,"num_object_clones":40,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":40,"num_read":114,"num_read_kb":17857,"num_write":1302,"num_write_kb":44488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"89'2367","reported_seq":5356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.063265+0000","last_change":"2024-03-09T09:29:40.063339+0000","last_active":"2024-03-09T09:29:40.063265+0000","last_peered":"2024-03-09T09:29:40.063265+0000","last_clean":"2024-03-09T09:29:40.063265+0000","last_became_active":"2024-03-09T08:52:18.235486+0000","last_became_peered":"2024-03-09T08:52:18.235486+0000","last_unstale":"2024-03-09T09:29:40.063265+0000","last_undegraded":"2024-03-09T09:29:40.063265+0000","last_fullsized":"2024-03-09T09:29:40.063265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2367","last_scrub_stamp":"2024-03-09T09:29:40.063175+0000","last_deep_scrub":"89'2367","last_deep_scrub_stamp":"2024-03-09T09:29:40.063175+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.063175+0000","objects_scrubbed":117,"log_size":2367,"log_dups_size":0,"ondisk_log_size":2367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:42:37.267229+0000","scrub_duration":194,"objects_trimmed":0,"snaptrim_duration":1.18447864,"stat_sum":{"num_bytes":1122276,"num_objects":117,"num_object_clones":43,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":43,"num_read":101,"num_read_kb":18409,"num_write":1364,"num_write_kb":39400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"89'1869","reported_seq":3064,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.624152+0000","last_change":"2024-03-09T09:29:26.624152+0000","last_active":"2024-03-09T09:29:26.624152+0000","last_peered":"2024-03-09T09:29:26.624152+0000","last_clean":"2024-03-09T09:29:26.624152+0000","last_became_active":"2024-03-09T08:52:17.270671+0000","last_became_peered":"2024-03-09T08:52:17.270671+0000","last_unstale":"2024-03-09T09:29:26.624152+0000","last_undegraded":"2024-03-09T09:29:26.624152+0000","last_fullsized":"2024-03-09T09:29:26.624152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1869","last_scrub_stamp":"2024-03-09T09:29:26.624016+0000","last_deep_scrub":"89'1869","last_deep_scrub_stamp":"2024-03-09T09:29:26.624016+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.624016+0000","objects_scrubbed":39,"log_size":1869,"log_dups_size":0,"ondisk_log_size":1869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:42:29.163043+0000","scrub_duration":102,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":1037,"num_read_kb":4764,"num_write":1840,"num_write_kb":108217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":338965,"num_omap_keys":539,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"89'441","reported_seq":612,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.080999+0000","last_change":"2024-03-09T09:29:31.081118+0000","last_active":"2024-03-09T09:29:31.080999+0000","last_peered":"2024-03-09T09:29:31.080999+0000","last_clean":"2024-03-09T09:29:31.080999+0000","last_became_active":"2024-03-09T08:52:28.585947+0000","last_became_peered":"2024-03-09T08:52:28.585947+0000","last_unstale":"2024-03-09T09:29:31.080999+0000","last_undegraded":"2024-03-09T09:29:31.080999+0000","last_fullsized":"2024-03-09T09:29:31.080999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'441","last_scrub_stamp":"2024-03-09T09:29:31.080900+0000","last_deep_scrub":"89'441","last_deep_scrub_stamp":"2024-03-09T09:29:31.080900+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.080900+0000","objects_scrubbed":37,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:15:41.261309+0000","scrub_duration":127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":44,"num_read_kb":0,"num_write":445,"num_write_kb":78896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":431086,"num_omap_keys":609,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"89'2307","reported_seq":5264,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:54.222346+0000","last_change":"2024-03-09T09:29:54.222371+0000","last_active":"2024-03-09T09:29:54.222346+0000","last_peered":"2024-03-09T09:29:54.222346+0000","last_clean":"2024-03-09T09:29:54.222346+0000","last_became_active":"2024-03-09T08:52:29.606530+0000","last_became_peered":"2024-03-09T08:52:29.606530+0000","last_unstale":"2024-03-09T09:29:54.222346+0000","last_undegraded":"2024-03-09T09:29:54.222346+0000","last_fullsized":"2024-03-09T09:29:54.222346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2307","last_scrub_stamp":"2024-03-09T09:29:54.222278+0000","last_deep_scrub":"89'2307","last_deep_scrub_stamp":"2024-03-09T09:29:54.222278+0000","last_clean_scrub_stamp":"2024-03-09T09:29:54.222278+0000","objects_scrubbed":123,"log_size":2307,"log_dups_size":0,"ondisk_log_size":2307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:15:53.907954+0000","scrub_duration":314,"objects_trimmed":0,"snaptrim_duration":4.4843044489999997,"stat_sum":{"num_bytes":1636574,"num_objects":123,"num_object_clones":44,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":44,"num_read":134,"num_read_kb":27719,"num_write":1302,"num_write_kb":59461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'2240","reported_seq":5247,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.183008+0000","last_change":"2024-03-09T09:29:30.183008+0000","last_active":"2024-03-09T09:29:30.183008+0000","last_peered":"2024-03-09T09:29:30.183008+0000","last_clean":"2024-03-09T09:29:30.183008+0000","last_became_active":"2024-03-09T08:52:18.213624+0000","last_became_peered":"2024-03-09T08:52:18.213624+0000","last_unstale":"2024-03-09T09:29:30.183008+0000","last_undegraded":"2024-03-09T09:29:30.183008+0000","last_fullsized":"2024-03-09T09:29:30.183008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2240","last_scrub_stamp":"2024-03-09T09:29:30.182824+0000","last_deep_scrub":"89'2240","last_deep_scrub_stamp":"2024-03-09T09:29:30.182824+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.182824+0000","objects_scrubbed":113,"log_size":2240,"log_dups_size":0,"ondisk_log_size":2240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:52:59.854784+0000","scrub_duration":187,"objects_trimmed":0,"snaptrim_duration":0.289202336,"stat_sum":{"num_bytes":424328,"num_objects":113,"num_object_clones":35,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":35,"num_read":94,"num_read_kb":17266,"num_write":1284,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"89'615","reported_seq":795,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:47.292509+0000","last_change":"2024-03-09T09:29:47.292559+0000","last_active":"2024-03-09T09:29:47.292509+0000","last_peered":"2024-03-09T09:29:47.292509+0000","last_clean":"2024-03-09T09:29:47.292509+0000","last_became_active":"2024-03-09T08:52:17.297753+0000","last_became_peered":"2024-03-09T08:52:17.297753+0000","last_unstale":"2024-03-09T09:29:47.292509+0000","last_undegraded":"2024-03-09T09:29:47.292509+0000","last_fullsized":"2024-03-09T09:29:47.292509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'615","last_scrub_stamp":"2024-03-09T09:29:47.292418+0000","last_deep_scrub":"89'615","last_deep_scrub_stamp":"2024-03-09T09:29:47.292418+0000","last_clean_scrub_stamp":"2024-03-09T09:29:47.292418+0000","objects_scrubbed":35,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:49:55.705231+0000","scrub_duration":76,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":29,"num_read_kb":0,"num_write":598,"num_write_kb":82993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":415266,"num_omap_keys":726,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"89'1635","reported_seq":1806,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.112003+0000","last_change":"2024-03-09T09:29:26.112003+0000","last_active":"2024-03-09T09:29:26.112003+0000","last_peered":"2024-03-09T09:29:26.112003+0000","last_clean":"2024-03-09T09:29:26.112003+0000","last_became_active":"2024-03-09T08:52:28.578235+0000","last_became_peered":"2024-03-09T08:52:28.578235+0000","last_unstale":"2024-03-09T09:29:26.112003+0000","last_undegraded":"2024-03-09T09:29:26.112003+0000","last_fullsized":"2024-03-09T09:29:26.112003+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1634","last_scrub_stamp":"2024-03-09T09:29:26.111801+0000","last_deep_scrub":"89'1634","last_deep_scrub_stamp":"2024-03-09T09:29:26.111801+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.111801+0000","objects_scrubbed":42,"log_size":1635,"log_dups_size":0,"ondisk_log_size":1635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:02:01.981447+0000","scrub_duration":129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25241505,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":3476,"num_write_kb":84820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":357598,"num_omap_keys":628,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"89'2357","reported_seq":5302,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:50.358047+0000","last_change":"2024-03-09T09:29:50.358047+0000","last_active":"2024-03-09T09:29:50.358047+0000","last_peered":"2024-03-09T09:29:50.358047+0000","last_clean":"2024-03-09T09:29:50.358047+0000","last_became_active":"2024-03-09T08:52:29.591607+0000","last_became_peered":"2024-03-09T08:52:29.591607+0000","last_unstale":"2024-03-09T09:29:50.358047+0000","last_undegraded":"2024-03-09T09:29:50.358047+0000","last_fullsized":"2024-03-09T09:29:50.358047+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2357","last_scrub_stamp":"2024-03-09T09:29:50.357806+0000","last_deep_scrub":"89'2357","last_deep_scrub_stamp":"2024-03-09T09:29:50.357806+0000","last_clean_scrub_stamp":"2024-03-09T09:29:50.357806+0000","objects_scrubbed":121,"log_size":2357,"log_dups_size":0,"ondisk_log_size":2357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:29:38.056079+0000","scrub_duration":189,"objects_trimmed":2,"snaptrim_duration":4.4503168610000001,"stat_sum":{"num_bytes":1665642,"num_objects":121,"num_object_clones":41,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":41,"num_read":126,"num_read_kb":42837,"num_write":1341,"num_write_kb":67584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'2445","reported_seq":5421,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.554842+0000","last_change":"2024-03-09T09:29:45.554842+0000","last_active":"2024-03-09T09:29:45.554842+0000","last_peered":"2024-03-09T09:29:45.554842+0000","last_clean":"2024-03-09T09:29:45.554842+0000","last_became_active":"2024-03-09T08:52:18.233465+0000","last_became_peered":"2024-03-09T08:52:18.233465+0000","last_unstale":"2024-03-09T09:29:45.554842+0000","last_undegraded":"2024-03-09T09:29:45.554842+0000","last_fullsized":"2024-03-09T09:29:45.554842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2445","last_scrub_stamp":"2024-03-09T09:29:45.554681+0000","last_deep_scrub":"89'2445","last_deep_scrub_stamp":"2024-03-09T09:29:45.554681+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.554681+0000","objects_scrubbed":137,"log_size":2445,"log_dups_size":0,"ondisk_log_size":2445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:29:22.203097+0000","scrub_duration":302,"objects_trimmed":0,"snaptrim_duration":0.56599842099999997,"stat_sum":{"num_bytes":1847778,"num_objects":137,"num_object_clones":49,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":49,"num_read":115,"num_read_kb":13750,"num_write":1371,"num_write_kb":55061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'665","reported_seq":845,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.902079+0000","last_change":"2024-03-09T09:29:39.902079+0000","last_active":"2024-03-09T09:29:39.902079+0000","last_peered":"2024-03-09T09:29:39.902079+0000","last_clean":"2024-03-09T09:29:39.902079+0000","last_became_active":"2024-03-09T08:52:17.277052+0000","last_became_peered":"2024-03-09T08:52:17.277052+0000","last_unstale":"2024-03-09T09:29:39.902079+0000","last_undegraded":"2024-03-09T09:29:39.902079+0000","last_fullsized":"2024-03-09T09:29:39.902079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'665","last_scrub_stamp":"2024-03-09T09:29:39.901928+0000","last_deep_scrub":"89'665","last_deep_scrub_stamp":"2024-03-09T09:29:39.901928+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.901928+0000","objects_scrubbed":39,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:49:03.340420+0000","scrub_duration":52,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":40,"num_read_kb":0,"num_write":698,"num_write_kb":70663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":499002,"num_omap_keys":859,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"89'119","reported_seq":337,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.183267+0000","last_change":"2024-03-09T09:29:41.183293+0000","last_active":"2024-03-09T09:29:41.183267+0000","last_peered":"2024-03-09T09:29:41.183267+0000","last_clean":"2024-03-09T09:29:41.183267+0000","last_became_active":"2024-03-09T08:52:08.578305+0000","last_became_peered":"2024-03-09T08:52:08.578305+0000","last_unstale":"2024-03-09T09:29:41.183267+0000","last_undegraded":"2024-03-09T09:29:41.183267+0000","last_fullsized":"2024-03-09T09:29:41.183267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'119","last_scrub_stamp":"2024-03-09T09:29:41.183205+0000","last_deep_scrub":"89'119","last_deep_scrub_stamp":"2024-03-09T09:29:41.183205+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.183205+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:10:29.103603+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'726","reported_seq":898,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.473372+0000","last_change":"2024-03-09T09:29:39.473372+0000","last_active":"2024-03-09T09:29:39.473372+0000","last_peered":"2024-03-09T09:29:39.473372+0000","last_clean":"2024-03-09T09:29:39.473372+0000","last_became_active":"2024-03-09T08:52:28.577524+0000","last_became_peered":"2024-03-09T08:52:28.577524+0000","last_unstale":"2024-03-09T09:29:39.473372+0000","last_undegraded":"2024-03-09T09:29:39.473372+0000","last_fullsized":"2024-03-09T09:29:39.473372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'726","last_scrub_stamp":"2024-03-09T09:29:39.473216+0000","last_deep_scrub":"89'726","last_deep_scrub_stamp":"2024-03-09T09:29:39.473216+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.473216+0000","objects_scrubbed":46,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:30:31.060980+0000","scrub_duration":110,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":743,"num_write_kb":95766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":577964,"num_omap_keys":893,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"89'2354","reported_seq":5295,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.733098+0000","last_change":"2024-03-09T09:29:35.733160+0000","last_active":"2024-03-09T09:29:35.733098+0000","last_peered":"2024-03-09T09:29:35.733098+0000","last_clean":"2024-03-09T09:29:35.733098+0000","last_became_active":"2024-03-09T08:52:29.598870+0000","last_became_peered":"2024-03-09T08:52:29.598870+0000","last_unstale":"2024-03-09T09:29:35.733098+0000","last_undegraded":"2024-03-09T09:29:35.733098+0000","last_fullsized":"2024-03-09T09:29:35.733098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2354","last_scrub_stamp":"2024-03-09T09:29:35.733000+0000","last_deep_scrub":"89'2354","last_deep_scrub_stamp":"2024-03-09T09:29:35.733000+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.733000+0000","objects_scrubbed":106,"log_size":2354,"log_dups_size":0,"ondisk_log_size":2354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:51:37.891297+0000","scrub_duration":155,"objects_trimmed":4,"snaptrim_duration":0.66264078000000004,"stat_sum":{"num_bytes":792938,"num_objects":106,"num_object_clones":38,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":38,"num_read":125,"num_read_kb":17823,"num_write":1366,"num_write_kb":27629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'2357","reported_seq":5314,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.688984+0000","last_change":"2024-03-09T09:29:41.689028+0000","last_active":"2024-03-09T09:29:41.688984+0000","last_peered":"2024-03-09T09:29:41.688984+0000","last_clean":"2024-03-09T09:29:41.688984+0000","last_became_active":"2024-03-09T08:53:00.437859+0000","last_became_peered":"2024-03-09T08:53:00.437859+0000","last_unstale":"2024-03-09T09:29:41.688984+0000","last_undegraded":"2024-03-09T09:29:41.688984+0000","last_fullsized":"2024-03-09T09:29:41.688984+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2357","last_scrub_stamp":"2024-03-09T09:29:41.688920+0000","last_deep_scrub":"89'2357","last_deep_scrub_stamp":"2024-03-09T09:29:41.688920+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.688920+0000","objects_scrubbed":115,"log_size":2357,"log_dups_size":0,"ondisk_log_size":2357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:28:30.276654+0000","scrub_duration":224,"objects_trimmed":0,"snaptrim_duration":0.79260830400000004,"stat_sum":{"num_bytes":1456716,"num_objects":115,"num_object_clones":40,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":40,"num_read":109,"num_read_kb":18518,"num_write":1301,"num_write_kb":37816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"89'547","reported_seq":725,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.610930+0000","last_change":"2024-03-09T09:29:29.611054+0000","last_active":"2024-03-09T09:29:29.610930+0000","last_peered":"2024-03-09T09:29:29.610930+0000","last_clean":"2024-03-09T09:29:29.610930+0000","last_became_active":"2024-03-09T08:52:17.283285+0000","last_became_peered":"2024-03-09T08:52:17.283285+0000","last_unstale":"2024-03-09T09:29:29.610930+0000","last_undegraded":"2024-03-09T09:29:29.610930+0000","last_fullsized":"2024-03-09T09:29:29.610930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'547","last_scrub_stamp":"2024-03-09T09:29:29.610727+0000","last_deep_scrub":"89'547","last_deep_scrub_stamp":"2024-03-09T09:29:29.610727+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.610727+0000","objects_scrubbed":46,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:29:15.793829+0000","scrub_duration":161,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":39,"num_read_kb":0,"num_write":538,"num_write_kb":99417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":457181,"num_omap_keys":720,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"89'758","reported_seq":930,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.479227+0000","last_change":"2024-03-09T09:29:41.479258+0000","last_active":"2024-03-09T09:29:41.479227+0000","last_peered":"2024-03-09T09:29:41.479227+0000","last_clean":"2024-03-09T09:29:41.479227+0000","last_became_active":"2024-03-09T08:52:28.588831+0000","last_became_peered":"2024-03-09T08:52:28.588831+0000","last_unstale":"2024-03-09T09:29:41.479227+0000","last_undegraded":"2024-03-09T09:29:41.479227+0000","last_fullsized":"2024-03-09T09:29:41.479227+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'758","last_scrub_stamp":"2024-03-09T09:29:41.479164+0000","last_deep_scrub":"89'758","last_deep_scrub_stamp":"2024-03-09T09:29:41.479164+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.479164+0000","objects_scrubbed":55,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:36:20.205971+0000","scrub_duration":156,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":780,"num_write_kb":116155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":563342,"num_omap_keys":842,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"89'2292","reported_seq":5252,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.732442+0000","last_change":"2024-03-09T09:29:37.732525+0000","last_active":"2024-03-09T09:29:37.732442+0000","last_peered":"2024-03-09T09:29:37.732442+0000","last_clean":"2024-03-09T09:29:37.732442+0000","last_became_active":"2024-03-09T08:52:29.597669+0000","last_became_peered":"2024-03-09T08:52:29.597669+0000","last_unstale":"2024-03-09T09:29:37.732442+0000","last_undegraded":"2024-03-09T09:29:37.732442+0000","last_fullsized":"2024-03-09T09:29:37.732442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2292","last_scrub_stamp":"2024-03-09T09:29:37.732276+0000","last_deep_scrub":"89'2292","last_deep_scrub_stamp":"2024-03-09T09:29:37.732276+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.732276+0000","objects_scrubbed":92,"log_size":2292,"log_dups_size":0,"ondisk_log_size":2292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:37:49.725086+0000","scrub_duration":199,"objects_trimmed":2,"snaptrim_duration":0.647865472,"stat_sum":{"num_bytes":999257,"num_objects":92,"num_object_clones":32,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":32,"num_read":116,"num_read_kb":42999,"num_write":1346,"num_write_kb":55653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"89'2284","reported_seq":5290,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.156522+0000","last_change":"2024-03-09T09:29:28.156605+0000","last_active":"2024-03-09T09:29:28.156522+0000","last_peered":"2024-03-09T09:29:28.156522+0000","last_clean":"2024-03-09T09:29:28.156522+0000","last_became_active":"2024-03-09T08:52:18.244878+0000","last_became_peered":"2024-03-09T08:52:18.244878+0000","last_unstale":"2024-03-09T09:29:28.156522+0000","last_undegraded":"2024-03-09T09:29:28.156522+0000","last_fullsized":"2024-03-09T09:29:28.156522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2284","last_scrub_stamp":"2024-03-09T09:29:28.156442+0000","last_deep_scrub":"89'2284","last_deep_scrub_stamp":"2024-03-09T09:29:28.156442+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.156442+0000","objects_scrubbed":97,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:48:56.755552+0000","scrub_duration":364,"objects_trimmed":0,"snaptrim_duration":1.951760857,"stat_sum":{"num_bytes":1333229,"num_objects":97,"num_object_clones":33,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":33,"num_read":100,"num_read_kb":17450,"num_write":1313,"num_write_kb":41729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"89'888","reported_seq":1066,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.953183+0000","last_change":"2024-03-09T09:29:37.953238+0000","last_active":"2024-03-09T09:29:37.953183+0000","last_peered":"2024-03-09T09:29:37.953183+0000","last_clean":"2024-03-09T09:29:37.953183+0000","last_became_active":"2024-03-09T08:52:17.286400+0000","last_became_peered":"2024-03-09T08:52:17.286400+0000","last_unstale":"2024-03-09T09:29:37.953183+0000","last_undegraded":"2024-03-09T09:29:37.953183+0000","last_fullsized":"2024-03-09T09:29:37.953183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'888","last_scrub_stamp":"2024-03-09T09:29:37.953099+0000","last_deep_scrub":"89'888","last_deep_scrub_stamp":"2024-03-09T09:29:37.953099+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.953099+0000","objects_scrubbed":43,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:06:19.595033+0000","scrub_duration":131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971641,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":912,"num_write_kb":111865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":576689,"num_omap_keys":942,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"89'842","reported_seq":1012,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.077990+0000","last_change":"2024-03-09T09:29:37.078044+0000","last_active":"2024-03-09T09:29:37.077990+0000","last_peered":"2024-03-09T09:29:37.077990+0000","last_clean":"2024-03-09T09:29:37.077990+0000","last_became_active":"2024-03-09T08:52:28.582850+0000","last_became_peered":"2024-03-09T08:52:28.582850+0000","last_unstale":"2024-03-09T09:29:37.077990+0000","last_undegraded":"2024-03-09T09:29:37.077990+0000","last_fullsized":"2024-03-09T09:29:37.077990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'842","last_scrub_stamp":"2024-03-09T09:29:37.077914+0000","last_deep_scrub":"89'842","last_deep_scrub_stamp":"2024-03-09T09:29:37.077914+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.077914+0000","objects_scrubbed":46,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:20:48.969270+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":51,"num_read_kb":0,"num_write":872,"num_write_kb":99652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":580759,"num_omap_keys":912,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"89'2337","reported_seq":5246,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.409316+0000","last_change":"2024-03-09T09:29:37.409360+0000","last_active":"2024-03-09T09:29:37.409316+0000","last_peered":"2024-03-09T09:29:37.409316+0000","last_clean":"2024-03-09T09:29:37.409316+0000","last_became_active":"2024-03-09T08:52:29.589846+0000","last_became_peered":"2024-03-09T08:52:29.589846+0000","last_unstale":"2024-03-09T09:29:37.409316+0000","last_undegraded":"2024-03-09T09:29:37.409316+0000","last_fullsized":"2024-03-09T09:29:37.409316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2337","last_scrub_stamp":"2024-03-09T09:29:37.409205+0000","last_deep_scrub":"89'2337","last_deep_scrub_stamp":"2024-03-09T09:29:37.409205+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.409205+0000","objects_scrubbed":100,"log_size":2337,"log_dups_size":0,"ondisk_log_size":2337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:14:29.152094+0000","scrub_duration":183,"objects_trimmed":2,"snaptrim_duration":4.4098675719999996,"stat_sum":{"num_bytes":603203,"num_objects":100,"num_object_clones":32,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":32,"num_read":95,"num_read_kb":13936,"num_write":1338,"num_write_kb":41727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"89'2316","reported_seq":5322,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.413234+0000","last_change":"2024-03-09T09:29:31.413310+0000","last_active":"2024-03-09T09:29:31.413234+0000","last_peered":"2024-03-09T09:29:31.413234+0000","last_clean":"2024-03-09T09:29:31.413234+0000","last_became_active":"2024-03-09T08:52:18.249915+0000","last_became_peered":"2024-03-09T08:52:18.249915+0000","last_unstale":"2024-03-09T09:29:31.413234+0000","last_undegraded":"2024-03-09T09:29:31.413234+0000","last_fullsized":"2024-03-09T09:29:31.413234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2316","last_scrub_stamp":"2024-03-09T09:29:31.413148+0000","last_deep_scrub":"89'2316","last_deep_scrub_stamp":"2024-03-09T09:29:31.413148+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.413148+0000","objects_scrubbed":119,"log_size":2316,"log_dups_size":0,"ondisk_log_size":2316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:25:38.834796+0000","scrub_duration":212,"objects_trimmed":0,"snaptrim_duration":0.91280589400000001,"stat_sum":{"num_bytes":4608718,"num_objects":119,"num_object_clones":37,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":37,"num_read":119,"num_read_kb":26183,"num_write":1317,"num_write_kb":37436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'716","reported_seq":894,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.567246+0000","last_change":"2024-03-09T09:29:27.567312+0000","last_active":"2024-03-09T09:29:27.567246+0000","last_peered":"2024-03-09T09:29:27.567246+0000","last_clean":"2024-03-09T09:29:27.567246+0000","last_became_active":"2024-03-09T08:52:17.271652+0000","last_became_peered":"2024-03-09T08:52:17.271652+0000","last_unstale":"2024-03-09T09:29:27.567246+0000","last_undegraded":"2024-03-09T09:29:27.567246+0000","last_fullsized":"2024-03-09T09:29:27.567246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'716","last_scrub_stamp":"2024-03-09T09:29:27.567172+0000","last_deep_scrub":"89'716","last_deep_scrub_stamp":"2024-03-09T09:29:27.567172+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.567172+0000","objects_scrubbed":44,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:24:56.572979+0000","scrub_duration":87,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":47,"num_read_kb":0,"num_write":724,"num_write_kb":103814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547853,"num_omap_keys":763,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"89'767","reported_seq":939,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.026274+0000","last_change":"2024-03-09T09:29:33.026328+0000","last_active":"2024-03-09T09:29:33.026274+0000","last_peered":"2024-03-09T09:29:33.026274+0000","last_clean":"2024-03-09T09:29:33.026274+0000","last_became_active":"2024-03-09T08:52:28.587071+0000","last_became_peered":"2024-03-09T08:52:28.587071+0000","last_unstale":"2024-03-09T09:29:33.026274+0000","last_undegraded":"2024-03-09T09:29:33.026274+0000","last_fullsized":"2024-03-09T09:29:33.026274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'767","last_scrub_stamp":"2024-03-09T09:29:33.026092+0000","last_deep_scrub":"89'767","last_deep_scrub_stamp":"2024-03-09T09:29:33.026092+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.026092+0000","objects_scrubbed":54,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:20:10.415787+0000","scrub_duration":182,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":59,"num_read_kb":0,"num_write":793,"num_write_kb":95673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":554506,"num_omap_keys":964,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"89'2494","reported_seq":5406,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.768604+0000","last_change":"2024-03-09T09:29:33.768673+0000","last_active":"2024-03-09T09:29:33.768604+0000","last_peered":"2024-03-09T09:29:33.768604+0000","last_clean":"2024-03-09T09:29:33.768604+0000","last_became_active":"2024-03-09T08:52:29.585863+0000","last_became_peered":"2024-03-09T08:52:29.585863+0000","last_unstale":"2024-03-09T09:29:33.768604+0000","last_undegraded":"2024-03-09T09:29:33.768604+0000","last_fullsized":"2024-03-09T09:29:33.768604+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2494","last_scrub_stamp":"2024-03-09T09:29:33.768471+0000","last_deep_scrub":"89'2494","last_deep_scrub_stamp":"2024-03-09T09:29:33.768471+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.768471+0000","objects_scrubbed":135,"log_size":2494,"log_dups_size":0,"ondisk_log_size":2494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:25:40.852627+0000","scrub_duration":235,"objects_trimmed":0,"snaptrim_duration":0.634097468,"stat_sum":{"num_bytes":572008,"num_objects":135,"num_object_clones":47,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":47,"num_read":117,"num_read_kb":9742,"num_write":1431,"num_write_kb":32918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"89'2353","reported_seq":5345,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.349389+0000","last_change":"2024-03-09T09:29:36.349440+0000","last_active":"2024-03-09T09:29:36.349389+0000","last_peered":"2024-03-09T09:29:36.349389+0000","last_clean":"2024-03-09T09:29:36.349389+0000","last_became_active":"2024-03-09T08:52:18.245926+0000","last_became_peered":"2024-03-09T08:52:18.245926+0000","last_unstale":"2024-03-09T09:29:36.349389+0000","last_undegraded":"2024-03-09T09:29:36.349389+0000","last_fullsized":"2024-03-09T09:29:36.349389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2353","last_scrub_stamp":"2024-03-09T09:29:36.349279+0000","last_deep_scrub":"89'2353","last_deep_scrub_stamp":"2024-03-09T09:29:36.349279+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.349279+0000","objects_scrubbed":105,"log_size":2353,"log_dups_size":0,"ondisk_log_size":2353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:48:34.518396+0000","scrub_duration":171,"objects_trimmed":0,"snaptrim_duration":0.81694827699999994,"stat_sum":{"num_bytes":851628,"num_objects":105,"num_object_clones":33,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":33,"num_read":114,"num_read_kb":26314,"num_write":1339,"num_write_kb":37517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"89'569","reported_seq":746,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.941125+0000","last_change":"2024-03-09T09:29:32.941257+0000","last_active":"2024-03-09T09:29:32.941125+0000","last_peered":"2024-03-09T09:29:32.941125+0000","last_clean":"2024-03-09T09:29:32.941125+0000","last_became_active":"2024-03-09T08:52:17.283356+0000","last_became_peered":"2024-03-09T08:52:17.283356+0000","last_unstale":"2024-03-09T09:29:32.941125+0000","last_undegraded":"2024-03-09T09:29:32.941125+0000","last_fullsized":"2024-03-09T09:29:32.941125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'569","last_scrub_stamp":"2024-03-09T09:29:32.941005+0000","last_deep_scrub":"89'569","last_deep_scrub_stamp":"2024-03-09T09:29:32.941005+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.941005+0000","objects_scrubbed":49,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:00:09.932346+0000","scrub_duration":177,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554454,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":604,"num_write_kb":91576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":684517,"num_omap_keys":1040,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"89'740","reported_seq":912,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.997100+0000","last_change":"2024-03-09T09:29:30.997177+0000","last_active":"2024-03-09T09:29:30.997100+0000","last_peered":"2024-03-09T09:29:30.997100+0000","last_clean":"2024-03-09T09:29:30.997100+0000","last_became_active":"2024-03-09T08:52:28.581904+0000","last_became_peered":"2024-03-09T08:52:28.581904+0000","last_unstale":"2024-03-09T09:29:30.997100+0000","last_undegraded":"2024-03-09T09:29:30.997100+0000","last_fullsized":"2024-03-09T09:29:30.997100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'740","last_scrub_stamp":"2024-03-09T09:29:30.996979+0000","last_deep_scrub":"89'740","last_deep_scrub_stamp":"2024-03-09T09:29:30.996979+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.996979+0000","objects_scrubbed":57,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:12:39.922259+0000","scrub_duration":202,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":63,"num_read_kb":0,"num_write":783,"num_write_kb":95503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":674714,"num_omap_keys":1159,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"89'2437","reported_seq":5342,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:54.585932+0000","last_change":"2024-03-09T09:29:54.586006+0000","last_active":"2024-03-09T09:29:54.585932+0000","last_peered":"2024-03-09T09:29:54.585932+0000","last_clean":"2024-03-09T09:29:54.585932+0000","last_became_active":"2024-03-09T08:52:29.610232+0000","last_became_peered":"2024-03-09T08:52:29.610232+0000","last_unstale":"2024-03-09T09:29:54.585932+0000","last_undegraded":"2024-03-09T09:29:54.585932+0000","last_fullsized":"2024-03-09T09:29:54.585932+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2437","last_scrub_stamp":"2024-03-09T09:29:54.585849+0000","last_deep_scrub":"89'2437","last_deep_scrub_stamp":"2024-03-09T09:29:54.585849+0000","last_clean_scrub_stamp":"2024-03-09T09:29:54.585849+0000","objects_scrubbed":133,"log_size":2437,"log_dups_size":0,"ondisk_log_size":2437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:55:53.175608+0000","scrub_duration":238,"objects_trimmed":4,"snaptrim_duration":4.1951373749999998,"stat_sum":{"num_bytes":1358361,"num_objects":133,"num_object_clones":45,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":45,"num_read":125,"num_read_kb":25752,"num_write":1343,"num_write_kb":45310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"89'2391","reported_seq":5377,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.368641+0000","last_change":"2024-03-09T09:29:34.368641+0000","last_active":"2024-03-09T09:29:34.368641+0000","last_peered":"2024-03-09T09:29:34.368641+0000","last_clean":"2024-03-09T09:29:34.368641+0000","last_became_active":"2024-03-09T08:52:18.244435+0000","last_became_peered":"2024-03-09T08:52:18.244435+0000","last_unstale":"2024-03-09T09:29:34.368641+0000","last_undegraded":"2024-03-09T09:29:34.368641+0000","last_fullsized":"2024-03-09T09:29:34.368641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2391","last_scrub_stamp":"2024-03-09T09:29:34.368459+0000","last_deep_scrub":"89'2391","last_deep_scrub_stamp":"2024-03-09T09:29:34.368459+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.368459+0000","objects_scrubbed":106,"log_size":2391,"log_dups_size":0,"ondisk_log_size":2391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:10:24.314293+0000","scrub_duration":184,"objects_trimmed":0,"snaptrim_duration":0.30155770100000001,"stat_sum":{"num_bytes":730111,"num_objects":106,"num_object_clones":35,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":35,"num_read":106,"num_read_kb":32982,"num_write":1381,"num_write_kb":56548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'813","reported_seq":992,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.585622+0000","last_change":"2024-03-09T09:29:28.585622+0000","last_active":"2024-03-09T09:29:28.585622+0000","last_peered":"2024-03-09T09:29:28.585622+0000","last_clean":"2024-03-09T09:29:28.585622+0000","last_became_active":"2024-03-09T08:52:17.277672+0000","last_became_peered":"2024-03-09T08:52:17.277672+0000","last_unstale":"2024-03-09T09:29:28.585622+0000","last_undegraded":"2024-03-09T09:29:28.585622+0000","last_fullsized":"2024-03-09T09:29:28.585622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'813","last_scrub_stamp":"2024-03-09T09:29:28.585444+0000","last_deep_scrub":"89'813","last_deep_scrub_stamp":"2024-03-09T09:29:28.585444+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.585444+0000","objects_scrubbed":64,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:35:23.927054+0000","scrub_duration":122,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":74,"num_read_kb":0,"num_write":870,"num_write_kb":87759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":821123,"num_omap_keys":1317,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"89'656","reported_seq":826,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:24.066554+0000","last_change":"2024-03-09T09:29:24.066590+0000","last_active":"2024-03-09T09:29:24.066554+0000","last_peered":"2024-03-09T09:29:24.066554+0000","last_clean":"2024-03-09T09:29:24.066554+0000","last_became_active":"2024-03-09T08:52:28.575717+0000","last_became_peered":"2024-03-09T08:52:28.575717+0000","last_unstale":"2024-03-09T09:29:24.066554+0000","last_undegraded":"2024-03-09T09:29:24.066554+0000","last_fullsized":"2024-03-09T09:29:24.066554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'656","last_scrub_stamp":"2024-03-09T09:29:24.066487+0000","last_deep_scrub":"89'656","last_deep_scrub_stamp":"2024-03-09T09:29:24.066487+0000","last_clean_scrub_stamp":"2024-03-09T09:29:24.066487+0000","objects_scrubbed":54,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:27:17.167110+0000","scrub_duration":113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":669,"num_write_kb":112438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":801869,"num_omap_keys":1283,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"89'2284","reported_seq":5228,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:47.078831+0000","last_change":"2024-03-09T09:29:47.078874+0000","last_active":"2024-03-09T09:29:47.078831+0000","last_peered":"2024-03-09T09:29:47.078831+0000","last_clean":"2024-03-09T09:29:47.078831+0000","last_became_active":"2024-03-09T08:52:29.575444+0000","last_became_peered":"2024-03-09T08:52:29.575444+0000","last_unstale":"2024-03-09T09:29:47.078831+0000","last_undegraded":"2024-03-09T09:29:47.078831+0000","last_fullsized":"2024-03-09T09:29:47.078831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2284","last_scrub_stamp":"2024-03-09T09:29:47.078751+0000","last_deep_scrub":"89'2284","last_deep_scrub_stamp":"2024-03-09T09:29:47.078751+0000","last_clean_scrub_stamp":"2024-03-09T09:29:47.078751+0000","objects_scrubbed":100,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:41:16.716934+0000","scrub_duration":178,"objects_trimmed":0,"snaptrim_duration":4.4375742420000002,"stat_sum":{"num_bytes":1378177,"num_objects":100,"num_object_clones":26,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":26,"num_read":107,"num_read_kb":18463,"num_write":1316,"num_write_kb":41976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"89'2284","reported_seq":5289,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.668543+0000","last_change":"2024-03-09T09:29:27.668608+0000","last_active":"2024-03-09T09:29:27.668543+0000","last_peered":"2024-03-09T09:29:27.668543+0000","last_clean":"2024-03-09T09:29:27.668543+0000","last_became_active":"2024-03-09T08:52:18.244828+0000","last_became_peered":"2024-03-09T08:52:18.244828+0000","last_unstale":"2024-03-09T09:29:27.668543+0000","last_undegraded":"2024-03-09T09:29:27.668543+0000","last_fullsized":"2024-03-09T09:29:27.668543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2284","last_scrub_stamp":"2024-03-09T09:29:27.668439+0000","last_deep_scrub":"89'2284","last_deep_scrub_stamp":"2024-03-09T09:29:27.668439+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.668439+0000","objects_scrubbed":103,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:36:45.259300+0000","scrub_duration":219,"objects_trimmed":0,"snaptrim_duration":2.3444653409999998,"stat_sum":{"num_bytes":3375625,"num_objects":103,"num_object_clones":31,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":31,"num_read":106,"num_read_kb":36185,"num_write":1311,"num_write_kb":52081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"89'425","reported_seq":603,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.568640+0000","last_change":"2024-03-09T09:29:34.568640+0000","last_active":"2024-03-09T09:29:34.568640+0000","last_peered":"2024-03-09T09:29:34.568640+0000","last_clean":"2024-03-09T09:29:34.568640+0000","last_became_active":"2024-03-09T08:52:17.290264+0000","last_became_peered":"2024-03-09T08:52:17.290264+0000","last_unstale":"2024-03-09T09:29:34.568640+0000","last_undegraded":"2024-03-09T09:29:34.568640+0000","last_fullsized":"2024-03-09T09:29:34.568640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'425","last_scrub_stamp":"2024-03-09T09:29:34.568493+0000","last_deep_scrub":"89'425","last_deep_scrub_stamp":"2024-03-09T09:29:34.568493+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.568493+0000","objects_scrubbed":39,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:10.520113+0000","scrub_duration":154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":412,"num_write_kb":99378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":459011,"num_omap_keys":713,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"89'611","reported_seq":1021,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.520367+0000","last_change":"2024-03-09T09:29:33.520455+0000","last_active":"2024-03-09T09:29:33.520367+0000","last_peered":"2024-03-09T09:29:33.520367+0000","last_clean":"2024-03-09T09:29:33.520367+0000","last_became_active":"2024-03-09T08:52:28.588127+0000","last_became_peered":"2024-03-09T08:52:28.588127+0000","last_unstale":"2024-03-09T09:29:33.520367+0000","last_undegraded":"2024-03-09T09:29:33.520367+0000","last_fullsized":"2024-03-09T09:29:33.520367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'611","last_scrub_stamp":"2024-03-09T09:29:33.520170+0000","last_deep_scrub":"89'611","last_deep_scrub_stamp":"2024-03-09T09:29:33.520170+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.520170+0000","objects_scrubbed":51,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:30:44.823125+0000","scrub_duration":171,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007622,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":290,"num_read_kb":993,"num_write":609,"num_write_kb":88247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":534239,"num_omap_keys":903,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"89'2348","reported_seq":5268,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:53.262544+0000","last_change":"2024-03-09T09:29:53.262611+0000","last_active":"2024-03-09T09:29:53.262544+0000","last_peered":"2024-03-09T09:29:53.262544+0000","last_clean":"2024-03-09T09:29:53.262544+0000","last_became_active":"2024-03-09T08:52:29.595494+0000","last_became_peered":"2024-03-09T08:52:29.595494+0000","last_unstale":"2024-03-09T09:29:53.262544+0000","last_undegraded":"2024-03-09T09:29:53.262544+0000","last_fullsized":"2024-03-09T09:29:53.262544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2348","last_scrub_stamp":"2024-03-09T09:29:53.262425+0000","last_deep_scrub":"89'2348","last_deep_scrub_stamp":"2024-03-09T09:29:53.262425+0000","last_clean_scrub_stamp":"2024-03-09T09:29:53.262425+0000","objects_scrubbed":131,"log_size":2348,"log_dups_size":0,"ondisk_log_size":2348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:15:48.392272+0000","scrub_duration":323,"objects_trimmed":2,"snaptrim_duration":4.4357263939999996,"stat_sum":{"num_bytes":1364696,"num_objects":131,"num_object_clones":50,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":50,"num_read":112,"num_read_kb":14161,"num_write":1321,"num_write_kb":29407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'2438","reported_seq":5411,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.285788+0000","last_change":"2024-03-09T09:29:29.285788+0000","last_active":"2024-03-09T09:29:29.285788+0000","last_peered":"2024-03-09T09:29:29.285788+0000","last_clean":"2024-03-09T09:29:29.285788+0000","last_became_active":"2024-03-09T08:52:18.243898+0000","last_became_peered":"2024-03-09T08:52:18.243898+0000","last_unstale":"2024-03-09T09:29:29.285788+0000","last_undegraded":"2024-03-09T09:29:29.285788+0000","last_fullsized":"2024-03-09T09:29:29.285788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2438","last_scrub_stamp":"2024-03-09T09:29:29.285678+0000","last_deep_scrub":"89'2438","last_deep_scrub_stamp":"2024-03-09T09:29:29.285678+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.285678+0000","objects_scrubbed":100,"log_size":2438,"log_dups_size":0,"ondisk_log_size":2438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:42:18.105349+0000","scrub_duration":147,"objects_trimmed":0,"snaptrim_duration":0.64721804299999997,"stat_sum":{"num_bytes":2530176,"num_objects":100,"num_object_clones":36,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":36,"num_read":119,"num_read_kb":17537,"num_write":1383,"num_write_kb":31574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'305","reported_seq":483,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.915455+0000","last_change":"2024-03-09T09:29:38.915505+0000","last_active":"2024-03-09T09:29:38.915455+0000","last_peered":"2024-03-09T09:29:38.915455+0000","last_clean":"2024-03-09T09:29:38.915455+0000","last_became_active":"2024-03-09T08:52:17.276573+0000","last_became_peered":"2024-03-09T08:52:17.276573+0000","last_unstale":"2024-03-09T09:29:38.915455+0000","last_undegraded":"2024-03-09T09:29:38.915455+0000","last_fullsized":"2024-03-09T09:29:38.915455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'305","last_scrub_stamp":"2024-03-09T09:29:38.915368+0000","last_deep_scrub":"89'305","last_deep_scrub_stamp":"2024-03-09T09:29:38.915368+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.915368+0000","objects_scrubbed":38,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:20:52.360008+0000","scrub_duration":69,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":41,"num_read_kb":0,"num_write":313,"num_write_kb":62737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":405052,"num_omap_keys":616,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"89'875","reported_seq":1044,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.337360+0000","last_change":"2024-03-09T09:29:25.337360+0000","last_active":"2024-03-09T09:29:25.337360+0000","last_peered":"2024-03-09T09:29:25.337360+0000","last_clean":"2024-03-09T09:29:25.337360+0000","last_became_active":"2024-03-09T08:52:28.585252+0000","last_became_peered":"2024-03-09T08:52:28.585252+0000","last_unstale":"2024-03-09T09:29:25.337360+0000","last_undegraded":"2024-03-09T09:29:25.337360+0000","last_fullsized":"2024-03-09T09:29:25.337360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'875","last_scrub_stamp":"2024-03-09T09:29:25.337193+0000","last_deep_scrub":"89'875","last_deep_scrub_stamp":"2024-03-09T09:29:25.337193+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.337193+0000","objects_scrubbed":46,"log_size":875,"log_dups_size":0,"ondisk_log_size":875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:21:18.748241+0000","scrub_duration":89,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":56,"num_read_kb":0,"num_write":880,"num_write_kb":124192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":605867,"num_omap_keys":715,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"89'2330","reported_seq":5251,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:48.498156+0000","last_change":"2024-03-09T09:29:48.498156+0000","last_active":"2024-03-09T09:29:48.498156+0000","last_peered":"2024-03-09T09:29:48.498156+0000","last_clean":"2024-03-09T09:29:48.498156+0000","last_became_active":"2024-03-09T08:52:29.581009+0000","last_became_peered":"2024-03-09T08:52:29.581009+0000","last_unstale":"2024-03-09T09:29:48.498156+0000","last_undegraded":"2024-03-09T09:29:48.498156+0000","last_fullsized":"2024-03-09T09:29:48.498156+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2330","last_scrub_stamp":"2024-03-09T09:29:48.497986+0000","last_deep_scrub":"89'2330","last_deep_scrub_stamp":"2024-03-09T09:29:48.497986+0000","last_clean_scrub_stamp":"2024-03-09T09:29:48.497986+0000","objects_scrubbed":102,"log_size":2330,"log_dups_size":0,"ondisk_log_size":2330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:03:50.415412+0000","scrub_duration":197,"objects_trimmed":6,"snaptrim_duration":4.2113280319999999,"stat_sum":{"num_bytes":5678953,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":106,"num_read_kb":14364,"num_write":1330,"num_write_kb":47101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"89'2273","reported_seq":5294,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.197398+0000","last_change":"2024-03-09T09:29:35.197398+0000","last_active":"2024-03-09T09:29:35.197398+0000","last_peered":"2024-03-09T09:29:35.197398+0000","last_clean":"2024-03-09T09:29:35.197398+0000","last_became_active":"2024-03-09T08:52:18.235285+0000","last_became_peered":"2024-03-09T08:52:18.235285+0000","last_unstale":"2024-03-09T09:29:35.197398+0000","last_undegraded":"2024-03-09T09:29:35.197398+0000","last_fullsized":"2024-03-09T09:29:35.197398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2273","last_scrub_stamp":"2024-03-09T09:29:35.197297+0000","last_deep_scrub":"89'2273","last_deep_scrub_stamp":"2024-03-09T09:29:35.197297+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.197297+0000","objects_scrubbed":124,"log_size":2273,"log_dups_size":0,"ondisk_log_size":2273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:16:48.175211+0000","scrub_duration":177,"objects_trimmed":0,"snaptrim_duration":0.096588679999999996,"stat_sum":{"num_bytes":1266796,"num_objects":124,"num_object_clones":39,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":39,"num_read":107,"num_read_kb":17887,"num_write":1323,"num_write_kb":41057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"89'647","reported_seq":826,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.093883+0000","last_change":"2024-03-09T09:29:28.093883+0000","last_active":"2024-03-09T09:29:28.093883+0000","last_peered":"2024-03-09T09:29:28.093883+0000","last_clean":"2024-03-09T09:29:28.093883+0000","last_became_active":"2024-03-09T08:52:17.286407+0000","last_became_peered":"2024-03-09T08:52:17.286407+0000","last_unstale":"2024-03-09T09:29:28.093883+0000","last_undegraded":"2024-03-09T09:29:28.093883+0000","last_fullsized":"2024-03-09T09:29:28.093883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'647","last_scrub_stamp":"2024-03-09T09:29:28.093716+0000","last_deep_scrub":"89'647","last_deep_scrub_stamp":"2024-03-09T09:29:28.093716+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.093716+0000","objects_scrubbed":53,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:23:58.108279+0000","scrub_duration":131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":60,"num_read_kb":0,"num_write":687,"num_write_kb":95812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":576914,"num_omap_keys":966,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"89'1219","reported_seq":1388,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.916765+0000","last_change":"2024-03-09T09:29:30.916912+0000","last_active":"2024-03-09T09:29:30.916765+0000","last_peered":"2024-03-09T09:29:30.916765+0000","last_clean":"2024-03-09T09:29:30.916765+0000","last_became_active":"2024-03-09T08:52:28.585386+0000","last_became_peered":"2024-03-09T08:52:28.585386+0000","last_unstale":"2024-03-09T09:29:30.916765+0000","last_undegraded":"2024-03-09T09:29:30.916765+0000","last_fullsized":"2024-03-09T09:29:30.916765+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1219","last_scrub_stamp":"2024-03-09T09:29:30.916643+0000","last_deep_scrub":"89'1219","last_deep_scrub_stamp":"2024-03-09T09:29:30.916643+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.916643+0000","objects_scrubbed":43,"log_size":1219,"log_dups_size":0,"ondisk_log_size":1219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:53:23.719504+0000","scrub_duration":101,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777306,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":54,"num_read_kb":0,"num_write":1250,"num_write_kb":83773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":524629,"num_omap_keys":853,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"89'2460","reported_seq":5375,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:47.421417+0000","last_change":"2024-03-09T09:29:47.421417+0000","last_active":"2024-03-09T09:29:47.421417+0000","last_peered":"2024-03-09T09:29:47.421417+0000","last_clean":"2024-03-09T09:29:47.421417+0000","last_became_active":"2024-03-09T08:52:29.578505+0000","last_became_peered":"2024-03-09T08:52:29.578505+0000","last_unstale":"2024-03-09T09:29:47.421417+0000","last_undegraded":"2024-03-09T09:29:47.421417+0000","last_fullsized":"2024-03-09T09:29:47.421417+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2460","last_scrub_stamp":"2024-03-09T09:29:47.421304+0000","last_deep_scrub":"89'2460","last_deep_scrub_stamp":"2024-03-09T09:29:47.421304+0000","last_clean_scrub_stamp":"2024-03-09T09:29:47.421304+0000","objects_scrubbed":96,"log_size":2460,"log_dups_size":0,"ondisk_log_size":2460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:13:28.293465+0000","scrub_duration":168,"objects_trimmed":4,"snaptrim_duration":4.2009582749999996,"stat_sum":{"num_bytes":273727,"num_objects":96,"num_object_clones":34,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":34,"num_read":130,"num_read_kb":37554,"num_write":1394,"num_write_kb":63694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"89'2354","reported_seq":5332,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.440003+0000","last_change":"2024-03-09T09:29:38.440041+0000","last_active":"2024-03-09T09:29:38.440003+0000","last_peered":"2024-03-09T09:29:38.440003+0000","last_clean":"2024-03-09T09:29:38.440003+0000","last_became_active":"2024-03-09T08:52:18.242140+0000","last_became_peered":"2024-03-09T08:52:18.242140+0000","last_unstale":"2024-03-09T09:29:38.440003+0000","last_undegraded":"2024-03-09T09:29:38.440003+0000","last_fullsized":"2024-03-09T09:29:38.440003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2354","last_scrub_stamp":"2024-03-09T09:29:38.439939+0000","last_deep_scrub":"89'2354","last_deep_scrub_stamp":"2024-03-09T09:29:38.439939+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.439939+0000","objects_scrubbed":133,"log_size":2354,"log_dups_size":0,"ondisk_log_size":2354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:24:51.287488+0000","scrub_duration":260,"objects_trimmed":0,"snaptrim_duration":0.19064995900000001,"stat_sum":{"num_bytes":1055946,"num_objects":133,"num_object_clones":51,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":51,"num_read":117,"num_read_kb":18332,"num_write":1305,"num_write_kb":38006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"89'1087","reported_seq":1265,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.588370+0000","last_change":"2024-03-09T09:29:32.588370+0000","last_active":"2024-03-09T09:29:32.588370+0000","last_peered":"2024-03-09T09:29:32.588370+0000","last_clean":"2024-03-09T09:29:32.588370+0000","last_became_active":"2024-03-09T08:52:17.290880+0000","last_became_peered":"2024-03-09T08:52:17.290880+0000","last_unstale":"2024-03-09T09:29:32.588370+0000","last_undegraded":"2024-03-09T09:29:32.588370+0000","last_fullsized":"2024-03-09T09:29:32.588370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1087","last_scrub_stamp":"2024-03-09T09:29:32.588105+0000","last_deep_scrub":"89'1087","last_deep_scrub_stamp":"2024-03-09T09:29:32.588105+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.588105+0000","objects_scrubbed":48,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:10:42.375813+0000","scrub_duration":154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":58,"num_read_kb":0,"num_write":1092,"num_write_kb":124321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":613889,"num_omap_keys":1009,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"89'340","reported_seq":510,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.177305+0000","last_change":"2024-03-09T09:29:27.177305+0000","last_active":"2024-03-09T09:29:27.177305+0000","last_peered":"2024-03-09T09:29:27.177305+0000","last_clean":"2024-03-09T09:29:27.177305+0000","last_became_active":"2024-03-09T08:52:28.580036+0000","last_became_peered":"2024-03-09T08:52:28.580036+0000","last_unstale":"2024-03-09T09:29:27.177305+0000","last_undegraded":"2024-03-09T09:29:27.177305+0000","last_fullsized":"2024-03-09T09:29:27.177305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'340","last_scrub_stamp":"2024-03-09T09:29:27.177105+0000","last_deep_scrub":"89'340","last_deep_scrub_stamp":"2024-03-09T09:29:27.177105+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.177105+0000","objects_scrubbed":43,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:44:35.621276+0000","scrub_duration":81,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":38,"num_read_kb":0,"num_write":328,"num_write_kb":62283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":599772,"num_omap_keys":972,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"89'2304","reported_seq":5248,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.141171+0000","last_change":"2024-03-09T09:29:42.141171+0000","last_active":"2024-03-09T09:29:42.141171+0000","last_peered":"2024-03-09T09:29:42.141171+0000","last_clean":"2024-03-09T09:29:42.141171+0000","last_became_active":"2024-03-09T08:52:29.585565+0000","last_became_peered":"2024-03-09T08:52:29.585565+0000","last_unstale":"2024-03-09T09:29:42.141171+0000","last_undegraded":"2024-03-09T09:29:42.141171+0000","last_fullsized":"2024-03-09T09:29:42.141171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2304","last_scrub_stamp":"2024-03-09T09:29:42.140985+0000","last_deep_scrub":"89'2304","last_deep_scrub_stamp":"2024-03-09T09:29:42.140985+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.140985+0000","objects_scrubbed":130,"log_size":2304,"log_dups_size":0,"ondisk_log_size":2304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:56:57.948152+0000","scrub_duration":246,"objects_trimmed":0,"snaptrim_duration":1.695386042,"stat_sum":{"num_bytes":3385389,"num_objects":130,"num_object_clones":42,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":42,"num_read":111,"num_read_kb":40727,"num_write":1306,"num_write_kb":67855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"89'2331","reported_seq":5308,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.804448+0000","last_change":"2024-03-09T09:29:37.804518+0000","last_active":"2024-03-09T09:29:37.804448+0000","last_peered":"2024-03-09T09:29:37.804448+0000","last_clean":"2024-03-09T09:29:37.804448+0000","last_became_active":"2024-03-09T08:52:18.244221+0000","last_became_peered":"2024-03-09T08:52:18.244221+0000","last_unstale":"2024-03-09T09:29:37.804448+0000","last_undegraded":"2024-03-09T09:29:37.804448+0000","last_fullsized":"2024-03-09T09:29:37.804448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2331","last_scrub_stamp":"2024-03-09T09:29:37.804328+0000","last_deep_scrub":"89'2331","last_deep_scrub_stamp":"2024-03-09T09:29:37.804328+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.804328+0000","objects_scrubbed":94,"log_size":2331,"log_dups_size":0,"ondisk_log_size":2331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:18:40.635031+0000","scrub_duration":134,"objects_trimmed":0,"snaptrim_duration":3.3666159750000002,"stat_sum":{"num_bytes":799596,"num_objects":94,"num_object_clones":30,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":30,"num_read":103,"num_read_kb":34868,"num_write":1326,"num_write_kb":46782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'1346","reported_seq":1523,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.949434+0000","last_change":"2024-03-09T09:29:35.949536+0000","last_active":"2024-03-09T09:29:35.949434+0000","last_peered":"2024-03-09T09:29:35.949434+0000","last_clean":"2024-03-09T09:29:35.949434+0000","last_became_active":"2024-03-09T08:52:17.281746+0000","last_became_peered":"2024-03-09T08:52:17.281746+0000","last_unstale":"2024-03-09T09:29:35.949434+0000","last_undegraded":"2024-03-09T09:29:35.949434+0000","last_fullsized":"2024-03-09T09:29:35.949434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'1346","last_scrub_stamp":"2024-03-09T09:29:35.949327+0000","last_deep_scrub":"89'1346","last_deep_scrub_stamp":"2024-03-09T09:29:35.949327+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.949327+0000","objects_scrubbed":48,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:59:02.731368+0000","scrub_duration":87,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":80,"num_read_kb":0,"num_write":1400,"num_write_kb":80149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":852662,"num_omap_keys":1288,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"89'461","reported_seq":632,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.801135+0000","last_change":"2024-03-09T09:29:38.801135+0000","last_active":"2024-03-09T09:29:38.801135+0000","last_peered":"2024-03-09T09:29:38.801135+0000","last_clean":"2024-03-09T09:29:38.801135+0000","last_became_active":"2024-03-09T08:52:28.568481+0000","last_became_peered":"2024-03-09T08:52:28.568481+0000","last_unstale":"2024-03-09T09:29:38.801135+0000","last_undegraded":"2024-03-09T09:29:38.801135+0000","last_fullsized":"2024-03-09T09:29:38.801135+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'461","last_scrub_stamp":"2024-03-09T09:29:38.801026+0000","last_deep_scrub":"89'461","last_deep_scrub_stamp":"2024-03-09T09:29:38.801026+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.801026+0000","objects_scrubbed":50,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:46.713037+0000","scrub_duration":123,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360162,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":462,"num_write_kb":95092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":374202,"num_omap_keys":642,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"89'2374","reported_seq":5309,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.430466+0000","last_change":"2024-03-09T09:29:45.430466+0000","last_active":"2024-03-09T09:29:45.430466+0000","last_peered":"2024-03-09T09:29:45.430466+0000","last_clean":"2024-03-09T09:29:45.430466+0000","last_became_active":"2024-03-09T08:52:29.579389+0000","last_became_peered":"2024-03-09T08:52:29.579389+0000","last_unstale":"2024-03-09T09:29:45.430466+0000","last_undegraded":"2024-03-09T09:29:45.430466+0000","last_fullsized":"2024-03-09T09:29:45.430466+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2374","last_scrub_stamp":"2024-03-09T09:29:45.430327+0000","last_deep_scrub":"89'2374","last_deep_scrub_stamp":"2024-03-09T09:29:45.430327+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.430327+0000","objects_scrubbed":103,"log_size":2374,"log_dups_size":0,"ondisk_log_size":2374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:27:17.101981+0000","scrub_duration":176,"objects_trimmed":0,"snaptrim_duration":0.64490501099999997,"stat_sum":{"num_bytes":1735737,"num_objects":103,"num_object_clones":32,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":32,"num_read":113,"num_read_kb":37575,"num_write":1379,"num_write_kb":57434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"89'2414","reported_seq":5378,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.180587+0000","last_change":"2024-03-09T09:29:43.180642+0000","last_active":"2024-03-09T09:29:43.180587+0000","last_peered":"2024-03-09T09:29:43.180587+0000","last_clean":"2024-03-09T09:29:43.180587+0000","last_became_active":"2024-03-09T08:53:00.254589+0000","last_became_peered":"2024-03-09T08:53:00.254589+0000","last_unstale":"2024-03-09T09:29:43.180587+0000","last_undegraded":"2024-03-09T09:29:43.180587+0000","last_fullsized":"2024-03-09T09:29:43.180587+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2414","last_scrub_stamp":"2024-03-09T09:29:43.180507+0000","last_deep_scrub":"89'2414","last_deep_scrub_stamp":"2024-03-09T09:29:43.180507+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.180507+0000","objects_scrubbed":123,"log_size":2414,"log_dups_size":0,"ondisk_log_size":2414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:11:55.562575+0000","scrub_duration":214,"objects_trimmed":0,"snaptrim_duration":0.471373141,"stat_sum":{"num_bytes":5513596,"num_objects":123,"num_object_clones":39,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":39,"num_read":114,"num_read_kb":29387,"num_write":1331,"num_write_kb":57801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'618","reported_seq":795,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.986665+0000","last_change":"2024-03-09T09:29:42.986696+0000","last_active":"2024-03-09T09:29:42.986665+0000","last_peered":"2024-03-09T09:29:42.986665+0000","last_clean":"2024-03-09T09:29:42.986665+0000","last_became_active":"2024-03-09T08:52:17.290238+0000","last_became_peered":"2024-03-09T08:52:17.290238+0000","last_unstale":"2024-03-09T09:29:42.986665+0000","last_undegraded":"2024-03-09T09:29:42.986665+0000","last_fullsized":"2024-03-09T09:29:42.986665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'618","last_scrub_stamp":"2024-03-09T09:29:42.986545+0000","last_deep_scrub":"89'618","last_deep_scrub_stamp":"2024-03-09T09:29:42.986545+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.986545+0000","objects_scrubbed":28,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:44:35.113222+0000","scrub_duration":92,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":24,"num_read_kb":0,"num_write":598,"num_write_kb":111422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":173539,"num_omap_keys":299,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"89'785","reported_seq":955,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.372377+0000","last_change":"2024-03-09T09:29:26.372377+0000","last_active":"2024-03-09T09:29:26.372377+0000","last_peered":"2024-03-09T09:29:26.372377+0000","last_clean":"2024-03-09T09:29:26.372377+0000","last_became_active":"2024-03-09T08:52:28.581334+0000","last_became_peered":"2024-03-09T08:52:28.581334+0000","last_unstale":"2024-03-09T09:29:26.372377+0000","last_undegraded":"2024-03-09T09:29:26.372377+0000","last_fullsized":"2024-03-09T09:29:26.372377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'785","last_scrub_stamp":"2024-03-09T09:29:26.372276+0000","last_deep_scrub":"89'785","last_deep_scrub_stamp":"2024-03-09T09:29:26.372276+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.372276+0000","objects_scrubbed":56,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:43:47.007320+0000","scrub_duration":104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":81,"num_read_kb":0,"num_write":849,"num_write_kb":91941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":871124,"num_omap_keys":1334,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"89'2639","reported_seq":5457,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.324027+0000","last_change":"2024-03-09T09:29:39.324027+0000","last_active":"2024-03-09T09:29:39.324027+0000","last_peered":"2024-03-09T09:29:39.324027+0000","last_clean":"2024-03-09T09:29:39.324027+0000","last_became_active":"2024-03-09T08:52:29.590403+0000","last_became_peered":"2024-03-09T08:52:29.590403+0000","last_unstale":"2024-03-09T09:29:39.324027+0000","last_undegraded":"2024-03-09T09:29:39.324027+0000","last_fullsized":"2024-03-09T09:29:39.324027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2639","last_scrub_stamp":"2024-03-09T09:29:39.323851+0000","last_deep_scrub":"89'2639","last_deep_scrub_stamp":"2024-03-09T09:29:39.323851+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.323851+0000","objects_scrubbed":111,"log_size":2639,"log_dups_size":0,"ondisk_log_size":2639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:21:51.409246+0000","scrub_duration":160,"objects_trimmed":0,"snaptrim_duration":4.4377723019999999,"stat_sum":{"num_bytes":1160060,"num_objects":111,"num_object_clones":35,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":35,"num_read":113,"num_read_kb":9969,"num_write":1381,"num_write_kb":17733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"89'2272","reported_seq":5281,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.941475+0000","last_change":"2024-03-09T09:29:28.941508+0000","last_active":"2024-03-09T09:29:28.941475+0000","last_peered":"2024-03-09T09:29:28.941475+0000","last_clean":"2024-03-09T09:29:28.941475+0000","last_became_active":"2024-03-09T08:52:18.250067+0000","last_became_peered":"2024-03-09T08:52:18.250067+0000","last_unstale":"2024-03-09T09:29:28.941475+0000","last_undegraded":"2024-03-09T09:29:28.941475+0000","last_fullsized":"2024-03-09T09:29:28.941475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2272","last_scrub_stamp":"2024-03-09T09:29:28.941403+0000","last_deep_scrub":"89'2272","last_deep_scrub_stamp":"2024-03-09T09:29:28.941403+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.941403+0000","objects_scrubbed":107,"log_size":2272,"log_dups_size":0,"ondisk_log_size":2272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:10:44.701638+0000","scrub_duration":152,"objects_trimmed":0,"snaptrim_duration":1.82863014,"stat_sum":{"num_bytes":1951923,"num_objects":107,"num_object_clones":29,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":29,"num_read":96,"num_read_kb":37066,"num_write":1331,"num_write_kb":53456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"89'601","reported_seq":1007,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.480009+0000","last_change":"2024-03-09T09:29:35.480088+0000","last_active":"2024-03-09T09:29:35.480009+0000","last_peered":"2024-03-09T09:29:35.480009+0000","last_clean":"2024-03-09T09:29:35.480009+0000","last_became_active":"2024-03-09T08:52:17.293938+0000","last_became_peered":"2024-03-09T08:52:17.293938+0000","last_unstale":"2024-03-09T09:29:35.480009+0000","last_undegraded":"2024-03-09T09:29:35.480009+0000","last_fullsized":"2024-03-09T09:29:35.480009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'601","last_scrub_stamp":"2024-03-09T09:29:35.479886+0000","last_deep_scrub":"89'601","last_deep_scrub_stamp":"2024-03-09T09:29:35.479886+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.479886+0000","objects_scrubbed":48,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:29:59.508107+0000","scrub_duration":106,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26007933,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":267,"num_read_kb":985,"num_write":604,"num_write_kb":87856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":409708,"num_omap_keys":587,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"89'504","reported_seq":673,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.476044+0000","last_change":"2024-03-09T09:29:29.476044+0000","last_active":"2024-03-09T09:29:29.476044+0000","last_peered":"2024-03-09T09:29:29.476044+0000","last_clean":"2024-03-09T09:29:29.476044+0000","last_became_active":"2024-03-09T08:52:28.586315+0000","last_became_peered":"2024-03-09T08:52:28.586315+0000","last_unstale":"2024-03-09T09:29:29.476044+0000","last_undegraded":"2024-03-09T09:29:29.476044+0000","last_fullsized":"2024-03-09T09:29:29.476044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'504","last_scrub_stamp":"2024-03-09T09:29:29.475650+0000","last_deep_scrub":"89'504","last_deep_scrub_stamp":"2024-03-09T09:29:29.475650+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.475650+0000","objects_scrubbed":43,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:28:49.825813+0000","scrub_duration":189,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":42,"num_read_kb":0,"num_write":506,"num_write_kb":99411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":543720,"num_omap_keys":804,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"89'2299","reported_seq":5236,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.419568+0000","last_change":"2024-03-09T09:29:44.419568+0000","last_active":"2024-03-09T09:29:44.419568+0000","last_peered":"2024-03-09T09:29:44.419568+0000","last_clean":"2024-03-09T09:29:44.419568+0000","last_became_active":"2024-03-09T08:52:29.582384+0000","last_became_peered":"2024-03-09T08:52:29.582384+0000","last_unstale":"2024-03-09T09:29:44.419568+0000","last_undegraded":"2024-03-09T09:29:44.419568+0000","last_fullsized":"2024-03-09T09:29:44.419568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2299","last_scrub_stamp":"2024-03-09T09:29:44.419446+0000","last_deep_scrub":"89'2299","last_deep_scrub_stamp":"2024-03-09T09:29:44.419446+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.419446+0000","objects_scrubbed":110,"log_size":2299,"log_dups_size":0,"ondisk_log_size":2299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:58:45.243233+0000","scrub_duration":160,"objects_trimmed":0,"snaptrim_duration":0.64610520400000004,"stat_sum":{"num_bytes":1318782,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":124,"num_read_kb":9990,"num_write":1296,"num_write_kb":22209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"89'721","reported_seq":894,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.127524+0000","last_change":"2024-03-09T09:29:38.127674+0000","last_active":"2024-03-09T09:29:38.127524+0000","last_peered":"2024-03-09T09:29:38.127524+0000","last_clean":"2024-03-09T09:29:38.127524+0000","last_became_active":"2024-03-09T08:52:28.585921+0000","last_became_peered":"2024-03-09T08:52:28.585921+0000","last_unstale":"2024-03-09T09:29:38.127524+0000","last_undegraded":"2024-03-09T09:29:38.127524+0000","last_fullsized":"2024-03-09T09:29:38.127524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'721","last_scrub_stamp":"2024-03-09T09:29:38.127212+0000","last_deep_scrub":"89'721","last_deep_scrub_stamp":"2024-03-09T09:29:38.127212+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.127212+0000","objects_scrubbed":54,"log_size":721,"log_dups_size":0,"ondisk_log_size":721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:49:46.042055+0000","scrub_duration":149,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":72,"num_read_kb":0,"num_write":777,"num_write_kb":92056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":568534,"num_omap_keys":965,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'2330","reported_seq":5319,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:26.253759+0000","last_change":"2024-03-09T09:29:26.253814+0000","last_active":"2024-03-09T09:29:26.253759+0000","last_peered":"2024-03-09T09:29:26.253759+0000","last_clean":"2024-03-09T09:29:26.253759+0000","last_became_active":"2024-03-09T08:52:18.240124+0000","last_became_peered":"2024-03-09T08:52:18.240124+0000","last_unstale":"2024-03-09T09:29:26.253759+0000","last_undegraded":"2024-03-09T09:29:26.253759+0000","last_fullsized":"2024-03-09T09:29:26.253759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2330","last_scrub_stamp":"2024-03-09T09:29:26.253681+0000","last_deep_scrub":"89'2330","last_deep_scrub_stamp":"2024-03-09T09:29:26.253681+0000","last_clean_scrub_stamp":"2024-03-09T09:29:26.253681+0000","objects_scrubbed":120,"log_size":2330,"log_dups_size":0,"ondisk_log_size":2330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:25:05.771093+0000","scrub_duration":172,"objects_trimmed":0,"snaptrim_duration":1.23951623,"stat_sum":{"num_bytes":1295292,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":13266,"num_write":1323,"num_write_kb":26325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"89'907","reported_seq":1084,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.004768+0000","last_change":"2024-03-09T09:29:42.004865+0000","last_active":"2024-03-09T09:29:42.004768+0000","last_peered":"2024-03-09T09:29:42.004768+0000","last_clean":"2024-03-09T09:29:42.004768+0000","last_became_active":"2024-03-09T08:52:17.281616+0000","last_became_peered":"2024-03-09T08:52:17.281616+0000","last_unstale":"2024-03-09T09:29:42.004768+0000","last_undegraded":"2024-03-09T09:29:42.004768+0000","last_fullsized":"2024-03-09T09:29:42.004768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'907","last_scrub_stamp":"2024-03-09T09:29:42.004658+0000","last_deep_scrub":"89'907","last_deep_scrub_stamp":"2024-03-09T09:29:42.004658+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.004658+0000","objects_scrubbed":46,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:27:34.410597+0000","scrub_duration":133,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":45,"num_read_kb":0,"num_write":880,"num_write_kb":119947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":513434,"num_omap_keys":845,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"89'2251","reported_seq":5202,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.149013+0000","last_change":"2024-03-09T09:29:41.149074+0000","last_active":"2024-03-09T09:29:41.149013+0000","last_peered":"2024-03-09T09:29:41.149013+0000","last_clean":"2024-03-09T09:29:41.149013+0000","last_became_active":"2024-03-09T08:52:29.590753+0000","last_became_peered":"2024-03-09T08:52:29.590753+0000","last_unstale":"2024-03-09T09:29:41.149013+0000","last_undegraded":"2024-03-09T09:29:41.149013+0000","last_fullsized":"2024-03-09T09:29:41.149013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2251","last_scrub_stamp":"2024-03-09T09:29:41.148890+0000","last_deep_scrub":"89'2251","last_deep_scrub_stamp":"2024-03-09T09:29:41.148890+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.148890+0000","objects_scrubbed":107,"log_size":2251,"log_dups_size":0,"ondisk_log_size":2251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:24:46.012715+0000","scrub_duration":179,"objects_trimmed":2,"snaptrim_duration":0.70511981300000004,"stat_sum":{"num_bytes":1915718,"num_objects":107,"num_object_clones":37,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":37,"num_read":111,"num_read_kb":24546,"num_write":1285,"num_write_kb":44187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"89'596","reported_seq":769,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.437230+0000","last_change":"2024-03-09T09:29:40.437230+0000","last_active":"2024-03-09T09:29:40.437230+0000","last_peered":"2024-03-09T09:29:40.437230+0000","last_clean":"2024-03-09T09:29:40.437230+0000","last_became_active":"2024-03-09T08:52:28.585331+0000","last_became_peered":"2024-03-09T08:52:28.585331+0000","last_unstale":"2024-03-09T09:29:40.437230+0000","last_undegraded":"2024-03-09T09:29:40.437230+0000","last_fullsized":"2024-03-09T09:29:40.437230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'596","last_scrub_stamp":"2024-03-09T09:29:40.437090+0000","last_deep_scrub":"89'596","last_deep_scrub_stamp":"2024-03-09T09:29:40.437090+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.437090+0000","objects_scrubbed":50,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:43:24.326589+0000","scrub_duration":114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":589,"num_write_kb":82799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":492001,"num_omap_keys":847,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'2568","reported_seq":5501,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.496342+0000","last_change":"2024-03-09T09:29:43.496392+0000","last_active":"2024-03-09T09:29:43.496342+0000","last_peered":"2024-03-09T09:29:43.496342+0000","last_clean":"2024-03-09T09:29:43.496342+0000","last_became_active":"2024-03-09T08:52:18.233041+0000","last_became_peered":"2024-03-09T08:52:18.233041+0000","last_unstale":"2024-03-09T09:29:43.496342+0000","last_undegraded":"2024-03-09T09:29:43.496342+0000","last_fullsized":"2024-03-09T09:29:43.496342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2568","last_scrub_stamp":"2024-03-09T09:29:43.496266+0000","last_deep_scrub":"89'2568","last_deep_scrub_stamp":"2024-03-09T09:29:43.496266+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.496266+0000","objects_scrubbed":93,"log_size":2568,"log_dups_size":0,"ondisk_log_size":2568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:04:36.024377+0000","scrub_duration":181,"objects_trimmed":0,"snaptrim_duration":0.79565864399999997,"stat_sum":{"num_bytes":2512650,"num_objects":93,"num_object_clones":29,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":29,"num_read":114,"num_read_kb":34867,"num_write":1447,"num_write_kb":63344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'447","reported_seq":623,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:41.448400+0000","last_change":"2024-03-09T09:29:41.448512+0000","last_active":"2024-03-09T09:29:41.448400+0000","last_peered":"2024-03-09T09:29:41.448400+0000","last_clean":"2024-03-09T09:29:41.448400+0000","last_became_active":"2024-03-09T08:52:17.290347+0000","last_became_peered":"2024-03-09T08:52:17.290347+0000","last_unstale":"2024-03-09T09:29:41.448400+0000","last_undegraded":"2024-03-09T09:29:41.448400+0000","last_fullsized":"2024-03-09T09:29:41.448400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'447","last_scrub_stamp":"2024-03-09T09:29:41.448251+0000","last_deep_scrub":"89'447","last_deep_scrub_stamp":"2024-03-09T09:29:41.448251+0000","last_clean_scrub_stamp":"2024-03-09T09:29:41.448251+0000","objects_scrubbed":47,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:48:44.843582+0000","scrub_duration":116,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":53,"num_read_kb":0,"num_write":480,"num_write_kb":79281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":589869,"num_omap_keys":880,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"89'2289","reported_seq":5241,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.144881+0000","last_change":"2024-03-09T09:29:44.144881+0000","last_active":"2024-03-09T09:29:44.144881+0000","last_peered":"2024-03-09T09:29:44.144881+0000","last_clean":"2024-03-09T09:29:44.144881+0000","last_became_active":"2024-03-09T08:52:29.588361+0000","last_became_peered":"2024-03-09T08:52:29.588361+0000","last_unstale":"2024-03-09T09:29:44.144881+0000","last_undegraded":"2024-03-09T09:29:44.144881+0000","last_fullsized":"2024-03-09T09:29:44.144881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2289","last_scrub_stamp":"2024-03-09T09:29:44.144740+0000","last_deep_scrub":"89'2289","last_deep_scrub_stamp":"2024-03-09T09:29:44.144740+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.144740+0000","objects_scrubbed":112,"log_size":2289,"log_dups_size":0,"ondisk_log_size":2289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:19:24.277820+0000","scrub_duration":274,"objects_trimmed":4,"snaptrim_duration":1.72318536,"stat_sum":{"num_bytes":768948,"num_objects":112,"num_object_clones":37,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":37,"num_read":126,"num_read_kb":26327,"num_write":1312,"num_write_kb":46360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"89'779","reported_seq":956,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.511927+0000","last_change":"2024-03-09T09:29:31.511927+0000","last_active":"2024-03-09T09:29:31.511927+0000","last_peered":"2024-03-09T09:29:31.511927+0000","last_clean":"2024-03-09T09:29:31.511927+0000","last_became_active":"2024-03-09T08:52:17.287957+0000","last_became_peered":"2024-03-09T08:52:17.287957+0000","last_unstale":"2024-03-09T09:29:31.511927+0000","last_undegraded":"2024-03-09T09:29:31.511927+0000","last_fullsized":"2024-03-09T09:29:31.511927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'779","last_scrub_stamp":"2024-03-09T09:29:31.511787+0000","last_deep_scrub":"89'779","last_deep_scrub_stamp":"2024-03-09T09:29:31.511787+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.511787+0000","objects_scrubbed":40,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:36:36.049606+0000","scrub_duration":89,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":32,"num_read_kb":0,"num_write":777,"num_write_kb":95156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":299260,"num_omap_keys":519,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"89'2434","reported_seq":5434,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.212607+0000","last_change":"2024-03-09T09:29:33.212607+0000","last_active":"2024-03-09T09:29:33.212607+0000","last_peered":"2024-03-09T09:29:33.212607+0000","last_clean":"2024-03-09T09:29:33.212607+0000","last_became_active":"2024-03-09T08:52:18.230753+0000","last_became_peered":"2024-03-09T08:52:18.230753+0000","last_unstale":"2024-03-09T09:29:33.212607+0000","last_undegraded":"2024-03-09T09:29:33.212607+0000","last_fullsized":"2024-03-09T09:29:33.212607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2434","last_scrub_stamp":"2024-03-09T09:29:33.212411+0000","last_deep_scrub":"89'2434","last_deep_scrub_stamp":"2024-03-09T09:29:33.212411+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.212411+0000","objects_scrubbed":128,"log_size":2434,"log_dups_size":0,"ondisk_log_size":2434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:40:06.160484+0000","scrub_duration":228,"objects_trimmed":0,"snaptrim_duration":0.65597337700000002,"stat_sum":{"num_bytes":5623986,"num_objects":128,"num_object_clones":44,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":44,"num_read":148,"num_read_kb":35274,"num_write":1372,"num_write_kb":72006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"89'513","reported_seq":682,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.596851+0000","last_change":"2024-03-09T09:29:32.596851+0000","last_active":"2024-03-09T09:29:32.596851+0000","last_peered":"2024-03-09T09:29:32.596851+0000","last_clean":"2024-03-09T09:29:32.596851+0000","last_became_active":"2024-03-09T08:52:28.578928+0000","last_became_peered":"2024-03-09T08:52:28.578928+0000","last_unstale":"2024-03-09T09:29:32.596851+0000","last_undegraded":"2024-03-09T09:29:32.596851+0000","last_fullsized":"2024-03-09T09:29:32.596851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'513","last_scrub_stamp":"2024-03-09T09:29:32.596705+0000","last_deep_scrub":"89'513","last_deep_scrub_stamp":"2024-03-09T09:29:32.596705+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.596705+0000","objects_scrubbed":42,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:55:34.301830+0000","scrub_duration":111,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":36,"num_read_kb":0,"num_write":514,"num_write_kb":86887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":450365,"num_omap_keys":682,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"89'2567","reported_seq":5464,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.890621+0000","last_change":"2024-03-09T09:29:46.890621+0000","last_active":"2024-03-09T09:29:46.890621+0000","last_peered":"2024-03-09T09:29:46.890621+0000","last_clean":"2024-03-09T09:29:46.890621+0000","last_became_active":"2024-03-09T08:52:29.584309+0000","last_became_peered":"2024-03-09T08:52:29.584309+0000","last_unstale":"2024-03-09T09:29:46.890621+0000","last_undegraded":"2024-03-09T09:29:46.890621+0000","last_fullsized":"2024-03-09T09:29:46.890621+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2567","last_scrub_stamp":"2024-03-09T09:29:46.890133+0000","last_deep_scrub":"89'2567","last_deep_scrub_stamp":"2024-03-09T09:29:46.890133+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.890133+0000","objects_scrubbed":147,"log_size":2567,"log_dups_size":0,"ondisk_log_size":2567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:58:06.268182+0000","scrub_duration":357,"objects_trimmed":0,"snaptrim_duration":3.9237505820000003,"stat_sum":{"num_bytes":5533714,"num_objects":147,"num_object_clones":51,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":51,"num_read":136,"num_read_kb":30612,"num_write":1437,"num_write_kb":51393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"89'670","reported_seq":847,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.424152+0000","last_change":"2024-03-09T09:29:40.424152+0000","last_active":"2024-03-09T09:29:40.424152+0000","last_peered":"2024-03-09T09:29:40.424152+0000","last_clean":"2024-03-09T09:29:40.424152+0000","last_became_active":"2024-03-09T08:52:17.284257+0000","last_became_peered":"2024-03-09T08:52:17.284257+0000","last_unstale":"2024-03-09T09:29:40.424152+0000","last_undegraded":"2024-03-09T09:29:40.424152+0000","last_fullsized":"2024-03-09T09:29:40.424152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'670","last_scrub_stamp":"2024-03-09T09:29:40.423973+0000","last_deep_scrub":"89'670","last_deep_scrub_stamp":"2024-03-09T09:29:40.423973+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.423973+0000","objects_scrubbed":50,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:39:53.222882+0000","scrub_duration":127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17956277,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":61,"num_read_kb":0,"num_write":701,"num_write_kb":108966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":563156,"num_omap_keys":856,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"89'2322","reported_seq":5314,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.810202+0000","last_change":"2024-03-09T09:29:35.810202+0000","last_active":"2024-03-09T09:29:35.810202+0000","last_peered":"2024-03-09T09:29:35.810202+0000","last_clean":"2024-03-09T09:29:35.810202+0000","last_became_active":"2024-03-09T08:52:18.229123+0000","last_became_peered":"2024-03-09T08:52:18.229123+0000","last_unstale":"2024-03-09T09:29:35.810202+0000","last_undegraded":"2024-03-09T09:29:35.810202+0000","last_fullsized":"2024-03-09T09:29:35.810202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2322","last_scrub_stamp":"2024-03-09T09:29:35.810044+0000","last_deep_scrub":"89'2322","last_deep_scrub_stamp":"2024-03-09T09:29:35.810044+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.810044+0000","objects_scrubbed":89,"log_size":2322,"log_dups_size":0,"ondisk_log_size":2322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:37:09.823607+0000","scrub_duration":129,"objects_trimmed":0,"snaptrim_duration":3.1639723850000001,"stat_sum":{"num_bytes":538194,"num_objects":89,"num_object_clones":30,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":30,"num_read":112,"num_read_kb":30041,"num_write":1325,"num_write_kb":47681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"89'828","reported_seq":997,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:27.420261+0000","last_change":"2024-03-09T09:29:27.420261+0000","last_active":"2024-03-09T09:29:27.420261+0000","last_peered":"2024-03-09T09:29:27.420261+0000","last_clean":"2024-03-09T09:29:27.420261+0000","last_became_active":"2024-03-09T08:52:28.585401+0000","last_became_peered":"2024-03-09T08:52:28.585401+0000","last_unstale":"2024-03-09T09:29:27.420261+0000","last_undegraded":"2024-03-09T09:29:27.420261+0000","last_fullsized":"2024-03-09T09:29:27.420261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'828","last_scrub_stamp":"2024-03-09T09:29:27.420151+0000","last_deep_scrub":"89'828","last_deep_scrub_stamp":"2024-03-09T09:29:27.420151+0000","last_clean_scrub_stamp":"2024-03-09T09:29:27.420151+0000","objects_scrubbed":60,"log_size":828,"log_dups_size":0,"ondisk_log_size":828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:46:33.569655+0000","scrub_duration":145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":79,"num_read_kb":0,"num_write":862,"num_write_kb":140804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":670100,"num_omap_keys":1071,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"89'2505","reported_seq":5435,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:51.660188+0000","last_change":"2024-03-09T09:29:51.660248+0000","last_active":"2024-03-09T09:29:51.660188+0000","last_peered":"2024-03-09T09:29:51.660188+0000","last_clean":"2024-03-09T09:29:51.660188+0000","last_became_active":"2024-03-09T08:52:29.591931+0000","last_became_peered":"2024-03-09T08:52:29.591931+0000","last_unstale":"2024-03-09T09:29:51.660188+0000","last_undegraded":"2024-03-09T09:29:51.660188+0000","last_fullsized":"2024-03-09T09:29:51.660188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2505","last_scrub_stamp":"2024-03-09T09:29:51.659988+0000","last_deep_scrub":"89'2505","last_deep_scrub_stamp":"2024-03-09T09:29:51.659988+0000","last_clean_scrub_stamp":"2024-03-09T09:29:51.659988+0000","objects_scrubbed":125,"log_size":2505,"log_dups_size":0,"ondisk_log_size":2505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:56:47.553326+0000","scrub_duration":345,"objects_trimmed":4,"snaptrim_duration":4.2173252530000003,"stat_sum":{"num_bytes":794839,"num_objects":125,"num_object_clones":42,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":42,"num_read":135,"num_read_kb":30019,"num_write":1425,"num_write_kb":47795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"89'2375","reported_seq":5368,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.797820+0000","last_change":"2024-03-09T09:29:28.797820+0000","last_active":"2024-03-09T09:29:28.797820+0000","last_peered":"2024-03-09T09:29:28.797820+0000","last_clean":"2024-03-09T09:29:28.797820+0000","last_became_active":"2024-03-09T08:52:18.237595+0000","last_became_peered":"2024-03-09T08:52:18.237595+0000","last_unstale":"2024-03-09T09:29:28.797820+0000","last_undegraded":"2024-03-09T09:29:28.797820+0000","last_fullsized":"2024-03-09T09:29:28.797820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2375","last_scrub_stamp":"2024-03-09T09:29:28.797595+0000","last_deep_scrub":"89'2375","last_deep_scrub_stamp":"2024-03-09T09:29:28.797595+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.797595+0000","objects_scrubbed":120,"log_size":2375,"log_dups_size":0,"ondisk_log_size":2375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:49:05.083497+0000","scrub_duration":320,"objects_trimmed":0,"snaptrim_duration":1.8676528270000001,"stat_sum":{"num_bytes":501236,"num_objects":120,"num_object_clones":38,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":38,"num_read":108,"num_read_kb":17494,"num_write":1356,"num_write_kb":32845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"89'2247","reported_seq":5189,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:48.181969+0000","last_change":"2024-03-09T09:29:48.182083+0000","last_active":"2024-03-09T09:29:48.181969+0000","last_peered":"2024-03-09T09:29:48.181969+0000","last_clean":"2024-03-09T09:29:48.181969+0000","last_became_active":"2024-03-09T08:52:29.600967+0000","last_became_peered":"2024-03-09T08:52:29.600967+0000","last_unstale":"2024-03-09T09:29:48.181969+0000","last_undegraded":"2024-03-09T09:29:48.181969+0000","last_fullsized":"2024-03-09T09:29:48.181969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2247","last_scrub_stamp":"2024-03-09T09:29:48.181828+0000","last_deep_scrub":"89'2247","last_deep_scrub_stamp":"2024-03-09T09:29:48.181828+0000","last_clean_scrub_stamp":"2024-03-09T09:29:48.181828+0000","objects_scrubbed":113,"log_size":2247,"log_dups_size":0,"ondisk_log_size":2247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:21:42.421161+0000","scrub_duration":249,"objects_trimmed":2,"snaptrim_duration":1.7132004649999999,"stat_sum":{"num_bytes":1622605,"num_objects":113,"num_object_clones":36,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":36,"num_read":103,"num_read_kb":9361,"num_write":1283,"num_write_kb":37703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"89'2455","reported_seq":5425,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.616751+0000","last_change":"2024-03-09T09:29:30.616751+0000","last_active":"2024-03-09T09:29:30.616751+0000","last_peered":"2024-03-09T09:29:30.616751+0000","last_clean":"2024-03-09T09:29:30.616751+0000","last_became_active":"2024-03-09T08:52:18.226643+0000","last_became_peered":"2024-03-09T08:52:18.226643+0000","last_unstale":"2024-03-09T09:29:30.616751+0000","last_undegraded":"2024-03-09T09:29:30.616751+0000","last_fullsized":"2024-03-09T09:29:30.616751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2455","last_scrub_stamp":"2024-03-09T09:29:30.616633+0000","last_deep_scrub":"89'2455","last_deep_scrub_stamp":"2024-03-09T09:29:30.616633+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.616633+0000","objects_scrubbed":99,"log_size":2455,"log_dups_size":0,"ondisk_log_size":2455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:37:02.165755+0000","scrub_duration":180,"objects_trimmed":0,"snaptrim_duration":0.73410826500000004,"stat_sum":{"num_bytes":968837,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":110,"num_read_kb":13537,"num_write":1403,"num_write_kb":47183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"89'2377","reported_seq":5277,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.048548+0000","last_change":"2024-03-09T09:29:37.048589+0000","last_active":"2024-03-09T09:29:37.048548+0000","last_peered":"2024-03-09T09:29:37.048548+0000","last_clean":"2024-03-09T09:29:37.048548+0000","last_became_active":"2024-03-09T08:52:29.575296+0000","last_became_peered":"2024-03-09T08:52:29.575296+0000","last_unstale":"2024-03-09T09:29:37.048548+0000","last_undegraded":"2024-03-09T09:29:37.048548+0000","last_fullsized":"2024-03-09T09:29:37.048548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2377","last_scrub_stamp":"2024-03-09T09:29:37.048480+0000","last_deep_scrub":"89'2377","last_deep_scrub_stamp":"2024-03-09T09:29:37.048480+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.048480+0000","objects_scrubbed":126,"log_size":2377,"log_dups_size":0,"ondisk_log_size":2377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:37:30.619929+0000","scrub_duration":197,"objects_trimmed":2,"snaptrim_duration":4.446018724,"stat_sum":{"num_bytes":5408806,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":112,"num_read_kb":29960,"num_write":1313,"num_write_kb":48116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"89'2347","reported_seq":5358,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.082223+0000","last_change":"2024-03-09T09:29:32.082223+0000","last_active":"2024-03-09T09:29:32.082223+0000","last_peered":"2024-03-09T09:29:32.082223+0000","last_clean":"2024-03-09T09:29:32.082223+0000","last_became_active":"2024-03-09T08:52:18.241290+0000","last_became_peered":"2024-03-09T08:52:18.241290+0000","last_unstale":"2024-03-09T09:29:32.082223+0000","last_undegraded":"2024-03-09T09:29:32.082223+0000","last_fullsized":"2024-03-09T09:29:32.082223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2347","last_scrub_stamp":"2024-03-09T09:29:32.082021+0000","last_deep_scrub":"89'2347","last_deep_scrub_stamp":"2024-03-09T09:29:32.082021+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.082021+0000","objects_scrubbed":110,"log_size":2347,"log_dups_size":0,"ondisk_log_size":2347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:38:10.823674+0000","scrub_duration":219,"objects_trimmed":0,"snaptrim_duration":1.706678991,"stat_sum":{"num_bytes":887348,"num_objects":110,"num_object_clones":37,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":37,"num_read":123,"num_read_kb":42348,"num_write":1334,"num_write_kb":66020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"89'2231","reported_seq":5193,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.130714+0000","last_change":"2024-03-09T09:29:43.130785+0000","last_active":"2024-03-09T09:29:43.130714+0000","last_peered":"2024-03-09T09:29:43.130714+0000","last_clean":"2024-03-09T09:29:43.130714+0000","last_became_active":"2024-03-09T08:52:29.588369+0000","last_became_peered":"2024-03-09T08:52:29.588369+0000","last_unstale":"2024-03-09T09:29:43.130714+0000","last_undegraded":"2024-03-09T09:29:43.130714+0000","last_fullsized":"2024-03-09T09:29:43.130714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2231","last_scrub_stamp":"2024-03-09T09:29:43.130621+0000","last_deep_scrub":"89'2231","last_deep_scrub_stamp":"2024-03-09T09:29:43.130621+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.130621+0000","objects_scrubbed":112,"log_size":2231,"log_dups_size":0,"ondisk_log_size":2231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:13:03.579083+0000","scrub_duration":222,"objects_trimmed":0,"snaptrim_duration":1.708282305,"stat_sum":{"num_bytes":596753,"num_objects":112,"num_object_clones":36,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":36,"num_read":111,"num_read_kb":13558,"num_write":1290,"num_write_kb":25078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"89'2507","reported_seq":5441,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:23.111217+0000","last_change":"2024-03-09T09:29:23.111217+0000","last_active":"2024-03-09T09:29:23.111217+0000","last_peered":"2024-03-09T09:29:23.111217+0000","last_clean":"2024-03-09T09:29:23.111217+0000","last_became_active":"2024-03-09T08:52:18.240521+0000","last_became_peered":"2024-03-09T08:52:18.240521+0000","last_unstale":"2024-03-09T09:29:23.111217+0000","last_undegraded":"2024-03-09T09:29:23.111217+0000","last_fullsized":"2024-03-09T09:29:23.111217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2507","last_scrub_stamp":"2024-03-09T09:29:23.111090+0000","last_deep_scrub":"89'2507","last_deep_scrub_stamp":"2024-03-09T09:29:23.111090+0000","last_clean_scrub_stamp":"2024-03-09T09:29:23.111090+0000","objects_scrubbed":112,"log_size":2507,"log_dups_size":0,"ondisk_log_size":2507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:16:12.677859+0000","scrub_duration":172,"objects_trimmed":0,"snaptrim_duration":0.39778581299999999,"stat_sum":{"num_bytes":1231522,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":112,"num_read_kb":16729,"num_write":1390,"num_write_kb":42674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"89'2284","reported_seq":5243,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:53.295284+0000","last_change":"2024-03-09T09:29:53.295284+0000","last_active":"2024-03-09T09:29:53.295284+0000","last_peered":"2024-03-09T09:29:53.295284+0000","last_clean":"2024-03-09T09:29:53.295284+0000","last_became_active":"2024-03-09T08:52:29.594359+0000","last_became_peered":"2024-03-09T08:52:29.594359+0000","last_unstale":"2024-03-09T09:29:53.295284+0000","last_undegraded":"2024-03-09T09:29:53.295284+0000","last_fullsized":"2024-03-09T09:29:53.295284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2284","last_scrub_stamp":"2024-03-09T09:29:53.295198+0000","last_deep_scrub":"89'2284","last_deep_scrub_stamp":"2024-03-09T09:29:53.295198+0000","last_clean_scrub_stamp":"2024-03-09T09:29:53.295198+0000","objects_scrubbed":102,"log_size":2284,"log_dups_size":0,"ondisk_log_size":2284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:39:45.832654+0000","scrub_duration":145,"objects_trimmed":0,"snaptrim_duration":4.4007626340000003,"stat_sum":{"num_bytes":767001,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":115,"num_read_kb":17556,"num_write":1334,"num_write_kb":40745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"89'2353","reported_seq":5331,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.521386+0000","last_change":"2024-03-09T09:29:38.521419+0000","last_active":"2024-03-09T09:29:38.521386+0000","last_peered":"2024-03-09T09:29:38.521386+0000","last_clean":"2024-03-09T09:29:38.521386+0000","last_became_active":"2024-03-09T08:52:18.238848+0000","last_became_peered":"2024-03-09T08:52:18.238848+0000","last_unstale":"2024-03-09T09:29:38.521386+0000","last_undegraded":"2024-03-09T09:29:38.521386+0000","last_fullsized":"2024-03-09T09:29:38.521386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2353","last_scrub_stamp":"2024-03-09T09:29:38.521306+0000","last_deep_scrub":"89'2353","last_deep_scrub_stamp":"2024-03-09T09:29:38.521306+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.521306+0000","objects_scrubbed":126,"log_size":2353,"log_dups_size":0,"ondisk_log_size":2353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:24:47.266752+0000","scrub_duration":250,"objects_trimmed":0,"snaptrim_duration":0.80249947799999999,"stat_sum":{"num_bytes":1587591,"num_objects":126,"num_object_clones":43,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":43,"num_read":108,"num_read_kb":24972,"num_write":1321,"num_write_kb":50275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"89'2530","reported_seq":5390,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.805397+0000","last_change":"2024-03-09T09:29:43.805397+0000","last_active":"2024-03-09T09:29:43.805397+0000","last_peered":"2024-03-09T09:29:43.805397+0000","last_clean":"2024-03-09T09:29:43.805397+0000","last_became_active":"2024-03-09T08:52:29.578713+0000","last_became_peered":"2024-03-09T08:52:29.578713+0000","last_unstale":"2024-03-09T09:29:43.805397+0000","last_undegraded":"2024-03-09T09:29:43.805397+0000","last_fullsized":"2024-03-09T09:29:43.805397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2530","last_scrub_stamp":"2024-03-09T09:29:43.805254+0000","last_deep_scrub":"89'2530","last_deep_scrub_stamp":"2024-03-09T09:29:43.805254+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.805254+0000","objects_scrubbed":108,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:53:27.143897+0000","scrub_duration":244,"objects_trimmed":0,"snaptrim_duration":3.6671456120000001,"stat_sum":{"num_bytes":827266,"num_objects":108,"num_object_clones":36,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":36,"num_read":108,"num_read_kb":21834,"num_write":1396,"num_write_kb":37656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"89'2300","reported_seq":5300,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:28.242654+0000","last_change":"2024-03-09T09:29:28.242727+0000","last_active":"2024-03-09T09:29:28.242654+0000","last_peered":"2024-03-09T09:29:28.242654+0000","last_clean":"2024-03-09T09:29:28.242654+0000","last_became_active":"2024-03-09T08:52:18.240632+0000","last_became_peered":"2024-03-09T08:52:18.240632+0000","last_unstale":"2024-03-09T09:29:28.242654+0000","last_undegraded":"2024-03-09T09:29:28.242654+0000","last_fullsized":"2024-03-09T09:29:28.242654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2300","last_scrub_stamp":"2024-03-09T09:29:28.242562+0000","last_deep_scrub":"89'2300","last_deep_scrub_stamp":"2024-03-09T09:29:28.242562+0000","last_clean_scrub_stamp":"2024-03-09T09:29:28.242562+0000","objects_scrubbed":110,"log_size":2300,"log_dups_size":0,"ondisk_log_size":2300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:28:46.353806+0000","scrub_duration":160,"objects_trimmed":0,"snaptrim_duration":1.410735039,"stat_sum":{"num_bytes":1374461,"num_objects":110,"num_object_clones":35,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":35,"num_read":104,"num_read_kb":29127,"num_write":1325,"num_write_kb":54062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"89'2339","reported_seq":5281,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.341840+0000","last_change":"2024-03-09T09:29:35.341840+0000","last_active":"2024-03-09T09:29:35.341840+0000","last_peered":"2024-03-09T09:29:35.341840+0000","last_clean":"2024-03-09T09:29:35.341840+0000","last_became_active":"2024-03-09T08:52:29.583501+0000","last_became_peered":"2024-03-09T08:52:29.583501+0000","last_unstale":"2024-03-09T09:29:35.341840+0000","last_undegraded":"2024-03-09T09:29:35.341840+0000","last_fullsized":"2024-03-09T09:29:35.341840+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2339","last_scrub_stamp":"2024-03-09T09:29:35.341694+0000","last_deep_scrub":"89'2339","last_deep_scrub_stamp":"2024-03-09T09:29:35.341694+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.341694+0000","objects_scrubbed":117,"log_size":2339,"log_dups_size":0,"ondisk_log_size":2339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:04:43.563628+0000","scrub_duration":157,"objects_trimmed":4,"snaptrim_duration":4.393423125,"stat_sum":{"num_bytes":646888,"num_objects":117,"num_object_clones":37,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":37,"num_read":111,"num_read_kb":21616,"num_write":1343,"num_write_kb":41561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"89'2510","reported_seq":5497,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.644972+0000","last_change":"2024-03-09T09:29:29.645003+0000","last_active":"2024-03-09T09:29:29.644972+0000","last_peered":"2024-03-09T09:29:29.644972+0000","last_clean":"2024-03-09T09:29:29.644972+0000","last_became_active":"2024-03-09T08:52:18.246914+0000","last_became_peered":"2024-03-09T08:52:18.246914+0000","last_unstale":"2024-03-09T09:29:29.644972+0000","last_undegraded":"2024-03-09T09:29:29.644972+0000","last_fullsized":"2024-03-09T09:29:29.644972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2510","last_scrub_stamp":"2024-03-09T09:29:29.644909+0000","last_deep_scrub":"89'2510","last_deep_scrub_stamp":"2024-03-09T09:29:29.644909+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.644909+0000","objects_scrubbed":130,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:35:58.503655+0000","scrub_duration":189,"objects_trimmed":0,"snaptrim_duration":2.440348755,"stat_sum":{"num_bytes":1448398,"num_objects":130,"num_object_clones":44,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":44,"num_read":125,"num_read_kb":18804,"num_write":1435,"num_write_kb":32798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"89'2406","reported_seq":5321,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.416254+0000","last_change":"2024-03-09T09:29:46.416254+0000","last_active":"2024-03-09T09:29:46.416254+0000","last_peered":"2024-03-09T09:29:46.416254+0000","last_clean":"2024-03-09T09:29:46.416254+0000","last_became_active":"2024-03-09T08:52:29.587060+0000","last_became_peered":"2024-03-09T08:52:29.587060+0000","last_unstale":"2024-03-09T09:29:46.416254+0000","last_undegraded":"2024-03-09T09:29:46.416254+0000","last_fullsized":"2024-03-09T09:29:46.416254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2406","last_scrub_stamp":"2024-03-09T09:29:46.416131+0000","last_deep_scrub":"89'2406","last_deep_scrub_stamp":"2024-03-09T09:29:46.416131+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.416131+0000","objects_scrubbed":98,"log_size":2406,"log_dups_size":0,"ondisk_log_size":2406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:14:54.896801+0000","scrub_duration":139,"objects_trimmed":4,"snaptrim_duration":0.64500296700000004,"stat_sum":{"num_bytes":1066603,"num_objects":98,"num_object_clones":32,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":32,"num_read":104,"num_read_kb":18198,"num_write":1399,"num_write_kb":47978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"89'2282","reported_seq":5256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.491786+0000","last_change":"2024-03-09T09:29:30.491850+0000","last_active":"2024-03-09T09:29:30.491786+0000","last_peered":"2024-03-09T09:29:30.491786+0000","last_clean":"2024-03-09T09:29:30.491786+0000","last_became_active":"2024-03-09T08:52:18.247428+0000","last_became_peered":"2024-03-09T08:52:18.247428+0000","last_unstale":"2024-03-09T09:29:30.491786+0000","last_undegraded":"2024-03-09T09:29:30.491786+0000","last_fullsized":"2024-03-09T09:29:30.491786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2282","last_scrub_stamp":"2024-03-09T09:29:30.491684+0000","last_deep_scrub":"89'2282","last_deep_scrub_stamp":"2024-03-09T09:29:30.491684+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.491684+0000","objects_scrubbed":107,"log_size":2282,"log_dups_size":0,"ondisk_log_size":2282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:52:34.304376+0000","scrub_duration":186,"objects_trimmed":0,"snaptrim_duration":0.62298635300000005,"stat_sum":{"num_bytes":1371558,"num_objects":107,"num_object_clones":34,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":34,"num_read":92,"num_read_kb":30679,"num_write":1263,"num_write_kb":63255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"89'2311","reported_seq":5219,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:50.532794+0000","last_change":"2024-03-09T09:29:50.532829+0000","last_active":"2024-03-09T09:29:50.532794+0000","last_peered":"2024-03-09T09:29:50.532794+0000","last_clean":"2024-03-09T09:29:50.532794+0000","last_became_active":"2024-03-09T08:52:29.588929+0000","last_became_peered":"2024-03-09T08:52:29.588929+0000","last_unstale":"2024-03-09T09:29:50.532794+0000","last_undegraded":"2024-03-09T09:29:50.532794+0000","last_fullsized":"2024-03-09T09:29:50.532794+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2311","last_scrub_stamp":"2024-03-09T09:29:50.532728+0000","last_deep_scrub":"89'2311","last_deep_scrub_stamp":"2024-03-09T09:29:50.532728+0000","last_clean_scrub_stamp":"2024-03-09T09:29:50.532728+0000","objects_scrubbed":108,"log_size":2311,"log_dups_size":0,"ondisk_log_size":2311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:01:50.403478+0000","scrub_duration":188,"objects_trimmed":0,"snaptrim_duration":4.1845584630000001,"stat_sum":{"num_bytes":1760124,"num_objects":108,"num_object_clones":35,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":35,"num_read":107,"num_read_kb":13622,"num_write":1279,"num_write_kb":27255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'2317","reported_seq":5312,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.041719+0000","last_change":"2024-03-09T09:29:34.041794+0000","last_active":"2024-03-09T09:29:34.041719+0000","last_peered":"2024-03-09T09:29:34.041719+0000","last_clean":"2024-03-09T09:29:34.041719+0000","last_became_active":"2024-03-09T08:52:18.239656+0000","last_became_peered":"2024-03-09T08:52:18.239656+0000","last_unstale":"2024-03-09T09:29:34.041719+0000","last_undegraded":"2024-03-09T09:29:34.041719+0000","last_fullsized":"2024-03-09T09:29:34.041719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2317","last_scrub_stamp":"2024-03-09T09:29:34.041570+0000","last_deep_scrub":"89'2317","last_deep_scrub_stamp":"2024-03-09T09:29:34.041570+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.041570+0000","objects_scrubbed":118,"log_size":2317,"log_dups_size":0,"ondisk_log_size":2317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:43:34.420743+0000","scrub_duration":202,"objects_trimmed":0,"snaptrim_duration":1.2778432559999999,"stat_sum":{"num_bytes":1308027,"num_objects":118,"num_object_clones":42,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":42,"num_read":112,"num_read_kb":14330,"num_write":1310,"num_write_kb":25655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"89'2444","reported_seq":5354,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:54.316911+0000","last_change":"2024-03-09T09:29:54.316967+0000","last_active":"2024-03-09T09:29:54.316911+0000","last_peered":"2024-03-09T09:29:54.316911+0000","last_clean":"2024-03-09T09:29:54.316911+0000","last_became_active":"2024-03-09T08:52:29.596743+0000","last_became_peered":"2024-03-09T08:52:29.596743+0000","last_unstale":"2024-03-09T09:29:54.316911+0000","last_undegraded":"2024-03-09T09:29:54.316911+0000","last_fullsized":"2024-03-09T09:29:54.316911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2444","last_scrub_stamp":"2024-03-09T09:29:54.316796+0000","last_deep_scrub":"89'2444","last_deep_scrub_stamp":"2024-03-09T09:29:54.316796+0000","last_clean_scrub_stamp":"2024-03-09T09:29:54.316796+0000","objects_scrubbed":115,"log_size":2444,"log_dups_size":0,"ondisk_log_size":2444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:14:12.787600+0000","scrub_duration":157,"objects_trimmed":2,"snaptrim_duration":4.4171876489999997,"stat_sum":{"num_bytes":1480146,"num_objects":115,"num_object_clones":41,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":41,"num_read":120,"num_read_kb":31406,"num_write":1383,"num_write_kb":52041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"89'2289","reported_seq":5300,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.401148+0000","last_change":"2024-03-09T09:29:46.401267+0000","last_active":"2024-03-09T09:29:46.401148+0000","last_peered":"2024-03-09T09:29:46.401148+0000","last_clean":"2024-03-09T09:29:46.401148+0000","last_became_active":"2024-03-09T08:52:18.236347+0000","last_became_peered":"2024-03-09T08:52:18.236347+0000","last_unstale":"2024-03-09T09:29:46.401148+0000","last_undegraded":"2024-03-09T09:29:46.401148+0000","last_fullsized":"2024-03-09T09:29:46.401148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2289","last_scrub_stamp":"2024-03-09T09:29:46.400913+0000","last_deep_scrub":"89'2289","last_deep_scrub_stamp":"2024-03-09T09:29:46.400913+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.400913+0000","objects_scrubbed":114,"log_size":2289,"log_dups_size":0,"ondisk_log_size":2289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:18:06.604961+0000","scrub_duration":192,"objects_trimmed":0,"snaptrim_duration":0.72006586699999997,"stat_sum":{"num_bytes":650831,"num_objects":114,"num_object_clones":39,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":39,"num_read":114,"num_read_kb":46474,"num_write":1326,"num_write_kb":53541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"89'2398","reported_seq":5341,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.745167+0000","last_change":"2024-03-09T09:29:25.745227+0000","last_active":"2024-03-09T09:29:25.745167+0000","last_peered":"2024-03-09T09:29:25.745167+0000","last_clean":"2024-03-09T09:29:25.745167+0000","last_became_active":"2024-03-09T08:52:29.590554+0000","last_became_peered":"2024-03-09T08:52:29.590554+0000","last_unstale":"2024-03-09T09:29:25.745167+0000","last_undegraded":"2024-03-09T09:29:25.745167+0000","last_fullsized":"2024-03-09T09:29:25.745167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2398","last_scrub_stamp":"2024-03-09T09:29:25.745039+0000","last_deep_scrub":"89'2398","last_deep_scrub_stamp":"2024-03-09T09:29:25.745039+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.745039+0000","objects_scrubbed":112,"log_size":2398,"log_dups_size":0,"ondisk_log_size":2398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T14:21:28.894700+0000","scrub_duration":266,"objects_trimmed":2,"snaptrim_duration":0.64248945300000004,"stat_sum":{"num_bytes":509009,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":145,"num_read_kb":34396,"num_write":1348,"num_write_kb":44845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"89'2411","reported_seq":5370,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.040932+0000","last_change":"2024-03-09T09:29:36.041005+0000","last_active":"2024-03-09T09:29:36.040932+0000","last_peered":"2024-03-09T09:29:36.040932+0000","last_clean":"2024-03-09T09:29:36.040932+0000","last_became_active":"2024-03-09T08:52:18.236534+0000","last_became_peered":"2024-03-09T08:52:18.236534+0000","last_unstale":"2024-03-09T09:29:36.040932+0000","last_undegraded":"2024-03-09T09:29:36.040932+0000","last_fullsized":"2024-03-09T09:29:36.040932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2411","last_scrub_stamp":"2024-03-09T09:29:36.040797+0000","last_deep_scrub":"89'2411","last_deep_scrub_stamp":"2024-03-09T09:29:36.040797+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.040797+0000","objects_scrubbed":100,"log_size":2411,"log_dups_size":0,"ondisk_log_size":2411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:12:02.963696+0000","scrub_duration":144,"objects_trimmed":0,"snaptrim_duration":1.291579509,"stat_sum":{"num_bytes":917223,"num_objects":100,"num_object_clones":30,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":30,"num_read":95,"num_read_kb":13353,"num_write":1387,"num_write_kb":42086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"89'2418","reported_seq":5320,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:53.559752+0000","last_change":"2024-03-09T09:29:53.559803+0000","last_active":"2024-03-09T09:29:53.559752+0000","last_peered":"2024-03-09T09:29:53.559752+0000","last_clean":"2024-03-09T09:29:53.559752+0000","last_became_active":"2024-03-09T08:52:29.601630+0000","last_became_peered":"2024-03-09T08:52:29.601630+0000","last_unstale":"2024-03-09T09:29:53.559752+0000","last_undegraded":"2024-03-09T09:29:53.559752+0000","last_fullsized":"2024-03-09T09:29:53.559752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2418","last_scrub_stamp":"2024-03-09T09:29:53.559673+0000","last_deep_scrub":"89'2418","last_deep_scrub_stamp":"2024-03-09T09:29:53.559673+0000","last_clean_scrub_stamp":"2024-03-09T09:29:53.559673+0000","objects_scrubbed":97,"log_size":2418,"log_dups_size":0,"ondisk_log_size":2418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:00:17.803009+0000","scrub_duration":215,"objects_trimmed":2,"snaptrim_duration":4.2066621079999997,"stat_sum":{"num_bytes":1356079,"num_objects":97,"num_object_clones":28,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":28,"num_read":106,"num_read_kb":29896,"num_write":1386,"num_write_kb":51017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"89'2345","reported_seq":5348,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:25.220830+0000","last_change":"2024-03-09T09:29:25.220935+0000","last_active":"2024-03-09T09:29:25.220830+0000","last_peered":"2024-03-09T09:29:25.220830+0000","last_clean":"2024-03-09T09:29:25.220830+0000","last_became_active":"2024-03-09T08:52:18.238112+0000","last_became_peered":"2024-03-09T08:52:18.238112+0000","last_unstale":"2024-03-09T09:29:25.220830+0000","last_undegraded":"2024-03-09T09:29:25.220830+0000","last_fullsized":"2024-03-09T09:29:25.220830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2345","last_scrub_stamp":"2024-03-09T09:29:25.220712+0000","last_deep_scrub":"89'2345","last_deep_scrub_stamp":"2024-03-09T09:29:25.220712+0000","last_clean_scrub_stamp":"2024-03-09T09:29:25.220712+0000","objects_scrubbed":121,"log_size":2345,"log_dups_size":0,"ondisk_log_size":2345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:29:53.288594+0000","scrub_duration":262,"objects_trimmed":0,"snaptrim_duration":0.313168905,"stat_sum":{"num_bytes":1092268,"num_objects":121,"num_object_clones":43,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":43,"num_read":123,"num_read_kb":25979,"num_write":1341,"num_write_kb":58516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"89'2398","reported_seq":5310,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:42.808913+0000","last_change":"2024-03-09T09:29:42.808976+0000","last_active":"2024-03-09T09:29:42.808913+0000","last_peered":"2024-03-09T09:29:42.808913+0000","last_clean":"2024-03-09T09:29:42.808913+0000","last_became_active":"2024-03-09T08:52:29.577848+0000","last_became_peered":"2024-03-09T08:52:29.577848+0000","last_unstale":"2024-03-09T09:29:42.808913+0000","last_undegraded":"2024-03-09T09:29:42.808913+0000","last_fullsized":"2024-03-09T09:29:42.808913+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2398","last_scrub_stamp":"2024-03-09T09:29:42.808820+0000","last_deep_scrub":"89'2398","last_deep_scrub_stamp":"2024-03-09T09:29:42.808820+0000","last_clean_scrub_stamp":"2024-03-09T09:29:42.808820+0000","objects_scrubbed":95,"log_size":2398,"log_dups_size":0,"ondisk_log_size":2398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:58:20.828745+0000","scrub_duration":224,"objects_trimmed":6,"snaptrim_duration":3.9232747450000001,"stat_sum":{"num_bytes":5183706,"num_objects":95,"num_object_clones":27,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":111,"num_read_kb":34166,"num_write":1371,"num_write_kb":49768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"89'2364","reported_seq":5337,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.609331+0000","last_change":"2024-03-09T09:29:30.609381+0000","last_active":"2024-03-09T09:29:30.609331+0000","last_peered":"2024-03-09T09:29:30.609331+0000","last_clean":"2024-03-09T09:29:30.609331+0000","last_became_active":"2024-03-09T08:52:18.235252+0000","last_became_peered":"2024-03-09T08:52:18.235252+0000","last_unstale":"2024-03-09T09:29:30.609331+0000","last_undegraded":"2024-03-09T09:29:30.609331+0000","last_fullsized":"2024-03-09T09:29:30.609331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2364","last_scrub_stamp":"2024-03-09T09:29:30.609256+0000","last_deep_scrub":"89'2364","last_deep_scrub_stamp":"2024-03-09T09:29:30.609256+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.609256+0000","objects_scrubbed":117,"log_size":2364,"log_dups_size":0,"ondisk_log_size":2364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:52:13.442415+0000","scrub_duration":168,"objects_trimmed":0,"snaptrim_duration":2.1326348209999999,"stat_sum":{"num_bytes":1866711,"num_objects":117,"num_object_clones":40,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":40,"num_read":109,"num_read_kb":17692,"num_write":1315,"num_write_kb":41853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"89'2322","reported_seq":5272,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:44.834916+0000","last_change":"2024-03-09T09:29:44.835021+0000","last_active":"2024-03-09T09:29:44.834916+0000","last_peered":"2024-03-09T09:29:44.834916+0000","last_clean":"2024-03-09T09:29:44.834916+0000","last_became_active":"2024-03-09T08:52:29.579346+0000","last_became_peered":"2024-03-09T08:52:29.579346+0000","last_unstale":"2024-03-09T09:29:44.834916+0000","last_undegraded":"2024-03-09T09:29:44.834916+0000","last_fullsized":"2024-03-09T09:29:44.834916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2322","last_scrub_stamp":"2024-03-09T09:29:44.834720+0000","last_deep_scrub":"89'2322","last_deep_scrub_stamp":"2024-03-09T09:29:44.834720+0000","last_clean_scrub_stamp":"2024-03-09T09:29:44.834720+0000","objects_scrubbed":123,"log_size":2322,"log_dups_size":0,"ondisk_log_size":2322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:53:24.127536+0000","scrub_duration":316,"objects_trimmed":0,"snaptrim_duration":3.5970959150000001,"stat_sum":{"num_bytes":1348477,"num_objects":123,"num_object_clones":45,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":45,"num_read":126,"num_read_kb":8825,"num_write":1341,"num_write_kb":33612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"89'2764","reported_seq":5598,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:48.357200+0000","last_change":"2024-03-09T09:29:48.357265+0000","last_active":"2024-03-09T09:29:48.357200+0000","last_peered":"2024-03-09T09:29:48.357200+0000","last_clean":"2024-03-09T09:29:48.357200+0000","last_became_active":"2024-03-09T08:52:18.239514+0000","last_became_peered":"2024-03-09T08:52:18.239514+0000","last_unstale":"2024-03-09T09:29:48.357200+0000","last_undegraded":"2024-03-09T09:29:48.357200+0000","last_fullsized":"2024-03-09T09:29:48.357200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2764","last_scrub_stamp":"2024-03-09T09:29:48.357121+0000","last_deep_scrub":"89'2764","last_deep_scrub_stamp":"2024-03-09T09:29:48.357121+0000","last_clean_scrub_stamp":"2024-03-09T09:29:48.357121+0000","objects_scrubbed":92,"log_size":2764,"log_dups_size":0,"ondisk_log_size":2764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:29:25.057956+0000","scrub_duration":188,"objects_trimmed":0,"snaptrim_duration":0.49950316500000003,"stat_sum":{"num_bytes":924300,"num_objects":92,"num_object_clones":25,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":25,"num_read":107,"num_read_kb":17449,"num_write":1480,"num_write_kb":44901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"89'2302","reported_seq":5262,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:51.243015+0000","last_change":"2024-03-09T09:29:51.243076+0000","last_active":"2024-03-09T09:29:51.243015+0000","last_peered":"2024-03-09T09:29:51.243015+0000","last_clean":"2024-03-09T09:29:51.243015+0000","last_became_active":"2024-03-09T08:52:29.586669+0000","last_became_peered":"2024-03-09T08:52:29.586669+0000","last_unstale":"2024-03-09T09:29:51.243015+0000","last_undegraded":"2024-03-09T09:29:51.243015+0000","last_fullsized":"2024-03-09T09:29:51.243015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2302","last_scrub_stamp":"2024-03-09T09:29:51.242923+0000","last_deep_scrub":"89'2302","last_deep_scrub_stamp":"2024-03-09T09:29:51.242923+0000","last_clean_scrub_stamp":"2024-03-09T09:29:51.242923+0000","objects_scrubbed":127,"log_size":2302,"log_dups_size":0,"ondisk_log_size":2302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:14:25.471485+0000","scrub_duration":296,"objects_trimmed":0,"snaptrim_duration":4.4099735320000004,"stat_sum":{"num_bytes":773517,"num_objects":127,"num_object_clones":46,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":46,"num_read":120,"num_read_kb":5950,"num_write":1331,"num_write_kb":13298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"89'2263","reported_seq":5262,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.714031+0000","last_change":"2024-03-09T09:29:31.714031+0000","last_active":"2024-03-09T09:29:31.714031+0000","last_peered":"2024-03-09T09:29:31.714031+0000","last_clean":"2024-03-09T09:29:31.714031+0000","last_became_active":"2024-03-09T08:52:18.233082+0000","last_became_peered":"2024-03-09T08:52:18.233082+0000","last_unstale":"2024-03-09T09:29:31.714031+0000","last_undegraded":"2024-03-09T09:29:31.714031+0000","last_fullsized":"2024-03-09T09:29:31.714031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2263","last_scrub_stamp":"2024-03-09T09:29:31.713804+0000","last_deep_scrub":"89'2263","last_deep_scrub_stamp":"2024-03-09T09:29:31.713804+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.713804+0000","objects_scrubbed":101,"log_size":2263,"log_dups_size":0,"ondisk_log_size":2263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:38:28.086397+0000","scrub_duration":247,"objects_trimmed":0,"snaptrim_duration":0.89838860799999998,"stat_sum":{"num_bytes":6604987,"num_objects":101,"num_object_clones":30,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":30,"num_read":109,"num_read_kb":14504,"num_write":1281,"num_write_kb":44083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"89'2258","reported_seq":5301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.156349+0000","last_change":"2024-03-09T09:29:40.156383+0000","last_active":"2024-03-09T09:29:40.156349+0000","last_peered":"2024-03-09T09:29:40.156349+0000","last_clean":"2024-03-09T09:29:40.156349+0000","last_became_active":"2024-03-09T08:52:29.584653+0000","last_became_peered":"2024-03-09T08:52:29.584653+0000","last_unstale":"2024-03-09T09:29:40.156349+0000","last_undegraded":"2024-03-09T09:29:40.156349+0000","last_fullsized":"2024-03-09T09:29:40.156349+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2258","last_scrub_stamp":"2024-03-09T09:29:40.156262+0000","last_deep_scrub":"89'2258","last_deep_scrub_stamp":"2024-03-09T09:29:40.156262+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.156262+0000","objects_scrubbed":99,"log_size":2258,"log_dups_size":0,"ondisk_log_size":2258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:12:59.205500+0000","scrub_duration":158,"objects_trimmed":0,"snaptrim_duration":0.71096335899999996,"stat_sum":{"num_bytes":637081,"num_objects":99,"num_object_clones":33,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":33,"num_read":204,"num_read_kb":24036,"num_write":1319,"num_write_kb":30441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"89'2452","reported_seq":5408,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:39.725646+0000","last_change":"2024-03-09T09:29:39.725731+0000","last_active":"2024-03-09T09:29:39.725646+0000","last_peered":"2024-03-09T09:29:39.725646+0000","last_clean":"2024-03-09T09:29:39.725646+0000","last_became_active":"2024-03-09T08:53:00.240319+0000","last_became_peered":"2024-03-09T08:53:00.240319+0000","last_unstale":"2024-03-09T09:29:39.725646+0000","last_undegraded":"2024-03-09T09:29:39.725646+0000","last_fullsized":"2024-03-09T09:29:39.725646+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2452","last_scrub_stamp":"2024-03-09T09:29:39.725567+0000","last_deep_scrub":"89'2452","last_deep_scrub_stamp":"2024-03-09T09:29:39.725567+0000","last_clean_scrub_stamp":"2024-03-09T09:29:39.725567+0000","objects_scrubbed":116,"log_size":2452,"log_dups_size":0,"ondisk_log_size":2452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:51:10.590974+0000","scrub_duration":202,"objects_trimmed":0,"snaptrim_duration":0.86880120699999996,"stat_sum":{"num_bytes":1614659,"num_objects":116,"num_object_clones":42,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":42,"num_read":120,"num_read_kb":18915,"num_write":1382,"num_write_kb":39172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"89'2231","reported_seq":5192,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:46.141554+0000","last_change":"2024-03-09T09:29:46.141590+0000","last_active":"2024-03-09T09:29:46.141554+0000","last_peered":"2024-03-09T09:29:46.141554+0000","last_clean":"2024-03-09T09:29:46.141554+0000","last_became_active":"2024-03-09T08:52:29.592432+0000","last_became_peered":"2024-03-09T08:52:29.592432+0000","last_unstale":"2024-03-09T09:29:46.141554+0000","last_undegraded":"2024-03-09T09:29:46.141554+0000","last_fullsized":"2024-03-09T09:29:46.141554+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2231","last_scrub_stamp":"2024-03-09T09:29:46.141469+0000","last_deep_scrub":"89'2231","last_deep_scrub_stamp":"2024-03-09T09:29:46.141469+0000","last_clean_scrub_stamp":"2024-03-09T09:29:46.141469+0000","objects_scrubbed":102,"log_size":2231,"log_dups_size":0,"ondisk_log_size":2231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:30:37.575836+0000","scrub_duration":191,"objects_trimmed":0,"snaptrim_duration":1.7177660380000002,"stat_sum":{"num_bytes":924110,"num_objects":102,"num_object_clones":34,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":34,"num_read":129,"num_read_kb":10365,"num_write":1260,"num_write_kb":33897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"89'2330","reported_seq":5321,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:45.101745+0000","last_change":"2024-03-09T09:29:45.101779+0000","last_active":"2024-03-09T09:29:45.101745+0000","last_peered":"2024-03-09T09:29:45.101745+0000","last_clean":"2024-03-09T09:29:45.101745+0000","last_became_active":"2024-03-09T08:52:18.225318+0000","last_became_peered":"2024-03-09T08:52:18.225318+0000","last_unstale":"2024-03-09T09:29:45.101745+0000","last_undegraded":"2024-03-09T09:29:45.101745+0000","last_fullsized":"2024-03-09T09:29:45.101745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2330","last_scrub_stamp":"2024-03-09T09:29:45.101686+0000","last_deep_scrub":"89'2330","last_deep_scrub_stamp":"2024-03-09T09:29:45.101686+0000","last_clean_scrub_stamp":"2024-03-09T09:29:45.101686+0000","objects_scrubbed":103,"log_size":2330,"log_dups_size":0,"ondisk_log_size":2330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:59:07.019534+0000","scrub_duration":190,"objects_trimmed":0,"snaptrim_duration":1.2105350770000001,"stat_sum":{"num_bytes":1326786,"num_objects":103,"num_object_clones":34,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":34,"num_read":111,"num_read_kb":18735,"num_write":1322,"num_write_kb":41042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"89'2467","reported_seq":5383,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:47.766832+0000","last_change":"2024-03-09T09:29:47.766884+0000","last_active":"2024-03-09T09:29:47.766832+0000","last_peered":"2024-03-09T09:29:47.766832+0000","last_clean":"2024-03-09T09:29:47.766832+0000","last_became_active":"2024-03-09T08:52:29.607865+0000","last_became_peered":"2024-03-09T08:52:29.607865+0000","last_unstale":"2024-03-09T09:29:47.766832+0000","last_undegraded":"2024-03-09T09:29:47.766832+0000","last_fullsized":"2024-03-09T09:29:47.766832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2467","last_scrub_stamp":"2024-03-09T09:29:47.766733+0000","last_deep_scrub":"89'2467","last_deep_scrub_stamp":"2024-03-09T09:29:47.766733+0000","last_clean_scrub_stamp":"2024-03-09T09:29:47.766733+0000","objects_scrubbed":111,"log_size":2467,"log_dups_size":0,"ondisk_log_size":2467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T09:37:32.471575+0000","scrub_duration":269,"objects_trimmed":4,"snaptrim_duration":4.0059579840000001,"stat_sum":{"num_bytes":936699,"num_objects":111,"num_object_clones":39,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":39,"num_read":129,"num_read_kb":34540,"num_write":1403,"num_write_kb":57337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"89'2378","reported_seq":5363,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.146738+0000","last_change":"2024-03-09T09:29:32.146738+0000","last_active":"2024-03-09T09:29:32.146738+0000","last_peered":"2024-03-09T09:29:32.146738+0000","last_clean":"2024-03-09T09:29:32.146738+0000","last_became_active":"2024-03-09T08:52:18.235786+0000","last_became_peered":"2024-03-09T08:52:18.235786+0000","last_unstale":"2024-03-09T09:29:32.146738+0000","last_undegraded":"2024-03-09T09:29:32.146738+0000","last_fullsized":"2024-03-09T09:29:32.146738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2378","last_scrub_stamp":"2024-03-09T09:29:32.146551+0000","last_deep_scrub":"89'2378","last_deep_scrub_stamp":"2024-03-09T09:29:32.146551+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.146551+0000","objects_scrubbed":101,"log_size":2378,"log_dups_size":0,"ondisk_log_size":2378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T19:49:53.410721+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0.91582108900000003,"stat_sum":{"num_bytes":1759504,"num_objects":101,"num_object_clones":31,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":31,"num_read":108,"num_read_kb":5845,"num_write":1367,"num_write_kb":19152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"89'2471","reported_seq":5368,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.085873+0000","last_change":"2024-03-09T09:29:35.085955+0000","last_active":"2024-03-09T09:29:35.085873+0000","last_peered":"2024-03-09T09:29:35.085873+0000","last_clean":"2024-03-09T09:29:35.085873+0000","last_became_active":"2024-03-09T08:52:29.573372+0000","last_became_peered":"2024-03-09T08:52:29.573372+0000","last_unstale":"2024-03-09T09:29:35.085873+0000","last_undegraded":"2024-03-09T09:29:35.085873+0000","last_fullsized":"2024-03-09T09:29:35.085873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2471","last_scrub_stamp":"2024-03-09T09:29:35.085785+0000","last_deep_scrub":"89'2471","last_deep_scrub_stamp":"2024-03-09T09:29:35.085785+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.085785+0000","objects_scrubbed":122,"log_size":2471,"log_dups_size":0,"ondisk_log_size":2471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:59:32.359356+0000","scrub_duration":207,"objects_trimmed":6,"snaptrim_duration":4.4857478520000003,"stat_sum":{"num_bytes":2540930,"num_objects":122,"num_object_clones":40,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":40,"num_read":127,"num_read_kb":51469,"num_write":1352,"num_write_kb":80787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"89'2314","reported_seq":5303,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.153266+0000","last_change":"2024-03-09T09:29:34.153266+0000","last_active":"2024-03-09T09:29:34.153266+0000","last_peered":"2024-03-09T09:29:34.153266+0000","last_clean":"2024-03-09T09:29:34.153266+0000","last_became_active":"2024-03-09T08:52:18.227694+0000","last_became_peered":"2024-03-09T08:52:18.227694+0000","last_unstale":"2024-03-09T09:29:34.153266+0000","last_undegraded":"2024-03-09T09:29:34.153266+0000","last_fullsized":"2024-03-09T09:29:34.153266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2314","last_scrub_stamp":"2024-03-09T09:29:34.153148+0000","last_deep_scrub":"89'2314","last_deep_scrub_stamp":"2024-03-09T09:29:34.153148+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.153148+0000","objects_scrubbed":96,"log_size":2314,"log_dups_size":0,"ondisk_log_size":2314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:05:43.864141+0000","scrub_duration":133,"objects_trimmed":0,"snaptrim_duration":3.1939797309999998,"stat_sum":{"num_bytes":1585107,"num_objects":96,"num_object_clones":31,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":31,"num_read":96,"num_read_kb":18307,"num_write":1352,"num_write_kb":50489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"89'2804","reported_seq":5569,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:56.532260+0000","last_change":"2024-03-09T09:29:56.532329+0000","last_active":"2024-03-09T09:29:56.532260+0000","last_peered":"2024-03-09T09:29:56.532260+0000","last_clean":"2024-03-09T09:29:56.532260+0000","last_became_active":"2024-03-09T08:52:29.602990+0000","last_became_peered":"2024-03-09T08:52:29.602990+0000","last_unstale":"2024-03-09T09:29:56.532260+0000","last_undegraded":"2024-03-09T09:29:56.532260+0000","last_fullsized":"2024-03-09T09:29:56.532260+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2804","last_scrub_stamp":"2024-03-09T09:29:56.532161+0000","last_deep_scrub":"89'2804","last_deep_scrub_stamp":"2024-03-09T09:29:56.532161+0000","last_clean_scrub_stamp":"2024-03-09T09:29:56.532161+0000","objects_scrubbed":122,"log_size":2804,"log_dups_size":0,"ondisk_log_size":2804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:12:57.354249+0000","scrub_duration":328,"objects_trimmed":0,"snaptrim_duration":4.4197025170000002,"stat_sum":{"num_bytes":1559225,"num_objects":122,"num_object_clones":42,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":42,"num_read":122,"num_read_kb":6495,"num_write":1460,"num_write_kb":38738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"89'2342","reported_seq":5327,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:34.551463+0000","last_change":"2024-03-09T09:29:34.551559+0000","last_active":"2024-03-09T09:29:34.551463+0000","last_peered":"2024-03-09T09:29:34.551463+0000","last_clean":"2024-03-09T09:29:34.551463+0000","last_became_active":"2024-03-09T08:52:18.243197+0000","last_became_peered":"2024-03-09T08:52:18.243197+0000","last_unstale":"2024-03-09T09:29:34.551463+0000","last_undegraded":"2024-03-09T09:29:34.551463+0000","last_fullsized":"2024-03-09T09:29:34.551463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2342","last_scrub_stamp":"2024-03-09T09:29:34.551354+0000","last_deep_scrub":"89'2342","last_deep_scrub_stamp":"2024-03-09T09:29:34.551354+0000","last_clean_scrub_stamp":"2024-03-09T09:29:34.551354+0000","objects_scrubbed":117,"log_size":2342,"log_dups_size":0,"ondisk_log_size":2342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:08:23.847477+0000","scrub_duration":251,"objects_trimmed":0,"snaptrim_duration":0.94133736700000004,"stat_sum":{"num_bytes":314683,"num_objects":117,"num_object_clones":39,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":39,"num_read":116,"num_read_kb":13620,"num_write":1329,"num_write_kb":42753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"89'2293","reported_seq":5316,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:49.476394+0000","last_change":"2024-03-09T09:29:49.476394+0000","last_active":"2024-03-09T09:29:49.476394+0000","last_peered":"2024-03-09T09:29:49.476394+0000","last_clean":"2024-03-09T09:29:49.476394+0000","last_became_active":"2024-03-09T08:52:29.587123+0000","last_became_peered":"2024-03-09T08:52:29.587123+0000","last_unstale":"2024-03-09T09:29:49.476394+0000","last_undegraded":"2024-03-09T09:29:49.476394+0000","last_fullsized":"2024-03-09T09:29:49.476394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2293","last_scrub_stamp":"2024-03-09T09:29:49.476186+0000","last_deep_scrub":"89'2293","last_deep_scrub_stamp":"2024-03-09T09:29:49.476186+0000","last_clean_scrub_stamp":"2024-03-09T09:29:49.476186+0000","objects_scrubbed":95,"log_size":2293,"log_dups_size":0,"ondisk_log_size":2293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:28:49.578823+0000","scrub_duration":150,"objects_trimmed":4,"snaptrim_duration":4.1785381859999999,"stat_sum":{"num_bytes":1532500,"num_objects":95,"num_object_clones":30,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":30,"num_read":195,"num_read_kb":18044,"num_write":1327,"num_write_kb":39414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"89'2396","reported_seq":5379,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.852175+0000","last_change":"2024-03-09T09:29:36.852219+0000","last_active":"2024-03-09T09:29:36.852175+0000","last_peered":"2024-03-09T09:29:36.852175+0000","last_clean":"2024-03-09T09:29:36.852175+0000","last_became_active":"2024-03-09T08:52:18.231505+0000","last_became_peered":"2024-03-09T08:52:18.231505+0000","last_unstale":"2024-03-09T09:29:36.852175+0000","last_undegraded":"2024-03-09T09:29:36.852175+0000","last_fullsized":"2024-03-09T09:29:36.852175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2396","last_scrub_stamp":"2024-03-09T09:29:36.852066+0000","last_deep_scrub":"89'2396","last_deep_scrub_stamp":"2024-03-09T09:29:36.852066+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.852066+0000","objects_scrubbed":130,"log_size":2396,"log_dups_size":0,"ondisk_log_size":2396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:11:10.627586+0000","scrub_duration":176,"objects_trimmed":0,"snaptrim_duration":3.298540461,"stat_sum":{"num_bytes":2039183,"num_objects":130,"num_object_clones":43,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":43,"num_read":126,"num_read_kb":25377,"num_write":1340,"num_write_kb":50110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"89'2329","reported_seq":5261,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.731308+0000","last_change":"2024-03-09T09:29:36.731381+0000","last_active":"2024-03-09T09:29:36.731308+0000","last_peered":"2024-03-09T09:29:36.731308+0000","last_clean":"2024-03-09T09:29:36.731308+0000","last_became_active":"2024-03-09T08:52:29.599327+0000","last_became_peered":"2024-03-09T08:52:29.599327+0000","last_unstale":"2024-03-09T09:29:36.731308+0000","last_undegraded":"2024-03-09T09:29:36.731308+0000","last_fullsized":"2024-03-09T09:29:36.731308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2329","last_scrub_stamp":"2024-03-09T09:29:36.731193+0000","last_deep_scrub":"89'2329","last_deep_scrub_stamp":"2024-03-09T09:29:36.731193+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.731193+0000","objects_scrubbed":120,"log_size":2329,"log_dups_size":0,"ondisk_log_size":2329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:17:35.549238+0000","scrub_duration":183,"objects_trimmed":0,"snaptrim_duration":0.62967200199999995,"stat_sum":{"num_bytes":347019,"num_objects":120,"num_object_clones":41,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":41,"num_read":119,"num_read_kb":17844,"num_write":1302,"num_write_kb":51829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"89'2770","reported_seq":5621,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.538685+0000","last_change":"2024-03-09T09:29:36.538725+0000","last_active":"2024-03-09T09:29:36.538685+0000","last_peered":"2024-03-09T09:29:36.538685+0000","last_clean":"2024-03-09T09:29:36.538685+0000","last_became_active":"2024-03-09T08:52:18.233519+0000","last_became_peered":"2024-03-09T08:52:18.233519+0000","last_unstale":"2024-03-09T09:29:36.538685+0000","last_undegraded":"2024-03-09T09:29:36.538685+0000","last_fullsized":"2024-03-09T09:29:36.538685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2770","last_scrub_stamp":"2024-03-09T09:29:36.538615+0000","last_deep_scrub":"89'2770","last_deep_scrub_stamp":"2024-03-09T09:29:36.538615+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.538615+0000","objects_scrubbed":112,"log_size":2770,"log_dups_size":0,"ondisk_log_size":2770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:57:17.020890+0000","scrub_duration":157,"objects_trimmed":0,"snaptrim_duration":2.0660642299999998,"stat_sum":{"num_bytes":1365000,"num_objects":112,"num_object_clones":38,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":38,"num_read":127,"num_read_kb":33469,"num_write":1475,"num_write_kb":57156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"89'2340","reported_seq":5280,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:55.432876+0000","last_change":"2024-03-09T09:29:55.432925+0000","last_active":"2024-03-09T09:29:55.432876+0000","last_peered":"2024-03-09T09:29:55.432876+0000","last_clean":"2024-03-09T09:29:55.432876+0000","last_became_active":"2024-03-09T08:52:29.597218+0000","last_became_peered":"2024-03-09T08:52:29.597218+0000","last_unstale":"2024-03-09T09:29:55.432876+0000","last_undegraded":"2024-03-09T09:29:55.432876+0000","last_fullsized":"2024-03-09T09:29:55.432876+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2340","last_scrub_stamp":"2024-03-09T09:29:55.432788+0000","last_deep_scrub":"89'2340","last_deep_scrub_stamp":"2024-03-09T09:29:55.432788+0000","last_clean_scrub_stamp":"2024-03-09T09:29:55.432788+0000","objects_scrubbed":109,"log_size":2340,"log_dups_size":0,"ondisk_log_size":2340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:48:15.104084+0000","scrub_duration":267,"objects_trimmed":0,"snaptrim_duration":4.4162208879999998,"stat_sum":{"num_bytes":1481141,"num_objects":109,"num_object_clones":37,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":37,"num_read":120,"num_read_kb":25693,"num_write":1333,"num_write_kb":50297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'2302","reported_seq":5294,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.522536+0000","last_change":"2024-03-09T09:29:38.522573+0000","last_active":"2024-03-09T09:29:38.522536+0000","last_peered":"2024-03-09T09:29:38.522536+0000","last_clean":"2024-03-09T09:29:38.522536+0000","last_became_active":"2024-03-09T08:52:18.234208+0000","last_became_peered":"2024-03-09T08:52:18.234208+0000","last_unstale":"2024-03-09T09:29:38.522536+0000","last_undegraded":"2024-03-09T09:29:38.522536+0000","last_fullsized":"2024-03-09T09:29:38.522536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2302","last_scrub_stamp":"2024-03-09T09:29:38.522458+0000","last_deep_scrub":"89'2302","last_deep_scrub_stamp":"2024-03-09T09:29:38.522458+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.522458+0000","objects_scrubbed":110,"log_size":2302,"log_dups_size":0,"ondisk_log_size":2302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:09:19.242940+0000","scrub_duration":165,"objects_trimmed":0,"snaptrim_duration":2.379810049,"stat_sum":{"num_bytes":472147,"num_objects":110,"num_object_clones":38,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":38,"num_read":115,"num_read_kb":25809,"num_write":1277,"num_write_kb":34061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"89'2409","reported_seq":5344,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:49.092025+0000","last_change":"2024-03-09T09:29:49.092025+0000","last_active":"2024-03-09T09:29:49.092025+0000","last_peered":"2024-03-09T09:29:49.092025+0000","last_clean":"2024-03-09T09:29:49.092025+0000","last_became_active":"2024-03-09T08:52:29.580058+0000","last_became_peered":"2024-03-09T08:52:29.580058+0000","last_unstale":"2024-03-09T09:29:49.092025+0000","last_undegraded":"2024-03-09T09:29:49.092025+0000","last_fullsized":"2024-03-09T09:29:49.092025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2409","last_scrub_stamp":"2024-03-09T09:29:49.091914+0000","last_deep_scrub":"89'2409","last_deep_scrub_stamp":"2024-03-09T09:29:49.091914+0000","last_clean_scrub_stamp":"2024-03-09T09:29:49.091914+0000","objects_scrubbed":99,"log_size":2409,"log_dups_size":0,"ondisk_log_size":2409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:47:15.002886+0000","scrub_duration":160,"objects_trimmed":0,"snaptrim_duration":4.4238349550000002,"stat_sum":{"num_bytes":1560364,"num_objects":99,"num_object_clones":31,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":31,"num_read":120,"num_read_kb":23106,"num_write":1406,"num_write_kb":36148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"89'2461","reported_seq":5435,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:33.333891+0000","last_change":"2024-03-09T09:29:33.333923+0000","last_active":"2024-03-09T09:29:33.333891+0000","last_peered":"2024-03-09T09:29:33.333891+0000","last_clean":"2024-03-09T09:29:33.333891+0000","last_became_active":"2024-03-09T08:52:18.252383+0000","last_became_peered":"2024-03-09T08:52:18.252383+0000","last_unstale":"2024-03-09T09:29:33.333891+0000","last_undegraded":"2024-03-09T09:29:33.333891+0000","last_fullsized":"2024-03-09T09:29:33.333891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2461","last_scrub_stamp":"2024-03-09T09:29:33.333818+0000","last_deep_scrub":"89'2461","last_deep_scrub_stamp":"2024-03-09T09:29:33.333818+0000","last_clean_scrub_stamp":"2024-03-09T09:29:33.333818+0000","objects_scrubbed":106,"log_size":2461,"log_dups_size":0,"ondisk_log_size":2461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T21:06:25.505859+0000","scrub_duration":154,"objects_trimmed":0,"snaptrim_duration":0.73072901000000001,"stat_sum":{"num_bytes":1439516,"num_objects":106,"num_object_clones":37,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":37,"num_read":117,"num_read_kb":43194,"num_write":1390,"num_write_kb":63251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"89'2203","reported_seq":5139,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:37.436790+0000","last_change":"2024-03-09T09:29:37.436790+0000","last_active":"2024-03-09T09:29:37.436790+0000","last_peered":"2024-03-09T09:29:37.436790+0000","last_clean":"2024-03-09T09:29:37.436790+0000","last_became_active":"2024-03-09T08:52:29.578241+0000","last_became_peered":"2024-03-09T08:52:29.578241+0000","last_unstale":"2024-03-09T09:29:37.436790+0000","last_undegraded":"2024-03-09T09:29:37.436790+0000","last_fullsized":"2024-03-09T09:29:37.436790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2203","last_scrub_stamp":"2024-03-09T09:29:37.436692+0000","last_deep_scrub":"89'2203","last_deep_scrub_stamp":"2024-03-09T09:29:37.436692+0000","last_clean_scrub_stamp":"2024-03-09T09:29:37.436692+0000","objects_scrubbed":91,"log_size":2203,"log_dups_size":0,"ondisk_log_size":2203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:51:14.275290+0000","scrub_duration":154,"objects_trimmed":4,"snaptrim_duration":4.2163343510000004,"stat_sum":{"num_bytes":526581,"num_objects":91,"num_object_clones":29,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":29,"num_read":100,"num_read_kb":25429,"num_write":1233,"num_write_kb":64615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"89'2261","reported_seq":5249,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:36.400325+0000","last_change":"2024-03-09T09:29:36.400325+0000","last_active":"2024-03-09T09:29:36.400325+0000","last_peered":"2024-03-09T09:29:36.400325+0000","last_clean":"2024-03-09T09:29:36.400325+0000","last_became_active":"2024-03-09T08:52:18.240213+0000","last_became_peered":"2024-03-09T08:52:18.240213+0000","last_unstale":"2024-03-09T09:29:36.400325+0000","last_undegraded":"2024-03-09T09:29:36.400325+0000","last_fullsized":"2024-03-09T09:29:36.400325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2261","last_scrub_stamp":"2024-03-09T09:29:36.400199+0000","last_deep_scrub":"89'2261","last_deep_scrub_stamp":"2024-03-09T09:29:36.400199+0000","last_clean_scrub_stamp":"2024-03-09T09:29:36.400199+0000","objects_scrubbed":102,"log_size":2261,"log_dups_size":0,"ondisk_log_size":2261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:15:16.762722+0000","scrub_duration":152,"objects_trimmed":0,"snaptrim_duration":0.46979842799999999,"stat_sum":{"num_bytes":631595,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":98,"num_read_kb":25968,"num_write":1286,"num_write_kb":41427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"89'2240","reported_seq":5193,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:43.433616+0000","last_change":"2024-03-09T09:29:43.433616+0000","last_active":"2024-03-09T09:29:43.433616+0000","last_peered":"2024-03-09T09:29:43.433616+0000","last_clean":"2024-03-09T09:29:43.433616+0000","last_became_active":"2024-03-09T08:52:29.583884+0000","last_became_peered":"2024-03-09T08:52:29.583884+0000","last_unstale":"2024-03-09T09:29:43.433616+0000","last_undegraded":"2024-03-09T09:29:43.433616+0000","last_fullsized":"2024-03-09T09:29:43.433616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2240","last_scrub_stamp":"2024-03-09T09:29:43.433506+0000","last_deep_scrub":"89'2240","last_deep_scrub_stamp":"2024-03-09T09:29:43.433506+0000","last_clean_scrub_stamp":"2024-03-09T09:29:43.433506+0000","objects_scrubbed":103,"log_size":2240,"log_dups_size":0,"ondisk_log_size":2240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T12:43:00.142548+0000","scrub_duration":145,"objects_trimmed":6,"snaptrim_duration":0.66506321800000001,"stat_sum":{"num_bytes":592790,"num_objects":103,"num_object_clones":35,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":35,"num_read":105,"num_read_kb":21771,"num_write":1301,"num_write_kb":41126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"89'2285","reported_seq":5254,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:30.343867+0000","last_change":"2024-03-09T09:29:30.343867+0000","last_active":"2024-03-09T09:29:30.343867+0000","last_peered":"2024-03-09T09:29:30.343867+0000","last_clean":"2024-03-09T09:29:30.343867+0000","last_became_active":"2024-03-09T08:52:18.249533+0000","last_became_peered":"2024-03-09T08:52:18.249533+0000","last_unstale":"2024-03-09T09:29:30.343867+0000","last_undegraded":"2024-03-09T09:29:30.343867+0000","last_fullsized":"2024-03-09T09:29:30.343867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2285","last_scrub_stamp":"2024-03-09T09:29:30.343749+0000","last_deep_scrub":"89'2285","last_deep_scrub_stamp":"2024-03-09T09:29:30.343749+0000","last_clean_scrub_stamp":"2024-03-09T09:29:30.343749+0000","objects_scrubbed":81,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T13:50:19.628014+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0.638707739,"stat_sum":{"num_bytes":1008318,"num_objects":81,"num_object_clones":24,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":24,"num_read":86,"num_read_kb":17604,"num_write":1317,"num_write_kb":38401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"89'2404","reported_seq":5308,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:47.125300+0000","last_change":"2024-03-09T09:29:47.125335+0000","last_active":"2024-03-09T09:29:47.125300+0000","last_peered":"2024-03-09T09:29:47.125300+0000","last_clean":"2024-03-09T09:29:47.125300+0000","last_became_active":"2024-03-09T08:52:29.594867+0000","last_became_peered":"2024-03-09T08:52:29.594867+0000","last_unstale":"2024-03-09T09:29:47.125300+0000","last_undegraded":"2024-03-09T09:29:47.125300+0000","last_fullsized":"2024-03-09T09:29:47.125300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2404","last_scrub_stamp":"2024-03-09T09:29:47.125222+0000","last_deep_scrub":"89'2404","last_deep_scrub_stamp":"2024-03-09T09:29:47.125222+0000","last_clean_scrub_stamp":"2024-03-09T09:29:47.125222+0000","objects_scrubbed":106,"log_size":2404,"log_dups_size":0,"ondisk_log_size":2404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:13:06.705903+0000","scrub_duration":210,"objects_trimmed":4,"snaptrim_duration":1.705573912,"stat_sum":{"num_bytes":534756,"num_objects":106,"num_object_clones":34,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":34,"num_read":108,"num_read_kb":25061,"num_write":1338,"num_write_kb":44408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"89'2229","reported_seq":5246,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.024384+0000","last_change":"2024-03-09T09:29:38.024384+0000","last_active":"2024-03-09T09:29:38.024384+0000","last_peered":"2024-03-09T09:29:38.024384+0000","last_clean":"2024-03-09T09:29:38.024384+0000","last_became_active":"2024-03-09T08:52:18.233279+0000","last_became_peered":"2024-03-09T08:52:18.233279+0000","last_unstale":"2024-03-09T09:29:38.024384+0000","last_undegraded":"2024-03-09T09:29:38.024384+0000","last_fullsized":"2024-03-09T09:29:38.024384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2229","last_scrub_stamp":"2024-03-09T09:29:38.024195+0000","last_deep_scrub":"89'2229","last_deep_scrub_stamp":"2024-03-09T09:29:38.024195+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.024195+0000","objects_scrubbed":118,"log_size":2229,"log_dups_size":0,"ondisk_log_size":2229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T10:50:01.538268+0000","scrub_duration":205,"objects_trimmed":0,"snaptrim_duration":1.1420017650000001,"stat_sum":{"num_bytes":1651219,"num_objects":118,"num_object_clones":44,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":44,"num_read":114,"num_read_kb":27021,"num_write":1266,"num_write_kb":36457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"89'2400","reported_seq":5338,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:52.285820+0000","last_change":"2024-03-09T09:29:52.285820+0000","last_active":"2024-03-09T09:29:52.285820+0000","last_peered":"2024-03-09T09:29:52.285820+0000","last_clean":"2024-03-09T09:29:52.285820+0000","last_became_active":"2024-03-09T08:52:29.595431+0000","last_became_peered":"2024-03-09T08:52:29.595431+0000","last_unstale":"2024-03-09T09:29:52.285820+0000","last_undegraded":"2024-03-09T09:29:52.285820+0000","last_fullsized":"2024-03-09T09:29:52.285820+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2400","last_scrub_stamp":"2024-03-09T09:29:52.285690+0000","last_deep_scrub":"89'2400","last_deep_scrub_stamp":"2024-03-09T09:29:52.285690+0000","last_clean_scrub_stamp":"2024-03-09T09:29:52.285690+0000","objects_scrubbed":121,"log_size":2400,"log_dups_size":0,"ondisk_log_size":2400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:56:43.335889+0000","scrub_duration":298,"objects_trimmed":2,"snaptrim_duration":4.3954307559999997,"stat_sum":{"num_bytes":1015705,"num_objects":121,"num_object_clones":38,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":38,"num_read":132,"num_read_kb":25133,"num_write":1351,"num_write_kb":59371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"89'2478","reported_seq":5445,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:40.377903+0000","last_change":"2024-03-09T09:29:40.377903+0000","last_active":"2024-03-09T09:29:40.377903+0000","last_peered":"2024-03-09T09:29:40.377903+0000","last_clean":"2024-03-09T09:29:40.377903+0000","last_became_active":"2024-03-09T08:52:18.242007+0000","last_became_peered":"2024-03-09T08:52:18.242007+0000","last_unstale":"2024-03-09T09:29:40.377903+0000","last_undegraded":"2024-03-09T09:29:40.377903+0000","last_fullsized":"2024-03-09T09:29:40.377903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2478","last_scrub_stamp":"2024-03-09T09:29:40.377714+0000","last_deep_scrub":"89'2478","last_deep_scrub_stamp":"2024-03-09T09:29:40.377714+0000","last_clean_scrub_stamp":"2024-03-09T09:29:40.377714+0000","objects_scrubbed":117,"log_size":2478,"log_dups_size":0,"ondisk_log_size":2478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T18:37:06.154933+0000","scrub_duration":204,"objects_trimmed":0,"snaptrim_duration":0.51876684699999998,"stat_sum":{"num_bytes":1407157,"num_objects":117,"num_object_clones":38,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":38,"num_read":113,"num_read_kb":14121,"num_write":1423,"num_write_kb":30929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"89'2261","reported_seq":5205,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:35.501013+0000","last_change":"2024-03-09T09:29:35.501013+0000","last_active":"2024-03-09T09:29:35.501013+0000","last_peered":"2024-03-09T09:29:35.501013+0000","last_clean":"2024-03-09T09:29:35.501013+0000","last_became_active":"2024-03-09T08:52:29.576470+0000","last_became_peered":"2024-03-09T08:52:29.576470+0000","last_unstale":"2024-03-09T09:29:35.501013+0000","last_undegraded":"2024-03-09T09:29:35.501013+0000","last_fullsized":"2024-03-09T09:29:35.501013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2261","last_scrub_stamp":"2024-03-09T09:29:35.500858+0000","last_deep_scrub":"89'2261","last_deep_scrub_stamp":"2024-03-09T09:29:35.500858+0000","last_clean_scrub_stamp":"2024-03-09T09:29:35.500858+0000","objects_scrubbed":116,"log_size":2261,"log_dups_size":0,"ondisk_log_size":2261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:46:12.193654+0000","scrub_duration":204,"objects_trimmed":2,"snaptrim_duration":4.1683436250000003,"stat_sum":{"num_bytes":2080769,"num_objects":116,"num_object_clones":43,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":43,"num_read":114,"num_read_kb":27228,"num_write":1275,"num_write_kb":54719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"89'2329","reported_seq":5313,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:29.999608+0000","last_change":"2024-03-09T09:29:29.999663+0000","last_active":"2024-03-09T09:29:29.999608+0000","last_peered":"2024-03-09T09:29:29.999608+0000","last_clean":"2024-03-09T09:29:29.999608+0000","last_became_active":"2024-03-09T08:52:18.245299+0000","last_became_peered":"2024-03-09T08:52:18.245299+0000","last_unstale":"2024-03-09T09:29:29.999608+0000","last_undegraded":"2024-03-09T09:29:29.999608+0000","last_fullsized":"2024-03-09T09:29:29.999608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2329","last_scrub_stamp":"2024-03-09T09:29:29.999459+0000","last_deep_scrub":"89'2329","last_deep_scrub_stamp":"2024-03-09T09:29:29.999459+0000","last_clean_scrub_stamp":"2024-03-09T09:29:29.999459+0000","objects_scrubbed":102,"log_size":2329,"log_dups_size":0,"ondisk_log_size":2329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:41:58.957669+0000","scrub_duration":225,"objects_trimmed":0,"snaptrim_duration":1.4282657350000001,"stat_sum":{"num_bytes":1076453,"num_objects":102,"num_object_clones":33,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":33,"num_read":106,"num_read_kb":19463,"num_write":1318,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"89'2532","reported_seq":5423,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.773752+0000","last_change":"2024-03-09T09:29:38.773752+0000","last_active":"2024-03-09T09:29:38.773752+0000","last_peered":"2024-03-09T09:29:38.773752+0000","last_clean":"2024-03-09T09:29:38.773752+0000","last_became_active":"2024-03-09T08:52:29.604285+0000","last_became_peered":"2024-03-09T08:52:29.604285+0000","last_unstale":"2024-03-09T09:29:38.773752+0000","last_undegraded":"2024-03-09T09:29:38.773752+0000","last_fullsized":"2024-03-09T09:29:38.773752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2532","last_scrub_stamp":"2024-03-09T09:29:38.773637+0000","last_deep_scrub":"89'2532","last_deep_scrub_stamp":"2024-03-09T09:29:38.773637+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.773637+0000","objects_scrubbed":116,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T15:29:01.867872+0000","scrub_duration":204,"objects_trimmed":2,"snaptrim_duration":0.63967549000000001,"stat_sum":{"num_bytes":3580963,"num_objects":116,"num_object_clones":37,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":37,"num_read":116,"num_read_kb":32310,"num_write":1440,"num_write_kb":66236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"89'2363","reported_seq":5357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:31.164540+0000","last_change":"2024-03-09T09:29:31.164540+0000","last_active":"2024-03-09T09:29:31.164540+0000","last_peered":"2024-03-09T09:29:31.164540+0000","last_clean":"2024-03-09T09:29:31.164540+0000","last_became_active":"2024-03-09T08:52:18.222613+0000","last_became_peered":"2024-03-09T08:52:18.222613+0000","last_unstale":"2024-03-09T09:29:31.164540+0000","last_undegraded":"2024-03-09T09:29:31.164540+0000","last_fullsized":"2024-03-09T09:29:31.164540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2363","last_scrub_stamp":"2024-03-09T09:29:31.164401+0000","last_deep_scrub":"89'2363","last_deep_scrub_stamp":"2024-03-09T09:29:31.164401+0000","last_clean_scrub_stamp":"2024-03-09T09:29:31.164401+0000","objects_scrubbed":112,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:04:08.484425+0000","scrub_duration":162,"objects_trimmed":0,"snaptrim_duration":0.47268980999999999,"stat_sum":{"num_bytes":1488567,"num_objects":112,"num_object_clones":34,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":34,"num_read":108,"num_read_kb":30029,"num_write":1350,"num_write_kb":42424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"89'2240","reported_seq":5183,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:49.447305+0000","last_change":"2024-03-09T09:29:49.447305+0000","last_active":"2024-03-09T09:29:49.447305+0000","last_peered":"2024-03-09T09:29:49.447305+0000","last_clean":"2024-03-09T09:29:49.447305+0000","last_became_active":"2024-03-09T08:52:29.591188+0000","last_became_peered":"2024-03-09T08:52:29.591188+0000","last_unstale":"2024-03-09T09:29:49.447305+0000","last_undegraded":"2024-03-09T09:29:49.447305+0000","last_fullsized":"2024-03-09T09:29:49.447305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2240","last_scrub_stamp":"2024-03-09T09:29:49.447119+0000","last_deep_scrub":"89'2240","last_deep_scrub_stamp":"2024-03-09T09:29:49.447119+0000","last_clean_scrub_stamp":"2024-03-09T09:29:49.447119+0000","objects_scrubbed":98,"log_size":2240,"log_dups_size":0,"ondisk_log_size":2240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T20:14:56.069116+0000","scrub_duration":229,"objects_trimmed":6,"snaptrim_duration":4.3763956530000003,"stat_sum":{"num_bytes":1440979,"num_objects":98,"num_object_clones":30,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":30,"num_read":104,"num_read_kb":22606,"num_write":1302,"num_write_kb":46651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"89'2151","reported_seq":5156,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:38.982552+0000","last_change":"2024-03-09T09:29:38.982552+0000","last_active":"2024-03-09T09:29:38.982552+0000","last_peered":"2024-03-09T09:29:38.982552+0000","last_clean":"2024-03-09T09:29:38.982552+0000","last_became_active":"2024-03-09T08:52:18.225464+0000","last_became_peered":"2024-03-09T08:52:18.225464+0000","last_unstale":"2024-03-09T09:29:38.982552+0000","last_undegraded":"2024-03-09T09:29:38.982552+0000","last_fullsized":"2024-03-09T09:29:38.982552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2151","last_scrub_stamp":"2024-03-09T09:29:38.982431+0000","last_deep_scrub":"89'2151","last_deep_scrub_stamp":"2024-03-09T09:29:38.982431+0000","last_clean_scrub_stamp":"2024-03-09T09:29:38.982431+0000","objects_scrubbed":85,"log_size":2151,"log_dups_size":0,"ondisk_log_size":2151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T17:13:17.068773+0000","scrub_duration":127,"objects_trimmed":0,"snaptrim_duration":1.344208157,"stat_sum":{"num_bytes":485403,"num_objects":85,"num_object_clones":27,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":27,"num_read":96,"num_read_kb":5635,"num_write":1204,"num_write_kb":28568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"89'2303","reported_seq":5228,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:52.273061+0000","last_change":"2024-03-09T09:29:52.273061+0000","last_active":"2024-03-09T09:29:52.273061+0000","last_peered":"2024-03-09T09:29:52.273061+0000","last_clean":"2024-03-09T09:29:52.273061+0000","last_became_active":"2024-03-09T08:52:29.594738+0000","last_became_peered":"2024-03-09T08:52:29.594738+0000","last_unstale":"2024-03-09T09:29:52.273061+0000","last_undegraded":"2024-03-09T09:29:52.273061+0000","last_fullsized":"2024-03-09T09:29:52.273061+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2303","last_scrub_stamp":"2024-03-09T09:29:52.272876+0000","last_deep_scrub":"89'2303","last_deep_scrub_stamp":"2024-03-09T09:29:52.272876+0000","last_clean_scrub_stamp":"2024-03-09T09:29:52.272876+0000","objects_scrubbed":87,"log_size":2303,"log_dups_size":0,"ondisk_log_size":2303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T11:53:22.068184+0000","scrub_duration":126,"objects_trimmed":4,"snaptrim_duration":4.4378791440000001,"stat_sum":{"num_bytes":906976,"num_objects":87,"num_object_clones":27,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":27,"num_read":110,"num_read_kb":26703,"num_write":1307,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"89'2242","reported_seq":5251,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-09T09:29:32.337831+0000","last_change":"2024-03-09T09:29:32.337831+0000","last_active":"2024-03-09T09:29:32.337831+0000","last_peered":"2024-03-09T09:29:32.337831+0000","last_clean":"2024-03-09T09:29:32.337831+0000","last_became_active":"2024-03-09T08:52:18.245021+0000","last_became_peered":"2024-03-09T08:52:18.245021+0000","last_unstale":"2024-03-09T09:29:32.337831+0000","last_undegraded":"2024-03-09T09:29:32.337831+0000","last_fullsized":"2024-03-09T09:29:32.337831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'2242","last_scrub_stamp":"2024-03-09T09:29:32.337704+0000","last_deep_scrub":"89'2242","last_deep_scrub_stamp":"2024-03-09T09:29:32.337704+0000","last_clean_scrub_stamp":"2024-03-09T09:29:32.337704+0000","objects_scrubbed":97,"log_size":2242,"log_dups_size":0,"ondisk_log_size":2242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T16:41:31.317870+0000","scrub_duration":144,"objects_trimmed":0,"snaptrim_duration":1.0826407490000001,"stat_sum":{"num_bytes":992571,"num_objects":97,"num_object_clones":32,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":32,"num_read":100,"num_read_kb":32945,"num_write":1310,"num_write_kb":54952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7626,"num_read_kb":1526336,"num_write":85965,"num_write_kb":2933421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151544,"ondisk_log_size":151544,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":877528733,"num_objects":1507,"num_object_clones":0,"num_object_copies":3014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1507,"num_whiteouts":0,"num_read":2924,"num_read_kb":5807,"num_write":25888,"num_write_kb":3145194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1225,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17386594,"num_omap_keys":27558,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1755119616,"data_stored":1755057466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23677,"ondisk_log_size":23677,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":101855205,"num_objects":7099,"num_object_clones":2364,"num_object_copies":14198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7099,"num_whiteouts":2364,"num_read":7230,"num_read_kb":1519173,"num_write":85971,"num_write_kb":2933405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":227172352,"data_stored":203710410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151399,"ondisk_log_size":151399,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798940451,"num_objects":1475,"num_object_clones":0,"num_object_copies":2950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1475,"num_whiteouts":0,"num_read":2854,"num_read_kb":5749,"num_write":25875,"num_write_kb":3105441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1213,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17656376,"num_omap_keys":27516,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597939712,"data_stored":1597880902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23637,"ondisk_log_size":23637,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575308,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":835936,"kb_used_data":502168,"kb_used_omap":1,"kb_used_meta":333758,"kb_avail":93535904,"statfs":{"total":96636764160,"available":95780765696,"internally_reserved":0,"allocated":514220032,"data_stored":507443895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":341768650},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575309,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716380,"kb_used_data":456276,"kb_used_omap":1,"kb_used_meta":260094,"kb_avail":93655460,"statfs":{"total":96636764160,"available":95903191040,"internally_reserved":0,"allocated":467226624,"data_stored":462408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":266336710},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575307,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":814680,"kb_used_data":454544,"kb_used_omap":1,"kb_used_meta":360126,"kb_avail":93557160,"statfs":{"total":96636764160,"available":95802531840,"internally_reserved":0,"allocated":465453056,"data_stored":458259869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":368769477},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575308,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":857476,"kb_used_data":551484,"kb_used_omap":1,"kb_used_meta":305982,"kb_avail":93514364,"statfs":{"total":96636764160,"available":95758708736,"internally_reserved":0,"allocated":564719616,"data_stored":557367936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":313326021},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575309,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":940952,"kb_used_data":544784,"kb_used_omap":1,"kb_used_meta":396158,"kb_avail":93430888,"statfs":{"total":96636764160,"available":95673229312,"internally_reserved":0,"allocated":557858816,"data_stored":550354976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":405666247},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575308,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462952,"kb_used_data":225056,"kb_used_omap":1,"kb_used_meta":237886,"kb_avail":93908888,"statfs":{"total":96636764160,"available":96162701312,"internally_reserved":0,"allocated":230457344,"data_stored":225591238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":243595717},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575308,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":973072,"kb_used_data":613000,"kb_used_omap":1,"kb_used_meta":360062,"kb_avail":93398768,"statfs":{"total":96636764160,"available":95640338432,"internally_reserved":0,"allocated":627712000,"data_stored":621647944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":368703941},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575310,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672440,"kb_used_data":380400,"kb_used_omap":1,"kb_used_meta":292030,"kb_avail":93699400,"statfs":{"total":96636764160,"available":95948185600,"internally_reserved":0,"allocated":389529600,"data_stored":383207215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":299039175},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":226504704,"data_stored":226493054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":218112000,"data_stored":218104019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92274688,"data_stored":92274688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":173154304,"data_stored":173146197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":201338880,"data_stored":201326769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":197980160,"data_stored":197974487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":223150080,"data_stored":223140893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":265424896,"data_stored":265420795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37781504,"data_stored":34251073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20246528,"data_stored":18282047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22822912,"data_stored":20809621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27308032,"data_stored":23727598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":37437440,"data_stored":33951899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":27983872,"data_stored":24565897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23818240,"data_stored":21628413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":29773824,"data_stored":26493862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":100741120,"data_stored":100738977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":360800256,"data_stored":360786519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96473088,"data_stored":96469310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":315428864,"data_stored":315415292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":289415168,"data_stored":289407328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":201334784,"data_stored":201327264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":201334784,"data_stored":201327208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":189591552,"data_stored":189585568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22835200,"data_stored":20540995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27480064,"data_stored":23882611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17813504,"data_stored":15444871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40894464,"data_stored":37473141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":35454976,"data_stored":32089192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":37081088,"data_stored":33799473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17850368,"data_stored":15719573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":27762688,"data_stored":24760554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-09T09:30:10.347 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-09T09:30:10.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.577+0000 7f320ac00640 1 Processor -- start 2024-03-09T09:30:10.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.577+0000 7f320ac00640 1 -- start start 2024-03-09T09:30:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.579+0000 7f320ac00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f32040c34f0 0x7f3204159430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.579+0000 7f320ac00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f32040c4030 0x7f3204159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.579+0000 7f320ac00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3204071b60 con 0x7f32041532b0 2024-03-09T09:30:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.579+0000 7f320ac00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32040729e0 con 0x7f32040c4030 2024-03-09T09:30:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.579+0000 7f320ac00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32040c4b50 con 0x7f32040c34f0 2024-03-09T09:30:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.579+0000 7f320a200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f32040c4030 0x7f3204159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.579+0000 7f320a200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f32040c4030 0x7f3204159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38062/0 (socket says 172.21.15.67:38062) 2024-03-09T09:30:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.579+0000 7f320a200640 1 -- 172.21.15.67:0/3615026280 learned_addr learned my addr 172.21.15.67:0/3615026280 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.580+0000 7f3208e00640 1 --2- 172.21.15.67:0/3615026280 >> v2:172.21.15.134:3300/0 conn(0x7f32040c34f0 0x7f3204159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.580+0000 7f320a200640 1 -- 172.21.15.67:0/3615026280 >> v2:172.21.15.134:3300/0 conn(0x7f32040c34f0 msgr2=0x7f3204159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.580+0000 7f320a200640 1 --2- 172.21.15.67:0/3615026280 >> v2:172.21.15.134:3300/0 conn(0x7f32040c34f0 0x7f3204159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.580+0000 7f320a200640 1 -- 172.21.15.67:0/3615026280 >> v1:172.21.15.134:6789/0 conn(0x7f32041532b0 legacy=0x7f320406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.580+0000 7f320a200640 1 -- 172.21.15.67:0/3615026280 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3204071880 con 0x7f32040c4030 2024-03-09T09:30:10.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.580+0000 7f3208e00640 1 --2- 172.21.15.67:0/3615026280 >> v2:172.21.15.134:3300/0 conn(0x7f32040c34f0 0x7f3204159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:30:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.581+0000 7f320a200640 1 --2- 172.21.15.67:0/3615026280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32040c4030 0x7f3204159970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3204072b60 tx=0x7f31fc02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cf6b8e0a007e08 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.581+0000 7f3203e00640 1 -- 172.21.15.67:0/3615026280 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc02c960 con 0x7f32040c4030 2024-03-09T09:30:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.581+0000 7f3203e00640 1 -- 172.21.15.67:0/3615026280 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc02cb00 con 0x7f32040c4030 2024-03-09T09:30:10.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.581+0000 7f3203e00640 1 -- 172.21.15.67:0/3615026280 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0343f0 con 0x7f32040c4030 2024-03-09T09:30:10.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.582+0000 7f320ac00640 1 -- 172.21.15.67:0/3615026280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32040c4030 msgr2=0x7f3204159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:10.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.582+0000 7f320ac00640 1 --2- 172.21.15.67:0/3615026280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32040c4030 0x7f3204159970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3204072b60 tx=0x7f31fc02bdc0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.582+0000 7f320ac00640 1 -- 172.21.15.67:0/3615026280 shutdown_connections 2024-03-09T09:30:10.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.582+0000 7f320ac00640 1 --2- 172.21.15.67:0/3615026280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32040c4030 0x7f3204159970 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.582+0000 7f320ac00640 1 --2- 172.21.15.67:0/3615026280 >> v2:172.21.15.134:3300/0 conn(0x7f32040c34f0 0x7f3204159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.582+0000 7f320ac00640 1 -- 172.21.15.67:0/3615026280 >> 172.21.15.67:0/3615026280 conn(0x7f32040c82a0 msgr2=0x7f32040c8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:10.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.582+0000 7f320ac00640 1 -- 172.21.15.67:0/3615026280 shutdown_connections 2024-03-09T09:30:10.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.582+0000 7f320ac00640 1 -- 172.21.15.67:0/3615026280 wait complete. 2024-03-09T09:30:10.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.584+0000 7f320ac00640 1 Processor -- start 2024-03-09T09:30:10.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.584+0000 7f320ac00640 1 -- start start 2024-03-09T09:30:10.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.585+0000 7f320ac00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32040c34f0 0x7f32041eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.585+0000 7f320ac00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32040c4030 0x7f32041e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.585+0000 7f320ac00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32041532b0 0x7f32041ef670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.585+0000 7f320ac00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320406b750 con 0x7f32041532b0 2024-03-09T09:30:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.585+0000 7f320ac00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32040c4870 con 0x7f32040c34f0 2024-03-09T09:30:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.585+0000 7f320ac00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320406b8d0 con 0x7f32040c4030 2024-03-09T09:30:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.586+0000 7f320a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32041532b0 0x7f32041ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.586+0000 7f320a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32041532b0 0x7f32041ef670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38066/0 (socket says 172.21.15.67:38066) 2024-03-09T09:30:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.586+0000 7f320a200640 1 -- 172.21.15.67:0/4194523578 learned_addr learned my addr 172.21.15.67:0/4194523578 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:10.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.586+0000 7f3208e00640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32040c4030 0x7f32041e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.586+0000 7f3209800640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32040c34f0 0x7f32041eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.586+0000 7f320a200640 1 -- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32040c4030 msgr2=0x7f32041e2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:10.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.586+0000 7f320a200640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32040c4030 0x7f32041e2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.586+0000 7f320a200640 1 -- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32040c34f0 msgr2=0x7f32041eb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:10.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.586+0000 7f320a200640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32040c34f0 0x7f32041eb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.586+0000 7f320a200640 1 -- 172.21.15.67:0/4194523578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32041ebd10 con 0x7f32041532b0 2024-03-09T09:30:10.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.587+0000 7f320a200640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32041532b0 0x7f32041ef670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f31fc036040 tx=0x7f31fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.587+0000 7f3201600640 1 -- 172.21.15.67:0/4194523578 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc043020 con 0x7f32041532b0 2024-03-09T09:30:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.587+0000 7f3201600640 1 -- 172.21.15.67:0/4194523578 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc03e450 con 0x7f32041532b0 2024-03-09T09:30:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.587+0000 7f3201600640 1 -- 172.21.15.67:0/4194523578 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc03e860 con 0x7f32041532b0 2024-03-09T09:30:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.587+0000 7f320ac00640 1 -- 172.21.15.67:0/4194523578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32040c7a70 con 0x7f32041532b0 2024-03-09T09:30:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.587+0000 7f320ac00640 1 -- 172.21.15.67:0/4194523578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32041eee70 con 0x7f32041532b0 2024-03-09T09:30:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.589+0000 7f320ac00640 1 -- 172.21.15.67:0/4194523578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320404f3c0 con 0x7f32041532b0 2024-03-09T09:30:10.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.589+0000 7f3201600640 1 -- 172.21.15.67:0/4194523578 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f31fc046020 con 0x7f32041532b0 2024-03-09T09:30:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.591+0000 7f3201600640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31e406b4d0 0x7f31e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.591+0000 7f3201600640 1 -- 172.21.15.67:0/4194523578 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f31fc0acea0 con 0x7f32041532b0 2024-03-09T09:30:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.591+0000 7f3209800640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31e406b4d0 0x7f31e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:10.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.592+0000 7f3209800640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31e406b4d0 0x7f31e406d990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f31f8007fd0 tx=0x7f31f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:10.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.596+0000 7f3201600640 1 -- 172.21.15.67:0/4194523578 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f31fc04d030 con 0x7f32041532b0 2024-03-09T09:30:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.804+0000 7f320ac00640 1 -- 172.21.15.67:0/4194523578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f3204149850 con 0x7f32041532b0 2024-03-09T09:30:10.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.809+0000 7f3201600640 1 -- 172.21.15.67:0/4194523578 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f31fc07b950 con 0x7f32041532b0 2024-03-09T09:30:10.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.810+0000 7f3201600640 1 -- 172.21.15.67:0/4194523578 <== mon.0 v2:172.21.15.67:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f31fc07b250 con 0x7f32041532b0 2024-03-09T09:30:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 -- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31e406b4d0 msgr2=0x7f31e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31e406b4d0 0x7f31e406d990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f31f8007fd0 tx=0x7f31f8009040 comp rx=0 tx=0).stop 2024-03-09T09:30:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 -- 172.21.15.67:0/4194523578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32041532b0 msgr2=0x7f32041ef670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:10.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32041532b0 0x7f32041ef670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f31fc036040 tx=0x7f31fc004a40 comp rx=0 tx=0).stop 2024-03-09T09:30:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 -- 172.21.15.67:0/4194523578 shutdown_connections 2024-03-09T09:30:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f31e406b4d0 0x7f31e406d990 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32041532b0 0x7f32041ef670 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32040c4030 0x7f32041e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 --2- 172.21.15.67:0/4194523578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32040c34f0 0x7f32041eb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 -- 172.21.15.67:0/4194523578 >> 172.21.15.67:0/4194523578 conn(0x7f32040c82a0 msgr2=0x7f3204159430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:10.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 -- 172.21.15.67:0/4194523578 shutdown_connections 2024-03-09T09:30:10.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:10.813+0000 7f320ac00640 1 -- 172.21.15.67:0/4194523578 wait complete. 2024-03-09T09:30:10.847 INFO:tasks.cephfs.filesystem:Destroying file system a and related pools 2024-03-09T09:30:10.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T09:30:11.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.064+0000 7f472ca00640 1 Processor -- start 2024-03-09T09:30:11.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.064+0000 7f472ca00640 1 -- start start 2024-03-09T09:30:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.067+0000 7f472ca00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f4728153300 0x7f47280b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.067+0000 7f472ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47280c5e40 0x7f47280b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.067+0000 7f472ca00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47280cadb0 0x7f4728156010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.067+0000 7f472ca00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4728157000 con 0x7f47280cadb0 2024-03-09T09:30:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.067+0000 7f472ca00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47280ae430 con 0x7f4728153300 2024-03-09T09:30:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.067+0000 7f472ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47280ae6f0 con 0x7f47280c5e40 2024-03-09T09:30:11.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.067+0000 7f4727200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47280cadb0 0x7f4728156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.067+0000 7f4727200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f47280cadb0 0x7f4728156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38074/0 (socket says 172.21.15.67:38074) 2024-03-09T09:30:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.067+0000 7f4727200640 1 -- 172.21.15.67:0/221778963 learned_addr learned my addr 172.21.15.67:0/221778963 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.068+0000 7f471de00640 1 --2- 172.21.15.67:0/221778963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47280c5e40 0x7f47280b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.068+0000 7f4726800640 1 --2- 172.21.15.67:0/221778963 >> v2:172.21.15.134:3300/0 conn(0x7f4728153300 0x7f47280b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.068+0000 7f4727200640 1 -- 172.21.15.67:0/221778963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47280c5e40 msgr2=0x7f47280b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.068+0000 7f4727200640 1 --2- 172.21.15.67:0/221778963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47280c5e40 0x7f47280b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.068+0000 7f4727200640 1 -- 172.21.15.67:0/221778963 >> v2:172.21.15.134:3300/0 conn(0x7f4728153300 msgr2=0x7f47280b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.068+0000 7f4727200640 1 --2- 172.21.15.67:0/221778963 >> v2:172.21.15.134:3300/0 conn(0x7f4728153300 0x7f47280b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.068+0000 7f4727200640 1 -- 172.21.15.67:0/221778963 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728155b10 con 0x7f47280cadb0 2024-03-09T09:30:11.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.068+0000 7f4726800640 1 --2- 172.21.15.67:0/221778963 >> v2:172.21.15.134:3300/0 conn(0x7f4728153300 0x7f47280b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:30:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.069+0000 7f4727200640 1 --2- 172.21.15.67:0/221778963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47280cadb0 0x7f4728156010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f47280ade40 tx=0x7f472002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590ce36ad61adfdf server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.069+0000 7f4725e00640 1 -- 172.21.15.67:0/221778963 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472002ca70 con 0x7f47280cadb0 2024-03-09T09:30:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.069+0000 7f4725e00640 1 -- 172.21.15.67:0/221778963 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f472002cc10 con 0x7f47280cadb0 2024-03-09T09:30:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.069+0000 7f4725e00640 1 -- 172.21.15.67:0/221778963 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47200343f0 con 0x7f47280cadb0 2024-03-09T09:30:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.070+0000 7f472ca00640 1 -- 172.21.15.67:0/221778963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47280cadb0 msgr2=0x7f4728156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:11.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.070+0000 7f472ca00640 1 --2- 172.21.15.67:0/221778963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47280cadb0 0x7f4728156010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f47280ade40 tx=0x7f472002bed0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.070+0000 7f472ca00640 1 -- 172.21.15.67:0/221778963 shutdown_connections 2024-03-09T09:30:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.070+0000 7f472ca00640 1 --2- 172.21.15.67:0/221778963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47280cadb0 0x7f4728156010 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.070+0000 7f472ca00640 1 --2- 172.21.15.67:0/221778963 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47280c5e40 0x7f47280b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.070+0000 7f472ca00640 1 --2- 172.21.15.67:0/221778963 >> v2:172.21.15.134:3300/0 conn(0x7f4728153300 0x7f47280b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.070+0000 7f472ca00640 1 -- 172.21.15.67:0/221778963 >> 172.21.15.67:0/221778963 conn(0x7f47280732d0 msgr2=0x7f47280ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.070+0000 7f472ca00640 1 -- 172.21.15.67:0/221778963 shutdown_connections 2024-03-09T09:30:11.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.070+0000 7f472ca00640 1 -- 172.21.15.67:0/221778963 wait complete. 2024-03-09T09:30:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.072+0000 7f472ca00640 1 Processor -- start 2024-03-09T09:30:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.072+0000 7f472ca00640 1 -- start start 2024-03-09T09:30:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.073+0000 7f472ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47280c5e40 0x7f47280ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.073+0000 7f472ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47280cadb0 0x7f47280abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.073+0000 7f472ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4728153300 0x7f47280ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.073+0000 7f472ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280c59e0 con 0x7f47280cadb0 2024-03-09T09:30:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.073+0000 7f472ca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4728159e90 con 0x7f4728153300 2024-03-09T09:30:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.073+0000 7f472ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4728156550 con 0x7f47280c5e40 2024-03-09T09:30:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.073+0000 7f471de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47280cadb0 0x7f47280abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.073+0000 7f471de00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47280cadb0 0x7f47280abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38076/0 (socket says 172.21.15.67:38076) 2024-03-09T09:30:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.073+0000 7f471de00640 1 -- 172.21.15.67:0/2705965596 learned_addr learned my addr 172.21.15.67:0/2705965596 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.073+0000 7f4727200640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4728153300 0x7f47280ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.074+0000 7f4726800640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47280c5e40 0x7f47280ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.074+0000 7f471de00640 1 -- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47280c5e40 msgr2=0x7f47280ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.074+0000 7f471de00640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47280c5e40 0x7f47280ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.074+0000 7f471de00640 1 -- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4728153300 msgr2=0x7f47280ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:11.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.074+0000 7f471de00640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4728153300 0x7f47280ac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.074+0000 7f471de00640 1 -- 172.21.15.67:0/2705965596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280c6b40 con 0x7f47280cadb0 2024-03-09T09:30:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.074+0000 7f4726800640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47280c5e40 0x7f47280ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:30:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.074+0000 7f471de00640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47280cadb0 0x7f47280abb60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f471000d800 tx=0x7f471000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.074+0000 7f471f400640 1 -- 172.21.15.67:0/2705965596 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710018020 con 0x7f47280cadb0 2024-03-09T09:30:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.074+0000 7f471f400640 1 -- 172.21.15.67:0/2705965596 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4710009640 con 0x7f47280cadb0 2024-03-09T09:30:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.074+0000 7f471f400640 1 -- 172.21.15.67:0/2705965596 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710009a30 con 0x7f47280cadb0 2024-03-09T09:30:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.075+0000 7f472ca00640 1 -- 172.21.15.67:0/2705965596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47280c7a80 con 0x7f47280cadb0 2024-03-09T09:30:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.075+0000 7f472ca00640 1 -- 172.21.15.67:0/2705965596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47280ab750 con 0x7f47280cadb0 2024-03-09T09:30:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.076+0000 7f472ca00640 1 -- 172.21.15.67:0/2705965596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472804f3c0 con 0x7f47280cadb0 2024-03-09T09:30:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.078+0000 7f471f400640 1 -- 172.21.15.67:0/2705965596 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f471001b020 con 0x7f47280cadb0 2024-03-09T09:30:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.079+0000 7f471f400640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f470406b500 0x7f470406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.079+0000 7f471f400640 1 -- 172.21.15.67:0/2705965596 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f471008a860 con 0x7f47280cadb0 2024-03-09T09:30:11.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.079+0000 7f4726800640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f470406b500 0x7f470406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.080+0000 7f4726800640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f470406b500 0x7f470406d9c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4718006fd0 tx=0x7f4718008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:11.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.084+0000 7f471f400640 1 -- 172.21.15.67:0/2705965596 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4710022030 con 0x7f47280cadb0 2024-03-09T09:30:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.332+0000 7f472ca00640 1 -- 172.21.15.67:0/2705965596 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47280689d0 con 0x7f47280cadb0 2024-03-09T09:30:11.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.333+0000 7f471f400640 1 -- 172.21.15.67:0/2705965596 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6247 (secure 0 0 0) 0x7f4710059280 con 0x7f47280cadb0 2024-03-09T09:30:11.341 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:30:11.341 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:17.736973+0000","modified":"2024-03-09T08:52:21.931108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T09:30:11.341 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 14 2024-03-09T09:30:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.336+0000 7f472ca00640 1 -- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f470406b500 msgr2=0x7f470406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.336+0000 7f472ca00640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f470406b500 0x7f470406d9c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4718006fd0 tx=0x7f4718008040 comp rx=0 tx=0).stop 2024-03-09T09:30:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.336+0000 7f472ca00640 1 -- 172.21.15.67:0/2705965596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47280cadb0 msgr2=0x7f47280abb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.336+0000 7f472ca00640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47280cadb0 0x7f47280abb60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f471000d800 tx=0x7f471000dcc0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.336+0000 7f472ca00640 1 -- 172.21.15.67:0/2705965596 shutdown_connections 2024-03-09T09:30:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.336+0000 7f472ca00640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f470406b500 0x7f470406d9c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.336+0000 7f472ca00640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4728153300 0x7f47280ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.336+0000 7f472ca00640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47280cadb0 0x7f47280abb60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.336+0000 7f472ca00640 1 --2- 172.21.15.67:0/2705965596 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47280c5e40 0x7f47280ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.336+0000 7f472ca00640 1 -- 172.21.15.67:0/2705965596 >> 172.21.15.67:0/2705965596 conn(0x7f47280732d0 msgr2=0x7f47281553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.337+0000 7f472ca00640 1 -- 172.21.15.67:0/2705965596 shutdown_connections 2024-03-09T09:30:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.337+0000 7f472ca00640 1 -- 172.21.15.67:0/2705965596 wait complete. 2024-03-09T09:30:11.362 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T09:30:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.565+0000 7f8e76c00640 1 Processor -- start 2024-03-09T09:30:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.565+0000 7f8e76c00640 1 -- start start 2024-03-09T09:30:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.567+0000 7f8e76c00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e7014dad0 0x7f8e7006e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.567+0000 7f8e76c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f8e7006b940 0x7f8e7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.567+0000 7f8e76c00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e700c30e0 con 0x7f8e7006f320 2024-03-09T09:30:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.567+0000 7f8e76c00640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e700c33c0 con 0x7f8e7014dad0 2024-03-09T09:30:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.567+0000 7f8e76c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e700c36a0 con 0x7f8e7006b940 2024-03-09T09:30:11.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.567+0000 7f8e75800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e7014dad0 0x7f8e7006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.567+0000 7f8e75800640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f8e7014dad0 0x7f8e7006e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38080/0 (socket says 172.21.15.67:38080) 2024-03-09T09:30:11.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.567+0000 7f8e75800640 1 -- 172.21.15.67:0/3166454976 learned_addr learned my addr 172.21.15.67:0/3166454976 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:11.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.568+0000 7f8e74e00640 1 --2- 172.21.15.67:0/3166454976 >> v2:172.21.15.134:3300/0 conn(0x7f8e7006b940 0x7f8e7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.568+0000 7f8e75800640 1 -- 172.21.15.67:0/3166454976 >> v2:172.21.15.134:3300/0 conn(0x7f8e7006b940 msgr2=0x7f8e7006ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.568+0000 7f8e75800640 1 --2- 172.21.15.67:0/3166454976 >> v2:172.21.15.134:3300/0 conn(0x7f8e7006b940 0x7f8e7006ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.568+0000 7f8e75800640 1 -- 172.21.15.67:0/3166454976 >> v1:172.21.15.134:6789/0 conn(0x7f8e7006f320 legacy=0x7f8e70153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.568+0000 7f8e75800640 1 -- 172.21.15.67:0/3166454976 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e700c3980 con 0x7f8e7014dad0 2024-03-09T09:30:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.569+0000 7f8e75800640 1 --2- 172.21.15.67:0/3166454976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e7014dad0 0x7f8e7006e470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8e7006b3c0 tx=0x7f8e6402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec383d658fb07447 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:11.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.570+0000 7f8e6fe00640 1 -- 172.21.15.67:0/3166454976 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64038070 con 0x7f8e7014dad0 2024-03-09T09:30:11.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.570+0000 7f8e6fe00640 1 -- 172.21.15.67:0/3166454976 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8e6402f050 con 0x7f8e7014dad0 2024-03-09T09:30:11.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.570+0000 7f8e6fe00640 1 -- 172.21.15.67:0/3166454976 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64031990 con 0x7f8e7014dad0 2024-03-09T09:30:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.570+0000 7f8e76c00640 1 -- 172.21.15.67:0/3166454976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e7014dad0 msgr2=0x7f8e7006e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.570+0000 7f8e76c00640 1 --2- 172.21.15.67:0/3166454976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e7014dad0 0x7f8e7006e470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8e7006b3c0 tx=0x7f8e6402b7f0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.571+0000 7f8e76c00640 1 -- 172.21.15.67:0/3166454976 shutdown_connections 2024-03-09T09:30:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.571+0000 7f8e76c00640 1 --2- 172.21.15.67:0/3166454976 >> v2:172.21.15.134:3300/0 conn(0x7f8e7006b940 0x7f8e7006ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.571+0000 7f8e76c00640 1 --2- 172.21.15.67:0/3166454976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e7014dad0 0x7f8e7006e470 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.571+0000 7f8e76c00640 1 -- 172.21.15.67:0/3166454976 >> 172.21.15.67:0/3166454976 conn(0x7f8e7006d650 msgr2=0x7f8e7006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.571+0000 7f8e76c00640 1 -- 172.21.15.67:0/3166454976 shutdown_connections 2024-03-09T09:30:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.571+0000 7f8e76c00640 1 -- 172.21.15.67:0/3166454976 wait complete. 2024-03-09T09:30:11.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.572+0000 7f8e76c00640 1 Processor -- start 2024-03-09T09:30:11.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.573+0000 7f8e76c00640 1 -- start start 2024-03-09T09:30:11.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e76c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e7006b940 0x7f8e70145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e76c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e7006f320 0x7f8e701df030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e76c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e7014dad0 0x7f8e701ef010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e76c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7014e030 con 0x7f8e7006b940 2024-03-09T09:30:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e76c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7014deb0 con 0x7f8e7006f320 2024-03-09T09:30:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e76c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7006e9b0 con 0x7f8e7014dad0 2024-03-09T09:30:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e75800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e7006b940 0x7f8e70145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e75800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e7006b940 0x7f8e70145520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38096/0 (socket says 172.21.15.67:38096) 2024-03-09T09:30:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e75800640 1 -- 172.21.15.67:0/1625501604 learned_addr learned my addr 172.21.15.67:0/1625501604 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e75800640 1 -- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e7014dad0 msgr2=0x7f8e701ef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e74e00640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e7006f320 0x7f8e701df030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e75800640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e7014dad0 0x7f8e701ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e75800640 1 -- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e7006f320 msgr2=0x7f8e701df030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.574+0000 7f8e75800640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e7006f320 0x7f8e701df030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.575+0000 7f8e75800640 1 -- 172.21.15.67:0/1625501604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e701ef550 con 0x7f8e7006b940 2024-03-09T09:30:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.575+0000 7f8e75800640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e7006b940 0x7f8e70145520 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8e6402b830 tx=0x7f8e640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.575+0000 7f8e6d600640 1 -- 172.21.15.67:0/1625501604 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64042070 con 0x7f8e7006b940 2024-03-09T09:30:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.575+0000 7f8e6d600640 1 -- 172.21.15.67:0/1625501604 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8e6403d890 con 0x7f8e7006b940 2024-03-09T09:30:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.575+0000 7f8e6d600640 1 -- 172.21.15.67:0/1625501604 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64038040 con 0x7f8e7006b940 2024-03-09T09:30:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.575+0000 7f8e76c00640 1 -- 172.21.15.67:0/1625501604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e70145c20 con 0x7f8e7006b940 2024-03-09T09:30:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.575+0000 7f8e76c00640 1 -- 172.21.15.67:0/1625501604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e701464e0 con 0x7f8e7006b940 2024-03-09T09:30:11.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.577+0000 7f8e76c00640 1 -- 172.21.15.67:0/1625501604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7004f3c0 con 0x7f8e7006b940 2024-03-09T09:30:11.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.577+0000 7f8e6d600640 1 -- 172.21.15.67:0/1625501604 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8e64045070 con 0x7f8e7006b940 2024-03-09T09:30:11.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.579+0000 7f8e6d600640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e5006b4d0 0x7f8e5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:11.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.579+0000 7f8e6d600640 1 -- 172.21.15.67:0/1625501604 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f8e640ac540 con 0x7f8e7006b940 2024-03-09T09:30:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.579+0000 7f8e74e00640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e5006b4d0 0x7f8e5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.580+0000 7f8e74e00640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e5006b4d0 0x7f8e5006d990 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c006fd0 tx=0x7f8e5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:11.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.583+0000 7f8e6d600640 1 -- 172.21.15.67:0/1625501604 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8e6407b110 con 0x7f8e7006b940 2024-03-09T09:30:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.824+0000 7f8e76c00640 1 -- 172.21.15.67:0/1625501604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e701475d0 con 0x7f8e7006b940 2024-03-09T09:30:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.825+0000 7f8e6d600640 1 -- 172.21.15.67:0/1625501604 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6247 (secure 0 0 0) 0x7f8e6407aa10 con 0x7f8e7006b940 2024-03-09T09:30:11.833 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:30:11.834 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:17.736973+0000","modified":"2024-03-09T08:52:21.931108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6835/340976284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":340976284},{"type":"v1","addr":"172.21.15.134:6835","nonce":340976284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4282,"qdb_cluster":[4282]},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T09:30:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 14 2024-03-09T09:30:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.828+0000 7f8e76c00640 1 -- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e5006b4d0 msgr2=0x7f8e5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.828+0000 7f8e76c00640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e5006b4d0 0x7f8e5006d990 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c006fd0 tx=0x7f8e5c008040 comp rx=0 tx=0).stop 2024-03-09T09:30:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.828+0000 7f8e76c00640 1 -- 172.21.15.67:0/1625501604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e7006b940 msgr2=0x7f8e70145520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.828+0000 7f8e76c00640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e7006b940 0x7f8e70145520 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8e6402b830 tx=0x7f8e640046c0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.828+0000 7f8e76c00640 1 -- 172.21.15.67:0/1625501604 shutdown_connections 2024-03-09T09:30:11.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.828+0000 7f8e76c00640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f8e5006b4d0 0x7f8e5006d990 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.828+0000 7f8e76c00640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e7014dad0 0x7f8e701ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.828+0000 7f8e76c00640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e7006f320 0x7f8e701df030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.828+0000 7f8e76c00640 1 --2- 172.21.15.67:0/1625501604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8e7006b940 0x7f8e70145520 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:11.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.828+0000 7f8e76c00640 1 -- 172.21.15.67:0/1625501604 >> 172.21.15.67:0/1625501604 conn(0x7f8e7006d650 msgr2=0x7f8e701dd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:11.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.829+0000 7f8e76c00640 1 -- 172.21.15.67:0/1625501604 shutdown_connections 2024-03-09T09:30:11.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:11.829+0000 7f8e76c00640 1 -- 172.21.15.67:0/1625501604 wait complete. 2024-03-09T09:30:11.855 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T09:30:12.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.055+0000 7f1fb6200640 1 Processor -- start 2024-03-09T09:30:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.055+0000 7f1fb6200640 1 -- start start 2024-03-09T09:30:12.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.056+0000 7f1fb6200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fb0153370 0x7f1fb00c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.056+0000 7f1fb6200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1fb00c1d70 0x7f1fb00ac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.056+0000 7f1fb6200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb0156d50 con 0x7f1fb00c2400 2024-03-09T09:30:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.056+0000 7f1fb6200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb00aadf0 con 0x7f1fb00c1d70 2024-03-09T09:30:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.057+0000 7f1fb6200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb00aa4c0 con 0x7f1fb0153370 2024-03-09T09:30:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.057+0000 7f1faea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1fb00c1d70 0x7f1fb00ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.057+0000 7f1faea00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f1fb00c1d70 0x7f1fb00ac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38112/0 (socket says 172.21.15.67:38112) 2024-03-09T09:30:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.057+0000 7f1faea00640 1 -- 172.21.15.67:0/3849441202 learned_addr learned my addr 172.21.15.67:0/3849441202 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:12.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.057+0000 7f1fafe00640 1 --1- 172.21.15.67:0/3849441202 >> v1:172.21.15.67:6789/0 conn(0x7f1fb00c2400 0x7f1fb00acb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41740/0 (socket says 172.21.15.67:41740) 2024-03-09T09:30:12.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.057+0000 7f1faf400640 1 --2- 172.21.15.67:0/3849441202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fb0153370 0x7f1fb00c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.057+0000 7f1faea00640 1 -- 172.21.15.67:0/3849441202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fb0153370 msgr2=0x7f1fb00c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:12.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.057+0000 7f1faea00640 1 --2- 172.21.15.67:0/3849441202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fb0153370 0x7f1fb00c07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.057+0000 7f1faea00640 1 -- 172.21.15.67:0/3849441202 >> v1:172.21.15.67:6789/0 conn(0x7f1fb00c2400 legacy=0x7f1fb00acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.057+0000 7f1faea00640 1 -- 172.21.15.67:0/3849441202 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb00aa7a0 con 0x7f1fb00c1d70 2024-03-09T09:30:12.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.057+0000 7f1fae000640 1 -- 172.21.15.67:0/3849441202 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262486933 0 0) 0x7f1fb0156d50 con 0x7f1fb00c2400 2024-03-09T09:30:12.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.059+0000 7f1faea00640 1 --2- 172.21.15.67:0/3849441202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb00c1d70 0x7f1fb00ac610 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009870 tx=0x7f1f9c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844a0cf4a20b2f6a server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:12.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.059+0000 7f1fae000640 1 -- 172.21.15.67:0/3849441202 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c035020 con 0x7f1fb00c1d70 2024-03-09T09:30:12.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.060+0000 7f1fae000640 1 -- 172.21.15.67:0/3849441202 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1f9c031640 con 0x7f1fb00c1d70 2024-03-09T09:30:12.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.060+0000 7f1fae000640 1 -- 172.21.15.67:0/3849441202 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c02d040 con 0x7f1fb00c1d70 2024-03-09T09:30:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.060+0000 7f1fb6200640 1 -- 172.21.15.67:0/3849441202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb00c1d70 msgr2=0x7f1fb00ac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.060+0000 7f1fb6200640 1 --2- 172.21.15.67:0/3849441202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb00c1d70 0x7f1fb00ac610 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009870 tx=0x7f1f9c02b8f0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.060+0000 7f1fb6200640 1 -- 172.21.15.67:0/3849441202 shutdown_connections 2024-03-09T09:30:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.060+0000 7f1fb6200640 1 --2- 172.21.15.67:0/3849441202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb00c1d70 0x7f1fb00ac610 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.060+0000 7f1fb6200640 1 --2- 172.21.15.67:0/3849441202 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fb0153370 0x7f1fb00c07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.060+0000 7f1fb6200640 1 -- 172.21.15.67:0/3849441202 >> 172.21.15.67:0/3849441202 conn(0x7f1fb00c7370 msgr2=0x7f1fb00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.060+0000 7f1fb6200640 1 -- 172.21.15.67:0/3849441202 shutdown_connections 2024-03-09T09:30:12.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.060+0000 7f1fb6200640 1 -- 172.21.15.67:0/3849441202 wait complete. 2024-03-09T09:30:12.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.062+0000 7f1fb6200640 1 Processor -- start 2024-03-09T09:30:12.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.062+0000 7f1fb6200640 1 -- start start 2024-03-09T09:30:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.063+0000 7f1fb6200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fb00c1d70 0x7f1fb01550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.063+0000 7f1fb6200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb00c2400 0x7f1fb01572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.063+0000 7f1fb6200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fb0153370 0x7f1fb01eec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.063+0000 7f1fb6200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb0153750 con 0x7f1fb00c2400 2024-03-09T09:30:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.063+0000 7f1fb6200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb00ad260 con 0x7f1fb0153370 2024-03-09T09:30:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.063+0000 7f1fb6200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb01538d0 con 0x7f1fb00c1d70 2024-03-09T09:30:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.063+0000 7f1faea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb00c2400 0x7f1fb01572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1faea00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb00c2400 0x7f1fb01572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38122/0 (socket says 172.21.15.67:38122) 2024-03-09T09:30:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1faea00640 1 -- 172.21.15.67:0/1327919776 learned_addr learned my addr 172.21.15.67:0/1327919776 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1faf400640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fb00c1d70 0x7f1fb01550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1faea00640 1 -- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fb00c1d70 msgr2=0x7f1fb01550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1fafe00640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fb0153370 0x7f1fb01eec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1faea00640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fb00c1d70 0x7f1fb01550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1faea00640 1 -- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fb0153370 msgr2=0x7f1fb01eec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1faea00640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fb0153370 0x7f1fb01eec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1faea00640 1 -- 172.21.15.67:0/1327919776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb01eb720 con 0x7f1fb00c2400 2024-03-09T09:30:12.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1faf400640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fb00c1d70 0x7f1fb01550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:30:12.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1fafe00640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fb0153370 0x7f1fb01eec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:30:12.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.064+0000 7f1faea00640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb00c2400 0x7f1fb01572c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c000c00 tx=0x7f1f9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.065+0000 7f1fa7400640 1 -- 172.21.15.67:0/1327919776 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c040070 con 0x7f1fb00c2400 2024-03-09T09:30:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.065+0000 7f1fa7400640 1 -- 172.21.15.67:0/1327919776 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1f9c039bf0 con 0x7f1fb00c2400 2024-03-09T09:30:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.065+0000 7f1fa7400640 1 -- 172.21.15.67:0/1327919776 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c035040 con 0x7f1fb00c2400 2024-03-09T09:30:12.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.065+0000 7f1fb6200640 1 -- 172.21.15.67:0/1327919776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb01eba00 con 0x7f1fb00c2400 2024-03-09T09:30:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.065+0000 7f1fb6200640 1 -- 172.21.15.67:0/1327919776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb01ebfc0 con 0x7f1fb00c2400 2024-03-09T09:30:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.066+0000 7f1fb6200640 1 -- 172.21.15.67:0/1327919776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb0075a00 con 0x7f1fb00c2400 2024-03-09T09:30:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.067+0000 7f1fa7400640 1 -- 172.21.15.67:0/1327919776 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1f9c04b060 con 0x7f1fb00c2400 2024-03-09T09:30:12.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.068+0000 7f1fa7400640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1f9006b4d0 0x7f1f9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.068+0000 7f1fa7400640 1 -- 172.21.15.67:0/1327919776 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f1f9c0ac080 con 0x7f1fb00c2400 2024-03-09T09:30:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.069+0000 7f1faf400640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1f9006b4d0 0x7f1f9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.070+0000 7f1faf400640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1f9006b4d0 0x7f1f9006d990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0006fd0 tx=0x7f1fa0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:12.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.077+0000 7f1fa7400640 1 -- 172.21.15.67:0/1327919776 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1f9c07aaa0 con 0x7f1fb00c2400 2024-03-09T09:30:12.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.281+0000 7f1fb6200640 1 -- 172.21.15.67:0/1327919776 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fb00c2df0 con 0x7f1fb00c2400 2024-03-09T09:30:12.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.284+0000 7f1fa7400640 1 -- 172.21.15.67:0/1327919776 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+20649 (secure 0 0 0) 0x7f1f9c07a3a0 con 0x7f1fb00c2400 2024-03-09T09:30:12.293 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:30:12.293 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T09:27:40.166424+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-03-09T08:52:16.185356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-03-09T08:52:16.756829+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":765,"snap_epoch":79,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":765,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":748,"length":18}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T09:30:12.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 -- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1f9006b4d0 msgr2=0x7f1f9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1f9006b4d0 0x7f1f9006d990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0006fd0 tx=0x7f1fa0008040 comp rx=0 tx=0).stop 2024-03-09T09:30:12.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 -- 172.21.15.67:0/1327919776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb00c2400 msgr2=0x7f1fb01572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb00c2400 0x7f1fb01572c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c000c00 tx=0x7f1f9c0046c0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 -- 172.21.15.67:0/1327919776 shutdown_connections 2024-03-09T09:30:12.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f1f9006b4d0 0x7f1f9006d990 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fb0153370 0x7f1fb01eec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1fb00c2400 0x7f1fb01572c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 --2- 172.21.15.67:0/1327919776 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1fb00c1d70 0x7f1fb01550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 -- 172.21.15.67:0/1327919776 >> 172.21.15.67:0/1327919776 conn(0x7f1fb00c7370 msgr2=0x7f1fb01526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:12.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 -- 172.21.15.67:0/1327919776 shutdown_connections 2024-03-09T09:30:12.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.287+0000 7f1fb6200640 1 -- 172.21.15.67:0/1327919776 wait complete. 2024-03-09T09:30:12.314 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail a 2024-03-09T09:30:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.515+0000 7f3c36800640 1 Processor -- start 2024-03-09T09:30:12.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.516+0000 7f3c36800640 1 -- start start 2024-03-09T09:30:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.517+0000 7f3c36800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c30153c00 0x7f3c30153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.517+0000 7f3c36800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3c300c7130 0x7f3c30154520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.517+0000 7f3c36800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c300aaa40 con 0x7f3c30066e90 2024-03-09T09:30:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.517+0000 7f3c36800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c300ab000 con 0x7f3c300c7130 2024-03-09T09:30:12.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.517+0000 7f3c36800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300aad20 con 0x7f3c30153c00 2024-03-09T09:30:12.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.518+0000 7f3c35e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3c300c7130 0x7f3c30154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.518+0000 7f3c35e00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f3c300c7130 0x7f3c30154520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.67:54020/0 (socket says 172.21.15.67:54020) 2024-03-09T09:30:12.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.518+0000 7f3c35e00640 1 -- 172.21.15.67:0/396715537 learned_addr learned my addr 172.21.15.67:0/396715537 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:12.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.518+0000 7f3c35400640 1 --2- 172.21.15.67:0/396715537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c30153c00 0x7f3c30153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.518+0000 7f3c35400640 1 -- 172.21.15.67:0/396715537 >> v2:172.21.15.134:3300/0 conn(0x7f3c300c7130 msgr2=0x7f3c30154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:12.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.518+0000 7f3c35400640 1 --2- 172.21.15.67:0/396715537 >> v2:172.21.15.134:3300/0 conn(0x7f3c300c7130 0x7f3c30154520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.518+0000 7f3c35400640 1 -- 172.21.15.67:0/396715537 >> v1:172.21.15.134:6789/0 conn(0x7f3c30066e90 legacy=0x7f3c300c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.518+0000 7f3c35400640 1 -- 172.21.15.67:0/396715537 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c300ad100 con 0x7f3c30153c00 2024-03-09T09:30:12.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.519+0000 7f3c35400640 1 --2- 172.21.15.67:0/396715537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c30153c00 0x7f3c30153fe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009d90 tx=0x7f3c2402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c477e9e4a75ce391 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:12.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.520+0000 7f3c2fe00640 1 -- 172.21.15.67:0/396715537 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24039070 con 0x7f3c30153c00 2024-03-09T09:30:12.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.520+0000 7f3c2fe00640 1 -- 172.21.15.67:0/396715537 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3c2402cbc0 con 0x7f3c30153c00 2024-03-09T09:30:12.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.520+0000 7f3c2fe00640 1 -- 172.21.15.67:0/396715537 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24034440 con 0x7f3c30153c00 2024-03-09T09:30:12.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.520+0000 7f3c36800640 1 -- 172.21.15.67:0/396715537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c30153c00 msgr2=0x7f3c30153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.520+0000 7f3c36800640 1 --2- 172.21.15.67:0/396715537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c30153c00 0x7f3c30153fe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009d90 tx=0x7f3c2402be10 comp rx=0 tx=0).stop 2024-03-09T09:30:12.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.520+0000 7f3c36800640 1 -- 172.21.15.67:0/396715537 shutdown_connections 2024-03-09T09:30:12.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.520+0000 7f3c36800640 1 --2- 172.21.15.67:0/396715537 >> v2:172.21.15.134:3300/0 conn(0x7f3c300c7130 0x7f3c30154520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.520+0000 7f3c36800640 1 --2- 172.21.15.67:0/396715537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c30153c00 0x7f3c30153fe0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.520+0000 7f3c36800640 1 -- 172.21.15.67:0/396715537 >> 172.21.15.67:0/396715537 conn(0x7f3c300a9740 msgr2=0x7f3c300a9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:12.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.521+0000 7f3c36800640 1 -- 172.21.15.67:0/396715537 shutdown_connections 2024-03-09T09:30:12.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.521+0000 7f3c36800640 1 -- 172.21.15.67:0/396715537 wait complete. 2024-03-09T09:30:12.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.522+0000 7f3c36800640 1 Processor -- start 2024-03-09T09:30:12.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.522+0000 7f3c36800640 1 -- start start 2024-03-09T09:30:12.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.523+0000 7f3c36800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c30066e90 0x7f3c301db070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.523+0000 7f3c36800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c300c7130 0x7f3c30151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.523+0000 7f3c36800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c30153c00 0x7f3c30152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.523+0000 7f3c36800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300c5890 con 0x7f3c30153c00 2024-03-09T09:30:12.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.523+0000 7f3c36800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300ad6c0 con 0x7f3c30066e90 2024-03-09T09:30:12.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.523+0000 7f3c36800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300c5a10 con 0x7f3c300c7130 2024-03-09T09:30:12.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.524+0000 7f3c35e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c30153c00 0x7f3c30152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.524+0000 7f3c34a00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c300c7130 0x7f3c30151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.524+0000 7f3c35e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c30153c00 0x7f3c30152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38130/0 (socket says 172.21.15.67:38130) 2024-03-09T09:30:12.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.524+0000 7f3c35e00640 1 -- 172.21.15.67:0/194131535 learned_addr learned my addr 172.21.15.67:0/194131535 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:12.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.524+0000 7f3c35400640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c30066e90 0x7f3c301db070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.524+0000 7f3c35e00640 1 -- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c300c7130 msgr2=0x7f3c30151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.524+0000 7f3c35e00640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c300c7130 0x7f3c30151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.524+0000 7f3c35e00640 1 -- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c30066e90 msgr2=0x7f3c301db070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.524+0000 7f3c35e00640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c30066e90 0x7f3c301db070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.524+0000 7f3c35e00640 1 -- 172.21.15.67:0/194131535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c30073010 con 0x7f3c30153c00 2024-03-09T09:30:12.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.525+0000 7f3c35e00640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c30153c00 0x7f3c30152220 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800e8a0 tx=0x7f3c2800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.525+0000 7f3c2d600640 1 -- 172.21.15.67:0/194131535 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c28017070 con 0x7f3c30153c00 2024-03-09T09:30:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.525+0000 7f3c2d600640 1 -- 172.21.15.67:0/194131535 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3c2800c940 con 0x7f3c30153c00 2024-03-09T09:30:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.525+0000 7f3c2d600640 1 -- 172.21.15.67:0/194131535 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800cd50 con 0x7f3c30153c00 2024-03-09T09:30:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.525+0000 7f3c36800640 1 -- 172.21.15.67:0/194131535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c301ee330 con 0x7f3c30153c00 2024-03-09T09:30:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.525+0000 7f3c36800640 1 -- 172.21.15.67:0/194131535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c301ee8f0 con 0x7f3c30153c00 2024-03-09T09:30:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.526+0000 7f3c36800640 1 -- 172.21.15.67:0/194131535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3004f3c0 con 0x7f3c30153c00 2024-03-09T09:30:12.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.527+0000 7f3c2d600640 1 -- 172.21.15.67:0/194131535 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3c2801a070 con 0x7f3c30153c00 2024-03-09T09:30:12.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.528+0000 7f3c2d600640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c1006b500 0x7f3c1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:12.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.528+0000 7f3c2d600640 1 -- 172.21.15.67:0/194131535 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f3c2808a990 con 0x7f3c30153c00 2024-03-09T09:30:12.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.529+0000 7f3c35400640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c1006b500 0x7f3c1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:12.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.530+0000 7f3c35400640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c1006b500 0x7f3c1006d9c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3c24036000 tx=0x7f3c24030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:12.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.537+0000 7f3c2d600640 1 -- 172.21.15.67:0/194131535 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3c28058cd0 con 0x7f3c30153c00 2024-03-09T09:30:12.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.778+0000 7f3c36800640 1 -- 172.21.15.67:0/194131535 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "a"} v 0) v1 -- 0x7f3c301dc520 con 0x7f3c30153c00 2024-03-09T09:30:12.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.782+0000 7f3c2d600640 1 -- 172.21.15.67:0/194131535 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "a"}]=0 a marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 151+0+0 (secure 0 0 0) 0x7f3c2805cd70 con 0x7f3c30153c00 2024-03-09T09:30:12.790 INFO:teuthology.orchestra.run.smithi067.stderr:a marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-09T09:30:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.785+0000 7f3c36800640 1 -- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c1006b500 msgr2=0x7f3c1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.785+0000 7f3c36800640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c1006b500 0x7f3c1006d9c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3c24036000 tx=0x7f3c24030000 comp rx=0 tx=0).stop 2024-03-09T09:30:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.785+0000 7f3c36800640 1 -- 172.21.15.67:0/194131535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c30153c00 msgr2=0x7f3c30152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:12.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.785+0000 7f3c36800640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c30153c00 0x7f3c30152220 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800e8a0 tx=0x7f3c2800ed60 comp rx=0 tx=0).stop 2024-03-09T09:30:12.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.788+0000 7f3c36800640 1 -- 172.21.15.67:0/194131535 shutdown_connections 2024-03-09T09:30:12.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.788+0000 7f3c36800640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f3c1006b500 0x7f3c1006d9c0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.788+0000 7f3c36800640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c30153c00 0x7f3c30152220 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.788+0000 7f3c36800640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3c300c7130 0x7f3c30151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.788+0000 7f3c36800640 1 --2- 172.21.15.67:0/194131535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c30066e90 0x7f3c301db070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:12.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.788+0000 7f3c36800640 1 -- 172.21.15.67:0/194131535 >> 172.21.15.67:0/194131535 conn(0x7f3c300a9740 msgr2=0x7f3c300a7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:12.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.788+0000 7f3c36800640 1 -- 172.21.15.67:0/194131535 shutdown_connections 2024-03-09T09:30:12.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:12.789+0000 7f3c36800640 1 -- 172.21.15.67:0/194131535 wait complete. 2024-03-09T09:30:12.816 INFO:tasks.ceph.mds.d.smithi134.stderr: -6827> 2024-03-09T09:29:34.780+0000 7f9245e00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 49838) UID: 0 2024-03-09T09:30:12.819 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm a --yes-i-really-mean-it 2024-03-09T09:30:12.907 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-03-09T09:30:13.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.017+0000 7fcbb0c00640 1 Processor -- start 2024-03-09T09:30:13.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.018+0000 7fcbb0c00640 1 -- start start 2024-03-09T09:30:13.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.019+0000 7fcbb0c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 0x7fcbac0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:13.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.019+0000 7fcbb0c00640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fcbac0c9490 0x7fcbac0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.019+0000 7fcbb0c00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac0c9a30 con 0x7fcbac0c5f10 2024-03-09T09:30:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.019+0000 7fcbb0c00640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac0c9d10 con 0x7fcbac0c9490 2024-03-09T09:30:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.019+0000 7fcbb0c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0ca340 con 0x7fcbac06b040 2024-03-09T09:30:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.019+0000 7fcbab400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fcbac0c5f10 0x7fcbac1567d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41744/0 (socket says 172.21.15.67:41744) 2024-03-09T09:30:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.019+0000 7fcbab400640 1 -- 172.21.15.67:0/1570430309 learned_addr learned my addr 172.21.15.67:0/1570430309 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:30:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.020+0000 7fcbaaa00640 1 --2- 172.21.15.67:0/1570430309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 0x7fcbac0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:13.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.020+0000 7fcbabe00640 1 --2- 172.21.15.67:0/1570430309 >> v2:172.21.15.134:3300/0 conn(0x7fcbac0c9490 0x7fcbac0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.020+0000 7fcbaa000640 1 -- 172.21.15.67:0/1570430309 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140061439 0 0) 0x7fcbac0c9a30 con 0x7fcbac0c5f10 2024-03-09T09:30:13.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.020+0000 7fcbaa000640 1 -- 172.21.15.67:0/1570430309 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb90003520 con 0x7fcbac0c5f10 2024-03-09T09:30:13.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.020+0000 7fcbaaa00640 1 -- 172.21.15.67:0/1570430309 >> v2:172.21.15.134:3300/0 conn(0x7fcbac0c9490 msgr2=0x7fcbac0cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:13.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.020+0000 7fcbaaa00640 1 --2- 172.21.15.67:0/1570430309 >> v2:172.21.15.134:3300/0 conn(0x7fcbac0c9490 0x7fcbac0cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:13.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.020+0000 7fcbaaa00640 1 -- 172.21.15.67:0/1570430309 >> v1:172.21.15.67:6789/0 conn(0x7fcbac0c5f10 legacy=0x7fcbac1567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:13.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.020+0000 7fcbaaa00640 1 -- 172.21.15.67:0/1570430309 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0ca620 con 0x7fcbac06b040 2024-03-09T09:30:13.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.021+0000 7fcbaaa00640 1 --2- 172.21.15.67:0/1570430309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 0x7fcbac0c62f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb980097a0 tx=0x7fcb9802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e4865796cdc3afd server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.022+0000 7fcbaa000640 1 -- 172.21.15.67:0/1570430309 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98031780 con 0x7fcbac06b040 2024-03-09T09:30:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.022+0000 7fcbaa000640 1 -- 172.21.15.67:0/1570430309 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcb98037070 con 0x7fcbac06b040 2024-03-09T09:30:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.022+0000 7fcbaa000640 1 -- 172.21.15.67:0/1570430309 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98031b20 con 0x7fcbac06b040 2024-03-09T09:30:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.022+0000 7fcbb0c00640 1 -- 172.21.15.67:0/1570430309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 msgr2=0x7fcbac0c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.022+0000 7fcbb0c00640 1 --2- 172.21.15.67:0/1570430309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 0x7fcbac0c62f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb980097a0 tx=0x7fcb9802b820 comp rx=0 tx=0).stop 2024-03-09T09:30:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.022+0000 7fcbb0c00640 1 -- 172.21.15.67:0/1570430309 shutdown_connections 2024-03-09T09:30:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.022+0000 7fcbb0c00640 1 --2- 172.21.15.67:0/1570430309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 0x7fcbac0c62f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.022+0000 7fcbb0c00640 1 --2- 172.21.15.67:0/1570430309 >> v2:172.21.15.134:3300/0 conn(0x7fcbac0c9490 0x7fcbac0cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.022+0000 7fcbb0c00640 1 -- 172.21.15.67:0/1570430309 >> 172.21.15.67:0/1570430309 conn(0x7fcbac0cb590 msgr2=0x7fcbac0cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.023+0000 7fcbb0c00640 1 -- 172.21.15.67:0/1570430309 shutdown_connections 2024-03-09T09:30:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.023+0000 7fcbb0c00640 1 -- 172.21.15.67:0/1570430309 wait complete. 2024-03-09T09:30:13.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.024+0000 7fcbb0c00640 1 Processor -- start 2024-03-09T09:30:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.024+0000 7fcbb0c00640 1 -- start start 2024-03-09T09:30:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.025+0000 7fcbb0c00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 0x7fcbac14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.025+0000 7fcbb0c00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbac0c5f10 0x7fcbac1ef170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.025+0000 7fcbb0c00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbac0c9490 0x7fcbac1e9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.025+0000 7fcbb0c00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac1537a0 con 0x7fcbac0c5f10 2024-03-09T09:30:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.025+0000 7fcbb0c00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac153620 con 0x7fcbac0c9490 2024-03-09T09:30:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.025+0000 7fcbb0c00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0cb080 con 0x7fcbac06b040 2024-03-09T09:30:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.025+0000 7fcbaaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbac0c5f10 0x7fcbac1ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.025+0000 7fcbaaa00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbac0c5f10 0x7fcbac1ef170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38142/0 (socket says 172.21.15.67:38142) 2024-03-09T09:30:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.025+0000 7fcbaaa00640 1 -- 172.21.15.67:0/178581868 learned_addr learned my addr 172.21.15.67:0/178581868 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.025+0000 7fcbab400640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 0x7fcbac14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcbabe00640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbac0c9490 0x7fcbac1e9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcbaaa00640 1 -- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 msgr2=0x7fcbac14ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:13.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcbaaa00640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 0x7fcbac14ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:13.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcbaaa00640 1 -- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbac0c9490 msgr2=0x7fcbac1e9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:13.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcbaaa00640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbac0c9490 0x7fcbac1e9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:13.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcbaaa00640 1 -- 172.21.15.67:0/178581868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac14af80 con 0x7fcbac0c5f10 2024-03-09T09:30:13.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcbab400640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 0x7fcbac14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:30:13.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcbaaa00640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbac0c5f10 0x7fcbac1ef170 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb980097a0 tx=0x7fcb980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:13.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcba3400640 1 -- 172.21.15.67:0/178581868 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98006e10 con 0x7fcbac0c5f10 2024-03-09T09:30:13.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcba3400640 1 -- 172.21.15.67:0/178581868 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcb98031780 con 0x7fcbac0c5f10 2024-03-09T09:30:13.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcbb0c00640 1 -- 172.21.15.67:0/178581868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac1efab0 con 0x7fcbac0c5f10 2024-03-09T09:30:13.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.026+0000 7fcba3400640 1 -- 172.21.15.67:0/178581868 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98031b90 con 0x7fcbac0c5f10 2024-03-09T09:30:13.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.027+0000 7fcbb0c00640 1 -- 172.21.15.67:0/178581868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac14b500 con 0x7fcbac0c5f10 2024-03-09T09:30:13.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.028+0000 7fcbb0c00640 1 -- 172.21.15.67:0/178581868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac04f3c0 con 0x7fcbac0c5f10 2024-03-09T09:30:13.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.030+0000 7fcba3400640 1 -- 172.21.15.67:0/178581868 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcb98031d30 con 0x7fcbac0c5f10 2024-03-09T09:30:13.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.031+0000 7fcba3400640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb8c06b4d0 0x7fcb8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:13.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.031+0000 7fcba3400640 1 -- 172.21.15.67:0/178581868 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fcb980acd70 con 0x7fcbac0c5f10 2024-03-09T09:30:13.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.031+0000 7fcbab400640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb8c06b4d0 0x7fcb8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:13.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.032+0000 7fcbab400640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb8c06b4d0 0x7fcb8c06d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c007fd0 tx=0x7fcb9c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:13.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.035+0000 7fcba3400640 1 -- 172.21.15.67:0/178581868 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcb9802f050 con 0x7fcbac0c5f10 2024-03-09T09:30:13.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.286+0000 7fcbb0c00640 1 -- 172.21.15.67:0/178581868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "a", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcbac0cc260 con 0x7fcbac0c5f10 2024-03-09T09:30:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.825+0000 7fcba3400640 1 -- 172.21.15.67:0/178581868 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "a", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 99+0+0 (secure 0 0 0) 0x7fcb9807b7f0 con 0x7fcbac0c5f10 2024-03-09T09:30:13.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.828+0000 7fcbb0c00640 1 -- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb8c06b4d0 msgr2=0x7fcb8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:13.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.828+0000 7fcbb0c00640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb8c06b4d0 0x7fcb8c06d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c007fd0 tx=0x7fcb9c009040 comp rx=0 tx=0).stop 2024-03-09T09:30:13.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.828+0000 7fcbb0c00640 1 -- 172.21.15.67:0/178581868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbac0c5f10 msgr2=0x7fcbac1ef170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:13.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.828+0000 7fcbb0c00640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbac0c5f10 0x7fcbac1ef170 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb980097a0 tx=0x7fcb980046c0 comp rx=0 tx=0).stop 2024-03-09T09:30:13.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.829+0000 7fcbb0c00640 1 -- 172.21.15.67:0/178581868 shutdown_connections 2024-03-09T09:30:13.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.829+0000 7fcbb0c00640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fcb8c06b4d0 0x7fcb8c06d990 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:13.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.829+0000 7fcbb0c00640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbac0c9490 0x7fcbac1e9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:13.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.829+0000 7fcbb0c00640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbac0c5f10 0x7fcbac1ef170 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:13.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.829+0000 7fcbb0c00640 1 --2- 172.21.15.67:0/178581868 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbac06b040 0x7fcbac14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:13.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.829+0000 7fcbb0c00640 1 -- 172.21.15.67:0/178581868 >> 172.21.15.67:0/178581868 conn(0x7fcbac0cb590 msgr2=0x7fcbac06b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:13.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.829+0000 7fcbb0c00640 1 -- 172.21.15.67:0/178581868 shutdown_connections 2024-03-09T09:30:13.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:13.829+0000 7fcbb0c00640 1 -- 172.21.15.67:0/178581868 wait complete. 2024-03-09T09:30:13.859 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm a_metadata a_metadata --yes-i-really-really-mean-it 2024-03-09T09:30:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.097+0000 7fc371000640 1 Processor -- start 2024-03-09T09:30:14.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.097+0000 7fc371000640 1 -- start start 2024-03-09T09:30:14.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.099+0000 7fc371000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7fc36c0c9380 0x7fc36c06b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.099+0000 7fc371000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36c0c9800 con 0x7fc36c153310 2024-03-09T09:30:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.099+0000 7fc371000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36c0c6130 con 0x7fc36c0c7e60 2024-03-09T09:30:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.099+0000 7fc371000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36c06b4c0 con 0x7fc36c0c9380 2024-03-09T09:30:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.099+0000 7fc36b400640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7fc36c153310 0x7fc36c1565c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41748/0 (socket says 172.21.15.67:41748) 2024-03-09T09:30:14.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.099+0000 7fc36b400640 1 -- 172.21.15.67:0/1695958171 learned_addr learned my addr 172.21.15.67:0/1695958171 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:30:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.099+0000 7fc36aa00640 1 --2- 172.21.15.67:0/1695958171 >> v2:172.21.15.134:3300/0 conn(0x7fc36c0c9380 0x7fc36c06b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.099+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312373790 0 0) 0x7fc35c001490 con 0x7fc36c153310 2024-03-09T09:30:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.099+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc350003520 con 0x7fc36c153310 2024-03-09T09:30:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1686650964 0 0) 0x7fc350003520 con 0x7fc36c153310 2024-03-09T09:30:14.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 >> v2:172.21.15.134:3300/0 conn(0x7fc36c0c9380 msgr2=0x7fc36c06b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 --2- 172.21.15.67:0/1695958171 >> v2:172.21.15.134:3300/0 conn(0x7fc36c0c9380 0x7fc36c06b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 >> v1:172.21.15.134:6789/0 conn(0x7fc36c0c7e60 legacy=0x7fc36c1604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c0c6410 con 0x7fc36c153310 2024-03-09T09:30:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7fc35c001bc0 con 0x7fc36c153310 2024-03-09T09:30:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 >> v1:172.21.15.67:6789/0 conn(0x7fc36c153310 legacy=0x7fc36c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 --2- 172.21.15.67:0/1695958171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc350003a00 0x7fc350003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 --2- 172.21.15.67:0/1695958171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc350004800 0x7fc350004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:14.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 --2- 172.21.15.67:0/1695958171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 0x7fc35000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:14.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c06b4c0 con 0x7fc3500052e0 2024-03-09T09:30:14.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35000c1e0 con 0x7fc350004800 2024-03-09T09:30:14.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35000c3e0 con 0x7fc350003a00 2024-03-09T09:30:14.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36be00640 1 --2- 172.21.15.67:0/1695958171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 0x7fc35000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:14.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.100+0000 7fc36b400640 1 --2- 172.21.15.67:0/1695958171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc350003a00 0x7fc350003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:14.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.101+0000 7fc36aa00640 1 --2- 172.21.15.67:0/1695958171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc350004800 0x7fc350004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:14.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.101+0000 7fc36be00640 1 -- 172.21.15.67:0/1695958171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc350003a00 msgr2=0x7fc350003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:14.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.101+0000 7fc36be00640 1 --2- 172.21.15.67:0/1695958171 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc350003a00 0x7fc350003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.101+0000 7fc36be00640 1 -- 172.21.15.67:0/1695958171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc350004800 msgr2=0x7fc350004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.101+0000 7fc36be00640 1 --2- 172.21.15.67:0/1695958171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc350004800 0x7fc350004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.101+0000 7fc36be00640 1 -- 172.21.15.67:0/1695958171 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc350003520 con 0x7fc3500052e0 2024-03-09T09:30:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.102+0000 7fc36be00640 1 -- 172.21.15.67:0/1695958171 reap_dead start 2024-03-09T09:30:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.102+0000 7fc36be00640 1 --2- 172.21.15.67:0/1695958171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 0x7fc35000b9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc364001750 tx=0x7fc350005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee46fa62b2a88e85 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.102+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364001370 con 0x7fc3500052e0 2024-03-09T09:30:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.102+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc364005310 con 0x7fc3500052e0 2024-03-09T09:30:14.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.102+0000 7fc36a000640 1 -- 172.21.15.67:0/1695958171 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364003850 con 0x7fc3500052e0 2024-03-09T09:30:14.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.103+0000 7fc371000640 1 -- 172.21.15.67:0/1695958171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 msgr2=0x7fc35000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.103+0000 7fc371000640 1 --2- 172.21.15.67:0/1695958171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 0x7fc35000b9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc364001750 tx=0x7fc350005830 comp rx=0 tx=0).stop 2024-03-09T09:30:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.103+0000 7fc371000640 1 -- 172.21.15.67:0/1695958171 shutdown_connections 2024-03-09T09:30:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.103+0000 7fc371000640 1 --2- 172.21.15.67:0/1695958171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 0x7fc35000b9c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.103+0000 7fc371000640 1 -- 172.21.15.67:0/1695958171 >> 172.21.15.67:0/1695958171 conn(0x7fc36c0acf90 msgr2=0x7fc36c0ad3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.103+0000 7fc371000640 1 -- 172.21.15.67:0/1695958171 shutdown_connections 2024-03-09T09:30:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.103+0000 7fc371000640 1 -- 172.21.15.67:0/1695958171 wait complete. 2024-03-09T09:30:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.106+0000 7fc371000640 1 Processor -- start 2024-03-09T09:30:14.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.106+0000 7fc371000640 1 -- start start 2024-03-09T09:30:14.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.107+0000 7fc371000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 0x7fc36c0c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc371000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc36c0c2550 0x7fc36c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc371000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc36c0c2b80 0x7fc36c0c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc371000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c0c5e00 con 0x7fc3500052e0 2024-03-09T09:30:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc371000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c0c66f0 con 0x7fc36c0c2550 2024-03-09T09:30:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc371000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c06cd30 con 0x7fc36c0c2b80 2024-03-09T09:30:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc36b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 0x7fc36c0c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc36b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 0x7fc36c0c33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38154/0 (socket says 172.21.15.67:38154) 2024-03-09T09:30:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc36b400640 1 -- 172.21.15.67:0/998001820 learned_addr learned my addr 172.21.15.67:0/998001820 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc36be00640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc36c0c2b80 0x7fc36c0c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc36aa00640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc36c0c2550 0x7fc36c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc36b400640 1 -- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc36c0c2b80 msgr2=0x7fc36c0c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc36b400640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc36c0c2b80 0x7fc36c0c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc36b400640 1 -- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc36c0c2550 msgr2=0x7fc36c14fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc36b400640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc36c0c2550 0x7fc36c14fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.108+0000 7fc36b400640 1 -- 172.21.15.67:0/998001820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c1e2a00 con 0x7fc3500052e0 2024-03-09T09:30:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.109+0000 7fc36b400640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 0x7fc36c0c33b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00aa30 tx=0x7fc35c00aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.109+0000 7fc363400640 1 -- 172.21.15.67:0/998001820 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c017020 con 0x7fc3500052e0 2024-03-09T09:30:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.109+0000 7fc363400640 1 -- 172.21.15.67:0/998001820 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc35c007970 con 0x7fc3500052e0 2024-03-09T09:30:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.109+0000 7fc371000640 1 -- 172.21.15.67:0/998001820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c1ec1e0 con 0x7fc3500052e0 2024-03-09T09:30:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.109+0000 7fc363400640 1 -- 172.21.15.67:0/998001820 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c013630 con 0x7fc3500052e0 2024-03-09T09:30:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.109+0000 7fc371000640 1 -- 172.21.15.67:0/998001820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c150040 con 0x7fc3500052e0 2024-03-09T09:30:14.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.110+0000 7fc371000640 1 -- 172.21.15.67:0/998001820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c04f430 con 0x7fc3500052e0 2024-03-09T09:30:14.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.111+0000 7fc363400640 1 -- 172.21.15.67:0/998001820 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc35c01a020 con 0x7fc3500052e0 2024-03-09T09:30:14.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.112+0000 7fc363400640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc34806b500 0x7fc34806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:14.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.112+0000 7fc363400640 1 -- 172.21.15.67:0/998001820 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fc35c08b090 con 0x7fc3500052e0 2024-03-09T09:30:14.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.113+0000 7fc36aa00640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc34806b500 0x7fc34806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:14.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.114+0000 7fc36aa00640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc34806b500 0x7fc34806d9c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc354009a80 tx=0x7fc354008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:14.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.117+0000 7fc363400640 1 -- 172.21.15.67:0/998001820 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc35c021020 con 0x7fc3500052e0 2024-03-09T09:30:14.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.319+0000 7fc371000640 1 -- 172.21.15.67:0/998001820 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "a_metadata", "pool2": "a_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc36c06cb10 con 0x7fc3500052e0 2024-03-09T09:30:14.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.844+0000 7fc363400640 1 -- 172.21.15.67:0/998001820 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "a_metadata", "pool2": "a_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'a_metadata' removed v91) v1 ==== 166+0+0 (secure 0 0 0) 0x7fc36c0c9800 con 0x7fc3500052e0 2024-03-09T09:30:14.853 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'a_metadata' removed 2024-03-09T09:30:14.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 -- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc34806b500 msgr2=0x7fc34806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:14.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc34806b500 0x7fc34806d9c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc354009a80 tx=0x7fc354008040 comp rx=0 tx=0).stop 2024-03-09T09:30:14.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 -- 172.21.15.67:0/998001820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 msgr2=0x7fc36c0c33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:14.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 0x7fc36c0c33b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00aa30 tx=0x7fc35c00aef0 comp rx=0 tx=0).stop 2024-03-09T09:30:14.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 -- 172.21.15.67:0/998001820 shutdown_connections 2024-03-09T09:30:14.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7fc34806b500 0x7fc34806d9c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:14.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc36c0c2b80 0x7fc36c0c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:14.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc36c0c2550 0x7fc36c14fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:14.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 --2- 172.21.15.67:0/998001820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc3500052e0 0x7fc36c0c33b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:14.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 -- 172.21.15.67:0/998001820 >> 172.21.15.67:0/998001820 conn(0x7fc36c0acf90 msgr2=0x7fc36c0c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:14.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 -- 172.21.15.67:0/998001820 shutdown_connections 2024-03-09T09:30:14.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:14.848+0000 7fc371000640 1 -- 172.21.15.67:0/998001820 wait complete. 2024-03-09T09:30:14.880 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm a_data a_data --yes-i-really-really-mean-it 2024-03-09T09:30:15.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.100+0000 7f94d2800640 1 Processor -- start 2024-03-09T09:30:15.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.101+0000 7f94d2800640 1 -- start start 2024-03-09T09:30:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.102+0000 7f94d2800640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f94cc153410 0x7f94cc0c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.102+0000 7f94d2800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 0x7f94cc06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.102+0000 7f94d2800640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94cc06ce30 con 0x7f94cc0bec10 2024-03-09T09:30:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.102+0000 7f94d2800640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94cc156b00 con 0x7f94cc153410 2024-03-09T09:30:15.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.102+0000 7f94d2800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc155c30 con 0x7f94cc0c95d0 2024-03-09T09:30:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.102+0000 7f94d1e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 0x7f94cc06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:15.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.102+0000 7f94d1e00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 0x7f94cc06b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.67:35822/0 (socket says 172.21.15.67:35822) 2024-03-09T09:30:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.103+0000 7f94d1e00640 1 -- 172.21.15.67:0/985710885 learned_addr learned my addr 172.21.15.67:0/985710885 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.103+0000 7f94d1400640 1 --2- 172.21.15.67:0/985710885 >> v2:172.21.15.134:3300/0 conn(0x7f94cc153410 0x7f94cc0c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.103+0000 7f94d1e00640 1 -- 172.21.15.67:0/985710885 >> v2:172.21.15.134:3300/0 conn(0x7f94cc153410 msgr2=0x7f94cc0c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.103+0000 7f94d1e00640 1 --2- 172.21.15.67:0/985710885 >> v2:172.21.15.134:3300/0 conn(0x7f94cc153410 0x7f94cc0c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.103+0000 7f94cbe00640 1 -- 172.21.15.67:0/985710885 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129577979 0 0) 0x7f94cc06ce30 con 0x7f94cc0bec10 2024-03-09T09:30:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.103+0000 7f94d1e00640 1 -- 172.21.15.67:0/985710885 >> v1:172.21.15.134:6789/0 conn(0x7f94cc0bec10 legacy=0x7f94cc06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:15.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.103+0000 7f94d1e00640 1 -- 172.21.15.67:0/985710885 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc155ef0 con 0x7f94cc0c95d0 2024-03-09T09:30:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.105+0000 7f94d1e00640 1 --2- 172.21.15.67:0/985710885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 0x7f94cc06b900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009e50 tx=0x7f94c402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a0b1766f42f379c server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.105+0000 7f94cbe00640 1 -- 172.21.15.67:0/985710885 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c402ca70 con 0x7f94cc0c95d0 2024-03-09T09:30:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.105+0000 7f94cbe00640 1 -- 172.21.15.67:0/985710885 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f94c402cc10 con 0x7f94cc0c95d0 2024-03-09T09:30:15.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.105+0000 7f94cbe00640 1 -- 172.21.15.67:0/985710885 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c40343f0 con 0x7f94cc0c95d0 2024-03-09T09:30:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.106+0000 7f94d2800640 1 -- 172.21.15.67:0/985710885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 msgr2=0x7f94cc06b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:15.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.106+0000 7f94d2800640 1 --2- 172.21.15.67:0/985710885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 0x7f94cc06b900 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009e50 tx=0x7f94c402bed0 comp rx=0 tx=0).stop 2024-03-09T09:30:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.106+0000 7f94d2800640 1 -- 172.21.15.67:0/985710885 shutdown_connections 2024-03-09T09:30:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.106+0000 7f94d2800640 1 --2- 172.21.15.67:0/985710885 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 0x7f94cc06b900 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.106+0000 7f94d2800640 1 --2- 172.21.15.67:0/985710885 >> v2:172.21.15.134:3300/0 conn(0x7f94cc153410 0x7f94cc0c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.106+0000 7f94d2800640 1 -- 172.21.15.67:0/985710885 >> 172.21.15.67:0/985710885 conn(0x7f94cc0ad0d0 msgr2=0x7f94cc0ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.106+0000 7f94d2800640 1 -- 172.21.15.67:0/985710885 shutdown_connections 2024-03-09T09:30:15.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.106+0000 7f94d2800640 1 -- 172.21.15.67:0/985710885 wait complete. 2024-03-09T09:30:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.108+0000 7f94d2800640 1 Processor -- start 2024-03-09T09:30:15.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.108+0000 7f94d2800640 1 -- start start 2024-03-09T09:30:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.109+0000 7f94d2800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94cc0bec10 0x7f94cc146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.109+0000 7f94d2800640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 0x7f94cc0ac330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.109+0000 7f94d2800640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94cc153410 0x7f94cc158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.109+0000 7f94d2800640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc156790 con 0x7f94cc0bec10 2024-03-09T09:30:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.109+0000 7f94d2800640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc1564b0 con 0x7f94cc153410 2024-03-09T09:30:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.109+0000 7f94d2800640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc06be40 con 0x7f94cc0c95d0 2024-03-09T09:30:15.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.109+0000 7f94d1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94cc0bec10 0x7f94cc146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.109+0000 7f94d1400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94cc0bec10 0x7f94cc146be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38156/0 (socket says 172.21.15.67:38156) 2024-03-09T09:30:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.109+0000 7f94d1400640 1 -- 172.21.15.67:0/49434987 learned_addr learned my addr 172.21.15.67:0/49434987 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.110+0000 7f94d0a00640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 0x7f94cc0ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.110+0000 7f94d1e00640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94cc153410 0x7f94cc158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.110+0000 7f94d1400640 1 -- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 msgr2=0x7f94cc0ac330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.110+0000 7f94d1400640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 0x7f94cc0ac330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.110+0000 7f94d1400640 1 -- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94cc153410 msgr2=0x7f94cc158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:15.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.110+0000 7f94d1400640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94cc153410 0x7f94cc158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.110+0000 7f94d1400640 1 -- 172.21.15.67:0/49434987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc158090 con 0x7f94cc0bec10 2024-03-09T09:30:15.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.110+0000 7f94d1400640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94cc0bec10 0x7f94cc146be0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f94c000e870 tx=0x7f94c000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.111+0000 7f94c9600640 1 -- 172.21.15.67:0/49434987 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0017070 con 0x7f94cc0bec10 2024-03-09T09:30:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.111+0000 7f94c9600640 1 -- 172.21.15.67:0/49434987 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f94c000c940 con 0x7f94cc0bec10 2024-03-09T09:30:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.111+0000 7f94c9600640 1 -- 172.21.15.67:0/49434987 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c000cd50 con 0x7f94cc0bec10 2024-03-09T09:30:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.111+0000 7f94d2800640 1 -- 172.21.15.67:0/49434987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94cc1ecc50 con 0x7f94cc0bec10 2024-03-09T09:30:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.111+0000 7f94d2800640 1 -- 172.21.15.67:0/49434987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94cc1ec3e0 con 0x7f94cc0bec10 2024-03-09T09:30:15.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.112+0000 7f94d2800640 1 -- 172.21.15.67:0/49434987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94cc1ec1f0 con 0x7f94cc0bec10 2024-03-09T09:30:15.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.113+0000 7f94c9600640 1 -- 172.21.15.67:0/49434987 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f94c001a070 con 0x7f94cc0bec10 2024-03-09T09:30:15.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.114+0000 7f94c9600640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f94ac06b4d0 0x7f94ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:15.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.115+0000 7f94c9600640 1 -- 172.21.15.67:0/49434987 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6295+0+0 (secure 0 0 0) 0x7f94c008a330 con 0x7f94cc0bec10 2024-03-09T09:30:15.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.115+0000 7f94d0a00640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f94ac06b4d0 0x7f94ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:15.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.116+0000 7f94d0a00640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f94ac06b4d0 0x7f94ac06d990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000f60 tx=0x7f94b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:15.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.119+0000 7f94c9600640 1 -- 172.21.15.67:0/49434987 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f94c0058f50 con 0x7f94cc0bec10 2024-03-09T09:30:15.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.322+0000 7f94d2800640 1 -- 172.21.15.67:0/49434987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "a_data", "pool2": "a_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94cc04f430 con 0x7f94cc0bec10 2024-03-09T09:30:15.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.854+0000 7f94c9600640 1 -- 172.21.15.67:0/49434987 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "a_data", "pool2": "a_data", "yes_i_really_really_mean_it": true}]=0 pool 'a_data' removed v92) v1 ==== 154+0+0 (secure 0 0 0) 0x7f94c0058850 con 0x7f94cc0bec10 2024-03-09T09:30:15.863 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'a_data' removed 2024-03-09T09:30:15.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.858+0000 7f94d2800640 1 -- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f94ac06b4d0 msgr2=0x7f94ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.858+0000 7f94d2800640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f94ac06b4d0 0x7f94ac06d990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000f60 tx=0x7f94b8009040 comp rx=0 tx=0).stop 2024-03-09T09:30:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.858+0000 7f94d2800640 1 -- 172.21.15.67:0/49434987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94cc0bec10 msgr2=0x7f94cc146be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.858+0000 7f94d2800640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94cc0bec10 0x7f94cc146be0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f94c000e870 tx=0x7f94c000ed30 comp rx=0 tx=0).stop 2024-03-09T09:30:15.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.861+0000 7f94d2800640 1 -- 172.21.15.67:0/49434987 shutdown_connections 2024-03-09T09:30:15.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.862+0000 7f94d2800640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f94ac06b4d0 0x7f94ac06d990 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.862+0000 7f94d2800640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94cc153410 0x7f94cc158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.862+0000 7f94d2800640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94cc0c95d0 0x7f94cc0ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.862+0000 7f94d2800640 1 --2- 172.21.15.67:0/49434987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94cc0bec10 0x7f94cc146be0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:15.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.862+0000 7f94d2800640 1 -- 172.21.15.67:0/49434987 >> 172.21.15.67:0/49434987 conn(0x7f94cc0ad0d0 msgr2=0x7f94cc1544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.862+0000 7f94d2800640 1 -- 172.21.15.67:0/49434987 shutdown_connections 2024-03-09T09:30:15.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:15.862+0000 7f94d2800640 1 -- 172.21.15.67:0/49434987 wait complete. 2024-03-09T09:30:15.910 INFO:tasks.cephfs.filesystem:Destroying file system b and related pools 2024-03-09T09:30:15.910 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T09:30:16.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.138+0000 7f990ca00640 1 Processor -- start 2024-03-09T09:30:16.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.138+0000 7f990ca00640 1 -- start start 2024-03-09T09:30:16.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.139+0000 7f990ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9908153300 0x7f99080b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.139+0000 7f990ca00640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9908157000 con 0x7f99080cadb0 2024-03-09T09:30:16.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.139+0000 7f990ca00640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99080ae430 con 0x7f99080c5e40 2024-03-09T09:30:16.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.139+0000 7f990ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99080ae6f0 con 0x7f9908153300 2024-03-09T09:30:16.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.140+0000 7f9907200640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f99080cadb0 0x7f9908160490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:41754/0 (socket says 172.21.15.67:41754) 2024-03-09T09:30:16.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.140+0000 7f9907200640 1 -- 172.21.15.67:0/1856960451 learned_addr learned my addr 172.21.15.67:0/1856960451 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:30:16.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.140+0000 7f9906800640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9908153300 0x7f99080b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.140+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192519183 0 0) 0x7f9908157000 con 0x7f99080cadb0 2024-03-09T09:30:16.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.140+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98ec003520 con 0x7f99080cadb0 2024-03-09T09:30:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.140+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 <== mon.1 v1:172.21.15.134:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1793351694 0 0) 0x7f99080ae430 con 0x7f99080c5e40 2024-03-09T09:30:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.140+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 --> v1:172.21.15.134:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9908157000 con 0x7f99080c5e40 2024-03-09T09:30:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.140+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 992354934 0 0) 0x7f98ec003520 con 0x7f99080cadb0 2024-03-09T09:30:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.140+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9908153300 msgr2=0x7f99080b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.140+0000 7f9905400640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9908153300 0x7f99080b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.140+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 >> v1:172.21.15.134:6789/0 conn(0x7f99080c5e40 legacy=0x7f9908156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9908155b10 con 0x7f99080cadb0 2024-03-09T09:30:16.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7f99000019f0 con 0x7f99080cadb0 2024-03-09T09:30:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 >> v1:172.21.15.67:6789/0 conn(0x7f99080cadb0 legacy=0x7f9908160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9905400640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98ec003be0 0x7f98ec003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9905400640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9908153300 0x7f98ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9905400640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98ec004d40 0x7f98ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99080ae6f0 con 0x7f98ec003be0 2024-03-09T09:30:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec00baa0 con 0x7f98ec004d40 2024-03-09T09:30:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec00bca0 con 0x7f9908153300 2024-03-09T09:30:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9906800640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98ec003be0 0x7f98ec003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9905e00640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9908153300 0x7f98ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9906800640 1 -- 172.21.15.67:0/1856960451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9908153300 msgr2=0x7f98ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9906800640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9908153300 0x7f98ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9906800640 1 -- 172.21.15.67:0/1856960451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98ec004d40 msgr2=0x7f98ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9906800640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98ec004d40 0x7f98ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9906800640 1 -- 172.21.15.67:0/1856960451 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec003520 con 0x7f98ec003be0 2024-03-09T09:30:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.141+0000 7f9905e00640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9908153300 0x7f98ec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:30:16.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.142+0000 7f9906800640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98ec003be0 0x7f98ec003fe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009a90 tx=0x7f98ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e250c87341f08485 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.143+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8039070 con 0x7f98ec003be0 2024-03-09T09:30:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.143+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f98f802c680 con 0x7f98ec003be0 2024-03-09T09:30:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.143+0000 7f9905400640 1 -- 172.21.15.67:0/1856960451 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f8009690 con 0x7f98ec003be0 2024-03-09T09:30:16.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.143+0000 7f990ca00640 1 -- 172.21.15.67:0/1856960451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98ec003be0 msgr2=0x7f98ec003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:16.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.143+0000 7f990ca00640 1 --2- 172.21.15.67:0/1856960451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98ec003be0 0x7f98ec003fe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009a90 tx=0x7f98ec005830 comp rx=0 tx=0).stop 2024-03-09T09:30:16.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.143+0000 7f9907200640 1 -- 172.21.15.67:0/1856960451 reap_dead start 2024-03-09T09:30:16.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.143+0000 7f990ca00640 1 -- 172.21.15.67:0/1856960451 shutdown_connections 2024-03-09T09:30:16.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.143+0000 7f990ca00640 1 -- 172.21.15.67:0/1856960451 >> 172.21.15.67:0/1856960451 conn(0x7f99080732d0 msgr2=0x7f99080ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:16.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.143+0000 7f990ca00640 1 -- 172.21.15.67:0/1856960451 shutdown_connections 2024-03-09T09:30:16.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.143+0000 7f990ca00640 1 -- 172.21.15.67:0/1856960451 wait complete. 2024-03-09T09:30:16.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.145+0000 7f990ca00640 1 Processor -- start 2024-03-09T09:30:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.145+0000 7f990ca00640 1 -- start start 2024-03-09T09:30:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.146+0000 7f990ca00640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98ec003be0 0x7f99080b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.146+0000 7f990ca00640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99080c6c80 0x7f990814bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.146+0000 7f990ca00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99080c70d0 0x7f990814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.146+0000 7f990ca00640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99080c59e0 con 0x7f99080c70d0 2024-03-09T09:30:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.146+0000 7f990ca00640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9908159e90 con 0x7f99080c6c80 2024-03-09T09:30:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.146+0000 7f990ca00640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9908156720 con 0x7f98ec003be0 2024-03-09T09:30:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.146+0000 7f9907200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99080c70d0 0x7f990814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.146+0000 7f9907200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99080c70d0 0x7f990814c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38180/0 (socket says 172.21.15.67:38180) 2024-03-09T09:30:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.146+0000 7f9907200640 1 -- 172.21.15.67:0/650289628 learned_addr learned my addr 172.21.15.67:0/650289628 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f9905e00640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99080c6c80 0x7f990814bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f9906800640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98ec003be0 0x7f99080b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f9907200640 1 -- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98ec003be0 msgr2=0x7f99080b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f9907200640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98ec003be0 0x7f99080b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f9907200640 1 -- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99080c6c80 msgr2=0x7f990814bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f9907200640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99080c6c80 0x7f990814bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f9907200640 1 -- 172.21.15.67:0/650289628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99081ef920 con 0x7f99080c70d0 2024-03-09T09:30:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f9906800640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98ec003be0 0x7f99080b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-09T09:30:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f9907200640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99080c70d0 0x7f990814c060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f99080adc60 tx=0x7f990000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f98fea00640 1 -- 172.21.15.67:0/650289628 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900004d60 con 0x7f99080c70d0 2024-03-09T09:30:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f98fea00640 1 -- 172.21.15.67:0/650289628 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f990000a5b0 con 0x7f99080c70d0 2024-03-09T09:30:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f98fea00640 1 -- 172.21.15.67:0/650289628 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990000a9a0 con 0x7f99080c70d0 2024-03-09T09:30:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.147+0000 7f990ca00640 1 -- 172.21.15.67:0/650289628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99080c8070 con 0x7f99080c70d0 2024-03-09T09:30:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.148+0000 7f990ca00640 1 -- 172.21.15.67:0/650289628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99080b0740 con 0x7f99080c70d0 2024-03-09T09:30:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.148+0000 7f990ca00640 1 -- 172.21.15.67:0/650289628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990804f3c0 con 0x7f99080c70d0 2024-03-09T09:30:16.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.149+0000 7f98fea00640 1 -- 172.21.15.67:0/650289628 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9900011070 con 0x7f99080c70d0 2024-03-09T09:30:16.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.150+0000 7f98fea00640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98e806b4d0 0x7f98e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.150+0000 7f98fea00640 1 -- 172.21.15.67:0/650289628 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5775+0+0 (secure 0 0 0) 0x7f99000897b0 con 0x7f99080c70d0 2024-03-09T09:30:16.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.151+0000 7f9906800640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98e806b4d0 0x7f98e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.152+0000 7f9906800640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98e806b4d0 0x7f98e806d990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f98f8036000 tx=0x7f98f8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:16.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.155+0000 7f98fea00640 1 -- 172.21.15.67:0/650289628 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f99000584c0 con 0x7f99080c70d0 2024-03-09T09:30:16.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.386+0000 7f990ca00640 1 -- 172.21.15.67:0/650289628 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99081eb870 con 0x7f99080c70d0 2024-03-09T09:30:16.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.387+0000 7f98fea00640 1 -- 172.21.15.67:0/650289628 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4979 (secure 0 0 0) 0x7f9900057dc0 con 0x7f99080c70d0 2024-03-09T09:30:16.395 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:30:16.395 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":364309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1268474005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1268474005},{"type":"v1","addr":"172.21.15.134:6835","nonce":1268474005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T09:30:16.395 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 16 2024-03-09T09:30:16.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.389+0000 7f990ca00640 1 -- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98e806b4d0 msgr2=0x7f98e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:16.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.389+0000 7f990ca00640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98e806b4d0 0x7f98e806d990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f98f8036000 tx=0x7f98f8030000 comp rx=0 tx=0).stop 2024-03-09T09:30:16.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.389+0000 7f990ca00640 1 -- 172.21.15.67:0/650289628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99080c70d0 msgr2=0x7f990814c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:16.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.389+0000 7f990ca00640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99080c70d0 0x7f990814c060 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f99080adc60 tx=0x7f990000cb70 comp rx=0 tx=0).stop 2024-03-09T09:30:16.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.390+0000 7f990ca00640 1 -- 172.21.15.67:0/650289628 shutdown_connections 2024-03-09T09:30:16.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.390+0000 7f990ca00640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f98e806b4d0 0x7f98e806d990 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.390+0000 7f990ca00640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99080c70d0 0x7f990814c060 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.390+0000 7f990ca00640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99080c6c80 0x7f990814bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.390+0000 7f990ca00640 1 --2- 172.21.15.67:0/650289628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98ec003be0 0x7f99080b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.390+0000 7f990ca00640 1 -- 172.21.15.67:0/650289628 >> 172.21.15.67:0/650289628 conn(0x7f99080732d0 msgr2=0x7f9908152470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:16.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.390+0000 7f990ca00640 1 -- 172.21.15.67:0/650289628 shutdown_connections 2024-03-09T09:30:16.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.390+0000 7f990ca00640 1 -- 172.21.15.67:0/650289628 wait complete. 2024-03-09T09:30:16.423 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-09T09:30:16.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.659+0000 7f2882200640 1 Processor -- start 2024-03-09T09:30:16.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.659+0000 7f2882200640 1 -- start start 2024-03-09T09:30:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.661+0000 7f2882200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f287c0c1d70 0x7f287c0ac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.661+0000 7f2882200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f287c0c2400 0x7f287c0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.661+0000 7f2882200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f287c156d50 con 0x7f287c153370 2024-03-09T09:30:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.661+0000 7f2882200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c0aadf0 con 0x7f287c0c1d70 2024-03-09T09:30:16.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.661+0000 7f2882200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0aa4c0 con 0x7f287c0c2400 2024-03-09T09:30:16.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.661+0000 7f287aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f287c0c1d70 0x7f287c0ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.661+0000 7f287aa00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f287c0c1d70 0x7f287c0ac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38196/0 (socket says 172.21.15.67:38196) 2024-03-09T09:30:16.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.661+0000 7f287aa00640 1 -- 172.21.15.67:0/2260289970 learned_addr learned my addr 172.21.15.67:0/2260289970 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.662+0000 7f287be00640 1 --2- 172.21.15.67:0/2260289970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f287c0c2400 0x7f287c0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.662+0000 7f287aa00640 1 -- 172.21.15.67:0/2260289970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f287c0c2400 msgr2=0x7f287c0acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.662+0000 7f287aa00640 1 --2- 172.21.15.67:0/2260289970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f287c0c2400 0x7f287c0acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.662+0000 7f287aa00640 1 -- 172.21.15.67:0/2260289970 >> v1:172.21.15.134:6789/0 conn(0x7f287c153370 legacy=0x7f287c0c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.662+0000 7f287aa00640 1 -- 172.21.15.67:0/2260289970 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c0aa7a0 con 0x7f287c0c1d70 2024-03-09T09:30:16.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.664+0000 7f287aa00640 1 --2- 172.21.15.67:0/2260289970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f287c0c1d70 0x7f287c0ac610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2868009870 tx=0x7f286802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255445b9ea0abdb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:16.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.665+0000 7f287a000640 1 -- 172.21.15.67:0/2260289970 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680317c0 con 0x7f287c0c1d70 2024-03-09T09:30:16.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.665+0000 7f287a000640 1 -- 172.21.15.67:0/2260289970 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2868037070 con 0x7f287c0c1d70 2024-03-09T09:30:16.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.665+0000 7f287a000640 1 -- 172.21.15.67:0/2260289970 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868031ba0 con 0x7f287c0c1d70 2024-03-09T09:30:16.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.666+0000 7f2882200640 1 -- 172.21.15.67:0/2260289970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f287c0c1d70 msgr2=0x7f287c0ac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:16.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.666+0000 7f2882200640 1 --2- 172.21.15.67:0/2260289970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f287c0c1d70 0x7f287c0ac610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2868009870 tx=0x7f286802b860 comp rx=0 tx=0).stop 2024-03-09T09:30:16.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.666+0000 7f2882200640 1 -- 172.21.15.67:0/2260289970 shutdown_connections 2024-03-09T09:30:16.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.666+0000 7f2882200640 1 --2- 172.21.15.67:0/2260289970 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f287c0c2400 0x7f287c0acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.666+0000 7f2882200640 1 --2- 172.21.15.67:0/2260289970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f287c0c1d70 0x7f287c0ac610 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.666+0000 7f2882200640 1 -- 172.21.15.67:0/2260289970 >> 172.21.15.67:0/2260289970 conn(0x7f287c0c7370 msgr2=0x7f287c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:16.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.667+0000 7f2882200640 1 -- 172.21.15.67:0/2260289970 shutdown_connections 2024-03-09T09:30:16.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.667+0000 7f2882200640 1 -- 172.21.15.67:0/2260289970 wait complete. 2024-03-09T09:30:16.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.670+0000 7f2882200640 1 Processor -- start 2024-03-09T09:30:16.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.671+0000 7f2882200640 1 -- start start 2024-03-09T09:30:16.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.672+0000 7f2882200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f287c0c1d70 0x7f287c1550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.672+0000 7f2882200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f287c0c2400 0x7f287c0c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.672+0000 7f2882200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f287c153370 0x7f287c0c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.672+0000 7f287b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f287c0c1d70 0x7f287c1550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.672+0000 7f287b400640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f287c0c1d70 0x7f287c1550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38206/0 (socket says 172.21.15.67:38206) 2024-03-09T09:30:16.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f287b400640 1 -- 172.21.15.67:0/3130123811 learned_addr learned my addr 172.21.15.67:0/3130123811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:16.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0c1030 con 0x7f287c0c1d70 2024-03-09T09:30:16.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0c0eb0 con 0x7f287c153370 2024-03-09T09:30:16.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c153750 con 0x7f287c0c2400 2024-03-09T09:30:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f287aa00640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f287c0c2400 0x7f287c0c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f287be00640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f287c153370 0x7f287c0c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f287b400640 1 -- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f287c0c2400 msgr2=0x7f287c0c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f287b400640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f287c0c2400 0x7f287c0c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f287b400640 1 -- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f287c153370 msgr2=0x7f287c0c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f287b400640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f287c153370 0x7f287c0c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f287b400640 1 -- 172.21.15.67:0/3130123811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c071350 con 0x7f287c0c1d70 2024-03-09T09:30:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.673+0000 7f287be00640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f287c153370 0x7f287c0c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-09T09:30:16.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.674+0000 7f287b400640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f287c0c1d70 0x7f287c1550c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f286c00b420 tx=0x7f286c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:16.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.674+0000 7f2873400640 1 -- 172.21.15.67:0/3130123811 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c011830 con 0x7f287c0c1d70 2024-03-09T09:30:16.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.675+0000 7f2873400640 1 -- 172.21.15.67:0/3130123811 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f286c016070 con 0x7f287c0c1d70 2024-03-09T09:30:16.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.675+0000 7f2873400640 1 -- 172.21.15.67:0/3130123811 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c011c30 con 0x7f287c0c1d70 2024-03-09T09:30:16.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.675+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c0c5400 con 0x7f287c0c1d70 2024-03-09T09:30:16.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.675+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c155ba0 con 0x7f287c0c1d70 2024-03-09T09:30:16.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.680+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c075a00 con 0x7f287c0c1d70 2024-03-09T09:30:16.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.681+0000 7f2873400640 1 -- 172.21.15.67:0/3130123811 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f286c01b460 con 0x7f287c0c1d70 2024-03-09T09:30:16.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.682+0000 7f2873400640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f285c06b5d0 0x7f285c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:16.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.682+0000 7f2873400640 1 -- 172.21.15.67:0/3130123811 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5775+0+0 (secure 0 0 0) 0x7f286c089f40 con 0x7f287c0c1d70 2024-03-09T09:30:16.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.684+0000 7f287aa00640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f285c06b5d0 0x7f285c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:16.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.686+0000 7f287aa00640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f285c06b5d0 0x7f285c06da90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2868036000 tx=0x7f286803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:16.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:16.691+0000 7f2873400640 1 -- 172.21.15.67:0/3130123811 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f286c058d60 con 0x7f287c0c1d70 2024-03-09T09:30:17.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.079+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f287c0c0a40 con 0x7f287c0c1d70 2024-03-09T09:30:17.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.080+0000 7f2873400640 1 -- 172.21.15.67:0/3130123811 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4979 (secure 0 0 0) 0x7f286c058660 con 0x7f287c0c1d70 2024-03-09T09:30:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:30:17.088 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/1904109808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6833","nonce":1904109808},{"type":"v1","addr":"172.21.15.67:6835","nonce":1904109808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1894094648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":1894094648},{"type":"v1","addr":"172.21.15.134:6837","nonce":1894094648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":364309,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/1268474005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":1268474005},{"type":"v1","addr":"172.21.15.134:6835","nonce":1268474005}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-09T08:52:29.037519+0000","modified":"2024-03-09T08:52:33.151162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"c","rank":0,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.67:6834/1699514524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":1699514524},{"type":"v1","addr":"172.21.15.67:6834","nonce":1699514524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":2}]} 2024-03-09T09:30:17.089 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 16 2024-03-09T09:30:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.082+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f285c06b5d0 msgr2=0x7f285c06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:17.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.082+0000 7f2882200640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f285c06b5d0 0x7f285c06da90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2868036000 tx=0x7f286803d040 comp rx=0 tx=0).stop 2024-03-09T09:30:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.082+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f287c0c1d70 msgr2=0x7f287c1550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.082+0000 7f2882200640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f287c0c1d70 0x7f287c1550c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f286c00b420 tx=0x7f286c00b8e0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.082+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 shutdown_connections 2024-03-09T09:30:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.082+0000 7f2882200640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f285c06b5d0 0x7f285c06da90 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.082+0000 7f2882200640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f287c153370 0x7f287c0c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.082+0000 7f2882200640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f287c0c2400 0x7f287c0c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.082+0000 7f2882200640 1 --2- 172.21.15.67:0/3130123811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f287c0c1d70 0x7f287c1550c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.082+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 >> 172.21.15.67:0/3130123811 conn(0x7f287c0c7370 msgr2=0x7f287c156390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:17.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.082+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 shutdown_connections 2024-03-09T09:30:17.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.083+0000 7f2882200640 1 -- 172.21.15.67:0/3130123811 wait complete. 2024-03-09T09:30:17.107 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-09T09:30:17.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.299+0000 7f356e200640 1 Processor -- start 2024-03-09T09:30:17.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.300+0000 7f356e200640 1 -- start start 2024-03-09T09:30:17.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.301+0000 7f356e200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f35680c1f70 0x7f3568150c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.301+0000 7f356e200640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f35680710a0 0x7f3568151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.301+0000 7f356e200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35680c81e0 con 0x7f35680c27a0 2024-03-09T09:30:17.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.301+0000 7f356e200640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680c32b0 con 0x7f35680710a0 2024-03-09T09:30:17.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.301+0000 7f356e200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680c2fd0 con 0x7f35680c1f70 2024-03-09T09:30:17.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.301+0000 7f3567e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f35680710a0 0x7f3568151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.301+0000 7f3567e00640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f35680710a0 0x7f3568151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38216/0 (socket says 172.21.15.67:38216) 2024-03-09T09:30:17.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.301+0000 7f3567e00640 1 -- 172.21.15.67:0/626198685 learned_addr learned my addr 172.21.15.67:0/626198685 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:17.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.302+0000 7f356ce00640 1 --2- 172.21.15.67:0/626198685 >> v2:172.21.15.134:3300/0 conn(0x7f35680c1f70 0x7f3568150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.302+0000 7f3567e00640 1 -- 172.21.15.67:0/626198685 >> v2:172.21.15.134:3300/0 conn(0x7f35680c1f70 msgr2=0x7f3568150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:17.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.302+0000 7f3567e00640 1 --2- 172.21.15.67:0/626198685 >> v2:172.21.15.134:3300/0 conn(0x7f35680c1f70 0x7f3568150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.302+0000 7f3567e00640 1 -- 172.21.15.67:0/626198685 >> v1:172.21.15.134:6789/0 conn(0x7f35680c27a0 legacy=0x7f3568153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:17.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.302+0000 7f3567e00640 1 -- 172.21.15.67:0/626198685 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35680c3700 con 0x7f35680710a0 2024-03-09T09:30:17.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.304+0000 7f3567e00640 1 --2- 172.21.15.67:0/626198685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35680710a0 0x7f3568151190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f356806e890 tx=0x7f355402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0a33731f49d66 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:17.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.304+0000 7f3567400640 1 -- 172.21.15.67:0/626198685 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355402e070 con 0x7f35680710a0 2024-03-09T09:30:17.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.304+0000 7f3567400640 1 -- 172.21.15.67:0/626198685 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3554037070 con 0x7f35680710a0 2024-03-09T09:30:17.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.304+0000 7f3567400640 1 -- 172.21.15.67:0/626198685 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554031a20 con 0x7f35680710a0 2024-03-09T09:30:17.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.304+0000 7f356e200640 1 -- 172.21.15.67:0/626198685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35680710a0 msgr2=0x7f3568151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:17.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.304+0000 7f356e200640 1 --2- 172.21.15.67:0/626198685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35680710a0 0x7f3568151190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f356806e890 tx=0x7f355402b880 comp rx=0 tx=0).stop 2024-03-09T09:30:17.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.305+0000 7f356e200640 1 -- 172.21.15.67:0/626198685 shutdown_connections 2024-03-09T09:30:17.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.305+0000 7f356e200640 1 --2- 172.21.15.67:0/626198685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35680710a0 0x7f3568151190 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.305+0000 7f356e200640 1 --2- 172.21.15.67:0/626198685 >> v2:172.21.15.134:3300/0 conn(0x7f35680c1f70 0x7f3568150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.305+0000 7f356e200640 1 -- 172.21.15.67:0/626198685 >> 172.21.15.67:0/626198685 conn(0x7f35680c7300 msgr2=0x7f35680c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:17.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.305+0000 7f356e200640 1 -- 172.21.15.67:0/626198685 shutdown_connections 2024-03-09T09:30:17.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.305+0000 7f356e200640 1 -- 172.21.15.67:0/626198685 wait complete. 2024-03-09T09:30:17.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.306+0000 7f356e200640 1 Processor -- start 2024-03-09T09:30:17.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.306+0000 7f356e200640 1 -- start start 2024-03-09T09:30:17.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.307+0000 7f356e200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35680710a0 0x7f3568154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.307+0000 7f356e200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35680c1f70 0x7f35680c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.307+0000 7f356e200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35680c27a0 0x7f35680c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.307+0000 7f356e200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3568070d50 con 0x7f35680c27a0 2024-03-09T09:30:17.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.307+0000 7f356e200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3568070830 con 0x7f35680c1f70 2024-03-09T09:30:17.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.307+0000 7f356e200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3568070ed0 con 0x7f35680710a0 2024-03-09T09:30:17.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.308+0000 7f356d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35680c27a0 0x7f35680c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.308+0000 7f356d800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35680c27a0 0x7f35680c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38222/0 (socket says 172.21.15.67:38222) 2024-03-09T09:30:17.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.308+0000 7f356d800640 1 -- 172.21.15.67:0/1200564937 learned_addr learned my addr 172.21.15.67:0/1200564937 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:17.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.308+0000 7f356ce00640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35680710a0 0x7f3568154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.308+0000 7f3567e00640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35680c1f70 0x7f35680c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.308+0000 7f356d800640 1 -- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35680710a0 msgr2=0x7f3568154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:17.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.308+0000 7f356d800640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35680710a0 0x7f3568154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.308+0000 7f356d800640 1 -- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35680c1f70 msgr2=0x7f35680c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:17.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.308+0000 7f356d800640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35680c1f70 0x7f35680c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.308+0000 7f356d800640 1 -- 172.21.15.67:0/1200564937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35680c4210 con 0x7f35680c27a0 2024-03-09T09:30:17.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.308+0000 7f356d800640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35680c27a0 0x7f35680c59c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f356000b9a0 tx=0x7f356000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:17.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.309+0000 7f3564c00640 1 -- 172.21.15.67:0/1200564937 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3560019070 con 0x7f35680c27a0 2024-03-09T09:30:17.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.309+0000 7f3564c00640 1 -- 172.21.15.67:0/1200564937 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f356000cba0 con 0x7f35680c27a0 2024-03-09T09:30:17.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.309+0000 7f3564c00640 1 -- 172.21.15.67:0/1200564937 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3560014440 con 0x7f35680c27a0 2024-03-09T09:30:17.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.309+0000 7f356e200640 1 -- 172.21.15.67:0/1200564937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3568155110 con 0x7f35680c27a0 2024-03-09T09:30:17.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.309+0000 7f356e200640 1 -- 172.21.15.67:0/1200564937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35680c4760 con 0x7f35680c27a0 2024-03-09T09:30:17.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.310+0000 7f356e200640 1 -- 172.21.15.67:0/1200564937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35680679a0 con 0x7f35680c27a0 2024-03-09T09:30:17.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.312+0000 7f3564c00640 1 -- 172.21.15.67:0/1200564937 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f356001c070 con 0x7f35680c27a0 2024-03-09T09:30:17.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.313+0000 7f3564c00640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f354806b4d0 0x7f354806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.313+0000 7f3564c00640 1 -- 172.21.15.67:0/1200564937 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5775+0+0 (secure 0 0 0) 0x7f3560089ee0 con 0x7f35680c27a0 2024-03-09T09:30:17.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.314+0000 7f356ce00640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f354806b4d0 0x7f354806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.315+0000 7f356ce00640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f354806b4d0 0x7f354806d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3558006fd0 tx=0x7f3558008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:17.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.318+0000 7f3564c00640 1 -- 172.21.15.67:0/1200564937 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35680679a0 con 0x7f35680c27a0 2024-03-09T09:30:17.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.514+0000 7f356e200640 1 -- 172.21.15.67:0/1200564937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35680728f0 con 0x7f35680c27a0 2024-03-09T09:30:17.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.516+0000 7f3564c00640 1 -- 172.21.15.67:0/1200564937 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16766 (secure 0 0 0) 0x7f3560058c80 con 0x7f35680c27a0 2024-03-09T09:30:17.525 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-09T09:30:17.525 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":92,"fsid":"aded9f77-d43e-4509-848a-632eae6a080f","created":"2024-03-09T08:51:53.424683+0000","modified":"2024-03-09T09:30:15.845334+0000","last_up_change":"2024-03-09T08:52:05.892984+0000","last_in_change":"2024-03-09T08:51:56.924885+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-09T08:52:06.749641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-03-09T08:52:27.484297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-03-09T08:52:28.060754+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":765,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d3c74a26-9960-44e3-a2cf-eae01cd1f1c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6805","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6806","nonce":3109561126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6810","nonce":3109561126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6816","nonce":3109561126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6811","nonce":3109561126},{"type":"v1","addr":"172.21.15.67:6813","nonce":3109561126}]},"public_addr":"172.21.15.67:6806/3109561126","cluster_addr":"172.21.15.67:6810/3109561126","heartbeat_back_addr":"172.21.15.67:6816/3109561126","heartbeat_front_addr":"172.21.15.67:6813/3109561126","state":["exists","up"]},{"osd":1,"uuid":"749c5cf3-91d5-4bb9-bd35-e34f9e83ceac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6815","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6817","nonce":1773797031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6819","nonce":1773797031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6823","nonce":1773797031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1773797031},{"type":"v1","addr":"172.21.15.67:6821","nonce":1773797031}]},"public_addr":"172.21.15.67:6817/1773797031","cluster_addr":"172.21.15.67:6819/1773797031","heartbeat_back_addr":"172.21.15.67:6823/1773797031","heartbeat_front_addr":"172.21.15.67:6821/1773797031","state":["exists","up"]},{"osd":2,"uuid":"2c83ccf2-0bc6-4d05-be62-5d788c8c20ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6801","nonce":74956722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6803","nonce":74956722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6809","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6812","nonce":74956722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":74956722},{"type":"v1","addr":"172.21.15.67:6807","nonce":74956722}]},"public_addr":"172.21.15.67:6801/74956722","cluster_addr":"172.21.15.67:6803/74956722","heartbeat_back_addr":"172.21.15.67:6812/74956722","heartbeat_front_addr":"172.21.15.67:6807/74956722","state":["exists","up"]},{"osd":3,"uuid":"aab9dcc8-7016-4aa6-aeb2-e030df702b71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6825","nonce":1424647284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6827","nonce":1424647284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6831","nonce":1424647284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":1424647284},{"type":"v1","addr":"172.21.15.67:6829","nonce":1424647284}]},"public_addr":"172.21.15.67:6825/1424647284","cluster_addr":"172.21.15.67:6827/1424647284","heartbeat_back_addr":"172.21.15.67:6831/1424647284","heartbeat_front_addr":"172.21.15.67:6829/1424647284","state":["exists","up"]},{"osd":4,"uuid":"24203e41-1515-47c8-b596-01c471e0de42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6825","nonce":294033811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6827","nonce":294033811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6831","nonce":294033811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":294033811},{"type":"v1","addr":"172.21.15.134:6829","nonce":294033811}]},"public_addr":"172.21.15.134:6825/294033811","cluster_addr":"172.21.15.134:6827/294033811","heartbeat_back_addr":"172.21.15.134:6831/294033811","heartbeat_front_addr":"172.21.15.134:6829/294033811","state":["exists","up"]},{"osd":5,"uuid":"db5b5ea7-6a92-4d06-b2f7-dcece773d296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6801","nonce":4161897388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6803","nonce":4161897388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6807","nonce":4161897388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":4161897388},{"type":"v1","addr":"172.21.15.134:6805","nonce":4161897388}]},"public_addr":"172.21.15.134:6801/4161897388","cluster_addr":"172.21.15.134:6803/4161897388","heartbeat_back_addr":"172.21.15.134:6807/4161897388","heartbeat_front_addr":"172.21.15.134:6805/4161897388","state":["exists","up"]},{"osd":6,"uuid":"2af12c64-6476-4e86-a977-2420ad5a4679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6809","nonce":3579212498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6811","nonce":3579212498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6815","nonce":3579212498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3579212498},{"type":"v1","addr":"172.21.15.134:6813","nonce":3579212498}]},"public_addr":"172.21.15.134:6809/3579212498","cluster_addr":"172.21.15.134:6811/3579212498","heartbeat_back_addr":"172.21.15.134:6815/3579212498","heartbeat_front_addr":"172.21.15.134:6813/3579212498","state":["exists","up"]},{"osd":7,"uuid":"14fb0a99-8675-41f7-b02d-adfcfd95f841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6817","nonce":128562112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6819","nonce":128562112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6823","nonce":128562112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":128562112},{"type":"v1","addr":"172.21.15.134:6821","nonce":128562112}]},"public_addr":"172.21.15.134:6817/128562112","cluster_addr":"172.21.15.134:6819/128562112","heartbeat_back_addr":"172.21.15.134:6823/128562112","heartbeat_front_addr":"172.21.15.134:6821/128562112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.574743+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.286635+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.178717+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.276340+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.484239+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.469093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.202223+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-09T08:52:02.279933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.134:6835/340976284":"2024-03-10T09:30:12.780488+0000","172.21.15.134:6834/340976284":"2024-03-10T09:30:12.780488+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-09T09:30:17.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 -- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f354806b4d0 msgr2=0x7f354806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:17.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f354806b4d0 0x7f354806d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3558006fd0 tx=0x7f3558008040 comp rx=0 tx=0).stop 2024-03-09T09:30:17.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 -- 172.21.15.67:0/1200564937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35680c27a0 msgr2=0x7f35680c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:17.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35680c27a0 0x7f35680c59c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f356000b9a0 tx=0x7f356000be60 comp rx=0 tx=0).stop 2024-03-09T09:30:17.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 -- 172.21.15.67:0/1200564937 shutdown_connections 2024-03-09T09:30:17.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f354806b4d0 0x7f354806d990 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35680c27a0 0x7f35680c59c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35680c1f70 0x7f35680c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 --2- 172.21.15.67:0/1200564937 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f35680710a0 0x7f3568154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 -- 172.21.15.67:0/1200564937 >> 172.21.15.67:0/1200564937 conn(0x7f35680c7300 msgr2=0x7f356814b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:17.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 -- 172.21.15.67:0/1200564937 shutdown_connections 2024-03-09T09:30:17.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.519+0000 7f356e200640 1 -- 172.21.15.67:0/1200564937 wait complete. 2024-03-09T09:30:17.546 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail b 2024-03-09T09:30:17.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.744+0000 7f39bf000640 1 Processor -- start 2024-03-09T09:30:17.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.744+0000 7f39bf000640 1 -- start start 2024-03-09T09:30:17.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.746+0000 7f39bf000640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7f39b80c5e00 0x7f39b80acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.746+0000 7f39bf000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f39b80c9380 0x7f39b8156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.746+0000 7f39bf000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b80c97d0 con 0x7f39b80c61e0 2024-03-09T09:30:17.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.746+0000 7f39bf000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b80c6690 con 0x7f39b80c9380 2024-03-09T09:30:17.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.746+0000 7f39bf000640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b806d740 con 0x7f39b80c5e00 2024-03-09T09:30:17.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.746+0000 7f39be600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f39b80c9380 0x7f39b8156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.746+0000 7f39be600640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f39b80c9380 0x7f39b8156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38236/0 (socket says 172.21.15.67:38236) 2024-03-09T09:30:17.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.746+0000 7f39be600640 1 -- 172.21.15.67:0/106709665 learned_addr learned my addr 172.21.15.67:0/106709665 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:17.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.746+0000 7f39bdc00640 1 --2- 172.21.15.67:0/106709665 >> v2:172.21.15.134:3300/0 conn(0x7f39b80c5e00 0x7f39b80acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.747+0000 7f39be600640 1 -- 172.21.15.67:0/106709665 >> v2:172.21.15.134:3300/0 conn(0x7f39b80c5e00 msgr2=0x7f39b80acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:17.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.747+0000 7f39be600640 1 --2- 172.21.15.67:0/106709665 >> v2:172.21.15.134:3300/0 conn(0x7f39b80c5e00 0x7f39b80acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.747+0000 7f39be600640 1 -- 172.21.15.67:0/106709665 >> v1:172.21.15.67:6789/0 conn(0x7f39b80c61e0 legacy=0x7f39b806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:17.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.747+0000 7f39be600640 1 -- 172.21.15.67:0/106709665 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b80ad7c0 con 0x7f39b80c9380 2024-03-09T09:30:17.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.748+0000 7f39be600640 1 --2- 172.21.15.67:0/106709665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39b80c9380 0x7f39b8156660 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009e80 tx=0x7f39b002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cee2e8d3eae49eb server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:17.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.748+0000 7f39b7e00640 1 -- 172.21.15.67:0/106709665 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b002cb00 con 0x7f39b80c9380 2024-03-09T09:30:17.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.748+0000 7f39b7e00640 1 -- 172.21.15.67:0/106709665 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f39b002cca0 con 0x7f39b80c9380 2024-03-09T09:30:17.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.748+0000 7f39b7e00640 1 -- 172.21.15.67:0/106709665 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0034630 con 0x7f39b80c9380 2024-03-09T09:30:17.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.749+0000 7f39bf000640 1 -- 172.21.15.67:0/106709665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39b80c9380 msgr2=0x7f39b8156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:17.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.749+0000 7f39bf000640 1 --2- 172.21.15.67:0/106709665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39b80c9380 0x7f39b8156660 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009e80 tx=0x7f39b002bf60 comp rx=0 tx=0).stop 2024-03-09T09:30:17.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.749+0000 7f39bf000640 1 -- 172.21.15.67:0/106709665 shutdown_connections 2024-03-09T09:30:17.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.749+0000 7f39bf000640 1 --2- 172.21.15.67:0/106709665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39b80c9380 0x7f39b8156660 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.749+0000 7f39bf000640 1 --2- 172.21.15.67:0/106709665 >> v2:172.21.15.134:3300/0 conn(0x7f39b80c5e00 0x7f39b80acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.749+0000 7f39bf000640 1 -- 172.21.15.67:0/106709665 >> 172.21.15.67:0/106709665 conn(0x7f39b80cb480 msgr2=0x7f39b80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:17.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.749+0000 7f39bf000640 1 -- 172.21.15.67:0/106709665 shutdown_connections 2024-03-09T09:30:17.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.749+0000 7f39bf000640 1 -- 172.21.15.67:0/106709665 wait complete. 2024-03-09T09:30:17.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.750+0000 7f39bf000640 1 Processor -- start 2024-03-09T09:30:17.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.750+0000 7f39bf000640 1 -- start start 2024-03-09T09:30:17.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.751+0000 7f39bf000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39b80c5e00 0x7f39b80af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.751+0000 7f39bf000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39b80c61e0 0x7f39b814e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.751+0000 7f39bf000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b80c9380 0x7f39b814ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.751+0000 7f39bf000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b806bb00 con 0x7f39b80c5e00 2024-03-09T09:30:17.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.751+0000 7f39bf000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b8153680 con 0x7f39b80c61e0 2024-03-09T09:30:17.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.751+0000 7f39bf000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b806bde0 con 0x7f39b80c9380 2024-03-09T09:30:17.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.751+0000 7f39bdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39b80c5e00 0x7f39b80af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.752+0000 7f39bdc00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39b80c5e00 0x7f39b80af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38252/0 (socket says 172.21.15.67:38252) 2024-03-09T09:30:17.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.752+0000 7f39bdc00640 1 -- 172.21.15.67:0/2679191668 learned_addr learned my addr 172.21.15.67:0/2679191668 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:17.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.752+0000 7f39bd200640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39b80c61e0 0x7f39b814e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.752+0000 7f39be600640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b80c9380 0x7f39b814ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.752+0000 7f39bdc00640 1 -- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b80c9380 msgr2=0x7f39b814ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:17.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.752+0000 7f39bdc00640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b80c9380 0x7f39b814ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.752+0000 7f39bdc00640 1 -- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39b80c61e0 msgr2=0x7f39b814e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:17.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.752+0000 7f39bdc00640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39b80c61e0 0x7f39b814e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:17.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.752+0000 7f39bdc00640 1 -- 172.21.15.67:0/2679191668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b80c5830 con 0x7f39b80c5e00 2024-03-09T09:30:17.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.752+0000 7f39bdc00640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39b80c5e00 0x7f39b80af360 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00d800 tx=0x7f39ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:17.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.753+0000 7f39b5600640 1 -- 172.21.15.67:0/2679191668 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00b6c0 con 0x7f39b80c5e00 2024-03-09T09:30:17.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.753+0000 7f39b5600640 1 -- 172.21.15.67:0/2679191668 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f39ac019070 con 0x7f39b80c5e00 2024-03-09T09:30:17.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.753+0000 7f39b5600640 1 -- 172.21.15.67:0/2679191668 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00baa0 con 0x7f39b80c5e00 2024-03-09T09:30:17.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.753+0000 7f39bf000640 1 -- 172.21.15.67:0/2679191668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b80c5270 con 0x7f39b80c5e00 2024-03-09T09:30:17.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.753+0000 7f39bf000640 1 -- 172.21.15.67:0/2679191668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b80c0830 con 0x7f39b80c5e00 2024-03-09T09:30:17.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.754+0000 7f39bf000640 1 -- 172.21.15.67:0/2679191668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b80ad260 con 0x7f39b80c5e00 2024-03-09T09:30:17.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.755+0000 7f39b5600640 1 -- 172.21.15.67:0/2679191668 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f39ac00bc40 con 0x7f39b80c5e00 2024-03-09T09:30:17.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.756+0000 7f39b5600640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f399806b4d0 0x7f399806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:17.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.756+0000 7f39b5600640 1 -- 172.21.15.67:0/2679191668 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5775+0+0 (secure 0 0 0) 0x7f39ac08a740 con 0x7f39b80c5e00 2024-03-09T09:30:17.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.756+0000 7f39bd200640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f399806b4d0 0x7f399806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:17.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.757+0000 7f39bd200640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f399806b4d0 0x7f399806d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f39b80be8f0 tx=0x7f39a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:17.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:17.760+0000 7f39b5600640 1 -- 172.21.15.67:0/2679191668 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f39b80ad260 con 0x7f39b80c5e00 2024-03-09T09:30:18.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.031+0000 7f39bf000640 1 -- 172.21.15.67:0/2679191668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "b"} v 0) v1 -- 0x7f39b804f3c0 con 0x7f39b80c5e00 2024-03-09T09:30:18.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.035+0000 7f39b5600640 1 -- 172.21.15.67:0/2679191668 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "b"}]=0 b marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 151+0+0 (secure 0 0 0) 0x7f39ac059580 con 0x7f39b80c5e00 2024-03-09T09:30:18.043 INFO:teuthology.orchestra.run.smithi067.stderr:b marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-09T09:30:18.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 -- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f399806b4d0 msgr2=0x7f399806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:18.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f399806b4d0 0x7f399806d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f39b80be8f0 tx=0x7f39a4009040 comp rx=0 tx=0).stop 2024-03-09T09:30:18.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 -- 172.21.15.67:0/2679191668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39b80c5e00 msgr2=0x7f39b80af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:18.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39b80c5e00 0x7f39b80af360 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00d800 tx=0x7f39ac00dcc0 comp rx=0 tx=0).stop 2024-03-09T09:30:18.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 -- 172.21.15.67:0/2679191668 shutdown_connections 2024-03-09T09:30:18.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f399806b4d0 0x7f399806d990 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39b80c9380 0x7f39b814ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39b80c61e0 0x7f39b814e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 --2- 172.21.15.67:0/2679191668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f39b80c5e00 0x7f39b80af360 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 -- 172.21.15.67:0/2679191668 >> 172.21.15.67:0/2679191668 conn(0x7f39b80cb480 msgr2=0x7f39b814f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 -- 172.21.15.67:0/2679191668 shutdown_connections 2024-03-09T09:30:18.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.038+0000 7f39bf000640 1 -- 172.21.15.67:0/2679191668 wait complete. 2024-03-09T09:30:18.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm b --yes-i-really-mean-it 2024-03-09T09:30:18.185 INFO:tasks.ceph.mds.c.smithi067.stdout:starting mds.c at 2024-03-09T09:30:18.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.250+0000 7f78ab000640 1 Processor -- start 2024-03-09T09:30:18.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.250+0000 7f78ab000640 1 -- start start 2024-03-09T09:30:18.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.251+0000 7f78ab000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a40c6240 0x7f78a414f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:18.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.251+0000 7f78ab000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f78a414fa80 0x7f78a40c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:18.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.251+0000 7f78ab000640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a40aa3b0 con 0x7f78a40ab940 2024-03-09T09:30:18.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.251+0000 7f78ab000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a40c6f00 con 0x7f78a414fa80 2024-03-09T09:30:18.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.251+0000 7f78ab000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40c50b0 con 0x7f78a40c6240 2024-03-09T09:30:18.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.252+0000 7f78a9000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f78a414fa80 0x7f78a40c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:18.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.252+0000 7f78a9000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f78a414fa80 0x7f78a40c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38262/0 (socket says 172.21.15.67:38262) 2024-03-09T09:30:18.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.252+0000 7f78a9000640 1 -- 172.21.15.67:0/4128822863 learned_addr learned my addr 172.21.15.67:0/4128822863 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:18.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.252+0000 7f789be00640 1 --2- 172.21.15.67:0/4128822863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a40c6240 0x7f78a414f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:18.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.252+0000 7f78a9000640 1 -- 172.21.15.67:0/4128822863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a40c6240 msgr2=0x7f78a414f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:18.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.252+0000 7f78a9000640 1 --2- 172.21.15.67:0/4128822863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a40c6240 0x7f78a414f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:18.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.252+0000 7f78a9000640 1 -- 172.21.15.67:0/4128822863 >> v1:172.21.15.134:6789/0 conn(0x7f78a40ab940 legacy=0x7f78a41561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:18.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.252+0000 7f78a9000640 1 -- 172.21.15.67:0/4128822863 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a406d5e0 con 0x7f78a414fa80 2024-03-09T09:30:18.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.253+0000 7f78a9000640 1 --2- 172.21.15.67:0/4128822863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a414fa80 0x7f78a40c5540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f789c009e50 tx=0x7f789c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce3891c48f09d9e server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:18.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.254+0000 7f78a3400640 1 -- 172.21.15.67:0/4128822863 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c02ca70 con 0x7f78a414fa80 2024-03-09T09:30:18.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.254+0000 7f78a3400640 1 -- 172.21.15.67:0/4128822863 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f789c02cc10 con 0x7f78a414fa80 2024-03-09T09:30:18.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.254+0000 7f78a3400640 1 -- 172.21.15.67:0/4128822863 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c0343f0 con 0x7f78a414fa80 2024-03-09T09:30:18.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.254+0000 7f78ab000640 1 -- 172.21.15.67:0/4128822863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a414fa80 msgr2=0x7f78a40c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:18.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.254+0000 7f78ab000640 1 --2- 172.21.15.67:0/4128822863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a414fa80 0x7f78a40c5540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f789c009e50 tx=0x7f789c02bed0 comp rx=0 tx=0).stop 2024-03-09T09:30:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.254+0000 7f78ab000640 1 -- 172.21.15.67:0/4128822863 shutdown_connections 2024-03-09T09:30:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.254+0000 7f78ab000640 1 --2- 172.21.15.67:0/4128822863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a414fa80 0x7f78a40c5540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.254+0000 7f78ab000640 1 --2- 172.21.15.67:0/4128822863 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a40c6240 0x7f78a414f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.254+0000 7f78ab000640 1 -- 172.21.15.67:0/4128822863 >> 172.21.15.67:0/4128822863 conn(0x7f78a406b040 msgr2=0x7f78a406b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.255+0000 7f78ab000640 1 -- 172.21.15.67:0/4128822863 shutdown_connections 2024-03-09T09:30:18.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.255+0000 7f78ab000640 1 -- 172.21.15.67:0/4128822863 wait complete. 2024-03-09T09:30:18.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.256+0000 7f78ab000640 1 Processor -- start 2024-03-09T09:30:18.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.256+0000 7f78ab000640 1 -- start start 2024-03-09T09:30:18.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.257+0000 7f78ab000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a40ab940 0x7f78a4142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:18.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.257+0000 7f78ab000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a40c6240 0x7f78a413d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:18.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.257+0000 7f78ab000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a414fa80 0x7f78a413dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:18.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.257+0000 7f78ab000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a406de80 con 0x7f78a414fa80 2024-03-09T09:30:18.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.257+0000 7f78ab000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a406dba0 con 0x7f78a40ab940 2024-03-09T09:30:18.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.257+0000 7f78ab000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40abd20 con 0x7f78a40c6240 2024-03-09T09:30:18.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.257+0000 7f78a9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a414fa80 0x7f78a413dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:18.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.257+0000 7f78a9000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a414fa80 0x7f78a413dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38270/0 (socket says 172.21.15.67:38270) 2024-03-09T09:30:18.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.257+0000 7f78a9000640 1 -- 172.21.15.67:0/2945651762 learned_addr learned my addr 172.21.15.67:0/2945651762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:18.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.258+0000 7f78a3e00640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a40ab940 0x7f78a4142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:18.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.258+0000 7f789be00640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a40c6240 0x7f78a413d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:18.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.258+0000 7f78a9000640 1 -- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a40c6240 msgr2=0x7f78a413d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:18.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.258+0000 7f78a9000640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a40c6240 0x7f78a413d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:18.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.258+0000 7f78a9000640 1 -- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a40ab940 msgr2=0x7f78a4142e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:18.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.258+0000 7f78a9000640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a40ab940 0x7f78a4142e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:18.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.258+0000 7f78a9000640 1 -- 172.21.15.67:0/2945651762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a4071440 con 0x7f78a414fa80 2024-03-09T09:30:18.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.258+0000 7f78a9000640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a414fa80 0x7f78a413dc20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f789c009e50 tx=0x7f789c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:18.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.259+0000 7f78a0c00640 1 -- 172.21.15.67:0/2945651762 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c043020 con 0x7f78a414fa80 2024-03-09T09:30:18.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.259+0000 7f78a0c00640 1 -- 172.21.15.67:0/2945651762 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f789c03e450 con 0x7f78a414fa80 2024-03-09T09:30:18.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.259+0000 7f78a0c00640 1 -- 172.21.15.67:0/2945651762 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c009510 con 0x7f78a414fa80 2024-03-09T09:30:18.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.259+0000 7f78ab000640 1 -- 172.21.15.67:0/2945651762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a4071720 con 0x7f78a414fa80 2024-03-09T09:30:18.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.259+0000 7f78ab000640 1 -- 172.21.15.67:0/2945651762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a40a9d30 con 0x7f78a414fa80 2024-03-09T09:30:18.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.260+0000 7f78ab000640 1 -- 172.21.15.67:0/2945651762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a40c7920 con 0x7f78a414fa80 2024-03-09T09:30:18.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.261+0000 7f78a0c00640 1 -- 172.21.15.67:0/2945651762 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f789c005d10 con 0x7f78a414fa80 2024-03-09T09:30:18.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.262+0000 7f78a0c00640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788006b500 0x7f788006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:18.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.262+0000 7f78a0c00640 1 -- 172.21.15.67:0/2945651762 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5861+0+0 (secure 0 0 0) 0x7f789c0ace10 con 0x7f78a414fa80 2024-03-09T09:30:18.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.262+0000 7f78a3e00640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788006b500 0x7f788006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:18.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.263+0000 7f78a3e00640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788006b500 0x7f788006d9c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7894001640 tx=0x7f7894009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:18.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.266+0000 7f78a0c00640 1 -- 172.21.15.67:0/2945651762 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f78a40c7920 con 0x7f78a414fa80 2024-03-09T09:30:18.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:18.510+0000 7f78ab000640 1 -- 172.21.15.67:0/2945651762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "b", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f78a40c4d80 con 0x7f78a414fa80 2024-03-09T09:30:19.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.038+0000 7f78a0c00640 1 -- 172.21.15.67:0/2945651762 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "b", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 99+0+0 (secure 0 0 0) 0x7f78a40c4d80 con 0x7f78a414fa80 2024-03-09T09:30:19.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.042+0000 7f78ab000640 1 -- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788006b500 msgr2=0x7f788006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:19.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.042+0000 7f78ab000640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788006b500 0x7f788006d9c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7894001640 tx=0x7f7894009040 comp rx=0 tx=0).stop 2024-03-09T09:30:19.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.042+0000 7f78ab000640 1 -- 172.21.15.67:0/2945651762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a414fa80 msgr2=0x7f78a413dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:19.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.042+0000 7f78ab000640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a414fa80 0x7f78a413dc20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f789c009e50 tx=0x7f789c004a40 comp rx=0 tx=0).stop 2024-03-09T09:30:19.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.042+0000 7f78ab000640 1 -- 172.21.15.67:0/2945651762 shutdown_connections 2024-03-09T09:30:19.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.042+0000 7f78ab000640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f788006b500 0x7f788006d9c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:19.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.042+0000 7f78ab000640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78a414fa80 0x7f78a413dc20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.042+0000 7f78ab000640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78a40c6240 0x7f78a413d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.042+0000 7f78ab000640 1 --2- 172.21.15.67:0/2945651762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78a40ab940 0x7f78a4142e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.043+0000 7f78ab000640 1 -- 172.21.15.67:0/2945651762 >> 172.21.15.67:0/2945651762 conn(0x7f78a406b040 msgr2=0x7f78a40aa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.043+0000 7f78ab000640 1 -- 172.21.15.67:0/2945651762 shutdown_connections 2024-03-09T09:30:19.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.043+0000 7f78ab000640 1 -- 172.21.15.67:0/2945651762 wait complete. 2024-03-09T09:30:19.072 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm b_metadata b_metadata --yes-i-really-really-mean-it 2024-03-09T09:30:19.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.281+0000 7f914d000640 1 Processor -- start 2024-03-09T09:30:19.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.282+0000 7f914d000640 1 -- start start 2024-03-09T09:30:19.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.283+0000 7f914d000640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9148153410 0x7f91480c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:19.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.283+0000 7f914d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91480c95d0 0x7f914806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:19.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.283+0000 7f914d000640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914806ce30 con 0x7f91480bec10 2024-03-09T09:30:19.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.283+0000 7f914d000640 1 -- --> v2:172.21.15.67:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9148156b00 con 0x7f9148153410 2024-03-09T09:30:19.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.283+0000 7f914d000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9148155c30 con 0x7f91480c95d0 2024-03-09T09:30:19.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.283+0000 7f9147400640 1 --2- >> v2:172.21.15.67:3300/0 conn(0x7f9148153410 0x7f91480c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:19.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.283+0000 7f9146a00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7f91480bec10 0x7f914806b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50818/0 (socket says 172.21.15.67:50818) 2024-03-09T09:30:19.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.283+0000 7f9146a00640 1 -- 172.21.15.67:0/2176439983 learned_addr learned my addr 172.21.15.67:0/2176439983 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:30:19.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.284+0000 7f9147e00640 1 --2- 172.21.15.67:0/2176439983 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91480c95d0 0x7f914806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:19.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.284+0000 7f9147400640 1 -- 172.21.15.67:0/2176439983 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91480c95d0 msgr2=0x7f914806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:19.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.284+0000 7f9147400640 1 --2- 172.21.15.67:0/2176439983 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91480c95d0 0x7f914806b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:19.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.284+0000 7f9147400640 1 -- 172.21.15.67:0/2176439983 >> v1:172.21.15.67:6789/0 conn(0x7f91480bec10 legacy=0x7f914806b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:19.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.284+0000 7f9147400640 1 -- 172.21.15.67:0/2176439983 --> v2:172.21.15.67:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9148155ef0 con 0x7f9148153410 2024-03-09T09:30:19.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.284+0000 7f9146000640 1 -- 172.21.15.67:0/2176439983 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983915432 0 0) 0x7f914806ce30 con 0x7f91480bec10 2024-03-09T09:30:19.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.286+0000 7f9147400640 1 --2- 172.21.15.67:0/2176439983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9148153410 0x7f91480c7f90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f91300097a0 tx=0x7f913002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cf84249c9ca437 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:19.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.286+0000 7f9146000640 1 -- 172.21.15.67:0/2176439983 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130031780 con 0x7f9148153410 2024-03-09T09:30:19.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.286+0000 7f9146000640 1 -- 172.21.15.67:0/2176439983 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9130037070 con 0x7f9148153410 2024-03-09T09:30:19.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.286+0000 7f9146000640 1 -- 172.21.15.67:0/2176439983 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130031b60 con 0x7f9148153410 2024-03-09T09:30:19.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.287+0000 7f914d000640 1 -- 172.21.15.67:0/2176439983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9148153410 msgr2=0x7f91480c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:19.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.287+0000 7f914d000640 1 --2- 172.21.15.67:0/2176439983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9148153410 0x7f91480c7f90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f91300097a0 tx=0x7f913002b820 comp rx=0 tx=0).stop 2024-03-09T09:30:19.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.287+0000 7f914d000640 1 -- 172.21.15.67:0/2176439983 shutdown_connections 2024-03-09T09:30:19.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.287+0000 7f914d000640 1 --2- 172.21.15.67:0/2176439983 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91480c95d0 0x7f914806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:19.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.287+0000 7f914d000640 1 --2- 172.21.15.67:0/2176439983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9148153410 0x7f91480c7f90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:19.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.287+0000 7f914d000640 1 -- 172.21.15.67:0/2176439983 >> 172.21.15.67:0/2176439983 conn(0x7f91480ad0d0 msgr2=0x7f91480ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:19.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.287+0000 7f914d000640 1 -- 172.21.15.67:0/2176439983 shutdown_connections 2024-03-09T09:30:19.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.287+0000 7f914d000640 1 -- 172.21.15.67:0/2176439983 wait complete. 2024-03-09T09:30:19.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.289+0000 7f914d000640 1 Processor -- start 2024-03-09T09:30:19.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.289+0000 7f914d000640 1 -- start start 2024-03-09T09:30:19.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f914d000640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91480bec10 0x7f9148148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:19.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f914d000640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91480c95d0 0x7f91480abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:19.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f914d000640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9148153410 0x7f91480ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:19.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f914d000640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9148156790 con 0x7f9148153410 2024-03-09T09:30:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f914d000640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91481564b0 con 0x7f91480bec10 2024-03-09T09:30:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f914d000640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914806be40 con 0x7f91480c95d0 2024-03-09T09:30:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f9147e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9148153410 0x7f91480ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:19.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f9147e00640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9148153410 0x7f91480ac4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49700/0 (socket says 172.21.15.67:49700) 2024-03-09T09:30:19.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f9147e00640 1 -- 172.21.15.67:0/2751375583 learned_addr learned my addr 172.21.15.67:0/2751375583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:19.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f9147400640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91480bec10 0x7f9148148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:19.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f9146a00640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91480c95d0 0x7f91480abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:19.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f9147e00640 1 -- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91480c95d0 msgr2=0x7f91480abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:19.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f9147e00640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91480c95d0 0x7f91480abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:19.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f9147e00640 1 -- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91480bec10 msgr2=0x7f9148148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:19.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f9147e00640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91480bec10 0x7f9148148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:19.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.290+0000 7f9147e00640 1 -- 172.21.15.67:0/2751375583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914814e260 con 0x7f9148153410 2024-03-09T09:30:19.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.291+0000 7f9147e00640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9148153410 0x7f91480ac4d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f914000c820 tx=0x7f914000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:19.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.291+0000 7f913f400640 1 -- 172.21.15.67:0/2751375583 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140015070 con 0x7f9148153410 2024-03-09T09:30:19.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.291+0000 7f913f400640 1 -- 172.21.15.67:0/2751375583 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9140004020 con 0x7f9148153410 2024-03-09T09:30:19.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.291+0000 7f913f400640 1 -- 172.21.15.67:0/2751375583 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140004430 con 0x7f9148153410 2024-03-09T09:30:19.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.291+0000 7f914d000640 1 -- 172.21.15.67:0/2751375583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914814e540 con 0x7f9148153410 2024-03-09T09:30:19.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.291+0000 7f914d000640 1 -- 172.21.15.67:0/2751375583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91480aca10 con 0x7f9148153410 2024-03-09T09:30:19.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.292+0000 7f914d000640 1 -- 172.21.15.67:0/2751375583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914804f430 con 0x7f9148153410 2024-03-09T09:30:19.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.293+0000 7f913f400640 1 -- 172.21.15.67:0/2751375583 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9140007830 con 0x7f9148153410 2024-03-09T09:30:19.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.294+0000 7f913f400640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f912406b4d0 0x7f912406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:19.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.294+0000 7f913f400640 1 -- 172.21.15.67:0/2751375583 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5861+0+0 (secure 0 0 0) 0x7f914008a170 con 0x7f9148153410 2024-03-09T09:30:19.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.295+0000 7f9147400640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f912406b4d0 0x7f912406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:19.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.295+0000 7f9147400640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f912406b4d0 0x7f912406d990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9130036000 tx=0x7f9130008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:19.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.298+0000 7f913f400640 1 -- 172.21.15.67:0/2751375583 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9140058f50 con 0x7f9148153410 2024-03-09T09:30:19.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:19.498+0000 7f914d000640 1 -- 172.21.15.67:0/2751375583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "b_metadata", "pool2": "b_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f914814ff10 con 0x7f9148153410 2024-03-09T09:30:20.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.050+0000 7f913f400640 1 -- 172.21.15.67:0/2751375583 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "b_metadata", "pool2": "b_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'b_metadata' removed v94) v1 ==== 166+0+0 (secure 0 0 0) 0x7f9140058850 con 0x7f9148153410 2024-03-09T09:30:20.058 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'b_metadata' removed 2024-03-09T09:30:20.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.054+0000 7f914d000640 1 -- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f912406b4d0 msgr2=0x7f912406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:20.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.054+0000 7f914d000640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f912406b4d0 0x7f912406d990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9130036000 tx=0x7f9130008030 comp rx=0 tx=0).stop 2024-03-09T09:30:20.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.054+0000 7f914d000640 1 -- 172.21.15.67:0/2751375583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9148153410 msgr2=0x7f91480ac4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:20.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.054+0000 7f914d000640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9148153410 0x7f91480ac4d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f914000c820 tx=0x7f914000cce0 comp rx=0 tx=0).stop 2024-03-09T09:30:20.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.054+0000 7f914d000640 1 -- 172.21.15.67:0/2751375583 shutdown_connections 2024-03-09T09:30:20.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.054+0000 7f914d000640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7f912406b4d0 0x7f912406d990 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:20.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.054+0000 7f914d000640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9148153410 0x7f91480ac4d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:20.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.054+0000 7f914d000640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91480c95d0 0x7f91480abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:20.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.054+0000 7f914d000640 1 --2- 172.21.15.67:0/2751375583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91480bec10 0x7f9148148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:20.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.054+0000 7f914d000640 1 -- 172.21.15.67:0/2751375583 >> 172.21.15.67:0/2751375583 conn(0x7f91480ad0d0 msgr2=0x7f91480c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:20.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.056+0000 7f914d000640 1 -- 172.21.15.67:0/2751375583 shutdown_connections 2024-03-09T09:30:20.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.057+0000 7f914d000640 1 -- 172.21.15.67:0/2751375583 wait complete. 2024-03-09T09:30:20.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm b_data b_data --yes-i-really-really-mean-it 2024-03-09T09:30:20.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.299+0000 7faf6a200640 1 Processor -- start 2024-03-09T09:30:20.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.299+0000 7faf6a200640 1 -- start start 2024-03-09T09:30:20.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.300+0000 7faf6a200640 1 --2- >> v2:172.21.15.134:3300/0 conn(0x7faf64153410 0x7faf640c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:20.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.300+0000 7faf6a200640 1 -- --> v1:172.21.15.67:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6406ce30 con 0x7faf640bec10 2024-03-09T09:30:20.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.300+0000 7faf6a200640 1 -- --> v1:172.21.15.134:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf64155c30 con 0x7faf640c95d0 2024-03-09T09:30:20.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.300+0000 7faf6a200640 1 -- --> v2:172.21.15.134:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf64156b00 con 0x7faf64153410 2024-03-09T09:30:20.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.300+0000 7faf63e00640 1 --1- >> v1:172.21.15.67:6789/0 conn(0x7faf640bec10 0x7faf6406b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.67:6789/0 says I am v1:172.21.15.67:50828/0 (socket says 172.21.15.67:50828) 2024-03-09T09:30:20.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.300+0000 7faf63e00640 1 -- 172.21.15.67:0/1438240036 learned_addr learned my addr 172.21.15.67:0/1438240036 (peer_addr_for_me v1:172.21.15.67:0/0) 2024-03-09T09:30:20.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf68e00640 1 --2- 172.21.15.67:0/1438240036 >> v2:172.21.15.134:3300/0 conn(0x7faf64153410 0x7faf640c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:20.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 <== mon.0 v1:172.21.15.67:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507078 0 0) 0x7faf6406ce30 con 0x7faf640bec10 2024-03-09T09:30:20.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 --> v1:172.21.15.67:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf48003520 con 0x7faf640bec10 2024-03-09T09:30:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 <== mon.0 v1:172.21.15.67:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2308400911 0 0) 0x7faf48003520 con 0x7faf640bec10 2024-03-09T09:30:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 >> v2:172.21.15.134:3300/0 conn(0x7faf64153410 msgr2=0x7faf640c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 --2- 172.21.15.67:0/1438240036 >> v2:172.21.15.134:3300/0 conn(0x7faf64153410 0x7faf640c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 >> v1:172.21.15.134:6789/0 conn(0x7faf640c95d0 legacy=0x7faf6406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 --> v1:172.21.15.67:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf64155ef0 con 0x7faf640bec10 2024-03-09T09:30:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 <== mon.0 v1:172.21.15.67:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3256363255 0 0) 0x7faf50001670 con 0x7faf640bec10 2024-03-09T09:30:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 >> v1:172.21.15.67:6789/0 conn(0x7faf640bec10 legacy=0x7faf6406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:20.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 --2- 172.21.15.67:0/1438240036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf480039b0 0x7faf48003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 --2- 172.21.15.67:0/1438240036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf48004800 0x7faf48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.301+0000 7faf63400640 1 --2- 172.21.15.67:0/1438240036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf48005210 0x7faf4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.302+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf64156b00 con 0x7faf48005210 2024-03-09T09:30:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.302+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4800c160 con 0x7faf480039b0 2024-03-09T09:30:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.302+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4800c360 con 0x7faf48004800 2024-03-09T09:30:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.302+0000 7faf69800640 1 --2- 172.21.15.67:0/1438240036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf48005210 0x7faf4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:20.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.302+0000 7faf68e00640 1 --2- 172.21.15.67:0/1438240036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf480039b0 0x7faf48003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.302+0000 7faf63e00640 1 --2- 172.21.15.67:0/1438240036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf48004800 0x7faf48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.302+0000 7faf69800640 1 -- 172.21.15.67:0/1438240036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf48004800 msgr2=0x7faf48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.302+0000 7faf69800640 1 --2- 172.21.15.67:0/1438240036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf48004800 0x7faf48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.302+0000 7faf69800640 1 -- 172.21.15.67:0/1438240036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf480039b0 msgr2=0x7faf48003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-09T09:30:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.302+0000 7faf69800640 1 --2- 172.21.15.67:0/1438240036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf480039b0 0x7faf48003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.302+0000 7faf69800640 1 -- 172.21.15.67:0/1438240036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf48003520 con 0x7faf48005210 2024-03-09T09:30:20.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.303+0000 7faf69800640 1 -- 172.21.15.67:0/1438240036 reap_dead start 2024-03-09T09:30:20.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.303+0000 7faf69800640 1 --2- 172.21.15.67:0/1438240036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf48005210 0x7faf4800b9c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf5c001f60 tx=0x7faf48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408b18c155e13397 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:20.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.304+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c001b30 con 0x7faf48005210 2024-03-09T09:30:20.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.304+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7faf5c003b90 con 0x7faf48005210 2024-03-09T09:30:20.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.304+0000 7faf63400640 1 -- 172.21.15.67:0/1438240036 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c005070 con 0x7faf48005210 2024-03-09T09:30:20.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.304+0000 7faf6a200640 1 -- 172.21.15.67:0/1438240036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf48005210 msgr2=0x7faf4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:20.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.304+0000 7faf6a200640 1 --2- 172.21.15.67:0/1438240036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf48005210 0x7faf4800b9c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faf5c001f60 tx=0x7faf48005830 comp rx=0 tx=0).stop 2024-03-09T09:30:20.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.304+0000 7faf6a200640 1 -- 172.21.15.67:0/1438240036 shutdown_connections 2024-03-09T09:30:20.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.304+0000 7faf6a200640 1 --2- 172.21.15.67:0/1438240036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf48005210 0x7faf4800b9c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:20.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.304+0000 7faf6a200640 1 -- 172.21.15.67:0/1438240036 >> 172.21.15.67:0/1438240036 conn(0x7faf640ad0d0 msgr2=0x7faf640ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:20.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.304+0000 7faf6a200640 1 -- 172.21.15.67:0/1438240036 shutdown_connections 2024-03-09T09:30:20.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.305+0000 7faf6a200640 1 -- 172.21.15.67:0/1438240036 wait complete. 2024-03-09T09:30:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.306+0000 7faf6a200640 1 Processor -- start 2024-03-09T09:30:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.306+0000 7faf6a200640 1 -- start start 2024-03-09T09:30:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.307+0000 7faf6a200640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf48005210 0x7faf640af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.307+0000 7faf6a200640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf641ebb90 0x7faf640ac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.307+0000 7faf6a200640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf640ab8e0 0x7faf64147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.307+0000 7faf6a200640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf64156790 con 0x7faf640ab8e0 2024-03-09T09:30:20.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.307+0000 7faf6a200640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf641564b0 con 0x7faf641ebb90 2024-03-09T09:30:20.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.307+0000 7faf6a200640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf640c7f90 con 0x7faf48005210 2024-03-09T09:30:20.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.307+0000 7faf69800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf640ab8e0 0x7faf64147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:20.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.308+0000 7faf69800640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf640ab8e0 0x7faf64147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49730/0 (socket says 172.21.15.67:49730) 2024-03-09T09:30:20.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.308+0000 7faf69800640 1 -- 172.21.15.67:0/37687443 learned_addr learned my addr 172.21.15.67:0/37687443 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-09T09:30:20.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.308+0000 7faf68e00640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf48005210 0x7faf640af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:20.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.308+0000 7faf63e00640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf641ebb90 0x7faf640ac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:20.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.308+0000 7faf69800640 1 -- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf48005210 msgr2=0x7faf640af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:20.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.308+0000 7faf69800640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf48005210 0x7faf640af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:20.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.308+0000 7faf69800640 1 -- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf641ebb90 msgr2=0x7faf640ac5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:20.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.308+0000 7faf69800640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf641ebb90 0x7faf640ac5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:20.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.308+0000 7faf69800640 1 -- 172.21.15.67:0/37687443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640c9f00 con 0x7faf640ab8e0 2024-03-09T09:30:20.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.308+0000 7faf69800640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf640ab8e0 0x7faf64147750 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7faf5c02c150 tx=0x7faf5c0015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:20.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.309+0000 7faf60c00640 1 -- 172.21.15.67:0/37687443 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c006d50 con 0x7faf640ab8e0 2024-03-09T09:30:20.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.309+0000 7faf60c00640 1 -- 172.21.15.67:0/37687443 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7faf5c02d4f0 con 0x7faf640ab8e0 2024-03-09T09:30:20.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.309+0000 7faf6a200640 1 -- 172.21.15.67:0/37687443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf640ca1e0 con 0x7faf640ab8e0 2024-03-09T09:30:20.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.309+0000 7faf60c00640 1 -- 172.21.15.67:0/37687443 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c02d8e0 con 0x7faf640ab8e0 2024-03-09T09:30:20.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.309+0000 7faf6a200640 1 -- 172.21.15.67:0/37687443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf640ab3c0 con 0x7faf640ab8e0 2024-03-09T09:30:20.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.310+0000 7faf6a200640 1 -- 172.21.15.67:0/37687443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf640ab5c0 con 0x7faf640ab8e0 2024-03-09T09:30:20.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.311+0000 7faf60c00640 1 -- 172.21.15.67:0/37687443 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faf5c040070 con 0x7faf640ab8e0 2024-03-09T09:30:20.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.312+0000 7faf60c00640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf4406b500 0x7faf4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-09T09:30:20.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.312+0000 7faf60c00640 1 -- 172.21.15.67:0/37687443 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5433+0+0 (secure 0 0 0) 0x7faf5c0aba60 con 0x7faf640ab8e0 2024-03-09T09:30:20.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.313+0000 7faf68e00640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf4406b500 0x7faf4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-09T09:30:20.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.314+0000 7faf68e00640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf4406b500 0x7faf4406d9c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7faf54009970 tx=0x7faf54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-09T09:30:20.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.317+0000 7faf60c00640 1 -- 172.21.15.67:0/37687443 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf640ab5c0 con 0x7faf640ab8e0 2024-03-09T09:30:20.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:20.518+0000 7faf6a200640 1 -- 172.21.15.67:0/37687443 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "b_data", "pool2": "b_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faf6404f430 con 0x7faf640ab8e0 2024-03-09T09:30:21.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.061+0000 7faf60c00640 1 -- 172.21.15.67:0/37687443 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "b_data", "pool2": "b_data", "yes_i_really_really_mean_it": true}]=0 pool 'b_data' removed v95) v1 ==== 154+0+0 (secure 0 0 0) 0x7faf5c048060 con 0x7faf640ab8e0 2024-03-09T09:30:21.069 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'b_data' removed 2024-03-09T09:30:21.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.064+0000 7faf6a200640 1 -- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf4406b500 msgr2=0x7faf4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:21.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.064+0000 7faf6a200640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf4406b500 0x7faf4406d9c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7faf54009970 tx=0x7faf54008040 comp rx=0 tx=0).stop 2024-03-09T09:30:21.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.064+0000 7faf6a200640 1 -- 172.21.15.67:0/37687443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf640ab8e0 msgr2=0x7faf64147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-09T09:30:21.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.064+0000 7faf6a200640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf640ab8e0 0x7faf64147750 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7faf5c02c150 tx=0x7faf5c0015b0 comp rx=0 tx=0).stop 2024-03-09T09:30:21.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.065+0000 7faf6a200640 1 -- 172.21.15.67:0/37687443 shutdown_connections 2024-03-09T09:30:21.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.065+0000 7faf6a200640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:6832/1750216114,v1:172.21.15.134:6833/1750216114] conn(0x7faf4406b500 0x7faf4406d9c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:21.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.065+0000 7faf6a200640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faf640ab8e0 0x7faf64147750 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:21.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.065+0000 7faf6a200640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf641ebb90 0x7faf640ac5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:21.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.065+0000 7faf6a200640 1 --2- 172.21.15.67:0/37687443 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf48005210 0x7faf640af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-09T09:30:21.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.065+0000 7faf6a200640 1 -- 172.21.15.67:0/37687443 >> 172.21.15.67:0/37687443 conn(0x7faf640ad0d0 msgr2=0x7faf640c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-09T09:30:21.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.065+0000 7faf6a200640 1 -- 172.21.15.67:0/37687443 shutdown_connections 2024-03-09T09:30:21.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-09T09:30:21.065+0000 7faf6a200640 1 -- 172.21.15.67:0/37687443 wait complete. 2024-03-09T09:30:21.098 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-09T09:30:21.098 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-09T09:30:21.098 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:30:21.131 INFO:tasks.ceph.mds.a:Stopped 2024-03-09T09:30:21.131 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-09T09:30:21.131 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:30:21.198 INFO:tasks.ceph.mds.c:Stopped 2024-03-09T09:30:21.198 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-09T09:30:21.198 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:30:21.226 INFO:tasks.ceph.mds.b:Stopped 2024-03-09T09:30:21.226 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-09T09:30:21.226 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:30:21.300 INFO:tasks.ceph.mds.d:Stopped 2024-03-09T09:30:21.300 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-09T09:30:21.301 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-09T09:30:21.301 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:30:27.403 INFO:tasks.ceph.osd.0:Stopped 2024-03-09T09:30:27.404 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-09T09:30:27.404 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:30:28.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-03-09T09:30:33.506 INFO:tasks.ceph.osd.1:Stopped 2024-03-09T09:30:33.507 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-09T09:30:33.507 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:30:34.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-03-09T09:30:34.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-03-09T09:30:39.611 INFO:tasks.ceph.osd.2:Stopped 2024-03-09T09:30:39.611 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-09T09:30:39.611 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:30:40.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-03-09T09:30:40.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-03-09T09:30:45.713 INFO:tasks.ceph.osd.3:Stopped 2024-03-09T09:30:45.714 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-09T09:30:45.714 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:30:46.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-03-09T09:30:46.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-03-09T09:30:51.816 INFO:tasks.ceph.osd.4:Stopped 2024-03-09T09:30:51.817 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-09T09:30:51.817 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:30:51.970 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-03-09T09:30:51.971 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-03-09T09:30:57.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-03-09T09:30:57.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-03-09T09:30:57.919 INFO:tasks.ceph.osd.5:Stopped 2024-03-09T09:30:57.919 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-09T09:30:57.920 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:30:58.008 INFO:tasks.ceph.osd.6:Stopped 2024-03-09T09:30:58.009 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-09T09:30:58.009 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:31:03.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-03-09T09:31:03.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-03-09T09:31:04.111 INFO:tasks.ceph.osd.7:Stopped 2024-03-09T09:31:04.112 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-09T09:31:04.112 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-09T09:31:04.112 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:31:04.212 INFO:tasks.ceph.mgr.y:Stopped 2024-03-09T09:31:04.212 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-09T09:31:04.212 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:31:04.311 INFO:tasks.ceph.mgr.x:Stopped 2024-03-09T09:31:04.312 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-09T09:31:04.312 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-09T09:31:04.312 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:31:04.367 INFO:tasks.ceph.mon.a:Stopped 2024-03-09T09:31:04.367 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-09T09:31:04.367 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:31:04.393 INFO:tasks.ceph.mon.b:Stopped 2024-03-09T09:31:04.394 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-09T09:31:04.394 INFO:teuthology.orchestra.run:waiting for 300 2024-03-09T09:31:04.459 INFO:tasks.ceph.mon.c:Stopped 2024-03-09T09:31:04.459 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-09T09:31:04.460 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-03-09T09:31:04.593 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T09:31:04.593 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-09T09:31:04.919 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T09:31:04.919 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-09T09:31:05.135 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T09:31:05.135 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-09T09:31:05.334 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T09:31:05.334 DEBUG:teuthology.orchestra.run.smithi067:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-09T09:31:05.563 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T09:31:05.563 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-09T09:31:06.945 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T09:31:06.945 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-09T09:31:07.168 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T09:31:07.169 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-09T09:31:07.369 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T09:31:07.369 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-09T09:31:07.594 INFO:tasks.ceph:Archiving mon data... 2024-03-09T09:31:07.596 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387/data/mon.a.tgz 2024-03-09T09:31:07.596 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-03-09T09:31:07.652 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.KZYPqi2jbv 2024-03-09T09:31:07.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.KZYPqi2jbv 2024-03-09T09:31:07.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.KZYPqi2jbv 2024-03-09T09:31:08.024 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.KZYPqi2jbv is 546KB 2024-03-09T09:31:08.142 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.KZYPqi2jbv 2024-03-09T09:31:08.184 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-03-09T09:31:08.202 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387/data/mon.b.tgz 2024-03-09T09:31:08.202 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-03-09T09:31:08.252 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.t53xXYB5Qa 2024-03-09T09:31:08.253 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.t53xXYB5Qa 2024-03-09T09:31:08.495 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.t53xXYB5Qa 2024-03-09T09:31:08.634 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.t53xXYB5Qa is 557KB 2024-03-09T09:31:08.735 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.t53xXYB5Qa 2024-03-09T09:31:08.784 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387/data/mon.c.tgz 2024-03-09T09:31:08.784 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-03-09T09:31:08.873 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.KhjHi5SPSp 2024-03-09T09:31:08.874 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.KhjHi5SPSp 2024-03-09T09:31:09.064 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.KhjHi5SPSp 2024-03-09T09:31:09.188 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.KhjHi5SPSp is 556KB 2024-03-09T09:31:09.290 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.KhjHi5SPSp 2024-03-09T09:31:09.338 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-09T09:31:09.339 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-09T09:31:09.346 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-09T09:31:09.523 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-09T09:31:09.573 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-09T09:31:09.573 INFO:tasks.ceph:Archiving crash dumps... 2024-03-09T09:31:09.575 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387/remote/smithi067/crash 2024-03-09T09:31:09.576 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-09T09:31:09.677 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387/remote/smithi134/crash 2024-03-09T09:31:09.678 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-09T09:31:09.757 INFO:tasks.ceph:Shutting down logrotate 2024-03-09T09:31:29.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-09T09:31:29.698 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-09T09:31:29.791 INFO:tasks.ceph:Compressing logs... 2024-03-09T09:31:29.792 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-09T09:31:29.799 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-09T09:31:29.884 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-09T09:31:29.885 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37371.log 2024-03-09T09:31:29.886 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42715.log 2024-03-09T09:31:29.887 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.37371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37331.log 2024-03-09T09:31:29.887 INFO:teuthology.orchestra.run.smithi134.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37371.log.gz 2024-03-09T09:31:29.888 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.1.42715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42715.log.gz 2024-03-09T09:31:29.888 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-09T09:31:29.889 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.37331.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.37331.log.gz 2024-03-09T09:31:29.889 INFO:teuthology.orchestra.run.smithi134.stderr: -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-09T09:31:29.890 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37171.log 2024-03-09T09:31:29.892 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-09T09:31:29.892 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78264.log 2024-03-09T09:31:29.893 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.37171.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37171.log.gz 2024-03-09T09:31:29.893 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37131.log 2024-03-09T09:31:29.893 INFO:teuthology.orchestra.run.smithi134.stderr: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-09T09:31:29.895 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42366.log 2024-03-09T09:31:29.895 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42450.log 2024-03-09T09:31:29.895 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78264.log.gz 2024-03-09T09:31:29.896 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37211.log/var/log/ceph/ceph-osd.4.log: 2024-03-09T09:31:29.896 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.37131.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37131.log.gz 2024-03-09T09:31:29.896 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-09T09:31:29.898 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.37211.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.37211.log.gz 2024-03-09T09:31:29.898 INFO:teuthology.orchestra.run.smithi134.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-09T09:31:29.898 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-09T09:31:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66041.log 2024-03-09T09:31:29.899 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42366.log.gz 2024-03-09T09:31:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42450.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42450.log.gz 2024-03-09T09:31:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66700.log 2024-03-09T09:31:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71621.log 2024-03-09T09:31:29.900 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.66041.log.gz 2024-03-09T09:31:29.901 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50842.log 2024-03-09T09:31:29.901 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66700.log.gz 2024-03-09T09:31:29.901 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60425.log 2024-03-09T09:31:29.901 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71621.log.gz 2024-03-09T09:31:29.902 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37411.log 2024-03-09T09:31:29.902 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42696.log 2024-03-09T09:31:29.903 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.37411.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37411.log.gz 2024-03-09T09:31:29.903 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.1.42696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42696.log.gz 2024-03-09T09:31:29.904 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44879.log 2024-03-09T09:31:29.904 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50842.log.gz 2024-03-09T09:31:29.904 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76940.log 2024-03-09T09:31:29.904 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60425.log.gz 2024-03-09T09:31:29.905 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58959.log 2024-03-09T09:31:29.906 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44879.log.gz 2024-03-09T09:31:29.906 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44339.log 2024-03-09T09:31:29.906 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76940.log.gz 2024-03-09T09:31:29.907 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-09T09:31:29.907 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37291.log 2024-03-09T09:31:29.908 INFO:teuthology.orchestra.run.smithi134.stderr: 93.7%/var/log/ceph/ceph.log: -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-09T09:31:29.908 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-09T09:31:29.909 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.37291.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37291.log.gz 2024-03-09T09:31:29.910 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77908.log 2024-03-09T09:31:29.910 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58959.log.gz 2024-03-09T09:31:29.910 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44339.log: gzip -5 --verbose 87.5% -- -- replaced with /var/log/ceph/ceph-client.admin.44339.log.gz /var/log/ceph/ceph-client.admin.51737.log 2024-03-09T09:31:29.911 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-03-09T09:31:29.911 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47029.log 2024-03-09T09:31:29.911 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77908.log.gz 2024-03-09T09:31:29.911 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78600.log 2024-03-09T09:31:29.911 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51737.log.gz 2024-03-09T09:31:29.912 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-09T09:31:29.913 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37251.log 2024-03-09T09:31:29.913 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47029.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47029.log.gz 2024-03-09T09:31:29.914 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47650.log 2024-03-09T09:31:29.914 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45383.log 2024-03-09T09:31:29.914 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78600.log.gz 2024-03-09T09:31:29.915 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77294.log 2024-03-09T09:31:29.916 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-09T09:31:29.918 INFO:teuthology.orchestra.run.smithi067.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.47650.log.gz 2024-03-09T09:31:29.918 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45383.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45383.log.gz 2024-03-09T09:31:29.919 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79427.log 2024-03-09T09:31:29.919 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57469.log 2024-03-09T09:31:29.919 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77294.log.gz 2024-03-09T09:31:29.919 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53969.log 2024-03-09T09:31:29.919 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79427.log.gz 2024-03-09T09:31:29.920 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57469.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-09T09:31:29.920 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57469.log.gz 2024-03-09T09:31:29.921 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.37251.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37251.log.gz 2024-03-09T09:31:29.924 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55134.log 2024-03-09T09:31:29.924 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53969.log.gz 2024-03-09T09:31:29.924 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45827.log 2024-03-09T09:31:29.924 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73705.log 2024-03-09T09:31:29.924 INFO:teuthology.orchestra.run.smithi067.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.55134.log.gz 2024-03-09T09:31:29.924 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68526.log 2024-03-09T09:31:29.925 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45827.log.gz 2024-03-09T09:31:29.925 INFO:teuthology.orchestra.run.smithi134.stderr: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-09T09:31:29.926 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70276.log 2024-03-09T09:31:29.927 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68526.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.73705.log.gz 2024-03-09T09:31:29.927 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68526.log.gz 2024-03-09T09:31:29.927 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62759.log 2024-03-09T09:31:29.927 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60295.log 2024-03-09T09:31:29.929 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70276.log.gz 2024-03-09T09:31:29.929 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68120.log 2024-03-09T09:31:29.929 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62759.log.gz 2024-03-09T09:31:29.930 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64755.log 2024-03-09T09:31:29.931 INFO:teuthology.orchestra.run.smithi067.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60295.log.gz 2024-03-09T09:31:29.931 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72382.log 2024-03-09T09:31:29.931 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68120.log.gz 2024-03-09T09:31:29.933 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56871.log 2024-03-09T09:31:29.933 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64755.log.gz 2024-03-09T09:31:29.933 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54362.log 2024-03-09T09:31:29.934 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72382.log.gz 2024-03-09T09:31:29.934 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55404.log 2024-03-09T09:31:29.934 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56871.log.gz 2024-03-09T09:31:29.935 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58076.log 2024-03-09T09:31:29.935 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54362.log.gz 2024-03-09T09:31:29.936 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47506.log 2024-03-09T09:31:29.937 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55404.log.gz 2024-03-09T09:31:29.938 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63123.log 2024-03-09T09:31:29.938 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58076.log.gz 2024-03-09T09:31:29.939 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59350.log 2024-03-09T09:31:29.939 INFO:teuthology.orchestra.run.smithi067.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47506.log.gz 2024-03-09T09:31:29.940 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50195.log 2024-03-09T09:31:29.940 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63123.log.gz 2024-03-09T09:31:29.941 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79259.log 2024-03-09T09:31:29.942 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59350.log.gz 2024-03-09T09:31:29.944 INFO:teuthology.orchestra.run.smithi134.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-09T09:31:29.944 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50195.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50195.log.gz 2024-03-09T09:31:29.945 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42408.log 2024-03-09T09:31:29.945 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64456.log 2024-03-09T09:31:29.945 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79259.log.gz 2024-03-09T09:31:29.946 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59118.log 2024-03-09T09:31:29.946 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42408.log.gz 2024-03-09T09:31:29.946 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44075.log 2024-03-09T09:31:29.947 INFO:teuthology.orchestra.run.smithi067.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.64456.log.gz 2024-03-09T09:31:29.948 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59118.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59118.log.gz 2024-03-09T09:31:29.948 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74060.log 2024-03-09T09:31:29.949 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71747.log 2024-03-09T09:31:29.949 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44075.log.gz 2024-03-09T09:31:29.950 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55587.log 2024-03-09T09:31:29.950 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74060.log.gz 2024-03-09T09:31:29.951 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59741.log 2024-03-09T09:31:29.952 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71747.log.gz 2024-03-09T09:31:29.952 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55587.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55587.log.gz 2024-03-09T09:31:29.953 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69780.log 2024-03-09T09:31:29.954 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59518.log 2024-03-09T09:31:29.954 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59741.log.gz 2024-03-09T09:31:29.955 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49110.log 2024-03-09T09:31:29.955 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69780.log.gz 2024-03-09T09:31:29.956 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46903.log 2024-03-09T09:31:29.957 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59518.log.gz 2024-03-09T09:31:29.957 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49110.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.49110.log.gz --verbose -- 2024-03-09T09:31:29.958 INFO:teuthology.orchestra.run.smithi067.stderr: /var/log/ceph/ceph-client.admin.44381.log 2024-03-09T09:31:29.958 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51881.log 2024-03-09T09:31:29.959 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46903.log.gz 2024-03-09T09:31:29.960 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44381.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44381.log.gz 2024-03-09T09:31:29.960 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78138.log 2024-03-09T09:31:29.961 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75261.log 2024-03-09T09:31:29.961 INFO:teuthology.orchestra.run.smithi067.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.51881.log.gz 2024-03-09T09:31:29.962 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44795.log 2024-03-09T09:31:29.963 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78138.log.gz 2024-03-09T09:31:29.963 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62175.log 2024-03-09T09:31:29.964 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75261.log.gz 2024-03-09T09:31:29.964 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61245.log 2024-03-09T09:31:29.964 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44795.log.gz 2024-03-09T09:31:29.965 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76297.log 2024-03-09T09:31:29.966 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62175.log.gz 2024-03-09T09:31:29.967 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71663.log 2024-03-09T09:31:29.968 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76297.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.61245.log.gz 2024-03-09T09:31:29.968 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52583.log 2024-03-09T09:31:29.968 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76297.log.gz 2024-03-09T09:31:29.969 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68222.log 2024-03-09T09:31:29.969 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.71663.log.gz 2024-03-09T09:31:29.971 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66871.log 2024-03-09T09:31:29.971 INFO:teuthology.orchestra.run.smithi067.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.52583.log.gz 2024-03-09T09:31:29.972 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44963.log 2024-03-09T09:31:29.972 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68222.log.gz 2024-03-09T09:31:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66871.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.66871.log.gz 2024-03-09T09:31:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42236.log 2024-03-09T09:31:29.973 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70976.log 2024-03-09T09:31:29.974 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44963.log.gz 2024-03-09T09:31:29.974 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62801.log 2024-03-09T09:31:29.975 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42236.log.gz 2024-03-09T09:31:29.976 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62888.log 2024-03-09T09:31:29.976 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70976.log.gz 2024-03-09T09:31:29.977 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73015.log 2024-03-09T09:31:29.977 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62801.log.gz 2024-03-09T09:31:29.978 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63017.log 2024-03-09T09:31:29.979 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62888.log.gz 2024-03-09T09:31:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73015.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73015.log.gz 2024-03-09T09:31:29.980 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76795.log 2024-03-09T09:31:29.981 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69532.log 2024-03-09T09:31:29.981 INFO:teuthology.orchestra.run.smithi067.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.63017.log.gz 2024-03-09T09:31:29.982 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55320.log 2024-03-09T09:31:29.982 INFO:teuthology.orchestra.run.smithi067.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.76795.log.gz 2024-03-09T09:31:29.983 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63749.log 2024-03-09T09:31:29.984 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69532.log.gz 2024-03-09T09:31:29.984 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44064.log 2024-03-09T09:31:29.985 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55320.log.gz 2024-03-09T09:31:29.986 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71400.log 2024-03-09T09:31:29.986 INFO:teuthology.orchestra.run.smithi067.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.63749.log.gz 2024-03-09T09:31:29.987 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55900.log 2024-03-09T09:31:29.987 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44064.log.gz 2024-03-09T09:31:29.988 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47189.log 2024-03-09T09:31:29.988 INFO:teuthology.orchestra.run.smithi067.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.71400.log.gz 2024-03-09T09:31:29.989 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73453.log 2024-03-09T09:31:29.990 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55900.log.gz 2024-03-09T09:31:29.991 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72298.log 2024-03-09T09:31:29.991 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47189.log.gz 2024-03-09T09:31:29.992 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66614.log 2024-03-09T09:31:29.992 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73453.log.gz 2024-03-09T09:31:29.994 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67536.log 2024-03-09T09:31:29.994 INFO:teuthology.orchestra.run.smithi067.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.72298.log.gz 2024-03-09T09:31:29.994 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67446.log 2024-03-09T09:31:29.995 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66614.log.gz 2024-03-09T09:31:29.996 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43736.log 2024-03-09T09:31:29.996 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67536.log.gz 2024-03-09T09:31:29.997 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63535.log 2024-03-09T09:31:29.997 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67446.log.gz 2024-03-09T09:31:29.998 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75329.log 2024-03-09T09:31:29.998 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43736.log.gz 2024-03-09T09:31:29.999 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47861.log 2024-03-09T09:31:29.999 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.63535.log.gz 2024-03-09T09:31:30.000 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60383.log 2024-03-09T09:31:30.002 INFO:teuthology.orchestra.run.smithi067.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.75329.log.gz 2024-03-09T09:31:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74102.log 2024-03-09T09:31:30.002 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47861.log.gz 2024-03-09T09:31:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67389.log 2024-03-09T09:31:30.003 INFO:teuthology.orchestra.run.smithi067.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60383.log.gz 2024-03-09T09:31:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78348.log 2024-03-09T09:31:30.004 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74102.log.gz 2024-03-09T09:31:30.004 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52151.log 2024-03-09T09:31:30.005 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67389.log.gz 2024-03-09T09:31:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45869.log 2024-03-09T09:31:30.006 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78348.log.gz 2024-03-09T09:31:30.007 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74186.log 2024-03-09T09:31:30.007 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52151.log.gz 2024-03-09T09:31:30.008 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45785.log 2024-03-09T09:31:30.009 INFO:teuthology.orchestra.run.smithi067.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45869.log.gz 2024-03-09T09:31:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47949.log 2024-03-09T09:31:30.010 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74186.log.gz 2024-03-09T09:31:30.011 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55278.log 2024-03-09T09:31:30.011 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45785.log.gz 2024-03-09T09:31:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65999.log 2024-03-09T09:31:30.012 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47949.log.gz 2024-03-09T09:31:30.013 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44408.log 2024-03-09T09:31:30.013 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.55278.log.gz 2024-03-09T09:31:30.014 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47590.log 2024-03-09T09:31:30.014 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65999.log.gz 2024-03-09T09:31:30.015 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43307.log 2024-03-09T09:31:30.015 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44408.log.gz 2024-03-09T09:31:30.017 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47590.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47590.log.gz 2024-03-09T09:31:30.017 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78516.log 2024-03-09T09:31:30.018 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56073.log 2024-03-09T09:31:30.018 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43307.log.gz 2024-03-09T09:31:30.019 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73351.log 2024-03-09T09:31:30.019 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78516.log.gz 2024-03-09T09:31:30.021 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57782.log 2024-03-09T09:31:30.021 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56073.log.gz 2024-03-09T09:31:30.022 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44371.log 2024-03-09T09:31:30.022 INFO:teuthology.orchestra.run.smithi067.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.73351.log.gz 2024-03-09T09:31:30.023 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57511.log 2024-03-09T09:31:30.023 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57782.log.gz 2024-03-09T09:31:30.024 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42534.log 2024-03-09T09:31:30.025 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44371.log.gz 2024-03-09T09:31:30.025 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42492.log 2024-03-09T09:31:30.026 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57511.log.gz 2024-03-09T09:31:30.026 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75937.log 2024-03-09T09:31:30.027 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42534.log.gz 2024-03-09T09:31:30.028 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43737.log 2024-03-09T09:31:30.028 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42492.log.gz 2024-03-09T09:31:30.029 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43135.log 2024-03-09T09:31:30.029 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75937.log.gz 2024-03-09T09:31:30.030 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57595.log 2024-03-09T09:31:30.031 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43737.log.gz 2024-03-09T09:31:30.032 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52007.log 2024-03-09T09:31:30.032 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43135.log.gz 2024-03-09T09:31:30.033 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44479.log 2024-03-09T09:31:30.033 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57595.log.gz 2024-03-09T09:31:30.034 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46713.log 2024-03-09T09:31:30.034 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52007.log.gz 2024-03-09T09:31:30.035 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66253.log 2024-03-09T09:31:30.035 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44479.log.gz 2024-03-09T09:31:30.036 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41749.log 2024-03-09T09:31:30.036 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46713.log.gz 2024-03-09T09:31:30.037 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48033.log 2024-03-09T09:31:30.038 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66253.log.gz 2024-03-09T09:31:30.039 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59244.log 2024-03-09T09:31:30.039 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41749.log.gz 2024-03-09T09:31:30.040 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61355.log 2024-03-09T09:31:30.040 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48033.log.gz 2024-03-09T09:31:30.041 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57141.log 2024-03-09T09:31:30.042 INFO:teuthology.orchestra.run.smithi067.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.59244.log.gz 2024-03-09T09:31:30.042 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47379.log 2024-03-09T09:31:30.043 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.61355.log.gz 2024-03-09T09:31:30.044 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44921.log 2024-03-09T09:31:30.044 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57141.log.gz 2024-03-09T09:31:30.045 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63251.log 2024-03-09T09:31:30.045 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47379.log.gz 2024-03-09T09:31:30.046 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71902.log 2024-03-09T09:31:30.046 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44921.log.gz 2024-03-09T09:31:30.047 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56157.log 2024-03-09T09:31:30.047 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63251.log.gz 2024-03-09T09:31:30.048 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46945.log 2024-03-09T09:31:30.049 INFO:teuthology.orchestra.run.smithi067.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.71902.log.gz 2024-03-09T09:31:30.050 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62388.log 2024-03-09T09:31:30.050 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56157.log.gz 2024-03-09T09:31:30.051 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72209.log 2024-03-09T09:31:30.051 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46945.log.gz 2024-03-09T09:31:30.052 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45617.log 2024-03-09T09:31:30.052 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.62388.log.gz 2024-03-09T09:31:30.053 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69227.log 2024-03-09T09:31:30.053 INFO:teuthology.orchestra.run.smithi067.stderr: 92.3%/var/log/ceph/ceph-client.admin.45617.log: -- replaced with /var/log/ceph/ceph-client.admin.72209.log.gz 2024-03-09T09:31:30.054 INFO:teuthology.orchestra.run.smithi067.stderr:gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45617.log.gz -5 2024-03-09T09:31:30.054 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47991.log 2024-03-09T09:31:30.055 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65913.log 2024-03-09T09:31:30.056 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69227.log.gz 2024-03-09T09:31:30.056 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65526.log 2024-03-09T09:31:30.057 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.47991.log.gz 2024-03-09T09:31:30.058 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58430.log 2024-03-09T09:31:30.058 INFO:teuthology.orchestra.run.smithi067.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.65913.log.gz 2024-03-09T09:31:30.065 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70234.log 2024-03-09T09:31:30.065 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65526.log.gz 2024-03-09T09:31:30.065 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71124.log 2024-03-09T09:31:30.065 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58430.log.gz 2024-03-09T09:31:30.066 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45131.log 2024-03-09T09:31:30.066 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70234.log.gz 2024-03-09T09:31:30.066 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78966.log 2024-03-09T09:31:30.066 INFO:teuthology.orchestra.run.smithi067.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.71124.log.gz 2024-03-09T09:31:30.066 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37261.log 2024-03-09T09:31:30.066 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45131.log.gz 2024-03-09T09:31:30.066 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46332.log 2024-03-09T09:31:30.066 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78966.log.gz 2024-03-09T09:31:30.067 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.37261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78894.log 2024-03-09T09:31:30.067 INFO:teuthology.orchestra.run.smithi067.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37261.log.gz 2024-03-09T09:31:30.068 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.46332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46332.log.gz 2024-03-09T09:31:30.068 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79175.log 2024-03-09T09:31:30.068 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75853.log 2024-03-09T09:31:30.069 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78894.log.gz 2024-03-09T09:31:30.070 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62259.log 2024-03-09T09:31:30.070 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79175.log.gz 2024-03-09T09:31:30.071 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74228.log 2024-03-09T09:31:30.071 INFO:teuthology.orchestra.run.smithi067.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.75853.log.gz 2024-03-09T09:31:30.072 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65808.log 2024-03-09T09:31:30.072 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62259.log.gz 2024-03-09T09:31:30.073 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55816.log 2024-03-09T09:31:30.074 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74228.log.gz 2024-03-09T09:31:30.075 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54609.log 2024-03-09T09:31:30.075 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65808.log.gz 2024-03-09T09:31:30.076 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58304.log 2024-03-09T09:31:30.076 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55816.log.gz 2024-03-09T09:31:30.077 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66913.log 2024-03-09T09:31:30.077 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.54609.log.gz 2024-03-09T09:31:30.078 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69101.log 2024-03-09T09:31:30.078 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58304.log.gz 2024-03-09T09:31:30.079 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55446.log 2024-03-09T09:31:30.080 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66913.log.gz 2024-03-09T09:31:30.080 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76024.log 2024-03-09T09:31:30.081 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69101.log.gz 2024-03-09T09:31:30.082 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66572.log 2024-03-09T09:31:30.083 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55446.log.gz 2024-03-09T09:31:30.084 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76567.log 2024-03-09T09:31:30.084 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76024.log.gz 2024-03-09T09:31:30.084 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45659.log 2024-03-09T09:31:30.085 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66572.log.gz 2024-03-09T09:31:30.086 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72644.log 2024-03-09T09:31:30.086 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.76567.log.gz 2024-03-09T09:31:30.087 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54693.log 2024-03-09T09:31:30.087 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45659.log.gz 2024-03-09T09:31:30.088 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45173.log 2024-03-09T09:31:30.088 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72644.log.gz 2024-03-09T09:31:30.089 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51611.log 2024-03-09T09:31:30.089 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54693.log.gz 2024-03-09T09:31:30.090 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48177.log 2024-03-09T09:31:30.090 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45173.log.gz 2024-03-09T09:31:30.091 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51611.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51611.log.gz 2024-03-09T09:31:30.092 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76525.log 2024-03-09T09:31:30.093 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48177.log: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.48177.log.gz 2024-03-09T09:31:30.094 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41939.log 2024-03-09T09:31:30.095 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75570.log 2024-03-09T09:31:30.095 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76525.log.gz 2024-03-09T09:31:30.096 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58388.log 2024-03-09T09:31:30.097 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41939.log.gz 2024-03-09T09:31:30.098 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76465.log 2024-03-09T09:31:30.098 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.75570.log.gz 2024-03-09T09:31:30.099 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47819.log 2024-03-09T09:31:30.101 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58388.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.58388.log.gz 2024-03-09T09:31:30.101 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47252.log 2024-03-09T09:31:30.101 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76465.log.gz 2024-03-09T09:31:30.103 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77368.log 2024-03-09T09:31:30.103 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47819.log.gz 2024-03-09T09:31:30.103 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51653.log 2024-03-09T09:31:30.103 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47252.log.gz 2024-03-09T09:31:30.105 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71303.log 2024-03-09T09:31:30.106 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77368.log.gz 2024-03-09T09:31:30.106 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78684.log 2024-03-09T09:31:30.106 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51653.log.gz 2024-03-09T09:31:30.108 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62133.log 2024-03-09T09:31:30.108 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71303.log.gz 2024-03-09T09:31:30.108 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54082.log 2024-03-09T09:31:30.109 INFO:teuthology.orchestra.run.smithi067.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.78684.log.gz 2024-03-09T09:31:30.109 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47129.log 2024-03-09T09:31:30.109 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62133.log.gz 2024-03-09T09:31:30.111 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79553.log 2024-03-09T09:31:30.111 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54082.log.gz 2024-03-09T09:31:30.112 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44068.log 2024-03-09T09:31:30.112 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47129.log.gz 2024-03-09T09:31:30.112 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78768.log 2024-03-09T09:31:30.112 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79553.log.gz 2024-03-09T09:31:30.115 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67494.log 2024-03-09T09:31:30.115 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44068.log.gz 2024-03-09T09:31:30.115 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71204.log 2024-03-09T09:31:30.116 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78768.log.gz 2024-03-09T09:31:30.116 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61981.log 2024-03-09T09:31:30.116 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67494.log.gz 2024-03-09T09:31:30.119 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53571.log 2024-03-09T09:31:30.120 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71204.log.gz 2024-03-09T09:31:30.120 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76109.log 2024-03-09T09:31:30.120 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61981.log.gz 2024-03-09T09:31:30.120 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49356.log 2024-03-09T09:31:30.121 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53571.log.gz 2024-03-09T09:31:30.121 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74312.log 2024-03-09T09:31:30.121 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76109.log.gz 2024-03-09T09:31:30.124 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47693.log 2024-03-09T09:31:30.124 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74312.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.49356.log.gz 2024-03-09T09:31:30.125 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74312.log.gz 2024-03-09T09:31:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54194.log 2024-03-09T09:31:30.125 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58742.log 2024-03-09T09:31:30.125 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47693.log.gz 2024-03-09T09:31:30.126 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71949.log 2024-03-09T09:31:30.126 INFO:teuthology.orchestra.run.smithi067.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.54194.log.gz 2024-03-09T09:31:30.128 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59567.log 2024-03-09T09:31:30.129 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58742.log.gz 2024-03-09T09:31:30.129 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76609.log 2024-03-09T09:31:30.129 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71949.log.gz 2024-03-09T09:31:30.129 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59567.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59567.log.gz 2024-03-09T09:31:30.129 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42954.log 2024-03-09T09:31:30.130 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65765.log 2024-03-09T09:31:30.130 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.76609.log.gz 2024-03-09T09:31:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52049.log 2024-03-09T09:31:30.131 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42954.log.gz 2024-03-09T09:31:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65765.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.65765.log.gz --verbose 2024-03-09T09:31:30.131 INFO:teuthology.orchestra.run.smithi067.stderr: -- /var/log/ceph/ceph-client.admin.52277.log 2024-03-09T09:31:30.132 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52685.log 2024-03-09T09:31:30.133 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52049.log.gz 2024-03-09T09:31:30.133 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49522.log 2024-03-09T09:31:30.133 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52277.log.gz 2024-03-09T09:31:30.134 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73976.log 2024-03-09T09:31:30.134 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.52685.log.gz 2024-03-09T09:31:30.136 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51965.log 2024-03-09T09:31:30.136 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49522.log.gz 2024-03-09T09:31:30.137 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60341.log 2024-03-09T09:31:30.137 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.73976.log.gz 2024-03-09T09:31:30.138 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43735.log 2024-03-09T09:31:30.138 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51965.log.gz 2024-03-09T09:31:30.139 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53529.log 2024-03-09T09:31:30.140 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43735.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60341.log.gz 2024-03-09T09:31:30.140 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43735.log.gz 2024-03-09T09:31:30.140 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77108.log 2024-03-09T09:31:30.141 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42324.log 2024-03-09T09:31:30.142 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53529.log.gz 2024-03-09T09:31:30.143 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61128.log 2024-03-09T09:31:30.143 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77108.log.gz 2024-03-09T09:31:30.143 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56973.log 2024-03-09T09:31:30.144 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61128.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42324.log.gz 2024-03-09T09:31:30.144 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.61128.log.gz 2024-03-09T09:31:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51821.log 2024-03-09T09:31:30.145 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77252.log 2024-03-09T09:31:30.146 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56973.log.gz 2024-03-09T09:31:30.147 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52403.log 2024-03-09T09:31:30.147 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51821.log.gz 2024-03-09T09:31:30.148 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53841.log 2024-03-09T09:31:30.148 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77252.log.gz 2024-03-09T09:31:30.149 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49007.log 2024-03-09T09:31:30.149 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52403.log.gz 2024-03-09T09:31:30.150 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54966.log 2024-03-09T09:31:30.150 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53841.log.gz 2024-03-09T09:31:30.151 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76982.log 2024-03-09T09:31:30.152 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0%/var/log/ceph/ceph-client.admin.54966.log: -- replaced with /var/log/ceph/ceph-client.admin.49007.log.gz 2024-03-09T09:31:30.152 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54966.log.gz 2024-03-09T09:31:30.153 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78852.log 2024-03-09T09:31:30.153 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60719.log 2024-03-09T09:31:30.154 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76982.log.gz 2024-03-09T09:31:30.155 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52643.log 2024-03-09T09:31:30.155 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78852.log.gz 2024-03-09T09:31:30.156 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78474.log 2024-03-09T09:31:30.157 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52643.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.60719.log.gz 2024-03-09T09:31:30.157 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52643.log.gz 2024-03-09T09:31:30.158 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45089.log 2024-03-09T09:31:30.159 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50237.log 2024-03-09T09:31:30.159 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78474.log.gz 2024-03-09T09:31:30.159 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67714.log 2024-03-09T09:31:30.160 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45089.log.gz 2024-03-09T09:31:30.160 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66744.log 2024-03-09T09:31:30.161 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50237.log.gz 2024-03-09T09:31:30.162 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46860.log 2024-03-09T09:31:30.162 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67714.log.gz 2024-03-09T09:31:30.163 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57950.log 2024-03-09T09:31:30.163 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66744.log.gz 2024-03-09T09:31:30.164 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51923.log 2024-03-09T09:31:30.164 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46860.log.gz 2024-03-09T09:31:30.165 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70434.log 2024-03-09T09:31:30.166 INFO:teuthology.orchestra.run.smithi067.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.57950.log.gz 2024-03-09T09:31:30.167 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56619.log 2024-03-09T09:31:30.167 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51923.log.gz 2024-03-09T09:31:30.168 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72602.log 2024-03-09T09:31:30.168 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70434.log.gz 2024-03-09T09:31:30.169 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45005.log 2024-03-09T09:31:30.169 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56619.log.gz 2024-03-09T09:31:30.170 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55858.log 2024-03-09T09:31:30.170 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72602.log.gz 2024-03-09T09:31:30.171 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60127.log 2024-03-09T09:31:30.172 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45005.log.gz 2024-03-09T09:31:30.172 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56703.log 2024-03-09T09:31:30.172 INFO:teuthology.orchestra.run.smithi067.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.55858.log.gz 2024-03-09T09:31:30.173 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50033.log 2024-03-09T09:31:30.174 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60127.log.gz 2024-03-09T09:31:30.175 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76151.log 2024-03-09T09:31:30.175 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56703.log.gz 2024-03-09T09:31:30.176 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58574.log 2024-03-09T09:31:30.176 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50033.log.gz 2024-03-09T09:31:30.177 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79343.log 2024-03-09T09:31:30.177 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76151.log.gz 2024-03-09T09:31:30.178 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55774.log 2024-03-09T09:31:30.179 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58574.log.gz 2024-03-09T09:31:30.179 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67620.log 2024-03-09T09:31:30.180 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79343.log.gz 2024-03-09T09:31:30.180 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71579.log 2024-03-09T09:31:30.181 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55774.log.gz 2024-03-09T09:31:30.181 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76693.log 2024-03-09T09:31:30.182 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.67620.log.gz 2024-03-09T09:31:30.183 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71794.log 2024-03-09T09:31:30.183 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71579.log.gz 2024-03-09T09:31:30.184 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52319.log 2024-03-09T09:31:30.184 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71794.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76693.log.gz 2024-03-09T09:31:30.185 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71794.log.gz 2024-03-09T09:31:30.185 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62931.log 2024-03-09T09:31:30.186 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55530.log 2024-03-09T09:31:30.187 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52319.log.gz 2024-03-09T09:31:30.187 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66295.log 2024-03-09T09:31:30.187 INFO:teuthology.orchestra.run.smithi067.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.62931.log.gz 2024-03-09T09:31:30.188 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45971.log 2024-03-09T09:31:30.189 INFO:teuthology.orchestra.run.smithi067.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55530.log.gz 2024-03-09T09:31:30.189 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67662.log 2024-03-09T09:31:30.191 INFO:teuthology.orchestra.run.smithi067.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.66295.log.gz 2024-03-09T09:31:30.191 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74570.log 2024-03-09T09:31:30.191 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45971.log.gz 2024-03-09T09:31:30.192 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56371.log 2024-03-09T09:31:30.193 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67662.log.gz 2024-03-09T09:31:30.194 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54566.log 2024-03-09T09:31:30.194 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74570.log.gz 2024-03-09T09:31:30.195 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67043.log 2024-03-09T09:31:30.195 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56371.log.gz 2024-03-09T09:31:30.196 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46312.log 2024-03-09T09:31:30.196 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54566.log.gz 2024-03-09T09:31:30.197 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52541.log 2024-03-09T09:31:30.197 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.67043.log.gz 2024-03-09T09:31:30.197 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.46312.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46312.log.gz 2024-03-09T09:31:30.198 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58616.log 2024-03-09T09:31:30.198 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45575.log 2024-03-09T09:31:30.199 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52541.log.gz 2024-03-09T09:31:30.200 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54924.log 2024-03-09T09:31:30.200 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58616.log.gz 2024-03-09T09:31:30.201 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54404.log 2024-03-09T09:31:30.201 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45575.log.gz 2024-03-09T09:31:30.202 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58784.log 2024-03-09T09:31:30.202 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54924.log.gz 2024-03-09T09:31:30.203 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54651.log 2024-03-09T09:31:30.203 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54404.log.gz 2024-03-09T09:31:30.205 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55236.log 2024-03-09T09:31:30.205 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58784.log.gz 2024-03-09T09:31:30.205 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79511.log 2024-03-09T09:31:30.206 INFO:teuthology.orchestra.run.smithi067.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.54651.log.gz 2024-03-09T09:31:30.207 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55236.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55236.log.gz 2024-03-09T09:31:30.207 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67001.log 2024-03-09T09:31:30.208 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64368.log 2024-03-09T09:31:30.208 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79511.log.gz 2024-03-09T09:31:30.209 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73789.log 2024-03-09T09:31:30.209 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67001.log.gz 2024-03-09T09:31:30.211 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79656.log 2024-03-09T09:31:30.211 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73789.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64368.log.gz 2024-03-09T09:31:30.212 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73789.log.gz 2024-03-09T09:31:30.212 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45341.log 2024-03-09T09:31:30.213 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47337.log 2024-03-09T09:31:30.214 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79656.log.gz 2024-03-09T09:31:30.214 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49570.log 2024-03-09T09:31:30.214 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45341.log.gz 2024-03-09T09:31:30.217 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47337.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-09T09:31:30.217 INFO:teuthology.orchestra.run.smithi067.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.47337.log.gz 2024-03-09T09:31:30.218 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44069.log 2024-03-09T09:31:30.218 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49570.log.gz 2024-03-09T09:31:30.218 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53757.log 2024-03-09T09:31:30.221 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66379.log 2024-03-09T09:31:30.221 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44069.log.gz 2024-03-09T09:31:30.221 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53757.log: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-09T09:31:30.222 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62973.log 2024-03-09T09:31:30.222 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53757.log.gz 2024-03-09T09:31:30.222 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43042.log 2024-03-09T09:31:30.224 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62973.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.66379.log.gz 2024-03-09T09:31:30.225 INFO:teuthology.orchestra.run.smithi067.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.62973.log.gz 2024-03-09T09:31:30.225 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76212.log 2024-03-09T09:31:30.225 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48261.log 2024-03-09T09:31:30.225 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43042.log.gz 2024-03-09T09:31:30.225 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58034.log 2024-03-09T09:31:30.225 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76212.log.gz 2024-03-09T09:31:30.226 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50754.log 2024-03-09T09:31:30.227 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48261.log.gz 2024-03-09T09:31:30.227 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64669.log 2024-03-09T09:31:30.228 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58034.log.gz 2024-03-09T09:31:30.228 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54735.log 2024-03-09T09:31:30.228 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50754.log.gz 2024-03-09T09:31:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66786.log 2024-03-09T09:31:30.229 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64669.log.gz 2024-03-09T09:31:30.230 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55008.log 2024-03-09T09:31:30.231 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54735.log.gz 2024-03-09T09:31:30.231 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52193.log 2024-03-09T09:31:30.231 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66786.log.gz 2024-03-09T09:31:30.233 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74528.log 2024-03-09T09:31:30.233 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55008.log.gz 2024-03-09T09:31:30.233 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67843.log 2024-03-09T09:31:30.234 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52193.log.gz 2024-03-09T09:31:30.235 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49859.log 2024-03-09T09:31:30.235 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.74528.log.gz 2024-03-09T09:31:30.236 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43475.log 2024-03-09T09:31:30.236 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.67843.log.gz 2024-03-09T09:31:30.237 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65870.log 2024-03-09T09:31:30.238 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49859.log.gz 2024-03-09T09:31:30.238 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67578.log 2024-03-09T09:31:30.239 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43475.log.gz 2024-03-09T09:31:30.240 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54863.log 2024-03-09T09:31:30.241 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.65870.log.gz 2024-03-09T09:31:30.241 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58829.log 2024-03-09T09:31:30.242 INFO:teuthology.orchestra.run.smithi067.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.67578.log.gz 2024-03-09T09:31:30.242 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74270.log 2024-03-09T09:31:30.242 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54863.log.gz 2024-03-09T09:31:30.243 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56329.log 2024-03-09T09:31:30.243 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58829.log.gz 2024-03-09T09:31:30.244 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64711.log 2024-03-09T09:31:30.245 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.74270.log.gz 2024-03-09T09:31:30.246 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60803.log 2024-03-09T09:31:30.246 INFO:teuthology.orchestra.run.smithi067.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.56329.log.gz 2024-03-09T09:31:30.247 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69320.log 2024-03-09T09:31:30.247 INFO:teuthology.orchestra.run.smithi067.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.64711.log.gz 2024-03-09T09:31:30.248 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60509.log 2024-03-09T09:31:30.248 INFO:teuthology.orchestra.run.smithi067.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.60803.log.gz 2024-03-09T09:31:30.249 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69320.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69320.log.gz 2024-03-09T09:31:30.249 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59981.log 2024-03-09T09:31:30.250 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66656.log 2024-03-09T09:31:30.251 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60509.log.gz 2024-03-09T09:31:30.251 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50481.log 2024-03-09T09:31:30.252 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59981.log.gz 2024-03-09T09:31:30.253 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66656.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66656.log.gz 2024-03-09T09:31:30.253 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77740.log 2024-03-09T09:31:30.254 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47548.log 2024-03-09T09:31:30.255 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50481.log.gz 2024-03-09T09:31:30.255 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47907.log 2024-03-09T09:31:30.255 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77740.log.gz 2024-03-09T09:31:30.256 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77698.log 2024-03-09T09:31:30.257 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47548.log.gz 2024-03-09T09:31:30.257 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64070.log 2024-03-09T09:31:30.258 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47907.log.gz 2024-03-09T09:31:30.259 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44421.log 2024-03-09T09:31:30.259 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77698.log.gz 2024-03-09T09:31:30.260 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61741.log 2024-03-09T09:31:30.260 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64070.log.gz 2024-03-09T09:31:30.261 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64627.log 2024-03-09T09:31:30.261 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44421.log.gz 2024-03-09T09:31:30.262 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78306.log 2024-03-09T09:31:30.262 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61741.log.gz 2024-03-09T09:31:30.264 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70586.log 2024-03-09T09:31:30.264 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64627.log.gz 2024-03-09T09:31:30.265 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71352.log 2024-03-09T09:31:30.265 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78306.log.gz 2024-03-09T09:31:30.266 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59897.log 2024-03-09T09:31:30.266 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.70586.log.gz 2024-03-09T09:31:30.267 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53926.log 2024-03-09T09:31:30.268 INFO:teuthology.orchestra.run.smithi067.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.71352.log.gz 2024-03-09T09:31:30.268 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63577.log 2024-03-09T09:31:30.268 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59897.log.gz 2024-03-09T09:31:30.269 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44066.log 2024-03-09T09:31:30.269 INFO:teuthology.orchestra.run.smithi067.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53926.log.gz 2024-03-09T09:31:30.271 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73892.log 2024-03-09T09:31:30.271 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63577.log.gz 2024-03-09T09:31:30.271 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65055.log 2024-03-09T09:31:30.272 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44066.log.gz 2024-03-09T09:31:30.273 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45257.log 2024-03-09T09:31:30.273 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73892.log.gz 2024-03-09T09:31:30.274 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70720.log 2024-03-09T09:31:30.274 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.65055.log.gz 2024-03-09T09:31:30.275 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56413.log 2024-03-09T09:31:30.275 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45257.log.gz 2024-03-09T09:31:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67347.log 2024-03-09T09:31:30.276 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70720.log.gz 2024-03-09T09:31:30.278 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56413.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56413.log.gz 2024-03-09T09:31:30.278 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59202.log 2024-03-09T09:31:30.279 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41840.log 2024-03-09T09:31:30.279 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59202.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67347.log.gz 2024-03-09T09:31:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70628.log 2024-03-09T09:31:30.280 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59202.log.gz 2024-03-09T09:31:30.281 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.41840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63833.log 2024-03-09T09:31:30.281 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41840.log.gz 2024-03-09T09:31:30.282 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74486.log 2024-03-09T09:31:30.283 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70628.log.gz 2024-03-09T09:31:30.283 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64582.log 2024-03-09T09:31:30.284 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63833.log.gz 2024-03-09T09:31:30.285 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62091.log 2024-03-09T09:31:30.285 INFO:teuthology.orchestra.run.smithi067.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.74486.log.gz 2024-03-09T09:31:30.285 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51779.log 2024-03-09T09:31:30.286 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64582.log.gz 2024-03-09T09:31:30.286 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77866.log 2024-03-09T09:31:30.287 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62091.log.gz 2024-03-09T09:31:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68570.log 2024-03-09T09:31:30.289 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51779.log.gz 2024-03-09T09:31:30.289 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48448.log 2024-03-09T09:31:30.290 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77866.log.gz 2024-03-09T09:31:30.291 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48135.log 2024-03-09T09:31:30.291 INFO:teuthology.orchestra.run.smithi067.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.68570.log.gz 2024-03-09T09:31:30.292 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42152.log 2024-03-09T09:31:30.292 INFO:teuthology.orchestra.run.smithi067.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.48448.log.gz 2024-03-09T09:31:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64927.log 2024-03-09T09:31:30.293 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48135.log.gz 2024-03-09T09:31:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75487.log 2024-03-09T09:31:30.295 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42152.log.gz 2024-03-09T09:31:30.295 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64927.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.64927.log.gz 2024-03-09T09:31:30.295 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65678.log 2024-03-09T09:31:30.296 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65354.log 2024-03-09T09:31:30.296 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75487.log.gz 2024-03-09T09:31:30.297 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61573.log 2024-03-09T09:31:30.298 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65678.log.gz 2024-03-09T09:31:30.299 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72167.log 2024-03-09T09:31:30.300 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65354.log.gz 2024-03-09T09:31:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62716.log 2024-03-09T09:31:30.300 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.61573.log.gz 2024-03-09T09:31:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63938.log 2024-03-09T09:31:30.302 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.72167.log.gz 2024-03-09T09:31:30.302 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75029.log 2024-03-09T09:31:30.302 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62716.log.gz 2024-03-09T09:31:30.304 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55092.log 2024-03-09T09:31:30.304 INFO:teuthology.orchestra.run.smithi067.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.63938.log.gz 2024-03-09T09:31:30.304 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73850.log 2024-03-09T09:31:30.304 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75029.log.gz 2024-03-09T09:31:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61931.log 2024-03-09T09:31:30.307 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55092.log.gz 2024-03-09T09:31:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44077.log 2024-03-09T09:31:30.307 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73850.log.gz 2024-03-09T09:31:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55176.log 2024-03-09T09:31:30.309 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.61931.log.gz 2024-03-09T09:31:30.309 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60996.log 2024-03-09T09:31:30.310 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44077.log.gz 2024-03-09T09:31:30.310 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57057.log 2024-03-09T09:31:30.310 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55176.log.gz 2024-03-09T09:31:30.312 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43093.log 2024-03-09T09:31:30.312 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.60996.log.gz 2024-03-09T09:31:30.312 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64024.log 2024-03-09T09:31:30.312 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57057.log.gz 2024-03-09T09:31:30.313 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63165.log 2024-03-09T09:31:30.313 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43093.log.gz 2024-03-09T09:31:30.314 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69998.log 2024-03-09T09:31:30.314 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.64024.log.gz 2024-03-09T09:31:30.315 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75625.log 2024-03-09T09:31:30.315 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63165.log.gz 2024-03-09T09:31:30.316 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64284.log 2024-03-09T09:31:30.316 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69998.log.gz 2024-03-09T09:31:30.317 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70888.log 2024-03-09T09:31:30.317 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75625.log.gz 2024-03-09T09:31:30.318 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64284.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-09T09:31:30.318 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64284.log.gz 2024-03-09T09:31:30.319 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66955.log 2024-03-09T09:31:30.319 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70888.log.gz 2024-03-09T09:31:30.320 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78390.log 2024-03-09T09:31:30.321 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43391.log 2024-03-09T09:31:30.321 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66955.log.gz 2024-03-09T09:31:30.322 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75811.log 2024-03-09T09:31:30.323 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78390.log.gz 2024-03-09T09:31:30.324 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65396.log 2024-03-09T09:31:30.324 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43391.log.gz 2024-03-09T09:31:30.325 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77452.log 2024-03-09T09:31:30.325 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75811.log.gz 2024-03-09T09:31:30.326 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42110.log 2024-03-09T09:31:30.326 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65396.log.gz 2024-03-09T09:31:30.327 INFO:teuthology.orchestra.run.smithi134.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-09T09:31:30.328 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70040.log 2024-03-09T09:31:30.328 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77452.log.gz 2024-03-09T09:31:30.329 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61086.log 2024-03-09T09:31:30.329 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42110.log.gz 2024-03-09T09:31:30.330 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59476.log 2024-03-09T09:31:30.330 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70040.log.gz 2024-03-09T09:31:30.331 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69738.log 2024-03-09T09:31:30.331 INFO:teuthology.orchestra.run.smithi067.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.61086.log.gz 2024-03-09T09:31:30.333 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37005.log 2024-03-09T09:31:30.333 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59476.log.gz 2024-03-09T09:31:30.334 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69738.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.47422.log 2024-03-09T09:31:30.334 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69738.log.gz 2024-03-09T09:31:30.335 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.37005.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37005.log.gz 2024-03-09T09:31:30.335 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78432.log 2024-03-09T09:31:30.336 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68396.log 2024-03-09T09:31:30.337 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47422.log.gz 2024-03-09T09:31:30.338 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78432.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78432.log.gz 2024-03-09T09:31:30.338 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66527.log 2024-03-09T09:31:30.339 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76066.log 2024-03-09T09:31:30.340 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66527.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.68396.log.gz 2024-03-09T09:31:30.341 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66527.log.gz 2024-03-09T09:31:30.341 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68078.log 2024-03-09T09:31:30.342 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63878.log 2024-03-09T09:31:30.343 INFO:teuthology.orchestra.run.smithi067.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.76066.log.gz 2024-03-09T09:31:30.343 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46461.log 2024-03-09T09:31:30.343 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7%/var/log/ceph/ceph-client.admin.63878.log: -- replaced with /var/log/ceph/ceph-client.admin.68078.log.gz 2024-03-09T09:31:30.344 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63878.log.gz 2024-03-09T09:31:30.344 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72560.log 2024-03-09T09:31:30.346 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73294.log 2024-03-09T09:31:30.346 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72560.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46461.log.gz 2024-03-09T09:31:30.347 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.72560.log.gz 2024-03-09T09:31:30.347 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56115.log 2024-03-09T09:31:30.348 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69014.log 2024-03-09T09:31:30.349 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73294.log.gz 2024-03-09T09:31:30.350 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56115.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56115.log.gz 2024-03-09T09:31:30.350 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60954.log 2024-03-09T09:31:30.351 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74018.log 2024-03-09T09:31:30.351 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60954.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69014.log.gz 2024-03-09T09:31:30.352 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.60954.log.gz 2024-03-09T09:31:30.352 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78810.log 2024-03-09T09:31:30.353 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58262.log 2024-03-09T09:31:30.353 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78810.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74018.log.gz 2024-03-09T09:31:30.354 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78810.log.gz 2024-03-09T09:31:30.355 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45425.log 2024-03-09T09:31:30.355 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64200.log 2024-03-09T09:31:30.356 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58262.log.gz 2024-03-09T09:31:30.357 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58658.log 2024-03-09T09:31:30.357 INFO:teuthology.orchestra.run.smithi067.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.45425.log.gz 2024-03-09T09:31:30.358 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77494.log 2024-03-09T09:31:30.359 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58658.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64200.log.gz 2024-03-09T09:31:30.359 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.58658.log.gz 2024-03-09T09:31:30.360 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45491.log 2024-03-09T09:31:30.360 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42744.log 2024-03-09T09:31:30.361 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77494.log.gz 2024-03-09T09:31:30.362 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45491.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45491.log.gz 2024-03-09T09:31:30.362 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59288.log 2024-03-09T09:31:30.363 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75895.log 2024-03-09T09:31:30.363 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42744.log.gz 2024-03-09T09:31:30.364 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78180.log 2024-03-09T09:31:30.365 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59288.log.gz 2024-03-09T09:31:30.366 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70349.log 2024-03-09T09:31:30.366 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75895.log.gz 2024-03-09T09:31:30.367 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72063.log 2024-03-09T09:31:30.367 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78180.log.gz 2024-03-09T09:31:30.368 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65722.log 2024-03-09T09:31:30.368 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70349.log.gz 2024-03-09T09:31:30.370 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64112.log 2024-03-09T09:31:30.370 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72063.log.gz 2024-03-09T09:31:30.371 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78222.log 2024-03-09T09:31:30.371 INFO:teuthology.orchestra.run.smithi067.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.65722.log.gz 2024-03-09T09:31:30.372 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66167.log 2024-03-09T09:31:30.372 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64112.log.gz 2024-03-09T09:31:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67255.log 2024-03-09T09:31:30.374 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78222.log.gz 2024-03-09T09:31:30.374 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59074.log 2024-03-09T09:31:30.375 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66167.log.gz 2024-03-09T09:31:30.376 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42067.log 2024-03-09T09:31:30.376 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67255.log.gz 2024-03-09T09:31:30.377 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53322.log 2024-03-09T09:31:30.377 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59074.log.gz 2024-03-09T09:31:30.378 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72751.log 2024-03-09T09:31:30.379 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42067.log.gz 2024-03-09T09:31:30.380 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63493.log 2024-03-09T09:31:30.381 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53322.log.gz 2024-03-09T09:31:30.381 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69578.log 2024-03-09T09:31:30.381 INFO:teuthology.orchestra.run.smithi067.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.72751.log.gz 2024-03-09T09:31:30.383 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42025.log 2024-03-09T09:31:30.383 INFO:teuthology.orchestra.run.smithi067.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.63493.log.gz 2024-03-09T09:31:30.384 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49255.log 2024-03-09T09:31:30.386 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42025.log: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.69578.log.gz 2024-03-09T09:31:30.386 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42025.log.gz 2024-03-09T09:31:30.386 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71991.log 2024-03-09T09:31:30.387 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57866.log 2024-03-09T09:31:30.387 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49255.log.gz 2024-03-09T09:31:30.388 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46545.log 2024-03-09T09:31:30.388 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71991.log.gz 2024-03-09T09:31:30.388 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69872.log 2024-03-09T09:31:30.389 INFO:teuthology.orchestra.run.smithi067.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.57866.log.gz 2024-03-09T09:31:30.390 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77950.log 2024-03-09T09:31:30.390 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46545.log.gz 2024-03-09T09:31:30.391 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69872.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69872.log.gz 2024-03-09T09:31:30.391 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42280.log 2024-03-09T09:31:30.392 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64156.log 2024-03-09T09:31:30.392 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77950.log.gz 2024-03-09T09:31:30.393 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46755.log 2024-03-09T09:31:30.394 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42280.log.gz 2024-03-09T09:31:30.395 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64498.log 2024-03-09T09:31:30.396 INFO:teuthology.orchestra.run.smithi067.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.64156.log.gz 2024-03-09T09:31:30.396 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42828.log 2024-03-09T09:31:30.396 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46755.log.gz 2024-03-09T09:31:30.397 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63207.log 2024-03-09T09:31:30.397 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64498.log.gz 2024-03-09T09:31:30.398 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77410.log 2024-03-09T09:31:30.398 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42828.log.gz 2024-03-09T09:31:30.399 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66083.log 2024-03-09T09:31:30.400 INFO:teuthology.orchestra.run.smithi067.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.63207.log.gz 2024-03-09T09:31:30.400 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55629.log 2024-03-09T09:31:30.401 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77410.log.gz 2024-03-09T09:31:30.401 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71040.log 2024-03-09T09:31:30.402 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.66083.log.gz 2024-03-09T09:31:30.403 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64326.log 2024-03-09T09:31:30.403 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55629.log.gz 2024-03-09T09:31:30.404 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78642.log 2024-03-09T09:31:30.404 INFO:teuthology.orchestra.run.smithi067.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.71040.log.gz 2024-03-09T09:31:30.405 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76880.log 2024-03-09T09:31:30.405 INFO:teuthology.orchestra.run.smithi067.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64326.log.gz 2024-03-09T09:31:30.406 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50379.log 2024-03-09T09:31:30.407 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78642.log.gz 2024-03-09T09:31:30.408 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61657.log 2024-03-09T09:31:30.408 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76880.log.gz 2024-03-09T09:31:30.409 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50379.log: /var/log/ceph/ceph-client.admin.67979.log 2024-03-09T09:31:30.410 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61657.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50379.log.gz 2024-03-09T09:31:30.410 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.61657.log.gz 2024-03-09T09:31:30.411 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44837.log 2024-03-09T09:31:30.411 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67801.log 2024-03-09T09:31:30.412 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67979.log.gz 2024-03-09T09:31:30.413 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48963.log 2024-03-09T09:31:30.413 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44837.log.gz 2024-03-09T09:31:30.414 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60023.log 2024-03-09T09:31:30.414 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67801.log.gz 2024-03-09T09:31:30.415 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45215.log 2024-03-09T09:31:30.415 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48963.log.gz 2024-03-09T09:31:30.416 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47735.log 2024-03-09T09:31:30.416 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.60023.log.gz 2024-03-09T09:31:30.417 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68735.log 2024-03-09T09:31:30.418 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45215.log.gz 2024-03-09T09:31:30.419 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42576.log 2024-03-09T09:31:30.419 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47735.log.gz 2024-03-09T09:31:30.420 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59939.log 2024-03-09T09:31:30.421 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68735.log.gz 2024-03-09T09:31:30.422 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65186.log 2024-03-09T09:31:30.423 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42576.log.gz 2024-03-09T09:31:30.423 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51569.log 2024-03-09T09:31:30.424 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59939.log.gz 2024-03-09T09:31:30.424 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46503.log 2024-03-09T09:31:30.424 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65186.log.gz 2024-03-09T09:31:30.426 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77066.log 2024-03-09T09:31:30.426 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51569.log.gz 2024-03-09T09:31:30.427 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78054.log 2024-03-09T09:31:30.427 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46503.log.gz 2024-03-09T09:31:30.428 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65482.log 2024-03-09T09:31:30.428 INFO:teuthology.orchestra.run.smithi067.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.77066.log.gz 2024-03-09T09:31:30.429 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62346.log 2024-03-09T09:31:30.429 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78054.log.gz 2024-03-09T09:31:30.430 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50800.log 2024-03-09T09:31:30.430 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.65482.log.gz 2024-03-09T09:31:30.431 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62346.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-09T09:31:30.431 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62346.log.gz 2024-03-09T09:31:30.432 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72934.log 2024-03-09T09:31:30.433 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50800.log.gz 2024-03-09T09:31:30.434 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46587.log 2024-03-09T09:31:30.435 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72934.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.72934.log.gz 2024-03-09T09:31:30.435 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68657.log 2024-03-09T09:31:30.436 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68864.log 2024-03-09T09:31:30.437 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46587.log: /var/log/ceph/ceph-client.admin.68657.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46587.log.gz 2024-03-09T09:31:30.438 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69143.log 2024-03-09T09:31:30.438 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68657.log.gz 2024-03-09T09:31:30.439 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55942.log 2024-03-09T09:31:30.439 INFO:teuthology.orchestra.run.smithi067.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.68864.log.gz 2024-03-09T09:31:30.440 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69143.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.69143.log.gz 2024-03-09T09:31:30.440 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57908.log 2024-03-09T09:31:30.441 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53631.log 2024-03-09T09:31:30.442 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55942.log: /var/log/ceph/ceph-client.admin.57908.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55942.log.gz 2024-03-09T09:31:30.443 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68780.log 2024-03-09T09:31:30.443 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57908.log.gz 2024-03-09T09:31:30.444 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70934.log 2024-03-09T09:31:30.445 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53631.log.gz 2024-03-09T09:31:30.446 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68780.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68780.log.gz 2024-03-09T09:31:30.447 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53673.log 2024-03-09T09:31:30.447 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61481.log 2024-03-09T09:31:30.448 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70934.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70934.log.gz 2024-03-09T09:31:30.448 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42786.log 2024-03-09T09:31:30.448 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53673.log.gz 2024-03-09T09:31:30.449 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57225.log 2024-03-09T09:31:30.450 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61481.log.gz 2024-03-09T09:31:30.450 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42786.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42786.log.gz 2024-03-09T09:31:30.451 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67937.log 2024-03-09T09:31:30.451 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43181.log 2024-03-09T09:31:30.453 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67937.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57225.log.gz 2024-03-09T09:31:30.453 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69185.log 2024-03-09T09:31:30.453 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67937.log.gz 2024-03-09T09:31:30.454 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62304.log 2024-03-09T09:31:30.455 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43181.log.gz 2024-03-09T09:31:30.456 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79614.log 2024-03-09T09:31:30.456 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69185.log.gz 2024-03-09T09:31:30.457 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56931.log 2024-03-09T09:31:30.458 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62304.log.gz 2024-03-09T09:31:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79614.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79614.log.gz -5 2024-03-09T09:31:30.458 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61397.log 2024-03-09T09:31:30.459 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60169.log 2024-03-09T09:31:30.459 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56931.log.gz 2024-03-09T09:31:30.460 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61397.log: gzip -5 89.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.61397.log.gz 2024-03-09T09:31:30.460 INFO:teuthology.orchestra.run.smithi067.stderr: /var/log/ceph/ceph-client.admin.63409.log 2024-03-09T09:31:30.462 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76753.log 2024-03-09T09:31:30.462 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.60169.log.gz 2024-03-09T09:31:30.463 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63409.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63409.log.gz 2024-03-09T09:31:30.463 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73063.log 2024-03-09T09:31:30.468 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69654.log 2024-03-09T09:31:30.468 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76753.log.gz 2024-03-09T09:31:30.469 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54320.log 2024-03-09T09:31:30.469 INFO:teuthology.orchestra.run.smithi067.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.73063.log.gz 2024-03-09T09:31:30.469 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79091.log 2024-03-09T09:31:30.469 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69654.log.gz 2024-03-09T09:31:30.469 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77024.log 2024-03-09T09:31:30.469 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54320.log.gz 2024-03-09T09:31:30.469 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53487.log 2024-03-09T09:31:30.470 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79091.log.gz 2024-03-09T09:31:30.470 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77024.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.77024.log.gz -5 2024-03-09T09:31:30.470 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.70670.log 2024-03-09T09:31:30.471 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50439.log 2024-03-09T09:31:30.471 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53487.log.gz 2024-03-09T09:31:30.472 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45047.log 2024-03-09T09:31:30.472 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70670.log.gz 2024-03-09T09:31:30.473 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53799.log 2024-03-09T09:31:30.473 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50439.log.gz 2024-03-09T09:31:30.474 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44384.log 2024-03-09T09:31:30.475 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45047.log.gz 2024-03-09T09:31:30.476 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59609.log 2024-03-09T09:31:30.476 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53799.log.gz 2024-03-09T09:31:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44384.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44384.log.gz 2024-03-09T09:31:30.477 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65228.log 2024-03-09T09:31:30.478 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65139.log 2024-03-09T09:31:30.478 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59609.log.gz 2024-03-09T09:31:30.479 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72518.log 2024-03-09T09:31:30.480 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65228.log.gz 2024-03-09T09:31:30.480 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65013.log 2024-03-09T09:31:30.481 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65139.log.gz 2024-03-09T09:31:30.482 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73663.log 2024-03-09T09:31:30.482 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72518.log.gz 2024-03-09T09:31:30.483 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62430.log 2024-03-09T09:31:30.483 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65013.log.gz 2024-03-09T09:31:30.484 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69956.log 2024-03-09T09:31:30.485 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73663.log.gz 2024-03-09T09:31:30.485 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71705.log 2024-03-09T09:31:30.486 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.62430.log.gz 2024-03-09T09:31:30.487 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76423.log 2024-03-09T09:31:30.487 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.69956.log.gz 2024-03-09T09:31:30.488 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44669.log 2024-03-09T09:31:30.488 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71705.log.gz 2024-03-09T09:31:30.489 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43517.log 2024-03-09T09:31:30.489 INFO:teuthology.orchestra.run.smithi067.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.76423.log.gz 2024-03-09T09:31:30.491 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46671.log 2024-03-09T09:31:30.491 INFO:teuthology.orchestra.run.smithi067.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44669.log.gz 2024-03-09T09:31:30.492 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48864.log 2024-03-09T09:31:30.492 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43517.log.gz 2024-03-09T09:31:30.493 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53884.log 2024-03-09T09:31:30.493 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46671.log.gz 2024-03-09T09:31:30.495 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62595.log 2024-03-09T09:31:30.495 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53884.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.48864.log.gz 2024-03-09T09:31:30.496 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79469.log 2024-03-09T09:31:30.496 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53884.log.gz 2024-03-09T09:31:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62595.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62595.log.gz 2024-03-09T09:31:30.497 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70544.log 2024-03-09T09:31:30.498 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70086.log 2024-03-09T09:31:30.499 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79469.log.gz 2024-03-09T09:31:30.499 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54236.log 2024-03-09T09:31:30.500 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.70544.log.gz 2024-03-09T09:31:30.501 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65097.log 2024-03-09T09:31:30.501 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70086.log.gz 2024-03-09T09:31:30.502 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56661.log 2024-03-09T09:31:30.502 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65097.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54236.log.gz 2024-03-09T09:31:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75707.log 2024-03-09T09:31:30.504 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.65097.log.gz 2024-03-09T09:31:30.504 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56661.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.56661.log.gz 2024-03-09T09:31:30.504 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.76255.log 2024-03-09T09:31:30.505 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59653.log 2024-03-09T09:31:30.506 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75707.log.gz 2024-03-09T09:31:30.507 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74144.log 2024-03-09T09:31:30.507 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76255.log.gz 2024-03-09T09:31:30.508 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59653.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59653.log.gz 2024-03-09T09:31:30.508 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67170.log 2024-03-09T09:31:30.509 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75769.log 2024-03-09T09:31:30.509 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74144.log.gz 2024-03-09T09:31:30.510 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43265.log 2024-03-09T09:31:30.510 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67170.log.gz 2024-03-09T09:31:30.511 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75769.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75769.log.gz 2024-03-09T09:31:30.511 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72476.log 2024-03-09T09:31:30.512 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73747.log 2024-03-09T09:31:30.513 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43265.log.gz 2024-03-09T09:31:30.514 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56829.log 2024-03-09T09:31:30.514 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72476.log.gz 2024-03-09T09:31:30.515 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67086.log 2024-03-09T09:31:30.515 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73747.log.gz 2024-03-09T09:31:30.516 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60574.log 2024-03-09T09:31:30.516 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56829.log.gz 2024-03-09T09:31:30.517 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61439.log 2024-03-09T09:31:30.517 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67086.log.gz 2024-03-09T09:31:30.518 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60574.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60574.log.gz 2024-03-09T09:31:30.518 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61313.log 2024-03-09T09:31:30.519 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57638.log 2024-03-09T09:31:30.520 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61313.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61439.log.gz 2024-03-09T09:31:30.520 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61313.log.gz 2024-03-09T09:31:30.521 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75131.log 2024-03-09T09:31:30.522 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59853.log 2024-03-09T09:31:30.522 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.57638.log.gz 2024-03-09T09:31:30.523 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52235.log 2024-03-09T09:31:30.523 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75131.log.gz 2024-03-09T09:31:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68822.log 2024-03-09T09:31:30.525 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59853.log.gz 2024-03-09T09:31:30.526 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68615.log 2024-03-09T09:31:30.526 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52235.log.gz 2024-03-09T09:31:30.527 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68822.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68822.log.gz 2024-03-09T09:31:30.527 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59699.log 2024-03-09T09:31:30.528 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44627.log 2024-03-09T09:31:30.529 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.68615.log.gz 2024-03-09T09:31:30.530 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59699.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59699.log.gz 2024-03-09T09:31:30.530 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50908.log 2024-03-09T09:31:30.530 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60845.log 2024-03-09T09:31:30.532 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44627.log.gz 2024-03-09T09:31:30.532 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53445.log 2024-03-09T09:31:30.532 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50908.log.gz 2024-03-09T09:31:30.532 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64540.log 2024-03-09T09:31:30.534 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60845.log.gz 2024-03-09T09:31:30.534 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43349.log 2024-03-09T09:31:30.534 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53445.log.gz 2024-03-09T09:31:30.535 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64540.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-09T09:31:30.535 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64540.log.gz 2024-03-09T09:31:30.536 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43349.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43349.log.gz 2024-03-09T09:31:30.536 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56203.log 2024-03-09T09:31:30.537 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73934.log 2024-03-09T09:31:30.538 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56203.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56203.log.gz 2024-03-09T09:31:30.538 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67128.log 2024-03-09T09:31:30.539 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64843.log 2024-03-09T09:31:30.539 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73934.log.gz 2024-03-09T09:31:30.540 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67128.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67128.log.gzgzip 2024-03-09T09:31:30.540 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61615.log 2024-03-09T09:31:30.541 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43741.log 2024-03-09T09:31:30.542 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64843.log.gz 2024-03-09T09:31:30.542 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56787.log 2024-03-09T09:31:30.543 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.61615.log.gz 2024-03-09T09:31:30.543 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60634.log 2024-03-09T09:31:30.544 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43741.log.gz 2024-03-09T09:31:30.545 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59160.log 2024-03-09T09:31:30.545 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.56787.log.gz 2024-03-09T09:31:30.546 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47464.log 2024-03-09T09:31:30.546 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60634.log.gz 2024-03-09T09:31:30.547 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60211.log 2024-03-09T09:31:30.548 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47464.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.59160.log.gz 2024-03-09T09:31:30.548 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47464.log.gz 2024-03-09T09:31:30.549 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66446.log 2024-03-09T09:31:30.550 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66337.log 2024-03-09T09:31:30.550 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.60211.log.gz 2024-03-09T09:31:30.551 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71860.log 2024-03-09T09:31:30.551 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66446.log.gz 2024-03-09T09:31:30.552 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44753.log 2024-03-09T09:31:30.552 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66337.log.gz 2024-03-09T09:31:30.553 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52109.log 2024-03-09T09:31:30.553 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71860.log.gz 2024-03-09T09:31:30.554 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69362.log 2024-03-09T09:31:30.555 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44753.log.gz 2024-03-09T09:31:30.556 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66125.log 2024-03-09T09:31:30.556 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52109.log.gz 2024-03-09T09:31:30.556 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69362.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69362.log.gz -5 2024-03-09T09:31:30.557 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.69272.log 2024-03-09T09:31:30.558 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43433.log 2024-03-09T09:31:30.558 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66125.log.gz 2024-03-09T09:31:30.559 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79385.log 2024-03-09T09:31:30.559 INFO:teuthology.orchestra.run.smithi067.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.69272.log.gz 2024-03-09T09:31:30.560 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72434.log 2024-03-09T09:31:30.561 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43433.log.gz 2024-03-09T09:31:30.561 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45743.log 2024-03-09T09:31:30.562 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79385.log.gz 2024-03-09T09:31:30.563 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71526.log 2024-03-09T09:31:30.563 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72434.log.gz 2024-03-09T09:31:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45743.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45743.log.gz 2024-03-09T09:31:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31028.log 2024-03-09T09:31:30.565 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63059.log 2024-03-09T09:31:30.566 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71526.log.gz 2024-03-09T09:31:30.566 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.tmp-client.admin.31028.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31028.log.gz 2024-03-09T09:31:30.566 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72709.log 2024-03-09T09:31:30.567 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55362.log 2024-03-09T09:31:30.568 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63059.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63059.log.gz 2024-03-09T09:31:30.568 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72709.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.66829.log -- replaced with /var/log/ceph/ceph-client.admin.72709.log.gz 2024-03-09T09:31:30.568 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-03-09T09:31:30.569 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70804.log 2024-03-09T09:31:30.570 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55362.log.gz 2024-03-09T09:31:30.570 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66829.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.66829.log.gzgzip 2024-03-09T09:31:30.571 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.70498.log 2024-03-09T09:31:30.572 INFO:teuthology.orchestra.run.smithi067.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-09T09:31:30.572 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67213.log 2024-03-09T09:31:30.573 INFO:teuthology.orchestra.run.smithi067.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.70804.log.gz 2024-03-09T09:31:30.573 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73196.log 2024-03-09T09:31:30.573 INFO:teuthology.orchestra.run.smithi067.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.70498.log.gz 2024-03-09T09:31:30.574 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58472.log 2024-03-09T09:31:30.575 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67213.log.gz 2024-03-09T09:31:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73196.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.73196.log.gz 2024-03-09T09:31:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44711.log 2024-03-09T09:31:30.577 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64885.log 2024-03-09T09:31:30.577 INFO:teuthology.orchestra.run.smithi067.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58472.log.gz 2024-03-09T09:31:30.578 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44711.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.44711.log.gz -5 2024-03-09T09:31:30.578 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.70132.log 2024-03-09T09:31:30.579 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57183.log 2024-03-09T09:31:30.579 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64885.log.gz 2024-03-09T09:31:30.581 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70132.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70132.log.gz 2024-03-09T09:31:30.581 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48322.log 2024-03-09T09:31:30.582 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74444.log 2024-03-09T09:31:30.582 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57183.log.gz 2024-03-09T09:31:30.583 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43739.log 2024-03-09T09:31:30.583 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48322.log.gz 2024-03-09T09:31:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77192.log 2024-03-09T09:31:30.584 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74444.log.gz 2024-03-09T09:31:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43739.log: gzip -5 87.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43739.log.gz 2024-03-09T09:31:30.585 INFO:teuthology.orchestra.run.smithi067.stderr: /var/log/ceph/ceph-client.admin.59392.log 2024-03-09T09:31:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45299.log 2024-03-09T09:31:30.586 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77192.log.gz 2024-03-09T09:31:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59392.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59392.log.gz -5 2024-03-09T09:31:30.587 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.64414.log 2024-03-09T09:31:30.587 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68162.log 2024-03-09T09:31:30.588 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45299.log.gz 2024-03-09T09:31:30.588 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64414.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64414.log.gz 2024-03-09T09:31:30.589 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55690.log 2024-03-09T09:31:30.589 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56745.log 2024-03-09T09:31:30.590 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.68162.log.gz 2024-03-09T09:31:30.591 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55690.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55690.log.gz 2024-03-09T09:31:30.591 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56287.log 2024-03-09T09:31:30.592 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42912.log 2024-03-09T09:31:30.592 INFO:teuthology.orchestra.run.smithi067.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56745.log.gz 2024-03-09T09:31:30.593 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68483.log 2024-03-09T09:31:30.593 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56287.log.gz 2024-03-09T09:31:30.594 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66209.log 2024-03-09T09:31:30.594 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42912.log.gz 2024-03-09T09:31:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58346.log 2024-03-09T09:31:30.595 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68483.log.gz 2024-03-09T09:31:30.596 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.66209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59434.log 2024-03-09T09:31:30.597 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66209.log.gz 2024-03-09T09:31:30.598 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69822.log 2024-03-09T09:31:30.598 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58346.log.gz 2024-03-09T09:31:30.599 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78726.log 2024-03-09T09:31:30.599 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59434.log.gz 2024-03-09T09:31:30.600 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71442.log 2024-03-09T09:31:30.600 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69822.log.gz 2024-03-09T09:31:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77620.log 2024-03-09T09:31:30.601 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78726.log.gz 2024-03-09T09:31:30.602 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62217.log 2024-03-09T09:31:30.602 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.71442.log.gz 2024-03-09T09:31:30.603 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68312.log 2024-03-09T09:31:30.604 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62217.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77620.log.gz 2024-03-09T09:31:30.605 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71484.log 2024-03-09T09:31:30.605 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.62217.log.gz 2024-03-09T09:31:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68312.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68312.log.gz 2024-03-09T09:31:30.606 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68354.log 2024-03-09T09:31:30.607 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79301.log 2024-03-09T09:31:30.607 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68354.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71484.log.gz 2024-03-09T09:31:30.608 INFO:teuthology.orchestra.run.smithi067.stderr:gzip 89.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.68354.log.gz --verbose 2024-03-09T09:31:30.608 INFO:teuthology.orchestra.run.smithi067.stderr: -- /var/log/ceph/ceph-client.admin.50279.log 2024-03-09T09:31:30.609 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79301.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.79301.log.gz 2024-03-09T09:31:30.609 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62023.log 2024-03-09T09:31:30.610 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65955.log 2024-03-09T09:31:30.611 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62023.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.50279.log.gz 2024-03-09T09:31:30.612 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62023.log.gz 2024-03-09T09:31:30.612 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61170.log 2024-03-09T09:31:30.613 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65955.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-09T09:31:30.613 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65955.log.gz 2024-03-09T09:31:30.614 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54821.log 2024-03-09T09:31:30.614 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61170.log.gz 2024-03-09T09:31:30.615 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57267.log 2024-03-09T09:31:30.616 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68270.log 2024-03-09T09:31:30.616 INFO:teuthology.orchestra.run.smithi067.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54821.log.gz 2024-03-09T09:31:30.617 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57553.log 2024-03-09T09:31:30.618 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57267.log.gz 2024-03-09T09:31:30.618 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65616.log 2024-03-09T09:31:30.619 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68270.log.gz 2024-03-09T09:31:30.620 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47295.log 2024-03-09T09:31:30.620 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57553.log.gz 2024-03-09T09:31:30.621 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57992.log 2024-03-09T09:31:30.621 INFO:teuthology.orchestra.run.smithi067.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65616.log.gz 2024-03-09T09:31:30.622 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57309.log 2024-03-09T09:31:30.623 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7%/var/log/ceph/ceph-client.admin.57992.log: -- replaced with /var/log/ceph/ceph-client.admin.47295.log.gz 2024-03-09T09:31:30.623 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57680.log 2024-03-09T09:31:30.624 INFO:teuthology.orchestra.run.smithi067.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57992.log.gz 2024-03-09T09:31:30.624 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57309.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.57309.log.gz 2024-03-09T09:31:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67759.log 2024-03-09T09:31:30.625 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78558.log 2024-03-09T09:31:30.626 INFO:teuthology.orchestra.run.smithi067.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57680.log.gz 2024-03-09T09:31:30.627 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49947.log 2024-03-09T09:31:30.627 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67759.log.gz 2024-03-09T09:31:30.628 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58514.log 2024-03-09T09:31:30.628 INFO:teuthology.orchestra.run.smithi067.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78558.log.gz 2024-03-09T09:31:30.629 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71082.log 2024-03-09T09:31:30.630 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49947.log.gz 2024-03-09T09:31:30.630 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50323.log 2024-03-09T09:31:30.631 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58514.log.gz 2024-03-09T09:31:30.631 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.71082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63293.log 2024-03-09T09:31:30.632 INFO:teuthology.orchestra.run.smithi067.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.71082.log.gz 2024-03-09T09:31:30.633 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46987.log 2024-03-09T09:31:30.633 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50323.log.gz 2024-03-09T09:31:30.634 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60677.log 2024-03-09T09:31:30.634 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63293.log.gz 2024-03-09T09:31:30.635 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76838.log 2024-03-09T09:31:30.635 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46987.log.gz 2024-03-09T09:31:30.636 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57388.log 2024-03-09T09:31:30.636 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60677.log.gz 2024-03-09T09:31:30.637 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37389.log 2024-03-09T09:31:30.638 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76838.log.gz 2024-03-09T09:31:30.638 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54486.log 2024-03-09T09:31:30.639 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57388.log.gz 2024-03-09T09:31:30.639 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.37389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53715.log 2024-03-09T09:31:30.640 INFO:teuthology.orchestra.run.smithi067.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37389.log.gz 2024-03-09T09:31:30.641 INFO:teuthology.orchestra.run.smithi134.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-09T09:31:30.641 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57015.log 2024-03-09T09:31:30.642 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54486.log.gz 2024-03-09T09:31:30.642 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64797.log 2024-03-09T09:31:30.643 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53715.log.gz 2024-03-09T09:31:30.643 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60253.log 2024-03-09T09:31:30.644 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.57015.log.gz 2024-03-09T09:31:30.644 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61889.log 2024-03-09T09:31:30.645 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64797.log.gz 2024-03-09T09:31:30.645 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79133.log 2024-03-09T09:31:30.646 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60253.log.gz 2024-03-09T09:31:30.647 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69696.log 2024-03-09T09:31:30.647 INFO:teuthology.orchestra.run.smithi067.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.61889.log.gz 2024-03-09T09:31:30.648 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78096.log 2024-03-09T09:31:30.648 INFO:teuthology.orchestra.run.smithi067.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79133.log.gz 2024-03-09T09:31:30.649 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58872.log 2024-03-09T09:31:30.649 INFO:teuthology.orchestra.run.smithi067.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.69696.log.gz 2024-03-09T09:31:30.650 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.78096.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54777.log 2024-03-09T09:31:30.650 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78096.log.gz 2024-03-09T09:31:30.651 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51695.log 2024-03-09T09:31:30.652 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54777.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.58872.log.gz 2024-03-09T09:31:30.652 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-09T09:31:30.653 INFO:teuthology.orchestra.run.smithi067.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.54777.log.gz 2024-03-09T09:31:30.654 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.51695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79217.log 2024-03-09T09:31:30.654 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51695.log.gz 2024-03-09T09:31:30.655 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64242.log 2024-03-09T09:31:30.656 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79217.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79217.log.gz 2024-03-09T09:31:30.657 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54011.log 2024-03-09T09:31:30.657 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64242.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64242.log.gz 2024-03-09T09:31:30.658 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57099.log 2024-03-09T09:31:30.659 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69056.log 2024-03-09T09:31:30.659 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54011.log.gz 2024-03-09T09:31:30.660 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69451.log 2024-03-09T09:31:30.660 INFO:teuthology.orchestra.run.smithi067.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57099.log.gz 2024-03-09T09:31:30.661 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69056.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.69056.log.gz 2024-03-09T09:31:30.662 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79009.log 2024-03-09T09:31:30.663 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77536.log 2024-03-09T09:31:30.663 INFO:teuthology.orchestra.run.smithi067.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69451.log.gz 2024-03-09T09:31:30.664 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62500.log 2024-03-09T09:31:30.664 INFO:teuthology.orchestra.run.smithi067.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79009.log.gz 2024-03-09T09:31:30.665 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77536.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77536.log.gz 2024-03-09T09:31:30.666 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72840.log 2024-03-09T09:31:30.666 INFO:teuthology.orchestra.run.smithi067.stderr: 87.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-09T09:31:30.667 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62500.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-09T09:31:30.667 INFO:teuthology.orchestra.run.smithi067.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.62500.log.gz 2024-03-09T09:31:30.668 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42618.log 2024-03-09T09:31:30.668 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72840.log.gz 2024-03-09T09:31:30.669 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61847.log 2024-03-09T09:31:30.670 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42618.log: gzip -5 88.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.42618.log.gz -- 2024-03-09T09:31:30.670 INFO:teuthology.orchestra.run.smithi067.stderr: /var/log/ceph/ceph-client.admin.60912.log 2024-03-09T09:31:30.671 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61847.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-09T09:31:30.672 INFO:teuthology.orchestra.run.smithi067.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61847.log.gz 2024-03-09T09:31:30.673 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42194.log 2024-03-09T09:31:30.673 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60912.log.gz 2024-03-09T09:31:30.674 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56031.log 2024-03-09T09:31:30.676 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42194.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42194.log.gz 2024-03-09T09:31:30.676 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46629.log 2024-03-09T09:31:30.678 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56031.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56031.log.gz 2024-03-09T09:31:30.678 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63707.log 2024-03-09T09:31:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46629.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46629.log.gz 2024-03-09T09:31:30.686 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48609.log 2024-03-09T09:31:30.686 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63707.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.63707.log.gz 2024-03-09T09:31:30.686 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48609.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48609.log.gz 2024-03-09T09:31:30.686 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63982.log 2024-03-09T09:31:30.686 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55050.log 2024-03-09T09:31:30.687 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63982.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63982.log.gz 2024-03-09T09:31:30.688 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55050.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.55050.log.gz 2024-03-09T09:31:30.688 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65312.log 2024-03-09T09:31:30.689 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73504.log 2024-03-09T09:31:30.690 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65312.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.65312.log.gz 2024-03-09T09:31:30.692 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73154.log 2024-03-09T09:31:30.693 INFO:teuthology.orchestra.run.smithi067.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73504.log.gz 2024-03-09T09:31:30.693 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47777.log 2024-03-09T09:31:30.695 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73154.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73154.log.gz 2024-03-09T09:31:30.695 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55488.log 2024-03-09T09:31:30.697 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47777.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47777.log.gz 2024-03-09T09:31:30.698 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68948.log 2024-03-09T09:31:30.698 INFO:teuthology.orchestra.run.smithi067.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55488.log.gz 2024-03-09T09:31:30.699 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68948.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68948.log.gz 2024-03-09T09:31:30.700 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42998.log 2024-03-09T09:31:30.702 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76381.log 2024-03-09T09:31:30.703 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42998.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42998.log.gz 2024-03-09T09:31:30.704 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76381.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.76381.log.gz 2024-03-09T09:31:30.705 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48364.log 2024-03-09T09:31:30.708 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48364.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.48364.log.gz 2024-03-09T09:31:30.708 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60761.log 2024-03-09T09:31:30.710 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48219.log 2024-03-09T09:31:30.711 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60761.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.60761.log.gz 2024-03-09T09:31:30.712 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48219.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48219.log.gz 2024-03-09T09:31:30.713 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76651.log 2024-03-09T09:31:30.714 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63791.log 2024-03-09T09:31:30.716 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76651.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.76651.log.gz 2024-03-09T09:31:30.717 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63791.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63791.log.gz 2024-03-09T09:31:30.718 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54278.log 2024-03-09T09:31:30.719 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44076.log 2024-03-09T09:31:30.720 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.54278.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54278.log.gz 2024-03-09T09:31:30.722 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44076.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44076.log.gz 2024-03-09T09:31:30.723 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63451.log 2024-03-09T09:31:30.724 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49903.log 2024-03-09T09:31:30.725 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63451.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63451.log.gz 2024-03-09T09:31:30.726 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49903.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49903.log.gz 2024-03-09T09:31:30.727 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68906.log 2024-03-09T09:31:30.729 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57824.log 2024-03-09T09:31:30.730 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68906.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.68906.log.gz 2024-03-09T09:31:30.731 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57824.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57824.log.gz 2024-03-09T09:31:30.732 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48805.log 2024-03-09T09:31:30.734 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48805.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48805.log.gz 2024-03-09T09:31:30.735 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44059.log 2024-03-09T09:31:30.736 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59032.log 2024-03-09T09:31:30.738 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44059.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44059.log.gz 2024-03-09T09:31:30.739 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59032.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59032.log.gz 2024-03-09T09:31:30.740 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68036.log 2024-03-09T09:31:30.741 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45701.log 2024-03-09T09:31:30.743 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68036.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68036.log.gz 2024-03-09T09:31:30.744 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.45701.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45701.log.gz 2024-03-09T09:31:30.744 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77578.log 2024-03-09T09:31:30.746 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55732.log 2024-03-09T09:31:30.747 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77578.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77578.log.gz 2024-03-09T09:31:30.748 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.55732.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55732.log.gz 2024-03-09T09:31:30.749 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77150.log 2024-03-09T09:31:30.750 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73112.log 2024-03-09T09:31:30.752 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77150.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77150.log.gz 2024-03-09T09:31:30.753 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/ceph-client.admin.73112.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63623.log 2024-03-09T09:31:30.755 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65438.log 2024-03-09T09:31:30.755 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73112.log.gz 2024-03-09T09:31:30.756 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63623.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63623.log.gz 2024-03-09T09:31:30.757 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62553.log 2024-03-09T09:31:30.758 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65438.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65438.log.gz 2024-03-09T09:31:30.759 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58917.log 2024-03-09T09:31:30.760 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62553.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.62553.log.gz 2024-03-09T09:31:30.761 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58917.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.58917.log.gz 2024-03-09T09:31:30.762 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59811.log 2024-03-09T09:31:30.763 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63365.log 2024-03-09T09:31:30.764 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.59811.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59811.log.gz 2024-03-09T09:31:30.766 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70192.log 2024-03-09T09:31:30.768 INFO:teuthology.orchestra.run.smithi067.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79049.log 2024-03-09T09:31:30.768 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63365.log.gz 2024-03-09T09:31:30.769 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44429.log 2024-03-09T09:31:30.771 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70192.log: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.70192.log.gz 2024-03-09T09:31:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.79049.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79049.log.gz 2024-03-09T09:31:30.775 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58118.log 2024-03-09T09:31:30.776 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.44429.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44429.log.gz 2024-03-09T09:31:30.776 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43733.log 2024-03-09T09:31:30.791 INFO:teuthology.orchestra.run.smithi067.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58118.log.gz 2024-03-09T09:31:30.791 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48093.log 2024-03-09T09:31:30.792 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43733.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43733.log.gz 2024-03-09T09:31:30.792 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48093.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48093.log.gz 2024-03-09T09:31:30.792 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42660.log 2024-03-09T09:31:30.792 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77782.log 2024-03-09T09:31:30.793 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42660.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42660.log.gz 2024-03-09T09:31:30.793 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.77782.log: /var/log/ceph/ceph-client.admin.62673.log 2024-03-09T09:31:30.793 INFO:teuthology.orchestra.run.smithi067.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77782.log.gz 2024-03-09T09:31:30.793 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52361.log 2024-03-09T09:31:30.793 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62673.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62673.log.gz 2024-03-09T09:31:30.794 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61038.log 2024-03-09T09:31:30.794 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.52361.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52361.log.gz 2024-03-09T09:31:30.794 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64970.log 2024-03-09T09:31:30.794 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61038.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.61038.log.gz 2024-03-09T09:31:30.795 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60467.log 2024-03-09T09:31:30.815 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.64970.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64970.log.gz 2024-03-09T09:31:30.815 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65270.log 2024-03-09T09:31:30.815 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60467.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60467.log.gz 2024-03-09T09:31:30.816 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65270.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.65270.log.gz 2024-03-09T09:31:30.816 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67895.log 2024-03-09T09:31:30.816 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72340.log 2024-03-09T09:31:30.817 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67895.log: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.67895.log.gz 2024-03-09T09:31:30.817 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56496.log 2024-03-09T09:31:30.817 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72340.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.72340.log.gz 2024-03-09T09:31:30.817 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58160.log 2024-03-09T09:31:30.818 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56496.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56496.log.gz 2024-03-09T09:31:30.819 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70846.log 2024-03-09T09:31:30.820 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.58160.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58160.log.gz 2024-03-09T09:31:30.820 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56577.log 2024-03-09T09:31:30.821 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70846.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.70846.log.gz 2024-03-09T09:31:30.821 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49991.log 2024-03-09T09:31:30.821 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56577.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56577.log.gz 2024-03-09T09:31:30.822 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69914.log 2024-03-09T09:31:30.822 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.49991.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.49991.log.gz 2024-03-09T09:31:30.822 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62844.log 2024-03-09T09:31:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.69914.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.69914.log.gz 2024-03-09T09:31:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76339.log 2024-03-09T09:31:30.823 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.62844.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62844.log.gz 2024-03-09T09:31:30.887 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.76339.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76339.log.gz 2024-03-09T09:31:30.887 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73238.log 2024-03-09T09:31:30.888 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48919.log 2024-03-09T09:31:30.888 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73238.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73238.log.gz 2024-03-09T09:31:30.888 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63665.log 2024-03-09T09:31:30.888 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48919.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.48919.log.gz 2024-03-09T09:31:30.888 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61699.log 2024-03-09T09:31:30.888 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.63665.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.63665.log.gz 2024-03-09T09:31:30.889 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61699.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61699.log.gz 2024-03-09T09:31:30.889 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72798.log 2024-03-09T09:31:30.889 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50952.log 2024-03-09T09:31:30.889 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.72798.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72798.log.gz 2024-03-09T09:31:30.889 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75979.log 2024-03-09T09:31:30.889 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.50952.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50952.log.gz 2024-03-09T09:31:30.889 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42870.log 2024-03-09T09:31:30.889 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.75979.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75979.log.gz 2024-03-09T09:31:30.890 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42870.log: /var/log/ceph/ceph-client.admin.43559.log 2024-03-09T09:31:30.890 INFO:teuthology.orchestra.run.smithi067.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42870.log.gz 2024-03-09T09:31:30.890 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77824.log 2024-03-09T09:31:30.890 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43559.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43559.log.gz 2024-03-09T09:31:30.890 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73402.log 2024-03-09T09:31:30.891 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.77824.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77824.log.gz 2024-03-09T09:31:30.891 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67297.log 2024-03-09T09:31:30.891 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.73402.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73402.log.gz 2024-03-09T09:31:30.891 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58202.log 2024-03-09T09:31:30.891 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.67297.log: 91.1%/var/log/ceph/ceph-client.admin.58202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65568.log 2024-03-09T09:31:30.892 INFO:teuthology.orchestra.run.smithi067.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58202.log.gz 2024-03-09T09:31:30.892 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67297.log.gz 2024-03-09T09:31:30.892 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43740.log 2024-03-09T09:31:30.893 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.65568.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65568.log.gz 2024-03-09T09:31:30.893 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43223.log 2024-03-09T09:31:30.893 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43740.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43740.log.gz 2024-03-09T09:31:30.894 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-09T09:31:30.894 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43223.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43223.log.gz 2024-03-09T09:31:30.894 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45533.log 2024-03-09T09:31:30.894 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.45533.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45533.log.gz 2024-03-09T09:31:30.895 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53403.log 2024-03-09T09:31:30.895 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.53403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74372.log 2024-03-09T09:31:30.895 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53403.log.gz 2024-03-09T09:31:30.895 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46798.log 2024-03-09T09:31:30.895 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.74372.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74372.log.gz 2024-03-09T09:31:30.895 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.46798.log: 93.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-09T09:31:30.895 INFO:teuthology.orchestra.run.smithi067.stderr:gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46798.log.gz 2024-03-09T09:31:30.896 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43722.log 2024-03-09T09:31:30.896 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68438.log 2024-03-09T09:31:30.896 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.43722.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43722.log.gz 2024-03-09T09:31:30.896 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73621.log 2024-03-09T09:31:30.896 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.68438.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68438.log.gz 2024-03-09T09:31:30.896 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.73621.log: -- /var/log/ceph/ceph-client.admin.61531.log 2024-03-09T09:31:30.897 INFO:teuthology.orchestra.run.smithi067.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73621.log.gz 2024-03-09T09:31:30.897 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75193.log 2024-03-09T09:31:30.897 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61531.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61531.log.gz 2024-03-09T09:31:30.897 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60065.log/var/log/ceph/ceph-client.admin.75193.log: 2024-03-09T09:31:30.897 INFO:teuthology.orchestra.run.smithi067.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75193.log.gz 2024-03-09T09:31:30.897 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42702.log 2024-03-09T09:31:30.897 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.60065.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60065.log.gz 2024-03-09T09:31:30.898 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48406.log 2024-03-09T09:31:30.898 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.42702.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42702.log.gz 2024-03-09T09:31:30.898 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56245.log 2024-03-09T09:31:30.898 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.48406.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.48406.log.gz 2024-03-09T09:31:30.898 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.56245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47071.log 2024-03-09T09:31:30.899 INFO:teuthology.orchestra.run.smithi067.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56245.log.gz 2024-03-09T09:31:30.899 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70762.log 2024-03-09T09:31:30.899 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.47071.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47071.log.gz 2024-03-09T09:31:30.899 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72256.log 2024-03-09T09:31:30.899 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.70762.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70762.log.gz 2024-03-09T09:31:30.900 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.72256.log: /var/log/ceph/ceph-client.admin.61783.log 2024-03-09T09:31:30.900 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.72256.log.gz 2024-03-09T09:31:30.900 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58700.log 2024-03-09T09:31:30.901 INFO:teuthology.orchestra.run.smithi134.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-09T09:31:30.903 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.61783.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61783.log.gz 2024-03-09T09:31:30.903 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.58700.log: /var/log/ceph/ceph-client.admin.57740.log 2024-03-09T09:31:30.904 INFO:teuthology.orchestra.run.smithi067.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.58700.log.gz 2024-03-09T09:31:30.904 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.57740.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57740.log.gz 2024-03-09T09:31:31.514 INFO:teuthology.orchestra.run.smithi067.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-09T09:31:40.855 INFO:teuthology.orchestra.run.smithi134.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-09T09:31:44.114 INFO:teuthology.orchestra.run.smithi067.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-09T09:31:44.980 INFO:teuthology.orchestra.run.smithi134.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-09T09:31:46.197 INFO:teuthology.orchestra.run.smithi134.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-09T09:31:46.363 INFO:teuthology.orchestra.run.smithi067.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-09T09:31:46.486 INFO:teuthology.orchestra.run.smithi134.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-09T09:31:46.489 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-03-09T09:31:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m16.641s 2024-03-09T09:31:46.489 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m56.168s 2024-03-09T09:31:46.490 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m4.791s 2024-03-09T09:31:48.159 INFO:teuthology.orchestra.run.smithi067.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-09T09:31:49.061 INFO:teuthology.orchestra.run.smithi067.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-09T09:32:46.473 INFO:teuthology.orchestra.run.smithi067.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-09T09:32:46.477 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-03-09T09:32:46.477 INFO:teuthology.orchestra.run.smithi067.stderr:real 1m16.628s 2024-03-09T09:32:46.477 INFO:teuthology.orchestra.run.smithi067.stderr:user 2m12.785s 2024-03-09T09:32:46.477 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m11.941s 2024-03-09T09:32:46.479 INFO:tasks.ceph:Archiving logs... 2024-03-09T09:32:46.479 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387/remote/smithi067/log 2024-03-09T09:32:46.480 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-09T09:33:26.677 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387/remote/smithi134/log 2024-03-09T09:33:26.678 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-09T09:34:04.120 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-09T09:34:04.130 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-09T09:34:04.131 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-09T09:34:04.139 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-09T09:34:04.292 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-09T09:34:04.302 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-09T09:34:04.302 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-09T09:34:04.311 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-09T09:34:04.359 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-03-09T09:34:04.364 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-03-09T09:34:04.369 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-09T09:34:04.369 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-03-09T09:34:04.369 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +2230us[+2185us] +/- 90ms 2024-03-09T09:34:04.370 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 +511us[ +466us] +/- 70ms 2024-03-09T09:34:04.370 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 -2061us[-2106us] +/- 65ms 2024-03-09T09:34:04.370 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T09:34:04.374 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-09T09:34:04.374 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-03-09T09:34:04.374 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +2185us[+1856us] +/- 90ms 2024-03-09T09:34:04.375 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv02.front.sepia.ceph.com 3 7 377 40 +496us[ +166us] +/- 71ms 2024-03-09T09:34:04.375 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -2121us[-2451us] +/- 65ms 2024-03-09T09:34:04.375 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-09T09:34:04.376 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-09T09:34:04.387 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-09T09:34:04.388 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-09T09:34:04.397 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-09T09:34:04.651 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-09T09:34:04.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-09T09:34:04.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T09:34:04.969 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-09T09:34:05.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T09:34:05.070 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-09T09:34:05.080 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-09T09:34:05.088 INFO:teuthology.task.internal:Duration was 3137.714696 seconds 2024-03-09T09:34:05.089 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-09T09:34:05.098 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-09T09:34:05.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-09T09:34:05.106 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-09T09:34:05.268 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-09T09:34:05.274 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-09T09:34:05.683 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-09T09:34:05.683 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2024-03-09T09:34:05.683 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-09T09:34:05.749 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-03-09T09:34:05.750 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-09T09:34:05.830 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-09T09:34:05.831 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-09T09:34:05.838 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-09T09:34:05.935 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-09T09:34:05.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-09T09:34:05.941 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-09T09:34:06.201 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-09T09:34:06.213 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-09T09:34:06.213 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-09T09:34:06.307 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-09T09:34:06.403 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-09T09:34:06.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-09T09:34:06.424 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-09T09:34:06.528 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-03-09T09:34:06.530 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2024-03-09T09:34:06.584 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-09T09:34:06.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T09:34:06.635 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-09T09:34:06.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-09T09:34:06.688 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-09T09:34:06.699 INFO:teuthology.task.internal:Transferring archived files... 2024-03-09T09:34:06.700 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387/remote/smithi067 2024-03-09T09:34:06.700 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-09T09:34:06.809 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387/remote/smithi134 2024-03-09T09:34:06.810 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-09T09:34:06.916 INFO:teuthology.task.internal:Removing archive directory... 2024-03-09T09:34:06.917 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-09T09:34:06.925 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-09T09:34:06.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-09T09:34:06.990 INFO:teuthology.task.internal:Not uploading archives. 2024-03-09T09:34:06.990 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-09T09:34:06.999 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-09T09:34:07.000 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-09T09:34:07.007 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-09T09:34:07.059 INFO:teuthology.orchestra.run.smithi067.stdout: 524586 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 9 09:34 /home/ubuntu/cephtest 2024-03-09T09:34:07.063 INFO:teuthology.orchestra.run.smithi134.stdout: 524892 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 9 09:34 /home/ubuntu/cephtest 2024-03-09T09:34:07.066 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-09T09:34:07.077 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-09T09:34:07.097 INFO:teuthology.nuke:Checking targets against current locks 2024-03-09T09:34:07.123 DEBUG:teuthology.nuke:shortname: smithi067 2024-03-09T09:34:07.124 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-09T09:34:07.150 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-09 08:27:53.275226', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAb/kX4Hsc0tiJSY05cLPFVNhJMK+e4k9JbctP5BGPUVT79kEHsaEBetdjSRmsLxj40HKNERLtYzcfUyQqlqn10='} 2024-03-09T09:34:07.151 DEBUG:teuthology.nuke:shortname: smithi134 2024-03-09T09:34:07.152 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-09T09:34:07.175 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-09_08:15:07-fs-wip-lusov-qdb-distro-default-smithi/7588387', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-09 08:27:53.273353', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCLbjgt5vP+vOklGDWn6iDhKnEH0HXyfVYCMuZrpZANGqm64uoLhdXIJPj/Q1yBtpcI7cHfNELey3q2WmQCGrMA='} 2024-03-09T09:34:07.204 INFO:teuthology.orchestra.console.smithi067:Power off 2024-03-09T09:34:07.204 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-09T09:34:07.228 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off 2024-03-09T09:34:07.229 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T09:34:07.245 INFO:teuthology.orchestra.console.smithi134:Power off 2024-03-09T09:34:07.246 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-09T09:34:07.256 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2024-03-09T09:34:07.268 DEBUG:teuthology.orchestra.console.smithi134:power off output: Chassis Power Control: Down/Off 2024-03-09T09:34:07.268 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T09:34:07.290 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-03-09T09:34:11.257 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T09:34:11.291 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T09:34:11.369 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2024-03-09T09:34:11.402 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-03-09T09:34:15.370 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T09:34:15.402 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T09:34:15.480 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off 2024-03-09T09:34:15.512 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-03-09T09:34:15.582 INFO:teuthology.orchestra.console.smithi067:Power off completed 2024-03-09T09:34:19.514 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T09:34:19.625 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is off 2024-03-09T09:34:19.725 INFO:teuthology.orchestra.console.smithi134:Power off completed 2024-03-09T09:34:19.832 INFO:teuthology.run:Summary data: client.0-kernel-sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 client.1-kernel-sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 description: fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} duration: 3137.7146961688995 failure_reason: 'Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi067 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8b896a9e145796119f0451201dc4d53ddac97db0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9229f5ab0faa4aaba8f4efbf6f4a4b0f status: fail success: false 2024-03-09T09:34:19.833 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-09T09:34:19.920 INFO:teuthology.run:FAIL